Challenges
Volume of user-generated content and artwork
As a marketplace for user-generated content and artwork, Redbubble had difficulty understanding the listed content because the listings/products can be very subjective.
The team relied heavily on artists/merchants tagging their own listings, which can be error-prone. This also created a challenge where tags could be manipulated to artificially boost particular works.
Challenges with Keyword Search
Redbubble has historically utilized OpenSearch. OpenSearch is effective for keyword search but there were known challenges for torso and tail queries where interaction signals were more sparse.
The team noticed the existing keyword search performed very well on the 1000 most popular one- and two-word queries. However, the team saw room for improvement outside these searches, particularly in three-plus-word queries.
Multimodal Vector Search at Scale
In 2021, OpenAI released the first CLIP model. The team ran a few experiments and knew they wanted to leverage the technology, but they quickly realized how challenging it is to build an end-to-end vector search engine.
![](https://cdn.prod.website-files.com/62dfa8e3960a6e2b47dc7fae/669e6eb2fc213e2b69b69d4f_quotes.svg)
Initially, we were thinking about CLIP from OpenAI as the AI model to do our own vector search, but after attempting to build it internally, we decided to evaluate a few vector search engines. Marqo was the most advanced in terms of functionality.
![Luan Nguyen](https://cdn.prod.website-files.com/62dfa8e3960a6e2b47dc7fae/66a323dd819a4c09e9bd5ecc_luan.nguyen.jpeg)
Luan Nguyen
![LinkedIn Icon](https://cdn.prod.website-files.com/62dfa8e3960a6e2b47dc7fae/669e6eb3fc213e2b69b69f46_linkedin-black.svg)
Tech Lead, Search & Recommendation at Redbubble
Solutions
Vector Search Approach
After extensive evaluation and piloting vector databases and other search solutions, the Redbubble team decided to use a purpose-built vector search engine for semantic search to complement its current keyword search system (OpenSearch).
The ability to use state-of-the-art CLIP models that understand both the artwork's image and its accompanying textual features captured a new understanding of the data. These new models could now generalize the understanding of relevance to unseen new works and better capture the affinity between the visual elements of the design and the products that best fit them. These models allow search results to be completely independent of the user-generated texts and tags added, which was a huge unlock for the team.
Why Marqo
While the team knew they wanted to embrace vector search, they needed to determine what solution would be the best for their needs.
Luan Nguyen, Technical Lead at Redbubble, explains, “We were choosing between Vespa, Marqo, open source tools, OpenSearch, and Pinecone. After piloting the cloud solution and seeing the support and expertise Marqo provided, the decision was easy to move from pilot to production. The Marqo team almost seemed a part of our team”
Marqo’s end-to-end embedding cloud, in combination with Marqtune, and Marqo’s embedding fine-tuning product, set Marqo apart from its competitors. Marqtune developed several fine-tuned models using Redbubble’s data, which understand relevance, historical popularity of works, and product-category affinity.
Results
Marqtune — Model Fine-tuning
While open-source CLIP models provided relevant results, they were not aligned with Redbubble's customer intent; Rebubble’s audience and content necessitate specific definitions of relevance. Traditional CLIP training provided some improvements, but it could not capture customer wants.
With Marqtune, new models are now trained with Redbubble’s historical sales information to align the model with user behavior. Unlike keyword search these models can generalize meaning that previously unsold works do not require a score to be easily surfaced in search - they simply must fit the style of works that are successful on redbubbles platform. The training of these models utilizes Marqo’s Generalised Contrastive Learning (GCL).
Long Tail Queries
A/B experiments showed that models fine-tuned with Marqtune significantly outperformed the existing keyword search on 3+ word queries, representing a third of Redbubble’s search volume.
Vector search has excelled at these tail queries, with experiments yielding up to 30% improvements in Add-to-Cart (ATC) rate compared to the existing search.
- Add-to-Cart (ATC) rate with searches of 4+ words - 19% improvement
- Add-to-Cart (ATC) rate with descriptive searches (5+ words) - 30% improvement
- Average order value increased by 2.3% across all queries
The road ahead
Marqo will continue to partner with Redbubble, intending to innovate in ecommerce search:
Search improvement to support head queries
Expanding on what we learned from multi-string queries, we are exploring further optimizations to the model to enable us to serve more relevant content to customers searching high-frequency terms.
Recommendations
Ameliorate information from users, interaction history, current cart, and queries to deliver real-time personalization of searches and recommendations.
Semantic Filtering
Query modification for filtering with no metadata, to allow for filtering to styles (e.g. “outline,” “cyberpunk,” “sketch”) in the existing catalog. Users could also construct personalized filters.