How to Improve Fashion Search Relevance and Revenue with Marqo
.png)
Research shows that 80% of consumers abandon websites due to ineffective search experiences, and 79% are more likely to purchase when search results are on target. These figures emphasize the importance of robust search systems—especially in the fashion industry.
Traditional search engines often fall short for many tasks, especially when they depend solely on keyword searches or text-based data. This limitation becomes even more apparent in fields like fashion, where effectively combining textual and visual information is crucial. Marqo provides an elegant solution by seamlessly integrating advanced multimodal search capabilities, combining text and visuals to deliver precise, context-aware results. With Marqo you can combine the best of embedding/vector search and keyword search into a powerful hybrid search system. Marqo’s state-of-the-art fashion embedding models ensure that shoppers find exactly what they’re looking for. This blog will guide you through implementing Marqo within your search systems for state-of-the-art results.
The Problem: Improving Search Relevance in Fashion
Shoppers searching for fashion items face unique challenges that traditional search engines struggle to address. Product searches require a deep understanding of both visual and textual inputs, as well as the nuances of personal style, material preferences, and intent. Here’s why most search systems fail—and how Marqo solves these issues.
- Inability to Handle Complex Queries: Fashion searches like “casual black dress for a summer wedding” combine style, occasion, and color—yet most traditional search engines treat these as disconnected keywords, leading to irrelevant results. Even more challenging, keyword-based systems require this data to be explicitly present in the text. Extracting this information and maintaining the necessary pipelines is both complex and error-prone. Marqo’s framework addresses this by integrating advanced multimodal models that understand both textual and visual cues in context, eliminating the need for rigid keyword extraction and delivering highly relevant results.
- Failure to Combine Relevance with Conversion Potential: Most search engines focus solely on relevance, yet the definition of relevance is often unclear and inconsistent, frequently relying on human judgments (or “vibe” checks) that may not align with business objectives. As a result, these systems overlook conversion potential—the likelihood that a user will actually purchase an item. Marqo uniquely addresses this challenge by balancing relevance (how well an item matches a query) with conversion potential (based on metrics like add-to-cart rates, conversions, and revenue). By aligning search results with real business goals, Marqo ensures that users not only see relevant products but are also presented with items more likely to drive purchases.
- Lack of Personalisation: Fashion is personal, yet search engines fail to adapt to individual tastes, overlooking critical factors such as style preferences, past behavior, and shopping intent. Additionally, personalization isn't just about individual user preferences—it also involves localization, ensuring that search results are tailored to different regions and cultural contexts. With Marqo, personalization is at the core of search, enabling systems to evolve based on user-specific signals and local nuances for a truly tailored shopping experience.
- Failure to Surface Relevant Alternatives: When shoppers seek alternatives, rigid systems fail to suggest nuanced options like similar styles, materials, or colours, limiting discovery and frustrating users. Marqo’s framework enables intelligent alternative suggestions by leveraging multimodal embeddings and historical interaction data, enhancing product discovery and user satisfaction.
We’ll now take a look at how you can build a personalized fashion search engine with Marqo. We’ll start by collecting and preparing product and historical search data, then guide you through creating an index, adding documents, and using advanced search techniques like exact match boosters and revenue modifiers to enhance relevance and drive conversions.
Step 1: Data Collection
To build an effective search system, you need two key types of data: product and historical search. The latter can be obtained using search logs. In this section, we’ll discuss and demonstrate these datasets.
Product Data
Product data contains static, descriptive attributes about each item in the catalogue. This includes fields such as product images, titles, descriptions, categories, and stock status. Unlike search data, product data remains independent of user interactions or sales performance but is essential for providing context and filtering search results.
Here’s an example of product data:
For this example, we use a ~15k subset of the Marqo/fashion200k dataset. To simulate a real-world scenario, we’ve added fields like product cost and stock status, which are crucial for filtering and ranking search results. You can find this dataset on our GitHub.
Search Log Data
Search logs capture event-level interactions between users and the search system. These logs record each action (e.g., view, click, add to cart, purchase) that a user performs after submitting a query. This data is essential for understanding user behavior and generating metrics that can be used to improve search relevance.
Here’s an example of raw search log data:
Each row represents a single interaction for a given query and product. However, to make this data useful for search optimization, it needs to be processed and aggregated. By grouping the raw event data by query-item pairs, we can derive key metrics such as total purchases, add-to-cart counts, total clicks, and revenue. This aggregated data, combined with product data, forms the basis of historical search data.
Historical Search Data
Historical data helps optimise search relevance by highlighting popular products for specific queries. For given queries, we are able to collect information about what products are returned in the search and how well these products perform. We can take our search log data from above and create our historical data. For information and instructions on how you can convert your search logs into historical data, see our GitHub.
Here is an example of historical search data.
We see 4 different queries, the corresponding product items that were returned for this query, and how popular they are (popularity here being based on purchases/add to cart/clicks). For example, when a user searched for “green candy dress”, the item “green short dress” (id: 71246356_0) received 17 total purchases. We see that this is a popular item for this query and so, if a user was to search this query again, featuring this item is likely to perform well. We will see later how we can leverage this historical data for improved search and revenue with Marqo.
Step 2: Creating Marqo Index
Now we have discussed the types of data needed for successful search, we can begin building with Marqo. First, we need to install the Marqo Python client:
Next, we define our Marqo API Key and set up our Marqo client. This establishes a connection to the Marqo API. To find your API Key, visit this article.
We now define the settings for our Marqo index. This is where we specify what embedding model we’re going to use. Embedding models are machine learning models that transform data, such as text or images, into dense vector representations, enabling efficient and accurate similarity search.
At Marqo, we created a general-purpose, state-of-the-art, fashion embedding model that you can use out-of-the-box, Marqo/marqo-fashionSigLIP. This model was trained using Marqo’s novel framework, Generalized Constrastive Learning (GCL), to optimize over seven fashion specific aspects including descriptions, titles, colors, details, categories, keywords and materials.
While our general model is a great starting point, fine-tuning on actual data from your specific use case can significantly improve search accuracy and relevance. Fine-tuning allows the model to learn nuances unique to your data, resulting in better understanding and representation of your products. Fine-tuning also scales well to billions of products [1].
If you want to fine-tune on your own data, you can leverage Marqtune—Marqo’s fine-tuning solution designed to make this process straightforward. Marqtune helps you enhance your embedding models by optimizing them for specific tasks, enabling even better retrieval performance for your application.
Let’s take a look at how we can use Marqo’s state-of-the-art fashion embedding model when creating an index:
It will take a few minutes to create your index. While your index is being created, the Marqo Cloud console will populate. The console will display a ‘Ready’ status when your index has been successfully created.
Step 3: Adding Documents
Now we have created our Marqo index, we can begin adding documents to it. As mentioned, we have two types of data: product and historical search. We will first take a look at how we can combine these two and then how to add these documents to our Marqo index.
Document Formatting
We can take our historical data and update our product data to include this information. We do this so that any product item that positively impacts revenue for a given query, is added to that document. For example, let’s take the following item from our product catalogue.
Let’s look at how this item is structured in our product data. We can see all the key information (name, category, cost, etc.) which are all important features in a search engine but let’s see if this item has any historical search data associated with it.
From our historical search data, we see that this item performed well from a revenue and click perspective for the query “green candy dress”. It received 80 total clicks and generated revenue in the last 5 days.
Given its popularity for this specific query, we can update this document with this information i.e. adding "one_day_revenue_modifiers.green_candy_dress": 36.74. Marqo supports maps of score modifiers which are a flexible data structure we can use to add arbitrary score boosting for documents. We can use modifiers at query time by inserting the user submitted query into the modifiers, if it matches then it will get boosted, if it doesn’t then it just adds 0 to the score and has no impact. By including these modifier fields in the documents, it allows these items to be weighted for better search results.
Another crucial aspect to a successful search engine is ensuring that if a user’s query matches exactly to that of the product name (and/or other attributes like title, the product name in another language, etc.) it must appear at the top of the search feed. For example, if a user searches for “gray guipure cape draped dress” and there is a product in the catalogue with that exact name or title, this item should be the top returned result. This can be incorporated by a field called exact_match_boosters which is used to rank items that match the exact query. The exact_match_boosters field maps the product_name to a large number which we can use to boost it in search.
Given the two CSV datasets, we can combine them into a single CSV dataset. By combining the original product data with exact match boosters and modifiers, we can expect our documents to look as follows in JSON format:
This can then be converted to JSON format ready to be loaded into Marqo. Each product item will look as follows:
This is then repeated for all entries in the historical search data that match id’s in the product catalogue data. Please note, we have this GitHub that contains all the code for data processing, creating modifiers and performing searches.
Adding Documents
We can now add these updated documents to our Marqo index. We have a JSON file which can be found on our GitHub here so you can get up and running but feel free to substitute your own search and product data here.
The product data is first loaded from a JSON file and then added to the index in batches for efficient processing. A key feature here is the use of a multimodal_combination field, which blends text data (product_name) and visual data (image_url) using assigned weights. By giving more weight to the image data, we ensure that the visual aspect of a product plays a dominant role in search queries. Additionally, the use of existing tensors, when available, speeds up the process by avoiding redundant computation. This approach helps create a rich, multimodal index that can provide more accurate and visually relevant search results for users.
Our dataset is ~15k items which you can expect to take roughly 10 minutes to add to your index. However, you can start searching while documents are being added. Let’s now take a look at how you can perform searches with Marqo.
Step 4: Searching
We will now take a look at searching over our index and explore 4 different search approaches. Please note, we have an easily deployable UI which allows you to simply select between these 4 search methods and observe the results. To do this, visit the GitHub repository.
Simple Search
We will start with a very basic search. We perform a vector search for the query “green candy dress”, return the top 20 results (limit=20), and filter the results to make sure that only items in stock are returned.
Note, you can also filter other items in here such as available regions. These are the results:
We see immediately that the Marqo/marqo-fashionSigLIP embedding model is surfacing relevant and accurate search results by showing items that match similar to the search query. This is great but let’s look at how we can improve this search even further.
Hybrid Search
We’ve seen the results from a tensor search but we can extend this to combine both tensor and lexical search methods with hybrid search. We specify the search method, search_method="HYBRID" as well as the hybrid parameters. Alpha is the linear weight of the tensor RRF score. At alpha=0.5, this balances both lexical and tensor search. Reciprocal rank fusion (rrfK) is the smoothing factor for RRF. The higher rrfK, the lower the contribution of RRF to the ranking. The product’s name is used as the attribute for lexical search. The rest of the settings are the same as before with the addition of attributes_to_retrieve which specifies which fields should be included in the search results.
With hybrid search, we expect more lexical based matches to appear in our searches which is exactly what we see — the top three items returned include “Green Candy Dress” and “Green Candy Off Shoulder Dress”.
The next question is, can we improve our search so that if someone inputs a query that matches a product name exactly, this will rank as the top result?
Exact Match Boosters
Hybrid search allows for the flexibility of both lexical and visual based searches. However, for more general searches, it may not return exact results. Take the query “green candy dress” for example as searched above. The top result is a “Green Short Dress” but the second result is an exact query to product name match. Here we can leverage the exact_match_boosters to ensure any matching items are returned as the top result. Let’s take a look at how we can do this with modifiers in Marqo.
All of our search settings remain the same as our previous hybrid search but this time we also specify scoreModifiersTensor and scoreModifiersLexical which boost the scores for documents that have exact matches in the specified fields. This ensures that results with a perfect match to the query are ranked higher. In this example, if the query matches the product name of any document, this will be boosted higher in the search results.
Let’s take a look at what this has done to our search results for the query “green candy dress”:
We can clearly see the ‘Green Candy Dress’ ranking top as desired. This was a great introduction to score modifiers in Marqo but they can be further leveraged — this time, taking advantage of historical data like revenue or clicks.
Revenue Modifiers
As mentioned earlier, we can use revenue values as modifiers. The process is the same as above, just with some additional fields. For this example, we use the revenue from the historical query data as well as revenue information from the product data itself.
First, let’s take a look at what our hybrid search with exact boost modifiers looks like for our previous query “green candy dress”.
We can now use our historic data to ensure that popular items are surfaced closer to the top. Here's some historical data for this query:
We see from our historical data that for the query “green candy dress”, the items 91256064_0 and 71246356_0 performed well amongst users — these items generated revenue and clicks. These items look as follows:
We can use modifiers in search to boost these popular items. Our search settings are exactly the same as before but this time we include modifiers for one, three, and five day revenue for a given query. That is, if an item performs well (from a revenue perspective in this case), we weight this item. The weights for revenue modifiers were derived through hyperparameter optimization.
Through this, we expect the items shown above to rank more highly when incorporated into the score modifiers when searching for “green candy dress”. Let’s look at the results:
Both of these items are now in the top 5 search results with the “Green Open-Back Dress” ranking higher than the “Green Sheath Dress” which we expect as this item generated more in revenue for this particular query. This particular example used revenue but this can be extended to any type of data.
Measuring Search Quality
We’ve performed four different search methods and now, we can evaluate and compare the performance of these search methods against a dataset by generating comparative reports.
Mean Average Precision (map@K)
This graph shows how well the search strategies rank the most relevant results at the top. Mean Average Precision (MAP) tells us, on average, how often the top-ranked items are what users actually want.
The blue line (tensor) performs poorly throughout, meaning it struggles to consistently rank relevant items at the top. The orange, green, and red lines (hybrid methods) perform significantly better. Among them, the red line (hybrid with exact boosters and modifiers) performs best, suggesting that this search method is optimal when wanting to rank relevant items at the top.
Normalized Discounted Cumulative Gain (ndcg@K)
NDCG is all about ranking quality, with extra emphasis on getting the most relevant results near the top of the list. A higher NDCG score means users see the best matches without needing to scroll too far.
Again, the blue line (tensor) performs significantly worse. If you’re building a search system where users need quick, high-quality results (like when searching for products), the hybrid approach with boosters and modifiers is the best option.
Recall (recall@K)
Recall measures how many of the relevant items the search strategy finds overall. A higher recall score means users are seeing more of the items that match their query.
The blue line (tensor) starts low and only improves gradually as K increases (i.e., when users look at a lot of results). The red line (hybrid_with_exact_boosters_and_modifiers) hits nearly perfect recall very quickly, meaning it retrieves almost all relevant items early on.
From this analysis, if you’re building a search experience where ranking quality and relevance matter, hybrid search with boosters and modifiers is the best option. This approach ensures users find exactly what they’re looking for, faster and more accurately.
Conclusion
In this blog, we demonstrated how Marqo can be configured to handle fashion-specific data, integrate historical search information, and apply advanced techniques like exact match boosters and revenue modifiers. The result? A smarter, context-aware search engine that evolves with user behaviour, ensuring that the most relevant products are always in front of your customers.
Next Steps
To get the exact same modifiers and configurations using the same historic data, see our GitHub which contains further information on how to obtain this.
As mentioned earlier in the article, the best search performances are with a model fine-tuned on your data. For more information on how Marqo can help you achieve this as well as a fully state-of-the-art search system, book a demo or contact us.
.png)
.png)
.png)
