Enhancing relevance with sparse vectors

Sparse vectors are a key component in ELSER, but their usefulness extends far beyond that. In this post, we’ll explore how sparse vectors can enhance search relevance in an e-commerce setting: boosting documents based on search behavior (like clicks) and user preferences.

What exactly are sparse vectors?

Vector search is a hot topic right now, but most conversations focus on dense vectors: compact numerical representations used in machine learning and neural search. Sparse vectors, on the other hand, take a different path.

Unlike dense vectors that pack data tightly, sparse vectors store information in a more interpretable and structured format, often with many zeros. Though less hyped, they can be incredibly powerful in the right context.

💡 Fun fact: Sparse vectors and inverted indexes both leverage sparsity to efficiently represent and retrieve information.

In Elasticsearch, you can store sparse vectors using the sparse_vector field type: no surprises there.

Querying with sparse vectors

Searching with sparse vectors in Elasticsearch feels similar to traditional keyword search, but with a twist. Rather than matching terms directly, sparse vector queries use weighted terms and the dot product to score documents based on how well they align with the query vector.

Use case 1: Signal boosting for better search ranking

Signal boosting refers to emphasizing certain features or terms to improve search ranking. This is especially useful when business logic or user behavior suggests that some results should appear higher.

Let’s say we’re working with a simple e-commerce index:

Now, let’s index two documents only using traditional full text type:

A basic search for “playstation” will return the controller first, not because it’s more relevant, but because BM25, the default lexical scoring algorithm, tends to favor shorter fields, causing the controller’s concise title to rank higher.

But we want to boost the console result, especially since it has a special offer!

One way to do this is by embedding boosting signals directly into the document via sparse vectors:

This document now carries extra weight for the search queries “playstation” and “game console”.

We can adjust our query to incorporate this sparse vector boost:

Thanks to the added score from the sparse vector match, the console now ranks above the controller, which is exactly what we want!

This approach offers an alternative to traditional boosting techniques, such as function_score queries or field-level weight tuning. By storing boosting information directly in the document using sparse vectors, you gain more flexibility and transparency in how relevance is adjusted. It also decouples business logic from query logic.

However, it’s worth noting the tradeoffs: traditional boosting can be simpler to implement for straightforward use cases and may have performance advantages in some scenarios. Sparse vectors shine when you need fine-grained, multi-dimensional control over boosting.

Reminder: The must clause filters and contributes to scoring, while the should clause adds to the score if the condition matches.

Use case 2: Personalization using sparse vectors

Sparse vectors also enable personalization. You can assign weights to customer traits or personas and use them to surface the most relevant products for individual users.

Here’s an example:

Let’s say Jim is a customer who prefers healthy, sustainable options:

We can tailor the search experience to reflect Jim’s preferences:

As a result, the healthier snack bar floats to the top of the search results because that’s what Jim is more likely to buy.

This method of personalization via sparse vectors builds on ideas like static segment tags, but makes them more dynamic and expressive. Instead of assigning a user to a single segment like "tech-savvy" or "healthy-conscious", sparse vectors allow you to represent multiple affinities with varying weights, all in a way that integrates directly into the search ranking process.

Using a function_score query to incorporate user preferences is a flexible alternative for personalization, but it can become complex and difficult to maintain as logic grows. Another common approach, collaborative filtering, relies on external systems to compute user-item similarities and typically requires additional infrastructure. Learning to Rank (LTR) can also be applied to personalization, offering powerful ranking capabilities, but it demands a high level of maturity, both in terms of feature engineering and model training.

Wrapping up

Sparse vectors are a versatile addition to your search toolbox. We’ve covered just two practical examples: boosting search results and personalizing based on user profiles. But the possibilities are broad.

By embedding structured, weighted information directly into your documents, you unlock smarter, more relevant search experiences with minimal complexity.

Elasticsearch is packed with new features to help you build the best search solutions for your use case. Dive into our sample notebooks to learn more, start a free cloud trial, or try Elastic on your local machine now.

Report an issue