Elasticsearch open inference API adds support for IBM watsonx.ai Slate embedding models

Elastic is excited to announce the expansion of our open inference API capabilities with the integration of IBM watsonx™ Slate embedding models, marking an important milestone in our ongoing partnership with the IBM watsonx team. With this announcement, Elasticsearch users gain immediate, simplified access to IBM’s Slate family of models while the IBM watsonx community can take advantage of Elasticsearch’s comprehensive AI search tooling and proven vector database capabilities.

Elastic’s open inference API, now generally available, enables you to create endpoints and use machine learning models from providers like IBM watsonx™. IBM® watsonx™ AI and Data Platform includes core components and AI assistants designed to scale and accelerate AI's impact using trusted data. The platform features open-sourced Slate embedding models (slate-125m, slate-30m) for retrieval-augmented generation, semantic search, and document comparison, and also the Granite family of LLMs trained on trusted enterprise data.

In this blog, we will explain how to use IBM watsonx™ Slate text embeddings when building Search AI experiences with Elasticsearch vector database. Elastic now supports the usage of these text embeddings, with the new semantic_text field chunking incoming text by default to fit the token limits of the platform’s models.

Prerequisites & Creation of Inference Endpoint

You will need an IBM Cloud® Databases for Elasticsearch deployment. You can provision one through the catalog, the Cloud Databases CLI plug-in, the Cloud Databases API, or Terraform. Once the account is set up successfully, you should land on the IBM cloud home page.

You can then provision a Kibana instance and connect to your Databases for Elasticsearch instance using the managed service model of IBM Cloud using these steps -

Set the Admin Password for your Elasticsearch deployment.
Install Docker to pull the Kibana container image and connect it to Databases for Elasticsearch.

Alternatively, if you prefer not to run Kibana locally or install Docker, you can deploy Kibana using IBM Cloud® Code Engine. For details, see the documentation on deploying Kibana with Code Engine and connecting it to your Databases for Elasticsearch instance.

Generate an API key

Go to IBM watsonx.ai cloud and log in using your credentials. You will land on the welcome page.

Go to the API keys page.
Create an API key.

Steps in Elasticsearch

Using DevTools in Kibana, create an inference endpoint using the watsonxai service for text_embedding

PUT _inference/text_embedding/ibm_watsonx_embedding
{
    "service": "watsonxai",
    "service_settings": {
        "api_key": "<api_key>",
        "url": "xxx.ml.cloud.ibm.com",
        "model_id": "ibm/slate-30m-english-rtrvr",
        "project_id": "<project_id>",
        "api_version": "2023-05-29"
    }
}

You will receive the following response on the successful creation of the inference endpoint:

{
  "inference_id": "ibm_watsonx_embedding",
  "task_type": "text_embedding",
  "service": "watsonxai",
  "service_settings": {
    "model_id": "ibm/slate-30m-english-rtrvr",
    "project_id": "<project_id>",
    "url": "xxx.ml.cloud.ibm.com",
    "api_version": "2023-05-29",
    "dimensions": 384,
    "similarity": "dot_product",
    "rate_limit": {
      "requests_per_minute": 120
    }
  },
  "task_settings": {}
}

Generate Embeddings

Below is an example of generating text_embedding for a single string

POST _inference/text_embedding/ibm_watsonx_embedding
{
     "input": "Embed this text"
}

You will receive the following response as embeddings:

{
  "text_embedding": [
    {
      "embedding": [
        0.009380317,
        0.05560313,
        ...,        
        0.03804972
      ]
    }
  ]
}

Additionally, let's look at a semantic_text mapping example

Create an index containing semantic_text field

PUT books
{
 "mappings": {
   "properties": {
     "title": {
       "type": "text"
     },
     "semantic_text": {
       "type": "semantic_text",
       "inference_id": "ibm_watsonx_embedding"
     },
     "description": {
       "type": "text",
       "copy_to": [
         "semantic_text"
       ]
     }
   }
 }
}

Insert some documents to the created index

POST /books/_bulk
{ "index": {"_id": "1" }}
{ "title": "The Art of Coding", "description": "A comprehensive guide to coding best practices.", "price": 39.99 }
{ "index": { "_id": "2" } }
{ "title": "Mysteries of the Universe", "description": "Exploring the unknown aspects of our universe.", "price":
24.50 }
{ "index": {"_id": "3" } }
{ "title": "The Journey Within", "description": "A deep dive into personal development.", "price": 18.75 }
{ "index": {"_id": "4" } }
{ "title": "The richest man in babylon", "description": "The Richest Man in Babylon by George S. Clason is a classic book that offers timeless financial wisdom through a series of parables set in ancient Babylon, one of history’s wealthiest cities. The book conveys its lessons through stories of Babylonian citizens who learn to achieve wealth, prosperity, and happiness by following basic financial principles. The central character, Arkad, who is the richest man in Babylon, shares his wisdom on wealth-building with friends and fellow citizens. He explains how he started as a poor scribe but grew rich by adhering to a set of practical rules, which are presented as the Seven Cures for a Lean Purse and the Five Laws of Gold. The Seven Cures for a Lean Purse emphasize essential steps for accumulating wealth: saving at least ten percent of earnings, controlling expenses to live below one’s means, making wise investments, protecting investments from loss, owning a home, ensuring future income, and continually increasing the ability to earn. The Five Laws of Gold focus on key principles such as accumulating money through savings, investing wisely, seeking advice from experienced investors, avoiding investments in unfamiliar areas, and steering clear of get-rich-quick schemes. The book uses various characters like Bansir, a chariot maker, and Kobbi, a musician, to illustrate common financial challenges, and they seek Arkad’s guidance to achieve financial security, learning the importance of discipline, thrift, and prudent investment. Other stories, such as those of Dabasir, who faces challenges repaying debts, and Rodan, who learns about wise lending, further reinforce these principles. The book emphasizes that wealth is not a matter of luck or fate but a result of adhering to sound financial practices and cultivating a mindset geared towards saving, investing, and learning continuously. Through its simple yet profound lessons, the book provides readers with practical advice on personal finance, helping them understand the value of money management, disciplined saving, and wise investment strategies to build and maintain wealth over time.", "price": 18.75 }

Next, run a query using semantic_text

GET books/_search
{
 "query": {
    "semantic" : {
       "field": "semantic_text",
       "query": "maintain wealth over time"
     }
 }
}

You will receive the following response from the query

{
  ...,
  "hits": {
    "max_score": 0.86763954,
    "hits": [
      {
        "_index": "books",
        "_id": "4",
        "_score": 0.86763954,
        "_source": {
          "semantic_text": {
            "inference": {
              "inference_id": "ibm_watsonx_embedding",
              "model_settings": {
                "task_type": "text_embedding",
                "dimensions": 384,
                "similarity": "dot_product",
                "element_type": "float"
              },
              "chunks": [
                {
                  "text": "The Richest Man in Babylon by George S. Clason is a classic book that offers timeless financial wisdom through a series of parables set in ancient Babylon, one of history’s wealthiest cities. The book conveys its lessons through stories of Babylonian citizens who learn to achieve wealth, prosperity, and happiness by following basic financial principles. The central character, Arkad, who is the richest man in Babylon, shares his wisdom on wealth-building with friends and fellow citizens. He explains how he started as a poor scribe but grew rich by adhering to a set of practical rules, which are presented as the Seven Cures for a Lean Purse and the Five Laws of Gold. The Seven Cures for a Lean Purse emphasize essential steps for accumulating wealth: saving at least ten percent of earnings, controlling expenses to live below one’s means, making wise investments, protecting investments from loss, owning a home, ensuring future income, and continually increasing the ability to earn. The Five Laws of Gold focus on key principles such as accumulating money through savings, investing wisely, seeking advice from experienced investors, avoiding investments in unfamiliar areas, and steering clear of get-rich-quick schemes. The book uses various characters like Bansir, a chariot maker, and Kobbi, a musician, to illustrate common financial challenges, and they seek Arkad’s guidance to achieve financial security, learning the importance of discipline, thrift, and prudent investment. Other stories, such as those of Dabasir, who faces challenges repaying debts, and Rodan, who learns about",
                  "embeddings": [
                    -0.0012952854,
                    0.054616664,
                    ...
                  ]
                },
                {
                  "text": " home, ensuring future income, and continually increasing the ability to earn. The Five Laws of Gold focus on key principles such as accumulating money through savings, investing wisely, seeking advice from experienced investors, avoiding investments in unfamiliar areas, and steering clear of get-rich-quick schemes. The book uses various characters like Bansir, a chariot maker, and Kobbi, a musician, to illustrate common financial challenges, and they seek Arkad’s guidance to achieve financial security, learning the importance of discipline, thrift, and prudent investment. Other stories, such as those of Dabasir, who faces challenges repaying debts, and Rodan, who learns about wise lending, further reinforce these principles. The book emphasizes that wealth is not a matter of luck or fate but a result of adhering to sound financial practices and cultivating a mindset geared towards saving, investing, and learning continuously. Through its simple yet profound lessons, the book provides readers with practical advice on personal finance, helping them understand the value of money management, disciplined saving, and wise investment strategies to build and maintain wealth over time.",
                  "embeddings": [
                    -0.025139397,
                    0.054932922,
                    0.09041961,
                    ...
                  ]
                }
              ]
            }
          },
          "price": 18.75,
          "description": "The Richest Man in Babylon by George S. Clason is a classic book that offers timeless financial wisdom through a series of parables set in ancient Babylon, one of history’s wealthiest cities. The book conveys its lessons through stories of Babylonian citizens who learn to achieve wealth, prosperity, and happiness by following basic financial principles. The central character, Arkad, who is the richest man in Babylon, shares his wisdom on wealth-building with friends and fellow citizens. He explains how he started as a poor scribe but grew rich by adhering to a set of practical rules, which are presented as the Seven Cures for a Lean Purse and the Five Laws of Gold. The Seven Cures for a Lean Purse emphasize essential steps for accumulating wealth: saving at least ten percent of earnings, controlling expenses to live below one’s means, making wise investments, protecting investments from loss, owning a home, ensuring future income, and continually increasing the ability to earn. The Five Laws of Gold focus on key principles such as accumulating money through savings, investing wisely, seeking advice from experienced investors, avoiding investments in unfamiliar areas, and steering clear of get-rich-quick schemes. The book uses various characters like Bansir, a chariot maker, and Kobbi, a musician, to illustrate common financial challenges, and they seek Arkad’s guidance to achieve financial security, learning the importance of discipline, thrift, and prudent investment. Other stories, such as those of Dabasir, who faces challenges repaying debts, and Rodan, who learns about wise lending, further reinforce these principles. The book emphasizes that wealth is not a matter of luck or fate but a result of adhering to sound financial practices and cultivating a mindset geared towards saving, investing, and learning continuously. Through its simple yet profound lessons, the book provides readers with practical advice on personal finance, helping them understand the value of money management, disciplined saving, and wise investment strategies to build and maintain wealth over time.",
          "title": "The richest man in babylon"
        }
      },
      ...
    ]
  }
}

Conclusion

With the integration of IBM watsonx™ text embeddings, the Elasticsearch Open Inference API continues to empower developers with enhanced capabilities for building powerful and flexible AI-powered search experiences. Explore more supported encoder foundation models available with watsonx.ai.

Elasticsearch has native integrations to industry leading Gen AI tools and providers. Check out our webinars on going Beyond RAG Basics, or building prod-ready apps Elastic Vector Database.

To build the best search solutions for your use case, start a free cloud trial or try Elastic on your local machine now.