Using AutoGen with Elasticsearch

Elasticsearch has native integrations to industry leading Gen AI tools and providers. Check out our webinars on going Beyond RAG Basics, or building prod-ready apps Elastic Vector Database.

To build the best search solutions for your use case, start a free cloud trial or try Elastic on your local machine now.

AutoGen is a Microsoft framework for building applications that can act with human intervention or autonomously. It provides a complete ecosystem with different abstraction levels, depending on how much you need to customize.

If you want to read more about agents and how they work, I recommend you read this article.

AutoGen Digaram — *Image source: https://github.com/microsoft/autogen*

AgentChat allows you to easily instantiate preset agents on top of the AutoGen core so you can configure the model prompts, tools, etc.

On top of AgentChat, you can use extensions that allow you to extend its functionalities. The extensions are both from the official library and community-based.

The highest level of abstraction is Magnetic-One, a generalist multi-agent system designed for complex tasks. It comes preconfigured in the paper explaining this approach.

AutoGen is known for fostering communication among agents, proposing groundbreaking patterns like:

In this article, we will create an agent that uses Elasticsearch as a semantic search tool to collaborate with other agents and look for the perfect match between candidate profiles stored in Elasticsearch and job offers online.

We will create a group of agents that share Elasticsearch and online information to try to match the candidates with job offers. We'll use the ´Group Chat´ pattern where an admin moderates the conversation and runs tasks while each agent specializes in a task.

The complete example is available in this Notebook.

steps

Install dependencies and import packages

Prepare data

setup keys

For the agent AI endpoint, we need to provide an OpenAI API key. We will also need a serper API key to give the agent search capabilities. serper gives 2,500 search calls for free on sign-up. We use serper to give our agent internet access capabilities, more specifically finding Google results. The agent can send a search query via API and serper will return the top Google results.

Elasticsearch client

Inference endpoint and mappings

To enable semantic search capabilities, we need to create an inference endpoint using ELsER. ELsER allows us to run semantic or hybrid queries, so we can give broad tasks to our agents and semantically related documents from Elasticsearch will show up with no need of typing keywords that are present in the documents.

Mappings

For the mappings, we are going to copy all the relevant text fields into the semantic_text field so we can run semantic or hybrid queries against the data.

Ingesting documents to Elasticsearch

We are going to load data about the job applicants and ask our agents to find the ideal job for each of them based on their experience and expected salary.

Configure agents

AI endpoint configuration

Let’s configure the AI endpoint based on the environment variables we defined in the first step.

Create agents

We´ll begin by creating the admin that will moderate the conversation and run the tasks the agents propose.

Then, we´ll create the agents that will carry out each task:

Admin: leads the conversation and executes the other agents’ actions.
Researcher: navigates online searching for job offers.
Retriever: looks up candidates in Elastic.
Matcher: tries to match the offers and the candidates.
Critic: evaluates the quality of a match before providing the final answer.

Configure tools

For this project, we need to create two tools: one to search in Elasticsearch and another to search online. Tools are a Python function that we will register and assign to agents next.

Tools methods

Assigning tools to agents

For the tools to work properly, we need to define a caller that will determine the parameters for the function and an executor that will run said function. We will define the admin as the executor and the respective agent as the caller.

Run task

We will now define a group chat with all agents, where the administrator assigns turns for each agent to define the task it wants to call and end it once the defined conditions are met based on previous instructions.

Reasoning

(Formatted for readability)

The output will look like this:

Next speaker: Matcher

Next speaker: Retriever

Next speaker: Admin

Admin (to chat_manager):

Researcher (to chat_manager):

Next speaker: Critic

Admin has been informed of the successful candidate-job offer matches.

Results

(Formatted for readability)

Note that at the end of each Elasticsearch stored candidate, you can see a match field with the job listing that best fits them!

Conclusion

AutoGen allows you to create groups of agents that work together to solve a problem with different degrees of complexity. One of the available patterns is 'group chat,' where an admin leads a conversation among agents to reach a successful solution.

You can add more features to the project by creating more agents. For example, storing the matches provided back into Elasticsearch, and then automatically applying to the job offers using the Websurfer agent. The Websurfer agent can navigate websites using visual models and a headless browser.

To index documents in Elasticsearch, you can use a tool similar to elasticsearch_hybrid_search, but with added ingestion logic. Then, create a special agent “ingestor” to achieve indexing. Once you have that, you can implement the Websurfer agent by following the official documentation.

Report an issue

steps

Install dependencies and import packages