Improving Copilot capabilities using <strong>elasticsearch</strong>

elasticsearch has native integrations to industry leading Gen AI tools and providers. Check out our webinars on going Beyond RAG Basics, or building prod-ready apps Elastic Vector Database.

To build the best search solutions for your use case, start a free cloud trial or try Elastic on your local machine now.

This article will guide you through integrating elasticsearch with Microsoft Copilot. We'll build a custom agent capable of retrieving and utilizing data from your elasticsearch documents.

What is Microsoft Copilot

Microsoft Copilot is an AI companion that helps you achieve tasks across the Microsoft ecosystem products, offering functionality similar to ChatGPT for various productivity needs.

By using Microsoft Copilot Studio and a UI interface, you can build customized agents and equip them with actions. These actions connect to your APIs to fetch data and respond to user queries.

An action is a tool the agent can use to help you solve your tasks. The actions can connect to built-in platform connectors or to APIs. The OpenAPI JSON specification provides a standard format for describing your API, allowing Copilot to understand its functions, available endpoints, and how to correctly structure queries, including automatic parameter population.

We are going to build an agent that gives information about invoices stored in elasticsearch, connecting via API. The agent will live on Microsoft Teams to work as a personal assistant.

This tutorial will guide you through the following steps:

All the code and files referenced in this blog post can be found in this GitHub repository.

Building the API

We’ll create a simple API that queries elasticsearch. We’ll build it in a Jupyter notebook using FastAPI.

Let’s review the installed tools before importing them:

fastapi, uvicorn: Generates a HTTP server to consume elasticsearch.
pyngrok: Creates a secure public IP address (tunnel) for your local server, enabling the Copilot agent to access it over the internet.
elasticsearch: elasticsearch’s Python client.
nest-asyncio: Needed to create the server inside the Notebook environment.

To use the API in a Copilot agent, we need it to be accessible through the internet. So, we’ll use ngrok to create a tunnel to connect the API to the internet. Create a ngrok account and get an auth token here.

Let’s configure the environment variables:

Now, instantiate the elasticsearch client:

Let’s start the server and create the following endpoints:

/search/semantic

Runs a semantic search based on the user’s query.

/search/by-date

Runs a search based on a date range.

In both cases, elasticsearch returns a list of documents from each query.

Let’s configure the tunnel and run the server.

The Public URL you get will be provided to our Microsoft Copilot agent.

Set up elasticsearch data

Mappings

Let’s index a list of invoices with the descriptions of the services provided, date, reference URL, total amount, and description.

We will also copy the properties we need for semantic search to a semantic_field field type.

Indexing data

We’ll upload a list of different invoices. The dataset can be found here.

Let’s create a function to run the invoices array and index it into elasticsearch using the bulk API.

Copilot and agent configuration

To use Copilot chat, we’ll need an Office 365 account. If you don’t have one, you can create a trial account here.

Create the agent and actions

After you create an account, go to the main Copilot Studio site and follow the instructions below:

1. Follow the Microsoft steps to create an agent.

2. Add actions from a REST API.

An action is an API endpoint. For this example, you need to add two: one for the semantic text queries and another one for date based queries.

To do this, you need to first generate an OpenAPI JSON reference of the API you just created. You can find the OpenAPI specification file in the GitHub repository dedicated to this article.
In the OpenAPI configuration, replace the existing 'host' value with the ngrok-generated URL. Make sure you remove the 'https://' prefix, leaving only the hostname and port.

3. Click on Settings and activate Generative AI to be able to use GenAI to process the conversation. Select Actions, Knowledge, and Topics, depending on what Copilot detects in the conversation.

4. Clicking on Overview and then scrolling down, you’ll find the Knowledge section. Deactivate the option Allow the AI to use its own general knowledge. This will prevent the agents from responding using the general info from their training since we want them to focus on our elasticsearch documents.

5. If you’re using the ngrok free layer, you must set the 'ngrok-skip-browser-warning' header with any value. This bypasses a browser verification step required by ngrok due to potential abuse of free tunnels.

To do this, go to “Actions > [Action] > Inputs > Ngrok Skip Browser Warning and click on “How will the agent fill this input?” and select “Set as value.” Add any value to Value and click on “Save” so you can use the endpoints with the ngrok tunnel:

Make the agent visible in Copilot

To use the agent from Microsoft 365 or Microsoft Teams, you must follow these steps:

1. From Copilot Studio, go to Channels and click on Teams + Microsoft 365:

2. Unselect the option “Make agent available in Microsoft 365 Copilot Chat.” This option is only useful if you want to use the agent in Microsoft 365 Copilot, but we want to use it with Microsoft Teams.

3. Now click on “Add channel.”

4. Once you have it on the channel, click See agent in Teams and add it directly.

After adding it, you can start to use it from Copilot simply by typing an @ and selecting it from the options.

Using the agent

When using actions for the first time, Copilot will warn you about permissions and show a box when you need to grant permissions for each action by clicking on Connect. You must do this once per action.

You are now ready to use the agent.

To test semantic search, let's ask the agent about invoices related to food expenses:

Let’s check the answer:

Thanks to the semantic search features, Copilot was able to find invoices that didn’t include the word food but instead contained words like “dinner” and “lunch.”

In the API logs, we can verify the called endpoint and the query generated by Copilot. In this case, it was “food consumption.”

Find the query in JSON format in this file, along with the raw hits returned by elasticsearch for the previous query.

Now, let's test the date range search by querying for invoices issued within a specific timeframe:

These are the results:

The invoices issued between April 20th and April 22nd are

Invoice ID: INV-0011
- Description: Business lunch with client
- Services: Lunch at La Terraza Bistro - $85.00
- Total Amount: $85.00
- Issue Date: 2025-04-20
- View Invoice
Invoice ID: INV-0012
- Description: Hotel accommodation during client visit
- Services: 3-night stay at Hotel Central - $450.00
- Total Amount: $450.00
- Issue Date: 2025-04-21
- View Invoice
Invoice ID: INV-0013
- Description: Team-building activity
- Services: Escape room experience for team - $200.00
- Total Amount: $200.00
- Issue Date: 2025-04-22
- View Invoice

By reviewing the API requests, we can confirm that the data and the selected endpoint were correctly interpreted by Copilot:

The range query looks like this, and you can see here the data returned by elasticsearch.

Conclusion

In this article, we successfully integrated elasticsearch with Microsoft Copilot and created a personal assistant with access to our indices.

Using the OpenAI specification of your API with clear descriptions enables the agent to effectively select and utilize the appropriate endpoints.

This architecture is similar to Model Context Protocol (MCP) and in a future article, we’ll explore the new Microsoft Copilot integration with MCP and an elasticsearch MCP server.

Report an issue