Building Elasticsearch APIs with FastAPI

Want to get Elastic certified? Find out when the next Elasticsearch Engineer training is running!

Elasticsearch is packed with new features to help you build the best search solutions for your use case. Dive into our sample notebooks to learn more, start a free cloud trial, or try Elastic on your local machine now.

From the Elasticsearch in JavaScript article, we learned that you should not expose your Elasticsearch instance to the internet but build an API layer instead. Exposing your cluster’s URL, index names, or API keys makes it easier for attackers to target your data and unnecessarily expands your attack surface. Even if you sanitize requests, attackers can send heavy or complex queries to overload your cluster. That’s why it’s best to hide not only the cluster, but also the query logic — let the user control only what’s needed, like the search keyword and never the full query.

Node.js is the alternative for JavaScript developers, and FastAPI is the alternative for Python developers. FastAPI has become popular due to its simplicity and out-of-the-box performance.

In this article, we will build the API layer between Elasticsearch and the client application (commonly a web browser) using FastAPI. Then, we’ll explore some common use cases that can be covered using FastAPI’s native capabilities.

You can find the notebook for this application here.

Preparing data

For this article, we will use a dataset with details of veterinary visits. Below is a sample document:

First, we need to install the Elasticsearch client to be able to query our cluster:

Now, we import the Elasticsearch client, helpers, and getpass to capture environment variables from the terminal.

We define the index name and initialize the Elasticsearch client with the Elasticsearch endpoint and API Key.

And let's create mappings:

Finally, download the dataset and put it in the same folder where the script is located. With it, we can ingest the data into Elasticsearch using the Bulk API:

If everything goes well, you should see the following message:

Now, the data is available in Elasticsearch and ready to use. In the next steps, we’ll build the API to showcase FastAPI’s features.

Hello, world!

To get started, we just need to install FastAPI and Uvicorn for the server creation, Pydantic for schema handling, and Elasticsearch to store and search data.

We start by importing the libraries and creating our FastAPI server instance.

Then, we can create a ping endpoint to check the status of our server.

Request:

Response:

Now, let’s create a search endpoint for the sake of protecting our Elasticsearch instance by just receiving the Elasticsearch query body as is and returning the response body as is too.

We are going to try by searching for dental cleaning visits using match_phrase:

Request:

Response:

Typed request and response

One of FastAPI’s key features is its integration with Pydantic for data schema handling, allowing you to define classes with type annotations and benefit from automatic data validation. In most cases, typing your data is essential for building robust and stable applications. It also enables model reuse, making the code easier to read, maintain, and document.

However, strict typing and schema enforcement can be limiting in scenarios where you need to handle highly dynamic or unpredictable data structures.

Let’s convert our request and response for the search endpoint.

We can create classes to validate field types, set default values, define enums, lists, etc. We are going to create classes for the user request and the Elasticsearch response by applying some of these concepts.

Now, the user just needs to send the search term, and optionally the results limit, and it will get back a list of hits with owner_name and visit_details.

Now the endpoint is much simpler for the user, and more secure for us because the user can only control a portion of the query and not all of it.

Giving the user total access to the _search query body is considered a security risk because a malicious user could produce a query that overwhelms the cluster. With this approach, the user can only set what’s inside the match phrase clause, making the endpoint more secure.

Same with the response. We can send a cleaner version of the result, hiding fields like _id, _score, _index, etc.

Request:

Response:

Background tasks

Another FastAPI feature we can leverage with Elasticsearch is background tasks. With background tasks, you can return something instantly to the user while continuing to execute the task in the background. This feature is especially useful for long-running tasks.

In Elasticsearch, we use the wait_for_completion=false parameter to get the task ID and close the connection instead of waiting for the task to finish. We can use the tasks API to check the task status. Some examples are _reindex, _update_by_query, and _delete_by_query.

Imagine you want to give your users a way to trigger a delete operation of a couple of million documents based on a field value and notify them when it finishes. You can achieve this by using FastAPI background tasks and Elasticsearch’s wait_for_completion.

We start by creating a function to check the tasks API every 2 seconds.

Now, we can create the endpoint that receives the value we want to use as a filter to delete the documents. Call the _delete_by_query API with wait_for_completion=false and use the task id we get back to create a background task that calls our check_task method.

The first parameter of background_tasks.add_task is the function we want to execute, and the rest of the parameters are the ones that the invoked function uses.

This is how you remove all the documents containing “Buddy” in the pet_name.keyword field:

Response:

Backend logs:

Run API

Add the following block to expose the server to port 8000:

Run the FastAPI application with the following command:

Conclusion

FastAPI makes it simple to build a safe and clean API layer for Elasticsearch. With type checks, async support, and background tasks out of the box, you can handle common use cases without much overhead.

These examples are just a starting point—expand them with auth, pagination, or even websockets as needed. The key is to keep your cluster safe and your API clear and easy to maintain.