Install Elasticsearch with Docker
editInstall Elasticsearch with Docker
editDocker images for Elasticsearch are available from the Elastic Docker registry. A list of all published Docker images and tags is available at www.docker.elastic.co. The source code is in GitHub.
This package contains both free and subscription features. Start a 30-day trial to try out all of the features.
If you just want to test Elasticsearch in local development, refer to Run Elasticsearch locally. Please note that this setup is not suitable for production environments.
Run Elasticsearch in Docker
editUse Docker commands to start a single-node Elasticsearch cluster for development or testing. You can then run additional Docker commands to add nodes to the test cluster or run Kibana.
This setup doesnR17;t run multiple Elasticsearch nodes or Kibana by default. To create a multi-node cluster with Kibana, use Docker Compose instead. See Start a multi-node cluster with Docker Compose.
Hardened Docker images
editYou can also use the hardened Wolfi image for additional security. Using Wolfi images requires Docker version 20.10.10 or higher.
To use the Wolfi image, append -wolfi
to the image tag in the Docker command.
For example:
docker pull docker.elastic.co/elasticsearch/elasticsearch-wolfi:8.17.0
Start a single-node cluster
edit-
Install Docker. Visit Get Docker to install Docker for your environment.
If using Docker Desktop, make sure to allocate at least 4GB of memory. You can adjust memory usage in Docker Desktop by going to Settings > Resources.
-
Create a new docker network.
docker network create elastic
-
Pull the Elasticsearch Docker image.
docker pull docker.elastic.co/elasticsearch/elasticsearch:8.17.0
-
Optional: Install Cosign for your environment. Then use Cosign to verify the Elasticsearch imageR17;s signature.
wget https://artifacts.elastic.co/cosign.pub cosign verify --key cosign.pub docker.elastic.co/elasticsearch/elasticsearch:8.17.0
The
cosign
command prints the check results and the signature payload in JSON format:Verification for docker.elastic.co/elasticsearch/elasticsearch:8.17.0 -- The following checks were performed on each of these signatures: - The cosign claims were validated - Existence of the claims in the transparency log was verified offline - The signatures were verified against the specified public key
-
Start an Elasticsearch container.
docker run --name es01 --net elastic -p 9200:9200 -it -m 1GB docker.elastic.co/elasticsearch/elasticsearch:8.17.0
Use the
-m
flag to set a memory limit for the container. This removes the need to manually set the JVM size.Machine learning features such as semantic search with ELSER require a larger container with more than 1GB of memory. If you intend to use the machine learning capabilities, then start the container with this command:
docker run --name es01 --net elastic -p 9200:9200 -it -m 6GB -e "xpack.ml.use_auto_machine_memory_percent=true" docker.elastic.co/elasticsearch/elasticsearch:8.17.0
The command prints the
elastic
user password and an enrollment token for Kibana. -
Copy the generated
elastic
password and enrollment token. These credentials are only shown when you start Elasticsearch for the first time. You can regenerate the credentials using the following commands.docker exec -it es01 /usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic docker exec -it es01 /usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana
We recommend storing the
elastic
password as an environment variable in your shell. Example:export ELASTIC_PASSWORD="your_password"
-
Copy the
http_ca.crt
SSL certificate from the container to your local machine.docker cp es01:/usr/share/elasticsearch/config/certs/http_ca.crt .
-
Make a REST API call to Elasticsearch to ensure the Elasticsearch container is running.
curl --cacert http_ca.crt -u elastic:$ELASTIC_PASSWORD https://localhost:9200
Add more nodes
edit-
Use an existing node to generate a enrollment token for the new node.
docker exec -it es01 /usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s node
The enrollment token is valid for 30 minutes.
-
Start a new Elasticsearch container. Include the enrollment token as an environment variable.
docker run -e ENROLLMENT_TOKEN="<token>" --name es02 --net elastic -it -m 1GB docker.elastic.co/elasticsearch/elasticsearch:8.17.0
-
Call the cat nodes API to verify the node was added to the cluster.
curl --cacert http_ca.crt -u elastic:$ELASTIC_PASSWORD https://localhost:9200/_cat/nodes
Run Kibana
edit-
Pull the Kibana Docker image.
docker pull docker.elastic.co/kibana/kibana:8.17.0
-
Optional: Verify the Kibana imageR17;s signature.
wget https://artifacts.elastic.co/cosign.pub cosign verify --key cosign.pub docker.elastic.co/kibana/kibana:8.17.0
-
Start a Kibana container.
docker run --name kib01 --net elastic -p 5601:5601 docker.elastic.co/kibana/kibana:8.17.0
- When Kibana starts, it outputs a unique generated link to the terminal. To access Kibana, open this link in a web browser.
-
In your browser, enter the enrollment token that was generated when you started Elasticsearch.
To regenerate the token, run:
docker exec -it es01 /usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana
-
Log in to Kibana as the
elastic
user with the password that was generated when you started Elasticsearch.To regenerate the password, run:
docker exec -it es01 /usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic
Remove containers
editTo remove the containers and their network, run:
# Remove the Elastic network docker network rm elastic # Remove Elasticsearch containers docker rm es01 docker rm es02 # Remove the Kibana container docker rm kib01
Next steps
editYou now have a test Elasticsearch environment set up. Before you start serious development or go into production with Elasticsearch, review the requirements and recommendations to apply when running Elasticsearch in Docker in production.
Start a multi-node cluster with Docker Compose
editUse Docker Compose to start a three-node Elasticsearch cluster with Kibana. Docker Compose lets you start multiple containers with a single command.
Configure and start the cluster
edit-
Install Docker Compose. Visit the Docker Compose docs to install Docker Compose for your environment.
If youR17;re using Docker Desktop, Docker Compose is installed automatically. Make sure to allocate at least 4GB of memory to Docker Desktop. You can adjust memory usage in Docker Desktop by going to Settings > Resources.
- Create or navigate to an empty directory for the project.
-
Download and save the following files in the project directory:
-
In the
.env
file, specify a password for theELASTIC_PASSWORD
andKIBANA_PASSWORD
variables.The passwords must be alphanumeric and canR17;t contain special characters, such as
!
or@
. The bash script included in thedocker-compose.yml
file only works with alphanumeric characters. Example:# Password for the 'elastic' user (at least 6 characters) ELASTIC_PASSWORD=changeme # Password for the 'kibana_system' user (at least 6 characters) KIBANA_PASSWORD=changeme ...
-
In the
.env
file, setSTACK_VERSION
to the current Elastic Stack version.... # Version of Elastic products STACK_VERSION=8.17.0 ...
-
By default, the Docker Compose configuration exposes port
9200
on all network interfaces.To avoid exposing port
9200
to external hosts, setES_PORT
to127.0.0.1:9200
in the.env
file. This ensures Elasticsearch is only accessible from the host machine.... # Port to expose Elasticsearch HTTP API to the host #ES_PORT=9200 ES_PORT=127.0.0.1:9200 ...
-
To start the cluster, run the following command from the project directory.
docker-compose up -d
- After the cluster has started, open http://localhost:5601 in a web browser to access Kibana.
-
Log in to Kibana as the
elastic
user using theELASTIC_PASSWORD
you set earlier.
Stop and remove the cluster
editTo stop the cluster, run docker-compose down
. The data in the Docker volumes
is preserved and loaded when you restart the cluster with docker-compose up
.
docker-compose down
To delete the network, containers, and volumes when you stop the cluster,
specify the -v
option:
docker-compose down -v
Next steps
editYou now have a test Elasticsearch environment set up. Before you start serious development or go into production with Elasticsearch, review the requirements and recommendations to apply when running Elasticsearch in Docker in production.
Using the Docker images in production
editThe following requirements and recommendations apply when running Elasticsearch in Docker in production.
Set vm.max_map_count
to at least 262144
editThe vm.max_map_count
kernel setting must be set to at least 262144
for production use.
How you set vm.max_map_count
depends on your platform.
Linux
editTo view the current value for the vm.max_map_count
setting, run:
grep vm.max_map_count /etc/sysctl.conf vm.max_map_count=262144
To apply the setting on a live system, run:
sysctl -w vm.max_map_count=262144
To permanently change the value for the vm.max_map_count
setting, update the
value in /etc/sysctl.conf
.
macOS with Docker for Mac
editThe vm.max_map_count
setting must be set within the xhyve virtual machine:
-
From the command line, run:
screen ~/Library/Containers/com.docker.docker/Data/vms/0/tty
-
Press enter and use
sysctl
to configurevm.max_map_count
:sysctl -w vm.max_map_count=262144
-
To exit the
screen
session, typeCtrl a d
.
Windows and macOS with Docker Desktop
editThe vm.max_map_count
setting must be set via docker-machine:
docker-machine ssh sudo sysctl -w vm.max_map_count=262144
Windows with Docker Desktop WSL 2 backend
editThe vm.max_map_count
setting must be set in the "docker-desktop" WSL instance before the
Elasticsearch container will properly start. There are several ways to do this, depending
on your version of Windows and your version of WSL.
If you are on Windows 10 before version 22H2, or if you are on Windows 10 version 22H2 using the
built-in version of WSL, you must either manually set it every time you restart Docker before starting
your Elasticsearch container, or (if you do not wish to do so on every restart) you must globally set
every WSL2 instance to have the vm.max_map_count
changed. This is because these versions of WSL
do not properly process the /etc/sysctl.conf file.
To manually set it every time you reboot, you must run the following commands in a command prompt or PowerShell window every time you restart Docker:
wsl -d docker-desktop -u root sysctl -w vm.max_map_count=262144
If you are on these versions of WSL and you do not want to have to run those commands every time you restart Docker, you can globally change every WSL distribution with this setting by modifying your %USERPROFILE%\.wslconfig as follows:
[wsl2] kernelCommandLine = "sysctl.vm.max_map_count=262144"
This will cause all WSL2 VMs to have that setting assigned when they start.
If you are on Windows 11, or Windows 10 version 22H2 and have installed the Microsoft Store version of WSL, you can modify the /etc/sysctl.conf within the "docker-desktop" WSL distribution, perhaps with commands like this:
wsl -d docker-desktop -u root vi /etc/sysctl.conf
and appending a line which reads:
vm.max_map_count = 262144
Configuration files must be readable by the elasticsearch
user
editBy default, Elasticsearch runs inside the container as user elasticsearch
using
uid:gid 1000:0
.
One exception is Openshift,
which runs containers using an arbitrarily assigned user ID.
Openshift presents persistent volumes with the gid set to 0
, which works without any adjustments.
If you are bind-mounting a local directory or file, it must be readable by the elasticsearch
user.
In addition, this user must have write access to the config, data and log dirs
(Elasticsearch needs write access to the config
directory so that it can generate a keystore).
A good strategy is to grant group access to gid 0
for the local directory.
For example, to prepare a local directory for storing data through a bind-mount:
mkdir esdatadir chmod g+rwx esdatadir chgrp 0 esdatadir
You can also run an Elasticsearch container using both a custom UID and GID. You must ensure that file permissions will not prevent Elasticsearch from executing. You can use one of two options:
-
Bind-mount the
config
,data
andlogs
directories. If you intend to install plugins and prefer not to create a custom Docker image, you must also bind-mount theplugins
directory. -
Pass the
--group-add 0
command line option todocker run
. This ensures that the user under which Elasticsearch is running is also a member of theroot
(GID 0) group inside the container.
Increase ulimits for nofile and nproc
editIncreased ulimits for nofile and nproc must be available for the Elasticsearch containers. Verify the init system for the Docker daemon sets them to acceptable values.
To check the Docker daemon defaults for ulimits, run:
docker run --rm docker.elastic.co/elasticsearch/elasticsearch:8.17.0 /bin/bash -c 'ulimit -Hn && ulimit -Sn && ulimit -Hu && ulimit -Su'
If needed, adjust them in the Daemon or override them per container.
For example, when using docker run
, set:
--ulimit nofile=65535:65535
Disable swapping
editSwapping needs to be disabled for performance and node stability. For information about ways to do this, see Disable swapping.
If you opt for the bootstrap.memory_lock: true
approach,
you also need to define the memlock: true
ulimit in the
Docker Daemon,
or explicitly set for the container as shown in the sample compose file.
When using docker run
, you can specify:
-e "bootstrap.memory_lock=true" --ulimit memlock=-1:-1
Randomize published ports
editThe image exposes
TCP ports 9200 and 9300. For production clusters, randomizing the
published ports with --publish-all
is recommended,
unless you are pinning one container per host.
Manually set the heap size
editBy default, Elasticsearch automatically sizes JVM heap based on a nodesR17;s roles and the total memory available to the nodeR17;s container. We recommend this default sizing for most production environments. If needed, you can override default sizing by manually setting JVM heap size.
To manually set the heap size in production, bind mount a JVM
options file under /usr/share/elasticsearch/config/jvm.options.d
that
includes your desired heap size settings.
For testing, you can also manually set the heap size using the ES_JAVA_OPTS
environment variable. For example, to use 1GB, use the following command.
docker run -e ES_JAVA_OPTS="-Xms1g -Xmx1g" -e ENROLLMENT_TOKEN="<token>" --name es01 -p 9200:9200 --net elastic -it docker.elastic.co/elasticsearch/elasticsearch:8.17.0
The ES_JAVA_OPTS
variable overrides all other JVM options.
We do not recommend using ES_JAVA_OPTS
in production.
Pin deployments to a specific image version
editPin your deployments to a specific version of the Elasticsearch Docker image. For
example docker.elastic.co/elasticsearch/elasticsearch:8.17.0
.
Always bind data volumes
editYou should use a volume bound on /usr/share/elasticsearch/data
for the following reasons:
- The data of your Elasticsearch node wonR17;t be lost if the container is killed
- Elasticsearch is I/O sensitive and the Docker storage driver is not ideal for fast I/O
- It allows the use of advanced Docker volume plugins
Avoid using loop-lvm
mode
editIf you are using the devicemapper storage driver, do not use the default loop-lvm
mode.
Configure docker-engine to use
direct-lvm.
Centralize your logs
editConsider centralizing your logs by using a different logging driver. Also note that the default json-file logging driver is not ideally suited for production use.
Configuring Elasticsearch with Docker
editWhen you run in Docker, the Elasticsearch configuration files are loaded from
/usr/share/elasticsearch/config/
.
To use custom configuration files, you bind-mount the files over the configuration files in the image.
You can set individual Elasticsearch configuration parameters using Docker environment variables. The sample compose file and the single-node example use this method. You can use the setting name directly as the environment variable name. If you cannot do this, for example because your orchestration platform forbids periods in environment variable names, then you can use an alternative style by converting the setting name as follows.
- Change the setting name to uppercase
-
Prefix it with
ES_SETTING_
-
Escape any underscores (
_
) by duplicating them -
Convert all periods (
.
) to underscores (_
)
For example, -e bootstrap.memory_lock=true
becomes
-e ES_SETTING_BOOTSTRAP_MEMORY__LOCK=true
.
You can use the contents of a file to set the value of the
ELASTIC_PASSWORD
or KEYSTORE_PASSWORD
environment variables, by
suffixing the environment variable name with _FILE
. This is useful for
passing secrets such as passwords to Elasticsearch without specifying them directly.
For example, to set the Elasticsearch bootstrap password from a file, you can bind mount the
file and set the ELASTIC_PASSWORD_FILE
environment variable to the mount location.
If you mount the password file to /run/secrets/bootstrapPassword.txt
, specify:
-e ELASTIC_PASSWORD_FILE=/run/secrets/bootstrapPassword.txt
You can override the default command for the image to pass Elasticsearch configuration parameters as command line options. For example:
docker run <various parameters> bin/elasticsearch -Ecluster.name=mynewclustername
While bind-mounting your configuration files is usually the preferred method in production, you can also create a custom Docker image that contains your configuration.
Mounting Elasticsearch configuration files
editCreate custom config files and bind-mount them over the corresponding files in the Docker image.
For example, to bind-mount custom_elasticsearch.yml
with docker run
, specify:
-v full_path_to/custom_elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
If you bind-mount a custom elasticsearch.yml
file, ensure it includes the
network.host: 0.0.0.0
setting. This setting ensures the node is reachable for
HTTP and transport traffic, provided its ports are exposed. The Docker imageR17;s
built-in elasticsearch.yml
file includes this setting by default.
The container runs Elasticsearch as user elasticsearch
using
uid:gid 1000:0
. Bind mounted host directories and files must be accessible by this user,
and the data and log directories must be writable by this user.
Create an encrypted Elasticsearch keystore
editBy default, Elasticsearch will auto-generate a keystore file for secure settings. This file is obfuscated but not encrypted.
To encrypt your secure settings with a password and have them persist outside
the container, use a docker run
command to manually create the keystore
instead. The command must:
-
Bind-mount the
config
directory. The command will create anelasticsearch.keystore
file in this directory. To avoid errors, do not directly bind-mount theelasticsearch.keystore
file. -
Use the
elasticsearch-keystore
tool with thecreate -p
option. YouR17;ll be prompted to enter a password for the keystore.
For example:
docker run -it --rm \ -v full_path_to/config:/usr/share/elasticsearch/config \ docker.elastic.co/elasticsearch/elasticsearch:8.17.0 \ bin/elasticsearch-keystore create -p
You can also use a docker run
command to add or update secure settings in the
keystore. YouR17;ll be prompted to enter the setting values. If the keystore is
encrypted, youR17;ll also be prompted to enter the keystore password.
docker run -it --rm \ -v full_path_to/config:/usr/share/elasticsearch/config \ docker.elastic.co/elasticsearch/elasticsearch:8.17.0 \ bin/elasticsearch-keystore \ add my.secure.setting \ my.other.secure.setting
If youR17;ve already created the keystore and donR17;t need to update it, you can
bind-mount the elasticsearch.keystore
file directly. You can use the
KEYSTORE_PASSWORD
environment variable to provide the keystore password to the
container at startup. For example, a docker run
command might have the
following options:
-v full_path_to/config/elasticsearch.keystore:/usr/share/elasticsearch/config/elasticsearch.keystore -e KEYSTORE_PASSWORD=mypassword
Using custom Docker images
editIn some environments, it might make more sense to prepare a custom image that contains
your configuration. A Dockerfile
to achieve this might be as simple as:
FROM docker.elastic.co/elasticsearch/elasticsearch:8.17.0 COPY --chown=elasticsearch:elasticsearch elasticsearch.yml /usr/share/elasticsearch/config/
You could then build and run the image with:
docker build --tag=elasticsearch-custom . docker run -ti -v /usr/share/elasticsearch/data elasticsearch-custom
Some plugins require additional security permissions. You must explicitly accept them either by:
-
Attaching a
tty
when you run the Docker image and allowing the permissions when prompted. -
Inspecting the security permissions and accepting them (if appropriate) by adding the
--batch
flag to the plugin install command.
See Plugin management for more information.
Troubleshoot Docker errors for Elasticsearch
editHere’s how to resolve common errors when running Elasticsearch with Docker.
elasticsearch.keystore is a directory
editException in thread "main" org.elasticsearch.bootstrap.BootstrapException: java.io.IOException: Is a directory: SimpleFSIndexInput(path="/usr/share/elasticsearch/config/elasticsearch.keystore") Likely root cause: java.io.IOException: Is a directory
A keystore-related docker run
command attempted
to directly bind-mount an elasticsearch.keystore
file that doesnR17;t exist. If
you use the -v
or --volume
flag to mount a file that doesnR17;t exist, Docker
instead creates a directory with the same name.
To resolve this error:
-
Delete the
elasticsearch.keystore
directory in theconfig
directory. -
Update the
-v
or--volume
flag to point to theconfig
directory path rather than the keystore fileR17;s path. For an example, see Create an encrypted Elasticsearch keystore. - Retry the command.
elasticsearch.keystore: Device or resource busy
editException in thread "main" java.nio.file.FileSystemException: /usr/share/elasticsearch/config/elasticsearch.keystore.tmp -> /usr/share/elasticsearch/config/elasticsearch.keystore: Device or resource busy
A docker run
command attempted to update the
keystore while directly bind-mounting the elasticsearch.keystore
file. To
update the keystore, the container requires access to other files in the
config
directory, such as keystore.tmp
.
To resolve this error:
-
Update the
-v
or--volume
flag to point to theconfig
directory path rather than the keystore fileR17;s path. For an example, see Create an encrypted Elasticsearch keystore. - Retry the command.