Elastic Agent to elasticsearch with Kafka as middleware message queue
- Ingest models
-
Elastic Agent to Logstash to Kafka to Logstash to elasticsearch: Kafka as middleware message queue.
Logstash reads data from Kafka and routes it to elasticsearch clusters and other destinations.Elastic Agent to Logstash to Kafka to Kafka ES Sink to elasticsearch: Kafka as middleware message queue.
Kafka ES sink connector reads from Kafka and writes to elasticsearch. - Use when
- You are standardizing on Kafka as middleware message queue between Elastic Agent and elasticsearch
- Notes
- The transformation from raw data to Elastic Common Schema (ECS) and any other enrichment can be handled by Logstash as described in Elastic Agent to Logstash (for enrichment) to elasticsearch.