You are viewing docs on Elastic's new documentation system, currently in technical preview. For all other Elastic docs, visit

Custom Kafka Logs

Collect data from kafka topic with Elastic Agent.

What is an Elastic integration?

This integration is powered by Elastic Agent. Elastic Agent is a single, unified way to add monitoring for logs, metrics, and other types of data to a host. It can also protect hosts from security threats, query data from operating systems, forward data from remote services or hardware, and more. Refer to our documentation for a detailed comparison between Beats and Elastic Agent.

Prefer to use Beats for this use case? See Filebeat modules for logs or Metricbeat modules for metrics.

The custom Kafka log integration is used to read from topics in a Kafka cluster.

To configure this integration, specify a list of one or more hosts in the cluster to bootstrap the connection with, a list of topics to track, and a group_id for the connection.


This Integration works with all Kafka versions in between 0.11 and 2.8.0. Older versions might work as well, but are not supported.

Ingest Pipelines

Custom ingest pipelines may be added by adding the name to the pipeline configuration option, creating custom ingest pipelines can be done either through the API or the Ingest Node Pipeline UI.


Enhancement View pull request
Rename ownership from obs-service-integrations to obs-infraobs-integrations
Enhancement View pull request
Added categories and/or subcategories.
Enhancement View pull request
Update ECS version to 8.5.1
Enhancement View pull request
Initial Release