- Elastic integrations
- Integrations quick reference
- 1Password
- Abnormal Security
- ActiveMQ
- Active Directory Entity Analytics
- Airflow
- Akamai
- Apache
- API (custom)
- Arbor Peakflow SP Logs
- Arista NG Firewall
- Atlassian
- Auditd
- Auth0
- authentik
- AWS
- Amazon CloudFront
- Amazon DynamoDB
- Amazon EBS
- Amazon EC2
- Amazon ECS
- Amazon EMR
- AWS API Gateway
- Amazon GuardDuty
- AWS Health
- Amazon Kinesis Data Firehose
- Amazon Kinesis Data Stream
- Amazon Managed Streaming for Apache Kafka (MSK)
- Amazon NAT Gateway
- Amazon RDS
- Amazon Redshift
- Amazon S3
- Amazon S3 Storage Lens
- Amazon Security Lake
- Amazon SNS
- Amazon SQS
- Amazon VPC
- Amazon VPN
- AWS Bedrock
- AWS Billing
- AWS CloudTrail
- AWS CloudWatch
- AWS ELB
- AWS Fargate
- AWS Inspector
- AWS Lambda
- AWS Logs (custom)
- AWS Network Firewall
- AWS Route 53
- AWS Security Hub
- AWS Transit Gateway
- AWS Usage
- AWS WAF
- Azure
- Activity logs
- App Service
- Application Gateway
- Application Insights metrics
- Application Insights metrics overview
- Application State Insights metrics
- Azure logs (v2 preview)
- Azure OpenAI
- Billing metrics
- Container instance metrics
- Container registry metrics
- Container service metrics
- Custom Azure Logs
- Custom Blob Storage Input
- Database Account metrics
- Event Hub input
- Firewall logs
- Frontdoor
- Functions
- Microsoft Entra ID
- Monitor metrics
- Network Watcher VNet
- Network Watcher NSG
- Platform logs
- Resource metrics
- Spring Cloud logs
- Storage Account metrics
- Virtual machines metrics
- Virtual machines scaleset metrics
- Barracuda
- BitDefender
- Bitwarden
- blacklens.io
- Blue Coat Director Logs
- BBOT (Bighuge BLS OSINT Tool)
- Box Events
- Bravura Monitor
- Broadcom ProxySG
- Canva
- Cassandra
- CEL Custom API
- Ceph
- Check Point
- Cilium Tetragon
- CISA Known Exploited Vulnerabilities
- Cisco
- Cisco Meraki Metrics
- Citrix
- Claroty CTD
- Cloudflare
- Cloud Asset Inventory
- CockroachDB Metrics
- Common Event Format (CEF)
- Containerd
- CoreDNS
- Corelight
- Couchbase
- CouchDB
- Cribl
- CrowdStrike
- Cyberark
- Cybereason
- CylanceProtect Logs
- Custom Websocket logs
- Darktrace
- Data Exfiltration Detection
- DGA
- Digital Guardian
- Docker
- Elastic APM
- Elastic Fleet Server
- Elastic Security
- Elastic Stack monitoring
- Elasticsearch Service Billing
- Envoy Proxy
- ESET PROTECT
- ESET Threat Intelligence
- etcd
- Falco
- F5
- File Integrity Monitoring
- FireEye Network Security
- First EPSS
- Forcepoint Web Security
- ForgeRock
- Fortinet
- Gigamon
- GitHub
- GitLab
- Golang
- Google Cloud
- Custom GCS Input
- GCP
- GCP Audit logs
- GCP Billing metrics
- GCP Cloud Run metrics
- GCP CloudSQL metrics
- GCP Compute metrics
- GCP Dataproc metrics
- GCP DNS logs
- GCP Firestore metrics
- GCP Firewall logs
- GCP GKE metrics
- GCP Load Balancing metrics
- GCP Metrics Input
- GCP PubSub logs (custom)
- GCP PubSub metrics
- GCP Redis metrics
- GCP Security Command Center
- GCP Storage metrics
- GCP VPC Flow logs
- GCP Vertex AI
- GoFlow2 logs
- Hadoop
- HAProxy
- Hashicorp Vault
- HTTP Endpoint logs (custom)
- IBM MQ
- IIS
- Imperva
- InfluxDb
- Infoblox
- Iptables
- Istio
- Jamf Compliance Reporter
- Jamf Pro
- Jamf Protect
- Jolokia Input
- Journald logs (custom)
- JumpCloud
- Kafka
- Keycloak
- Kubernetes
- LastPass
- Lateral Movement Detection
- Linux Metrics
- Living off the Land Attack Detection
- Logs (custom)
- Lumos
- Lyve Cloud
- Mattermost
- Memcached
- Menlo Security
- Microsoft
- Microsoft 365
- Microsoft Defender for Cloud
- Microsoft Defender for Endpoint
- Microsoft DHCP
- Microsoft DNS Server
- Microsoft Entra ID Entity Analytics
- Microsoft Exchange Online Message Trace
- Microsoft Exchange Server
- Microsoft Graph Activity Logs
- Microsoft M365 Defender
- Microsoft Office 365 Metrics Integration
- Microsoft Sentinel
- Microsoft SQL Server
- Mimecast
- ModSecurity Audit
- MongoDB
- MongoDB Atlas
- MySQL
- Nagios XI
- NATS
- NetFlow Records
- Netskope
- Network Beaconing Identification
- Network Packet Capture
- Nginx
- Okta
- Oracle
- OpenCanary
- Osquery
- Palo Alto
- pfSense
- PHP-FPM
- PingOne
- PingFederate
- Pleasant Password Server
- PostgreSQL
- Prometheus
- Proofpoint TAP
- Proofpoint On Demand
- Pulse Connect Secure
- Qualys VMDR
- QNAP NAS
- RabbitMQ Logs
- Radware DefensePro Logs
- Rapid7
- Redis
- Rubrik RSC Metrics Integration
- Salesforce
- SentinelOne
- ServiceNow
- Slack Logs
- Snort
- Snyk
- SonicWall Firewall
- Sophos
- Spring Boot
- SpyCloud Enterprise Protection
- SQL Input
- Squid Logs
- SRX
- STAN
- Statsd Input
- Sublime Security
- Suricata
- StormShield SNS
- Symantec
- Symantec Endpoint Security
- Sysmon for Linux
- Sysdig
- Syslog Router Integration
- System
- System Audit
- Tanium
- TCP Logs (custom)
- Teleport
- Tenable
- Threat intelligence
- ThreatConnect
- Threat Map
- Thycotic Secret Server
- Tines
- Traefik
- Trellix
- Trend Micro
- TYCHON Agentless
- UDP Logs (custom)
- Universal Profiling
- Vectra Detect
- VMware
- WatchGuard Firebox
- WebSphere Application Server
- Windows
- Wiz
- Zeek
- ZeroFox
- Zero Networks
- ZooKeeper Metrics
- Zoom
- Zscaler
container-logs
editcontainer-logs
editVersion |
1.70.0 (View all) |
Compatible Kibana version(s) |
8.15.0 or higher |
Supported Serverless project types |
Security |
Subscription level |
Basic |
container-logs integration collects and parses logs of Kubernetes containers.
It requires access to the log files in each Kubernetes node where the container logs are stored.
This defaults to /var/log/containers/*${kubernetes.container.id}.log
.
By default only container parser is enabled. Additional log parsers can be added as an advanced options configuration.
Rerouting and preserve original event based on pod annotations
editYou can customize the routing of container logs events and sending them to different datasets and namespaces,
as well as enable preserve_original_event
based on using pods' annotations.
Customization can happen at:
- pod definition time, e.g., using a deployment.
-
pod runtime, annotating pods using
kubectl
.
Set at pod definition time
editHere is an example of an Nginx deployment where we set both elastic.co/dataset
and elastic.co/namespace
annotations to route the container logs to specific datasets and namespace, respectively.
# nginx-deployment.yaml apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment spec: replicas: 1 selector: matchLabels: app: nginx template: metadata: annotations: elastic.co/dataset: kubernetes.container_logs.nginx elastic.co/namespace: nginx elastic.co/preserve_original_event: "true" labels: app: nginx app.kubernetes.io/name: myservice app.kubernetes.io/version: v0.1.2 app.kubernetes.io/instance: myservice-abcxzy spec: containers: - name: nginx-container image: nginx:latest ports: - containerPort: 80
Set at runtime
editSuppose you want to change the container logs routing and enable preserve_original_event
on a running container.
In that case, you can annotate the pod using kubectl
, and the integration will apply it immediately sending all the following documents to the new destination:
Here is an example where we route the container logs for a pod running the Elastic Agent to the kubernetes.container_logs.agents
dataset:
kubectl annotate pods elastic-agent-managed-daemonset-6p22g elastic.co/dataset=kubernetes.container_logs.agents
Here’s a similar example to change the namespace on a pod running Nginx:
kubectl annotate pods elastic-agent-managed-daemonset-6p22g elastic.co/namespace=nginx
Here is an example to enable preserve_original_event
on a pod running Nginx:
kubectl annotate pods elastic-agent-managed-daemonset-6p22g elastic.co/preserve_original_event=true
You can restore the standard settings by removing the annotations:
kubectl annotate pods elastic-agent-managed-daemonset-6p22g elastic.co/dataset- kubectl annotate pods elastic-agent-managed-daemonset-6p22g elastic.co/namespace- kubectl annotate pods elastic-agent-managed-daemonset-6p22g elastic.co/preserve_original_event-
Annotations Reference
editHere are the annotations available to customize routing:
Label | Description |
---|---|
|
Defines the target data stream’s dataset for this pod. |
|
Defines the target data stream’s namespace for this pod. |
|
Enables preserve_original_event for this pod. Use string true (case-insensitive) to enable |
Changelog
editChangelog
Version | Details | Kibana version(s) |
---|---|---|
1.70.0 |
Enhancement (View pull request) |
8.15.0 or higher |
1.69.0 |
Enhancement (View pull request) |
8.15.0 or higher |
1.68.1 |
Bug fix (View pull request) |
8.15.0 or higher |
1.68.0 |
Enhancement (View pull request) |
8.15.0 or higher |
1.67.0 |
Enhancement (View pull request) |
8.15.0 or higher |
1.66.4 |
Bug fix (View pull request) |
8.15.0 or higher |
1.66.3 |
Enhancement (View pull request) |
8.15.0 or higher |
1.66.2 |
Bug fix (View pull request) |
8.15.0 or higher |
1.66.1 |
Bug fix (View pull request) |
8.15.0 or higher |
1.66.0 |
Enhancement (View pull request) |
8.15.0 or higher |
1.65.0 |
Enhancement (View pull request) |
8.15.0 or higher |
1.64.0 |
Enhancement (View pull request) |
8.15.0 or higher |
1.63.1 |
Bug fix (View pull request) |
8.15.0 or higher |
1.63.0 |
Enhancement (View pull request) |
8.15.0 or higher |
1.62.1 |
Bug fix (View pull request) |
8.14.0 or higher |
1.62.0 |
Enhancement (View pull request) |
8.14.0 or higher |
1.61.1 |
Bug fix (View pull request) |
8.14.0 or higher |
1.61.0 |
Enhancement (View pull request) |
8.14.0 or higher |
1.60.0 |
Bug fix (View pull request) |
8.14.0 or higher |
1.59.0 |
Enhancement (View pull request) |
8.14.0 or higher |
1.58.0 |
Enhancement (View pull request) |
8.12.0 or higher |
1.57.0 |
Enhancement (View pull request) |
8.12.0 or higher |
1.56.0 |
Enhancement (View pull request) |
8.12.0 or higher |
1.55.1 |
Enhancement (View pull request) |
8.11.0 or higher |
1.55.0 |
Enhancement (View pull request) |
8.11.0 or higher |
1.54.0 |
Enhancement (View pull request) |
8.11.0 or higher |
1.53.0 |
Enhancement (View pull request) |
8.11.0 or higher |
1.52.0 |
Enhancement (View pull request) |
8.11.0 or higher |
1.51.0 |
Enhancement (View pull request) |
8.10.2 or higher |
1.50.0 |
Enhancement (View pull request) |
8.10.2 or higher |
1.49.0 |
Enhancement (View pull request) |
8.10.2 or higher |
1.48.0 |
Enhancement (View pull request) |
8.10.2 or higher |
1.47.0 |
Enhancement (View pull request) |
8.10.2 or higher |
1.46.0 |
Enhancement (View pull request) |
8.10.1 or higher |
1.45.0 |
Enhancement (View pull request) |
8.10.0 or higher |
1.44.0 |
Enhancement (View pull request) |
8.10.0 or higher |
1.43.1 |
Enhancement (View pull request) |
8.8.0 or higher |
1.43.0 |
Enhancement (View pull request) |
8.8.0 or higher |
1.42.0 |
Enhancement (View pull request) |
8.8.0 or higher |
1.41.0 |
Enhancement (View pull request) |
8.8.0 or higher |
1.40.0 |
Bug fix (View pull request) |
8.8.0 or higher |
1.40.0-beta.2 |
Bug fix (View pull request) |
— |
1.40.0-beta.1 |
Bug fix (View pull request) |
— |
1.40.0-beta |
Enhancement (View pull request) |
— |
1.39.0 |
Enhancement (View pull request) |
8.6.1 or higher |
1.38.1 |
Enhancement (View pull request) |
8.6.1 or higher |
1.38.0 |
Enhancement (View pull request) |
8.6.1 or higher |
1.37.0 |
Enhancement (View pull request) |
8.6.1 or higher |
1.36.0 |
Enhancement (View pull request) |
8.6.1 or higher |
1.35.0 |
Enhancement (View pull request) |
8.6.1 or higher |
1.34.1 |
Enhancement (View pull request) |
8.6.1 or higher |
1.34.0 |
Enhancement (View pull request) |
8.6.1 or higher |
1.33.0 |
Enhancement (View pull request) |
8.6.1 or higher |
1.32.2 |
Enhancement (View pull request) |
8.6.1 or higher |
1.32.1 |
Enhancement (View pull request) |
8.6.1 or higher |
1.32.0 |
Enhancement (View pull request) |
8.6.1 or higher |
1.31.2 |
Enhancement (View pull request) |
8.6.1 or higher |
1.31.1 |
Enhancement (View pull request) |
8.6.1 or higher |
1.31.0 |
Enhancement (View pull request) |
8.6.0 or higher |
1.30.0 |
Enhancement (View pull request) |
8.6.0 or higher |
1.29.2 |
Bug fix (View pull request) |
8.5.0 or higher |
1.29.1 |
Bug fix (View pull request) |
8.5.0 or higher |
1.29.0 |
Bug fix (View pull request) |
8.5.0 or higher |
1.28.2 |
Bug fix (View pull request) |
8.5.0 or higher |
1.28.1 |
Enhancement (View pull request) |
8.5.0 or higher |
1.28.0 |
Enhancement (View pull request) |
8.5.0 or higher |
1.27.1 |
Enhancement (View pull request) |
8.5.0 or higher |
1.27.0 |
Enhancement (View pull request) |
8.5.0 or higher |
1.26.0 |
Enhancement (View pull request) |
8.4.0 or higher |
1.25.0 |
Enhancement (View pull request) |
8.4.0 or higher |
1.24.0 |
Enhancement (View pull request) |
8.4.0 or higher |
1.23.1 |
Enhancement (View pull request) |
8.4.0 or higher |
1.23.0 |
Enhancement (View pull request) |
8.4.0 or higher |
1.22.1 |
Enhancement (View pull request) |
8.4.0 or higher |
1.22.0 |
Enhancement (View pull request) |
8.4.0 or higher |
1.21.2 |
Bug fix (View pull request) |
8.3.0 or higher |
1.21.1 |
Enhancement (View pull request) |
8.3.0 or higher |
1.21.0 |
Enhancement (View pull request) |
8.3.0 or higher |
1.20.0 |
Enhancement (View pull request) |
8.2.0 or higher |
1.19.1 |
Enhancement (View pull request) |
8.2.0 or higher |
1.19.0 |
Enhancement (View pull request) |
8.2.0 or higher |
1.18.1 |
Enhancement (View pull request) |
8.2.0 or higher |
1.18.0 |
Enhancement (View pull request) |
8.2.0 or higher |
1.17.3 |
Bug fix (View pull request) |
7.16.0 or higher |
1.17.2 |
Bug fix (View pull request) |
7.16.0 or higher |
1.17.1 |
Enhancement (View pull request) |
— |
1.17.0 |
Enhancement (View pull request) |
— |
1.16.0 |
Enhancement (View pull request) |
— |
1.15.0 |
Enhancement (View pull request) |
— |
1.14.3 |
Bug fix (View pull request) |
— |
1.14.2 |
Bug fix (View pull request) |
— |
1.14.1 |
Bug fix (View pull request) |
— |
1.14.0 |
Enhancement (View pull request) |
— |
1.13.0 |
Enhancement (View pull request) |
— |
1.12.0 |
Enhancement (View pull request) |
— |
1.11.0 |
Enhancement (View pull request) |
— |
1.10.0 |
Enhancement (View pull request) |
— |
1.9.0 |
Enhancement (View pull request) |
7.16.0 or higher |
1.8.1 |
Bug fix (View pull request) |
7.16.0 or higher |
1.8.0 |
Enhancement (View pull request) |
7.16.0 or higher |
1.7.0 |
Enhancement (View pull request) |
7.16.0 or higher |
1.6.0 |
Enhancement (View pull request) |
7.16.0 or higher |
1.5.0 |
Enhancement (View pull request) |
7.16.0 or higher |
1.4.2 |
Enhancement (View pull request) |
— |
1.4.1 |
Enhancement (View pull request) |
8.0.0 or higher |
1.4.0 |
Enhancement (View pull request) |
— |
1.3.3 |
Bug fix (View pull request) |
— |
1.3.2 |
Enhancement (View pull request) |
— |
1.3.1 |
Enhancement (View pull request) |
— |
1.3.0 |
Enhancement (View pull request) |
— |
1.2.1 |
Bug fix (View pull request) |
— |
1.2.0 |
Enhancement (View pull request) |
— |
1.1.1 |
Bug fix (View pull request) |
— |
1.1.0 |
Enhancement (View pull request) |
7.15.0 or higher |
1.0.0 |
Enhancement (View pull request) |
— |
0.14.1 |
Enhancement (View pull request) |
— |
0.14.0 |
Enhancement (View pull request) |
— |
0.13.0 |
Enhancement (View pull request) |
— |
0.12.2 |
Bug fix (View pull request) |
— |
0.12.1 |
Bug fix (View pull request) |
— |
0.12.0 |
Enhancement (View pull request) |
— |
0.11.1 |
Enhancement (View pull request) |
— |
0.11.0 |
Enhancement (View pull request) |
— |
0.10.0 |
Enhancement (View pull request) |
— |
0.9.1 |
Bug fix (View pull request) |
— |
0.9.0 |
Enhancement (View pull request) |
— |
0.8.0 |
Enhancement (View pull request) |
— |
0.7.0 |
Enhancement (View pull request) |
— |
0.6.0 |
Enhancement (View pull request) |
— |
0.5.3 |
Enhancement (View pull request) |
— |
0.5.2 |
Bug fix (View pull request) |
— |
0.5.1 |
Bug fix (View pull request) |
— |
0.5.0 |
Enhancement (View pull request) |
— |
0.4.5 |
Enhancement (View pull request) |
— |
0.4.4 |
Enhancement (View pull request) |
— |
0.4.3 |
Bug fix (View pull request) |
— |
0.4.2 |
Bug fix (View pull request) |
— |
0.4.1 |
Enhancement (View pull request) |
— |
0.1.0 |
Enhancement (View pull request) |
— |
On this page
ElasticON events are back!
Learn about the Elastic Search AI Platform from the experts at our live events.
Register now