- Elastic integrations
- Integrations quick reference
- 1Password
- Abnormal Security
- ActiveMQ
- Active Directory Entity Analytics
- Airflow
- Akamai
- Apache
- API (custom)
- Arbor Peakflow SP Logs
- Arista NG Firewall
- Atlassian
- Auditd
- Auth0
- authentik
- AWS
- Amazon CloudFront
- Amazon DynamoDB
- Amazon EBS
- Amazon EC2
- Amazon ECS
- Amazon EMR
- AWS API Gateway
- Amazon GuardDuty
- AWS Health
- Amazon Kinesis Data Firehose
- Amazon Kinesis Data Stream
- Amazon Managed Streaming for Apache Kafka (MSK)
- Amazon NAT Gateway
- Amazon RDS
- Amazon Redshift
- Amazon S3
- Amazon S3 Storage Lens
- Amazon Security Lake
- Amazon SNS
- Amazon SQS
- Amazon VPC
- Amazon VPN
- AWS Bedrock
- AWS Billing
- AWS CloudTrail
- AWS CloudWatch
- AWS ELB
- AWS Fargate
- AWS Inspector
- AWS Lambda
- AWS Logs (custom)
- AWS Network Firewall
- AWS Route 53
- AWS Security Hub
- AWS Transit Gateway
- AWS Usage
- AWS WAF
- Azure
- Activity logs
- App Service
- Application Gateway
- Application Insights metrics
- Application Insights metrics overview
- Application State Insights metrics
- Azure logs (v2 preview)
- Azure OpenAI
- Billing metrics
- Container instance metrics
- Container registry metrics
- Container service metrics
- Custom Azure Logs
- Custom Blob Storage Input
- Database Account metrics
- Event Hub input
- Firewall logs
- Frontdoor
- Functions
- Microsoft Entra ID
- Monitor metrics
- Network Watcher VNet
- Network Watcher NSG
- Platform logs
- Resource metrics
- Spring Cloud logs
- Storage Account metrics
- Virtual machines metrics
- Virtual machines scaleset metrics
- Barracuda
- BitDefender
- Bitwarden
- blacklens.io
- Blue Coat Director Logs
- BBOT (Bighuge BLS OSINT Tool)
- Box Events
- Bravura Monitor
- Broadcom ProxySG
- Canva
- Cassandra
- CEL Custom API
- Ceph
- Check Point
- Cilium Tetragon
- CISA Known Exploited Vulnerabilities
- Cisco
- Cisco Meraki Metrics
- Citrix
- Claroty CTD
- Cloudflare
- Cloud Asset Inventory
- CockroachDB Metrics
- Common Event Format (CEF)
- Containerd
- CoreDNS
- Corelight
- Couchbase
- CouchDB
- Cribl
- CrowdStrike
- Cyberark
- Cybereason
- CylanceProtect Logs
- Custom Websocket logs
- Darktrace
- Data Exfiltration Detection
- DGA
- Digital Guardian
- Docker
- Elastic APM
- Elastic Fleet Server
- Elastic Security
- Elastic Stack monitoring
- Elasticsearch Service Billing
- Envoy Proxy
- ESET PROTECT
- ESET Threat Intelligence
- etcd
- Falco
- F5
- File Integrity Monitoring
- FireEye Network Security
- First EPSS
- Forcepoint Web Security
- ForgeRock
- Fortinet
- Gigamon
- GitHub
- GitLab
- Golang
- Google Cloud
- Custom GCS Input
- GCP
- GCP Audit logs
- GCP Billing metrics
- GCP Cloud Run metrics
- GCP CloudSQL metrics
- GCP Compute metrics
- GCP Dataproc metrics
- GCP DNS logs
- GCP Firestore metrics
- GCP Firewall logs
- GCP GKE metrics
- GCP Load Balancing metrics
- GCP Metrics Input
- GCP PubSub logs (custom)
- GCP PubSub metrics
- GCP Redis metrics
- GCP Security Command Center
- GCP Storage metrics
- GCP VPC Flow logs
- GCP Vertex AI
- GoFlow2 logs
- Hadoop
- HAProxy
- Hashicorp Vault
- HTTP Endpoint logs (custom)
- IBM MQ
- IIS
- Imperva
- InfluxDb
- Infoblox
- Iptables
- Istio
- Jamf Compliance Reporter
- Jamf Pro
- Jamf Protect
- Jolokia Input
- Journald logs (custom)
- JumpCloud
- Kafka
- Keycloak
- Kubernetes
- LastPass
- Lateral Movement Detection
- Linux Metrics
- Living off the Land Attack Detection
- Logs (custom)
- Lumos
- Lyve Cloud
- Mattermost
- Memcached
- Menlo Security
- Microsoft
- Microsoft 365
- Microsoft Defender for Cloud
- Microsoft Defender for Endpoint
- Microsoft DHCP
- Microsoft DNS Server
- Microsoft Entra ID Entity Analytics
- Microsoft Exchange Online Message Trace
- Microsoft Exchange Server
- Microsoft Graph Activity Logs
- Microsoft M365 Defender
- Microsoft Office 365 Metrics Integration
- Microsoft Sentinel
- Microsoft SQL Server
- Mimecast
- ModSecurity Audit
- MongoDB
- MongoDB Atlas
- MySQL
- Nagios XI
- NATS
- NetFlow Records
- Netskope
- Network Beaconing Identification
- Network Packet Capture
- Nginx
- Okta
- Oracle
- OpenCanary
- Osquery
- Palo Alto
- pfSense
- PHP-FPM
- PingOne
- PingFederate
- Pleasant Password Server
- PostgreSQL
- Prometheus
- Proofpoint TAP
- Proofpoint On Demand
- Pulse Connect Secure
- Qualys VMDR
- QNAP NAS
- RabbitMQ Logs
- Radware DefensePro Logs
- Rapid7
- Redis
- Rubrik RSC Metrics Integration
- Salesforce
- SentinelOne
- ServiceNow
- Slack Logs
- Snort
- Snyk
- SonicWall Firewall
- Sophos
- Spring Boot
- SpyCloud Enterprise Protection
- SQL Input
- Squid Logs
- SRX
- STAN
- Statsd Input
- Sublime Security
- Suricata
- StormShield SNS
- Symantec
- Symantec Endpoint Security
- Sysmon for Linux
- Sysdig
- Syslog Router Integration
- System
- System Audit
- Tanium
- TCP Logs (custom)
- Teleport
- Tenable
- Threat intelligence
- ThreatConnect
- Threat Map
- Thycotic Secret Server
- Tines
- Traefik
- Trellix
- Trend Micro
- TYCHON Agentless
- UDP Logs (custom)
- Universal Profiling
- Vectra Detect
- VMware
- WatchGuard Firebox
- WebSphere Application Server
- Windows
- Wiz
- Zeek
- ZeroFox
- Zero Networks
- ZooKeeper Metrics
- Zoom
- Zscaler
Containerd integration
editContainerd integration
editVersion |
0.4.0 [beta] This functionality is in beta and is subject to change. The design and code is less mature than official GA features and is being provided as-is with no warranties. Beta features are not subject to the support SLA of official GA features. (View all) |
Compatible Kibana version(s) |
8.13.0 or higher |
Supported Serverless project types |
Security |
Subscription level |
Basic |
This integration is used to collect metrics from containerd runtime.
It collects cpu, memory and blkio statistics about running containers controlled by containerd.
The current datasets are: cpu
, blkio
and memory
and are enabled by default.
Compatibility
editThe Containerd integration is currently tested with Containerd version v1.5.2.
Prerequisites
editContainerd
daemon has to be configured to provide metrics before enabling containerd integration.
In the configuration file located in /etc/containerd/config.toml
metrics endpoint needs to
be set and containerd daemon needs to be restarted.
[metrics] address = "127.0.0.1:1338"
Integration-specific configuration notes
editFor cpu data stream, if Calculate cpu usage percentage
setting is set to true
, cpu usage percentages will be calculated
and, more specifically, fields containerd.cpu.usage.total.pct
, containerd.cpu.usage.kernel.pct
, containerd.cpu.usage.user.pct
.
Default value is true
.
For memory data stream, if Calculate memory usage percentage
setting is set to true
, memory usage percentages will be calculated,
and, more specifically, fields containerd.memory.usage.pct
and containerd.memory.workingset.pct
.
Default value is true
.
Metrics
editcpu
editThis is the cpu
dataset of the Containerd
package. It collects cpu related metrics
from containerd’s metrics APIs.
Exported fields
Field | Description | Type | Unit | Metric Type |
---|---|---|---|---|
@timestamp |
Event timestamp. |
date |
||
agent.id |
Unique identifier of this agent (if one exists). Example: For Beats this would be beat.id. |
keyword |
||
cloud.account.id |
The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier. |
keyword |
||
cloud.availability_zone |
Availability zone in which this host, resource, or service is located. |
keyword |
||
cloud.instance.id |
Instance ID of the host machine. |
keyword |
||
cloud.provider |
Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean. |
keyword |
||
cloud.region |
Region in which this host, resource, or service is located. |
keyword |
||
container.cpu.usage |
Total CPU usage normalized by the number of CPU cores. |
scaled_float |
percent |
gauge |
container.id |
Unique container id. |
keyword |
||
containerd.cpu.system.total |
Total user and system CPU time spent in seconds. |
double |
s |
gauge |
containerd.cpu.usage.cpu.*.ns |
CPU usage nanoseconds in this cpu. |
object |
||
containerd.cpu.usage.kernel.ns |
CPU Kernel usage nanoseconds |
double |
nanos |
gauge |
containerd.cpu.usage.kernel.pct |
Percentage of time in kernel space normalized by the number of CPU cores. |
scaled_float |
percent |
gauge |
containerd.cpu.usage.total.ns |
CPU total usage nanoseconds |
double |
nanos |
gauge |
containerd.cpu.usage.total.pct |
Percentage of total CPU time normalized by the number of CPU cores |
scaled_float |
percent |
gauge |
containerd.cpu.usage.user.ns |
CPU User usage nanoseconds |
double |
nanos |
gauge |
containerd.cpu.usage.user.pct |
Percentage of time in user space normalized by the number of CPU cores. |
scaled_float |
percent |
gauge |
containerd.namespace |
Containerd namespace |
keyword |
||
data_stream.dataset |
Data stream dataset. |
constant_keyword |
||
data_stream.namespace |
Data stream namespace. |
constant_keyword |
||
data_stream.type |
Data stream type. |
constant_keyword |
||
ecs.version |
ECS version this event conforms to. |
keyword |
||
host.name |
Name of the host. It can contain what |
keyword |
||
service.address |
Address where data about this service was collected from. This should be a URI, network address (ipv4:port or [ipv6]:port) or a resource path (sockets). |
keyword |
||
service.type |
The type of the service data is collected from. The type can be used to group and correlate logs and metrics from one service type. Example: If logs or metrics are collected from Elasticsearch, |
keyword |
Example
An example event for cpu
looks as following:
{ "container": { "id": "c9b16941acc7ec09a2b5e677d2863dd5c235b95304e7e8432a403f28e4f3e939" }, "agent": { "name": "kind-control-plane", "id": "e76bdfd9-382f-4f81-be2d-7031ea2f3e0b", "ephemeral_id": "09df445e-765a-4333-bb12-ab043d85149a", "type": "metricbeat", "version": "8.1.0" }, "elastic_agent": { "id": "e76bdfd9-382f-4f81-be2d-7031ea2f3e0b", "version": "8.1.0", "snapshot": false }, "containerd": { "namespace": "k8s.io", "cpu": { "usage": { "total": { "pct": 0.004503286370848513, "ns": 497538744548 }, "kernel": { "pct": 0.0006661590189871613, "ns": 91920000000 }, "user": { "pct": 0.0022482866890816693, "ns": 250140000000 } } } }, "@timestamp": "2022-01-17T10:56:05.804Z", "ecs": { "version": "8.0.0" }, "service": { "address": "http://localhost:1338/v1/metrics", "type": "containerd" }, "data_stream": { "namespace": "default", "type": "metrics", "dataset": "containerd.cpu" }, "host": { "hostname": "kind-control-plane", "os": { "kernel": "5.10.47-linuxkit", "codename": "Core", "name": "CentOS Linux", "type": "linux", "family": "redhat", "version": "7 (Core)", "platform": "centos" }, "containerized": true, "ip": [ "10.244.0.1", "10.244.0.1", "10.244.0.1", "192.168.0.2", "172.18.0.2", "fc00:f853:ccd:e793::2", "fe80::42:acff:fe12:2" ], "name": "kind-control-plane", "id": "8572202eb1aa48f1a380b1d48c9ff9c3", "mac": [ "a2:9c:01:82:64:e2", "d6:4a:2e:9f:3a:19", "4e:3b:ac:65:8e:2e", "02:42:c0:a8:00:02", "02:42:ac:12:00:02" ], "architecture": "x86_64" }, "metricset": { "period": 10000, "name": "cpu" }, "event": { "duration": 38067862, "agent_id_status": "verified", "ingested": "2022-01-17T10:56:06Z", "module": "containerd", "dataset": "containerd.cpu" } }
memory
editThis is the memory
dataset of the Containerd
package. It collects memory related metrics
from containerd’s metrics APIs.
Exported fields
Field | Description | Type | Unit | Metric Type |
---|---|---|---|---|
@timestamp |
Event timestamp. |
date |
||
agent.id |
Unique identifier of this agent (if one exists). Example: For Beats this would be beat.id. |
keyword |
||
cloud.account.id |
The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier. |
keyword |
||
cloud.availability_zone |
Availability zone in which this host, resource, or service is located. |
keyword |
||
cloud.instance.id |
Instance ID of the host machine. |
keyword |
||
cloud.provider |
Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean. |
keyword |
||
cloud.region |
Region in which this host, resource, or service is located. |
keyword |
||
container.id |
Unique container id. |
keyword |
||
container.memory.usage |
Memory usage percentage. |
scaled_float |
percent |
gauge |
containerd.memory.activeFiles |
Total active file bytes. |
long |
byte |
gauge |
containerd.memory.cache |
Total cache bytes. |
long |
byte |
gauge |
containerd.memory.inactiveFiles |
Total inactive file bytes. |
long |
byte |
gauge |
containerd.memory.kernel.fail.count |
Kernel fail counter. |
scaled_float |
counter |
|
containerd.memory.kernel.limit |
Kernel memory limit. |
long |
byte |
gauge |
containerd.memory.kernel.max |
Kernel max memory usage. |
long |
byte |
gauge |
containerd.memory.kernel.total |
Kernel total memory usage. |
long |
byte |
gauge |
containerd.memory.rss |
Total memory resident set size. |
long |
byte |
gauge |
containerd.memory.swap.fail.count |
Swap fail counter. |
scaled_float |
counter |
|
containerd.memory.swap.limit |
Swap memory limit. |
long |
byte |
gauge |
containerd.memory.swap.max |
Swap max memory usage. |
long |
byte |
gauge |
containerd.memory.swap.total |
Swap total memory usage. |
long |
byte |
gauge |
containerd.memory.usage.fail.count |
Fail counter. |
scaled_float |
counter |
|
containerd.memory.usage.limit |
Memory usage limit. |
long |
byte |
gauge |
containerd.memory.usage.max |
Max memory usage. |
long |
byte |
gauge |
containerd.memory.usage.pct |
Total allocated memory percentage. |
scaled_float |
percent |
gauge |
containerd.memory.usage.total |
Total memory usage. |
long |
byte |
gauge |
containerd.memory.workingset.pct |
Memory working set percentage. |
scaled_float |
percent |
gauge |
containerd.namespace |
Containerd namespace |
keyword |
||
data_stream.dataset |
Data stream dataset. |
constant_keyword |
||
data_stream.namespace |
Data stream namespace. |
constant_keyword |
||
data_stream.type |
Data stream type. |
constant_keyword |
||
ecs.version |
ECS version this event conforms to. |
keyword |
||
host.name |
Name of the host. It can contain what |
keyword |
||
service.address |
Address where data about this service was collected from. This should be a URI, network address (ipv4:port or [ipv6]:port) or a resource path (sockets). |
keyword |
||
service.type |
The type of the service data is collected from. The type can be used to group and correlate logs and metrics from one service type. Example: If logs or metrics are collected from Elasticsearch, |
keyword |
Example
An example event for memory
looks as following:
{ "container": { "id": "9c46c226469ff14544c97e003f403a457eda5398b9de757179e7d704ac1c8f72" }, "agent": { "name": "kind-control-plane", "id": "e76bdfd9-382f-4f81-be2d-7031ea2f3e0b", "ephemeral_id": "09df445e-765a-4333-bb12-ab043d85149a", "type": "metricbeat", "version": "8.1.0" }, "elastic_agent": { "id": "e76bdfd9-382f-4f81-be2d-7031ea2f3e0b", "version": "8.1.0", "snapshot": false }, "containerd": { "memory": { "activeFiles": 2568192, "cache": 41631744, "rss": 11759616, "swap": { "fail": { "count": 0 }, "total": 54820864, "max": 56819712, "limit": 9223372036854772000 }, "kernel": { "fail": { "count": 0 }, "total": 1294336, "max": 1773568, "limit": 9223372036854772000 }, "usage": { "pct": 5.9436899846332405e-12, "fail": { "count": 0 }, "total": 54820864, "max": 56819712, "limit": 9223372036854772000 }, "workingset": { "pct": 1.7084111902931917e-12 }, "inactiveFiles": 39063552 }, "namespace": "k8s.io" }, "@timestamp": "2022-01-17T10:53:55.882Z", "ecs": { "version": "8.0.0" }, "service": { "address": "http://localhost:1338/v1/metrics", "type": "containerd" }, "data_stream": { "namespace": "default", "type": "metrics", "dataset": "containerd.memory" }, "host": { "hostname": "kind-control-plane", "os": { "kernel": "5.10.47-linuxkit", "codename": "Core", "name": "CentOS Linux", "family": "redhat", "type": "linux", "version": "7 (Core)", "platform": "centos" }, "containerized": true, "ip": [ "10.244.0.1", "10.244.0.1", "10.244.0.1", "192.168.0.2", "172.18.0.2", "fc00:f853:ccd:e793::2", "fe80::42:acff:fe12:2" ], "name": "kind-control-plane", "id": "8572202eb1aa48f1a380b1d48c9ff9c3", "mac": [ "a2:9c:01:82:64:e2", "d6:4a:2e:9f:3a:19", "4e:3b:ac:65:8e:2e", "02:42:c0:a8:00:02", "02:42:ac:12:00:02" ], "architecture": "x86_64" }, "metricset": { "period": 10000, "name": "memory" }, "event": { "duration": 916030, "agent_id_status": "verified", "ingested": "2022-01-17T10:53:56Z", "module": "containerd", "dataset": "containerd.memory" } }
blkio
editThis is the blkio
dataset of the Containerd
package. It collects blkio related metrics
from containerd’s metrics APIs.
Exported fields
Field | Description | Type | Unit | Metric Type |
---|---|---|---|---|
@timestamp |
Event timestamp. |
date |
||
agent.id |
Unique identifier of this agent (if one exists). Example: For Beats this would be beat.id. |
keyword |
||
cloud.account.id |
The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier. |
keyword |
||
cloud.availability_zone |
Availability zone in which this host, resource, or service is located. |
keyword |
||
cloud.instance.id |
Instance ID of the host machine. |
keyword |
||
cloud.provider |
Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean. |
keyword |
||
cloud.region |
Region in which this host, resource, or service is located. |
keyword |
||
container.disk.read.bytes |
Bytes read during the life of the container |
long |
counter |
|
container.disk.write.bytes |
Bytes written during the life of the container |
long |
byte |
counter |
container.id |
Unique container id. |
keyword |
||
containerd.blkio.device |
Name of block device |
keyword |
||
containerd.blkio.read.bytes |
Bytes read during the life of the container |
long |
byte |
gauge |
containerd.blkio.read.ops |
Number of reads during the life of the container |
long |
gauge |
|
containerd.blkio.summary.bytes |
Bytes read and written during the life of the container |
long |
byte |
gauge |
containerd.blkio.summary.ops |
Number of I/O operations during the life of the container |
long |
gauge |
|
containerd.blkio.write.bytes |
Bytes written during the life of the container |
long |
byte |
gauge |
containerd.blkio.write.ops |
Number of writes during the life of the container |
long |
gauge |
|
containerd.namespace |
Containerd namespace |
keyword |
||
data_stream.dataset |
Data stream dataset. |
constant_keyword |
||
data_stream.namespace |
Data stream namespace. |
constant_keyword |
||
data_stream.type |
Data stream type. |
constant_keyword |
||
ecs.version |
ECS version this event conforms to. |
keyword |
||
host.name |
Name of the host. It can contain what |
keyword |
||
service.address |
Address where data about this service was collected from. This should be a URI, network address (ipv4:port or [ipv6]:port) or a resource path (sockets). |
keyword |
||
service.type |
The type of the service data is collected from. The type can be used to group and correlate logs and metrics from one service type. Example: If logs or metrics are collected from Elasticsearch, |
keyword |
Example
An example event for blkio
looks as following:
{ "container": { "id": "42e49b56969eba4a3657745fba5bf7375e66f1a4b7a55a873f365e86fdc0de9b" }, "agent": { "name": "kind-control-plane", "id": "e76bdfd9-382f-4f81-be2d-7031ea2f3e0b", "ephemeral_id": "09df445e-765a-4333-bb12-ab043d85149a", "type": "metricbeat", "version": "8.1.0" }, "elastic_agent": { "id": "e76bdfd9-382f-4f81-be2d-7031ea2f3e0b", "version": "8.1.0", "snapshot": false }, "containerd": { "blkio": { "summary": { "ops": 37, "bytes": 2838528 }, "read": { "ops": 37, "bytes": 2838528 }, "device": "/dev/vda", "write": { "ops": 0, "bytes": 0 } }, "namespace": "k8s.io" }, "@timestamp": "2022-01-17T10:05:17.991Z", "ecs": { "version": "8.0.0" }, "service": { "address": "http://localhost:1338/v1/metrics", "type": "containerd" }, "data_stream": { "namespace": "default", "type": "metrics", "dataset": "containerd.blkio" }, "host": { "hostname": "kind-control-plane", "os": { "kernel": "5.10.47-linuxkit", "codename": "Core", "name": "CentOS Linux", "family": "redhat", "type": "linux", "version": "7 (Core)", "platform": "centos" }, "ip": [ "10.244.0.1", "10.244.0.1", "10.244.0.1", "192.168.0.2", "172.18.0.2", "fc00:f853:ccd:e793::2", "fe80::42:acff:fe12:2" ], "containerized": true, "name": "kind-control-plane", "id": "8572202eb1aa48f1a380b1d48c9ff9c3", "mac": [ "a2:9c:01:82:64:e2", "d6:4a:2e:9f:3a:19", "4e:3b:ac:65:8e:2e", "02:42:c0:a8:00:02", "02:42:ac:12:00:02" ], "architecture": "x86_64" }, "metricset": { "period": 10000, "name": "blkio" }, "event": { "duration": 36666537, "agent_id_status": "verified", "ingested": "2022-01-17T10:05:18Z", "module": "containerd", "dataset": "containerd.blkio" } }
Changelog
editChangelog
Version | Details | Kibana version(s) |
---|---|---|
0.4.0 |
Enhancement (View pull request) |
— |
0.3.0 |
Enhancement (View pull request) |
— |
0.2.6 |
Enhancement (View pull request) |
— |
0.2.5 |
Enhancement (View pull request) |
— |
0.2.4 |
Enhancement (View pull request) |
— |
0.2.3 |
Enhancement (View pull request) |
— |
0.2.2 |
Enhancement (View pull request) |
— |
0.2.1 |
Enhancement (View pull request) |
— |
0.2.0 |
Enhancement (View pull request) |
— |
0.1.0 |
Enhancement (View pull request) |
— |
On this page