Cassandra | Elastic docs

What is an Elastic integration?

This integration is powered by Elastic Agent. Elastic Agent is a single, unified way to add monitoring for logs, metrics, and other types of data to a host. It can also protect hosts from security threats, query data from operating systems, forward data from remote services or hardware, and more. Refer to our documentation for a detailed comparison between Beats and Elastic Agent.

Prefer to use Beats for this use case? See Filebeat modules for logs or Metricbeat modules for metrics.

Get started with integrations

See the integrations quick start guides to get started:

Overview

This integration periodically fetches metrics from Cassandra using jolokia agent. It can parse System logs.

Logs

Cassandra system logs from cassandra.log files.

An example event for log looks as following:

{
    "@timestamp": "2021-10-05T07:40:26.558Z",
    "agent": {
        "ephemeral_id": "37f2fc52-9bf4-4fe3-a5c6-ac3a89d10c87",
        "hostname": "docker-fleet-agent",
        "id": "55e3ef5d-1870-4291-8cff-32abd8b8b84b",
        "name": "docker-fleet-agent",
        "type": "filebeat",
        "version": "7.15.0"
    },
    "cassandra": {
        "log": {
            "meta": ""
        }
    },
    "data_stream": {
        "dataset": "cassandra.log",
        "namespace": "ep",
        "type": "logs"
    },
    "ecs": {
        "version": "1.12.0"
    },
    "elastic_agent": {
        "id": "55e3ef5d-1870-4291-8cff-32abd8b8b84b",
        "snapshot": true,
        "version": "7.15.0"
    },
    "event": {
        "agent_id_status": "verified",
        "category": [
            "database"
        ],
        "dataset": "cassandra.log",
        "ingested": "2021-10-05T07:40:53Z",
        "kind": "event",
        "original": "INFO  [main] 2021-10-05 07:40:26,558 YamlConfigurationLoader.java:92 - Configuration location: file:/etc/cassandra/cassandra.yaml",
        "type": "info"
    },
    "host": {
        "architecture": "x86_64",
        "containerized": true,
        "hostname": "docker-fleet-agent",
        "id": "6505f7ca36739e7eb909bdb52bf3ec18",
        "ip": [
            "172.27.0.10"
        ],
        "mac": [
            "02:42:ac:1b:00:0a"
        ],
        "name": "docker-fleet-agent",
        "os": {
            "codename": "Core",
            "family": "redhat",
            "kernel": "3.10.0-1062.el7.x86_64",
            "name": "CentOS Linux",
            "platform": "centos",
            "type": "linux",
            "version": "7 (Core)"
        }
    },
    "input": {
        "type": "log"
    },
    "log": {
        "file": {
            "path": "/tmp/service_logs/cassandra/system.log"
        },
        "level": "INFO",
        "offset": 0,
        "origin": {
            "file": {
                "line": "92",
                "name": "YamlConfigurationLoader.java"
            }
        }
    },
    "message": "Configuration location: file:/etc/cassandra/cassandra.yaml",
    "process": {
        "thread": {
            "name": "main"
        }
    }
}

Exported fields

Field	Description	Type
@timestamp	Event timestamp.	date
cassandra.log.meta	Log meta infos like java stack_trace	keyword
data_stream.dataset	Data stream dataset.	constant_keyword
data_stream.namespace	Data stream namespace.	constant_keyword
data_stream.type	Data stream type.	constant_keyword
ecs.version	ECS version this event conforms to. `ecs.version` is a required field and must exist in all events. When querying across multiple indices -- which may conform to slightly different ECS versions -- this field lets integrations adjust to the schema version of the events.	keyword
error.message	Error message.	match_only_text
event.category	This is one of four ECS Categorization Fields, and indicates the second level in the ECS category hierarchy. `event.category` represents the "big buckets" of ECS categories. For example, filtering on `event.category:process` yields all events relating to process activity. This field is closely related to `event.type`, which is used as a subcategory. This field is an array. This will allow proper categorization of some events that fall in multiple categories.	keyword
event.ingested	Timestamp when an event arrived in the central data store. This is different from `@timestamp`, which is when the event originally occurred. It's also different from `event.created`, which is meant to capture the first time an agent saw the event. In normal conditions, assuming no tampering, the timestamps should chronologically look like this: `@timestamp` < `event.created` < `event.ingested`.	date
event.kind	This is one of four ECS Categorization Fields, and indicates the highest level in the ECS category hierarchy. `event.kind` gives high-level information about what type of information the event contains, without being specific to the contents of the event. For example, values of this field distinguish alert events from metric events. The value of this field can be used to inform how these kinds of events should be handled. They may warrant different retention, different access control, it may also help understand whether the data coming in at a regular interval or not.	keyword
input.type	Input type	keyword
log.file.path	Full path to the log file this event came from, including the file name. It should include the drive letter, when appropriate. If the event wasn't read from a log file, do not populate this field.	keyword
log.flags	Log flags	keyword
log.level	Original log level of the log event. If the source of the event provides a log level or textual severity, this is the one that goes in `log.level`. If your source doesn't specify one, you may put your event transport's severity here (e.g. Syslog severity). Some examples are `warn`, `err`, `i`, `informational`.	keyword
log.offset	Log offset	long
log.origin.file.line	The line number of the file containing the source code which originated the log event.	integer
log.origin.file.name	The name of the file containing the source code which originated the log event. Note that this field is not meant to capture the log file. The correct field to capture the log file is `log.file.path`.	keyword
message	For log events the message field contains the log message, optimized for viewing in a log viewer. For structured logs without an original message field, other fields can be concatenated to form a human-readable summary of the event. If multiple messages exist, they can be combined into one message.	match_only_text
process.thread.name	Thread name.	keyword
tags	List of keywords used to tag each event.	keyword

Metrics

Cassandra metrics using jolokia agent installed on cassandra.

An example event for metrics looks as following:

{
    "@timestamp": "2021-10-05T07:38:10.135Z",
    "agent": {
        "ephemeral_id": "d0cc6862-f7cf-4d0c-8ab1-f5bf6f859ed8",
        "hostname": "docker-fleet-agent",
        "id": "55e3ef5d-1870-4291-8cff-32abd8b8b84b",
        "name": "docker-fleet-agent",
        "type": "metricbeat",
        "version": "7.15.0"
    },
    "cassandra": {
        "metrics": {
            "cache": {
                "key_cache": {
                    "capacity": 99614720,
                    "one_minute_hit_rate": 0.704736648238381,
                    "requests": {
                        "one_minute_rate": 8.440473533066658
                    }
                },
                "row_cache": {
                    "capacity": 0,
                    "requests": {
                        "one_minute_rate": 0
                    }
                }
            },
            "client": {
                "connected_native_clients": 0
            },
            "client_request": {
                "casread": {
                    "one_minute_rate": 0
                },
                "caswrite": {
                    "one_minute_rate": 0
                },
                "range_slice": {
                    "one_minute_rate": 0.014712537947741825,
                    "total_latency": 45132
                },
                "read": {
                    "count": 2,
                    "one_minute_rate": 0.02942507589548365,
                    "timeouts": 0,
                    "total_latency": 190512,
                    "unavailables": 0
                },
                "write": {
                    "count": 1,
                    "one_minute_rate": 0.014712537947741825,
                    "timeouts": 0,
                    "timeoutsms": 0,
                    "total_latency": 147610,
                    "unavailables": 0,
                    "unavailablesms": 0
                }
            },
            "column_family": {
                "total_disk_space_used": 72600
            },
            "compaction": {
                "completed": 44,
                "pending": 0
            },
            "dropped_message": {
                "batch_remove": 0,
                "batch_store": 0,
                "counter_mutation": 0,
                "hint": 0,
                "mutation": 0,
                "paged_range": 0,
                "range_slice": 0,
                "read": 0,
                "read_repair": 0,
                "request_response": 0,
                "trace": 0
            },
            "gc": {
                "concurrent_mark_sweep": {
                    "collection_count": 1,
                    "collection_time": 139
                },
                "par_new": {
                    "collection_count": 2,
                    "collection_time": 188
                }
            },
            "memory": {
                "heap_usage": {
                    "committed": 2009071616,
                    "init": 2051014656,
                    "max": 2009071616,
                    "used": 210515520
                },
                "other_usage": {
                    "committed": 62955520,
                    "init": 2555904,
                    "max": -1,
                    "used": 61118512
                }
            },
            "storage": {
                "exceptions": 0,
                "load": 72600,
                "total_hint_in_progress": 0,
                "total_hints": 0
            },
            "system": {
                "cluster": "Test Cluster",
                "data_center": "datacenter1",
                "joining_nodes": [],
                "leaving_nodes": [],
                "live_nodes": [
                    "172.28.0.2"
                ],
                "moving_nodes": [],
                "rack": "rack1",
                "unreachable_nodes": [],
                "version": "3.11.11"
            },
            "table": {
                "all_memtables_heap_size": 5148,
                "all_memtables_off_heap_size": 0,
                "live_disk_space_used": 72600,
                "live_ss_table_count": 11
            },
            "task": {
                "complete": 56,
                "pending": 0,
                "total_commitlog_size": 67108864
            },
            "thread_pools": {
                "counter_mutation_stage": {
                    "request": {
                        "active": 0,
                        "pending": 0
                    }
                },
                "mutation_stage": {
                    "request": {
                        "active": 0,
                        "pending": 0
                    }
                },
                "read_repair_stage": {
                    "request": {
                        "active": 0,
                        "pending": 0
                    }
                },
                "read_stage": {
                    "request": {
                        "active": 0,
                        "pending": 0
                    }
                },
                "request_response_stage": {
                    "request": {
                        "active": 0,
                        "pending": 0
                    }
                }
            }
        }
    },
    "data_stream": {
        "dataset": "cassandra.metrics",
        "namespace": "ep",
        "type": "metrics"
    },
    "ecs": {
        "version": "1.12.0"
    },
    "elastic_agent": {
        "id": "55e3ef5d-1870-4291-8cff-32abd8b8b84b",
        "snapshot": true,
        "version": "7.15.0"
    },
    "event": {
        "agent_id_status": "verified",
        "dataset": "cassandra.metrics",
        "duration": 321822352,
        "ingested": "2021-10-05T07:38:13Z",
        "module": "jolokia"
    },
    "host": {
        "architecture": "x86_64",
        "containerized": true,
        "hostname": "docker-fleet-agent",
        "id": "6505f7ca36739e7eb909bdb52bf3ec18",
        "ip": [
            "172.27.0.10"
        ],
        "mac": [
            "02:42:ac:1b:00:0a"
        ],
        "name": "docker-fleet-agent",
        "os": {
            "codename": "Core",
            "family": "redhat",
            "kernel": "3.10.0-1062.el7.x86_64",
            "name": "CentOS Linux",
            "platform": "centos",
            "type": "linux",
            "version": "7 (Core)"
        }
    },
    "metricset": {
        "name": "jmx",
        "period": 10000
    },
    "service": {
        "address": "http://elastic-package-service_cassandra_1:8778/jolokia/%3FignoreErrors=true\u0026canonicalNaming=false",
        "type": "jolokia"
    }
}

Exported fields

Field	Description	Type
@timestamp	Event timestamp.	date
cassandra.metrics.cache.key_cache.capacity		long
cassandra.metrics.cache.key_cache.one_minute_hit_rate		long
cassandra.metrics.cache.key_cache.requests.one_minute_rate		long
cassandra.metrics.cache.row_cache.capacity		long
cassandra.metrics.cache.row_cache.one_minute_hit_rate		long
cassandra.metrics.cache.row_cache.requests.one_minute_rate		long
cassandra.metrics.client.connected_native_clients		long
cassandra.metrics.client_request.casread.one_minute_rate		double
cassandra.metrics.client_request.caswrite.one_minute_rate		double
cassandra.metrics.client_request.range_slice.one_minute_rate		double
cassandra.metrics.client_request.range_slice.total_latency		double
cassandra.metrics.client_request.read.count		long
cassandra.metrics.client_request.read.one_minute_rate		double
cassandra.metrics.client_request.read.timeouts	Number of read timeouts encountered	double
cassandra.metrics.client_request.read.timeoutsms		double
cassandra.metrics.client_request.read.total_latency		double
cassandra.metrics.client_request.read.unavailables	Number of read unavailables encountered	double
cassandra.metrics.client_request.read.unavailablesms		double
cassandra.metrics.client_request.write.count		long
cassandra.metrics.client_request.write.one_minute_rate		double
cassandra.metrics.client_request.write.timeouts		double
cassandra.metrics.client_request.write.timeoutsms		double
cassandra.metrics.client_request.write.total_latency		double
cassandra.metrics.client_request.write.unavailables		double
cassandra.metrics.client_request.write.unavailablesms		double
cassandra.metrics.column_family.total_disk_space_used		long
cassandra.metrics.compaction.completed	compaction completed tasks	long
cassandra.metrics.compaction.pending	compaction pending tasks	long
cassandra.metrics.dropped_message.batch_remove		long
cassandra.metrics.dropped_message.batch_store		long
cassandra.metrics.dropped_message.counter_mutation		long
cassandra.metrics.dropped_message.hint		long
cassandra.metrics.dropped_message.mutation		long
cassandra.metrics.dropped_message.paged_range		long
cassandra.metrics.dropped_message.range_slice		long
cassandra.metrics.dropped_message.read		long
cassandra.metrics.dropped_message.read_repair		long
cassandra.metrics.dropped_message.request_response		long
cassandra.metrics.dropped_message.trace		long
cassandra.metrics.gc.concurrent_mark_sweep.collection_count	Total number of CMS collections that have occurred.	long
cassandra.metrics.gc.concurrent_mark_sweep.collection_time	Approximate accumulated CMS collection elapsed time in milliseconds.	long
cassandra.metrics.gc.mbean	Mbean that this event is related to	keyword
cassandra.metrics.gc.par_new.collection_count	Total number of ParNew collections that have occurred.	long
cassandra.metrics.gc.par_new.collection_time	Approximate accumulated ParNew collection elapsed time in milliseconds.	long
cassandra.metrics.memory.heap_usage.committed	Committed heap memory usage	long
cassandra.metrics.memory.heap_usage.init	Initial heap memory usage	long
cassandra.metrics.memory.heap_usage.max	Max heap memory usage	long
cassandra.metrics.memory.heap_usage.used	Used heap memory usage	long
cassandra.metrics.memory.mbean	Mbean that this event is related to	keyword
cassandra.metrics.memory.other_usage.committed	Committed non-heap memory usage	long
cassandra.metrics.memory.other_usage.init	Initial non-heap memory usage	long
cassandra.metrics.memory.other_usage.max	Max non-heap memory usage	long
cassandra.metrics.memory.other_usage.used	Used non-heap memory usage	long
cassandra.metrics.storage.exceptions	The number of the total exceptions	long
cassandra.metrics.storage.load	Storage used for Cassandra data in bytes	long
cassandra.metrics.storage.mbean	Mbean that this event is related to	keyword
cassandra.metrics.storage.total_hint_in_progress	The number of the total hits in progress	long
cassandra.metrics.storage.total_hints	The number of the total hits	long
cassandra.metrics.system.cluster		keyword
cassandra.metrics.system.data_center		keyword
cassandra.metrics.system.joining_nodes		keyword
cassandra.metrics.system.leaving_nodes		keyword
cassandra.metrics.system.live_nodes		keyword
cassandra.metrics.system.moving_nodes		keyword
cassandra.metrics.system.rack		keyword
cassandra.metrics.system.unreachable_nodes		keyword
cassandra.metrics.system.version		keyword
cassandra.metrics.table.all_memtables_heap_size		long
cassandra.metrics.table.all_memtables_off_heap_size		long
cassandra.metrics.table.live_disk_space_used		long
cassandra.metrics.table.live_ss_table_count		long
cassandra.metrics.task.complete	completed tasks	long
cassandra.metrics.task.pending	pending tasks	long
cassandra.metrics.task.total_commitlog_size	total commitlog size of tasks	long
cassandra.metrics.thread_pools.counter_mutation_stage.request.active		long
cassandra.metrics.thread_pools.counter_mutation_stage.request.pending		long
cassandra.metrics.thread_pools.mutation_stage.request.active		long
cassandra.metrics.thread_pools.mutation_stage.request.pending		long
cassandra.metrics.thread_pools.read_repair_stage.request.active		long
cassandra.metrics.thread_pools.read_repair_stage.request.pending		long
cassandra.metrics.thread_pools.read_stage.request.active		long
cassandra.metrics.thread_pools.read_stage.request.pending		long
cassandra.metrics.thread_pools.request_response_stage.request.active		long
cassandra.metrics.thread_pools.request_response_stage.request.pending		long
data_stream.dataset	Data stream dataset.	constant_keyword
data_stream.namespace	Data stream namespace.	constant_keyword
data_stream.type	Data stream type.	constant_keyword
destination.domain	Destination domain.	keyword
ecs.version	ECS version this event conforms to. `ecs.version` is a required field and must exist in all events. When querying across multiple indices -- which may conform to slightly different ECS versions -- this field lets integrations adjust to the schema version of the events.	keyword
error.message	Error message.	match_only_text
event.category	This is one of four ECS Categorization Fields, and indicates the second level in the ECS category hierarchy. `event.category` represents the "big buckets" of ECS categories. For example, filtering on `event.category:process` yields all events relating to process activity. This field is closely related to `event.type`, which is used as a subcategory. This field is an array. This will allow proper categorization of some events that fall in multiple categories.	keyword
event.created	event.created contains the date/time when the event was first read by an agent, or by your pipeline. This field is distinct from @timestamp in that @timestamp typically contain the time extracted from the original event. In most situations, these two timestamps will be slightly different. The difference can be used to calculate the delay between your source generating an event, and the time when your agent first processed it. This can be used to monitor your agent's or pipeline's ability to keep up with your event source. In case the two timestamps are identical, @timestamp should be used.	date
event.kind	This is one of four ECS Categorization Fields, and indicates the highest level in the ECS category hierarchy. `event.kind` gives high-level information about what type of information the event contains, without being specific to the contents of the event. For example, values of this field distinguish alert events from metric events. The value of this field can be used to inform how these kinds of events should be handled. They may warrant different retention, different access control, it may also help understand whether the data coming in at a regular interval or not.	keyword
event.outcome	This is one of four ECS Categorization Fields, and indicates the lowest level in the ECS category hierarchy. `event.outcome` simply denotes whether the event represents a success or a failure from the perspective of the entity that produced the event. Note that when a single transaction is described in multiple events, each event may populate different values of `event.outcome`, according to their perspective. Also note that in the case of a compound event (a single event that contains multiple logical events), this field should be populated with the value that best captures the overall success or failure from the perspective of the event producer. Further note that not all events will have an associated outcome. For example, this field is generally not populated for metric events, events with `event.type:info`, or any events for which an outcome does not make logical sense.	keyword
file.path	Full path to the file, including the file name. It should include the drive letter, when appropriate.	keyword
http.request.method	HTTP request method. Prior to ECS 1.6.0 the following guidance was provided: "The field value must be normalized to lowercase for querying." As of ECS 1.6.0, the guidance is deprecated because the original case of the method may be useful in anomaly detection. Original case will be mandated in ECS 2.0.0	keyword
http.request.referrer	Referrer for this HTTP request.	keyword
http.response.body.bytes	Size in bytes of the response body.	long
http.response.status_code	HTTP response status code.	long
http.version	HTTP version.	keyword
log.file.path	Full path to the log file this event came from, including the file name. It should include the drive letter, when appropriate. If the event wasn't read from a log file, do not populate this field.	keyword
log.level	Original log level of the log event. If the source of the event provides a log level or textual severity, this is the one that goes in `log.level`. If your source doesn't specify one, you may put your event transport's severity here (e.g. Syslog severity). Some examples are `warn`, `err`, `i`, `informational`.	keyword
message	For log events the message field contains the log message, optimized for viewing in a log viewer. For structured logs without an original message field, other fields can be concatenated to form a human-readable summary of the event. If multiple messages exist, they can be combined into one message.	match_only_text
process.pid	Process id.	long
process.thread.id	Thread ID.	long
service.address	Address where data about this service was collected from. This should be a URI, network address (ipv4:port or [ipv6]:port) or a resource path (sockets).	keyword
service.type	The type of the service data is collected from. The type can be used to group and correlate logs and metrics from one service type. Example: If logs or metrics are collected from Elasticsearch, `service.type` would be `elasticsearch`.	keyword
source.address	Some event source addresses are defined ambiguously. The event will sometimes list an IP, a domain or a unix socket. You should always store the raw address in the `.address` field. Then it should be duplicated to `.ip` or `.domain`, depending on which one it is.	keyword
source.as.number	Unique number allocated to the autonomous system. The autonomous system number (ASN) uniquely identifies each network on the Internet.	long
source.as.organization.name	Organization name.	keyword
source.domain	Source domain.	keyword
source.geo.city_name	City name.	keyword
source.geo.continent_name	Name of the continent.	keyword
source.geo.country_iso_code	Country ISO code.	keyword
source.geo.country_name	Country name.	keyword
source.geo.location	Longitude and latitude.	geo_point
source.geo.region_iso_code	Region ISO code.	keyword
source.geo.region_name	Region name.	keyword
source.ip	IP address of the source (IPv4 or IPv6).	ip
tags	List of keywords used to tag each event.	keyword
tls.cipher	String indicating the cipher used during the current connection.	keyword
tls.version	Numeric part of the version parsed from the original string.	keyword
tls.version_protocol	Normalized lowercase protocol name parsed from original string.	keyword
url.domain	Domain of the url, such as "www.elastic.co". In some cases a URL may refer to an IP and/or port directly, without a domain name. In this case, the IP address would go to the `domain` field. If the URL contains a literal IPv6 address enclosed by `[` and `]` (IETF RFC 2732), the `[` and `]` characters should also be captured in the `domain` field.	keyword
url.extension	The field contains the file extension from the original request url, excluding the leading dot. The file extension is only set if it exists, as not every url has a file extension. The leading period must not be included. For example, the value must be "png", not ".png". Note that when the file name has multiple extensions (example.tar.gz), only the last one should be captured ("gz", not "tar.gz").	keyword
url.original	Unmodified original url as seen in the event source. Note that in network monitoring, the observed URL may be a full URL, whereas in access logs, the URL is often just represented as a path. This field is meant to represent the URL as it was observed, complete or not.	wildcard
url.path	Path of the request, such as "/search".	wildcard
url.query	The query field describes the query string of the request, such as "q=elasticsearch". The `?` is excluded from the query string. If a URL contains no `?`, there is no query field. If there is a `?` but no query, the query field exists with an empty string. The `exists` query can be used to differentiate between the two cases.	keyword
user.name	Short name or login of the user.	keyword
user_agent.device.name	Name of the device.	keyword
user_agent.name	Name of the user agent.	keyword
user_agent.original	Unparsed user_agent string.	keyword
user_agent.os.full	Operating system name, including the version or code name.	keyword
user_agent.os.name	Operating system name, without the version.	keyword
user_agent.os.version	Operating system version as a raw string.	keyword
user_agent.version	Version of the user agent.	keyword

Changelog

Version	Details
1.1.0	Enhancement View pull request Release cassandra package for v8.0.0
1.0.0	Enhancement View pull request GA Release
0.0.1	Enhancement View pull request Initial draft of the package