The AWS CloudTrail integration allows you to monitor AWS CloudTrail.

Use the AWS CloudTrail integration to collect and parse logs related to account activity across your AWS infrastructure. Then visualize that data in Kibana, create alerts to notify you if something goes wrong, and reference logs when troubleshooting an issue.

For example, you could use the data from this integration to spot unusual activity in your AWS accounts—like excessive failed AWS console sign in attempts.

IMPORTANT: Extra AWS charges on AWS API requests will be generated by this integration. Please refer to the AWS integration for more details.

Data streams

The AWS CloudTrail integration collects one type of data: logs.

Logs help you keep a record of every event that CloudTrail receives. These logs are useful for many scenarios, including security and access audits. See more details in the Logs reference.

Requirements

You need Elasticsearch for storing and searching your data and Kibana for visualizing and managing it. You can use our hosted Elasticsearch Service on Elastic Cloud, which is recommended, or self-manage the Elastic Stack on your own hardware.

Before using any AWS integration you will need:

• AWS Credentials to connect with your AWS account.
• AWS Permissions to make sure the user you're using to connect has permission to share the relevant data.

For more details about these requirements, please take a look at the AWS integration documentation.

Setup

Use this integration if you only need to collect data from the AWS CloudTrail service.

If you want to collect data from two or more AWS services, consider using the AWS integration. When you configure the AWS integration, you can collect data from as many AWS services as you'd like.

For step-by-step instructions on how to set up an integration, see the Getting started guide.

Advanced options

CloudWatch

The CloudWatch logs input has several advanced options to fit specific use cases.

Latency

AWS CloudWatch Logs sometimes takes extra time to make the latest logs available to clients like the Agent.

The CloudWatch integration offers the latency setting to address this scenario. Latency translates the query's time range to consider the CloudWatch Logs latency. For example, a 5m latency means the integration will query CloudWatch for logs available 5 minutes ago.

Number of workers

If you are collecting log events from multiple log groups using log_group_name_prefix, you should review the value of the number_of_workers.

The number_of_workers setting defines the number of workers assigned to reading from log groups. Each log group matching the log_group_name_prefix requires a worker to keep log ingestion as close to real-time as possible. For example, if log_group_name_prefix matches five log groups, then number_of_workers should be set to 5. The default value is 1.

Logs reference

The cloudtrail data stream collects AWS CloudTrail logs. CloudTrail monitors events like user activity and API usage in AWS services. If a user creates a trail, it delivers those events as log files to a specific Amazon S3 bucket.

Note: Use the CloudTrail Digest Logs regex setting to define regex to match the path of the CloudTrail Digest S3 Objects you'd like to read. If blank, CloudTrail Digest logs will be skipped.

Exported fields

An example event for cloudtrail looks as following:

{
    "data_stream": {
        "namespace": "default",
        "type": "logs",
        "dataset": "aws.cloudtrail"
    },
    "source": {
        "address": "127.0.0.1",
        "ip": "127.0.0.1"
    },
    "tags": [
        "preserve_original_event"
    ],
    "cloud": {
        "region": "us-east-1",
        "account": {
            "id": "123456789012"
        }
    },
    "@timestamp": "2020-01-08T20:53:12.000Z",
    "ecs": {
        "version": "8.0.0"
    },
    "related": {
        "user": [
            "Alice",
            "Bob",
            "Robert"
        ]
    },
    "event": {
        "ingested": "2021-10-05T23:06:12.229540200Z",
        "original": "{\"eventVersion\":\"1.05\",\"userIdentity\":{\"type\":\"IAMUser\",\"principalId\":\"EX_PRINCIPAL_ID\",\"arn\":\"arn:aws:iam::123456789012:user/Alice\",\"accountId\":\"123456789012\",\"accessKeyId\":\"EXAMPLE_KEY_ID\",\"userName\":\"Alice\"},\"eventTime\":\"2020-01-08T20:53:12Z\",\"eventSource\":\"iam.amazonaws.com\",\"eventName\":\"UpdateUser\",\"awsRegion\":\"us-east-1\",\"sourceIPAddress\":\"127.0.0.1\",\"userAgent\":\"aws-cli/1.16.310 Python/3.8.1 Darwin/18.7.0 botocore/1.13.46\",\"requestParameters\":{\"userName\":\"Bob\",\"newUserName\":\"Robert\"},\"responseElements\":null,\"requestID\":\"3a6b3260-739d-465e-9406-bcEXAMPLE\",\"eventID\":\"9150d546-3564-4262-8e62-110EXAMPLE\",\"eventType\":\"AwsApiCall\",\"recipientAccountId\":\"123456789012\"}",
        "provider": "iam.amazonaws.com",
        "created": "2020-01-08T20:53:12.000Z",
        "kind": "event",
        "action": "UpdateUser",
        "id": "9150d546-3564-4262-8e62-110EXAMPLE",
        "type": [
            "user",
            "change"
        ],
        "category": [
            "iam"
        ],
        "outcome": "success"
    },
    "aws": {
        "cloudtrail": {
            "event_version": "1.05",
            "flattened": {
                "request_parameters": {
                    "userName": "Bob",
                    "newUserName": "Robert"
                }
            },
            "user_identity": {
                "access_key_id": "EXAMPLE_KEY_ID",
                "type": "IAMUser",
                "arn": "arn:aws:iam::123456789012:user/Alice"
            },
            "event_type": "AwsApiCall",
            "recipient_account_id": "123456789012",
            "request_parameters": "{newUserName=Robert, userName=Bob}"
        }
    },
    "user": {
        "name": "Alice",
        "changes": {
            "name": "Robert"
        },
        "id": "EX_PRINCIPAL_ID",
        "target": {
            "name": "Bob"
        }
    },
    "user_agent": {
        "name": "aws-cli",
        "original": "aws-cli/1.16.310 Python/3.8.1 Darwin/18.7.0 botocore/1.13.46",
        "device": {
            "name": "Spider"
        },
        "version": "1.16.310"
    }
}

Changelog