- Elastic integrations
- Integrations quick reference
- 1Password
- Abnormal Security
- ActiveMQ
- Active Directory Entity Analytics
- Airflow
- Akamai
- Apache
- API (custom)
- Arbor Peakflow SP Logs
- Arista NG Firewall
- Atlassian
- Auditd
- Auth0
- authentik
- AWS
- Amazon CloudFront
- Amazon DynamoDB
- Amazon EBS
- Amazon EC2
- Amazon ECS
- Amazon EMR
- AWS API Gateway
- Amazon GuardDuty
- AWS Health
- Amazon Kinesis Data Firehose
- Amazon Kinesis Data Stream
- Amazon Managed Streaming for Apache Kafka (MSK)
- Amazon NAT Gateway
- Amazon RDS
- Amazon Redshift
- Amazon S3
- Amazon S3 Storage Lens
- Amazon Security Lake
- Amazon SNS
- Amazon SQS
- Amazon VPC
- Amazon VPN
- AWS Bedrock
- AWS Billing
- AWS CloudTrail
- AWS CloudWatch
- AWS ELB
- AWS Fargate
- AWS Inspector
- AWS Lambda
- AWS Logs (custom)
- AWS Network Firewall
- AWS Route 53
- AWS Security Hub
- AWS Transit Gateway
- AWS Usage
- AWS WAF
- Azure
- Activity logs
- App Service
- Application Gateway
- Application Insights metrics
- Application Insights metrics overview
- Application State Insights metrics
- Azure logs (v2 preview)
- Azure OpenAI
- Billing metrics
- Container instance metrics
- Container registry metrics
- Container service metrics
- Custom Azure Logs
- Custom Blob Storage Input
- Database Account metrics
- Event Hub input
- Firewall logs
- Frontdoor
- Functions
- Microsoft Entra ID
- Monitor metrics
- Network Watcher VNet
- Network Watcher NSG
- Platform logs
- Resource metrics
- Spring Cloud logs
- Storage Account metrics
- Virtual machines metrics
- Virtual machines scaleset metrics
- Barracuda
- BitDefender
- Bitwarden
- blacklens.io
- Blue Coat Director Logs
- BBOT (Bighuge BLS OSINT Tool)
- Box Events
- Bravura Monitor
- Broadcom ProxySG
- Canva
- Cassandra
- CEL Custom API
- Ceph
- Check Point
- Cilium Tetragon
- CISA Known Exploited Vulnerabilities
- Cisco
- Cisco Meraki Metrics
- Citrix
- Claroty CTD
- Cloudflare
- Cloud Asset Inventory
- CockroachDB Metrics
- Common Event Format (CEF)
- Containerd
- CoreDNS
- Corelight
- Couchbase
- CouchDB
- Cribl
- CrowdStrike
- Cyberark
- Cybereason
- CylanceProtect Logs
- Custom Websocket logs
- Darktrace
- Data Exfiltration Detection
- DGA
- Digital Guardian
- Docker
- Elastic APM
- Elastic Fleet Server
- Elastic Security
- Elastic Stack monitoring
- Elasticsearch Service Billing
- Envoy Proxy
- ESET PROTECT
- ESET Threat Intelligence
- etcd
- Falco
- F5
- File Integrity Monitoring
- FireEye Network Security
- First EPSS
- Forcepoint Web Security
- ForgeRock
- Fortinet
- Gigamon
- GitHub
- GitLab
- Golang
- Google Cloud
- Custom GCS Input
- GCP
- GCP Audit logs
- GCP Billing metrics
- GCP Cloud Run metrics
- GCP CloudSQL metrics
- GCP Compute metrics
- GCP Dataproc metrics
- GCP DNS logs
- GCP Firestore metrics
- GCP Firewall logs
- GCP GKE metrics
- GCP Load Balancing metrics
- GCP Metrics Input
- GCP PubSub logs (custom)
- GCP PubSub metrics
- GCP Redis metrics
- GCP Security Command Center
- GCP Storage metrics
- GCP VPC Flow logs
- GCP Vertex AI
- GoFlow2 logs
- Hadoop
- HAProxy
- Hashicorp Vault
- HTTP Endpoint logs (custom)
- IBM MQ
- IIS
- Imperva
- InfluxDb
- Infoblox
- Iptables
- Istio
- Jamf Compliance Reporter
- Jamf Pro
- Jamf Protect
- Jolokia Input
- Journald logs (custom)
- JumpCloud
- Kafka
- Keycloak
- Kubernetes
- LastPass
- Lateral Movement Detection
- Linux Metrics
- Living off the Land Attack Detection
- Logs (custom)
- Lumos
- Lyve Cloud
- Mattermost
- Memcached
- Menlo Security
- Microsoft
- Microsoft 365
- Microsoft Defender for Cloud
- Microsoft Defender for Endpoint
- Microsoft DHCP
- Microsoft DNS Server
- Microsoft Entra ID Entity Analytics
- Microsoft Exchange Online Message Trace
- Microsoft Exchange Server
- Microsoft Graph Activity Logs
- Microsoft M365 Defender
- Microsoft Office 365 Metrics Integration
- Microsoft Sentinel
- Microsoft SQL Server
- Mimecast
- ModSecurity Audit
- MongoDB
- MongoDB Atlas
- MySQL
- Nagios XI
- NATS
- NetFlow Records
- Netskope
- Network Beaconing Identification
- Network Packet Capture
- Nginx
- Okta
- Oracle
- OpenCanary
- Osquery
- Palo Alto
- pfSense
- PHP-FPM
- PingOne
- PingFederate
- Pleasant Password Server
- PostgreSQL
- Prometheus
- Proofpoint TAP
- Proofpoint On Demand
- Pulse Connect Secure
- Qualys VMDR
- QNAP NAS
- RabbitMQ Logs
- Radware DefensePro Logs
- Rapid7
- Redis
- Rubrik RSC Metrics Integration
- Salesforce
- SentinelOne
- ServiceNow
- Slack Logs
- Snort
- Snyk
- SonicWall Firewall
- Sophos
- Spring Boot
- SpyCloud Enterprise Protection
- SQL Input
- Squid Logs
- SRX
- STAN
- Statsd Input
- Sublime Security
- Suricata
- StormShield SNS
- Symantec
- Symantec Endpoint Security
- Sysmon for Linux
- Sysdig
- Syslog Router Integration
- System
- System Audit
- Tanium
- TCP Logs (custom)
- Teleport
- Tenable
- Threat intelligence
- ThreatConnect
- Threat Map
- Thycotic Secret Server
- Tines
- Traefik
- Trellix
- Trend Micro
- TYCHON Agentless
- UDP Logs (custom)
- Universal Profiling
- Vectra Detect
- VMware
- WatchGuard Firebox
- WebSphere Application Server
- Windows
- Wiz
- Zeek
- ZeroFox
- Zero Networks
- ZooKeeper Metrics
- Zoom
- Zscaler
Lateral Movement Detection Model
editLateral Movement Detection Model
editVersion |
2.1.4 (View all) |
Compatible Kibana version(s) |
8.9.0 or higher |
Supported Serverless project types |
Security |
Subscription level |
Platinum |
Level of support |
Elastic |
The Lateral movement detection model package contains assets that detect lateral movement based on file transfer activity and Windows RDP events. This package requires a Platinum subscription. Please ensure that you have a Trial, Platinum, or Enterprise subscription before proceeding. This package is licensed under Elastic License 2.0.
For more detailed information refer to the following blogs:
Installation
edit- Upgrading: If upgrading from a version below v2.0.0, see the section v2.0.0 and beyond.
- Add the Integration Package: Install the package via Management > Integrations > Add Lateral Movement Detection. Configure the integration name and agent policy. Click Save and Continue.
-
Check the health of the transform: The transform is scheduled to run every hour. This transform creates the index
ml-rdp-lmd
. To check the health of the transform go to Management > Stack Management > Data > Transforms underlogs-lmd.pivot_transform-default-<FLEET-TRANSFORM-VERSION>
. -
Create data views for anomaly detection jobs: The anomaly detection jobs under this package rely on two indices. One has file transfer events (
logs-*
), and the other index (ml-rdp-lmd
) collects RDP session information from a transform. Before enabling the anomaly detection jobs, create a data view with both index patterns.- Go to Stack Management > Kibana > Data Views and click Create data view.
-
Enter the name of your respective index patterns in the Index pattern box, i.e.,
logs-*, ml-rdp-lmd
, and copy the same in the Name field. -
Select
@timestamp
under the Timestamp field and click on Save data view to Kibana. -
Use the new data view (
logs-*, ml-rdp-lmd
) to create anomaly detection jobs for this package.
-
Add preconfigured anomaly detection jobs: In Machine Learning > Anomaly Detection, when you create a job, you should see an option to
Use preconfigured jobs
with a card for Lateral Movement Detection. When you select the card, you will see pre-configured anomaly detection jobs that you can enable depending on what makes the most sense for your environment. Note: In the Machine Learning app, these configurations are available only when data exists that matches the query specified in the lmd-ml file. For example, this would be available inlogs-endpoint.events.*
if you used Elastic Defend to collect events. -
Data view configuration for Dashboards: For the dashboard to work as expected, the following settings need to be configured in Kibana.
- You have started the above anomaly detection jobs.
-
You have read access to
.ml-anomalies-shared
index or are assigned themachine_learning_user
role. For more information on roles, please refer to Built-in roles in Elastic. Please be aware that a user who has access to the underlying machine learning results indices can see the results of all jobs in all spaces. Be mindful of granting permissions if you use Kibana spaces to control which users can see which machine learning results. For more information on machine learning privileges, refer to setup-privileges. -
After enabling the jobs, go to Management > Stack Management > Kibana > Data Views. Click on Create data view with the following settings:
-
Name:
.ml-anomalies-shared
-
Index pattern :
.ml-anomalies-shared
- Select Show Advanced settings enable Allow hidden and system indices
-
Custom data view ID:
.ml-anomalies-shared
-
Name:
-
Enabling detection rules: You can also enable detection rules to alert on Lateral Movement activity in your environment, based on anomalies flagged by the above ML jobs. As of version 2.0.0 of this package, these rules are available as part of the Detection Engine, and can be found using the tag
Use Case: Lateral Movement Detection
. See this documentation for more information on importing and enabling the rules. - Use with Living off the Land Detection: This integration package can be used along with Living off the Land detection, see the section Install Living off the Land package to detect malicious processes.
In Security > Rules, filtering with the “Use Case: Lateral Movement Detection” tag
Dashboard
editAfter the anomaly detectors and the data views for the dashboard are configured, the Lateral Movement Detection Dashboard is available under Analytics > Dashboard. This dashboard gives an overview of anomalies triggered for the lateral movement detection package.
Install ProblemChild package to detect malicious processes
editTo detect malicious RDP processes started in a session, install the Living off the Land Attack (LotL) Detection package. Follow the steps under the package overview to install the related assets. Use the below filter query to examine model predictions on RDP events only.
Clone the anomaly detection jobs available under the Living off the Land Attack (LotL) Detection package and follow the below steps to customize them only to process Windows RDP events in the datafeed:
- Click on the Actions panel at the right-most corner of the anomaly detection job and then select the Edit job option.
- Under the Datafeed panel, enter the below query to filter malicious RDP processes.
{ "bool": { "minimum_should_match": 1, "should": [ { "match": { "problemchild.prediction": 1 } }, { "match": { "blocklist_label": 1 } } ], "must_not": [ { "terms": { "user.name": [ "system" ] } } ], "filter": [ { "exists": { "field": "process.Ext.session_info.client_address" } }, { "exists": { "field": "process.Ext.authentication_id" } }, { "exists": { "field": "host.ip" } }, { "term": { "event.category": "process" } }, { "term": { "process.Ext.session_info.logon_type": "RemoteInteractive" } } ] } }
Anomaly Detection Jobs
editDetects potential lateral movement activity by identifying malicious file transfers and RDP sessions in an environment.
Job | Description |
---|---|
lmd_high_count_remote_file_transfer |
Detects unusually high file transfers to a remote host in the network. |
lmd_high_file_size_remote_file_transfer |
Detects unusually high size of files shared with a remote host in the network. |
lmd_rare_file_extension_remote_transfer |
Detects rare file extensions shared with a remote host in the network. |
lmd_rare_file_path_remote_transfer |
Detects unusual folders and directories on which a file is transferred (by a host). |
lmd_high_mean_rdp_session_duration |
Detects unusually high mean of RDP session duration. |
lmd_high_var_rdp_session_duration |
Detects unusually high variance in RDP session duration. |
lmd_high_sum_rdp_number_of_processes |
Detects unusually high number of processes started in a single RDP session. |
lmd_unusual_time_weekday_rdp_session_start |
Detects an RDP session started at an usual time or weekday. |
lmd_high_rdp_distinct_count_source_ip_for_destination |
Detects a high count of source IPs making an RDP connection with a single destination IP. |
lmd_high_rdp_distinct_count_destination_ip_for_source |
Detects a high count of destination IPs establishing an RDP connection with a single source IP. |
lmd_high_mean_rdp_process_args |
Detects unusually high number of process arguments in an RDP session. |
v2.0.0 and beyond
editv2.0.0 of the package introduces breaking changes, namely deprecating detection rules from the package. To continue receiving updates to Lateral Movement Detection, we recommend upgrading to v2.0.0 after doing the following:
-
Delete existing ML jobs: Navigate to Machine Learning > Anomaly Detection and delete jobs corresponding to the following IDs:
- high-count-remote-file-transfer
- high-file-size-remote-file-transfer
- rare-file-extension-remote-transfer
- rare-file-path-remote-transfer
- high-mean-rdp-session-duration
- high-var-rdp-session-duration
- high-sum-rdp-number-of-processes
- unusual-time-weekday-rdp-session-start
- high-rdp-distinct-count-source-ip-for-destination
- high-rdp-distinct-count-destination-ip-for-source
- high-mean-rdp-process-args
Depending on the version of the package you’re using, you might also be able to search for the above jobs using the group lateral_movement
.
-
Uninstall existing rules associated with this package: Navigate to Security > Rules and delete the following rules:
- Spike in Remote File Transfers
- Unusual Remote File Size
- Unusual Remote File Directory
- Unusual Remote File Extension
- Malicious Remote File Creation
- Remote File Creation on a Sensitive Directory
- Spike in number of processes in an RDP session
- High mean of RDP session duration
- High variance in RDP session duration
- Unusually high number of process arguments in an RDP session
- Spike in number of connections made to a source IP
- Spike in number of connections made to a destination IP
- Unusual time or day for an RDP session start
Depending on the version of the package you’re using, you might also be able to search for the above rules using the tag Lateral Movement
.
- Upgrade the Lateral Movement Detection package to v2.0.0 using the steps here
- Install the new rules as described in the Enabling detection rules section below
In version 2.1.2, the package ignores data in cold and frozen data tiers to reduce heap memory usage, avoid running on outdated data, and to follow best practices.
Licensing
editUsage in production requires that you have a license key that permits use of machine learning features.
Changelog
editChangelog
Version | Details | Kibana version(s) |
---|---|---|
2.1.4 |
Enhancement (View pull request) |
8.9.0 or higher |
2.1.3 |
Enhancement (View pull request) |
8.9.0 or higher |
2.1.2 |
Enhancement (View pull request) |
8.9.0 or higher |
2.1.1 |
Enhancement (View pull request) |
8.9.0 or higher |
2.1.0 |
Enhancement (View pull request) |
8.9.0 or higher |
2.0.0 |
Enhancement (View pull request) |
8.9.0 or higher |
1.0.2 |
Enhancement (View pull request) |
8.8.0 or higher |
1.0.1 |
Enhancement (View pull request) |
8.5.0 or higher |
1.0.0 |
Enhancement (View pull request) |
8.5.0 or higher |
0.0.1 |
Enhancement (View pull request) |
— |
On this page