- Elastic integrations
- Integrations quick reference
- 1Password
- Abnormal Security
- ActiveMQ
- Active Directory Entity Analytics
- Admin By Request EPM integration
- Airflow
- Akamai
- Apache
- API (custom)
- Arbor Peakflow SP Logs
- Arista NG Firewall
- Atlassian
- Auditd
- Auth0
- authentik
- AWS
- Amazon CloudFront
- Amazon DynamoDB
- Amazon EBS
- Amazon EC2
- Amazon ECS
- Amazon EMR
- AWS API Gateway
- Amazon GuardDuty
- AWS Health
- Amazon Kinesis Data Firehose
- Amazon Kinesis Data Stream
- Amazon MQ
- Amazon Managed Streaming for Apache Kafka (MSK)
- Amazon NAT Gateway
- Amazon RDS
- Amazon Redshift
- Amazon S3
- Amazon S3 Storage Lens
- Amazon Security Lake
- Amazon SNS
- Amazon SQS
- Amazon VPC
- Amazon VPN
- AWS Bedrock
- AWS Billing
- AWS CloudTrail
- AWS CloudWatch
- AWS ELB
- AWS Fargate
- AWS Inspector
- AWS Lambda
- AWS Logs (custom)
- AWS Network Firewall
- AWS Route 53
- AWS Security Hub
- AWS Transit Gateway
- AWS Usage
- AWS WAF
- Azure
- Activity logs
- App Service
- Application Gateway
- Application Insights metrics
- Application Insights metrics overview
- Application State Insights metrics
- Azure logs (v2 preview)
- Azure OpenAI
- Billing metrics
- Container instance metrics
- Container registry metrics
- Container service metrics
- Custom Azure Logs
- Custom Blob Storage Input
- Database Account metrics
- Event Hub input
- Firewall logs
- Frontdoor
- Functions
- Microsoft Entra ID
- Monitor metrics
- Network Watcher VNet
- Network Watcher NSG
- Platform logs
- Resource metrics
- Spring Cloud logs
- Storage Account metrics
- Virtual machines metrics
- Virtual machines scaleset metrics
- Barracuda
- BeyondInsight and Password Safe Integration
- BitDefender
- Bitwarden
- blacklens.io
- Blue Coat Director Logs
- BBOT (Bighuge BLS OSINT Tool)
- Box Events
- Bravura Monitor
- Broadcom ProxySG
- Canva
- Cassandra
- CEL Custom API
- Ceph
- Check Point
- Cilium Tetragon
- CISA Known Exploited Vulnerabilities
- Cisco
- Cisco Meraki Metrics
- Citrix
- Claroty CTD
- Cloudflare
- Cloud Asset Inventory
- CockroachDB Metrics
- Common Event Format (CEF)
- Containerd
- CoreDNS
- Corelight
- Couchbase
- CouchDB
- Cribl
- CrowdStrike
- Cyberark
- Cybereason
- CylanceProtect Logs
- Custom Websocket logs
- Darktrace
- Data Exfiltration Detection
- DGA
- Digital Guardian
- Docker
- DomainTools Real Time Unified Feeds
- Elastic APM
- Elastic Fleet Server
- Elastic Security
- Elastic Stack monitoring
- Elasticsearch Service Billing
- Envoy Proxy
- ESET PROTECT
- ESET Threat Intelligence
- etcd
- Falco
- F5
- File Integrity Monitoring
- FireEye Network Security
- First EPSS
- Forcepoint Web Security
- ForgeRock
- Fortinet
- Gigamon
- GitHub
- GitLab
- Golang
- Google Cloud
- Custom GCS Input
- GCP
- GCP Audit logs
- GCP Billing metrics
- GCP Cloud Run metrics
- GCP CloudSQL metrics
- GCP Compute metrics
- GCP Dataproc metrics
- GCP DNS logs
- GCP Firestore metrics
- GCP Firewall logs
- GCP GKE metrics
- GCP Load Balancing metrics
- GCP Metrics Input
- GCP PubSub logs (custom)
- GCP PubSub metrics
- GCP Redis metrics
- GCP Security Command Center
- GCP Storage metrics
- GCP VPC Flow logs
- GCP Vertex AI
- GoFlow2 logs
- Hadoop
- HAProxy
- Hashicorp Vault
- HTTP Endpoint logs (custom)
- IBM MQ
- IIS
- Imperva
- InfluxDb
- Infoblox
- Iptables
- Istio
- Jamf Compliance Reporter
- Jamf Pro
- Jamf Protect
- Jolokia Input
- Journald logs (custom)
- JumpCloud
- Kafka
- Keycloak
- Kubernetes
- LastPass
- Lateral Movement Detection
- Linux Metrics
- Living off the Land Attack Detection
- Logs (custom)
- Lumos
- Lyve Cloud
- Mattermost
- Memcached
- Menlo Security
- Microsoft
- Microsoft 365
- Microsoft Defender for Cloud
- Microsoft Defender for Endpoint
- Microsoft DHCP
- Microsoft DNS Server
- Microsoft Entra ID Entity Analytics
- Microsoft Exchange Online Message Trace
- Microsoft Exchange Server
- Microsoft Graph Activity Logs
- Microsoft M365 Defender
- Microsoft Office 365 Metrics Integration
- Microsoft Sentinel
- Microsoft SQL Server
- Mimecast
- ModSecurity Audit
- MongoDB
- MongoDB Atlas
- MySQL
- Nagios XI
- NATS
- NetFlow Records
- Netskope
- Network Beaconing Identification
- Network Packet Capture
- Nginx
- Okta
- Oracle
- OpenAI
- OpenCanary
- Osquery
- Palo Alto
- pfSense
- PHP-FPM
- PingOne
- PingFederate
- Pleasant Password Server
- PostgreSQL
- Prometheus
- Proofpoint TAP
- Proofpoint On Demand
- Pulse Connect Secure
- Qualys VMDR
- QNAP NAS
- RabbitMQ Logs
- Radware DefensePro Logs
- Rapid7
- Redis
- Rubrik RSC Metrics Integration
- Sailpoint Identity Security Cloud
- Salesforce
- SentinelOne
- ServiceNow
- Slack Logs
- Snort
- Snyk
- SonicWall Firewall
- Sophos
- Spring Boot
- SpyCloud Enterprise Protection
- SQL Input
- Squid Logs
- SRX
- STAN
- Statsd Input
- Sublime Security
- Suricata
- StormShield SNS
- Symantec
- Symantec Endpoint Security
- Sysmon for Linux
- Sysdig
- Syslog Router Integration
- System
- System Audit
- Tanium
- TCP Logs (custom)
- Teleport
- Tenable
- Threat intelligence
- ThreatConnect
- Threat Map
- Thycotic Secret Server
- Tines
- Traefik
- Trellix
- Trend Micro
- TYCHON Agentless
- UDP Logs (custom)
- Universal Profiling
- Vectra Detect
- VMware
- WatchGuard Firebox
- WebSphere Application Server
- Windows
- Wiz
- Zeek
- ZeroFox
- Zero Networks
- ZooKeeper Metrics
- Zoom
- Zscaler
Recorded Future Integration
editRecorded Future Integration
editVersion |
1.29.0 (View all) |
Compatible Kibana version(s) |
8.13.0 or higher |
Supported Serverless project types |
Security |
Subscription level |
Basic |
Level of support |
Elastic |
The Recorded Future integration fetches risklists from the Recorded Future API.
It supports domain
, hash
, ip
and url
entities.
In order to use it you need to define the entity
and list
to fetch. Check with
Recorded Future for the available lists for each entity. To fetch indicators
from multiple entities, it’s necessary to define one integration for each.
Alternatively, it’s also possible to use the integration to fetch custom Fusion files by supplying the URL to the CSV file as the Custom URL configuration option.
Expiration of Indicators of Compromise (IOCs)
editThe ingested IOCs expire after certain duration. An Elastic Transform is created to faciliate only active IOCs be available to the end users. This transform creates a destination index named logs-ti_recordedfuture_latest.threat-1
which only contains active and unexpired IOCs. The destination index also has an alias logs-ti_recordedfuture_latest.threat
. When setting up indicator match rules, use this latest destination index to avoid false positives from expired IOCs. Please read ILM Policy below which is added to avoid unbounded growth on source .ds-logs-ti_recordedfuture.threat-*
indices.
ILM Policy
editTo facilitate IOC expiration, source datastream-backed indices .ds-logs-ti_recordedfuture.threat-*
are allowed to contain duplicates from each polling interval. ILM policy is added to these source indices so it doesn’t lead to unbounded growth. This means data in these source indices will be deleted after 5 days
from ingested date.
For large risklist downloads, adjust the timeout setting so that the Agent has enough time to download and process the risklist.
Example
An example event for threat
looks as following:
{ "@timestamp": "2024-08-02T06:24:04.201Z", "agent": { "ephemeral_id": "25d7a936-2b7c-4476-9181-82d1296ce9df", "id": "8299ae35-ee0e-4107-9acb-1b6acfdda1fb", "name": "docker-fleet-agent", "type": "filebeat", "version": "8.13.0" }, "data_stream": { "dataset": "ti_recordedfuture.threat", "namespace": "67234", "type": "logs" }, "ecs": { "version": "8.11.0" }, "elastic_agent": { "id": "8299ae35-ee0e-4107-9acb-1b6acfdda1fb", "snapshot": false, "version": "8.13.0" }, "event": { "agent_id_status": "verified", "category": [ "threat" ], "created": "2024-08-02T06:24:04.201Z", "dataset": "ti_recordedfuture.threat", "ingested": "2024-08-02T06:24:16Z", "kind": "enrichment", "original": "{\"EvidenceDetails\":\"{\\\"EvidenceDetails\\\": [{\\\"Name\\\": \\\"suspectedCncDnsName\\\", \\\"EvidenceString\\\": \\\"1 sighting on 1 source: ThreatFox Infrastructure Analysis. ThreatFox identified ubykou33.top as possible TA0011 (Command and Control) for CryptBot on December 26, 2023. Most recent link (Dec 26, 2023): https://threatfox.abuse.ch/ioc/1223634\\\", \\\"CriticalityLabel\\\": \\\"Unusual\\\", \\\"MitigationString\\\": \\\"\\\", \\\"Rule\\\": \\\"Historical Suspected C\\\\\u0026C DNS Name\\\", \\\"SourcesCount\\\": 1.0, \\\"Sources\\\": [\\\"source:sIoEOQ\\\"], \\\"Timestamp\\\": \\\"2023-12-26T17:06:29.000Z\\\", \\\"SightingsCount\\\": 1.0, \\\"Criticality\\\": 1.0}, {\\\"Name\\\": \\\"malwareSiteDetected\\\", \\\"EvidenceString\\\": \\\"2 sightings on 2 sources: External Sensor Data Analysis, Bitdefender. ubykou33.top is observed to be a malware site domain that navigates to malicious content including executables, drive-by infection sites, malicious scripts, viruses, trojans, or code.\\\", \\\"CriticalityLabel\\\": \\\"Unusual\\\", \\\"MitigationString\\\": \\\"\\\", \\\"Rule\\\": \\\"Historically Detected Malware Operation\\\", \\\"SourcesCount\\\": 2.0, \\\"Sources\\\": [\\\"source:kBB1fk\\\", \\\"source:d3Awkm\\\"], \\\"Timestamp\\\": \\\"2024-01-26T00:00:00.000Z\\\", \\\"SightingsCount\\\": 2.0, \\\"Criticality\\\": 1.0}, {\\\"Name\\\": \\\"malwareSiteSuspected\\\", \\\"EvidenceString\\\": \\\"1 sighting on 1 source: Bitdefender. Detected malicious behavior from an endpoint agent via global telemetry. Last observed on Jan 26, 2024.\\\", \\\"CriticalityLabel\\\": \\\"Unusual\\\", \\\"MitigationString\\\": \\\"\\\", \\\"Rule\\\": \\\"Historically Suspected Malware Operation\\\", \\\"SourcesCount\\\": 1.0, \\\"Sources\\\": [\\\"source:d3Awkm\\\"], \\\"Timestamp\\\": \\\"2024-01-26T00:00:00.000Z\\\", \\\"SightingsCount\\\": 1.0, \\\"Criticality\\\": 1.0}, {\\\"Name\\\": \\\"recentMalwareSiteDetected\\\", \\\"EvidenceString\\\": \\\"1 sighting on 1 source: External Sensor Data Analysis. ubykou33.top is observed to be a malware site domain that navigates to malicious content including executables, drive-by infection sites, malicious scripts, viruses, trojans, or code.\\\", \\\"CriticalityLabel\\\": \\\"Malicious\\\", \\\"MitigationString\\\": \\\"\\\", \\\"Rule\\\": \\\"Recently Detected Malware Operation\\\", \\\"SourcesCount\\\": 1.0, \\\"Sources\\\": [\\\"source:kBB1fk\\\"], \\\"Timestamp\\\": \\\"2024-05-08T23:11:43.601Z\\\", \\\"SightingsCount\\\": 1.0, \\\"Criticality\\\": 3.0}]}\",\"Name\":\"ubykou33.top\",\"Risk\":\"67\",\"RiskString\":\"4/52\"}", "risk_score": 67, "type": [ "indicator" ] }, "input": { "type": "httpjson" }, "recordedfuture": { "evidence_details": [ { "criticality": 1, "criticality_label": "Unusual", "evidence_string": "1 sighting on 1 source: ThreatFox Infrastructure Analysis. ThreatFox identified ubykou33.top as possible TA0011 (Command and Control) for CryptBot on December 26, 2023. Most recent link (Dec 26, 2023): https://threatfox.abuse.ch/ioc/1223634", "mitigation_string": "", "name": "suspectedCncDnsName", "rule": "Historical Suspected C&C DNS Name", "sightings_count": 1, "sources": [ "source:sIoEOQ" ], "sources_count": 1, "timestamp": "2023-12-26T17:06:29.000Z" }, { "criticality": 1, "criticality_label": "Unusual", "evidence_string": "2 sightings on 2 sources: External Sensor Data Analysis, Bitdefender. ubykou33.top is observed to be a malware site domain that navigates to malicious content including executables, drive-by infection sites, malicious scripts, viruses, trojans, or code.", "mitigation_string": "", "name": "malwareSiteDetected", "rule": "Historically Detected Malware Operation", "sightings_count": 2, "sources": [ "source:kBB1fk", "source:d3Awkm" ], "sources_count": 2, "timestamp": "2024-01-26T00:00:00.000Z" }, { "criticality": 1, "criticality_label": "Unusual", "evidence_string": "1 sighting on 1 source: Bitdefender. Detected malicious behavior from an endpoint agent via global telemetry. Last observed on Jan 26, 2024.", "mitigation_string": "", "name": "malwareSiteSuspected", "rule": "Historically Suspected Malware Operation", "sightings_count": 1, "sources": [ "source:d3Awkm" ], "sources_count": 1, "timestamp": "2024-01-26T00:00:00.000Z" }, { "criticality": 3, "criticality_label": "Malicious", "evidence_string": "1 sighting on 1 source: External Sensor Data Analysis. ubykou33.top is observed to be a malware site domain that navigates to malicious content including executables, drive-by infection sites, malicious scripts, viruses, trojans, or code.", "mitigation_string": "", "name": "recentMalwareSiteDetected", "rule": "Recently Detected Malware Operation", "sightings_count": 1, "sources": [ "source:kBB1fk" ], "sources_count": 1, "timestamp": "2024-05-08T23:11:43.601Z" } ], "list": "test", "name": "ubykou33.top", "risk_string": "4/52" }, "tags": [ "preserve_original_event", "forwarded", "recordedfuture" ], "threat": { "feed": { "name": "Recorded Future" }, "indicator": { "provider": [ "ThreatFox Infrastructure Analysis", "External Sensor Data Analysis", "Bitdefender" ], "scanner_stats": 5, "sightings": 5, "type": "domain-name", "url": { "domain": "ubykou33.top" } } } }
Exported fields
Field | Description | Type |
---|---|---|
@timestamp |
Event timestamp. |
date |
cloud.image.id |
Image ID for the cloud instance. |
keyword |
data_stream.dataset |
Data stream dataset name. |
constant_keyword |
data_stream.namespace |
Data stream namespace. |
constant_keyword |
data_stream.type |
Data stream type. |
constant_keyword |
event.dataset |
Event dataset |
constant_keyword |
event.module |
Event module |
constant_keyword |
host.containerized |
If the host is a container. |
boolean |
host.os.build |
OS build information. |
keyword |
host.os.codename |
OS codename, if any. |
keyword |
input.type |
Type of Filebeat input. |
keyword |
labels.is_ioc_transform_source |
Indicates whether an IOC is in the raw source data stream, or the in latest destination index. |
constant_keyword |
log.flags |
Flags for the log file. |
keyword |
log.offset |
Offset of the entry in the log file. |
long |
recordedfuture.evidence_details.criticality |
double |
|
recordedfuture.evidence_details.criticality_label |
keyword |
|
recordedfuture.evidence_details.evidence_string |
keyword |
|
recordedfuture.evidence_details.mitigation_string |
keyword |
|
recordedfuture.evidence_details.name |
keyword |
|
recordedfuture.evidence_details.rule |
keyword |
|
recordedfuture.evidence_details.sightings_count |
integer |
|
recordedfuture.evidence_details.sources |
keyword |
|
recordedfuture.evidence_details.sources_count |
integer |
|
recordedfuture.evidence_details.timestamp |
date |
|
recordedfuture.list |
User-configured risklist. |
keyword |
recordedfuture.name |
Indicator value. |
keyword |
recordedfuture.risk_string |
Details of risk rules observed. |
keyword |
threat.feed.name |
Display friendly feed name |
constant_keyword |
threat.indicator.first_seen |
The date and time when intelligence source first reported sighting this indicator. |
date |
threat.indicator.last_seen |
The date and time when intelligence source last reported sighting this indicator. |
date |
threat.indicator.modified_at |
The date and time when intelligence source last modified information for this indicator. |
date |
Changelog
editChangelog
Version | Details | Kibana version(s) |
---|---|---|
1.29.0 |
Enhancement (View pull request) |
8.13.0 or higher |
1.28.0 |
Enhancement (View pull request) |
8.13.0 or higher |
1.27.0 |
Enhancement (View pull request) |
8.13.0 or higher |
1.26.3 |
Bug fix (View pull request) |
8.13.0 or higher |
1.26.2 |
Bug fix (View pull request) |
8.13.0 or higher |
1.26.1 |
Bug fix (View pull request) |
8.13.0 or higher |
1.26.0 |
Enhancement (View pull request) |
8.13.0 or higher |
1.25.1 |
Bug fix (View pull request) |
8.12.0 or higher |
1.25.0 |
Enhancement (View pull request) |
8.12.0 or higher |
1.24.0 |
Enhancement (View pull request) |
8.12.0 or higher |
1.23.0 |
Enhancement (View pull request) |
8.12.0 or higher |
1.22.0 |
Enhancement (View pull request) |
8.12.0 or higher |
1.21.0 |
Enhancement (View pull request) |
8.8.0 or higher |
1.20.2 |
Enhancement (View pull request) |
8.8.0 or higher |
1.20.1 |
Bug fix (View pull request) |
8.8.0 or higher |
1.20.0 |
Enhancement (View pull request) |
8.8.0 or higher |
1.19.0 |
Enhancement (View pull request) |
8.8.0 or higher |
1.18.1 |
Bug fix (View pull request) |
8.8.0 or higher |
1.18.0 |
Enhancement (View pull request) |
8.8.0 or higher |
1.17.0 |
Enhancement (View pull request) |
8.8.0 or higher |
1.16.0 |
Enhancement (View pull request) Enhancement (View pull request) |
8.8.0 or higher |
1.15.1 |
Bug fix (View pull request) |
— |
1.15.0 |
Enhancement (View pull request) |
8.8.0 or higher |
1.14.0 |
Enhancement (View pull request) |
8.8.0 or higher |
1.13.0 |
Enhancement (View pull request) |
8.8.0 or higher |
1.12.0 |
Enhancement (View pull request) |
8.8.0 or higher |
1.11.0 |
Enhancement (View pull request) |
8.8.0 or higher |
1.10.0 |
Enhancement (View pull request) |
8.8.0 or higher |
1.9.0 |
Enhancement (View pull request) |
8.8.0 or higher |
1.8.0 |
Enhancement (View pull request) |
8.7.1 or higher |
1.7.0 |
Enhancement (View pull request) |
8.0.0 or higher |
1.6.0 |
Enhancement (View pull request) |
8.0.0 or higher |
1.5.0 |
Enhancement (View pull request) |
8.0.0 or higher |
1.4.1 |
Bug fix (View pull request) |
8.0.0 or higher |
1.4.0 |
Enhancement (View pull request) |
— |
1.3.0 |
Enhancement (View pull request) |
8.0.0 or higher |
1.2.1 |
Bug fix (View pull request) Bug fix (View pull request) |
8.0.0 or higher |
1.2.0 |
Enhancement (View pull request) |
8.0.0 or higher |
1.1.0 |
Enhancement (View pull request) |
8.0.0 or higher |
1.0.1 |
Enhancement (View pull request) |
8.0.0 or higher |
1.0.0 |
Enhancement (View pull request) |
8.0.0 or higher |
0.1.3 |
Enhancement (View pull request) |
— |
0.1.2 |
Enhancement (View pull request) |
— |
0.1.1 |
Enhancement (View pull request) |
— |
0.1.0 |
Enhancement (View pull request) |
— |