Collect logs from Zscaler Internet Access (ZIA) with Elastic Agent.
What is an Elastic integration?
This integration is powered by Elastic Agent. Elastic Agent is a single, unified way to add monitoring for logs, metrics, and other types of data to a host. It can also protect hosts from security threats, query data from operating systems, forward data from remote services or hardware, and more. Refer to our documentation for a detailed comparison between Beats and Elastic Agent.
Prefer to use Beats for this use case? See Filebeat modules for logs or Metricbeat modules for metrics.
See the integrations quick start guides to get started:
This integration is for Zscaler Internet Access logs. It can be used to receive logs sent by NSS log server on respective TCP ports.
The log message is expected to be in JSON format. The data is mapped to ECS fields where applicable and the remaining fields are written under zscaler_zia.<data-stream-name>.*.
Please make sure to use the given response formats for NSS and Cloud NSS Feeds.
Note: Please make sure to use latest version of given response formats.
This package has been tested against Zscaler Internet Access version 6.1
See: Zscaler Vendor documentation
Zscaler response format (v1):
<%d{syslogid}>%s{Monthname} %2d{Dayofmonth} %02d{Hour}:%02d{Minutes}:%02d{Seconds} [%s{Deviceip}] ZscalerNSS: %s{Eventinfo}\n
Sample Response:
<114>Dec 10 14:04:28 [175.16.199.1] ZscalerNSS: Zscaler cloud configuration connection to 175.16.199.1:443 lost and unavailable for the past 2325.00 minutes
See: Zscaler Vendor documentation
Zscaler response format (v1):
\{ "sourcetype" : "zscalernss-dns", "event" :\{"datetime":"%s{time}","user":"%s{elogin}","department":"%s{edepartment}","location":"%s{elocation}","reqaction":"%s{reqaction}","resaction":"%s{resaction}","reqrulelabel":"%s{reqrulelabel}","resrulelabel":"%s{resrulelabel}","dns_reqtype":"%s{reqtype}","dns_req":"%s{req}","dns_resp":"%s{res}","srv_dport":"%d{sport}","durationms":"%d{durationms}","clt_sip":"%s{cip}","srv_dip":"%s{sip}","category":"%s{domcat}","deviceowner":"%s{deviceowner}","devicehostname":"%s{devicehostname}"\}\}
Sample Response:
{ "sourcetype" : "zscalernss-dns", "event" :{"datetime":"Fri Dec 17 07:27:54 2021","user":"some_user@example.com","department":"Unknown","location":"TestLoc%20DB","reqaction":"REQ_ALLOW","resaction":"Some Response Action","reqrulelabel":"Access%20Blocked","resrulelabel":"None","dns_reqtype":"Some type","dns_req":"example.com","dns_resp":"Some response string","srv_dport":"8080","durationms":"123456","clt_sip":"81.2.69.193","srv_dip":"81.2.69.144","category":"Professional Services","deviceowner":"Owner77","devicehostname":"Machine9000"}}
See: Zscaler Vendor documentation
Zscaler response format (v1):
\{ "sourcetype" : "zscalernss-fw", "event" :\{"datetime":"%s{time}","user":"%s{elogin}","department":"%s{edepartment}","locationname":"%s{elocation}","cdport":"%d{cdport}","csport":"%d{csport}","sdport":"%d{sdport}","ssport":"%d{ssport}","csip":"%s{csip}","cdip":"%s{cdip}","ssip":"%s{ssip}","sdip":"%s{sdip}","tsip":"%s{tsip}","tunsport":"%d{tsport}","tuntype":"%s{ttype}","action":"%s{action}","dnat":"%s{dnat}","stateful":"%s{stateful}","aggregate":"%s{aggregate}","nwsvc":"%s{nwsvc}","nwapp":"%s{nwapp}","proto":"%s{ipproto}","ipcat":"%s{ipcat}","destcountry":"%s{destcountry}","avgduration":"%d{avgduration}","rulelabel":"%s{erulelabel}","inbytes":"%ld{inbytes}","outbytes":"%ld{outbytes}","duration":"%d{duration}","durationms":"%d{durationms}","numsessions":"%d{numsessions}","ipsrulelabel":"%s{ipsrulelabel}","threatcat":"%s{threatcat}","threatname":"%s{ethreatname}","deviceowner":"%s{deviceowner}","devicehostname":"%s{devicehostname}"\}\}
Sample Response:
{ "sourcetype" : "zscalernss-fw", "event" :{"datetime":"Fri Dec 17 07:27:54 2021","user":"some_user@example.com","department":"Unknown","locationname":"TestLoc%20DB","cdport":443,"csport":55018,"sdport":443,"ssport":0,"csip":"0.0.0.0","cdip":"0.0.0.0","ssip":"0.0.0.0","sdip":"0.0.0.0","tsip":"0.0.0.0","tunsport":0,"tuntype":"ZscalerClientConnector","action":"Drop","dnat":"No","stateful":"Yes","aggregate":"No","nwsvc":"HTTPS","nwapp":"http","proto":"TCP","ipcat":"Test Name","destcountry":"Ireland","avgduration":486,"rulelabel":"Access%20Blocked","inbytes":19052,"outbytes":1734,"duration":0,"durationms":486,"numsessions":1,"ipsrulelabel":"None","threatcat":"None","threatname":"None","deviceowner":"admin77","devicehostname":"Machine9000"}}
See: Zscaler Vendor documentation
Zscaler response format (v1):
\{ "sourcetype" : "zscalernss-tunnel", "event" : \{"datetime":"%s{datetime}","Recordtype":"%s{tunnelactionname}","tunneltype":"%s{tunneltype}","user":"%s{vpncredentialname}","location":"%s{elocationname}","sourceip":"%s{sourceip}","destinationip":"%s{destvip}","sourceport":"%d{srcport}","event":"%s{event}","eventreason":"%s{eventreason}","recordid":"%d{recordid}"\}\}
\{ "sourcetype" : "zscalernss-tunnel", "event" : \{"datetime":"%s{datetime}","Recordtype":"%s{tunnelactionname}","tunneltype":"%s{tunneltype}","user":"%s{vpncredentialname}","location":"%s{elocationname}","sourceip":"%s{sourceip}","destinationip":"%s{destvip}","sourceport":"%d{srcport}","txbytes":"%lu{txbytes}","rxbytes":"%lu{rxbytes}","dpdrec":"%d{dpdrec}","recordid":"%d{recordid}"\}\}
\{ "sourcetype" : "zscalernss-tunnel", "event" : \{"datetime":"%s{datetime}","Recordtype":"%s{tunnelactionname}","tunneltype":"IPSEC IKEV %d{ikeversion}","user":"%s{vpncredentialname}","location":"%s{elocationname}","sourceip":"%s{sourceip}","destinationip":"%s{destvip}","sourceport":"%d{srcport}","destinationport":"%d{dstport}","lifetime":"%d{lifetime}","ikeversion":"%d{ikeversion}","spi_in":"%lu{spi_in}","spi_out":"%lu{spi_out}","algo":"%s{algo}","authentication":"%s{authentication}","authtype":"%s{authtype}","recordid":"%d{recordid}"\}\}
\{ "sourcetype" : "zscalernss-tunnel", "event" : \{"datetime":"%s{datetime}","Recordtype":"%s{tunnelactionname}","tunneltype":"IPSEC IKEV %d{ikeversion}","user":"%s{vpncredentialname}","location":"%s{elocationname}","sourceip":"%s{sourceip}","destinationip":"%s{destvip}","sourceport":"%d{srcport}","sourceportstart":"%d{srcportstart}","destinationportstart":"%d{destportstart}","srcipstart":"%s{srcipstart}","srcipend":"%s{srcipend}","destinationipstart":"%s{destipstart}","destinationipend":"%s{destipend}","lifetime":"%d{lifetime}","ikeversion":"%d{ikeversion}","lifebytes":"%d{lifebytes}","spi":"%d{spi}","algo":"%s{algo}","authentication":"%s{authentication}","authtype":"%s{authtype}","protocol":"%s{protocol}","tunnelprotocol":"%s{tunnelprotocol}","policydirection":"%s{policydirection}","recordid":"%d{recordid}"\}\}
Sample Response:
{ "sourcetype" : "zscalernss-tunnel", "event" : {"datetime":"Thu Dec 30 11:40:27 2021","Recordtype":"IPSec Phase1","tunneltype":"IPSEC IKEV 2","user":"81.2.69.145","location":"some-location","sourceip":"81.2.69.145","destinationip":"81.2.69.143","sourceport":"500","destinationport":"500","lifetime":"0","ikeversion":"2","spi_in":"00000000000000000000","spi_out":"11111111111111111111","algo":"AES-CBS","authentication":"HMAC-SHA1-96","authtype":"PSK","recordid":"1111111111111111111"}}
See: Zscaler Vendor documentation
Zscaler response format (v2):
\{ "sourcetype" : "zscalernss-web", "event" :\{"time":"%s{time}","login":"%s{login}","proto":"%s{proto}","eurl":"%s{eurl}","action":"%s{action}","appname":"%s{appname}","appclass":"%s{appclass}","reqsize":"%d{reqsize}","respsize":"%d{respsize}","stime":"%d{stime}","ctime":"%d{ctime}","urlclass":"%s{urlclass}","urlsupercat":"%s{urlsupercat}","urlcat":"%s{urlcat}","malwarecat":"%s{malwarecat}","threatname":"%s{threatname}","riskscore":"%d{riskscore}","dlpeng":"%s{dlpeng}","dlpdict":"%s{dlpdict}","location":"%s{location}","dept":"%s{dept}","cip":"%s{cip}","sip":"%s{sip}","reqmethod":"%s{reqmethod}","respcode":"%s{respcode}","eua":"%s{eua}","ereferer":"%s{ereferer}","ruletype":"%s{ruletype}","rulelabel":"%s{rulelabel}","contenttype":"%s{contenttype}","unscannabletype":"%s{unscannabletype}","deviceowner":"%s{deviceowner}","devicehostname":"%s{devicehostname}"\}\}
Sample Response:
{ "sourcetype" : "zscalernss-web", "event" :{"time":"Fri Dec 17 07:04:57 2021","login":"test@example.com","proto":"HTTP_PROXY","eurl":"browser.events.data.msn.com:443","action":"Blocked","appname":"General Browsing","appclass":"General Browsing","reqsize":"600","respsize":"65","stime":"0","ctime":"0","urlclass":"Business Use","urlsupercat":"Information Technology","urlcat":"Web Search","malwarecat":"None","threatname":"None","riskscore":"0","dlpeng":"None","dlpdict":"None","location":"Test DB","dept":"Unknown","cip":"81.2.69.193","sip":"81.2.69.145","reqmethod":"CONNECT","respcode":"200","eua":"Windows%20Microsoft%20Windows%2010%20Pro%20ZTunnel%2F1.0","ereferer":"None","ruletype":"FwFilter","rulelabel":"Zscaler Proxy Traffic","contenttype":"Other","unscannabletype":"None","deviceowner":"administrator1","devicehostname":"TestMachine35"}}
Caveats:
network.protocol value that is not http or https will be implicitly converted to https for the purposes of URL parsing. The original value of network.protocol will be preserved.This is the alerts dataset.
An example event for alerts looks as following:
{
"@timestamp": "2023-12-10T13:40:32.000Z",
"agent": {
"ephemeral_id": "19a4d9cc-161f-44b9-803e-3d5f3f7668d7",
"id": "fc4affb9-ab52-48ec-b9ce-f65f4390f0b9",
"name": "docker-fleet-agent",
"type": "filebeat",
"version": "8.6.0"
},
"data_stream": {
"dataset": "zscaler_zia.alerts",
"namespace": "ep",
"type": "logs"
},
"destination": {
"address": "81.2.69.193",
"ip": "81.2.69.193",
"port": 9012
},
"ecs": {
"version": "8.6.0"
},
"elastic_agent": {
"id": "fc4affb9-ab52-48ec-b9ce-f65f4390f0b9",
"snapshot": false,
"version": "8.6.0"
},
"event": {
"agent_id_status": "verified",
"dataset": "zscaler_zia.alerts",
"ingested": "2023-02-24T09:40:42Z"
},
"input": {
"type": "tcp"
},
"log": {
"source": {
"address": "172.29.0.7:55310"
},
"syslog": {
"priority": 114
}
},
"message": "ZscalerNSS: SIEM Feed connection \"DNS Logs Feed\" to 81.2.69.193:9012 lost and unavailable for the past 2440.00 minutes",
"related": {
"ip": [
"81.2.69.193"
]
},
"tags": [
"forwarded",
"zscaler_zia-alerts"
],
"zscaler_zia": {
"alerts": {
"connection_lost_minutes": 2440,
"log_feed_name": "DNS Logs Feed"
}
}
}
Exported fields
| Field | Description | Type |
|---|---|---|
@timestamp | Event timestamp. | date |
data_stream.dataset | Data stream dataset. | constant_keyword |
data_stream.namespace | Data stream namespace. | constant_keyword |
data_stream.type | Data stream type. | constant_keyword |
event.dataset | Event dataset. | constant_keyword |
event.module | Event module. | constant_keyword |
input.type | Type of Filebeat input. | keyword |
log.offset | Log offset. | long |
log.source.address | Source address from which the log event was read / sent from. | keyword |
tags | User defined tags. | keyword |
zscaler_zia.alerts.connection_lost_minutes | Amount of time after loosing connection to a server in Minutes. | double |
zscaler_zia.alerts.log_feed_name | Name of the NSS log feed. | keyword |
This is the dns dataset.
An example event for dns looks as following:
{
"@timestamp": "2021-12-17T07:27:54.000Z",
"agent": {
"ephemeral_id": "f4bda282-f29f-4110-ae6f-3e625eb7e71b",
"id": "fc4affb9-ab52-48ec-b9ce-f65f4390f0b9",
"name": "docker-fleet-agent",
"type": "filebeat",
"version": "8.6.0"
},
"data_stream": {
"dataset": "zscaler_zia.dns",
"namespace": "ep",
"type": "logs"
},
"destination": {
"as": {
"number": 29518,
"organization": {
"name": "Bredband2 AB"
}
},
"geo": {
"city_name": "Linköping",
"continent_name": "Europe",
"country_iso_code": "SE",
"country_name": "Sweden",
"location": {
"lat": 58.4167,
"lon": 15.6167
},
"region_iso_code": "SE-E",
"region_name": "Östergötland County"
},
"ip": "89.160.20.156",
"port": 8080
},
"dns": {
"answers": {
"name": "Some response string"
},
"question": {
"name": "example.com",
"type": "Some type"
}
},
"ecs": {
"version": "8.6.0"
},
"elastic_agent": {
"id": "fc4affb9-ab52-48ec-b9ce-f65f4390f0b9",
"snapshot": false,
"version": "8.6.0"
},
"event": {
"agent_id_status": "verified",
"category": [
"network"
],
"dataset": "zscaler_zia.dns",
"duration": 123456000000,
"ingested": "2023-02-24T09:42:00Z",
"kind": "event",
"type": [
"info"
]
},
"input": {
"type": "tcp"
},
"log": {
"source": {
"address": "172.29.0.7:53784"
}
},
"network": {
"protocol": "dns"
},
"related": {
"hosts": [
"Machine9000"
],
"ip": [
"89.160.20.112",
"89.160.20.156"
],
"user": [
"Owner77"
]
},
"source": {
"as": {
"number": 29518,
"organization": {
"name": "Bredband2 AB"
}
},
"geo": {
"city_name": "Linköping",
"continent_name": "Europe",
"country_iso_code": "SE",
"country_name": "Sweden",
"location": {
"lat": 58.4167,
"lon": 15.6167
},
"region_iso_code": "SE-E",
"region_name": "Östergötland County"
},
"ip": "89.160.20.112"
},
"tags": [
"forwarded",
"zscaler_zia-dns"
],
"user": {
"email": "some_user@example.com",
"name": "Owner77"
},
"zscaler_zia": {
"dns": {
"department": "Unknown",
"dom": {
"category": "Professional Services"
},
"duration": {
"milliseconds": 123456
},
"hostname": "Machine9000",
"location": "TestLoc DB",
"request": {
"action": "REQ_ALLOW",
"rule": {
"label": "Access Blocked"
}
},
"response": {
"action": "Some Response Action",
"rule": {
"label": "None"
}
}
}
}
}
Exported fields
| Field | Description | Type |
|---|---|---|
@timestamp | Event timestamp. | date |
data_stream.dataset | Data stream dataset. | constant_keyword |
data_stream.namespace | Data stream namespace. | constant_keyword |
data_stream.type | Data stream type. | constant_keyword |
event.dataset | Event dataset. | constant_keyword |
event.module | Event module. | constant_keyword |
input.type | Type of Filebeat input. | keyword |
log.offset | Log offset. | long |
log.source.address | Source address from which the log event was read / sent from. | keyword |
tags | User defined tags. | keyword |
zscaler_zia.dns.department | Department of the user. | keyword |
zscaler_zia.dns.dom.category | URL Category of the FQDN in the DNS request. | keyword |
zscaler_zia.dns.duration.milliseconds | Duration of the DNS request in milliseconds. | long |
zscaler_zia.dns.hostname | N/A | keyword |
zscaler_zia.dns.location | Gateway location or sub-location of the source. | keyword |
zscaler_zia.dns.request.action | Name of the action that was applied to the DNS request. | keyword |
zscaler_zia.dns.request.rule.label | Name of the rule that was applied to the DNS request. | keyword |
zscaler_zia.dns.response.action | Name of the action that was applied to the DNS response. | keyword |
zscaler_zia.dns.response.rule.label | Name of the rule that was applied to the DNS response. | keyword |
This is the firewall dataset.
An example event for firewall looks as following:
{
"@timestamp": "2021-12-17T07:27:54.000Z",
"agent": {
"ephemeral_id": "83dc07a4-89ea-45b3-b7aa-f72b7e84f8b8",
"id": "fc4affb9-ab52-48ec-b9ce-f65f4390f0b9",
"name": "docker-fleet-agent",
"type": "filebeat",
"version": "8.6.0"
},
"data_stream": {
"dataset": "zscaler_zia.firewall",
"namespace": "ep",
"type": "logs"
},
"destination": {
"bytes": 19052,
"geo": {
"country_name": "Ireland"
},
"ip": "0.0.0.0",
"port": 443
},
"ecs": {
"version": "8.6.0"
},
"elastic_agent": {
"id": "fc4affb9-ab52-48ec-b9ce-f65f4390f0b9",
"snapshot": false,
"version": "8.6.0"
},
"event": {
"action": "drop",
"agent_id_status": "verified",
"category": [
"network"
],
"dataset": "zscaler_zia.firewall",
"duration": 486000000,
"ingested": "2023-02-24T09:43:15Z",
"kind": "event",
"type": [
"info"
]
},
"host": {
"hostname": "Machine9000"
},
"input": {
"type": "tcp"
},
"log": {
"source": {
"address": "172.29.0.7:46908"
}
},
"network": {
"application": "http",
"community_id": "1:hQwW1HWTOUYlk7y4+T2D+UPDU1c=",
"protocol": "https",
"transport": "tcp"
},
"related": {
"hosts": [
"Machine9000"
],
"ip": [
"0.0.0.0"
],
"user": [
"admin77"
]
},
"rule": {
"name": [
"Access Blocked",
"None"
]
},
"source": {
"bytes": 1734,
"ip": "0.0.0.0",
"port": 55018
},
"tags": [
"forwarded",
"zscaler_zia-firewall"
],
"user": {
"email": "some_user@example.com",
"name": "admin77"
},
"zscaler_zia": {
"firewall": {
"aggregate": "No",
"client": {
"destination": {
"ip": "0.0.0.0",
"port": 443
}
},
"department": "Unknown",
"duration": {
"avg": 486,
"milliseconds": 486,
"seconds": 0
},
"ip_category": "Test Name",
"location": {
"name": "TestLoc DB"
},
"nat": "No",
"server": {
"source": {
"ip": "0.0.0.0",
"port": 0
}
},
"session": {
"count": 1
},
"stateful": "Yes",
"threat": {
"category": "None",
"name": "None"
},
"tunnel": {
"ip": "0.0.0.0",
"port": 0,
"type": "ZscalerClientConnector"
}
}
}
}
Exported fields
| Field | Description | Type |
|---|---|---|
@timestamp | Event timestamp. | date |
data_stream.dataset | Data stream dataset. | constant_keyword |
data_stream.namespace | Data stream namespace. | constant_keyword |
data_stream.type | Data stream type. | constant_keyword |
event.dataset | Event dataset. | constant_keyword |
event.module | Event module. | constant_keyword |
input.type | Type of Filebeat input. | keyword |
log.offset | Log offset. | long |
log.source.address | Source address from which the log event was read / sent from. | keyword |
tags | User defined tags. | keyword |
zscaler_zia.firewall.aggregate | keyword | |
zscaler_zia.firewall.client.destination.ip | Client destination IP address. For aggregated sessions, this is the client destination IP address of the last session in the aggregate. | ip |
zscaler_zia.firewall.client.destination.port | Client destination port. For aggregated sessions, this is the client destination port of the last session in the aggregate. | long |
zscaler_zia.firewall.department | Department of the user. | keyword |
zscaler_zia.firewall.duration.avg | Average session duration, in milliseconds, if the sessions were aggregated. | long |
zscaler_zia.firewall.duration.milliseconds | Session or request duration in milliseconds. | long |
zscaler_zia.firewall.duration.seconds | Average session duration, in milliseconds, if the sessions were aggregated. | long |
zscaler_zia.firewall.ip_category | URL category that corresponds to the server IP address. | keyword |
zscaler_zia.firewall.location.name | Name of the location from which the session was initiated. | keyword |
zscaler_zia.firewall.nat | Indicates if the destination NAT policy was applied. | keyword |
zscaler_zia.firewall.server.source.ip | Server source IP address. For aggregated sessions, this is the server source IP address of the last session in the aggregate. | ip |
zscaler_zia.firewall.server.source.port | Server source port. For aggregated sessions, this is the server source port of the last session in the aggregate. | long |
zscaler_zia.firewall.session.count | Number of sessions that were aggregated. | double |
zscaler_zia.firewall.stateful | keyword | |
zscaler_zia.firewall.threat.category | Category of the threat in the Firewall session by the IPS engine. | keyword |
zscaler_zia.firewall.threat.name | Name of the threat detected in the Firewall session by the IPS engine. | keyword |
zscaler_zia.firewall.tunnel.ip | Tunnel IP address of the client (source). For aggregated sessions, this is the client's tunnel IP address corresponding to the last session in the aggregate. | ip |
zscaler_zia.firewall.tunnel.port | Tunnel port on the client side. For aggregated sessions, this is the client's tunnel port corresponding to the last session in the aggregate. | long |
zscaler_zia.firewall.tunnel.type | Traffic forwarding method used to send the traffic to the firewall. | keyword |
This is the tunnel dataset.
An example event for tunnel looks as following:
{
"@timestamp": "2021-12-30T11:20:12.000Z",
"agent": {
"ephemeral_id": "acd73b21-110d-4a61-a22c-0b5a38e01cd4",
"id": "fc4affb9-ab52-48ec-b9ce-f65f4390f0b9",
"name": "docker-fleet-agent",
"type": "filebeat",
"version": "8.6.0"
},
"data_stream": {
"dataset": "zscaler_zia.tunnel",
"namespace": "ep",
"type": "logs"
},
"destination": {
"ip": "81.2.69.143"
},
"ecs": {
"version": "8.6.0"
},
"elastic_agent": {
"id": "fc4affb9-ab52-48ec-b9ce-f65f4390f0b9",
"snapshot": false,
"version": "8.6.0"
},
"event": {
"agent_id_status": "verified",
"category": [
"network"
],
"dataset": "zscaler_zia.tunnel",
"id": "1111111111111111111",
"ingested": "2023-02-24T09:44:32Z",
"kind": "event",
"type": [
"info"
]
},
"input": {
"type": "tcp"
},
"log": {
"source": {
"address": "172.29.0.7:46832"
}
},
"network": {
"transport": "ipsec ikev 1"
},
"related": {
"ip": [
"81.2.69.143",
"81.2.69.145"
],
"user": [
"81.2.69.145"
]
},
"source": {
"ip": "81.2.69.145",
"port": 0
},
"tags": [
"forwarded",
"zscaler_zia-tunnel"
],
"user": {
"name": "81.2.69.145"
},
"zscaler_zia": {
"tunnel": {
"action": {
"type": "IPSec Phase2"
},
"authentication": {
"algorithm": "HMAC-SHA-1",
"type": "None"
},
"destination": {
"end": {
"ip": "81.2.69.143"
},
"start": {
"ip": "81.2.69.143",
"port": 0
}
},
"encryption": {
"algorithm": "AES"
},
"ike": {
"version": 1
},
"life": {
"bytes": 0,
"time": 3600
},
"location": {
"name": "some-location"
},
"policy": {
"direction": "Inbound SA Policy",
"protocol": "Any"
},
"protocol": "ESP",
"source": {
"end": {
"ip": "81.2.69.145"
},
"start": {
"ip": "81.2.69.145",
"port": 0
}
},
"spi": "123456789",
"type": "IPSEC IKEV 1"
}
}
}
Exported fields
| Field | Description | Type |
|---|---|---|
@timestamp | Event timestamp. | date |
data_stream.dataset | Data stream dataset. | constant_keyword |
data_stream.namespace | Data stream namespace. | constant_keyword |
data_stream.type | Data stream type. | constant_keyword |
event.dataset | Event dataset. | constant_keyword |
event.module | Event module. | constant_keyword |
input.type | Type of Filebeat input. | keyword |
log.offset | Log offset. | long |
log.source.address | Source address from which the log event was read / sent from. | keyword |
tags | User defined tags. | keyword |
zscaler_zia.tunnel.action.type | Type of the record. Possible values [ WL_TUNNEL_IPSECPHASE1, WL_TUNNEL_IPSECPHASE2, WL_TUNNEL_EVENT, WL_TUNNEL_SAMPLES ]. | keyword |
zscaler_zia.tunnel.authentication.algorithm | Authentication algorithm. | keyword |
zscaler_zia.tunnel.authentication.type | Authentication type. | keyword |
zscaler_zia.tunnel.destination.end.ip | Phase 2 policy proposal - Destination IP end. | ip |
zscaler_zia.tunnel.destination.start.ip | Phase 2 policy proposal - Destination IP start. | ip |
zscaler_zia.tunnel.destination.start.port | Phase 2 policy proposal - Destination port end. | long |
zscaler_zia.tunnel.dpd_packets | Number of DPD packets received in 60-second sample window. | keyword |
zscaler_zia.tunnel.encryption.algorithm | Encryption algorithm. | keyword |
zscaler_zia.tunnel.ike.version | IKE version (1 or 2). | long |
zscaler_zia.tunnel.life.bytes | Life bytes (number of traffic to be transacted through tunnel before renegotiation). | long |
zscaler_zia.tunnel.life.time | Lifetime of IKE Phase 1/2 in seconds. | long |
zscaler_zia.tunnel.location.name | Location name. | keyword |
zscaler_zia.tunnel.policy.direction | N/A | keyword |
zscaler_zia.tunnel.policy.protocol | Phase 2 policy proposal - Protocol. | keyword |
zscaler_zia.tunnel.protocol | IPSec tunnel protocol type (Zscaler only supports ESP). | keyword |
zscaler_zia.tunnel.source.end.ip | Phase 2 policy proposal - Source IP end. | ip |
zscaler_zia.tunnel.source.start.ip | Phase 2 policy proposal - Source IP start. | ip |
zscaler_zia.tunnel.source.start.port | Phase 2 policy proposal - Source port start. | long |
zscaler_zia.tunnel.spi | Security Parameter Index. | keyword |
zscaler_zia.tunnel.spi_in | Initiator cookie. | keyword |
zscaler_zia.tunnel.spi_out | Responder cookie. | keyword |
zscaler_zia.tunnel.type | Tunnel type. | keyword |
zscaler_zia.tunnel.vendor.name | Vendor name of the edge device. | keyword |
This is the web dataset.
An example event for web looks as following:
{
"@timestamp": "2021-12-31T08:08:08.000Z",
"agent": {
"ephemeral_id": "444ca1f4-28b9-45cb-8287-ba44516c521b",
"id": "08fc14c0-5a92-4649-93f3-68fb5d6c5fbc",
"name": "docker-fleet-agent",
"type": "filebeat",
"version": "8.6.1"
},
"data_stream": {
"dataset": "zscaler_zia.web",
"namespace": "ep",
"type": "logs"
},
"destination": {
"ip": "1.128.3.4"
},
"ecs": {
"version": "8.6.0"
},
"elastic_agent": {
"id": "08fc14c0-5a92-4649-93f3-68fb5d6c5fbc",
"snapshot": false,
"version": "8.6.1"
},
"event": {
"action": "blocked",
"agent_id_status": "verified",
"category": [
"web"
],
"dataset": "zscaler_zia.web",
"ingested": "2023-03-15T21:02:55Z",
"kind": "event",
"risk_score": 0,
"type": [
"info"
]
},
"http": {
"request": {
"bytes": 600,
"method": "CONNECT",
"mime_type": "Other",
"referrer": "None"
},
"response": {
"bytes": 65,
"status_code": 200
}
},
"input": {
"type": "http_endpoint"
},
"network": {
"protocol": "http_proxy"
},
"related": {
"hosts": [
"TestMachine35"
],
"ip": [
"1.128.3.4"
],
"user": [
"administrator1",
"test"
]
},
"rule": {
"name": "Zscaler Proxy Traffic",
"ruleset": "FwFilter"
},
"source": {
"nat": {
"ip": "1.128.3.4"
},
"user": {
"name": "administrator1"
}
},
"tags": [
"forwarded",
"zscaler_zia-web"
],
"url": {
"domain": "www.example.com",
"full": "https://www.example.com",
"original": "https://www.example.com",
"path": "",
"scheme": "https"
},
"user": {
"domain": "example.com",
"email": "test@example.com",
"name": "test"
},
"user_agent": {
"device": {
"name": "Other"
},
"name": "Other",
"original": "Windows Microsoft Windows 10 Pro ZTunnel/1.0",
"os": {
"full": "Windows 10",
"name": "Windows",
"version": "10"
}
},
"zscaler_zia": {
"web": {
"app": {
"class": "General Browsing",
"name": "General Browsing"
},
"ctime": 0,
"department": "Unknown",
"device": {
"hostname": "TestMachine35"
},
"dpl": {
"dictionaries": "None",
"engine": "None"
},
"location": "Test DB",
"malware": {
"category": "None"
},
"stime": 0,
"threat": {
"name": "None"
},
"unscannable": {
"type": "None"
},
"url": {
"category": {
"sub": "Web Search",
"super": "Information Technology"
},
"class": "Business Use"
}
}
}
}
Exported fields
| Field | Description | Type |
|---|---|---|
@timestamp | Event timestamp. | date |
data_stream.dataset | Data stream dataset. | constant_keyword |
data_stream.namespace | Data stream namespace. | constant_keyword |
data_stream.type | Data stream type. | constant_keyword |
event.dataset | Event dataset. | constant_keyword |
event.module | Event module. | constant_keyword |
input.type | Type of Filebeat input. | keyword |
log.offset | Log offset. | long |
log.source.address | Source address from which the log event was read / sent from. | keyword |
tags | User defined tags. | keyword |
zscaler_zia.web.app.class | The web application class of the application that was accessed. Equivalent to module. | keyword |
zscaler_zia.web.app.name | Cloud application name. | keyword |
zscaler_zia.web.bandwidth_throttle | Indicates whether the transaction was throttled due to a configured bandwidth policy. | keyword |
zscaler_zia.web.ctime | The time from when the first byte of the request hits the ZEN to the time in which the last byte of the response is sent from the ZEN back to the browser. | long |
zscaler_zia.web.department | Department of the user. | keyword |
zscaler_zia.web.device.hostname | The obfuscated version of the device owner. This field must be changed manually. | keyword |
zscaler_zia.web.dpl.dictionaries | The DLP dictionaries that were matched, if any. | keyword |
zscaler_zia.web.dpl.engine | The DLP engine that was matched, if any. | keyword |
zscaler_zia.web.encoded_host | Encoded version of the destination host name. | keyword |
zscaler_zia.web.file.class | Type of file associated with the transaction. | keyword |
zscaler_zia.web.file.type | Type of file associated with the transaction. | keyword |
zscaler_zia.web.location | Gateway location or sub-location of the source. | keyword |
zscaler_zia.web.malware.category | The category of malware that was detected in the transaction, if any. Also indicates if a file was submitted to the Sandbox engine for analysis and the result of the analysis. | keyword |
zscaler_zia.web.malware.class | The class of malware that was detected in the transaction, if any. | keyword |
zscaler_zia.web.record.id | N/A | keyword |
zscaler_zia.web.stime | The round trip time between the ZEN request and the server. | long |
zscaler_zia.web.threat.name | The name of the threat that was detected in the transaction, if any. | keyword |
zscaler_zia.web.total.size | Total size, in bytes, of the HTTP transaction; sum of the total request size and total response size. | long |
zscaler_zia.web.unscannable.type | Unscannable file type. | keyword |
zscaler_zia.web.url.category.sub | Category of the destination URL. | keyword |
zscaler_zia.web.url.category.super | Super category of the destination URL. | keyword |
zscaler_zia.web.url.class | Class of the destination URL. | keyword |
| Version | Details |
|---|---|
2.7.3 | Enhancement View pull request Map web login user details to ECS. |
2.7.2 | Enhancement View pull request Added categories and/or subcategories. |
2.7.1 | Enhancement View pull request Add support for dynamic ECS mapping. Bug fix View pull request Resolve the issue related to the user agent field. Bug fix View pull request Resolve the issue related to fields that contain a NA value. |
2.7.0 | Enhancement View pull request Update package to ECS 8.6.0. |
2.6.1 | Bug fix View pull request Remove duplicate fields. |
2.6.0 | Enhancement View pull request Update Aggregation visualizations to Lens, Add an on_failure processor to the convert, geo_ip, uri_parts and date processors, remove unnecessary white spaces, mapped to related ecs field and convert double quotes to single quotes. |
2.5.0 | Enhancement View pull request Update package to ECS 8.5.0. |
2.4.1 | Bug fix View pull request Remap network.protocol to valid values for web data stream. |
2.4.0 | Enhancement View pull request Update package to ECS 8.4.0 |
2.3.1 | Enhancement View pull request Updated the documentation links in README file |
2.3.0 | Bug fix View pull request Fix issue related URI parts processor. Enhancement View pull request Added support for optionally configuring secret header and secret value for cloud NSS input. |
2.2.0 | Enhancement View pull request Update package to ECS 8.3.0. |
2.1.0 | Enhancement View pull request Make GA |
2.0.0 | Enhancement View pull request Added input for Cloud NSS using HTTP Endpoint input type. |
0.2.0 | Enhancement View pull request Update ECS to 8.2 |
0.1.3 | Enhancement View pull request Updated the image file reference in README file. |
0.1.2 | Enhancement View pull request Add documentation for multi-fields. |
0.1.1 | Enhancement View pull request Updated the README to describe the Zscaler ZIA setup process in detail. |
0.1.0 | Enhancement View pull request Initial draft of the package. |