Security Information Exchange (SIE) Feed Guide
About Security Information Exchange (SIE)
The Security Information Exchange (SIE), from Farsight Security® Inc. (now part of DomainTools), is a scalable and adaptable real-time data streaming and information sharing platform. SIE collects and provides access to more than 800,000 observations per-second of raw data from its global sensor network. Farsight also applies unique and patented methods for improving usability of the data, directly sharing the refined intelligence with SIE customers and DNSDB®, one of the world’s largest passive DNS (pDNS) databases.
The diverse set of data available from SIE includes the following and is relevant and useful for practitioners in various technology roles:
- Real-time actionable insights for domains and hostnames, fully qualified domain names (FQDNs), when they are first successfully resolved on the Internet or reactivated after having not been seen for at least 10 days
- Full copies of emails sent to spamtrap email addresses, phishing URLs for malicious sites involved in phishing campaigns, and connection attempts from malware-infected systems that monitor Conficker activity
- Network traffic blocked by Intrusion Detection Systems (IDS) and firewall devices
- Passive DNS observations where the responding server returned the “NXDomain” error.
Each unique set of data in SIE is known as a feed (or channel) and the data acquired from a specific channel can be customized to meet the needs of each customer, enabling you to subscribe to and access only the channels needed to solve your problem. A channel in SIE may be the result from analyzing the data or a subset of data from other channels.
Why Passive DNS (pDNS)?
- Visiting a website? Your system uses DNS to resolve the IP address of the hostname for the website you are attempting to access
- Sending an email? Email uses DNS to resolve the IP address of the mail exchange server your message should be delivered to
DNS serves as early warning and detection solution for phishing, spam, malicious and suspicious behaviors, and other attacks. DNS intelligence is considered the only source of “ground truth” information for the Internet.
Farsight Security’s mission is to make the Internet a safer place. We provide security solutions that empower customers with meaningful and relevant intelligence. This information provides customers with insights about the network configuration of a threat and the surrounding network on the Internet for improving the value and impact of threat intelligence and research.
DNS is a critical component of Internet communication and almost all Internet transactions begin with a DNS query and response.
The Security Information Exchange (SIE), from Farsight Security Inc., is designed with privacy in mind. The passive DNS (pDNS) sensors do not collect Personally Identifiable Information (PII) from client resolvers (also known as stub) by deliberately collecting between recursive resolvers and authoritative servers.
The data from SIE enables security professionals to accurately identify, map, and protect their networks from cybercriminal activity by providing global visibility. It provides immediate access to a real-time global sensor network without the need to develop or deploy your own data collection infrastructure.
Methods to access and acquire data from SIE feeds are available using SIE Direct Connect, SIE Remote Access (SRA), SIE Batch, or AXAMD. These methods are described in the SIE Technical Reference document. Due to the technical limitations of transporting high bitrate SIE feeds across the Internet, some access methods are not available for specific SIE feeds. These restrictions are noted below.
Based on your needs, you can subscribe to an individual feed or a bundle of commonly used feeds. A Farsight sales representative or Solution Architect (SAs) can help you select the channels that will best meet your needs.
SIE Feed Guide
In the following table, the average and max bitrates (in bits per second) for each feed to indicate how much network bandwidth is required to acquire and receive data from a feed. The average and max payloads (per second) indicate the number of records that need to be processed data from a feed is acquired in real-time.
| Feed | Name | Description | Bitrate (Max) | Payloads (Max) |
| 24 | Spam-Full | Full copies of emails sent to spamtrap email addresses. | 2.4kbps (104kbps) | 2/sec (14/sec) |
| 25 | Spam-Select | Select fields from the emails sent to Feed 24 (Spam-Full). | 2kbps (105kbps) | 1/sec (3/sec) |
| 27 | Phishing URLs | PhishLabs data for malicious sites involved in phishing campaigns. | <1kbps (2kbps) | <1/sec (2/sec) |
| 42 | IDS and Firewall Log Data | ThreatStop data of blocking action from IDS and Firewall devices. | 7mbps (35mbps) | 390/sec (2kbps) |
| 80 | Conficker Sinkhole | Connection attempts from infected clients to sinkholes that monitor Conficker activity. | 385kbps (520kbps) | 210/sec (280/sec) |
| 115 | DDos Events | Evidence of DDoS and DRDoS (Distributed Reflection Denial of Service) attacks based on analysis of data based on analysis of data from captured network packets destined from unused network space. | <1kbps (<1kbps) | <1/sec (1.5/sec) |
| 207 | DNSDB De-duplicated Data | Passive DNS observations after the deduplication processing phase and immediately prior to the verification phase. | 144mbps (170mbps) | 122kbps (144kbps) |
| 208 | DNSDB Verified Data | Passive DNS observations after the verification processing phase and prior to filtering. | 88mbps (117mbps) | 66kbps (87kbps) |
| 211 | Newly Active Domains (NAD) | Domains that have been observed after having not been seen for at least 10 days. | 62kbps (900kbps) | 53/sec (760/sec) |
| 212 | Newly Observed Domains (NOD) | Passive DNS observations of base domains not previously seen when compared to the DNSDB historical database. | 3kbps (18kbps) | 2/sec (13/sec) |
| 213 | Newly Observed Hostnames (NOH) | Fully Qualified Domain Names (FQDNs) not previously seen when compared to the DNSDB historical database. | 1.6mbps(4mbps) | 1.2kbps (3.7kbps) |
| 214 | DNS Changes | Passive DNS observations where some aspect of the query or response was not found when compared to the DNSDB historical database. | 4.9mbps (8mbps) | 3kbps(5.8kbps) |
| 220 | DNS Errors | Queries where the authoritative DNS servers answered with a non-zero error code. | 570mbps (860mbps) | 207kbps (330kbps) |
| 221 | NX Domains | Passive DNS observations where the responding server returned the “NXDomain” error. | 52mbps(73mbps) | 67kbps (94kbps) |
| 255 | Heartbeat | Repeating data used for SIE health monitoring. | 1kbps (1kbps) | 1kbps (1kbps) |
Note: Quoted bitrates and payloads are representative of SIE traffic as of February 2023.
Data Formats and SIE Access Methods
Intelligence data acquired from SIE is delivered in various data formats depending on the feed and access method. Data formats follow.
| Data Format | Information |
| NMSG | Farsight’s Network Message (NMSG) Encapsulation format. See the SIE NMSG User Guide for details. |
| JSON | JavaScript Object Notation |
| NDJSON | Newline delimited JSON |
| PCAP | Packet Capture |
The data format for each feed and access method follow.
- SIE Direct Connect: NMSG unless noted below
- SIE Remote Access (SRA): NMSG unless noted below
- SIE Batch: NDJSON unless noted below
- AXAMD: JSON unless noted below
| Feed | Name | Direct Connect | SRA | SIE Batch | AXAMD |
| 24 | Spam-Full | Yes | Yes | Yes | Yes |
| 25 | Spam-Select | Yes | Yes | Yes | Yes |
| 27 | Phishing URLs | Yes | Yes | Yes | Yes |
| 42 | IDS and Firewall Log Data | Yes | Yes | Yes | No |
| 80 | Conficker Sinkhole | Yes | Yes | Yes | No |
| 115 | DDos Events | Yes | Yes | Yes | Yes |
| 207 | DNSDB De-duplicated Data | Yes | No | Yes (NMSG) | No |
| 208 | DNSDB Verified Data | Yes | No | Yes (NMSG) | No |
| 211 | Newly Active Domains | Yes | Yes | Yes | Yes |
| 212 | Newly Observed Domains (NOD) | Yes | Yes | Yes | Yes |
| 213 | Newly Observed Hostnames (NOH) | Yes | Yes | Yes | Yes |
| 214 | DNS Changes | Yes | Yes | Yes | No |
| 220 | DNS Errors | Yes | No | No | No |
| 221 | NX Domains | Yes | Yes | Yes (NMSG) | No |
| 255 | Heartbeat | Yes | Yes | No | Yes |
SIE Access Methods
Data from SIE can be accessed and acquired using the following methods:
- Direct Connect: Connect a system to the SIE network. This 1.) requires a server to be installed in a data center where Farsight has a point of presence, and 2.) then ordering a network cross connect between your server and the SIE network. Customers can optionally, and prefer to, lease a blade server from Farsight
- SIE Remote Access (SRA): Remotely connect to the SIE network using an encrypted tunnel from your workstation or a server in your local data center
- SIE Batch: Provides on-demand access for downloading data from SIE feeds using a RESTful API or web-based interface. You select the feed and duration of time you are interested in, and then download the data for analysis. The duration of available data is dependent on the feed, but is typically the most recent 12-18 hours
For additional information about SIE access methods, please see the SIE Technical Overview document.
Direct Connect
SIE Direct Connect allows a customer to physically connect a server to the Farsight SIE network for maximum data throughput. This can be done in one of two ways:
- Blade Server: Pre-configured blade servers co-located in one of Farsight’s data centers that can be leased by customers for direct access to SIE feeds
- Customer Server: Customer (owned, managed, and operated) servers that can be installed in one of Farsight’s data centers and physically connected to the SIE network with a network cross-connect
If a blade server is leased from Farsight, it will be pre-installed with the essential software components needed to acquire, process, compress, buffer, and transfer data from SIE feeds to the customer’s data center for additional analysis, enrichment, and storage.
If a customer uses their own server, an order can be submitted for a cross-connect to the SIE switches hosted at select Equinix data centers (Ashburn DC3 and Palo Alto SV8). An FSI account manager can help guide cross-connect provisioning details, hosting, or colocation options.
For additional information about SIE connection methods, please see the SIE Technical Overview document. A Farsight’s sales representatives is happy to share a copy of this document with you. This will help inform and guide you in understanding which connection method will work best for you.
SIE Remote Access (SRA)
SIE Remote Access (SRA) enables a customer to remotely connect to the Security Information Exchange (SIE) from anywhere on the Internet. SRA provides access to SIE feed data on customer’s local servers, allowing their analysis and processing systems to be located in their own data centers rather than physically co-located at a Farsight’s data center.
Due to the technical limitations of transporting high bitrate SIE feeds across the Internet, the SRA access method is not available for all SIE feeds.
SRA uses the Advanced Exchange Access (AXA) transport protocol which enables SRA sessions to perform the following:
- Select which SIE feed or feeds to monitor and acquire data from
- Define user-specified search or filtering criteria to match IP or DNS traffic
- Control rate-limits and other AXA parameters
The streaming search and filtering capabilities of AXA enables SRA to access and acquire meaningful and relevant data from SIE while avoiding the costs of transporting enormous volumes of data across the Internet.
Note: For high volume feeds accessed using SRA, it is expected that customers will specify a search or filter for IP addresses and DNS domain names or hostnames of interest. The SRA service will only collect and send data matching the specified criteria across the Internet to the customer.
SIE Batch
SIE Batch provides on-demand access for downloading data from SIE feeds using a RESTful API or web-based interface. You select the feeds and duration of time you are interested in, and then download the data for analysis. The duration of available data is dependent on the feed, but is typically the most recent 12-18 hours. SIE Batch allows you to acquire data from SIE feed using two (2) methods:
- API: Allows you to write tools to programmatically download data from SIE feeds for analysis
- Interactively: Web-based interface to the API that enables you to select and download SIE feed data on-demand
Advanced Exchange Access Middleware Daemon (AXAMD)
Farsight also provides a RESTful middleware layer in front of its AXA service. This service is called the AXA Middleware Daemon (AXAMD) and provides a RESTful capability that adds a streaming HTTP interface on top of the AXA toolkit. This enables web-application developers to interface with SIE using SRA. Farsight also published a command line tool and Python extension library called axamd_client. This toolkit is licensed under the Apache 2.0 license.
The Advanced Exchange Access (AXA) toolkit contains tools and a C library to bring Farsight’s real-time data and services directly from the Farsight Security Information Exchange (SIE) to the customers network.
Advanced Exchange Access Middleware Daemon (AXAMD) is a suite of tools and library code to bring Farsight’s real-time data and services directly from the Farsight Security Information Exchange (SIE) to the customers network.
Due to the technical limitations of transporting high bitrate SIE feeds across the Internet, the AXAMD access method is not available for all SIE feeds.
