Tag title

Security Information Exchange (SIE) Newly Active Domains

Published on:

November 7, 2022

On This Page

Channel 211, the SIE Newly Active Domains channel is a source of DNS intelligence for previously seen domains observed in channel 204 after 10 days of inactivity. This enables customers to observe and monitor known domains that have been inactive and then notified when they become active again.

Newly Active Domains is one of several channels that tracks domain observations, creation, and changes. These channels follow:

Channel 211: Newly Active Domains: Previously seen domains observed in channel 204 after 10 days of inactivity
Channel 212: Newly Observed Domains (NOD): Base domains¹ that have never been observed in DNSDB
Channel 213: Newly Observed Hostnames (NOH): Hostnames, also known as Fully Qualified Domain Names (FQDNs), that have never been observed in DNSDB. Both the RRname (left-side) and Rdata (right-side) of a DNS resource record (RR) are checked
Channel 214: DNS Changes: Domains, hostnames, or record data that is unknown to DNSDB, either because the data is for a new domain or hostname or because the record data for a domain or hostname has changed. These changes may include new RR types, new or changed IP addresses, or a change in the authoritative name servers for a domain

¹ A base domain is one label followed by a suffix. See the Public Suffix List for information on the current list of official suffixes. Suffixes are a superset of the Top Level Domains (TLDs).

These channels use Channel 204 Processed DNS Data, that is used by DNSDB, astheir authoritative data source. The DNS data available from channel 204 isafter the deduplication and verification phases from the Passive DNS Processing"Waterfall" Model. Domains and hostnames are checked for historic observationsin DNSDB back to June 2010.

For more information about Channel 204 Processed DNS Data, please refer to the SIE Technical Overview guide.

About Security Information Exchange (SIE)

The Security Information Exchange (SIE), from Farsight Security® Inc., (now a part of DomainTools) is a scalable and adaptable real-time data streaming and information sharing platform. SIE collects and provides access to more than 200,000 observations per-second of raw data from its global sensor network. Farsight also applies unique and proprietary methods for improving usability of the data, directly sharing the refined intelligence with SIE customers and DNSDB®, one of the world's largest passive DNS (pDNS) databases.

The diverse set of data available from SIE includes the following and isrelevant and useful for practitioners in various technology roles:

Raw and processed passive DNS data
Darknet/darkspace telescope data
SPAM sources and URLs
Phishing URLs and associated targeted brands
Connection attempts from malware-infected systems (as seen by a sinkhole)
Network traffic blocked by Intrusion Detection Systems (IDS) and firewall devices

Each unique set of data in SIE is known as a channel and the data acquiredfrom a specific channel can be customized to meet the needs of each customer,enabling you to subscribe to and access only the channels needed to solve yourproblem. A channel in SIE may be the result from analyzing the data or a subsetof data from other channels.

Why Passive DNS (pDNS)?

DNS is a critical component of Internet communication and almost all Internettransactions begin with a DNS query and response.

Visiting a website? Your system uses DNS to resolve the IP address of the hostname for the website you are attempting to access
Sending an email? Email uses DNS to resolve the IP address of the mail exchange server your message should be delivered to

DNS serves as early warning and detection solution for phishing, spam, maliciousand suspicious behaviors, and other attacks. DNS intelligence is considered theonly source of "ground truth" information for the Internet.

Passive DNS (pDNS) begins with raw DNS traffic that is observed and collected bypassive DNS sensors and contributed to Farsight's Security Information Exchange(SIE) by pDNS sensor operators. Once the data is sent to SIE, the data thenpasses through a series of processing phases:

Deduplication: Channel 207, DNSDB Deduplicated Data
Verification: Channel 208, DNSDB Verified Data
Filtering: Channel 204, Processed DNS Data (which used by DNSDB)

The end result is the highest-quality and most comprehensive passive DNSdatabase, DNSDB, of its kind-with more than 100 billion unique DNS resourcerecords since 2010.

Farsight Security's mission is to make the Internet a safer place. We providesecurity solutions that empower customers with meaningful and relevantintelligence. This information provides customers with insights about thenetwork configuration of a threat and the surrounding network on the Internetfor improving the value and impact of threat intelligence and research.

The Security Information Exchange (SIE), from Farsight Security Inc., isdesigned with privacy in mind. The passive DNS (pDNS) sensors do not collectPersonally Identifiable Information (PII) from client resolvers (also known asstub) by deliberately collecting between recursive resolvers and authoritativeservers.

The data from SIE enables security professionals to accurately identify, map,and protect their networks from cybercriminal activity by providing globalvisibility. It provides immediate access to a real-time global sensor networkwithout the need to develop or deploy your own data collection infrastructure.

About SIE Newly Active Domains

Channel 211 provides insights about DNS activity for previously observed basedomains seen in channel 204 after 10 days of inactivity. If DNS activity meetsthis criteria, the intelligence is sent to channel 211.

A base domain is one label followed by a suffix and a domain that users can register with a registrar. See the Public Suffix List for information on the current list of official suffixes. Suffixes are a superset of the Top Level Domains (TLDs). For example, farsightsecurity.com is a "base domain".

When is DNS intelligence for Newly Active Domains sent to channel 211?

The following narrative will inform and guide you in understanding when DNSresource record data is sent to one of the channels that tracks domainobservations, creation, and changes.

Suppose you are creating a new website. When you register a new domain for the website, that DNS registration will be sent to channel 214 DNS Changes.
When you start to create and test the website, accessing it will generate DNS requests. These DNS requests will be sent to channel 212 Newly Observed Domains (NOD) and channel 213 Newly Observed Hostnames (NOH) as the first time the domain or hostname was observed being resolved.
When you are done creating and testing the website, but before it is announced to the public and starts being used, activity on the website and associated DNS requests will stop being observed. If the duration of inactivity is more than 10 days for the domain and then a DNS request is observed, that information will be sent to channel 211 Newly Active Domains. If the duration between DNS requests is less than 10 days, it is considered active and no information is sent.

For a good introduction into the opportunities and possibilities of these channels, see New (and Newly-Changed) Fully Qualified Domain Names (FQDNs): A View of Worldwide Changes to the Internet's DNS from Black Hat Europe, 2015.

Use Cases for SIE Newly Active Domains

Newly Active Domains is an important tool in recognizing potential bad actor domains and enables customers to observe and monitor known domains that have been inactive and then notified when they become active again.

In most environments, domains and hostnames are created and staged long beforethey "light up" and become active. With malware, ransomware, and othercriminal activities, domains often become active shortly after registration andcreation. By monitoring and tracking this activity, it is possible to identifydomains and hostnames used by bad actors and block access to them before theycan be used to cause malicious actions or problems in your environment.

Malware and other bad actor domains and hostnames often have two behaviors:

Domain is registered / created and then shortly followed by DNS activity
Dormant domain becomes active after a period of inactivity

The first behavior can be easily recognized when a domain is observed in the Channel 212 Newly Observed Domains (NOD) and then observed after 10 days of inactivity and is sent to Channel 211 Newly Active Domains This type of activity can be an indicator that site is risky.

The second behavior is less clear, since it can be an indicator of risky activity or a legitimate site that is being brought live. Channel 211 Newly Active Domains can inform customers about idle to active behaviors, but further may be analysis needed to understand whether the activity is risky

Organizations have utilized DNS intelligence from Newly Observed Domains (NOD) and Newly Active Domains and implemented countermeasures with the following criteria. If domains are first observed in NOD and then observed in Newly Active Domains, then a short-term block known as a "greylist" is implemented to prevent access to the suspicious domains.

While the suspicious domains are greylisted, teams analyze and validate todetermine whether they are legitimate or risky. Once analysis is complete,access to a domain is either permitted or it is added to a block list and accessis denied.

Implementing this type of countermeasure could prevent email for the suspiciousdomains from being received, may deny and/or reject connection attempts to awebsite, or both. There is rarely an urgent need to access a new website andgreylists can help mitigate the risk of a new or unknown domain.

Channel 211 Newly Active Domains is an essential tool that enables you to identify domains that were previously observed, dormant, and then become active again. This channel empowers an organization to create policies and decide whether to permit or deny access to the domains and how long access to should be prevented.

Channel Information for SIE Channel 211

Channel NameNewly Active DomainsChannel Number211DescriptionDomains that have been observed after having not been seen for at least 10 days.SchemaSIE:newdomain

To see current channel traffic volumes and service options for accessing it, please see the Security Information Exchange (SIE) Channel Guide.

Data Format for SIE Channel 211

The Newly Active Domains channel data uses the SIE NMSG newdomain schema. DNS Query and Response resource record schema that observes and collects data returned from a query. The data available from this channel contains NMSG SIE:newdomain type messages that include the following fields:

The NMSG header includes the following fields:

KEYVALUEdomainDomain name of the query observed by pDNS.time_seenTime that pDNS observed the base domain after 10 days of inactivity.rrnameDomain name of the query observed by pDNS.rrclassRR CLASS is always "Internet (IN)", which is decimal value "1".rrtypeRR TYPE describes the type of RR, e.g., A(1), NS(2), CNAME(5).rdataData that describes the RR type, returned as an array.keysAlways empty or null.new_rrAlways empty or null.

Note: Time-based strings are in the YYYY-MM-DD HH:MM:SS format. The month"MM" starts at 01 for January and ends with 12 for December. The hours"HH" are 00-23, and minutes "MM" and seconds "SS" are 00-59. Thetimes are recorded at UTC (GMT) and daylight savings time (DST) is notapplicable.

Example Message from SIE Channel 211

Data acquired from Channel 211 Newly Active Domains is returned in NMSG format when using the Direct Connect or SIE Remote Access (SRA) access methods. NMSG is an adaptable container format that allows for consistent or variable message types. If data is downloaded for Channel 211 using SIE Batch, the data is already delivered in ND-JSON format, and the nmsgtool step below can be skipped.

The nmsgtool program is a tool for acquiring a variety of different inputs,like data streams from the network, capturing data from network interfaces,reading data from files, or even standard input and making NMSG payloadsavailable to one or more outputs. The nmsgtool program can acquire data fromSIE Channel 211 and convert it to a ND-JSON (newline-delimited JSON) text formatfor display or additional processing and analysis. nmsgtool is a programwritten by Farsight and released as open source.

See the following pages for instructions on how to install software packages fora specific distribution.

After data for Channel 211 has been acquired, written, and saved to a file, youneed to decode it to ND-JSON using nmsgtool. The [-r ch211_nad.nmsg] optiontells nmsgtool to read binary NMSG data from a file, [-c 1] limits theoutput to single NMSG payload, and [-J -] displays the record in ND-JSONformat to stdout, which is typically the screen.

$ nmsgtool -r ch211_nad.nmsg -c 1 -J - {"time":"2020-03-12 16:13:03.567192526","vname":"SIE","mname":"newdomain", "message":{"domain":"example.com.","time_seen":"2020-03-12 15:39:45", "rrname":"example.com.","rrclass":"IN","rrtype":"NS", "rdata":["nsg1.example.com.","nsg2.example.com."],"keys":[],"new_rr":[]}}

Once the data has been formatted to ND-JSON, a record from the Newly ActiveDomains channel will look similar to the following. The followingoutput can be sent to another tool for additional processing.

{"time":"2020-03-12 16:13:03.567192526","vname":"SIE","mname":"newdomain", "message":{"domain":"example.com.","time_seen":"2020-03-12 15:39:45", "rrname":"example.com.","rrclass":"IN","rrtype":"NS", "rdata":["nsg1.example.com.","nsg2.example.com."],"keys":[],"new_rr":[]}}

If you want to display a pretty-printed output of ND-JSON formatted records, we recommend using jq, a lightweight and flexible command-line JSON processor. The open source software package is available on Debian and can be installed using $ sudo apt-get install jq. The output from nmsgtool in JSON format [-J -] can be piped to jq using the following:

$ nmsgtool -r ch211_nad.nmsg -c 1 -J - | jq -r '.' { "time": "2020-03-12 16:13:03.567192526", "vname": "SIE", "mname": "newdomain", "message": { "domain": "example.com.", "time_seen": "2020-03-12 15:39:45", "rrname": "example.com.", "rrclass": "IN", "rrtype": "NS", "rdata": [ "nsg1.example.com.", "nsg2.example.com." ], "keys": [], "new_rr": [] } }

SIE Access Methods

Data from SIE can be accessed and acquired using the following methods:

Direct Connect: Connect a system to the SIE network. This 1.) requires a server to be installed in a data center where Farsight has a point of presence, and 2.) then ordering a network cross connect between your server and the SIE network. Customers can optionally, and prefer to, lease a blade server from Farsight
SIE Remote Access (SRA): Remotely connect to the SIE network using an encrypted tunnel from your workstation or a server in your local data center
SIE Batch: Provides on-demand access for downloading data from SIE channels using a RESTful API or web-based interface. You select the channel and duration of time you are interested in, and then download the data for analysis. The duration of available data is dependent on the channel, but is typically the most recent 12-18 hours

For additional information about SIE access methods, please see the SIETechnical Overview document.

Direct Connect

SIE Direct Connect allows a customer to physically connect a server to theFarsight SIE network for maximum data throughput. This can be done in one oftwo ways:

Blade Server: Pre-configured blade servers co-located in one of Farsight's data centers that can be leased by customers for direct access to SIE channels
Customer Server: Customer (owned, managed, and operated) servers that can be installed in one of Farsight's data centers and physically connected to the SIE network with a network cross-connect

If a blade server is leased from Farsight, it will be pre-installed with theessential software components needed to acquire, process, compress, buffer, andtransfer data from SIE channels to the customer's data center for additionalanalysis, enrichment, and storage.

If a customer uses their own server, an order can be submitted for across-connect to the SIE switches hosted at select Equinix data centers (AshburnDC3 and Palo Alto SV8). An FSI account manager can help guide cross-connectprovisioning details, hosting, or colocation options.

For additional information about SIE connection methods, please see the SIETechnical Overview document. A Farsight's sales representatives is happy toshare a copy of this document with you. This will help inform and guide you inunderstanding which connection method will work best for you.

SIE Remote Access (SRA)

SIE Remote Access (SRA) enables a customer to remotely connect to the SecurityInformation Exchange (SIE) from anywhere on the Internet. SRA provides accessto SIE channel data on customer's local servers, allowing their analysis andprocessing systems to be located in their own data centers rather thanphysically co-located at a Farsight's data center.

Due to the technical limitations of transporting high bitrate SIE channels across the Internet, the SRA access method is not available for all SIE channels. Please reference the SIE Channel Guide for channels that can be accessed using SRA.

SRA uses the Advanced Exchange Access (AXA) transport protocol which enables SRAsessions to perform the following:

Select which SIE channel or channels to monitor and acquire data from
Define user-specified search or filtering criteria to match IP or DNS traffic
Control rate-limits and other AXA parameters

The streaming search and filtering capabilities of AXA enables SRA to access andacquire meaningful and relevant data from SIE while avoiding the costs oftransporting enormous volumes of data across the Internet.

Note: For high volume channels accessed using SRA, it is expected thatcustomer's will specify a search or filter for IP addresses and DNS domain namesor hostnames of interest. The SRA service will only collect and send datamatching the specified criteria across the Internet to the customer.

SIE Batch

SIE Batch provides on-demand access for downloading data from SIE channels usinga RESTful API or web-based interface. You select the channel and duration oftime you are interested in, and then download the data for analysis. Theduration of available data is dependent on the channel, but is typically themost recent 12-18 hours. SIE Batch allows you to acquire data from SIE channelusing two (2) methods:

API: Allows you to write tools to programmatically download data from SIE channels for analysis
Interactively: Web-based interface to the API that enables you to select and download SIE channel data on-demand

Advanced Exchange Access Middleware Daemon (AXAMD)

Farsight also provides a RESTful middleware layer in front of its AXA service. This service is called the AXA Middleware Daemon (AXAMD) and provides a RESTful capability that adds a streaming HTTP interface on top of the AXA toolkit. This enables web-application developers to interface with SIE using SRA. Farsight also published a command line tool and Python extension library called axamd_client. This toolkit is licensed under the Apache 2.0 license.

The Advanced Exchange Access (AXA) toolkit contains tools and a C library to bring Farsight's real-time data and services directly from the Farsight Security Information Exchange (SIE) to the customers network.

Advanced Exchange Access Middleware Daemon (AXAMD) is a suite of tools andlibrary code to bring Farsight's real-time data and services directly from theFarsight Security Information Exchange (SIE) to the customers network.

Due to the technical limitations of transporting high bitrate SIE channelsacross the Internet, the AXAMD access method is not available for all SIEchannels.