Skip to main content
DomainTools Announces Predictive Threat Feeds - Powering Preemptive Exposure Management
Tag title

Newly Observed Domains (NOD) User Guide

Published on: 
November 19, 2022
On This Page
TOC Element
Share:

Farsight Security® Inc.'s (now a part of DomainTools) Newly Observed Domains™ (NOD) provides security teams with real-time actionable information based on the age of domain names. By using NOD, subscribers can block spam and malware from newly observed domains until security providers have an opportunity to catch up.

This document is intended for UNIX and Linux System Administrators whowant to download and use Farsight Security Newly Observed Domains (NOD)data in one or more of the following scenarios:

  1. NOD Response Policy Zones (RPZ)
  2. NOD DNS Blackhole List Zones Zones (DNSBL)

Getting Started

When choosing a Newly Observed Domains (NOD) subscription you will needto determine the format and transport you want to use to retrieve NODdata sets. At the beginning of the provisioning process for the NODsubscription you will be asked to provide one of the following dependingon what delivery methods you have chosen.

Incremental Zone Transfers (IXFR)

If you choose to use Incremental Zone Transfers (IXFR) you will need togive Farsight a list of IPv4 and IPv6 addresses to which you would likeDNS NOTIFY messages to be sent, as well as IPv4 and IPv6 addresses orsmall address blocks that will be allowed access (the two lists aretypically the same).

Please use restraint when listing IPv4 or IPv6 address ranges andlimit the requested address ranges to only those that will be used todownload the files.

Rsync

If you choose to use rsync you will need to create an SSH key pair andshare the public key with Farsight. Farsight has a strong preference for4096 bit RSA keys. Using ssh in an automated script requires that thessh key be generated with a null passphrase.

You can create such a key pair with the following command:

$ ssh-keygen -t rsa -b 4096 -N '' -C farsight_security -f /path/to/filename

Generating public/private rsa key pair.
Your identification has been saved in /path/to/filename.
Your public key has been saved in /path/to/filename.pub.
The key fingerprint is:
SHA256:r1IKQPxpp9erHZ9tAaAnllp5mIrai6N1441p0NMueOg farsight_security

You will need to share the file created by ssh-keygen with the fileextension .pub.

Note: The public SSH key can be freely shared. You do not need toencrypt your public key when sending it to us. We welcome PGP/GPG-signedemail, and/or S/MIME-signed email from customers who are comfortablecryptographically signing their email.

Retrieve DNS Zones via Incremental Zone Transfers (IXFR)

A Zone Transfer is a term used to refer to the process by which thecontents of a DNS Zone file is copied from a primary DNS Server to asecondary DNS server. IXFR is a term used to refer to a incremental zonetransfer vs a full zone transfer (AXFR).

Farsight recommends using Zone Transfers to consume NOD as it provides anear real time mechanism to retrieve NOD updates.

Note: To configure your DNS infrastructure to use Incremental DNSZone Transfers as the transport for NOD, Farsight will need a list ofIPv4 and IPv6 addresses to which you would like DNS NOTIFY messages tobe sent, as well as IPv4 and IPv6 addresses or small address blocks thatwill be allowed access (the two lists are typically the same). If youhave not already done so, please share those with Farsight so youraccount can be configured.

You will receive a DNS TSIG key, which will look like the following:

key "FSI-####-#-key" {
algorithm HMAC-SHA512;
secret “SECRET”;
}

The zones are served by the following masters. This list is subject tochange as Farsight grows the service over time and you will be givenreasonable notice to reconfigure your name server.

masters “fsi-ixfr-masters” {
104.244.13.88 key “FSI-####-#-key”;
104.244.14.88 key “FSI-####-#-key”;
2620:11c:f004::88 key “FSI-####-#-key”;
2620:11c:f008::88 key “FSI-####-#-key”;
};

These settings are attached as named.fsi-####-#.conf where ####-# isyour account number. Farsight recommends that you use this file withBIND's include statement in your configuration to simplify futureupdates. Also included will be an example configuration that you mayincorporate into your existing name server's configuration.

You will need to add rules to your firewall's access control list(s) forFarsight hosts to send UDP packets to port 53 of your DNS server so thatit can receive the DNS NOTIFY packets for updates. This will allow yourDNS server to receive incremental updates every few seconds. If you donot add these firewall rules your zones will only update every fewminutes as per the refresh field in the zone's Start of Authorityrecord.

Using TSIGs for authentication requires reasonably synchronized systemclocks. Ensure that your server is enabled to use NTP for clocksynchronization.

NOD Response Policy Zones (RPZ)

Farsight Security makes NOD available via Domain Name Service Response Policy Zones (DNS RPZ). DNS RPZ is a method that allows a name server administrator to overlay custom information on top of the global DNS to provide alternate responses to queries. You can read more about DNS RPZ at DNS Response Policy Zone

Response Policy Zones are delivered as one of seven DNS zones suitablefor deployment as a DNS Response Policy Zone. Each zone file namecontains a time which corresponds to how old the domains in the zone arebelieved to be. Farsight recommends the use of the 3h.rpz.dns-nod.netzone as a starting point. You will receive DNS NOTIFY messages for allseven zones but the preference is that you only download the zones thatyou will actively use, to avoid duplication:

  • 5m.rpz.dns-nod.net
  • 10m.rpz.dns-nod.net
  • 30m.rpz.dns-nod.net
  • 1h.rpz.dns-nod.net
  • 3h.rpz.dns-nod.net
  • 12h.rpz.dns-nod.net
  • 24h.rpz.dns-nod.net

The domains included in the seven DNS zones correspond to the age of the domain as thenames were first observed by our sensor network. Think of RPZ as various rolling windowfrom five minutes to 24 hours. In other words, newly observed domains (NOD) are bucketedby age and formed into seven different zone files.

When a NOD is first observed, it is inserted into all seven zone files and as the domainages, our systems remove the name from the appropriate RPZ file starting with the 5m filefirst rolling up through the files to the last 24h file. A once newly observed domain willultimately age out and it will no longer be included/found in any of the RPZ zone files.

Farsight does not support an rpz files with domains older than 24 hours old.

In the meantime, here is a link that includes additional information about DNSD ResponsePolicy Zones that you might find helpful:

Further technical information may be found in the NOD Technical Data Sheet and NOD UserDocumentation.

Example config files below, see the welcome email for personalized examples.

File named.fsi-####-#.conf: Note: You'll want to make sure the name ofthis file matches the name of the file you get in the welcome email.

key “FSI-####-#-key” {
algorithm HMAC-SHA512
secret “SECRET
};

masters “fsi-ixfr-masters” {
104.244.13.88 key “FSI-####-#-key”;
104.244.14.88 key “FSI-####-#-key”;
2620:11c:f004::88 key “FSI-####-#-key”;
2620:11c:f008::88 key “FSI-####-#-key”;
};

File named.fsi-nod.conf

// add this to your options clause
options {
response-policy {
zone “3h.rpz.dns-nod.net” policy given;
// zone “3h.rpz.dns-nod.net” policy passthru; # audit with logging
};
};


// optionally, add something like this to your logging clause and send
// to your SIEM

logging {
channel named-rpz {
file “/var/log/rpz.log” versions 3 size 250k;
print-time yes;
print-category yes;
print-severity yes;
severity info;
};

category rpz {
named-rpz;
};

};

// include the keyfile we created above
include “/etc/bind/named.fsi-####-#.conf”;

zone “3h.rpz.dns-nod.net” {
type slave;
file “3h.rpz.dns-nod.net.zone”;
masters { fsi-ixfr-masters; };
allow-query {localhost;};
allow-transfer {none;};
};

NOD DNS Blackhole List Zone (DNSBL)

Farsight Security makes NOD available in DNS-based Blackhole List(DNSBL) Zone file format. DNSBLs convey information over DNS and allowsubsequent processes make decisions based on the provided answers.DNSBLs are most commonly used to assist in the scoring of SPAM email.

Using the named.fsi-####-#.conf configuration file as described above;you can use the following snip-it in your name server configuration fileto consume the NOD DNSBL file via IXFR.

include “/etc/bind/named.fsi-####-#.conf”;

zone “v1.bl.dns-nod.net” {
type slave;
file “v1.bl.dns-nod.net.zone”;
masters { fsi-ixfr-masters; };
allow-query {localhost;};
allow-transfer {none;};
};

Using dig for troubleshooting IXFR

IXFR depends on retrieving the SOA (Start Of Authority) record for thezone: this record contains a serial number, and the first step in theIXFR process is to compare the serial number with what the recipientserver has locally. In practice, many connectivity issues can beunderstood by looking at what happens when a request is made to retrievethe SOA record for the zone.

The basic command looks like this:

dig -y '<hashing algorithm>:<your key name>:<your secret>' @<one of our servers> \
<one of our zones> SOA

Using the example data given above:

dig -y 'HMAC-SHA512:FSI-####-#-key:NOT=REALLY=YOUR=SECRET' @104.244.13.88 v1.bl.dns-nod.net SOA

; <<>> DiG 9.8.3-P1 <<>> -y HMAC-SHA512 @104.244.13.88 v1.bl.dns-nod.net SOA
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 14186
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;v1.bl.dns-nod.net. IN SOA

;; ANSWER SECTION:
v1.bl.dns-nod.net. 86400 IN SOA a.ns.dns-nod.net.v1.bl.dns-nod.net.
nod-admin.fsi.io. 1479754872 600 300 86400 300

;; AUTHORITY SECTION:
v1.bl.dns-nod.net. 86400 IN NS a.ns.dns-nod.net.
v1.bl.dns-nod.net. 86400 IN NS b.ns.dns-nod.net.

;; TSIG PSEUDOSECTION:
fsi-####-#-key. 0 ANY TSIG hmac-sha512. 1479754882 300 64
NOT=REALLY=YOUR=SECRET 14186 NOERROR 0

;; Query time: 71 msec
;; SERVER: 104.244.13.88#53(104.244.13.88)
;; WHEN: Mon Nov 21 11:01:22 2016
;; MSG SIZE rcvd: 258

That shows a successful response. The two most important pieces of
information are:

  • the query status shown on the HEADER line
  • the TSIG section which will show any key-related issues

and of course, that you got a response at all. (If you're sniffing
packets, it's a lot easier to look for the SOA query than to try to
recognize a zone transfer which probably spans multiple packets.)

Retrieve DNS Zones via Rsync

NOD RPZ files can be retrieved via Rsync when DNS Zone Transfers are not
desired. Rsync is a file copying tool, it is known for its
delta-transfer algorithm, which reduces the amount of data sent over the
network by sending only the differences between the source files and the
existing files at the destination.

NOD Response Policy Zones (RPZ) Starting Recommendation

Farsight recommends the use of the 3h.rpz.dns-nod.net zone as a starting
point. There are separate files based on the age of the domain names.
Please only download the file for the zone that you will actively use.

path - filename:

  • nod/rpz/5m.rpz.dns-nod.net.zone
  • nod/rpz/10m.rpz.dns-nod.net.zone
  • nod/rpz/30m.rpz.dns-nod.net.zone
  • nod/rpz/1h.rpz.dns-nod.net.zone
  • nod/rpz/3h.rpz.dns-nod.net.zone
  • nod/rpz/12h.rpz.dns-nod.net.zone
  • nod/rpz/24h.rpz.dns-nod.net.zone

Example shell command to download the three hour NOD RPZ Zone file to
the directory /srv/nod:

rsync -az -e "ssh -p 49222 –i/path/to/sshkey" \
USERNAME@rsync.dns-nod.net:nod/rpz/3h.rpz.dns-nod.net.zone /srv/nod/

The following is an example BIND configuration for RPZ:

options {
response-policy { zone "3h.rpz.dns-nod.net"; };
};

zone "3h.rpz.dns-nod.net" {
type master;
file "3h.rpz.dns-nod.net.zone";
allow-query {localhost;};
allow-transfer {none;};
allow-update {none;};
};

Example shell script to be run by cron to download and load the three hour NOD RPZ zone file using BIND utility rndc. If your operating system distribution includes the flock - manage locks from shell scripts command, using it will prevent multiple cronned instances from overrunning each other (its semantics are similar to time ):

#!/bin/bash

rsync -az -e "ssh -p 49222 –i/path/to/sshkey" \
USERNAME@rsync.dns-nod.net:nod/rpz/3h.rpz.dns-nod.net.zone /srv/nod/

rndc reload /srv/nod/3h.rpz.dns-nod.net

Using NOD RPZ this way will cause DNS queries for any domains in the
selected database to return an NXDOMAIN response, should those domains
be queried by an end-user.

NOD DNS Blackhole List Zone (DNSBL) Availability

Farsight Security makes NOD available in DNS-based Blackhole List
(DNSBL) Zone file format. DNSBLs convey information over DNS and allow
subsequent processes make decisions based on the provided answers.
DNSBLs are most commonly used to assist in the scoring of SPAM email.

Farsight NOD DNSBL Zones can be loaded and served by Rbldnsd to
applications like Spamassassin and Postfix.

Your user name on the Farsight server will be "USERNAME".

Example shell command to download the one hour NOD DNSBL Zone file to
the directory /srv/nod:

rsync -az -e "ssh -p 49222 –i/path/to/sshkey" USERNAME@rsync.dns-nod.net:v1/nod.rbldnsd /srv/nod/

Use Rbldnsd to serve the NOD DNSBL Zone

*Rbldnsd (http://www.corpit.ru/mjt/rbldnsd.html> is a small
authoritate-only DNS name server designed to serve DNS-based blocklists
(DNSBLs).

Configure Rbldnsd to load the NOD DNSBL Zone

The following is an example of the Rbldnsd command-line arguments

Starting Rbldnsd:

rbldnsd -b 127.0.0.1/5053 -r /srv/nod/ v1.bl.dns-nod.net:dnset:nod.rbldnsd -p /var/run/rbldnsd.pid

Configuring BIND to forward DNSBL queries to Rbldnsd

Sample BIND configuration:

zone "v1.bl.dns-nod.net" IN {
type forward;
forward first;
forwarders {
127.0.0.1 port 5053;
};
};

Using the NOD DNSBL

Many applications can be configured to use a DNSBL. This user guide
gives guidance on two open source applications that can be configured to
consume a DNSBL. (SpamAssassin and Postfix)

When using NOD in this manner, DNS queries for A type (IP address)
resource records of the form domain.v1.bl.dns-nod.net will return an
address that indicates the age of domain if it is in the NOD database.
The response should be interpreted as follows:

PeriodResponse0-5 minute127.0.0.25-10 minute127.0.0.310-30 minute127.0.0.430-60 minute127.0.0.51-3 hours127.0.0.63-12 hours127.0.0.712-24 hours127.0.0.8

“NXDOMAIN” (no such domain) response will be returned if the domain is
not in the NOD database.

TXT DNS queries will return more detailed information about the domain
if it is in the database. For example:

$ dig +short svetlanovskiy.accountant.v1.bl.dns-nod.net
127.0.0.2$ dig +short svetlanovskiy.accountant.v1.bl.dns-nod.net txt
"first_seen=1461953815"

The first_seen date is in Unix seconds. That value can be converted to
"human time" with a command such as:

$ date -r 1461953815
Fri Apr 29 11:16:55 PDT 2016

You may test your setup using your favorite DNS query tool and the
special test domains test.dns-nod.net and invalid.dns-nod.net.

$ host test.dns-nod.net.v1.bl.dns-nod.net
test.dns-nod.net.v1.bl.dns-nod.net has address 127.0.0.2$ host invalid.dns-nod.net.v1.bl.dns-nod.net
Host invalid.dns-nod.net.v1.bl.dns-nod.net not found: 3(NXDOMAIN)

Configure SpamAssassin to use NOD DNSBL

To configure SpamAssassin to use NOD DNSBL, append the following to
/etc/spamassassin/local.cf or ~/.spamassassin/user_prefs, adjusting
scores to taste:

urirhssub URIBL_NOD_5M v1.bl.dns-nod.net. A 127.0.0.2
body URIBL_NOD_5M eval:check_uridnsbl('URIBL_NOD_5M')
describe URIBL_NOD_5M Contains an URL that is only 5 minutes old in
Farsight Passive DNS
tflags URIBL_NOD_5M net domains_only
score URIBL_NOD_5M 5

urirhssub URIBL_NOD_10M v1.bl.dns-nod.net. A 127.0.0.3
body URIBL_NOD_10M eval:check_uridnsbl('URIBL_NOD_10M')
describe URIBL_NOD_10M Contains an URL that is only 10 minutes old in
Farsight Passive DNS
tflags URIBL_NOD_10M net domains_only
score URIBL_NOD_10M 5

urirhssub URIBL_NOD_30M v1.bl.dns-nod.net. A 127.0.0.4
body URIBL_NOD_30M eval:check_uridnsbl('URIBL_NOD_30M')
describe URIBL_NOD_30M Contains an URL that is only 30 minutes old in
Farsight Passive DNS
tflags URIBL_NOD_30M net domains_only
score URIBL_NOD_30M 5

urirhssub URIBL_NOD_1H v1.bl.dns-nod.net. A 127.0.0.5
body URIBL_NOD_1H eval:check_uridnsbl('URIBL_NOD_1H')
describe URIBL_NOD_1H Contains an URL that is only one hour old in
Farsight Passive DNS
tflags URIBL_NOD_1H net domains_only
score URIBL_NOD_1H 5

urirhssub URIBL_NOD_3H v1.bl.dns-nod.net. A 127.0.0.6
body URIBL_NOD_3H eval:check_uridnsbl('URIBL_NOD_3H')
describe URIBL_NOD_3H Contains an URL that is only three hours old in
Farsight Passive DNS
tflags URIBL_NOD_3H net domains_only
score URIBL_NOD_3H 5

urirhssub URIBL_NOD_12H v1.bl.dns-nod.net. A 127.0.0.7
body URIBL_NOD_12H eval:check_uridnsbl('URIBL_NOD_12H')
describe URIBL_NOD_12H Contains an URL that is only twelve hours old in
Farsight Passive DNS
tflags URIBL_NOD_12H net domains_only
score URIBL_NOD_12H 5

urirhssub URIBL_NOD_24H v1.bl.dns-nod.net. A 127.0.0.8
body URIBL_NOD_24H eval:check_uridnsbl('URIBL_NOD_24H')
describe URIBL_NOD_24H Contains an URL that is only 24 hours old in
Farsight Passive DNS
tflags URIBL_NOD_24H net domains_only
score URIBL_NOD_24H 5

Configure Postfix to use NOD DNSBL

For postfix to reject everything:

reject_rhsbl_sender = v1.bl.dns-nod.net

To cut off at a particular age threshold using postfix version 2.8 or higher:

reject_rhsbl_sender = v1.bl.dns-nod.net=127.0.0.[2-8]

The individual domain age ranges must be explicitly listed when using
postfix versions 2.1 through 2.7:

reject_rhsbl_sender = v1.bl.dns-nod.net=127.0.0.2,…,v1.bl.dns- nod.net=127.0.0.8

NOD and Local Firewalls

If you need to configure your firewalls to allow access to the NOD systems at Farsight,
please be aware that servers may be rotated in and out of service without notice, so we
recommend you do not configure access to specific IP addresses. You should configure to
allow access through 104.244.12.0/22 to be able to access NOD services without
interruption.

Additional Information