User Guides

Newly Observed Domains (NOD) User Guide

Share this entry

Farsight Security® Inc.’s (now a part of DomainTools) Newly Observed Domains™ (NOD) provides security teams with real-time actionable information based on the age of domain names. By using NOD, subscribers can block spam and malware from newly observed domains until security providers have an opportunity to catch up.

This document is intended for UNIX and Linux System Administrators who want to download and use Farsight Security Newly Observed Domains (NOD) data in one or more of the following scenarios:

  1. NOD Response Policy Zones (RPZ)
  2. NOD DNS Blackhole List Zones Zones (DNSBL)

Getting Started

When choosing a Newly Observed Domains (NOD) subscription you will need to determine the format and transport you want to use to retrieve NOD data sets. At the beginning of the provisioning process for the NOD subscription you will be asked to provide one of the following depending on what delivery methods you have chosen.

Incremental Zone Transfers (IXFR)

If you choose to use Incremental Zone Transfers (IXFR) you will need to give Farsight a list of IPv4 and IPv6 addresses to which you would like DNS NOTIFY messages to be sent, as well as IPv4 and IPv6 addresses or small address blocks that will be allowed access (the two lists are typically the same).

Please use restraint when listing IPv4 or IPv6 address ranges and limit the requested address ranges to only those that will be used to download the files.

Rsync

If you choose to use rsync you will need to create an SSH key pair and share the public key with Farsight. Farsight has a strong preference for 4096 bit RSA keys. Using ssh in an automated script requires that the ssh key be generated with a null passphrase.

You can create such a key pair with the following command:

$ ssh-keygen -t rsa -b 4096 -N '' -C farsight_security -f /path/to/filename

Generating public/private rsa key pair.
Your identification has been saved in /path/to/filename.
Your public key has been saved in /path/to/filename.pub.
The key fingerprint is:
SHA256:r1IKQPxpp9erHZ9tAaAnllp5mIrai6N1441p0NMueOg farsight_security

You will need to share the file created by ssh-keygen with the file extension .pub.

Note: The public SSH key can be freely shared. You do not need to encrypt your public key when sending it to us. We welcome PGP/GPG-signed email, and/or S/MIME-signed email from customers who are comfortable cryptographically signing their email.

Retrieve DNS Zones via Incremental Zone Transfers (IXFR)

A Zone Transfer is a term used to refer to the process by which the contents of a DNS Zone file is copied from a primary DNS Server to a secondary DNS server. IXFR is a term used to refer to a incremental zone transfer vs a full zone transfer (AXFR).

Farsight recommends using Zone Transfers to consume NOD as it provides a near real time mechanism to retrieve NOD updates.

Note: To configure your DNS infrastructure to use Incremental DNS Zone Transfers as the transport for NOD, Farsight will need a list of IPv4 and IPv6 addresses to which you would like DNS NOTIFY messages to be sent, as well as IPv4 and IPv6 addresses or small address blocks that will be allowed access (the two lists are typically the same). If you have not already done so, please share those with Farsight so your account can be configured.

You will receive a DNS TSIG key, which will look like the following:

  key "FSI-####-#-key" {
    algorithm HMAC-SHA512;
    secret “SECRET”;
}

The zones are served by the following masters. This list is subject to change as Farsight grows the service over time and you will be given reasonable notice to reconfigure your name server.

  masters “fsi-ixfr-masters” {
    104.244.13.88 key “FSI-####-#-key”;
    104.244.14.88 key “FSI-####-#-key”;
    2620:11c:f004::88 key “FSI-####-#-key”;
    2620:11c:f008::88 key “FSI-####-#-key”;
  };

These settings are attached as named.fsi-####-#.conf where ####-# is your account number. Farsight recommends that you use this file with BIND’s include statement in your configuration to simplify future updates. Also included will be an example configuration that you may incorporate into your existing name server’s configuration.

You will need to add rules to your firewall’s access control list(s) for Farsight hosts to send UDP packets to port 53 of your DNS server so that it can receive the DNS NOTIFY packets for updates. This will allow your DNS server to receive incremental updates every few seconds. If you do not add these firewall rules your zones will only update every few minutes as per the refresh field in the zone’s Start of Authority record.

Using TSIGs for authentication requires reasonably synchronized system clocks. Ensure that your server is enabled to use NTP for clock synchronization.

NOD Response Policy Zones (RPZ)

Farsight Security makes NOD available via Domain Name Service Response Policy Zones (DNS RPZ). DNS RPZ is a method that allows a name server administrator to overlay custom information on top of the global DNS to provide alternate responses to queries. You can read more about DNS RPZ at DNS Response Policy Zone

Response Policy Zones are delivered as one of seven DNS zones suitable for deployment as a DNS Response Policy Zone. Each zone file name contains a time which corresponds to how old the domains in the zone are believed to be. Farsight recommends the use of the 3h.rpz.dns-nod.net zone as a starting point. You will receive DNS NOTIFY messages for all seven zones but the preference is that you only download the zones that you will actively use, to avoid duplication:

  • 5m.rpz.dns-nod.net
  • 10m.rpz.dns-nod.net
  • 30m.rpz.dns-nod.net
  • 1h.rpz.dns-nod.net
  • 3h.rpz.dns-nod.net
  • 12h.rpz.dns-nod.net
  • 24h.rpz.dns-nod.net

The domains included in the seven DNS zones correspond to the age of the domain as the names were first observed by our sensor network. Think of RPZ as various rolling window from five minutes to 24 hours. In other words, newly observed domains (NOD) are bucketed by age and formed into seven different zone files.

When a NOD is first observed, it is inserted into all seven zone files and as the domain ages, our systems remove the name from the appropriate RPZ file starting with the 5m file first rolling up through the files to the last 24h file. A once newly observed domain will ultimately age out and it will no longer be included/found in any of the RPZ zone files.

Farsight does not support an rpz files with domains older than 24 hours old.

In the meantime, here is a link that includes additional information about DNSD Response Policy Zones that you might find helpful:

Further technical information may be found in the NOD Technical Data Sheet and NOD User Documentation.

Example config files below, see the welcome email for personalized examples.

File named.fsi-####-#.conf: Note: You’ll want to make sure the name of this file matches the name of the file you get in the welcome email.

key “FSI-####-#-key” {
  algorithm HMAC-SHA512
  secret “SECRET
};

masters “fsi-ixfr-masters” {
  104.244.13.88 key “FSI-####-#-key”;
  104.244.14.88 key “FSI-####-#-key”;
  2620:11c:f004::88 key “FSI-####-#-key”;
  2620:11c:f008::88 key “FSI-####-#-key”;
};

File named.fsi-nod.conf

// add this to your options clause
options {
   response-policy {
     zone “3h.rpz.dns-nod.net” policy given;
     // zone “3h.rpz.dns-nod.net” policy passthru; # audit with logging
   };
};


// optionally, add something like this to your logging clause and send
// to your SIEM

logging {
   channel named-rpz {
     file “/var/log/rpz.log” versions 3 size 250k;
     print-time yes;
     print-category yes;
     print-severity yes;
     severity info;
   };

   category rpz {
     named-rpz;
   };

};

// include the keyfile we created above
include “/etc/bind/named.fsi-####-#.conf”;

zone “3h.rpz.dns-nod.net” {
   type slave;
   file “3h.rpz.dns-nod.net.zone”;
   masters { fsi-ixfr-masters; };
   allow-query {localhost;};
   allow-transfer {none;};
};

NOD DNS Blackhole List Zone (DNSBL)

Farsight Security makes NOD available in DNS-based Blackhole List (DNSBL) Zone file format. DNSBLs convey information over DNS and allow subsequent processes make decisions based on the provided answers. DNSBLs are most commonly used to assist in the scoring of SPAM email.

Using the named.fsi-####-#.conf configuration file as described above; you can use the following snip-it in your name server configuration file to consume the NOD DNSBL file via IXFR.

include “/etc/bind/named.fsi-####-#.conf”;

zone “v1.bl.dns-nod.net” {
   type slave;
   file “v1.bl.dns-nod.net.zone”;
   masters { fsi-ixfr-masters; };
   allow-query {localhost;};
   allow-transfer {none;};
};

Using dig for troubleshooting IXFR

IXFR depends on retrieving the SOA (Start Of Authority) record for the zone: this record contains a serial number, and the first step in the IXFR process is to compare the serial number with what the recipient server has locally. In practice, many connectivity issues can be understood by looking at what happens when a request is made to retrieve the SOA record for the zone.

The basic command looks like this:

dig -y '<hashing algorithm>:<your key name>:<your secret>' @<one of our servers> \
    <one of our zones> SOA

Using the example data given above:

dig -y 'HMAC-SHA512:FSI-####-#-key:NOT=REALLY=YOUR=SECRET' @104.244.13.88 v1.bl.dns-nod.net SOA

; <<>> DiG 9.8.3-P1 <<>> -y HMAC-SHA512 @104.244.13.88 v1.bl.dns-nod.net SOA
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 14186
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;v1.bl.dns-nod.net.             IN      SOA

;; ANSWER SECTION:
v1.bl.dns-nod.net.      86400   IN      SOA     a.ns.dns-nod.net.v1.bl.dns-nod.net.
    nod-admin.fsi.io. 1479754872 600 300 86400 300

;; AUTHORITY SECTION:
v1.bl.dns-nod.net.      86400   IN      NS      a.ns.dns-nod.net.
v1.bl.dns-nod.net.      86400   IN      NS      b.ns.dns-nod.net.

;; TSIG PSEUDOSECTION:
fsi-####-#-key.         0       ANY     TSIG    hmac-sha512. 1479754882 300 64
    NOT=REALLY=YOUR=SECRET 14186 NOERROR 0

;; Query time: 71 msec
;; SERVER: 104.244.13.88#53(104.244.13.88)
;; WHEN: Mon Nov 21 11:01:22 2016
;; MSG SIZE  rcvd: 258

That shows a successful response. The two most important pieces of
information are:

  • the query status shown on the HEADER line
  • the TSIG section which will show any key-related issues

and of course, that you got a response at all. (If you’re sniffing
packets, it’s a lot easier to look for the SOA query than to try to
recognize a zone transfer which probably spans multiple packets.)

Retrieve DNS Zones via Rsync

NOD RPZ files can be retrieved via Rsync when DNS Zone Transfers are not
desired. Rsync is a file copying tool, it is known for its
delta-transfer algorithm, which reduces the amount of data sent over the
network by sending only the differences between the source files and the
existing files at the destination.

NOD Response Policy Zones (RPZ) Starting Recommendation

Farsight recommends the use of the 3h.rpz.dns-nod.net zone as a starting
point. There are separate files based on the age of the domain names.
Please only download the file for the zone that you will actively use.

path – filename:

  • nod/rpz/5m.rpz.dns-nod.net.zone
  • nod/rpz/10m.rpz.dns-nod.net.zone
  • nod/rpz/30m.rpz.dns-nod.net.zone
  • nod/rpz/1h.rpz.dns-nod.net.zone
  • nod/rpz/3h.rpz.dns-nod.net.zone
  • nod/rpz/12h.rpz.dns-nod.net.zone
  • nod/rpz/24h.rpz.dns-nod.net.zone

Example shell command to download the three hour NOD RPZ Zone file to
the directory /srv/nod:

rsync -az -e "ssh -p 49222 –i/path/to/sshkey" \
    [email protected]:nod/rpz/3h.rpz.dns-nod.net.zone /srv/nod/

The following is an example BIND configuration for RPZ:

options {
  response-policy { zone "3h.rpz.dns-nod.net"; };
};

zone "3h.rpz.dns-nod.net" {
  type master;
  file "3h.rpz.dns-nod.net.zone";
  allow-query {localhost;};
  allow-transfer {none;};
  allow-update {none;};
};

Example shell script to be run by cron to download and load the three hour NOD RPZ zone file using BIND utility rndc. If your operating system distribution includes the flock – manage locks from shell scripts command, using it will prevent multiple cronned instances from overrunning each other (its semantics are similar to time ):

#!/bin/bash

rsync -az -e "ssh -p 49222 –i/path/to/sshkey" \
    [email protected]:nod/rpz/3h.rpz.dns-nod.net.zone /srv/nod/

rndc reload /srv/nod/3h.rpz.dns-nod.net

Using NOD RPZ this way will cause DNS queries for any domains in the
selected database to return an NXDOMAIN response, should those domains
be queried by an end-user.

NOD DNS Blackhole List Zone (DNSBL) Availability

Farsight Security makes NOD available in DNS-based Blackhole List
(DNSBL) Zone file format. DNSBLs convey information over DNS and allow
subsequent processes make decisions based on the provided answers.
DNSBLs are most commonly used to assist in the scoring of SPAM email.

Farsight NOD DNSBL Zones can be loaded and served by Rbldnsd to
applications like Spamassassin and Postfix.

Your user name on the Farsight server will be “USERNAME”.

Example shell command to download the one hour NOD DNSBL Zone file to
the directory /srv/nod:

rsync -az -e "ssh -p 49222 –i/path/to/sshkey" [email protected]:v1/nod.rbldnsd /srv/nod/

Use Rbldnsd to serve the NOD DNSBL Zone

*Rbldnsd (http://www.corpit.ru/mjt/rbldnsd.html> is a small
authoritate-only DNS name server designed to serve DNS-based blocklists
(DNSBLs).

Configure Rbldnsd to load the NOD DNSBL Zone

The following is an example of the Rbldnsd command-line arguments

Starting Rbldnsd:

rbldnsd -b 127.0.0.1/5053 -r /srv/nod/ v1.bl.dns-nod.net:dnset:nod.rbldnsd -p /var/run/rbldnsd.pid

Configuring BIND to forward DNSBL queries to Rbldnsd

Sample BIND configuration:

zone "v1.bl.dns-nod.net" IN {
  type forward;
  forward first;
  forwarders {
    127.0.0.1 port 5053;
  };
};

Using the NOD DNSBL

Many applications can be configured to use a DNSBL. This user guide
gives guidance on two open source applications that can be configured to
consume a DNSBL. (SpamAssassin and Postfix)

When using NOD in this manner, DNS queries for A type (IP address)
resource records of the form domain.v1.bl.dns-nod.net will return an
address that indicates the age of domain if it is in the NOD database.
The response should be interpreted as follows:

PeriodResponse
0-5 minute127.0.0.2
5-10 minute127.0.0.3
10-30 minute127.0.0.4
30-60 minute127.0.0.5
1-3 hours127.0.0.6
3-12 hours127.0.0.7
12-24 hours127.0.0.8

“NXDOMAIN” (no such domain) response will be returned if the domain is
not in the NOD database.

TXT DNS queries will return more detailed information about the domain
if it is in the database. For example:

$ dig +short svetlanovskiy.accountant.v1.bl.dns-nod.net
127.0.0.2
$ dig +short svetlanovskiy.accountant.v1.bl.dns-nod.net txt
"first_seen=1461953815"

The first_seen date is in Unix seconds. That value can be converted to
“human time” with a command such as:

$ date -r 1461953815
Fri Apr 29 11:16:55 PDT 2016

You may test your setup using your favorite DNS query tool and the
special test domains test.dns-nod.net and invalid.dns-nod.net.

$ host test.dns-nod.net.v1.bl.dns-nod.net
test.dns-nod.net.v1.bl.dns-nod.net has address 127.0.0.2
$ host invalid.dns-nod.net.v1.bl.dns-nod.net
Host invalid.dns-nod.net.v1.bl.dns-nod.net not found: 3(NXDOMAIN)

Configure SpamAssassin to use NOD DNSBL

To configure SpamAssassin to use NOD DNSBL, append the following to
/etc/spamassassin/local.cf or ~/.spamassassin/user_prefs, adjusting
scores to taste:

urirhssub URIBL_NOD_5M  v1.bl.dns-nod.net.  A 127.0.0.2
body      URIBL_NOD_5M  eval:check_uridnsbl('URIBL_NOD_5M')
describe  URIBL_NOD_5M  Contains an URL that is only 5 minutes old in
    Farsight Passive DNS
tflags    URIBL_NOD_5M  net domains_only
score     URIBL_NOD_5M  5

urirhssub URIBL_NOD_10M v1.bl.dns-nod.net.  A 127.0.0.3
body      URIBL_NOD_10M eval:check_uridnsbl('URIBL_NOD_10M')
describe  URIBL_NOD_10M Contains an URL that is only 10 minutes old in
    Farsight Passive DNS
tflags    URIBL_NOD_10M net domains_only
score     URIBL_NOD_10M 5

urirhssub URIBL_NOD_30M v1.bl.dns-nod.net.  A 127.0.0.4
body      URIBL_NOD_30M eval:check_uridnsbl('URIBL_NOD_30M')
describe  URIBL_NOD_30M Contains an URL that is only 30 minutes old in
    Farsight Passive DNS
tflags    URIBL_NOD_30M net domains_only
score     URIBL_NOD_30M 5

urirhssub URIBL_NOD_1H  v1.bl.dns-nod.net.  A 127.0.0.5
body      URIBL_NOD_1H  eval:check_uridnsbl('URIBL_NOD_1H')
describe  URIBL_NOD_1H  Contains an URL that is only one hour old in
    Farsight Passive DNS
tflags    URIBL_NOD_1H  net domains_only
score     URIBL_NOD_1H  5

urirhssub URIBL_NOD_3H  v1.bl.dns-nod.net.  A 127.0.0.6
body      URIBL_NOD_3H  eval:check_uridnsbl('URIBL_NOD_3H')
describe  URIBL_NOD_3H  Contains an URL that is only three hours old in
    Farsight Passive DNS
tflags    URIBL_NOD_3H  net domains_only
score     URIBL_NOD_3H  5

urirhssub URIBL_NOD_12H v1.bl.dns-nod.net.  A 127.0.0.7
body      URIBL_NOD_12H eval:check_uridnsbl('URIBL_NOD_12H')
describe  URIBL_NOD_12H Contains an URL that is only twelve hours old in
    Farsight Passive DNS
tflags    URIBL_NOD_12H net domains_only
score     URIBL_NOD_12H 5

urirhssub URIBL_NOD_24H v1.bl.dns-nod.net.  A 127.0.0.8
body      URIBL_NOD_24H eval:check_uridnsbl('URIBL_NOD_24H')
describe  URIBL_NOD_24H Contains an URL that is only 24 hours old in
    Farsight Passive DNS
tflags    URIBL_NOD_24H net domains_only
score     URIBL_NOD_24H 5

Configure Postfix to use NOD DNSBL

For postfix to reject everything:

reject_rhsbl_sender = v1.bl.dns-nod.net

To cut off at a particular age threshold using postfix version 2.8 or higher:

reject_rhsbl_sender = v1.bl.dns-nod.net=127.0.0.[2-8]

The individual domain age ranges must be explicitly listed when using
postfix versions 2.1 through 2.7:

reject_rhsbl_sender = v1.bl.dns-nod.net=127.0.0.2,…,v1.bl.dns- nod.net=127.0.0.8

NOD and Local Firewalls

If you need to configure your firewalls to allow access to the NOD systems at Farsight,
please be aware that servers may be rotated in and out of service without notice, so we
recommend you do not configure access to specific IP addresses. You should configure to
allow access through 104.244.12.0/22 to be able to access NOD services without
interruption.

Additional Information

Related Content

Farsight Security: A DomainTools Company logo in front of black-gray gradient
User Guides

The AXA User Guide

Read more
Farsight Security: A DomainTools Company logo in front of black-gray gradient
User Guides

The AXA Technical Overview

Read more
Farsight Security: A DomainTools Company logo in front of black-gray gradient
User Guides

The AXA API Reference

Read more
DomainTools Logo
Pricing Contact Login Support Request a Demo

© 2024 DomainTools

DomainTools® and DomainTools™ are owned by DomainTools, all rights reserved.

Privacy Policy    |    California Privacy Notice
Do Not Sell My Personal Information    |    Terms of Service    |    Sitemap