Introduction
In an environment as large and complex as the Internet, it’s difficult to keeptrack of things that are new.
Today, we’re going to talk about something that might seem almost impossible,and that’s keeping track of new Internet-visible hostnames — on ahostname-by-hostname basis — Internet-wide.
Before we do that, however, let’s quickly recap an earlier FarsightSecurity, Inc. (FSI) product, Newly Observed Domains, or “NOD,” so you’ll havea foundation for understanding both it and our new product.
Newly Observed DOMAINS
A year ago, FSI announced our highly-popular Newly Observed Domains (NOD)product. That channel consists of a near-real-time stream of newly-observed2nd-level domain names (“com” is a “base domain name,” and “example.com” wouldbe considered a “2nd-level domain name.”)
By monitoring that channel, subscribers could learn about new 2nd-level domainnames virtually as soon as they began to be used. That visibility was (andstill is) useful for a variety of purposes including but not limited to:
- automatically blocking spam and potentially-malicious newly-created domains for a number of hours (e.g., until domain name reputation systems have had a chance to catch up),
- brand protection work,
- corporate intelligence,
- detection of DGA-related botnet activity, and
- search engine seeding
Some may have (incorrectly) assumed that NOD was merely an aggregation ofpublicly-available Zone File Access program (ZFA) data. It’s not. The core ofNOD consists of data from actual live monitoring performed by FSI’s worldwidenetwork of over 500 sensor nodes.
Thus, NOD includes results for newly observed domains from TLDs which don’toffer ZFA programs at all, and from 2nd level-domains created aboveeffective-TLDs (as defined by the Mozilla Public Suffix list), as well as newly observed domains from conventional gTLDs andccTLDs. It’s truly a unique resource, and an impressive accomplishment in itsown right. That said, we hope you’ll agree that our new product is even cooler.
Newly Observed HOSTNAMES
Earlier this month at BlackHat Las Vegas — one of the world’s largest andlongest-running system and network security conferences — FSI announced ourNewly Observed HOSTNAMES (NOH) channel.
Newly Observed Hostnames compliments our existing Newly Observed Domainschannel by “zooming in” and tracking the creation of individualInternet-visible hostnames (e.g., “fully qualified domain names” or FQDNs) ona hostname-by-hostname basis worldwide.
Conceptually, you can think of NOH as taking a raw stream of DNS hostnames (asseen at Farsight’s Security Information Exchange (SIE), and then screening each of those hostnames againstour Passive DNS database (DNSBD), reporting only those that haven’t previously been seen by any of our over500 sensor nodes.
A typical Newly Observed Hostname entry (in presentation format) looks like:
domain: mutualofomaha.com. <-- base domain
time_seen: 2015-08-12 17:10:14
bailiwick: mutualofomaha.com.
rrname: bgzpzftw.mutualofomaha.com. <-- full hostname
rrclass: IN (1)
rrtype: A (1)
rdata: 170.31.64.50
Averaged over the day, typically there will be about ninety Newly ObservedHostnames per second (versus just one or two Newly Observed Domains persecond). Newly Observed Hostnames typically consumes about a 100 Kbps worth ofnetwork bandwidth. An average newly-observed hostname is published about 150seconds from the time of its first observation.
So now that you know what NOH is, let’s look at some use-cases.
The Need For Newly Observed Hostnames: PHISHING
Some may wonder why anyone might want or need a feed of Newly ObservedHostnames. To understand that, it may help to think about the sort of domainnames that routinely show up in conjunction with abuse. For example, considerphishing domains.
PhishTank is one of several sites that listsuser-reported phishing URLs. When I checked that site on August 11th, 2015 itincluded a variety of apparently PayPal-related URLs such as the following(de-fanged here to prevent anyone from accidentally visiting these URLs, andto keep this article from potentially making any domain reputation systemstwitch):
- http://p4yp4l[dot]gq/co.uk/validate/uk/…
- http://ppaaypal[dot]com/us/webapps/m64/home
- http://epapyal[dot]com/login/
Note that those domains are the sort of domains that could be detected from aNOD feed: the potential phishing-related signature content, bolded above, is anintegral part of the 2nd-level domain name (which is what you’d see in NOD).
But now consider some other domain names, also apparently PayPal-related, alsofrom PhishTank:
- http://loginpaypal[dot]backgammonplayer[dot]com/update…
- http://www[dot]paiypalservice[dot]esy[dot]es/webapps/…
- http://please[dot]logintoupdate[dot]yourinformation[dot]pyapal[dot]comresusa[dot]com/updat…
- http://paypall[dot]ksarpress24[dot]com/ppl/update-information-security-login-w…
- http://paypal[dot]pp-srvrx.com/ssl-gateway/paypal.de/paypal.com/0/
- http://www[dot]paypal[dot]kundencenter[dot]com[dot]de/589…
If we reduce those URLs to just their 2nd-level domains (e.g., just the bitthat’s shown in bold above), there’s nothing inherently suspicious about those2nd-level domain names. They just appear to be regular domains.
It is only when we have the ability to see the full hostname, as you can inFarsight’s Newly Observed HOSTNAMES, that we can see potential phishing-relatedpatterns that would likely trigger further review and action by anti-phishingspecialists.
Rapid identification of suspicious hostnames, as enabled by Newly ObservedHostnames, translates to quicker takedowns, lower levels of phished accountsand financial losses, and thus happier banks, payment card companies, andother financial businesses — to say nothing of their customers.
The Need for Newly Observed Domains: BRAND PROTECTION
Another example of how it can be critical to have visibility into completehostnames (rather than “merely” 2nd-level domain names), can be seen in thebrand enforcement area. Assume, for example that you’re interested in any/alldomains that include either the trademarked name “rolex” or the trademarkedname “gucci”.
One easy way to watch the Newly Observed Hostnames for hosts containing eitherof those words is to rent a blade server from Farsight at the SecurityInformation Exchange, subscribing to the Newly Observed Hosts channel. Onceyou’ve done that, you can simply say:
$ nmsgtool -C ch213 | grep "rolex\|gucci"
The first part of that command pipeline,
nmsgtool -C ch213
, will get trafficfrom the Newly Observed Hostname channel (Channel 213), while the second halfof that command,
grep "rolex\|gucci"
, will match and print any records witheither of our matching strings of interest, even if those strings appear buriedin the middle of a hostname.
Rather not work on a blade server at SIE? You can also use SIE Remote Access (“SRA”) to securely tunnelch213 traffic back to your own location, if that’s more convenient for yourneeds.
We can also deliver rolling hourly snapshots of NOH in CSV format as anotheroption.
These simple approaches to mining Newly Observed Hostnames may be all thatcustomers need or want to process that data stream. Other Newly ObservedHostname customers may want to take advantage of our API to tightly integrateFSI’s Newly Observed Hostname data with their own code.
The Need for Newly Observed Hostnames: IDENTIFICATION OF WILDCARDED DOMAIN NAMES
Another phenomena that you can easily identify in our Newly Observed Hostnameschannel is wildcardeddomain names. Wildcarded domain names can be used for many different purposes,ranging from marketers attempting to track individual responses to one of theirsolicitations, to less savory types trying to “stay under the radar” byavoiding any single hostname showing up as running “too hot.”
Of course, if you’re watching Newly Observed Hostnames, wildcarded domain namesrepresent a phenomena that’s pretty hard to miss.
For example, taking a 50,000 observation sample of Newly Observed Hostnames(less than 10 minutes worth of data), we can easily find hostnames such as thefollowing (note that the select hostnames shown here are just a small subsetof all matching hostnames, and the names shown have been de-fanged,partially-redacted, and are shown here in reversed, easily sorted format):
- com[dot]kddflk.nsxpu2[snip]
- com[dot]kddflk.nsxpuh[snip]
- com[dot]kddflk.nsxpvb[snip]
- com[dot]kddflk.nsxpvq[snip]
- com[dot]kddflk.nsxpw5[snip]
- com[dot]kddflk.nsxpwk[snip]
- com[dot]kddflk.nsxpwz[snip]
- com[dot]kddflk.nsxpxei[snip]
- com[dot]kddflk.nsxpxtc[snip]
- com[dot]kddflk.nsxpy8[snip]
- [etc]
If we use
to manually testother arbitrary potential hostnames in the kddflk[dot]com domain, we see thatthe nameservers for that domain will “answer” for any arbitrary hostname wespecify for that domain. This is behavior consistent with a wildcarded domainname.
The
kddflk[dot]com
resolves to
158[dot]58[dot]173[dot]5
.
Checking Passive DNS for
158[dot]58[dot]173[dot]5
, we see that over amillion unique
kddflk[dot]com
wildcard domain names exist on that IP alone.
If we further check
kddflk[dot]com
, we see that:
- The domain name’s whois point of contact information is hidden behind a privacy protection service (not a “smoking gun” in and of itself, but an often-relevant indicator)
- It’s listed on the SURBL domain block list,
- It has a negative reputation assigned by MyWOT, and
- It’s listed on Fortiguard as having had multiple malware infections over a period of years.
Our point in drawing your attention to a wildcarded domain of this sort is tomake the point that wildcarded domains are often (although certainly notalways) an example of a phenomena of interest to security staff.
With a subscription to Farsight Security’s Newly Observed HOSTNAMES channel,you can easily identify wildcarded domain names.
The Need for Newly Observed Hostnames: ENTERPRISE/AGENCY SECURITY AWARENESS
Most of the Newly Observed Hostnames that Farsight sees are fully qualifieddomain names that are intentionally made available to the Internet.
Occasionally, however, an enterprise or government agency might have internalhosts that have “inadvertently” become Internet visible.
These intranet-only hosts may then end up in FSI’s Newly Observed Hostnameschannel.
Simply knowing that those hostnames (hostnames which were never meant to “see the light of the Internet”) are now being publicly resolved can be a real warning bell/wake up call that a review and corrective reconfiguration may be needed ASAP.
The Need for Newly Observed Hostnames: ACADEMIC RESEARCH
Another example of a niche for Newly Observed Hostnames is in the academicresearch area, particularly labs that may be focused on measuringInternet-related phenomena such as adoption of new gTLDs, uptake of IPv6,Internet mapping, etc.
Getting More Information About Newly Observed Hostnames
For more information about subscribing to Newly Observed Hostnames, pleasecontact the Farsight Security Sales department at [email protected], or check here.
Joe St Sauver, Ph.D. is a Scientist with Farsight Security, Inc.
Heading
