I. Introduction
In the final episode of the first season of the TV series Mad Men, adman Don Draper pitched a new campaign for Kodak’s newly-invented rotary slide projector. Kodak’s new projector was technically advanced for its time, but that wasn’t how Don chose to advertise it. Instead, Don explained how the projector acted as a sort of family “time machine,” helping families to go back in time to reminisce about good times they’d spent together. (Check out Mad Men: The Carousel if you don’t recall if you’ve previously watched that episode.)
Farsight’s passive DNS database, DNSDB, is also a sort of “time machine,” albeit one that allows you to go “back in time” so you can better understand bad times and puzzling things that may have occurred in conjunction with systems and networks.
This may seem a little counter-intuitive, at least at first. Do we really want to look back and “dwell on the bad stuff” that may have happened? Absolutely, yes! That said, we know that when confronted with something that’s gone badly, the overwhelming urge is to repress the unpleasant memory and move on.
If we can overcome the urge to repress memories of our failures, we can often find useful clues about how things went wrong. We can “learn from our mistakes.” This is one of the reasons why baseball coaches review the previous-day’s game tapes, why police detectives watch CCTV recordings of crime scenes, and why the National Transportation Safety Board (NTSB) reviews “black box” flight recorders.
DNSDB serves as a post-attack “time machine” or “black box” flight recorder for the Internet. DNSDB contains near-real time DNS data about what’s just happened in DNS. That’s a great “rear view mirror” for those seeking to understand what’s just happened. DNSDB also lets you go back in time to focus on just a particular epoch of historical interest. This is an incredibly useful forensic capability.
We normally call that ability to limit our results to just a particular time period, “time fencing.” For the purposes of this blog post, we’ll explain what time fencing is and how it works in DNSDB, and then show you a few examples of how time fencing works in practical terms.
Note: while time fencing support is implemented in a standardized form in DNSDB API, it may be used differently by various DNSDB clients and integrations. We don’t have room to discuss all those variations here today. We do need to be able to illustrate the concepts introduced in this article, so we have to pick some interface implementation to use for our examples.
We’ll be using Farsight’s dnsdbq (formerly the “dnsdb_query C language command line interface reference client”) for this post. We’ll describe how to get, build and configure the dnsdbq client in the next section.
II. The dnsdbq Command Line Client
dnsdbq is one of the most feature-rich command line interfaces (CLIs) available for use with Farsight Security’s DNSDB API.
You can get a copy of it from https://github.com/dnsdb/dnsdbq/
Begin by reviewing the README on that Github page. As described there, install
janssonlibcurl
Once jansson and libcurl are in place, download and unzip the zip file from the Download ZIP link you’ll see when you click the green clone or download button:
Figure 1. dnsdbq Client Github Download
After unzipping that file, change down into the resulting directory, use make to compile the code, and then install the executable and man page with sudo make install
Figure 2. Building and Installing dnsdbq command line client
$ make $ sudo make install
You’ll also need to tell dnsdbq about your DNSDB API key in order to be able to access DNSDB.
Use your favorite editor to add your API key to ~/.dnsdb-query.conf using the format
APIKEY="yourAPIkeyhere"
[Note the required leading dot on the .dnsdb-query.conf file name]
The dnsdbq client should then be ready to use.
To learn a little about some of the dnsdbq client’s options, see:
$ man dnsdbq or $ dnsdbq -h
III. The Default DNSDB API Perspective: A “Womb-to-Tomb” Longitudinal Historical Review…
Normally DNSDB will return the full breadth of historical results available for whatever query a user makes.
For example, you can ask to see the history of www.farsightsecurity.com "A" records in reverse chronological order by issuing the command:
Example 1. Non-time-fenced output
$ dnsdbq -S -k last -r www.farsightsecurity.com/A
That command returns three unique results, collectively covering a period of some four years, sorted in reverse order (that’s what the -S specifies) using as a key (-k) the time each answer was last seen:
;; record times: 2015-04-01 13:07:24 .. 2018-01-15 14:25:17 ;; count: 40534; bailiwick: farsightsecurity.com. www.farsightsecurity.com. A 104.244.13.104 ;; record times: 2013-09-25 20:02:10 .. 2015-04-01 09:51:39 ;; count: 5059; bailiwick: farsightsecurity.com. www.farsightsecurity.com. A 66.160.140.81 ;; record times: 2013-07-01 17:37:26 .. 2013-09-24 17:14:08 ;; count: 164; bailiwick: farsightsecurity.com. www.farsightsecurity.com. A 149.20.4.207
The first seen and last seen record times shown reflect the first time and last time Farsight saw that domain with that IP address.
This sort of “womb-to-tomb” memory is perfect if you’re conducting a comprehensive historical review.
Sometimes, though, you only care about what happened during a recent period of time, or what happened during some specific period of time that was months or years back.
For simple cases or just a few queries, you can just visually scan a sorted list of results from DNSDB to see what (if anything) was going on during a given time period of interest.
However, when you’re doing more complex analyses, or LOTS of runs, manually digging out the right results quickly gets to be kind of tedious.
That’s where automatic time fencing begins to show its power and convenience.
IV. Some Backfill: What Gets Tracked
DNSDB keeps track of the first and last time it sees each unique answer to a question from the DNS:
Figure 3. DNSDB Timepoints
Thus, for example, when we asked DNSDB about "A" records for www.farsightsecurity.com in Example 1:
- We first saw the response
104.244.13.104for that name at 2015-04-01 13:07:24 UTC, - We last saw that response
104.244.13.104for that name at 2018-01-15 14:25:17 UTC, and - Within that interval we saw that particular response a total of 40,534 times,
See the sample output above. The same reported count would be returned regardless of whether the time fencing period specified “barely intersects” the observation period or fully encompasses it. Observations are not tracked nor allocated “per day” or “per week,” etc.
PART A. TIME FENCING WITH “AFTER”
V. The -A (“After”) Qualifier in Default “Loose” Mode
Let’s begin by focusing on the -A (“After”) qualifier as used in the current dnsdbq client. In that client, the -A qualifier stands for “After this date:”
Example 2. “After” time fencing, default loose mode
$ dnsdbq -r www.farsightsecurity.com/A -A 2015-08-25 ;; record times: 2015-04-01 13:07:24 .. 2018-01-25 17:35:44 ;; count: 41370; bailiwick: farsightsecurity.com. www.farsightsecurity.com. A 104.244.13.104
Because the last_seen date reported by DNSDB comes after the user-specified -A date, and because we’re using “loose” mode by default, that observation was properly included in the output.
Important: Loose mode generally increases the number of results returned in response to a query.
Note that the matching observation does NOT need to be “strictly” or “completely” after the user-specified after date in “loose” mode. This is illustrated by the middle arrow in Figure 4:
Figure 4: After qualifier (loose mode)
VI. The -A (“After”) Qualifier in “Complete” (or “Strict”) Mode
The alternative to loose mode is “complete” (or “strict”) mode. To select it, specify the -c option to dnsdbq
In “complete” mode, an observation must “completely” or “fully” satisfy the selection criteria in order to be returned. See figure 5. In this example, only observations whose first_seen dates are after the user specified date get displayed. This is useful when you’re trying to identify the impact of a specific change or event, and you’re only interested in things that were first seen after a specific moment.
Figure 5. After qualifier (complete mode)
Since DNSDB has no new "A" records for www.farsightsecurity.com which were created after 2015-08-25, we’d see:
Example 3. “After” time fencing, complete (“strict”) mode enabled with -c
$ dnsdbq -r www.farsightsecurity.com/A -A 2015-08-25 -c API: Error: no results found for query. libcurl: 404 (https://api.dnsdb.info/lookup/rrset/name/www.farsightsecurity.com/A?time_first_after=1440489120) please note: 404 usually just means that no records matched the search
If we pull the -A date back further in time, we can find some matching observations, even in “complete” mode:
Example 4. “After” time fencing, complete mode, with a sufficiently old date to see a hit
$ dnsdbq -r www.farsightsecurity.com/A -A 2015-01-25 -c ;; record times: 2015-04-01 13:07:24 .. 2018-01-15 14:25:17 ;; count: 40534; bailiwick: farsightsecurity.com. www.farsightsecurity.com. A 104.244.13.104
PART B. TIME FENCING WITH “BEFORE”
VII. The -B (“Before”) Qualifier in Default “Loose” Mode
What about the -B (“Before”) qualifier? It compares the user-supplied date against the DNSDB first_seen date in “loose mode” by default. Because the -B qualifier is a “loose” match by default, the observed response does NOT need to be COMPLETELY before the user specified date, it just needs to have been seen at least a little bit before that time:
Figure 6. Before Qualifier (loose mode)
An example of a -B query in loose mode:
Example 5. Before qualifier (loose mode)
$ dnsdbq -r www.farsightsecurity.com/A -B 2013-08-25 ;; record times: 2013-07-01 17:37:26 .. 2013-09-24 17:14:08 ;; count: 164; bailiwick: farsightsecurity.com. www.farsightsecurity.com. A 149.20.4.207
The “hit” shown in Example 5 is a match because the time first seen as reported by DNSDB is less than 2013-08-25.
VIII. The -B Qualifier in “Complete” (or “Strict”) Mode
In “complete” or “strict” mode, the observation’s last_seen date must be before the user specified date:
Figure 7. Before Qualifier (complete mode)
For example:
Example 6. Before qualifier (complete mode)
$ dnsdbq -r www.farsightsecurity.com/A -B 2013-09-27 -c ;; record times: 2013-07-01 17:37:26 .. 2013-09-24 17:14:08 ;; count: 164; bailiwick: farsightsecurity.com. www.farsightsecurity.com. A 149.20.4.207
PART C. USING BOTH “BEFORE” AND “AFTER” AT THE SAME TIME
IX. Using BOTH The -A and The -B Qualifiers, Loose Mode
We can constrain our time window on “both sides” by using -A and -B simultaneously. We have the option of using “loose” mode (the default) or “complete” (“strict”) mode. Let’s begin by considering the default “loose” mode:
Figure 8. BOTH before AND after in loose mode
Loose mode gives you results responsive to the question, “So what did we see during the specified time period? Tell us what was seen even if it’s something that was happening before the time period started, or something that continued after the specified time period ended.”
An example of this:
Example 7. Before AND After Time fencing, Loose Mode, Sorted in Descending Order
$ dnsdbq -r bcnetworks.com/A -A "2014-06-27" -B "2015-12-10" -S ;; record times: 2015-12-04 19:24:49 .. 2018-01-16 03:25:54 ;; count: 6640; bailiwick: bcnetworks.com. bcnetworks.com. A 54.247.91.90 ;; record times: 2014-06-28 19:06:38 .. 2015-12-04 16:30:20 ;; count: 2099; bailiwick: bcnetworks.com. bcnetworks.com. A 162.13.25.152
The first result started during the period of interest, but went on after the period of interest ended.
The second result started before the beginning time, but went through much of the period of interest.
Both are “good hits” under the default “loose mode.”
X. Using BOTH The -A and The -B Qualifiers in Complete Mode
Now let’s consider using BOTH -A and -B in “complete” or “strict” mode.
In that mode, the specified constraints must be strictly satisfied. That is to say, dnsdbq will ONLY return results that fall COMPLETELY within the specified bounds. This capability might be just what you want if you’re “hunting just for what went crazy” during a particular well-defined period.
Figure 9. BOTH before AND after in complete mode
When the dnsdbq client uses “complete” or “strict” mode:
- first_seen must be after than the user-specified after date and
- last_seen must be before the user-specified before date
This rule is “ONLY tell me about stuff that ONLY happened during the specified time period. If we saw it before or after that, I’m not interested, even if it was ALSO going on during the specified time period.”
Example 8. Before AND After Filtering, “Complete” or “Strict” mode (note the -c)
$ dnsdbq -r bcnetworks.com/A -A "2014-06-27" -B "2015-12-10" -S -c ;; record times: 2014-06-28 19:06:38 .. 2015-12-04 16:30:20 ;; count: 2099; bailiwick: bcnetworks.com. bcnetworks.com. A 162.13.25.152
That “hit” has a first seen date that’s AFTER 2014-06-27 (e.g., we first saw the domain on 2014-06-28) and a last seen date that BEFORE 2015-12-10 (e.g., we last saw the domain on 2015-12-04).
The returned answers are completely BETWEEN the time fencing points, and the returned answer does NOT run BEYOND the specified window. Think of this rule as just returning RRsets that are TOTALLY CONTAINED within, or STRICTLY BOUNDED BY, the specified time window.
XI. Time AND Date Stamps. Implicit Time Zone.
Sometimes you may want to specify a finer-than-single-day granularity, e.g., a particular date AND time stamp. An example of doing this is shown in Example 9:
Example 9. Specifying a date and time stamp
$ dnsdbq -r www.farsightsecurity.com/A -A "2015-04-01 12:00:00" ;; record times: 2015-04-01 13:07:24 .. 2018-01-15 17:28:41 ;; count: 40534; bailiwick: farsightsecurity.com. www.farsightsecurity.com. A 104.244.13.104
Note that dates with time stamps MUST be enclosed within quotes.
When times are mentioned, they’re assumed to be in “UTC” (NOT Eastern, Central, Pacific, etc.)
XII. Relative times and the -A Qualifier in dnsdbq
Many analysts don’t care what happened months or years ago. They may just want to see what happened during the last few hours (4h), the last week (7d), or some other brief period. When that’s the case, relative times can be an easy way to focus on just recent results. For example:
Example 10. Asking for records seen for www.uoregon.edu in the last week
$ dnsdbq -r www.uoregon.edu -A 7d ;; record times: 2014-12-29 16:09:30 .. 2018-01-15 16:58:23 ;; count: 1203051; bailiwick: uoregon.edu. www.uoregon.edu.CNAME drupal-cluster5.uoregon.edu.
You can even ask to go back some combination of weeks, days, hours, minutes and/or seconds — just string the relative time elements of interest together, e.g.,-A6w3d12h30s
XIII. Time Fencing Example Case Study
Finally, let’s look at an incident that Fox-IT B.V. (NL) recently reported. That company stated
[…] In the early morning of September 19 2017, an attacker accessed the DNS records for the Fox-IT.com domain at our third party domain registrar. The attacker initially modified a DNS record for one particular server to point to a server in their possession and to intercept and forward the traffic to the original server that belongs to Fox-IT. This type of attack is called a Man-in-the-Middle (MitM) attack. The attack was specifically aimed at ClientPortal, Fox-IT’s document exchange web application, which we use for secure exchange of files with customers, suppliers and other organizations. We believe that the attacker’s goal was to carry out a sustained MitM attack. Because we detected and addressed the breach quickly we limited the total effective MitM time to 10 hours and 24 minutes. [continues]
See https://www.fox-it.com/en/insights/blogs/blog/fox-hit-cyber-attack/
What do we see in DNSDB? We believe that this was a brief and quickly mitigated event involving unauthorized changes to their nameservers:
Example 11. What Do Fox-It’s name servers look like?
$ dnsdbq -r fox-it.com/ns/com ;; zone times: 2017-12-23 17:02:27 .. 2018-01-15 17:02:35 ;; count: 24; bailiwick: com. fox-it.com. NS cf1.fox-it.com. fox-it.com. NS cf2.fox-it.com. ;; zone times: 2010-04-24 16:12:21 .. 2017-12-22 17:02:25 ;; count: 2788; bailiwick: com. fox-it.com. NS ns1.fox-it.com. fox-it.com. NS ns2.fox-it.com. ;; record times: 2017-12-22 18:22:33 .. 2018-01-16 23:34:57 ;; count: 5446; bailiwick: com. fox-it.com. NS cf1.fox-it.com. fox-it.com. NS cf2.fox-it.com. ;; record times: 2010-06-24 10:43:31 .. 2017-12-22 17:40:52 ;; count: 480904; bailiwick: com. fox-it.com. NS ns1.fox-it.com. fox-it.com. NS ns2.fox-it.com. ;; record times: 2017-09-18 22:37:15 .. 2017-09-19 05:20:38 ;; count: 51; bailiwick: com. fox-it.com. NS ali.ns.cloudflare.com. fox-it.com. NS dom.ns.cloudflare.com.
In fact, while Farsight saw the name servers change, as shown above, none of our sensors actually saw the clientportal.fox-it.com server resolve to attacker-specified bogus value. All we see in the Example 12 output was the normally expected "A" record for that domain:
Example 12: What did we see for clientportal.fox-it.com? Just business as usual
$ dnsdbq -r clientportal.fox-it.com -A 2017-01-01 ;; record times: 2010-11-12 03:48:55 .. 2018-01-15 07:10:17 ;; count: 603; bailiwick: fox-it.com. clientportal.fox-it.com. A 178.250.144.146
This is a very positive indicator showing just how efficiently fox-it.com detected and handled their incident, good job!
XIV. Conclusion
You’ve now seen:
• That time fencing allows DNSDB to act as a sort of “time machine,” letting you “go back in time” to the time when an incident occurred.
• DNSDB’s time fencing can also act as a helpful “shield” to ensure that ancient no-longer-relevant data doesn’t complicate a current cyber investigation.
• We’ve shown you how to get, build and use dnsdbq for time fencing.
• We’ve illustrated the difference between “loose” time fencing (as used by dnsbq by default) and “complete” (or “strict”) time fencing, and when each can be useful.
• How to specify absolute dates, absolute dates AND times, and how to use relative time fencing periods.
• We finished up with a case study showing how time fencing can help you to look at a real-world incident.
We hope you’ve found all the above about time fencing to be interesting!
For more information about getting access to DNSDB API, please see https://www.farsightsecurity.com/order-services/
Joe St Sauver Ph.D. is a Distinguished Scientist with Farsight Security, Inc.
