1. Introduction
SIE Batch provides an easy way to download batches of data from channels on the Farsight Security Information Exchange. If you’re curious about what data’s available there, there’s a list of publicly available SIE channels.
If you want to try SIE Batch, contact Farsight Security Sales to arrange for access to one or more SIE channels of interest. The Farsight Security Sales Team can be reached at [email protected]. Be sure to mention that you want to access that channel/those channels via SIE Batch.
Access to the SIE Batch GUI, API, and related API documentation is only available to current subscribers.
2. Interactive Use of SIE Batch
SIE Batch can be used two different ways: interactively, or via the SIE Batch API.
This article is focused on showing an example of how to create a simple command line client with the SIE Batch API.
For information on the interactive use of SIE Batch via the web interface, read our blog article, What’s SIE Batch? Why Might I Be Interested In It?.
The web interface is great if you just want to grab a quick chunk of data from an SIE channel that you’ve subscribed to. That said, we realize that some users may want a command line interface to SIE Batch (perhaps for routine scripting use), or the ability to directly integrate SIE Batch into their own products.
That’s where SIE Batch’s API comes into play. We’ll focus on it for the rest of this article.
3. SIE Batch API
In a nutshell, the core SIE Batch API functionality involves doing an HTTPS POST to a defined endpoint, and passing query parameters specifying:
- Your SIE Batch API key (note: NOT the same as your DNSDB API key)
- The channel number you’re interested in and subscribed to
- A starting UTC datetime in 24 hour format, and
- An ending UTC datetime in 24 hour format.
You can use pretty much any programming or scripting language to do this, including shell scripting languages (such as bash), Python, Perl, Ruby, Java, Scala, C, C++, etc.
4. Sample Command Line Client In C With libcurl (As An Example of Using the SIE Batch API)
This article’s focus is on showing how you can use C with
libcurl
to query SIE Batch API, much as we’ve previously shared sample C program to query the DNSDB API with
libcurl
(see Making Programmatic DNSDB Queries With libcurl).
We’ll begin by explaining our thinking about how we’re going to approach this task.
Sample Command Invocation: We want our command line client to handle two basic query formats:
- One query format will use explicit UTC starting and ending datetime values, such as:
- $ sie_get 212 “2020-01-07 00:13:00” “2020-01-07 00:28:00”
- Another query format will start at the current time (“now”), and go back some integer number of minutes:
- $ sie_get 212 now 5
The first case is obviously more general, but suffers from being verbose. The second case is simpler (and thus probably more likely to be used), but less general. Fortunately, we can support both, and don’t need to choose just one or the other.
Output: Output from the client will be written to files. The files will be in our current directory, with names such as:
sie-ch212-{2020-01-09@19:46:42}-{2020-01-09@20:01:42}.jsonl
sie-ch208-{2020-01-09@21:39:44}-{2020-01-09@21:44:44}.nmsg
Ignoring punctuation in the filenames, the filenames are composed from the following elements:
sie literal filename prefix signaling the data source
ch212 channel number (with literal ch prefix)
2020-01-09@19:46:42 starting datetime
2020-01-09@20:01:42 ending datetime
jsonl file extension conoting the file (and channel) data format
The filename as shown above is also what you’ll see if you look at those filenames at the command line (e.g., using the Un*x
ls
command from within a terminal window).
Some operating systems may change how those files are represented or displayed in graphical file browsers. For example, in Finder under Mac osX Catalina, you’ll see the colons in the filenames displayed as slashes (although in reality the filenames are unchanged):
5. Building
The code for this application can be seen in Appendicies I-III of this article, or obtain a copy from here:
Prerequisite Libraries: We leverage two third-party libraries in this sample: libcurl and
. We use
libcurl
to do the “heavy lifting” for our
https
calls, and
json-c
to deserialize (e.g., “to extract” or “to unmarshall”) the
JSON
format status code information we need.
When installing json-c, if you don’t install it into the standard /usr/local/lib directory, you may need to explicitly declare it’s location in the Makefile; in my case I said:
JSON_C_DIR=/usr/local/Cellar/json-c/0.13.1
Normal Build: Once you’ve got the prerequisite libraries installed and the provided
Makefile
tweaked to suit your system, you should just be able to use the typical:
$ make
$ sudo make install
to compile, link and install the
sie_get
executable.
6. Setting Up Your SIE Batch API Key
Access to SIE Batch is controlled via use of SIE Batch API keys. Your SIE Batch API key will be configured to give you access to the SIE channels you’ve purchased from Farsight. (Note that your SIE Batch API key is NOT the same as the API key you may already have for DNSDB API).
For the purpose of this command line example client, use your favorite editor to put your SIE Batch API key (with no additional text or punctuation) into the file:
~/.sie-get-key.txt
Note the easy-to-overlook leading dot near the beginning of that filename. Create that file prior to running the
sie_get
client. If you forget to do so and ~/.sie-get-key.txt doesn’t exist, you’ll see an error such as:
ERROR: no API key file at /home/jsmith/.sie-get-key.txt
If the file exists (but the API key in that file is missing or invalid), you’ll see:
403 "API key not present"
7. Running The Client Interactively
You can run the
sie_get
client interactively, or you can set up a
cron
job to periodically run
sie_get
.
As previously mentioned, one approach is to explicitly specify a channel you’re interested in, plus a starting and ending datetime value in UTC (naturally, you must be subscribed to any SIE channel you want to access).
$ sie_get 212 "2020-01-07 00:13:00" "2020-01-07 00:28:00"
You must provide full 19-character long values as shown, zero padded as may be required throughout (e.g., Midnight UTC on February 1st, 2020 would be “2020-02-01 00:00:00”). The date AND the time must always be provided. Datetime values MUST be quoted as shown (you’re passing a channel number and TWO datetime values, you’re NOT passing a channel number plus FOUR additional values).
If you find yourself struggling to convert local time to UTC, check out
$ date -u
Being lazy, we also created a mode that will let you ask for minutes worth of traffic by “rewinding” minutes from the current time. For example, to ask for five minutes of channel 212:
$ sie_get 212 now 5
When your
sie_get
job finishes, you’ll be able to find your output file in your current directory. You should see files that look like:
$ ls -t sie-ch*
sie-ch212-{2020-01-10@06:58:34}-{2020-01-10@07:08:34}.jsonl
Given some of the comparatively uncommon characters in those filenames, you should plan on quoting those filenames when you work with them. For example:
$ more "sie-ch212-{2020-01-10@06:58:34}-{2020-01-10@07:08:34}.jsonl"
{"time":"2020-01-10 06:59:01.162622928","vname":"SIE","mname":"newdomain",
"source":"a1ba02cf","message":{"domain":"m3wx3m.duckdns.org.","time_seen":
"2020-01-10 06:57:15","bailiwick":"duckdns.org.","rrname":
.duckdns.org.","rrclass":"IN","rrtype":"A","rdata":["5.189.132.177"],
[],"new_rr":[]}}
8. Scheduling The sie_get Client To Run Periodically With
cron
As noted here,
vixie cron
is yet another contribution to the community from Farsight Security CEO Dr. Paul Vixie.
cron
makes it easy to run a program (such as
sie_get
) on a routinely-scheduled basis.
For example, maybe you want to grab a five minute sample of data from SIE Channel 212 at the top of every hour. After installing
sie_get
, you could create a
crontab
entry like the following one:
$ crontab -ei@hourly /usr/local/bin/sie_get 212 now 5<ESC>:wq
Note that the
crontab
program typically uses “
vim
-like” semantics, with text being inserted into the
crontab
after you enter an
i
(until you hit the
<ESC>
character). If you need to correct an error, get out of insert mode by hitting
<ESC>
then move to your error with your cursor keys and hit the
x
key to delete the erroneous character. To write that file and quit, enter :wq )
After that
cron
job has been running for a while, you should see your downloads begin to accumulate:
$ ls -lat sie-ch*
-rw-r--r-- 1 jsmith staff 114451 Jan 10 17:00 sie-ch212-{2020-01-11@00:55:00}-{2020-01-11@01:00:00}.jsonl
-rw-r--r-- 1 jsmith staff 80861 Jan 10 16:00 sie-ch212-{2020-01-10@23:55:00}-{2020-01-11@00:00:00}.jsonl
-rw-r--r-- 1 jsmith staff 99017 Jan 10 15:00 sie-ch212-{2020-01-10@22:55:00}-{2020-01-10@23:00:00}.jsonl
-rw-r--r-- 1 jsmith staff 108448 Jan 10 14:00 sie-ch212-{2020-01-10@21:55:00}-{2020-01-10@22:00:00}.jsonl
-rw-r--r-- 1 jsmith staff 80108 Jan 10 13:00 sie-ch212-{2020-01-10@20:55:00}-{2020-01-10@21:00:00}.jsonl
-rw-r--r-- 1 jsmith staff 91057 Jan 10 12:00 sie-ch212-{2020-01-10@19:55:00}-{2020-01-10@20:00:00}.jsonl
-rw-r--r-- 1 jsmith staff 288624 Jan 10 11:00 sie-ch212-{2020-01-10@18:55:00}-{2020-01-10@19:00:00}.jsonl
-rw-r--r-- 1 jsmith staff 212893 Jan 10 10:00 sie-ch212-{2020-01-10@17:55:00}-{2020-01-10@18:00:00}.jsonl
-rw-r--r-- 1 jsmith staff 1877907 Jan 10 09:00 sie-ch212-{2020-01-10@16:55:00}-{2020-01-10@17:00:00}.jsonl
If you change your mind and want to stop running that
cron
job, you can edit the
crontab
again and delete the no-longer wanted entry with the dd (delete line) command followed by :wq (write and quit)
You now know an easy way to periodically grab SIE Batch data downloads!
9. Some Q&A
Q1) When I try to build the client, my system can’t find strlcpy and strcat!”
A1) The stock
strcat
and
strcpy
commands are well known for being potentially dangerous. (A nice discussion of this problem can be seen in chapter 2 of “Secure Coding in C and C++, 2nd Edition,” conveniently available here).
strlcpy
is generally accepted as being a better option. One discussion: “strlcpy and strlcat—Consistent, Safe, String Copy and Concatenation” by Todd C. Miller (University of Colorado, Boulder) and Theo de Raadt (OpenBSD project) is available here.
Mac OS X, being rooted in BSD, has
strlcpy
and
strlcat
natively. Those of you running in some other environments may not have these routines. (If curious why not, see for example The ups and downs of strlcpy()).
An easy way to get
strlcat
and
strlcpy
(if you don’t already have them natively) may be by downloading and including:
-- http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/lib/libc/string/strlcat.c
-- http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/lib/libc/string/strlcpy.c
Q2) “What if I mess up and forget the channel number or forget to put in the double quotes around the datetime values, etc.?”
A2) You’ll see an error message. Just try it again. The error message does its’ best to offer some hints:
ERROR: Wrong number of command line arguments supplied.
Usage: sie_get channel "startdatetime" "enddatetime"
Example: sie_get 212 "2020-01-07 00:13:00" "2020-01-07 00:28:00"
Notes: UTC datetimes must within half a day or so and must be double quoted.
Zero pad any single digit months, days, hours or minutes.
Seconds must be part of the UTC datetimes (but are ignored.)
Need the current UTC time? Try $ date -u
or
Usage: sie_get channel "now" minutesBack
Example: sie_get 212 now 20
Note: Generally: 0 < minutesBack <= 720 (and integer valued)
TERMINATED
Q3) “Why do you say that ‘UTC datetimes must within half a day or so” and minutesBack should be between 0 and 720?
A3) SIE Batch maintains a buffer of data sent over various SIE channels, typically at least half a day or so. If you attempt to ask for data from last week/last month/last year, the data you’re asking for won’t still be in that cache. (If you review the SIE Batch API, you’ll see that you can make another call via the API that will give you explicit details about the available data for each channel, but since this is just a simple sample client, we suggest using the stated rule of thumb instead).
Note: long intervals will result in very large files on some of the busier channels.
Q4) “How big ARE the files that I’ll be getting?”
A4) Traffic levels varies by channel and from day-to-day and hour-to-hour. The eagle-eyed will notice that volume estimates are available under the channel names in the web interface to SIE Batch. An easy way to get a sense for traffic levels is to pull a small sample or two by way of example:
As you can see from that display, five minutes of Ch208 (from 21:39:44 to 21:44:44) totaled 773.1 megabytes (even in very space-efficient binary
nmsg
format).
On the other hand, fifteen minutes of Ch212 (from 19:46:42 to 20:01:42) was just 548 kilobytes (even in the more-verbose JSON Lines format).
Q5) ‘How do I work with nmsg files? They’re binary rather than text format, aren’t they? Can I convert them into some sort of presentation (“human readable”) format?’
A5)
nmsg
format files from SIE Batch can be read using nmsgtool. For example, perhaps we’ve grabbed two minutes of Ch204:
$ sie_get 204 now 2
$ ls -lh sie-ch*
-rw-r--r-- 1 jsmith staff 190M Jan 10 00:04 sie-ch204-{2020-01-10@08:01:32}-{2020-01-10@08:03:32}.nmsg
To turn that
nmsg
data into human-readable presentation format, we could say:
$ nmsgtool -r "sie-ch204-{2020-01-10@08:01:32}-{2020-01-10@08:03:32}.nmsg" -o -
[...]
[90] [2020-01-10 08:01:59.971034147] [2:1 SIE dnsdedupe] [00000000] [] []
type: INSERTION
count: 1
time_first: 2020-01-10 08:00:52
time_last: 2020-01-10 08:00:52
response_ip: 217.160.81.29
bailiwick: lunar-exploration.net.
rrname: www.lunar-exploration.net.
rrclass: IN (1)
rrtype: A (1)
rrttl: 3600
rdata: 74.208.236.115
[...]
Q6) “What if I need JSON Lines rather than presentation format?”
A6) To get
JSON
Lines format, add the
-J -
option to the end of the
nmsgtool
command:
$ nmsgtool -r "sie-ch204-{2020-01-10@08:01:32}-{2020-01-10@08:03:32}.nmsg" -J -
[...]
{"time":"2020-01-10 08:01:59.971034147","vname":"SIE","mname":"dnsdedupe","message": {"type":"INSERTION","count":1,"time_first":"2020-01-10 08:00:52","time_last":"2020-01-10 08:00:52","response_ip":"217.160.81.29","bailiwick":"lunar-exploration.net.","rrname":
"www.lunar-exploration.net.","rrclass":"IN","rrtype":"A","rrttl":3600,"rdata":["74.208.236.115"]}}
[...]
[Note: new lines (carriage returns) have been added to the above text to break it up so it will better fit this document’s format — in reality each observation from
nmsgtool
will be on a single long line.]
Q7) “Okay, I’ve got JSON Lines format output — now what can I do?”
A7) Once you’re using JSON Lines format, you can easily see how many observations are in your data file:
$ nmsgtool -r "sie-ch204-{2020-01-10@08:01:32}-{2020-01-10@08:03:32}.nmsg" -J - | wc -l
2883963
Note that’s nearly 2.9 million observations from just a two minute chunk of Channel 204!
You can also use
to extract a select field of interest, e.g.:
$ nmsgtool -r "sie-ch204-{2020-01-10@08:01:32}-{2020-01-10@08:03:32}.nmsg" -J - | jq -r '.message.rrname'
You can also use
jq
with
nmsgtool
to extract multiple fields, simplifying subsetting. For example, if you wanted to drop all records with RRSIG or NSEC3 rrtypes, and skip all .in-addr.arpa rrnames:
$ nmsgtool -r "sie-ch204-{2020-01-10@08:01:32}-{2020-01-10@08:03:32}.nmsg" -J - | jq -r '"\(.message.rrname) \(.message.rrtype)"' | grep -v " RRSIG" | grep -v " NSEC3" | grep -v ".in-addr.arpa. "
10. Conclusion
We hope you’ve found this an interesting example of how you can write a little command line client using the SIE Batch API.
We encourage you to give SIE Batch a try, and welcome your feedback.
Note: This Source Code Form is subject to the terms of the Mozilla Public License, v.2.0. If a copy of the MPL was not distributed with this file. You can obtain one here.
Appendix I. sie_get.c
/* Note: change EndpointHostnameHere to the hostname you're given by Farsight at provisioning */
/* sie_get.c */
#include <stdlib.h>
#include <string.h>
#include <time.h>
#include <curl/curl.h>
#define ENDPOINTHERE hostnameProvidedToYouByFarsight
/* write_data function from https://curl.haxx.se/libcurl/c/url2file.html */
static size_t
write_data(void * ptr, size_t size, size_t nmemb, void * stream)
{
size_t written = fwrite(ptr, size, nmemb, (FILE *) stream);
return written;
}
/* function declarations */
extern char *
getAPIkey(void);
/* main **************************************************************** */
#define BIG 200
int
main(int argc, char ** argv)
{
/* declarations (alphabetically ordered by variable name) */
char arguments[BIG];
int chanflag = 0;
CURL * curl_handle;
char currentGMTtimestring[BIG];
char enddatetime[BIG];
time_t epoch;
char * filetype = NULL;
const char * fmt = "%Y-%m-%d %H:%M:%S";
char * fullcommand;
struct tm * gmt;
unsigned long i;
size_t lengthofstring1;
size_t lengthofstring2;
char * myapikey = NULL;
int mysecondsback = 0;
static char * nowliteral = "now";
FILE * outputfile;
char outputfilename[BIG];
CURLcode res;
char startdatetime[BIG];
struct tm * start_from_offset_val;
char string1[BIG];
char string2[BIG];
time_t t;
/* if we don't have the number of arguments we expect, provide precis */
if (argc != 4) {
printf("\nsie_get\n\n");
printf("ERROR: Wrong number of command line arguments supplied.\n\n");
printf(" Usage: sie_get channel \"startdatetime\" \"enddatetime\"\n");
printf(" Example: sie_get 212 \"2020-01-07 00:13:00\" \"2020-01-07 00:28:00\"\n");
printf(" Notes: UTC datetimes must within half a day or so and must be double quoted.\n");
printf(" Zero pad any single digit months, days, hours or minutes.\n");
printf(" Seconds must be part of the UTC datetimes (but are ignored.)\n");
printf(" Need the current UTC time? Try $ date -u\n");
printf("or\n\n");
printf(" Usage: sie_get channel \"now\" minutesBack\n");
printf(" Example: sie_get 212 now 20\n");
printf(" Note: Generally: 0 < minutesBack <= 720 (and integer valued)\n\n");
printf("TERMINATED\n\n");
exit(EXIT_FAILURE);
}
/* get the command line arguments */
chanflag = atoi(argv[1]);
char * chanflagstring = strdup(argv[1]);
strlcpy(startdatetime, argv[2], sizeof startdatetime);
strlcpy(enddatetime, argv[3], sizeof enddatetime);
/* save the current UTC time */
t = time(NULL);
gmt = gmtime(&t);
strftime(currentGMTtimestring, sizeof currentGMTtimestring, fmt, gmt);
/* get and validate the user's SIE-Batch API key */
myapikey = getAPIkey();
/* allow either specify absolute datetime strings, or "now" and */
/* integer minutesBack. this block handles the 2nd case */
if (strncmp(startdatetime, nowliteral, 3) == 0) {
/* enddatetime is actually minutesBack field saying how far back
* we want to go from "now", so we save it as such in a new int var */
mysecondsback = atoi(enddatetime) * 60;
/* set the endtime to the current GMT time (as a human datetime) */
/* (replacing the number of minutes specified by the user) */
strlcpy(enddatetime, currentGMTtimestring, sizeof enddatetime);
strlcpy(string2, currentGMTtimestring, sizeof string2);
/* set the start time to the calculated earlier human datetime, */
/* replacing "now" */
epoch = t - mysecondsback;
start_from_offset_val = gmtime(&epoch);
strftime(startdatetime, sizeof(startdatetime), fmt, start_from_offset_val);
strlcpy(string1, startdatetime, sizeof string1);
} else {
/* in the explicit form, we've already startdatetime and enddatetime */
/* we just need a copy we can munge to form the filename */
strlcpy(string1, startdatetime, sizeof string1);
strlcpy(string2, enddatetime, sizeof string2);
}
/* don't like spaces in filenames. (this is a framework for fixing other */
/* unwanted chars, should we need to do so, too). we'll fix start and end */
strlcpy(string1, startdatetime, sizeof string1);
lengthofstring1 = strlen(string1);
for (i = 0; i <= lengthofstring1; i++) {
if (string1[i] == ' ') string1[i] = '@';
}
strlcpy(string2, enddatetime, sizeof string2);
lengthofstring2 = strlen(string2);
for (i = 0; i <= lengthofstring2; i++) {
if (string2[i] == ' ') string2[i] = '@';
}
/* we want a file extension reflective of the file's contents. A number */
/* of channels use native NMSG format, the rest are JSON Lines */
if ((chanflag == 204) ||
(chanflag == 206) ||
(chanflag == 207) ||
(chanflag == 208) ||
(chanflag == 221))
filetype = ".nmsg";
else filetype = ".jsonl";
/* now build the output filename by concatenating elements into a name */
strlcpy(outputfilename, "sie-ch", sizeof outputfilename);
strlcat(outputfilename, chanflagstring, sizeof outputfilename);
strlcat(outputfilename, "-{", sizeof outputfilename);
strlcat(outputfilename, string1, sizeof outputfilename);
strlcat(outputfilename, "}-{", sizeof outputfilename);
strlcat(outputfilename, string2, sizeof outputfilename);
strlcat(outputfilename, "}", sizeof outputfilename);
strlcat(outputfilename, filetype, sizeof outputfilename);
/* initialize curl. this must be called once and only once. */
if (curl_global_init(CURL_GLOBAL_ALL) != 0) {
printf("curl_global_init() failed\n");
exit(EXIT_FAILURE);
}
/* the POST we're going to be doing needs some arguments to work */
/* we'll assemble those arguments now */
strlcpy(arguments, "{\"apikey\":\"", sizeof arguments);
strlcat(arguments, myapikey, sizeof arguments);
strlcat(arguments, "\",\"channel\":", sizeof arguments);
strlcat(arguments, chanflagstring, sizeof arguments);
strlcat(arguments, ",\"start_time\":\"", sizeof arguments);
strlcat(arguments, startdatetime, sizeof arguments);
strlcat(arguments, "\",\"end_time\":\"", sizeof arguments);
strlcat(arguments, enddatetime, sizeof arguments);
strlcat(arguments, "\"}", sizeof arguments);
curl_handle = curl_easy_init();
fullcommand = "https://EndpointHostNameHere/siebatchd/v1/siebatch/chfetch";
curl_easy_setopt(curl_handle, CURLOPT_URL, fullcommand);
curl_easy_setopt(curl_handle, CURLOPT_POSTFIELDS, arguments);
/* we're going to write the curl output to the file with the odd name */
curl_easy_setopt(curl_handle, CURLOPT_WRITEFUNCTION, write_data);
outputfile = fopen(outputfilename, "wb");
if (outputfile) {
/* write the page body to this file handle */
curl_easy_setopt(curl_handle, CURLOPT_WRITEDATA, outputfile);
/* do the actual curl command */
res = curl_easy_perform(curl_handle);
if (res != CURLE_OK) {
return (EXIT_FAILURE);
}
fclose(outputfile);
}
curl_easy_cleanup(curl_handle);
curl_global_cleanup();
free(myapikey);
return (EXIT_SUCCESS);
} /* main */
Appendix II. getAPIkey.c
/* getAPIkey */
/* Note: change EndpointHostnameHere to the hostname you're given by Farsight at provisioning */
#include <stdlib.h>
#include <string.h>
#include <curl/curl.h>
#include <time.h>
#include <json.h>
size_t len = 0;
int apikeylen;
FILE * stream;
char * myapikey;
CURL * curl_handle;
CURLcode res;
char arguments[500];
char * arguments1 = "{\"apikey\":\"";
char * arguments2 = NULL;
char * arguments3 = "\"}";
/* download to memory: see https://curl.haxx.se/libcurl/c/getinmemory.html */
struct MemoryStruct {
char * memory;
size_t size;
};
static size_t
WriteMemoryCallback(void * contents, size_t size, size_t nmemb, void * userp)
{
size_t realsize = size * nmemb;
struct MemoryStruct * mem = (struct MemoryStruct *) userp;
char * ptr = realloc(mem->memory, mem->size + realsize + 1);
if (ptr == NULL) {
/* out of memory! */
printf("not enough memory (realloc returned NULL)\n");
return 0;
}
mem->memory = ptr;
memcpy(&(mem->memory[mem->size]), contents, realsize);
mem->size += realsize;
mem->memory[mem->size] = 0;
return realsize;
}
/* prototype */
char *
getAPIkey(void);
char *
getAPIkey(void)
{
/* declarations (in alphabetical order) */
static char * apikeybasefilename = ".sie-get-key.txt";
static char * arguments1 = "{\"apikey\":\"";
struct MemoryStruct chunk;
char * filepath = NULL;
static char * fullcommand = "https://EndpointHostnameHere/siebatchd/v1/validate";
char * home_dir = NULL;
struct json_object * new_obj;
struct json_object * new_obj_extract;
size_t size;
static char * slash = "/";
int status_code = 0;
home_dir = getenv("HOME");
size = strlen(home_dir) + strlen(slash)
+ strlen(apikeybasefilename) + 1;
filepath = malloc(size);
snprintf(filepath, size, "%s%s%s", home_dir, slash, apikeybasefilename);
chunk.memory = malloc(1); /* will be grown as needed by the realloc above */
chunk.size = 0; /* no data at this point */
stream = fopen(filepath, "r");
if (stream == NULL) {
printf("ERROR: no API key file at %s\n", filepath);
exit(EXIT_FAILURE);
}
getline(&myapikey, &len, stream);
fclose(stream);
/* clean up the sie-batch apikey */
/* remove newline if present */
apikeylen = strlen(myapikey);
if (myapikey[apikeylen - 1] == '\n')
myapikey[apikeylen - 1] = 0;
/* validate sie-batch apikey */
curl_handle = curl_easy_init();
strlcpy(arguments, arguments1, sizeof arguments);
strlcat(arguments, myapikey, sizeof arguments);
strlcat(arguments, arguments3, sizeof arguments);
curl_easy_setopt(curl_handle, CURLOPT_URL, fullcommand);
/* send all data to this function */
curl_easy_setopt(curl_handle, CURLOPT_WRITEFUNCTION, WriteMemoryCallback);
/* we pass our 'chunk' struct to the callback function */
curl_easy_setopt(curl_handle, CURLOPT_WRITEDATA, (void *) &chunk);
curl_easy_setopt(curl_handle, CURLOPT_POSTFIELDS, arguments);
res = curl_easy_perform(curl_handle);
if (res != CURLE_OK) {
printf("curl_easy_perform() failed: %s\n",
curl_easy_strerror(res));
} else {
new_obj = json_tokener_parse(chunk.memory);
new_obj_extract = json_object_object_get(new_obj, "status");
status_code = json_object_get_int(new_obj_extract);
if (status_code != 0) {
printf("%d ", status_code);
new_obj_extract = json_object_object_get(new_obj, "_message");
printf("%s\n", json_object_to_json_string(new_obj_extract));
}
}
curl_easy_cleanup(curl_handle);
free(chunk.memory);
curl_global_cleanup();
free(filepath);
if (status_code != 0) exit(EXIT_FAILURE);
return myapikey;
} /* getAPIkey */
Appendix III. Makefile
EXEC = sie_get
PREFIX = /usr/local
JSON_C_DIR=/usr/local/Cellar/json-c/0.13.1
CC = clang
CFLAGS = -O3 -Wall
CPPFLAGS = -I /usr/local/include
LDFLAGS = -L /usr/local/lib -Wall
LDLIBS = -lcurl -ljson-c
CFLAGS += -I$(JSON_C_DIR)/include/json-c
LDFLAGS+= -L$(JSON_C_DIR)/lib
.SUFFIXES:
.SUFFIXES: .c .o
objects = sie_get.o getAPIkey.o
all: $(objects)
$(CC) -o $(EXEC) $(objects) $(LDFLAGS) $(LDLIBS)
sie_get.o :
getAPIkey.o :
.PHONY: install
install:
mkdir -p $(PREFIX)/bin
cp sie_get $(PREFIX)/bin/.
chmod a+rx $(PREFIX)/bin/$(EXEC)
.PHONY: clean
clean:
@rm -f $(EXEC) $(objects)
Joe St Sauver Ph.D. is a Distinguished Scientist with Farsight Security®, Inc.
Read the next part in this series: New Whitepaper: Working With The SIE Batch API: A Command Line Client In Ruby, Perl, Python and C
Heading
