SIE Batch REST API Reference
Conventions
Every response from the SIE Batch API will contain two status indicating keys. The first
is _status which is a string that will contain either OK or NOK indicating whether
the API was successful or not. The second field is _message which will contain a short
description of what went wrong. If the _status is OK, then _message will be present
but empty.
If _status is NOK then additional fields giving details of the exception will be
present, conforming to RFC 7807 .
Exception response examples are provided for each API below where appropriate.
Authentication
Validate – Validate an API Key
Validates a given API Key, returns profile information associated with it.
POST
https://batch.sie-remote.net/siebatchd/v1/validate
Permission: public
Example cURL
APIKEY=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
curl -d '{"apikey":"'$APIKEY'"}' https://batch.sie-remote.net/siebatchd/v1/validate
Example JS XMLHttpRequest
const apikey = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx";
let xhr = new XMLHttpRequest();
xhr.open("POST", "https://batch.sie-remote.net/siebatchd/v1/validate");
xhr.send(JSON.stringify({apikey: apikey}));
xhr.onload = function() {
do_something();
};
xhr.onprogress = function(event) {
do_something_else();
};
xhr.onerror = function() {
do_something_error();
};
Example JS React, Axios
const apikey = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx";
axios({
url: "https://batch.sie-remote.net/siebatchd/v1/validate",
method: "POST",
data: {
apikey: apikey,
},
}).then(
response => {
if(response.status === 200) {
do_something();
} else {
do_something_else();
}
},
error => {
do_something_error();
}
);
Example Object
{
"apikey": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
}
Parameter
| Field | Type | Description |
|---|---|---|
| profile | Dictionary | A dictionary containing user profile information (provisioning). The following keys are only some of the possible keys available. |
| username | String | The username associated with your API Key. |
| siebatch | Dictionary | The SIE batch channels you have access to. |
| _status | String | OK |
| _message | String | An account status message (“” [empty] if no message). |
Success-Response
HTTP/1.1 200 OK
{
"profile": {
"username": "siebatch-customer",
"siebatch": {
"ch212": {
"description": "Newly Observed Domains"
},
"ch213": {
"description": "Newly Observed Fully Qualified Domain Names"
},
}
},
"_status": "OK",
"_message": ""
}
Error 4xx 5xx
| Field | Type | Description |
|---|---|---|
| apikey | String | Your Farsight SIE Batch API Key |
| channels | List | List (array) of one or more channel numbers (integers). Valid channels are 1-255. |
MissingAPIKey
HTTP/1.1 403
{
"_message": "API key not present",
"_status": "NOK",
"logid": "30fd25e0-0130-4ab1-a375-fabf1d57430e",
"status": 403,
"type": "missing-api-key"
}Batched Channel Access
siebatch chdetails – Get Channel Details
Request the details of a channel such as possible datetime ranges, download filetype, and
the size of data available.
These details will update as data is stored and pruned on the server every minute; channel data is always in motion so the details represent a moving window of data that is available for download. This means that the size and earliest/latest times will change frequently. Because channels are continuously recording new data it’s not possible to always be up to date if you were to repeatedly request the latest “earliest” and “latest” details for a channel; it’s an archival reference only.
POST
https://batch.sie-remote.net/siebatchd/v1/siebatch/chdetailsPermission: user
Example cURL
APIKEY=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
curl -d '{"apikey":"'$APIKEY'","channels":[212, 213]}' \
-X POST https://batch.sie-remote.net/siebatchd/v1/siebatch/chdetailsExample JS XMLHttpRequest
const apikey = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx";
let xhr = new XMLHttpRequest();
xhr.open("POST", "https://batch.sie-remote.net/siebatchd/v1/siebatch/chdetails");
xhr.send(JSON.stringify({apikey: apikey, channels: [212, 213]}));
xhr.onload = function() {
do_something();
};
xhr.onprogress = function(event) {
do_something_else();
};
xhr.onerror = function() {
do_something_error();
};Example React, Axios
const apikey = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx";
axios({
url: "https://batch.sie-remote.net/siebatchd/v1/siebatch/chdetails",
method: "POST",
data: {
apikey: apikey,
channels: [212, 213]
},
}).then(
response => {
if(response.status === 200) {
do_something();
} else {
do_something_else();
}
},
error => {
do_something_error();
}
);Example Object
{
"apikey": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
"channels": [212, 213]
}Parameter
| Field | Type | Description |
|---|---|---|
| apikey | String | Your Farsight SIE Batch API Key |
| channels | List | List (array) of one or more channel numbers (integers). Valid channels are 1-255. |
Success 200
| Field | Type | Description |
|---|---|---|
| _status | String | NOK |
| _message | String | A short description of what went wrong. |
| status | Integer | HTTP status code |
| type | String | A hyphenated mnemonic for the error. This can be used to key on a local i18n translation. |
| detail | String | Optional String that contains more details about the specific error. |
Success Response Example
HTTP/1.1 200 OK
{
"channels": {
"ch212": {
"earliest":"2020-01-28 00:40:00",
"latest":"2020-01-28 20:06:00",
"size":83837922,
"mimetype": "application/x-ndjson"
},
"ch213": {
"earliest":"2020-01-28 00:40:00",
"latest": "2020-01-28 20:06:01",
"size":10397567992,
"mimetype": "application/x-ndjson"
}
},
"_status": "OK",
"_message":""
}Error 4xx 5xx
| Name | Type | Description |
|---|---|---|
| _status | String | NOK |
| _message | String | A short description of what went wrong. |
| status | Integer | HTTP status code |
| type | String | A hyphenated mnemonic for the error. This can be used to key on a local i18n translation. |
| detail | String | Optional String that contains more details about the specific error. |
SessionFailure
HTTP/1.1 403
{
"_message": "<Message will contain details>",
"_status": "NOK",
"logid": "30fd25e0-0130-4ab1-a375-fabf1d57430e",
"status": 403,
"type": "session-failure"
}
InvalidParameters
HTTP/1.1 403
{
"_message": "<message will be specific wrt to which parameter(s) is missing>",
"_status": "NOK",
"logid": "30fd25e0-0130-4ab1-a375-fabf1d57430e",
"status": 403,
"type": "invalid-parameters",
"invalid_parameters": {
"name": "foobar",
"reason": "should be an integer but received a string"
}
}
MissingAPIKey
HTTP/1.1 403
{
"_message": "API key not present",
"_status": "NOK",
"logid": "30fd25e0-0130-4ab1-a375-fabf1d57430e",
"status": 403,
"type": "missing-api-key"
}
siebatch chfetch – Fetch Channel Data
Download data associated with a given channel.
As mentioned in chdetails, the earliest data available is pruned frequently, so you are not guaranteed to retrieve that data if you request it.
The seconds portion of the start/end times are ignored, but for API consistency, we require seconds to be given. This means that if you ask for “12:10:30” thru “12:45:30” you will get “12:10:00” thru “12:45:00”.
If no data is available then an HTTP 404 is returned – otherwise it returns 200, sets the appropriate HTTP headers, and the download begins.
POST
https://batch.sie-remote.net/siebatchd/v1/siebatch/chfetch
Permission: User
Example cURL
APIKEY=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
curl -d '{"apikey":"'$APIKEY'","channel":213,"start_time":"2020-01-28 12:00:00","end_time":"2020-01-28 12:01:00"}' \
-X POST https://batch.sie-remote.net/siebatchd/v1/siebatch/chfetch
Example JS XMLHttpRequest
const apikey = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx";
let xhr = new XMLHttpRequest();
xhr.open("POST", "https://batch.sie-remote.net/siebatchd/v1/siebatch/chfetch");
xhr.send(JSON.stringify({apikey: apikey, channel: 213, start_time: "2020-01-28 12:00:00",end_time:"2020-01-28 12:01:00"}));
xhr.onload = function() {
do_something();
};
xhr.onprogress = function(event) {
do_something_else();
};
xhr.onerror = function() {
do_something_error();
};
Example JS React, Axios
const apikey = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx";
axios({
url: "https://batch.sie-remote.net/siebatchd/v1/siebatch/chfetch",
method: "POST",
data: {
apikey: apikey,
channel: 213,
start_time: "2020-01-28 12:00:00",
end_time: "2020-01-28 12:01:00",
},
}).then(
response => {
if(response.status === 200) {
do_something();
} else {
do_something_else();
}
},
error => {
do_something_error();
}
);
Example Object
{
"apikey": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
"channel": 213,
"start_time": "2020-01-28 12:00:00",
"end_time": "2020-01-28 12:01:00"
}
Parameter
| Field | Type | Description |
|---|---|---|
| _status | String | NOK |
| _message | String | A short description of what went wrong. |
| status | Integer | HTTP status code |
| type | String | A hyphenated mnemonic for the error. This can be used to key on a local i18n translation. |
| detail | String | Optional String that contains more details about the specific error. |
Error 4xx 5xx
| Name | Type | Description |
|---|---|---|
| _status | String | NOK |
| _message | String | A short description of what went wrong. |
| status | Integer | HTTP status code |
| type | String | A hyphenated mnemonic for the error. This can be used to key on a local i18n translation. |
| detail | String | Optional String that contains more details about the specific error. |
SessionFailure
HTTP/1.1 403
{
"_message": "<Message will contain details>",
"_status": "NOK",
"logid": "30fd25e0-0130-4ab1-a375-fabf1d57430e",
"status": 403,
"type": "session-failure"
}
InvalidParameters
HTTP/1.1 403
{
"_message": "<message will be specific wrt to which parameter(s) is missing>",
"_status": "NOK",
"logid": "30fd25e0-0130-4ab1-a375-fabf1d57430e",
"status": 403,
"type": "invalid-parameters",
"invalid_parameters": {
"name": "foobar",
"reason": "should be an integer but received a string"
}
}
MissingAPIKey
HTTP/1.1 403
{
"_message": "API key not present",
"_status": "NOK",
"logid": "30fd25e0-0130-4ab1-a375-fabf1d57430e",
"status": 403,
"type": "missing-api-key"
}
siebatch makeurl – Make URL
Generate a URL that can be used to download data associated with a channel later or independently of the SIE Batch webserver.
As mentioned in chdetails, the earliest data available is pruned frequently, so you are not guaranteed to retrieve it if it’s about to be pruned.
The seconds portion of the start/end times are ignored, but for API consistency, we require seconds to be given. This means that if you ask for “12:10:30” thru “12:45:30” you will get “12:10:00” thru “12:45:00”.
Using makeurl as opposed to chfetch is advantageous as generated URLs can be more easily shared and accessed later on different computers and by fellow users, independently of the user that originally requested it to be generated and without credentials being revealed. Generated URLs are the preferred method for sharing snippets of channel data.
The expire keyword allows you to specify how long the URL is valid for. The maximum is 168 hours (1 week), the minimum is 1 hour. If you specify an expiration that does not make sense then a 404 will be returned. For example, if a channel collects the last 12 hours of data and you request a URL with an expiration of 13 hours then no data would be available for the final hour that the URL is valid. As a rule of thumb, the maximum requested expiration time should be no more than chXXX[“latest”] – chXXX[“earliest”] where chXXX is the channel you’re generating URLs for.
POST
https://batch.sie-remote.net/siebatchd/v1/siebatch/makeurl
Permission: user
Example cURL
APIKEY=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
curl -d '{"apikey":"'$APIKEY'","channel":213,"start_time":"2020-01-28 12:00:00","end_time":"2020-01-28 12:01:00"}' \
-X POST https://batch.sie-remote.net/siebatchd/v1/siebatch/makeurl
Example JS XMLHttpRequest
const apikey = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx";
let xhr = new XMLHttpRequest();
xhr.open("POST", "https://batch.sie-remote.net/siebatchd/v1/siebatch/makeurl");
xhr.send(JSON.stringify({apikey: apikey, channel: 213, start_time: "2020-01-28 12:00:00",end_time:"2020-01-28 12:01:00"}));
xhr.onload = function() {
do_something();
};
xhr.onprogress = function(event) {
do_something_else();
};
xhr.onerror = function() {
do_something_error();
};
Example JS React, Axios
const apikey = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx";
axios({
url: "https://batch.sie-remote.net/siebatchd/v1/siebatch/makeurl",
method: "POST",
data: {
apikey: apikey,
channel: 213,
start_time: "2020-01-28 12:00:00",
end_time: "2020-01-28 12:01:00"
},
}).then(
response => {
if(response.status === 200) {
do_something();
} else {
do_something_else();
}
},
error => {
do_something_error();
}
);
Example Object
{
"apikey": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
"channel": 213,
"start_time": "2020-01-28 12:00:00",
"end_time": "2020-01-28 12:01:00"
}
Parameter
| Field | Type | Description |
|---|---|---|
| _status | String | OK |
| _message | String | “” (empty if no message) |
| url | String | The presigned shareable URL to the specified SIE data |
Success 200
| Field | Type | Description |
|---|---|---|
| _status | String | OK |
| _message | String | “” (empty if no message) |
| url | String | The presigned shareable URL to the specified SIE data |
Success-Response
HTTP/1.1 200 OK
{"_status": "OK",
"_message": "",
"url": "https://batch-dl.sie-remote.net/range/ch213/20200128.1200.20200128.1201?
X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=..."
}Error 4xx 5xx
| Name | Type | Description |
|---|---|---|
| _status | String | NOK |
| _message | String | A short description of what went wrong. |
| status | Integer | HTTP status code |
| type | String | A hyphenated mnemonic for the error. This can be used to key on a local i18n translation. |
| detail | String | Optional String that contains more details about the specific error. |
SessionFailure
HTTP/1.1 403
{
"_message": "<Message will contain details>",
"_status": "NOK",
"logid": "30fd25e0-0130-4ab1-a375-fabf1d57430e",
"status": 403,
"type": "session-failure"
}InvalidParameters
HTTP/1.1 403
{
"_message": "<message will be specific wrt to which parameter(s) is missing>",
"_status": "NOK",
"logid": "30fd25e0-0130-4ab1-a375-fabf1d57430e",
"status": 403,
"type": "invalid-parameters",
"invalid_parameters": {
"name": "foobar",
"reason": "should be an integer but received a string"
}
}MissingAPIKey
HTTP/1.1 403
{
"_message": "API key not present",
"_status": "NOK",
"logid": "30fd25e0-0130-4ab1-a375-fabf1d57430e",
"status": 403,
"type": "missing-api-key"
}
