Groups

Binary Parser

Binary Parser is a function for Soracom Air devices that parses binary data with a fixed format and converts the data into easy-to-use JSON data, which can then be passed to Soracom Beam, Funnel, Funk, and Harvest Data. Binary Parser is available for Air for Cellular, Air for Sigfox, and Air for LoRaWAN.

When using Binary Parser, you must select either a Predefined Format or define a Custom Format.

How Data is Parsed

With the Binary Parser option enabled, the binary payload sent from a device to Beam, Funnel, Funk, Harvest Data, or Unified Endpoint is automatically parsed into a JSON object, and the resulting JSON data is sent to the configured destination. There is no need to modify the Beam, Funnel, Funk, or Harvest Data destination configuration.

When binary data is converted to JSON, a binaryParserEnabled: true attribute will be added to the JSON data.

Configuration

The Binary Parser option is found within Soracom Air Group settings.

1. Login to the User Console. From the Menu, open the Groups screen.

2. From the list of groups, click the name of the group you want to configure to open its settings page.

3. From the Basic Settings tab, click the SORACOM Air for Cellular, SORACOM Air for Sigfox, or SORACOM Air for LoRaWAN panel to expand its settings.

   https://console.soracom.io

   

4. Enable the Binary Parser option by switching the option to ON.

5. Enter the Binary Parser Format to use.

6. Click Save at the bottom of the panel.

Options

• Format - A Predefined Format or Custom Format string that defines how Binary Parser should parse binary data.

Predefined Format

Predefined formats are available for select devices and libraries. When using a supported device or library, simply enter the Format string when configuring Binary Parser settings.

Supported devices:

Refer to the Soracom blog for an example of using Binary Parser with a supported device.

Custom Format

When defining a custom format, individual values can be parsed from the payload using the following definition:

[name]:[byte index]:[type]:[options]

Multiple values can be parsed by creating multiple definitions, each separated with a space.

For example, the following Format definition will produce a JSON object that looks like this:

flag:0:bool:7 temp:1:int:13:/10 humid:3:uint:8:/100 lat::float:32 long::float:32

{
  "flag": true,
  "temp": 18.0,
  "humid": 80.0,
  "lat": 39.8283,
  "long": 98.5795
}

Name

The JSON object key where the parsed value should be stored. Available characters: letters, numbers, -, _, #, and $.

Note that the following words cannot be used, as they are reserved for use in Sigfox and LoRaWAN paylods:

• Sigfox reserved words: device, time, station, lat, lng, rssi, data, seqNumber, and data
• LoRaWAN reserved words: date, gatewayData, data, and deveui

Byte Index

Defines which byte of the payload should be read in order to parse the value, with 0 corresponding to the first byte of the payload.

Type

Defines the type to use for the value when parsing data. The following value types are available:

• bool - returns true or false
• int - returns a signed integer
• uint - returns an unsigned integer
• float - returns a decimal value
• char - returns an ASCII character

Options

Available options vary depending on the defined type. For types that support multiple options, each option should be separated using a colon :.

bool

int, uint

float

char

Advanced Formatting

Endianness

By default, Binary Parser will parse numerical values using big-endian format, with most-significant bit first and least-significant bit last. You can override this behavior to parse the bit sequence using little-endian by appending :little-endian to the end of the format definition.

Example: value:0:float:64:little-endian

Byte Index Shorthand

Normally, the byte index defines which byte in the payload contains the value to be parsed. However, the byte index can be left blank using ::, which will cause the Binary Parser to use the next byte as the byte index, following the previously-used byte index.

For example, with 2-byte payload and a definition value1:0:int:6 value2::int:8:

• value1 will be parsed using the first 6 bits of the first byte (byte index: 0), and
• value2 will be parsed using all 8 bits of the second byte;
• the last two bits of the first byte are ignored

However, when using the byte index shorthand with multiple consecutive definitions that are type bool, Binary Parser will re-use the previously-used byte index. Any non-bool definition after a sequence of bool definitions will resume the normal byte index shorthand behavior of using the next byte index.

This allows you to read each of the individual bits of a byte as separate boolean values, using a definition such as:

flagA:0:bool:7 flagB::bool:6 flagC::bool:5 value:0:int:5:4

This format will parse the following values:

• flagA will be parsed using the first bit (byte offset: 7) of the first byte (byte index: 0);
• flagB will be parsed using the second bit (byte offset: 6) of the same byte (byte index shorthand);
• flagC will be parsed using the third bit (byte offset: 5) of the same byte (byte index shorthand);
• value will be parsed using the last five bits (byte offset: 4, length: 5) of the first byte (byte index: 0)

Note that when using the byte index shorthand with multiple type bool definitions, Binary Parser will not keep track of bit offsets, and will continue to re-use the same byte index for all consecutive type bool definitions. For example, to parse the first ten bits of a 2-byte sequence as boolean values, you can use the byte index shorthand for the 2nd-8th definitions. However, you must re-specify the byte index in the 9th definition in order to move the byte index forward. Afterwards, you can use the byte index shorthand again for the 10th definition.

Limitations

• When using Binary Parser with Beam, the following protocol conversions are supported:
  • TCP to HTTP/HTTPS entry point
  • UDP to HTTP/HTTPS entry point
• When using Binary Parser with Funnel or Harvest, the following forwarding protocols are supported:
  • TCP
  • UDP
• While there is no forwarding byte size limit, network MTU may affect parsing.
• Binary Parser cannot be combined with Soracom Binary Format v1.

Error Handling

Any parse errors are stored in an errors key within the JSON object.

Soracom does not provide any tools for inspecting or debugging binary data.

Advanced Configuration

Binary Parser settings can also be configured through the Soracom API or CLI by using the SoracomAir namespace.

Configuration Structure

  "binaryParserEnabled": true|false,
  "binaryParserFormat": "",

Parameters

Enable or disable Binary Parser:

• key - binaryParserEnabled
• value (boolean, default: false) - Enables or disables the Binary Parser option.

Modify the Binary Parser format:

• key - binaryParserFormat
• value (string, default: "") - Defines the parser format to use when Binary Parser is enabled.

Example

The following configuration will set and enable Binary Parser:

[
  {
    "key": "binaryParserEnabled",
    "value": true
  },
  {
    "key": "binaryParserFormat",
    "value": "flag:0:bool:7 temp:1:int:13:/10 humid:3:uint:8:/100 lat::float:32 long::float:32"
  }
]

Programmatic Usage

You can use the advanced configuration structure to configure the Binary Parser option programmatically using the Soracom API and Soracom CLI.

curl -X PUT \
>  -H 'X-Soracom-API-Key: <MY-API-KEY>' \
>  -H 'X-Soracom-Token: <MY-TOKEN>' \
>  -H 'Content-Type: application/json' \
>  -d '[
>        {
>          "key": "binaryParserEnabled",
>          "value": true
>        },
>        {
>          "key": "binaryParserFormat",
>          "value": "flag:0:bool:7 temp:1:int:13:/10 humid:3:uint:8:/100 lat::float:32 long::float:32"
>        }
>      ]' \
>  https://g.api.soracom.io/v1/groups/<GROUP-ID>/configuration/SoracomAir

curl -X PUT \
>  -H 'X-Soracom-API-Key: <MY-API-KEY>' \
>  -H 'X-Soracom-Token: <MY-TOKEN>' \
>  -H 'Content-Type: application/json' \
>  -d '[
>        {
>          "key": "binaryParserEnabled",
>          "value": true
>        },
>        {
>          "key": "binaryParserFormat",
>          "value": "flag:0:bool:7 temp:1:int:13:/10 humid:3:uint:8:/100 lat::float:32 long::float:32"
>        }
>      ]' \
>  https://jp.api.soracom.io/v1/groups/<GROUP-ID>/configuration/SoracomAir

soracom groups put-config --group-id "<GROUP-ID>" --namespace "SoracomAir" --body "<CONFIG>" --coverage-type g

soracom groups put-config --group-id "<GROUP-ID>" --namespace "SoracomAir" --body "<CONFIG>" --coverage-type jp