Soracom Harvest

Configuration

Soracom Harvest settings are found in Soracom Air or Soracom Inventory group settings. To send data to Harvest, refer to the Harvest Data section below. To upload files to Harvest, refer to the Harvest Files section below.

Harvest Data

Harvest Data settings are shared among all device types within the same group. That means that enabling Harvest for one device type (cellular, Sigfox, LoRaWAN, or Inventory) will also enable Harvest for other device types within that group.

To configure Harvest, open the settings page of the group that you want to configure.

Underneath the Basic Settings tab, click the SORACOM Harvest Data panel to expand its settings.

Enable Harvest Data by switching the option to ON.

Harvest Files

Harvest Files settings are only available for Air for Cellular devices.

To configure Harvest Files, open the settings page of the group that you want to configure.

Underneath the Basic Settings tab, click the SORACOM Harvest Files panel to expand its settings.

Enable Harvest Files by switching the option to ON.

Parameters

Harvest Files allows you to configure options to control where files are saved, and how uploaded files can be accessed by devices.

• Default Path - The directory structure where Harvest Files will store uploaded files. Variables :imsi and :time are permitted, and will automatically insert the corresponding IMSI and timestamp accordingly. Refer to the Uploading Files section for more information.
• Assumed Role - The ID of the Soracom Access Management (SAM) Role containing the file read/write permissions that should be applied to subscribers in the group. For more information, refer to the Access Control documentation for more information.

In addition, the Log file uploading event to Harvest Data option allows you to create a data entry in Harvest Data whenever a file is uploaded, consisting of the file path and upload timestamp. This option can be combined with the Dynamic Image Panel in Lagoon in order to dynamically show new images in a Lagoon dashboard as they are uploaded from a device.

• Path patterns - A list of file path patterns (regular expressions allowed) that determine which file upload events should be logged in Harvest Data. When uploading a file to Harvest Files, if the file path does not match one of the patterns, the file upload event will not be logged.

Harvest Data does not need to be enabled in order to use the Log file uploading event to Harvest Data option. In addition, file upload events that are logged to Harvest Data do not incur any Harvest Data fees service fees.

---

Extended Data Retention

By default, Harvest data retention period is 40 days, once the retention period has passed, data older than 40 days will be deleted. However, users can subscribe to the Harvest Data Extended Retention Option, which will extend data retention to 731 days. Please refer to our Pricing & Fee Schedule for the cost of enabling this option.

To enable Extended Data Retention, log into the Soracom User Console, select Soracom Harvest Data. The alert at the top of the page will indicate that the default retention period of 40 days is enabled. Click the link within the alert to subscribe

---

Advanced Configuration

Harvest Data and Harvest Files can also be configured through the Soracom API or CLI by using the SoracomHarvest and SoracomHarvesetFiles namespaces, respectively.

Configuration Structure

For Harvest Data, the group configuration looks like this:

"SoracomHarvest": {
  "enabled": true|false
}

For Harvest Files, the group configuration looks like this:

"SoracomHarvestFiles": {
  "enabled": true|false,
  "defaultPath": "/path/to/file",
  "assumedRoleId": "myRole",
  "logFileUploadingEventToHarvestData": {
    "enabled": true,
    "pathPatterns": [
      "/lagoon/.*"
    ]
  }
}

Parameters

For the SoracomHarvest namespace:

Enable or disable Harvest Data:

• key (string, required) - enabled
• value (boolean, default: false) - Enables or disables Harvest Data.

For the SoracomHarvestFiles namespace:

Enable or disable Harvest Files:

• key (string, required) - enabled
• value (boolean, default: false) - Enables or disables Harvest Files.

Set the default path for upload files:

• key (string, required) - defaultPath
• value (string, required) - The path Harvest Files should use to save the uploaded file, if no file path is specified by the device when uploading.

Control the device read and write permissions:

• key (string, required) - assumedRoleId
• value (string, default: "") - The ID of the Soracom Access Management (SAM) Role which should be applied to the group.

Configure Log file uploading event to Harvest Data option:

• key (string, required) - logFileUploadingEventToHarvestData
• value (object, required) - Option configuration.
  • enabled (boolean, required) - Enables or disables the Log file uploading event to Harvest Data option.
  • pathPatterns (array of strings, required) - An array of regular expressions used to determine which file upload events should be logged.

Sample

Enable Harvest Data:

[
  {
    "key": "enabled",
    "value": true
  }
]

Enable Harvest Files, while specifying that each file uploaded should be given a file path (filename) according to the IMSI of the subscriber, and the timestamp when the file is uploaded:

[
  {
    "key": "enabled",
    "value": true
  },
  {
    "key": "defaultPath",
    "value": "/:imsi/:time"
  }
]