Inspection Mode

Junction's Inspection mode will parse all packets passing through the VPG and send statistical information to a specified cloud service using a service adapter. From the cloud service, the data can then be connected with a tool in order to build visualizations and perform network analysis.

The following cloud service adapters are available:


Example Implementation

As each adapter provides access to different cloud services, implementation instructions will vary according to the data analysis tools you use. This guide demonstrates a practical implementation of Junction Inspection with the following architecture:

Junction Inspection example

Requirements


Elastic Cloud Setup

Let's first start by setting up Elasticsearch and Kibana on Elastic Cloud. If you don't have an account, you can sign up for a 14-day trial .

  1. Login to Elastic Cloud.

  2. Click Create Cluster.

  3. Enter the following options:

    https://cloud.elastic.co

    Missing

    • Cluster Size - 1GB memory and 24GB storage.
    • Region - Select a region near you.

    Keep the defaults for the remaining options, then create the cluster.

  4. Once the cluster is created, a elastic user and its password will be displayed. Make a note of this information, as we will need it later.

  5. Next, from the Kibana section, click the Enable button.

    https://cloud.elastic.co

    Missing

Now we need to test that we can connect to Elasticsearch.

  1. Open the cluster Overview page. Then from the Endpoints section, click HTTPS. Use the elastic username and password noted earlier to authenticate the connection. If successful, you should see a JSON response similar to the following:

    Missing

  2. We also need to check that we can access Kibana. Follow the same process, using the same elastic username and password from earlier.

    Missing

Last, we need to register a template in Elasticsearch, which will tell Elasticsearch about the format of the statistical data that Junction will be sending.

  1. Download the Soracom Realtime VPG Metrics template file to your computer.

  2. Register it to Elasticsearch by making an HTTP PUT request to your cluster's HTTPS endpoint, specifying the elastic username and password, and using the template file as the HTTP request body:

    curl -X PUT \
    >  -H 'Content-Type:application/json' \
    >  --user elastic:<PASSWORD> \
    >  -d '@/path/to/soracom-realtime-vpg-metrics-template.json' \
    >  "https://xxxxxxxxxxxxxxxxxx.ap-northeast-1.aws.found.io:9243/_template/soracom-vpg"

    Elasticsearch will return a {"acknowledged":true} response when the template is successfully registered.


Kinesis Stream Setup

Next, we need to set up a Kinesis Stream, which is where Junction will send Inspection data.

  1. Login to the AWS Management Console . From the Services menu, open the Kinesis dashboard. Then click the Data streams button.

    https://console.aws.amazon.com

    Data streams

  2. Click the Create data stream button.

    https://console.aws.amazon.com

    Create data stream

  3. Enter a name for this stream, such as junction-inspection-escloud. We will keep the default settings for this stream. Then click the Create data stream button.

After the Kinesis stream has been created, we need to create credentials that Junction can use in order to connect to Kinesis. For security, we will create a new AWS IAM role that only has access to Kinesis.

  1. From the AWS Services menu, open the IAM dashboard. Then from the Roles, click the Create role button.

    https://console.aws.amazon.com

    Create role

  2. Select AWS account as the trusted entity type and add an External ID. This External ID will be needed for our Soracom Credential Set later. Then click Next.

    https://console.aws.amazon.com

    Role

  3. Select AmazonKinesisFullAccess as the permission policy and click Next.

    https://console.aws.amazon.com

    Attach Policy

  4. Enter a descriptive name for the role, then click Create role.

  5. From the main Roles screen, click on your newly created role and make a note of the Role ARN. This will be needed for the creation of our Soracom Credential Set later.

Lambda Setup

Now with Elasticsearch and Kinesis set up, we can create a Lambda function that will take care of sending our Inspection data from Kinesis to Elasticsearch.

In order to let Lambda access Kinesis, we need to set up a Role.

  1. In the IAM dashboard, click the Roles section. Then click the Create role button.

    https://console.aws.amazon.com

    Create role

  2. In the Select role type screen, select AWS service. Under Use case, select Lambda. Then click Next.

    https://console.aws.amazon.com

    Select settings

  3. In the Add permissions screen, search for the AWSLambdaKinesisExecutionRole policy and click its to attach it to the new role. Then click Next.

    https://console.aws.amazon.com

    Add permissions

  4. Last, enter a Role name. We will use this role when setting up Lambda next. Here, we use the name lambda_kinesis_execution. Leave the rest of the settings as default and click Create role.

Now we can create the Lambda function.

  1. From the AWS Services menu, open the Lambda dashboard. Click the Create function button.

    https://console.aws.amazon.com

    Create function

  2. Using the Author from scratch option, enter a descriptive function name, then choose Python 3.8 under Runtime and x86_64 under Architecture. Under Permissions click Use an existing role, then select the lambda_kinesis_execution we created earlier. Finally, click Create function.

    https://console.aws.amazon.com

    Lambda settings

  3. In the Code source section copy and paste the Soracom Junction Lambda function code.

    https://console.aws.amazon.com

    Code

  4. The Elasticsearch cluster endpoint and credentials can now be configured as environment variables. Under the Configuration tab click Environment variables, then click Edit.

    https://console.aws.amazon.com

    Environment variables

  5. Add the following environment variables then click Save:

    Key Value
    ES_BASE_URL Your Elasticsearch cluster HTTPS endpoint, without the trailing /.
    ES_USERNAME elastic
    ES_PASSWORD The password created earlier for the elastic user.
    ES_INDEX A prefix added to the data. Set this as soracom, as this is the format the Elasticsearch template expects.
    ES_TYPE Stats

Configure Junction

So far, we've completed the following configuration:

Let's configure our VPG to send Inspection data to our Kinesis stream.

Follow the Inspection configuration instructions from the Junction Configuration documentation, providing the following parameters:

https://console.soracom.io

Missing

Once we save these settings, Junction will immediately begin sending Inspection data to our Kinesis stream. Our Lambda trigger will then send the data to Elasticsearch, and we can start to visualize the VPG traffic.

If any Air SIM subscribers were not previously attached to the VPG, you must disconnect and reconnect their cellular connection in order to connect to the VPG.


Kibana Configuration

Now that everything is hooked up, we just need to configure our Kibana visualization.

We need to define an Index Pattern that will match the data being sent from Lambda function.

  1. Login to Kibana and click Management, then click Index Pattern.

  2. Enter soracom-* as the Index name or pattern. Then click the Create button.

    https://cloud.elastic.co

    Inspection settings

Next, let's import a dashboard template that is pre-configured.

  1. Download the Soracom Realtime VPG Metrics Dashboard template file.

  2. From Kibana, click Management Saved Objects. Then click the Import button, and import the dashboard template.

    https://cloud.elastic.co

    Elastic Cloud


Our Inspection configuration is now fully configured. As traffic passes through our Soracom VPG, Junction will take care of parsing the packets for statistical data. Kinesis, Lambda, and Elasticsearch will take care of handling the data, and we can now view realtime statistics from Kibana.

From Kibana, click Dashboard Soracom.

https://cloud.elastic.co

Kibana dashboard

The dashboard template will include information about throughput, protocol, packet destination, and so on.

https://cloud.elastic.co

Kibana dashboard