Inspection Mode

Junction's Inspection mode will parse all packets passing through the VPG and send statistical information to a specified cloud service using a service adapter. From the cloud service, the data can then be connected with a tool in order to build visualizations and perform network analysis.

The following cloud service adapters are available:


Example Implementation

As each adapter provides access to different cloud services, implementation instructions will vary according to the data analysis tools you use. This guide demonstrates a practical implementation of Junction Inspection with the following architecture:

Junction Inspection example

Requirements


Elastic Cloud Setup

Let's first start by setting up Elasticsearch and Kibana on Elastic Cloud. If you don't have an account, you can sign up for a 14-day trial .

  1. Login to Elastic Cloud.

  2. Click Create Cluster.

  3. Enter the following options:

    https://cloud.elastic.co

    Missing

    • Cluster Size - 1GB memory and 24GB storage.
    • Region - Select a region near you.

    Keep the defaults for the remaining options, then create the cluster.

  4. Once the cluster is created, a elastic user and its password will be displayed. Make a note of this information, as we will need it later.

  5. Next, from the Kibana section, click the Enable button.

    https://cloud.elastic.co

    Missing

Now we need to test that we can connect to Elasticsearch.

  1. Open the cluster Overview page. Then from the Endpoints section, click HTTPS. Use the elastic username and password noted earlier to authenticate the connection. If successful, you should see a JSON response similar to the following:

    Missing

  2. We also need to check that we can access Kibana. Follow the same process, using the same elastic username and password from earlier.

    Missing

Last, we need to register a template in Elasticsearch, which will tell Elasticsearch about the format of the statistical data that Junction will be sending.

  1. Download the Soracom Realtime VPG Metrics template file to your computer.

  2. Register it to Elasticsearch by making an HTTP PUT request to your cluster's HTTPS endpoint, specifying the elastic username and password, and using the template file as the HTTP request body:

    curl -X PUT \
    >  -H 'Content-Type:application/json' \
    >  --user elastic:<PASSWORD> \
    >  -d '@/path/to/soracom-realtime-vpg-metrics-template.json' \
    >  "https://xxxxxxxxxxxxxxxxxx.ap-northeast-1.aws.found.io:9243/_template/soracom-vpg"

    Elasticsearch will return a {"acknowledged":true} response when the template is successfully registered.


Kinesis Stream Setup

Next, we need to set up a Kinesis Stream, which is where Junction will send Inspection data.

  1. Login to the AWS Management Console . From the Services menu, open the Kinesis dashboard. Then click the Go to the Streams console button.

    https://console.aws.amazon.com

    Missing

  2. Click the Create Kinesis stream button.

    https://console.aws.amazon.com

    Missing

  3. Enter a name for this stream, such as junction-inspection-escloud. We will keep the default settings for this stream. Then click the Create Kinesis stream button.

    https://console.aws.amazon.com

    Missing

After the Kinesis stream has been created, we need to create credentials that Junction can use in order to connect to Kinesis. For security, we will create a new AWS IAM user with programmatic access which only has permission to access Kinesis.

  1. From the AWS Services menu, open the IAM dashboard. Then from the Users, click the Add user button.

    https://console.aws.amazon.com

    Missing

  2. Enter a User name, and enable Programmatic access. Then click the Next: Permissions button.

    https://console.aws.amazon.com

    Missing

  3. Under the Set permission section, click Attach existing policies directly. Search for the AmazonKinesisFullAccess policy and click its to attach it to the new user.

    https://console.aws.amazon.com

    Missing

  4. Continue the Add user wizard by clicking the Next buttons, keeping the default values for the remaining sections. Then click Create user to complete the wizard.

    https://console.aws.amazon.com

    Missing

Once the new user has been created, it will appear along with its Access key ID and Secret access key. Save these keys in a secure location.


Lambda Setup

Now with Elasticsearch and Kinesis set up, we can create a Lambda function that will take care of sending our Inspection data from Kinesis to Elasticsearch.

In order to let Lambda access Kinesis, we need to set up a Role.

  1. In the IAM dashboard, click the Roles section. Then click the Create new role button.

    https://console.aws.amazon.com

    Missing

  2. In the Select role type screen, select AWS Lambda.

    https://console.aws.amazon.com

    create role

  3. In the Attach Policy screen, search for the AWSLambdaKinesisExecutionRole policy and click its to attach it to the new role.

    https://console.aws.amazon.com

    create role

  4. Last, enter a Role name. We will use this role when setting up Lambda next. Here, we use the name lambda_kinesis_execution.

    https://console.aws.amazon.com

    create role

Now we can create the Lambda function.

  1. From the AWS Services menu, open the Lambda dashboard. Click the Create a Lambda function button.

    https://console.aws.amazon.com

    Missing

  2. In the Select blueprint screen, choose Blank Function.

    https://console.aws.amazon.com

    Missing

  3. In the Configure triggers screen, select Kinesis, then select the junction-inspection-escloud stream we created earlier. Then click Activate Trigger.

    https://console.aws.amazon.com

    Missing

  4. Now we can write the Lambda function that will take our data from Kinesis and send it to Elasticsearch. Copy and paste the Soracom Junction Lambda function . For runtime, select Python 3.6.

    https://console.aws.amazon.com

    Missing

  5. Rather than entering our Elasticsearch cluster endpoint and credentials directly in the function, let's add them as environment variables in the next section.

    https://console.aws.amazon.com

    Missing

    Environment variable Value
    ES_BASE_URL Your Elasticsearch cluster HTTPS endpoint, without the trailing /.
    ES_USERNAME elastic
    ES_PASSWORD The password created earlier for the elastic user.
    ES_INDEX A prefix added to the data. Set this as soracom, as this is the format the Elasticsearch template expects.
    ES_TYPE Stats
  6. Then set configure this function to use the lambda_kinesis_execution role we created earlier.

  7. Last, set the timeout to 1 minute. Then click Create function.

Configure Junction

So far, we've completed the following configuration:

Let's configure our VPG to send Inspection data to our Kinesis stream.

Follow the Inspection configuration instructions from the Junction Configuration documentation, providing the following parameters:

https://console.soracom.io

Missing

Once we save these settings, Junction will immediately begin sending Inspection data to our Kinesis stream. Our Lambda trigger will then send the data to Elasticsearch, and we can start to visualize the VPG traffic.

If any Air SIM subscribers were not previously attached to the VPG, you must disconnect and reconnect their cellular connection in order to connect to the VPG.


Kibana Configuration

Now that everything is hooked up, we just need to configure our Kibana visualization.

We need to define an Index Pattern that will match the data being sent from Lambda function.

  1. Login to Kibana and click Management, then click Index Pattern.

  2. Enter soracom-* as the Index name or pattern. Then click the Create button.

    https://cloud.elastic.co

    Inspection settings

Next, let's import a dashboard template that is pre-configured.

  1. Download the Soracom Realtime VPG Metrics Dashboard template file.

  2. From Kibana, click Management Saved Objects. Then click the Import button, and import the dashboard template.

    https://cloud.elastic.co

    Elastic Cloud


Our Inspection configuration is now fully configured. As traffic passes through our Soracom VPG, Junction will take care of parsing the packets for statistical data. Kinesis, Lambda, and Elasticsearch will take care of handling the data, and we can now view realtime statistics from Kibana.

From Kibana, click Dashboard Soracom.

https://cloud.elastic.co

Kibana dashboard

The dashboard template will include information about throughput, protocol, packet destination, and so on.

https://cloud.elastic.co

Kibana dashboard