Using Flux with Generative AI to Analyze Image Data and Send Slack Alerts

In this example project, we provide guidance on sending image data to Soracom, using Flux with an AI platform to analyze the data, and sending back alerts when conditions are met.

Requirements

If you don't already have a Soracom account or a registered SIM, follow the steps here.

Project Overview - Warehouse Safety Monitoring

Project Scenario

Warehouse Monitoring with Generative AI for Safety Alerts:

A camera continuously monitors a warehouse, capturing images for analysis. The system uses AI image analytics to identify workers and equipment. If a person is detected, Generative AI evaluates the scene. If an anomaly, such as a worker falling or not wearing a helmet, is found, an alert with a description is sent to Slack, notifying supervisors to take immediate action.

• The Soracom Harvest Files service is used to collect image data.
• The process within the dotted line represents an application created using Flux to manage this project.

Project Steps

Enable Harvest Files on Your Soracom SIMs

Configure your device to send image files to the Soracom platform through Soracom Harvest. Image files uploaded to Soracom will serve as the event source for your application.

Learn how to configure your account to enable Harvest Files as an event source for Flux here.

Create a Flux application

A Flux App can integrate various forms of inputs and and apply sophisticated business logic to achieve desired outcomes.

1. Create a new Flux application

   • Login to the User Console. From the Menu, open the SORACOM FLUX screen. Next, select Flux Apps

   

   • + Create a new Flux app

   

   • Name and describe the Flux app, then click Create

   

2. Access SORACOM Flux Studio
   
   
   • The Flux app will be created, and the STUDIO tab in SORACOM Flux Studio will automatically be displayed

Create a Flux channel

Create a Flux channel to detect potentially dangerous situations in the warehouse. A channel is a component that can connect an event source to actions in your Flux app.

1. Open the SORACOM Flux Studio and create a channel

   • Navigate to the STUDIO tab and click on + Create Channel

2. Select Event Source

   • Choose SORACOM Harvest Files as the event source
   • Click Next

3. Configure Channel Settings

   • NAME: Enter a name for your channel
   • DESCRIPTION: Provide a brief summary of the channel
   • EVENT SOURCE CONFIG: Define conditions for file changes that will trigger messages to this channel

4. Configure Event Types

   Event Source Configurations for this Project
   
   

   Item	Details
   ENABLED	Set this condition to enabled or disabled. Choose enabled.
   FILE PATH PREFIX	Enter the file path for Harvest Files. Files that start with the path specified here will be the target of the event source.
   EVENT TYPE	Select the type of file change. Check File Created and File Updated.
   File Created	A new file has been uploaded.
   File Updated	A file with the same name as an existing file was uploaded.
   File Deleted	The file was deleted.

   

5. Create Channel

   • Click Create a new channel
   • A new channel linked to the event source will be created and displayed
     
     

6. Close and Verify

   • Click the X to close the dialog to verify the setup
   • Return to the STUDIO tab in SORACOM Flux Studio to view details and / or modify settings

   You can click on components in SORACOM Flux Studio to view or modify settings.

   

Create a Flux action to have AI analyze events

An action is a component that can be added to a channel that can conditionally process information from an event source to execute a desired effect.

1. Create the AI action

   • Click on the channel you've created in the previous step to open the menu. Choose the ACTIONS tab
   • Click + Add Action

   

2. Configure action

   • In the Create a new action dialog, choose AI as your action type, then click OK
     • NAME: Enter a name for the action
     • DESCRIPTION: Provide a brief summary
     • ENABLED: enable
     • ACTION CONDITION: Create a condition that can trigger the action. In the example below, payload.contentType=="image/jpeg" instructs the action to respond to JPEG images.

   Click the ? Specify condition to... section of the dialog to reveal a full guide on example conditions, variables, operators, and functions.

   

   Action configurations for this project
   
   

   Item	Description
   CONFIG Tab	Configures the AI model to for the action.
   AI MODEL	Select Azure OpenAI (GPT-4o).
   PROMPT (enter the following)	Analyze the attached image. If people are visible, give me the total number and tell me how many are not wearing helmets. Return in JSON format, like this: `{ "people": 1, "people_with_no_helmet": 1 }`
   USE JSON-FORMATTED AI RESPONSE	Check this box.
   USE IMAGE	Check this box and enter ${payload.presignedUrls.get} into the input field.
   OUTPUT	Specifies whether the AI action response should be sent to a separate channel.
   Republish the action output to another channel	Enable.
   DESTINATION CHANNEL	Select CREATE A NEW CHANNEL.
   Channel Name	Enter the channel name you'll use to receive the output (e.g. Output)

   
   

3. Complete the setup

   • Click Create
   • Click X to close and return to the STUDIO tab to see the new AI Action and Output channel

   You can click on components in SORACOM Flux Studio to view or modify settings.

   

Create a Flux action for AI responses to Slack

In this section we'll create an action that will send out a Slack alert if conditions set in the previous section are met.

Prepare Slack Integration

1. Create a Slack Notification action

   • Click on the Output channel created in the previous section and select the Actions tab
   • Click Add Action to create a new action
   • Choose Slack Notification, then click OK

   

2. Configure the Slack Notification action

   Item	Details
   NAME	Enter a name for the Slack Notification action.
   DESCRIPTION	Enter a summary for your Slack Notification action.
   ENABLED	Set this condition to enabled or disabled. Choose enabled.
   ACTION CONDITION	Set the condition to trigger the action when the number of people is greater than 0, using payload.output.people > 0

   Remember, you can click the ? Specify condition to... section of the dialog to reveal a full guide on example conditions, variables, operators, and functions.

   

3. Configure the message and webhook URL

   • Enter the Webhook URL associated with the Slack channel you want to notify
   • In the payload section, specify the output message with payload.output.people and payload.output.people_with_no_helmet
   • Add the event payload to specify when to trigger the Slack Notification action based on the event and image data
     
     

   Item	Details
   CONFIG	Configure the message to send to Slack and the webhook URL for the Incoming Webhook integration.
   URL	Enter the Webhook URL associated with the Slack channel you want to notify. For details on how to issue a Webhook URL, see Issue a Webhook URL for Incoming Webhooks in the Slack App.
   PAYLOAD (Enter the following information)	There are ${payload.output.people} people. Of those, there are ${payload.output.people_with_no_helmet} people not wearing helmets!!
   OUTPUT	Configure whether the response of the Slack notification action should be sent to a separate channel. (not necessary per the scenario being described above here)
   Republish the action output to another channel	Disable.

4. Create the Slack notification action

   • Click Create to save the Slack notification action

Your settings should reflect the image below.

The AI action should now be successfully set up to analyze images and send Slack notifications for any anomalies detected.

Testing

1. Test the Slack Notification action

   • Upload a JPEG image to Harvest Files containing a person without a helmet
     • If there are no people without a helmet in the file, no further action will be taken.
   • Check that the message is sent to Slack as configured
2. Check Slack Notification and app history
   • Verify that the message has been sent to the Slack channel
   • From the Menu, open the SORACOM FLUX screen. Next, select Flux Apps
     • Select your Flux app for this project
     • Select the HISTORY tab to review actions taken by the app
   • Here, you can review records such as MESSAGE, CONTEXT, INPUT, and OUTPUT for all channels in the app. See further details on Execution History here.

Troubleshooting and Expected Behavior

File and Data Requirements

• Ensure that the image file sent to Soracom Harvest Files is of the type specified in the Flux action (e.g., JPEG).
• If a non-image file is uploaded to Soracom Harvest Files, the app will not take action.

AI Detection Conditions

• The Flux app will send a message only if the AI detects people in the image file.
  • Additionally, the AI must detect an anomaly in the image (e.g., people not wearing helmets) to trigger a notificaiton.
• Ensure that test images contain the specified conditions (e.g., people without helmets) to generate the expected response.

Configuration and Settings

• Ensure the AI model is configured correctly in the CONFIG tab. Verify that both Use JSON-Formatted AI Response and Use Image options are selected, and check that the correct prompt format is used.

Action Triggers and Conditions

• If no action is triggered when expected, double-check the action condition in both the AI and Slack actions. Ensure that conditions like payload.contentType=="image/jpeg" and payload.output.people > 0 are set accurately to trigger only when the conditions are met.

Output and Channel Connections

• If no output or incorrect data is sent to Slack or other channels, review the HISTORY tab in Soracom Flux Studio to inspect the output data.
• Verify that each action is correctly linked to the destination channel. If output republishing is enabled, ensure the destination channel is correctly specified.