Toolkit v1.5.0b3 released

[Takenori Sato]

Hello developers,

We are pleased to announce that the new version of the Toolkit was released and is available on the APT repository as 1.5.0b3-3.

It includes new features to enrich an edge-ai application as well as some critical bug fixes as below.

New Features

• Options
• Custom Overlay
• Continuous Recording

Bug Fixes

• Delay In Double EAPs
• Broken Action Recording on Nano

Reflecting these changes, the new toolkit_home is available as v1.1.0 with various improvements.

Options and Custom Overlay

The Options is a way to allow an AI Model Developer to define an arbitrary number of configuration items that can be set by an end user on the Device Console at runtime. There are two types of Options. Callback and GStreamer. A callback option is a variable you can access in a callback through a signal, while a GStreamer option is a property of GStreamer element you can expose to an end user.

Custom Overlay is a way to draw a custom overlay by making drawing instructions in a callback. This allows an AI Model Developer to show rich information on a view to tell an end user what is happening. There are some limitations, but NvDsDisplayMeta is used for drawings.

EMI Vehicle Counter (right) and EMI Vehicle Color Watcher (left) are new templates to illustrate these new features as Double EAPs.

They are configured as follows.

EMI Vehicle Counter

This is a parent app that does a basic traffic counting.

    "pipeline_configuration": {
        "meta-source-id": "emi_vehicle_dcf",
        "input": {
            "batch-size": 1,
            "width": 1280,
            "height": 720
        },
        "primary": {
            "process-mode": 1,
            "config-file-path": "dstest1_pgie_config.txt"
        },
        "tracker": {
            "ll-config-file": "tracker_config.yml",
            "ll-lib-file": "libnvds_nvdcf.so"
        },
        "overlay": {
            "display-clock": 1
        },
        "aimeta": {
            "signal-interval": 10
        }
    }

EMI Vehicle Color Watcher

This is a child app that watches a particular color of vehicles and notifies when detected.

    "pipeline_configuration": {
        "input": {
            "batch-size": 1,
            "width": 1280,
            "height": 720
        },
        "primary": {},
        "dsmetatransfer": {
            "listen-to": "emi_vehicle_dcf"
        },
        "tracker": {
            "ll-config-file": "tracker_config.yml",
            "ll-lib-file": "libnvds_nvdcf.so"
        },
        "secondary": {
            "process-mode": 2,
            "config-file-path": "dstest2_sgie1_config.txt"
        },
        "overlay": {
            "display-clock": 1
        },
        "aimeta": {
            "signal-interval": 10
        }
    },
    "options": [
        {
            "key": "watching_color_id",
            "option_type": "callback",
            "value_type": "number"
        },
        {
            "key": {
                "element": "aimeta",
                "property": "signal-interval"
            },
            "option_type": "gstreamer",
            "value_type": "number"
        }
    ]

As you can see, there are definition of options that can be set by an end user. Here's such user defined configs in the new stream, vehicle_colorwatcher_stream.

  "options": [
    {
      "key": {
        "element": "aimeta",
        "property": "signal-interval"
      },
      "value": 10
    },
    {
      "key": "watching_color_id",
      "value": 10
    }
  ]

Those definitions of Options configurations can be done on a toolkit as below.

Details will be published on the tutorial document soon.

Continuous Recording

The continuous recording is another on-device recording that will keep recording as configured below by an end user.

pedestrian stream

  "continuous_recording": {
    "duration_in_minutes": 1,
    "max_files": 1440,
    "bitrate": "1m"
  }

This is configured to record for 1 minute each, up to 1,440 files (1 day), at the 1 M bps. The encoder in a pipeline is shared among record action and webrtc clients. But this continuous recording uses its own encoder instance to manage its own bitrate.

This will be further managed on the DeviceConsole by an end user as below (design screenshot, coming soon).

Note that the max_files is managed by the DeviceAgent. So, it won't be managed on a Toolkit box.

Installation

To install the new v1.5.0, you just need to run the following command.

/mnt/nvme/toolkit_home/bin$ ./update_toolkit.sh

It will install several new packages and upgrade others.

Please note that meson will be installed as the dependency with apt. If you have installed with pip3 by following the instruction of building EMCustom with v1.4.3, please remove it because the version is difference (meson.build in examples folder is not compatible).

$ pip3 uninstall meson

Known Issues

fifo error (hardware error)

The GStreamer Daemon process called gstd may crash with an error as below.

e.g.) dmesg will show many lines of errors along with

nvgpu: 57000000.gpu                    fifo_error_isr:2596 [ERR]  channel reset initiated from fifo_error_isr; intr=0x00000100

gstc concurrency error

The client of gstd may experience a race condition and crash when running more than one apps. This was exposed by having fixed the delay issue and two pipeline run in sync, which would make two gstd clients access almost at the same time.