对象检测

对象检测服务支持使用 Multimedia API 从图像中提取车辆、人员、图案、子图像等对象。

默认情况下,ecu.test 支持用于识别图像内子图像的简单算法(模板匹配和基于特征的匹配)。也可以使用 UserPyModules 集成额外的对象检测算法,例如 YOLO 这类可检测人员、车辆、动物等不同对象类别的 AI 模型。

用户自定义对象检测

ecu.test 中的对象检测使用对象检测器识别图像中的各个对象,并将其输出为 MatchList。为此,必须提供一个 Python 模块。要使其被识别为有效的 UserObjectDetector,必须满足以下要求:

  • 该模块必须位于当前 Workspace(工作区)的 UserPyModules 文件夹中。

  • 必须在脚本顶层定义变量 MODULE_TYPE = ‘ObjectDetector’。

  • 必须实现一个名称任意、但满足以下接口的 ObjectDetector 类:

    • __init__

    • GetId

    • GetDocumentation

    • DetectObjects

    • GetAvailableParameters

    • GetParameter

    • SetParamter

  • 不得存在两个具有相同 ID 的对象检测器。

有关该接口的更多细节,请参见下一节中的示例。

新增或修改后的对象检测器会在更新用户库后被 ecu.test 识别,此操作可在主菜单“Extras”中执行。

要通过内部 Multimedia API 使用用户自定义对象检测器,可以通过以下方式创建实例:

  • api.Multimedia.GetObjectDetector

返回对象上可以直接调用该接口中已实现的方法,例如

api.Multimedia.GetObjectDetector('ExampleObjectDetector').GetParamter('objects').

对象检测最小示例

自定义对象检测器的示例实现可在 ecu.test 安装目录中找到:

  • Templates/DefaultData/UserPyModules/UserObjectDetector/ExampleObjectDetector.py

ExampleObjectDetector.py 中的最小示例包含以下代码。有关各个方法的更多信息可参见文档字符串。

from __future__ import annotations

from typing import TYPE_CHECKING, Any

import numpy as np

if TYPE_CHECKING:
    import numpy.typing as npt

MODULE_TYPE = 'ObjectDetector'


class ExampleObjectDetector:

    def __init__(self):
        """
        The object detector can be created when *GetObjectDetector(detectorId)* is called on
        the internal multimedia API, where *detectorId* is the identifier returned by *GetId()*.
        """
        self._params: dict[str, Any] = {
            param['name']: param['default'] for param in self.GetAvailableParameters()
        }

    def DetectObjects(self, img: npt.NDArray) -> list[dict[str, int | float]]:
        """
        Calls the detection algorithm of you object detector. This example detector creates
        (pseudo-random) object matches in the image dimensions and returns it.

        :param img: Numpy image (width x height x channels)
        :return: A list of dictionaries that describe the (pseudo-random) objects in the image with
                 position, dimensions and confidence.
        """
        objects = []
        # retrieve image dimensions
        height, width = img.shape[:2]
        # create pseudo random number generator with fixed seed
        gen = np.random.default_rng(seed=42)
        # create objects at random positions. The number of objects is defined in the 'objects'
        # parameter
        for _ in range(self._params.get('objects', 10)):
            objWidth = gen.random() * 100
            objHeight = gen.random() * 100
            x = gen.random() * (width - objWidth)
            y = gen.random() * (height - objHeight)
            objects.append(
                {
                    'x': x,
                    'y': y,
                    'width': objWidth,
                    'height': objHeight,
                    'confidence': gen.random(),
                }
            )
        return objects

    @staticmethod
    def GetDocumentation() -> str:
        """
        Returns a short documentation for your ObjectDetector class.

        :return: A documentation string.
        """
        return (
            'This is an example object detector that detects a parameterized number of objects at '
            'random positions.'
        )

    @staticmethod
    def GetId() -> str:
        """
        The identifier of the object detector which is used to create the detector via the
        multimedia api.

        :return: Identifier of the object detector.
        """
        return 'ExampleObjectDetector'

    def GetAvailableParameters(self) -> list[dict[str, Any]]:
        """
        Returns a list of all configureable parameters. Each parameter must be described by a unique
        name and a default value. A short description is optional.

        :return: List of all available paramters
        """
        return [
            {
                'name': 'objects',
                'default': 10,
                'description': 'Number of objects to detect',
            }
        ]

    def GetParameter(self, name: str) -> Any:
        """
        Get the current value of a given parameter.

        :param name: Name of the parameter. Available parameters can be queried via
                     GetAvailableParameters() method.
        :return: Value of the given parameter.
        """
        return self._params.get(name, None)

    def SetParameter(self, name: str, value: Any) -> None:
        """
        Set a new value to the given parameter.

        :param name: Name of the parameter. Available parameters can be queried via
                     GetAvailableParameters() method.
        :param value: The new value of the given parameter.
        """
        self._params[name] = value