Data labelling has become a major problem in industries aiming to create and use ground truth labels from massive multi-sensor archives to feed into Artificial Intelligence (AI) applications. Annotation of multi-sensor set-ups with multiple cameras and LIDAR is now particularly relevant for the industry aiming to build autonomous capabilities. The paper presents the Video Content Description (VCD), as the first open-source metadata structure and set of tools, able to structure annotations for such complex scenes, including unprecedented flexibility to label 2D and 3D objects, pixel-wise labels, actions, events, contexts, semantic relations, odometry, and calibration. Several example cases are reported to demonstrate the flexibility of the VCD.
The application of the publication outcomes within the SmaCS framework is essential, as it will help identifying and tracking the nature of objects and items detected in the cabin of the aircraft that might be of potential danger during the Taxi, Take off and landing – TTL operations, thus, the outcomes will be of outmost relevance to the project.
The paper was published in January 2021 over the SoftwareX the biannual peer-reviewed open-access scientific journal, covering scientific software by Elsevier (full publication is available here. Even though VCD was originally designed in the context of automotive applications, it will be of direct use in the context of SmaCS as well, as it is the first open-source metadata structure and toolset capable of structuring annotations for highly complex scenes, which will be performed in the project.