CYCLIST+IMU: A synchronized visual–inertial dataset for cyclist orientation and perception in urban environments

2.1 Data acquisition system

Cyclist images and synchronized visual–inertial data were captured using the CYCLOPS system (cyclists’ orientation data acquisition system using RGB camera and inertial measurement units). This original development consists of a node located on a vehicle and another on a bicycle. The vehicle node includes a monocular RGB camera, an inertial measurement unit (IMU), an RF transceiver, and a microcontroller. The bicycle node includes an IMU, an RF transceiver, and a microcontroller. The system facilitates the acquisition of images of a moving cyclist and associates each image with both acceleration and orientation angles (Ax, Ay, Az, Roll, Pitch, Yaw), as illustrated in Figure 2. Similarly, camera acceleration and orientation angles are acquired at the vehicle to subsequently obtain relative values (cyclist relative to camera) and establish the cyclist’s real orientation in each image acquired from the vehicle while both are moving in an urban environment.

Figure 2. Frame assignment for both, the camera attached to a car’s windshield (over the vehicle) and the bicycle’s top tube (cyclist).

For the vehicle, the axes have been named Xv, Yv, and Zv, and rotations around the axis are ROLLv, PITCHv, and YAWv (respectively). Similarly, the frame for the cyclist has axes Xc, Yc, and Zc, and rotations around the axis are ROLLc, PITCHc, and YAWc (respectively). Source: Adapted from the original in Arias-Correa et al. (2024).

A significant achievement of the CYCLOPS acquisition system lies in the synchronization between images and 6-axis motion data for both actors. Full details regarding the open-source hardware architecture, the IMU-based sensor fusion, and the software suite (VideoCapture) are documented in Arias-Correa et al. (2024), ensuring complete experimental reproducibility. A general diagram of the hardware and software for each node (cyclist node and camera node) of the CYCLOPS system is presented in Figure 3.

Figure 3. Diagram of the hardware and software for each node of the CYCLOPS system.

The block Cyclist includes a printed circuit board (PCB), which comprises an IMU, an Arduino Nano board (Microcontroller-C), and a transceiver HC12 in transmission mode (Transceiver-C) with an antenna. Similarly, the block camera includes the RGB camera mounted on the vehicle’s windshield, an IMU (IMU-V), an Arduino Nano board (Microcontroller-V), and a transceiver HC12 (Transceiver-V) with an antenna. Both the camera and the PCB send data to the acquisition software running on a computer. Source: Adapted from the original in Arias-Correa et al. (2024).

2.2 Visual–inertial data synchronization

The CYCLOPS system implements a hardware-level synchronization protocol to ensure precise temporal registration between the visual–inertial data from the RGB camera and the high-frequency inertial data from the IMU sensors. This synchronization is executed at the point of acquisition via a deterministic software architecture that manages concurrent sensor triggering and logging across both bicycle-mounted and vehicle-mounted nodes. Specifically, each RGB frame is hardware-level synchronized to a discrete set of inertial measurements captured at the exact timestamp t_k of the camera’s shutter release. This one-to-one temporal mapping constitutes the fundamental visual–inertial sample of the dataset. For every image captured by the vehicle node, the system captures the cyclist’s instantaneous kinematic state, including orientation variables such as yawb and rollb referenced to the coordinate frames established during system calibration. Crucially, this process is managed in real-time using a unified system clock, which precludes the need for subsequent corrections such as temporal interpolation, resampling, or offline alignment. By avoiding these post-processing steps, the CYCLOPS dataset maintains the integrity of the raw sensor data, providing a high-fidelity, temporally consistent snapshot of the cyclist’s pose and motion at the precise moment of visual capture as seen in Figure 4.

Figure 4. Result of the CYCLOPS acquisition process.

The figure shows the output generated by the CYCLOPS system for several consecutive RGB image frames acquired by the vehicle-mounted camera, together with their corresponding inertial measurements stored in format. Each image frame is temporally synchronized with a discrete set of inertial data captured at the exact acquisition instant, establishing a one-to-one correspondence between visual and inertial information. This synchronized visual–inertial data product represents the final output of the CYCLOPS framework.

All details regarding the CYCLOPS hardware components (including camera and IMU models), synchronization strategy, calibration procedures, and data verification experiments are fully described and validated in Arias-Correa et al. (2024) and are therefore not repeated here.

Each RGB image captured by the vehicle-mounted camera is associated with a corresponding set of inertial measurements describing the cyclist’s motion and orientation at the time of capture. Orientation variables, including yaw and roll, are derived from the inertial measurements and expressed in their respective reference frames as defined by the system configuration. Although the dataset also includes pitch measurements, this variable is not explicitly discussed here because the subsequent analysis and data augmentation procedures focus exclusively on yaw and roll.

2.3 Data Acquisition Protocol

Data were collected across multiple independent acquisition sessions conducted in urban environments. Each session is identified using the label Adq_x and corresponds to a continuous recording sequence acquired while the cyclist and the acquisition vehicle were in motion under real traffic conditions.

During each acquisition session, RGB images of the cyclist were recorded using the vehicle-mounted camera, while inertial measurements describing the cyclist’s motion and orientation were simultaneously captured by the bicycle-mounted and vehicle-mounted IMUs. A single sample is defined as an RGB image associated with its corresponding synchronized inertial measurements, including orientation angles such as yaw and roll, as described in Section 2.2.

The acquisition protocol was designed to capture natural cyclist behavior in unconstrained urban scenarios. Data were collected under daytime lighting conditions on public roads, without imposing specific trajectories or maneuvers on the cyclist beyond normal riding behavior.

Following data collection, a basic quality control process was applied to the acquired data. Samples exhibiting acquisition failures, severe occlusions of the cyclist, or loss of visual–inertial synchronization were excluded from the dataset. Only samples in which the cyclist was clearly visible, and the corresponding inertial data were successfully recorded and retained for further processing and inclusion in the dataset.

In this work, a total of 3,606 RGB images were acquired, each associated with its corresponding inertial measurements obtained from the IMU units of the CYCLOPS system. Images and inertial data are stored following a hierarchical file structure designed to preserve the temporal correspondence between each image and its associated orientation and acceleration records. After data acquisition, the RGB images were manually annotated using the DarkLabel tool ( https://github.com/darkpgmr/DarkLabel ) to identify and delineate the cyclist region of interest (ROI). The resulting annotations were exported in YOLO format (*.txt files), providing normalized bounding box coordinates for each cyclist and enabling their direct use in object detection and subsequent analysis pipelines.

2.4 Image processing

To construct a dataset suitable for computer vision tasks, additional processing was applied to the cyclist region-of-interest (ROI) images extracted from the original RGB images. This processing stage includes manual semantic segmentation of the cyclist and the generation of relative depth maps, as illustrated in Figure 5.

Figure 5. RGB image processing stages.

(a) Manual segmentation of the cyclist from RGB images using the LabelMe annotation tool, where the region corresponding to the cyclist is precisely delineated. (b) Depth map estimation from RGB images using an encoder–decoder model, generating a complementary geometric representation of the cyclist and the surrounding environment.

Polygon-based semantic segmentation of the cyclist was performed using the LabelMe annotation tool (Russell et al., 2008), which supports polygon-based annotation and is widely used in computer vision applications. As illustrated in Figure 5a, for each RGB image, the region corresponding to the cyclist was manually delineated using polygonal annotations. This procedure was applied to the entire dataset to ensure a consistent and precise definition of the region of interest across all samples. The resulting annotations were stored in JSON format, preserving the geometric information required for subsequent processing and reuse.

In parallel, the RGB images were processed to obtain depth maps using the Depth Anything model (Yang et al., 2024), as shown in Figure 5b. Depth Anything follows a base-model paradigm, and is trained at a large scale using unlabeled data, enabling robust generalization across diverse visual scenes. Although alternative approaches such as MiDaS have demonstrated strong performance through supervised and weakly supervised training on curated datasets (Ranftl et al., 2020; Birkl et al., 2023). The Depth Anything model was selected due to its ability to produce consistent relative depth estimates without relying on task-specific supervision. The depth maps included in the CYCLIST+IMU dataset represent relative depth and have been stored as 8-bit grayscale images, where depth values (non-metric) were normalized and linearly mapped to the range [0, 255] before saving in JPEG format. These depth maps are provided as complementary geometric context to the RGB images and semantic annotations.

2.5 Data augmentation

To enhance the coverage of cyclist orientations and increase the angular diversity of the dataset, a data augmentation strategy based on geometric transformations was applied. This strategy was designed to expand the range of represented orientations while preserving the physical coherence and visual consistency of the samples.

Data augmentation was performed using a horizontal flipping transformation applied to the RGB images. This operation generates additional samples by flipping the original images along the vertical axis, thereby increasing orientation diversity without introducing artificial visual artifacts or altering the geometric structure of the cyclist and the surrounding environment.

To maintain consistency between the visual content and the associated orientation labels, yaw and roll values were deterministically updated following the transformation. Under horizontal flipping, yaw angles were transformed according to: yaw′ = 360° − yaw. Meanwhile, roll values were inverted as: roll′ = −roll. This transformation was applied only to samples belonging to selected underrepresented yaw intervals, as identified during the dataset validation stage (Section 3.1).

An illustrative example of the data augmentation process is shown in Figure 6, where the original image, the augmented image, and the corresponding updates to yaw and roll angles are presented.

Figure 6. Example of data augmentation using horizontal image flipping.

The original RGB image and the corresponding augmented image obtained by horizontal flipping are shown. Yaw and roll angles are deterministically updated to preserve geometric consistency, with (yaw’ = 360°-yaw) and (roll’ = −roll ).

2.6 Dataset structure and organization.

The dataset is organized using a hierarchical directory structure designed to preserve traceability between original acquisitions and all derived data products. At the top level, the dataset is divided into two main directories: Original_Image, containing the raw RGB images, and Image_Crops, which stores all processed cyclist-centered data.

Within Image_Crops, data are grouped into acquisition-level subdirectories labeled Adq_x, where x denotes an independent synchronized recording session, integrating RGB imagery and inertial measurement unit (IMU) data. Each Adq_x directory contains four modality-specific subfolders: RGB_IMAGE (cropped RGB images), Detection (region-of-interest detection files), Polygons (polygon-based semantic segmentation annotations in JSON format), and Depth_Map (relative depth maps).

Each acquisition session also includes a metadata file (Adq_x.xlsx) consolidating inertial information, including cyclist orientation parameters (yaw and roll), and unique identifiers linking all data modalities. File naming follows the convention Adq_x (n), ensuring a strict one-to-one correspondence among RGB images, detection files, segmentation annotations, depth maps, and inertial measurements.

An overview of the dataset structure is provided in Figure 7, and a summary of dataset contents and file formats is presented in Table 2.

Figure 7. Hierarchical organization of the CYCLIST+IMU dataset.

The dataset is structured to preserve traceability between the original multimodal acquisitions and all derived data products. Raw RGB images are stored in the Original_Image directory, while processed cyclist-centered data are organized under Image_Crops by acquisition session (Adq_x). Each Adq_x represents a synchronized acquisition unit integrating RGB imagery and inertial measurement unit (IMU) data, including cyclist orientation parameters such as yaw and roll observed from the acquisition vehicle. For each session, cropped RGB images, region-of-interest detection files, relative depth maps, and polygon-based semantic segmentation annotations are stored in modality-specific subdirectories using a consistent naming convention that ensures a strict one-to-one correspondence across data types.

Table 2 presents the complete description of the dataset generated in this work.

3.1 Angular consistency and coverage.

The CYCLOPS orientation angles include yaw and roll, which require different statistical treatments due to their measurement domains. Yaw is a periodic variable defined over [0°, 360°), while roll is restricted to a narrow, non-periodic range around zero (approximately −20° to 20°).

Yaw distributions were analysed using circular statistics to avoid discontinuities at the 0°/360° boundary, computing circular mean, median, and deviation with the pycircstat Python library. Roll values were characterised using standard linear descriptive statistics (mean, median, and standard deviation).

Figure 8 shows the yaw and roll distributions prior to data augmentation, and Table 3 reports the corresponding descriptive statistics, confirming adequate angular coverage of cyclist orientations under real-world riding conditions.

Figure 8. Distribution of angular variables acquired by the CYCLOPS system.

(a) Circular distribution of cyclist orientation angles (yaw), represented using circular statistics to account for the periodic nature of the variable over the [0°, 360°) range. (b) Linear distribution of cyclist inclination angles (roll), analysed using linear statistics due to their bounded range around 0°.

3.2 Validation of the data augmentation strategy.

The data augmentation procedure based on horizontal image flipping was validated to ensure geometric consistency between RGB images and orientation angles. After augmentation, yaw and roll values were deterministically updated following the transformation rules defined in the Methods section.

Consistency between augmented images, updated orientation angles, and the associated segmentation polygons, detection files, and relative depth maps was verified through visual inspection. Figure 6 illustrates the augmentation process, while Figure 9 shows the post-augmentation distributions of yaw and roll, highlighting the angular regions targeted to improve coverage.

Figure 9. Post-augmentation distribution of cyclist orientation and inclination variables.

(a) Circular distribution of orientation angles (yaw) after data augmentation. Angular bins highlighted in red indicate the yaw intervals selected for horizontal flipping and sample augmentation. (b) Linear distribution of inclination angles (roll ) after data augmentation. Red-highlighted bins correspond to the roll range associated with the augmented samples, while blue bins represent the remaining original and augmented data.

Table 4 summarizes the descriptive statistics after augmentation, confirming enhanced angular coverage without altering the overall structure of the original dataset.