feat: camera images and 2d annotations generation

[MagdalenaKotynia]

Description

This PR contains changes needed to convert rosbags to t4dataset with camera images and annotations (including 2d bounding boxes annotations in object_ann.json file).

How to review

How to test

test data

Example test rosbag

Example test config yaml file:

task: convert_rosbag2_to_t4
description:
  scene: synthetic
conversion:
  input_base: <path to rosbags directory>
  output_base: <path to output directory>
  workers_number: 1
  skip_timestamp: 2.0
  num_load_frames: 200
  crop_frames_unit: 50  
  camera_latency_sec: 0.00
  object_topic_name: /ground_truth/objects
  object_msg_type: TrackedObjects
  world_frame_id: world
  lidar_sensor:
    topic: /lidar/concatenated/pointcloud
    channel: LIDAR_CONCAT
  camera_sensors: 
    - topic: /sensing/camera/camera4/image_rect_color
      channel: CAM_FRONT
    - topic: /sensing/camera/camera5/image_rect_color
      channel: CAM_BACK

test command

python -m perception_dataset.convert --config <path to config yaml file> --synthetic --generate-bbox-from-cuboid

Reference

Notes for reviewer

enable synthetic camera images decoding - compressed_msg_to_numpy has been changed to be able to decode both synthetic and real data (synthetic data have bgr8 encoding).

fixed camera view orientation for synthetic data - this change was done to fix the orientation of camera view and to be able to map the annotations to images properly

increased _TIMESTAMP_DIFF to be able to generate syntetic data - work… - this change was done to workaround the issue with too big timestamp mismatch between camera and lidar corresponding frames (should be decreased when the issue is fixed)

implemented generation of 2d bounding boxes annotations in object_ann… - this is implementation of mapping annotations to 2d bounding boxes annotations of images

[pawel-kotowski]

The code runs ok on my side.

[MagdalenaKotynia]

@yukke42
I noticed that the code cannot be run after rebase to last three main commits (starting from this commit d283ecb). I see that in this commit 2caa1a9 some packages from autoware_auto_perception_msgs were migrated to tier4_perception_msgs. I cloned repository again and built and installed ros dependencies from the instructions in README but the Python interpreter does not find the module tier4_perception_msgs so I cannot test the code after rebase to main. Should I revert the rebase to a version that works on my side, or is the current version fine for you?

[yukke42]

@MagdalenaKotynia Did you run source install/setup.bash or something? https://github.com/tier4/tier4_perception_dataset#setup
All tests are passed, there doesn't seem to be any issue.

[MagdalenaKotynia]

@MagdalenaKotynia Did you run source install/setup.bash or something? https://github.com/tier4/tier4_perception_dataset#setup All tests are passed, there doesn't seem to be any issue.

@yukke42
Yes, I run source install/setup.bash.
When I changed colcon build --symlink-install --cmake-args -DCMAKE_BUILD_TYPE=Release --packages-up-to autoware_auto_perception_msgs from instruction
to colcon build --symlink-install --cmake-args -DCMAKE_BUILD_TYPE=Release --packages-up-to tier4_perception_msgs, it works now.

[miursh]

@yukke42 Is this pull request ready to be merged?