Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(obstacle_cruise_planner): support pointcloud-based obstacles #6907

Conversation

mitukou1109
Copy link
Contributor

@mitukou1109 mitukou1109 commented May 1, 2024

Description

Make obstacle_cruise_planner compatible with sensor_msgs::msg::PointCloud2-based obstacles.

The node subscribes to a sensor_msgs::msg::PointCloud2 topic and converts the point cloud into obstacles by the following steps:

  1. transform the cloud into the trajectory frame
  2. downsample the cloud by voxel grid and apply Euclidean clustering to get candidates of obstacles
  3. check if each point in a cluster is within the stop / slow down margin
  4. when any points are inside the stop / slow down area, create a new obstacle object containing:
    • the nearest point from the ego for stop obstacles
    • the nearest and the farthest from the ego for slow down obstacles

Stop and slow down planning are performed with the same trajectory generation algorithm as for ML-based (autoware_perception_msgs::msg::PredictedObject) obstacles.

Cruising and yielding are not supported since point cloud-based obstacles have no velocity information.

Margin adjustment on curves (#4952) is also unsupported due to the following reason:

  • Point cloud-based obstacles do not have shape information.
  • Unlike PredictedObject polygons, point clouds do not necessarily capture the entire obstacle. This could lead the margin to be smaller than in the ML-based case.

launch PR: autowarefoundation/autoware_launch#980

Tests performed

Psim

ros2 launch autoware_launch planning_simulator.launch.xml map_path:=$HOME/autoware_map/sample-map-planning vehicle_model:=sample_vehicle sensor_model:=sample_sensor_kit

with the following changes to obstacle_cruise_planner.param.yaml:

/**:
  ros__parameters:
    common:
     stop_obstacle_type:
        unknown: false
        car: false
        truck: false
        bus: false
        trailer: false
        motorcycle: false
        bicycle: false
        pedestrian: false
        pointcloud: true

     slow_down_obstacle_type:
        unknown: false
        car: false
        truck: false
        bus: false
        trailer: false
        motorcycle: false
        bicycle: false
        pedestrian: false
        pointcloud: true

    behavior_determination:
      pointcloud_search_radius: 5.0
      pointcloud_voxel_grid_x: 0.05
      pointcloud_voxel_grid_y: 0.05
      pointcloud_voxel_grid_z: 100000.0
      pointcloud_cluster_tolerance: 1.0
      pointcloud_min_cluster_size: 1
      pointcloud_max_cluster_size: 100000
Screencast.from.2024.06.27.16.40.32.webm

Evaluator:

Effects on system behavior

It is confirmed that the changes do not affect the behavior when the point cloud handling is disabled.

Pre-review checklist for the PR author

The PR author must check the checkboxes below when creating the PR.

In-review checklist for the PR reviewers

The PR reviewers must check the checkboxes below before approval.

  • The PR follows the pull request guidelines.
  • The PR has been properly tested.
  • The PR has been reviewed by the code owners.

Post-review checklist for the PR author

The PR author must check the checkboxes below before merging.

  • There are no open discussions or they are tracked via tickets.
  • The PR is ready for merge.

After all checkboxes are checked, anyone who has write access can merge the PR.

@github-actions github-actions bot added the component:planning Route planning, decision-making, and navigation. (auto-assigned) label May 1, 2024
@@ -87,6 +117,8 @@ struct Obstacle
std::vector<PredictedPath> predicted_paths;
double ego_to_obstacle_distance;
double lat_dist_from_obstacle_to_traj;
PointCloud pointcloud;
bool pointcloud_repr;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you explain the purpose of this variable?

twist.linear.x = twist.linear.y = twist.angular.z = 0.0;

Eigen::Vector4d centroid;
pcl::compute3DCentroid(pointcloud, centroid);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you calculate centroid of pointcloud?

@github-actions github-actions bot added the component:launch Launch files, scripts and initialization tools. (auto-assigned) label May 1, 2024
if (
(obstacle.source_type == Obstacle::SourceType::PREDICTED_OBJECT &&
!isStopObstacle(obstacle.classification.label)) ||
(obstacle.source_type == Obstacle::SourceType::POINTCLOUD && !use_pointcloud_for_stop_)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[imo] it's a little bit difficult to understand this condition.

This is my suggestion.

  if (
    (obstacle.source_type == Obstacle::SourceType::PREDICTED_OBJECT &&
     !isStopObstacle(obstacle.classification.label)) {
    return std::nullopt;
  }

  if (
    (obstacle.source_type == Obstacle::SourceType::POINTCLOUD && !use_pointcloud_for_stop_)) {
    return std::nullopt;
  }

!enable_slow_down_planning_ ||
(obstacle.source_type == Obstacle::SourceType::PREDICTED_OBJECT &&
!isSlowDownObstacle(obstacle.classification.label)) ||
(obstacle.source_type == Obstacle::SourceType::POINTCLOUD && !use_pointcloud_for_slow_down_)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@satoshi-ota satoshi-ota added run:build-and-test-differential Mark to enable build-and-test-differential workflow. (used-by-ci) and removed run:build-and-test-differential Mark to enable build-and-test-differential workflow. (used-by-ci) labels Jul 2, 2024
@satoshi-ota satoshi-ota added run:build-and-test-differential Mark to enable build-and-test-differential workflow. (used-by-ci) and removed tag:run-clang-tidy-differential labels Jul 3, 2024
Copy link

codecov bot commented Jul 3, 2024

Codecov Report

Attention: Patch coverage is 18.39080% with 142 lines in your changes missing coverage. Please review.

Project coverage is 28.32%. Comparing base (c9aa86c) to head (8e88283).
Report is 1 commits behind head on main.

Files Patch % Lines
...ning/autoware_obstacle_cruise_planner/src/node.cpp 19.13% 126 Missing and 5 partials ⚠️
...utoware/obstacle_cruise_planner/common_structs.hpp 0.00% 10 Missing ⚠️
..._obstacle_cruise_planner/src/planner_interface.cpp 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6907      +/-   ##
==========================================
- Coverage   28.69%   28.32%   -0.37%     
==========================================
  Files        1587     1585       -2     
  Lines      116143   115614     -529     
  Branches    49559    49305     -254     
==========================================
- Hits        33326    32750     -576     
- Misses      73723    73871     +148     
+ Partials     9094     8993     -101     
Flag Coverage Δ *Carryforward flag
differential 6.62% <18.39%> (?)
total 28.34% <ø> (-0.35%) ⬇️ Carriedforward from 6a70824

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@satoshi-ota satoshi-ota merged commit 9e47b4d into autowarefoundation:main Jul 3, 2024
49 of 51 checks passed
palas21 pushed a commit to palas21/autoware.universe that referenced this pull request Jul 12, 2024
…towarefoundation#6907)

* add pointcloud to obstacle properties

* add tf listener & pointcloud subscriber

* add parameters for pointcloud obstacle

* add type aliases

* convert pointcloud to obstacle

* add type alias

* add polygon conversion for pointcloud obstacle

* initialize twist & pose of pointcloud obstacle

* overload to handle both obstacle & predicted path

* implement ego behavior determination against pointcloud obstacles

* generate obstacle from point

* revert getCollisionIndex()

* generate obstacle from each point in cloud

* set pointcloud obstacle velocity to 0

* use tf buffer & listener with pointers

* update latest pointcloud data

* add topic remap

* remove unnecessary includes

* set slow down obstacle velocity to 0

* add flag to consider pointcloud obstacle for stopping & slowing down

* style(pre-commit): autofix

* downsample pointcloud using voxel grid

* change  shape type of pointcloud obstacle to polygon

* convert pointcloud to obstacle by clustering

* add parameters for clustering

* add max_num_points parameter to dummy object

* downsample pointcloud when the number of points is larger than max_num_points

* add max_num_points property to dummy bus

* add parameters for pointcloud based obstacles

* store pointcloud in obstacle struct

* change obstacle conversion method

* migrate previous changes to new package

* store necessary points only

* move use_pointcloud to common parameter

* extract necessary points from pointcloud

* add use_pointcloud parameter to planner interface

* fix obstacle conversion

* fix collision point determination

* simplify pointcloud transformation

* style(pre-commit): autofix

* fix collision point determination

* pick nearest stop collision point

* check collision for every point in cluster

* migrate previous changes to new files

* reduce diff

* remove use_pointcloud parameter

* add parameters for pointcloud filtering

* add autoware namespace

* Revert "add max_num_points parameter to dummy object"

This reverts commit 98bcd08.

* Revert "downsample pointcloud when the number of points is larger than max_num_points"

This reverts commit fb00b59.

* Revert "add max_num_points property to dummy bus"

This reverts commit 5f9e4ab.

* feat(diagnostic_graph_utils): add logging tool

Signed-off-by: Takagi, Isamu <[email protected]>

* fix all OK

Signed-off-by: Takagi, Isamu <[email protected]>

* feat(default_ad_api): add log when operation mode change fails

Signed-off-by: Takagi, Isamu <[email protected]>

* get only the necessary one of object or pointcloud data

* addfield for obstacle source type

* enable simultaneous use of PredictedObjects and PointCloud

* separate convertToObstacles() by source type

* avoid using pointer

* reduce diff

* make nest shallower

* define vector concatenate function

* shorten variable names

* fix redundant condition

---------

Signed-off-by: Takagi, Isamu <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Takagi, Isamu <[email protected]>
Co-authored-by: Satoshi OTA <[email protected]>
Signed-off-by: palas21 <[email protected]>
tby-udel pushed a commit to tby-udel/autoware.universe that referenced this pull request Jul 14, 2024
…towarefoundation#6907)

* add pointcloud to obstacle properties

* add tf listener & pointcloud subscriber

* add parameters for pointcloud obstacle

* add type aliases

* convert pointcloud to obstacle

* add type alias

* add polygon conversion for pointcloud obstacle

* initialize twist & pose of pointcloud obstacle

* overload to handle both obstacle & predicted path

* implement ego behavior determination against pointcloud obstacles

* generate obstacle from point

* revert getCollisionIndex()

* generate obstacle from each point in cloud

* set pointcloud obstacle velocity to 0

* use tf buffer & listener with pointers

* update latest pointcloud data

* add topic remap

* remove unnecessary includes

* set slow down obstacle velocity to 0

* add flag to consider pointcloud obstacle for stopping & slowing down

* style(pre-commit): autofix

* downsample pointcloud using voxel grid

* change  shape type of pointcloud obstacle to polygon

* convert pointcloud to obstacle by clustering

* add parameters for clustering

* add max_num_points parameter to dummy object

* downsample pointcloud when the number of points is larger than max_num_points

* add max_num_points property to dummy bus

* add parameters for pointcloud based obstacles

* store pointcloud in obstacle struct

* change obstacle conversion method

* migrate previous changes to new package

* store necessary points only

* move use_pointcloud to common parameter

* extract necessary points from pointcloud

* add use_pointcloud parameter to planner interface

* fix obstacle conversion

* fix collision point determination

* simplify pointcloud transformation

* style(pre-commit): autofix

* fix collision point determination

* pick nearest stop collision point

* check collision for every point in cluster

* migrate previous changes to new files

* reduce diff

* remove use_pointcloud parameter

* add parameters for pointcloud filtering

* add autoware namespace

* Revert "add max_num_points parameter to dummy object"

This reverts commit 98bcd08.

* Revert "downsample pointcloud when the number of points is larger than max_num_points"

This reverts commit fb00b59.

* Revert "add max_num_points property to dummy bus"

This reverts commit 5f9e4ab.

* feat(diagnostic_graph_utils): add logging tool

Signed-off-by: Takagi, Isamu <[email protected]>

* fix all OK

Signed-off-by: Takagi, Isamu <[email protected]>

* feat(default_ad_api): add log when operation mode change fails

Signed-off-by: Takagi, Isamu <[email protected]>

* get only the necessary one of object or pointcloud data

* addfield for obstacle source type

* enable simultaneous use of PredictedObjects and PointCloud

* separate convertToObstacles() by source type

* avoid using pointer

* reduce diff

* make nest shallower

* define vector concatenate function

* shorten variable names

* fix redundant condition

---------

Signed-off-by: Takagi, Isamu <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Takagi, Isamu <[email protected]>
Co-authored-by: Satoshi OTA <[email protected]>
KhalilSelyan pushed a commit that referenced this pull request Jul 22, 2024
)

* add pointcloud to obstacle properties

* add tf listener & pointcloud subscriber

* add parameters for pointcloud obstacle

* add type aliases

* convert pointcloud to obstacle

* add type alias

* add polygon conversion for pointcloud obstacle

* initialize twist & pose of pointcloud obstacle

* overload to handle both obstacle & predicted path

* implement ego behavior determination against pointcloud obstacles

* generate obstacle from point

* revert getCollisionIndex()

* generate obstacle from each point in cloud

* set pointcloud obstacle velocity to 0

* use tf buffer & listener with pointers

* update latest pointcloud data

* add topic remap

* remove unnecessary includes

* set slow down obstacle velocity to 0

* add flag to consider pointcloud obstacle for stopping & slowing down

* style(pre-commit): autofix

* downsample pointcloud using voxel grid

* change  shape type of pointcloud obstacle to polygon

* convert pointcloud to obstacle by clustering

* add parameters for clustering

* add max_num_points parameter to dummy object

* downsample pointcloud when the number of points is larger than max_num_points

* add max_num_points property to dummy bus

* add parameters for pointcloud based obstacles

* store pointcloud in obstacle struct

* change obstacle conversion method

* migrate previous changes to new package

* store necessary points only

* move use_pointcloud to common parameter

* extract necessary points from pointcloud

* add use_pointcloud parameter to planner interface

* fix obstacle conversion

* fix collision point determination

* simplify pointcloud transformation

* style(pre-commit): autofix

* fix collision point determination

* pick nearest stop collision point

* check collision for every point in cluster

* migrate previous changes to new files

* reduce diff

* remove use_pointcloud parameter

* add parameters for pointcloud filtering

* add autoware namespace

* Revert "add max_num_points parameter to dummy object"

This reverts commit 98bcd08.

* Revert "downsample pointcloud when the number of points is larger than max_num_points"

This reverts commit fb00b59.

* Revert "add max_num_points property to dummy bus"

This reverts commit 5f9e4ab.

* feat(diagnostic_graph_utils): add logging tool

Signed-off-by: Takagi, Isamu <[email protected]>

* fix all OK

Signed-off-by: Takagi, Isamu <[email protected]>

* feat(default_ad_api): add log when operation mode change fails

Signed-off-by: Takagi, Isamu <[email protected]>

* get only the necessary one of object or pointcloud data

* addfield for obstacle source type

* enable simultaneous use of PredictedObjects and PointCloud

* separate convertToObstacles() by source type

* avoid using pointer

* reduce diff

* make nest shallower

* define vector concatenate function

* shorten variable names

* fix redundant condition

---------

Signed-off-by: Takagi, Isamu <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Takagi, Isamu <[email protected]>
Co-authored-by: Satoshi OTA <[email protected]>
Ariiees pushed a commit to Ariiees/autoware.universe that referenced this pull request Jul 22, 2024
…towarefoundation#6907)

* add pointcloud to obstacle properties

* add tf listener & pointcloud subscriber

* add parameters for pointcloud obstacle

* add type aliases

* convert pointcloud to obstacle

* add type alias

* add polygon conversion for pointcloud obstacle

* initialize twist & pose of pointcloud obstacle

* overload to handle both obstacle & predicted path

* implement ego behavior determination against pointcloud obstacles

* generate obstacle from point

* revert getCollisionIndex()

* generate obstacle from each point in cloud

* set pointcloud obstacle velocity to 0

* use tf buffer & listener with pointers

* update latest pointcloud data

* add topic remap

* remove unnecessary includes

* set slow down obstacle velocity to 0

* add flag to consider pointcloud obstacle for stopping & slowing down

* style(pre-commit): autofix

* downsample pointcloud using voxel grid

* change  shape type of pointcloud obstacle to polygon

* convert pointcloud to obstacle by clustering

* add parameters for clustering

* add max_num_points parameter to dummy object

* downsample pointcloud when the number of points is larger than max_num_points

* add max_num_points property to dummy bus

* add parameters for pointcloud based obstacles

* store pointcloud in obstacle struct

* change obstacle conversion method

* migrate previous changes to new package

* store necessary points only

* move use_pointcloud to common parameter

* extract necessary points from pointcloud

* add use_pointcloud parameter to planner interface

* fix obstacle conversion

* fix collision point determination

* simplify pointcloud transformation

* style(pre-commit): autofix

* fix collision point determination

* pick nearest stop collision point

* check collision for every point in cluster

* migrate previous changes to new files

* reduce diff

* remove use_pointcloud parameter

* add parameters for pointcloud filtering

* add autoware namespace

* Revert "add max_num_points parameter to dummy object"

This reverts commit 98bcd08.

* Revert "downsample pointcloud when the number of points is larger than max_num_points"

This reverts commit fb00b59.

* Revert "add max_num_points property to dummy bus"

This reverts commit 5f9e4ab.

* feat(diagnostic_graph_utils): add logging tool

Signed-off-by: Takagi, Isamu <[email protected]>

* fix all OK

Signed-off-by: Takagi, Isamu <[email protected]>

* feat(default_ad_api): add log when operation mode change fails

Signed-off-by: Takagi, Isamu <[email protected]>

* get only the necessary one of object or pointcloud data

* addfield for obstacle source type

* enable simultaneous use of PredictedObjects and PointCloud

* separate convertToObstacles() by source type

* avoid using pointer

* reduce diff

* make nest shallower

* define vector concatenate function

* shorten variable names

* fix redundant condition

---------

Signed-off-by: Takagi, Isamu <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Takagi, Isamu <[email protected]>
Co-authored-by: Satoshi OTA <[email protected]>
@mitukou1109 mitukou1109 deleted the obstacle_cruise_planner_pointcloud branch September 13, 2024 04:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component:launch Launch files, scripts and initialization tools. (auto-assigned) component:planning Route planning, decision-making, and navigation. (auto-assigned) run:build-and-test-differential Mark to enable build-and-test-differential workflow. (used-by-ci)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants