Milestone of graspnet

[MaxWaterhout]

Description

The default of graspnet samples different grasps that are not in collision and do not overlap. You can read the paper here.

We created two forks for graspnet-baseline and GraspnetAPI
Download the checkpoint here

Don't forget to syn uv:

uv sync --group graspnet -v

Note: Uv sync can take a while because torch also downloads some bigger cuda packages, that is why sometimes -v is recommended to see the progress.

Start the simulation in a first terminal:

ros2 launch rcdt_launch robotics.launch.py configuration:=franka_realsense

Start the ui in a second terminal:

ros2 run rcdt_utilities simple_gui.py

Open the ui (http://localhost:8080) and execute the actions:

• update image (color), update image (depth), update camera info
• generate grasps
• visualize grasps
• create plan
• visualize plan
• execute plan

Todo

• Filter grasps poses
• Fix update functions in UI sometimes fail
• Fix visualize grasp is incorrect first time
• Add gripper control
• Add obstacles based on pointcloud

Testing

Explain how you tested your changes.

Documentation

• I have updated the documentation (if necessary)

Additional Notes

Any relevant screenshots, logs, or context.