Leetop Jetson Cloud Demo 7新功能预览

Isaac ROS Dev Base Container

l Isaac ROS Pose Estimation

此demo将使用NVIDIA GPU加速包为3D对象姿态预估做加速。使用深度学习的姿态估计模型和单目相机，isaac_ros_dope和isaac-ros_ centerpose包可以实现估计目标对象的6自由度姿态。

l 配置环境

下载源码包

$ mkdir -p ~/workspaces/isaac_ros-dev/src

$ cd ~/workspaces/isaac_ros-dev/src

$ git clone [https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common)

$ git clone [https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_nitros](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_nitros)

$ git clone [https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_pose_estimation](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_pose_estimation)

$ git clone https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_dnn_inference

下载ROS Bag数据包

$ cd ~/workspaces/isaac_ros-dev/src/isaac_ros_pose_estimation && \

 git lfs pull -X "" -I "resources/rosbags/"

运行run_dev.sh脚本搭建Docker容器

$ cd ~/workspaces/isaac_ros-dev/src/isaac_ros_common && \

 ./scripts/run_dev.sh

$ mkdir -p /tmp/models/

在DOPE GitHub下载本例程使用的Ketchup.pth模型，在PC(x86_64)机上将模型转成onnx文件，然后存放于Docker里的/tmp/models/目录中

## 注意：此步骤在PC端执行

$ python3 /workspaces/isaac_ros-dev/src/isaac_ros_pose_estimation/isaac_ros_dope/scripts/dope_converter.py --format onnx --input /tmp/models/Ketchup.pth

## 注意：此步骤在Jetson执行，假设onnx模型放于~/Downloads

$ cd ~/Downloads && \

docker cp Ketchup.onnx isaac_ros_dev-aarch64-container:/tmp/models

l 编译 && 运行

进入上面搭建好的Dokcer容器

$ cd ~/workspaces/isaac_ros-dev/src/isaac_ros_common && \

 ./scripts/run_dev.sh

$ cd /workspaces/isaac_ros-dev && \

 colcon build --symlink-install && \

 source install/setup.bash

Note: 可运行 colcon test --executor sequential 以验证编译安装的完整性和正确性

运行以下启动文件以启动此软件包的演示：

执行isaac_ros_dope

$ ros2 launch isaac_ros_dope isaac_ros_dope_tensor_rt.launch.py model_file_path:=/tmp/models/Ketchup.onnx engine_file_path:=/tmp/models/Ketchup.plan

打开另一个终端窗口，并进入Docker容器

$ cd ~/workspaces/isaac_ros-dev/src/isaac_ros_common && \

 ./scripts/run_dev.sh

然后运行ROS bag

$ ros2 bag play -l src/isaac_ros_pose_estimation/resources/rosbags/dope_rosbag/

打开第三个终端窗口并进入Docker查看pose信息

$ cd ~/workspaces/isaac_ros-dev/src/isaac_ros_common && \

 ./scripts/run_dev.sh

$ ros2 topic echo /poses

Note: 当前显示的/poses是从/dope/pose_array这个原始topic获取的poses信息

使用rviz2显示pose数据

在第三个进入Docker的终端里输入

$ rviz2

此时将出现rviz2工具的窗口界面：

image.png

然后点击Add按钮，在By topic 里的PoseArray下选择/poses选项，确保将固定帧更新为摄像机。最后，可通过改变Shape的Axes信息来更新显示位置。
Note: 为了获得最佳效果，请将输入图像裁剪或调整大小，使其与DNN模型所期望的尺寸相同

支持如下的设备：
Module Jetpack Orin devkit module Orin 32G 35.0 EA V2 Orin devkit module Orin 32G 35.0 EA V2 Orin devkit 35.0 EA V2 Orin devkit 35.0 EA V2 Xavier devkit module Xavier 32G 35.0 EA V2 Xavier Industry Xavier Industry Xavier NX 35.0 EA V2 Xavier NX 16G 35.0 EA V2 Nano R 32.7.2 TX2 NX R 32.7.2 Xavier devkit module Xavier 32G R 32.7.2 Xavier Industry Xavier Industry Xavier NX R 32.7.2 Xavier NX 16G R 32.7.2 config Software other SW HW
35.0 EA V2 default DS + Full Jetpack + Pytorch + TF Todesk NV devkit
default DS + Full Jetpack + Pytorch + TF Todesk NV devkit
35.0 EA V2 default DS + Full Jetpack + Pytorch + TF Todesk Leetop carry board
default DS + Full Jetpack + Pytorch + TF Todesk Leetop carry board
Orin nx 16G DS + Full Jetpack + Pytorch + TF Todesk
Orin nx 8G DS + Full Jetpack + Pytorch + TF Todesk
35.0 EA V2 default DS + Full Jetpack + Pytorch + TF Todesk NV devkit
default DS + Full Jetpack + Pytorch + TF Todesk Leetop carry board
35.0 EA V2 default DS + Full Jetpack + Pytorch + TF Todesk Leetop carry board
35.0 EA V2 default DS + Full Jetpack + Pytorch + TF Todesk NV devkit
default DS + Full Jetpack + Pytorch + TF Todesk Leetop carry board
default DS + Full Jetpack + Pytorch + TF Todesk Leetop carry board
default DS + Full Jetpack + Pytorch + TF Todesk Leetop carry board
default DS + Full Jetpack + Pytorch + TF Todesk Leetop carry board
R 32.7.2 default DS + Full Jetpack + Pytorch + TF Todesk NV devkit
default DS + Full Jetpack + Pytorch + TF Todesk Leetop carry board
R 32.7.2 default DS + Full Jetpack + Pytorch + TF Todesk Leetop carry board
R 32.7.2 default DS + Full Jetpack + Pytorch + TF Todesk NV devkit
default DS + Full Jetpack + Pytorch + TF Todesk Leetop carry board
default DS + Full Jetpack + Pytorch + TF Todesk Leetop carry board