Training Detectnet_v2 on Jetson AGX Orin

[ABDELRAHMAN123491]

Hi @dusty-nv,
Thanks for the continuous effort that you put in upgrading this repository. Recently, I was able to convert the peoplenet model based on detectnet_v2 to .engine extension and run the inference on Jetson AGX Orin using detectnet.py thanks to your hints. Now, I want to customize the model to detect new classes, such as PPEs (helmet, gloves, etc). So, is it possible to train such model using the functions you wrote for transfer learning? Or better train it using TAO toolkit, then convert to .engine and run the inference? What about retraining detectnet_v1, which is supported from the very beginning in this work? Can I retrain the network on Orin device with JetPack 5.0.1. If yes, then do I need to Swap some space for training?

Thanks in Advance!

Please keep up the amazing work!

[ABDELRAHMAN123491]

up!

[dusty-nv]

Hi @ABDELRAHMAN123491, detectnet_v2 is only trained with TAO and TAO supports x86, so it would need to be done on a PC/server that has a discrete NVIDIA GPU. The old detectnet_v1 architecture is outdated and shouldn't be used for new models and was trained with DIGITS and I haven't used that in years.

[ABDELRAHMAN123491]

Hi @ABDELRAHMAN123491, detectnet_v2 is only trained with TAO and TAO supports x86, so it would need to be done on a PC/server that has a discrete NVIDIA GPU. The old detectnet_v1 architecture is outdated and shouldn't be used for new models and was trained with DIGITS and I haven't used that in years.

Thanks for the valuable information! I meant by detectnet_v1:= the SSD mobilenet model - sorry for the confusion - which you have elegantly incorporated in this work for custom data training. Does it support training on jetson orin? and if not, could you provide me with hints to train detectnet_v2?
I have already read the steps on the Nvidia website, and I can tell that the process seems to be difficult and indeed intimidating. starting from KITTI format passing by converting to TFrecord file and ending up with file configuration and training.

I am really grateful for the amazing work you done here! I was able to use the gstreamer with .pt yolo models and the latency is not noticeable at all! I may add this feature to this repo if you permit, which, I believe, can help Yolo users to get rid of opencv, have some stability in viewing the detections, and achieve very low latency.

Thanks @dusty-nv

[dusty-nv]

I meant by detectnet_v1:= the SSD mobilenet model - sorry for the confusion - which you have elegantly incorporated in this work for custom data training. Does it support training on jetson orin?

Oh yes it does, through the same train_ssd.py script using PyTorch, it runs on Jetson Orin just the same as before.

I am really grateful for the amazing work you done here! I was able to use the gstreamer with .pt yolo models and the latency is not noticeable at all! I may add this feature to this repo if you permit, which, I believe, can help Yolo users to get rid of opencv, have some stability in viewing the detections, and achieve very low latency.

That would be great, which version of YOLO are you have working?
The reason I haven't adopted YOLO in this repo, is because there is always new version coming out, and people want the newest ones. Whereas I need a stable network architecture that I know the training & inference scripts are stable running across many different Jetsons and different versions of JetPack for all the people using it.

[ABDELRAHMAN123491]

I meant by detectnet_v1:= the SSD mobilenet model - sorry for the confusion - which you have elegantly incorporated in this work for custom data training. Does it support training on jetson orin?

Oh yes it does, through the same train_ssd.py script using PyTorch, it runs on Jetson Orin just the same as before.

I am really grateful for the amazing work you done here! I was able to use the gstreamer with .pt yolo models and the latency is not noticeable at all! I may add this feature to this repo if you permit, which, I believe, can help Yolo users to get rid of opencv, have some stability in viewing the detections, and achieve very low latency.

That would be great, which version of YOLO are you have working? The reason I haven't adopted YOLO in this repo, is because there is always new version coming out, and people want the newest ones. Whereas I need a stable network architecture that I know the training & inference scripts are stable running across many different Jetsons and different versions of JetPack for all the people using it.

I am using version "5" of Yolo; however, I believe that the way that I adopted the Yolo model is universal and can support the new versions of Yolo. Testing Yolov5, starting from the small version to the large one, I can tell that they perform very well on the Jetson Orin. As may be expected, the small version is processed faster (nearly real-time). It is worth noting that I am running the torch files, i.e., ".pt" not the ".engine" file. If we support running ".engine" yolo-based models, then the processing time shall be 5x faster.

For your kind reference, I have the 32 GB version of the Orin with the utilities shown in the Figure below:

[dusty-nv]

I believe that the way that I adopted the Yolo model is universal and can support the new versions of Yolo.

Oh that's great. Are you using TensorRT like jetson-inference does to run the YOLO inferencing, or PyTorch?

[ABDELRAHMAN123491]

I believe that the way that I adopted the Yolo model is universal and can support the new versions of Yolo.

Oh that's great. Are you using TensorRT like jetson-inference does run the YOLO inferencing or PyTorch?

Actually, I am using Pytorch. The conversion from yolo to TensorRT, and then carrying out the inference using Python is something that I am working on. There are multiple issues that I encountered through the steps of running TensorRT yolo-based models. For instance, the conversion on Jetson Orin with the JetPack that I have seems to lack support and inferring the “.engine” is not straightforward as well.

[ABDELRAHMAN123491]

I attempted to use the export.py file, available in the yolov5 repository, to convert to .engine; however, It yields the following error: Expected all tensors to be on the same device, but found at least two devices...

[dusty-nv]

Typically that means that some tensor is on CPU and some on GPU (i.e. .cuda() or .to(device=cuda0)), or that .cuda() wasn't called on the model. It doesn't really matter if the tensors are on the CPU or GPU for ONNX export, but whatever you pick they should be the same.

[ABDELRAHMAN123491]

Typically that means that some tensor is on CPU and some on GPU (i.e. .cuda() or .to(device=cuda0)), or that .cuda() wasn't called on the model. It doesn't really matter if the tensors are on the CPU or GPU for ONNX export, but whatever you pick they should be the same.

The intriguing thing is that I explicitly defined the device parameter and made sure that it runs only on GPU. Have you come across the export.py that is available in the yolov5 repository before? They provide conversion to wide range of extensions including .engine. The key thing is that using their script shall ease the inference part, as you can use directly torch.hub.load to load the TensorRt model and use it easily in the code as any normal yolo model. They support conversions to FP32 and FP16 precisions. This shall make the detections faster 5x. So, if you could give export.py a shot and examine the conversions to .engine model, that would be awesome.

Besides, here is my personal email: abdulrhman123491@gmail.com for easier future communication. I am excited to add to this work, and share with you the script of supporting running Yolo models using Pytorch until we find a way to support the .engine extensions.

Thanks @dusty-nv

[ligaz]

@dusty-nv You can check a "native" Yolo integration in jetson-inference in my fork here.

It includes:

• efficientNMS plugin support.
• Letterboxing pre and post processing.

I would be happy to figure out a way how we can upstream these changes. My first attempt was this PR.

I have some other ideas like having a separate yolonet class (apart from detectnet) that has those integrated.

[ABDELRAHMAN123491]

@dusty-nv You can check a "native" Yolo integration in jetson-inference in my fork here.

It includes:

• efficientNMS plugin support.
• Letterboxing pre and post processing.

I would be happy to figure out a way how we can upstream these changes. My first attempt was this PR.

I have some other ideas like having a separate yolonet class (apart from detectnet) that has those integrated.

Hi @ligaz,
I can see that you support Yolov7 in your code, is that correct? Have you tested the detectnet. cpp with yolo models? How did you find the performance?

[ligaz]

Yolo v7 tiny runs at 55-60ms on the Jetson Nano.

Although the method name is yolov7 those changes will work with any model that uses efficientNMSPlugin TensorRT plugin including yolo v5, v7 and v8. You can check this repo as a reference how you can export a YOLO model with efficientNMSPlugin.

[VitorHugoAguiar]

Hi @ligaz.
How to test a yolo model? There is any way of testing using my-detection.py?