Real-time GPU Object Detection with Joynext

Table of Contents

1. Introduction
2. Software Architecture
3. Hardware Setup
4. Object Detection in the Cloud
5. Object Detection in the Edge
6. Comparison of Results
7. Modifying the Eurovision Models
8. Live Camera Inputs in the Edge
9. Conclusion
10. Issues Faced

Article

Introduction

Hi everyone! I'm Kushon from China, and I'm excited to share with You our recent work on live streaming object detection by leveraging GPU with AVA platform on Evo. In this article, I will give you a brief overview of our project and the different aspects we've explored.

Software Architecture

Our software architecture comprises both cloud and Edge sites. In the cloud, we utilize Amazon EC2 G5g instances powered by AWS Graviton2 processors, Based on the ARM architecture. We also make use of a media key GeForce RTX 3080 GPU. On the Edge side, we use Ava platform with an ARM-based CPU and manually plug in an Nvidia GPU GeForce RTX 3080. This ensures that both the cloud and Edge sides have a parity environment for consistent results.

Hardware Setup

To run the AI models and leverage the GPU, we first install the Nvidia GPU driver in Evo. We then use NGC containers, which come pre-installed with Coda toolkit and support popular AI frameworks. Our final hardware setup consists of Nvidia GPU, ARM-based CPU on Ava platform, and Raspberry Pi with a camera capturing the road video.

Object Detection in the Cloud

To demonstrate the object detection process, we start by running the RTSP server container to Read the video file. We then run the Euro object detector container, which performs object detections on each frame and saves the results either to files or sends them through WebRTC. The output video can be viewed on a browser of another PC with WebRTC.

Object Detection in the Edge

In the Edge side, we face a limitation where Eva running on the platform does not support camera inputs. As an alternative, we run the RTSP server container on Raspberry Pi with the camera capturing the road video. The Euro object detector container runs on Ava platform with the Nvidia GPU, performing object detections. The results of live streaming object detection can be viewed on the browser of another PC with WebRTC.

Comparison of Results

When comparing the results of object detection in both the cloud and Edge sides, we find that the accuracy of object detection is the same on each frame. However, we Notice a difference in FPS (frames per Second). The cloud side achieves around 24 FPS, while the Edge side achieves around 15 FPS. Despite the difference in FPS, the accuracy remains consistent.

Modifying the Eurovision Models

One of the advantages of our setup is the ability to easily and quickly modify the Eurovision models in the container. We can then pull and update the container image on our Edge site. This flexibility allows us to experiment with different models and improve the accuracy and performance of object detection.

Live Camera Inputs in the Edge

In addition to using video files, we also perform live object detection with camera inputs on the Edge side. The process involves checking the OS and GPU information on Ava platform, starting the Euro V5 and WebRTC server, and running the RTSP server on Raspberry Pi to process live camera inputs. The results of live streaming object detection can be viewed in real-time on a browser.

Conclusion

In this project, we have verified the complete process of applying live streaming object detection by leveraging GPU with Ava platform on Evo. We have successfully developed and deployed the necessary software and hardware components. Despite facing a few issues, such as the lack of camera device support in the Current version of Evo platform and the size limitation of container images, we have achieved promising results.

Issues Faced

During the course of our work, we encountered a couple of issues. Firstly, we found that the current version of Evo platform doesn't support camera devices, requiring us to connect the camera to Raspberry Pi separately. Secondly, we faced difficulties in building and deploying container images due to the large size of the image, which includes PIP packages and CoDA toolkit. We Are actively working on addressing these issues for future improvements.

And that concludes our article on live streaming object detection by leveraging GPU with Ava platform on Evo. Thank you for reading!

Highlights

• Live streaming object detection using GPU with Ava platform on Evo
• Software architecture for cloud and Edge sites
• Hardware setup with Nvidia GPU and ARM-based CPU
• Object detection process in the cloud and Edge
• Comparison of object detection results between cloud and Edge
• Modifying and updating Eurovision models for improved accuracy
• Live camera inputs for real-time object detection in the Edge
• Challenges and issues faced during the project

FAQ

Q: What is the software architecture used for live streaming object detection? A: Our software architecture comprises both cloud and Edge sites. In the cloud, we leverage Amazon EC2 G5g instances powered by AWS Graviton2 processors and a media key GeForce RTX 3080 GPU. On the Edge side, we utilize Ava platform with an ARM-based CPU and Nvidia GPU GeForce RTX 3080.

Q: How do you perform object detection in the cloud and Edge? A: In the cloud, we run the RTSP server container to read the video file and the Euro object detector container for object detection. On the Edge side, we capture the road video using a Raspberry Pi with a camera and run the RTSP server container. The Euro object detector container performs object detection using the Nvidia GPU.

Q: What is the difference in object detection results between the cloud and Edge sides? A: The accuracy of object detection remains consistent on each frame, but there is a difference in frames per second (FPS). The cloud side achieves around 24 FPS, while the Edge side achieves around 15 FPS. Despite the difference in FPS, the accuracy of object detection is the same.

Q: Can the Eurovision models be modified and updated? A: Yes, one of the advantages of our setup is the ability to easily modify and update the Eurovision models in the container. We can experiment with different models and improve the accuracy and performance of object detection.

Q: Can live camera inputs be used for object detection in the Edge? A: Yes, in addition to using video files, we can perform live object detection with camera inputs in the Edge side. The process involves running the RTSP server on Raspberry Pi to process live camera inputs and viewing the results in real-time on a browser.

Q: What challenges did you face during the project? A: We encountered a couple of issues during the project. Firstly, the current version of Evo platform doesn't support camera devices, necessitating the use of Raspberry Pi for camera inputs. Additionally, the size of the container image, which includes PIP packages and CoDA toolkit, posed challenges in building and deploying. We are actively working on addressing these issues for future improvements.