Building a Custom AI Drone: Tracking Humans with Precision

Table of Contents

1. Introduction
2. Building Stanley, the Drone
3. Programming Stanley to Follow
4. Using the Oak-D Camera
5. Comparison with Other Cameras
6. Depth Maps and Inference
7. Integrating the Code
8. testing and Debugging
9. Subsumption Architecture
10. Final Testing and Conclusion
11. Challenges and Lessons Learned

Introduction

In this article, we will explore the journey of building and programming Stanley, a drone powered by an NVIDIA Jetson Nano. We will dive into the process of programming Stanley to follow the user using artificial intelligence. Additionally, we will discover the capabilities of the Oak-D camera, a compact yet powerful camera with a vision processing unit capable of running AI models. We will compare the Oak-D camera with other cameras on the market and explore the concept of depth maps and inference. Throughout this article, we will delve into the challenges and lessons learned, providing a comprehensive understanding of the process. So let's strap in and take flight with Stanley!

Building Stanley, the Drone

Stanley, the drone, is powered by an NVIDIA Jetson Nano and equipped with an Oak-D camera. The journey of building Stanley hasn't been without its obstacles, as we've encountered technical issues along the way. Initially, Stanley experienced damage to its arm and GPS antenna, requiring repairs. However, with determination and the right tools, Stanley was brought back to a functioning state. The process of building and repairing drones requires attention to detail and problem-solving skills. Despite the setbacks, Stanley was soon ready to take flight again.

Programming Stanley to Follow

Now that Stanley was back in good Shape, the next step was to program it to follow the user. This required utilizing artificial intelligence and harnessing the power of the Oak-D camera. The Oak-D camera is known for its depth maps and impressive processing capabilities. By leveraging the SDK provided by exenous, developers can easily communicate with the Oak-D camera and integrate it into their applications. Playing around with the depth AI SDK, our aim was to achieve real-time detection of people. With some experiments using Tiny YOLO, we were able to successfully detect and track the user.

Using the Oak-D Camera

The Oak-D camera proves itself as a worthy successor to the Xbox Kinect, with its quality depth maps and affordable price. With a Myriad X Vision Processing Unit onboard, the camera can perform at an impressive four trillion operations per Second. It supports AI models such as MobileNet SSD and YOLO, allowing users to run their own models directly on the camera. The compact size of the camera makes it suitable for various applications, and its specifications make it a powerful tool in the research community. To learn more about the Oak-D camera, visit their website for detailed specifications and information.

Comparison with Other Cameras

While the Oak-D camera is a remarkable piece of technology, it's important to consider other options on the market. One such camera is the Bottlenose, available in both stereo and mono options. Manufactured by Lab Forge, the Bottlenose camera offers promising features and is currently available for pre-order. Its stereo capabilities provide depth Perception, making it an interesting competitor to the Oak-D camera. A thorough review of the Bottlenose camera and a comparison to the Oak-D camera will be conducted to determine its performance in various scenarios. Stay tuned for future updates on this exciting camera.

Depth Maps and Inference

To truly understand the capabilities of the Oak-D camera, it's essential to explore depth maps and how they enable accurate inference. In previous parts of this series, we covered the generation of depth maps using stereo algorithms such as Stereo BM and Stereo SGBM. These depth maps serve as valuable input for AI models running on the Jetson Nano. The combination of depth maps and inference allows for real-time tracking and detection of objects, making it possible for Stanley to follow the user with precision.

Integrating the Code

With the Oak-D camera and AI models in place, the next challenge was integrating the code and ensuring smooth operation. A trial and error approach was employed, with continuous testing and refining of the code. To minimize risks and avoid potential accidents, a virtual drone was used initially to simulate the behavior of Stanley. This allowed for thorough testing and ironing out any issues in a controlled environment. Additionally, a visualization tool was developed to observe and analyze the behavior of the code in response to changes in the user's position. These measures ensured a smoother development process and minimized the chances of errors when deploying the code on the actual hardware.

Testing and Debugging

Testing and debugging are crucial phases in any project, and Stanley's development was no exception. Several issues surfaced during the testing phase, highlighting the importance of meticulous debugging. One notable issue was related to the drone's behavior during takeoff in guided mode. A careful inspection of the code revealed a divide by zero problem, which was promptly rectified. This emphasized the importance of thorough testing and continuous improvement when developing complex systems like Stanley.

Subsumption Architecture

To achieve reliable and intelligent behavior in Stanley, a subsumption architecture was implemented. This architecture allowed for the prioritization of certain rules based on the presence of objects or people. By assigning different levels of priority to each rule, the system could make decisions based on the most Relevant information at any given time. For example, if a person was detected, the rule to fly towards them would take precedence, while the rule to hover in place would be overridden. This approach ensured that Stanley could react and adapt to various situations, making it a more reliable and intelligent drone.

Final Testing and Conclusion

After refining the code and addressing the bugs, it was time to conduct final testing in real-world environments. Stanley was taken to a test location where various scenarios were simulated to evaluate its performance. Although some initial tests didn't go as planned, they provided valuable insights and led to further refinement of the code. With perseverance and determination, Stanley was finally able to track and follow the user successfully. The project, a year in the making, showcased the incredible progress made in the field of robotics and artificial intelligence.

Challenges and Lessons Learned

Throughout the development of Stanley, numerous challenges were encountered. From physical damages to technical issues, the journey was not without its setbacks. However, these challenges served as valuable learning opportunities. Troubleshooting techniques, debugging skills, and the importance of thorough testing were all lessons learned along the way. The determination to overcome obstacles and the commitment to continuous improvement were critical factors in the success of this project. With each challenge conquered, the team grew more resilient and capable of tackling future endeavors.

Highlights:

• Building Stanley, a drone powered by NVIDIA Jetson Nano
• Programming Stanley to follow the user using AI
• Exploring the features and capabilities of the Oak-D camera
• Comparing the Oak-D camera with other cameras in the market
• Understanding depth maps and their role in inference
• Integrating the code and ensuring seamless operation
• Thorough testing and debugging to iron out issues
• Implementing a subsumption architecture for intelligent behavior
• Final testing and conclusion of the project's success
• Challenges faced throughout the development process and lessons learned

FAQ

Q: How long did it take to build Stanley? A: The process of building and programming Stanley took approximately a year, including debugging and refinement.

Q: Can the Oak-D camera be used with other drones? A: Yes, the Oak-D camera can be integrated into various drone models to enable AI-based tracking and detection.

Q: What were the main challenges faced during the project? A: The main challenges included physical damages to the drone, technical issues with the GPS antenna, and debugging complex code.

Q: Is the subsumption architecture applicable to other robotics projects? A: Yes, the subsumption architecture can be utilized in various robotics projects to prioritize and control different behavior rules.

Q: What lessons were learned from building Stanley? A: Lessons learned include the importance of thorough testing, continuous improvement, and the use of virtual simulations for debugging.