Building a Smart Conveyor Belt with Computer Vision and AI: A Fascinating Journey

Table of Contents

1. Introduction
2. Chris Nogich's Journey in the OpenCV Spatial AI Competition
3. The Project: Building a Smart Conveyor Belt with Computer Vision and AI
   • The Project Origin: Geological Mining Industry
   • Challenges in Creating a Dataset
   • Switching from Rocks to Lego Bricks
   • Goals of the Project
   • Tools and Technologies Used
   • Building the Prototype
   • Training the AI model
   • Hardware Issues and Solutions
   • Lessons Learned
4. Conclusion

Introduction

In this article, we will explore the journey of Chris Nogich in the OpenCV Spatial AI Competition. Chris developed a project that involved creating a smart conveyor belt operated by computer vision and AI algorithms. We will Delve into the challenges faced, the goals of the project, the tools and technologies used, and the final results. Join us as we take a closer look at this fascinating project and the lessons learned along the way.

Chris Nogich's Journey in the OpenCV Spatial AI Competition

Chris Nogich, a participant in the OpenCV Spatial AI Competition in 2022, shares his experiences and insights in this article. Despite working as a consultant with a focus on Microsoft products, Chris developed a keen interest in machine learning, advanced data analytics, and computer vision. He seized the opportunity to participate in the competition, where he aimed to Create a smart conveyor belt using computer vision and AI algorithms.

The Project: Building a Smart Conveyor Belt with Computer Vision and AI

The Project Origin: Geological Mining Industry

The inspiration for Chris's project came from the geological mining industry. His friend, a geologist, sparked his interest in the process of sorting and categorizing various rocks obtained from mining operations. The need for efficient rock selection and categorization motivated Chris to develop a solution using computer vision and AI algorithms.

Challenges in Creating a Dataset

One of the initial challenges faced by Chris was creating a dataset for training the AI model. The available data was skewed, with an uneven distribution of different rock types. Additionally, the number of rock samples was relatively small, making it difficult to train the model effectively. To overcome these challenges, Chris decided to switch from using real rocks to Lego bricks, which offered a more manageable dataset.

Switching from Rocks to Lego Bricks

Chris made the decision to switch from rocks to Lego bricks for several reasons. Firstly, the available rock samples were limited and did not provide a balanced dataset. Secondly, the size and weight of the rocks posed practical challenges for the model and conveyor belt. Finally, Lego bricks offered a more diverse and easily accessible dataset, allowing for better training and testing of the AI model.

Goals of the Project

The main goals of Chris's project were to build a real replica of a conveyor belt that could intelligently select and sort objects Based on computer vision and AI algorithms. The project aimed to remove unwanted objects from the belt and categorize the remaining objects into specific classes. Additionally, the project aimed to count objects within each class and estimate the volume of objects using the spatial capabilities of the cameras.

Tools and Technologies Used

To realize his project, Chris utilized various tools and technologies. He used the Mindstorms Robot Inventor Kit from Lego to build the physical prototype of the conveyor belt. The prototype included Lego motors, lights, and cameras. For image annotation and augmentation, Chris used the Roboflow platform. He trained the AI model using the Azure Custom Vision service and the MobileNet V2 network. Additionally, he employed Python and the Mindstorms Python library for controlling and running the conveyor belt.

Building the Prototype

Chris detailed the process of building the physical prototype of the conveyor belt using Lego bricks and components. The prototype went through various stages, including the construction of the conveyor belt, the addition of pushers and assorters, and the integration of cameras for object detection and classification. The final model featured an impressive replica of a conveyor belt capable of intelligent object selection and sorting.

Training the AI model

The training of the AI model involved capturing approximately 900 images of Lego bricks from various angles and classes. These images were annotated and augmented using Roboflow, enhancing the dataset for improved model performance. Chris experimented with different AI models, including YOLO, but found that MobileNet V2 performed slightly better for his project. He utilized the Azure Custom Vision service for training and exporting the models, which were then converted to depth AI platform blobs for deployment.

Hardware Issues and Solutions

While developing the project, Chris encountered several hardware-related challenges. One of the main issues was the vibration caused by the movement of the conveyor belt and motors. The vibrations affected the focus of the cameras and impacted object detection. To mitigate this issue, Chris made adjustments to the structure of the conveyor belt and added additional support through diagonal bars. He also strategically placed lead blocks on the motors to further reduce vibrations.

Lessons Learned

Chris shared various lessons learned throughout his journey in the OpenCV Spatial AI Competition. He emphasized the importance of not assuming that certain steps will be easy, as unforeseen challenges can arise. He also emphasized the need to plan the time for each phase of the project and stick to the schedule. Additionally, Chris highlighted the importance of not working alone and the benefits of having additional team members to share the workload and expertise.

Conclusion

In conclusion, Chris Nogich's project of building a smart conveyor belt using computer vision and AI algorithms highlights the potential applications of these technologies in industries such as mining. By utilizing Lego bricks instead of real rocks, Chris overcame challenges in dataset creation and ultimately demonstrated a successful prototype. His journey serves as inspiration for aspiring participants in the field of computer vision and AI.