Revolutionizing Conveyor Belts: A Journey from Front-end Developer to Computer Vision Expert
Table of Contents:
- Introduction
- Chris Nogich: A Journey from Front-end Developer to Computer Vision Expert
- The Project Concept: Smart Conveyor Belt for Sorting Rocks
- Challenges in Data Collection and Training
a. Skewed Data and Imbalanced Classes
b. Limited Number of Training Samples
c. Switching from Rocks to Lego Bricks
- Tools and Technologies Used
a. Lego Mindstorms Robot Inventor Kit
b. Luxonis Oak-D Cameras
c. Azure Custom Vision Service
d. Roboflow for Image Annotation and Augmentation
e. Python and Mindstorms Python Library for Application Development
- Developing the Prototype
a. Building the Conveyor Belt, Pusher, and Assorter Spoon
b. Positioning and Securing the Cameras
- Training the AI Models
a. Capturing and Annotating Lego Block Images
b. Augmenting the Data
c. Using Custom Vision and MobileNet for Transfer Learning
- Overcoming Challenges and Lessons Learned
a. Dealing with Faulty Cameras and Hardware Issues
b. Managing Vibrations and Belt Speed
c. Solving Unwanted Detections and Labeling Challenges
- The Final Model and Presentation
a. Demonstrating the Functionality of the Conveyor Belt
b. Calculating Object Quantities and Classifying Blocks
- Conclusion
a. Key Takeaways and Achievements
b. Recommendations for Future Improvements
Introduction:
In this article, we will Delve into the exciting journey of Chris Nogich, who went from being a front-end web developer to a computer vision expert. We will explore his project that made it to the top 10 in the OpenCV Spatial AI competition of 2022, where he created a smart conveyor belt using computer vision and AI algorithms. Throughout this article, we will discuss the challenges faced during data collection and training, the tools and technologies used, the development of the prototype, the training of AI models, and the final presentation. Join us as we unravel the process behind this innovative project and gain insights into Chris's experiences and lessons learned along the way.
Chris Nogich: A Journey from Front-end Developer to Computer Vision Expert
Before we delve into the project itself, let's take a moment to appreciate the journey that led Chris Nogich from being a front-end web developer to becoming a proficient computer vision expert. Starting his career in databases and front-end development, Chris eventually found himself drawn to the realm of machine learning and advanced data analytics. With a background in economics and a keen interest in statistics, he made a seamless transition into the world of AI and computer vision.
The Project Concept: Smart Conveyor Belt for Sorting Rocks
The project that Chris Nogich undertook for the OpenCV Spatial AI competition was inspired by the geological mining industry. The concept revolved around creating a smart conveyor belt that could intelligently select and sort various minerals from the mining output. The goal was to categorize different types of rocks Based on their properties, discard unwanted rubble, and ensure accurate classification. However, Chris encountered several challenges along the way, including skewed data, limited training samples, and the need to switch from rocks to Lego bricks for better experimentation.
Challenges in Data Collection and Training
One of the primary hurdles Chris faced was the skewed distribution of data. To train the model properly, he had to ensure an equal representation of each Type of rock. However, the number of samples for each class varied, with pebbles being the most abundant and quartz rocks being the most scarce. Moreover, the overall number of samples was relatively small, demanding a strategic approach to maximize the learning potential.
Another challenge arose from the sheer size of the rocks selected for the initial models. Given their weight and Dimensions, they proved difficult to process on the conveyor belt and caused technical limitations. As a result, Chris switched to Lego blocks, which offered greater flexibility and ease of experimentation.
Tools and Technologies Used
To bring his project to life, Chris utilized various tools and technologies. The Lego Mindstorms Robot Inventor Kit served as the basis for the conveyor belt model, enabling motorized control and customizable configurations. Luxonis Oak-D cameras were incorporated for object detection and classification, providing spatial capabilities to enhance the accuracy of the sorting process. Azure Custom Vision service was used for image annotation, training, and deployment of AI models, offering a user-friendly interface and seamless integration with other components. Additionally, Roboflow aided in data annotation and augmentation, streamlining the preparation of the training dataset. The application development was powered by Python and the Mindstorms Python library, offering extensive functionalities and flexibility for programming the Lego hub and controlling the conveyor belt.
Developing the Prototype
The development of the prototype involved building the physical components of the conveyor belt, including the motorized belt itself, pusher mechanisms for removing unwanted objects, and the assorter spoon for sorting objects into their respective classes. Careful calibration and positioning of the Luxonis cameras were crucial for accurate object detection and spatial measurement. The process required creativity and technical expertise, culminating in a fully operational prototype that emulated a real-world conveyor belt system.
Training the AI Models
Training the AI models was a critical part of the project. Chris captured approximately 900 images of Lego blocks across different classes, ensuring diverse angles, lighting conditions, and backgrounds. These images were annotated and augmented using Roboflow, enabling robust training of the models. Custom Vision service from Azure facilitated the training process, leveraging transfer learning with MobileNetV2 for efficient and reliable classification. The models were optimized to balance accuracy and performance, enabling real-time object detection and classification on the conveyor belt.
Overcoming Challenges and Lessons Learned
Throughout the project, Chris encountered several challenges and learned valuable lessons. Limited Python library support for Lego Mindstorms hindered real-time monitoring and control of the conveyor belt's movements. As a result, he had to rely on sequential programming and pre-determined delay times. Hardware issues, including faulty cameras and vibrations caused by the motorized components, required innovative solutions. Chris addressed the vibration issue by redesigning the leg structure of the prototype and introducing lead blocks to dampen the vibrations. Additionally, unwanted detections were mitigated by placing blank paper in the field of view to avoid self-detection.
The Final Model and Presentation
The culmination of Chris's hard work and innovation resulted in a fully functional model of the smart conveyor belt. During the final presentation for the OpenCV Spatial AI competition, he showcased the prototype's capabilities, demonstrating the accurate sorting and counting of Lego blocks. The intelligent operation of the conveyor belt, combined with spatial measurements, allowed for efficient object removal and classification. Overall, the model proved to be a success, meeting the project's goals and showcasing Chris's dedication and technical prowess.
Conclusion
Chris Nogich's journey from a front-end web developer to a skilled computer vision expert showcases the transformative potential of AI and machine learning. His project, focused on a smart conveyor belt for sorting rocks using computer vision and AI algorithms, highlights the practical application of these technologies in real-world scenarios. Despite the challenges faced along the way, Chris's innovation, problem-solving skills, and adaptability enabled him to overcome obstacles and Create a fully functional prototype. From data collection and training to the development of the physical model, every step of the process demonstrated his passion and commitment to the project's success. Ultimately, this project serves as an inspiration for aspiring computer vision enthusiasts and showcases the power of imagination, determination, and creativity when combined with state-of-the-art technologies.
FAQ:
Q: What inspired Chris Nogich to create a smart conveyor belt for sorting rocks?
A: Chris's inspiration came from the geological mining industry, where the task of selecting and categorizing rocks often proves to be a labor-intensive and inefficient process. He aimed to leverage computer vision and AI algorithms to automate and streamline this operation, enhancing productivity and accuracy in the mining industry.
Q: What were some of the challenges Chris faced during data collection and training?
A: Chris encountered challenges such as skewed data distribution, limited training samples, and the need to switch from rocks to Lego bricks for better experimentation. Skewed data and imbalanced classes made it challenging to train accurate models, while the limited number of samples for certain classes posed a data scarcity issue. Additionally, training models on large rocks proved to be technically challenging, leading to the decision to switch to Lego bricks.
Q: Which tools and technologies did Chris utilize for his project?
A: Chris used the Lego Mindstorms Robot Inventor Kit for building the physical components of the conveyor belt, Luxonis Oak-D cameras for object detection, Azure Custom Vision service for image annotation and AI model training, Roboflow for data augmentation, and Python with the Mindstorms Python library for application development.
Q: How did Chris overcome the challenge of vibrations in the model?
A: To address the vibrations caused by the motorized components, Chris redesigned the leg structure of the prototype and introduced lead blocks to dampen the vibrations. These modifications helped enhance the accuracy of object detection and improved the overall performance of the conveyor belt system.
Q: What lessons did Chris learn throughout the project?
A: Chris learned the importance of not assuming tasks to be easy and suggested planning time for each phase of the project. He also highlighted the value of working collaboratively rather than tackling a project alone. Additionally, he emphasized the significance of adapting to challenges, making innovative engineering modifications, and finding simple yet effective solutions to overcome technical limitations.
Q: How long did it take for Chris to train his AI model on the Microsoft Azure Custom Vision service?
A: Chris trained his AI model, specifically using MobileNetV2 for transfer learning, in less than one hour using the Microsoft Azure Custom Vision service. The service provided a user-friendly interface and seamless integration for accelerating the training process.
Q: What are some key takeaways from Chris Nogich's project?
A: Chris's project demonstrates the transformative potential of AI and computer vision in various industries, including mining. It showcases how innovative solutions can optimize processes, enhance efficiency, and lead to tangible results. Furthermore, his journey serves as an inspiration for those looking to explore the practical applications and possibilities within the field of computer vision.
