From Self-Study to Deepfake and Game AI: My 9-Month AI Journey

Table of Contents

1. Introduction
2. Background and Learning Journey
   1. Starting Point: Self-Studying Machine Learning
   2. Initial Project: Predicting Instagram Likes
   3. Exploring Open Source Code on GitHub
   4. Deepfake Experiments with Face Swapping
   5. Music Generation and Voice Cloning
3. The Billy Eilish Project
   1. Challenges with Lip Sync and Pose Detection
   2. Deploying the First Order Motion Model
   3. Successes and Lessons Learned
4. Exploring Recommendation Engines
   1. Failed Project: New Hobbies Recommendation Engine
   2. Discovering Online Dance Classes
   3. Fusion of AI and Dance: AI Choreography
5. Deployment and Web Development
   1. Mastering Model Deployment
   2. The Dance God Project: Deepfake Dance Video Generator
   3. Overcoming Challenges for Deployment
6. Continuing Education with Andrew Ng's Courses
   1. The Value of Andrew Ng's Courses
   2. Project-Based Learning vs. Theoretical Understanding
7. Reinforcement Learning and Game AI
   1. Transition to Game Development
   2. Studying Reinforcement Learning Techniques
   3. Entering the Proc Gen Competition by OpenAI
8. Reflection and Future Plans
   1. Building a Broad Understanding through Project Work
   2. Implementation vs. Idea Generation
   3. Future Plans and ML Engineer Interviews
   4. The Power of Sponsorship and Tech Domains
   5. Blogging and Sharing Resources on Willkwon.tech

My AI Journey: From Self-Study to Deepfake, Music Generation, and Game AI

In today's rapidly advancing technological landscape, artificial intelligence (AI) has become increasingly prevalent across various industries. As an aspiring machine learning enthusiast, I embarked on a nine-month journey of self-study, exploring different AI applications and projects. In this article, I will take you through my learning journey and highlight the projects and lessons learned along the way.

Introduction

AI has always fascinated me, but it wasn't until last December that I made a conscious decision to delve deeper into the field. Armed with a background in programming, thanks to my years of experience, I yearned for a return to the more creative aspects of the craft. My goal was to build exciting projects that brought me joy, rather than solely focusing on practical programming tasks or problem-solving.

Background and Learning Journey

Starting Point: Self-Studying Machine Learning

Before embarking on my machine learning journey, I already had some basic knowledge thanks to university courses. However, I had never truly pursued it independently. To kickstart my foray into the world of AI, I decided to start with a simple regression model project. I aimed to predict the number of likes an Instagram photo would receive, with the possibility of extending this concept to YouTube videos. While aware that the accuracy might not be exceptional, it nevertheless presented an exciting challenge.

Initial Project: Predicting Instagram Likes

To lay the groundwork for my Instagram likes prediction model, I began by researching existing work. I discovered numerous complicated machine learning projects open-sourced on GitHub, often with minimal documentation. It struck me as a missed opportunity to see such technically intricate work go unused. This realization sparked the idea of copying and modifying code from these projects to create my own video-based projects. While this approach may not have been the most conducive to my personal learning, it played a crucial role in honing my skills in reading and modifying academic code.

Exploring Open Source Code on GitHub

During my exploration of GitHub repositories, I stumbled upon a pre-trained model specifically designed for predicting Instagram likes. Intrigued by the prospects, I decided to enhance my project's appeal by incorporating the generation of photorealistic faces. With enthusiasm and determination, I completed this project in just a few weeks. It soon dawned on me that there was a world of possibilities within the realm of deepfakes.

Deepfake Experiments with Face Swapping

Deepfakes, the art of swapping faces in videos, had gained significant attention at this time. Keen to experiment with this technology, I discovered "DeepFaceLab" and created my first deepfake video. Although the result wasn't exceptional, it served as a stark reminder of the effort required to produce high-quality deepfake videos. Frame-by-frame masking and meticulous settings adjustments were crucial, dashing any illusions of Instant success.

Music Generation and Voice Cloning

Feeling dissatisfied with my initial attempts at generating music using AI, I decided to explore facial synthesis while simultaneously adding voice cloning capabilities. This marked the beginning of the Billy Eilish project. Although it took me two months to plan and complete the video, this endeavor presented numerous challenges. One major obstacle was syncing the mouth movements to match the cloned voice. It involved extensive experimentation with various projects until I stumbled upon the First Order Motion Model project, which revolutionized my progress. This project proved to be a valuable resource for many meme pages and introduced me to the concept of deepfake dancing.

📸 The Billy Eilish Project