Maximize your AI resources with Vast.ai
Table of Contents
- Introduction
- Setting Up Your AI Rig
- Port Forwarding
- Machine Setup
- Starting the Installation
- Accessing the BIOS
- Boot Sequence Configuration
- Memory Map IO Setup
- Performance Configuration
- Advanced Configuration
- Installing Ubuntu
- Choosing the Installation Type
- Disk Partitioning
- Setting Up File Systems
- Installation Process
- Configuring Ubuntu
- Updating Packages
- Installing OpenSSH
- Installing NVIDIA Drivers
- Finalizing the Setup
- Disabling Desktop Version
- Rebooting the Machine
- Connecting to Vast AI Platform
- Registering an Account
- Running the Vast AI Command
- Using Vast AI
- Monitoring Your Machines
- Listing Your Prices
- Populating Machine Information
Setting Up Your AI Rig
In order to get started with the Vast AI and the IA marketplace, You will need to set up your AI rig. This guide is aimed at intermediate to advanced users who are familiar with Linux. If you are new to Linux or have difficulty with basic networking concepts, it is recommended to Seek assistance or consider other options.
Port Forwarding
The first step is to port forward ports 40,000 to 40,019. This is the range of ports recommended by Vast AI for optimal performance. Port forwarding allows external access to your machine and is necessary for your rig to communicate with the Vast AI platform.
Machine Setup
Once you have completed the port forwarding, you will need to set up your AI rig. It is important to place the machine in a location where it will remain permanently, as reliability is crucial. Treat your machine like a data center and ensure it remains up and running 24/7. Unreliable machines may result in penalties, regardless of whether they are currently assigned work.
Accessing the BIOS
To start the installation process, you need to access the BIOS of your machine. Connect a keyboard, mouse, and monitor to your rig, and turn it on. The ECC memory and other components on the boot cycle may cause a slight delay in the boot-up process. Once the machine starts, press F2 on your keyboard to enter the BIOS.
Boot Sequence Configuration
In the BIOS, navigate to the boot sequence configuration. Remove any unnecessary boot options and ensure that only the USB containing the Ubuntu 20.04 desktop version is selected as the boot device. This will simplify the installation process and avoid potential errors.
Memory Map IO Setup
Next, locate the memory map IO option in the BIOS. Enable the option to ensure your system has access to memory above 4G, which is essential for running AI workloads efficiently.
Performance Configuration
In the performance configuration section, enable Hyper-threading if it is disabled by default. Hyper-threading allows your CPU to utilize multiple Threads, significantly improving performance. Additionally, disable Intel Turbo Boost to maintain control over the CPU and GPU performance.
Advanced Configuration
In the advanced configuration section, set the PCIe link to Gen 3. This setting optimizes the connection between your machine and external devices, ensuring fast and stable data transfer.
Installing Ubuntu
Now that the BIOS settings are configured, it's time to install the Ubuntu operating system. The desktop version is recommended for a smoother installation process. Although it is possible to make the system SSH-only later, starting with the desktop version simplifies initial setup.
Choosing the Installation Type
During the Ubuntu installation, choose the minimal installation option without downloading updates or installing third-party software. This will ensure a clean installation without unnecessary packages.
Disk Partitioning
In the disk partitioning step, select the "Something else" option. This allows manual configuration for machines with multiple SSDs. Adjust the partition sizes accordingly, allocating enough space for the operating system, swap area, EFI, and file system.
Setting Up File Systems
Set up the file system for the main partition as XFS with journaling enabled. This ensures efficient and reliable storage for your AI rig. Additionally, set up a small EFI partition for system booting and a swap area to improve system performance.
Installation Process
Confirm your settings and proceed with the installation. Wait for the process to complete, which may take a few minutes depending on your system specifications.
Configuring Ubuntu
After the installation, you will need to configure some settings in Ubuntu. Open a terminal window or use the shortcut Ctrl+Alt+T. Update the packages using the command sudo apt update. Once the update is complete, proceed with the installation of OpenSSH by running the command sudo apt install openssh-server.
Installing NVIDIA Drivers
To utilize the full potential of your AI rig, it is important to install the appropriate NVIDIA drivers. Run the command sudo apt install nvidia-driver-525 to install the recommended driver version. This process may take a few minutes as the driver Package is downloaded and installed.
Finalizing the Setup
To optimize your system, run the command sudo service gdm3 restart. This disables the desktop version and ensures your resources are dedicated solely to AI workloads. Your system will now operate as an Ubuntu server.
Connecting to Vast AI Platform
To connect your AI rig to the Vast AI platform, you need to register an account on their Website. Once you have an account, log in and navigate to the hosting section. Read the documentation provided and follow the instructions to obtain the Vast AI command.
Running the Vast AI Command
As root, run the Vast AI command provided in the documentation to link your rig with the platform. This command will establish the connection and enable the monitoring and management of your machine through the Vast AI platform.
Using Vast AI
Once your rig is connected and registered on the Vast AI platform, you can start monitoring its performance and assigning prices for usage. The platform provides various tools and features for efficient management of your AI resources.
Monitoring Your Machines
Use the Vast AI platform to monitor the status, workload, and performance of your AI machines. You can view machine details, resource utilization, and other Relevant information through the platform's user-friendly interface.
Listing Your Prices
In the pricing section of the Vast AI platform, you can define the prices for utilizing your AI rig. Set your desired rates Based on factors such as demand, hour of usage, and internet bandwidth. This will ensure you can earn revenue from the utilization of your AI resources.
Populating Machine Information
Allow some time for the Vast AI platform to populate your machine's information. This process might take a while, but once complete, you will have full visibility and control over your AI rig through the Vast AI platform.
Conclusion
Setting up your own AI rig and connecting it to the Vast AI platform opens up a world of possibilities for AI workloads and monetization. By following the steps outlined in this guide, you can effectively configure your rig, install the necessary software, and start utilizing the Vast AI platform to maximize the potential of your AI resources.
