Unlock the Power of Conversational AI in Unreal Engine

Table of Contents

1. Introduction
2. Step 1: Downloading the Convey Plugin
3. Step 2: Installing the Convey Plugin
4. Step 3: Obtaining the API Key
5. Step 4: Setting Up the Project in Unreal Engine
6. Step 5: Adding a Simple Character
7. Step 6: Modifying the Character's Backstory and Personality
8. Step 7: Adding a Metahuman Character
9. Step 8: Adding Animations to the Metahuman
10. Step 9: Adding Support for Third-Person Game Mode
11. Step 10: Implementing Actions for the Character
12. Step 11: Demo Project and Ready Player Me Characters
13. Conclusion

Adding Conversational AI to Unreal Engine

In this guide, we will explore how to integrate Conversational AI into Unreal Engine. Conversational AI can bring life to virtual characters and enhance the interactive experience in your virtual environment. By following these step-by-step instructions, you will be able to add Conversational AI to your Unreal Engine project and Create engaging conversations with virtual characters.

Step 1: Downloading the Convey Plugin

To start off, we need to download the Convey plugin from the Unreal Marketplace. You can easily find the plugin by searching for "Convey" in the marketplace. Once you have found it, proceed to install it into your Unreal Engine project. Please note that this guide assumes you are using Unreal Engine 5.1, but instructions for installing the plugin in Unreal Engine 4 can be found in our documentation.

Step 2: Installing the Convey Plugin

After downloading the Convey plugin, ensure that it is properly installed in Unreal Engine. To do this, go to the "Edit" menu, search for "Convey" under plugins, and enable the plugin by clicking the checkbox. Restart your project to Apply the changes.

Step 3: Obtaining the API Key

To utilize the full capabilities of the Convey plugin, you will need an API key. Head to convey.com and sign in to your account. Once logged in, navigate to the playground page and click on the key icon in the upper right corner to obtain your API key.

Step 4: Setting Up the Project in Unreal Engine

In Unreal Engine, go to the "Project Settings" under the "Edit" menu. Scroll down to the "Plugins" section and find Convey. Enter your API key into the provided input field. Additionally, you will need to edit the default engine.ini file in your project's directory to enable voice. Open the file with a text editor and add the lines "voice" and "be enabled equals true". Save the file and restart Unreal Engine.

Step 5: Adding a Simple Character

Let's start by adding a simple character that we can have a conversation with. In the content browser, create a new blueprint class and select the "Convey Base Character" as the parent class. Give the blueprint a name and drag it into the environment. Open the character's side panel and fill in the required character ID field. This ID can be obtained from the Convey Website in the dashboard page. It is a unique identifier for your character.

Step 6: Modifying the Character's Backstory and Personality

The character's backstory defines their personality and influences their responses in conversations. On the Convey website, in the character description section, you can modify the character's name, select different voices, and change the backstory. Explore the different options to see how the character's responses vary Based on their personality.

Step 7: Adding a Metahuman Character

Instead of having an empty character, let's add a Metahuman character that we can Interact with. Open the Quixel Bridge from the Window menu and select a default Metahuman character, such as "Cooper". After adding the Metahuman, make some edits to the blueprint by changing the parent class to "Convey Base Character". Additionally, add animations for the body and face components to bring the character to life.

Step 8: Adding Animations to the Metahuman

To animate the Metahuman character, open the body and face components in the blueprint and assign the Convey Meta-Human Body Animation and Convey Meta-Human Face Animation, respectively, as the animation classes. This will enable the character to display lifelike movements and facial expressions during conversations.

Step 9: Adding Support for Third-Person Game Mode

If desired, you can add support for the third-person game mode. Search for "Third Person" in the "Add Feature or Content Pack" section under the "Window" menu and add the third-person pack to your project. Modify the class settings of the player blueprint to use the "Convey Bass Player" as the parent class. This will enable smooth interaction between the character and the player in third-person mode.

Step 10: Implementing Actions for the Character

In addition to conversations, characters can perform actions such as moving or following the player. To enable this feature, search for "Navigation Nav Mesh Bounds Volume" in the "Place Actors" section under the "Window" menu. Place the bounds volume and Scale it to cover the desired area. This will allow the character to navigate the environment. Test the character's actions by interacting with them through conversation.

Step 11: Demo Project and Ready Player Me Characters

For a more advanced implementation, consider exploring our demo project that showcases lip sync and Ready Player Me characters. This project, available on our documentation pages, includes a variety of characters with predefined IDs. Simply open the project, set your API key in the plugin settings, and start interacting with the characters using conversations.

Conclusion

Congratulations! You have successfully integrated Conversational AI into your Unreal Engine project. By following the steps outlined in this guide, you can create immersive and engaging conversations with virtual characters. Explore the different settings and features to customize the behavior of your characters. Stay tuned for more deep dive tutorials and further enhancements to your Conversational AI experience.