This plugin is a comprehensive Unreal Engine wrapper for the OpenAI API. It supports all OpenAI endpoints, including:
- Models
- Completions
- Chat
- Images
- Embeddings
- Audio
- Files
- Moderations
- Fine-tuning
- Functions
Edits[deprecated]Fine-tunes[deprecated]
All requests are available in both C++ and Blueprints:
void AAPIOverview::CreateImage()
{
auto* Provider = NewObject<UOpenAIProvider>();
Provider->SetLogEnabled(false);
Provider->OnRequestError().AddUObject(this, &ThisClass::OnRequestError);
Provider->OnCreateImageCompleted().AddLambda(
[](const FImageResponse& Response)
{
FString OutputString{};
Algo::ForEach(Response.Data, [&](const FString& Data) { OutputString.Append(Data).Append(LINE_TERMINATOR); });
UE_LOG(LogAPIOverview, Display, TEXT("%s"), *OutputString);
});
FOpenAIImage Image;
Image.Prompt = "Tiger is eating pizza";
Image.Size = UOpenAIFuncLib::OpenAIImageSizeToString(EImageSize::Size_512x512);
Image.Response_Format = UOpenAIFuncLib::OpenAIImageFormatToString(EOpenAIImageFormat::URL);
Image.N = 2;
Provider->CreateImage(Image, Auth);
}
- Unreal Engine 5.3 (master), 5.2
- Plugin overview and usage example [english subtitles].
- How to use functions.
- Create a new C++ project in Unreal Engine.
- Create a
Plugins
folder at the root of your project. - Add the OpenAI plugin to your Unreal Engine
Plugins
folder. - The easiest way to add plugins is by adding the current plugin as a submodule. Use the following command in the root folder of your project:
git submodule add https://github.com/life-exe/UnrealOpenAIPlugin Plugins/OpenAI
- Alternatively, you can download the source code from the current repository and copy the files to your
Plugins
folder. - When complete, your Unreal Engine project structure should resemble the following:
- Do the authentication steps
- Generate Visual Studio project files.
- Add
OpenAI
plugin to yourYourProject.Build.cs
:
public class YourProject : ModuleRules
{
public YourProject(ReadOnlyTargetRules Target) : base(Target)
{
PCHUsage = PCHUsageMode.UseExplicitOrSharedPCHs;
PublicDependencyModuleNames.AddRange(new string[] { "Core", "CoreUObject", "Engine", "InputCore", "OpenAI" });
PublicIncludePaths.AddRange(new string[] { "YourProject" });
}
}
- Build your project and run Editor.
- In the Editor, navigate to
Edit->Plugins
. Find and activate theOpenAI
plugin. - Restart the editor.
- Make sure that your
.uproject
file contains plugin:
{
"FileVersion": 3,
"EngineAssociation": "5.x",
"Category": "",
"Description": "",
"Modules": [
{
"Name": "ProjectName",
"Type": "Runtime",
"LoadingPhase": "Default",
"AdditionalDependencies": [
"Engine"
]
}
],
"Plugins": [
{
"Name": "OpenAI",
"Enabled": true,
"MarketplaceURL": "com.epicgames.launcher://ue/marketplace/product/97eaf1e101ab4f29b5acbf7dacbd4d16"
}
]
}
- Start using the plugin.
- Create a blueprint project.
- Create a
Plugins
folder in the root of your project. - Download precompiled plugin from the Releases.
- Unzip plugin to the
Plugins
folder, specifically toPlugins/OpenAI
. - Do the authentication steps
- Run the Editor
<YourProjectName>.uproject
- In the Editor, navigate to
Edit->Plugins
. Find and activate theOpenAI
plugin. - Restart the editor.
- Make sure that your
.uproject
file contains plugin:
{
"FileVersion": 3,
"EngineAssociation": "5.x",
"Category": "",
"Description": "",
"Modules": [
{
"Name": "ProjectName",
"Type": "Runtime",
"LoadingPhase": "Default",
"AdditionalDependencies": [
"Engine"
]
}
],
"Plugins": [
{
"Name": "OpenAI",
"Enabled": true,
"MarketplaceURL": "com.epicgames.launcher://ue/marketplace/product/97eaf1e101ab4f29b5acbf7dacbd4d16"
}
]
}
- Start using the plugin.
-
Create an OpenAI account
-
Generate and store your API Key:
sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
-
Record your Organization ID:
org-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
-
At the root of your Unreal Engine project, create a file named
OpenAIAuth.ini
(you can usesetup_auth_files.bat
in the plugin root folder. It will copy.ini
files from template folder) with the following content:
APIKey=sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
OrganizationID=org-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
- Once completed, your Unreal Engine project structure might look like this:
-
Actually you can left
OrganizationID
empty. It doesn't affect on auth. -
Do the authentication steps for services if you need them.
Finally, compile your project and launch the Unreal Editor.
This provides a basic example of request usage with Blueprints.
- Make sure that plugin content activated:
- Open the plugin content folder.
- Open the level:
- Run the game.
- Test the API with the widget examples:
Feel free to modify requests in the widget code to test the API with different parameters:
This is the Chat-GPT implementation with token streaming support.
- Open the plugin content folder.
- Navigate to the
EditorWidgets
folder. - Right-click on the ChatGPT editor utility widget:
- Start chatting:
- You can save the chat history to a file using the
Dump
button. The specific location where the history is saved can be checked in the chat or logs:
You can build your own services (addons, plugins) on top of ChatGPT:
Currently the plugin has two services available by default:
- Weather
- News
I use free API for services. But you need to create accounts to get API keys.
- Weather serivces: https://weatherstack.com. Once your account is created, you will find
API Access Key
on your dashboard. - News service: https://newsapi.org. Once your account is created, you will find
API Key
on your dashboard. - In the root of your Unreal Engine project, create a file called
OnlineServicesAuth.ini
(you can usesetup_auth_files.bat
in the plugin root folder. It will copy.ini
files from template folder) with the following content:
WeatherstackAccessKey=dbe5fcdd54f2196d2cdc5194cf5
NewsApiOrgApiKey=1dec1793ed3940e880cc93b4087fcf96
- Enter your APi keys for each service.
- Compile your project, launch the Unreal Editor, launch ChatGPT editor widget, select the services that you want to use.
- Ask for the weather or latest news somewhere. Example prompts:
What is the weather like in Oslo?
What is the latest news from Microsoft? (2 headlines)
- Always check the logs when you encounter an error.
This blueprint demonstrates all available nodes.
- Open the plugin content folder:
- Open
BP_APIOverview
blueprint. - Check the available nodes: functions and structs:
There are also several nodes that could be useful in other projects. Feel free to copy and paste the plugin code if you need them:
- Open
Plugins\OpenAI\Source\OpenAI\Private\Sample\APIOverview.cpp
actor - Uncomment the function that executes the request you want to test. Navigate to the function and adjust all the request parameters as needed.
- Compile your project and open Unreal Editor.
- Drop the
AAPIOverview
actor into any level. - Run the game.
- Check the Output Log.
OpenAIModule
- core classes.OpenAIEditorModule
- Chat-GPT editor utility widget.OpenAITestRunnerModule
- unit tests.
I highly recommend reading the OpenAI documentation for a better understanding of how all requests work:
In addition plugin code has its own documentation generated with help of Doxygen:
You can generate the plugin documentation locally by following these steps:
- Update all submodules. You can use the batch script located at the root of the plugin folder:
update_submodules.bat
- Ensure that Doxygen is installed on your system.
- Ensure that Python is installed on your system.
- Generate the documentation using the batch script at the root of the plugin folder:
generate_docs.bat
- After the generation process, the documentation will be available in the
Documentation
folder at the root of the plugin:Documentation\html\index.html
Unit tests are available in the OpenAITestRunnerModule
. You can initiate them using the Session Frontend
:
There are also API tests that request a real endpoints. These have stress filters. They are mostly for my personal use to check API changes. But you can use test spec file to learn how to make requests, because testing in software development is a part of documentation:
- Verify that your .uproject file contains the following:
{
"Name": "OpenAI",
"Enabled": true
}
- Package the project as you always do (kudos if you use a CI solution like Jenkins).
- You can handle authentication in your project as you see fit. However, if you choose to use a file-based method, such as in a plugin, please remember to include the
OpenAIAuth.ini
andOnlineServicesAuth.ini
files in your packaged folder:
Build/Windows/<YourProjectName>/OpenAIAuth.ini
Build/Windows/<YourProjectName>/OnlineServicesAuth.ini
If you are having problems loading the file, please check the logs to see where it might be located:
LogOpenAIFuncLib: Error: Failed loading file: C:/_Projects/UE5/OpenAICpp/Build/Windows/OpenAICpp/OpenAIAuth.ini
Chat GPT-4 models are not available for everyone via the API. You need to request access for it.Chat GPT-4 models are now available for everyone.- OpenAI hosts a variety of different models. Please check the models that are compatible with the particular request.
I would appreciate bug reports and pull-request fixes.
Enjoy! ๐๏ธ