Git Product home page Git Product logo

chatgpt_api_dotnet's Introduction

example gif...

ChatGPT integration for .NET (+DI)

Nuget.NET
OpenAI Chat Completions API (ChatGPT) integration with DI and EF Core support. It allows you to use the API in your .NET applications. Also, the client supports streaming responses (like ChatGPT) via async streams.

Changelog

2024.8

  • Add OpenRouter and Azure OpenAI support
  • Add support for GPT-4-o, GPT-4-o-mini
  • Improve JSON mode support
  • BREAKING CHANGES: AddChatGptEntityFrameworkIntegration now requires builder.Configuration as a mandatory parameter.
  • Add a setup example

2023.11

  • Add GPT-4-Turbo
  • Add JSON mode support
  • StructuredResponse module allows you to get structured responses from the API as C# object. See: StructuredResponse section.

Content

Preparation

First, you need to create an OpenAI account and get an API key (or OpenRouter or Azure OpenAI). You can do this at https://platform.openai.com/account/api-keys.

Installation

The easiest way to use ChatGPT service in your .NET project with DI and persistence (EF Core) supporting is to install the NuGet package OpenAI.ChatGPT.EntityFrameworkCore:

Install-Package OpenAI.ChatGPT.EntityFrameworkCore

If you don't want to use EF Core, you can install the package OpenAI.ChatGPT.AspNetCore and implement your own storage for chat history, using IChatHistoryStorage interface.

Usage

TL;DR: See an example in a reference project.

  1. Set the OpenAI API key or even host (optional) in your project user secrets, or the appsettings.json file (not safe):
{
  "AIProvider": "openai", // or openrouter or azure_openai
  "OpenAICredentials": { //optional
    "ApiKey": "your-api-key-from-openai",
    "ApiHost": "https://api.openai.com/v1/"
  },
  "AzureOpenAICredentials": { //optional
    "ApiKey": "your-api-key-from-azure-openai",
    "ApiHost": "https://{your-host}.openai.azure.com/",
    "DeploymentName": "gpt-4-turbo-preview"
  },
  "OpenRouterCredentials": { //optional
    "ApiKey": "your-api-key-from-openrouter",
    "ApiHost": "https://openrouter.ai/api/v1"
  }
}

Also, you can specify OpenAI API key as environment variable ASPNETCORE_OpenAICredentials:ApiKey.

  1. Add ChatGPT integration with EF to your DI container:
builder.Services.AddChatGptEntityFrameworkIntegration(
    builder.Configuration,
    options => options.UseSqlite("Data Source=chats.db"));

Instead of options.UseSqlite("Data Source=chats.db") use your own db and connection string.

  1. Inject ChatGPTFactory to your service and use it to create ChatGPT instance:
public class YourService
{
    private readonly ChatGPTFactory _chatGptFactory;

    public YourService(ChatGPTFactory chatGptFactory)
    {
        _chatGptFactory = chatGptFactory;
    }

    public async Task<string> GetAnswer(string text)
    {
        ChatGPT chatGpt = await _chatGptFactory.Create(userId);
        var chatService = await chatGpt.ContinueOrStartNewTopic();
        response = await _chatService.GetNextMessageResponse(_prompt);
        return response;
    }
}

See Blazor Example.

If you want to configure request parameters, you can do it in appsettings.json configuration or in ChatGPTFactory.Create or in ChatGPT.CreateChat methods.

{
  "ChatGPTConfig": {
      "InitialSystemMessage": "You are a helpful and kind assistant.",
      "InitialUserMessage": null,
      "MaxTokens": null,
      "Model": null,
      "Temperature": null,
      "PassUserIdToOpenAiRequests": true
  }
}

See parameters description inside ChatGPTConfig.

Exceptions

If the server response is not a success status code, the client will throw a NotExpectedResponseException. The exception will contain the error message from the OpenAI API.
By default, requesting cancellation or ChatService.Stop() method calling will throw OperationCanceledException. If you don't want to throw it (relevant for streaming responses), you can set throwOnCancellation parameter to false:

await foreach (string chunk in chatService.StreamNextMessageResponse(text, throwOnCancellation: false))
{
    //...
}

Thread safety and async

ChatGPTFactory, ChatGPT classes thread-safety is depend on the IChatHistoryStorage implementation. If you use ChatGPTFactory with entity framework, it's NOT thread-safe. ChatService class is not thread-safe.
Anyways, these services are designed to be used safely with DI, so you don't need to worry about it.
All the methods from all the packages are designed to be used in async context and use ConfigureAwait(false) (thanks for the ConfigureAwait.Fody package).

Retries, timeouts and other policies

Since ChatGPTFactory depends on IHttpClientFactory, you can easily use any of the available policies for it, like Polly.

Modules

StructuredResponse

This module allows you to get structured responses from the API as C# object. It's useful if you want to use the API for something more than just chat.

record City(string Name, int YearOfFoundation, string Country);

var message = Dialog
    .StartAsSystem("Return requested data.")
    .ThenUser("I need info about Almaty city");
City almaty = await _client.GetStructuredResponse<City>(message, model: ChatCompletionModels.Gpt4Turbo);
Console.WriteLine(almaty); // Name: "Almaty", Country: "Kazakhstan", YearOfFoundation: 1854

Under the hood, it uses the new json mode of the API for GPT4Turbo and for the gpt-3.5-turbo-1106. Regular GPT4 and GPT3.5Turbo models are also supported, but GPT3.5 responses may be unstable (for GPT3.5 it's strictly recommended to provide examples parameter).

More complex examples with arrays, nested objects and enums are available in tests:

NuGet: https://www.nuget.org/packages/OpenAI.ChatGPT.Modules.StructuredResponse

Translator

This module allows you to translate messages from one language to another.

string textToTranslate = "Hello, world!";
string translatedText = await _client.TranslateText(textToTranslate, "English", "Russian");
Console.WriteLine(translatedText); // "Привет, мир!"

Also, it's possible to translate entire object in pair with StructuredResponse module:

var objectToTranslate = new Order(
    new List<Order.Item> 
    {
        new(1,"Book", 5),
        new(2,"Pen", 10),
        new(3,"Notebook", 3)
    }
);
Order translatedObject = await _client.TranslateObject(objectToTranslate, "English", "Russian");

See full example in tests:

public async Task Translate_object_from_English_to_Russian()

NuGet: https://www.nuget.org/packages/OpenAI.ChatGPT.Modules.Translator

Examples

API Parameters

Here is a list of the main parameters that can be used in the ChatCompletions (ChatGPT) API request (OpenAI.ChatGpt/Models/ChatCompletion/ChatCompletionRequest.cs). Some of them are taken from this article: https://towardsdatascience.com/gpt-3-parameters-and-prompt-design-1a595dc5b405
Below listed parameters for ChatCompletions API.

Model

The prediction-generating AI model is specified by the engine parameter. The available models are described below: https://platform.openai.com/docs/models

C# Model API Model
ChatCompletionModels.Gpt4Turbo gpt-4-1106-preview
ChatCompletionModels.Gpt4 gpt-4
ChatCompletionModels.Gpt4_0613 gpt-4-0613
ChatCompletionModels.Gpt4_32k gpt-4-32k
ChatCompletionModels.Gpt4_32k_0613 gpt-4-32k-0613
ChatCompletionModels.Gpt3_5_Turbo gpt-3.5-turbo
ChatCompletionModels.Gpt3_5_Turbo_1106 gpt-3.5-turbo-1106
ChatCompletionModels.Gpt3_5_Turbo_16k gpt-3.5-turbo-16k
ChatCompletionModels.Gpt3_5_Turbo_0613 gpt-3.5-turbo-0613
ChatCompletionModels.Gpt3_5_Turbo_16k_0613 gpt-3.5-turbo-16k-0613
ChatCompletionModels.Gpt4_0314 gpt-4-0314
ChatCompletionModels.Gpt4_32k_0314 gpt-4-32k-0314
ChatCompletionModels.Gpt3_5_Turbo_0301 gpt-3.5-turbo-0301

MaxTokens

The maximum number of tokens allowed for the generated answer. Defaults to ChatCompletionRequest.MaxTokensDefault (64).

  • This value is validated and limited with ChatCompletionModels.GetMaxTokensLimitForModel method.
  • It's possible to calculate approximately tokens count using ChatCompletionMessage.CalculateApproxTotalTokenCount method
  • The number of tokens can be retrieved from the API response: ChatCompletionResponse.Usage.TotalTokens. As a rule of thumb for English, 1 token is around 4 characters (so 100 tokens ≈ 75 words). See tokenizer from OpenAI: https://platform.openai.com/tokenizer
  • Encoding algorithm can be found here: https://github.com/latitudegames/GPT-3-Encoder

Temperature

What sampling temperature to use, between 0 and 2.

  • Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic.
  • Predefined values are available in ChatCompletionTemperatures.
  • Default value is: ChatCompletionTemperatures.Balanced (0.5).

Description: Before being mapped into probabilities, the model outputs unnormalized values (logits). The logits are typically used with a function such as softmax to convert them into probabilities.

But, before applying the softmax function, we can use a trick inspired by thermodynamics and scale the logits with the temperature parameter, i.e. softmax(logits/temperature).

A temperature parameter close to 1 would mean that the logits are passed through the softmax function without modification. If the temperature is close to zero, the highest probable tokens will become very likely compared to the other tokens, i.e. the model becomes more deterministic and will always output the same set of tokens after a given sequence of words.

More parameters description can be found here: Some of them are taken from this article: https://towardsdatascience.com/gpt-3-parameters-and-prompt-design-1a595dc5b405

Using raw client without DI

If you don't need DI and chat history, you can use only the NuGet package OpenAI.ChatGPT:

Install-Package OpenAI.ChatGPT

Then create an instance of OpenAIClient:

_client = new OpenAiClient("{YOUR_OPENAI_API_KEY}");

Simple usage of the Chat Completions API (raw client)

string text = "Who are you?";
string response = await _client.GetChatCompletions(new UserMessage(text), maxTokens: 80);
Console.WriteLine(response);

Streaming response with async streams (like ChatGPT)

var text = "Write the world top 3 songs of Soul genre";
await foreach (string chunk in _client.StreamChatCompletions(new UserMessage(text), maxTokens: 80))
{
    Console.Write(chunk);
}

Continue dialog with ChatGPT (message history)

Use ThenAssistant and ThenUser methods to create a dialog:

var dialog = Dialog.StartAsUser("How many meters are in a kilometer? Write just the number.") //the message from user
          .ThenAssistant("1000") // response from the assistant
          .ThenUser("Convert it to hex. Write just the number."); // the next message from user

await foreach (var chunk in _client.StreamChatCompletions(dialog, maxTokens: 80))
{
    Console.Write(chunk);
}

Or just send message history as a collection.

chatgpt_api_dotnet's People

Contributors

rodion-m avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

chatgpt_api_dotnet's Issues

Using ChatGPT.CreateInMemoryChat, add response on ChatGPT's behalf?

Using the example here, which uses CreateInMemoryChat():
https://github.com/rodion-m/ChatGPT_API_dotnet/blob/master/samples/ChatGpt.ConsoleExample/Program.cs

Is there a way to "inject" a response on GPT's behalf in the middle of a conversation?

For example, a user is having a conversation with GPT, and based on one of their inputs we want to give our own response, but pass that response to GPT so that it knows that was it's last response.

Update Nuget Package

Hi. I love your library. It is easy to use and ligthweight. Could you please update de Nuget package? I need to set the MaxTokens value, but it is a protected variable in 1.1.0 version.
Thanks.

Please integrate apipie.ai

Why bother integrating, together, openrouter, eden, etc? just integrate APIpie to access them all from one place and plus it also provides:

-The most affordable, reliable and fastest AI available
-One API to access ~500 Models and growing
-Language, embedding, voice, image, vision and more
-Global AI load balancing, route queries based on price or latency
-Redundancy for major models providing the greatest up time possible
-Global reporting of AI availability, pricing and performance

Cant create pull requests

I have added a guard to the samples.Blazor project to check for a valid model and API key configuration. Unfortunately, I'm unable to create a pull request for this repo?

When the package will support chatGPT 4o-mini?

GPT-4o mini surpasses GPT-3.5 Turbo and other small models on academic benchmarks across both textual intelligence and multimodal reasoning, and supports the same range of languages as GPT-4o. Therefore, I believe the package should support it

How to use function calling and model config in your package?

Hi @rodion-m, thanks for your updates recently, that helps me a lot. However, I still have a question. Currently, I want to convert config from python code to c#, the python-dev give me these config but I only apply some of them, can you help me check if I can config FUNCTION and other parameters of MODEL CONFIG below.

PROMPT:
{hidden} **_Config Done_**

FUNCTION: (Need help)
{
  "type": "object",
  "title": "classify task",
  "required": [
    "index"
  ],
  "properties": {
    "index": {
      "type": "integer",
      "description": "description task of index"
    }
  },
  "description": "description task"
}

MODEL CONFIG: (Need help)
Model: gpt-4o-mini **_Config Done_**
Temperature: 0.2 **_Config Done_**
Top P: 0.1
Presence Penalty: 0
Frequency Penalty: 0

How to use gpt-4-vision-preview ?

Hi There,

love your work and this repo!
Tried to use gpt-4-vision-preview but only got an exception:
var result = await client.GetChatCompletions(message, model: "gpt-4-vision-preview");

And I got:
image

Thank you
Sebastian

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.