diff --git a/docs/README.md b/docs/README.md index fa29a63..7c7091f 100644 --- a/docs/README.md +++ b/docs/README.md @@ -5,3 +5,4 @@ - [Hugging Face](hugging-face.md) - [LG](lg.md) - [OpenAI](openai.md) +- [Upstage](upstage.md) diff --git a/docs/upstage.md b/docs/upstage.md new file mode 100644 index 0000000..50aeeea --- /dev/null +++ b/docs/upstage.md @@ -0,0 +1,225 @@ +# OpenChat Playground with Upstage + +This page describes how to run OpenChat Playground (OCP) with [Upstage Solar](https://www.upstage.ai/solar-llm) models integration. + +## Get the repository root + +1. Get the repository root. + + ```bash + # bash/zsh + REPOSITORY_ROOT=$(git rev-parse --show-toplevel) + ``` + + ```powershell + # PowerShell + $REPOSITORY_ROOT = git rev-parse --show-toplevel + ``` + +## Run on local machine + +1. Make sure you are at the repository root. + + ```bash + cd $REPOSITORY_ROOT + ``` + +1. Add Upstage API Key for Upstage Solar connection. Make sure you should replace `{{UPSTAGE_API_KEY}}` with your Upstage API key. + + ```bash + # bash/zsh + dotnet user-secrets --project $REPOSITORY_ROOT/src/OpenChat.PlaygroundApp \ + set Upstage:ApiKey "{{UPSTAGE_API_KEY}}" + ``` + + ```powershell + # PowerShell + dotnet user-secrets --project $REPOSITORY_ROOT/src/OpenChat.PlaygroundApp ` + set Upstage:ApiKey "{{UPSTAGE_API_KEY}}" + ``` + + > For more details about Upstage API Keys, refer to the doc, [Upstage Console](https://console.upstage.ai/). + +1. Run the app. The default model OCP uses is [solar-1-mini-chat](https://developers.upstage.ai/docs/apis/chat). + + ```bash + # bash/zsh + dotnet run --project $REPOSITORY_ROOT/src/OpenChat.PlaygroundApp -- \ + --connector-type Upstage + ``` + + ```powershell + # PowerShell + dotnet run --project $REPOSITORY_ROOT/src/OpenChat.PlaygroundApp -- ` + --connector-type Upstage + ``` + + Alternatively, if you want to run with a different model, say [solar-pro](https://developers.upstage.ai/docs/apis/chat), other than the default one, you can specify it as an argument: + + ```bash + # bash/zsh + dotnet run --project $REPOSITORY_ROOT/src/OpenChat.PlaygroundApp -- \ + --connector-type Upstage \ + --model solar-pro + ``` + + ```powershell + # PowerShell + dotnet run --project $REPOSITORY_ROOT/src/OpenChat.PlaygroundApp -- ` + --connector-type Upstage ` + --model solar-pro + ``` + +1. Open your web browser, navigate to `http://localhost:5280`, and enter prompts. + +## Run in local container + +1. Make sure you are at the repository root. + + ```bash + cd $REPOSITORY_ROOT + ``` + +1. Build a container. + + ```bash + docker build -f Dockerfile -t openchat-playground:latest . + ``` + +1. Get Upstage API Key. + + ```bash + # bash/zsh + API_KEY=$(dotnet user-secrets --project ./src/OpenChat.PlaygroundApp list --json | \ + sed -n '/^\/\//d; p' | jq -r '."Upstage:ApiKey"') + ``` + + ```powershell + # PowerShell + $API_KEY = (dotnet user-secrets --project ./src/OpenChat.PlaygroundApp list --json | ` + Select-String -NotMatch '^//(BEGIN|END)' | ConvertFrom-Json).'Upstage:ApiKey' + ``` + +1. Run the app. The default model OCP uses is [solar-1-mini-chat](https://developers.upstage.ai/docs/apis/chat). + + ```bash + # bash/zsh - from locally built container + docker run -i --rm -p 8080:8080 openchat-playground:latest --connector-type Upstage \ + --api-key $API_KEY + ``` + + ```powershell + # PowerShell - from locally built container + docker run -i --rm -p 8080:8080 openchat-playground:latest --connector-type Upstage ` + --api-key $API_KEY + ``` + + ```bash + # bash/zsh - from GitHub Container Registry + docker run -i --rm -p 8080:8080 ghcr.io/aliencube/open-chat-playground/openchat-playground:latest \ + --connector-type Upstage \ + --api-key $API_KEY + ``` + + ```powershell + # PowerShell - from GitHub Container Registry + docker run -i --rm -p 8080:8080 ghcr.io/aliencube/open-chat-playground/openchat-playground:latest ` + --connector-type Upstage ` + --api-key $API_KEY + ``` + + Alternatively, if you want to run with a different model, say [solar-pro](https://developers.upstage.ai/docs/apis/chat), other than the default one, you can specify it as an argument: + + ```bash + # bash/zsh - from locally built container with custom model + docker run -i --rm -p 8080:8080 openchat-playground:latest --connector-type Upstage \ + --api-key $API_KEY \ + --model solar-pro + ``` + + ```powershell + # PowerShell - from locally built container with custom model + docker run -i --rm -p 8080:8080 openchat-playground:latest --connector-type Upstage ` + --api-key $API_KEY ` + --model solar-pro + ``` + +1. Open your web browser, navigate to `http://localhost:8080`, and enter prompts. + +## Run on Azure + +1. Make sure you are at the repository root. + + ```bash + cd $REPOSITORY_ROOT + ``` + +1. Login to Azure. + + ```bash + azd auth login + ``` + +1. Check login status. + + ```bash + azd auth login --check-status + ``` + +1. Initialize `azd` template. + + ```bash + azd init + ``` + + > **NOTE**: You will be asked to provide environment name for provisioning. + +1. Get Upstage API Key. + + ```bash + # bash/zsh + API_KEY=$(dotnet user-secrets --project ./src/OpenChat.PlaygroundApp list --json | \ + sed -n '/^\/\//d; p' | jq -r '."Upstage:ApiKey"') + ``` + + ```powershell + # PowerShell + $API_KEY = (dotnet user-secrets --project ./src/OpenChat.PlaygroundApp list --json | ` + Select-String -NotMatch '^//(BEGIN|END)' | ConvertFrom-Json).'Upstage:ApiKey' + ``` + +1. Set Upstage API Key to azd environment variables. + + ```bash + azd env set UPSTAGE_API_KEY $API_KEY + ``` + + The default model OCP uses is [solar-1-mini-chat](https://developers.upstage.ai/docs/apis/chat). If you want to run with a different model, say [solar-pro](https://developers.upstage.ai/docs/apis/chat), other than the default one, add it to azd environment variables. + + ```bash + azd env set UPSTAGE_MODEL solar-pro + ``` + +1. Set the connector type to `Upstage`. + + ```bash + azd env set CONNECTOR_TYPE Upstage + ``` + +1. Run the following commands in order to provision and deploy the app. + + ```bash + azd up + ``` + + > **NOTE**: You will be asked to provide Azure subscription and location for deployment. + + Once deployed, you will be able to see the deployed OCP app URL. + +1. Open your web browser, navigate to the OCP app URL, and enter prompts. + +1. Clean up all the resources. + + ```bash + azd down --force --purge + ``` \ No newline at end of file diff --git a/infra/main.bicep b/infra/main.bicep index 28ced17..16809ba 100644 --- a/infra/main.bicep +++ b/infra/main.bicep @@ -37,6 +37,10 @@ param openAIModel string = '' @secure() param openAIApiKey string = '' // Upstage +param upstageModel string = '' +param upstageBaseUrl string = '' +@secure() +param upstageApiKey string = '' @allowed([ 'NC24-A100' @@ -88,6 +92,9 @@ module resources 'resources.bicep' = { lgModel: lgModel openAIModel: openAIModel openAIApiKey: openAIApiKey + upstageModel: upstageModel + upstageBaseUrl: upstageBaseUrl + upstageApiKey: upstageApiKey gpuProfileName: gpuProfileName openchatPlaygroundAppExists: openchatPlaygroundAppExists } diff --git a/infra/main.parameters.json b/infra/main.parameters.json index c9c1a5b..b17eb69 100644 --- a/infra/main.parameters.json +++ b/infra/main.parameters.json @@ -38,6 +38,15 @@ "openAIApiKey": { "value": "${OPENAI_API_KEY}" }, + "upstageModel": { + "value": "${UPSTAGE_MODEL}" + }, + "upstageBaseUrl": { + "value": "${UPSTAGE_BASE_URL}" + }, + "upstageApiKey": { + "value": "${UPSTAGE_API_KEY}" + }, "gpuProfileName": { "value": "${GPU_PROFILE_NAME=NC8as-T4}" }, diff --git a/infra/resources.bicep b/infra/resources.bicep index 0483ad3..a989660 100644 --- a/infra/resources.bicep +++ b/infra/resources.bicep @@ -32,6 +32,10 @@ param openAIModel string = '' @secure() param openAIApiKey string = '' // Upstage +param upstageModel string = '' +param upstageBaseUrl string = '' +@secure() +param upstageApiKey string = '' @allowed([ 'NC24-A100' @@ -248,6 +252,22 @@ var envOpenAI = connectorType == 'OpenAI' ? concat(openAIModel != '' ? [ } ] : []) : [] // Upstage +var envUpstage = connectorType == 'Upstage' ? concat(upstageModel != '' ? [ + { + name: 'Upstage__Model' + value: upstageModel + } +] : [], upstageBaseUrl != '' ? [ + { + name: 'Upstage__BaseUrl' + value: upstageBaseUrl + } +] : [], upstageApiKey != '' ? [ + { + name: 'Upstage__ApiKey' + secretRef: 'upstage-api-key' + } +] : []) : [] module openchatPlaygroundApp 'br/public:avm/res/app/container-app:0.18.1' = { name: 'openchatPlaygroundApp' @@ -273,6 +293,11 @@ module openchatPlaygroundApp 'br/public:avm/res/app/container-app:0.18.1' = { name: 'openai-api-key' value: openAIApiKey } + ] : [], upstageApiKey != '' ? [ + { + name: 'upstage-api-key' + value: upstageApiKey + } ] : []) containers: [ { @@ -302,7 +327,9 @@ module openchatPlaygroundApp 'br/public:avm/res/app/container-app:0.18.1' = { envHuggingFace, envOllama, envLG, - envOpenAI, useOllama == true ? [ + envOpenAI, + envUpstage, + useOllama == true ? [ { name: connectorType == 'LG' ? 'LG__BaseUrl' : (connectorType == 'Ollama' ? 'Ollama__BaseUrl' : 'HuggingFace__BaseUrl') value: 'https://${ollama.outputs.fqdn}' diff --git a/src/OpenChat.PlaygroundApp/Abstractions/LanguageModelConnector.cs b/src/OpenChat.PlaygroundApp/Abstractions/LanguageModelConnector.cs index f2857ba..006b41c 100644 --- a/src/OpenChat.PlaygroundApp/Abstractions/LanguageModelConnector.cs +++ b/src/OpenChat.PlaygroundApp/Abstractions/LanguageModelConnector.cs @@ -41,6 +41,7 @@ public static async Task CreateChatClientAsync(AppSettings settings ConnectorType.HuggingFace => new HuggingFaceConnector(settings), ConnectorType.LG => new LGConnector(settings), ConnectorType.OpenAI => new OpenAIConnector(settings), + ConnectorType.Upstage => new UpstageConnector(settings), _ => throw new NotSupportedException($"Connector type '{settings.ConnectorType}' is not supported.") }; diff --git a/src/OpenChat.PlaygroundApp/Connectors/UpstageConnector.cs b/src/OpenChat.PlaygroundApp/Connectors/UpstageConnector.cs new file mode 100644 index 0000000..8a45cd7 --- /dev/null +++ b/src/OpenChat.PlaygroundApp/Connectors/UpstageConnector.cs @@ -0,0 +1,64 @@ +using System.ClientModel; + +using Microsoft.Extensions.AI; + +using OpenAI; + +using OpenChat.PlaygroundApp.Abstractions; +using OpenChat.PlaygroundApp.Configurations; + +namespace OpenChat.PlaygroundApp.Connectors; + +/// +/// This represents the connector entity for Upstage. +/// +public class UpstageConnector(AppSettings settings) : LanguageModelConnector(settings.Upstage) +{ + /// + public override bool EnsureLanguageModelSettingsValid() + { + var settings = this.Settings as UpstageSettings; + if (settings is null) + { + throw new InvalidOperationException("Missing configuration: Upstage."); + } + + if (string.IsNullOrWhiteSpace(settings.BaseUrl?.Trim()) == true) + { + throw new InvalidOperationException("Missing configuration: Upstage:BaseUrl."); + } + + if (string.IsNullOrWhiteSpace(settings.ApiKey?.Trim()) == true) + { + throw new InvalidOperationException("Missing configuration: Upstage:ApiKey."); + } + + if (string.IsNullOrWhiteSpace(settings.Model?.Trim()) == true) + { + throw new InvalidOperationException("Missing configuration: Upstage:Model."); + } + + return true; + } + + /// + public override async Task GetChatClientAsync() + { + var settings = this.Settings as UpstageSettings; + + var credential = new ApiKeyCredential(settings?.ApiKey ?? + throw new InvalidOperationException("Missing configuration: Upstage:ApiKey.")); + + var options = new OpenAIClientOptions + { + Endpoint = new Uri(settings.BaseUrl ?? + throw new InvalidOperationException("Missing configuration: Upstage:BaseUrl.")) + }; + + var client = new OpenAIClient(credential, options); + var chatClient = client.GetChatClient(settings.Model) + .AsIChatClient(); + + return await Task.FromResult(chatClient).ConfigureAwait(false); + } +} diff --git a/test/OpenChat.PlaygroundApp.Tests/Connectors/UpstageConnectorTests.cs b/test/OpenChat.PlaygroundApp.Tests/Connectors/UpstageConnectorTests.cs new file mode 100644 index 0000000..88d158c --- /dev/null +++ b/test/OpenChat.PlaygroundApp.Tests/Connectors/UpstageConnectorTests.cs @@ -0,0 +1,261 @@ +using OpenChat.PlaygroundApp.Abstractions; +using OpenChat.PlaygroundApp.Configurations; +using OpenChat.PlaygroundApp.Connectors; + +namespace OpenChat.PlaygroundApp.Tests.Connectors; + + + +public class UpstageConnectorTests +{ + private const string BaseUrl = "https://api.upstage.ai/v1/solar"; + private const string ApiKey = "test-api-key"; + private const string Model = "solar-1-mini-chat"; + + private static AppSettings BuildAppSettings(string? baseUrl = BaseUrl, string? apiKey = ApiKey, string? model = Model) + { + return new AppSettings + { + ConnectorType = ConnectorType.Upstage, + Upstage = new UpstageSettings + { + BaseUrl = baseUrl, + ApiKey = apiKey, + Model = model + } + }; + } + + [Trait("Category", "UnitTest")] + [Theory] + [InlineData(typeof(LanguageModelConnector), typeof(UpstageConnector), true)] + [InlineData(typeof(UpstageConnector), typeof(LanguageModelConnector), false)] + public void Given_BaseType_Then_It_Should_Be_AssignableFrom_DerivedType(Type baseType, Type derivedType, bool expected) + { + // Act + var result = baseType.IsAssignableFrom(derivedType); + + // Assert + result.ShouldBe(expected); + } + + [Trait("Category", "UnitTest")] + [Fact] + public void Given_Settings_Is_Null_When_EnsureLanguageModelSettingsValid_Invoked_Then_It_Should_Throw() + { + // Arrange + var appSettings = new AppSettings { ConnectorType = ConnectorType.Upstage, Upstage = null }; + var connector = new UpstageConnector(appSettings); + + // Act + Action action = () => connector.EnsureLanguageModelSettingsValid(); + + // Assert + action.ShouldThrow() + .Message.ShouldContain("Missing configuration: Upstage."); + } + + [Trait("Category", "UnitTest")] + [Theory] + [InlineData(null, typeof(InvalidOperationException), "Upstage:BaseUrl")] + [InlineData("", typeof(InvalidOperationException), "Upstage:BaseUrl")] + [InlineData(" ", typeof(InvalidOperationException), "Upstage:BaseUrl")] + public void Given_Invalid_BaseUrl_When_EnsureLanguageModelSettingsValid_Invoked_Then_It_Should_Throw(string? baseUrl, Type expectedType, string expectedMessage) + { + // Arrange + var appSettings = BuildAppSettings(baseUrl: baseUrl); + var connector = new UpstageConnector(appSettings); + + // Act + Action action = () => connector.EnsureLanguageModelSettingsValid(); + + // Assert + action.ShouldThrow(expectedType) + .Message.ShouldContain(expectedMessage); + } + + [Trait("Category", "UnitTest")] + [Theory] + [InlineData(null, typeof(InvalidOperationException), "Upstage:ApiKey")] + [InlineData("", typeof(InvalidOperationException), "Upstage:ApiKey")] + [InlineData(" ", typeof(InvalidOperationException), "Upstage:ApiKey")] + public void Given_Invalid_ApiKey_When_EnsureLanguageModelSettingsValid_Invoked_Then_It_Should_Throw(string? apiKey, Type expectedType, string expectedMessage) + { + // Arrange + var appSettings = BuildAppSettings(apiKey: apiKey); + var connector = new UpstageConnector(appSettings); + + // Act + Action action = () => connector.EnsureLanguageModelSettingsValid(); + + // Assert + action.ShouldThrow(expectedType) + .Message.ShouldContain(expectedMessage); + } + + [Trait("Category", "UnitTest")] + [Theory] + [InlineData(null, typeof(InvalidOperationException), "Upstage:Model")] + [InlineData("", typeof(InvalidOperationException), "Upstage:Model")] + [InlineData(" ", typeof(InvalidOperationException), "Upstage:Model")] + public void Given_Invalid_Model_When_EnsureLanguageModelSettingsValid_Invoked_Then_It_Should_Throw(string? model, Type expectedType, string expectedMessage) + { + // Arrange + var appSettings = BuildAppSettings(model: model); + var connector = new UpstageConnector(appSettings); + + // Act + Action action = () => connector.EnsureLanguageModelSettingsValid(); + + // Assert + action.ShouldThrow(expectedType) + .Message.ShouldContain(expectedMessage); + } + + [Trait("Category", "UnitTest")] + [Fact] + public void Given_Valid_Settings_When_EnsureLanguageModelSettingsValid_Invoked_Then_It_Should_Return_True() + { + // Arrange + var appSettings = BuildAppSettings(); + var connector = new UpstageConnector(appSettings); + + // Act + var result = connector.EnsureLanguageModelSettingsValid(); + + // Assert + result.ShouldBeTrue(); + } + + [Trait("Category", "UnitTest")] + [Fact] + public async Task Given_Valid_Settings_When_GetChatClient_Invoked_Then_It_Should_Return_ChatClient() + { + // Arrange + var settings = BuildAppSettings(); + var connector = new UpstageConnector(settings); + + // Act + var client = await connector.GetChatClientAsync(); + + // Assert + client.ShouldNotBeNull(); + } + + [Trait("Category", "UnitTest")] + [Fact] + public void Given_Settings_Is_Null_When_GetChatClientAsync_Invoked_Then_It_Should_Throw() + { + // Arrange + var appSettings = new AppSettings { ConnectorType = ConnectorType.Upstage, Upstage = null }; + var connector = new UpstageConnector(appSettings); + + // Act + Func func = async () => await connector.GetChatClientAsync(); + + // Assert + func.ShouldThrow() + .Message.ShouldContain("Missing configuration: Upstage:ApiKey."); + } + + [Trait("Category", "UnitTest")] + [Theory] + [InlineData(null, typeof(InvalidOperationException), "Upstage:ApiKey")] + [InlineData("", typeof(ArgumentException), "key")] + public void Given_Missing_ApiKey_When_GetChatClient_Invoked_Then_It_Should_Throw(string? apiKey, Type expected, string message) + { + // Arrange + var settings = BuildAppSettings(apiKey: apiKey); + var connector = new UpstageConnector(settings); + + // Act + Func func = async () => await connector.GetChatClientAsync(); + + // Assert + func.ShouldThrow(expected) + .Message.ShouldContain(message); + } + + [Trait("Category", "UnitTest")] + [Theory] + [InlineData(null, typeof(InvalidOperationException), "Upstage:BaseUrl")] + [InlineData("", typeof(UriFormatException), "empty")] + public void Given_Missing_BaseUrl_When_GetChatClient_Invoked_Then_It_Should_Throw(string? baseUrl, Type expected, string message) + { + // Arrange + var settings = BuildAppSettings(baseUrl: baseUrl); + var connector = new UpstageConnector(settings); + + // Act + Func func = async () => await connector.GetChatClientAsync(); + + // Assert + func.ShouldThrow(expected) + .Message.ShouldContain(message); + } + + [Trait("Category", "UnitTest")] + [Theory] + [InlineData(null, typeof(ArgumentNullException), "model")] + [InlineData("", typeof(ArgumentException), "model")] + public void Given_Missing_Model_When_GetChatClient_Invoked_Then_It_Should_Throw(string? model, Type expected, string message) + { + // Arrange + var settings = BuildAppSettings(model: model); + var connector = new UpstageConnector(settings); + + // Act + Func func = async () => await connector.GetChatClientAsync(); + + // Assert + func.ShouldThrow(expected) + .Message.ShouldContain(message); + } + + [Trait("Category", "UnitTest")] + [Fact] + public async Task Given_Valid_Settings_When_CreateChatClientAsync_Invoked_Then_It_Should_Return_ChatClient() + { + // Arrange + var settings = BuildAppSettings(); + + // Act + var client = await LanguageModelConnector.CreateChatClientAsync(settings); + + // Assert + client.ShouldNotBeNull(); + } + + [Trait("Category", "UnitTest")] + [Theory] + [InlineData("apiKey", null, typeof(InvalidOperationException), "Upstage:ApiKey")] + [InlineData("apiKey", "", typeof(InvalidOperationException), "Upstage:ApiKey")] + [InlineData("apiKey", " ", typeof(InvalidOperationException), "Upstage:ApiKey")] + [InlineData("apiKey", "\t\n\r", typeof(InvalidOperationException), "Upstage:ApiKey")] + [InlineData("baseUrl", null, typeof(InvalidOperationException), "Upstage:BaseUrl")] + [InlineData("baseUrl", "", typeof(InvalidOperationException), "Upstage:BaseUrl")] + [InlineData("baseUrl", " ", typeof(InvalidOperationException), "Upstage:BaseUrl")] + [InlineData("baseUrl", "\t\n\r", typeof(InvalidOperationException), "Upstage:BaseUrl")] + [InlineData("model", null, typeof(InvalidOperationException), "Upstage:Model")] + [InlineData("model", "", typeof(InvalidOperationException), "Upstage:Model")] + [InlineData("model", " ", typeof(InvalidOperationException), "Upstage:Model")] + [InlineData("model", "\t\n\r", typeof(InvalidOperationException), "Upstage:Model")] + public void Given_Invalid_Settings_When_CreateChatClientAsync_Invoked_Then_It_Should_Throw(string parameterName, string? invalidValue, Type expectedType, string expectedMessage) + { + // Arrange + var settings = parameterName switch + { + "apiKey" => BuildAppSettings(apiKey: invalidValue), + "baseUrl" => BuildAppSettings(baseUrl: invalidValue), + "model" => BuildAppSettings(model: invalidValue), + _ => throw new ArgumentException($"Unknown parameter: {parameterName}") + }; + + // Act + Func func = async () => await LanguageModelConnector.CreateChatClientAsync(settings); + + // Assert + func.ShouldThrow(expectedType) + .Message.ShouldContain(expectedMessage); + } +}