Model Client Wrapper

[Spartee]

Description
I thought it was confusing how the instructions here (https://docs.arcade.dev/home/supported-models/anthropic) used the OpenAI client to interact with Claude. I think this proof-of-concept helps to decouple the from OpenAI just a little bit.

Proposed Feature
Provide a clear and concise description of the feature you would like implemented.
example: main...nickagliano:arcade-ai:main

posted on behalf of @nickagliano

[nickagliano]

Thanks @Spartee ! If this is going in the right direction I'll clean it up and play around with actually running the test suite.

[Spartee]

@wdawson @nbarbettini what do you think about a wrapper? I think it makes sense, plus we don't EXACTLY conform to OpenAI spec anyway.

[wdawson]

Is the suggestion here that when interacting with Arcade's chat API, you should use this wrapper rather than the OpenAI (or Anthropic) client directly?

If so, I think part of the magic of Arcade is that we're compatible with the official OpenAI (and in the future, Anthropic) APIs. You don't need to change all the code if you want to start using Arcade, you can just change one line to point your client to Arcade.

[nickagliano]

That makes sense, and I'm not familiar with your design philosophies at all so I'll (obviously) defer to you all.

I just thought it was unexpected that you're using the OpenAI client in the Anthropic example.

To send a message to Anthropic, you can use the OpenAI Client. The Arcade Engine will convert the OpenAI requests and responses to and from the Anthropic chat formats.

import os
from openai import OpenAI
 
client = OpenAI(
    base_url="https://api.arcade.dev/v1",
    api_key=os.environ.get("ARCADE_API_KEY"))
 
response = client.chat.completions.create(
    model = "claude-3-5-sonnet-20240620",
    user = "user@arcade.dev",
    messages = [
        {"role": "user", "content": "Summarize my last 10 emails"}
    ]
)

Here's an example, swapping out OpenAI for ModelClientWrapper or ArcadeChatClient, or whatever you want to name it.

import os
from arcade_ai import ModelClientWrapper  # or ArcadeChatClient, etc.

SUPPORTED_PROVIDERS = ["openai", "anthropic", ...] 
 
client = ModelClientWrapper(
    provider="anthropic",
    api_key=os.environ.get("ARCADE_API_KEY"))
 
response = client.chat.completions.create(
    model = "claude-3-5-sonnet-20240620",
    user = "user@arcade.dev",
    messages = [
        {"role": "user", "content": "Summarize my last 10 emails"}
    ]
)

You'd still be compatible with OpenAI, Anthropic, and whoever else you'd want to support—I just think this is more clear you're choosing a specific model provider. And it's still close to a one-line change.

(Also IMO consolidating chat(), handle_chat_interaction(), handle_tool_authorization() into the wrapper is a nice abstraction instead of passing around the client in cli/utils.py.)

[wdawson]

That's a great point. I think maybe it would be nice to support all of it. Right now it's just the OpenAI client but I would like to see us support the Anthropic client and Groq, Gemini, llama, etc. in the future.

Maybe having a wrapper also would help be more clear for the devex. Wdyt?

[nickagliano]

Exactly! I think it would really help chat API devs as you aim to support more model providers as well as improve the experience for users of the chat API.

(I still plan on cleaning this up, just haven't quite had the time to get around to it.)

[wdawson]

Awesome! Yeah, if there's a way to still support using the openai client directly, I'm all for this!