You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This directory contains the implementation of the Cloud LLM brick, which provides an interface to interact with cloud-based Large Language Models (LLMs) through their REST API.
3
+
The Cloud LLM Brick provides a seamless interface to interact with cloud-based Large Language Models (LLMs) such as OpenAI's GPT, Anthropic's Claude, and Google's Gemini. It abstracts the complexity of REST APIs, enabling you to send prompts, receive responses, and maintain conversational context within your Arduino projects.
4
4
5
5
## Overview
6
6
7
-
The Cloud LLM brick allows users to send prompts to a specified LLM service and receive generated responses.
8
-
It can be configured to work with a curated set of LLM providers that offer RESTful APIs, notably: ChatGPT, Claude and Gemini.
7
+
This Brick acts as a gateway to powerful AI models hosted in the cloud. It is designed to handle the nuances of network communication, authentication, and session management. Whether you need a simple one-off answer or a continuous conversation with memory, the Cloud LLM Brick provides a unified API for different providers.
8
+
9
+
## Features
10
+
11
+
-**Multi-Provider Support**: Compatible with major LLM providers including Anthropic (Claude), OpenAI (GPT), and Google (Gemini).
12
+
-**Conversational Memory**: Built-in support for windowed history, allowing the AI to remember context from previous exchanges.
13
+
-**Streaming Responses**: Receive text chunks in real-time as they are generated, ideal for responsive user interfaces.
14
+
-**Configurable Behavior**: Customize system prompts, temperature (creativity), and request timeouts.
15
+
-**Simple API**: Unified `chat` and `chat_stream` methods regardless of the underlying model provider.
9
16
10
17
## Prerequisites
11
18
12
-
Before using the Cloud LLM brick, ensure you have the following:
13
-
- An account with a cloud-based LLM service (e.g., OpenAI, Cohere, etc.).
14
-
- API access credentials (API key or token) for the LLM service.
15
-
- Network connectivity to access the LLM service endpoint.
19
+
-**Internet Connection**: The board must be connected to the internet to reach the LLM provider's API.
20
+
-**API Key**: A valid API key for the chosen service (e.g., OpenAI API Key, Anthropic API Key).
21
+
-**Python Dependencies**: The Brick relies on LangChain integration packages (`langchain-anthropic`, `langchain-openai`, `langchain-google-genai`).
16
22
17
-
## Features
23
+
## Code Example and Usage
24
+
25
+
### Basic Conversation
18
26
19
-
- Send prompts to a cloud-based LLM service.
20
-
- Receive and process responses from the LLM.
21
-
- Supports both one-shot requests and memory for follow-up questions and answers.
22
-
- Supports a curated set of LLM providers.
27
+
This example initializes the Brick with an OpenAI model and performs a simple chat interaction.
23
28
24
-
## Code example and usage
25
-
Here is a basic example of how to use the Cloud LLM brick:
29
+
**Note:** The API key is not hardcoded. It is retrieved automatically from the **Brick Configuration** in App Lab.
26
30
27
31
```python
28
-
from arduino.app_bricks.cloud_llm import CloudLLM
32
+
import os
33
+
from arduino.app_bricks.cloud_llm import CloudLLM, CloudModel
29
34
from arduino.app_utils import App
30
35
31
-
llm = CloudLLM(api_key="your_api_key_here")
36
+
# Initialize the Brick (API key is loaded from configuration)
37
+
llm = CloudLLM(
38
+
model=CloudModel.OPENAI_GPT,
39
+
system_prompt="You are a helpful assistant for an IoT device."
40
+
)
32
41
33
-
App.start_bricks()
42
+
defsimple_chat():
43
+
# Send a prompt and print the response
44
+
response = llm.chat("What is the capital of Italy?")
45
+
print(f"AI: {response}")
34
46
35
-
response = llm.chat("What is the capital of France?")
36
-
print(response)
47
+
# Run the application
48
+
App.run(simple_chat)
49
+
```
50
+
51
+
### Streaming with Memory
52
+
53
+
This example demonstrates how to enable conversational memory and process the response as a stream of tokens.
54
+
55
+
```python
56
+
from arduino.app_bricks.cloud_llm import CloudLLM, CloudModel
57
+
from arduino.app_utils import App
37
58
38
-
App.stop_bricks()
59
+
# Initialize with memory enabled (keeps last 10 messages)
60
+
# API Key is retrieved automatically from Brick Configuration
61
+
llm = CloudLLM(
62
+
model=CloudModel.ANTHROPIC_CLAUDE
63
+
).with_memory(max_messages=10)
64
+
65
+
defchat_loop():
66
+
whileTrue:
67
+
user_input =input("You: ")
68
+
if user_input.lower() in ["exit", "quit"]:
69
+
break
70
+
71
+
print("AI: ", end="", flush=True)
72
+
73
+
# Stream the response token by token
74
+
for token in llm.chat_stream(user_input):
75
+
print(token, end="", flush=True)
76
+
print() # Newline after response
77
+
78
+
App.run(chat_loop)
39
79
```
80
+
81
+
## Configuration
82
+
83
+
The Brick is initialized with the following parameters:
|`api_key`|`str`|`os.getenv("API_KEY")`| The authentication key for the LLM provider. **Recommended:** Set this via the **Brick Configuration** menu in App Lab instead of code. |
88
+
|`model`|`str`\|`CloudModel`|`CloudModel.ANTHROPIC_CLAUDE`| The specific model to use. Accepts a `CloudModel` enum or its string value. |
89
+
|`system_prompt`|`str`|`""`| A base instruction that defines the AI's behavior and persona. |
90
+
|`temperature`|`float`|`0.7`| Controls randomness. `0.0` is deterministic, `1.0` is creative. |
91
+
|`timeout`|`int`|`30`| Maximum time (in seconds) to wait for a response. |
92
+
93
+
### Supported Models
94
+
95
+
You can select a model using the `CloudModel` enum or by passing the corresponding raw string identifier.
96
+
97
+
| Enum Constant | Raw String ID | Provider Documentation |
description: "Cloud LLM Brick enables seamless integration with cloud-based Large Language Models (LLMs) for advanced AI capabilities in your Arduino projects."
4
-
disabled: true
5
-
6
4
variables:
7
-
- API_KEY
5
+
- name: API_KEY
6
+
description: API Key for the cloud-based LLM service
0 commit comments