You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+86-7Lines changed: 86 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -34,18 +34,97 @@ dependencies:
34
34
35
35
In the future, this SDK will be added to the [ESP component registry](https://components.espressif.com).
36
36
37
-
## Examples
37
+
## Documentation
38
38
39
-
This repository contains several examples you can use as a starting point for your project:
40
-
- [Voice agent](./examples/voice_agent/)
41
-
- *More examples coming soon*
39
+
Please refer to the [LiveKit Docs](https://docs.livekit.io/home/) for an introduction to the platform and its features, or to the API reference (*TODO: Not published yet*) for specifics about this SDK.
42
40
43
-
See the README located in each example directory for setup information and build instructions.
41
+
## Example: Voice Agent
44
42
45
-
## Documentation
43
+
This repository contains an example application built with this SDK and [LiveKit Agents](https://docs.livekit.io/agents/), enabling bidirectional voice communication with an AI agent. The agent can interact with hardware in response to user requests. Below is an example of a conversation between a user and the agent:
46
44
47
-
Please refer to the [LiveKit Docs](https://docs.livekit.io/home/) for an introduction to the platform and its features, or to the API reference (*TODO: Not published yet*) for specifics about this SDK.
45
+
> **User:** What is the current CPU temperature? \
46
+
> **Agent:** The CPU temperature is currently 33°C.
47
+
48
+
> **User:** Turn on the blue LED. \
49
+
> **Agent:** *[turns blue LED on]*
50
+
51
+
> **User:** Turn on the yellow LED. \
52
+
> **Agent:** I'm sorry, the board does not have a yellow LED.
53
+
54
+
### Requirements
55
+
56
+
- Software:
57
+
- [IDF](https://docs.espressif.com/projects/esp-idf/en/stable/esp32/get-started/index.html) release v5.4 or later
58
+
- Python 3.9 or later
59
+
- LiveKit Cloud Project
60
+
- Sandbox Token Server (created from your cloud project)
61
+
- API keys for OpenAI, Deepgram, and Cartesia.
62
+
- Hardware
63
+
- Dev board: [ESP32-S3-Korvo-2](https://docs.espressif.com/projects/esp-adf/en/latest/design-guide/dev-boards/user-guide-esp32-s3-korvo-2.html)
64
+
- Two micro USB cables: one for power, one for flashing
65
+
- Mono enclosed speaker (example from [Adafruit](https://www.adafruit.com/product/3351))
66
+
67
+
### Run example
68
+
69
+
To run the example on your board, begin in your terminal by navigating to the example's root directory: *[examples/voice_agent](./examples/voice_agent/)*.
70
+
71
+
#### 1. Configuration
72
+
73
+
The example requires a network connection and Sandbox ID from your [LiveKit Cloud Project](https://cloud.livekit.io/projects/p_/sandbox/templates/token-server). To configure these settings from your terminal, launch *menuconfig*:
74
+
```sh
75
+
idf.py menuconfig
76
+
```
77
+
78
+
With *menuconfig* open, navigate to the *LiveKit Example* menu and configure the following settings:
79
+
80
+
- Network → Wi-Fi SSID
81
+
- Network → Wi-Fi password
82
+
- Room connection → Sandbox ID
83
+
84
+
For more information about available options, please refer to [this guide](./examples/README.md#configuration).
85
+
86
+
#### 2. Build & flash
87
+
88
+
Begin by connecting your dev board via USB. With the board connected, use the following command
89
+
to build the example, flash it to your board, and monitor serial output:
90
+
91
+
```sh
92
+
idf.py flash monitor
93
+
```
94
+
95
+
Once running on device, the example will establish a network connection and then connect to a
96
+
LiveKit room. Once connected, you will see the following log message:
97
+
98
+
```sh
99
+
I (17208) livekit_example: Connected to room
100
+
```
101
+
102
+
If you encounter any issues during this process, please refer to the example [troubleshooting guide](./examples/README.md/#troubleshooting).
103
+
104
+
### Run agent
105
+
106
+
With the example running on your board, the next step is to run the agent so it can join the room.
107
+
Begin by navigating to the agent's source directory in your terminal: *[examples/voice_agent/agent](./examples/voice_agent/agent)*.
108
+
109
+
In this directory, create a *.env* file containing the required API keys:
110
+
111
+
```sh
112
+
DEEPGRAM_API_KEY=<your Deepgram API Key>
113
+
OPENAI_API_KEY=<your OpenAI API Key>
114
+
CARTESIA_API_KEY=<your Cartesia API Key>
115
+
LIVEKIT_API_KEY=<your API Key>
116
+
LIVEKIT_API_SECRET=<your API Secret>
117
+
LIVEKIT_URL=<your server URL>
118
+
```
119
+
120
+
With the API keys in place, download the required files and run the agent in development mode as follows:
121
+
122
+
```sh
123
+
python agent.py download-files
124
+
python agent.py dev
125
+
```
48
126
127
+
With the agent running, it will discover and join the room, and you will now be able to engage in two-way conversation. Try asking some of the questions shown above.
This directory contains example applications using the LiveKit ESP-32 SDK, demonstrating various features and
4
+
use cases. For setup instructions specific to each example, see the example's README.
5
+
6
+
## Configuration
7
+
8
+
This guide lists the settings available for all examples. From within the example's root directory, you can set these options in two ways:
9
+
10
+
1.**Using *menuconfig***: run `idf.py menuconfig`, navigate to the *LiveKit Example* submenu, and configure the
11
+
list of available settings.
12
+
13
+
2.**In an *sdkconfig* file**: place setting key-value pairs in an *sdkconfig* file in the format shown below.
14
+
15
+
### Codec board type
16
+
17
+
For examples that support multiple development boards, you can set the model of the board you are using as follows:
18
+
19
+
```ini
20
+
CONFIG_CODEC_BOARD_TYPE="S3_Korvo_V2"
21
+
```
22
+
23
+
See the README for each example for information about supported boards.
24
+
25
+
### Network
26
+
27
+
All examples require a network connection in order to connect to a LiveKit server. Choose one of the following connection options:
28
+
29
+
#### Wi-Fi
30
+
31
+
Connect using Wi-Fi by specifying your network SSID and password as follows:
32
+
33
+
```ini
34
+
CONFIG_NETWORK_MODE_WIFI=y
35
+
CONFIG_WIFI_SSID="<your SSID>"
36
+
CONFIG_WIFI_PASSWORD="<your password>"
37
+
```
38
+
39
+
#### Ethernet
40
+
41
+
You can choose to connect using Ethernet on supported boards such as the [ESP32-P4-Function-EV-Board](https://docs.espressif.com/projects/esp-dev-kits/en/latest/esp32p4/esp32-p4-function-ev-board/user_guide.html).
42
+
43
+
TODO:
44
+
45
+
### Room connection
46
+
47
+
In production, your backend server is responsible for [generating tokens](https://docs.livekit.io/home/server/generating-tokens/) for users to connect to a room.
48
+
49
+
However, for the purposes of demonstration, each example supports two methods to supply the required tokens without
50
+
creating a custom backend. Choose one of the following methods:
51
+
52
+
#### Option A: Sandbox token server (recommended)
53
+
54
+
A Sandbox Token Server is simple hosted component that exposes an HTTP endpoint to generate tokens—much like your own
55
+
server would in production. To use this option, create a [Sandbox Token Server](https://cloud.livekit.io/projects/p_/sandbox/templates/token-server) for your LiveKit Cloud project, and set its ID in _sdkconfig_:
56
+
57
+
```ini
58
+
CONFIG_LK_USE_SANDBOX=y
59
+
CONFIG_LK_SANDBOX_ID="<your sandbox id>"
60
+
```
61
+
62
+
(Optional) If you would like the token to be generated with a specific room or participant name, you can also add the following keys:
63
+
64
+
```ini
65
+
CONFIG_LK_SANDBOX_ROOM_NAME="robot-control"
66
+
CONFIG_LK_SANDBOX_PARTICIPANT_NAME="esp-32"
67
+
```
68
+
69
+
#### Option B: Pre-generated token
70
+
71
+
Connect to a room by proving a manually generated token and custom LiveKit server URL:
72
+
73
+
```ini
74
+
CONFIG_LK_USE_PREGENERATED=y
75
+
CONFIG_LK_TOKEN="your-jwt-token"
76
+
CONFIG_LK_SERVER_URL="ws://localhost:7880"
77
+
```
78
+
79
+
This option can be useful for local development or troubleshooting. Tokens can be generated using the *token* subcommand on the [LiveKit CLI](https://docs.livekit.io/home/cli/cli-setup/#generate-access-token), and you can follow [this guide](https://docs.livekit.io/home/self-hosting/local/) to install LiveKit server to run from your local machine.
80
+
81
+
## Troubleshooting
82
+
83
+
### No serial ports found
84
+
85
+
If your board's serial port cannot be detected, check to see if it is recognized by your operating system:
86
+
87
+
- macOS: Run `ls /dev/cu.*` and look for `/dev/cu.usbserial-*` or similar.
88
+
- Linux: Run `ls /dev/ttyUSB*` or `ls /dev/ttyACM*`.
89
+
- Windows: Check Device Manager under "Ports (COM & LPT)" for the COM port (e.g. _COM3_).
90
+
91
+
If you see your board listed, you can manually specify its port to *idf.py* as follows:
0 commit comments