Commit 55c8d83
authored
added two new embedding model's encoding (#247)
Library doesn't support two new embedding model's encoding mapper
- `text-embedding-3-small`
- `text-embedding-3-large`
Added Encoding mapper for 2 new embedding models. The source of mapping
is taken from https://github.com/openai/openai-cookbook/blob/main/examples/How_to_count_tokens_with_tiktoken.ipynb1 parent 6cc3a46 commit 55c8d83
1 file changed
+2
-0
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
26 | 26 | | |
27 | 27 | | |
28 | 28 | | |
| 29 | + | |
| 30 | + | |
29 | 31 | | |
30 | 32 | | |
31 | 33 | | |
| |||
0 commit comments