Skip to content

Commit 0bdacc0

Browse files
committed
added facts notebook
1 parent 108a9fc commit 0bdacc0

File tree

2 files changed

+967
-2
lines changed

2 files changed

+967
-2
lines changed

10-FACTS-benchmarks/README.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,16 +2,17 @@
22

33
This repo provides functionality to evaluate Contextual AI Grounded Language Model (GLM) responses using the FACTS Grounding benchmark framework.
44
A [script](GLMv2_FACTS.py) is inclulded to generate responses for evaluating Contextual AI for the FACTS benchmark.
5-
An upcoming notebook will walk through the process of assessing factual grounding and quality across multiple LLM models using LLM judges including Gemini-1.5-pro, GPT-4o, and Claude-3-5-sonnet.
5+
A [notebook](facts-starter.ipynb) walks through generating responses and evaluating the models for the FACTS benchmark on the public.
66

77
## **Prerequisites**
88
* [Contextual API key](https://app.contextual.ai/) [Notebook and Script]
99
* Google AI Studio API key [Notebook only]
1010
* OpenAI API key [Notebook only]
1111
* Anthropic API key [Notebook only]
12+
* FACTS public benchmark [Notebook only]
1213

1314
## **Dataset**
14-
The script uses the [FACTS Grounding 1.0 Public Examples](https://kaggle.com/datasets/deepmind/FACTS-grounding-examples/data) dataset containing:
15+
The notebook uses the [FACTS Grounding 1.0 Public Examples](https://kaggle.com/datasets/deepmind/FACTS-grounding-examples/data) dataset containing:
1516
* 860 public examples (out of 1,719 total examples)
1617

1718
## **Notes**

0 commit comments

Comments
 (0)