Skip to content

Conversation

@wanyingz-dis
Copy link
Contributor

What this PR does / why we need it

This PR introduces the design document for the first version of the graphql server.
The goal is to define how the dashboard will fetch experiment data through a simple, read-only graphql API layer. The document covers the proposed schema, API scope, directory structure, and the integration plan with fastapi and strawberry.

Why this is needed

The dashboard (Issue #61) requires a way to fetch experiments, trials, runs, and metrics efficiently.
A GraphQL endpoint allows the frontend to request exactly the data it needs with a single query per view, which avoids multiple REST calls

@InftyAI-Agent InftyAI-Agent added needs-triage Indicates an issue or PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. do-not-merge/needs-kind Indicates a PR lacks a label and requires one. labels Nov 16, 2025
@kerthcet
Copy link
Member

/assign
will review this later.

Copy link
Member

@kerthcet kerthcet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @wanyingz-dis
General LGTM, we can discuss more on the implementaton.

## 4. Schema Proposal (v0.1)
### 4.1 Types
```
type Experiment {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add Projects as well. Projects is for multi-tenant, for example, for multi-teams, each one will have a different project_id.

trial_id: ID!
meta: JSON
created_at: DateTime
metrics: [Metric]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's remove metrics from here. Metrics maybe query with trial_id, because all the metrics reflects the trial condition. We can add it in the future when needed.

experiment(uuid: ID!): Experiment
trials(experiment_uuid: ID!): [Trial]
runs(trial_uuid: ID!): [Run]
metrics(run_uuid: ID!): [Metric]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rename to trial_metrics(trial_id). We can add run_metrics in the future. Also let's use id rather than uuid as the parameter, e.g. run_uuid -> run_id.

### 4.2 Queries
```
type Query {
experiments: [Experiment]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also projects here.

- experiments
- experiment(uuid)
- nested queries (experiment --> trials --> runs)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two blank lines are enough.

Is read-only sufficient for v0.1?
(Default assumption: yes, until dashboard requires creation workflows.)
Do we want nested queries (Experiment --> Trials --> Runs) or only flat queries?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto



## 10. Open Questions
Is read-only sufficient for v0.1?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes.

## 10. Open Questions
Is read-only sufficient for v0.1?
(Default assumption: yes, until dashboard requires creation workflows.)
Do we want nested queries (Experiment --> Trials --> Runs) or only flat queries?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's make it optional, the frontend can query when they want.

Copy link
Member

@kerthcet kerthcet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's use id rather than uuid for all the parameters.

type Query {
experiments: [Experiment]
experiment(uuid: ID!): Experiment
trials(experiment_uuid: ID!): [Trial]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add trial and run queries as well.

… query, replaced uuid param with id; renamed metric, updated schema, formatting
@wanyingz-dis
Copy link
Contributor Author

Thanks for the review and feedback @kerthcet , I have updated the design doc according to the review. Please take another look.

@kerthcet
Copy link
Member

/retest

@kerthcet
Copy link
Member

/lgtm
/approve
/kind documentation

@InftyAI-Agent InftyAI-Agent added lgtm Looks good to me, indicates that a PR is ready to be merged. approved Indicates a PR has been approved by an approver from all required OWNERS files. documentation Categorizes issue or PR as related to documentation. labels Nov 17, 2025
@kerthcet
Copy link
Member

Let's start to build the api server then!

@InftyAI-Agent InftyAI-Agent removed the do-not-merge/needs-kind Indicates a PR lacks a label and requires one. label Nov 17, 2025
@InftyAI-Agent InftyAI-Agent merged commit 3fb9a0c into InftyAI:main Nov 17, 2025
24 of 26 checks passed
@kerthcet
Copy link
Member

part of #59

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. documentation Categorizes issue or PR as related to documentation. lgtm Looks good to me, indicates that a PR is ready to be merged. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants