feat(spans): Span V2 kafka schema #440

jjbayer · 2025-09-30T07:03:58Z

Spans written to Kafka by Relay are now a superset of the Span V2 schema.

See https://linear.app/getsentry/project/align-sentry-span-kafka-schema-with-span-v2-a77fced29f22/overview.

Closes INGEST-540.

linear · 2025-09-30T07:04:01Z

INGEST-540 Clean up sentry-kafka-schemas

github-actions · 2025-09-30T08:02:36Z

versions in use:

The following repositories use one of the schemas you are editing. It is recommended to roll out schema changes in small PRs, meaning that if those used versions lag behind the latest, it is probably best to update those services before rolling out your change.

latest version: 2.1.3

changes that might be breaking:

schemas/ingest-spans.v1.schema.json

Added a new property, but the consumer has been ignoring additional properties so far. This is probably still fine, but please double-check that the producer does not already send this property with a different type in practice than you defined in this schema.
```
{"path": "", "change": {"PropertyAdd": {"lhs_additional_properties": true, "added": "attributes"}}}
```
Added a new property, but the consumer has been ignoring additional properties so far. This is probably still fine, but please double-check that the producer does not already send this property with a different type in practice than you defined in this schema.
```
{"path": "", "change": {"PropertyAdd": {"lhs_additional_properties": true, "added": "downsampled_retention_days"}}}
```
Added a new property, but the consumer has been ignoring additional properties so far. This is probably still fine, but please double-check that the producer does not already send this property with a different type in practice than you defined in this schema.
```
{"path": "", "change": {"PropertyAdd": {"lhs_additional_properties": true, "added": "end_timestamp"}}}
```
Added a new property, but the consumer has been ignoring additional properties so far. This is probably still fine, but please double-check that the producer does not already send this property with a different type in practice than you defined in this schema.
```
{"path": "", "change": {"PropertyAdd": {"lhs_additional_properties": true, "added": "is_remote"}}}
```
Added a new property, but the consumer has been ignoring additional properties so far. This is probably still fine, but please double-check that the producer does not already send this property with a different type in practice than you defined in this schema.
```
{"path": "", "change": {"PropertyAdd": {"lhs_additional_properties": true, "added": "kind"}}}
```
Added a new property, but the consumer has been ignoring additional properties so far. This is probably still fine, but please double-check that the producer does not already send this property with a different type in practice than you defined in this schema.
```
{"path": "", "change": {"PropertyAdd": {"lhs_additional_properties": true, "added": "links"}}}
```
Added a new property, but the consumer has been ignoring additional properties so far. This is probably still fine, but please double-check that the producer does not already send this property with a different type in practice than you defined in this schema.
```
{"path": "", "change": {"PropertyAdd": {"lhs_additional_properties": true, "added": "name"}}}
```
Added a new property, but the consumer has been ignoring additional properties so far. This is probably still fine, but please double-check that the producer does not already send this property with a different type in practice than you defined in this schema.
```
{"path": "", "change": {"PropertyAdd": {"lhs_additional_properties": true, "added": "start_timestamp"}}}
```
Added a new property, but the consumer has been ignoring additional properties so far. This is probably still fine, but please double-check that the producer does not already send this property with a different type in practice than you defined in this schema.
```
{"path": "", "change": {"PropertyAdd": {"lhs_additional_properties": true, "added": "status"}}}
```

{"path": "", "change": {"RequiredAdd": {"property": "end_timestamp"}}}

{"path": "", "change": {"RequiredAdd": {"property": "is_remote"}}}

{"path": "", "change": {"RequiredAdd": {"property": "name"}}}

{"path": "", "change": {"RequiredAdd": {"property": "start_timestamp"}}}

{"path": "", "change": {"RequiredAdd": {"property": "status"}}}

benign changes

schemas/buffered-segments.v1.schema.json

{"path": ".spans.?", "change": {"TypeAdd": {"added": "string"}}}

{"path": ".spans.?", "change": {"TypeAdd": {"added": "number"}}}

{"path": ".spans.?", "change": {"TypeAdd": {"added": "integer"}}}

{"path": ".spans.?", "change": {"TypeAdd": {"added": "array"}}}

{"path": ".spans.?", "change": {"TypeAdd": {"added": "boolean"}}}

{"path": ".spans.?", "change": {"TypeAdd": {"added": "null"}}}

Removed a property data from .spans.?, but it is still accepted via additionalProperties=true

{"path": ".spans.?", "change": {"PropertyRemove": {"lhs_additional_properties": true, "removed": "data"}}}

Removed a property description from .spans.?, but it is still accepted via additionalProperties=true

{"path": ".spans.?", "change": {"PropertyRemove": {"lhs_additional_properties": true, "removed": "description"}}}

Removed a property downsampled_retention_days from .spans.?, but it is still accepted via additionalProperties=true

{"path": ".spans.?", "change": {"PropertyRemove": {"lhs_additional_properties": true, "removed": "downsampled_retention_days"}}}

Removed a property duration_ms from .spans.?, but it is still accepted via additionalProperties=true

{"path": ".spans.?", "change": {"PropertyRemove": {"lhs_additional_properties": true, "removed": "duration_ms"}}}

Removed a property end_timestamp_precise from .spans.?, but it is still accepted via additionalProperties=true

{"path": ".spans.?", "change": {"PropertyRemove": {"lhs_additional_properties": true, "removed": "end_timestamp_precise"}}}

Removed a property event_id from .spans.?, but it is still accepted via additionalProperties=true

{"path": ".spans.?", "change": {"PropertyRemove": {"lhs_additional_properties": true, "removed": "event_id"}}}

Removed a property is_segment from .spans.?, but it is still accepted via additionalProperties=true

{"path": ".spans.?", "change": {"PropertyRemove": {"lhs_additional_properties": true, "removed": "is_segment"}}}

Removed a property key_id from .spans.?, but it is still accepted via additionalProperties=true

{"path": ".spans.?", "change": {"PropertyRemove": {"lhs_additional_properties": true, "removed": "key_id"}}}

Removed a property links from .spans.?, but it is still accepted via additionalProperties=true

{"path": ".spans.?", "change": {"PropertyRemove": {"lhs_additional_properties": true, "removed": "links"}}}

Removed a property measurements from .spans.?, but it is still accepted via additionalProperties=true

{"path": ".spans.?", "change": {"PropertyRemove": {"lhs_additional_properties": true, "removed": "measurements"}}}

Removed a property organization_id from .spans.?, but it is still accepted via additionalProperties=true

{"path": ".spans.?", "change": {"PropertyRemove": {"lhs_additional_properties": true, "removed": "organization_id"}}}

Removed a property parent_span_id from .spans.?, but it is still accepted via additionalProperties=true

{"path": ".spans.?", "change": {"PropertyRemove": {"lhs_additional_properties": true, "removed": "parent_span_id"}}}

Removed a property profile_id from .spans.?, but it is still accepted via additionalProperties=true

{"path": ".spans.?", "change": {"PropertyRemove": {"lhs_additional_properties": true, "removed": "profile_id"}}}

Removed a property project_id from .spans.?, but it is still accepted via additionalProperties=true

{"path": ".spans.?", "change": {"PropertyRemove": {"lhs_additional_properties": true, "removed": "project_id"}}}

Removed a property received from .spans.?, but it is still accepted via additionalProperties=true

{"path": ".spans.?", "change": {"PropertyRemove": {"lhs_additional_properties": true, "removed": "received"}}}

Removed a property retention_days from .spans.?, but it is still accepted via additionalProperties=true

{"path": ".spans.?", "change": {"PropertyRemove": {"lhs_additional_properties": true, "removed": "retention_days"}}}

Removed a property segment_id from .spans.?, but it is still accepted via additionalProperties=true

{"path": ".spans.?", "change": {"PropertyRemove": {"lhs_additional_properties": true, "removed": "segment_id"}}}

Removed a property sentry_tags from .spans.?, but it is still accepted via additionalProperties=true

{"path": ".spans.?", "change": {"PropertyRemove": {"lhs_additional_properties": true, "removed": "sentry_tags"}}}

Removed a property span_id from .spans.?, but it is still accepted via additionalProperties=true

{"path": ".spans.?", "change": {"PropertyRemove": {"lhs_additional_properties": true, "removed": "span_id"}}}

Removed a property start_timestamp_ms from .spans.?, but it is still accepted via additionalProperties=true

{"path": ".spans.?", "change": {"PropertyRemove": {"lhs_additional_properties": true, "removed": "start_timestamp_ms"}}}

Removed a property start_timestamp_precise from .spans.?, but it is still accepted via additionalProperties=true

{"path": ".spans.?", "change": {"PropertyRemove": {"lhs_additional_properties": true, "removed": "start_timestamp_precise"}}}

Removed a property tags from .spans.?, but it is still accepted via additionalProperties=true

{"path": ".spans.?", "change": {"PropertyRemove": {"lhs_additional_properties": true, "removed": "tags"}}}

Removed a property trace_id from .spans.?, but it is still accepted via additionalProperties=true

{"path": ".spans.?", "change": {"PropertyRemove": {"lhs_additional_properties": true, "removed": "trace_id"}}}

{"path": ".spans.?", "change": {"RequiredRemove": {"property": "duration_ms"}}}

{"path": ".spans.?", "change": {"RequiredRemove": {"property": "end_timestamp_precise"}}}

{"path": ".spans.?", "change": {"RequiredRemove": {"property": "is_segment"}}}

{"path": ".spans.?", "change": {"RequiredRemove": {"property": "organization_id"}}}

{"path": ".spans.?", "change": {"RequiredRemove": {"property": "project_id"}}}

{"path": ".spans.?", "change": {"RequiredRemove": {"property": "received"}}}

{"path": ".spans.?", "change": {"RequiredRemove": {"property": "retention_days"}}}

{"path": ".spans.?", "change": {"RequiredRemove": {"property": "span_id"}}}

{"path": ".spans.?", "change": {"RequiredRemove": {"property": "start_timestamp_ms"}}}

{"path": ".spans.?", "change": {"RequiredRemove": {"property": "start_timestamp_precise"}}}

{"path": ".spans.?", "change": {"RequiredRemove": {"property": "trace_id"}}}

schemas/ingest-spans.v1.schema.json

Removed a property data from ``, but it is still accepted via additionalProperties=true

{"path": "", "change": {"PropertyRemove": {"lhs_additional_properties": true, "removed": "data"}}}

Removed a property description from ``, but it is still accepted via additionalProperties=true

{"path": "", "change": {"PropertyRemove": {"lhs_additional_properties": true, "removed": "description"}}}

Removed a property duration_ms from ``, but it is still accepted via additionalProperties=true

{"path": "", "change": {"PropertyRemove": {"lhs_additional_properties": true, "removed": "duration_ms"}}}

Removed a property end_timestamp_precise from ``, but it is still accepted via additionalProperties=true

{"path": "", "change": {"PropertyRemove": {"lhs_additional_properties": true, "removed": "end_timestamp_precise"}}}

Removed a property measurements from ``, but it is still accepted via additionalProperties=true

{"path": "", "change": {"PropertyRemove": {"lhs_additional_properties": true, "removed": "measurements"}}}

Removed a property profile_id from ``, but it is still accepted via additionalProperties=true

{"path": "", "change": {"PropertyRemove": {"lhs_additional_properties": true, "removed": "profile_id"}}}

Removed a property segment_id from ``, but it is still accepted via additionalProperties=true

{"path": "", "change": {"PropertyRemove": {"lhs_additional_properties": true, "removed": "segment_id"}}}

Removed a property sentry_tags from ``, but it is still accepted via additionalProperties=true

{"path": "", "change": {"PropertyRemove": {"lhs_additional_properties": true, "removed": "sentry_tags"}}}

Removed a property start_timestamp_ms from ``, but it is still accepted via additionalProperties=true

{"path": "", "change": {"PropertyRemove": {"lhs_additional_properties": true, "removed": "start_timestamp_ms"}}}

Removed a property start_timestamp_precise from ``, but it is still accepted via additionalProperties=true

{"path": "", "change": {"PropertyRemove": {"lhs_additional_properties": true, "removed": "start_timestamp_precise"}}}

Removed a property tags from ``, but it is still accepted via additionalProperties=true

{"path": "", "change": {"PropertyRemove": {"lhs_additional_properties": true, "removed": "tags"}}}

{"path": ".event_id", "change": {"TypeAdd": {"added": "number"}}}

{"path": ".event_id", "change": {"TypeAdd": {"added": "integer"}}}

{"path": ".event_id", "change": {"TypeAdd": {"added": "object"}}}

{"path": ".event_id", "change": {"TypeAdd": {"added": "array"}}}

{"path": ".event_id", "change": {"TypeAdd": {"added": "boolean"}}}

{"path": ".event_id", "change": {"TypeAdd": {"added": "null"}}}

{"path": ".key_id", "change": {"TypeAdd": {"added": "string"}}}

{"path": ".key_id", "change": {"TypeAdd": {"added": "number"}}}

{"path": ".key_id", "change": {"TypeAdd": {"added": "object"}}}

{"path": ".key_id", "change": {"TypeAdd": {"added": "array"}}}

{"path": ".key_id", "change": {"TypeAdd": {"added": "boolean"}}}

{"path": ".key_id", "change": {"TypeAdd": {"added": "null"}}}

{"path": ".key_id", "change": {"RangeRemove": {"removed": "minimum", "value": 0.0}}}

{"path": ".organization_id", "change": {"TypeAdd": {"added": "string"}}}

{"path": ".organization_id", "change": {"TypeAdd": {"added": "number"}}}

{"path": ".organization_id", "change": {"TypeAdd": {"added": "object"}}}

{"path": ".organization_id", "change": {"TypeAdd": {"added": "array"}}}

{"path": ".organization_id", "change": {"TypeAdd": {"added": "boolean"}}}

{"path": ".organization_id", "change": {"TypeAdd": {"added": "null"}}}

{"path": ".organization_id", "change": {"RangeRemove": {"removed": "minimum", "value": 0.0}}}

{"path": ".project_id", "change": {"TypeAdd": {"added": "string"}}}

{"path": ".project_id", "change": {"TypeAdd": {"added": "number"}}}

{"path": ".project_id", "change": {"TypeAdd": {"added": "object"}}}

{"path": ".project_id", "change": {"TypeAdd": {"added": "array"}}}

{"path": ".project_id", "change": {"TypeAdd": {"added": "boolean"}}}

{"path": ".project_id", "change": {"TypeAdd": {"added": "null"}}}

{"path": ".project_id", "change": {"RangeRemove": {"removed": "minimum", "value": 0.0}}}

{"path": ".received", "change": {"TypeAdd": {"added": "string"}}}

{"path": ".received", "change": {"TypeAdd": {"added": "object"}}}

{"path": ".received", "change": {"TypeAdd": {"added": "array"}}}

{"path": ".received", "change": {"TypeAdd": {"added": "boolean"}}}

{"path": ".received", "change": {"TypeAdd": {"added": "null"}}}

{"path": ".received", "change": {"RangeRemove": {"removed": "minimum", "value": 0.0}}}

{"path": ".retention_days", "change": {"TypeAdd": {"added": "string"}}}

{"path": ".retention_days", "change": {"TypeAdd": {"added": "number"}}}

{"path": ".retention_days", "change": {"TypeAdd": {"added": "object"}}}

{"path": ".retention_days", "change": {"TypeAdd": {"added": "array"}}}

{"path": ".retention_days", "change": {"TypeAdd": {"added": "boolean"}}}

{"path": ".retention_days", "change": {"TypeAdd": {"added": "null"}}}

{"path": ".retention_days", "change": {"RangeRemove": {"removed": "minimum", "value": 0.0}}}

{"path": ".retention_days", "change": {"RangeRemove": {"removed": "maximum", "value": 65535.0}}}

{"path": ".trace_id", "change": {"TypeAdd": {"added": "number"}}}

{"path": ".trace_id", "change": {"TypeAdd": {"added": "integer"}}}

{"path": ".trace_id", "change": {"TypeAdd": {"added": "object"}}}

{"path": ".trace_id", "change": {"TypeAdd": {"added": "array"}}}

{"path": ".trace_id", "change": {"TypeAdd": {"added": "boolean"}}}

{"path": ".trace_id", "change": {"TypeAdd": {"added": "null"}}}

{"path": "", "change": {"RequiredRemove": {"property": "duration_ms"}}}

{"path": "", "change": {"RequiredRemove": {"property": "end_timestamp_precise"}}}

{"path": "", "change": {"RequiredRemove": {"property": "start_timestamp_ms"}}}

{"path": "", "change": {"RequiredRemove": {"property": "start_timestamp_precise"}}}

⚠️ This PR contains breaking changes. Normally you should avoid that and make
your consumer backwards-compatible (meaning that updated consumers can still
accept old messages). There are a few exceptions:

If consumers already require these invariants in practice, and you're
just adjusting the JSON schema to reality, ignore this warning.
If you know what you are doing, this change could potentially be rolled out
to producers first, but that's not a flow we support.

jjbayer · 2025-09-30T09:26:56Z

schemas/buffered-segments.v1.schema.json

      "type": "array",
      "minItems": 1,
      "items": {
-        "$ref": "#/definitions/SegmentSpan"


Segment spans and ingest spans have the same format now.

schemas/ingest-spans.v1.schema.json

jjbayer · 2025-09-30T09:29:40Z

schemas/ingest-spans.v1.schema.json

      "properties": {
        "event_id": {
-          "$ref": "#/definitions/UUID"
+          "$ref": "file://./ingest-spans.v1.schema.json#/definitions/UUID"


I had to use full URLs here to keep the build script & tests happy.

wip

3aa5eb2

jjbayer and others added 3 commits September 30, 2025 09:10

example

bb3e246

fix imports

314b902

style(lint): Auto commit lint changes

55c317c

getsantry bot and others added 6 commits September 30, 2025 08:02

style(lint): Auto commit lint changes

fbe5986

fix, replace example

c926b63

style(lint): Auto commit lint changes

7389c3b

fix relative file paths for tests

3b4b698

style(lint): Auto commit lint changes

a312cdb

style(lint): Auto commit lint changes

1a28ed6

jjbayer commented Sep 30, 2025

View reviewed changes

jjbayer changed the title ~~feat(spans): New Kafka schema~~ feat(spans): Span V2 kafka schema Sep 30, 2025

jjbayer marked this pull request as ready for review September 30, 2025 09:30

jjbayer requested a review from a team as a code owner September 30, 2025 09:30

jjbayer requested a review from a team September 30, 2025 09:30

jjbayer requested a review from a team as a code owner September 30, 2025 09:30

jjbayer and others added 2 commits September 30, 2025 14:40

feat: relax attribute schema

0644639

style(lint): Auto commit lint changes

8049db4

jjbayer mentioned this pull request Sep 30, 2025

ref(spans): Use Span V2 kafka schema in span consumers getsentry/sentry#100181

Merged

Update schemas/ingest-spans.v1.schema.json

b6e3982

Dav1dde approved these changes Oct 1, 2025

View reviewed changes

jjbayer merged commit 7a89664 into main Oct 1, 2025
17 of 18 checks passed

jjbayer deleted the feat/span-v2 branch October 1, 2025 08:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

feat(spans): Span V2 kafka schema #440

feat(spans): Span V2 kafka schema #440

Uh oh!

jjbayer commented Sep 30, 2025 •

edited

Loading

Uh oh!

linear bot commented Sep 30, 2025

Uh oh!

github-actions bot commented Sep 30, 2025 •

edited

Loading

Uh oh!

jjbayer Sep 30, 2025

Uh oh!

Uh oh!

jjbayer Sep 30, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

feat(spans): Span V2 kafka schema #440

feat(spans): Span V2 kafka schema #440

Uh oh!

Conversation

jjbayer commented Sep 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

linear bot commented Sep 30, 2025

Uh oh!

github-actions bot commented Sep 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jjbayer Sep 30, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jjbayer Sep 30, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jjbayer commented Sep 30, 2025 •

edited

Loading

github-actions bot commented Sep 30, 2025 •

edited

Loading