feat(eap): create LW delete eap-items schema and topic #438
Conversation
onewland
changed the title
|
@phacops asks on Slack:
I did consider this, and landed on the current approach. My reasoning was: The only existing lightweight deletes are on the generic-events dataset. Currently, the consumer uses the storage in the CLI to decide which topic to watch, and it has no logic for filtering which messages it processes. There's a bit of logic in Snuba around mapping storages to Kafka topics: I'm assuming that over the medium term (after migrating issue occurrences) we can expect that all lightweight deletions will happen on eap_items, and that we can remove most of this mapping code, and at that time it will be preferable to land all traffic on an Two alternatives, which could make sense, are:
For either of those we could keep the topic name as-is, though as I said it seems to me like we're assuming that the only pre-items use case is going to go away in not that much time.
If we want to just reuse the original schema and make it more flexible, I'm fine with making that change. The schema isn't that important for us (since it's all internal to Snuba) |
Provide mechanism for lightweight deletes API to handle eap_items. https://linear.app/getsentry/issue/EAP-283/enable-eap-items-storage-to-delete-by-trace-id Depends on getsentry/sentry-kafka-schemas#438 Tested a delete locally by: - setting is_enabled on the storage deletion_settings to 1 - running `curl -H "Content-Type: application/json" -X DELETE http://localhost:1218/eap_items -d "@test-delete.json"` test-delete.json contents: ``` { "query": { "columns": { "project_id": [1], "trace_id": ["85cc9542-ef90-4282-917e-5659721fb940"] } } } ``` Before and after delete: ``` 66bbc2e1b854 :) select trace_id, count() from eap_items_1_local where trace_id='85cc9542-ef90-4282-917e-5659721fb940' group by trace_id SELECT trace_id, count() FROM eap_items_1_local WHERE trace_id = '85cc9542-ef90-4282-917e-5659721fb940' GROUP BY trace_id Query id: f68fc09c-57df-4a53-99b8-e5653800c78a ┌─trace_id─────────────────────────────┬─count()─┐ 1. │ 85cc9542-ef90-4282-917e-5659721fb940 │ 1 │ └──────────────────────────────────────┴─────────┘ 1 row in set. Elapsed: 0.007 sec. 66bbc2e1b854 :) select trace_id, count() from eap_items_1_local where trace_id='85cc9542-ef90-4282-917e-5659721fb940' group by trace_id SELECT trace_id, count() FROM eap_items_1_local WHERE trace_id = '85cc9542-ef90-4282-917e-5659721fb940' GROUP BY trace_id Query id: dec2a45e-1e72-4a3e-a7da-63552141fe37 Ok. 0 rows in set. Elapsed: 0.008 sec. ```
3 participants
create schema, topic, and example for eap-items deletions
in support of https://linear.app/getsentry/issue/EAP-282/create-new-rpc-endpoint-for-deletion-from-eap-items