-
-
Notifications
You must be signed in to change notification settings - Fork 124
Description
Improve documentation
Link
Describe the problem
Currently it is a bit unclear what the role of the various parts of the Destination trait being implemented in the example represents, mainly due to the comments being a bit unclear compared to the actual comments on the trait in https://github.com/supabase/etl/blob/main/etl/src/destination/base.rs. It is hard to understand the full scope of the trait without reading the comments for it (outside of the tutorial). Initially I thought that write_table_rows would be handling the real time traffic, but that was not the case.
Describe the improvement
Imo, it would be beneficial to elaborate on the inputs to the functions in the trait implementations making sure to mention the difference between write_events and write_table_rows, also mentioning that write_events is handling the realtime updates as the comments for the Destination trait do.
On top of that for the understanding of the reader, it might also be helpful to elaborate the possible type of Event enums that flow into the write_events function to really illustrate how a custom destination would be handling real time statements and would actually be handling inserts/updates/deletes, but may want to ignore the Commit events if they aren't relevant for your destination (as BigQuery does for ex)