Add Storage API Acceleration via Apache Arrow Deserialization#646
Add Storage API Acceleration via Apache Arrow Deserialization#646danieljbruce wants to merge 1 commit intomainfrom
Conversation
This commit adds support for picosecond timestamp precision in the BigQuery Storage Read API with Apache Arrow. Due to limitations in the Apache Arrow JavaScript library, a validation hook is added to fall back to microsecond precision when picosecond precision is requested, preventing deserialization errors. Changes: - Updated `arrow.proto` to include `PicosecondTimestampPrecision` enum. - Regenerated protos with new field. - Injected validation hook in `BigQueryReadClient.createReadSession` to handle fallback. - Added documentation about the limitation in `README`. - Added unit tests to verify the fallback mechanism. Co-authored-by: danieljbruce <8935272+danieljbruce@users.noreply.github.com>
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
product-auto-label
bot
added
size: l
1 participant
Implemented Storage API Acceleration via Apache Arrow Deserialization with a graceful degradation fallback for picosecond precision. Updated protos, added validation hooks, updated documentation, and included comprehensive unit tests.
PR created automatically by Jules for task 14717700606008930848 started by @danieljbruce