-
Notifications
You must be signed in to change notification settings - Fork 4.9k
chore(source-gcs) update to airbyte-cdk v7 #66671
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
👋 Greetings, Airbyte Team Member!Here are some helpful tips and reminders for your convenience. Helpful Resources
PR Slash CommandsAirbyte Maintainers (that's you!) can execute the following slash commands on your PR:
|
/format-fix
|
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Deploy preview for airbyte-docs ready! ✅ Preview Built with commit c5e5ca1. |
|
Regression tests:local run as gha wasn't able to find connections ![]()
GHA: https://github.com/airbytehq/airbyte/actions/runs/18012486891 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have a concern regarding code duplication that I hope would facilitate our upcoming work. WDYT?
airbyte-integrations/connectors/source-gcs/source_gcs/stream_reader.py
Outdated
Show resolved
Hide resolved
airbyte-integrations/connectors/source-gcs/source_gcs/stream_reader.py
Outdated
Show resolved
Hide resolved
airbyte-integrations/connectors/source-gcs/source_gcs/stream_reader.py
Outdated
Show resolved
Hide resolved
airbyte-integrations/connectors/source-gcs/integration_tests/spec.json
Outdated
Show resolved
Hide resolved
regression tests: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Awesome! Thanks for adding leverage to the CDK. It looks clean in the source without duplication
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
APPROVED
Thanks for improving this.
delivery_method: Union[DeliverRecords, DeliverRawFiles] = Field( | ||
title="Delivery Method", | ||
discriminator="delivery_type", | ||
type="object", | ||
order=3, | ||
display_type="radio", | ||
group="advanced", | ||
default="use_records_transfer", | ||
airbyte_hidden=True, | ||
) | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
def __init__(self, blob: Any, displayed_uri: str = None, **kwargs): | ||
super().__init__(**kwargs) | ||
self.blob = blob | ||
self.displayed_uri = displayed_uri | ||
self.id = self.blob.id | ||
self.created_at = self.blob.time_created.strftime("%Y-%m-%dT%H:%M:%S.%fZ") | ||
self.updated_at = self.blob.updated.strftime("%Y-%m-%dT%H:%M:%S.%fZ") | ||
|
||
@property | ||
def size(self) -> int: | ||
return self.blob.size | ||
|
||
def download_to_local_directory(self, local_file_path: str) -> None: | ||
self.blob.download_to_filename(local_file_path) | ||
|
||
@property | ||
def source_file_relative_path(self) -> str: | ||
return urllib.parse.unquote(self.blob.path) | ||
|
||
@property | ||
def file_uri_for_logging(self) -> str: | ||
return urllib.parse.unquote(self.blob.path) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Cool @darynaishchenko
Co-authored-by: Octavia Squidington III <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Octavia Squidington III <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
What
resolves: https://github.com/airbytehq/airbyte-internal-issues/issues/14340
How
Review guide
User Impact
Can this PR be safely reverted and rolled back?