feat: add s3 implementation for storage layer #10170

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Closed

jordanrfrazier wants to merge 24 commits into release-1.6.5 from 1.6.4-patch-s3-impl

Collaborator

jordanrfrazier commented Oct 8, 2025

No description provided.


          Basic s3 abstractions and framework for storage

5d20af4

Contributor

coderabbitai bot commented Oct 8, 2025 •

edited

Loading

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

✨ Finishing touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment
Commit unit tests in branch 1.6.4-patch-s3-impl

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

github-actions bot added the enhancement label


          [autofix.ci] apply automated fixes

6bb3b12

github-actions bot added enhancement and removed enhancement labels


          [autofix.ci] apply automated fixes (attempt 2/3)

fd02b38

github-actions bot added enhancement and removed enhancement labels

jordanrfrazier added 2 commits

October 8, 2025 01:18


          Remove duplicate code, simplifications, comments

c4b70ed


          csv and json to data comp fixes

5e30e9d

github-actions bot added enhancement and removed enhancement labels

jordanrfrazier commented

View reviewed changes

src/backend/base/langflow/components/data/file.py

    
                  TEXT_EXTENSIONS = TEXT_FILE_TYPES

                  # Extensions that require Docling for processing (images, advanced office formats, etc.)

                  DOCLING_ONLY_EXTENSIONS = [

Collaborator Author

jordanrfrazier Oct 8, 2025

sync with eric

jordanrfrazier commented

View reviewed changes

src/backend/base/langflow/components/data/file.py

    
                      )

                      return file_path.lower().endswith(docling_exts)

                  async def _get_local_file_for_docling(self, file_path: str) -> tuple[str, bool]:

Collaborator Author

jordanrfrazier Oct 8, 2025

untested

jordanrfrazier changed the base branch from release-1.6.4 to release-1.6.5

October 8, 2025 05:59

github-actions bot added enhancement and removed enhancement labels


          Add hash lock to postgres migrations

000fb4c

github-actions bot added enhancement and removed enhancement labels

jordanrfrazier added 2 commits

October 9, 2025 16:26


          Add guarddrails against using some components while in s3 mode

aa2880f


          Clean up tests; use real env vars

48c46cc

github-actions bot added enhancement and removed enhancement labels


          [autofix.ci] apply automated fixes

865d8a1

github-actions bot added enhancement and removed enhancement labels


          [autofix.ci] apply automated fixes (attempt 2/3)

github-actions bot removed the enhancement label


          use issubclass in the pool creation

6b89a33

github-actions bot added enhancement and removed enhancement labels


          [autofix.ci] apply automated fixes

0ed8661

github-actions bot added enhancement and removed enhancement labels


          Use static lock key for alembic migrations - TESTING

cb38603

github-actions bot added enhancement and removed enhancement labels


          [autofix.ci] apply automated fixes

64c5712

github-actions bot added enhancement and removed enhancement labels


          Try adding explicit connect args with prepare threshold to disable al…

d818213

…embic prepared stmts

github-actions bot added enhancement and removed enhancement labels


          Fix previous -- only add prepare threshold for postgres

8b9e7a9

github-actions bot added enhancement and removed enhancement labels


          add LANGFLOW_MIGRATION_LOCK_NAMESPACE for the lock key

586dce7

github-actions bot added enhancement and removed enhancement labels

sonarqubecloud bot commented Oct 15, 2025

Quality Gate failed

Failed conditions
4.1% Duplication on New Code (required ≤ 3%)
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE


          [autofix.ci] apply automated fixes

87ea318

github-actions bot added enhancement and removed enhancement labels


          Scope sessions to read/write and normalize using single mgmt function (…

2ba02d8

…#10377)

* Scope sessions to read/write and normalize using single mgmt function

* [autofix.ci] apply automated fixes

* comments and dedupe

* Fix not persisted issues

* [autofix.ci] apply automated fixes

* Re-add initialization of sut :wq
peruser which adds default folder

* Add debug log

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>

github-actions bot added enhancement and removed enhancement labels

jordanrfrazier closed this

codeflash-ai bot mentioned this pull request

⚡️ Speed up method S3StorageService._get_client by 276% in PR #10384 (1.6.4-patch-s3-impl) #10388

Closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels