-
Notifications
You must be signed in to change notification settings - Fork 3.6k
[Improve](segment) Put Segment footer into index page cache #56382
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
run buildall |
run buildall |
This commit modified the caching logic for the Segment footer. Previously, the Segment footer was cached in the `DATA_PAGE` cache. This was incorrect as the footer is metadata, not data. This commit changes it to use the more appropriate `INDEX_PAGE` cache. Additionally, the calculation for `_meta_mem_usage` has been adjusted for better accuracy. The footer size is no longer included in the calculation since it's now managed by the `StoragePageCache`. The memory estimation for column readers is also updated to use `config::max_segment_partial_column_cache_size` for a more realistic value.
145fea1
to
9c57d4b
Compare
run buildall |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
PR approved by anyone and no changes requested. |
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
ClickBench: Total hot run time: 30.68 s
|
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
PR approved by at least one committer and no changes requested. |
This commit modified the caching logic for the Segment footer. Previously, the Segment footer was cached in the `DATA_PAGE` cache. This was incorrect as the footer is metadata, not data. This commit changes it to use the more appropriate `INDEX_PAGE` cache. Additionally, the calculation for `_meta_mem_usage` has been adjusted for better accuracy. The footer size is no longer included in the calculation since it's now managed by the `StoragePageCache`. The memory estimation for column readers is also updated to use `config::max_segment_partial_column_cache_size` for a more realistic value.
This commit modified the caching logic for the Segment footer. Previously, the Segment footer was cached in the `DATA_PAGE` cache. This was incorrect as the footer is metadata, not data. This commit changes it to use the more appropriate `INDEX_PAGE` cache. Additionally, the calculation for `_meta_mem_usage` has been adjusted for better accuracy. The footer size is no longer included in the calculation since it's now managed by the `StoragePageCache`. The memory estimation for column readers is also updated to use `config::max_segment_partial_column_cache_size` for a more realistic value.
This commit modified the caching logic for the Segment footer.
Previously, the Segment footer was cached in the
DATA_PAGE
cache. This was incorrect as the footer is metadata, not data. This commit changes it to use the more appropriateINDEX_PAGE
cache.Additionally, the calculation for
_meta_mem_usage
has been adjusted for better accuracy. The footer size is no longer included in the calculation since it's now managed by theStoragePageCache
. The memory estimation for column readers is also updated to useconfig::max_segment_partial_column_cache_size
for a more realistic value.What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)