Skip to content

Merge upstream 05 03 26#17

Open
Mopsgeschwindigkeit wants to merge 634 commits intomasterfrom
merge-upstream-05-03-26
Open

Merge upstream 05 03 26#17
Mopsgeschwindigkeit wants to merge 634 commits intomasterfrom
merge-upstream-05-03-26

Conversation

@Mopsgeschwindigkeit
Copy link

@Mopsgeschwindigkeit Mopsgeschwindigkeit commented Mar 5, 2026

Note: Before submitting this pull request, please review our contributing
guidelines.

Description

Please describe your pull request.

NOTE: All patches should be made against master, not a maintenance branch like
3.1, 2.5, etc. That is unless the bug is already fixed in master, but not in
that version series.

If it fixes a bug or resolves a feature request,
be sure to link to that issue via (Fixes #4412) for example.

@jpesenhofer
removed logs
496901b
@jpesenhofer
put flush in own function
32261ee
@jpesenhofer
flush on revoked partition
6d9211c
@jpesenhofer
removed offsets from cache
67137b3
@jpesenhofer
removed some functions
9dd45b1
@jpesenhofer
added value to check exception on set
9e98cb1
@jpesenhofer
turn off batcher
f43b84c
@jpesenhofer
added batcher again with different mutation approach
f5af306
@jpesenhofer
revival of the mutation buffer
e747b54
@jpesenhofer
fixed some stuff in mutation buffer
b2b0f8f
@jpesenhofer
remove items method
782c2da
@jpesenhofer
added mutatation rows and values in init
64f60d3
@jpesenhofer
fixed list cast
18ceeb4
@jpesenhofer
removed force
2987e47
@jpesenhofer
fixed tests and moved mutation buffer to cache
675afe5
@jpesenhofer
fixed flush mutations
0b7a24c
@jpesenhofer
flush after apply changelog batch
a91b65c
@jpesenhofer
added warning with traceback
63013af
@jpesenhofer
fixed mutation rows
63e8308
@jpesenhofer
changed min flush mutations
f3d9ee1
@jpesenhofer
refactored log
bb66e30
@jpesenhofer
try faster iteritems
c1736e4
@jpesenhofer
don't write offsets in the cache
27977cb
@jpesenhofer
removed set after get
e21b158
@jpesenhofer
fixed iterkeys
3ef552d
@jpesenhofer
fixed iteritems again and added logs for revoking and assigning
cdb7536 
partitions
@jpesenhofer
mitigate race conditions while logging
50974ba
@jpesenhofer
adjusted logging and faster iteritems if value cache is enabled
e3dba58
@jpesenhofer
always flush.
0ebe9c5
@jpesenhofer
move reset of total mutation count up
9744db4
jpesenhofer and others added 29 commits November 27, 2023 10:10
@jpesenhofer
Merge pull request #8 from smaxtec/fix-wrong-return-in-bigtable-get
7b760a4 
fixed return value and added testcase
@jpesenhofer
fixed bug with mutation batcher in get requests if a delete mutation …
5291ac0 
…was found
@jpesenhofer
added partition revoking and log
06b3997
@jpesenhofer
Merge pull request #9 from smaxtec/add_revoke_partitions
e16dcc7 
added partition revoking and log
@jpesenhofer
revoke all tps also the ones that are not active
66df010 
This is just for logging for now.
The partitions that are revoked don't neet to be active.
In order to log it correctly, we need to replace the call to _get_active_changelogtopic_partitions
@jpesenhofer
Merge pull request #10 from smaxtec/fix_revoke_partitions_for_bigtable
2096f40 
revoke all tps also the ones that are not active
Approved by @Mopsgeschwindigkeit
@jpesenhofer
moved logging and added todo
3266ae7
@jpesenhofer
Merge pull request #11 from smaxtec/debounce-logging-in-bigtable
afbca06 
moved logging and added todo
@jpesenhofer
added option and added tests
a6e5e0f 
also set the default value of startup cache ttl to - 1
@jpesenhofer
Merge pull request #12 from smaxtec/add-option-for-endless-startup-cache
5db83dc 
added option and added tests
@jpesenhofer
updated unit tests and bigtable store
4f50676 
This update enhances the startup cache with partitions.
It means, that now with every access to the cache a partition needs to be specified.
This change enables, that the additional memory is freed faster on a rebalance and that
additional partitions are not assigned redundantly
@jpesenhofer
added some logs
32dc779
@jpesenhofer
added gc collect
d8b2cce
@jpesenhofer
introduced locking on rebalance and fixed unit tests
4b7384b
@jpesenhofer
fill startup cache after recovery
199949c 
This ensures that partition assignments are fast adn recovery loops are not possible
Also the startup cache is removed from the apply_changelog_batch function
@jpesenhofer
made some adjustments to the startup cache and fixed tests accordingly
2e7730e
@jpesenhofer
adjusted some more stuff
7b68855
@jpesenhofer
create no existing partitions on assign in value cache
ecb1269
@jpesenhofer
also create empty cache if partition is not in startup cache already
711b615
@jpesenhofer
fixed bug
cb095f5
@jpesenhofer
removed log
2d338e9
@jpesenhofer
Merge pull request #14 from smaxtec/faster-bigtable-startup-revokes
8092517 
Faster bigtable startup revokes
@jpesenhofer
Revert "Merge pull request #14 from smaxtec/faster-bigtable-startup-r…
2661270 
…evokes"

This reverts commit 8092517, reversing
changes made to 5db83dc.
@jpesenhofer
Merge pull request #15 from smaxtec/revert-bigtable-merge
e4d50d6 
Revert "Merge pull request #14 from smaxtec/faster-bigtable-startup-r…
Merge remote-tracking branch 'upstream/master' into merge-upstream-05…
7206241 
…-03-26
Fix bigtable tests and aiokafka producer compatibility
e760baf
Handle aiokafka consumer api_version kwarg compatibility
e2efb82
Use request builders for aiokafka admin/metadata calls
bdf989e
Use Faust assignor when changelog tables are present
ecc7214
@Mopsgeschwindigkeit Mopsgeschwindigkeit changed the base branch from sxmaster to master March 9, 2026 12:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@Mopsgeschwindigkeit @jpesenhofer