-
Notifications
You must be signed in to change notification settings - Fork 4.4k
zzip improvements: constant time deletion, unit tests, better internal apis #82675
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
An obvious question is how the magic checksum is guaranteed not to be possible to be generated naturally, or, if it can be, how it is guarded from being considered a "tombstone" (only special case together with empty file, and empty file not generating that checksum?). |
|
2a4e130
to
5d1cc43
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Auto-requesting reviews from non-collaborators: @jbytheway
3f90c96
to
e4da926
Compare
Found one more bug with the test I was mentally stuck on how to write. Yay! |
Don't merge until the ASAN test runs please. |
Followup #83059 |
Summary
None
Purpose of change
Some heftier but needed improvements to the zzip implementation.
Describe the solution
0xDE1337ED
(deleeted) which is interpreted as 'this file was deleted'. When reading a zzip from the beginning, if this magic entry is encountered, the program removes any prior instance of the file from internal datastructures. Subsequent entries however are interpreted normally.mmap_file
(essentially a mock) which supports all the same operations normalmmap_file
supports, but not backed by a real file.mmap_file
does not have a 'path'. This simplified the internal apis greatly, and also resulted in a better separation of concerns for zzip. zzip does not care about filesystem anymore. Responsibility for filesystem operations like moving files is moved to the callers. Instead zzip largely just operates directly on existingmmap_file
s. This worked out nicely.Describe alternatives you've considered
Testing
The new unit tests I added which will run on this PR!
Additional context