Skip to content

Conversation

akrieger
Copy link
Member

Summary

None

Purpose of change

Some heftier but needed improvements to the zzip implementation.

  • deletion requires rewriting the entire zzip to remove one file.
  • no unit tests.
  • the zzip api was... acceptable. But it could be better.

Describe the solution

  • Deleting a file now writes a 'tombstone' which is an empty file entry with a magic constant checksum value 0xDE1337ED (deleeted) which is interpreted as 'this file was deleted'. When reading a zzip from the beginning, if this magic entry is encountered, the program removes any prior instance of the file from internal datastructures. Subsequent entries however are interpreted normally.
  • To validate the implementation, since it is new and moderately invasive, I added unit tests for zzip. Specifically, an in-memory 'malloc' based implementation of mmap_file (essentially a mock) which supports all the same operations normal mmap_file supports, but not backed by a real file.
  • To make zzip unit testable, I had to refactor the api to remove the concept of path from zzip, because malloc based mmap_file does not have a 'path'. This simplified the internal apis greatly, and also resulted in a better separation of concerns for zzip. zzip does not care about filesystem anymore. Responsibility for filesystem operations like moving files is moved to the callers. Instead zzip largely just operates directly on existing mmap_files. This worked out nicely.

Describe alternatives you've considered

Testing

The new unit tests I added which will run on this PR!

Additional context

@github-actions github-actions bot added Map / Mapgen Overmap, Mapgen, Map extras, Map display Code: Tests Measurement, self-control, statistics, balancing. [C++] Changes (can be) made in C++. Previously named `Code` Character / World Generation Issues and enhancements concerning stages of creating a character or a world Code: Tooling Tooling that is not part of the main game but is part of the repo. labels Aug 30, 2025
@github-actions github-actions bot added json-styled JSON lint passed, label assigned by github actions astyled astyled PR, label is assigned by github actions labels Aug 30, 2025
@PatrikLundell
Copy link
Contributor

An obvious question is how the magic checksum is guaranteed not to be possible to be generated naturally, or, if it can be, how it is guarded from being considered a "tombstone" (only special case together with empty file, and empty file not generating that checksum?).

@akrieger
Copy link
Member Author

akrieger commented Aug 30, 2025

Well, its a combination of magic checksum and zero length file that make up the tombstone. The natural checksum for the empty file is not the magic checksum. yeah what you said.

@akrieger akrieger force-pushed the delteeted branch 2 times, most recently from 2a4e130 to 5d1cc43 Compare September 2, 2025 05:53
@github-actions github-actions bot added the BasicBuildPassed This PR builds correctly, label assigned by github actions label Sep 2, 2025
@github-actions github-actions bot removed the BasicBuildPassed This PR builds correctly, label assigned by github actions label Sep 11, 2025
@akrieger akrieger marked this pull request as ready for review September 11, 2025 18:52
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Auto-requesting reviews from non-collaborators: @jbytheway

@akrieger akrieger marked this pull request as draft September 11, 2025 23:07
@github-actions github-actions bot added the BasicBuildPassed This PR builds correctly, label assigned by github actions label Sep 12, 2025
@akrieger akrieger force-pushed the delteeted branch 2 times, most recently from 3f90c96 to e4da926 Compare September 22, 2025 00:54
@akrieger akrieger marked this pull request as ready for review September 22, 2025 00:55
@akrieger
Copy link
Member Author

Found one more bug with the test I was mentally stuck on how to write. Yay!

@github-actions github-actions bot removed the BasicBuildPassed This PR builds correctly, label assigned by github actions label Sep 24, 2025
@github-actions github-actions bot added BasicBuildPassed This PR builds correctly, label assigned by github actions and removed BasicBuildPassed This PR builds correctly, label assigned by github actions labels Sep 25, 2025
@akrieger
Copy link
Member Author

Don't merge until the ASAN test runs please.

@github-actions github-actions bot added the BasicBuildPassed This PR builds correctly, label assigned by github actions label Sep 25, 2025
@Maleclypse Maleclypse merged commit b85a5a5 into CleverRaven:master Sep 25, 2025
39 of 58 checks passed
@akrieger akrieger deleted the delteeted branch September 26, 2025 03:38
@akrieger
Copy link
Member Author

Followup #83059

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
astyled astyled PR, label is assigned by github actions BasicBuildPassed This PR builds correctly, label assigned by github actions [C++] Changes (can be) made in C++. Previously named `Code` Character / World Generation Issues and enhancements concerning stages of creating a character or a world Code: Tests Measurement, self-control, statistics, balancing. Code: Tooling Tooling that is not part of the main game but is part of the repo. json-styled JSON lint passed, label assigned by github actions Map / Mapgen Overmap, Mapgen, Map extras, Map display
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants