Skip to content

ZSTD decoder: cocotb utils and decoder testcases, verilog helper modules #3035

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 30 commits into
base: main
Choose a base branch
from
Open

ZSTD decoder: cocotb utils and decoder testcases, verilog helper modules #3035

wants to merge 30 commits into from
+11,326 −91

Conversation

rw1nkler
Copy link
Contributor

@rw1nkler rw1nkler commented Sep 10, 2025 •
edited
Loading

Added functionality:

  • multiple testcases covering the decoding of frames with "compressed" blocks of different type
  • cocotb testing utilities for common operations on XLS channels
  • collection of PnR benchmarks
  • ZSTD frame generator
  • tests of axi ram writer/reader procs as well as memreader & memwriter

Supersedes #2717

rw1nkler and others added 30 commits September 10, 2025 17:03
@rw1nkler
Add a separate test case for each test file
01f521a 
Signed-off-by: Robert Winkler <[email protected]>
@sgizler @rw1nkler
Add "pytype binary, library" hint
e9e5608
@koblonczek @rw1nkler
Add cocotb tests
1267946 
Add cocotb testing utilities:
- XLSStruct for easier handling and serializing/deserializing XLS structs
- XLSChannel that serves as a dummy receiving channel
- XLSMonitor that monitors transactions on an XLS channel
- XLSDriver that can send data on an XLS channel
- LatencyScoreboard that can measure latency between corresponding transactions on input and output buses
- File-backed AXI memory python model

Add cocotb tests for:
- modules/zstd/memory/MemReader
- modules/zstd/memory/AxiWriter
- modules/zstd/memory/MemWriter

Co-authred-by: Pawel Czarnecki <[email protected]>
Co-authred-by: Robert Winkler <[email protected]>
Co-authred-by: Michal Czyz <[email protected]>
Signed-off-by: Michal Czyz <[email protected]>
Signed-off-by: Pawel Czarnecki <[email protected]>
Signed-off-by: Robert Winkler <[email protected]>
Signed-off-by: Krzysztof Obłonczek <[email protected]>
@lpawelcz @rw1nkler
xls/modules/zstd: expose fifo verilog module
fcc3c17 
Signed-off-by: Pawel Czarnecki <[email protected]>
@lpawelcz @rw1nkler
modules/zstd/cocotb: Add ZSTD frame generator library
6937ea0 
This reverts commit 04ad379225b706ddf492d440c673e77348d7a409.
@lpawelcz @rw1nkler
dependency_support: Add zstandard python library
5bdc5e2 
Signed-off-by: Pawel Czarnecki <[email protected]>
@lpawelcz @rw1nkler
modules/zstd: Add verilog simulation of the ZstdDecoder
286f041 
Signed-off-by: Pawel Czarnecki <[email protected]>
@koblonczek @rw1nkler
Use frame generator in tests
548decc 
Co-authored-by: Robert Winkler <[email protected]>
Signed-off-by: Robert Winkler <[email protected]>
Signed-off-by: Krzysztof Obłonczek <[email protected]>
@lpawelcz @rw1nkler
modules/zstd/zstd_dec: Write decoded data to the memory
0deb2be 
ZstdDecoder:
  * Cocotb tests:
    * Move third-party verilog modules (AXI Interconnect) to external directory
    * Replace AXI Interconnect with AXI Crossbar that handles simultaneous AXI Read
      and Write transactions
    * Add reference memory and fill it with expected data for comparison
      against testbench memory at the end of the decoding

Internal-tag: [#67272]
Signed-off-by: Pawel Czarnecki <[email protected]>
@lpawelcz @rw1nkler
modules/zstd/zstd_dec: Remove stream-based output interface
17c4c04 
Adjust cocotb test to:
* Removal of Repacketizer proc
* Removal of stream-based output channels from
  * SequenceExecutor
  * ZstdDecoder

Internal-tag: [#67272]
Signed-off-by: Pawel Czarnecki <[email protected]>
@lpawelcz @rw1nkler
modules/zstd/zstd_dec_cocotb_test: Improve Verilog simulation
d832902 
* Decode multiple ZSTD frames in a single cocotb testbench
* Add one cocotb testbench per type of the ZSTD frames:
  * Frames with RAW blocks only
  * Frames with RLE blocks only
  * Frames with Compressed blocks only (disabled)
  * Frames with mixed blocks (disabled)

Internal-tag: [#67272]
Signed-off-by: Pawel Czarnecki <[email protected]>
@lpawelcz @rw1nkler
modules/zstd/memory/mem_writer: Reduce the amount of random test cases
efb880a 
Caused by timeouts after rebase - looks like regression in the
performance of codegen

Signed-off-by: Pawel Czarnecki <[email protected]>
@lpawelcz @rw1nkler
modules/zstd/memory: Move verilog sources to rtl subdirectory
deb9ccb 
Signed-off-by: Pawel Czarnecki <[email protected]>
@lpawelcz @rw1nkler
xls/modules/zstd/external: Add docstring and license for third party …
b63b1b5 
…sources

Signed-off-by: Pawel Czarnecki <[email protected]>
@lpawelcz @rw1nkler
modules/zstd: Adjust cocotb test to removal of RESET CSR
137e922 
Signed-off-by: Pawel Czarnecki <[email protected]>
@wsipak @lpawelcz @rw1nkler
Style changes
2b221b6 
Improve formatting, wording and fix lint issues

Co-authored-by: Pawel Czarnecki <[email protected]>
Signed-off-by: Wojciech Sipak <[email protected]>
@rw1nkler
Extend cocotb tests of ZSTD decoder
b8fe20f 
Co-authored-by: Pawel Czarnecki <[email protected]>
Co-authored-by: Krzysztof Oblonczek <[email protected]>
Co-authored-by: Wojciech Sipak <[email protected]>
Co-authored-by: Dominik Lau <[email protected]>
Co-authored-by: Szymon Gizler <[email protected]>

Signed-off-by: Robert Winkler <[email protected]>
@rw1nkler
Collect P&R benchmarks
4a1522b 
Signed-off-by: Robert Winkler <[email protected]>
@rw1nkler
Add fixed version of RamDemux together with a test
7402291 
Signed-off-by: Robert Winkler <[email protected]>
@sgizler @rw1nkler
Document decoder testing
a856511
@rw1nkler
dependency_support: Add verilog_axi package
67c7c1f 
Signed-off-by: Robert Winkler <[email protected]>
@rw1nkler
Use bazel package with verilog_axi
fe7671e 
Signed-off-by: Robert Winkler <[email protected]>
@rw1nkler
Downgrade dependencies
7c63750 
Signed-off-by: Robert Winkler <[email protected]>
@rw1nkler
Run tests selectively in GitHub CI
b587217 
Signed-off-by: Robert Winkler <[email protected]>
@rw1nkler
Remove zstandard python library
53c0d85 
Signed-off-by: Robert Winkler <[email protected]>
@rw1nkler
modules: zstd: update readme
b59f420 
Co-autored-by: Dominik Lau <[email protected]>
Signed-off-by: Robert Winkler <[email protected]>
@rw1nkler
Add cocotb check for verifying huffman codes
1200aa1 
Signed-off-by: Robert Winkler <[email protected]>
@rw1nkler
modules: zstd: cocotb: drop unused testcase
7230243 
Co-autored-by: Dominik Lau <[email protected]>
Signed-off-by: Robert Winkler <[email protected]>
@rw1nkler
modules: zstd: disable huffman code builder PnR
258fc49 
Co-autored-by: Dominik Lau <[email protected]>
Signed-off-by: Robert Winkler <[email protected]>
@rw1nkler
modules: zstd: fix formatting
2b62dd2 
Co-autored-by: Dominik Lau <[email protected]>
Signed-off-by: Robert Winkler <[email protected]>
@rw1nkler rw1nkler mentioned this pull request Sep 10, 2025
Closed
@dplassgit dplassgit requested a review from proppy September 10, 2025 18:55
@rw1nkler
Copy link
Contributor Author

@proppy Please let us know if there’s anything else we can do to move this forward

proppy
proppy reviewed Sep 16, 2025
View reviewed changes
dependency_support/com_github_alexforencich_verilog_axi/BUILD.bazel
@@ -0,0 +1,15 @@
# Copyright 2025 The XLS Authors
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think there is a precedent for our project to import SystemVerilog third_party dependency, I'll have to figure out how to make this work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@proppy proppy proppy left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@rw1nkler @proppy @sgizler @koblonczek @lpawelcz @wsipak