Why does ds_read_addtid_b32 require a following buffer_wbinvl1_vol

[Greensaid]

for an mir like below:

name: addtid
body: |
  bb.0:
    $m0 = S_MOV_B32 0
    $vgpr0 = DS_READ_ADDTID_B32 0, 0, implicit $m0, implicit $exec
    S_ENDPGM 0

if we run with the command: llc -march=amdgcn -mcpu=gfx900 -run-pass si-memory-legalizer,
the result will contain a buffer_wbinvl1_vol instruction after ds_read_addtid_b32.
I have found the handling logic in the pass: SIMemoryLegalizer , but I was wondering why there is such a logic not only for vmem instructions, but also for ds instructions. Will ds_read_addtid_b32 (and also other ds instructions?) affect L1 cache?

[ppanchad-amd]

Hi @Greensaid. Internal ticket has been created to assist with your issue. Thanks

[schung-amd]

Hi @Greensaid, I'm trying to find the answer to this but it isn't documented anywhere I've seen thus far. From internal discussions we don't expect the ds instructions to touch the L1 cache in general, but we must be inserting buffer_wbinvl1_vol for a reason here. Notably this is target-specific, I see the same thing with -mcpu=gfx90a but not -mcpu=gfx942, so this is likely due to some detail in the memory model of those architectures. Will update when I have more clarity on this.

[Greensaid]

Hi @schung-amd, with -mcpu=gfx942, we will see there is a buffer_inval 17 after ds_read_addtid_b32. This 17 should be equal to sc0 sc1，which means it will invalidate both L1 and L2 cache lines. Furthermore, with -mcpu=gfx1010/gfx1100, ds_read_addtid_b32 will be followed by buffer_gl0_inv and buffer_gl1_inv insts, which invalidate L0 and L1 cache on GFX10 targets.
Therefore, I think this is a general feature for AMD targets based on the code in the SIMemoryLegalizer pass(but different target may end up with different cache-invalidating insts). Is this behavior really due to AMDGPU memory model, or just the code structure forces all DS insts go through cache-handling process as VMEM insts? Based on the code, since ds_read_addtid_b32 has a default SIMemOpInfo (it has no memory operands), it will always be followed by cache-invalidating insts.

[schung-amd]

Thanks for the info. As you say it's unclear why this is being done, still reaching out internally for answers.