[TASK] Locate Performance Bottlenecks in DeepSeek R1 Decode Subgraph

[zhanghb97]

Deliverables

An experimental report in this issue.

Task Description

• Execute the default f32 inference workflow of DeepSeek and identify the decode subgraph(build/examples/BuddyDeepSeekR1/subgraph0_decode.mlir).

• Following the method used in BuddyNext, split the subgraph into separate MLIR files for detailed analysis.

• Conduct performance evaluation using AOT execution.

• Summarize the performance metrics of each component, for example:

Timeline

2025.10.30 – 2025.10.31
Code Review: Begins 2025.11.1

If completed early, the review process may start ahead of schedule.

[zhanghb97]

Asignee: @GradientRipple

[GradientRipple]

DeepSeek R1 Decoding Subgraph Performance Analysis

Model Layer	Model Structure	Single Module Performance	Single Layer Performance	Stacked Performance	Overall Performance
Input Layer	Embedding	10.7209648	23.25137515		139.1450697
	Positional Encoding (RoPE)	10.7209648
	Layer Norm (RMSNorm)	1.809445550
Core Stacked Layers (28 layers)	MHSA-qkv	1.1718314	6.73889215	98.4338376
	MHSA-core	1.1718314
	MHSA-context	1.1718314
	Layer Norm (RMSNorm)	1.809445550
	FFN	1.4139524
Output Layer	Output Layer	10.7209648	10.7209648

[zhanghb97]

Deliverables

• An experimental report in this issue.

Task Description

• Execute the default f32 inference workflow of DeepSeek and identify the decode subgraph(build/examples/BuddyDeepSeekR1/subgraph0_decode.mlir).
• Manually insert rtclock profiling code. (For example: https://github.com/buddy-compiler/buddy-mlir/blob/main/examples/BuddyTransformer/subgraph0_timed.mlir#L23)
  Profiling should be done at the layer level (e.g., FFN, Attention, Norm, etc.).
  You may use AI assistance for identifying insertion points.
  After inserting the profiling code, simply run ninja buddy-deepseek-r1-run to perform incremental compilation.
• Refer to the profiling method used in:
  https://github.com/buddy-compiler/buddy-mlir/blob/main/examples/BuddyTransformer/build_and_run_timed.sh

Timeline

• Coding phase: 2025.11.17 – 2025.11.18
• Code review: Starts on 2025.11.18

[zhanghb97]

Assignee: @Yssx-g

[yssx-l]

When I conducted separate timing statistics for each layer,it was evident that the Layer Norm module in the ffn layer and the input layer accounted for a significant portion of the time.

Therefore,our optimization efforts should be focused on these two layers.

Next,we can observe that the performance bottlenecks are primarily concentrated in the`linalg.matmul`function and the`ffn`and`qkv`modules. Therefore,we still need to continue optimizing the matmul function,or rather,optimize the other operations within these two layers.

[zhanghb97]

@yssx-l Thanks for your evaluation. Please continue optimizing the matmul for the decode size.