-
Couldn't load subscription status.
- Fork 17
Open
Description
CPU work on 4.8GHz
Memory 128G
pulsar2 llm_build --input_path qwen2.5-7b-intrstruct --output_path qwen2.5-7b-w4a16/ --kv_cache_len 1023 -w s4 --hidden_state_type bf16 --prefill_len 128 --chip AX650
Config(
model_name='qwen2.5-7b',
model_type='qwen2',
num_hidden_layers=28,
num_attention_heads=28,
num_key_value_heads=4,
hidden_size=3584,
head_dim=0,
intermediate_size=18944,
vocab_size=152064,
rope_theta=1000000.0,
max_position_embeddings=32768,
rope_partial_factor=1.0,
rms_norm_eps=1e-06,
norm_type='rms_norm',
hidden_act='silu',
hidden_act_param=0.03,
scale_depth=1.4,
scale_emb=1,
dim_model_base=256,
origin_model_type='',
quant=False,
quant_sym=False,
quant_bits=4,
quant_group_size=128
)
2025-02-11 12:47:06.984 | SUCCESS | yamain.command.llm_build:llm_build:134 - prepare llm model done!
building llm decode layers ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 28/28 1:59:42
building llm post layer ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1/1 0:05:03
btw: what's the BSP_MSP_DIR?
Metadata
Metadata
Assignees
Labels
No labels