Skip to content

Fix CI ImportError: FlashAttention2 and decorator order for all parameterized tests #4176

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Oct 1, 2025
Merged

Fix CI ImportError: FlashAttention2 and decorator order for all parameterized tests #4176

Changes from 9 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
2837bd9
Run CI slow tests on push to debug
albertvillanova Sep 30, 2025
24df301
Install flash-attn for CI slow tests
albertvillanova Sep 30, 2025
94049d1
Install flash-attn for CI slow tests on multi GPU
albertvillanova Sep 30, 2025
cd07d0a
Do not run multi GPU tests for debugging
albertvillanova Sep 30, 2025
23e9dbc
Skip test if GPUs are not Ampere or newer
albertvillanova Sep 30, 2025
5a298fe
Fix order of test decorators
albertvillanova Sep 30, 2025
cf2dd02
Fix
albertvillanova Sep 30, 2025
a856c5e
Fix
albertvillanova Sep 30, 2025
0aaa2ed
Revert debug changes
albertvillanova Sep 30, 2025
fb42f81
Fix decorator order for all parameterized tests
albertvillanova Sep 30, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions tests/slow/test_grpo_slow.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -208,14 +208,14 @@ def test_training_with_transformers_paged(self, model_name):

release_memory(model, trainer)

@require_flash_attn
@require_bitsandbytes
@require_peft
@parameterized.expand(
[
("HuggingFaceTB/SmolVLM-Instruct",), # Only test the smaller model to avoid OOM
]
)
@require_flash_attn
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok so @require_X must be after parametrized. For consistency, can you also modify

  • test_training_vlm_and_vllm (it's deactivated but still, it's good to have it right)
  • test_training_with_judge
  • test_training_with_transformers_paged
  • test_xpo_trainer_judge_training

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, also for:

  • test_dpo_trainer_with_liger

@require_bitsandbytes
@require_peft
def test_vlm_training(self, model_name):
"""
Test VLM training with aggressive memory optimization.
Expand Down