Skip to content

Running out of memory with version v0.5.17 #617

@Aleksis99

Description

@Aleksis99

I used v0.5.6 but upgraded to version v0.5.17 since it uses the newer version of unsloth, which is supposed to be much more memory efficient, especially for MOE models and RL. But with the newer version, I get an out-of-memory error for the same model and the same setup with which v0.5.6 worked fine. I'm training Qwen2.5-14B on A100 GPU. Now, even if I try to run Qwen2.5-7B I get out of memory while loading the model, not even when training starts.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions