Skip to content

Conversation

@cmpatino
Copy link
Collaborator

@cmpatino cmpatino commented Oct 8, 2025

What does this PR do?

This pull request updates the apply_chat_template function in trl/data_utils.py to read the chat_template_kwargs from a dataset.

The version in main ignores the chat_template_kwargs and degrades the performance when setting chat template settings through a dataset. We ran tests using the DPOTrainer on the it-SFT checkpoint from HuggingFaceTB/SmolLM3-3B-checkpoints and saw improvements of ~10 points in AIME25 and GPQA with this fix.

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline,
    Pull Request section?
  • Was this discussed/approved via a GitHub issue? Please add a link
    to it if that's the case.
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@cmpatino cmpatino marked this pull request as ready for review October 8, 2025 14:59
Copy link
Member

@lewtun lewtun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix, LGTM!

I wonder if it makes sense to also include a regression test, but I'll defer to @qgallouedec's judgement on that

Copy link
Collaborator

@edbeeching edbeeching left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for porting this over

Copy link
Member

@qgallouedec qgallouedec left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks!

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@qgallouedec qgallouedec merged commit aab21eb into huggingface:main Oct 10, 2025
8 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants