Replace setup with pyproject and fix packaging unintended modules

[albertvillanova]

Replace setup with pyproject, as discussed internally in HF open source team.

This PR replaces the legacy setup.cfg configuration with a modern pyproject.toml setup.

Additionally, this PR fixes a long-standing packaging issue, as unintended modules (examples, scripts and tests) were being included in the distributed package:

In [1]: !pip install trl

In [2]: !ls /usr/local/lib/python3.12/dist-packages/scripts
Out[2]:
add_copyrights.py             generate_zen_image_dataset.py
generate_harmony_dataset.py   generate_zen_multi_image_dataset.py
generate_tiny_models.py       log_example_reports.py
generate_toolcall_dataset.py  log_reports.py
generate_zen_dataset.py       __pycache__

These have now been properly excluded to ensure only the intended source code is packaged.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[qgallouedec]

Suggested change

	[tool.setuptools]
	include-package-data = true
	[tool.setuptools.package-data]
	"trl" = [
	"templates/*.md",
	"accelerate_configs/*.yaml",
	"LICENSE",
	"CONTRIBUTING.md",
	"README.md",
	]
	[tool.setuptools.exclude-package-data]
	"*" = ["__pycache__"]

maybe we can also drop MANIFEST.in and have everything here

[qgallouedec]

Suggested change

	where = ["."]
	where = ["trl*"]

otherwise building will include the examples, which we don't want

[qgallouedec]

and maybe you don't even need exclude = ["tests*"] in this case

[qgallouedec]

I'm realizing that setup.cfg was actually mistakenly including examples and scripts.

[albertvillanova]

I'm realizing that setup.cfg was actually mistakenly including examples and scripts.

Yes, @qgallouedec!!

Initially, my goal was just to replicate the previous behavior while migrating from setup.cfg to pyproject.toml.

But after looking deeper, I noticed that we were previously packaging and distributing unintended modules!

I did some investigation and found that we're not the only ones. For example, see the issue I opened in spanner-graph-notebook (which is preinstalled in all Collab environments):

• Unintended distribution of packages cloudspannerecosystem/spanner-graph-notebook#61

So this PR will actually fix a long-standing packaging issue on our side as well.

[albertvillanova]

At the end I had to keep MANIFEST.in because it is the only way we can exclude tests from being packaged due to default behavior of setuptools backend. See: https://setuptools.pypa.io/en/latest/userguide/miscellaneous.html#controlling-files-in-the-distribution

More precisely, the following files are included in a source distribution by default:
...

• Files that match the following glob patterns: tests/test*.py, test/test*.py;

[qgallouedec]

Ok, this solution works for me. I’ve tested it locally and can confirm that the packaged version contains exactly the required files.