-
Notifications
You must be signed in to change notification settings - Fork 3.2k
Description
What's the problem this feature will solve?
Hello, pip maintainers!
This is (another) proposal for a plugin/extension system for pip. My goals with it are twofold:
- Define a minimal plugin API that allows plugins, but keeps them at an arm's length from
pip's API internals - Define a subcommand API that allows commands under the
pip ext CMDhierarchy, allowing existingpip-tooling (including tools that can't easily be integrated intopipitself or shouldn't be) to provide a better and more consistent UX.
TL;DR: a minimal plugin architecture for pip would allow for better integrations with external tooling, including codebases (e.g. cryptographic codebases with native components) that cannot be easily or desirably vendored into pip itself.
Describe the solution you'd like
I have two things in mind:
- A limited entry points-based plugin architecture for
pip, allowing third-party packages to register plugins. - A
pip ext ...subcommand hierarchy, populated by plugins that register the appropriate entry point, allowing third-party packages to register wholly independent subcommands.
I think both of these would be nice to have, but I think either also makes a good proposal. So I'm curious to hear what others think!
Plugin architecture
My high level idea: pip gains awareness of the pip.plugins entry point group.
For example, a plugin might register as:
[project.entry-points."pip.plugins"]
quux = "sampleproject.plugin"...where plugin is a module object with the following minimal interface:
def plugin_type() -> PluginType:
...and PluginType is:
PluginType = Literal["some"] | Literal["another"]from here, the remaining attributes of the plugin module are determined by PluginType; the intended contract between pip and the plugin is that pip will ignore (and warn on?) any plugin of a type it does not recognize.
(I have ideas for an initial trial-run PluginType, but I want to make sure this basic approach/architecture is amenable before I get into the details there!)
pip ext commands
pip ext subcommands would be a specialization of the above architecture. For example, to register pip ext frobulate, a third-party package might register the following:
[project.entry-points."pip.plugins.ext"]
frobulate = "sampleproject.cli"From here, the cli attribute is expected to be a module with the following attributes:
def description() -> str:
return "a brief oneline description of the command"
def main(args: list[str]) -> None:
......where args is the list of arguments passed after pip ext frobulate.
Under this model, subcommands under pip ext are entirely responsible for their own lifecycle: pip provides no public APIs, no additional context besides args (and os.environ), and the subcommand is expected to handle its own errors.
The description callable is used solely to populate pip ext --list, e.g. to an effect like this (probably more nicely rendered):
$ pip ext --list
plugin description
frobulate a brief oneline description of the command
wangjangle randomly install a python package
compile run pip-compileTimeline
Either (or both) of these would be a significant feature addition to pip. As such, my thinking is that they should go through pip --use-feature like other experimental features, e.g.:
pip --use-feature=plugins
pip --use-feature=extensions
# or combined, no distinction?
pip --use-feature=pluginsFrom there, plugin/pip ext developers could experiment with either feature before they're fully stabilized, without pip committing to an exact API/interface until stabilization.
Alternative Solutions
The minimal alternative here is "do nothing." 🙂
However, for each of the above:
-
pipplugins: expect people to wrappipinstead via its public CLI (or a wrapper likepip-api. Where this isn't sufficiently introspective, users/communities could build their own one-off tools. This is more or less the status quo, and results in a lot of duplication/tools that buggily wrappip(like some of my tools). -
pip extsubcommands: Continue the status quo of people (informally) signaling the adjacency of their tool topipviapip-, e.g.pip-compile,pip-tools,pip-audit, etc. This is workable, although it's not the nicest UX compared to a unified subcommand CLI. Moreover, it can result in weird mismatches (e.g. wherepipuses one Python/environment andpip-compileuses another).
Additional context
A lot of ink has been spilled over plugin architectures before: #3999 and #3121 are probably the oldest and most immediately relevant, but there are references to user requests for various plugin/API architectures scattered throughout the issues. I can try to collate all of them, if desired 🙂
After discussion, if some variant of this proposal is amenable, I (and my colleagues) will happily implement it and provide ongoing maintenance for it (like we do for PyPI, twine, gh-action-pypi-publish, etc.) -- our objective is not to drop a pile of new code on pip and run away, but to work closely with you all and make sure that anything we propose strikes the right balance between value provided to end users, potential new error modes, and your limited maintenance time.
Code of Conduct
- I agree to follow the PSF Code of Conduct.