Gaps and Obstacles in Developing Collections with Ansible Creator

[Andersson007]

The context: we are now working on the Improve the Partner Certification Experience ANSTRAT-1330 .
To complete the Document the gaps and obstacles on collection workflow story, every community engineering team member independently went through a typical collection development workflow: created a collection with a module from scratch, configured GitHub including CI, published it on Galaxy, etc. trying to use Ansible development tools as much as possible and reflecting their experiences in the Obstacles and issues found with ADT Google doc along the way.

This issue summarizes the challenges and obstacles encountered by the Community Engineering team while developing collections using Ansible Creator. This feedback outlines key areas for improvement within Creator, focusing on actionable solutions to enhance user experience. We've also created an issue with general feedback on using ADT.

Documentation:

• The Creating ansible content using ansible-creator and VS Code Ansible extension page obviously incomplete with large TODO entries
• No documentation on how to run/enable GHA workflows on GitHub.
• The files (e.g. GHA workflow files created) contain no comments whatsoever.
  • Example: in the release workflow file, there’s neither information about what’s going on, how to get the token, where to add it, etc. nor corresponding documentation references.
  • This is relevant to all/most of the created files.
• No information/references in creator’s docs about the files that creator generates and a user needs to fill in before releasing and submitting a collection.
• Misleading guidelines:
  • Example: https://ansible.readthedocs.io/projects/creator/installing/#example After I ran $ ansible-creator init collection andersson007.adt ~/ansible_collections/ it scaffolded it right under ~/ansible_collections/, not under ~/ansible_collections/andersson007/adt as described in the guide. To fix this, you need to create ~/ansible_collections/andersson007/adt first and then pass it as the path, i.e. $ ansible-creator init collection andersson007.adt ~/ansible_collections/andersson007/adt
  • No clear explanation about where to scaffold your collection on the filesystem in general to be able to successfully run ansible-test and pytest afterwards from CLI and VSCode. This should take into account possible conflicts when there are already installed collections in the system.

Usability:

• Scaffolded collections are too verbose by default. It creates way too many files all at once, which is somewhat overwhelming. We spent a lot more time trying to figure out what’s what in the scaffolded collection and then carefully removing resources (and we are people very familiar with collection development) than we would have spent had we just built things out by hand.
  • How would it feel to a person w/o great experience? Probably spooky and confusing.
  • It would probably also look repellant for experts who want to have only files really needed.
  • Examples:
    • You want to create a simple collection containing only modules, but you get more than a hundred files and folders even if you need max a dozen.
    • You get things like extensions/eda resources even if your collection doesn’t have any EDA content.
    • It’s unlikely that a collection in average case will need even a half of the files generated by default.
  • Possible solution: Creator should support a more granular/minimalist, purpose-oriented way of collection initialization, specifically it could be:
    • Start as minimalist as possible (something like ansible-galaxy collection init does + generates README.md).
    • Then use options (that need to be implemented in Creator first) that:
      • Add files for galaxy publishing like galaxy.yml, CODE_OF_CONDUCT, LICENSE, etc.: not everybody is going to publish their collections
      • Add GitHub related files like workflow files: not everyone has their collections on GitHub
      • Generate folders/files exclusively for modules: most collections I saw contain either modules/plugins OR roles, etc.
      • Generate folders/files exclusively for roles
      • Generate folders/files exclusively for EDA
      • Same for different kinds of testing. Separate options to scaffold unit and integration tests, other kinds of tests.
      • Maybe an option to create all possible files (e.g. –all) like it does not by default.
• VSCode initialization should be well tested on a few OS distributions
• Scaffolding should be opinionated in galaxy.yml to use the manifest directive by default and turn what is inside a collection tarball to be an opt-in instead of an opt-out by default. This would prevent things like .ansible and tokens being included in the tarball. We define the common manifest that ‘just works’ and then partners can modify it if they need to add other files inside their collection tarball.
• Creator puts first import before DOCUMENTATION block, which ansible-test sanity –test validate-modules flags as an error. Also the DOCUMENTATION block for the sample module uses ‘name:’ instead of ‘module’ causing more errors.
• Scaffolded examples fail sanity tests.
• The tests/integration/test_integration.py file is really confusing. Do we need Molecule to test our module and collection? It seems like molecule is more for testing roles.
• There's no action entry to run integration tests in .github/workflows/tests.yml, .github/workflows/tests.yml should be updated to run integrations too if they’re present.
• The scaffolded collection basically locks you into a test framework with tox-ansible that seems to be quite complex.