From 31aa16e0e4b5d844c3bf847aa604f9339ceb36ff Mon Sep 17 00:00:00 2001 From: Tim Meusel Date: Sun, 16 Mar 2025 19:39:18 +0100 Subject: [PATCH 1/2] Add ToC --- README.md | 54 ++++++++++++++++++++++++++++++++---------------------- 1 file changed, 32 insertions(+), 22 deletions(-) diff --git a/README.md b/README.md index 414369f2..2a20800c 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,4 @@ -ModuleSync -=========== +# ModuleSync [![License](https://img.shields.io/github/license/voxpupuli/modulesync.svg)](https://github.com/voxpupuli/modulesync/blob/master/LICENSE) [![Test](https://github.com/voxpupuli/modulesync/actions/workflows/ci.yml/badge.svg)](https://github.com/voxpupuli/modulesync/actions/workflows/ci.yml) @@ -9,26 +8,40 @@ ModuleSync [![RubyGem Downloads](https://img.shields.io/gem/dt/modulesync.svg)](https://rubygems.org/gems/modulesync) [![Donated by Puppet Inc](https://img.shields.io/badge/donated%20by-Puppet%20Inc-fb7047.svg)](#transfer-notice) -Table of Contents ------------------ - -1. [Usage TLDR](#usage-tldr) -2. [Overview](#overview) -3. [How it works](#how-it-works) -4. [Installing](#installing) -5. [Workflow](#workflow) -6. [The Templates](#the-templates) - -Usage TLDR ----------- +## Table of Contents + +* [Usage TLDR](#usage-tldr) +* [Overview](#overview) +* [How it works](#how-it-works) + * [Default mode](#default-mode) +* [Installing](#installing) +* [Workflow](#workflow) + * [Default mode](#default-mode) + * [Using Forks and Non-master branches](#using-forks-and-non-master-branches) + * [Dry run](#dry-run-1) + * [Damage mode](#damage-mode-1) + * [Configuring ModuleSync defaults](#configuring-modulesync-defaults) + * [Filtering Repositories](#filtering-repositories) + * [Pushing to a different remote branch](#pushing-to-a-different-remote-branch) + * [Automating updates](#automating-updates-1) + * [Updating metadata.json](#updating-metadatajson) + * [Tagging repositories](#tagging-repositories) + * [Setting the tag pattern](#setting-the-tag-pattern) + * [Updating the CHANGELOG](#updating-the-changelog) + * [Working with templates](#working-with-templates) +* [The Templates](#the-templates) +* [Transfer Notice](#transfer-notice) +* [License](#license) +* [Release information](#release-information) + +## Usage TLDR ``` gem install modulesync msync --help ``` -Overview --------- +## Overview ModuleSync was written as a simple script with ERB templates to help the Puppet Labs module engineers manage the zoo of Puppet modules on GitHub, and @@ -58,8 +71,7 @@ Another advantage of ModuleSync is the ability to run in no-op mode, which makes local changes and shows the diffs, but does not make permanent changes in the remote repository. -How It Works ------------- +## How It Works ModuleSync is a gem that uses the GitHub workflow to clone, update, and push module repositories. It expects to be activated from a directory containing @@ -88,8 +100,7 @@ branch you're working on or the remote to clone from and push to. You can also define these parameters in a file named `modulesync.yml` in the configuration directory. -Installing ----------- +## Installing ``` gem install modulesync @@ -102,8 +113,7 @@ gem build modulesync.gemspec gem install modulesync-*.gem ``` -Workflow --------- +## Workflow ### Default mode From a7bfa548cbb38552f151b870e59d45c08b234af5 Mon Sep 17 00:00:00 2001 From: Tim Meusel Date: Sun, 16 Mar 2025 19:47:27 +0100 Subject: [PATCH 2/2] reformart README.md --- README.md | 226 +++++++++++++++++++++--------------------------------- 1 file changed, 89 insertions(+), 137 deletions(-) diff --git a/README.md b/README.md index 2a20800c..01073127 100644 --- a/README.md +++ b/README.md @@ -43,62 +43,39 @@ msync --help ## Overview -ModuleSync was written as a simple script with ERB templates to help the -Puppet Labs module engineers manage the zoo of Puppet modules on GitHub, and -has now been restructured and generalized to be used within other -organizations. Puppet modules within an organization tend to have a number of -meta-files that are identical or very similar between modules, such as the -`Gemfile`, `.travis.yml`, `.gitignore`, or `spec_helper.rb`. If a file needs to -change in one module, it likely needs to change in the same way in every other -module that the organization manages. - -One approach to this problem is to use sed in a bash for loop on the modules to -make a single change across every module. This approach falls short if there is -a single file that is purposefully different than the others, but still needs -to be managed. Moreover, this approach does not help if two files are -structured differently but need to be changed with the same meaning; for -instance, if the .travis.yml of one module uses a list of environments to -include, and another uses a matrix of includes with a list of excludes, adding -a test environment to both modules will require entirely different approaches. - -ModuleSync provides the advantage of defining a standard template for each -file to follow, so it becomes clear what a file is supposed to look like. Two -files with the same semantics should also have the same syntax. A difference -between two files should have clear reasons, and old cruft should not be left -in files of one module as the files of another module march forward. - -Another advantage of ModuleSync is the ability to run in no-op mode, which -makes local changes and shows the diffs, but does not make permanent changes in -the remote repository. +ModuleSync was written as a simple script with ERB templates to help the Puppet Labs module engineers manage the zoo of Puppet modules on GitHub, and has now been restructured and generalized to be used within other organizations. +Puppet modules within an organization tend to have a number of meta-files that are identical or very similar between modules, such as the `Gemfile`, `.travis.yml`, `.gitignore`, or `spec_helper.rb`. +If a file needs to change in one module, it likely needs to change in the same way in every other module that the organization manages. + +One approach to this problem is to use sed in a bash for loop on the modules to make a single change across every module. +This approach falls short if there is a single file that is purposefully different than the others, but still needs to be managed. +Moreover, this approach does not help if two files are structured differently but need to be changed with the same meaning; +for instance, if the .travis.yml of one module uses a list of environments to include, and another uses a matrix of includes with a list of excludes, adding a test environment to both modules will require entirely different approaches. + +ModuleSync provides the advantage of defining a standard template for each file to follow, so it becomes clear what a file is supposed to look like. +Two files with the same semantics should also have the same syntax. +A difference between two files should have clear reasons, and old cruft should not be left in files of one module as the files of another module march forward. + +Another advantage of ModuleSync is the ability to run in no-op mode, which makes local changes and shows the diffs, but does not make permanent changes in the remote repository. ## How It Works -ModuleSync is a gem that uses the GitHub workflow to clone, update, and push module -repositories. It expects to be activated from a directory containing -configuration for modulesync and the modules, or you can pass it the location -of this configuration directory. [The configuration for the Puppet Labs -modules](https://github.com/puppetlabs/modulesync_configs), can be used as an -example for your own configuration. The configuration directory contains a -directory called moduleroot which mirrors the structure of a module. The files -in the moduleroot are ERB templates, and MUST be named after the target file, -with `.erb.` appended. The templates are -rendered using values from a file called `config_defaults.yml` in the root (not -moduleroot) of the configuration directory. The default values can be -overridden or extended by adding a file called `.sync.yml` to the module itself. -This allows us to, for example, have a set of "required" gems that are added -to all Gemfiles, and a set of "optional" gems that a single module might add. - -Within the templates, values can be accessed in the `@configs` hash, which is -merged from the values under the keys `:global` and the target file name (no -`.erb` suffix). - -The list of modules to manage is in `managed_modules.yml` in the configuration -directory. This lists just the names of the modules to be managed. - -ModuleSync can be called from the command line with parameters to change the -branch you're working on or the remote to clone from and push to. You can also -define these parameters in a file named `modulesync.yml` in the configuration -directory. +ModuleSync is a gem that uses the GitHub workflow to clone, update, and push module repositories. +It expects to be activated from a directory containing configuration for modulesync and the modules, or you can pass it the location of this configuration directory. +[The configuration for the Puppet Labs modules](https://github.com/puppetlabs/modulesync_configs), can be used as an example for your own configuration. +The configuration directory contains a directory called moduleroot which mirrors the structure of a module. +The files in the moduleroot are ERB templates, and MUST be named after the target file, with `.erb.` appended. +The templates are rendered using values from a file called `config_defaults.yml` in the root (not moduleroot) of the configuration directory. +The default values can be overridden or extended by adding a file called `.sync.yml` to the module itself. +This allows us to, for example, have a set of "required" gems that are added to all Gemfiles, and a set of "optional" gems that a single module might add. + +Within the templates, values can be accessed in the `@configs` hash, which is merged from the values under the keys `:global` and the target file name (no `.erb` suffix). + +The list of modules to manage is in `managed_modules.yml` in the configuration directory. +This lists just the names of the modules to be managed. + +ModuleSync can be called from the command line with parameters to change the branch you're working on or the remote to clone from and push to. +You can also define these parameters in a file named `modulesync.yml` in the configuration directory. ## Installing @@ -117,25 +94,28 @@ gem install modulesync-*.gem ### Default mode -With no additional arguments, ModuleSync clones modules from the puppetlabs -github organization and pushes to the master branch. +With no additional arguments, ModuleSync clones modules from the puppetlabs github organization and pushes to the master branch. #### Make changes -Make changes to a file in the moduleroot. For sanity's sake you should commit -and push these changes, but in this mode the update will be rendered from the -state of the files locally. +Make changes to a file in the moduleroot. +For sanity's sake you should commit and push these changes, but in this mode the update will be rendered from the state of the files locally. #### Dry-run -Do a dry-run to see what files will be changed, added and removed. This clones -the modules to `modules/-` in the current working -directory, or if the modules are already cloned, does an effective `git fetch -origin; git checkout master; git reset --hard origin/master` on the modules. -Don't run modulesync if the current working directory contains a modules/ -directory with changes you want to keep. The dry-run makes local changes there, -but does not commit or push changes. It is still destructive in that it -overwrites local changes. +Do a dry-run to see what files will be changed, added and removed. +This clones the modules to `modules/-` in the current working directory, or if the modules are already cloned, does an effective + +``` +git fetch origin +git checkout master +git reset --hard origin/master +``` + +on the modules. +Don't run modulesync if the current working directory contains a `modules/` directory with changes you want to keep. +The dry-run makes local changes there, but does not commit or push changes. +It is still destructive in that it overwrites local changes. ``` msync update --noop @@ -143,11 +123,10 @@ msync update --noop #### Offline support -The --offline flag was added to allow a user to disable git support within -msync. One reason for this is because the user wants to control git commands -external to msync. Note, when using this command, msync assumes you have -create the folder structure and git repositories correctly. If not, msync will -fail to update correctly. +The --offline flag was added to allow a user to disable git support within msync. +One reason for this is because the user wants to control git commands external to msync. +Note, when using this command, msync assumes you have create the folder structure and git repositories correctly. +If not, msync will fail to update correctly. ``` msync update --offline @@ -155,9 +134,8 @@ msync update --offline #### Damage mode -Make changes for real and push them back to master. This operates on the -pre-cloned modules from the dry-run or clones them fresh if the modules aren't -found. +Make changes for real and push them back to master. +This operates on the pre-cloned modules from the dry-run or clones them fresh if the modules aren't found. ``` msync update -m "Commit message" @@ -169,8 +147,8 @@ Amend the commit if changes are needed. msync update --amend ``` -For most workflows you will need to force-push an amended commit. Not required -for gerrit. +For most workflows you will need to force-push an amended commit. +Not required for gerrit. ``` msync update --amend --force @@ -178,17 +156,14 @@ msync update --amend --force #### Automating Updates -You can install a pre-push git hook to automatically clone, update, and push -modules upon pushing changes to the configuration directory. This does not -include a noop mode. +You can install a pre-push git hook to automatically clone, update, and push modules upon pushing changes to the configuration directory. +This does not include a noop mode. ``` msync hook activate ``` -If you have activated the hook but want to make changes to the configuration -directory (such as changes to `managed_modules.yml` or `modulesync.yml`) without -touching the modules, you can deactivate the hook. +If you have activated the hook but want to make changes to the configuration directory (such as changes to `managed_modules.yml` or `modulesync.yml`) without touching the modules, you can deactivate the hook. ``` msync hook deactivate @@ -196,21 +171,16 @@ msync hook deactivate #### Submitting PRs/MRs to GitHub or GitLab -You can have modulesync submit Pull Requests on GitHub or Merge Requests on -GitLab automatically with the `--pr` CLI option. +You can have modulesync submit Pull Requests on GitHub or Merge Requests on GitLab automatically with the `--pr` CLI option. ``` msync update --pr ``` -In order for GitHub PRs or GitLab MRs to work, you must either provide -the `GITHUB_TOKEN` or `GITLAB_TOKEN` environment variables, -or set them per repository in `managed_modules.yml`, using the `github` or -`gitlab` keys respectively. +In order for GitHub PRs or GitLab MRs to work, you must either provide the `GITHUB_TOKEN` or `GITLAB_TOKEN` environment variables. +You can also set them per repository in `managed_modules.yml`, using the `github` or `gitlab` keys respectively. -For GitHub Enterprise and self-hosted GitLab instances you also need to set the -`GITHUB_BASE_URL` or `GITLAB_BASE_URL` environment variables, or specify the -`base_url` parameter in `modulesync.yml`: +For GitHub Enterprise and self-hosted GitLab instances you also need to set the `GITHUB_BASE_URL` or `GITLAB_BASE_URL` environment variables, or specify the `base_url` parameter in `modulesync.yml`: ```yaml --- @@ -227,13 +197,9 @@ repo2: Then: -* Set the PR/MR title with `--pr-title` or in `modulesync.yml` with the - `pr_title` attribute. -* Assign labels to the PR/MR with `--pr-labels` or in `modulesync.yml` with - the `pr_labels` attribute. **NOTE:** `pr_labels` should be a list. When - using the `--pr-labels` CLI option, you should use a comma separated list. -* Set the target branch with `--pr-target-branch` or in `modulesync.yml` with - the `pr_target_branch` attribute. +* Set the PR/MR title with `--pr-title` or in `modulesync.yml` with the `pr_title` attribute. +* Assign labels to the PR/MR with `--pr-labels` or in `modulesync.yml` with the `pr_labels` attribute. **NOTE:** `pr_labels` should be a list. When using the `--pr-labels` CLI option, you should use a comma separated list. +* Set the target branch with `--pr-target-branch` or in `modulesync.yml` with the `pr_target_branch` attribute. More details for GitHub: @@ -241,19 +207,15 @@ More details for GitHub: ### Using Forks and Non-master branches -The default functionality is to run ModuleSync on the puppetlabs modules, but -you can use this on your own organization's modules. This functionality also -applies if you want to work on a fork of the puppetlabs modules or work on a -non-master branch of any organization's modules. ModuleSync does not support -cloning from one remote and pushing to another, you are expected to fork -manually. It does not support automating pull requests. +The default functionality is to run ModuleSync on the puppetlabs modules, but you can use this on your own organization's modules. +This functionality also applies if you want to work on a fork of the puppetlabs modules or work on a non-master branch of any organization's modules. +ModuleSync does not support cloning from one remote and pushing to another, you are expected to fork manually. +It does not support automating pull requests. #### Dry-run -If you dry-run before doing the live update, you need to specify what namespace -to clone from because the live update will not re-clone if the modules are -already cloned. The namespace should be your fork, not the upstream module (if -working on a fork). +If you dry-run before doing the live update, you need to specify what namespace to clone from because the live update will not re-clone if the modules are already cloned. +The namespace should be your fork, not the upstream module (if working on a fork). ``` msync update -n puppetlabs --noop @@ -261,10 +223,9 @@ msync update -n puppetlabs --noop #### Damage mode -You don't technically need to specify the namespace if the modules are already -cloned from the dry-run, but it doesn't hurt. You do need to specify the -namespace if the modules are not pre-cloned. You need to specify a branch to -push to if you are not pushing to master. +You don't technically need to specify the namespace if the modules are already cloned from the dry-run, but it doesn't hurt. +You do need to specify the namespace if the modules are not pre-cloned. +You need to specify a branch to push to if you are not pushing to master. ``` msync update -n puppetlabs -b sync_branch -m "Commit message" @@ -272,11 +233,9 @@ msync update -n puppetlabs -b sync_branch -m "Commit message" #### Configuring ModuleSync defaults -If you're not using the puppetlabs modules or only ever pushing to a fork of -them, then specifying the namespace and branch every time you use ModuleSync -probably seems excessive. You can create a file called modulesync.yml in the -configuration directory that provides these arguments automatically. This file -has a form such as: +If you're not using the puppetlabs modules or only ever pushing to a fork of them, then specifying the namespace and branch every time you use ModuleSync probably seems excessive. +You can create a file called modulesync.yml in the configuration directory that provides these arguments automatically. +This file has a form such as: ```yaml --- @@ -294,9 +253,7 @@ msync update -m "Commit message" Available parameters for modulesync.yml * `git_base` : The default URL to git clone from (Default: 'git@github.com:') -* `namespace` : Namespace of the projects to manage (Default: 'puppetlabs'). - This value can be overridden in the module name (e.g. 'namespace/mod') or by - using the `namespace` key for the module in `managed_modules.yml`. +* `namespace` : Namespace of the projects to manage (Default: 'puppetlabs'). This value can be overridden in the module name (e.g. 'namespace/mod') or by using the `namespace` key for the module in `managed_modules.yml`. * `branch` : Branch to push to (Default: 'master') * `remote_branch` : Remote branch to push to (Default: Same value as branch) * `message` : Commit message to apply to updated modules. @@ -367,10 +324,9 @@ msync update -r master_new -m "Commit message" #### Automating updates -If you install a git hook, you need to tell it what remote and branch to push -to. This may not work properly if you already have the modules cloned from a -different remote. The hook will also look in modulesync.yml for default -arguments. +If you install a git hook, you need to tell it what remote and branch to push to. +This may not work properly if you already have the modules cloned from a different remote. +The hook will also look in modulesync.yml for default arguments. ``` msync hook activate -n puppetlabs -b sync_branch @@ -378,8 +334,7 @@ msync hook activate -n puppetlabs -b sync_branch #### Updating metadata.json -Modulesync can optionally bump the minor version in `metadata.json` for each -modified modules if you add the `--bump` flag to the command line: +Modulesync can optionally bump the minor version in `metadata.json` for each modified modules if you add the `--bump` flag to the command line: ``` msync update -m "Commit message" --bump @@ -387,8 +342,7 @@ msync update -m "Commit message" --bump #### Tagging repositories -If you wish to tag the modified repositories with the newly bumped version, -you can do so by using the `--tag` flag: +If you wish to tag the modified repositories with the newly bumped version, you can do so by using the `--tag` flag: ``` msync update -m "Commit message" --bump --tag @@ -396,8 +350,7 @@ msync update -m "Commit message" --bump --tag #### Setting the tag pattern -You can also set the format of the tag to be used (`printf`-formatted) -by setting the `tag_pattern` option: +You can also set the format of the tag to be used (`printf`-formatted) by setting the `tag_pattern` option: ``` msync update -m "Commit message" --bump --tag --tag_pattern 'v%s' @@ -407,8 +360,7 @@ The default for the tag pattern is `%s`. #### Updating the CHANGELOG -When bumping the version in `metadata.json`, modulesync can let you -updating `CHANGELOG.md` in each modified repository. +When bumping the version in `metadata.json`, modulesync can let you updating `CHANGELOG.md` in each modified repository. This is one by using the `--changelog` flag: @@ -416,15 +368,15 @@ This is one by using the `--changelog` flag: msync update -m "Commit message" --bump --changelog ``` -This flag will cause the `CHANGELOG.md` file to be updated with the -current date, bumped (minor) version, and commit message. +This flag will cause the `CHANGELOG.md` file to be updated with the current date, bumped (minor) version, and commit message. If `CHANGELOG.md` is absent in the repository, nothing will happen. #### Working with templates -As mentioned, files in the moduleroot directory must be ERB templates (they must have an .erb extension, or they will be ignored). These files have direct access to @configs hash, which gets values from config_defaults.yml file and from the module being processed: +As mentioned, files in the moduleroot directory must be ERB templates (they must have an .erb extension, or they will be ignored). +These files have direct access to @configs hash, which gets values from config_defaults.yml file and from the module being processed: ```erb <%= @configs[:git_base] %> @@ -432,8 +384,8 @@ As mentioned, files in the moduleroot directory must be ERB templates (they must <%= @configs[:puppet_module] %> ``` -Alternatively some meta data is passed to the template. This will allow you to add custom Ruby extensions inside the -template, reading other files from the module, to make the template system more adaptive. +Alternatively some meta data is passed to the template. +This will allow you to add custom Ruby extensions inside the template, reading other files from the module, to make the template system more adaptive. ```erb module: <%= @metadata[:module_name] %>