Skip to content

Conversation

nfawbert
Copy link
Contributor

No description provided.

EmlynK and others added 30 commits January 20, 2022 10:13
…cs with backup (#94)

* Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs).

* Set up nightly backup crons in separate files in /etc/cron.d

* Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms.

* Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump.

* Use Ansible modules to look up RDS host and to copy the nightly backup into place.

* Delegate PATH setup in db backup cron to localhost.

* Shell bad. Command good. But makes it convoluted. Oh well.

* Used wrong database name in source database copy.

* Try and disable the ReplaceUnhealthy auto scale process during syncs.

* Fix Drupal cron roles when deferring to deploy server.
* Including an ami.yml for packing AMIs on build finale.

* New api_call role, focused on GitLab for now.

* Optionally trigger an infra build with an API call.
…104)

* Making the MySQL dump command for routine back-ups less aggressive.

* Making max_allowed_packets a variable we can set.

* Adding dump command to a re-usable central var.

* Renaming _init var so it's easier to distinguish in the code.
* Making the MySQL dump command for routine back-ups less aggressive.

* Making max_allowed_packets a variable we can set.

* Adding dump command to a re-usable central var.

* Renaming _init var so it's easier to distinguish in the code.

* Defaults file must be the first param for mysqldump.
* Don't recurse through site directory when setting permissions during config_generate step.

* Update drupal7 config_generate perms update task and use true/false instead of yes/no.
* Pass -l option in drush commands to specify site name, which should match the site folder name.

* Move -l option in drush commands to the first option instead of the last.
* Attempt to fix static credentials handling when deploying to an ASG.

* Include build_type in static password file location.

* Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now.

* Add a couple of comments to legacy handling of static credentials.
* Add a files_sync role.

* Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role.

* Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself.

* Use rsync instead of copy when syncing files to target server as copy is so slow.

* Ensure file sync tasks are run only once.
* Add a drupal_sync_tasks role to run Drupal specific tasks during syncs.

* Move sync database_apply files into tasks subdirectory.

* Woops, used bad role names in sync database update roles.

* Remove feature reverting from drupal7 database_apply sync role.
* Fixing GRANT query for MySQL > 8.0.

* Create what though?
* Adding SquashFS option to syncing.

* Make 'tarball' the default mount type so nothing existing breaks.

* Altering deploy_path so we can build in another location for SquashFS builds.

* Ensuring the build dir exists if doing a SquashFS build.

* Force symlink creation as the deploy dir may not exist yet.

* Running mount commands with sudo.

* Tweaking link command and destination for live links.

* Slight bug in link path handling for SquashFS.

* Checking for existing mount and using remount operation.

* Stop ce-deploy trying to delete from read-only SquashFS mount.

* Formatting error, these special jinja2 things are not filters - no spaces.

* Reloading services to make sure mounting doesn't fail.

* Picking more sensible loop var name.

* Moving to shell for mount check and fixing jinja2 filter names.

* Trying with the posix mount module instead of command.

* Working through user/sudo issues.

* Altering symlink handling slightly so we have the deploy directory always set.

* Simplifying 'when' checks.
* Use a different variable for config imports during a sync so they can be optional.

* Actually, no. Use a variable to determine if config should be imported during a sync.

* Fix some logic with config import during syncs and add a comment to explain when the task runs.
* Adding SquashFS option to syncing.

* Make 'tarball' the default mount type so nothing existing breaks.

* Altering deploy_path so we can build in another location for SquashFS builds.

* Ensuring the build dir exists if doing a SquashFS build.

* Force symlink creation as the deploy dir may not exist yet.

* Running mount commands with sudo.

* Tweaking link command and destination for live links.

* Slight bug in link path handling for SquashFS.

* Checking for existing mount and using remount operation.

* Stop ce-deploy trying to delete from read-only SquashFS mount.

* Formatting error, these special jinja2 things are not filters - no spaces.

* Reloading services to make sure mounting doesn't fail.

* Picking more sensible loop var name.

* Moving to shell for mount check and fixing jinja2 filter names.

* Trying with the posix mount module instead of command.

* Working through user/sudo issues.

* Altering symlink handling slightly so we have the deploy directory always set.

* Simplifying 'when' checks.

* Adding a revert behaviour for SquashFS.

* Making some wording a little less ambiguous.

* Fixing bad image path and adding clauses to stat check.

* Trying to make remounting SquashFS images a bit safer.

* Adding when clauses to SquashFS image path check.

* Removing revert code, as it cannot work.
…159)

* Add cache clears to Drupal deployments, before database updates and config import are run.

* Move cache clears out of meta roles and only run in database_apply role on non-initial builds.

* Add cache clear stuff to drupal7 database_apply role.
)

* Avoid leaving exponentially growing sqsh files in build locations!

* Use mount point instead of /tmp for stowing previous sqsh file.
* Avoid leaving exponentially growing sqsh files in build locations!

* Use mount point instead of /tmp for stowing previous sqsh file.

* Making sure image/tarball filenames are build specific in /tmp.
* Removing unnecessary lines in Drupal config generation.

* Drupal 7 lacked the install_command var.
* Ensuring dump directory exists on backup step.

* Acts on shared storage, so can run once.
* Supporting option to suppress reverting backups.

* Fixing namespacing in database_backup role.
* Handling the 'drush deploy' command more elegantly for Drupal 8+.

* Changing logic to AND for cache rebuild.
gregharvey and others added 27 commits January 31, 2024 19:10
* Fixing linting so CI can work.

* More linting fixes.

* Downgrading Ubuntu due to Docker issues with ce-dev and latest.

* Making pipefail shell commands use /bin/bash.

* Adding pipefail code to cachetool installer block.

* Adding executable for drush install checker.

* Fixing D7 ctools behaviour to match features handling.

* Jinja2 template looking for settings.php using wrong path.

* Fixing config_generate for Drupal.

* Making host checking more robust.

* Removing variable declaration that will break host check.

* Allow us to use deploy_code with a completely custom build and do nothing.

* Make PHP cachetool install optional.

* Fixing bug introduced by failed_when - need to check for return code on mount points.

* Moving where opcache clears get called so they can be excluded if necessary.

* Fixing linting errors around white space.

* The unarchive module requires 'dest' to be a directory.

* Fixing Drupal example templates.

* Logically splitting tests and removing LGD because it's broken.

* Adding extra plugin LGD needs.

* Clearing stat and apcu cache for PHP by default.

* Adding some perms handling for Drupal 8+ config sync where deploy and web user are different.

* Rolling back APCu cache by default as the package must be installed.

* Making the remount squashfs code more defensive by using a block.

* Trying with 'throttle' keyword to restrict to one machine at a time.

* Sadly 'throttle' seems to work on a task level only, not a block level.

* Adding hash salt generation for Drupal 8+.

* Trying a different syntax for passing params to the password plugin.

* Forgot the constants import in plugin.

* Reinstating the SOPS plugin - we *do* need it.

* Making online/offline tasks use current live code/database.

* We need build paths to bring the site back online if it's a squashfs build.

* Ensuring _common.sh includes /etc/profile.d for Python venv path.

* Making linter happy.

* Using the drush_live_bin var in sync jobs for Drupal 8+.

* Fixing database dump revert.

* Fixing gunzip command.

* Having to do a double step to unpack SQL file in /tmp.

* Load bash profiles before executing a build.

* Fixing linting error, use -n instead of ! -z.

* Adding support for specifying path to Ansible and path to Python interpreter.

* Moving installation of cachetool to application includes, where they are PHP apps.
* Adding a pause if a replica is declared, so the replica can catch up with the writer.

* Making read replica support more robust.
* Adding a pause if a replica is declared, so the replica can catch up with the writer.

* Making read replica support more robust.

* Ensuring mysql backup takes no action on replicas.

* Suppressing backup jobs for replica databases.

* Minor docs update.
* 448: Added setup for the Drupal default.settings.php file, when used in a multi-site context.

* 448: Corrected linting issue.

* 448: Corrected code style issue.

* 448: Added a Drupal default.settings.php file to the templates directory, for use with multi-site Drupal setups.

* 448: Added a .j2 to the end of the default.settings.php template reference.

---------

Co-authored-by: Philip Norton <[email protected]>
* New StatusCake check handling role.

* Updating maintenance mode defaults and options.

* Following StatusCake task naming conventions.
* New StatusCake check handling role.

* Updating maintenance mode defaults and options.

* Following StatusCake task naming conventions.

* Missed the body_format param in StatusCake URI call.
* New StatusCake check handling role.

* Updating maintenance mode defaults and options.

* Following StatusCake task naming conventions.

* Missed the body_format param in StatusCake URI call.

* Ensuring we pass the test ID as a string to the StatusCake URI call.

* Fixing namespaces.

* Stashing the original deploy_path in initial_deploy_path for squashfs builds.

* Adding a double online step for Drupal builds using SquashFS so both the new site and the currently live site come out of maintenance mode.

* We need the path to the previous build for maintenance mode online.

* Changing order of tasks so previous_build_number is set earlier.

* We also need a previous_drush_bin.
* Adding extra healthcheck options for ECS services.

* Renaming scale up/down to in/out to avoid confusion.

* Spotted annoying typo.
* Fixing linting so CI can work.

* More linting fixes.

* Downgrading Ubuntu due to Docker issues with ce-dev and latest.

* Making pipefail shell commands use /bin/bash.

* Adding pipefail code to cachetool installer block.

* Adding executable for drush install checker.

* Fixing D7 ctools behaviour to match features handling.

* Jinja2 template looking for settings.php using wrong path.

* Fixing config_generate for Drupal.

* Making host checking more robust.

* Removing variable declaration that will break host check.

* Allow us to use deploy_code with a completely custom build and do nothing.

* Make PHP cachetool install optional.

* Fixing bug introduced by failed_when - need to check for return code on mount points.

* Moving where opcache clears get called so they can be excluded if necessary.

* Fixing linting errors around white space.

* The unarchive module requires 'dest' to be a directory.

* Fixing Drupal example templates.

* Logically splitting tests and removing LGD because it's broken.

* Adding extra plugin LGD needs.

* Clearing stat and apcu cache for PHP by default.

* Adding some perms handling for Drupal 8+ config sync where deploy and web user are different.

* Rolling back APCu cache by default as the package must be installed.

* Making the remount squashfs code more defensive by using a block.

* Trying with 'throttle' keyword to restrict to one machine at a time.

* Sadly 'throttle' seems to work on a task level only, not a block level.

* Adding hash salt generation for Drupal 8+.

* Trying a different syntax for passing params to the password plugin.

* Forgot the constants import in plugin.

* Reinstating the SOPS plugin - we *do* need it.

* Making online/offline tasks use current live code/database.

* We need build paths to bring the site back online if it's a squashfs build.

* Ensuring _common.sh includes /etc/profile.d for Python venv path.

* Making linter happy.

* Using the drush_live_bin var in sync jobs for Drupal 8+.

* Fixing database dump revert.

* Fixing gunzip command.

* Having to do a double step to unpack SQL file in /tmp.

* Load bash profiles before executing a build.

* Fixing linting error, use -n instead of ! -z.

* Adding support for specifying path to Ansible and path to Python interpreter.

* Moving installation of cachetool to application includes, where they are PHP apps.

* Fixing minor typo.
* Adding pre-flight checks and lock file behaviour to ce-deploy.

* Moving lock file paths to variables.
* Fixing linting so CI can work.

* More linting fixes.

* Downgrading Ubuntu due to Docker issues with ce-dev and latest.

* Making pipefail shell commands use /bin/bash.

* Adding pipefail code to cachetool installer block.

* Adding executable for drush install checker.

* Fixing D7 ctools behaviour to match features handling.

* Jinja2 template looking for settings.php using wrong path.

* Fixing config_generate for Drupal.

* Making host checking more robust.

* Removing variable declaration that will break host check.

* Allow us to use deploy_code with a completely custom build and do nothing.

* Make PHP cachetool install optional.

* Fixing bug introduced by failed_when - need to check for return code on mount points.

* Moving where opcache clears get called so they can be excluded if necessary.

* Fixing linting errors around white space.

* The unarchive module requires 'dest' to be a directory.

* Fixing Drupal example templates.

* Logically splitting tests and removing LGD because it's broken.

* Adding extra plugin LGD needs.

* Clearing stat and apcu cache for PHP by default.

* Adding some perms handling for Drupal 8+ config sync where deploy and web user are different.

* Rolling back APCu cache by default as the package must be installed.

* Making the remount squashfs code more defensive by using a block.

* Trying with 'throttle' keyword to restrict to one machine at a time.

* Sadly 'throttle' seems to work on a task level only, not a block level.

* Adding hash salt generation for Drupal 8+.

* Trying a different syntax for passing params to the password plugin.

* Forgot the constants import in plugin.

* Reinstating the SOPS plugin - we *do* need it.

* Making online/offline tasks use current live code/database.

* We need build paths to bring the site back online if it's a squashfs build.

* Ensuring _common.sh includes /etc/profile.d for Python venv path.

* Making linter happy.

* Using the drush_live_bin var in sync jobs for Drupal 8+.

* Fixing database dump revert.

* Fixing gunzip command.

* Having to do a double step to unpack SQL file in /tmp.

* Load bash profiles before executing a build.

* Fixing linting error, use -n instead of ! -z.

* Adding support for specifying path to Ansible and path to Python interpreter.

* Moving installation of cachetool to application includes, where they are PHP apps.

* Fixing minor typo.

* Cleaning up namespacing.

* Making quotes consistent.

* Making quotes consistent.

* Making quotes consistent.

* Allowing for Drupal cron rollback on revert.

* Only running maintenance mode enable on deploy.
* Adding a new Drupal 10 meta role.

* Bad copy/paste in docs.
* r69347-db-import-from-deploy-server

* r69347-db-import-from-deploy-server

* r69347-db-import-from-deploy-server

* typo fixes and tidying up
* Fixing linting so CI can work.

* More linting fixes.

* Downgrading Ubuntu due to Docker issues with ce-dev and latest.

* Making pipefail shell commands use /bin/bash.

* Adding pipefail code to cachetool installer block.

* Adding executable for drush install checker.

* Fixing D7 ctools behaviour to match features handling.

* Jinja2 template looking for settings.php using wrong path.

* Fixing config_generate for Drupal.

* Making host checking more robust.

* Removing variable declaration that will break host check.

* Allow us to use deploy_code with a completely custom build and do nothing.

* Make PHP cachetool install optional.

* Fixing bug introduced by failed_when - need to check for return code on mount points.

* Moving where opcache clears get called so they can be excluded if necessary.

* Fixing linting errors around white space.

* The unarchive module requires 'dest' to be a directory.

* Fixing Drupal example templates.

* Logically splitting tests and removing LGD because it's broken.

* Adding extra plugin LGD needs.

* Clearing stat and apcu cache for PHP by default.

* Adding some perms handling for Drupal 8+ config sync where deploy and web user are different.

* Rolling back APCu cache by default as the package must be installed.

* Making the remount squashfs code more defensive by using a block.

* Trying with 'throttle' keyword to restrict to one machine at a time.

* Sadly 'throttle' seems to work on a task level only, not a block level.

* Adding hash salt generation for Drupal 8+.

* Trying a different syntax for passing params to the password plugin.

* Forgot the constants import in plugin.

* Reinstating the SOPS plugin - we *do* need it.

* Making online/offline tasks use current live code/database.

* We need build paths to bring the site back online if it's a squashfs build.

* Ensuring _common.sh includes /etc/profile.d for Python venv path.

* Making linter happy.

* Using the drush_live_bin var in sync jobs for Drupal 8+.

* Fixing database dump revert.

* Fixing gunzip command.

* Having to do a double step to unpack SQL file in /tmp.

* Load bash profiles before executing a build.

* Fixing linting error, use -n instead of ! -z.

* Adding support for specifying path to Ansible and path to Python interpreter.

* Moving installation of cachetool to application includes, where they are PHP apps.

* Fixing minor typo.

* Cleaning up namespacing.

* Making quotes consistent.

* Making quotes consistent.

* Making quotes consistent.

* Allowing for Drupal cron rollback on revert.

* Only running maintenance mode enable on deploy.

* Moving previous_deploy_path out of the squashfs block.

* Committed installer by accident.
* 472 Ensure Mautic cache is cleared on every host

* 472 Add Symfony cache clear task

* 472 Make console path configurable

* 472 Fix command variable
* Adding syncing for feature branches.

* Cannot loop over import_role, changing for include_role.

* Adding ability to specify an exact filename for a settings template for Drupal 8 and above.

* First pass at NGINX vhost handling in ce-deploy.

* Adding in the SSL role from ce-provision.

* Fixing role paths and ensuring NGINX ssl.yml is available.

* Fixing NGINX role location.

* Tweaking location of domain.yml so it loads the templates correctly.
* additional_symlink_task_fix_for_squashfs

* additional_symlink_task_fix_for_squashfs

* additional_symlink_task_fix_for_squashfs

* additional_symlink_task_fixes

* additional_symlink_task_fixes
@nfawbert nfawbert self-assigned this Sep 26, 2024
@nfawbert nfawbert merged commit db1c5d9 into devel Sep 26, 2024
3 of 5 checks passed
@nfawbert nfawbert deleted the r70771-strip-incompatible-sandbox-syntax-PR-devel branch September 26, 2024 09:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants