Modularize Terratest into 27 independent modules

[james00012]

Summary

This PR converts Terratest from a monolithic Go module into 28 independent submodules, allowing users to import only the modules they need without pulling in unnecessary dependencies.

Changes

• Created 28 independent Go modules under modules/ and internal/
• Set up Go workspace (go.work) for local development
• Added GitHub Action to automatically tag submodules on release

Backward Compatibility Testing

Extensive testing was performed using a fork repository (james00012/terratest-modularization-test) to validate backward compatibility:

1. Consumer Simulation Test

Created an external project importing 13+ modules simultaneously:

• modules/terraform, modules/terragrunt, modules/aws, modules/k8s, modules/docker, modules/helm, modules/collections, modules/random, modules/logger, modules/shell, modules/retry, modules/files, modules/environment

Result: PASS - No ambiguous import errors

2. Backward Compatibility Tests

Validated that existing user code works without modification:

Module	Test	Result
terraform	Existing terraform.Options{}, terraform.Init, terraform.Apply	PASS
aws	Existing aws.GetRandomRegion, aws.CreateS3Bucket	PASS
k8s	Existing k8s.KubectlOptions{}, k8s.GetPod	PASS
docker	Existing docker.Options{}, docker.Build, docker.Run	PASS
helm	Existing helm.Options{}, helm.Install	PASS

Migration for Users

No changes required. Existing import paths continue to work:

import (
    "github.com/gruntwork-io/terratest/modules/terraform"
    "github.com/gruntwork-io/terratest/modules/aws"
)

Release Process

A new GitHub Action (.github/workflows/release-modules.yml) handles submodule tagging:

How It Works

1. Create a GitHub Release (e.g., v0.55.0) through the GitHub UI or API
2. Workflow automatically triggers when the release is published
3. Auto-discovers all modules by finding go.mod files (excludes root and test directories)
4. Creates tags for all submodules:
   • modules/terraform/v0.55.0
   • modules/aws/v0.55.0
   • modules/k8s/v0.55.0
   • ... (all modules)
5. Pushes tags to the repository

Setup Required

Add GH_TOKEN as a repository secret:

1. Go to Settings → Secrets and variables → Actions
2. Create a new secret named GH_TOKEN
3. Use a Personal Access Token with contents: write permission

Manual Trigger

The workflow can also be triggered manually:

1. Go to Actions → Release Module Tags
2. Click Run workflow
3. Enter the version (e.g., 0.55.0)

Auto-Discovery

The workflow automatically discovers modules - no need to update the workflow when adding new modules:

find . -name "go.mod" -not -path "./go.mod" -not -path "./test/*"

Why Submodule Tags?

Go modules require tags in the format <path>/v<version> for submodules. Without these tags, users cannot pin specific versions:

go get github.com/gruntwork-io/terratest/modules/[email protected]

Tested On Fork

The workflow was tested on james00012/terratest-modularization-test:

• Successfully created tags for v0.55.0
• All 28 submodule tags created and pushed correctly via auto-discovery

Benefits

• Reduced dependencies: Import only what you need
• Faster builds: Smaller dependency trees
• Future-proof: New modules automatically tagged without workflow changes

Test plan

• Consumer simulation test passes (no ambiguous imports)
• Backward compatibility tests pass (existing code works unchanged)
• CI pipeline passes
• Release workflow tested on fork (v0.55.0)
• Auto-discovery tested (28 modules found)
• Add GH_TOKEN secret after merge
• Manual verification of release workflow on first production release

[james00012]

Closing this PR from fork. Will reopen from the main repository so CircleCI can run properly.