Add end-to-end VPN connectivity tests using network namespaces

.ansible-lint

@@ -2,7 +2,6 @@
 exclude_paths:
   - .cache/
   - .github/
-  - tests/legacy-lxd/
   - tests/
   - files/cloud-init/  # Cloud-init files have special format requirements
   - playbooks/  # These are task files included by other playbooks, not standalone playbooks

.github/workflows/integration-tests.yml

@@ -33,7 +33,7 @@ jobs:
       - uses: actions/setup-python@83679a892e2d95755f2dac6acb0bfd1e9ac5d548  # v6.1.0
         with:
           python-version: '3.11'
-          cache: 'pip'
+          # Note: No pip cache - we use uv for dependency management
 
       - name: Install system dependencies
         run: |
@@ -46,7 +46,9 @@ jobs:
             dnsmasq \
             qrencode \
             openssl \
-            linux-headers-$(uname -r)
+            linux-headers-$(uname -r) \
+            libxml2-utils \
+            dnsutils
 
       - name: Install uv
         run: curl -LsSf https://astral.sh/uv/install.sh | sh
@@ -203,6 +205,26 @@ jobs:
             sudo ipsec statusall | grep -E "INSTALLED|ESTABLISHED" || echo "No active IPsec connections (expected)"
           fi
 
+      - name: Run E2E VPN connectivity tests
+        env:
+          VPN_TYPE: ${{ matrix.vpn_type }}
+        run: |
+          chmod +x tests/e2e/test-vpn-connectivity.sh
+          sudo tests/e2e/test-vpn-connectivity.sh "${VPN_TYPE}"
+
+      - name: Collect E2E debug info on failure
+        if: failure()
+        run: |
+          echo "=== E2E Test Debug Information ==="
+          echo "=== Network Namespaces ==="
+          ip netns list || true
+          echo "=== WireGuard Config (alice) ==="
+          cat configs/localhost/wireguard/alice.conf 2>/dev/null || echo "Not found"
+          echo "=== IPsec Certificates ==="
+          ls -la configs/localhost/ipsec/.pki/certs/ 2>/dev/null || echo "Not found"
+          echo "=== iptables NAT ==="
+          sudo iptables -t nat -L -n -v || true
+
       - name: Upload configs as artifacts
         if: always()
         uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4  # v5.0.0

tests/README.md

@@ -1,150 +1,87 @@
-# Algo VPN Test Suite
-
-## Current Test Coverage
-
-### What We Test Now
-1. **Basic Sanity** (`test_basic_sanity.py`)
-   - Python version >= 3.11
-   - pyproject.toml exists and has dependencies
-   - config.cfg is valid YAML
-   - Ansible playbook syntax
-   - Shell scripts pass shellcheck
-   - Dockerfile exists and is valid
-
-2. **Docker Build** (`test_docker_build.py`)
-   - Docker image builds successfully
-   - Container can start
-   - Ansible is available in container
-
-3. **Configuration Generation** (`test-local-config.sh`)
-   - Ansible templates render without errors
-   - Basic configuration can be generated
-
-4. **Config Validation** (`test_config_validation.py`)
-   - WireGuard config format validation
-   - Base64 key format checking
-   - IP address and CIDR notation
-   - Mobile config XML validation
-   - Port range validation
-
-5. **Certificate Validation** (`test_certificate_validation.py`)
-   - OpenSSL availability
-   - Certificate subject formats
-   - Key file permissions (600)
-   - Password complexity
-   - IPsec cipher suite security
-
-6. **User Management** (`test_user_management.py`) - Addresses #14745, #14746, #14738, #14726
-   - User list parsing from config
-   - Server selection string parsing
-   - SSH key preservation
-   - CA password handling
-   - User config path generation
-   - Duplicate user detection
-
-7. **OpenSSL Compatibility** (`test_openssl_compatibility.py`) - Addresses #14755, #14718
-   - OpenSSL version detection
-   - Legacy flag support detection
-   - Apple device key format compatibility
-   - Certificate generation compatibility
-   - PKCS#12 export for mobile devices
-
-8. **Cloud Provider Configs** (`test_cloud_provider_configs.py`) - Addresses #14752, #14730, #14762
-   - Cloud provider configuration validation
-   - Hetzner server type updates (cx11 → cx22)
-   - Azure dependency compatibility
-   - Region format validation
-   - Server size naming conventions
-   - OS image naming validation
-
-### What We DON'T Test Yet
-
-#### 1. VPN Functionality
-- **WireGuard configuration validation**
-  - Private/public key generation
-  - Client config file format
-  - QR code generation
-  - Mobile config profiles
-- **IPsec configuration validation**
-  - Certificate generation and validation
-  - StrongSwan config format
-  - Apple profile generation
-- **SSH tunnel configuration**
-  - Key generation
-  - SSH config file format
-
-#### 2. Cloud Provider Integrations
-- DigitalOcean API interactions
-- AWS EC2/Lightsail deployments
-- Azure deployments
-- Google Cloud deployments
-- Other providers (Vultr, Hetzner, etc.)
-
-#### 3. User Management
-- Adding new users
-- Removing users
-- Updating user configurations
-
-#### 4. Advanced Features
-- DNS ad-blocking configuration
-- On-demand VPN settings
-- MTU calculations
-- IPv6 configuration
-
-#### 5. Security Validations
-- Certificate constraints
-- Key permissions
-- Password generation
-- Firewall rules
-
-## Potential Improvements
-
-### Short Term (Easy Wins)
-1. **Add job names** to fix zizmor warnings
-2. **Test configuration file generation** without deployment:
-   ```python
-   def test_wireguard_config_format():
-       # Generate a test config
-       # Validate it has required sections
-       # Check key format with regex
-   ```
-
-3. **Test user management scripts** in isolation:
-   ```bash
-   # Test that update-users generates valid YAML
-   ./algo update-users --dry-run
-   ```
-
-4. **Add XML validation** for mobile configs:
-   ```bash
-   xmllint --noout generated_configs/*.mobileconfig
-   ```
-
-### Medium Term
-1. **Mock cloud provider APIs** to test deployment logic
-2. **Container-based integration tests** using Docker Compose
-3. **Test certificate generation** without full deployment
-4. **Validate generated configs** against schemas
-
-### Long Term
-1. **End-to-end tests** with actual VPN connections (using network namespaces)
-2. **Performance testing** for large user counts
-3. **Upgrade path testing** (old configs → new configs)
-4. **Multi-platform client testing**
-
-## Security Improvements (from zizmor)
-
-Current status: ✅ No security issues found
-
-Recommendations:
-1. Add explicit job names for better workflow clarity
-2. Consider pinning Ubuntu runner versions to specific releases
-3. Add GITHUB_TOKEN with minimal permissions when needed for API checks
-
-## Test Philosophy
-
-Our approach focuses on:
-1. **Fast feedback** - Tests run in < 3 minutes
-2. **No flaky tests** - Avoid complex networking setups
-3. **Test what matters** - Config generation, not VPN protocols
-4. **Progressive enhancement** - Start simple, add coverage gradually
+# Tests
+
+## Running Tests
+
+```bash
+# Run all linters (same as CI)
+ansible-lint . && yamllint . && ruff check . && shellcheck scripts/*.sh
+
+# Run Python unit tests
+pytest tests/unit/ -q
+
+# Run E2E connectivity tests (requires deployed Algo on localhost)
+sudo tests/e2e/test-vpn-connectivity.sh both
+```
+
+## Directory Structure
+
+```
+tests/
+├── unit/                    # Python unit tests (pytest)
+│   ├── test_basic_sanity.py
+│   ├── test_config_validation.py
+│   ├── test_template_rendering.py
+│   └── ...
+├── e2e/                     # End-to-end connectivity tests
+│   └── test-vpn-connectivity.sh
+├── integration/             # Integration test helpers
+│   └── mock_modules/
+├── fixtures/                # Shared test data
+│   └── test_variables.yml
+└── conftest.py              # Pytest configuration
+```
+
+## Test Coverage
+
+| Category | Tests | What's Verified |
+|----------|-------|-----------------|
+| Sanity | `test_basic_sanity.py` | Python version, config syntax, playbook validity |
+| Config | `test_config_validation.py` | WireGuard/IPsec config formats, key validation |
+| Templates | `test_template_rendering.py` | Jinja2 template syntax, filter compatibility |
+| Certificates | `test_certificate_validation.py` | OpenSSL compatibility, PKCS#12 export |
+| Cloud Providers | `test_cloud_provider_configs.py` | Region formats, instance types, OS images |
+| E2E | `test-vpn-connectivity.sh` | WireGuard handshake, IPsec connection, DNS through VPN |
+
+## CI Workflows
+
+| Workflow | Trigger | What It Does |
+|----------|---------|--------------|
+| `lint.yml` | All PRs | ansible-lint, yamllint, ruff, shellcheck |
+| `main.yml` | Push to master | Syntax check, unit tests, Docker build |
+| `integration-tests.yml` | PRs to roles/ | Full localhost deployment + E2E tests |
+| `smart-tests.yml` | All PRs | Runs subset based on changed files |
+
+## Writing Tests
+
+### Python Unit Tests
+
+Place in `tests/unit/`. Use fixtures from `conftest.py`:
+
+```python
+def test_something(mock_ansible_module, jinja_env):
+    # mock_ansible_module - mocked AnsibleModule
+    # jinja_env - Jinja2 environment with Ansible filters
+    pass
+```
+
+### Shell Scripts
+
+Use bash strict mode and pass shellcheck:
+
+```bash
+#!/bin/bash
+set -euo pipefail
+```
+
+## Troubleshooting
+
+**E2E tests fail with "namespace already exists"**
+```bash
+sudo ip netns del algo-client
+```
+
+**Template tests fail with "filter not found"**
+Add the filter to the mock in `conftest.py`.
+
+**CI fails but local passes**
+Check Python/Ansible versions match CI (Python 3.11, Ansible 12+).