|
1 | | -# Algo VPN Test Suite |
2 | | - |
3 | | -## Current Test Coverage |
4 | | - |
5 | | -### What We Test Now |
6 | | -1. **Basic Sanity** (`test_basic_sanity.py`) |
7 | | - - Python version >= 3.11 |
8 | | - - pyproject.toml exists and has dependencies |
9 | | - - config.cfg is valid YAML |
10 | | - - Ansible playbook syntax |
11 | | - - Shell scripts pass shellcheck |
12 | | - - Dockerfile exists and is valid |
13 | | - |
14 | | -2. **Docker Build** (`test_docker_build.py`) |
15 | | - - Docker image builds successfully |
16 | | - - Container can start |
17 | | - - Ansible is available in container |
18 | | - |
19 | | -3. **Configuration Generation** (`test-local-config.sh`) |
20 | | - - Ansible templates render without errors |
21 | | - - Basic configuration can be generated |
22 | | - |
23 | | -4. **Config Validation** (`test_config_validation.py`) |
24 | | - - WireGuard config format validation |
25 | | - - Base64 key format checking |
26 | | - - IP address and CIDR notation |
27 | | - - Mobile config XML validation |
28 | | - - Port range validation |
29 | | - |
30 | | -5. **Certificate Validation** (`test_certificate_validation.py`) |
31 | | - - OpenSSL availability |
32 | | - - Certificate subject formats |
33 | | - - Key file permissions (600) |
34 | | - - Password complexity |
35 | | - - IPsec cipher suite security |
36 | | - |
37 | | -6. **User Management** (`test_user_management.py`) - Addresses #14745, #14746, #14738, #14726 |
38 | | - - User list parsing from config |
39 | | - - Server selection string parsing |
40 | | - - SSH key preservation |
41 | | - - CA password handling |
42 | | - - User config path generation |
43 | | - - Duplicate user detection |
44 | | - |
45 | | -7. **OpenSSL Compatibility** (`test_openssl_compatibility.py`) - Addresses #14755, #14718 |
46 | | - - OpenSSL version detection |
47 | | - - Legacy flag support detection |
48 | | - - Apple device key format compatibility |
49 | | - - Certificate generation compatibility |
50 | | - - PKCS#12 export for mobile devices |
51 | | - |
52 | | -8. **Cloud Provider Configs** (`test_cloud_provider_configs.py`) - Addresses #14752, #14730, #14762 |
53 | | - - Cloud provider configuration validation |
54 | | - - Hetzner server type updates (cx11 → cx22) |
55 | | - - Azure dependency compatibility |
56 | | - - Region format validation |
57 | | - - Server size naming conventions |
58 | | - - OS image naming validation |
59 | | - |
60 | | -### What We DON'T Test Yet |
61 | | - |
62 | | -#### 1. VPN Functionality |
63 | | -- **WireGuard configuration validation** |
64 | | - - Private/public key generation |
65 | | - - Client config file format |
66 | | - - QR code generation |
67 | | - - Mobile config profiles |
68 | | -- **IPsec configuration validation** |
69 | | - - Certificate generation and validation |
70 | | - - StrongSwan config format |
71 | | - - Apple profile generation |
72 | | -- **SSH tunnel configuration** |
73 | | - - Key generation |
74 | | - - SSH config file format |
75 | | - |
76 | | -#### 2. Cloud Provider Integrations |
77 | | -- DigitalOcean API interactions |
78 | | -- AWS EC2/Lightsail deployments |
79 | | -- Azure deployments |
80 | | -- Google Cloud deployments |
81 | | -- Other providers (Vultr, Hetzner, etc.) |
82 | | - |
83 | | -#### 3. User Management |
84 | | -- Adding new users |
85 | | -- Removing users |
86 | | -- Updating user configurations |
87 | | - |
88 | | -#### 4. Advanced Features |
89 | | -- DNS ad-blocking configuration |
90 | | -- On-demand VPN settings |
91 | | -- MTU calculations |
92 | | -- IPv6 configuration |
93 | | - |
94 | | -#### 5. Security Validations |
95 | | -- Certificate constraints |
96 | | -- Key permissions |
97 | | -- Password generation |
98 | | -- Firewall rules |
99 | | - |
100 | | -## Potential Improvements |
101 | | - |
102 | | -### Short Term (Easy Wins) |
103 | | -1. **Add job names** to fix zizmor warnings |
104 | | -2. **Test configuration file generation** without deployment: |
105 | | - ```python |
106 | | - def test_wireguard_config_format(): |
107 | | - # Generate a test config |
108 | | - # Validate it has required sections |
109 | | - # Check key format with regex |
110 | | - ``` |
111 | | - |
112 | | -3. **Test user management scripts** in isolation: |
113 | | - ```bash |
114 | | - # Test that update-users generates valid YAML |
115 | | - ./algo update-users --dry-run |
116 | | - ``` |
117 | | - |
118 | | -4. **Add XML validation** for mobile configs: |
119 | | - ```bash |
120 | | - xmllint --noout generated_configs/*.mobileconfig |
121 | | - ``` |
122 | | - |
123 | | -### Medium Term |
124 | | -1. **Mock cloud provider APIs** to test deployment logic |
125 | | -2. **Container-based integration tests** using Docker Compose |
126 | | -3. **Test certificate generation** without full deployment |
127 | | -4. **Validate generated configs** against schemas |
128 | | - |
129 | | -### Long Term |
130 | | -1. **End-to-end tests** with actual VPN connections (using network namespaces) |
131 | | -2. **Performance testing** for large user counts |
132 | | -3. **Upgrade path testing** (old configs → new configs) |
133 | | -4. **Multi-platform client testing** |
134 | | - |
135 | | -## Security Improvements (from zizmor) |
136 | | - |
137 | | -Current status: ✅ No security issues found |
138 | | - |
139 | | -Recommendations: |
140 | | -1. Add explicit job names for better workflow clarity |
141 | | -2. Consider pinning Ubuntu runner versions to specific releases |
142 | | -3. Add GITHUB_TOKEN with minimal permissions when needed for API checks |
143 | | - |
144 | | -## Test Philosophy |
145 | | - |
146 | | -Our approach focuses on: |
147 | | -1. **Fast feedback** - Tests run in < 3 minutes |
148 | | -2. **No flaky tests** - Avoid complex networking setups |
149 | | -3. **Test what matters** - Config generation, not VPN protocols |
150 | | -4. **Progressive enhancement** - Start simple, add coverage gradually |
| 1 | +# Tests |
| 2 | + |
| 3 | +## Running Tests |
| 4 | + |
| 5 | +```bash |
| 6 | +# Run all linters (same as CI) |
| 7 | +ansible-lint . && yamllint . && ruff check . && shellcheck scripts/*.sh |
| 8 | + |
| 9 | +# Run Python unit tests |
| 10 | +pytest tests/unit/ -q |
| 11 | + |
| 12 | +# Run E2E connectivity tests (requires deployed Algo on localhost) |
| 13 | +sudo tests/e2e/test-vpn-connectivity.sh both |
| 14 | +``` |
| 15 | + |
| 16 | +## Directory Structure |
| 17 | + |
| 18 | +``` |
| 19 | +tests/ |
| 20 | +├── unit/ # Python unit tests (pytest) |
| 21 | +│ ├── test_basic_sanity.py |
| 22 | +│ ├── test_config_validation.py |
| 23 | +│ ├── test_template_rendering.py |
| 24 | +│ └── ... |
| 25 | +├── e2e/ # End-to-end connectivity tests |
| 26 | +│ └── test-vpn-connectivity.sh |
| 27 | +├── integration/ # Integration test helpers |
| 28 | +│ └── mock_modules/ |
| 29 | +├── fixtures/ # Shared test data |
| 30 | +│ └── test_variables.yml |
| 31 | +└── conftest.py # Pytest configuration |
| 32 | +``` |
| 33 | + |
| 34 | +## Test Coverage |
| 35 | + |
| 36 | +| Category | Tests | What's Verified | |
| 37 | +|----------|-------|-----------------| |
| 38 | +| Sanity | `test_basic_sanity.py` | Python version, config syntax, playbook validity | |
| 39 | +| Config | `test_config_validation.py` | WireGuard/IPsec config formats, key validation | |
| 40 | +| Templates | `test_template_rendering.py` | Jinja2 template syntax, filter compatibility | |
| 41 | +| Certificates | `test_certificate_validation.py` | OpenSSL compatibility, PKCS#12 export | |
| 42 | +| Cloud Providers | `test_cloud_provider_configs.py` | Region formats, instance types, OS images | |
| 43 | +| E2E | `test-vpn-connectivity.sh` | WireGuard handshake, IPsec connection, DNS through VPN | |
| 44 | + |
| 45 | +## CI Workflows |
| 46 | + |
| 47 | +| Workflow | Trigger | What It Does | |
| 48 | +|----------|---------|--------------| |
| 49 | +| `lint.yml` | All PRs | ansible-lint, yamllint, ruff, shellcheck | |
| 50 | +| `main.yml` | Push to master | Syntax check, unit tests, Docker build | |
| 51 | +| `integration-tests.yml` | PRs to roles/ | Full localhost deployment + E2E tests | |
| 52 | +| `smart-tests.yml` | All PRs | Runs subset based on changed files | |
| 53 | + |
| 54 | +## Writing Tests |
| 55 | + |
| 56 | +### Python Unit Tests |
| 57 | + |
| 58 | +Place in `tests/unit/`. Use fixtures from `conftest.py`: |
| 59 | + |
| 60 | +```python |
| 61 | +def test_something(mock_ansible_module, jinja_env): |
| 62 | + # mock_ansible_module - mocked AnsibleModule |
| 63 | + # jinja_env - Jinja2 environment with Ansible filters |
| 64 | + pass |
| 65 | +``` |
| 66 | + |
| 67 | +### Shell Scripts |
| 68 | + |
| 69 | +Use bash strict mode and pass shellcheck: |
| 70 | + |
| 71 | +```bash |
| 72 | +#!/bin/bash |
| 73 | +set -euo pipefail |
| 74 | +``` |
| 75 | + |
| 76 | +## Troubleshooting |
| 77 | + |
| 78 | +**E2E tests fail with "namespace already exists"** |
| 79 | +```bash |
| 80 | +sudo ip netns del algo-client |
| 81 | +``` |
| 82 | + |
| 83 | +**Template tests fail with "filter not found"** |
| 84 | +Add the filter to the mock in `conftest.py`. |
| 85 | + |
| 86 | +**CI fails but local passes** |
| 87 | +Check Python/Ansible versions match CI (Python 3.11, Ansible 12+). |
0 commit comments