Skip to content
This repository was archived by the owner on Oct 7, 2025. It is now read-only.

Commit bbb645b

Browse files
matej5drazenCEMatej Stajduhargregharveyklausi
authored
Aurora backup validation pr devel 2.x (#2699)
* Fixing-email-title-for-backup-validation (#2657) Co-authored-by: Matej Stajduhar <[email protected]> * Adding-task-to-create-aurora-cluster (#2659) * Adding-task-to-create-aurora-cluster * Adding-region-profile-and-tags-to-aurora-cluster * Updating-engine-for-aurora-cluster * Updating-parameter-group-engine * Updating-engine-version * Updating-engine-version-2 * Disabling-automated-backups * Disabling-automated-backups-2 * Disabling-automated-backups-3 * Disabling-automated-backups-4 * Skipping-task-if-not-aurora * Adding-subnet-group-to-instances * Adding-subnet-group-to-instances * Updating-SG-return-values * Updating-SG-return-values-2 * Updating-SG-return-values-3 * Updating-SG-return-values-4 * Updating-SG-return-value-debug * Updating-SG-return-value-debug-2 * Updating-SG-return-value-debug-3 * Removing-debug-tasks * Removing-init-var-for-SG-list * Adding-character-set-option --------- Co-authored-by: Matej Stajduhar <[email protected]> * Fixing truthy variable mistakes. (#2662) * Fixing installer variable bug. * Fixing tests for external PRs. * Testing with a fork. * Adding repo owner's username into installer string. * Refactoring config repo detection to simplify. * No longer permitted to use an integer as a truthy value. * No longer permitted to use existence check as a truthy value. * Can't see a reason why linotp var shouldn't be a boolean. * No longer permitted to use existence check as a truthy value. * Fixing truthy errors in ce_deploy role. * No longer permitted to use an integer as a truthy value. * feat(php): Add FPM slow logrotate (#2625) * feat(php): Support removal of APCU, add FPM slow logrotate * simplify condition * revert apcu installed setting, not needed * r73458-install-php-gmp-by-default2 (#2667) * r73458-install-php-gmp-by-default2 * re-add required packages * Wazuh-mitre-report-setup (#2588) * Wazuh-mitre-report-setup * Wazuh-mitre-shellshock-longurl-block * Fixing-vars * Wazuh-mitre-report-setup-PR-2.x * Wazuh mitre report setup pr 2.x (#2669) * Wazuh-mitre-report-setup * Wazuh-mitre-shellshock-longurl-block * Fixing-vars * Wazuh-mitre-report-setup-PR-2.x * Wazuh-mitre-report-setup-PR-2.x * pin_ansible_version (#2671) * pin_ansible_version * pin_ansible_version * pin_ansible_version * pin_ansible_version * pin_ansible_version_fix_upgrade_timer * pin_ansible_version_fix_upgrade_timer * pin_ansible_version_fix_upgrade_timer * pin_ansible_version_disable_upgrade_timer * pin_ansible_version_disable_upgrade_timer * pin_ansible_version_disable_upgrade_timer * pin_ansible_version_disable_upgrade_timer * Fixing-ce-provision-vars (#2678) * Updating-string (#2507) * Updating-string * Updating-string-3 --------- Co-authored-by: Matej Stajduhar <[email protected]> * Added-tasks-to-backup-Aurora-and-copy-AMI-to-safe-region (#2682) * Added-tasks-to-backup-Aurora-and-copy-AMI-to-safe-region * Fixing-aurora-backup-tasks * Fixing-aurora-backup-tasks-2 * Fixing-aurora-backup-tasks-3 * Fixing-aurora-backup-tasks-5 * Adding-aurora-template * Updating-aurora-vars * Adding-handler-to-defaults-for-CF --------- Co-authored-by: Matej Stajduhar <[email protected]> * SG-creation-update (#2605) * SG-creation-update * Updating-lambda-tasks-to-handle-various-file-options * Updating-lambda-tasks-for-url-handling * Updating-aws_admin_tools-for-aws_lambda * Updating-aws_admin_tools-for-aws_lambda * Setting-loop-item * Setting-loop-item-2 * Updating-vpc-sec-group-vars * Removing-extra-vars-for-git-module * Adding-default-for-git_url * Cleaning-up-tasks * Updating-ansible-lint * Updating-ansible-lint * Ommiting-name-if-no-sec_group-name-defined * Removing-loop-var --------- Co-authored-by: Matej Stajduhar <[email protected]> * Fixing-copy-AMI-to-backup-region (#2684) Co-authored-by: Matej Stajduhar <[email protected]> * Fixing-ami-copy-task (#2686) Co-authored-by: Matej Stajduhar <[email protected]> * Bug fixes pr 2.x (#2690) * Fixing installer variable bug. * Fixing tests for external PRs. * Testing with a fork. * Adding repo owner's username into installer string. * Refactoring config repo detection to simplify. * No longer permitted to use an integer as a truthy value. * No longer permitted to use existence check as a truthy value. * Can't see a reason why linotp var shouldn't be a boolean. * No longer permitted to use existence check as a truthy value. * Fixing truthy errors in ce_deploy role. * No longer permitted to use an integer as a truthy value. * Updating clamav command to use flock avoiding duplicate processes running. * 73569 allowing webp nginx pr 2.x (#2692) * allowing webp extension * adding webp mime type --------- Co-authored-by: filip <[email protected]> * extending provision.sh to support tags in plays (#2431) Co-authored-by: filip <[email protected]> * Adding-option-for-Aurora-RDS-for-backup-validation (#2635) Co-authored-by: Matej Stajduhar <[email protected]> * Fixing-aws_vpc-override (#2688) * Fixing-aws_vpc-override * Adding-defaults * Fixing-register-command * Defaulting-tags * Defaulting-tags-2 * Updating-region * Updating-iam_role-vars * Updating-iam_role-vars-2 * Updating-when-statement * Updating-when-statement-for-backups * Updating-when-statement-for-iam-policy * Updating-when-statement-for-iam-policy * Updating-vars-for-SG-creation * Updating-when-statement-for-iam-role * Updating-handle-git-url * Updating-handle-git-url-2 * Updating-handle-git-url-3 * Updating-handle-git-url-4 * Updating-handle-git-url-5 * Updating-handle-git-url-6 * Updating-handle-git-url-7 * Fixing-indentation --------- Co-authored-by: Matej Stajduhar <[email protected]> * Updating-pam-ldap-condition (#2695) * Updating-pam-ldap-condition * Updating-pam-ldap-condition-PR-2.x * Fixing-when-statement * Updating-LE-tasks * Adding-from_json-for-systemd-timers * Adding-from_json-for-systemd-timers-2 * Removin-from_json-for-systemd-timers * Updating-pam_ldap-when-statements * Updating-pam_ldap-when-statements-2 * Updated-Backup-validation-role * Updated-trusted-entity-file-name * Updated-event-patterns * Dropped-default-aurora-retention-to-1 * Bug-fixes * Moving-iam-policy * Updating-tasks * Updating-tasks-2 * Updating-return-value * Updating-file-names * Updating-file-names-2 * Updating-file-names-3 * Updating-file-names-4 * Adding-debug * Adding-debug-2 * Adding-debug-3 * Updating-source-for-iam * Removing-handle-zip-for-lambda * Updating-regex-search * Updating-regex-search * Updating-lambda-function-handling * Updating-lambda-function-handling * Updating-lambda-function-handling-2 * Updating-event-bridge-role-arn * Moving-functions-to-gitlab * Updating-event-pattern * Updating-iam-role * Updating-defaults * Reverting-wazuh-changes * Removing-files * Adding-LE-vars-for-apache --------- Co-authored-by: drazenCE <[email protected]> Co-authored-by: Matej Stajduhar <[email protected]> Co-authored-by: Greg Harvey <[email protected]> Co-authored-by: Klaus Purer <[email protected]> Co-authored-by: nfawbert <[email protected]> Co-authored-by: tymofiisobchenko <[email protected]> Co-authored-by: Filip Rupic <[email protected]> Co-authored-by: filip <[email protected]>
1 parent 961af1c commit bbb645b

File tree

20 files changed

+252
-511
lines changed

20 files changed

+252
-511
lines changed

roles/aws/aws_admin_tools/templates/api_get_list_of_ec2.py.j2

Lines changed: 0 additions & 48 deletions
This file was deleted.

roles/aws/aws_admin_tools/templates/default_s3_object.j2

Lines changed: 0 additions & 1 deletion
This file was deleted.

roles/aws/aws_admin_tools/templates/get_infra_data_from_s3.py.j2

Whitespace-only changes.

roles/aws/aws_backup_validation/defaults/main.yml

Lines changed: 42 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,46 @@ aws_backup_validation:
88
runtime: "python3.12"
99
handler: "lambda_handler"
1010
resources:
11-
- EC2
12-
- RDS
11+
- name: ec2_test_instance
12+
git_url: [email protected]:functions/ec2_test_instance.git
13+
type: EC2
14+
lambda_policy:
15+
- "backup:PutRestoreValidationResult"
16+
- "ssm:GetCommandInvocation"
17+
- "ssm:GetConnectionStatus"
18+
- "ssm:SendCommand"
19+
- "ec2:DescribeInstances"
20+
- name: rds_test_instance
21+
git_url: [email protected]:functions/rds_test_instance.git
22+
type: RDS
23+
lambda_policy:
24+
- "backup:PutRestoreValidationResult"
25+
- "ssm:GetCommandInvocation"
26+
- "ssm:SendCommand"
27+
- "ec2:DescribeInstances"
28+
- "rds:DescribeDBInstances"
29+
- name: aurora_create_instance
30+
git_url: [email protected]:functions/aurora_create_instance.git
31+
type: Aurora
32+
lambda_policy:
33+
- "lambda:InvokeFunction"
34+
- name: aurora_test_instance
35+
git_url: [email protected]:functions/aurora_test_instance.git
36+
type: Aurora
37+
event_pattern: '{ "source": ["aws.rds"], "detail-type": ["RDS DB Instance Event"], "resources": [{ "prefix": "arn:aws:rds:eu-west-1:{{ _acc_id }}:db:restoretest" }], "detail": { "EventID": ["RDS-EVENT-0005"] } }'
38+
lambda_policy:
39+
- "backup:PutRestoreValidationResult"
40+
- "ec2:DescribeInstances"
41+
- "rds:DescribeDBInstances"
42+
- "rds:DescribeDBClusters"
43+
- "rds:DeleteDBInstance"
44+
- name: validation_report
45+
git_url: [email protected]:functions/validation_report.git
46+
type: Schedule
47+
schedule: "cron(0 0 ? * MON *)"
48+
lambda_policy:
49+
- "backup:ListRestoreJobs"
50+
- "ses:SendEmail"
51+
- "ec2:DescribeImages"
52+
- "rds:DescribeDbSnapshots"
1353
#- EFS

roles/aws/aws_backup_validation/files/trusted_entitites.j2

Lines changed: 0 additions & 12 deletions
This file was deleted.
Lines changed: 107 additions & 70 deletions
Original file line numberDiff line numberDiff line change
@@ -1,36 +1,100 @@
11
---
2-
- name: Create a role and attach policies for Lambda backup validation.
2+
- name: Get account ID for ARN.
3+
ansible.builtin.command: >-
4+
aws sts get-caller-identity
5+
--query Account
6+
--output text
7+
register: _acc_id
8+
9+
- name: Setting previous command output into variable.
10+
ansible.builtin.set_fact:
11+
_acc_id: "{{ _acc_id.stdout | from_json }}"
12+
13+
# Passing Api throug the resource ( triger )
14+
15+
- name: Configure API Gateway if defined.
16+
ansible.builtin.include_tasks: create_mock.yml
17+
when: _api_index | length == 0
18+
19+
- name: Create a role and attach policies for events.
320
ansible.builtin.include_role:
421
name: aws/aws_iam_role
522
vars:
623
aws_iam_role:
7-
name: LambdaBackupRestoreRole
24+
name: "{{ item.name }}_event"
25+
source: "{{ item.name}}"
26+
aws_profile: "{{ _aws_profile }}"
27+
inline_policies:
28+
name: "{{ item.name }}_event"
29+
resource: "arn:aws:lambda:{{ _aws_region }}:{{ _acc_id }}:function:{{ item.name }}"
30+
action:
31+
- "lambda:InvokeFunction"
32+
policy_document: "{{ lookup('template', 'event_document_policy.json.j2') }}"
33+
loop: "{{ aws_backup_validation.resources }}"
34+
loop_control:
35+
extended: true
36+
extended_allitems: false
37+
38+
- name: Create a role and attach policies for Lambda functions.
39+
ansible.builtin.include_role:
40+
name: aws/aws_iam_role
41+
vars:
42+
aws_iam_role:
43+
name: "{{ item.name}}_lambda"
44+
source: "{{ item.name}}"
845
aws_profile: "{{ _aws_profile }}"
946
managed_policies:
10-
- arn:aws:iam::aws:policy/AmazonEC2FullAccess
11-
- arn:aws:iam::aws:policy/AWSBackupFullAccess
12-
- arn:aws:iam::aws:policy/AmazonRDSFullAccess
1347
- arn:aws:iam::aws:policy/CloudWatchLogsFullAccess
14-
- arn:aws:iam::aws:policy/AmazonSSMFullAccess
15-
- arn:aws:iam::aws:policy/AmazonSESFullAccess
16-
policy_document: "{{ lookup('file', 'trusted_entitites.j2') }}"
48+
inline_policies:
49+
name: "{{ item.name }}_lambda"
50+
resource: "*"
51+
action: "{{ item.lambda_policy }}"
52+
policy_document: "{{ lookup('template', 'trusted_entitites.json.j2') }}"
53+
loop: "{{ aws_backup_validation.resources }}"
54+
loop_control:
55+
extended: true
56+
extended_allitems: false
57+
58+
- name: Get info about newly created restore testing plan.
59+
ansible.builtin.command: >
60+
aws backup list-restore-testing-plans --region {{ _aws_region }}
61+
register: _testing_plans
62+
63+
- name: Create Lambda functions from templates.
64+
ansible.builtin.include_role:
65+
name: aws/aws_lambda
66+
vars:
67+
aws_lambda:
68+
name: "{{ item.name }}"
69+
description: "Lambda functions for {{ item.type }} validation."
70+
timeout: "{{ aws_backup_validation.timeout }}"
71+
role: "{{ aws_iam_role._result[item.name + '_lambda'] }}"
72+
runtime: "{{ aws_backup_validation.runtime }}"
73+
function_file: "{{ lookup('template', item.name + '.py.j2') }}"
74+
s3_bucket: "{{ aws_backup_validation.s3_bucket }}"
75+
s3_bucket_prefix: "lambda-functions"
76+
tags:
77+
Name: "{{ item.name }}"
78+
loop: "{{ aws_backup_validation.resources }}"
79+
when: item.git_url is not defined
1780

18-
- name: Create backup validation Lambda functions.
81+
- name: Create Lambda functions from git url.
1982
ansible.builtin.include_role:
2083
name: aws/aws_lambda
2184
vars:
2285
aws_lambda:
23-
name: "{{ aws_backup_validation.name }}_{{ item }}"
24-
description: "{{ aws_backup_validation.description }}"
86+
name: "{{ item.name }}"
87+
description: "Lambda functions for {{ item.type }} validation."
2588
timeout: "{{ aws_backup_validation.timeout }}"
26-
role: "{{ aws_iam_role._result['LambdaBackupRestoreRole'] }}"
89+
role: "{{ aws_iam_role._result[item.name + '_lambda'] }}"
2790
runtime: "{{ aws_backup_validation.runtime }}"
28-
function_file: "{{ lookup('template', item + '_validation.py.j2') }}"
91+
function_file: "{{ item.git_url }}"
2992
s3_bucket: "{{ aws_backup_validation.s3_bucket }}"
3093
s3_bucket_prefix: "lambda-functions"
3194
tags:
32-
Name: "{{ item }}_backup_validation"
95+
Name: "{{ item.name }}"
3396
loop: "{{ aws_backup_validation.resources }}"
97+
when: item.git_url is defined
3498

3599
- name: Create an IAM Managed Policy for passing roles and setup IAM role.
36100
ansible.builtin.include_role:
@@ -53,79 +117,52 @@
53117
#- name: Get verified domain.
54118
# ansible.builtin.include_tasks: get_valid_email.yml
55119

56-
- name: Get info about newly created restore testing plan.
57-
ansible.builtin.command: >
58-
aws backup list-restore-testing-plans --region {{ _aws_region }}
59-
register: _testing_plans
60-
61-
- name: Create validation report function.
62-
ansible.builtin.include_role:
63-
name: aws/aws_lambda
64-
vars:
65-
aws_lambda:
66-
name: "validation_report"
67-
description: "{{ aws_backup_validation.description }}"
68-
timeout: "30"
69-
role: "{{ aws_iam_role._result['LambdaBackupRestoreRole'] }}"
70-
runtime: "{{ aws_backup_validation.runtime }}"
71-
function_file: "{{ lookup('template', 'validation_report.py.j2') }}"
72-
s3_bucket: "{{ aws_backup_validation.s3_bucket }}"
73-
s3_bucket_prefix: "lambda-functions"
74-
tags:
75-
Name: "validation_report"
76-
77-
- name: Get account ID for ARN.
78-
ansible.builtin.command: >-
79-
aws sts get-caller-identity
80-
--query Account
81-
--output text
82-
register: _acc_id
83-
84-
- name: Setting previous command output into variable.
85-
ansible.builtin.set_fact:
86-
_acc_id: "{{ _acc_id.stdout | from_json }}"
87-
88-
- name: Create EventBridge for validation functions.
120+
- name: Create EventBridge with lambda functions.
89121
amazon.aws.cloudwatchevent_rule:
90-
name: "RestoreValidation_{{ item }}"
122+
name: "{{ item.name }}"
91123
description: "{{ aws_backup_validation.description }}"
92124
state: present
93125
region: "{{ _aws_region }}"
94-
event_pattern: '{ "source": ["aws.backup"], "detail-type": ["Restore Job State Change"], "detail": { "resourceType": ["{{ item }}"], "status": ["COMPLETED"] } }'
126+
role_arn: "arn:aws:iam::{{ _acc_id }}:role/{{ item.name }}_event"
127+
event_pattern: >-
128+
{{ item.event_pattern | default(
129+
{
130+
"source": ["aws.backup"],
131+
"detail-type": ["Restore Job State Change"],
132+
"detail": {
133+
"resourceType": [ item.type ],
134+
"status": ["COMPLETED"]
135+
}
136+
} | to_json
137+
) }}
95138
targets:
96-
- id: "RestoreValidation_{{ item }}"
97-
arn: "arn:aws:lambda:{{ _aws_region }}:{{ _acc_id }}:function:RestoreValidation_{{ item }}"
139+
- id: "{{ item.name }}"
140+
arn: "arn:aws:lambda:{{ _aws_region }}:{{ _acc_id }}:function:{{ item.name }}"
98141
loop: "{{ aws_backup_validation.resources }}"
142+
when: item.type != "Schedule"
99143
register: _event_bridges
100144

101145
- name: Create schedule for validation reports.
102146
amazon.aws.cloudwatchevent_rule:
103-
name: validation_report
104-
schedule_expression: "cron(0 0 ? * MON *)"
105-
description: Run validation reporting
147+
name: "{{ item.name }}"
148+
schedule_expression: "{{ item.schedule }}"
149+
description: "Run validation reporting."
106150
region: "{{ _aws_region }}"
151+
role_arn: "arn:aws:iam::{{ _acc_id }}:role/{{ item.name }}_event"
107152
targets:
108153
- id: validation_report
109-
arn: "{{ (aws_lambda._result['validation_report'].configuration.function_arn.split(':') | map('trim'))[:-1] | join(':') }}" # Remove the version number from ARN
154+
arn: "arn:aws:lambda:{{ _aws_region }}:{{ _acc_id }}:function:{{ item.name }}"
155+
loop: "{{ aws_backup_validation.resources }}"
156+
when: item.type == "Schedule"
110157
register: _validation_event
111158

112-
- name: Update Lambda policy.
159+
- name: Update Lambda policies.
113160
amazon.aws.lambda_policy:
114161
state: present
115-
function_name: "{{ item.rule.name }}"
116-
statement_id: "{{ item.rule.name }}"
162+
function_name: "{{ item.name }}"
163+
statement_id: "{{ item.name }}"
117164
action: lambda:InvokeFunction
118165
principal: events.amazonaws.com
119-
source_arn: "{{ item.rule.arn }}"
120-
region: "{{ _aws_region }}"
121-
loop: "{{ _event_bridges.results }}"
122-
123-
- name: Update lambda validation report policy.
124-
amazon.aws.lambda_policy:
125-
state: present
126-
function_name: "validation_report"
127-
statement_id: "validation_report"
128-
action: lambda:InvokeFunction
129-
principal: events.amazonaws.com
130-
source_arn: "{{ _validation_event.rule.arn }}"
166+
source_arn: "arn:aws:events:{{ _aws_region }}:{{ _acc_id }}:rule/{{ item.name }}"
131167
region: "{{ _aws_region }}"
168+
loop: "{{ aws_backup_validation.resources }}"

roles/aws/aws_backup_validation/tasks/testing_resources.yml

Lines changed: 5 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -68,34 +68,26 @@
6868
instance: "EC2"
6969
file-system: "EFS"
7070
db: "RDS"
71-
cluster: "AURORA"
71+
cluster: "Aurora"
7272

7373
- name: Set instance type for template.
7474
ansible.builtin.set_fact:
7575
_instance_type_restore: "{{ instance_type[backup.resource_type] }}"
76-
_template_prefix: "{{ instance_type[backup.resource_type] }}"
7776
when: backup.resource_type != 'file-system'
7877

79-
- name: Set instance type to Aurora if defined.
80-
ansible.builtin.set_fact:
81-
_instance_type_restore: "Aurora"
82-
when:
83-
- backup.resource_type == 'db'
84-
- "'aurora' in aws_rds.engine"
85-
8678
- name: Create restore testing query file.
8779
ansible.builtin.template:
88-
src: "{{ _template_prefix }}_restore_testing.j2"
80+
src: "{{ _instance_type_restore }}_restore_testing.j2"
8981
dest: /tmp/restore_testing.json
9082
register: _restore_testing_query
91-
when: _template_prefix is defined
83+
when: _instance_type_restore is defined
9284

9385
- name: Check if protected reource exist.
9486
ansible.builtin.command: >
9587
aws backup list-protected-resources --query "Results[?ResourceArn=='{{ _resource_arn }}']" --region {{ _aws_region }}
9688
register: _protected_res
9789

98-
- name: Assign {{ _template_prefix }} resource to AWS restore testing plan.
90+
- name: Assign {{ _instance_type_restore }} resource to AWS restore testing plan.
9991
ansible.builtin.command: >
10092
aws backup create-restore-testing-selection --cli-input-json file:///tmp/restore_testing.json --region {{ _aws_region }}
101-
when: _template_prefix is defined and _testing_plan_info.stdout != "null" and _testing_selection_exists.stdout | length == 0 and _protected_res.stdout | length != 0
93+
when: _instance_type_restore is defined and _testing_plan_info.stdout != "null" and _testing_selection_exists.stdout | length == 0 and _protected_res.stdout | length != 0
File renamed without changes.

0 commit comments

Comments
 (0)