-
Notifications
You must be signed in to change notification settings - Fork 28
Open
Description
The current way of passing creds in airflow for taar jobs is not ideal – the cleanest way to do this right now is probably to write to a GCS bucket for all the outputs, and sync the bucket to AWS using the GoogleCloudStorageToS3Operator
Most of the changes are isolated to the s3-interfacing methods here: https://github.com/mozilla/python_mozetl/blob/master/mozetl/taar/taar_utils.py
Metadata
Metadata
Assignees
Labels
No labels