SecretsManagerBackend

Amazon

Retrieves Connection or Variables from AWS Secrets Manager

View Source

Last Updated: Apr. 2, 2021

Access Instructions

Install the Amazon provider package into your Airflow environment.

Update your airflow.cfg per the instructions in the docs below.

Parameters

connections_prefixstrSpecifies the prefix of the secret to read to get Connections. If set to None (null), requests for connections will not be sent to AWS Secrets Manager
variables_prefixstrSpecifies the prefix of the secret to read to get Variables. If set to None (null), requests for variables will not be sent to AWS Secrets Manager
config_prefixstrSpecifies the prefix of the secret to read to get Variables. If set to None (null), requests for configurations will not be sent to AWS Secrets Manager
profile_namestrThe name of a profile to use. If not given, then the default profile is used.
sepstrseparator used to concatenate secret_prefix and secret_id. Default: "/"

Documentation

Retrieves Connection or Variables from AWS Secrets Manager

Configurable via airflow.cfg like so:

[secrets] backend = airflow.providers.amazon.aws.secrets.secrets_manager.SecretsManagerBackend backend_kwargs = {"connections_prefix": "airflow/connections"}

For example, if secrets prefix is airflow/connections/smtp_default, this would be accessible if you provide {"connections_prefix": "airflow/connections"} and request conn_id smtp_default. If variables prefix is airflow/variables/hello, this would be accessible if you provide {"variables_prefix": "airflow/variables"} and request variable key hello. And if config_prefix is airflow/config/sql_alchemy_conn, this would be accessible if you provide {"config_prefix": "airflow/config"} and request config key sql_alchemy_conn.

You can also pass additional keyword arguments like aws_secret_access_key, aws_access_key_id or region_name to this class and they would be passed on to Boto3 client.

Example DAGs

Improve this module by creating an example DAG.

View Source
  1. Add an `example_dags` directory to the top-level source of the provider package with an empty `__init__.py` file.
  2. Add your DAG to this directory. Be sure to include a well-written and descriptive docstring
  3. Create a pull request against the source code. Once the package gets released, your DAG will show up on the Registry.

Was this page helpful?