CloudSecretManagerBackend

Google

Retrieves Connection object from Google Cloud Secrets Manager

View Source

Last Updated: Mar. 6, 2021

Access Instructions

Install the Google provider package into your Airflow environment.

Update your airflow.cfg per the instructions in the docs below.

Parameters

connections_prefixstrSpecifies the prefix of the secret to read to get Connections. If set to None (null), requests for connections will not be sent to GCP Secrets Manager
variables_prefixstrSpecifies the prefix of the secret to read to get Variables. If set to None (null), requests for variables will not be sent to GCP Secrets Manager
config_prefixstrSpecifies the prefix of the secret to read to get Airflow Configurations containing secrets. If set to None (null), requests for configurations will not be sent to GCP Secrets Manager
gcp_key_pathstrPath to Google Cloud Service Account key file (JSON). Mutually exclusive with gcp_keyfile_dict. use default credentials in the current environment if not provided.
gcp_keyfile_dictdictDictionary of keyfile parameters. Mutually exclusive with gcp_key_path.
gcp_scopesstrComma-separated string containing OAuth2 scopes
project_idstrProject ID to read the secrets from. If not passed, the project ID from credentials will be used.
sepstrSeparator used to concatenate connections_prefix and conn_id. Default: "-"

Documentation

Retrieves Connection object from Google Cloud Secrets Manager

Configurable via airflow.cfg as follows:

[secrets] backend = airflow.providers.google.cloud.secrets.secret_manager.CloudSecretManagerBackend backend_kwargs = {"connections_prefix": "airflow-connections", "sep": "-"}

For example, if the Secrets Manager secret id is airflow-connections-smtp_default, this would be accessible if you provide {"connections_prefix": "airflow-connections", "sep": "-"} and request conn_id smtp_default.

If the Secrets Manager secret id is airflow-variables-hello, this would be accessible if you provide {"variables_prefix": "airflow-variables", "sep": "-"} and request Variable Key hello.

The full secret id should follow the pattern “[a-zA-Z0-9-_]”.

Example DAGs

Improve this module by creating an example DAG.

View Source
  1. Add an `example_dags` directory to the top-level source of the provider package with an empty `__init__.py` file.
  2. Add your DAG to this directory. Be sure to include a well-written and descriptive docstring
  3. Create a pull request against the source code. Once the package gets released, your DAG will show up on the Registry.

Was this page helpful?