VaultBackend

Hashicorp

Retrieves Connections and Variables from Hashicorp Vault.

View Source

Last Updated: Nov. 20, 2020

Access Instructions

Install the Hashicorp provider package into your Airflow environment.

Update your airflow.cfg per the instructions in the docs below.

Parameters

connections_pathstrSpecifies the path of the secret to read to get Connections. (default: 'connections'). If set to None (null), requests for connections will not be sent to Vault.
variables_pathstrSpecifies the path of the secret to read to get Variable. (default: 'variables'). If set to None (null), requests for variables will not be sent to Vault.
config_pathstrSpecifies the path of the secret to read Airflow Configurations (default: 'config'). If set to None (null), requests for configurations will not be sent to Vault.
urlstrBase URL for the Vault instance being addressed.
auth_typestrAuthentication Type for Vault. Default is token. Available values are: ('approle', 'aws_iam', 'azure', 'github', 'gcp', 'kubernetes', 'ldap', 'radius', 'token', 'userpass')
auth_mount_pointstrIt can be used to define mount_point for authentication chosen Default depends on the authentication method used.
mount_pointstrThe "path" the secret engine was mounted on. Default is "secret". Note that this mount_point is not used for authentication if authentication is done via a different engine. For authentication mount_points see, auth_mount_point.
kv_engine_versionintSelect the version of the engine to run (1 or 2, default: 2).
tokenstrAuthentication token to include in requests sent to Vault. (for token and github auth_type)
token_pathstrpath to file containing authentication token to include in requests sent to Vault (for token and github auth_type).
usernamestrUsername for Authentication (for ldap and userpass auth_type).
passwordstrPassword for Authentication (for ldap and userpass auth_type).
key_idstrKey ID for Authentication (for aws_iam and ''azure`` auth_type).
secret_idstrSecret ID for Authentication (for approle, aws_iam and azure auth_types).
role_idstrRole ID for Authentication (for approle, aws_iam auth_types).
kubernetes_rolestrRole for Authentication (for kubernetes auth_type).
kubernetes_jwt_pathstrPath for kubernetes jwt token (for kubernetes auth_type, default: /var/run/secrets/kubernetes.io/serviceaccount/token).
gcp_key_pathstrPath to Google Cloud Service Account key file (JSON) (for gcp auth_type). Mutually exclusive with gcp_keyfile_dict.
gcp_keyfile_dictdictDictionary of keyfile parameters. (for gcp auth_type). Mutually exclusive with gcp_key_path.
gcp_scopesstrComma-separated string containing OAuth2 scopes (for gcp auth_type).
azure_tenant_idstrThe tenant id for the Azure Active Directory (for azure auth_type).
azure_resourcestrThe configured URL for the application registered in Azure Active Directory (for azure auth_type).
radius_hoststrHost for radius (for radius auth_type).
radius_secretstrSecret for radius (for radius auth_type).
radius_portstrPort for radius (for radius auth_type).

Documentation

Retrieves Connections and Variables from Hashicorp Vault.

Configurable via airflow.cfg as follows:

[secrets] backend = airflow.providers.hashicorp.secrets.vault.VaultBackend backend_kwargs = { "connections_path": "connections", "url": "http://127.0.0.1:8200", "mount_point": "airflow" }

For example, if your keys are under connections path in airflow mount_point, this would be accessible if you provide {"connections_path": "connections"} and request conn_id smtp_default.

Example DAGs

Improve this module by creating an example DAG.

View Source
  1. Add an `example_dags` directory to the top-level source of the provider package with an empty `__init__.py` file.
  2. Add your DAG to this directory. Be sure to include a well-written and descriptive docstring
  3. Create a pull request against the source code. Once the package gets released, your DAG will show up on the Registry.

Was this page helpful?