Connection to Druid overlord for ingestion

View Source

Last Updated: Apr. 5, 2021

Access Instructions

Install the Druid provider package into your Airflow environment.

Import the module into your DAG file and instantiate it with your desired params.


druid_ingest_conn_idstrThe connection id to the Druid overlord machine which accepts index jobs
timeoutintThe interval between polling the Druid job for the status of the ingestion job. Must be greater than or equal to 1
max_ingestion_timeintThe maximum ingestion time before assuming the job failed


Connection to Druid overlord for ingestion

To connect to a Druid cluster that is secured with the druid-basic-security extension, add the username and password to the druid ingestion connection.

Example DAGs

Improve this module by creating an example DAG.

View Source
  1. Add an `example_dags` directory to the top-level source of the provider package with an empty `__init__.py` file.
  2. Add your DAG to this directory. Be sure to include a well-written and descriptive docstring
  3. Create a pull request against the source code. Once the package gets released, your DAG will show up on the Registry.

Was this page helpful?