QuboleFileSensor

Qubole

Wait for a file or folder to be present in cloud storage and check for its presence via QDS APIs

View Source

Last Updated: Dec. 8, 2020

Access Instructions

Install the Qubole provider package into your Airflow environment.

Import the module into your DAG file and instantiate it with your desired params.

Parameters

qubole_conn_idstrConnection id which consists of qds auth_token
datadicta JSON object containing payload, whose presence needs to be checked Check this example for sample payload structure.

Documentation

Wait for a file or folder to be present in cloud storage and check for its presence via QDS APIs

Note

Both data and qubole_conn_id fields support templating. You can also use .txt files for template-driven use cases.

Example DAGs

Was this page helpful?