FivetranSensor

Fivetran

FivetranSensor monitors a Fivetran sync job for completion.

View Source

Last Updated: Apr. 28, 2021

Access Instructions

Install the Fivetran provider package into your Airflow environment.

Import the module into your DAG file and instantiate it with your desired params.

Parameters

fivetran_conn_idstrConn ID of the Connection to be used to configure the hook.
connector_idstrID of the Fivetran connector to sync, found on the Connector settings page in the Fivetran Dashboard.
poke_intervalintTime in seconds that the job should wait in between each tries
fivetran_retry_limitOptional[int]# of retries when encountering API errors
fivetran_retry_delayintTime to wait before retrying API request

Documentation

FivetranSensor monitors a Fivetran sync job for completion.

Monitoring with FivetranSensor allows you to trigger downstream processes only when the Fivetran sync jobs have completed, ensuring data consistency. You can use multiple instances of FivetranSensor to monitor multiple Fivetran connectors. Note, it is possible to monitor a sync that is scheduled and managed from Fivetran; in other words, you can use FivetranSensor without using FivetranOperator. If used in this way, your DAG will wait until the sync job starts on its Fivetran-controlled schedule and then completes. FivetranSensor requires that you specify the connector_id of the sync job to start. You can find connector_id in the Settings page of the connector you configured in the Fivetran dashboard.

Example DAGs

Was this page helpful?