NamedHivePartitionSensor

Hive

Waits for a set of partitions to show up in Hive.

View on GitHub

Last Updated: May. 7, 2021

Access Instructions

Install the Hive provider package into your Airflow environment.

Import the module into your DAG file and instantiate it with your desired params.

Parameters

partition_namesRequiredlist[str]List of fully qualified names of the partitions to wait for. A fully qualified name is of the form schema.table/pk1=pv1/pk2=pv2, for example, default.users/ds=2016-01-01. This is passed as is to the metastore Thrift client get_partitions_by_name method. Note that you cannot use logical or comparison operators as in HivePartitionSensor.
metastore_conn_idstrreference to the metastore thrift service connection id

Documentation

Waits for a set of partitions to show up in Hive.

Was this page helpful?