DataprocJobSensor

Google

Check for the state of a previously submitted Dataproc job.

View on GitHub

Last Updated: May. 7, 2021

Access Instructions

Install the Google provider package into your Airflow environment.

Import the module into your DAG file and instantiate it with your desired params.

Parameters

project_idRequiredstrThe ID of the google cloud project in which to create the cluster. (templated)
dataproc_job_idRequiredstrThe Dataproc job ID to poll. (templated)
locationRequiredstrRequired. The Cloud Dataproc region in which to handle the request. (templated)
gcp_conn_idstrThe connection ID to use connecting to Google Cloud Platform.

Documentation

Check for the state of a previously submitted Dataproc job.

Was this page helpful?