Monitor a Livy sessions for termination.

View Source

Last Updated: Mar. 17, 2021

Access Instructions

Install the Livy provider package into your Airflow environment.

Import the module into your DAG file and instantiate it with your desired params.


livy_conn_idstrreference to a pre-defined Livy connection
batch_idUnion[int, str]identifier of the monitored batch
extra_optionsA dictionary of options, where key is string and value depends on the option that's being modified.


Monitor a Livy sessions for termination.

Example DAGs

Improve this module by creating an example DAG.

View Source
  1. Add an `example_dags` directory to the top-level source of the provider package with an empty `` file.
  2. Add your DAG to this directory. Be sure to include a well-written and descriptive docstring
  3. Create a pull request against the source code. Once the package gets released, your DAG will show up on the Registry.

Was this page helpful?