EmrCreateJobFlowOperator

Amazon

Creates an EMR JobFlow, reading the config from the EMR connection. A dictionary of JobFlow overrides can be passed that override the config from the connection.

View Source

Last Updated: Apr. 2, 2021

Access Instructions

Install the Amazon provider package into your Airflow environment.

Import the module into your DAG file and instantiate it with your desired params.

Parameters

aws_conn_idstraws connection to uses
emr_conn_idstremr connection to use
job_flow_overridesdict|strboto3 style arguments or reference to an arguments file (must be '.json') to override emr_connection extra. (templated)

Documentation

Creates an EMR JobFlow, reading the config from the EMR connection. A dictionary of JobFlow overrides can be passed that override the config from the connection.

Was this page helpful?