GreatExpectationsOperator

Great Expectations

An operator to leverage Great Expectations as a task in your Airflow DAG.

View on GitHub

Last Updated: Aug. 23, 2021

Access Instructions

Install the Great Expectations provider package into your Airflow environment.

Import the module into your DAG file and instantiate it with your desired params.

Parameters

run_nameOptional[str]Identifies the validation run (defaults to timestamp if not specified)
data_context_root_dirOptional[str]Path of the great_expectations directory
data_contextOptional[BaseDataContext]A great_expectations DataContext object
expectation_suite_nameOptional[str]The name of the Expectation Suite to use for validation
batch_kwargsOptional[dict]The batch_kwargs to use for validation
assets_to_validateOptional[list[dict]]A list of dictionaries of batch_kwargs + Expectation Suites to use for validation
checkpoint_nameOptional[str]A Checkpoint name to use for validation
validation_operator_nameOptional[str]name of a Great Expectations validation operator, defaults to action_list_operator
fail_task_on_validation_failureOptiopnal[bool]Fail the Airflow task if the Great Expectation validation fails
validation_failure_callbackCallable[[CheckpointResult], None]Called when the Great Expectations validation fails
**kwargsOptional[dict]kwargs

Documentation

An operator to leverage Great Expectations as a task in your Airflow DAG.

Current list of expectations types: https://docs.greatexpectations.io/en/latest/reference/glossary_of_expectations.html

How to create expectations files: https://docs.greatexpectations.io/en/latest/guides/tutorials/how_to_create_expectations.html

Was this page helpful?