GreatExpectationsOperator

Great Expectations

An operator to leverage Great Expectations as a task in your Airflow DAG.

View on GitHub

Last Updated: Mar. 13, 2022

Access Instructions

Install the Great Expectations provider package into your Airflow environment.

Import the module into your DAG file and instantiate it with your desired params.

Parameters

run_nameOptional[str]Identifies the validation run (defaults to timestamp if not specified)
data_context_root_dirOptional[str]Path of the great_expectations directory
data_context_configOptional[DataContextConfig]A great_expectations DataContextConfig object
checkpoint_nameOptional[str]A Checkpoint name to use for validation
checkpoint_configOptional[CheckpointConfig]A great_expectations CheckpointConfig object to use for validation
checkpoint_kwargsOptional[Dict]A dictionary whose keys match the parameters of CheckpointConfig which can be used to update and populate the Operator’s Checkpoint at runtime
fail_task_on_validation_failureOptiopnal[bool]Fail the Airflow task if the Great Expectation validation fails
validation_failure_callbackCallable[[CheckpointResult], None]Called when the Great Expectations validation fails
return_json_dictboolIf True, returns a json-serializable dictionary instead of a CheckpointResult object
**kwargsOptional[dict]kwargs

Documentation

An operator to leverage Great Expectations as a task in your Airflow DAG.

Current list of expectations types: https://docs.greatexpectations.io/en/latest/reference/glossary_of_expectations.html

How to create expectations files: https://docs.greatexpectations.io/en/latest/guides/tutorials/how_to_create_expectations.html

Was this page helpful?