BigQueryIntervalCheckOperator

Google

Checks that the values of metrics given as SQL expressions are within a certain tolerance of the ones from days_back before.

View Source

Last Updated: May. 7, 2021

Access Instructions

Install the Google provider package into your Airflow environment.

Import the module into your DAG file and instantiate it with your desired params.

Parameters

tablestrthe table name
days_backintnumber of days between ds and the ds we want to check against. Defaults to 7 days
metrics_thresholddicta dictionary of ratios indexed by metrics, for example 'COUNT(*)': 1.5 would require a 50 percent or less difference between the current day, and the prior days_back.
use_legacy_sqlboolWhether to use legacy SQL (true) or standard SQL (false).
gcp_conn_idstr(Optional) The connection ID used to connect to Google Cloud.
bigquery_conn_idstr(Deprecated) The connection ID used to connect to Google Cloud. This parameter has been deprecated. You should pass the gcp_conn_id parameter instead.
locationstrThe geographic location of the job. See details at: https://cloud.google.com/bigquery/docs/locations#specifying_your_location
impersonation_chainUnion[str, Sequence[str]]Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in the list, which will be impersonated in the request. If set as a string, the account must grant the originating account the Service Account Token Creator IAM role. If set as a sequence, the identities from the list must grant Service Account Token Creator IAM role to the directly preceding identity, with first account from the list granting this role to the originating account (templated).

Documentation

Checks that the values of metrics given as SQL expressions are within a certain tolerance of the ones from days_back before.

This method constructs a query like so

SELECT {metrics_threshold_dict_key} FROM {table}
WHERE {date_filter_column}=<date>

See also

For more information on how to use this operator, take a look at the guide: Compare metrics over time

Was this page helpful?