PostgresOperator

Postgres

Executes sql code in a specific Postgres database

View on GitHub

Last Updated: Oct. 31, 2022

Access Instructions

Install the Postgres provider package into your Airflow environment.

Import the module into your DAG file and instantiate it with your desired params.

Parameters

sqlthe SQL code to be executed as a single string, or a list of str (sql statements), or a reference to a template file. Template references are recognized by str ending in ‘.sql’
postgres_conn_idThe postgres conn id reference to a specific postgres database.
autocommitif True, each command is automatically committed. (default value: False)
parameters(optional) the parameters to render the SQL query with.
databasename of database which overwrite defined one in connection
runtime_parametersa mapping of runtime params added to the final sql being executed. For example, you could set the schema via {“search_path”: “CUSTOM_SCHEMA”}.

Documentation

Executes sql code in a specific Postgres database

Example DAGs

Execute Parameterized Queries in Greenplum

A DAG that runs multiple, parameterized dynamic queries against a Greenplum database based on a list of provided states using the PostgresOperator, and then sends an email upon successful completion u…

Apache Airflow Postgres
Databases
Perform Data Quality Checks in Redshift with Airflow

An example pipeline demonstrating how to perform data quality checks in Redshift using SQL Check Operators.

Apache Airflow AmazonPostgres
Data Management & GovernanceETL/ELTDatabases
CrateDB Table Export

This DAG performs a daily export from a CrateDB time-series table to an S3 bucket as JSON files.

Apache Airflow Postgres
Data Management & GovernanceStorageDatabasesWork Management
Great Expectations Redshift Example

Example DAG showcasing loading and data quality checking with Redshift and Great Expectations.

Apache Airflow Great ExpectationsAmazonPostgres
Data QualityETL/ELT
Perform Data Integrity Checks from S3 to Redshift

This is the second in a series of DAGs showing an EL pipeline with data integrity checking of data in S3 as well as Redshift.

Apache Airflow AmazonPostgres
Data Management & GovernanceETL/ELTDatabases
Advanced Data Integrity Checks from S3 to Redshift

This is the third in a series of DAGs showing an EL pipeline with data integrity and data quality checking for data in S3 and Redshift using ETag verification and row-based data quality checks where t…

Apache Airflow AmazonPostgres
Data Management & GovernanceETL/ELTDatabases
Example Reporting DAG

Example DAG that queries the Airflow Metadata Database and moves data through GCS to Postgres.

PostgresGoogle
ETL/ELTDatabases
Crate Data Quality Checks DAG

Imports local files to S3, then to CrateDB and checks several data quality properties

PostgresAmazonSlackCommon SQL
ETL/ELTData QualityDatabases

Was this page helpful?