append

AstroCertified

Append the results of a source table onto a target table. This function assumes there are no conflicts between the schemas of both tables.

View on GitHub

Last Updated: Dec. 16, 2021

Access Instructions

Install the Astro provider package into your Airflow environment.

Import the module into your DAG file and instantiate it with your desired params.

Parameters

append_tableRequiredastro.sql.table.TableThe Table containing the rows to append to a target table.
main_tableRequiredastro.sql.table.TableThe target ``Table`` to which rows should be appended.
columnsList[str]A list of columns to use for the append operation. If no columns are specified, all columns from the append_table will be used.
casted_columnsdictA mapping of column:datatype pairs which will be used to cast columns as particular datatypes when appending i.e. ``CAST(column AS datatype)``.

Documentation

Having transformed a table, you might want to append the results to a reporting table. An example of this might be to aggregate daily data on a "main" table that analysts use for timeseries analysis. The append function merges tables assuming that there are no conflicts. You can choose to merge the data 'as-is' or cast it to a new value if needed. Note that this query will fail if there is a merge conflict.

Example:
foo = append(
conn_id="postgres_conn",
database="postgres",
append_table=APPEND_TABLE,
columns=["Bedrooms", "Bathrooms"],
casted_columns={"Age": "INTEGER"},
main_table=MAIN_TABLE,
)

Was this page helpful?