Apache Airflow Provider - Astro

Astro

OfficialData Processing

A library for data-centric, functional DAG authoring focused on data movement and transformation.

Quick Install

Available Modules

Modules are Python callables available from this provider package.

Types:

ObjectsDecoratorsFunctions
TempTable
Astro

A metadata representation of database table within the Astro ecosystem that will not be persisted after corresponding operations are complete.

ObjectsUpdated a month ago
run_raw_sql
Astro

Execute SQL that is not expected to return data like DDL or DML operations.

DecoratorsUpdated a month ago
save_file
Astro

Export a database table as a CSV or Parquet file to local storage, Amazon S3, or Google Cloud Storage.

FunctionsUpdated a month ago
merge
Astro

Merge data into an existing table in situations where there may be conflicts. This function adds data to a table with either an "update" or "ignore" s…

FunctionsUpdated a month ago
append
Astro

Append the results of a source table onto a target table. This function assumes there are no conflicts between the schemas of both tables.

FunctionsUpdated a month ago
dataframe
Astro

Convert a Table object into a Pandas DataFrame or persist a DataFrame result to a database table.

DecoratorsUpdated a month ago
Table
Astro

A metadata representation of an existing or to-be-created database table within the Astro ecosystem.

ObjectsUpdated a month ago
aggregate_check
Astro

Validate the result from a SQL which performs an aggregation matches an expected value or falls within a provided range.

FunctionsUpdated a month ago
load_file
Astro

Load CSV or Parquet files from local storage, Amazon S3, or Google Cloud Storage into a SQL database.

FunctionsUpdated a month ago
truncate
Astro

Perform a TRUNCATE operation on a given table.

FunctionsUpdated a month ago
transform_file
Astro

Execute a SELECT SQL statement contained in a file. Data returned from this SQL is inserted into a temporary table which can used by other downstream …

FunctionsUpdated a month ago
transform
Astro

Execute an explicit, SELECT SQL statement. Data returned from this SQL is inserted into a temporary table which can used by other downstream tasks.

DecoratorsUpdated a month ago