AwsGlueCrawlerOperator

Amazon

Creates, updates and triggers an AWS Glue Crawler. AWS Glue Crawler is a serverless service that manages a catalog of metadata tables that contain the inferred schema, format and data types of data stores within the AWS cloud.

View on GitHub

Last Updated: May. 7, 2021

Access Instructions

Install the Amazon provider package into your Airflow environment.

Import the module into your DAG file and instantiate it with your desired params.

Parameters

configdictConfigurations for the AWS Glue crawler
aws_conn_idOptional[str]aws connection to use
poll_intervalOptional[int]Time (in seconds) to wait between two consecutive calls to check crawler status

Documentation

Creates, updates and triggers an AWS Glue Crawler. AWS Glue Crawler is a serverless service that manages a catalog of metadata tables that contain the inferred schema, format and data types of data stores within the AWS cloud.

Was this page helpful?