ECSOperator

Amazon

Execute a task on AWS ECS (Elastic Container Service)

View on GitHub

Last Updated: May. 29, 2021

Access Instructions

Install the Amazon provider package into your Airflow environment.

Import the module into your DAG file and instantiate it with your desired params.

Parameters

task_definitionRequiredstrthe task definition name on Elastic Container Service
clusterRequiredstrthe cluster name on Elastic Container Service
overridesRequireddictthe same parameter that boto3 will receive (templated): https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ecs.html#ECS.Client.run_task
aws_conn_idstrconnection id of AWS credentials / region name. If None, credential boto3 strategy will be used (http://boto3.readthedocs.io/en/latest/guide/configuration.html).
region_namestrregion name to use in AWS Hook. Override the region_name in connection (if provided)
launch_typestrthe launch type on which to run your task (‘EC2’ or ‘FARGATE’)
groupstrthe name of the task group associated with the task
placement_constraintslistan array of placement constraint objects to use for the task
placement_strategylistan array of placement strategy objects to use for the task
platform_versionstrthe platform version on which your task is running
network_configurationdictthe network configuration for the task
tagsdicta dictionary of tags in the form of {‘tagKey’: ‘tagValue’}.
awslogs_groupstrthe CloudWatch group where your ECS container logs are stored. Only required if you want logs to be shown in the Airflow UI after your job has finished.
awslogs_regionstrthe region in which your CloudWatch logs are stored. If None, this is the same as the region_name parameter. If that is also None, this is the default AWS region based on your connection settings.
awslogs_stream_prefixstrthe stream prefix that is used for the CloudWatch logs. This is usually based on some custom name combined with the name of the container. Only required if you want logs to be shown in the Airflow UI after your job has finished.
reattachboolIf set to True, will check if a task from the same family is already running. If so, the operator will attach to it instead of starting a new task.
quota_retrydictConfig if and how to retry _start_task() for transient errors.

Documentation

Execute a task on AWS ECS (Elastic Container Service)

See also

For more information on how to use this operator, take a look at the guide: ECS Operator

Was this page helpful?