[Python] Passing parameters from AWS EventBridge Pipe to ECS task read by a python application

Discussão em 'Python' iniciado por Stack, Março 24, 2025.

  1. Stack

    Stack Membro Participativo

    I am totally at a loss trying to figure out a seemingly simple thing and all searches are taking me in different directions, except offering the clear solution.

    I have a python application (not flask/fastapi based) that looks for a command line parameter (an event message from SQS) and process it. I have setup an eventbridge pipe, connected to the SQS queue, which should 'trigger' this Task. So, instead of, say, invoking a lambda function, I am trying to run an ECS task to process this message, on-demand.

    However, the issue I am running into is the eventbridge pipe is not passing the message to the application. Here are the relevant portions of the code and configurations:

    if __name__ == '__main__':
    if len(sys.argv) > 1:
    foobar (sys.argv[1])
    else:
    print("No command line arguments were passed. Nothing to do. Terminating")


    The output is always:


    No command line arguments were passed. Nothing to do. Terminating
    ​

    Eventbridge pipe is also clearing the message from the queue (so I can't even have my application fetch it from the queue).

    Here are the configurations I used. Can someone please guide me on what I seem to be missing? Most answers seem to suggest I need to edit the "Container Override" something I have not been able to find so far either in the task definition or in the eventbridge pipe spec.

    Dockerfile:(redacted)

    FROM python:3.13-alpine
    ...
    ...
    CMD ["python", "app.py"]


    Task Definition (redacted as necessary):

    {
    "taskDefinitionArn": "arn:aws:ecs:us-east-1:999999999999:task-definition/taskdefFooBar:1",
    "containerDefinitions": [
    {
    "name": "taskFooBar",
    "image": "999999999999.dkr.ecr.us-east-1.amazonaws.com/foorepo/foobar",
    "cpu": 1024,
    "memory": 2048,
    "memoryReservation": 1024,
    "portMappings": [],
    "essential": true,
    "environment": [],
    "environmentFiles": [],
    "mountPoints": [],
    "volumesFrom": [],
    "ulimits": [],
    "logConfiguration": {
    "logDriver": "awslogs",
    "options": {
    "awslogs-group": "/loggroupFooBar",
    "mode": "non-blocking",
    "awslogs-create-group": "true",
    "max-buffer-size": "25m",
    "awslogs-region": "us-east-1",
    "awslogs-stream-prefix": "ecs"
    },
    "secretOptions": []
    },
    "systemControls": []
    }
    ],
    "family": "taskdefFooBar",
    "taskRoleArn": "arn:aws:iam::999999999999:role/roleFoobar",
    "executionRoleArn": "arn:aws:iam::999999999999:role/foobarExecutionRole",
    "networkMode": "awsvpc",
    "revision": 1,
    "volumes": [],
    "status": "ACTIVE",
    "requiresAttributes": [
    {
    "name": "com.amazonaws.ecs.capability.logging-driver.awslogs"
    },
    {
    "name": "ecs.capability.execution-role-awslogs"
    },
    {
    "name": "com.amazonaws.ecs.capability.ecr-auth"
    },
    {
    "name": "com.amazonaws.ecs.capability.docker-remote-api.1.19"
    },
    {
    "name": "com.amazonaws.ecs.capability.docker-remote-api.1.28"
    },
    {
    "name": "com.amazonaws.ecs.capability.docker-remote-api.1.21"
    },
    {
    "name": "com.amazonaws.ecs.capability.task-iam-role"
    },
    {
    "name": "ecs.capability.execution-role-ecr-pull"
    },
    {
    "name": "com.amazonaws.ecs.capability.docker-remote-api.1.18"
    },
    {
    "name": "ecs.capability.task-eni"
    },
    {
    "name": "com.amazonaws.ecs.capability.docker-remote-api.1.29"
    }
    ],
    "placementConstraints": [],
    "compatibilities": [
    "EC2",
    "FARGATE"
    ],
    "requiresCompatibilities": [
    "FARGATE"
    ],
    "cpu": "1024",
    "memory": "2048",
    "runtimePlatform": {
    "cpuArchitecture": "X86_64",
    "operatingSystemFamily": "LINUX"
    },
    "registeredAt": "2025-03-23T03:55:41.028Z",
    "registeredBy": "arn:aws:iam::999999999999:user/userme",
    "enableFaultInjection": false,
    "tags": []
    }


    Event Bridge Configuration:

    {
    "AWSTemplateFormatVersion": "2010-09-09",
    "Description": "CloudFormation template for EventBridge PipeFooBar",
    "Resources": {
    "Pipexxxxxxxx": {
    "Type": "AWS::pipes::pipe",
    "DependsOn": [],
    "Properties": {
    "RoleArn": "arn:aws:iam::999999999999:role/service-role/Amazon_EventBridge_Pipe_FooBar_5aaaa999",
    "Name": "ebFooBar",
    "Description": "Used to invoke ECS task when there is a new message in SQS",
    "DesiredState": "RUNNING",
    "Source": {
    "Fn::Sub": "arn:${AWS::partition}:sqs:${AWS::Region}:${AWS::AccountId}:queueFoo"
    },
    "SourceParameters": {
    "SqsQueueParameters": {
    "BatchSize": 1
    }
    },
    "LogConfiguration": {
    "CloudwatchLogsLogDestination": {
    "LogGroupArn": {
    "Fn::Sub": "arn:${AWS::partition}:logs:${AWS::Region}:${AWS::AccountId}:log-group:/logFoobars"
    }
    },
    "Level": "INFO"
    },
    "TargetParameters": {
    "InputTemplate": "'<$.body>'",
    "EcsTaskParameters": {
    "TaskDefinitionArn": "arn:aws:ecs:us-east-1:999999999999:task-definition/taskdefFooBar",
    "TaskCount": 1,
    "LaunchType": "FARGATE",
    "NetworkConfiguration": {
    "awsvpcConfiguration": {
    "Subnets": ["subnet-xxxxxxxxxxxxxxxxx"],
    "SecurityGroups": ["sg-yyyyyyyyyyyyyyyyy"],
    "AssignPublicIp": "ENABLED"
    }
    },
    "EnableECSManagedTags": true,
    "EnableExecuteCommand": false
    }
    },
    "Target": {
    "Fn::Sub": "arn:${AWS::partition}:ecs:${AWS::Region}:${AWS::AccountId}:cluster/clstrFooBar"
    }
    }
    }
    },
    "Parameters": {}
    }


    Update: I've looked to see if the environment variables has the event message, but no luck.

    Continue reading...

Compartilhe esta Página