[Python] Cloud Watch: Custom log stream name set by container

I am attempting to manually specify the log stream name published by AWS Batch to Cloud Watch. From docs, tutorials, etc. I have pieced together the below

def custom_logger(logger_config):
# Get the root logger
logger = logging.getLogger()
logger.handlers = []
logger.setLevel(logging.INFO)
cloudwatch_client = boto3.client(
'logs',
region_name=os.environ["AWS_REGION"],
)
job_id = os.environ.get('AWS_BATCH_JOB_ID', uuid.uuid4().hex[:8])
log_stream_name = f"{logger_config['specialId']"
watchtower_handler = watchtower.CloudWatchLogHandler(
log_group_name=os.environ["LOG_GROUP_NAME"],
log_stream_name=log_stream_name,
boto3_client=cloudwatch_client,
send_interval=1,
create_log_group=False
)
logger.addHandler(watchtower_handler)


However, when I invoke this function, something unexpected happens. Logs are no longer published at all! They once were but now don't.

Through CDK, I have set the environment variable, LOG_GROUP_NAME, to a real log group. However, I am assuming that AWS_REGION is an environment variable set by AWS.

Perhaps I am using watchtower incorrectly? It seems to be failing silently as my AWS Batch jobs are indeed succeeding; their logs are just no longer published.

Continue reading...