[Python] Get package level logs into webservice between logging and structlog

So the situation is the following:

• multiple internal packages written in python which do rarely log and if only use, these use the basic python logging module, since this is as far as I have read best practice see here
• now since we also want to have a twelve factor app and only have json logs in our Kubernetes cluster. Our python services use structlog. We are very happy with structlog, its easy configuration and use

But now we want to see the error logs which are made in the packages and see them in a best case as json log entry, but atleast in some way even if it means just a blank string. Currently the package logs are not logged to std.out when we run the service and trigger the functionality.

Here is an example of the package logging.logger config:

import logging
logger = logging.getLogger(__name__)

def anyfunc():
logger.info("hello world")



This is our configuration of the structlog logger in one of the webservices:

import structlog

structlog.configure(
processors=[
structlog.stdlib.add_log_level,
structlog.processors.ExceptionRenderer(),
structlog.processors.TimeStamper("iso", key="ts"),
structlog.processors.EventRenamer("msg"),
structlog.processors.JSONRenderer(sort_keys=True),
],
cache_logger_on_first_use=True,
)

logger = structlog.get_logger()



As said we want to see the logs from the package but in some way maybe structlog is surpressing the standard logger? Any help would be appreciated. I wonder that I could not find a question about this. Is this a bad setup because I would assume it is quite common, structlog for services and logging for packages(in case other people do not want to use structlog).

Any help is welcome.

Continue reading...