[Python] How to efficiently log a byte string message in Python 3

I am trying to port a Python2 library to Python3. (My reason is a bit silly: just that Python3 ships with Ubuntu and so if my library works in Python3 then there are fewer installation steps.)

It is frustrating because all of my byte strings (i.e. bytes objects) are getting escaped, padded with b'', and converted to Unicodes (i.e. str objects) and back via decoding and re-encoding. For example, I have some simple log statements with byte string parameters:

# some application stuff
tar_inc_page = 1024
elem = lambda:None
elem.name = b'_demo_vec'
elem.base = 0x2000003c
elem.size = 4

#logging
import logging
logging.basicConfig(level=logging.INFO)
LOG = logging.getLogger(__name__)
LOG.info(b"using tar inc page size of %d" % tar_inc_page)
LOG.info(b"%8s = 0x%08x + 0x%04x" % (elem.name, elem.base, elem.size))


In Python2 I get:

INFO:__main__:using tar inc page size of 1024
INFO:__main__:_demo_vec = 0x2000003c + 0x0004


But in Python3 I get:

INFO:__main__:b'using tar inc page size of 1024'
INFO:__main__:b'_demo_vec = 0x2000003c + 0x0004'


It looks like Python3 is adding the b'' to my byte string, escaping all non-ascii characters (as well as control characters and 0x7f), converting the result to a Unicode, then UTF-8 encoding the result for my terminal. How do I stop the lavish munging and extra conversions? I just want to send the original byte strings to the file or terminal as it worked in Python2.

My byte strings are UTF-8 encoded, but 99.99% of software shouldn't care what encoding my file, socket, or terminal is using because I already know that my files, sockets, and terminals all use the same encoding. For example, I use the cat utility to send files to terminals. Furthermore, any encoding/decoding of strings other than for prettyprinting or text editing is inefficient, bug-prone, and general evidence of poor design. 99.99% of string operations can be done perfectly well on byte strings.

Does Python3's built-in logging module provide an option to send the byte strings directly to the output? Otherwise is there a Python2-compatible logging module I can use instead of the one that ships with Python3?

Continue reading...