[Python] Python3 ThreadingHTTPServer fails to send chunked encoded response

I'm implementing a simple reverse proxy in Python3 and I need to send a response with transfer-encoding chunked mode.

I've taken my cues from this post but I have some problems when sending the chunks in the format described here

If I send chunks of length <= 9 bytes the message is received correctly by the client, otherwise when sending chunks of length >= 10 bytes, it seems that some of them are not received and the message remains stuck in the client waiting indefinitely

Here is an example of non working code:

from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer


class ProxyHTTPRequestHandler(BaseHTTPRequestHandler):
protocol_version = 'HTTP/1.1'

def do_GET(self, body=True):

# HTTP 200 + minimal HTTP headers in response
self.send_response(200)
self.send_header('transfer-encoding', 'chunked')
self.send_header('Content-Type', 'text/plain')
self.end_headers()

# writing 5 chunks of 10 characters
for i in range(5):
text = str(i+1) * 10 # concatenate 10 chars
chunk = '{0:d}\r\n'.format(len(text)) + text + '\r\n'
self.wfile.write(chunk.encode(encoding='utf-8'))

# writing close sequence
close_chunk = '0\r\n\r\n'
self.wfile.write(close_chunk.encode(encoding='utf-8'))


def main():
try:
server_address = ('127.0.0.1', 8099)

# I use ThreadingHTTPServer but the problem persists also with HTTPServer
httpd = ThreadingHTTPServer(server_address, ProxyHTTPRequestHandler)
print('http server is running')
httpd.serve_forever()
except KeyboardInterrupt:
print(" ^C entered, stopping web server...")
httpd.socket.close()


if __name__ == '__main__':
main()


In this case, after several seconds, and only if I manually stop python execution the result in Postman is the following. Please note the missing "2222222222" chunk



But if I use this length instead:

# writing the same 5 chunks of 9 characters
for i in range(5):
text = str(i+1) * 9 # concatenate 9 chars
chunk = '{0:d}\r\n'.format(len(text)) + text + '\r\n'
self.wfile.write(chunk.encode(encoding='utf-8'))

# writing close sequence
close_chunk = '0\r\n\r\n'
self.wfile.write(close_chunk.encode(encoding='utf-8'))


The communication ends correctly (after 6ms all the 5 chunks are interpreted correctly)

Some version informations:

HTTP Client: Postman 8.10

(venv) manuel@MBP ReverseProxy % python -V
Python 3.9.2

(venv) manuel@MBP ReverseProxy % pip freeze
certifi==2021.10.8
charset-normalizer==2.0.6
idna==3.2
requests==2.26.0
urllib3==1.26.7


Thanks in advance for any hints!

Continue reading...