[Python] mitmproxy fails on specific WiFi networks where Charles/ProxyPin work, likely due to...

I've developed a client application that uses a mitmproxy script to intercept traffic by setting the system's HTTP proxy. On most networks, it works perfectly. However, on certain WiFi networks (e.g., some hotel WiFi), my mitmproxy setup fails while other tools like Charles or ProxyPin work flawlessly.

Here's a detailed breakdown of the problem and my observations:

The Core Problem:

When connected to these specific WiFi networks and with my mitmproxy proxy enabled, browsers show a "proxy connection failed" error. My mitmproxy instance, which is running correctly, receives no traffic. Specifically, the request() event in my addon is never triggered, and no logs are printed from it. This indicates the connection fails before the HTTP request is even processed.

My Hypothesis:

I strongly suspect this is caused by advanced network security policies on these WiFi networks, such as:

• TLS Fingerprinting (JA3): The network might be blocking clients whose TLS fingerprint matches known tools like mitmproxy/Python, while allowing fingerprints that look like mainstream browsers (which Charles/ProxyPin might be better at impersonating).
• Captive Portal / Session Initialization: The network might require a specific, implicit handshake to fully "open" the connection, which ProxyPin seems to trigger successfully.

My Question:

Given these observations, how can I make my mitmproxy script more robust to handle these kinds of restrictive network environments? Specifically:

• How can I replicate the "network initialization" behavior that ProxyPin seems to perform?
• What is the best way to modify mitmproxy's TLS fingerprint to impersonate a standard browser and bypass such network policies?

Has anyone encountered a similar scenario and found a reliable solution? Thank you!

mitmproxy==12.1.1 python==3.12.3

Key Observations & Debugging Steps:

1. 
   Partial Success with --ssl-insecure: When I run mitmproxy with the --ssl-insecure flag, I see a small improvement. A few, sporadic requests (for 2-3 specific domains) actually make it through and are logged in my request() handler. However, the vast majority of requests still fail. This suggests that SSL/TLS interception by the network is happening, but it's not the only issue.
   
   
2. 
   "Unblocking" with ProxyPin: I discovered a very strange workaround on one of the problematic hotel WiFi networks. The sequence is:
   
   • Start my mitmproxy client -> Fails.
   • Stop my client.
   • Start ProxyPin, let it successfully proxy a request, then shut it down.
   • Start my mitmproxy client again -> It now works perfectly and intercepts all traffic.
3. 
   Charles Doesn't "Unblock": Interestingly, performing the same sequence with Charles (run Charles first, then mitmproxy) does not solve the problem for mitmproxy. Mitmproxy still fails. This suggests that ProxyPin might be performing a specific network "handshake" or "initialization" that Charles and my script are not.
   

Continue reading...