[Python] Why does importlib.resources read_text raise UnicodeDecodeError on Windows but not...

I am writing a small web app (https://github.com/r-owen/base_loom_server) and it runs on macOS and Raspberry Pi, but a Windows user reports an exception when Python reads a UTF-8-encoded resource file using importlib.resources.files.read_text() with no arguments. The resource is JavaScript, but I doubt it matters. This is the error:

UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 20834: character maps to <undefined>
​

I believe it is complaining about the first of four arrow characters in that file: "←" = LEFTWARDS ARROW. The file has the same style of arrow in all 4 directions: U+2190 through U+2193. Those are the only non-ASCII characters in the file (other than a few unnecessary • in a comment earlier in the file, which I will replace with *).

With no arguments, read_file decodes using UTF-8 in strict mode, which is what I want.

I would like to understand what's going on so I can work around this, plus any potential problems with letters that have diacritical marks (because the app also uses read_file to load language translation files as needed).

The failure occurs here

Continue reading...