[Python] requests_html html.render() works on Ubuntu laptop but not on headless Raspberry Pi

I've written a program that uses requests_html to fetch, render Javascript, and then pick specific bits (using CSS selectors) out of certain web pages. It works on my Ubuntu laptop but not on the headless Raspberry Pi (where I want to run it as a cron job). I've done some tests in REPL and found this discrepancy:

laptop in idle3, python 3.11.0:


s = requests_html.HTMLSession()
r = s.get(url)
h = r.html
len(h.html)
342318
h.render()
len(h.html)
461598


That's what I expected -- the javascript adds the content that I want to the page.

Pi (over ssh in python 3.11.2)


>>> s = requests_html.HTMLSession()
>>> r = s.get(url)
>>> h = r.html
>>> len(h.html)
341539
>>> h.render()
>>> len(h.html)
339279


That's obviously not right!

I would be grateful for any ideas on how to fix this.

Continue reading...