[Python] Selenium wire failing to intercept request randomly

I have a scraping script that navigates to a website that contains the widget I want to scrape. The driver clicks a button that should trigger a xhr request and has never failed to do so in my manual testing but it will randomly not be able to find the request and I cannot figure out why.

browser.scopes = [
".url I am lookinmg for.*"
]
del browser.requests
utils.random_sleep(user_config, "scroll_sleep_time")
browser.get('https://www.' + url)

visit_types = utils.go_to_date_page(browser, wait, user_config, initial_steps, pid)

utils.random_sleep(user_config, "page_wait_time")
try:
WebDriverWait(browser, 10).until(
EC.presence_of_element_located((By.ID, "main-view"))
)
logger.info("Element with id 'main-view' is present — likely safe to fetch network requests.")
except TimeoutException:
logger.warning("Element with id 'main-view' did not appear in time.")
booking_requests = [
request for request in browser.requests
if request.url == user_config.get("urls").get("booking_request_url")
]

if len(booking_requests)==0:

logger.warning("No booking requests found — refreshing browser and retrying")
browser.refresh()
utils.random_sleep(user_config, "page_wait_time")
attempt += 1
continue

request = booking_requests[0]


I even have it checking if the page is fully loaded before trying to grab the URL yet this does not stop it. I can't predict when it is going to happen and once it does happen refreshing the browser and retrying does not lead to the request being found.

Continue reading...