[Python] How do I scrape product links from Kroger using Playwright inside FastAPI on Windows...

I'm building a FastAPI app on Windows 10 using Python 3.11, and I want to scrape product links from the Kroger website using Playwright. I'm using the synchronous Playwright API (sync_playwright) inside a blocking function, which is then called via loop.run_in_executor() in an async FastAPI route.

Here’s a simplified version of my setup:

from fastapi import FastAPI
import asyncio
from playwright.sync_api import sync_playwright

app = FastAPI()

def sync_search_kroger(query: str):
with sync_playwright() as p:
browser = p.chromium.launch(headless=True)
page = browser.new_page()
page.goto(f"https://www.kroger.com/search?query={query}")
# extract product links...
browser.close()

@app.get("/search_kroger")
async def search_kroger(query: str):
loop = asyncio.get_event_loop()
return await loop.run_in_executor(None, sync_search_kroger, query)


But when I run it, I get the following error:

NotImplementedError
File ".../playwright/_impl/_transport.py", line ...
self._proc = await asyncio.create_subprocess_exec(...)

What I’ve Tried:

• I know that ProactorEventLoop is the default on Windows and doesn't support create_subprocess_exec.
• I tried setting asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy()), but it felt hacky and possibly unsafe with FastAPI and Uvicorn.
• I want to keep scraping product links (e.g., URLs from .kds-Card a[data-qa="product-card-link"]) from Kroger.

✅ What’s the Best Way to Do This?

• How can I properly use Playwright inside FastAPI on Windows to avoid this error?
• Should I switch to async_playwright()?
• Can you provide a working example that returns product links from a Kroger search page?

️ System Details:

• Python 3.11
• FastAPI + Uvicorn
• Windows 10
• Playwright v1.43+
• Goal: Extract product links like https://www.kroger.com/p/{upc} from search results

Continue reading...