[Python] How to fetch flow run data from Prefect Cloud (v3.x) using Python REST API for...

I'm trying to build an Excel-based dashboard from data in Prefect Cloud (3.x).

What I'm trying to do:

Track each flow's number of runs

Count how many succeeded, failed, or are still running

Export this to an Excel file using Python (pandas, openpyxl)

My environment:

Prefect Cloud (3.x)

prefect cloud login -k <API_KEY> works successfully

I'm using a virtual environment with prefect==3.4.3, pandas, and requests

What’s not working:

1. 
   When I try to hit this URL from Python code:
   
   url = "https://api.prefect.cloud/api/accounts/<ACCOUNT_ID>/workspaces/<WORKSPACE_ID>/flow_runs/filter"
   

With this POST payload:

payload = {
"flow_filter": {
"id": {
"any_": ["<FLOW_ID>"]
}
},
"limit": 10
}


and proper Authorization headers

headers = {
"Authorization": f"Bearer {API_KEY}",
"Content-Type": "application/json"
}


Getting output as: Status code: 404 Response: {"detail":"Not Found"}

1. 
   The same URL works in browser/cURL on my local machine but fails from my VM.
   
   
2. 
   I’ve confirmed the API key is valid (cloud login works), and I’m able to run prefect flow ls and prefect flow-run ls from CLI.
   
   
3. 
   When debugging with requests, I also hit this error sometimes:
   

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

which usually means the response is empty or not JSON.


What I've tried:

Verified the full URL structure

Tested connectivity with curl and ping from my VM

Ensured Authorization: Bearer <API_KEY> header is set

Added debug logging to print response.status_code and response.text

Recreated venv, reinstalled Prefect

What I'm asking:

What's the correct way to query flow runs from Prefect Cloud 3.x using Python and REST API?

Am I missing something in headers, payload, or URL structure?

Do I need any special workspace authentication/permissions beyond API key?

Any known gotchas with VM access to Prefect Cloud API?


Any help is deeply appreciated — I’ve been banging my head on this all day

Continue reading...