[Python] In GA4 API, how do you get page referral information? All queries are coming up null

What I'm trying to do:

I want to query GA4 to get a matrix of the top 1000 pages that match a specific regex, which is this:

r"^/page/[a-zA-Z]+$"

I have run other queries with this regex, like getting unique visitors per day per page, and that's no problem.

But now I'm trying to do something more ambitious, and every solution has failed.

The output I want

A table in which each row and column is a page path that meets the above regex. The cell value is the amount of traffic referred from the page path of the row to the page path of the column.

What I've tried

I've tried using things like "pageReferrer" and "lastPagePath" but both have not worked. And I've run my script through 3 different AIs which have just made things worse.

The problem

Every time I execute the script, it exports an empty csv file. No data. It's not a bad regex (which AI keeps suggesting when I ask it). Those are legit pages. I am starting to wonder if there is something about Google Analytics that doesn't allow referral info.

Current script

from google.analytics.data_v1beta.types import RunReportRequest, DateRange, Dimension, Metric, Filter, FilterExpression
import pandas as pd
from collections import defaultdict

# Initialize GA4 API client
client = BetaAnalyticsDataClient(credentials=creds)

# GA4 property ID
property_id = "properties/1234567890"

# Step 1: Get top 1000 page paths matching the pattern ^/name/[a-zA-Z]+$
regex_pattern = r"^/page/[a-zA-Z]+$"
request_top = RunReportRequest(
property=property_id,
date_ranges=[DateRange(start_date="2022-08-01", end_date="2025-06-15")],
dimensions=[Dimension(name="pagePath")],
metrics=[Metric(name="screenPageViews")],
dimension_filter=FilterExpression(
filter=Filter(
field_name="pagePath",
string_filter=Filter.StringFilter(
value=regex_pattern,
match_type=Filter.StringFilter.MatchType.MATCH_TYPE_UNSPECIFIED
)
)
),
limit=1000,
order_bys=[
{"metric": {"metric_name": "screenPageViews"}, "desc": True}
]
)

response_top = client.run_report(request_top)
top_pages = [row.dimension_values[0].value for row in response_top.rows]
print(f"Top {len(top_pages)} pages:")
print(top_pages[:10])

# Step 2: Get referral traffic between these top pages
page_set = set(top_pages)
traffic_data = defaultdict(lambda: defaultdict(int))

for to_page in top_pages:
print(f"Checking referrals to: {to_page}")
filter_expression = FilterExpression(
and_group=FilterExpression.AndGroup(
expressions=[
FilterExpression(
filter=Filter(
field_name="pagePath",
string_filter=Filter.StringFilter(value=to_page, match_type=Filter.StringFilter.MatchType.EXACT)
)
),
FilterExpression(
filter=Filter(
field_name="previousPagePath",
in_list_filter=Filter.InListFilter(values=list(page_set))
)
)
]
)
)

request = RunReportRequest(
property=property_id,
date_ranges=[DateRange(start_date="2022-08-01", end_date="2025-06-15")],
dimensions=[Dimension(name="previousPagePath")],
metrics=[Metric(name="eventCount")],
dimension_filter=filter_expression,
limit=10000
)

response = client.run_report(request)
print(f"Returned {len(response.rows)} referrer rows")
for row in response.rows:
from_page = row.dimension_values[0].value
if from_page in page_set:
traffic_data[from_page][to_page] += int(row.metric_values[0].value)

# Step 3: Build matrix DataFrame
traffic_df = pd.DataFrame(index=top_pages, columns=top_pages).fillna(0)

for from_page in top_pages:
for to_page in top_pages:
traffic_df.at[from_page, to_page] = traffic_data[from_page][to_page]

# Save to CSV
traffic_df.to_csv("referral_matrix.csv")
print("Saved referral matrix to referral_matrix.csv")```

Continue reading...