[Python] Why does pytrends return different values than Google Trends UI even after...

I'm using the pytrends Python library to collect interest-over-time data for a list of movies from multiple countries. However, the values I get in the resulting DataFrame are often different from what I see in the Google Trends web interface, even though the timeframe, region, and keywords are exactly the same.I'm using the pytrends Python library to collect interest-over-time data for a list of movies from multiple countries. However, the values I get in the resulting DataFrame are often different from what I see in the Google Trends web interface, even though the timeframe, region, and keywords are exactly the same.

I'm using classic movie titles (e.g., Casablanca, Gone with the Wind) across the US, France, and the UK with a 5-year timeframe ('today 5-y'). I’m also setting custom headers to avoid 429 errors.

What I expect: The values in the CSV to be the same as in the Google Trends UI for the same keyword, region, and timeframe.

The data from pytrends often shows different peak weeks, different values (sometimes scaled differently) abd differences across keywords when compared to the UI

Here is the code:

import pandas as pd
from pytrends.request import TrendReq as UTrendReq
import time
import random
from functools import reduce

REFERENCE_MOVIE = "Gone with the Wind"
movies = [
"Gone with the Wind",
"Casablanca",
"The Godfather",
"Citizen Kane",
"The Sound of Music",
"12 Angry Men",
"Psycho",
"Singin' in the Rain"
]
movies_nonref = [m for m in movies if m != REFERENCE_MOVIE]

countries = {
"US": "United States",
"FR": "France",
"GB": "United Kingdom"
}

category = 0
timeframe = "today 5-y"
gprop = ""

class TrendReq(UTrendReq):
def _get_data(self, url, method='get', trim_chars=0, **kwargs):
headers = {
'accept': 'application/json, text/plain, */*',
'accept-language': 'en-US,en;q=0.9',
'content-type': 'application/json;charset=UTF-8',
'origin': 'https://trends.google.com',
'referer': 'https://trends.google.com/trends/',
'user-agent': 'Mozilla/5.0'
}
return super()._get_data(url, method=method, trim_chars=trim_chars, headers=headers, **kwargs)

pytrends = TrendReq(hl='en-US', tz=360)

def chunk(lst, n):
for i in range(0, len(lst), n):
yield lst[i:i + n]

all_dfs = []

for country_code, country_name in countries.items():
batches = list(chunk(movies_nonref, 4))
for batch in batches:
batch_movies = [REFERENCE_MOVIE] + batch
for attempt in range(3):
try:
pytrends.build_payload(batch_movies, cat=category, timeframe=timeframe, geo=country_code, gprop=gprop)
df = pytrends.interest_over_time().drop(columns=['isPartial'], errors='ignore')
df = df.rename(columns={movie: f"{movie}: ({country_name})" for movie in batch_movies})
df = df.reset_index()
all_dfs.append(df)
break
except Exception as e:
if attempt < 2:
time.sleep(random.uniform(5, 10))
time.sleep(random.uniform(5, 10))

if all_dfs:
result = reduce(lambda left, right: pd.merge(left, right, on="date", how="outer"), all_dfs)
result = result.rename(columns={"date": "Week"})

movie_row = [""] + [col.split(": (")[0] for col in result.columns if col != "Week"]
country_row = ["Week"] + [col.split(": (")[1][:-1] for col in result.columns if col != "Week"]

final_df = pd.DataFrame([movie_row, country_row], columns=result.columns)
final_df = pd.concat([final_df, result], ignore_index=True)

final_df.to_csv("movie_trends_interest_over_time.csv", header=False, index=False)
else:
print("No data collected.")

Continue reading...