[Python] Microsoft Forms API on Databricks - JWE token validation

I'm retrieving data from Microsoft Forms using the Forms API, which is undocumented (I found the endpoint here, but there's no official documentation). I have several flows running that pull data from Forms into Databricks. However, starting on Wednesday, it suddenly stopped working and began throwing a 403 Forbidden error:

Access token obtained: <class 'azure.core.credentials.AccessToken'> Request failed: 403 Client Error: Forbidden for url: https://forms.office.com/formapi/api/[REDACTED]/users/[REDACTED]/light/forms('[REDACTED]')/responses?$expand=comments&$top=7&$skip=0

Below is the code I’m using:

# Start SparkSession
spark = SparkSession.builder.appName("FormsAPI").getOrCreate()

# Get access token
try:
cred = DefaultAzureCredential()
scope = "https://forms.office.com/.default"
tok = cred.get_token(scope)
print(f"Access token obtained: {type(tok)}")
except Exception as e:
print("Failed to acquire token:", e)
exit(1)

# Check environment variables
if not all([forms_id, tenantid, userid]):
print("One or more environment variables are not set.")
exit(1)

# API URL
url = f"https://forms.office.com/formapi/api/{tenantid}/users/{userid}/light/forms('{forms_id}')/responses?$expand=comments&$top=7&$skip=0"

# Set access token in the header
headers = {
"Authorization": f"Bearer {tok.token}",
"Accept": "application/json",
"Content-Type": "application/json"
}

def process_responses_with_nulls(response_json):
# Identify all possible question IDs
all_question_ids = set()
rows = []

for item in response_json.get("value", []):
# Base response data
response_data = {
"id": item.get("id"),
"releaseDate": item.get("releaseDate"),
"startDate": item.get("startDate"),
"submitDate": item.get("submitDate"),
"responder": item.get("responder"),
"responderName": item.get("responderName")
}

# Parse answers
answers = json.loads(item.get("answers", "[]"))

# Track all question IDs
all_question_ids.update(answer.get("questionId") for answer in answers)

# Process answers
for answer in answers:
question_id = answer.get("questionId")
answer_value = answer.get("answer1", None)
response_data[question_id] = answer_value

# Ensure all possible question IDs are in the response
for qid in all_question_ids:
if qid not in response_data:
response_data[qid] = None

rows.append(Row(**response_data))

return rows, all_question_ids

# Make the API request
try:
r = requests.get(url, headers=headers)
r.raise_for_status()
response_json = r.json()

# Process responses with null-filling
rows, question_ids = process_responses_with_nulls(response_json)

# Create DataFrame
df = spark.createDataFrame(rows)

except requests.exceptions.RequestException as e:
print("Request failed:", str(e))
except Exception as e:
print("Error processing responses:", str(e))


Any ideas why this suddenly started returning 403? Could it be a change in token scope, permissions, or the (unofficial) Forms API itself?

NOTE: No changes on Azure App registration either.

Continue reading...