[Python] Azure DevOps API: "Expecting Value" Error and Users Not Found Despite Matching Emails

I am working on a Python script to remove direct assignments for users in Azure DevOps based on their email addresses from a CSV file. The script uses the Azure DevOps Graph API to fetch users and match their mailAddress or principalName to the emails provided in the CSV.

Problem: When I attempt to fetch users using the API endpoint https://vssps.dev.azure.com/{organization}/_apis/graph/users?api-version=7.1-preview.1, I encounter this error:

ERROR:__main__:Error fetching users: Expecting value: line 3 column 1 (char 4)
The raw response indicates an empty or malformed response from the API.


Despite ensuring the mailAddress matches (with normalized case-insensitive comparison), I always see:

WARNING:__main__:User not found in Azure DevOps: john.doe@example.com. Check casing and presence in Azure DevOps.


Note : The email john.doe@example.com exists in Azure DevOps as John.Doe@example.com.

What I’ve Tried: Normalized Emails: Converted both the CSV and fetched mailAddress to lowercase for comparison.

Fallback to principalName: Used principalName as a secondary identifier if mailAddress was not found.

Logging: Logged all fetched users and their attributes for debugging:

logger.debug(f"Fetched users: {[user.get('mailAddress') for user in users]}")
The log shows no users being fetched (Fetched 0 users from Azure DevOps).


Validated API Permissions:

The Personal Access Token (PAT) has the Graph (Read) and Manage User Entitlements scopes. The PAT works for other API calls in Azure DevOps. Script Highlights: Here’s the function to fetch users:

def fetch_all_users():
users = []
continuation_token = None
while True:
url = f"{BASE_URL}/graph/users?api-version=7.1-preview.1"
if continuation_token:
url += f"&continuationToken={continuation_token}"
try:
response = requests.get(url, headers=HEADERS, timeout=30)
logger.debug(f"Raw response: {response.status_code}, Content: {response.text}")
response.raise_for_status()
data = response.json()
users.extend(data.get("value", []))
continuation_token = data.get("continuationToken")
if not continuation_token:
break
except Exception as e:
logger.error(f"Error fetching users: {e}")
break
logger.info(f"Fetched {len(users)} users from Azure DevOps.")
return users


Environment: Organization URL: https://dev.azure.com/myorganization Base API URL: https://vssps.dev.azure.com/myorganization/_apis API Version: 7.1-preview.1 Python Version: 3.10 Questions: Why does the /graph/users endpoint return an empty or malformed response? Is there a different endpoint I should use? Is there a way to reliably fetch users with case-insensitive email matching in Azure DevOps? Could this be related to organization-level settings in Azure DevOps that restrict access to user data? How can I debug this further to ensure the users are correctly fetched and matched? Any guidance or suggestions would be greatly appreciated! Let me know if additional details are needed.

Here is the entire script :

import os
import csv
import requests
import logging
import time
from requests.exceptions import RequestException
from collections import defaultdict
from concurrent.futures import ThreadPoolExecutor

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

ORGANIZATION = "company"
PAT = "your_actual_pat_value"

CSV_FILE_PATH = "...Report.csv"
BASE_URL = "https://vssps.dev.azure.com/company/_apis"
HEADERS = {
"Content-Type": "application/json",
"Authorization": f"Basic {requests.auth._basic_auth_str('', PAT)}"
}

def validate_csv_columns(required_columns, csv_reader):
missing_columns = [col for col in required_columns if col not in csv_reader.fieldnames]
if missing_columns:
raise ValueError(f"CSV is missing required columns: {missing_columns}. Found: {csv_reader.fieldnames}")

def fetch_all_users():
users = []
continuation_token = None
while True:
url = f"{BASE_URL}/graph/users?api-version=7.1-preview.1"
if continuation_token:
url += f"&continuationToken={continuation_token}"
try:
response = requests.get(url, headers=HEADERS, timeout=30)
logger.debug(f"Raw response: {response.status_code}, Content: {response.text}")
response.raise_for_status()
data = response.json()
users.extend(data.get("value", []))
continuation_token = data.get("continuationToken")
if not continuation_token:
break
except RequestException as e:
logger.error(f"Error fetching users: {e}")
break
except ValueError as ve:
logger.error(f"Malformed JSON response: {ve}")
break
logger.info(f"Fetched {len(users)} users from Azure DevOps.")
logger.debug(f"Fetched users: {[user.get('mailAddress') for user in users]}")
return users

def get_user_descriptor_map():
users = fetch_all_users()
user_map = defaultdict(str)
for user in users:
email = user.get("mailAddress", "").strip()
principal_name = user.get("principalName", "").strip()
descriptor = user.get("descriptor")
if email:
user_map[email.lower()] = descriptor
if principal_name and principal_name.lower() not in user_map:
user_map[principal_name.lower()] = descriptor
return user_map

def remove_direct_assignment(user_descriptor):
url = f"{BASE_URL}/graph/memberships/{user_descriptor}?api-version=7.1-preview.1"
try:
response = requests.delete(url, headers=HEADERS)
response.raise_for_status()
if response.status_code == 204:
logger.info(f"Successfully removed direct assignment for user: {user_descriptor}")
except RequestException as e:
logger.error(f"Error removing direct assignment for {user_descriptor}: {e}")

def process_users_in_parallel(user_descriptors):
with ThreadPoolExecutor(max_workers=10) as executor:
executor.map(remove_direct_assignment, user_descriptors)

def process_csv():
try:
user_map = get_user_descriptor_map()
with open(CSV_FILE_PATH, mode='r') as file:
csv_reader = csv.DictReader(file)
validate_csv_columns(["UserEmail"], csv_reader)
user_descriptors = []
for row in csv_reader:
email = row.get("UserEmail", "").strip().lower()
if not email:
logger.warning("Skipping row with missing UserEmail field")
continue
logger.info(f"Processing user: {email}")
user_descriptor = user_map.get(email)
if user_descriptor:
user_descriptors.append(user_descriptor)
else:
logger.warning(f"User not found in Azure DevOps: {email}. Check casing and presence in Azure DevOps.")
process_users_in_parallel(user_descriptors)
except FileNotFoundError:
logger.error(f"CSV file not found: {CSV_FILE_PATH}")
except ValueError as ve:
logger.error(f"CSV validation error: {ve}")
except Exception as e:
logger.error(f"Unexpected error: {e}")

if __name__ == "__main__":
try:
logger.info("Starting the process to remove direct assignments.")
process_csv()
logger.info("Script completed successfully.")
import sys
sys.exit(0)
except Exception as e:
logger.error(f"Script terminated with errors: {e}")
sys.exit(1)

Continue reading...