[Python] GCP API Response: Control Characters Forbidden: Invalid encoding

I built an API on GCP. The health endpoint is working properly:

curl -H "Authorization: bearer $(gcloud auth print-identity-token)" \
https://my-api-123.us-central1.run.app/health
{"status":"healthy"}


The following calls are returning the error:

{"error":"control characters forbidden: invalid-encoding","success":false}
​

curl -X POST "https://my-api-123.us-central1.run.app" \
-H "Authorization: bearer $(gcloud auth print-identity-token)" \
-H "Content-Type: application/json"


Application logs include the "[Start] Starting job scraping process" message, but then an unknown issue occurs is in the scrape_all_countries() function:

def scrape_all_countries():
"""Main function to scrape all countries and all search terms"""
try:
logger.info("[START] Starting job scraping process")

conn = get_db_connection()
grand_total = 0

try:
for country in COUNTRIES:
# Scrape all term categories for this country
country_total = 0
country_total += scrape_country(country, GENERAL_TERMS, conn)
country_total += scrape_country(country, DATABASE_TERMS, conn)
country_total += scrape_country(country, ADOBE_TERMS, conn)

grand_total += country_total
logger.info(f"[SUCCESS] Country '{country}' complete: {country_total} total jobs")

# Longer delay between countries
time.sleep(5)

logger.info(f"[COMPLETE] SCRAPING COMPLETE! Grand total: {grand_total} new jobs across all countries")

except Exception as e:
# Clean error message
error_msg = str(e).encode('ascii', 'ignore').decode('ascii')
logger.error(f"[ERROR] Error during scraping: {error_msg}")
raise Exception(error_msg)
finally:
conn.close()
logger.info("[CLOSED] Database connection closed")

return {"success": True, "total_jobs": grand_total}

except Exception as e:
# Ensure error message has no control characters
error_msg = str(e).encode('ascii', 'ignore').decode('ascii')
logger.error(f"[ERROR] Fatal error in scrape_all_countries: {error_msg}")
return {"success": False, "error": error_msg}

Continue reading...