[Python] 2nd gen Cloud function response size too large

I have a 2nd gen HTTP Python cloud function that queries a large Bigquery table and returns large amounts of data, like this:

from google.cloud import bigquery
import functions_framework


client = bigquery.Client()


@functions_framework.http
def qry_bq(request):

request_json = request.get_json(silent=True)

df = client.query(qry).to_dataframe()
return df.to_json()



Sometimes, when the returned results of the query are large, the response results in a 500 error with the error message:

Response size was too large. Please consider reducing response size.

I saw here that the HTTP response size is limited to 32MB. Are there any suggestions to circumvent this issue for my use case?

Continue reading...