We called the person-events API to get events data, and get "504 Gateway time-out" error after a few requests. set 5000 'offset' per request to get full events in the 24 hours. The job usually took less than 30 mins, but after 8/12, we noticed it took over 10 hours (20 hours in one of the days) to complete the job. It disrupts our downstream job executions. We found the error message in response. It's server's side error likely caused by overwhelming volume or throttling. Getting events data through the person-events API endpoint, get "504 Gateway time-out" after a few requests ('offset' param goes to 85000 or more). And won't be able to get Banno events data. We have jobs running hours waiting to get response which usually took 30 mins. It started to have this issue the past week. Sample code is provided in the screenshot. Error message "504 Gateway time-out" in the response. We investigated the past week, it happened after successfully getting over 70000-80000 records, (hence "offset" param 85000 in the request). while True: sleep(5) # add waiting time to avoid get token error access_token = get_access_token() decoded = jwt.decode(access_token, options={"verify_signature": False}) end_point = f'/a/history/api/v0/institutions/{decoded["institutionId"]}/person-events' print(f'pulling data, from: {from_datetime}, offset: {tot_cnt}') resp = requests.get(base_url + end_point, headers = {'Authorization': f"Bearer {access_token}"}, params = {'to': to_datetime, 'from': from_datetime, 'offset': tot_cnt, 'count': BANNOAPI_MAX_BATCH_SIZE}) try: events = json.loads(resp.text) except: print(f'error when loading json, from: {from_datetime}, to: {to_datetime}, will retry') sleep(60) # add extra waiting time continue if not events: break tot_cnt += len(events) # offset count start_dt = events[-1]['date'] end_dt = events[0]['date'] start_dt_2 = dateutil.parser.parse(start_dt).strftime('%Y-%m-%dT%H%M%S') end_dt_2 = dateutil.parser.parse(end_dt).strftime('%Y-%m-%dT%H%M%S') pd.to_pickle(events, f'./{start_dt_2}_to_{end_dt_2}.pkl') sample request code parse json failed because of 504 error Expect to get Banno events data from response, but failed. Continue reading...
