[Python] Ierating a mongodb collection - Exception has occured - cursor id not found

I have the following code that connects to mongodb (v7.0.14) running on Ubuntu 24.04.

It retrieves about 1,500 urls and then iterates them.



Randomly I get a CursorNotFound exception which I understand sometimes has to to with timeout issues.

I've specified NoCursorTimeout - and processing inside the loop is certainly less than 30 minutes.

The code is running on the same machine as MONGODB.

Can anyone sujjest what might be causing this issue?

Thanks !

with mdb_client.start_session() as session:

# Fetch the connection string for the mongodb database from the config.ini file.
mongo_conn_str = config.get_mongodb_connection_string()
if (mongo_conn_str == "ERROR"):
return (False)

# Try connecting to the mongodb server. If success, just close the connection and continue. If failure - just exit this program.
mdb_client = None
try:
mdb_client = pymongo.MongoClient(mongo_conn_str)
mdb_client.server_info()
except:
print(f"{color.bcolors.FAIL}ERROR: Connection to the MONGODB Server failed. Please check the connection parameters in the config.ini file.")
return (False)

collection = mdb_client.bp2.geocatlinks.find({"last_collected_date": {"$eq": None}}, no_cursor_timeout=True)

for geocatrec in collection: // The exception happend on this line of code.

# collection now contains 1500 urls

url = geocatrec['geocat_link']

# Fetch the url and extract data

mdb_client.bp2.geocatlinks.update_one({"_id": document_id}, update_fields)


collection.close()

print(f"{color.bcolors.OKCYAN}STATUS: {now.strftime('%Y-%m-%d %H:%M:%S')} Processing Begins.....")

Continue reading...