I'm trying to add some functionality to my FastAPI application, which will asynchronously update an ML model (pulled from an S3 bucket.) The idea is to update this model once hourly without blocking the API's ability to respond to CPU-bound tasks, such as model inference requests. # Global model variable ml_model = None # Async function to download model from S3 async def download_model_from_s3(path="integration-tests/artifacts/MULTI.joblib"): global ml_model s3_client = boto3.client("s3") bucket = os.environ.get("BUCKET_BUCKET", "artifacts_bucket") try: local_model_path = './model.joblib' download_coroutine = s3_client.download_file(bucket, path, local_model_path) await download_coroutine ml_model = joblib.load(local_model_path) http://logging.info(f"Model updated.") except Exception as e: logging.exception(f"Error downloading or loading model: {e}") # Asynchronous scheduler function that updates the model every interval async def scheduler(bucket_name: str, model_key: str, interval=60): while True: # Sleep for the specified interval (in minutes) await asyncio.sleep(interval * 60) # Call the download function to update the model await download_model_from_s3(bucket_name, model_key) app = FastAPI() # Startup event to start the scheduler @app.on_event("startup") async def startup_event(): # BLOCKING: Download the model once at startup to ensure it is available download_model_from_s3() # Blocking, ensures model is available # Start the scheduler to update the model every 60 minutes (async, non-blocking) await scheduler(bucket_name, model_key, interval=60) I’m familiar with FastAPI but relatively new to async programming. My question: Is this the right way to asynchronously pull data from S3? Is a separate async S3 client required? I’ve opted to use a while true statement over explicitly scheduling the job once hourly. However, I’m uncertain about the appropriateness of this technique. Continue reading...
