I'm facing this situation: I have a FastAPI application running on, for example, 10.11.12.13:8000, and I "register" multiple "workers" with that address. Clients can send requests (each request is unique) to a worker through the registered address, but I registered all the workers with the same address, so all requests are sent to 10.11.12.13:8000. But here's the part I'm trying to handle: A client can send the same unique request to multiple workers at the same time (through an asyncio.gather). How can my application handle the request only once, and return the response to the client as if each worker performs the task independently? Example code: app.py from fastapi import FastAPI import uvicorn app = FastAPI() work_count = 0 @app.get("/") def handle_workload(): global work_count work_count += 1 return {"message": f"Hello world! {work_count}"} def start_server(): uvicorn.run( app, host="0.0.0.0", port=8000, log_level="debug" ) if __name__ == "__main__": start_server() client.py import asyncio import aiohttp import random async def send_request(worker_url: str): async with aiohttp.ClientSession() as session: async with session.get(worker_url) as response: if response.status == 200: return await response.json() #I registered all the worker with the same address: worker_servers = [ ("worker_1", "http://localhost:8000"), ("worker_2", "http://localhost:8000"), ("worker_3", "http://localhost:8000"), ("worker_4", "http://localhost:8000") ] async def main(): worker_urls = [worker[1] for worker in random.sample(worker_servers, 3)] responses = await asyncio.gather(*[send_request(url) for url in worker_urls]) print(responses) if __name__ == "__main__": asyncio.run(main()) Can you give me some advice about how to implement this? I thought of batching the requests coming within a short time period, and using cache, but each request is unique so I don't think using cache is a good idea. Continue reading...
