I am testing nlu model called LaBSE. I want to know how much gpu memory is required when i run it. Here is example how to run it on sentence and turn it into embeddings: from sentence_transformers import SentenceTransformer sentences = ["This is an example sentence"] model = SentenceTransformer('sentence-transformers/LaBSE') embeddings = model.encode(sentences) print(embeddings) It shows that 1.9GB of gpu memory are used. But i want to know how much gpu memory will be used when i run it with 10, 25, 50 requests per second. How could I do that? How could i run model.encode on 25 requests per second for example? I basically want to know how much gpu and RAM memory is required if I deploy it and there will be multiple requests per second. Continue reading...
