[Python] Azure function with service bus trigger - ManagedIdentityCredential performance

I am working on an Azure Function that uses a ServiceBusTrigger and queries Azure Table Storage. In order to process multiple messages as quickly as possible we're using the MaxConcurrentCalls setting to enable parallel message processing (e.g. we set MaxConcurrentCalls to 200). We're using Managed Identity to access the Service bus and the Azure Table Storage via DefaultAzureCredential.

Performance testing shows that multiple instances of the Function app are instantiated and are processing messages as expected, however each instance makes a call to the Azure /msi/token endpoint to obtain a ManagedIdentityCredential, and this call is the bottleneck, taking anywhere from 200ms to 5000ms. I.e using the above setting, if 200 messages get dropped onto the service bus then 200 "instances" of the azure function will start processing them, and make 200 calls to get a ManagedIdentityCredential.

1. What is the mechanism behind how Azure functions are processing messages concurrently, will it create multiple processes or multiple threads within the same process?
2. is thee a wa to share/cache the credential once it's obtained to be used by the other message processing instances as well and eliminate the redundant calls to /msi/token ?

We're using Python as programming language.

Continue reading...