I am currently running a QA model using load_qa_with_sources_chain(). However, when I run it with three chunks of each up to 10,000 tokens, it takes about 35s to return an answer. I would like to speed this up. Can somebody explain what influences the speed of the function and if there is any way to reduce the time to output. If that's not possible, what other changes could you undertake to increase the speed of QA with sources? I tried changing the size of the text chunks but that did not have a significant effect. I am using the map_reduce chain. I am using Python3.10. Continue reading...
