I'm working on file merges and i have this code wrapped in a function with open('out.txt','ab') as out: for d,_,f in path.walk(): with open(os.path.join(d,f),'rb') as in: out.write( in.read() ) I've noticed that running this code the first time, it takes up so much time. At first, It could take as much as five minutes to run, and sometimes up to 10 minutes. But soon after I run the first time, it takes a few seconds, less than five seconds to run the entire process all over again. I don't think it is the file size since, I've tested it with files of 3 MB and ~500MB and in both occurs the same, first 5min and then 2-5 seconds. Do you happen to know why this happens and whether it can be optimized? I know that under the hood, python does things like compilation to bytecode and the os may influence as well. But this happens every single time. I've tried to search if python does warmup like Java but it appears that it doesn't. And I wonder if all of this affect this much to performance. Thanks a lot Continue reading...
