I am pulling data from a slow URL link which does not respond for more than 30 seconds and the downloading file is either timed out or too big. I need to pull the data in python pandas data frame and run some transformations. If it is running it can go for hours for 215 mb file which will increase weekly. I used shell scripting as well but it also faces the same problem. How can i make it automated and fault tolerant? Is there a better way to do this? csv_url='URL' #Delivery_Orders Link # csv_url='delivery.csv' #this is after i manually download it df=pd.read_csv(csv_url) Continue reading...
