[Python] download large file from URL using python

Discussão em 'Python' iniciado por Stack, Março 22, 2025.

  1. Stack

    Stack Membro Participativo

    I have a task to download around 16K+ ( max size is of 1GB) files from given URL to location. Files are of different format like pdf, ppt, doc, docx, zip, jpg, iso etc. So had written below piece of code which

    1. downloads file some times and some times only 26KB file only will be downloaded.
    2. Also sometimes get error message " [Errno 10054] An existing connection was forcibly closed by the remote host"

    def download_file(s):
    for row in sheet.iter_rows(min_row=2):
    try:
    url = row[6].value #reading from excel
    # Send GET request to the URL
    response = s.get(url)
    if response.status_code == 200:
    with open(save_path, 'wb') as file:
    file.write(response.content)
    except Exception as e:
    print(f"Error: {e}")


    if __name__ == "__main__":
    with requests.session() as s:
    res = s.post(login_url, data=login_data)
    download_file(s)


    Tried alternative approach using shutil and downloading in chunks . still the issue is observed. reffered solutions from here and here

    import shutil
    with requests.get(url, stream=True) as r:
    with open(local_filename, 'wb') as f:
    shutil.copyfileobj(r.raw, f)


    response = requests.get(url, stream=True)
    with open(book_name, 'wb') as f:
    for chunk in response.iter_content(1024 * 1024 * 2):
    f.write(chunk)

    Continue reading...

Compartilhe esta Página