1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.
  2. Anuncie Aqui
    Anuncie aqui você Também: fdantas@4each.com.br

[Python] Unable to post large file with python requests library

Discussão em 'Python' iniciado por Stack, Setembro 17, 2025.

  1. Stack

    Stack Membro Participativo

    I try to post large file and read response as streaming with requests python library.

    I have to send a large text file (2 million lines of log ) and receive response back as streaming. If I did by small batch (30 thousands lines), it is ok , I get immediate stream response. For large file, nothing happens and it is finished by a timeout if I specify a timeout (without timeout it is hanging indefinitely).

    import requests

    url='http://********:59599'
    header = {'specific-app-header':'01-fr-open-edition-03'}

    def post(file_path):
    with open(file_path, "rb") as f:
    r = requests.post(
    url, headers=header, data=f, stream=True, proxies=None, timeout=180
    )
    for line in r.iter_lines(decode_unicode=True):
    print(line)

    post('/tmp/30_000_lines.log')
    # .... lines is display
    post('/tmp/2_000_000_lines.log')
    # ... requests.exceptions.ConnectionError: ('Connection aborted.', TimeoutError('timed out'))


    My configuration is:

    $python -m requests.help
    {
    "implementation": {
    "name": "CPython",
    "version": "3.11.2"
    },
    "platform": {
    "release": "6.1.0-31-amd64",
    "system": "Linux"
    },
    "requests": {
    "version": "2.32.3"
    },
    "urllib3": {
    "version": "2.3.0"
    },
    .....
    }

    Continue reading...

Compartilhe esta Página