• Nir Soffer's avatar
    net/http: add Transport.ReadBufferSize and WriteBufferSize · c706d422
    Nir Soffer authored
    Previously transport was using the hardcoded bufio.defaultBufSize
    (4096), limiting throughput and increasing cpu usage when uploading or
    downloading large files.
    
    Add options to allow users to configure the buffer sizes as needed.
    
    I tested the maximum benefit of this change by uploading data from
    /dev/zero to a server discarding the bytes. Here is an example upload
    using the default buffer size:
    
    $ time ./upload 10 https://localhost:8000/
    Uploaded 10.00g in 25.13 seconds (407.49m/s)
    
    real	0m25.135s
    user	0m5.167s
    sys	0m11.643s
    
    With this change, using 128k buffer size:
    
    $ time ./upload 10 https://localhost:8000/
    Uploaded 10.00g in 7.93 seconds (1291.51m/s)
    
    real	0m7.935s
    user	0m4.517s
    sys	0m2.603s
    
    In real world usage the difference will be smaller, depending on the
    local and remote storage and the network.
    
    See https://github.com/nirs/http-bench for more info.
    
    Fixes #22618
    
    Change-Id: Iac99ed839c7b95d6dc66602ba8fe1fc5b500c47c
    Reviewed-on: https://go-review.googlesource.com/c/go/+/76410Reviewed-by: default avatarBrad Fitzpatrick <bradfitz@golang.org>
    c706d422
transport.go 74.4 KB