1. 21 Apr, 2016 1 commit
  2. 20 Apr, 2016 1 commit
  3. 19 Apr, 2016 3 commits
    • Kirill Smelkov's avatar
      dropbear: Don't waste transfer time in favour of small-memory machines defaults · 605e564b
      Kirill Smelkov authored
      I recently discovered that dropbear, compared to openssh over same link
      with same host and user keys, transfers files several times slower -
      sometimes up to an order of magnitude slower. This equally applies to
      both fast LAN IPv4 networks (tried on 1Gbps) and relatively slow "over
      re6stnet" networks with geographically separated client and server.
      
      The problem turned out to be dropbear has very small receive window size
      default:
      
          https://github.com/mkj/dropbear/blob/DROPBEAR_0.53.1/options.h#L258
      
          ---- 8< ----
          /* Window size limits. These tend to be a trade-off between memory
             usage and network performance: */
          /* Size of the network receive window. This amount of memory is allocated
             as a per-channel receive buffer. Increasing this value can make a
             significant difference to network performance. 24kB was empirically
             chosen for a 100mbit ethernet network. The value can be altered at
             runtime with the -W argument. */
          #ifndef DEFAULT_RECV_WINDOW
          #define DEFAULT_RECV_WINDOW 24576
          #endif
          ---- 8< ----
      
      this is maybe appropriate for embedded systems, but small receive window
      affects networking throughput a lot: e.g. I've tried to run several
      file downloading benchmarks and changing receive window size from
      default 24K to 1M (this is maximum according to dropbear documentation) on
      client side made the downloading several times faster and comparable to
      downloading speed with using openssh as client.
      
      Dropbear has -W option for setting receive window size at runtime, so we could
      go through all places which use dropbear and add appropriate `-W ...` there.
      But I think fixing dropbear itself to have more sane defaults for
      not-very-memory-constrained devices (= devices we run slapos on) is preferable
      - this way all services will automatically benefit from transfer speedup
      without taking action on their side with only doing recompilation/redeployment.
      
      Besides changing only recv window size at runtime breaks compatibility with
      openssh: if we only do `-W 1M` on server and try to upload data with openssh as
      client, dropbear complains
      
          [3302] Apr 17 23:10:06 Exit (slapuser2): Bad packet size 32777
      
      and connection terminates. Thus RECV_MAX_PAYLOAD_LEN increase is also
      required, which cannot be done via option at runtime:
      
          https://github.com/mkj/dropbear/blob/DROPBEAR_0.53.1/options.h#L268
      
          ---- 8< ----
          /* Maximum size of a received SSH data packet - this _MUST_ be >= 32768
             in order to interoperate with other implementations */
          #ifndef RECV_MAX_PAYLOAD_LEN
          #define RECV_MAX_PAYLOAD_LEN 32768
          #endif
          ---- 8< ----
      
      So let's increase DEFAULT_RECV_WINDOW to 1M and RECV_MAX_PAYLOAD_LEN
      appropriately (experimentally found that at 512K the complain goes
      away).
      
      Below is rough benchmark before and after the change (timings are not
      very precise, because machines are otherwise busy and network
      fluctuates, but I think they are reasonable in showing we have at least
      several times improvement):
      
                                              before  after
      
          download 1M (re6stnet, slow link)   ~12s    ~4s
          upload   1M (re6stnet, slow link)   ~12s    ~4s
      
          download 100M (1Gbps LAN, IPv4)     ~12s    ~4s
          upload   100M (1Gbps LAN, IPv4)     ~12s    ~4s
      
              legend:
                  client=dbclient
                  server=dropbear
      
      NOTE the original problem equally applies to latest dropbear master, so it is
          not because we are running old 0.53.1 ( from 2011 ! )
      
      NOTE wrt transfer speed openssh is still faster - not an order of
          magnitude, but somewhat.  Also openssh uses much less CPU compared to
          dropbear.
      
      /cc @Tyagov, @klaus
      
      @nexedi, why at all we use dropbear in the first place instead of openssh?
      
      /reviewed-by @rafael
      /reviewed-on nexedi/slapos!68
      605e564b
    • Jérome Perrin's avatar
      recipe.dropbear.AuthorizedKeysFile: Fix returned paths · ec4bdad8
      Jérome Perrin authored
      This recipe only created ~/.ssh/authorized_keys, so it should not return
      the full ~/.ssh/ directory because uninstallation will delete "too
      much".
      
      @alain.takoudjou @rafael after machine was restarted yesterday, the keys I added in ~/.ssh/ of my webrunner were not here anymore. I think this is the reason, thanks for taking a look when you have time.
      
      /reviewed-on nexedi/slapos!37
      ec4bdad8
    • Jérome Perrin's avatar
  4. 18 Apr, 2016 1 commit
  5. 15 Apr, 2016 7 commits
  6. 13 Apr, 2016 6 commits
  7. 11 Apr, 2016 1 commit
  8. 08 Apr, 2016 1 commit
  9. 06 Apr, 2016 1 commit
  10. 05 Apr, 2016 1 commit
  11. 04 Apr, 2016 15 commits
  12. 01 Apr, 2016 2 commits