1. 07 Dec, 2020 15 commits
  2. 06 Dec, 2020 3 commits
  3. 05 Dec, 2020 9 commits
  4. 04 Dec, 2020 13 commits
    • Bongsu Jeon's avatar
      nfc: s3fwrn5: skip the NFC bootloader mode · 4fb7b98c
      Bongsu Jeon authored
      If there isn't a proper NFC firmware image, Bootloader mode will be
      skipped.
      Signed-off-by: default avatarBongsu Jeon <bongsu.jeon@samsung.com>
      Reviewed-by: default avatarKrzysztof Kozlowski <krzk@kernel.org>
      Link: https://lore.kernel.org/r/20201203225257.2446-1-bongsu.jeon@samsung.comSigned-off-by: default avatarJakub Kicinski <kuba@kernel.org>
      4fb7b98c
    • Jakub Kicinski's avatar
      Merge branch 'perf-optimizations-for-tcp-recv-zerocopy' · 43be3a3c
      Jakub Kicinski authored
      Arjun Roy says:
      
      ====================
      Perf. optimizations for TCP Recv. Zerocopy
      
      This patchset contains several optimizations for TCP Recv. Zerocopy.
      
      Summarized:
      1. It is possible that a read payload is not exactly page aligned -
      that there may exist "straggler" bytes that we cannot map into the
      caller's address space cleanly. For this, we allow the caller to
      provide as argument a "hybrid copy buffer", turning
      getsockopt(TCP_ZEROCOPY_RECEIVE) into a "hybrid" operation that allows
      the caller to avoid a subsequent recvmsg() call to read the
      stragglers.
      
      2. Similarly, for "small" read payloads that are either below the size
      of a page, or small enough that remapping pages is not a performance
      win - we allow the user to short-circuit the remapping operations
      entirely and simply copy into the buffer provided.
      
      Some of the patches in the middle of this set are refactors to support
      this "short-circuiting" optimization.
      
      3. We allow the user to provide a hint that performing a page zap
      operation (and the accompanying TLB shootdown) may not be necessary,
      for the provided region that the kernel will attempt to map pages
      into. This allows us to avoid this expensive operation while holding
      the socket lock, which provides a significant performance advantage.
      
      With all of these changes combined, "medium" sized receive traffic
      (multiple tens to few hundreds of KB) see significant efficiency gains
      when using TCP receive zerocopy instead of regular recvmsg(). For
      example, with RPC-style traffic with 32KB messages, there is a roughly
      15% efficiency improvement when using zerocopy. Without these changes,
      there is a roughly 60-70% efficiency reduction with such messages when
      employing zerocopy.
      ====================
      
      Link: https://lore.kernel.org/r/20201202225349.935284-1-arjunroy.kdev@gmail.comSigned-off-by: default avatarJakub Kicinski <kuba@kernel.org>
      43be3a3c
    • Arjun Roy's avatar
      net-zerocopy: Defer vm zap unless actually needed. · 94ab9eb9
      Arjun Roy authored
      Zapping pages is required only if we are calling vm_insert_page into a
      region where pages had previously been mapped. Receive zerocopy allows
      reusing such regions, and hitherto called zap_page_range() before
      calling vm_insert_page() in that range.
      
      zap_page_range() can also be triggered from userspace with
      madvise(MADV_DONTNEED). If userspace is configured to call this before
      reusing a segment, or if there was nothing mapped at this virtual
      address to begin with, we can avoid calling zap_page_range() under the
      socket lock. That said, if userspace does not do that, then we are
      still responsible for calling zap_page_range().
      
      This patch adds a flag that the user can use to hint to the kernel
      that a zap is not required. If the flag is not set, or if an older
      user application does not have a flags field at all, then the kernel
      calls zap_page_range as before. Also, if the flag is set but a zap is
      still required, the kernel performs that zap as necessary. Thus
      incorrectly indicating that a zap can be avoided does not change the
      correctness of operation. It also increases the batchsize for
      vm_insert_pages and prefetches the page struct for the batch since
      we're about to bump the refcount.
      
      An alternative mechanism could be to not have a flag, assume by
      default a zap is not needed, and fall back to zapping if needed.
      However, this would harm performance for older applications for which
      a zap is necessary, and thus we implement it with an explicit flag
      so newer applications can opt in.
      
      When using RPC-style traffic with medium sized (tens of KB) RPCs, this
      change yields an efficency improvement of about 30% for QPS/CPU usage.
      Signed-off-by: default avatarArjun Roy <arjunroy@google.com>
      Signed-off-by: default avatarEric Dumazet <edumazet@google.com>
      Signed-off-by: default avatarSoheil Hassas Yeganeh <soheil@google.com>
      Signed-off-by: default avatarJakub Kicinski <kuba@kernel.org>
      94ab9eb9
    • Arjun Roy's avatar
      net-zerocopy: Set zerocopy hint when data is copied · 0c3936d3
      Arjun Roy authored
      Set zerocopy hint, event when falling back to copy, so that the
      pending data can be efficiently received using zerocopy when
      possible.
      Signed-off-by: default avatarArjun Roy <arjunroy@google.com>
      Signed-off-by: default avatarEric Dumazet <edumazet@google.com>
      Signed-off-by: default avatarSoheil Hassas Yeganeh <soheil@google.com>
      Signed-off-by: default avatarJakub Kicinski <kuba@kernel.org>
      0c3936d3
    • Arjun Roy's avatar
      net-zerocopy: Introduce short-circuit small reads. · f21a3c48
      Arjun Roy authored
      Sometimes, we may call tcp receive zerocopy when inq is 0,
      or inq < PAGE_SIZE, or inq is generally small enough that
      it is cheaper to copy rather than remap pages.
      
      In these cases, we may want to either return early (inq=0) or
      attempt to use the provided copy buffer to simply copy
      the received data.
      
      This allows us to save both system call overhead and
      the latency of acquiring mmap_sem in read mode for cases where
      it would be useless to do so.
      
      This patchset enables this behaviour by:
      1. Returning quickly if inq is 0.
      2. Attempting to perform a regular copy if a hybrid copybuffer is
         provided and it is large enough to absorb all available bytes.
      3. Return quickly if no such buffer was provided and there are less
         than PAGE_SIZE bytes available.
      
      For small RPC ping-pong workloads, normally we would have
      1 getsockopt(), 1 recvmsg() and 1 sendmsg() call per RPC. With this
      change, we remove the recvmsg() call entirely, reducing the syscall
      overhead by about 33%. In testing with small (hundreds of bytes)
      RPC traffic, this yields a syscall reduction of about 33% and
      an efficiency gain of about 3-5% when defined as QPS/CPU Util.
      Signed-off-by: default avatarArjun Roy <arjunroy@google.com>
      Signed-off-by: default avatarEric Dumazet <edumazet@google.com>
      Signed-off-by: default avatarSoheil Hassas Yeganeh <soheil@google.com>
      Signed-off-by: default avatarJakub Kicinski <kuba@kernel.org>
      f21a3c48
    • Arjun Roy's avatar
      net-zerocopy: Fast return if inq < PAGE_SIZE · 936ced41
      Arjun Roy authored
      Sometimes, we may call tcp receive zerocopy when inq is 0,
      or inq < PAGE_SIZE, in which case we cannot remap pages. In this case,
      simply return the appropriate hint for regular copying without taking
      mmap_sem.
      Signed-off-by: default avatarArjun Roy <arjunroy@google.com>
      Signed-off-by: default avatarEric Dumazet <edumazet@google.com>
      Signed-off-by: default avatarSoheil Hassas Yeganeh <soheil@google.com>
      Signed-off-by: default avatarJakub Kicinski <kuba@kernel.org>
      936ced41
    • Arjun Roy's avatar
      net-zerocopy: Refactor frag-is-remappable test. · 98917cf0
      Arjun Roy authored
      Refactor frag-is-remappable test for tcp receive zerocopy. This is
      part of a patch set that introduces short-circuited hybrid copies
      for small receive operations, which results in roughly 33% fewer
      syscalls for small RPC scenarios.
      Signed-off-by: default avatarArjun Roy <arjunroy@google.com>
      Signed-off-by: default avatarEric Dumazet <edumazet@google.com>
      Signed-off-by: default avatarSoheil Hassas Yeganeh <soheil@google.com>
      Signed-off-by: default avatarJakub Kicinski <kuba@kernel.org>
      98917cf0
    • Arjun Roy's avatar
      net-zerocopy: Refactor skb frag fast-forward op. · 7fba5309
      Arjun Roy authored
      Refactor skb frag fast-forwarding for tcp receive zerocopy. This is
      part of a patch set that introduces short-circuited hybrid copies
      for small receive operations, which results in roughly 33% fewer
      syscalls for small RPC scenarios.
      
      skb_advance_to_frag(), given a skb and an offset into the skb,
      iterates from the first frag for the skb until we're at the frag
      specified by the offset. Assuming the offset provided refers to how
      many bytes in the skb are already read, the returned frag points to
      the next frag we may read from, while offset_frag is set to the number
      of bytes from this frag that we have already read.
      
      If frag is not null and offset_frag is equal to 0, then we may be able
      to map this frag's page into the process address space with
      vm_insert_page(). However, if offset_frag is not equal to 0, then we
      cannot do so.
      Signed-off-by: default avatarArjun Roy <arjunroy@google.com>
      Signed-off-by: default avatarEric Dumazet <edumazet@google.com>
      Signed-off-by: default avatarSoheil Hassas Yeganeh <soheil@google.com>
      Signed-off-by: default avatarJakub Kicinski <kuba@kernel.org>
      7fba5309
    • Arjun Roy's avatar
      net-tcp: Introduce tcp_recvmsg_locked(). · 2cd81161
      Arjun Roy authored
      Refactor tcp_recvmsg() by splitting it into locked and unlocked
      portions. Callers already holding the socket lock and not using
      ERRQUEUE/cmsg/busy polling can simply call tcp_recvmsg_locked().
      This is in preparation for a short-circuit copy performed by
      TCP receive zerocopy for small (< PAGE_SIZE, or otherwise requested
      by the user) reads.
      Signed-off-by: default avatarArjun Roy <arjunroy@google.com>
      Signed-off-by: default avatarEric Dumazet <edumazet@google.com>
      Signed-off-by: default avatarSoheil Hassas Yeganeh <soheil@google.com>
      Signed-off-by: default avatarJakub Kicinski <kuba@kernel.org>
      2cd81161
    • Arjun Roy's avatar
      net-zerocopy: Copy straggler unaligned data for TCP Rx. zerocopy. · 18fb76ed
      Arjun Roy authored
      When TCP receive zerocopy does not successfully map the entire
      requested space, it outputs a 'hint' that the caller should recvmsg().
      
      Augment zerocopy to accept a user buffer that it tries to copy this
      hint into - if it is possible to copy the entire hint, it will do so.
      This elides a recvmsg() call for received traffic that isn't exactly
      page-aligned in size.
      
      This was tested with RPC-style traffic of arbitrary sizes. Normally,
      each received message required at least one getsockopt() call, and one
      recvmsg() call for the remaining unaligned data.
      
      With this change, almost all of the recvmsg() calls are eliminated,
      leading to a savings of about 25%-50% in number of system calls
      for RPC-style workloads.
      Signed-off-by: default avatarArjun Roy <arjunroy@google.com>
      Signed-off-by: default avatarEric Dumazet <edumazet@google.com>
      Signed-off-by: default avatarSoheil Hassas Yeganeh <soheil@google.com>
      Signed-off-by: default avatarJakub Kicinski <kuba@kernel.org>
      18fb76ed
    • Jakub Kicinski's avatar
      Merge branch 'seg6-add-support-for-srv6-end-dt4-dt6-behavior' · 4be986c8
      Jakub Kicinski authored
      Andrea Mayer says:
      
      ====================
      seg6: add support for SRv6 End.DT4/DT6 behavior
      
      This patchset provides support for the SRv6 End.DT4 and End.DT6 (VRF mode)
      behaviors.
      
      The SRv6 End.DT4 behavior is used to implement multi-tenant IPv4 L3 VPNs. It
      decapsulates the received packets and performs IPv4 routing lookup in the
      routing table of the tenant. The SRv6 End.DT4 Linux implementation leverages a
      VRF device in order to force the routing lookup into the associated routing
      table.
      The SRv6 End.DT4 behavior is defined in the SRv6 Network Programming [1].
      
      The Linux kernel already offers an implementation of the SRv6 End.DT6 behavior
      which allows us to set up IPv6 L3 VPNs over SRv6 networks. This new
      implementation of DT6 is based on the same VRF infrastructure already exploited
      for implementing the SRv6 End.DT4 behavior. The aim of the new SRv6 End.DT6 in
      VRF mode consists in simplifying the construction of IPv6 L3 VPN services in
      the multi-tenant environment.
      Currently, the two SRv6 End.DT6 implementations (legacy and VRF mode)
      coexist seamlessly and can be chosen according to the context and the user
      preferences.
      
      - Patch 1 is needed to solve a pre-existing issue with tunneled packets
        when a sniffer is attached;
      
      - Patch 2 improves the management of the seg6local attributes used by the
        SRv6 behaviors;
      
      - Patch 3 adds support for optional attributes in SRv6 behaviors;
      
      - Patch 4 introduces two callbacks used for customizing the
        creation/destruction of a SRv6 behavior;
      
      - Patch 5 is the core patch that adds support for the SRv6 End.DT4
        behavior;
      
      - Patch 6 introduces the VRF support for SRv6 End.DT6 behavior;
      
      - Patch 7 adds the selftest for SRv6 End.DT4 behavior;
      
      - Patch 8 adds the selftest for SRv6 End.DT6 (VRF mode) behavior.
      
      Regarding iproute2, the support for the new "vrftable" attribute, required by
      both SRv6 End.DT4 and End.DT6 (VRF mode) behaviors, is provided in a different
      patchset that will follow shortly.
      
      I would like to thank David Ahern for his support during the development of
      this patchset.
      
      [1] https://tools.ietf.org/html/draft-ietf-spring-srv6-network-programming
      ====================
      
      Link: https://lore.kernel.org/r/20201202130517.4967-1-andrea.mayer@uniroma2.itSigned-off-by: default avatarJakub Kicinski <kuba@kernel.org>
      4be986c8
    • Andrea Mayer's avatar
      selftests: add selftest for the SRv6 End.DT6 (VRF) behavior · 2bc03553
      Andrea Mayer authored
      this selftest is designed for evaluating the new SRv6 End.DT6 (VRF) behavior
      used, in this example, for implementing IPv6 L3 VPN use cases.
      Signed-off-by: default avatarAndrea Mayer <andrea.mayer@uniroma2.it>
      Signed-off-by: default avatarPaolo Lungaroni <paolo.lungaroni@cnit.it>
      Signed-off-by: default avatarJakub Kicinski <kuba@kernel.org>
      2bc03553
    • Andrea Mayer's avatar
      selftests: add selftest for the SRv6 End.DT4 behavior · 2195444e
      Andrea Mayer authored
      this selftest is designed for evaluating the new SRv6 End.DT4 behavior
      used, in this example, for implementing IPv4 L3 VPN use cases.
      Signed-off-by: default avatarAndrea Mayer <andrea.mayer@uniroma2.it>
      Reviewed-by: default avatarDavid Ahern <dsahern@kernel.org>
      Signed-off-by: default avatarJakub Kicinski <kuba@kernel.org>
      2195444e