• Haggai Eran's avatar
    IB/core: Implement support for MMU notifiers regarding on demand paging regions · 882214e2
    Haggai Eran authored
    * Add an interval tree implementation for ODP umems. Create an
      interval tree for each ucontext (including a count of the number of
      ODP MRs in this context, semaphore, etc.), and register ODP umems in
      the interval tree.
    * Add MMU notifiers handling functions, using the interval tree to
      notify only the relevant umems and underlying MRs.
    * Register to receive MMU notifier events from the MM subsystem upon
      ODP MR registration (and unregister accordingly).
    * Add a completion object to synchronize the destruction of ODP umems.
    * Add mechanism to abort page faults when there's a concurrent invalidation.
    
    The way we synchronize between concurrent invalidations and page
    faults is by keeping a counter of currently running invalidations, and
    a sequence number that is incremented whenever an invalidation is
    caught. The page fault code checks the counter and also verifies that
    the sequence number hasn't progressed before it updates the umem's
    page tables. This is similar to what the kvm module does.
    
    In order to prevent the case where we register a umem in the middle of
    an ongoing notifier, we also keep a per ucontext counter of the total
    number of active mmu notifiers. We only enable new umems when all the
    running notifiers complete.
    Signed-off-by: default avatarSagi Grimberg <sagig@mellanox.com>
    Signed-off-by: default avatarShachar Raindel <raindel@mellanox.com>
    Signed-off-by: default avatarHaggai Eran <haggaie@mellanox.com>
    Signed-off-by: default avatarYuval Dagan <yuvalda@mellanox.com>
    Signed-off-by: default avatarRoland Dreier <roland@purestorage.com>
    882214e2
ib_umem_odp.h 5.18 KB