• Thomas Gleixner's avatar
    mm/vmalloc: dont purge usable blocks unnecessarily · 77e50af0
    Thomas Gleixner authored
    Purging fragmented blocks is done unconditionally in several contexts:
    
      1) From drain_vmap_area_work(), when the number of lazy to be freed
         vmap_areas reached the threshold
    
      2) Reclaiming vmalloc address space from pcpu_get_vm_areas()
    
      3) _vm_unmap_aliases()
    
    #1 There is no reason to zap fragmented vmap blocks unconditionally, simply
       because reclaiming all lazy areas drains at least
    
          32MB * fls(num_online_cpus())
    
       per invocation which is plenty.
    
    #2 Reclaiming when running out of space or due to memory pressure makes a
       lot of sense
    
    #3 _unmap_aliases() requires to touch everything because the caller has no
       clue which vmap_area used a particular page last and the vmap_area lost
       that information too.
    
       Except for the vfree + VM_FLUSH_RESET_PERMS case, which removes the
       vmap area first and then cares about the flush. That in turn requires
       a full walk of _all_ vmap areas including the one which was just
       added to the purge list.
    
       But as this has to be flushed anyway this is an opportunity to combine
       outstanding TLB flushes and do the housekeeping of purging freed areas,
       but like #1 there is no real good reason to zap usable vmap blocks
       unconditionally.
    
    Add a @force_purge argument to the newly split out block purge function and
    if not true only purge fragmented blocks which have less than 1/4 of their
    capacity left.
    
    Rename purge_vmap_area_lazy() to reclaim_and_purge_vmap_areas() to make it
    clear what the function does.
    
    [lstoakes@gmail.com: correct VMAP_PURGE_THRESHOLD check]
      Link: https://lkml.kernel.org/r/3e92ef61-b910-4576-88e7-cf43211fd4e7@lucifer.local
    Link: https://lkml.kernel.org/r/20230525124504.864005691@linutronix.deSigned-off-by: default avatarThomas Gleixner <tglx@linutronix.de>
    Signed-off-by: default avatarLorenzo Stoakes <lstoakes@gmail.com>
    Reviewed-by: default avatarBaoquan He <bhe@redhat.com>
    Cc: Christoph Hellwig <hch@lst.de>
    Cc: Lorenzo Stoakes <lstoakes@gmail.com>
    Cc: Uladzislau Rezki (Sony) <urezki@gmail.com>
    Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
    77e50af0
vmalloc.c 115 KB