• Kirill Smelkov's avatar
    bigfile/virtmem: Introduce "mmap overlay" mode · fae045cc
    Kirill Smelkov authored
    with the intention to later use WCFS through it.
    
    Before this patch virtmem had only one mode: a BigFile backend was
    providing loadblk and storeblk methods, and on every block access
    loadblk was called to load block data into allocated RAM page.
    
    However with WCFS virtmem won't be needed to do anything to load data -
    because loading from head/bigfile/f mmaped through OS will be handled by
    OS directly. Thus for wcfs, that leaves virtmem only to handle dirtying
    and writeout.
    
    -> Introduce "mmap overlay" mode into virtmem to handle WCFS-like
    BigFile backends - that can provide read-only base layer suitable for
    mmapping.
    
    This patch is organized as follows:
    
    - fileh_open is added flags argument to indicate which mode to use for
      opened fileh. BigFileH is added .mmap_overlay bitfield correspondingly.
      (virtmem.h)
    
    - struct bigfile_ops is extended with 3 optional methods that a BigFile
      backend might provide to support mmap-overlay mode:
    
      * mmap_setup_read,
      * remmap_blk_read, and
      * munmap
    
      (see file.h changes for documentation of this new interface)
    
    - if opened with MMAP_OVERLAY flag, virtmem is using those methods to
      organize VMA views backed by read-only base mmap layer and writeout
      for such VMAs (virtmem.c)
    
    - a test is added to exercise MMAP_OVERLAY virtmem mode (test_virtmem.c)
    
    - everything else, including bigfile.py, is switched to use
      DONT_MMAP_OVERLAY unconditionally for now.
    
    In internal comments inside virtmem new mode is interchangeable called
    "mmap overlay" and "wcfs", even though wcfs is not hooked to be used
    mmap-overlaying yet.
    
    Some preliminary history:
    
    fb6932a2    X Split PAGE_LOADED -> PAGE_LOADED, PAGE_LOADED_FOR_WRITE
    4a20a573    X Settled on what should happen after writeout for wcfs case
    f084ff9b    X Transition to all VMA under 1 fileh to be either all based on wcfs or all based on !wcfs
    fae045cc
test_virtmem.c 54.1 KB