• Linus Torvalds's avatar
    modules: catch concurrent module loads, treat them as idempotent · 9b9879fc
    Linus Torvalds authored
    This is the new-and-improved attempt at avoiding huge memory load spikes
    when the user space boot sequence tries to load hundreds (or even
    thousands) of redundant duplicate modules in parallel.
    
    See commit 9828ed3f ("module: error out early on concurrent load of
    the same module file") for background and an earlier failed attempt that
    was reverted.
    
    That earlier attempt just said "concurrently loading the same module is
    silly, just open the module file exclusively and return -ETXTBSY if
    somebody else is already loading it".
    
    While it is true that concurrent module loads of the same module is
    silly, the reason that earlier attempt then failed was that the
    concurrently loaded module would often be a prerequisite for another
    module.
    
    Thus failing to load the prerequisite would then cause cascading
    failures of the other modules, rather than just short-circuiting that
    one unnecessary module load.
    
    At the same time, we still really don't want to load the contents of the
    same module file hundreds of times, only to then wait for an eventually
    successful load, and have everybody else return -EEXIST.
    
    As a result, this takes another approach, and treats concurrent module
    loads from the same file as "idempotent" in the inode.  So if one module
    load is ongoing, we don't start a new one, but instead just wait for the
    first one to complete and return the same return value as it did.
    
    So unlike the first attempt, this does not return early: the intent is
    not to speed up the boot, but to avoid a thundering herd problem in
    allocating memory (both physical and virtual) for a module more than
    once.
    
    Also note that this does change behavior: it used to be that when you
    had concurrent loads, you'd have one "winner" that would return success,
    and everybody else would return -EEXIST.
    
    In contrast, this idempotent logic goes all Oprah on the problem, and
    says "You are a winner! And you are a winner! We are ALL winners".  But
    since there's no possible actual real semantic difference between "you
    loaded the module" and "somebody else already loaded the module", this
    is more of a feel-good change than an actual honest-to-goodness semantic
    change.
    
    Of course, any true Johnny-come-latelies that don't get caught in the
    concurrency filter will still return -EEXIST.  It's no different from
    not even getting a seat at an Oprah taping.  That's life.
    
    See the long thread on the kernel mailing list about this all, which
    includes some numbers for memory use before and after the patch.
    
    Link: https://lore.kernel.org/lkml/20230524213620.3509138-1-mcgrof@kernel.org/Reviewed-by: default avatarJohan Hovold <johan@kernel.org>
    Tested-by: default avatarJohan Hovold <johan@kernel.org>
    Tested-by: default avatarLuis Chamberlain <mcgrof@kernel.org>
    Tested-by: default avatarDan Williams <dan.j.williams@intel.com>
    Tested-by: default avatarRudi Heitbaum <rudi@heitbaum..com>
    Tested-by: default avatarDavid Hildenbrand <david@redhat.com>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    9b9879fc
main.c 85.2 KB