• Hugo Wen's avatar
    Fix a stack overflow in pinbox allocator · 9e8546e2
    Hugo Wen authored
    MariaDB supports a "wait-free concurrent allocator based on pinning addresses".
    In `lf_pinbox_real_free()` it tries to sort the pinned addresses for better
    performance to use binary search during "real free". `alloca()` was used to
    allocate stack memory and copy addresses.
    
    To prevent a stack overflow when allocating the stack memory the function checks
    if there's enough stack space. However, the available stack size was calculated
    inaccurately which eventually caused database crash due to stack overflow.
    
    The crash was seen on MariaDB 10.6.11 but the same code defect exists on all
    MariaDB versions.
    
    A similar issue happened previously and the fix in fc2c1e43 was to add a
    `ALLOCA_SAFETY_MARGIN` which is 8192 bytes. However, that safety margin is not
    enough during high connection workloads.
    
    MySQL also had a similar issue and the fix
    https://github.com/mysql/mysql-server/commit/b086fda was to remove the use of
    `alloca` and replace qsort approach by a linear scan through all pointers (pins)
    owned by each thread.
    
    This commit is mostly the same as it is the only way to solve this issue as:
    1. Frame sizes in different architecture can be different.
    2. Number of active (non-null) pinned addresses varies, so the frame
       size for the recursive sorting function `msort_with_tmp` is also hard
       to predict.
    3. Allocating big memory blocks in stack doesn't seem to be a very good
       practice.
    
    For further details see the mentioned commit in MySQL and the inline comments.
    
    All new code of the whole pull request, including one or several files
    that are either new files or modified ones, are contributed under the
    BSD-new license. I am contributing on behalf of my employer Amazon Web
    Services, Inc.
    9e8546e2
lf_alloc-pin.c 14.9 KB