• Marko Mäkelä's avatar
    MDEV-15587 AES test fails, segfaults in EVP_CipherInit_ex · fc168c3a
    Marko Mäkelä authored
    When HAVE_YASSL is defined (due to cmake -DWITH_SSL=bundled
    or otherwise), mysys_ssl/my_crypt.cc will #include "yassl.cc"
    from the same directory.
    
    When MariaDB 10.2 or later is compiled with GCC 8 and optimizations
    are enabled, then the check
      if (iv)
    in EVP_CipherInit_ex() can be wrongly optimized away.
    The reason appears to be that __attribute__((nonnull)) is attached
    to the variable iv, because there is a (no-op) call
    memcpy(oiv, iv, ivlen=0) earlier in the code path.
    
    It is possible that this started failing after the code was
    refactored in MDEV-10332 (MariaDB 10.2.6). In MariaDB 10.1,
    there is a similar memcpy() call in MyCTX_nopad::init(),
    but the code appears to work fine.
    fc168c3a
my_crypt.cc 9.09 KB