Commit fc168c3a authored by Marko Mäkelä's avatar Marko Mäkelä

MDEV-15587 AES test fails, segfaults in EVP_CipherInit_ex

When HAVE_YASSL is defined (due to cmake -DWITH_SSL=bundled
or otherwise), mysys_ssl/my_crypt.cc will #include "yassl.cc"
from the same directory.

When MariaDB 10.2 or later is compiled with GCC 8 and optimizations
are enabled, then the check
  if (iv)
in EVP_CipherInit_ex() can be wrongly optimized away.
The reason appears to be that __attribute__((nonnull)) is attached
to the variable iv, because there is a (no-op) call
memcpy(oiv, iv, ivlen=0) earlier in the code path.

It is possible that this started failing after the code was
refactored in MDEV-10332 (MariaDB 10.2.6). In MariaDB 10.1,
there is a similar memcpy() call in MyCTX_nopad::init(),
but the code appears to work fine.
parent 8fcd9478
/*
Copyright (c) 2014 Google Inc.
Copyright (c) 2014, 2017 MariaDB Corporation
Copyright (c) 2014, 2019, MariaDB Corporation.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
......@@ -93,7 +93,8 @@ class MyCTX_nopad : public MyCTX
this->key= key;
this->klen= klen;
this->buf_len= 0;
memcpy(oiv, iv, ivlen);
if (ivlen)
memcpy(oiv, iv, ivlen);
DBUG_ASSERT(ivlen == 0 || ivlen == sizeof(oiv));
int res= MyCTX::init(cipher, encrypt, key, klen, iv, ivlen);
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment