GitLab

Before this fix, a thread instrumented for the performance schema,
that would perform file io operations, could crash inside the LF_HASH
implementation, in cases when my_thread_init is not called.

The crash itself has not been reported in 5.5 but similar crashes have
been found in 5.6-based development branches, using LF_HASH for
more instrumentation.

The possibility of a crash in 5.5 is confirmed by code analysis.

The problem is that, when my_thread_init() is not called,
which can happen for threads in storage engines or thirs party code,
my_thread_var is NULL.

Using my_thread_var->stacks_ends_here in mysys/lf_alloc-pin.c is unsafe.

Given that my_thread_var is used:
- only for stacks_ends_here
- only on platform with HAVE_ALLOCA
- only when there is enough room on the stack
and given that the LF_HASH implementation has a fallback
algorythm implemented already when using alloca is not possible,
using my_thread_var->stacks_ends_here is in fact not a strict requirement,
and can be relaxed.

The fix is to:
- test explicitly if my_thread_var is NULL, to account for cases
  when my_thread_init() is not used by the calling thread.
- not use alloca in this case, and rely on the fall back code already in place.
  so that the LF_HASH can be supported even without my_thread_init().

The implementation of mysys/lf_alloc-pin.c has been fixed to support this new usage.
The units tests in unittest/mysys/lf-t.c have been adjusted accordingly.