Part 2. Function QUOTE() was not multi-byte safe.

  @ mysql-test/r/ctype_ucs.result
  @ mysql-test/t/ctype_ucs.test
  Adding tests

  @ sql/item_strfunc.cc
  Fixing Item_func_quote::val_str to be multi-byte safe.

  @ sql/item_strfunc.h
  Multiple size needed for quote characters to mbmaxlen

      
Problem: wrong character set pointer was passed to my_strtoll10_mb2,
which led to DBUG_ASSERT failure in some cases.

  @ mysql-test/r/func_encrypt_ucs2.result
  @ mysql-test/t/func_encrypt_ucs2.test
  @ mysql-test/r/ctype_ucs.result
  @ mysql-test/t/ctype_ucs.test
  Adding tests

  @ sql/item_func.cc
  "cs" initialization was wrong (res does not necessarily point to &str_value)

  @ sql/item_strfunc.cc
  Item_func_dec_encrypt::val_str() and Item_func_des_descrypt::val_str()
  did not set character set for tmp_value (the returned value),
  so the old value, which was previously copied from args[1]->val_str(),
  was incorrectly returned with tmp_value.

               ordered data from archive tables

Archive was using wrong memory address to check if field
is NULL (after filesort, when reading record again).

mysql-test/r/archive.result:
  A test case for BUG#11764339.
mysql-test/t/archive.test:
  A test case for BUG#11764339.
storage/archive/ha_archive.cc:
  Null bytes are restored to "record" buffer, which may
  or may not be equal to record buffer for field. Check
  null bits in "record" buffer, instead of Field::null_ptr.

Bug#11765108 - Bug#58036: CLIENT UTF32, UTF16, UCS2 SHOULD BE DISALLOWED, THEY CRASH SERVER 

Fixing wrong usage of DBUG_ASSERT.
In non-debug version thd_init_client_charset
was not executed at all.

Result File Correction For Bug Fix

Problem: a byte behind the end of input string was read
in case of a broken XML not having a quote or doublequote
character closing a string value.

Fix: changing condition not to read behind the end of input string

  @ mysql-test/r/xml.result
  @ mysql-test/t/xml.test
  Adding tests

  @ strings/xml.c
  When checking if the closing quote/doublequote was found,
  using p->cur[0] us unsafe, as p->cur can point to the byte after the value.
  Comparing p->cur to p->beg instead.

      
This is a backport of the patch for MySQL Bug#50574.
      
Adding a SPATIAL INDEX on non-geometrical columns caused a
segmentation fault when the table was subsequently 
inserted into.
            
A test was added in mysql_prepare_create_table to explicitly
check whether non-geometrical columns are used in a
spatial index, and throw an error if so.
      
For MySQL 5.5 and later, a new and more meaningful error 
message was introduced.  For 5.1, we (re-)use an existing
error code.

               ("-") IN DATABASE NAMES IN ALTER DATABASE.

mysqldump did not quote database name in 'ALTER DATABASE'
statements in its output. This can further cause a failure
while loading if database name contains a hyphen '-'.

This happened as, while printing the 'ALTER DATABASE'
statements, the database name was not quoted.

Fixed by quoting the database name.


client/mysqldump.c:
  Bug#11766310 : 59398: MYSQLDUMP 5.1 CAN'T HANDLE A DASH
                 ("-") IN DATABASE NAMES IN ALTER DATABASE.
  
  Modified the print statement in order to print the quoted
  database name for 'ALTER DATABASE' statements.
mysql-test/r/mysqldump.result:
  Added a test case for bug#11766310.
mysql-test/t/mysqldump.test:
  Added a test case for bug#11766310.

A separate fix for 5.1 (as 5.1 and 5.5 have seriously
differged in the related pieces of the code).
A patch for 5.5 was approved earlier.

Problem: ucs2 was correctly disallowed in "SET NAMES" only,
while mysql_real_connect() and mysql_change_user() still allowed
to use ucs2, which made server crash.

Fix: disallow ucs2 in mysql_real_connect() and mysql_change_user().

  @ sql/sql_priv.h
    - changing return type for thd_init_client_charset() to bool,
      to return errors to the caller

  @ sql/sql_var.cc
    - using new function

  @ sql/sql_connect.cc
    - thd_client_charset_init:
      in case of unsupported client character set send error and return true;
      in case of success return false
    - check_connection:
      Return error if character set initialization failed

  @ sql/sql_parse.cc
    - check charset in the very beginnig of the CMD_CHANGE_USER handling code

  @ tests/mysql_client_test.c
    - adding tests

The loop that was looping over subqueries' references to outer field used a
local boolean variable to tell whether the field was grouped or not. But the
implementor failed to reset the variable after each iteration. Thus a field
that was not directly aggregated appeared to be.

Fixed by resetting the variable upon each new iteration.

Thanks to Kristian Nielsen for finding out the root cause for the
failure, see: https://bugs.launchpad.net/maria/+bug/677407

          memory reference

There are two issues present here.
  1) There is a possibility that we test a byte beyond the
     allocated buffer

  2) We compare a byte that might never have been
     initalized to see if it's 0.

The first issue is not triggered by existing code, but an
ASSERT has been added to safe-guard against introducing
new code that triggers it.

The second issue is what triggers the Valgrind warnings
reported in the bug report. A buffer is allocated in
class String to hold the value. This buffer is populated
by the character data constituting the string, but is not
zero-terminated in most cases.  Testing if it is indeed
zero-terminated means that we check a byte that has never
been explicitly set, thus causing Valgrind to trigger.

Note that issue 2 is not a serious problem.  The variable
is read, and if it's not zero, we will set it to zero.
There are no further consequences.

Note that this patch does not fix the underlying problems
with issue 1, as it is deemed too risky to fix at this
point (as noted in the bug report).  As discussed in
the report, the c_ptr() method should probably be
replaced, but this requires a thorough analysis of the
~200 calls to the method.


sql/set_var.cc:
  These two cases have been reported to fail
  with Valgrind.

Assertion `bitmap_is_set_all(&table->s->all_set)' failed in
handler::ha_reset

This followup fixes the compilation warning
  'test_bit' may be used uninitialized in this function
introduced by the previous patch.

Assertion `bitmap_is_set_all(&table->s->all_set)' failed in
handler::ha_reset

This assertion could be triggered if two connections simultaneously
executed two bitmap test functions on the same bitmap. For example,
the assertion could be triggered if one connection executed UPDATE
while a second connection executed SELECT on the same table.

Even if bitmap test functions have read-only semantics and have
const bitmaps as parameter, several of them modified the internal
state of the bitmap. With interleaved execution of two such functions
it was possible for one function to modify the state of the same
bitmap that the other function had just modified. This lead to an
inconsistent state and could trigger the assert.

Internally the bitmap uses 32 bit words for storage. Since bitmaps
can contain any number of bits, the last word in the bitmap may
not be fully used. A 32 bit mask is maintained where a bit is set
if the corresponding bit in the last bitmap word is unused.
The problem was that several test functions applied this mask to
the last word. Sometimes the mask was negated and used to zero out
the remainder of the last word and sometimes the mask was used as-is
to fill the remainder of the last word with 1's. This meant that if
a function first used the negated mask and another function then
used the mask as-is (or vice-versa), the first function would then
get the wrong result.

This patch fixes the problem by changing the implementation of
9 bitmap functions that modified the bitmap state even if the 
bitmap was declared const. These functions now preserve the
internal state of the bitmap. This makes it possible for
two connections to concurrently execute two of these functions
on the same bitmap without issues.

The patch also removes dead testing code from my_bitmap.c.
These tests have already been moved to unittest/mysys/bitmap-t.c.
Existing test coverage of my_bitmap has been extended.

No MTR test case added as this would require adding several sync
points to the bitmap functions. The patch has been tested with
a non-deterministic test case posted on the bug report.


include/my_bit.h:
  Removed my_count_bits_ushort() which is not needed anymore.
  Added my_count_bits_uint32().
unittest/mysys/bitmap-t.c:
  Extended test coverage of my_bitmap.

attempt to create spatial index on char > 31 bytes".

Attempt to create spatial index on char field with length
greater than 31 byte led to assertion failure on server
compiled with safemutex support.

The problem occurred in mi_create() function which was called
to create a new version of table being altered. This function
failed since it detected an attempt to create a spatial key
on non-binary column and tried to return an error.
On its error path it tried to unlock THR_LOCK_myisam mutex
which has not been not locked at this point. Indeed such an
incorrect behavior was caught by safemutex wrapper and caused
assertion failure.

This patch fixes the problem by ensuring that mi_create()
doesn't releases THR_LOCK_myisam mutex on error path if it was
not acquired.

mysql-test/r/gis.result:
  Added test for bug @59888 "debug assertion when attempt to
  create spatial index on char > 31 bytes".
mysql-test/t/gis.test:
  Added test for bug @59888 "debug assertion when attempt to
  create spatial index on char > 31 bytes".
storage/myisam/mi_create.c:
  Changed mi_create() not to release THR_LOCK_myisam mutex on
  error path if it was not acquired.

rw_lock_create_func(): Initialize lock->writer_thread, so that Valgrind
will not complain even when Valgrind instrumentation is not enabled.
Flag lock->writer_thread uninitialized, so that Valgrind can complain
when it is used uninitialized.

rw_lock_set_writer_id_and_recursion_flag(): Revert the bogus Valgrind
instrumentation that was pushed in the first attempt to fix this bug.

index gives failures.

Approved by Marko

by silencing a bogus Valgrind warning:

==4392== Conditional jump or move depends on uninitialised value(s)
==4392==    at 0x5A18416: rw_lock_set_writer_id_and_recursion_flag (sync0rw.ic:283)
==4392==    by 0x5A1865C: rw_lock_x_lock_low (sync0rw.c:558)
==4392==    by 0x5A18481: rw_lock_x_lock_func (sync0rw.c:617)
==4392==    by 0x597EEE6: mtr_x_lock_func (mtr0mtr.ic:271)
==4392==    by 0x597EBBD: fsp_header_init (fsp0fsp.c:970)
==4392==    by 0x5A15E78: innobase_start_or_create_for_mysql (srv0start.c:1508)
==4392==    by 0x598B789: innobase_init(void*) (ha_innodb.cc:2282)

os_compare_and_swap_thread_id() is defined as
__sync_bool_compare_and_swap(). From the GCC doc:

`bool __sync_bool_compare_and_swap (TYPE *ptr, TYPE oldval TYPE newval, ...)'
  ...
  The "bool" version returns true if the comparison is successful and
  NEWVAL was written.

So it is not possible that the return value is uninitialized, no matter what
the arguments to os_compare_and_swap_thread_id() are. Probably Valgrind gets
confused by the implementation of the GCC internal function
__sync_bool_compare_and_swap().