Bug#17867117 - ERROR RESULT WHEN "COUNT + DISTINCT + CASE WHEN" NEED MERGE_WALK 

Problem:
COUNT DISTINCT gives incorrect result when it uses a Unique
Tree and its last inserted record has null value.

Here is how COUNT DISTINCT is processed, given that this query is not
using loose index scan.

When a row is produced as a result of joining tables (there is only
one table here), we store the SELECTed value in a Unique tree. This
allows elimination of any duplicates, and thus implements DISTINCT.

When we have processed all rows like this, we walk the Unique tree,
counting its elements, in Aggregator_distinct::endup() (tree->walk());
for each element we call Item_sum_count::add(). Such function wants to
ignore any NULL value, for that it checks item_sum -> args[0] ->
null_value. It is a mistake: when walking the Unique tree, the value
to be aggregated is not item_sum ->args[0] but rather table ->
field[0].

Solution:
instead of item_sum -> args[0] -> null_value, use arg_is_null(), which
knows where to look (like in fix for bug 57932).

As a consequence of this solution, we have to make arg_is_null() a
little more general:
1) Because it was so far only used for AVG() (which always has a
single argument), this function was looking at a single argument; now
that it has to work with COUNT(DISTINCT expression1,expression2), it
must look at all arguments.
2) Because we start using arg_is_null () for COUNT(DISTINCT), i.e. in
Item_sum_count::add (), it implies that we are also using it for
COUNT(no DISTINCT) (same add ()). For COUNT(no DISTINCT), the
nullness to check is that of item_sum -> args[0]. But the null_value
of such item is reliable only if val_*() has been called on it. So far
arg_is_null() was always used after a call to arg_val*(), so could
rely on null_value; but for COUNT, there is no call to arg_val*(), so
arg_is_null() has to call is_null() instead.

Testcase for 16539979 by Neeraj. Testcase for 17867117 contributed by
Xiaobin Lin from Taobao.

Fix :
-------	

Created separate suites called innodb_zip ans i_innodb_zip that contain all compression tests.

Running the new suites with following compression-related parameters :

* innodb_compression_level = {1/9}
* innodb_log_compressed_pages = {ON/OFF}

No commit message

MALFORMED XPATH EXP
Problem:
A malformed XPATH expression in the ExtractValue query is causing
a server crash. This malformed XPATH expression is resulted when 
the position attribute in the substring function contains ".." in
the beginning.
Solution:
The original crash is happening because the "../" is being evaluated
prematurely. It tries to access XML while it hasn't been parsed yet.
The premature evaluation is happening because the val_nodeset function
is being set to constant, in which case we proceed to evaluate them in
JOIN:prepare stage only. The solution to this is setting the val_nodeset
functions as non-constant. This forces us to evaluate the function in
the JOIN:exec stage and thus avoid any premature evaluation of the 
XML strings.

WITH MALFORMED XPATH EXP
Problem:
A malformed XPATH expression in the ExtractValue query is 
causing a server crash. This malformed XPATH expression is
resulted when the position attribute in the substring function
contains ".." in the beginning.
Solution:
The original crash is happening because the "../" is being 
evaluated prematurely. It tries to access XML while it 
hasn't been parsed yet. The premature evaluation is happening
because the val_nodeset function is being set to constant, 
in which case we proceed to evaluate them in JOIN:prepare
stage only. The solution to this is setting the val_nodeset
functions as non-constant. This forces us to evaluate the function
 in the JOIN:exec stage and thus avoid any premature evaluation of
the XML strings.

              SPECIFIED WITH THE BASEDIR OPTION 

Description: The mysql_plugin client attempts to remove any
filename specified to the --basedir option. The problem is
that if the filename does not end with a slash, it will 
attempt to unlink it, which succeeds for files, but not for
directories.

Analysis: When we are starting mysql_plugin with basedir 
option and if we are giving path of a file as basedir, it 
deletes that file. It was because it uses a function 
my_delete which unlinks the file path given.

Fix:  As a fix we replace that line using another function 
my_free, which will only free the  pointer which is having 
that file path.

revid:mattias.jonsson@oracle.com-20131119103616-u6t82s8cpgp0q3ex

Use of uninitialized memory in the priority queue used for returning records
in sorted order.

It happens if no previous partition have returned a row since the
beginning of index_init + an index_read* call returned
HA_ERR_KEY_NOT_FOUND for all partitions (otherwise the record
buffer/priority queue would be initialized) + an index_next/prev
call where all partitions returns HA_ERR_END_OF_FILE.

                WITH SORT ABORTED LEAKS FILE DESCRIPTORS

ISSUE : IO_CACHE used for index_merge quick select
is freed only on successful retrieval of all rows
from index merge.
Suppose if there is a interrupt( or failure) to
this operation of row retrieval (let it be a
KILL_QUERY signal) then we are not freeing the IO_CACHE
resources allocated by index_merge quick select.
And hence temp file associated with it is also not closed.
This lead to a file descriptor leak.

SOLUTION : As part of file sort operation now we always 
free the IO_CACHE allocated by index_merge quick select.

sql/filesort.cc:
  In filesort function we try to free if any
  IO_CACHE allocated by index_merge quick select
  and if it is not yet freed.

No commit message

Problem: The "--local-install" service does not perform as expected for, at least,
         Windows.

Fix: A NULL pointer was dereferenced due to which there was crash.A check was introduced
     for NULL string before dereferencing it.No test cases written as it is a bug during 
     installation.

Problem:
When log_warnings is greater than 1, master prints binlog
dump thread information in mysqld.1.err file.
The information contains slave server id, binlog file and
binlog position. The slave server id is uint32 and the print
format was wrongly specifified (%d instead of %u).
Hence a server id which is more than 2 billion is getting
printed with a negative value.
Eg: Start binlog_dump to slave_server(-1340259414),
pos(mysql-bin.001663, 325187493)

Fix: Changed the uint32 format to %u.

                RESULTING MY_WC_T RESULT IS NOT USED
Issue         : handler functions my_ismbchar_utf8,
              my_well_formed_len_mb for charset utf8
              is calling unicode converion function
              to validate and to find the character
              length. Because of this, instructions
              which will convert the utf8 to unicode
              are executed for no use.
              A similar issue exist with charset utf8mb4
Solution      : reorganized the code such that character
              validation part of unicode conversion
              handler is extracted(duplicated) in to
              separate function. Hence
              my_ismbchar_utf8, my_well_formed_len_mb
              will call the new function which only
              validates and return the length of mb(utf8).
              A similar fix for charset utf8mb4.

strings/ctype-utf8.c:
  New functions has been added for charset utf8 and utf8mb4
  to validate and to get the length of the character.

Fix Windows build break

"SHOW BINLOG EVENTS"

Fixing post push test issue. 
Changing the debug simulation.

Problem:-
We have created a table with UTF8_BIN collation.
In case, when in our query we have ORDER BY clause over a function 
call we are getting result in incorrect order.
Note:the bug is not there in 5.5.

Analysis:
In 5.5, for UTF16_BIN, we have min and max multi-byte length is 2 and 4 
respectively.In make_sortkey(),for 2 byte character character we are 
assuming that the resultant length will be 2 byte/character. But when we 
use my_strnxfrm_unicode_full_bin(), we store sorting weights using 3 bytes 
per character.This result in truncated result.

Same thing happen for UTF8MB4, where we have 1 byte min multi-byte and 
4 byte max multi-byte.We will accsume resultant data as 1 byte/character, 
which result in truncated result.

Solution:-
use strnxfrm(means use of MY_CS_STRNXFRM macro) is used for sort, in 
which the resultant length is not dependent on source length.

No commit message

"SHOW BINLOG EVENTS"

Problem:
========
mysql was crashed after executing "show binlog events in
'mysql-bin.000005' from 99", the crash happened randomly.

Analysis:
========
During construction of LOAD EVENT or NEW LOAD EVENT object
if the starting offset is provided as incorrect value then
all the object members that are retrieved from the offset
are also invalid.  Some times it will lead to out of bound
address offsets.  In the bug scenario, the file name is
extracrated from an invalid address and the same is fed to
strlen(fname) function. Passing invalid address to strlen
will lead to crash.

Fix:
===
Validate if the given offset falls within the event boundary
or not.

sql/log_event.cc:
  Added code to validate fname's address. "fname" should
  be within event boundary. Added code to find invalid
  invents.

The pthread_cond_timedwait(3P) api
uses a const struct timespec for parameter 3.

The instrumentation api for the same, mysql_cond_timedwait,
which expands to inline_mysql_cond_timedwait,
should also take a const parameter for the timespec.

This fix add the missing const to inline_mysql_cond_timedwait.

               CAN RETURN WRONG RESULT SET

PROBLEM
-------
In ha_partition::cmp_ref() we were only calling the 
underlying cmp_ref() of storage engine if the records
are in the same partiton,else we sort by partition and
returns the result.But the index merge intersect 
algorithm expects first to sort by row-id first and 
then by partition id.

FIX
---
Compare the refernces first using storage engine cmp_ref
and then if references are equal(only happens if 
non clustered index is used) then sort it by partition id.

[Approved by Mattiasj #rb3755]
-

Item_func_trim::val_str: we were using the non-mb algorithm for skipping leading spaces
in a multibyte-charset string.

    UPPER CASE HOST NAME ANYMORE
Null merge from mysql-5.1 to mysql-5.5

No commit message

get_cost_calc_buff_size() could return wrong value for the size of imerge_cost_buff.

No commit message

No commit message

No commit message

    UPPER CASE HOST NAME ANYMORE
Merging from mysql-5.1 to mysql-5.5

    UPPER CASE HOST NAME ANYMORE

Description:
It is not possible to drop users with host names with upper case
letters in them. i.e DROP USER 'root'@'Tmp_Host_Name'; is failing
with error.

Analysis: Since the fix 11748570 we came up with lower case hostnames
as standard. But in the current bug the hostname is created by
mysql_install_db script is still having upper case hostnames. 
So, if we have the hostname with upper case letters like(Tmp_Host_Name)
then we will have as it is stored in the mysql.user table. 
In this case if use "'DROP USER 'root'@'Tmp_Host_Name';" it gives 
error because we do compare with the lower case of hostname since the 
11748570 fix.

Fix: We need to convert the hostname to lower case before storing into 
the mysql.user table when we run the mysql_install_db script.

No commit message