YASSL-COMPILED SERVER/CLIENT

Description: thread_pool.thread_pool_connect hangs when the server and
client are compiled with yaSSL.

Bug-fix: Test thread_pool.thread_pool_connect was temporary disabled for
yaSSL. However, now that yaSSL is fixed it runs OK. The bug was
introduced by one of the yaSSL updates. set_current was not working for
i == 0. Now this is fixed. YASSL is updated to 2.3.7d

INITIAL STARTUP

Description: By using mysql_ssl_rsa_setup to get SSL enabled server
(after running mysqld --initialize) server don't answer properly
to "mysqladmin ping" first 30 secs after startup.

Bug-fix: YASSL validated certificate date to the minute but should have
to the second. This is why the ssl on the server side was not up right
away after new certs were created with mysql_ssl_rsa_setup. The fix for
that was submitted by Todd. YASSL was updated to 2.3.7c.

Affects at least 5.6 and 5.7. In customer case, the "client" happened to
be a replication slave, therefore his server crashed.

Bug-fix:
The bug was in yassl. Todd Ouska has provided us with the patch.

(cherry picked from commit 42ffa91aad898b02f0793b669ffd04f5c178ce39)

                MYSQLADMIN -U ROOT -P

DESCRIPTION
===========
Crash occurs when no command is given while executing
mysqladmin utility.

ANALYSIS
========
In mask_password() the final write to array 'temp_argv'
is done without checking if corresponding index 'argc'
is valid (non-negative) or not. In case its negative
(would happen when this function is called with 'argc'=0),
it may cause a SEGFAULT. Logically in such a case,
mask_password() should not have been called as it would do
no valid thing.

FIX
===
mask_password() is now called after checking 'argc'. This
function is now called only when 'argc' is positive
otherwise the process terminates

Problem :
---------
The specific issue reported in this bug is with range/list column
value that is allocated and initialized by evaluating partition
expression(item tree) during execution. After evaluation the range
list value is marked fixed [part_column_list_val]. During next
execution, we don't re-evaluate the expression and use the old value
since it is marked fixed.

Solution :
----------
One way to solve the issue is to mark all column values as not fixed
during clone so that the expression is always re-evaluated once we
attempt partition_info::fix_column_value_functions() after cloning
the part_info object during execution of DDL on partitioned table.
Reviewed-by: Jimmy Yang <Jimmy.Yang@oracle.com>
Reviewed-by: Mattias Jonsson <mattias.jonsson@oracle.com>

RB: 9424

Follow up patch to fix sys_vars.query_cache_min_res_unit_basic_32 test failure.

Valid min value for query_cache_min_res_unit is 512. But attempt
to set value greater than or equal to the ULONG_MAX(max value) is
resulting query_cache_min_res_unit value to 0. This result in
crash while searching for memory block lesser than the valid
min value to store query results.

Free memory blocks in query cache are stored in bins according
to their size. The bins are stored in size descending order.
For the memory block request the appropriate bin is searched using
binary search algorithm. The minimum free memory block request
expected is 512 bytes. And the appropriate bin is searched for block
greater than or equals to 512 bytes.

Because of the bug the query_cache_min_res_unit is set to 0. Due
to which there is a chance of request for memory blocks lesser
than the minimum size in free memory block bins. Search for bin
for this invalid input size fails and returns garbage index.
Accessing bins array element with this index is causing the issue
reported.

The valid value range for the query_cache_min_res_unit is
512 to ULONG_MAX(when value is greater than the max allowed value,
max allowed value is used i.e ULONG_MAX). While setting result unit
block size (query_cache_min_res_unit), size is memory aligned by
using a macro ALIGN_SIZE. The ALIGN_SIZE logic is as below,

  (input_size + sizeof(double) - 1) & ~(sizeof(double) - 1)

For unsigned long type variable when input_size is greater than
equal to ULONG_MAX-(sizeof(double)-1), above expression is
resulting in value 0.

Fix:
-----
Comparing value set for query_cache_min_res_unit with max
aligned value which can be stored in ulong type variable.
If it is greater then setting it to the max aligned value for
ulong type variable.

                MULTIPLE THREADS

Description:- The utility "mysqlimport" does not use
multiple threads for the execution with option
"--use-threads". "mysqlimport" while importing multiple
files and multiple tables, uses a single thread even if the
number of threads are specified with "--use-threads" option.

Analysis:- This utility uses ifdef HAVE_LIBPTHREAD to check
for libpthread library and if defined uses libpthread
library for mutlithreaing. Since HAVE_LIBPTHREAD is not
defined anywhere in the source, "--use-threads" option is
silently ignored.

Fix:- "-DTHREADS" is set to the COMPILE_FLAGS  which will
enable pthreads. HAVE_LIBPTHREAD macro is removed.

Problem :
---------
Issue-1: The root cause for the issues is that (col1 > 1) is not a
valid partition function and we should have thrown error at much early
stage [partition_info::check_partition_info]. We are not checking
sub-partition expression when partition expression is NULL.

Issue-2: Potential issue for future if any partition function needs to
change item tree during open/fix_fields. We should release changed
items, if any, before doing closefrm when we open the partitioned table
during creation in create_table_impl.

Solution :
----------
1.check_partition_info() - Check for sub-partition expression even if no
partition expression.
[partition by ... columns(...) subpartition by hash(<expr>)]

2.create_table_impl() - Assert that the change list is empty before doing
closefrm for partitioned table. Currently no supported partition function
seems to be changing item tree during open.
Reviewed-by: Mattias Jonsson <mattias.jonsson@oracle.com>

RB: 9345

Post push fix.  The function cmp_dtuple_rec() was used without a prototype
in the file row0purge.c.  Adding the include file rem0cmp.h to row0purge.c
to resolve this issue.

approved by Krunal over IM.

    AVOID DEADLOCK AFTER RESTORE

    Post push test fix.

Problem:

If we add a referential integrity constraint with a duplicate
name, an error occurs.  The foreign key object would not have
been added to the dictionary cache.  In the error path, there
is an attempt to remove this foreign key object. Since this
object is not there, the search returns a NULL result.
De-referencing the null object results in this crash.

Solution:

If the search to the foreign key object failed, then don't
attempt to access it.

rb#9309 approved by Marko.

Description: The newest RHEL/CentOS/SL 6.6 openssl package
(1.0.1e-30.el6_6.9; published around 6/4/2015) contains a fix for
LogJam. RedHat's fix for this was to limit the use
of any SSL DH key sizes to a minimum of 768 bits. This breaks any
DHE SSL ciphers for MySQL clients as soon as you install the
openssl update, since in vio/viosslfactories.c, the default
DHPARAM is a 512 bit one. This cannot be changed in
configuration/runtime; and needs a recompile. Because of this the
client connection with --ssl-cipher=DHE-RSA-AES256-SHA is not
able to connect the server.

Analysis: Openssl has changed Diffie-Hellman key from the 512 to
1024 due to some reasons(please see the details at
http://openssl.org/news/secadv_20150611.txt) Because of this the client
with DHE cipher is failing to connect the server. This change took
place from the openssl-1.0.1n onwards.

Fix: Similar bug fix is already pushed to mysql-5.7 under bug#18367167.
Hence we backported the same fix to mysql-5.5 and mysql-5.6.

Backport from 5.6 to 5.5
This makes filesort robust to misc variants of order by / group by
on columns/expressions with zero length.

Fixed the syntax in mysql-systemd-start script

No commit message

No commit message

Description:- mysqlslap is a diagnostic utility designed to
emulate client load for a MySQL server and to report the
timing of each stage. This utility crashes when invalid
values are passed to the options 'num_int_cols_opt' or
'num_chars_cols_opt' or 'engine'.

Analysis:- mysqlslap uses "parse_option()" to parse the
values specified to the options 'num_int_cols_opt',
'num_chars_cols_opt' and 'engine'. These options takes
values separated by commas. In "parse_option()", the comma
separated values are separated and copied into a buffer
without checking the length of the string to be copied. The
size of the buffer is defined by a macro HUGE_STRING_LENGTH
whose value is 8196. So if the length of the any of the
comma separated value exceeds HUGE_STRING_LENGTH, will
result in a buffer overflow.

Fix:- A check is introduced in "parse_option()" to check
whether the size of the string to be copied is more than
HUGE_STRING_LENGTH. If it is more, an error, "Invalid value
specified for the option 'xxx'" is thrown.
Option length was incorrectly calculated for the last comma
separated value. So fixed that as well.

Problem :
---------
This is a regression of Bug#19138298. In purge_node_t::validate_pcur
we are trying to get offsets for all columns of clustered index from
stored record in persistent cursor. This would fail when stored record
is not having all fields of the index. The stored record stores only
fields that are needed to uniquely identify the entry.

Solution :
----------
1. Use pcur.old_n_fields to get fields that are stored
2. Add comment to note dependency between stored fields in purge node
ref and stored cursor.
3. Return if the cursor record is not already stored as it is not safe
to access cursor record directly without latch.
Reviewed-by: Marko Makela <marko.makela@oracle.com>

RB: 9139

Problem :
---------
This is a regression of bug-19138298. During purge, if
btr_pcur_restore_position fails, we set found_clust to FALSE
so that it can find a possible clustered index record in future
calls for the same undo entry. This, however, overwrites the
old_rec_buf while initializing pcur again in next call.

The leak is reproducible in local environment and with the
test provided along with bug-19138298.

Solution :
----------
If btr_pcur_restore_position() fails close the cursor.
Reviewed-by: Marko Makela <Marko.Makela@oracle.com>
Reviewed-by: Annamalai Gurusami <Annamalai.Gurusami@oracle.com>

RB: 9074

No commit message

As man page of open(2) suggested, we should open the same file in the same
mode, to have better performance. For some data files, we will first call
os_file_create_simple_no_error_handling_func() to open them, and then call
os_file_create_func() again. We have to make sure if DIRECT IO is specified,
these two functions should both open file with O_DIRECT.
Reviewed-by: Sunny Bains <sunny.bains@oracle.com>
RB: 8981

The MySQL server uses Henry Spencer's library for regular
expressions to support the REGEXP/RLIKE string operator.
This changeset adapts a recent fix from the upstream for
better 32-bit compatiblity. (Note that we cannot simply use
the current upstream version as a drop-in replacement
for the version used by the server as the latter has
been extended to understand MySQL charsets etc.)

AVOID DEADLOCK AFTER RESTORE

Test Failure Fix.

AVOID DEADLOCK AFTER RESTORE

Analysis
--------
Accessing the restored NDB table in an active multi-statement
transaction was resulting in deadlock found error.

MySQL Server needs to discover metadata of NDB table from
data nodes after table is restored from backup. Metadata
discovery happens on the first access to restored table.
Current code mandates this statement to be the first one
in the transaction. This is because discover needs exclusive
metadata lock on the table. Lock upgrade at this point can
lead to MDL deadlock and the code was written at the time
when MDL deadlock detector was not present. In case when
discovery attempted in the statement other than the first
one in transaction ER_LOCK_DEADLOCK error is reported
pessimistically.

Fix:
---
Removed the constraint as any potential deadlock will be
handled by deadlock detector. Also changed code in discover
to keep metadata locks of active transaction.

Same issue was present in table auto repair scenario. Same
fix is added in repair path also.

Scenario:

1. The purge thread takes an undo log record and parses it and forms
   the record to be purged. We have the primary and secondary keys
   to locate the actual records.
2. Using the secondary index key, we search in the secondary index.
   One record is found.
3. Then it is checked if this record can be purged.  The answer is we
   can purge this record.  To determine this we look up the clustered
   index record.  Either there is no corresponding clustered index
   record, or the matching clustered index record is delete marked.
4. Then we check whether the secondary index record is delete marked.
   We find that it is not delete marked.  We report warning in optimized
   build and assert in debug build.

Problem:

In step 3, we report that the record is purgeable even though it is
not delete marked.  This is because of inconsistency between the
following members of purge_node_t structure - found_clust, ref and pcur.

Solution:

In the row_purge_reposition_pcur(), if the persistent cursor restore
fails, then reset the purge_node_t->found_clust member.  This will
keep the members of purge_node_t structure in a consistent state.

rb#8813 approved by Marko.

As part of the fix find_files() prototype has been modified and
mysql-cluster uses find_files() function. Hence modified find_files() call
in ha_ndbcluster_binlog.cc file to make mysql-cluster build successful.

                 HOST WHEN IT CONTAINS WILDCARD

Description :- Incorrect access privileges are provided to a
user due to wrong sorting of users when wildcard characters
is present in the hostname.

Analysis :- Function "get_sorts()" is used to sort the
strings of user name, hostname, database name. It is used
to arrange the users in the access privilege matching order.
When a user connects, it checks in the sorted user access
privilege list and finds a corresponding matching entry for
the user. Algorithm used in "get_sort()" sorts the strings
inappropriately. As a result, when a user connects to the
server, it is mapped to incorrect user access privileges.
Algorithm used in "get_sort()" counts the number of
characters before the first occurence of any one of the
wildcard characters (single-wildcard character '_' or
multi-wildcard character '%') and sorts in that order.
As a result of inconnect sorting it treats hostname "%" and
"%.mysql.com" as equally-specific values and therefore
the order is indeterminate.

Fix:- The "get_sort()" algorithm has been modified to treat
"%" seperately. Now "get_sort()" returns a number which, if
sorted in descending order, puts strings in the following
order:-
* strings with no wildcards
* strings containg wildcards and non-wildcard characters
* single muilt-wildcard character('%')
* empty string.

Description: On an example MySQL instance with 28k empty
InnoDB tables, a specific query to information_schema.tables
and information_schema.columns leads to memory consumption
over 38GB RSS.

Analysis: In get_all_tables() call, we fill the I_S tables
from frm files and storage engine. As part of that process
we call make_table_name_list() and allocate memory for all
the 28k frm file names in the THD mem_root through
make_lex_string_root(). Since it has been called around
28k * 28k times there is a huge memory getting hogged in
THD mem_root. This causes the RSS to grow to 38GB.

Fix: As part of fix we are creating a temporary mem_root
in get_all_tables and passing it to fill_fiels(). There we
replace the THD mem_root with the temporary mem_root and
allocates the file names in temporary mem_root and frees
it once we fill the I_S tables in get_all_tables and
re-assign the original mem_root back to THD mem_root.

Note: Checked the massif out put with the fix now the memory growth is just around 580MB at peak.