Problem
=======

When facing decoding of corrupt binary log files, server may misbehave
without detecting the events corruption.

This patch makes MySQL server more resilient to binary log decoding.

Fixes for events de-serialization and apply
===========================================

@sql/log_event.cc

Query_log_event::Query_log_event: added a check to ensure query length
is respecting event buffer limits.

Query_log_event::do_apply_event: extended a debug print, added a check
to character set to determine if it is "parseable" or not, verified if
database name is valid for system collation.

Start_log_event_v3::do_apply_event: report an error on applying a
non-supported binary log version.

Load_log_event::copy_log_event: added a check to table_name length.

User_var_log_event::User_var_log_event: added checks to avoid reading
out of buffer limits.

User_var_log_event::do_apply_event: reported an sanity check error
properly and added individual sanity checks for variable types that
expect fixed (or minimum) amount of bytes to be read.

Rows_log_event::Rows_log_event: added checks to avoid reading out of
buffer limits.

@sql/log_event_old.cc

Old_rows_log_event::Old_rows_log_event: added a sanity check to avoid
reading out of buffer limits.

@sql/sql_priv.h

Added a sanity check to available_buffer() function.

              STATEMENTS

ANALYSIS:
=========
A user not having FILE privilege is not allowed to create
custom data/index directories for a table or for its
partitions via CREATE TABLE but is allowed to do so via
ALTER TABLE statement.

ALTER TABLE ignores DATA DIRECTORY and INDEX DIRECTORY when
given as table options. The issue occurs during the
creation of partitions for a table via ALTER TABLE
statement with the DATA DIRECTORY and/or INDEX DIRECTORY
options. The issue exists because of the absence of FILE
privilege check for the user.

FIX:
====
A FILE privilege check has been introduced for resolving
the above scenario.

 Followup: now that the man pages have actually been removed,
 we no longer need to take deliberate action to ignore them.
 Thus we can remove that part of the original change.

 RPM: drop the conditional removal
 DEB: remove from the exclude list

No commit message

No commit message

              IS DROPPED

ANALYSIS:
=========
It is advised not to tamper with the system tables.
When primary key is dropped from a system table, certain
operations on the table which tries to access the table key
information may lead to server exit.

FIX:
====
An appropriate error is now reported in such a case.

               ROOT

DESCRIPTION
===========
If the .pid file is created at a world-writable location,
it can be compromised by replacing the server's pid with
another running server's (or some other non-mysql process)
PID causing abnormal behaviour.

ANALYSIS
========
In such a case, user should be warned that .pid file is
being created at a world-writable location.

FIX
===
A new function is_file_or_dir_world_writable() is defined
and it is called in create_pid_file() before .pid file
creation. If the location is world-writable, a relevant
warning is thrown.

NOTE
====
1. PID file is always created with permission bit 0664, so
for outside world its read-only.
2. Ignoring the case when permission is denied to get the
dir stats since the .pid file creation would fail anyway in
such a case.

              MDL_CONTEXT::TRY_ACQUIRE_LOCK_IMPL

ANALYSIS:
=========
Server sometimes exited when multiple threads tried to
acquire and release metadata locks simultaneously (for
example, necessary to access a table). The same problem
could have occurred when new objects were registered/
deregistered in Performance Schema.

The problem was caused by a bug in LF_HASH - our lock free
hash implementation which is used by metadata locking
subsystem in 5.7 branch. In 5.5 and 5.6 we only use LF_HASH
in Performance Schema Instrumentation implementation. So
for these versions, the problem was limited to P_S.

The problem was in my_lfind() function, which searches for
the specific hash element by going through the elements
list. During this search it loads information about element
checked such as key pointer and hash value into local
variables. Then it confirms that they are not corrupted by
concurrent delete operation (which will set pointer to 0)
by checking if element is still in the list. The latter
check did not take into account that compiler (and
processor) can reorder reads in such a way that load of key
pointer will happen after it, making result of the check
invalid.

FIX:
====
This patch fixes the problem by ensuring that no such
reordering can take place. This is achieved by using
my_atomic_loadptr() which contains compiler and processor
memory barriers for the check mentioned above and other
similar places.

The default (for non-Windows systems) implementation of
my_atomic*() relies on old __sync intrisics and implements
my_atomic_loadptr() as read-modify operation. To avoid
scalability/performance penalty associated with addition of
my_atomic_loadptr()'s we change the my_atomic*() to use
newer __atomic intrisics when available. This new default
implementation doesn't have such a drawback.

PROBLEM
-------

This warning message is printed when trx_sys->rseg_history_len is greater than some
arbitrary magic number (2000000). By seeing the reproducing scenario where we keep
a read view open and do a lot of transactions on table which increases the hitsory
length it is entirely possible that trx_sys->rseg_history_len can exceed 2000000.
So this is not a bug due to corruption of history length.The warning message was
just added to test some scenario and not removed.

FIX
---

1.Print this warning message only for debug versions.
2.Modified the warning message with more detailed information.
3.Don't crash even in debug versions.

[#rb 17929 Reviewed by jimmy and satya]

Issue:
------
VALUES doesn't have a type() function and is considered a
Item_field.

Solution for 5.7:
-----------------
Add a new type() function for Item_values_insert.

On 8.0 and trunk it was fixed by Mithun's Bug#19601973.

Solution for 5.6:
-----------------
Additionally Bug#17458914 is backported.

This will address the problem of using VALUES() in
INSERT ... ON DUPLICATE KEY UPDATE. Create a field object
only if it is in the UPDATE clause, else return a NULL
item.

This will also address the problems mentioned in
Bug#14789787 and Bug#16756402.

Solution for 5.5:
-----------------
As mentioned above Bug#17458914 is backported.

Additionally Bug#14786324 is also backported.

When VALUES() is detected outside its meaningful place,
it should be treated as NULL and is thus replaced with a
Field_null object, with the same name as the original
field.

Fields with type NULL are generally not handled well inside
the server (e.g Innodb will not accept them and it is
impossible to create them in regular tables). So create a
new const NULL item instead.

	- Update the default plugin directory for debug builds

Remove cmake code for signing executables.
Automatic signing has always failed anyways.
It should be done manually as part of the release process.

  Removed relevant man pages from file lists for RPM and DEB

  RPM: added conditional removal of them, so it works both before and
  after man pages are actually removed

  DEB: added to exclude list (5.6+)

               NOT UPDATE FILE ON DISK

Description:- When the server variable, "myisam_use_mmap" is
enabled, MyISAM tables on windows are not updating the file
on disk even when the server variable "flush" is set to 1.
This is inturn making the table corrupted when encountering
a power failure.

Analysis:- When the server variable "myisam_use_mmap" is set,
files of MyISAM tables will be memory mapped using the OS
APIs mmap()/munmap()/msync() on Unix and CreateFileMapping()
/UnmapViewOfFile()/FlushViewOfFile() on Windows. msync() and
FlushViewOfFile() is responsible for flushing the changes
made to the in-core copy of a file that was mapped into
memory using mmap()/CreateFileMapping() back to the
file system.  FLUSH is determined by the OS unless
explicitly called using msync()/FlushViewOfFile().

When the server variables "myisam_use_mmap" and "flush" are
enabled, MyISAM is only flushing the files from file system
cache to disc using "mysql_file_sync()" and not the memory
mapped file from memory to FS cache using "my_msync()".
["my_msync()" inturn calls  msync() on Unix and
FlushViewOfFile() on Windows.

Fix:- As part of the fix, if server variable
"myisam_use_mmap" is enabled along with  "flush",
"my_msync()" is invoked to flush the data in memory to file
system cache and followed by "mysql_file_sync()" which will
flush the data from file system cache to disk.

              UNEXPECTED ERROR

ANALYSIS:
=========
Creating many indexes with large amount of index
information causes a server exit.

FIX:
====
A appropriate error is reported when the cumulative index
information length exceeds the 2 byte range (i.e 65535).

               AND PARAM IN ORDER BY

Issue:
------
This issue can occur when the ORDER BY list refers to a
column that contains a parameter in the select list.

Solution:
---------
In JOIN::update_depend_map and get_sort_by_table, the
ORDER BY list's used_tables isn't checked for parameters.
This can result in incorrect behavior.

This is a partial backport of Roy's

Building with ninja shows the problem:
cmake .. -G Ninja
ninja
ninja: error: dependency cycle: sql/GenServerSource -> sql/CMakeFiles/GenServerSource -> sql/sql_builtin.cc -> cmake_order_depends_target_sq
 sql/GenServerSource

Bug#16877045 5.6-CLUSTER-7.3 WIN32 SQL_YACC.CC BUILD PROBLEM
 - Somewhat circular dependency caused by the configured files sql_builtin.cc  being included as
    part of the files to generate in sql/
 - Move sql_builtin.cc out of GEN_SOURCES variable.
 - Create new variable CONF_SOURCES to be used for configured files.

No commit message

DESCRIPTION:
===========
The bug is related to incorrect parsing of SQL queries
when typed in on the CLI. The incorrect parsing can
result in unexpected results.

ANALYSIS:
========
The scenarios mainly happens for identifier names
with a typical combination of backslashes and backticks.
The incorrect parsing can either result in executing
additional queries or can result in query truncation.
This can impact mysqldump as well.

FIX:
===
The fix makes sure that such identifier names are
correctly parsed and a proper query is sent to the
server for execution.

(cherry picked from commit 31a372aa1c2b93dc75267d1f05a7f7fca6080dc0)

No commit message

Reintroduce environment variable MYSQL_GROUP_SUFFIX to be used as
--default-group-suffix value if not already set.

The environment variable was accidentally renamed to DEFAULT_GROUP_SUFFIX_ENV
in MySQL server 5.5.

DESCRIPTION:
===========
The bug is related to incorrect parsing of SQL queries
when typed in on the CLI. The incorrect parsing can
result in unexpected results.

ANALYSIS:
========
The scenarios mainly happens for identifier names
with a typical combination of backslashes and backticks.
The incorrect parsing can either result in executing
additional queries or can result in query truncation.
This can impact mysqldump as well.

FIX:
===
The fix makes sure that such identifier names are
correctly parsed and a proper query is sent to the
server for execution.

   - Add hostname package for docker rpm builds.

               CHARSETS AND SHA2

Missed pushing to 5.5.

No commit message

                       COL), NAME_CONST('NAME', NULL))

Backport of Bug#19143243 fix.

NAME_CONST item can return NULL_ITEM type in case of incorrect arguments.
NULL_ITEM has special processing in Item_func_in function.
In Item_func_in::fix_length_and_dec an array of possible comparators is
created. Since NAME_CONST function has NULL_ITEM type, corresponding
array element is empty. Then NAME_CONST is wrapped to ITEM_CACHE.
ITEM_CACHE can not return proper type(NULL_ITEM) in Item_func_in::val_int(),
so the NULL_ITEM is attempted compared with an empty comparator.
The fix is to disable the caching of Item_name_const item.

              LAST BYTE ONLY (YASSL)


Description:- TLS cipher negociation happens incorrectly
leading to the use of a different

Analysis:- YaSSL based MySQL server will compare only the
last byte of each cipher sent in the Client Hello message.
This can cause TLS connections to fail, due to the server
picking a cipher which the client doesn't actually support.

Fix:- A fix for detecting cipher suites with non leading
zeros is included as YaSSL only supports cipher suites with
leading zeros.

Analysis
========
CREATE TABLE of InnoDB table with a partition name
which exceeds the path limit can cause the server
to exit.

During the preparation of the partition name,
there was no check to identify whether the complete
path name for partition exceeds the max supported
path length, causing the server to exit during
subsequent processing.

Fix
===
During the preparation of partition name, check and report
an error if the partition path name exceeds the maximum path
name limit.

This is a 5.5 patch.

Backport from 5.7 to 5.5 Field_set::val_str()
should return String("", 0, cs) rather than String(NULL, 0, cs)

Problem & Analysis: Slave's Receiver thread, Applier thread and worker
    threads are created with LOCAL-INFILE option enabled. As the document
    says https://dev.mysql.com/doc/refman/5.7/en/load-data-local.html,
    there are some issues if a thread enables local infile.
    This flag should be enabled with care. But for the above mentioned
    internal threads, server is enabling it at the time of creation.

Fix: Further analysis on the code shows that none of threads really
    need this flag to be enabled at any time as Slave never executes
    "LOAD DATA LOCAL INFILE" after reading it from Relay log.
    Applier thread removes "LOCAL" before start executing the query.

Issue:
------
Running MTR with the --no-reorder option by specifying test cases on the
command line, without prefixing the suite name results in an error saying the
test case was not found in the main suite. This is because MTR looks for the
test case only in the main suite, and no other suites.

Fix:
----
The fix involves searching for the test in every suite if only the test name
is specified. This back-ports two bug fixes: Bug#24967869 and Bug#24365783
Reviewed-by: Pavan Naik <pavan.naik@oracle.com>
RB: 16812

    - Add mysqlcheck tool to docker rpms for upgrade

Corrected the revert.

(cherry picked from commit f637e524bf9b692c3ed46d856e2beac193b42a3e)