MDEV-9084 Calling a stored function from a nested select from temporary table causes unpredictable behavior

Cherry-pick: f4a0af070ce49abae60040f6f32e1074309c27fb
Author: Dmitry Lenev <dmitry.lenev@oracle.com>
Date:   Mon Jul 25 16:06:52 2016 +0300

  Fix for bug #16672723 "CAN'T FIND TEMPORARY TABLE".

  Attempt to execute prepared CREATE TABLE SELECT statement which used
  temporary table in the subquery in FROM clause and stored function
  failed with unwarranted ER_NO_SUCH_TABLE error. The same happened
  when such statement was used in stored procedure and this procedure
  was re-executed.

  The problem occurred because execution of such prepared statement/its
  re-execution as part of stored procedure incorrectly set
  Query_table_list::query_tables_own_last marker, indicating the last
  table which is directly used by statement. As result temporary table
  used in the subquery was treated as indirectly used/belonging to
  prelocking list and was not pre-opened by open_temporary_tables()
  call before statement execution. Thus causing ER_NO_SUCH_TABLE errors
  since our code assumes that temporary tables need to be correctly
  pre-opened before statement execution.

  This problem became visible only in version 5.6 after patches related to
  bug 11746602/27480 "EXTEND CREATE TEMPORARY TABLES PRIVILEGE TO ALLOW
  TEMP TABLE OPERATIONS" since they have introduced pre-opening of temporary
  tables for statements.

  Incorrect setting of Query_table_list::query_tables_own_last happened
  in LEX::first_lists_tables_same() method which is called by CREATE TABLE
  SELECT implementation as part of LEX::unlink_first_table(), which temporary
  excludes table list element for table being created from the query table
  list before handling SELECT part.

  LEX::first_lists_tables_same() tries to ensure that global table list of
  the statement starts with the first table list element from the first
  statement select. To do this it moves such table list element to the head
  of the global table list. If this table happens to be last directly-used
  table for the statement, query_tables_own_last marker is pointing to it.
  Since this marker was not updated when table list element was moved we
  ended up with all tables except the first table separated by it as if
  they were not directly used by statement (i.e. belonged to prelocked
  tables list).

  This fix changes code of LEX::first_lists_tables_same() to update
  query_tables_own_last marker in cases when it points to the table
  being moved. It is set to the table which precedes table being moved
  in this case.

Make the slave SQL thread always output to the error log the message "Slave
SQL thread exiting, replication stopped in ..." whenever it previously
outputted "Slave SQL thread initialized, starting replication ...".

Before this patch, it was somewhat inconsistent in which cases the message
would be output and in which not, depending on the exact time and cause of
the condition that caused the SQL thread to stop.

MTR raises default wait_for_pos_timeout from 300 to 1500 when tests
are run with valgrind. The same needs to be done for other
replication-related waits.

The change should fix one of failures mentioned in MDEV-10653
(rpl.rpl_parallel fails in buildbot with timeout), the one
on the valgrind builder; but not all of them

Add diagnostics output if any Sphinx components aren't found

The fix for bug mdev-5104 did not take into account that
for any call of setup_order the size of ref_array must
be big enough. This patch fixes this problem.

Memory was leaked when ALTER TABLE is attempted on a table
that contains corrupted indexes.
The memory leak was reported by AddressSanitizer for the test
innodb.innodb_corrupt_bit. The leak was introduced into
MariaDB Server 10.0.26, 10.1.15, 10.2.1 by the following:

commit c081c978
Merge: 1d21b221 a482e76e
Author: Sergei Golubchik <serg@mariadb.org>
Date:   Tue Jun 21 14:11:02 2016 +0200

   Merge branch '5.5' into bb-10.0

The warning is "blocks are still reachable in loss record",
happens in malloc / _dl_close_worker. Suppression added

When the test is run as a part of the suite with valgrind,
only allow it to be executed if --big-test is set.
If the test is run by specifying its name explicitly, it
will still be executed, even with valgrind without big-test,
MTR has special logic for that

The guilty part of the test checks for performance degradation on
a query with numerous joins on an empty table. The test expects
the query to take less than 1 second, and fails if it is not so
(which can happen on very slow builders).

The solution is to add more JOINs to the query. On a fixed server,
it should not have any noticeable impact on the query execution,
while on the unfixed version the query would take several times
longer (e.g. 6.5 sec vs 1.5 sec). Thus, we can increase the margin
for the error, and make the test fail when the query takes longer
than 5 seconds.

- fix the test to avoid false-negatives before MDEV-5114 patch;
- fix the race condition which made the test fail on slow builders

correct the error message in case of setuid/setgid failures

Ability to print lock type added.
Restoring correct lock type for CREATE VIEW added.

Backport the fix to 5.5, because it fails there too

The patch fixes two test failures:
- on slow builders, sometimes a connection attempt which should
  fail due to the exceeded number of thread_pool_max_threads
  actually succeeds;
- on even slow builders, MTR sometimes cannot establish the
  initial connection, and check-testcase fails prior to the
  test start

The problem with check-testcase was caused by connect-timeout=2
which was set for all clients in the test config file. On slow
builders it might be not enough.
There is no way to override it for the pre-test check, so it needed
to be substantially increased or removed.

The other problem was caused by a race condition between sleeps
that the test performs in existing connections and the connect
timeout for the connection attempt which was expected to fail.
If sleeps finished before the connect-timeout was exceeded, it
would allow the connection to succeed.

To solve each problem without making the other one worse,
connect-timeout should be configured dynamically during the test.
Due to the nature of the test (all connections must be busy
at the moment when we need to change the timeout, and cannot execute
SET GLOBAL ...), it needs to be done independently from the server.

The solution:
- recognize 'connect_timeout' as a connection option in mysqltest's
  "connect" command;
- remove connect-timeout from the test configuration file;
- use the new connect_timeout option for those connections which
  are expected to fail;
- re-arrange the test flow to allow running a huge SLEEP
  without affecting the test execution time (because it would be
  interrupted after the main test flow is finished).

The test is still subject to false negatives, e.g. if the connection
fails due to timeout rather than due to the exceeded number of
allowed threads, or if the connection on extra port succeeds due
to a race condition and not because the special logic for the extra
port. But those false negatives have always been possible there
on slow builders, they should not be critical because faster builders
should catch such failures if they appear.

Conflicts:
	client/mysqltest.cc
	mysql-test/r/pool_of_threads.result
	mysql-test/t/pool_of_threads.test

main.log_slow might leave mysql.slow_log table non-empty,
and tests which later use it might fail. Make sure that the table
is properly truncated

MariaDB Server 10.0.28 and 10.1.19 merged code from Percona XtraDB
that introduced support for compressed columns. Much but not all
of this code was disabled by placing #ifdef HAVE_PERCONA_COMPRESSED_COLUMNS
around it.

Among the unused but not disabled code is code to access
some new system tables related to compressed columns.

The creation of these system tables SYS_ZIP_DICT and SYS_ZIP_DICT_COLS
would cause a crash in --innodb-read-only mode when upgrading
from an earlier version to 10.0.28 or 10.1.19.

Let us remove all the dead code related to compressed columns.
Users who already upgraded to 10.0.28 and 10.1.19 will have the two
above mentioned empty tables in their InnoDB system tablespace.
Subsequent versions of MariaDB Server will completely ignore those tables.

The patch fixes two test failures:
- on slow builders, sometimes a connection attempt which should
  fail due to the exceeded number of thread_pool_max_threads
  actually succeeds;
- on even slow builders, MTR sometimes cannot establish the
  initial connection, and check-testcase fails prior to the
  test start

The problem with check-testcase was caused by connect-timeout=2
which was set for all clients in the test config file. On slow
builders it might be not enough.
There is no way to override it for the pre-test check, so it needed
to be substantially increased or removed.

The other problem was caused by a race condition between sleeps
that the test performs in existing connections and the connect
timeout for the connection attempt which was expected to fail.
If sleeps finished before the connect-timeout was exceeded, it
would allow the connection to succeed.

To solve each problem without making the other one worse,
connect-timeout should be configured dynamically during the test.
Due to the nature of the test (all connections must be busy
at the moment when we need to change the timeout, and cannot execute
SET GLOBAL ...), it needs to be done independently from the server.

The solution:
- recognize 'connect_timeout' as a connection option in mysqltest's
  "connect" command;
- remove connect-timeout from the test configuration file;
- use the new connect_timeout option for those connections which
  are expected to fail;
- re-arrange the test flow to allow running a huge SLEEP
  without affecting the test execution time (because it would be
  interrupted after the main test flow is finished).

The test is still subject to false negatives, e.g. if the connection
fails due to timeout rather than due to the exceeded number of
allowed threads, or if the connection on extra port succeeds due
to a race condition and not because the special logic for the extra
port. But those false negatives have always been possible there
on slow builders, they should not be critical because faster builders
should catch such failures if they appear.

Problem:- In replication if slave has extra persistent column then these
column are not computed while applying write-set from master.

Solution:- While applying row events from server, we will generate values
for extra persistent columns.

Test is very slow with valgrind, and pointless because it is
initially about a race condition which is hardly achievable
with valgrind

- in DOMNODELIST::DropItem
  if (Listp == NULL || Listp->length <= n)
    return true;
is wrong, should be:
  if (Listp == NULL || Listp->length < n)
    return true;
- Crash in discovery with libxml2 in XMLColumns because:
            if (!tdp->Usedom)    // nl was destroyed
              vp->nl = vp->pn->GetChildElements(g);
is executed with vp->pn uninitialized. Fixed by adding:
          vp->pn = node;
line 264.
-In discovery with libxml2 some columns are not found.
Because list was not recovered properly, nodes being modified and not reallocated.
Fixed lines 214 and 277.
  modified:   storage/connect/domdoc.cpp
  modified:   storage/connect/tabxml.cpp

Add support for zipped table files
  modified:   storage/connect/domdoc.cpp
  modified:   storage/connect/domdoc.h
  modified:   storage/connect/filamap.cpp
  modified:   storage/connect/filamap.h
  modified:   storage/connect/filamzip.cpp
  modified:   storage/connect/filamzip.h
  modified:   storage/connect/ha_connect.cc
  modified:   storage/connect/libdoc.cpp
  modified:   storage/connect/plgdbutl.cpp
  modified:   storage/connect/plgxml.cpp
  modified:   storage/connect/plgxml.h
  modified:   storage/connect/tabdos.cpp
  modified:   storage/connect/tabdos.h
  modified:   storage/connect/tabfmt.cpp
  modified:   storage/connect/tabjson.cpp
  modified:   storage/connect/tabxml.cpp

and use is_supported_parser_charset() instead of direct check

In file sql/filesort.cc,when merge_buffers() is called then
- queue_remove(&queue,0) is called
- For the function queue_remove there is assertion states that the element to be removed should have index >=1
- this is causing the assertion to fail.

Fixed by removing the top element.

MDEV-10386 Assertion `fixed == 1' failed in virtual String* Item_func_conv_charset::val_str(String*)

The patch b96c196f added a new call for
safe_charset_converter() without a corresponding fix_fields().
In case of a sub-query the created Item remained in non-fixed state.
The problem did not show up with literal derived expressions, only
subselects were affected. This patch adds a corresponding fix_fields()
to the previously added safe_charset_converter().

Added test case.

Do not use 0x%p to output thd address, use %p

count duplicate of UNION SELECT separately to awoid influence on lokal LIMIT clause.

Fix st_select_lex::is_merged_child_of to work across merged views or
derived tables.

The bug occurred when a subquery
- has a reference to outside, to grand-parent query or further up
- is converted to a semi-join (i.e. merged into its parent).

Then the reference to outside had form Item_ref(Item_field(...)).
- Conversion to semi-join would call item->fix_after_pullout() for the
  outside reference.
- Item_ref::fix_after_pullout would call Item_field->fix_after_pullout
- The Item_field would construct a new Name_resolution_context object
  This process ignored the fact that the Item_field does not belong to
  any of the subselects being flattened.
The result was crash in the next call to Item_field::fix_fields(), where
we would try to use an invalid Name_resolution_context object.

Fixed by not creating Name_resolution_context object if the Item_field's
context does not belong to the subselect(s) that were flattened.