mysql-test/r/union.result:
  Update test results (checked, it should be "DEPENDENT SUBQUERY" as it will actually run subquery's join multiple times. There is no reason to do that but executor will still do it)

Fix for the PBXT copy of subselect.test.

Bug#48623: Multiple subqueries are optimized incorrectly
      
The function setup_semijoin_dups_elimination() has a major loop that
goes through every table in the JOIN object. Usually, there is a normal
"plus one" increment in the for loop that implements this, but each semijoin
nest is treated as one entity and there is another increment that skips past
the semijoin nest to the next table in the JOIN object. However, when
combining these two increments, the next joined table is skipped, and if that
happens to be the start of another semijoin nest, the correct processing
for that nest will not be carried out.

mysql-test/r/subselect_sj.result:
  Added test results for bug#48623
mysql-test/r/subselect_sj_jcl6.result:
  Added test results for bug#48623
mysql-test/t/subselect_sj.test:
  Added test case for bug#48623
sql/opt_subselect.cc:
  Omitted the "plus one" increment in the for loop, added "plus one"
  in the remaining switch case, fixed coding style issue in remaining
  increment operations.

Fix two problems:
1. Let optimize_semijoin_nests() reset sj_nest->sjmat_info irrespectively  
   of value of optimizer_flag. We need this in case somebody has turned optimization
   off between reexecutions of the same statement.

2. Do not pull out constant tables out of semi-join nests. The problem is that pullout 
   operation is not undoable, and if a table is constant because it is 1/0-row table it 
   may cease to be constant on the next execution. Note that tables that are constant
   because of possible eq_ref(const) access will still be pulled out as they are
   considered functionally-dependent.

Bug#48213 Materialized subselect crashes if using GEOMETRY type

The problem occurred because during semi-join a materialized table
was created which contained a GEOMETRY column, which is a specialized
BLOB column.  This caused an segmentation fault because such tables will
have extra columns, and the semi-join code was not prepared for that.

The solution is to disable materialization when Blob/Geometry columns would 
need to be materialized.  Blob columns cannot be used for index look-up 
anyway, so it does not makes sense to use materialization.

This fix implies that it is detected earlier that subquery materialization
can not be used.  The result of that is that in->exist optimization may
be performed for such queries.  Hence, extended query plans for such
queries had to be updated.

mysql-test/r/subselect_mat.result:
  Update extended query plan for subqueries that cannot use materialization
  due to Blobs.
mysql-test/r/subselect_sj.result:
  Updated result file.
mysql-test/r/subselect_sj_jcl6.result:
  Update result file.
mysql-test/t/subselect_sj.test:
  Add test case for Bug#48213 that verifies that semi-join works when subquery select list contain Blob columns.  Also verify that materialization is not
  used.
sql/opt_subselect.cc:
  Disable materialization for semi-join/subqueries when the subquery select list
  contain Blob columns.

BUG#50019: Wrong result for IN-subquery with materialization
- Fix equality substitution in presense of semi-join materialization, lookup and scan variants
  (started off from fix by Evgen Potemkin, then modified it to work in all cases)

2. don't copy st_mysql_plugin structure unnecessary (sizeof hasn't changed)

This patch does three things:
- It adds the possibility to force the execution of top-level [NOT] IN
  subquery predicates via the IN=>EXISTS transformation. This is done by
  setting both optimizer switches partial_match_rowid_merge and
  partial_match_table_scan to "off".
- It adjusts all test cases where the complete optimizer_switch is
  selected because now we have two more switches.
- For those test cases where the plan changes because of the new available
  strategies, we switch off both partial match strategies in order to
  force the "old" IN=>EXISTS strategy. This is done because most of these
  test cases specifically test bugs in this strategy.

sql/opt_subselect.cc:
  Adds the possibility to force the execution of top-level [NOT] IN
  subquery predicates via the IN=>EXISTS transformation. This is done by
  setting both optimizer switches partial_match_rowid_merge and
  partial_match_table_scan to "off".

storage/myisam/mi_create.c:
  Don't set HA_OPTION_NULL_FIELDS if table is not using CHECKSUM as this makes the table incompatible with MySQL.

- Now DELETE IGNORE skips over rows with a foreign key constraints (as it was supposed to do)


mysql-test/r/foreign_key.result:
  Test case for Bug#44987 DELETE IGNORE and FK constraint
mysql-test/t/foreign_key.test:
  Test case for Bug#44987 DELETE IGNORE and FK constraint
sql/sql_delete.cc:
  Firx for Bug#44987 DELETE IGNORE and FK constraint
  Now DELETE IGNORE skips over rows with a foreign key constraints (as it was supposed to do)
  Bug fix inspired by: Moritz Mertinkat

Add some extra error output and code cleanup in an attempt to fix/debug
a rare random testsuite problem in check_warnings, where the exit code
from mysqltest is somehow corrupted inside mysql-test-run.pl.

include/my_global.h:
  Fix compiler warnings on some platforms.
mysql-test/lib/My/SafeProcess.pm:
  Move dereference of $? subprocess exit code closer to where it is generated,
  to make the code more robust and on the chance that this will fix the
  occasional problems in check_warnings we see in Buildbot.
mysql-test/mysql-test-run.pl:
  When check_warnings failed, also log the mysqld server for which it failed.
sql/sql_lex.cc:
  Fix compiler warning about possibly uninitialised value, by rewriting a for()
  loop that is always executed at least once into a do .. while() loop with an
  assert.
sql/table.cc:
  Fix compiler warning about uninitialised value.
storage/federatedx/ha_federatedx.cc:
  Fix uninitialised variable.
storage/maria/ma_delete.c:
  Fix compiler warning about uninitialised value.
storage/maria/ma_loghandler.c:
  Fix compiler warning about uninitialised value.
storage/myisam/ft_stopwords.c:
  Fix compiler warning.
storage/myisam/mi_write.c:
  Fix compiler warning about possibly uninitialised value, by rewriting a while()
  loop that is always executed at least once into a do .. while() loop with an
  assert.
storage/xtradb/btr/btr0cur.c:
  Fix compiler warning about possibly uninitialised value.
support-files/compiler_warnings.supp:
  Fix warning suppression to cover all cases in yassl.
vio/viossl.c:
  Fix compiler warning.

 - Adjust timing in test case, to avoid test failures caused by high load
   on machines and consequent race conditions in the test case.

 - Add another variant of Valgrind suppressions for memory leak in system
   libraries when unloading dynamic object files.


mysql-test/r/information_schema.result:
  Adjust timing to avoid test failures due to races.
mysql-test/t/information_schema.test:
  Adjust timing to avoid test failures due to races.
mysql-test/valgrind.supp:
  Add another variant of valgrind suppression for leak in system libs.

query:
- When subquery is located in ORDER BY, EXPLAIN will run as follows:
  select_describe() will run JOIN::prepare()/optimize() for the subquery;
  then at some point subselect_single_select_engine::prepare() will be called,
  which will create another join and run join->prepare().

  In mainline mysql this is not a problem because subquery's join will be 
  destroyed after the first call. 
  In MariaDB, it won't (table elimination needs to keep JOIN objects around
  for longer in order to know which tables were eliminated when constructing
  EXPLAIN EXTENDED warning).

  Fix the problem of memory leak by calling select_lex->cleanup() in 
  subselect_single_select_engine::prepare().

tmp_table_size can now be set to 0 (to disable in memory internal temp tables)
Improved speed for internal Maria temp tables:
- Don't use packed keys, except with long text fields.
- Don't copy key all accessed pages during key search.
Some new benchmark tests to sql-bench (for group by)

BUILD/compile-pentium64-gcov:
  Update script to use same pentium_config flags as other tests
BUILD/compile-pentium64-gprof:
  Update script to use same pentium_config flags as other tests
include/my_sys.h:
  Added count of my_sync calls
mysql-test/r/variables.result:
  tmp_table_size can now be set to 0
sql-bench/test-select.sh:
  Added some new test for GROUP BY on a not key field and group by with different order by
sql/mysqld.cc:
  Added count of my_sync calls
  tmp_table_size can now be set to 0 (to disable in memory internal temp tables)
sql/sql_select.cc:
  If tmp_table_size is 0, don't use in memory temp tables (good for benchmarking MyISAM/Maria temp tables)
  Don't pack keys for Maria tables;  The 8K page size makes packed keys too slow for temp tables.
storage/maria/ma_key_recover.h:
  Moved definition to maria_def.h
storage/maria/ma_page.c:
  Moved code used to simplify comparing of identical Maria tables to own function (page_cleanup())
  Fixed that one can read a page with a read lock.
storage/maria/ma_rkey.c:
  For not exact key reads, cache the page where we found key (to speed up future read-next/read-prev calls)
storage/maria/ma_search.c:
  Moved code to cache last key page to separate function.
  Instead of copying pages, only get a link to the page. This notable speeds up key searches on bigger tables.
storage/maria/ma_write.c:
  Added comment
storage/maria/maria_def.h:
  Moved page_cleanup() to separate function.

Bug#46949: memory leak with failed alter table to create partitions based on extract()

Bug#51830: Incorrect partition pruning on range partition (regression)

Fixed valgrind failure in select_describe(), read of uninitialized 
  Item_subselect::eliminated.

PBXT test file updates to reflect changes done in MySQL.

mysql-test/suite/pbxt/r/partition_error.result:
  Result file update following MySQL 5.1.44 changes.
mysql-test/suite/pbxt/r/partition_pruning.result:
  Result file update following MySQL 5.1.44 changes.
mysql-test/suite/pbxt/t/partition_error.test:
  Test file update following MySQL 5.1.44 changes.
sql/item_subselect.cc:
    Fixed valgrind failure in select_describe(), read of uninitialized 
    Item_subselect::eliminated: 
    - it turns out we can call select_describe() without having fixed 
      subquery items for child subselects. These are not the kind of subqueries
      that we could eliminate, so the fix is to ensure that 
      item_subselect->eliminated==FALSE even before fix_fields is called.
      Also added code to reset item_subselect->eliminated back to FALSE in
      Item::reset() call.
sql/item_subselect.h:
    Fixed valgrind failure in select_describe(), read of uninitialized 
    Item_subselect::eliminated: 
    - it turns out we can call select_describe() without having fixed 
      subquery items for child subselects. These are not the kind of subqueries
      that we could eliminate, so the fix is to ensure that 
      item_subselect->eliminated==FALSE even before fix_fields is called.
      Also added code to reset item_subselect->eliminated back to FALSE in
      Item::reset() call.
sql/sql_partition.cc:
  Fix Bug#51830: Revert part of the patch for Bug#49742, which caused the regression.
sql/table.cc:
  Fix Bug#46949: memory leak in failed ALTER TABLE with partitioning.

Automerge with 5.3-subqueries

* Implemented a second partial matching strategy via table scan.
  This strategy is a fallback when there is no memory for rowid merging.

* Refactored the selection and creation of partial matching strategies,
  so that the choice of strategy is encapsulated in a separate method
  choose_partial_match_strategy().

* Refactored the representation of partial match strategies so that:
  - each strategy is represented by a polymorphic class, and
  - the base class for all partial match strategies contains common
    execution code.

* Added an estimate of the memory needed for the rowid merge strategy,
  and the system variable "rowid_merge_buff_size" to control the maximum
  memory to be used by the rowid merge algorithm.

* Added two optimizer_switch system variables to control the choice of
  partial match strategy:
  "partial_match_rowid_merge", "partial_match_table_scan".

* Fixed multiple problems with deallocation of resources by the partial
  match strategies.


sql/mysql_priv.h:
  * Added two optimizer_switch system variables to control the choice of
    partial match strategy:
    "partial_match_rowid_merge", "partial_match_table_scan".
sql/mysqld.cc:
  * Added two optimizer_switch system variables to control the choice of
    partial match strategy:
    "partial_match_rowid_merge", "partial_match_table_scan".
  * Added a system variable "rowid_merge_buff_size" to control the maximum
    memory to be used by the rowid merge algorithm.
sql/set_var.cc:
  * Added a system variable "rowid_merge_buff_size" to control the maximum
    memory to be used by the rowid merge algorithm.
sql/sql_class.h:
  * Added a system variable "rowid_merge_buff_size" to control the maximum
    memory to be used by the rowid merge algorithm.
support-files/build-tags:
  Newer versions of BZR require the recursive flag in order to list all files.

- The problem was that DuplicateWeedout strategy setup code wasn't aware of the 
  fact that join buffering will be used and applied optimization that doesn't work
  together with join buffering. Fixed by making DuplicateWeedout setup code to have 
  a pessimistic check about whether there is a chance that join buffering will be 
  used.
- Make JOIN_CACHE_BKA::init() correctly process Copy_field elements that denote saving
  current rowids in the join buffer.

mysql-test/r/subselect_sj2.result:
  Update test results
mysql-test/r/subselect_sj2_jcl6.result:
  Update test results
mysql-test/r/subselect_sj_jcl6.result:
  Testcase
mysql-test/t/subselect_sj2.test:
  Update test results
mysql-test/t/subselect_sj_jcl6.test:
  Testcase
sql/opt_subselect.cc:
  - The problem was that DuplicateWeedout strategy setup code wasn't aware of the 
    fact that join buffering will be used and applied optimization that doesn't work
    together with join buffering. Fixed by making DuplicateWeedout setup code to have 
    a pessimistic check about whether there is a chance that join buffering will be 
    used.
sql/sql_join_cache.cc:
  Make JOIN_CACHE_BKA::init() correctly process Copy_field elements that denote saving current rowids in the join buffer.
sql/sql_select.cc:
  Added a question note

The function JOIN_CACHE::read_all_record_fields could return 0
for an incremental join cache in two cases:
1. there were no more records in the associated join buffer
2. there was no table fields stored in the join buffer.
As a result the function JOIN_CACHE::get_record() could
return prematurely and did not read all needed fields from
join buffers into the record buffer.

Now the function JOIN_CACHE::read_all_record_fields returns
-1 if there are no more records in the associated join buffer.

Save no-records constant tables in JOIN::const_table_map before we invoke
eliminate_tables(). Failure to do so caused crash when the same table was
marked as constant two times

mysql-test/r/table_elim.result:
  LPBUG#524025 Running RQG outer_join test leads to crash
  - Testcase
mysql-test/t/table_elim.test:
  LPBUG#524025 Running RQG outer_join test leads to crash
  - Testcase
sql/sql_select.cc:
  LPBUG#524025 Running RQG outer_join test leads to crash
  Save no-records constant tables in JOIN::const_table_map before we invoke eliminate_tables(). Failure to do so caused crash when the same table was marked as constant two times.

Made sure that join buffers could be used for inner tables of
any semi-join when the first match strategy is employed.

it's working on the WHERE clause.

Re-worked fix of Tor Didriksen:
 The problem was that fix_after_pullout() after semijoin conversion 
 wasn't propagated from the view to the underlying table. 
 On subesequent executions of the prepared statement, 
 we would mark the underlying table as 'dependent' and the predicate 
 anlysis would lead to a different (and illegal) execution plan.