Completed the fix for this bug.
Note: in 5.3 the affected 'if' statement in Item_in_subselect::single_value_transformer()
starting with the  condition (thd->variables.sql_mode & MODE_ONLY_FULL_GROUP_BY)
should be removed altogether. The change from table.cc is not needed either.
This is because in 5.3
 - min/max transformation for subqueries are done at the optimization phase
 - evaluation of the expensive subqueries is done at the execution phase.

Added an EXPLAIN EXTENDED to the test case for bug #12329653. 

The MIN/MAX optimizer code from the function opt_sum_query erroneously
did not take into account conjunctive conditions that did not depend on
any table, yet were not identified as constant items. These could be
items containing rand() or PS/SP parameters. These items are supposed
to be evaluated at the execution phase. That's why if such conditions
can be extracted from the WHERE condition the MIN/MAX optimization is
not applied as currently it is always done at the optimization phase.

(In 5.3 expensive subqueries are also evaluated only at the execution
phase. So, if a constant condition with such subquery can be extracted
from the WHERE clause the MIN/MAX optimization should not be applied 
in 5.3.)

IF an IN/ALL/SOME predicate with a constant left part is transformed
into an EXISTS subquery the resulting subquery should not be considered
uncacheable if the right part of the predicate is not uncacheable.

Backported the function dbug_print_item() from 5.3. The function is used
only for debugging.  
 

Problematic query:
insert ignore into `t1_federated` (`c1`) select `c1` from  `t1_local` a
where not exists (select 1 from `t1_federated` b where a.c1 = b.c1);
When this query is killed in another connection it could lead to crash.
The problem is follwing:
An attempt to obtain table statistics for subselect table in killed query
fails with an error. So JOIN::optimize() for subquery is failed but
it does not prevent further subquery evaluation.
At the first subquery execution JOIN::optimize() is called
(see subselect_single_select_engine::exec()) and fails with
an error. 'executed' flag is set to TRUE and it prevents
further subquery evaluation. At the second call
JOIN::optimize() does not happen as 'JOIN::optimized' is TRUE
and in case of uncacheable subquery the 'executed' flag is set
to FALSE before subquery evaluation. So we loose 'optimize stage'
error indication (see subselect_single_select_engine::exec()).
In other words 'executed' flag is used for two purposes, for
error indication at JOIN::optimize() stage and for an
indication of subquery execution. And it seems it's wrong
as the flag could be reset.


mysql-test/r/error_simulation.result:
  test case
mysql-test/t/error_simulation.test:
  test case
sql/item_subselect.cc:
  added new flag subselect_single_select_engine::optimize_error
  which is used for error detection which could happen at optimize
  stage.
sql/item_subselect.h:
  added new flag subselect_single_select_engine::optimize_error
sql/sql_select.cc:
  test case

  
  ALL subquery should return TRUE if subquery rowa set is empty independently
  of left part.  The problem was that Item_func_(eq,ne,gt,ge,lt,le) do not
  call execution of second argument if first is NULL no in this case subquery
  will not be executed and when Item_func_not_all calls any_value() of the
  subquery or aggregation function which report that there was rows. So for
  NULL < ALL (SELECT...) result was FALSE instead of TRUE.
  
  Fix is just swapping of arguments of Item_func_(eq,ne,gt,ge,lt,le) (with
  changing the operation if it is needed) so that result will be the same
  (for examole a < b is equal to b > a). This fix exploit the fact that
  first argument will be executed in any case.

For ANY subqueries NULLs should be ignored (if there is other values) when finding max min.
For ALL subqueries NULLs should be saved if they found.

Optimisation for ALL suqbueries if NULL is possible in the SELECT list with max/min aggregate function switched off.

Some test changed where NULL is not used but optimization with max/min aggregate function important so NOT NULL added.

mysql-test/r/explain.result:
  Forced old optimization.
mysql-test/r/subselect.result:
  Forced old optimization.
  
  New test suite.
mysql-test/t/explain.test:
  Forced old optimization.
mysql-test/t/subselect.test:
  Forced old optimization.
  
  New test suite.
sql/item_subselect.cc:
  Store converted subquery type.
  
  Switch off aggregate function optimisation for ALL and nulls.
sql/sql_class.cc:
  Fixed NULL comparison.
sql/sql_class.h:
  Store converted subquery type.

The query was re-written *after* we had tagged it with NON_AGG_FIELD_USED.
Remove the flag before continuing.


mysql-test/r/explain.result:
  Update test case for Bug#48295.
mysql-test/r/subselect.result:
  New test case.
mysql-test/t/explain.test:
  Update test case for Bug#48295.
mysql-test/t/subselect.test:
  New test case.
sql/item.cc:
  Use accessor functions for non_agg_field_used/agg_func_used.
sql/item_subselect.cc:
  Remove non_agg_field_used when we rewrite query '1 < some (...)' => '1 < max(...)'
sql/item_sum.cc:
  Use accessor functions for non_agg_field_used/agg_func_used.
sql/mysql_priv.h:
  Remove unused #defines.
sql/sql_lex.cc:
  Initialize new member variables.
sql/sql_lex.h:
  Replace full_group_by_flag with two boolean flags,
  and itroduce accessors for manipulating them.
sql/sql_select.cc:
  Use accessor functions for non_agg_field_used/agg_func_used.

> ------------------------------------------------------------
> revno: 3520
> revision-id: sergey.glukhov@oracle.com-20101214093303-wmo9mqcb8rz0wv9f
> parent: tor.didriksen@oracle.com-20101213161301-81lprlbune7r98dl
> committer: Sergey Glukhov <sergey.glukhov@oracle.com>
> branch nick: mysql-5.1-bugteam
> timestamp: Tue 2010-12-14 12:33:03 +0300
> message:
>   Fixed following problems:
>   --Bug#52157 various crashes and assertions with multi-table update, stored function
>   --Bug#54475 improper error handling causes cascading crashing failures in innodb/ndb
>   --Bug#57703 create view cause Assertion failed: 0, file .\item_subselect.cc, line 846
>   --Bug#57352 valgrind warnings when creating view
>   --Recently discovered problem when a nested materialized derived table is used
>     before being populated and it leads to incorrect result
>   
>   We have several modes when we should disable subquery evaluation.
>   The reasons for disabling are different. It could be
>   uselessness of the evaluation as in case of 'CREATE VIEW'
>   or 'PREPARE stmt', or we should disable subquery evaluation
>   if tables are not locked yet as it happens in bug#54475, or
>   too early evaluation of subqueries can lead to wrong result
>   as it happened in Bug#19077.
>   Main problem is that if subquery items are treated as const
>   they are evaluated in ::fix_fields(), ::fix_length_and_dec()
>   of the parental items as a lot of these methods have
>   Item::val_...() calls inside.
>   We have to make subqueries non-const to prevent unnecessary
>   subquery evaluation. At the moment we have different methods
>   for this. Here is a list of these modes:
>   
>   1. PREPARE stmt;
>   We use UNCACHEABLE_PREPARE flag.
>   It is set during parsing in sql_parse.cc, mysql_new_select() for
>   each SELECT_LEX object and cleared at the end of PREPARE in
>   sql_prepare.cc, init_stmt_after_parse(). If this flag is set
>   subquery becomes non-const and evaluation does not happen.
>   
>   2. CREATE|ALTER VIEW, SHOW CREATE VIEW, I_S tables which
>      process FRM files
>   We use LEX::view_prepare_mode field. We set it before
>   view preparation and check this flag in
>   ::fix_fields(), ::fix_length_and_dec().
>   Some bugs are fixed using this approach,
>   some are not(Bug#57352, Bug#57703). The problem here is
>   that we have a lot of ::fix_fields(), ::fix_length_and_dec()
>   where we use Item::val_...() calls for const items.
>   
>   3. Derived tables with subquery = wrong result(Bug19077)
>   The reason of this bug is too early subquery evaluation.
>   It was fixed by adding Item::with_subselect field
>   The check of this field in appropriate places prevents
>   const item evaluation if the item have subquery.
>   The fix for Bug19077 fixes only the problem with
>   convert_constant_item() function and does not cover
>   other places(::fix_fields(), ::fix_length_and_dec() again)
>   where subqueries could be evaluated.
>   
>   Example:
>   CREATE TABLE t1 (i INT, j BIGINT);
>   INSERT INTO t1 VALUES (1, 2), (2, 2), (3, 2);
>   SELECT * FROM (SELECT MIN(i) FROM t1
>   WHERE j = SUBSTRING('12', (SELECT * FROM (SELECT MIN(j) FROM t1) t2))) t3;
>   DROP TABLE t1;
>   
>   4. Derived tables with subquery where subquery
>      is evaluated before table locking(Bug#54475, Bug#52157)
>   
>   Suggested solution is following:
>   
>   -Introduce new field LEX::context_analysis_only with the following
>    possible flags:
>    #define CONTEXT_ANALYSIS_ONLY_PREPARE 1
>    #define CONTEXT_ANALYSIS_ONLY_VIEW    2
>    #define CONTEXT_ANALYSIS_ONLY_DERIVED 4
>   -Set/clean these flags when we perform
>    context analysis operation
>   -Item_subselect::const_item() returns
>    result depending on LEX::context_analysis_only.
>    If context_analysis_only is set then we return
>    FALSE that means that subquery is non-const.
>    As all subquery types are wrapped by Item_subselect
>    it allow as to make subquery non-const when
>    it's necessary.

- Removed files specific to compiling on OS/2
- Removed files specific to SCO Unix packaging
- Removed "libmysqld/copyright", text is included in documentation
- Removed LaTeX headers for NDB Doxygen documentation
- Removed obsolete NDB files
- Removed "mkisofs" binaries
- Removed the "cvs2cl.pl" script
- Changed a few GPL texts to use "program" instead of "library"

--Bug#52157 various crashes and assertions with multi-table update, stored function
--Bug#54475 improper error handling causes cascading crashing failures in innodb/ndb
--Bug#57703 create view cause Assertion failed: 0, file .\item_subselect.cc, line 846
--Bug#57352 valgrind warnings when creating view
--Recently discovered problem when a nested materialized derived table is used
  before being populated and it leads to incorrect result

We have several modes when we should disable subquery evaluation.
The reasons for disabling are different. It could be
uselessness of the evaluation as in case of 'CREATE VIEW'
or 'PREPARE stmt', or we should disable subquery evaluation
if tables are not locked yet as it happens in bug#54475, or
too early evaluation of subqueries can lead to wrong result
as it happened in Bug#19077.
Main problem is that if subquery items are treated as const
they are evaluated in ::fix_fields(), ::fix_length_and_dec()
of the parental items as a lot of these methods have
Item::val_...() calls inside.
We have to make subqueries non-const to prevent unnecessary
subquery evaluation. At the moment we have different methods
for this. Here is a list of these modes:

1. PREPARE stmt;
We use UNCACHEABLE_PREPARE flag.
It is set during parsing in sql_parse.cc, mysql_new_select() for
each SELECT_LEX object and cleared at the end of PREPARE in
sql_prepare.cc, init_stmt_after_parse(). If this flag is set
subquery becomes non-const and evaluation does not happen.

2. CREATE|ALTER VIEW, SHOW CREATE VIEW, I_S tables which
   process FRM files
We use LEX::view_prepare_mode field. We set it before
view preparation and check this flag in
::fix_fields(), ::fix_length_and_dec().
Some bugs are fixed using this approach,
some are not(Bug#57352, Bug#57703). The problem here is
that we have a lot of ::fix_fields(), ::fix_length_and_dec()
where we use Item::val_...() calls for const items.

3. Derived tables with subquery = wrong result(Bug19077)
The reason of this bug is too early subquery evaluation.
It was fixed by adding Item::with_subselect field
The check of this field in appropriate places prevents
const item evaluation if the item have subquery.
The fix for Bug19077 fixes only the problem with
convert_constant_item() function and does not cover
other places(::fix_fields(), ::fix_length_and_dec() again)
where subqueries could be evaluated.

Example:
CREATE TABLE t1 (i INT, j BIGINT);
INSERT INTO t1 VALUES (1, 2), (2, 2), (3, 2);
SELECT * FROM (SELECT MIN(i) FROM t1
WHERE j = SUBSTRING('12', (SELECT * FROM (SELECT MIN(j) FROM t1) t2))) t3;
DROP TABLE t1;

4. Derived tables with subquery where subquery
   is evaluated before table locking(Bug#54475, Bug#52157)

Suggested solution is following:

-Introduce new field LEX::context_analysis_only with the following
 possible flags:
 #define CONTEXT_ANALYSIS_ONLY_PREPARE 1
 #define CONTEXT_ANALYSIS_ONLY_VIEW    2
 #define CONTEXT_ANALYSIS_ONLY_DERIVED 4
-Set/clean these flags when we perform
 context analysis operation
-Item_subselect::const_item() returns
 result depending on LEX::context_analysis_only.
 If context_analysis_only is set then we return
 FALSE that means that subquery is non-const.
 As all subquery types are wrapped by Item_subselect
 it allow as to make subquery non-const when
 it's necessary.


mysql-test/r/derived.result:
  test case
mysql-test/r/multi_update.result:
  test case
mysql-test/r/view.result:
  test case
mysql-test/suite/innodb/r/innodb_multi_update.result:
  test case
mysql-test/suite/innodb/t/innodb_multi_update.test:
  test case
mysql-test/suite/innodb_plugin/r/innodb_multi_update.result:
  test case
mysql-test/suite/innodb_plugin/t/innodb_multi_update.test:
  test case
mysql-test/t/derived.test:
  test case
mysql-test/t/multi_update.test:
  test case
mysql-test/t/view.test:
  test case
sql/item.cc:
  --removed unnecessary code
sql/item_cmpfunc.cc:
  --removed unnecessary checks
  --THD::is_context_analysis_only() is replaced with LEX::is_ps_or_view_context_analysis()
sql/item_func.cc:
  --refactored context analysis checks
sql/item_row.cc:
  --removed unnecessary checks
sql/item_subselect.cc:
  --removed unnecessary code
  --added DBUG_ASSERT into Item_subselect::exec()
    which asserts that subquery execution can not happen
    if LEX::context_analysis_only is set, i.e. at context
    analysis stage.
  --Item_subselect::const_item()
    Return FALSE if LEX::context_analysis_only is set.
    It prevents subquery evaluation in ::fix_fields &
    ::fix_length_and_dec at context analysis stage.
sql/item_subselect.h:
  --removed unnecessary code
sql/mysql_priv.h:
  --Added new set of flags.
sql/sql_class.h:
  --removed unnecessary code
sql/sql_derived.cc:
  --added LEX::context_analysis_only analysis intialization/cleanup
sql/sql_lex.cc:
  --init LEX::context_analysis_only field
sql/sql_lex.h:
  --New LEX::context_analysis_only field
sql/sql_parse.cc:
  --removed unnecessary code
sql/sql_prepare.cc:
  --removed unnecessary code
  --added LEX::context_analysis_only analysis intialization/cleanup
sql/sql_select.cc:
  --refactored context analysis checks
sql/sql_show.cc:
  --added LEX::context_analysis_only analysis intialization/cleanup
sql/sql_view.cc:
  --added LEX::context_analysis_only analysis intialization/cleanup

create_sort_index() function overwrites original JOIN_TAB::type field.
At re-execution of subquery overwritten JOIN_TAB::type(JT_ALL) is
used instead of JT_FT. It misleads test_if_skip_sort_order() and
the function tries to find suitable key for the order that should
not be allowed for FULLTEXT(JT_FT) table.
The fix is to restore JOIN_TAB strucures for subselect on re-execution
for EXPLAIN.
Additional fix:
Update TABLE::maybe_null field which
affects list_contains_unique_index() behaviour as it
could have the value(maybe_null==TRUE) based on the
assumption that this join is outer
(see setup_table_map() func).


mysql-test/r/explain.result:
  test case
mysql-test/t/explain.test:
  test case
sql/item_subselect.cc:
  Make subquery uncacheable in case of EXPLAIN. It allows to keep
  original JOIN_TAB::type(see JOIN::save_join_tab) and restore it
  on re-execution.
sql/sql_select.cc:
  -restore JOIN_TAB strucures for subselect on re-execution for EXPLAIN
  -Update TABLE::maybe_null field as it could have
   the value(maybe_null==TRUE) based on the assumption
   that this join is outer(see setup_table_map() func).
   This change is not related to the crash problem but
   affects EXPLAIN results in the test case.

The crash happens because original join table is replaced with temporary table
at execution stage and later we attempt to use this temporary table in
select_describe. It might happen that
Item_subselect::update_used_tables() method which sets const_item flag
is not called by some reasons (no where/having conditon in subquery for example).
It prevents JOIN::join_tmp creation and breaks original join.
The fix is to call ::update_used_tables() before ::const_item() check.


mysql-test/r/ps.result:
  test case
mysql-test/t/ps.test:
  test case
sql/item_subselect.cc:
  call ::update_used_tables() before ::const_item() check.

            result

Row subqueries producing no rows were not handled as UNKNOWN
values in row comparison expressions.

That was a result of the following two problems:

1. Item_singlerow_subselect did not mark the resulting row
value as NULL/UNKNOWN when no rows were produced.

2. Arg_comparator::compare_row() did not take into account that
a whole argument may be NULL rather than just individual scalar
values.

Before bug#34384 was fixed, the above problems were hidden
because an uninitialized (i.e. without any stored value) cached
object would appear as NULL for scalar values in a row subquery
returning an empty result. After the fix
Arg_comparator::compare_row() would try to evaluate
uninitialized cached objects.

Fixed by removing the aforementioned problems.


mysql-test/r/row.result:
  Added a test case for bug #54190.
mysql-test/r/subselect.result:
  Updated the result for a test relying on wrong behavior.
mysql-test/t/row.test:
  Added a test case for bug #54190.
sql/item_cmpfunc.cc:
  If either of the argument rows is NULL, return NULL as the
  result of comparison.
sql/item_subselect.cc:
  Adjust null_value for Item_singlerow_subselect depending on
  whether a row has been produced by the row subquery.

      
The EXISTS transformation has additional switches to catch the known corner
cases that appear when transforming an IN predicate into EXISTS. Guarded
conditions are used which are deactivated when a NULL value is seen in the
outer expression's row. When the inner query block supplies NULL values,
however, they are filtered out because no distinction is made between the
guarded conditions; guarded NOT x IS NULL conditions in the HAVING clause that
filter out NULL values cannot be de-activated in isolation from those that
match values or from the outer expression or NULL's.

The above problem is handled by making the guarded conditions remember whether
they have rejected a NULL value or not, and index access methods are taking
this into account as well. 

The bug consisted of 

1) Not resetting the property for every nested loop iteration on the inner
   query's result.

2) Not propagating the NULL result properly from inner query to IN optimizer.

3) A hack that may or may not have been needed at some point. According to a
   comment it was aimed to fix #2 by returning NULL when FALSE was actually
   the result. This caused failures when #2 was properly fixed. The hack is
   now removed.

The fix resolves all three points.

The issue was that we didn't always check result of ha_rnd_init() which caused a problem for handlers that returned an error in this code.
- Changed prototype of ha_rnd_init() to ensure that we get a compile warning if result is not checked.
- Added ha_rnd_init_with_error() that prints error on failure.
- Checked all usage of ha_rnd_init() and ensure we generate an error message on failures.
- Changed init_read_record() to return 1 on failure.




sql/create_options.cc:
  Fixed wrong printf
sql/event_db_repository.cc:
  Check result from init_read_record()
sql/events.cc:
  Check result from init_read_record()
sql/filesort.cc:
  Check result from ha_rnd_init()
sql/ha_partition.cc:
  Check result from ha_rnd_init()
sql/ha_partition.h:
  Fixed compiler warning
sql/handler.cc:
  Added ha_rnd_init_with_error()
  Check result from ha_rnd_init()
sql/handler.h:
  Added ha_rnd_init_with_error()
  Changed prototype of ha_rnd_init() to ensure that we get a compile warning if result is not checked
sql/item_subselect.cc:
  Check result from ha_rnd_init()
sql/log.cc:
  Check result from ha_rnd_init()
sql/log_event.cc:
  Check result from ha_rnd_init()
sql/log_event_old.cc:
  Check result from ha_rnd_init()
sql/mysql_priv.h:
  init_read_record() now returns error code on failure
sql/opt_range.cc:
  Check result from ha_rnd_init()
sql/records.cc:
  init_read_record() now returns error code on failure
  Check result from ha_rnd_init()
sql/sql_acl.cc:
  Check result from init_read_record()
sql/sql_cursor.cc:
  Print error if ha_rnd_init() fails
sql/sql_delete.cc:
  Check result from init_read_record()
sql/sql_help.cc:
  Check result from init_read_record()
sql/sql_plugin.cc:
  Check result from init_read_record()
sql/sql_select.cc:
  Check result from ha_rnd_init()
  Print error if ha_rnd_init() fails.
sql/sql_servers.cc:
  Check result from init_read_record()
sql/sql_table.cc:
  Check result from init_read_record()
sql/sql_udf.cc:
  Check result from init_read_record()
sql/sql_update.cc:
  Check result from init_read_record()
storage/example/ha_example.cc:
  Don't return error on rnd_init()
storage/ibmdb2i/ha_ibmdb2i.cc:
  Removed not relevant comment

            strict aliasing violations.

One somewhat major source of strict-aliasing violations and
related warnings is the SQL_LIST structure. For example,
consider its member function `link_in_list` which takes
a pointer to pointer of type T (any type) as a pointer to
pointer to unsigned char. Dereferencing this pointer, which
is done to reset the next field, violates strict-aliasing
rules and might cause problems for surrounding code that
uses the next field of the object being added to the list.

The solution is to use templates to parametrize the SQL_LIST
structure in order to deference the pointers with compatible
types. As a side bonus, it becomes possible to remove quite
a few casts related to acessing data members of SQL_LIST.

sql/handler.h:
  Use the appropriate template type argument.
sql/item.cc:
  Remove now-unnecessary cast.
sql/item_subselect.cc:
  Remove now-unnecessary casts.
sql/item_sum.cc:
  Use the appropriate template type argument.
  Remove now-unnecessary cast.
sql/mysql_priv.h:
  Move SQL_LIST structure to sql_list.h
  Use the appropriate template type argument.
sql/sp.cc:
  Remove now-unnecessary casts.
sql/sql_delete.cc:
  Use the appropriate template type argument.
  Remove now-unnecessary casts.
sql/sql_derived.cc:
  Remove now-unnecessary casts.
sql/sql_lex.cc:
  Remove now-unnecessary casts.
sql/sql_lex.h:
  SQL_LIST now takes a template type argument which must
  match the type of the elements of the list. Use forward
  declaration when the type is not available, it is used
  in pointers anyway.
sql/sql_list.h:
  Rename SQL_LIST to SQL_I_List. The template parameter is
  the type of object that is stored in the list.
sql/sql_olap.cc:
  Remove now-unnecessary casts.
sql/sql_parse.cc:
  Remove now-unnecessary casts.
sql/sql_prepare.cc:
  Remove now-unnecessary casts.
sql/sql_select.cc:
  Remove now-unnecessary casts.
sql/sql_show.cc:
  Remove now-unnecessary casts.
sql/sql_table.cc:
  Remove now-unnecessary casts.
sql/sql_trigger.cc:
  Remove now-unnecessary casts.
sql/sql_union.cc:
  Remove now-unnecessary casts.
sql/sql_update.cc:
  Remove now-unnecessary casts.
sql/sql_view.cc:
  Remove now-unnecessary casts.
sql/sql_yacc.yy:
  Remove now-unnecessary casts.
storage/myisammrg/ha_myisammrg.cc:
  Remove now-unnecessary casts.

We should disable const subselect item evaluation because
subselect transformation does not happen in view_prepare_mode
and thus val_...() methods can not be called.


mysql-test/r/ctype_ucs.result:
  test case
mysql-test/r/view.result:
  test case
mysql-test/t/ctype_ucs.test:
  test case
mysql-test/t/view.test:
  test case
sql/item.cc:
  disabled const subselect item evaluation in
  view prepare mode.
sql/item_subselect.cc:
  added Item_subselect::safe_charset_converter which
  prevents const item evaluation in view prepare mode.
sql/item_subselect.h:
  added Item_subselect::safe_charset_converter which
  prevents const item evaluation in view prepare mode.

query:
- When subquery is located in ORDER BY, EXPLAIN will run as follows:
  select_describe() will run JOIN::prepare()/optimize() for the subquery;
  then at some point subselect_single_select_engine::prepare() will be called,
  which will create another join and run join->prepare().

  In mainline mysql this is not a problem because subquery's join will be 
  destroyed after the first call. 
  In MariaDB, it won't (table elimination needs to keep JOIN objects around
  for longer in order to know which tables were eliminated when constructing
  EXPLAIN EXTENDED warning).

  Fix the problem of memory leak by calling select_lex->cleanup() in 
  subselect_single_select_engine::prepare().

Bug#46949: memory leak with failed alter table to create partitions based on extract()

Bug#51830: Incorrect partition pruning on range partition (regression)

Fixed valgrind failure in select_describe(), read of uninitialized 
  Item_subselect::eliminated.

PBXT test file updates to reflect changes done in MySQL.

mysql-test/suite/pbxt/r/partition_error.result:
  Result file update following MySQL 5.1.44 changes.
mysql-test/suite/pbxt/r/partition_pruning.result:
  Result file update following MySQL 5.1.44 changes.
mysql-test/suite/pbxt/t/partition_error.test:
  Test file update following MySQL 5.1.44 changes.
sql/item_subselect.cc:
    Fixed valgrind failure in select_describe(), read of uninitialized 
    Item_subselect::eliminated: 
    - it turns out we can call select_describe() without having fixed 
      subquery items for child subselects. These are not the kind of subqueries
      that we could eliminate, so the fix is to ensure that 
      item_subselect->eliminated==FALSE even before fix_fields is called.
      Also added code to reset item_subselect->eliminated back to FALSE in
      Item::reset() call.
sql/item_subselect.h:
    Fixed valgrind failure in select_describe(), read of uninitialized 
    Item_subselect::eliminated: 
    - it turns out we can call select_describe() without having fixed 
      subquery items for child subselects. These are not the kind of subqueries
      that we could eliminate, so the fix is to ensure that 
      item_subselect->eliminated==FALSE even before fix_fields is called.
      Also added code to reset item_subselect->eliminated back to FALSE in
      Item::reset() call.
sql/sql_partition.cc:
  Fix Bug#51830: Revert part of the patch for Bug#49742, which caused the regression.
sql/table.cc:
  Fix Bug#46949: memory leak in failed ALTER TABLE with partitioning.

The problem was that killing a query during the optimization
phase of a subselect would lead to crashes. The root of the
problem is that the subselect execution engine ignores failures
(eg: killed) during the optimization phase (JOIN::optimize),
leading to a crash once the subquery is executed due to
partially initialized structures (in this case a join tab).

The optimal solution would be to cleanup certain optimizer
structures if the optimization phase fails, but currently
there is no infrastructure to properly to track and cleanup
the structures. To workaround the whole problem one somewhat
good solution is to avoid executing a subselect if the query
has been killed. Cutting short any problems caused by failures
during the optimization phase.

sql/item_subselect.cc:
  Do not execute a subselect if the session or query has been killed.

If an outer query is broken, a subquery might not even get set up.
EXPLAIN EXTENDED did not expect this and merrily tried to de-ref all
of the half-setup info.

We now catch this case and print as much as we have, as it doesn't cost us
anything (doesn't make regular execution slower).

backport from 5.1

mysql-test/r/explain.result:
  Show that EXPLAIN EXTENDED with subquery and illegal out query doesn't crash.
  Show also that SHOW WARNINGS will render an additional Note in the hope of
  being, well, helpful.
mysql-test/t/explain.test:
  If we have only half a query for EXPLAIN EXTENDED to print (i.e.,
  incomplete subquery info as outer query is illegal), we should
  provide the user with as much info as we easily can if they ask
  for it. What we should not do is crash when they come asking for
  help, that violates etiquette in some countries.
sql/item_subselect.cc:
  If the sub-query's actually set up, print it. Otherwise, elide.

> ------------------------------------------------------------
> revno: 2818.1.19
> revision-id: kostja@sun.com-20091103165854-7di545xruez8w207
> parent: li-bing.song@sun.com-20091103090041-zj7nedx6ok5jgges
> committer: Konstantin Osipov <kostja@sun.com>
> branch nick: 5.0-41756
> timestamp: Tue 2009-11-03 19:58:54 +0300
> message:
>   A fix and a test case for
>   Bug#41756 "Strange error messages about locks from InnoDB".
>   
>   In JT_EQ_REF (join_read_key()) access method,
>   don't try to unlock rows in the handler, unless certain that
>   a) they were locked
>   b) they are not used.
>   
>   Unlocking of rows is done by the logic of the nested join loop,
>   and is unaware of the possible caching that the access method may
>   have. This could lead to double unlocking, when a row
>   was unlocked first after reading into the cache, and then
>   when taken from cache, as well as to unlocking of rows which
>   were actually used (but taken from cache).
>   
>   Delegate part of the unlocking logic to the access method,
>   and in JT_EQ_REF count how many times a record was actually
>   used in the join. Unlock it only if it's usage count is 0.
>   
>   Implemented review comments.

> ------------------------------------------------------------
> revno: 3190 [merge]
> revision-id: kostja@sun.com-20091103174552-bfpak6r7ngf5cbjb
> parent: magnus.blaudd@sun.com-20091103170719-6b64sjnivsiyz6xy
> parent: kostja@sun.com-20091103165854-7di545xruez8w207
> committer: Konstantin Osipov <kostja@sun.com>
> branch nick: 5.1-41756
> timestamp: Tue 2009-11-03 20:45:52 +0300
> message:
>   A fix and a test case for 
>   Bug#41756 "Strange error messages about locks from InnoDB".
>         
>   In JT_EQ_REF (join_read_key()) access method, 
>   don't try to unlock rows in the handler, unless certain that 
>   a) they were locked
>   b) they are not used.
>   
>   Unlocking of rows is done by the logic of the nested join loop,
>   and is unaware of the possible caching that the access method may
>   have. This could lead to double unlocking, when a row
>   was unlocked first after reading into the cache, and then 
>   when taken from cache, as well as to unlocking of rows which
>   were actually used (but taken from cache).
>         
>   Delegate part of the unlocking logic to the access method,
>   and in JT_EQ_REF count how many times a record was actually 
>   used in the join. Unlock it only if it's usage count is 0.
>   
>   Implemented review comments.
> ------------------------------------------------------------
> Use --include-merges or -n0 to see merged revisions.

> ------------------------------------------------------------
> revno: 3148.9.3
> revision-id: azundris@mysql.com-20091029230154-jp2xqvzw2nhj9q41
> parent: azundris@mysql.com-20091027095316-54lwjr9vqkscq1ik
> committer: Tatiana A. Nurnberg <azundris@mysql.com>
> branch nick: 51-48295
> timestamp: Thu 2009-10-29 16:01:54 -0700
> message:
>   Bug#48295: explain extended crash with subquery and ONLY_FULL_GROUP_BY sql_mode
>   
>   If an outer query is broken, a subquery might not even get set up.
>   EXPLAIN EXTENDED did not expect this and merrily tried to de-ref all
>   of the half-setup info.
>   
>   We now catch this case and print as much as we have, as it doesn't cost us
>   anything (doesn't make regular execution slower).

When values of different types are compared they're converted to a type that
allows correct comparison. This conversion is done for each comparison and
takes some time. When a constant is being compared it's possible to cache the
value after conversion to speedup comparison. In some cases (large dataset,
complex WHERE condition with many type conversions) query might be executed
7% faster.

A test case isn't provided because all changes are internal and isn't visible
outside.

The behavior of the Item_cache is changed to cache values on the first request
of cached value rather than at the moment of storing item to be cached.
A flag named value_cached is added to the Item_cache class. It's set to TRUE
when cache holds the value of the last stored item.
Function named cache_value() is added to the Item_cache class and derived classes.
This function actually caches the value of the saved item.
Item_cache_xxx::store functions now only store item to be cached and set
value_cached flag to FALSE.
Item_cache_xxx::val_xxx functions are changed to call cache_value function
prior to returning cached value if value_cached is FALSE.
The Arg_comparator::set_cmp_func function now calls cache_converted_constant
to cache constants if they need a type conversion.
The Item_cache::get_cache function is overloaded to allow setting of the
cache type.
The cache_converted_constant function is added to the Arg_comparator class.
It checks whether a value can and should be cached and if so caches it.

sql/item.cc:
  Bug#34384: Slow down on constant conversion.
  Function named cache_value() is added to the Item_cache class and derived classes.
  This function actually caches the value of the saved item.
  Item_cache_xxx::store functions now only store item to be cached and set
  value_cached flag to FALSE.
  Item_cache_xxx::val_xxx functions are changed to call cache_value function
  prior to returning cached value if value_cached is FALSE.
  The Item_cache::get_cache function is overloaded to allow setting of the
  cache type.
sql/item.h:
  Bug#34384: Slow down on constant conversion.
  A flag named value_cached is added to the Item_cache class. It's set to TRUE
  when we need to start caching values when the store method is called.
  Function named cache_value() is added to the Item_cache class and derived classes.
sql/item_cmpfunc.cc:
  Bug#34384: Slow down on constant conversion.
  A helper function cache_converted_constant is added to the Arg_comparator class.
  It checks whether a given item can and should be cached and caches it if so.
  The Arg_comparator::set_cmp_func function now calls cache_converted_constant
  to cache constants if they need a type conversion.
sql/item_cmpfunc.h:
  Bug#34384: Slow down on constant conversion.
  The cache_converted_constant function is added to the Arg_comparator class.
  It checks whether a value can and should be cached and if so caches it.
sql/item_subselect.cc:
  Bug#34384: Slow down on constant conversion.
  Force immediate caching of subselect result.
sql/item_xmlfunc.cc:
  Bug#34384: Slow down on constant conversion.
sql/sp_rcontext.cc:
  Bug#34384: Slow down on constant conversion.
  Force immediate caching of values of an SP CASE function.

Bug#41756 "Strange error messages about locks from InnoDB".

In JT_EQ_REF (join_read_key()) access method,
don't try to unlock rows in the handler, unless certain that
a) they were locked
b) they are not used.

Unlocking of rows is done by the logic of the nested join loop,
and is unaware of the possible caching that the access method may
have. This could lead to double unlocking, when a row
was unlocked first after reading into the cache, and then
when taken from cache, as well as to unlocking of rows which
were actually used (but taken from cache).

Delegate part of the unlocking logic to the access method,
and in JT_EQ_REF count how many times a record was actually
used in the join. Unlock it only if it's usage count is 0.

Implemented review comments.


mysql-test/r/bug41756.result:
  Add result file (Bug#41756)
mysql-test/t/bug41756-master.opt:
  Use --innodb-locks-unsafe-for-binlog, as in 5.0 just
  using read_committed isolation is not sufficient to 
  reproduce the bug.
mysql-test/t/bug41756.test:
  Add a test file (Bug#41756)
sql/item_subselect.cc:
  Complete struct READ_RECORD initialization with a new
  member to unlock records.
sql/records.cc:
  Extend READ_RECORD API with a method to unlock read records.
sql/sql_select.cc:
  In JT_EQ_REF (join_read_key()) access method,
  don't try to unlock rows in the handler, unless certain that
  a) they were locked
  b) they are not used.
sql/sql_select.h:
  Add members to TABLE_REF to count TABLE_REF buffer usage count.
sql/structs.h:
  Update declarations.

If an outer query is broken, a subquery might not even get set up.
EXPLAIN EXTENDED did not expect this and merrily tried to de-ref all
of the half-setup info.

We now catch this case and print as much as we have, as it doesn't cost us
anything (doesn't make regular execution slower).

mysql-test/r/explain.result:
  Show that EXPLAIN EXTENDED with subquery and illegal out query doesn't crash.
  Show also that SHOW WARNINGS will render an additional Note in the hope of
  being, well, helpful.
mysql-test/t/explain.test:
  If we have only half a query for EXPLAIN EXTENDED to print (i.e.,
  incomplete subquery info as outer query is illegal), we should
  provide the user with as much info as we easily can if they ask
  for it. What we should not do is crash when they come asking for
  help, that violates etiquette in some countries.
sql/item_subselect.cc:
  If the sub-query's actually set up, print it. Otherwise, elide.

The original code comes, as far as I know, from Google (Mark Callaghan's team) with additional work from Percona, Ourdelta and Weldon Whipple.

This code provides the same functionallity, but with a lot of changes to make it faster and better fit the MariaDB infrastucture.

Added new status variables:
- Com_show_client_statistics, Com_show_index_statistics, Com_show_table_statistics, Com_show_user_statistics
- Access_denied_errors, Busy_time (clock time), Binlog_bytes_written, Cpu_time, Empty_queries, Rows_sent, Rows_read

Added new variable / startup option 'userstat' to control if user statistics should be enabled or not

Added my_getcputime(); Returns cpu time used by this thread.
New FLUSH commands:
- FLUSH SLOW QUERY LOG
- FLUSH TABLE_STATISTICS
- FLUSH INDEX_STATISTICS
- FLUSH USER_STATISTICS
- FLUSH CLIENT_STATISTICS

New SHOW commands:
- SHOW CLIENT_STATISTICS
- SHOW USER_STATISTICS
- SHOW TABLE_STATISTICS
- SHOW INDEX_STATISTICS

New Information schemas:
- CLIENT_STATISTICS
- USER_STATISTICS
- INDEX_STATISTICS
- TABLE_STATISTICS

Added support for all new flush commands to mysqladmin

Added handler::ha_... wrappers for all handler read calls to do statistics counting
- Changed all code to use new ha_... calls
- Count number of read rows, changed rows and rows read trough an index

Added counting of number of bytes sent to binary log (status variable Binlog_bytes_written)
Added counting of access denied errors (status variable Access_denied_erors)

Bugs fixed:
- Fixed bug in add_to_status() and add_diff_to_status() where longlong variables where threated as long
- CLOCK_GETTIME was not propely working on Linuxm

client/mysqladmin.cc:
  Added support for all new flush commmands and some common combinations:
  
  flush-slow-log
  flush-table-statistics
  flush-index-statistics
  flush-user-statistics
  flush-client-statistics
  flush-all-status
  flush-all-statistics
configure.in:
  Added checking if clock_gettime needs the librt.
  (Fixes Bug #37639 clock_gettime is never used/enabled in Linux/Unix)
include/my_sys.h:
  Added my_getcputime()
include/mysql_com.h:
  Added LIST_PROCESS_HOST_LEN & new REFRESH target defines
mysql-test/r/information_schema.result:
  New information schema tables added
mysql-test/r/information_schema_all_engines.result:
  New information schema tables added
mysql-test/r/information_schema_db.result:
  New information schema tables added
mysql-test/r/log_slow.result:
  Added testing that flosh slow query logs is accepted
mysql-test/r/status_user.result:
  Basic testing of user, client, table and index statistics
mysql-test/t/log_slow.test:
  Added testing that flosh slow query logs is accepted
mysql-test/t/status_user-master.opt:
  Ensure that we get a fresh restart before running status_user.test
mysql-test/t/status_user.test:
  Basic testing of user, client, table and index statistics
mysys/my_getsystime.c:
  Added my_getcputime()
  Returns cpu time used by this thread.
sql/authors.h:
  Updated authors to have core and original MySQL developers first.
sql/event_data_objects.cc:
  Updated call to mysql_reset_thd_for_next_command()
sql/event_db_repository.cc:
  Changed to use new ha_... calls
sql/filesort.cc:
  Changed to use new ha_... calls
sql/ha_partition.cc:
  Changed to use new ha_... calls
  Fixed comment syntax
sql/handler.cc:
  Changed to use new ha_... calls
  Reset table statistics
  Added code to update global table and index status
  Added counting of rows changed
sql/handler.h:
  Added table and index statistics variables
  Added function reset_statistics()
  Added handler::ha_... wrappers for all handler read calls to do statistics counting
  Protected all normal read calls to ensure we use the new calls in the server.
  Made ha_partition a friend class so that partition code can call the old read functions
sql/item_subselect.cc:
  Changed to use new ha_... calls
sql/lex.h:
  Added keywords for new information schema tables and flush commands
sql/log.cc:
  Added flush_slow_log()
  Added counting of number of bytes sent to binary log
  Removed not needed test of thd (It's used before, so it's safe to use)
  Added THD object to MYSQL_BIN_LOG::write_cache() to simplify statistics counting
sql/log.h:
  Added new parameter to write_cache()
  Added flush_slow_log() functions.
sql/log_event.cc:
  Updated call to mysql_reset_thd_for_next_command()
  Changed to use new ha_... calls
sql/log_event_old.cc:
  Updated call to mysql_reset_thd_for_next_command()
  Changed to use new ha_... calls
sql/mysql_priv.h:
  Updated call to mysql_reset_thd_for_next_command()
  Added new statistics functions and variables needed by these.
sql/mysqld.cc:
  Added new statistics variables and structures to handle these
  Added new status variables:
  - Com_show_client_statistics, Com_show_index_statistics, Com_show_table_statistics, Com_show_user_statistics
  - Access_denied_errors, Busy_time (clock time), Binlog_bytes_written, Cpu_time, Empty_queries, Rows_set, Rows_read
  Added new option 'userstat' to control if user statistics should be enabled or not
sql/opt_range.cc:
  Changed to use new ha_... calls
sql/opt_range.h:
  Changed to use new ha_... calls
sql/opt_sum.cc:
  Changed to use new ha_... calls
sql/records.cc:
  Changed to use new ha_... calls
sql/set_var.cc:
  Added variable 'userstat'
sql/sp.cc:
  Changed to use new ha_... calls
sql/sql_acl.cc:
  Changed to use new ha_... calls
  Added counting of access_denied_errors
sql/sql_base.cc:
  Added call to statistics functions
sql/sql_class.cc:
  Added usage of org_status_var, to store status variables at start of command
  Added functions THD::update_stats(), THD::update_all_stats()
  Fixed bug in add_to_status() and add_diff_to_status() where longlong variables where threated as long
sql/sql_class.h:
  Added new status variables to status_var
  Moved variables that was not ulong in status_var last.
  Added variables to THD for storing temporary values during statistics counting
sql/sql_connect.cc:
  Variables and functions to calculate user and client statistics
  Added counting of access_denied_errors and lost_connections
sql/sql_cursor.cc:
  Changed to use new ha_... calls
sql/sql_handler.cc:
  Changed to use new ha_... calls
sql/sql_help.cc:
  Changed to use new ha_... calls
sql/sql_insert.cc:
  Changed to use new ha_... calls
sql/sql_lex.h:
  Added SQLCOM_SHOW_USER_STATS, SQLCOM_SHOW_TABLE_STATS, SQLCOM_SHOW_INDEX_STATS, SQLCOM_SHOW_CLIENT_STATS
sql/sql_parse.cc:
  Added handling of:
  - SHOW CLIENT_STATISTICS
  - SHOW USER_STATISTICS
  - SHOW TABLE_STATISTICS
  - SHOW INDEX_STATISTICS
  Added handling of new FLUSH commands:
  - FLUSH SLOW QUERY LOGS
  - FLUSH TABLE_STATISTICS
  - FLUSH INDEX_STATISTICS
  - FLUSH USER_STATISTICS
  - FLUSH CLIENT_STATISTICS
  Added THD parameter to mysql_reset_thd_for_next_command()
  Added initialization and calls to user statistics functions
  Added increment of statistics variables empty_queries, rows_sent and access_denied_errors.
  Added counting of cpu time per query
sql/sql_plugin.cc:
  Changed to use new ha_... calls
sql/sql_prepare.cc:
  Updated call to mysql_reset_thd_for_next_command()
sql/sql_select.cc:
  Changed to use new ha_... calls
  Indentation changes
sql/sql_servers.cc:
  Changed to use new ha_... calls
sql/sql_show.cc:
  Added counting of access denied errors
  Added function for new information schema tables:
  - CLIENT_STATISTICS
  - USER_STATISTICS
  - INDEX_STATISTICS
  - TABLE_STATISTICS
  Changed to use new ha_... calls
sql/sql_table.cc:
  Changed to use new ha_... calls
sql/sql_udf.cc:
  Changed to use new ha_... calls
sql/sql_update.cc:
  Changed to use new ha_... calls
sql/sql_yacc.yy:
  Add new show and flush commands
sql/structs.h:
  Add name_length to KEY to avoid some strlen
  Added cache_name to KEY for fast storage of keyvalue in cache
  Added structs USER_STATS, TABLE_STATS, INDEX_STATS
  Added function prototypes for statistics functions
sql/table.cc:
  Store db+table+index name into keyinfo->cache_name
sql/table.h:
  Added new information schema tables
sql/tztime.cc:
  Changed to use new ha_... calls

on subquery inside a SP 

Problem: repeated call of a SP containing an incorrect query with a 
subselect may lead to failed ASSERT().

Fix: set proper sublelect's state in case of error occured during 
subquery transformation.


mysql-test/r/sp.result:
  Fix for bug#46629: Item_in_subselect::val_int(): Assertion `0' 
  on subquery inside a SP 
    - test result.
mysql-test/t/sp.test:
  Fix for bug#46629: Item_in_subselect::val_int(): Assertion `0' 
  on subquery inside a SP 
    - test case.
sql/item_subselect.cc:
  Fix for bug#46629: Item_in_subselect::val_int(): Assertion `0' 
  on subquery inside a SP 
    - don't set Item_subselect::changed in the Item_subselect::fix_fields()
  if an error occured during subquery transformation.
  That prevents us of further processing incorrect subqueries after 
  Item_in_subselect::select_in_like_transformer().

- Addressing review feedback, generation 4.

include/my_global.h:
  Make ALIGN_PTR's action correspond to that of ALIGN_SIZE
sql/item.cc:
  MWL#17: Table-elimination
  - Review feedback: function renames, better comments
sql/item.h:
  MWL#17: Table-elimination
  - Review feedback: function renames, better comments
sql/item_cmpfunc.cc:
  MWL#17: Table-elimination
  - Review feedback: function renames, better comments
sql/item_subselect.cc:
  MWL#17: Table-elimination
  - Review feedback: function renames, better comments
sql/item_subselect.h:
  MWL#17: Table-elimination
  - Review feedback: function renames, better comments
sql/opt_table_elimination.cc:
  MWL#17: Table-elimination
  - Addressing review feedback, generation 4: abstract everything in case
    we would need to change it for something else in the future.
sql/sql_list.h:
  MWL#17: Table-elimination
  - Introduce exchange_sort(List<T> ...) template function
sql/sql_select.cc:
  MWL#17: Table-elimination
  - Review feedback: function renames, better comments

In a subselect all fields from outer selects are marked as dependent on
selects they are belong to. In some cases optimizer substitutes it for an
equivalent expression. For example "a_field IN (SELECT outer_field)" is
substituted with "a_field = outer_field". As we moved the outer_field to the
upper select it's not really outer anymore. But it was left marked as outer.
If exists an index over a_field optimizer choose wrong execution plan and thus
return wrong result.

Now the Item_in_subselect::single_value_transformer function removes dependent
marking from fields when a subselect is optimized away.

mysql-test/r/subselect.result:
  Added a test case for the bug#46051.
mysql-test/t/subselect.test:
  Added a test case for the bug#46051.
sql/item_subselect.cc:
  Bug#46051: Incorrectly market field caused wrong result.
  Now the Item_in_subselect::single_value_transformer function removes dependent
  marking from fields when a subselect is optimized away.

- Moved table elimination code to sql/opt_table_elimination.cc
- Added comments 

.bzrignore:
  MWL#17: Table elimination
  - Moved table elimination code to sql/opt_table_elimination.cc
libmysqld/Makefile.am:
  MWL#17: Table elimination
  - Moved table elimination code to sql/opt_table_elimination.cc
sql/CMakeLists.txt:
  MWL#17: Table elimination
  - Moved table elimination code to sql/opt_table_elimination.cc
sql/Makefile.am:
  MWL#17: Table elimination
  - Moved table elimination code to sql/opt_table_elimination.cc

- Make elimination check to be able detect cases like  t.primary_key_col1=othertbl.col AND t.primary_key_col2=func(t.primary_key_col1).
  These are needed to handle e.g. the case of func() being a correlated subquery that selects the latest value.
- If we've removed a condition with subquery predicate, EXPLAIN [EXTENDED] won't show the subquery anymore

sql/item.cc:
  MWL#17: Table elimination
  - Add tem_field::check_column_usage_processor(). it allows to check which key parts a condition depends on.
sql/item.h:
  MWL#17: Table elimination
  - Add tem_field::check_column_usage_processor(). it allows to check which key parts a condition depends on.
sql/item_subselect.cc:
  MWL#17: Table elimination
  - Item_subselect got 'eliminated' attribute. It is used only to determine if the subselect should be printed by EXPLAIN.
  - Item_subselect got List<Item> refers_to - a list of item in the current select that are referred to from within the subselect.
  - Added Item_*::check_column_usage_processor(). it allows to check which key parts a condition depends on.
  - Added a comment about possible problem in Item_subselect::walk
sql/item_subselect.h:
  MWL#17: Table elimination
  - Item_subselect got 'eliminated' attribute. It is used only to determine if the subselect should be printed by EXPLAIN.
  - Item_subselect got List<Item> refers_to - a list of item in the current select that are referred to from within the subselect.
  - Added Item_*::check_column_usage_processor(). it allows to check which key parts a condition depends on.
sql/item_sum.cc:
  MWL#17: Table elimination
sql/sql_lex.cc:
  MWL#17: Table elimination
sql/sql_lex.h:
  MWL#17: Table elimination
sql/sql_select.h:
  MWL#17: Table elimination

with gcc 4.3.2
      
Compiling MySQL with gcc 4.3.2 and later produces a number of 
warnings, many of which are new with the recent compiler
versions.
      
This bug will be resolved in more than one patch to limit the
size of changesets. This is the first patch, fixing a number 
of the warnings, predominantly "suggest using parentheses 
around && in ||", and empty for and while bodies.

with gcc 4.3.2

Compiling MySQL with gcc 4.3.2 and later produces a number of 
warnings, many of which are new with the recent compiler
versions.

This bug will be resolved in more than one patch to limit the
size of changesets. This is the first patch, fixing a number 
of the warnings, predominantly "suggest using parentheses 
around && in ||", and empty for and while bodies.

- Added braces around expressions with &&, ||, & and |
- Added empty line before ; for empty while and for loops
- Added () around if with assignments
- Removed const before function returning simple type

Changed BUILD scripts to not build with NDB 


BUILD/SETUP.sh:
  By default, don't build ndb with --max in Maria tree.
  NDB is not kept up to date anyway in 5.1
client/mysql.cc:
  Added braces around && to get rid of compiler warnings
sql/event_db_repository.cc:
  Added braces around && to get rid of compiler warnings
sql/events.cc:
  Added braces around && to get rid of compiler warnings
sql/field.cc:
  Added braces around && to get rid of compiler warnings
  Fixed for loops
sql/field.h:
  Added braces around & to get rid of compiler warnings
sql/field_conv.cc:
  Added braces around && to get rid of compiler warnings
  Fixed bug when copying between DATETIME fields and strict dates are used
  Removed not needeed else
sql/gstream.cc:
  Added braces around && to get rid of compiler warnings
sql/ha_ndbcluster.cc:
  Added braces around && to get rid of compiler warnings
  Added {} to get rid of compiler warnings
sql/handler.cc:
  Added braces around && to get rid of compiler warnings
sql/item.cc:
  Added braces around && to get rid of compiler warnings
sql/item_cmpfunc.cc:
  Added braces around && to get rid of compiler warnings
  Removed some not needed space
sql/item_func.cc:
  Added braces around && to get rid of compiler warnings
sql/item_strfunc.cc:
  Added braces around && to get rid of compiler warnings
sql/item_subselect.cc:
  Added braces around && to get rid of compiler warnings
sql/item_sum.cc:
  Added braces around && to get rid of compiler warnings
sql/item_timefunc.cc:
  Added braces around && to get rid of compiler warnings
sql/item_xmlfunc.cc:
  Added empty line before ; for empty while and for loops
sql/log.cc:
  Added braces around && to get rid of compiler warnings
sql/log_event.cc:
  Added braces around && to get rid of compiler warnings
  Removed not needed else
sql/log_event_old.cc:
  Added braces around && to get rid of compiler warnings
sql/opt_range.cc:
  Added braces around && to get rid of compiler warnings
sql/opt_sum.cc:
  Added braces around && to get rid of compiler warnings
sql/set_var.cc:
  Added empty line before ; for empty while and for loops
  Added () around if with assignments
sql/slave.cc:
  Added braces around && to get rid of compiler warnings
  Added empty line before ; for empty while and for loops
sql/spatial.h:
  Added braces around && to get rid of compiler warnings
sql/sql_acl.cc:
  Added braces around && to get rid of compiler warnings
sql/sql_analyse.cc:
  Added empty line before ; for empty while and for loops
sql/sql_base.cc:
  Added braces around && to get rid of compiler warnings
sql/sql_connect.cc:
  Added braces around && to get rid of compiler warnings
sql/sql_db.cc:
  Added braces around && to get rid of compiler warnings
sql/sql_delete.cc:
  Added braces around && to get rid of compiler warnings
sql/sql_help.cc:
  Added empty line before ; for empty while and for loops
sql/sql_insert.cc:
  Added braces around && to get rid of compiler warnings
  Added () around if with assignments
sql/sql_lex.cc:
  Cast char array references to uchar; Fixed wrong array referencing when using characters > ASCII 128 in SQL statments
  Added empty line before ; for empty while and for loops
  Trivial indent fixes
  Added braces around && to get rid of compiler warnings
sql/sql_load.cc:
  Added braces around && to get rid of compiler warnings
sql/sql_parse.cc:
  Added braces around && to get rid of compiler warnings
sql/sql_partition.cc:
  Added braces around && to get rid of compiler warnings
sql/sql_plugin.cc:
  Fixed bug in detecing if option variable should be readonly
  Added empty line before ; for empty while and for loops
sql/sql_prepare.cc:
  Added braces around && to get rid of compiler warnings
sql/sql_select.cc:
  Added braces around && to get rid of compiler warnings
  Added () around if with assignments
  Added empty line before ; for empty while and for loops
sql/sql_show.cc:
  Added braces around && to get rid of compiler warnings
sql/sql_table.cc:
  Added braces around && to get rid of compiler warnings
sql/sql_trigger.cc:
  Added braces around && to get rid of compiler warnings
sql/sql_update.cc:
  Added braces around && to get rid of compiler warnings
sql/sql_yacc.yy:
  Added braces around && to get rid of compiler warnings
sql/table.cc:
  Added braces around && to get rid of compiler warnings
sql/table.h:
  Added braces around && to get rid of compiler warnings
sql/time.cc:
  Added braces around && to get rid of compiler warnings
sql/tztime.cc:
  Added braces around && to get rid of compiler warnings
sql/uniques.cc:
  Added braces around && to get rid of compiler warnings
storage/federated/ha_federated.cc:
  Fixed bug in testing of variable to ha_info() (Not critical)
storage/heap/ha_heap.cc:
  Added braces around && to get rid of compiler warnings
storage/maria/ha_maria.cc:
  Fixed bug: Mark that maria_log_dir_path is readonly
  Added braces around && to get rid of compiler warnings
storage/ndb/include/ndbapi/NdbEventOperation.hpp:
  Removed const before function returning simple type
storage/ndb/include/ndbapi/NdbOperation.hpp:
  Removed const before function returning simple type
storage/ndb/src/ndbapi/Ndb.cpp:
  Added empty line before ; for empty while and for loops
storage/ndb/src/ndbapi/NdbEventOperation.cpp:
  Removed const before function returning simple type
storage/ndb/src/ndbapi/NdbEventOperationImpl.cpp:
  Removed const before function returning simple type
storage/ndb/src/ndbapi/NdbEventOperationImpl.hpp:
  Removed const before function returning simple type
storage/ndb/src/ndbapi/NdbRecAttr.cpp:
  Added empty line before ; for empty while and for loops
storage/ndb/src/ndbapi/TransporterFacade.hpp:
  Added braces around && to get rid of compiler warnings

When there is an error executing EXISTS predicates they return NULL as their string
or decimal value but don't set the NULL value flag.
Fixed by returning 0 (as a decimal or a string) on error exectuting the subquery.
Note that we can't return NULL as EXISTS is not supposed to return NULL.

mysql-test/r/subselect.result:
  Bug#37627: test case
mysql-test/t/subselect.test:
  Bug#37627: test case
sql/item_subselect.cc:
  Bug#37627: return decimal (or string) 0 isntead of a NULL pointer on
  error calculating an EXISTS predicate.

Mixing aggregate functions and non-grouping columns is not allowed in the
ONLY_FULL_GROUP_BY mode. However in some cases the error wasn't thrown because
of insufficient check.

In order to check more thoroughly the new algorithm employs a list of outer
fields used in a sum function and a SELECT_LEX::full_group_by_flag.
Each non-outer field checked to find out whether it's aggregated or not and
the current select is marked accordingly.
All outer fields that are used under an aggregate function are added to the
Item_sum::outer_fields list and later checked by the Item_sum::check_sum_func
function.


mysql-test/t/group_by.test:
  Added a test case for the bug#27219: Aggregate functions in ORDER BY.
mysql-test/r/group_by.result:
  Added a test case for the bug#27219: Aggregate functions in ORDER BY.
sql/sql_select.cc:
  Bug#27219: Aggregate functions in ORDER BY.
  Implementation of new check for mixing non aggregated fields and aggregation
  function in the ONLY_FULL_GROUP_BY mode.
sql/sql_lex.cc:
  Bug#27219: Aggregate functions in ORDER BY.
  Initialization of the full_group_by_flag bitmap.
  SELECT_LEX::test_limit function doesn't reset ORDER BY
  clause anymore.
sql/sql_lex.h:
  Bug#27219: Aggregate functions in ORDER BY.
  The full_group_by_flag is added to the SELECT_LEX class.
sql/item_sum.h:
  Bug#27219: Aggregate functions in ORDER BY.
  The outer_fields list is added to the Item_sum class.
sql/mysql_priv.h:
  Bug#27219: Aggregate functions in ORDER BY.
  Defined a set of constants used in the new check for mixing non aggregated
  fields and sum functions in the ONLY_FULL_GROUP_BY_MODE.
sql/item_subselect.cc:
  Bug#27219: Aggregate functions in ORDER BY.
  The Item_in_subselect::select_in_like_transformer function now drops
  ORDER BY clause in all selects in a subquery.
sql/item_sum.cc:
  Bug#27219: Aggregate functions in ORDER BY.
  Now the Item_sum::check_sum_func function now checks whether fields in the
  outer_fields list are aggregated or not and marks selects accordingly.
sql/item.cc:
  Bug#27219: Aggregate functions in ORDER BY.
  Now the Item_field::fix_fields function checks whether the field is aggregated
  or not and marks its select_lex accordingly.

Queries like:

  SELECT ROW(1, 2) IN (SELECT t1.a, 2)
    FROM t1 GROUP BY t1.a

or 

  SELECT ROW(1, 2) IN (SELECT t1.a, 2 FROM t2)
    FROM t1 GROUP BY t1.a

lead to assertion failure in the
Item_in_subselect::row_value_transformer method in debugging
build, or to unexpected error message in release build:

  ERROR 1247 (42S22): Reference '<list ref>' not supported (forward
                      reference in item list)

Unexpected error message and assertion failure have been
eliminated.


mysql-test/r/subselect3.result:
  Added test case for bug #34763.
mysql-test/t/subselect3.test:
  Added test case for bug #34763.
sql/item.cc:
  Fixed bug #34763.
  The Item_ref::fix_fields method has been modified to silently
  ignore not fixed outer references: by the definition, those
  references should be fixed later by the call to the
  fix_inner_refs function.
sql/item_subselect.cc:
  Fixed bug #34763.
  The Item_in_subselect::row_value_transformer method has been
  modified to eliminate assertion failure on not fixed outer
  references: by the definition those references are allowed in
  this context and should be fixed later by the call to the
  fix_inner_refs function.

between 5.0 and 5.1.
  
The problem was that in the patch for Bug#11986 it was decided
to store original query in UTF8 encoding for the INFORMATION_SCHEMA.
This approach however turned out to be quite difficult to implement
properly. The main problem is to preserve the same IS-output after
dump/restore.
  
So, the fix is to rollback to the previous functionality, but also
to fix it to support multi-character-set-queries properly. The idea
is to generate INFORMATION_SCHEMA-query from the item-tree after
parsing view declaration. The IS-query should:
  - be completely in UTF8;
  - not contain character set introducers.
  
For more information, see WL4052.


mysql-test/include/ddl_i18n.check_views.inc:
  Add a test case for Bug#30217.
mysql-test/r/ddl_i18n_koi8r.result:
  Update result file.
mysql-test/r/ddl_i18n_utf8.result:
  Update result file.
mysql-test/r/information_schema.result:
  Update result file.
mysql-test/r/information_schema_db.result:
  Update result file.
mysql-test/r/mysqldump.result:
  Update result file.
mysql-test/r/show_check.result:
  Update result file.
mysql-test/t/ddl_i18n_koi8r.test:
  Add a test case for Bug#30217.
mysql-test/t/ddl_i18n_utf8.test:
  Add a test case for Bug#30217.
mysql-test/t/mysqldump.test:
  Add a test case for Bug#30217.
sql/ha_ndbcluster.cc:
  Add a parameter to print().
sql/item.cc:
  1. Add a parameter to print().
  2. Item_string::print():
        - Do not append character set introducer to the text literal
          if we're building a query for INFORMATION_SCHEMA;
        - Convert text literal to UTF8 if we're building a query
          for INFORMATION_SCHEMA.
sql/item.h:
  Add a parameter to print().
sql/item_cmpfunc.cc:
  Add a parameter to print().
sql/item_cmpfunc.h:
  Add a parameter to print().
sql/item_func.cc:
  Add a parameter to print().
sql/item_func.h:
  Add a parameter to print().
sql/item_geofunc.h:
  Add a parameter to print().
sql/item_row.cc:
  Add a parameter to print().
sql/item_row.h:
  Add a parameter to print().
sql/item_strfunc.cc:
  Add a parameter to print().
sql/item_strfunc.h:
  Add a parameter to print().
sql/item_subselect.cc:
  Add a parameter to print().
sql/item_subselect.h:
  Add a parameter to print().
sql/item_sum.cc:
  Add a parameter to print().
sql/item_sum.h:
  Add a parameter to print().
sql/item_timefunc.cc:
  Add a parameter to print().
sql/item_timefunc.h:
  Add a parameter to print().
sql/mysql_priv.h:
  Add a parameter to print().
sql/sp_head.cc:
  Add a parameter to print().
sql/sql_lex.cc:
  Add a parameter to print().
sql/sql_lex.h:
  Add a parameter to print().
sql/sql_parse.cc:
  Add a parameter to print().
sql/sql_select.cc:
  Add a parameter to print().
sql/sql_show.cc:
  Add a parameter to print().
sql/sql_test.cc:
  Add a parameter to print().
sql/sql_view.cc:
  Build INFORMATION_SCHEMA query from Item-tree.
sql/sql_yacc.yy:
  Build INFORMATION_SCHEMA query from Item-tree.
sql/table.h:
  Add a parameter to print().