SUPER is not required to change binlog format for session

A user without SUPER privileges can change the value of the
session variable BINLOG_FORMAT, causing problems for a DBA.

This changeset requires a user to have SUPER privileges to
change the value of the session variable BINLOG_FORMAT, and
not only the global variable BINLOG_FORMAT.


mysql-test/suite/binlog/t/binlog_grant.test:
  Adding test to test grants needed for SQL_LOG_BIN and BINLOG_FORMAT.
sql/set_var.cc:
  Adding code to check that user has SUPER permission
  needed to change the value of BINLOG_FORMAT.
sql/set_var.h:
  Adding function sys_var_thd_binlog_format::check()

configure.in:
  change server version number to 5.1.29

Added a rule that uses gcc to generate preprocessor
output (gcc -E) that can be compared to an already
generated output using the diff utility.

icheck has been removed and replaced by gcc -E
because icheck does not support C++.

Makefile.am:
  Added a rule for checking that the abi/api
  has not changed.
  
  The following rules are followed in the rule in makefile.am
  
  1) Generate preprocessor output for the
     files that need to be tested for abi/ 
     api changes. use -nostdinc to prevent
     generation of preprocessor output for
     system headers. This results in
     messages in stderr saying that these
     headers were not found. Redirect the
     stderr output to /dev/null to prevent
     seeing these messages.
  2) sed the output to 
     2.1) remove blank lines and lines that
          begin with "# "
     2.2) When gcc -E is run on the Mac OS 
          and solaris sparc platforms it
          introduces a line of output that
          shows up as a difference between
          the .pp and .out files. Remove
          these OS specific preprocessor
          text inserted by the preprocessor.
  3) diff the generated file and the canons
     (.pp files already in the repository).
  4) delete the .out file that is generated.
     If the diff fails, the generated file 
     is not removed. This will be useful
     for analysis of ABI differences (e.g.
     using a visual diff tool).
  
  A ABI change that causes a build to fail will always be accompanied by new canons (.out files). The .out files that are not removed will be replaced as the new .pp files.
  
  e.g. If include/mysql/plugin.h has an ABI
       change then this rule would leave a
       <build directory>/abi_check.out file.
  
  A developer with a justified API change will then do a 
  
  mv <build directory>/abi_check.out include/mysql/plugin.pp 
  
  to replace the old canons with the new ones.
configure.in:
  1) Removed the part of the file that was
     icheck related
  2) Added an entry for the configure
     variable DIFF
  3) Ensured that the abi_check rule is run
     only if gcc is available
include/Makefile.am:
  1) Removed the icheck related entries
include/mysql.h.pp:
  The pre-processor output cannon file for
  include/mysql.h
include/mysql/plugin.h.pp:
  The pre-processor output cannon file
  for include/mysql/plugin.h
include/mysql_h.ic:
  Removed the cannon file related to icheck.
sql/mysql_priv.h.pp:
  The pre-processor output cannon file for
  sql/mysql_priv.h

Problem: SELECT ... REGEXP BINARY NULL may lead to server crash/hang.

Fix: properly handle NULL regular expressions.


mysql-test/r/func_regexp.result:
  Fix for bug #39021: SELECT REGEXP BINARY NULL never returns
    - test result.
mysql-test/t/func_regexp.test:
  Fix for bug #39021: SELECT REGEXP BINARY NULL never returns
    - test case.
sql/item_cmpfunc.cc:
  Fix for bug #39021: SELECT REGEXP BINARY NULL never returns
    - checking regular expressions' null_value
  we tested it without a val_xxx() call before, which is wrong.
  Now Item_func_regex::regcomp() returns -1 in the case
  and allows to handle NULL expessions properly.
sql/item_cmpfunc.h:
  Fix for bug #39021: SELECT REGEXP BINARY NULL never returns
    - checking regular expressions' null_value
  we tested it without a val_xxx() call before, which is wrong.
  Now Item_func_regex::regcomp() returns -1 in the case
  and allows to handle NULL expessions properly.

Incremental fixes: updating a comment and fixing a result file.

sql/sql_class.h:
  Changing comment.

The assertion indicates that some data was left in the transaction
cache when the server was shut down, which means that a previous
statement did not commit or rollback correctly.

What happened was that a bug in the rollback of a transactional
table caused the transaction cache to be emptied, but not reset.
The error can be triggered by having a failing UPDATE or INSERT,
on a transactional table, causing an implicit rollback.

Fixed by always flushing the pending event to reset the state
properly.


mysql-test/extra/rpl_tests/rpl_row_basic.test:
  Testing that a failed update (that writes some rows to the
  transaction cache) does not cause the transaction cache to
  hold on to the data or forget to reset the transaction cache.
sql/log.cc:
  Added call to remove pending event when the transaction cache
  is emptied instead of written to binary log. The call will also
  clear the outstanding table map count so that the cache is not
  left it in a state of "empty but not reset".
  
  Added function MYSQL_BIN_LOG::remove_pending_rows_event().
sql/log.h:
  Added function MYSQL_BIN_LOG::remove_pending_rows_event().
sql/sql_class.cc:
  Adding function THD::binlog_remove_pending_rows_event().
sql/sql_class.h:
  Adding function THD::binlog_remove_pending_rows_event().

This patch also fixes bugs 36963 and 35600.
                      
- In many places a view was confused with an anonymous derived
  table, i.e. access checking was skipped. Fixed by introducing a
  predicate to tell the difference between named and anonymous
  derived tables.
                      
- When inserting fields for "SELECT * ", there was no 
  distinction between base tables and views, where one should be
  made. View privileges are checked elsewhere.

mysql-test/include/grant_cache.inc:
  Bug#36086: Changed test case.
mysql-test/r/grant2.result:
  Bug#36086: Changed test result.
mysql-test/r/grant_cache_no_prot.result:
  Bug#36086: Changed test result.
mysql-test/r/grant_cache_ps_prot.result:
  Bug#36086: Changed test result.
mysql-test/r/view_grant.result:
  Bug#36086: Test result.
mysql-test/t/grant2.test:
  Bug#36086: Changed test case.
mysql-test/t/view_grant.test:
  Bug#36086: Test case.
sql/item.cc:
  Bug#36086: Replaced conditional with new methods.
sql/sql_acl.cc:
  Bug no 35600: 
  In mysql_table_grant:
    Replaced conditional with the new accessor method.
  
  In check_grant:
   - Changed the requirement table->derived != null to 
     checking all anonymous derived tables.
   - Use of the accessor methods for getting object and database 
     names.
      
  Bug#36086: In check_grant_all_columns:
    - Updated comment. This function is now called for views
      as well.
    - The error message should not disclose any column names 
      unless the user has privilege to see all column names.
    - Changed names of Field_iterator_table_ref methods.
sql/sql_base.cc:
  Bug no 36963: In insert_fields()
    - Commented.
    - We should call check_grant_all_columns() for views in  
      this case.        
    - Changed names of Field_iterator_table_ref methods.
    - We should not disclose column names in the error message
      when the user has no approprate privilege.
sql/sql_cache.cc:
  Bug#36086: Replaced test with new predicate method.
sql/sql_derived.cc:
  Bug#36086: commenting only. Updated and doxygenated
  comment for mysql_derived_prepare().
sql/sql_parse.cc:
  Bug no 35600: 
  - In check_single_table_access:
    Due to the bug, check_grant would raise an error for a
    SHOW CREATE TABLE command for a TEMPTABLE view. It should in
    fact not be be invoked in this case. This table privilege
    is checked already.
    There is a test case for this in information_schema_db.test.
      
  - In check_access: replaced table->derived
sql/table.cc:
  Bug#36086: 
  
  - In TABLE_LIST::set_underlying_merge(): 
    Commenting only. Doxygenated, corrected spelling,
    added.
  
  - Renamed table_name() and db_name() methods of 
    Field_iterator_table_ref in order to be consistent
    with new methods in TABLE_LIST.
sql/table.h:
  Bug#36086: 
    - Commented GRANT_INFO.
    - Added a predicate is_anonymous_derived_table() to    
      TABLE_LIST.
    - Added get_table_name() and get_db_name() to   
      TABLE_LIST in order to hide the disparate   
      representation of these properties.

in open_table()

Problem: repeating "CREATE... ( AUTOINCREMENT) ... SELECT" may lead to
an assertion failure.

Fix: reset table->auto_increment_field_not_null after each record 
writing.


mysql-test/r/create.result:
  Fix for bug#38821: Assert table->auto_increment_field_not_null failed 
  in open_table()
    - test result.
mysql-test/t/create.test:
  Fix for bug#38821: Assert table->auto_increment_field_not_null failed 
  in open_table()
    - test case.
sql/sql_insert.cc:
  Fix for bug#38821: Assert table->auto_increment_field_not_null failed 
  in open_table()
    - reset table->auto_increment_field_not_null after writing a record
  for "{CREATE, INSERT}..SELECT".

      
The replication filtering rules were inappropiately applied when
executing BINLOG pseudo-query.  The rules are supposed to be active
only at times when the slave's sql thread executes an event.
            
Fixed with correcting a condition to call replication rules only if
the slave sql thread executes the event.

mysql-test/suite/rpl/r/rpl_binlog_query_filter_rules.result:
  new result file
mysql-test/suite/rpl/t/rpl_binlog_query_filter_rules-master.opt:
  a filtering option that would refuse to replicate a row event of the main test on
  slave
mysql-test/suite/rpl/t/rpl_binlog_query_filter_rules.test:
  a regression test for the bug
sql/log_event.cc:
  avoiding to call the filtering rules if the execution thread is not a slave.

  INSERT .. SELECT .. ON DUPLICATE KEY UPDATE col=DEFAULT

In order to get correct values from update fields that
belongs to the SELECT part in the INSERT .. SELECT .. ON
DUPLICATE KEY UPDATE statement, the server adds referenced
fields to the select list. Part of the code that does this
transformation is shared between implementations of
the DEFAULT(col) function and the DEFAULT keyword (in
the col=DEFAULT expression), and an implementation of
the DEFAULT keyword is incomplete.


mysql-test/r/default.result:
  Added test case for bug #39002.
mysql-test/t/default.test:
  Added test case for bug #39002.
sql/item.cc:
  The Item_default_value::transform() function has been
  modified to take into account the fact that the DEFAULT
  keyword has no arguments unlike the DEFAULT(col) function
  that always has an argument.

            returns unexpected result

If:
  1. a table has a not nullable BIT column c1 with a length
     shorter than 8 bits and some additional not nullable
     columns c2 etc, and
  2. the WHERE clause is like: (c1 = constant) AND c2 ...,
the SELECT query returns unexpected result set.


The server stores BIT columns in a tricky way to save disk
space: if column's bit length is not divisible by 8, the
server places reminder bits among the null bits at the start
of a record. The rest bytes are stored in the record itself,
and Field::ptr points to these rest bytes.

However if a bit length of the whole column is less than 8,
there are no remaining bytes, and there is nothing to store in
the record at its regular place. In this case Field::ptr points
to bytes actually occupied by the next column in a record.
If both columns (BIT and the next column) are NOT NULL,
the Field::eq function incorrectly deduces that this is the
same column, so query transformation/equal item elimination
code (see build_equal_items_for_cond) may mix these columns
and damage conditions containing references to them.


mysql-test/r/type_bit.result:
  Added test case for bug #37799.
mysql-test/t/type_bit.test:
  Added test case for bug #37799.
sql/field.h:
  1. The Field::eq function has been modified to take types of
  comparing columns into account to distinguish between BIT and
  not BIT columns referencing the same bytes in a record.
  
  2. Unnecessary type comparison has been removed from the
  Field_bit::eq function (moved to Field::eq).

      
When switching to indexed ORDER BY we must be sure to reset the index read
flag if we are switching from a covering index to non-covering.

mysql-test/r/subselect.result:
  Bug#37548: test case
mysql-test/t/subselect.test:
  Bug#37548: test case
sql/sql_select.cc:
  Bug#37548: update the index read flag if the index for indexed ORDER BY is not
      covering.

used causes server crash.
      
When the loose index scan access method is used values of aggregated functions
are precomputed by it. Aggregation of such functions shouldn't be performed
in this case and functions should be treated as normal ones.
The create_tmp_table function wasn't taking this into account and this led to
a crash if a query has MIN/MAX aggregate functions and employs temporary table
and loose index scan.
Now the JOIN::exec and the create_tmp_table functions treat MIN/MAX aggregate
functions as normal ones when the loose index scan is used.


mysql-test/r/group_min_max.result:
  Added a test case for the bug#38195.
mysql-test/t/group_min_max.test:
  Added a test case for the bug#38195.
sql/sql_select.cc:
  Bug#38195: Incorrect handling of aggregate functions when loose index scan is
  used causes server crash.
  The JOIN::exec and the create_tmp_table functions treat MIN/MAX aggregate
  functions as normal ones when the loose index scan is used.

            tables open

When executing a DROP DATABASE statement in ROW mode and having temporary
tables open at the same time, the existance of temporary tables prevent
the server from switching back to row mode after temporarily switching to
statement mode to handle the logging of the statement.

Fixed the problem by removing the code to switch to statement mode and added
code to temporarily disable the binary log while dropping the objects in the
database.


mysql-test/extra/binlog_tests/database.test:
  Added test to ensure that DROP DATABASE does not affect the replication mode.
sql/sql_db.cc:
  Removed code that clears the current_stmt_binlog_row_based flag.
  Added code to disable the binary log while dropping the objects
  in a database.

post push fix

Updated partition_symlink since different error behavior
if embedded (or not partitioned)

mysql-test/r/partition_symlink.result:
  Bug#32167 another privilege bypass with DATA/INDEX DIRECTORY
  
  updated result file
mysql-test/t/partition_symlink.test:
  Bug#32167 another privilege bypass with DATA/INDEX DIRECTORY
  
  Fix for embedded