It is well-known that due to concurrency issues, a slave can become
inconsistent when a transaction contains updates to both transaction and
non-transactional tables.
                    
In a nutshell, the current code-base tries to preserve causality among the
statements by writing non-transactional statements to the txn-cache which
is flushed upon commit. However, modifications done to non-transactional
tables on behalf of a transaction become immediately visible to other
connections but may not immediately get into the binary log and therefore
consistency may be broken.
            
In general, it is impossible to automatically detect causality/dependency
among statements by just analyzing the statements sent to the server. This
happen because dependency may be hidden in the application code and it is
necessary to know a priori all the statements processed in the context of
a transaction such as in a procedure. Moreover, even for the few cases that
we could automatically address in the server, the computation effort
required could make the approach infeasible.
            
So, in this patch we introduce the option
      - "--binlog-direct-non-transactional-updates" that can be used to bypass
      the current behavior in order to write directly to binary log statements
      that change non-transactional tables.

Besides, it is used to enable the WL#2687 which is disabled by default.

mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test:
  Changes the result set as the STMT mode behaves as both the ROW and MIXED modes and as such uses the non-trx-cache and takes longer to fill the trx-cache up and trigger an error.
mysql-test/extra/rpl_tests/rpl_implicit_commit_binlog.test:
  Changes the result set as the STMT mode behaves as both the ROW and MIXED modes and as such uses the non-trx-cache. It also fixes comments.
mysql-test/extra/rpl_tests/rpl_mixing_engines.test:
  The STMT mode is unsafe when mixed-statements are executed thus making slaves to go out of sync. For that reason, it checks consistency if not in STMT mode.
mysql-test/include/default_mysqld.cnf:
  Makes binlog-direct-non-transactional-updates "TRUE" by default in the test
  cases.
mysql-test/r/mysqld--help-notwin.result:
  Updates the result file with the new option.
mysql-test/r/mysqld--help-win.result:
  Updates the result file with the new option.
mysql-test/suite/binlog/r/binlog_multi_engine.result:
  Updates the result file because non-trx-changes are written ahead of the
  transaction.
mysql-test/suite/binlog/r/binlog_switch_inside_trans.result:
  Verifies if the user cannot change the opion binlog_direct_non_transactional_updates
  within a transaction or a procedure/function/trigger.
mysql-test/suite/binlog/t/binlog_stm_mix_innodb_myisam-master.opt:
  Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing
  the test case and its result file.
mysql-test/suite/binlog/t/binlog_switch_inside_trans.test:
  Verifies if the user cannot change the opion binlog_direct_non_transactional_updates
  within a transaction or a procedure/function/trigger.
mysql-test/suite/ndb/r/ndb_binlog_format.result:
  Updates the result file because non-trx-changes are written ahead of the
  transaction.
mysql-test/suite/rpl/r/rpl_begin_commit_rollback.result:
  Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing the test case and its result file.
mysql-test/suite/rpl/r/rpl_concurrency_error.result:
  Updates the result file because non-trx-changes are written ahead of the
  transaction
mysql-test/suite/rpl/r/rpl_mixed_implicit_commit_binlog.result:
  Updates the result file because non-trx-changes are written ahead of the
  transaction.
mysql-test/suite/rpl/r/rpl_mixed_mixing_engines.result:
  Updates the result file because non-trx-changes are written ahead of the
  transaction.
mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result:
  Changes the result set as the STMT mode behaves as both the ROW and MIXED modes and as such uses the non-trx-cache and takes longer to fill the trx-cache up and trigger an error.
mysql-test/suite/rpl/r/rpl_stm_implicit_commit_binlog.result:
  Updates the result file because non-trx-changes are written ahead of the
  transaction.
mysql-test/suite/rpl/r/rpl_stm_mixing_engines.result:
  Updates the result file because non-trx-changes are written ahead of the
  transaction.
mysql-test/suite/rpl/r/rpl_stm_start_stop_slave.result:
  Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing the test case and its result file.
mysql-test/suite/rpl/r/rpl_stm_stop_middle_group.result:
  Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing the test case and its result file.
mysql-test/suite/rpl/t/rpl_begin_commit_rollback.test:
  Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing the test case and its result file.
mysql-test/suite/rpl/t/rpl_stm_start_stop_slave.test:
  Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing the test case and its result file.
mysql-test/suite/rpl/t/rpl_stm_stop_middle_group.test:
  Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing the test case and its result file.
sql/log.cc:
  Verifies if changes should be written to either the trx-cache or non-trx-cache through the use of the function use_trans_cache(). It also organizes the code.
sql/log.h:
  Changes the signature of some functions by adding the modifier "const" to the thd parameter. Specifically, the following functions are changed: 
          bool trans_has_updated_trans_table(const THD* thd);
          bool stmt_has_updated_trans_table(const THD *thd);
          bool use_trans_cache(const THD*, bool is_transactional);
sql/share/errmsg-utf8.txt:
  Creates error messages to report when an user tries to change the new option
  binlog_direct_non_transactional_updates within a transaction or a procedure/
  function/trigger.
sql/share/errmsg.txt:
  Creates error messages to report when an user tries to change the new option
  binlog_direct_non_transactional_updates within a transaction or a procedure/
  function/trigger.
sql/sql_class.h:
  Adds the new option binlog_direct_non_transactional_updates.
sql/sys_vars.cc:
  Adds the new option binlog_direct_non_transactional_updates.
support-files/my-small.cnf.sh:
  Adds binlog-direct-non-transactional-updates to the example file. By default
  the option is disabled.

      
It is well-known that due to concurrency issues, a slave can become
inconsistent when a transaction contains updates to both transaction and
non-transactional tables.
                    
In a nutshell, the current code-base tries to preserve causality among the
statements by writing non-transactional statements to the txn-cache which
is flushed upon commit. However, modifications done to non-transactional
tables on behalf of a transaction become immediately visible to other
connections but may not immediately get into the binary log and therefore
consistency may be broken.
            
In general, it is impossible to automatically detect causality/dependency
among statements by just analyzing the statements sent to the server. This
happen because dependency may be hidden in the application code and it is
necessary to know a priori all the statements processed in the context of
a transaction such as in a procedure. Moreover, even for the few cases that
we could automatically address in the server, the computation effort
required could make the approach infeasible.
            
So, in this patch we introduce the option
      - "--binlog-direct-non-transactional-updates" that can be used to bypass
      the current behavior in order to write directly to binary log statements
      that change non-transactional tables.

Besides, it is used to enable the WL#2687 which is disabled by default.

mysql-test/extra/rpl_tests/rpl_binlog_max_cache_size.test:
  Changes the result set as the STMT mode behaves as both the ROW and MIXED modes and as such uses the non-trx-cache and takes longer to fill the trx-cache up and trigger an error.
mysql-test/extra/rpl_tests/rpl_implicit_commit_binlog.test:
  Changes the result set as the STMT mode behaves as both the ROW and MIXED modes and as such uses the non-trx-cache. It also fixes comments.
mysql-test/extra/rpl_tests/rpl_mixing_engines.test:
  The STMT mode is unsafe when mixed-statements are executed thus making slaves to go out of sync. For that reason, it checks consistency if not in STMT mode.
mysql-test/include/default_mysqld.cnf:
  Makes binlog-direct-non-transactional-updates "TRUE" by default in the test
  cases.
mysql-test/r/mysqld--help-notwin.result:
  Updates the result file with the new option.
mysql-test/r/mysqld--help-win.result:
  Updates the result file with the new option.
mysql-test/suite/binlog/r/binlog_multi_engine.result:
  Updates the result file because non-trx-changes are written ahead of the
  transaction.
mysql-test/suite/binlog/r/binlog_switch_inside_trans.result:
  Verifies if the user cannot change the opion binlog_direct_non_transactional_updates
  within a transaction or a procedure/function/trigger.
mysql-test/suite/binlog/t/binlog_stm_mix_innodb_myisam-master.opt:
  Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing
  the test case and its result file.
mysql-test/suite/binlog/t/binlog_switch_inside_trans.test:
  Verifies if the user cannot change the opion binlog_direct_non_transactional_updates
  within a transaction or a procedure/function/trigger.
mysql-test/suite/ndb/r/ndb_binlog_format.result:
  Updates the result file because non-trx-changes are written ahead of the
  transaction.
mysql-test/suite/rpl/r/rpl_begin_commit_rollback.result:
  Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing the test case and its result file.
mysql-test/suite/rpl/r/rpl_concurrency_error.result:
  Updates the result file because non-trx-changes are written ahead of the
  transaction
mysql-test/suite/rpl/r/rpl_mixed_implicit_commit_binlog.result:
  Updates the result file because non-trx-changes are written ahead of the
  transaction.
mysql-test/suite/rpl/r/rpl_mixed_mixing_engines.result:
  Updates the result file because non-trx-changes are written ahead of the
  transaction.
mysql-test/suite/rpl/r/rpl_stm_binlog_max_cache_size.result:
  Changes the result set as the STMT mode behaves as both the ROW and MIXED modes and as such uses the non-trx-cache and takes longer to fill the trx-cache up and trigger an error.
mysql-test/suite/rpl/r/rpl_stm_implicit_commit_binlog.result:
  Updates the result file because non-trx-changes are written ahead of the
  transaction.
mysql-test/suite/rpl/r/rpl_stm_mixing_engines.result:
  Updates the result file because non-trx-changes are written ahead of the
  transaction.
mysql-test/suite/rpl/r/rpl_stm_start_stop_slave.result:
  Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing the test case and its result file.
mysql-test/suite/rpl/r/rpl_stm_stop_middle_group.result:
  Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing the test case and its result file.
mysql-test/suite/rpl/t/rpl_begin_commit_rollback.test:
  Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing the test case and its result file.
mysql-test/suite/rpl/t/rpl_stm_start_stop_slave.test:
  Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing the test case and its result file.
mysql-test/suite/rpl/t/rpl_stm_stop_middle_group.test:
  Sets binlog_direct_non_transactional_updates to FALSE in order to avoid changing the test case and its result file.
sql/log.cc:
  Verifies if changes should be written to either the trx-cache or non-trx-cache through the use of the function use_trans_cache(). It also organizes the code.
sql/log.h:
  Changes the signature of some functions by adding the modifier "const" to the thd parameter. Specifically, the following functions are changed: 
          bool trans_has_updated_trans_table(const THD* thd);
          bool stmt_has_updated_trans_table(const THD *thd);
          bool use_trans_cache(const THD*, bool is_transactional);
sql/share/errmsg-utf8.txt:
  Creates error messages to report when an user tries to change the new option
  binlog_direct_non_transactional_updates within a transaction or a procedure/
  function/trigger.
sql/share/errmsg.txt:
  Creates error messages to report when an user tries to change the new option
  binlog_direct_non_transactional_updates within a transaction or a procedure/
  function/trigger.
sql/sql_class.h:
  Adds the new option binlog_direct_non_transactional_updates.
sql/sys_vars.cc:
  Adds the new option binlog_direct_non_transactional_updates.
support-files/my-small.cnf.sh:
  Adds binlog-direct-non-transactional-updates to the example file. By default
  the option is disabled.

in errmsg.txt. Kept the one in that matched the location
in errmsg-utf8.txt, ie after ER_UNKNOWN_LOCALE.

The reason for the crash is using uinitialized mutex attribute (MY_MUTEX_FAST_INIT)
in pthread_mutex_init.

The fix is to initialize the attribute before the first use.

for regular PB builds.

        
Problem: The test case failed because: (i) warning text in 
         result file differed from the warning output by the 
         server, and (ii) binlog contents in result file did
         not show the statements logged wrapped in BEGIN/COMMIT
         as it is the case after WL 2687.
      
Solution: We update the result file, but first we change the
          unsafe warning text to also refer to performance_schema
          table(s). This required changing the result files for
          existing test cases that provide output for warnings 
          related to ER_BINLOG_UNSAFE_SYSTEM_TABLE. "Grepping" in
          result files, shows that only binlog_unsafe contained
          reference to such a warning.
      
          We also update the result file with the missing 
          BEGIN/COMMIT statements.

Conflicts:
  Text conflict in sql/sql_insert.cc 

STMT mode aborts the SQL Thread if it receives changes in ROW mode and such
changes are not processed.

When starting mysqld it did not recognize most of the options given on
the command line when it was compiled for 32-bit Solaris using Sun
Studio compiler. The cause for this was that most of the entries in
the my_long_options array contained "garbage" data. The garbage data
was caused by a compiler bug. When initilizing the def_value member
for the "default-storage-engine" entry it was initialized like this:

  (longlong)"MyISAM"

i.e. casting a 32 bit pointer to a 64 bit integer value. Due to the
compiler bug only 4 bytes was allocated (instead of 8 bytes). This
caused everything following this entry to be stored at a location that
was 4 byte wrong.

The fix/work-around for this problem is initialize the def_value
for default-storage-engine in my_long_options to 0 and instead
initialize the default_storage_engine variable to "MyISAM" in
init_common_variables().

sql/mysqld.cc:
  Due to a bug in Sun Studio compiler when generating 32 bit code the 
  initialization of the def_value member of the default-storage-engine entry 
  in my_long_options only got 4 bytes allocated instead of 8 bytes. 
  The compiler bug was triggered by casting a 32 bit pointer to a 64 bit 
  integer value in the initialization code for my_long_options. To avoid 
  triggering the compiler bug the intialization of the def_value in
  my_long_options is set to 0 and instead the default_storage_engine
  is initialized to "MyISAM" in init_common_variables().

  modified:
  include/m_ctype.h
  - Changing type for tolower/toupper members, to store values >= 0xFFFF.
  - Adding function prototypes

  mysql-test/r/ctype_big5.result
  mysql-test/r/ctype_cp932_binlog_stm.result
  mysql-test/r/ctype_eucjpms.result*
  mysql-test/r/ctype_euckr.result
  mysql-test/r/ctype_gb2312.result
  mysql-test/r/ctype_gbk.result
  mysql-test/r/ctype_sjis.result
  mysql-test/r/ctype_ujis.result
  mysql-test/t/ctype_big5.test
  mysql-test/t/ctype_cp932_binlog_stm.test
  mysql-test/t/ctype_eucjpms.test
  mysql-test/t/ctype_euckr.test
  mysql-test/t/ctype_gb2312.test
  mysql-test/t/ctype_gbk.test
  mysql-test/t/ctype_sjis.test
  mysql-test/t/ctype_ujis.test
  -  Adding tests

  strings/ctype-big5.c
  strings/ctype-cp932.c
  strings/ctype-euc_kr.c
  strings/ctype-eucjpms.c
  strings/ctype-gb2312.c
  strings/ctype-gbk.c
  strings/ctype-sjis.c
  - Adding upper/lower case conversion data

  strings/ctype-mb.c
  - Adding handling of upper/lower conversion for multi-byte characters.

  strings/ctype-ujis.c
  - Implementing shared upper/lower conversion
    functions  for ujis and eucjpms
  - Adding upper/lower case conversion data for ujis

Conflicts:
  Text conflict in sql/log.cc
  Text conflict in sql/slave.cc
  Text conflict in sql/sql_base.cc

         replicating

Replace c_ptr() calls with c_ptr_safe() calls to
avoid valgrind warnings.

Adding code to to handle the case that no metadata
was present in the table map for the column.

Allow first parameter to unpack_row() to be NULL,
in which case no source tables is used and hence
no checks nor conversions are done.

Clarifying some comments and fixing documentation
for unpack_row().

run in PB2 as it ought to be. Otherwise test will fail because
variable is no recognized:

1193: Unknown system variable 'slave_type_conversions'

  - mysqld--help-win
    Updated result so that it contains missing
    value for slave-type-conversions

  - rpl_idempotency
    This seems a bad merge. In BUG#39934, the contents of
    this file had been split into rpl_row_idempontency and
    rpl_idempotency. The patch was pushed to 5.1-rep+3 which
    was later merged in rep+2-delivery1 which in turn was
    merged in 5.1-rpl-merge. Now while merging next-mr in
    5.1-rpl-merge, the file got back it's old content (which
    is in rpl_row_idempotency now because of BUG#39934). This
    cset reverts the bad merge:

    bzr merge -r revid:dao-gang.qu@sun.com-20100112120709-ioxp11yl9bvquaqd..\
    before:revid:dao-gang.qu@sun.com-20100112120709-ioxp11yl9bvquaqd\
    suite/rpl/t/rpl_idempotency.test

  - sys_vars.rpl_init_slave_func
    Added suppression for the unsafe warning.

 - sys_vars.all_vars:
   Added test case for slave_type_conversions variable
 - rpl_row_idempotency
   Removed ER_SLAVE_AMBIGOUS_EXEC_MODE (which was removed by WL 4738)
   from the test case. Using ER_WRONG_VALUE_FOR_VAR instead.
 - mysqld--help-win
   Added missing help for --slave-type-conversions from the
   result file.