variable

The User_var_log_event was not serializing the unsigned
flag. This would cause the slave to always assume signed values.

We fix this by extending the User_var_log_event to also contain
information on the unsigned_flag, meaning that it gets into the
binlog as well, therefore the slave will get this information as
well. Events without information on unsigned flag (old events)
are treated as they were before (always signed: unsigned_flag=
FALSE).

The information on the unsigned_flag, is shipped in an extra byte
appended to the end of the User_var_log_event and added by this
patch. This extra byte holds values for general purpose
User_var_log_event flags which are now packed in the binlog as
well. One of these flags contains information about whether the
value is signed or unsigned (currently this extra byte is only
used to hold data on the unsigned flag, in the future we can use
it to pack extra flags if there is the need to).

In auto-commit mode, updating both trx and non-trx tables (i.e. issuing a mixed
statement) causes the following sequence of events:

1 - "Flush trx changes" (MYSQL_BIN_LOG::write) - T1:
  1.1 - mutex_lock (&LOCK_log)
  1.2 - mutex_lock (&LOCK_prep_xids)
  1.3 - increase prepared_xids
  1.4 - mutex_unlock (&LOCK_prep_xids)
  1.5 - mutex_unlock (&LOCK_log)

2 - "Flush non-trx changes" (MYSQL_BIN_LOG::write) - T1:
  2.1 - mutex_lock (&LOCK_log)
  2.2 - mutex_unlock (&LOCK_log)

3. "unlog" - T1
  3.1 - mutex_lock (&LOCK_prep_xids)
  3.2 - decrease prepared xids
  3.3 - pthread_cond_signal(&COND_prep_xids);
  3.4 - mutex_unlock (&LOCK_prep_xids)

The "FLUSH logs" command produces the following sequence of events:

1 - "FLUSH logs" command (MYSQL_BIN_LOG::new_file_impl) - user thread:
  1.1 - mutex_lock (&LOCK_log)
  1.2 - mutex_lock (&LOCK_prep_xids)
  1.3 - while (prepared_xids)  pthread_cond_wait(..., &LOCK_prep_xids);
  1.4 - mutex_unlock (&LOCK_prep_xids)
  1.5 - mutex_unlock (&LOCK_log)

A deadlock will arise if T1 flushes the trx changes and thus increases
prepared_xids but before it is able to continue the execution and flush the
non-trx changes, an user thread calls the "FLUSH logs" command and wait that
the prepared_xids is decreased and gets to zero. However, T1 cannot proceed
with the call to "Flush non-trx changes" because it will block in the mutex
"LOCK_log" and by consequence cannot complete the execution and call the
unlog to decrease the prepared_xids.

To fix the problem, we ensure that the non-trx changes are always flushed
before the trx changes.

Note that if you call "Flush non-trx changes" and a concurrent "FLUSH logs" is
issued, the "Flush non-trx changes" may block, but a deadlock will never happen
because the prepared_xids will eventually get to zero. Bottom line, there will
not be any transaction able to increase the prepared_xids because they will
block in the mutex "LOCK_log" (MYSQL_BIN_LOG::write) and those that increased
the prepared_xids will eventually commit and decrease the prepared_xids.

      
When @@session.binlog_format is modified inside a transaction,
it can cause slave to go out of sync.
      
To fix the problem, make the session variable 'binlog_format' 
read-only inside a transaction.


mysql-test/suite/binlog/r/binlog_format_switch_inside_trans.result:
  Test result for bug#47863.
mysql-test/suite/binlog/t/binlog_format_switch_inside_trans.test:
  Added test file to verify if the session variable 'binlog_format' 
  is read-only inside a transaction and in sub-statements.
sql/set_var.cc:
  Added code to make the session variable 'binlog_format'
  read-only inside a transaction.

The slave thread changed the format of the information it used to
connect to the master after patch for BUG 13963.  This resulted
in old master getting confused, thence rejecting the slave
connection attempt.

In particular, patch for BUG 13963 removed the rpl_recovery_rank
variable which was, at that time, packed together with the rest
of the information which the slave would use to register itself
on the master. Based on this data, the master would then assert
that the number of bytes received in the connection command was
consistent to what it was expecting. 

Therefore, given that a slave, patched with the aforementioned
patch, would not pack the four bytes related to the
rpl_recovery_rank variable, the old master would reject the
connection attempt. It would assume that the data was
inconsistent (fewer bytes than it was expecting) and return 
an error.

We fix this by faking an rpl_recovery_rank variable when
registering the slave on the master. In practice this reverts a
small part of patch for BUG 13963, the one related to the slave
connecting to the master.

sql/repl_failsafe.cc:
  Added bypassing of removed rpl_recovery_rank variable information in
  packet. This should also make more sense when old servers connect
  to a new master (ie, master with patch for BUG 13963). If this was not
  done, the new master could interpert information an old slave sends as
  master_id, when in fact it could be the rpl_recovery_rank data.
sql/slave.cc:
  Faking a rpl_recovery_rank so that we can register as a slave in
  an old master.

Problem: The test was written before BUG#45827 was fixed.
The test contained code that assumed the wrong behavior,
pre-BUG#45827. Then, the fix for BUG#45827 was merged
from 5.1-rep+2 to 5.1-rep+3. Since the test case assumed
the wrong behavior, it failed. This should have been fixed
by making the test assume the correct behavior, but was
fixed by updating the result file to assert failure.
Fix 1: fix the test to assume correct behavior
(post-BUG#45827), update result file.
Fix 2: make test fail with 'die' instead of 'exit' when
wrong behavior is detected. Thus, the test cannot be
silenced with a wrong result file in case the behavior
will change again.


mysql-test/extra/rpl_tests/create_recursive_construct.inc:
  Replaced 'exit' by 'die' to avoid similar
mysql-test/suite/binlog/r/binlog_unsafe.result:
  Updated result file.
mysql-test/suite/binlog/t/binlog_unsafe.test:
  Since BUG#45827 is now fixed, we need to update the test case
  in two places where it expects the wrong result because of
  BUG#45827.

Support for flushing individual logs, so that the user can
selectively flush a subset of the server logs.

Flush of individual logs is done according to the 
following syntax:

  FLUSH <log_category> LOGS;

The syntax is extended so that the user is able to flush a
subset of logs:

  FLUSH [log_category LOGS,];

where log_category is one of:
  SLOW
  ERROR
  BINARY
  ENGINE
  GENERAL
  RELAY.


mysql-test/suite/rpl/r/rpl_flush_logs.result:
  Test result for WL#5142.
mysql-test/suite/rpl/t/rpl_flush_logs.test:
  Added the test file to verify if the 'flush individual log' 
  statement works fine.
sql/log.cc:
  Added the two functions to flush slow and general log.
sql/sql_parse.cc:
  Added code to flush specified logs against the option.
sql/sql_yacc.yy:
  Added code to parse the 'flush * log' statement syntax and 
  set its option to Lex->type.

merging from 5.1 to rep+2 starting at gca(5.1, next-mr) == build@mysql.com-20091104182209-iui387z35159aoyw

NOTE: backporting BUG#42150 into next-mr
      Includes latest Andrei's patch (see [2 Feb 18:40] Bugs System)
      and merge.test post-push fix (see [3 Feb 18:04] Bugs System)

The reason of the bug appeared to be overreacting on absense of a
binlog file although the file name had been presented in in the master
binlog index file.
By convention, there should have been only a warning printed and the rest of
`reset master' logics completed.  This did not happen on windows
due to incorrect value of my_errno returned from nt_share_delete().
      
Fixed with correcting my_errno assignment in nt_share_delete() to be ENOENT in 
he event of no binlog file. Some minor refactoring has been made.

beyond unsigned long.
BUG#44779: binlog.binlog_max_extension may be causing failure on 
next test in PB
            
NOTE1: this is the backport to next-mr.
NOTE2: already includes patch for BUG#44779.
            
Binlog file extensions would turn into negative numbers once the
variable used to hold the value reached maximum for signed
long. Consequently, incrementing value to the next (negative) number
would lead to .000000 extension, causing the server to fail.
                        
This patch addresses this issue by not allowing negative extensions
and by returning an error on find_uniq_filename, when the limit is
reached. Additionally, warnings are printed to the error log when the
limit is approaching. FLUSH LOGS will also report warnings to the
user, if the extension number has reached the limit. The limit has been
set to 0x7FFFFFFF as the maximum.

mysql-test/suite/binlog/t/binlog_max_extension.test:
  Test case added that checks the maximum available number for
  binlog extensions.
sql/log.cc:
  Changes to find_uniq_filename and test_if_number.
sql/log.h:
  Added macros with values for MAX_LOG_UNIQUE_FN_EXT and
  LOG_WARN_UNIQUE_FN_EXT_LEFT, as suggested in review.

The mentioned on the bug report set of bugs fixes have not be pushed to the main trees.

Fixed with extracting commits done to 6.0-rpl tree and applying them to the main 5.1.
Notes.
1. part of changes - the mtr's specific - were packported to the main 5.0 tree for mtr v1
   as http://lists.mysql.com/commits/46562
   However, there is no that fix anymore in the mtr v2. (This fact was mailed to mtr maintaining
   people).

2. Bug@36929  crash in kill_zombie_dump_threads-> THD::awake() with replication tests
   is not backported because the base code of the patch is libevent and that was removed
   from the main trees due to its instability.

client/mysqlbinlog.cc:
  fixes for BUG#35546
mysql-test/suite/rpl/r/rpl_bug41902.result:
  the new tests result file is added.
mysql-test/suite/rpl/t/rpl_bug41902-slave.opt:
  conf file for bug41902 testing is added.
mysql-test/suite/rpl/t/rpl_bug41902.test:
  regression tests for Bug #41902 is added.
sql/log.cc:
  collection of changes due to Bug #48463.
sql/log.h:
  collection of changes due to Bug #48463.
sql/rpl_rli.h:
  collection of changes due to Bug #48463.
sql/slave.cc:
  collection of changes due to Bug #48463.
sql/sql_repl.cc:
  collection of changes due to Bug #48463.

Post-push fix: Removed MTRv1 arguments according to the
original patch. Although there is a version check, the patch
was pushed to a 5.1 GA staging tree, while the version check 
considers version 5.2. This makes the deprecated parameters 
to be used, despite the fact that they are not valid anymore.

Part of MTRv1 is currently used in RQG semisync test, and this
was causing the test to fail on slave startup.

It should be safe to uncomment when merging up to celosia.

After fix of bug46322, logging to table is turned off, each test
that need this should turn on it in it's opt file.

Add suppression to unsafe statement warnings for binlog_unsafe.test.

mysql-test/suite/binlog/t/binlog_unsafe-master.opt:
  After fix of bug46322, logging to table is turned off, each test that need this should turn on it in it's opt file.
mysql-test/suite/binlog/t/binlog_unsafe.test:
  suppress Unsafe statement warnings

      
Replication info files are not being flushed and synced when the
command 'STOP SLAVE' is issued. This means that one cannot just
rely on existing values on those files when the slave has been
stopped. Having consistent, uncorrupted and up-to-date info files
when stopping the slave would be most useful, for instance, for
snapshotting purposes (a procedure that is often used for
restoring slaves).
      
This patch addresses this by instrumenting the
terminate_slave_threads function so that it also flushes and
syncs the *info files as well as the relay log whenever it gets
called, ie, on 'STOP SLAVE'.  Although this imposes a performance
trade-off (specifically when stopping the slave), it should have
no negative influence on overall replication performance (impact
is only noticeable on 'STOP SLAVE').

Post-push fix: Reverting change in Makefile.am which was
causing windows not to build client/.

CONFLICTS
=========

Text conflict in sql/sql_yacc.yy
1 conflicts encountered. 

      
One statement that have more than one different tables to update with 
autoinc columns just was marked as unsafe in mixed mode, so the unsafe 
warning can't be produced in statement mode.
      
To fix the problem, mark the statement as unsafe in statement mode too.


mysql-test/extra/rpl_tests/rpl_insert_id.test:
  The test case is updated due to the patch of bug#45827.
mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result:
  The test result is updated due to the patch of bug#45827.
mysql-test/suite/binlog/r/binlog_unsafe.result:
  Test result for bug#45827.
mysql-test/suite/binlog/t/binlog_unsafe.test:
  Added test to verify if stmt that have more than one
  different tables to update with autoinc columns will 
  produce unsafe warning
mysql-test/suite/rpl/r/rpl_stm_auto_increment_bug33029.result:
  The test result is updated due to the patch of bug#45827.
mysql-test/suite/rpl/t/rpl_trigger.test:
  The test case is updated due to the patch of bug#45827.
mysql-test/suite/rpl/t/rpl_variables_stm.test:
  The test case is updated due to the patch of bug#45827.
sql/sql_base.cc:
  Reomved the 'set_current_stmt_binlog_row_based_if_mixed' function 
  for producing unsafe warnings by executing 'decide_logging_format' 
  function later in statement mode

Create a set of test cases to see if some DDL statements implicitly commit
a transaction on the NDB and are written directly to the binary log without
going through either the Statement- or Transactional-Cache.

mysql-test/extra/binlog_tests/binlog_failure_mixing_engines.test:
  Merged the test case into rpl_mixing_engines.test.
mysql-test/extra/rpl_tests/rpl_mixing_engines.inc:
  Incorporated some tests from binlog_failure_mixing_engines.test and
  fixed after BUG#47323.
mysql-test/extra/rpl_tests/rpl_mixing_engines.test:
  Incorporated some tests from binlog_failure_mixing_engines.test and
  fixed after BUG#47323.
mysql-test/suite/binlog/r/binlog_mixed_failure_mixing_engines.result:
  Merged the test case into rpl_mixing_engines.test.
mysql-test/suite/binlog/r/binlog_row_failure_mixing_engines.result:
  Merged the test case into rpl_mixing_engines.test.
mysql-test/suite/binlog/t/binlog_mixed_failure_mixing_engines.test:
  Merged the test case into rpl_mixing_engines.test.
mysql-test/suite/binlog/t/binlog_row_failure_mixing_engines.test:
  Merged the test case into rpl_mixing_engines.test.
mysql-test/suite/rpl/r/rpl_mixed_mixing_engines.result:
  Incorporated some tests from binlog_failure_mixing_engines.test and
  changed after BUG#47323.
mysql-test/suite/rpl/r/rpl_row_mixing_engines.result:
  Incorporated some tests from binlog_failure_mixing_engines.test and
  changed after BUG#47323.
mysql-test/suite/rpl/r/rpl_stm_mixing_engines.result:
  Incorporated some tests from binlog_failure_mixing_engines.test and
  changed after BUG#47323.

mysql-test/extra/rpl_tests/rpl_implicit_commit_binlog.test:
  Re-enabled some parts of the test after BUG#46572
mysql-test/suite/rpl/r/rpl_mixed_implicit_commit_binlog.result:
  Re-enabled some parts of the test after BUG#46572
mysql-test/suite/rpl/r/rpl_mysql_upgrade.result:
  Suppressed warning messages due to unsafe statements.
mysql-test/suite/rpl/r/rpl_row_implicit_commit_binlog.result:
  Re-enabled some parts of the test after BUG#46572
mysql-test/suite/rpl/r/rpl_stm_implicit_commit_binlog.result:
  Re-enabled some parts of the test after BUG#46572
mysql-test/suite/rpl/t/rpl_mixed_implicit_commit_binlog.test:
  Re-enabled some parts of the test after BUG#46572
mysql-test/suite/rpl/t/rpl_mysql_upgrade.test:
  Suppressed warning messages due to unsafe statements.
mysql-test/suite/rpl/t/rpl_row_implicit_commit_binlog.test:
  Re-enabled some parts of the test after BUG#46572
mysql-test/suite/rpl/t/rpl_stm_implicit_commit_binlog.test:
  Re-enabled some parts of the test after BUG#46572

Removed test case that was left without significance after
backporting the deprecated constructs from 6.0 codebase.

merging 5.1 main -> 5.1-rep+2 -> 5.1-rep+3; binlog_unsafe , rpl_mysql_upgrade fail and are under treatment

NOTE: Backport of:

bzr log -r revid:sp1r-serg@sergbook.mysql.com-20070505200319-38337
------------------------------------------------------------
revno: 2469.263.4
committer: serg@sergbook.mysql.com
timestamp: Sat 2007-05-05 13:03:19 -0700
message:
  Removing deprecated features:
  --master-XXX command-line options
  log_bin_trust_routine_creators
  table_type
  BACKUP TABLE ...
  RESTORE TABLE ...
  SHOW PLUGIN
  LOAD TABLE ... FROM MASTER
  LOAD DATA FROM MASTER
  SHOW INNODB STATUS
  SHOW MUTEX STATUS
  SHOW TABLE TYPES
  ... TIMESTAMP(N)
  ... TYPE=engine
  
  RESET SLAVE don't reset connection parameters anymore
  LOAD DATA: check opt_secure_file_priv before access(filename)
  improved WARN_DEPRECATED macro

Non-transactional updates that take place inside a transaction present problems
for logging because they are visible to other clients before the transaction
is committed, and they are not rolled back even if the transaction is rolled
back. It is not always possible to log correctly in statement format when both
transactional and non-transactional tables are used in the same transaction.

In the current patch, we ensure that such scenario is completely safe under the
ROW and MIXED modes.

Bug#41756 "Strange error messages about locks from InnoDB".
      
In JT_EQ_REF (join_read_key()) access method, 
don't try to unlock rows in the handler, unless certain that 
a) they were locked
b) they are not used.

Unlocking of rows is done by the logic of the nested join loop,
and is unaware of the possible caching that the access method may
have. This could lead to double unlocking, when a row
was unlocked first after reading into the cache, and then 
when taken from cache, as well as to unlocking of rows which
were actually used (but taken from cache).
      
Delegate part of the unlocking logic to the access method,
and in JT_EQ_REF count how many times a record was actually 
used in the join. Unlock it only if it's usage count is 0.

Implemented review comments.

mysql-test/r/innodb_lock_wait_timeout_1.result:
  Update results (Bug41756).
mysql-test/t/innodb_lock_wait_timeout_1.test:
  Add a test case (Bug#41756).
sql/item_subselect.cc:
  Complete struct READ_RECORD initialization with a new
  member to unlock records.
sql/records.cc:
  Extend READ_RECORD API with a method to unlock read records.
sql/sql_select.cc:
  In JT_EQ_REF (join_read_key()) access method, 
  don't try to unlock rows in the handler, unless certain that 
  a) they were locked
  b) they are not used.
sql/sql_select.h:
  Add members to TABLE_REF to count TABLE_REF buffer usage count.
sql/structs.h:
  Update declarations.