when replicating

Post-merge fixes. Updating result files.

         when replicating

The function create_virtual_tmp_table does not
set db_low_byte_first in the same way as
create_tmp_table does, causing copying from
the virtual table to a real table to get strange
values for SET types on big-endian machines.


mysql-test/extra/rpl_tests/type_conversions.test:
  Adding tests for converting between different-sized sets.
sql/sql_select.cc:
  Settng dh_low_byte_first for tables created
  with create_virtual_tmp_table.

merging from 5.1 to rep+2 starting at gca(5.1, next-mr) == build@mysql.com-20091208092611-pbno5awyb0v38hs7

Fixed conflicts in:

- binlog.binlog_unsafe
- rpl.rpl_slow_query_log

Post-merge fixes to update result files.

         replicating

Fixes to make mysqlbinlog work correctly and updates
to rpl_ndb result files.

sql/log_event.cc:
  The processing of the MYSQL_STRING_TYPE is now done
  in table_def::type(), so the support functions for
  mysqlbinlog are changed to work correctly.

Fixes to get it to compile on MacOSX.

Fixing error codes that changed after merge.

           for InnoDB

The class Field_bit_as_char stores the metadata for the
field incorrecly because bytes_in_rec and bit_len are set
to (field_length + 7 ) / 8 and 0 respectively, while
Field_bit has the correct values field_length / 8 and
field_length % 8.

Solved the problem by re-computing the values for the
metadata based on the field_length instead of using the
bytes_in_rec and bit_len variables.

To handle compatibility with old server, a table map
flag was added to indicate that the bit computation is
exact. If the flag is clear, the slave computes the
number of bytes required to store the bit field and
compares that instead, effectively allowing replication
*without conversion* from any field length that require
the same number of bytes to store.

mysql-test/suite/rpl/t/rpl_typeconv_innodb.test:
  Adding test to check compatibility for bit field
  replication when using InnoDB.
sql/field.cc:
  Extending compatible_field_size() with flags from
  table map to allow fields to check master info.
sql/field.h:
  Extending compatible_field_size() with flags from
  table map to allow fields to check master info.
sql/log.cc:
  Removing table map flags since they are not used
  outside table map class.
sql/log_event.cc:
  Removing flags parameter from table map constructor
  since it is not used and does not have to be exposed.
sql/log_event.h:
  Adding flag to denote that bit length for bit field type
  is exact and not potentially rounded to even bytes.
sql/rpl_utility.cc:
  Adding fields to table_def to store table map flags.
sql/rpl_utility.h:
  Removing obsolete comment and adding flags to store
  table map flags from master.

Fixing minor error when printing SQL types from master and cleaning some code.

Updating result files.

Row-based replication requires the types of columns on the
master and slave to be approximately the same (some safe
conversions between strings are allowed), but does not
allow safe conversions between fields of similar types such
as TINYINT and INT.

This patch implement type conversions between similar fields
on the master and slave.

The conversions are controlled using a new variable
SLAVE_TYPE_CONVERSIONS of type SET('ALL_LOSSY','ALL_NON_LOSSY').

Non-lossy conversions are any conversions that do not run the
risk of losing any information, while lossy conversions can
potentially truncate the value. The column definitions are
checked to decide if the conversion is acceptable.

If neither conversion is enabled, it is required that the
definitions of the columns are identical on master and slave.

Conversion is done by creating an internal conversion table,
unpacking the master data into it, and then copy the data to
the real table on the slave.

.bzrignore:
  New files added
client/Makefile.am:
  New files added
client/mysqlbinlog.cc:
  Functions in rpl_utility.cc is now needed by mysqlbinlog.cc.
libmysqld/Makefile.am:
  New files added
mysql-test/extra/rpl_tests/check_type.inc:
  Test include file to check a single type conversion.
mysql-test/extra/rpl_tests/rpl_extraSlave_Col.test:
  Switching to use INT instead of TEXT for column that should not have matching types.
mysql-test/extra/rpl_tests/rpl_row_basic.test:
  Adding code to enable type conversions for BIT tests since InnoDB
  cannot handle them properly due to incorrect information stored as
  metadata.
mysql-test/extra/rpl_tests/type_conversions.test:
  Test file to check a set of type conversions
  with current settings of slave_type_conversions.
mysql-test/suite/rpl/t/rpl_typeconv.test:
  Test file to test conversions from master to slave with
  all possible values for slave_type_conversions.
  
  The test also checks that the slave_type_conversions
  variable works as expected.
sql/field.cc:
  Changing definition of compatible_field_size to both check if 
  two field with identical base types are compatible and give an
  order between them if they are compatible.
  
  This only implement checking on the slave, so it will not affect
  replication from an old master to a new slave.
sql/field.h:
  Changing prototypes for functions:
  - compatible_field_size()
  - init_for_tmp_table()
  - row_pack_length()
sql/log_event.cc:
  Changing compability checks to build a conversion table if the fields
  are compatible, but does not have the same base type.
sql/log_event_old.cc:
  Changing compability checks to build a conversion table if the fields
  are compatible, but does not have the same base type.
sql/mysql_priv.h:
  Adding global option variable for SLAVE_TYPE_CONVERSIONS
sql/mysqld.cc:
  Adding SLAVE_TYPE_CONVERSIONS global server variable.
sql/rpl_record.cc:
  Changing unpack_row to use the conversion table if present.
sql/rpl_rli.h:
  Removing function get_tabledef and replacing it with get_table_data().
  This function retrieve data for table opened for replication, not just
  table definition.
sql/rpl_utility.cc:
  Function table_def::compatible_with is changed to compare table on master
  and slave for compatibility and generate a conversions table if they are
  compatible.
  
  Computing real type of fields from metadata for ENUM and SET types.
  Computing pack_length correctly for ENUM, SET, and BLOB types.
  
  Adding optimization to not check compatibility if no
  slave type conversions are enabled.
sql/rpl_utility.h:
  Changing prototypes since implementation has changed.
  
  Modifying table_def::type() to return real type instead of stored type.
sql/set_var.cc:
  Adding SLAVE_TYPE_CONVERSIONS variable.
sql/set_var.h:
  Adding SLAVE_TYPE_CONVERSIONS variable.
sql/share/errmsg.txt:
  Adding error messages for slave type conversions.
sql/sql_class.h:
  Adding SLAVE_TYPE_CONVERSIONS variable.
sql/sql_select.cc:
  Correcting create_virtual_tmp_table() to compute null bit positions
  correctly in the presence of bit fields.

Support for flushing individual logs, so that the user can
selectively flush a subset of the server logs.

Flush of individual logs is done according to the 
following syntax:

  FLUSH <log_category> LOGS;

The syntax is extended so that the user is able to flush a
subset of logs:

  FLUSH [log_category LOGS,];

where log_category is one of:
  SLOW
  ERROR
  BINARY
  ENGINE
  GENERAL
  RELAY.


mysql-test/suite/rpl/r/rpl_flush_logs.result:
  Test result for WL#5142.
mysql-test/suite/rpl/t/rpl_flush_logs.test:
  Added the test file to verify if the 'flush individual log' 
  statement works fine.
sql/log.cc:
  Added the two functions to flush slow and general log.
sql/sql_parse.cc:
  Added code to flush specified logs against the option.
sql/sql_yacc.yy:
  Added code to parse the 'flush * log' statement syntax and 
  set its option to Lex->type.

      
Add the test to disabled list and remove from the experimental list.

the last IF ELSE part which decides the plugin name is not relevant as we still have
to substitute the occurences of INNOBASE with INNODB_PLUGIN.
Remove the last IF ELSE part in CMakeLists.txt as it doesn't make sense in 5.1.

manual merge 5.0-->5.1, updating InnoDB plugin.

The bug 38816 changed the lock that protects THD::query from
LOCK_thread_count to LOCK_thd_data, but didn't update the associated
InnoDB functions.

1. The innobase_mysql_prepare_print_arbitrary_thd and the
innobase_mysql_end_print_arbitrary_thd InnoDB functions have been
removed, since now we have a per-thread mutex: now we don't need to wrap
several inter-thread access tries to THD::query with a single global
LOCK_thread_count lock, so we can simplify the code.

2. The innobase_mysql_print_thd function has been modified to lock
LOCK_thd_data in direct way.

and rpl.rpl_get_master_version_and_clock.

applied revisions: r6201, r6202, r6207, r6208, r6210

r6202 - port fix for BUG#48469 to plugin

Detailed revision comments:

r6201 | marko | 2009-11-19 14:09:11 +0200 (Thu, 19 Nov 2009) | 2 lines
branches/zip: ha_innobase::add_index(): Clarify the comment
on orphaned tables when creating a primary key.
r6202 | jyang | 2009-11-19 15:01:00 +0200 (Thu, 19 Nov 2009) | 8 lines
branches/zip: Function fseg_free() is no longer defined
in branches/zip. To port fix for bug #48469 to zip,
we can use btr_free_root() which frees the page,
and also does not require mini-transaction.

Approved by Marko.


r6207 | vasil | 2009-11-20 10:19:14 +0200 (Fri, 20 Nov 2009) | 54 lines
branches/zip: Merge r6198:6206 from branches/5.1:

(r6203 was skipped as it is already in branches/zip)

  ------------------------------------------------------------------------
  r6200 | vasil | 2009-11-19 12:14:23 +0200 (Thu, 19 Nov 2009) | 4 lines
  Changed paths:
     M /branches/5.1/btr/btr0btr.c
  
  branches/5.1:
  
  White space fixup - indent under the opening (
  
  ------------------------------------------------------------------------
  r6203 | jyang | 2009-11-19 15:12:22 +0200 (Thu, 19 Nov 2009) | 8 lines
  Changed paths:
     M /branches/5.1/btr/btr0btr.c
  
  branches/5.1: Use btr_free_root() instead of fseg_free() for
  the fix of bug #48469, because fseg_free() is not defined
  in the zip branch. And we could save one mini-trasaction started
  by fseg_free().
  
  Approved by Marko.
  
  
  ------------------------------------------------------------------------
  r6205 | jyang | 2009-11-20 07:55:48 +0200 (Fri, 20 Nov 2009) | 11 lines
  Changed paths:
     M /branches/5.1/handler/ha_innodb.cc
  
  branches/5.1: Add a special case to handle the Duplicated Key error
  and return DB_ERROR instead. This is to avoid a possible SIGSEGV
  by mysql error handling re-entering the storage layer for dup key
  info without proper table handle.
  This is to prevent a server crash when error situation in bug
  #45961 "DDL on partitioned innodb tables leaves data dictionary
  in an inconsistent state" happens.
  
  rb://157 approved by Sunny Bains.
  
  
  ------------------------------------------------------------------------
  r6206 | jyang | 2009-11-20 09:38:43 +0200 (Fri, 20 Nov 2009) | 5 lines
  Changed paths:
     M /branches/5.1/handler/ha_innodb.cc
  
  branches/5.1: Fix a minor code formating issue for 
  the parenthesis iplacement of the if condition in
  rename_table().
  
  
  ------------------------------------------------------------------------

r6208 | vasil | 2009-11-20 10:49:24 +0200 (Fri, 20 Nov 2009) | 4 lines
branches/zip:

Add ChangeLog entry for c6207.

r6210 | vasil | 2009-11-20 23:39:48 +0200 (Fri, 20 Nov 2009) | 3 lines
branches/zip:

Whitespace fixup.

applied revisions: r6197, r6198, r6199
r6197 - BUG#48317
r6199 - BUG#48526

Detailed revision comments:

r6197 | calvin | 2009-11-19 09:32:55 +0200 (Thu, 19 Nov 2009) | 6 lines
branches/zip: merge the fix of bug#48317 (CMake file)

Due to MySQL changes to the CMake, it is no longer able
to build InnoDB plugin as a static library on Windows.
The fix is proposed by Vlad of MySQL.

r6198 | vasil | 2009-11-19 09:44:31 +0200 (Thu, 19 Nov 2009) | 4 lines
branches/zip:

Add ChangeLog entry for r6197.

r6199 | vasil | 2009-11-19 12:10:12 +0200 (Thu, 19 Nov 2009) | 31 lines
branches/zip: Merge r6159:6198 from branches/5.1:

  ------------------------------------------------------------------------
  r6187 | jyang | 2009-11-18 05:27:30 +0200 (Wed, 18 Nov 2009) | 9 lines
  Changed paths:
     M /branches/5.1/btr/btr0btr.c
  
  branches/5.1: Fix bug #48469 "when innodb tablespace is
  configured too small, crash and corruption!". Function
  btr_create() did not check the return status of fseg_create(),
  and continue the index creation even there is no sufficient
  space.
  
  rb://205 Approved by Marko
  
  
  ------------------------------------------------------------------------
  r6188 | jyang | 2009-11-18 07:14:23 +0200 (Wed, 18 Nov 2009) | 8 lines
  Changed paths:
     M /branches/5.1/data/data0type.c
  
  branches/5.1: Fix bug #48526 "Data type for float and
  double is incorrectly reported in InnoDB table monitor".
  Certain datatypes are not printed correctly in
  dtype_print().
  
  rb://204 Approved by Marko.
  
  
  ------------------------------------------------------------------------

applied revisions: r6185, r6186, r6189, r6194

r6185 - only code changes incorporated, changesets which change
        innodb tests in the main mysql suite are discarded

r61889 - Fixes BUG#48782

Detailed revision comments:

r6185 | marko | 2009-11-17 16:44:20 +0200 (Tue, 17 Nov 2009) | 16 lines
branches/zip: Report duplicate table names
to the client connection, not to the error log.  This change will allow
innodb-index.test to be re-enabled.  It was previously disabled, because
mysql-test-run does not like output in the error log.

row_create_table_for_mysql(): Do not output anything to the error log
when reporting DB_DUPLICATE_KEY.  Let the caller report the error.
Add a TODO comment that the dict_table_t object is apparently not freed
when an error occurs.

create_table_def(): Convert InnoDB table names to the character set
of the client connection for reporting.  Use my_error(ER_WRONG_COLUMN_NAME)
for reporting reserved column names.  Report my_error(ER_TABLE_EXISTS_ERROR)
when row_create_table_for_mysql() returns DB_DUPLICATE_KEY.

rb://206
r6186 | vasil | 2009-11-17 16:48:14 +0200 (Tue, 17 Nov 2009) | 4 lines
branches/zip:

Add ChangeLog entry for r6185.

r6189 | marko | 2009-11-18 11:36:18 +0200 (Wed, 18 Nov 2009) | 5 lines
branches/zip: ha_innobase::add_index(): When creating the primary key
and the table is being locked by another transaction,
do not attempt to drop the table. (Bug #48782)

Approved by Sunny Bains over IM
r6194 | vasil | 2009-11-19 09:24:45 +0200 (Thu, 19 Nov 2009) | 5 lines
branches/zip:

Increment version number from 1.0.5 to 1.0.6 since 1.0.5 was just released
by MySQL and we will soon release 1.0.6.

Fixes several test cases.

applied revisions: r6181, r6182, r6183, r6184

r6183 - changes are made only to tests in innodb suite which is only 
        innodb-consistent-master.opt

Detailed revision comments:

r6181 | vasil | 2009-11-17 12:21:41 +0200 (Tue, 17 Nov 2009) | 33 lines
branches/zip:

At the end of innodb-index.test: restore the environment as it was before
the test was started to silence this warning:

  MTR's internal check of the test case 'main.innodb-index' failed.
  This means that the test case does not preserve the state that existed
  before the test case was executed.  Most likely the test case did not
  do a proper clean-up.
  This is the diff of the states of the servers before and after the
  test case was executed:
  mysqltest: Logging to '/tmp/autotest.sh-20091117_033000-zip.btyZwu/mysql-5.1/mysql-test/var/tmp/check-mysqld_1.log'.
  mysqltest: Results saved in '/tmp/autotest.sh-20091117_033000-zip.btyZwu/mysql-5.1/mysql-test/var/tmp/check-mysqld_1.result'.
  mysqltest: Connecting to server localhost:13000 (socket /tmp/autotest.sh-20091117_033000-zip.btyZwu/mysql-5.1/mysql-test/var/tmp/mysqld.1.sock) as 'root', connection 'default', attempt 0 ...
  mysqltest: ... Connected.
  mysqltest: Start processing test commands from './include/check-testcase.test' ...
  mysqltest: ... Done processing test commands.
  --- /tmp/autotest.sh-20091117_033000-zip.btyZwu/mysql-5.1/mysql-test/var/tmp/check-mysqld_1.result	2009-11-17 13:10:40.000000000 +0300
  +++ /tmp/autotest.sh-20091117_033000-zip.btyZwu/mysql-5.1/mysql-test/var/tmp/check-mysqld_1.reject	2009-11-17 13:10:54.000000000 +0300
  @@ -84,7 +84,7 @@
   INNODB_DOUBLEWRITE	ON
   INNODB_FAST_SHUTDOWN	1
   INNODB_FILE_FORMAT	Antelope
  -INNODB_FILE_FORMAT_CHECK	Antelope
  +INNODB_FILE_FORMAT_CHECK	Barracuda
   INNODB_FILE_PER_TABLE	OFF
   INNODB_FLUSH_LOG_AT_TRX_COMMIT	1
   INNODB_FLUSH_METHOD	
  
  mysqltest: Result content mismatch
  
  not ok

r6182 | marko | 2009-11-17 13:49:15 +0200 (Tue, 17 Nov 2009) | 1 line
branches/zip: Set svn:eol-style on mysql-test files.
r6183 | marko | 2009-11-17 13:51:16 +0200 (Tue, 17 Nov 2009) | 1 line
branches/zip: Prepend loose_ to plugin-only mysql-test options.
r6184 | marko | 2009-11-17 13:52:01 +0200 (Tue, 17 Nov 2009) | 1 line
branches/zip: innodb-index.test: Restore innodb_file_format_check.

merging from 5.1 to rep+2 starting at gca(5.1, next-mr) == build@mysql.com-20091104182209-iui387z35159aoyw

1. add testcase for BUG#46676
2. Allow CREATE INDEX to be interrupted
3. ha_innobase::change_active_index(): When the history is
   missing, report it to the client, not to the error log
4. ChangeLog entries

appplied revisions:r6169, r6170, r6175, r6177, r6179

Detailed revision comments:

r6169 | calvin | 2009-11-12 14:40:43 +0200 (Thu, 12 Nov 2009) | 6 lines
branches/zip: add test case for bug#46676

This crash is reproducible with InnoDB plugin 1.0.4 + MySQL 5.1.37.
But no longer reproducible after MySQL 5.1.38 (with plugin 1.0.5).
Add test case to catch future regression.

r6170 | marko | 2009-11-12 15:49:08 +0200 (Thu, 12 Nov 2009) | 4 lines
branches/zip: Allow CREATE INDEX to be interrupted.  (Issue #354)

rb://183 approved by Heikki Tuuri

r6175 | vasil | 2009-11-16 20:07:39 +0200 (Mon, 16 Nov 2009) | 4 lines
branches/zip:

Wrap line at 78th char in the ChangeLog

r6177 | calvin | 2009-11-16 20:20:38 +0200 (Mon, 16 Nov 2009) | 2 lines
branches/zip: add an entry to ChangeLog for r6065

r6179 | marko | 2009-11-17 10:19:34 +0200 (Tue, 17 Nov 2009) | 2 lines
branches/zip: ha_innobase::change_active_index(): When the history is
missing, report it to the client, not to the error log.

applied revisions: r6161, r6162, r6163

Detailed revision comments:

r6161 | vasil | 2009-11-11 15:36:16 +0200 (Wed, 11 Nov 2009) | 4 lines
branches/zip:

Add changelog entry for r6160.

r6162 | vasil | 2009-11-11 16:00:12 +0200 (Wed, 11 Nov 2009) | 4 lines
branches/zip:

Add ChangeLog for r6157.

r6163 | calvin | 2009-11-11 17:53:20 +0200 (Wed, 11 Nov 2009) | 8 lines
branches/zip: Exclude thd_binlog_filter_ok() when building
with older version of MySQL.

thd_binlog_filter_ok() is introduced in MySQL 5.1.41. But the
plugin can be built with MySQL prior to 5.1.41.

Approved by Heikki (on IM).

applied revisions: r6160

Detailed revision comments:

r6160 | vasil | 2009-11-11 15:33:49 +0200 (Wed, 11 Nov 2009) | 72 lines
branches/zip: Merge r6152:6159 from branches/5.1:

(r6158 was skipped as an equivallent change has already been merged from MySQL)

  ------------------------------------------------------------------------
  r6154 | calvin | 2009-11-11 02:51:17 +0200 (Wed, 11 Nov 2009) | 17 lines
  Changed paths:
     M /branches/5.1/include/os0file.h
     M /branches/5.1/os/os0file.c
  
  branches/5.1: fix bug#3139: Mysql crashes: 'windows error 995'
  after several selects on a large DB
  
  During stress environment, Windows AIO may fail with error code
  ERROR_OPERATION_ABORTED. InnoDB does not handle the error, rather
  crashes. The cause of the error is unknown, but likely due to
  faulty hardware or driver.
  
  This patch introduces a new error code OS_FILE_OPERATION_ABORTED,
  which maps to Windows ERROR_OPERATION_ABORTED (995). When the error
  is detected during AIO, the InnoDB will issue a synchronous retry
  (read/write).
  
  This patch has been extensively tested by MySQL support.
  
  Approved by: Marko
  rb://196
  ------------------------------------------------------------------------
  r6158 | vasil | 2009-11-11 14:52:14 +0200 (Wed, 11 Nov 2009) | 37 lines
  Changed paths:
     M /branches/5.1/handler/ha_innodb.cc
     M /branches/5.1/handler/ha_innodb.h
  
  branches/5.1:
  
  Merge a change from MySQL:
  (this has been reviewed by Calvin and Marko, and Calvin says Luis has
  incorporated Marko's suggestions)
  
    ------------------------------------------------------------
    revno: 3092.5.1
    committer: Luis Soares <luis.soares@sun.com>
    branch nick: mysql-5.1-bugteam
    timestamp: Thu 2009-09-24 15:52:52 +0100
    message:
      BUG#42829: binlogging enabled for all schemas regardless of
      binlog-db-db / binlog-ignore-db
            
      InnoDB will return an error if statement based replication is used
      along with transaction isolation level READ-COMMITTED (or weaker),
      even if the statement in question is filtered out according to the
      binlog-do-db rules set. In this case, an error should not be printed.
            
      This patch addresses this issue by extending the existing check in
      external_lock to take into account the filter rules before deciding to
      print an error. Furthermore, it also changes decide_logging_format to
      take into consideration whether the statement is filtered out from 
      binlog before decision is made.
    added:
      mysql-test/suite/binlog/r/binlog_stm_do_db.result
      mysql-test/suite/binlog/t/binlog_stm_do_db-master.opt
      mysql-test/suite/binlog/t/binlog_stm_do_db.test
    modified:
      sql/sql_base.cc
      sql/sql_class.cc
      storage/innobase/handler/ha_innodb.cc
      storage/innobase/handler/ha_innodb.h
      storage/innodb_plugin/handler/ha_innodb.cc
      storage/innodb_plugin/handler/ha_innodb.h
  
  ------------------------------------------------------------------------

applied revisions: r6157

Detailed revision comments:

r6157 | jyang | 2009-11-11 14:27:09 +0200 (Wed, 11 Nov 2009) | 10 lines
branches/zip: Fix an issue that a local variable defined
in innodb_file_format_check_validate() is being referenced
across function in innodb_file_format_check_update().
In addition, fix "set global innodb_file_format_check =
DEFAULT" call.
Bug #47167: "set global innodb_file_format_check" cannot
set value by User-Defined Variable."
rb://169 approved by Sunny Bains and Marko.