into  weblab.(none):/home/marcsql/TREE/mysql-5.1-21554-merge

The problem reported is a compile bug,
reported by the development GCC team with GCC 4.2.

The original issue can no longer be reproduced in MySQL 5.1,
since the configure script no longer define HAVE_ATOMIC_ADD,
which caused the Linux atomic functions to be used (and cause a problem
with an invalid cast).

This patch implements some code cleanup for 5.1 only, which was identified
during the investigation of this issue.

With this patch, statistics maintained in THD::status_var are by definition
owned by the running thread, and do not need to be protected against race
conditions. These statistics are maintained by the status_var_* helpers,
which do not require any lock.


include/my_global.h:
  General cleanup of thread_safe_increment / statistic_increment
include/my_pthread.h:
  General cleanup of thread_safe_increment / statistic_increment
sql/filesort.cc:
  General cleanup of thread_safe_increment / statistic_increment
sql/handler.cc:
  General cleanup of thread_safe_increment / statistic_increment
sql/sql_insert.cc:
  General cleanup of thread_safe_increment / statistic_increment
sql/sql_parse.cc:
  General cleanup of thread_safe_increment / statistic_increment
sql/sql_prepare.cc:
  General cleanup of thread_safe_increment / statistic_increment
sql/sql_select.cc:
  General cleanup of thread_safe_increment / statistic_increment

tests/mysql_client_test.c:
  Add a test case for bug#28075 - just send COM_DEBUG and check whether
  the server is still alive.

Uninitialized in the constructor member variables were
pointing to nirvana and causing a crash when debug information
of the Event Scheduler was dumped in result to COM_DEBUG
packet sent to the server.


sql/event_queue.cc:
  Initialize member variables or they will point to
  nirvana and could possible cause a crash during
  dumping of debug information.
sql/event_queue.h:
  remove unneeded line
sql/event_scheduler.cc:
  Initialize member variables or they will point to
  nirvana and could possible cause a crash during
  dumping of debug information.
sql/event_scheduler.h:
  remove unneeded state

into  mockturtle.local:/home/dlenev/src/mysql-5.1-alter


sql/sql_base.cc:
  Auto merged

and invalidation in the most general case (non-temporary table and
not simple RENAME or ENABLE/DISABLE KEYS or partitioning command).

See comment for sql/sql_table.cc for more information.

These changes are prerequisite for 5.1 version of fix for bug #23667
"CREATE TABLE LIKE is not isolated from alteration by other connections"


mysql-test/include/mix1.inc:
  Extended coverage for behavior of ALTER TABLE statement under LOCK TABLES,
  which should be consistent across all platforms and for all engines.
mysql-test/r/alter_table-big.result:
  Changed test for bug #25044 to use @@debug and injected sleeps
  infrastructure. Extended test coverage for ALTER TABLE's behavior
  under concurrency.
mysql-test/r/alter_table.result:
  Extended coverage for behavior of ALTER TABLE statement under LOCK TABLES,
  which should be consistent across all platforms and for all engines.
mysql-test/r/innodb_mysql.result:
  Extended coverage for behavior of ALTER TABLE statement under LOCK TABLES,
  which should be consistent across all platforms and for all engines.
mysql-test/t/alter_table-big.test:
  Changed test for bug #25044 to use @@debug and injected sleeps
  infrastructure. Extended test coverage for ALTER TABLE's behavior
  under concurrency.
mysql-test/t/alter_table.test:
  Extended coverage for behavior of ALTER TABLE statement under LOCK TABLES,
  which should be consistent across all platforms and for all engines.
sql/mysql_priv.h:
  Made functions reopen_table() and close_handle_and_leave_table_as_lock()
  available outside of sql_base.cc file.
  Changed close_data_tables() in such way that after closing handler
  for the table it leaves TABLE object for it in table cache not as
  placeholder for ordinary name-lock but as placeholder for an exclusive
  name-lock. Renamed this routine to close_data_files_and_morph_locks().
sql/sql_base.cc:
  Made functions reopen_table() and close_handle_and_leave_table_as_lock()
  available outside of sql_base.cc file.
  Changed close_data_tables() in such way that after closing handler
  for the table it leaves TABLE object for it in table cache not as
  placeholder for ordinary name-lock but as placeholder for an exclusive
  name-lock. Renamed this routine to close_data_files_and_morph_locks().
  Also adjusted it so it can work properly not only in LOCK TABLES mode.
sql/sql_table.cc:
  Changed the way in which ALTER TABLE implementation handles table
  locking and invalidation in the most general case (non-temporary table
  and not simple RENAME or ENABLE/DISABLE KEYS or partitioning command)
  
  Now after preparing new version of the table we:
  1) Wait until all other threads close old version of table.
  2) Close instances of table open by this thread and replace them
     with exclusive name-locks.
  3) Rename the old table to a temp name, rename the new one to the
     old name.
  4) If we are under LOCK TABLES and don't do ALTER TABLE ... RENAME
     we reopen new version of table.
  5) Write statement to the binary log.
  6) If we are under LOCK TABLES and do ALTER TABLE ... RENAME we
     remove name-locks from list of open tables and table cache.
  7) If we are not not under LOCK TABLES we rely on close_thread_tables()
     call to remove name-locks from table cache and list of open table. 
  
  Such approach:
  a) Eliminates possibility for concurrent statement to sneak in and get
     access to the new version of the table before ALTER TABLE gets logged
     into binary log.
  b) Ensures that ALTER TABLE behaves under LOCK TABLES in the same way
     on all platforms and for all engines (in 5.0 this was not true) 
  c) Preserves nice invariant that if table is open in some connection
     there is a guarantee that .FRM file for this table exists and is
     properly named.

into  adventure.(none):/home/thek/Development/cpp/mysql-5.1-runtime


mysql-test/t/sp-vars.test:
  Auto merged
sql/item_func.cc:
  Auto merged
sql/item_func.h:
  Auto merged
mysql-test/r/sp-vars.result:
  manual merge

- Adding variable m_cached_result_type to keep the variable type consistent
  during the execution of a statement.
- Before each result set is returned to the client the description of each
  column is sent as meta data.
  Previously the result type for a column could change if the hash variable
  entry changed between statements. This caused the result set of the query
  to alternate column types in certain cases which is not supported by MySQL
  client-server protocol. Example:
  Previously this sequence:
    SET @A:=1;
    SELECT @A:="text", @A;
  would return "text", "text";
 
  After the change the SELECT returns "text", 0
  The reson for this is that previously the result set from 'SELECT @A;'
  would always be of the type STRING, whereas now the type of the variable
  is taken from the last SET statement. However, 'SELECT @A:="text"' will
  return type of STRING since the right side of the assignment is used.


mysql-test/r/ps_2myisam.result:
  Changed test result because SQL type of a user variable now
  more accurately represents its Item type: since Item type of a variable
  can be either STRING, INT, DECIMAL or DOUBLE, SQL type of the
  result set metadata now can be either MYSQL_TYPE_VARCHAR,
  MYSQL_TYPE_LONGLONG, MYSQL_TYPE_NEWDECIMAL or MYSQL_TYPE_DOUBLE.
  
  Previously it was always MYSQL_TYPE_VARCHAR.
  
  In particular, integer variables now have changed from
  MYSQL_TYPE_VARCHAR to MYSQL_TYPE_LONGLONG.
mysql-test/r/ps_3innodb.result:
  Changed test result because SQL type of a user variable now
  more accurately represents its Item type: since Item type of a variable
  can be either STRING, INT, DECIMAL or DOUBLE, SQL type of the
  result set metadata now can be either MYSQL_TYPE_VARCHAR,
  MYSQL_TYPE_LONGLONG, MYSQL_TYPE_NEWDECIMAL or MYSQL_TYPE_DOUBLE.
  
  Previously it was always MYSQL_TYPE_VARCHAR.
  
  In particular, integer variables now have changed from
  MYSQL_TYPE_VARCHAR to MYSQL_TYPE_LONGLONG.
mysql-test/r/ps_4heap.result:
  Changed test result because SQL type of a user variable now
  more accurately represents its Item type: since Item type of a variable
  can be either STRING, INT, DECIMAL or DOUBLE, SQL type of the
  result set metadata now can be either MYSQL_TYPE_VARCHAR,
  MYSQL_TYPE_LONGLONG, MYSQL_TYPE_NEWDECIMAL or MYSQL_TYPE_DOUBLE.
  
  Previously it was always MYSQL_TYPE_VARCHAR.
  
  In particular, integer variables now have changed from
  MYSQL_TYPE_VARCHAR to MYSQL_TYPE_LONGLONG.
mysql-test/r/ps_5merge.result:
  Changed test result because SQL type of a user variable now
  more accurately represents its Item type: since Item type of a variable
  can be either STRING, INT, DECIMAL or DOUBLE, SQL type of the
  result set metadata now can be either MYSQL_TYPE_VARCHAR,
  MYSQL_TYPE_LONGLONG, MYSQL_TYPE_NEWDECIMAL or MYSQL_TYPE_DOUBLE.
  
  Previously it was always MYSQL_TYPE_VARCHAR.
  
  In particular, integer variables now have changed from
  MYSQL_TYPE_VARCHAR to MYSQL_TYPE_LONGLONG.
mysql-test/r/ps_7ndb.result:
  Changed test result because SQL type of a user variable now
  more accurately represents its Item type: since Item type of a variable
  can be either STRING, INT, DECIMAL or DOUBLE, SQL type of the
  result set metadata now can be either MYSQL_TYPE_VARCHAR,
  MYSQL_TYPE_LONGLONG, MYSQL_TYPE_NEWDECIMAL or MYSQL_TYPE_DOUBLE.
  
  Previously it was always MYSQL_TYPE_VARCHAR.
  
  In particular, integer variables now have changed from
  MYSQL_TYPE_VARCHAR to MYSQL_TYPE_LONGLONG.
mysql-test/r/sp-vars.result:
  Added test case. Previously variables could change their variable type during 
  the execution of a statement.
  Which variable type to use in the statement is specified in
  any previous statement.
mysql-test/r/type_date.result:
  This test case result is changed because it is no longer allowed for user
  variables to change their variable type during the execution of a statement.
  The determination of which variable type to use in the statement is specified in
  any previous statement.
mysql-test/r/user_var.result:
  This test case result is changed because it is no longer allowed for user
  variables to change their variable type during the execution of a statement.
  The determination of which variable type to use in the statement is specified in
  any previous statement.
mysql-test/t/sp-vars.test:
  Added test case. Previously variables could change their variable type during 
  the execution of a statement.
  Which variable type to use in the statement is specified in
  any previous statement.
mysql-test/t/type_date.test:
  This test case result is changed because it is no longer allowed for user
  variables to change their variable type during the execution of a statement.
  The determination of which variable type to use in the statement is specified in
  any previous statement.
sql/item_func.cc:
  Adding variable m_cached_result_type to keep the variable type consistent
  during the execution of a statement.
  Previously the result type could change if the hash variable entry changed
  between statements. This caused the result set of the query to alternate
  column types in certain cases.
sql/item_func.h:
  Adding variable m_cached_result_type to keep the variable type consistent
  during the execution of a statement.
  Previously the result type could change if the hash variable entry changed
  between statements. This caused the result set of the query to alternate
  column types in certain cases.

into  vajra.(none):/opt/local/work/mysql-5.1-runtime


mysql-test/r/sp-error.result:
  Auto merged
mysql-test/r/sp-prelocking.result:
  Auto merged
mysql-test/r/trigger.result:
  Auto merged
mysql-test/t/sp-error.test:
  Auto merged
mysql-test/t/sp-prelocking.test:
  Auto merged
mysql-test/t/trigger.test:
  Auto merged
sql/sql_base.cc:
  Auto merged

Adjust the check that defines the error message to be returned.


mysql-test/r/sp-error.result:
  Update results (more accurate error code)
mysql-test/r/sp-prelocking.result:
  Update results (more accurate error code)
mysql-test/r/trigger.result:
  Update results (more accurate error code)
mysql-test/t/sp-error.test:
  ER_NOT_LOCKED -> ER_NO_SUCH_TABLE
mysql-test/t/sp-prelocking.test:
  Add a test case for Bug#27907
mysql-test/t/trigger.test:
  ER_NOT_LOCKED -> ER_NO_SUCH_TABLE
sql/sql_base.cc:
  Adjust the check for where-we-are for a better error message.

storage/ndb/include/mgmapi/mgmapi.h:
  A tiny fix to make sure MGM API does not clutter doxygen index
  page for the entire MySQL Server.

into  adventure.(none):/home/thek/Development/cpp/mysql-5.0-runtime

into  adventure.(none):/home/thek/Development/cpp/mysql-5.1-runtime

into  adventure.(none):/home/thek/Development/cpp/bug27415/my51-bug27415


mysql-test/r/sp-vars.result:
  Auto merged
mysql-test/t/sp-vars.test:
  Auto merged

 - Problem was reported as a SP variable using itself as 
   right value inside SUBSTR caused corruption of data. 
 - This bug could not be verified in either 5.0bk or 5.1bk
 - Added test case to prevent future regressions.


mysql-test/r/sp-vars.result:
  Added test case for a reported regression which couldn't be
  verified.
mysql-test/t/sp-vars.test:
  Added test case for a reported regression which couldn't be
  verified.

sql/event_data_objects.cc:
  Fix typo.

(Bug#26338 "events_bugs.test fail on Debian")


mysql-test/r/events_bugs.result:
  Update results.
mysql-test/t/events_bugs.test:
  Make a stab at fixing events_bugs.test failure on Debian.
  The problem is purely in the race inherent in the test case: an
  event that was started doesn't go away fast enough and clutters the 
  processlist. This patch remove the part of the event that
  we can not wait on synchronously (there is no table 'hashed_num'
  referenced anywhere).

open bug report, reproduced in the runtime team tree).


sql/event_data_objects.cc:
  Make a stub at fixing a race in event_bugs.test under valgrind:
  read of uninitialized byte in SHOW PROCESSLIST from an event thread.

into  vajra.(none):/opt/local/work/mysql-5.1-21483


sql/sql_insert.cc:
  Auto merged

sql/sql_insert.cc:
  Do not access delayed thread memory without a mutex.

sql/sql_insert.cc:
  Correct a merge error (wrong declaration was used).

into  vajra.(none):/opt/local/work/mysql-5.1-21483


sql/sp_head.cc:
  Auto merged
sql/sql_base.cc:
  Auto merged
sql/sql_lex.h:
  Auto merged
mysql-test/r/insert.result:
  Manual merge.
mysql-test/t/insert.test:
  Manual merge.
sql/sql_insert.cc:
  Manual merge.

into  vajra.(none):/opt/local/work/mysql-5.0-21483


sql/sp_head.cc:
  Auto merged
sql/sql_base.cc:
  Auto merged
sql/sql_insert.cc:
  Auto merged
sql/sql_lex.h:
  Auto merged

Bug#21483 "Server abort or deadlock on INSERT DELAYED with another
implicit insert"
Also fixes and adds test cases for bugs:
20497 "Trigger with INSERT DELAYED causes Error 1165"
21714 "Wrong NEW.value and server abort on INSERT DELAYED to a
table with a trigger".
Post-review fixes.

Problem:
In MySQL INSERT DELAYED is a way to pipe all inserts into a
given table through a dedicated thread. This is necessary for
simplistic storage engines like MyISAM, which do not have internal
concurrency control or threading and thus can not
achieve efficient INSERT throughput without support from SQL layer.
DELAYED INSERT works as follows:
For every distinct table, which can accept DELAYED inserts and has
pending data to insert, a dedicated thread is created to write data
to disk. All user connection threads that attempt to
delayed-insert into this table interact with the dedicated thread in
producer/consumer fashion: all records to-be inserted are pushed
into a queue of the dedicated thread, which fetches the records and 
writes them.
In this design, client connection threads never open or lock
the delayed insert table.
This functionality was introduced in version 3.23 and does not take 
into account existence of triggers, views, or pre-locking.
E.g. if INSERT DELAYED is called from a stored function, which,
in turn, is called from another stored function that uses the delayed
table, a deadlock can occur, because delayed locking by-passes
pre-locking. Besides:
 * the delayed thread works directly with the subject table through
   the storage engine API and does not invoke triggers
 * even if it was patched to invoke triggers, if triggers,
   in turn, used other tables, the delayed thread would
   have to open and lock involved tables (use pre-locking).
 * even if it was patched to use pre-locking, without deadlock
   detection the delayed thread could easily lock out user 
   connection threads in case when the same table is used both
   in a trigger and on the right side of the insert query: 
   the delayed thread would not release locks until all inserts 
   are complete, and user connection can not complete inserts 
   without having locks on the tables used on the right side of the
   query.

Solution:

These considerations suggest two general alternatives for the
future of INSERT DELAYED:
 * it is considered a full-fledged alternative to normal INSERT
 * it is regarded as an optimisation that is only relevant 
   for simplistic engines.
Since we missed our chance to provide complete support of new
features when 5.0 was in development, the first alternative
currently renders infeasible.
However, even the second alternative, which is to detect
new features and convert DELAYED insert into a normal insert, 
is not easy to implement.
The catch-22 is that we don't know if the subject table has triggers
or is a view before we open it, and we only open it in the
delayed thread. We don't know if the query involves pre-locking
until we have opened all tables, and we always first create
the delayed thread, and only then open the remaining tables.
This patch detects the problematic scenarios and converts
DELAYED INSERT to a normal INSERT using the following approach:
 * if the statement is executed under pre-locking (e.g. from
   within a stored function or trigger) or the right
   side may require pre-locking, we detect the situation
   before creating a delayed insert thread and convert the statement
   to a conventional INSERT.
  * if the subject table is a view or has triggers, we shutdown
   the delayed thread and convert the statement to a conventional
   INSERT.


mysql-test/r/insert.result:
  Update test results.
mysql-test/t/insert.test:
  Add a test case for Bug#21483, Bug#20497, Bug#21714 (INSERT DELAYED
  and stored routines, triggers).
sql/sp_head.cc:
  Upgrade lock type to TL_WRITE when computing the pre-locking set.
sql/sql_base.cc:
  Use a new method.
sql/sql_insert.cc:
  INSERT DELAYED and pre-locking:
  - if  under pre-locking, upgrade the lock type to TL_WRITE
  and proceed as a normal write
  - if DELAYED table has triggers, also request a lock upgrade.
  - make sure errors in the delayed thread are propagated
  correctly
sql/sql_lex.h:
  Add a method to check if a parsed tree refers to stored
  routines.

into  vajra.(none):/opt/local/work/mysql-5.1-runtime


sql/item.cc:
  Auto merged
sql/item_func.cc:
  Auto merged
sql/mysql_priv.h:
  Auto merged
sql/set_var.cc:
  Auto merged
sql/sql_class.h:
  Auto merged
sql/sql_insert.cc:
  Auto merged
sql/sql_parse.cc:
  Auto merged
sql/sql_prepare.cc:
  Auto merged
sql/sql_yacc.yy:
  Auto merged
mysql-test/t/disabled.def:
  Manual merge.

into  vajra.(none):/opt/local/work/mysql-5.1-runtime


mysql-test/include/mix1.inc:
  Auto merged
mysql-test/r/innodb_mysql.result:
  Auto merged
mysql-test/r/ps_1general.result:
  Auto merged
mysql-test/t/disabled.def:
  Auto merged
mysql-test/t/ps_1general.test:
  Auto merged
sql/ha_ndbcluster.cc:
  Auto merged
sql/ha_ndbcluster_binlog.cc:
  Auto merged
sql/item.cc:
  Auto merged
sql/item_func.cc:
  Auto merged
sql/mysql_priv.h:
  Auto merged
sql/set_var.cc:
  Auto merged
sql/sql_base.cc:
  Auto merged
sql/sql_cache.cc:
  Auto merged
sql/sql_class.cc:
  Auto merged
sql/sql_class.h:
  Auto merged
sql/sql_lex.cc:
  Auto merged
sql/sql_lex.h:
  Auto merged
sql/sql_parse.cc:
  Auto merged
sql/sql_partition.cc:
  Auto merged
sql/sql_prepare.cc:
  Auto merged
sql/sql_table.cc:
  Auto merged
sql/sql_yacc.yy:
  Auto merged
sql/table.h:
  Auto merged
sql/sql_insert.cc:
  SCCS merged

into  vajra.(none):/opt/local/work/mysql-5.0-runtime

into  vajra.(none):/opt/local/work/mysql-5.0-runtime


sql/item.cc:
  Auto merged
sql/item_func.cc:
  Auto merged
sql/mysql_priv.h:
  Auto merged
sql/set_var.cc:
  Auto merged
sql/sql_class.h:
  Auto merged
sql/sql_insert.cc:
  Auto merged
sql/sql_parse.cc:
  Auto merged
sql/sql_prepare.cc:
  Auto merged
sql/sql_yacc.yy:
  Auto merged

into  vajra.(none):/opt/local/work/mysql-4.1-runtime

into  mockturtle.local:/home/dlenev/src/mysql-5.1-cts-3

TABLES" and failures of alter_table.test on Windows which occured after
pushing fix for bugs #20662, #20903, #24508, #24738 (various problems
with CREATE TABLE SELECT).

ALTER TABLE statements which were handled using "fast" alter table
optimization were not properly working under LOCK TABLES if table
was transactional (for all table types under Windows).

Code implementing "fast" version of ALTER TABLE tried to open and
lock table using open_ltable() after renaming .FRM files (which
corresponds to renaming tables in normal case) in some cases
(for transactional tables or on Windows). This caused problems
under LOCK TABLES and conflicted with name-lock taken by 
ALTER TABLE RENAME on target tables.

This patch solves this issue by using reopen_name_locked_table()
instead of open_ltable().


mysql-test/include/mix1.inc:
  Added test for bug #28415 "Some ALTER TABLE statements no longer work
  under LOCK TABLES" and minimal coverage for fast ALTER TABLE behaviour
  for transactional tables.
mysql-test/r/innodb_mysql.result:
  Added test for bug #28415 "Some ALTER TABLE statements no longer work
  under LOCK TABLES" and minimal coverage for fast ALTER TABLE behaviour
  for transactional tables.
sql/sql_table.cc:
  mysql_alter_table():
    Fixed handling of transactional tables (and all tables on Windows)
    by "fast" ALTER TABLE.
    Code implementing "fast" version of ALTER TABLE tried to open and
    lock table using open_ltable() after renaming .FRM files (which
    corresponds to renaming tables in normal case) for such tables.
    This caused problems under LOCK TABLES and conflicted with name-lock
    taken by ALTER TABLE RENAME on target tables. We solve this issue by
    using reopen_name_locked_table() instead of open_ltable().

into  vajra.(none):/opt/local/work/mysql-5.1-runtime


sql/sql_insert.cc:
  Manual merge.

sql/sql_insert.cc:
  Fix a compile-time warning (log_on is not used in embedded).

into  bk-internal.mysql.com:/data0/bk/mysql-5.1-opt


sql/field.cc:
  Auto merged