Bug#25422 (Hang with log tables)
Bug 17876 (Truncating mysql.slow_log in a SP after using cursor locks the
          thread)
Bug 23044 (Warnings on flush of a log table)
Bug 29129 (Resetting general_log while the GLOBAL READ LOCK is set causes
           a deadlock)

Prior to this fix, the server would hang when performing concurrent
ALTER TABLE or TRUNCATE TABLE statements against the LOG TABLES,
which are mysql.general_log and mysql.slow_log.

The root cause traces to the following code:
in sql_base.cc, open_table()
  if (table->in_use != thd)
  {
    /* wait_for_condition will unlock LOCK_open for us */
    wait_for_condition(thd, &LOCK_open, &COND_refresh);
  }
The problem with this code is that the current implementation of the
LOGGER creates 'fake' THD objects, like
- Log_to_csv_event_handler::general_log_thd
- Log_to_csv_event_handler::slow_log_thd
which are not associated to a real thread running in the server,
so that waiting for these non-existing threads to release table locks
cause the dead lock.

In general, the design of Log_to_csv_event_handler does not fit into the
general architecture of the server, so that the concept of general_log_thd
and slow_log_thd has to be abandoned:
- this implementation does not work with table locking
- it will not work with commands like SHOW PROCESSLIST
- having the log tables always opened does not integrate well with DDL
operations / FLUSH TABLES / SET GLOBAL READ_ONLY

With this patch, the fundamental design of the LOGGER has been changed to:
- always open and close a log table when writing a log
- remove totally the usage of fake THD objects
- clarify how locking of log tables is implemented in general.

See WL#3984 for details related to the new locking design.

Additional changes (misc bugs exposed and fixed):

1)

mysqldump which would ignore some tables in dump_all_tables_in_db(),
 but forget to ignore the same in dump_all_views_in_db().

2)

mysqldump would also issue an empty "LOCK TABLE" command when all the tables
to lock are to be ignored (numrows == 0), instead of not issuing the query.

3)

Internal errors handlers could intercept errors but not warnings
(see sql_error.cc).

4)

Implementing a nested call to open tables, for the performance schema tables,
exposed an existing bug in remove_table_from_cache(), which would perform:
  in_use->some_tables_deleted=1;
against another thread, without any consideration about thread locking.
This call inside remove_table_from_cache() was not required anyway,
since calling mysql_lock_abort() takes care of aborting -- cleanly -- threads
that might hold a lock on a table.
This line (in_use->some_tables_deleted=1) has been removed.


sql/handler.cc:
  Moved logic for system / log tables in the SQL layer.
sql/handler.h:
  Moved logic for system / log tables in the SQL layer.
sql/lock.cc:
  Revised locking of log tables
sql/log.cc:
  Major cleanup: changed how log tables are locked / written to.
sql/log.h:
  Major cleanup: changed how log tables are locked / written to.
sql/mysql_priv.h:
  performance schema helpers
sql/slave.cc:
  open_ltable() lock flags
sql/sp.cc:
  open_ltable() lock flags
sql/sql_acl.cc:
  open_ltable() lock flags
sql/sql_class.h:
  performance schema helpers
sql/sql_delete.cc:
  log tables cleanup in TRUNCATE
sql/sql_error.cc:
  Internal handlers can also intercept warnings
sql/sql_insert.cc:
  open_ltable() lock flags
sql/sql_parse.cc:
  performance schema helpers
sql/sql_plugin.cc:
  open_ltable() lock flags
sql/sql_rename.cc:
  log tables cleanup in RENAME
sql/sql_servers.cc:
  open_ltable() lock flags
sql/sql_show.cc:
  Move INFORMATION_SCHEMA_NAME to table.cc
sql/sql_table.cc:
  log tables cleanup (admin operations, ALTER TABLE)
sql/sql_udf.cc:
  open_ltable() lock flags
sql/table.cc:
  Implemented TABLE_CATEGORY.
sql/share/errmsg.txt:
  Changed the wording and name of ER_CANT_READ_LOCK_LOG_TABLE
sql/table.h:
  Implemented TABLE_CATEGORY.
storage/csv/ha_tina.cc:
  Moved logic for system / log tables in the SQL layer.
storage/csv/ha_tina.h:
  Moved logic for system / log tables in the SQL layer.
storage/myisam/ha_myisam.cc:
  Moved logic for system / log tables in the SQL layer.
storage/myisam/ha_myisam.h:
  Moved logic for system / log tables in the SQL layer.
client/mysqldump.c:
  Don't lock tables in the ignore list.
  Don't issue empty LOCK TABLES queries.
sql/sql_base.cc:
  log tables cleanup
  performance schema helpers
mysql-test/r/ps.result:
  Adjust test results
mysql-test/r/show_check.result:
  Adjust test results
mysql-test/r/status.result:
  Adjust test results
mysql-test/t/log_state.test:
  Added tests for Bug#29129
mysql-test/t/ps.test:
  Make the test output deterministic
mysql-test/t/show_check.test:
  Make the test output deterministic
mysql-test/r/log_state.result:
  Changed the default location of the log output to LOG_FILE,
  for backward compatibility with MySQL 5.0
  ---
  Adjust test results
mysql-test/r/log_tables.result:
  cleanup for -ps-protocol
mysql-test/t/log_tables.test:
  cleanup for -ps-protocol
sql/set_var.cc:
  Changed the default location of the log output to LOG_FILE,
  for backward compatibility with MySQL 5.0
  ---
  log tables cleanup

into  bodhi.(none):/opt/local/work/mysql-5.1-runtime


mysql-test/r/create.result:
  Auto merged
mysql-test/t/create.test:
  Auto merged
sql/sql_class.h:
  Auto merged

into  bodhi.(none):/opt/local/work/mysql-5.1-runtime


mysql-test/r/create.result:
  Auto merged
mysql-test/r/innodb.result:
  Auto merged
mysql-test/t/create.test:
  Auto merged
mysql-test/t/innodb.test:
  Auto merged
sql/sql_class.h:
  Auto merged
sql/sql_insert.cc:
  Auto merged

into  bodhi.(none):/opt/local/work/mysql-5.0-runtime


mysql-test/r/create.result:
  Auto merged
mysql-test/t/create.test:
  Auto merged
sql/sql_class.h:
  Auto merged

into  gleb.loc:/home/uchum/work/bk/5.1-opt


mysql-test/r/cast.result:
  Auto merged
mysql-test/t/cast.test:
  Auto merged
sql/item_timefunc.cc:
  Auto merged

into  moonbone.local:/mnt/gentoo64/work/29898-bug-5.0-opt-mysql

sql/sql_lex.cc:
  Fix the style.
sql/sql_lex.h:
  Fix the style.

into  bodhi.(none):/opt/local/work/mysql-5.1-runtime


sql/sql_base.cc:
  Auto merged

into  gleb.loc:/home/uchum/work/bk/5.1-opt


sql/set_var.cc:
  Auto merged
sql/sql_class.h:
  Auto merged

into  gleb.loc:/home/uchum/work/bk/5.1-opt


mysql-test/t/disabled.def:
  Auto merged
sql/set_var.cc:
  Auto merged
sql/sql_class.h:
  Auto merged
sql/sql_insert.cc:
  SCCS merged
tests/mysql_client_test.c:
  SCCS merged

into  gleb.loc:/home/uchum/work/bk/5.0-opt


sql/set_var.cc:
  Auto merged
sql/sql_class.h:
  Auto merged

into  gleb.loc:/home/uchum/work/bk/5.1-opt


mysql-test/r/func_gconcat.result:
  Auto merged
mysql-test/t/func_gconcat.test:
  Auto merged
sql/item_sum.cc:
  Auto merged

The Item_date_typecast::val_int function doesn't reset null_value flag.
This makes all values that follows the first null value to be treated as nulls
and led to a wrong result.

Now the Item_date_typecast::val_int function correctly sets the null_value flag
for both null and non-null values.


mysql-test/t/cast.test:
  Added a test case for the bug#29898:  Item_date_typecast::val_int doesn't reset
  the null_value flag.
mysql-test/r/cast.result:
  Added a test case for the bug#29898:  Item_date_typecast::val_int doesn't reset
  the null_value flag.
sql/item_timefunc.cc:
  Bug#29898: Item_date_typecast::val_int doesn't reset the null_value flag.
  Now the Item_date_typecast::val_int function correctly sets the null_value flag
  for both null and non-null values.

tests/mysql_client_test.c:
  rc assignement added

into  moonbone.local:/mnt/gentoo64/work/29850-bug-5.0-opt-mysql

crash" in 5.1


sql/sql_insert.cc:
  Additional safety fix: do not assume we already have a share in 
  get_local_table.

into  bodhi.(none):/opt/local/work/mysql-5.1-runtime


sql/sql_insert.cc:
  Manual merge.

a temporary table.

The result string of the Item_func_group_concat wasn't initialized in the 
copying constructor of the Item_func_group_concat class. This led to a
wrong charset of GROUP_CONCAT result when the select employs a temporary
table.

The copying constructor of the Item_func_group_concat class now correctly
initializes the charset of the result string.


mysql-test/t/func_gconcat.test:
  Added a test case for the bug#29850: Wrong charset of the GROUP_CONCAT result
  when the select employs a temporary table.
mysql-test/r/func_gconcat.result:
  Added a test case for the bug#29850: Wrong charset of the GROUP_CONCAT result
  when the select employs a temporary table.
sql/item_sum.cc:
  Bug#29850: Wrong charset of GROUP_CONCAT result when the select employs
  a temporary table.
  The copying constructor of the Item_func_group_concat class now correctly
  initializes the charset of the result string.

into  mysql.com:/home/hf/work/29687/my51-29687

into  mysql.com:/home/hf/work/29687/my51-29687

into  mysql.com:/home/hf/work/29687/my50-29687

tests/mysql_client_test.c:
  test case added

into  mysql.com:/home/hf/work/29687/my50-29687

into  mysql.com:/home/hf/work/29687/my51-29687


libmysql/libmysql.c:
  Auto merged

into  bk-internal.mysql.com:/data0/bk/tsmith-tmp/mysql-5.1-maint


sql/mysql_priv.h:
  Auto merged

into  bk-internal.mysql.com:/data0/bk/tsmith-tmp/mysql-5.0-maint

into  bodhi.(none):/opt/local/work/mysql-5.1-runtime


mysql-test/r/sp-prelocking.result:
  Auto merged
mysql-test/r/trigger.result:
  Auto merged
mysql-test/t/create.test:
  Auto merged
mysql-test/t/innodb.test:
  Auto merged
mysql-test/t/sp-prelocking.test:
  Auto merged
mysql-test/t/trigger.test:
  Auto merged
sql/sql_class.h:
  Auto merged
mysql-test/r/create.result:
  Use local.
mysql-test/r/innodb.result:
  Manual merge.

for consistency.

No test case, since the bug requires a stress case with 30 INSERT DELAYED
threads and 1 killer thread to repeat. The patch is verified
manually.
Review fixes.

The server that is running DELAYED inserts would deadlock itself
or crash under high load if some of the delayed threads were KILLed
in the meanwhile.

The fix is to change internal lock acquisition order of delayed inserts
subsystem and to ensure that
Delayed_insert::table_list::db does not point to volatile memory in some 
cases.
For details, please see a comment for sql_insert.cc.


sql/sql_insert.cc:
  A fix for Bug#29431 killing an insert delayed thread causes crash
  
  1) The deadlock was caused by different lock acquisition order
  between delayed_get_table and handle_delayed_insert.
  
  delayed_get_table would:
  - acquire LOCK_delayed_create
  - create a new Delayed_insert instance
  - acquire instance mutex (di->mutex)
  - "lock the instance in memory" by 
  increasing di->locked_in_memory variable
  - start the delayed thread
  - release di->mutex
  - let the delayed thread open the delayed table
  - discover that the delayed thread was killed
  - try to unlock() the delayed insert instance in memory
  - in Delayed_insert::unlock() do
   * lock LOCK_delayed_insert
   * decrease locks_in_memory and discover it's 0
   * attempt to lock di->mutex to broadcast di->cond condition <-- deadlock
   here
  
  Meanwhile, the delayed thread would
   * lock di->mutex
   * open the table
   * get killed
   * not notice that and attempt to lock LOCK_delayed_insert
  to register itself in the delayed insert list <-- deadlock here.
  
  Simply put, delayed_get_table used to lock LOCK_delayed_insert and then 
  di->mutex, and handle_delayed_insert would lock di->mutex and then 
  LOCK_delayed_insert.
  
  Fixed by moving registration in the list of delayed insert threads from 
  handle_delayed_insert to delayed_get_table - so that now
  handle_delayed_insert doesn't need to acquire LOCK_delayed_insert mutex.
  
  
  2) di->table_list.db was copied by-pointer from table_list.db of the first
  producer -- the one who initiated creation of the delayed insert thread.
  This producer might be long gone when the member is needed
  (handle_delayed_insert:open_ltable),
  e.g. by having been killed with KILL CONNECTION statement.
  Fixed by using a pointer to the consumer's deep copy of THD::db.
  
  3) In find_handler, we shouldn't assume that Delayed_insert object
  already (or still) has a table opened all the time it is
  present in the delayed insert list. E.g. it's not the case
  when Delayed_insert decided to terminate, closed the table, but haven't
  yet unregistered from the list (see the end of handle_delayed_insert).

into  gleb.loc:/home/uchum/work/bk/5.1-opt


sql/sql_select.cc:
  Auto merged
storage/myisam/ha_myisam.cc:
  Auto merged

into  gleb.loc:/home/uchum/work/bk/5.1-opt


client/mysqldump.c:
  Auto merged
include/my_base.h:
  Auto merged
mysql-test/include/mix1.inc:
  Auto merged
mysql-test/r/innodb_mysql.result:
  Auto merged
mysql-test/t/disabled.def:
  Auto merged
sql/sql_class.h:
  Auto merged
sql/sql_insert.cc:
  Auto merged
sql/sql_select.cc:
  Auto merged
sql/sql_table.cc:
  Auto merged
sql/sql_update.cc:
  Auto merged
storage/myisam/ha_myisam.cc:
  Auto merged

into  gleb.loc:/home/uchum/work/bk/5.0-opt


sql/ha_myisam.cc:
  Auto merged
sql/sql_select.cc:
  Auto merged

  Post-merge fix.


mysql-test/r/mysqldump.result:
  Post-merge fix.

into  gleb.loc:/home/uchum/work/bk/5.1-opt


client/mysqldump.c:
  Auto merged
mysql-test/t/mysqldump.test:
  Auto merged
mysql-test/r/mysqldump.result:
  Merge with 5.0-opt.

into  gleb.loc:/home/uchum/work/bk/5.0-opt

into  ramayana.hindu.god:/home/tsmith/m/bk/maint/51

  Don't run test with embedded server; requires external client connections


mysql-test/t/ddl_i18n_koi8r.test:
  Don't run test with embedded server; requires external client connections
mysql-test/t/ddl_i18n_utf8.test:
  Don't run test with embedded server; requires external client connections