rb://914
approved by: Marko Makela

Poll in fil_rename_tablespace() after setting ::stop_ios flag can
result in a hang because the other thread actually dispatching the IO
won't wake IO helper threads or flush the tablespace before starting
wait in fil_mutex_enter_and_prepare_for_io().

No commit message

Fix for #36428/#38364 backported into 5.0.

ERROR LOG

Fixed error meesage

+get_tty_password  this is the only external symbol in get_password.c,
                   which is explicitly listed in CLIENT_SOURCES
+handle_options    this is in mysys/my_getopt.c
                   adding this sysmbol pulls in the other externals:
T getopt_compare_strings
T getopt_double_limit_value
T getopt_ll_limit_value
T getopt_ull_limit_value
T handle_options
T my_cleanup_options
T my_getopt_register_get_addr
T my_print_help
T my_print_variables

asked by Calvin

storage/innobase/ibuf/ibuf0ibuf.c:
  Cast to avoid gcc warning

This fix does not remove the underlying cause of the assertion
failure. It just works around the problem, allowing a corrupted
secondary index to be fixed by DROP INDEX and CREATE INDEX (or in the
worst case, by re-creating the table).

ibuf_delete(): If the record to be purged is the last one in the page
or it is not delete-marked, refuse to purge it. Instead, write an
error message to the error log and let a debug assertion fail.

ibuf_set_del_mark(): If the record to be delete-marked is not found,
display some more information in the error log and let a debug
assertion fail.

row_undo_mod_del_unmark_sec_and_undo_update(),
row_upd_sec_index_entry(): Let a debug assertion fail when the record
to be delete-marked is not found.

buf_page_print(): Add ut_ad(0) so that corruption will be more
prominent in stress testing with debug binaries. Add ut_ad(0) here and
there where corruption is noticed.

btr_corruption_report(): Display some data on page_is_comp() mismatch.

btr_assert_not_corrupted(): A wrapper around btr_corruption_report().
Assert that page_is_comp() agrees with the table flags.

rb:911 approved by Inaam Rana

BUG#13519696 - 62940: SELECT RESULTS VARY WITH VERSION AND
WITH/WITHOUT INDEX RANGE SCAN
BUG#13453382 - REGRESSION SINCE 5.1.39, RANGE OPTIMIZER WRONG
RESULTS WITH DECIMAL CONVERSION
BUG#13463488 - 63437: CHAR & BETWEEN WITH INDEX RETURNS WRONG
RESULT AFTER MYSQL 5.1.
Those are all cases where the range optimizer got it wrong
with > and >=.

mysql-test/r/range.result:
  Without the code fix for DECIMAL, "select count(val) from t2 where val > 0.1155"
  (which uses a range scan) returned 127 instead of 128);
  Moreover, both
  select * from t1 force  index (primary) where a=1 and c>= 2.9;
  and
  select * from t1 force  index (primary) where a=1 and c> 2.9;
  would miss "1	1	3".
  Without the code fix for strings, both
  SELECT * FROM t1 WHERE F1 >= 'A    ';
  and
  SELECT * FROM t1 WHERE F1 BETWEEN 'A    ' AND 'AAAAA';
  would miss "A	A	A".
sql/item.cc:
  Preamble to the explanations below: opt_range.cc:get_mm_leaf() does
  this (this is not changed by the patch): changes
  column > value
  to
  column OP V
  where:
  * V is what is in "column" after we stored "value" in it
  (such store operation may have done rounding...)
  * OP is > or >=, depending on what's correct.
  For example, if c is an INT column,
  c > 2.9 is changed to
  c OP 3
  where OP is >= ('>' would not be correct).
  The bugs below are cases where we chose OP wrongly.
  Note that such transformations are visible in the optimizer trace.
  
  1) Fix for STRING. In the scenario with CHAR(5) in range.test, this happens,
  in get_mm_tree(), for the condition F1>='A    ':
  * value->save_in_field_no_warnings(field, 1) wants to store the right argument
  (named 'item') into the CHAR(5) field; this stores 'A    ' (the item's value)
  padded with spaces (which changes nothing: still 'A    ')
  * we come to
    case Item_func::GE_FUNC:
      /* Don't use open ranges for partial key_segments */
      if ((!(key_part->flag & HA_PART_KEY_SEG)) &&
          (stored_field_cmp_to_item(param->thd, field, value) < 0))
        tree->min_flag= NEAR_MIN;
      tree->max_flag=NO_MAX_RANGE;
  What this wants to do is: if the field's value is strictly smaller
  than the item's, then ">=" can be changed to ">" (this is an optimization,
  it can help pruning one useless partition).
  * stored_field_cmp_to_item() is called; it compares the field's
  and item's values: the item's value (Item_string::val_str()) is
  'A    ') and the field's value (Field_string::val_str()) is
  'A' (yes val_str() removes end spaces unless sql_mode='PAD_CHAR_TO_FULL_LENGTH');
  and the comparison is done with stringcmp() which considers
  end spaces as relevant; as end spaces differ, function returns a
  negative number, and ">='A    '" becomes ">'A'" (i.e. the NEAR_MIN
  flag is turned on).
  During execution the index range scan code will search for "A", find
  a match, but exclude it (because of ">"), wrongly.
  The badness is the string comparison done by stored_field_cmp_to_item():
  we use the reply of this function to determine where the index search
  should start, so it should do comparison like index search does
  comparisons; index search comparisons are ha_key_cmp() which uses
  a collation-aware comparison (in our case, my_strnncollsp_simple(),
  which ignores end spaces); so stored_field_cmp_to_item()
  needs to do the same. When this is fixed, condition becomes
  ">='A    '".
  
  2) Fix for DECIMAL: just like in other comparisons in stored_field_cmp_to_item(),
  we must first pass the field and then the item; otherwise expectations
  on what <0 and >0 mean (inferiority, superiority) get violated.
  In the test in range.test about c>2.9: c is an INT column, so 2.9
  gets stored as 3, then stored_field_cmp_to_item() compares 3
  and 2.9; because of the wrong order of arguments passed
  to my_decimal_cmp(), range optimizer
  thinks that 3 is < 2.9 and thus changes "c> 2.9" to "c> 3".
  After fixing the order, it changes to the correct "c>= 3".
  In the test in range.inc for val > 0.1155, it was changed to
  val > 0.116, now it is changed to val >= 0.116.

rpl_heartbeat_basic test fails sporadically on pushbuild because did
not received all heartbeats from slave in circular replication.

MASTER_HEARTBEAT_PERIOD had the default value (slave_net_timeout/2) so
wait on "Heartbeat event received on master", that only waits for 1
minute, sometimes timeout before heartbeat arrives. Fixed setting a
smaller period value.

Bug#12985021 SIMPLE QUERY WITH DECIMAL NUMBERS TAKE AN

When parsing the fractional part of a string which
is to be converted to double, we can stop after a few digits:
the extra digits will not contribute to the actual result anyways.


mysql-test/r/func_str.result:
  New tests.
mysql-test/t/func_str.test:
  New tests.
strings/dtoa.c:
  The problem was s2b() multiplying and adding hundreds-of-thousands
  of ever smaller fractions.

Bug#11758543 50756: BIGINT '100' MATCHES 1.001E2

Expressions of the form
      BIGINT_COL <compare> <non-integer constant>

      should be done either as decimal, or float.

      Currently however, such comparisons are done as int,
      which means that the constant may be truncated,
      and yield false positives/negatives for all queries
      where compare is '>' '<' '>=' '<=' '=' '!='.

      BIGINT_COL IN <list of contstants>
      and
      BIGINT_COL BETWEEN <constant> AND <constant>
      are also affected.



mysql-test/r/bigint.result:
  New tests.
mysql-test/r/func_in.result:
  BIGINT <=> string comparison should be done as float,
  so a warning for the value 'abc' is appropriate.
mysql-test/t/bigint.test:
  New tests.
sql/item_cmpfunc.cc:
  In convert_constant_item() we verify that the constant item
  can be stored in the given field.
  For BIGINT columns (MYSQL_TYPE_LONGLONG) we must verify that the
  stored constant value is actually comparable as int,
  i.e. that the value was not truncated.
  
  For between: compare as int only if both arguments convert correctly to int.

MEMORY LEAK.

Background:
 - There are caches for stored functions and stored procedures (SP-cache);
 - There is no similar cache for events;
 - Triggers are cached together with TABLE objects;
 - Those SP-caches are per-session (i.e. specific to each session);
 - A stored routine is represented by a sp_head-instance internally;
 - SP-cache basically contains sp_head-objects of stored routines, which
   have been executed in a session;
 - sp_head-object is added into the SP-cache before the corresponding
   stored routine is executed;
 - SP-cache is flushed in the end of the session.

The problem was that SP-cache might grow without any limit. Although this
was not a pure memory leak (the SP-cache is flushed when session is closed),
this is still a problem, because the user might take much memory by
executing many stored routines.

The patch fixes this problem in the least-intrusive way. A soft limit
(similar to the size of table definition cache) is introduced. To represent
such limit the new runtime configuration parameter 'stored_program_cache'
is introduced. The value of this parameter is stored in the new global
variable stored_program_cache_size that used to control the size of SP-cache
to overflow. 

The parameter 'stored_program_cache' limits number of cached routines for
each thread. It has the following min/default/max values given from support:
  min = 256, default = 256, max = 512 * 1024.
Also it should be noted that this parameter limits the size of 
each cache (for stored procedures and for stored functions) separately.

The SP-cache size is checked after top-level statement is parsed.
If SP-cache size exceeds the limit specified by parameter
'stored_program_cache' then SP-cache is flushed and memory allocated for
cache objects is freed. Such approach allows to flush cache safely 
when there are dependencies among stored routines.


sql/mysqld.cc:
  Added global variable stored_program_cache_size to store value of
  configuration parameter 'stored-program-cache'.
sql/mysqld.h:
  Added declaration of global variable stored_program_cache_size.
sql/sp_cache.cc:
  Extended interface for sp_cache by adding helper routine
  sp_cache_enforce_limit to control size of stored routines cache for
  overflow. Also added method enforce_limit into class sp_cache that
  implements control of cache size for overflow.
sql/sp_cache.h:
  Extended interface for sp_cache by adding standalone routine
  sp_cache_enforce_limit to control size of stored routines cache
  for overflow.
sql/sql_parse.cc:
  Added flush of sp_cache after processing of next sql-statement
  received from a client.
sql/sql_prepare.cc:
  Added flush of sp_cache after preparation/execution of next prepared
  sql-statement received from a client.
sql/sys_vars.cc:
  Added support for configuration parameter stored-program-cache.

and some Valgrind instrumentation.

This patch changes the mechanism by which the client enables a 
plugin. Instead of using INSERT IGNORE to reload a plugin library,
it now uses REPLACE INTO. This allows users to load a library
multiple times replacing the existing values in the mysql.plugin
table. This allows users to replace the symbol reference to a
different dl name in the table. Thus permitting enabling of 
multiple versions of the same library without first disabling
the old version.

A regression test was added to ensure this feature works.

- Reverting the patch for Bug # 12584302
  The patch will be reverted in 5.1 and 5.5.
  The patch will not be reverted in 5.6, the change will
  be properly documented in 5.6.
- Backporting DBUG_ASSERT not to crash on '0000-01-00'
  (already fixed in mysql-trunk (5.6))

The wait_for_ndb_to_binlog.inc include file used by the blow rpl_tests
common for rpl and rpl_ndb suite is simply doing a "sleep 5", this is
not deterministic and wastes lot of test time uneccessarily. The test
should be rewritten to check if the condition it wait for has been
reached or not.

For NDB engine all events will be added by NDB injector so tests only 
can continue after injector is ready, this test waits for proper
injector thread state.

Problem : The basic problem is the way the thread sleeps in mysql-5.5 and also in mysql-5.1
          when we execute a stop slave on windows platform.
          On windows platform if the stop slave is executed after the master dies, we have 
          this long wait before the stop slave return a value. This is because there is a 
          sleep of the thread. The sleep is uninterruptable in the two above version,
          which was fixed by Davi patch for the BUG#11765860 for mysql-trunk. Backporting 
          his patch for mysql-5.5 fixes the problem. 

Solution : A new pair of mutex and condition variable is introduced to synchronize thread 
           sleep and finalization. A new mutex is required because the slave threads are 
           terminated while holding the slave thread locks (run_lock), which can not be 
           relinquished during termination as this would affect the lock order.

mysql-test/suite/rpl/r/rpl_start_stop_slave.result:
  The result file associated with the test added.
mysql-test/suite/rpl/t/rpl_start_stop_slave.test:
  A test to check the new functionality.
sql/rpl_mi.cc:
  The constructor using the new mutex and condition variables for the master_info.
sql/rpl_mi.h:
  The condition variable and mutex have been added for the master_info.
sql/rpl_rli.cc:
  The constructor using the new mutex and condition variables for the realy_log_info.
sql/rpl_rli.h:
  The condition variable and mutex have been added for the relay_log_info.
sql/slave.cc:
  Use a timed wait on a condition variable to implement a interruptible sleep. 
  The wait is registered with the THD object so that the thread will be woken 
  up if killed.

Introducing new collations:
utf8_general_mysql500_ci and ucs2_general_mysql500_ci,
to reproduce behaviour of utf8_general_ci and ucs2_general_ci
from mysql-5.1.23 (and earlier).

The collations are added to simplify upgrade from mysql-5.1.23 and earlier.

Note: The patch does not make new server start over old data automatically.
Some manual upgrade procedures are assumed.

Paul: please get in touch with me to discuss upgrade procedures
when documenting this bug.

modified:
  include/m_ctype.h
  mysql-test/r/ctype_utf8.result
  mysql-test/t/ctype_utf8.test
  mysys/charset-def.c
  strings/ctype-ucs2.c
  strings/ctype-utf8.c

Fix valgrind warning introduced by fix for bug 11765450.

CREATES A FILE IN AN IMPROPER LOCATION.

Fixed by using $MYSQLTEST_VARDIR, as proposed by
Davi Arnaut.

Thank you Davi!

- Fixed the checks to properly check for plugin_dir containing a trailing slash or backslash.
- Fixed a under-configuration in udf_skip_grants that was preventing the test 
  from running even when there was a udf plugin.

The issue is that xa.test failed sporadically on some platforms.
The reason for the test failure is a race condition in xa.test.
The race condition occures between connection that executes statement
INSERT INTO t2 SELECT FROM t1 and other connection that tries to run
statements DELETE FROM t1 and COMMIT. If COMMIT statement had been executed
before the statement INSERT INTO t2 SELECT FROM t1 was locked by lock
on table t1 (as a result of query from table t1) then the INSERT statement
is executed successfully and a following test for deadlock would failed.

This patch fixes this race condition by moving COMMIT statement after commit
of distributed transaction from concurrent session.

If an error message contains '\' backslash it is displayed correctly
through show-slave-status or
query_get_value(SHOW SLAVE STATUS, Last_IO_Error, 1);. But when
SELECT REPLACE(...) is applied backslash is escaped resulting in a
different test output.

Disabled backslash escape on show_slave_status.inc and replaced '\' for
'/' using replace_regex function in order to achieve the same test
output when different path separators are used.