This fixes a bug that gave ER_FILE_TOO_SHORT error when scanning Aria tables.

storage/maria/ma_blockrec.c:
  Changed code so that we update share->state.state.data_file_length after page cache write is done.
  When scanning pages, stop when you are at 'data_file_length'.
  (We can't trust the bitmap as there may be reserved pages in the bitmap that are not yet written).
  This fixes a bug that gave ER_FILE_TOO_SHORT error when scanning Aria tables.
storage/maria/maria_def.h:
  Updated struct for stopping scanning at end of file

Implement os_event_wait_time() for POSIX systems.

In the purge thread, use os_event_wait_time() when sleeping rather than sleep,
and signal the event when server shuts down, so we do not need to wait for
upto 10 seconds until the purge thread wakes up.

Also fix bug that warnings that were pushed after we call set_ok_status() were
not included in the waning count sent to the client in the result packet.

Also in mysqltest, in recursive die() invocation at least print the message
before aborting.

client/mysqltest.cc:
  If we detect recursive die(), at least print the message before aborting.
mysql-test/r/warnings_debug.result:
  Test case.
mysql-test/t/warnings_debug.test:
  Test case.
sql/handler.cc:
  Force generation of a warning with specific debug option, for testing.
sql/protocol.cc:
  Fix wrong DBUG_ENTER
sql/sql_class.h:
  Add method to count warnings pushed after set_ok_status() is called.
sql/sql_error.cc:
  Also count warnings pushed after set_ok_status() is called.
storage/xtradb/include/os0sync.h:
  Implement working os_sync_wait_time() for POSIX.
storage/xtradb/include/srv0srv.h:
  Make the purge thread wait for an event when sleeping, so we can signal it to
  wakeup immediately at shutdown.
storage/xtradb/log/log0log.c:
  Make the purge thread wait for an event when sleeping, so we can signal it to
  wakeup immediately at shutdown.
storage/xtradb/os/os0sync.c:
  Implement working os_sync_wait_time() for POSIX.
storage/xtradb/srv/srv0srv.c:
  Make the purge thread wait for an event when sleeping, so we can signal it to
  wakeup immediately at shutdown.

Fixed a bug in Aria when two threads was inserting into the same table and row page and one thread did an abort becasue of duplicate key.



mysys/thr_lock.c:
  Fixed valgrind warning
sql/sql_base.cc:
  Remove not used variable
storage/maria/ma_bitmap.c:
  Added ma_bitmap_lock() & ma_bitmap_unlock() to protect against two threads using the bitmap at the same time.
  More DBUG_PRINT()
storage/maria/ma_blockrec.c:
  Fixed a bug in Aria when two threads was inserting into the same table and row page and one thread did an abort becasue of duplicate key.
  Fix was that we block other threads to modify the bitmap while we are removing the row with a duplicate key.
storage/maria/ma_blockrec.h:
  Added ma_bitmap_lock() & ma_bitmap_unlock() to protect against two threads using the bitmap at the same time.
storage/maria/maria_def.h:
  Changed flush_all_requested to be a counter.
storage/myisam/mi_locking.c:
  Fixed compiler error on windows (typo).

This bug in the MRR code for Maria engine caused wrong results when
MRR was used to scan ranges for each record.

Added test cases for bugs 669420 and 669423 (as a duplicate of 669420).

Added locking of lock mutex when updating status in external_unlock() for Aria and MyISAM tables.
Fixed that 'source' command doesn't cause mysql command line tool to exit on error.
DEBUG_EXECUTE() and DEBUG_EVALUATE_IF() should not execute things based on wildcards. (Allows one to run --debug with mysql-test-run scripts that uses @debug)
Fixed several core dump, deadlock and crashed table bugs in handling of LOCK TABLE with MERGE tables:
- Added priority of locks to avoid crashes with MERGE tables.
- Added thr_lock_merge() to allow one to merge two results of thr_lock().
Fixed 'not found row' bug in REPLACE with Aria tables.
Mark MyISAM tables that are part of MERGE with HA_OPEN_MERGE_TABLE and set the locks to have priority THR_LOCK_MERGE_PRIV.
- By sorting MERGE tables last in thr_multi_unlock() it's safer to release and relock them many times (can happen when TRIGGERS are created)
Avoid printing (null) in debug file (to easier find out wrong NULL pointer usage with %s).



client/mysql.cc:
  Fixed that 'source' command doesn't cause mysql command line tool to exit on error.
client/mysqltest.cc:
  Don't send NULL to fn_format(). (Can cause crash on Solaris when using --debug)
dbug/dbug.c:
  DEBUG_EXECUTE() and DEBUG_EVALUATE_IF() should not execute things based on wildcards.
include/my_base.h:
  Added flag to signal if one opens a MERGE table.
  Added extra() command to signal that one is not part of a MERGE table anymore.
include/thr_lock.h:
  Added priority for locks (needed to fix bug in thr_lock when using MERGE tables)
  Added option to thr_unlock() if get_status() should be called.
  Added prototype for thr_merge_locks().
mysql-test/mysql-test-run.pl:
  Ignore crashed table warnings for tables named 'crashed'.
mysql-test/r/merge.result:
  Renamed triggers to make debugging easier.
  Added some CHECK TABLES to catch errors earlier.
  Additional tests.
mysql-test/r/merge_debug.result:
  Test of error handling when reopening MERGE tables.
mysql-test/r/udf_query_cache.result:
  Added missing flush status
mysql-test/suite/parts/r/partition_repair_myisam.result:
  Update results
mysql-test/t/merge.test:
  Renamed triggers to make debugging easier.
  Added some CHECK TABLES to catch errors earlier.
  Additional tests.
mysql-test/t/merge_debug.test:
  Test of error handling when reopening MERGE tables.
mysql-test/t/udf_query_cache.test:
  Added missing flush status
mysys/my_getopt.c:
  Removed not used variable
mysys/my_symlink2.c:
  Changed (null) to (NULL) to make it easier to find NULL arguments to DBUG_PRINT() functions.
  (On linux, NULL to sprintf is printed 'null')
mysys/thr_lock.c:
  Added priority of locks to avoid crashes with MERGE tables.
  Added thr_lock_merge() to allow one to merge two results of thr_lock().
  - This is needed for MyISAM as all locked table must share the same status. If not, you will not see newly inserted rows in other instances of the table.
  If calling thr_unlock() with THR_UNLOCK_UPDATE_STATUS, call update_status() and restore_status() for the locks. This is needed in some rare cases where we call thr_unlock() followed by thr_lock() without calling external_unlock/external_lock in between.
  Simplify loop in thr_multi_lock().
  Added 'start_trans', which is called at end of thr_multi_lock() when all locks are taken.
  - This was needed by Aria to ensure that transaction is started when we got all locks, not at get_status(). Without this, some rows could not be visible when we lock two tables at the same time, causing REPLACE using two tables to fail unexpectedly.
sql/handler.cc:
  Add an assert() in handler::print_error() for "impossible errors" (like table is crashed) when --debug-assert-if-crashed-table is used.
sql/lock.cc:
  Simplify mysql_lock_tables() code if get_lock_data() returns 0 locks.
  Added new parameter to thr_multi_unlock()
  In mysql_unlock_read_tables(), call first externa_unlock(), then thr_multi_unlock();  This is same order as we do in mysql_unlock_tables().
  Don't abort locks in mysql_lock_abort() for merged tables when a MERGE table is deleted; Would cause a spin lock.
  Added call to thr_merge_locks() in mysql_lock_merge() to ensure consistency in thr_locks().
  - New locks of same type and table is stored after the old lock to ensure that we get the status from the original lock.
sql/mysql_priv.h:
  Added debug_assert_if_crashed_table
sql/mysqld.cc:
  Added --debug-assert-if-crashed-table
sql/parse_file.cc:
  Don't print '(null)' in DBUG_PRINT of no dir given
sql/set_var.cc:
  Increase default size of buffer for @debug variable.
sql/sql_base.cc:
  In case of error from reopen_table() in reopen_tables(), call unlock_open_table() and restart loop.
  - This fixed bug when we twice deleted same table from open_cache.
  Don't take name lock for already name locked table in open_unireg_entry().
  - Fixed bug when doing repair in reopen_table().
  - In detach_merge_children(), always detach if 'clear_refs' is given. We can't trust parent->children_attached as this function can be called twice, first time with clear_refs set to 0.
sql/sql_class.cc:
  Changed printing of (null) to "" in set_thd_proc_info()
sql/sql_parse.cc:
  Added DBUG
sql/sql_trigger.cc:
  Don't call unlink_open_table() if reopen_table() fails as the table may already be freed.
storage/maria/ma_bitmap.c:
  Fixed DBUG_ASSERT() in allocate_tail()
storage/maria/ma_blockrec.c:
  Fixed wrong calculation of row length for very small rows in undo_row_update().
  - Fixes ASSERT() when doing undo.
storage/maria/ma_blockrec.h:
  Added _ma_block_start_trans() and _ma_block_start_trans_no_versioning()
storage/maria/ma_locking.c:
  Call _ma_update_status_with_lock() when releasing write locks.
  - Fixes potential problem with updating status without the proper lock.
storage/maria/ma_open.c:
  Changed to use start_trans() instead of get_status() to ensure that we see all rows in all locked tables when we got the locks.
  - Fixed 'not found row' bug in REPLACE with Aria tables.
storage/maria/ma_state.c:
  Added _ma_update_status_with_lock() and _ma_block_start_trans().
  This is to ensure that we see all rows in all locked tables when we got the locks.
storage/maria/ma_state.h:
  Added _ma_update_status_with_lock()
storage/maria/ma_write.c:
  More DBUG_PRINT
storage/myisam/mi_check.c:
  Fixed error message
storage/myisam/mi_extra.c:
  Added HA_EXTRA_DETACH_CHILD:
  - Detach MyISAM table to not be part of MERGE table (remove flag & lock priority).
storage/myisam/mi_locking.c:
  Call mi_update_status_with_lock() when releasing write locks.
  - Fixes potential problem with updating status without the proper lock.
  Change to use new HA_OPEN_MERGE_TABLE flag to test if MERGE table.
  Added mi_fix_status(), called by thr_merge().
storage/myisam/mi_open.c:
  Added marker if part of MERGE table.
  Call mi_fix_status() in thr_lock() for transactional tables.
storage/myisam/myisamdef.h:
  Change my_once_flag to uint, as it stored different values than just 0/1
  Added 'open_flag' to store state given to mi_open()
storage/myisammrg/ha_myisammrg.cc:
  Add THR_LOCK_MERGE_PRIV to THR_LOCK_DATA to get MERGE locks sorted after other types of locks.
storage/myisammrg/myrg_locking.c:
  Remove windows specific code.
storage/myisammrg/myrg_open.c:
  Use HA_OPEN_MERGE_TABLE to mi_open().
  Set HA_OPEN_MERGE_TABLE for linked MyISAM tables.
storage/xtradb/buf/buf0buf.c:
  Fixed compiler warning
storage/xtradb/buf/buf0lru.c:
  Initialize variable that could be used not initialized.

Fixed compiler warnings.

include/queues.h:
  Added prototype for queue_replace()
mysys/queues.c:
  Fixed wrong queue_replace()
mysys/thr_alarm.c:
  Added DBUG_PRINT
sql/item_subselect.cc:
  Check return value of ha_rnd_init().
  (Fixes compiler warnings)
sql/sql_class.cc:
  Fixed wrong test
sql/sql_show.cc:
  Removed not used variable.

don't set view_prepare_mode when opening a base table
(either in SHOW CREATE or in I_S.TABLES)

The bug could cause wrong results for queries over Maria tables when
index condition pushdown was used.

Also fix missing zero termination in DBUG_PRINT, causing garbage output
in --debug output.

The set of Ordered keys of a rowid merge engine is dense. Thus when
we decide not to create a key for a column that has only NULLs, this
column shouldn't be counted.

Notice that the caller has already precomputed the correct total
number of keys that should be created.

Post-review fix - avoid re-evaluation of the having clause
when it evaluates to true.

The main.ps_ddl test does SELECT * FROM mysql.general_log; that can be really
expensive with --valgrind if previous test cases put lots of data in the
general log since last server restart. Fix by truncating the log at test start.

sql/sql_expression_cache.cc:
  Type of the variable fixed.
sql/sql_expression_cache.h:
  Type of the variable fixed.

The cause for this bug is that MariaDB 5.3 still processes derived tables
(subqueries in the FROM clause) by fully executing them during the parse
phase. This will be remedied by MWL#106 once merged into the main 5.3.

The assert statement is triggered when MATERIALIZATION is ON for EXPLAIN
EXTENDED for derived tables with an IN subquery as follows:
- mysql_parse calls JOIN::exec for the derived table as if it is regular
  execution (not explain).
- When materialization is ON, this call goes all the way to
  subselect_hash_sj_engine::exec, which creates a partial match engine
  because of NULL presence.
- In order to proceed with normal execution, the hash_sj engine substitutes
  itself with the created partial match engine.
- After the parse phase it turns out that this execution was part of
  EXPLAIN EXTENDED, which in turn calls
  Item_cond::print -> ... -> Item_subselect::print,
  which calls engine->print().
  Since subselect_hash_sj_engine::exec substituted the current
  Item_subselect engine with a  partial match engine, eventually we call
  its ::print() method. However the partial match engines are designed only
  for execution, hence there is no implementation of this print() method.

The fix temporarily removes the assert, until this code is merged with
MWL#106.

The bug was a result of missing logic to handle the case
when there are 'expensive' predicates that are not evaluated
during constant table optimization. Such is the case for
the IN predicate, which is considered expensive if it is
computed via materialization. In general this bug can be
triggered with any expensive predicate instead of IN.

When FALSE constant predicates are not evaluated during constant
optimization, the execution path changes so that instead of
setting JOIN::zero_result_cause after make_join_select, and
exiting JOIN::exec via the call to return_zero_rows(), execution
ends in JOIN::exec in the branch:
if (join->tables == join->const_tables)
{
  ...
  else if (join->send_row_on_empty_set())
     ...
     rc= join->result->send_data(*columns_list);
}
Unlike return_zero_rows(), this branch didn't evaluate the
having clause of the query.

The patch adds a call to evaluate the HAVING clause of a query even
when all tables are constant, because even for an empty result set
some aggregate functions may produce a NULL value.

bugfix: engine defined table options were not showing up in INFORMATION_SCHEMA.TABLES.CREATE_OPTIONS

BUG#26447 prefer a clustered key for an index scan, as secondary index is always slower
  ... which was fixed to cause
BUG#35850 queries take 50% longer
  ... and was reverted

and

BUG#39653 prefer a secondary index for an index scan, as clustered key is always slower
  ... which was fixed to cause
BUG#55656 mysqldump takes six days instead of half an hour
  ... and was amended with a special case workaround


sql/opt_range.cc:
  move get_index_only_read_time() into the handler class
sql/sql_select.cc:
  use cost not an index length when choosing a cheaper index