Fix for LP#697610 ha_index_prev(uchar*): Assertion `inited==INDEX' failed with HANDLER + InnoDB in maria-5.3


mysql-test/suite/handler/innodb.result:
  Added test case
mysql-test/suite/handler/innodb.test:
  Added test case
sql/handler.h:
  Move setting/resetting of active_index to ha_index_init()/ha_index_end() to simplify handler functions index_init()/index_end()
  Fixed that get_index() returns MAX_KEY if index is not inited (this fixed LP#697610)
storage/federated/ha_federated.cc:
  Settting of active_index is not needed anymore
storage/maria/ma_pagecache.c:
  Added error message if we have too little memory for Maria page cache

storage/maria/ma_pagecache.c:
  - Ensure that we write also the last buffer in flush if buffer overflows.
  - Don't set PCBLOCK_IN_FLUSH if we don't flush the page
  - Added some new safety asserts.

Don't allow too low value of pagecache_buffer_size for mysqld

storage/maria/ha_maria.cc:
  Don't allow too low value of pagecache_buffer_size (as it will stop mysqld from starting)
storage/maria/maria_read_log.c:
  Allow one to decrease page-buffer-size down to 1M (from 128M)

storage/maria/maria_read_log.c:
  Allow smaller size for page-buffer-size (down to 1M)
  Sorted options for --help

storage/maria/ma_rsame.c:
  Fixed wrong test of index usage
storage/maria/ma_search.c:
  Fixed test to avoid compiler warnings.
  Safety fix to ensure that my_error is properly set in case of errors.

- Fix for MySQL BUG#52357 added NESTED_JOIN::is_fully_covered() which would 
  not take into account that MariaDB's table elimination could eliminate tables
  from join plan (and so, from join nest).
  Fixed the check in the function to compare post-table-elimination numbers.

- Set lastinx= ~0 when last_key.keyinfo is set.


storage/maria/ma_check.c:
  Set lastinx= ~0 when last_key.keyinfo is set
storage/maria/ma_ft_boolean_search.c:
  Set lastinx= ~0 when last_key.keyinfo is set
storage/maria/ma_rt_index.c:
  Remove setting of info->lastkey.keyinfo, as this should already be set by caller
storage/maria/ma_search.c:
  Added ASSERT to ensure that info->last_key.keyinfo is properly set
storage/maria/ma_unique.c:
   Set lastinx= ~0 when last_key.keyinfo is set

- Added test case for Aria
- Tested HANDLER with HEAP (changes to HEAP code will be pushed in 5.3)
- Moved all HANDLER test to suite/handler.


mysql-test/Makefile.am:
  Added suite/handler
mysql-test/mysql-test-run.pl:
  Added suite/handler
mysql-test/r/lock_multi.result:
  Remove test that is already in handler test suite
mysql-test/suite/handler/aria.result:
  Test for HANDLER with Aria storage engine
mysql-test/suite/handler/aria.test:
  Test for HANDLER with Aria storage engine
mysql-test/suite/handler/handler.inc:
  Extended the general handler test
  Moved interface testing to 'interface.test'
mysql-test/suite/handler/init.inc:
  Common init for handler tests.
mysql-test/suite/handler/innodb.result:
  New results
mysql-test/suite/handler/innodb.test:
  Update to use new include files
mysql-test/suite/handler/interface.result:
  Test of HANDLER interface (not storage engine dependent parts)
mysql-test/suite/handler/interface.test:
  Test of HANDLER interface (not storage engine dependent parts)
mysql-test/suite/handler/myisam.result:
  New results
mysql-test/suite/handler/myisam.test:
  Update to use new include files
mysql-test/t/lock_multi.test:
  Remove test that is already in handler test suite
mysys/tree.c:
  Added missing handling of read previous (showed up in HEAP testing)
sql/handler.cc:
  Don't marka 'HA_ERR_RECORD_CHANGED' as fatal (can be used with HANDLER READ, especially with MEMORY ENGINE)
sql/handler.h:
  Added prototype for can_continue_handler_scan()
sql/sql_handler.cc:
  Re-initialize search if we switch from key to table search.
  Check if handler can continue searching between calls (via can_continue_handler_scan())
  Don't write common not fatal errors to log
storage/maria/ma_extra.c:
  Don't set index 0 as default. This forces call to ma_check_index() to set up index variables.
storage/maria/ma_ft_boolean_search.c:
  Ensure that info->last_key.keyinfo is set
storage/maria/ma_open.c:
  Don't set index 0 as default. This forces call to ma_check_index() to set up index variables.
storage/maria/ma_rkey.c:
  Trivial optimization
storage/maria/ma_rnext.c:
  Added missing code from mi_rnext.c to ensure that handler next/prev works.
storage/maria/ma_rsame.c:
  Simple optimizations
storage/maria/ma_search.c:
  Initialize info->last_key once and for all when we change keys.
storage/maria/ma_unique.c:
  Ensure that info->last_key.keyinfo is up to date.

mysql_convert_table_format ignored --engine option.
Fix that zerofill() doesn't write out wrong data to client if run with auto repair.
Ensure that pagecache is properly flushed, even in case of errors.
Handle checksum errors in BLOCK_RECORD format.


scripts/mysql_convert_table_format.sh:
  Fixed that --engine option works
storage/maria/ha_maria.cc:
  Fix that zerofill() doesn't write out wrong data to client if run with auto repair.
storage/maria/ma_check.c:
  Set in_check_table when scanning table to not get DBUG_ASSERT for checksum error.
  Ensure that pagecache is properly flushed, even in case of errors.
  Handle checksum errors in BLOCK_RECORD format.
storage/maria/ma_sort.c:
  Set in_check_table when scanning table to not get DBUG_ASSERT for checksum error.

Patch provided by Dolf Schimmel

- Don't delete pages without flushing that has had a tail or head information in pagecache_delete()
  This fixes a case where REPAIR could find old deleted rows.


storage/maria/ha_maria.cc:
  Remove calls to depricated function ha_statistic_increment
storage/maria/ma_blockrec.c:
  Don't delete pages without flushing that has had a tail or head information in pagecache_delete()
storage/maria/ma_pagecache.c:
  Added possibility to mark pages to not be deleted by pagecache_delete() without beeing flushed.
storage/maria/ma_pagecache.h:
  Added new prototype

- Fixed some bugs in recovery of blobs
- Don't ASSERT() on checksum errors when running check table
- Added to maria_read_log option --tables-to-redo=list-of-tables to only recover some tables (good for debugging)


storage/maria/ma_blockrec.c:
  Don't ASSERT() on checksum errors when running check table
  Fixed bug in recovery of blog page that was not in dirty pages list
storage/maria/ma_check.c:
  Don't ASSERT() on checksum errors when running check table
storage/maria/ma_recovery.c:
  Handling of --tables-to-redo
storage/maria/ma_recovery.h:
  Handling of --tables-to-redo
storage/maria/ma_recovery_util.c:
  Give better warning if table was not in dirty pages list
storage/maria/maria_def.h:
  Added in_check_table
storage/maria/maria_read_log.c:
  Added --tables-to-redo=list-of-tables to only recover some tables (good for debugging)
  Cleaned up message when wrong arguments

This could only happen with very small rows on very full pages with old deleted information in middle of page.

Fixes LP#695006 converting HEAP to Aria" status do not respond to KILL QUERY

sql/sql_select.cc:
  Make copy from heap to MyISAM / Aria killable.

LP#619731: Aria recovery corruption "Page 1: Row: 1 has an extent with wrong information in bitmap

storage/maria/ma_bitmap.c:
  Don't send broadcast if no one is waiting for it
storage/maria/ma_blockrec.c:
  Don't update bitmap if the page is not in the dirty_page list (or LSN is after checkpoint start)
  Fixes the case where we have in the log redo_free_block followed by another redo entry for the same page which is ignored.
  Also fixed that _ma_apply_redo_insert_row_blobs() doesn't update the bitmap in similar circumstances.
storage/maria/ma_blockrec.h:
  Updated prototype
storage/maria/ma_check.c:
  Added printing of bitmap information if used with maria_chk -vvv (for debugging)
storage/maria/ma_recovery.c:
  Updated call parameters to _ma_apply_redo_free_blocks().

The patch also fixes a race in rpl_stop_slave.test.

On machines with lots of CPU and memory, something like `mtr --parallel=10`
can speed up the test suite enormously. However, we have a few test cases
that run for long (several minutes), and if we are unlucky and happen to
schedule those towards the end of the test suite, we end up with most
workers idle while waiting for the last slow test to end, significantly
delaying the finish of the entire suite.

Improve this by marking the offending tests as taking "long", and trying
to schedule those tests early. This reduces the time towards the end of
the test suite run where some workers are waiting with nothing to do for
the remaining workers each to finish their last test.

Also, the rpl_stop_slave test had a race which could cause it to take
a 300 seconds debug_sync timeout; this is fixed.

Testing on a 4-core 8GB machine, this patch speeds up the test suite with
around 30% for --parallel=10 (debug build), allowing to run the entire
suite in 5 minutes.

 * move a capability from a virtual handler method to table_flags()
 * rephrase error messages to avoid hard-coded English parts
 * admit in test cases that they need xtradb, not innodb

mysql-test/suite/vcol/t/rpl_vcol.test:
  this test needs xtradb, it will fail with innodb
mysql-test/suite/vcol/t/vcol_blocked_sql_funcs_innodb.test:
  this test needs xtradb, it will fail with innodb
mysql-test/suite/vcol/t/vcol_column_def_options_innodb.test:
  this test needs xtradb, it will fail with innodb
mysql-test/suite/vcol/t/vcol_handler_innodb.test:
  this test needs xtradb, it will fail with innodb
mysql-test/suite/vcol/t/vcol_ins_upd_innodb.test:
  this test needs xtradb, it will fail with innodb
mysql-test/suite/vcol/t/vcol_keys_innodb.test:
  this test needs xtradb, it will fail with innodb
mysql-test/suite/vcol/t/vcol_non_stored_columns_innodb.test:
  this test needs xtradb, it will fail with innodb
mysql-test/suite/vcol/t/vcol_partition_innodb.test:
  this test needs xtradb, it will fail with innodb
mysql-test/suite/vcol/t/vcol_select_innodb.test:
  this test needs xtradb, it will fail with innodb
mysql-test/suite/vcol/t/vcol_supported_sql_funcs_innodb.test:
  this test needs xtradb, it will fail with innodb
mysql-test/suite/vcol/t/vcol_trigger_sp_innodb.test:
  this test needs xtradb, it will fail with innodb
mysql-test/suite/vcol/t/vcol_view_innodb.test:
  this test needs xtradb, it will fail with innodb
sql/ha_partition.h:
  check_if_supported_virtual_columns() -> HA_CAN_VIRTUAL_COLUMNS
sql/handler.h:
  check_if_supported_virtual_columns() -> HA_CAN_VIRTUAL_COLUMNS
sql/share/errmsg.txt:
  no hard-coded english parts in the error messages (ER_UNSUPPORTED_ACTION_ON_VIRTUAL_COLUMN)
sql/sql_table.cc:
  no hard-coded english parts in the error messages
sql/table.cc:
  * check_if_supported_virtual_columns() -> HA_CAN_VIRTUAL_COLUMNS
  * no "csv workaround" is needed
  * no hard-coded english parts in the error messages
storage/maria/ha_maria.cc:
  check_if_supported_virtual_columns() -> HA_CAN_VIRTUAL_COLUMNS
storage/maria/ha_maria.h:
  check_if_supported_virtual_columns() -> HA_CAN_VIRTUAL_COLUMNS
storage/myisam/ha_myisam.cc:
  check_if_supported_virtual_columns() -> HA_CAN_VIRTUAL_COLUMNS
storage/myisam/ha_myisam.h:
  check_if_supported_virtual_columns() -> HA_CAN_VIRTUAL_COLUMNS
storage/xtradb/handler/ha_innodb.cc:
  check_if_supported_virtual_columns() -> HA_CAN_VIRTUAL_COLUMNS
storage/xtradb/handler/ha_innodb.h:
  check_if_supported_virtual_columns() -> HA_CAN_VIRTUAL_COLUMNS

don't fill in handlerton::commit member, as it's not used
and makes MySQL believe that Aria is transactional.
Fix the TRANSACTIONAL=1 warning.

When the optimizer creates items out of other items it does
not have to call the fix_fields method. Usually in these
cases it calls quick_fix_field() that just marks the 
created item as fixed. If the created item is an Item_func
object then calling quick_fix_field() works fine if the
arguments of the created functional item are already fixed.
Otherwise some unfixed nodes remain in the item tree and
it triggers an assertion failure whenever the item is 
evaluated.

Fixed the problem by making the method quick_fix_field
virtual and providing an implementation for the class
Item_func objects that recursively calls the method
for unfixed arguments of any functional item.

BUILD/compile-pentium64:
  Added --with-zlib-dir=bundled when doing static build.
mysql-test/suite/maria/r/maria.result:
  Added test case
mysql-test/suite/maria/t/maria.test:
  Added test case
storage/maria/ma_blockrec.c:
  We need to flush blob pages to disk to ensure they overwrite any reused head/tail pages.
  If not, REPAIR will find rows that was previously deleted.

set thd->password appropriately also for cases when a user was not found.

It was missing from EXTRA_DIST in Makefile.am.

(Includes patch for overrun detected by valgrind thanks to previous my_alloca() -> my_malloc() patch)

mysys/thr_lock.c:
  Fixed typo that caused compile failure

- This allows us to detect missing my_afree() calls and also find overruns (when running with valgrind) to alloca() areas.
- Added missing my_afree() calls
- Fixed wrong call to my_afree()


include/my_sys.h:
  When compiling with valgrind, change my_alloca() to use my_malloc()
mysql-test/suite/innodb/t/innodb_bug57255.test:
  Speed up taste case (patch from Stewart Smith)
mysql-test/suite/innodb_plugin/t/innodb_bug57255.test:
  Speed up taste case (patch from Stewart Smith)
sql/ha_partition.cc:
  Removed casts from my_afree()
sql/opt_range.cc:
  Add missing my_afree() calls.
storage/maria/ma_rt_split.c:
  Fixed wrong parameter to my_afree()

storage/maria/ma_check.c:
  Allocate also memory for nodflag and transid's
storage/maria/ma_write.c:
  Allow nodflag and transid as part of key.
  (This has nothing to do with the bug report, but it's a safer check).
storage/maria/maria_def.h:
  Define MARIA_MAX_POINTER_LENGTH (length of pointer to node)
  Added node pointer length to MARIA_INDEX_OVERHEAD_SIZE, as this is part of the key.
  (Safety fix)

Made archive.test a bit more safe



mysql-test/r/archive.result:
  Added removal of files to make rerun of failed test work
mysql-test/t/archive.test:
  Added removal of files to make rerun of failed test work
mysys/thr_lock.c:
  Better warning message if lock test fails