The bug was that when using bulk insert combined with lock table, we intitalized the io cache with the wrong file position.
This fixed a bug where MariaDB could not read in a table dump done with mysqldump.


mysql-test/suite/maria/r/locking.result:
  Test case for locking + write cache bug
mysql-test/suite/maria/t/locking.test:
  Test case for locking + write cache bug
storage/maria/ma_extra.c:
  Initialize write cache used with bulk insert to correct file length.
  (The old code didn't work if one was using LOCK TABLE for the given table).

Fixes bugs found by testcase for lp:815022 and lp:726374 "ma_blockrec.c:3000: write_block_record: Assertion `cur_block[1].page_count == 0' failed with a multi-index Aria workload"
The issues was:
- For some tables with a lot of not packed fields, we didn't allocate enough memory in head page which caused DBUG_ASSERT's
- Removed wrong DBUG_ASSERT()
- Fixed a problem with underflow() where it generates a key page where all keys didn't fit.
- Max key length is now limited by block_size/3  (was block_size /2).  This is required for underflow() to work with packed keys.




mysql-test/lib/v1/mysql-test-run.pl:
  Remove --alignment=8 as this doesn't work on 64 bit systems
mysql-test/suite/maria/r/small_blocksize.result:
  Test case for Aria bug
mysql-test/suite/maria/t/small_blocksize-master.opt:
  Test case for Aria bug
mysql-test/suite/maria/t/small_blocksize.test:
  Test case for Aria bug
storage/maria/ha_maria.cc:
  Fixed comment
storage/maria/ma_bitmap.c:
  Fixed wrong variable usage in find_where_to_split_row() where we allocated too little memory for head page.
  We did not take into account space for head extents (long VARCHAR) when trying to split row on head page. This caused us to allocate too little space from bitmap which lead to ASSERT failures later.
storage/maria/ma_blockrec.c:
  Made some argument const (to ensure they was not accidently changed)
  Removed wrong DBUG_ASSERT()
storage/maria/ma_blockrec.h:
  Removed not used variable
storage/maria/ma_delete.c:
  Added my_afree() in case of error
  More comments and DBUG_ASSERT() for underflow()
storage/maria/ma_open.c:
  Make keyinfo->underflow_block_length smaller for packed keys. This has to be done as for long packed keys, underflow() otherwise generates a key page where all keys didn't fit.
storage/maria/ma_page.c:
  New DBUG_ASSERT()
storage/maria/ma_write.c:
  Fixed comment
storage/maria/maria_def.h:
  We have to have space for at least 3 keys on a key page.
  (Otherwise the underflow() code doesn't work for packed keys, even when we have an underflow() for an empty key page)

Fixed lp:814231 Aria post-recovery error "Bitmap at page 0 has pages reserved outside of data file length"
The bug was a wrong check in aria_chk;  The table was fine.


storage/maria/ma_bitmap.c:
  Print whole bitmap to find errors in last bitmap
storage/maria/ma_check.c:
  Fixed wrong test if bitmap was overallocated.

Fixed bug lp:814054 'Assertion `block->hash_link == hash_link && hash_link->block == block' in ma_pagecache.c:2275 with Aria'
- Replaced old DBUG_ASSERT with a new correct one + a comment.

storage/maria/ma_pagecache.c:
  Replaced old DBUG_ASSERT with a new correct one + a comment.

This fixes a bug that when you use mysqldump --no-create-info to generate a dump that you want to merge with an existing table,
you can get an innodb table with duplicated unique keys.
Patch orignally by Eric Bergen.


client/mysqldump.c:
  Only use UNIQUE_CHECKS=0 for tables that are created.
  This solves the issue that you can't get duplicate unique keys when merging two dumps.
mysql-test/r/mysqldump.result:
  Test for mysqldump --no-create-info

storage/maria/ma_blockrec.c:
  Unlock bitmaps earlier (no reason to have them unlocked over _ma_write_clr())
storage/maria/ma_extra.c:
  Don't lock THR_LOCK_maria for HA_EXTRA_PREPARE_FOR_RENAME (upper level ensures that we are not opening the same table during this call)
  We don't need to have share->intern_lock locked over _ma_flush_table_files()
storage/maria/ma_open.c:
  Update comment

Problem was the parsing of test suite files for various tags and options.
This was done inefficiently, and include files were re-parsed for every
place they were included. This caused a delay of 20 seconds or so before
the first test started to run.

By parsing more efficiently and re-using first parse for subsequent
inclusion of the same file, time spent parsing is reduced to less than
1 second, and start appears instantaneous.

(With this patch, full ./mtr runs in 3 minutes on my laptop (release
build.)

mysql-test/suite/innodb_plugin/t/innodb_bug52663.test:
  Test is fairly slow, so try to avoid getting stuck with it at the end
  while other workers are idle.

Fixed lp:814237: Wrong mutex usage in Aria


storage/maria/ma_bitmap.c:
  Added call to _ma_bitmap_mark_file_changed() when flushing bitmap. This fixes lp:814237

scripts/mysqldumpslow.sh:
  Remove not used --basedir option
  Ensure that the last --datadir option is used from the my.cnf files.

The bug happens when one uses MAX_ROWS=# with Aria & row_format=page and one insert more than # rows.


mysql-test/mysql-test-run.pl:
  Ignore table is full error messages
mysql-test/suite/maria/r/max_length.result:
  Test case for 'Table is full'
mysql-test/suite/maria/t/max_length.test:
  Test case for 'Table is full'
storage/maria/ma_bitmap.c:
  Ensure that we don't allocate bits outside of max_data_file_size.
  Adjust max_data_file_size based on bitmap alignments.
  Backport fix to adjust wrong first_bitmap_with_space.
storage/maria/ma_blockrec.c:
  Calculate value of max_data_file_length
storage/maria/ma_blockrec.h:
  Updated prototype for _ma_bitmap_init()
storage/maria/ma_check.c:
  Give warnings if file sizes are above max file sizes.
  Give more warnings in case of errors.
  Have maria_chk write if table is recreated.
storage/maria/ma_create.c:
  Better calculation of max_data_file_length and thus data pointer length.
  Fixes some wrong pointer lengths when using MAX_ROWS=#
storage/maria/ma_open.c:
  Removed duplicate assigment.
  Use block size from file instead of global variable.
storage/maria/maria_chk.c:
  Remove -1 from printed file length
storage/maria/maria_def.h:
  Update struct st_maria_file_bitmap

Fixed explains of previous patch.

mysql-test/r/explain.result:
  Fixed explains of previous patch.
mysql-test/r/join_outer.result:
  Fixed explains of previous patch.
mysql-test/r/negation_elimination.result:
  Fixed explains of previous patch.
mysql-test/r/view.result:
  Fixed explains of previous patch.
mysql-test/suite/innodb/r/innodb_mysql.result:
  Removed duplicate test suite.
mysql-test/suite/innodb/t/innodb_mysql.test:
  Removed duplicate test suite.
mysql-test/suite/innodb_plugin/r/innodb_mysql.result:
  Removed duplicate test suite.
mysql-test/suite/innodb_plugin/t/innodb_mysql.test:
  Removed duplicate test suite.
sql/opt_range.h:
  Removed incorrect fix.
sql/records.cc:
  Removed incorrect fix.

There are 2 volatile condition constructions AND/OR constructions and fields(references) when first
good supported to be top elements of conditions because it is normal practice
(see copy_andor_structure for example) fields without any expression in the condition is really rare
and mostly useless case however it could lead to problems when optimiser changes/moves them unaware
of other variables referring to them. An easy solution of this problem is just to replace single field
in a condition with equivalent expression well supported by the server (<field> -> <field> != 0).

mysql-test/r/view.result:
  New test added.
mysql-test/t/view.test:
  New test added.
sql/sql_parse.cc:
  <field> -> <field> != 0
sql/sql_yacc.yy:
  <field> -> <field> != 0

Fix is to replace uint in public header with unsigned int.  uint is not guaranteed to be defined by system headers.

before strlen(db) we need to be sure that
db lies within packet boundaries.
same for client_plugin.

before strlen(db) we need to be sure that
db lies within packet boundaries

- Fixed multi-user problem with one thread doing inserts and another doing scans that gave error 175
- Fixed bug that caused assert in move_to_next_bitmap() & _ma_read_bitmap_page()
- Much more DBUG_ASSERT(!maria_assert_if_crashed_table) to detect errors early
- EXTERNAL_LOCKING -> MARIA_EXTERNAL_LOCKING (to use same define everywhere

storage/maria/ma_bitmap.c:
  More secure handling of first_bitmap_with_space (now we also take care of wrong values)
  Don't set page to -1; This fixed unlikely bug that caused assert in move_to_next_bitmap() & _ma_read_bitmap_page()
storage/maria/ma_blockrec.c:
  More DBUG_ASSERT()'s
  Fixed multi-user problem with one thread doing inserts and another doing scans that gave error 175
  (We should use data_file_length from start of scan, not new value as new bitmap page may not yet be in page cache)
storage/maria/ma_check.c:
  EXTERNAL_LOCKING -> MARIA_EXTERNAL_LOCKING (to use same define everywhere)
storage/maria/ma_checkpoint.c:
  Made maria_checkpoint_min_activity static so that one can change it in debugger while testing.
  Fixed long standing performance problem that caused write of state info at checkpoint for any file that was ever changed since open.
storage/maria/ma_create.c:
  EXTERNAL_LOCKING -> MARIA_EXTERNAL_LOCKING (to use same define everywhere
storage/maria/ma_dynrec.c:
  Added missing MARIA_EXTERNAL_LOCKING (minor performance improvement)
storage/maria/ma_locking.c:
  EXTERNAL_LOCKING -> MARIA_EXTERNAL_LOCKING (to use same define everywhere
storage/maria/ma_open.c:
  EXTERNAL_LOCKING -> MARIA_EXTERNAL_LOCKING (to use same define everywhere
storage/maria/ma_pagecache.c:
  Added assert to detect reads outside of data file
storage/maria/maria_def.h:
  Added checkpoint_state to cache state writes in checkpoint

- Fixed performance bug in alter table with Aria; Aria didn't use disable keys + enable keys
- Fixed wrong warning about 'Wrong CRC on datapage' from REPAIR TABLE with aria block tables.
- Fixed bug in aria_chk that disabled performance counters.
- Added --translog_buffer_size to maria_read_log.

storage/maria/ha_maria.cc:
  Fixed performance bug in alter table with Aria; Aria didn't use disable keys + enable keys
storage/maria/ma_bitmap.c:
  Added some DBUG_ASSERT()'s and made code more uniform
storage/maria/ma_check.c:
  Fixed wrong warning about 'Wrong CRC on datapage' from REPAIR TABLE with aria block tables.
storage/maria/ma_pagecache.c:
  Copy read pages to buffer even if there was an error (to be able to detect zero filled pages)
storage/maria/maria_chk.c:
  Fixed bug in aria_chk that disabled performance counters.
storage/maria/maria_read_log.c:
  Added option to set translog_buffer_size

Fixed several errors in Aria discovered by test case for lp:727869 ma_pagecache.c:2103: find_block: Assertion `block->rlocks == 0
- Fixed assert in transaction log handler when aria_check was run on block-record table that was much bigger than expected.
- Fixed warnings about wrong mutex order between bitmap and intern_lock
- Fixed error in bitmap that could cause two rows to use same block for a block record.
- Fixed wrong test that could cause error if last page for a bitmap was used by a blob.
- Fixed several bugs in pagecache for the case where pagecase had very few blocks and there was a lot of threads competing to get the blocks (very unlikely case).


mysql-test/suite/maria/r/maria-recovery3.result:
  Updated results
sql/mysqld.cc:
  Allow mi_check() to send information messages for log file
storage/maria/ma_bitmap.c:
  Fixed problem with wrong mutex order when bitmap was the first page that was flushed out of page cache
  - Fixed by introducing _ma_bitmap_mark_file_changed() that marks file changed without a bitmap lock.
  - Fixed one case in _ma_change_bitmap_page() where we didn't mark the bitmap changed. This could cause to rows to reuse same block if this was the only change to the bitmap.
  - Split _ma_bitmap_get_page_bits() in two parts to not take a bitmap lock when we already have it
  - Fixed bug in _ma_bitmap_set_full_page_bits() that caused an error if last page for a bitmap was used by a blob
storage/maria/ma_check.c:
  Better handling of wrong file length.
  Fixed bug when we tried to write to transaction log when it was not opened (happened when block record file was bigger than expected)
storage/maria/ma_pagecache.c:
  Fixed several bugs in pagecache for the case where pagecase had very few blocks and there was a lot of threads competing to get the blocks:
  - In link_block() mark a block given to another thread with PCBLOCK_REASSIGNED to ensure that no other threads can start re-using the block
    before the thread that requsted a block.
  - In free_block(), don't reset status for a block that is in re-assign by link_block() (we don't want to loose the PCBLOCK_REASSIGNED flag).
  - Added call to wait_for_flush() when we got a new block in find_block() to ensure that we don't use a block that is beeing flushed by another thread.
  - Moved setting of hits_left and last_hit_time in find_block() to where we assign the block.
  
  
  Code cleanup and making code uniform:
  - Changed a lot of KEYCACHE_DBUG_PRINT to use DBUG_PRINT
  - Streamlined all reporting of 'signal' and 'wait' between threads to be identical.
  - Use thread name instead of thread number (for each match against --debug)
  - Added more DBUG_ENTER, DBUG_PRINT and DBUG_ASSERT()
  - Added more comments
storage/myisam/ha_myisam.cc:
  Only print information about that we make a backup if we are really making a backup.
storage/myisam/mi_check.c:
  Inform mysqld that we are creating a backup of the data file (for inclusion in error log).