Merge to 10.3
- changed a few LEX_STRING to LEX_CSTRING

The XtraDB storage engine was already replaced by InnoDB
and disabled in MariaDB Server 10.2. Let us remove it altogether
to avoid dragging dead code around.

Replace some references to XtraDB with references to InnoDB.

rpl_get_position_info(): Remove.

Remove the mysql-test-run --suite=percona, because it only contains
tests specific to XtraDB, many of which were disabled already in
earlier MariaDB versions.

In MariaDB 10.2, this column is practically always reported as 0,
even before the data field trx_t::has_search_latch was removed.

Remove the adjustments for some parameters that were deprecated in
MariaDB 10.2 and removed in 10.3.

This will also change the minimum and maximum value of
innodb_log_file_size to 1MiB and 512GiB, respectively.

MDEV-12975 InnoDB redo log minimum size check uses detected file size instead of requested innodb_log_file_size

log_calc_max_ages(): Use the requested size in the check, instead of
the detected redo log size. The redo log will be resized at startup
if it differs from what has been requested.

The option was basically duplicating InnoDB functionality.
Persistent statistics can be accessed via the tables
mysql.innodb_table_stats and mysql.innodb_index_stats.

These functions cannot possibly work in MariaDB 10.2, and it is
uncertain if they really work in 10.1 either.

Additional tests for MDEV-10309 COALESCE(12345678900) makes a column of a wrong type and truncates the data

This is another attempt to fix the bug mdev-12992.
This patch introduces st_select_lex::context_analysis_place for
the place in SELECT where context analysis is currently performed.
It's similar to st_select_lex::parsing_place, but it is used at
the preparation stage.

             lock not released after timeout

Release GRL if FLUSH TABLES phase failed.

When the btr_search_latch was split into an array of latches
in MySQL 5.7.8 as part of the Oracle Bug#20985298 fix, the "caching"
of the latch across storage engine API calls was removed, and
the field trx->has_search_latch would only be set during a short
time frame in the execution of row_search_mvcc(), which was
formerly called row_search_for_mysql().

This means that the column
INFORMATION_SCHEMA.INNODB_TRX.TRX_ADAPTIVE_HASH_LATCHED will always
report 0. That column cannot be removed in MariaDB 10.2, but it
can be removed in future releases.

trx_t::has_search_latch: Remove.

trx_assert_no_search_latch(): Remove.

row_sel_try_search_shortcut_for_mysql(): Remove a redundant condition
on trx->has_search_latch (it was always true).

sync_check_iterate(): Make the parameter const.

sync_check_functor_t: Make the operator() const, and remove result()
and the virtual destructor. There is no need to have mutable state
in the functors.

sync_checker<bool>: Replaces dict_sync_check and btrsea_sync_check.

sync_check: Replaces btrsea_sync_check.

dict_sync_check: Instantiated from sync_checker.

sync_allowed_latches: Use std::find() directly on the array.
Remove the std::vector.

TrxInInnoDB::enter(), TrxInInnoDB::exit(): Remove obviously redundant
debug assertions on trx->in_depth, and use equality comparison against 0
because it could be more efficient on some architectures.

The sole purpose of handlerton::release_temporary_latches and its wrapper
function was to release the InnoDB adaptive hash index latch
(btr_search_latch).

When the btr_search_latch was split into an array of latches
in MySQL 5.7.8 as part of the Oracle Bug#20985298 fix, the "caching"
of the latch across storage engine API calls was removed. As part of that,
the function trx_search_latch_release_if_reserved() was changed to an
assertion and the function trx_reserve_search_latch_if_not_reserved()
was removed, and handlerton::release_temporary_latches() practically
became a no-op.

Note: MDEV-12121 replaced the function
trx_search_latch_release_if_reserved()
with the more appropriately named macro trx_assert_no_search_latch().

Merge 10.2 into bb-10.2-ext; also, fix MDEV-13015 After restart, InnoDB wrongly thinks that a SEQUENCE is a TABLE

sql_sequence.read_only: Show that the sequence can be read in
both read-only and read-write mode, and that the sequence remains
accessible after a server restart.

dict_sys_tables_type_to_tf(): Change the parameter n_cols to not_redundant.

dict_tf_is_valid_not_redundant(): Refactored from dict_tf_is_valid().

dict_sys_tables_type_valid(): Replaces dict_sys_tables_type_validate().
Use the common function dict_tf_is_valid_not_redundant(), which validates
PAGE_COMPRESSION_LEVEL more strictly.

DICT_TF_GET_UNUSED(flags): Remove.

innodb.table_flags: Adjust the test case. Due to the MDEV-12873 fix
in 10.2, the corrupted flags for table test.td would be converted,
and a tablespace flag mismatch will occur when trying to open the file.

MDEV-12873 InnoDB SYS_TABLES.TYPE incompatibility for PAGE_COMPRESSED=YES in MariaDB 10.2.2 to 10.2.6

Remove the SHARED_SPACE flag that was erroneously introduced in
MariaDB 10.2.2, and shift the SYS_TABLES.TYPE flags back to where
they were before MariaDB 10.2.2. While doing this, ensure that
tables created with affected MariaDB versions can be loaded,
and also ensure that tables created with MySQL 5.7 using the
TABLESPACE attribute cannot be loaded.

MariaDB 10.2.2 picked the SHARED_SPACE flag from MySQL 5.7,
shifting the MariaDB 10.1 flags PAGE_COMPRESSION, PAGE_COMPRESSION_LEVEL,
ATOMIC_WRITES by one bit. The SHARED_SPACE flag would always
be written as 0 by MariaDB, because MariaDB does not support
CREATE TABLESPACE or CREATE TABLE...TABLESPACE for InnoDB.

So, instead of the bits AALLLLCxxxxxxx we would have
AALLLLC0xxxxxxx if the table was created with MariaDB 10.2.2
to 10.2.6. (AA=ATOMIC_WRITES, LLLL=PAGE_COMPRESSION_LEVEL,
C=PAGE_COMPRESSED, xxxxxxx=7 bits that were not moved.)

PAGE_COMPRESSED=NO implies LLLLC=00000. That is not a problem.

If someone created a table in MariaDB 10.2.2 or 10.2.3 with
the attribute ATOMIC_WRITES=OFF (value 2; AA=10) and without
PAGE_COMPRESSED=YES or PAGE_COMPRESSION_LEVEL, the table should be
rejected. We ignore this problem, because it should be unlikely
for anyone to specify ATOMIC_WRITES=OFF, and because 10.2.2 and
10.2.2 were not mature releases. The value ATOMIC_WRITES=ON (1)
would be interpreted as ATOMIC_WRITES=OFF, but starting with
MariaDB 10.2.4 the ATOMIC_WRITES attribute is ignored.

PAGE_COMPRESSED=YES implies that PAGE_COMPRESSION_LEVEL be between
1 and 9 and that ROW_FORMAT be COMPACT or DYNAMIC. Thus, the affected
wrong bit pattern in SYS_TABLES.TYPE is of the form AALLLL10DB00001
where D signals the presence of a DATA DIRECTORY attribute and B is 1
for ROW_FORMAT=DYNAMIC and 0 for ROW_FORMAT=COMPACT. We must interpret
this bit pattern as AALLLL1DB00001 (discarding the extraneous 0 bit).

dict_sys_tables_rec_read(): Adjust the affected bit pattern when
reading the SYS_TABLES.TYPE column. In case of invalid flags,
report both SYS_TABLES.TYPE (after possible adjustment) and
SYS_TABLES.MIX_LEN.

dict_load_table_one(): Replace an unreachable condition on
!dict_tf2_is_valid() with a debug assertion. The flags will already
have been validated by dict_sys_tables_rec_read(); if that validation
fails, dict_load_table_low() will have failed.

fil_ibd_create(): Shorten an error message about a file pre-existing.

Datafile::validate_to_dd(): Clarify an error message about tablespace
flags mismatch.

ha_innobase::open(): Remove an unnecessary warning message.

dict_tf_is_valid(): Simplify and stricten the logic. Validate the
values of PAGE_COMPRESSION. Remove error log output; let the callers
handle that.

DICT_TF_BITS: Remove ATOMIC_WRITES, PAGE_ENCRYPTION, PAGE_ENCRYPTION_KEY.
The ATOMIC_WRITES is ignored once the SYS_TABLES.TYPE has been validated;
there is no need to store it in dict_table_t::flags. The PAGE_ENCRYPTION
and PAGE_ENCRYPTION_KEY are unused since MariaDB 10.1.4 (the GA release
was 10.1.8).

DICT_TF_BIT_MASK: Remove (unused).

FSP_FLAGS_MEM_ATOMIC_WRITES: Remove (the flags are never read).

row_import_read_v1(): Display an error if dict_tf_is_valid() fails.

dict_table_t::thd: Remove. This was only used by btr_root_block_get()
for reporting decryption failures, and it was only assigned by
ha_innobase::open(), and never cleared. This could mean that if a
connection is closed, the pointer would become stale, and the server
could crash while trying to report the error. It could also mean
that an error is being reported to the wrong client. It is better
to use current_thd in this case, even though it could mean that if
the code is invoked from an InnoDB background operation, there would
be no connection to which to send the error message.

Remove dict_table_t::crypt_data and dict_table_t::page_0_read.
These fields were never read.

fil_open_single_table_tablespace(): Remove the parameter "table".

BULK execution moved to a new command.

innodb.row_format_redundant: Really corrupt the SYS_TABLES.MIX_LEN,
and do not use any debug instrumentation. For tables created in the
system tablespace, the contents of the column will be ignored.
Only the table t1 will refuse to load.

dict_load_table_one(): Remove the DBUG_EXECUTE_IF instrumentation.
Omit a redundant error message "incorrect flags in SYS_TABLES".

dict_sys_tables_rec_read(): Partially revert the Oracle Bug#21644827
fix, and always report errors by the return value.

fts_create_in_mem_aux_table(): Do not rely on dict_table_t::flags2,
but instead evaluate the tablespace ID.

DICT_TF2_BITS: Reduce to the correct value of 7. The two extra
high-order bits were specific to MySQL 5.7.

Cover innodb.table_flags with the new innodb_page_size.combinations
32k and 64k.

dict_sys_tables_type_validate(): Remove an assertion that made a
check in the function redundant. Remove the excessive output to
the error log, as the invalid SYS_TABLES.TYPE value is already being
output.

Add a missing #include "sync0types.h" that was removed in MDEV-12674.

Add a test case for corrupting SYS_TABLES.TYPE,
and for ROW_FORMAT=REDUNDANT, the unused field SYS_TABLES.MIX_LEN
that must be ignored (InnoDB before MySQL 5.5 wrote uninitialized
garbage to this column).

MariaDB 10.0 appears to validate the SYS_TABLES.TYPE properly.
This is a test-only change.

CLOCKS_PER_SEC not used anymore.
start_timer()/mysql_end_timer() don't make much sense.
Buffer size requirement increased by 1 byte.

As svoj points out, my_timer_microseconds uses gettimeofday which is
affected by "discontinuous jumps in the system time", according to
man.

This patch changes to use microsecond_interval_timer which is "not
perfect, but is less affected by these jumps", and also does not
require an include of my_rdtsc.h header.

https://github.com/MariaDB/server/pull/332#discussion_r114708923

"No space before "=" and space after cast please"
https://github.com/MariaDB/server/pull/332#discussion_r114708923

Oversight spotted by svoj:
https://github.com/MariaDB/server/pull/332#discussion_r114708923

based upon: https://github.com/dveeden/mysql-server/commit/d5e46428075d86dbdc333d27951bb06cb4c11a32

MariaDB [test]> select sleep(0.123);
+--------------+
| sleep(0.123) |
+--------------+
|            0 |
+--------------+
1 row in set (0.123 sec)

"More exact timing for mysql client based on my_timer_microseconds"

Based on suggestion from @grooverdan on https://github.com/mysql/mysql-server/pull/112

This patch is slightly bigger because the original did not preserve the
return type of my_timer_microseconds and this patch does.
   (my_timer_microseconds returns ulonglong, not simply ulong)

Also I believe the correct place to do the division of microseconds to
seconds is better in the caller of nice_time because nice_time takes a
"double sec" param, so we should convert before calling; the other caller
of nice_time does not call with microseconds.

in innodb_read_only mode.

The reason for the hang is that there was no notification received about
completed read io. File handles are bound to completion_port, and there
were no background "write" threads that would be waiting on completion_port,
only 2 "read" threads waiting on read_completion_port were active.

The fix is to use a single IO completion port for all IOs, if
innodb_read_only is set.

log_crypt(): Remove the useless error log output that was
accidentally introduced in MDEV-11782. These messages could be emitted
to the server error log during crash recovery.

10.2 follow-up to MDEV-13039 innodb_fast_shutdown=0 crash due premature purge shutdown before fts_optimize_shutdown()

srv_start_state_t: Document the flags. Replace SRV_START_STATE_STAT
with SRV_START_STATE_REDO. The srv_bg_undo_sources replaces the
original use of SRV_START_STATE_STAT.

dict_stats_thread_started, buf_dump_thread_started,
buf_flush_page_cleaner_thread_started: Remove (unused).

srv_shutdown_all_bg_threads(): Always wait for the I/O threads
to exit, also in read-only mode.

os_thread_free(): Remove.