When undo log pages pre-exist from an upgrade, the
TRX_UNDO_PAGE_TYPE could be TRX_UNDO_INSERT==1 (for insert_undo)
TRX_UNDO_UPDATE==2 (for update_undo), instead of the new unified
page type 0 that was introduced in MDEV-12288.

The undo log page type does not really matter much, because the
undo log record type identifies the records independently
of the page type. So, the debug assertions can simply allow any
potential value of the TRX_UNDO_PAGE_TYPE (0, 1, or 2).

MDEV-13475 InnoDB: Failing assertion: lsn == log_sys->lsn || srv_force_recovery == SRV_FORCE_NO_LOG_REDO

At shutdown, we would have lsn == srv_start_lsn == 0 if the server
was started up in --innodb-read-only mode with an old-format redo log.

This regression was caused by MDEV-13430, which skips some redo log
processing at startup when the redo log file format differs (and the
redo log has been determined to be logically empty).

Even though the MDEV-13430 change was introduced in MariaDB 10.2.8,
the MariaDB Server 10.2 series is unaffected by this, because
it will refuse to start up from a version-tagged redo log that is
not tagged to be in the MySQL 5.7.9 or MariaDB 10.2.2 format.
Starting with MariaDB 10.3.1, there are multiple version-tagged
redo log formats.

recv_recovery_from_checkpoint_start(): When skipping an empty
different-format redo log, initialize srv_start_lsn and
recv_sys->recovered_lsn from the redo log file.

Most of this fix was merged from a backport to 10.2.

recv_find_max_checkpoint(): Do not return DB_SUCCESS on an error.

The XtraDB storage engine is not used in MariaDB 10.2, and
it no longer exists starting with MariaDB 10.3.1.

row_log_table_apply_convert_mrec(), row_log_table_apply_insert_low(),
row_log_table_apply_insert(), row_log_table_apply_update(): Remove
the trx_id parameter. The clustered index may contain DB_TRX_ID=0,
but PAGE_MAX_TRX_ID=0 is invalid for secondary indexes. Use the
current transaction ID when inserting secondary index records.

row_log_table_apply_op(): Remove the trx_id_col parameter.

The inserted SYS_INDEXES record for the ADD INDEX operation will also
be counted as an un-purgeable transaction.

Disable change buffering, so that some data that was previously
written to the encrypted redo log will not end up being copied
to the unencrypted redo log due to change buffer merge.

The thd_destructor_proxy detects that no transactions are active and
starts srv_shutdown_bg_undo_sources(), but fails to take into account
that new transactions can still start, especially be slave but also
by other threads. In addition there is no mutex when checking for
active transaction so this is not safe.

We relax the failing InnoDB debug assertion by allowing the execution
of user transactions after the purge thread has been shut down.

FIXME: If innodb_fast_shutdown=0, we should somehow guarantee that no
new transactions can start after thd_destructor_proxy observed that
trx_sys_any_active_transactions() did not hold.

row_update_for_mysql(): Remove the wrapper function and
rename the function from row_update_for_mysql_using_upd_graph().
Remove the unused parameter mysql_rec.

This is basically a duplicate or a reincarnation of MDEV-117.
For some reason, the test innodb.mdev-117 started failing in 10.2.

It is uncertain when this test started failing. The test is
nondeterministic, because there is a race condition between the
concurrently executing DELETE IGNORE and DELETE statements.

When a deadlock is reported for DELETE IGNORE, the SQL layer would
call handler::print_error() but then proceed to the next row,
as if no error had happened (which is the purpose of DELETE IGNORE).
So, when it proceeded to handler::ha_rnd_next(), InnoDB would hit an
assertion failure, because the transaction no longer exists, and we
are not executing at the start of a statement.

handler::print_error(): If thd_mark_transaction_to_rollback(thd, true)
was called, clear the ME_JUST_WARNING and ME_JUST_INFO errflags, so
that a note or warning will be promoted to an error if the transaction
was aborted by a storage engine.

        Check for duplicating keys added.

outside.

        The result_limit variable wasn't always initialized in
        Item_func_json_array::fix_length_and_dec().

The file wait_innodb_all_purged.inc waited for InnoDB purge in a way
that only worked in debug builds. The file wait_all_purged.inc
provides a better mechanism.

If the latest InnoDB redo log checkpoint was stored in the
first checkpoint slot and not the second one, InnoDB would
incorrectly set log_sys->log.lsn to the previous checkpoint.

It is possible that this logic error did not exist before
commit 86927cc7, which
removed traces of multiple InnoDB redo logs, to prepare for
MDEV-12548 (Mariabackup for MariaDB 10.2). In the worst case,
this error could mean that InnoDB unnecessarily fails to
recover from redo log when the last-but-one checkpoint was
overwritten, but the last checkpoint is intact.

recv_find_max_checkpoint(), recv_find_max_checkpoint_0():
Do not overwrite the fields of log_sys->log with the information
of an older checkpoint.

recv_find_max_checkpoint(): Do not return DB_SUCCESS on an error.

recv_recovery_from_checkpoint_start(): Return early if the log is
in a version-tagged format but not in the latest format. (In this case,
the log must be logically empty, and there is nothing to apply.)

Always read full page 0 to determine does tablespace contain
encryption metadata. Tablespaces that are page compressed or
page compressed and encrypted do not compare checksum as
it does not exists. For encrypted tables use checksum
verification written for encrypted tables and normal tables
use normal method.

buf_page_is_checksum_valid_crc32
buf_page_is_checksum_valid_innodb
buf_page_is_checksum_valid_none
        Modify Innochecksum logging to file to avoid compilation
	warnings.

fil0crypt.cc fil0crypt.h
        Modify to be able to use in innochecksum compilation and
        move fil_space_verify_crypt_checksum to end of the file.
        Add innochecksum logging to file.

univ.i
        Add innochecksum strict_verify, log_file and cur_page_num
        variables as extern.

page_zip_verify_checksum
        Add innochecksum logging to file and remove unnecessary code.

innochecksum.cc
        Lot of changes most notable able to read encryption
        metadata from page 0 of the tablespace.

Added test case where we corrupt intentionally
FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION (encryption key version)
FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION+4 (post encryption checksum)
FIL_DATA+10 (data)

        Comparison fixed to take the actual type of JSON value into
        account. Bug in escaping handling fixed.

Added extra memcpy to get rid of valgrind warning for sequence tables with
InnoDB. When reading a row from InnoDB, some of the bytes in the row
are marked as not initialized. Needs to be investigated later, but
this is a safe patch for now.

Problem was that SEQUENCE::table was shared among threads, which caused
several threads to use the same object at the same time.

Conflicts:
	mysql-test/r/func_json.result
	mysql-test/r/win.result
	mysql-test/t/func_json.test
	mysql-test/t/win.test
	sql/share/errmsg-utf8.txt
	storage/rocksdb/ha_rocksdb.cc
	storage/rocksdb/mysql-test/rocksdb/r/tbl_opt_data_index_dir.result

Problem was introduced with the InnoDB 5.7 merge, the code related to
avoiding extra fsync at the end of commit when binlog is enabled. The
MariaDB method for this was removed, but the replacement MySQL method
based on thd_get_durability_property() is not functional in MariaDB.

This commit reverts the offending parts of the merge and adds a test
case, to fix the problem for InnoDB. But other storage engines are
likely to have a similar problem.

- Support first_linear_tab() traversal for degenerate joins

The debug flag recv_no_log_write prohibits writes of redo log records for
modifying page data. The debug assertion was failing when fil_names_clear()
was writing the informative MLOG_FILE_NAME and MLOG_CHECKPOINT records
which do not modify any data.

log_reserve_and_open(), log_write_low(): Remove the debug assertion.

log_pad_current_log_block(), mtr_write_log(),
mtr_t::Command::prepare_write(): Add the debug assertion.

During InnoDB startup, change buffer merge operations are prohibited
before recv_apply_hashed_log_recs(true), which performs the last phase
of redo log apply. Before this call, ibuf_init_at_db_start() would be
invoked, and it could trigger the debug assertion.

ibuf_init_at_db_start(): Do not declare the mini-transaction as
"inside change buffer", because nothing is being written in the
mini-transaction. The purpose of this function is only to initialize
the memory data structures from the persistent data structures.

        Fixed the path comparison.

        Options handling implemented for ST_AsGeoJSON.

Also removed clang-3.9
Signed-off-by: Daniel Black <daniel.black@au.ibm.com>