log_copying_thread(): Keep copying redo log until the end has been
reached. (Previously, we would stop copying as soon as
the first batch of xtrabackup_copy_logfile() returned.)

log_copying: Remove. Use log_copying_running instead.

copy_logfile: Remove. Log copying will now only be invoked from
2 places: from xtrabackup_backup_func() for the initial batch,
and from log_copying_thread() until all of the log has been read.
Use the global variable metadata_to_lsn for determining if the
final part of the log is being copied.

xtrabackup_copy_log(): Add diagnostic messages for terminating
the copying. These messages should be dead code, because
log_group_read_log_seg() should be checking for the same.

xtrabackup_copy_logfile(): Correct the retrying logic.
If anything was successfully read, process the portion that
was read. On failure, let the caller close dst_log_file.

io_watching_thread(): Stop throttling during the last phase
of copying the log (metadata_to_lsn!=0). The final copying
of the log will now be performed in log_copying_thread().

stop_backup_threads(): Clean up the message about stopping
the log copying thread.

xtrabackup_backup_low(): Read metadata_to_lsn from the latest
checkpoint header page, even if it is the first page.
Let the log_copying_thread take care of copying all of
the redo log.

InnoDB insisted on closing the file handle before renaming a file.
Renaming a file should never be a problem on POSIX systems. Also on
Windows it should work if the file was opened in FILE_SHARE_DELETE
mode.

fil_space_t::stop_ios: Remove. We no longer need to stop file access
during rename operations.

fil_mutex_enter_and_prepare_for_io(): Remove the wait for stop_ios.

fil_rename_tablespace(): Remove the retry logic; do not close the
file handle. Remove the unused fault injection that was added along
with the DATA DIRECTORY functionality (MySQL WL#5980).

os_file_create_simple_func(), os_file_create_func(),
os_file_create_simple_no_error_handling_func(): Include FILE_SHARE_DELETE
in the share_mode. (We will still prevent multiple InnoDB instances
from using the same files by not setting FILE_SHARE_WRITE.)

Revert part of 64094e12 because mysqld_safe isn't just used for
Debian. As such references to specific packaging files like
/etc/mysql/debian.cnf shouldn't have passed a review.

There's also no history of why 64094e12 was needed based on
the history of debian/patches/38_scripts__mysqld_safe.sh__signals.dpatch

Other distos have survived without mysqld_safe handling signals.

c902d5a4 changed this to bash.
This isn't approprate as mysqld_safe is used by FreeBSD where
there is no bash by default.

The original reason to use bash seems related to MDEV-3279 which
was a dash bug fixed in 2009 - https://git.kernel.org/pub/scm/utils/dash/dash.git/commit/?id=3800d4934391b144fd261a7957aea72ced7d47ea

Working around 9 year old fixed bugs shouldn't be done here.

ha_innobase::optimize(): If both innodb_defragment and
innodb_optimize_fulltext_only are at their default settings (OFF),
fall back to ALTER TABLE. Else process one or both options.

ha_innobase::set_partition_owner_stats(): Remove (unused function).

ha_innobase::ha_partition_stats: Remove (the variable is never read).

Remove unused ut_timer functions.

The problem described in the bug report happened because the code
did not test check_cols(1) after fix_fields() in a few places.

Additionally, fix_fields() could be called multiple times for SP variables,
because they are all fixed at a early stage in append_for_log().

Solution:
1. Adding a few helper methods
   - fix_fields_if_needed()
   - fix_fields_if_needed_for_scalar()
   - fix_fields_if_needed_for_bool()
   - fix_fields_if_needed_for_order_by()
  and using it in many cases instead of fix_fields() where
  the "fixed" status is not definitely known to be "false".

2. Adding DBUG_ASSERT(!fixed) into Item_splocal*::fix_fields()
   to catch double execution.

3. Adding tests.

As a good side effect, the patch removes a lot of duplicate code (~60 lines):

   if (!item->fixed &&
       item->fix_fields(..) &&
       item->check_cols(1))
     return true;

After a failed ADD INDEX, dict_index_remove_from_cache_low()
could iterate the index fields and dereference a freed virtual
column object when trying to remove the index from the v_indexes
of the virtual column.

This regression was caused by a merge of
MDEV-16119 InnoDB lock->index refers to a freed object.

ha_innobase_inplace_ctx::clear_added_indexes(): Detach the
indexes of uncommitted indexes from virtual columns, so that
the iteration in dict_index_remove_from_cache_low() can be avoided.

ha_innobase::prepare_inplace_alter_table(): Ignore uncommitted
corrupted indexes when rejecting ALTER TABLE. (This minor bug was
revealed by the extension of the test case.)

dict_index_t::detach_columns(): Detach an index from virtual columns.
Invoked by both dict_index_remove_from_cache_low() and
ha_innobase_inplace_ctx::clear_added_indexes().

dict_col_t::detach(const dict_index_t& index): Detach an index from
a column.

dict_col_t::is_virtual(): Replaces dict_col_is_virtual().

dict_index_t::has_virtual(): Replaces dict_index_has_virtual().

log_crypt_101_read_block(): Mimic MariaDB 10.1, and use the first
encryption key if the key for the checkpoint cannot be found.
Redo log encryption key rotation was ultimately disabled
in MDEV-9422 (MariaDB 10.1.13) due to design issues. So, from
MariaDB 10.1.13 onwards only one log encryption key should matter.

recv_log_format_0_recover(): Add the parameter 'bool crypt'.
Indicate when the log cannot be decrypted for upgrade, instead of
making a possibly false claim that the log requires crash recovery.

init_crypt_key(): Remove extra space from a message.

just before they're sent to the client. Because mysql->host
value is known only after the connection is established,
and it's needed for the "_sever_host" attribute.

This fixes ASAN use-after-free warning in rpl tests

followup for ee8dfc68

MDEV-15991 Server crashes in setup_on_expr upon calling SP or function executing DML on versioned tables

Do not try to set versioning conditions on every SP call. It may work
incorrectly, but it's a general bug described in MDEV-774.
This patch makes system versioning stuff consistent with other code and
also fixes a use-after-free bug.

Closes #756

* Always-false vers_select_conds_t::from_query
* Debug helpers

Closes #684

This partially reverts commit dc0a7660.

Make mariadb crc32 lib platform independent

It looks strange that someone can make use of 2 crc libraries
(Power64 or AArch64) at the same time.

The patch sets macros 'CRC32_LIBRARY' to make platform independence as an optional crc32 library.

Change-Id: I68bbf73cafb6a12f7fb105ad57d117b114a8c4af
Signed-off-by: Yuqi Gu <yuqi.gu@arm.com>

Just spacing out the links on separate linkes, they were running together
in a block of text and could be a little hard for some people to
differentiate where one begins and the next ends. Seems silly, but just
trying to help in the formatting a bit.

This bug caused crashes for queries with unreferenced non-recursive
CTEs specified by unions.It happened because the function
st_select_lex_unit::prepare() tried to use the value of the field 'derived'
that could not be set for unferenced CTEs as there was no derived
table associated with an unreferenced CTE.

INNOBASE_DEFAULTS: Replace ALTER_ADD_COLUMN with the more appropriate
ALTER_ADD_STORED_BASE_COLUMN. This clean-up causes no change of
behaviour, because ALGORITHM=INPLACE would be refused for
ALTER_ADD_STORED_GENERATED_COLUMN, and default values are not computed
nor substituted for ALTER_ADD_VIRTUAL_COLUMN.

Disable /permissive- flag, so MFC  is found, and upgrade_wizard is built.

MFC code is not standard C++, so it doesn not play well with /permissive-

commit 2dbeebdb accidentally changed
ALTER_COLUMN_OPTION and ALTER_COLUMN_STORAGE_TYPE to be separate flags.
InnoDB and Mroonga are only checking for the latter;
the example storage engine is checking for the former only.

The impact of this bug should be incorrect operation of Mroonga when
the column options GROONGA_TYPE, FLAGS are changed.

InnoDB does not define any column options, only table options,
so the flag ALTER_COLUMN_OPTION should never have been set.

Also, remove the unused flag ALTER_DROP_HISTORICAL.

Fix type mismatches in the unit test mdev10259().

btr_search_info_get_ref_count(): Do not return early if !table->space.
We can simply access table->space_id even after the tablespace has
been discarded.

btr_get_search_latch(): Relax a debug assertion to allow
!index->table->space.

Include all the Makefiles that define variables that can be useful
within debian/rules. This includes buildflags.mk as well.

Use the standard variable names and don't define our own.

For details see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=852728

For details see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=865737