null_value was not properly reset when setting parameter value

If a crash occurs during ALTER TABLE…ALGORITHM=COPY, InnoDB would spend
a lot of time rolling back writes to the intermediate copy of the table.
To reduce the amount of busy work done, a work-around was introduced in
commit fd069e2b in MySQL 4.1.8 and 5.0.2,
to commit the transaction after every 10,000 inserted rows.

A proper fix would have been to disable the undo logging altogether and
to simply drop the intermediate copy of the table on subsequent server
startup. This is what happens in MariaDB 10.3 with MDEV-14717,MDEV-14585.
In MariaDB 10.2, the intermediate copy of the table would be left behind
with a name starting with the string #sql.

This is a backport of a bug fix from MySQL 8.0.0 to MariaDB,
contributed by jixianliang <271365745@qq.com>.

Unlike recent MySQL, MariaDB supports ALTER IGNORE. For that operation
InnoDB must for now keep the undo logging enabled, so that the latest
row can be rolled back in case of an error.

In Galera cluster, the LOAD DATA statement will retain the existing
behaviour and commit the transaction after every 10,000 rows if
the parameter wsrep_load_data_splitting=ON is set. The logic to do
so (the wsrep_load_data_split() function and the call
handler::extra(HA_EXTRA_FAKE_START_STMT)) are joint work
by Ji Xianliang and Marko Mäkelä.

The original fix:

Author: Thirunarayanan Balathandayuthapani <thirunarayanan.balathandayuth@oracle.com>
Date:   Wed Dec 2 16:09:15 2015 +0530

Bug#17479594 AVOID INTERMEDIATE COMMIT WHILE DOING ALTER TABLE ALGORITHM=COPY

Problem:

During ALTER TABLE, we commit and restart the transaction for every
10,000 rows, so that the rollback after recovery would not take so long.

Fix:

Suppress the undo logging during copy alter operation. If fts_index is
present then insert directly into fts auxiliary table rather
than doing at commit time.

ha_innobase::num_write_row: Remove the variable.

ha_innobase::write_row(): Remove the hack for committing every 10000 rows.

row_lock_table_for_mysql(): Remove the extra 2 parameters.

lock_get_src_table(), lock_is_table_exclusive(): Remove.
Reviewed-by: Marko Mäkelä <marko.makela@oracle.com>
Reviewed-by: Shaohua Wang <shaohua.wang@oracle.com>
Reviewed-by: Jon Olav Hauglid <jon.hauglid@oracle.com>

Fortify wsrep_hton so that wsrep calls are not done to NULL-pointers.

- Galera tests that was not updated with connection change
  messages
- Disabled some TokuDB tests that always timed out.
  These should be enabled again when we have an option to
  specicy timeouts per tests.

Do not SET DEBUG_DBUG=-d,... in tests. To disable debug instrumentation,
save and restore the original value of the variable DEBUG_DBUG.
Assigning -d,... will enable the output of a lot of unrelated DBUG
messages to the server error log.

The patch for MDEV-15107 fixed this problem. Adding tests only.

This reverts commit 3486135b.

The commit comment ended in the words: "This is needed later."
Apparently the "later" never arrived.

MDEV-15107 Add virtual Field::sp_prepare_and_store_item(), make sp_rcontext symmetric for scalar and ROW

After MDEV-14212, the Virtual_tmp_table instance that stores a ROW
variable elements is accessible from the underlying Field_row
(rather than Item_field_row).

This patch makes some further changes by moving the code from
sp_instr_xxx, sp_rcontext, Item_xxx to Virtual_tmp_table and Field_xxx.

The data type specific code (scalar vs ROW) now resides in
a new virtual method Field_xxx::sp_prepare_and_store_item().
The the code in sp_rcontext::set_variable() and sp_eval_expr()
is now symmetric for scalar and ROW values.
The code in sp_rcontext::set_variable_row_field(), sp_rcontext::set_variable_row_field(), sp_rcontext::set_variable_row()
is now symmetric for ROW elements (i.e. scalar and ROW elements inside a ROW).

Rationale:

Prepare the code to implement these tasks soon easier:

- MDEV-12252 ROW data type for stored function return values
- MDEV-12307 ROW data type for built-in function return values
- MDEV-6121 Data type: Array
- MDEV-10593 sql_mode=ORACLE: TYPE .. AS OBJECT: basic functionality
- ROW with ROW fields (no MDEV yet)

Details:

1. Moving the code in sp_eval_expr() responsible to backup/restore
   thd->count_cuted_fields, thd->abort_on_warning,
   thd->transaction.stmt.modified_non_trans_table
   into a new helper class Sp_eval_expr_state, to reuse it easier.
   Fixing sp_eval_expr() to use this new class.

2. Moving sp_eval_expr() and sp_prepare_func_item() from public functions
   to methods in THD, so they can be reused in *.cc files easier without
   a need to include "sp_head.h".

   Splitting sp_prepare_func_item() into two parts.
   Adding a new function sp_fix_func_item(), which fixes
   the underlying items, but does not do check_cols() for them.
   Reusing sp_fix_func_item() in Field_row::sp_prepare_and_store_item().

3. Moving the code to find ROW fields by name from Item to Virtual_tmp_table

   Moving the code searching for ROW fields by their names
   from Item_field_row::element_index_by_name() to a new method
   Item_field_row to Virtual_tmp_table::sp_find_field_by_name().

   Adding wrapper methods sp_rcontext::find_row_field_by_name() and
   find_row_field_by_name_or_error(), to search for a ROW variable
   fields by the variable offset and its field name.

   Changing Item_splocal_row_field_by_name::fix_fields() to do
   use sp_rcontext::find_row_field_by_name_or_error().
   Removing virtual Item::element_index_by_name().

4. Splitting sp_rcontext::set_variable()

   Adding a new virtual method Field::sp_prepare_and_store_item().
   Spliting the two branches of the code in sp_rcontext::set_variable()
   into two virtual implementations of Field::sp_prepare_and_store_item(),
   (for Field and for Field_row).

   Moving the former part of sp_rcontext::set_variable() with the loop
   doing set_null() for all ROW fields into a new method
   Virtual_tmp_table::set_all_fields_to_null() and using it in
   Field_row::sp_prepare_and_store_item().

   Moving the former part of sp_rcontext::set_variable() with the loop
   doing set_variable_row_field() into a new method
   Virtual_tmp_table::set_all_fields_from_item() and using it in
   Field_row::sp_prepare_and_store_item().
   The loop in the new method now uses sp_prepare_and_store_item()
   instead of set_variable_row_field(), because saving/restoring
   THD flags is now done on the upper level. No needs to save/restore
   on every iteration.

5. Fixing sp_eval_expr() to simply do two things:
   - backup/restore THD flags
   - call result_field->sp_prepare_and_store_item()
   So now sp_eval_expr() can be used for both scalar and ROW variables.
   Reusing it in sp_rcontext::set_variable*().

6. Moving the loop in sp_rcontext::set_variable_row() into a
   new method Virtual_tmp_table::sp_set_all_fields_from_item_list().

   Changing the loop body to call field->sp_prepare_and_store_item()
   instead of doing set_variable_row_field(). This removes
   saving/restoring of the THD flags from every interation.
   Instead, adding the code to save/restore the flags around
   the entire loop in set_variable_row(), using Sp_eval_expr_state.
   So now saving/restoring is done only once for the entire ROW
   (a slight performance improvement).

7. Removing the code in sp_instr_set::exec_core() that sets
   a variable to NULL if the value evaluation failed.
   sp_rcontext::set_variable() now makes sure to reset
   the variable properly by effectively calling sp_eval_expr(),
   which calls virtual Field::sp_prepare_and_store_item().

   Removing the similar code from sp_instr_set_row_field::exec_core()
   and sp_instr_set_row_field_by_name::exec_core().

   Removing the method sp_rcontext::set_variable_row_field_to_null(),
   as it's not used any more.

8. Removing the call for sp_prepare_func_item() from
   sp_rcontext::set_variable_row_field(), as it was duplicate:
   it was done inside sp_eval_expr(). Now it's done inside
   virtual Field::sp_prepare_and_store_item().

9. Moving the code from sp_instr_set_row_field_by_name::exec_core()
   into sp_rcontext::set_variable_row_field_by_name(), for symmetry
   with other sp_instr_set*::exec_core()/sp_rcontext::set_variable*() pairs.
   Now sp_instr_set_row_field_by_name::exec_core() calls
   sp_rcontext::set_variable_row_field_by_name().

10. Misc:
   - Adding a helper private method sp_rcontext::virtual_tmp_table_for_row(),
     reusing it in a new sp_rcontext methods.
   - Removing Item_field_row::get_row_field(), as it's not used any more.
   - Removing the "Item *result_item" from sp_eval_expr(),
     as it's not needed any more.

Solve 3 way deadlock between plugin_initialiaze(), THD::init() and
mysql_sys_var_char().

The deadlock exists because of the lock order inversion between
LOCK_global_system_variables mutex and LOCK_system_variables_hash
read-write lock-

In this case, it is enough to change LOCK_system_variables_hash to prefer
reads to fix the deadlock, i.e change it to mysql_prlock_t

One can use -DTRASH_FREED_MEMORY to enable TRASH
macros. Useful to do when one suspects that there
is accesses to freed memory.

Extended my_free() to TRASH freed memory

Conflicts:
	sql/table.cc

Only warnings, should not have caused any bugs in old code

I disabled rocksdb in ASAN build as I got a link error when
it's included

           replicate_events_marked_for_skip=FILTER_ON_MASTER

When events of a big transaction are binlogged offsetting over 2GB from
the beginning of the log the semisync master's dump thread
lost such events.
The events were skipped by the Dump thread that found their skipping
status erroneously.

The current fixes make sure the skipping status is computed correctly.
The test verifies them simulating the 2GB offset.

do not include test suite in release zip anymore.

Probem was that dict_sys mutex was owned when calling function
dict_stats_save_defrag_stats() that assumes we do not own
dict_sys mutex.

Assertion `count > 0' failed in rpl_parallel_thread_pool::
get_thread, rpl.rpl_parallel failed in buildbot

The reason for this is that one thread can call
rpl_parallel_resize_pool_if_no_slaves() while
another thread calls at the same time
rpl_parallel_activate_pool(). If rpl_parallel_active_pool() is
called before rpl_parallel_resize_pool_if_no_slaves() has
finished, pool->count will be set to 0 even if there exists
active slave threads.

Added a mutex lock in rpl_parallel_activate_pool() to protect against this scenario, which seams to fix this issue.

Conflicts:
	mysql-test/r/sp.result
	mysql-test/t/sp.test

It crashed because we accessed lex->current_select when
it was a NULL, which is the case for SP parameters or
local variables.

While the bug was reported as a regression of
MDEV-11025 Make number of page cleaner threads variable dynamic
in MariaDB Server 10.3, the code that MariaDB Server 10.2
inherited from MySQL 5.7.4 (WL#6642) looks prone to similar errors.

pc_flush_slot(): If there is no work to do, reset the is_requested
signal, to avoid potential busy-waiting in
buf_flush_page_cleaner_worker(). If the coordinator thread has shut
down, avoid resetting the is_requested event, to avoid a potential
hang at shutdown if there are multiple worker threads.

Signed-off-by: Daniel Black <daniel@linux.vnet.ibm.com>

FULLTEXT auxiliary tables

Change the logic to take mdl lock on all tables before tablespaces are
copied, rather than  lock every single tablespace just before it is
copied.

ibuf_merge_or_delete_for_page(): Invoke fil_space_acquire_silent()
instead of fil_space_acquire() in order to avoid displaying
a useless message.

We know perfectly well that a tablespace can be dropped while a
change buffer merge is pending, because change buffer merges skip
any transactional locks.

innobase_commit_by_xid(), innobase_rollback_by_xid(): Decrement
the reference count before freeing the transaction object to the pool.
Failure to do so might corrupt the transaction bookkeeping
if trx_create_low() returns the same object to another thread
before we are done with it.

trx_sys_close(): Detach the recovered XA PREPARE transactions from
trx_sys->rw_trx_list before freeing them.

join_tab->distinct=true means "Before doing record read with this
join_tab, call join_tab->remove_duplicates() to eliminate duplicates".
remove_duplicates() assumes that
- there is a temporary table $T with rows that are to be de-duplicated
- there is a previous join_tab (e.g. with join_tab->fields) which was
used to populate the temp.table $T.

When the query has "Impossible WHERE" and window function, then the above
conditions are not met (but we still might need a window function
computation step when the query has implicit grouping).

The fix is to not add remove_duplicates step if the select execution is
degenerate (and we'll have at most one row in the output anyway).