This fixes mdev_26541.test, and the new clean_up_spider.inc will be
useful for other tests where part of deinit_spider does not apply,
e.g. those testing spider initialisation only.

Fix spider init bugs (MDEV-22979, MDEV-27233, MDEV-28218) while
preventing regression on old ones (MDEV-30370, MDEV-29904)

Two things are changed:

First, Spider initialisation is made fully synchronous, i.e. it no
longer happens in a background thread. Adapted from the original fix
by nayuta for MDEV-27233. This change itself would cause failure when
spider is initialised early, by plugin-load-add, due to dependency on
Aria and udf function creation, which are fixed in the second and
third parts below. Requires SQL Service, thus porting earlier versions
requires MDEV-27595

Second, if spider is initialised before udf_init(), create udf by
inserting into `mysql.func`, otherwise do it by `CREATE FUNCTION` as
usual. This change may be generalised in MDEV-31401.

Also factor out some clean-up queries from deinit_spider.inc for use
of spider init tests.

A minor caveat is that early spider initialisation will fail if the
server is bootstrapped for the first time, due to missing `mysql`
database which needs to be created by the bootstrap script.

Removing procedures that were created and dropped during init.

This also fixes a race condition where mtr test with
plugin-load-add=ha_spider.so causes post test check to fail as it
expects the procedures to still be there.

There are several plugins in ha_spider: spider, spider_alloc_mem,
spider_wrapper_protocols, spider_rewrite etc.

INSTALL PLUGIN foo SONAME ha_spider causes all the other ones to be
installed by the init queries where foo is any of the plugins.

This introduces unnecessary complexiy. For example it reads
mysql.plugins to find all other plugins, causing the hack of moving
spider plugin init to a separate thread.

To install all spider related plugins, install soname ha_spider should
be used instead.

This also fixes spurious rows in mysql.plugin when installing say only
the spider plugin with `plugin-load-add=SPIDER=ha_spider.so`:

select * from mysql.plugin;
name	dl
spider_alloc_mem	ha_spider.so # should not be here
spider_wrapper_protocols	ha_spider.so # should not be here

Adapted from part of the reverted commit
c160a115.

We introduce simple plugin dependency. A plugin init function may
return HA_ERR_RETRY_INIT. If this happens during server startup when
the server is trying to initialise all plugins, the failed plugins
will be retried, until no more plugins succeed in initialisation or
want to be retried.

This will fix spider init bugs which is caused in part by its
dependency on Aria for initialisation.

The reason we need a new return code, instead of treating every
failure as a request for retry, is that it may be impossible to clean
up after a failed plugin initialisation. Take InnoDB for example, it
has a global variable `buf_page_cleaner_is_active`, which may not
satisfy an assertion during a second initialisation try, probably
because InnoDB does not expect the initialisation to be called
twice. A test that may fail because of this is
`encryption.corrupted_during_recovery`, see for example[1], which is
tested at 73835f64b7fc245d38812380685aca03bef72bb5, a previous commit
where we retry every failed plugin.

[1] https://buildbot.mariadb.org/#/builders/369/builds/10107/steps/7/logs/stdio

The server needs to have a unique name

Will re-enable once MDEV-31101 is no longer blocked by MDEV-22979,
as the patch for the latter might fix the former.

spider_db_mbase_util::open_item_func() is a monster function.
It is difficult to maintain while it is expected that we need to
modify it when a new SQL function or a new func_type is added.

We split the function into two distinct functions: one handles the
case of str != NULL and the other handles the case of str == NULL.

This refactoring was done in a conservative way because we do not
have comprehensive tests on the function.

It also fixes MDEV-29447 and MDEV-31338 where field items that are
arguments of a func item may be used before created / initialised.

Note this commit is adapted from a patch by Nayuta for MDEV-26285.

Extract the indexed string memcopy pattern in spd_trx.cc to a static
inline function.

Also updated the ubsan check in mdev_26541.test (h/t roel).

When setting the server lockwait timeout, spider should do some basic
checks first, like whether the remote server is still reachable. So
instead of directly calling spider_db_mbase::exec_query(), it should
call spider_db_query().

The reset of the lock wait timeout does not need to do such checks,
because they happen after the successfully setting the lock wait
timeout, implying the checks have been passed already.

This is mainly done to be able to push to 11.0
The test is moved in 10.6 to another file, which will add back the test.

The reason for the crash wad that 'best splitting' optimization
predicted less rows to be found than what opt_range did.
This code in apply_selectivity_for_table(), when using use_cond_selectivity=1,
was not prepared for this case which caused an assert in debug builds.
Production builds is not affected.

The fix is to choose the smaller of the two row counts. It will have a
minimum on costs when using use_cond_selectivity=1 and should not cause
any problems in production.

MDEV-31445 Server crashes in ha_partition::index_blocks / cost_for_index_read

The crash happened in the case where partition pruning finds 0
partitions.

It could cause wrong range estimation for GROUP BY queries that are
using 'WHERE index_part >= constant'.
(The function was trying to check for 'index_part = constant')

Reporter: Yuty Chaikou

The test gcol.gcol_purge would reliably hang on 10.6 on a
Microsoft Windows builder without this adjustment.
A similar adjustment was applied in
commit 3e40f9a7
to the tests innodb.dml_purge and innodb.instant_alter_purge.

innodb_undo_log_truncate_update(): A callback function. If
SET GLOBAL innodb_undo_log_truncate=ON, invoke
srv_wake_purge_thread_if_not_active().

srv_wake_purge_thread_if_not_active(): If innodb_undo_log_truncate=ON,
always wake up the purge subsystem.

srv_do_purge(): If the history is empty, invoke
trx_purge_truncate_history() in order to free undo log pages.

trx_purge_truncate_history(): If head.trx_no==0, consider the
cached undo logs to be free.

trx_purge(): Remove the parameter "bool truncate" and let the
caller invoke trx_purge_truncate_history() directly.

Reviewed by: Vladislav Lesin

purge_sys_t::sees(): Wrapper for view.sees().

trx_purge_truncate_history(): Invoke purge_sys.sees() instead of
comparing to head.trx_no, to determine if undo pages can be safely freed.

The test innodb.cursor-restore-locking was adjusted by Vladislav Lesin,
as was the the debug instrumentation in row_purge_del_mark().

Reviewed by: Vladislav Lesin

This test case exposed 2 different bugs:
- When replacing a range with an index scan on a covering key
  in test_if_skip_sort_order() we didn't disable filtering.
  Filtering does not make much sense in this case.
  - Fixed by disabling filtering in this case.
- Range_rowid_filter::fill() did not take into account that keyread
  could already active, which caused an assert when it tried to
  activate another keyread.
  - Fixed by remembering old keyread state at start and restoring it
    at end.

Other things:
- ha_start_keyread() allowed multiple calls. This is wrong, especially
  as we do no check if the index changed!
  I added an assert() to ensure that we don't call it there is already
  an active keyread.
- ha_end_keyread() always called ha_extra(), even if keyread was not
  active.  Added a check to avoid the extra call.

This patch also fixes
MDEV-31391 Assertion `((best.records_out) == 0.0 ... failed

Cost changes caused by this change:
- range queries with join buffer now have a notable smaller cost.
- range ranges are bit more expensive as the MULTI_RANGE_COST is now
  properly applied to it in all cases (this extra cost is equal to a
  key lookup).
- table scan cost is slight smaller as we now assume data is cached in
  the engine after the first scan pass. (We did this before for range
  scans and other access methods).
- partition tables had wrong values for max_row_blocks and
  max_index_blocks.  Correcting this, causes range access on
  partitioned tables to have slightly higher cost because of the
  increased estimated IO.
- Using first match + join buffer caused 'filtered' to be calcualted
  wrong.  (Only affected EXPLAIN, not query costs).
- Added cost_without_join_buffer to optimizer_trace.
- check_quick_select() adjusted the number of rows according to persistent
  statistics, but did not adjust cost. Now fixed.

The big change in the patch are:

- In best_access_path(), where we now are using storing the cost in
  'ALL_READ_COST cost' and only converting it to a double at the end.
   This allows us to more exactly calculate the effect of the join_cache.
- In JOIN_TAB::estimate_scan_time(), store the cost also in a
  ALL_READ_COST object.

One of effect if this change is that when joining very small tables:

t1    some_access_method
t2    range
t3    ALL         Use join buffer

This is swiched to

t1      some_access_method
t3      ALL
t2      range      use join buffer

Both plans has the same cost, but as table scan in this case has less
cost than rang, the table scan will be considered first and thus have
precidence.

Test case changes:
- optimizer_trace          - Addition of cost_without_join_buffer
- subselect_mat_cost_bugs  - Small tables and scan versus range
- range & range_mrr_icp    - Range + join_cache is faster than ref
- optimizer_trace          - cost_without_join_buffer, smaller scan cost,
                             range setup cost.
- mrr                      - range+join_buffer used as smaller cost