Spiral Patch 057: 057_mariadb-10.2.0.partition_index_end.diff MDEV-12999

Original author: Kentoku SHIBA
First reviewer:  Jacob Mathew
Second reviewer: Michael Widenius

- In Spider, calling cmp_ref() can be very expensive. In ha_partition.cc
  we don't anymore sort rows according to position for the Spider
  engine.
- Removed Spider specific call info(HA_EXTRA_STARTING_ORDERED_INDEX_SCAN)
  from handle_ordered_index_scan(). It's caused performance issues and
  does not change results for queries with ORDER BY.
- The visible effect of this patch is that for some storage engines,
  rows may be returned in a different order if there is no ORDER BY clause.

- Based in Spiral Patch 052:
  052_mariadb-10.2.0.add_partition_skip_pk_sort_for_non_clustered_index.diff
  MDEV-7748
- The major difference from original patch is that there is no variable to
  get the old behaviour.

- Other things:
  - Optimized ha_partition::cmp_ref() and cmp_part_ids() to make them
    simpler and faster.
  - Changed arguments to cmp_key_part_id() to be same as cmp_key_rowid_part_id
    to simplify code.

Original author: Kentoku SHIBA
First reviewer:  Jacob Mathew
Second reviewer: Michael Widenius

Contains Spiral patches:
- Spiral Patch 022: 022_mariadb-10.2.0.auto_increment.diff          MDEV-7720
- Spiral Patch 030: 030_mariadb-10.2.0.partition_auto_inc_init.diff MDEV-7726

These patches have the following differences compared to the original
patches:
- Added the new #defines for the feature in spd_environ.h instead of in
  handler.h because these #defines are needed by Spider and are not needed by
  the server.
- Cleaned up code related to the removed variable m_need_info_for_auto_inc
. Changed variable assignment in lock_auto_increment() and
  unlock_auto_increment() so that the assignments are done under locks.
- Added a test case.
- Added test result changes resulting from a bug that was fixed by these
  patches.

Original author: Kentoku SHIBA
First reviewer:  Jacob Mathew
Second reviewer: Michael Widenius

Contains Spiral patches:
- Spiral Patch 007: 007_mariadb-10.2.0.partition_fulltext.diff  MDEV-7705
- Spiral Patch 038: 038_mariadb-10.2.0.partition_fulltext2.diff MDEV-7734

- This commit has the following differences compared to the original patches:
  - Added necessary full text search cleanup at the storage engine layer
    that was omitted in the original patch.
  - Added test case.
- A lot of code cleanups to make the code notable smaller.
- Changed SQL code to use ha_ft_end() instead of ft_end()

Original author: Kentoku SHIBA
First reviewer:  Jacob Mathew
Second reviewer: Michael Widenius

Spider patches 026 (MDEV-7723), 031 (MDEV-7727) and 058 (MDEV-12532)

This allows the storage engine to internally compute sum and count
operations.

- Enhance sum items to be able to store the sum value directly.
- return_record_by_parent() is enabled in spider as
  HANDLER_HAS_DIRECT_AGGREGATE is defined
- Added spd_environ.h to spider. This is loaded first to ensure that all
  MariaDB specific defines that are used by include files are properly
  defined.
- This code is tested by the existing spider tests direct_aggregate.test
  and direct_aggregate_part.test and also partition.test

Done to simplify spider patch for direct aggregates

Other things:
- Cleanup of allocated bitmaps done in open(), which
  simplifies init_partition_bitmaps()
- Add needed defines in ha_spider.cc to enable new spider code
- Fixed some DBUG_PRINT() to be consistent with normal code
- Removed end space
- The changes in test cases partition_innodb, partition_range,
  partition_pruning etc are becasue partitions can now more exactly calculate
  the number of rows in a range.

Contains spider patches:
014,015,023,033,035,037,040,042,044,045,049,050,051,053,059

Allow partition engines to access table->s->connect_string for engines
that uses HTON_CAN_READ_CONNECT_STRING_IN_PARTITION
Don't reset table->s->connect_string in ha_partition::open

- Support HA_EXTRA_ADD_CHILDREN_LIST,HA_EXTRA_ATTACH_CHILDREN,
  HA_EXTRA_IS_ATTACHED_CHILDREN and HA_EXTRA_DETACH_CHILDREN
  in partition handler for handlers that has HA_CAN_MULTISTEPL_MERGE flag
- Added HA_CAN_MULTISTEPL_MERGE to MERGE handler.
- Added handler::get_child_handlers()
- Change m_num_lock to contain total number of locks. This was needed as we
  now adjust number of locks when extra(HA_EXTRA_ATTACH_CHILDREN) is called.

- Enable HA_EXTRA_WRITE_CAN_REPLACE and HA_EXTRA_WRITE_CANNOT_REPLACE for
  partition engine.

- Added cond_push() and cond_pop() to ha_partition.cc

013_mariadb-10.0.15.vp_handler.diff
034_mariadb-10.0.15.vp_handler2.diff
005_mariadb-10.0.15.hs.diff
041_mariadb-10.0.15.vp_handler2.diff
+ Fixes from Kentoku
+ Added handler/suite.pm and handler/suite.opt to be able to run test cases
  in spider/handler

trx_undo_assign_undo(): Restore the fault injection.

Import the test innodb.alter_crash

The redo log is divided into 512-byte blocks. Sometimes, some of
the strings end up at a block boundary and will not be found.
Look for only one match.

full server recovery is performed .
We start "mysqld" with  --bootstrap
and pass bootstrap script consisting of several FLUSH TABLES FOR export/
UNLOCK TABLES

recv_reset_logs(): Initialize the redo log buffer, so that no data
from the old redo log can be written to the new redo log.

This bug has very little impact before MariaDB 10.2. The
innodb_log_encrypt option that was introduced in MariaDB 10.1
increases the impact. If the redo log used to be encrypted, and
it is being resized and encryption disabled, then previously
encrypted data could end up being written to the new redo log
in clear text. This resulted in encryption.innodb_encrypt_log
test failures in MariaDB 10.2.

The SEARCH_ABORT logic was not working as intended
(tests were not being aborted).

In XtraDB, buf_block_get_frame() can return NULL, and this value
can be passed to buf_page_print(). Do not declare the parameter
as nonnull.

The files ibtmp1 and ib_buffer_pool are sometimes created, sometimes not.
Remove them if they exist.

MDEV-13523: Group By in a View, called within a Stored Routine causes Error Code 1356 when a non-root user runs the routine for a second time

it is actually test suite for second code chunk fix in MDEV-13439

The bug only affects ROW_FORMAT=DYNAMIC tables.
It is reproducible already with 2 records in the table.

Keep testing with ROW_FORMAT=REDUNDANT just in case.

The bug only affected innodb_page_size=64k. There is no point
running the test with smaller page sizes.

buf_page_print(): Remove the parameter 'flags',
and when a server abort is intended, perform that in the caller.

In this way, page corruption reports due to different reasons
can be distinguished better.

This is non-functional code refactoring that does not fix any
page corruption issues. The change is only made to avoid falsely
grouping together unrelated causes of page corruption.

Rdb_io_watchdog can not be built on  OS X. Disable it.

Was reported as 'rpl.rpl_gtid_stop_start fails with Valgrind in buildbot and outside'.
The 'Conditional jump or move depends on uninitialized value' valgrind complaint is valid
and means THD::m_current_stage_key that is not initialized indeed by constructor.

Fixed with its initialization added to the initializer list of
THD::THD.  Double checked with mysql to have found a similar fixes for
the very same issue which was discovered after P_S merge to Maria
(60589aee where the member is introduced) had been done.

A new $MYSQLD_LAST_CMD evaluation was too late in case --manual-gdb.
Now it is done before the server restart type branches which is safe
and the args value has been fully computed by the new point of
evaluation.

save thd->select_number between parsing and executions (in case it was not complete executed due to errors (for example epsent table))

This is a backport of the following:

MDEV-13009 10.1.24 does not compile on architectures without 64-bit atomics

Add a missing #include "sync0types.h" that was removed in MDEV-12674.

* Fix bash expansion issue in wsrep_sst_rsync. https://github.com/docker-library/mariadb/issues/126

* Fix permissions