MDEV-18992 Crash when using 'insert into on duplicate update'if session charset different from table charset (#1290)

This reverts commit 81f6a3b0.

           depends on uninitialised value

Initialized THD::force_read_stats introduced in the patch for MDEV-17605.
Leaving this field uninitialized in the constructor of the THD class may
trigger reading statistical data that is not needed.

These were broken in commit b5615eff

Adjust a test for the merge commit e6bdf77e

Test for MDEV-11874 "Data too long for column" instead of "Invalid default value for" upon ALTER in strict mode

This problem was earlier fixed by the patch for:
    MDEV-16421 Make system tables crash safe

Adding tests only.

This is fixup for the merge commit e6bdf77e

FIXME: Find out why the results differ between release and debug builds.

If a select query was of the form (SELECT ... ORDER BY ...) LIMIT ...
then in most cases it returned incorrect result. It happened because
SELECT ... ORDER BY ... was wrapped into a select with materialized
derived table:
  SELECT ... ORDER BY ... =>
  SELECT * FROM (SELECT ... ORDER BY ...) dt.
Yet for any materialized derived table ORDER BY without LIMIT is ignored.
This patch resolves the problem by the conversion
  (SELECT ... ORDER BY ...) LIMIT ... =>
  SELECT ... ORDER BY ... LIMIT ...
at the parser stage.

Similarly
  ((SELECT ... UNION ...) ORDER BY ...) LIMIT ...
is converted to
  (SELECT ... UNION ...) ORDER BY ... LIMIT ...
This conversion optimizes execution of the query because the result of
(SELECT ... UNION ...) ORDER BY ... is not materialized into a temporary
table anymore.

Add const_table support to the direct join feature and condition pushdown feature

In is_eits_usable(), we disable an assertion that fails due to
MDEV-19334.

Part of MDEV-7974 - backport fix for mysql bug#12161 (XA and binlog)

Part of MDEV-7974 - backport fix for mysql bug#12161 (XA and binlog)

Part of MDEV-7974 - backport fix for mysql bug#12161 (XA and binlog)

Simplified away XA_NOTR, use XID_STATE::is_explicit_XA() instead.

Part of MDEV-7974 - backport fix for mysql bug#12161 (XA and binlog)

Let xid_cache_insert()/xid_cache_delete() handle xa_state.

Let session tracker use is_explicit_XA() rather than xa_state != XA_NOTR.

Fixed open_tables() to refuse data access in XA_ROLLBACK_ONLY state.

Removed dead code from THD::cleanup(). It was supposed to be a reminder,
but it got messed up over time.

spider_internal_start_trx() is called either with XA_NOTR or XA_ACTIVE,
which is guarded by server callers. Thus is_explicit_XA() is acceptable
replacement for XA_ACTIVE check (which was likely wrong anyway).

Setting xa_state to XA_PREPARED in spider_internal_xa_prepare() isn't
meaningful, as this value is never accessed later. It can't be accessed
by current thread and it can't be recovered either. It can only be
accessed by spider internally, which never happens.

Make spider_xa_lock()/spider_xa_unlock() static.

Part of MDEV-7974 - backport fix for mysql bug#12161 (XA and binlog)

XID_STATE::rm_error is never used by internal 2PC, it is intended to be
used by explicit XA only.

Also removed redundant xid reset from THD::init_for_queries(). Must've
been done already either by THD::transaction constructor or by
THD::cleanup().

Part of MDEV-7974 - backport fix for mysql bug#12161 (XA and binlog)

Part of MDEV-7974 - backport fix for mysql bug#12161 (XA and binlog)

Adding new virtual methods in Field:
- make_empty_rec_store_default_value()
- make_empty_rec_reset()

This simplifies the logic for every Field type,
and makes the code more friendly to pluggable data types.

InnoDB could return the same list again and again if the buffer
passed to trx_recover_for_mysql() is smaller than the number of
transactions that InnoDB recovered in XA PREPARE state.

We introduce the transaction state TRX_PREPARED_RECOVERED, which
is like TRX_PREPARED, but will be set during trx_recover_for_mysql()
so that each transaction will only be returned once.

Because init_server_components() is invoking ha_recover() twice,
we must reset the state of the transactions back to TRX_PREPARED
after returning the complete list, so that repeated traversals
will see the complete list again, instead of seeing an empty list.
Without this tweak, the test main.tc_heuristic_recover would hang
in MariaDB 10.1.

            failed in compare_order_elements function

The issue here is the function compare_order_lists() is called for the order by list of the window functions
so that those window function that can be computed together are adjacent.
So in the function compare_order_list we iterate over all the elements in the order list of the two functions and
compare the items in their order by clause.
The function compare_order_elements() is called for each item in the
order by clause. This function assumes that all the items that are in the order by list would be of the type
Item::FIELD_ITEM.

The case we have is that we have constants in the order by clause. We should ignore the constant and only compare
items of the type Item::FIELD_ITEM in compare_order_elements()

Problem:
========
The mysqlbinlog tool is leaking memory, causing failures in various tests when
compiling and testing with AddressSanitizer or LeakSanitizer like this:

cmake -DCMAKE_BUILD_TYPE=Debug -DWITH_ASAN:BOOL=ON /path/to/source
make -j$(nproc)
cd mysql-test
ASAN_OPTIONS=abort_on_error=1 ./mtr --parallel=auto rpl.rpl_row_mysqlbinlog

CURRENT_TEST: rpl.rpl_row_mysqlbinlog

Direct leak of 112 byte(s) in 1 object(s) allocated from:
#0 0x4eff87 in __interceptor_malloc (/dev/shm/5.5/client/mysqlbinlog+0x4eff87)
#1 0x60eaab in my_malloc /mariadb/5.5/mysys/my_malloc.c:41:10
#2 0x5300dd in Log_event::read_log_event(char const*, unsigned int, char const**,
   Format_description_log_event const*, char) /mariadb/5.5/sql/log_event.cc:1568:
#3 0x564a9c in dump_remote_log_entries(st_print_event_info*, char const*)
/mariadb/5.5/client/mysqlbinlog.cc:1978:17

Analysis:
========
'mysqlbinlog' tool is being used to read binary log events from a remote server.
While reading binary log, if a fake rotate event is found following actions are
taken.

If 'to-last-log' option is specified, then fake rotate event is processed.
In the absence of 'to-last-log' skip the fake rotate event.

In this skipped case the fake rotate event object is not getting cleaned up
resulting in memory leak.

Fix:
===
Cleanup the fake rotate event.

This issues is already fixed in MariaDB 10.0.23 and higher versions as part of
commit c3018b0f

           with GROUP BY + ORDER BY

The method JOIN::create_postjoin_aggr_table() should not call
call JOIN::add_sorting_to_table() unless the first non-constant join
table is passed as the first parameter to the method.

dict_create_foreign_constraints_low(): Tolerate the keywords
IGNORE and ONLINE between the keywords ALTER and TABLE.

We should really remove the hacky FOREIGN KEY constraint parser
from InnoDB.

Otherwise, it is possible for "prepare" to exit with error code 0,
even if it did not work at all.

MDEV-9465 The constructor StringBuffer(const char *str, size_t length, const CHARSET_INFO *cs) looks suspicious

Removing the suspicious constructor, it's not used in the code anyway.

The command SHOW INDEXES ignored setting of the system variable
use_stat_tables to the value of 'preferably' and and showed statistical
data received from the engine. Similarly queries over the table
STATISTICS from INFORMATION_SCHEMA ignored this setting. It happened
because the function fill_schema_table_by_open() did not read any data
from statistical tables.

In the case of error when object shapes are half-collected we need
to set the NULL at the vertice's list.

The bug appears because not all conditions are found to be knowingly
true or false in WHERE after HAVING pushdown optimization.
Impossible WHERE can be found much earlier compared with how it is done now.

To fix it and_new_conditions_to_optimized_cond() is changed.