Backport 7c03edf2 from xtradb to innodb

also cover USE and other built-in commands

"tokudb_alter_table.drop_add_pk_part_104 leaves a temporary file behind"

Fixed by copying 3 lines from 10.1 to 10.0 that cleaned up the temporary
file for partitioning tables.

May also fix: MDEV-14970 "MariaDB crashed with signal 11 and Aria table"

I am not able to reproduce a crash, however there was no protection in
print_keydup_error() if the storage engine reported the wrong key number.

This patch adds such a protection and should stop any further crashes
in this case.

Other things:
- Added extra protection in Aria to not set errkey to more than number of
  keys. (Don't think this is cause of this crash, but better safe than
  sorry)
- Extend test_if_equal_repl_errors() to handle different cases of
  ER_DUP_ENTRY. This is just mainly precaution for the future.

mem_heap_free_heap_top(): Remove UNIV_MEM_ASSERT_W() and unpoison
the memory region first, because part of it may have been poisoned
by an earlier mem_heap_free_top() call.
Poison the address range at the end.

mem_heap_block_free(): Poison the address range at the end.

UNIV_MEM_ASSERT_AND_ALLOC(): Replace with UNIV_MEM_ALLOC().
We want to keep the address ranges poisoned (unaccessible) as
long as possible.

UNIV_MEM_ASSERT_AND_FREE(): Replace with UNIV_MEM_FREE().

MDEV-14957: JOIN::prepare gets unusable "conds" as argument

Do not touch merged derived (it is irreversible)

Fix first argument of in_optimizer for calls possible before fix_fields()

escape all charecters less or equal 0x1F (control symbols)
(shorter sequence are not used to make code simple, long encoding is always legal according to the rfc4627)

Problem:-
 If we create table using myisam/aria then this crashes the server.
  CREATE TABLE t1(a bit(1), b int auto_increment , index(a,b));
  insert into t1 values(1,1);
 Or this query
  CREATE TABLE t1 (b BIT(1), pk INTEGER AUTO_INCREMENT PRIMARY KEY);
  ALTER TABLE t1 ADD INDEX(b,pk);
  INSERT INTO t1 VALUES (1,b'1');
  ALTER TABLE t1 DROP PRIMARY KEY;

Reason:-
 The reason for this is
 1st- find_ref_key() finds what key an auto_increment field belongs to by
  comparing key_part->offset and field->ptr. But BIT fields might have
  zero length in the record, so a key might have many key parts with the
  same offset. That is, comparing offsets cannot uniquely identify the
  correct key part.
 2nd- Since next_number_key_offset is zero it myisam/aria will think that
  auto_increment is in first part of key.
 3nd- myisam/aria will call retrieve_auto_key which will see first key_part
  field as a bit field and call assert(0)

Solution:-
  Many key parts might have the same offset, but BIT fields do not
  support auto_increment. So, we can skip all key parts over BIT fields,
  and then comparing offsets will be unambiguous.

/home/kevg/work/mariadb/sql/sql_partition.cc:286:47: error: cannot initialize a parameter of type 'HA_CREATE_INFO *' (aka 'st_ha_create_information *') with an rvalue of type 'ulonglong' (aka 'unsigned long long')
                                              (ulonglong)0, (uint)0);
                                              ^~~~~~~~~~~~
/home/kevg/work/mariadb/sql/partition_info.h:281:72: note: passing argument to parameter 'info' here
  bool set_up_defaults_for_partitioning(handler *file, HA_CREATE_INFO *info,
                                                                       ^

We don't check for DLSYM in CMake, check for DLOPEN instead.

PCOLRES::Length is the length in characters, not in bytes
(because it's printed as length in "VARCHAR(N)").
So convert it into characters using cs->mbmaxlen.

translate clang __has_feature to gcc macros

InnoDB is issuing a 'noise' message that is not a sign of abnormal
operation. The only issuers of it are the debug function
lock_rec_block_validate() and the change buffer merge.
While the error should ideally never occur in transactional locking,
we happen to know that DISCARD TABLESPACE and TRUNCATE TABLE and
possibly DROP TABLE are breaking InnoDB table locks.

When it comes to the change buffer merge, the message simply is useless
noise. We know perfectly well that a tablespace can be dropped while a
change buffer merge is pending. And the code is prepared to handle that,
which is demonstrated by the fact that whenever the message was issued,
InnoDB did not crash.

fil_inc_pending_ops(): Remove the parameter print_err.

The assertion failure was caused by an incorrectly set read_set for
functions in the ORDER BY clause in part of a union, when we are using
a mergeable view and the order by clause can be skipped (removed).

An order by clause can be skipped if it's part of one part of the UNION as
the result set is not meaningful when multiple SELECT queries are UNIONed. The
server is aware of this optimization and tries to remove the order by
clause before JOIN::prepare. The problem is that we need to throw an
error when the ORDER BY clause contains invalid columns. To do this, we
attempt resolving the ORDER BY expressions, then subsequently drop them
if resolution succeeded. However, ORDER BY resolution had the side
effect of adding the expressions to the all_fields list, which is used
to construct temporary tables to store the result. We may be ignoring
the ORDER BY statement, but the tmp table still tried to compute the
values for the expressions, even if the columns are never used.

The assertion only shows itself if the order by clause contains members
which were not previously in the select list, and are part of a
function.

There is an additional question as to why this only manifests when using
VIEWS and not when using a regular table. The difference lies with the
"reset" of the read_set for the temporary table during
SELECT_LEX::update_used_tables() in JOIN::optimize(). The changes
introduced in fdf789a7 cleared the
read_set when a mergeable view is encountered in the TABLE_LIST
defintion.

Upon initial order_list resolution, the table's read_set is updated
correctly. JOIN::optimize() will only reset the read_set if it
encounters a VIEW. Since we no longer have ORDER BY clause in
JOIN::optimize() we never get to correctly update the read_set again.

Other relevant commit by Timour, which first introduced the order
resolution when we "can_skip_sort_order":
883af99e

Solution:
Don't add the resolved ORDER BY elements to all_fields. We only resolve
them to check if an error should be returned for the query. Ignore them
completely otherwise.

It has its limitations, e.g. it assumes that there's only one
gdb and only one valgrind process is running. And a hard-coded
one-second delay might be too short for slow machines.

Still, it's better than "doesn't work at all"

instrument table->record[0], table->record[1] and share->default_values.

One should not access record image beyond share->reclength, even
if table->record[0] has some unused space after it (functions that
work with records, might get a copy of the record as an argument,
and that copy - not being record[0] - might not have this buffer space
at the end). See b80fa400 and 444587d8

more complete TRASH-ing of memroots

mark freed memory as not accessible, not merely undefined

TRASH was mapped to TRASH_FREE and was supposed to be used for memory
that should not be accessed anymore, while TRASH_ALLOC() is to be
used for uninitialized but to-be-used memory.

But sometimes TRASH() was used in the latter sense.

Remove TRASH() macro, always use explicit TRASH_ALLOC() or TRASH_FREE().

InnoDB limited the maximum number of bytes per character to 4.
But, the filename character set that was introduced in MySQL 5.1
uses up to 5 bytes per character.

To allow InnoDB tables to be created with wider characters, let
us split the mbminmaxlen fields into mbminlen, mbmaxlen, and increase
the limit to 7 bytes per character. This will increase the payload size
of dtype_t and dict_col_t by one bit. The storage size will be unchanged
(54 bits and 77 bits will use the same number of bytes as the
previous sizes 53 and 76 bits).

Resolving a stacktrace including functions in dynamic libraries requires
us to look inside the libraries for the symbols. Addr2line needs to be
started with the correct binary for each address on the stack. To do this,
figure out which library it is using dladdr, then if the addr2line
binary was started with a different binary, fork it again with the
correct one.

We only have one addr2line process running at any point during the
stacktrace resolving step. The maximum number of forks for addr2line should
generally be around 6.

One for server stacktrace code, one for plugin code, one when going back
into server code, one for pthread library, one for libc, one for the
_start function in the server. More can come up if plugin calls server
function which goes back to a plugin, etc.

In this case we were using the optimization derived_with_keys but we could not create a key
because the length of the key was greater than the max allowed(MI_MAX_KEY_LENGTH).
To do the join we needed to create a hash join key instead, but in the explain output it
showed that we were still referring to derived keys which were created but not used.

In the function JOIN::shrink_join_buffers the iteration over joined
tables was organized in a wrong way. This could cause a crash if
the optimizer chose to materialize a semi-join that used join caches
for which the sizes must be adjusted.