MDEV-17297: stats.records=0 for a table of Archive engine when it has rows, when we run ANALYZE command

Archive storage engine assumed that any query that attempts to read from
the table will call ha_archive::info() beforehand. ha_archive would flush
un-written data in that call (this would make it visible for the reads).

Break this assumption. Flush the data when the table is opened for reading.

This way, one can do multiple write statements without causing a flush, but
as soon as we might need the data, we flush it.

For single table updates and multi-table updates , engine independent statistics were not being
read even if the statistics were collected.
Fixed it, so when the optimizer_use_condition_selectivity > 2 then we would read the available
statistics for update queries.

before:
(gdb) p sizeof(PFS_events_waits)
$1 = 184

after:
(gdb) p sizeof(PFS_events_waits)
$1 = 160

no functional changes

Always set SERVER_MORE_RESULTS_EXIST when executing stored procedure statements

If statements produce a result, EOF packet needs this flag (SP ends
with an OK packet). IF statetement does not produce a result, affected rows
count are part of the final OK packet.

within stored procedure

Always set SERVER_MORE_RESULTS_EXIST when executing stored procedure.
statements

If statements produce a result, EOF packet needs this flag (SP ends with
an OK packet). IF statetement does not produce a result, affected rows
count are part of the final OK packet.

Fixed by adding more rows to a table

Other things:
- Speed up index_merge tests 20% by adding begin/commit around loops that
  generated rows.

For some reason, GCC 8 did not issue warnings for all such comparisons.

InnoDB and XtraDB had redundant assertions for checking that
function parameters that were declared as nonnull were not NULL.

it was supposed to be `*(p-1)` not `*p-1`
(the crash happens if `*p==0`)

in particular, don't search for libjemalloc.a, which is only
needed for tokudb's ftcxx tests, when the tests aren't going
to be built.

`zypper install krb5-devel` installs executables outside of $PATH.
It also installs /etc/profile.d/krb5.sh that is sourced by a new
shell to add the new location to the $PATH. But this doesn't affect
the current shell.

Now decent Linux distros remind the user to run `. /etc/profile`
to reload paths in such a case. SUSE doesn't and for a good reason -
it doesn't work there. Because SUSE sets PROFILEREAD=true in the
environment and /etc/profile does not do anything.

By this point, one should not really expect `unset PROFILEREAD` to help,
and it does not - PROFILEREAD is readonly, and cannot be unset.

Apparently SUSE really *really* wants you to re-login between installing
MariaDB build dependencies and actually running the rpmbuild.

Which we cannot do it buildbot. And it would look very user-un-friendly
in the Build Instructions section of the manual.

So, we work around it - by adding SUSE krb5 path to the search list.

THIS IS SUSEEEEEE!!!

Filter out most common file types from automatic dependency collection.
This makes `make package`  in centos73 VM three times faster

This is follow-up for commit 619d22dd
to fix the cmake -DWITH_EMBEDDED_SERVER build.

InnoDB includes 3 parsers, which use 3 lexical analyzers that
are generated with flex. Flex versions before 2.6 emitted
the keyword "register", which is deprecated in C++17.

The lexical analyzers were regenerated as follows:

for s in storage/innobase storage/xtradb
do
	(cd "$s"/pars; ./make_flex.sh)
	touch "$s"/fts/*.l
	make -C "$s"/fts -f Makefile.query
done

The MDEV-11738/MDEV-11581 fix was supposed to add the column
ROTATING_OR_FLUSHING to the INFORMATION_SCHEMA table
INNODB_TABLESPACES_ENCRYPTION, but it also added that column to
INNODB_TABLESPACES_SCRUBBING in InnoDB (not XtraDB).

The extra column was never initialized. We will remove it,
because key rotation has nothing to do with the scrubbing of
tablespace data.

This reverts commit 21b2fada
and commit 81d71ee6.

The MDEV-18464 change introduces a few data race issues. Contrary to
the documentation, the field trx_t::victim is not always being protected
by lock_sys_t::mutex and trx_t::mutex. Most importantly, it seems
that KILL QUERY could wrongly avoid acquiring both mutexes when
invoking lock_trx_handle_wait_low(), in case another thread had
already set trx->victim=true.

We also revert MDEV-12009, because it should depend on the MDEV-18464
fix being present.

As noted on kill_one_thread SUPER should be able to kill even
system threads i.e. threads/query flagged as high priority or
wsrep applier thread. Normal user, should not able to kill
threads/query flagged as high priority (BF) or wsrep applier
thread.

Pushed the decision for innodb transaction and system
locking down to lock0lock.cc level. With this,
we can avoid releasing these mutexes for executions
where these mutexes were acquired upfront.

This patch will also fix BF aborting of native threads, e.g.
threads which have declared wsrep_on=OFF. Earlier, we have
used, for innodb trx locks, was_chosen_as_deadlock_victim
flag, for marking inodb transactions, which are victims for
wsrep BF abort. With native threads (wsrep_on==OFF), re-using
was_chosen_as_deadlock_victim flag may lead to inteference
with real deadlock, and to deal with this, the patch has added new
flag for marking wsrep BF aborts only: victim=true

Similar way if replication decides to abort one of the threads
we mark victim by: victim=true

innobase_kill_query
	Remove lock sys and trx mutex handling.

wsrep_innobase_kill_one_trx
	Mark victim trx with victim=true

trx0trx.h
	Remove trx_abort_t type and abort type variable from
	trx struct. Add victim variable to trx.

wsrep_kill_victim
	Remove abort_type

lock_report_waiters_to_mysql
	Take also trx mutex and mark trx as a victim for
	replication abort.

lock_trx_handle_wait_low
	New low level function to check whether the transaction
	has already been rolled back because it was selected as
	a deadlock victim, or if it has to wait then cancel
	the wait lock.

lock_trx_handle_wait
	If transaction is not marked as victim take lock sys
	and trx mutex before calling lock_trx_handle_wait_low
	and release them after that.

row_search_for_mysql
	Remove lock sys and trx mutex taking and releasing.

trx_rollback_to_savepoint_for_mysql_low
trx_commit_in_memory
	Clean up victim variable.

MDEV-18466 Unsafe to log updates on tables referenced by foreign keys with triggers in statement format

ignore FK-prelocked tables when looking for write-prelocked tables
with auto-increment to complain about "Statement is unsafe because
it invokes a trigger or a stored function that inserts into an
AUTO_INCREMENT column"

special cases:

* change systemd detection to use CHECK_LIBRARY_EXISTS at least once,
  to have it detected by build_depends.cmake
* similarly, use find_library for pam
* unixODBC is weird, libodbc.so is in the unixODBC package, not
  in the unixODBC-devel, where normally all .so files belong.
  Packaging bug? As a workaround, use find_file(sql.h) instead of
  find_path(sql.h) to make sure that /usr/include/sql.h (not /usr/include)
  is cached by cmake, and later build_depends.cmake will select
  unixODBC-devel, as a package owning /usr/include/sql.h file.

automatic BuildRequires for source RPM: for every FILEPATH and
"Have library XXX" cached variable, detect what rpm package it comes from
and add it to the list of dependencies.

That is, the source RPM will BuildRequire all those packages that
were found by cmake when the source RPM was built. Presumably, our
CMakeLists.txt won't check for libraries that aren't needed for a build.

It supports libraries/executables/files found with
  FIND_LIBRARY
  FIND_FILE
  FIND_PROGRAM
  CHECK_LIBRARY_EXISTS

create source RPM cpack-way

when building binary packages, this source rpm will use
same BUILD_CONFIG and WITH_SSL values that were used when
creating the source RPM.

Only do it for a reasonably new cmake, where
source rpms are known to work (3.10.2 is ok, 3.5.2 is not).

And force a shorter CPACK_RPM_BUILD_SOURCE_DIRS_PREFIX so that
a source rpm could be built from a standard location in /usr/src

instead remove internal modules from Requires/Provides

add defensive $ for filenames, don't include .gitattributes
and *.rpm, correct rules for *.gz and *.zip

since long we use a different workaround, our own CPackRPM wrapper

now we can afford it. Fix -Werror errors. Note:
* old gcc is bad at detecting uninit variables, disable it.
* time_t is int or long, cast it for printf's

Fix mariabackup to crash if opening tablespace fails, insitead of
continuing after an error.

In a Ubuntu Xenial build environment, the compiler identified as
g++-5.real (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
seems to be emitting incorrect code for the compilation unit
trx0rec.cc, triggering a bogus-looking AddressSanitizer report
of an invalid read of something in the function trx_undo_rec_get_pars().
This is potentially affecting any larger tests where the InnoDB
purge subsystem is being exercised.

When the optimization level of trx0rec.cc is limited to -O1, no
bogus failure is being reported. With -O2 or -O3, a lot of things
seemed to be inlined in the function, and the disassembly of the
generated code did not make sense to me.

select from I_S

Problem:
========
When applier thread tries to access 'variable_name' of
INFORMATION_SCHEMA.SESSION_VARIABLES table through triggers, it results in an
abnormal exit of slave server.

Analysis:
========
At the time of replication of stored routines and triggers, their associated
security context will be sent by the master. The applier thread on the slave
server will use this information to set the required security context for the
execution of stored routines and triggers. This is achieved as follows.

->The stored routine object has a member named 'm_security_ctx' which holds the
  security context received from master.
->The applier thread's security_ctx is stored into a 'backup' object.

->Set the applier thread's security_ctx to 'm_security_ctx'.

->Upon the completion of stored routine execution restore the original security
  context of applier thread from the backup.

During the above process the 'm_security_ctx' object is not initialized
properly. Hence the 'external_user' of 'm_security_ctx' has invalid value for
this variable and accessing this variable results in abnormal exit of server.

Fix:
===
Invoke the Security_context::init() call from the constructor of stored routine
so that 'm_security_ctx' gets initialized properly.

Item_cond::eval_not_null_tables(): Use Item::eval_const_cond(),
just like Item_cond::fix_fields().

This inconsistency was found while merging to 10.3, where the
Microsoft compiler is configured to report an error for comparing
longlong to bool.