with joins, SQ, ORDER BY, semijoin=on

A bug in get_sort_by_table() could mislead the function
setup_semijoin_dups_elimination(). As a result the optimizer
could produce invalid execution plans for queries with ORDER BY
and subquery predicates that could be converted to semi-joins.

Remove non prepared (and so belonging to removed clauses FT functions) from the list.

in later version it will be fixed by building the list during preparation.

Signed-off-by: Daniel Black <daniel@linux.vnet.ibm.com>

Fix the call to correspond protocoll of pagecache call.
Fix of misleading variables names.

This bug happens when locking the same Aria "transactional" table
(page format) more then once with LOCK TABLES and inserting into one
of them with INSERT ... SELECT when the table is empty.

Fixed by ensuring we don't use fast bulk insert if table is opened
twice with LOCK TABLES (as this changes table->s->state)

Code changes:
- Added use_count to MARIA_USED_TABLES to be able to check if
  table is opened twice for a statement/lock table
- Don't clear history or reset info->start_state if we
  don't have versioning. One reason for the bug was
  was that info->start_state was set to point to different
  states for the two tables.  If there is no versioning
  info->start_state should always point to info->s->state.common.

Other things:
- Fixed also some typos that was noticed while scanning the code
- More DBUG_PRINT

The warning was originally added in
commit c6766305
(MySQL 4.1.12, 5.0.3) to trace claimed undo log corruption that
was analyzed in https://lists.mysql.com/mysql/176250
on November 9, 2004.

Originally, the limit was 20,000 undo log headers or transactions,
but in commit 9d6d1902
in MySQL 5.5.11 it was increased to 2,000,000.

The message can be triggered when the progress of purge is prevented
by a long-running transaction (or just an idle transaction whose
read view was started a long time ago), by running many transactions
that UPDATE or DELETE some records, then starting another transaction
with a read view, and finally by executing more than 2,000,000
transactions that UPDATE or DELETE records in InnoDB tables. Finally,
when the oldest long-running transaction is completed, purge would
run up to the next-oldest transaction, and there would still be more
than 2,000,000 transactions to purge.

Because the message can be triggered when the database is obviously
not corrupted, it should be removed. Heavy users of InnoDB should be
monitoring the "History list length" in SHOW ENGINE INNODB STATUS;
there is no need to spam the error log.

Roll back to most general duplicate removing strategi in case of different stratagies for one position.

Backport the fix from 10.0.33 to 5.5, in case someone compiles XtraDB
with -DUNIV_LOG_ARCHIVE

The XtraDB option innodb_track_changed_pages causes
the function log_group_read_log_seg() to be invoked
even when recv_sys==NULL, leading to the SIGSEGV.

This regression was caused by
MDEV-11027 InnoDB log recovery is too noisy

trx_undo_rec_get_partial_row(): When the PRIMARY KEY includes a
column prefix of an externally stored column, the already parsed
part of the undo log record may contain a reference to
an off-page column. This is the case in the bug58912 test in
innodb.innodb.

This is a regression caused by MDEV-14051 'Undo log record is too big.'

Purge in the secondary index is wrongly skipped in
row_purge_upd_exist_or_extern() because node->row only does not contain all
indexed columns.

trx_undo_rec_get_partial_row(): Add the parameter for node->update
so that the updated columns will be copied from the initial part
of the undo log record.

* don't use Env module in tests, use $ENV{xxx} instead
* collateral changes:
** $file in the error message was unset
** $file in the other error message was unset too :)
** source file arguments are conventionally upper-cased
** abort the test (die) on error, don't just echo/exit

If translation table present when we materialize the derived table then
change it to point to the materialized table.

Added debug info to see really what happens with what derived.

In the function make_sortkey a tmp buffer was defined and in the absence of
param->tmp_buffer, tmp buffer used the sort_keys buffer. sort_keys buffer
has a length defined in sort_field->length, while param->tmp_buffer is
stored in param->rec_length. Make sure to use the appropriate length
based on which buffer we are using otherwise we'll overflow.

Also added a type cast to size_t during the calculation of the sort keys
buffer size to avoid an oveflow if the buffer size exceeds 32 bits.

Correctly print separator string in single quotes.

This reverts commit 7603463a.

The commit itself is fine, however when disabling volatile, compiler
optimizations mess up our double results due to precision differences.
Revert the patch till a proper solution is found.

This was added in c7964159 but would hurt all other compilers
because of Visual Studio. Hopefully this has been fixed now.
Signed-off-by: Daniel Black <daniel@linux.vnet.ibm.com>

For BIT field null_bit is not set to 0 even for a field defined as NOT NULL.
So now in the function TABLE::create_key_part_by_field, if the bit field is not nullable
then the null_bit is explicitly set to 0

MariaDB adjustments to test case innodb-replace-debug.

Imported missing test case from MySQL 5.7 for

commit 25781c154396dbbc21023786aa3be070057d6999
Author: Annamalai Gurusami <annamalai.gurusami@oracle.com>
Date:   Mon Feb 24 14:00:03 2014 +0530

    Bug #17604730 ASSERTION: *CURSOR->INDEX->NAME == TEMP_INDEX_PREFIX

MariaDB 5.5 does not seem to be affected.

uses alias in HAVING when sql_mode = 'ONLY_FULL_GROUP_BY'

This patch corrects the patch for bug#18739: non-standard
HAVING extension was allowed in strict ANSI sql mode
added in 2006 by commit 4b7c4cd2.
As a result of incompleteness of the fix in the above commit
if a query with GROUP BY contained an aggregate function with an
alias and this alias was used in the HAVING clause of the query
the server reported an error when sql_mode was set to
'ONLY_FULL_GROUP_BY'.

don't close stdout/stderr, redirect them to /dev/null instead.
otherwise redirections like >&2 fail with "invalid file descriptor"

present.

        thread_pool_server_audit.test fixed.
        plugin version updated.

present.

        Escape special characters (like \r \n \t) instead of
        replacing them with spaces.

Replace all references in InnoDB and XtraDB error log messages
to bugs.mysql.com with references to https://jira.mariadb.org/.

InnoDB was writing unnecessary information to the
update undo log records. Most notably, if an indexed column is updated,
the old value of the column would be logged twice: first as part of
the update vector, and then another time because it is an indexed column.

Because the InnoDB undo log record must fit in a single page,
this would cause unnecessary failure of certain updates.

Even after this fix, InnoDB still seems to be unnecessarily logging
indexed column values for non-updated columns. It seems that non-updated
secondary index columns only need to be logged when a PRIMARY KEY
column is updated. To reduce risk, we are not fixing this remaining flaw
in GA versions.

trx_undo_page_report_modify(): Log updated indexed columns only once.

based on:

commit f7316aa0
Author: Ajo Robert <ajo.robert@oracle.com>
Date:   Thu Aug 24 17:03:21 2017 +0530

    Bug#26361149  MYSQL SERVER CRASHES AT: COL IN(IFNULL(CONST,
                           COL), NAME_CONST('NAME', NULL))

    Backport of Bug#19143243 fix.

    NAME_CONST item can return NULL_ITEM type in case of incorrect arguments.
    NULL_ITEM has special processing in Item_func_in function.
    In Item_func_in::fix_length_and_dec an array of possible comparators is
    created. Since NAME_CONST function has NULL_ITEM type, corresponding
    array element is empty. Then NAME_CONST is wrapped to ITEM_CACHE.
    ITEM_CACHE can not return proper type(NULL_ITEM) in Item_func_in::val_int(),
    so the NULL_ITEM is attempted compared with an empty comparator.
    The fix is to disable the caching of Item_name_const item.

Partition wasn't setting HA_OPTION_PACK_RECORD on ALTER TABLE
if the row format was PAGE.

(so one bit in the null bitmap was reserved for a deleted bit -
see make_empty_rec - and all actual null bits were one off)

if it's a DROP TABLE, we cannot detect whether a table is
temporary by looking in thd->temporary_tables - because the
table might simply not exist at all.

backport ce6c0e58

MDEV-8960: Can't refer the same column twice in one ALTER TABLE

Problem was that if column was created in alter table when
it was refered again it was not tried to find from list
of current columns.

mysql_prepare_alter_table:
  There is two cases
    (1) If alter table adds a new column and then later alter
        changes the field definition, there was no check from
	list of new columns, instead an incorrect error was given.
    (2) If alter table adds a new column and then later alter
        changes the default, there was no check from list of
	new columns, instead an incorrect error was given.

always search in compile-time specified paths
INSTALL_BINDIR, INSTALL_SBINDIR, INSTALL_MYSQLSHAREDIR. User
can set them to arbitrary values, it's not enough to search only
in their usual values of bin, sbin and libexec, share and share/mysql.

1. detect resolveip location, don' assume it's in $basedir/bin
2. don't guess $scriptdir to (incorrectly) construct the $0 path
3. rename find_in_basedir -> find_in_dirs, don't prepend $basedir
   automatically. This allows to use identical path lists in
   find_in_dirs and in cannot_find_file.
4. move search path lists to CMakeLists.txt to avoid specifying the
   same path list twice (in find_in_dirs and in cannot_find_file).

mostly caused by -Wimplicit-fallthrough