MDEV-16374: Filtered shows 0 for materilization scan for a semi join, which makes optimizer always picks
materialization scan over materialization lookup

For non-mergeable semi-joins we don't store the estimates of the IN subquery in table->file->stats.records.
In the function TABLE_LIST::fetch_number_of_rows, we store the number of rows in the tables
(estimates in case of derived table/views).
Currently we don't store the estimates for non-mergeable semi-joins, which leads to a problem of selecting
materialization scan over materialization lookup.
Fixed this by storing these estimated appropriately

Fixed  CreateFile() for the log file
- TRUNCATE_EXISTING does not create a new file, use CREATE_ALWAYS
- Make log file handle inheritable

InnoDB insisted on closing the file handle before renaming a file.
Renaming a file should never be a problem on POSIX systems. Also on
Windows it should work if the file was opened in FILE_SHARE_DELETE
mode.

fil_space_t::stop_ios: Remove. We no longer need to stop file access
during rename operations.

fil_mutex_enter_and_prepare_for_io(): Remove the wait for stop_ios.

fil_rename_tablespace(): Remove the retry logic; do not close the
file handle. Remove the unused fault injection that was added along
with the DATA DIRECTORY functionality (MySQL WL#5980).

os_file_create_simple_func(), os_file_create_func(),
os_file_create_simple_no_error_handling_func(): Include FILE_SHARE_DELETE
in the share_mode. (We will still prevent multiple InnoDB instances
from using the same files by not setting FILE_SHARE_WRITE.)

i_s_innodb_buffer_page_fill(), i_s_innodb_buf_page_lru_fill():
Only invoke Field::set_notnull() if the index was found.

MDEV-16123 ASAN heap-use-after-free handler::ha_index_or_rnd_end
MDEV-13828 Segmentation fault on RENAME TABLE

Problem was that destructor called methods for closed table.
Fixed by removing code in destructor.

Problem was that handle_if_exists_options() didn't correct
alter_info->flags when things was removed from the list.

Problem was that detection of temporary tables was all wrong for
RENAME TABLE.
(Temporary tables where opened by top level call to
open_temporary_tables(), which can't detect if a temporary table
was renamed to something and then reused).

Fixed by adding proper parsing of rename list to check against
the current name of a table at each rename stage.
Also change do_rename_temporary() to check against the current
state of temporary tables, not according to the state of start
of RENAME TABLE.

MDEV-10130 Assertion `share->in_trans == 0' failed in storage/maria/ma_close.c
MDEV-10378 Assertion `trn' failed in virtual int ha_maria::start_stmt

The problem was that maria_handler->trn was not properly reset
at commit/rollback and ha_maria::exernal_lock() could get confused
because.

There was some old code in ha_maria::implicit_commit() that tried
to take care of this, but it was not bullet proof.

Fixed by adding list of all tables that is part of the maria transaction to
TRN.

A nice side effect was of the fix is that loops in
ha_maria::implict_commit() got to be much simpler.

Other things:
- Fixed a bug in mysql_admin_table() where argument open_for_modify
  was wrongly reset for the next table in the chain
- rollback admin command also in case of fatal error.
- Split _ma_set_trn_for_table() to three version to simplify code
  and debugging.
- Several new asserts to detect the original problem (that file was
  not properly removed from trn before calling ma_close())

fix a typo that broke the main.view test

followup for ef295c31

It should work ok on all Unixes, but on Windows ,only worked by accident
in the past, with client not being Unicode safe.

It stopped working with Visual Studio 2017 15.7 update now.

When Item_insert_value needs a dummy field,
use zero-length Field_string, not Field_null.
The latter isn't compatible with CREATE ... SELECT.

Fixed by extending unique_table() with a flag to not allow usage of
the replaced table.

I also cleaned up find_dup_table() to not use goto next.
I also added more comments to the code in find_dup_table()

Analyze core independently of max-save-datadir and max-save-core setting.

Increment $num_saved_cores only if core was actually saved.

"Move any core files from e.g. mysqltest" independently of
max-save-datadir setting. Note: it may overwrite core from mysqld, which
might not be desired (it did work this way even before).

- Added missing test case for MyISAM

Problem was that if copy_data_between_tables() didn't do proper
clean up in case of failures:
- copy object was not properly freed
- end_bulk_insert() was not called
- mysql_trans_prepare_alter_copy_data() set THD->transaction.on to
  false which was not properly restored

The last part caused a crash in Aria as Aria depends on that THD
is correct.

Other things:
- Reset info->switched_transactional after usage (safety)
- Reset bulk_insert_single_undo (safety)

MDEV-654 Assertion `share->now_transactional' failed in flush_log_for_bitmap on concurrent workload with Aria tables

Problem was that we the bitmap needs to be flushed before disabling
logging of redo entires, as writing the bitmap to disk by
background checkpoint may cause redo entries.

Problem:
When FTS index is added into a table which doesn't have 'FTS_DOC_ID'
column, Innodb rebuilds table to add column 'FTS_DOC_ID'. when this FTS
index is dropped from this table. Innodb doesn't not rebuild table to
remove 'FTS_DOC_ID' column and deletes FTS index auxiliary tables.
But it doesn't delete FTS common auxiliary tables.
Later when the database having this table is renamed, FTS auxiliary
tables are not renamed because table's flags2 (dict_table_t.flags2)
has been resetted for DICT_TF2_FTS flag during FTS index drop operation.
Now when we drop old database, it leads to an assert.

Fix:
During renaming of FTS auxiliary tables, ORed a condition to check if
table has DICT_TF2_FTS_HAS_DOC_ID flag set.

RB: 18769
Reviewed by : Jimmy.Yang@oracle.com

Problem:
=======
Multiple insert statement in table contains FULLTEXT KEY and a
FTS_DOC_ID column aborts the server if the FTS_DOC_ID exceeds
FTS_DOC_ID_MAX_STEP.

Solution:
========
Remove the exception for first committed insert statement.

Reviewed-by: Jimmy Yang<jimmy.yang@oracle.com>
RB: 18023

When Oracle fixed MDEV-13899 in their own way, they moved the
condition to the only caller of PageConverter::update_records().
Thus, the merge of 5.6.40 into MariaDB added a redundant condition.

PageConverter::update_records(): Move the page_is_leaf() condition
to the only caller, PageConverter::update_index_page().

QUERY_DML_NO_SELECT flag added.

QUERY_DML_NO_SELECT flag added.

The problem is hard to repeat, and I failed to create a deterministic
test case. Online index creation creates stubs for to-be-created indexes.
If index creation fails, we could remove these stubs while locks exist
in the indexes. (This would require that the index creation was completed,
and a concurrent DML operation acquired a lock on a record in the
uncommitted index. If a duplicate key error occurs in an uncommitted
index, the error will be reported for the CREATE UNIQUE INDEX, not for
the DML operation that tried to insert the duplicate.)

dict_table_try_drop_aborted(), row_merge_drop_indexes(): If transactional
locks exist on the table, keep the table->indexes intact.

MDEV-15216 Assertion `! is_set() || m_can_overwrite_status' failed in Diagnostics_area::set_error_status upon operation inside XA

don't implicitly commit or rollback in mysql_admin_table()
unless the statement has CF_IMPLICIT_COMMIT_END flag.

ha_innobase::commit_inplace_alter_table(): Defer the freeing of ctx->trx
until after the operation has been successfully committed. In this way,
rollback on a partitioned table will be possible.

rollback_inplace_alter_table(): Handle ctx->new_table == NULL when
ctx->trx != NULL.

storage/connect/tabjson.cpp:198:10: warning: converting to non-pointer type ‘int’ from NULL [-Wconversion-null]
   return NULL;

Call _exit() from signal handler. main() can just do return.

Merge a fix from MySQL 8.0.11.

row_log_table_apply_delete(): Remove the unused parameter thr.
(This removal was not part of the Oracle fix.)

Issue:
------
Prefix for externally stored columns were being stored in online_log when a
table is altered and alter causes table to be rebuilt. Space in online_log is
limited and if length of prefix of externally stored columns is very big, then
it is being written to online log without making sure if it fits. This leads to
memory corruption.

Fix:
----
After fix for Bug#16544143, there is no need to store prefixes of externally
stored columnd in online_log. Thus remove the code which stores column prefixes
for externally stored columns. Also, before writing anything on online_log,
make sure it fits to available memory to avoid memory corruption.

Read RB page for more details.
Reviewed-by: Annamalai Gurusami <annamalai.gurusami@oracle.com>
RB: 18239

a better fix