This is a partial port of my patch in MySQL 8.0.
In MySQL 8.0, all InnoDB references to DBUG_OFF were replaced
with UNIV_DEBUG. We will not do that in MariaDB.

InnoDB used two independent compile-time flags that distinguish
debug and non-debug builds, which is confusing.

Also, make ut_ad() and alias of DBUG_ASSERT().

In the InnoDB internal SQL parser, there is the keyword
DOES_NOT_FIT_IN_MEMORY that is never specified in any CREATE TABLE
statement that is passed to the InnoDB SQL parser
(que_eval_sql() or pars_sql() or yyparse()). If this keyword were
ever present, it would set the flag dict_table_t::does_not_fit_in_memory
which is only present in debug builds.

Let us remove all traces of this.

Also, fix storage/innobase/pars/make_flex.sh so that no the generated
file storage/innobase/pars/lexyy.cc works as is.

FIXME: Always generate the InnoDB Bison files at build time, similar
to how sql/sql_yacc.yy is handled. (This would still leave the
generated scanner files, unless we want to add a build-time dependency
for Flex.)

The fix is about filling the space beyond the end of VARCHAR values with zeroes.
VARCHAR NULLs already has its buffer filled with zeros. So when the VARCHAR fields
are not NULL, then we explicitly fill the buffer with zeros.

Added conditions so that the WITH queries are also added to the query cache

fix proposed by Diego

dict_create_or_check_foreign_constraint_tables(): Change the warning
about the foreign key metadata table creation to a note.

Remove messages after metadata table creation. If the creation fails,
startup will abort with a message. Normally the creation succeeds on
bootstrap, and the messages would only be noise.

Remove the related suppressions from the tests.

        Now let's check JSON length to fit the max_allowed packet.

        In get_mm_tree we have to change Field_geom::geom_type to
        GEOMETRY as we have to let storing all types of the spatial features
        in the field. So now we restore the original geom_type as it's
        done.

character (").

        The my_wildcmp function doesn't expect the string parameter to
        have escapements, only the template. So the string
        should be unescaped if necessary.

row_purge_remove_sec_if_poss_leaf(): Add a debug assertion.

row_purge_parse_undo_rec(): Add a debug assertion that the table cannot
be a temporary table. Remove a dead condition.

- add PAGESIZE property to the MSI installer
- add combobox  to the MSI UI to select innodb page size
- add new parameter --innodb_page_size for mysql_install_db.exe.
this is passed down to bootstrap and also stored in my.ini.

MSI will call mysql_install_db.exe with --innodb_page_size set to the
PAGESIZE property

fixed C++ comment in C code

Starting with MySQL 5.7, temporary tables in InnoDB are handled
differently from persistent tables. Because temporary tables are
private to a connection, concurrency control and multi-versioning
(MVCC) are not applicable. For performance reasons, purge is
disabled as well. Rollback is supported for temporary tables;
that is why we have the temporary undo logs in the first place.

Because MVCC and purge are disabled for temporary tables, we should
discard all temporary undo logs already at transaction commit,
just like we discard the persistent insert_undo logs. Before this
change, update_undo logs were being preserved.

trx_temp_undo_t: A wrapper for temporary undo logs, comprising
a rollback segment and a single temporary undo log.

trx_rsegs_t::m_noredo: Use trx_temp_undo_t.
(Instead of insert_undo, update_undo, there will be a single undo.)

trx_is_noredo_rseg_updated(), trx_is_rseg_assigned(): Remove.

trx_undo_add_page(): Remove the parameter undo_ptr.
Acquire and release the rollback segment mutex inside the function.

trx_undo_free_last_page(): Remove the parameter trx.

trx_undo_truncate_end(): Remove the parameter trx, and add the
parameter is_temp. Clean up the code a bit.

trx_undo_assign_undo(): Split the parameter undo_ptr into rseg, undo.

trx_undo_commit_cleanup(): Renamed from trx_undo_insert_cleanup().
Replace the parameter undo_ptr with undo.
This will discard the temporary undo or insert_undo log at
commit/rollback.

trx_purge_add_update_undo_to_history(), trx_undo_update_cleanup():
Remove 3 parameters. Always operate on the persistent update_undo.

trx_serialise(): Renamed from trx_serialisation_number_get().

trx_write_serialisation_history(): Simplify the code flow.
If there are no persistent changes, do not update MONITOR_TRX_COMMIT_UNDO.

trx_commit_in_memory(): Simplify the logic, and add assertions.

trx_undo_page_report_modify(): Keep a direct reference to the
persistent update_undo log.

trx_undo_report_row_operation(): Simplify some code.
Always assign TRX_UNDO_INSERT for temporary undo logs.

trx_prepare_low(): Keep only one parameter. Prepare all 3 undo logs.

trx_roll_try_truncate(): Remove the parameter undo_ptr.
Try to truncate all 3 undo logs of the transaction.

trx_roll_pop_top_rec_of_trx_low(): Remove.

trx_roll_pop_top_rec_of_trx(): Remove the redundant parameter
trx->roll_limit. Clear roll_limit when exhausting the undo logs.
Consider all 3 undo logs at once, prioritizing the persistent
undo logs.

row_undo(): Minor cleanup. Let trx_roll_pop_top_rec_of_trx()
reset the trx->roll_limit.

In the 10.1 InnoDB Plugin, a call os_event_free(buf_flush_event) was
misplaced. The event could be triggered by rollback of resurrected
transactions while shutdown was in progress. This bug was caught
by cmake -DWITH_ASAN testing. This call was only present in the
10.1 InnoDB Plugin, not in other versions, or in XtraDB.

That said, the bug affects all InnoDB versions. Shutdown assumes the
cessation of any page-dirtying activity, including the activity of
the background rollback thread. InnoDB only waited for the background
rollback to finish as part of a slow shutdown (innodb_fast_shutdown=0).
The default is a clean shutdown (innodb_fast_shutdown=1). In a scenario
where InnoDB is killed, restarted, and shut down soon enough, the data
files could become corrupted.

logs_empty_and_mark_files_at_shutdown(): Wait for the
rollback to finish, except if innodb_fast_shutdown=2
(crash-like shutdown) was requested.

trx_rollback_or_clean_recovered(): Before choosing the next
recovered transaction to roll back, terminate early if non-slow
shutdown was initiated. Roll back everything on slow shutdown
(innodb_fast_shutdown=0).

srv_innodb_monitor_mutex: Declare as static, because the mutex
is only used within one module.

In 10.2, os_event_destroy() sets the event to a NULL pointer,
while os_event_free() in earlier versions did not do that.

os_sync_free(), which hid resource leaks, was removed in MySQL 5.7.

Take into acount result of open table operation in mysql_admin_table.

Compatibility links ended up looking like this:
libmysqlclient.so.18 -> /tmp/buildd/mariadb-10.2-10.2.0/debian/tmp/usr/lib/i386-linux-gnu/libmariadb.so.3

This change fixes ln syntax to create links with correct target paths.

Reason : after running sysbench tests (rw,update_no_index), in all cases
the throughout was considerably (approx 15%) better with the patch removed.

Reverted commit info
commit 8dc03bee3ade2edcc53a3a257346f4a0a9f0b44c
Author: Yasufumi Kinoshita <yasufumi.kinoshita@oracle.com>
Date:   Wed Nov 20 17:02:24 2013 +0900

Bug #17824101 : WL#7050 CAUSES RW PERFORMANCE REGRESSION AT SOME WINDOWS ENVIRONMENT

InnoDB undo logs currently always use the innodb_page_size,
whether they are stored in the system tablespace, in a
dedicated undo tablespace, or in the temporary tablespace.
Remove redundant page_size parameters.

TrxUndoRsegsIterator::set_next(): return bool instead of page_size.

trx_rseg_mem_create(): Remove the parameter rseg_array.
Update trx_sys->rseg_array directly.

TrxUndoRsegsIterator::m_purge_sys: Remove. There is only one purge_sys.

purge_sys_t: Renamed from trx_purge_t. Define a constructor and
destructor. Allocate rseg_iter, purge_queue inline.

purge_sys->trx: Remove. Use purge_sys->sess->trx instead.

purge_sys->view_active: Remove. Access to purge_sys->view is always
protected by purge_sys->latch.

trx_purge_sys_create(): Replaced by purge_sys_t::purge_sys_t().

trx_purge_sys_close(): Replaced by purge_sys_t::~purge_sys_t().

Also in log_checkpoint(), where fil_flush_file_spaces() was
not done on Windows, it is not done.

 Remove srv_win_file_flush_method

- Rename srv_unix_file_flush_method to srv_file_flush_method, and
  rename constants to remove UNIX from them, i.e SRV_UNIX_FSYNC=>SRV_FSYNC

- Add SRV_ALL_O_DIRECT_FSYNC corresponding to current Windows default
(no buffering for either log or data, flush on both log and data)

- change os_file_open on Windows to behave identically to Unix wrt
O_DIRECT and O_DSYNC settings. map O_DIRECT to FILE_FLAG_NO_BUFFERING and
O_DSYNC to FILE_FLAG_WRITE_THROUGH

- remove various #ifdef _WIN32

MDEV-11363: Assertion `!derived->first_sel ect()->first_inner_unit() || derived->first_select()->first_inner_unit()->first_select()-> exclude_from_table_unique_test' failed in TABLE_LIST::set_check_materialized()

Do not try to materialize derived table which already merged (irreversebly).

InnoDB needs to collect transactions from the persistent data files
in trx_rseg_array_init() before trx_lists_init_at_db_start() is
executed. But there is no need to create purge_sys->purge_queue
separately from purge_sys.

trx_sys_init_at_db_start(): Change the return type to void.
Remove the direct call to trx_rseg_array_init(). It will be called
by trx_lists_init_at_db_start(), which we are calling.
Initialize the purge system read view.

trx_lists_init_at_db_start(): Call trx_purge_sys_create(), which will
invoke trx_rseg_array_init() to read the undo log segments.

trx_purge_sys_create(): Remove the parameters. Do not initialize
the purge system read view, because trx_sys->rw_trx_list has not
been recovered yet. The purge_sys->view will be initialized at
the end of trx_sys_init_at_db_start().

trx_rseg_array_init(): Remove the parameter. Use purge_sys->purge_queue
directly.

innobase_start_or_create_for_mysql(): Remove the local variable
purge_queue. Do not call trx_purge_sys_create(), because it will be
called by trx_sys_init_at_db_start().

In MySQL 5.7, there is some redundant code for supposedly handling
an upgrade from an earlier version of InnoDB.

An upgrade of InnoDB between major versions should include a
slow shutdown (innodb_fast_shutdown=0) of the previous version.

A comment in trx_lists_init_at_db_start() confused clean shutdown
and slow shutdown. A clean shutdown does not necessarily guarantee
that there are no active transactions. A slow shutdown guarantees
that.

Because there was no code to handle rollback of recovered transactions
that happened to use the rollback segment slots that MySQL 5.7.2
repurposed for temporary undo logs, the upgrade is not working in all
cases, and we may as well remove the code to handle purging.

trx_sys_t::pending_purge_rseg_array: Remove.

trx_undo_get_undo_rec_low(): Define as static. Remove the parameter
is_redo_rseg.

trx_undo_get_undo_rec(), trx_rseg_get_on_id(): Remove the parameter
is_redo_rseg.

trx_rseg_mem_free(): Remove the second parameter.

trx_sys_get_nth_rseg(): Replace with trx_rseg_get_on_id().

trx_rseg_schedule_pending_purge(): Remove.

Prepare os ANALYZE now respond as EXPLAIN.

This fixes some GCC 6.3.0 warnings and makes the code a little
more debugging-friendly.

After starting MariaDB 10.2 with an invalid value of
--innodb-flush-method= (the empty string), shutdown would
attempt to dereference some NULL pointers. This was probably broken
in commit 81b7fe9d which implemented
shutdown after aborted startup.

logs_empty_and_mark_files_at_shutdown(): Allow shutdown even if
lock_sys, log_sys, or fil_system is NULL.

os_aio_free(): Tolerate os_aio_segment_wait_events==NULL.

innobase_start_or_create_for_mysql(): Do not invoke
srv_init_abort() before initializing all mutexes for the temporary files.

innodb_shutdown(): Tolerate buf_pool_ptr==NULL.

Provide more useful progress reporting of crash recovery.

recv_sys_t::progress_time: The time of the last report.

recv_sys_t::report(ib_time_t): Determine whether progress should
be reported.

recv_scan_print_counter: Remove.

log_group_read_log_seg(): After after each I/O request, invoke
recv_sys_t::report() and report progress if needed.

recv_apply_hashed_log_recs(): Change the return type back to void
(DB_SUCCESS was always returned), and rename the parameter to last_batch.
At the start of each batch, if there are pages to be recovered,
issue a message.