MDEV-33299 Assertion `(tm->tv_usec % (int) log_10_int[6 - dec]) == 0' failed in void my_timestamp_to_binary(const timeval*, uchar*, uint)

This original query:

(1)  SELECT ts0 FROM t1
     WHERE DATE(ts0) <= '2024-01-23';

was rewritten (by MDEV-8320) to:

(2)  SELECT ts0 FROM t1
     WHERE ts0 <= '2024-01-23 23:59.59.999999';
     -- DATETIME comparison, Item_datetime on the right side

which was further optimized (by MDEV-32148) to:

(3)  SELECT ts0 FROM t1
     WHERE ts0 <= TIMESTAMP/* WITH LOCAL TIME ZONE*/ '2024-01-23 23:59.59.999999';
     -- TIMESTAMP comparison, Item_timestamp_literal on the right side

The origin of the problem was in (2) - in the MDEV-8320 related code.
The recent new code for MDEV-32148 revealed this problem.

Item_datetime on step (2) was always created in an inconsistent way:
- with Item::decimals==0
- with ltime.second_part==999999,
  without taking into account the precision of the left side
  (e.g. ts0 in the above example)

On step (3), Item_timestamp_literal was created in an inconsistent way too,
because it copied the inconsistent data from step (2):
- with Item::decimals==0       (copied from Item_datetime::decimals)
- with m_value.tv_usec==999999 (copied from ltime.second_part of Item_datetime)

Later, the Item_timestamp_literal performed save_in_field()
and crashed in my_timestamp_to_binary() on a DBUG_ASSERT checking
consistency between the fractional precision and the fractional seconds value.

Fix:

On step (2) create Item_datetime with truncating maximum possible
second_part value of 999999 according to the the left side fractional
second precision. So for example it sets second_part as follows:
- 000000 for TIMESTAMP(0)
- 999000 for TIMESTAMP(3)
- 999999 for TIMESTAMP(6)

This automatically makes the code create a consistent Item_timestamp_literal
on step (3).

This also makes TIMESTAMP comparison work faster, because now
Item_timestamp_literal is created with Item::decimals value equal
to the Item_field (which is on the other side of the comparison),
so the low level function Type_handler_timestamp_common::cmp_native()
goes the fastest execution path optimized for the case when both sides
have equal fractional precision.

Adding a helper class TimeOfDay to reuse the code when populating:
- the last datetime point for YEAR()
- the last datetime point for DATE()
with a given fractional precision.

This class also helped to unify the equal code in create_start_bound()
and create_end_bound() into a single method create_bound().

Changing the way how a the following conditions are evaluated:

    WHERE timestamp_column=datetime_const_expr

(for all comparison operators: =, <=>, <, >, <=, >=, <> and for NULLIF)

Before the change it was always performed as DATETIME.
That was not efficient, as involved per-row TIMESTAMP->DATETIME conversion
for timestamp_column. For example, in case of the SYSTEM time zone
it involved a localtime_r() call, which is known to be slow.

After the change it's performed as TIMESTAMP in many cases.
This allows to avoid per-row conversion, as it works the other way around:
datetime_const_expr is converted to TIMESTAMP once before the execution stage.

Note, datetime_const_expr must be inside monotone continuous periods of
the current time zone, i.e. not near these anomalies:
- DST changes (spring forward, fall back)
- leap seconds

When innodb_undo_log_truncate=ON causes an InnoDB undo tablespace
to be truncated, we must guarantee that the undo tablespace will
be rebuilt atomically: After mtr_t::commit_shrink() has durably
written the mini-transaction that rebuilds the undo tablespace,
we must not write any old pages to the tablespace.

To guarantee this, in trx_purge_truncate_history() we used to
traverse the entire buf_pool.flush_list in order to acquire
exclusive latches on all pages for the undo tablespace that
reside in the buffer pool, so that those pages cannot be written
and will be evicted during mtr_t::commit_shrink(). But, this
traversal may interfere with the page writing activity of
buf_flush_page_cleaner(). It would be better to lazily discard
the old pages of the truncated undo tablespace.

fil_space_t::is_being_truncated, fil_space_t::clear_stopping(): Remove.

fil_space_t::create_lsn: A new field, identifying the LSN of the
latest rebuild of a tablespace.

buf_page_t::flush(), buf_flush_try_neighbors(): Evict pages whose
FIL_PAGE_LSN is below fil_space_t::create_lsn.

mtr_t::commit_shrink(): Update fil_space_t::create_lsn and
fil_space_t::size right before the log is durably written and the
tablespace file is being truncated.

fsp_page_create(), trx_purge_truncate_history(): Simplify the logic.

Reviewed by: Thirunarayanan Balathandayuthapani, Vladislav Lesin
Performance tested by: Axel Schwenke
Correctness tested by: Matthias Leich

trx_purge_free_segment(), trx_purge_truncate_rseg_history():
Do not claim that the blocks will be modified in the mini-transaction,
because that will not always be the case. Whenever there is a
modification, mtr_t::set_modified() will flag it.

The debug assertion that failed in recovery is checking that all
changes to data pages are covered by log records. Due to these
incorrect calls, we would unnecessarily write unmodified data pages,
which is something that commit 05fa4558
aims to avoid.

The incorrect calls had originally been added in
commit de31ca6a (MDEV-32820) and
commit 86767bcc (MDEV-29593).

Reviewed by: Vladislav Lesin
Tested by: Elena Stepanova

perfschema thread walker needs to take thread's LOCK_thd_kill to prevent
the thread from disappearing why it's being looked at.
But there's no need to lock it for the current thread.

In fact, it was harmful as some code down the stack might take
LOCK_thd_kill (e.g. set_killed() does it, and my_malloc_size_cb_func()
calls set_killed()). And it caused a bunch of mutexes being locked under
LOCK_thd_kill, which created problems later when my_malloc_size_cb_func()
called set_killed() at some unspecified point under some
random mutexes.

same assertion with spider. spider status variables
didn't expect to be queried from a different thread
without LOCK_thd_data.

And they didn't expect to be queried under LOCK_thd_data either
(because spider_get_trx() calls thd_set_ha_data()).

reduce code duplication

need to protect access to thread-local cache_mngr with LOCK_thd_data

technically only access from different threads has to be protected,
but this is the SHOW STATUS code path, so the difference is neglectable

use utf8mb4 with PCRE2, not utf8mb3

- Move it from delete.test to delete_innodb.test
- Use --source include/innodb_stable_estimates.inc to make it predicatable.

buf_flush_page_cleaner(): A continue or break inside DBUG_EXECUTE_IF
actually is a no-op. Use an explicit call to _db_keyword_() to
actually avoid advancing the checkpoint.

buf_flush_list_now_set(): Invoke os_aio_wait_until_no_pending_writes()
to ensure that the page write to the system tablespace is completed.

MDEV-33101 Server crashes when starting the server with innodb-force-recovery=6 and enabling the innodb_truncate_temporary_tablespace_now variable

The issue is introduced by "MDEV-28699: Shrink temporary tablespaces
without restart". SRV_FORCE_NO_LOG_REDO forces server to read only mode
and we don't initialize temporary tablespace in read only mode.

solution: innodb_truncate_temporary_tablespace_now should be no-op in
read only mode.

srv_start(): Move a read only mode startup tweak from
innodb_init_params() to the correct location. Also if
innodb_force_recovery=6 we will disable the doublewrite buffer,
because InnoDB must run in read-only mode to prevent further corruption.

This change only affects debug checks. Whenever srv_read_only_mode holds,
the buf_pool.flush_list will be empty, that is, there will be no writes
of persistent InnoDB data pages.

Reviewed by: Thirunarayanan Balathandayuthapani

- innodb.doublewrite_debug should avoid the checkpoint
before killing the server. So used debug sync and
innodb_flush_sync to avoid the checkpoint completely.
Test case allowed to skip on MSAN builder due to extra
checkpoint.

wsrep_plugin_init(), wsrep_plugin_deinit(): Remove these dummy functions
in order to fix an error that would be flagged by cmake -DWITH_UBSAN=ON
when using clang.

wsrep_show_ready(), wsrep_show_bf_aborts(): Correct the signature.

If a query has a HAVING clause that contains a predicate with a constant
IN subquery whose lef part in its turn is a subquery and the predicate is
subject to pushdown from HAVING to WHERE then execution of the query could
cause a crash of the server.

The cause of the problem was the missing implementation of the walk()
method for the class Item_in_optimizer. As a result in some cases the left
operand of the Item_in_optimizer condition could be traversed twice by
the walk procedure. For many call-back functions used as an argument of
this procedure it does not matter. Yet it matters for the call-back
function cleanup_excluding_immutables_processor() used in pushdown of
predicates from HAVING to WHERE. If the processed item is marked with
the IMMUTABLE_FL flag then the processor just removes this flag, otherwise
it performs cleanup of the item making it unfixed. If an item is marked
with an the IMMUTABLE_FL and it traversed with this processor twice then
it becomes unfixed after the second traversal though the flag indicates
that the item should not be cleaned up.

Approved by Oleksandr Byelkin <sanja@mariadb.com>

Problem:

sp_cache erroneously looked up fully qualified SP names (e.g. `DB`.`SP`),
in case insensitive style. It was wrong, because only the "name"
part is always case insensitive, while the "db" part should be compared
according to lower_case_table_names (case sensitively for 0,
case insensitively for 1 and 2).

Fix:

Adding a "casedn_name" parameter make_qname() to tell
if the name part should be lower cased:
  `DB1`.`SP` -> "DB1.SP"  (when casedn_name=false)
  `DB1`.`SP` -> "DB1.sp"  (when casedn_name=true)
and using make_qname() with casedn_name=true when creating
sp_cache hash lookup keys.

Details:

As a result, it now works as follows:
- sp_head::m_db is converted to lower case if lower_case_table_names>0
  during the sp_name initialization phase. So when make_qname() is called,
  sp_head::m_db is already normalized. There are no changes in here.

- The initialization phase of sp_head when creating sp_head::m_qname
  now calls make_qname() with casedn_name=true,
  so sp_head::m_name gets written to sp_head::m_qname in lower case.

- sp_cache_lookup() now also calls make_qname() with casedn_name=true,
  so sp_head::m_name gets written to the temporary lookup key in lower case.

- sp_cache::m_hashtable now uses case sensitive comparison

Part#1 A non-functional change

Changing the signature of Identifier_chain2::make_qname() from

  bool make_qname(MEM_ROOT *mem_root, LEX_CSTRING *dst) const;

to

  LEX_CSTRING make_qname(MEM_ROOT *mem_root) const;

Now the result is returned as LEX_CSTRING from the function rather than
is passed as a parameter.
The return value {NULL,0} means "EOM".

This is a requirement step to fix and merge easier
  MDEV-33019 The database part is not case sensitive in SP names

The original MDEV-31991 commit commend:

- Moving some of Database_qualified_name methods into a new class
  Identifier_chain2.

- Changing the data type of the following variables from
  Database_qualified_name to Identifier_chain2:

  * q_pkg_proc in LEX::call_statement_start()
  * q_pkg_func in LEX::make_item_func_call_generic()

Rationale:

The data type of Database_qualified_name::m_db will be changed
to Lex_ident_db soon. So Database_qualified_name won't be able
to store the `pkg.routine` part of `db.pkg.routine` any more,
because `pkg` must not depend on lower-case-table-names.

Marko reported DBUG_ASSERT from dict_stats_shutdown().

destroy_background_thd() does not like when current_thd is set.
In galera, it can be the case, dict_stats_shutdown() can be called from
user thread, to stop and later restart stats recalculations.

MDEV-31003 has introduced second execution for SELECTs that execute
under ps-protocol. The following tests in galera suites do not support
this mode of execution, disable it:

  galera.MDEV-27862
  galera.galera_log_output_csv
  galera.galera_query_cache
  galera.galera_query_cache_sync_wait
  galera_3nodes_sr.GCF-336
  galera_3nodes_sr.galera_sr_isolate_master
  galera_sr.galera_sr_large_fragment
  galera_sr.galera_sr_many_fragments
Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>