Two Problems
1. Upgrade wizard failed to retrieve path to service executable,
if it contained non-ASCII.
Fixed by setlocale(LC_ALL, "en_US.UTF8"), which was missing in upgrade wizard

2.mysql_upgrade_service only updated (converted to UTF8) the server's sections
leaving client's as-is

Corrected typo.

3. Fixed assertion in my_getopt, turns out to be too strict.

Translate username, password and database from UTF8 into desired charset,
if  non-auto default-character-set was used, on Windows10 1903

This change is implemented only in the command line client, and mainly to
allow users with non-UTF8 passwords to login.

The user is supposed to use the same charset that was used during setting
password (usually, console CP if used in CLI)

Add a test to document the behavior.

If someone on whatever reasons uses --default-character-set=cp850,
this will avoid incorrect display, and inserting incorrect data.

Adjusting console codepage sometimes also needs to happen with
--default-charset=auto, on older Windows. This is because autodetection
is not always exact. For example, console codepage on US editions of
Windows is 437. Client autodetects it as cp850, a rather loose
approximation, given 46 code point differences. We change the console
codepage to cp850, so that there is no discrepancy.

That fix is currently Windows-only, and serves people who used combination
of chcp to achieve WYSIWYG effect (although, this would mostly likely used
with utf8 in the past)

Now, --default-character-set would be a replacement for that.

Fix fs_character_set() detection of current codepage.

Add mysql_install_db test with some i18n, for data dir and root password

If last character in command is ampersand, do not use my_popen/my_pclose
in "exec", instead do CreateProcess() without a window.

Handle upgrade - on Windows that is capable of UTF8 codepage, convert
entries in my.ini from ANSI to UTF8, during upgrade

Reason is that for the server, paths such as datadir, innodb directories,
location of SSL certificates must now be utf8.

For the client programs , user name,  database etc should be in UTF8,
too, as UTF-8 is now the default charset.

- Tolerate situation, when datadir for service seems invalid/non-existing
prior to upgrade. It could be that my.ini contains legacy ANSI characters
for the data directory. Those can't be read correctly by
mysql_upgrade_service, which uses a different ANSI codepage(UTF8)
.
- schedule upgrade_config_file at later stage, because once we
convert it to UTF-8 (followup patch), this will render config file uselss
with the older version of mariadbd.exe

- Refactor upgrade_conf_file.cc, prepare for UTF-8 conversion.

Also, fix the "UTF8" option in MSI, which is responsible for character-set-server
setting

- Use corresponding entry in the manifest, as described in
https://docs.microsoft.com/en-us/windows/apps/design/globalizing/use-utf8-code-page

- If if ANSI codepage is UTF8 (i.e for Windows 1903 and later)
  Use UTF8 as default client charset
  Set console codepage(s) to UTF8, in case process is using console

- Allow some previously disabled MTR tests, that used Unicode for in "exec",
  for the recent Windows versions

Previously, password was passed as hex(clear_text_password).
The hex encoding was used to avoid masking apostrophe and backslash etc.

However, bootstrap still manages to misinterpert UTF8 password, so that
root would not connect later.

So the fix is to compute the native password hash inside mysql_install_db
already instead, and create user with that hash, rather than letting bootstrap
calculate it by using PASSWORD() function.

CreateServiceA, OpenServiceA, and couple of other functions do not work
correctly with non-ASCII character, in the special case where application
has defined activeCodePage=UTF8.

Workaround by redefining affected ANSI functions to own wrapper, which
works by converting narrow(ANSI) to wide, then calling wide function.

Deprecate original ANSI service functions, via declspec, so that we can catch
their use.

Corresponding Windows bug  https://github.com/microsoft/terminal/issues/4551

Use ReadConsoleW instead and convert to console's input codepage, to
workaround.

Also, disable VT sequences in the console output, as we do not knows what
type of data comes with SELECT, we do not want VT escapes there.

Remove my_cgets()

Prior to patch, get_password would echo multple mask characters '*', for
a single multibyte input character.

Fixed the behavior by using "wide" version of getch, _getwch.
Also take care of possible characters outside of BMP (i.e we do not print
'*' for high surrogates).

The function will now internally construct the "wide" password string,
and conver to the console codepage. Some characters could still be lost
in that conversion, unless the codepage is utf8, but this is not any new
bug.

and galera_3nodes.galera_ipv6_mariabackup_section
until their corresponding MDEV's are fixed.

They fail almost everywhere.

SQL statments could have table entries added in lexer. This entries
should not invalidate `next_global` member.
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

- Addressing the format issue in deferred_dblwr() and changed the
function comment.

  InnoDB fails to identify the deferred tablespace after recovery.
Problem is that InnoDB fails to rename the tablespace present
in recovered tablespace. Fix is that InnoDB should try to rename the
recovered tablespace when tablespace is being deferred

  This patch reverts the commit cab8f4b5.
InnoDB fails to restore page0 from doublewrite buffer when the
tablespace is being deferred. In that case, InnoDB doesn't find
INIT_PAGE redo log record for page0 and it leads to failure.
InnoDB should recovery page0 from doublewrite buffer for the
deferred tablespace before applying the redo log records.

Added deferred_dblwr() to restore page0 of deferred tablespace
from doublewrite buffer

In commit 49e2c8f0 (MDEV-25743)
some more use of the printf-style format "%.*s" was added.
The length parameter is of type int, not size_t.
On 64-bit platforms that follow the LLP64 convention (such as
64-bit Microsoft Windows), sizeof(int)==4 and sizeof(size_t)==8.

Let us explicitly cast the lengths to the correct type in order
to avoid any trouble.

[Adjusting Sergei Krivonos's patch]

"duplicates_removal" may contain multiple elements inside it and
so should have a JSON array as a value (and not object).

BUG#31761802 STATISTICS ANY QUERIES USING VIEWS ARE SUMMARIZED TOGETHER WITH THE VIEW DEFINITION SELECT

test case only

now when SLES12.3 is gone, we can enforce it

During startup, InnoDB must write a FILE_CHECKPOINT record.
However, before MDEV-12353 (in MariaDB Server 10.2, 10.3, 10.4)
the corresponding record MLOG_CHECKPOINT was encoded in a different way.

When we are upgrading from a logically empty 10.2, 10.3, or 10.4 redo log,
we must not write anything to the old log file, because if the server were
killed during the upgrade, we would end up with a corrupted log file, and
both the old and the new server would refuse to start up.

On upgrade, we must simply create a new logically empty log file
and replace the old ib_logfile0 with that.

This is a low hanging fruit. Before this patch std::map::emplace() was
a ~50% of the whole recv_sys_t::parse() operation in by test.
After the fix it's only ~20%.

recv_sys_t::parse() recv_sys_t::pages is a collection of all pages
to recovery. Often, there are multiple changes for a single page.
Often, they go in a row and for such cases let's avoid
lookup in a std::map. cached_pages_it serves as a cache
of size 1.

recv_sys_t::add(): replace page_id argument with a std::map::iterator

This reverts commit 2d21917e.

No explainations, lots of code moved, wrong cmake changes