- Don't use 'res' uninitialized
- multiply it by col_non_nulls before set_if_bigger(...) call.

Signed-off-by: Michael Okoko <okokomichaels@outlook.com>

Signed-off-by: Michael Okoko <okokomichaels@outlook.com>

Signed-off-by: Michael Okoko <okokomichaels@outlook.com>

* Also merges tests relating to JSON statistics into one file
Signed-off-by: Michael Okoko <okokomichaels@outlook.com>

Signed-off-by: Michael Okoko <okokomichaels@outlook.com>

Signed-off-by: Michael Okoko <okokomichaels@outlook.com>

Signed-off-by: Michael Okoko <okokomichaels@outlook.com>

Signed-off-by: Michael Okoko <okokomichaels@outlook.com>

* it also adds an "explain select" statement to the test so that the fprintf calls
  can print the computed intervals to mysqld.1.err
Signed-off-by: Michael Okoko <okokomichaels@outlook.com>

Signed-off-by: Michael Okoko <okokomichaels@outlook.com>

This fixes the wrong calculation for avg_frequency in json histograms
by replacing the specific histogram objects with the generic Histogram_base class.

It also restores get/set size functions as they were useful in calculating fields
for binary histogram.
Signed-off-by: Michael Okoko <okokomichaels@outlook.com>

A demo of how to use in-memory data structure for histogram.
The patch shows how to
* convert string form of data to binary form
* compare two values in binary form
* compute a fraction for val in [X, Y] range.

grep for GSOC-TODO for notes.

Signed-off-by: Michael Okoko <okokomichaels@outlook.com>

Signed-off-by: Michael Okoko <okokomichaels@outlook.com>

Signed-off-by: Michael Okoko <okokomichaels@outlook.com>

Signed-off-by: Michael Okoko <okokomichaels@outlook.com>

Preparation for handling different kinds of histograms:

- In Column_statistics, change "Histogram histogram" into
  "Histogram *histogram_".  This allows for different kinds
  of Histogram classes with virtual functions.

- [Almost] remove the usage of Histogram->set_values and
  Histogram->set_size. The code outside the histogram should
  not make any assumptions about what/how is stored in the Histogram.

- Introduce drafts of methods to read/save histograms to/from disk.

Signed-off-by: Michael Okoko <okokomichaels@outlook.com>

Signed-off-by: Michael Okoko <okokomichaels@outlook.com>

Signed-off-by: Michael Okoko <okokomichaels@outlook.com>

Signed-off-by: Michael Okoko <okokomichaels@outlook.com>

Signed-off-by: Michael Okoko <okokomichaels@outlook.com>

This fixes the memory allocation for json histogram builder and add more column types for testing.
Some challenges at the moment include:
* Garbage value at the end of JSON array still persists.
* Garbage value also gets appended to bucket values if the column is a primary key.
* There's a memory leak resulting in a "Warning: Memory not freed" message at the end of tests.
Signed-off-by: Michael Okoko <okokomichaels@outlook.com>

Signed-off-by: Michael Okoko <okokomichaels@outlook.com>

Signed-off-by: Michael Okoko <okokomichaels@outlook.com>

Signed-off-by: Michael Okoko <okokomichaels@outlook.com>

Signed-off-by: Michael Okoko <okokomichaels@outlook.com>

Two Problems
1. Upgrade wizard failed to retrieve path to service executable,
if it contained non-ASCII.
Fixed by setlocale(LC_ALL, "en_US.UTF8"), which was missing in upgrade wizard

2.mysql_upgrade_service only updated (converted to UTF8) the server's sections
leaving client's as-is

Corrected typo.

3. Fixed assertion in my_getopt, turns out to be too strict.

Log messages like total size = 17179869184, chunk size = 134217728
get hard to read. If we normalize it down to IEC units is easier.

Idea thanks to Axel Schwenke.

Review thanks to Eugene Kosov and Marko Mäkelä

$ mariadblocal --innodb-buffer-pool-size=30G --innodb-log-file-size=128M
Installing MariaDB/MySQL system tables in '/tmp/build-mariadb-server-10.7-datadir' ...
2021-12-09  9:54:04 0 [Note] /home/dan/repos/build-mariadb-server-10.7/sql/mysqld (server 10.7.2-MariaDB) starting as process 250473 ...
2021-12-09  9:54:04 0 [Note] InnoDB: The first data file './ibdata1' did not exist. A new tablespace will be created!
2021-12-09  9:54:04 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2021-12-09  9:54:04 0 [Note] InnoDB: Number of transaction pools: 1
2021-12-09  9:54:04 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
2021-12-09  9:54:04 0 [Note] InnoDB: Using liburing
2021-12-09  9:54:04 0 [Note] InnoDB: Initializing buffer pool, total size = 128.000MiB, chunk size = 128.000MiB
2021-12-09  9:54:04 0 [Note] InnoDB: Completed initialization of buffer pool
2021-12-09  9:54:04 0 [Note] InnoDB: Setting O_DIRECT on file ./ibdata1 failed
2021-12-09  9:54:04 0 [Note] InnoDB: Setting file './ibdata1' size to 12.000MiB. Physically writing the file full; Please wait ...
2021-12-09  9:54:04 0 [Note] InnoDB: File './ibdata1' size is now 12.000MiB.
2021-12-09  9:54:04 0 [Note] InnoDB: Setting log file ./ib_logfile101 size to 96.000MiB
2021-12-09  9:54:04 0 [Note] InnoDB: Renaming log file ./ib_logfile101 to ./ib_logfile0
2021-12-09  9:54:04 0 [Note] InnoDB: New log file created, LSN=10317
2021-12-09  9:54:04 0 [Note] InnoDB: Doublewrite buffer not found: creating new
2021-12-09  9:54:04 0 [Note] InnoDB: Doublewrite buffer created
2021-12-09  9:54:04 0 [Note] InnoDB: 128 rollback segments are active.
2021-12-09  9:54:04 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2021-12-09  9:54:04 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ...
2021-12-09  9:54:04 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB.
2021-12-09  9:54:04 0 [Note] InnoDB: 10.7.2 started; log sequence number 0; transaction id 3
OK
2021-12-09  9:54:04 0 [Note] sql/mysqld (server 10.7.2-MariaDB) starting as process 250501 ...
2021-12-09  9:54:04 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2021-12-09  9:54:04 0 [Note] InnoDB: Number of transaction pools: 1
2021-12-09  9:54:04 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
2021-12-09  9:54:04 0 [Note] InnoDB: Using liburing
2021-12-09  9:54:04 0 [Note] InnoDB: Initializing buffer pool, total size = 30.000GiB, chunk size = 128.000MiB
2021-12-09  9:54:04 0 [Note] InnoDB: Completed initialization of buffer pool
2021-12-09  9:54:04 0 [Note] InnoDB: Setting O_DIRECT on file ./ibdata1 failed
2021-12-09  9:54:04 0 [Note] InnoDB: Resizing redo log from 96.000MiB to 128.000MiB; LSN=41361
2021-12-09  9:54:04 0 [Note] InnoDB: Starting to delete and rewrite log file.
2021-12-09  9:54:04 0 [Note] InnoDB: Setting log file ./ib_logfile101 size to 128.000MiB
2021-12-09  9:54:04 0 [Note] InnoDB: Renaming log file ./ib_logfile101 to ./ib_logfile0
2021-12-09  9:54:04 0 [Note] InnoDB: New log file created, LSN=41361
2021-12-09  9:54:04 0 [Note] InnoDB: 128 rollback segments are active.
2021-12-09  9:54:04 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2021-12-09  9:54:04 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ...
2021-12-09  9:54:04 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB.
2021-12-09  9:54:04 0 [Note] InnoDB: 10.7.2 started; log sequence number 41349; transaction id 14
2021-12-09  9:54:04 0 [Note] InnoDB: Loading buffer pool(s) from /tmp/build-mariadb-server-10.7-datadir/ib_buffer_pool
2021-12-09  9:54:04 0 [Note] Plugin 'FEEDBACK' is disabled.
2021-12-09  9:54:04 0 [Note] InnoDB: Buffer pool(s) load completed at 211209  9:54:04
2021-12-09  9:54:04 0 [Note] sql/mysqld: ready for connections.
Version: '10.7.2-MariaDB'  socket: '/tmp/build-mariadb-server-10.7.sock'  port: 0  Source distribution
2021-12-09  9:56:57 0 [Note] sql/mysqld (initiated by: unknown): Normal shutdown
2021-12-09  9:56:57 0 [Note] InnoDB: FTS optimize thread exiting.
2021-12-09  9:56:57 0 [Note] InnoDB: Starting shutdown...
2021-12-09  9:56:57 0 [Note] InnoDB: Dumping buffer pool(s) to /tmp/build-mariadb-server-10.7-datadir/ib_buffer_pool
2021-12-09  9:56:57 0 [Note] InnoDB: Buffer pool(s) dump completed at 211209  9:56:57
2021-12-09  9:56:57 0 [Note] InnoDB: Removed temporary tablespace data file: "./ibtmp1"
2021-12-09  9:56:57 0 [Note] InnoDB: Shutdown completed; log sequence number 42602; transaction id 15
2021-12-09  9:56:57 0 [Note] sql/mysqld: Shutdown complete

The previous default innodb_buffer_pool_chunk_size of 128M
made sense when the innodb buffer pool size was a few GB.

When the pool size is 128GB this means the chunk size is 0.1%
of this. Fine tuning the buffer pool size on such a fine
increment doesn't make practical sense. Also on extremely
large buffer pool systems, initializing on the default 128M can
also take a considerable amount of time.

When large pages are enabled, the chunk size has to be a multiple
of an available large page size or memory allocation without
use can occur.

Previously the default 0 was documented as disabling resizing.
With srv_buf_pool_chunk_unit > 0 assertions in the code and the
minimium value set, I doubt this was ever the case.

As such the autosizing (based on default 0) takes place as follows:
* a 64th of the innodb_buffer_pool_size
* if large pages, this is rounded down the the nearest multiple
  of the large page size.
* If less than 1MB, set to 1MB.

This does mean the new default innodb_buffer_pool_chunk size is
2MB, derived form the above formular with 128MB as the buffer pool
size.

The innodb_buffer_pool_chunk_size is changed to a size_t for
better compatiblity with the memory allocations which use size_t.
The previous upper limit is changed to the maxium of a size_t. The
maximium value used is the buffer pool size anyway.

Getting this default value of the chunk size to a more practical
size facilitates further development of more automated resizing
without significant overhead or memory fragmentation.

innodb_buffer_pool_resize test adjusted based on 1M default
chunk size thanks Wlad.