- 22 May, 2021 1 commit
-
-
Julius Goryavsky authored
-
- 21 May, 2021 2 commits
-
-
Julius Goryavsky authored
Another batch of changes that should make the SST process more reliable in all scenarios: 1) Added hostname or CN verification when stunnel is used with certificate chain verification (verifyChain = yes); 2) Added check for the absence of the stunnel utility for mtr tests; 3) Deletion of working files before and after SST is done more accurately; 4) rsync on joiner can be run even if the path to its configuration file contains spaces; 5) More accurate directory creation (for data files and for logs); 6) IST with mysqldump no longer turns off statement logging; 7) Reset password for mysqldump when password is empty but username is specified; 8) More reliable quoting when generating statements in wsrep_sst_mysqldump; 9) Added explicit generation of 2048-bit Diffie-Hellman parameters for sockat < 1.7.3, by analogy with xtrabackup; 10) Compression parameters for qpress are read from all suitable server groups in configuration file, as well as from the [sst] and [xtrabackup] groups; 11) Added a test that checks compression using qpress; 12) Checking for optional utilities is modified to work even if they implemented as built-in shell commands (unlikely on real systems, but more reliable).
-
mkaruza authored
MDEV-25562 Assertion `pause_seqno_.is_undefined() == false' failed in void wsrep::server_state::resume() If pause() is not executed in galera and returns seqno = -1 we should skip resume().
-
- 18 May, 2021 4 commits
-
-
Marko Mäkelä authored
The functions fil_file_readdir_next_file(), os_file_opendir(), os_file_closedir() became dead code in the server in MariaDB 10.4.0 with commit 09af00cb (the removal of the crash recovery logic for the TRUNCATE TABLE implementation that was replaced in MDEV-13564). os_file_opendir(), os_file_closedir(): Define as macros.
-
Marko Mäkelä authored
-
Marko Mäkelä authored
-
Marko Mäkelä authored
ha_innobase::index_read(): If an autocommit non-locking transaction was already started, refuse to access a SPATIAL INDEX. Once a non-locking autocommit transaction has started, it must remain in that mode (not acquire any locks). This should fix one cause of the assertion failure that would occur in DeadlockChecker::check_and_resolve() under heavy load, presumably due to concurrent execution of trx_commit_in_memory().
-
- 17 May, 2021 12 commits
-
-
Julius Goryavsky authored
Fixed bugs caused by inaccuracies in automatic merging from other branches: 1) Authentication information is not removed from the connection address, which causes some tests to fail; 2) wsrep_debug=on should be replaced with wsrep_debug=1; 3) Added missing "connection" lines to test result file; 4) Some tests have been corrected for Galera 4.x (10.4+).
-
Julius Goryavsky authored
-
Julius Goryavsky authored
-
Julius Goryavsky authored
-
Julius Goryavsky authored
1) This commit implements reading all sections from configuration files while looking for the current value of any server variable, which were previously only read from the [mysqld.suffix] group and from [mysqld], but not from other groups such as [mariadb.suffix], [mariadb] or, for example, [server]. 2) This commit also fixes misrecognition of some parameters when parsing a command line containing a special marker for the end of the list of options ("--") or when short option names (such as "-s", "-a" and "-h arg") chained together (like a "-sah arg"). Such parameters can be passed to the SST script in the list of arguments after "--mysqld-args" if the server is started with a complex set of options - this was revealed during manual testing of changes to read configuration files. 3) The server-side preparation code for the "--mysqld-args" option list has also been simplified to make it easier to change in the future (if needed), and has been improved to properly handle the special backquote ("`") character in the argument values.
-
Julius Goryavsky authored
-
Julius Goryavsky authored
1) This commit implements reading all sections from configuration files while looking for the current value of any server variable, which were previously only read from the [mysqld.suffix] group and from [mysqld], but not from other groups such as [mariadb.suffix], [mariadb] or, for example, [server]. 2) This commit also fixes misrecognition of some parameters when parsing a command line containing a special marker for the end of the list of options ("--") or when short option names (such as "-s", "-a" and "-h arg") chained together (like a "-sah arg"). Such parameters can be passed to the SST script in the list of arguments after "--mysqld-args" if the server is started with a complex set of options - this was revealed during manual testing of changes to read configuration files. 3) The server-side preparation code for the "--mysqld-args" option list has also been simplified to make it easier to change in the future (if needed), and has been improved to properly handle the special backquote ("`") character in the argument values.
-
Julius Goryavsky authored
-
Julius Goryavsky authored
-
Julius Goryavsky authored
This commit contains a large set of further bug fixes and improvements to SST scripts for Galera, continuing the work that was started in MDEV-24962 to make SST scripts work smoothly in different network configurations (especially using ipv6) and with different environment settings: 1) The ipv6 addresses were incorrectly handled in the SST script for rsync (incorrect address substitution for establishing a connection, incorrect address substitution for bind, and so on); 2) Checking the locality of the ip-address in SST scripts did not support ipv6 addresses (such as "[::1]"), which were falsely identified as non-local ip, which further did not allow running two SSTs on different local addresses on the same machine. On the other hand, this bug masked some other errors (related to handling ipv6 addresses); 3) The code for checking the locality of the ip address was different in the SST scripts for rsync and for mysqldump, with individual flaws. This code is now made common and moved to wsrep_sst_common; 4) Waiting for the start of the transport channel (socat, nc, rsync, stunnel) in the wait_for_listen() and check_pid_and_port() functions did not process ipv6 addresses correctly in all cases (not for all branches); 5) Waiting for the start of the transport channel (socat, nc, rsync, stunnel) in the wait_for_listen() and check_pid_and_port() functions for some code branches could give a false positive result due to the textual match of prefixes in the port number and/or PID of the process; 6) Waiting for the start of the transport channel (socat, nc, rsync, stunnel) was supported through different utilities in SST scripts for mariabackup and for rsync, and with various minor flaws in the code. Now the code is still different in these scripts, but it supports a common set of utilities (lsof, ss, sockstat) and is synchronized across patterns that used to check the output of these utilities; 7) In SST via mariabackup, the signal about readiness to receive data is sometimes sent too early - immediately after listen(), and not after accept() (which are called by socat or netcat utility). 8) Checking availability of the some options of some utilities was done using the grep pattern, which easily gives false positives; 9) Common name (CN) for local addresses, if not explicitly specified, is now always replaced to "localhost" to avoid the need to generate many separate certificates for local addresses of one machine and not to depend on which the local address is currently used in test (ipv4 or ipv6, etc.); 10) In tests galera_sst_mariabackup_encrypt_with_key_server and galera_sst_rsync_encrypt_with_key_server the correct certificate is selected to avoid commonname (CN) mismatch problems; 11) Further refactoring to protect against spaces in file names. 12) Further general refactoring to eliminate bash-specific constructs or to improve code readability; 13) The code for setting options for the nc (netcat) utility was different in different scripts for SST - now it is made identical. 14) Fixed long-time broken encryption via xbcrypt in combination with mariabackup and added support for key-based encryption via openssl utility, which is now enabled by default for encrypt=1 mode (this default mode can be changed using a new configuration file option "encypt-format=openssl|xbcrypt", which can be placed in the [mysqld], [sst] or in the [xtrabackup] section) - this change will allow us to use and to test the encypt=1 encryption without installing non-standard third-party utilities.
-
Sujatha authored
Problem: ======== Aborting OPTIMIZE TABLE still logs in binary logs and replicates to the Slave server. "Optimize table" command under execution, is killed by using "Ctrl-C" as shown below. MariaDB [test]> optimize table t2; ^CCtrl-C -- query killed. Continuing normally. In spite of query execution being interrupted the query gets written to binary log. Analysis: ======== Admin command execution logic is not handling KILL command, hence it ignores the KILL command and completes its execution. Fix: === Check for thread killed notification, during admin command execution and handle it. If thread kill occurs prior to any table modification the query will not be written to binary log. If kill happens after at least one table is modified then the query will be written to binary log. Ex: command in execution is 'OPTIMIZE TABLE t1,t2' and the thread kill happens after t1 table is modified then 'OPTIMIZE TABLE t1,t2' will be written to binary log as admin commands will not make the slave to diverge from master.
-
Sujatha authored
Problem: ======= In slave_parallel_mode=optimistic configuration, when admin commands and DML operation on the same table are scheduled simultaneously for execution, it results in lock conflict and slave server either hangs due to deadlock or goes down with an assert. Analysis: ======== Admin commands OPTIMIZE, REPAIR and ANALYZE are written to binary log as ordinary transactions. When 'slave_parallel_mode' is 'optimistic' DMLs are allowed to run in parallel. But these locks are not detected by parallel replication deadlock detection-and-handling mechanism. At times they result in deadlock or assertion. Fix: === Flag admin commands as DDL in Gtid_log_event at the time of writing to binary log. Add a new bit EXECUTED_TABLE_ADMIN_CMD to 'm_unsafe_rollback_flags'. During 'mysql_admin_table' command execution it accepts a list of tables to be processed and executes them in a loop. Upon successful execution enable 'EXECUTED_TABLE_ADMIN_CMD' bit in thd->transaction.stmt_unsafe_rollback_flags. Gtid_log_event constructor will notice this flag and mark the current transaction with 'FL_DDL' flag. Gtid_log_events marked as FL_DDL will not be scheduled parallel execution, on the slave. They will execute in isolation to prevent deadlocks. Note: Removed the call to 'trans_commit_implicit' from 'mysql_admin_table' function as 'mysql_execute_command' will take care of invoking 'trans_commit_implicit'.
-
- 16 May, 2021 1 commit
-
-
Daniel Black authored
No longer a MySQL server, "his" is the wrong pronoun for a server. Thanks Michael Newton for highlighting these problems Also changed slave -> replica.
-
- 15 May, 2021 2 commits
-
-
Julius Goryavsky authored
1) This commit implements reading all sections from configuration files while looking for the current value of any server variable, which were previously only read from the [mysqld.suffix] group and from [mysqld], but not from other groups such as [mariadb.suffix], [mariadb] or, for example, [server]. 2) This commit also fixes misrecognition of some parameters when parsing a command line containing a special marker for the end of the list of options ("--") or when short option names (such as "-s", "-a" and "-h arg") chained together (like a "-sah arg"). Such parameters can be passed to the SST script in the list of arguments after "--mysqld-args" if the server is started with a complex set of options - this was revealed during manual testing of changes to read configuration files. 3) The server-side preparation code for the "--mysqld-args" option list has also been simplified to make it easier to change in the future (if needed), and has been improved to properly handle the special backquote ("`") character in the argument values.
-
Julius Goryavsky authored
-
- 14 May, 2021 4 commits
-
-
Igor Babaev authored
If a select query contained an ORDER BY clause that followed a LIMIT clause or an ORDER BY clause or ORDER BY with LIMIT the EXPLAIN output for the query showed an execution plan different from that was actually executed. Approved by Roman Nozdrin <roman.nozdrin@mariadb.com>
-
Sachin Kumar authored
Problem:- When slave is shutdown, we will get this assertion failure sql/sql_list.h:642: void ilink::assert_linked(): Assertion `prev != 0 && next != 0' failed. Solution:- In close_connections when we call threads.get() it resets to prev and next to NULL. And in parallel worker thread(handle_rpl_parallel_thread) calls unlink_not_visible_thd() which assert on prev and next being not NULL. .unlink_not_visible_thd() should be always called first before threads.get() is called. To make sure worker calls unlink_not_visible_thd() in slave_prepare_for_shutdown() we are deactivating the worker thread pool which in turn will close all worker threads. Since this is already done in 10.4 and 10.5 I am backPorting MDEV-20821 and MDEV-22370 to 10.2. Mdev-22370 is improving the MDEV-20821 patch.
-
Sachin Kumar authored
MDEV-22370 safe_mutex: Trying to lock uninitialized mutex at /data/src/10.4-bug/sql/rpl_parallel.cc, line 470 upon shutdown during FTWRL Problem:- When we issue FTWRL with shutdown in parallel, there is race between FTWRL and shutdown. Shutdown might destroy the mutex (pool->LOCK_rpl_thread_pool) before FTWRL can lock it. So we can get crash on FTWRL thread Solution:- mysql_mutex_destroy(pool->LOCK_rpl_thread_pool) should wait for FTWRL thread to complete its work , and then destroy. So slave_prepare_for_shutdown will just deactivate the pool, and mutex is destroyed later in end_slave()
-
Andrei Elkin authored
Parallel slave server shutdown found to be hanging in close_connections() triggered by shutdown due to a slave worker thread would not be notified to exit in case the worker was sitting idle. Fixed with destroying the worker pool earlier that is in slave_prepare_for_shutdown() when all their driver threads have already left. A test file is added to simulate the bug condition as well as check multi-sourced and not-idle worker cases.
-
- 11 May, 2021 5 commits
-
-
Sergei Golubchik authored
-
Ramesh Sivaraman authored
-
Julius Goryavsky authored
-
Julius Goryavsky authored
-
Julius Goryavsky authored
This commit contains a large set of further bug fixes and improvements to SST scripts for Galera, continuing the work that was started in MDEV-24962 to make SST scripts work smoothly in different network configurations (especially using ipv6) and with different environment settings: 1) The ipv6 addresses were incorrectly handled in the SST script for rsync (incorrect address substitution for establishing a connection, incorrect address substitution for bind, and so on); 2) Checking the locality of the ip-address in SST scripts did not support ipv6 addresses (such as "[::1]"), which were falsely identified as non-local ip, which further did not allow running two SSTs on different local addresses on the same machine. On the other hand, this bug masked some other errors (related to handling ipv6 addresses); 3) The code for checking the locality of the ip address was different in the SST scripts for rsync and for mysqldump, with individual flaws. This code is now made common and moved to wsrep_sst_common; 4) Waiting for the start of the transport channel (socat, nc, rsync, stunnel) in the wait_for_listen() and check_pid_and_port() functions did not process ipv6 addresses correctly in all cases (not for all branches); 5) Waiting for the start of the transport channel (socat, nc, rsync, stunnel) in the wait_for_listen() and check_pid_and_port() functions for some code branches could give a false positive result due to the textual match of prefixes in the port number and/or PID of the process; 6) Waiting for the start of the transport channel (socat, nc, rsync, stunnel) was supported through different utilities in SST scripts for mariabackup and for rsync, and with various minor flaws in the code. Now the code is still different in these scripts, but it supports a common set of utilities (lsof, ss, sockstat) and is synchronized across patterns that used to check the output of these utilities; 7) In SST via mariabackup, the signal about readiness to receive data is sometimes sent too early - immediately after listen(), and not after accept() (which are called by socat or netcat utility). 8) Checking availability of the some options of some utilities was done using the grep pattern, which easily gives false positives; 9) Common name (CN) for local addresses, if not explicitly specified, is now always replaced to "localhost" to avoid the need to generate many separate certificates for local addresses of one machine and not to depend on which the local address is currently used in test (ipv4 or ipv6, etc.); 10) In tests galera_sst_mariabackup_encrypt_with_key_server and galera_sst_rsync_encrypt_with_key_server the correct certificate is selected to avoid commonname (CN) mismatch problems; 11) Further refactoring to protect against spaces in file names. 12) Further general refactoring to eliminate bash-specific constructs or to improve code readability; 13) The code for setting options for the nc (netcat) utility was different in different scripts for SST - now it is made identical. 14) Fixed long-time broken encryption via xbcrypt in combination with mariabackup and added support for key-based encryption via openssl utility, which is now enabled by default for encrypt=1 mode (this default mode can be changed using a new configuration file option "encypt-format=openssl|xbcrypt", which can be placed in the [mysqld], [sst] or in the [xtrabackup] section) - this change will allow us to use and to test the encypt=1 encryption without installing non-standard third-party utilities.
-
- 10 May, 2021 6 commits
-
-
Daniel Bartholomew authored
-
Daniel Bartholomew authored
-
Daniel Bartholomew authored
-
Julius Goryavsky authored
This commit contains a large set of further bug fixes and improvements to SST scripts for Galera, continuing the work that was started in MDEV-24962 to make SST scripts work smoothly in different network configurations (especially using ipv6) and with different environment settings: 1) The ipv6 addresses were incorrectly handled in the SST script for rsync (incorrect address substitution for establishing a connection, incorrect address substitution for bind, and so on); 2) Checking the locality of the ip-address in SST scripts did not support ipv6 addresses (such as "[::1]"), which were falsely identified as non-local ip, which further did not allow running two SSTs on different local addresses on the same machine. On the other hand, this bug masked some other errors (related to handling ipv6 addresses); 3) The code for checking the locality of the ip address was different in the SST scripts for rsync and for mysqldump, with individual flaws. This code is now made common and moved to wsrep_sst_common; 4) Waiting for the start of the transport channel (socat, nc, rsync, stunnel) in the wait_for_listen() and check_pid_and_port() functions did not process ipv6 addresses correctly in all cases (not for all branches); 5) Waiting for the start of the transport channel (socat, nc, rsync, stunnel) in the wait_for_listen() and check_pid_and_port() functions for some code branches could give a false positive result due to the textual match of prefixes in the port number and/or PID of the process; 6) Waiting for the start of the transport channel (socat, nc, rsync, stunnel) was supported through different utilities in SST scripts for mariabackup and for rsync, and with various minor flaws in the code. Now the code is still different in these scripts, but it supports a common set of utilities (lsof, ss, sockstat) and is synchronized across patterns that used to check the output of these utilities; 7) In SST via mariabackup, the signal about readiness to receive data is sometimes sent too early - immediately after listen(), and not after accept() (which are called by socat or netcat utility). 8) Checking availability of the some options of some utilities was done using the grep pattern, which easily gives false positives; 9) Common name (CN) for local addresses, if not explicitly specified, is now always replaced to "localhost" to avoid the need to generate many separate certificates for local addresses of one machine and not to depend on which the local address is currently used in test (ipv4 or ipv6, etc.); 10) In tests galera_sst_mariabackup_encrypt_with_key_server and galera_sst_rsync_encrypt_with_key_server the correct certificate is selected to avoid commonname (CN) mismatch problems; 11) Further refactoring to protect against spaces in file names. 12) Further general refactoring to eliminate bash-specific constructs or to improve code readability; 13) The code for setting options for the nc (netcat) utility was different in different scripts for SST - now it is made identical. 14) Fixed long-time broken encryption via xbcrypt in combination with mariabackup and added support for key-based encryption via openssl utility, which is now enabled by default for encrypt=1 mode (this default mode can be changed using a new configuration file option "encypt-format=openssl|xbcrypt", which can be placed in the [mysqld], [sst] or in the [xtrabackup] section) - this change will allow us to use and to test the encypt=1 encryption without installing non-standard third-party utilities.
-
Marko Mäkelä authored
-
Marko Mäkelä authored
-
- 09 May, 2021 1 commit
-
-
Vladislav Vaintroub authored
This is incosistent with other single config generators, where the default type has always been RelWithDebInfo. Fixed by moving setting of CMAKE_BUILD_TYPE right before PROJECT.
-
- 08 May, 2021 2 commits
-
-
Sergei Golubchik authored
unless explicitly overruled with -DWITH_JEMALLOC=no
-
Sergei Golubchik authored
XA transaction only allows to access data in specific states, in ACTIVE, but not in IDLE or PREPARE. But even then one should be able to run SHOW STATUS.
-