- 17 May, 2021 12 commits
-
-
Julius Goryavsky authored
Fixed bugs caused by inaccuracies in automatic merging from other branches: 1) Authentication information is not removed from the connection address, which causes some tests to fail; 2) wsrep_debug=on should be replaced with wsrep_debug=1; 3) Added missing "connection" lines to test result file; 4) Some tests have been corrected for Galera 4.x (10.4+).
-
Julius Goryavsky authored
-
Julius Goryavsky authored
1) This commit implements reading all sections from configuration files while looking for the current value of any server variable, which were previously only read from the [mysqld.suffix] group and from [mysqld], but not from other groups such as [mariadb.suffix], [mariadb] or, for example, [server]. 2) This commit also fixes misrecognition of some parameters when parsing a command line containing a special marker for the end of the list of options ("--") or when short option names (such as "-s", "-a" and "-h arg") chained together (like a "-sah arg"). Such parameters can be passed to the SST script in the list of arguments after "--mysqld-args" if the server is started with a complex set of options - this was revealed during manual testing of changes to read configuration files. 3) The server-side preparation code for the "--mysqld-args" option list has also been simplified to make it easier to change in the future (if needed), and has been improved to properly handle the special backquote ("`") character in the argument values.
-
Julius Goryavsky authored
-
Julius Goryavsky authored
This commit contains a large set of further bug fixes and improvements to SST scripts for Galera, continuing the work that was started in MDEV-24962 to make SST scripts work smoothly in different network configurations (especially using ipv6) and with different environment settings: 1) The ipv6 addresses were incorrectly handled in the SST script for rsync (incorrect address substitution for establishing a connection, incorrect address substitution for bind, and so on); 2) Checking the locality of the ip-address in SST scripts did not support ipv6 addresses (such as "[::1]"), which were falsely identified as non-local ip, which further did not allow running two SSTs on different local addresses on the same machine. On the other hand, this bug masked some other errors (related to handling ipv6 addresses); 3) The code for checking the locality of the ip address was different in the SST scripts for rsync and for mysqldump, with individual flaws. This code is now made common and moved to wsrep_sst_common; 4) Waiting for the start of the transport channel (socat, nc, rsync, stunnel) in the wait_for_listen() and check_pid_and_port() functions did not process ipv6 addresses correctly in all cases (not for all branches); 5) Waiting for the start of the transport channel (socat, nc, rsync, stunnel) in the wait_for_listen() and check_pid_and_port() functions for some code branches could give a false positive result due to the textual match of prefixes in the port number and/or PID of the process; 6) Waiting for the start of the transport channel (socat, nc, rsync, stunnel) was supported through different utilities in SST scripts for mariabackup and for rsync, and with various minor flaws in the code. Now the code is still different in these scripts, but it supports a common set of utilities (lsof, ss, sockstat) and is synchronized across patterns that used to check the output of these utilities; 7) In SST via mariabackup, the signal about readiness to receive data is sometimes sent too early - immediately after listen(), and not after accept() (which are called by socat or netcat utility). 8) Checking availability of the some options of some utilities was done using the grep pattern, which easily gives false positives; 9) Common name (CN) for local addresses, if not explicitly specified, is now always replaced to "localhost" to avoid the need to generate many separate certificates for local addresses of one machine and not to depend on which the local address is currently used in test (ipv4 or ipv6, etc.); 10) In tests galera_sst_mariabackup_encrypt_with_key_server and galera_sst_rsync_encrypt_with_key_server the correct certificate is selected to avoid commonname (CN) mismatch problems; 11) Further refactoring to protect against spaces in file names. 12) Further general refactoring to eliminate bash-specific constructs or to improve code readability; 13) The code for setting options for the nc (netcat) utility was different in different scripts for SST - now it is made identical. 14) Fixed long-time broken encryption via xbcrypt in combination with mariabackup and added support for key-based encryption via openssl utility, which is now enabled by default for encrypt=1 mode (this default mode can be changed using a new configuration file option "encypt-format=openssl|xbcrypt", which can be placed in the [mysqld], [sst] or in the [xtrabackup] section) - this change will allow us to use and to test the encypt=1 encryption without installing non-standard third-party utilities.
-
Marko Mäkelä authored
During data file creation, InnoDB holds dict_sys mutex, tries to write page 0 of the file and flushes the file. This not only causing unnecessary contention but also a deviation from the write-ahead logging protocol. The clean sequence of operations is that we first start a dictionary transaction and write SYS_TABLES and SYS_INDEXES records that identify the tablespace. Then, we durably write a FILE_CREATE record to the write-ahead log and create the file. Recovery should not unnecessarily insist that the first page of each data file that is referred to by the redo log is valid. It must be enough that page 0 of the tablespace can be initialized based on the redo log contents. We introduce a new data structure deferred_spaces that keeps track of corrupted-looking files during recovery. The data structure holds the last LSN of a FILE_ record referring to the data file, the tablespace identifier, and the last known file name. There are two scenarios can happen during recovery: i) Sufficient memory: InnoDB can reconstruct the tablespace after parsing all redo log records. ii) Insufficient memory(multiple apply phase): InnoDB should store the deferred tablespace redo logs even though tablespace is not present. InnoDB should start constructing the tablespace when it first encounters deferred tablespace id. Mariabackup copies the zero filled ibd file in backup_fix_ddl() as the extension of .new file. Mariabackup test case does page flushing when it deals with DDL operation during backup operation. fil_ibd_create(): Remove the write of page0 and flushing of file fil_ibd_load(): Return FIL_LOAD_DEFER if the tablespace has zero filled page0 Datafile: Clean up the error handling, and do not report errors if we are in the middle of recovery. The caller will check Datafile::m_defer. fil_node_t::deferred: Indicates whether the tablespace loading was deferred during recovery FIL_LOAD_DEFER: Returned by fil_ibd_load() to indicate that tablespace file was cannot be loaded. recv_sys_t::recover_deferred(): Invoke deferred_spaces.create() to initialize fil_space_t based on buffered metadata and records to initialize page 0. Ignore the flags in fil_name_t, because they are intentionally invalid. fil_name_process(): Update deferred_spaces. recv_sys_t::parse(): Store the redo log if the tablespace id is present in deferred spaces recv_sys_t::recover_low(): Should recover the first page of the tablespace even though the tablespace instance is not present recv_sys_t::apply(): Initialize the deferred tablespace before applying the deferred tablespace records recv_validate_tablespace(): Skip the validation for deferred_spaces. recv_rename_files(): Moved and revised from recv_sys_t::apply(). For deferred-recovery tablespaces, do not attempt to rename the file if a deferred-recovery tablespace is associated with the name. recv_recovery_from_checkpoint_start(): Invoke recv_rename_files() and initialize all deferred tablespaces before applying redo log. fil_node_t::read_page0(): Skip page0 validation if the tablespace is deferred buf_page_create_deferred(): A variant of buf_page_create() when the fil_space_t is not available yet This is joint work with Thirunarayanan Balathandayuthapani, who implemented an initial prototype.
-
Marko Mäkelä authored
Compilers older than GCC 4.8 are not supported anyway.
-
Sujatha authored
Post push fixes. * Set binlog_expire_logs_seconds max valid value to 8553600 i.e 99 * 24 * 60 * 60 (99 days)
-
Marko Mäkelä authored
The counter metadata_table_reference_count was not updated consistently ever since mysql-server@65c0af9a1dedae43b63797134aff6b32304ced52 or commit 2e814d47 introduced dict_table_t::release(). The counter metadata_table_handles_closed was being incremented unconditionally in ha_innobase::close(), while the corresponding counter metadata_table_handles_opened would be incremented in ha_innobase::open() if the function returned early due to an error.
-
Marko Mäkelä authored
The error code DB_TABLE_IS_BEING_USED was orphaned in commit ddc663ef (MySQL 3.23.37). Let us finally remove it.
-
Marko Mäkelä authored
MONITOR_TRX_ACTIVE: Remove. The count is not being updated consistently, and it would also include read-only transactions that are otherwise fully invisible to any other threads. If it later turns out that a reliable count of active transactions is needed, it can be exposed via a different interface. trx_commit_for_mysql(): If the transaction was not started, return immediately.
-
Marko Mäkelä authored
row_drop_table_after_create_fail(): Remove. This function was only used during InnoDB initialization (or upgrade) when creating a system table failed. Rollback will clean up a failed CREATE just fine, by invoking dict_drop_index_tree().
-
- 15 May, 2021 1 commit
-
-
Jan Lindström authored
Added DDL logging to applier and replaying also so that DDL is logged on other than originating node. wsrep.h Removed wsrep_thd_is_local conditions and cleaned up the macros. Removed WSREP_TO_ISOLATION_END. Event_job_data::execute change_password acl_set_default_role mysql_execute_command Replaced macro by function call wsrep_to_isolation_begin wsrep_to_isolation_end If execution is not local log DDL-information when wsrep_debug is enabled No new tests required as current regression setting is already testing these code paths.
-
- 14 May, 2021 4 commits
-
-
Marko Mäkelä authored
Ever since MDEV-18518 made DDL operations mostly crash-safe inside InnoDB, it became obvious that Mariabackup might not be entirely safe with regard to concurrent DDL operations. check_if_skip_table(): Do not skip files whose name starts with #sql. We cannot know whether a DDL operation is in progress and the table might in fact be needed later.
-
Marko Mäkelä authored
-
Marko Mäkelä authored
-
Marko Mäkelä authored
-
- 13 May, 2021 3 commits
-
-
Sergei Petrunia authored
Mark the JSON_TABLE function as SBR-unsafe. It is not unsafe for the current implementation. But we still mark it as such in order to be future-proof and keep it possible to change JSON data representation in the future.
-
Marko Mäkelä authored
-
Sujatha authored
Problem: ======= Test fails with 3 different symptoms connection slave; Assertion text: 'Last_Seen_Transaction should show .' Assertion condition: '"0-1-1" = ""' Assertion condition, interpolated: '"0-1-1" = ""' Assertion result: '0' connection slave; Assertion text: 'Value returned by SSS and PS table for Last_Error_Number should be same.' Assertion condition: '"1146" = "0"' Assertion condition, interpolated: '"1146" = "0"' Assertion result: '0' connection slave; Assertion text: 'Value returned by PS table for worker_idle_time should be >= 1' Assertion condition: '"0" >= "1"' Assertion condition, interpolated: '"0" >= "1"' Assertion result: '0' Fix1: ==== Performance schema table's Last_Seen_Transaction is compared with 'SELECT gtid_slave_pos'. Since DDLs are not transactional changes to user table and gtid_slave_pos table are not guaranteed to be synchronous. To fix the issue Gtid_IO_Pos value from SHOW SLAVE STATUS command will be used to verify the correctness of Performance schema specific Last_Seen_Transaction. Fix2: ==== On error worker thread information is stored as part of backup pool. Access to this backup pool should be protected by 'LOCK_rpl_thread_pool' mutex so that simultaneous START SLAVE cannot destroy the backup pool, while it is being queried by performance schema. Fix3: ==== When a worker is waiting for events if performance schema table is queried, at present it just returns the difference between current_time and start_time. This is incorrect. It should be worker_idle_time + (current_time - start_time). For example a worker thread was idle for 10 seconds and then it got events to process. Upon completion it goes to idle state, now if the pfs table is queried it should return current_idle time + worker_idle_time.
-
- 12 May, 2021 3 commits
-
-
Nikita Malyavin authored
-
Kentoku SHIBA authored
==22532== Conditional jump or move depends on uninitialised value(s) ==22532== at 0x9EEDFA: String::append(char const*, unsigned long, charset_info_st const*) (sql_string.cc:587) ==22532== by 0x17C12BA7: spider_string::append(char const*, unsigned int, charset_info_st const*) (spd_malloc.cc:913) ==22532== by 0x17C68BD0: spider_db_mysql_util::append_column_value(ha_spider*, spider_string*, Field*, unsigned char const*, charset_info_st const*) (spd_db_mysql.cc:4573) ==22532== by 0x17B7D298: spider_db_append_key_where_internal(spider_string*, spider_string*, spider_string*, st_key_range const*, st_key_range const*, ha_spider*, bool, unsigned long, unsigned int) (spd_db_conn.cc:1978) ==22532== by 0x17C81A7C: spider_mbase_handler::append_key_where(spider_string*, spider_string*, spider_string*, st_key_range const*, st_key_range const*, unsigned long, bool) (spd_db_mysql.cc:11146) ==22532== by 0x17C819B7: spider_mbase_handler::append_key_where_part(st_key_range const*, st_key_range const*, unsigned long) (spd_db_mysql.cc:11130) ==22532== by 0x17C4B0F0: ha_spider::append_key_where_sql_part(st_key_range const*, st_key_range const*, unsigned long) (ha_spider.cc:15012) ==22532== by 0x17B7FC8F: spider_db_append_key_where(st_key_range const*, st_key_range const*, ha_spider*) (spd_db_conn.cc:2707) ==22532== by 0x17C28DF6: ha_spider::multi_range_read_next_first(void**) (ha_spider.cc:4559) ==22532== by 0x17C2E0F5: ha_spider::multi_range_read_next(void**) (ha_spider.cc:5879) ==22532== by 0xE27019: QUICK_RANGE_SELECT::get_next() (opt_range.cc:12647) Conflicts: storage/spider/spd_malloc.cc
-
Marko Mäkelä authored
-
- 11 May, 2021 4 commits
-
-
Monty authored
-
Monty authored
The problem was that in a timeout event, thd->lex->restore_backup_query_tables_list() was called when it should not have been. Patch tested with the script in MDEV-25651 (not suitable for mtr)
-
Brandon Nesterenko authored
Problem: ======= MariaDB's command line utilities (e.g., mysql, mysqldump, etc) silently ignore connection property options (e.g., --port and --socket) when protocol is not explicitly set via the command-line for localhost connections. Fix: === If connection properties are specified without a protocol, override the protocol to be consistent. For example, if --port is specified, automatically set protocol=tcp. Caveats: ======= * When multiple connection properties are specified, nothing is overridden * If protocol is is set via the command-line, its value is used Reviewers: ======== Sergei Golubchik <serg@mariadb.com> Vladislav Vaintroub <wlad@mariadb.com>
-
Marko Mäkelä authored
In commit 38fd7b7d (MDEV-21452), 2 of the 3 callers of recv_sys.set_corrupt_fs() are already holding recv_sys.mutex.
-
- 10 May, 2021 8 commits
-
-
Monty authored
The bug is that we don't have a a lock on the trigger name, so it is possible for two threads to try to create the same trigger at the same time and both thinks that they have succeed. Same thing can happen with drop trigger or a combinations of create and drop trigger. Fixed by adding a mdl lock for the trigger name for the duration of the create/drop. Patch tested by Elena
-
Daniel Bartholomew authored
-
Marko Mäkelä authored
-
Marko Mäkelä authored
-
Marko Mäkelä authored
-
Marko Mäkelä authored
-
Sujatha authored
Part2: Test scripts
-
Sujatha authored
Part1: Functional changes Backporting upstream changes. commit a7e1ef858ee82493dd8ad9a76bc9c22fe3b8c05b Author: Neha Kumari <neha.n.kumari@oracle.com> Note: From the upstream patch only the new option binlog_expire_logs_seconds specific changes are taken. * Unlike in the upstream patch 'binlog_expire_logs_seconds' does not replace the "old" 'expire_logs_days', to preserve backward-compatibility. * Datatype of 'expire_logs_days' variable is changed to double. * Default value of 'binlog_expire_logs_seconds=0' similar to 'expire_logs_days'. * The purge_time can be specified in days with the micro-day precision. Eg: expire_logs_days=1 is the same as expire_logs_days=1.000000 to make binlog_expire_logs_seconds=86400. binlog_expire_logs_seconds=1 is the same as expire_logs_days=0.000012. * If binary log is disabled and option 'expire_logs_days' or 'binlog_expire_logs_seconds' used with purge_time > 0 a warning will be issued.
-
- 09 May, 2021 5 commits
-
-
Vladislav Vaintroub authored
-
Vladislav Vaintroub authored
-
Vladislav Vaintroub authored
If IPO is enabled, MSVC compiles with /GL which is not compatible with cmake's WINDOWS_EXPORT_ALL_SYMBOLS. Since server.dll can't autoexport symbols from /GL compiled objects, IPO must be disabled for dbug, string,mysys, and sql targets (in case we build "plugin" DLLs that need server.dll's symbols)
-
Vladislav Vaintroub authored
The warning brought by code analysis, unsafe arithmetic in timer related code
-
Vladislav Vaintroub authored
Avoid calling constructor for Protocol_text, if not necessary.
-