- 07 Sep, 2018 10 commits
-
-
Marko Mäkelä authored
This is a backport of commit 07e9ff1f. Allow DROP TABLE `#mysql50##sql-...._.` to drop tables that were being rebuilt by ALGORITHM=INPLACE NOTE: If the server is killed after the table-rebuilding ALGORITHM=INPLACE commits inside InnoDB but before the .frm file has been replaced, then the recovery will involve something else than DROP TABLE. NOTE: If the server is killed in a true inplace ALTER TABLE commits inside InnoDB but before the .frm file has been replaced, then we are really out of luck. To properly handle that situation, we would need a transactional mysql.ddl_fixup table that directs recovery to rename or remove files. prepare_inplace_alter_table_dict(): Use the altered_table->s->table_name for generating the new_table_name. table_name_t::part_suffix: The start of the partition name suffix. table_name_t::dbend(): Return the end of the schema name. table_name_t::dblen(): Return the length of the schema name, in bytes. table_name_t::basename(): Return the name without the schema name. table_name_t::part(): Return the partition name, or NULL if none. row_drop_table_for_mysql(): Assert for #sql, not #sql-ib.
-
Marko Mäkelä authored
This is a backport of commit 0bc36758 and commit 9eb3fcc9. InnoDB in MariaDB 10.2 appears to only write MLOG_FILE_RENAME2 redo log records during table-rebuilding ALGORITHM=INPLACE operations. We must write the records for any .ibd file renames, so that the operations are crash-safe. If InnoDB is killed during a RENAME TABLE operation, it can happen that the transaction for updating the data dictionary will be rolled back. But, nothing will roll back the renaming of the .ibd file (the MLOG_FILE_RENAME2 only guarantees roll-forward), or for that matter, the renaming of the dict_table_t::name in the dict_sys cache. We introduce the undo log record TRX_UNDO_RENAME_TABLE to fix this. fil_space_for_table_exists_in_mem(): Remove the parameters adjust_space, table_id and some code that was trying to work around these deficiencies. fil_name_write_rename(): Write a MLOG_FILE_RENAME2 record. dict_table_rename_in_cache(): Invoke fil_name_write_rename(). trx_undo_rec_copy(): Set the first 2 bytes to the length of the copied undo log record. trx_undo_page_report_rename(), trx_undo_report_rename(): Write a TRX_UNDO_RENAME_TABLE record with the old table name. row_rename_table_for_mysql(): Invoke trx_undo_report_rename() before modifying any data dictionary tables. row_undo_ins_parse_undo_rec(): Roll back TRX_UNDO_RENAME_TABLE by invoking dict_table_rename_in_cache(), which will take care of both renaming the table and the file. ha_innobase::truncate(): Remove a work-around.
-
Marko Mäkelä authored
Remove the innodb_undo suite, and move and adapt the tests. Remove unnecessary restarts, and add innodb_page_size_small.inc for combinations. innodb.undo_truncate is the merge of innodb_undo.truncate and innodb_undo.truncate_multi_client. Add the global status variable innodb_undo_truncations. Without this, the test innodb.undo_truncate would occasionally report that truncation did not happen. The test was only waiting for the history list length to reach 0, but the undo tablespace truncation would only take place some time after that. Undo tablespace truncation will only occasionally occur with innodb_page_size=32k, and typically never occur (with this amount of undo log operations) with innodb_page_size=64k. We disable these combinations. innodb.undo_truncate_recover was formerly called innodb_undo.truncate_recover.
-
Marko Mäkelä authored
Implement undo tablespace truncation via normal redo logging. Implement TRUNCATE TABLE as a combination of RENAME to #sql-ib name, CREATE, and DROP. Note: Orphan #sql-ib*.ibd may be left behind if MariaDB Server 10.2 is killed before the DROP operation is committed. If MariaDB Server 10.2 is killed during TRUNCATE, it is also possible that the old table was renamed to #sql-ib*.ibd but the data dictionary will refer to the table using the original name. In MariaDB Server 10.3, RENAME inside InnoDB is transactional, and #sql-* tables will be dropped on startup. So, this new TRUNCATE will be fully crash-safe in 10.3. ha_mroonga::wrapper_truncate(): Pass table options to the underlying storage engine, now that ha_innobase::truncate() will need them. rpl_slave_state::truncate_state_table(): Before truncating mysql.gtid_slave_pos, evict any cached table handles from the table definition cache, so that there will be no stale references to the old table after truncating. == TRUNCATE TABLE == WL#6501 in MySQL 5.7 introduced separate log files for implementing atomic and crash-safe TRUNCATE TABLE, instead of using the InnoDB undo and redo log. Some convoluted logic was added to the InnoDB crash recovery, and some extra synchronization (including a redo log checkpoint) was introduced to make this work. This synchronization has caused performance problems and race conditions, and the extra log files cannot be copied or applied by external backup programs. In order to support crash-upgrade from MariaDB 10.2, we will keep the logic for parsing and applying the extra log files, but we will no longer generate those files in TRUNCATE TABLE. A prerequisite for crash-safe TRUNCATE is a crash-safe RENAME TABLE (with full redo and undo logging and proper rollback). This will be implemented in MDEV-14717. ha_innobase::truncate(): Invoke RENAME, create(), delete_table(). Because RENAME cannot be fully rolled back before MariaDB 10.3 due to missing undo logging, add some explicit rename-back in case the operation fails. ha_innobase::delete(): Introduce a variant that takes sqlcom as a parameter. In TRUNCATE TABLE, we do not want to touch any FOREIGN KEY constraints. ha_innobase::create(): Add the parameters file_per_table, trx. In TRUNCATE, the new table must be created in the same transaction that renames the old table. create_table_info_t::create_table_info_t(): Add the parameters file_per_table, trx. row_drop_table_for_mysql(): Replace a bool parameter with sqlcom. row_drop_table_after_create_fail(): New function, wrapping row_drop_table_for_mysql(). dict_truncate_index_tree_in_mem(), fil_truncate_tablespace(), fil_prepare_for_truncate(), fil_reinit_space_header_for_table(), row_truncate_table_for_mysql(), TruncateLogger, row_truncate_prepare(), row_truncate_rollback(), row_truncate_complete(), row_truncate_fts(), row_truncate_update_system_tables(), row_truncate_foreign_key_checks(), row_truncate_sanity_checks(): Remove. row_upd_check_references_constraints(): Remove a check for TRUNCATE, now that the table is no longer truncated in place. The new test innodb.truncate_foreign uses DEBUG_SYNC to cover some race-condition like scenarios. The test innodb-innodb.truncate does not use any synchronization. We add a redo log subformat to indicate backup-friendly format. MariaDB 10.4 will remove support for the old TRUNCATE logging, so crash-upgrade from old 10.2 or 10.3 to 10.4 will involve limitations. == Undo tablespace truncation == MySQL 5.7 implements undo tablespace truncation. It is only possible when innodb_undo_tablespaces is set to at least 2. The logging is implemented similar to the WL#6501 TRUNCATE, that is, using separate log files and a redo log checkpoint. We can simply implement undo tablespace truncation within a single mini-transaction that reinitializes the undo log tablespace file. Unfortunately, due to the redo log format of some operations, currently, the total redo log written by undo tablespace truncation will be more than the combined size of the truncated undo tablespace. It should be acceptable to have a little more than 1 megabyte of log in a single mini-transaction. This will be fixed in MDEV-17138 in MariaDB Server 10.4. recv_sys_t: Add truncated_undo_spaces[] to remember for which undo tablespaces a MLOG_FILE_CREATE2 record was seen. namespace undo: Remove some unnecessary declarations. fil_space_t::is_being_truncated: Document that this flag now only applies to undo tablespaces. Remove some references. fil_space_t::is_stopping(): Do not refer to is_being_truncated. This check is for tablespaces of tables. Potentially used tablespaces are never truncated any more. buf_dblwr_process(): Suppress the out-of-bounds warning for undo tablespaces. fil_truncate_log(): Write a MLOG_FILE_CREATE2 with a nonzero page number (new size of the tablespace in pages) to inform crash recovery that the undo tablespace size has been reduced. fil_op_write_log(): Relax assertions, so that MLOG_FILE_CREATE2 can be written for undo tablespaces (without .ibd file suffix) for a nonzero page number. os_file_truncate(): Add the parameter allow_shrink=false so that undo tablespaces can actually be shrunk using this function. fil_name_parse(): For undo tablespace truncation, buffer MLOG_FILE_CREATE2 in truncated_undo_spaces[]. recv_read_in_area(): Avoid reading pages for which no redo log records remain buffered, after recv_addr_trim() removed them. trx_rseg_header_create(): Add a FIXME comment that we could write much less redo log. trx_undo_truncate_tablespace(): Reinitialize the undo tablespace in a single mini-transaction, which will be flushed to the redo log before the file size is trimmed. recv_addr_trim(): Discard any redo logs for pages that were logged after the new end of a file, before the truncation LSN. If the rec_list becomes empty, reduce n_addrs. After removing any affected records, actually truncate the file. recv_apply_hashed_log_recs(): Invoke recv_addr_trim() right before applying any log records. The undo tablespace files must be open at this point. buf_flush_or_remove_pages(), buf_flush_dirty_pages(), buf_LRU_flush_or_remove_pages(): Add a parameter for specifying the number of the first page to flush or remove (default 0). trx_purge_initiate_truncate(): Remove the log checkpoints, the extra logging, and some unnecessary crash points. Merge the code from trx_undo_truncate_tablespace(). First, flush all to-be-discarded pages (beyond the new end of the file), then trim the space->size to make the page allocation deterministic. At the only remaining crash injection point, flush the redo log, so that the recovery can be tested.
-
Marko Mäkelä authored
-
Marko Mäkelä authored
recv_addr_state, recv_addr_t: Define in log0recv.cc only.
-
Marko Mäkelä authored
-
Marko Mäkelä authored
-
Marko Mäkelä authored
Because innodb_file_per_table can be enabled at runtime after it was disabled at startup, it is better to always register the same innobase_hton->tablefile_extensions. Besides, innodb_file_per_table=OFF does not prevent loading tables that may have been created earlier with the .ibd file extension.
-
Vladislav Vaintroub authored
Remove plugin-load option from mariabackup. It does not needed to be an option (we only need to store the plugin-load value during backup phase, and reuse the same value during --prepare). Fix is to read plugin-load from backup-my.cnf during prepare.
-
- 06 Sep, 2018 2 commits
-
-
Sergei Golubchik authored
-
Vladislav Vaintroub authored
Fix exit condition for the log copying thread.
-
- 05 Sep, 2018 1 commit
-
-
Vladislav Vaintroub authored
(https://github.com/facebook/rocksdb/issues/4344) Also, disable /permissive- flag if set, it breaks rocksdb compilation in 10.3 on older versions of Windows 8.1 SDK.
-
- 03 Sep, 2018 1 commit
-
-
Sergei Petrunia authored
It uses --slave-use-idempotent-for-recovery which is not available in MariaDB.
-
- 01 Sep, 2018 2 commits
-
-
Sergei Petrunia authored
-
Sergei Petrunia authored
Remove the mismatch betwen list of man pages in CMakeLists.txt and actual files.
-
- 31 Aug, 2018 9 commits
-
-
Marko Mäkelä authored
-
Marko Mäkelä authored
-
Sergei Petrunia authored
-
Sergei Petrunia authored
Direct IO doesnt work on all filesystems, e.g. it doesnt work on the disk of kvm-bintar-trusty-amd64.
-
Jan Lindström authored
-
Marko Mäkelä authored
-
Sergei Petrunia authored
-
Marko Mäkelä authored
This regression was introduced in MDEV-16515. We would fail to drop a temporary table on client disconnect, because trx_is_interrupted() would hold. To add insult to injury, in MariaDB 10.1, InnoDB temporary tables are actually persistent, so the garbage temporary tables will never be dropped. row_drop_table_for_mysql(): If several iterations of buf_LRU_drop_page_hash_for_tablespace() are needed, do not interrupt dropping a temporary table even after the transaction was marked as killed. Server shutdown will still terminate the loop, and also DROP TABLE of persistent tables will keep checking if the execution was aborted.
-
Marko Mäkelä authored
This reverts commit 75dfd4ac.
-
- 30 Aug, 2018 8 commits
-
-
Sergei Petrunia authored
It uses a feature that's not available in MariaDB
-
Sergei Petrunia authored
It uses a feature that's not available in MariaDB
-
Sergei Petrunia authored
-
Sergei Petrunia authored
-
Alexey Botchkov authored
Plugin version name fixed.
-
Alexey Botchkov authored
Let '\\' and '/' symbols to be part of a group name.
-
Sergei Petrunia authored
-
Igor Babaev authored
Field_iterator_table_ref::set_field_iterator Several functions that processed different prepare statements missed the DT_INIT flag in last parameter of the open_normal_and_derived_tables() calls. It made context analysis of derived tables dependent on the order in which the derived tables were processed by mysql_handle_derived(). This order was induced by the order of SELECTs in all_select_list. In 10.4 the order of SELECTs in all_select_list became different and lack of the DT_INIT flags in some open_normal_and_derived_tables() call became critical as some derived tables were not identified as such.
-
- 29 Aug, 2018 3 commits
-
-
Jan Lindström authored
Merged https://github.com/MariaDB/server/pull/617 authored by Colin Mollenhour.
-
Sergei Petrunia authored
-
Ian Gilfillan authored
-
- 28 Aug, 2018 4 commits
-
-
Sergei Petrunia authored
-
Sergei Petrunia authored
-
Sergei Petrunia authored
-
Sergei Petrunia authored
-