MDEV-13542: Crashing on corrupted page is unhelpful

The approach to handling corruption that was chosen by Oracle in
commit 177d8b0c
is not really useful. Not only did it actually fail to prevent InnoDB
from crashing, but it is making things worse by blocking attempts to
rescue data from or rebuild a partially readable table.

We will try to prevent crashes in a different way: by propagating
errors up the call stack. We will never mark the clustered index
persistently corrupted, so that data recovery may be attempted by
reading from the table, or by rebuilding the table.

This should also fix MDEV-13680 (crash on btr_page_alloc() failure);
it was extensively tested with innodb_file_per_table=0 and a
non-autoextend system tablespace.

We should now avoid crashes in many cases, such as when a page
cannot be read or allocated, or an inconsistency is detected when
attempting to update multiple pages. We will not crash on double-free,
such as on the recovery of DDL in system tablespace in case something
was corrupted.

Crashes on corrupted data are still possible. The fault injection mechanism
that is introduced in the subsequent commit may help catch more of them.

buf_page_import_corrupt_failure: Remove the fault injection, and instead
corrupt some pages using Perl code in the tests.

btr_cur_pessimistic_insert(): Always reserve extents (except for the
change buffer), in order to prevent a subsequent allocation failure.

btr_pcur_open_at_rnd_pos(): Merged to the only caller ibuf_merge_pages().

btr_assert_not_corrupted(), btr_corruption_report(): Remove.
Similar checks are already part of btr_block_get().

FSEG_MAGIC_N_BYTES: Replaces FSEG_MAGIC_N_VALUE.

dict_hdr_get(), trx_rsegf_get_new(), trx_undo_page_get(),
trx_undo_page_get_s_latched(): Replaced with error-checking calls.

trx_rseg_t::get(mtr_t*): Replaces trx_rsegf_get().

trx_rseg_header_create(): Let the caller update the TRX_SYS page if needed.

trx_sys_create_sys_pages(): Merged with trx_sysf_create().

dict_check_tablespaces_and_store_max_id(): Do not access
DICT_HDR_MAX_SPACE_ID, because it was already recovered in dict_boot().
Merge dict_check_sys_tables() with this function.

dir_pathname(): Replaces os_file_make_new_pathname().

row_undo_ins_remove_sec(): Do not modify the undo page by adding
a terminating NUL byte to the record.

btr_decryption_failed(): Report decryption failures

dict_set_corrupted_by_space(), dict_set_encrypted_by_space(),
dict_set_corrupted_index_cache_only(): Remove.

dict_set_corrupted(): Remove the constant parameter dict_locked=false.
Never flag the clustered index corrupted in SYS_INDEXES, because
that would deny further access to the table. It might be possible to
repair the table by executing ALTER TABLE or OPTIMIZE TABLE, in case
no B-tree leaf page is corrupted.

dict_table_skip_corrupt_index(), dict_table_next_uncorrupted_index(),
row_purge_skip_uncommitted_virtual_index(): Remove, and refactor
the callers to read dict_index_t::type only once.

dict_table_is_corrupted(): Remove.

dict_index_t::is_btree(): Determine if the index is a valid B-tree.

BUF_GET_NO_LATCH, BUF_EVICT_IF_IN_POOL: Remove.

UNIV_BTR_DEBUG: Remove. Any inconsistency will no longer trigger
assertion failures, but error codes being returned.

buf_corrupt_page_release(): Replaced with a direct call to
buf_pool.corrupted_evict().

fil_invalid_page_access_msg(): Never crash on an invalid read;
let the caller of buf_page_get_gen() decide.

btr_pcur_t::restore_position(): Propagate failure status to the caller
by returning CORRUPTED.

opt_search_plan_for_table(): Simplify the code.

row_purge_del_mark(), row_purge_upd_exist_or_extern_func(),
row_undo_ins_remove_sec_rec(), row_undo_mod_upd_del_sec(),
row_undo_mod_del_mark_sec(): Avoid mem_heap_create()/mem_heap_free()
when no secondary indexes exist.

row_undo_mod_upd_exist_sec(): Simplify the code.

row_upd_clust_step(), dict_load_table_one(): Return DB_TABLE_CORRUPT
if the clustered index (and therefore the table) is corrupted, similar
to what we do in row_insert_for_mysql().

fut_get_ptr(): Replace with buf_page_get_gen() calls.

buf_page_get_gen(): Return nullptr and *err=DB_CORRUPTION
if the page is marked as freed. For other modes than
BUF_GET_POSSIBLY_FREED or BUF_PEEK_IF_IN_POOL this will
trigger a debug assertion failure. For BUF_GET_POSSIBLY_FREED,
we will return nullptr for freed pages, so that the callers
can be simplified. The purge of transaction history will be
a new user of BUF_GET_POSSIBLY_FREED, to avoid crashes on
corrupted data.

buf_page_get_low(): Never crash on a corrupted page, but simply
return nullptr.

fseg_page_is_allocated(): Replaces fseg_page_is_free().

fts_drop_common_tables(): Return an error if the transaction
was rolled back.

fil_space_t::set_corrupted(): Report a tablespace as corrupted if
it was not reported already.

fil_space_t::io(): Invoke fil_space_t::set_corrupted() to report
out-of-bounds page access or other errors.

Clean up mtr_t::page_lock()

buf_page_get_low(): Validate the page identifier (to check for
recently read corrupted pages) after acquiring the page latch.

buf_page_t::read_complete(): Flag uninitialized (all-zero) pages
with DB_FAIL. Return DB_PAGE_CORRUPTED on page number mismatch.

mtr_t::defer_drop_ahi(): Renamed from mtr_defer_drop_ahi().

recv_sys_t::free_corrupted_page(): Only set_corrupt_fs()
if any log records exist for the page. We do not mind if read-ahead
produces corrupted (or all-zero) pages that were not actually needed
during recovery.

recv_recover_page(): Return whether the operation succeeded.

recv_sys_t::recover_low(): Simplify the logic. Check for recovery error.

Thanks to Matthias Leich for testing this extensively and to the
authors of https://rr-project.org for making it easy to diagnose
and fix any failures that were found during the testing.

extra/mariabackup/xtrabackup.cc

@@ -584,8 +584,8 @@ void CorruptedPages::zero_out_free_pages()
              space_it->second.pages.begin();
          page_it != space_it->second.pages.end(); ++page_it)
     {
-      bool is_free= fseg_page_is_free(space, *page_it);
-      if (!is_free) {
+      if (fseg_page_is_allocated(space, *page_it))
+      {
         space_info_t &space_info = non_free_pages[space_id];
         space_info.pages.insert(*page_it);
         if (space_info.space_name.empty())

mysql-test/suite/encryption/r/innodb-bad-key-change.result

 call mtr.add_suppression("Plugin 'file_key_management' init function returned error");
 call mtr.add_suppression("Plugin 'file_key_management' registration.*failed");
 call mtr.add_suppression("InnoDB: Table `test`\\.`t[12]` (has an unreadable root page|is corrupted)");
-call mtr.add_suppression("InnoDB: The page \\[page id: space=[1-9][0-9]*, page number=[1-9][0-9]*\\] in file '.*test.t[12]\\.ibd' cannot be decrypted\\.");
-call mtr.add_suppression("failed to read or decrypt \\[page id: space=[1-9][0-9]*, page number=[1-9][0-9]*\\]");
+call mtr.add_suppression("InnoDB: The page \\[page id: space=[1-9][0-9]*, page number=[1-9][0-9]*\\] in file '.*test.t[12]\\.ibd' cannot be decrypted; key_version=1");
+call mtr.add_suppression("failed to read \\[page id: space=[1-9][0-9]*, page number=[1-9][0-9]*\\]");
 call mtr.add_suppression("InnoDB: Encrypted page \\[page id: space=[1-9][0-9]*, page number=3\\] in file .*test.t1.ibd looks corrupted; key_version=1");
 call mtr.add_suppression("InnoDB: Table `test`\\.`t[12]` is corrupted");
 call mtr.add_suppression("File '.*mysql-test.std_data.keysbad3\\.txt' not found");

mysql-test/suite/encryption/r/innodb-bad-key-change2.result

 call mtr.add_suppression("InnoDB: Table `test`\\.`t1` (has an unreadable root page|is corrupted|does not exist.*is trying to rename)");
-call mtr.add_suppression("InnoDB: The page \\[page id: space=[1-9][0-9]*, page number=[1-9][0-9]*\\] in file '.*test.t1(new)?\\.ibd' cannot be decrypted\\.");
+call mtr.add_suppression("InnoDB: The page \\[page id: space=[1-9][0-9]*, page number=[1-9][0-9]*\\] in file '.*test.t1(new)?\\.ibd' cannot be decrypted; key_version=1");
 call mtr.add_suppression("failed to read or decrypt \\[page id: space=[1-9][0-9]*, page number=[1-9][0-9]*\\]");
 call mtr.add_suppression("Couldn't load plugins from 'file_key_management");
 call mtr.add_suppression("InnoDB: Tablespace for table \`test\`.\`t1\` is set as discarded\\.");

mysql-test/suite/encryption/r/innodb-bad-key-change4.result

 call mtr.add_suppression("InnoDB: Table `test`\\.`t1` (has an unreadable root page|is corrupted)");
-call mtr.add_suppression("InnoDB: The page \\[page id: space=[1-9][0-9]*, page number=[1-9][0-9]*\\] in file '.*test.t1\\.ibd' cannot be decrypted\\.");
+call mtr.add_suppression("InnoDB: The page \\[page id: space=[1-9][0-9]*, page number=[1-9][0-9]*\\] in file '.*test.t1\\.ibd' cannot be decrypted; key_version=1");
 call mtr.add_suppression("failed to read or decrypt \\[page id: space=[1-9][0-9]*, page number=[1-9][0-9]*\\]");
 call mtr.add_suppression("Couldn't load plugins from 'file_key_management");
 call mtr.add_suppression("InnoDB: Table `test`\\.`t1` is corrupted");

mysql-test/suite/encryption/r/innodb-compressed-blob.result

-call mtr.add_suppression("InnoDB: The page \\[page id: space=[1-9][0-9]*, page number=[1-9][0-9]*\\] in file '.*test.t[123]\\.ibd' cannot be decrypted\\.");
+call mtr.add_suppression("InnoDB: The page \\[page id: space=[1-9][0-9]*, page number=[1-9][0-9]*\\] in file '.*test.t[123]\\.ibd' cannot be decrypted; key_version=1");
 call mtr.add_suppression("failed to read or decrypt \\[page id: space=[1-9][0-9]*, page number=[1-9][0-9]*\\]");
 call mtr.add_suppression("InnoDB: Unable to decompress ..test.t[1-3]\\.ibd\\[page id: space=[1-9][0-9]*, page number=[0-9]+\\]");
 call mtr.add_suppression("InnoDB: Table `test`\\.`t[12]` is corrupted");

mysql-test/suite/encryption/r/innodb-redo-badkey.result

 call mtr.add_suppression("Plugin 'file_key_management'");
 call mtr.add_suppression("Plugin 'InnoDB' init function returned error.");
-call mtr.add_suppression("InnoDB: The page \\[page id: space=[1-9][0-9]*, page number=[1-9][0-9]*\\] in file '.*test.t[1-4]\\.ibd' cannot be decrypted");
+call mtr.add_suppression("InnoDB: The page \\[page id: space=[1-9][0-9]*, page number=[1-9][0-9]*\\] in file '.*test.t[1-4]\\.ibd' cannot be decrypted; key_version=1");
+call mtr.add_suppression("InnoDB: File '.*test/t[1234]\\.ibd' is corrupted");
 call mtr.add_suppression("failed to read or decrypt \\[page id: space=[1-9][0-9]*, page number=[1-9][0-9]*\\]");
 call mtr.add_suppression("InnoDB: Unable to decompress .*.test.t[12]\\.ibd\\[page id: space=[1-9][0-9]*, page number=[0-9]+\\]");
 call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed read of file '.*test.t[12]\\.ibd'");

mysql-test/suite/encryption/t/innodb-bad-key-change.test

@@ -11,8 +11,8 @@
 call mtr.add_suppression("Plugin 'file_key_management' init function returned error");
 call mtr.add_suppression("Plugin 'file_key_management' registration.*failed");
 call mtr.add_suppression("InnoDB: Table `test`\\.`t[12]` (has an unreadable root page|is corrupted)");
-call mtr.add_suppression("InnoDB: The page \\[page id: space=[1-9][0-9]*, page number=[1-9][0-9]*\\] in file '.*test.t[12]\\.ibd' cannot be decrypted\\.");
-call mtr.add_suppression("failed to read or decrypt \\[page id: space=[1-9][0-9]*, page number=[1-9][0-9]*\\]");
+call mtr.add_suppression("InnoDB: The page \\[page id: space=[1-9][0-9]*, page number=[1-9][0-9]*\\] in file '.*test.t[12]\\.ibd' cannot be decrypted; key_version=1");
+call mtr.add_suppression("failed to read \\[page id: space=[1-9][0-9]*, page number=[1-9][0-9]*\\]");
 call mtr.add_suppression("InnoDB: Encrypted page \\[page id: space=[1-9][0-9]*, page number=3\\] in file .*test.t1.ibd looks corrupted; key_version=1");
 call mtr.add_suppression("InnoDB: Table `test`\\.`t[12]` is corrupted");
 call mtr.add_suppression("File '.*mysql-test.std_data.keysbad3\\.txt' not found");

mysql-test/suite/encryption/t/innodb-bad-key-change2.test

@@ -9,7 +9,7 @@
 # MDEV-8727: Server/InnoDB hangs on shutdown after trying to read an encrypted table with a wrong key
 #
 call mtr.add_suppression("InnoDB: Table `test`\\.`t1` (has an unreadable root page|is corrupted|does not exist.*is trying to rename)");
-call mtr.add_suppression("InnoDB: The page \\[page id: space=[1-9][0-9]*, page number=[1-9][0-9]*\\] in file '.*test.t1(new)?\\.ibd' cannot be decrypted\\.");
+call mtr.add_suppression("InnoDB: The page \\[page id: space=[1-9][0-9]*, page number=[1-9][0-9]*\\] in file '.*test.t1(new)?\\.ibd' cannot be decrypted; key_version=1");
 call mtr.add_suppression("failed to read or decrypt \\[page id: space=[1-9][0-9]*, page number=[1-9][0-9]*\\]");
 # Suppression for builds where file_key_management plugin is linked statically
 call mtr.add_suppression("Couldn't load plugins from 'file_key_management");

mysql-test/suite/encryption/t/innodb-bad-key-change4.test

@@ -8,7 +8,7 @@
 #
 
 call mtr.add_suppression("InnoDB: Table `test`\\.`t1` (has an unreadable root page|is corrupted)");
-call mtr.add_suppression("InnoDB: The page \\[page id: space=[1-9][0-9]*, page number=[1-9][0-9]*\\] in file '.*test.t1\\.ibd' cannot be decrypted\\.");
+call mtr.add_suppression("InnoDB: The page \\[page id: space=[1-9][0-9]*, page number=[1-9][0-9]*\\] in file '.*test.t1\\.ibd' cannot be decrypted; key_version=1");
 call mtr.add_suppression("failed to read or decrypt \\[page id: space=[1-9][0-9]*, page number=[1-9][0-9]*\\]");
 # Suppression for builds where file_key_management plugin is linked statically
 call mtr.add_suppression("Couldn't load plugins from 'file_key_management");

mysql-test/suite/encryption/t/innodb-compressed-blob.test

@@ -4,7 +4,7 @@
 # embedded does not support restart
 -- source include/not_embedded.inc
 
-call mtr.add_suppression("InnoDB: The page \\[page id: space=[1-9][0-9]*, page number=[1-9][0-9]*\\] in file '.*test.t[123]\\.ibd' cannot be decrypted\\.");
+call mtr.add_suppression("InnoDB: The page \\[page id: space=[1-9][0-9]*, page number=[1-9][0-9]*\\] in file '.*test.t[123]\\.ibd' cannot be decrypted; key_version=1");
 call mtr.add_suppression("failed to read or decrypt \\[page id: space=[1-9][0-9]*, page number=[1-9][0-9]*\\]");
 call mtr.add_suppression("InnoDB: Unable to decompress ..test.t[1-3]\\.ibd\\[page id: space=[1-9][0-9]*, page number=[0-9]+\\]");
 call mtr.add_suppression("InnoDB: Table `test`\\.`t[12]` is corrupted");

mysql-test/suite/encryption/t/innodb-redo-badkey.test

@@ -9,7 +9,8 @@
 
 call mtr.add_suppression("Plugin 'file_key_management'");
 call mtr.add_suppression("Plugin 'InnoDB' init function returned error.");
-call mtr.add_suppression("InnoDB: The page \\[page id: space=[1-9][0-9]*, page number=[1-9][0-9]*\\] in file '.*test.t[1-4]\\.ibd' cannot be decrypted");
+call mtr.add_suppression("InnoDB: The page \\[page id: space=[1-9][0-9]*, page number=[1-9][0-9]*\\] in file '.*test.t[1-4]\\.ibd' cannot be decrypted; key_version=1");
+call mtr.add_suppression("InnoDB: File '.*test/t[1234]\\.ibd' is corrupted");
 call mtr.add_suppression("failed to read or decrypt \\[page id: space=[1-9][0-9]*, page number=[1-9][0-9]*\\]");
 call mtr.add_suppression("InnoDB: Unable to decompress .*.test.t[12]\\.ibd\\[page id: space=[1-9][0-9]*, page number=[0-9]+\\]");
 call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed read of file '.*test.t[12]\\.ibd'");

mysql-test/suite/innodb/disabled.def

-##############################################################################
-#
-#  List the test cases that are to be disabled temporarily.
-#
-#  Separate the test case name and the comment with ':'.
-#
-#    <testcasename> : BUG#<xxxx> <date disabled> <disabler> <comment>
-#
-#  Do not use any TAB characters for whitespace.
-#
-##############################################################################
-
-create-index-debug : MDEV-13680 InnoDB may crash when btr_page_alloc() fails

mysql-test/suite/innodb/r/create-index-debug.result

-SET @saved_debug_dbug = @@SESSION.debug_dbug;
-#
-#BUG#21326304 INNODB ONLINE ALTER TABLE ENDS IN CRASH ON DISK FULL
-#
-CREATE TABLE t1(f1 CHAR(255) NOT NULL, f2 CHAR(255) NOT NULL, f3
-CHAR(255) NOT NULL, f4 CHAR(255) NOT NULL, f5 CHAR(255) NOT NULL,f6
-CHAR(255) NOT NULL, f7 CHAR(255) NOT NULL, f8 CHAR(255) NOT NULL,f9
-CHAR(255) NOT NULL, f10 CHAR(255) NOT NULL, f11 CHAR(255) NOT NULL,f12
-CHAR(255) NOT NULL, f13 CHAR(255) NOT NULL, f14 CHAR(255) NOT NULL,f15
-CHAR(255) NOT NULL, f16 CHAR(255) NOT NULL, f17 CHAR(255) NOT NULL,f18
-CHAR(255) NOT NULL)
-ENGINE=INNODB ROW_FORMAT=DYNAMIC;
-INSERT INTO t1
-VALUES('a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r');
-INSERT INTO t1 SELECT * FROM t1;
-INSERT INTO t1 SELECT * FROM t1;
-INSERT INTO t1 SELECT * FROM t1;
-INSERT INTO t1 SELECT * FROM t1;
-SET debug_dbug = '+d,disk_is_full';
-ALTER TABLE t1 FORCE, ALGORITHM=INPLACE;
-ERROR HY000: The table 't1' is full
-SET debug_dbug= @saved_debug_dbug;
-DROP TABLE t1;

mysql-test/suite/innodb/r/import_corrupted.result

 call mtr.add_suppression("Table `test`.`t2` should have 2 indexes but the tablespace has 1 indexes");
 call mtr.add_suppression("Index for table 't2' is corrupt; try to repair it");
 call mtr.add_suppression("Trying to read .* bytes at .* outside the bounds of the file: \\..test.t2\\.ibd");
+call mtr.add_suppression("InnoDB: File '.*test/t2\\.ibd' is corrupted");
 CREATE TABLE t1 (
 id INT AUTO_INCREMENT PRIMARY KEY,
 not_id INT,

mysql-test/suite/innodb/r/innodb-wl5522-debug.result

@@ -9,6 +9,8 @@ call mtr.add_suppression("InnoDB: Page for tablespace ");
 call mtr.add_suppression("InnoDB: Invalid FSP_SPACE_FLAGS=");
 call mtr.add_suppression("InnoDB: Unknown index id .* on page");
 call mtr.add_suppression("InnoDB: Cannot save statistics for table `test`\\.`t1` because the \\.ibd file is missing");
+call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed read of file '.*ibdata1' page");
+call mtr.add_suppression("InnoDB: File '.*ibdata1' is corrupted");
 FLUSH TABLES;
 SET GLOBAL innodb_file_per_table = 1;
 CREATE TABLE t1 (c1 INT) ENGINE = InnoDB;
@@ -862,10 +864,8 @@ ALTER TABLE t1 DISCARD TABLESPACE;
 SELECT COUNT(*) FROM t1;
 ERROR HY000: Tablespace has been discarded for table `t1`
 restore: t1 .ibd and .cfg files
-SET SESSION debug_dbug="+d,buf_page_import_corrupt_failure";
 ALTER TABLE t1 IMPORT TABLESPACE;
-ERROR HY000: Internal error: Cannot reset LSNs in table `test`.`t1` : Data structure corruption
-SET SESSION debug_dbug=@saved_debug_dbug;
+ERROR HY000: Index for table 't1' is corrupt; try to repair it
 DROP TABLE t1;
 unlink: t1.ibd
 unlink: t1.cfg

mysql-test/suite/innodb/r/page_id_innochecksum.result

@@ -3,4 +3,7 @@ create table t1(f1 int not null)engine=innodb;
 insert into t1 values(1), (2), (3);
 # Change the page offset
 FOUND 1 /page id mismatch/ in result.log
+SET GLOBAL innodb_purge_rseg_truncate_frequency=1;
+InnoDB		0 transactions not purged
 drop table t1;
+call mtr.add_suppression("InnoDB: Failed to read page 3 from file '.*test/t1\\.ibd': Page read from tablespace is corrupted\\.");

mysql-test/suite/innodb/t/corrupted_during_recovery.test

@@ -6,8 +6,8 @@ call mtr.add_suppression("Plugin 'InnoDB' init function returned error");
 call mtr.add_suppression("Plugin 'InnoDB' registration as a STORAGE ENGINE failed");
 call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed read of file '.*test.t1\\.ibd' page");
 call mtr.add_suppression("InnoDB: Failed to read page 3 from file '.*test.t1\\.ibd': Page read from tablespace is corrupted.");
-call mtr.add_suppression("InnoDB: Background Page read failed to read or decrypt \\[page id: space=\\d+, page number=3\\]");
 call mtr.add_suppression("InnoDB: Table `test`.`t1` is corrupted. Please drop the table and recreate.");
+call mtr.add_suppression("InnoDB: File '.*test/t1\\.ibd' is corrupted");
 --enable_query_log
 
 let INNODB_PAGE_SIZE=`select @@innodb_page_size`;

mysql-test/suite/innodb/t/create-index-debug.test

---source include/have_innodb.inc
---source include/have_innodb_16k.inc
---source include/have_debug.inc
-
-SET @saved_debug_dbug = @@SESSION.debug_dbug;
-
---echo #
---echo #BUG#21326304 INNODB ONLINE ALTER TABLE ENDS IN CRASH ON DISK FULL
---echo #
-CREATE TABLE t1(f1 CHAR(255) NOT NULL, f2 CHAR(255) NOT NULL, f3
-CHAR(255) NOT NULL, f4 CHAR(255) NOT NULL, f5 CHAR(255) NOT NULL,f6
-CHAR(255) NOT NULL, f7 CHAR(255) NOT NULL, f8 CHAR(255) NOT NULL,f9
-CHAR(255) NOT NULL, f10 CHAR(255) NOT NULL, f11 CHAR(255) NOT NULL,f12
-CHAR(255) NOT NULL, f13 CHAR(255) NOT NULL, f14 CHAR(255) NOT NULL,f15
-CHAR(255) NOT NULL, f16 CHAR(255) NOT NULL, f17 CHAR(255) NOT NULL,f18
-CHAR(255) NOT NULL)
-ENGINE=INNODB ROW_FORMAT=DYNAMIC;
-
-INSERT INTO t1
-VALUES('a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r');
-
-INSERT INTO t1 SELECT * FROM t1;
-INSERT INTO t1 SELECT * FROM t1;
-INSERT INTO t1 SELECT * FROM t1;
-INSERT INTO t1 SELECT * FROM t1;
-
-SET debug_dbug = '+d,disk_is_full';
-
---error ER_RECORD_FILE_FULL
-ALTER TABLE t1 FORCE, ALGORITHM=INPLACE;
-
-SET debug_dbug= @saved_debug_dbug;
-
-DROP TABLE t1;

mysql-test/suite/innodb/t/import_corrupted.test

@@ -3,6 +3,7 @@
 call mtr.add_suppression("Table `test`.`t2` should have 2 indexes but the tablespace has 1 indexes");
 call mtr.add_suppression("Index for table 't2' is corrupt; try to repair it");
 call mtr.add_suppression("Trying to read .* bytes at .* outside the bounds of the file: \\..test.t2\\.ibd");
+call mtr.add_suppression("InnoDB: File '.*test/t2\\.ibd' is corrupted");
 
 let MYSQLD_DATADIR = `SELECT @@datadir`;
 

mysql-test/suite/innodb/t/innodb-wl5522-debug.test

@@ -29,6 +29,8 @@ call mtr.add_suppression("InnoDB: Page for tablespace ");
 call mtr.add_suppression("InnoDB: Invalid FSP_SPACE_FLAGS=");
 call mtr.add_suppression("InnoDB: Unknown index id .* on page");
 call mtr.add_suppression("InnoDB: Cannot save statistics for table `test`\\.`t1` because the \\.ibd file is missing");
+call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed read of file '.*ibdata1' page");
+call mtr.add_suppression("InnoDB: File '.*ibdata1' is corrupted");
 FLUSH TABLES;
 
 let MYSQLD_DATADIR =`SELECT @@datadir`;
@@ -1173,20 +1175,29 @@ ALTER TABLE t1 DISCARD TABLESPACE;
 SELECT COUNT(*) FROM t1;
 
 # Restore files
+let INNODB_PAGE_SIZE=`select @@innodb_page_size`;
+
 perl;
 do "$ENV{MTR_SUITE_DIR}/include/innodb-util.pl";
 ib_restore_tablespaces("test", "t1");
+my $page;
+my $ps = $ENV{INNODB_PAGE_SIZE};
+my $file = "$ENV{MYSQLD_DATADIR}/test/t1.ibd";
+open(FILE, "+<$file") || die "Unable to open $file";
+binmode FILE;
+sysseek(FILE, 3*$ps, 0) || die "Unable to seek $file\n";
+die "Unable to read $file" unless sysread(FILE, $page, $ps) == $ps;
+substr($page,24,2)='42';
+sysseek(FILE, 3*$ps, 0) || die "Unable to rewind $file\n";
+syswrite(FILE, $page, $ps)==$ps || die "Unable to write $file\n";
+close(FILE) || die "Unable to close $file";
 EOF
 
-SET SESSION debug_dbug="+d,buf_page_import_corrupt_failure";
-
 --replace_regex /'.*t1.cfg'/'t1.cfg'/
 
---error ER_INTERNAL_ERROR
+--error ER_NOT_KEYFILE
 ALTER TABLE t1 IMPORT TABLESPACE;
 
-SET SESSION debug_dbug=@saved_debug_dbug;
-
 DROP TABLE t1;
 
 perl;

mysql-test/suite/innodb/t/innodb_bug14147491.test

@@ -12,6 +12,7 @@ call mtr.add_suppression("InnoDB: Table `test`\\.`t1` is corrupted\\. Please dro
 call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed read of file '.*test.t1\\.ibd' page");
 call mtr.add_suppression("InnoDB: We detected index corruption in an InnoDB type table");
 call mtr.add_suppression("Index for table 't1' is corrupt; try to repair it");
+call mtr.add_suppression("InnoDB: File '.*test/t1\\.ibd' is corrupted");
 --enable_query_log
 
 --echo # Ensure that purge will not crash on the table after we corrupt it.
@@ -70,13 +71,13 @@ EOF
 
 --echo # Now t1 is corrupted but we should not crash
 
---error 1030,1712,1932
+--error ER_GET_ERRNO,ER_NOT_KEYFILE,ER_INDEX_CORRUPT,ER_NO_SUCH_TABLE_IN_ENGINE
 SELECT * FROM t1;
 
---error 126,1030,1034,1712,1932
+--error 126,ER_GET_ERRNO,ER_NOT_KEYFILE,ER_INDEX_CORRUPT,ER_NO_SUCH_TABLE_IN_ENGINE
 INSERT INTO t1(b) VALUES('abcdef');
 
---error 1030,1712,1932
+--error ER_GET_ERRNO,ER_NOT_KEYFILE,ER_INDEX_CORRUPT,ER_NO_SUCH_TABLE_IN_ENGINE
 UPDATE t1 set b = 'deadbeef' where a = 1;
 
 --echo # Cleanup, this must be possible

mysql-test/suite/innodb/t/leaf_page_corrupted_during_recovery.test

@@ -3,13 +3,13 @@
 
 --disable_query_log
 call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed read of file '.*test.t1\\.ibd' page");
-call mtr.add_suppression("InnoDB: Background Page read failed to read or decrypt \\[page id: space=\\d+, page number=19\\]");
 call mtr.add_suppression("\\[ERROR\\] InnoDB: Failed to read page 19 from file '.*test.t1\\.ibd': Page read from tablespace is corrupted\\.");
 call mtr.add_suppression("\\[ERROR\\] InnoDB: Plugin initialization aborted at srv0start\\.cc.* with error Data structure corruption");
 call mtr.add_suppression("\\[ERROR\\] Plugin 'InnoDB' (init function|registration)");
 call mtr.add_suppression("\\[ERROR\\] InnoDB: We detected index corruption");
 call mtr.add_suppression("\\[ERROR\\] (mysqld|mariadbd).*: Index for table 't1' is corrupt; try to repair it");
 call mtr.add_suppression("InnoDB: btr_pcur_open_low level: 0 table: `test`\\.`t1` index: `PRIMARY`");
+call mtr.add_suppression("InnoDB: File '.*test/t1\\.ibd' is corrupted");
 --enable_query_log
 CREATE TABLE t1 (pk INT PRIMARY KEY, c CHAR(255))ENGINE=InnoDB STATS_PERSISTENT=0;
 

mysql-test/suite/innodb/t/page_id_innochecksum.test

@@ -61,9 +61,11 @@ let SEARCH_PATTERN=page id mismatch;
 --source include/search_pattern_in_file.inc
 
 --remove_file $resultlog
-# prevent purge from crashing on page ID mismatch
-let $restart_parameters=--innodb-force-recovery=2;
+let $restart_parameters=--innodb-force-recovery=1;
 --source include/start_mysqld.inc
+SET GLOBAL innodb_purge_rseg_truncate_frequency=1;
+--source include/wait_all_purged.inc
 drop table t1;
+call mtr.add_suppression("InnoDB: Failed to read page 3 from file '.*test/t1\\.ibd': Page read from tablespace is corrupted\\.");
 let $restart_parameters=;
 --source include/restart_mysqld.inc

mysql-test/suite/innodb_zip/r/wl5522_debug_zip.result

@@ -9,6 +9,8 @@ call mtr.add_suppression("InnoDB: The error means");
 call mtr.add_suppression("InnoDB: Cannot open datafile .*t1\\.ibd");
 call mtr.add_suppression("InnoDB: Ignoring tablespace for test/t1 ");
 call mtr.add_suppression("InnoDB: Cannot save statistics for table `test`\\.`t1` because the \\.ibd file is missing");
+call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed read of file '.*ibdata1' page");
+call mtr.add_suppression("InnoDB: File '.*ibdata1' is corrupted");
 FLUSH TABLES;
 SET SESSION innodb_strict_mode=1;
 CREATE TABLE t1 (c1 INT) ENGINE = Innodb
@@ -325,10 +327,8 @@ ALTER TABLE t1 DISCARD TABLESPACE;
 SELECT COUNT(*) FROM t1;
 ERROR HY000: Tablespace has been discarded for table `t1`
 restore: t1 .ibd and .cfg files
-SET SESSION debug_dbug="+d,buf_page_import_corrupt_failure";
 ALTER TABLE t1 IMPORT TABLESPACE;
-ERROR HY000: Internal error: Cannot reset LSNs in table `test`.`t1` : Data structure corruption
-SET SESSION debug_dbug=@saved_debug_dbug;
+Got one of the listed errors
 DROP TABLE t1;
 unlink: t1.ibd
 unlink: t1.cfg

mysql-test/suite/innodb_zip/t/wl5522_debug_zip.test

@@ -25,6 +25,8 @@ call mtr.add_suppression("InnoDB: The error means");
 call mtr.add_suppression("InnoDB: Cannot open datafile .*t1\\.ibd");
 call mtr.add_suppression("InnoDB: Ignoring tablespace for test/t1 ");
 call mtr.add_suppression("InnoDB: Cannot save statistics for table `test`\\.`t1` because the \\.ibd file is missing");
+call mtr.add_suppression("InnoDB: Database page corruption on disk or a failed read of file '.*ibdata1' page");
+call mtr.add_suppression("InnoDB: File '.*ibdata1' is corrupted");
 FLUSH TABLES;
 
 let MYSQLD_DATADIR =`SELECT @@datadir`;
@@ -435,19 +437,29 @@ ALTER TABLE t1 DISCARD TABLESPACE;
 SELECT COUNT(*) FROM t1;
 
 # Restore files
+let INNODB_PAGE_SIZE=`select @@innodb_page_size`;
+let MYSQLD_DATADIR=`select @@datadir`;
+
 perl;
 do "$ENV{MTR_SUITE_DIR}/../innodb/include/innodb-util.pl";
 ib_restore_tablespaces("test", "t1");
+my $page;
+my $ps = $ENV{INNODB_PAGE_SIZE};
+my $file = "$ENV{MYSQLD_DATADIR}/test/t1.ibd";
+open(FILE, "+<$file") || die "Unable to open $file";
+binmode FILE;
+sysseek(FILE, 4*$ps, 0) || die "Unable to seek $file\n";
+die "Unable to read $file" unless sysread(FILE, $page, $ps) == $ps;
+substr($page,24,2)='42';
+sysseek(FILE, 4*$ps, 0) || die "Unable to rewind $file\n";
+syswrite(FILE, $page, $ps)==$ps || die "Unable to write $file\n";
+close(FILE) || die "Unable to close $file";
 EOF
 
-SET SESSION debug_dbug="+d,buf_page_import_corrupt_failure";
-
 # Following alter is failing
---error ER_INTERNAL_ERROR
+--error ER_INTERNAL_ERROR,ER_NOT_KEYFILE
 ALTER TABLE t1 IMPORT TABLESPACE;
 
-SET SESSION debug_dbug=@saved_debug_dbug;
-
 DROP TABLE t1;
 
 perl;

storage/innobase/CMakeLists.txt

@@ -148,7 +148,6 @@ SET(INNOBASE_SOURCES
 	include/fts0types.h
 	include/fts0types.inl
 	include/fts0vlc.h
-	include/fut0fut.h
 	include/fut0lst.h
 	include/gis0geo.h
 	include/gis0rtree.h
@@ -241,7 +240,6 @@ SET(INNOBASE_SOURCES
 	include/trx0rec.inl
 	include/trx0roll.h
 	include/trx0rseg.h
-	include/trx0rseg.inl
 	include/trx0sys.h
 	include/trx0trx.h
 	include/trx0trx.inl

storage/innobase/btr/btr0bulk.cc

 /*****************************************************************************
 
 Copyright (c) 2014, 2019, Oracle and/or its affiliates. All Rights Reserved.
-Copyright (c) 2017, 2021, MariaDB Corporation.
+Copyright (c) 2017, 2022, MariaDB Corporation.
 
 This program is free software; you can redistribute it and/or modify it under
 the terms of the GNU General Public License as published by the Free Software
@@ -61,17 +61,22 @@ PageBulk::init()
 		m_index->set_modified(alloc_mtr);
 
 		uint32_t n_reserved;
-		if (!fsp_reserve_free_extents(&n_reserved,
-					      m_index->table->space,
-					      1, FSP_NORMAL, &alloc_mtr)) {
+		dberr_t err = fsp_reserve_free_extents(
+			&n_reserved, m_index->table->space, 1, FSP_NORMAL,
+			&alloc_mtr);
+		if (UNIV_UNLIKELY(err != DB_SUCCESS)) {
+oom:
 			alloc_mtr.commit();
 			m_mtr.commit();
-			return(DB_OUT_OF_FILE_SPACE);
+			return err;
 		}
 
 		/* Allocate a new page. */
 		new_block = btr_page_alloc(m_index, 0, FSP_UP, m_level,
-					   &alloc_mtr, &m_mtr);
+					   &alloc_mtr, &m_mtr, &err);
+		if (!new_block) {
+			goto oom;
+		}
 
 		m_index->table->space->release_free_extents(n_reserved);
 
@@ -103,9 +108,12 @@ PageBulk::init()
 	} else {
 		new_block = btr_block_get(*m_index, m_page_no, RW_X_LATCH,
 					  false, &m_mtr);
+		if (!new_block) {
+			m_mtr.commit();
+			return(DB_CORRUPTION);
+		}
 
 		new_page = buf_block_get_frame(new_block);
-		ut_ad(new_block->page.id().page_no() == m_page_no);
 
 		ut_ad(page_dir_get_n_heap(new_page) == PAGE_HEAP_NO_USER_LOW);
 
@@ -840,38 +848,19 @@ PageBulk::release()
 }
 
 /** Start mtr and latch the block */
-dberr_t
-PageBulk::latch()
+void PageBulk::latch()
 {
-	m_mtr.start();
-	m_index->set_modified(m_mtr);
-
-	ut_ad(m_block->page.buf_fix_count());
-
-	/* In case the block is U-latched by page_cleaner. */
-	if (!buf_page_optimistic_get(RW_X_LATCH, m_block, m_modify_clock,
-				     &m_mtr)) {
-		/* FIXME: avoid another lookup */
-		m_block = buf_page_get_gen(page_id_t(m_index->table->space_id,
-						     m_page_no),
-					   0, RW_X_LATCH,
-					   m_block, BUF_GET_IF_IN_POOL,
-					   &m_mtr, &m_err);
-
-		if (m_err != DB_SUCCESS) {
-			return (m_err);
-		}
-
-		ut_ad(m_block != NULL);
-	}
-
-	ut_d(const auto buf_fix_count =) m_block->page.unfix();
-
-	ut_ad(buf_fix_count);
-	ut_ad(m_cur_rec > m_page);
-	ut_ad(m_cur_rec < m_heap_top);
-
-	return (m_err);
+  m_mtr.start();
+  m_index->set_modified(m_mtr);
+#ifdef BTR_CUR_HASH_ADAPT
+  ut_ad(!m_block->index);
+#endif
+  m_block->page.lock.x_lock();
+  ut_ad(m_block->page.buf_fix_count());
+  m_mtr.memo_push(m_block, MTR_MEMO_PAGE_X_FIX);
+
+  ut_ad(m_cur_rec > m_page);
+  ut_ad(m_cur_rec < m_heap_top);
 }
 
 /** Split a page
@@ -1199,6 +1188,13 @@ BtrBulk::finish(dberr_t	err)
 		ut_ad(last_page_no != FIL_NULL);
 		last_block = btr_block_get(*m_index, last_page_no, RW_X_LATCH,
 					   false, &mtr);
+		if (!last_block) {
+			err = DB_CORRUPTION;
+err_exit:
+			mtr.commit();
+			return err;
+		}
+
 		first_rec = page_rec_get_next(
 			page_get_infimum_rec(last_block->page.frame));
 		ut_ad(page_rec_is_user_rec(first_rec));
@@ -1206,18 +1202,18 @@ BtrBulk::finish(dberr_t	err)
 		/* Copy last page to root page. */
 		err = root_page_bulk.init();
 		if (err != DB_SUCCESS) {
-			mtr.commit();
-			return(err);
+			goto err_exit;
 		}
 		root_page_bulk.copyIn(first_rec);
 		root_page_bulk.finish();
 
 		/* Remove last page. */
-		btr_page_free(m_index, last_block, &mtr);
-
+		err = btr_page_free(m_index, last_block, &mtr);
 		mtr.commit();
 
-		err = pageCommit(&root_page_bulk, NULL, false);
+		if (dberr_t e = pageCommit(&root_page_bulk, NULL, false)) {
+			err = e;
+		}
 		ut_ad(err == DB_SUCCESS);
 	}
 

storage/innobase/btr/btr0defragment.cc

 /*****************************************************************************
 
 Copyright (C) 2012, 2014 Facebook, Inc. All Rights Reserved.
-Copyright (C) 2014, 2021, MariaDB Corporation.
+Copyright (C) 2014, 2022, MariaDB Corporation.
 
 This program is free software; you can redistribute it and/or modify it under
 the terms of the GNU General Public License as published by the Free Software
@@ -283,7 +283,8 @@ btr_defragment_calc_n_recs_for_size(
 /*************************************************************//**
 Merge as many records from the from_block to the to_block. Delete
 the from_block if all records are successfully merged to to_block.
-@return the to_block to target for next merge operation. */
+@return the to_block to target for next merge operation.
+@retval nullptr if corruption was noticed */
 static
 buf_block_t*
 btr_defragment_merge_pages(
@@ -330,9 +331,9 @@ btr_defragment_merge_pages(
 	// reorganizing the page, otherwise we need to reorganize the page
 	// first to release more space.
 	if (move_size > max_ins_size) {
-		if (!btr_page_reorganize_block(page_zip_level,
-					       to_block, index,
-					       mtr)) {
+		dberr_t err = btr_page_reorganize_block(page_zip_level,
+                                                        to_block, index, mtr);
+		if (err != DB_SUCCESS) {
 			if (!dict_index_is_clust(index)
 			    && page_is_leaf(to_page)) {
 				ibuf_reset_free_bits(to_block);
@@ -341,23 +342,30 @@ btr_defragment_merge_pages(
 			// not compressable. There's no point to try
 			// merging into this page. Continue to the
 			// next page.
-			return from_block;
+			return err == DB_FAIL ? from_block : nullptr;
 		}
 		ut_ad(page_validate(to_page, index));
 		max_ins_size = page_get_max_insert_size(to_page, n_recs);
-		ut_a(max_ins_size >= move_size);
+		if (max_ins_size < move_size) {
+			return nullptr;
+		}
 	}
 
 	// Move records to pack to_page more full.
 	orig_pred = NULL;
 	target_n_recs = n_recs_to_move;
+	dberr_t err;
 	while (n_recs_to_move > 0) {
 		rec = page_rec_get_nth(from_page,
 					n_recs_to_move + 1);
 		orig_pred = page_copy_rec_list_start(
-			to_block, from_block, rec, index, mtr);
+			to_block, from_block, rec, index, mtr, &err);
 		if (orig_pred)
 			break;
+		if (err != DB_FAIL) {
+			return nullptr;
+		}
+
 		// If we reach here, that means compression failed after packing
 		// n_recs_to_move number of records to to_page. We try to reduce
 		// the targeted data size on the to_page by
@@ -396,19 +404,20 @@ btr_defragment_merge_pages(
 		}
 	}
 	btr_cur_t parent;
-	if (n_recs_to_move == n_recs) {
+	if (!btr_page_get_father(index, from_block, mtr, &parent)) {
+		to_block = nullptr;
+	} else if (n_recs_to_move == n_recs) {
 		/* The whole page is merged with the previous page,
 		free it. */
-		const page_id_t from{from_block->page.id()};
-		lock_update_merge_left(*to_block, orig_pred, from);
+		lock_update_merge_left(*to_block, orig_pred,
+				       from_block->page.id());
 		btr_search_drop_page_hash_index(from_block);
-		ut_a(DB_SUCCESS == btr_level_list_remove(*from_block, *index,
-							 mtr));
-		btr_page_get_father(index, from_block, mtr, &parent);
-		btr_cur_node_ptr_delete(&parent, mtr);
-		/* btr_blob_dbg_remove(from_page, index,
-		"btr_defragment_n_pages"); */
-		btr_page_free(index, from_block, mtr);
+		if (btr_level_list_remove(*from_block, *index, mtr)
+		    != DB_SUCCESS
+		    || btr_cur_node_ptr_delete(&parent, mtr) != DB_SUCCESS
+		    || btr_page_free(index, from_block, mtr) != DB_SUCCESS) {
+			return nullptr;
+		}
 	} else {
 		// There are still records left on the page, so
 		// increment n_defragmented. Node pointer will be changed
@@ -424,15 +433,20 @@ btr_defragment_merge_pages(
 						    orig_pred,
 						    from_block);
 			// FIXME: reuse the node_ptr!
-			btr_page_get_father(index, from_block, mtr, &parent);
-			btr_cur_node_ptr_delete(&parent, mtr);
+			if (btr_cur_node_ptr_delete(&parent, mtr)
+			    != DB_SUCCESS) {
+				return nullptr;
+			}
 			rec = page_rec_get_next(
 				page_get_infimum_rec(from_page));
 			node_ptr = dict_index_build_node_ptr(
 				index, rec, page_get_page_no(from_page),
 				heap, level);
-			btr_insert_on_non_leaf_level(0, index, level+1,
-						     node_ptr, mtr);
+			if (btr_insert_on_non_leaf_level(0, index, level+1,
+                                                         node_ptr, mtr)
+                            != DB_SUCCESS) {
+				return nullptr;
+                        }
 		}
 		to_block = from_block;
 	}
@@ -507,6 +521,9 @@ btr_defragment_n_pages(
 
 		blocks[i] = btr_block_get(*index, page_no, RW_X_LATCH, true,
 					  mtr);
+		if (!blocks[i]) {
+			return nullptr;
+		}
 	}
 
 	if (n_pages == 1) {
@@ -517,7 +534,8 @@ btr_defragment_n_pages(
 				return NULL;
 			/* given page is the last page.
 			Lift the records to father. */
-			btr_lift_page_up(index, block, mtr);
+			dberr_t err;
+			btr_lift_page_up(index, block, mtr, &err);
 		}
 		return NULL;
 	}
@@ -580,6 +598,9 @@ btr_defragment_n_pages(
 		if (new_block != current_block) {
 			n_defragmented ++;
 			current_block = new_block;
+			if (!new_block) {
+				break;
+			}
 		}
 	}
 	mem_heap_free(heap);
@@ -667,17 +688,19 @@ static void btr_defragment_chunk(void*)
 		mtr_start(&mtr);
 		dict_index_t *index = item->pcur->btr_cur.index;
 		index->set_modified(mtr);
-		/* To follow the latching order defined in WL#6326, acquire index->lock X-latch.
-		This entitles us to acquire page latches in any order for the index. */
+		/* To follow the latching order defined in WL#6326,
+		acquire index->lock X-latch.  This entitles us to
+		acquire page latches in any order for the index. */
 		mtr_x_lock_index(index, &mtr);
-		/* This will acquire index->lock SX-latch, which per WL#6363 is allowed
+		/* This will acquire index->lock U latch, which is allowed
 		when we are already holding the X-latch. */
-		item->pcur->restore_position(BTR_MODIFY_TREE, &mtr);
-		buf_block_t* first_block = btr_pcur_get_block(item->pcur);
 		if (buf_block_t *last_block =
-		    btr_defragment_n_pages(first_block, index,
-					   srv_defragment_n_pages,
-					   &mtr)) {
+		    item->pcur->restore_position(BTR_MODIFY_TREE, &mtr)
+		    == btr_pcur_t::CORRUPTED
+		    ? nullptr
+		    : btr_defragment_n_pages(btr_pcur_get_block(item->pcur),
+					     index, srv_defragment_n_pages,
+					     &mtr)) {
 			/* If we haven't reached the end of the index,
 			place the cursor on the last record of last page,
 			store the cursor position, and put back in queue. */

storage/innobase/btr/btr0pcur.cc

@@ -299,21 +299,14 @@ btr_pcur_t::restore_position(ulint restore_latch_mode, mtr_t *mtr)
 	if (UNIV_UNLIKELY
 	    (rel_pos == BTR_PCUR_AFTER_LAST_IN_TREE
 	     || rel_pos == BTR_PCUR_BEFORE_FIRST_IN_TREE)) {
-		dberr_t err = DB_SUCCESS;
-
 		/* In these cases we do not try an optimistic restoration,
 		but always do a search */
 
-		err = btr_cur_open_at_index_side(
+		if (btr_cur_open_at_index_side(
 			rel_pos == BTR_PCUR_BEFORE_FIRST_IN_TREE,
 			index, restore_latch_mode,
-			&btr_cur, 0, mtr);
-
-		if (err != DB_SUCCESS) {
-			ib::warn() << " Error code: " << err
-				   << " btr_pcur_t::restore_position "
-				   << " table: " << index->table->name
-				   << " index: " << index->name;
+			&btr_cur, 0, mtr) != DB_SUCCESS) {
+			return restore_status::CORRUPTED;
 		}
 
 		latch_mode =
@@ -412,16 +405,20 @@ btr_pcur_t::restore_position(ulint restore_latch_mode, mtr_t *mtr)
 		mode = PAGE_CUR_L;
 		break;
 	default:
-		ut_error;
+		MY_ASSERT_UNREACHABLE();
 		mode = PAGE_CUR_UNSUPP;
 	}
 
-	btr_pcur_open_with_no_init_func(index, tuple, mode, restore_latch_mode,
-					this,
+	if (btr_pcur_open_with_no_init_func(
+		index, tuple, mode, restore_latch_mode,
+		this,
 #ifdef BTR_CUR_HASH_ADAPT
-					NULL,
+		nullptr,
 #endif /* BTR_CUR_HASH_ADAPT */
-					mtr);
+		mtr) != DB_SUCCESS) {
+		mem_heap_free(heap);
+		return restore_status::CORRUPTED;
+        }
 
 	/* Restore the old search mode */
 	search_mode = old_mode;
@@ -473,7 +470,7 @@ Moves the persistent cursor to the first record on the next page. Releases the
 latch on the current page, and bufferunfixes it. Note that there must not be
 modifications on the current page, as then the x-latch can be released only in
 mtr_commit. */
-void
+dberr_t
 btr_pcur_move_to_next_page(
 /*=======================*/
 	btr_pcur_t*	cursor,	/*!< in: persistent cursor; must be on the
@@ -487,11 +484,6 @@ btr_pcur_move_to_next_page(
 	cursor->old_stored = false;
 
 	const page_t* page = btr_pcur_get_page(cursor);
-
-	if (UNIV_UNLIKELY(!page)) {
-		return;
-	}
-
 	const uint32_t next_page_no = btr_page_get_next(page);
 
 	ut_ad(next_page_no != FIL_NULL);
@@ -505,28 +497,31 @@ btr_pcur_move_to_next_page(
 		mode = BTR_MODIFY_LEAF;
 	}
 
+	dberr_t err;
 	buf_block_t* next_block = btr_block_get(
 		*btr_pcur_get_btr_cur(cursor)->index, next_page_no, mode,
-		page_is_leaf(page), mtr);
+		page_is_leaf(page), mtr, &err);
 
 	if (UNIV_UNLIKELY(!next_block)) {
-		return;
+		return err;
 	}
 
 	const page_t* next_page = buf_block_get_frame(next_block);
-#ifdef UNIV_BTR_DEBUG
-	ut_a(page_is_comp(next_page) == page_is_comp(page));
-	ut_a(btr_page_get_prev(next_page)
-	     == btr_pcur_get_block(cursor)->page.id().page_no());
-#endif /* UNIV_BTR_DEBUG */
+
+	if (UNIV_UNLIKELY(memcmp_aligned<4>(next_page + FIL_PAGE_PREV,
+					    page + FIL_PAGE_OFFSET, 4))) {
+		return DB_CORRUPTION;
+	}
 
 	btr_leaf_page_release(btr_pcur_get_block(cursor), mode, mtr);
 
 	page_cur_set_before_first(next_block, btr_pcur_get_page_cur(cursor));
 
 	ut_d(page_check_dir(next_page));
+	return err;
 }
 
+MY_ATTRIBUTE((nonnull,warn_unused_result))
 /*********************************************************//**
 Moves the persistent cursor backward if it is on the first record of the page.
 Commits mtr. Note that to prevent a possible deadlock, the operation
@@ -537,17 +532,13 @@ return, but it may happen that the cursor is not positioned on the last
 record of any page, because the structure of the tree may have changed
 during the time when the cursor had no latches. */
 static
-void
+bool
 btr_pcur_move_backward_from_page(
 /*=============================*/
 	btr_pcur_t*	cursor,	/*!< in: persistent cursor, must be on the first
 				record of the current page */
 	mtr_t*		mtr)	/*!< in: mtr */
 {
-	ulint		prev_page_no;
-	page_t*		page;
-	buf_block_t*	prev_block;
-
 	ut_ad(btr_pcur_is_before_first_on_page(cursor));
 	ut_ad(!btr_pcur_is_before_first_in_tree(cursor));
 
@@ -563,43 +554,39 @@ btr_pcur_move_backward_from_page(
 	static_assert(BTR_SEARCH_PREV == (4 | BTR_SEARCH_LEAF), "");
 	static_assert(BTR_MODIFY_PREV == (4 | BTR_MODIFY_LEAF), "");
 
-	cursor->restore_position(4 | latch_mode, mtr);
-
-	page = btr_pcur_get_page(cursor);
+	if (UNIV_UNLIKELY(cursor->restore_position(4 | latch_mode, mtr)
+			  == btr_pcur_t::CORRUPTED)) {
+		return true;
+	}
 
-	prev_page_no = btr_page_get_prev(page);
+	buf_block_t* prev_block = btr_pcur_get_btr_cur(cursor)->left_block;
 
-	if (prev_page_no == FIL_NULL) {
+	if (!page_has_prev(btr_pcur_get_page(cursor))) {
 	} else if (btr_pcur_is_before_first_on_page(cursor)) {
-
-		prev_block = btr_pcur_get_btr_cur(cursor)->left_block;
-
 		btr_leaf_page_release(btr_pcur_get_block(cursor),
 				      latch_mode, mtr);
 
 		page_cur_set_after_last(prev_block,
 					btr_pcur_get_page_cur(cursor));
 	} else {
-
 		/* The repositioned cursor did not end on an infimum
 		record on a page. Cursor repositioning acquired a latch
 		also on the previous page, but we do not need the latch:
 		release it. */
-
 		prev_block = btr_pcur_get_btr_cur(cursor)->left_block;
-
 		btr_leaf_page_release(prev_block, latch_mode, mtr);
 	}
 
 	cursor->latch_mode = latch_mode;
 	cursor->old_stored = false;
+	return false;
 }
 
 /*********************************************************//**
 Moves the persistent cursor to the previous record in the tree. If no records
 are left, the cursor stays 'before first in tree'.
 @return TRUE if the cursor was not before first in tree */
-ibool
+bool
 btr_pcur_move_to_prev(
 /*==================*/
 	btr_pcur_t*	cursor,	/*!< in: persistent cursor; NOTE that the
@@ -612,53 +599,13 @@ btr_pcur_move_to_prev(
 	cursor->old_stored = false;
 
 	if (btr_pcur_is_before_first_on_page(cursor)) {
-
-		if (btr_pcur_is_before_first_in_tree(cursor)) {
-
-			return(FALSE);
-		}
-
-		btr_pcur_move_backward_from_page(cursor, mtr);
-
-		return(TRUE);
-	}
-
-	btr_pcur_move_to_prev_on_page(cursor);
-
-	return(TRUE);
-}
-
-/**************************************************************//**
-If mode is PAGE_CUR_G or PAGE_CUR_GE, opens a persistent cursor on the first
-user record satisfying the search condition, in the case PAGE_CUR_L or
-PAGE_CUR_LE, on the last user record. If no such user record exists, then
-in the first case sets the cursor after last in tree, and in the latter case
-before first in tree. The latching mode must be BTR_SEARCH_LEAF or
-BTR_MODIFY_LEAF. */
-void
-btr_pcur_open_on_user_rec(
-	dict_index_t*	index,		/*!< in: index */
-	const dtuple_t*	tuple,		/*!< in: tuple on which search done */
-	page_cur_mode_t	mode,		/*!< in: PAGE_CUR_L, ... */
-	ulint		latch_mode,	/*!< in: BTR_SEARCH_LEAF or
-					BTR_MODIFY_LEAF */
-	btr_pcur_t*	cursor,		/*!< in: memory buffer for persistent
-					cursor */
-	mtr_t*		mtr)		/*!< in: mtr */
-{
-	btr_pcur_open_low(index, 0, tuple, mode, latch_mode, cursor, 0, mtr);
-
-	if ((mode == PAGE_CUR_GE) || (mode == PAGE_CUR_G)) {
-
-		if (btr_pcur_is_after_last_on_page(cursor)) {
-
-			btr_pcur_move_to_next_user_rec(cursor, mtr);
+		if (btr_pcur_is_before_first_in_tree(cursor)
+		    || btr_pcur_move_backward_from_page(cursor, mtr)) {
+			return false;
 		}
 	} else {
-		ut_ad((mode == PAGE_CUR_LE) || (mode == PAGE_CUR_L));
-
-		/* Not implemented yet */
-
-		ut_error;
+		btr_pcur_move_to_prev_on_page(cursor);
 	}
+
+	return true;
 }

storage/innobase/btr/btr0sea.cc

@@ -1436,19 +1436,13 @@ void btr_search_drop_page_hash_when_freed(const page_id_t page_id)
 	block = buf_page_get_gen(page_id, 0, RW_X_LATCH, NULL,
 				 BUF_PEEK_IF_IN_POOL, &mtr);
 
-	if (block) {
-		/* If AHI is still valid, page can't be in free state.
-		AHI is dropped when page is freed. */
-		DBUG_ASSERT(!block->page.is_freed());
-
-		if (block->index) {
-			/* In all our callers, the table handle should
-			be open, or we should be in the process of
-			dropping the table (preventing eviction). */
-			DBUG_ASSERT(block->index->table->get_ref_count()
-				    || dict_sys.locked());
-			btr_search_drop_page_hash_index(block);
-		}
+	if (block && block->index) {
+		/* In all our callers, the table handle should
+		be open, or we should be in the process of
+		dropping the table (preventing eviction). */
+		DBUG_ASSERT(block->index->table->get_ref_count()
+			    || dict_sys.locked());
+		btr_search_drop_page_hash_index(block);
 	}
 
 	mtr_commit(&mtr);

storage/innobase/buf/buf0dblwr.cc

@@ -83,7 +83,13 @@ bool buf_dblwr_t::create()
 start_again:
   mtr.start();
 
+  dberr_t err;
   buf_block_t *trx_sys_block= buf_dblwr_trx_sys_get(&mtr);
+  if (!trx_sys_block)
+  {
+    mtr.commit();
+    return false;
+  }
 
   if (mach_read_from_4(TRX_SYS_DOUBLEWRITE + TRX_SYS_DOUBLEWRITE_MAGIC +
                        trx_sys_block->page.frame) ==
@@ -98,10 +104,10 @@ bool buf_dblwr_t::create()
 
   if (UT_LIST_GET_FIRST(fil_system.sys_space->chain)->size < 3 * size)
   {
-too_small:
     ib::error() << "Cannot create doublewrite buffer: "
                    "the first file in innodb_data_file_path must be at least "
                 << (3 * (size >> (20U - srv_page_size_shift))) << "M.";
+fail:
     mtr.commit();
     return false;
   }
@@ -109,9 +115,13 @@ bool buf_dblwr_t::create()
   {
     buf_block_t *b= fseg_create(fil_system.sys_space,
                                 TRX_SYS_DOUBLEWRITE + TRX_SYS_DOUBLEWRITE_FSEG,
-                                &mtr, false, trx_sys_block);
+                                &mtr, &err, false, trx_sys_block);
     if (!b)
-      goto too_small;
+    {
+      ib::error() << "Cannot create doublewrite buffer: " << err;
+      goto fail;
+    }
+
     ib::info() << "Doublewrite buffer not found: creating new";
 
     /* FIXME: After this point, the doublewrite buffer creation
@@ -126,8 +136,9 @@ bool buf_dblwr_t::create()
   for (uint32_t prev_page_no= 0, i= 0, extent_size= FSP_EXTENT_SIZE;
        i < 2 * size + extent_size / 2; i++)
   {
-    buf_block_t *new_block= fseg_alloc_free_page(fseg_header, prev_page_no + 1,
-                                                 FSP_UP, &mtr);
+    buf_block_t *new_block=
+      fseg_alloc_free_page_general(fseg_header, prev_page_no + 1, FSP_UP,
+                                   false, &mtr, &mtr, &err);
     if (!new_block)
     {
       ib::error() << "Cannot create doublewrite buffer: "
@@ -395,9 +406,12 @@ void buf_dblwr_t::recover()
                             physical_size, read_buf);
 
     if (UNIV_UNLIKELY(fio.err != DB_SUCCESS))
+    {
        ib::warn() << "Double write buffer recovery: " << page_id
                   << " ('" << space->chain.start->name
                   << "') read failed with error: " << fio.err;
+       continue;
+    }
 
     if (buf_is_zeroes(span<const byte>(read_buf, physical_size)))
     {

storage/innobase/dict/dict0defrag_bg.cc

@@ -194,7 +194,7 @@ static void dict_stats_process_entry_from_defrag_pool(THD *thd)
   {
     if (dict_index_t *index= !table->corrupted
         ? dict_table_find_index_on_id(table, index_id) : nullptr)
-      if (!index->is_corrupted())
+      if (index->is_btree())
         dict_stats_save_defrag_stats(index);
     dict_table_close(table, false, thd, mdl);
   }
@@ -307,7 +307,8 @@ btr_get_size_and_reserved(
 		return(ULINT_UNDEFINED);
 	}
 
-	buf_block_t* root = btr_root_block_get(index, RW_SX_LATCH, mtr);
+	dberr_t err;
+	buf_block_t* root = btr_root_block_get(index, RW_SX_LATCH, mtr, &err);
 	*used = 0;
 	if (!root) {
 		return ULINT_UNDEFINED;