- 15 Aug, 2023 40 commits
-
-
Nikita Malyavin authored
-
Sergei Golubchik authored
so that table->update_default_fields() would know what to update
-
Nikita Malyavin authored
We shouldn't rely on `fill_extra_persistent_columns`, as it only updates fields which have an index > cols->n_bits (replication bitmap width). Actually, it should never be used, as its approach is error-prone. Normal update_virtual_fields+update_default_fields should be done.
-
Sergei Golubchik authored
-
Sergei Golubchik authored
if ALTER TABLE ... LOCK=xxx is executed under LOCK TABLES, ignore the LOCK clause, because ALTER should not downgrade already taken EXCLUSIVE table lock to SHARED or NONE. This commit preserves the existing behavior (LOCK was de facto ignored), but makes it explicit.
-
Nikita Malyavin authored
ALTER ONLINE TABLE acquires table with TL_READ. Myisam normally acquires TL_WRITE for DML, which makes it hang until table is freed. We deadlock once ALTER upgrades its MDL lock. Solution: Unlock table earlier. We don't need to hold TL_READ once we finished copying. Relay log replication requires no data locks on `from` table.
-
Sergei Golubchik authored
MDEV-28967 Assertion `marked_for_write_or_computed()' failed in Field_new_decimal::store_value / online_alter_read_from_binlog` in the catch-up phase of the online alter we apply row events, they're unpacked into `from->record[0]` and then converted to `to->record[0]`. This needs all fields of `from` to be in the `write_set`. Although practically `Field::unpack()` does not assert the `write_set`, and `Field::reset()` - used when a field value is not present in the after-image - also doesn't assert the `write_set` for many types, `Field_new_decimal::reset()` does.
-
Sergei Golubchik authored
in RBR - only show warnings for values that are to be written into a table, that is, only for the after-image. Don't show data conversion warnings for the before-image.
-
Sergei Golubchik authored
-
Sergei Golubchik authored
* don't disable warnings when catching up * do propagate warnings up like copy_data_between_tables() does
-
Sergei Golubchik authored
-
Nikita Malyavin authored
If online alter fails, TABLE_SHARE can be freed while concurrent transactions still have row events in their online_alter_cache_data. On commit they try'll to flush them, writing to TABLE_SHARE's Cache_flip_event_log, which is already freed. This causes a crash in main.alter_table_online_debug test
-
Sergei Golubchik authored
don't simply set tdc->flushed, use flush_unused(1) that removes opened but unused TABLE instances (that would otherwise prevent TABLE_SHARE from being closed by keeping the ref_count>0).
-
Sergei Golubchik authored
-
Sergei Golubchik authored
-
Sergei Golubchik authored
-
Sergei Golubchik authored
even if called from binlog_rollback()
-
Nikita Malyavin authored
-
Sergei Golubchik authored
-
Sergei Golubchik authored
instead use only one (trx) IO_CACHE and truncate it if the statement is rolled back. don't use binlog_cache_mngr to accumulate the data, use binlog_cache_data instead. (binlog_cache_data owns one IO_CACHE, binlog_cache_mngr owns two binlog_cache_data's, trx and stmt).
-
Sergei Golubchik authored
because ALTER TABLE ... DROP SYSTEM VERSIONING is not just a change in the table structure, it also deletes all historical rows
-
Sergei Golubchik authored
also * don't modify write_set * backup/restore rpl_write_set
-
Sergei Golubchik authored
-
Sergei Golubchik authored
so any other value of ALGORITHM or LOCK disables online alter
-
Sergei Golubchik authored
ht->start_consistent_snapshot() is also not a way, because some engines (e.g. rocksdb) only do it readonly. instead, downgrade the lock after reading the first row (which implicitly opens a read view).
-
Sergei Golubchik authored
no functional changes here
-
Sergei Golubchik authored
-
Sergei Golubchik authored
two new tests: * alter table times out because of a long concurrent trx * alter table adds a column in the middle
-
Nikita Malyavin authored
* Log rows in online_alter_binlog. * Table online data is replicated within dedicated binlog file * Cached data is written on commit. * Versioning is fully supported. * Works both wit and without binlog enabled. * For now savepoints setup is forbidden while ONLINE ALTER goes on. Extra support is required. We can simply log the SAVEPOINT query events and replicate them together with row events. But it's not implemented for now. * Cache flipping: We want to care for the possible bottleneck in the online alter binlog reading/writing in advance. IO_CACHE does not provide anything better that sequential access, besides, only a single write is mutex-protected, which is not suitable, since we should write a transaction atomically. To solve this, a special layer on top Event_log is implemented. There are two IO_CACHE files underneath: one for reading, and one for writing. Once the read cache is empty, an exclusive lock is acquired (we can wait for a currently active transaction finish writing), and flip() is emitted, i.e. the write cache is reopened for read, and the read cache is emptied, and reopened for writing. This reminds a buffer flip that happens in accelerated graphics (DirectX/OpenGL/etc). Cache_flip_event_log is considered non-blocking for a single reader and a single writer in this sense, with the only lock held by reader during flip. An alternative approach by implementing a fair concurrent circular buffer is described in MDEV-24676. * Cache managers: We have two cache sinks: statement and transactional. It is important that the changes are first cached per-statement and per-transaction. If a statement fails, then only statement data is rolled back. The transaction moves along, however. Turns out, there's no guarantee that TABLE well persist in thd->open_tables to the transaction commit moment. If an error occurs, tables from statement are purged. Therefore, we can't store te caches in TABLE. Ideally, it should be handlerton, but we cut the corner and store it in THD in a list.
-
Nikita Malyavin authored
Event_log is supposed to be a basic logging class that can write events in a single file. MYSQL_BIN_LOG in comparison will have: * rotation support * index files * purging * gtid and transactional information handling. * is dedicated for a general-purpose binlog
-
Nikita Malyavin authored
* Eliminate most usages of THD::use_trans_table. Only 3 left, and they are at quite high levels, and really essential. * Eliminate is_transactional argument when possible. Lots of places are left though, because of some WSREP error handling in MYSQL_BIN_LOG::set_write_error. * Remove junk binlog functions from THD * binlog_prepare_pending_rows_event is moved to log.cc inside MYSQL_BIN_LOG and is not anymore template. Instead it accepls event factory with a type code, and a callback to a constructing function in it.
-
Nikita Malyavin authored
pump up binlog and cache manager to level of binlog_log_row_internal
-
Nikita Malyavin authored
-
Nikita Malyavin authored
1. Change m_size to uint. This removes some implicit conversions. See unpack_row, for instance: uint max_cols= MY_MIN(tabledef->size(), cols->n_bits); 2. Improve table_def memory layout by reordering columns
-
Nikita Malyavin authored
-
Sergei Golubchik authored
alter_table_online -> alter_table_locknone gis-alter_table_online -> gis-alter_table
-
Sergei Golubchik authored
-
Sergei Golubchik authored
it was redundant, duplicating vcol_type == VCOL_GENERATED_STORED. Note that VCOL_DEFAULT is not "stored", "stored vcol" means that after rnd_next or index_read/etc the field value is already in the record[0] and does not need to be calculated separately
-
Sergei Golubchik authored
unpack_row() must calculate all stored and indexed vcols (in fill_extra_persistent_columns()). Also Update and Delete row events must mark in read_set all columns needed for calculating all stored and indexed vcols. If it's done properly in do_apply_event(), it no longer needs to be repeated per row.
-
Sergei Golubchik authored
it was created to handle the case of the concurrent inplace add index. So it should only work in that case, otherwise it's doing server's job (marks virtual column dependencies) and hides server's bugs. And it was redundant.
-