1. 25 Nov, 2021 4 commits
  2. 24 Nov, 2021 9 commits
  3. 23 Nov, 2021 3 commits
  4. 20 Nov, 2021 1 commit
    • Marko Mäkelä's avatar
      MDEV-27058 fixup: Fix MemorySanitizer, and GCC 4.8.5 ICE on ARMv8 · 9436c778
      Marko Mäkelä authored
      buf_LRU_scan_and_free_block(): It turns out that even with
      -fno-expensive-optimizations, GCC 4.8.5 may fail to split an instruction.
      For the non-embedded server, -O1 would fail and -Og would seem to work,
      while the embedded server build seems to require -O0.
      
      buf_block_init(): Correct the MemorySanitizer instrumentation.
      
      buf_page_get_low(): Do not read dirty data from read-fixed blocks.
      These data races were identified by MemorySanitizer. If a read-fixed
      block is being accessed, we must acquire and release a page latch,
      so that the read-fix (and the exclusive page latch) will be released
      and it will be safe to read the page frame contents if needed,
      even before acquiring the final page latch. We do that in
      buf_read_ahead_linear() and for the allow_ibuf_merge check.
      
      mtr_t::page_lock(): Assert that the block is not read-fixed.
      9436c778
  5. 19 Nov, 2021 6 commits
  6. 18 Nov, 2021 7 commits
    • Sergei Krivonos's avatar
      Json_writer_object add integers · d4d71153
      Sergei Krivonos authored
      d4d71153
    • Sergei Krivonos's avatar
    • Marko Mäkelä's avatar
      MDEV-27058: Reduce the size of buf_block_t and buf_page_t · aaef2e1d
      Marko Mäkelä authored
      buf_page_t::frame: Moved from buf_block_t::frame.
      All 'thin' buf_page_t describing compressed-only ROW_FORMAT=COMPRESSED
      pages will have frame=nullptr, while all 'fat' buf_block_t
      will have a non-null frame pointing to aligned innodb_page_size bytes.
      This eliminates the need for separate states for
      BUF_BLOCK_FILE_PAGE and BUF_BLOCK_ZIP_PAGE.
      
      buf_page_t::lock: Moved from buf_block_t::lock. That is, all block
      descriptors will have a page latch. The IO_PIN state that was used
      for discarding or creating the uncompressed page frame of a
      ROW_FORMAT=COMPRESSED block is replaced by a combination of read-fix
      and page X-latch.
      
      page_zip_des_t::fix: Replaces state_, buf_fix_count_, io_fix_, status
      of buf_page_t with a single std::atomic<uint32_t>. All modifications
      will use store(), fetch_add(), fetch_sub(). This space was previously
      wasted to alignment on 64-bit systems. We will use the following encoding
      that combines a state (partly read-fix or write-fix) and a buffer-fix
      count:
      
      buf_page_t::NOT_USED=0 (previously BUF_BLOCK_NOT_USED)
      buf_page_t::MEMORY=1 (previously BUF_BLOCK_MEMORY)
      buf_page_t::REMOVE_HASH=2 (previously BUF_BLOCK_REMOVE_HASH)
      buf_page_t::FREED=3 + fix: pages marked as freed in the file
      buf_page_t::UNFIXED=1U<<29 + fix: normal pages
      buf_page_t::IBUF_EXIST=2U<<29 + fix: normal pages; may need ibuf merge
      buf_page_t::REINIT=3U<<29 + fix: reinitialized pages (skip doublewrite)
      buf_page_t::READ_FIX=4U<<29 + fix: read-fixed pages (also X-latched)
      buf_page_t::WRITE_FIX=5U<<29 + fix: write-fixed pages (also U-latched)
      buf_page_t::WRITE_FIX_IBUF=6U<<29 + fix: write-fixed; may have ibuf
      buf_page_t::WRITE_FIX_REINIT=7U<<29 + fix: write-fixed (no doublewrite)
      
      buf_page_t::write_complete(): Change WRITE_FIX or WRITE_FIX_REINIT to
      UNFIXED, and WRITE_FIX_IBUF to IBUF_EXIST, before releasing the U-latch.
      
      buf_page_t::read_complete(): Renamed from buf_page_read_complete().
      Change READ_FIX to UNFIXED or IBUF_EXIST, before releasing the X-latch.
      
      buf_page_t::can_relocate(): If the page latch is being held or waited for,
      or the block is buffer-fixed or io-fixed, return false. (The condition
      on the page latch is new.)
      
      Outside buf_page_get_gen(), buf_page_get_low() and buf_page_free(), we
      will acquire the page latch before fix(), and unfix() before unlocking.
      
      buf_page_t::flush(): Replaces buf_flush_page(). Optimize the
      handling of FREED pages.
      
      buf_pool_t::release_freed_page(): Assume that buf_pool.mutex is held
      by the caller.
      
      buf_page_t::is_read_fixed(), buf_page_t::is_write_fixed(): New predicates.
      
      buf_page_get_low(): Ignore guesses that are read-fixed because they
      may not yet be registered in buf_pool.page_hash and buf_pool.LRU.
      
      buf_page_optimistic_get(): Acquire latch before buffer-fixing.
      
      buf_page_make_young(): Leave read-fixed blocks alone, because they
      might not be registered in buf_pool.LRU yet.
      
      recv_sys_t::recover_deferred(), recv_sys_t::recover_low():
      Possibly fix MDEV-26326, by holding a page X-latch instead of
      only buffer-fixing the page.
      aaef2e1d
    • Marko Mäkelä's avatar
      MDEV-27058: Move buf_page_t::slot to IORequest::slot · db915f73
      Marko Mäkelä authored
      MDEV-23855 and MDEV-23399 already moved some transient data fields
      from buffer pool page descriptors to IORequest, but the write buffer
      of PAGE_COMPRESSED or ENCRYPTED tables was missed. Since is only needed
      during asynchronous page write requests, it belongs to IORequest.
      db915f73
    • Marko Mäkelä's avatar
      MDEV-26769 follow-up: Remove unnecessary page locking · 02e72f7b
      Marko Mäkelä authored
      btr_cur_optimistic_latch_leaves(): Use transactional_shared_lock_guard.
      
      btr_cur_latch_leaves(): Avoid acquiring some page latches, because
      the changes are already blocked by index->lock.
      
      btr_cur_search_to_nth_level_func(): Remove a redundant variable
      retrying_for_search_prev=!!prev_tree_blocks, and avoid acquiring
      some page latches.
      02e72f7b
    • Marko Mäkelä's avatar
      MDEV-27069: heap-use-after-free in dict_stats_recalc_pool_del() · 14c5178f
      Marko Mäkelä authored
      dict_stats_recalc_pool_del(): Always reposition the iterators after
      releasing and reacquiring the mutex. Another thread could have modified
      recalc_pool, causing reallocation of the underlying memory while
      we were waiting.
      
      This fixes a regression that was caused by
      commit 45a05fda (MDEV-25919).
      14c5178f
    • Marko Mäkelä's avatar
      862eccd5
  7. 17 Nov, 2021 7 commits
    • Marko Mäkelä's avatar
      Merge 10.5 into 10.6 · 0a168398
      Marko Mäkelä authored
      0a168398
    • Marko Mäkelä's avatar
      Merge 10.4 into 10.5 · 5489ce0a
      Marko Mäkelä authored
      5489ce0a
    • Marko Mäkelä's avatar
      Merge 10.3 into 10.4 · 70e788b1
      Marko Mäkelä authored
      70e788b1
    • Marko Mäkelä's avatar
      Merge 10.2 into 10.3 · 9962cda5
      Marko Mäkelä authored
      9962cda5
    • Marko Mäkelä's avatar
      MDEV-23805 fixup: Adjsut the MDEV-16131 and MDEV-24730 tests · 878f7e38
      Marko Mäkelä authored
      MDEV-23805 simplified the treatment of empty tables during ALTER TABLE,
      which could prevent the scenarios that were previously reported and
      fixed as MDEV-16131 and MDEV-24730.
      
      With the MDEV-23805 fix, the statement
      SET DEBUG_SYNC = 'now WAIT_FOR copied';
      could occasionally time out, depending on timing.
      Apparently, there was a race condition where purge could resume
      (and empty the table) before ALTER TABLE got the chance to execute.
      We must prevent the purge of history from running before
      ALTER TABLE has started executing.
      878f7e38
    • Eugene Kosov's avatar
      MDEV-26747 improve corruption check for encrypted tables on ALTER IMPORT · ed0a224b
      Eugene Kosov authored
      fil_space_decrypt(): change signature to return status via dberr_t only.
      Also replace impossible condition with an assertion and prove it via
      test cases.
      ed0a224b
    • Igor Babaev's avatar
      MDEV-26825 Bogus error for query with two usage of CTE referring another CTE · 8f24f5fe
      Igor Babaev authored
        This bug affected queries with two or more references to a CTE referring
      another CTE if the definition of the latter contained an invocation of
      a stored function that used a base table. The bug could lead to a bogus
      error message or to an assertion failure.
        For any non-first reference to CTE cte1 With_element::clone_parsed_spec()
      is called that parses the specification of cte1 to construct the unit
      structure for this usage of cte1. If cte1 refers to another CTE cte2
      outside of the specification of cte1 then With_element::clone_parsed_spec()
      has to be called for cte2 as well. This call is made by the function
      LEX::resolve_references_to_cte() within the invocation of the function
      With_element::clone_parsed_spec() for cte1.
        When the specification of a CTE is parsed all table references encountered
      in it must be added to the global list of table references for the query.
      As the specification for the non-first usage of a CTE is parsed at a
      recursive call of the parser the function With_element::clone_parsed_spec()
      invoked at this recursive call should takes care of appending the list of
      table references encountered in the specification of this CTE cte1 to the
      list of table references created for the query. And it should do it after
      the call of LEX::resolve_references_to_cte() that resolves references to
      CTEs defined outside of the specification of cte1 because this call may
      invoke the parser again for specifications of other CTEs and  the table
      references from their specifications must ultimately appear in the global
      list of table references of the query.
        The code of With_element::clone_parsed_spec() misplaced the call of
      LEX::resolve_references_to_cte(). As a result LEX::query_tables_last used
      for the query that was supposed to point to the field 'next_global' of the
      last element in the global list of table references actually pointed to
      'next_global' of the previous element.
        The above inconsistency certainly caused serious problems when table
      references used in the stored functions invoked in cloned specifications
      of CTEs were added to the global list of table references.
      8f24f5fe
  8. 16 Nov, 2021 3 commits
    • Marko Mäkelä's avatar
      Merge 10.5 into 10.6 · 6841d1af
      Marko Mäkelä authored
      6841d1af
    • Marko Mäkelä's avatar
      MDEV-27059 page_zip_dir_insert() may corrupt ROW_FORMAT=COMPRESSED tables · ebb15f98
      Marko Mäkelä authored
      In commit 7ae21b18 (MDEV-12353)
      the recovery of ROW_FORMAT=COMPRESSED tables was changed.
      Changes would be logged in a physical format for the compressed
      page image, so that the page need not be decompressed or compressed
      during recovery.
      
      page_zip_write_rec(): Log any update of the delete-mark flag in the
      ROW_FORMAT=COMPRESSED page.
      
      page_zip_dir_insert(): Copy the delete-mark flag. A delete-marked
      record may be inserted by btr_cur_pessimistic_update() via
      btr_cur_insert_if_possible(), page_cur_tuple_insert(),
      page_cur_insert_rec_zip(). In the observed scenario, it was
      an ROLLBACK. Presumably, the test case involved repeated DELETE
      and INSERT of the same key, or updating a key back and forth.
      This change alone might make the adjustment in page_zip_write_rec()
      redundant, but we play it safe because we failed to create a
      minimal test case for this scenario.
      ebb15f98
    • Marko Mäkelä's avatar
      MDEV-27028 [ERROR] [FATAL] InnoDB: SYS_VIRTUAL.TABLE_ID mismatch · 89ab2538
      Marko Mäkelä authored
      If the server is killed during any DDL operation that is about to
      delete an .ibd file, recovery could crash when attempting to load
      the table definition of the being-dropped table. By design of
      commit 1bd681c8 (MDEV-25506 part 3),
      a table whose name starts with #sql-ib in the data dictionary may
      belong to an uncommitted transaction. So, we must ignore any missing
      SYS_COLUMNS, SYS_FIELDS, and SYS_VIRTUAL records for such tables.
      
      The "ID mismatch" error messages were misleading; they really mean
      "record not found".
      89ab2538