1. 27 Nov, 2020 2 commits
    • Sujatha's avatar
      MDEV-16437: merge 5.7 P_S replication instrumentation and tables · 5582a45a
      Sujatha authored
      Merge 'replication_connection_configuration' table.
      
      Replaced following columns:
        - CHANNEL_NAME with CONNECTION_NAME
        - AUTO_POSITION with MASTER_USE_GTID
      Added new columns for:
        - IGNORE_SERVER_IDS
        - DO_DOMAIN_IDS
        - IGNORE_SERVER_IDS
      Removed following columns as they are not part of mariadb replication
      connection configuration:
        - NETWORK_INTERFACE
        - TLS_VERSION
      5582a45a
    • Sujatha's avatar
      MDEV-20220: Merge 5.7 P_S replication table 'replication_applier_status_by_worker · e5fc78f8
      Sujatha authored
      Fix:
      ===
      Iterate through rpl_parallel_thread_pool and display slave worker thread
      specific information as part of 'replication_applier_status_by_worker' table.
      
      ---------------------------------------------------------------------------------
      |Column Name:           |        Description:                                   |
      |-------------------------------------------------------------------------------|
      |                       |                                                       |
      |THREAD_ID              | Thread_Id as displayed in 'performance_schema.threads'|
      |                       | table for thread with name                            |
      |                       | 'thread/sql/rpl_parallel_thread'                      |
      |                       |                                                       |
      |                       | THREAD_ID will be NULL when worker threads are stopped|
      |                       | due to error/force stop                               |
      |                       |                                                       |
      |SERVICE_STATE          | Thread is running or not                              |
      |                       |                                                       |
      |LAST_SEEN_TRANSACTION  | Last GTID executed by worker                          |
      |                       |                                                       |
      |LAST_ERROR_NUMBER      | Last Error that occurred on a particular worker       |
      |                       |                                                       |
      |LAST_ERROR_MESSAGE     | Last error specific message                           |
      |                       |                                                       |
      |LAST_ERROR_TIMESTAMP   | Time stamp of last error                              |
      |                       |                                                       |
      |WORKER_IDLE_TIME       | Total idle time in seconds that the worker thread has |
      |                       | spent waiting for work from SQL thread                |
      |                       |                                                       |
      |LAST_TRANS_RETRY_COUNT | Total number of retries attempted by last transaction |
      ---------------------------------------------------------------------------------
      
      
      In case STOP SLAVE is executed worker threads will be gone, hence worker
      threads will be unavailable. Querying the table at this stage will give empty
      rows. To address this case when worker threads are about to stop, due to an
      error or forced stop, create a backup pool and preserve the data which is
      relevant to populate performance schema table. Clear the backup pool upon
      slave start.
      e5fc78f8
  2. 26 Nov, 2020 1 commit
  3. 25 Nov, 2020 11 commits
    • Marko Mäkelä's avatar
      MDEV-24280 InnoDB triggers too many independent periodic tasks · 657fcdf4
      Marko Mäkelä authored
      A side effect of MDEV-16264 is that a large number of threads will
      be created at server startup, to be destroyed after a minute or two.
      
      One source of such thread creation is srv_start_periodic_timer().
      InnoDB is creating 3 periodic tasks: srv_master_callback (1Hz)
      srv_error_monitor_task (1Hz), and srv_monitor_task (0.2Hz).
      
      It appears that we can merge srv_error_monitor_task and srv_monitor_task
      and have them invoked 4 times per minute (every 15 seconds). This will
      affect our ability to enforce innodb_fatal_semaphore_wait_threshold and
      some computations around BUF_LRU_STAT_N_INTERVAL.
      
      We could remove srv_master_callback along with the DROP TABLE queue
      at some point of time in the future. We must keep it independent
      of the innodb_fatal_semaphore_wait_threshold detection, because
      the background DROP TABLE queue could get stuck due to dict_sys
      being locked by another thread. For now, srv_master_callback
      must be invoked once per second, so that
      innodb_flush_log_at_timeout=1 can work.
      
      BUF_LRU_STAT_N_INTERVAL: Reduce the precision and extend the time
      from 50*1 second to 4*15 seconds.
      
      srv_error_monitor_timer: Remove.
      
      MAX_MUTEX_NOWAIT: Increase from 20*1 second to 2*15 seconds.
      
      srv_refresh_innodb_monitor_stats(): Avoid a repeated call to time(NULL).
      Change the interval to less than 60 seconds.
      
      srv_monitor(): Renamed from srv_monitor_task.
      
      srv_monitor_task(): Renamed from srv_error_monitor_task().
      Invoked only once in 15 seconds. Invoke also srv_monitor().
      Increase the fatal_cnt threshold from 10*1 second to 1*15 seconds.
      
      sync_array_print_long_waits_low(): Invoke time(NULL) only once.
      Remove a bogus message about printouts for 30 seconds. Those
      printouts were effectively already disabled in MDEV-16264
      (commit 5e62b6a5).
      657fcdf4
    • Marko Mäkelä's avatar
      MDEV-24278 InnoDB page cleaner keeps waking up on idle server · 7b1252c0
      Marko Mäkelä authored
      The purpose of the InnoDB page cleaner subsystem is to write out
      modified pages from the buffer pool to data files. When the
      innodb_max_dirty_pages_pct_lwm is not exceeded or
      innodb_adaptive_flushing=ON decides not to write out anything,
      the page cleaner should keep sleeping indefinitely until the state
      of the system changes: a dirty page is added to the buffer pool such
      that the page cleaner would no longer be idle.
      
      buf_flush_page_cleaner(): Explicitly note when the page cleaner is idle.
      When that happens, use mysql_cond_wait() instead of mysql_cond_timedwait().
      
      buf_flush_insert_into_flush_list(): Wake up the page cleaner if needed.
      
      innodb_max_dirty_pages_pct_update(),
      innodb_max_dirty_pages_pct_lwm_update():
      Wake up the page cleaner just in case.
      
      Note: buf_flush_ahead(), buf_flush_wait_flushed() and shutdown are
      already waking up the page cleaner thread.
      7b1252c0
    • Marko Mäkelä's avatar
      MDEV-24270: Clarify some comments · f693b725
      Marko Mäkelä authored
      f693b725
    • Vladislav Vaintroub's avatar
      Fix misspelling. · 2de95f7a
      Vladislav Vaintroub authored
      Kudos to Marko for finding.
      2de95f7a
    • Vladislav Vaintroub's avatar
      Cleanup. Remove obsolete comment · af98fddc
      Vladislav Vaintroub authored
      af98fddc
    • Vladislav Vaintroub's avatar
    • Vladislav Vaintroub's avatar
      Partially Revert "MDEV-24270: Collect multiple completed events at a time" · 78df9e37
      Vladislav Vaintroub authored
      This partially reverts commit 6479006e.
      
      Remove the constant tpool::aio::N_PENDING, which has no
      intrinsic meaning for the tpool.
      78df9e37
    • Marko Mäkelä's avatar
    • Marko Mäkelä's avatar
      4a22056c
    • Marko Mäkelä's avatar
      MDEV-24270: Collect multiple completed events at a time · 6479006e
      Marko Mäkelä authored
      tpool::aio::N_PENDING: Replaces OS_AIO_N_PENDING_IOS_PER_THREAD.
      This limits two similar things: the number of outstanding requests
      that a thread may io_submit(), and the number of completed requests
      collected at a time by io_getevents().
      6479006e
    • Marko Mäkelä's avatar
      MDEV-24270 Misuse of io_getevents() causes wake-ups at least twice per second · 7a9405e3
      Marko Mäkelä authored
      In the asynchronous I/O interface, InnoDB is invoking io_getevents()
      with a timeout value of half a second, and requesting exactly 1 event
      at a time.
      
      The reason to have such a short timeout is to facilitate shutdown.
      
      We can do better: Use an infinite timeout, wait for a larger maximum
      number of events. On shutdown, we will invoke io_destroy(), which
      should lead to the io_getevents system call reporting EINVAL.
      
      my_getevents(): Reimplement the libaio io_getevents() by only invoking
      the system call. The library implementation would try to elide the
      system call and return 0 immediately if aio_ring_is_empty() holds.
      Here, we do want a blocking system call, not 100% CPU usage. Neither
      do we want the aio_ring_is_empty() trigger SIGSEGV because it is
      dereferencing some memory that was freed by io_destroy().
      7a9405e3
  4. 24 Nov, 2020 13 commits
  5. 23 Nov, 2020 4 commits
  6. 20 Nov, 2020 5 commits
    • Marko Mäkelä's avatar
      MDEV-24167: Remove PFS instrumentation of buf_block_t · 1e5d989d
      Marko Mäkelä authored
      We always defined PFS_SKIP_BUFFER_MUTEX_RWLOCK, that is,
      the latches of the buffer pool blocks were never instrumented
      in PERFORMANCE_SCHEMA.
      
      For some reason, the debug_latch (which enforce proper usage of
      buffer-fixing in debug builds) was instrumented.
      1e5d989d
    • Marko Mäkelä's avatar
      MDEV-22871 fixup: Relax a debug assertion · 156cb94b
      Marko Mäkelä authored
      In commit bf3c862f we introduced
      an assertion that may dereference a null pointer.
      
      This regression was caught by running the following:
      ./mtr --parallel=auto --suite=innodb \
      --mysqld=--loose-innodb-adaptive-hash-index
      
      The adaptive hash index is disabled by default since
      commit 88cdfc5c (MDEV-20487)
      and hence the problem was not caught earlier.
      156cb94b
    • Marko Mäkelä's avatar
      Run innodb_wl6326_big only in debug builds · 3c8ecb5b
      Marko Mäkelä authored
      The test seems to deterministically fail on RelWithDebInfo builds
      due to a timeout in wait_condition.inc.
      
      According to Matthias Leich (the original author of the test),
      the failure rate would reduce if we disabled the purge of
      transaction history by setting innodb_force_recovery=2.
      
      For now, let us run this stress test on debug builds only.
      3c8ecb5b
    • Marko Mäkelä's avatar
      Cleanup: Fix build problems with the Intel compiler · 8ac19be8
      Marko Mäkelä authored
      fil_space_t::flush_low(): Define and declare without inline.
      
      ut_is_2pow(): Remove UNIV_LIKELY. This is almost exclusively
      used in debug assertions. UNIV_LIKELY is not compatible with
      static_assert in some compilers.
      8ac19be8
    • Marko Mäkelä's avatar
      9c455945
  7. 19 Nov, 2020 2 commits
  8. 18 Nov, 2020 1 commit
    • Marko Mäkelä's avatar
      MDEV-24224 Gap lock on delete in 10.5 using READ COMMITTED · 33d41167
      Marko Mäkelä authored
      When MDEV-19544 (commit 1a6f4704)
      simplified the initialization of the local variable
      set_also_gap_locks, an inadvertent change was included.
      Essentially, all code branches that are executed when
      set_also_gap_locks hold must also ensure that
      trx->isolation_level > TRX_ISO_READ_COMMITTED holds.
      This was being violated in a few code paths.
      
      It turns out that there is an even simpler fix: Remove the test
      of thd_is_select() completely. In that way, the first part of
      UPDATE or DELETE should work exactly like SELECT...FOR UPDATE.
      
      thd_is_select(): Remove.
      33d41167
  9. 17 Nov, 2020 1 commit