1. 03 Feb, 2022 1 commit
  2. 02 Feb, 2022 3 commits
  3. 01 Feb, 2022 2 commits
    • Oleksandr Byelkin's avatar
      new CC 3.3 · 83cdcf5b
      Oleksandr Byelkin authored
      83cdcf5b
    • Andrei's avatar
      MDEV-27687 Assertion `!thd->rgi_fake || ... · 2b554116
      Andrei authored
      The added by MDEV-11675 assert is incorrected assuming
      a replayed from binlog transaction can't contain invoke
      a FD event's apply method.
      
      In fact it can do that through BINLOG event.
      The test case like
      
        BEGIN;
          INSERT INTO t1 VALUES(10);
          BINLOG '
          SOgWTg8BAAAAbgAAAHIAAAAAAAQANS42LjMtbTUtZGVidWctbG9nAAAAAAAAAAAAAAAAAAAAAAAA
          AAAAAAAAAAAAAAAAAABI6BZOEzgNAAgAEgAEBAQEEgAAVgAEGggAAAAICAgCAAAAAAVAYI8=';
      
      is provided (by yet to merged atm) MDEV-27536 fixes.
      
      The assert is removed.
      2b554116
  4. 31 Jan, 2022 2 commits
    • Andrei's avatar
      MDEV-11675. Convert the new session var to bool type and test changes · fe2d90cc
      Andrei authored
      The new @@binlog_alter_two_phase is converted to `my_bool` type.
      fe2d90cc
    • Brandon Nesterenko's avatar
      MDEV-4989: Support for GTID in mysqlbinlog · a64991df
      Brandon Nesterenko authored
      This patch fixes two issues:
      
      First, it fixes test failure due to GTID List events
      having inconsistent ordering of domain ids. In
      particular, this patch ensures that a GTID list log
      event will have its GTIDs ordered by domain id
      (ascending) followed by sequence number (ascending).
      
      Second, it fixes an assert which could use an
      unintialized variable.
      
      Reviewed By:
      ============
      Andrei Elkin <andrei.elkin@mariadb.com>
      a64991df
  5. 27 Jan, 2022 2 commits
    • Sachin's avatar
      MDEV-11675 Lag Free Alter On Slave · 0c5d1342
      Sachin authored
      This commit implements two phase binloggable ALTER.
      When a new
      
            @@session.binlog_alter_two_phase = YES
      
      ALTER query gets logged in two parts, the START ALTER and the COMMIT
      or ROLLBACK ALTER. START Alter is written in binlog as soon as
      necessary locks have been acquired for the table. The timing is
      such that any concurrent DML:s that update the same table are either
      committed, thus logged into binary log having done work on the old
      version of the table, or will be queued for execution on its new
      version.
      
      The "COMPLETE" COMMIT or ROLLBACK ALTER are written at the very point
      of a normal "single-piece" ALTER that is after the most of
      the query work is done. When its result is positive COMMIT ALTER is
      written, otherwise ROLLBACK ALTER is written with specific error
      happened after START ALTER phase.
      Replication of two-phase binloggable ALTER is
      cross-version safe. Specifically the OLD slave merely does not
      recognized the start alter part, still being able to process and
      memorize its gtid.
      
      Two phase logged ALTER is read from binlog by mysqlbinlog to produce
      BINLOG 'string', where 'string' contains base64 encoded
      Query_log_event containing either the start part of ALTER, or a
      completion part. The Query details can be displayed with `-v` flag,
      similarly to ROW format events.  Notice, mysqlbinlog output containing
      parts of two-phase binloggable ALTER is processable correctly only by
      binlog_alter_two_phase server.
      
      @@log_warnings > 2 can reveal details of binlogging and slave side
      processing of the ALTER parts.
      
      The current commit also carries fixes to the following list of
      reported bugs:
      MDEV-27511, MDEV-27471, MDEV-27349, MDEV-27628, MDEV-27528.
      
      Thanks to all people involved into early discussion of the feature
      including Kristian Nielsen, those who helped to design, implement and
      test: Sergei Golubchik, Andrei Elkin who took the burden of the
      implemenation completion, Sujatha Sivakumar, Brandon
      Nesterenko, Alice Sherepa, Ramesh Sivaraman, Jan Lindstrom.
      0c5d1342
    • Marko Mäkelä's avatar
      MDEV-27621 Backup fails with FATAL ERROR: Was only able to copy log · c64e507f
      Marko Mäkelä authored
      In commit 685d958e (MDEV-14425)
      a bug was introduced to mariadb-backup --backup for the case when
      the log is wrapping around to log_sys.START_OFFSET (12288).
      
      This could also cause a "Missing FILE_CHECKPOINT" error during
      mariadb-backup --prepare, in case the log copying resumed after
      the server had produced a multiple of innodb_log_file_size-12288
      bytes of more log so that the last mini-transaction would end
      exactly at the end of the log file.
      
      xtrabackup_copy_logfile(): If the log wraps around, read everything
      to the end of the log file, and then the rest from log_sys.START_OFFSET.
      c64e507f
  6. 26 Jan, 2022 30 commits
    • Brandon Nesterenko's avatar
      MDEV-4989: Support for GTID in mysqlbinlog · 79e3ee00
      Brandon Nesterenko authored
      New Feature:
      ===========
      This commit extends the mariadb-binlog capabilities to allow events
      to be filtered by GTID ranges. More specifically, the
      --start-position and --stop-position arguments have been extended to
      accept values formatted as a list of GTID positions, e.g.
      --start-position=0-1-0,1-2-55. The following specific capabilities
      are addressed:
         1) GTIDs can be used to filter results on local binlog files
         2) GTIDs can be used to filter results from remote servers
         3) Implemented --gtid-strict-mode that ensures the GTID event
            stream in each domain is monotonically increasing
         4) Added new level of verbosity in mysqlbinlog -vvv to print
            additional diagnostic information/warnings about invalid GTID
            states
         5) For a given GTID range, its start and stop position parameters
            aim to mimic the behaviors of
            CHANGE MASTER TO MASTER_USE_GTID=slave_pos and
            START SLAVE UNTIL master_gtid_pos=<GTID>, respectively. In
            particular, the start-position list expresses a gtid state of
            the server, similarly to how @@global.gtid_slave_pos expresses
            the gtid state of a slave server when connecting to a master
            with MASTER_USE_GTID=slave_pos.
            The GTID start-position list is exclusive and the
            stop-position list is inclusive. This allows users to receive
            events strictly after those that they already have, and is
            useful in  cases of point in (logical) time recovery including
            1) events were received out of order and should be re-sent, or
            2) specifying the gtid state of a slave to get events newer
            than their current state. If a seq_no is 0 for start-position,
            it means to include the entirety of the domain. If a seq_no is
            0 for stop-position, it means to exclude all events from that
            domain. The GTIDs provided in a start position argument must
            match with the GTID state of the first processed log (i.e.
            those listed in the Gtid_list event). If a stop position is
            provided, the events that are output are limited to only those
            with domain ids listed in the argument. When specifying
            combinations of start and stop positions, the following
            behaviors are expected:
      
      [--start-position without --stop-position]: Events that have domain
      ids in the start position are output if their seq_no occurs after
      the respective start position. Events with domain ids that are
      unspecified in the start position list are also output. Note that if
      the Gtid_list event of the first binary log is populated (i.e.
      non-empty), each domain in the Gtid_list must be present in the
      start-position list with a seq_no at or after the listed value.
      This behavior mimics how a slave only processes events after the
      state provided by @@global.gtid_slave_pos when connecting to a
      master with CHANGE MASTER TO MASTER_USE_GTID=slave_pos.
      
      [--stop-position without --start-position]: Output is limited to
      only events with both 1) domain ids that are present in the given
      stop position list and 2) seq_nos that are less than or equal to
      their respective stop GTID. Once all GTIDs in the stop position
      list have been processed, the program will stop processing log
      files. This behavior mimics how
      START SLAVE UNTIL master_gtid_pos=<G>
      has a slave only process events with domain ids present in G with
      their seq_nos at or before the respective gtid.
      
      [--start-position and --stop-position]: Output consists of the
      intersection between the events permitted by both the start and stop
      position rules. More concretely, the output can be defined by a
      union of the following rules:
      
        1. For domains which exist in both the start and stop position
           lists, the events which exist in-between these positions
           (exclusive start, inclusive stop) are output
        2. For all other events, the rules of
           [--stop-position without --start-position] are followed
      
      This is due to the implicit filtering within each individual rule.
      Even though the start position rule always includes events from
      unspecified domains, the stop position rule takes precedence because
      it always excludes events from unspecified domains. In other words,
      events which the start position rule would have included would then
      always be excluded by the stop position rule.
      
      [neither --start-position nor --stop-position]: Events are not
      omitted based on GTID positioning; however, --gtid-strict-mode and
      -vvv can still analyze gtid correctness for warning and error
      reporting.
      
      [repeated specification of --start-position or --stop-position]:
      Subsequent specifications of start and stop positions completely
      override previous ones. E.g., if invoked as
      mysqlbinlog --start-position=<G1> --start-position=<G2> ...
      All GTIDs specified in G1 are ignored and only those specified in G2
      are used for the start position.
      
      A few additional notes:
       1) this commit squashes together the commits:
      f4319661120e-78a9d49907ba
      
       2) Changed rpl.rpl_blackhole_row_annotate test because it has
      out of order GTIDs in its binlog, so I added
      --skip-gtid-strict-mode
      
       3) After all binlog events have been written, the session server
          id and domain id are reset to their values in the global state
      
      Reviewed By:
      ===========
      Andrei Elkin: <andrei.elkin@mariadb.com>
      79e3ee00
    • Sergei Golubchik's avatar
      bump the version and maturity · 343134fc
      Sergei Golubchik authored
      343134fc
    • Rucha Deodhar's avatar
      MDEV-26238: Remove inconsistent behaviour of --default-* options · 5217211e
      Rucha Deodhar authored
      in my_print_defaults
      
      Analysis: --defaults* option is recognized anywhere in the commandline
      instead of only at the beginning because handle_options() recognizes
      options in any order.
      Fix: use get_defaults_options() which recognizes --defaults* options only at
      the beginning. After this is done, we only want to recognize other options
      given in any order which can be done using handle_options(). So only skip
      --defaults* options and pass rest of them to handle_options().
      Also, removed -e, -g and -c because only my_print_defaults supports them.
      5217211e
    • Sergei Golubchik's avatar
      MDEV-27398 DESC index causes wrong (empty) result on Federated tables · b18697fd
      Sergei Golubchik authored
      take descending indexes into account when generating a query
      b18697fd
    • Sergei Golubchik's avatar
      cleanup: FederatedX · f00236ac
      Sergei Golubchik authored
      f00236ac
    • Sergei Golubchik's avatar
      MDEV-27581 Wrong result with DESC key on partitioned Spider table · 820aa11e
      Sergei Golubchik authored
      take descending indexes into account when generating a query
      
      also fixes MDEV-27617
      820aa11e
    • Sergei Golubchik's avatar
      MDEV-27590 Auto-increment on Spider tables with DESC PK does not work properly · 24ee346a
      Sergei Golubchik authored
      use index_first for DESC keys
      24ee346a
    • Sergei Golubchik's avatar
      MDEV-27586 Auto-increment does not work with DESC on MERGE table · d751f42a
      Sergei Golubchik authored
      also fix handler::get_auto_increment()
      d751f42a
    • Sergei Golubchik's avatar
      MDEV-27434 DESC attribute does not work with auto-increment on secondary column of multi-part index · 72b5b8b2
      Sergei Golubchik authored
      when searching for the last auto-inc value, it's HA_READ_PREFIX_LAST for
      the ASC keypart, but HA_READ_PREFIX for the DESC one
      
      also fixes MDEV-27585
      72b5b8b2
    • Sergei Golubchik's avatar
    • Sergei Golubchik's avatar
      MDEV-27407 Different ASC/DESC index attributes on MERGE and underlying table... · d7e7f48e
      Sergei Golubchik authored
      MDEV-27407 Different ASC/DESC index attributes on MERGE and underlying table can cause wrong results
      
      detect if merge children are "differently defined" regarding ASC/DESC
      d7e7f48e
    • Sergei Golubchik's avatar
      MDEV-27406 Index on a HEAP table retains DESC attribute despite being hash · 58195449
      Sergei Golubchik authored
      just like in SHOW KEYS, suppress DESC in SHOW CREATE if not HA_READ_ORDER
      58195449
    • Sergei Golubchik's avatar
      MDEV-27393 Timezone tables cannot have descending indexes · aea076dc
      Sergei Golubchik authored
      replace the assert with an if().
      
      asserts should not be used on the input (even without DESC indexes
      the table could've been corrupted)
      aea076dc
    • Sergei Golubchik's avatar
      MDEV-27309 Server crash or ASAN memcpy-param-overlap upon INSERT into... · ddbb3d14
      Sergei Golubchik authored
      MDEV-27309 Server crash or ASAN memcpy-param-overlap upon INSERT into Aria/MyISAM table with DESC key
      
      MyiSAM and Aria, indexes with prefix compression, where the first keypart
      could be NULL - in this case they didn't expect the next key after the
      not NULL key to be NULL.
      
      Expect the first keypart of the next key to have zero length even
      if store_not_null==1, this combination means keypart is NULL,
      don't pack it.
      
      also fixes MDEV-27340
      ddbb3d14
    • Sergei Golubchik's avatar
      cleanup: reduce code duplication · 799a3066
      Sergei Golubchik authored
      799a3066
    • Sergei Golubchik's avatar
      MDEV-27303 Table corruption after insert into a non-InnoDB table with DESC index · 775e7ce6
      Sergei Golubchik authored
      optimized prefix search didn't take into account descending indexes
      
      also fixes MDEV-27330
      775e7ce6
    • Sergei Golubchik's avatar
      MDEV-27396 DESC index attribute remains in Archive table definition, despite... · d6ab34b3
      Sergei Golubchik authored
      MDEV-27396 DESC index attribute remains in Archive table definition, despite being apparently ignored
      
      disallow descending indexes in archive
      d6ab34b3
    • Sergei Golubchik's avatar
      MDEV-27591 Connect tables (FIX/DOS) don't work with DESC keys - wrong results · 4c6e8fc5
      Sergei Golubchik authored
      disallow descending indexes in connect
      4c6e8fc5
    • Sergei Golubchik's avatar
      MDEV-27408 DESC index on a Mroonga table causes wrong order of result set · ae8600d5
      Sergei Golubchik authored
      disallow descending indexes in mroonga
      ae8600d5
    • Sergei Golubchik's avatar
      RocksDB doesn't support DESC indexes yet · 918f5244
      Sergei Golubchik authored
      disallow descending indexes in rocksdb
      918f5244
    • Sergei Petrunia's avatar
      MDEV-27529 Wrong result upon query using index_merge with DESC key (#2) · fca37e49
      Sergei Petrunia authored
      ROR-index_merge relies on Rowid-ordered-retrieval property: a ROR scan,
      e.g. a scan on equality range
      
        tbl.key=const
      
      should return rows ordered by their Rowid. Also, handler->cmp_ref() should
      compare rowids according to the Rowid ordering.
      
      When the table's primary key uses DESC keyparts, ROR scans return rows
      according to the PK's ordering.
      
      But ha_innobase::cmp_ref() compared rowids as if PK used ASC keyparts.
      This caused wrong query results with index_merge.
      
      Fixed this by making ha_innobase::cmp_ref() compare according to the PK
      defintion, including keypart's DESC property.
      fca37e49
    • Sergei Petrunia's avatar
      MDEV-27426 Wrong result upon query using index_merge with DESC key · 62760af4
      Sergei Petrunia authored
      Make QUICK_RANGE_SELECT::cmp_next() aware of reverse-ordered key parts.
      
      (QUICK_RANGE_SELECT::cmp_prev() uses key_cmp() and so it already works
      correctly)
      62760af4
    • Sergei Petrunia's avatar
      MDEV-26996 Reverse-ordered indexes: improve print-out · 96bdda6c
      Sergei Petrunia authored
      When printing a range into optimizer trace, print DESC for columns
      that are reverse-ordered, for example:
      
         "(4) <= (key1 DESC) <= (2)"
      96bdda6c
    • Sergei Petrunia's avatar
      MDEV-26996 Reverse-ordered indexes: remove SEL_ARG::is_ascending · d6c6f79f
      Sergei Petrunia authored
      Instead, Get the "is_ascending" value from the array of KEY_PART
      structures that describes the [pseudo-]index that is being analyzed.
      d6c6f79f
    • Sergei Petrunia's avatar
      MDEV-26996 Support descending indexes in the range optimizer · cbfe6a5e
      Sergei Petrunia authored
      - Code cleanup
      - Disable "Using index for GROUP BY" over indexes with DESC keyparts
      cbfe6a5e
    • Sergei Petrunia's avatar
      Descending indexes code exposed a gap in fix for MDEV-25858. · 3a82c256
      Sergei Petrunia authored
      Extend the fix for MDEV-25858 to handle non-reverse-ordered ORDER BY:
      
      If test_if_skip_sort_order() decides to use an index to produce rows
      in the required ordering, it should disable "Range Checked for Each Record".
      
      The fix needs to be backported to earlier versions.
      3a82c256
    • Sergei Petrunia's avatar
      MDEV-26996 Support descending indexes in the range optimizer · 791146b9
      Sergei Petrunia authored
      Make the Range Optimizer support descending index key parts.
      
      We follow the approach taken in MySQL-8.
      
      See HowRangeOptimizerHandlesDescKeyparts for the description.
      791146b9
    • Sergei Golubchik's avatar
      MDEV-26938 Support descending indexes internally in InnoDB (server part) · a4cac0e0
      Sergei Golubchik authored
      * preserve DESC index property in the parser
      * store it in the frm (only for HA_KEY_ALG_BTREE)
      * read it from the frm
      * show it in SHOW CREATE
      * skip DESC indexes in opt_range.cc and opt_sum.cc
      * ORDER BY test
      
      This includes a fix of MDEV-27432.
      a4cac0e0
    • Marko Mäkelä's avatar
      MDEV-26938 Support descending indexes internally in InnoDB · 358921ce
      Marko Mäkelä authored
      This is loosely based on the InnoDB changes in
      mysql/mysql-server@97fd8b1b6993340b361fa7f85da86a308f0b5e0c
      that I had developed in 2015 or 2016.
      
      For each B-tree key field, we will allow a flag ASC/DESC to be associated.
      When PRIMARY KEY fields are internally appended to secondary indexes,
      the ASC/DESC attribute will be inherited, so that covering index scans
      will work as expected.
      
      Note: Until the subsequent commit, the DESC attribute will be ignored
      (no HA_REVERSE_SORT flag will be written to .frm files).
      
      dict_field_t::descending: A new flag to denote descending order.
      
      cmp_data(), cmp_dfield_dfield(): Add a new parameter descending.
      
      cmp_dtuple_rec(), cmp_dtuple_rec_with_match(): Add a parameter "index".
      
      dtuple_coll_eq(): Replaces dtuple_coll_cmp().
      
      cmp_dfield_dfield_eq_prefix(): Replaces cmp_dfield_dfield_like_prefix().
      
      dict_index_t::is_btree(): Check whether the index is a regular
      B-tree index (not SPATIAL, FULLTEXT, or the ibuf.index,
      or a corrupted index.
      
      btr_cur_search_to_nth_level_func(): Only attempt to use
      the adaptive hash index if index->is_btree().
      This function may also be invoked on ibuf.index, and
      cmp_dtuple_rec_with_match_bytes() will no longer work on ibuf.index
      because it assumes that the index and record fields exactly match.
      The ibuf.index is a special variadic index tree.
      
      Thanks to Thirunarayanan Balathandayuthapani for fixing some bugs:
      MDEV-27439, MDEV-27374/MDEV-27445.
      358921ce
    • Sergei Golubchik's avatar
      cleanup: tests · 349a595b
      Sergei Golubchik authored
      * combine two test files with seemingly the same name
      * comments
      349a595b