1. 17 Jul, 2024 12 commits
    • Rucha Deodhar's avatar
      MDEV-32456: incorrect result of gis function in view protocol · 1f28350b
      Rucha Deodhar authored
      There are 3 diff in result:
      1) NULL value from SELECT
      Due to incorrect truncating of the hex value, incorrect value is
      written instead of original value to the view frm. This results in reading
      incorrect value from frm, so eventual result is NULL.
      2) 'Name_exp1' in column name (in gis.test)
      This was because the identifier in SELECT is longer than 64 characters,
      so 'Name_exp1' alias is also written to the view frm.
      3)diff in explain extended
      This was because the query plan for view protocol doesn't
      contain database name. As a fix, disable view protocol for that particular
      query.
      1f28350b
    • Souradeep Saha's avatar
      Refactor import * with only required imports · 4a89f79b
      Souradeep Saha authored
      Import only the required functions instead of all the functions from the
      module to reduce the unnecessary functions in the namespace and prevent
      shadowing. Note: All code changes are non-functional.
      
      All new code of the whole pull request, including one or several
      files that are either new files or modified ones, are contributed
      under the BSD-new license. I am contributing on behalf of my
      employer Amazon Web Services, Inc.
      4a89f79b
    • Robin Newhouse's avatar
      GitLab CI Upgrade CentOS 8 to CentOS 9 build · 008bddaa
      Robin Newhouse authored
      > After May 31, 2024, CentOS Stream 8 will be archived and no further
      updates will be provided. [1]
      
      CentOS Stream 8 is now EOL and should be updated to using CentOS Stream
      9 for compatibility testing in GitLab CI.
      
      [1] https://blog.centos.org/2023/04/end-dates-are-coming-for-centos-stream-8-and-centos-linux-7/
      https://www.centos.org/centos-linux-eol/
      
      All new code of the whole pull request, including one or several files
      that are either new files or modified ones, are contributed under the
      BSD-new license. I am contributing on behalf of my employer Amazon Web
      Services.
      008bddaa
    • Oleksandr Byelkin's avatar
      new CC version · e7c2e25b
      Oleksandr Byelkin authored
      e7c2e25b
    • Oleksandr Byelkin's avatar
      new PCRE2-10.44 · 7478fabc
      Oleksandr Byelkin authored
      7478fabc
    • Alexander Barkov's avatar
      MDEV-28345 ASAN: use-after-poison or unknown-crash in my_strtod_int from... · b777b749
      Alexander Barkov authored
      MDEV-28345 ASAN: use-after-poison or unknown-crash in my_strtod_int from charset_info_st::strntod or test_if_number
      
      This patch fixes two problems:
      
      - The code inside my_strtod_int() in strings/dtoa.c could test the byte
        behind the end of the string when processing the mantissa.
        Rewriting the code to avoid this.
      
      - The code in test_if_number() in sql/sql_analyse.cc called my_atof()
        which is unsafe and makes the called my_strtod_int() look behind
        the end of the string if the input string is not 0-terminated.
        Fixing test_if_number() to use my_strtod() instead, passing the correct
        end pointer.
      b777b749
    • Sutou Kouhei's avatar
      MDEV-21166 Add Mroonga initialized check to Mroonga UDFs · 383d53ed
      Sutou Kouhei authored
      Mroonga UDFs can't be used without loading Mroonga.
      383d53ed
    • PinkFreud's avatar
      MDEV-34604 mytop - fix specifying filters in .mytop · 49dff5a4
      PinkFreud authored
      Specifying filters (filter_status, filter_user, etc) in the mytop config
      previously wouldn't work, because any filter specified here was added to
      the config hash as a literal string.
      
      This change fixes that - if filter_* is defined in the config and matches
      an existing filter_* key, then run the value through StringOrRegex() and
      assign to the config hash.
      49dff5a4
    • Yuchen Pei's avatar
      MDEV-30408 Reset explicit_limit in exists2in · 03a35037
      Yuchen Pei authored
      Item_exists_subselect::fix_length_and_dec() sets explicit_limit to 1.
      In the exists2in transformation it resets select_limit to NULL. For
      consistency we should reset explicity_limit too.
      
      This fixes a bug where spider table returns wrong results for queries
      that gets through exists2in transformation when semijoin is off.
      03a35037
    • Yuchen Pei's avatar
      MDEV-32627 [fixup] Spider: Fix conn key length · 8416fd32
      Yuchen Pei authored
      To avoid off-by-one in spider_get_share.
      
      And document conn key and conn key length.
      8416fd32
    • Yuchen Pei's avatar
      MDEV-32627 Distinguish between absence of a keyword and empty value for the keyword · 20d99f3f
      Yuchen Pei authored
      Distinguish them in two place:
      
      when constructing connection key in create_conn_key and
      spider_create_conn for both ordinary queries and spider_direct_sql
      
      For spider_create_conn and spider_udf_direct_sql_create_conn, the
      created conn gets assigned a field from the source object if and only
      if source->field is non-null.
      
      For spider_create_conn_keys and spider_udf_direct_sql_create_conn_key,
      we update the encoding so that absence of keyword and keyword with an
      empty value result in different keys. More specifically, if the i-th
      keyword has a value, the corresponding part in the conn key begins
      with the char \i, followed by the possibly empty value and ends with a
      \0. If the i-th keyword is not specified, then it does not get a
      mention in the conn key.
      
      As part of this change, we also update table param / option parsing to
      recognise a singleton empty string when creating an string list,
      instead of writing it off as NULL.
      20d99f3f
    • Yuchen Pei's avatar
      MDEV-32627 Spider: add tests for connection param overriding · 6f3baec4
      Yuchen Pei authored
      When connection parameters are specified in COMMENT, CONNECTION and
      SERVER, they are overriden in the following order:
      
      COMMENT > CONNECTION > SERVER
      6f3baec4
  2. 16 Jul, 2024 10 commits
    • Tuukka Pasanen's avatar
      MDEV-33837: Workaround chown warnings · 62dfd0c0
      Tuukka Pasanen authored
      Blindly recursive chown is not way to do it.
      This Workaround is not much better than just chown -R but
      there is small adjustment just chown MariaDB statedir and logdir
      then with find only chown those files that are not correctly
      owned.
      
      Fixes lintian warnings:
       * W: mariadb-server: recursive-privilege-change "chown -R" [postinst:*]
       * W: mariadb-server: recursive-privilege-change "chown -R" [postinst:*]
      62dfd0c0
    • Oleg Smirnov's avatar
      MDEV-33010 Crash when pushing condition with CHARSET()/COERCIBILITY() into derived table · 972879f4
      Oleg Smirnov authored
      Based on the current logic, objects of classes Item_func_charset and
      Item_func_coercibility (responsible for CHARSET() and COERCIBILITY()
      functions) are always considered constant.
      However, SQL syntax allows their use in a non-constant manner, such as
      CHARSET(t1.a), COERCIBILITY(t1.a).
      
      In these cases, the `used_tables()` parameter corresponds to table names
      in the function parameters, creating an inconsistency: the item is marked
      as constant but accesses tables. This leads to crashes when
      conditions with CHARSET()/COERCIBILITY() are pushed into derived tables.
      
      This commit addresses the issue by setting `used_tables()` to 0 for
      `Item_func_charset` and `Item_func_coercibility`. Additionally, the items
      now store the return values during the preparation phase and return
      them during the execution phase. This ensures that the items do not call
      its arguments methods during the execution and are truly constant.
      
      Reviewer: Alexander Barkov <bar@mariadb.com>
      972879f4
    • Yuchen Pei's avatar
      MDEV-34421 Check the SQL command when resolving storage engine · 384ec03e
      Yuchen Pei authored
      ENGINE_SUBSTITUTION only applies to CREATE TABLE and ALTER TABLE, and
      Storage_engine_name::resolve_storage_engine_with_error() could be
      called when executing any sql command.
      384ec03e
    • Yuchen Pei's avatar
      MDEV-34541 Clean up spider self reference check · 132270d3
      Yuchen Pei authored
      SPIDER_CONN::loop_check_meraged_first is useless, because all
      SPIDER_CONN_LOOP_CHECKs are in SPIDER_CONN::loop_check_queue, which in
      spider_db_conn::fin_loop_check() is iterated over.
      
      This fixes the use-after-free issue when there are three spider tables
      sharing the same remote, and their corresponding
      SPIDER_CONN_LOOP_CHECKs getting merged in
      spider_conn_queue_and_merge_loop_check()
      
      This also fixes MDEV-34555
      132270d3
    • Yuchen Pei's avatar
      MDEV-29962 Spider: creates connections if needed before lock_tables() · 0bb98628
      Yuchen Pei authored
      Same cause as MDEV-31996. This prevents reuse of conns freed
      previously, e.g. during the commit of previous statement.
      
      It does not occur in 10.4 because of the commit for MDEV-19002 removed
      the call to spider_check_trx_and_get_conn().
      0bb98628
    • Yuchen Pei's avatar
      MDEV-27902 Spider check trx and get conn in rnd_next() · 696c2497
      Yuchen Pei authored
      This allows creation of SPIDER_CONN on demand, if the previous one was
      freed.
      
      Also, the first_link_idx's are reset during
      spider_check_trx_and_get_conn(), which in the case of remote HANDLER
      commands, might not match the link to use correct first_link_idx for
      remote HANDLER statement that was later set in
      ha_spider::rnd_handler_init() (causing testing regressions in the
      spider/handler suite). Therefore we fix the first_link_idx there.
      696c2497
    • Yuchen Pei's avatar
    • Yuchen Pei's avatar
      MDEV-32492 Delete and remove trx_ha on spider share mismatch · 14c40509
      Yuchen Pei authored
      A SPIDER_TRX_HA associated with a SPIDER_TRX could have longer
      lifetime than its associated SPIDER_SHARE. And it is identified with
      the associated table name. When the SPIDER_SHARE no longer valid, e.g.
      when the associated spider table has been dropped and recreated, the
      SPIDER_TRX_HA should be reset too.
      
      Since spider could create a new SPIDER_SHARE with the exact same
      address of a freed SPIDER_SHARE, we try to mark all SPIDER_TRX_HAs
      associated with a SPIDER_SHARE invalid when the SPIDER_SHARE is about
      to be freed.
      14c40509
    • Yuchen Pei's avatar
      MDEV-32492 MDEV-29676 Spider: some code documentation and cleanup · 653050ac
      Yuchen Pei authored
      Some documentations come from commit for MDEV-29676, and the rest are
      mainly related to SPIDER_TRX_HA.
      
      Removed SPIDER_TRX_HA::trx as it is unused.
      653050ac
    • Vladislav Vaintroub's avatar
      MDEV-33265 mariadb-secure-installation fails with --defaults-group-suffix · 26192a46
      Vladislav Vaintroub authored
      Reason for the error is that --defaults-group-suffix is passed twice
      to the command line client, and option parser is not prepared for this.
      
      Double occurence of comes from 2 invocations of parse_arguments(), which
      appends unparsed arguments each time it is called.
      
      Fixed by treating --defaults-group-suffix like other "--defaults-"
      (--defaults-file/--defaults-extra-file). it will be parsed, and thus
      passed only once to the command line client.
      26192a46
  3. 15 Jul, 2024 5 commits
    • Oleksandr Byelkin's avatar
      New wolfssl v5.7.2-stable · 3a38a7a4
      Oleksandr Byelkin authored
      3a38a7a4
    • Daniel Black's avatar
      MDEV-34099: AddressSanitizer running out of memory regardless of stack_thread size · cf1c381b
      Daniel Black authored
      Address Sanitizer's know how to detect stack overrun, so there's
      no point in us doing it.
      
      As evidenced by perfschema tests where signficant test failures
      because this function failed under ASAN (MDEV-33210).
      
      Also, so since clang-16, we cannot assume much about how local
      variables are allocated on the stack (MDEV-31605).
      
      Disabling check idea thanks to Sanja.
      cf1c381b
    • Oleg Smirnov's avatar
      MDEV-34490 get_copy() and build_clone() may return an instance of an ancestor... · 405613eb
      Oleg Smirnov authored
      MDEV-34490 get_copy() and build_clone() may return an instance of an ancestor class instead of a copy/clone
      
      The `Item` class methods `get_copy()`, `build_clone()`, and `clone_item()`
      face an issue where they may be defined in a descendant class
      (e.g., `Item_func`) but not in a further descendant (e.g., `Item_func_child`).
      This can lead to scenarios where `build_clone()`, when operating on an
      instance of `Item_func_child` with a pointer to the base class (`Item`),
      returns an instance of `Item_func` instead of `Item_func_child`.
      
      Since this limitation cannot be resolved at compile time, this commit
      introduces runtime type checks for the copy/clone operations.
      A debug assertion will now trigger in case of a type mismatch.
      
      `get_copy()`, `build_clone()`, and `clone_item()` are no more virtual,
      but virtual `do_get_copy()`, `do_build_clone()`, and `do_clone_item()`
      are added to the protected section of the class `Item`.
      
      Additionally, const qualifiers have been added to certain methods
      to enhance code reliability.
      
      Reviewer: Oleksandr Byelkin <sanja@mariadb.com>
      405613eb
    • Thirunarayanan Balathandayuthapani's avatar
      MDEV-34474 InnoDB: Failing assertion: stat_n_leaf_pages > 0 in... · 46d95ae2
      Thirunarayanan Balathandayuthapani authored
      MDEV-34474 InnoDB: Failing assertion: stat_n_leaf_pages > 0 in ha_innobase::estimate_rows_upper_bound
      
      - Fixing the compilation issue.
      46d95ae2
    • Yuchen Pei's avatar
      MDEV-34449 Default ha_spider::bulk_size to 0. · 3b956f83
      Yuchen Pei authored
      This fixes a valgrind failure where the bulk_size is used before
      initialised in ha_spider::end_bulk_insert().
      3b956f83
  4. 14 Jul, 2024 1 commit
  5. 12 Jul, 2024 3 commits
    • Thirunarayanan Balathandayuthapani's avatar
      MDEV-34542 Assertion `lock_trx_has_sys_table_locks(trx) == __null' failed in... · 00d2c7f7
      Thirunarayanan Balathandayuthapani authored
      MDEV-34542  Assertion `lock_trx_has_sys_table_locks(trx) == __null' failed in void row_mysql_unfreeze_data_dictionary(trx_t*)
      
      - During XA PREPARE, InnoDB releases the non-exclusive locks.
      But it fails to remove the non-exclusive table lock from the
      transaction table locks. In the mean time, main thread evicts
      the table from the LRU cache. While rollbacking the XA transaction,
      InnoDB iterates through the table locks to check whether it
      holds lock on any system tables and wrongly assumes the
      evicted table as system table since the table id is 0
      
      Fix:
      ===
      During XA PREPARE, remove the table locks of the transaction while
      releasing the non-exclusive locks.
      00d2c7f7
    • Thirunarayanan Balathandayuthapani's avatar
      MDEV-34543 Shutdown hangs while freeing the asynchronous i/o slots · 3662f8ca
      Thirunarayanan Balathandayuthapani authored
      Problem:
      ========
      - During shutdown, InnoDB tries to free the asynchronous
      I/O slots and hangs. The reason is that InnoDB disables
      asynchronous I/O before waiting for pending
      asynchronous I/O to finish.
      
      buf_load(): InnoDB aborts the buffer pool load due to
      user requested shutdown and doesn't wait for the asynchronous
      read to get completed. This could lead to debug assertion
      in buf_flush_buffer_pool() during shutdown
      
      Fix:
      ===
      os_aio_free(): Should wait all read_slots and write_slots
      to finish before disabling the aio.
      
      buf_load(): Should wait for pending read request to complete
      even though it was aborted.
      3662f8ca
    • Daniel Black's avatar
      MDEV-34568 rpl.rpl_mdev12179 - correct for Windows · e8bcc4e4
      Daniel Black authored
      Simplify in an attempt to avoid:
      
      mysqltest: At line 275: File already exist: on the write_file
      lines.
      
      Using write_line as that's what a lot of other tests
      do for writing small bits to a expect file.
      
      Review thanks Valdislav Vaintroub
      e8bcc4e4
  6. 11 Jul, 2024 3 commits
    • Oleg Smirnov's avatar
      MDEV-34041 Display additional information for materialized subqueries in... · aae3233c
      Oleg Smirnov authored
      MDEV-34041 Display additional information for materialized subqueries in EXPLAIN/ANALYZE FORMAT=JSON
      
      This commits adds the "materialization" block to the output of
      EXPLAIN/ANALYZE FORMAT=JSON when materialized subqueries are involved
      into processing. In the case of ANALYZE additional runtime information
      is displayed, such as:
        - chosen strategy of materialization
        - number of partial match/index lookup loops
        - sizes of partial match buffers
      aae3233c
    • Galina Shalygina's avatar
      MDEV-32608: Expression with constant subquery causes a crash in pushdown · a5e4c349
      Galina Shalygina authored
      from HAVING
      
      The bug is caused by refixing of the constant subquery in pushdown from
      HAVING into WHERE optimization.
      
      Similarly to MDEV-29363 in the problematic query two references of the
      constant subquery are used. After the pushdown one of the references of the
      subquery is pushed into WHERE-clause and the second one remains as the part
      of the HAVING-clause.
      Before the represented fix, the constant subquery reference that was going to
      be pushed into WHERE was cleaned up and fixed. That caused the changes of
      the subquery itself and, therefore, changes for the second reference that
      remained in HAVING. These changes caused a crash.
      
      To fix this problem all constant objects that are going to be pushed into
      WHERE should be marked with an IMMUTABLE_FL flag. Objects marked with this
      flag are not cleaned up or fixed in the pushdown optimization.
      
      Approved by Igor Babaev <igor@mariadb.com>
      a5e4c349
    • Daniel Black's avatar
      mtr: remove not_valgrind_build · eaf7c0cb
      Daniel Black authored
      The version test on not_valgrind_build.inc was
      broken as in BB the sp-no-valgrind.test was
      executed.
      
      The implication that it wouldn't work on ASAN
      was also incorrect as ASAN tests show it running
      fine there.
      
      Correct sp-no-valgrind.test for not_valgrind.inc.
      eaf7c0cb
  7. 10 Jul, 2024 3 commits
    • Brandon Nesterenko's avatar
      MDEV-33921: Replication breaks when filtering two-phase XA transactions · ea986950
      Brandon Nesterenko authored
      There are two problems.
      
      First, replication fails when XA transactions are used where the
      slave has replicate_do_db set and the client has touched a different
      database when running DML such as inserts. This is because XA
      commands are not treated as keywords, and are thereby not exempt
      from the replication filter. The effect of this is that during an XA
      transaction, if its logged “use db” from the master is filtered out
      by the replication filter, then XA END will be ignored, yet its
      corresponding XA PREPARE will be executed in an invalid state,
      thereby breaking replication.
      
      Second, if the slave replicates an XA transaction which results in
      an empty transaction, the XA START through XA PREPARE first phase of
      the transaction won’t be binlogged, yet the XA COMMIT will be
      binlogged. This will break replication in chain configurations.
      
      The first problem is fixed by treating XA commands in
      Query_log_event as keywords, thus allowing them to bypass the
      replication filter. Note that Query_log_event::is_trans_keyword() is
      changed to accept a new parameter to define its mode, to either
      check for XA commands or regular transaction commands, but not both.
      In addition, mysqlbinlog is adapted to use this mode so its
      --database filter does not remove XA commands from its output.
      
      The second problem fixed by overwriting the XA state in the XID
      cache to be XA_ROLLBACK_ONLY, so at commit time, the server knows to
      rollback the transaction and skip its binlogging. If the xid cache
      is cleared before an XA transaction receives its completion command
      (e.g. on server shutdown), then before reporting ER_XAER_NOTA when
      the completion command is executed, the filter is first checked if
      the database is ignored, and if so, the error is ignored.
      
      Reviewed By:
      ============
      Kristian Nielsen <knielsen@knielsen-hq.org>
      Andrei Elkin <andrei.elkin@mariadb.com>
      ea986950
    • Vladislav Vaintroub's avatar
      MDEV-34546 Windows - no error log entries after startup in XAMPP · 9fdc0e54
      Vladislav Vaintroub authored
      The server does not log errors after startup when it is started without the
      --console parameter and not as a service. This issue arises due to an
      undocumented behavior of FreeConsole() in Windows when only a single
      process (mariadbd/mysqld) is attached to it, causing the window to close.
      
      In this case stderr is redirected to a file before FreeConsole()
      is called. Procmon shows FreeConsole closing file handle
      subsequent writes to stderr fail with ERROR_INVALID_HANDLE because
      WriteFile() cannot operate on the closed handle. This results in losing
      all messages after startup, including warnings, errors, notes, and
      crash reports.
      
      Additionally, some users reported stderr being redirected to
      multi-master.info and failing at startup, but this could not be reproduced
      here.
      
      The workaround involves calling FreeConsole() right before the redirection of
      stdout/stderr. This fix has been tested with XAMPP and via cmd.exe using
      "start mysqld". Automated testing using MTR is challenging for this case.
      
      The fix is only applicable to version 10.5. In later versions, the
      FreeConsole() call has been removed.
      9fdc0e54
    • Thirunarayanan Balathandayuthapani's avatar
      MDEV-34474 InnoDB: Failing assertion: stat_n_leaf_pages > 0 in... · 60125a77
      Thirunarayanan Balathandayuthapani authored
      MDEV-34474 InnoDB: Failing assertion: stat_n_leaf_pages > 0 in ha_innobase::estimate_rows_upper_bound
      
      - Column stat_value and sample_size in mysql.innodb_index_stats
      table is declared as BIGINT UNSIGNED without any check constraint.
      user manually updates the value of stat_value and sample_size
      to zero. InnoDB aborts the server while reading the statistics
      information because InnoDB expects at least one leaf
      page to exist for the index.
      
      - To fix this issue, InnoDB should interpret the value of
      stat_n_leaf_pages, stat_index_size in innodb_index_stats
      stat_clustered_index_size, stat_sum_of_other_index_sizes
      in innodb_table_stats as valid one even though user
      mentioned it as 0.
      60125a77
  8. 08 Jul, 2024 3 commits