1. 17 May, 2021 6 commits
    • Julius Goryavsky's avatar
      MDEV-25693: SST failed due to incorrect connection address · 74091762
      Julius Goryavsky authored
      Fixed bugs caused by inaccuracies in automatic merging
      from other branches:
      
      1) Authentication information is not removed from the connection
         address, which causes some tests to fail;
      2) wsrep_debug=on should be replaced with wsrep_debug=1;
      3) Added missing "connection" lines to test result file;
      4) Some tests have been corrected for Galera 4.x (10.4+).
      74091762
    • Julius Goryavsky's avatar
      2947cf64
    • Julius Goryavsky's avatar
      MDEV-25669: SST scripts should check all server groups in config files · 527675d5
      Julius Goryavsky authored
      1) This commit implements reading all sections from configuration
      files while looking for the current value of any server variable,
      which were previously only read from the [mysqld.suffix] group and
      from [mysqld], but not from other groups such as [mariadb.suffix],
      [mariadb] or, for example, [server].
      
      2) This commit also fixes misrecognition of some parameters when
      parsing a command line containing a special marker for the end
      of the list of options ("--") or when short option names (such
      as "-s", "-a" and "-h arg") chained together (like a "-sah arg").
      Such parameters can be passed to the SST script in the list of
      arguments after "--mysqld-args" if the server is started with a
      complex set of options - this was revealed during manual testing
      of changes to read configuration files.
      
      3) The server-side preparation code for the "--mysqld-args"
      option list has also been simplified to make it easier to change
      in the future (if needed), and has been improved to properly
      handle the special backquote ("`") character in the argument
      values.
      527675d5
    • Julius Goryavsky's avatar
    • Julius Goryavsky's avatar
      MDEV-23580: WSREP_SST: [ERROR] rsync daemon port has been taken · d57e60d7
      Julius Goryavsky authored
      This commit contains a large set of further bug fixes and
      improvements to SST scripts for Galera, continuing the work
      that was started in MDEV-24962 to make SST scripts work smoothly
      in different network configurations (especially using ipv6) and
      with different environment settings:
      
       1) The ipv6 addresses were incorrectly handled in the SST script
          for rsync (incorrect address substitution for establishing a
          connection, incorrect address substitution for bind, and so on);
       2) Checking the locality of the ip-address in SST scripts did not
          support ipv6 addresses (such as "[::1]"), which were falsely
          identified as non-local ip, which further did not allow running
          two SSTs on different local addresses on the same machine.
          On the other hand, this bug masked some other errors (related
          to handling ipv6 addresses);
       3) The code for checking the locality of the ip address was different
          in the SST scripts for rsync and for mysqldump, with individual
          flaws. This code is now made common and moved to wsrep_sst_common;
       4) Waiting for the start of the transport channel (socat, nc, rsync,
          stunnel) in the wait_for_listen() and check_pid_and_port() functions
          did not process ipv6 addresses correctly in all cases (not for all
          branches);
       5) Waiting for the start of the transport channel (socat, nc, rsync,
          stunnel) in the wait_for_listen() and check_pid_and_port() functions
          for some code branches could give a false positive result due to
          the textual match of prefixes in the port number and/or PID of
          the process;
       6) Waiting for the start of the transport channel (socat, nc, rsync,
          stunnel) was supported through different utilities in SST scripts
          for mariabackup and for rsync, and with various minor flaws in
          the code. Now the code is still different in these scripts, but
          it supports a common set of utilities (lsof, ss, sockstat) and
          is synchronized across patterns that used to check the output
          of  these utilities;
       7) In SST via mariabackup, the signal about readiness to receive data
          is sometimes sent too early - immediately after listen(), and not
          after accept() (which are called by socat or netcat utility).
       8) Checking availability of the some options of some utilities was
          done using the grep pattern, which easily gives false positives;
       9) Common name (CN) for local addresses, if not explicitly specified,
          is now always replaced to "localhost" to avoid the need to generate
          many separate certificates for local addresses of one machine and
          not to depend on which the local address is currently used in test
          (ipv4 or ipv6, etc.);
      10) In tests galera_sst_mariabackup_encrypt_with_key_server and
          galera_sst_rsync_encrypt_with_key_server the correct certificate
          is selected to avoid commonname (CN) mismatch problems;
      11) Further refactoring to protect against spaces in file names.
      12) Further general refactoring to eliminate bash-specific constructs
          or to improve code readability;
      13) The code for setting options for the nc (netcat) utility was
          different in different scripts for SST - now it is made identical.
      14) Fixed long-time broken encryption via xbcrypt in combination with
          mariabackup and added support for key-based encryption via openssl
          utility, which is now enabled by default for encrypt=1 mode (this
          default mode can be changed using a new configuration file option
          "encypt-format=openssl|xbcrypt", which can be placed in the [mysqld],
          [sst] or in the [xtrabackup] section) - this change will allow us
          to use and to test the encypt=1 encryption without installing
          non-standard third-party utilities.
      d57e60d7
    • Alexey Botchkov's avatar
      MDEV-25690 Plugins can't execute sql statements with the Galera enabled. · c22d567e
      Alexey Botchkov authored
      For the plugin thread during the server start the wsrep_on should be 0.
      c22d567e
  2. 11 May, 2021 2 commits
  3. 10 May, 2021 5 commits
  4. 09 May, 2021 1 commit
  5. 08 May, 2021 5 commits
  6. 07 May, 2021 7 commits
  7. 06 May, 2021 1 commit
    • Alexey Yurchenko's avatar
      MDEV-25418: Improve mariabackup SST script compliance with native MariaDB SSL practices · 54d7ba96
      Alexey Yurchenko authored
      and configuration.
      
      1. Pass joiner's authentication information to donor together with address
         in State Transfer Request. This allows joiner to authenticate donor on
         connection. Previously joiner would accept data from anywhere.
      
      2. Deprecate custom SSL configuration variables tca, tcert and tkey in favor
         of more familiar ssl-ca, ssl-cert and ssl-key. For backward compatibility
         tca, tcert and tkey are still supported.
      
      3. Allow falling back to server-wide SSL configuration in [mysqld] if no SSL
         configuration is found in [sst] section of the config file.
      
      4. Introduce ssl-mode variable in [sst] section that takes standard values
         and has following effects:
          - old-style SSL configuration present in [sst]: no effect
            otherwise:
          - ssl-mode=DISABLED or absent: retains old, backward compatible behavior
            and ignores any other SSL configuration
          - ssl-mode=VERIFY*: verify joiner's certificate and CN on donor,
                              verify donor's secret on joiner
                              (passed to donor via State Transfer Request)
                              BACKWARD INCOMPATIBLE BEHAVIOR
          - anything else enables new SSL configuration convetions but does not
            require verification
      
          ssl-mode should be set to VERIFY only in a fully upgraded cluster.
      
          Examples:
      
          [mysqld]
          ssl-cert=/path/to/cert
          ssl-key=/path/to/key
          ssl-ca=/path/to/ca
      
          [sst]
      
           -- server-wide SSL configuration is ignored, SST does not use SSL
      
          [mysqld]
          ssl-cert=/path/to/cert
          ssl-key=/path/to/key
          ssl-ca=/path/to/ca
      
          [sst]
          ssl-mode=REQUIRED
      
           -- use server-wide SSL configuration for SST but don't attempt to
              verify the peer identity
      
          [sst]
          ssl-cert=/path/to/cert
          ssl-key=/path/to/key
          ssl-ca=/path/to/ca
          ssl-mode=VERIFY_CA
      
           -- use SST-specific SSL configuration for SST and require verification
              on both sides
      Signed-off-by: default avatarJulius Goryavsky <julius.goryavsky@mariadb.com>
      54d7ba96
  8. 05 May, 2021 13 commits