- 05 Oct, 2023 1 commit
-
-
Yuchen Pei authored
Spider GBH's query rewrite of table joins is overly complex and error-prone. We replace it with something closer to what dbug_print() (more specifically, print_join()) does, but catered to spider. More specifically, we replace the body of spider_db_mbase_util::append_from_and_tables() with a call to spider_db_mbase_util::append_join(), and remove downstream append_X functions. We make it handle const tables by rewriting them as (select 1). This fixes the main issue in MDEV-26247. We also ban semijoin from spider gbh, which fixes MDEV-31645 and MDEV-30392, as semi-join is an "internal" join, and "semi join" does not parse, and it is different from "join" in that it deduplicates the right hand side Not all queries passed to a group by handler are valid (MDEV-32273), for example, a join on expr may refer outer fields not in the current context. We detect this during the handler creation when walking the join. See also gbh_outer_fields_in_join.test. It also skips eliminated tables, which fixes MDEV-26193.
-
- 04 Oct, 2023 8 commits
-
-
Yuchen Pei authored
-
Yuchen Pei authored
Spider gbh query rewrite should get table for fields in a simple way. Add a method spider_fields::find_table that searches its table holders to find table for a given field. This way we will be able to get rid of the first pass during the gbh creation where field_chains and field_holders are created. We also check that the field belongs to a spider table while walking through the query, so we could remove all_query_fields_are_query_table_members(). However, this requires an earlier creation of the table_holder so that tables are added before checking. We do that, and in doing so, also decouple table_holder and spider_fields Remove unused methods and fields. Add comments.
-
Yuchen Pei authored
Two methods from spider_fields. There are probably more of these conn_holder related methods that can be removed reappend_tables_part() reappend_tables()
-
Yuchen Pei authored
-
Yuchen Pei authored
Removed some redundant hint related string literals from spd_db_conn.cc Clean up SPIDER_PARAM_*_[CHAR]LEN[S] Adding tests covering monitoring_kind=2. What it does is that it reads from mysql.spider_link_mon_servers with matching db_name, table_name, link_id, and does not do anything about that... How monitoring_* can be useful: in the deprecated spider high availability feature, when one remote fails, spider will try another remote, which apparently makes use of these table parameters. A test covering the query_cache_sync table param. Some further tests on some spider table params. Wrapper should be case insensitive. Code documentation on spider priority binary tree. Add an assertion that static_key_cardinality is always -1. All tests pass still
-
Yuchen Pei authored
This helps eliminate "server exists" failures Also, spider/bugfix.mdev_29676, when enabled after MDEV-29525 is pushed will fail because we have not --recorded the result. But the failure will only emerge when working on MDEV-31138 where we manually re-enable this test, so let's worry about that then.
-
Yuchen Pei authored
The direct aggregate mechanism sems to be only intended to work when otherwise a full table scan query will be executed from the spider node and the aggregation done at the spider node too. Typically this happens in sub_select(). In the test spider.direct_aggregate_part direct aggregate allows to send COUNT statements directly to the data nodes and adds up the results at the spider node, instead of iterating over the rows one by one at the spider node. By contrast, the group by handler (GBH) typically sends aggregated queries directly to data nodes, in which case DA does not improve the situation here. That is why we should fix it by disabling DA when GBH is used. There are other reasons supporting this change. First, the creation of GBH results in a call to change_to_use_tmp_fields() (as opposed to setup_copy_fields()) which causes the spider DA function spider_db_fetch_for_item_sum_funcs() to work on wrong items. Second, the spider DA function only calls direct_add() on the items, and the follow-up add() needs to be called by the sql layer code. In do_select(), after executing the query with the GBH, it seems that the required add() would not necessarily be called. Disabling DA when GBH is used does fix the bug. There are a few other things included in this commit to improve the situation with spider DA: 1. Add a session variable that allows user to disable DA completely, this will help as a temporary measure if/when further bugs with DA emerge. 2. Move the increment of direct_aggregate_count to the spider DA function. Currently this is done in rather bizarre and random locations. 3. Fix the spider_db_mbase_row creation so that the last of its row field (sentinel) is NULL. The code is already doing a null check, but somehow the sentinel field is on an invalid address, causing the segfaults. With a correct implementation of the row creation, we can avoid such segfaults.
-
Yuchen Pei authored
This function trivially returns false
-
- 13 Jul, 2023 2 commits
-
-
Yuchen Pei authored
Spider connection string is a comma-separated parameter definitions, where each definition is of the form "<param_title> <param_value>", where <param_value> is quote delimited on both ends, with backslashes acting as an escaping prefix. The code however treated param title the same way as param value when assigning, and have nonsensical fields like delim_title_len and delim_title. We remove these. We also clean up the spider comment connection string parsing, including: - Factoring out some code from the parsing function - Rewriting the struct `st_spider_param_string_parse`, including replacing its messy methods with cleaner ones - And any necessary changes caused by the above changes
-
Yuchen Pei authored
The existing (incorrect) overriding mechanism is: Non-minus-one var value overrides table param overrides default value. Before MDEV-27169, unspecified var value is -1. So if the user sets both the var to be a value other than -1 and the table param, the var value will prevail, which is incorrect. After MDEV-27169, unspecified var value is default value. So if the user does not set the var but sets the table param, the default value will prevail, which is even more incorrect. This patch fixes it so that table param, if specified, always overrides var value, and the latter if not specified or set to -1, falls back to the default value We achieve this by replacing all such overriding in spd_param.cc with macros that override in the correct way, and removing all the "overriding -1" lines involving table params in spider_set_connect_info_default() except for those table params not defined as sysvar/thdvar in spd_params.cc We also introduced macros for non-overriding sysvar and thdvar, so that the code is cleaner and less error-prone In server versions where MDEV-27169 has not been applied, we also backport the patch, that is, replacing -1 default values with real default values In server versions where MDEV-28006 has not been applied, we do the same for udf params
-
- 04 Jul, 2023 12 commits
-
-
Yuchen Pei authored
Fix spider init bugs (MDEV-22979, MDEV-27233, MDEV-28218) while preventing regression on old ones (MDEV-30370, MDEV-29904) Two things are changed: First, Spider initialisation is made fully synchronous, i.e. it no longer happens in a background thread. Adapted from the original fix by nayuta for MDEV-27233. This change itself would cause failure when spider is initialised early, by plugin-load-add, due to dependency on Aria and udf function creation, which are fixed in the second and third parts below. Requires SQL Service, thus porting earlier versions requires MDEV-27595 Second, if spider is initialised before udf_init(), create udf by inserting into `mysql.func`, otherwise do it by `CREATE FUNCTION` as usual. This change may be generalised in MDEV-31401. Also factor out some clean-up queries from deinit_spider.inc for use of spider init tests. A minor caveat is that early spider initialisation will fail if the server is bootstrapped for the first time, due to missing `mysql` database which needs to be created by the bootstrap script.
-
Yuchen Pei authored
Removing procedures that were created and dropped during init. This also fixes a race condition where mtr test with plugin-load-add=ha_spider.so causes post test check to fail as it expects the procedures to still be there.
-
Yuchen Pei authored
There are several plugins in ha_spider: spider, spider_alloc_mem, spider_wrapper_protocols, spider_rewrite etc. INSTALL PLUGIN foo SONAME ha_spider causes all the other ones to be installed by the init queries where foo is any of the plugins. This introduces unnecessary complexiy. For example it reads mysql.plugins to find all other plugins, causing the hack of moving spider plugin init to a separate thread. To install all spider related plugins, install soname ha_spider should be used instead. This also fixes spurious rows in mysql.plugin when installing say only the spider plugin with `plugin-load-add=SPIDER=ha_spider.so`: select * from mysql.plugin; name dl spider_alloc_mem ha_spider.so # should not be here spider_wrapper_protocols ha_spider.so # should not be here Adapted from part of the reverted commit c160a115.
-
Yuchen Pei authored
We introduce simple plugin dependency. A plugin init function may return HA_ERR_RETRY_INIT. If this happens during server startup when the server is trying to initialise all plugins, the failed plugins will be retried, until no more plugins succeed in initialisation or want to be retried. This will fix spider init bugs which is caused in part by its dependency on Aria for initialisation. The reason we need a new return code, instead of treating every failure as a request for retry, is that it may be impossible to clean up after a failed plugin initialisation. Take InnoDB for example, it has a global variable `buf_page_cleaner_is_active`, which may not satisfy an assertion during a second initialisation try, probably because InnoDB does not expect the initialisation to be called twice.
-
Yuchen Pei authored
This fixes mdev_26541.test, and the new clean_up_spider.inc will be useful for other tests where part of deinit_spider does not apply, e.g. those testing spider initialisation only.
-
Yuchen Pei authored
-
Yuchen Pei authored
generated by the language server ccls
-
Yuchen Pei authored
The server needs to have a unique name
-
Yuchen Pei authored
Will re-enable once MDEV-31101 is no longer blocked by MDEV-22979, as the patch for the latter might fix the former.
-
Yuchen Pei authored
spider_db_mbase_util::open_item_func() is a monster function. It is difficult to maintain while it is expected that we need to modify it when a new SQL function or a new func_type is added. We split the function into two distinct functions: one handles the case of str != NULL and the other handles the case of str == NULL. This refactoring was done in a conservative way because we do not have comprehensive tests on the function. It also fixes MDEV-29447 and MDEV-31338 where field items that are arguments of a func item may be used before created / initialised. Note this commit is adapted from a patch by Nayuta for MDEV-26285.
-
Yuchen Pei authored
Extract the indexed string memcopy pattern in spd_trx.cc to a static inline function. Also updated the ubsan check in mdev_26541.test (h/t roel).
-
Yuchen Pei authored
The error code is non-deterministic, presumably due to some race condition from the SLEEP statement above. The correct error should be 12701 ER_SPIDER_REMOTE_SERVER_GONE_AWAY_NUM as it is the last failure. Nevertheless, this contrived test is needed to cover the error reporting when setting lock wait timeout, until we find a better test case and/or fixing the non-deterministic error reporting (MDEV-31586)
-
- 02 Jul, 2023 2 commits
-
-
Trevor Gross authored
Allocate a temporary buffer instead of using the same buffer in some cases, and add assertions to verify the buffers do not overlap. See [1] for reasonsing. [1] https://github.com/MariaDB/server/pull/2438#discussion_r1137403645Signed-off-by:
Trevor Gross <tmgross@umich.edu>
-
Trevor Gross authored
This patch ensures that all direct and indirect calls to encryption_crypt provide a `dlen` value correctly initialized to the destination buffer length, allowing encryption plugins to verify available space. It also adds assertions to verify related invariants. Signed-off-by:
Trevor Gross <tmgross@umich.edu>
-
- 28 Jun, 2023 2 commits
-
-
Marko Mäkelä authored
-
Yuchen Pei authored
When setting the server lockwait timeout, spider should do some basic checks first, like whether the remote server is still reachable. So instead of directly calling spider_db_mbase::exec_query(), it should call spider_db_query(). The reset of the lock wait timeout does not need to do such checks, because they happen after the successfully setting the lock wait timeout, implying the checks have been passed already.
-
- 27 Jun, 2023 5 commits
-
-
Marko Mäkelä authored
-
Marko Mäkelä authored
-
Marko Mäkelä authored
-
Marko Mäkelä authored
-
Marko Mäkelä authored
recv_sys_t::parse(): For undo tablespace truncation mini-transactions, remember the start_lsn instead of the end LSN. This is what we expect after commit 461402a5 (MDEV-30479).
-
- 26 Jun, 2023 3 commits
-
-
Marko Mäkelä authored
-
Marko Mäkelä authored
-
Rucha Deodhar authored
mysql.proc. The table is probably corrupted" Analysis: When mysql_upgrade runs statements for upgrade, characterset is converted to utf8mb4 because server starts with old_mode that interprets utf8 to utf8mb4, but mysql.proc table has "utf8mb3" as hardcoded, so it crashes with corrupted table. Fix: Changed Table_check_intact::check() definition to allow both utf8mb3 and utf8mb4 by checking prefix and changing the upgrade scripts to explicitly use utf8mb3
-
- 25 Jun, 2023 5 commits
-
-
Monty authored
-
Monty authored
This is mainly done to be able to push to 11.0 The test is moved in 10.6 to another file, which will add back the test.
-
Michael Widenius authored
MDEV-29253 Detect incompatible MySQL partition scheme and either convert them or report to user and in error log. This task is about converting in place MySQL 5.6 and 5.7 partition tables to MariaDB as part of mariadb-upgrade. - Update TABLE_SHARE::init_from_binary_frm_image() to be able to read MySQL frm files with partitions. - Create .par file, if it do not exists, on open of partitioned table. Executing mariadb-upgrade will create all the missing .par files. The MySQL .frm file will be changed to MariaDB format after next ALTER TABLE. Other changes: - If we are using stored mysql_version to distingush between MySQL and MariaDB .frm file information, do not upgrade mysql_version in the .frm file as part of CHECK TABLE .. FOR UPGRADE as this would cause problems next time we parse the .frm file.
-
Monty authored
- Moved view checks after privilege tables are fixed. This is to avoid warnings about wrongly defined mysql.proc when checking views. - Don't use stat tables before they have been fixed. - Don't run mysql_fix_view() if 'FOR MYSQL' is used if the view is already a MariaDB view. - Added 'FOR UPGRADE' as an option for 'REPAIR VIEW' to be able to detect if the REPAIR command comes from mariadb_upgrade. In this case we get a warning, instead of an error, if a definer of a view does not exists.
-
Vicentiu Ciorbaru authored
In this case, one would end up calling my_error(ER_BAD_DB_ERROR, ...) with a null ptr for DB, which caused a NULL ptr dereference.
-