Commit 236f825e authored by Sachin Agarwal's avatar Sachin Agarwal Committed by Marko Mäkelä

Bug #31576731 INNODB_FT_TOTAL_CACHE_SIZE NOT CAPPED WHEN SET TO 32000000

Problem:
Server throws OOM error when we execute twitter load with SELECTs
for UPDATE + UPDATES, and SELECT queries on tables with full-text
index.
FTS cache->total_memory store count of total memory allocated to FTS
cache (for all fulltext indexes on a table).
For each word in fts cache, we store doc-id & word position in a
node->ilist.
we increment cache->total_memory with size of doc-id & word position
whereas we allocate ilist in chuck of 16, 32 ,64 bytes or 1.2 times
of last size.
When we wil insert huge amount of data into the FTS aux index tables
then collectively these small chucks for each token become huge
unaccounted memory allocated for FTS cache.

Fix:
Incremented cache->total_memory by size of chunk allocated to
node->ilist.

RB: 25286
Reviewed by : Rahul Agarkar <rahul.agarkar@oracle.com>

mysql/mysql-server@7ab5707f1c4482ed050ed9fa739e9ec0e2fc0ffa
parent c4295b9b
/*****************************************************************************
Copyright (c) 2011, 2018, Oracle and/or its affiliates. All Rights Reserved.
Copyright (c) 2016, 2020, MariaDB Corporation.
Copyright (c) 2011, 2021, Oracle and/or its affiliates.
Copyright (c) 2016, 2021, MariaDB Corporation.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
......@@ -1299,6 +1299,9 @@ fts_cache_node_add_positions(
ptr = ilist + node->ilist_size;
node->ilist_size_alloc = new_size;
if (cache) {
cache->total_size += new_size;
}
}
ptr_start = ptr;
......@@ -1325,6 +1328,9 @@ fts_cache_node_add_positions(
if (node->ilist_size > 0) {
memcpy(ilist, node->ilist, node->ilist_size);
ut_free(node->ilist);
if (cache) {
cache->total_size -= node->ilist_size;
}
}
node->ilist = ilist;
......@@ -1332,10 +1338,6 @@ fts_cache_node_add_positions(
node->ilist_size += enc_len;
if (cache) {
cache->total_size += enc_len;
}
if (node->first_doc_id == FTS_NULL_DOC_ID) {
node->first_doc_id = doc_id;
}
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment