Skip to content

M
MariaDB
Commit 485c9b1f authored by Junqi Xie's avatar Junqi Xie Committed by Andrew Hutchings
Browse files
Options

MDEV-30610 Update RocksDB to v8.1.1 

MariaRocks is currently lagging behind the main branch of the RocksDB engine. This commit brings MariaRocks up to date with the latest release of RocksDB by backporting changes from Facebook’s MyRocks. These changes include API updates, bug fixes, and improvements for compatibility with RocksDB v8.1.1. Some system variables and metadata tables are modified to reflect the internal changes in RocksDB.

Additionally, this commit backports improved and more stable test cases from Facebook’s MyRocks, including tests for the write_unprepared isolation level of RocksDB. It also reverts workarounds for MDEV-29875 and MDEV-31057 and adds support for the latest compilation options.

The default value of the following system variables are changed:
* rocksdb_stats_level: 1 (kExceptHistogramOrTimers)
* rocksdb_wal_recovery_mode: 2 (kPointInTimeRecovery)

The following system variables are added:
* rocksdb_cancel_manual_compactions
* rocksdb_enable_iterate_bounds
* rocksdb_enable_pipelined_write
* rocksdb_enable_remove_orphaned_dropped_cfs
* rocksdb_manual_compaction_bottommost_level
* rocksdb_max_background_compactions
* rocksdb_max_background_flushes
* rocksdb_max_bottom_pri_background_compactions
* rocksdb_skip_locks_if_skip_unique_check
* rocksdb_track_and_verify_wals_in_manifest
* rocksdb_write_batch_flush_threshold

The following system variables are deprecated:
* rocksdb_hash_index_allow_collision
* rocksdb_new_table_reader_for_compaction_inputs

The following dynamic metadata table is added:
* INFORMATION_SCHEMA.ROCKSDB_LIVE_FILES_METADATA

The following status variables are added:
* rocksdb_manual_compactions_cancelled
* rocksdb_manual_compactions_pending

The following status variables are removed:
* rocksdb_block_cache_filter_bytes_evict
* rocksdb_block_cache_index_bytes_evict
* rocksdb_block_cachecompressed_hit
* rocksdb_block_cachecompressed_miss
* rocksdb_no_file_closes
* rocksdb_num_iterators
* rocksdb_number_deletes_filtered
* rocksdb_write_timedout
parent c035f3af
Expand all Hide whitespace changes
Inline Side-by-side
Showing with 11927 additions and 2942 deletions
.gitignore
View file @ 485c9b1f
......@@ -240,8 +240,12 @@ storage/rocksdb/ldb
storage/rocksdb/myrocks_hotbackup
storage/rocksdb/mysql_ldb
storage/rocksdb/mysql_ldb.1
storage/rocksdb/mariadb-ldb
storage/rocksdb/rdb_source_revision.h
storage/rocksdb/sst_dump
storage/rocksdb/mysql_sst_dump
storage/rocksdb/mysql_sst_dump.1
storage/rocksdb/mariadb-sst-dump
strings/conf_to_src
support-files/MySQL-shared-compat.spec
support-files/binary-configure
......@@ -615,7 +619,6 @@ scripts/mariadb-secure-installation
scripts/mariadb-setpermission
sql/mariadbd
sql/mariadb-tzinfo-to-sql
storage/rocksdb/mariadb-ldb
strings/ctype-uca1400data.h
strings/uca-dump
tests/mariadb-client-test
......
cmake/symlinks.cmake
View file @ 485c9b1f
......@@ -33,6 +33,7 @@ REGISTER_SYMLINK("mariadb-secure-installation" "mysql_secure_installation")
REGISTER_SYMLINK("mariadb-setpermission" "mysql_setpermission")
REGISTER_SYMLINK("mariadb-show" "mysqlshow")
REGISTER_SYMLINK("mariadb-slap" "mysqlslap")
REGISTER_SYMLINK("mariadb-sst-dump" "mysql_sst_dump")
REGISTER_SYMLINK("mariadb-test" "mysqltest")
REGISTER_SYMLINK("mariadb-test-embedded" "mysqltest_embedded")
REGISTER_SYMLINK("mariadb-tzinfo-to-sql" "mysql_tzinfo_to_sql")
......
debian/autobake-deb.sh
View file @ 485c9b1f
......@@ -29,10 +29,6 @@ remove_rocksdb_tools()
{
sed '/rocksdb-tools/d' -i debian/control
sed '/sst_dump/d' -i debian/not-installed
if ! grep -q sst_dump debian/mariadb-plugin-rocksdb.install
then
echo "usr/bin/sst_dump" >> debian/mariadb-plugin-rocksdb.install
fi
}
add_lsb_base_depends()
......
debian/mariadb-plugin-rocksdb.install
View file @ 485c9b1f
etc/mysql/mariadb.conf.d/rocksdb.cnf
usr/bin/mariadb-ldb
usr/bin/mariadb-sst-dump
usr/bin/myrocks_hotbackup
usr/bin/mysql_ldb
usr/bin/mysql_sst_dump
usr/lib/mysql/plugin/ha_rocksdb.so
usr/share/man/man1/mariadb-ldb.1
usr/share/man/man1/mariadb-sst-dump.1
usr/share/man/man1/myrocks_hotbackup.1
usr/share/man/man1/mysql_ldb.1
usr/share/man/man1/mysql_sst_dump.1
mysql-test/valgrind.supp
View file @ 485c9b1f
......@@ -1504,6 +1504,33 @@
fun:_ZN7myrocksL27rdb_init_rocksdb_db_optionsEv
}
{
Still reachable for once-per-process initializations
Memcheck:Leak
match-leak-kinds: reachable
...
fun:_ZN10my_rocksdb24CacheEntryStatsCollectorINS_13InternalStats19CacheEntryRoleStatsEE9GetSharedEPNS_5CacheEPNS_11SystemClockEPSt10shared_ptrIS3_E
...
}
{
Still reachable for once-per-process initializations
Memcheck:Leak
match-leak-kinds: reachable
...
fun:_ZN10my_rocksdb12_GLOBAL__N_18Registry8RegisterEPFvRKNS_5SliceEPvENS_14CacheEntryRoleE
...
}
{
Still reachable for once-per-process initializations
Memcheck:Leak
match-leak-kinds: reachable
...
fun:_ZN10my_rocksdb12_GLOBAL__N_111GetRegistryEv
...
}
{
Still reachable for thread local storage initialization (SetHandle)
Memcheck:Leak
......
sql/field.cc
View file @ 485c9b1f
......@@ -8593,7 +8593,7 @@ void Field_blob::store_length(uchar *i_ptr, uint i_packlength, uint32 i_number)
}
uint32 Field_blob::get_length(const uchar *pos, uint packlength_arg) const
uint32 Field_blob::get_length(const uchar *pos, uint packlength_arg)
{
return (uint32)read_lowendian(pos, packlength_arg);
}
......
sql/field.h
View file @ 485c9b1f
......@@ -4540,7 +4540,7 @@ class Field_blob :public Field_longstr {
}
inline uint32 get_length(my_ptrdiff_t row_offset= 0) const
{ return get_length(ptr+row_offset, this->packlength); }
uint32 get_length(const uchar *ptr, uint packlength) const;
static uint32 get_length(const uchar *ptr, uint packlength);
uint32 get_length(const uchar *ptr_arg) const
{ return get_length(ptr_arg, this->packlength); }
inline uchar *get_ptr() const { return get_ptr(ptr); }
......
sql/share/errmsg-utf8.txt
View file @ 485c9b1f
......@@ -11675,10 +11675,10 @@ ER_RDB_TTL_UNSUPPORTED
sw "Uauniaji wa TTL umezimwa kwa sasa wakati jedwali lina PK iliyofichwa."
ER_RDB_TTL_COL_FORMAT
chi "Myrocks中的TTL列(%s)必须是一个无符号的非空64位整数，存在于表内，并具有伴随的TTL持续时间。"
eng "TTL column (%s) in MyRocks must be an unsigned non-null 64-bit integer, exist inside the table, and have an accompanying ttl duration."
spa "La columna TTL (%s) en MyRocks debe de ser un entero sin signo no-null de 64-bit, debe de existir dentro de la tabla y debe de tener una duración ttl acompañante."
sw "Safu wima ya TTL (%s) katika MyRocks lazima iwe nambari kamili ya biti 64 ambayo haijatiwa saini, iwe ndani ya jedwali, na iwe na muda wa ttl unaoandamana."
chi "Myrocks中的TTL列(%s)必须是一个无符号的非空64位整数或非空时间戳，存在于表内，并具有伴随的TTL持续时间。"
eng "TTL column (%s) in MyRocks must be an unsigned non-null 64-bit integer or non-null timestamp, exist inside the table, and have an accompanying ttl duration."
spa "La columna TTL (%s) en MyRocks debe de ser un entero sin signo no-null de 64-bit o una marca de tiempo no-null, debe de existir dentro de la tabla y debe de tener una duración ttl acompañante."
sw "Safu wima ya TTL (%s) katika MyRocks lazima iwe nambari kamili ya biti 64 ambayo haijatiwa saini au muhuri wa muda usio batili, iwe ndani ya jedwali, na iwe na muda wa ttl unaoandamana."
ER_PER_INDEX_CF_DEPRECATED
chi "已弃用每个索引列族选项"
......@@ -11697,6 +11697,7 @@ ER_SK_POPULATE_DURING_ALTER
eng "MyRocks failed populating secondary key during alter."
spa "MyRocks falló al poblar clave secundaria duante el 'alter'."
sw "MyRocks imeshindwa kujaza ufunguo wa sekondari wakati wa kubadilisha."
# MyRocks messages end
ER_SUM_FUNC_WITH_WINDOW_FUNC_AS_ARG
......@@ -12300,3 +12301,9 @@ ER_JSON_INVALID_VALUE_FOR_KEYWORD
ER_JSON_SCHEMA_KEYWORD_UNSUPPORTED
eng "%s keyword is not supported"
sw "%s neno kuu halitumiki"
ER_CF_DROPPED
chi "列族 ('%s') 正在被删除。"
eng "Column family ('%s') is being dropped."
ER_CANT_DROP_CF
chi "不能删除列族 ('%s')，因为它正在使用或不存在。"
eng "Cannot drop Column family ('%s') because it is in use or does not exist."
storage/rocksdb/CMakeLists.txt
View file @ 485c9b1f
......@@ -4,8 +4,6 @@ SET(CPACK_RPM_rocksdb-engine_PACKAGE_SUMMARY "RocksDB storage engine for MariaDB
SET(CPACK_RPM_rocksdb-engine_PACKAGE_DESCRIPTION "The RocksDB storage engine is a high performance storage engine, aimed
at maximising storage efficiency while maintaining InnoDB-like performance." PARENT_SCOPE)
MY_CHECK_AND_SET_COMPILER_FLAG(-Wno-range-loop-construct)
MACRO(SKIP_ROCKSDB_PLUGIN msg)
MESSAGE_ONCE(SKIP_ROCKSDB_PLUGIN "Can't build rocksdb engine - ${msg}")
ADD_FEATURE_INFO(ROCKSDB "OFF" "Storage Engine")
......@@ -58,41 +56,29 @@ IF(MSVC_ARM64)
SKIP_ROCKSDB_PLUGIN("Windows ARM64 not supported")
ENDIF()
#
# Also, disable on ARM64 when not Linux
# Requires submodule update to v6.16.3
# containing commit https://github.com/facebook/rocksdb/commit/ee4bd4780b321ddb5f92a0f4eb956f2a2ebd60dc
#
IF(CMAKE_SYSTEM_PROCESSOR MATCHES "(arm64|aarch64)" AND NOT CMAKE_SYSTEM_NAME STREQUAL "Linux")
SKIP_ROCKSDB_PLUGIN("ARM64 disabled on all except Linux")
ENDIF()
# This plugin needs recent C++ compilers (it is using C++11 features)
# This plugin needs recent C++ compilers (it is using C++17 features)
# Skip build for the old compilers
SET(CXX11_FLAGS)
SET(OLD_COMPILER_MSG "requires c++11 -capable compiler (minimal supported versions are g++ 4.8, clang 3.3, VS2015)")
SET(CXX17_FLAGS)
SET(OLD_COMPILER_MSG "requires c++17 -capable compiler (minimal supported versions are g++ 7, clang 5, VS2017)")
IF(CMAKE_CXX_COMPILER_ID MATCHES "GNU")
EXECUTE_PROCESS(COMMAND ${CMAKE_CXX_COMPILER} -dumpversion OUTPUT_VARIABLE GCC_VERSION)
IF (GCC_VERSION VERSION_LESS 4.8)
IF (GCC_VERSION VERSION_LESS 7)
SKIP_ROCKSDB_PLUGIN("${OLD_COMPILER_MSG}")
ENDIF()
SET(CXX11_FLAGS "-std=c++11")
IF (GCC_VERSION VERSION_LESS 5.0)
SET(CXX11_FLAGS "-std=c++11 -Wno-missing-field-initializers")
ENDIF()
SET(CXX17_FLAGS "-std=c++17")
ELSEIF (CMAKE_CXX_COMPILER_ID MATCHES "Clang")
IF ((CMAKE_CXX_COMPILER_VERSION AND CMAKE_CXX_COMPILER_VERSION VERSION_LESS 3.3) OR
(CLANG_VERSION_STRING AND CLANG_VERSION_STRING VERSION_LESS 3.3))
IF ((CMAKE_CXX_COMPILER_VERSION AND CMAKE_CXX_COMPILER_VERSION VERSION_LESS 5) OR
(CLANG_VERSION_STRING AND CLANG_VERSION_STRING VERSION_LESS 5))
SKIP_ROCKSDB_PLUGIN("${OLD_COMPILER_MSG}")
ENDIF()
SET(CXX11_FLAGS "-std=c++11 -stdlib=libstdc++")
SET(CXX17_FLAGS "-std=c++17 -stdlib=libstdc++")
IF(MSVC)
# clang-cl does not work yet
SKIP_ROCKSDB_PLUGIN("Clang-cl is not supported")
ENDIF()
ELSEIF(MSVC)
IF (MSVC_VERSION LESS 1900)
IF (MSVC_VERSION LESS 1914)
SKIP_ROCKSDB_PLUGIN("${OLD_COMPILER_MSG}")
ENDIF()
ELSE()
......@@ -100,9 +86,9 @@ ELSE()
ENDIF()
IF(CMAKE_VERSION GREATER 3.0)
SET(CMAKE_CXX_STANDARD 11)
ELSEIF(CXX11_FLAGS)
ADD_DEFINITIONS(${CXX11_FLAGS})
SET(CMAKE_CXX_STANDARD 17)
ELSEIF(CXX17_FLAGS)
ADD_DEFINITIONS(${CXX17_FLAGS})
ENDIF()
SET(ROCKSDB_SE_SOURCES
......@@ -135,11 +121,6 @@ SET(ROCKSDB_SE_SOURCES
rdb_converter.h
)
# MariaDB: the following is added in build_rocksdb.cmake, when appropriate:
# This is a strong requirement coming from RocksDB. No conditional checks here.
#ADD_DEFINITIONS(-DROCKSDB_PLATFORM_POSIX -DROCKSDB_LIB_IO_POSIX
#)
if (HAVE_GCC_C11_ATOMICS_WITH_LIBATOMIC)
SET(ATOMIC_EXTRA_LIBS -latomic)
else()
......@@ -160,7 +141,7 @@ IF(NOT TARGET rocksdb)
RETURN()
ENDIF()
INSTALL_MANPAGES(rocksdb-engine mariadb-ldb.1 myrocks_hotbackup.1)
INSTALL_MANPAGES(rocksdb-engine mariadb-ldb.1 mariadb-sst-dump.1 myrocks_hotbackup.1)
CHECK_CXX_SOURCE_COMPILES("
#if defined(_MSC_VER) && !defined(__thread)
......@@ -193,7 +174,6 @@ ADD_CONVENIENCE_LIBRARY(rocksdb_aux_lib
rdb_perf_context.h
rdb_buff.h
rdb_mariadb_port.h
nosql_access.cc nosql_access.h
)
ADD_DEPENDENCIES(rocksdb_aux_lib GenError)
......@@ -204,37 +184,8 @@ if (UNIX AND NOT APPLE AND NOT CMAKE_SYSTEM_NAME STREQUAL "OpenBSD")
TARGET_LINK_LIBRARIES(rocksdb_aux_lib -lrt)
endif()
TARGET_LINK_LIBRARIES(rocksdb_aux_lib ${ATOMIC_EXTRA_LIBS})
# IF (WITH_JEMALLOC)
# FIND_LIBRARY(JEMALLOC_LIBRARY
# NAMES libjemalloc${PIC_EXT}.a jemalloc
# HINTS ${WITH_JEMALLOC}/lib)
# SET(rocksdb_static_libs ${rocksdb_static_libs}
# ${JEMALLOC_LIBRARY})
# ADD_DEFINITIONS(-DROCKSDB_JEMALLOC)
# ADD_DEFINITIONS(-DROCKSDB_MALLOC_USABLE_SIZE)
# ENDIF()
# MariaDB: Q: why does the upstream add libunwind for a particular
# storage engine?
#IF (WITH_UNWIND)
# FIND_LIBRARY(UNWIND_LIBRARY
# NAMES libunwind${PIC_EXT}.a unwind
# HINTS ${WITH_UNWIND}/lib)
# SET(rocksdb_static_libs ${rocksdb_static_libs}
# ${UNWIND_LIBRARY})
#ENDIF()
TARGET_LINK_LIBRARIES(rocksdb rocksdb_aux_lib)
CHECK_FUNCTION_EXISTS(sched_getcpu HAVE_SCHED_GETCPU)
IF(HAVE_SCHED_GETCPU)
ADD_DEFINITIONS(-DHAVE_SCHED_GETCPU=1)
# MariaDB: don't do this:
# ADD_DEFINITIONS(-DZSTD_STATIC_LINKING_ONLY)
ENDIF()
IF (WITH_TBB)
FIND_LIBRARY(TBB_LIBRARY
NAMES libtbb${PIC_EXT}.a tbb
......@@ -244,13 +195,6 @@ IF (WITH_TBB)
ADD_DEFINITIONS(-DTBB)
ENDIF()
#
# MariaDB: Dynamic plugin build is not suitable with unittest ATM
#
#IF(WITH_UNIT_TESTS AND WITH_EMBEDDED_SERVER)
# ADD_SUBDIRECTORY(unittest)
#ENDIF()
if (UNIX AND NOT APPLE AND NOT CMAKE_SYSTEM_NAME STREQUAL "OpenBSD")
SET(rocksdb_static_libs ${rocksdb_static_libs} "-lrt")
endif()
......@@ -262,8 +206,8 @@ ADD_LIBRARY(rocksdb_tools STATIC
rocksdb/tools/sst_dump_tool.cc
)
MYSQL_ADD_EXECUTABLE(sst_dump rocksdb/tools/sst_dump.cc COMPONENT rocksdb-engine)
TARGET_LINK_LIBRARIES(sst_dump rocksdblib)
MYSQL_ADD_EXECUTABLE(mariadb-sst-dump tools/mysql_sst_dump.cc COMPONENT rocksdb-engine)
TARGET_LINK_LIBRARIES(mariadb-sst-dump rocksdblib)
MYSQL_ADD_EXECUTABLE(mariadb-ldb tools/mysql_ldb.cc COMPONENT rocksdb-engine)
TARGET_LINK_LIBRARIES(mariadb-ldb rocksdb_tools rocksdb_aux_lib dbug)
......@@ -285,7 +229,7 @@ IF(MSVC)
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /wd4267")
ENDIF()
ELSEIF(CMAKE_CXX_COMPILER_ID MATCHES "GNU" OR CMAKE_CXX_COMPILER_ID MATCHES "Clang")
SET_TARGET_PROPERTIES(rocksdb_tools sst_dump mariadb-ldb PROPERTIES COMPILE_FLAGS "-Wno-error")
SET_TARGET_PROPERTIES(rocksdb_tools mariadb-sst-dump mariadb-ldb PROPERTIES COMPILE_FLAGS "-Wno-error")
ENDIF()
IF(GIT_EXECUTABLE AND EXISTS ${CMAKE_CURRENT_SOURCE_DIR}/rocksdb/.git)
......
storage/rocksdb/build_rocksdb.cmake
View file @ 485c9b1f
This diff is collapsed.
storage/rocksdb/get_rocksdb_files.sh deleted 100755 → 0
View file @ c035f3af
#!/bin/bash
MKFILE=`mktemp`
# create and run a simple makefile
# include rocksdb make file relative to the path of this script
echo "include ./storage/rocksdb/rocksdb/src.mk
all:
@echo \$(LIB_SOURCES)" > $MKFILE
for f in `make --makefile $MKFILE`
do
echo ./rocksdb/$f
done
rm $MKFILE
# create build_version.cc file. Only create one if it doesn't exists or if it is different
# this is so that we don't rebuild mysqld every time
bv=storage/rocksdb/rocksdb/util/build_version.cc
date=$(date +%F)
git_sha=$(pushd storage/rocksdb/rocksdb >/dev/null && git rev-parse HEAD 2>/dev/null && popd >/dev/null)
if [ ! -f $bv ] || [ -z $git_sha ] || [ ! `grep $git_sha $bv` ]
then
echo "#include \"build_version.h\"
const char* rocksdb_build_git_sha =
\"rocksdb_build_git_sha:$git_sha\";
const char* rocksdb_build_git_date =
\"rocksdb_build_git_date:$date\";
const char* rocksdb_build_compile_date = __DATE__;" > $bv
fi
storage/rocksdb/ha_rocksdb.cc
View file @ 485c9b1f
This source diff could not be displayed because it is too large. You can view the blob instead.
storage/rocksdb/ha_rocksdb.h
View file @ 485c9b1f
......@@ -93,6 +93,7 @@ enum collations_used {
#if 0 // MARIAROCKS_NOT_YET : read-free replication is not supported
extern char *rocksdb_read_free_rpl_tables;
extern ulong rocksdb_max_row_locks;
#if defined(HAVE_PSI_INTERFACE)
extern PSI_rwlock_key key_rwlock_read_free_rpl_tables;
#endif
......@@ -290,12 +291,18 @@ class ha_rocksdb : public my_core::handler {
*/
bool m_insert_with_update;
/* TRUE if last time the insertion failed due to duplicated PK */
bool m_dup_pk_found;
/*
TRUE if last time the insertion failed due to duplicate key error.
(m_dupp_errkey holds the key# that we've had error for)
*/
bool m_dup_key_found;
#ifndef DBUG_OFF
/* Last retreived record for sanity checking */
String m_dup_pk_retrieved_record;
/*
Last retrieved record (for duplicate PK) or index tuple (for duplicate
unique SK). Used for sanity checking.
*/
String m_dup_key_retrieved_record;
#endif
/**
......@@ -347,10 +354,10 @@ class ha_rocksdb : public my_core::handler {
MY_ATTRIBUTE((__nonnull__));
void release_scan_iterator(void);
rocksdb::Status get_for_update(
Rdb_transaction *const tx,
rocksdb::ColumnFamilyHandle *const column_family,
const rocksdb::Slice &key, rocksdb::PinnableSlice *value) const;
rocksdb::Status get_for_update(Rdb_transaction *const tx,
const Rdb_key_def &kd,
const rocksdb::Slice &key,
rocksdb::PinnableSlice *value) const;
int get_row_by_rowid(uchar *const buf, const char *const rowid,
const uint rowid_size, const bool skip_lookup = false,
......@@ -392,13 +399,6 @@ class ha_rocksdb : public my_core::handler {
void set_last_rowkey(const uchar *const old_data);
/*
For the active index, indicates which columns must be covered for the
current lookup to be covered. If the bitmap field is null, that means this
index does not cover the current lookup for any record.
*/
MY_BITMAP m_lookup_bitmap = {nullptr, nullptr, 0, 0};
int alloc_key_buffers(const TABLE *const table_arg,
const Rdb_tbl_def *const tbl_def_arg,
bool alloc_alter_buffers = false)
......@@ -670,7 +670,7 @@ class ha_rocksdb : public my_core::handler {
*/
private:
struct key_def_cf_info {
rocksdb::ColumnFamilyHandle *cf_handle;
std::shared_ptr<rocksdb::ColumnFamilyHandle> cf_handle;
bool is_reverse_cf;
bool is_per_partition_cf;
};
......@@ -771,14 +771,14 @@ class ha_rocksdb : public my_core::handler {
int get_pk_for_update(struct update_row_info *const row_info);
int check_and_lock_unique_pk(const uint key_id,
const struct update_row_info &row_info,
bool *const found)
bool *const found, const bool skip_unique_check)
MY_ATTRIBUTE((__warn_unused_result__));
int check_and_lock_sk(const uint key_id,
const struct update_row_info &row_info,
bool *const found)
bool *const found, const bool skip_unique_check)
MY_ATTRIBUTE((__warn_unused_result__));
int check_uniqueness_and_lock(const struct update_row_info &row_info,
bool pk_changed)
bool pk_changed, const bool skip_unique_check)
MY_ATTRIBUTE((__warn_unused_result__));
bool over_bulk_load_threshold(int *err)
MY_ATTRIBUTE((__warn_unused_result__));
......@@ -923,6 +923,8 @@ class ha_rocksdb : public my_core::handler {
MY_ATTRIBUTE((__warn_unused_result__));
int create_table(const std::string &table_name, const TABLE *table_arg,
ulonglong auto_increment_value);
int truncate_table(Rdb_tbl_def *tbl_def, TABLE *table_arg,
ulonglong auto_increment_value);
bool check_if_incompatible_data(HA_CREATE_INFO *const info,
uint table_changes) override
MY_ATTRIBUTE((__warn_unused_result__));
......@@ -982,6 +984,9 @@ class ha_rocksdb : public my_core::handler {
bool is_read_free_rpl_table() const;
#endif
void build_decoder();
void check_build_decoder();
#ifdef MARIAROCKS_NOT_YET // MDEV-10976
public:
virtual void rpl_before_delete_rows() override;
......@@ -997,6 +1002,9 @@ class ha_rocksdb : public my_core::handler {
bool m_in_rpl_update_rows;
bool m_force_skip_unique_check;
/* Need to build decoder on next read operation */
bool m_need_build_decoder;
};
/*
......
storage/rocksdb/ha_rocksdb_proto.h
View file @ 485c9b1f
......@@ -46,6 +46,9 @@ void rdb_handle_io_error(const rocksdb::Status status,
MY_ATTRIBUTE((noinline,noclone));
#endif
bool rdb_is_tablename_normalized(const std::string &tablename)
MY_ATTRIBUTE((__warn_unused_result__));
int rdb_normalize_tablename(const std::string &tablename, std::string *str)
MY_ATTRIBUTE((__nonnull__, __warn_unused_result__));
......@@ -54,6 +57,11 @@ int rdb_split_normalized_tablename(const std::string &fullname, std::string *db,
std::string *partition = nullptr)
MY_ATTRIBUTE((__warn_unused_result__));
void rdb_gen_normalized_tablename(const std::string *db,
const std::string *table,
const std::string *partition,
std::string *fullname);
std::vector<std::string> rdb_get_open_table_names(void);
class Rdb_perf_counters;
......@@ -66,6 +74,8 @@ void rdb_get_global_perf_counters(Rdb_perf_counters *counters)
void rdb_queue_save_stats_request();
extern const std::string TRUNCATE_TABLE_PREFIX;
/*
Access to singleton objects.
*/
......
storage/rocksdb/mariadb-ldb.1
View file @ 485c9b1f
'\" t
.\"
.TH "\fBMARIADB-LDB\fR" "1" "15 May 2020" "MariaDB 10.11" "MariaDB Database System"
.TH "\FBMARIADB-LDB\FR" "1" "15 May 2023" "MariaDB 11.2" "MariaDB Database System"
.\" -----------------------------------------------------------------
.\" * set default formatting
.\" -----------------------------------------------------------------
......@@ -11,6 +11,6 @@
.SH NAME
mariadb-ldb \- RocksDB tool (mysql_ldb is now a symlink to mariadb-ldb)
.SH DESCRIPTION
Use \fBmysql_ldb \-\-help\fR for details on usage\.
Use \fBmariadb-ldb \-\-help\fR for details on usage\.
.PP
For more information, please refer to the MariaDB Knowledge Base, available online at https://mariadb.com/kb/
storage/rocksdb/mariadb-sst-dump.1 0 → 100644
View file @ 485c9b1f
'\" t
.\"
.TH "\FBMARIADB-SST-DUMP\FR" "1" "15 May 2023" "MariaDB 11.2" "MariaDB Database System"
.\" -----------------------------------------------------------------
.\" * set default formatting
.\" -----------------------------------------------------------------
.\" disable hyphenation
.nh
.\" disable justification (adjust text to left margin only)
.ad l
.SH NAME
mariadb-sst-dump \- RocksDB tool (mysql_sst_dump is now a symlink to mariadb-sst-dump)
.SH DESCRIPTION
Use \fBmariadb-sst-dump \-\-help\fR for details on usage\.
.PP
For more information, please refer to the MariaDB Knowledge Base, available online at https://mariadb.com/kb/
storage/rocksdb/mysql-test/rocksdb/combinations
View file @ 485c9b1f
......@@ -3,3 +3,7 @@ loose-rocksdb_write_policy=write_committed
[write_prepared]
loose-rocksdb_write_policy=write_prepared
[write_unprepared]
loose-rocksdb_write_policy=write_unprepared
loose-rocksdb_write_batch_flush_threshold=1
storage/rocksdb/mysql-test/rocksdb/r/bloomfilter2.result
View file @ 485c9b1f
......@@ -2,7 +2,7 @@ CREATE TABLE t0 (id1 VARCHAR(30), id2 INT, value INT, PRIMARY KEY (id1, id2)) EN
select variable_value into @u from information_schema.global_status where variable_name='rocksdb_bloom_filter_prefix_useful';
SELECT COUNT(*) FROM t0 WHERE id1='X' AND id2>=1;
COUNT(*)
10000
100000
select case when variable_value-@u = 0 then 'true' else 'false' end from information_schema.global_status where variable_name='rocksdb_bloom_filter_prefix_useful';
case when variable_value-@u = 0 then 'true' else 'false' end
true
......@@ -11,14 +11,14 @@ CREATE TABLE t1 (id1 BIGINT, id2 INT, id3 BIGINT, value INT, PRIMARY KEY (id1, i
select variable_value into @u from information_schema.global_status where variable_name='rocksdb_bloom_filter_prefix_useful';
SELECT COUNT(*) FROM t1 WHERE id1=1 AND id2=1 AND id3>=2;
COUNT(*)
9999
99999
select case when variable_value-@u = 0 then 'true' else 'false' end from information_schema.global_status where variable_name='rocksdb_bloom_filter_prefix_useful';
case when variable_value-@u = 0 then 'true' else 'false' end
true
select variable_value into @u from information_schema.global_status where variable_name='rocksdb_bloom_filter_prefix_useful';
SELECT COUNT(*) FROM t1 WHERE id1=1 AND id2>=1 AND id3>=2;
COUNT(*)
9999
99999
select case when variable_value-@u = 0 then 'true' else 'false' end from information_schema.global_status where variable_name='rocksdb_bloom_filter_prefix_useful';
case when variable_value-@u = 0 then 'true' else 'false' end
true
......
storage/rocksdb/mysql-test/rocksdb/r/cancel_mc-master.opt 0 → 100644
View file @ 485c9b1f
--rocksdb_rate_limiter_bytes_per_sec=256k --rocksdb_default_cf_options=write_buffer_size=64k;target_file_size_base=64k;max_bytes_for_level_base=1m;compression_per_level=kNoCompression;max_compaction_bytes=256k
\ No newline at end of file
storage/rocksdb/mysql-test/rocksdb/r/cancel_mc.result 0 → 100644
View file @ 485c9b1f
CREATE FUNCTION is_compaction_stopped() RETURNS INT
BEGIN
SELECT variable_value INTO @cwb_1 FROM information_schema.global_status WHERE variable_name="rocksdb_compact_write_bytes";
SELECT SLEEP(3) INTO @a;
SELECT variable_value INTO @cwb_2 FROM information_schema.global_status WHERE variable_name="rocksdb_compact_write_bytes";
IF @cwb_2 > @cwb_1 THEN
RETURN 0;
END IF;
RETURN 1;
END//
DROP TABLE IF EXISTS t1;
create table t1 (id int auto_increment, value int, value2 varchar(500), primary key (id), index(value2) comment 'cf_i') engine=rocksdb;
SET SESSION rocksdb_bulk_load_allow_sk=1;
SET SESSION rocksdb_bulk_load=1;
LOAD DATA INFILE '<DATADIR>/se_loaddata.dat' INTO TABLE t1
FIELDS TERMINATED BY ',' (id, value, value2);
SET SESSION rocksdb_bulk_load=0;
SET SESSION rocksdb_bulk_load_allow_sk=0;
connect con1,localhost,root,,;
connect con2,localhost,root,,;
#
# Cancelling MC, including pending MCs, by KILL statements
#
connection default;
SELECT variable_value into @executed_start FROM information_schema.global_status WHERE variable_name="rocksdb_manual_compactions_cancelled";
connection con1;
SET GLOBAL rocksdb_compact_cf='default';
connection con2;
set session rocksdb_manual_compaction_threads=2;
SET GLOBAL rocksdb_compact_cf='cf_i';
connection default;
connection con1;
ERROR HY000: Internal error: Manual Compaction Failed. Reason: Cancelled by client.
connection con2;
ERROR HY000: Internal error: Manual Compaction Failed. Reason: Cancelled by client.
connection default;
SELECT variable_value into @executed_end FROM information_schema.global_status WHERE variable_name="rocksdb_manual_compactions_cancelled";
SELECT @executed_end-@executed_start AS CANCELLED_MC;
CANCELLED_MC
2
#
# Cancelling MC by global opt
#
connection default;
SELECT variable_value into @executed_start FROM information_schema.global_status WHERE variable_name="rocksdb_manual_compactions_cancelled";
connection con1;
set session rocksdb_manual_compaction_threads=2;
SET GLOBAL rocksdb_compact_cf='default';
connection default;
connection con2;
SET GLOBAL rocksdb_compact_cf='cf_i';
connection default;
set global rocksdb_cancel_manual_compactions=ON;
connection con1;
ERROR HY000: Internal error: Manual Compaction Failed. Reason: Cancelled by server.
connection con2;
ERROR HY000: Internal error: Manual Compaction Failed. Reason: Cancelled by server.
connection default;
SELECT variable_value into @executed_end FROM information_schema.global_status WHERE variable_name="rocksdb_manual_compactions_cancelled";
SELECT @executed_end-@executed_start AS CANCELLED_MC;
CANCELLED_MC
2
#
# Cancelling OPTIMIZE TABLE by global opt
#
connection con1;
OPTIMIZE TABLE t1;
connection default;
SELECT variable_value into @current_cwb FROM information_schema.global_status WHERE variable_name="rocksdb_compact_write_bytes";
set global rocksdb_cancel_manual_compactions=ON;
connection con1;
Table Op Msg_type Msg_text
test.t1 optimize Error Got error 7 'Result incomplete: Manual compaction paused' from ROCKSDB
test.t1 optimize error Unknown - internal error 215 during operation
connection default;
select 'PRIMARY' as idx, COUNT(*) as cnt FROM t1 FORCE INDEX (PRIMARY)
UNION ALL SELECT 'value2' as idx, COUNT(*) as cnt FROM t1 FORCE INDEX (value2);
idx cnt
PRIMARY 500000
value2 500000
#
# Dropping Indexes complete even after cancelling MC
#
INSERT INTO t1 (id) VALUES (null);
set global rocksdb_force_flush_memtable_now=ON;
SELECT variable_value into @current_cwb FROM information_schema.global_status WHERE variable_name="rocksdb_compact_write_bytes";
DROP TABLE t1;
set global rocksdb_cancel_manual_compactions=ON;
DROP FUNCTION is_compaction_stopped;
storage/rocksdb/mysql-test/rocksdb/r/cancel_mc_timeout.result 0 → 100644
View file @ 485c9b1f
connect con1,localhost,root,,;
connect con2,localhost,root,,;
connection default;
set global rocksdb_debug_manual_compaction_delay=80;
connection con1;
SET GLOBAL rocksdb_compact_cf='default';
connection default;
connection con2;
set session rocksdb_manual_compaction_threads=2;
SET GLOBAL rocksdb_compact_cf='default';
connection default;
connection con2;
ERROR HY000: Internal error: Manual Compaction Failed. Reason: Cancelled by client.
connection con1;
ERROR HY000: Internal error: Manual Compaction Failed. Reason: Cancelled by client. (timeout)
connection default;
set global rocksdb_debug_manual_compaction_delay=0;
connection con1;
SET GLOBAL rocksdb_compact_cf='default';
storage/rocksdb/mysql-test/rocksdb/r/drop_table3.result
View file @ 485c9b1f
call mtr.add_suppression("Column family 'cf1' not found");
call mtr.add_suppression("Column family 'rev:cf2' not found");
DROP TABLE IF EXISTS t1;
call mtr.add_suppression("Column family 'cf1' not found");
call mtr.add_suppression("Column family 'rev:cf2' not found");
set global rocksdb_compact_cf = 'cf1';
set global rocksdb_compact_cf = 'rev:cf2';
set global rocksdb_signal_drop_index_thread = 1;
# restart
CREATE TABLE t1 (
a int not null,
b int not null,
......@@ -15,6 +9,10 @@ primary key (a,b) comment 'cf1',
key (b) comment 'rev:cf2'
) ENGINE=RocksDB;
DELETE FROM t1;
set global rocksdb_compact_cf = 'cf1';
set global rocksdb_compact_cf = 'rev:cf2';
set global rocksdb_signal_drop_index_thread = 1;
# restart
select variable_value into @a from information_schema.global_status where variable_name='rocksdb_compact_read_bytes';
drop table t1;
select case when variable_value-@a < 500000 then 'true' else 'false' end from information_schema.global_status where variable_name='rocksdb_compact_read_bytes';
......
storage/rocksdb/mysql-test/rocksdb/r/i_s.result
View file @ 485c9b1f
......@@ -11,6 +11,7 @@ ROCKSDB_DDL
ROCKSDB_DEADLOCK
ROCKSDB_GLOBAL_INFO
ROCKSDB_INDEX_FILE_MAP
ROCKSDB_LIVE_FILES_METADATA
ROCKSDB_LOCKS
ROCKSDB_PERF_CONTEXT
ROCKSDB_PERF_CONTEXT_GLOBAL
......
storage/rocksdb/mysql-test/rocksdb/r/information_schema.result
View file @ 485c9b1f
......@@ -55,6 +55,9 @@ CF_FLAGS 4 cf_c [0]
CF_FLAGS 5 rev:cf_d [1]
CREATE TABLE t3 (a INT, PRIMARY KEY (a)) ENGINE=ROCKSDB;
insert into t3 (a) values (1), (2), (3);
SELECT IF(count(*) > 0, "TRUE", "FALSE") as metadata_exist FROM INFORMATION_SCHEMA.ROCKSDB_LIVE_FILES_METADATA where CF_NAME = 'default';
metadata_exist
TRUE
SET @ORIG_ROCKSDB_PAUSE_BACKGROUND_WORK = @@GLOBAL.ROCKSDB_PAUSE_BACKGROUND_WORK;
SHOW GLOBAL VARIABLES LIKE 'ROCKSDB_PAUSE_BACKGROUND_WORK';
Variable_name Value
......
storage/rocksdb/mysql-test/rocksdb/r/issue243_transactionStatus.result
View file @ 485c9b1f
......@@ -60,7 +60,7 @@ LIST OF SNAPSHOTS FOR EACH SESSION:
---SNAPSHOT, ACTIVE NUM sec
MySQL thread id TID, OS thread handle PTR, query id QID localhost root ACTION
SHOW ENGINE rocksdb TRANSACTION STATUS
lock count 8, write count 4
lock count 4, write count 4
insert count 2, update count 1, delete count 1
----------LATEST DETECTED DEADLOCKS----------
-----------------------------------------
......@@ -70,7 +70,7 @@ END OF ROCKSDB TRANSACTION MONITOR OUTPUT
ROLLBACK;
SHOW ENGINE rocksdb TRANSACTION STATUS;
Type Name Status
rocksdb
rocksdb
============================================================
TIMESTAMP ROCKSDB TRANSACTION MONITOR OUTPUT
============================================================
......@@ -87,7 +87,7 @@ START TRANSACTION;
INSERT INTO t1 VALUES(40,40,40);
SHOW ENGINE rocksdb TRANSACTION STATUS;
Type Name Status
rocksdb
rocksdb
============================================================
TIMESTAMP ROCKSDB TRANSACTION MONITOR OUTPUT
============================================================
......@@ -98,7 +98,7 @@ LIST OF SNAPSHOTS FOR EACH SESSION:
---SNAPSHOT, ACTIVE NUM sec
MySQL thread id TID, OS thread handle PTR, query id QID localhost root ACTION
SHOW ENGINE rocksdb TRANSACTION STATUS
lock count 2, write count 1
lock count 1, write count 1
insert count 1, update count 0, delete count 0
----------LATEST DETECTED DEADLOCKS----------
-----------------------------------------
......@@ -108,7 +108,7 @@ END OF ROCKSDB TRANSACTION MONITOR OUTPUT
COMMIT;
SHOW ENGINE rocksdb TRANSACTION STATUS;
Type Name Status
rocksdb
rocksdb
============================================================
TIMESTAMP ROCKSDB TRANSACTION MONITOR OUTPUT
============================================================
......@@ -138,7 +138,43 @@ UPDATE t2 SET value=3 WHERE id2=2;
DELETE FROM t2 WHERE id1=10;
SHOW ENGINE rocksdb TRANSACTION STATUS;
Type Name Status
rocksdb
rocksdb
============================================================
TIMESTAMP ROCKSDB TRANSACTION MONITOR OUTPUT
============================================================
---------
SNAPSHOTS
---------
LIST OF SNAPSHOTS FOR EACH SESSION:
---SNAPSHOT, ACTIVE NUM sec
MySQL thread id TID, OS thread handle PTR, query id QID localhost root ACTION
SHOW ENGINE rocksdb TRANSACTION STATUS
lock count 5, write count 7
insert count 2, update count 1, delete count 1
----------LATEST DETECTED DEADLOCKS----------
-----------------------------------------
END OF ROCKSDB TRANSACTION MONITOR OUTPUT
=========================================
ROLLBACK;
SET AUTOCOMMIT=1;
DROP TABLE t2;
DROP TABLE IF EXISTS t2;
CREATE TABLE t2 (
id1 INT,
id2 INT,
value INT,
PRIMARY KEY (id1),
UNIQUE KEY (id2)
) ENGINE=rocksdb;
SET AUTOCOMMIT=0;
START TRANSACTION;
INSERT INTO t2 VALUES(1,2,0),(10,20,30);
UPDATE t2 SET value=3 WHERE id2=2;
DELETE FROM t2 WHERE id1=10;
SHOW ENGINE rocksdb TRANSACTION STATUS;
Type Name Status
rocksdb
============================================================
TIMESTAMP ROCKSDB TRANSACTION MONITOR OUTPUT
============================================================
......@@ -149,7 +185,7 @@ LIST OF SNAPSHOTS FOR EACH SESSION:
---SNAPSHOT, ACTIVE NUM sec
MySQL thread id TID, OS thread handle PTR, query id QID localhost root ACTION
SHOW ENGINE rocksdb TRANSACTION STATUS
lock count 9, write count 7
lock count 4, write count 7
insert count 2, update count 1, delete count 1
----------LATEST DETECTED DEADLOCKS----------
-----------------------------------------
......
storage/rocksdb/mysql-test/rocksdb/r/issue896.result
View file @ 485c9b1f
......@@ -15,3 +15,20 @@ SELECT COUNT(*) FROM t1 FORCE INDEX(d);
COUNT(*)
1
DROP TABLE t1;
CREATE TABLE `t1` (
`a` bigint(20) NOT NULL,
`b` varchar(10) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL,
`u` timestamp NOT NULL,
`d` bigint(20) DEFAULT NULL,
PRIMARY KEY (`a`,`b`),
KEY `d` (`d`)
) ENGINE=ROCKSDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin COMMENT='ttl_duration=1000;ttl_col=u';
INSERT INTO t1 VALUES (100, 'aaabbb', NOW(), 200);
EXPLAIN SELECT COUNT(*) FROM t1;
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE t1 index NULL d 9 NULL # Using index
# segfault here without the fix
SELECT COUNT(*) FROM t1;
COUNT(*)
1
DROP TABLE t1;
storage/rocksdb/mysql-test/rocksdb/r/manual_compaction_bottommost_level.result 0 → 100644
View file @ 485c9b1f
CREATE PROCEDURE compact_start()
BEGIN
select variable_value into @c from information_schema.global_status where variable_name='rocksdb_compact_write_bytes';
END//
CREATE PROCEDURE compact_end()
BEGIN
select case when variable_value-@c > 0 then 'true' else 'false' end as checked from information_schema.global_status where variable_name='rocksdb_compact_write_bytes';
END//
DROP TABLE IF EXISTS t1;
CREATE TABLE t1 (
a int not null,
b int not null,
primary key (a,b),
key (b)
) ENGINE=RocksDB;
DELETE FROM t1;
set @@session.rocksdb_manual_compaction_bottommost_level='kSkip';
call compact_start();
set @@global.rocksdb_compact_cf = 'default';
call compact_end();
checked
true
set @@session.rocksdb_manual_compaction_bottommost_level='kSkip';
call compact_start();
set @@global.rocksdb_compact_cf = 'default';
call compact_end();
checked
false
set @@session.rocksdb_manual_compaction_bottommost_level='kForceOptimized';
call compact_start();
set @@global.rocksdb_compact_cf = 'default';
call compact_end();
checked
true
DROP PROCEDURE compact_start;
DROP PROCEDURE compact_end;
drop table t1;
storage/rocksdb/mysql-test/rocksdb/r/mariadb_plugin.result
View file @ 485c9b1f
......@@ -23,6 +23,7 @@ Warning 1620 Plugin is busy and will be uninstalled on shutdown
Warning 1620 Plugin is busy and will be uninstalled on shutdown
Warning 1620 Plugin is busy and will be uninstalled on shutdown
Warning 1620 Plugin is busy and will be uninstalled on shutdown
Warning 1620 Plugin is busy and will be uninstalled on shutdown
SELECT ENGINE, SUPPORT FROM INFORMATION_SCHEMA.ENGINES WHERE ENGINE='ROCKSDB';
ENGINE SUPPORT
ROCKSDB NO
......
storage/rocksdb/mysql-test/rocksdb/r/mariadb_port_fixes.result
View file @ 485c9b1f
......@@ -83,6 +83,7 @@ ROCKSDB_INDEX_FILE_MAP Stable
ROCKSDB_LOCKS Stable
ROCKSDB_TRX Stable
ROCKSDB_DEADLOCK Stable
ROCKSDB_LIVE_FILES_METADATA Stable
#
# MDEV-12466 : Assertion `thd->transaction.stmt.is_empty() || thd->in_sub_stmt || ...
#
......
storage/rocksdb/mysql-test/rocksdb/r/max_row_locks.result 0 → 100644
View file @ 485c9b1f
create table t1 (id1 bigint, id2 bigint, c1 bigint, c2 bigint, c3 bigint, c4 bigint, c5 bigint, c6 bigint, c7 bigint, primary key (id1, id2), index i(c1, c2));
begin;
select * from t1 where c3=1 for update;
id1 id2 c1 c2 c3 c4 c5 c6 c7
SHOW ENGINE rocksdb TRANSACTION STATUS;
Type Name Status
rocksdb
============================================================
TIMESTAMP ROCKSDB TRANSACTION MONITOR OUTPUT
============================================================
---------
SNAPSHOTS
---------
LIST OF SNAPSHOTS FOR EACH SESSION:
---SNAPSHOT, ACTIVE NUM sec
MySQL thread id TID, OS thread handle PTR, query id QID localhost root ACTION
SHOW ENGINE rocksdb TRANSACTION STATUS
lock count 0, write count 0
insert count 0, update count 0, delete count 0
----------LATEST DETECTED DEADLOCKS----------
-----------------------------------------
END OF ROCKSDB TRANSACTION MONITOR OUTPUT
=========================================
select * from t1 where c7 between 101 and 110 for update;
id1 id2 c1 c2 c3 c4 c5 c6 c7
101 0 101 0 0 0 0 0 101
102 0 102 0 0 0 0 0 102
103 0 103 0 0 0 0 0 103
104 0 104 0 0 0 0 0 104
105 0 105 0 0 0 0 0 105
106 0 106 0 0 0 0 0 106
107 0 107 0 0 0 0 0 107
108 0 108 0 0 0 0 0 108
109 0 109 0 0 0 0 0 109
110 0 110 0 0 0 0 0 110
SHOW ENGINE rocksdb TRANSACTION STATUS;
Type Name Status
rocksdb
============================================================
TIMESTAMP ROCKSDB TRANSACTION MONITOR OUTPUT
============================================================
---------
SNAPSHOTS
---------
LIST OF SNAPSHOTS FOR EACH SESSION:
---SNAPSHOT, ACTIVE NUM sec
MySQL thread id TID, OS thread handle PTR, query id QID localhost root ACTION
SHOW ENGINE rocksdb TRANSACTION STATUS
lock count 10, write count 0
insert count 0, update count 0, delete count 0
----------LATEST DETECTED DEADLOCKS----------
-----------------------------------------
END OF ROCKSDB TRANSACTION MONITOR OUTPUT
=========================================
rollback;
set session rocksdb_lock_scanned_rows=on;
begin;
select * from t1 where c3=1 for update;
id1 id2 c1 c2 c3 c4 c5 c6 c7
SHOW ENGINE rocksdb TRANSACTION STATUS;
Type Name Status
rocksdb
============================================================
TIMESTAMP ROCKSDB TRANSACTION MONITOR OUTPUT
============================================================
---------
SNAPSHOTS
---------
LIST OF SNAPSHOTS FOR EACH SESSION:
---SNAPSHOT, ACTIVE NUM sec
MySQL thread id TID, OS thread handle PTR, query id QID localhost root ACTION
SHOW ENGINE rocksdb TRANSACTION STATUS
lock count 1000, write count 0
insert count 0, update count 0, delete count 0
----------LATEST DETECTED DEADLOCKS----------
-----------------------------------------
END OF ROCKSDB TRANSACTION MONITOR OUTPUT
=========================================
rollback;
set session rocksdb_lock_scanned_rows=off;
SET @start_rocksdb_max_row_locks = @@global.rocksdb_max_row_locks;
set global rocksdb_max_row_locks = 20;
select * from t1 where c3=1 for update;
id1 id2 c1 c2 c3 c4 c5 c6 c7
select * from t1 where c7 between 101 and 110 for update;
id1 id2 c1 c2 c3 c4 c5 c6 c7
101 0 101 0 0 0 0 0 101
102 0 102 0 0 0 0 0 102
103 0 103 0 0 0 0 0 103
104 0 104 0 0 0 0 0 104
105 0 105 0 0 0 0 0 105
106 0 106 0 0 0 0 0 106
107 0 107 0 0 0 0 0 107
108 0 108 0 0 0 0 0 108
109 0 109 0 0 0 0 0 109
110 0 110 0 0 0 0 0 110
set global rocksdb_max_row_locks = 5;
select * from t1 where c3=1 for update;
id1 id2 c1 c2 c3 c4 c5 c6 c7
select * from t1 where c7 between 101 and 110 for update;
ERROR HY000: Got error 10 'Operation aborted: Failed to acquire lock due to rocksdb_max_row_locks limit' from ROCKSDB
set session rocksdb_lock_scanned_rows=on;
select * from t1 where c3=1 for update;
ERROR HY000: Got error 10 'Operation aborted: Failed to acquire lock due to rocksdb_max_row_locks limit' from ROCKSDB
select * from t1 where c7 between 101 and 110 for update;
ERROR HY000: Got error 10 'Operation aborted: Failed to acquire lock due to rocksdb_max_row_locks limit' from ROCKSDB
set session rocksdb_lock_scanned_rows=off;
set @@global.rocksdb_max_row_locks = @start_rocksdb_max_row_locks;
drop table t1;
storage/rocksdb/mysql-test/rocksdb/r/perf_context.result
View file @ 485c9b1f
......@@ -161,10 +161,6 @@ BEGIN;
INSERT INTO t2 VALUES (1), (2);
INSERT INTO t2 VALUES (3), (4);
COMMIT;
SELECT COUNT(*) from INFORMATION_SCHEMA.ROCKSDB_PERF_CONTEXT
WHERE TABLE_NAME = 't2'
AND STAT_TYPE = 'IO_WRITE_NANOS'
AND VALUE > 0;
COUNT(*)
0
SELECT COUNT(*) from INFORMATION_SCHEMA.ROCKSDB_PERF_CONTEXT_GLOBAL
......
storage/rocksdb/mysql-test/rocksdb/r/rocksdb.result
View file @ 485c9b1f
......@@ -888,6 +888,7 @@ rocksdb_cache_dump ON
rocksdb_cache_high_pri_pool_ratio 0.000000
rocksdb_cache_index_and_filter_blocks ON
rocksdb_cache_index_and_filter_with_high_priority ON
rocksdb_cancel_manual_compactions OFF
rocksdb_checksums_pct 100
rocksdb_collect_sst_properties ON
rocksdb_commit_in_the_middle OFF
......@@ -918,6 +919,9 @@ rocksdb_delete_obsolete_files_period_micros 21600000000
rocksdb_enable_2pc ON
rocksdb_enable_bulk_load_api ON
rocksdb_enable_insert_with_update_caching ON
rocksdb_enable_iterate_bounds ON
rocksdb_enable_pipelined_write OFF
rocksdb_enable_remove_orphaned_dropped_cfs ON
rocksdb_enable_thread_tracking ON
rocksdb_enable_ttl ON
rocksdb_enable_ttl_read_filtering ON
......@@ -945,10 +949,14 @@ rocksdb_lock_wait_timeout 1
rocksdb_log_dir
rocksdb_log_file_time_to_roll 0
rocksdb_manifest_preallocation_size 4194304
rocksdb_manual_compaction_bottommost_level kForceOptimized
rocksdb_manual_compaction_threads 0
rocksdb_manual_wal_flush ON
rocksdb_master_skip_tx_api OFF
rocksdb_max_background_compactions -1
rocksdb_max_background_flushes -1
rocksdb_max_background_jobs 2
rocksdb_max_bottom_pri_background_compactions 0
rocksdb_max_latest_deadlocks 5
rocksdb_max_log_file_size 0
rocksdb_max_manifest_file_size 1073741824
......@@ -979,10 +987,11 @@ rocksdb_signal_drop_index_thread OFF
rocksdb_sim_cache_size 0
rocksdb_skip_bloom_filter_on_read OFF
rocksdb_skip_fill_cache OFF
rocksdb_skip_locks_if_skip_unique_check OFF
rocksdb_skip_unique_check_tables .*
rocksdb_sst_mgr_rate_bytes_per_sec 0
rocksdb_stats_dump_period_sec 600
rocksdb_stats_level 0
rocksdb_stats_level 1
rocksdb_stats_recalc_rate 0
rocksdb_store_row_debug_checksums OFF
rocksdb_strict_collation_check OFF
......@@ -991,6 +1000,7 @@ rocksdb_table_cache_numshardbits 6
rocksdb_table_stats_sampling_pct 10
rocksdb_tmpdir
rocksdb_trace_sst_api OFF
rocksdb_track_and_verify_wals_in_manifest ON
rocksdb_two_write_queues ON
rocksdb_unsafe_for_binlog OFF
rocksdb_update_cf_options
......@@ -1003,10 +1013,11 @@ rocksdb_validate_tables 1
rocksdb_verify_row_debug_checksums OFF
rocksdb_wal_bytes_per_sync 0
rocksdb_wal_dir
rocksdb_wal_recovery_mode 1
rocksdb_wal_recovery_mode 2
rocksdb_wal_size_limit_mb 0
rocksdb_wal_ttl_seconds 0
rocksdb_whole_key_filtering ON
rocksdb_write_batch_flush_threshold 0
rocksdb_write_batch_max_bytes 0
rocksdb_write_disable_wal OFF
rocksdb_write_ignore_missing_column_families OFF
......@@ -1365,9 +1376,10 @@ CREATE TABLE t1 (pk INT PRIMARY KEY, a int) ENGINE=RocksDB;
set @a=-1;
insert into t1 select (@a:=@a+1), 1234 from information_schema.session_variables limit 100;
set @tmp1= @@rocksdb_max_row_locks;
set rocksdb_max_row_locks= 20;
set GLOBAL rocksdb_max_row_locks= 20;
update t1 set a=a+10;
ERROR HY000: Got error 10 'Operation aborted: Failed to acquire lock due to rocksdb_max_row_locks limit' from ROCKSDB
set @@global.rocksdb_max_row_locks = @tmp1;
DROP TABLE t1;
#
# Test AUTO_INCREMENT behavior problem,
......@@ -1500,19 +1512,15 @@ Rocksdb_block_cache_data_bytes_insert #
Rocksdb_block_cache_data_hit #
Rocksdb_block_cache_data_miss #
Rocksdb_block_cache_filter_add #
Rocksdb_block_cache_filter_bytes_evict #
Rocksdb_block_cache_filter_bytes_insert #
Rocksdb_block_cache_filter_hit #
Rocksdb_block_cache_filter_miss #
Rocksdb_block_cache_hit #
Rocksdb_block_cache_index_add #
Rocksdb_block_cache_index_bytes_evict #
Rocksdb_block_cache_index_bytes_insert #
Rocksdb_block_cache_index_hit #
Rocksdb_block_cache_index_miss #
Rocksdb_block_cache_miss #
Rocksdb_block_cachecompressed_hit #
Rocksdb_block_cachecompressed_miss #
Rocksdb_bloom_filter_full_positive #
Rocksdb_bloom_filter_full_true_positive #
Rocksdb_bloom_filter_prefix_checked #
......@@ -1531,14 +1539,14 @@ Rocksdb_get_hit_l1 #
Rocksdb_get_hit_l2_and_up #
Rocksdb_getupdatessince_calls #
Rocksdb_iter_bytes_read #
Rocksdb_manual_compactions_cancelled #
Rocksdb_manual_compactions_pending #
Rocksdb_manual_compactions_processed #
Rocksdb_manual_compactions_running #
Rocksdb_memtable_hit #
Rocksdb_memtable_miss #
Rocksdb_no_file_closes #
Rocksdb_no_file_errors #
Rocksdb_no_file_opens #
Rocksdb_num_iterators #
Rocksdb_number_block_not_compressed #
Rocksdb_number_db_next #
Rocksdb_number_db_next_found #
......@@ -1546,7 +1554,6 @@ Rocksdb_number_db_prev #
Rocksdb_number_db_prev_found #
Rocksdb_number_db_seek #
Rocksdb_number_db_seek_found #
Rocksdb_number_deletes_filtered #
Rocksdb_number_keys_read #
Rocksdb_number_keys_updated #
Rocksdb_number_keys_written #
......@@ -1582,7 +1589,6 @@ Rocksdb_wal_group_syncs #
Rocksdb_wal_synced #
Rocksdb_write_other #
Rocksdb_write_self #
Rocksdb_write_timedout #
Rocksdb_write_wal #
select VARIABLE_NAME from INFORMATION_SCHEMA.global_status where VARIABLE_NAME LIKE 'rocksdb%' and VARIABLE_NAME NOT LIKE '%num_get_for_update%';
VARIABLE_NAME
......@@ -1611,19 +1617,15 @@ ROCKSDB_BLOCK_CACHE_DATA_BYTES_INSERT
ROCKSDB_BLOCK_CACHE_DATA_HIT
ROCKSDB_BLOCK_CACHE_DATA_MISS
ROCKSDB_BLOCK_CACHE_FILTER_ADD
ROCKSDB_BLOCK_CACHE_FILTER_BYTES_EVICT
ROCKSDB_BLOCK_CACHE_FILTER_BYTES_INSERT
ROCKSDB_BLOCK_CACHE_FILTER_HIT
ROCKSDB_BLOCK_CACHE_FILTER_MISS
ROCKSDB_BLOCK_CACHE_HIT
ROCKSDB_BLOCK_CACHE_INDEX_ADD
ROCKSDB_BLOCK_CACHE_INDEX_BYTES_EVICT
ROCKSDB_BLOCK_CACHE_INDEX_BYTES_INSERT
ROCKSDB_BLOCK_CACHE_INDEX_HIT
ROCKSDB_BLOCK_CACHE_INDEX_MISS
ROCKSDB_BLOCK_CACHE_MISS
ROCKSDB_BLOCK_CACHECOMPRESSED_HIT
ROCKSDB_BLOCK_CACHECOMPRESSED_MISS
ROCKSDB_BLOOM_FILTER_FULL_POSITIVE
ROCKSDB_BLOOM_FILTER_FULL_TRUE_POSITIVE
ROCKSDB_BLOOM_FILTER_PREFIX_CHECKED
......@@ -1642,14 +1644,14 @@ ROCKSDB_GET_HIT_L1
ROCKSDB_GET_HIT_L2_AND_UP
ROCKSDB_GETUPDATESSINCE_CALLS
ROCKSDB_ITER_BYTES_READ
ROCKSDB_MANUAL_COMPACTIONS_CANCELLED
ROCKSDB_MANUAL_COMPACTIONS_PENDING
ROCKSDB_MANUAL_COMPACTIONS_PROCESSED
ROCKSDB_MANUAL_COMPACTIONS_RUNNING
ROCKSDB_MEMTABLE_HIT
ROCKSDB_MEMTABLE_MISS
ROCKSDB_NO_FILE_CLOSES
ROCKSDB_NO_FILE_ERRORS
ROCKSDB_NO_FILE_OPENS
ROCKSDB_NUM_ITERATORS
ROCKSDB_NUMBER_BLOCK_NOT_COMPRESSED
ROCKSDB_NUMBER_DB_NEXT
ROCKSDB_NUMBER_DB_NEXT_FOUND
......@@ -1657,7 +1659,6 @@ ROCKSDB_NUMBER_DB_PREV
ROCKSDB_NUMBER_DB_PREV_FOUND
ROCKSDB_NUMBER_DB_SEEK
ROCKSDB_NUMBER_DB_SEEK_FOUND
ROCKSDB_NUMBER_DELETES_FILTERED
ROCKSDB_NUMBER_KEYS_READ
ROCKSDB_NUMBER_KEYS_UPDATED
ROCKSDB_NUMBER_KEYS_WRITTEN
......@@ -1693,7 +1694,6 @@ ROCKSDB_WAL_GROUP_SYNCS
ROCKSDB_WAL_SYNCED
ROCKSDB_WRITE_OTHER
ROCKSDB_WRITE_SELF
ROCKSDB_WRITE_TIMEDOUT
ROCKSDB_WRITE_WAL
# RocksDB-SE's status variables are global internally
# but they are shown as both session and global, like InnoDB's status vars.
......@@ -1724,19 +1724,15 @@ ROCKSDB_BLOCK_CACHE_DATA_BYTES_INSERT
ROCKSDB_BLOCK_CACHE_DATA_HIT
ROCKSDB_BLOCK_CACHE_DATA_MISS
ROCKSDB_BLOCK_CACHE_FILTER_ADD
ROCKSDB_BLOCK_CACHE_FILTER_BYTES_EVICT
ROCKSDB_BLOCK_CACHE_FILTER_BYTES_INSERT
ROCKSDB_BLOCK_CACHE_FILTER_HIT
ROCKSDB_BLOCK_CACHE_FILTER_MISS
ROCKSDB_BLOCK_CACHE_HIT
ROCKSDB_BLOCK_CACHE_INDEX_ADD
ROCKSDB_BLOCK_CACHE_INDEX_BYTES_EVICT
ROCKSDB_BLOCK_CACHE_INDEX_BYTES_INSERT
ROCKSDB_BLOCK_CACHE_INDEX_HIT
ROCKSDB_BLOCK_CACHE_INDEX_MISS
ROCKSDB_BLOCK_CACHE_MISS
ROCKSDB_BLOCK_CACHECOMPRESSED_HIT
ROCKSDB_BLOCK_CACHECOMPRESSED_MISS
ROCKSDB_BLOOM_FILTER_FULL_POSITIVE
ROCKSDB_BLOOM_FILTER_FULL_TRUE_POSITIVE
ROCKSDB_BLOOM_FILTER_PREFIX_CHECKED
......@@ -1755,14 +1751,14 @@ ROCKSDB_GET_HIT_L1
ROCKSDB_GET_HIT_L2_AND_UP
ROCKSDB_GETUPDATESSINCE_CALLS
ROCKSDB_ITER_BYTES_READ
ROCKSDB_MANUAL_COMPACTIONS_CANCELLED
ROCKSDB_MANUAL_COMPACTIONS_PENDING
ROCKSDB_MANUAL_COMPACTIONS_PROCESSED
ROCKSDB_MANUAL_COMPACTIONS_RUNNING
ROCKSDB_MEMTABLE_HIT
ROCKSDB_MEMTABLE_MISS
ROCKSDB_NO_FILE_CLOSES
ROCKSDB_NO_FILE_ERRORS
ROCKSDB_NO_FILE_OPENS
ROCKSDB_NUM_ITERATORS
ROCKSDB_NUMBER_BLOCK_NOT_COMPRESSED
ROCKSDB_NUMBER_DB_NEXT
ROCKSDB_NUMBER_DB_NEXT_FOUND
......@@ -1770,7 +1766,6 @@ ROCKSDB_NUMBER_DB_PREV
ROCKSDB_NUMBER_DB_PREV_FOUND
ROCKSDB_NUMBER_DB_SEEK
ROCKSDB_NUMBER_DB_SEEK_FOUND
ROCKSDB_NUMBER_DELETES_FILTERED
ROCKSDB_NUMBER_KEYS_READ
ROCKSDB_NUMBER_KEYS_UPDATED
ROCKSDB_NUMBER_KEYS_WRITTEN
......@@ -1806,7 +1801,6 @@ ROCKSDB_WAL_GROUP_SYNCS
ROCKSDB_WAL_SYNCED
ROCKSDB_WRITE_OTHER
ROCKSDB_WRITE_SELF
ROCKSDB_WRITE_TIMEDOUT
ROCKSDB_WRITE_WAL
#
# Fix issue #9: HA_ERR_INTERNAL_ERROR when running linkbench
......
storage/rocksdb/mysql-test/rocksdb/r/rocksdb_bottom_pri_compaction.result 0 → 100644
View file @ 485c9b1f
1
DROP TABLE IF EXISTS t1;
CREATE TABLE t1 (
a int not null,
b int not null,
primary key (a,b),
key (b)
) ENGINE=RocksDB;
DELETE FROM t1;
set @@global.rocksdb_compact_cf = 'default';
Bottom thread priority:
19
Bottom thread counts:
1
drop table t1;
storage/rocksdb/mysql-test/rocksdb/r/rocksdb_bottom_pri_compaction2.result 0 → 100644
View file @ 485c9b1f
1
Bottom thread counts:
5
"Increasing bottom pri compaction threads online."
SET GLOBAL rocksdb_max_bottom_pri_background_compactions=10;
1
Bottom thread counts:
10
"Decreasing bottom pri compaction threads online."
SET GLOBAL rocksdb_max_bottom_pri_background_compactions=2;
1
Bottom thread counts:
2
SET GLOBAL rocksdb_max_bottom_pri_background_compactions=0;
ERROR HY000: Error when executing command SET: max_bottom_pri_background_compactions can't be changed to 0 online.
SET GLOBAL rocksdb_max_bottom_pri_background_compactions=5;
storage/rocksdb/mysql-test/rocksdb/r/rocksdb_datadir.result
View file @ 485c9b1f
Check for MANIFEST files
MANIFEST-000006
MANIFEST-000005
storage/rocksdb/mysql-test/rocksdb/r/rocksdb_deadlock_detect_rc.result
View file @ 485c9b1f
......@@ -55,7 +55,6 @@ select * from t;
i
rollback;
connection con2;
i
rollback;
connection con1;
i
......
storage/rocksdb/mysql-test/rocksdb/r/rocksdb_deadlock_detect_rr.result
View file @ 485c9b1f
......@@ -55,7 +55,6 @@ select * from t;
i
rollback;
connection con2;
i
rollback;
connection con1;
i
......
storage/rocksdb/mysql-test/rocksdb/r/rocksdb_manual_compaction_bottommost_level_basic.result 0 → 100644
View file @ 485c9b1f
CREATE TABLE valid_values (value varchar(255)) ENGINE=myisam;
INSERT INTO valid_values VALUES('kSkip');
INSERT INTO valid_values VALUES('kIfHaveCompactionFilter');
INSERT INTO valid_values VALUES('kForce');
INSERT INTO valid_values VALUES('kForceOptimized');
CREATE TABLE invalid_values (value varchar(255)) ENGINE=myisam;
INSERT INTO invalid_values VALUES('\'aaa\'');
SET @start_global_value = @@global.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL;
SELECT @start_global_value;
@start_global_value
kForceOptimized
SET @start_session_value = @@session.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL;
SELECT @start_session_value;
@start_session_value
kForceOptimized
'# Setting to valid values in global scope#'
"Trying to set variable @@global.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL to kSkip"
SET @@global.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL = kSkip;
SELECT @@global.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL;
@@global.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL
kSkip
"Setting the global scope variable back to default"
SET @@global.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL = DEFAULT;
SELECT @@global.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL;
@@global.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL
kForceOptimized
"Trying to set variable @@global.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL to kIfHaveCompactionFilter"
SET @@global.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL = kIfHaveCompactionFilter;
SELECT @@global.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL;
@@global.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL
kIfHaveCompactionFilter
"Setting the global scope variable back to default"
SET @@global.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL = DEFAULT;
SELECT @@global.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL;
@@global.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL
kForceOptimized
"Trying to set variable @@global.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL to kForce"
SET @@global.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL = kForce;
SELECT @@global.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL;
@@global.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL
kForce
"Setting the global scope variable back to default"
SET @@global.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL = DEFAULT;
SELECT @@global.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL;
@@global.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL
kForceOptimized
"Trying to set variable @@global.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL to kForceOptimized"
SET @@global.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL = kForceOptimized;
SELECT @@global.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL;
@@global.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL
kForceOptimized
"Setting the global scope variable back to default"
SET @@global.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL = DEFAULT;
SELECT @@global.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL;
@@global.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL
kForceOptimized
'# Setting to valid values in session scope#'
"Trying to set variable @@session.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL to kSkip"
SET @@session.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL = kSkip;
SELECT @@session.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL;
@@session.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL
kSkip
"Setting the session scope variable back to default"
SET @@session.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL = DEFAULT;
SELECT @@session.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL;
@@session.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL
kForceOptimized
"Trying to set variable @@session.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL to kIfHaveCompactionFilter"
SET @@session.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL = kIfHaveCompactionFilter;
SELECT @@session.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL;
@@session.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL
kIfHaveCompactionFilter
"Setting the session scope variable back to default"
SET @@session.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL = DEFAULT;
SELECT @@session.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL;
@@session.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL
kForceOptimized
"Trying to set variable @@session.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL to kForce"
SET @@session.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL = kForce;
SELECT @@session.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL;
@@session.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL
kForce
"Setting the session scope variable back to default"
SET @@session.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL = DEFAULT;
SELECT @@session.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL;
@@session.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL
kForceOptimized
"Trying to set variable @@session.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL to kForceOptimized"
SET @@session.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL = kForceOptimized;
SELECT @@session.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL;
@@session.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL
kForceOptimized
"Setting the session scope variable back to default"
SET @@session.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL = DEFAULT;
SELECT @@session.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL;
@@session.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL
kForceOptimized
'# Testing with invalid values in global scope #'
"Trying to set variable @@global.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL to 'aaa'"
SET @@global.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL = 'aaa';
Got one of the listed errors
SELECT @@global.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL;
@@global.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL
kForceOptimized
SET @@global.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL = @start_global_value;
SELECT @@global.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL;
@@global.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL
kForceOptimized
SET @@session.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL = @start_session_value;
SELECT @@session.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL;
@@session.ROCKSDB_MANUAL_COMPACTION_BOTTOMMOST_LEVEL
kForceOptimized
DROP TABLE valid_values;
DROP TABLE invalid_values;
storage/rocksdb/mysql-test/rocksdb/r/truncate_failures.result 0 → 100644
View file @ 485c9b1f
set global rocksdb_strict_collation_exceptions = 't1,t2';
CREATE TABLE t1 (
pk INT,
a VARCHAR(64),
PRIMARY KEY (pk),
KEY (a)
) ENGINE=ROCKSDB DEFAULT CHARSET=latin1 COLLATE=latin1_general_ci;
INSERT INTO t1 VALUES (1, 'one');
RENAME TABLE t1 TO t1_trunc;
SELECT * FROM t1_trunc ORDER BY pk;
pk a
1 one
TRUNCATE TABLE t1_trunc;
SELECT * FROM t1_trunc;
pk a
CREATE TABLE t2 (
pk INT,
a VARCHAR(64),
PRIMARY KEY (pk),
KEY (a)
) ENGINE=ROCKSDB DEFAULT CHARSET=latin1 COLLATE=latin1_general_ci
PARTITION BY RANGE (pk) (
PARTITION p0 VALUES LESS THAN (3),
PARTITION p1 VALUES LESS THAN (6),
PARTITION p2 VALUES LESS THAN MAXVALUE
);
INSERT INTO t2 VALUES (1, 'one'), (4, 'four'), (7, 'seven');
RENAME TABLE t2 to t2_trunc;
SELECT * FROM t2_trunc ORDER BY pk;
pk a
1 one
4 four
7 seven
ALTER TABLE t2_trunc TRUNCATE PARTITION p0;
SELECT * FROM t2_trunc ORDER BY pk;
pk a
4 four
7 seven
TRUNCATE TABLE t2_trunc;
SELECT * FROM t2_trunc ORDER BY pk;
pk a
set global rocksdb_strict_collation_exceptions = default;
DROP TABLE t1_trunc, t2_trunc;
CREATE TABLE t1 (
pk INT PRIMARY KEY
) ENGINE=ROCKSDB;
INSERT INTO t1 VALUES (1), (2);
SELECT * FROM t1;
pk
1
2
SET DEBUG_DBUG = "+d,rocksdb_truncate_failure";
TRUNCATE TABLE t1;
ERROR HY000: Internal error: Simulated truncation failure.
SET DEBUG_DBUG = "-d,rocksdb_truncate_failure";
SELECT * FROM t1;
pk
1
2
CREATE TABLE t2 (
a INT
) ENGINE=ROCKSDB
PARTITION BY RANGE (a) (
PARTITION p0 VALUES LESS THAN (3),
PARTITION p1 VALUES LESS THAN (6),
PARTITION p2 VALUES LESS THAN MAXVALUE
);
INSERT INTO t2 VALUES (1), (4), (7);
SET DEBUG_DBUG = "+d,rocksdb_truncate_failure";
TRUNCATE TABLE t2;
ERROR HY000: Internal error: Simulated truncation failure.
ALTER TABLE t2 TRUNCATE PARTITION p1;
ERROR HY000: Internal error: Simulated truncation failure.
SET DEBUG_DBUG = "-d,rocksdb_truncate_failure";
SELECT * FROM t2;
a
1
4
7
DROP TABLE t1, t2;
CREATE TABLE t1_crash (
pk INT PRIMARY KEY
) ENGINE=ROCKSDB;
INSERT INTO t1_crash VALUES (100), (1000);
SELECT * FROM t1_crash;
pk
100
1000
SET DEBUG_DBUG = "+d,rocksdb_truncate_failure_crash";
TRUNCATE TABLE t1_crash;
ERROR HY000: Lost connection to server during query
MyRocks: Removing truncated leftover table test.#truncate_tmp#t1_crash
DROP TABLE t1_crash;
Warnings:
Warning 1932 Table 'test.t1_crash' doesn't exist in engine
storage/rocksdb/mysql-test/rocksdb/r/truncate_table3.result
View file @ 485c9b1f
call mtr.add_suppression("Column family 'cf1' not found");
call mtr.add_suppression("Column family 'rev:cf2' not found");
DROP TABLE IF EXISTS t1;
call mtr.add_suppression("Column family 'cf1' not found");
call mtr.add_suppression("Column family 'rev:cf2' not found");
set global rocksdb_compact_cf = 'cf1';
set global rocksdb_compact_cf = 'rev:cf2';
set global rocksdb_signal_drop_index_thread = 1;
# restart
CREATE TABLE t1 (
a int not null,
b int not null,
......@@ -15,6 +9,10 @@ primary key (a,b) comment 'cf1',
key (b) comment 'rev:cf2'
) ENGINE=RocksDB;
DELETE FROM t1;
set global rocksdb_compact_cf = 'cf1';
set global rocksdb_compact_cf = 'rev:cf2';
set global rocksdb_signal_drop_index_thread = 1;
# restart
select variable_value into @a from information_schema.global_status where variable_name='rocksdb_compact_read_bytes';
truncate table t1;
select case when variable_value-@a < 500000 then 'true' else 'false' end from information_schema.global_status where variable_name='rocksdb_compact_read_bytes';
......
storage/rocksdb/mysql-test/rocksdb/r/ttl_secondary_read_filtering.result
View file @ 485c9b1f
......@@ -368,6 +368,57 @@ a b
10 10
DROP TABLE t1;
CREATE TABLE t1 (
a int,
b int,
ts TIMESTAMP NOT NULL,
PRIMARY KEY (a),
KEY kb (b)
) ENGINE=rocksdb
COMMENT='ttl_duration=1;ttl_col=ts;';
set global rocksdb_debug_ttl_rec_ts = 100;
INSERT INTO t1 VALUES (1, 1, NOW());
INSERT INTO t1 VALUES (2, 2, NOW());
INSERT INTO t1 VALUES (3, 3, NOW());
INSERT INTO t1 VALUES (4, 4, NOW());
INSERT INTO t1 VALUES (5, 5, NOW());
INSERT INTO t1 VALUES (6, 6, NOW());
INSERT INTO t1 VALUES (7, 7, NOW());
INSERT INTO t1 VALUES (8, 8, NOW());
INSERT INTO t1 VALUES (9, 9, NOW());
INSERT INTO t1 VALUES (10, 10, NOW());
set global rocksdb_debug_ttl_rec_ts = 0;
set global rocksdb_force_flush_memtable_now=1;
# None are expired
SELECT a, b FROM t1 FORCE INDEX (kb);
a b
1 1
2 2
3 3
4 4
5 5
6 6
7 7
8 8
9 9
10 10
set global rocksdb_debug_ttl_rec_ts = -100;
UPDATE t1 SET ts=(ADDTIME(NOW(), 1)) WHERE a IN (4, 7);
set global rocksdb_debug_ttl_rec_ts = 0;
set global rocksdb_force_flush_memtable_now=1;
set global rocksdb_compact_cf='default';
# 4 and 7 should be gone
SELECT a, b FROM t1 FORCE INDEX (kb);
a b
1 1
2 2
3 3
5 5
6 6
8 8
9 9
10 10
DROP TABLE t1;
CREATE TABLE t1 (
c1 INT,
c2 INT,
name VARCHAR(25) NOT NULL,
......
storage/rocksdb/mysql-test/rocksdb/r/ttl_secondary_with_partitions.result
View file @ 485c9b1f
......@@ -334,6 +334,67 @@ c1
9
DROP TABLE t1;
CREATE TABLE t1 (
c1 BIGINT,
c2 TIMESTAMP DEFAULT NOW() NOT NULL,
name VARCHAR(25) NOT NULL,
event DATE,
PRIMARY KEY (`c1`) COMMENT 'custom_p0_cfname=foo;custom_p1_cfname=bar;custom_p2_cfname=baz;',
KEY kc2 (`c2`)
) ENGINE=ROCKSDB
COMMENT="ttl_duration=1;custom_p1_ttl_duration=100;custom_p1_ttl_col=c2;custom_p2_ttl_duration=5000;"
PARTITION BY LIST(c1) (
PARTITION custom_p0 VALUES IN (1, 2, 3),
PARTITION custom_p1 VALUES IN (4, 5, 6),
PARTITION custom_p2 VALUES IN (7, 8, 9)
);
set global rocksdb_debug_ttl_rec_ts = -300;
INSERT INTO t1 VALUES (1, NOW(), "one", null);
INSERT INTO t1 VALUES (2, NOW(), "two", null);
INSERT INTO t1 VALUES (3, NOW(), "three", null);
set global rocksdb_debug_ttl_rec_ts = 0;
INSERT INTO t1 VALUES (4, NOW(), "four", null);
INSERT INTO t1 VALUES (5, NOW(), "five", null);
INSERT INTO t1 VALUES (6, NOW(), "six", null);
INSERT INTO t1 VALUES (7, NOW(), "seven", null);
INSERT INTO t1 VALUES (8, NOW(), "eight", null);
INSERT INTO t1 VALUES (9, NOW(), "nine", null);
set global rocksdb_force_flush_memtable_now=1;
set @@global.rocksdb_compact_cf = 'foo';
set @@global.rocksdb_compact_cf = 'baz';
set @@global.rocksdb_compact_cf = 'bar';
set @@global.rocksdb_compact_cf = 'default';
SELECT c1 FROM t1 FORCE INDEX (PRIMARY);
c1
4
5
6
7
8
9
SELECT c1 FROM t1 FORCE INDEX (kc2);
c1
4
5
6
7
8
9
set global rocksdb_debug_ttl_snapshot_ts = 600;
set @@global.rocksdb_compact_cf = 'bar';
set @@global.rocksdb_compact_cf = 'default';
set global rocksdb_debug_ttl_snapshot_ts = 0;
SELECT c1 FROM t1 FORCE INDEX (PRIMARY);
c1
7
8
9
SELECT c1 FROM t1 FORCE INDEX (kc2);
c1
7
8
9
DROP TABLE t1;
CREATE TABLE t1 (
c1 INT,
c2 INT,
PRIMARY KEY (`c1`) COMMENT 'custom_p0_cfname=foo;'
......
storage/rocksdb/mysql-test/rocksdb/r/unique_sec.result
View file @ 485c9b1f
......@@ -219,3 +219,34 @@ CHECK TABLE t EXTENDED;
Table Op Msg_type Msg_text
test.t check status OK
DROP TABLE t;
#
# Validate lock acquired on SK during delete
#
CREATE TABLE t3 (pk INT PRIMARY KEY, sk INT, UNIQUE KEY (sk)) ENGINE=ROCKSDB;
INSERT INTO t3 VALUES (1, 100);
BEGIN;
# This should acquire a lock on sk = 100
DELETE FROM t3 WHERE pk = 1;
connect con1, localhost, root,,;
connection con1;
set session rocksdb_lock_wait_timeout=1;
set session transaction_isolation='REPEATABLE-READ';
SELECT * FROM t3;
pk sk
1 100
# RR: This should fail with lock wait timeout due to sk = 100
INSERT INTO t3 VALUES (2, 100);
ERROR HY000: Lock wait timeout exceeded; try restarting transaction
set session transaction_isolation='READ-COMMITTED';
# RC: This should fail with lock wait timeout due to sk = 100
INSERT INTO t3 VALUES (2, 100);
ERROR HY000: Lock wait timeout exceeded; try restarting transaction
connection default;
# The original transaction should be able to perform an insert
INSERT INTO t3 VALUES (3, 100);
COMMIT;
SELECT * FROM t3;
pk sk
3 100
disconnect con1;
DROP TABLE t3;
storage/rocksdb/mysql-test/rocksdb/suite.pm
View file @ 485c9b1f
......@@ -15,8 +15,8 @@ use strict;
my $sst_dump=
::mtr_exe_maybe_exists(
"$::bindir/storage/rocksdb$::multiconfig/sst_dump",
"$::path_client_bindir/sst_dump");
"$::bindir/storage/rocksdb$::multiconfig/mariadb-sst-dump",
"$::path_client_bindir/mariadb-sst-dump");
return "RocksDB is not compiled, no sst_dump" unless $sst_dump;
$ENV{MARIAROCKS_SST_DUMP}="$sst_dump";
......
storage/rocksdb/mysql-test/rocksdb/t/bloomfilter2.test
View file @ 485c9b1f
......@@ -4,7 +4,7 @@
CREATE TABLE t0 (id1 VARCHAR(30), id2 INT, value INT, PRIMARY KEY (id1, id2)) ENGINE=rocksdb collate latin1_bin;
--disable_query_log
let $i = 1;
while ($i <= 10000) {
while ($i <= 100000) {
let $insert = INSERT INTO t0 VALUES('X', $i, $i);
inc $i;
eval $insert;
......@@ -24,7 +24,7 @@ CREATE TABLE t1 (id1 BIGINT, id2 INT, id3 BIGINT, value INT, PRIMARY KEY (id1, i
--disable_query_log
let $i = 1;
while ($i <= 10000) {
while ($i <= 100000) {
let $insert = INSERT INTO t1 VALUES(1, 1, $i, $i);
eval $insert;
inc $i;
......@@ -48,7 +48,7 @@ DROP TABLE t1;
CREATE TABLE t2 (id1 INT, id2 VARCHAR(100), id3 BIGINT, value INT, PRIMARY KEY (id1, id2, id3)) ENGINE=rocksdb collate latin1_bin;
--disable_query_log
let $i = 1;
while ($i <= 10000) {
while ($i <= 100000) {
let $insert = INSERT INTO t2 VALUES($i, $i, $i, $i);
inc $i;
eval $insert;
......@@ -76,7 +76,7 @@ DROP TABLE t2;
CREATE TABLE t3 (id1 BIGINT, id2 BIGINT, id3 BIGINT, id4 BIGINT, PRIMARY KEY (id1, id2, id3, id4)) ENGINE=rocksdb collate latin1_bin;
--disable_query_log
let $i = 1;
while ($i <= 10000) {
while ($i <= 100000) {
if ($i != 5000) {
let $insert = INSERT INTO t3 VALUES(1, $i, $i, $i);
eval $insert;
......
storage/rocksdb/mysql-test/rocksdb/t/cancel_mc.test 0 → 100644
View file @ 485c9b1f
--source include/have_rocksdb.inc
DELIMITER //;
CREATE FUNCTION is_compaction_stopped() RETURNS INT
BEGIN
SELECT variable_value INTO @cwb_1 FROM information_schema.global_status WHERE variable_name="rocksdb_compact_write_bytes";
SELECT SLEEP(3) INTO @a;
SELECT variable_value INTO @cwb_2 FROM information_schema.global_status WHERE variable_name="rocksdb_compact_write_bytes";
IF @cwb_2 > @cwb_1 THEN
RETURN 0;
END IF;
RETURN 1;
END//
DELIMITER ;//
let $datadir = `SELECT @@datadir`;
--disable_warnings
DROP TABLE IF EXISTS t1;
--enable_warnings
create table t1 (id int auto_increment, value int, value2 varchar(500), primary key (id), index(value2) comment 'cf_i') engine=rocksdb;
let $datadir = `SELECT @@datadir`;
# generating test data
--let MYSQLD_DATADIR=$datadir
perl;
my $row_count = 500000;
my $datadir = $ENV{'MYSQLD_DATADIR'};
open(DATA, ">", $datadir . "/se_loaddata.dat") || die "Can't open file: $!\n";
my $value = 'x' x 250;
for (my $i= 1; $i <= $row_count; $i++) {
print DATA "$i,$i,$value\n";
}
close(DATA);
EOF
SET SESSION rocksdb_bulk_load_allow_sk=1;
SET SESSION rocksdb_bulk_load=1;
--replace_result $datadir <DATADIR>
eval
LOAD DATA INFILE '$datadir/se_loaddata.dat' INTO TABLE t1
FIELDS TERMINATED BY ',' (id, value, value2);
SET SESSION rocksdb_bulk_load=0;
SET SESSION rocksdb_bulk_load_allow_sk=0;
--remove_file $datadir/se_loaddata.dat
--connect (con1,localhost,root,,)
--connect (con2,localhost,root,,)
# Note that finishing cancelling MC may take a few seconds after instruction,
# so testing with very large datasets or slowing down compaction is needed
# to make tests reliable. rocksdb_rate_limiter_bytes_per_sec=256k
# (in -master.opt) is an option to do the latter so that RocksDB has enough
# time to cancel compactions.
--echo #
--echo # Cancelling MC, including pending MCs, by KILL statements
--echo #
connection default;
SELECT variable_value into @executed_start FROM information_schema.global_status WHERE variable_name="rocksdb_manual_compactions_cancelled";
connection con1;
--let $con1_id = `SELECT CONNECTION_ID()`
send SET GLOBAL rocksdb_compact_cf='default';
connection con2;
--let $con2_id = `SELECT CONNECTION_ID()`
set session rocksdb_manual_compaction_threads=2;
send SET GLOBAL rocksdb_compact_cf='cf_i';
connection default;
let $wait_condition=
select variable_value = 1 from information_schema.global_status WHERE variable_name="rocksdb_manual_compactions_running";
--source include/wait_condition.inc
let $wait_condition=
select variable_value = 1 from information_schema.global_status WHERE variable_name="rocksdb_manual_compactions_pending";
--source include/wait_condition.inc
--disable_query_log
eval kill query $con1_id;
eval kill query $con2_id;
--enable_query_log
connection con1;
--error ER_INTERNAL_ERROR
reap;
connection con2;
--error ER_INTERNAL_ERROR
reap;
connection default;
let $wait_condition=
select variable_value = 0 from information_schema.global_status WHERE variable_name="rocksdb_manual_compactions_pending";
--source include/wait_condition.inc
SELECT variable_value into @executed_end FROM information_schema.global_status WHERE variable_name="rocksdb_manual_compactions_cancelled";
SELECT @executed_end-@executed_start AS CANCELLED_MC;
let $wait_condition=
select is_compaction_stopped() = 1 from dual;
--source include/wait_condition.inc
--echo #
--echo # Cancelling MC by global opt
--echo #
connection default;
SELECT variable_value into @executed_start FROM information_schema.global_status WHERE variable_name="rocksdb_manual_compactions_cancelled";
connection con1;
set session rocksdb_manual_compaction_threads=2;
send SET GLOBAL rocksdb_compact_cf='default';
# waiting for compaction to actually start
connection default;
let $wait_condition=
select variable_value = 1 from information_schema.global_status WHERE variable_name="rocksdb_manual_compactions_running";
--source include/wait_condition.inc
# Issuing second MC (pending)
connection con2;
send SET GLOBAL rocksdb_compact_cf='cf_i';
connection default;
let $wait_condition=
select variable_value = 1 from information_schema.global_status WHERE variable_name="rocksdb_manual_compactions_pending";
--source include/wait_condition.inc
set global rocksdb_cancel_manual_compactions=ON;
# first MC (running state) is cancelled
connection con1;
--error ER_INTERNAL_ERROR
reap;
# second MC (pending state) is also cancelled
connection con2;
--error ER_INTERNAL_ERROR
reap;
connection default;
SELECT variable_value into @executed_end FROM information_schema.global_status WHERE variable_name="rocksdb_manual_compactions_cancelled";
SELECT @executed_end-@executed_start AS CANCELLED_MC;
let $wait_condition=
select is_compaction_stopped() = 1 from dual;
--source include/wait_condition.inc
--echo #
--echo # Cancelling OPTIMIZE TABLE by global opt
--echo #
connection con1;
send OPTIMIZE TABLE t1;
connection default;
SELECT variable_value into @current_cwb FROM information_schema.global_status WHERE variable_name="rocksdb_compact_write_bytes";
let $wait_condition=
select count(*) = 1 from information_schema.processlist
where info IN ("OPTIMIZE TABLE t1");
--source include/wait_condition.inc
# compacting more than 100kb (at least a few percent)
let $wait_condition=
select variable_value > (@current_cwb + 100000) FROM information_schema.global_status WHERE variable_name="rocksdb_compact_write_bytes";
--source include/wait_condition.inc
set global rocksdb_cancel_manual_compactions=ON;
connection con1;
reap;
connection default;
select 'PRIMARY' as idx, COUNT(*) as cnt FROM t1 FORCE INDEX (PRIMARY)
UNION ALL SELECT 'value2' as idx, COUNT(*) as cnt FROM t1 FORCE INDEX (value2);
--echo #
--echo # Dropping Indexes complete even after cancelling MC
--echo #
INSERT INTO t1 (id) VALUES (null);
set global rocksdb_force_flush_memtable_now=ON;
SELECT variable_value into @current_cwb FROM information_schema.global_status WHERE variable_name="rocksdb_compact_write_bytes";
DROP TABLE t1;
let $wait_condition=
select variable_value > @current_cwb FROM information_schema.global_status WHERE variable_name="rocksdb_compact_write_bytes";
set global rocksdb_cancel_manual_compactions=ON;
let $wait_condition = select count(*) = 0
as c from information_schema.rocksdb_global_info
where TYPE = 'DDL_DROP_INDEX_ONGOING';
--source include/wait_condition.inc
DROP FUNCTION is_compaction_stopped;
storage/rocksdb/mysql-test/rocksdb/t/cancel_mc_timeout.test 0 → 100644
View file @ 485c9b1f
--source include/have_rocksdb.inc
--connect (con1,localhost,root,,)
--connect (con2,localhost,root,,)
connection default;
# longer than compaction wait timeout (60 seconds)
set global rocksdb_debug_manual_compaction_delay=80;
connection con1;
--let $con1_id = `SELECT CONNECTION_ID()`
send SET GLOBAL rocksdb_compact_cf='default';
connection default;
let $wait_condition=
select variable_value = 1 from information_schema.global_status WHERE variable_name="rocksdb_manual_compactions_running";
--source include/wait_condition.inc
connection con2;
--let $con2_id = `SELECT CONNECTION_ID()`
set session rocksdb_manual_compaction_threads=2;
send SET GLOBAL rocksdb_compact_cf='default';
connection default;
let $wait_condition=
select variable_value = 1 from information_schema.global_status WHERE variable_name="rocksdb_manual_compactions_pending";
--source include/wait_condition.inc
--disable_query_log
eval kill query $con1_id;
eval kill query $con2_id;
--enable_query_log
# pending compaction can be cancelled instantly
connection con2;
--error ER_INTERNAL_ERROR
reap;
# running compaction hits timeout
connection con1;
--error ER_INTERNAL_ERROR
reap;
connection default;
set global rocksdb_debug_manual_compaction_delay=0;
# next compaction starts after previous (running) compaction finishes
connection con1;
SET GLOBAL rocksdb_compact_cf='default';
storage/rocksdb/mysql-test/rocksdb/t/disabled.def
View file @ 485c9b1f
......@@ -69,6 +69,7 @@ gap_lock_issue254: MDEV-11735: MyRocks: Gap Lock detector support
gap_lock_raise_error: MDEV-11735: MyRocks: Gap Lock detector support
show_engine : MariaRocks: MariaDB doesnt support SHOW ENGINE rocksdb TRANSACTION STATUS
issue243_transactionStatus: MariaDB doesnt support SHOW ENGINE rocksdb TRANSACTION STATUS
max_row_locks: MariaDB doesnt support SHOW ENGINE rocksdb TRANSACTION STATUS
rpl_row_not_found : MariaDB doesnt support slave_exec_mode='SEMI_STRICT'
rpl_row_not_found_rc : MariaDB doesnt support slave_exec_mode='SEMI_STRICT'
......
storage/rocksdb/mysql-test/rocksdb/t/drop_table3.inc
View file @ 485c9b1f
--source include/have_rocksdb.inc
call mtr.add_suppression("Column family 'cf1' not found");
call mtr.add_suppression("Column family 'rev:cf2' not found");
--disable_warnings
DROP TABLE IF EXISTS t1;
--enable_warnings
......@@ -10,12 +7,6 @@ DROP TABLE IF EXISTS t1;
call mtr.add_suppression("Column family 'cf1' not found");
call mtr.add_suppression("Column family 'rev:cf2' not found");
# Start from clean slate
set global rocksdb_compact_cf = 'cf1';
set global rocksdb_compact_cf = 'rev:cf2';
set global rocksdb_signal_drop_index_thread = 1;
--source include/restart_mysqld.inc
CREATE TABLE t1 (
a int not null,
b int not null,
......@@ -29,6 +20,12 @@ let $max = 50000;
let $table = t1;
--source drop_table3_repopulate_table.inc
# Start from clean slate
set global rocksdb_compact_cf = 'cf1';
set global rocksdb_compact_cf = 'rev:cf2';
set global rocksdb_signal_drop_index_thread = 1;
--source include/restart_mysqld.inc
select variable_value into @a from information_schema.global_status where variable_name='rocksdb_compact_read_bytes';
if ($truncate_table)
{
......
storage/rocksdb/mysql-test/rocksdb/t/information_schema.test
View file @ 485c9b1f
......@@ -66,6 +66,9 @@ select * from INFORMATION_SCHEMA.ROCKSDB_GLOBAL_INFO where TYPE = 'CF_FLAGS';
CREATE TABLE t3 (a INT, PRIMARY KEY (a)) ENGINE=ROCKSDB;
insert into t3 (a) values (1), (2), (3);
SELECT IF(count(*) > 0, "TRUE", "FALSE") as metadata_exist FROM INFORMATION_SCHEMA.ROCKSDB_LIVE_FILES_METADATA where CF_NAME = 'default';
SET @ORIG_ROCKSDB_PAUSE_BACKGROUND_WORK = @@GLOBAL.ROCKSDB_PAUSE_BACKGROUND_WORK;
--let $t3_index_id = query_get_value(SELECT * FROM INFORMATION_SCHEMA.ROCKSDB_DDL WHERE TABLE_NAME = 't3', INDEX_NUMBER, 1)
--let $t3_cf_id = query_get_value(SELECT * FROM INFORMATION_SCHEMA.ROCKSDB_DDL WHERE TABLE_NAME = 't3', COLUMN_FAMILY, 1)
......
storage/rocksdb/mysql-test/rocksdb/t/issue243_transactionStatus.test
View file @ 485c9b1f
......@@ -76,5 +76,34 @@ DELETE FROM t2 WHERE id1=10;
--replace_regex /[0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2}/TIMESTAMP/ /(ACTIVE) [0-9]+ /\1 NUM / /(thread id) [0-9]+/\1 TID/ /0x[0-9a-f]+/PTR/ /(query id) [0-9]+/\1 QID/ /(root) [a-z ]+/\1 ACTION/
SHOW ENGINE rocksdb TRANSACTION STATUS;
ROLLBACK;
SET AUTOCOMMIT=1;
DROP TABLE t2;
#
# Secondary Key with uniq key Tests
#
--disable_warnings
DROP TABLE IF EXISTS t2;
--enable_warnings
CREATE TABLE t2 (
id1 INT,
id2 INT,
value INT,
PRIMARY KEY (id1),
UNIQUE KEY (id2)
) ENGINE=rocksdb;
SET AUTOCOMMIT=0;
START TRANSACTION;
INSERT INTO t2 VALUES(1,2,0),(10,20,30);
UPDATE t2 SET value=3 WHERE id2=2;
DELETE FROM t2 WHERE id1=10;
--replace_regex /[0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2}/TIMESTAMP/ /(ACTIVE) [0-9]+ /\1 NUM / /(thread id) [0-9]+/\1 TID/ /0x[0-9a-f]+/PTR/ /(query id) [0-9]+/\1 QID/ /(root) [a-z ]+/\1 ACTION/
SHOW ENGINE rocksdb TRANSACTION STATUS;
ROLLBACK;
SET AUTOCOMMIT=1;
DROP TABLE t2;
storage/rocksdb/mysql-test/rocksdb/t/issue896.test
View file @ 485c9b1f
......@@ -15,3 +15,18 @@ EXPLAIN SELECT COUNT(*) FROM t1 FORCE INDEX(d);
--echo # segfault here without the fix
SELECT COUNT(*) FROM t1 FORCE INDEX(d);
DROP TABLE t1;
CREATE TABLE `t1` (
`a` bigint(20) NOT NULL,
`b` varchar(10) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL,
`u` timestamp NOT NULL,
`d` bigint(20) DEFAULT NULL,
PRIMARY KEY (`a`,`b`),
KEY `d` (`d`)
) ENGINE=ROCKSDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin COMMENT='ttl_duration=1000;ttl_col=u';
INSERT INTO t1 VALUES (100, 'aaabbb', NOW(), 200);
--replace_column 9 #
EXPLAIN SELECT COUNT(*) FROM t1;
--echo # segfault here without the fix
SELECT COUNT(*) FROM t1;
DROP TABLE t1;
storage/rocksdb/mysql-test/rocksdb/t/manual_compaction_bottommost_level-master.opt 0 → 100644
View file @ 485c9b1f
--rocksdb_max_subcompactions=1
--rocksdb_default_cf_options=target_file_size_base=100k;max_bytes_for_level_multiplier=1;max_bytes_for_level_base=1m;target_file_size_multiplier=1
storage/rocksdb/mysql-test/rocksdb/t/manual_compaction_bottommost_level.test 0 → 100644
View file @ 485c9b1f
--source include/have_rocksdb.inc
DELIMITER //;
CREATE PROCEDURE compact_start()
BEGIN
select variable_value into @c from information_schema.global_status where variable_name='rocksdb_compact_write_bytes';
END//
CREATE PROCEDURE compact_end()
BEGIN
select case when variable_value-@c > 0 then 'true' else 'false' end as checked from information_schema.global_status where variable_name='rocksdb_compact_write_bytes';
END//
DELIMITER ;//
--disable_warnings
DROP TABLE IF EXISTS t1;
--enable_warnings
CREATE TABLE t1 (
a int not null,
b int not null,
primary key (a,b),
key (b)
) ENGINE=RocksDB;
# Populate tables
let $max = 1000;
let $table = t1;
--source drop_table_repopulate_table.inc
# compact to move all data in files in Lmax level
set @@session.rocksdb_manual_compaction_bottommost_level='kSkip';
call compact_start();
set @@global.rocksdb_compact_cf = 'default';
call compact_end(); # should return true as compaction of lower levels is performed
# skip compaction of files in bottommost level i.e. Lmax->Lmax compaction and ensure rocksdb_compact_write_byte
# does not increase after compaction
set @@session.rocksdb_manual_compaction_bottommost_level='kSkip';
call compact_start();
set @@global.rocksdb_compact_cf = 'default';
call compact_end(); # should return false as files only in bottommost layer
# restore 'kForceOptimized' default setting and check that value of rocksdb_compact_write_byte increses after compaction
set @@session.rocksdb_manual_compaction_bottommost_level='kForceOptimized';
call compact_start();
set @@global.rocksdb_compact_cf = 'default';
call compact_end(); # should return true
# cleanup
DROP PROCEDURE compact_start;
DROP PROCEDURE compact_end;
drop table t1;
storage/rocksdb/mysql-test/rocksdb/t/max_row_locks.test 0 → 100644
View file @ 485c9b1f
This diff is collapsed.
storage/rocksdb/mysql-test/rocksdb/t/rocksdb.test
View file @ 485c9b1f
......@@ -1150,9 +1150,10 @@ CREATE TABLE t1 (pk INT PRIMARY KEY, a int) ENGINE=RocksDB;
set @a=-1;
insert into t1 select (@a:=@a+1), 1234 from information_schema.session_variables limit 100;
set @tmp1= @@rocksdb_max_row_locks;
set rocksdb_max_row_locks= 20;
set GLOBAL rocksdb_max_row_locks= 20;
--error ER_GET_ERRMSG
update t1 set a=a+10;
set @@global.rocksdb_max_row_locks = @tmp1;
DROP TABLE t1;
......
storage/rocksdb/mysql-test/rocksdb/t/truncate_failures.test 0 → 100644
View file @ 485c9b1f
This diff is collapsed.
storage/rocksdb/mysql-test/rocksdb/t/ttl_primary.inc 0 → 100644
View file @ 485c9b1f
This diff is collapsed.
storage/rocksdb/mysql-test/rocksdb/t/ttl_secondary.inc 0 → 100644
View file @ 485c9b1f
This diff is collapsed.
storage/rocksdb/nosql_access.cc deleted 100644 → 0
View file @ c035f3af
This diff is collapsed.
storage/rocksdb/nosql_access.h deleted 100644 → 0
View file @ c035f3af
This diff is collapsed.
storage/rocksdb/rdb_cf_manager.cc
View file @ 485c9b1f
This diff is collapsed.
storage/rocksdb/rdb_cf_manager.h
View file @ 485c9b1f
This diff is collapsed.
storage/rocksdb/rdb_cf_options.cc
View file @ 485c9b1f
This diff is collapsed.
storage/rocksdb/rdb_converter.cc
View file @ 485c9b1f
This diff is collapsed.
storage/rocksdb/rdb_converter.h
View file @ 485c9b1f
This diff is collapsed.
storage/rocksdb/rdb_datadic.cc
View file @ 485c9b1f
This diff is collapsed.
storage/rocksdb/rdb_datadic.h
View file @ 485c9b1f
This diff is collapsed.
storage/rocksdb/rdb_i_s.cc
View file @ 485c9b1f
This diff is collapsed.
storage/rocksdb/rdb_i_s.h
View file @ 485c9b1f
This diff is collapsed.
storage/rocksdb/rdb_psi.cc
View file @ 485c9b1f
This diff is collapsed.
storage/rocksdb/rdb_psi.h
View file @ 485c9b1f
This diff is collapsed.
storage/rocksdb/rdb_threads.h
View file @ 485c9b1f
This diff is collapsed.
rocksdb @ 6a436150
This diff is collapsed.
storage/rocksdb/tools/mysql_ldb.cc
View file @ 485c9b1f
This diff is collapsed.
storage/rocksdb/tools/mysql_sst_dump.cc 0 → 100644
View file @ 485c9b1f
This diff is collapsed.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7