Commit 0cbabcb8 authored by Narayanan V's avatar Narayanan V

Bug#44856 IBMDB2I gives misleading 2504 error

Occasionally, if both the partition_pruning
and partition_range tests are run sequentially
against the IBMDB2I engine, the partition_range
test will fail.

Compiler padding on a 64-bit build allowed
garbage data in the hash key used for
caching open iconv descriptors. As a
result, cached descriptors were not found,
and multiple duplicate iconv descriptors
were opened for a single character set.
Eventually, the maximum number of open
iconv descriptors was reached, and further
iconv_open() calls would fail, leading the
storage engine to report incorrectly that
the character set was not supported.

This patch widens the 16-bit members of the
hash key to 32 bits to eliminate compiler
padding. The entire length of the hash key
is now initialized correctly on both 32-bit
and 64-bit builds.

storage/ibmdb2i/db2i_charsetSupport.cc:
  Bug#44856 IBMDB2I gives misleading 2504 error
  
  widen the 16-bit members of the
  hash key to 32 bits to eliminate
  compiler padding.
parent 3a854447
......@@ -129,8 +129,8 @@ struct IconvMap
{
struct HashKey
{
uint16 direction; // This is a uint16 instead of a uchar to avoid garbage data in the key from compiler padding
uint16 db2CCSID;
uint32 direction; // These are uint32s to avoid garbage data in the key from compiler padding
uint32 db2CCSID;
const CHARSET_INFO* myCharset;
} hashKey;
iconv_t iconvDesc;
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment