storage engine returns errno 12

If there is not enough memory to store or update blob record
(while allocating record buffer), myisam marks table as crashed.

With this fix myisam attempts to roll an index back and return
an error, not marking a table as crashed.

Affects myisam tables with blobs only. No test case for this fix.


myisam/mi_dynrec.c:
  If there is not enough memory to store or update blob record
  (while allocating record buffer), return HA_ERR_OUT_OF_MEM
  instead of ENOMEM. In this case storage engine can simply
  roll an index back and return an error, not marking table
  as crashed.
myisam/mi_update.c:
  In some cases do not mark a table as crashed if we run out of
  memory. Instead roll an index back and return an error. These
  cases are signalled with my_errno set to HA_ERR_OUT_OF_MEM.
myisam/mi_write.c:
  In some cases do not mark a table as crashed if we run out of
  memory. Instead roll an index back and return an error. These
  cases are signalled with my_errno set to HA_ERR_OUT_OF_MEM.

into  mysql.com:/home/svoj/devel/mysql/BUG23312/mysql-4.1-engines

            tables,alter table
Deadlock could happen if there are delayed insert + flush tables + alter table
running concurrently.

This is fixed by removing a redundant mutex lock when killing a delayed thread.


sql/sql_insert.cc:
  Removed redundant delayed thread mutex lock, that could result in a deadlock.
  LOCK_delayed_insert is sufficient to be sure that no other thread frees delayed
  handler.

into  chilla.local:/home/mydev/mysql-4.1-merge

into  chilla.local:/home/mydev/mysql-4.1-bug22119

When compiling with a default key block size greater than the
smallest key block size used in a table, checking that table
failed with bogus errors. The table was marked corrupt. This
affected myisamchk and the server.

The problem was that the default key block size was used at
some places where sizes less or equal to the block size of the
index in check was required.

We do now use the key block size of the particular index
when checking.

A test case is available for later versions only.


myisam/mi_check.c:
  Bug#22119 - Changing MI_KEY_BLOCK_LENGTH makes a wrong myisamchk
  Changed check_k_link() and chk_index_down() to use the block
  size of the index in check or MI_MIN_KEY_BLOCK_LENGTH where
  required. Formerly myisam_block_size or MYISAM_SHARE::blocksize
  was used wrongly.

            really damaged MyISAM tables

When unpacking a blob column from broken row server crash
could happen. This could rather happen when trying to repair
a table using either REPAIR TABLE or myisamchk, though it
also could happend when trying to access broken row using
other SQL statements like SELECT if table is not marked as
crashed.

Fixed ulong overflow when trying to extract blob from
broken row.

Affects MyISAM only.


myisam/mi_dynrec.c:
  Fixed ulong overflow when trying to extract blob from
  broken row. It happens when there are not enough bytes
  to store blob length in `from' buffer. In this case
  (ulong) (from_end - from) - size_length value is huge,
  close to ULONG_MAX.

into  dl145s.mysql.com:/data/bk/team_tree_merge/MERGE/mysql-4.1-opt


sql/sql_base.cc:
  Auto merged
sql/sql_select.cc:
  Auto merged

into  mysql.com:/home/svoj/devel/mysql/engines/mysql-4.1-engines


mysql-test/r/repair.result:
  Manual merge.
mysql-test/t/repair.test:
  Manual merge.

into  dl145s.mysql.com:/data/bk/team_tree_merge/MERGE/mysql-4.1-opt


sql/sql_select.cc:
  Auto merged

into  mysql.com:/home/svoj/devel/mysql/engines/mysql-4.1-engines

into  chilla.local:/home/mydev/mysql-4.1-merge

Repair table could crash a server if there is not sufficient
memory (myisam_sort_buffer_size) to operate. Affects not only
repair, but also all statements that use create index by sort:
repair by sort, parallel repair, bulk insert.

Return an error if there is not sufficient memory to store at
least one key per BUFFPEK.

Also fixed memory leak if thr_find_all_keys returns an error.


myisam/sort.c:
  maxbuffer is number of BUFFPEK-s for repair. It is calculated
  as records / keys. keys is number of keys that can be stored
  in memory (myisam_sort_buffer_size). There must be sufficient
  memory to store both BUFFPEK-s and keys. It was checked
  correctly before this patch. However there is another
  requirement that wasn't checked: there must be sufficient
  memory for at least one key per BUFFPEK, otherwise repair
  by sort/parallel repair cannot operate.
  
  Return an error if there is not sufficient memory to store at
  least one key per BUFFPEK.
  
  Also fixed memory leak if thr_find_all_keys returns an error.
mysql-test/r/repair.result:
  A test case for BUG#23175.
mysql-test/t/repair.test:
  A test case for BUG#23175.

into  chilla.local:/home/mydev/mysql-4.1-bug12240


sql/sql_select.cc:
  Auto merged

into  rakia.(none):/home/kgeorge/mysql/autopush/B14019-4.1-opt

into  chilla.local:/home/mydev/mysql-4.1-bug12240

   When resolving unqualified name references MySQL was not
   checking what is the item type for the reference. Thus
   e.g a string literal item that has by convention a name
   equal to its string value will also work as a reference to 
   a SELECT list item or a table field.
   Fixed by allowing only Item_ref or Item_field to referenced by
   (unqualified) name.


mysql-test/r/func_gconcat.result:
  Bug #14019: group by converts literal string to column name
   - removed undeterministic testcase : order by a constant 
     means no order.
mysql-test/r/group_by.result:
  Bug #14019: group by converts literal string to column name
   - test case
mysql-test/t/func_gconcat.test:
  Bug #14019: group by converts literal string to column name
   - removed undeterministic testcase : order by a constant 
     means no order.
mysql-test/t/group_by.test:
  Bug #14019: group by converts literal string to column name
   - test case
sql/sql_base.cc:
  Bug #14019: group by converts literal string to column name
   - resolve unqualified by name refs only for real references

into  moonlight.intranet:/home/tomash/src/mysql_ab/mysql-4.1-bug9678

into  moonlight.intranet:/home/tomash/src/mysql_ab/mysql-4.1-bug9678


BitKeeper/deleted/.del-lib_vio.c~d779731a1e391220:
  Use local.
include/violite.h:
  Use local.
sql/net_serv.cc:
  Use local.
vio/viosocket.c:
  Use local.

into  chilla.local:/home/mydev/mysql-4.1-bug8283-one


include/my_sys.h:
  Auto merged

            hangs on Linux

If REPAIR TABLE ... USE_FRM is issued for table that is located in different
than default database server crash could happen.

In reopen_name_locked_table take database name from table_list (user specified
or default database) instead of from thd (default database).

Affects 4.1 only.


mysql-test/r/repair.result:
  A test case for BUG#22562.
mysql-test/t/repair.test:
  A test case for BUG#22562.
sql/sql_base.cc:
  In reopen_name_locked_table take database name from table_list (user specified
  or default database) instead of from thd (default database).

Examined rows are counted for every join part. The per-join-part
counter was incremented over all iterations. The result variable
was replaced at the end of every iteration. The final result was
the number of examined rows by the join part that ended its
execution as the last one. The numbers of other join parts was
lost.

Now we reset the per-join-part counter before every iteration and
add it to the result variable at the end of the iteration. That
way we get the sum of all iterations of all join parts.

No test case. Testing this needs a look into the slow query log.
I don't know of a way to do this portably with the test suite.


sql/sql_select.cc:
  Bug#12240 - Rows Examined in Slow Log showing incorrect number?
  Fixed reseting and accumulation of examined rows counts.

into  neptunus.(none):/home/msvensson/mysql/mysql-4.1

into  mysql.com:/home/bk/MERGE/mysql-4.1-merge

into  chilla.local:/home/mydev/mysql-4.1-bug8283-one


myisam/mi_check.c:
  Auto merged
myisam/mi_packrec.c:
  Auto merged
myisam/sort.c:
  Auto merged
mysql-test/r/myisam.result:
  Bug#8283 - OPTIMIZE TABLE causes data loss
  Manual merge
mysql-test/t/myisam.test:
  Bug#8283 - OPTIMIZE TABLE causes data loss
  Manual merge

OPTIMIZE TABLE with myisam_repair_threads > 1 performs a non-quick 
parallel repair. This means that it does not only rebuild all 
indexes, but also the data file.

Non-quick parallel repair works so that there is one thread per 
index. The first of the threads rebuilds also the new data file.

The problem was that all threads shared the read io cache on the
old data file. If there were holes (deleted records) in the table,
the first thread skipped them, writing only contiguous, non-deleted
records to the new data file. Then it built the new index so that
its entries pointed to the correct record positions. But the other
threads didn't know the new record positions, but put the positions
from the old data file into the index.

The new design is so that there is a shared io cache which is filled
by the first thread (the data file writer) with the new contiguous
records and read by the other threads. Now they know the new record
positions.

Another problem was that for the parallel repair of compressed
tables a common bit_buff and rec_buff was used. I changed it so
that thread specific buffers are used for parallel repair.

A similar problem existed for checksum calculation. I made this
multi-thread safe too.


include/my_sys.h:
  Bug#8283 - OPTIMIZE TABLE causes data loss
  Redesign of io_cache_share.
include/myisam.h:
  Bug#8283 - OPTIMIZE TABLE causes data loss
  Redesign of checksum calculation in mi_check.c.
  'calc_checksum' is now in myisamdef.h:st_mi_sort_param.
myisam/mi_check.c:
  Bug#8283 - OPTIMIZE TABLE causes data loss
  Implemented a new parallel repair design.
  Using a synchronized shared read/write cache.
  Allowed for thread specific bit_buff, rec_buff, and calc_checksum.
myisam/mi_open.c:
  Bug#8283 - OPTIMIZE TABLE causes data loss
  Added DBUG output.
myisam/mi_packrec.c:
  Bug#8283 - OPTIMIZE TABLE causes data loss
  Allowed for thread specific bit_buff and rec_buff.
myisam/myisamdef.h:
  Bug#8283 - OPTIMIZE TABLE causes data loss
  Commented on checksum calculation variables.
  Allowed for thread specific bit_buff.
  Added DBUG output for better table crash detection.
myisam/sort.c:
  Bug#8283 - OPTIMIZE TABLE causes data loss
  Added implications of the new parallel repair design.
  Renamed 'info' -> 'sort_param'.
  Added DBUG output.
mysql-test/r/myisam.result:
  Bug#8283 - OPTIMIZE TABLE causes data loss
  Added test results.
mysql-test/t/myisam.test:
  Bug#8283 - OPTIMIZE TABLE causes data loss
  Added test cases.
mysys/mf_iocache.c:
  Bug#8283 - OPTIMIZE TABLE causes data loss
  Redesign of io_cache_share.
  We do now allow a writer to synchronize himself with the
  readers of a shared cache. When all threads join in the lock,
  the writer copies the data from his write buffer to the shared
  read buffer.

into  may.pils.ru:/home/svoj/devel/bk/mysql-4.1-engines

into  mysql.com:/home/svoj/devel/mysql/BUG22937/mysql-4.1-engines

This is addition to fix for bug21617. Valgrind reports an error when
opening merge table that has underlying tables with less indexes than
in a merge table itself.

Copy at most min(file->keys, table->key_parts) elements from rec_per_key array.
This fixes problems when merge table and subtables have different number of keys.


sql/ha_myisammrg.cc:
  Copy at most min(file->keys, table->key_parts) elements from rec_per_key array.
  This fixes problems when merge table and subtables have different number of keys.

into  mysql.com:/home/svoj/devel/mysql/BUG21381/mysql-4.1-engines

Though this is not storage engine specific problem, I was able to
repeat this problem with BDB and NDB engines only. That was the
reason to add a test case into ndb_update.test. As a result
different bad things could happen.

BDB has removed duplicate rows which is not expected.
NDB returns an error.

For multi table update notify storage engine about UPDATE IGNORE
as it is done in single table UPDATE.


mysql-test/r/ndb_update.result:
  A test case for bug#21381.
mysql-test/t/ndb_update.test:
  A test case for bug#21381.
sql/sql_update.cc:
  For multi table update notify storage engine about UPDATE IGNORE
  as it is done in single table UPDATE.

into  mysql.com:/home/gluh/MySQL/Merge/4.1-kt


include/m_ctype.h:
  Auto merged
mysql-test/r/ctype_utf8.result:
  Auto merged
mysql-test/t/ctype_utf8.test:
  Auto merged
sql/table.cc:
  Auto merged
sql/unireg.cc:
  Auto merged

into  neptunus.(none):/home/msvensson/mysql/mysql-4.1-maint

into  neptunus.(none):/home/msvensson/mysql/mysql-4.1-maint

into  neptunus.(none):/home/msvensson/mysql/mysql-4.1-maint


sql/item_func.cc:
  Auto merged

into  neptunus.(none):/home/msvensson/mysql/mysql-4.1-maint

into  shellback.(none):/home/msvensson/mysql/mysql-4.1-maint

into  mysql.com:/usr/home/bar/mysql-4.1-rpl


mysql-test/r/cast.result:
  Auto merged
mysql-test/r/ps.result:
  Auto merged
sql/item_func.cc:
  Auto merged