Accessing a file that is bigger than 2G may report that read/write operation
failed. This may affect anything that uses my_pread/my_pwrite functions, e.g.
MyISAM, ARCHIVE, binary log.

For MyISAM INSERT may report that table is crashed when writing to a table
that is bigger than 2G.

This is fixed by using proper offset type in my_pread/my_pwrite functions on
systems that do not have native pread/pwrite calls.

Affects systems that do not have native pread/pwrite calls, e.g. Windows.

No test case for this fix, since it requires huge table.


mysys/my_pread.c:
  Use proper offset type for restoring position on systems that do not have native
  pread/pwrite calls.

* Modified Federated memory allocation to use MEM_ROOT
* Modified sql_servers and federated to allocate share connection
  parameters to use MEM_ROOT
* Modified Federated to allow tablename in addition to server name
* Implicit flushing of tables using altered/dropped server name
* Added tests to prove new functionality works

Contributors to this patch: Patrick Galbraith, Antony Curtis


mysql-test/r/federated_server.result:
  BUG #26257 New Federated Server Functionality Doesn't support differently named tables
  
    New test results
mysql-test/t/federated_server.test:
  BUG #26257 New Federated Server Functionality Doesn't support differently named tables
  
  New test which ensures that one can use the new 'create server'
  functionality and have tables point to the correct table, using CONNECTION='server',
  CONNECTION="server/tablename" and CONNECTION="mysql://...url"
sql/mysql_priv.h:
  BUG #26257 New Federated Server Functionality Doesn't support differently named tables
  
  new function: close_cached_connection_tables()
sql/sql_base.cc:
  BUG #26257 New Federated Server Functionality Doesn't support differently named tables
  
  new function: close_cached_connection_tables()
    closes all open tables which match connection string
    provides functionality to allow flushing of altered/dropped server names.
sql/sql_servers.cc:
  BUG #26257 New Federated Server Functionality Doesn't support differently named tables
  
  * Added function clone_server() to allocate a new server for use by
    get_server_by_name() when creating a federated table
  
  * Now using MEM_ROOT allocation (mark and sweep) to account for meta
    data parameters being allocated properly, particularly with regards to
    to SERVER object. Also cleans up code allocating share.
  
  * Tables using the old definition of server name are now flushed on successful
    execution of ALTER/DROP SERVER.
  
  style: fixed some line-wrapping
sql/sql_servers.h:
  BUG #26257 New Federated Server Functionality Doesn't support differently named tables
  
  * change in prototype to get_server_by_name()
    caller can now provide mem_root which strings will be copied in to.
storage/federated/ha_federated.cc:
  BUG #26257 New Federated Server Functionality Doesn't support differently named tables
  
  * Simplified share and share member memory allocaton to use MEM_ROOT
  * Modified parse_url to parse table names along with server names
storage/federated/ha_federated.h:
  BUG #26257 New Federated Server Functionality Doesn't support differently named tables
  
  * Added MEM_ROOT share member

  "Concurrent ALTER/CREATE SERVER can lead to deadlock"
  Deadlock caused by inconsistant use of mutexes in sql_server.cc
  One mutex has been removed to resolve deadlock.
  Many functions were made private which should not be exported.
  Unused variables and function removed.


mysql-test/r/federated_server.result:
  Bug 25721 "Concurrent ALTER/CREATE SERVER can lead to deadlock"
  
  New test results
mysql-test/t/federated_server.test:
  Bug 25721 "Concurrent ALTER/CREATE SERVER can lead to deadlock"
  
  Test for bug by using stored procedure. Unpatched server would deadlock frequently.
sql/sql_parse.cc:
  Bug 25721 "Concurrent ALTER/CREATE SERVER can lead to deadlock"
  
  check for correct error code when dropping server
sql/sql_servers.cc:
  Bug 25721 "Concurrent ALTER/CREATE SERVER can lead to deadlock"
  
  Removed unneccessary mutex, only need THR_LOCK_servers rwlock to
  guard data structures against race conditions. Misuse of other mutex
  caused deadlock by inconsistant ordering of mutex lock operations.
  Alter order of some operations to hit memory before disk.
  Removed unused function.
  Removed servers_version and servers_cache_initialised variables.
  Made many internal functions static.
sql/sql_servers.h:
  Bug 25721 "Concurrent ALTER/CREATE SERVER can lead to deadlock"
  
  remove internal functions from being exported.

into  chilla.local:/home/mydev/mysql-5.1-bug26782

After merge fix
In conjunction with the newest 5.1 we always need to
create a real path name to be able to detect already
open tables.

into  chilla.local:/home/mydev/mysql-5.1-bug26996

into  chilla.local:/home/mydev/mysql-5.1-bug26996


mysql-test/r/heap_btree.result:
  Auto merged
mysql-test/t/heap_btree.test:
  Auto merged
storage/heap/hp_write.c:
  Auto merged

into  chilla.local:/home/mydev/mysql-5.0-bug26996


heap/hp_write.c:
  Auto merged
mysql-test/r/heap_btree.result:
  Auto merged
mysql-test/t/heap_btree.test:
  Bug#26996 - Update of a Field in a Memory Table ends with wrong result
  Manual merge from 4.1

into  mysql.com:/home/svoj/devel/mysql/BUG25908/mysql-5.1-engines


storage/myisam/ha_myisam.cc:
  Auto merged

Opening certain tables that have different definitions in .MYI and
.frm may result in a server crash.

Compare .MYI and .frm definition when myisam table is opened. In case
definitions are diffirent refuse to open such table.

No test case, since it requires broken table.


storage/myisam/ha_myisam.cc:
  Compare .MYI and .frm definition when myisam table is opened. In case
  definitions are diffirent refuse to open such table.

Using a MEMORY table BTREE index for scanning for updatable rows
could lead to an infinite loop.

Everytime a key was inserted into a btree index, the position
in the index scan was cleared. The search started from the
beginning and found the same key again.

Now we do not clear the position on key insert an more.


heap/hp_write.c:
  Bug#26996 - Update of a Field in a Memory Table ends with wrong result
  Removed the index-scan-breaking nulling of last_pos.
  The comment behind this line ("For heap_rnext/heap_rprev")
  was misleading. It should have been "Breaks heap_rnext/heap_rprev".
mysql-test/r/heap_btree.result:
  Bug#26996 - Update of a Field in a Memory Table ends with wrong result
  Added test result.
mysql-test/t/heap_btree.test:
  Bug#26996 - Update of a Field in a Memory Table ends with wrong result
  Added test.

into  chilla.local:/home/mydev/mysql-5.1-bug26782


storage/myisam/mi_create.c:
  Auto merged

into  blade08.mysql.com:/data0/istruewing/autopush/mysql-5.1-bug25289


storage/myisam/ha_myisam.cc:
  Auto merged

into  xiphis.org:/home/antony/work2/p1-bug25671.4


sql/sql_parse.cc:
  Auto merged

into  chilla.local:/home/mydev/mysql-5.1-bug25289


storage/myisam/ha_myisam.cc:
  Bug#25289 - repair table causes "my_seek.c:56:
              my_seek: Assertion `fd != -1' failed"
  Manual merge from 5.0

into  chilla.local:/home/mydev/mysql-5.0-bug25289


sql/ha_myisam.cc:
  Bug#25289 - repair table causes "my_seek.c:56:
              my_seek: Assertion `fd != -1' failed"
  Manual merge from 4.1

into  mockturtle.local:/home/dlenev/src/mysql-5.1-bg25966


sql/mysqld.cc:
  Auto merged
sql/sp_head.cc:
  Auto merged
sql/sql_parse.cc:
  Auto merged

into  mockturtle.local:/home/dlenev/src/mysql-5.0-bg25966-2


sql/mysqld.cc:
  Auto merged

TABLE ... WRITE".

Memory and CPU hogging occured when connection which had to wait for table
lock was serviced by thread which previously serviced connection that was
killed (note that connections can reuse threads if thread cache is enabled).
One possible scenario which exposed this problem was when thread which
provided binlog dump to replication slave was implicitly/automatically
killed when the same slave reconnected and started pulling data through
different thread/connection.
The problem also occured when one killed particular query in connection
(using KILL QUERY) and later this connection had to wait for some table
lock.

This problem was caused by the fact that thread-specific mysys_var::abort
variable, which indicates that waiting operations on mysys layer should
be aborted (this includes waiting for table locks), was set by kill
operation but was never reset back. So this value was "inherited" by the
following statements or even other connections (which reused the same
physical thread). Such discrepancy between this variable and THD::killed
flag broke logic on SQL-layer and caused CPU and memory hogging.

This patch tries to fix this problem by properly resetting this member.

There is no test-case associated with this patch since it is hard to test
for memory/CPU hogging conditions in our test-suite.


sql/mysqld.cc:
  We should not forget to reset THD::mysys_var::abort after kill operation
  if we are going to use thread to which this operation was applied for
  handling of other connections.
sql/sp_head.cc:
  We should not forget to reset THD::mysys_var::abort after kill operation
  if we are going to use thread to which this operation was applied for
  handling of further statements.
sql/sql_parse.cc:
  We should not forget to reset THD::mysys_var::abort after kill operation
  if we are going to use thread to which this operation was applied for
  handling of further statements.

TABLE ... WRITE".

CPU hogging occured when connection which had to wait for table lock was
serviced by thread which previously serviced connection that was killed
(note that connections can reuse threads if thread cache is enabled).
One possible scenario which exposed this problem was when thread which
provided binlog dump to replication slave was implicitly/automatically
killed when the same slave reconnected and started pulling data through
different thread/connection.
In 5.* versions memory hogging was added to CPU hogging. Moreover in
those versions the problem also occured when one killed particular query
in connection (using KILL QUERY) and later this connection had to wait for
some table lock.

This problem was caused by the fact that thread-specific mysys_var::abort
variable, which indicates that waiting operations on mysys layer should
be aborted (this includes waiting for table locks), was set by kill
operation but was never reset back. So this value was "inherited" by the
following statements or even other connections (which reused the same
physical thread). Such discrepancy between this variable and THD::killed
flag broke logic on SQL-layer and caused CPU and memory hogging.

This patch tries to fix this problem by properly resetting this member.

There is no test-case associated with this patch since it is hard to test
for memory/CPU hogging conditions in our test-suite.


sql/mysqld.cc:
  We should not forget to reset THD::mysys_var::abort after kill operation
  if we are going to use thread to which this operation was applied for
  handling of other connections.

into  mockturtle.local:/home/dlenev/src/mysql-5.1-bg25966

into  mysql.com:/home/kent/bk/tmp/mysql-5.1-build

into  mysql.com:/home/kent/bk/tmp/mysql-5.0-build

into  mysql.com:/home/kent/bk/tmp/mysql-4.1-build

  Updated to version 0.6 of the text


EXCEPTIONS-CLIENT:
  Updated to version 0.6 of the text

            my_seek: Assertion `fd != -1' failed"

In difficult optimize/repair situations the server could crash.
Under some circumstances the server retries an optimize/repair
with more elaborate options. But it did not check if the first
attempt failed so badly that a second one must not be tried.

This could happen when a new data file has been created
but it was not possible to open it. In this case the
repair leaves behind a table with closed data file.
This must not be used for another repair attempt.

We do now detect the closed data file and do not try
another repair attempt in this situation.

No test case. The required table corruption can not be
repeated easily. There is a test program attached to
bug 25433.


sql/ha_myisam.cc:
  Bug#25289 - repair table causes "my_seek.c:56:
              my_seek: Assertion `fd != -1' failed"
  Added code to detect a closed data file. It could be closed
  by a preceeding repair attempt. We must not try another
  repair then.

into  mysql.com:/home/kent/bk/tmp/mysql-5.1-build


configure.in:
  Auto merged
storage/ndb/src/ndbapi/NdbBlob.cpp:
  Auto merged
storage/ndb/test/ndbapi/testBlobs.cpp:
  Auto merged

into  mysql.com:/home/kent/bk/tmp/mysql-5.1-build

into  mysql.com:/home/kent/bk/tmp/mysql-5.0-build


configure.in:
  Auto merged

into  mysql.com:/home/kent/bk/tmp/mysql-5.0-build

into  mysql.com:/home/kent/bk/tmp/mysql-4.1-build

into  mysql.com:/home/kent/bk/tmp/mysql-4.1-build


configure.in:
  SCCS merged

  Added test for sched_yield() possibly in -lposix4 on Solaris


configure.in:
  Added test for sched_yield() possibly in -lposix4 on Solaris

into  blade08.mysql.com:/data0/istruewing/autopush/mysql-5.1-bug25460

into  mysql.com:/home/svoj/devel/mysql/BUG26881/mysql-5.1-engines


storage/myisam/mi_open.c:
  Auto merged

  "CREATE/DROP/ALTER SERVER should require privileges"
  Add check for SUPER privilege when executing CREATE/DROP/ALTER SERVER.
  Previously, any user even with only USAGE priv can execute those commands.


mysql-test/r/federated_server.result:
  Bug25671 - CREATE/DROP/ALTER SERVER should require privileges
mysql-test/t/federated_server.test:
  Bug25671 - CREATE/DROP/ALTER SERVER should require privileges
sql/sql_parse.cc:
  Bug25671 - CREATE/DROP/ALTER SERVER should require privileges
    Add check for SUPER privilege when executing CREATE/DROP/ALTER SERVER

into  blade08.mysql.com:/data0/istruewing/autopush/mysql-5.1-bug25460

The previous two patches for this bug worked together so that
no permanent table was memory mapped. The first patch tried to
avoid mapping while a table is in use. It allowed mapping only
if there was exactly one lock on the table, assuming that the
calling thread owned it. During mi_open(), a different call to
memory mapping was coded, which did not have this limitation.

The second patch tried to remove the code duplication and just
called mi_extra() from mi_open() an thus inherited the limitation.
But on open, a thread does not have a lock on the table...

A possible solution would be to check for zero or one lock.
But since I learned that it is safe to memory map a file while
normal file I/O is done on it, I removed the restriction altogether
and allow to memory map while a table is in use.

No test case. I do not see a chance to verify with the test suite
which kind of I/O is used on a table.


storage/myisam/mi_extra.c:
  Bug#25460 - High concurrency MyISAM access causes severe mysqld crash.
  Allow to memory map while table is in use.