GitLab

 - Always make rnd_pos reposition the cursor in the fetched result set.


mysql-test/r/federated.result:
  Update test result
mysql-test/t/federated.test:
  Add test for bug#17377
sql/ha_federated.cc:
  Remove the "scan_flag" variable. rnd_pos should always move to a new position in the result set. 
  The "scan" flag in rnd_init will protect from the old result set being 
  closed and a new fecthed. I think the "scan_flag" was added before check of the "scan" argument in rnd_init was added.
sql/ha_federated.h:
  Remove class variable "scan_flag"

Added fixes to make last_insert_id() to work.


mysql-test/r/federated.result:
  BUG #14768
  
  New test results for last_insert_id()
mysql-test/t/federated.test:
  BUG #14768
  
  Tests for last_insert_id()
sql/ha_federated.cc:
  BUG# 14768
  
  * Added code to set last_insert_id()
  * Added code to free share->scheme
sql/ha_federated.h:
  BUG #14768
  
  New method for setting last_insert_id()

sql/ha_federated.cc:
  if the mysql_error(mysql) contained any %-format specifiers, my_snprintf would try to interppret them. Essentially replacing printf(str) with printf("%s", str);
sql/ha_federated.h:
  removed unused remote_error_len variable

BUG#12659 error handling in federated needs improvement. Sanja pointed out that we were copying an entire buffer, most of which was probably garbage, and possibly hidden by null termination in the middle of the buffer. Now we are only copying the error message.


sql/ha_federated.cc:
  now recording only the error message, and not additional garbage. Storing the buffer length.
sql/ha_federated.h:
  added buffer length variable

Now many direct references to my_error() have been removed.


mysql-test/r/federated_archive.result:
  new results with new error message
mysql-test/t/federated_archive.test:
  corrected error number
sql/ha_federated.cc:
  some minor 80 column formatting
  corrected some error handling to be more handler print_error friendly
  moved duplicate code into new "stash_remote_error" function
sql/ha_federated.h:
  added error number, and places to stash error message and number
  added private method to stash error

sql/ha_federated.h:
  Fix cache issues

The idea of the patch
is that every cursor gets its own lock id for table level locking.
Thus cursors are protected from updates performed within the same 
connection. Additionally a list of transient (must be closed at
commit) cursors is maintained and all transient cursors are closed
when necessary. Lastly, this patch adds support for deadlock
timeouts to TLL locking when using cursors.
+ post-review fixes.


include/thr_lock.h:
  - add a notion of lock owner to table level locking. When using
    cursors, lock owner can not be identified by a thread id any more, 
    as we must protect cursors from updates issued within the same 
    connection (thread). So, each cursor has its own lock identifier to 
    use with table level locking.
  - extend return values of thr_lock and thr_multi_lock with
    THR_LOCK_TIMEOUT and THR_LOCK_DEADLOCK, since these conditions
    are now possible (see comments to thr_lock.c)
mysys/thr_lock.c:
  Better support for cursors:
  - use THR_LOCK_OWNER * as lock identifier, not pthread_t.
  - check and return an error for a trivial deadlock case, when an
    update statement is issued to a table locked by a cursor which has 
    been previously opened in the same connection.
  - add support for locking timeouts: with use of cursors, trivial 
    deadlocks can occur. For now the only remedy is the lock wait timeout,
    which is initialized from a new global variable 'table_lock_wait_timeout'
    Example of a deadlock (assuming the storage engine does not downgrade 
    locks):
    con1: open cursor for select * from t1;
    con2: open cursor for select * from t2;
    con1: update t2 set id=id*2;  -- blocked
    con2: update t1 set id=id*2;  -- deadlock
    Lock timeouts are active only if a connection is using cursors.
  - the check in the wait_for_lock loop has been changed from
    data->cond != cond to data->cond != 0. data->cond is zeroed
    in every place it's changed. 
  - added comments
sql/examples/ha_archive.cc:
  - extend the handlerton with the info about cursor behaviour at commit.
sql/examples/ha_archive.h:
  - ctor moved to .cc to make use of archive handlerton
sql/examples/ha_example.cc:
  - add handlerton instance, init handler::ht with it
sql/examples/ha_example.h:
  - ctor moved to .cc to make use of ha_example handlerton
sql/examples/ha_tina.cc:
  - add handlerton instance, init handler::ht with it
sql/examples/ha_tina.h:
  - ctor moved to .cc to make use of CSV handlerton
sql/ha_berkeley.cc:
  - init handlerton::flags and handler::ht
sql/ha_berkeley.h:
  - ctor moved to .cc to make use of BerkeleyDB handlerton
sql/ha_blackhole.cc:
  - add handlerton instance, init handler::ht with it
sql/ha_blackhole.h:
  - ctor moved to .cc to make use of blackhole handlerton
sql/ha_federated.cc:
  - add handlerton instance, init handler::ht with it
sql/ha_federated.h:
  - ctor moved to .cc to make use of federated handlerton
sql/ha_heap.cc:
  - add handlerton instance, init handler::ht with it
sql/ha_heap.h:
  - ctor moved to .cc to make use of ha_heap handlerton
sql/ha_innodb.cc:
  - init handlerton::flags and handler::ht of innobase storage engine
sql/ha_innodb.h:
  - ctor moved to .cc to make use of archive handlerton
sql/ha_myisam.cc:
  - add handlerton instance, init handler::ht with it
sql/ha_myisam.h:
  - ctor moved to .cc to make use of MyISAM handlerton
sql/ha_myisammrg.cc:
  - init handler::ht in the ctor
sql/ha_myisammrg.h:
  - ctor moved to .cc to make use of MyISAM MERGE handlerton
sql/ha_ndbcluster.cc:
  - init handlerton::flags and handler::ht
sql/handler.cc:
  - drop support for ISAM storage engine, which was removed from 5.0
  - close all "transient" cursors at COMMIT/ROLLBACK. A "transient"
    SQL level cursor is a cursor that uses tables that have a transaction-
    specific state.
sql/handler.h:
  - extend struct handlerton with flags, add handlerton *ht to every
    handler instance.
sql/lock.cc:
  - extend mysql_lock_tables to send error to the client if 
    thr_multi_lock returns a timeout or a deadlock error.
sql/mysqld.cc:
  - add server option --table_lock_wait_timeout (in seconds)
sql/set_var.cc:
  - add new global variable 'table_lock_wait_timeout' to specify
  a wait timeout for table-level locks of MySQL (in seconds). The default
  timeout is 50 seconds. The timeout is active only if the connection
  has open cursors.
sql/sql_class.cc:
  - implement Statement_map::close_transient_cursors
  - safety suggests that we need an assert ensuring 
   llock_info->n_cursors is functioning properly, adjust destruction of
   the Statement_map to allow such assert in THD::~THD
sql/sql_class.h:
  - add support for Cursors registry to Statement map.
sql/sql_prepare.cc:
  - maintain a list of cursors that must be closed at commit/rollback.
sql/sql_select.cc:
  - extend class Cursor to support specific at-COMMIT/ROLLBACK behavior.
  If a cursor uses tables of a storage engine that 
  invalidates all open tables at COMMIT/ROLLBACK, it must be closed
  before COMMIT/ROLLBACK is executed.
sql/sql_select.h:
  - add an own lock_id and commit/rollback status flag to class Cursor
tests/mysql_client_test.c:
  A test case for Bug#10760 and complementary issues: test a simple
  deadlock case too.
mysql-test/var:
  New BitKeeper file ``mysql-test/var''

as well as changes to the test.


mysql-test/r/federated.result:
  New test results
mysql-test/t/federated.test:
  Removed explains per brian's suggestion since on his 64-bit amd, you get 
  different information, something to be ironed out later, as well as it 
  having nothing to do with federated.
sql/ha_federated.cc:
  - Eric Herman's implementation of repair and optimize!!!
  - Eric's changes to 1EQ0 to FALSE
sql/ha_federated.h:
  Eric's addition of LEN for each SQL string tokens.

WHERE statement"


sql/ha_federated.cc:
  Changes per Jimw's review. 
  
  notes about some suggestions:
  
  1. Tried to replace my_sprintf with strxmov, but had problems. Couldn't 
     get the test suite to run at all, and could not find any particular
     error. Since this is something that is not critical, I'll leave it
     as is for now.
  2. I would like to use my_error to create all of my error messages, but 
     cannot figure out how to get it to take more than one argument to be
     passed to the error message (I tried this once but had errors, and 
     cannot find an example of how to get it to work despite seeing messages
     with obviously more than one arg
  3. Replaced all string building from literals to defined SQL string pieces
     such as "SELECT ", "FROM " ...
  4. Format fixes, spacing fixes completed
  5. Removed records_in_range records calculation functionality
sql/ha_federated.h:
  Formatting changes, new defines for SQL query string tokens
sql/share/errmsg.txt:
  changed 'SRC' to 'SOURCE' per Jimw's review

deallocation cleanup


mysql-test/r/federated.result:
  new test results
mysql-test/t/federated.test:
  New tests, and added explains per Timour's suggestion. This will keep
          track of whether changes have been made to the optimiser that might affect
          Federated.
          Also changed the error codes to federated error codes in the error tests
sql/ha_federated.cc:
  1. Not using MAX_REMOTE_SIZE, so removed
           2. Changed all defines that were IO_SIZE to FEDERATED_QUERY_BUFFER_SIZE
           3. Now using Federated Error Codes
           4. Implemented ::info (gets valid 'records' number)
           5. Implemented ::read_range_first - now uses indexes in range operations
           6. Better allocation in get_share
           7. Cleaned up memory bug in get_share
           8. Better initial query "SELECT * FROM t1 WHERE 1=0" for check_foreign_data_src
           9. emit_key_part_name and emit_key_part_element for read_range_first to not
              have redundancy
           10. Made sure all 'append's pass length, and use append("") to ensure proper
               null termination of strings being built.
           11. Better message creation. If ER_QUERY_ON_FOREIGN_DATA_SRC, I also pass
               the actual error code and message from the foreign data src.
           12. Cleaned up how the result set and share->scheme is freed.
sql/share/errmsg.txt:
  New Federated error messages that are more tailored to the type of information
  we want to see when federated has an error.
mysql-test/include/federated.inc:
  New bitkeeper file, include file for separating setup from main test suite
mysql-test/include/federated_cleanup.inc:
  New BitKeeper file ``mysql-test/include/federated_cleanup.inc''
sql/ha_federated.h:
  Added comments where needed, one was already commented about

sql/ha_federated.h:
  Change mode to -rw-rw-r--

This is the first changeset of suggested changes recommended in Kostja's 
review of my patch, 1.1846, which includes only functionality changes. 
Style changes/Documentation patch to follow.


include/mysql.h:
  removed declaration of cli_fetch_lengths per Kostja's suggestion
libmysql/libmysql.c:
  moved mysql_fetch_lengths to client.c (for server to access) per Kostja's
  suggestion
sql-common/client.c:
  added back 'static' to function definition, added mysql_fetch_lengths
sql/ha_federated.cc:
  changed to use defines as opposed to hardcoded values
sql/ha_federated.h:
  took out duplicate table_flag, fixed a resolve mistake

Testing on hp3750 shows these fixes fix the compile problems on HPUX, but I have 
a problem where when I run the tests, the test shows that the tables default to MyISAM!


include/mysql.h:
  HPUX's compiler does not like 'ulong' - the compile fails due to this.
sql/ha_federated.h:
  Serg's recommendation to fix bug #8599 (which is a good fix since it's obviously 
  from when I added the method via cut and paste ;). Also caught some comment style issues.

sql/ha_federated.h:
  copy-paste fix
sql/sql_parse.cc:
  after-merge fix

for bugs 8033, 8065, 8535, 8582

This particular changeset contains style changes per code review suggestions 
and does not contain bug fixes in of itself.

ha_federated.h:
  standardised code style to conform to internals.texi
ha_federated.cc:
  more code standardisation to conform to internals.texi.
  - casts
  - declarations
  - comments
  - 80 char width
  also, append using string1.append(string2) and not 
  string1.append(string2.c_ptr_quick())


sql/ha_federated.cc:
  more code standardisation to conform to internals.texi.
  - casts
  - declarations
  - comments
  - 80 char width
  
  also, append using string1.append(string2) and not 
  string1.append(string2.c_ptr_quick())
sql/ha_federated.h:
  standardised code style to conform to internals.texi

This changeset/patch is on top of changesets 1.1814 and 1.1846 
(for bugs 8033 and 8065) and now fixes bug 8535.

These changes have been built and tested successfully on build.mysql.com

handler.cc:
  Added hooks for federated_db_init() and federated_db_end(), 
  as done with ha_archive_db does, per suggestion by Ingo in
  code review of patch 1.1846.
ha_federated.h:
  declaration of federated_db_init() and federated_db_end()
ha_federated.cc:
  - Fixed some indentation problems from indent-ex (mainly to do with
    cases where "variablename += value"
  - Added federated_db_init() and federated_db_end(), as done with
    archive, which also handler more elegantly one of the memory leaks
    from bug 8033 where the federated_mutex was not freed
  - Removed extrenous debug messages in parse_url()
  - Fixed bug 8535, caused by NULL being quoted in write_row. This used to
    work (incorrectly) but a recent change was made in the server that 
    exposed this


sql/ha_federated.cc:
   - Fixed some indentation problems from indent-ex (mainly to do with
      cases where "variablename += value"
    - Added federated_db_init() and federated_db_end(), as done with
      archive, which also handler more elegantly one of the memory leaks
      from bug 8033 where the federated_mutex was not freed
    - Removed extrenous debug messages in parse_url()
sql/ha_federated.h:
  declaration of federated_db_init() and federated_db_end()
sql/handler.cc:
  Added hooks for federated_db_init() and federated_db_end(), as done with ha_archive_db does.

This patch contains all that my previous patch (1.1814) contained, with the addition of using cli_fetch_lengths for
handling binary data (Bar noted this on the review of 1.1814, Guilhem suggested using cli_fetch_lenghts by 
making available via removal of static in method definition and declaration in mysql.h, but
Konstantin had some reservations, but he said to commit the patch using this anyway,
and I suppose this can be discussed. I abandoned 1.1814 because Monty made a couple
fixes to my code as well as formatting changes, and I thought it would just be easier
to hand-edit my changes into a fresh clone and then make a patch. 

The reason for using cli_fetch_lengths is so that I can correctly get the length of
the field I am setting into the field. I was previously using 'strlen' but Bar pointed out this
won't correctly get the length of binary data and is also less effecient. Upon testing,
it was in fact verified that binary data in a blob table was being inserted correctly,
but not being retrieved correctly, all due to not having the correct value for the
field:

(*field)->store(row[x], strlen(row[x]), &my_charset_bin);

was changed to:

(*field)->store(row[x], lengths[x], &my_charset_bin);

lengths being a unsigned long pointer to the values of the field lengths from a 
MYSQL_ROW.

Since the server doesn't have the function "mysql_fetch_lengths" available, I tried 
to use "result->lengths", but this isn't set, so I finally successfully used 
cli_fetch_lenghts, which does give the correct lengths, and now the binary data gets
retrieved correctly.

I've also run the code through indent-ex and am using Brian's vimrc to ensure correct formatting!

This code passes the entire test suite, without any errors or warning on both my 
workstation and build.mysql.com


include/mysql.h:
  added cli_fetch_lengths to mysql.h in order to use this function in the federated handler
mysql-test/r/federated.result:
  - Moved countries to be created and inserted prior to federated test table
  - Added a test of inserting binary values into a blob table
mysql-test/t/federated.test:
  - Moved order of countries table creation to prior to test table creation
  - Test insertion of binary values in a blob table
sql-common/client.c:
  removed 'static' to allow cli_fetch_lengths to be used in the federated handler
sql/ha_federated.cc:
  1. share->scheme that was created in parse_url was not being freed
  2. HASH federated_open_tables was being deleted, but not freed
  3. 'result' from mysql_store_result was not being free in several instances
  4. Fixed the problem where a table scan was being performed after
  index_read_idx, which didn't cause a problem because the result set from
  idx_read_idx was not being freed, but once the result set was properly freed,
  it broke update_row. Now, I'm using the bool 'scan' to determine if I need to
  perform a table scan, which it magically is false when the query is an update
  with an index.
  5. Changed all stings containing the query to perform in mysql_real_query
  calls from string.c_ptr_quick() to string.ptr() per Monty's suggestion
  (better performance)
  6. Fixed various cast/type/truth compile warnings.
  7. Removed 'load_conn_info' and just let 'parse_url' handle it.
  8. Added the use of cli_fetch_lengths, needed to fix binary values being retrieved 
  from the database in rnd_next/convert_row_to_internal_format
  9. Formatting changes by using indent-ex!
sql/ha_federated.h:
  added scan flag, setting defaults for result and scan_flag

sql/ha_federated.cc:
  changes to get rid of compile warnings
sql/ha_federated.h:
  changes to get rid of compile warnings

-New tests and results

logging_ok:
  Logging to logging@openlogging.org accepted
ha_federated.h:
  removed quote_data and type_quote (now in the Field class)
ha_federated.cc:
  moved quote_data and type_quote to field class
field.h:
  new methods quote_data and needs_quotes declared
field.cc:
  new field class methods quote_data and needs_quotes (per Monty's request)
federated.test:
  more tests, joins, index tests
have_federated_db.require:
  new name of federated system var
federated.result:
  new test results for federated handler
have_federated_db.inc:
  changed name of variable in test due to change in vars
sql_analyse.cc:
  over-ridden append_escaped to take (String *, char *, uint) per requirements of 'create_where_from_key' method in federated handler.
mysql_priv.h:
  define over-ridden append_escaped to take arguments from 'create_where_from_key' method in federated handler 
ha_federated.cc:
  implemented "create_where_from_key" to deal properly with two-byte prefix and multi keys. Initial testing shows it works, but I still need to move quoting to field class and also look at changes per Segei's suggestions.


sql/mysql_priv.h:
  define over-ridden append_escaped to take arguments from 'create_where_from_key' method in federated handler
sql/sql_analyse.cc:
  over-ridden append_escaped to take (String *, char *, uint) per requirements of 'create_where_from_key' method in federated handler.
mysql-test/include/have_federated_db.inc:
  changed name of variable in test due to change in vars
mysql-test/r/federated.result:
  new test results for federated handler
mysql-test/r/have_federated_db.require:
  new name of federated system var
mysql-test/t/federated.test:
  more tests, joins, index tests
sql/field.cc:
  new field class methods quote_data and needs_quotes (per Monty's request)
sql/field.h:
  new methods quote_data and needs_quotes declared
sql/ha_federated.cc:
  moved quote_data and type_quote to field class
sql/ha_federated.h:
  removed quote_data and type_quote (now in the Field class)
BitKeeper/etc/logging_ok:
  Logging to logging@openlogging.org accepted

sql/ha_federated.cc:
  added code to handle multi-keys (per monty). This code still fails to compile because sql_analyse.cc's method, append_escaped only accepts two string pointers. I don't know if append_escaped will be overloaded to accept string pointer, byte/char pointer, and length, as is being passed in this example. Need to talk to devs/monty about this.
sql/ha_federated.h:
  added method declaration of create_where_from_key

mysql-test/r/federated.result:
  new test results
mysql-test/t/federated.test:
  added order by, group by
sql/ha_federated.cc:
  - added 'scheme' to URL
  - added proper escaping
  - made sure &my_charset_bin is being used throughout handler
  - made sure create_table catches improper URL in comment upon table creation
sql/ha_federated.h:
  added scheme to share

First commit to mysql-5.0 tree to include MySQL Federated Storage Handler. This includes both the source and header files, test (results, test, require), and modifications to server and handler base files, and autoconf modifications to properly build federated handler.


configure.in:
  inclusion of federated handler autoheader macro
mysql-test/mysql-test-run.sh:
  allow usage of replication tests for federated handler
sql/Makefile.am:
  inclusion of federated header and source file
sql/field.h:
  overloaded method val_str() to work with fields in 'old_data' in 'update_row()'
sql/handler.cc:
  added code to include federated handler
sql/handler.h:
  add db type for federated
sql/mysql_priv.h:
  added code for federated handler
sql/mysqld.cc:
  added code for federated handler
sql/set_var.cc:
  added code for federated handler

New handler::index_flags() definition to make it easy to check the full used key and a specific key part.
Added key part to optimize_range() to fix problems when using fields in key parts.



sql/examples/ha_archive.h:
  New handler::index_flags() definition
sql/examples/ha_example.h:
  New handler::index_flags() definition
sql/field.cc:
  New optimize_range() definition
sql/field.h:
  New optimize_range() definition
sql/ha_berkeley.cc:
  New handler::index_flags() definition
sql/ha_berkeley.h:
  New handler::index_flags() definition
sql/ha_heap.h:
  New handler::index_flags() definition
sql/ha_innodb.h:
  New handler::index_flags() definition
sql/ha_isam.h:
  New handler::index_flags() definition
sql/ha_isammrg.h:
  New handler::index_flags() definition
sql/ha_myisam.h:
  New handler::index_flags() definition
sql/ha_myisammrg.h:
  New handler::index_flags() definition
sql/ha_ndbcluster.cc:
  New handler::index_flags() definition
sql/ha_ndbcluster.h:
  New handler::index_flags() definition
sql/handler.h:
  New handler::index_flags() definition
sql/log.cc:
  Fixed compiler warnings
sql/log_event.cc:
  Fixed compiler warnings (and renamed short variable name)
sql/opt_range.cc:
  New handler::index_flags() definition
sql/opt_sum.cc:
  New handler::index_flags() definition
sql/set_var.cc:
  Removed compiler warnings
sql/sql_db.cc:
  Removed compiler warnings
sql/sql_select.cc:
  New handler::index_flags() definition
sql/sql_show.cc:
  Removed compiler warnings
sql/sql_update.cc:
  Removed compiler warnings
sql/table.cc:
  New handler::index_flags() definition

mysql-test/r/bdb.result:
  bug#4000
mysql-test/t/bdb.test:
  bug#4000
sql/sql_select.cc:
  cleanup

A couple of small changes to ha_example (mainly comments). 


sql/examples/ha_archive.cc:
  Changes from Serg.
  A few styles changes, fix for potential hash insert gone wrong, removed bad tables if create fails, gzflush used for flushing by default (had problems with OSX 10.2), and some spelling corrections.
sql/examples/ha_archive.h:
  Removed fast_key_read() and changed the name of a conflicting variable.
sql/examples/ha_example.cc:
  Added a few new comments.
sql/examples/ha_example.h:
  Added pragma interface. Removed fast_key_read() and Serg's request

Updating range_in_rows for the include file for ha_example (the main .cc file has already been updated). 


sql/examples/ha_example.h:
  Updating ha_example.h for new records_in_range. ha_example.cc has already been updated.

sql/examples/ha_example.cc:
  Documentation updates (lots of comments in the code).
sql/examples/ha_example.h:
  Documenation update, lots of comments in the code.

  Fixed spelling of example share and fxed indention.
ha_example.cc:
  Fixed spelling of example and removed a few unneeded printf pieces.
my_base.h:
  Adde HA_ERR_NOT_IMPLEMENTED at Monty's request


include/my_base.h:
  Adde HA_ERR_NOT_IMPLEMENTED at Monty's request
sql/examples/ha_example.cc:
  Fixed spelling of example and removed a few unneeded printf pieces.
sql/examples/ha_example.h:
  Fixed spelling of example share and fxed indention.

First commit of the skeleton storage engine. Use this example as a template to build storage engines. Also includes a new degine for marking storage engine methods as not being implemented.


sql/examples/ha_example.h:
  First commit of example storage engine.
BitKeeper/etc/logging_ok:
  Logging to logging@openlogging.org accepted