Continued implementation of WL#1324 (table name to filename encoding)

The intermediate (not temporary) files of the new table
during ALTER TABLE was visible for SHOW TABLES. These
intermediate files are copies of the original table with
the changes done by ALTER TABLE. After all the data is
copied over from the original table, these files are renamed 
to the original tables file names. So they are not temporary 
files. They persist after ALTER TABLE, but just with another 
name.

In 5.0 the intermediate files are invisible for SHOW TABLES
because all file names beginning with "#sql" were suppressed.

This failed since 5.1.6 because even temporary table names were
converted when making file names from them. The prefix became
converted to "@0023sql". Converting the prefix during SHOW TABLES
would suppress the listing of user tables that start with "#sql".

The solution of the problem is to continue the implementation of
the table name to file name conversion feature. One requirement
is to suppress the conversion for temporary table names.

This change is straightforward for real temporary tables as there
is a function that creates temporary file names.

But the generated path names are located in TMPDIR and have no
relation to the internal table name. This cannot be used for
ALTER TABLE. Its intermediate files need to be in the same
directory as the old table files. And it is necessary to be
able to deduce the same path from the same table name repeatedly.

Consequently the intermediate table files must be handled like normal
tables. Their internal names shall start with tmp_file_prefix
(#sql) and they shall not be converted like normal table names.

I added a flags parameter to all relevant functions that are
called from ALTER TABLE. It is used to suppress the conversion
for the intermediate table files.

The outcome is that the suppression of #sql in SHOW TABLES
works again. It does not suppress user tables as these are
converted to @0023sql on file level.

This patch does also fix ALTER TABLE ... RENAME, which could not 
rename a table with non-ASCII characters in its name.

It does also fix the problem that a user could create a table like
`#sql-xxxx-yyyy`, where xxxx is mysqld's pid and yyyy is the thread
ID of some other thread, which prevented this thread from running 
ALTER TABLE.

Some of the above problems are mentioned in Bug 1405, which can
be closed with this patch.

This patch does also contain some minor fixes for other forgotten
conversions. Still known problems are reported as bugs 21370,
21373, and 21387.


mysql-test/r/alter_table.result:
  Bug#18775 - Temporary table from alter table visible to other threads
  Continued implementation of WL#1324 (table name to filename encoding)
  Added test results.
mysql-test/r/backup.result:
  Bug#18775 - Temporary table from alter table visible to other threads
  Continued implementation of WL#1324 (table name to filename encoding)
  Added test results.
mysql-test/r/repair.result:
  Bug#18775 - Temporary table from alter table visible to other threads
  Continued implementation of WL#1324 (table name to filename encoding)
  Added test results.
mysql-test/t/alter_table.test:
  Bug#18775 - Temporary table from alter table visible to other threads
  Continued implementation of WL#1324 (table name to filename encoding)
  Added test cases.
mysql-test/t/backup.test:
  Bug#18775 - Temporary table from alter table visible to other threads
  Continued implementation of WL#1324 (table name to filename encoding)
  Added test cases.
mysql-test/t/repair.test:
  Bug#18775 - Temporary table from alter table visible to other threads
  Continued implementation of WL#1324 (table name to filename encoding)
  Added a test case.
sql/ha_myisam.cc:
  Bug#18775 - Temporary table from alter table visible to other threads
  Continued implementation of WL#1324 (table name to filename encoding)
  Added missing table name mapping calls to backup() and restore().
sql/ha_myisammrg.cc:
  Bug#18775 - Temporary table from alter table visible to other threads
  Continued implementation of WL#1324 (table name to filename encoding)
  Added an zero argument for the new 'flags' parameter.
sql/ha_ndbcluster.cc:
  Bug#18775 - Temporary table from alter table visible to other threads
  Continued implementation of WL#1324 (table name to filename encoding)
  Added an zero argument for the new 'flags' parameter.
sql/ha_ndbcluster_binlog.cc:
  Bug#18775 - Temporary table from alter table visible to other threads
  Continued implementation of WL#1324 (table name to filename encoding)
  Added an zero argument for the new 'flags' parameter.
sql/ha_ndbcluster_binlog.h:
  Bug#18775 - Temporary table from alter table visible to other threads
  Continued implementation of WL#1324 (table name to filename encoding)
  Removed unnecessary check for wrong temp file prefix.
sql/mysql_priv.h:
  Bug#18775 - Temporary table from alter table visible to other threads
  Continued implementation of WL#1324 (table name to filename encoding)
  Extended quick_rm_table(), mysql_rename_table(), and
  build_table_filename() by an flags argument, which can indicate
  temporary table names that should not be converted.
  Added symbolic flag values.
sql/sql_acl.cc:
  Bug#18775 - Temporary table from alter table visible to other threads
  Continued implementation of WL#1324 (table name to filename encoding)
  Added an zero argument for the new 'flags' parameter.
sql/sql_base.cc:
  Bug#18775 - Temporary table from alter table visible to other threads
  Continued implementation of WL#1324 (table name to filename encoding)
  Fixed a comment.
  Added DBUG calls.
sql/sql_db.cc:
  Bug#18775 - Temporary table from alter table visible to other threads
  Continued implementation of WL#1324 (table name to filename encoding)
  Added an zero argument for the new 'flags' parameter.
sql/sql_delete.cc:
  Bug#18775 - Temporary table from alter table visible to other threads
  Continued implementation of WL#1324 (table name to filename encoding)
  Added an zero argument for the new 'flags' parameter.
sql/sql_insert.cc:
  Bug#18775 - Temporary table from alter table visible to other threads
  Continued implementation of WL#1324 (table name to filename encoding)
  Added an zero argument for the new 'flags' parameter.
sql/sql_partition.cc:
  Bug#18775 - Temporary table from alter table visible to other threads
  Continued implementation of WL#1324 (table name to filename encoding)
  Added an zero argument for the new 'flags' parameter.
sql/sql_rename.cc:
  Bug#18775 - Temporary table from alter table visible to other threads
  Continued implementation of WL#1324 (table name to filename encoding)
  Added an zero argument for the new 'flags' parameter.
sql/sql_show.cc:
  Bug#18775 - Temporary table from alter table visible to other threads
  Continued implementation of WL#1324 (table name to filename encoding)
  Reverted the former fix for this bug. tmp_file_prefix is now used
  verbatim in the comparison of file names.
sql/sql_table.cc:
  Bug#18775 - Temporary table from alter table visible to other threads
  Continued implementation of WL#1324 (table name to filename encoding)
  Added a check for a tmp_file_prefix file name to
  filename_to_tablename(). These names are now accepted without
  conversion.
  Extended quick_rm_table(), mysql_rename_table(), and
  build_table_filename() by an flags argument, which can indicate
  temporary table names that should not be converted.
  Removed the table to file name conversion from
  build_tmptable_filename().
  Disabled REPAIR TABLE ... USE_FRM for temporary tables.
  Added the forgotten conversion to mysql_alter_table() for the case
  of ALTER TABLE ... RENAME.
  Added comments and DBUG calls.
sql/sql_trigger.cc:
  Bug#18775 - Temporary table from alter table visible to other threads
  Continued implementation of WL#1324 (table name to filename encoding)
  Added an zero argument for the new 'flags' parameter.
sql/sql_view.cc:
  Bug#18775 - Temporary table from alter table visible to other threads
  Continued implementation of WL#1324 (table name to filename encoding)
  Added an zero argument for the new 'flags' parameter.
sql/table.cc:
  Bug#18775 - Temporary table from alter table visible to other threads
  Continued implementation of WL#1324 (table name to filename encoding)
  Replaced a literal ".frm" by reg_ext.
  Added DBUG calls.
storage/innobase/row/row0mysql.c:
  Bug#18775 - Temporary table from alter table visible to other threads
  Continued implementation of WL#1324 (table name to filename encoding)
  Changed back the encoded temp file prefix to #sql.

into  chilla.local:/home/mydev/mysql-5.1-aid


sql/ha_federated.cc:
  Auto merged

into  chilla.local:/home/mydev/mysql-5.1-amerge


sql/sql_insert.cc:
  Auto merged
sql/sql_parse.cc:
  Auto merged
sql/sql_trigger.cc:
  Auto merged

Bug #20902 Alter table invalid schema version with statement-based binlogging
(latter caused by the above)

into  devsrv-b.mysql.com:/data0/knielsen/tmp-5.1

Small post-patch fixes


mysql-test/r/federated.result:
  BUG #19773
  
  Test result fixes
mysql-test/t/federated.test:
  BUG #19774
  
  Removed failing line to test.
sql/ha_federated.cc:
  BUG #19773
  
  method declartion and definition not the same.

into  bodhi.local:/opt/local/work/mysql-5.1-runtime-merge-with-5.0

into  devsrv-b.mysql.com:/data0/knielsen/tmp-5.1

into  poseidon.ndb.mysql.com:/home/tomas/mysql-5.1-ndb

- keep accounting for aoutstanding drop trigger requests
- also lock table object in suma while doing that (should be impossible right now though since dict serializes all requests)

Disable test cases ndb_alter_table3, ndb_autodiscover3, ndb_dd_advance,
and ndb_multi in statement-based binlogging mode until bug can be fixed.


mysql-test/t/ndb_alter_table3.test:
  Disable test in statement-based binlogging mode.
mysql-test/t/ndb_autodiscover3.test:
  Disable test in statement-based binlogging mode.
mysql-test/t/ndb_dd_advance.test:
  Disable test in statement-based binlogging mode.
mysql-test/t/ndb_multi.test:
  Disable test in statement-based binlogging mode.

server-tools/instance-manager/parse.h:
  Post-merge fixes: fix Windows build.
sql/log_event.cc:
  uint32 -> uint
sql/rpl_filter.cc:
  uint32 -> uint
sql/rpl_filter.h:
  uint32 -> uint

into  bodhi.netgear:/opt/local/work/mysql-5.1-runtime-merge-with-5.0

sql/events.cc:
  Fix a bug I introduced with the merge: dbchanged is read
  even if it's not passed to sp_use_new_db.

into  chilla.local:/home/mydev/mysql-5.1-amerge

sql/event_timed.cc:
  After merge fixes: dbname can never be NULL.
sql/events.cc:
  After merge fixes: use a new signature of sp_use_new_db
sql/sp_head.cc:
  After merge fixes: replace assert with an if for events code to work.
sql/sql_parse.cc:
  After merge fixes: put back free_items, it's used in partitioning.
  Whether it is not a bug is to be investigated.
  Remove check_db_used, as planned.

into  govinda.patg.net:/home/patg/mysql-build/mysql-5.1-merge1


mysql-test/r/federated.result:
  BUG #19773
  
  Hand-merge conflicts from 5.0 to 5.1
mysql-test/t/federated.test:
  BUG #19773
  
  Hand-merge conflicts from 5.0 to 5.1
sql/ha_federated.cc:
  BUG #19773
  
  Hand-merge conflicts from 5.0 to 5.1
sql/ha_federated.h:
  BUG #19773
  
  Hand-merge conflicts from 5.0 to 5.1

into  ymer.(none):/usr/local/mysql/tmp-5.1


mysql-test/t/wait_timeout.test:
  Auto merged

into  bodhi.netgear:/opt/local/work/mysql-5.1-runtime-merge

removed alter table partition error test since it cannot be made predictable... name will change of created file

Fix random failures in test 'wait_timeout' that depend on exact timing.

1. Force a reconnect initially if necessary, as otherwise slow startup
might have caused a connection timeout before the test can even start.

2. Explicitly disconnect the first connection to remove confusion about
which connection aborts from timeout, causing test failure.


mysql-test/r/wait_timeout.result:
  Fix two races in test.
mysql-test/t/wait_timeout.test:
  Fix two races in test.

into  chilla.local:/home/mydev/mysql-5.1-amerge


scripts/make_binary_distribution.sh:
  Auto merged
support-files/mysql.spec.sh:
  Auto merged

into  poseidon.ndb.mysql.com:/home/tomas/mysql-5.1-ndb

into  bodhi.netgear:/opt/local/work/mysql-5.1-runtime-merge-with-5.0


mysql-test/r/create.result:
  Auto merged
mysql-test/r/ps.result:
  Auto merged
mysql-test/r/sp.result:
  Auto merged
mysql-test/t/create.test:
  Auto merged
mysql-test/t/ps.test:
  Auto merged
mysql-test/t/sp.test:
  Auto merged
sql/item_strfunc.cc:
  Auto merged
sql/sp.cc:
  Auto merged
sql/sp.h:
  Auto merged
sql/sp_head.cc:
  Auto merged
sql/sp_head.h:
  Auto merged
sql/sql_db.cc:
  Auto merged
sql/sql_insert.cc:
  Auto merged
sql/sql_lex.h:
  Auto merged
sql/sql_parse.cc:
  Auto merged
sql/sql_trigger.cc:
  Auto merged
sql/sql_view.cc:
  Auto merged
sql/table.h:
  Auto merged
sql/tztime.cc:
  Auto merged
include/my_sys.h:
  Manual merge (second attempt).
mysql-test/r/bdb.result:
  Manual merge (second attempt).
mysql-test/t/bdb.test:
  Manual merge (second attempt).
mysys/my_malloc.c:
  Manual merge (second attempt).
mysys/safemalloc.c:
  Manual merge (second attempt).
sql/ha_federated.cc:
  Manual merge (second attempt).
sql/log_event.cc:
  Manual merge (second attempt).
sql/set_var.cc:
  Manual merge (second attempt).
sql/set_var.h:
  Manual merge (second attempt).
sql/slave.cc:
  Manual merge (second attempt).
sql/slave.h:
  Manual merge (second attempt).
sql/sql_class.h:
  Manual merge (second attempt).
sql/sql_table.cc:
  Manual merge (second attempt).
sql/sql_udf.cc:
  Manual merge (second attempt).
sql/sql_yacc.yy:
  Manual merge (second attempt).

into  bodhi.netgear:/opt/local/work/mysql-5.1-runtime-merge


BitKeeper/etc/ignore:
  auto-union

mysql-test/t/ndb_dd_backuprestore.test:
  make sure only run in default cluster
mysql-test/t/rpl_ndb_dd_advance.test:
  make sure only run in default cluster
mysql-test/t/rpl_ndb_sync.test:
  make sure only run in default cluster

into  poseidon.ndb.mysql.com:/home/tomas/mysql-5.1-ndb


mysql-test/r/ndb_restore.result:
  manual merge
mysql-test/t/ndb_restore.test:
  manual merge
storage/ndb/tools/restore/consumer_restore.cpp:
  manual merge

into trift2.:/M51/bug20216-5.1


scripts/Makefile.am:
  Auto merged
scripts/make_binary_distribution.sh:
  Auto merged
support-files/mysql.spec.sh:
  Auto merged

into  chilla.local:/home/mydev/mysql-5.1-ateam


sql/field.h:
  Auto merged
sql/sql_select.cc:
  Auto merged
sql/sql_trigger.cc:
  Auto merged
sql/field.cc:
  Manual merge
sql/sql_insert.cc:
  Manual merge
sql/table.cc:
  Manual merge

into trift2.:/M51/merge-5.1

into  poseidon.ndb.mysql.com:/home/tomas/mysql-5.1-ndb

- missing order by

mysql-test/r/archive.result:
  After merge fix. It might come from the fix for
  bug 1662 (ALTER TABLE LIKE ignores DATA/INDEX DIRECTPORY)
sql/time.cc:
  After merge fix. Auto resolve failed because this piece
  of code was moved from another file to here.

into  poseidon.ndb.mysql.com:/home/tomas/mysql-5.1-ndb


storage/ndb/src/ndbapi/DictCache.cpp:
  Auto merged

into  poseidon.ndb.mysql.com:/home/tomas/mysql-5.0


ndb/src/ndbapi/DictCache.cpp:
  Auto merged

- don't invalidate tables that are in state RETRIEVING

into  chilla.local:/home/mydev/mysql-5.1-ateam


libmysqld/lib_sql.cc:
  Auto merged
libmysqld/libmysqld.c:
  Auto merged
mysql-test/r/func_sapdb.result:
  Auto merged
mysql-test/r/func_time.result:
  Auto merged
mysql-test/r/gis-rtree.result:
  Auto merged
mysql-test/r/myisam.result:
  Auto merged
mysql-test/r/symlink.result:
  Auto merged
mysql-test/t/func_time.test:
  Auto merged
mysql-test/t/key.test:
  Auto merged
mysql-test/t/myisam.test:
  Auto merged
scripts/make_binary_distribution.sh:
  Auto merged
sql/field.cc:
  Auto merged
sql-common/client.c:
  Auto merged
sql/opt_sum.cc:
  Auto merged
sql/sql_class.cc:
  Auto merged
sql/sql_parse.cc:
  Auto merged
sql/table.cc:
  Auto merged
storage/myisam/mi_check.c:
  Auto merged
storage/myisam/mi_create.c:
  Auto merged
storage/myisam/mi_delete_table.c:
  Auto merged
storage/myisam/mi_dynrec.c:
  Auto merged
storage/myisam/mi_key.c:
  Auto merged
storage/myisam/mi_rkey.c:
  Auto merged
storage/myisam/rt_index.c:
  Auto merged
storage/myisam/rt_mbr.c:
  Auto merged
support-files/mysql.spec.sh:
  Auto merged
mysql-test/r/ctype_utf8.result:
  Manual merge
mysql-test/r/key.result:
  Manual merge
mysql-test/t/ctype_utf8.test:
  Manual merge
sql/item_timefunc.cc:
  Manual merge