WL#3951 - MyISAM: Additional Error Logs for Data Corruption
Fixed partition_repair_myisam.test.

mysql-test/suite/parts/t/partition_repair_myisam.test:
  Suppress some extra warnings produced by MyISAM.

Implemented a new INFORMATION_SCHEMA table, which is intended to
provide information about tablespaces.

mysql-test/r/information_schema.result:
  Updated test result according to WL#2511. With this WL
  I_S has new TABLESPACES schema.
mysql-test/r/information_schema_db.result:
  Updated test result according to WL#2511. With this WL
  I_S has new TABLESPACES schema.
mysql-test/r/mysqlshow.result:
  Updated test result according to WL#2511. With this WL
  I_S has new TABLESPACES schema.
mysql-test/suite/funcs_1/r/is_columns_is.result:
  Updated test result according to WL#2511. With this WL
  I_S has new TABLESPACES schema.
mysql-test/suite/funcs_1/r/is_tables_is.result:
  Updated test result according to WL#2511. With this WL
  I_S has new TABLESPACES schema.
sql/handler.h:
  Added SCH_TABLESPACES to enum_schema_tables.
sql/mysql_priv.h:
  Added human readable definitions for I_S.TABLESPACES field
  identifiers.
sql/sql_show.cc:
  Added I_S.TABLESPACES schema. The code which handles I_S.FILES
  is capable to handle I_S.TABLESPACES as well. Thus we reuse
  this code and let functions/variables have more generic names.

Part II, engines instrumentation

    
When a .CSV file for table in the CSV engine contains
\X characters as part of unquoted fields, e.g.
    
2,naraya\nan
    
\n is not interpreted as a new line (it is however interpreted as a
newline in a quoted field).
    
The old algorithm copied the entire value for a unquoted field without
parsing the \X characters. 
    
The new algorithm adds the capability to handle \X characters in the 
unquoted fields of a .CSV file.

mysql-test/r/csv.result:
  Bug#40814 CSV engine does not parse \X characters when they occur in unquoted fields
  
  Contains additional test output corresponding to the new 
  tests added.
mysql-test/t/csv.test:
  Bug#40814 CSV engine does not parse \X characters when they occur in unquoted fields
  
  Contains additional tests for testing the behaviour of the CSV 
  storage engine when the fields are not enclosed in quotes and
  contain \X characters.
storage/csv/ha_tina.cc:
  Bug#40814 CSV engine does not parse \X characters when they occur in unquoted fields
  
  Changes the parsing logic of the rows in a CSV file, to parse
  \X characters that might be present in the unquoted fields.

Introduce a flag that will enable the REPLACE
command to work correctly with an underlying
storage engine that does not report unique key
conflicts in the ascending order.

sql/handler.h:
  WL#4454 change sql_insert.cc::last_uniq_key to match keys in any order
  
  Adds the flag that will be set by a
  SE that does not report unique key
  conflicts in the ascending order.
sql/sql_insert.cc:
  WL#4454 change sql_insert.cc::last_uniq_key to match keys in any order
  
  modifies the function used for a last row
  replace optimization to check for the
  HA_DUPLICATE_KEY_NOT_IN_ORDER flag.

The attached patch adds a method
handlerton::fill_is_table that can be used
instead of having to create specific
handlerton::fill_*_table methods.

sql/ha_ndbcluster.cc:
  WL#4448 Generalize the handlerton::fill_files_table call with handlerton::fill_is_table
  
  Implemented the method ndbcluster_fill_is_table
  that uses the supplied table id to switch to the
  appropriate fill_*_table method.
sql/handler.h:
  WL#4448 Generalize the handlerton::fill_files_table call with handlerton::fill_is_table
  
  Moved the enum_schema_table enumeration
  from table.h to here. 
  
  contains the declaration for the new method
  fill_is_tables.
sql/sql_show.cc:
  WL#4448 Generalize the handlerton::fill_files_table call with handlerton::fill_is_table
  
  calls the fill_is_table method instead of the
  fill_files_table method.
sql/table.h:
  WL#4448 Generalize the handlerton::fill_files_table call with handlerton::fill_is_table
  
  removed the earlier definition of enum_schema_tables.

Text conflict in mysql-test/collections/default.experimental
Text conflict in mysql-test/r/show_check.result
Text conflict in mysql-test/r/sp-code.result
Text conflict in mysql-test/suite/binlog/r/binlog_tmp_table.result
Text conflict in mysql-test/suite/rpl/t/disabled.def
Text conflict in mysql-test/t/show_check.test
Text conflict in mysys/my_delete.c
Text conflict in sql/item.h
Text conflict in sql/item_cmpfunc.h
Text conflict in sql/log.cc
Text conflict in sql/mysqld.cc
Text conflict in sql/repl_failsafe.cc
Text conflict in sql/slave.cc
Text conflict in sql/sql_parse.cc
Text conflict in sql/sql_table.cc
Text conflict in sql/sql_yacc.yy
Text conflict in storage/myisam/ha_myisam.cc

Corrected results for
 stm_auto_increment_bug33029.reject      2009-12-01
		20:01:49.000000000 +0300
       <andrei> @@ -42,9 +42,6 @@
       <andrei>  RETURN i;
       <andrei>  END//
       <andrei>  CALL p1();
       <andrei> -Warnings:
       <andrei> -Note   1592    Statement may not be safe to log in statement
		format.
       <andrei> -Note   1592    Statement may not be safe to log in statement
		format.

There should be indeed no Note present because there is in fact autoincrement 
top-level query in sp() that triggers inserting in yet another auto-inc table.
(todo: alert DaoGang to improve the test).

mysqld crashed in network_init()).

The problem was that current_thd was not ready at that point in mysqld life,
so ER() macro could not be used.

The fix is to use ER_DEFAULT() macro, which is intented for such cases.

Part 1: Instrumentation interface

merging from 5.1 to rep+2 starting at gca(5.1, next-mr) == build@mysql.com-20091104182209-iui387z35159aoyw

      
When the data directory contained a symbolic link to another
file system, and the DATA or INDEX DIRECTORY clause of a
CREATE TABLE statement referred to a subdirectory of the data
directory, this was accepted.
      
The problem was the use of a table file path name, which included
the table name without an extension, for the comparison against
the data directory path name. This was almost always a
non-existent file. The internal algorithm failed to resolve
symbolic links for non-existent files. So we compared unrelated
path names.
      
Fixed by truncating the table name from the path before resolving
symlinks. If this is also a non-existent path, the creation of
the table will fail anyway.

Backport to 5.6.0.    6.0-codebase revid: 2599.60.1


sql/sql_table.cc:
  Changed test for data directory to exclude the table name from the
  comparison.

                      .frm file
      
Added FLUSH TABLES before myisampack --join operation to fix the test 
warnings or errors
Removed unused variable in create_dest_frm() method

table .frm file
            
Problem:
========
Myisampack --join did not create the destination table .frm file. 
The user had to copy one of the source table .frm file as destination .frm 
file for mysql server to recognize. This is just 'user-friendliness' issue.
            
How it was solved
=================
After successful join and compression we copy the frm file from the first 
source table.
            
Functionality added
===================
myisampack --join=/path/t3 /path/t1 /path/t2 creates 
/path/t3.frm (which is bascially copied from first table's frm /path/t1)
      
Tests
=====
Modified myisampack.test to test two scenario's
1. Positive myisampack --join test
   In this case after the join operation is done,we test if the destination 
   table is accessible from the server
2. Positive myisampack --join test with an existing .frm file.
   We test the above case with an existing .frm file for the destination 
   table. It should return success even in this case.
3. Positive myisampack --join test with no .frm file for source tables
   We test the join operation with no .frm files for source tables. It should
   complete the join operation without any warnings and error messages
4. Negative myisampack --join test
   We test myisampack --join with existing .MYI,.MDI,.frm files for the 
   destination table. It should fail with exit status 2 in this case.

branch due to BUG#46853
      
Commented out the testcase for Bug#45816 in mysql-pe branch
due to valgrind warnings. Please see Bug#46853

Select queries on archive tables when joined on their primary keys
returns no results(empty set)
      
Archive storage doesn't inform the handler about the fetched record 
status when it is found. Fixed the archive storage engine to update
the record status when it fetches successfully

WL#3951 - MyISAM: Additional Error Logs for Data Corruption

When table corruption is detected, in addition to current error message
provide following information:
- list of threads (and queries) accessing a table;
- thread_id of a thread that detected corruption;
- source file name and line number where this corruption was detected;
- optional extra information (string).

mysql-test/r/myisam_crash_before_flush_keys.result:
  Adjusted a test case according to WL#3951.
mysql-test/t/myisam_crash_before_flush_keys.test:
  Adjusted a test case according to WL#3951.
storage/myisam/CMakeLists.txt:
  Added mi_extrafunc.h to myisam sources list.
storage/myisam/Makefile.am:
  Added mi_extrafunc.h to myisam headers list.
storage/myisam/ha_myisam.cc:
  Added _mi_report_crashed() function (reports additional information
  whenever table corruption is detected).
storage/myisam/mi_extrafunc.h:
  All standalone programs must define their version of _mi_report_crashed()
  by including mi_extrafunc.h.
storage/myisam/mi_locking.c:
  For every call to mi_lock_database(F_[RD|WR|EXTRA_]LCK) - add
  current thread to the list of threads accessing this table.
  
  For every call to mi_lock_database(F_UNLCK) - remove current
  thread from the list of threads accessing this table.
storage/myisam/mi_test1.c:
  All standalone programs must define their version of _mi_report_crashed()
  by including mi_extrafunc.h.
storage/myisam/mi_test2.c:
  All standalone programs must define their version of _mi_report_crashed()
  by including mi_extrafunc.h.
storage/myisam/mi_test3.c:
  All standalone programs must define their version of _mi_report_crashed()
  by including mi_extrafunc.h.
storage/myisam/myisam_ftdump.c:
  All standalone programs must define their version of _mi_report_crashed()
  by including mi_extrafunc.h.
storage/myisam/myisamchk.c:
  All standalone programs must define their version of _mi_report_crashed()
  by including mi_extrafunc.h.
storage/myisam/myisamdef.h:
  Extra elements are added to MI_INFO and MYISAM_SHARE structures.
  MI_INFO is extended with LIST element, that holds a pointer to
  THD object accessing a table. MYISAM_SHARE is extended with LIST
  (list of threads accessing a table).
  
  Whenever table is marked as crashed, call mi_report_crashed() macro
  to provide useful information.
storage/myisam/myisamlog.c:
  All standalone programs must define their version of _mi_report_crashed()
  by including mi_extrafunc.h.
storage/myisam/myisampack.c:
  All standalone programs must define their version of _mi_report_crashed()
  by including mi_extrafunc.h.
storage/myisam/rt_test.c:
  All standalone programs must define their version of _mi_report_crashed()
  by including mi_extrafunc.h.
storage/myisam/sp_test.c:
  All standalone programs must define their version of _mi_report_crashed()
  by including mi_extrafunc.h.
storage/myisammrg/ha_myisammrg.cc:
  For each unedrlying table initialize `in_use' variable.

The following 6.0 revisions were analyzed:
  - sp1r-brian@zim.(none)-20071228102738-21894
  - sp1r-brian@zim.(none)-20071228121841-56447
  - sp1r-brian@zim.(none)-20071228205403-56423
  - sp1r-brian@zim.(none)-20071228221139-55341
  - sp1r-brian@zim.(none)-20071228233443-55352
  - sp1r-brian@zim.(none)-20071229094527-61763
  - sp1r-brian@zim.(none)-20071230203739-61746
  - sp1r-brian@zim.(none)-20080102213805-61741
  - sp1r-brian@zim.(none)-20080103201041-61746
  - sp1r-brian@zim.(none)-20080104234927-59875
  - sp1r-brian@zim.(none)-20080105005827-59874
  - sp1r-brian@zim.(none)-20080105012020-59865
  - sp1r-brian@zim.(none)-20080106003858-59857
  - sp1r-brian@zim.(none)-20080123195552-31680
  - sp1r-brian@zim.(none)-20080124201848-29999
  - sp1r-brian@zim.(none)-20080129075144-36991
  - sp1r-cbell/Chuck@mysql_cab_desk.-20080215041806-21954
  - sp1r-vvaintroub@wva.-20080212124547-06272
  - sp1r-dfischer/mysqldev@mysql.com/production.mysql.com-20071223184258-15140
  - sp1r-brian@zim.(none)-20080206122216-35774
  - sp1r-baker@bk-internal.mysql.com-20080209005622-35947
  - sp1r-baker@bk-internal.mysql.com-20080224215608-24613
  - sp1r-baker@bk-internal.mysql.com-20080307170710-63543
  - sp1r-baker@bk-internal.mysql.com-20080312233205-13069
  - sp1r-Reggie@core.-20080402175211-28643
  - kpettersson@mysql.com-20080901101150-4ne74r8v0492vv42
  - alik@sun.com-20090805173811-9fzt0ymcp9tsvn7k
  - alik@sun.com-20090805173937-fcv1fdveodq5x9gb
  - alik@sun.com-20090805175009-g1od16i3t1xkw2qr
  - kostja@sun.com-20090805200643-j9i4fy7ii8ijho5c
  - alik@sun.com-20090807195303-j4fb5m4l1dgdahwo
  - alik@sun.com-20090808114848-3rkzr9kifrijzaqy
  - alik@sun.com-20090810041739-ugmx34h34uid8mox
  - alik@sun.com-20090810105306-rf43rfyzzblsy5e7
  - alik@sun.com-20090810123113-ccdjwai68b5woqdm
  - alik@sun.com-20090811080423-gb7pibec1znaydzy
  - alik@sun.com-20090811082130-5uckar1vx3kdsw7g
  - alik@sun.com-20090812202051-uqkfwwxxcjvo5ean

The following bugfixes are also backported within this patch:
  - Bug#34292: netdb.h missing in hostname.cc
  - Bug#39153: Failing to lookup a host name can lead to crash
    in current IPv6 implementation
  - Bug#38247: Server does not resolve connecting ip's
  - Bug#43006: main.skip_name_resolve fails on Windows in PB2
  - Bug#45606: ACL requires IPv4-mapped addresses to be used
  - Bug#45584: Host name cache does not work as a cache