Merge work:/my/mysql into tik.mysql.fi:/home/my/mysql

Docs/manual.texi:
  Auto merged

Docs/manual.texi

@@ -5467,9 +5467,10 @@ something that is of course not true.  We could make things even worse
 by just taking the test where PostgreSQL performs worst and claim that 
 MySQL is more than 2000 times faster than PostgreSQL.
 
-The case is that MySQL does a lot of optimizations that PostgreSQL doesn't 
-do and the other way around.  An SQL optimizer is a very complex thing, and
-a company could spend years on just making the optimizer faster and faster.
+The case is that MySQL does a lot of optimizations that PostgreSQL
+doesn't do. This is of course also true the other way around.  An SQL
+optimizer is a very complex thing, and a company could spend years on
+just making the optimizer faster and faster.
 
 When looking at the benchmark results you should look for things that
 you do in your application and just use these results to decide which
@@ -5604,12 +5605,14 @@ MySQL with 2000 simultaneous connections doing 400 queries per second.
 
 It sounded like he was using a Linux kernel that either had some
 problems with many threads, such as kernels before 2.4, which had a problem 
-with this but we have documented how to fix this and Tim should be aware of
-this problem.  The other possible problem could have been an old glibc
-library and that Tim didn't use a MySQL binary from our site, which is
-linked with a corrected glibc library, but had compiled a version of his
-own with.  In any of the above cases, the symptom would have been exactly
-what Tim had measured.
+with many threads on multi-CPU machines. We have documented in this manual
+how to fix this and Tim should be aware of this problem.
+
+The other possible problem could have been an old glibc library and
+that Tim didn't use a MySQL binary from our site, which is linked with
+a corrected glibc library, but had compiled a version of his own with.
+In any of the above cases, the symptom would have been exactly what Tim
+had measured.
 
 We asked Tim if we could get access to his data so that we could repeat
 the benchmark and if he could check the MySQL version on the machine to
@@ -5618,6 +5621,16 @@ He has not done that yet.
 
 Because of this we can't put any trust in this benchmark either :(
 
+Over time things also changes and the above benchmarks are not that
+relevant anymore.  MySQL now have a couple of different table handlers
+with different speed/concurrency tradeoffs.  @xref{Table types}. It
+would be interesting to see how the above tests would run with the
+different transactional table types in MySQL.  PostgreSQL has of course
+also got new features since the test was made.  As the above test are
+not publicly available there is no way for us to know how the
+database would preform in the same tests today.
+
+
 Conclusion:
 
 The only benchmarks that exist today that anyone can download and run
@@ -5632,15 +5645,15 @@ The thing we find strange is that every test we have seen about
 PostgreSQL, that is impossible to reproduce, claims that PostgreSQL is
 better in most cases while our tests, which anyone can reproduce,
 clearly shows otherwise.  With this we don't want to say that PostgreSQL
-isn't good at many things (it is!).  We would just like to see a fair test
-where they are very good so that we could get some friendly competition
-going!
+isn't good at many things (it is!) or that it isn't faster than MySQL
+under certain conditions.  We would just like to see a fair test where
+they are very good so that we could get some friendly competition going!
 
 For more information about our benchmarks suite @xref{MySQL Benchmarks}. 
 
-We are working on an even better benchmark suite, including much better
-documentation of what the individual tests really do, and how to add more
-tests to the suite.
+We are working on an even better benchmark suite, including multi user
+tests, and a better documentation of what the individual tests really
+do and how to add more tests to the suite.
 
 
 @node TODO,  , Comparisons, Introduction
@@ -6149,9 +6162,9 @@ Nothing;  In the long run we plan to be fully ANSI 92 / ANSI 99 compliant.
 @menu
 * Quick Standard Installation::  Quick Standard Installation of MySQL
 * General Installation Issues::  General Installation Issues
-* Installing source::            Installing a MySQL source distribution
-* Post-installation::            Post-installation setup and testing
-* Upgrade::                      Upgrading/Downgrading MySQL
+* Installing source::           Installing a MySQL source distribution
+* Post-installation::           Post-installation setup and testing
+* Upgrade::                     Upgrading/Downgrading MySQL
 * Operating System Specific Notes::  Operating System Specific Notes
 @end menu
 
@@ -8202,7 +8215,7 @@ If your client programs are using threads, you need to also compile a
 thread-safe version of the MySQL client library with the
 @code{--with-thread-safe-client} configure options. This will create a
 @code{libmysqlclient_r} library with which you should link your threaded
-applications.  @xref{Thread-safe clients}.
+applications.  @xref{Threaded clients}.
 
 @item
 Options that pertain to particular systems can be found in the
@@ -38964,9 +38977,10 @@ likely it is that we can fix the problem!
 * C API datatypes::             C API Datatypes
 * C API function overview::     C API Function Overview
 * C API functions::             C API Function Descriptions
+* C Thread functions::          
 * C API problems::              Common questions and problems when using the C API
 * Building clients::            Building Client Programs
-* Thread-safe clients::         How to Make a Thread-safe Client
+* Threaded clients::            How to Make a Threaded Client
 @end menu
 
 The C API code is distributed with MySQL. It is included in the
@@ -39007,7 +39021,7 @@ the buffer associated with a connection is not decreased until the connection
 is closed, at which time client memory is reclaimed.
 
 For programming with threads, consult the 'how to make a thread-safe
-client' chapter. @xref{Thread-safe clients}.
+client' chapter. @xref{Threaded clients}.
 
 
 @node C API datatypes, C API function overview, C, C
@@ -39455,7 +39469,7 @@ recently invoked function that can succeed or fail, allowing you to determine
 when an error occurred and what it was.
 
 
-@node C API functions, C API problems, C API function overview, C
+@node C API functions, C Thread functions, C API function overview, C
 @subsection C API Function Descriptions
 
 @menu
@@ -40373,7 +40387,7 @@ of @code{mysql_field_count()} whether or not the statement was a
 
 @code{MYSQL_FIELD_OFFSET mysql_field_seek(MYSQL_RES *result, MYSQL_FIELD_OFFSET offset)}
 
-* Thread-safe clients::         How to Make a Thread-safe Client
+* Threaded clients::         How to Make a Threaded Client
 @subsubheading Description
 
 Sets the field cursor to the given offset.   The next call to
@@ -41650,8 +41664,71 @@ The connection to the server was lost during the query.
 An unknown error occurred.
 @end table
 
+@node C Thread functions, C API problems, C API functions, C
+@subsection C Threaded Function Descriptions
+
+You need to use the following functions when you want to create a
+threaded client. @xref{Threaded clients}.
+
+@menu
+* my_init::                     
+* my_thread_init()::            
+* my_thread_end()::             
+@end menu
+
+@node my_init, my_thread_init(), C Thread functions, C Thread functions
+@subsubsection @code{my_init()}
+
+@findex @code{my_init()}
+
+@subsubheading Description
+
+This function needs to be called once in the program before calling any
+MySQL function.  This initializes some global variables that MySQL
+needs.  If you are using a thread safe client library, this will also
+call @code{my_thread_init()} for this thread.
+
+This is automaticly called by @code{mysql_init()}
+and @code{mysql_connect()}.
+
+@subsubheading Return Values
+
+none.
+
+@node my_thread_init(), my_thread_end(), my_init, C Thread functions
+@subsubsection @code{my_thread_init()}
+
+@findex @code{my_thread_init()}
+
+@subsubheading Description
+
+This function needs to be called for each created thread to initialize
+thread specific variables.
+
+This is automaticly called by @code{my_init()} and @code{mysql_connect()}.
+
+@subsubheading Return Values
+
+none.
+
+@node my_thread_end(),  , my_thread_init(), C Thread functions
+@subsubsection @code{my_thread_end()}
+
+@findex @code{my_thread_end()}
+
+@subsubheading Description
 
-@node C API problems, Building clients, C API functions, C
+This function needs to be called before calling @code{pthread_exit()} to
+freed memory allocated by @code{my_thread_init()}.
+
+Note that this function is NOT invoked automaticly be the client
+library!
+
+@subsubheading Return Values
+
+none.
+
+@node C API problems, Building clients, C Thread functions, C
 @subsection Common questions and problems when using the C API
 
 @tindex @code{mysql_query()}
@@ -41806,7 +41883,7 @@ If this happens on your system, you must include the math library by
 adding @code{-lm} to the end of the compile/link line.
 
 
-@node Building clients, Thread-safe clients, C API problems, C
+@node Building clients, Threaded clients, C API problems, C
 @subsection Building Client Programs
 
 @cindex client programs, building
@@ -41827,11 +41904,11 @@ For clients that use MySQL header files, you may need to specify a
 files.
 
 
-@node Thread-safe clients,  , Building clients, C
-@subsection How to Make a Thread-safe Client
+@node Threaded clients,  , Building clients, C
+@subsection How to Make a Threaded Client
 
-@cindex clients, thread-safe
-@cindex thread-safe clients
+@cindex clients, threaded
+@cindex threaded clients
 
 The client library is almost thread safe. The biggest problem is
 that the subroutines in @file{net.c} that read from sockets are not
@@ -41846,20 +41923,21 @@ Windows binaries are by default compiled to be thread safe).
 Newer binary distributions should have both a normal and a
 thread-safe client library.
 
-To get a really thread-safe client where you can interrupt the client
-from other threads and set timeouts when talking with the MySQL
-server, you should use the @code{-lmysys}, @code{-lstring}, and @code{-ldbug}
-libraries and the @code{net_serv.o} code that the server uses.
+To get a threaded client where you can interrupt the client from other
+threads and set timeouts when talking with the MySQL server, you should
+use the @code{-lmysys}, @code{-lstring}, and @code{-ldbug} libraries and
+the @code{net_serv.o} code that the server uses.
 
 If you don't need interrupts or timeouts, you can just compile a thread
 safe client library @code{(mysqlclient_r)} and use this.  @xref{C,,
 MySQL C API}.  In this case you don't have to worry about the
 @code{net_serv.o} object file or the other MySQL libraries.
 
-When using a threaded client and you want to use timeouts and interrupts,
-you can make great use of the routines in the @file{thr_alarm.c} file.
-If you are using routines from the @code{mysys} library, the only thing
-you must remember is to call @code{my_init()} first!
+When using a threaded client and you want to use timeouts and
+interrupts, you can make great use of the routines in the
+@file{thr_alarm.c} file.  If you are using routines from the
+@code{mysys} library, the only thing you must remember is to call
+@code{my_init()} first! @xref{C Thread functions}.
 
 All functions except @code{mysql_real_connect()} are by default
 thread safe.  The following notes describe how to compile a thread safe
@@ -41906,11 +41984,38 @@ If you program with POSIX threads, you can use
 establish and release a mutex lock.
 @end itemize
 
+You need to know the following if you have a thread that is calling
+MySQL functions, but that thread has not created the connection to the
+MySQL database:
+
+When you call @code{mysql_init()} or @code{mysql_connect()}, MySQL will
+create a thread specific variable for the thread that is used by the
+debug library (among other things).
+
+If you have in a thread call a MySQL function, before a thread has
+called @code{mysql_init()} or @code{mysql_connect()}, the thread will
+not have the necessary thread specific variables in place and you are
+likely to end up with a core dump sooner or later.
+ 
+The get things to work smoothly you have to do the following:
+
+@enumerate
+@item
+Call @code{my_init()} at the start of your program if it calls
+any other MySQL function before calling @code{mysql_real_connect()}.
+@item
+Call @code{my_thread_init()} in the thread handler before calling
+any MySQL function.
+@item
+In the thread, call @code{my_thread_end()} before calling
+@code{pthread_exit()}.  This will free the memory used by MySQL thread
+specific variables.
+@end enumerate
+
 You may get some errors because of undefined symbols when linking your
 client with @code{mysqlclient_r}. In most cases this is because you haven't
 included the thread libraries on the link/compile line.
 
-
 @node Cplusplus, Java, C, Clients
 @section MySQL C++ APIs
 
@@ -44221,34 +44326,6 @@ thread that is waiting on the disk-full condition will allow the other
 threads to continue.
 @end itemize
 
-You need to know the following if you have a thread that is calling
-MySQL functions, but that thread has not created the connection to the
-MySQL database:
-
-When you call @code{mysql_init()} or @code{mysql_connect()}, MySQL will
-create a thread specific variable for the thread that is used by the
-debug library (among other things).
-
-If you have in a thread call a MySQL function, before a thread has
-called @code{mysql_init()} or @code{mysql_connect()}, the thread will
-not have the necessary thread specific variables in place and you are
-likely to end up with a core dump sooner or later.
- 
-The get things to work smoothly you have to do the following:
-
-@enumerate
-@item
-Call @code{my_init()} at the start of your program if it calls
-any other MySQL function before calling @code{mysql_real_connect()}.
-@item
-Call @code{my_thread_init()} in the thread handler before calling
-any MySQL function.
-@item
-In the thread, call @code{my_thread_end()} before calling
-@code{pthread_exit()}.  This will free the memory used by MySQL thread
-specific variables.
-@end enumerate
-
 Exceptions to the above behaveour is when you use @code{REPAIR} or
 @code{OPTIMIZE} or when the indexes are created in a batch after an
 @code{LOAD DATA INFILE} or after an @code{ALTER TABLE} statement.
@@ -46707,6 +46784,8 @@ not yet 100% confident in this code.
 @appendixsubsec Changes in release 3.23.42
 @itemize @bullet
 @item
+Fixed problem when using @code{LOCK TABLES} and @code{BDB} tables.
+@item
 Fixed problem with @code{REPAIR TABLE} on MyISAM tables with row lengths
 between 65517 - 65520 bytes
 @item

sql-bench/test-create.sh

@@ -39,13 +39,21 @@ $opt_loop_count=10000; # Change this to make test harder/easier
 chomp($pwd = `pwd`); $pwd = "." if ($pwd eq '');
 require "$pwd/bench-init.pl" || die "Can't read Configuration file: $!\n";
 
+$create_loop_count=$opt_loop_count;
 if ($opt_small_test)
 {
   $opt_loop_count/=100;
+  $create_loop_count/=1000;
 }
 
 $max_tables=min($limits->{'max_tables'},$opt_loop_count);
 
+if ($opt_small_test)
+{
+  $max_tables=10;
+}
+
+
 print "Testing the speed of creating and droping tables\n";
 print "Testing with $max_tables tables and $opt_loop_count loop count\n\n";
 
@@ -177,7 +185,7 @@ print "Testing create+drop\n";
 
 $loop_time=new Benchmark;
 
-for ($i=1 ; $i <= $opt_loop_count ; $i++)
+for ($i=1 ; $i <= $create_loop_count ; $i++)
 {
   do_many($dbh,$server->create("bench_$i",
 			       ["i int NOT NULL",
@@ -190,7 +198,7 @@ for ($i=1 ; $i <= $opt_loop_count ; $i++)
 }
 
 $end_time=new Benchmark;
-print "Time for create+drop ($opt_loop_count): " .
+print "Time for create+drop ($create_loop_count): " .
     timestr(timediff($end_time, $loop_time),"all") . "\n";
 
 if ($opt_fast && defined($server->{vacuum}))

sql/ha_berkeley.cc

@@ -1658,12 +1658,15 @@ int ha_berkeley::external_lock(THD *thd, int lock_type)
   {
     if (!thd->transaction.bdb_lock_count++)
     {
+      changed_rows=0;
       /* First table lock, start transaction */
-      if ((thd->options & (OPTION_NOT_AUTO_COMMIT | OPTION_BEGIN)) &&
+      if ((thd->options & (OPTION_NOT_AUTO_COMMIT | OPTION_BEGIN |
+			   OPTION_TABLE_LOCK)) &&
 	  !thd->transaction.all.bdb_tid)
       {
+	DBUG_ASSERT(thd->transaction.stmt.bdb_tid != 0);
 	/* We have to start a master transaction */
-	DBUG_PRINT("trans",("starting transaction"));
+	DBUG_PRINT("trans",("starting transaction all"));
 	if ((error=txn_begin(db_env, 0,
 			     (DB_TXN**) &thd->transaction.all.bdb_tid,
 			     0)))
@@ -1671,8 +1674,10 @@ int ha_berkeley::external_lock(THD *thd, int lock_type)
 	  thd->transaction.bdb_lock_count--;	// We didn't get the lock /* purecov: inspected */
 	  DBUG_RETURN(error); /* purecov: inspected */
 	}
+	if (thd->in_lock_tables)
+	  DBUG_RETURN(0);			// Don't create stmt trans
       }
-      DBUG_PRINT("trans",("starting transaction for statement"));
+      DBUG_PRINT("trans",("starting transaction stmt"));
       if ((error=txn_begin(db_env,
 			   (DB_TXN*) thd->transaction.all.bdb_tid,
 			   (DB_TXN**) &thd->transaction.stmt.bdb_tid,
@@ -1684,7 +1689,6 @@ int ha_berkeley::external_lock(THD *thd, int lock_type)
       }
     }
     transaction= (DB_TXN*) thd->transaction.stmt.bdb_tid;
-    changed_rows=0;
   }
   else
   {
@@ -1722,6 +1726,7 @@ int ha_berkeley::start_stmt(THD *thd)
   DBUG_ENTER("ha_berkeley::start_stmt");
   if (!thd->transaction.stmt.bdb_tid)
   {
+    DBUG_PRINT("trans",("starting transaction stmt"));
     error=txn_begin(db_env, (DB_TXN*) thd->transaction.all.bdb_tid,
 		    (DB_TXN**) &thd->transaction.stmt.bdb_tid,
 		    0);

sql/mysql_priv.h

@@ -163,7 +163,8 @@ void kill_one_thread(THD *thd, ulong id);
 #define OPTION_BIN_LOG          OPTION_BUFFER_RESULT*2
 #define OPTION_NOT_AUTO_COMMIT	OPTION_BIN_LOG*2
 #define OPTION_BEGIN		OPTION_NOT_AUTO_COMMIT*2
-#define OPTION_QUICK		OPTION_BEGIN*2
+#define OPTION_TABLE_LOCK	OPTION_BEGIN*2
+#define OPTION_QUICK		OPTION_TABLE_LOCK*2
 #define OPTION_QUOTE_SHOW_CREATE OPTION_QUICK*2
 #define OPTION_INTERNAL_SUBTRANSACTIONS OPTION_QUOTE_SHOW_CREATE*2
 

sql/sql_base.cc

@@ -413,7 +413,10 @@ void close_thread_tables(THD *thd, bool locked)
   DBUG_ENTER("close_thread_tables");
 
   if (thd->locked_tables)
+  {
+    ha_commit_stmt(thd);			// If select statement
     DBUG_VOID_RETURN;				// LOCK TABLES in use
+  }
 
   TABLE *table,*next;
   bool found_old_table=0;

sql/sql_parse.cc

@@ -81,7 +81,8 @@ static void init_signals(void)
 inline bool end_active_trans(THD *thd)
 {
   int error=0;
-  if (thd->options & (OPTION_NOT_AUTO_COMMIT | OPTION_BEGIN))
+  if (thd->options & (OPTION_NOT_AUTO_COMMIT | OPTION_BEGIN |
+		      OPTION_TABLE_LOCK))
   {
     thd->options&= ~(ulong) (OPTION_BEGIN | OPTION_STATUS_NO_TRANS_UPDATE);
     thd->server_status&= ~SERVER_STATUS_IN_TRANS;
@@ -1825,7 +1826,11 @@ mysql_execute_command(void)
     {
       thd->lock=thd->locked_tables;
       thd->locked_tables=0;			// Will be automaticly closed
+    }
+    if (thd->options & OPTION_TABLE_LOCK)
+    {
       end_active_trans(thd);
+      thd->options&= ~(ulong) (OPTION_TABLE_LOCK);
     }
     if (thd->global_read_lock)
     {
@@ -1847,12 +1852,15 @@ mysql_execute_command(void)
     if (check_db_used(thd,tables) || end_active_trans(thd))
       goto error;
     thd->in_lock_tables=1;
+    thd->options|= OPTION_TABLE_LOCK;
     if (!(res=open_and_lock_tables(thd,tables)))
     {
       thd->locked_tables=thd->lock;
       thd->lock=0;
       send_ok(&thd->net);
     }
+    else
+      thd->options&= ~(ulong) (OPTION_TABLE_LOCK);
     thd->in_lock_tables=0;
     break;
   case SQLCOM_CREATE_DB: