MDEV-8889 - Assertion `next_insert_id == 0' failed in handler::ha_external_lock

There was a race condition between delayed insert thread and connection thread
actually performing INSERT/REPLACE DELAYED. It was triggered by concurrent
INSERT/REPLACE DELAYED and statements that flush the same table either
explicitely or implicitely (like FLUSH TABLE, ALTER TABLE, ...).

This race condition was caused by a gap in delayed thread shutdown logic,
which allowed concurrent connection running INSERT/REPLACE DELAYED to change
essential data consequently leaving table in semi-consistent state.

Specifically query thread could decrease "tables_in_use" reference counter in
this gap, causing delayed insert thread to shutdown without releasing auto
increment and table lock.

Fixed by extending condition so that delayed insert thread won't shutdown
until there're locked tables.

Also removed volatile qualifier from tables_in_use and stacked_inserts since
they're supposed to be protected by mutexes.

sql/sql_insert.cc

@@ -1978,7 +1978,7 @@ public:
   TABLE *table;
   mysql_mutex_t mutex;
   mysql_cond_t cond, cond_client;
-  volatile uint tables_in_use,stacked_inserts;
+  uint tables_in_use, stacked_inserts;
   volatile bool status;
   /**
     When the handler thread starts, it clones a metadata lock ticket
@@ -2863,7 +2863,8 @@ pthread_handler_t handle_delayed_insert(void *arg)
         lock_count=di->lock_count();
         mysql_mutex_unlock(&LOCK_delayed_insert);
         mysql_mutex_lock(&di->mutex);
-        if (!lock_count && !di->tables_in_use && !di->stacked_inserts)
+        if (!lock_count && !di->tables_in_use && !di->stacked_inserts &&
+            !thd->lock)
           break;					// Time to die
       }