There was two problems:
The first was the symptom, caused by bad error handling in
ha_partition. It did not handle print_error etc. when
having no partitions (when used by dummy handler).

The second was the real problem that when dropping tables
it reused the table type (storage engine) from when the lock
was asked for, not the table type that it had when gaining
the exclusive name lock. So that it tried to delete tables
from wrong storage engines.

Solutions for the first problem was to accept some handler
calls to the partitioning handler even if it was not setup
with any partitions, and also if possible fallback
to use the base handler's default functions.

Solution for the second problem was to remove the optimization
to reuse the definition from the cache, instead always check
the frm-file when holding the LOCK_open mutex

(updated with a fix for a debug print crash and better
comments as required by reviewer, and removed optimization
to avoid reading the frm-file).

mysql-test/r/partition_debug_sync.result:
  Bug#42438: Crash ha_partition::change_table_ptr
  
  New result file using DEBUG_SYNC for deterministic results.
mysql-test/t/partition_debug_sync.test:
  Bug#42438: Crash ha_partition::change_table_ptr
  
  New test file using DEBUG_SYNC for deterministic results.
sql/ha_partition.cc:
  Bug#42438: Crash ha_partition::change_table_ptr
  
  allow some handler calls, used by error handling, even when
  no partitions are setup. Fallback to default handling if possible.
sql/sql_base.cc:
  Bug#42438: Crash ha_partition::change_table_ptr
  
  Added DEBUG_SYNC point for deterministic test cases.
sql/sql_table.cc:
  Bug#42438: Crash ha_partition::change_table_ptr
  
  Always use the table type written in the .frm-file
  (i.e. the current table type) when deleting a table.
  
  Moved the check for log-table to not depend of the cache.
  
  Added DEBUG_SYNC points for deterministic test cases.