Skip to content

Draft: component_package: release all import lock levels to avoid ABBA deadlock

Jérome Perrin requested to merge fix/py2-deadlock-at-startup into master

On CPython 2, the import lock is re-entrant: each nested __import__ increments import_lock_level, and imp.release_lock() decrements it by one and only release when the count is 0 ( see _PyImport_AcquireLock and _PyImport_ReleaseLock in https://github.com/python/cpython/blob/v2.7.18/Python/import.c#L291-L330 )

void
_PyImport_AcquireLock(void)
{
    ...
    if (import_lock_thread == me) {
        import_lock_level++;
        return;
    }
    if (import_lock_thread != -1 || !PyThread_acquire_lock(import_lock, 0))
    {
        PyThreadState *tstate = PyEval_SaveThread();
        PyThread_acquire_lock(import_lock, 1);
        PyEval_RestoreThread(tstate);
    }
    import_lock_thread = me;
    import_lock_level = 1;
}

int
_PyImport_ReleaseLock(void)
{
    ...
    import_lock_level--;
    if (import_lock_level == 0) {
        PyThread_release_lock(import_lock);
    }
    return 1;
}

When find_module or load_module is called during a nested import (lock level >= 2), a single release() decrements the level (e.g. 2 -> 1) but does not actually release the underlying lock. Any other thread that needs the import lock remains blocked.

This causes an ABBA deadlock in the following real-world scenario:

  • Thread A

    • Holds aq_method_lock (dynamic_module.__getattr__ → generateLazyPortalTypeClass → InitializeClass)
    • Executes from AccessControl.Permission import ...
    • Blocks waiting for the import lock
  • Thread B

    • Holds the import lock at level >= 2:
      • __import__('Products.X.Y') (level=1)
      • module body performs another import (level=2)
      • find_module is called with a Products path
    • Calls release() once (level 2 -> 1)
    • The underlying thread lock remains held

Fix: loop global_import_lock.release() until held() returns False (draining all levels), then re-acquire the same count in the finally block. This ensures the actual lock is freed regardless of nesting depth, allowing the other thread to proceed.

Regression introduced in 673e7dd3 ("py3.12: imp module has been removed", 2025-10-21), which changed the old try/except RuntimeError pattern to a lock_held() check + single release.

Edited by Jérome Perrin

Merge request reports