[Python] Loading standard C++ library makes ctypesgen's Python binding crash if library is not...

I'm experiencing a wierd crash when trying to run a C++ library entry point from a Python script. I'm under Windows 10.

I isolated this into a very simple example.

Here is my C++ code:

sdetests_lib_bind_python.h:

#pragma once

#ifdef SDETESTS_LIB_BIND_PYTHON_EXPORTS
/** Export symbols statement */
#define SDETESTS_LIB_BIND_PYTHON __declspec(dllexport)
#else
/** Import symbols statement */
#define SDETESTS_LIB_BIND_PYTHON __declspec(dllimport)
#endif

#ifdef __cplusplus
extern "C" {
#endif

SDETESTS_LIB_BIND_PYTHON void func(int lock);


#ifdef __cplusplus
}
#endif


sdetests_lib_bind_python.cpp:

#include "sdetests_lib_bind_python.h"

#include <iostream>

#include <mutex>

SDETESTS_LIB_BIND_PYTHON void func( int lock )
{
if (lock!=0)
{
std::cout << "Creating mutex" << std::endl;
std::recursive_mutex mutex;
std::cout << "Created mutex" << std::endl;
std::cout << "Locking mutex" << std::endl;
std::lock_guard<std::recursive_mutex> lock_guard(mutex);
std::cout << "Locked mutex" << std::endl;
}
std::cout << "Hello world" << std::endl;
}


I compile this as a shared library using Visual Studio 2022 (MSVC 19.43.34808.0)

Later I run ctypesgen to generate a binding sdetests_lib_bind_python.py and I use it from a very simple Python script:

import sdetests_lib_bind_python

print("Calling func without lock")
sdetests_lib_bind_python.func(0)

print("Calling func with lock")
sdetests_lib_bind_python.func(1)


This works fine and output:

Hello from Python script without lock
Hello world
Hello from Python script with lock
Creating mutex
Created mutex
Locking mutex
Locked mutex
Hello world


However, if I load PyQt from my Python script, being now:

from PyQt5.QtCore import QT_VERSION_STR, PYQT_VERSION_STR
print("Qt: v", QT_VERSION_STR, "\tPyQt: v", PYQT_VERSION_STR)

import sdetests_lib_bind_python

print("Calling func without lock")
sdetests_lib_bind_python.func(0)

print("Calling func with lock")
sdetests_lib_bind_python.func(1)


Now the program crashs with output:

Qt: v 5.15.2 PyQt: v 5.15.9
Hello from Python script without lock
Hello world
Hello from Python script with lock
Creating mutex
Created mutex
Locking mutex
Traceback (most recent call last):
File "example_sdetests_lib_bind_python_def.py", line 10, in <module>
sdetests_lib_bind_python_def.func(1)
OSError: exception: access violation reading 0x0000000000000000


So the stcipt crashs if PyQt is loaded BEFORE (see below) sdetests_lib_bind_python and if sdetests_lib_bind_python tries to use a std::recursive_mutex.

Note that the script also crashs with other modules than PyQt, like rclpy (all this comes from a ROS2 environment before I isolated it as an MCVE here).

Note that NO crash is observed:

• If import sdetests_lib_bind_python statement is the very first line of the script, then I can load PyQt, rclpy, and everything works fine. It only crashs if those other modules are loaded BEFORE sdetests_lib_bind_python
• If I compile with an older compiler (MSVC 19.38.33135.0)...this crash appeared recently in my enviropnment after a compiler upgrade
• If my library does not use std::recursive_mutex, loading <mutex> has no impact, but creating a mutex makes the whole thing crash with OSError.

I'm packaging my library for external users, so asking them to load it first or, if this could be a solution, to upgrade their PyQt or rlypy modules (and whatever other module leading to a crash) is not an acceptable solution for me...

Continue reading...