Random exception of the Python interpreter in a free-threaded build. about cpython HOT 6 OPEN

I'm not sure any of this is related to the free-threaded build, but there are a few bugs in your code example:

• Py_NewInterpreterFromConfig sets the active thread-state. The subsequent PyEval_RestoreThread() crashes
• You are missing a Py_EndInterpreter()

Here is a similar, fixed example:

    PyThreadState *prev = PyThreadState_Swap(NULL);

    PyInterpreterConfig conf = { 0 };
    conf.check_multi_interp_extensions = 1;
    conf.allow_threads = 1;
    conf.gil = PyInterpreterConfig_OWN_GIL;
    
    PyThreadState* tstate;
    Py_NewInterpreterFromConfig(&tstate, &conf); // creating new interpreter in new thread

    ExecuteScript(name, console_id, argc, argv); //PyRun_String-like func

    Py_EndInterpreter(tstate);
    PyThreadState_Swap(prev);

from cpython.

Oh yes, this is the rough code, the PyEval_RestoreThread() call has now been removed. However, I don't consider the lack of calling Py_EndInterpreter() to be the cause of the error, as it is just a memory leak. But in the error we can see that there seems to be object confusion in cpython.

from cpython.

I don't consider the lack of calling Py_EndInterpreter() to be the cause of the error, as it is just a memory leak

The lack of Py_EndInterpreter() is not just a memory leak. It also swaps the active thread state. Your original code example calls PyThreadState_Clear() with the wrong active interpreter. That can lead to all sorts of problems, including the ones you've described above.

I don't think there's anything more we can do with this issue without additional details that are sufficient to reproduce a bug in CPython.

from cpython.

Your fix just throws an exception (C0000005, EXCEPTION_ACCESS_VIOLATION)! in progress. The code I left above should be enough to reproduce the problem.

EXPORT VOID RunScript(char* name, int console_id, int argc, char* argv[]) {
//	auto new_thread = PyThreadState_New(g_state->interp); //creating sub-main thread

	auto prev = PyThreadState_Swap(0); //swap

	PyInterpreterConfig conf = { 0 };
	conf.check_multi_interp_extensions = 1;
	conf.allow_threads = 1;
	conf.gil = PyInterpreterConfig_OWN_GIL;
	
	PyThreadState* tstate;
	Py_NewInterpreterFromConfig(&tstate, &conf); // creating new interpreter in new thread

	ExecuteScript(name, console_id, argc, argv); //PyRun_String-like func

	Py_EndInterpreter(tstate);
	PyThreadState_Swap(prev);
//	PyThreadState_Clear(new_thread);
//	PyThreadState_Delete(new_thread);
}

from cpython.

It's not enough to reproduce the issue because the code run by ExecuteScript is not shown.

Here is the code I've tested that runs without issues:

static PyObject *
issue117864(PyObject *module, PyObject *arg)
{
    PyThreadState *prev = PyThreadState_Swap(NULL);

    PyInterpreterConfig conf = { 0 };
    conf.check_multi_interp_extensions = 1;
    conf.allow_threads = 1;
    conf.gil = PyInterpreterConfig_OWN_GIL;

    PyThreadState* tstate;
    Py_NewInterpreterFromConfig(&tstate, &conf); // creating new interpreter in new thread

    PyObject *globals = PyDict_New();
    PyObject *locals = PyDict_New();

    PyObject *r = PyRun_String("print('Hello, World!')", Py_file_input, globals, locals);
    if (r == NULL) {
        PyErr_Print();
    }
    Py_XDECREF(r);

    Py_EndInterpreter(tstate);
    PyThreadState_Swap(prev);
    Py_RETURN_NONE;
}

from cpython.

My program calls this code several times, so that inside the first ExecuteScript call there is another ExecuteScript call (for example, the autorun.py file runs other scripts through this code, which, by the way, is part of a custom cpython module). The first call (including the nested one) succeeds, but the second does not.
Callstack:

from cpython.