Repository navigation
gh-135893: Lazily initialize the libffi allocator in ctypes - #161844
ZeroIntensity wants to merge 4 commits into
Conversation
|
🤖 New build scheduled with the buildbot fleet by @ZeroIntensity for commit 878a8a5 🤖 Results will be shown at: https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F161844%2Fmerge If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again. |
|
The Android failure was caused by the buildbot running out of disk space, and is probably not relevant to this PR. |
| // See https://github.com/python/cpython/issues/128485 | ||
| // This allocates some memory and then frees it to ensure that libffi's | ||
| // dlmalloc allocator initializes itself exactly once, avoiding data races | ||
| // between threads that don't share a GIL. |
There was a problem hiding this comment.
I don't understand well how _PyOnceFlag_CallOnce() avoid reintroducing #128485. If two threads call _ctypes_alloc_callback(), the first one calls this closure_alloc_init() function and the other one calls Py_ffi_closure_alloc() which is a wrapper to ffi_closure_alloc(). At the end, the two threads call ``ffi_closure_alloc()` in parallel and we are back to #128485 issue, no?
There was a problem hiding this comment.
The whole point of a once flag is that multiple threads can't call it in parallel. It's called once, per the name. When _PyOnceFlag_CallOnce is called concurrently, the other threads will wait until the winner finishes initialization. Here's the scenario:
- Thread 1 enters
_ctypes_alloc_callback. - Thread 2 enters the same code path.
- Thread 1 calls
_PyOnceFlag_CallOnce. - The once flag is unlocked, so the thread calls
closure_alloc_initto initialize the libffi allocator. - Thread 2 calls
_PyOnceFlag_CallOnceat the same time, but it sees that it's in the "locked" state, so it spins until thread 1 marks the once flag as initialized. - Thread 2 does not call
_ctypes_alloc_callbackand proceeds; the allocator has been initialized by thread 1.
There was a problem hiding this comment.
Thread 2 calls _PyOnceFlag_CallOnce at the same time, but it sees that it's in the "locked" state, so it spins until thread 1 marks the once flag as initialized.
Oh, I missed that part of _PyOnceFlag_CallOnce(). I didn't know that other threads block until the function is initialized. In that case, ok, I understand that the change doesn't reintroduce the bug.
No. I had assumed that the fix had come with a test case, but apparently not. I think it would make sense to add a test for this rather than manually verifying. |
This prevents libffi from causing side effects when
ctypesis imported.