Race condition when waiting on Promise resolved in other thread #87
Description
I'm using with LibUsb to achieve asynchronous transfers. Without going into too much details I'm using two threads:
- Worker: creates promise (P) and starts async USB transfer with supplied callback (C) that resolves promise. After starting transfer, thread wait for promise result.
- LibUSB event loop: triggers event processing in LibUSB, which will call callback C which will resolve promise and resume Worker thread.
All calls to LibUSB are native calls (ctypes).
Occasionally (one in hundreds) I'm getting AssertionFailed exception on this line: https://github.com/syrusakbary/promise/blob/master/promise/promise.py#L414 with traceback:
Traceback (most recent call last):
File "C:\Python38\lib\threading.py", line 932, in _bootstrap_inner
self.run()
File "C:\Python38\lib\threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "<snip>", line 66, in _run
packet = self._segmentation.read_packet()
File "<snip>", line 60, in read_packet
payload = self._low.low_read(part_size)
File "<snip>.py", line 56, in low_read
return bytes(ret.get())
File "C:\Python38\lib\site-packages\promise\promise.py", line 511, in get
self._wait(timeout or DEFAULT_TIMEOUT)
File "C:\Python38\lib\site-packages\promise\promise.py", line 506, in _wait
self.wait(self, timeout)
File "C:\Python38\lib\site-packages\promise\promise.py", line 502, in wait
async_instance.wait(promise, timeout)
File "C:\Python38\lib\site-packages\promise\async_.py", line 117, in wait
target.scheduler.wait(target, timeout)
File "C:\Python38\lib\site-packages\promise\schedulers\immediate.py", line 24, in wait
promise._then(on_resolve_or_reject, on_resolve_or_reject)
File "C:\Python38\lib\site-packages\promise\promise.py", line 577, in _then
target._add_callbacks(did_fulfill, did_reject, promise)
File "C:\Python38\lib\site-packages\promise\promise.py", line 414, in _add_callbacks
assert not self._rejection_handler0"
AssertionError
Traceback indicates that assertion failed in context on Worker thread.
I was able to create (sort of...) reproduction of this issue: https://github.com/Novakov/promise/tree/race-condition Unfortunately it's not great. Running test https://github.com/Novakov/promise/blob/race-condition/tests/test_thread_safety.py#L122 shows the issue in most cases.
Investigation of Promise state suggests that in line https://github.com/syrusakbary/promise/blob/master/promise/promise.py#L577 promise is still pending, than context switch happens, promise gets resolved in loop thread, context switch happends again and _add_callbacks continues on now resolved promise.
Machine details
OS: Windows 10 x64
CPU: 4 cores, 8 logical threads
Python: 3.8.1 (tags/v3.8.1:1b293b6, Dec 18 2019, 23:11:46) [MSC v.1916 64 bit (AMD64)]
I was not able to reproduce the issue on Linux machine, however it maybe matter of probability