We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent 8c3bc3b commit 15c50f7Copy full SHA for 15c50f7
vllm/v1/engine/core.py
@@ -1044,7 +1044,10 @@ def process_input_sockets(
1044
request = generic_decoder.decode(data_frames)
1045
1046
if request_type == EngineCoreRequestType.ABORT:
1047
- # Aborts are added to *both* queues.
+ # Aborts are added to *both* queues, allows us to eagerly
1048
+ # process aborts while also ensuring ordering in the input
1049
+ # queue to avoid leaking requests. This is ok because
1050
+ # aborting in the scheduler is idempotent.
1051
self.aborts_queue.put_nowait(request)
1052
1053
# Push to input queue for core busy loop.
0 commit comments