Redefine command-buffer simultaneous-use

[EwanC]

As discussed in #891 the current definition of simultaneous use is hard for users to reason about.

Instead a better model is one where simultaneous-use isn't a constraint on when a command-buffer can be submitted for execution, but the execution itself. That is, simultaneous-use occurs when a command-buffer is enqueued for execution while any previous submission of the command-buffer is still in-flight, and there any no scheduling dependencies expressed to serialize execution on the in-flight submission.

This means that pipelined submissions of a command-buffer, where there is an in-order queue, barrier, or cl_events to serialize execution, is always valid usage without the optional simultaneous-use device feature.

To avoid the runtime having to incur overheads from monitoring when simultaneous-use occurs so it can throw an error, violating this valid usage for non simultaneous-use command-buffers is UB. Following from this and relation to #1311, the pending state has also been removed from the command-buffer lifecycle, and there is only left the binary states of recording and executable (note that this conflicts with the change proposed in #1382, but it I believe it is straightforward to resolve). This is because a user can use the existing OpenCL mechanisms of host waits and event queries to inspect the state of individual command-buffer enqueues to avoid simultaneous-use, and having this stored as a command-buffer state incurs the runtime overhead of tracking a previous submissions.

The implications for updating a command-buffer is that the error behavior is removed from update, so simultaneous-use is a property of execution rather than enqueue. In fact update is the only scenario where simultaneous use now adds value, as without it there will be race conditions during execution. But with command-buffer update the command-buffer operations can work on different data. As a result i've moved the simultaneous capability from the base extension to mutable-dispatch.

I've outlined the CTS changes that would be required for this change in KhronosGroup/OpenCL-CTS#2473 and drafted them in KhronosGroup/OpenCL-CTS#2477

[EwanC]

@bashbaug I've requested you re-review the PR, as since you approved it I've pushed a commit that moves the simultaneous-use capability from the base extension to cl_khr_command_buffer_mutable_dispatch. While updating the CTS tests I realized I couldn't think of a scenario without update where it adds value, as any concurrent execution of the command-buffer would probably be race condition. Whereas with mutable-dispatch you can update your kernel to new inputs/outputs to use that would allow two concurrent executions to operate on different data. We haven't discussed this in the group yet, so I can revert it if we think it's undesirable.

[bashbaug]

I've pushed a commit that moves the simultaneous-use capability from the base extension to cl_khr_command_buffer_mutable_dispatch. While updating the CTS tests I realized I couldn't think of a scenario without update where it adds value, as any concurrent execution of the command-buffer would probably be race condition.

Interesting. My first reaction was that this goes a bit too far, but after thinking about it a bit more, I'm starting to agree. Without mutable dispatch, every execution of the command buffer is necessarily reading from and writing to the same locations in memory, which is why "any concurrent execution of the command-buffer would probably be race condition".

Note that Vulkan has the command buffer simultaneous usage flag VK_COMMAND_BUFFER_USAGE_SIMULTANEOUS_USE_BIT, but Vulkan also allows modifying the command buffer via vkCmdBindDescriptorSets and vkCmdPushConstants to avoid race conditions.

Let's discuss a bit more in our call on Tuesday before merging this change, just to be sure.

CC @bcalidas @kpet @paulfradgley

[bashbaug]

One more thought: "simultaneous use" is going to apply to mutable dispatch, plus mutable memory commands (#1065), plus potentially any other future extensions that allow modifying a command buffer (e.g. #1279). So, practically speaking, we still may want to keep the "simultaneous use" concept and enums in the base-level extension and have the layered extensions refer to them vs. replicating everything into each of the layered extensions.

Even if we do this, I think it's worth an informative note cautioning against concurrent execution of a command buffer without modification due to the possibility of data races, since this wasn't immediately obvious to me.

[bashbaug]

Discussed in the August 12th teleconference:

• We are leaning towards moving the simultaneous use bit to the mutable dispatch extension as suggested. Note that the other extensions that enable modifying the command buffer layer on top of the mutable dispatch extension so they can reuse the same simultaneous use bit.
• We are going to explore adding a capability to indicate that the command buffer many be modified while it is in use, separate from the capability to indicate that the command buffer may be re-executed while it is in use (after it has been updated).
• We added "focused review" to the related CTS tests changes for simultaneous use so they will be ready we decide to merge the spec changes, see CTS PR #2477.

[bashbaug]

Merging as discussed in the August 26th teleconference.