Richer data model: local out-of-band objects

[smuir]

There are various enhancements to the data model that could be useful in different types of Sirius app, that can probably be implemented without changing the underlying key-value abstraction for the consensus protocol.

One relatively straightforward one is to allow object(s) to be passed across the enqueue/handle interface without them being part of the consensus protocol. This is useful if the back-end requires additional data to efficiently process a (K, V) pair received by the handler, e.g., it might need a handle for the client that submitted a request to the app front-end.

This would save developers from having to stash such objects in a queue in the front-end and then retrieve them in the back-end by matching the key (or something similar). It also provides an alternative way to pass values back from back-end to front-end than returning a value via a Future. Finally, it allows the app front-end to be implemented as fully asynchronous ('fire and forget'), since it no longer needs to check the status of enqueued operations in order to respond to clients.

Obviously only the node that performed the enqueue operation would receive the out-of-band data, but presumably only that node needs the additional data to respond to the client.

[comcast-jonm]

So you are suggesting something like these additional methods?

public interface Sirius {
  ...
  Future<SiriusResult> <T> enqueuePut(String key, byte[] body, T localData);
}

interface RequestHandler {
  ...
  SiriusResult <T> handlePut(String key, byte[] body, T localData);
}

Possibly the <T> parameters would want to get moved up onto the interfaces so that we could statically check that the RequestHandler can handle the same types of localData that were being enqueued.

[smuir]

Yes, exactly like that. I like your suggestion about pushing up the types, as one of the motivations is to give stronger static type checking between enqueue and handle than just reducing everything to Object/AnyRef as in the SiriusResult type.

[smuir]

I hacked up a prototype of this interface that build on the Sirius1Dot2 stuff. Essentially it creates new interfaces for the RequestHandler and Sirius instance that are parametrized by two types T and U, and a factory method that is parametrized similarly. The prototypes for enqueuePut and handlePut are as follows (with other interface methods being analogous):

enqueuePut(key: String, body: Array[Byte], tag: T): Future[U]

handlePut(key: String, body: Array[Byte]: tag: Option[T]): U

These are intentionally as general as possible, allowing users to pass in, say, Option as the return type (U). The handler is passed SomeT when receiving a locally-enqueued Put, and None when receiving either a remotely-enqueued Put or during recovery from the WAL. Exceptions in the handler are passed back using the standard Java Future interface, i.e., by throwing an ExecutionException in handlePut, with the thrown exception as the cause value.

The prototype includes sufficient implementation glue to make this all work transparently with the existing interfaces, and satisfy all existing unit tests. It includes the Promise-based workaround I described in Issue #6 for return values not currently being propagated correctly, but I came to the conclusion that that is not a good long-term solution and we should just fix #6.