diff --git a/mrpc/proto.md b/mrpc/proto.md new file mode 100644 index 0000000..cae69ec --- /dev/null +++ b/mrpc/proto.md @@ -0,0 +1,208 @@ +# Protocol + +The mediocre-rpc protocol is designed to operate over nearly any network +protocol. It exists entirely in the data layer, and only relies on being carried +by a protocol which supports a request/response paradaigm, and garauntees +order/reception. This includes HTTP and raw TCP sockets, and likely many others. + +RPC calls and responses are JSON encoded objects. The protocol supports single +object calls/responses, as well as streaming multiple objects for either. There +is also support for streaming raw byte blobs. + +## General + +A couple common rules which apply across all subsequent documentation for this +spec: + +* In all JSON object specs, a field which is not required can be omitted + entirely, and its value is assumed to be the expected type's empty value (e.g. + `""` for strings, `0` for numbers, `{}` for objects, `null` for any-JSON + types). + +* For multiple JSON objects appearing back-to-back on the wire there may or may + not be white-space separating them. + +## Calls + +### Single call + +A single call looks like the following: + + { + "method":"methodName (required)", + "args":"anyJSON", + "debug":{ "foo":"anyJSON" } + } + +`method` is required and indicates the name of the method being called. Its +value has no restrictions on the protocol level, it's up to the caller and +handler to know ahead of time which methods are available. + +`args` are the arguments to the method, and can be anything. + +`debug` is metadata about the call which can be made accessible to the handler +for purposes of tracing, logging, etc... `debug` must be a JSON object. A good +rule to know if something should be debug or part of the arguments is that if +any `debug` field were to be deleted on any request the response wouldn't +change. If the response were to change then that field should be in `args`. + +### Stream call + +A stream call consists of a leading JSON object which looks like a single call, +a body of zero or more JSON objects containing single elements of a stream, and +a tail which indicates the end of the stream. + +A stream method call whose body is all JSON strings might look this (newlines +added for clarity, they are optional in the protocol): + + { + "method":"methodName (required)", + "args":"anyJSON", + "debug":{ "foo":"anyJSON" }, + "streamStart":true, + "streamLen":3 + } + + { "el":"anyJSON" } + + { "elBytesFrame":"RANDFRAME" } + RANDFRAMEsome very cool arbitrary bytes + which may contain whitespace or anythingRANDFRAME + + { + "elBytesFrame":"RANDFRAME", + "elBytesLen":10 + } + RANDFRAMEsome-bytesRANDFRAME + + { + "args":"anyJSON", + "debug":{ "foo":"anyJSON" }, + "streamEnd":true + } + +The head is the first JSON value in the stream. It looks and operates much like a +single call's head, but with the added `streamStart` field. The `streamLen` +field is optional, but may be used if the number of elements in the stream is +known beforehand. + +Each element in the stream is a JSON object, and can be either a single JSON +value or a blob of arbitrary bytes. + +If the element is a JSON value the JSON object will have an `el` field whose +value is the JSON value. + +If the element is a blob of arbitrary bytes the JSON object will have an +`elBytesFrame` field whose value is a set of random alphanumeric characters +which will be used to prefix and suffix the bytes. Following the JSON object may +be some whitespace, and then the arbitrary bytes with the `elBytesFrame` +prefixed and suffixed immediately around it. The JSON object for the element may +optionally have the `elBytesLen` field if the length of the blob is known +beforehand. The arbitrary bytes _must_ be prefixed/suffixed by the frame even if +`elBytesLen` is given. + +TODO elBytesFrame size recommendation + +The tail is the last JSON value in the stream and indicates the stream +has ended. It is required even if `streamLen` was given in the head. The tail +can also have its own `args` and `debug`, independent of the head's but subject +to the same rules. + +## Response + +### Single response + +A single response looks like the following: + + { + "res":"anyJSON", + "err":{ + "msg":"some presumably helpful string", + "meta":"anyJSON" + }, + "debug":{ "foo":"anyJSON" } + } + +`res` is the response from the call, and can be anything. + +`err` is mutually exclusive with `res` (if one is set the other should be `null` +or unset). The `msg` is be any arbitrary string. `meta` is optional and contains +any extra information which might be actionable by the client receiving the +error. + +`debug` is metadata about the response which can be made accessible to the +caller for purposes of tracing, logging, etc... `debug` must be a JSON object. A +good rule to know if something should be debug or part of the results is that +if any `debug` field were to be deleted on any response the client would act in +the same way. If the client's subsequent actions were to change then that field +should be in `res`. + +### Stream response + +A stream response looks and acts very similar to a stream call, and the +documentation for the stream call can be referenced for details on the +following: + + { + "res":"anyJSON", + "debug":{ "foo":"anyJSON" }, + "streamStart":true, + "streamLen":3 + } + + { "el":"anyJSON" } + + { "elBytesFrame":"RANDFRAME" } + RANDFRAMEsome very cool arbitrary bytes + which may contain whitespace or anythingRANDFRAME + + { + "elBytesFrame":"RANDFRAME", + "elBytesLen":10 + } + RANDFRAMEsome-bytesRANDFRAME + + { + "args":"anyJSON", + "debug":{ "foo":"anyJSON" }, + "streamEnd":true + } + +The one note specific to stream responses is that a stream response _cannot_ +contain an `err` field. + +## Network details + +In the case of a stream call the handler may send back a response before the +stream has been completely sent. In this case the client should ignore the fact +that the stream wasn't completely sent and return the response as it receives +it. + +In the case of a stream call and a stream response both the client and handler +can be sending their respective streams to the other simultaneously. As in the +previous case, if the handler ends the stream with the tail object the caller +should treat the call as successfully completed. + +The general rule is: If the client receives either a single response or the tail +of a stream response then the call should be treated as completed. + +## TODO + +* In the case of pipelining then short-circuiting ought to be implemented for + the case of the stream response having been completed but the stream call + hasn't. In that case the stream call should short-circuit the stream so the + connection can be reused asap + +* Figure out the naming + +* Len is weird, cause if a stream is short-circuited then the length will have + just been a hint + +* Maybe just make bytes a thing which can be bolted onto any of the JSON + objects, either the single, head, elements, or tail. + +* Maybe instead of merely defining a single-level stream, do something where all + elements in the stream are the same (either a json value or bytes), and + additionally each can declare that it is the beginning of a stream of further + objects. That gets weird with both `method` and `err`, but it's kinda nice in + that it would potentially simplify the code interface greatly.