On Tue, Nov 06, 2012 at 11:40:53AM -0600, Tony Asleson wrote:
On 11/06/2012 09:49 AM, Dan Kenigsberg wrote:
> On Mon, Oct 29, 2012 at 10:20:04AM -0500, Adam Litke wrote:
>> Hi everyone,
>>
>> libvdsm is listed as a release feature for 3.2 (preview only)[1][2]. There is
a
>> set of patches up in gerrit that could use a wide review from the community.
>> The plan is to merge the new json-rpc server[3] first so if you could
>> concentrate your reviews there it would yield the greatest benefit. Thanks!
>>
>> [1]
http://wiki.ovirt.org/wiki/OVirt_3.2_release-management
>> [2]
http://wiki.ovirt.org/wiki/Features/libvdsm
>> [3]
http://gerrit.ovirt.org/#/c/8614/
>
> [3] defines the format of each message as
>
> <size><json-data>
>
> where <size> is a binary value, used to split a (tcp) stream into
> messages. I would like to consider another splitting scheme, which I
> find better suited to the textual nature of jsonrpc: terminate each
> message with the newline character. It makes the protocol easier to
> sniff and debug (in case you've missed part of a message).
>
> The down size is that we would need to require clients to
> escape literal newlines, and unescape them in responses (both are done
> by python's json module, and the latter is part of the json standard).
I use json-rpc for IPC in libStoragemgmt (out of process plug-ins) with
unix domain sockets. I adopted the <size><json-data> model as well*.
I chose this because it allows the use of non-stream capable json
parsers. I wanted to ensure that the transport and protocol would be
language and parser agnostic.
You could achieve the message separation with new lines as you suggest,
but then you may have to parse the message stream twice. Once to find
the message delimiter and once again to parse the json itself, depending
on json parser. Having the size at the beginning of the message is
incredibly convenient from a coding efficiency standpoint.
As for debug, I just log the message payload if needed. I haven't had
the need to use a packet trace, but I'm not sure having a single newline
separating messages would be obvious in a single frame capture?
Would it be possible to compromise and leave the length and add the
newline as the end? So <size><payload><new line>? You could then
pass
the message payload to the parser with without having to escape the
newlines?
Thanks for weighing in on this! If you use the size and newline, how do you
account for the newline char in the <size> value? It seems unnecessary to
include this character to me since you can use a combination of logfile analysis
and scanning the data stream for 'id': to find method calls and responses.
--
Adam Litke <agl(a)us.ibm.com>
IBM Linux Technology Center