7:50 p.m.
On 2012-7-10 2:52, Saggi Mizrahi wrote:
----- Original Message -----
> From: "Itamar Heim" <iheim(a)redhat.com>
> To: "Saggi Mizrahi" <smizrahi(a)redhat.com>
> Cc: "Adam Litke" <agl(a)us.ibm.com>, vdsm-devel(a)lists.fedorahosted.org
> Sent: Monday, July 9, 2012 11:03:43 AM
> Subject: Re: [vdsm] [RFC] An alternative way to provide a supported interface --
libvdsm
>
> On 07/09/2012 05:56 PM, Saggi Mizrahi wrote:
>> I don't think AMQP is a good low level supported protocol as it's a
>> very complex protocol to set up and support.
>> Also brokers are known to have their differences in standard
>> implementation which means supporting them all is a mess.
>>
>> It looks like the most accepted route is the libvirt route of
>> having a c library abstracting away client server communication
>> and having more advanced consumers build protocol specific bridges
>> that may have different support standards.
>>
>> On a more personal note, I think brokerless messaging is the way to
>> go in ovirt because, unlike traditional clustering, worker nodes
>> are not interchangeable so direct communication is the way to go,
>> rendering brokers pretty much useless.
> but brokerless doesn't let multiple consumers which a bus provides?
All consumers can connect to the host and *some* events can be broadcasted to all
connected clients.
The real question is weather you want to depend on AMQP's routing \ message storing
Also, if you find it preferable to have a centralized host (single point of failure) to
get all events from all hosts for the price of some clients (I assume read only clients)
not needing to know the locations of all worker nodes.
But IMHO we already have something like that, it's called the ovirt-engine, and it
could send aggregated events about the cluster (maybe with some extra enginy data).
The question is what does mandating a broker gives us something that an "AMQP
bridge" wouldn't.
The only thing I can think of is vdsm can assume unmoderated vdsm to vdsm communication
bypassing the engine.
I don't fully understand the sentence above. I think
"AMQP bridge" can
also provide a way for unmoderated vdsm to vdsm communication bypassing
the engine. That is, one VDSM can post a message into the bridge and the
other interested VDSM can consume the message.
This means that VDSM can have some clustered behavior that requires
no engine intervention.
Further more, the engine can send a request and let the nodes decide who is performing
the operation among themselves.
Essentially:
[ engine ] [ engine ]
| | VS |
[vdsm][vdsm] [ broker ]
| |
[vdsm][vdsm]
*All links are two way links
Where is the broker sit in? In one of the host nodes with VDSM? or in
a specific node?
This has dire consequences on API usability and supportability. So we need to converge on
that.
There needs to be a good reason why the aforementioned logic code can't sit on a
another ovirt specific entity (lets call it ovirt-dynamo) that uses VDSM's supported
API
What aforementioned logic code you are talking about here?
but it's own APIs (or more likely messaging algorithms) are
unsupported.
[ engine ]
| | |
| [ broker ] |
| | | |
[vdsm]-[dynamo] : [dynamo]-[vdsm]
Host A : Host B
*All links are two way links
>> ----- Original Message -----
>>> From: "Adam Litke" <agl(a)us.ibm.com>
>>> To: "Itamar Heim" <iheim(a)redhat.com>
>>> Cc: vdsm-devel(a)lists.fedorahosted.org
>>> Sent: Monday, July 9, 2012 9:56:17 AM
>>> Subject: Re: [vdsm] [RFC] An alternative way to provide a
>>> supported interface -- libvdsm
>>>
>>> On Fri, Jul 06, 2012 at 03:53:08PM +0300, Itamar Heim wrote:
>>>> On 07/06/2012 01:15 AM, Robert Middleswarth wrote:
>>>>> On 07/05/2012 04:45 PM, Adam Litke wrote:
>>>>>> On Thu, Jul 05, 2012 at 03:47:42PM -0400, Saggi Mizrahi wrote:
>>>>>>> ----- Original Message -----
>>>>>>>> From: "Adam Litke" <agl(a)us.ibm.com>
>>>>>>>> To: "Saggi Mizrahi"
<smizrahi(a)redhat.com>
>>>>>>>> Cc: "Anthony Liguori"
<anthony(a)codemonkey.ws>, "VDSM Project
>>>>>>>> Development"
<vdsm-devel(a)lists.fedorahosted.org>
>>>>>>>> Sent: Thursday, July 5, 2012 2:34:50 PM
>>>>>>>> Subject: Re: [RFC] An alternative way to provide a
supported
>>>>>>>> interface -- libvdsm
>>>>>>>>
>>>>>>>> On Wed, Jun 27, 2012 at 02:50:02PM -0400, Saggi Mizrahi
>>>>>>>> wrote:
>>>>>>>>> The idea of having a supported C API was something I
was
>>>>>>>>> thinking
>>>>>>>>> about doing
>>>>>>>>> (But I'd rather use gobject introspection and not
schema
>>>>>>>>> generation) But the
>>>>>>>>> problem is not having a C API is using the current
XML RPC
>>>>>>>>> API
>>>>>>>>> as
>>>>>>>>> it's base
>>>>>>>> I want to disect this a bit to find out exactly where
there
>>>>>>>> might be
>>>>>>>> agreement
>>>>>>>> and disagreement.
>>>>>>>>
>>>>>>>> C API is a good thing to implement - Agreed.
>>>>>>>>
>>>>>>>> I also want to use gobject introspection but I don't
agree
>>>>>>>> that
>>>>>>>> using
>>>>>>>> glib
>>>>>>>> precludes the use of a formalized schema. My proposal
is
>>>>>>>> that
>>>>>>>> we
>>>>>>>> write a schema
>>>>>>>> definition and generate the glib C code from that
schema.
>>>>>>>>
>>>>>>>> I agree that the _current_ xmlrpc API makes a pretty bad
base
>>>>>>>> from
>>>>>>>> which to
>>>>>>>> start a supportable API. XMLRPC is a perfectly
reasonable
>>>>>>>> remote/wire protocol
>>>>>>>> and I think we should continue using it as a base for
the
>>>>>>>> next
>>>>>>>> generation API.
>>>>>>>> Using a schema will ensure that the new API is
>>>>>>>> well-structured.
>>>>>>> There major problems with XML-RPC (and to some extent with
>>>>>>> REST
>>>>>>> as
>>>>>>> well) are high call overhead and no two way communication
>>>>>>> (push
>>>>>>> events). Basing on XML-RPC means that we will never be able
to
>>>>>>> solve
>>>>>>> these issues.
>>>>>> I am not sure I am ready to conceed that XML-RPC is too slow
>>>>>> for
>>>>>> our
>>>>>> needs. Can
>>>>>> you provide some more detail around this point and possibly
>>>>>> suggest an
>>>>>> alternative that has even lower overhead without sacrificing
>>>>>> the
>>>>>> ubiquity and
>>>>>> usability of XML-RPC? As far as the two-way communication
>>>>>> point,
>>>>>> what
>>>>>> are the
>>>>>> options besides AMQP/ZeroMQ? Aren't these even worse from
an
>>>>>> overhead
>>>>>> perspective than XML-RPC? Regarding two-way communication: you
>>>>>> can
>>>>>> write AMQP
>>>>>> brokers based on the C API and run one on each vdsm host.
>>>>>> Assuming
>>>>>> the C API
>>>>>> supports events, what else would you need?
>>>>> I personally think that using something like AMQP for inter-node
>>>>> communication and engine - node would be optimal. With a rest
>>>>> interface
>>>>> that just send messages though something like AMQP.
>>>> I would also not dismiss AMQP so soon
>>>> we want a bug with more than a single listener at engine side
>>>> (engine, history db, maybe event correlation service).
>>>> collectd as a means for statistics already supports it as well.
>>>> I'm for having REST as well, but not sure as main one for a
>>>> consumer
>>>> like ovirt engine.
>>> I agree that a message bus could be a very useful model of
>>> communication between
>>> ovirt-engine components and multiple vdsm instances. But the
>>> complexities and
>>> dependencies of AMQP do not make it suitable for use as a
>>> low-level
>>> API. AMQP
>>> will repel new adopters. Why not establish a libvdsm that is more
>>> minimalist
>>> and can be easily used by everyone? Then AMQP brokers can be
>>> built
>>> on top of
>>> the stable API with ease. All AMQP should require of the
>>> low-level
>>> API are
>>> standard function calls and an events mechanism.
>>>
>>>>> Thanks
>>>>> Robert
>>>>>>>>> The current XML-RPC API contains a lot of decencies
and
>>>>>>>>> inefficiencies and we
>>>>>>>>> would like to retire it as soon as we possibly can.
Engine
>>>>>>>>> would
>>>>>>>>> like us to
>>>>>>>>> move to a message based API and 3rd parties want
something
>>>>>>>>> simple
>>>>>>>>> like REST so
>>>>>>>>> it looks like no one actually wants to use XML-RPC.
Not even
>>>>>>>>> us.
>>>>>>>> I am proposing that AMQP brokers and REST APIs could be
>>>>>>>> written
>>>>>>>> against the
>>>>>>>> public API. In fact, they need not even live in the
vdsm
>>>>>>>> tree
>>>>>>>> anymore if that
>>>>>>>> is what we choose. Core vdsm would only be responsible
for
>>>>>>>> providing
>>>>>>>> libvdsm
>>>>>>>> and whatever language bindings we want to support.
>>>>>>> If we take the libvdsm route, the only reason to even have a
>>>>>>> REST
>>>>>>> bridge is only to support OSes other then Linux which is
>>>>>>> something
>>>>>>> I'm not sure we care about at the moment.
>>>>>> That might be true regarding the current in-tree
>>>>>> implementation.
>>>>>> However, I can
>>>>>> almost guarantee that someone wanting to write a web GUI on top
>>>>>> of
>>>>>> standalone
>>>>>> vdsm would want a REST API to talk to. But libvdsm makes this
>>>>>> use
>>>>>> case of no
>>>>>> concern to the core vdsm developers.
>>>>>>
>>>>>>>>> I do think that having C supportability in our API is
a good
>>>>>>>>> idea,
>>>>>>>>> but the
>>>>>>>>> current API should not be used as the base.
>>>>>>>> Let's _start_ with a schema document that describes
today's
>>>>>>>> API
>>>>>>>> and
>>>>>>>> then clean
>>>>>>>> it up. I think that will work better than starting from
>>>>>>>> scratch.
>>>>>>>> Once my
>>>>>>>> schema is written I will post it and we can
'patch' it as a
>>>>>>>> community
>>>>>>>> until we
>>>>>>>> arrive at a 1.0 version we are all happy with.
>>>>>>> +1
>>>>>> Ok. Redoubling my efforts to get this done. Describing the
>>>>>> output of
>>>>>> list(True) takes awhile :)
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> vdsm-devel mailing list
>>>>> vdsm-devel(a)lists.fedorahosted.org
>>>>> https://fedorahosted.org/mailman/listinfo/vdsm-devel
>>>>
>>>> _______________________________________________
>>>> vdsm-devel mailing list
>>>> vdsm-devel(a)lists.fedorahosted.org
>>>> https://fedorahosted.org/mailman/listinfo/vdsm-devel
>>> --
>>> Adam Litke <agl(a)us.ibm.com>
>>> IBM Linux Technology Center
>>>
>>> _______________________________________________
>>> vdsm-devel mailing list
>>> vdsm-devel(a)lists.fedorahosted.org
>>> https://fedorahosted.org/mailman/listinfo/vdsm-devel
>>>
>
>
_______________________________________________
vdsm-devel mailing list
vdsm-devel(a)lists.fedorahosted.org
https://fedorahosted.org/mailman/listinfo/vdsm-devel
--
Shu Ming <shuming(a)linux.vnet.ibm.com>
IBM China Systems and Technology Laboratory