[RFC] An alternative way to provide a supported interface -- libvdsm

[Anthony Liguori]

Hi,

I've been reading through the API threads here and considering the options.  To 
be honest, I worry a lot about the scope of these discussions and that there's a 
tremendous amount of work before we have a useful end result.

I wonder if we can solve this problem by adding another layer of abstraction...

As Adam is currently building a schema for VDSM's XML-RPC, we could use the QAPI 
code generators to build a libvdsm that provided a programmatic C interface for 
the XML-RPC interface.

It would take some tweaking, but this could be made a supportable C interface. 
The rules for having a supportable C interface are basically:

1) Never change function signatures

2) Never remove functions

3) Always allocate structures in the library and/or pad

4) Only add to structures, never remove or reorder

5) Provide flags that default to zero to indicate that fields/features are not 
present.

6) Always zero-initialize structures

Having a libvdsm would allow the transport to change over time w/o affecting 
end-users.  There are lots of good tools for documenting C APIs and dealing with 
versioning of C APIs.

While we can start out with a schema-generated API, over time, we can implement 
libvdsm in an open-coded fashion allowing old APIs to be reimplemented in terms 
of new APIs.

 From a compatibility perspective, libvdsm would be fully backwards compatible 
with old versions of VDSM (so it would keep XML-RPC support forever) but may 
require new versions of libvdsm to talk to new versions of VDSM.  That would 
allow for APIs to be deprecated within VDSM without breaking old clients.

I think this would be an incremental approach to building a supportable API 
today while still giving the flexibility to make changes in the long term.

And it should be fairly easy to generate a JNI binding and also port 
ovirt-engine to use an interface like this (since it already uses the XML-RPC API).

Regards,

Anthony Liguori

The idea of having a supported C API was something I was thinking about doing (But I'd
rather use gobject introspection and not schema generation)
But the problem is not having a C API is using the current XML RPC API as it's base

The current XML-RPC API contains a lot of decencies and inefficiencies and we would like
to retire it as soon as we possibly can. Engine would like us to move to a message based
API and 3rd parties want something simple like REST so it looks like no one actually wants
to use XML-RPC. Not even us.

I do think that having C supportability in our API is a good idea, but the current API
should not be used as the base.

----- Original Message -----
>  From: "Anthony Liguori" <anthony(a)codemonkey.ws&gt;
>  To: "VDSM Project Development" <vdsm-devel(a)lists.fedorahosted.org&gt;
>  Cc: "Adam Litke" <agl(a)us.ibm.com&gt;, "Saggi Mizrahi"
> <smizrahi(a)redhat.com&gt;
>  Sent: Monday, June 25, 2012 10:18:33 AM
>  Subject: [RFC] An alternative way to provide a supported interface -- libvdsm
>  
>  Hi,
>  
>  I've been reading through the API threads here and considering the
>  options.  To
>  be honest, I worry a lot about the scope of these discussions and
>  that there's a
>  tremendous amount of work before we have a useful end result.
>  
>  I wonder if we can solve this problem by adding another layer of
>  abstraction...
>  
>  As Adam is currently building a schema for VDSM's XML-RPC, we could
>  use the QAPI
>  code generators to build a libvdsm that provided a programmatic C
>  interface for
>  the XML-RPC interface.
>  
>  It would take some tweaking, but this could be made a supportable C
>  interface.
>  The rules for having a supportable C interface are basically:
>  
>  1) Never change function signatures
>  
>  2) Never remove functions
>  
>  3) Always allocate structures in the library and/or pad
>  
>  4) Only add to structures, never remove or reorder
>  
>  5) Provide flags that default to zero to indicate that
>  fields/features are not
>  present.
>  
>  6) Always zero-initialize structures
>  
>  Having a libvdsm would allow the transport to change over time w/o
>  affecting
>  end-users.  There are lots of good tools for documenting C APIs and
>  dealing with
>  versioning of C APIs.
>  
>  While we can start out with a schema-generated API, over time, we can
>  implement
>  libvdsm in an open-coded fashion allowing old APIs to be
>  reimplemented in terms
>  of new APIs.
>  
>   From a compatibility perspective, libvdsm would be fully backwards
>   compatible
>  with old versions of VDSM (so it would keep XML-RPC support forever)
>  but may
>  require new versions of libvdsm to talk to new versions of VDSM.
>   That would
>  allow for APIs to be deprecated within VDSM without breaking old
>  clients.
>  
>  I think this would be an incremental approach to building a
>  supportable API
>  today while still giving the flexibility to make changes in the long
>  term.
>  
>  And it should be fairly easy to generate a JNI binding and also port
>  ovirt-engine to use an interface like this (since it already uses the
>  XML-RPC API).
>  
>  Regards,
>  
>  Anthony Liguori
>

----- Original Message -----
>  The idea of having a supported C API was something I was thinking
>  about doing (But I'd rather use gobject introspection and not schema
>  generation)
>  But the problem is not having a C API is using the current XML RPC
>  API as it's base
>  
>  The current XML-RPC API contains a lot of decencies and
>  inefficiencies and we would like to retire it as soon as we possibly
>  can. Engine would like us to move to a message based API and 3rd
>  parties want something simple like REST so it looks like no one
>  actually wants to use XML-RPC. Not even us.
>  
>  I do think that having C supportability in our API is a good idea,
>  but the current API should not be used as the base. 
The current API is irrelevant, we should focus on the API we will deliver for Fedora 18 /
RHEL 7.
I also really don't see what xmlrpc has to do with it.  We specifically separated the
transport from the API and the c bindings should have nothing to do with xmlrpc.

>  
>  ----- Original Message -----
>  > From: "Anthony Liguori" <anthony(a)codemonkey.ws&gt;
>  > To: "VDSM Project Development" <vdsm-devel(a)lists.fedorahosted.org&gt;
>  > Cc: "Adam Litke" <agl(a)us.ibm.com&gt;, "Saggi Mizrahi"
>  > <smizrahi(a)redhat.com&gt;
>  > Sent: Monday, June 25, 2012 10:18:33 AM
>  > Subject: [RFC] An alternative way to provide a supported interface
>  > -- libvdsm
>  > 
>  > Hi,
>  > 
>  > I've been reading through the API threads here and considering the
>  > options.  To
>  > be honest, I worry a lot about the scope of these discussions and
>  > that there's a
>  > tremendous amount of work before we have a useful end result.
>  > 
>  > I wonder if we can solve this problem by adding another layer of
>  > abstraction...
>  > 
>  > As Adam is currently building a schema for VDSM's XML-RPC, we could
>  > use the QAPI
>  > code generators to build a libvdsm that provided a programmatic C
>  > interface for
>  > the XML-RPC interface.
>  > 
>  > It would take some tweaking, but this could be made a supportable C
>  > interface.
>  > The rules for having a supportable C interface are basically:
>  > 
>  > 1) Never change function signatures
>  > 
>  > 2) Never remove functions
>  > 
>  > 3) Always allocate structures in the library and/or pad
>  > 
>  > 4) Only add to structures, never remove or reorder
>  > 
>  > 5) Provide flags that default to zero to indicate that
>  > fields/features are not
>  > present.
>  > 
>  > 6) Always zero-initialize structures
>  > 
>  > Having a libvdsm would allow the transport to change over time w/o
>  > affecting
>  > end-users.  There are lots of good tools for documenting C APIs and
>  > dealing with
>  > versioning of C APIs.
>  > 
>  > While we can start out with a schema-generated API, over time, we
>  > can
>  > implement
>  > libvdsm in an open-coded fashion allowing old APIs to be
>  > reimplemented in terms
>  > of new APIs.
>  > 
>  >  From a compatibility perspective, libvdsm would be fully backwards
>  >  compatible
>  > with old versions of VDSM (so it would keep XML-RPC support
>  > forever)
>  > but may
>  > require new versions of libvdsm to talk to new versions of VDSM.
>  >  That would
>  > allow for APIs to be deprecated within VDSM without breaking old
>  > clients.
>  > 
>  > I think this would be an incremental approach to building a
>  > supportable API
>  > today while still giving the flexibility to make changes in the
>  > long
>  > term.
>  > 
>  > And it should be fairly easy to generate a JNI binding and also
>  > port
>  > ovirt-engine to use an interface like this (since it already uses
>  > the
>  > XML-RPC API).
>  > 
>  > Regards,
>  > 
>  > Anthony Liguori
>  > 
>  _______________________________________________
>  vdsm-devel mailing list
>  vdsm-devel(a)lists.fedorahosted.org
>  https://fedorahosted.org/mailman/listinfo/vdsm-devel
>

[Anthony Liguori]

On 06/27/2012 05:13 PM, Ayal Baron wrote:
>  ----- Original Message -----
> > The idea of having a supported C API was something I was thinking
> > about doing (But I'd rather use gobject introspection and not schema
> > generation)
> > But the problem is not having a C API is using the current XML RPC
> > API as it's base
> >
> > The current XML-RPC API contains a lot of decencies and
> > inefficiencies and we would like to retire it as soon as we possibly
> > can. Engine would like us to move to a message based API and 3rd
> > parties want something simple like REST so it looks like no one
> > actually wants to use XML-RPC. Not even us.
> >
> > I do think that having C supportability in our API is a good idea,
> > but the current API should not be used as the base.
>
>  The current API is irrelevant, we should focus on the API we will deliver for Fedora 18 /
> RHEL 7.
>  I also really don't see what xmlrpc has to do with it.  We specifically separated the
> transport from the API and the c bindings should have nothing to do with xmlrpc. 
And I'm suggesting is to make the C bindings the supported interface and make 
the wire protocol an implementation detail.

You could then implement the C bindings (using GObject or whatever) via the 
XML-RPC transport if you liked.

It also presents a better integration story as you could more easily expose the 
VDSM API as part of a large CIM/AMQP model data model if you like.

Regards,

Anthony Liguori

> >
> > ----- Original Message -----
> >> From: "Anthony Liguori"<anthony(a)codemonkey.ws>
> >> To: "VDSM Project
> Development"<vdsm-devel(a)lists.fedorahosted.org>
> >> Cc: "Adam Litke"<agl(a)us.ibm.com>, "Saggi Mizrahi"
> >> <smizrahi(a)redhat.com&gt;
> >> Sent: Monday, June 25, 2012 10:18:33 AM
> >> Subject: [RFC] An alternative way to provide a supported interface
> >> -- libvdsm
> >>
> >> Hi,
> >>
> >> I've been reading through the API threads here and considering the
> >> options.  To
> >> be honest, I worry a lot about the scope of these discussions and
> >> that there's a
> >> tremendous amount of work before we have a useful end result.
> >>
> >> I wonder if we can solve this problem by adding another layer of
> >> abstraction...
> >>
> >> As Adam is currently building a schema for VDSM's XML-RPC, we could
> >> use the QAPI
> >> code generators to build a libvdsm that provided a programmatic C
> >> interface for
> >> the XML-RPC interface.
> >>
> >> It would take some tweaking, but this could be made a supportable C
> >> interface.
> >> The rules for having a supportable C interface are basically:
> >>
> >> 1) Never change function signatures
> >>
> >> 2) Never remove functions
> >>
> >> 3) Always allocate structures in the library and/or pad
> >>
> >> 4) Only add to structures, never remove or reorder
> >>
> >> 5) Provide flags that default to zero to indicate that
> >> fields/features are not
> >> present.
> >>
> >> 6) Always zero-initialize structures
> >>
> >> Having a libvdsm would allow the transport to change over time w/o
> >> affecting
> >> end-users.  There are lots of good tools for documenting C APIs and
> >> dealing with
> >> versioning of C APIs.
> >>
> >> While we can start out with a schema-generated API, over time, we
> >> can
> >> implement
> >> libvdsm in an open-coded fashion allowing old APIs to be
> >> reimplemented in terms
> >> of new APIs.
> >>
> >>   From a compatibility perspective, libvdsm would be fully backwards
> >>   compatible
> >> with old versions of VDSM (so it would keep XML-RPC support
> >> forever)
> >> but may
> >> require new versions of libvdsm to talk to new versions of VDSM.
> >>   That would
> >> allow for APIs to be deprecated within VDSM without breaking old
> >> clients.
> >>
> >> I think this would be an incremental approach to building a
> >> supportable API
> >> today while still giving the flexibility to make changes in the
> >> long
> >> term.
> >>
> >> And it should be fairly easy to generate a JNI binding and also
> >> port
> >> ovirt-engine to use an interface like this (since it already uses
> >> the
> >> XML-RPC API).
> >>
> >> Regards,
> >>
> >> Anthony Liguori
> >>
> > _______________________________________________
> > vdsm-devel mailing list
> > vdsm-devel(a)lists.fedorahosted.org
> > https://fedorahosted.org/mailman/listinfo/vdsm-devel
> >

----- Original Message -----
>  On 06/27/2012 05:13 PM, Ayal Baron wrote:
>  >
>  >
>  > ----- Original Message -----
>  >> The idea of having a supported C API was something I was thinking
>  >> about doing (But I'd rather use gobject introspection and not
>  >> schema
>  >> generation)
>  >> But the problem is not having a C API is using the current XML RPC
>  >> API as it's base
>  >>
>  >> The current XML-RPC API contains a lot of decencies and
>  >> inefficiencies and we would like to retire it as soon as we
>  >> possibly
>  >> can. Engine would like us to move to a message based API and 3rd
>  >> parties want something simple like REST so it looks like no one
>  >> actually wants to use XML-RPC. Not even us.
>  >>
>  >> I do think that having C supportability in our API is a good idea,
>  >> but the current API should not be used as the base.
>  >
>  > The current API is irrelevant, we should focus on the API we will
>  > deliver for Fedora 18 / RHEL 7.
>  > I also really don't see what xmlrpc has to do with it.  We
>  > specifically separated the transport from the API and the c
>  > bindings should have nothing to do with xmlrpc.
>  
>  And I'm suggesting is to make the C bindings the supported interface
>  and make
>  the wire protocol an implementation detail.
>  
>  You could then implement the C bindings (using GObject or whatever)
>  via the
>  XML-RPC transport if you liked.
>  
>  It also presents a better integration story as you could more easily
>  expose the
>  VDSM API as part of a large CIM/AMQP model data model if you like. 
Sounds good then.

Your previous comment:
>  >>>   From a compatibility perspective, libvdsm would be
> fully
>  >>>   backwards
>  >>>   compatible
>  >>> with old versions of VDSM (so it would keep XML-RPC support
>  >>> forever) 
Threw me off there as we would not want to keep on supporting XML-RPC.
What's more, the current API has way too many things that have to go so needing to
keep those would just be sad and painful.

> 
> Regards,
> 
> Anthony Liguori
> 
> >
> >>
> >> ----- Original Message -----
> >>> From: "Anthony Liguori"<anthony(a)codemonkey.ws>
> >>> To: "VDSM Project
Development"<vdsm-devel(a)lists.fedorahosted.org>
> >>> Cc: "Adam Litke"<agl(a)us.ibm.com>, "Saggi
Mizrahi"
> >>> <smizrahi(a)redhat.com&gt;
> >>> Sent: Monday, June 25, 2012 10:18:33 AM
> >>> Subject: [RFC] An alternative way to provide a supported
> >>> interface
> >>> -- libvdsm
> >>>
> >>> Hi,
> >>>
> >>> I've been reading through the API threads here and considering
> >>> the
> >>> options.  To
> >>> be honest, I worry a lot about the scope of these discussions and
> >>> that there's a
> >>> tremendous amount of work before we have a useful end result.
> >>>
> >>> I wonder if we can solve this problem by adding another layer of
> >>> abstraction...
> >>>
> >>> As Adam is currently building a schema for VDSM's XML-RPC, we
> >>> could
> >>> use the QAPI
> >>> code generators to build a libvdsm that provided a programmatic C
> >>> interface for
> >>> the XML-RPC interface.
> >>>
> >>> It would take some tweaking, but this could be made a supportable
> >>> C
> >>> interface.
> >>> The rules for having a supportable C interface are basically:
> >>>
> >>> 1) Never change function signatures
> >>>
> >>> 2) Never remove functions
> >>>
> >>> 3) Always allocate structures in the library and/or pad
> >>>
> >>> 4) Only add to structures, never remove or reorder
> >>>
> >>> 5) Provide flags that default to zero to indicate that
> >>> fields/features are not
> >>> present.
> >>>
> >>> 6) Always zero-initialize structures
> >>>
> >>> Having a libvdsm would allow the transport to change over time
> >>> w/o
> >>> affecting
> >>> end-users.  There are lots of good tools for documenting C APIs
> >>> and
> >>> dealing with
> >>> versioning of C APIs.
> >>>
> >>> While we can start out with a schema-generated API, over time, we
> >>> can
> >>> implement
> >>> libvdsm in an open-coded fashion allowing old APIs to be
> >>> reimplemented in terms
> >>> of new APIs.
> >>>
>  >>>   From a compatibility perspective, libvdsm would be
> fully
>  >>>   backwards
>  >>>   compatible
>  >>> with old versions of VDSM (so it would keep XML-RPC support
>  >>> forever) 
> >>> but may
> >>> require new versions of libvdsm to talk to new versions of VDSM.
> >>>   That would
> >>> allow for APIs to be deprecated within VDSM without breaking old
> >>> clients.
> >>>
> >>> I think this would be an incremental approach to building a
> >>> supportable API
> >>> today while still giving the flexibility to make changes in the
> >>> long
> >>> term.
> >>>
> >>> And it should be fairly easy to generate a JNI binding and also
> >>> port
> >>> ovirt-engine to use an interface like this (since it already uses
> >>> the
> >>> XML-RPC API).
> >>>
> >>> Regards,
> >>>
> >>> Anthony Liguori
> >>>
> >> _______________________________________________
> >> vdsm-devel mailing list
> >> vdsm-devel(a)lists.fedorahosted.org
> >> https://fedorahosted.org/mailman/listinfo/vdsm-devel
> >>
> 
>

[Anthony Liguori]

On 06/27/2012 05:38 PM, Ayal Baron wrote:
>  ----- Original Message -----
> > On 06/27/2012 05:13 PM, Ayal Baron wrote:
> >>
> >>
> >> ----- Original Message -----
> >>> The idea of having a supported C API was something I was thinking
> >>> about doing (But I'd rather use gobject introspection and not
> >>> schema
> >>> generation)
> >>> But the problem is not having a C API is using the current XML RPC
> >>> API as it's base
> >>>
> >>> The current XML-RPC API contains a lot of decencies and
> >>> inefficiencies and we would like to retire it as soon as we
> >>> possibly
> >>> can. Engine would like us to move to a message based API and 3rd
> >>> parties want something simple like REST so it looks like no one
> >>> actually wants to use XML-RPC. Not even us.
> >>>
> >>> I do think that having C supportability in our API is a good idea,
> >>> but the current API should not be used as the base.
> >>
> >> The current API is irrelevant, we should focus on the API we will
> >> deliver for Fedora 18 / RHEL 7.
> >> I also really don't see what xmlrpc has to do with it.  We
> >> specifically separated the transport from the API and the c
> >> bindings should have nothing to do with xmlrpc.
> >
> > And I'm suggesting is to make the C bindings the supported interface
> > and make
> > the wire protocol an implementation detail.
> >
> > You could then implement the C bindings (using GObject or whatever)
> > via the
> > XML-RPC transport if you liked.
> >
> > It also presents a better integration story as you could more easily
> > expose the
> > VDSM API as part of a large CIM/AMQP model data model if you like.
>
>  Sounds good then.
>
>  Your previous comment:
> >>>>     From a compatibility perspective, libvdsm would be fully
> >>>>    backwards
> >>>>    compatible
> >>>> with old versions of VDSM (so it would keep XML-RPC support
> >>>> forever)
>
>  Threw me off there as we would not want to keep on supporting XML-RPC.
>  What's more, the current API has way too many things that have to go so needing to
> keep those would just be sad and painful. 

libvdsm would keep XML-RPC support--not VDSM itself.

IOW, libvdsm would need enough XML-RPC support to talk to old versions of VDSM. 
  That doesn't mean libvdsm needs to expose ever possible thing that's in the 
current XML-RPC interface.

Regards,

Anthony Liguori

> >
> > Regards,
> >
> > Anthony Liguori
> >
> >>
> >>>
> >>> ----- Original Message -----
> >>>> From: "Anthony Liguori"<anthony(a)codemonkey.ws>
> >>>> To: "VDSM Project
> Development"<vdsm-devel(a)lists.fedorahosted.org>
> >>>> Cc: "Adam Litke"<agl(a)us.ibm.com>, "Saggi
> Mizrahi"
> >>>> <smizrahi(a)redhat.com&gt;
> >>>> Sent: Monday, June 25, 2012 10:18:33 AM
> >>>> Subject: [RFC] An alternative way to provide a supported
> >>>> interface
> >>>> -- libvdsm
> >>>>
> >>>> Hi,
> >>>>
> >>>> I've been reading through the API threads here and considering
> >>>> the
> >>>> options.  To
> >>>> be honest, I worry a lot about the scope of these discussions and
> >>>> that there's a
> >>>> tremendous amount of work before we have a useful end result.
> >>>>
> >>>> I wonder if we can solve this problem by adding another layer of
> >>>> abstraction...
> >>>>
> >>>> As Adam is currently building a schema for VDSM's XML-RPC, we
> >>>> could
> >>>> use the QAPI
> >>>> code generators to build a libvdsm that provided a programmatic C
> >>>> interface for
> >>>> the XML-RPC interface.
> >>>>
> >>>> It would take some tweaking, but this could be made a supportable
> >>>> C
> >>>> interface.
> >>>> The rules for having a supportable C interface are basically:
> >>>>
> >>>> 1) Never change function signatures
> >>>>
> >>>> 2) Never remove functions
> >>>>
> >>>> 3) Always allocate structures in the library and/or pad
> >>>>
> >>>> 4) Only add to structures, never remove or reorder
> >>>>
> >>>> 5) Provide flags that default to zero to indicate that
> >>>> fields/features are not
> >>>> present.
> >>>>
> >>>> 6) Always zero-initialize structures
> >>>>
> >>>> Having a libvdsm would allow the transport to change over time
> >>>> w/o
> >>>> affecting
> >>>> end-users.  There are lots of good tools for documenting C APIs
> >>>> and
> >>>> dealing with
> >>>> versioning of C APIs.
> >>>>
> >>>> While we can start out with a schema-generated API, over time, we
> >>>> can
> >>>> implement
> >>>> libvdsm in an open-coded fashion allowing old APIs to be
> >>>> reimplemented in terms
> >>>> of new APIs.
> >>>>
> >>>>    From a compatibility perspective, libvdsm would be fully
> >>>>    backwards
> >>>>    compatible
> >>>> with old versions of VDSM (so it would keep XML-RPC support
> >>>> forever)
> >>>> but may
> >>>> require new versions of libvdsm to talk to new versions of VDSM.
> >>>>    That would
> >>>> allow for APIs to be deprecated within VDSM without breaking old
> >>>> clients.
> >>>>
> >>>> I think this would be an incremental approach to building a
> >>>> supportable API
> >>>> today while still giving the flexibility to make changes in the
> >>>> long
> >>>> term.
> >>>>
> >>>> And it should be fairly easy to generate a JNI binding and also
> >>>> port
> >>>> ovirt-engine to use an interface like this (since it already uses
> >>>> the
> >>>> XML-RPC API).
> >>>>
> >>>> Regards,
> >>>>
> >>>> Anthony Liguori
> >>>>
> >>> _______________________________________________
> >>> vdsm-devel mailing list
> >>> vdsm-devel(a)lists.fedorahosted.org
> >>> https://fedorahosted.org/mailman/listinfo/vdsm-devel
> >>>
> >
> >

On Wed, Jun 27, 2012 at 02:50:02PM -0400, Saggi Mizrahi wrote:
>  The idea of having a supported C API was something I was thinking
> about doing
>  (But I'd rather use gobject introspection and not schema generation) But the
>  problem is not having a C API is using the current XML RPC API as it's base 
I want to disect this a bit to find out exactly where there might be agreement
and disagreement.

C API is a good thing to implement - Agreed.

I also want to use gobject introspection but I don't agree that using glib
precludes the use of a formalized schema.  My proposal is that we write a schema
definition and generate the glib C code from that schema.

I agree that the _current_ xmlrpc API makes a pretty bad base from which to
start a supportable API.  XMLRPC is a perfectly reasonable remote/wire protocol
and I think we should continue using it as a base for the next generation API.
Using a schema will ensure that the new API is well-structured.
 
>  The current XML-RPC API contains a lot of decencies and
> inefficiencies and we
>  would like to retire it as soon as we possibly can. Engine would like us to
>  move to a message based API and 3rd parties want something simple like REST so
>  it looks like no one actually wants to use XML-RPC. Not even us. 
I am proposing that AMQP brokers and REST APIs could be written against the
public API.  In fact, they need not even live in the vdsm tree anymore if that
is what we choose.  Core vdsm would only be responsible for providing libvdsm
and whatever language bindings we want to support.

>  I do think that having C supportability in our API is a good idea,
> but the
>  current API should not be used as the base. 
Let's _start_ with a schema document that describes today's API and then clean
it up.  I think that will work better than starting from scratch.  Once my
schema is written I will post it and we can 'patch' it as a community until we
arrive at a 1.0 version we are all happy with.

If we are going to break compatibility with the current xmlrpc interface, I
would like to have a commitment from ovirt-engine to support the new library
interface so that we can drop xmlrpc.

--
Adam Litke <agl(a)us.ibm.com&gt;
IBM Linux Technology Center

----- Original Message -----
>  From: "Adam Litke" <agl(a)us.ibm.com&gt;
>  To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt;
>  Cc: "Anthony Liguori" <anthony(a)codemonkey.ws&gt;, "VDSM Project
> Development" <vdsm-devel(a)lists.fedorahosted.org&gt;
>  Sent: Thursday, July 5, 2012 2:34:50 PM
>  Subject: Re: [RFC] An alternative way to provide a supported interface -- libvdsm
>  
>  On Wed, Jun 27, 2012 at 02:50:02PM -0400, Saggi Mizrahi wrote:
>  > The idea of having a supported C API was something I was thinking
>  > about doing
>  > (But I'd rather use gobject introspection and not schema
>  > generation) But the
>  > problem is not having a C API is using the current XML RPC API as
>  > it's base
>  
>  I want to disect this a bit to find out exactly where there might be
>  agreement
>  and disagreement.
>  
>  C API is a good thing to implement - Agreed.
>  
>  I also want to use gobject introspection but I don't agree that using
>  glib
>  precludes the use of a formalized schema.  My proposal is that we
>  write a schema
>  definition and generate the glib C code from that schema.
>  
>  I agree that the _current_ xmlrpc API makes a pretty bad base from
>  which to
>  start a supportable API.  XMLRPC is a perfectly reasonable
>  remote/wire protocol
>  and I think we should continue using it as a base for the next
>  generation API.
>  Using a schema will ensure that the new API is well-structured. 
There major
problems with XML-RPC (and to some extent with REST as well) are high call overhead and no
two way communication (push events). Basing on XML-RPC means that we will never be able to
solve these issues.

>   
>  > The current XML-RPC API contains a lot of decencies and
>  > inefficiencies and we
>  > would like to retire it as soon as we possibly can. Engine would
>  > like us to
>  > move to a message based API and 3rd parties want something simple
>  > like REST so
>  > it looks like no one actually wants to use XML-RPC. Not even us.
>  
>  I am proposing that AMQP brokers and REST APIs could be written
>  against the
>  public API.  In fact, they need not even live in the vdsm tree
>  anymore if that
>  is what we choose.  Core vdsm would only be responsible for providing
>  libvdsm
>  and whatever language bindings we want to support. 
If we take the libvdsm route,
the only reason to even have a REST bridge is only to support OSes other then Linux which
is something I'm not sure we care about at the moment.

>  
>  > I do think that having C supportability in our API is a good idea,
>  > but the
>  > current API should not be used as the base.
>  
>  Let's _start_ with a schema document that describes today's API and
>  then clean
>  it up.  I think that will work better than starting from scratch.
>   Once my
>  schema is written I will post it and we can 'patch' it as a community
>  until we
>  arrive at a 1.0 version we are all happy with. 
+1
>  
>  If we are going to break compatibility with the current xmlrpc
>  interface, I
>  would like to have a commitment from ovirt-engine to support the new
>  library
>  interface so that we can drop xmlrpc.
>  
>  --
>  Adam Litke <agl(a)us.ibm.com&gt;
>  IBM Linux Technology Center
>  
>

On Thu, Jul 05, 2012 at 03:47:42PM -0400, Saggi Mizrahi wrote:
>  
>  
>  ----- Original Message -----
>  > From: "Adam Litke" <agl(a)us.ibm.com&gt;
>  > To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt;
>  > Cc: "Anthony Liguori" <anthony(a)codemonkey.ws&gt;, "VDSM Project
> Development" <vdsm-devel(a)lists.fedorahosted.org&gt;
>  > Sent: Thursday, July 5, 2012 2:34:50 PM
>  > Subject: Re: [RFC] An alternative way to provide a supported interface -- libvdsm
>  > 
>  > On Wed, Jun 27, 2012 at 02:50:02PM -0400, Saggi Mizrahi wrote:
>  > > The idea of having a supported C API was something I was thinking
>  > > about doing
>  > > (But I'd rather use gobject introspection and not schema
>  > > generation) But the
>  > > problem is not having a C API is using the current XML RPC API as
>  > > it's base
>  > 
>  > I want to disect this a bit to find out exactly where there might be
>  > agreement
>  > and disagreement.
>  > 
>  > C API is a good thing to implement - Agreed.
>  > 
>  > I also want to use gobject introspection but I don't agree that using
>  > glib
>  > precludes the use of a formalized schema.  My proposal is that we
>  > write a schema
>  > definition and generate the glib C code from that schema.
>  > 
>  > I agree that the _current_ xmlrpc API makes a pretty bad base from
>  > which to
>  > start a supportable API.  XMLRPC is a perfectly reasonable
>  > remote/wire protocol
>  > and I think we should continue using it as a base for the next
>  > generation API.
>  > Using a schema will ensure that the new API is well-structured.
>  There major problems with XML-RPC (and to some extent with REST as well) are high call
> overhead and no two way communication (push events). Basing on XML-RPC means that we will
> never be able to solve these issues. 
I am not sure I am ready to conceed that XML-RPC is too slow for our needs.  Can
you provide some more detail around this point and possibly suggest an
alternative that has even lower overhead without sacrificing the ubiquity and
usability of XML-RPC?  As far as the two-way communication point, what are the
options besides AMQP/ZeroMQ?  Aren't these even worse from an overhead
perspective than XML-RPC?  Regarding two-way communication: you can write AMQP
brokers based on the C API and run one on each vdsm host.  Assuming the C API
supports events, what else would you need?

>  > > The current XML-RPC API contains a lot of decencies and
>  > > inefficiencies and we
>  > > would like to retire it as soon as we possibly can. Engine would
>  > > like us to
>  > > move to a message based API and 3rd parties want something simple
>  > > like REST so
>  > > it looks like no one actually wants to use XML-RPC. Not even us.
>  > 
>  > I am proposing that AMQP brokers and REST APIs could be written
>  > against the
>  > public API.  In fact, they need not even live in the vdsm tree
>  > anymore if that
>  > is what we choose.  Core vdsm would only be responsible for providing
>  > libvdsm
>  > and whatever language bindings we want to support.
>  If we take the libvdsm route, the only reason to even have a REST bridge is only to
> support OSes other then Linux which is something I'm not sure we care about at the
> moment. 
That might be true regarding the current in-tree implementation.  However, I can
almost guarantee that someone wanting to write a web GUI on top of standalone
vdsm would want a REST API to talk to.  But libvdsm makes this use case of no
concern to the core vdsm developers.

>  > > I do think that having C supportability in our API is a
> good idea,
>  > > but the
>  > > current API should not be used as the base.
>  > 
>  > Let's _start_ with a schema document that describes today's API and
>  > then clean
>  > it up.  I think that will work better than starting from scratch.
>  >  Once my
>  > schema is written I will post it and we can 'patch' it as a community
>  > until we
>  > arrive at a 1.0 version we are all happy with.
>  +1 
Ok.  Redoubling my efforts to get this done.  Describing the output of
list(True) takes awhile :)

-- 
Adam Litke <agl(a)us.ibm.com&gt;
IBM Linux Technology Center

----- Original Message -----
>  From: "Adam Litke" <agl(a)us.ibm.com&gt;
>  To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt;
>  Cc: "Anthony Liguori" <anthony(a)codemonkey.ws&gt;, "VDSM Project
> Development" <vdsm-devel(a)lists.fedorahosted.org&gt;
>  Sent: Thursday, July 5, 2012 4:45:08 PM
>  Subject: Re: [RFC] An alternative way to provide a supported interface -- libvdsm
>  
>  On Thu, Jul 05, 2012 at 03:47:42PM -0400, Saggi Mizrahi wrote:
>  > 
>  > 
>  > ----- Original Message -----
>  > > From: "Adam Litke" <agl(a)us.ibm.com&gt;
>  > > To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt;
>  > > Cc: "Anthony Liguori" <anthony(a)codemonkey.ws&gt;, "VDSM
> Project
>  > > Development" <vdsm-devel(a)lists.fedorahosted.org&gt;
>  > > Sent: Thursday, July 5, 2012 2:34:50 PM
>  > > Subject: Re: [RFC] An alternative way to provide a supported
>  > > interface -- libvdsm
>  > > 
>  > > On Wed, Jun 27, 2012 at 02:50:02PM -0400, Saggi Mizrahi wrote:
>  > > > The idea of having a supported C API was something I was
>  > > > thinking
>  > > > about doing
>  > > > (But I'd rather use gobject introspection and not schema
>  > > > generation) But the
>  > > > problem is not having a C API is using the current XML RPC API
>  > > > as
>  > > > it's base
>  > > 
>  > > I want to disect this a bit to find out exactly where there might
>  > > be
>  > > agreement
>  > > and disagreement.
>  > > 
>  > > C API is a good thing to implement - Agreed.
>  > > 
>  > > I also want to use gobject introspection but I don't agree that
>  > > using
>  > > glib
>  > > precludes the use of a formalized schema.  My proposal is that we
>  > > write a schema
>  > > definition and generate the glib C code from that schema.
>  > > 
>  > > I agree that the _current_ xmlrpc API makes a pretty bad base
>  > > from
>  > > which to
>  > > start a supportable API.  XMLRPC is a perfectly reasonable
>  > > remote/wire protocol
>  > > and I think we should continue using it as a base for the next
>  > > generation API.
>  > > Using a schema will ensure that the new API is well-structured.
>  > There major problems with XML-RPC (and to some extent with REST as
>  > well) are high call overhead and no two way communication (push
>  > events). Basing on XML-RPC means that we will never be able to
>  > solve these issues.
>  
>  I am not sure I am ready to conceed that XML-RPC is too slow for our
>  needs.  Can
>  you provide some more detail around this point and possibly suggest
>  an
>  alternative that has even lower overhead without sacrificing the
>  ubiquity and
>  usability of XML-RPC?  As far as the two-way communication point,
>  what are the
>  options besides AMQP/ZeroMQ?  Aren't these even worse from an
>  overhead
>  perspective than XML-RPC?  Regarding two-way communication: you can
>  write AMQP
>  brokers based on the C API and run one on each vdsm host.  Assuming
>  the C API
>  supports events, what else would you need? 
If we plan to go with the libvdsm route
the only transports I think are appropriate are either raw sockets (like libvirt) or ZMQ
(just to take advantage of it managing connection and message encapsulation but it might
be an overkill). Other then that ZMQ\AMQP\REST\XML-RPC bridges are not really a priority
for me as engine will not be using any of the bridges.
>  
>  > > > The current XML-RPC API contains a lot of decencies and
>  > > > inefficiencies and we
>  > > > would like to retire it as soon as we possibly can. Engine
>  > > > would
>  > > > like us to
>  > > > move to a message based API and 3rd parties want something
>  > > > simple
>  > > > like REST so
>  > > > it looks like no one actually wants to use XML-RPC. Not even
>  > > > us.
>  > > 
>  > > I am proposing that AMQP brokers and REST APIs could be written
>  > > against the
>  > > public API.  In fact, they need not even live in the vdsm tree
>  > > anymore if that
>  > > is what we choose.  Core vdsm would only be responsible for
>  > > providing
>  > > libvdsm
>  > > and whatever language bindings we want to support.
>  > If we take the libvdsm route, the only reason to even have a REST
>  > bridge is only to support OSes other then Linux which is something
>  > I'm not sure we care about at the moment.
>  
>  That might be true regarding the current in-tree implementation.
>   However, I can
>  almost guarantee that someone wanting to write a web GUI on top of
>  standalone
>  vdsm would want a REST API to talk to.  But libvdsm makes this use
>  case of no
>  concern to the core vdsm developers.
>  
>  > > > I do think that having C supportability in our API is a good
>  > > > idea,
>  > > > but the
>  > > > current API should not be used as the base.
>  > > 
>  > > Let's _start_ with a schema document that describes today's API
>  > > and
>  > > then clean
>  > > it up.  I think that will work better than starting from scratch.
>  > >  Once my
>  > > schema is written I will post it and we can 'patch' it as a
>  > > community
>  > > until we
>  > > arrive at a 1.0 version we are all happy with.
>  > +1
>  
>  Ok.  Redoubling my efforts to get this done.  Describing the output
>  of
>  list(True) takes awhile :)
>  
>  --
>  Adam Litke <agl(a)us.ibm.com&gt;
>  IBM Linux Technology Center
>  
>

Adding danpb and DV to get some perspective from libvirt...

On Thu, Jul 05, 2012 at 05:02:59PM -0400, Saggi Mizrahi wrote:
>  
>  
>  ----- Original Message -----
>  > From: "Adam Litke" <agl(a)us.ibm.com&gt;
>  > To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt;
>  > Cc: "Anthony Liguori" <anthony(a)codemonkey.ws&gt;, "VDSM Project
> Development" <vdsm-devel(a)lists.fedorahosted.org&gt;
>  > Sent: Thursday, July 5, 2012 4:45:08 PM
>  > Subject: Re: [RFC] An alternative way to provide a supported interface -- libvdsm
>  > 
>  > On Thu, Jul 05, 2012 at 03:47:42PM -0400, Saggi Mizrahi wrote:
>  > > 
>  > > 
>  > > ----- Original Message -----
>  > > > From: "Adam Litke" <agl(a)us.ibm.com&gt;
>  > > > To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt;
>  > > > Cc: "Anthony Liguori" <anthony(a)codemonkey.ws&gt;, "VDSM
> Project
>  > > > Development" <vdsm-devel(a)lists.fedorahosted.org&gt;
>  > > > Sent: Thursday, July 5, 2012 2:34:50 PM
>  > > > Subject: Re: [RFC] An alternative way to provide a supported
>  > > > interface -- libvdsm
>  > > > 
>  > > > On Wed, Jun 27, 2012 at 02:50:02PM -0400, Saggi Mizrahi wrote:
>  > > > > The idea of having a supported C API was something I was
>  > > > > thinking
>  > > > > about doing
>  > > > > (But I'd rather use gobject introspection and not schema
>  > > > > generation) But the
>  > > > > problem is not having a C API is using the current XML RPC API
>  > > > > as
>  > > > > it's base
>  > > > 
>  > > > I want to disect this a bit to find out exactly where there might
>  > > > be
>  > > > agreement
>  > > > and disagreement.
>  > > > 
>  > > > C API is a good thing to implement - Agreed.
>  > > > 
>  > > > I also want to use gobject introspection but I don't agree that
>  > > > using
>  > > > glib
>  > > > precludes the use of a formalized schema.  My proposal is that we
>  > > > write a schema
>  > > > definition and generate the glib C code from that schema.
>  > > > 
>  > > > I agree that the _current_ xmlrpc API makes a pretty bad base
>  > > > from
>  > > > which to
>  > > > start a supportable API.  XMLRPC is a perfectly reasonable
>  > > > remote/wire protocol
>  > > > and I think we should continue using it as a base for the next
>  > > > generation API.
>  > > > Using a schema will ensure that the new API is well-structured.
>  > > There major problems with XML-RPC (and to some extent with REST as
>  > > well) are high call overhead and no two way communication (push
>  > > events). Basing on XML-RPC means that we will never be able to
>  > > solve these issues.
>  > 
>  > I am not sure I am ready to conceed that XML-RPC is too slow for our
>  > needs.  Can
>  > you provide some more detail around this point and possibly suggest
>  > an
>  > alternative that has even lower overhead without sacrificing the
>  > ubiquity and
>  > usability of XML-RPC?  As far as the two-way communication point,
>  > what are the
>  > options besides AMQP/ZeroMQ?  Aren't these even worse from an
>  > overhead
>  > perspective than XML-RPC?  Regarding two-way communication: you can
>  > write AMQP
>  > brokers based on the C API and run one on each vdsm host.  Assuming
>  > the C API
>  > supports events, what else would you need?
>  If we plan to go with the libvdsm route the only transports I think are appropriate are
> either raw sockets (like libvirt) or ZMQ (just to take advantage of it managing connection
> and message encapsulation but it might be an overkill). Other then that
> ZMQ\AMQP\REST\XML-RPC bridges are not really a priority for me as engine will not be using
> any of the bridges. 
I think (and others have also suggested) that ZMQ is not very suitable as the
foundation of an API due to it's heavy weight.  I would really appreciate the
perspective of the libvirt guys and others in the community on this topic.
Between you and I, the only approach that hasn't been nacked is:
raw sockets and xdr ala libvirt.  Are there any detractors of this idea?
Anything better to consider?  xdr is ancient :)

>  > 
>  > > > > The current XML-RPC API contains a lot of decencies and
>  > > > > inefficiencies and we
>  > > > > would like to retire it as soon as we possibly can. Engine
>  > > > > would
>  > > > > like us to
>  > > > > move to a message based API and 3rd parties want something
>  > > > > simple
>  > > > > like REST so
>  > > > > it looks like no one actually wants to use XML-RPC. Not even
>  > > > > us.
>  > > > 
>  > > > I am proposing that AMQP brokers and REST APIs could be written
>  > > > against the
>  > > > public API.  In fact, they need not even live in the vdsm tree
>  > > > anymore if that
>  > > > is what we choose.  Core vdsm would only be responsible for
>  > > > providing
>  > > > libvdsm
>  > > > and whatever language bindings we want to support.
>  > > If we take the libvdsm route, the only reason to even have a REST
>  > > bridge is only to support OSes other then Linux which is something
>  > > I'm not sure we care about at the moment.
>  > 
>  > That might be true regarding the current in-tree implementation.
>  >  However, I can
>  > almost guarantee that someone wanting to write a web GUI on top of
>  > standalone
>  > vdsm would want a REST API to talk to.  But libvdsm makes this use
>  > case of no
>  > concern to the core vdsm developers.
>  > 
>  > > > > I do think that having C supportability in our API is a good
>  > > > > idea,
>  > > > > but the
>  > > > > current API should not be used as the base.
>  > > > 
>  > > > Let's _start_ with a schema document that describes today's API
>  > > > and
>  > > > then clean
>  > > > it up.  I think that will work better than starting from scratch.
>  > > >  Once my
>  > > > schema is written I will post it and we can 'patch' it as a
>  > > > community
>  > > > until we
>  > > > arrive at a 1.0 version we are all happy with.
>  > > +1
>  > 
>  > Ok.  Redoubling my efforts to get this done.  Describing the output
>  > of
>  > list(True) takes awhile :)
>  > 
>  > --
>  > Adam Litke <agl(a)us.ibm.com&gt;
>  > IBM Linux Technology Center
>  > 
>  > 
>   
-- 
Adam Litke <agl(a)us.ibm.com&gt;
IBM Linux Technology Center

[Daniel P. Berrange]

On Fri, Jul 06, 2012 at 07:17:44AM -0500, Adam Litke wrote:
>  Adding danpb and DV to get some perspective from libvirt...
>  
>  On Thu, Jul 05, 2012 at 05:02:59PM -0400, Saggi Mizrahi wrote:
>  > 
>  > > 
>  > > I am not sure I am ready to conceed that XML-RPC is too slow for our
>  > > needs.  Can
>  > > you provide some more detail around this point and possibly suggest
>  > > an
>  > > alternative that has even lower overhead without sacrificing the
>  > > ubiquity and
>  > > usability of XML-RPC?  As far as the two-way communication point,
>  > > what are the
>  > > options besides AMQP/ZeroMQ?  Aren't these even worse from an
>  > > overhead
>  > > perspective than XML-RPC?  Regarding two-way communication: you can
>  > > write AMQP
>  > > brokers based on the C API and run one on each vdsm host.  Assuming
>  > > the C API
>  > > supports events, what else would you need?
>  > If we plan to go with the libvdsm route the only transports I think are appropriate
> are either raw sockets (like libvirt) or ZMQ (just to take advantage of it managing
> connection and message encapsulation but it might be an overkill). Other then that
> ZMQ\AMQP\REST\XML-RPC bridges are not really a priority for me as engine will not be using
> any of the bridges.
>  
>  I think (and others have also suggested) that ZMQ is not very suitable as the
>  foundation of an API due to it's heavy weight.  I would really appreciate the
>  perspective of the libvirt guys and others in the community on this topic.
>  Between you and I, the only approach that hasn't been nacked is:
>  raw sockets and xdr ala libvirt.  Are there any detractors of this idea?
>  Anything better to consider?  xdr is ancient :) 
I think I've already given my views on this earlier in the thread.
To recap, I think that it is very important for the core functionality
of VDSM to be easy to get up & running / deployment. Adding a serious
messaging layer like AMQP or similar, as a mandatory component puts a
non-negligable barrier to entry.

Probably one of the most important things that drove adoption of libvirt
was that you can use it remotely & securely with no external infrastructure.
You just start libvirtd & can access it via its SSH tunnel facility. Of
course for most production deployments you wouldn't use this, but it is
important for encouraging developers to work on it & making it easy for
users to just try it out on a small scale.

WRT to XDR, remember that it is not a protocol - it is simply a spec for
how to encode various data structures in an efficient binary format. I
don't think how old it is really relevant, since there's only so many
different ways you want to encode binary data structures.

Since it is just a data format encoding, you have the job of defining
the actual RPC protocol on top of that. Now this isn't particularly
hard to get going at first, but the complexity does add up over time.
I'd still suggest aiming for some variant of REST for its simplicity
and the fact that you can then use standard HTTP server & client
components and solely focus on your APIs, rather than infrastructure.

Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

[Daniel Veillard]

On Fri, Jul 06, 2012 at 01:30:43PM +0100, Daniel P. Berrange wrote:
>  On Fri, Jul 06, 2012 at 07:17:44AM -0500, Adam Litke wrote:
>  > Adding danpb and DV to get some perspective from libvirt...
>  > 
>  > On Thu, Jul 05, 2012 at 05:02:59PM -0400, Saggi Mizrahi wrote:
>  > > 
>  > > > 
>  > > > I am not sure I am ready to conceed that XML-RPC is too slow for our
>  > > > needs.  Can
>  > > > you provide some more detail around this point and possibly suggest
>  > > > an
>  > > > alternative that has even lower overhead without sacrificing the
>  > > > ubiquity and
>  > > > usability of XML-RPC?  As far as the two-way communication point,
>  > > > what are the
>  > > > options besides AMQP/ZeroMQ?  Aren't these even worse from an
>  > > > overhead
>  > > > perspective than XML-RPC?  Regarding two-way communication: you can
>  > > > write AMQP
>  > > > brokers based on the C API and run one on each vdsm host.  Assuming
>  > > > the C API
>  > > > supports events, what else would you need?
>  > > If we plan to go with the libvdsm route the only transports I think are
> appropriate are either raw sockets (like libvirt) or ZMQ (just to take advantage of it
> managing connection and message encapsulation but it might be an overkill). Other then
> that ZMQ\AMQP\REST\XML-RPC bridges are not really a priority for me as engine will not be
> using any of the bridges.
>  > 
>  > I think (and others have also suggested) that ZMQ is not very suitable as the
>  > foundation of an API due to it's heavy weight.  I would really appreciate the
>  > perspective of the libvirt guys and others in the community on this topic.
>  > Between you and I, the only approach that hasn't been nacked is:
>  > raw sockets and xdr ala libvirt.  Are there any detractors of this idea?
>  > Anything better to consider?  xdr is ancient :)
>  
>  I think I've already given my views on this earlier in the thread.
>  To recap, I think that it is very important for the core functionality
>  of VDSM to be easy to get up & running / deployment. Adding a serious
>  messaging layer like AMQP or similar, as a mandatory component puts a
>  non-negligable barrier to entry.
>  
>  Probably one of the most important things that drove adoption of libvirt
>  was that you can use it remotely & securely with no external infrastructure.
>  You just start libvirtd & can access it via its SSH tunnel facility. Of
>  course for most production deployments you wouldn't use this, but it is
>  important for encouraging developers to work on it & making it easy for
>  users to just try it out on a small scale.
>  
>  WRT to XDR, remember that it is not a protocol - it is simply a spec for
>  how to encode various data structures in an efficient binary format. I
>  don't think how old it is really relevant, since there's only so many
>  different ways you want to encode binary data structures.
>  
>  Since it is just a data format encoding, you have the job of defining
>  the actual RPC protocol on top of that. Now this isn't particularly
>  hard to get going at first, but the complexity does add up over time.
>  I'd still suggest aiming for some variant of REST for its simplicity
>  and the fact that you can then use standard HTTP server & client
>  components and solely focus on your APIs, rather than infrastructure. 
  As Dan said xdr as the encoder is fine, but won't provide a protocol.
If you go that way this also mean that everybody will have to use the
library stub to talk to your server API, point in case nobody ever used
the API at the RPC level to talk to libvirtd (though we work very hard
to keep that ABI stable too).
  I guess that in a situation like talking to vdsm you don't want to
force people to go though a library you have to maintain/control/distribute
which is why I would suggest too to avoid the mess and export with a
common and easy to use protocol. My experience is that if you want an
API to be used you have to make it as easy to use as possible to get
traction (especially when you are competing with other solutions), and
forcing to go through a given library should be avoided if you can.
Having to maintain a zillion bindings (having spend just way to much
time lately with libvirt-java) takes time that could be invested better
on the core functionalities by just using a remote protocol available
in a variety of language.
  IMHO speed of encoding/decoding is pure noise. HTTP should be a proof
that you can design a slow solution and have it take over the world just
because it's easier to implement, use and debug. Pick an ubiquitous
solution or be ready to spend an awful lot of time maintainaing client
side entry points, bindings, distribution, work around protocol problems
etc...

  my 2 cents,

Daniel

-- 
Daniel Veillard      | libxml Gnome XML XSLT toolkit  http://xmlsoft.org/
daniel(a)veillard.com  | Rpmfind RPM search engine http://rpmfind.net/
http://veillard.com/ | virtualization library  http://libvirt.org/

On 07/05/2012 04:45 PM, Adam Litke wrote:
>  On Thu, Jul 05, 2012 at 03:47:42PM -0400, Saggi Mizrahi wrote:
> >
> > ----- Original Message -----
> >> From: "Adam Litke" <agl(a)us.ibm.com&gt;
> >> To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt;
> >> Cc: "Anthony Liguori" <anthony(a)codemonkey.ws&gt;, "VDSM Project
> Development" <vdsm-devel(a)lists.fedorahosted.org&gt;
> >> Sent: Thursday, July 5, 2012 2:34:50 PM
> >> Subject: Re: [RFC] An alternative way to provide a supported interface --
> libvdsm
> >>
> >> On Wed, Jun 27, 2012 at 02:50:02PM -0400, Saggi Mizrahi wrote:
> >>> The idea of having a supported C API was something I was thinking
> >>> about doing
> >>> (But I'd rather use gobject introspection and not schema
> >>> generation) But the
> >>> problem is not having a C API is using the current XML RPC API as
> >>> it's base
> >> I want to disect this a bit to find out exactly where there might be
> >> agreement
> >> and disagreement.
> >>
> >> C API is a good thing to implement - Agreed.
> >>
> >> I also want to use gobject introspection but I don't agree that using
> >> glib
> >> precludes the use of a formalized schema.  My proposal is that we
> >> write a schema
> >> definition and generate the glib C code from that schema.
> >>
> >> I agree that the _current_ xmlrpc API makes a pretty bad base from
> >> which to
> >> start a supportable API.  XMLRPC is a perfectly reasonable
> >> remote/wire protocol
> >> and I think we should continue using it as a base for the next
> >> generation API.
> >> Using a schema will ensure that the new API is well-structured.
> > There major problems with XML-RPC (and to some extent with REST as well) are high
> call overhead and no two way communication (push events). Basing on XML-RPC means that we
> will never be able to solve these issues.
>  I am not sure I am ready to conceed that XML-RPC is too slow for our needs.  Can
>  you provide some more detail around this point and possibly suggest an
>  alternative that has even lower overhead without sacrificing the ubiquity and
>  usability of XML-RPC?  As far as the two-way communication point, what are the
>  options besides AMQP/ZeroMQ?  Aren't these even worse from an overhead
>  perspective than XML-RPC?  Regarding two-way communication: you can write AMQP
>  brokers based on the C API and run one on each vdsm host.  Assuming the C API
>  supports events, what else would you need? 
I personally think that using something
like AMQP for inter-node 
communication and engine - node would be optimal.  With a rest interface 
that just send messages though something like AMQP.

Thanks
Robert
> >>> The current XML-RPC API contains a lot of decencies and
> >>> inefficiencies and we
> >>> would like to retire it as soon as we possibly can. Engine would
> >>> like us to
> >>> move to a message based API and 3rd parties want something simple
> >>> like REST so
> >>> it looks like no one actually wants to use XML-RPC. Not even us.
> >> I am proposing that AMQP brokers and REST APIs could be written
> >> against the
> >> public API.  In fact, they need not even live in the vdsm tree
> >> anymore if that
> >> is what we choose.  Core vdsm would only be responsible for providing
> >> libvdsm
> >> and whatever language bindings we want to support.
> > If we take the libvdsm route, the only reason to even have a REST bridge is only to
> support OSes other then Linux which is something I'm not sure we care about at the
> moment.
>  That might be true regarding the current in-tree implementation.  However, I can
>  almost guarantee that someone wanting to write a web GUI on top of standalone
>  vdsm would want a REST API to talk to.  But libvdsm makes this use case of no
>  concern to the core vdsm developers.
>
> >>> I do think that having C supportability in our API is a good idea,
> >>> but the
> >>> current API should not be used as the base.
> >> Let's _start_ with a schema document that describes today's API and
> >> then clean
> >> it up.  I think that will work better than starting from scratch.
> >>   Once my
> >> schema is written I will post it and we can 'patch' it as a community
> >> until we
> >> arrive at a 1.0 version we are all happy with.
> > +1
>  Ok.  Redoubling my efforts to get this done.  Describing the output of
>  list(True) takes awhile :)
>

On 07/06/2012 01:15 AM, Robert Middleswarth wrote:
>  On 07/05/2012 04:45 PM, Adam Litke wrote:
> > On Thu, Jul 05, 2012 at 03:47:42PM -0400, Saggi Mizrahi wrote:
> >>
> >> ----- Original Message -----
> >>> From: "Adam Litke" <agl(a)us.ibm.com&gt;
> >>> To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt;
> >>> Cc: "Anthony Liguori" <anthony(a)codemonkey.ws&gt;, "VDSM
> Project
> >>> Development" <vdsm-devel(a)lists.fedorahosted.org&gt;
> >>> Sent: Thursday, July 5, 2012 2:34:50 PM
> >>> Subject: Re: [RFC] An alternative way to provide a supported
> >>> interface -- libvdsm
> >>>
> >>> On Wed, Jun 27, 2012 at 02:50:02PM -0400, Saggi Mizrahi wrote:
> >>>> The idea of having a supported C API was something I was thinking
> >>>> about doing
> >>>> (But I'd rather use gobject introspection and not schema
> >>>> generation) But the
> >>>> problem is not having a C API is using the current XML RPC API as
> >>>> it's base
> >>> I want to disect this a bit to find out exactly where there might be
> >>> agreement
> >>> and disagreement.
> >>>
> >>> C API is a good thing to implement - Agreed.
> >>>
> >>> I also want to use gobject introspection but I don't agree that using
> >>> glib
> >>> precludes the use of a formalized schema.  My proposal is that we
> >>> write a schema
> >>> definition and generate the glib C code from that schema.
> >>>
> >>> I agree that the _current_ xmlrpc API makes a pretty bad base from
> >>> which to
> >>> start a supportable API.  XMLRPC is a perfectly reasonable
> >>> remote/wire protocol
> >>> and I think we should continue using it as a base for the next
> >>> generation API.
> >>> Using a schema will ensure that the new API is well-structured.
> >> There major problems with XML-RPC (and to some extent with REST as
> >> well) are high call overhead and no two way communication (push
> >> events). Basing on XML-RPC means that we will never be able to solve
> >> these issues.
> > I am not sure I am ready to conceed that XML-RPC is too slow for our
> > needs.  Can
> > you provide some more detail around this point and possibly suggest an
> > alternative that has even lower overhead without sacrificing the
> > ubiquity and
> > usability of XML-RPC?  As far as the two-way communication point, what
> > are the
> > options besides AMQP/ZeroMQ?  Aren't these even worse from an overhead
> > perspective than XML-RPC?  Regarding two-way communication: you can
> > write AMQP
> > brokers based on the C API and run one on each vdsm host.  Assuming
> > the C API
> > supports events, what else would you need?
>  I personally think that using something like AMQP for inter-node
>  communication and engine - node would be optimal.  With a rest interface
>  that just send messages though something like AMQP. 
I would also not dismiss AMQP so soon
we want a bug with more than a single listener at engine side (engine, 
history db, maybe event correlation service).
collectd as a means for statistics already supports it as well.
I'm for having REST as well, but not sure as main one for a consumer 
like ovirt engine.


>  Thanks
>  Robert
> >>>> The current XML-RPC API contains a lot of decencies and
> >>>> inefficiencies and we
> >>>> would like to retire it as soon as we possibly can. Engine would
> >>>> like us to
> >>>> move to a message based API and 3rd parties want something simple
> >>>> like REST so
> >>>> it looks like no one actually wants to use XML-RPC. Not even us.
> >>> I am proposing that AMQP brokers and REST APIs could be written
> >>> against the
> >>> public API.  In fact, they need not even live in the vdsm tree
> >>> anymore if that
> >>> is what we choose.  Core vdsm would only be responsible for providing
> >>> libvdsm
> >>> and whatever language bindings we want to support.
> >> If we take the libvdsm route, the only reason to even have a REST
> >> bridge is only to support OSes other then Linux which is something
> >> I'm not sure we care about at the moment.
> > That might be true regarding the current in-tree implementation.
> > However, I can
> > almost guarantee that someone wanting to write a web GUI on top of
> > standalone
> > vdsm would want a REST API to talk to.  But libvdsm makes this use
> > case of no
> > concern to the core vdsm developers.
> >
> >>>> I do think that having C supportability in our API is a good idea,
> >>>> but the
> >>>> current API should not be used as the base.
> >>> Let's _start_ with a schema document that describes today's API and
> >>> then clean
> >>> it up.  I think that will work better than starting from scratch.
> >>>   Once my
> >>> schema is written I will post it and we can 'patch' it as a
> community
> >>> until we
> >>> arrive at a 1.0 version we are all happy with.
> >> +1
> > Ok.  Redoubling my efforts to get this done.  Describing the output of
> > list(True) takes awhile :)
> >
>
>
>  _______________________________________________
>  vdsm-devel mailing list
>  vdsm-devel(a)lists.fedorahosted.org
>  https://fedorahosted.org/mailman/listinfo/vdsm-devel

On Fri, Jul 06, 2012 at 03:53:08PM +0300, Itamar Heim wrote:
>  On 07/06/2012 01:15 AM, Robert Middleswarth wrote:
>  >On 07/05/2012 04:45 PM, Adam Litke wrote:
>  >>On Thu, Jul 05, 2012 at 03:47:42PM -0400, Saggi Mizrahi wrote:
>  >>>
>  >>>----- Original Message -----
>  >>>>From: "Adam Litke" <agl(a)us.ibm.com&gt;
>  >>>>To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt;
>  >>>>Cc: "Anthony Liguori" <anthony(a)codemonkey.ws&gt;, "VDSM
> Project
>  >>>>Development" <vdsm-devel(a)lists.fedorahosted.org&gt;
>  >>>>Sent: Thursday, July 5, 2012 2:34:50 PM
>  >>>>Subject: Re: [RFC] An alternative way to provide a supported
>  >>>>interface -- libvdsm
>  >>>>
>  >>>>On Wed, Jun 27, 2012 at 02:50:02PM -0400, Saggi Mizrahi wrote:
>  >>>>>The idea of having a supported C API was something I was thinking
>  >>>>>about doing
>  >>>>>(But I'd rather use gobject introspection and not schema
>  >>>>>generation) But the
>  >>>>>problem is not having a C API is using the current XML RPC API as
>  >>>>>it's base
>  >>>>I want to disect this a bit to find out exactly where there might be
>  >>>>agreement
>  >>>>and disagreement.
>  >>>>
>  >>>>C API is a good thing to implement - Agreed.
>  >>>>
>  >>>>I also want to use gobject introspection but I don't agree that
> using
>  >>>>glib
>  >>>>precludes the use of a formalized schema.  My proposal is that we
>  >>>>write a schema
>  >>>>definition and generate the glib C code from that schema.
>  >>>>
>  >>>>I agree that the _current_ xmlrpc API makes a pretty bad base from
>  >>>>which to
>  >>>>start a supportable API.  XMLRPC is a perfectly reasonable
>  >>>>remote/wire protocol
>  >>>>and I think we should continue using it as a base for the next
>  >>>>generation API.
>  >>>>Using a schema will ensure that the new API is well-structured.
>  >>>There major problems with XML-RPC (and to some extent with REST as
>  >>>well) are high call overhead and no two way communication (push
>  >>>events). Basing on XML-RPC means that we will never be able to solve
>  >>>these issues.
>  >>I am not sure I am ready to conceed that XML-RPC is too slow for our
>  >>needs.  Can
>  >>you provide some more detail around this point and possibly suggest an
>  >>alternative that has even lower overhead without sacrificing the
>  >>ubiquity and
>  >>usability of XML-RPC?  As far as the two-way communication point, what
>  >>are the
>  >>options besides AMQP/ZeroMQ?  Aren't these even worse from an overhead
>  >>perspective than XML-RPC?  Regarding two-way communication: you can
>  >>write AMQP
>  >>brokers based on the C API and run one on each vdsm host.  Assuming
>  >>the C API
>  >>supports events, what else would you need?
>  >I personally think that using something like AMQP for inter-node
>  >communication and engine - node would be optimal.  With a rest interface
>  >that just send messages though something like AMQP.
>  
>  I would also not dismiss AMQP so soon
>  we want a bug with more than a single listener at engine side
>  (engine, history db, maybe event correlation service).
>  collectd as a means for statistics already supports it as well.
>  I'm for having REST as well, but not sure as main one for a consumer
>  like ovirt engine. 
I agree that a message bus could be a very useful model of communication between
ovirt-engine components and multiple vdsm instances.  But the complexities and
dependencies of AMQP do not make it suitable for use as a low-level API.  AMQP
will repel new adopters.  Why not establish a libvdsm that is more minimalist
and can be easily used by everyone?  Then AMQP brokers can be built on top of
the stable API with ease.  All AMQP should require of the low-level API are
standard function calls and an events mechanism.

>  >
>  >Thanks
>  >Robert
>  >>>>>The current XML-RPC API contains a lot of decencies and
>  >>>>>inefficiencies and we
>  >>>>>would like to retire it as soon as we possibly can. Engine would
>  >>>>>like us to
>  >>>>>move to a message based API and 3rd parties want something simple
>  >>>>>like REST so
>  >>>>>it looks like no one actually wants to use XML-RPC. Not even us.
>  >>>>I am proposing that AMQP brokers and REST APIs could be written
>  >>>>against the
>  >>>>public API.  In fact, they need not even live in the vdsm tree
>  >>>>anymore if that
>  >>>>is what we choose.  Core vdsm would only be responsible for providing
>  >>>>libvdsm
>  >>>>and whatever language bindings we want to support.
>  >>>If we take the libvdsm route, the only reason to even have a REST
>  >>>bridge is only to support OSes other then Linux which is something
>  >>>I'm not sure we care about at the moment.
>  >>That might be true regarding the current in-tree implementation.
>  >>However, I can
>  >>almost guarantee that someone wanting to write a web GUI on top of
>  >>standalone
>  >>vdsm would want a REST API to talk to.  But libvdsm makes this use
>  >>case of no
>  >>concern to the core vdsm developers.
>  >>
>  >>>>>I do think that having C supportability in our API is a good idea,
>  >>>>>but the
>  >>>>>current API should not be used as the base.
>  >>>>Let's _start_ with a schema document that describes today's API
> and
>  >>>>then clean
>  >>>>it up.  I think that will work better than starting from scratch.
>  >>>>  Once my
>  >>>>schema is written I will post it and we can 'patch' it as a
> community
>  >>>>until we
>  >>>>arrive at a 1.0 version we are all happy with.
>  >>>+1
>  >>Ok.  Redoubling my efforts to get this done.  Describing the output of
>  >>list(True) takes awhile :)
>  >>
>  >
>  >
>  >_______________________________________________
>  >vdsm-devel mailing list
>  &gt;vdsm-devel(a)lists.fedorahosted.org
>  >https://fedorahosted.org/mailman/listinfo/vdsm-devel
>  
>  
>  _______________________________________________
>  vdsm-devel mailing list
>  vdsm-devel(a)lists.fedorahosted.org
>  https://fedorahosted.org/mailman/listinfo/vdsm-devel 
-- 
Adam Litke <agl(a)us.ibm.com&gt;
IBM Linux Technology Center

I don't think AMQP is a good low level supported protocol as it's a very complex
protocol to set up and support.
Also brokers are known to have their differences in standard implementation which means
supporting them all is a mess.

It looks like the most accepted route is the libvirt route of having a c library
abstracting away client server communication and having more advanced consumers build
protocol specific bridges that may have different support standards.

On a more personal note, I think brokerless messaging is the way to go in ovirt because,
unlike traditional clustering, worker nodes are not interchangeable so direct
communication is the way to go, rendering brokers pretty much useless.

----- Original Message -----
>  From: "Adam Litke" <agl(a)us.ibm.com&gt;
>  To: "Itamar Heim" <iheim(a)redhat.com&gt;
>  Cc: vdsm-devel(a)lists.fedorahosted.org
>  Sent: Monday, July 9, 2012 9:56:17 AM
>  Subject: Re: [vdsm] [RFC] An alternative way to provide a supported interface -- libvdsm
>  
>  On Fri, Jul 06, 2012 at 03:53:08PM +0300, Itamar Heim wrote:
>  > On 07/06/2012 01:15 AM, Robert Middleswarth wrote:
>  > >On 07/05/2012 04:45 PM, Adam Litke wrote:
>  > >>On Thu, Jul 05, 2012 at 03:47:42PM -0400, Saggi Mizrahi wrote:
>  > >>>
>  > >>>----- Original Message -----
>  > >>>>From: "Adam Litke" <agl(a)us.ibm.com&gt;
>  > >>>>To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt;
>  > >>>>Cc: "Anthony Liguori" <anthony(a)codemonkey.ws&gt;,
> "VDSM Project
>  > >>>>Development" <vdsm-devel(a)lists.fedorahosted.org&gt;
>  > >>>>Sent: Thursday, July 5, 2012 2:34:50 PM
>  > >>>>Subject: Re: [RFC] An alternative way to provide a supported
>  > >>>>interface -- libvdsm
>  > >>>>
>  > >>>>On Wed, Jun 27, 2012 at 02:50:02PM -0400, Saggi Mizrahi wrote:
>  > >>>>>The idea of having a supported C API was something I was
>  > >>>>>thinking
>  > >>>>>about doing
>  > >>>>>(But I'd rather use gobject introspection and not schema
>  > >>>>>generation) But the
>  > >>>>>problem is not having a C API is using the current XML RPC API
>  > >>>>>as
>  > >>>>>it's base
>  > >>>>I want to disect this a bit to find out exactly where there
>  > >>>>might be
>  > >>>>agreement
>  > >>>>and disagreement.
>  > >>>>
>  > >>>>C API is a good thing to implement - Agreed.
>  > >>>>
>  > >>>>I also want to use gobject introspection but I don't agree that
>  > >>>>using
>  > >>>>glib
>  > >>>>precludes the use of a formalized schema.  My proposal is that
>  > >>>>we
>  > >>>>write a schema
>  > >>>>definition and generate the glib C code from that schema.
>  > >>>>
>  > >>>>I agree that the _current_ xmlrpc API makes a pretty bad base
>  > >>>>from
>  > >>>>which to
>  > >>>>start a supportable API.  XMLRPC is a perfectly reasonable
>  > >>>>remote/wire protocol
>  > >>>>and I think we should continue using it as a base for the next
>  > >>>>generation API.
>  > >>>>Using a schema will ensure that the new API is well-structured.
>  > >>>There major problems with XML-RPC (and to some extent with REST
>  > >>>as
>  > >>>well) are high call overhead and no two way communication (push
>  > >>>events). Basing on XML-RPC means that we will never be able to
>  > >>>solve
>  > >>>these issues.
>  > >>I am not sure I am ready to conceed that XML-RPC is too slow for
>  > >>our
>  > >>needs.  Can
>  > >>you provide some more detail around this point and possibly
>  > >>suggest an
>  > >>alternative that has even lower overhead without sacrificing the
>  > >>ubiquity and
>  > >>usability of XML-RPC?  As far as the two-way communication point,
>  > >>what
>  > >>are the
>  > >>options besides AMQP/ZeroMQ?  Aren't these even worse from an
>  > >>overhead
>  > >>perspective than XML-RPC?  Regarding two-way communication: you
>  > >>can
>  > >>write AMQP
>  > >>brokers based on the C API and run one on each vdsm host.
>  > >> Assuming
>  > >>the C API
>  > >>supports events, what else would you need?
>  > >I personally think that using something like AMQP for inter-node
>  > >communication and engine - node would be optimal.  With a rest
>  > >interface
>  > >that just send messages though something like AMQP.
>  > 
>  > I would also not dismiss AMQP so soon
>  > we want a bug with more than a single listener at engine side
>  > (engine, history db, maybe event correlation service).
>  > collectd as a means for statistics already supports it as well.
>  > I'm for having REST as well, but not sure as main one for a
>  > consumer
>  > like ovirt engine.
>  
>  I agree that a message bus could be a very useful model of
>  communication between
>  ovirt-engine components and multiple vdsm instances.  But the
>  complexities and
>  dependencies of AMQP do not make it suitable for use as a low-level
>  API.  AMQP
>  will repel new adopters.  Why not establish a libvdsm that is more
>  minimalist
>  and can be easily used by everyone?  Then AMQP brokers can be built
>  on top of
>  the stable API with ease.  All AMQP should require of the low-level
>  API are
>  standard function calls and an events mechanism.
>  
>  > >
>  > >Thanks
>  > >Robert
>  > >>>>>The current XML-RPC API contains a lot of decencies and
>  > >>>>>inefficiencies and we
>  > >>>>>would like to retire it as soon as we possibly can. Engine
>  > >>>>>would
>  > >>>>>like us to
>  > >>>>>move to a message based API and 3rd parties want something
>  > >>>>>simple
>  > >>>>>like REST so
>  > >>>>>it looks like no one actually wants to use XML-RPC. Not even
>  > >>>>>us.
>  > >>>>I am proposing that AMQP brokers and REST APIs could be written
>  > >>>>against the
>  > >>>>public API.  In fact, they need not even live in the vdsm tree
>  > >>>>anymore if that
>  > >>>>is what we choose.  Core vdsm would only be responsible for
>  > >>>>providing
>  > >>>>libvdsm
>  > >>>>and whatever language bindings we want to support.
>  > >>>If we take the libvdsm route, the only reason to even have a
>  > >>>REST
>  > >>>bridge is only to support OSes other then Linux which is
>  > >>>something
>  > >>>I'm not sure we care about at the moment.
>  > >>That might be true regarding the current in-tree implementation.
>  > >>However, I can
>  > >>almost guarantee that someone wanting to write a web GUI on top
>  > >>of
>  > >>standalone
>  > >>vdsm would want a REST API to talk to.  But libvdsm makes this
>  > >>use
>  > >>case of no
>  > >>concern to the core vdsm developers.
>  > >>
>  > >>>>>I do think that having C supportability in our API is a good
>  > >>>>>idea,
>  > >>>>>but the
>  > >>>>>current API should not be used as the base.
>  > >>>>Let's _start_ with a schema document that describes today's
> API
>  > >>>>and
>  > >>>>then clean
>  > >>>>it up.  I think that will work better than starting from
>  > >>>>scratch.
>  > >>>>  Once my
>  > >>>>schema is written I will post it and we can 'patch' it as a
>  > >>>>community
>  > >>>>until we
>  > >>>>arrive at a 1.0 version we are all happy with.
>  > >>>+1
>  > >>Ok.  Redoubling my efforts to get this done.  Describing the
>  > >>output of
>  > >>list(True) takes awhile :)
>  > >>
>  > >
>  > >
>  > >_______________________________________________
>  > >vdsm-devel mailing list
>  > &gt;vdsm-devel(a)lists.fedorahosted.org
>  > >https://fedorahosted.org/mailman/listinfo/vdsm-devel
>  > 
>  > 
>  > _______________________________________________
>  > vdsm-devel mailing list
>  > vdsm-devel(a)lists.fedorahosted.org
>  > https://fedorahosted.org/mailman/listinfo/vdsm-devel
>  
>  --
>  Adam Litke <agl(a)us.ibm.com&gt;
>  IBM Linux Technology Center
>  
>  _______________________________________________
>  vdsm-devel mailing list
>  vdsm-devel(a)lists.fedorahosted.org
>  https://fedorahosted.org/mailman/listinfo/vdsm-devel
>

On 07/09/2012 05:56 PM, Saggi Mizrahi wrote:
>  I don't think AMQP is a good low level supported protocol as
> it's a very complex protocol to set up and support.
>  Also brokers are known to have their differences in standard implementation which means
> supporting them all is a mess.
>
>  It looks like the most accepted route is the libvirt route of having a c library
> abstracting away client server communication and having more advanced consumers build
> protocol specific bridges that may have different support standards.
>
>  On a more personal note, I think brokerless messaging is the way to go in ovirt because,
> unlike traditional clustering, worker nodes are not interchangeable so direct
> communication is the way to go, rendering brokers pretty much useless. 
but brokerless doesn't let multiple consumers which a bus provides?

>  ----- Original Message -----
> > From: "Adam Litke" <agl(a)us.ibm.com&gt;
> > To: "Itamar Heim" <iheim(a)redhat.com&gt;
> > Cc: vdsm-devel(a)lists.fedorahosted.org
> > Sent: Monday, July 9, 2012 9:56:17 AM
> > Subject: Re: [vdsm] [RFC] An alternative way to provide a supported interface --
> libvdsm
> >
> > On Fri, Jul 06, 2012 at 03:53:08PM +0300, Itamar Heim wrote:
> >> On 07/06/2012 01:15 AM, Robert Middleswarth wrote:
> >>> On 07/05/2012 04:45 PM, Adam Litke wrote:
> >>>> On Thu, Jul 05, 2012 at 03:47:42PM -0400, Saggi Mizrahi wrote:
> >>>>>
> >>>>> ----- Original Message -----
> >>>>>> From: "Adam Litke" <agl(a)us.ibm.com&gt;
> >>>>>> To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt;
> >>>>>> Cc: "Anthony Liguori" <anthony(a)codemonkey.ws&gt;,
> "VDSM Project
> >>>>>> Development" <vdsm-devel(a)lists.fedorahosted.org&gt;
> >>>>>> Sent: Thursday, July 5, 2012 2:34:50 PM
> >>>>>> Subject: Re: [RFC] An alternative way to provide a supported
> >>>>>> interface -- libvdsm
> >>>>>>
> >>>>>> On Wed, Jun 27, 2012 at 02:50:02PM -0400, Saggi Mizrahi wrote:
> >>>>>>> The idea of having a supported C API was something I was
> >>>>>>> thinking
> >>>>>>> about doing
> >>>>>>> (But I'd rather use gobject introspection and not schema
> >>>>>>> generation) But the
> >>>>>>> problem is not having a C API is using the current XML RPC
> API
> >>>>>>> as
> >>>>>>> it's base
> >>>>>> I want to disect this a bit to find out exactly where there
> >>>>>> might be
> >>>>>> agreement
> >>>>>> and disagreement.
> >>>>>>
> >>>>>> C API is a good thing to implement - Agreed.
> >>>>>>
> >>>>>> I also want to use gobject introspection but I don't agree
> that
> >>>>>> using
> >>>>>> glib
> >>>>>> precludes the use of a formalized schema.  My proposal is that
> >>>>>> we
> >>>>>> write a schema
> >>>>>> definition and generate the glib C code from that schema.
> >>>>>>
> >>>>>> I agree that the _current_ xmlrpc API makes a pretty bad base
> >>>>>> from
> >>>>>> which to
> >>>>>> start a supportable API.  XMLRPC is a perfectly reasonable
> >>>>>> remote/wire protocol
> >>>>>> and I think we should continue using it as a base for the next
> >>>>>> generation API.
> >>>>>> Using a schema will ensure that the new API is well-structured.
> >>>>> There major problems with XML-RPC (and to some extent with REST
> >>>>> as
> >>>>> well) are high call overhead and no two way communication (push
> >>>>> events). Basing on XML-RPC means that we will never be able to
> >>>>> solve
> >>>>> these issues.
> >>>> I am not sure I am ready to conceed that XML-RPC is too slow for
> >>>> our
> >>>> needs.  Can
> >>>> you provide some more detail around this point and possibly
> >>>> suggest an
> >>>> alternative that has even lower overhead without sacrificing the
> >>>> ubiquity and
> >>>> usability of XML-RPC?  As far as the two-way communication point,
> >>>> what
> >>>> are the
> >>>> options besides AMQP/ZeroMQ?  Aren't these even worse from an
> >>>> overhead
> >>>> perspective than XML-RPC?  Regarding two-way communication: you
> >>>> can
> >>>> write AMQP
> >>>> brokers based on the C API and run one on each vdsm host.
> >>>> Assuming
> >>>> the C API
> >>>> supports events, what else would you need?
> >>> I personally think that using something like AMQP for inter-node
> >>> communication and engine - node would be optimal.  With a rest
> >>> interface
> >>> that just send messages though something like AMQP.
> >>
> >> I would also not dismiss AMQP so soon
> >> we want a bug with more than a single listener at engine side
> >> (engine, history db, maybe event correlation service).
> >> collectd as a means for statistics already supports it as well.
> >> I'm for having REST as well, but not sure as main one for a
> >> consumer
> >> like ovirt engine.
> >
> > I agree that a message bus could be a very useful model of
> > communication between
> > ovirt-engine components and multiple vdsm instances.  But the
> > complexities and
> > dependencies of AMQP do not make it suitable for use as a low-level
> > API.  AMQP
> > will repel new adopters.  Why not establish a libvdsm that is more
> > minimalist
> > and can be easily used by everyone?  Then AMQP brokers can be built
> > on top of
> > the stable API with ease.  All AMQP should require of the low-level
> > API are
> > standard function calls and an events mechanism.
> >
> >>>
> >>> Thanks
> >>> Robert
> >>>>>>> The current XML-RPC API contains a lot of decencies and
> >>>>>>> inefficiencies and we
> >>>>>>> would like to retire it as soon as we possibly can. Engine
> >>>>>>> would
> >>>>>>> like us to
> >>>>>>> move to a message based API and 3rd parties want something
> >>>>>>> simple
> >>>>>>> like REST so
> >>>>>>> it looks like no one actually wants to use XML-RPC. Not even
> >>>>>>> us.
> >>>>>> I am proposing that AMQP brokers and REST APIs could be written
> >>>>>> against the
> >>>>>> public API.  In fact, they need not even live in the vdsm tree
> >>>>>> anymore if that
> >>>>>> is what we choose.  Core vdsm would only be responsible for
> >>>>>> providing
> >>>>>> libvdsm
> >>>>>> and whatever language bindings we want to support.
> >>>>> If we take the libvdsm route, the only reason to even have a
> >>>>> REST
> >>>>> bridge is only to support OSes other then Linux which is
> >>>>> something
> >>>>> I'm not sure we care about at the moment.
> >>>> That might be true regarding the current in-tree implementation.
> >>>> However, I can
> >>>> almost guarantee that someone wanting to write a web GUI on top
> >>>> of
> >>>> standalone
> >>>> vdsm would want a REST API to talk to.  But libvdsm makes this
> >>>> use
> >>>> case of no
> >>>> concern to the core vdsm developers.
> >>>>
> >>>>>>> I do think that having C supportability in our API is a good
> >>>>>>> idea,
> >>>>>>> but the
> >>>>>>> current API should not be used as the base.
> >>>>>> Let's _start_ with a schema document that describes
> today's API
> >>>>>> and
> >>>>>> then clean
> >>>>>> it up.  I think that will work better than starting from
> >>>>>> scratch.
> >>>>>>   Once my
> >>>>>> schema is written I will post it and we can 'patch' it as
> a
> >>>>>> community
> >>>>>> until we
> >>>>>> arrive at a 1.0 version we are all happy with.
> >>>>> +1
> >>>> Ok.  Redoubling my efforts to get this done.  Describing the
> >>>> output of
> >>>> list(True) takes awhile :)
> >>>>
> >>>
> >>>
> >>> _______________________________________________
> >>> vdsm-devel mailing list
> >>> vdsm-devel(a)lists.fedorahosted.org
> >>> https://fedorahosted.org/mailman/listinfo/vdsm-devel
> >>
> >>
> >> _______________________________________________
> >> vdsm-devel mailing list
> >> vdsm-devel(a)lists.fedorahosted.org
> >> https://fedorahosted.org/mailman/listinfo/vdsm-devel
> >
> > --
> > Adam Litke <agl(a)us.ibm.com&gt;
> > IBM Linux Technology Center
> >
> > _______________________________________________
> > vdsm-devel mailing list
> > vdsm-devel(a)lists.fedorahosted.org
> > https://fedorahosted.org/mailman/listinfo/vdsm-devel
> >

----- Original Message -----
>  From: "Itamar Heim" <iheim(a)redhat.com&gt;
>  To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt;
>  Cc: "Adam Litke" <agl(a)us.ibm.com&gt;, vdsm-devel(a)lists.fedorahosted.org
>  Sent: Monday, July 9, 2012 11:03:43 AM
>  Subject: Re: [vdsm] [RFC] An alternative way to provide a supported interface -- libvdsm
>  
>  On 07/09/2012 05:56 PM, Saggi Mizrahi wrote:
>  > I don't think AMQP is a good low level supported protocol as it's a
>  > very complex protocol to set up and support.
>  > Also brokers are known to have their differences in standard
>  > implementation which means supporting them all is a mess.
>  >
>  > It looks like the most accepted route is the libvirt route of
>  > having a c library abstracting away client server communication
>  > and having more advanced consumers build protocol specific bridges
>  > that may have different support standards.
>  >
>  > On a more personal note, I think brokerless messaging is the way to
>  > go in ovirt because, unlike traditional clustering, worker nodes
>  > are not interchangeable so direct communication is the way to go,
>  > rendering brokers pretty much useless.
>  
>  but brokerless doesn't let multiple consumers which a bus provides? 
All
consumers can connect to the host and *some* events can be broadcasted to all connected
clients.

The real question is weather you want to depend on AMQP's routing \ message storing
Also, if you find it preferable to have a centralized host (single point of failure) to
get all events from all hosts for the price of  some clients (I assume read only clients)
not needing to know the locations of all worker nodes.
But IMHO we already have something like that, it's called the ovirt-engine, and it
could send aggregated events about the cluster (maybe with some extra enginy data).

The question is what does mandating a broker gives us something that an "AMQP
bridge" wouldn't.
The only thing I can think of is vdsm can assume unmoderated vdsm to vdsm communication
bypassing the engine.
This means that VDSM can have some clustered behavior that requires no engine
intervention.
Further more, the engine can send a request and let the nodes decide who is performing the
operation among themselves.

Essentially:

[  engine  ]          [  engine  ]
   |     |      VS          |
[vdsm][vdsm]          [  broker  ]
                         |     |
                      [vdsm][vdsm]

*All links are two way links

This has dire consequences on API usability and supportability. So we need to converge on
that.

There needs to be a good reason why the aforementioned logic code can't sit on a
another ovirt specific entity (lets call it ovirt-dynamo) that uses VDSM's supported
API but it's own APIs (or more likely messaging algorithms) are unsupported.
                                  
 [            engine           ]
   |            |            |
   |      [   broker   ]     |
   |        |        |       |
[vdsm]-[dynamo] : [dynamo]-[vdsm]
    Host A      :      Host B

*All links are two way links
>  
>  >
>  > ----- Original Message -----
>  >> From: "Adam Litke" <agl(a)us.ibm.com&gt;
>  >> To: "Itamar Heim" <iheim(a)redhat.com&gt;
>  >> Cc: vdsm-devel(a)lists.fedorahosted.org
>  >> Sent: Monday, July 9, 2012 9:56:17 AM
>  >> Subject: Re: [vdsm] [RFC] An alternative way to provide a
>  >> supported interface -- libvdsm
>  >>
>  >> On Fri, Jul 06, 2012 at 03:53:08PM +0300, Itamar Heim wrote:
>  >>> On 07/06/2012 01:15 AM, Robert Middleswarth wrote:
>  >>>> On 07/05/2012 04:45 PM, Adam Litke wrote:
>  >>>>> On Thu, Jul 05, 2012 at 03:47:42PM -0400, Saggi Mizrahi wrote:
>  >>>>>>
>  >>>>>> ----- Original Message -----
>  >>>>>>> From: "Adam Litke" <agl(a)us.ibm.com&gt;
>  >>>>>>> To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt;
>  >>>>>>> Cc: "Anthony Liguori"
> <anthony(a)codemonkey.ws&gt;, "VDSM Project
>  >>>>>>> Development" <vdsm-devel(a)lists.fedorahosted.org&gt;
>  >>>>>>> Sent: Thursday, July 5, 2012 2:34:50 PM
>  >>>>>>> Subject: Re: [RFC] An alternative way to provide a
> supported
>  >>>>>>> interface -- libvdsm
>  >>>>>>>
>  >>>>>>> On Wed, Jun 27, 2012 at 02:50:02PM -0400, Saggi Mizrahi
>  >>>>>>> wrote:
>  >>>>>>>> The idea of having a supported C API was something I
> was
>  >>>>>>>> thinking
>  >>>>>>>> about doing
>  >>>>>>>> (But I'd rather use gobject introspection and not
> schema
>  >>>>>>>> generation) But the
>  >>>>>>>> problem is not having a C API is using the current XML
> RPC
>  >>>>>>>> API
>  >>>>>>>> as
>  >>>>>>>> it's base
>  >>>>>>> I want to disect this a bit to find out exactly where there
>  >>>>>>> might be
>  >>>>>>> agreement
>  >>>>>>> and disagreement.
>  >>>>>>>
>  >>>>>>> C API is a good thing to implement - Agreed.
>  >>>>>>>
>  >>>>>>> I also want to use gobject introspection but I don't
> agree
>  >>>>>>> that
>  >>>>>>> using
>  >>>>>>> glib
>  >>>>>>> precludes the use of a formalized schema.  My proposal is
>  >>>>>>> that
>  >>>>>>> we
>  >>>>>>> write a schema
>  >>>>>>> definition and generate the glib C code from that schema.
>  >>>>>>>
>  >>>>>>> I agree that the _current_ xmlrpc API makes a pretty bad
> base
>  >>>>>>> from
>  >>>>>>> which to
>  >>>>>>> start a supportable API.  XMLRPC is a perfectly reasonable
>  >>>>>>> remote/wire protocol
>  >>>>>>> and I think we should continue using it as a base for the
>  >>>>>>> next
>  >>>>>>> generation API.
>  >>>>>>> Using a schema will ensure that the new API is
>  >>>>>>> well-structured.
>  >>>>>> There major problems with XML-RPC (and to some extent with
>  >>>>>> REST
>  >>>>>> as
>  >>>>>> well) are high call overhead and no two way communication
>  >>>>>> (push
>  >>>>>> events). Basing on XML-RPC means that we will never be able to
>  >>>>>> solve
>  >>>>>> these issues.
>  >>>>> I am not sure I am ready to conceed that XML-RPC is too slow
>  >>>>> for
>  >>>>> our
>  >>>>> needs.  Can
>  >>>>> you provide some more detail around this point and possibly
>  >>>>> suggest an
>  >>>>> alternative that has even lower overhead without sacrificing
>  >>>>> the
>  >>>>> ubiquity and
>  >>>>> usability of XML-RPC?  As far as the two-way communication
>  >>>>> point,
>  >>>>> what
>  >>>>> are the
>  >>>>> options besides AMQP/ZeroMQ?  Aren't these even worse from an
>  >>>>> overhead
>  >>>>> perspective than XML-RPC?  Regarding two-way communication: you
>  >>>>> can
>  >>>>> write AMQP
>  >>>>> brokers based on the C API and run one on each vdsm host.
>  >>>>> Assuming
>  >>>>> the C API
>  >>>>> supports events, what else would you need?
>  >>>> I personally think that using something like AMQP for inter-node
>  >>>> communication and engine - node would be optimal.  With a rest
>  >>>> interface
>  >>>> that just send messages though something like AMQP.
>  >>>
>  >>> I would also not dismiss AMQP so soon
>  >>> we want a bug with more than a single listener at engine side
>  >>> (engine, history db, maybe event correlation service).
>  >>> collectd as a means for statistics already supports it as well.
>  >>> I'm for having REST as well, but not sure as main one for a
>  >>> consumer
>  >>> like ovirt engine.
>  >>
>  >> I agree that a message bus could be a very useful model of
>  >> communication between
>  >> ovirt-engine components and multiple vdsm instances.  But the
>  >> complexities and
>  >> dependencies of AMQP do not make it suitable for use as a
>  >> low-level
>  >> API.  AMQP
>  >> will repel new adopters.  Why not establish a libvdsm that is more
>  >> minimalist
>  >> and can be easily used by everyone?  Then AMQP brokers can be
>  >> built
>  >> on top of
>  >> the stable API with ease.  All AMQP should require of the
>  >> low-level
>  >> API are
>  >> standard function calls and an events mechanism.
>  >>
>  >>>>
>  >>>> Thanks
>  >>>> Robert
>  >>>>>>>> The current XML-RPC API contains a lot of decencies and
>  >>>>>>>> inefficiencies and we
>  >>>>>>>> would like to retire it as soon as we possibly can.
> Engine
>  >>>>>>>> would
>  >>>>>>>> like us to
>  >>>>>>>> move to a message based API and 3rd parties want
> something
>  >>>>>>>> simple
>  >>>>>>>> like REST so
>  >>>>>>>> it looks like no one actually wants to use XML-RPC. Not
> even
>  >>>>>>>> us.
>  >>>>>>> I am proposing that AMQP brokers and REST APIs could be
>  >>>>>>> written
>  >>>>>>> against the
>  >>>>>>> public API.  In fact, they need not even live in the vdsm
>  >>>>>>> tree
>  >>>>>>> anymore if that
>  >>>>>>> is what we choose.  Core vdsm would only be responsible for
>  >>>>>>> providing
>  >>>>>>> libvdsm
>  >>>>>>> and whatever language bindings we want to support.
>  >>>>>> If we take the libvdsm route, the only reason to even have a
>  >>>>>> REST
>  >>>>>> bridge is only to support OSes other then Linux which is
>  >>>>>> something
>  >>>>>> I'm not sure we care about at the moment.
>  >>>>> That might be true regarding the current in-tree
>  >>>>> implementation.
>  >>>>> However, I can
>  >>>>> almost guarantee that someone wanting to write a web GUI on top
>  >>>>> of
>  >>>>> standalone
>  >>>>> vdsm would want a REST API to talk to.  But libvdsm makes this
>  >>>>> use
>  >>>>> case of no
>  >>>>> concern to the core vdsm developers.
>  >>>>>
>  >>>>>>>> I do think that having C supportability in our API is a
> good
>  >>>>>>>> idea,
>  >>>>>>>> but the
>  >>>>>>>> current API should not be used as the base.
>  >>>>>>> Let's _start_ with a schema document that describes
> today's
>  >>>>>>> API
>  >>>>>>> and
>  >>>>>>> then clean
>  >>>>>>> it up.  I think that will work better than starting from
>  >>>>>>> scratch.
>  >>>>>>>   Once my
>  >>>>>>> schema is written I will post it and we can 'patch'
> it as a
>  >>>>>>> community
>  >>>>>>> until we
>  >>>>>>> arrive at a 1.0 version we are all happy with.
>  >>>>>> +1
>  >>>>> Ok.  Redoubling my efforts to get this done.  Describing the
>  >>>>> output of
>  >>>>> list(True) takes awhile :)
>  >>>>>
>  >>>>
>  >>>>
>  >>>> _______________________________________________
>  >>>> vdsm-devel mailing list
>  >>>> vdsm-devel(a)lists.fedorahosted.org
>  >>>> https://fedorahosted.org/mailman/listinfo/vdsm-devel
>  >>>
>  >>>
>  >>> _______________________________________________
>  >>> vdsm-devel mailing list
>  >>> vdsm-devel(a)lists.fedorahosted.org
>  >>> https://fedorahosted.org/mailman/listinfo/vdsm-devel
>  >>
>  >> --
>  >> Adam Litke <agl(a)us.ibm.com&gt;
>  >> IBM Linux Technology Center
>  >>
>  >> _______________________________________________
>  >> vdsm-devel mailing list
>  >> vdsm-devel(a)lists.fedorahosted.org
>  >> https://fedorahosted.org/mailman/listinfo/vdsm-devel
>  >>
>  
>  
>

On 07/09/2012 09:52 PM, Saggi Mizrahi wrote:
>  ----- Original Message -----
> > From: "Itamar Heim" <iheim(a)redhat.com&gt;
> > To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt;
> > Cc: "Adam Litke" <agl(a)us.ibm.com&gt;, vdsm-devel(a)lists.fedorahosted.org
> > Sent: Monday, July 9, 2012 11:03:43 AM
> > Subject: Re: [vdsm] [RFC] An alternative way to provide a supported interface --
> libvdsm
> >
> > On 07/09/2012 05:56 PM, Saggi Mizrahi wrote:
> >> I don't think AMQP is a good low level supported protocol as it's a
> >> very complex protocol to set up and support.
> >> Also brokers are known to have their differences in standard
> >> implementation which means supporting them all is a mess.
> >>
> >> It looks like the most accepted route is the libvirt route of
> >> having a c library abstracting away client server communication
> >> and having more advanced consumers build protocol specific bridges
> >> that may have different support standards.
> >>
> >> On a more personal note, I think brokerless messaging is the way to
> >> go in ovirt because, unlike traditional clustering, worker nodes
> >> are not interchangeable so direct communication is the way to go,
> >> rendering brokers pretty much useless.
> >
> > but brokerless doesn't let multiple consumers which a bus provides?
>  All consumers can connect to the host and *some* events can be broadcasted to all
> connected clients.
>
>  The real question is weather you want to depend on AMQP's routing \ message storing
>  Also, if you find it preferable to have a centralized host (single point of failure) to
> get all events from all hosts for the price of  some clients (I assume read only clients)
> not needing to know the locations of all worker nodes.
>  But IMHO we already have something like that, it's called the ovirt-engine, and it
> could send aggregated events about the cluster (maybe with some extra enginy data).
>
>  The question is what does mandating a broker gives us something that an "AMQP
> bridge" wouldn't.
>  The only thing I can think of is vdsm can assume unmoderated vdsm to vdsm communication
> bypassing the engine.
>  This means that VDSM can have some clustered behavior that requires no engine
> intervention.
>  Further more, the engine can send a request and let the nodes decide who is performing
> the operation among themselves.
>
>  Essentially:
>
>  [  engine  ]          [  engine  ]
>      |     |      VS          |
>  [vdsm][vdsm]          [  broker  ]
>                            |     |
>                         [vdsm][vdsm]
>
>  *All links are two way links
>
>  This has dire consequences on API usability and supportability. So we need to converge on
> that.
>
>  There needs to be a good reason why the aforementioned logic code can't sit on a
> another ovirt specific entity (lets call it ovirt-dynamo) that uses VDSM's supported
> API but it's own APIs (or more likely messaging algorithms) are unsupported.
>
>    [            engine           ]
>      |            |            |
>      |      [   broker   ]     |
>      |        |        |       |
>  [vdsm]-[dynamo] : [dynamo]-[vdsm]
>       Host A      :      Host B
>
>  *All links are two way links 
1. we have engine today 'in the path' to the history db. but it makes no 
sense for engine to be aware of each statistic we want to keep in the 
history db.
same would be for an event/stats correlation service.
they don't need to depend on each other for availability/redundancy.

2. we are already looking at quantum integration, which is doing engine 
to nodes communication via amqp.

3. with somewhat of a forward looking - moving some scheduling logic 
"down to vdsm" will probably mean we'll want one of the nodes to listen 
to statistics and state from the other nodes.

to all of these, setting up a bus which allows multiple peer listeners 
seems more robust

On 07/17/2012 01:19 AM, Itamar Heim wrote:
>  On 07/09/2012 09:52 PM, Saggi Mizrahi wrote:
> >
> >
> > ----- Original Message -----
> >> From: "Itamar Heim" <iheim(a)redhat.com&gt;
> >> To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt;
> >> Cc: "Adam Litke" <agl(a)us.ibm.com&gt;,
> vdsm-devel(a)lists.fedorahosted.org
> >> Sent: Monday, July 9, 2012 11:03:43 AM
> >> Subject: Re: [vdsm] [RFC] An alternative way to provide a supported
> >> interface -- libvdsm
> >>
> >> On 07/09/2012 05:56 PM, Saggi Mizrahi wrote:
> >>> I don't think AMQP is a good low level supported protocol as it's a
> >>> very complex protocol to set up and support.
> >>> Also brokers are known to have their differences in standard
> >>> implementation which means supporting them all is a mess.
> >>>
> >>> It looks like the most accepted route is the libvirt route of
> >>> having a c library abstracting away client server communication
> >>> and having more advanced consumers build protocol specific bridges
> >>> that may have different support standards.
> >>>
> >>> On a more personal note, I think brokerless messaging is the way to
> >>> go in ovirt because, unlike traditional clustering, worker nodes
> >>> are not interchangeable so direct communication is the way to go,
> >>> rendering brokers pretty much useless.
> >>
> >> but brokerless doesn't let multiple consumers which a bus provides?
> > All consumers can connect to the host and *some* events can be
> > broadcasted to all connected clients.
> >
> > The real question is weather you want to depend on AMQP's routing \
> > message storing
> > Also, if you find it preferable to have a centralized host (single
> > point of failure) to get all events from all hosts for the price of
> > some clients (I assume read only clients) not needing to know the
> > locations of all worker nodes.
> > But IMHO we already have something like that, it's called the
> > ovirt-engine, and it could send aggregated events about the cluster
> > (maybe with some extra enginy data).
> >
> > The question is what does mandating a broker gives us something that
> > an "AMQP bridge" wouldn't.
> > The only thing I can think of is vdsm can assume unmoderated vdsm to
> > vdsm communication bypassing the engine.
> > This means that VDSM can have some clustered behavior that requires no
> > engine intervention.
> > Further more, the engine can send a request and let the nodes decide
> > who is performing the operation among themselves.
> >
> > Essentially:
> >
> > [  engine  ]          [  engine  ]
> >     |     |      VS          |
> > [vdsm][vdsm]          [  broker  ]
> >                           |     |
> >                        [vdsm][vdsm]
> >
> > *All links are two way links
> >
> > This has dire consequences on API usability and supportability. So we
> > need to converge on that.
> >
> > There needs to be a good reason why the aforementioned logic code
> > can't sit on a another ovirt specific entity (lets call it
> > ovirt-dynamo) that uses VDSM's supported API but it's own APIs (or
> > more likely messaging algorithms) are unsupported.
> >
> >   [            engine           ]
> >     |            |            |
> >     |      [   broker   ]     |
> >     |        |        |       |
> > [vdsm]-[dynamo] : [dynamo]-[vdsm]
> >      Host A      :      Host B
> >
> > *All links are two way links
>
>  1. we have engine today 'in the path' to the history db. but it makes no
>  sense for engine to be aware of each statistic we want to keep in the
>  history db.
>  same would be for an event/stats correlation service.
>  they don't need to depend on each other for availability/redundancy.
>
>  2. we are already looking at quantum integration, which is doing engine
>  to nodes communication via amqp.
>
>  3. with somewhat of a forward looking - moving some scheduling logic
>  "down to vdsm" will probably mean we'll want one of the nodes to listen
>  to statistics and state from the other nodes.
>
>  to all of these, setting up a bus which allows multiple peer listeners
>  seems more robust 

I'm still against developing a C level binding for amqp and rest support 
over a codebase which is in python.
rest and amqp allow for both local and remote bindings in any language.
C bindings should/could be a parallel implementation, but they seem like 
an unneeded overhead and complexity in the middle of the codebase.

On Thu, Jul 26, 2012 at 11:47:51AM +0300, Itamar Heim wrote:
>  On 07/17/2012 01:19 AM, Itamar Heim wrote:
>  >On 07/09/2012 09:52 PM, Saggi Mizrahi wrote:
>  >>
>  >>
>  >>----- Original Message -----
>  >>>From: "Itamar Heim" <iheim(a)redhat.com&gt;
>  >>>To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt;
>  >>>Cc: "Adam Litke" <agl(a)us.ibm.com&gt;,
> vdsm-devel(a)lists.fedorahosted.org
>  >>>Sent: Monday, July 9, 2012 11:03:43 AM
>  >>>Subject: Re: [vdsm] [RFC] An alternative way to provide a supported
>  >>>interface -- libvdsm
>  >>>
>  >>>On 07/09/2012 05:56 PM, Saggi Mizrahi wrote:
>  >>>>I don't think AMQP is a good low level supported protocol as it's
> a
>  >>>>very complex protocol to set up and support.
>  >>>>Also brokers are known to have their differences in standard
>  >>>>implementation which means supporting them all is a mess.
>  >>>>
>  >>>>It looks like the most accepted route is the libvirt route of
>  >>>>having a c library abstracting away client server communication
>  >>>>and having more advanced consumers build protocol specific bridges
>  >>>>that may have different support standards.
>  >>>>
>  >>>>On a more personal note, I think brokerless messaging is the way to
>  >>>>go in ovirt because, unlike traditional clustering, worker nodes
>  >>>>are not interchangeable so direct communication is the way to go,
>  >>>>rendering brokers pretty much useless.
>  >>>
>  >>>but brokerless doesn't let multiple consumers which a bus provides?
>  >>All consumers can connect to the host and *some* events can be
>  >>broadcasted to all connected clients.
>  >>
>  >>The real question is weather you want to depend on AMQP's routing \
>  >>message storing
>  >>Also, if you find it preferable to have a centralized host (single
>  >>point of failure) to get all events from all hosts for the price of
>  >>some clients (I assume read only clients) not needing to know the
>  >>locations of all worker nodes.
>  >>But IMHO we already have something like that, it's called the
>  >>ovirt-engine, and it could send aggregated events about the cluster
>  >>(maybe with some extra enginy data).
>  >>
>  >>The question is what does mandating a broker gives us something that
>  >>an "AMQP bridge" wouldn't.
>  >>The only thing I can think of is vdsm can assume unmoderated vdsm to
>  >>vdsm communication bypassing the engine.
>  >>This means that VDSM can have some clustered behavior that requires no
>  >>engine intervention.
>  >>Further more, the engine can send a request and let the nodes decide
>  >>who is performing the operation among themselves.
>  >>
>  >>Essentially:
>  >>
>  >>[  engine  ]          [  engine  ]
>  >>    |     |      VS          |
>  >>[vdsm][vdsm]          [  broker  ]
>  >>                          |     |
>  >>                       [vdsm][vdsm]
>  >>
>  >>*All links are two way links
>  >>
>  >>This has dire consequences on API usability and supportability. So we
>  >>need to converge on that.
>  >>
>  >>There needs to be a good reason why the aforementioned logic code
>  >>can't sit on a another ovirt specific entity (lets call it
>  >>ovirt-dynamo) that uses VDSM's supported API but it's own APIs (or
>  >>more likely messaging algorithms) are unsupported.
>  >>
>  >>  [            engine           ]
>  >>    |            |            |
>  >>    |      [   broker   ]     |
>  >>    |        |        |       |
>  >>[vdsm]-[dynamo] : [dynamo]-[vdsm]
>  >>     Host A      :      Host B
>  >>
>  >>*All links are two way links
>  >
>  >1. we have engine today 'in the path' to the history db. but it makes no
>  >sense for engine to be aware of each statistic we want to keep in the
>  >history db.
>  >same would be for an event/stats correlation service.
>  >they don't need to depend on each other for availability/redundancy.
>  >
>  >2. we are already looking at quantum integration, which is doing engine
>  >to nodes communication via amqp.
>  >
>  >3. with somewhat of a forward looking - moving some scheduling logic
>  >"down to vdsm" will probably mean we'll want one of the nodes to
> listen
>  >to statistics and state from the other nodes.
>  >
>  >to all of these, setting up a bus which allows multiple peer listeners
>  >seems more robust
>  
>  
>  I'm still against developing a C level binding for amqp and rest
>  support over a codebase which is in python.
>  rest and amqp allow for both local and remote bindings in any language.
>  C bindings should/could be a parallel implementation, but they seem
>  like an unneeded overhead and complexity in the middle of the
>  codebase. 
Sure, it's probably possible to bind a REST or AMQP API in other languages but I
don't think there is an automatic way of doing it.  That means having to keep up
with maintenance of each and every binding every time the API changes.  If we
look at libvirt, they will say this is a large source of pain that they have
recommended we avoid.

For the C/gobject approach, we write a single API schema file.  From that, we
automatically generate the C API and bindings.  Sure, the generation could be a
bit complex but much of it will be someone else's codebase (and one that is used
by lots of Gnome projects).

-- 
Adam Litke <agl(a)us.ibm.com&gt;
IBM Linux Technology Center

On 07/26/2012 04:33 PM, Adam Litke wrote:
>  On Thu, Jul 26, 2012 at 11:47:51AM +0300, Itamar Heim wrote:
> > On 07/17/2012 01:19 AM, Itamar Heim wrote:
> >> On 07/09/2012 09:52 PM, Saggi Mizrahi wrote:
> >>>
> >>>
> >>> ----- Original Message -----
> >>>> From: "Itamar Heim" <iheim(a)redhat.com&gt;
> >>>> To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt;
> >>>> Cc: "Adam Litke" <agl(a)us.ibm.com&gt;,
> vdsm-devel(a)lists.fedorahosted.org
> >>>> Sent: Monday, July 9, 2012 11:03:43 AM
> >>>> Subject: Re: [vdsm] [RFC] An alternative way to provide a supported
> >>>> interface -- libvdsm
> >>>>
> >>>> On 07/09/2012 05:56 PM, Saggi Mizrahi wrote:
> >>>>> I don't think AMQP is a good low level supported protocol as
> it's a
> >>>>> very complex protocol to set up and support.
> >>>>> Also brokers are known to have their differences in standard
> >>>>> implementation which means supporting them all is a mess.
> >>>>>
> >>>>> It looks like the most accepted route is the libvirt route of
> >>>>> having a c library abstracting away client server communication
> >>>>> and having more advanced consumers build protocol specific bridges
> >>>>> that may have different support standards.
> >>>>>
> >>>>> On a more personal note, I think brokerless messaging is the way to
> >>>>> go in ovirt because, unlike traditional clustering, worker nodes
> >>>>> are not interchangeable so direct communication is the way to go,
> >>>>> rendering brokers pretty much useless.
> >>>>
> >>>> but brokerless doesn't let multiple consumers which a bus provides?
> >>> All consumers can connect to the host and *some* events can be
> >>> broadcasted to all connected clients.
> >>>
> >>> The real question is weather you want to depend on AMQP's routing \
> >>> message storing
> >>> Also, if you find it preferable to have a centralized host (single
> >>> point of failure) to get all events from all hosts for the price of
> >>> some clients (I assume read only clients) not needing to know the
> >>> locations of all worker nodes.
> >>> But IMHO we already have something like that, it's called the
> >>> ovirt-engine, and it could send aggregated events about the cluster
> >>> (maybe with some extra enginy data).
> >>>
> >>> The question is what does mandating a broker gives us something that
> >>> an "AMQP bridge" wouldn't.
> >>> The only thing I can think of is vdsm can assume unmoderated vdsm to
> >>> vdsm communication bypassing the engine.
> >>> This means that VDSM can have some clustered behavior that requires no
> >>> engine intervention.
> >>> Further more, the engine can send a request and let the nodes decide
> >>> who is performing the operation among themselves.
> >>>
> >>> Essentially:
> >>>
> >>> [  engine  ]          [  engine  ]
> >>>     |     |      VS          |
> >>> [vdsm][vdsm]          [  broker  ]
> >>>                           |     |
> >>>                        [vdsm][vdsm]
> >>>
> >>> *All links are two way links
> >>>
> >>> This has dire consequences on API usability and supportability. So we
> >>> need to converge on that.
> >>>
> >>> There needs to be a good reason why the aforementioned logic code
> >>> can't sit on a another ovirt specific entity (lets call it
> >>> ovirt-dynamo) that uses VDSM's supported API but it's own APIs (or
> >>> more likely messaging algorithms) are unsupported.
> >>>
> >>>   [            engine           ]
> >>>     |            |            |
> >>>     |      [   broker   ]     |
> >>>     |        |        |       |
> >>> [vdsm]-[dynamo] : [dynamo]-[vdsm]
> >>>      Host A      :      Host B
> >>>
> >>> *All links are two way links
> >>
> >> 1. we have engine today 'in the path' to the history db. but it makes no
> >> sense for engine to be aware of each statistic we want to keep in the
> >> history db.
> >> same would be for an event/stats correlation service.
> >> they don't need to depend on each other for availability/redundancy.
> >>
> >> 2. we are already looking at quantum integration, which is doing engine
> >> to nodes communication via amqp.
> >>
> >> 3. with somewhat of a forward looking - moving some scheduling logic
> >> "down to vdsm" will probably mean we'll want one of the nodes to
> listen
> >> to statistics and state from the other nodes.
> >>
> >> to all of these, setting up a bus which allows multiple peer listeners
> >> seems more robust
> >
> >
> > I'm still against developing a C level binding for amqp and rest
> > support over a codebase which is in python.
> > rest and amqp allow for both local and remote bindings in any language.
> > C bindings should/could be a parallel implementation, but they seem
> > like an unneeded overhead and complexity in the middle of the
> > codebase.
>
>  Sure, it's probably possible to bind a REST or AMQP API in other languages but I
>  don't think there is an automatic way of doing it.  That means having to keep up
>  with maintenance of each and every binding every time the API changes.  If we
>  look at libvirt, they will say this is a large source of pain that they have
>  recommended we avoid. 
we'd need this on top of the C api as well - but it would probably be 
simpler doing it over the python api, rather than the C one.

>  For the C/gobject approach, we write a single API schema file.  From that, we
>  automatically generate the C API and bindings.  Sure, the generation could be a
>  bit complex but much of it will be someone else's codebase (and one that is used
>  by lots of Gnome projects).
>  
that's a critical part of the question for the C api - what is needed 
for adding a verb / parameter.

I'm not against having c bindings. what i don't understand is why to put 
them in the 'middle', rather than as peers to the others on top of the 
python api classes.

I'll clarify.

Using gobject introspection means that you bindings are non existent for dynamic languages
and auto-generated (JGIR for java) for strongly typed languages.

This means that you don't maintain the bindings yourself, you maintain a schema that
generates gobject code and it, in turn, generates the binding.

You get integration for events, properties, and objects for free.

As for binary compatibility. The way to would work is that all objects internally will be
similar to python. All local object will hav an internal map and the getters and setters
will modify these in a type safe manner. This means that you can expend the API and still
keep BC.

As for REST and AMQP.

REST is pure request response, has high latency, and is pure CRUD which is very limiting.
AMQP (or any brokered messaging) needs a lot of setting up and makes no sense in a
supported API because it implies VDSM a cognizant entity in the cluster which makes it
less of an API slave and has more BL that might change and be engine specific.

The reason we are choosing to take the libvdsm approach with bridges is because remote
binding usually let themselves to some underlying design philosophy. This means ALL
bindings suffer.

I've uploaded a small POC[1] to show how the gobject binding might work.

The way it works is that you always have access to the send_command verb and the
BaseObject
This means that if you want to use something that you don't currently support you can
get\set\query the objects you received.
But, most objects will have types and their own type safe accessors to make everything
easier.

You could see that the python bindings "just work" because we use GObject no
additional code has to be written to load libvdsm.

http://gerrit.ovirt.org/6659

----- Original Message -----
>  From: "Itamar Heim" <iheim(a)redhat.com&gt;
>  To: "Adam Litke" <agl(a)us.ibm.com&gt;
>  Cc: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt;,
> vdsm-devel(a)lists.fedorahosted.org
>  Sent: Thursday, July 26, 2012 9:39:16 AM
>  Subject: Re: [vdsm] [RFC] An alternative way to provide a supported interface -- libvdsm
>  
>  On 07/26/2012 04:33 PM, Adam Litke wrote:
>  > On Thu, Jul 26, 2012 at 11:47:51AM +0300, Itamar Heim wrote:
>  >> On 07/17/2012 01:19 AM, Itamar Heim wrote:
>  >>> On 07/09/2012 09:52 PM, Saggi Mizrahi wrote:
>  >>>>
>  >>>>
>  >>>> ----- Original Message -----
>  >>>>> From: "Itamar Heim" <iheim(a)redhat.com&gt;
>  >>>>> To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt;
>  >>>>> Cc: "Adam Litke" <agl(a)us.ibm.com&gt;,
>  >>>>> vdsm-devel(a)lists.fedorahosted.org
>  >>>>> Sent: Monday, July 9, 2012 11:03:43 AM
>  >>>>> Subject: Re: [vdsm] [RFC] An alternative way to provide a
>  >>>>> supported
>  >>>>> interface -- libvdsm
>  >>>>>
>  >>>>> On 07/09/2012 05:56 PM, Saggi Mizrahi wrote:
>  >>>>>> I don't think AMQP is a good low level supported protocol
> as
>  >>>>>> it's a
>  >>>>>> very complex protocol to set up and support.
>  >>>>>> Also brokers are known to have their differences in standard
>  >>>>>> implementation which means supporting them all is a mess.
>  >>>>>>
>  >>>>>> It looks like the most accepted route is the libvirt route of
>  >>>>>> having a c library abstracting away client server
>  >>>>>> communication
>  >>>>>> and having more advanced consumers build protocol specific
>  >>>>>> bridges
>  >>>>>> that may have different support standards.
>  >>>>>>
>  >>>>>> On a more personal note, I think brokerless messaging is the
>  >>>>>> way to
>  >>>>>> go in ovirt because, unlike traditional clustering, worker
>  >>>>>> nodes
>  >>>>>> are not interchangeable so direct communication is the way to
>  >>>>>> go,
>  >>>>>> rendering brokers pretty much useless.
>  >>>>>
>  >>>>> but brokerless doesn't let multiple consumers which a bus
>  >>>>> provides?
>  >>>> All consumers can connect to the host and *some* events can be
>  >>>> broadcasted to all connected clients.
>  >>>>
>  >>>> The real question is weather you want to depend on AMQP's
>  >>>> routing \
>  >>>> message storing
>  >>>> Also, if you find it preferable to have a centralized host
>  >>>> (single
>  >>>> point of failure) to get all events from all hosts for the price
>  >>>> of
>  >>>> some clients (I assume read only clients) not needing to know
>  >>>> the
>  >>>> locations of all worker nodes.
>  >>>> But IMHO we already have something like that, it's called the
>  >>>> ovirt-engine, and it could send aggregated events about the
>  >>>> cluster
>  >>>> (maybe with some extra enginy data).
>  >>>>
>  >>>> The question is what does mandating a broker gives us something
>  >>>> that
>  >>>> an "AMQP bridge" wouldn't.
>  >>>> The only thing I can think of is vdsm can assume unmoderated
>  >>>> vdsm to
>  >>>> vdsm communication bypassing the engine.
>  >>>> This means that VDSM can have some clustered behavior that
>  >>>> requires no
>  >>>> engine intervention.
>  >>>> Further more, the engine can send a request and let the nodes
>  >>>> decide
>  >>>> who is performing the operation among themselves.
>  >>>>
>  >>>> Essentially:
>  >>>>
>  >>>> [  engine  ]          [  engine  ]
>  >>>>     |     |      VS          |
>  >>>> [vdsm][vdsm]          [  broker  ]
>  >>>>                           |     |
>  >>>>                        [vdsm][vdsm]
>  >>>>
>  >>>> *All links are two way links
>  >>>>
>  >>>> This has dire consequences on API usability and supportability.
>  >>>> So we
>  >>>> need to converge on that.
>  >>>>
>  >>>> There needs to be a good reason why the aforementioned logic
>  >>>> code
>  >>>> can't sit on a another ovirt specific entity (lets call it
>  >>>> ovirt-dynamo) that uses VDSM's supported API but it's own APIs
>  >>>> (or
>  >>>> more likely messaging algorithms) are unsupported.
>  >>>>
>  >>>>   [            engine           ]
>  >>>>     |            |            |
>  >>>>     |      [   broker   ]     |
>  >>>>     |        |        |       |
>  >>>> [vdsm]-[dynamo] : [dynamo]-[vdsm]
>  >>>>      Host A      :      Host B
>  >>>>
>  >>>> *All links are two way links
>  >>>
>  >>> 1. we have engine today 'in the path' to the history db. but it
>  >>> makes no
>  >>> sense for engine to be aware of each statistic we want to keep in
>  >>> the
>  >>> history db.
>  >>> same would be for an event/stats correlation service.
>  >>> they don't need to depend on each other for
>  >>> availability/redundancy.
>  >>>
>  >>> 2. we are already looking at quantum integration, which is doing
>  >>> engine
>  >>> to nodes communication via amqp.
>  >>>
>  >>> 3. with somewhat of a forward looking - moving some scheduling
>  >>> logic
>  >>> "down to vdsm" will probably mean we'll want one of the nodes
> to
>  >>> listen
>  >>> to statistics and state from the other nodes.
>  >>>
>  >>> to all of these, setting up a bus which allows multiple peer
>  >>> listeners
>  >>> seems more robust
>  >>
>  >>
>  >> I'm still against developing a C level binding for amqp and rest
>  >> support over a codebase which is in python.
>  >> rest and amqp allow for both local and remote bindings in any
>  >> language.
>  >> C bindings should/could be a parallel implementation, but they
>  >> seem
>  >> like an unneeded overhead and complexity in the middle of the
>  >> codebase.
>  >
>  > Sure, it's probably possible to bind a REST or AMQP API in other
>  > languages but I
>  > don't think there is an automatic way of doing it.  That means
>  > having to keep up
>  > with maintenance of each and every binding every time the API
>  > changes.  If we
>  > look at libvirt, they will say this is a large source of pain that
>  > they have
>  > recommended we avoid.
>  
>  we'd need this on top of the C api as well - but it would probably be
>  simpler doing it over the python api, rather than the C one.
>  
>  >
>  > For the C/gobject approach, we write a single API schema file.
>  >  From that, we
>  > automatically generate the C API and bindings.  Sure, the
>  > generation could be a
>  > bit complex but much of it will be someone else's codebase (and one
>  > that is used
>  > by lots of Gnome projects).
>  >
>  
>  that's a critical part of the question for the C api - what is needed
>  for adding a verb / parameter.
>  
>  I'm not against having c bindings. what i don't understand is why to
>  put
>  them in the 'middle', rather than as peers to the others on top of
>  the
>  python api classes.
>  
>

On 2012-7-10 2:52, Saggi Mizrahi wrote:
>  ----- Original Message -----
> > From: "Itamar Heim" <iheim(a)redhat.com&gt;
> > To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt;
> > Cc: "Adam Litke" <agl(a)us.ibm.com&gt;, vdsm-devel(a)lists.fedorahosted.org
> > Sent: Monday, July 9, 2012 11:03:43 AM
> > Subject: Re: [vdsm] [RFC] An alternative way to provide a supported interface --
> libvdsm
> >
> > On 07/09/2012 05:56 PM, Saggi Mizrahi wrote:
> >> I don't think AMQP is a good low level supported protocol as it's a
> >> very complex protocol to set up and support.
> >> Also brokers are known to have their differences in standard
> >> implementation which means supporting them all is a mess.
> >>
> >> It looks like the most accepted route is the libvirt route of
> >> having a c library abstracting away client server communication
> >> and having more advanced consumers build protocol specific bridges
> >> that may have different support standards.
> >>
> >> On a more personal note, I think brokerless messaging is the way to
> >> go in ovirt because, unlike traditional clustering, worker nodes
> >> are not interchangeable so direct communication is the way to go,
> >> rendering brokers pretty much useless.
> > but brokerless doesn't let multiple consumers which a bus provides?
>  All consumers can connect to the host and *some* events can be broadcasted to all
> connected clients.
>
>  The real question is weather you want to depend on AMQP's routing \ message storing
>  Also, if you find it preferable to have a centralized host (single point of failure) to
> get all events from all hosts for the price of  some clients (I assume read only clients)
> not needing to know the locations of all worker nodes.
>  But IMHO we already have something like that, it's called the ovirt-engine, and it
> could send aggregated events about the cluster (maybe with some extra enginy data).
>
>  The question is what does mandating a broker gives us something that an "AMQP
> bridge" wouldn't.
>  The only thing I can think of is vdsm can assume unmoderated vdsm to vdsm communication
> bypassing the engine. 
I don't fully understand the sentence above.  I think
"AMQP bridge" can 
also provide a way for unmoderated vdsm to vdsm communication bypassing 
the engine. That is, one VDSM can post a message into the bridge and the 
other interested VDSM can consume the message.
>  This means that VDSM can have some clustered behavior that requires
> no engine intervention.
>  Further more, the engine can send a request and let the nodes decide who is performing
> the operation among themselves.
>
>  Essentially:
>
>  [  engine  ]          [  engine  ]
>      |     |      VS          |
>  [vdsm][vdsm]          [  broker  ]
>                            |     |
>                         [vdsm][vdsm]
>
>  *All links are two way links 
Where is the broker sit in?  In one of the host nodes with VDSM?  or in 
a specific node?

>  This has dire consequences on API usability and supportability. So we need to converge on
> that.
>
>  There needs to be a good reason why the aforementioned logic code can't sit on a
> another ovirt specific entity (lets call it ovirt-dynamo) that uses VDSM's supported
> API 
What aforementioned logic code you are talking about here?

>    but it's own APIs (or more likely messaging algorithms) are
> unsupported.
>                                     
>    [            engine           ]
>      |            |            |
>      |      [   broker   ]     |
>      |        |        |       |
>  [vdsm]-[dynamo] : [dynamo]-[vdsm]
>       Host A      :      Host B
>
>  *All links are two way links
> >> ----- Original Message -----
> >>> From: "Adam Litke" <agl(a)us.ibm.com&gt;
> >>> To: "Itamar Heim" <iheim(a)redhat.com&gt;
> >>> Cc: vdsm-devel(a)lists.fedorahosted.org
> >>> Sent: Monday, July 9, 2012 9:56:17 AM
> >>> Subject: Re: [vdsm] [RFC] An alternative way to provide a
> >>> supported interface -- libvdsm
> >>>
> >>> On Fri, Jul 06, 2012 at 03:53:08PM +0300, Itamar Heim wrote:
> >>>> On 07/06/2012 01:15 AM, Robert Middleswarth wrote:
> >>>>> On 07/05/2012 04:45 PM, Adam Litke wrote:
> >>>>>> On Thu, Jul 05, 2012 at 03:47:42PM -0400, Saggi Mizrahi wrote:
> >>>>>>> ----- Original Message -----
> >>>>>>>> From: "Adam Litke" <agl(a)us.ibm.com&gt;
> >>>>>>>> To: "Saggi Mizrahi"
> <smizrahi(a)redhat.com&gt;
> >>>>>>>> Cc: "Anthony Liguori"
> <anthony(a)codemonkey.ws&gt;, "VDSM Project
> >>>>>>>> Development"
> <vdsm-devel(a)lists.fedorahosted.org&gt;
> >>>>>>>> Sent: Thursday, July 5, 2012 2:34:50 PM
> >>>>>>>> Subject: Re: [RFC] An alternative way to provide a
> supported
> >>>>>>>> interface -- libvdsm
> >>>>>>>>
> >>>>>>>> On Wed, Jun 27, 2012 at 02:50:02PM -0400, Saggi Mizrahi
> >>>>>>>> wrote:
> >>>>>>>>> The idea of having a supported C API was something I
> was
> >>>>>>>>> thinking
> >>>>>>>>> about doing
> >>>>>>>>> (But I'd rather use gobject introspection and not
> schema
> >>>>>>>>> generation) But the
> >>>>>>>>> problem is not having a C API is using the current
> XML RPC
> >>>>>>>>> API
> >>>>>>>>> as
> >>>>>>>>> it's base
> >>>>>>>> I want to disect this a bit to find out exactly where
> there
> >>>>>>>> might be
> >>>>>>>> agreement
> >>>>>>>> and disagreement.
> >>>>>>>>
> >>>>>>>> C API is a good thing to implement - Agreed.
> >>>>>>>>
> >>>>>>>> I also want to use gobject introspection but I don't
> agree
> >>>>>>>> that
> >>>>>>>> using
> >>>>>>>> glib
> >>>>>>>> precludes the use of a formalized schema.  My proposal
> is
> >>>>>>>> that
> >>>>>>>> we
> >>>>>>>> write a schema
> >>>>>>>> definition and generate the glib C code from that
> schema.
> >>>>>>>>
> >>>>>>>> I agree that the _current_ xmlrpc API makes a pretty bad
> base
> >>>>>>>> from
> >>>>>>>> which to
> >>>>>>>> start a supportable API.  XMLRPC is a perfectly
> reasonable
> >>>>>>>> remote/wire protocol
> >>>>>>>> and I think we should continue using it as a base for
> the
> >>>>>>>> next
> >>>>>>>> generation API.
> >>>>>>>> Using a schema will ensure that the new API is
> >>>>>>>> well-structured.
> >>>>>>> There major problems with XML-RPC (and to some extent with
> >>>>>>> REST
> >>>>>>> as
> >>>>>>> well) are high call overhead and no two way communication
> >>>>>>> (push
> >>>>>>> events). Basing on XML-RPC means that we will never be able
> to
> >>>>>>> solve
> >>>>>>> these issues.
> >>>>>> I am not sure I am ready to conceed that XML-RPC is too slow
> >>>>>> for
> >>>>>> our
> >>>>>> needs.  Can
> >>>>>> you provide some more detail around this point and possibly
> >>>>>> suggest an
> >>>>>> alternative that has even lower overhead without sacrificing
> >>>>>> the
> >>>>>> ubiquity and
> >>>>>> usability of XML-RPC?  As far as the two-way communication
> >>>>>> point,
> >>>>>> what
> >>>>>> are the
> >>>>>> options besides AMQP/ZeroMQ?  Aren't these even worse from
> an
> >>>>>> overhead
> >>>>>> perspective than XML-RPC?  Regarding two-way communication: you
> >>>>>> can
> >>>>>> write AMQP
> >>>>>> brokers based on the C API and run one on each vdsm host.
> >>>>>> Assuming
> >>>>>> the C API
> >>>>>> supports events, what else would you need?
> >>>>> I personally think that using something like AMQP for inter-node
> >>>>> communication and engine - node would be optimal.  With a rest
> >>>>> interface
> >>>>> that just send messages though something like AMQP.
> >>>> I would also not dismiss AMQP so soon
> >>>> we want a bug with more than a single listener at engine side
> >>>> (engine, history db, maybe event correlation service).
> >>>> collectd as a means for statistics already supports it as well.
> >>>> I'm for having REST as well, but not sure as main one for a
> >>>> consumer
> >>>> like ovirt engine.
> >>> I agree that a message bus could be a very useful model of
> >>> communication between
> >>> ovirt-engine components and multiple vdsm instances.  But the
> >>> complexities and
> >>> dependencies of AMQP do not make it suitable for use as a
> >>> low-level
> >>> API.  AMQP
> >>> will repel new adopters.  Why not establish a libvdsm that is more
> >>> minimalist
> >>> and can be easily used by everyone?  Then AMQP brokers can be
> >>> built
> >>> on top of
> >>> the stable API with ease.  All AMQP should require of the
> >>> low-level
> >>> API are
> >>> standard function calls and an events mechanism.
> >>>
> >>>>> Thanks
> >>>>> Robert
> >>>>>>>>> The current XML-RPC API contains a lot of decencies
> and
> >>>>>>>>> inefficiencies and we
> >>>>>>>>> would like to retire it as soon as we possibly can.
> Engine
> >>>>>>>>> would
> >>>>>>>>> like us to
> >>>>>>>>> move to a message based API and 3rd parties want
> something
> >>>>>>>>> simple
> >>>>>>>>> like REST so
> >>>>>>>>> it looks like no one actually wants to use XML-RPC.
> Not even
> >>>>>>>>> us.
> >>>>>>>> I am proposing that AMQP brokers and REST APIs could be
> >>>>>>>> written
> >>>>>>>> against the
> >>>>>>>> public API.  In fact, they need not even live in the
> vdsm
> >>>>>>>> tree
> >>>>>>>> anymore if that
> >>>>>>>> is what we choose.  Core vdsm would only be responsible
> for
> >>>>>>>> providing
> >>>>>>>> libvdsm
> >>>>>>>> and whatever language bindings we want to support.
> >>>>>>> If we take the libvdsm route, the only reason to even have a
> >>>>>>> REST
> >>>>>>> bridge is only to support OSes other then Linux which is
> >>>>>>> something
> >>>>>>> I'm not sure we care about at the moment.
> >>>>>> That might be true regarding the current in-tree
> >>>>>> implementation.
> >>>>>> However, I can
> >>>>>> almost guarantee that someone wanting to write a web GUI on top
> >>>>>> of
> >>>>>> standalone
> >>>>>> vdsm would want a REST API to talk to.  But libvdsm makes this
> >>>>>> use
> >>>>>> case of no
> >>>>>> concern to the core vdsm developers.
> >>>>>>
> >>>>>>>>> I do think that having C supportability in our API is
> a good
> >>>>>>>>> idea,
> >>>>>>>>> but the
> >>>>>>>>> current API should not be used as the base.
> >>>>>>>> Let's _start_ with a schema document that describes
> today's
> >>>>>>>> API
> >>>>>>>> and
> >>>>>>>> then clean
> >>>>>>>> it up.  I think that will work better than starting from
> >>>>>>>> scratch.
> >>>>>>>>    Once my
> >>>>>>>> schema is written I will post it and we can
> 'patch' it as a
> >>>>>>>> community
> >>>>>>>> until we
> >>>>>>>> arrive at a 1.0 version we are all happy with.
> >>>>>>> +1
> >>>>>> Ok.  Redoubling my efforts to get this done.  Describing the
> >>>>>> output of
> >>>>>> list(True) takes awhile :)
> >>>>>>
> >>>>>
> >>>>> _______________________________________________
> >>>>> vdsm-devel mailing list
> >>>>> vdsm-devel(a)lists.fedorahosted.org
> >>>>> https://fedorahosted.org/mailman/listinfo/vdsm-devel
> >>>>
> >>>> _______________________________________________
> >>>> vdsm-devel mailing list
> >>>> vdsm-devel(a)lists.fedorahosted.org
> >>>> https://fedorahosted.org/mailman/listinfo/vdsm-devel
> >>> --
> >>> Adam Litke <agl(a)us.ibm.com&gt;
> >>> IBM Linux Technology Center
> >>>
> >>> _______________________________________________
> >>> vdsm-devel mailing list
> >>> vdsm-devel(a)lists.fedorahosted.org
> >>> https://fedorahosted.org/mailman/listinfo/vdsm-devel
> >>>
> >
> >
>  _______________________________________________
>  vdsm-devel mailing list
>  vdsm-devel(a)lists.fedorahosted.org
>  https://fedorahosted.org/mailman/listinfo/vdsm-devel 

-- 
Shu Ming <shuming(a)linux.vnet.ibm.com&gt;
IBM China Systems and Technology Laboratory

Adam,

Maybe,  I don't fully understand your proposal.  Here is my 
understanding of libvdsm in the picture. Please check the following link 
for the picture.

http://www.ovirt.org/wiki/File:Libvdsm.JPG


http://www.ovirt.org/wiki/File:Libvdsm.JPG

On 2012-7-9 21:56, Adam Litke wrote:
>  On Fri, Jul 06, 2012 at 03:53:08PM +0300, Itamar Heim wrote:
> > On 07/06/2012 01:15 AM, Robert Middleswarth wrote:
> >> On 07/05/2012 04:45 PM, Adam Litke wrote:
> >>> On Thu, Jul 05, 2012 at 03:47:42PM -0400, Saggi Mizrahi wrote:
> >>>> ----- Original Message -----
> >>>>> From: "Adam Litke" <agl(a)us.ibm.com&gt;
> >>>>> To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt;
> >>>>> Cc: "Anthony Liguori" <anthony(a)codemonkey.ws&gt;,
> "VDSM Project
> >>>>> Development" <vdsm-devel(a)lists.fedorahosted.org&gt;
> >>>>> Sent: Thursday, July 5, 2012 2:34:50 PM
> >>>>> Subject: Re: [RFC] An alternative way to provide a supported
> >>>>> interface -- libvdsm
> >>>>>
> >>>>> On Wed, Jun 27, 2012 at 02:50:02PM -0400, Saggi Mizrahi wrote:
> >>>>>> The idea of having a supported C API was something I was
> thinking
> >>>>>> about doing
> >>>>>> (But I'd rather use gobject introspection and not schema
> >>>>>> generation) But the
> >>>>>> problem is not having a C API is using the current XML RPC API
> as
> >>>>>> it's base
> >>>>> I want to disect this a bit to find out exactly where there might be
> >>>>> agreement
> >>>>> and disagreement.
> >>>>>
> >>>>> C API is a good thing to implement - Agreed.
> >>>>>
> >>>>> I also want to use gobject introspection but I don't agree that
> using
> >>>>> glib
> >>>>> precludes the use of a formalized schema.  My proposal is that we
> >>>>> write a schema
> >>>>> definition and generate the glib C code from that schema.
> >>>>>
> >>>>> I agree that the _current_ xmlrpc API makes a pretty bad base from
> >>>>> which to
> >>>>> start a supportable API.  XMLRPC is a perfectly reasonable
> >>>>> remote/wire protocol
> >>>>> and I think we should continue using it as a base for the next
> >>>>> generation API.
> >>>>> Using a schema will ensure that the new API is well-structured.
> >>>> There major problems with XML-RPC (and to some extent with REST as
> >>>> well) are high call overhead and no two way communication (push
> >>>> events). Basing on XML-RPC means that we will never be able to solve
> >>>> these issues.
> >>> I am not sure I am ready to conceed that XML-RPC is too slow for our
> >>> needs.  Can
> >>> you provide some more detail around this point and possibly suggest an
> >>> alternative that has even lower overhead without sacrificing the
> >>> ubiquity and
> >>> usability of XML-RPC?  As far as the two-way communication point, what
> >>> are the
> >>> options besides AMQP/ZeroMQ?  Aren't these even worse from an overhead
> >>> perspective than XML-RPC?  Regarding two-way communication: you can
> >>> write AMQP
> >>> brokers based on the C API and run one on each vdsm host.  Assuming
> >>> the C API
> >>> supports events, what else would you need?
> >> I personally think that using something like AMQP for inter-node
> >> communication and engine - node would be optimal.  With a rest interface
> >> that just send messages though something like AMQP.
> > I would also not dismiss AMQP so soon
> > we want a bug with more than a single listener at engine side
> > (engine, history db, maybe event correlation service).
> > collectd as a means for statistics already supports it as well.
> > I'm for having REST as well, but not sure as main one for a consumer
> > like ovirt engine.
>  I agree that a message bus could be a very useful model of communication between
>  ovirt-engine components and multiple vdsm instances.  But the complexities and
>  dependencies of AMQP do not make it suitable for use as a low-level API.  AMQP
>  will repel new adopters.  Why not establish a libvdsm that is more minimalist
>  and can be easily used by everyone?  Then AMQP brokers can be built on top of
>  the stable API with ease.  All AMQP should require of the low-level API are
>  standard function calls and an events mechanism.
>
> >> Thanks
> >> Robert
> >>>>>> The current XML-RPC API contains a lot of decencies and
> >>>>>> inefficiencies and we
> >>>>>> would like to retire it as soon as we possibly can. Engine would
> >>>>>> like us to
> >>>>>> move to a message based API and 3rd parties want something
> simple
> >>>>>> like REST so
> >>>>>> it looks like no one actually wants to use XML-RPC. Not even us.
> >>>>> I am proposing that AMQP brokers and REST APIs could be written
> >>>>> against the
> >>>>> public API.  In fact, they need not even live in the vdsm tree
> >>>>> anymore if that
> >>>>> is what we choose.  Core vdsm would only be responsible for
> providing
> >>>>> libvdsm
> >>>>> and whatever language bindings we want to support.
> >>>> If we take the libvdsm route, the only reason to even have a REST
> >>>> bridge is only to support OSes other then Linux which is something
> >>>> I'm not sure we care about at the moment.
> >>> That might be true regarding the current in-tree implementation.
> >>> However, I can
> >>> almost guarantee that someone wanting to write a web GUI on top of
> >>> standalone
> >>> vdsm would want a REST API to talk to.  But libvdsm makes this use
> >>> case of no
> >>> concern to the core vdsm developers.
> >>>
> >>>>>> I do think that having C supportability in our API is a good
> idea,
> >>>>>> but the
> >>>>>> current API should not be used as the base.
> >>>>> Let's _start_ with a schema document that describes today's
> API and
> >>>>> then clean
> >>>>> it up.  I think that will work better than starting from scratch.
> >>>>>   Once my
> >>>>> schema is written I will post it and we can 'patch' it as a
> community
> >>>>> until we
> >>>>> arrive at a 1.0 version we are all happy with.
> >>>> +1
> >>> Ok.  Redoubling my efforts to get this done.  Describing the output of
> >>> list(True) takes awhile :)
> >>>
> >>
> >> _______________________________________________
> >> vdsm-devel mailing list
> >> vdsm-devel(a)lists.fedorahosted.org
> >> https://fedorahosted.org/mailman/listinfo/vdsm-devel
> >
> > _______________________________________________
> > vdsm-devel mailing list
> > vdsm-devel(a)lists.fedorahosted.org
> > https://fedorahosted.org/mailman/listinfo/vdsm-devel 

-- 
Shu Ming <shuming(a)linux.vnet.ibm.com&gt;
IBM China Systems and Technology Laboratory

I'm sorry, but I don't really understand the drawing

----- Original Message -----
>  From: "Shu Ming" <shuming(a)linux.vnet.ibm.com&gt;
>  To: "Adam Litke" <agl(a)us.ibm.com&gt;
>  Cc: vdsm-devel(a)lists.fedorahosted.org
>  Sent: Wednesday, July 11, 2012 10:24:49 AM
>  Subject: Re: [vdsm] [RFC] An alternative way to provide a supported interface -- libvdsm
>  
>  Adam,
>  
>  Maybe,  I don't fully understand your proposal.  Here is my
>  understanding of libvdsm in the picture. Please check the following
>  link
>  for the picture.
>  
>  http://www.ovirt.org/wiki/File:Libvdsm.JPG
>  
>  
>  http://www.ovirt.org/wiki/File:Libvdsm.JPG
>  
>  On 2012-7-9 21:56, Adam Litke wrote:
>  > On Fri, Jul 06, 2012 at 03:53:08PM +0300, Itamar Heim wrote:
>  >> On 07/06/2012 01:15 AM, Robert Middleswarth wrote:
>  >>> On 07/05/2012 04:45 PM, Adam Litke wrote:
>  >>>> On Thu, Jul 05, 2012 at 03:47:42PM -0400, Saggi Mizrahi wrote:
>  >>>>> ----- Original Message -----
>  >>>>>> From: "Adam Litke" <agl(a)us.ibm.com&gt;
>  >>>>>> To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt;
>  >>>>>> Cc: "Anthony Liguori" <anthony(a)codemonkey.ws&gt;,
> "VDSM Project
>  >>>>>> Development" <vdsm-devel(a)lists.fedorahosted.org&gt;
>  >>>>>> Sent: Thursday, July 5, 2012 2:34:50 PM
>  >>>>>> Subject: Re: [RFC] An alternative way to provide a supported
>  >>>>>> interface -- libvdsm
>  >>>>>>
>  >>>>>> On Wed, Jun 27, 2012 at 02:50:02PM -0400, Saggi Mizrahi wrote:
>  >>>>>>> The idea of having a supported C API was something I was
>  >>>>>>> thinking
>  >>>>>>> about doing
>  >>>>>>> (But I'd rather use gobject introspection and not
> schema
>  >>>>>>> generation) But the
>  >>>>>>> problem is not having a C API is using the current XML RPC
>  >>>>>>> API as
>  >>>>>>> it's base
>  >>>>>> I want to disect this a bit to find out exactly where there
>  >>>>>> might be
>  >>>>>> agreement
>  >>>>>> and disagreement.
>  >>>>>>
>  >>>>>> C API is a good thing to implement - Agreed.
>  >>>>>>
>  >>>>>> I also want to use gobject introspection but I don't agree
>  >>>>>> that using
>  >>>>>> glib
>  >>>>>> precludes the use of a formalized schema.  My proposal is that
>  >>>>>> we
>  >>>>>> write a schema
>  >>>>>> definition and generate the glib C code from that schema.
>  >>>>>>
>  >>>>>> I agree that the _current_ xmlrpc API makes a pretty bad base
>  >>>>>> from
>  >>>>>> which to
>  >>>>>> start a supportable API.  XMLRPC is a perfectly reasonable
>  >>>>>> remote/wire protocol
>  >>>>>> and I think we should continue using it as a base for the next
>  >>>>>> generation API.
>  >>>>>> Using a schema will ensure that the new API is
>  >>>>>> well-structured.
>  >>>>> There major problems with XML-RPC (and to some extent with REST
>  >>>>> as
>  >>>>> well) are high call overhead and no two way communication (push
>  >>>>> events). Basing on XML-RPC means that we will never be able to
>  >>>>> solve
>  >>>>> these issues.
>  >>>> I am not sure I am ready to conceed that XML-RPC is too slow for
>  >>>> our
>  >>>> needs.  Can
>  >>>> you provide some more detail around this point and possibly
>  >>>> suggest an
>  >>>> alternative that has even lower overhead without sacrificing the
>  >>>> ubiquity and
>  >>>> usability of XML-RPC?  As far as the two-way communication
>  >>>> point, what
>  >>>> are the
>  >>>> options besides AMQP/ZeroMQ?  Aren't these even worse from an
>  >>>> overhead
>  >>>> perspective than XML-RPC?  Regarding two-way communication: you
>  >>>> can
>  >>>> write AMQP
>  >>>> brokers based on the C API and run one on each vdsm host.
>  >>>>  Assuming
>  >>>> the C API
>  >>>> supports events, what else would you need?
>  >>> I personally think that using something like AMQP for inter-node
>  >>> communication and engine - node would be optimal.  With a rest
>  >>> interface
>  >>> that just send messages though something like AMQP.
>  >> I would also not dismiss AMQP so soon
>  >> we want a bug with more than a single listener at engine side
>  >> (engine, history db, maybe event correlation service).
>  >> collectd as a means for statistics already supports it as well.
>  >> I'm for having REST as well, but not sure as main one for a
>  >> consumer
>  >> like ovirt engine.
>  > I agree that a message bus could be a very useful model of
>  > communication between
>  > ovirt-engine components and multiple vdsm instances.  But the
>  > complexities and
>  > dependencies of AMQP do not make it suitable for use as a low-level
>  > API.  AMQP
>  > will repel new adopters.  Why not establish a libvdsm that is more
>  > minimalist
>  > and can be easily used by everyone?  Then AMQP brokers can be built
>  > on top of
>  > the stable API with ease.  All AMQP should require of the low-level
>  > API are
>  > standard function calls and an events mechanism.
>  >
>  >>> Thanks
>  >>> Robert
>  >>>>>>> The current XML-RPC API contains a lot of decencies and
>  >>>>>>> inefficiencies and we
>  >>>>>>> would like to retire it as soon as we possibly can. Engine
>  >>>>>>> would
>  >>>>>>> like us to
>  >>>>>>> move to a message based API and 3rd parties want something
>  >>>>>>> simple
>  >>>>>>> like REST so
>  >>>>>>> it looks like no one actually wants to use XML-RPC. Not
> even
>  >>>>>>> us.
>  >>>>>> I am proposing that AMQP brokers and REST APIs could be
>  >>>>>> written
>  >>>>>> against the
>  >>>>>> public API.  In fact, they need not even live in the vdsm tree
>  >>>>>> anymore if that
>  >>>>>> is what we choose.  Core vdsm would only be responsible for
>  >>>>>> providing
>  >>>>>> libvdsm
>  >>>>>> and whatever language bindings we want to support.
>  >>>>> If we take the libvdsm route, the only reason to even have a
>  >>>>> REST
>  >>>>> bridge is only to support OSes other then Linux which is
>  >>>>> something
>  >>>>> I'm not sure we care about at the moment.
>  >>>> That might be true regarding the current in-tree implementation.
>  >>>> However, I can
>  >>>> almost guarantee that someone wanting to write a web GUI on top
>  >>>> of
>  >>>> standalone
>  >>>> vdsm would want a REST API to talk to.  But libvdsm makes this
>  >>>> use
>  >>>> case of no
>  >>>> concern to the core vdsm developers.
>  >>>>
>  >>>>>>> I do think that having C supportability in our API is a
> good
>  >>>>>>> idea,
>  >>>>>>> but the
>  >>>>>>> current API should not be used as the base.
>  >>>>>> Let's _start_ with a schema document that describes
> today's
>  >>>>>> API and
>  >>>>>> then clean
>  >>>>>> it up.  I think that will work better than starting from
>  >>>>>> scratch.
>  >>>>>>   Once my
>  >>>>>> schema is written I will post it and we can 'patch' it
> as a
>  >>>>>> community
>  >>>>>> until we
>  >>>>>> arrive at a 1.0 version we are all happy with.
>  >>>>> +1
>  >>>> Ok.  Redoubling my efforts to get this done.  Describing the
>  >>>> output of
>  >>>> list(True) takes awhile :)
>  >>>>
>  >>>
>  >>> _______________________________________________
>  >>> vdsm-devel mailing list
>  >>> vdsm-devel(a)lists.fedorahosted.org
>  >>> https://fedorahosted.org/mailman/listinfo/vdsm-devel
>  >>
>  >> _______________________________________________
>  >> vdsm-devel mailing list
>  >> vdsm-devel(a)lists.fedorahosted.org
>  >> https://fedorahosted.org/mailman/listinfo/vdsm-devel
>  
>  
>  --
>  Shu Ming <shuming(a)linux.vnet.ibm.com&gt;
>  IBM China Systems and Technology Laboratory
>  
>  
>  _______________________________________________
>  vdsm-devel mailing list
>  vdsm-devel(a)lists.fedorahosted.org
>  https://fedorahosted.org/mailman/listinfo/vdsm-devel
>

Basically,  my understanding is that we can generate two versions of 
libvdsm from the schema file for both the node and the management 
application.  First, the transportation protocols(XMLRPC, REST-API) will 
depend on libvdsm(node version) to export the APIs to remote management 
application.  Secondly, the management application can use 
libvdsm(application version ) to emit the remote call to the node.   
Also, transportation protocols like REST API and XML RPC API can also be 
generated automatically by the schema file with C, Java, Python bindings.

On 2012-7-12 2:29, Saggi Mizrahi wrote:
>  I'm sorry, but I don't really understand the drawing
>
>  ----- Original Message -----
> > From: "Shu Ming" <shuming(a)linux.vnet.ibm.com&gt;
> > To: "Adam Litke" <agl(a)us.ibm.com&gt;
> > Cc: vdsm-devel(a)lists.fedorahosted.org
> > Sent: Wednesday, July 11, 2012 10:24:49 AM
> > Subject: Re: [vdsm] [RFC] An alternative way to provide a supported interface --
> libvdsm
> >
> > Adam,
> >
> > Maybe,  I don't fully understand your proposal.  Here is my
> > understanding of libvdsm in the picture. Please check the following
> > link
> > for the picture.
> >
> > http://www.ovirt.org/wiki/File:Libvdsm.JPG
> >
> >
> > http://www.ovirt.org/wiki/File:Libvdsm.JPG
> >
> > On 2012-7-9 21:56, Adam Litke wrote:
> >> On Fri, Jul 06, 2012 at 03:53:08PM +0300, Itamar Heim wrote:
> >>> On 07/06/2012 01:15 AM, Robert Middleswarth wrote:
> >>>> On 07/05/2012 04:45 PM, Adam Litke wrote:
> >>>>> On Thu, Jul 05, 2012 at 03:47:42PM -0400, Saggi Mizrahi wrote:
> >>>>>> ----- Original Message -----
> >>>>>>> From: "Adam Litke" <agl(a)us.ibm.com&gt;
> >>>>>>> To: "Saggi Mizrahi" <smizrahi(a)redhat.com&gt;
> >>>>>>> Cc: "Anthony Liguori"
> <anthony(a)codemonkey.ws&gt;, "VDSM Project
> >>>>>>> Development" <vdsm-devel(a)lists.fedorahosted.org&gt;
> >>>>>>> Sent: Thursday, July 5, 2012 2:34:50 PM
> >>>>>>> Subject: Re: [RFC] An alternative way to provide a supported
> >>>>>>> interface -- libvdsm
> >>>>>>>
> >>>>>>> On Wed, Jun 27, 2012 at 02:50:02PM -0400, Saggi Mizrahi
> wrote:
> >>>>>>>> The idea of having a supported C API was something I was
> >>>>>>>> thinking
> >>>>>>>> about doing
> >>>>>>>> (But I'd rather use gobject introspection and not
> schema
> >>>>>>>> generation) But the
> >>>>>>>> problem is not having a C API is using the current XML
> RPC
> >>>>>>>> API as
> >>>>>>>> it's base
> >>>>>>> I want to disect this a bit to find out exactly where there
> >>>>>>> might be
> >>>>>>> agreement
> >>>>>>> and disagreement.
> >>>>>>>
> >>>>>>> C API is a good thing to implement - Agreed.
> >>>>>>>
> >>>>>>> I also want to use gobject introspection but I don't
> agree
> >>>>>>> that using
> >>>>>>> glib
> >>>>>>> precludes the use of a formalized schema.  My proposal is
> that
> >>>>>>> we
> >>>>>>> write a schema
> >>>>>>> definition and generate the glib C code from that schema.
> >>>>>>>
> >>>>>>> I agree that the _current_ xmlrpc API makes a pretty bad
> base
> >>>>>>> from
> >>>>>>> which to
> >>>>>>> start a supportable API.  XMLRPC is a perfectly reasonable
> >>>>>>> remote/wire protocol
> >>>>>>> and I think we should continue using it as a base for the
> next
> >>>>>>> generation API.
> >>>>>>> Using a schema will ensure that the new API is
> >>>>>>> well-structured.
> >>>>>> There major problems with XML-RPC (and to some extent with REST
> >>>>>> as
> >>>>>> well) are high call overhead and no two way communication (push
> >>>>>> events). Basing on XML-RPC means that we will never be able to
> >>>>>> solve
> >>>>>> these issues.
> >>>>> I am not sure I am ready to conceed that XML-RPC is too slow for
> >>>>> our
> >>>>> needs.  Can
> >>>>> you provide some more detail around this point and possibly
> >>>>> suggest an
> >>>>> alternative that has even lower overhead without sacrificing the
> >>>>> ubiquity and
> >>>>> usability of XML-RPC?  As far as the two-way communication
> >>>>> point, what
> >>>>> are the
> >>>>> options besides AMQP/ZeroMQ?  Aren't these even worse from an
> >>>>> overhead
> >>>>> perspective than XML-RPC?  Regarding two-way communication: you
> >>>>> can
> >>>>> write AMQP
> >>>>> brokers based on the C API and run one on each vdsm host.
> >>>>>   Assuming
> >>>>> the C API
> >>>>> supports events, what else would you need?
> >>>> I personally think that using something like AMQP for inter-node
> >>>> communication and engine - node would be optimal.  With a rest
> >>>> interface
> >>>> that just send messages though something like AMQP.
> >>> I would also not dismiss AMQP so soon
> >>> we want a bug with more than a single listener at engine side
> >>> (engine, history db, maybe event correlation service).
> >>> collectd as a means for statistics already supports it as well.
> >>> I'm for having REST as well, but not sure as main one for a
> >>> consumer
> >>> like ovirt engine.
> >> I agree that a message bus could be a very useful model of
> >> communication between
> >> ovirt-engine components and multiple vdsm instances.  But the
> >> complexities and
> >> dependencies of AMQP do not make it suitable for use as a low-level
> >> API.  AMQP
> >> will repel new adopters.  Why not establish a libvdsm that is more
> >> minimalist
> >> and can be easily used by everyone?  Then AMQP brokers can be built
> >> on top of
> >> the stable API with ease.  All AMQP should require of the low-level
> >> API are
> >> standard function calls and an events mechanism.
> >>
> >>>> Thanks
> >>>> Robert
> >>>>>>>> The current XML-RPC API contains a lot of decencies and
> >>>>>>>> inefficiencies and we
> >>>>>>>> would like to retire it as soon as we possibly can.
> Engine
> >>>>>>>> would
> >>>>>>>> like us to
> >>>>>>>> move to a message based API and 3rd parties want
> something
> >>>>>>>> simple
> >>>>>>>> like REST so
> >>>>>>>> it looks like no one actually wants to use XML-RPC. Not
> even
> >>>>>>>> us.
> >>>>>>> I am proposing that AMQP brokers and REST APIs could be
> >>>>>>> written
> >>>>>>> against the
> >>>>>>> public API.  In fact, they need not even live in the vdsm
> tree
> >>>>>>> anymore if that
> >>>>>>> is what we choose.  Core vdsm would only be responsible for
> >>>>>>> providing
> >>>>>>> libvdsm
> >>>>>>> and whatever language bindings we want to support.
> >>>>>> If we take the libvdsm route, the only reason to even have a
> >>>>>> REST
> >>>>>> bridge is only to support OSes other then Linux which is
> >>>>>> something
> >>>>>> I'm not sure we care about at the moment.
> >>>>> That might be true regarding the current in-tree implementation.
> >>>>> However, I can
> >>>>> almost guarantee that someone wanting to write a web GUI on top
> >>>>> of
> >>>>> standalone
> >>>>> vdsm would want a REST API to talk to.  But libvdsm makes this
> >>>>> use
> >>>>> case of no
> >>>>> concern to the core vdsm developers.
> >>>>>
> >>>>>>>> I do think that having C supportability in our API is a
> good
> >>>>>>>> idea,
> >>>>>>>> but the
> >>>>>>>> current API should not be used as the base.
> >>>>>>> Let's _start_ with a schema document that describes
> today's
> >>>>>>> API and
> >>>>>>> then clean
> >>>>>>> it up.  I think that will work better than starting from
> >>>>>>> scratch.
> >>>>>>>    Once my
> >>>>>>> schema is written I will post it and we can 'patch'
> it as a
> >>>>>>> community
> >>>>>>> until we
> >>>>>>> arrive at a 1.0 version we are all happy with.
> >>>>>> +1
> >>>>> Ok.  Redoubling my efforts to get this done.  Describing the
> >>>>> output of
> >>>>> list(True) takes awhile :)
> >>>>>
> >>>> _______________________________________________
> >>>> vdsm-devel mailing list
> >>>> vdsm-devel(a)lists.fedorahosted.org
> >>>> https://fedorahosted.org/mailman/listinfo/vdsm-devel
> >>> _______________________________________________
> >>> vdsm-devel mailing list
> >>> vdsm-devel(a)lists.fedorahosted.org
> >>> https://fedorahosted.org/mailman/listinfo/vdsm-devel
> >
> > --
> > Shu Ming <shuming(a)linux.vnet.ibm.com&gt;
> > IBM China Systems and Technology Laboratory
> >
> >
> > _______________________________________________
> > vdsm-devel mailing list
> > vdsm-devel(a)lists.fedorahosted.org
> > https://fedorahosted.org/mailman/listinfo/vdsm-devel
> > 

-- 
Shu Ming <shuming(a)linux.vnet.ibm.com&gt;
IBM China Systems and Technology Laboratory

On Thu, Jul 12, 2012 at 08:11:17AM +0800, Shu Ming wrote:
>  Basically,  my understanding is that we can generate two versions of
>  libvdsm from the schema file for both the node and the management
>  application.  First, the transportation protocols(XMLRPC, REST-API)
>  will depend on libvdsm(node version) to export the APIs to remote
>  management application.  Secondly, the management application can
>  use libvdsm(application version ) to emit the remote call to the
>  node.   Also, transportation protocols like REST API and XML RPC API
>  can also be generated automatically by the schema file with C, Java,
>  Python bindings. 
I think this might be a bit too complex of a model.  Here's how I see it...

The schema generates C/gObject code which can be compiled into libvdsm.  We can
use the gObject introspection library to automatically generate language
bindings for Java, Python, Perl, etc.

The libvdsm library talks to vdsmd using a wire protocol that works locally and
remotely.  This wire protocol is completely hidden from library users.  It's an
implementation detail that can be changed later if necessary.  Today I would
recommend that we use xmlrpc.  This means that ovirt-engine or another remote
program could use libvdsm in the exact same manner as a local program.  The
library user just needs to call libvdsm.connect(uri).

Finally, REST and AMQP bridges would be written solely against libvdsm.  These
bridges are probably not suitable for code generation (but we can revisit that
as a separate issue because it's up to the bridge writer to determine the best
approach).

>  
>  On 2012-7-12 2:29, Saggi Mizrahi wrote:
>  >I'm sorry, but I don't really understand the drawing
>  >
>  >----- Original Message -----
>  >>From: "Shu Ming" <shuming(a)linux.vnet.ibm.com&gt;
>  >>To: "Adam Litke" <agl(a)us.ibm.com&gt;
>  >>Cc: vdsm-devel(a)lists.fedorahosted.org
>  >>Sent: Wednesday, July 11, 2012 10:24:49 AM
>  >>Subject: Re: [vdsm] [RFC] An alternative way to provide a supported interface --
> libvdsm
>  >>
>  >>Adam,
>  >>
>  >>Maybe,  I don't fully understand your proposal.  Here is my
>  >>understanding of libvdsm in the picture. Please check the following
>  >>link
>  >>for the picture.
>  >>
>  >>http://www.ovirt.org/wiki/File:Libvdsm.JPG
>  >>
>  >>
>  >>http://www.ovirt.org/wiki/File:Libvdsm.JPG
>  >>
>  >>On 2012-7-9 21:56, Adam Litke wrote:
>  >>>On Fri, Jul 06, 2012 at 03:53:08PM +0300, Itamar Heim wrote:
>  >>>>On 07/06/2012 01:15 AM, Robert Middleswarth wrote:
>  >>>>>On 07/05/2012 04:45 PM, Adam Litke wrote:
>  >>>>>>On Thu, Jul 05, 2012 at 03:47:42PM -0400, Saggi Mizrahi wrote:
>  >>>>>>>----- Original Message -----
>  >>>>>>>>From: "Adam Litke" <agl(a)us.ibm.com&gt;
>  >>>>>>>>To: "Saggi Mizrahi"
> <smizrahi(a)redhat.com&gt;
>  >>>>>>>>Cc: "Anthony Liguori"
> <anthony(a)codemonkey.ws&gt;, "VDSM Project
>  >>>>>>>>Development"
> <vdsm-devel(a)lists.fedorahosted.org&gt;
>  >>>>>>>>Sent: Thursday, July 5, 2012 2:34:50 PM
>  >>>>>>>>Subject: Re: [RFC] An alternative way to provide a
> supported
>  >>>>>>>>interface -- libvdsm
>  >>>>>>>>
>  >>>>>>>>On Wed, Jun 27, 2012 at 02:50:02PM -0400, Saggi Mizrahi
> wrote:
>  >>>>>>>>>The idea of having a supported C API was something I
> was
>  >>>>>>>>>thinking
>  >>>>>>>>>about doing
>  >>>>>>>>>(But I'd rather use gobject introspection and not
> schema
>  >>>>>>>>>generation) But the
>  >>>>>>>>>problem is not having a C API is using the current
> XML RPC
>  >>>>>>>>>API as
>  >>>>>>>>>it's base
>  >>>>>>>>I want to disect this a bit to find out exactly where
> there
>  >>>>>>>>might be
>  >>>>>>>>agreement
>  >>>>>>>>and disagreement.
>  >>>>>>>>
>  >>>>>>>>C API is a good thing to implement - Agreed.
>  >>>>>>>>
>  >>>>>>>>I also want to use gobject introspection but I don't
> agree
>  >>>>>>>>that using
>  >>>>>>>>glib
>  >>>>>>>>precludes the use of a formalized schema.  My proposal is
> that
>  >>>>>>>>we
>  >>>>>>>>write a schema
>  >>>>>>>>definition and generate the glib C code from that
> schema.
>  >>>>>>>>
>  >>>>>>>>I agree that the _current_ xmlrpc API makes a pretty bad
> base
>  >>>>>>>>from
>  >>>>>>>>which to
>  >>>>>>>>start a supportable API.  XMLRPC is a perfectly
> reasonable
>  >>>>>>>>remote/wire protocol
>  >>>>>>>>and I think we should continue using it as a base for the
> next
>  >>>>>>>>generation API.
>  >>>>>>>>Using a schema will ensure that the new API is
>  >>>>>>>>well-structured.
>  >>>>>>>There major problems with XML-RPC (and to some extent with
> REST
>  >>>>>>>as
>  >>>>>>>well) are high call overhead and no two way communication
> (push
>  >>>>>>>events). Basing on XML-RPC means that we will never be able
> to
>  >>>>>>>solve
>  >>>>>>>these issues.
>  >>>>>>I am not sure I am ready to conceed that XML-RPC is too slow for
>  >>>>>>our
>  >>>>>>needs.  Can
>  >>>>>>you provide some more detail around this point and possibly
>  >>>>>>suggest an
>  >>>>>>alternative that has even lower overhead without sacrificing the
>  >>>>>>ubiquity and
>  >>>>>>usability of XML-RPC?  As far as the two-way communication
>  >>>>>>point, what
>  >>>>>>are the
>  >>>>>>options besides AMQP/ZeroMQ?  Aren't these even worse from
> an
>  >>>>>>overhead
>  >>>>>>perspective than XML-RPC?  Regarding two-way communication: you
>  >>>>>>can
>  >>>>>>write AMQP
>  >>>>>>brokers based on the C API and run one on each vdsm host.
>  >>>>>>  Assuming
>  >>>>>>the C API
>  >>>>>>supports events, what else would you need?
>  >>>>>I personally think that using something like AMQP for inter-node
>  >>>>>communication and engine - node would be optimal.  With a rest
>  >>>>>interface
>  >>>>>that just send messages though something like AMQP.
>  >>>>I would also not dismiss AMQP so soon
>  >>>>we want a bug with more than a single listener at engine side
>  >>>>(engine, history db, maybe event correlation service).
>  >>>>collectd as a means for statistics already supports it as well.
>  >>>>I'm for having REST as well, but not sure as main one for a
>  >>>>consumer
>  >>>>like ovirt engine.
>  >>>I agree that a message bus could be a very useful model of
>  >>>communication between
>  >>>ovirt-engine components and multiple vdsm instances.  But the
>  >>>complexities and
>  >>>dependencies of AMQP do not make it suitable for use as a low-level
>  >>>API.  AMQP
>  >>>will repel new adopters.  Why not establish a libvdsm that is more
>  >>>minimalist
>  >>>and can be easily used by everyone?  Then AMQP brokers can be built
>  >>>on top of
>  >>>the stable API with ease.  All AMQP should require of the low-level
>  >>>API are
>  >>>standard function calls and an events mechanism.
>  >>>
>  >>>>>Thanks
>  >>>>>Robert
>  >>>>>>>>>The current XML-RPC API contains a lot of decencies
> and
>  >>>>>>>>>inefficiencies and we
>  >>>>>>>>>would like to retire it as soon as we possibly can.
> Engine
>  >>>>>>>>>would
>  >>>>>>>>>like us to
>  >>>>>>>>>move to a message based API and 3rd parties want
> something
>  >>>>>>>>>simple
>  >>>>>>>>>like REST so
>  >>>>>>>>>it looks like no one actually wants to use XML-RPC.
> Not even
>  >>>>>>>>>us.
>  >>>>>>>>I am proposing that AMQP brokers and REST APIs could be
>  >>>>>>>>written
>  >>>>>>>>against the
>  >>>>>>>>public API.  In fact, they need not even live in the vdsm
> tree
>  >>>>>>>>anymore if that
>  >>>>>>>>is what we choose.  Core vdsm would only be responsible
> for
>  >>>>>>>>providing
>  >>>>>>>>libvdsm
>  >>>>>>>>and whatever language bindings we want to support.
>  >>>>>>>If we take the libvdsm route, the only reason to even have a
>  >>>>>>>REST
>  >>>>>>>bridge is only to support OSes other then Linux which is
>  >>>>>>>something
>  >>>>>>>I'm not sure we care about at the moment.
>  >>>>>>That might be true regarding the current in-tree implementation.
>  >>>>>>However, I can
>  >>>>>>almost guarantee that someone wanting to write a web GUI on top
>  >>>>>>of
>  >>>>>>standalone
>  >>>>>>vdsm would want a REST API to talk to.  But libvdsm makes this
>  >>>>>>use
>  >>>>>>case of no
>  >>>>>>concern to the core vdsm developers.
>  >>>>>>
>  >>>>>>>>>I do think that having C supportability in our API is
> a good
>  >>>>>>>>>idea,
>  >>>>>>>>>but the
>  >>>>>>>>>current API should not be used as the base.
>  >>>>>>>>Let's _start_ with a schema document that describes
> today's
>  >>>>>>>>API and
>  >>>>>>>>then clean
>  >>>>>>>>it up.  I think that will work better than starting from
>  >>>>>>>>scratch.
>  >>>>>>>>   Once my
>  >>>>>>>>schema is written I will post it and we can
> 'patch' it as a
>  >>>>>>>>community
>  >>>>>>>>until we
>  >>>>>>>>arrive at a 1.0 version we are all happy with.
>  >>>>>>>+1
>  >>>>>>Ok.  Redoubling my efforts to get this done.  Describing the
>  >>>>>>output of
>  >>>>>>list(True) takes awhile :)
>  >>>>>>
>  >>>>>_______________________________________________
>  >>>>>vdsm-devel mailing list
>  &gt;&gt;&gt;&gt;&gt;vdsm-devel(a)lists.fedorahosted.org
>  >>>>>https://fedorahosted.org/mailman/listinfo/vdsm-devel
>  >>>>_______________________________________________
>  >>>>vdsm-devel mailing list
>  &gt;&gt;&gt;&gt;vdsm-devel(a)lists.fedorahosted.org
>  >>>>https://fedorahosted.org/mailman/listinfo/vdsm-devel
>  >>
>  >>--
>  >>Shu Ming <shuming(a)linux.vnet.ibm.com&gt;
>  >>IBM China Systems and Technology Laboratory
>  >>
>  >>
>  >>_______________________________________________
>  >>vdsm-devel mailing list
>  &gt;&gt;vdsm-devel(a)lists.fedorahosted.org
>  >>https://fedorahosted.org/mailman/listinfo/vdsm-devel
>  >>
>  
>  
>  -- 
>  Shu Ming <shuming(a)linux.vnet.ibm.com&gt;
>  IBM China Systems and Technology Laboratory
>  
>   
-- 
Adam Litke <agl(a)us.ibm.com&gt;
IBM Linux Technology Center

On 2012-7-12 20:41, Adam Litke wrote:
>  On Thu, Jul 12, 2012 at 08:11:17AM +0800, Shu Ming wrote:
> > Basically,  my understanding is that we can generate two versions of
> > libvdsm from the schema file for both the node and the management
> > application.  First, the transportation protocols(XMLRPC, REST-API)
> > will depend on libvdsm(node version) to export the APIs to remote
> > management application.  Secondly, the management application can
> > use libvdsm(application version ) to emit the remote call to the
> > node.   Also, transportation protocols like REST API and XML RPC API
> > can also be generated automatically by the schema file with C, Java,
> > Python bindings.
>  I think this might be a bit too complex of a model.  Here's how I see it...
>
>  The schema generates C/gObject code which can be compiled into libvdsm.  We can
>  use the gObject introspection library to automatically generate language
>  bindings for Java, Python, Perl, etc. 
Do you mean Java, Python, Perl, etc bindings
of libvdsm?

>  The libvdsm library talks to vdsmd using a wire protocol that works locally and
>  remotely.  This wire protocol is completely hidden from library users.  It's an
>  implementation detail that can be changed later if necessary.  Today I would
>  recommend that we use xmlrpc.  This means that ovirt-engine or another remote
>  program could use libvdsm in the exact same manner as a local program.  The
>  library user just needs to call libvdsm.connect(uri). 
Except libvdsm.connet() or libvdsm.disconnet(), do you expect the engine 
will not change any of its xml-rpc interfaces now after libvdsm is 
introduced?  I mean engine may not want to change much of its current 
code, so it expect the least change.

>  Finally, REST and AMQP bridges would be written solely against libvdsm.  These 
So we should write REST, AMQP bridges manually with Java, Python, Perl 
language on various bindings of libvdsm?

>  bridges are probably not suitable for code generation (but we can
> revisit that
>  as a separate issue because it's up to the bridge writer to determine the best
>  approach).
>
> > On 2012-7-12 2:29, Saggi Mizrahi wrote:
> >> I'm sorry, but I don't really understand the drawing
> >>
> >> ----- Original Message -----
> >>> From: "Shu Ming" <shuming(a)linux.vnet.ibm.com&gt;
> >>> To: "Adam Litke" <agl(a)us.ibm.com&gt;
> >>> Cc: vdsm-devel(a)lists.fedorahosted.org
> >>> Sent: Wednesday, July 11, 2012 10:24:49 AM
> >>> Subject: Re: [vdsm] [RFC] An alternative way to provide a supported interface
> -- libvdsm
> >>>
> >>> Adam,
> >>>
> >>> Maybe,  I don't fully understand your proposal.  Here is my
> >>> understanding of libvdsm in the picture. Please check the following
> >>> link
> >>> for the picture.
> >>>
> >>> http://www.ovirt.org/wiki/File:Libvdsm.JPG
> >>>
> >>>
> >>> http://www.ovirt.org/wiki/File:Libvdsm.JPG
> >>>
> >>> On 2012-7-9 21:56, Adam Litke wrote:
> >>>> On Fri, Jul 06, 2012 at 03:53:08PM +0300, Itamar Heim wrote:
> >>>>> On 07/06/2012 01:15 AM, Robert Middleswarth wrote:
> >>>>>> On 07/05/2012 04:45 PM, Adam Litke wrote:
> >>>>>>> On Thu, Jul 05, 2012 at 03:47:42PM -0400, Saggi Mizrahi
> wrote:
> >>>>>>>> ----- Original Message -----
> >>>>>>>>> From: "Adam Litke" <agl(a)us.ibm.com&gt;
> >>>>>>>>> To: "Saggi Mizrahi"
> <smizrahi(a)redhat.com&gt;
> >>>>>>>>> Cc: "Anthony Liguori"
> <anthony(a)codemonkey.ws&gt;, "VDSM Project
> >>>>>>>>> Development"
> <vdsm-devel(a)lists.fedorahosted.org&gt;
> >>>>>>>>> Sent: Thursday, July 5, 2012 2:34:50 PM
> >>>>>>>>> Subject: Re: [RFC] An alternative way to provide a
> supported
> >>>>>>>>> interface -- libvdsm
> >>>>>>>>>
> >>>>>>>>> On Wed, Jun 27, 2012 at 02:50:02PM -0400, Saggi
> Mizrahi wrote:
> >>>>>>>>>> The idea of having a supported C API was
> something I was
> >>>>>>>>>> thinking
> >>>>>>>>>> about doing
> >>>>>>>>>> (But I'd rather use gobject introspection and
> not schema
> >>>>>>>>>> generation) But the
> >>>>>>>>>> problem is not having a C API is using the
> current XML RPC
> >>>>>>>>>> API as
> >>>>>>>>>> it's base
> >>>>>>>>> I want to disect this a bit to find out exactly where
> there
> >>>>>>>>> might be
> >>>>>>>>> agreement
> >>>>>>>>> and disagreement.
> >>>>>>>>>
> >>>>>>>>> C API is a good thing to implement - Agreed.
> >>>>>>>>>
> >>>>>>>>> I also want to use gobject introspection but I
> don't agree
> >>>>>>>>> that using
> >>>>>>>>> glib
> >>>>>>>>> precludes the use of a formalized schema.  My
> proposal is that
> >>>>>>>>> we
> >>>>>>>>> write a schema
> >>>>>>>>> definition and generate the glib C code from that
> schema.
> >>>>>>>>>
> >>>>>>>>> I agree that the _current_ xmlrpc API makes a pretty
> bad base
> >>>>>>>>> from
> >>>>>>>>> which to
> >>>>>>>>> start a supportable API.  XMLRPC is a perfectly
> reasonable
> >>>>>>>>> remote/wire protocol
> >>>>>>>>> and I think we should continue using it as a base for
> the next
> >>>>>>>>> generation API.
> >>>>>>>>> Using a schema will ensure that the new API is
> >>>>>>>>> well-structured.
> >>>>>>>> There major problems with XML-RPC (and to some extent
> with REST
> >>>>>>>> as
> >>>>>>>> well) are high call overhead and no two way communication
> (push
> >>>>>>>> events). Basing on XML-RPC means that we will never be
> able to
> >>>>>>>> solve
> >>>>>>>> these issues.
> >>>>>>> I am not sure I am ready to conceed that XML-RPC is too slow
> for
> >>>>>>> our
> >>>>>>> needs.  Can
> >>>>>>> you provide some more detail around this point and possibly
> >>>>>>> suggest an
> >>>>>>> alternative that has even lower overhead without sacrificing
> the
> >>>>>>> ubiquity and
> >>>>>>> usability of XML-RPC?  As far as the two-way communication
> >>>>>>> point, what
> >>>>>>> are the
> >>>>>>> options besides AMQP/ZeroMQ?  Aren't these even worse
> from an
> >>>>>>> overhead
> >>>>>>> perspective than XML-RPC?  Regarding two-way communication:
> you
> >>>>>>> can
> >>>>>>> write AMQP
> >>>>>>> brokers based on the C API and run one on each vdsm host.
> >>>>>>>   Assuming
> >>>>>>> the C API
> >>>>>>> supports events, what else would you need?
> >>>>>> I personally think that using something like AMQP for inter-node
> >>>>>> communication and engine - node would be optimal.  With a rest
> >>>>>> interface
> >>>>>> that just send messages though something like AMQP.
> >>>>> I would also not dismiss AMQP so soon
> >>>>> we want a bug with more than a single listener at engine side
> >>>>> (engine, history db, maybe event correlation service).
> >>>>> collectd as a means for statistics already supports it as well.
> >>>>> I'm for having REST as well, but not sure as main one for a
> >>>>> consumer
> >>>>> like ovirt engine.
> >>>> I agree that a message bus could be a very useful model of
> >>>> communication between
> >>>> ovirt-engine components and multiple vdsm instances.  But the
> >>>> complexities and
> >>>> dependencies of AMQP do not make it suitable for use as a low-level
> >>>> API.  AMQP
> >>>> will repel new adopters.  Why not establish a libvdsm that is more
> >>>> minimalist
> >>>> and can be easily used by everyone?  Then AMQP brokers can be built
> >>>> on top of
> >>>> the stable API with ease.  All AMQP should require of the low-level
> >>>> API are
> >>>> standard function calls and an events mechanism.
> >>>>
> >>>>>> Thanks
> >>>>>> Robert
> >>>>>>>>>> The current XML-RPC API contains a lot of
> decencies and
> >>>>>>>>>> inefficiencies and we
> >>>>>>>>>> would like to retire it as soon as we possibly
> can. Engine
> >>>>>>>>>> would
> >>>>>>>>>> like us to
> >>>>>>>>>> move to a message based API and 3rd parties want
> something
> >>>>>>>>>> simple
> >>>>>>>>>> like REST so
> >>>>>>>>>> it looks like no one actually wants to use
> XML-RPC. Not even
> >>>>>>>>>> us.
> >>>>>>>>> I am proposing that AMQP brokers and REST APIs could
> be
> >>>>>>>>> written
> >>>>>>>>> against the
> >>>>>>>>> public API.  In fact, they need not even live in the
> vdsm tree
> >>>>>>>>> anymore if that
> >>>>>>>>> is what we choose.  Core vdsm would only be
> responsible for
> >>>>>>>>> providing
> >>>>>>>>> libvdsm
> >>>>>>>>> and whatever language bindings we want to support.
> >>>>>>>> If we take the libvdsm route, the only reason to even
> have a
> >>>>>>>> REST
> >>>>>>>> bridge is only to support OSes other then Linux which is
> >>>>>>>> something
> >>>>>>>> I'm not sure we care about at the moment.
> >>>>>>> That might be true regarding the current in-tree
> implementation.
> >>>>>>> However, I can
> >>>>>>> almost guarantee that someone wanting to write a web GUI on
> top
> >>>>>>> of
> >>>>>>> standalone
> >>>>>>> vdsm would want a REST API to talk to.  But libvdsm makes
> this
> >>>>>>> use
> >>>>>>> case of no
> >>>>>>> concern to the core vdsm developers.
> >>>>>>>
> >>>>>>>>>> I do think that having C supportability in our
> API is a good
> >>>>>>>>>> idea,
> >>>>>>>>>> but the
> >>>>>>>>>> current API should not be used as the base.
> >>>>>>>>> Let's _start_ with a schema document that
> describes today's
> >>>>>>>>> API and
> >>>>>>>>> then clean
> >>>>>>>>> it up.  I think that will work better than starting
> from
> >>>>>>>>> scratch.
> >>>>>>>>>    Once my
> >>>>>>>>> schema is written I will post it and we can
> 'patch' it as a
> >>>>>>>>> community
> >>>>>>>>> until we
> >>>>>>>>> arrive at a 1.0 version we are all happy with.
> >>>>>>>> +1
> >>>>>>> Ok.  Redoubling my efforts to get this done.  Describing the
> >>>>>>> output of
> >>>>>>> list(True) takes awhile :)
> >>>>>>>
> >>>>>> _______________________________________________
> >>>>>> vdsm-devel mailing list
> >>>>>> vdsm-devel(a)lists.fedorahosted.org
> >>>>>> https://fedorahosted.org/mailman/listinfo/vdsm-devel
> >>>>> _______________________________________________
> >>>>> vdsm-devel mailing list
> >>>>> vdsm-devel(a)lists.fedorahosted.org
> >>>>> https://fedorahosted.org/mailman/listinfo/vdsm-devel
> >>> --
> >>> Shu Ming <shuming(a)linux.vnet.ibm.com&gt;
> >>> IBM China Systems and Technology Laboratory
> >>>
> >>>
> >>> _______________________________________________
> >>> vdsm-devel mailing list
> >>> vdsm-devel(a)lists.fedorahosted.org
> >>> https://fedorahosted.org/mailman/listinfo/vdsm-devel
> >>>
> >
> > -- 
> > Shu Ming <shuming(a)linux.vnet.ibm.com&gt;
> > IBM China Systems and Technology Laboratory
> >
> > 

-- 
Shu Ming <shuming(a)linux.vnet.ibm.com&gt;
IBM China Systems and Technology Laboratory

On Thu, Jul 12, 2012 at 11:13:18PM +0800, Shu Ming wrote:
>  On 2012-7-12 20:41, Adam Litke wrote:
>  >On Thu, Jul 12, 2012 at 08:11:17AM +0800, Shu Ming wrote:
>  >>Basically,  my understanding is that we can generate two versions of
>  >>libvdsm from the schema file for both the node and the management
>  >>application.  First, the transportation protocols(XMLRPC, REST-API)
>  >>will depend on libvdsm(node version) to export the APIs to remote
>  >>management application.  Secondly, the management application can
>  >>use libvdsm(application version ) to emit the remote call to the
>  >>node.   Also, transportation protocols like REST API and XML RPC API
>  >>can also be generated automatically by the schema file with C, Java,
>  >>Python bindings.
>  >I think this might be a bit too complex of a model.  Here's how I see it...
>  >
>  >The schema generates C/gObject code which can be compiled into libvdsm.  We can
>  >use the gObject introspection library to automatically generate language
>  >bindings for Java, Python, Perl, etc.
>  Do you mean Java, Python, Perl, etc bindings of libvdsm? 
Yes.

>  >The libvdsm library talks to vdsmd using a wire protocol that
> works locally and
>  >remotely.  This wire protocol is completely hidden from library users.  It's an
>  >implementation detail that can be changed later if necessary.  Today I would
>  >recommend that we use xmlrpc.  This means that ovirt-engine or another remote
>  >program could use libvdsm in the exact same manner as a local program.  The
>  >library user just needs to call libvdsm.connect(uri).
>  
>  Except libvdsm.connet() or libvdsm.disconnet(), do you expect the
>  engine will not change any of its xml-rpc interfaces now after
>  libvdsm is introduced?  I mean engine may not want to change much of
>  its current code, so it expect the least change. 
You are right that vdsm must preserve backwards compatibility.  ovirt-engine
will not want to rewrite all of its vdsm interaction code all at once.  For this
reason, we will need to keep today's xmlrpc interface around until it can be
deprecated.

There is another issue to consider with regards to backwards compatibility.
Once application developers start writing code against libvdsm, they will want
to target new and old vdsm versions.  For example, they will want their code to
run correctly against oVirt 3.0 clusters and oVirt 4.0 clusters.  In order to do
this, we need to think of how libvdsm can talk to today's vdsm.

>  >
>  >Finally, REST and AMQP bridges would be written solely against libvdsm.  These
>  
>  So we should write REST, AMQP bridges manually with Java, Python,
>  Perl language on various bindings of libvdsm? 
No!  REST and AMQP should not be thought of as transports but more like language
bindings themselves.  The only reason an API consumer would want REST is because
they want to speak REST directly.  This would be the case for a simple web UI
where the client-side AJAX code wants to query the vdsm API.  If C/Java bindings
are preferred, then the application would just use the regular libvdsm language
bindings.

>  >bridges are probably not suitable for code generation (but we can
> revisit that
>  >as a separate issue because it's up to the bridge writer to determine the best
>  >approach).
>  >
>  >>On 2012-7-12 2:29, Saggi Mizrahi wrote:
>  >>>I'm sorry, but I don't really understand the drawing
>  >>>
>  >>>----- Original Message -----
>  >>>>From: "Shu Ming" <shuming(a)linux.vnet.ibm.com&gt;
>  >>>>To: "Adam Litke" <agl(a)us.ibm.com&gt;
>  >>>>Cc: vdsm-devel(a)lists.fedorahosted.org
>  >>>>Sent: Wednesday, July 11, 2012 10:24:49 AM
>  >>>>Subject: Re: [vdsm] [RFC] An alternative way to provide a supported
> interface -- libvdsm
>  >>>>
>  >>>>Adam,
>  >>>>
>  >>>>Maybe,  I don't fully understand your proposal.  Here is my
>  >>>>understanding of libvdsm in the picture. Please check the following
>  >>>>link
>  >>>>for the picture.
>  >>>>
>  >>>>http://www.ovirt.org/wiki/File:Libvdsm.JPG
>  >>>>
>  >>>>
>  >>>>http://www.ovirt.org/wiki/File:Libvdsm.JPG
>  >>>>
>  >>>>On 2012-7-9 21:56, Adam Litke wrote:
>  >>>>>On Fri, Jul 06, 2012 at 03:53:08PM +0300, Itamar Heim wrote:
>  >>>>>>On 07/06/2012 01:15 AM, Robert Middleswarth wrote:
>  >>>>>>>On 07/05/2012 04:45 PM, Adam Litke wrote:
>  >>>>>>>>On Thu, Jul 05, 2012 at 03:47:42PM -0400, Saggi Mizrahi
> wrote:
>  >>>>>>>>>----- Original Message -----
>  >>>>>>>>>>From: "Adam Litke"
> <agl(a)us.ibm.com&gt;
>  >>>>>>>>>>To: "Saggi Mizrahi"
> <smizrahi(a)redhat.com&gt;
>  >>>>>>>>>>Cc: "Anthony Liguori"
> <anthony(a)codemonkey.ws&gt;, "VDSM Project
>  >>>>>>>>>>Development"
> <vdsm-devel(a)lists.fedorahosted.org&gt;
>  >>>>>>>>>>Sent: Thursday, July 5, 2012 2:34:50 PM
>  >>>>>>>>>>Subject: Re: [RFC] An alternative way to provide
> a supported
>  >>>>>>>>>>interface -- libvdsm
>  >>>>>>>>>>
>  >>>>>>>>>>On Wed, Jun 27, 2012 at 02:50:02PM -0400, Saggi
> Mizrahi wrote:
>  >>>>>>>>>>>The idea of having a supported C API was
> something I was
>  >>>>>>>>>>>thinking
>  >>>>>>>>>>>about doing
>  >>>>>>>>>>>(But I'd rather use gobject introspection
> and not schema
>  >>>>>>>>>>>generation) But the
>  >>>>>>>>>>>problem is not having a C API is using the
> current XML RPC
>  >>>>>>>>>>>API as
>  >>>>>>>>>>>it's base
>  >>>>>>>>>>I want to disect this a bit to find out exactly
> where there
>  >>>>>>>>>>might be
>  >>>>>>>>>>agreement
>  >>>>>>>>>>and disagreement.
>  >>>>>>>>>>
>  >>>>>>>>>>C API is a good thing to implement - Agreed.
>  >>>>>>>>>>
>  >>>>>>>>>>I also want to use gobject introspection but I
> don't agree
>  >>>>>>>>>>that using
>  >>>>>>>>>>glib
>  >>>>>>>>>>precludes the use of a formalized schema.  My
> proposal is that
>  >>>>>>>>>>we
>  >>>>>>>>>>write a schema
>  >>>>>>>>>>definition and generate the glib C code from that
> schema.
>  >>>>>>>>>>
>  >>>>>>>>>>I agree that the _current_ xmlrpc API makes a
> pretty bad base
>  >>>>>>>>>>from
>  >>>>>>>>>>which to
>  >>>>>>>>>>start a supportable API.  XMLRPC is a perfectly
> reasonable
>  >>>>>>>>>>remote/wire protocol
>  >>>>>>>>>>and I think we should continue using it as a base
> for the next
>  >>>>>>>>>>generation API.
>  >>>>>>>>>>Using a schema will ensure that the new API is
>  >>>>>>>>>>well-structured.
>  >>>>>>>>>There major problems with XML-RPC (and to some extent
> with REST
>  >>>>>>>>>as
>  >>>>>>>>>well) are high call overhead and no two way
> communication (push
>  >>>>>>>>>events). Basing on XML-RPC means that we will never
> be able to
>  >>>>>>>>>solve
>  >>>>>>>>>these issues.
>  >>>>>>>>I am not sure I am ready to conceed that XML-RPC is too
> slow for
>  >>>>>>>>our
>  >>>>>>>>needs.  Can
>  >>>>>>>>you provide some more detail around this point and
> possibly
>  >>>>>>>>suggest an
>  >>>>>>>>alternative that has even lower overhead without
> sacrificing the
>  >>>>>>>>ubiquity and
>  >>>>>>>>usability of XML-RPC?  As far as the two-way
> communication
>  >>>>>>>>point, what
>  >>>>>>>>are the
>  >>>>>>>>options besides AMQP/ZeroMQ?  Aren't these even worse
> from an
>  >>>>>>>>overhead
>  >>>>>>>>perspective than XML-RPC?  Regarding two-way
> communication: you
>  >>>>>>>>can
>  >>>>>>>>write AMQP
>  >>>>>>>>brokers based on the C API and run one on each vdsm
> host.
>  >>>>>>>>  Assuming
>  >>>>>>>>the C API
>  >>>>>>>>supports events, what else would you need?
>  >>>>>>>I personally think that using something like AMQP for
> inter-node
>  >>>>>>>communication and engine - node would be optimal.  With a
> rest
>  >>>>>>>interface
>  >>>>>>>that just send messages though something like AMQP.
>  >>>>>>I would also not dismiss AMQP so soon
>  >>>>>>we want a bug with more than a single listener at engine side
>  >>>>>>(engine, history db, maybe event correlation service).
>  >>>>>>collectd as a means for statistics already supports it as well.
>  >>>>>>I'm for having REST as well, but not sure as main one for a
>  >>>>>>consumer
>  >>>>>>like ovirt engine.
>  >>>>>I agree that a message bus could be a very useful model of
>  >>>>>communication between
>  >>>>>ovirt-engine components and multiple vdsm instances.  But the
>  >>>>>complexities and
>  >>>>>dependencies of AMQP do not make it suitable for use as a low-level
>  >>>>>API.  AMQP
>  >>>>>will repel new adopters.  Why not establish a libvdsm that is more
>  >>>>>minimalist
>  >>>>>and can be easily used by everyone?  Then AMQP brokers can be built
>  >>>>>on top of
>  >>>>>the stable API with ease.  All AMQP should require of the low-level
>  >>>>>API are
>  >>>>>standard function calls and an events mechanism.
>  >>>>>
>  >>>>>>>Thanks
>  >>>>>>>Robert
>  >>>>>>>>>>>The current XML-RPC API contains a lot of
> decencies and
>  >>>>>>>>>>>inefficiencies and we
>  >>>>>>>>>>>would like to retire it as soon as we
> possibly can. Engine
>  >>>>>>>>>>>would
>  >>>>>>>>>>>like us to
>  >>>>>>>>>>>move to a message based API and 3rd parties
> want something
>  >>>>>>>>>>>simple
>  >>>>>>>>>>>like REST so
>  >>>>>>>>>>>it looks like no one actually wants to use
> XML-RPC. Not even
>  >>>>>>>>>>>us.
>  >>>>>>>>>>I am proposing that AMQP brokers and REST APIs
> could be
>  >>>>>>>>>>written
>  >>>>>>>>>>against the
>  >>>>>>>>>>public API.  In fact, they need not even live in
> the vdsm tree
>  >>>>>>>>>>anymore if that
>  >>>>>>>>>>is what we choose.  Core vdsm would only be
> responsible for
>  >>>>>>>>>>providing
>  >>>>>>>>>>libvdsm
>  >>>>>>>>>>and whatever language bindings we want to
> support.
>  >>>>>>>>>If we take the libvdsm route, the only reason to even
> have a
>  >>>>>>>>>REST
>  >>>>>>>>>bridge is only to support OSes other then Linux which
> is
>  >>>>>>>>>something
>  >>>>>>>>>I'm not sure we care about at the moment.
>  >>>>>>>>That might be true regarding the current in-tree
> implementation.
>  >>>>>>>>However, I can
>  >>>>>>>>almost guarantee that someone wanting to write a web GUI
> on top
>  >>>>>>>>of
>  >>>>>>>>standalone
>  >>>>>>>>vdsm would want a REST API to talk to.  But libvdsm makes
> this
>  >>>>>>>>use
>  >>>>>>>>case of no
>  >>>>>>>>concern to the core vdsm developers.
>  >>>>>>>>
>  >>>>>>>>>>>I do think that having C supportability in
> our API is a good
>  >>>>>>>>>>>idea,
>  >>>>>>>>>>>but the
>  >>>>>>>>>>>current API should not be used as the base.
>  >>>>>>>>>>Let's _start_ with a schema document that
> describes today's
>  >>>>>>>>>>API and
>  >>>>>>>>>>then clean
>  >>>>>>>>>>it up.  I think that will work better than
> starting from
>  >>>>>>>>>>scratch.
>  >>>>>>>>>>   Once my
>  >>>>>>>>>>schema is written I will post it and we can
> 'patch' it as a
>  >>>>>>>>>>community
>  >>>>>>>>>>until we
>  >>>>>>>>>>arrive at a 1.0 version we are all happy with.
>  >>>>>>>>>+1
>  >>>>>>>>Ok.  Redoubling my efforts to get this done.  Describing
> the
>  >>>>>>>>output of
>  >>>>>>>>list(True) takes awhile :)
>  >>>>>>>>
>  >>>>>>>_______________________________________________
>  >>>>>>>vdsm-devel mailing list
>  &gt;&gt;&gt;&gt;&gt;&gt;&gt;vdsm-devel(a)lists.fedorahosted.org
>  >>>>>>>https://fedorahosted.org/mailman/listinfo/vdsm-devel
>  >>>>>>_______________________________________________
>  >>>>>>vdsm-devel mailing list
>  &gt;&gt;&gt;&gt;&gt;&gt;vdsm-devel(a)lists.fedorahosted.org
>  >>>>>>https://fedorahosted.org/mailman/listinfo/vdsm-devel
>  >>>>--
>  >>>>Shu Ming <shuming(a)linux.vnet.ibm.com&gt;
>  >>>>IBM China Systems and Technology Laboratory
>  >>>>
>  >>>>
>  >>>>_______________________________________________
>  >>>>vdsm-devel mailing list
>  &gt;&gt;&gt;&gt;vdsm-devel(a)lists.fedorahosted.org
>  >>>>https://fedorahosted.org/mailman/listinfo/vdsm-devel
>  >>>>
>  >>
>  >>-- 
>  >>Shu Ming <shuming(a)linux.vnet.ibm.com&gt;
>  >>IBM China Systems and Technology Laboratory
>  >>
>  >>
>  
>  
>  -- 
>  Shu Ming <shuming(a)linux.vnet.ibm.com&gt;
>  IBM China Systems and Technology Laboratory
>  
>   
-- 
Adam Litke <agl(a)us.ibm.com&gt;
IBM Linux Technology Center