Re: [vdsm] Migration regression on master

 | ----- Original Message -----
| | From: "Assaf Muller" <amuller(a)redhat.com&gt;
| | To: "Michal Skrivanek" <michal.skrivanek(a)redhat.com&gt;
| | Cc: &quot;vdsm-devel(a)lists.fedorahosted.org Development"
| | <vdsm-devel(a)lists.fedorahosted.org&gt;
| | Sent: Thursday, May 23, 2013 2:12:47 PM
| | Subject: Re: [vdsm] Migration regression on master
| | 
| | As you can see in a previous patch set I checked if the alias attribute
| | exists instead of assuming it exists.
| | I then changed my mind with Dan's blessing, and decided to assume it does
| | exist, exactly for cases like this.
| | Even if we check if the alias exists, what do we do if we find out it
| | doesn't? We're at a problem and need to understand why the alias doesn't
| | exist because it should - For all devices.
| | 
| | We definitely need to deal with this issue - Can you provide the domxml of
| | the VM during creation, and during migration?
 
Hi Assaf, Peter today has reproduced the same error and provided me
the output log, you can find it here:
http://etherpad.ovirt.org/p/migration-errors
a previous patch blocked by this issue has been accepted
(http://gerrit.ovirt.org/#/c/14203/),
I've been able to do a migration using the engine.
Feel free to ask if you need more details,
I will look into this issue in the next few days.
 
Giuseppe
 
| 
| | 
| | ----- Original Message -----
| | From: "Michal Skrivanek" <michal.skrivanek(a)redhat.com&gt;
| | To: "Vinzenz Feenstra" <vfeenstr(a)redhat.com&gt;, "Assaf
Muller"
| | <amuller(a)redhat.com&gt;
| | Cc: &quot;vdsm-devel(a)lists.fedorahosted.org Development"
| | <vdsm-devel(a)lists.fedorahosted.org&gt;, "Dan Kenigsberg"
<danken(a)redhat.com&gt;
| | Sent: Wednesday, May 22, 2013 3:56:22 PM
| | Subject: Re: [vdsm] Migration regression on master
| | 
| | 
| | On May 22, 2013, at 10:14 , Vinzenz Feenstra <vfeenstr(a)redhat.com&gt; wrote:
| | 
| | > Hi,
| | > 
| | > During the verification of the refactoring in VDSM (libvirtvm + vm == vm)
| | > we have encountered some issue.
| | > There seems to be a regression on master branch regarding device aliases.
| | > 
| | > On the target machine _updateDevicesDomxmlCache accesses dev.alias
| | > without
| | > the attribute being present.
| | > 
| | > This happens to happen at least on balloon devices.
| | > I am not sure why we do have this, however it is reproducible and I am
| | > not
| | > sure why we do this now.
| | > 
| | > I know it has been introduced by http://gerrit.ovirt.org/#/c/13876 and
| | > it's
| | > actually in the commit message that we'd fail.
| | :-) you mean:
| | * We now crash horribly if an alias was not found for a device
| | when updating devices domxml cache after migrating
| | 
| | > Which is the case... So can somebody please enlighten me and tell me if
| | > this is a configuration issue of the VM or if this really a regression.
| | not all devices have alias so I think this is indeed a regression
| | 
| | 
| | > It's possible that the tests aren't properly configured, however that
| | > would
| | > from my understanding mean that we potentially will run into troubles
| | > during migrations from 3.2 -> 3.3 vdsm.
| | > 
| | > PS:
| | > This kind of blocks a bit the refactoring verification :/
| | > 
| | > --
| | > Regards,
| | > 
| | > Vinzenz Feenstra | Senior Software Engineer
| | > RedHat Engineering Virtualization R & D
| | > Phone: [phone]
| | > IRC: vfeenstr or evilissimo
| | > 
| | > Better technology. Faster innovation. Powered by community collaboration.
| | > See how it works at redhat.com
| | > 
| | > _______________________________________________
| | > vdsm-devel mailing list
| | > vdsm-devel(a)lists.fedorahosted.org
| | > https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
| | 
| | _______________________________________________
| | vdsm-devel mailing list
| | vdsm-devel(a)lists.fedorahosted.org
| | https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
| | 
|

Migration regression on master

[Dan Kenigsberg]

On Tue, May 28, 2013 at 11:54:45AM -0400, Giuseppe Vallarelli wrote:
>   | ----- Original Message -----
>  | | From: "Assaf Muller" <amuller(a)redhat.com&gt;
>  | | To: "Michal Skrivanek" <michal.skrivanek(a)redhat.com&gt;
>  | | Cc: &quot;vdsm-devel(a)lists.fedorahosted.org Development"
>  | | <vdsm-devel(a)lists.fedorahosted.org&gt;
>  | | Sent: Thursday, May 23, 2013 2:12:47 PM
>  | | Subject: Re: [vdsm] Migration regression on master
>  | | 
>  | | As you can see in a previous patch set I checked if the alias attribute
>  | | exists instead of assuming it exists.
>  | | I then changed my mind with Dan's blessing, and decided to assume it does
>  | | exist, exactly for cases like this.
>  | | Even if we check if the alias exists, what do we do if we find out it
>  | | doesn't? We're at a problem and need to understand why the alias doesn't
>  | | exist because it should - For all devices.
>  | | 
>  | | We definitely need to deal with this issue - Can you provide the domxml of
>  | | the VM during creation, and during migration?
>   
>  Hi Assaf, Peter today has reproduced the same error and provided me
>  the output log, you can find it here:
>  http://etherpad.ovirt.org/p/migration-errors 
Would you add the original vmCreate line, and domxml, that was used to
create the VM at the source host?

>  a previous patch blocked by this issue has been accepted
>  (http://gerrit.ovirt.org/#/c/14203/),
>  I've been able to do a migration using the engine.
>  Feel free to ask if you need more details,
>  I will look into this issue in the next few days.
>   
>  Giuseppe

Migration regression on master

On 05/29/2013 07:17 AM, Dan Kenigsberg wrote:
>  On Tue, May 28, 2013 at 11:54:45AM -0400, Giuseppe Vallarelli wrote:
> >   | ----- Original Message -----
> > | | From: "Assaf Muller" <amuller(a)redhat.com&gt;
> > | | To: "Michal Skrivanek" <michal.skrivanek(a)redhat.com&gt;
> > | | Cc: &quot;vdsm-devel(a)lists.fedorahosted.org Development"
> > | | <vdsm-devel(a)lists.fedorahosted.org&gt;
> > | | Sent: Thursday, May 23, 2013 2:12:47 PM
> > | | Subject: Re: [vdsm] Migration regression on master
> > | |
> > | | As you can see in a previous patch set I checked if the alias attribute
> > | | exists instead of assuming it exists.
> > | | I then changed my mind with Dan's blessing, and decided to assume it does
> > | | exist, exactly for cases like this.
> > | | Even if we check if the alias exists, what do we do if we find out it
> > | | doesn't? We're at a problem and need to understand why the alias
> doesn't
> > | | exist because it should - For all devices.
> > | |
> > | | We definitely need to deal with this issue - Can you provide the domxml of
> > | | the VM during creation, and during migration?
> >
> > Hi Assaf, Peter today has reproduced the same error and provided me
> > the output log, you can find it here:
> > http://etherpad.ovirt.org/p/migration-errors
>
>  Would you add the original vmCreate line, and domxml, that was used to
>  create the VM at the source host? 
http://pastebin.test.redhat.com/144447
http://pastebin.test.redhat.com/144446

-- 
Peter V. Saveliev

Migration regression on master

[Dan Kenigsberg]

On Wed, May 29, 2013 at 02:13:02PM +0200, Peter V. Saveliev wrote:
>  On 05/29/2013 07:17 AM, Dan Kenigsberg wrote:
>  >On Tue, May 28, 2013 at 11:54:45AM -0400, Giuseppe Vallarelli wrote:
>  >>  | ----- Original Message -----
>  >>| | From: "Assaf Muller" <amuller(a)redhat.com&gt;
>  >>| | To: "Michal Skrivanek" <michal.skrivanek(a)redhat.com&gt;
>  >>| | Cc: &quot;vdsm-devel(a)lists.fedorahosted.org Development"
>  >>| | <vdsm-devel(a)lists.fedorahosted.org&gt;
>  >>| | Sent: Thursday, May 23, 2013 2:12:47 PM
>  >>| | Subject: Re: [vdsm] Migration regression on master
>  >>| |
>  >>| | As you can see in a previous patch set I checked if the alias attribute
>  >>| | exists instead of assuming it exists.
>  >>| | I then changed my mind with Dan's blessing, and decided to assume it
> does
>  >>| | exist, exactly for cases like this.
>  >>| | Even if we check if the alias exists, what do we do if we find out it
>  >>| | doesn't? We're at a problem and need to understand why the alias
> doesn't
>  >>| | exist because it should - For all devices.
>  >>| |
>  >>| | We definitely need to deal with this issue - Can you provide the domxml of
>  >>| | the VM during creation, and during migration?
>  >>
>  >>Hi Assaf, Peter today has reproduced the same error and provided me
>  >>the output log, you can find it here:
>  >>http://etherpad.ovirt.org/p/migration-errors
>  >
>  >Would you add the original vmCreate line, and domxml, that was used to
>  >create the VM at the source host?
>  
>  http://pastebin.test.redhat.com/144447
>  http://pastebin.test.redhat.com/144446 
That's an internal pastebin instance. Could you simply inline the
answer? Hyperlinks makes a casual reader of the mailing list not
understand what's going on.

Note that we need vmCreate command, the response is less
interesting, thow seeing the whole vdsm thread would not harm.

Dan.

Migration regression on master

    * We still need to see the vmCreate command used to learn more about the issue 
    * We can always implement a hack that checks if the alias attribute exists. However,
there is probably a deeper issue at hand and this is a good opportunity to find and solve
it 
    * Dan has a theory that the issue is with migrating a VM from a 3.0 VDSM version host
to a 3.2 host. Perhaps the balloon device isn't created properly at the source host 

From: "Dan Kenigsberg" <danken(a)redhat.com&gt; 
To: "Peter V. Saveliev" <peet(a)redhat.com&gt; 
Cc: vdsm-devel(a)lists.fedorahosted.org 
Sent: Wednesday, May 29, 2013 4:41:24 PM 
Subject: Re: [vdsm] Migration regression on master 

On Wed, May 29, 2013 at 02:13:02PM +0200, Peter V. Saveliev wrote: 
>  On 05/29/2013 07:17 AM, Dan Kenigsberg wrote: 
>  >On Tue, May 28, 2013 at 11:54:45AM -0400, Giuseppe Vallarelli wrote: 
>  >> | ----- Original Message ----- 
>  >>| | From: "Assaf Muller" <amuller(a)redhat.com&gt;
>
>  >>| | To: "Michal Skrivanek" <michal.skrivanek(a)redhat.com&gt; 
>  >>| | Cc: &quot;vdsm-devel(a)lists.fedorahosted.org Development" 
>  >>| | <vdsm-devel(a)lists.fedorahosted.org&gt; 
>  >>| | Sent: Thursday, May 23, 2013 2:12:47 PM 
>  >>| | Subject: Re: [vdsm] Migration regression on master 
>  >>| | 
>  >>| | As you can see in a previous patch set I checked if the alias attribute 
>  >>| | exists instead of assuming it exists. 
>  >>| | I then changed my mind with Dan's blessing, and decided to assume it does
>
>  >>| | exist, exactly for cases like this. 
>  >>| | Even if we check if the alias exists, what do we do if we find out it 
>  >>| | doesn't? We're at a problem and need to understand why the alias
> doesn't 
>  >>| | exist because it should - For all devices. 
>  >>| | 
>  >>| | We definitely need to deal with this issue - Can you provide the domxml of 
>  >>| | the VM during creation, and during migration? 
>  >> 
>  >>Hi Assaf, Peter today has reproduced the same error and provided me 
>  >>the output log, you can find it here: 
>  >>http://etherpad.ovirt.org/p/migration-errors 
>  > 
>  >Would you add the original vmCreate line, and domxml, that was used to 
>  >create the VM at the source host? 
>  
>  http://pastebin.test.redhat.com/144447 
>  http://pastebin.test.redhat.com/144446  
That's an internal pastebin instance. Could you simply inline the 
answer? Hyperlinks makes a casual reader of the mailing list not 
understand what's going on. 

Note that we need vmCreate command, the response is less 
interesting, thow seeing the whole vdsm thread would not harm. 

Dan. 
_______________________________________________ 
vdsm-devel mailing list 
vdsm-devel(a)lists.fedorahosted.org 
https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel

Migration regression on master

Interesting Theory.
IIRC in the past the balloon device was auto created by libvirt, and
at some point vdsm started removing it. So if the migration is from
an old version where the device is created to a new one which supports
a balloon created by the engine, it may be missing some parameters.
So indeed we need to understand more about the source vdsm and how
its VMs were created.

----- Original Message -----
>  From: "Assaf Muller" <amuller(a)redhat.com&gt;
>  To: "Dan Kenigsberg" <danken(a)redhat.com&gt;
>  Cc: vdsm-devel(a)lists.fedorahosted.org
>  Sent: Thursday, May 30, 2013 10:00:49 AM
>  Subject: Re: [vdsm] Migration regression on master
>  
>  
>  
>      * We still need to see the vmCreate command used to learn more about the
>      issue
>      * We can always implement a hack that checks if the alias attribute
>      exists. However, there is probably a deeper issue at hand and this is a
>      good opportunity to find and solve it
>      * Dan has a theory that the issue is with migrating a VM from a 3.0 VDSM
>      version host to a 3.2 host. Perhaps the balloon device isn't created
>      properly at the source host
>  
>  From: "Dan Kenigsberg" <danken(a)redhat.com&gt;
>  To: "Peter V. Saveliev" <peet(a)redhat.com&gt;
>  Cc: vdsm-devel(a)lists.fedorahosted.org
>  Sent: Wednesday, May 29, 2013 4:41:24 PM
>  Subject: Re: [vdsm] Migration regression on master
>  
>  On Wed, May 29, 2013 at 02:13:02PM +0200, Peter V. Saveliev wrote:
>  > On 05/29/2013 07:17 AM, Dan Kenigsberg wrote:
>  > >On Tue, May 28, 2013 at 11:54:45AM -0400, Giuseppe Vallarelli wrote:
>  > >> | ----- Original Message -----
>  
>  > >>| | From: "Assaf Muller" <amuller(a)redhat.com&gt;
>  > >>| | To: "Michal Skrivanek" <michal.skrivanek(a)redhat.com&gt;
>  > >>| | Cc: &quot;vdsm-devel(a)lists.fedorahosted.org Development"
>  > >>| | <vdsm-devel(a)lists.fedorahosted.org&gt;
>  > >>| | Sent: Thursday, May 23, 2013 2:12:47 PM
>  > >>| | Subject: Re: [vdsm] Migration regression on master
>  > >>| | 
>  > >>| | As you can see in a previous patch set I checked if the alias
>  > >>| | attribute
>  > >>| | exists instead of assuming it exists.
>  > >>| | I then changed my mind with Dan's blessing, and decided to assume
> it
>  > >>| | does
>  > >>| | exist, exactly for cases like this.
>  > >>| | Even if we check if the alias exists, what do we do if we find out it
>  > >>| | doesn't? We're at a problem and need to understand why the
> alias
>  > >>| | doesn't
>  > >>| | exist because it should - For all devices.
>  > >>| | 
>  > >>| | We definitely need to deal with this issue - Can you provide the
>  > >>| | domxml of
>  > >>| | the VM during creation, and during migration?
>  > >> 
>  > >>Hi Assaf, Peter today has reproduced the same error and provided me
>  > >>the output log, you can find it here:
>  > >>http://etherpad.ovirt.org/p/migration-errors
>  > > 
>  > >Would you add the original vmCreate line, and domxml, that was used to
>  > >create the VM at the source host?
>  > 
>  > http://pastebin.test.redhat.com/144447
>  > http://pastebin.test.redhat.com/144446
>  
>  That's an internal pastebin instance. Could you simply inline the
>  answer? Hyperlinks makes a casual reader of the mailing list not
>  understand what's going on.
>  
>  Note that we need vmCreate command, the response is less
>  interesting, thow seeing the whole vdsm thread would not harm.
>  
>  Dan.
>  _______________________________________________
>  vdsm-devel mailing list
>  vdsm-devel(a)lists.fedorahosted.org
>  https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
>  
>  
>  _______________________________________________
>  vdsm-devel mailing list
>  vdsm-devel(a)lists.fedorahosted.org
>  https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
>

Migration regression on master

----- Original Message -----
| From: "Assaf Muller" <amuller(a)redhat.com&gt;
| To: "Dan Kenigsberg" <danken(a)redhat.com&gt;
| Cc: "Peter V. Saveliev" <peet(a)redhat.com&gt;,
vdsm-devel(a)lists.fedorahosted.org, "Giuseppe Vallarelli"
| <gvallare(a)redhat.com&gt;, "michal skrivanek"
<michal.skrivanek(a)redhat.com&gt;
| Sent: Thursday, May 30, 2013 9:00:49 AM
| Subject: Re: [vdsm] Migration regression on master
| 
| 
| 
|     * We still need to see the vmCreate command used to learn more about the
|     issue
|     * We can always implement a hack that checks if the alias attribute
|     exists. However, there is probably a deeper issue at hand and this is a
|     good opportunity to find and solve it
|     * Dan has a theory that the issue is with migrating a VM from a 3.0 VDSM
|     version host to a 3.2 host. Perhaps the balloon device isn't created
|     properly at the source host

I've been able to reproduce the failure here there are more details:

On the source host

* vmCreate http://pastebin.com/yEZ3zkyN
* vm xml http://pastebin.com/YSy4TYca

On the dest host

* vmMigrationCreate http://pastebin.com/rwdPC8Ef
* _srcDomXml nicely formatted http://pastebin.com/pjE8sMMW

Giuseppe


| 
| From: "Dan Kenigsberg" <danken(a)redhat.com&gt;
| To: "Peter V. Saveliev" <peet(a)redhat.com&gt;
| Cc: vdsm-devel(a)lists.fedorahosted.org
| Sent: Wednesday, May 29, 2013 4:41:24 PM
| Subject: Re: [vdsm] Migration regression on master
| 
| On Wed, May 29, 2013 at 02:13:02PM +0200, Peter V. Saveliev wrote:
| > On 05/29/2013 07:17 AM, Dan Kenigsberg wrote:
| > >On Tue, May 28, 2013 at 11:54:45AM -0400, Giuseppe Vallarelli wrote:
| > >> | ----- Original Message -----
| 
| > >>| | From: "Assaf Muller" <amuller(a)redhat.com&gt;
| > >>| | To: "Michal Skrivanek" <michal.skrivanek(a)redhat.com&gt;
| > >>| | Cc: &quot;vdsm-devel(a)lists.fedorahosted.org Development"
| > >>| | <vdsm-devel(a)lists.fedorahosted.org&gt;
| > >>| | Sent: Thursday, May 23, 2013 2:12:47 PM
| > >>| | Subject: Re: [vdsm] Migration regression on master
| > >>| | 
| > >>| | As you can see in a previous patch set I checked if the alias
| > >>| | attribute
| > >>| | exists instead of assuming it exists.
| > >>| | I then changed my mind with Dan's blessing, and decided to assume
it
| > >>| | does
| > >>| | exist, exactly for cases like this.
| > >>| | Even if we check if the alias exists, what do we do if we find out it
| > >>| | doesn't? We're at a problem and need to understand why the
alias
| > >>| | doesn't
| > >>| | exist because it should - For all devices.
| > >>| | 
| > >>| | We definitely need to deal with this issue - Can you provide the
| > >>| | domxml of
| > >>| | the VM during creation, and during migration?
| > >> 
| > >>Hi Assaf, Peter today has reproduced the same error and provided me
| > >>the output log, you can find it here:
| > >>http://etherpad.ovirt.org/p/migration-errors
| > > 
| > >Would you add the original vmCreate line, and domxml, that was used to
| > >create the VM at the source host?
| > 
| > http://pastebin.test.redhat.com/144447
| > http://pastebin.test.redhat.com/144446
| 
| That's an internal pastebin instance. Could you simply inline the
| answer? Hyperlinks makes a casual reader of the mailing list not
| understand what's going on.
| 
| Note that we need vmCreate command, the response is less
| interesting, thow seeing the whole vdsm thread would not harm.
| 
| Dan.
| _______________________________________________
| vdsm-devel mailing list
| vdsm-devel(a)lists.fedorahosted.org
| https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
| 
|