More test coverage for Boxes

[Jiri Eischmann]

Hi,
Boxes is the pre-installed solution for VMs on Workstation, but it
doesn't get as much attention as it should.
Currently development composes of F24 don't even work in Boxes on F23,
so users who'd like to test F24 in the default VM app are out of luck.
It clearly violates one of the release criteria:https://fedoraproject.o
rg/wiki/Fedora_24_Beta_Release_Criteria#Guest_on_current_stable_release

The problem is that even though Boxes is the default VM app in
Workstation it's not among recommended Fedora virtualization tools the
criteria is referring to: https://fedoraproject.org/wiki/Getting_starte
d_with_virtualization
Fedora QA also use solely virt-manager and they only test Boxes from
time to time to make sure the basic functionality works.

Boxes have been here for some time, it's our default solution, we
should make sure it's the best working desktop VM on Fedora and work
with Fedora QA on ways to improve the test coverage. I think we should
start with adding Boxes to recommended virt tools. Does anyone have an
idea if it needs to go through some committee or is it just "go ahead
and edit" wiki page?

Jiri

[Cole Robinson]

On 04/14/2016 07:54 AM, Jiri Eischmann wrote:
>  Hi,
>  Boxes is the pre-installed solution for VMs on Workstation, but it
>  doesn't get as much attention as it should.
>  Currently development composes of F24 don't even work in Boxes on F23,
>  so users who'd like to test F24 in the default VM app are out of luck.
>  It clearly violates one of the release criteria:https://fedoraproject.o
>  rg/wiki/Fedora_24_Beta_Release_Criteria#Guest_on_current_stable_release
>   
More test coverage is good, but the larger problem may be that gnome-boxes
lacks some fedora dev love; The bug tracker is filled with dupes, the last f22
build was in july, last f23 build was in october

>  The problem is that even though Boxes is the default VM app in
>  Workstation it's not among recommended Fedora virtualization tools the
>  criteria is referring to: https://fedoraproject.org/wiki/Getting_starte
>  d_with_virtualization
>  Fedora QA also use solely virt-manager and they only test Boxes from
>  time to time to make sure the basic functionality works.
>   
The more complete picture AFAIK is that virt-manager is part of the workflow
for some fedora QA people, while gnome boxes is something that they (maybe)
test. virt-manager/virt-install have at least one feature that boxes lacks
that I know QA use often, and that's installing directly from a URL tree by
pulling down kernel/initrd. Saves having to fetch an iso for quick tests

>  Boxes have been here for some time, it's our default solution,
> we
>  should make sure it's the best working desktop VM on Fedora and work
>  with Fedora QA on ways to improve the test coverage. I think we should
>  start with adding Boxes to recommended virt tools. Does anyone have an
>  idea if it needs to go through some committee or is it just "go ahead
>  and edit" wiki page? 
No oversight needed, edit the page as you see fit. The virt wiki docs are in a
pretty sorry state, I've been meaning to clean them up for a long time

- Cole

[Matthias Clasen]

On Thu, 2016-04-14 at 09:41 -0400, Cole Robinson wrote:

>  The more complete picture AFAIK is that virt-manager is part of the
>  workflow
>  for some fedora QA people, while gnome boxes is something that they
>  (maybe)
>  test. virt-manager/virt-install have at least one feature that boxes
>  lacks
>  that I know QA use often, and that's installing directly from a URL
>  tree by
>  pulling down kernel/initrd. Saves having to fetch an iso for quick
>  tests 
Zeeshan, is that something that we could add to boxes ?

I'm afraid that qa people may very well make use of some of the
"toolbox" aspects of virt-manager that are not present in boxes - like
being able to set up very specific vm configurations in the UI.

But if we can make "boot this image in boxes" level qa easier by adding
a hidden "install from url" feature, we should maybe investigate that.

[Chris Murphy]

I go back and forth with virt-manager and Boxes. The top two problems
I'm having in Boxes that don't happen with virt-manager that relate to
QA testing:


GRUB2 doesn't respond to ctrl-x or f10
https://bugzilla.gnome.org/show_bug.cgi?id=764002

So I can't add boot parameters like rd.break or rd.debug or
systemd.log_level=debug and then F10 or CntrlX to boot. I can only
issue cntl-alt-back, cntrl-alt-f1,2,7. And sometimes the GUI isn't on
tty1, it's on 4, 5, 6 (like I think it is with netinstall?) once I
switch to tty2, I'm hosed, I can't go back to the GUI.

screen flicker at GDM, mouse click erratic response, black login screen
https://bugzilla.gnome.org/show_bug.cgi?id=763960

This is Fedora 24 not usable once installed; the live environment is
OK. I think the issue is that there isn't enough information to debug
this but I don't know how to get more information.

On Thu, Apr 14, 2016 at 5:54 AM, Jiri Eischmann <eischmann(a)redhat.com&gt; wrote:
>  Hi,
>  Boxes is the pre-installed solution for VMs on Workstation, but it
>  doesn't get as much attention as it should.
>  Currently development composes of F24 don't even work in Boxes on F23,
>  so users who'd like to test F24 in the default VM app are out of luck.
>  It clearly violates one of the release criteria:https://fedoraproject.o
>  rg/wiki/Fedora_24_Beta_Release_Criteria#Guest_on_current_stable_release 

I'll make a RHBZ of the screen flicker bug and nominate it. But as far
as I know that criterion applies to virt-manager. If it works there
but not Boxes, I doubt the bug would block. Maybe not even final
release (?).


Chris Murphy

[Chris Murphy]

On Thu, Apr 14, 2016 at 11:24 AM, Chris Murphy <lists(a)colorremedies.com&gt; wrote:

>  this but I don't know how to get more information.
>
>  On Thu, Apr 14, 2016 at 5:54 AM, Jiri Eischmann <eischmann(a)redhat.com&gt; wrote:
> > Hi,
> > Boxes is the pre-installed solution for VMs on Workstation, but it
> > doesn't get as much attention as it should.
> > Currently development composes of F24 don't even work in Boxes on F23,
> > so users who'd like to test F24 in the default VM app are out of luck.
> > It clearly violates one of the release criteria:https://fedoraproject.o
> > rg/wiki/Fedora_24_Beta_Release_Criteria#Guest_on_current_stable_release
>
>
>  I'll make a RHBZ of the screen flicker bug and nominate it. But as far
>  as I know that criterion applies to virt-manager. If it works there
>  but not Boxes, I doubt the bug would block. Maybe not even final
>  release (?). 
Nevermind. I'm not sure how to file this as a RHBZ because I don't
know if it's due to a problem in a Fedora 24 component that's only
acting out in Fedora 23 Boxes. Or if it's a Boxes bug. And if it's a
Boxes bug, that can't block F24.

-- 
Chris Murphy

[Matthias Clasen]

On Thu, 2016-04-14 at 11:24 -0600, Chris Murphy wrote:
>  
>  screen flicker at GDM, mouse click erratic response, black login
>  screen
>  https://bugzilla.gnome.org/show_bug.cgi?id=763960
>  
>  This is Fedora 24 not usable once installed; the live environment is
>  OK. I think the issue is that there isn't enough information to debug
>  this but I don't know how to get more information. 
When I had issue with flickering in boxes a while ago, it turned out to
be a spice issue:

https://bugzilla.redhat.com/show_bug.cgi?id=1266484

Any chance that yours is the same ?

[Chris Murphy]

On Thu, Apr 14, 2016 at 12:08 PM, Matthias Clasen <mclasen(a)redhat.com&gt; wrote:
>  On Thu, 2016-04-14 at 11:24 -0600, Chris Murphy wrote:
> >
> > screen flicker at GDM, mouse click erratic response, black login
> > screen
> > https://bugzilla.gnome.org/show_bug.cgi?id=763960
> >
> > This is Fedora 24 not usable once installed; the live environment is
> > OK. I think the issue is that there isn't enough information to debug
> > this but I don't know how to get more information.
>
>  When I had issue with flickering in boxes a while ago, it turned out to
>  be a spice issue:
>
>  https://bugzilla.redhat.com/show_bug.cgi?id=1266484
>
>  Any chance that yours is the same ? 
How would I find out? In my case, it was an X session always, not Wayland.

-- 
Chris Murphy

[Jiri Eischmann]

Jiri Eischmann píše v Čt 14. 04. 2016 v 13:54 +0200:
>  Hi,
>  Boxes is the pre-installed solution for VMs on Workstation, but it
>  doesn't get as much attention as it should.
>  Currently development composes of F24 don't even work in Boxes on
>  F23,
>  so users who'd like to test F24 in the default VM app are out of
>  luck.
>  It clearly violates one of the release criteria:https://fedoraproject
>  .o
>  rg/wiki/Fedora_24_Beta_Release_Criteria#Guest_on_current_stable_relea
>  se
>  
>  The problem is that even though Boxes is the default VM app in
>  Workstation it's not among recommended Fedora virtualization tools
>  the
>  criteria is referring to: https://fedoraproject.org/wiki/Getting_star
>  te
>  d_with_virtualization
>  Fedora QA also use solely virt-manager and they only test Boxes from
>  time to time to make sure the basic functionality works.
>  
>  Boxes have been here for some time, it's our default solution, we
>  should make sure it's the best working desktop VM on Fedora and work
>  with Fedora QA on ways to improve the test coverage. I think we
>  should
>  start with adding Boxes to recommended virt tools. Does anyone have
>  an
>  idea if it needs to go through some committee or is it just "go ahead
>  and edit" wiki page?
>  
>  Jiri
>  -- 
An addition to the original message: today I stumbled on a comment from
a user who was very disappointed with F24 Beta saying that even basic
things such as starting up an app didn't work for him in F24.
In the end, it turned out that it was because he was running Fedora in
GNOME Boxes. It was difficult to explain him that Fedora doesn't run
well in our default desktop VM tool and that F24 Beta is actually a
great release, he should give it another try in a different tool.

Jiri

[Michael Catanzaro]

On Wed, 2016-05-11 at 14:20 +0200, Jiri Eischmann wrote:
>  An addition to the original message: today I stumbled on a comment
>  from
>  a user who was very disappointed with F24 Beta saying that even basic
>  things such as starting up an app didn't work for him in F24.
>  In the end, it turned out that it was because he was running Fedora
>  in
>  GNOME Boxes. It was difficult to explain him that Fedora doesn't run
>  well in our default desktop VM tool and that F24 Beta is actually a
>  great release, he should give it another try in a different tool. 
Why are apps not starting? Is there a bug report? I've *never* had any
trouble running Fedora in Boxes. This sounds like a serious regression.

Michael

[Jiri Eischmann]

Michael Catanzaro píše v St 11. 05. 2016 v 08:03 -0500:
>  On Wed, 2016-05-11 at 14:20 +0200, Jiri Eischmann wrote:
>  > An addition to the original message: today I stumbled on a comment
>  > from
>  > a user who was very disappointed with F24 Beta saying that even
>  > basic
>  > things such as starting up an app didn't work for him in F24.
>  > In the end, it turned out that it was because he was running Fedora
>  > in
>  > GNOME Boxes. It was difficult to explain him that Fedora doesn't
>  > run
>  > well in our default desktop VM tool and that F24 Beta is actually a
>  > great release, he should give it another try in a different tool.
>  
>  Why are apps not starting? Is there a bug report? I've *never* had
>  any
>  trouble running Fedora in Boxes. This sounds like a serious
>  regression. 
F24 Alpha didn't work in F23's Boxes at all, not only for me, but for
others around me in the office.
Now, I'm already on F24 now and the experience is:
login screen is flickering and I can't click any button on the login
screen, I have to choose my account and log in using keyboard. When I
log in I get a black screen with only thing: clock on the top panel, by
trying I figured out that switching Boxes to full screen and clicking
randomly on the black screen I make it somehow re-render the screen and
it finally works.

Frankly, I don't know what info to collect to make a useful bug report.
I believe it can be hardware specific, but I've never had such issues
with virt-manager. The only long-term solution is to work with the QA
to make sure that it gets enough test coverage and provides the best VM
experience with Fedora. Right now, QA don't use Boxes because it's not
as useful to them as virt-manager and they're not sure if the bugs and
RFEs they report will be looked into and thus it worth the effort.

Jiri

[Matthias Clasen]

On Wed, 2016-05-11 at 16:22 +0200, Jiri Eischmann wrote:
>  Michael Catanzaro píše v St 11. 05. 2016 v 08:03 -0500:
>  > On Wed, 2016-05-11 at 14:20 +0200, Jiri Eischmann wrote:
>  > > An addition to the original message: today I stumbled on a
>  > > comment
>  > > from
>  > > a user who was very disappointed with F24 Beta saying that even
>  > > basic
>  > > things such as starting up an app didn't work for him in F24.
>  > > In the end, it turned out that it was because he was running
>  > > Fedora
>  > > in
>  > > GNOME Boxes. It was difficult to explain him that Fedora doesn't
>  > > run
>  > > well in our default desktop VM tool and that F24 Beta is actually
>  > > a
>  > > great release, he should give it another try in a different tool.
>  > 
>  > Why are apps not starting? Is there a bug report? I've *never* had
>  > any
>  > trouble running Fedora in Boxes. This sounds like a serious
>  > regression.
>  
>  F24 Alpha didn't work in F23's Boxes at all, not only for me, but for
>  others around me in the office.
>  Now, I'm already on F24 now and the experience is:
>  login screen is flickering and I can't click any button on the login
>  screen, I have to choose my account and log in using keyboard. When I
>  log in I get a black screen with only thing: clock on the top panel,
>  by
>  trying I figured out that switching Boxes to full screen and clicking
>  randomly on the black screen I make it somehow re-render the screen
>  and
>  it finally works. 
As far as I know, this is a spice <> wayland issue. Some fixes were
attempted in https://bugzilla.redhat.com/show_bug.cgi?id=1266484

[Chris Murphy]

On Wed, May 11, 2016 at 8:32 AM, Matthias Clasen <mclasen(a)redhat.com&gt; wrote:
>  On Wed, 2016-05-11 at 16:22 +0200, Jiri Eischmann wrote:
> > Michael Catanzaro píše v St 11. 05. 2016 v 08:03 -0500:
> > > On Wed, 2016-05-11 at 14:20 +0200, Jiri Eischmann wrote:
> > > > An addition to the original message: today I stumbled on a
> > > > comment
> > > > from
> > > > a user who was very disappointed with F24 Beta saying that even
> > > > basic
> > > > things such as starting up an app didn't work for him in F24.
> > > > In the end, it turned out that it was because he was running
> > > > Fedora
> > > > in
> > > > GNOME Boxes. It was difficult to explain him that Fedora doesn't
> > > > run
> > > > well in our default desktop VM tool and that F24 Beta is actually
> > > > a
> > > > great release, he should give it another try in a different tool.
> > >
> > > Why are apps not starting? Is there a bug report? I've *never* had
> > > any
> > > trouble running Fedora in Boxes. This sounds like a serious
> > > regression.
> >
> > F24 Alpha didn't work in F23's Boxes at all, not only for me, but for
> > others around me in the office.
> > Now, I'm already on F24 now and the experience is:
> > login screen is flickering and I can't click any button on the login
> > screen, I have to choose my account and log in using keyboard. When I
> > log in I get a black screen with only thing: clock on the top panel,
> > by
> > trying I figured out that switching Boxes to full screen and clicking
> > randomly on the black screen I make it somehow re-render the screen
> > and
> > it finally works.
>
>  As far as I know, this is a spice <> wayland issue. Some fixes were
>  attempted in https://bugzilla.redhat.com/show_bug.cgi?id=1266484 
I'm running into a different problem, at least it's not wayland
related near as I can tell.
https://bugzilla.gnome.org/show_bug.cgi?id=763960

I hit it with Boxes but not virt-manager.


-- 
Chris Murphy