Atomic (rpm-ostree+flatpak+oc cluster up) workstation

[Colin Walters]

Hey, so I'd like to keep pushing forward on

https://fedoraproject.org/wiki/Workstation/AtomicWorkstation
(Which is related to:
 https://fedoraproject.org/wiki/Changes/WorkstationOstree )

I actually just revamped the first page completely.  One
thing I'd like to propose that's new is that we focus a lot on `oc cluster up`
as a local developer model for *server* applications.  I just
installed it by default:

https://pagure.io/atomic-ws/c/c5f77d9b1a9c15c65cf0abbc1490896b85d9ba48?branc...

Now let's back up; the current state is we
have two efforts, one in:
https://pagure.io/atomic-ws
which lives in CentOS CI currently, and
https://pagure.io/workstation-ostree-config/commits/master
which is in Fedora, but not as actively maintained.

I'd like to migrate my work into Fedora.   This content should be generated
based on Fedora 25, potentially with some overrides.
We should integrate with the existing Fedora tools, such as
https://openqa.fedoraproject.org/
for installer testing.  I'd also like this content to be GPG signed
(and ideally transport protected) the same way other content
is.  But like Atomic Host we should have both a more agile *and* slower release
process that's decoupled from the 6months base/daily-bodhi model
which I don't really think makes sense.  Basically we'd aim for updates
to the core at most once a week (modulo async errata).  The auto-generated flatpaks though
would be daily or faster potentially.  We need the ability to do async
Firefox security errata quickly.

In the short or potentially medium term, this would be a sidecar
to the Workstation which is based on the "big bag of packages" model.
There is an existing base of people who are interested in this, and I think
with some minimal CI/integration and maintenance we can
produce something that's very different from the package bag,
and has some powerful advantages, but also without distracting
too much from it.

A major milestone for this will be removing firefox from the base
tree, and relying on an autogenerated flatpak.

This email was sent from atomic-ws btw; I do find it practical
today for everyday use, but as you can see I've layered some
things like keepassx that aren't flatpak'd yet:

● atomic-ws:atomicws/fedora/x86_64/continuous
       Version: 25.2017.34 (2017-02-15 20:26:05)
    BaseCommit: ef0587aa74b1135566c1a0d79bb3c053da55016bd1fa314df3c78332edc79f68
        Commit: a49a1417c87cc7495a6d66141effba6bebe4d142f466ba79dce8d47a0d010fa7
        OSName: fedora
      Packages: ansible emacs fuse-sshfs gdb gimp keepassx krb5-workstation libgit2 libvirt opensc pcsc-lite-ccid powerline strace vagrant-hostmanager vagrant-libvirt vagrant-sshfs virt-manager xchat xsel ykclient ykpers
      Unlocked: development

[Adam Williamson]

On Thu, 2017-02-16 at 11:21 -0500, Colin Walters wrote:
> I'd like to migrate my work into Fedora.   This content should be generated
> based on Fedora 25, potentially with some overrides.
> We should integrate with the existing Fedora tools, such as
> https://openqa.fedoraproject.org/
> for installer testing.

Glad to see you thinking about this. openQA also has desktop tests
which we could certainly run on Atomic Workstation images; in fact I've
had that on my todo list for months, stymied only by the fact there are
no extant images to work with.

I'd say the reasonable requirement to get the openQA tests run for
Atomic Workstation images is that they be built as part of a typical
releng Pungi 4 compose (and thus we get a standard fedmsg 'compose
complete' notification for them). If we have that, the only work that
needs to be done is adding the Atomic Workstation image to the openQA
scheduler's list of tested images and specifying the tests that should
be run on it, which would take me about fifteen minutes.

The results would get forwarded to ResultsDB like all other openQA
results; there are various options for consuming them from there.

We *can* hack things up for openQA tests to run on less sanely-produced 
composes - as we used to do for the old non-Pungi 4 two-week Atomic
composes, and as we currently do for the semi-official post-release
live respin composes (fedfind has some awful hacks to deal with these,
and the openQA server runs a cron job every half an hour to see if a
new one showed up) - but I'd really prefer to avoid that kind of
hackery for more official / important deliverables. And, of course, if
the composes are 'proper' releng composes it makes integration with all
other systems (Taskotron etc.) easier as well.

[Colin Walters]

On Thu, Feb 16, 2017, at 12:01 PM, Adam Williamson wrote:
> Glad to see you thinking about this. openQA also has desktop tests
> which we could certainly run on Atomic Workstation images;

This is somewhat of a tangent but I was just thinking that openQA
tests the installer *only*.  One of the whole major goals of this is
that by using rpm-ostree to assemble content on the server side,
*that* is the base of the compose (plus any apps/containers to test)  Which means that it's
very valid to have a cached set of existing systems, and do in-place upgrades,
rather than doing fresh installs each time.  Because of the image-based
nature of ostree, one can have a high degree of confidence that the result
of an update is exactly identical to a fresh install[1].

While you've done a lot of heroics maintaining needles for openCV, and
it does make sense to test the installer that way (particularly since
the installer is almost identical, we can share most of that across products),
I'd like to have a variety of systems that test the ostree commit via a mix of inplace
updates and fresh installs, like we do for Atomic Host.

So I guess this isn't really a tangent at all - the concept that the system
is assembled as a unit with a checksum, version etc., which is what
becomes the fundamental unit of test is actually a major goal of
doing this - it makes the system very different in practice from
other systems which snapshot packages on the client side.

[1] Although since ostree operates on the filesystem and not block layer,
      it means systems use retain their initial filesystem layout, we can't
      upgrade that layer; - this bit us for Atomic Host with XFS + f_type + overlayfs:
     https://bugzilla.redhat.com/show_bug.cgi?id=1288162#c8

[Adam Williamson]

On Thu, 2017-02-16 at 21:19 -0500, Colin Walters wrote:
> On Thu, Feb 16, 2017, at 12:01 PM, Adam Williamson wrote:
> > Glad to see you thinking about this. openQA also has desktop tests
> > which we could certainly run on Atomic Workstation images;
>
> This is somewhat of a tangent but I was just thinking that openQA
> tests the installer *only*.  One of the whole major goals of this is
> that by using rpm-ostree to assemble content on the server side,
> *that* is the base of the compose (plus any apps/containers to test)  Which means that it's
> very valid to have a cached set of existing systems, and do in-place upgrades,
> rather than doing fresh installs each time.  Because of the image-based
> nature of ostree, one can have a high degree of confidence that the result
> of an update is exactly identical to a fresh install[1].
>
> While you've done a lot of heroics maintaining needles for openCV, and
> it does make sense to test the installer that way (particularly since
> the installer is almost identical, we can share most of that across products),
> I'd like to have a variety of systems that test the ostree commit via a mix of inplace
> updates and fresh installs, like we do for Atomic Host.

Well, sure, that would be great. But do you have a variety of systems
that can run such tests in place? If not, it doesn't seem like it'd do
any harm to run openQA's desktop tests for now. Then at least you'd
know if the terminal works. :p

Also, openQA isn't tied to installer testing; we could certainly have
it deploy a given Atomic Workstation image - via the installer or not -
then update to a later ostree and run its desktop tests. I'm currently
working on a mechanism to have openQA automatically run its post-
install tests on every critical path update that hits Bodhi, for
instance. Well, I'm working on that and ten other things, you know how
it goes. But it's not something openQA has any trouble doing.

[Colin Walters]

On Thu, Feb 16, 2017, at 11:21 AM, Colin Walters wrote:
> Hey, so I'd like to keep pushing forward on
>
> https://fedoraproject.org/wiki/Workstation/AtomicWorkstation

I've updated this page a bit again based on some in-person discussions.
One thing I'd like to note is the proposed use of Flatpak here is more
for the sandboxing and versioning aspect, not 3rd party Flatpaks, although
that's obviously going to be supported in the same way we support pulling
RPMs and Docker/OCI images from elsewhere too.

Basically, in this proposal the default Firefox flatpak is version locked with the RPM.
And if we go down this path I'm not really sure it makes sense to have a Flatpak runtime
that's versioned/shipped separately from the base ostree.

[M. Edward (Ed) Borasky]

I'm interesting in testing this - is there a page / repository / tutorial
somewhere I can put up on a virtual machine?

On Wed, Feb 22, 2017 at 7:13 AM Colin Walters  wrote:
> On Thu, Feb 16, 2017, at 11:21 AM, Colin Walters wrote:
> > Hey, so I'd like to keep pushing forward on
> >
> > https://fedoraproject.org/wiki/Workstation/AtomicWorkstation
>
> I've updated this page a bit again based on some in-person discussions.
> One thing I'd like to note is the proposed use of Flatpak here is more
> for the sandboxing and versioning aspect, not 3rd party Flatpaks, although
> that's obviously going to be supported in the same way we support pulling
> RPMs and Docker/OCI images from elsewhere too.
>
> Basically, in this proposal the default Firefox flatpak is version locked
> with the RPM.
> And if we go down this path I'm not really sure it makes sense to have a
> Flatpak runtime
> that's versioned/shipped separately from the base ostree.
> _______________________________________________
> desktop mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]

[Colin Walters]

On Wed, Feb 22, 2017, at 12:03 PM, M. Edward (Ed) Borasky wrote:
> I'm interesting in testing this - is there a page / repository /
> tutorial somewhere I can put up on a virtual machine?

There's an ISO linked from:

https://pagure.io/atomic-ws

[Colin Walters]

On Thu, Feb 16, 2017, at 11:21 AM, Colin Walters wrote:
> One
> thing I'd like to propose that's new is that we focus a lot on `oc cluster up`
> as a local developer model for *server* applications.  I just
> installed it by default:
>
> https://pagure.io/atomic-ws/c/c5f77d9b1a9c15c65cf0abbc1490896b85d9ba48?branc...

There's definitely more to do to make this a nicer out of the box experience.
If anyone is playing with this now, see:
https://github.com/openshift/origin/pull/13112
for example.

(I actually built openshift from current git master since I wanted
 https://github.com/openshift/origin/pull/12456 )

[Owen Taylor]

Lot of different things here! Answering in no particular order:

* Yes, merging the two efforts would be great. We need to figure out a
minimal set of things that we need to change in Fedora infrastructure
to make the atomic workstation builds there a reasonable development
target. Colin and I had an off-line conversation about this, and I'll
send mail separately to summarize.

* Having CI targeted at the Atomic Workstation builds is a very
important part of that.

* We definitely should promote OpenShift as a cool and Fedora-aligned
way of doing development of server applications and make it easy as
possible to get going. Shipping the necessary bits to run openshift out
of the box on Fedora Atomic Workstation seems like a good idea to me.

* I don't think OpenShift is a *universal* development solution - if
you want to 'pip install' a few packages and then try things out on the
command line, it's extremely heavy weight, and we should be looking for
good solutions for lightweight pet containers - whether an evolution of
my PurpleEgg idea or something else.

Thanks for pushing this forward, Colin!
Owen

On Thu, 2017-02-16 at 11:21 -0500, Colin Walters wrote:
> Hey, so I'd like to keep pushing forward on
>
> https://fedoraproject.org/wiki/Workstation/AtomicWorkstation
> (Which is related to:
>  https://fedoraproject.org/wiki/Changes/WorkstationOstree )
>
> I actually just revamped the first page completely.  One
> thing I'd like to propose that's new is that we focus a lot on `oc
> cluster up`
> as a local developer model for *server* applications.  I just
> installed it by default:
>
> https://pagure.io/atomic-ws/c/c5f77d9b1a9c15c65cf0abbc1490896b85d9ba4
> 8?branch=master
>
> Now let's back up; the current state is we
> have two efforts, one in:
> https://pagure.io/atomic-ws
> which lives in CentOS CI currently, and
> https://pagure.io/workstation-ostree-config/commits/master
> which is in Fedora, but not as actively maintained.
>
> I'd like to migrate my work into Fedora.   This content should be
> generated
> based on Fedora 25, potentially with some overrides.
> We should integrate with the existing Fedora tools, such as
> https://openqa.fedoraproject.org/
> for installer testing.  I'd also like this content to be GPG signed
> (and ideally transport protected) the same way other content
> is.  But like Atomic Host we should have both a more agile *and*
> slower release
> process that's decoupled from the 6months base/daily-bodhi model
> which I don't really think makes sense.  Basically we'd aim for
> updates
> to the core at most once a week (modulo async errata).  The auto-
> generated flatpaks though
> would be daily or faster potentially.  We need the ability to do
> async
> Firefox security errata quickly.
>
> In the short or potentially medium term, this would be a sidecar
> to the Workstation which is based on the "big bag of packages" model.
> There is an existing base of people who are interested in this, and I
> think
> with some minimal CI/integration and maintenance we can
> produce something that's very different from the package bag,
> and has some powerful advantages, but also without distracting
> too much from it.
>
> A major milestone for this will be removing firefox from the base
> tree, and relying on an autogenerated flatpak.
>
> This email was sent from atomic-ws btw; I do find it practical
> today for everyday use, but as you can see I've layered some
> things like keepassx that aren't flatpak'd yet:
>
> ● atomic-ws:atomicws/fedora/x86_64/continuous
>        Version: 25.2017.34 (2017-02-15 20:26:05)
>     BaseCommit:
> ef0587aa74b1135566c1a0d79bb3c053da55016bd1fa314df3c78332edc79f68
>         Commit:
> a49a1417c87cc7495a6d66141effba6bebe4d142f466ba79dce8d47a0d010fa7
>         OSName: fedora
>       Packages: ansible emacs fuse-sshfs gdb gimp keepassx krb5-
> workstation libgit2 libvirt opensc pcsc-lite-ccid powerline strace
> vagrant-hostmanager vagrant-libvirt vagrant-sshfs virt-manager xchat
> xsel ykclient ykpers
>       Unlocked: development
> _______________________________________________
> desktop mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]