Re: [ovirt-devel] Adding s390 support to oVirt

[Viktor Mihajlovski]

On 16.11.2017 18:27, Barak Korren wrote:
>  On 16 November 2017 at 18:52, Viktor Mihajlovski
>  <mihajlov(a)linux.vnet.ibm.com&gt; wrote:
> >
> > Short update, with yesterday's API model 4.2.25 release, there's basic
> > support for s390 available in ovirt-engine. At this point in time, there
> > are no ovirt yum repositories for the s390x architecture - not sure what
> > the process would be to add s390x repositories and how to build the
> > binary RPMs at least for the host packages (i.e. vdsm-*). Maybe it would
> > possible to use the s390-koji infrastructure used to build Fedora for s390x?
>  
>  Koji is very opinionated about how RPMs and specfiles should look,
>  AFAIK A pretty massive amount of work would be needed to make
>  everything that is needed for a node to build on it, and then we would
>  end up with a process that is quite different with how we currently do
>  builds for other platforms.
>  
>  (I might be wrong about this, since some oVirt packages get also built
>  as part of the CentOS virt SIG, and that is done using Koji as well)
>  
>  More specifically, Koji usually assumes the starting point for the
>  build process would be a specfile, while in oVirt we typically
>  generated the specfile and then the RPM as part of a bigger build
>  process.
>  
>  Does fedora have an s390x server associated to it? 
There's a build system for
Fedora on s390x:
https://s390.koji.fedoraproject.org/koji
>  We do use the same basic environment setup tool - mock - as the
> basis
>  of our build infrastructure, so if Fedora is actually emulating s390x
>  is some way while using mock, we might be able to do the same thing.
>  
>  Just for general knowledge, the process for building oVirt repos, is
>  to have *-build-artifacts-* jobs for each project that build RPM's
>  after patches get merged, and then have the change-queue to collect
>  the built packages, rung them through ovirt-system-tests (a.k.a. OST)
>  and finally deposit them into the 'tested' rpoe, from which they are
>  copied nightly to the '*-snapshot' repos.
>  
>  OST only tests for CentOS 7/x86_64 ATM, but we bring along packages
>  for other distros and architectures via the same process while
>  assuming that if a package for a given commit works for CentOS
>  7/x86_64 it would probably work for other platforms as well...
>   
I've noticed that there's no test automation for non-x86 arches. How are
the ppc64le packages then built and stored in the repos?

Would it be conceivable to do the following?
After a successfull build and OST of a package, take the SRPM and submit
to s390-koji to produce the s390x binary RPMs.
Then copy the binary RPMs into the respective repository, e.g.
*-snapshot/rpm/el7/s390x (and update the repository metadata).

-- 

Mit freundlichen Grüßen/Kind Regards
   Viktor Mihajlovski

[ovirt-devel] Adding s390 support to oVirt

[Peter Robinson]

On Fri, Nov 17, 2017 at 8:45 AM, Viktor Mihajlovski
<mihajlov(a)linux.vnet.ibm.com&gt; wrote:
>  On 16.11.2017 18:27, Barak Korren wrote:
> > On 16 November 2017 at 18:52, Viktor Mihajlovski
> > <mihajlov(a)linux.vnet.ibm.com&gt; wrote:
> >>
> >> Short update, with yesterday's API model 4.2.25 release, there's basic
> >> support for s390 available in ovirt-engine. At this point in time, there
> >> are no ovirt yum repositories for the s390x architecture - not sure what
> >> the process would be to add s390x repositories and how to build the
> >> binary RPMs at least for the host packages (i.e. vdsm-*). Maybe it would
> >> possible to use the s390-koji infrastructure used to build Fedora for s390x?
> >
> > Koji is very opinionated about how RPMs and specfiles should look,
> > AFAIK A pretty massive amount of work would be needed to make
> > everything that is needed for a node to build on it, and then we would
> > end up with a process that is quite different with how we currently do
> > builds for other platforms.
> >
> > (I might be wrong about this, since some oVirt packages get also built
> > as part of the CentOS virt SIG, and that is done using Koji as well)
> >
> > More specifically, Koji usually assumes the starting point for the
> > build process would be a specfile, while in oVirt we typically
> > generated the specfile and then the RPM as part of a bigger build
> > process.
> >
> > Does fedora have an s390x server associated to it?
>  There's a build system for Fedora on s390x:
>  https://s390.koji.fedoraproject.org/koji 
That is now deprecated, all the architectures including s390x build in
the primary Fedora koji infra

https://koji.fedoraproject.org/koji/packageinfo?packageID=12944

You'd just have to make the appropriate changes to the vdsm package
and it would build for Fedora on s390x

> > We do use the same basic environment setup tool - mock - as the
> basis
> > of our build infrastructure, so if Fedora is actually emulating s390x
> > is some way while using mock, we might be able to do the same thing.
> >
> > Just for general knowledge, the process for building oVirt repos, is
> > to have *-build-artifacts-* jobs for each project that build RPM's
> > after patches get merged, and then have the change-queue to collect
> > the built packages, rung them through ovirt-system-tests (a.k.a. OST)
> > and finally deposit them into the 'tested' rpoe, from which they are
> > copied nightly to the '*-snapshot' repos.
> >
> > OST only tests for CentOS 7/x86_64 ATM, but we bring along packages
> > for other distros and architectures via the same process while
> > assuming that if a package for a given commit works for CentOS
> > 7/x86_64 it would probably work for other platforms as well...
> >
>  I've noticed that there's no test automation for non-x86 arches. How are
>  the ppc64le packages then built and stored in the repos? 
This is slowly improving, it's about having available infrastructure
and people to assist in getting it running and supporting it. For
example I believe there's now ppc64le testing in openQA now.

>  Would it be conceivable to do the following?
>  After a successfull build and OST of a package, take the SRPM and submit
>  to s390-koji to produce the s390x binary RPMs.
>  Then copy the binary RPMs into the respective repository, e.g.
>  *-snapshot/rpm/el7/s390x (and update the repository metadata). 
We don't currently have s390x EPEL

[ovirt-devel] Adding s390 support to oVirt

[Neale Ferguson]

http://download.sinenomine.net/epel/ - EPEL for our CentOS clone


On 11/17/17, 8:21 PM, "Peter Robinson" <pbrobinson(a)gmail.com&gt; wrote:

> We don't currently have s390x EPEL
> _______________________________________________
> s390x mailing list -- s390x(a)lists.fedoraproject.org
> To unsubscribe send an email to s390x-leave(a)lists.fedoraproject.org

Adding s390 support to oVirt

[R P Herrold]

On Fri, 17 Nov 2017, Peter Robinson wrote:

>  We don't currently have s390x EPEL 
ummm -- I think Peter is suggesting EPEL executables -- this 
is a solved problem

The ClefOS s390x distribution has all of RHEL 6, RHEL 7, EPEL 
6, EPEL 7, and more in s390x, and has had for years. Neale 
Ferguson (SNA) has 'banged the drum' on the product at trade 
shows etc, and had a nice Docker demonstration under s390x

I have covered all of the s390x and z/VM mailing lists for 
nearly a decade on this topic, and it gets mentions at least 
monthly.  Some may recall SNA principal David Boyes' 
demonstration (with Debian kit) of spinning up north of 40k 
instances under the older IFL model.  Rich Troth (long time 
CMS developer) uses ClefOS, and he was over at my office 
earlier this month on trying to do some evangalism on how to 
make installations easier for people coming from 'Distributed'
	
The initial builds were done in Hercules emulators [I started 
doing this probably 8 years ago, and demonstrated at the 
Louisville VMWorkShop five years ago], then completely 
re-built starting mid RHEL 6 (after the IBM EOL on older Z 
hardware), naively  

# uname -a
Linux lclef01.lf-dev.marist.edu 3.10.0-693.5.2.el7.s390x #1 
	SMP Fri Oct 27 20:15:11 EDT 2017 s390x s390x s390x GNU/Linux

SNA hosts the primary mirror, but I anticipate getting a 
mirror up in higher bandwidth at Marist University (near the 
IBM facility at Poughkeepsie), hopefully yet this year

[herrold@centos-7 ~]$ lynx -dump \
	"http://mirrors.sinenomine.net/epel?releasever=7&arch=s390x&repo=epel"
https://download.sinenomine.net/epel/epel-7

-- Russ herrold

Adding s390 support to oVirt

[Neale Ferguson]

Only EPEL7. I didn't build for 6.


-------- Original message --------
From: R P Herrold <herrold(a)owlriver.com&gt;
Date: 11/18/17 06:54 (GMT+10:00)
To: s390x(a)lists.fedoraproject.org
Cc: devel <devel(a)ovirt.org&gt;, Barak Korren <bkorren(a)redhat.com&gt;
Subject: Adding s390 support to oVirt

On Fri, 17 Nov 2017, Peter Robinson wrote:

>  We don't currently have s390x EPEL 
ummm -- I think Peter is suggesting EPEL executables -- this
is a solved problem

The ClefOS s390x distribution has all of RHEL 6, RHEL 7, EPEL
6, EPEL 7, and more in s390x, and has had for years. Neale
Ferguson (SNA) has 'banged the drum' on the product at trade
shows etc, and had a nice Docker demonstration under s390x

I have covered all of the s390x and z/VM mailing lists for
nearly a decade on this topic, and it gets mentions at least
monthly.  Some may recall SNA principal David Boyes'
demonstration (with Debian kit) of spinning up north of 40k
instances under the older IFL model.  Rich Troth (long time
CMS developer) uses ClefOS, and he was over at my office
earlier this month on trying to do some evangalism on how to
make installations easier for people coming from 'Distributed'

The initial builds were done in Hercules emulators [I started
doing this probably 8 years ago, and demonstrated at the
Louisville VMWorkShop five years ago], then completely
re-built starting mid RHEL 6 (after the IBM EOL on older Z
hardware), naively

# uname -a
Linux lclef01.lf-dev.marist.edu 3.10.0-693.5.2.el7.s390x #1
        SMP Fri Oct 27 20:15:11 EDT 2017 s390x s390x s390x GNU/Linux

SNA hosts the primary mirror, but I anticipate getting a
mirror up in higher bandwidth at Marist University (near the
IBM facility at Poughkeepsie), hopefully yet this year

[herrold@centos-7 ~]$ lynx -dump \
       
"http://mirrors.sinenomine.net/epel?releasever=7&arch=s390x&repo=epel"
https://download.sinenomine.net/epel/epel-7

-- Russ herrold
_______________________________________________
s390x mailing list -- s390x(a)lists.fedoraproject.org
To unsubscribe send an email to s390x-leave(a)lists.fedoraproject.org

f-s390] Re: Adding s390 support to oVirt

[R P Herrold]

On Fri, 17 Nov 2017, Neale Ferguson wrote:

>  Only EPEL7. I didn't build for 6. 
ahh -- my mistake and faulty recollection -- I'd moved on to 7 
pretty much completely a couple of point updates ago

If there is demand for EPEL 6, and Neale's additional archive 
'earlier', I can accomodate

-- Russ herrold

[ovirt-devel] Adding s390 support to oVirt

[Barak Korren]

On 17 November 2017 at 10:45, Viktor Mihajlovski
<mihajlov(a)linux.vnet.ibm.com&gt; wrote:
>  On 16.11.2017 18:27, Barak Korren wrote:
> > On 16 November 2017 at 18:52, Viktor Mihajlovski
> > <mihajlov(a)linux.vnet.ibm.com&gt; wrote:
> >>
> >> Short update, with yesterday's API model 4.2.25 release, there's basic
> >> support for s390 available in ovirt-engine. At this point in time, there
> >> are no ovirt yum repositories for the s390x architecture - not sure what
> >> the process would be to add s390x repositories and how to build the
> >> binary RPMs at least for the host packages (i.e. vdsm-*). Maybe it would
> >> possible to use the s390-koji infrastructure used to build Fedora for s390x?
> >
> > Koji is very opinionated about how RPMs and specfiles should look,
> > AFAIK A pretty massive amount of work would be needed to make
> > everything that is needed for a node to build on it, and then we would
> > end up with a process that is quite different with how we currently do
> > builds for other platforms.
> >
> > (I might be wrong about this, since some oVirt packages get also built
> > as part of the CentOS virt SIG, and that is done using Koji as well)
> >
> > More specifically, Koji usually assumes the starting point for the
> > build process would be a specfile, while in oVirt we typically
> > generated the specfile and then the RPM as part of a bigger build
> > process.
> >
> > Does fedora have an s390x server associated to it?
>  There's a build system for Fedora on s390x:
>  https://s390.koji.fedoraproject.org/koji
> > We do use the same basic environment setup tool - mock - as the basis
> > of our build infrastructure, so if Fedora is actually emulating s390x
> > is some way while using mock, we might be able to do the same thing.
> >
> > Just for general knowledge, the process for building oVirt repos, is
> > to have *-build-artifacts-* jobs for each project that build RPM's
> > after patches get merged, and then have the change-queue to collect
> > the built packages, rung them through ovirt-system-tests (a.k.a. OST)
> > and finally deposit them into the 'tested' rpoe, from which they are
> > copied nightly to the '*-snapshot' repos.
> >
> > OST only tests for CentOS 7/x86_64 ATM, but we bring along packages
> > for other distros and architectures via the same process while
> > assuming that if a package for a given commit works for CentOS
> > 7/x86_64 it would probably work for other platforms as well...
> >
>  I've noticed that there's no test automation for non-x86 arches. How are
>  the ppc64le packages then built and stored in the repos? 
There is no test automation but there IS build automation (Supporting
just PPC64LE for now).

As I've said, we pass all the packages through the same pipeline and
just publish them as the equivalent x86_64 packages get built.

>  Would it be conceivable to do the following?
>  After a successfull build and OST of a package, take the SRPM and submit
>  to s390-koji to produce the s390x binary RPMs.
>  Then copy the binary RPMs into the respective repository, e.g.
>  *-snapshot/rpm/el7/s390x (and update the repository metadata). 
We don't have support of post-OST triggering in our framework ATM.
This would also mean that the s390x would hit the repos out-of-sync
with all other packages.

I'd much rather find some way to build the packages on oVirt's
existing infrastructure rather then out-source it to Fedora.,

While we are related to Fedora in some ways, oVirt is its own project
with its own tooling and processes.

-- 
Barak Korren
RHV DevOps team , RHCE, RHCi
Red Hat EMEA
redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted