openQA, "compose check reports", Wiki release validation:
CONSTRUCTION IN PROGRESS
by Adam Williamson
Hi, folks! I thought I'd better send out a notice so people know what's
going on here. Any errors in this explanation are my own, dgilmore will
correct me if I'm wrong on any details.
release engineering is currently working to switch the compose process
over to Pungi 4. Specifically, the old nightly compose process for
Rawhide has been disabled, and it was never enabled for Fedora 24.
The whole process of running openQA tests on new composes, generating
the "compose check reports", and creating release validation events on
the wiki was tied to the way the old compose process worked. All those
things need to be changed somewhat for the new compose process. We are
rebuilding the airplane in flight, if you could please hold on to the
engine and/or wing closest to you, that would help. ;)
# openQA
We have rewritten the openQA job scheduler to cue off Pungi 4 "compose
complete!" messages and use Pungi 4 / productmd'ish ways to identify
composes and images. I'm right in the middle of pushing those changes
out now. This *should* mean that openQA tests hand over relatively
smoothly, and there will be openQA test runs for every Pungi 4 compose,
as it lands. There may be a few teething problems with this, we'll see
as we go along. Note for anyone who follows openQA closely or does
anything with its output, the format we use for openQA 'BUILD's and
'FLAVOR's has changed.
Previously, the 'BUILD' value was based on my fedfind/wikitcms version
concept, it was simply the fedfind 'release', 'milestone' and 'compose'
joined with underscores (e.g. 23_Beta_RC2 or 24_Branched_20160225). We
now use the Pungi 4 / productmd 'compose ID' value. This looks like
'Fedora-RELEASE-DATE(.TYPE).RESPIN', where RELEASE is a release number
or 'Rawhide', DATE is the date the compose happened in YYYYMMDD format,
TYPE is the 'compose type' ('n' for "nightly" composes, 't' for "test"
composes, and nothing - this part is omitted - for "production"
composes), and RESPIN is an integer that gets incremented when you do
more than one compose which otherwise would have the same id (e.g. your
compose blows up and you have to do it again). If you need to parse
productmd "compose IDs", the python-productmd package has some bits for
doing it.
The 'FLAVOR' similarly used to be based on fedfind concepts (it was
'PAYLOAD_IMAGETYPE'), it's now based on productmd concepts - flavor is
now 'PAYLOAD-TYPE-FORMAT'. TYPE and FORMAT are productmd properties (we
replace dashes with underscores so parsing the flavor back out works).
PAYLOAD we unfortunately have to synthesize, because productmd doesn't
really provide any convenient property for, say, "the KDE live image".
productmd has a 'variant' concept, but all desktop lives are part of
the "Spins" variant, so that's no use (we can't use it to tell KDE from
LXDE). So we just have to kinda hack this up for now. I'm trying to get
it improved in productmd/pungi somehow.
# check-compose
"Compose check reports" are going to be broken for a bit, probably a
day or two. They're quite heavily tied to fedfind versioning and to an
assumption which is no longer true: there will be exactly one nightly
compose per day. When we decide what compose to compare the latest
compose against, we simply pick the compose for the same release from
the previous day. This doesn't work any more, because of how pungi4 has
'respins' and the respin identifier goes into the compose ID. When we
have 'Fedora-24-20160224.n.0' we don't know what to compare it to,
because we don't know how many respins there were on 20160223. What
we'll have to do instead is query PDC to find the previous successful
compose of the same type, and compare to that. It will also make sense
to use productmd-style image identification rather than fedfind-style.
The question of *when to run* check-compose is also an interesting one;
again it's currently cheating by making assumptions (it starts up a
couple of hours before the nightly compose is expected to complete and
waits 8 hours, on the assumption the compose will show up and openQA
testing will complete during that 8 hour span). What it ideally should
do is run when openQA sends out a message that testing of a nightly
compose has completed. This is a bit difficult at present because
openQA doesn't do that. =) So I'm hoping to add support for that into
openQA somehow or other, but in the worst case it might have to keep
running on a dumb timer for a while if that proves harder than
expected, or perhaps run as a daemon which checks in with openQA every
half hour with some clever query to figure out if a compose has
completed testing in the meantime.
# Wikitcms
The whole wiki release validation process is probably going to be the
hardest to adapt, and also won't be possible to do until we have an
idea how we're going to handle milestone releases and TCs/RCs, so I'm
kinda planning to start working on it once the Pungi 4 switchover has
shaken down and releng have some cycles to discuss and think about
that.
There will be some subjective human decisions to be made for sure, I
think, so when we get to it, there'll be some discussion threads.
Prepare yourself :)
For the present, there won't be any more nightly validation events
created automatically. If the changeover drags on too long before we
decide what to do about TCs/RCs, I'll do something about that, so we're
not stuck on an ancient validation event (with the systemd/selinux
bug).
Ideally, python-wikitcms validation event creation will also cue off
fedmsg's; for now it should probably run when any compose completes and
make some simple decisions about whether to create an event for that
compose. Perhaps in future the decision should take openQA automated
test results into account (so we don't create validation events for
completely busted composes), but that might be slightly further down
the road.
Thoughts? Comments? questions? Wanna help? Please let us know! Thanks
:)
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
8 years, 2 months
three questions: f24 branch, python3 default, where to put
/etc/rsyslog.d/gluster.conf.example
by Kaleb S. KEITHLEY
Hi,
1) usually after the branch I build new packages for rawhide (i.e.
$branch+1). But atm in the master branch, a `fedpkg build` gives me
Could not execute build: Package glusterfs-3.7.8-1.fc24 has already
been built
Am I just too early? Or is there something missing that's preventing
builds for f25.
2) Despite several articles out in the wild claiming that {F22,F23,F24}
will switch to Python3 as the default, [1] would seem to indicate that
the only thing we're going to see is 1) the update of Python3 from 3.4
to 3.5, and 2) Python3 system-python and system-python-libs will be
split out; however the default python will remain Python2.
Is that correct?
3) The Fedora and upstream glusterfs.spec file package example rsyslog
config files for some distributions. (Although since Fedora hasn't used
rsyslog by default since F20 or so, these aren't actually installed on
recent Fedora releases.) The packaging guidelines don't appear to say
anything on the topic. Absent anything in the packaging guidelines, is
there a consensus for where they should be installed on systems that do
use rsyslog, e.g. EPEL/EL6?
In /etc/rsyslog.d/glusterfs.conf.example? Or somewhere else, e.g.
/usr/share/doc/glusterfs/rsyslog.d/glusterfs.conf.example?
[1] https://fedoraproject.org/wiki/Releases/24/ChangeSet
Thanks,
--
Kaleb
8 years, 2 months
Re: 4.4 rebase coming to F23 soon
by Xose Vazquez Perez
Jóhann_B._Guðmundsson wrote:
> Arguably the regression in 4.4 with drm/i915 that causes screen
> flickering in dual monitor setups needs to be looked at before this gets
> released since it will get tiresome quite quickly for end users trying
> to do some work when suddenly one of the screen turns itself off and
> then on again.
I don't see any related patch in Greg's stable-queue for 4.4:
https://git.kernel.org/cgit/linux/kernel/git/stable/stable-queue.git/tree...
PD: the performance of HyperKitty is really awful!
8 years, 2 months
Soname change for jsoncpp
by Sébastien Willmann
Hi,
I’m planning to update jsoncpp in rawhide to version 1.6.5 in a week
(when f24 is branched)
(https://bugzilla.redhat.com/show_bug.cgi?id=1209654)
The output of dnf repoquery --whatrequires jsoncpp --alldeps is:
cmake-0:3.3.2-1.fc23.x86_64
cmake-0:3.4.1-4.fc23.i686
cmake-0:3.4.1-4.fc23.x86_64
cmake-gui-0:3.3.2-1.fc23.x86_64
cmake-gui-0:3.4.1-4.fc23.x86_64
engrid-0:2.0.0-0.5.gitbaef0ce.fc23.i686
engrid-0:2.0.0-0.5.gitbaef0ce.fc23.x86_64
jsoncpp-devel-0:0.6.0-0.18.rc2.fc23.i686
jsoncpp-devel-0:0.6.0-0.18.rc2.fc23.x86_64
kopete-0:15.04.2-2.fc23.x86_64
kopete-0:15.12.1-1.fc23.x86_64
libASL-0:0.1.6-1.fc23.i686
libASL-0:0.1.6-1.fc23.x86_64
minetest-0:0.4.12-4.fc23.x86_64
minetest-server-0:0.4.12-4.fc23.x86_64
mrpt-2d-slam-0:1.3.0-1.fc23.x86_64
mrpt-apps-0:1.3.0-1.fc23.x86_64
mrpt-camera-calibration-0:1.3.0-1.fc23.x86_64
mrpt-gridmap-navigation-0:1.3.0-1.fc23.x86_64
mrpt-libs-0:1.3.0-1.fc23.i686
mrpt-libs-0:1.3.0-1.fc23.x86_64
mrpt-navlog-viewer-0:1.3.0-1.fc23.x86_64
mrpt-rawlog-viewer-0:1.3.0-1.fc23.x86_64
mrpt-reactive-navigation-0:1.3.0-1.fc23.x86_64
mrpt-scene-viewer-0:1.3.0-1.fc23.x86_64
mrpt-stereo-camera-calibration-0:1.3.0-1.fc23.x86_64
orthanc-0:0.8.6-5.fc23.x86_64
orthancclient-devel-0:0.8.6-5.fc23.i686
orthancclient-devel-0:0.8.6-5.fc23.x86_64
orthancclient-lib-0:0.8.6-5.fc23.i686
orthancclient-lib-0:0.8.6-5.fc23.x86_64
vfrnav-0:20150429-1.fc23.i686
vfrnav-0:20150429-1.fc23.x86_64
vtk-0:6.2.0-8.fc23.i686
vtk-0:6.2.0-8.fc23.x86_64
vtk-examples-0:6.2.0-8.fc23.x86_64
vtk-testing-0:6.2.0-8.fc23.x86_64
Copr with the new version:
https://copr.fedorainfracloud.org/coprs/wilqu/jsoncpp/
8 years, 2 months
Orphaning python-django-haystack and python-django-pipeline in Fedora
by Stephen Gallagher
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
I no longer need these packages in Fedora (they were there to support Review
Board which is now only in EPEL due to Django incompatibilities), so I've
orphaned them.
I will continue to maintain them on EPEL 7.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
iEYEARECAAYFAlbN1XMACgkQeiVVYja6o6NT9gCcD6Wa+j+xSSqHWzoylvOG+hK1
NMoAoKeZR7kHSqkoWwjGGBjzrceQLAJN
=t/Ku
-----END PGP SIGNATURE-----
8 years, 2 months
4.4 rebase coming to F23 soon
by Laura Abbott
Hi,
4.4.2 is currently building and should be in updates-testing soon. As usual,
please test and give karma appropriately (negative karma for new issues,
not existing issues).
Thanks,
Laura
8 years, 2 months
Golang packaging: now and the future
by Jan Chaloupka
Hi guys,
at the moment there is approximately 250 packages shipping projects
written in go. In general, each project has its own set of dependencies
and provided import paths (golang packages). As usually each package has
four active branches. Thus, we are dealing with approximately 1000 spec
files, all potential candidates for rebases/updates and bug fixes. Each
update can break backward compatibility. Some projects may require
different commits of the same dependency.
What are the most hot packaging problems in the golang ecosystem?
- bundled vs. de-bundled dependencies
- updates of spec files
- backward compatibility issues
- integration testing
- best practises
*bundled vs. de-bundled dependencies)*
As mentioned in [1] some projects can not be build from de-bundled
dependencies. Either caused by requirement for more commits of the same
dependency or due to daily updates which require reviews of new
packages. The current effort is to minimize number of bundled dependencies.
*updates of spec files)*
In general, each spec file has to be manually updated. Either if it is a
list of build-time (runtime) dependencies, list of provided packages or
a list of tests. Secondly, you never know if you break dependencies of
other projects when you change project's API. Last but not least, when
you want to update to newer release (or commit) of a golang project, new
dependencies may appear. Resulting in generating of new spec files,
reviews, overrides and dreamless hours of routine steps. The current
effort is to detect all dependencies and provide warnings about their
removal and to detect portion of API that is used by other projects. At
the same time to provide more complex tooling to generate better spec
file with less dependencies.
*backward compatibility issues)*
With 1000 spec files it is difficult to keep API of all projects
consistent. No matter what you do you will always break something. At
the moment we can provide a dependency graph only. It can tell you if a
given dependency is available and what projects requires the dependency.
*integration testing)*
Some golang projects need additional services running. It would be great
to be able to CI all golang packages to see what is broken and what is
suitable for fixing.
*best practises)*
Well...
And there are other issues which when resolved can make the world better
place to live. Mentioning some:
- generator of self-standing dvel subpackages [2]
- detectors of bundled deps [3]
- multi spec file generators [4]
- partial spec file generator [5]
- better lint
*What can you do to make the world a better place?*
Join us and contribute on gofed [6]. Currently we are in a phase of
implementing new architecture, extending current and introducing new
functionality. At the moment we aim to support golang language. In
future, there is no reason not to extend the support to other languages.
The top most goal is to implement health check analysis. Once detection
of used symbols in API is done, we can provide better feedback during
updates of packages and make the ecosystem health enough to be able to
grow fresh and strong golang projects.
What can you do right away? Play with gofed and infrastructure, help
testing and documenting.
Ideas and improvements are welcome.
[1] https://fedorahosted.org/fpc/ticket/382#comment:37
[2] https://github.com/gofed/gofed/issues/36
[3] https://github.com/gofed/gofed/issues/42
[4] https://github.com/gofed/gofed/issues/49
[5] https://github.com/gofed/gofed/issues/55
[6] https://github.com/gofed/gofed
[7] https://github.com/gofed/infra
Go nuts!!!
Jan
8 years, 2 months