[AutoQA] #113: rpmguard: Behaviour when comparing identical packages?
by fedora-badges
#113: rpmguard: Behaviour when comparing identical packages?
-------------------------+--------------------------------------------------
Reporter: kparal | Owner: kparal
Type: enhancement | Status: new
Priority: minor | Milestone:
Component: tests | Version: 1.0
Keywords: rpmguard |
-------------------------+--------------------------------------------------
Currently rpmguard compares packages even when they are identical. Is
there a better approach? Should rpmguard for identical packages just skip
comparison and print info? Should identity be judged just from NVRA or
must the packages be really identical on file level? Is there some RPM
checksum that could help us decide identity?
--
Ticket URL: <https://fedorahosted.org/autoqa/ticket/113>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project
14 years, 2 months
Re: rpmguard: comparing identical packages
by Kamil Paral
----- "Kamil Paral" <kparal(a)redhat.com> wrote:
>
> Generally
> there is one use case where rpmguard compares the new package with
> itself. It is when the updated package moves to official repositories
> before the rpmguard test is performed.
>
As I have mentioned, it is currently possible for a package to "fly
under our radar" by simply moving to official repositories (stable,
updates, updates-testing) too soon. It caused by our current strategy
to compare the package against its most recent version in official
repositories.
So how to tackle it?
Solution #1: Leave it be
I suppose this events will not happen too often. If you deem it
appropriate, we can just let it be. Currently it's mainly proof
of concept. In the future if we have a policy that no package can
move from -updates-candidate unless it has been checked by AutoQA
test, then the problem is automatically solved.
Solution #2: Re-implement "search for latest" strategy
I have been playing with koji command and it seems to be able
to list all the history of builds with particular tag, not just
the latest build. You can try it yourself:
$ koji list-tag-history --package=PackageKit
$ koji list-tagged dist-f12-updates PackageKit
Therefore we can re-implement/add a new method to our library
that would not just provide the latest build of a package, but
all of them from the history. Then we would pick the one that
suits us most. The API clearly offers this capabilities.
There is also one question that is directly connected to this topic.
Which tags do we want to search through when looking for previous
builds?
a) Currently we search through stable, -updates and -updates-testing.
b) Would it make sense to search also through -updates-candidate?
The differences would always show to the very latest build, doesn't
have to be released.
c) Or the other way round, would it be better to search through only
stable and -updates (skip -updates-testing)?
When more builds are in testing repo, the differences would be
displayed always against the latest build in stable repos (stable or
-updates).
I would say solution #2 shouldn't be that hard, I would do it. And as
for repositories searched, maybe c) would be better than a), but I don't
know packagers' needs.
14 years, 3 months
rpmguard: comparing identical packages
by Kamil Paral
----- "Will Woods" <wwoods(a)redhat.com> wrote:
> No, because apparently PackageKit-0.5.6-1.fc12 is already public in
> f12-updates. The test is supposed to check things that land in
> dist-f12-updates-candidate (e.g. new builds) against whatever is
> currently public.
>
> So somehow PackageKit-0.5.6-1.fc12 appeared in updates-candidate
> despite
> it already being in the updates repo? I wonder how that happened.
>
I have dug into that issue and it's little more complicated. It
doesn't matter if James started the test manually or not. Generally
there is one use case where rpmguard compares the new package with
itself. It is when the updated package moves to official repositories
before the rpmguard test is performed.
Usually it goes like this:
1. Maintainer builds a new package, it's tagged as -updates-candidate.
2. AutoQA detects the updated package and using its library rpmguard
looks for the most recent package in stable, -updates and
-updates-testing.
3. Rpmguard compares new package from 1) with older package from 2).
4. Some time later a package from 1) is re-tagged as -updates or
-updates-testing.
The problem arises when step 4) is between steps 1) and 2), ie.
when maintainer moves
his new package from -updates-candidate to either -updates or
-updates-testing too soon. If AutoQA watcher is run only after that,
then obviously the package version from 1) and from 2) are equal.
And rpmguard simply compares them.
So that's the explanation of the situation. Of course I can modify
rpmguard to simply skip checking of identical packages, but that
does not solve the problem of that particular package not being
checked against the older version. I will cover possible solutions
in a future email, I have to play with it a little more.
One more thing:
> Previous package for dist-f12-updates was PackageKit-0.5.6-1.fc12
> Previous package for dist-f12-updates was PackageKit-0.5.6-1.fc12
> Previous package for dist-f12 was PackageKit-0.5.4-0.1.20091029git.fc12
This is not weird at all. It just means that there is no PackageKit
in dist-f12-updates-testing, and according to tag inheritance [1]
it found the most recent PackageKit in dist-f12-updates (which is
obviously the same result as when checking the dist-f12-updates
tag itself), hence the same line.
If there hadn't been any PackageKit updates since F12 release,
there would've been three identical lines mentioning dist-12 tag.
[1] http://koji.fedoraproject.org/koji/taginfo?tagID=106
14 years, 3 months
Re: rpmguard: 0 warnings for package PackageKit-0.5.6-1.fc12
by James Laska
On Thu, 2010-01-14 at 12:03 -0500, autoqa(a)fedoraproject.org wrote:
> N: Comparing PackageKit-yum-0.5.6-1.fc12 and PackageKit-yum-0.5.6-1.fc12 (archs: ppc, x86_64, ppc64, i686) ...
> N: ----
> N: Comparing PackageKit-docs-0.5.6-1.fc12 and PackageKit-docs-0.5.6-1.fc12 (archs: noarch) ...
> N: ----
> N: Comparing PackageKit-backend-devel-0.5.6-1.fc12 and PackageKit-backend-devel-0.5.6-1.fc12 (archs: ppc, x86_64, ppc64, i686) ...
> N: ----
> N: Comparing PackageKit-gtk-module-0.5.6-1.fc12 and PackageKit-gtk-module-0.5.6-1.fc12 (archs: ppc, x86_64, ppc64, i686) ...
> N: ----
> N: Comparing PackageKit-cron-0.5.6-1.fc12 and PackageKit-cron-0.5.6-1.fc12 (archs: ppc, x86_64, ppc64, i686) ...
> N: ----
> N: Comparing PackageKit-command-not-found-0.5.6-1.fc12 and PackageKit-command-not-found-0.5.6-1.fc12 (archs: ppc, x86_64, ppc64, i686) ...
> N: ----
> N: Comparing PackageKit-gstreamer-plugin-0.5.6-1.fc12 and PackageKit-gstreamer-plugin-0.5.6-1.fc12 (archs: ppc, x86_64, ppc64, i686) ...
> N: ----
> N: Comparing PackageKit-browser-plugin-0.5.6-1.fc12 and PackageKit-browser-plugin-0.5.6-1.fc12 (archs: ppc, x86_64, ppc64, i686) ...
> N: ----
> N: Comparing PackageKit-0.5.6-1.fc12 and PackageKit-0.5.6-1.fc12 (archs: ppc, x86_64, ppc64, i686) ...
> N: ----
> N: Comparing PackageKit-glib-devel-0.5.6-1.fc12 and PackageKit-glib-devel-0.5.6-1.fc12 (archs: ppc, x86_64, ppc64, i686) ...
> N: ----
> N: Comparing PackageKit-qt-0.5.6-1.fc12 and PackageKit-qt-0.5.6-1.fc12 (archs: ppc, x86_64, ppc64, i686) ...
> N: ----
> N: Comparing PackageKit-yum-plugin-0.5.6-1.fc12 and PackageKit-yum-plugin-0.5.6-1.fc12 (archs: ppc, x86_64, ppc64, i686) ...
> N: ----
> N: Comparing PackageKit-debug-install-0.5.6-1.fc12 and PackageKit-debug-install-0.5.6-1.fc12 (archs: ppc, x86_64, ppc64, i686) ...
> N: ----
> N: Comparing PackageKit-device-rebind-0.5.6-1.fc12 and PackageKit-device-rebind-0.5.6-1.fc12 (archs: ppc, x86_64, ppc64, i686) ...
> N: ----
> N: Comparing PackageKit-glib-0.5.6-1.fc12 and PackageKit-glib-0.5.6-1.fc12 (archs: ppc, x86_64, ppc64, i686) ...
> N: ----
> N: Comparing PackageKit-smart-0.5.6-1.fc12 and PackageKit-smart-0.5.6-1.fc12 (archs: ppc, x86_64, ppc64, i686) ...
> N: ----
> N: Comparing PackageKit-qt-devel-0.5.6-1.fc12 and PackageKit-qt-devel-0.5.6-1.fc12 (archs: ppc, x86_64, ppc64, i686) ...
> N: ----
Not sure what's going on, but trying to investigate why this test was
comparing two packages of the same nvr.
>>> import autoqa.koji_utils
>>> koji = autoqa.koji_utils.SimpleKojiClientSession()
>>> for pb in koji.list_previous_releases("PackageKit", "dist-f12-updates-candidate").values():
... print "Previous package for %s was %s" % ( pb['tag_name'], pb['nvr'] )
...
Previous package for dist-f12-updates was PackageKit-0.5.6-1.fc12
Previous package for dist-f12-updates was PackageKit-0.5.6-1.fc12
Previous package for dist-f12 was PackageKit-0.5.4-0.1.20091029git.fc12
I'm not sure whether list_previous_releases() is in error for returning
a duplicate entry or whether the rpmguard control file should handle
duplicate entries and keep scanning for the previous package?
Thanks,
James
14 years, 3 months
Re: architecture idea
by Kamil Paral
----- "James Laska" <jlaska(a)redhat.com> wrote:
>
> One challenge I see is designing a schema that can efficiently
> accommodate current and future front-end reporting needs. I often
> worry
> if we try to build something too universal, will it become useless?
> If
> the cost (time+energy) of making something generic is too high, I'd
> certainly be in favor of multiple front-ends and results servers. Do
> we
> have a good handle on what the specific schema needs each front-end
> might have?
Well this is certainly one of the harder parts. Unfortunately I don't
really know much about databases and I have just basic knowledge about
database design practices. But I believe this should not be rushed,
because a lot of code would have to be re-implemented after such change.
So I think it will take some week(s) of discussion.
If we know all the test types (installer tests, repository tests,
rpm package tests, etc), we should be able to create a schema generic
enough to accommodate all of those. If some new test type appears later,
then it may be quite difficult to put it into an existing schema
(unless the schema is really very generic, but then it's harder to
work with).
It is also possible to have several databases and some daemon listening
to all incoming results and classify them into relevant databases
according to the result type.
Do we have any database expert amongst us? Anyone? :)
PS: On IRC James mentioned we can have a look at autotest DB schema, so
I attach a link:
http://autotest.kernel.org/wiki/TkoDatabase
14 years, 3 months
architecture idea
by Kamil Paral
As we were discussing in the latest threads, we will probably
need some results database server in the future so we can then
access it by different front-ends or RPC calls. I tried to
think about the whole concept of AutoQA and create a picture
that would correspond to my ideas. Have a look at it at:
http://kparal.fedorapeople.org/misc/autoqa/architecture_idea.png
What are your opinions on this topic, which things need to be discussed,
what do you disagree with? I believe we should discuss these issues
properly so won't implement something that would need to be re-worked
soon after.
I see the database server as the biggest and nearest challenge now,
because the current irb site can easily display installer tests results,
but you can hardly use it for displaying package update results (rpmlint,
rpmguard). When we want to do something more with it than just sending
it to the mailing list, we will need the results server, I reckon.
Will, is it possible to separate the current irb database from the irb
front-end itself, so we could use it as a basis for the results database
server? And then connect the current front-end to it, as well as other
front-ends?
Thanks for responses,
Kamil
14 years, 3 months
Re: [Fwd: rpmguard: 0 warnings for package xorg-x11-server-1.7.4-2.fc12]
by Kamil Paral
----- "Will Woods" <wwoods(a)redhat.com> wrote:
> It would probably be better if it didn't send out email if there's
> nothing bad to report...
>
> -w
I can make that change for rpmlint and rpmguard, no problem. Other
tests currently send PASS results, but it's true it's a little
different category (and also the email volume is much lower).
Does anyone have another opinion, or shall I commit the patch?
I would just wait with rpmguard until its birth problems are
solved, so we just know if it works or not easily.
14 years, 3 months
