[AutoQA] #363: depcheck: don't require specific release

[fedora-badges]

#363: depcheck: don't require specific release
-------------------------+--------------------------------------------------
 Reporter:  kparal       |       Owner:             
     Type:  enhancement  |      Status:  new        
 Priority:  minor        |   Milestone:  Finger Food
Component:  tests        |    Keywords:             
-------------------------+--------------------------------------------------
 Depcheck currently requires machine with a specific Fedora release
 installed to test matching repositories.

 From tests/depcheck/control.autoqa:
 {{{
 # because we simulate installing packages, the autotest label of the
 correct distribution
 # must be present (like 'fc13') If proper label is not found the test will
 not execute
 if autoqa_args.has_key('nvrs'):
     from autoqa.util import get_distro
     distro = get_distro(autoqa_args['nvrs'][0])
     labels.append(distro)
 }}}

 Josef says it's possible to cancel this constraint. That would allow us to
 test arbitrary repository on arbitrary machine (improving performance and
 maybe allowing us to have less machines).

 There is one thing worth mentioning: Depcheck is tightly tied to yum, and
 any weird behavior might be caused by yum. If we cancel above-mentioned
 constraint, we can no longer suppose that "depcheck crashed on
 f15-updates, therefore I should try to reproduce it on my F15 machine". It
 will be necessary to look which machine the test was executed on and try
 to reproduce it in the same environment if possible.

 The goal of this ticket is test properly whether cancelling the
 requirement works correctly. That means to test executing depcheck for F15
 repos on F14 machine, or vice versa, and comparing the results with
 behavior prior to this change.

-- 
Ticket URL: <https://fedorahosted.org/autoqa/ticket/363>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project

[fedora-badges]

#363: depcheck: don't require specific release
-------------------------+--------------------------
 Reporter:  kparal       |       Owner:  jskladan
     Type:  enhancement  |      Status:  assigned
 Priority:  minor        |   Milestone:  Finger Food
Component:  tests        |  Resolution:
 Keywords:               |  Blocked By:
 Blocking:               |
-------------------------+--------------------------
Changes (by jskladan):

 * owner:   => jskladan
 * status:  new => assigned


Comment:

 As tested on F14 vs F16, depcheck has no 'hard' requirement for the
 specific distribution. But one needs to bear in mind, that if yum's
 depsolver would behave differently in (e.g.) F17, then running depcheck on
 F!=17 would return false results (beacause depcheck uses yum to do all the
 work).

-- 
Ticket URL: <https://fedorahosted.org/autoqa/ticket/363#comment:1>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project

[fedora-badges]

#363: depcheck: don't require specific release
-------------------------+--------------------------
 Reporter:  kparal       |       Owner:  kparal
     Type:  enhancement  |      Status:  new
 Priority:  minor        |   Milestone:  Finger Food
Component:  tests        |  Resolution:
 Keywords:               |  Blocked By:
 Blocking:               |
-------------------------+--------------------------
Changes (by kparal):

 * owner:  jskladan => kparal
 * status:  assigned => new


Comment:

 Because we don't currently want to maintain Fedora 17 Branched clients on
 staging and production, we decided to cancel this constraint. But I'll
 leave this ticket open until I confirm with yum developers whether our
 concern is well founded or extremely unlikely.

 Changed in commit 6a34bad.

-- 
Ticket URL: <https://fedorahosted.org/autoqa/ticket/363#comment:2>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project

[fedora-badges]

#363: depcheck: don't require specific release
-------------------------+---------------------
 Reporter:  kparal       |       Owner:  kparal
     Type:  enhancement  |      Status:  closed
 Priority:  minor        |   Milestone:  0.8.0
Component:  tests        |  Resolution:  fixed
 Keywords:               |  Blocked By:
 Blocking:               |
-------------------------+---------------------
Changes (by kparal):

 * status:  new => closed
 * resolution:   => fixed
 * milestone:  Finger Food => 0.8.0


Comment:

 This is a log of the conversation with yum developers. It seems that at
 least until libsolv starts to be used we should be fine with our current
 approach.

 {{{
 <kparal> basically we use yum's depsolving features inside our depcheck
 test inside AutoQA
 <kparal> because we are using mash, we found out that we can easily
 depsolve F17's repos on a F16 machine and it works great
 <kparal> so we don't need to maintain F17 machines just to be able to
 check whether new packages coming to F17 repos break dependencies or not
 <kparal> but there is a slight catch
 <kparal> if it ever happened that the depsolving algorithm is different
 for F16 yum and for F17 yum, we would produce invalid results
 <kparal> and my question is - is that likely?
 <kparal> skvidal: ^^
 <skvidal> geppetto: ^^
 <skvidal> kparal: I would guess, no it is not likely to change in the f17
 timeframe
 <skvidal> kparal: but I'm not the one working on these things
 <kparal> my question is more general than just  F16 and F17
 <kparal> can it happen in the future?
 <kparal> that the depsolving would change for one Fedora release, but not
 for others?
 <skvidal> sure.
 <kparal> so basically I should suppose it is likely to happen in some
 foreseeable future
 <kparal> that means I should always use F-XY machine to depsolve F-XY
 repos?
 <skvidal> kparal: which office are you located in?
 <kparal> skvidal: Brno
 <skvidal> kparal: you  know the dev conference coming up?
 <nphilipp> kparal: with the plans for a libsolv-based consolidated
 depsolver for rpm and yum (and others), I'm sure that there are subtle
 changes between how current yum code solves it now and how that solver
 will solve it when ready.
 <skvidal> kparal: ask pknirsch about his talk
 <kparal> skvidal: yep, but I won't be there. but my colleague will be
 there
 <kparal> we know that there are plans for libsolve
 <kparal> but the question is related strictly to yum
 <skvidal> that's in yum
 <kparal> ah
 <skvidal> that's the point
 <nphilipp> and when yum and rpm use libsolv (or whatever else other than
 yum), the depsolving will be different
 <skvidal> change will happen along there
 <pknirsch> well, depsolving can basically always be different, kparal
 <pknirsch> yum tries pretty successfully to keep it very stable
 <nphilipp> kparal: one reason for this is to have a "one-stop-shop" for
 depsolving
 <nphilipp> kparal: so regardless of the tool used, depsolving will come to
 the same solutions
 <pknirsch> but yes, with the switch to libsolv at one point in the future
 it's likely that the solution will look different
 <pknirsch> nphilipp: not with multiple possible solutions
 <pknirsch> if there is only one possible solution, true
 <kparal> we know that we will have to re-work depcheck once libsolv is
 being used
 <kparal> we re concerned with  time until that happens
 <pknirsch> otherwise, there is no guarantee that the new depsolver will
 produce always the same results as the one that is in yum right now
 <pknirsch> oh
 <nphilipp> pknirsch: if there is one depsolver (library), it should come
 to the same solution regardless of it being used by yum, PackageKit,
 whatever else
 <pknirsch> until then nothing will change
 <pknirsch> we will keep yum as it is right now
 <pknirsch> and introduce the new tool into Fedora once we feel it's ready
 to be used for testing
 <nphilipp> pknirsch: same set of inputs should generate the same output if
 the algo is the same :-)
 <pknirsch> then we'll keep it there for as long as we and everyone else
 isn't happy with the quality
 <nphilipp> pknirsch: "until we and everyone else is happy with the
 quality". positive phrases, you know :-)
 <kparal> ok, so IIUC, it's safe to suppose that until libsolv is being
 used, it's safe to assume that F17 yum and F18 yum uses the same algorithm
 <pknirsch> yep
 <kparal> or F16 and F17 yum. etc
 <kparal> ok, thanks very much
 <nphilipp> kparal: module bug fixes that affect depsolving
 <skvidal> kparal: but I would not assume that is true for the EL distros
 <nphilipp> *modulo
 <kparal> fortunately we don't need to handle EL now
 <nphilipp> kparal: you're using this for dep-checking, rather than
 solving? so you're use case is deciding whether dependencies of packages
 can be solved?
 <kparal> and the bug fixes are likely to be pushed to all supported
 releases I assume
 <kparal> nphilipp: yes, whether the new packages can be installed
 <kparal> without the actual installation
 <nphilipp> kparal: that should stay even more stable than actually solving
 how dependencies shall be fulfilled
 <kparal> great
 <nphilipp> kparal: AIUI, for your purposes your content to know that
 there's some package fulfilling dependency X of another package. Right?
 <nphilipp> *you're
 <nphilipp> I'm getting old
 <kparal> basically yes. we don't care what dependencies it pulls in, just
 that they can be satisfied
 <kparal> and we don't want to maintain Branched machines to check Branched
 repos. we want to do that on stable fedora releases
 <nphilipp> kparal: Then I guess what we expect to change shouldn't affect
 you much. There'll be an extensive testing period once we have something
 testable (as pknirsch said), be sure to participate then :-).
 <kparal> we will surely be interested
 <kparal> thanks for info
 <geppetto> kparal: I would say not … we've changed
"compare_providers" a
 few times, so it's certainly possible that they'll be small differences …
 I'm not sure if AutoQA will hit them though.
 <geppetto> kparal: What kinds of things do you do?
 <pknirsch> geppetto: but if they only do depchecking, not depsolving, then
 even a change to compare_provides() wouldn't affect their result, right?
 <pknirsch> as the solution would still be correct
 <geppetto> That's true … but that should be true with a libsolv yum too
 <pknirsch> aye
 <nphilipp> sure
 <pknirsch> so for your use case, kparal, even with the new tool you should
 still get the same results actually
 * geppetto isn't 100% sure the AutoQA thing is a true depcheck though
 <kparal> great. I have to admit it's a bit of black magic for us. it was
 written long time ago by wwoods. it uses private yum APIs. we hope that
 once libsolv is available we will be able to use its public APIs instead
 <kparal> but that's future
 <pknirsch> ah, wwoods, that np then, we're working closely with him on the
 PPC secondary arch stuff anyway :)
 <kparal> he's no longer working with Fedora QA, he's now an Anaconda guy
 <skvidal> geppetto: I was under the impression that autoqa is more doing
 an advanced kind of repoclosure
 <kparal> from what I know depcheck forces yum to think that all packages
 from stable repos are installed. and then asks yum whether we can install
 proposed updates
 <kparal> it has some corner cases but usually it works quite fine
 <nphilipp> kparal: unsure how this should work 100% or don't we have
 conflicting packages anymore?
 <jskladan> nphilipp: this is the "usually works quite fine" ;)
 <kparal> yeah, that's one of the few problems :)
 <nphilipp> haha
 <jskladan> but there is not that much conflicting packages as one would
 think
 <nphilipp> I would hope so :-9
 <jskladan> maybe kparal still remebers the number
 <kparal> nope
 }}}

-- 
Ticket URL: <https://fedorahosted.org/autoqa/ticket/363#comment:3>
AutoQA <http://autoqa.fedorahosted.org>
Automated QA project