New test for autoqa - upgradepath

[Vojtěch Aschenbrenner]

Hi,

I created new test for autoqa. Upgradepath will check if version of
pushing package has good version. See ticket #123 at
https://fedorahosted.org/autoqa/ticket/123 . This is first version,
which will NOT consider update-testing repository (just dist-fXX and
dist-fXX-update).

You can get it from upgradepath branch in git.
Try for example:

$ autoqa post-bodhi-update --updateid FEDORA-2010-11132        \
--title mc-4.7.1-2.fc11 --targettag dist-f11 1:mc-4.7.1-2.fc11 \
--arch x86_64 -t upgradepath --local

This will check if 1:mc-4.7.1-2.fc11 could be pushed to dist-f11. So
every lower repos have to have lower or equal version and higher repos
higher or equal version.

Next version will consider update-testing repos. Request for comments
:-).

Vojta

[Kamil Paral]

----- "Vojtěch Aschenbrenner" <vaschenb(a)redhat.com&gt; wrote:

>  Hi,
>  
>  I created new test for autoqa. Upgradepath will check if version of
>  pushing package has good version. See ticket #123 at
>  https://fedorahosted.org/autoqa/ticket/123 . This is first version,
>  which will NOT consider update-testing repository (just dist-fXX and
>  dist-fXX-update).
>  
>  You can get it from upgradepath branch in git.
>  Try for example:
>  
>  $ autoqa post-bodhi-update --updateid FEDORA-2010-11132        \
>  --title mc-4.7.1-2.fc11 --targettag dist-f11 1:mc-4.7.1-2.fc11 \
>  --arch x86_64 -t upgradepath --local
>  
>  This will check if 1:mc-4.7.1-2.fc11 could be pushed to dist-f11. So
>  every lower repos have to have lower or equal version and higher
>  repos
>  higher or equal version. 
This raises a few interesting questions:

1) Pushing to dist-f11 is nonsense, right? That repo is unpushable, we
can only push to dist-f11-updates or dist-f11-updates-testing. But is
it a purpose of this script to check this issue? Should it report an
error or just leave out these semantics?

2) Let's say we want to push to dist-f11-updates. We want to push foo-1.1
package. But foo-1.1 is already present in dist-f11-updates! Again,
should this test be concerned with this? It is not exactly upgrade path
issue, it's another problem elsewhere.

So in both issues (and maybe some other similar ones) I think we should
just check what should be checked (upgrade path breakage) and overlook
any other circumstances that may be weird but that are not our concern.

What do you think?

>  
>  Next version will consider update-testing repos. Request for comments
>  :-).
>  
>  Vojta
>  _______________________________________________
>  autoqa-devel mailing list
>  autoqa-devel(a)lists.fedorahosted.org
>  https://fedorahosted.org/mailman/listinfo/autoqa-devel

[James Laska]

On Fri, 2010-08-06 at 08:12 -0400, Kamil Paral wrote:
>  ----- "Vojtěch Aschenbrenner" <vaschenb(a)redhat.com&gt;
> wrote:
>  
>  > Hi,
>  > 
>  > I created new test for autoqa. Upgradepath will check if version of
>  > pushing package has good version. See ticket #123 at
>  > https://fedorahosted.org/autoqa/ticket/123 . This is first version,
>  > which will NOT consider update-testing repository (just dist-fXX and
>  > dist-fXX-update).
>  > 
>  > You can get it from upgradepath branch in git.
>  > Try for example:
>  > 
>  > $ autoqa post-bodhi-update --updateid FEDORA-2010-11132        \
>  > --title mc-4.7.1-2.fc11 --targettag dist-f11 1:mc-4.7.1-2.fc11 \
>  > --arch x86_64 -t upgradepath --local
>  > 
>  > This will check if 1:mc-4.7.1-2.fc11 could be pushed to dist-f11. So
>  > every lower repos have to have lower or equal version and higher
>  > repos
>  > higher or equal version.
>  
>  This raises a few interesting questions:
>  
>  1) Pushing to dist-f11 is nonsense, right? That repo is unpushable, we
>  can only push to dist-f11-updates or dist-f11-updates-testing. But is
>  it a purpose of this script to check this issue? Should it report an
>  error or just leave out these semantics? 
For released versions of Fedora, I think you're right.  After we spin up
the final RC, dist-fNN doesn't change.  Only the contents of
dist-fNN-updates and dist-fNN-updates-testing change at that point.

The branched release is an exception though.  During release
stabilization, dist-fNN and dist-fNN-updates-testing both receive new
packages, while dist-fNN-updates isn't used.

That said, the watcher will be providing the value for --targettag
right?  This comes from post-bodhi-update?  So if the bodhi watcher is
requesting an update for a package in dist-fNN to a released Fedora ...
that would be a watcher/bodhi or a policy change.

>  2) Let's say we want to push to dist-f11-updates. We want to push
> foo-1.1
>  package. But foo-1.1 is already present in dist-f11-updates! Again,
>  should this test be concerned with this? It is not exactly upgrade path
>  issue, it's another problem elsewhere. 
I don't think koji lets you into this situation since can't build the
same NVR into a tag.  Still, noting that outcome might be a good idea.

Thanks,
James

[Vojtěch Aschenbrenner]

On Fri, Aug 06, 2010 at 01:00:29PM +0200, Vojtěch Aschenbrenner wrote:
>  Next version will consider update-testing repos. Request for
> comments
>  :-). 
Here comes the problem. According to possibility of reverting package
versions from updates-testing repos, this test is useless. When the test
will pass one day, next day somebody can delete package from higher
repository and the test is invalid and will fail.

So the question is, is it necessary to test update-testing repos in
upgradepath test? Vojta, Kamil and Joseph say NO.

[James Laska]

On Tue, 2010-08-10 at 14:40 +0200, Vojtěch Aschenbrenner wrote:
>  On Fri, Aug 06, 2010 at 01:00:29PM +0200, Vojtěch Aschenbrenner
> wrote:
>  > Next version will consider update-testing repos. Request for comments
>  > :-).
>  
>  Here comes the problem. According to possibility of reverting package
>  versions from updates-testing repos, this test is useless. When the test
>  will pass one day, next day somebody can delete package from higher
>  repository and the test is invalid and will fail.
>  
>  So the question is, is it necessary to test update-testing repos in
>  upgradepath test? Vojta, Kamil and Joseph say NO. 
Geez, never fully thought through all these possibilities!  It's good to
know we have the option of additional updates-testing handling if needed
in the future, but I see no reason to include that now.  If the code is
already there, can this be a command-line option for the test (used only
when manually testing)?

Otherwise, I agree with you guys, let's start with testing upgrade path
for stable and released updates.  We can adjust as needed in the future.

Thanks,
James