11:37 a.m.
Hello,
I've noticed in
https://bodhi.fedoraproject.org/updates/FEDORA-2023-f34afe57a9
https://bodhi.fedoraproject.org/updates/FEDORA-2023-a61dbc3c1d
https://bodhi.fedoraproject.org/updates/FEDORA-2023-a61dbc3c1d
That the update is marked as critpath (which is probably correct because
python3-devel Requires it and many Python package BuildRequire it) and hence
the OpenQA tests run. However the probably unused in those tests, considering
it contains RPM macros.
Unless the OpenQA tests actually rebuild everything involved with this package
(which I doubt, because it's not that slow) before running the tests, what is
the point of running them for this update? Should it not be in critpath? Or
should such tests only run if the produced package would be installed in the
test environment even when not explicitly installed?
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
1:10 p.m.
New subject: OpenQA critpath Bodhi tests for packages that are most likely
not used during the tests
On Fri, 2023-03-03 at 18:37 +0100, Miro Hrončok wrote:
Hello,
I've noticed in
https://bodhi.fedoraproject.org/updates/FEDORA-2023-f34afe57a9
https://bodhi.fedoraproject.org/updates/FEDORA-2023-a61dbc3c1d
https://bodhi.fedoraproject.org/updates/FEDORA-2023-a61dbc3c1d
That the update is marked as critpath (which is probably correct because
python3-devel Requires it and many Python package BuildRequire it) and hence
the OpenQA tests run. However the probably unused in those tests, considering
it contains RPM macros.
Unless the OpenQA tests actually rebuild everything involved with this package
(which I doubt, because it's not that slow) before running the tests, what is
the point of running them for this update? Should it not be in critpath? Or
should such tests only run if the produced package would be installed in the
test environment even when not explicitly installed?
There is no point, but there may be nothing much we can do about it.
I have been improving things in this area recently with a lot of work
under the general heading "grouped critical path", but if you mouseover
the critpath icon for the update, you'll see the package is in the core
group, and the kde, gnome and server critical path groups (which means
something explicitly listed in each of those groups ultimately depends
on it).
If something is in the core group, we kind of have to schedule all the
openQA tests for it. Things in core could potentially break *anything*.
It is an unavoidable hazard that, sometimes, via long dep chains, this
results in us running the tests on things they don't really cover.
Similarly, we have to run the GNOME tests for things in the GNOME
critpath group, the KDE tests for things in the KDE critpath group, and
the Server tests for the things in the Server critpath group. So even
if we manage to get it out of @core , we would still have to run the
KDE, GNOME and Server tests on it unless we could get it out of those
groups too.
So the question really is, what causes it to be in those groups? I
suspect it's something in the dep chain of pyproject-srpm-macros , but
at a quick glance I can't figure out what. If anyone has a good dep
visualizer they can throw at the problem it'd help, I guess.
--
Adam Williamson (he/him/his)
Fedora QA
Fedora Chat: @adamwill:fedora.im | Mastodon: @adamw(a)fosstodon.org
https://www.happyassassin.net
3:24 p.m.
On 03. 03. 23 20:10, Adam Williamson wrote:
On Fri, 2023-03-03 at 18:37 +0100, Miro Hrončok wrote:
> Hello,
> I've noticed in
>
> https://bodhi.fedoraproject.org/updates/FEDORA-2023-f34afe57a9
> https://bodhi.fedoraproject.org/updates/FEDORA-2023-a61dbc3c1d
> https://bodhi.fedoraproject.org/updates/FEDORA-2023-a61dbc3c1d
>
> That the update is marked as critpath (which is probably correct because
> python3-devel Requires it and many Python package BuildRequire it) and hence
> the OpenQA tests run. However the probably unused in those tests, considering
> it contains RPM macros.
>
> Unless the OpenQA tests actually rebuild everything involved with this package
> (which I doubt, because it's not that slow) before running the tests, what is
> the point of running them for this update? Should it not be in critpath? Or
> should such tests only run if the produced package would be installed in the
> test environment even when not explicitly installed?
There is no point, but there may be nothing much we can do about it.
I have been improving things in this area recently with a lot of work
under the general heading "grouped critical path", but if you mouseover
the critpath icon for the update, you'll see the package is in the core
group, and the kde, gnome and server critical path groups (which means
something explicitly listed in each of those groups ultimately depends
on it).
If something is in the core group, we kind of have to schedule all the
openQA tests for it. Things in core could potentially break *anything*.
It is an unavoidable hazard that, sometimes, via long dep chains, this
results in us running the tests on things they don't really cover.
Similarly, we have to run the GNOME tests for things in the GNOME
critpath group, the KDE tests for things in the KDE critpath group, and
the Server tests for the things in the Server critpath group. So even
if we manage to get it out of @core , we would still have to run the
KDE, GNOME and Server tests on it unless we could get it out of those
groups too.
So the question really is, what causes it to be in those groups? I
suspect it's something in the dep chain of pyproject-srpm-macros , but
at a quick glance I can't figure out what. If anyone has a good dep
visualizer they can throw at the problem it'd help, I guess.
Thanks for the answer. I think it's:
# dnf install @critical-path-build
...
Installing group/module packages:
redhat-rpm-config
...
Installing dependencies:
pyproject-srpm-macros
...
So the question is: Are there test that test building RPM packages? If not, why
is @critical-path-build included?
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
5:31 p.m.
New subject: OpenQA critpath Bodhi tests for packages that are most likely
not used during the tests
On Fri, 2023-03-03 at 22:24 +0100, Miro Hrončok wrote:
On 03. 03. 23 20:10, Adam Williamson wrote:
> On Fri, 2023-03-03 at 18:37 +0100, Miro Hrončok wrote:
> > Hello,
> > I've noticed in
> >
> > https://bodhi.fedoraproject.org/updates/FEDORA-2023-f34afe57a9
> > https://bodhi.fedoraproject.org/updates/FEDORA-2023-a61dbc3c1d
> > https://bodhi.fedoraproject.org/updates/FEDORA-2023-a61dbc3c1d
> >
> > That the update is marked as critpath (which is probably correct because
> > python3-devel Requires it and many Python package BuildRequire it) and hence
> > the OpenQA tests run. However the probably unused in those tests, considering
> > it contains RPM macros.
> >
> > Unless the OpenQA tests actually rebuild everything involved with this package
> > (which I doubt, because it's not that slow) before running the tests, what
is
> > the point of running them for this update? Should it not be in critpath? Or
> > should such tests only run if the produced package would be installed in the
> > test environment even when not explicitly installed?
>
> There is no point, but there may be nothing much we can do about it.
>
> I have been improving things in this area recently with a lot of work
> under the general heading "grouped critical path", but if you mouseover
> the critpath icon for the update, you'll see the package is in the core
> group, and the kde, gnome and server critical path groups (which means
> something explicitly listed in each of those groups ultimately depends
> on it).
>
> If something is in the core group, we kind of have to schedule all the
> openQA tests for it. Things in core could potentially break *anything*.
> It is an unavoidable hazard that, sometimes, via long dep chains, this
> results in us running the tests on things they don't really cover.
>
> Similarly, we have to run the GNOME tests for things in the GNOME
> critpath group, the KDE tests for things in the KDE critpath group, and
> the Server tests for the things in the Server critpath group. So even
> if we manage to get it out of @core , we would still have to run the
> KDE, GNOME and Server tests on it unless we could get it out of those
> groups too.
>
> So the question really is, what causes it to be in those groups? I
> suspect it's something in the dep chain of pyproject-srpm-macros , but
> at a quick glance I can't figure out what. If anyone has a good dep
> visualizer they can throw at the problem it'd help, I guess.
Thanks for the answer. I think it's:
# dnf install @critical-path-build
...
Installing group/module packages:
redhat-rpm-config
...
Installing dependencies:
pyproject-srpm-macros
...
So the question is: Are there test that test building RPM packages? If not, why
is @critical-path-build included?
No, that's a misunderstanding. critical-path-build is one of the very
new groups I created recently specifically to *help* with this problem.
critical-path-build exists precisely so that openQA can *not* run tests
on packages that are (only) in that group.
The problem here is that the package is in a bunch of critpath groups,
including the key ones I listed above - @core, @critical-path-kde,
@critical-path-gnome, @critical-path-server. We run all the openQA
tests for anything that's in @core, because, well, we kinda have to. We
also run the openQA KDE tests on anything that's in @critical-path-kde
, the GNOME tests on anything that's in @critical-path-gnome, and the
Server tests on anything that's in @critical-path-server (for obvious
reasons). So as long as it's in @core, *all* the tests will be run on
it. If we get it out of @core but it's still in the other three groups,
we'll still run the KDE, GNOME and Server tests on it (which is nearly
all of the tests).
Interestingly, this is only the case for the F38 update. Check the F39
update:
https://bodhi.fedoraproject.org/updates/FEDORA-2023-23789cd657
it's *only* in @critical-path-build, it's not in @core or the kde,
gnome or server groups. So, as you can see, no tests are run on it.
I know why this would happen for F36 or F37, but I'm surprised it's
happening for F38. F38 and F39 ought to behave basically the same so
far as this stuff is concerned. I'll poke around a bit and see if I can
figure out why.
--
Adam Williamson (he/him/his)
Fedora QA
Fedora Chat: @adamwill:fedora.im | Mastodon: @adamw(a)fosstodon.org
https://www.happyassassin.net
5:45 p.m.
On 04. 03. 23 0:31, Adam Williamson wrote:
On Fri, 2023-03-03 at 22:24 +0100, Miro Hrončok wrote:
> On 03. 03. 23 20:10, Adam Williamson wrote:
>> On Fri, 2023-03-03 at 18:37 +0100, Miro Hrončok wrote:
>>> Hello,
>>> I've noticed in
>>>
>>> https://bodhi.fedoraproject.org/updates/FEDORA-2023-f34afe57a9
>>> https://bodhi.fedoraproject.org/updates/FEDORA-2023-a61dbc3c1d
>>> https://bodhi.fedoraproject.org/updates/FEDORA-2023-a61dbc3c1d
>>>
>>> That the update is marked as critpath (which is probably correct because
>>> python3-devel Requires it and many Python package BuildRequire it) and hence
>>> the OpenQA tests run. However the probably unused in those tests,
considering
>>> it contains RPM macros.
>>>
>>> Unless the OpenQA tests actually rebuild everything involved with this
package
>>> (which I doubt, because it's not that slow) before running the tests,
what is
>>> the point of running them for this update? Should it not be in critpath? Or
>>> should such tests only run if the produced package would be installed in the
>>> test environment even when not explicitly installed?
>>
>> There is no point, but there may be nothing much we can do about it.
>>
>> I have been improving things in this area recently with a lot of work
>> under the general heading "grouped critical path", but if you
mouseover
>> the critpath icon for the update, you'll see the package is in the core
>> group, and the kde, gnome and server critical path groups (which means
>> something explicitly listed in each of those groups ultimately depends
>> on it).
>>
>> If something is in the core group, we kind of have to schedule all the
>> openQA tests for it. Things in core could potentially break *anything*.
>> It is an unavoidable hazard that, sometimes, via long dep chains, this
>> results in us running the tests on things they don't really cover.
>>
>> Similarly, we have to run the GNOME tests for things in the GNOME
>> critpath group, the KDE tests for things in the KDE critpath group, and
>> the Server tests for the things in the Server critpath group. So even
>> if we manage to get it out of @core , we would still have to run the
>> KDE, GNOME and Server tests on it unless we could get it out of those
>> groups too.
>>
>> So the question really is, what causes it to be in those groups? I
>> suspect it's something in the dep chain of pyproject-srpm-macros , but
>> at a quick glance I can't figure out what. If anyone has a good dep
>> visualizer they can throw at the problem it'd help, I guess.
>
> Thanks for the answer. I think it's:
>
> # dnf install @critical-path-build
> ...
> Installing group/module packages:
> redhat-rpm-config
> ...
> Installing dependencies:
> pyproject-srpm-macros
> ...
>
> So the question is: Are there test that test building RPM packages? If not, why
> is @critical-path-build included?
No, that's a misunderstanding. critical-path-build is one of the very
new groups I created recently specifically to *help* with this problem.
critical-path-build exists precisely so that openQA can *not* run tests
on packages that are (only) in that group.
The problem here is that the package is in a bunch of critpath groups,
including the key ones I listed above - @core, @critical-path-kde,
@critical-path-gnome, @critical-path-server. We run all the openQA
tests for anything that's in @core, because, well, we kinda have to. We
also run the openQA KDE tests on anything that's in @critical-path-kde
, the GNOME tests on anything that's in @critical-path-gnome, and the
Server tests on anything that's in @critical-path-server (for obvious
reasons). So as long as it's in @core, *all* the tests will be run on
it. If we get it out of @core but it's still in the other three groups,
we'll still run the KDE, GNOME and Server tests on it (which is nearly
all of the tests).
Interestingly, this is only the case for the F38 update. Check the F39
update:
https://bodhi.fedoraproject.org/updates/FEDORA-2023-23789cd657
it's *only* in @critical-path-build, it's not in @core or the kde,
gnome or server groups. So, as you can see, no tests are run on it.
I know why this would happen for F36 or F37, but I'm surprised it's
happening for F38. F38 and F39 ought to behave basically the same so
far as this stuff is concerned. I'll poke around a bit and see if I can
figure out why.
Oh I see that all 3 updates are in critical-path-base, but only the F38 one is
in kde/gnome etc.
Either way, when I dnf install @core on any Fedora version, I never get the
package.
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
7:21 p.m.
New subject: OpenQA critpath Bodhi tests for packages that are most likely
not used during the tests
On Fri, 2023-03-03 at 15:31 -0800, Adam Williamson wrote:
Interestingly, this is only the case for the F38 update. Check the F39
update:
https://bodhi.fedoraproject.org/updates/FEDORA-2023-23789cd657
it's *only* in @critical-path-build, it's not in @core or the kde,
gnome or server groups. So, as you can see, no tests are run on it.
I know why this would happen for F36 or F37, but I'm surprised it's
happening for F38. F38 and F39 ought to behave basically the same so
far as this stuff is concerned. I'll poke around a bit and see if I can
figure out why.
So I dug into this a bit and it looks like we probably haven't been
updating the F38 critical path since freeze, which might be related.
I've sent a PR to fix that:
https://pagure.io/releng/pull-request/11322
When I generate critpath locally with that patch, pyproject-rpm-macros
only winds up in critical-path-build for F38, so I'm hoping if we get
that merged and critpath regenerated, this will be fixed for future
updates.
I'm still a bit baffled at how it wound up in so *many* groups, though.
I don't see how that could have happened, I can't reproduce that result
currently. Worst case it should be in critical-path-base and critical-
path-build. I don't see how it can have gotten into the other groups...
--
Adam Williamson (he/him/his)
Fedora QA
Fedora Chat: @adamwill:fedora.im | Mastodon: @adamw(a)fosstodon.org
https://www.happyassassin.net
7:25 p.m.
On Sat, Mar 4, 2023, 02:21 Adam Williamson <adamwill(a)fedoraproject.org>
wrote:
On Fri, 2023-03-03 at 15:31 -0800, Adam Williamson wrote:
>
> Interestingly, this is only the case for the F38 update. Check the F39
> update:
> https://bodhi.fedoraproject.org/updates/FEDORA-2023-23789cd657
>
> it's *only* in @critical-path-build, it's not in @core or the kde,
> gnome or server groups. So, as you can see, no tests are run on it.
>
> I know why this would happen for F36 or F37, but I'm surprised it's
> happening for F38. F38 and F39 ought to behave basically the same so
> far as this stuff is concerned. I'll poke around a bit and see if I can
> figure out why.
So I dug into this a bit and it looks like we probably haven't been
updating the F38 critical path since freeze, which might be related.
I've sent a PR to fix that:
https://pagure.io/releng/pull-request/11322
When I generate critpath locally with that patch, pyproject-rpm-macros
only winds up in critical-path-build for F38, so I'm hoping if we get
that merged and critpath regenerated, this will be fixed for future
updates.
I'm still a bit baffled at how it wound up in so *many* groups, though.
I don't see how that could have happened, I can't reproduce that result
currently. Worst case it should be in critical-path-base and critical-
path-build. I don't see how it can have gotten into the other groups...
Maybe it was caused by the dependency of pkgconf -> redhat-rpm-macros that
was dropped again a few days ago? That's the only recent change I can think
of that could explain this, and the timing seems to line up.
Fabio
--
Adam Williamson (he/him/his)
Fedora QA
Fedora Chat: @adamwill:fedora.im | Mastodon: @adamw(a)fosstodon.org
https://www.happyassassin.net
_______________________________________________
devel mailing list -- devel(a)lists.fedoraproject.org
To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it:
https://pagure.io/fedora-infrastructure/new_issue
7:30 p.m.
New subject: OpenQA critpath Bodhi tests for packages that are most likely
not used during the tests
On Sat, 2023-03-04 at 02:25 +0100, Fabio Valentini wrote:
On Sat, Mar 4, 2023, 02:21 Adam Williamson
<adamwill(a)fedoraproject.org>
wrote:
> On Fri, 2023-03-03 at 15:31 -0800, Adam Williamson wrote:
> >
> > Interestingly, this is only the case for the F38 update. Check the F39
> > update:
> > https://bodhi.fedoraproject.org/updates/FEDORA-2023-23789cd657
> >
> > it's *only* in @critical-path-build, it's not in @core or the kde,
> > gnome or server groups. So, as you can see, no tests are run on it.
> >
> > I know why this would happen for F36 or F37, but I'm surprised it's
> > happening for F38. F38 and F39 ought to behave basically the same so
> > far as this stuff is concerned. I'll poke around a bit and see if I can
> > figure out why.
>
> So I dug into this a bit and it looks like we probably haven't been
> updating the F38 critical path since freeze, which might be related.
> I've sent a PR to fix that:
>
> https://pagure.io/releng/pull-request/11322
>
> When I generate critpath locally with that patch, pyproject-rpm-macros
> only winds up in critical-path-build for F38, so I'm hoping if we get
> that merged and critpath regenerated, this will be fixed for future
> updates.
>
> I'm still a bit baffled at how it wound up in so *many* groups, though.
> I don't see how that could have happened, I can't reproduce that result
> currently. Worst case it should be in critical-path-base and critical-
> path-build. I don't see how it can have gotten into the other groups...
>
Maybe it was caused by the dependency of pkgconf -> redhat-rpm-macros that
was dropped again a few days ago? That's the only recent change I can think
of that could explain this, and the timing seems to line up.
Oooh, yeah, that could definitely be it, and would line up with the
critpath generation for F38 stopping when it froze. Thanks! I like when
mysteries are solved. :D
--
Adam Williamson (he/him/his)
Fedora QA
Fedora Chat: @adamwill:fedora.im | Mastodon: @adamw(a)fosstodon.org
https://www.happyassassin.net
427
days inactive
428
days old
7 comments
3 participants
participants (3)
-
Adam Williamson -
Fabio Valentini -
Miro Hrončok