11:47 p.m.
Hi,
I'm considering dropping some packages from sway:rolling modulemd
definition, however I don't clearly understand what would be the
resulting upgrade behavior.
Let's assume that the module is already enabled and all the packages are
installed in the user's system. Also, the module was containing the same
set of packages during f32 release (i.e. in fedora-modular).
In an optimistic case I'd like to believe that once the package is
removed from the modulemd file and a new module update is published, the
override magic hacked into dnf would stop masking package from
non-modular repository. NEVRA would be the only factor deciding the
update priority and when we push new build of the package into
f32-updates it would update the remaining modular build of the package
with lower EVR.
In a pessimistic case I expect that the package removed from the module
will never be updated by dnf in the same release cycle since the already
installed version originates from the modular repository. Upgrade to f33
would require module reset.
I have zero knowledge on the modularity implementation so I would
appreciate if anyone could clarify which behavior is more likely to
happen before we start experimenting on real users :)
--
With best regards,
Aleksei Bavshin
4:04 a.m.
On 17. 07. 20 6:47, Aleksei Bavshin wrote:
In an optimistic case I'd like to believe that once the package
is
removed from the modulemd file and a new module update is published, the
override magic hacked into dnf would stop masking package from
non-modular repository. NEVRA would be the only factor deciding the
update priority and when we push new build of the package into
f32-updates it would update the remaining modular build of the package
with lower EVR.
If the package was part of the module during the release of Fedora 32, dnf will
see 2 versions/builds of the module-stream:
- the one that existed during the release (containing the package)
- the one from updates (no longer containing the package)
The "original" version will still filter out the non-modular version of the
package from any transaction.
And given the way modular filtering works, a package cannot technically be
"demodularized" this was during one release, unless you force a module reset
from another package's scriptlet.
In a pessimistic case I expect that the package removed from the
module
will never be updated by dnf in the same release cycle since the already
installed version originates from the modular repository. Upgrade to f33
would require module reset.
During the update to Fedora 33 a modular reset should happen anyway. But even if
it doesn't I believe that the packages would be upgraded to non-modular,
assuming their EVR is higher. I might be wrong thou, because the concept of
removing packages from modules and making them non-modular is full of booby traps.
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
5:27 a.m.
Dne 17. 07. 20 v 11:04 Miro Hrončok napsal(a):
On 17. 07. 20 6:47, Aleksei Bavshin wrote:
> In an optimistic case I'd like to believe that once the package is
> removed from the modulemd file and a new module update is published, the
> override magic hacked into dnf would stop masking package from
> non-modular repository. NEVRA would be the only factor deciding the
> update priority and when we push new build of the package into
> f32-updates it would update the remaining modular build of the package
> with lower EVR.
If the package was part of the module during the release of Fedora 32,
dnf will see 2 versions/builds of the module-stream:
- the one that existed during the release (containing the package)
- the one from updates (no longer containing the package)
The "original" version will still filter out the non-modular version of
the package from any transaction.
And given the way modular filtering works, a package cannot technically
be "demodularized" this was during one release, unless you force a
module reset from another package's scriptlet.
Thanks Miro,
That exactly describes what happens.
> In a pessimistic case I expect that the package removed from the
module
> will never be updated by dnf in the same release cycle since the already
> installed version originates from the modular repository. Upgrade to f33
> would require module reset.
During the update to Fedora 33 a modular reset should happen anyway. But
even if it doesn't I believe that the packages would be upgraded to
non-modular, assuming their EVR is higher. I might be wrong thou,
because the concept of removing packages from modules and making them
non-modular is full of booby traps.
The module reset is essential, because otherwise DNF will keep using
fail-safe module data on disk and that would still keep the package
filtered.
I suggest making the change in F33 / Rawhide.
Rawhide shouldn't break, because there's only one (the latest) version
of each module in the repodata and if you release a new modulemd without
the package, everything should work as expected.
7:15 a.m.
On 7/17/20 3:27 AM, Daniel Mach wrote:
Dne 17. 07. 20 v 11:04 Miro Hrončok napsal(a):
> If the package was part of the module during the release of Fedora 32,
> dnf will see 2 versions/builds of the module-stream:
>
> - the one that existed during the release (containing the package)
> - the one from updates (no longer containing the package)
>
> The "original" version will still filter out the non-modular version
> of the package from any transaction.
>
> And given the way modular filtering works, a package cannot
> technically be "demodularized" this was during one release, unless you
> force a module reset from another package's scriptlet.
Thanks for clarifying this, Miro. So the filter is built based on the
module metadata instead of the installed packages. I guess I
misunderstood that part.
That's just sad. I woke up with the bright idea about removing the
packages from modulemd and then adding `sway-module-obsoletes` which
Obsoletes everything in the module that's behind f32-updates. We have
several packages like this since the last module rebuild.
If there's a cached non-updateable module metadata from fedora-modular
which is always considered by dnf together with the updated version from
fedora-updates-modular, that had no chance of working.
> During the update to Fedora 33 a modular reset should happen
anyway.
> But even if it doesn't I believe that the packages would be upgraded
> to non-modular, assuming their EVR is higher. I might be wrong thou,
> because the concept of removing packages from modules and making them
> non-modular is full of booby traps.
The module reset is essential, because otherwise DNF will keep using
fail-safe module data on disk and that would still keep the package
filtered.
I suggest making the change in F33 / Rawhide.
Rawhide shouldn't break, because there's only one (the latest) version
of each module in the repodata and if you release a new modulemd without
the package, everything should work as expected.
Your suggestion implies that
there's a way to build different set of
packages for different releases within the same module stream, right?
Because with what Miro said, we are stuck with the requirement to update
every package we have in the module until f32 is EOL. I can't just drop
package x from modulemd and say "x won't be ever updated in f32 because
modularity" :(
--
With best regards,
Aleksei Bavshin
3:50 p.m.
Dne 17. 07. 20 v 14:15 Aleksei Bavshin napsal(a):
On 7/17/20 3:27 AM, Daniel Mach wrote:
> Dne 17. 07. 20 v 11:04 Miro Hrončok napsal(a):
>> If the package was part of the module during the release of Fedora 32,
>> dnf will see 2 versions/builds of the module-stream:
>>
>> - the one that existed during the release (containing the package)
>> - the one from updates (no longer containing the package)
>>
>> The "original" version will still filter out the non-modular version
>> of the package from any transaction.
>>
>> And given the way modular filtering works, a package cannot
>> technically be "demodularized" this was during one release, unless you
>> force a module reset from another package's scriptlet.
Thanks for clarifying this, Miro. So the filter is built based on the
module metadata instead of the installed packages. I guess I
misunderstood that part.
That's just sad. I woke up with the bright idea about removing the
packages from modulemd and then adding `sway-module-obsoletes` which
Obsoletes everything in the module that's behind f32-updates. We have
several packages like this since the last module rebuild.
If there's a cached non-updateable module metadata from fedora-modular
which is always considered by dnf together with the updated version from
fedora-updates-modular, that had no chance of working.
That's exactly how it
works.
Unlike RPMs where usually the latest version is the result of a
transaction, module metadata are additive.
Sum of all versions make a stream and it's content (incl. the old RPM
versions) so you can eventually downgrade your package if you need to.
We know about the issue and we want to introduce some de-modularization
feature, but we need to fix more serious issues first.
>> During the update to Fedora 33 a modular reset should happen anyway.
>> But even if it doesn't I believe that the packages would be upgraded
>> to non-modular, assuming their EVR is higher. I might be wrong thou,
>> because the concept of removing packages from modules and making them
>> non-modular is full of booby traps.
>
> The module reset is essential, because otherwise DNF will keep using
> fail-safe module data on disk and that would still keep the package
> filtered.
>
> I suggest making the change in F33 / Rawhide.
> Rawhide shouldn't break, because there's only one (the latest) version
> of each module in the repodata and if you release a new modulemd without
> the package, everything should work as expected.
Your suggestion implies that there's a way to build different set of
packages for different releases within the same module stream, right?
Because with what Miro said, we are stuck with the requirement to update
every package we have in the module until f32 is EOL. I can't just drop
package x from modulemd and say "x won't be ever updated in f32 because
modularity" :(
The F33 change I suggested doesn't retrospectively apply on F32.
You're really stuck maintaining the package until F32 EOL.
4:14 p.m.
On 7/17/20 1:50 PM, Daniel Mach wrote:
Dne 17. 07. 20 v 14:15 Aleksei Bavshin napsal(a):
> On 7/17/20 3:27 AM, Daniel Mach wrote:
>> Dne 17. 07. 20 v 11:04 Miro Hrončok napsal(a):
>>> If the package was part of the module during the release of Fedora 32,
>>> dnf will see 2 versions/builds of the module-stream:
>>>
>>> - the one that existed during the release (containing the package)
>>> - the one from updates (no longer containing the package)
>>>
>>> The "original" version will still filter out the non-modular
version
>>> of the package from any transaction.
>>>
>>> And given the way modular filtering works, a package cannot
>>> technically be "demodularized" this was during one release, unless
you
>>> force a module reset from another package's scriptlet.
>
> Thanks for clarifying this, Miro. So the filter is built based on the
> module metadata instead of the installed packages. I guess I
> misunderstood that part.
>
> That's just sad. I woke up with the bright idea about removing the
> packages from modulemd and then adding `sway-module-obsoletes` which
> Obsoletes everything in the module that's behind f32-updates. We have
> several packages like this since the last module rebuild.
>
> If there's a cached non-updateable module metadata from fedora-modular
> which is always considered by dnf together with the updated version from
> fedora-updates-modular, that had no chance of working.
That's exactly how it works.
Unlike RPMs where usually the latest version is the result of a
transaction, module metadata are additive.
Sum of all versions make a stream and it's content (incl. the old RPM
versions) so you can eventually downgrade your package if you need to.
We know about the issue and we want to introduce some de-modularization
feature, but we need to fix more serious issues first.
>
>>> During the update to Fedora 33 a modular reset should happen anyway.
>>> But even if it doesn't I believe that the packages would be upgraded
>>> to non-modular, assuming their EVR is higher. I might be wrong thou,
>>> because the concept of removing packages from modules and making them
>>> non-modular is full of booby traps.
>>
>> The module reset is essential, because otherwise DNF will keep using
>> fail-safe module data on disk and that would still keep the package
>> filtered.
>>
>> I suggest making the change in F33 / Rawhide.
>> Rawhide shouldn't break, because there's only one (the latest) version
>> of each module in the repodata and if you release a new modulemd without
>> the package, everything should work as expected.
> Your suggestion implies that there's a way to build different set of
> packages for different releases within the same module stream, right?
>
> Because with what Miro said, we are stuck with the requirement to update
> every package we have in the module until f32 is EOL. I can't just drop
> package x from modulemd and say "x won't be ever updated in f32 because
> modularity" :(
>
The F33 change I suggested doesn't retrospectively apply on F32.
You're really stuck maintaining the package until F32 EOL.
Yes, that's something I already accepted.
The real question is how to do the change in f33 considering that f32
and f33 modules must be built from the same modulemd file. I only see
the ability to disable module stream build for f33.
And now I'm curious what would happen if I specify `platform: [-f33]`
and publish new module build. Would that remove previously published
metadata from rawhide?
I guess the right answer is somewhere around servicelevels and `eol`
specification.
--
With best regards,
Aleksei Bavshin
6:46 a.m.
On Fri, Jul 17, 2020 at 02:14:49PM -0700, Aleksei Bavshin wrote:
The real question is how to do the change in f33 considering that
f32
and f33 modules must be built from the same modulemd file.
You wrap a %files section of the package you want to remove with a condition
based on the Fedora version:
%if %{fedora} < 33
%files
...
%end
This will cause the when building the module component, rpmbuild won't produce
the unwanted binary package on F33, but it will produce it on F32.
The only problem is if that is the only binary package produced by the spec
file. Then rpmbuild would report an error, because it would consider it
a packaging mistake. But it can be worked around by produced a dummy package
instead:
%if %{fedora} < 33
%files
...
%else
%files -n dummy
%end
and filtering out the dummy package from the module on modulemd level with
/data/filetr/rpms/dummy entry.
And now I'm curious what would happen if I specify `platform:
[-f33]`
and publish new module build. Would that remove previously published
metadata from rawhide?
No. Rawhide repository is composed from the latest builds tagged into
a rawhide tag. That means Rawhide would still contain the last module built
from f33 platform.
I guess the right answer is somewhere around servicelevels and `eol`
specification.
Theoretically yes. You can file a releng request
<https://pagure.io/releng/new_issue> of moduel_eol type to shorten the EOL.
But I worry that a compose process does not respect the EOL dates and instead
the EOLed modules are removed from a compose configuration by relengs
manually once before branching a new Fedora release (33 is the next one).
-- Petr
11:42 a.m.
Petr and Daniel, thanks a lot for your replies.
I hope some of mentioned limitations could be reflected in the
modularity documentation. I read through most of it (including the
drafts for f33 change) and I don't feel that the consequences of adding
package to the module are clearly communicated.
And maybe something could be added to the modulemd spec to reflect that
the specific package is no longer required starting from release x. I
don't expect that to work from updates-modular for branched releases,
but having the ability to drop packages from module on a development
release would be awesome.
On 7/20/20 4:46 AM, Petr Pisar wrote:
On Fri, Jul 17, 2020 at 02:14:49PM -0700, Aleksei Bavshin wrote:
> The real question is how to do the change in f33 considering that f32
> and f33 modules must be built from the same modulemd file.
You wrap a %files section of the package you want to remove with a condition
based on the Fedora version:
%if %{fedora} < 33
%files
...
%end
This will cause the when building the module component, rpmbuild won't produce
the unwanted binary package on F33, but it will produce it on F32.
The only problem is if that is the only binary package produced by the spec
file. Then rpmbuild would report an error, because it would consider it
a packaging mistake. But it can be worked around by produced a dummy package
instead:
%if %{fedora} < 33
%files
...
%else
%files -n dummy
%end
and filtering out the dummy package from the module on modulemd level with
/data/filetr/rpms/dummy entry.>
> And now I'm curious what would happen if I specify `platform: [-f33]`
> and publish new module build. Would that remove previously published
> metadata from rawhide?
No. Rawhide repository is composed from the latest builds tagged into
a rawhide tag. That means Rawhide would still contain the last module built
from f33 platform.
> I guess the right answer is somewhere around servicelevels and `eol`
> specification.
>
Theoretically yes. You can file a releng request
<https://pagure.io/releng/new_issue> of moduel_eol type to shorten the EOL.
But I worry that a compose process does not respect the EOL dates and instead
the EOLed modules are removed from a compose configuration by relengs
manually once before branching a new Fedora release (33 is the next one).
-- Petr
--
With best regards,
Aleksei Bavshin
2:20 a.m.
Dne 17. 07. 20 v 23:14 Aleksei Bavshin napsal(a):
Yes, that's something I already accepted.
The real question is how to do the change in f33 considering that f32
and f33 modules must be built from the same modulemd file. I only see
the ability to disable module stream build for f33.
Hmm, it seems this is caused by
the dist-git implementation of
modularity source control management.
There's no name or stream in the yaml file. If I understand it
correctly, the values are obtained from following places:
name == name of the dist-git repo
stream == name of the dist-git branch
I'll make a note for myself to make the fields mandatory so you can use
any any branch name. That would allow you maintaining 2 versions of the
source yaml file and build the modules differently for different platforms.
And now I'm curious what would happen if I specify `platform: [-f33]`
and publish new module build. Would that remove previously published
metadata from rawhide?
I guess the right answer is somewhere around servicelevels and `eol`
specification.
It probably depends on the implementation.
Not sure if Fedora infrastructure releases module for the latest module
version (which would be none in this case) or the latest available
(which would be the previous module build).
1383
days inactive
1387
days old
8 comments
4 participants
participants (4)
-
Aleksei Bavshin -
Daniel Mach -
Miro Hrončok -
Petr Pisar