10:53 p.m.
I'm still very confused about how to do modular packaging in Fedora. I
don't know:
1. How do I create a module for a new Fedora package?
2. How do I create a module for my existing non-modular Fedora package?
3. How do I declare BuildRequires on my build dependencies from another module?
4. How do I declare Requires on my runtime dependencies from another module?
5. How do I know what modules are available?
6. How do I figure out which packages are in a particular module?
7. How do I find out what module a particular package is in?
Learning modularity, now that the primary maintainer for most of the
Java build ecosystem in Fedora is orphaning their non-modular
packages, has become very important for several of my packages, but
I'm still struggling to find where these basic answers are documented.
I'm a loyal Fedora user, but a novice packager, and still occasionally
struggle with basic non-modular packaging tasks. Understanding modular
packaging seems to be even harder to find clear, step-by-step
instructions to follow, than regular non-modular packaging
instructions, which I'm still not an expert with.
The answers to these questions should be clear, step-by-step
command-line instructions, or specific examples of a SPEC file
modification. Where can I find that kind of documentation? I know I've
attempted to ask similar questions in the past, but I just haven't
seen good documentation in response. Is there anybody who really,
truly, understands how to do modular packaging well enough to write a
clear step-by-step instructions for the rest of us to follow?
Thanks,
Christopher
1:02 a.m.
On Wed, Oct 03, 2018 at 11:53:25PM -0400, Christopher wrote:
I'm still very confused about how to do modular packaging in
Fedora. I
don't know:
1. How do I create a module for a new Fedora package?
2. How do I create a module for my existing non-modular Fedora package?
https://docs.fedoraproject.org/en-US/modularity/making-modules/adding-new...
3. How do I declare BuildRequires on my build dependencies from
another module?
4. How do I declare Requires on my runtime dependencies from another module?
No changes on the RPM level. You also have dependencies on
the module level, which makes their packages available to you.
At buildtime it means their packages are available for you to
install in the buildroot, at runtime that they are available as
installable deps. Besides ensuring you depend on the modules
that provide your packages in the appropriate situations,
you don't need to do anything.
5. How do I know what modules are available?
This one is tricky.
If you only care about runtime and what's available to Fedora
end users directly, you can just run "dnf module list" or inspect
the repodata (*modules.yaml.gz) of the release you're targeting.
Not all modules are included in the repos. Some are used
as building blocks and discovering these is more difficult.
You can query the MBS database to inspect everything there is:
https://mbs.fedoraproject.org/module-build-service/1/module-builds/
The REST API is documented here:
https://pagure.io/fm-orchestrator
6. How do I figure out which packages are in a particular module?
7. How do I find out what module a particular package is in?
These also need more work and depdends on what you're looking for.
"dnf module info" and "dnf module provides" or looking at the
repodata of your targetted release can help you identify what
binary packages are provided by particular modules. This doesn't
include building block modules as those aren't in the repo.
MBS database (see above) has links to all modular component
builds in koji. You could use that, too. This includes
everything, sources and binaries, available in the repos or not.
For sources and build instructions, the database is enough but
you could also just pull the modulemd files from dist-git.
Learning modularity, now that the primary maintainer for most of the
Java build ecosystem in Fedora is orphaning their non-modular
packages, has become very important for several of my packages, but
I'm still struggling to find where these basic answers are documented.
I'm a loyal Fedora user, but a novice packager, and still occasionally
struggle with basic non-modular packaging tasks. Understanding modular
packaging seems to be even harder to find clear, step-by-step
instructions to follow, than regular non-modular packaging
instructions, which I'm still not an expert with.
Thanks for asking these questions. We're aware of some gaps
but sometimes you just don't know what people don't know.
Modules also have the concept of defaults, making their
packages available at runtime, even as dependencies for
non-modular packages. There are also mechanisms to make select
modules available in the buildroot for everyone but this isn't
currently possible in Fedora. We would like to change that
at some point, though. With these, you shouldn't be forced to
switch to modularity if you don't want to and the maintainers
of your dependencies handle their transition properly.
The answers to these questions should be clear, step-by-step
command-line instructions, or specific examples of a SPEC file
modification. Where can I find that kind of documentation? I know I've
attempted to ask similar questions in the past, but I just haven't
seen good documentation in response. Is there anybody who really,
truly, understands how to do modular packaging well enough to write a
clear step-by-step instructions for the rest of us to follow?
So, we have this:
https://docs.fedoraproject.org/en-US/modularity/
Suggestions for improvements or even PRs are definitely
appreciated.
P
12:46 p.m.
> 4. How do I declare Requires on my runtime dependencies from
another module?
No changes on the RPM level. You also have dependencies on the module level,
which makes their packages available to you. At buildtime it means their packages
are available for you to install in the buildroot, at runtime that they are available as
installable deps. Besides ensuring you depend on the modules that provide your
packages in the appropriate situations, you don't need to do anything.
Isn't this what comps are designed for?
# dnf group install $name
3:23 p.m.
On Thu, Oct 04, 2018 at 05:46:54PM -0000, Raphael Groner wrote:
>> 4. How do I declare Requires on my runtime dependencies from
another module?
> No changes on the RPM level. You also have dependencies on the module level,
> which makes their packages available to you. At buildtime it means their packages
> are available for you to install in the buildroot, at runtime that they are
available as
> installable deps. Besides ensuring you depend on the modules that provide your
> packages in the appropriate situations, you don't need to do anything.
Isn't this what comps are designed for?
# dnf group install $name
Unless I'm missing something, comps don't really pin you to
sets of NVRs for the same name.
If I have, for example, two conflicting stacks available, say
perl:5.26 and perl:5.28, with modules I can choose which of the
two I have available in the buildroot and can later select the
package set at runtime. With comps I'd need to hardcode this in
package names and reflect that in every Perl package in Fedora.
At runtime I also get updates for perl:5.26 with the usual
"dnf upgrade", without being upgraded to 5.28, unless I choose
to switch.
Maybe you had something else in mind. In that case, please
elaborate.
P
10:08 a.m.
Il 04/10/18 05:53, Christopher ha scritto:
The answers to these questions should be clear, step-by-step
command-line instructions, or specific examples of a SPEC file
modification. Where can I find that kind of documentation?
There's a lot of information about that at
https://docs.fedoraproject.org/en-US/modularity/
However, I'm still confused too... what's the difference between
Requires and BuildRequires in .specfile and build and runtime
dependencies in module definition file? Aren't Requires and
BuildRequires enough?
For example, I would like to build a module for Skychart; Skychart needs
FreePascal and Lazarus as buildrequires; should I also add them to
module definition as build dependencies? If a new FreePascal or Lazarus
version is built, it will trigger a new Skychart module build, or I
still need to do it manually?
Thanks
Mattia
10:22 a.m.
On Thu, Oct 4, 2018 at 10:09 AM Mattia Verga
<mattia.verga(a)protonmail.com> wrote:
Il 04/10/18 05:53, Christopher ha scritto:
>
> The answers to these questions should be clear, step-by-step
> command-line instructions, or specific examples of a SPEC file
> modification. Where can I find that kind of documentation?
There's a lot of information about that at
https://docs.fedoraproject.org/en-US/modularity/
However, I'm still confused too... what's the difference between
Requires and BuildRequires in .specfile and build and runtime
dependencies in module definition file? Aren't Requires and
BuildRequires enough?
In most cases, yes. The only time you need to include them in the
module definition file is if you explicitly need a *different* version
of your dependencies than what would show up in the standard repos.
(E.g. Fedora 29 will ship Node.js 10.x by default, but the application
you are packaging requires Node.js 8.x or older, so you would add a
buildrequires to the module to tell it your module stream also relies
on a non-default stream of Node.js.
For example, I would like to build a module for Skychart; Skychart
needs
FreePascal and Lazarus as buildrequires; should I also add them to
module definition as build dependencies? If a new FreePascal or Lazarus
version is built, it will trigger a new Skychart module build, or I
still need to do it manually?
If FreePascal and Lazarus of the correct version are available in the
standard repository (i.e. you don't have to specify a non-default
stream of a module to get the correct version), you do not need to
specify them explicitly in the module definition.
10:54 a.m.
Il 04/10/18 17:22, Stephen Gallagher ha scritto:
If FreePascal and Lazarus of the correct version are available in the
standard repository (i.e. you don't have to specify a non-default
stream of a module to get the correct version), you do not need to
specify them explicitly in the module definition.
Thanks!
Just another question: what about if I need to use different .specfiles
for different Fedora releases?
For example, if FreePascal is upgraded in F29 and not in F28 I may need
to patch Skychart sources differently for F29 and F28...
10:57 a.m.
On Thu, Oct 4, 2018 at 10:54 AM Mattia Verga
<mattia.verga(a)protonmail.com> wrote:
Il 04/10/18 17:22, Stephen Gallagher ha scritto:
>
> If FreePascal and Lazarus of the correct version are available in the
> standard repository (i.e. you don't have to specify a non-default
> stream of a module to get the correct version), you do not need to
> specify them explicitly in the module definition.
Thanks!
Just another question: what about if I need to use different .specfiles
for different Fedora releases?
For example, if FreePascal is upgraded in F29 and not in F28 I may need
to patch Skychart sources differently for F29 and F28...
It depends on the case. It *might* be an example of when you might
decide that it's better to standardize on one version of FreePascal as
a dependency and make it a module. So if you wanted to standardize on
the upgraded version, you would create a FreePascal module stream for
F28 of the newer version and have your Skychart module require the
newer one there. Then you only have to maintain support for one of
them.
12:44 p.m.
Hi Christopher,
other contributors are wondering, too. I tried recently to ask your question and got an
anwser that "it depends" with the dependencies where they're given, although
I fail to see a significant difference.
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.o...
Regards, Raphael
7:07 p.m.
On 10/3/18 9:53 PM, Christopher wrote:
I'm still very confused about how to do modular packaging in
Fedora. I
don't know:
1. How do I create a module for a new Fedora package?
2. How do I create a module for my existing non-modular Fedora package?
3. How do I declare BuildRequires on my build dependencies from another module?
4. How do I declare Requires on my runtime dependencies from another module?
5. How do I know what modules are available?
6. How do I figure out which packages are in a particular module?
7. How do I find out what module a particular package is in?
The documentation noted elsewhere in this thread is pretty handy; I've read
through it a couple of times now and really appreciate the work that has gone
into it -- thank you. I do have some suggestions:
-- I'm going to be a little cranky and say the use of "ursine" is very
confusing to me; it's a cute pun, but we have actual bears -- who are
intensely ursine, as it were -- that live nearby so I constantly have
to remind myself of context and translate since proper use of that word
has been familiar to me for many years. This could just be me being an
old codger, however, and I freely admit that :).
-- Could we put the answers to the questions above into the FAQ?
-- And the one question I have to add on to Christopher's wonderful list:
I have a package where upstream releases about once a month, and each
new release must by definition be backwards compatible (acpica-tools,
specifically). I can think of no scenario where a module provides value
to me or end-users; in fact, using anything other than the most recent
causes problems. Do I have to create and maintain a module for this
package anyway? Or are the defaults robust enough that a package can
remain a package without touching modularity at all? The answer to this
is completely unclear to me -- what I've read seems to imply that I must
create a module definition regardless.
Thanks again for all the documentation work so far!
--
ciao,
al
-----------------------------------
Al Stone
Software Engineer
Red Hat, Inc.
ahs3(a)redhat.com
-----------------------------------
5:07 a.m.
On Tue, Oct 09, 2018 at 06:07:44PM -0600, Al Stone wrote:
On 10/3/18 9:53 PM, Christopher wrote:
> I'm still very confused about how to do modular packaging in Fedora. I
> don't know:
>
> 1. How do I create a module for a new Fedora package?
> 2. How do I create a module for my existing non-modular Fedora package?
> 3. How do I declare BuildRequires on my build dependencies from another module?
> 4. How do I declare Requires on my runtime dependencies from another module?
> 5. How do I know what modules are available?
> 6. How do I figure out which packages are in a particular module?
> 7. How do I find out what module a particular package is in?
The documentation noted elsewhere in this thread is pretty handy; I've read
through it a couple of times now and really appreciate the work that has gone
into it -- thank you. I do have some suggestions:
-- I'm going to be a little cranky and say the use of "ursine" is very
confusing to me; it's a cute pun, but we have actual bears -- who are
intensely ursine, as it were -- that live nearby so I constantly have
to remind myself of context and translate since proper use of that word
has been familiar to me for many years. This could just be me being an
old codger, however, and I freely admit that :).
sadbearissad.jpeg
You should know we also have this service named Ursa Major...
-- Could we put the answers to the questions above into the FAQ?
Good idea.
-- And the one question I have to add on to Christopher's
wonderful list:
I have a package where upstream releases about once a month, and each
new release must by definition be backwards compatible (acpica-tools,
specifically). I can think of no scenario where a module provides value
to me or end-users; in fact, using anything other than the most recent
causes problems. Do I have to create and maintain a module for this
package anyway? Or are the defaults robust enough that a package can
remain a package without touching modularity at all? The answer to this
is completely unclear to me -- what I've read seems to imply that I must
create a module definition regardless.
First of all, you don't have to create and maintain a module.
You can continue packaging this like you're used to.
But if you decide to embrace modularity, there's nothing wrong
with providing a single stream with just one package. You could
call it the "latest", or, as some people prefer, "stable" (uh).
That way you could benefit from some of the build automation
and maintain just one SPEC file for all releases, at least.
I see there are packages that depend on yours. At runtime
this isn't a real problem as long as your module is enabled by
default; however, moving your package to modules only would
make it unavailable to non-modular content in the buildroot
(until the above-mentioned Ursa Major is a thing in Fedora).
Module defaults only select default module streams and default
installation profiles (similar to package groups) for each.
If you don't package your software as a module, you don't have
to care about this at all.
P
2:15 p.m.
On Tue, Oct 09, 2018 at 06:07:44PM -0600, Al Stone wrote:
-- And the one question I have to add on to Christopher's
wonderful
list: I have a package where upstream releases about once a month,
and each new release must by definition be backwards compatible
(acpica-tools, specifically). I can think of no scenario where a
module provides value to me or end-users; in fact, using anything
other than the most recent causes problems. Do I have to create and
maintain a module for this package anyway? Or are the defaults
robust enough that a package can remain a package without touching
modularity at all? The answer to this is completely unclear to me --
what I've read seems to imply that I must create a module definition
regardless.
This actually seems like the ideal case for a single stream -- instead of
maintaining rawhide, f29, f28, epel7, you'd just maintain "latest",
and that would get build into all of these releases simultaneously.
--
Matthew Miller
<mattdm(a)fedoraproject.org>
Fedora Project Leader
2:17 p.m.
On Wed, Oct 10, 2018 at 3:15 PM Matthew Miller <mattdm(a)fedoraproject.org> wrote:
On Tue, Oct 09, 2018 at 06:07:44PM -0600, Al Stone wrote:
> -- And the one question I have to add on to Christopher's wonderful
> list: I have a package where upstream releases about once a month,
> and each new release must by definition be backwards compatible
> (acpica-tools, specifically). I can think of no scenario where a
> module provides value to me or end-users; in fact, using anything
> other than the most recent causes problems. Do I have to create and
> maintain a module for this package anyway? Or are the defaults
> robust enough that a package can remain a package without touching
> modularity at all? The answer to this is completely unclear to me --
> what I've read seems to imply that I must create a module definition
> regardless.
This actually seems like the ideal case for a single stream -- instead of
maintaining rawhide, f29, f28, epel7, you'd just maintain "latest",
and that would get build into all of these releases simultaneously.
I still don't get why this subset case requires all the extra module
goop? Couldn't we just have fedpkg have an "fedpkg build
--all-releases" switch to just trigger on the same commit for all
releases?
--
真実はいつも一つ！/ Always, there's only one truth!
2:45 p.m.
On Wed, Oct 10, 2018 at 03:17:13PM -0400, Neal Gompa wrote:
I still don't get why this subset case requires all the extra
module
goop? Couldn't we just have fedpkg have an "fedpkg build
--all-releases" switch to just trigger on the same commit for all
releases?
It does seem like something like that should be possible.
Once the Freshmaker thing is deployed, though, there's a nice twist with
MBS: the system can automatically do the builds on commit, completely
removing that step. I'd kind of like it if the module.md could live in the
rpm namespace in that case, though.
--
Matthew Miller
<mattdm(a)fedoraproject.org>
Fedora Project Leader
2:54 p.m.
On Wed, Oct 10, 2018 at 3:46 PM Matthew Miller <mattdm(a)fedoraproject.org> wrote:
On Wed, Oct 10, 2018 at 03:17:13PM -0400, Neal Gompa wrote:
> I still don't get why this subset case requires all the extra module
> goop? Couldn't we just have fedpkg have an "fedpkg build
> --all-releases" switch to just trigger on the same commit for all
> releases?
It does seem like something like that should be possible.
This is possible already, due to a fedpkg update a couple months ago:
https://docs.pagure.org/fedpkg/releases/1.35.html
3:16 p.m.
On Wed, Oct 10, 2018 at 03:54:18PM -0400, Stephen Gallagher wrote:
> It does seem like something like that should be possible.
This is possible already, due to a fedpkg update a couple months ago:
https://docs.pagure.org/fedpkg/releases/1.35.html
Do we have policy around that?
One reason one might want a module anyway is for the (future) EPEL case.
Right now, EPEL has two problems:
1. As a maintainer, I don't necessarily want to commit to a 10+ package
lifetime, and probably do not want to commit to RHEL-like backports.
2. Some people do though. So, as a user, it's unclear with the update
policies and lifecycle for any given package.
Theeeeoretically, modules should solve this, because I can declare "I'm
building this for EPEL, but the lifecycle will be the same as in Fedora".
(Or, for packages which have a 'slow' stream: "I'm planning on
maintaining
this version for three years in EPEL, and, hey, might as well also make that
available on Fedora releases too.)
--
Matthew Miller
<mattdm(a)fedoraproject.org>
Fedora Project Leader
2:28 p.m.
On Wed, Oct 10, 2018 at 3:15 PM Matthew Miller <mattdm(a)fedoraproject.org> wrote:
On Tue, Oct 09, 2018 at 06:07:44PM -0600, Al Stone wrote:
> -- And the one question I have to add on to Christopher's wonderful
> list: I have a package where upstream releases about once a month,
> and each new release must by definition be backwards compatible
> (acpica-tools, specifically). I can think of no scenario where a
> module provides value to me or end-users; in fact, using anything
> other than the most recent causes problems. Do I have to create and
> maintain a module for this package anyway? Or are the defaults
> robust enough that a package can remain a package without touching
> modularity at all? The answer to this is completely unclear to me --
> what I've read seems to imply that I must create a module definition
> regardless.
This actually seems like the ideal case for a single stream -- instead of
maintaining rawhide, f29, f28, epel7, you'd just maintain "latest",
and that would get build into all of these releases simultaneously.
What is the overhead of maintaining a module for a single package,
plus the package itself, vs just maintaining the package the current
way?
My understanding-- from skimming the documentation a few times and
reading discussions about modularity-- is that I'd now need to keep
track of two dist-git repositories, and two different metadata files.
This feels like a lot of extra overhead. It also requires learning
about a new thing-- modulemd files.
Is this really less work? I admit I haven't tried to do it myself yet,
so I don't know. But part of the reason I haven't tried it is because
I'm not sure if it will actually be better...
I guess it would be nice to read a sort of "modularity for the
skeptical contributor" document or article that answers questions like
this.
Ben Rosser
2:47 p.m.
On Wed, Oct 10, 2018 at 03:28:02PM -0400, Ben Rosser wrote:
My understanding-- from skimming the documentation a few times and
reading discussions about modularity-- is that I'd now need to keep
track of two dist-git repositories, and two different metadata files.
This feels like a lot of extra overhead. It also requires learning
about a new thing-- modulemd files.
Usually, the modulemd doesn't need to be updated very often, so it's a
little bit of setup overhead but then not something to worry about. I agree
that this could be made easier.
(Well, except right now when you have to actually make a commit in the
module repo to make a build. But that shouldn't be necessary.)
--
Matthew Miller
<mattdm(a)fedoraproject.org>
Fedora Project Leader
2025
days inactive
2031
days old
18 comments
10 participants
participants (10)
-
Al Stone -
Ben Rosser -
Christopher -
Matthew Miller -
Mattia Verga -
Miro Hrončok -
Neal Gompa -
Petr Šabata -
Raphael Groner -
Stephen Gallagher