On Mon, 22 Jun 2020 at 08:14, Zbigniew Jędrzejewski-Szmek
On Mon, Jun 22, 2020 at 04:55:10AM +0200, clime wrote:
> >> > > Hello Josh,
> >> > >
> >> > > you can change the artifact type while keeping interface the same
> >> > > it would be a _HUGE_ win because it would make modularity
> >> > > understandable for mere humans and better maintainable.
> >> > >
> >> > > Namely, modules should become rpms and therefore obey standard
> >> >
> >> > I'm not sure I entirely understand what you mean, but it sounds
> >> > you have some interesting ideas.
> >> >
> >> > I'm looking forward to seeing what you and the community can
> >> > from them, and how they could be brought into RHEL 10+! That kind of
> >> > collaboration is what makes Fedora great.
> >> I know this probably won't change anything because this was mentioned
> >> many times (by me at least) and nothing has changed but still...
> >> Currently, modules are essentially yum sub-repos, they are not really
> >> "modules", instead they are collections of rpms that reinvent
> >> relations (obsoletes, requires, build-requires, etc.).
> >> There is no reason for this wheel-reintervention. Modules (the
> >> collections) can be simply squashed into an rpm by automation and this
> >> resulting rpm can go to the modular repo together with other modules.
I agree with this general idea, even if not with the exact implementation
(comments below). In the past this was stated as "divorcing the build ordering
mechanism from the rpm delivery mechanism". The fact that we have two layers
of dependencies make Modularity conceptually hard and destroy the interaction
with the dependency solver. Also, if we disconnect the build and delivery
mechanisms, we can iterate and improve both separately.
> >> That way we don't have two types of objects we complex inter-relations
> >> but only one we well-known behavior.
> >> I wonder if this is clear to everyone but nobody really cares or
> >> doesn't really want to say it or I don't know.
> >> Is this clear to everyone? I mean either I am stating an obvious stuff
> >> that nobody really considers worth typing or idk.
> > How would this work when there are optional rpms in the module?
> > You do not need to install every rpm in eg the php module (different
graphics/database backends) for that module to be useful, but every version of the module
will have the rpm as an option which wont work outside a module of multiple rpms.
> Glad you ask, I wasn't precise...
> Well, I didn't mean everything always needs to be squashed, instead,
> it would be an optional step in modulemd processing.
So... if it's only optional, that means that the general case where
squashing is not done needs to be solved anyway. And once you have
solved the general case, what would the point of squashing be?
Thus, I don't find squashing useful.
> For some
> use-cases (like delicately compiled postgresql server), you can create
> a single rpm that contains all - postgresql-server, postgresql,
> postgresql-libs compiled in a specific way, optionally with some
> postgresql modules pre-included, so it would be let's say time-series
> optimized postgresql. Here it makes sense to make a single rpm from it
> - you install that and you are all set up for your use-case.
> Then there are language stacks where you might want to build things in
> a specific order - there nothing really needs to be squashed (or
> certain subset can if it makes sense) but you can still use modularity
> to easily batch-build certain rpms. If there are runtime optional
> deps, they can be described by Recommends/Suggests.
> Basically, once a "module" (things that comes from modulemd) is built,
> it should be put into normal repos and the "module" boundary should be
> forgotten (unless it is a single rpm), i.e. "module" is a built-time
> thing, at install-time we just have standard packages with standard
The unanswered question is what mechanism would be used make sure that
the rpms from the "module" are all installed. One option would be to
somehow mangle rpm names, another option would be to add some kind of
Provides/Requires, etc. But *some* mechanism is needed, because without
that dnf would often pick other rpms.
In Modularity the solution is that the rpms from the module shadow
rpms with the same name from outside. That's probably the single
feature of Modularity that causes the most problems.
Yeah, I could notice modularity causes quite a lot of troubles.
The question is whether those troubles can be somehow technically
justified, i.e. by the benefit modularity will eventually bring.
But I personally don't see it. Actually, (big revelation) I have never
understood modularity and I was even there in RH when it was just
starting to be born. Probably, one of the reasons why I didn't
understand it at that point was that there seemed to be no clear
specification floating around. We knew modularity should solve "too
fast, too slow" problem of distributions but it wasn't exactly clear
how. It was a cool buzz word but nobody seemed to know what it means
(at least that was my view of the situation).
I and perhaps others were thinking that it wants to provide parallel
availability+installability of the distribution software but after
some time, it was cleared out this isn't the case and that the goal is
just "parallel availability".
OK, but even this isn't clear to me.
I mean I understand the usefulness of modularity for build-time where
you can create a recipe (modulemd) that will build your packages with
interdependencies in a predictable and automatic manner. I think
that's a cool thing to have.
But what about run-time (or install-time in other words)? That's the
part I don't understand. And I have spent quite some time trying to
understand it but never managed. So it's possible that I am missing
something all these years...in that case, it would be great if
somebody could shed a light on it for me. Here, I would really like to
give modularity a chance.
From what I understand, the use of modularity in run-time is to
provide rpm namespaces. Natural way to do this would be to use
separate repositories ala COPR where rpms are namespaced by repo ID
but I know one of the requirements of modularity was to use a single
repo for those namespaces with an argument that dnf is slow when
working with a large number of repos...to me that reason always seemed
quite artificial...something is slow...ok, then it can be made faster.
I could understand if we were talking about let's say thousands of
modules - there I would believe that initiating a thousand (or
multiplied by few) new downloads of repo files might already have its
price. But okay, if thousands of modules were the plan, then I could
understand this argument.
But now comes even more curious part. So...run-time modularity
provides rpm namespacing if I understand it correctly. Basically
<module>/<stream>/<package_name>. The easy solution for this would be
to put the namespace implicitly into package name like python does it
when there should be multiple pythons available, e.g. currently in
CentOS7/EPEL7, there is python34-requests and python36-requests (I
understand there will be a dot between major and minor at some point
so e.g. python3.6-requests but that's another thing :)). So if we have
different rpm names (because the namespace is already included in the
rpm name itself), then there is no problem to provide multiple
variants of the "same package" (the same thing but intended e.g. for a
different python interpreter) in the same repo.
So I would be willing to accept that this is a hacky solution or just
a workaround (even though I am not sure it is). But even if I accept
that it is just a hack and we need a more proper solution, I still
have an issue in my mind. Let's say we have this two-level namespacing
(<module>/<stream>/) and it enables us to have a package of the same
name twice or more times in the same repo and it enables us to avoid
mangling the rpm names. Great, isn't it? Well...but what if those
different variants of the same package are actually
parallel-installable and a user would benefit from having them
parallel-installable (because it's a dev working with different
versions of the same language at the same time)? We can only install a
package of a certain name once into the system so that's why
modularity enables us to use always just a single stream from all
available streams of a module, i.e. you can only switch between the
individual streams, having multiple of them enabled at the same time
is not possible.
So basically, modularity gives parallel-availability but at the same
time, it disables the option of parallel-installability which could be
achieved through alternatives and some smart packaging for probably
all the language stacks if I understand correctly. I think that's a
too much of a limitation. To avoid it, we would need to keep an rpm DB
per the namespace (<module>/<stream>/) and these various DBs would be
handled by dnf, which would basically mean, rpm command itself
wouldn't know what's all installed on the system - hard to imagine
that people would be alright with it. ...Or we can bring the notion of
the namespaces into rpm itself (that's where my suggestion of "Stream"
rpm attribute comes from but it could also be called just
"Namespace"). But then there is the argument: "Why not just put the
namespace into rpm name itself?" I mean...I wouldn't mind having it as
a separate attribute but the usefulness of it would need to be
So I don't really get even after almost five years where modularity is
going or what it wants to achieve. I don't understand its use-case for
any of Fedora, RHEL, and CentOS because disabling
parallel-installability to allow parallel availability is imho not
really an option. But yeah...maybe I am missing some angle. In that
case, please, explain it to me because I would really like to
> > dnf interface could be kept given that we "Stream" rpm property is
> > added. This is still a bit rough what I am saying but hopefully it
> > makes at least a bit of sense...
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct:
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: