On Sat, Jul 23, 2022 at 7:47 AM Fabio Valentini <decathorpe(a)gmail.com> wrote:
On Sat, Jul 23, 2022 at 12:56 PM Neal Gompa <ngompa13(a)gmail.com> wrote:
>
> On Sat, Jul 23, 2022 at 4:14 AM Dan Čermák
> <dan.cermak(a)cgc-instruments.com> wrote:
> >
> > Richard Shaw <hobbes1069(a)gmail.com> writes:
> >
> > > Replying in general...
> > >
> > > I've asked about a "one script to rule them all" a few times
over my 10+
> > > year Fedora packaging career and it's fallen on deaf ears.
> > >
> > > I hope something will happen this time. There should really be only ONE
way
> > > to determine what packages need to be rebuilt, even if it's not
perfect, we
> > > can deal with the corner cases but everyone doing their own thing has
> > > definitely been worse.
> >
> > In a perfect world koji or koschei would figure this out themselves and
> > perform the rebuilds for us so that we can finally stop thinking about
> > build orders and dependencies ourselves.
>
> The sad part is that Koschei can do it, but the build system folks
> have so far refused to enhance Koji and Koschei to do this for
> creating *real builds* that are auto-submitted.
Alright Neal, this is a bit off-topic, but I'll bite.
Auto-submitting real builds is something that I will, except in very
narrowly defined exceptions, always disagree with.
Until now, automagic builds have only caused trouble. Just look at the
mess that's regularly made in the podman/container stack, or by stuff
that was automatically submitted by packit. It's one of the reasons
why we now have a policy that requires actual people to be responsible
for all builds submitted by bots.
Even if we had a mechanism to trigger automatic rebuilds of dependent
packages (i.e. "I have detected that the sonames of the libraries in
this package have changed, let me also rebuild dependent packages for
this!") only works *if* (and that's a big *if*) the ABI change isn't
accompanied by breaking API changes, as well. What would you want to
happen then? I'm pretty sure software isn't smart enough yet to
determine in advance if any breaking API changes affect any dependent
packages.
So how would the mechanism you want work?
1. packager submits a build of libfoo version X that contains an
soname bump to koji
2. magic mechanism (TM) detects this ~somehow~ and does ... what? the
only thing I can think of is ...
3. magic mechanism (TM) tags the build into an on-demand side-tag N
instead of rawhide after it finishes
4. magic mechanism (TM) queries dependent packages
5. magic mechanism (TM) bumps dependent packages with "Rebuilt for
libfoo soname bump."
This step wouldn't happen. Bumping packages automatically is dangerous
and creates problems.
6. magic mechanism (TM) submits builds of dependent packages to
on-demand side-tag N
7. magic mechanism (TM) requests bodhi to merge the side tag *if and
only if* all builds succeed and are installable
It would also have to pass relevant gating tests. Right now gating
doesn't do much, but that is already changing over time. And ideally
we could run OpenQA tests on all of these in the future.
That's all well and good so long as everything *just works*, but
will
instantly break the second any part of this will need human
intervention, or if there are any race conditions:
- dependent packages need patches for breaking API changes
- dependent packages need a compat package libfooXminus1 because they
can't be ported
Build failure would cause it to not get merged. But the side tag would
exist for human action to resolve the chain.
- the sets of dependent packages overlap between concurrently created
side tags
So what? If we're not tracking rebuilds in Git anymore, this is no
longer a serious problem. The build system knows every time a commit
is requested and increments the counter accordingly.
- dependent packages fail to build for unrelated reasons
- dependent packages fail to install for related (or unrelated) reasons
- packagers have pushed changes to dist-git that they didn't want to
get built yet (eugh)
Okay, and? They fail, it doesn't merge, and we deal with it as we do
now. Or if it looks like a fluke, poke it to try to rebuild the failed
ones again.
I just see so many points of failure in a process like the one you
want that I just can't ever see this becoming a reality unless we
enforce much stricter rules around 1) concurrent side tags / using a
mutex mechanism for packages in koji, 2) forbidding to push changes to
dist-git that must not be built yet, 3) drastically reducing the
number of FTBFS / FTI bugs in rawhide ~somehow~.
Unless you have ways to address at least some of those issues and
failure points with the mechanism that you want (and I cannot see how
that would be doable), I don't see automagic rebuilds happening.
The fundamental difference between your thought and mine is that you
want it to be perfect. I want it to be good enough to eliminate the
*majority* of provenpackager grunt work and stop punishing people so
hard for bringing useful software library packages into the
distribution.
Failure is okay. Human intervention is fine. Design a process that
handles 80% and make it gracefully fail for the remaining 20%.
You seem to think it's unworkable, but I work in a Linux distribution
that operates this way and has for twenty years! I *know* it works.
It's not perfect, and there's certainly going to be some human
intervention and probably some configuration stuff to resolve things
machines can't figure out on their own (like build cycles and things
of that nature), but it is absolutely possible to do this and make the
lives of the majority of packagers easier.
--
真実はいつも一つ!/ Always, there's only one truth!