Hi Nicolas,
I'm embarrassed to admit that before I sent my mail I carefully read over
... the old PackageDrafts/Go :-( My only excuse is that it was in my
browser history.
Having actually read the relevant parts of "More Go Packaging", the
explanation of compat packages and notification procedures does make the
intended operation clearer, though the social and technical barrier to a
packager new to Fedora will still be high if packaging their target package
requires creating a compat package and fixing multiple other packages. As
you say, once Fedora packaging catches up to the modern state, things will
be better, because the onus won't be on the new packager to do the work of
updating old libraries.
I still worry that Fedora is not big enough to move the status-quo in the
Go world - to get the point where Go programs require
github.com/foo/bar >=
1.2.3 and actually have been tested with a multiple versions in that range,
not exactly the one vendored version shipped with the program. Sp we might
end up with lots of Fedora-specific bugs that the upstream doesn't care
about. But if the Fedora Go community can get some size and momentum and
start pushing fixes upstream, that will certainly be a positive force for
reducing Go programs stuck on random old versions of dependencies!
I haven't had time to read through the entire proposal, but it certainly
looks like a major step forward!
Owen
On Thu, Feb 1, 2018 at 5:21 AM, <nicolas.mailhot(a)laposte.net> wrote:
----- Mail original -----
De: "Owen Taylor"
Hi Owen,
> Is there a guide for Fedora packagers about how to handle unbundling for
> golang packages? The draft guidelines don't seem to go into any details.
I don't think there is, nor that it is necessarily needed. The posted
guidelines should be sufficient technically, there are no magic I know of I
didn't document, the rest is just a lot of work (but see ↓)
> I've looked at packaging a few golang packages unbundled, and have
> immediately run into:
>
> A) lots of unpackaged dependencies
> B) dependencies that are packaged in Fedora with different, often much
> older versions
Yes the state of Go packages in Fedora is pretty sad right now. I wouldn't
expect anyone to be able to package anything but the most trivial app in
unbundled mode. Too many common parts are missing, when they are not
missing they are too old, trying to update an existing Go package is an
exercise in frustration (too much package-specific shell code, that is
difficult to understand and does not really work with new code versions),
and trying to update or add missing parts just reveals more breakage and
work to be done.
However, accepting to package complex Go apps in bundled mode is how
Fedora got to this state in the first place. In plain speak, bundling
(vendoring in Go linguo) just does not scale. You need an awful lot of
manpower to audit the hundreds of other projects each app bundles, bundling
removes all the package tooling that may help you to do so, and the result
is not shared with any other package, or with other versions of the same
package, so you get zero positive network effects. Worse, big bundled apps
that do not actually try to work in unbundled mode, do not actually test
the code they export (they are bundling, remember) so the result is toxic
to small Go packages that try to work with them.
So bundling parts is a direct road to bundling everything, bundling
everything is a direct road to bundling everything blindly, bundling
everything blindly is error-prone and dangerous (because upstreams are only
human and do make lots of mistakes), and pretty much removes any value
Fedora can add to end users, to other parts of Fedora that would like to
integrate with Go software, or to the upstream projects themselves (no
Fedora QA, no stream of Fedora-originated fixes, no Fedora pressure to
stabilize parts when upstream is lost in tunnel effect mode and does not
realize that it is wasting everyone's time starting with its own).
Therefore, trying to get all this it a better state, requires the
following steps IMHO:
1. completely refactor our Go packaging style it's less painful to update
Go packages and they do not need a packager with deep knowledge of
package-specific shell glue. That takes documentation, and factoring out
common Go packaging functions in shared rpm code (macros and autodeps) →
that's what I posted
2. using the new documentation and tooling to clean up years of Fedora
technical debt, and create a new set of up-to-date Go packages that can
serve as new baseline → I have hundreds of specs that I'm waiting for step
1 to complete to submit. They won't constitute a full baseline by
themselves, they're not all 100% done, it's too much work to do alone but
working with others requires the common macros and documentation to be
merged and adopted. This step is going to be painful I'm afraid, Fedora dug
itself a deep hole, leaving it is going to be hard.
3. hopefully, the result will be streamlined enough it won't be too
painful to keep up to date, having an up to date baseline will help attract
new Go packagers like you, everyone will benefit and be happy. We had to
package some ostree bits for example to create this baseline, and I'm
pretty sure ostree people within Fedora would prefer to maintain those
themselves, if the rest of the Fedora go universe didn't make it too hard
> B) more
> intimidating. It seems like to package the new package, I have to get the
> maintainer of the library to upgrade the version,
The guidelines and automation aim at making upgrading easy, and avoid one
package or packager blocking others
> and someone needs to
> rebuild everything that depends on the rebuilt library and test that the
> rebuilt packages work.
I hope that normalizing Fedora Go packages means it will be possible to
automate QA tests such as "try to rebuild all the packages that depend on
golang(foo) every time you see the package providing golang(foo) changing,
and report what broke"
That would be expensive computing-side but a lot less expensive and long
than expecting each Go packager to do it himself with his own means. And
that is certainly not overkill, given how lax Go projects are about
maintaining API stability.
And then in case of breakage, revert or create a compat package. That's
why there is a long chapter dedicated to compat package creation in the
proposed guidelines.
Regards,
--
Nicolas Mailhot
_______________________________________________
devel mailing list -- devel(a)lists.fedoraproject.org
To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org