4:52 a.m.
----- Original Message -----
From: "Sam Whited" <sam(a)samwhited.com>
To: "rprichard via golang-dev" <golang-dev(a)googlegroups.com>
Sent: Monday, March 4, 2019 7:06:37 PM
Subject: Re: [golang-dev] proposal: public module authentication with the Go notary
TL;DR — Right now when fetching a module Google isn't involved, and I'd
like it to stay that way. Please let me configure and use different
notaries without also using a proxy.
> There is no plan to allow use of alternate notaries, which would add
> complexity and potentially reduce the overall security of the system,
> allowing different users to be attacked by compromising different
> notaries.
If you're somewhere that Google doesn't service (eg. due to U.S. export
laws) does this mean you're entirely out of luck and can't use the new
security features? It seems unfortunate that a developer in Iran would
have to fall back to the TOFU model just because Google decided to bless
themselves and not allow anyone else to run a notary or jump through
extra hoops to configure or run a proxy.
> We originally considered having multiple notaries signing individual
> go.sum entries and requiring the go command to collect signatures from
> a quorum of notaries before accepting an entry. That design depended
> on the uptime of multiple services and could still be compromised
> undetectably by compromising enough notaries. That is, that design
> would blindly trust a quorum of notaries.
This seems strictly superior to blindly trusting a single notary. Why
does Google think it will be more secure than Google and Mozilla, or
Google and Microsoft, or Google and <your company>?
As far as uptime is concerned, you're always limited to the uptime of
your least available notary, I don't necessarily think Google's uptime
will be any better than any other large company that might choose to run
a notary, so that seems fine: I have the option of trusting only
notaries with high availability, meaning that it's likely no worse than
if I just trusted Google.
> The design presented here uses the transparent log eliminates blind
> trust in a quorum of notaries and instead uses a “trust but verify”
> model with a single notary.
We could also "trust but verify" a quorum of notaries, so this seems
like a false dichotomy.
On a more personal note, having a non-Google controlled notary seems
like an absolute requirement to me and I would prefer to avoid using a
Google run service entirely if possible. Google doesn't need to know
anything about what modules my company is using. I do appreciate the
privacy section, which addresses this, but tying the notary use to using
a proxy or running your own proxy is likely out of reach for many
individuals (including me).
—Sam
I have a similar fears and worries here. First thing that popped on my mind(personally as
a external observer) this is a move of Google to fully productize(data mine) a Go users
community(for very limited payback), even if it is not a main driver it will make it
irresistibly easy to do so(with google hosted notary).
IMHO this feature should be opt-in and the actual default instance of the notary should be
run by trustworthy and independent 3rd party(ideally NPO) with clear and transparent
privacy policy and on community accessible infrastructure(so a community members can
actually participate in the maintenance and verify policies surrounding it, i.e. not like
a current Go infra is run, only Google employees can contribute/touch it).
As I will carefully evaluate all the details of the final implementation, but from this
first look I'm currently leaning to "patch out" or de-configure by default
this feature in Fedora when/if it lands in upstream GC, to preserve the privacy of the
users.
JC
PS: This is just my personal Fedora's GC maintainer take on this issue.
>
> On Mon, Mar 4, 2019, at 17:32, Russ Cox wrote:
> > Hi all,
> >
> > I wanted to let people here know about the proposal design we just
> > published: golang.org/design/25530-notary, for golang.org/issue/25530.
> > (We also mentioned this general idea in our blog post from back in
> > December, https://blog.golang.org/modules2019.)
> >
> > As usual, comments are welcome on the issue or, if you prefer not to
> > use the issue tracker, here in this thread.
> >
> > Thanks. Russ
> >
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "golang-dev" group. To unsubscribe from this group and stop
> > receiving emails from it, send an email to golang-
> > dev+unsubscribe(a)googlegroups.com. For more options, visit
> > https://groups.google.com/d/optout.
>
> --
> Sam Whited
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-dev+unsubscribe(a)googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
2:03 p.m.
New subject: [golang-dev] proposal: public module authentication with the Go notary
On Thu, Mar 7, 2019 at 5:52 AM Jakub Cajka <jcajka(a)redhat.com> wrote:
As I will carefully evaluate all the details of the final
implementation,
but from this first look I'm currently leaning to "patch out" or
de-configure by default this feature in Fedora when/if it lands in upstream
GC, to preserve the privacy of the users.
Thanks for this feedback. I would only ask that you remember that
publishing the design proposal is the *first* step in the public
discussion, not the last. Please remember to check back in for the final
result before you decide to apply local patches in Fedora.
Russ
3:16 a.m.
New subject: [golang-dev] proposal: public module authentication with the Go
notary
----- Original Message -----
From: "Russ Cox" <rsc(a)golang.org>
To: "Jakub Cajka" <jcajka(a)redhat.com>
Cc: "Sam Whited" <sam(a)samwhited.com>, "rprichard via
golang-dev" <golang-dev(a)googlegroups.com>,
golang(a)lists.fedoraproject.org
Sent: Thursday, March 7, 2019 9:02:50 PM
Subject: Re: [golang-dev] proposal: public module authentication with the Go notary
On Thu, Mar 7, 2019 at 5:52 AM Jakub Cajka <jcajka(a)redhat.com> wrote:
> As I will carefully evaluate all the details of the final implementation,
> but from this first look I'm currently leaning to "patch out" or
> de-configure by default this feature in Fedora when/if it lands in upstream
> GC, to preserve the privacy of the users.
>
Thanks for this feedback. I would only ask that you remember that
publishing the design proposal is the *first* step in the public
discussion, not the last. Please remember to check back in for the final
result before you decide to apply local patches in Fedora.
Russ
I will be observing the future iterations. I'm looking forward to them :).
JC
4:06 a.m.
New subject: [golang-dev] proposal: public module authentication with the Go
notary
Hi Jakub,
The notary part of Go modules, like the rest of the module
implementation, suffers from a lack of understanding of integration and
QA workflows, and a simplistic dev-centric worldview.
As designed, it will have to be patched out by every single entity
trying to perform large scale Go code integration and QA, starting with
the usual suspects, Linux distributions. Where that will leave upstream
Go, since its main target platform is Linux, I have no idea (I suspect
there will be some angst @Google about it).
I write this from the POW of the person that tries to update the Fedora
Linux Go tooling so we can ship Go with module mode on in august.
Whatever we do at that time will be mirrored by Fedora downstreams like
RHEL and Centos and all the other Linux variants that get their
inspiration Fedora-side.
It's not an armchair analysis, I already wrote way too much module-
oriented custom code because the upstream tooling is deficient, and I've
read my share of upstream issue reports.
CONTINUOUS TRANSPARENT INTERNET LOOKUPS ARE NOT ACCEPTABLE POST−DEV
When you integrate large volumes of code, targeting multiple hardware
architectures, it’s not acceptable to have the result of the Arm QA run
differ from the result of the x86_64 run just because it was scheduled
some minutes later on the QA farm, the internet state changed in the
meanwhile, and the Go tools decided to “help” you by looking at the
internet behind your back and updated transparently the state being
QAed. That's nothing new of Go specific, and that’s why our build system
disables network access in build containers after populating them with
the code to be built and tested, and already did so before someone
decided to invent Go.
But, that QA reproducibility constrain was not understood by Go
developers, and pretty much all the Go tools that were ported to module
mode, will attempt Internet accesses at the slightest occasion, and
change the project state based on those access results, with no ability
to control or disable it, and nasty failure modes if the internet access
fails.
The Go issue tracker is filling up with reports of people expressing
their incredulity (or more) after hitting a variation of this problem on
their QA systems.
QA−ED CODE IS NOT PRISTINE DEV CODE
No code is ever perfect.
When you integrate large volumes of code, targeting multiple hardware
architectures, you *will* hit problems missed by the original upstream
dev team. And you *will* have to patch them up, and you *will* have to
build the result without waiting for upstream to look at it because, in
production, the show must go on, some of those will eventually have
zero-day security implications, and the upstreaming lag is not
acceptable (assuming upstream is available and friendly, which is not
necessarily the case).
So no serious shop is ever going to build its information system from
pristine upstream dev code or pristine upstream Go modules. Pristine
upstream code building is a nice ideal, but in the real world, it’s
utopia.
If you don’t believe me, take your favorite app, and try to build a
system containing it from genuine unadultered upstream code (that means,
without relying on Fedora, Debian or whatever, since we all know they
are full of “nasty” patched code).
Again, that technical real-world-is-not-perfect was not understood by Go
developers. Sure they gave us the replace directive in Go module. But
that is opt-in. So it only works on a small scale, when fixing third
party code is an exception, and you only have a handful of projects that
use this third-party code.
On a large scale everyone will just preventively rename everything by
default just to retain the ability to perform fixes easily. You'll end
up with blanket replacement of every "upstream" module name with
"qaed/upstream" forks. And probably easier to rewrite all the imports to
point to qaed/upstream by default in the source code.
That will result in a huge mess, much larger than the current GOPATH
vendoring mess.
All because the design does not take into account the last mile QA
patching that occurs before putting code in production.
Blindly enforcing renaming rules because you don't understand or accept
the existence of QA middlemen produced things like Iceweasel. And I
don't think Mozilla or Debian were ever happy about it.
PUBLISHING QA-ED CODE CAN NOT DEPEND ON A THIRD PARTY
At least, not if you want the result to work in a free software
ecosystem. Freedom 4 “The freedom to distribute copies of your modified
versions to others” is not “The freedom to distribute copies of your
modified versions to others, but only if you tell this third party
first”.
And Google is free to choose not to target a free software ecosystem,
but that means all the entities that do target a free software ecosystem
(like Linux distributions) or have happily built their infrastructure on
free software products (like every single major cloud operator out
there), will have fork Go, or reorient their software investments
somewhere else, or some combination of those. And I don't think Google
wants that, or will be happy about it.
But, that's the situation the current notary design will create.
Because in trying to enforce a dev utopia, it is going to break the
workflows of a large proportion of the current Go ecosystem.
Any working system will need a way to declare trust in a local authority
(the shop QA team) or a set of authorities (the shop QA team, and
trusted third parties), and not have this trust rely on continuous
access to a server, regardless of who controls it (so detached digital
signatures, not direct server hash lookups).
And I’ll stop here before I write things I will regret later.
Regards,
--
Nicolas Mailhot
5:28 a.m.
New subject: Go modules in Fedora
----- Original Message -----
From: "Nicolas Mailhot"
<nicolas.mailhot(a)laposte.net>
To: golang(a)lists.fedoraproject.org, "Sam Whited" <sam(a)samwhited.com>
Cc: "rprichard via golang-dev" <golang-dev(a)googlegroups.com>
Sent: Friday, March 8, 2019 11:06:44 AM
Subject: Re: [golang-dev] proposal: public module authentication with the Go notary
Hi Jakub,
The notary part of Go modules, like the rest of the module
implementation, suffers from a lack of understanding of integration and
QA workflows, and a simplistic dev-centric worldview.
As designed, it will have to be patched out by every single entity
trying to perform large scale Go code integration and QA, starting with
the usual suspects, Linux distributions. Where that will leave upstream
Go, since its main target platform is Linux, I have no idea (I suspect
there will be some angst @Google about it).
I write this from the POW of the person that tries to update the Fedora
Linux Go tooling so we can ship Go with module mode on in august.
Whatever we do at that time will be mirrored by Fedora downstreams like
RHEL and Centos and all the other Linux variants that get their
inspiration Fedora-side.
It's not an armchair analysis, I already wrote way too much module-
oriented custom code because the upstream tooling is deficient, and I've
read my share of upstream issue reports.
CONTINUOUS TRANSPARENT INTERNET LOOKUPS ARE NOT ACCEPTABLE POST−DEV
When you integrate large volumes of code, targeting multiple hardware
architectures, it’s not acceptable to have the result of the Arm QA run
differ from the result of the x86_64 run just because it was scheduled
some minutes later on the QA farm, the internet state changed in the
meanwhile, and the Go tools decided to “help” you by looking at the
internet behind your back and updated transparently the state being
QAed. That's nothing new of Go specific, and that’s why our build system
disables network access in build containers after populating them with
the code to be built and tested, and already did so before someone
decided to invent Go.
But, that QA reproducibility constrain was not understood by Go
developers, and pretty much all the Go tools that were ported to module
mode, will attempt Internet accesses at the slightest occasion, and
change the project state based on those access results, with no ability
to control or disable it, and nasty failure modes if the internet access
fails.
The Go issue tracker is filling up with reports of people expressing
their incredulity (or more) after hitting a variation of this problem on
their QA systems.
QA−ED CODE IS NOT PRISTINE DEV CODE
No code is ever perfect.
When you integrate large volumes of code, targeting multiple hardware
architectures, you *will* hit problems missed by the original upstream
dev team. And you *will* have to patch them up, and you *will* have to
build the result without waiting for upstream to look at it because, in
production, the show must go on, some of those will eventually have
zero-day security implications, and the upstreaming lag is not
acceptable (assuming upstream is available and friendly, which is not
necessarily the case).
So no serious shop is ever going to build its information system from
pristine upstream dev code or pristine upstream Go modules. Pristine
upstream code building is a nice ideal, but in the real world, it’s
utopia.
If you don’t believe me, take your favorite app, and try to build a
system containing it from genuine unadultered upstream code (that means,
without relying on Fedora, Debian or whatever, since we all know they
are full of “nasty” patched code).
Again, that technical real-world-is-not-perfect was not understood by Go
developers. Sure they gave us the replace directive in Go module. But
that is opt-in. So it only works on a small scale, when fixing third
party code is an exception, and you only have a handful of projects that
use this third-party code.
On a large scale everyone will just preventively rename everything by
default just to retain the ability to perform fixes easily. You'll end
up with blanket replacement of every "upstream" module name with
"qaed/upstream" forks. And probably easier to rewrite all the imports to
point to qaed/upstream by default in the source code.
That will result in a huge mess, much larger than the current GOPATH
vendoring mess.
All because the design does not take into account the last mile QA
patching that occurs before putting code in production.
Blindly enforcing renaming rules because you don't understand or accept
the existence of QA middlemen produced things like Iceweasel. And I
don't think Mozilla or Debian were ever happy about it.
PUBLISHING QA-ED CODE CAN NOT DEPEND ON A THIRD PARTY
At least, not if you want the result to work in a free software
ecosystem. Freedom 4 “The freedom to distribute copies of your modified
versions to others” is not “The freedom to distribute copies of your
modified versions to others, but only if you tell this third party
first”.
And Google is free to choose not to target a free software ecosystem,
but that means all the entities that do target a free software ecosystem
(like Linux distributions) or have happily built their infrastructure on
free software products (like every single major cloud operator out
there), will have fork Go, or reorient their software investments
somewhere else, or some combination of those. And I don't think Google
wants that, or will be happy about it.
But, that's the situation the current notary design will create.
Because in trying to enforce a dev utopia, it is going to break the
workflows of a large proportion of the current Go ecosystem.
Any working system will need a way to declare trust in a local authority
(the shop QA team) or a set of authorities (the shop QA team, and
trusted third parties), and not have this trust rely on continuous
access to a server, regardless of who controls it (so detached digital
signatures, not direct server hash lookups).
And I’ll stop here before I write things I will regret later.
Regards,
--
Nicolas Mailhot
Hello,
first this is kind of off topic to the notary issue discussed in the thread, so stating
new one.
I'm happy to see that you have notice the go modules and complexities that it will
bring along :). I have been trying to bring this up on the SIG and at meetings for past
several months(thanks for bringing it up on the last meeting, just read the log), but
nobody has been interested in discussing this ahead of time(I don't blame you/them). I
don't believe that I'm the right person that you want to focus on with this long
list of notes(what you are writing, I can't really do anything about). You should
definitively bring this up on the upstream golang-dev list(AFAIK you need to be subscribed
there to post), they are really keen for any feedback.
JC
> _______________________________________________
> golang mailing list -- golang(a)lists.fedoraproject.org
> To unsubscribe send an email to golang-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/golang@lists.fedoraproject.org
>
9:28 a.m.
New subject: [golang-dev] proposal: public module authentication with the Go notary
On Fri, Mar 8, 2019 at 8:09 AM 'Nicolas Mailhot' via golang-dev <
golang-dev(a)googlegroups.com> wrote:
The notary part of Go modules, like the rest of the module
implementation, suffers from a lack of understanding of integration and
QA workflows, and a simplistic dev-centric worldview.
This is not an auspicious beginning.
This mail came across as trying more to be antagonistic than constructive.
Even so, I really would like to understand your concerns
and either help address misunderstandings or adjust the
design appropriately.
CONTINUOUS TRANSPARENT INTERNET LOOKUPS ARE NOT ACCEPTABLE POST−DEV
[snip]
But, that QA reproducibility constrain was not understood by Go
developers, and pretty much all the Go tools that were ported to module
mode, will attempt Internet accesses at the slightest occasion, and
change the project state based on those access results, with no ability
to control or disable it, and nasty failure modes if the internet access
fails.
This really couldn't be farther from the truth.
You can turn off module changes using -mod=readonly in CI/CD systems.
We added it explicitly for that use case. Quoting golang.org/issue/26361 (July
2018):
CI systems need a way to test what's in the go.mod and fail if
something is missing instead of looking to satisfy dependencies
automatically. We meant for -getmode=local to mean this, but it
really means something a little bit different. We should add a
-getmode that does mean "you can't change go.mod". Maybe
-getmode=noauto.
The eventual spelling was -mod=readonly, and it shipped in Go 1.11
with the initial module preview.
More generally, we understand the importance of reproducibility.
For example, see this post <https://research.swtch.com/vgo-repro> and this
talk <https://www.youtube.com/watch?v=F8nrpe0XWRg>;.
The Go issue tracker is filling up with reports of people expressing
their incredulity (or more) after hitting a variation of this problem
on
their QA systems.
If you could point to specific examples, that would help me understand
your concerns a bit better. I did try to find some but failed:
$ issue cmd/go QA
26746 proposal: doc/install: define minimum supported VCS versions
14812 runtime: GC causes latency spikes
$ issue is:closed cmd/go QA
24301 cmd/go: add package version support to Go toolchain
22491 proposal: cmd/link: support BUILD_PATH_PREFIX_MAP for reproducible
binaries when built under varying path
27160 x/build: set up AIX builder
12660 x/mobile: support Xcode 7
$
QA−ED CODE IS NOT PRISTINE DEV CODE
No code is ever perfect.
When you integrate large volumes of code, targeting multiple hardware
architectures, you *will* hit problems missed by the original upstream
dev team. And you *will* have to patch them up, and you *will* have to
build the result without waiting for upstream to look at it because, in
production, the show must go on, some of those will eventually have
zero-day security implications, and the upstreaming lag is not
acceptable (assuming upstream is available and friendly, which is not
necessarily the case).
So no serious shop is ever going to build its information system from
pristine upstream dev code or pristine upstream Go modules. Pristine
upstream code building is a nice ideal, but in the real world, it’s
utopia.
If you don’t believe me, take your favorite app, and try to build a
system containing it from genuine unadultered upstream code (that means,
without relying on Fedora, Debian or whatever, since we all know they
are full of “nasty” patched code).
Again, that technical real-world-is-not-perfect was not understood by Go
developers. Sure they gave us the replace directive in Go module. But
that is opt-in. So it only works on a small scale, when fixing third
party code is an exception, and you only have a handful of projects that
use this third-party code.
On a large scale everyone will just preventively rename everything by
default just to retain the ability to perform fixes easily. You'll end
up with blanket replacement of every "upstream" module name with
"qaed/upstream" forks. And probably easier to rewrite all the imports to
point to qaed/upstream by default in the source code.
That will result in a huge mess, much larger than the current GOPATH
vendoring mess.
All because the design does not take into account the last mile QA
patching that occurs before putting code in production.
Blindly enforcing renaming rules because you don't understand or accept
the existence of QA middlemen produced things like Iceweasel. And I
don't think Mozilla or Debian were ever happy about it.
I've read this a couple times and found it a little hard to follow,
but I think what you are saying is that the go.sum and notary checks
are going to cause serious problems because Fedora and other
distributions want to create modified copies of module versions and
use them for building the software they ship. I don't see why that
would be the case.
I would have expected that if Fedora modified a library, they
would give it a different version number, so that for example modifying
v1.2.2 would produce v1.2.3-fedora.1 (it would be nice if SemVer
had a kind of 'post-release' syntax; maybe some day).
Go.sum and notary checks would only trigger at all if Fedora
were to take v1.2.2, modify it, and then try to pass it off as the
original v1.2.2, but that's indistinguishable from a man-in-the-middle
attack. Adding a suffix to the version string to indicate the presence
of Fedora-specific patches does not seem, at least to me, to be
anywhere near the level of renaming Firefox to IceWeasel, and it
avoids confusion about "which" v1.2.2 a build is using.
Furthermore, once you have created those versions and want to
build software using them, all it takes is to create a new module
with just a go.mod file (no source code) listing those versions as
requirements and then run builds referring to the unmodified original
top-level targets. Those targets' dependencies will be bumped forward
to the Fedora-patched versions listed in the go.mod file, without
any need to modify any of the client modules at all.
Or maybe I misunderstood what you were trying to say in this section.
PUBLISHING QA-ED CODE CAN NOT DEPEND ON A THIRD PARTY
At least, not if you want the result to work in a free software
ecosystem. Freedom 4 “The freedom to distribute copies of your modified
versions to others” is not “The freedom to distribute copies of your
modified versions to others, but only if you tell this third party
first”.
And Google is free to choose not to target a free software
ecosystem,
but that means all the entities that do target a free software ecosystem
(like Linux distributions) or have happily built their infrastructure on
free software products (like every single major cloud operator out
there), will have fork Go, or reorient their software investments
somewhere else, or some combination of those. And I don't think Google
wants that, or will be happy about it.
But, that's the situation the current notary design will create.
Because in trying to enforce a dev utopia, it is going to break the
workflows of a large proportion of the current Go ecosystem.
I'm still not sure exactly what the dev utopia is.
Can you help me understand better?
Is the utopia the fact there would be a consistent meaning for
mymodule(a)v1.2.2 across all possible references and that any
modified source code would have to present a modified version number?
Is the inability to silently modify code without changing the version
number what's going to break the workflows of a large proportion of
the Go ecosystem? I'd like to understand that better if so.
Any working system will need a way to declare trust in a local
authority
(the shop QA team) or a set of authorities (the shop QA team, and
trusted third parties), and not have this trust rely on continuous
access to a server, regardless of who controls it (so detached digital
signatures, not direct server hash lookups).
Your use of the word "continuous", both in this quoted text and in the
heading earlier, makes me think you might not realize that the notary
access is only needed when adding a new line to the go.sum file.
The steady state of a module that a module's go.mod file lists specific
versions of all its direct dependencies, and its go.sum file lists the
cryptographic hashes of all its direct and indirect dependencies.
The only time the system falls out of the steady state is when a code
change adds an import of a new module; the next build of that changed
code reestablishes the steady state. And the 'go mod tidy' command's
job is exactly to establish that state for all possible builds in the
module.
Once that state is established - once go.sum has a hash for each
dependency needed in a build - there is *zero* notary access during
any builds of that module. And as I mentioned above, a CI/CD system
or any other system can use -mod=readonly (either on the command line
or in the $GOFLAGS environment variable) to disable any attempt to
reestablish the steady state (that is, disable any attempt to modify go.mod
or go.sum) except by an explicit 'go get' or 'go mod tidy' command.
So the notary access is occasional, predictable, and controllable.
It is *not* continuous.
Put a different way, the notary is how the go command populates go.sum
by default. If a QA system needs to populate it a different way, that's
fine:
one option is 'go get -insecure', another option is GONOVERIFY=*, and
a third option is to just write the desired go.sum directly:
it's a simple, line-oriented text file. Even if a QA team insists on
modifying
mymodule(a)v1.2.2 without giving it a new version, if they also update the
go.sum files of modules using mymodule(a)v1.2.2, the builds will succeed,
all without any notary access.
Stepping back from the specific details, again I am more than happy
to understand everyone's concerns with the notary design and work to
address them. We've been having a productive conversation on the
GitHub issue, and I hope we can have a productive conversation here too.
Best,
Russ
6:52 p.m.
New subject: [golang-dev] proposal: public module authentication with the Go
notary
Le vendredi 08 mars 2019 à 10:28 -0500, Russ Cox a écrit :
On Fri, Mar 8, 2019 at 8:09 AM 'Nicolas Mailhot' via
golang-dev <
golang-dev(a)googlegroups.com> wrote:
> The notary part of Go modules, like the rest of the module
> implementation, suffers from a lack of understanding of integration
> and QA workflows, and a simplistic dev-centric worldview.
This is not an auspicious beginning.
This mail came across as trying more to be antagonistic than
constructive
I'm sorry, my level of English is not sufficient to convey information
you want to ignore, and dress it up so you feel good about it. I've
tried to stick to plain facts. If you object to plain facts I can't do
anything about it.
Even so, I really would like to understand your concerns
and either help address misunderstandings or adjust the
design appropriately.
> CONTINUOUS TRANSPARENT INTERNET LOOKUPS ARE NOT ACCEPTABLE POST−DEV
>
> [snip]
>
> But, that QA reproducibility constrain was not understood by Go
> developers, and pretty much all the Go tools that were ported to
> module
> mode, will attempt Internet accesses at the slightest occasion, and
> change the project state based on those access results, with no
> ability
> to control or disable it, and nasty failure modes if the internet
> access
> fails.
This really couldn't be farther from the truth.
You can turn off module changes using -mod=readonly in CI/CD systems.
It would be nice if it where true. Unfortunately even a simple command
like tell me what you know about the local code tree
go list -json -mod=readonly ./...
will abort if the CI/CD system cuts network access to make sure builds
do not depend on Internet state.
That's what the nasty failure modes are about. If the go command is not
sure about anything, it will “solve” things by trying to download new
bits from the Internet. If the Internet is not available, it will abort
violently, not degrade gracefully and work with what it has.
The no-modification no-internet CI/CD constrain was not taken into
account into the original design. It was bolted on later, and the
bolting is imperfect. Just take a vacation in some paradisiac place with
no internet access, and see how far you can do go coding in module mode.
*That* will replicate our CI/CD constrains (paradisiac place as a bonus,
we don't have those in our build farms – see I'm trying not to be
antagonistic).
> QA−ED CODE IS NOT PRISTINE DEV CODE
>
I've read this a couple times and found it a little hard to follow,
but I think what you are saying is that the go.sum and notary checks
are going to cause serious problems because Fedora and other
distributions want to create modified copies of module versions and
use them for building the software they ship. I don't see why that
would be the case.
I would have expected that if Fedora modified a library, they
would give it a different version number, so that for example
modifying v1.2.2 would produce v1.2.3-fedora.1
And that's not the case neither for Fedora, nor RHEL, nor Debian, nor
pretty much any large scale integrator, because when you integrate
masses of third-party code you will eventually hit bugs in pretty much
every component, so having to run patched code at every layer is the
norm not the exception.
It would be terribly inconvenient to have to rename or renumber or
replace everything, and then have to convince all the other components
to use the renamed or renumbered versions of the components they depend
on. It would be even more inconvenient to do it just in time and
continuously flip flop between upstream and local names and numbers.
That would actually add friction to merging back and returning to
pristine upstream code, because merging back would now cost a
rename/renumber, instead of being a net win. Patching is forced on you
(you hit a bug in upstream code). Upstreaming is a deliberate virtous
choice (you've already fixed your problem locally). If you make
upstreaming more expensive, people just stop doing it.
There are legal ways to force distributions to do the wasteful
renaming/renumbering dance. They will take it as an hostile imposition.
Where do you think Iceweasel came from? No love lost here.
Go.sum and notary checks would only trigger at all if Fedora
were to take v1.2.2, modify it, and then try to pass it off as the
original v1.2.2
That's the standard Linux distribution workflow.
but that's indistinguishable from a man-in-the-middle attack.
The whole purpose of a distribution, is to be a giant middleman, freely
chosen by the end user, between the code released upstream, and this end
user¹. All middlemen are not hostile. You need to understand that or
your system will not work.
In the real world you have friendly middlemen. Sometimes layers of them
(it is quite common for local organisations to add another level of
changes over distro changes). Sometimes those middlemen make changes.
Sometimes they just check things for nastiness.
A correct trust test is not “it exists exactly this way on the
Internet”. What kind of assurance is that? If I dump malware on a public
URL, it’s trusted as long as no AV touches it mid-flight?
A correct trust test is “has the state been signed by an entity the user
trusts”.
So, digital signatures. A way to configure the public keys of trusted
third parties. And, if no signature by a trusted third party is found
locally, at last resort, and only if the user asked for it, look on the
internet if a notary signature exists. Of course, that's a piss-poor
level or trust, but better that than nothing.
Furthermore, once you have created those versions and want to
build software using them, all it takes is to create a new module
with just a go.mod file (no source code) listing those versions as
requirements and then run builds referring to the unmodified original
top-level targets. Those targets' dependencies will be bumped forward
to the Fedora-patched versions listed in the go.mod file, without
any need to modify any of the client modules at all.
# dnf repoquery --disablerepo=* --enablerepo=rawhide \
--whatprovides 'golang(*)' |wc -l
766
(here Debian people are laughing at me because we are lagging behind
them on the Go integration front)
We have several hundreds more Go components in the integration queue
waiting for review.
Some of those will end up as several Go modules.
And you want us to renumber all this, and patch all the other module
files that use those with the new numbers, and redo it all every time
something changes, all year round, just because the go command can’t
accept that the code state Fedora builds from, may differ from the one
observed on the Internet?
Really?
REALLY ?
Do you really REALLY think any sane integrator will play this dance long
instead of patching out notary checks from its go command, and be done
with it?
Is the utopia the fact there would be a consistent meaning for
mymodule(a)v1.2.2 across all possible references and that any
modified source code would have to present a modified version number?
The utopia is thinking that everything is released in a perfect state by
upstream, so changes downstream need not happen, and therefore any
downstream change is necessarily an attempt to inject malware by Mr
Nasty.
Is the inability to silently modify code without changing the
version
number what's going to break the workflows of a large proportion of
the Go ecosystem? I'd like to understand that better if so.
Distributions distinguish between the upstream version number and their
own build id. So you have
mymodule(a)v1.2.2 release 1
mymodule(a)v1.2.2 release 2
and so on. Each release id is a separate build that can involve
different patches (some release ids are more complex than a single
number).
Only a single release can exist within the distribution at a given time,
so makefiles, go mod files and so on need not differentiate between
release X and release Y, they will only see one of those at any time and
they better work with it because they won't be given or allowed any
other.
System artefacts (libraries, binaries, go modules) are built from plain
upstream source code, as downloaded by distribution processes directly
from uptream VCS or website, after preparation (removal of problem
parts, patching, etc) in pristine containers, isolated from the
Internet, and populated only with a minimal distribution installation
and the content of the distribution components necessary for their
build.
So to build github.com/my/thing version x.y.z that declares in its
module file
module github.com/my/thing
require (
github.com/some/dependency v1.2.3
github.com/another/dependency/v4 v4.0.0
)
A. We will populate a clean container with
1. a minimal system
2. the go compiler
3. the most recent system component that provides the go module
github.com/some/dependency ≥ 1.2.3 and < 2
and all its dependencies
(go module as produced by our own built of this other component, not
as downloaded by go get from the internet)
4. the most recent system component that provides the go module
github.com/another/dependency/v4 ≥ 4.0.0 and < 5
and all its dependencies (ditto)
5. the source code for github.com/my/thing x.y.z as downloaded and
checked from the internet by a human (not by go get), and then sealed
B. We will cut internet access of this container
C. We will prepare the github.com/my/thing x.y.z source (remove problem
parts, patch bugs, remove vendored code, remove at least local replaces
in go.mod)
I'm fairly certain we will nuke go.sum because we want to build from our
reference state against our reference versions, not reproduce whatever
upstream tested on ubuntu or windows.
D. We will point the go compiler to the local module files (via GOPROXY)
E. We will ask a go command to transform our github.com/my/thing x.y.z
source state in a module files that can be used by other Go code (zip
ziphash info mod files) once deployed in our GOPROXY directory. It would
be nice if go mod pack existed upstream otherwise we will write and use
our own utility.
That will involve sanitizing the upstream mod file (remove indirect
requires, remove local replaces, not sure about non-local replaces yes,
by gut feeling is that we should remove them too and force the fixing of
imports in source preparation, but I'm not sure yet)
For various technical reasons it's not possible to deploy directly in
GOPROXY at this stage, so at this point you have the prepared
github.com/my/thing x.y.z source code in unpacked state in a directory,
the module it needs in GOPROXY, and a packed version of
github.com/my/thing x.y.z in a staging directory.
D. we will ask the go compiler to build the various binaries that need
to be produced from github.com/my/thing version x.y.z
E. we will pack the result files (binaries and go modules) in system
components, so they can be deployed at need. The component containing
github.com/my/thing x.y.z will record a need for
github.com/some/dependency ≥ 1.2.3 and < 2 and
github.com/another/dependency/v4 ≥ 4.0.0 and < 5
So the internet access won't exist when you think it exists, the files
arrive on disk without go get, and the state they are available on is
not the upstream state.
Having worked in a proprietary integration shop before, and working with
proprietary integrators today, the workflows are no so different, except
proprietary shops tend to be a lot laxer, allowing internet access when
they should not, and either processing code as downloaded from the
internet, with no checks, or forking it to death, without trying to re-
attach to upstream.
Best regards,
¹ The function of this middleman is beat upstream code into shape so the
user does not have to do it itself. Beating into shape does involve
large-scale changing of upstream code.
Users choose the distribution system because their alternatives are:
1. to hope every single upstream they use never makes a mistake
requiring a last-mile fix (fat chance on that)
2. wasting their time doing the last-mile fixing themselves, instead of
delegating this function to distributors.
It's all free software. An unhappy distribution user can take the source
code he needs and get it integrated somewhere else. No strings attached.
--
Nicolas Mailhot
7:50 p.m.
New subject: [golang-dev] proposal: public module authentication with the Go notary
On Fri, Mar 8, 2019 at 4:52 PM 'Nicolas Mailhot' via golang-dev
<golang-dev(a)googlegroups.com> wrote:
Le vendredi 08 mars 2019 à 10:28 -0500, Russ Cox a écrit :
> On Fri, Mar 8, 2019 at 8:09 AM 'Nicolas Mailhot' via golang-dev <
> golang-dev(a)googlegroups.com> wrote:
> > The notary part of Go modules, like the rest of the module
> > implementation, suffers from a lack of understanding of integration
> > and QA workflows, and a simplistic dev-centric worldview.
>
> This is not an auspicious beginning.
> This mail came across as trying more to be antagonistic than
> constructive
I'm sorry, my level of English is not sufficient to convey information
you want to ignore, and dress it up so you feel good about it. I've
tried to stick to plain facts. If you object to plain facts I can't do
anything about it.
You say that this is information that we want to ignore. Why leads
you to say such a thing? What makes you think that we want to ignore
it? We have never claimed to know everything. Adding modules to the
go tool is a work in progress.
Problems can be identified and solved but a necessary step is that
everyone treat each other with respect. When you accuse others of
wanting to ignore you, that makes your message seem like an attack,
and indeed makes people more likely to ignore it. Your message would
be a lot easier to read if you did just stick to plain facts.
Thanks.
Ian
8:01 p.m.
New subject: [golang-dev] proposal: public module authentication with the Go notary
On Fri, Mar 8, 2019 at 4:52 PM 'Nicolas Mailhot' via golang-dev <
golang-dev(a)googlegroups.com> wrote:
It would be nice if it where true. Unfortunately even a simple
command
like tell me what you know about the local code tree
go list -json -mod=readonly ./...
will abort if the CI/CD system cuts network access to make sure builds
do not depend on Internet state.
The behavior you described sounds like an issue to me. However, I wasn't
able to easily reproduce it locally: I tried running "strace -f go list
-json -mod=readonly ./... |& grep connect" under a few scenarios (e.g.,
missing go.sum, or incomplete go.mod file), and none of them seemed to
result in network access.
Can you provide more detailed reproduction steps?
The no-modification no-internet CI/CD constrain was not taken into
account into the original design. It was bolted on later, and the
bolting is imperfect.
I don't think this is true. Google's internal build system has used this
same "no-modification no-internet" constraint for around a decade:
http://google-engtools.blogspot.com/2011/09/build-in-cloud-distributing-b...
https://mike-bland.com/2012/10/01/tools.html#blaze-forge-srcfs-objfs
Supporting this usage pattern has been and remains very important to Go.
I would have expected that if Fedora modified a library, they
> would give it a different version number, so that for example
> modifying v1.2.2 would produce v1.2.3-fedora.1
And that's not the case neither for Fedora, nor RHEL, nor Debian, nor
pretty much any large scale integrator,
When I run "gcc --version" on my work Debian machine, I see:
gcc (Debian 7.3.0-5) 7.3.0
If Debian is okay naming their version of GCC "Debian 7.3.0-5", I'd think
they're okay releasing a patched Go package as v1.2.3-debian.1.
You even seem to suggest this later when talking about "mymodule(a)v1.2.2
release 1" and "mymodule(a)v1.2.2 release 2." Russ is just talking about a
different way of encoding those version strings.
because when you integrate
masses of third-party code you will eventually hit bugs in pretty
much
every component, so having to run patched code at every layer is the
norm not the exception.
The need to make changes to open source packages is familiar to Google.
E.g., Google's internal build system incorporates a lot of open source
packages, and many need to be modified to accommodate Google's internal
development idiosyncracies. Even the Go toolchain and standard library
themselves are patched internally.
I think it would help if you could highlight the specific technical hurdles
you're running into trying to bundle Go packages into RPMs. I expect the
mechanisms are in place to do what you need, but I can believe the tooling
could use improvements to handle Linux-distribution-scale integration
efforts better, and I think the Go project is interested in supporting
those efforts.
And you want us to renumber all this, and patch all the other module
files that use those with the new numbers, and redo it all every
time
something changes, all year round, just because the go command can’t
accept that the code state Fedora builds from, may differ from the one
observed on the Internet?
Really?
REALLY ?
Do you really REALLY think any sane integrator will play this dance long
instead of patching out notary checks from its go command, and be done
with it?
I would expect the amount of manual integration work needed would scale
with the amount of local changes required, not with the amount of
dependencies. E.g., if you have to modify a core Go module to work better
on Fedora, I would think you make that one change, and your build system
and tooling would handle automatically rebuilding dependencies as
appropriate. I wouldn't expect you to have to manually renumber/rename
every downstream dependency.
If you're finding that's not the case, please share the issues you're
running into so they can be discussed concretely and addressed.
The utopia is thinking that everything is released in a perfect state by
upstream, so changes downstream need not happen,
As pointed out above, Google and Go's build systems are not built for this
utopia, but for the real world where downstream changes are needed.
Finally, you included a detailed description of your package build system
(which sounds very similar to Google's internal build system), but didn't
seem to highlight how the Go module system or proposed Go notary system
cause problems.
E.g., you mention that in step A.3 that you download and install the Fedora
version of any package dependencies. I would think these packages can
contain any extra metadata for step C to be able to rename/renumber the
current package (if even necessary) without Internet access.
3:11 a.m.
New subject: [golang-dev] proposal: public module authentication with the Go notary
Last time this discussion came up in person, someone from the Go team
suggested replacing all the go.mod files with new ones that are
entirely based on package metadata -- i.e. declared dependencies,
their installed location etc.
This seems like a reasonable solution, and it would be a pity if the
notary made this impossible.
On Sat, Mar 9, 2019 at 3:01 AM 'Matthew Dempsky' via golang-dev
<golang-dev(a)googlegroups.com> wrote:
On Fri, Mar 8, 2019 at 4:52 PM 'Nicolas Mailhot' via golang-dev
<golang-dev(a)googlegroups.com> wrote:
>
> It would be nice if it where true. Unfortunately even a simple command
> like tell me what you know about the local code tree
> go list -json -mod=readonly ./...
>
> will abort if the CI/CD system cuts network access to make sure builds
> do not depend on Internet state.
The behavior you described sounds like an issue to me. However, I wasn't able to
easily reproduce it locally: I tried running "strace -f go list -json -mod=readonly
./... |& grep connect" under a few scenarios (e.g., missing go.sum, or incomplete
go.mod file), and none of them seemed to result in network access.
Can you provide more detailed reproduction steps?
> The no-modification no-internet CI/CD constrain was not taken into
> account into the original design. It was bolted on later, and the
> bolting is imperfect.
I don't think this is true. Google's internal build system has used this same
"no-modification no-internet" constraint for around a decade:
http://google-engtools.blogspot.com/2011/09/build-in-cloud-distributing-b...
https://mike-bland.com/2012/10/01/tools.html#blaze-forge-srcfs-objfs
Supporting this usage pattern has been and remains very important to Go.
> > I would have expected that if Fedora modified a library, they
> > would give it a different version number, so that for example
> > modifying v1.2.2 would produce v1.2.3-fedora.1
>
> And that's not the case neither for Fedora, nor RHEL, nor Debian, nor
> pretty much any large scale integrator,
When I run "gcc --version" on my work Debian machine, I see:
gcc (Debian 7.3.0-5) 7.3.0
If Debian is okay naming their version of GCC "Debian 7.3.0-5", I'd think
they're okay releasing a patched Go package as v1.2.3-debian.1.
You even seem to suggest this later when talking about "mymodule(a)v1.2.2 release
1" and "mymodule(a)v1.2.2 release 2." Russ is just talking about a different
way of encoding those version strings.
> because when you integrate
> masses of third-party code you will eventually hit bugs in pretty much
> every component, so having to run patched code at every layer is the
> norm not the exception.
The need to make changes to open source packages is familiar to Google. E.g.,
Google's internal build system incorporates a lot of open source packages, and many
need to be modified to accommodate Google's internal development idiosyncracies. Even
the Go toolchain and standard library themselves are patched internally.
I think it would help if you could highlight the specific technical hurdles you're
running into trying to bundle Go packages into RPMs. I expect the mechanisms are in place
to do what you need, but I can believe the tooling could use improvements to handle
Linux-distribution-scale integration efforts better, and I think the Go project is
interested in supporting those efforts.
> And you want us to renumber all this, and patch all the other module
> files that use those with the new numbers, and redo it all every time
> something changes, all year round, just because the go command can’t
> accept that the code state Fedora builds from, may differ from the one
> observed on the Internet?
>
> Really?
>
> REALLY ?
>
> Do you really REALLY think any sane integrator will play this dance long
> instead of patching out notary checks from its go command, and be done
> with it?
I would expect the amount of manual integration work needed would scale with the amount
of local changes required, not with the amount of dependencies. E.g., if you have to
modify a core Go module to work better on Fedora, I would think you make that one change,
and your build system and tooling would handle automatically rebuilding dependencies as
appropriate. I wouldn't expect you to have to manually renumber/rename every
downstream dependency.
If you're finding that's not the case, please share the issues you're running
into so they can be discussed concretely and addressed.
> The utopia is thinking that everything is released in a perfect state by
> upstream, so changes downstream need not happen,
As pointed out above, Google and Go's build systems are not built for this utopia,
but for the real world where downstream changes are needed.
Finally, you included a detailed description of your package build system (which sounds
very similar to Google's internal build system), but didn't seem to highlight how
the Go module system or proposed Go notary system cause problems.
E.g., you mention that in step A.3 that you download and install the Fedora version of
any package dependencies. I would think these packages can contain any extra metadata for
step C to be able to rename/renumber the current package (if even necessary) without
Internet access.
--
You received this message because you are subscribed to the Google Groups
"golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
golang-dev+unsubscribe(a)googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
Benny
9:29 a.m.
New subject: [golang-dev] proposal: public module authentication with the Go
notary
Le vendredi 08 mars 2019 à 18:01 -0800, Matthew Dempsky a écrit :
On Fri, Mar 8, 2019 at 4:52 PM 'Nicolas Mailhot' via
golang-dev <
golang-dev(a)googlegroups.com> wrote:
> It would be nice if it where true. Unfortunately even a simple
> command
> like tell me what you know about the local code tree
> go list -json -mod=readonly ./...
>
> will abort if the CI/CD system cuts network access to make sure
> builds
> do not depend on Internet state.
The behavior you described sounds like an issue to me. However, I
wasn't able to easily reproduce it locally: I tried running "strace -f
go list -json -mod=readonly ./... |& grep connect" under a few
scenarios (e.g., missing go.sum, or incomplete go.mod file), and none
of them seemed to result in network access.
Can you provide more detailed reproduction steps?
That's because upstream go workflows (from a distribution POW) cheat by
downloading blindly things from the internet before they have been
checked, filling the go cache silently with them, and grandfathering
directly downloaded third party code in a build as original source code.
We do not allow that. We make the promise to our users that we only make
available things, that we created from direct upstream source code, and
third party things, that already passed distribution QA checks in their
own separate build.
And that, only if this those party things were identified as necessarily
for the build¹ and installed (for go modules) in the GOPROXY directory
used for this specific build via system components.
At the very very first steps of a build recipe when we ask the copy of
the codebase being built "what are you, what are you composed of, what
to you need for building" there is literally only the codebase being
inspected and the go compiler available and GOPROXY is empty. Because we
need the answer to this question, to translate it into system component
ids, so the build container can be populated with those system
components, making the module files available inside GOPROXY.
But in module mode, go will assume everything is available all the time,
and if it is not available it can be downloaded directly from the
internet, so instead of answering the question from direct first-level
elements (the codebase and nothing else) it will try to work
transitively and abort because second level code is not available yet.
> > I would have expected that if Fedora modified a library,
they
> > would give it a different version number, so that for example
> > modifying v1.2.2 would produce v1.2.3-fedora.1
>
> And that's not the case neither for Fedora, nor RHEL, nor Debian,
> nor
> pretty much any large scale integrator,
When I run "gcc --version" on my work Debian machine, I see:
gcc (Debian 7.3.0-5) 7.3.0
And when you run
$ ls -l /usr/lib
you won't see any libfoo.so.x.y.z-debian or libfoo.so.x.y.z-5
Because the only libfoo.so.x.y.z allowed on system at any given time is
the one built from (debian-patched) foo x.y.z source code, no matter if
it's release 1 2 3 or 5 of the Debian build, no matter the level of
Debian patching and massaging each build carries.
So giving the underlying build command the "you're building 7.3.0 build
release 5 debian" info so it appears in version info of the binary is OK
and encouraged (we'll even pass it as ldflags info to make sure it is
recorded in reach binary).
But having the underlying build command try to discriminate between
different build releases, or the build releases or the third party
artefacts brought in the build root (like would happen if the build
release was exposed to the golang semver resolution engine) is not OK.
The underlying build command has not business doing any processing on
release info, it's system component metadata, the system component
engine will give it the release it needs to work with at any given time,
if the build command tries to second guess the system component engine
things start breaking fast.
If Debian is okay naming their version of GCC "Debian
7.3.0-5", I'd
think they're okay releasing a patched Go package as v1.2.3-debian.1.
You even seem to suggest this later when talking about "
mymodule(a)v1.2.2 release 1" and "mymodule(a)v1.2.2 release 2." Russ is
just talking about a different way of encoding those version strings.
> because when you integrate
> masses of third-party code you will eventually hit bugs in pretty
> much
> every component, so having to run patched code at every layer is the
> norm not the exception.
The need to make changes to open source packages is familiar to
Google. E.g., Google's internal build system incorporates a lot of
open source packages, and many need to be modified to accommodate
Google's internal development idiosyncracies. Even the Go toolchain
and standard library themselves are patched internally.
I think it would help if you could highlight the specific technical
hurdles you're running into trying to bundle Go packages into RPMs. I
expect the mechanisms are in place to do what you need, but I can
believe the tooling could use improvements to handle Linux-
distribution-scale integration efforts better, and I think the Go
project is interested in supporting those efforts.
Ok, thanks a lot, that would make things a lot simpler.
I'll list here the various commands we need implemented to plug go
modules in our system. Implemented means by upstream go or ourselves,
though we obviously would prefer it to be upstream go side, because that
ensures they are not broken in the future by upstream go tooling
changes, that other distros use the same commands as us, that we can
limit divergence and share the QA and fixing burden.
I believe that the commands needed apt-side for Debian and Ubuntu, are
pretty much the same, because apt/deb and dnf/yum/rpm have been on a
convergent evolution track for a long time, and at this point their
high-level architecture is the same, only implementation details differ
(we customarily scrouge Debian patches and fixes and they scrouge ours,
and we metoo each other's problem reports in upstream issue trackers).
Like I already wrote, our build process starts with a bare raw unpacked
copy of upstream sources in a specific directory. This copy will have
already been modified to remove things we do not like and patch out
problems². Obviously the patching out is an iterative process that's why
each release of a build may carry slightly different patches.
This copy typically does not include any VCS info³. Since go expects to
find the module version information there, and does not record it in the
project go.mod like other languages, life already sucks for us because
we need to carry a variable with this info and pass it manually to later
commands.
[available resources:
— minimal system,
— go compiler,
— empty system GOPROXY,
— prepared unpacked project sources in a specific directory]
A. We need a command to identify all the go module trees present in the
source directory. A variation of
$ find . -type f -name go.mod
would probably work, but that lacks all the sanity checks the upstream
go may want to apply now or in the future, to make sure a "go.mod" file
is a legitimate go module descriptor.
[available resources:
— minimal system,
— go compiler,
— empty system GOPROXY,
— prepared unpacked project sources in a specific directory]
B. We need another command (or the same) that lists:
1. the go module name of each of those trees,
2. their consolidated first-level requirements (not the list of their
individual first-level requirements)
Probably not both at once, or if both in a structured format we can
reprocess. The most convenient command is
<command> <directory> --provides → one module name per line
<command> <directory> --requires → one constrain
(module + associated version constrain) per line
The --requires command needs to filter out the modules names already
present in the source tree, because in a multi-module X project that
contains X/foo and X/bar modules, we want to build X/foo against the
local version of X/bar, not some other one
(and we will have some auditing work on the --provides output to make
sure every provides line actually belongs to the project being built and
is not a copy of someone else's code)
I lack the necessary perspective today to say how replaces should be
handled.
Local replaces clearly need ignoring, if some code needs the X third-
party module, we want it to use our curated X module, not some project-
local directory containing a fork of X.
Non-local replaces? That's less clear to me.
If a non local replace of B by C in module A means that every downstream
user of module A will then use C instead of B when processing A code,
that's fine with us, the fact B exists is an internal go compiler
information, we only need to know C is needed in the A build container
and in any container that uses the A module.
But, if a non local replace of B by C in module A only affects direct A
builds, and indirect A builds will still use B, we'll just cry and yell
and curse in a quiet corner, and then either remove the replacing in the
A module file, or make it transitive by patching our version of A code
imports statements to use C everywhere instead of A.
The result of the listing command will then be translated in rpm system
identifiers, and used to populate the build GOPROXY.
Translation basically means:
1. applying some form of namespacing:
“I need the go module named X” becomes “I need golang-module(X)”
(sadly the golang() namespace is already taken by GOPATH sources, and
can not be shared because go keeps GOPATH and module mode separate)
2. translating the golang version constrain syntax in rpm version
constrain syntax. That's pretty easy for semver:
“I need X(a)v1.2.xn--3-rhn becomes
“I need golang-module(X) ≥ 1.2.3 and < 2”
For non-semver constrains like a specific hash-identified snapshot
that's less nice, we do not allow hashes as valid version IDs rpm
side (because hash ordering can not be deduced from the hashes
themselves) so anything that absolutely require a specific snapshot
of X will be translated into “I need golang-module(X)(commit=hash)”
and golang-module(X)(commit=hash) is a separate ID from
golang-module(X)
Because dependency cycles exist in the real-world, and because we are
extra-careful to build only from things we already vetted, there will be
cases where we won't be able to satisfy upstream project requirements
If A needs B that needs C that needs A we will have build one of A B C
in reduced more at one point to break the cycle (we call that
boostrapping). No idea if it's better to inform the listing command that
one of the module needs it reports will be ignored, or if it's better to
ignore it silently.
[available resources:
— minimal system,
— go compiler,
— system GOPROXY populated with distro-produced go modules corresponding
to identified project requirements,
— prepared unpacked project sources in a specific director]
C. we need a go mod pack command that creates packed module files from
the unpacked project sources into a staging GOPROXY directory.
We can pass it the project version as argument (since it can not read it
in the upstream go.mod files).
Since VCS info is not available, the time component of the info file can
not be populated with it. And, we definitely do not want the command to
record the current time (because then the file content would depend on
when it was built, and we have processes that check that arch-agnostic
files files produced by different architecture builders are bit-for-bit
identical, and we have security auditors that replay builds later in
their own system and are alarmed when the replay produces different
results). And, frankly, the time info in VCSes is messy and unreliable,
and mtimes can not be used (the source preparation process can apply
patches, and that will change mtimes, even if you apply the very same
patch in two different runs).
Thus, we'd prefer this time info not to exist in our own info files, or
be set to zero, or, if go really needs it, to pass it as parameter to go
mod pack, (but that's more manual work our side to decide what to pass).
This command needs to operate in bulk on all the module trees previously
identified (so if it found 3 module trees, create 3 packed modules in
the staging GOPROXY), or a subset (explicitly provided module list).
This command needs to output the list of created module files (the zip
ziphash info mod files). The list file is a special case, we would
consider its file enveloppe as belonging to each of the versions of a
module, and the file content belonging to no one in particular (ghost
files in rpm tech). So it can be listed or not either way we will apply
special processing to it. Probably, better to list it for cleanliness.
The "files created" list can be either outputted to stdout (in which
case it outputs nothing else to stdout) or to a list file specified as
argument. No particular preference, we can handle both.
This list is used to ventilate the produced files in specific system
components: upstream project A may contain sources for A/B/C A/B/D A/E
modules, and we can choose to put all of those in a single system
component, or separate in two A/B/C+A/B/D and A/E, or go full granular
with three system components. The level of splitting depends on the
consequences in the system component dependency graph.
This go mod pack will need to be more discriminating than just “bulk
pack every file under the go.mod root directory, .gitignore included”
like I see go is doing right now.
If a project includes font files (for example the go font), we'll want
to expose those in /usr/share/fonts so they are not restricted to the go
compiler. If a project includes documentation files we will deploy them
in /usr/share/doc not in the module zip file. And so on (protobuf files
come to the mind)
So, typically, we'd want go mod pack to pack *only* the module tree Go
source code (things that can be used by the go compiler) ideally only
the source code that can be used on our platform (GOOS=linux), and vet
the packing of any other file.
So a basic go mod pack invocation only packs go source files and
testdata.
You have an info or dry run mode that says "default packing ignored/will
ignore all those files in the origin tree"
You can pass lists of regexes to the go mod pack command to either
include other non source code files, or to exclude already selected
files from the packing (--include regex and --exclude regex flags).
The regexes are obviously very module specific, depend on the module
resource needs and the distribution packaging policies. The generic go
tooling need not worry about those policies, just apply the produced
regexes.
Ideally, we could pass the go mod pack a --without module[@version]
argument, that causes it to not pack any the code that requires
module[@version]. That's necessary while bootstrapping and to cull very
expensive unit or integration tests that require hundreds of third-party
modules to run. Otherwise we can approximate it via exclude (but that
will require more Fedora manual work to identify the required excludes).
Please understand, we are not doing all this filtering and culling in
opposition to upstreams Go projects. We have no special wish to deviate
from them. Our ideal world is usptream code releases that can be used
directly as-is without any patching or filtering.
But, a lot of upstream projects lack discipline.
They will include files they have no legal right to distribute (and we
will remove those before step A).
They will include files that can be distributed, but not modified (and
entities like Fedora and Debian will think very hard if they really want
to relay those, because they are contrary to our free software policies,
and our users like to know that everything we ship can be modified
without legal problems).
They will reference third-party modules with incompatible (or completely
missing) licensing.
They will ship integration code that makes no sense outside of their own
information system.
They will ship for years broken and bitrotten project/test/example
code.
For them, continuing to include this code within their project is free.
their go get will happily download hundreds of unveted and unnecessary
Go modules from the internet. No one is likely to sue them for small
legal mistakes.
But shipping this code is *not* free for us. A distribution is big
enough it can be sued. Any failing unit test costs human time to check
it is an harmless failure. Any module dep pulled by this unnecessary
code is yet another software project that needs to be checked and
audited and integrated and maintained at the system component level
before it is available in the CI system.
So, we need to aggressively cull anything not necessary, to keep the our
integration costs and risks down.
And we really really would appreciate if project unit tests (+ testdata)
and project production code were split in separate zip files, with
separate build requirements, so the unit test dependency costs were
optional (we'd typically pay the project A unit test costs when
integrating project A, not when integrating something that uses project
A).
For all those reasons, the go modules files produced at this stage will
definitely *not* match the hashes produced by the notary, even if we do
not change a sigle line of go code.
[available resources:
– minimal system,
– go compiler,
– system GOPROXY populated with distro-produced go modules corresponding
to identified project requirements
– prepared unpacked project sources in a specific directory,
— staging GOPROXY containing candidate go modules corresponding to the
built project]
D. we need a command to build the project binaries from the files in the
system GOPROXY and the unpacked project sources,
OR from the system GOPROXY and the staging GOPROXY contents
(ideally, the second option, to make sure the candidate module files in
the staging directory are complete)
After this step we've hopefully finished producing files from the
prepared unpacked project sources. At this stage rpm deploys every
produced candidate file in a location that mirrors the target deployment
paths, under a specific prefix
So:
– a file in /usr/bin is a binary installed from existing system
components
– a file in /prefix/usr/bin is a candidate binary produced from the
ongoing build
— a file in GOPROXY is a Go module installed from existing system
components
— a file in /prefix/GOPROXY is a candidate Go module file produced from
the ongoing build
[available resources:
– minimal system,
– go compiler,
– system GOPROXY populated with distro-produced go modules corresponding
to identified project requirements
– prepared unpacked project sources in a specific directory,
— candidate tree under /prefix containing all the files that will end up
in new system modules. Some of those will replace existing files in /.
]
E. we need a command to run the unit tests of every produced go proxy
module (in the staging GOPROXY). Any failure (returning error code)
aborts the build process.
We'd really like this command to only take into account the candidate
tree, and the existing / tree (for files that do not belong to the
target system components).
[available resources:
– minimal system,
– go compiler,
– system GOPROXY populated with distro-produced go modules corresponding
to identified project requirements
– prepared unpacked project sources in a specific directory,
— candidate tree under /prefix containing all the files that will end up
in new system modules. Some of those will replace existing files in /.
]
F. At this point rpm starts ventilating the files contained in the
staging tree in new system components. Therefore, it needs to compute
the metadata of each of those system components
So, we have a new run of "what are you and what do you need" command,
this time on every mod/zip/zipphash/info fileset in /prefix/GOPROXY,
instead of the content of the prepared unpacked project sources. So,
probably some variation of the command in B.
Practically, those inspection runs can not be triggered by complex
objects like filesets, only individual files, so we'll probably trigger
them on the mod files, and assume that is the mod file is present, the
rest is likely to be here too.
rpm keeps track or where each file will end up, and attributes the
result of the query to the corresponding system component (it does not
tell the queried fileset where it will end up)
So we now have brand-new shiny system components that contain clean
audited GOPROXY module filesets, and declare they provide
golang-module(foo) = x.y.z, and that they need
golang-module(bar) ≥ a.b.c and < a+1
Installation of one of those system components will make the module
fileset exist in GOPROXY
But, that is not sufficient for go, because of the list files.
So, we need a final command, that rpm can invoque when it adds or
removes files in GOPROXY, to compute the list version indexes.
It could be approximated by a simple stupid shell script, but the go mod
cache code upstream performs more sanity checks than that, and we'd like
those sanity checks to be replicated in the "update list files" command.
And, that's pretty much all. We create clean new module files, we put
them in system components, installation of those components causes the
module files to exist in GOPROXY, and we'd like for the compiler to make
use of those module files without bothering us with notary checks (that
will fail pretty much all the time,due to how the whole integration
process is structured). The system components are digitally signed but
the build process has no access to the signing key (that is done on a
separate system with an HSM for security reasons) so the build process
can not generate detached signature files.
I would expect the amount of manual integration work needed would
scale with the amount of local changes required, not with the amount
of dependencies. E.g., if you have to modify a core Go module to work
better on Fedora, I would think you make that one change, and your
build system and tooling would handle automatically rebuilding
dependencies as appropriate.
Yes that's how it works, but only because the underlying build commands
are not supposed to discriminate between fedora build ids (releases)
We build
mymodule(a)v1.2.2 with some level of fixing of patching (release 1). That
creates a mod/zip/zipphash/info fileset
We build all the things that need mymodule(a)v1.2.2
Some time later a new issue causes us to adjust the fixing done for
mymodule(a)v1.2.2
So we build mymodule(a)v1.2.2 release 2 with the new fixes. For a lot of
languages that would be enough, and produce a new dll, transparently
used by all downstream users.
With Go static building that creates a new
mod/zip/zipphash/info fileset for mymodule(a)v1.2.2 (with new hashes)
and we have to perform the additional step of rebuild everything that
uses mymodule(a)v1.2.2, recursively.
We don't want the compiler to complain about the hash change or to
compare it to some internet notary.
We can certainly pass the release id info to the pack command in C so it
is recorded somewhere in the produced mod/zip/zipphash/info fileset and
each release fileset is unique.
That is, as long as it is only recorded for information purposes, and
the go command does not try to treat the result as different from
mymodule(a)v1.2.2, to discriminate between releases, or request a specific
release.
Best regards,
¹ Because first, sometimes the on-disk representation of two components
will clash. That's not supposed to happen with Go modules but our system
is not Go specific, so it was not specced around Go module
particularities, and who knows at this point if the module no-clash
design will be robust WRT the weird things upstreams like to invent in
the real word.
And second, we want to be sure that what we know about component
relationships is the truth, so security teams can rely on the system
component dependency graph when analysing incidents. The best way to
make sure it is the truth is to disallow anything not identified in the
CI container.
² (vendored copies of third party code, things we can not distribute
legaly like for example arial.ttf when upstream needs a "free" font and
does not understand copyright law, etc).
³ For historical reasons, because out CI infrastructure design antedates
modern VCSes, and because we absolutely do not want the build commands
to start pulling things from VCS history that do not match the version
we're attempting to build
--
Nicolas Mailhot
1:55 p.m.
New subject: [golang-dev] proposal: public module authentication with the Go notary
Quoting 'Matthew Dempsky' via golang-dev (2019-03-08 21:01:32)
Can you provide more detailed reproduction steps?
Here's a reproducer:
$ go mod init example.org/my-project
go: creating new go.mod: module example.org/my-project
$ cat >> main.go << "EOF"
package main
import _ "example.com/other-module"
func main() {}
EOF
$ go list -mod=readonly ./...
build example.org/my-project: cannot load example.com/other-module:
import lookup disabled by -mod=readonly
I think we'd both mis-understood Nicolas's complaint the first time --
it's not that it still tries to access the network with -mod=readonly,
but that even for basic commands for which a read-only mod file/no
network access really *shouldn't* be a problem, it will still just bomb
out.
I'm not familiar with the code, but this sounds like early in the common
program logic there's a check as to whether any of the module data needs
updating, then abort if it can't be done -- before even looking at what
the user has asked us to do.
I agree this is kinda sad; it seems like it would be better if the go
tool still tried to fulfill the user's request to the extent possible
when -mod=readonly is set.
-Ian
9:43 a.m.
New subject: [golang-dev] proposal: public module authentication with the Go notary
On Sat, Mar 9, 2019 at 2:55 PM Ian Denhardt <ian(a)zenhack.net> wrote:
Here's a reproducer:
> $ go mod init example.org/my-project
> go: creating new go.mod: module example.org/my-project
> $ cat >> main.go << "EOF"
> package main
> import _ "example.com/other-module"
> func main() {}
> EOF
> $ go list -mod=readonly ./...
> build example.org/my-project: cannot load example.com/other-module:
> import lookup disabled by -mod=readonly
I think we'd both mis-understood Nicolas's complaint the first time --
it's not that it still tries to access the network with -mod=readonly,
but that even for basic commands for which a read-only mod file/no
network access really *shouldn't* be a problem, it will still just bomb
out.
The go command is "bombing out" here because the code in question
is trying to import "example.com/other-module", yet no dependency listed
in go.mod provides that package (there are no dependencies in go.mod at
all!).
[One of the things go list computes is the dependency graph
(visible if you use go list -json or go list -f '{{.Deps}}').]
It's true that if you have a CI/CD system, then for a successful build
you need to push complete code to it, and that includes a go.mod with
listed dependencies sufficient to cover the relevant imports,
as I described in my original reply to Nicolas. If go.mod were up to date
(for example, if 'go mod tidy' found no work to do) and the local
Go module cache already contained the listed dependencies,
then the 'go list -mod=readonly' command you showed would complete
just fine. You can't push code with an incomplete dependency list to the
CI/CD system and expect it to build, any more than you can push
code with missing function definitions and expect it to build.
Best,
Russ
10:01 a.m.
New subject: [golang-dev] proposal: public module authentication with the Go
notary
Le 2019-03-11 15:43, Russ Cox a écrit :
On Sat, Mar 9, 2019 at 2:55 PM Ian Denhardt <ian(a)zenhack.net>
wrote:
> Here's a reproducer:
>
>> $ go mod init example.org/my-project [1]
>> go: creating new go.mod: module example.org/my-project [1]
>> $ cat >> main.go << "EOF"
>> package main
>> import _ "example.com/other-module [2]"
>> func main() {}
>> EOF
>> $ go list -mod=readonly ./...
>> build example.org/my-project [1]: cannot load
> example.com/other-module [2]:
>> import lookup disabled by -mod=readonly
>
> I think we'd both mis-understood Nicolas's complaint the first time
> --
> it's not that it still tries to access the network with
> -mod=readonly,
> but that even for basic commands for which a read-only mod file/no
> network access really *shouldn't* be a problem, it will still just
> bomb
> out.
The go command is "bombing out" here because the code in question
is trying to import "example.com/other-module [2]", yet no dependency
listed
in go.mod provides that package (there are no dependencies in go.mod
at all!).
[One of the things go list computes is the dependency graph
(visible if you use go list -json or go list -f '{{.Deps}}').]
It's true that if you have a CI/CD system, then for a successful build
you need to push complete code to it, and that includes a go.mod with
listed dependencies sufficient to cover the relevant imports,
as I described in my original reply to Nicolas. If go.mod were up to
date
(for example, if 'go mod tidy' found no work to do) and the local
Go module cache already contained the listed dependencies,
then the 'go list -mod=readonly' command you showed would complete
just fine. You can't push code with an incomplete dependency list to
the
CI/CD system and expect it to build, any more than you can push
code with missing function definitions and expect it to build.
And as I have explained in the detailed description Matthew requested,
our construction of the CI/CD environment is incremental, so the
assumption in the go tool code that "everything is there and it it is
not it can be downloaded directly" does not work for us
Regards,
--
Nicolas Mailhot
12:54 p.m.
New subject: [golang-dev] proposal: public module authentication with the Go notary
On Mon, Mar 11, 2019 at 8:01 AM Nicolas Mailhot <nicolas.mailhot(a)laposte.net>
wrote:
And as I have explained in the detailed description Matthew
requested,
To be clear, I was asking for details to reproduce the technical issues
you're running into. I may have missed them, but I don't believe you've
provided these.
our construction of the CI/CD environment is incremental, so the
assumption in the go tool code that "everything is there and it
it is
not it can be downloaded directly" does not work for us
If your build system provides the Go source/packages, then Go won't try to
download them directly itself. This is how Go works within Google's build
system (which, again, has the same no-network-access limitation as the
build system you're describing, and yet supports building Go
programs/packages).
It sounds like you're having trouble with providing the Go source/packages
in the format expected by cmd/go. If you would provide reproduction steps
of what you're doing and the problems you're running into, we can help
advise what changes you need to make, and maybe identify tooling
improvements to make that easier.
1:09 p.m.
New subject: [golang-dev] proposal: public module authentication with the Go notary
* via golang-dev:
If your build system provides the Go source/packages, then Go
won't
try to download them directly itself. This is how Go works within
Google's build system (which, again, has the same no-network-access
limitation as the build system you're describing, and yet supports
building Go programs/packages).
How do you plan to bypass the notary requirement? Do you have a
non-Internet connection to it?
Or do you pre-populate the expected data before starting the builds,
similar to OCSP stapling?
Thanks,
Florian
1:16 p.m.
New subject: [golang-dev] proposal: public module authentication with the Go notary
On Mon, Mar 11, 2019 at 11:09 AM Florian Weimer <fweimer(a)redhat.com> wrote:
How do you plan to bypass the notary requirement?
I don't work on the Go-inside-Google integration work, so someone else may
have to weigh in if it's appropriate to share those plans. My point with
mentioning Google's hermetic build system is more to emphasize that the Go
developers are very familiar with that build model and care about keeping
it working. The claims that the Go module system is being built without
consideration of those requirements seem very mistaken.
That said, the notary is only involved when adding new lines to the go.sum
file to handle adding or updating dependencies. There's no requirement to
contact the Go notary when the go.sum file is already complete.
3:17 a.m.
New subject: [golang-dev] proposal: public module authentication with the Go
notary
Le 2019-03-11 19:15, Matthew Dempsky a écrit :
On Mon, Mar 11, 2019 at 11:09 AM Florian Weimer
<fweimer(a)redhat.com>
wrote:
> How do you plan to bypass the notary requirement?
I don't work on the Go-inside-Google integration work, so someone else
may have to weigh in if it's appropriate to share those plans. My
point with mentioning Google's hermetic build system is more to
emphasize that the Go developers are very familiar with that build
model and care about keeping it working. The claims that the Go module
system is being built without consideration of those requirements seem
very mistaken.
That said, the notary is only involved when adding new lines to the
go.sum file to handle adding or updating dependencies. There's no
requirement to contact the Go notary when the go.sum file is already
complete.
Which we be pretty much the case all the time for us, since we build
against out own curated set of modules, not the ones upstream found on
the Internet.
In fact, I'm pretty sure we will start each build by removing the
upstream go.sum file altogether.
Regards,
--
Nicolas Mailhot
1:21 p.m.
New subject: [golang-dev] proposal: public module authentication with the Go notary
Quoting Matthew Dempsky (2019-03-11 13:54:07)
On Mon, Mar 11, 2019 at 8:01 AM Nicolas Mailhot
<[1]nicolas.mailhot(a)laposte.net> wrote:
And as I have explained in the detailed description Matthew
requested,
To be clear, I was asking for details to reproduce the technical issues
you're running into. I may have missed them, but I don't believe you've
provided these.
our construction of the CI/CD environment is incremental, so the
assumption in the go tool code that "everything is there and it it
is
not it can be downloaded directly" does not work for us
If your build system provides the Go source/packages, then Go won't try
to download them directly itself. This is how Go works within Google's
build system (which, again, has the same no-network-access limitation
as the build system you're describing, and yet supports building Go
programs/packages).
It sounds like you're having trouble with providing the Go
source/packages in the format expected by cmd/go. If you would provide
reproduction steps of what you're doing and the problems you're running
into, we can help advise what changes you need to make, and maybe
identify tooling improvements to make that easier.
In a parallel thread, a Nix developer was asking about basically the
same use case, and was pointed at `go build -mod=vendor`. It seems like
this does exactly what is wanted here -- just use the code we have
locally. Nicolas, does that address your use case?
-Ian
3:22 a.m.
New subject: [golang-dev] proposal: public module authentication with the Go
notary
Le 2019-03-11 19:18, Ian Denhardt a écrit :
Quoting Matthew Dempsky (2019-03-11 13:54:07)
> On Mon, Mar 11, 2019 at 8:01 AM Nicolas Mailhot
> <[1]nicolas.mailhot(a)laposte.net> wrote:
>
> And as I have explained in the detailed description Matthew
> requested,
>
> To be clear, I was asking for details to reproduce the technical
> issues
> you're running into. I may have missed them, but I don't believe
> you've
> provided these.
>
> our construction of the CI/CD environment is incremental, so the
> assumption in the go tool code that "everything is there and it
> it
> is
> not it can be downloaded directly" does not work for us
>
> If your build system provides the Go source/packages, then Go won't
> try
> to download them directly itself. This is how Go works within
> Google's
> build system (which, again, has the same no-network-access
> limitation
> as the build system you're describing, and yet supports building Go
> programs/packages).
> It sounds like you're having trouble with providing the Go
> source/packages in the format expected by cmd/go. If you would
> provide
> reproduction steps of what you're doing and the problems you're
> running
> into, we can help advise what changes you need to make, and maybe
> identify tooling improvements to make that easier.
In a parallel thread, a Nix developer was asking about basically the
same use case, and was pointed at `go build -mod=vendor`. It seems like
this does exactly what is wanted here -- just use the code we have
locally. Nicolas, does that address your use case?
That's definitely *not* what we want (our processes call for removing
vendor and its equivalents in other languages before going to the build
step). We do *not* like vendor. We do *not* like GOPATH. Modules are
good. As long as we can create and use *our* modules, not someone else's
idea of how a module content should look like. Our modules are basically
the same thing as upstream modules (API compatible) with lots of warts
removed.
As I already noted, there is a deep lack of understanding Go upstream
side of how we work (for all computer system languages, not just for
Go). What they imagine is not what we need or want.
Regards,
--
Nicolas Mailhot
9:27 p.m.
New subject: [golang-dev] proposal: public module authentication with the Go notary
Quoting Nicolas Mailhot (2019-03-12 04:22:45)
> In a parallel thread, a Nix developer was asking about basically
the
> same use case, and was pointed at `go build -mod=vendor`. It seems like
> this does exactly what is wanted here -- just use the code we have
> locally. Nicolas, does that address your use case?
That's definitely *not* what we want (our processes call for removing
vendor and its equivalents in other languages before going to the
build step).
Not sure we're on the same page re: what was suggested. The idea is
you'd point the vendor directory at something that contains the distro's
versions of the dependencies, and use -mod=vendor to bypass all of the
smarts around fetching modules and such. I was not proposing using an
existing vendor directory provided by upstream or anything like that.
This is more or less what Jakub described in a sibling comment, as far
as I can tell.
As I already noted, there is a deep lack of understanding Go
upstream
side of how we work (for all computer system languages, not just for
Go). What they imagine is not what we need or want.
This definitely jives with my own memories from the last time I was a
maintainer on a distro -- language package managers and module systems
generally tend to be much more oriented towards developers using the
modules than to distro maintainers packaging them. It's to the point
where as an end-user of a distro I have a boat load of executables in
places like ~/.local/bin, because I use enough oddball tools that aren't
popular enough to be maintained by the distros themselves, and packaging
them myself just isn't worth the trouble. I do wish these tools played
more nicely together.
3:14 a.m.
New subject: [golang-dev] proposal: public module authentication with the Go
notary
Le 2019-03-13 03:24, Ian Denhardt a écrit :
Quoting Nicolas Mailhot (2019-03-12 04:22:45)
> > In a parallel thread, a Nix developer was asking about basically the
> > same use case, and was pointed at `go build -mod=vendor`. It seems like
> > this does exactly what is wanted here -- just use the code we have
> > locally. Nicolas, does that address your use case?
>
> That's definitely *not* what we want (our processes call for removing
> vendor and its equivalents in other languages before going to the
> build step).
Not sure we're on the same page re: what was suggested. The idea is
you'd point the vendor directory at something that contains the
distro's
versions of the dependencies, and use -mod=vendor to bypass all of the
smarts around fetching modules and such.
But, again, I don't *want* a vendor directory, distro or otherwise.
vendor (and GOPATH) have always been a major PITA to assemble and
maintain (and I speak as the person, that wrote at least half of the
code that assembles and maintains them Fedora-side).
Ideally, I'd like proper shared libs, because rebuilding every dependent
on changes is an obstacle to robust security (you forget one rebuild and
poof, you're owned).
Baring that, I'd settle for a directory of modules, which was what
GOPROXY was supposed to deliver, before Go upstream embarked in its mad
crusade to squeeze out anything between dev and production.
--
Nicolas Mailhot
4:07 a.m.
New subject: [golang-dev] proposal: public module authentication with the Go notary
On Wed, Mar 13, 2019, 09:14 Nicolas Mailhot <nicolas.mailhot(a)laposte.net>
wrote:
Le 2019-03-13 03:24, Ian Denhardt a écrit :
> Quoting Nicolas Mailhot (2019-03-12 04:22:45)
>
>> > In a parallel thread, a Nix developer was asking about basically the
>> > same use case, and was pointed at `go build -mod=vendor`. It seems
like
>> > this does exactly what is wanted here -- just use the code we have
>> > locally. Nicolas, does that address your use case?
>>
>> That's definitely *not* what we want (our processes call for removing
>> vendor and its equivalents in other languages before going to the
>> build step).
>
> Not sure we're on the same page re: what was suggested. The idea is
> you'd point the vendor directory at something that contains the
> distro's
> versions of the dependencies, and use -mod=vendor to bypass all of the
> smarts around fetching modules and such.
But, again, I don't *want* a vendor directory, distro or otherwise.
vendor (and GOPATH) have always been a major PITA to assemble and
maintain (and I speak as the person, that wrote at least half of the
code that assembles and maintains them Fedora-side).
Ideally, I'd like proper shared libs, because rebuilding every dependent
on changes is an obstacle to robust security (you forget one rebuild and
poof, you're owned).
Go has supported -buildmode=shared on all major architectures for some
time. I'm curious why nobody uses that yet.
Fabio
Baring that, I'd settle for a directory of modules, which was
what
GOPROXY was supposed to deliver, before Go upstream embarked in its mad
crusade to squeeze out anything between dev and production.
--
Nicolas Mailhot
_______________________________________________
golang mailing list -- golang(a)lists.fedoraproject.org
To unsubscribe send an email to golang-leave(a)lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/golang@lists.fedoraproject.org
4:40 a.m.
New subject: [golang-dev] proposal: public module authentication with the Go
notary
----- Original Message -----
From: "Fabio Valentini" <decathorpe(a)gmail.com>
To: golang(a)lists.fedoraproject.org
Sent: Wednesday, March 13, 2019 10:07:35 AM
Subject: Re: [golang-dev] proposal: public module authentication with the Go notary
On Wed, Mar 13, 2019, 09:14 Nicolas Mailhot < nicolas.mailhot(a)laposte.net >
wrote:
Le 2019-03-13 03:24, Ian Denhardt a écrit :
> Quoting Nicolas Mailhot (2019-03-12 04:22:45)
>
>> > In a parallel thread, a Nix developer was asking about basically the
>> > same use case, and was pointed at `go build -mod=vendor`. It seems like
>> > this does exactly what is wanted here -- just use the code we have
>> > locally. Nicolas, does that address your use case?
>>
>> That's definitely *not* what we want (our processes call for removing
>> vendor and its equivalents in other languages before going to the
>> build step).
>
> Not sure we're on the same page re: what was suggested. The idea is
> you'd point the vendor directory at something that contains the
> distro's
> versions of the dependencies, and use -mod=vendor to bypass all of the
> smarts around fetching modules and such.
But, again, I don't *want* a vendor directory, distro or otherwise.
vendor (and GOPATH) have always been a major PITA to assemble and
maintain (and I speak as the person, that wrote at least half of the
code that assembles and maintains them Fedora-side).
Ideally, I'd like proper shared libs, because rebuilding every dependent
on changes is an obstacle to robust security (you forget one rebuild and
poof, you're owned).
Go has supported -buildmode=shared on all major architectures for some time.
I'm curious why nobody uses that yet.
Fabio
From the Fedora perspective it is due to the stability of the upstream code(API changes)
there is nearly no effective difference compared to the non shared(you will have to
rebuild "everything" anyway). I have been toying for a long time with switching
at least sdtlib to be dynamically linked in the BR, but haven't got around to push it
out(i.e. coerce maintainers to BR it, IMHO we have bigger nuts to crack atm). Other thing
is that ABI for the dynamic libs is not guaranteed between the Go releases, on other hand
with my experiments I haven't seen breaking breakages, but it is not guaranteed. I
might have been just lucky with my fairly small and simple test cases.
On practical note, nobody is barring you from BR-ing golang-shared(it currently exists on
all arches) and actually start building with dynamically linked stdlib. With current
pre-Go-module environment it should just work, you don't even need to explicitly
require it at runtime. RPM magic will require it automatically.
I guess from the regular Go user point of view is that static linking(of Go code bits) is
still the new and shining(hey I don't need any runtime deps, apart from libc), I
guess. And nobody really rediscovered all the pitfalls in production.
JC
>
>
>
> Baring that, I'd settle for a directory of modules, which was what
> GOPROXY was supposed to deliver, before Go upstream embarked in its mad
> crusade to squeeze out anything between dev and production.
>
> --
> Nicolas Mailhot
> _______________________________________________
> golang mailing list -- golang(a)lists.fedoraproject.org
> To unsubscribe send an email to golang-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/golang@lists.fedoraproject.org
>
> _______________________________________________
> golang mailing list -- golang(a)lists.fedoraproject.org
> To unsubscribe send an email to golang-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/golang@lists.fedoraproject.org
>
4:58 a.m.
New subject: [golang-dev] proposal: public module authentication with the Go notary
* Jakub Cajka:
From the Fedora perspective it is due to the stability of the
upstream
code(API changes) there is nearly no effective difference compared to
the non shared(you will have to rebuild "everything" anyway). I have
been toying for a long time with switching at least sdtlib to be
dynamically linked in the BR, but haven't got around to push it
out(i.e. coerce maintainers to BR it, IMHO we have bigger nuts to
crack atm).
I don't think linking the run time dynamically is feasible.
The Go 1 compatibility strategy does not mention not adding private
fields to structs as a breaking change, for example:
<https://golang.org/doc/go1compat>
But each time you allocate a struct, the caller inlines the size, so
adding private fields is very much a breaking change.
Thanks,
Florian
5 a.m.
New subject: [golang-dev] proposal: public module authentication with the Go
notary
----- Original Message -----
From: "Florian Weimer" <fweimer(a)redhat.com>
To: "Jakub Cajka" <jcajka(a)redhat.com>
Cc: golang(a)lists.fedoraproject.org
Sent: Wednesday, March 13, 2019 10:58:16 AM
Subject: Re: [golang-dev] proposal: public module authentication with the Go notary
* Jakub Cajka:
> From the Fedora perspective it is due to the stability of the upstream
> code(API changes) there is nearly no effective difference compared to
> the non shared(you will have to rebuild "everything" anyway). I have
> been toying for a long time with switching at least sdtlib to be
> dynamically linked in the BR, but haven't got around to push it
> out(i.e. coerce maintainers to BR it, IMHO we have bigger nuts to
> crack atm).
I don't think linking the run time dynamically is feasible.
The Go 1 compatibility strategy does not mention not adding private
fields to structs as a breaking change, for example:
<https://golang.org/doc/go1compat>
But each time you allocate a struct, the caller inlines the size, so
adding private fields is very much a breaking change.
Thanks,
Florian
This is not(shouldn't be) a runtime, just the standard library.
JC
> _______________________________________________
> golang mailing list -- golang(a)lists.fedoraproject.org
> To unsubscribe send an email to golang-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/golang@lists.fedoraproject.org
>
5:44 a.m.
New subject: [golang-dev] proposal: public module authentication with the Go notary
* Jakub Cajka:
----- Original Message -----
> From: "Florian Weimer" <fweimer(a)redhat.com>
> To: "Jakub Cajka" <jcajka(a)redhat.com>
> Cc: golang(a)lists.fedoraproject.org
> Sent: Wednesday, March 13, 2019 10:58:16 AM
> Subject: Re: [golang-dev] proposal: public module authentication with the Go notary
>
> * Jakub Cajka:
>
> > From the Fedora perspective it is due to the stability of the upstream
> > code(API changes) there is nearly no effective difference compared to
> > the non shared(you will have to rebuild "everything" anyway). I have
> > been toying for a long time with switching at least sdtlib to be
> > dynamically linked in the BR, but haven't got around to push it
> > out(i.e. coerce maintainers to BR it, IMHO we have bigger nuts to
> > crack atm).
>
> I don't think linking the run time dynamically is feasible.
>
> The Go 1 compatibility strategy does not mention not adding private
> fields to structs as a breaking change, for example:
>
> <https://golang.org/doc/go1compat>
>
> But each time you allocate a struct, the caller inlines the size, so
> adding private fields is very much a breaking change.
>
> Thanks,
> Florian
This is not(shouldn't be) a runtime, just the standard library.
Uhm, what's the difference between the two? There is no separate
run-time library.
Thanks,
Florian
6:38 a.m.
New subject: [golang-dev] proposal: public module authentication with the Go
notary
----- Original Message -----
From: "Florian Weimer" <fweimer(a)redhat.com>
To: "Jakub Cajka" <jcajka(a)redhat.com>
Cc: golang(a)lists.fedoraproject.org
Sent: Wednesday, March 13, 2019 11:44:09 AM
Subject: Re: [golang-dev] proposal: public module authentication with the Go notary
* Jakub Cajka:
> ----- Original Message -----
>> From: "Florian Weimer" <fweimer(a)redhat.com>
>> To: "Jakub Cajka" <jcajka(a)redhat.com>
>> Cc: golang(a)lists.fedoraproject.org
>> Sent: Wednesday, March 13, 2019 10:58:16 AM
>> Subject: Re: [golang-dev] proposal: public module authentication with the
>> Go notary
>>
>> * Jakub Cajka:
>>
>> > From the Fedora perspective it is due to the stability of the upstream
>> > code(API changes) there is nearly no effective difference compared to
>> > the non shared(you will have to rebuild "everything" anyway). I
have
>> > been toying for a long time with switching at least sdtlib to be
>> > dynamically linked in the BR, but haven't got around to push it
>> > out(i.e. coerce maintainers to BR it, IMHO we have bigger nuts to
>> > crack atm).
>>
>> I don't think linking the run time dynamically is feasible.
>>
>> The Go 1 compatibility strategy does not mention not adding private
>> fields to structs as a breaking change, for example:
>>
>> <https://golang.org/doc/go1compat>
>>
>> But each time you allocate a struct, the caller inlines the size, so
>> adding private fields is very much a breaking change.
>>
>> Thanks,
>> Florian
>
> This is not(shouldn't be) a runtime, just the standard library.
Uhm, what's the difference between the two? There is no separate
run-time library.
Thanks,
Florian
I have a feeling that we have had this discussion in the past.
If I'm not mistaken then https://golang.org/pkg/ is the stdlib(including
"runtime" package that enables interaction with runtime) and runtime is
separated part of a language/compiler/produced binary, i.e. it is baked in to each binary
it is not "public" lib in C sense and AFAIK can't be linked in dynamically.
For reference https://golang.org/doc/faq#runtime .
Hope I'm getting it right.
JC
6:56 a.m.
New subject: [golang-dev] proposal: public module authentication with the Go notary
* Jakub Cajka:
I have a feeling that we have had this discussion in the past. If
I'm
not mistaken then https://golang.org/pkg/ is the stdlib(including
"runtime" package that enables interaction with runtime) and runtime
is separated part of a language/compiler/produced binary, i.e. it is
baked in to each binary it is not "public" lib in C sense and AFAIK
can't be linked in dynamically. For reference
https://golang.org/doc/faq#runtime . Hope I'm getting it right.
I expect that the garbage collector would be linked dynamically as well.
It should be easy enough to check by disassembling a dynamically linked
Go program. However, -linkshared does not currently work in Fedora 29
(after installing golang-shared):
$ go build -linkshared t.go
go build runtime/internal/atomic: open
/usr/lib/golang/pkg/linux_amd64_dynlink/runtime/internal/atomic.a: permission denied
go build internal/cpu: open /usr/lib/golang/pkg/linux_amd64_dynlink/internal/cpu.a:
permission denied
go build sync/atomic: open /usr/lib/golang/pkg/linux_amd64_dynlink/sync/atomic.a:
permission denied
go build runtime/cgo: open /usr/lib/golang/pkg/linux_amd64_dynlink/runtime/cgo.a:
permission denied
go build vendor/golang_org/x/crypto/curve25519: open
/usr/lib/golang/pkg/linux_amd64_dynlink/vendor/golang_org/x/crypto/curve25519.a:
permission denied
Thanks,
Florian
7:08 a.m.
New subject: [golang-dev] proposal: public module authentication with the Go
notary
----- Original Message -----
From: "Florian Weimer" <fweimer(a)redhat.com>
To: "Jakub Cajka" <jcajka(a)redhat.com>
Cc: golang(a)lists.fedoraproject.org
Sent: Wednesday, March 13, 2019 12:56:57 PM
Subject: Re: [golang-dev] proposal: public module authentication with the Go notary
* Jakub Cajka:
> I have a feeling that we have had this discussion in the past. If I'm
> not mistaken then https://golang.org/pkg/ is the stdlib(including
> "runtime" package that enables interaction with runtime) and runtime
> is separated part of a language/compiler/produced binary, i.e. it is
> baked in to each binary it is not "public" lib in C sense and AFAIK
> can't be linked in dynamically. For reference
> https://golang.org/doc/faq#runtime . Hope I'm getting it right.
I expect that the garbage collector would be linked dynamically as well.
It should be easy enough to check by disassembling a dynamically linked
Go program. However, -linkshared does not currently work in Fedora 29
(after installing golang-shared):
$ go build -linkshared t.go
go build runtime/internal/atomic: open
/usr/lib/golang/pkg/linux_amd64_dynlink/runtime/internal/atomic.a:
permission denied
go build internal/cpu: open
/usr/lib/golang/pkg/linux_amd64_dynlink/internal/cpu.a: permission denied
go build sync/atomic: open
/usr/lib/golang/pkg/linux_amd64_dynlink/sync/atomic.a: permission denied
go build runtime/cgo: open
/usr/lib/golang/pkg/linux_amd64_dynlink/runtime/cgo.a: permission denied
go build vendor/golang_org/x/crypto/curve25519: open
/usr/lib/golang/pkg/linux_amd64_dynlink/vendor/golang_org/x/crypto/curve25519.a:
permission denied
Thanks,
Florian
Interesting. I haven't touched and put much time in to that since doing the initial
scoping ~3y ago(so I would take anything that I have said with grain of salt). Would you
mind opening BZ?
JC
> _______________________________________________
> golang mailing list -- golang(a)lists.fedoraproject.org
> To unsubscribe send an email to golang-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/golang@lists.fedoraproject.org
>
7:17 a.m.
New subject: [golang-dev] proposal: public module authentication with the Go notary
* Jakub Cajka:
Interesting. I haven't touched and put much time in to that
since
doing the initial scoping ~3y ago(so I would take anything that I have
said with grain of salt). Would you mind opening BZ?
Fair enough: <https://bugzilla.redhat.com/show_bug.cgi?id=1688261>
Thanks,
Florian
7:42 a.m.
New subject: [golang-dev] proposal: public module authentication with the Go
notary
----- Original Message -----
From: "Florian Weimer" <fweimer(a)redhat.com>
To: "Jakub Cajka" <jcajka(a)redhat.com>
Cc: golang(a)lists.fedoraproject.org
Sent: Wednesday, March 13, 2019 1:17:16 PM
Subject: Re: [golang-dev] proposal: public module authentication with the Go notary
* Jakub Cajka:
> Interesting. I haven't touched and put much time in to that since
> doing the initial scoping ~3y ago(so I would take anything that I have
> said with grain of salt). Would you mind opening BZ?
Fair enough: <https://bugzilla.redhat.com/show_bug.cgi?id=1688261>
Thanks,
Florian
Thanks :).
JC
4:45 a.m.
New subject: [golang-dev] proposal: public module authentication with the Go notary
* Fabio Valentini:
Go has supported -buildmode=shared on all major architectures for
some
time. I'm curious why nobody uses that yet.
There is no ABI document like this one for Go:
<https://github.com/itanium-cxx-abi/cxx-abi>
I assume that the Go compiler changes ABI between releases, just like
GCC for C++ did before GCC 3.4 or thereabouts.
In addition to toolchain support, participating packages would have to
follow a set of rules like this:
<https://community.kde.org/Policies/Binary_Compatibility_Issues_With_C%2B%...
Of course, the exact rules will be different for Go, but some of the
issues are similar.
Thanks,
Florian
2:42 p.m.
New subject: [golang-dev] proposal: public module authentication with the Go notary
* Florian Weimer:
* Fabio Valentini:
> Go has supported -buildmode=shared on all major architectures for some
> time. I'm curious why nobody uses that yet.
There is no ABI document like this one for Go:
<https://github.com/itanium-cxx-abi/cxx-abi>
I assume that the Go compiler changes ABI between releases, just like
GCC for C++ did before GCC 3.4 or thereabouts.
Here's a recent example of such a change:
| This patch by Cherry Zhang changes the Go frontend and libgo to pass
| the old slice's ptr/len/cap by value to growslice. In the C calling
| convention, on AMD64, and probably a number of other architectures,
| a 3-word struct argument is passed on stack. This is less efficient
| than passing in three registers. Further, this may affect the code
| generation in other part of the program, even if the function is not
| actually called.
<https://gcc.gnu.org/ml/gcc-patches/2019-03/msg00948.html>
12:17 p.m.
New subject: [golang-dev] proposal: public module authentication with the Go
notary
Le 2019-03-13 10:07, Fabio Valentini a écrit :
Hi Fabio
Go has supported -buildmode=shared on all major architectures for
some
time. I'm curious why nobody uses that yet.
Probably because no one has figured yet how to derive "standard" dynamic
library names from upstream artefact naming and versioning.
Go modules are a step forward, because they give us versions and the
module name as derivation root. But someone still needs to define a
deriving recipe.
As we've all seen while naming go distribution packages, it's not so
easy to define names that do not exist upstream
Regards,
--
Nicolas Mailhot
4:49 a.m.
New subject: [golang-dev] proposal: public module authentication with the Go
notary
----- Original Message -----
From: "Ian Denhardt" <ian(a)zenhack.net>
To: "Matthew Dempsky" <mdempsky(a)google.com>, "Nicolas Mailhot"
<nicolas.mailhot(a)laposte.net>
Cc: "Russ Cox" <rsc(a)golang.org>, "'Matthew Dempsky' via
golang-dev" <golang-dev(a)googlegroups.com>,
golang(a)lists.fedoraproject.org
Sent: Monday, March 11, 2019 7:18:49 PM
Subject: Re: [golang-dev] proposal: public module authentication with the Go notary
Quoting Matthew Dempsky (2019-03-11 13:54:07)
> On Mon, Mar 11, 2019 at 8:01 AM Nicolas Mailhot
> <[1]nicolas.mailhot(a)laposte.net> wrote:
>
> And as I have explained in the detailed description Matthew
> requested,
>
> To be clear, I was asking for details to reproduce the technical issues
> you're running into. I may have missed them, but I don't believe
you've
> provided these.
>
> our construction of the CI/CD environment is incremental, so the
> assumption in the go tool code that "everything is there and it it
> is
> not it can be downloaded directly" does not work for us
>
> If your build system provides the Go source/packages, then Go won't try
> to download them directly itself. This is how Go works within Google's
> build system (which, again, has the same no-network-access limitation
> as the build system you're describing, and yet supports building Go
> programs/packages).
> It sounds like you're having trouble with providing the Go
> source/packages in the format expected by cmd/go. If you would provide
> reproduction steps of what you're doing and the problems you're running
> into, we can help advise what changes you need to make, and maybe
> identify tooling improvements to make that easier.
In a parallel thread, a Nix developer was asking about basically the
same use case, and was pointed at `go build -mod=vendor`. It seems like
this does exactly what is wanted here -- just use the code we have
locally. Nicolas, does that address your use case?
-Ian
I think that our issue in Fedora is that we would like to continue using something that
resembles current "GOPATH/vendor" approach where the tooling is not trying to
kind of out smart us. As we have in distribution environment full control about the build
environment and dependencies and we want/will have to retain it. I believe that having
some "dumb mode" for operating all the go commands/tools(without the hard need
for a modules (verification), even with reduced features) would be much appreciated(so you
can use it as any other compiler/interpreter by just throwing code at it).
IMHO we will tap the metadata that the Go modules carry to simplify our processes and
automations, but for example having to run our own notary(even just in build environment)
just to be able to use go build/go list/go vet... might be really costly and we will be
most probably looking for any alternative.
It has been mentioned several times, in this bunch of treads, that Google's internal
build infrastructure has same requirements and is facing similar challenges. It would be
great to hear how do they plan to "workaround" up coming module changes.
JC
> --
> You received this message because you are subscribed to the Google Groups
> "golang-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-dev+unsubscribe(a)googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
8:38 a.m.
New subject: [golang-dev] proposal: public module authentication with the Go notary
Don't google use Blaze/Bazel for builds internally? I don't think go
get and modules are hugely relevant in that context (see
https://github.com/bazelbuild/rules_go#does-this-work-with-go-modules).
On Tue, 12 Mar 2019 at 09:49, Jakub Cajka <jcajka(a)redhat.com> wrote:
----- Original Message -----
> From: "Ian Denhardt" <ian(a)zenhack.net>
> To: "Matthew Dempsky" <mdempsky(a)google.com>, "Nicolas
Mailhot" <nicolas.mailhot(a)laposte.net>
> Cc: "Russ Cox" <rsc(a)golang.org>, "'Matthew Dempsky' via
golang-dev" <golang-dev(a)googlegroups.com>,
> golang(a)lists.fedoraproject.org
> Sent: Monday, March 11, 2019 7:18:49 PM
> Subject: Re: [golang-dev] proposal: public module authentication with the Go notary
>
> Quoting Matthew Dempsky (2019-03-11 13:54:07)
> > On Mon, Mar 11, 2019 at 8:01 AM Nicolas Mailhot
> > <[1]nicolas.mailhot(a)laposte.net> wrote:
> >
> > And as I have explained in the detailed description Matthew
> > requested,
> >
> > To be clear, I was asking for details to reproduce the technical issues
> > you're running into. I may have missed them, but I don't believe
you've
> > provided these.
> >
> > our construction of the CI/CD environment is incremental, so the
> > assumption in the go tool code that "everything is there and it it
> > is
> > not it can be downloaded directly" does not work for us
> >
> > If your build system provides the Go source/packages, then Go won't try
> > to download them directly itself. This is how Go works within Google's
> > build system (which, again, has the same no-network-access limitation
> > as the build system you're describing, and yet supports building Go
> > programs/packages).
> > It sounds like you're having trouble with providing the Go
> > source/packages in the format expected by cmd/go. If you would provide
> > reproduction steps of what you're doing and the problems you're
running
> > into, we can help advise what changes you need to make, and maybe
> > identify tooling improvements to make that easier.
>
> In a parallel thread, a Nix developer was asking about basically the
> same use case, and was pointed at `go build -mod=vendor`. It seems like
> this does exactly what is wanted here -- just use the code we have
> locally. Nicolas, does that address your use case?
>
> -Ian
>
I think that our issue in Fedora is that we would like to continue using something that
resembles current "GOPATH/vendor" approach where the tooling is not trying to
kind of out smart us. As we have in distribution environment full control about the build
environment and dependencies and we want/will have to retain it. I believe that having
some "dumb mode" for operating all the go commands/tools(without the hard need
for a modules (verification), even with reduced features) would be much appreciated(so you
can use it as any other compiler/interpreter by just throwing code at it).
IMHO we will tap the metadata that the Go modules carry to simplify our processes and
automations, but for example having to run our own notary(even just in build environment)
just to be able to use go build/go list/go vet... might be really costly and we will be
most probably looking for any alternative.
It has been mentioned several times, in this bunch of treads, that Google's internal
build infrastructure has same requirements and is facing similar challenges. It would be
great to hear how do they plan to "workaround" up coming module changes.
JC
> --
> You received this message because you are subscribed to the Google Groups
> "golang-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-dev+unsubscribe(a)googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
golang-dev+unsubscribe(a)googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
Tristan Colgate-McFarlane
----
"You can get all your daily vitamins from 52 pints of guiness, and a
glass of milk"
6:04 a.m.
New subject: [golang-dev] proposal: public module authentication with the Go
notary
[resending to keep the golang-dev listserver happy]
Hi Jakub,
The notary part of Go modules, like the rest of the module
implementation, suffers from a lack of understanding of integration and
QA workflows, and a simplistic dev-centric worldview.
As designed, it will have to be patched out by every single entity
trying to perform large scale Go code integration and QA, starting with
the usual suspects, Linux distributions. Where that will leave upstream
Go, since its main target platform is Linux, I have no idea (I suspect
there will be some angst @Google about it).
I write this from the POW of the person that tries to update the Fedora
Linux Go tooling so we can ship Go with module mode on in august.
Whatever we do at that time will be mirrored by Fedora downstreams like
RHEL and Centos and all the other Linux variants that get their
inspiration Fedora-side.
It's not an armchair analysis, I already wrote way too much module-
oriented custom code because the upstream tooling is deficient, and I've
read my share of upstream issue reports.
CONTINUOUS TRANSPARENT INTERNET LOOKUPS ARE NOT ACCEPTABLE POST−DEV
When you integrate large volumes of code, targeting multiple hardware
architectures, it’s not acceptable to have the result of the Arm QA run
differ from the result of the x86_64 run just because it was scheduled
some minutes later on the QA farm, the internet state changed in the
meanwhile, and the Go tools decided to “help” you by looking at the
internet behind your back and updated transparently the state being
QAed. That's nothing new of Go specific, and that’s why our build system
disables network access in build containers after populating them with
the code to be built and tested, and already did so before someone
decided to invent Go.
But, that QA reproducibility constrain was not understood by Go
developers, and pretty much all the Go tools that were ported to module
mode, will attempt Internet accesses at the slightest occasion, and
change the project state based on those access results, with no ability
to control or disable it, and nasty failure modes if the internet access
fails.
The Go issue tracker is filling up with reports of people expressing
their incredulity (or more) after hitting a variation of this problem on
their QA systems.
QA−ED CODE IS NOT PRISTINE DEV CODE
No code is ever perfect.
When you integrate large volumes of code, targeting multiple hardware
architectures, you *will* hit problems missed by the original upstream
dev team. And you *will* have to patch them up, and you *will* have to
build the result without waiting for upstream to look at it because, in
production, the show must go on, some of those will eventually have
zero-day security implications, and the upstreaming lag is not
acceptable (assuming upstream is available and friendly, which is not
necessarily the case).
So no serious shop is ever going to build its information system from
pristine upstream dev code or pristine upstream Go modules. Pristine
upstream code building is a nice ideal, but in the real world, it’s
utopia.
If you don’t believe me, take your favorite app, and try to build a
system containing it from genuine unadultered upstream code (that means,
without relying on Fedora, Debian or whatever, since we all know they
are full of “nasty” patched code).
Again, that technical real-world-is-not-perfect was not understood by Go
developers. Sure they gave us the replace directive in Go module. But
that is opt-in. So it only works on a small scale, when fixing third
party code is an exception, and you only have a handful of projects that
use this third-party code.
On a large scale everyone will just preventively rename everything by
default just to retain the ability to perform fixes easily. You'll end
up with blanket replacement of every "upstream" module name with
"qaed/upstream" forks. And probably easier to rewrite all the imports to
point to qaed/upstream by default in the source code.
That will result in a huge mess, much larger than the current GOPATH
vendoring mess.
All because the design does not take into account the last mile QA
patching that occurs before putting code in production.
Blindly enforcing renaming rules because you don't understand or accept
the existence of QA middlemen produced things like Iceweasel. And I
don't think Mozilla or Debian were ever happy about it.
PUBLISHING QA-ED CODE CAN NOT DEPEND ON A THIRD PARTY
At least, not if you want the result to work in a free software
ecosystem. Freedom 4 “The freedom to distribute copies of your modified
versions to others” is not “The freedom to distribute copies of your
modified versions to others, but only if you tell this third party
first”.
And Google is free to choose not to target a free software ecosystem,
but that means all the entities that do target a free software ecosystem
(like Linux distributions) or have happily built their infrastructure on
free software products (like every single major cloud operator out
there), will have fork Go, or reorient their software investments
somewhere else, or some combination of those. And I don't think Google
wants that, or will be happy about it.
But, that's the situation the current notary design will create.
Because in trying to enforce a dev utopia, it is going to break the
workflows of a large proportion of the current Go ecosystem.
Any working system will need a way to declare trust in a local authority
(the shop QA team) or a set of authorities (the shop QA team, and
trusted third parties), and not have this trust rely on continuous
access to a server, regardless of who controls it (so detached digital
signatures, not direct server hash lookups).
And I’ll stop here before I write things I will regret later.
Regards,
--
Nicolas Mailhot
1863
days inactive
1875
days old
golang@lists.fedoraproject.org
38 comments
11 participants
participants (11)
-
Benny Siegert -
Fabio Valentini -
Florian Weimer -
Florian Weimer -
Ian Denhardt -
Ian Lance Taylor -
Jakub Cajka -
Matthew Dempsky -
Nicolas Mailhot -
Russ Cox -
Tristan Colgate