New %{ld_ldflags} macro?

[Dridi Boukelmoune]

Greetings,

This weekend I played with the Hare [1] toolchain and ran into an
interesting case. The build system assumes an LDFLAGS variable for
direct invocation, not through GCC or Clang. Ironically, the hare(1)
command will also hide the ld(1) command just like GCC or Clang,
but it doesn't support the -Wl,flag syntax to pass flags to the linker.

So I came up with a macro derived from %{build_ldflags} that I called
%{ld_ldflags} to have a similar naming convention and convey the idea
of direct ld execution:
> %global ld_ldflags %{lua:
> sep=''
> for gcc_flag in string.gmatch(macros.build_ldflags, '%S+') do
>    if gcc_flag:match '^-Wl,' then
>        ld_flag=string.gsub(gcc_flag, '-Wl,', '')
>        print(sep..string.gsub(ld_flag, ',', ' '))
>        sep=' '
>    end
> end}

Please note that I'm not a proficient Lua developer, nor an advanced
RPM macro author, so I took the path of least resistance to reach the
WorksOnMyMachine™ point.

I attached the current work-in-progress spec file where I used this
macro, but you won't be able to test it without a compatible harec
package. I quickly hacked a harec refresh together [2] but I don't
have the bandwidth to maintain that kind of stack. I have yet to
submit my trivial patch upstream, maybe next weekend I will make
more progress on Hare packaging.

I thought it was an interesting case of inadequate LDFLAGS and
that the macro I came up with could help in that regard. I didn't take
the time to see whether other packages were running into the need
for "non GCC" ld flags, so even before someone picks hare up, it
could be useful for existing packages.

Best,
Dridi

[1] https://harelang.org/
[2] https://src.fedoraproject.org/rpms/harec/pull-request/1

[Florian Weimer]

* Dridi Boukelmoune:
> This weekend I played with the Hare [1] toolchain and ran into an
> interesting case. The build system assumes an LDFLAGS variable for
> direct invocation, not through GCC or Clang. Ironically, the hare(1)
> command will also hide the ld(1) command just like GCC or Clang,
> but it doesn't support the -Wl,flag syntax to pass flags to the
> linker.

The hare toolchain should not use LDFLAGS if it doesn't use a
GCC-compatible compiler driver.

But I don't see where it does that.  Do you have a more precise
reference?

Thanks,
Florian

[Dridi Boukelmoune]

> The hare toolchain should not use LDFLAGS if it doesn't use a
> GCC-compatible compiler driver.
>
> But I don't see where it does that.  Do you have a more precise
> reference?

See the man page:

https://git.sr.ht/~sircmpwn/hare/tree/master/item/docs/hare.scd

Dridi

[Florian Weimer]

* Dridi Boukelmoune:
> > The hare toolchain should not use LDFLAGS if it doesn't use a
> > GCC-compatible compiler driver.
> >
> > But I don't see where it does that.  Do you have a more precise
> > reference?
>
> See the man page:
>
> https://git.sr.ht/~sircmpwn/hare/tree/master/item/docs/hare.scd

Ah, looked at the wrong repository.  I think we can't use LDFLAGS at all
with hare.  Its startup probably won't support all our hardening.

Thanks,
Florian

[Neal Gompa]

On Mon, Apr 17, 2023 at 7:39 AM Florian Weimer  wrote:
> Dridi Boukelmoune:
> > > The hare toolchain should not use LDFLAGS if it doesn't use a
> > > GCC-compatible compiler driver.
> > >
> > > But I don't see where it does that.  Do you have a more precise
> > > reference?
> >
> > See the man page:
> >
> > https://git.sr.ht/~sircmpwn/hare/tree/master/item/docs/hare.scd
>
> Ah, looked at the wrong repository.  I think we can't use LDFLAGS at all
> with hare.  Its startup probably won't support all our hardening.

Hare seems to have taken a lot of inspiration from Go on its tooling.
Maybe they'll also add the external linker flag that the Go compiler
has so that our linker flags can be used normally.

[Benson Muite]

> Hare seems to have taken a lot of inspiration from Go on its tooling.
> Maybe they'll also add the external linker flag that the Go compiler
> has so that our linker flags can be used normally.

Please also see:
https://bugzilla.redhat.com/show_bug.cgi?id=2154514

[Dridi Boukelmoune]

On Mon, Apr 17, 2023 at 11:38 AM Florian Weimer  wrote:
> Dridi Boukelmoune:
> > > The hare toolchain should not use LDFLAGS if it doesn't use a
> > > GCC-compatible compiler driver.
> > >
> > > But I don't see where it does that.  Do you have a more precise
> > > reference?
> >
> > See the man page:
> >
> > https://git.sr.ht/~sircmpwn/hare/tree/master/item/docs/hare.scd
>
> Ah, looked at the wrong repository.  I think we can't use LDFLAGS at all
> with hare.  Its startup probably won't support all our hardening.

At least haredoc (built by hare) runs just fine on my machine. And
hare was built with the same %{ld_ldflags}.

Please note that the way my %{ld_ldflags} macro works, only "straight"
arguments are passed and those coming from specs (redhat-hardened-ld
and redhat-annobin-cc1) are simply ignored.

[Dridi Boukelmoune]

On Mon, Apr 17, 2023 at 12:11 PM Dridi Boukelmoune
 wrote:
> On Mon, Apr 17, 2023 at 11:38 AM Florian Weimer  wrote:
> > Dridi Boukelmoune:
> > > > The hare toolchain should not use LDFLAGS if it doesn't use a
> > > > GCC-compatible compiler driver.
> > > >
> > > > But I don't see where it does that.  Do you have a more precise
> > > > reference?
> > >
> > > See the man page:
> > >
> > > https://git.sr.ht/~sircmpwn/hare/tree/master/item/docs/hare.scd
> >
> > Ah, looked at the wrong repository.  I think we can't use LDFLAGS at all
> > with hare.  Its startup probably won't support all our hardening.
>
> At least haredoc (built by hare) runs just fine on my machine. And
> hare was built with the same %{ld_ldflags}.
>
> Please note that the way my %{ld_ldflags} macro works, only "straight"
> arguments are passed and those coming from specs (redhat-hardened-ld
> and redhat-annobin-cc1) are simply ignored.

It loos like somebody had already suggested moving away from LDFLAGS,
using LDLINKFLAGS instead, and the patch was merged 2 days after I
started this thread:

https://todo.sr.ht/~sircmpwn/hare/784

So that's one conflict gone, but there is still no RPM macro support for
distro-wide linker flags for toolchains not relying on LDFLAGS in their
current form.

My %{ld_ldflags} macro could be taking the problem from the wrong end: we
could instead have some %{link_ldflags} macro from which %{build_ldflags}
would be derived, turning command line arguments into `-Wl` options and
adding GCC-specific -specs options.

Cheers

[Florian Weimer]

* Dridi Boukelmoune:
> It loos like somebody had already suggested moving away from LDFLAGS,
> using LDLINKFLAGS instead, and the patch was merged 2 days after I
> started this thread:
>
> https://todo.sr.ht/~sircmpwn/hare/784
>
> So that's one conflict gone, but there is still no RPM macro support for
> distro-wide linker flags for toolchains not relying on LDFLAGS in their
> current form.
>
> My %{ld_ldflags} macro could be taking the problem from the wrong end: we
> could instead have some %{link_ldflags} macro from which %{build_ldflags}
> would be derived, turning command line arguments into `-Wl` options and
> adding GCC-specific -specs options.

Do you know if LDLINKFLAGS comes before -shared or after?

If we use LDLINKFLAGS to enable PIE, it has to come first, so that
-shared overrides it.  We can handle this with -specs independently of
order, but not directly in ld.

Thanks,
Florian

[Dridi Boukelmoune]

> Do you know if LDLINKFLAGS comes before -shared or after?
>
> If we use LDLINKFLAGS to enable PIE, it has to come first, so that
> -shared overrides it.  We can handle this with -specs independently of
> order, but not directly in ld.

I think you can only statically build programs, and libraries are just
source code from your HAREPATH.

But I checked where the linker flags are put in the command line, and
it's right after the $(LD) command, before other options like -o
managed by hare(1). The object files and archives come last.

Dridi

[Jan Drögehoff]

The problem is that Hare directly passes LDFLAGS on to ld, expecting them to be flags for the linker.
Instead LDFLAGS is commonly used for compiler flags for linking, only actually passing on -Wl flags.

`%global build_ldflags %(echo %{build_ldflags} | sed "s/-Wl,//g")` may be a simple drop-in solution for this, but I don't know if ld will accept the comma separated format or if thats something that the compiler will need to implement.

This may be worth mentioning to devault to figure out if this is worth including in hare itself

[Dridi Boukelmoune]

On Mon, Apr 17, 2023 at 11:51 AM Jan Drögehoff  wrote:
> The problem is that Hare directly passes LDFLAGS on to ld, expecting them to be flags for the linker.
> Instead LDFLAGS is commonly used for compiler flags for linking, only actually passing on -Wl flags.
>
> `%global build_ldflags %(echo %{build_ldflags} | sed "s/-Wl,//g")` may be a simple drop-in solution for this, but I don't know if ld will accept the comma separated format or if thats something that the compiler will need to implement.
>
> This may be worth mentioning to devault to figure out if this is worth including in hare itself

I initially had a shell one-liner but you need more than one sed to
both get rid of --spec options and -Wl commas.

Dridi