mingw sysroot paths (and generalizing them)

[Florian Weimer]

Looking at

Information for RPM mingw64-zlib-1.2.13-2.fc38.noarch.rpm
  https://koji.fedoraproject.org/koji/rpminfo?rpmID=33118048

sysroot paths look like this:

/usr/x86_64-w64-mingw32/sys-root/mingw/bin/zlib1.dll
  /usr/x86_64-w64-mingw32/sys-root/mingw/include/zconf.h
  /usr/x86_64-w64-mingw32/sys-root/mingw/include/zlib.h
  /usr/x86_64-w64-mingw32/sys-root/mingw/lib/libz.dll.a
  /usr/x86_64-w64-mingw32/sys-root/mingw/lib/pkgconfig/zlib.pc

Is the /mingw/ part of the sysroot path, or is it within the sysroot?
Would I use --sysroot=/usr/x86_64-w64-mingw32/sys-root or
--sysroot=/usr/x86_64-w64-mingw32/sys-root/mingw to build against the
sysroot?

I assumed the latter, but now I wonder if /mingw in the sysroot is the
analogue of /usr in GNU/Linux sysroots.

Eventually, we want to produce some GNU/Linux sysroots.  I picked

/usr/%{_arch}-redhat-linux/sys-root/fc%{fedora}

or (depending on the operating system)

/usr/%{_arch}-redhat-linux/sys-root/el%{rhel}

or (as the fallback)

/usr/%{_arch}-redhat-linux/sys-root/root

and we have /usr inside the sysroot, e.g. <stdio.h> is

/usr/x86_64-redhat-linux/sys-root/fc35/usr/include/stdio.h

in a Fedora 35 sysroot on x86-64 (although the Fedora 35 update with
this never actually went out).  We essentially use the /mingw/ part as
an OS ABI version indicator, to make the different versions
co-installable.

I want to pick this up again and would like to solicit comments if that
OS ABI variant thing is the right approach, or if we should drop it.
Sysroot packages are by nature relocatable, and do not have to be
installed in /, so it's perhaps not a great loss if their version
variants are not co-installable.

We need something like this before we can drop i686 on ELN; it's only
way to enable -m32 in GCC in a sustainable fashion.  But we don't need
OS ABI variants for that.

Thanks,
Florian

[Richard W.M. Jones]

On Fri, Apr 28, 2023 at 02:22:40PM +0200, Florian Weimer wrote:
> Looking at
>
> Information for RPM mingw64-zlib-1.2.13-2.fc38.noarch.rpm
>   https://koji.fedoraproject.org/koji/rpminfo?rpmID=33118048
>
> sysroot paths look like this:
>
> /usr/x86_64-w64-mingw32/sys-root/mingw/bin/zlib1.dll
>   /usr/x86_64-w64-mingw32/sys-root/mingw/include/zconf.h
>   /usr/x86_64-w64-mingw32/sys-root/mingw/include/zlib.h
>   /usr/x86_64-w64-mingw32/sys-root/mingw/lib/libz.dll.a
>   /usr/x86_64-w64-mingw32/sys-root/mingw/lib/pkgconfig/zlib.pc
>
> Is the /mingw/ part of the sysroot path, or is it within the sysroot?
> Would I use --sysroot=/usr/x86_64-w64-mingw32/sys-root or
> --sysroot=/usr/x86_64-w64-mingw32/sys-root/mingw to build against the
> sysroot?
>
> I assumed the latter, but now I wonder if /mingw in the sysroot is the
> analogue of /usr in GNU/Linux sysroots.

FWIW:

$ x86_64-w64-mingw32-gcc -print-sysroot
/usr/x86_64-w64-mingw32/sys-root

which would indicate that you are correct that /mingw is somehow
"inside" the sysroot.

However I have no idea about what the correct sysroot should be, just
that what we have now works and changing it would be a massive PITA so
I wouldn't want us to do it without very good reasons.

Rich.
> Eventually, we want to produce some GNU/Linux sysroots.  I picked
>
> /usr/%{_arch}-redhat-linux/sys-root/fc%{fedora}
>
> or (depending on the operating system)
>
> /usr/%{_arch}-redhat-linux/sys-root/el%{rhel}
>
> or (as the fallback)
>
> /usr/%{_arch}-redhat-linux/sys-root/root
>
> and we have /usr inside the sysroot, e.g. <stdio.h> is
>
> /usr/x86_64-redhat-linux/sys-root/fc35/usr/include/stdio.h
>
> in a Fedora 35 sysroot on x86-64 (although the Fedora 35 update with
> this never actually went out).  We essentially use the /mingw/ part as
> an OS ABI version indicator, to make the different versions
> co-installable.
>
> I want to pick this up again and would like to solicit comments if that
> OS ABI variant thing is the right approach, or if we should drop it.
> Sysroot packages are by nature relocatable, and do not have to be
> installed in /, so it's perhaps not a great loss if their version
> variants are not co-installable.
>
> We need something like this before we can drop i686 on ELN; it's only
> way to enable -m32 in GCC in a sustainable fashion.  But we don't need
> OS ABI variants for that.
>
> Thanks,
> Florian

[Neal Gompa]

On Fri, Apr 28, 2023 at 8:23 AM Florian Weimer  wrote:
> Looking at
>
> Information for RPM mingw64-zlib-1.2.13-2.fc38.noarch.rpm
>   https://koji.fedoraproject.org/koji/rpminfo?rpmID=33118048
>
> sysroot paths look like this:
>
> /usr/x86_64-w64-mingw32/sys-root/mingw/bin/zlib1.dll
>   /usr/x86_64-w64-mingw32/sys-root/mingw/include/zconf.h
>   /usr/x86_64-w64-mingw32/sys-root/mingw/include/zlib.h
>   /usr/x86_64-w64-mingw32/sys-root/mingw/lib/libz.dll.a
>   /usr/x86_64-w64-mingw32/sys-root/mingw/lib/pkgconfig/zlib.pc
>
> Is the /mingw/ part of the sysroot path, or is it within the sysroot?
> Would I use --sysroot=/usr/x86_64-w64-mingw32/sys-root or
> --sysroot=/usr/x86_64-w64-mingw32/sys-root/mingw to build against the
> sysroot?
>
> I assumed the latter, but now I wonder if /mingw in the sysroot is the
> analogue of /usr in GNU/Linux sysroots.
>
> Eventually, we want to produce some GNU/Linux sysroots.  I picked
>
> /usr/%{_arch}-redhat-linux/sys-root/fc%{fedora}
>
> or (depending on the operating system)
>
> /usr/%{_arch}-redhat-linux/sys-root/el%{rhel}
>
> or (as the fallback)
>
> /usr/%{_arch}-redhat-linux/sys-root/root
>
> and we have /usr inside the sysroot, e.g. <stdio.h> is
>
> /usr/x86_64-redhat-linux/sys-root/fc35/usr/include/stdio.h
>
> in a Fedora 35 sysroot on x86-64 (although the Fedora 35 update with
> this never actually went out).  We essentially use the /mingw/ part as
> an OS ABI version indicator, to make the different versions
> co-installable.
>
> I want to pick this up again and would like to solicit comments if that
> OS ABI variant thing is the right approach, or if we should drop it.
> Sysroot packages are by nature relocatable, and do not have to be
> installed in /, so it's perhaps not a great loss if their version
> variants are not co-installable.
>
> We need something like this before we can drop i686 on ELN; it's only
> way to enable -m32 in GCC in a sustainable fashion.  But we don't need
> OS ABI variants for that.

Years ago, when we originally started talking about multilib and
multiarch stuff on the list (I think in the context of both x86 and
ARM), the Exherbo developer brought up the idea of reworking the whole
operating system to sysroots[1].

I was actually in favor of this idea, and I prototyped a distribution
setup with it. The result of that was I found a few issues with how
RPM handles dependency generation for supporting such a setup. One of
the biggest ones is in how we generate the library dependencies, which
I attempted to fix a few years ago[2].

Unfortunately, it hasn't gone anywhere even though I'd still like to do it...

[1]: https://lists.fedoraproject.org/archives/list/[e-mail]/...
[2]: https://github.com/rpm-software-management/rpm/pull/1038

[Daniel P. Berrangé]

On Fri, Apr 28, 2023 at 01:41:05PM +0100, Richard W.M. Jones wrote:
> On Fri, Apr 28, 2023 at 02:22:40PM +0200, Florian Weimer wrote:
> > Looking at
> >
> > Information for RPM mingw64-zlib-1.2.13-2.fc38.noarch.rpm
> >   https://koji.fedoraproject.org/koji/rpminfo?rpmID=33118048
> >
> > sysroot paths look like this:
> >
> > /usr/x86_64-w64-mingw32/sys-root/mingw/bin/zlib1.dll
> >   /usr/x86_64-w64-mingw32/sys-root/mingw/include/zconf.h
> >   /usr/x86_64-w64-mingw32/sys-root/mingw/include/zlib.h
> >   /usr/x86_64-w64-mingw32/sys-root/mingw/lib/libz.dll.a
> >   /usr/x86_64-w64-mingw32/sys-root/mingw/lib/pkgconfig/zlib.pc
> >
> > Is the /mingw/ part of the sysroot path, or is it within the sysroot?
> > Would I use --sysroot=/usr/x86_64-w64-mingw32/sys-root or
> > --sysroot=/usr/x86_64-w64-mingw32/sys-root/mingw to build against the
> > sysroot?
> >
> > I assumed the latter, but now I wonder if /mingw in the sysroot is the
> > analogue of /usr in GNU/Linux sysroots.
>
> FWIW:
>
> $ x86_64-w64-mingw32-gcc -print-sysroot
> /usr/x86_64-w64-mingw32/sys-root
>
> which would indicate that you are correct that /mingw is somehow
> "inside" the sysroot.

Then on the Meson side we define

root = '/usr/i686-w64-mingw32/sys-root/mingw'

in the /usr/share/mingw/toolchain-mingw32.meson spec, while with
configure we invoke

configure \
    --host=i686-w64-mingw32 \
    --build=x86_64-redhat-linux-gnu \
    --target=i686-w64-mingw32 \
    --prefix=/usr/i686-w64-mingw32/sys-root/mingw \
    --exec-prefix=/usr/i686-w64-mingw32/sys-root/mingw \
    --bindir=/usr/i686-w64-mingw32/sys-root/mingw/bin \
    --sbindir=/usr/i686-w64-mingw32/sys-root/mingw/sbin \
    --sysconfdir=/usr/i686-w64-mingw32/sys-root/mingw/etc \
    --datadir=/usr/i686-w64-mingw32/sys-root/mingw/share \
    --includedir=/usr/i686-w64-mingw32/sys-root/mingw/include \
    --libdir=/usr/i686-w64-mingw32/sys-root/mingw/lib \
    --libexecdir=/usr/i686-w64-mingw32/sys-root/mingw/libexec \
    --localstatedir=/usr/i686-w64-mingw32/sys-root/mingw/var \
    --sharedstatedir=/usr/i686-w64-mingw32/sys-root/mingw/com \
    --mandir=/usr/i686-w64-mingw32/sys-root/mingw/share/man \
    --infodir=/usr/i686-w64-mingw32/sys-root/mingw/share/info \

is there any difference between the notion of the 'sysroot' vs
the install 'prefix' ?

The pkg-config files for mingw also typically define

prefix=/usr/i686-w64-mingw32/sys-root/mingw

With regards,
Daniel

[Daniel P. Berrangé]

On Fri, Apr 28, 2023 at 02:22:40PM +0200, Florian Weimer wrote:
> Looking at
>
> Information for RPM mingw64-zlib-1.2.13-2.fc38.noarch.rpm
>   https://koji.fedoraproject.org/koji/rpminfo?rpmID=33118048
>
> sysroot paths look like this:
>
> /usr/x86_64-w64-mingw32/sys-root/mingw/bin/zlib1.dll
>   /usr/x86_64-w64-mingw32/sys-root/mingw/include/zconf.h
>   /usr/x86_64-w64-mingw32/sys-root/mingw/include/zlib.h
>   /usr/x86_64-w64-mingw32/sys-root/mingw/lib/libz.dll.a
>   /usr/x86_64-w64-mingw32/sys-root/mingw/lib/pkgconfig/zlib.pc
>
> Is the /mingw/ part of the sysroot path, or is it within the sysroot?
> Would I use --sysroot=/usr/x86_64-w64-mingw32/sys-root or
> --sysroot=/usr/x86_64-w64-mingw32/sys-root/mingw to build against the
> sysroot?
>
> I assumed the latter, but now I wonder if /mingw in the sysroot is the
> analogue of /usr in GNU/Linux sysroots.
>
> Eventually, we want to produce some GNU/Linux sysroots.  I picked
>
> /usr/%{_arch}-redhat-linux/sys-root/fc%{fedora}
>
> or (depending on the operating system)
>
> /usr/%{_arch}-redhat-linux/sys-root/el%{rhel}
>
> or (as the fallback)
>
> /usr/%{_arch}-redhat-linux/sys-root/root
>
> and we have /usr inside the sysroot, e.g. <stdio.h> is
>
> /usr/x86_64-redhat-linux/sys-root/fc35/usr/include/stdio.h
>
> in a Fedora 35 sysroot on x86-64 (although the Fedora 35 update with
> this never actually went out).  We essentially use the /mingw/ part as
> an OS ABI version indicator, to make the different versions
> co-installable.

Parallel install of different versions of mingw has never been
something we needed to consider. We only need parallel install
the different ABIs, which is handled by the higher level dir
in the path. So from that POV, I don't think mingw particularly
cares what semantics are attached to the 4th path component
'mingw'. As long as we don't have to change our paths currently
used in Fedora, I think it'd be OK for you to define the 4th
component as the OS version indicator.

With regards,
Daniel

[David Michael]

On Fri, Apr 28, 2023 at 9:13 AM Daniel P. Berrangé  wrote:
> On Fri, Apr 28, 2023 at 01:41:05PM +0100, Richard W.M. Jones wrote:
> > On Fri, Apr 28, 2023 at 02:22:40PM +0200, Florian Weimer wrote:
> > > Looking at
> > >
> > > Information for RPM mingw64-zlib-1.2.13-2.fc38.noarch.rpm
> > >   https://koji.fedoraproject.org/koji/rpminfo?rpmID=33118048
> > >
> > > sysroot paths look like this:
> > >
> > > /usr/x86_64-w64-mingw32/sys-root/mingw/bin/zlib1.dll
> > >   /usr/x86_64-w64-mingw32/sys-root/mingw/include/zconf.h
> > >   /usr/x86_64-w64-mingw32/sys-root/mingw/include/zlib.h
> > >   /usr/x86_64-w64-mingw32/sys-root/mingw/lib/libz.dll.a
> > >   /usr/x86_64-w64-mingw32/sys-root/mingw/lib/pkgconfig/zlib.pc
> > >
> > > Is the /mingw/ part of the sysroot path, or is it within the sysroot?
> > > Would I use --sysroot=/usr/x86_64-w64-mingw32/sys-root or
> > > --sysroot=/usr/x86_64-w64-mingw32/sys-root/mingw to build against the
> > > sysroot?
> > >
> > > I assumed the latter, but now I wonder if /mingw in the sysroot is the
> > > analogue of /usr in GNU/Linux sysroots.
> >
> > FWIW:
> >
> > $ x86_64-w64-mingw32-gcc -print-sysroot
> > /usr/x86_64-w64-mingw32/sys-root
> >
> > which would indicate that you are correct that /mingw is somehow
> > "inside" the sysroot.
>
> Then on the Meson side we define
>
> root = '/usr/i686-w64-mingw32/sys-root/mingw'
>
> in the /usr/share/mingw/toolchain-mingw32.meson spec, while with
> configure we invoke
>
> configure \
>         --host=i686-w64-mingw32 \
>         --build=x86_64-redhat-linux-gnu \
>         --target=i686-w64-mingw32 \
>         --prefix=/usr/i686-w64-mingw32/sys-root/mingw \
>         --exec-prefix=/usr/i686-w64-mingw32/sys-root/mingw \
>         --bindir=/usr/i686-w64-mingw32/sys-root/mingw/bin \
>         --sbindir=/usr/i686-w64-mingw32/sys-root/mingw/sbin \
>         --sysconfdir=/usr/i686-w64-mingw32/sys-root/mingw/etc \
>         --datadir=/usr/i686-w64-mingw32/sys-root/mingw/share \
>         --includedir=/usr/i686-w64-mingw32/sys-root/mingw/include \
>         --libdir=/usr/i686-w64-mingw32/sys-root/mingw/lib \
>         --libexecdir=/usr/i686-w64-mingw32/sys-root/mingw/libexec \
>         --localstatedir=/usr/i686-w64-mingw32/sys-root/mingw/var \
>         --sharedstatedir=/usr/i686-w64-mingw32/sys-root/mingw/com \
>         --mandir=/usr/i686-w64-mingw32/sys-root/mingw/share/man \
>         --infodir=/usr/i686-w64-mingw32/sys-root/mingw/share/info \
>
> is there any difference between the notion of the 'sysroot' vs
> the install 'prefix' ?

There is at least one minor difference which I recently encountered
with musl packaging:  The gcc/cpp/ld --sysroot option rewrites paths
that include the configured /usr prefix, so e.g.

gcc --sysroot=/usr/x86_64-SYSROOT-linux-gnu -E -Wp,-v -xc /dev/null

shows that it will only search
/usr/x86_64-SYSROOT-linux-gnu/usr/include for headers.  It would be
nice if all new sysroots standardized on a prefix of $SYSROOT/usr
since it's useful for quickly building small things without needing a
build system, but it's a minor detail overall.

Thanks.

David

[Florian Weimer]

* Daniel P. Berrangé:
> Then on the Meson side we define
>
> root = '/usr/i686-w64-mingw32/sys-root/mingw'
>
> in the /usr/share/mingw/toolchain-mingw32.meson spec, while with
> configure we invoke
>
> configure \
>    --host=i686-w64-mingw32 \
>    --build=x86_64-redhat-linux-gnu \
>    --target=i686-w64-mingw32 \
>    --prefix=/usr/i686-w64-mingw32/sys-root/mingw \
>    --exec-prefix=/usr/i686-w64-mingw32/sys-root/mingw \
>    --bindir=/usr/i686-w64-mingw32/sys-root/mingw/bin \
>    --sbindir=/usr/i686-w64-mingw32/sys-root/mingw/sbin \
>    --sysconfdir=/usr/i686-w64-mingw32/sys-root/mingw/etc \
>    --datadir=/usr/i686-w64-mingw32/sys-root/mingw/share \
>    --includedir=/usr/i686-w64-mingw32/sys-root/mingw/include \
>    --libdir=/usr/i686-w64-mingw32/sys-root/mingw/lib \
>    --libexecdir=/usr/i686-w64-mingw32/sys-root/mingw/libexec \
>    --localstatedir=/usr/i686-w64-mingw32/sys-root/mingw/var \
>    --sharedstatedir=/usr/i686-w64-mingw32/sys-root/mingw/com \
>    --mandir=/usr/i686-w64-mingw32/sys-root/mingw/share/man \
>    --infodir=/usr/i686-w64-mingw32/sys-root/mingw/share/info \
>
> is there any difference between the notion of the 'sysroot' vs
> the install 'prefix' ?

The --sysroot option makes it less likely that artifacts from the host
pollute the build, for example headers from /usr/include.  The most
important difference for the Fedora sysroot will be that the absolute
path in the libc.so linker script will be interpreted as local to the
--sysroot tree (kind of like chroot).  If you just use -L…, it would go
to /usr/lib64 outside of the sysroot instead.

A configure invocation in the above style would not produce working
binaries for many GNU/Linux packages because it results in host paths
(not target paths) baked into the binary.  For example, a cross-built
info viewer is still expected to look into /usr/share/info, and not the
sysroot-prefixed path.

However, --sysroot needs to be enabled in the toolchain, and I don't
know if we do that for the mingw toolchain.

Thanks,
Florian

[Kevin Kofler]

Florian Weimer wrote:
> Is the /mingw/ part of the sysroot path, or is it within the sysroot?
> Would I use --sysroot=/usr/x86_64-w64-mingw32/sys-root or
> --sysroot=/usr/x86_64-w64-mingw32/sys-root/mingw to build against the
> sysroot?
>
> I assumed the latter, but now I wonder if /mingw in the sysroot is the
> analogue of /usr in GNU/Linux sysroots.

/mingw is a prefix like /usr. Where this comes from is the native MSYS/MSYS2 
toolchains, where / (and if present, /usr) contains binaries dependent on 
the MSYS DLL ("MSYS binaries"), whereas the subtree /mingw is what contains 
"MinGW binaries", i.e., ones that do NOT require an MSYS DLL. The separation 
helps MSYS because, if it invokes an MSYS binary, no path translation is 
done and the binary receives POSIX paths, whereas a MinGW binary gets its 
command line automatically translated to Windows paths. The rule is that 
everything either outside of the MSYS root or inside the /mingw subdirectory 
is assumed to be a MinGW (or MSVC) binary whereas everything with the MSYS 
root but not under /mingw is assumed to be an MSYS binary.

Cross toolchains are not really affected by this feature, but since MinGW 
build scripts written for MSYS(2) will generally assume native binaries to 
be under /mingw, it is probably best to keep this subdirectory in the 
sysroot.

Kevin Kofler

[Florian Weimer]

* Daniel P. Berrangé:
> Parallel install of different versions of mingw has never been
> something we needed to consider. We only need parallel install
> the different ABIs, which is handled by the higher level dir
> in the path. So from that POV, I don't think mingw particularly
> cares what semantics are attached to the 4th path component
> 'mingw'. As long as we don't have to change our paths currently
> used in Fedora, I think it'd be OK for you to define the 4th
> component as the OS version indicator.

Okay, then I'll keep the path layout as we have it today for now.  It's
toolchain-internal for now anyway, so we can adjust it easily later.

Thanks,
Florian