Mono Packaging Issues

[Tom Callaway]

As I look into lifting the hold on Mono packages in Fedora Extras (aka,
blessing the mono packaging guidelines found here:
http://fedoraproject.org/wiki/Packaging/Mono), I'd like to state my
opinions on the following blocker issues:

1. Redefining libdir for mono packages:

If mono and its tools cannot find a library on a 64bit architecture when
it lives in %{_libdir} (/usr/lib64), then mono is fundamentally broken
on that architecture, and needs to be fixed. This is a serious flaw, and
I will not have us doing ugly hacks in spec files to work around mono's
ineptitude. Every other core component is able to handle this, this bug
should be filed and fixed.

2. Putting DLLs in %{_datadir} vs %{_libdir}:

The FHS says that /usr/share (aka, %{_datadir}) is for "Architecture
Independent Data". These DLL files do not seem to qualify as
"Architecture Independent Data". I think that having them live in the:
%{_libdir}/mono/ hierarchy seems to be the most appropriate for them at
this point in time. Obviously, this can be revisited if upstream changes
the default location. The gacutil command should be putting these DLL
files in the right place.

3. Forcing local copies of DLL libraries instead of using system
installed DLLs.
This concept is documented upstream here:
http://www.mono-project.com/Assemblies_and_the_GAC#Libraries_with_Unstabl...
IMHO, this is a really stupid and braindead idea. In fact, I've yet to
talk to anyone who actually thinks this is a good idea. This is the same
thing as static linking a private copy of a library instead of using the
system shared lib. We do NOT permit this sort of behavior in Fedora, and
mono is no exception.

I've copied alexl on this email, since he is the mono owner in Fedora
Core, and likely has an opinion or two. ;)

~spot
-- 
Tom "spot" Callaway: Red Hat Senior Sales Engineer || GPG ID: 93054260
Fedora Extras Steering Committee Member (RPM Standards and Practices)
Aurora Linux Project Leader: http://auroralinux.org
Lemurs, llamas, and sparcs, oh my!

[Toshio Kuratomi]

Hi spot!

On Mon, 2006-06-12 at 18:23 -0500, Tom 'spot' Callaway wrote:
>  1. Redefining libdir for mono packages:
>  
>  If mono and its tools cannot find a library on a 64bit architecture
>  when
>  it lives in %{_libdir} (/usr/lib64), then mono is fundamentally broken
>  on that architecture, and needs to be fixed. 
I agree.

>  2. Putting DLLs in %{_datadir} vs %{_libdir}:
>  
>  The FHS says that /usr/share (aka, %{_datadir}) is for "Architecture
>  Independent Data". These DLL files do not seem to qualify as
>  "Architecture Independent Data". I think that having them live in the:
>  %{_libdir}/mono/ hierarchy seems to be the most appropriate for them
>  at
>  this point in time. Obviously, this can be revisited if upstream
>  changes
>  the default location. The gacutil command should be putting these DLL
>  files in the right place.
>   
My understanding is .dlls and .exes are architecture independent.
Should they end up in %{_libdir} or always end up in /usr/lib?  Fedora's
Python and Perl place arch independent data into /usr/lib, (I assume so
that they can be noarch) should mono as well?  All the *-sharp packages
on my system currently follow this convention but beagle, tomboy, and
banshee end up in %{_libdir}.  Perhaps relevantly, these three are
primarily applications but they do supply .dlls.

I have only one package on my system that installs something
into /usr/lib(!mono):
gtk-sharp2 =>  /usr/lib/gtk-sharp-2.0/gconfsharp-schemagen.exe.
By contrast the beagle/tomboy/banshee triumvirate install to
%{_libdir}/[PKGNAME]
Should this be allowed by the guidelines or should all mono packages go
under /usr/lib/mono (similar to the python and perl hierarchies
in /usr/lib)

The above examples illustrate that applications and libraries are
currently defaulting to different areas.  Does it make sense to put
applications in a different area than libraries?  Python and Perl seem
to use /usr/lib for arch independent libraries, %{_libdir} for arch
dependent libraries, and %{_datadir}/[PKGNAME] for arch-independent
program-specific scripts. (ex: rpmlint).  There is one mono application
under review (nant) that places its files in %{_datadir)/[PKGNAME].  Do
we need to move this or is it fine as is?

If we separate applications and libraries, how do we determine
which .dlls belong in an application directory and which belong in a
library directory?  ELF shared libraries have ldconfig as a tipoff.  I
can think of the following tipoffs for a mono library, are there others?
  The .dll is listed in a pkgconfig .pc file.
  The package attempts to setup the .dll in the GAC. (What are the
telltales of this?)

Forcing everything to /usr/lib/mono would be the easiest to review but
would cause a lot of hard to justify changing of upstream's install
locations.  OTOH, the hodgepodge of locations we have now is not
conducive to good reviewing because it leaves all these ambiguous areas.
Where do you see the logical lines falling?

>  3. Forcing local copies of DLL libraries instead of using system
>  installed DLLs.
>  This concept is documented upstream here:
>  http://www.mono-project.com/Assemblies_and_the_GAC#Libraries_with_Unstabl...
>  IMHO, this is a really stupid and braindead idea. In fact, I've yet to
>  talk to anyone who actually thinks this is a good idea. This is the
>  same
>  thing as static linking a private copy of a library instead of using
>  the
>  system shared lib. We do NOT permit this sort of behavior in Fedora,
>  and
>  mono is no exception.  
Here here!

What are the easy ways to detect this issue?

1) Upstream tarball contains .dlls that were not rebuilt from source
contained in the package.
2) Look through the installed .dlls for any that have the same name as
system .dlls or are suspiciously out of place (Package is myDiary and
contains mysql.dll, sqlite.dll, and gtk-sharp.dll)
3) Source directories look odd:
PKGNAME/
  src/
  data/
  libs/
    gtk-sharp/
    atk-sharp/

Anything else?

-Toshio

[Ville Skyttä]

On Mon, 2006-06-12 at 22:41 -0700, Toshio Kuratomi wrote:

>  Fedora's Python and Perl place arch independent data into
> /usr/lib, 
It can be argued that this is a packaging glitch (which is hard to fix
in a backwards compatible manner) in both of them.

[Toshio Kuratomi]

On Tue, 2006-06-13 at 09:35 +0300, Ville Skyttä wrote:
>  On Mon, 2006-06-12 at 22:41 -0700, Toshio Kuratomi wrote:
>  
>  > Fedora's Python and Perl place arch independent data into /usr/lib,
>  
>  It can be argued that this is a packaging glitch (which is hard to fix
>  in a backwards compatible manner) in both of them. 
True.  But in my response to spot, I'm using these examples to ask if
all the arch independent data should be consolidated into one place
rather than whether that place is /usr/lib or /usr/share.

If you want to bring that point up you can but after thinking about it
this weekend I couldn't come up with a truly compelling reason.

-Toshio

[Nicolas Mailhot]

Le Mar 13 juin 2006 09:33, Toshio Kuratomi a écrit :
>  On Tue, 2006-06-13 at 09:35 +0300, Ville Skyttä wrote:
> > On Mon, 2006-06-12 at 22:41 -0700, Toshio Kuratomi wrote:
> >
> > > Fedora's Python and Perl place arch independent data into /usr/lib,
> >
> > It can be argued that this is a packaging glitch (which is hard to fix
> > in a backwards compatible manner) in both of them.
>
>  True.  But in my response to spot, I'm using these examples to ask if
>  all the arch independent data should be consolidated into one place
>  rather than whether that place is /usr/lib or /usr/share. 
Some languages like java DO put arch-indep stuff in /usr/share, so you
can't take python and perl as the argument Fedora puts arch-indep stuff in
/usr/lib so let's ignore the LSB and mono should too

-- 
Nicolas Mailhot

[Paul F. Johnson]

Hi,

I've asked about the architecture agnostisism on the mono-developers
list and had this back

8-->
On Wed, 2006-06-14 at 09:49 +0100, PFJ wrote:
>  Is there any sure fire way to detect if either a dll or binary
compiled
>  with mono is platform agnostic or platform specific? 
It all depends on what your definition of "platform agnostic" and
"platform specific" is. :-)

The best example of a "platform specific" assembly would be a "mixed
mode" assembly, in which actual x86/AMD64/Itanium/etc. code is placed
into the assembly along with the platform agnostic CIL code.

However, only .NET tools can create such mixed mode assemblies, and Mono
doesn't support their execution on any platform, so they're currently
not a concern (and likely won't ever be, for a variety of reasons).

Consequently, any assembly of consequence executing under Mono will
contain *only* platform agnostic CIL code.

The only thing left to determine whether an assembly is platform
specific is the use of Platform Invoke (P/Invoke), and this is _not_
trivial, as "portable" P/Invoke code can be written, but
platform-specific platform code can also be written.  Classic example:
`struct stat' in C#:

        public struct stat {
                public ulong st_dev, st_ino;
                public uint st_mode;
                private uint _padding_;
                public ulong st_nlink, st_uid, st_gid, st_rdev;
                public long st_size, st_blksize, st_blocks, st_atime,
                        st_mtime, st_ctime;
        }

Question: is that structure definition portable for use with fstat(2)?

Answer: Maybe. :-)  It should work on 64-bit ABIs (since long/ulong are
64-bit integer types), but it will fail badly on 32-bit ABIs, and it
will fail on any ABI in which the member ordering differs.
Consequently, such a structure would be platform specific if used with a
DllImport statement, but there is NO EASY WAY for a program to make this
determination. :-/

(Plus, it can get more complicated, particularly if such a structure is
instead used with some native library which will accept that structure,
such as Mono.Posix.dll & libMonoPosixHelper.so, which uses a 64-bit ABI
throughout, and libMonoPosixHelper.so converts this to a 32-bit ABI as
appropriate.)

So it's easiest to assume that all CIL code is platform agnostic.

>  There is currently a debate about this on the fedora-packagers list,
but
>  as yet, no one is able to determine a simple way to see where things
>  need to go (in respects to agnostic going in /usr/share and platform
>  specific in /usr/lib or /usr/lib64). As FC has only recently taken 
mono
>  as part of it's core distro, the packagers (me included) want to
> get
>  this right from the outset. 
I would suggest looking at what SuSE and Debian/Ubuntu have done with
this, as they have all encountered these issues.  Following in their
footsteps would help consistency, and make it easier to repackage
existing programs.

I believe that they currently use /usr/lib for everything.

However, this gets considerably more complicated, as applications may
bundle native libraries with them -- take Blam! and F-Spot as examples:

        $ ls /usr/lib/f-spot
        FlickrNet.dll      libfspoteog.so.0.0.0       libfspot.so.0
        f-spot.exe         libfspotjpegtran.so        libfspot.so.0.0.0
        f-spot.exe.config  libfspotjpegtran.so.0
libgphoto2-sharp.dll
        libfspoteog.so     libfspotjpegtran.so.0.0.0
libgphoto2-sharp.dll.config
        libfspoteog.so.0   libfspot.so                SemWeb.dll

The *.dll and *.exe files may be platform agnostic, but the *.so* files
certainly are NOT.

So the easiest solution may be to check to see if any native libraries
are bundled with the app, and if that's the case stick it under the
appropriate /usr/lib or /usr/lib64 directory.  Otherwise, stick
with /usr/lib.
<--8

It doesn't exactly help, but does confirm that really, %_libdir should
be getting used. I do find it odd though that they suggest looking at
Debian and SuSE as they both do things differently (out of the two, I
think I'd go with the SuSE system rather than the Debian one)

TTFN

Paul

-- 
ich liebe Ashleigh, eins zwei drei 
ich liebe Ashleigh, auf meinem Knee zu hüpfen

[Tom Callaway]

On Wed, 2006-06-14 at 19:43 +0100, Paul wrote:
>  
>  The *.dll and *.exe files may be platform agnostic, but the *.so*
>  files
>  certainly are NOT.
>  
>  So the easiest solution may be to check to see if any native libraries
>  are bundled with the app, and if that's the case stick it under the
>  appropriate /usr/lib or /usr/lib64 directory.  Otherwise, stick
>  with /usr/lib.  
In the attempt to make the hurting stop, I propose that we do the
following:

If the mono package includes .so files, it should use
%{_libdir}/%{name}.

If the mono package does not include .so files, it should be BuildArch:
noarch and use /usr/lib.

EXE files should have a symlink to %{_bindir}.

Mono apps should NOT ship local copies of existant system DLLs without a
really really good reason.

I don't like it, but I think its easier to just draw the line there than
try to rework an entire language.

~spot
-- 
Tom "spot" Callaway: Red Hat Senior Sales Engineer || GPG ID: 93054260
Fedora Extras Steering Committee Member (RPM Standards and Practices)
Aurora Linux Project Leader: http://auroralinux.org
Lemurs, llamas, and sparcs, oh my!

[Nicolas Mailhot]

Le mercredi 14 juin 2006 à 14:56 -0500, Tom 'spot' Callaway a écrit :

>  If the mono package does not include .so files, it should be
> BuildArch:
>  noarch and use /usr/lib. 
Why on earth do you want it in /usr/lib if you know it's
arch-independant ?

-- 
Nicolas Mailhot

[Tom Callaway]

On Wed, 2006-06-14 at 22:17 +0200, Nicolas Mailhot wrote:
>  Le mercredi 14 juin 2006 à 14:56 -0500, Tom 'spot' Callaway a
> écrit :
>  
>  > If the mono package does not include .so files, it should be BuildArch:
>  > noarch and use /usr/lib.
>  
>  Why on earth do you want it in /usr/lib if you know it's
>  arch-independant ? 
Because they're still libraries, in a weird perverted Windowsy way?

It might be worth examining exactly what Debian/Ubuntu/SuSE do here.

~spot
-- 
Tom "spot" Callaway: Red Hat Senior Sales Engineer || GPG ID: 93054260
Fedora Extras Steering Committee Member (RPM Standards and Practices)
Aurora Linux Project Leader: http://auroralinux.org
Lemurs, llamas, and sparcs, oh my!

[Nicolas Mailhot]

Le mercredi 14 juin 2006 à 15:34 -0500, Tom 'spot' Callaway a écrit :
>  On Wed, 2006-06-14 at 22:17 +0200, Nicolas Mailhot wrote:
>  > Le mercredi 14 juin 2006 à 14:56 -0500, Tom 'spot' Callaway a écrit :
>  > 
>  > > If the mono package does not include .so files, it should be BuildArch:
>  > > noarch and use /usr/lib.
>  > 
>  > Why on earth do you want it in /usr/lib if you know it's
>  > arch-independant ?
>  
>  Because they're still libraries, in a weird perverted Windowsy way? 
So what ?
Are their any less libraries than jar files ? lisp packages ? 

All those end up in /usr/share in Fedora now, as the FHS demands

The "it's code, therefore it shouldn't go in /usr/share" argument is
bogus. We have a ton of code in /usr/share, both shared and app-specific

-- 
Nicolas Mailhot

[Tom Callaway]

On Wed, 2006-06-14 at 22:44 +0200, Nicolas Mailhot wrote:
>  Le mercredi 14 juin 2006 à 15:34 -0500, Tom 'spot' Callaway a
> écrit :
>  > On Wed, 2006-06-14 at 22:17 +0200, Nicolas Mailhot wrote:
>  > > Le mercredi 14 juin 2006 à 14:56 -0500, Tom 'spot' Callaway a écrit :
>  > > 
>  > > > If the mono package does not include .so files, it should be BuildArch:
>  > > > noarch and use /usr/lib.
>  > > 
>  > > Why on earth do you want it in /usr/lib if you know it's
>  > > arch-independant ?
>  > 
>  > Because they're still libraries, in a weird perverted Windowsy way?
>  
>  So what ?
>  Are their any less libraries than jar files ? lisp packages ? 
>  
>  All those end up in /usr/share in Fedora now, as the FHS demands
>  
>  The "it's code, therefore it shouldn't go in /usr/share" argument is
>  bogus. We have a ton of code in /usr/share, both shared and app-specific 
OK, so if we put everything in %{_datadir}/%{name}, the following
questions arise:

1. Does this work? Or do the apps stop working?
2. Can we still put the .so files in %{_libdir}/%{name}, symlink them
back to %{_datadir}/%{name}? Does this work? Or do the apps stop
working?

~spot
-- 
Tom "spot" Callaway: Red Hat Senior Sales Engineer || GPG ID: 93054260
Fedora Extras Steering Committee Member (RPM Standards and Practices)
Aurora Linux Project Leader: http://auroralinux.org
Lemurs, llamas, and sparcs, oh my!

[Paul F. Johnson]

Hi,

>  > The "it's code, therefore it shouldn't go in
> /usr/share" argument is
>  > bogus. We have a ton of code in /usr/share, both shared and app-specific
>  
>  OK, so if we put everything in %{_datadir}/%{name}, the following
>  questions arise:
>  
>  1. Does this work? Or do the apps stop working? 
Quite a few just die. Some don't. Some give you UK weather results
(depends on what you're doing). You don't get this if they're
in /usr/lib

>  2. Can we still put the .so files in %{_libdir}/%{name}, symlink
> them
>  back to %{_datadir}/%{name}? Does this work? Or do the apps stop
>  working? 
See above. Mono is *very* picky on where things go. See my other posting
on the Windows style that has to be adopted for Mono to work on
different platforms.

TTFN

Paul
-- 
ich liebe Ashleigh, eins zwei drei 
ich liebe Ashleigh, auf meinem Knee zu hüpfen

[Nicolas Mailhot]

Le mercredi 14 juin 2006 à 16:20 -0500, Tom 'spot' Callaway a écrit :
>  On Wed, 2006-06-14 at 22:44 +0200, Nicolas Mailhot wrote:
>  > Le mercredi 14 juin 2006 à 15:34 -0500, Tom 'spot' Callaway a écrit :
>  > > On Wed, 2006-06-14 at 22:17 +0200, Nicolas Mailhot wrote:
>  > > > Le mercredi 14 juin 2006 à 14:56 -0500, Tom 'spot' Callaway a
> écrit :
>  > > > 
>  > > > > If the mono package does not include .so files, it should be
> BuildArch:
>  > > > > noarch and use /usr/lib.
>  > > > 
>  > > > Why on earth do you want it in /usr/lib if you know it's
>  > > > arch-independant ?
>  > > 
>  > > Because they're still libraries, in a weird perverted Windowsy way?
>  > 
>  > So what ?
>  > Are their any less libraries than jar files ? lisp packages ? 
>  > 
>  > All those end up in /usr/share in Fedora now, as the FHS demands
>  > 
>  > The "it's code, therefore it shouldn't go in /usr/share" argument
> is
>  > bogus. We have a ton of code in /usr/share, both shared and app-specific
>  
>  OK, so if we put everything in %{_datadir}/%{name}, the following
>  questions arise: 
1. You probably want to create a %{_datadir}/mono or %{_datadir}/c# root
2. if the mono stack is too dumb to work if the .dll and .so are not in
the same dir, you should symlink the dlls to %{_libdir} since it's
difficult to create a symlink pointing both to /usr/lib and /usr/lib64
on a x86_64 system

>  1. Does this work? Or do the apps stop working?
>  2. Can we still put the .so files in %{_libdir}/%{name}, symlink them
>  back to %{_datadir}/%{name}? Does this work? Or do the apps stop
>  working? 
You know after reading this thread I fear the apps will stop working
whatever way we choose, and we won't avoid fixing mono to work with our
chosen policy.

Regards,

-- 
Nicolas Mailhot

[Paul F. Johnson]

Hi,

>  You know after reading this thread I fear the apps will stop working
>  whatever way we choose, and we won't avoid fixing mono to work with our
>  chosen policy. 
If we don't change where the mono installer wants to put things (other
than /usr/lib for shared libs), then everything works out the box.

Unless it relies on gtk-2.9 and then it dies horribly (look what happens
if you use monodevelop [either my rpm or from SVN] and try to create a
new project!)

TTFN

Paul
-- 
ich liebe Ashleigh, eins zwei drei 
ich liebe Ashleigh, auf meinem Knee zu hüpfen

[Enrico Scholz]

tcallawa(a)redhat.com ("Tom 'spot' Callaway") writes:

>  2. Can we still put the .so files in %{_libdir}/%{name}, symlink
> them
>  back to %{_datadir}/%{name}? Does this work? 
It will not work, when /usr/share is NFS-shared between an i386 and
x86_64 machine.



Enrico

[Ian Burrell]

On 6/14/06, Nicolas Mailhot <nicolas.mailhot(a)laposte.net&gt; wrote:
>  Le mercredi 14 juin 2006 à 15:34 -0500, Tom 'spot' Callaway a
> écrit :
>  > On Wed, 2006-06-14 at 22:17 +0200, Nicolas Mailhot wrote:
>  > >
>  > > Why on earth do you want it in /usr/lib if you know it's
>  > > arch-independant ?
>  >
>  > Because they're still libraries, in a weird perverted Windowsy way?
>
>  So what ?
>  Are their any less libraries than jar files ? lisp packages ?
>
>  All those end up in /usr/share in Fedora now, as the FHS demands
>  
Are they any less libraries than Perl, Ruby, or Python code?  Those
all end up in /usr/lib.

>  The "it's code, therefore it shouldn't go in
> /usr/share" argument is
>  bogus. We have a ton of code in /usr/share, both shared and app-specific
>  
Those three languages are grandfathered in because they expect to
install code and shared libraries in /usr/lib. They now put shared
libraries and code that uses them under /usr/lib64 and noarch packages
in /usr/lib.

Mono is probably in the same boat.  It might be possible to put the
arch-independent code in /usr/share but it would create compatibility
problems with existing packages, other distributions, and the
installers expectations. Probably not worth the trouble for LHS
purity.

 - Ian

[Nicolas Mailhot]

Le mercredi 14 juin 2006 à 15:33 -0700, Ian Burrell a écrit :

>  Those three languages are grandfathered in because they expect to
>  install code and shared libraries in /usr/lib. They now put shared
>  libraries and code that uses them under /usr/lib64 and noarch packages
>  in /usr/lib.
>  
>  Mono is probably in the same boat.  It might be possible to put the
>  arch-independent code in /usr/share but it would create compatibility
>  problems with existing packages, other distributions, and the
>  installers expectations. Probably not worth the trouble for FHS
>  purity. 
The other distros do not have consistent practice, mono in Fedora is
very new with few packages so far, it *is* the right time to make the
right choice before you have enough old baggage so your argument holds
sway. Red Hat customers always end up forcing Red Hat to follow 'FHS
purity' even if Red Hat doesn't want it¹, so if your argument is it's
hard now why it'll be harder later and you'll have to do it anyway²
(mark my words).

lib64 stuff requiring lib stuff (or the reverse) is going to be a
terrific can of worms as soon as the 64bit OO.o port completes and
people start looking at killing /usr/lib on 64bit systems.

multilib and the FHS are both old enough we shouldn't be making such
stupid mistakes today. 

Regards,

¹ Red Hat didn't like /srv or media it still is slowly migrating to them
anyway
² but perl and python are doing it it can't be so bad - did you really
miss the threads on python noarch problems some months ago ?

-- 
Nicolas Mailhot

[Nicolas Mailhot]

Le mercredi 14 juin 2006 à 15:33 -0700, Ian Burrell a écrit :
>  Probably not worth the trouble for LHS purity. 
If you don't aim for LHS purity, the FSB secret police will get at you.

More seriously you're confusing the LSB as a whole with the FHS it
references.

LSB is crap nobody expects you to follow it.
FHS is fairly good and followed closely my all big distros. Users do
expect Fedora to follow it. As previous RHL/FC examples proved, you can
duck your head on the sand and say the FHS didn't write some_decision,
but some_decision will still be online in the spec users read, and after
a few years of harassment by users demanding why you don't apply
some_decision you'll have a change of heart and decide applying it is
not so painful after all (certainly less than handling all the bug
reports / support requests not applying it generates)

-- 
Nicolas Mailhot

[Paul F. Johnson]

Hi,

>  It might be worth examining exactly what Debian/Ubuntu/SuSE do here.
SuSE declares everything to be noarch - doesn't matter what it is. This
really screws everything up.

Debian/Ubuntu have an awful mess to contend with. Let's have something
that works. I agree with what you've suggested Tom. I'll get on and
check that my packages are not including .so files.

TTFN

Paul
-- 
ich liebe Ashleigh, eins zwei drei 
ich liebe Ashleigh, auf meinem Knee zu hüpfen

[Paul F. Johnson]

Hi,

>  > It might be worth examining exactly what Debian/Ubuntu/SuSE do
> here.
>  
>  SuSE declares everything to be noarch - doesn't matter what it is. This
>  really screws everything up.
>  
>  Debian/Ubuntu have an awful mess to contend with. Let's have something
>  that works. I agree with what you've suggested Tom. I'll get on and
>  check that my packages are not including .so files. 
Problem : the configure script generates the .pc file as well as finding
the bits required from pre-installed packages. Having noarch in the
BuildArch there causes the configure script to moan and die. It also
runs the automake things to generate the makefiles.

This could be a problem Huston....

TTFN

Paul
-- 
ich liebe Ashleigh, eins zwei drei 
ich liebe Ashleigh, auf meinem Knee zu hüpfen

[Tom Callaway]

On Wed, 2006-06-14 at 22:06 +0100, Paul wrote:
>  Hi,
>  
>  > > It might be worth examining exactly what Debian/Ubuntu/SuSE do here.
>  > 
>  > SuSE declares everything to be noarch - doesn't matter what it is. This
>  > really screws everything up.
>  > 
>  > Debian/Ubuntu have an awful mess to contend with. Let's have something
>  > that works. I agree with what you've suggested Tom. I'll get on and
>  > check that my packages are not including .so files.
>  
>  Problem : the configure script generates the .pc file as well as finding
>  the bits required from pre-installed packages. Having noarch in the
>  BuildArch there causes the configure script to moan and die. It also
>  runs the automake things to generate the makefiles. 
How exactly is this dying? Is it dying because of something that the %
configure macro is passing?

~spot
-- 
Tom "spot" Callaway: Red Hat Senior Sales Engineer || GPG ID: 93054260
Fedora Extras Steering Committee Member (RPM Standards and Practices)
Aurora Linux Project Leader: http://auroralinux.org
Lemurs, llamas, and sparcs, oh my!

[Paul F. Johnson]

Hi,

>  > Problem : the configure script generates the .pc file as well as
> finding
>  > the bits required from pre-installed packages. Having noarch in the
>  > BuildArch there causes the configure script to moan and die. It also
>  > runs the automake things to generate the makefiles.
>  
>  How exactly is this dying? Is it dying because of something that the %
>  configure macro is passing? 
checking build system type... i686-redhat-linux-gnu
checking host system type... i686-redhat-linux-gnu
checking target system type... Invalid configuration
`noarch-redhat-linux-gnu': machine `noarch-redhat' not recognised

It looks like it doesn't like the target.

TTFN

Paul
-- 
ich liebe Ashleigh, eins zwei drei 
ich liebe Ashleigh, auf meinem Knee zu hüpfen

[Ralf Corsepius]

On Wed, 2006-06-14 at 22:22 +0100, Paul wrote:
>  Hi,
>  
>  > > Problem : the configure script generates the .pc file as well as finding
>  > > the bits required from pre-installed packages. Having noarch in the
>  > > BuildArch there causes the configure script to moan and die. It also
>  > > runs the automake things to generate the makefiles.
>  > 
>  > How exactly is this dying? Is it dying because of something that the %
>  > configure macro is passing?
>  
>  checking build system type... i686-redhat-linux-gnu
>  checking host system type... i686-redhat-linux-gnu
>  checking target system type... Invalid configuration
>  `noarch-redhat-linux-gnu': machine `noarch-redhat' not recognised
>  
>  It looks like it doesn't like the target. 
Please show us the corresponding call to configure.

Ralf

[Toshio Kuratomi]

On Wed, 2006-06-14 at 19:43 +0100, Paul wrote:
>  Hi,
>  
>  I've asked about the architecture agnostisism on the mono-developers
>  list and had this back
>  
>  8-->
>  On Wed, 2006-06-14 at 09:49 +0100, PFJ wrote:
>  The best example of a "platform specific" assembly would be a "mixed
>  mode" assembly, in which actual x86/AMD64/Itanium/etc. code is placed
>  into the assembly along with the platform agnostic CIL code.
>  
>  However, only .NET tools can create such mixed mode assemblies, and Mono
>  doesn't support their execution on any platform, so they're currently
>  not a concern (and likely won't ever be, for a variety of reasons).
>   
Good.  So as long as all source is rebuilt with mono we don't have to
worry about this issue.  DLLs and EXEs will contain CIL only.

>  The only thing left to determine whether an assembly is platform
>  specific is the use of Platform Invoke (P/Invoke), and this is _not_
>  trivial, as "portable" P/Invoke code can be written, but
>  platform-specific platform code can also be written.  Classic example:
>  `struct stat' in C#:
>  
>          public struct stat {
>                  public ulong st_dev, st_ino;
>                  public uint st_mode;
>                  private uint _padding_;
>                  public ulong st_nlink, st_uid, st_gid, st_rdev;
>                  public long st_size, st_blksize, st_blocks, st_atime,
>                          st_mtime, st_ctime;
>          }
>  
>  Question: is that structure definition portable for use with fstat(2)?
>  
>  Answer: Maybe. :-)  It should work on 64-bit ABIs (since long/ulong are
>  64-bit integer types), but it will fail badly on 32-bit ABIs, and it
>  will fail on any ABI in which the member ordering differs.
>  Consequently, such a structure would be platform specific if used with a
>  DllImport statement, but there is NO EASY WAY for a program to make this
>  determination. :-/
>  
>  (Plus, it can get more complicated, particularly if such a structure is
>  instead used with some native library which will accept that structure,
>  such as Mono.Posix.dll & libMonoPosixHelper.so, which uses a 64-bit ABI
>  throughout, and libMonoPosixHelper.so converts this to a 32-bit ABI as
>  appropriate.)
>  
>  So it's easiest to assume that all CIL code is platform agnostic.
>   
I don't see how the OTP came to this conclusion.  The example is
illustrating that there is no easy way to tell if CIL code is platform
specific or platform independent.  In the absence of knowing that, we
should be assuming the code is platform specific, not platform agnostic
as he suggests.

>  I would suggest looking at what SuSE and Debian/Ubuntu have done
> with
>  this, as they have all encountered these issues.  Following in their
>  footsteps would help consistency, and make it easier to repackage
>  existing programs.
>  
>  I believe that they currently use /usr/lib for everything.
>   
AFAIK Debian does not have multilib so our separation of %{_libdir}
from /usr/lib does not occur there.  I don't know about SuSE.

>  However, this gets considerably more complicated, as applications
> may
>  bundle native libraries with them -- take Blam! and F-Spot as examples: 
[snip
example]
>  The *.dll and *.exe files may be platform agnostic, but the *.so*
> files
>  certainly are NOT.
>  
>  So the easiest solution may be to check to see if any native libraries
>  are bundled with the app, and if that's the case stick it under the
>  appropriate /usr/lib or /usr/lib64 directory.  Otherwise, stick
>  with /usr/lib. 
Given that it is difficult to determine whether a .dll contains platform
specific or platform agnostic code, I'd argue that everything should go
to %{_libdir}.  My reading of Debian's switch from %{_datadir} to
%{_libdir} implies that they think the same way.

BTW: Here's Debian's guidelines.  All DLLs they reference goes to their
arch specific directory (Remember, they are non-multilib):

The package's applications, libraries and meta-data must be installed
into /usr/lib/packagename.

Libraries that will be installed into the GAC should be installed
into /usr/lib/cli/packagename-X.Y (for more details about the X.Y
version see GAC versioning). The commonly seen /usr/lib/mono/packagename
path should only be used for Mono project packages.

Never install native "glue" libraries into /usr/lib, instead install
them at /usr/lib/cli/packagename-X.Y. When moving libraries update the
references to the new location using a DLL Map. See the Mono DLL maps
secion for an example.

The only exception here is for native libraries that are of wider use;
can be used other packages. Native libraries should be packaged
according to the Library Packaging Guide in a Debain Policy conformant
way.

Never ever install application files (.exe) directly into /usr/bin.
Instead create a wrapper script into /usr/bin to allow them to be run
without the .exe suffix.


http://pkg-mono.alioth.debian.org/cli-policy/ch-packaging.html#s-file-loc...

-Toshio

[Ralf Corsepius]

On Tue, 2006-06-13 at 09:35 +0300, Ville Skyttä wrote:
>  On Mon, 2006-06-12 at 22:41 -0700, Toshio Kuratomi wrote:
>  
>  > Fedora's Python and Perl place arch independent data into /usr/lib,
>  
>  It can be argued that this is a packaging glitch 
A different view is: It's
generalization and simplification.

>   (which is hard to fix
>  in a backwards compatible manner) in both of them. 
What would it buy you? ... In
first place complexity ...

Ralf

[Ville Skyttä]

On Tue, 2006-06-13 at 09:43 +0200, Ralf Corsepius wrote:
>  On Tue, 2006-06-13 at 09:35 +0300, Ville Skyttä wrote:
>  > On Mon, 2006-06-12 at 22:41 -0700, Toshio Kuratomi wrote:
>  > 
>  > > Fedora's Python and Perl place arch independent data into /usr/lib,
>  > 
>  > It can be argued that this is a packaging glitch
>  A different view is: It's generalization and simplification. 
YMMV, but the way I read the FHS ("Architecture independent data goes
to /usr/share"), this is against it.  Ignoring backwards compatibility
issues, it would be hardly any more difficult to have them in /usr/share
than /usr/lib. 

>  >  (which is hard to fix
>  > in a backwards compatible manner) in both of them.
>  What would it buy you? ... In first place complexity ... 
FHS (and thus LSB) compliance, again the way I read it.  Whether it's
worth the pain of changing is another story, but IMO if there's no
historical baggage to take into account (such as possibly in the Mono
case there isn't), one should try to follow the standards.

[Ralf Corsepius]

On Tue, 2006-06-13 at 18:49 +0300, Ville Skyttä wrote:
>  On Tue, 2006-06-13 at 09:43 +0200, Ralf Corsepius wrote:
>  > On Tue, 2006-06-13 at 09:35 +0300, Ville Skyttä wrote:
>  > > On Mon, 2006-06-12 at 22:41 -0700, Toshio Kuratomi wrote:
>  > > 
>  > > > Fedora's Python and Perl place arch independent data into /usr/lib,
>  > > 
>  > > It can be argued that this is a packaging glitch
>  > A different view is: It's generalization and simplification.
>  
>  YMMV, but the way I read the FHS ("Architecture independent data goes
>  to /usr/share"), this is against it. 
Note: "data".

Which kind of data are we talking about? Scripts and arch-independent
executables aren't necessarily "data".

In my understanding, the FHS is talking about classical "data" files
(non-executables), such as "pictures" and the like.

Ralf

[Nicolas Mailhot]

Le mardi 13 juin 2006 à 18:10 +0200, Ralf Corsepius a écrit :

>  Which kind of data are we talking about? Scripts and
> arch-independent
>  executables aren't necessarily "data". 
Doesn't mean anything in english, the specific work we have in French
for doing computer stuff if often translated as data processing, so the
LSB author might just have wanted to write "Architecture independent
stuff" elegantly.

My personal reading is the authors of the spec wanted stuff which
couldn't be nfs-shared between systems with different architectures
in /usr/lib, and the rest in /usr/share. In that case the rest does
include the dlls.

The thing is not LSB vs how Red Hat used to do stuff like some people
try to put it, lots of Red Hat packagers have chosen for a long time to
put their arch-independant (byte)code in /usr/share :

- java bytecode in /usr/share/java
- lisp code in /usr/share/(x)emacs
- xslt code in /usr/share/sgml/ or /usr/share/xml
- php code in /usr/share/webappname (squirelmail for example)
- tcl code in /usr/share/tclfoo/

dssl, tex also comes to mind

The *vast* majority of noarch/interpreted languages have long
chosen /usr/share as home. Perl and python are the exception, probably
because they mix native with arch-independant code, and no one ever
asked their packagers to clean up their packaging style.

-- 
Nicolas Mailhot

[Nicolas Mailhot]

Le Mar 13 juin 2006 07:41, Toshio Kuratomi a écrit :

> > 3. Forcing local copies of DLL libraries instead of using system
> > installed DLLs.
> > This concept is documented upstream here:
> > http://www.mono-project.com/Assemblies_and_the_GAC#Libraries_with_Unstabl...
> > IMHO, this is a really stupid and braindead idea. In fact, I've yet to
> > talk to anyone who actually thinks this is a good idea. This is the
> > same
> > thing as static linking a private copy of a library instead of using
> > the
> > system shared lib. We do NOT permit this sort of behavior in Fedora,
> > and
> > mono is no exception.
>
>  Here here!
>
>  What are the easy ways to detect this issue? 
If upstream bundles dlls they don't even rebuild from sources, you can
steal a trick from the java packages and always end up %setup with a
blanket rm of DLL files

(won't take care of common stuff which *is* rebuilt, but a lot of the
times windoze developpers don't bother with this step when they do
bundling)

-- 
Nicolas Mailhot

[Toshio Kuratomi]

On Mon, 2006-06-12 at 22:41 -0700, Toshio Kuratomi wrote:
>  On Mon, 2006-06-12 at 18:23 -0500, Tom 'spot' Callaway
> wrote:
>  > 2. Putting DLLs in %{_datadir} vs %{_libdir}:
>  > 
>  > The FHS says that /usr/share (aka, %{_datadir}) is for "Architecture
>  > Independent Data". These DLL files do not seem to qualify as
>  > "Architecture Independent Data". I think that having them live in the:
>  > %{_libdir}/mono/ hierarchy seems to be the most appropriate for them
>  > at
>  > this point in time. Obviously, this can be revisited if upstream
>  > changes
>  > the default location. The gacutil command should be putting these DLL
>  > files in the right place.
>  > 
>  My understanding is .dlls and .exes are architecture independent. 
Ugh.  Sorry guys, I think my understanding of the architecture
independence of .dlls and .exes is mistaken:

[1] Miguel posts to the Debian list about problems with their packaging
guidelines, including the tidbit that although the current Mono .dlls
are compiled to CIL and therefore arch independent, they do not have to
be this way::

http://lists.alioth.debian.org/pipermail/pkg-mono-devel/2005-February/000...

[2] The old, problematic guidelines Miguel is critiquing.  Among other
things, it contains the assumptions about arch-independence that Miguel
says are wrong::
  http://wiki.debian.org/?MonoConventions

[3] A new Debian roadmap for mono integration written up shortly after
one of the Debian developers talked with Miguel via IRC.  This is short
and succinct with several useful tidbits::
  http://wiki.debian.org/?MonoDebianPlan

[4] The Debian mono packaging guidelines draft.  This is longer and a
work in progress.  I haven't finished plowing through it but it sounds
like Debian has been examining this issue and has started to
canonicalize some of their knowledge so it should be good fodder::
  http://pkg-mono.alioth.debian.org/cli-policy/

Given this, my questions change to::
* Do the Core mono packages belong in %{_libdir} (ie, /usr/lib64 on
x86_64 and /usr/lib on 32bit systems) rather than /usr/lib?

  - If so, is this a requirement before we can place Extras packages
into the proper, %{_libdir}, directories?

* Do all mono packages belong under %{_libdir}/mono or should there be
more flexibility?  How much? (Allow %{_libdir}/[PKGNAME]?  Allow
%{_datadir}/PKGNAME because upstream should know if their package is
truly arch independent?)

  - If we allow more flexibility (for instance, allowing nant to install
to %{_datadir}) how do we check that the .dlls and .exes are truly
platform independent?

  - Related questions I have after reading [3]: Do .dlls and .exes
become platform dependent simply by linking to C-language glue
libraries?  Do we need to consider all mono packages as being platform
dependent because of AOT-compile?  I tried the mentioned "grep -i
dllimport" test here and failed to come up with any matches so I
obviously don't fully understand that page.

* Once again, putting everything under %{_libdir}/mono would be easiest
for doing reviews but it may require us to do a lot of patching to get
things into the right directories.

FWIW, Debian installs Nant into /usr/lib/mono/ so it appears they have
decided patching upstream's install locations is the proper procedure.

http://packages.debian.org/cgi-bin/search_contents.pl?searchmode=filelist...

-Toshio

[Paul F. Johnson]

Hi,

>  Given this, my questions change to::
>  * Do the Core mono packages belong in %{_libdir} (ie, /usr/lib64 on
>  x86_64 and /usr/lib on 32bit systems) rather than /usr/lib? 
If I've read the replies so far, yes.

>    - If so, is this a requirement before we can place Extras packages
>  into the proper, %{_libdir}, directories? 
The problem is that for libraries, if something needs to link to it,
the .pc file isn't usually found if you've installed to the default 64
bit directory.

>  * Do all mono packages belong under %{_libdir}/mono or should there
> be
>  more flexibility?  How much? (Allow %{_libdir}/[PKGNAME]?  Allow
>  %{_datadir}/PKGNAME because upstream should know if their package is
>  truly arch independent?) 
should know and do know are not the same. For quite a few packages (such
as gtk-sharp), they are already in %{_libdir}/[PKGNAME]

>    - If we allow more flexibility (for instance, allowing nant to
> install
>  to %{_datadir}) how do we check that the .dlls and .exes are truly
>  platform independent? 
No idea - I would imagine in the same way as you would check something
written in Java. 

>  * Once again, putting everything under %{_libdir}/mono would be
> easiest
>  for doing reviews but it may require us to do a lot of patching to get
>  things into the right directories. 
It depends if we patch the spec file or the configure files. The spec is
much simpler than the configure scripts.

>  FWIW, Debian installs Nant into /usr/lib/mono/ so it appears they
> have
>  decided patching upstream's install locations is the proper procedure. 
I'd rather we come up with a FC specific solution than rely on two
conflicting points of view coming from the SuSE and Debian camps -
having used both, I can say that I wouldn't trust either to get things
even remotely right.

TTFN

Paul

-- 
ich liebe Ashleigh, eins zwei drei 
ich liebe Ashleigh, auf meinem Knee zu hüpfen

[Toshio Kuratomi]

On Tue, 2006-06-13 at 22:22 +0100, Paul wrote:
>  Toshio wrote: 
>  >   - If so, is this a requirement before we can place Extras
> packages
>  > into the proper, %{_libdir}, directories?
>  
>  The problem is that for libraries, if something needs to link to it,
>  the .pc file isn't usually found if you've installed to the default 64
>  bit directory.
>   
The .pc file isn't found by what?  pkg-config?  A configure script?
Some other program in the build process?  We have to know what's making
the mistakes in order to fix it.

>  > * Do all mono packages belong under %{_libdir}/mono or should
> there be
>  > more flexibility?  How much? (Allow %{_libdir}/[PKGNAME]?  Allow
>  > %{_datadir}/PKGNAME because upstream should know if their package is
>  > truly arch independent?)
>  
>  should know and do know are not the same. For quite a few packages (such
>  as gtk-sharp), they are already in %{_libdir}/[PKGNAME]
>   
Do you mean gtk-sharp2?
Do you mean it installs a helper app into /usr/lib/[PKGNAME]?

I'm being very careful to differentiate between /usr/lib and %{_libdir}
because /usr/lib has multiple roles on x86_64 in Fedora Core. (Place for
32bit libraries.  Repository for arch independent python and perl
libraries.)

>  >   - If we allow more flexibility (for instance, allowing nant to
> install
>  > to %{_datadir}) how do we check that the .dlls and .exes are truly
>  > platform independent? 
>  No idea - I would imagine in the same way as you would check
> something
>  written in Java. 
>   
Java jars are bytecode.  Native code is compiled into a .so.  From
reading the links I posted earlier, it appears that .dlls and .exes can
contain both arch independent and platform specific code.  So filename
extension is not an indicator.  I haven't found any indication that
file(1) knows the difference either.

-Toshio

[Paul F. Johnson]

Hi,

>   
>  > The problem is that for libraries, if something needs to link to it,
>  > the .pc file isn't usually found if you've installed to the default 64
>  > bit directory.
>  > 
>  The .pc file isn't found by what?  pkg-config?  A configure script?
>  Some other program in the build process?  We have to know what's making
>  the mistakes in order to fix it. 
I'm not actually sure, though I would imagine it's pkg-config. When I
run a spec file, it executes the configure script which itself calls
pkg-config for the libs.

>  > > * Do all mono packages belong under %{_libdir}/mono or
> should there be
>  > > more flexibility?  How much? (Allow %{_libdir}/[PKGNAME]?  Allow
>  > > %{_datadir}/PKGNAME because upstream should know if their package is
>  > > truly arch independent?)
>  > 
>  > should know and do know are not the same. For quite a few packages (such
>  > as gtk-sharp), they are already in %{_libdir}/[PKGNAME]
>  > 
>  Do you mean gtk-sharp2? 
Yes. Sorry. It's been a damned long day.

>  Do you mean it installs a helper app into /usr/lib/[PKGNAME]? 
It installs gapi2xml.pl, gapi_codegen.exe, gapi_fixup.exe,
gapi-parser.exe, gapi_pp.pl and gconfsharp-schemagen.exe

>  I'm being very careful to differentiate between /usr/lib and
> %{_libdir}
>  because /usr/lib has multiple roles on x86_64 in Fedora Core. (Place for
>  32bit libraries.  Repository for arch independent python and perl
>  libraries.) 
Thanks - I knew there was something special about /usr/lib on 64 bit,
but never actually what it was!

>  > >   - If we allow more flexibility (for instance, allowing
> nant to install
>  > > to %{_datadir}) how do we check that the .dlls and .exes are truly
>  > > platform independent?
>  
>  > No idea - I would imagine in the same way as you would check something
>  > written in Java. 
>  > 
>  Java jars are bytecode.  Native code is compiled into a .so.  From
>  reading the links I posted earlier, it appears that .dlls and .exes can
>  contain both arch independent and platform specific code.  So filename
>  extension is not an indicator.  I haven't found any indication that
>  file(1) knows the difference either. 
I'm actually there is a safe way to do it other than having a 64 bit box
with everything 64 bit on it and see if the 32 bit binary works. Not a
very good idea though - too many false positives.

I've looked at monodis, but that won't do the job and can't find
anything on google either.

TTFN

Paul
-- 
ich liebe Ashleigh, eins zwei drei 
ich liebe Ashleigh, auf meinem Knee zu hüpfen

[Ralf Corsepius]

On Tue, 2006-06-13 at 15:28 -0700, Toshio Kuratomi wrote:

>  I'm being very careful to differentiate between /usr/lib and
> %{_libdir}
>  because /usr/lib has multiple roles on x86_64 in Fedora Core. (Place for
>  32bit libraries.  Repository for arch independent python and perl
>  libraries.) 
Well, actually you have to distinguish between
/usr/lib
and the multilib dirs, which actually are
/usr/lib/.
and
/usr/lib/../lib64

Ralf

[Tom Callaway]

On Mon, 2006-06-12 at 22:41 -0700, Toshio Kuratomi wrote:

>  My understanding is .dlls and .exes are architecture independent.
This doesn't sound right to me. That doesn't mean that it isn't right,
it just sounds weird.

file says that the mono dll files are:
MS-DOS executable PE  for MS Windows (DLL) (console) Intel 80386 32-bit

and file thinks that the mono exe files are:
MS-DOS executable PE  for MS Windows (console) Intel 80386 32-bit

Now, my understanding is that mono uses these DLL files as libraries,
the EXE files as binaries. Are they compiled from source, or provided as
is for use? If they're not compiled from source, this seems like a
violation of Fedora policies (no binary only bits, please). 

Presuming that they are compiled from source, they should be
architecture specific. I did not think that C# was an interpreted
language like python/perl... is it more like Java (runtime)?

>  Should they end up in %{_libdir} or always end up in /usr/lib? 
> Fedora's
>  Python and Perl place arch independent data into /usr/lib, (I assume so
>  that they can be noarch) should mono as well?  
Well, there are several questions there:

1. Are the DLLs and EXEs truly architecture independent?
2. Are the EXEs used like binaries? (They certainly seem to be)
3. How much will these mono apps break if we move the EXE location to
%{_bindir}? (In my tests with tomboy, it seems to work fine if i move
the EXE to %{_bindir} and edit the /usr/bin/tomboy wrapper script)
4. How much will mono (and its app) go nuts if we move the DLLs to
to /usr/share ?
4. How much will upstream (and unpackaged mono apps built from source)
go nuts if we move our packaged mono bits out of the fake /usr/lib tree?

Let's leave perl and python out of this for the time being and focus on
mono, where we have the opportunity to do things "correctly" early on,
without several years of baggage and precedence.

>   All the *-sharp packages
>  on my system currently follow this convention but beagle, tomboy, and
>  banshee end up in %{_libdir}.  
Well, to be more specific, they end up in: %{_libdir}/%{name}


>  If we separate applications and libraries, how do we determine
>  which .dlls belong in an application directory and which belong in a
>  library directory?  
I'm hoping this is as clear cut as "DLL == library, EXE == binary".

>  What are the easy ways to detect this issue? 
Man, I really don't think there are easy ways to detect this. Is a
package including a DLL file that exists in another package?

>  1) Upstream tarball contains .dlls that were not rebuilt from source
>  contained in the package. 
Yeah, thats a huge freaking no-no. That alone should invalidate the
package for Fedora inclusion.

>  2) Look through the installed .dlls for any that have the same name
> as
>  system .dlls or are suspiciously out of place (Package is myDiary and
>  contains mysql.dll, sqlite.dll, and gtk-sharp.dll) 
Yeah. It is worth noting that banshee seems to be doing this:

/usr/lib/banshee/hal-sharp.dll

But I don't see anything else using mono(hal-sharp).

>  3) Source directories look odd:
>  PKGNAME/
>    src/
>    data/
>    libs/
>      gtk-sharp/
>      atk-sharp/
>  
>  Anything else? 
Yeah, not that I can think of.

Mmmm, this gives a new meaning to DLL hell. Thoughts?

~spot
-- 
Tom "spot" Callaway: Red Hat Senior Sales Engineer || GPG ID: 93054260
Fedora Extras Steering Committee Member (RPM Standards and Practices)
Aurora Linux Project Leader: http://auroralinux.org
Lemurs, llamas, and sparcs, oh my!

[Paul F. Johnson]

Hi,

>  Now, my understanding is that mono uses these DLL files as
> libraries,
>  the EXE files as binaries. Are they compiled from source, or provided as
>  is for use? If they're not compiled from source, this seems like a
>  violation of Fedora policies (no binary only bits, please).  
From what I've packaged, everything is source built.

>  Presuming that they are compiled from source, they should be
>  architecture specific. I did not think that C# was an interpreted
>  language like python/perl... is it more like Java (runtime)? 
Yes.

>  > Should they end up in %{_libdir} or always end up in /usr/lib? 
> Fedora's
>  > Python and Perl place arch independent data into /usr/lib, (I assume so
>  > that they can be noarch) should mono as well? 
>  
>  Well, there are several questions there:
>  
>  1. Are the DLLs and EXEs truly architecture independent? 
No. I have code which identifies itself as 64 bit and 32 bit.

>  2. Are the EXEs used like binaries? (They certainly seem to be)
Yes. The are effectively binaries.

>  3. How much will these mono apps break if we move the EXE location
> to
>  %{_bindir}? (In my tests with tomboy, it seems to work fine if i move
>  the EXE to %{_bindir} and edit the /usr/bin/tomboy wrapper script) 
It depends on the app. monodevelop hates being moved.

>  4. How much will mono (and its app) go nuts if we move the DLLs to
>  to /usr/share ? 
A lot - unless you specify to look somewhere, the likes of MD goes
bizerk.

>  4. How much will upstream (and unpackaged mono apps built from
> source)
>  go nuts if we move our packaged mono bits out of the fake /usr/lib tree? 
That's less of a problem IMO. I don't see them going loopy over the
Debian or Mandriva bods.

>  > If we separate applications and libraries, how do we determine
>  > which .dlls belong in an application directory and which belong in a
>  > library directory? 
>  
>  I'm hoping this is as clear cut as "DLL == library, EXE == binary". 
Speaking with some of my mates on the darkside over the last couple of
days, they refer to the exe as binaries and dlls as libs. Okay, that
could just be for ease of conversation...

>  > What are the easy ways to detect this issue?
>  
>  Man, I really don't think there are easy ways to detect this. Is a
>  package including a DLL file that exists in another package? 
Quite probably, yes.

>  > 1) Upstream tarball contains .dlls that were not rebuilt from
> source
>  > contained in the package.
>  
>  Yeah, thats a huge freaking no-no. That alone should invalidate the
>  package for Fedora inclusion. 
Sometimes though they have to do that to build the binary which then
rebuilds the source (sort of chicken and egg)

>  Mmmm, this gives a new meaning to DLL hell. Thoughts? 
Welcome to the wonderful world of Microsoft lads.

Seriously, this is one of the problems of .NET that people scream about
on Win32 platforms and there doesn't seem to be anything that can be
done as the code has to work in exactly the same way irrespective of
platform (the CIL does the grunt work). Upshot, they have to have the
directory structure they have - the app looks in a particular directory
to try and find the dll, but there is nothing in the system that says
look in the common area.

gac is a nice attempt to get around this, but even that is only so far.

TTFN

Paul
-- 
ich liebe Ashleigh, eins zwei drei 
ich liebe Ashleigh, auf meinem Knee zu hüpfen

[Toshio Kuratomi]

On Tue, 2006-06-13 at 19:25 -0500, Tom 'spot' Callaway wrote:
>  On Mon, 2006-06-12 at 22:41 -0700, Toshio Kuratomi wrote:
>  
>  > My understanding is .dlls and .exes are architecture independent.
>  
>  This doesn't sound right to me. That doesn't mean that it isn't right,
>  it just sounds weird.
>   
If you've gotten to my latest message, you'll see I'm partially wrong
about this.

>  file says that the mono dll files are:
>  MS-DOS executable PE  for MS Windows (DLL) (console) Intel 80386 32-bit
>  
>  and file thinks that the mono exe files are:
>  MS-DOS executable PE  for MS Windows (console) Intel 80386 32-bit
>   
I believe that most of those dll and exe's are compiled to CIL (Common
Intermediate Language) which is then run by the CLR (in our case, mono)
similar to how java bytecode is run on the java runtime.

The PE is a container for the CIL vaguely similar to a jar file being a
renamed zip.

>  Now, my understanding is that mono uses these DLL files as
> libraries,
>  the EXE files as binaries. Are they compiled from source, or provided as
>  is for use? If they're not compiled from source, this seems like a
>  violation of Fedora policies (no binary only bits, please). 
>   
In the boo package review there was one revision where the upstream
tarball was all precompiled .dlls and .exes.  After pointing this out,
the packager (Paul Johnson) found the tarball with source and packaged
that instead.  If it's common for upstream projects to distribute both
source and binary versions of their mono packages we'll have to be
vigilant about checking the source tarballs really contain sources and
not precompiled CIL PEs that install with make install.

>  Presuming that they are compiled from source, they should be
>  architecture specific. I did not think that C# was an interpreted
>  language like python/perl... is it more like Java (runtime)?
>   
It's similar to Java byte code.  However, if I'm reading Miguel's
post
to the Debian list correctly, the PE's can become non-portable if they
consume functions from native libraries.  I'm unclear whether this is
something that happens when the developer writes the code or when it is
linked.

>  > Should they end up in %{_libdir} or always end up in /usr/lib? 
> Fedora's
>  > Python and Perl place arch independent data into /usr/lib, (I assume so
>  > that they can be noarch) should mono as well? 
>  
>  Well, there are several questions there:
>  
>  1. Are the DLLs and EXEs truly architecture independent? 
At this point, the answer
is, they may or may not be and I haven't run
across an easy way to check yet.  This is one of the reasons Debian has
changed their mind about putting them into %{_datadir}.

The other reeason is that the .dlls can be precompiled to native code to
make startup quicker.  It seems that at the time Debian changed away
from %{_datadir}, these precompiled libraries had to be placed alongside
the .dlls.  This would obviously be wrong for %{_datadir}.

>  2. Are the EXEs used like binaries? (They certainly seem to be)
Yes.  But the standard usage seems to be to include a script that is
installed to %{_bindir} and invoked the application where it resides
with the rest of the code.  I know of some python applications (rpmlint)
and java applications (azureus) that do the same thing.

>  3. How much will these mono apps break if we move the EXE location
> to
>  %{_bindir}? (In my tests with tomboy, it seems to work fine if i move
>  the EXE to %{_bindir} and edit the /usr/bin/tomboy wrapper script) 
I don't
know.  It's interesting to note that Debian was placing the
EXE's into %{_datadir} and writing a wrapper script to place in
%{_bindir} before they started discussing this with Miguel.  In that
discussion they found that upstream projects were being encouraged to
write their own wrappers and put the EXEs into /usr/lib (I don't know if
this was %{_libdir} or /usr/lib).

Note: I just moved banshee.exe to /var/tmp and changed /usr/bin/banshee
to point at it.  /usr/bin/banshee fails with an error that it could not
find Banshee.Base.  Banshee.Base.dll lives in %{_libdir}/banshee/ which
is normally where banshee.exe would be.

>  4. How much will mono (and its app) go nuts if we move the DLLs to
>  to /usr/share ? 
Since it seems there's no good way to tell if a .dll is
dependent on the
arch or not, I'm withdrawing the idea of moving them to %{_datadir} so
this question is now moot.  I presently think they should go to
%{_libdir} (rather than /usr/lib where Core's mono package currently
lives.)

>  4. How much will upstream (and unpackaged mono apps built from
> source)
>  go nuts if we move our packaged mono bits out of the fake /usr/lib tree?
>   
This mailing list post[1]_ says that PLD has been using %{_libdir} since
before 2005 and feels that upstream accepts their position.  Debian was
using %{_datadir} and Miguel gave them reasons he thought that was
wrong.  They seem to have changed to using /usr/lib but I'm unclear
whether /usr/lib == %{_libdir} or the way we use /usr/lib in Fedora
multilib (does Debian have multilib?)

[1]_
http://lists.pld-linux.org/mailman/pipermail/pld-devel-en/2005-February/0...

>  Let's leave perl and python out of this for the time being and
> focus on
>  mono, where we have the opportunity to do things "correctly" early on,
>  without several years of baggage and precedence.
>   
True.  Although we'd still have upstream to shift to a new location.

>  >  All the *-sharp packages
>  > on my system currently follow this convention but beagle, tomboy, and
>  > banshee end up in %{_libdir}. 
>  
>  Well, to be more specific, they end up in: %{_libdir}/%{name}
>   
True.

>  
>  > If we separate applications and libraries, how do we determine
>  > which .dlls belong in an application directory and which belong in a
>  > library directory? 
>  
>  I'm hoping this is as clear cut as "DLL == library, EXE == binary".
>   
I think my Banshee experiment shows this is harder than that.  My guess
is if we separate banshee.exe from the DLLs it's intended to reside in
the same directory with, then those DLLs need to be registered with the
GAC.  If we do that are we placing something that isn't intended to be
shared into a shared area?

>  > What are the easy ways to detect this issue?
>  
>  Man, I really don't think there are easy ways to detect this. Is a
>  package including a DLL file that exists in another package?
>  
>  > 1) Upstream tarball contains .dlls that were not rebuilt from source
>  > contained in the package.
>  
>  Yeah, thats a huge freaking no-no. That alone should invalidate the
>  package for Fedora inclusion.
>  
>  > 2) Look through the installed .dlls for any that have the same name as
>  > system .dlls or are suspiciously out of place (Package is myDiary and
>  > contains mysql.dll, sqlite.dll, and gtk-sharp.dll)
>  
>  Yeah. It is worth noting that banshee seems to be doing this:
>  
>  /usr/lib/banshee/hal-sharp.dll
>  
>  But I don't see anything else using mono(hal-sharp).
>  
>  > 3) Source directories look odd:
>  > PKGNAME/
>  >   src/
>  >   data/
>  >   libs/
>  >     gtk-sharp/
>  >     atk-sharp/
>  > 
>  > Anything else?
>  
>  Yeah, not that I can think of.
>  
>  Mmmm, this gives a new meaning to DLL hell. Thoughts? 
Give me back my Linux :-)

My current gut feeling is that all EXE and DLLs belong in %{_libdir}
rather than /usr/lib.  DLLs that are meant to be used only as part of a
single package belong with the EXE.  The EXE + DLLs go in
%{_libdir}/[PKGNAME].  Other DLLs (I'm assuming all other dlls are meant
to be used publically) belong in %{_libdir}/mono/ as part of the GAC.
This seems reviewable and hopefully does not require too much work to
make upstream packages conform.

Reviewing packages for suspect DLLs is going to be hard.  Eventually we
might be able to compile a list of known dlls that we have on the system
and so shouldn't be included in a package but it's going to be an uphill
battle as new libraries (and new versions of libraries) are created.
Convincing the mono-project that their recommendation has security
implications and they should consider withdrawing it may help.

-Toshio

[Nicolas Mailhot]

Le Mer 14 juin 2006 09:20, Toshio Kuratomi a écrit :

>  In the boo package review there was one revision where the upstream
>  tarball was all precompiled .dlls and .exes.  After pointing this out,
>  the packager (Paul Johnson) found the tarball with source and packaged
>  that instead.  If it's common for upstream projects to distribute both
>  source and binary versions of their mono packages we'll have to be
>  vigilant about checking the source tarballs really contain sources and
>  not precompiled CIL PEs that install with make install. 
I believe that since .Net cloned a big part of Java, Java and Mono target
the same kind of windows developper with no rpm of FHS background, both
mono and java upstreams are pretty close WRT the packaging problems they
cause.

If the java and mono Red Hat / Fedora packagers are not religious about
the langage bit, they may find more than worth their time cooperating on
common packaging solutions

-- 
Nicolas Mailhot

[Paul F. Johnson]

Hi,

>  In the boo package review there was one revision where the upstream
>  tarball was all precompiled .dlls and .exes.  After pointing this out,
>  the packager (Paul Johnson) found the tarball with source and packaged
>  that instead.  If it's common for upstream projects to distribute both
>  source and binary versions of their mono packages we'll have to be
>  vigilant about checking the source tarballs really contain sources and
>  not precompiled CIL PEs that install with make install. 
The boo packagers made a boo-boo with 0.7.6-2334 which was fixed in 2337
and that mistake was they didn't include the source.

TTFN

Paul
-- 
"Logic, my dear Zoe, is merely the ability to be wrong with authority" -
Dr Who

[Paul F. Johnson]

Hi,

>  1. Redefining libdir for mono packages:
>  
>  If mono and its tools cannot find a library on a 64bit architecture when
>  it lives in %{_libdir} (/usr/lib64), then mono is fundamentally broken
>  on that architecture, and needs to be fixed. This is a serious flaw, and
>  I will not have us doing ugly hacks in spec files to work around mono's
>  ineptitude. Every other core component is able to handle this, this bug
>  should be filed and fixed. 
I've moaned about this until I'm blue in the face on the mono-devel
lists for almost 8 months now (basically since I moved to 64 bit
architecture) and there doesn't seem to be any real desire to do
anything.

I think the problem has been disguised by how SuSE package mono
applications. In their somewhat insane spec files, they define all mono
packages as being noarch, which means everything goes into /usr/lib
rather than anywhere else.

This, to me, is as bad a hack as my libdir one.

I have a feeling that the only way it will get fixed is if one of us
sits down and fixes the problem. It could possibly be just a quick
change to the configure.ac scripts so that /usr/lib is set as libdir in
there.

Thanks for having a look Tom, much appreciated. Hope the hotel room
isn't that bad...

TTFN

Paul
-- 
ich liebe Ashleigh, eins zwei drei 
ich liebe Ashleigh, auf meinem Knee zu hüpfen

[Ian Burrell]

On 6/12/06, Paul <paul(a)all-the-johnsons.co.uk&gt; wrote:
>  Hi,
>
>  > 1. Redefining libdir for mono packages:
>  >
>  > If mono and its tools cannot find a library on a 64bit architecture when
>  > it lives in %{_libdir} (/usr/lib64), then mono is fundamentally broken
>  > on that architecture, and needs to be fixed. This is a serious flaw, and
>  > I will not have us doing ugly hacks in spec files to work around mono's
>  > ineptitude. Every other core component is able to handle this, this bug
>  > should be filed and fixed.
>
>  I've moaned about this until I'm blue in the face on the mono-devel
>  lists for almost 8 months now (basically since I moved to 64 bit
>  architecture) and there doesn't seem to be any real desire to do
>  anything.
>
>  I think the problem has been disguised by how SuSE package mono
>  applications. In their somewhat insane spec files, they define all mono
>  packages as being noarch, which means everything goes into /usr/lib
>  rather than anywhere else.
>
>  This, to me, is as bad a hack as my libdir one.
>
>  I have a feeling that the only way it will get fixed is if one of us
>  sits down and fixes the problem. It could possibly be just a quick
>  change to the configure.ac scripts so that /usr/lib is set as libdir in
>  there.
>  
It sounds like what is needed is another variable which determines
where the arch-independent code for mono goes.  libdir would have its
normal arch-dependent meaning, /usr/lib or /usr/lib64.  The new
variable, lets call it monodir for example, would always be
/usr/lib/mono.

There will always be packages which use libdir for installing shared
libraries and mono code and using libdir for both purposes will always
create problems.

 - Ian

[Toshio Kuratomi]

Here's a recap of the blocker issues as I see them presently standing:

On Mon, 2006-06-12 at 18:23 -0500, Tom 'spot' Callaway wrote:
>  1. Redefining libdir for mono packages: 
>  If mono and its tools cannot find a library on a 64bit architecture
> when
>  it lives in %{_libdir} (/usr/lib64), then mono is fundamentally broken
>  on that architecture, and needs to be fixed. 
I think we're in agreement about this.  But Paul Johnson (Nodoid) has
run into some issues with this and making the packages noarch.  A look
at the lat approval:
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=177580

and the package in cvs shows that this is likely an overzealous
configure.ac.  Paul Howarth should ask upstream why they are using
AC_CANONICAL_TARGET in their configure.ac.  AFAICT there is no reason
for it.  Removing it from the spec and regenerating configure means that
we do not have to play tricks with either %{_libdir} or
%{_target_platform}.

(Diff to lat.spec as attachment #1)

>  
>  2. Putting DLLs in %{_datadir} vs %{_libdir}:
>   
A) DLLs are mostly architecture independent.  Testing of DLLImport
appears to show that code that is not architecture independent has to be
rewritten or it will fail at runtime on unsupported architectures.  A
simple recompile won't fix things.  The simple test case that I used was
a C library with a structure that contained an unsigned long type.  In
C, this can be a 32 bit or 64 bit value depending on the platform it is
compiled on.  In mono this structure is defined as either ulong or uint
for 64bit or 32bit respectively.  The mono code will fail to work
correctly on the platform it is not coded for.  Either the C library has
to be coded to explicitly use a 32bit or a 64bit integer or the mono
library will have to detect the number of bits at runtime and translate
its values correctly before passing to C (I don't know how this is done
but I've read that Mono's POSIX library does this.)

I have no objection to DLLs being noarch due to this.

Attachment #2 is a tarball of the code for my test case.  Run make on an
x86_64.  The resulting files will run fine.  Run make on an ix86.  The
resulting files will generate incorrect answers.

B) Using Ahead-of-time compilation currently has to place the .so files
in the same directory as the DLLs.  Putting those into /usr/share is
plainly wrong.  We have two options: i) Explicitly do not support ahead
of time compilation until mono can be enhanced to recognize an aot .so
in a different directory [Is this in upstream's roadmap?] ii) Put DLLs
in arch specific directories (somewhere under %{_libdir}).

To test I used nant as my test program.  This could probably be done
with tomboy/banshee/etc as  well.
1) cp -pr /usr/share/nant ~/nant
2) export MONO_PATH=$MONO_PATH:~/nant:~/nant/lib:~/nant/lib/mono:\
~/nant/lib/mono/1.0:~nant/lib/mono/2.0:~/nant/lib/net/1.0:\
~/nant/lib/net/1.1:~/nant/lib/net/2.0
3) cd ~/nant
4) sed 's^/usr/share/NAnt/bin^$HOME/nant^' < /usr/bin/nant > nant
5) mono --aot ./nant
6) strace ./nant &> aot-in-same-dir.log
7) strace /usr/bin/nant &> aot-in-monpath.log
8) Compare the two logs and notice that running our copy of nant that
uses dlls from ~/nant finds the .dll.so files there.  The system nant
does not even though the .dll.so's are in MONO_PATH.

>  3. Forcing local copies of DLL libraries instead of using system
>  installed DLLs. 
Everyone is against this.  So far, we've only discovered one possible
instance of this in banshee.  We have a list of possible ways to check
for this but it isn't foolproof.  We would like the monoproject to
rethink their position.

-Toshio

[Nicolas Mailhot]

Le dimanche 18 juin 2006 à 23:39 -0700, Toshio Kuratomi a écrit :
>  Put DLLs
>  in arch specific directories (somewhere under %{_libdir}). 
This is what Java does

-- 
Nicolas Mailhot

[Toshio Kuratomi]

On Mon, 2006-06-19 at 08:55 +0200, Nicolas Mailhot wrote:
>  Le dimanche 18 juin 2006 à 23:39 -0700, Toshio Kuratomi a écrit :
>  > Put DLLs
>  > in arch specific directories (somewhere under %{_libdir}).
>  
>  This is what Java does
>   
I think this is a little different from java (Note: You are a java
packager and I'm not, so correct me if I'm wrong.)  In java, the jars go
into %{_javadir} which is defined as %{_datadir}/java.  Most jars are
compiled to native code which goes somewhere under %{_libdir}.

Because mono is only checking for precompiled DLLs in the same directory
as the DLL you can't separate the DLLs from the dll.so's as you can JARS
from jar.so's.

-Toshio