[Fedora-packaging] Re: python packaging - icons/desktop files and /usr/bin accessibility

[Toshio Kuratomi]

Mary Ellen Foster wrote:
> On Mon, May 5, 2008 at 10:54 PM, Toshio Kuratomi <a.badger at gmail.com> wrote:
>>  I took a closer look at Ice and I think there's a few things that should
>> change.
>>
>>  1) AFAICS there's no reason for the python bindings to be part of the Ice
>> package.  It comes in a separate tarball and appears to build on its own.  I
>> can't find a mention of this in the review so I don't know if there's
>> something special about it that I'm missing. [1]_
> 
> Actually, upstream is very soon releasing a new version of Ice where
> all of the language bindings are in a single tarball, so I think I'll
> stick with the monolithic SRPM.
>
Okay.  That's not something I'd have done but I'm not upstream :-/ 
Choosing to follow upstream on your part is sane.

>>  2) The .pth file looks like it's being used to compensate for bad imports
>> and upstream packaging.
>>   * There should be an __init__.py file in
>> %{_libdir}/pythonX.Y/site-packages/Ice
>>
>>   * The following files have imports that need to be changed:
>>     IceBox/__init__.py
>>     Glacier2/__init__.py
>>     IceGrid/__init__.py
>>     IceStorm/__init__.py
>>     IcePatch2/__init__.py
>>
>>     change imports like::
>>       import IceBox_IceBox_ice
>>     to this::
>>       from Ice import IceBox_IceBox_ice
> 
> The problem is, those files are automatically generated using other
> parts of the Ice program (using slice2py, to be specific). There's a
> documented way that slice2py deals with packages and modules
> (http://zeroc.com/doc/Ice-3.2.1/manual/Python.23.15.html#75777) that
> I'm reluctant to touch.
> 
> I could remove the "Ice.pth" file and require people to set
> PYTHONPATH, but unfortunately I don't think I can go changing the
> actual (generated) __init__.py files. :(
> 
PYTHONPATH would be wrong for a module, you're correct.  Let me look at 
slice2py to see if we can submit a patch upstream for that program to do 
things right.  Reading the documentation makes it more apparent that 
using a .pth is wrong... If ice becomes popular there will be a lot of 
namespace pollution due to that.  Their example of:

   module OS {
     [...]
   }

coupled with a .pth file quickly makes you realize that it would be very 
easy for an ice generated module to conflict with other python modules 
if they are not given their own namespace.

ie: without a .pth file, you'd have to do this::
   import ice.os

with the .pth file this can quickly happen::
   import os

Now, is that the ice os module or the python standard library os module?

>>  3) Additionally, on x86_64, the python module is spread over two
>> directories: /usr/lib64/python2.5/site-packages/Ice and
>> /usr/lib/python2.5/site-packages/Ice.  Python doesn't handle that very well.
>> All the files of a module should be in a single directory.  Since this
>> module builds an ELF shared object (IcePy.so/IcePy.so.3.2.1) all files
>> should be installed into %{python_sitearch}/  (This is what lead to BZ
>> #392751).
> 
> Okay, sure, that's easy to fix. So I should put *.py* into site_arch
> always, and put nothing into site_lib? 

For ice or any other module that has a compiled object (.so) as part of 
the module, yes.  Basically, you can create a module in 
%{pythonsite_arch}/MODULE *or* %{pythonsite_lib}/MODULE.  Trying to 
create pieces of the module in both will just confuse the python 
interpreter.

Most of the python modules that are bindings for C libraries (for 
instance, psycopg2 which provides bindings for the postgresql libs) end 
up in sitearch for that reason even if they have significant pieces 
written in pure python.  Other packages (for instance yum combined with 
yum-metadata-parser) split the pure python and compiled portions into 
two separate modules.  The pure python module can then reside in sitelib 
and import the compiled module from sitearch.

> The Python packaging guidelines
> aren't amazingly clear on this point to a non-Python programmer like
> me. :)
> 
These monolithic packages end up harder to maintain because of that... 
you need to have domain specific knowledge about all the languages and 
quirks involved rather than just one but if upstream's moving to a 
single tarball, then it's not an issue we can easily solve in distro 
packaging.

If you would like to make this clearer, feel free to jot down some 
wording and tell me where it would make the most sense to appear in the 
Guidelines.  Additions to clarify things are always welcome.

-Toshio