As many of you may be aware, FPC has been fielding several tickets surrounding python packaging lately, and the last one was a sizable reorganization that unfortunately was based on an older version of the guidelines. While merging it all together I just said screw it and just cleaned up the page like I've always wanted to do.
I suggest anyone who hasn't lost sanity due to this whole guidelines revision process pile onto: https://fedorahosted.org/fpc/ticket/552#comment:4
Here's the contents of that comment to save you a click.
OK, this was a lot of stuff, and the more I worked on it, the more annoyed I became. Plus the wiki kept eating my edits and I kept getting lost in the overhuge page. So, I've probably gone too far in cleaning up but I'll present what I have and if people yell at me then I'll try something less ambitious.
There are now three pages: https://fedoraproject.org/wiki/User:Tibbs/PythonCleanup2 https://fedoraproject.org/wiki/Packaging:Python_F21 https://fedoraproject.org/wiki/User:Tibbs/PythonAppendix
The first page incorporates the following changes: * Notes that these guidelines apply only to F22+ and EPEL7. Points at the 2nd page for old guidelines. * Includes the information about not retiring python-version-specific subpackages in stable releases. * Makes BuildRequires: section much more succinct. * Makes sure python2-foo provide is versioned, mentions %python_provide macro. * Makes the macro table collapsible and collapsed by default. * Remove EL6-specific cruft (it's in the old guidelines page). * Move the parts of the byte compilation section that nobody ever uses off to the Appendix page. * Make the single-rpm single-dir case the default. Move the entire multiple-dir case to the appendix. * Completely remove %with_python3 from the example spec. We want people to build for python3. People just paste this in without knowing why they would ever need it. If you're using the new macros you can't just conditionalize for EL6 anyway. This really makes the spec look nice. Macro-izing the summary and description would make it look even nicer once the description gets longer than a line. * Uses a generic "example" module as an example. Two different approved proposals changed it to two different things; I just punted. * Puts the example spec all together without a bunch of text and admons and whatnot in the middle. * Moves the 2to3 section to the appendix. * Simply refers to the eggs section. That whole part at the bottom was quite awkward and I don't think most people even looked down that far. * Loads and loads of typo fixes, grammar fixes, and cleanups.
The current main guideline page is now a bit over half the length it was and on the page it's less because of the collapsed macro table. I find it to be far more readable, and the sample spec no longer turns my stomach.
The old guidelines page is just as it was before writing up the new macros, except that I added a short explanatory section at the top.
The appendix page conveniently holds things which we probably should document but pretty much nobody would actually want to read without some specific reason.
Functionally I do not believe I have gone beyond anything upon which we voted but I wanted to toss this out there and see if anyone yells before I copy it into place. Been trying to get this done for three days now and I think I'm finally there.
- J<
----- Original Message -----
As many of you may be aware, FPC has been fielding several tickets surrounding python packaging lately, and the last one was a sizable reorganization that unfortunately was based on an older version of the guidelines. While merging it all together I just said screw it and just cleaned up the page like I've always wanted to do.
I suggest anyone who hasn't lost sanity due to this whole guidelines revision process pile onto: https://fedorahosted.org/fpc/ticket/552#comment:4
Here's the contents of that comment to save you a click.
OK, this was a lot of stuff, and the more I worked on it, the more annoyed I became. Plus the wiki kept eating my edits and I kept getting lost in the overhuge page. So, I've probably gone too far in cleaning up but I'll present what I have and if people yell at me then I'll try something less ambitious.
There are now three pages: https://fedoraproject.org/wiki/User:Tibbs/PythonCleanup2 https://fedoraproject.org/wiki/Packaging:Python_F21 https://fedoraproject.org/wiki/User:Tibbs/PythonAppendix
<snip>
Functionally I do not believe I have gone beyond anything upon which we voted but I wanted to toss this out there and see if anyone yells before I copy it into place. Been trying to get this done for three days now and I think I'm finally there.
Hi Jason, thanks for doing this, excelent work! I have only three comments: - I'm not sure what the "%{python_provides:%python_provide python2-%{pypi_name}}" are supposed to do, did you mean to use "%{?python_provide:...}" (i.e. without the "s" and with question mark)? - My last point from comment [1], later described more at [2], still deserves more discussion, I'd like to hear opinions of other folks as well. - Same applies to the fourth point from my comment at [1].
Slavek
- J<
[1] https://fedorahosted.org/fpc/ticket/281#comment:29 [2] https://fedorahosted.org/fpc/ticket/281#comment:32
"BK" == Bohuslav Kabrda slavek@redhat.com writes:
BK> comments: BK> - I'm not sure what the "%{python_provides:%python_provide BK> python2-%{pypi_name}}" are supposed to do, did you mean to use BK> "%{?python_provide:...}" (i.e. without the "s" and with question BK> mark)?
I just pasted those from one of the drafts, so I guess the original was screwed up. There were several instances of this and I guess I didn't fix them all.
BK> - My last point from comment [1], later described more at [2], still BK> deserves more discussion, I'd like to hear opinions of other folks BK> as well.
I think if I were trying to do functional changes (instead of structural ones like moving things to the appendix and fixing grammar) I would either leave this unspecified or indeed mandate that the python3 version be the default. It wasn't, though, my desire to change anything functionally that hadn't had a vote. I'm just unclear as to whether that actually was included in any of the changes I was trying to merge.
- J<
On 28.7.2015 07:02, Jason L Tibbitts III wrote:
As many of you may be aware, FPC has been fielding several tickets surrounding python packaging lately, and the last one was a sizable reorganization that unfortunately was based on an older version of the guidelines. While merging it all together I just said screw it and just cleaned up the page like I've always wanted to do.
I suggest anyone who hasn't lost sanity due to this whole guidelines revision process pile onto: https://fedorahosted.org/fpc/ticket/552#comment:4
Here's the contents of that comment to save you a click.
OK, this was a lot of stuff, and the more I worked on it, the more annoyed I became. Plus the wiki kept eating my edits and I kept getting lost in the overhuge page. So, I've probably gone too far in cleaning up but I'll present what I have and if people yell at me then I'll try something less ambitious.
There are now three pages: https://fedoraproject.org/wiki/User:Tibbs/PythonCleanup2 https://fedoraproject.org/wiki/Packaging:Python_F21 https://fedoraproject.org/wiki/User:Tibbs/PythonAppendix
The first page incorporates the following changes:
- Notes that these guidelines apply only to F22+ and EPEL7. Points at the 2nd page for old guidelines.
- Includes the information about not retiring python-version-specific subpackages in stable releases.
- Makes BuildRequires: section much more succinct.
- Makes sure python2-foo provide is versioned, mentions %python_provide macro.
- Makes the macro table collapsible and collapsed by default.
- Remove EL6-specific cruft (it's in the old guidelines page).
- Move the parts of the byte compilation section that nobody ever uses off to the Appendix page.
- Make the single-rpm single-dir case the default. Move the entire multiple-dir case to the appendix.
- Completely remove %with_python3 from the example spec. We want people to build for python3. People just paste this in without knowing why they would ever need it. If you're using the new macros you can't just conditionalize for EL6 anyway. This really makes the spec look nice. Macro-izing the summary and description would make it look even nicer once the description gets longer than a line.
- Uses a generic "example" module as an example. Two different approved proposals changed it to two different things; I just punted.
- Puts the example spec all together without a bunch of text and admons and whatnot in the middle.
- Moves the 2to3 section to the appendix.
- Simply refers to the eggs section. That whole part at the bottom was quite awkward and I don't think most people even looked down that far.
- Loads and loads of typo fixes, grammar fixes, and cleanups.
The current main guideline page is now a bit over half the length it was and on the page it's less because of the collapsed macro table. I find it to be far more readable, and the sample spec no longer turns my stomach.
The old guidelines page is just as it was before writing up the new macros, except that I added a short explanatory section at the top.
The appendix page conveniently holds things which we probably should document but pretty much nobody would actually want to read without some specific reason.
Functionally I do not believe I have gone beyond anything upon which we voted but I wanted to toss this out there and see if anyone yells before I copy it into place. Been trying to get this done for three days now and I think I'm finally there.
- J<
python-devel mailing list python-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/python-devel
Awesome.
Few hints:
* in example spec, you mix srcname and pypi_name macros
* For example, the python 3 version of "coverage" must ship executables /usr/bin/coverage, /usr/bin/coverage-2 and /usr/bin/coverage-2.7, while the python 3 version must provide /usr/bin/coverage-3 and /usr/bin/coverage-3.4 (assuming python3 is Python 3.4).
Should be:
* For example, the python 3 version of "coverage" must ship executables /usr/bin/coverage, /usr/bin/coverage-3 and /usr/bin/coverage-3.4, while the python 2 version must provide /usr/bin/coverage-2 and /usr/bin/coverage-2.7 (assuming python3 is Python 3.4 and python2 is 2.7).
* Must: If you build for a single python runtime you must add %python_provide python-$module so that the current default python is provided from the unversioned python package.
I'm quite confused with this and don't understand it all. Generally %python_provide is not explained at all... ?
"MH" == Miro Hrončok mhroncok@redhat.com writes:
MH> * in example spec, you mix srcname and pypi_name macros
Yeah, fixed that up.
MH> * For example, the python 3 version of "coverage" must ship MH> executables MH> /usr/bin/coverage, /usr/bin/coverage-2 and /usr/bin/coverage-2.7, MH> while the python 3 version must provide /usr/bin/coverage-3 and MH> /usr/bin/coverage-3.4 (assuming python3 is Python 3.4).
That was just copied over as far as I know, but I've fixed it up in my draft. Good that this is resulting in some bug fixing.
MH> * Must: If you build for a single python runtime you must add MH> %python_provide python-$module so that the current default python is MH> provided from the unversioned python package.
MH> I'm quite confused with this and don't understand it all. Generally MH> %python_provide is not explained at all... ?
I need to get with tomspur and bang out documentation for the new macros, which will go at the beginning of the document.
- J<
After some additional discussion and cleanup, I've gone ahead and moved my drafts into place in the main guidelines. Please let me know if I've made any mistakes or typos or left out anything important.
However, there is one thing I'm trying to understand about the new guidelines (which came from one of the submitted drafts, not something that I wrote). They say:
" The following is a very simple spec file for a module building for both python2 and python3. It builds both versions in the same directory; this is possible because setuptools uses different build directories for different python versions and architectures. In addition, python3 will include the version of the interpreter in the names of generated files, so the build products don't conflict. (Of course this only works if a package builds for a single python2 version, which should always be the case in Fedora.) "
Which is fine, except that I must be missing something about the second sentence. Setuptools in f22 and rawhide (which I know are different versions) seems to use "build" regardless of which python version is used to execute setup.py. Am I misunderstanding what that sentence is trying to tell me?
For the package I'm using for testing (python-requests) it turns out that the results of %py2_build and %py3_build are completely identical, but this might just be coincidence.
- J<
Also, When using the python_provide macro as detailed in the guidelines, I can't seem to get fedpkg to generate an srpm:
fedpkg srpm
error: line 36: Unknown tag: ERROR: not recognized. error: query of specfile /home/tibbs/work/fpkg/python-requests/python-requests.spec failed, can't parse
That line is just: %{python_provide:%python_provide python2-%{srcname}}
Of course if I use %if %defined python_provide everything's OK, but that's three lines instead of one.
I know tomspur tested this so I must be missing something else. I just can't figure out what it might be.
- J<
"JLT" == Jason L Tibbitts tibbs@math.uh.edu writes:
JLT> Also, When using the python_provide macro as detailed in the JLT> guidelines, I can't seem to get fedpkg to generate an srpm:
OK, I think a question mark somehow got converted to a % in the draft.
%{?python_provide:%python_provide python2-%{srcname}}
I'll fix that up.
- J<
Jason L Tibbitts III tibbs@math.uh.edu schrieb am Fr., 31. Juli 2015 um 03:51 Uhr:
"JLT" == Jason L Tibbitts tibbs@math.uh.edu writes:
JLT> Also, When using the python_provide macro as detailed in the JLT> guidelines, I can't seem to get fedpkg to generate an srpm:
OK, I think a question mark somehow got converted to a % in the draft.
%{?python_provide:%python_provide python2-%{srcname}}
Yes. Thanks!
-Tom
On Thu, Jul 30, 2015 at 07:34:48PM -0500, Jason L Tibbitts III wrote:
After some additional discussion and cleanup, I've gone ahead and moved my drafts into place in the main guidelines. Please let me know if I've made any mistakes or typos or left out anything important.
There's a dead link in "Example common spec file".
However, there is one thing I'm trying to understand about the new guidelines (which came from one of the submitted drafts, not something that I wrote). They say:
" The following is a very simple spec file for a module building for both python2 and python3. It builds both versions in the same directory; this is possible because setuptools uses different build directories for different python versions and architectures. In addition, python3 will include the version of the interpreter in the names of generated files, so the build products don't conflict. (Of course this only works if a package builds for a single python2 version, which should always be the case in Fedora.) "
Which is fine, except that I must be missing something about the second sentence. Setuptools in f22 and rawhide (which I know are different versions) seems to use "build" regardless of which python version is used to execute setup.py. Am I misunderstanding what that sentence is trying to tell me?
I wrote that. I turns out, which I didn't know at the time, that this is only true for binary modules (for example python-systemd has build/{lib,temp}.linux-x86_64-{2.7,3.4}). For pure-python modules it seems to use a single build directory, but separate build/scripts directories.
[ It seems that there's no nice way to tell setuptools/distutils/distribute to always use separate build directories. It still uses the same build directory even if 2to3 translation is specified in the setup.py file, which is probably a bug. In a pinch, it is possible to override the build dir with --build-lib. So something like this works, even if 2to3 is used: %{__python2} setup.py build --build-lib build/2 %{__python3} setup.py build --build-lib build/3 Fortunately this is rarely necessary.]
So I think the detailed justification should be removed, and the first paragraph of "Example common spec file" replaced with something like this:
"The following is a very simple spec file for a module building for both python2 and python3. It builds both versions in the same directory; this is possible because build products for different versions of Python usually do not conflict."
For the package I'm using for testing (python-requests) it turns out that the results of %py2_build and %py3_build are completely identical, but this might just be coincidence.
Zbyszek
Zbigniew Jędrzejewski-Szmek zbyszek@in.waw.pl schrieb am Fr., 31. Juli 2015 um 06:06 Uhr:
On Thu, Jul 30, 2015 at 07:34:48PM -0500, Jason L Tibbitts III wrote:
After some additional discussion and cleanup, I've gone ahead and moved my drafts into place in the main guidelines. Please let me know if I've made any mistakes or typos or left out anything important.
There's a dead link in "Example common spec file".
However, there is one thing I'm trying to understand about the new guidelines (which came from one of the submitted drafts, not something that I wrote). They say:
" The following is a very simple spec file for a module building for both python2 and python3. It builds both versions in the same directory; this is possible because setuptools uses different build directories for different python versions and architectures. In addition, python3 will include the version of the interpreter in the names of generated files, so the build products don't conflict. (Of course this only works if a package builds for a single python2 version, which should always be the case in Fedora.) "
Which is fine, except that I must be missing something about the second sentence. Setuptools in f22 and rawhide (which I know are different versions) seems to use "build" regardless of which python version is used to execute setup.py. Am I misunderstanding what that sentence is trying to tell me?
I wrote that. I turns out, which I didn't know at the time, that this is only true for binary modules (for example python-systemd has build/{lib,temp}.linux-x86_64-{2.7,3.4}). For pure-python modules it seems to use a single build directory, but separate build/scripts directories.
[ It seems that there's no nice way to tell setuptools/distutils/distribute to always use separate build directories. It still uses the same build directory even if 2to3 translation is specified in the setup.py file, which is probably a bug. In a pinch, it is possible to override the build dir with --build-lib. So something like this works, even if 2to3 is used: %{__python2} setup.py build --build-lib build/2 %{__python3} setup.py build --build-lib build/3 Fortunately this is rarely necessary.]
So either this is a bug in the case of 2to3 or the %py{2,3}_build macros should implement this to work around it? Otherwise these packages would need to be build within different build directories and the current pushd/popd boilerplate.
-Tom
So I think the detailed justification should be removed, and the first paragraph of "Example common spec file" replaced with something like this:
"The following is a very simple spec file for a module building for both python2 and python3. It builds both versions in the same directory; this is possible because build products for different versions of Python usually do not conflict."
For the package I'm using for testing (python-requests) it turns out that the results of %py2_build and %py3_build are completely identical, but this might just be coincidence.
Zbyszek _______________________________________________ python-devel mailing list python-devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/python-devel
"ZJ" == Zbigniew Jędrzejewski-Szmek zbyszek@in.waw.pl writes:
ZJ> There's a dead link in "Example common spec file".
Oops, missed a colon. Fixed, thanks.
ZJ> I wrote that. I turns out, which I didn't know at the time, that ZJ> this is only true for binary modules (for example python-systemd has ZJ> build/{lib,temp}.linux-x86_64-{2.7,3.4}). For pure-python modules ZJ> it seems to use a single build directory, but separate build/scripts ZJ> directories.
Oops.
ZJ> So I think the detailed justification should be removed, and the ZJ> first paragraph of "Example common spec file" replaced with ZJ> something like this:
ZJ> "The following is a very simple spec file for a module building for ZJ> both python2 and python3. It builds both versions in the same ZJ> directory; this is possible because build products for different ZJ> versions of Python usually do not conflict."
I've done that. However, the following occurs to me: don't the existing %py*_build and %py*_install macros have enough information to do the necessary directory making, copying and pushd/popds to make this "just work" regardless? Can't we simply do that, hide all of the details and nuke the whole "separate build directories" thing from the guidelines?
- J<
python-devel@lists.fedoraproject.org
-
Bohuslav Kabrda -
Jason L Tibbitts III -
Miro Hrončok -
Thomas Spura -
Zbigniew Jędrzejewski-Szmek