Draft: Macros to tell what Python versions to package for

[Miro Hrončok]

Hello Pythonistas,

I've prepared a draft for Python packaging that introduces some new 
macros that should ease packaging for Fedoras, EPELs and even potential 
new RHELs, when it comes to python stacks.

I don't do much ifs in specfiles and prefer to leverage git branches for 
this, so I don't know what bothers you most. The proposal with example 
spec file is at:

https://fedoraproject.org/wiki/User:Churchyard/Packaging:PythonMustMayMacros

All you have to do to test it, is add the following block to your spec 
(that won't be needed once this is actually implemented):

     %if 0%{?fedora}
     %global py3_must 1
     %global py3_may 1
     %global py2_may 1
     %endif

     %if 0%{?rhel} &&  0%{?rhel} <= 7
     %global py3_may 1
     %global py2_may 1
     %endif

     %if 0%{?rhel} > 7
     ... (use your best judgment here)
     %endif

Could you please provide feedback? Ask questions?

There is a note at the bottom of the draft about how you could possibly 
start dropping Python 2 subpackages from Fedora.

-- 
Miro Hrončok
--
Phone: [phone]
IRC: mhroncok

[Petr Viktorin]

I like to use bconds, i.e. "%if %{with python2}". They're easy to 
override for local builds, allowing easy experimentation with, for 
example, dropping Python 2.
I'd be happy if our official recommendation used bconds. If we want 
people to copy-paste something, let's make it good.

Also, "with_python2" is the current de-facto best practice. It's a good, 
descriptive name, and people are used to it.
The "%if 0%{?py3_may}" is not as descriptive. If that is used throughout 
the specfile, people will need to know/read the docs – or just consider 
it magic.

So, I'd prefer adding macros that set "with_python2" bconds, and using 
that throughout the specfile.


For a more concrete idea, would something like this work? Naming to be 
bikeshedded of course.

# Build for python2 if it MUST be done
%py2_bcond_if required
# Build for python3 if it MAY be done
%py3_bcond_if available

%build

%{?with_python2:py2_build}
%{?with_python3:py3_build}

%install

%if %{with python2}
%py2_install
%endif
%if %{with python3}
%py3_install
%endif


(And some kind of %py_build_all and %py_install_all as the next baby 
step? Or am I reinventing a wheel with that line of thought?)


Also, I'd like to do some wordsmithing on the proposal when the 
technicalities are sorted out, so the following is obvious to people who 
just skim it:
- This new complexity is *only* meant for packagers who share specs 
across distros; others can stop reading now.
- FYI, we would love it if you dropped your py2 subpackage from Fedora.


On 03/02/18 10:36, Miro Hrončok wrote:
> Hello Pythonistas,
> 
> I've prepared a draft for Python packaging that introduces some new 
> macros that should ease packaging for Fedoras, EPELs and even potential 
> new RHELs, when it comes to python stacks.
> 
> I don't do much ifs in specfiles and prefer to leverage git branches for 
> this, so I don't know what bothers you most. The proposal with example 
> spec file is at:
> 
> https://fedoraproject.org/wiki/User:Churchyard/Packaging:PythonMustMayMacros 
> 
> 
> All you have to do to test it, is add the following block to your spec 
> (that won't be needed once this is actually implemented):
> 
>      %if 0%{?fedora}
>      %global py3_must 1
>      %global py3_may 1
>      %global py2_may 1
>      %endif
> 
>      %if 0%{?rhel} &&  0%{?rhel} <= 7
>      %global py3_may 1
>      %global py2_may 1
>      %endif
> 
>      %if 0%{?rhel} > 7
>      ... (use your best judgment here)
>      %endif
> 
> Could you please provide feedback? Ask questions?
> 
> There is a note at the bottom of the draft about how you could possibly 
> start dropping Python 2 subpackages from Fedora.
>

[Miro Hrončok]

On 2.3.2018 11:54, Petr Viktorin wrote:
>  I like to use bconds, i.e. "%if %{with python2}".
> They're easy to 
>  override for local builds, allowing easy experimentation with, for 
>  example, dropping Python 2.
>  I'd be happy if our official recommendation used bconds. If we want 
>  people to copy-paste something, let's make it good.
>  
>  Also, "with_python2" is the current de-facto best practice. It's a good, 
>  descriptive name, and people are used to it.
>  The "%if 0%{?py3_may}" is not as descriptive. If that is used throughout 
>  the specfile, people will need to know/read the docs – or just consider 
>  it magic.
>  
>  So, I'd prefer adding macros that set "with_python2" bconds, and using 
>  that throughout the specfile.
>  
>  
>  For a more concrete idea, would something like this work? Naming to be 
>  bikeshedded of course.
>  
>  # Build for python2 if it MUST be done
>  %py2_bcond_if required
>  # Build for python3 if it MAY be done
>  %py3_bcond_if available
>  
>  %build
>  
>  %{?with_python2:py2_build}
>  %{?with_python3:py3_build}
>  
>  %install
>  
>  %if %{with python2}
>  %py2_install
>  %endif
>  %if %{with python3}
>  %py3_install
>  %endif 
That looks good!

>  
>  (And some kind of %py_build_all and %py_install_all as the next baby 
>  step? Or am I reinventing a wheel with that line of thought?) 
That would be awesome, but let's keep that for another discussion maybe?


>  Also, I'd like to do some wordsmithing on the proposal when the 
>  technicalities are sorted out, so the following is obvious to people who 
>  just skim it:
>  - This new complexity is *only* meant for packagers who share specs 
>  across distros; others can stop reading now.
>  - FYI, we would love it if you dropped your py2 subpackage from Fedora. 
Yes, yes!


-- 
Miro Hrončok
--
Phone: [phone]
IRC: mhroncok

[Jason L Tibbitts III]

> >>>> "PV" == Petr Viktorin
> <pviktori(a)redhat.com&gt; writes: 
PV> %install

PV> %install

PV> %if %{with python2}
PV> %py2_install
PV> %endif
PV> %if %{with python3}
PV> %py3_install
PV> %endif

So... why not just make %py2_install and %py3_install just do the
%{with python*} internally, so the result is just

%install
%py2_install
%py3_install

And things just work.  That way you only need the conditionals when
you're doing something not covered by the macros.

 - J<

[Miro Hrončok]

On 2.3.2018 18:06, Jason L Tibbitts III wrote:
> >>>>> "PV" == Petr Viktorin
> <pviktori(a)redhat.com&gt; writes:
>  
>  PV> %install
>  
>  PV> %install
>  
>  PV> %if %{with python2}
>  PV> %py2_install
>  PV> %endif
>  PV> %if %{with python3}
>  PV> %py3_install
>  PV> %endif
>  
>  So... why not just make %py2_install and %py3_install just do the
>  %{with python*} internally, so the result is just
>  
>  %install
>  %py2_install
>  %py3_install
>  
>  And things just work.  That way you only need the conditionals when
>  you're doing something not covered by the macros. 
Not everybody wants to use those conditionals at all. I don't. I use git 
branches for this kind of things. When I see:

     %install
     %py2_install
     %py3_install

I expect it to be called for both 2 and 3 unconditionally. Hiding the 
condition in the maro makes things too magical for me.

-- 
Miro Hrončok
--
Phone: [phone]
IRC: mhroncok

[Jason L Tibbitts III]

> >>>> "MH" == Miro Hrončok
> <mhroncok(a)redhat.com&gt; writes: 
MH> I expect it to be called for both 2 and 3 unconditionally. Hiding
MH> the condition in the maro makes things too magical for me.

I don't think that stance holds up well as we increasingly hide things
in macros and will continue to hide even more.  (I'd like to see even
more hidden with macros, up to having whole classes of specfiles
completely generated by macros.)

And I can certainly understand the git branch thing, but if we
universally adopt and integrate %{with python3} as the conditional and
you're manually passing --without python3 then why would you really want
%py3_build to be executed unconditionally?  The principle of least
surprise would suggest that it isn't executed.

 - J<

[Petr Viktorin]

On 03/02/18 18:24, Jason L Tibbitts III wrote:
> >>>>> "MH" == Miro Hrončok
> <mhroncok(a)redhat.com&gt; writes:
>  
>  MH> I expect it to be called for both 2 and 3 unconditionally. Hiding
>  MH> the condition in the maro makes things too magical for me.
>  
>  I don't think that stance holds up well as we increasingly hide things
>  in macros and will continue to hide even more.  (I'd like to see even
>  more hidden with macros, up to having whole classes of specfiles
>  completely generated by macros.) 
Sure, let's hide things in macros. But I see this discussion as one 
level below that: it's the thing that we'll want to hide in those macros.

As I said in my proposal: Let's (later) add "%py_build_all" that 
conditionally calls py3_build and py3_build. Discussing that is 
intentionally left as the next step, though.

IOW, let's design the explicit version, and only hide *that* in macros, 
so advanced users can go one level down and still find a well-designed 
system underneath. An unstructured magic macro is a pain to debug.

>  And I can certainly understand the git branch thing, but if we
>  universally adopt and integrate %{with python3} as the conditional and
>  you're manually passing --without python3 then why would you really want
>  %py3_build to be executed unconditionally?  The principle of least
>  surprise would suggest that it isn't executed. 
I *really* don't want to change the behavior of %py3_build (which we 
currently define and promote as best practice), or of %with_python3 
(which many packages currently define).
Generating whole classes of specfiles completely by macros is a worthy 
goal, but it needs to be opt-in. We have hundreds of packages that are 
either lovingly hand-crafted or horribly cargo-culted, and we don't want 
to break them more than necessary.