python2->python3 mass rebuild and auto tools

[Steve Grubb]

Hello,

I have a package that fails to build because libraries aren't where they are 
suposed to be. I looked at the project page

https://fedoraproject.org/wiki/Changes/Python_means_Python3

and there is no mention of the effect on autotools.

I have pyexec_PYTHON. What is supposed to be there? And since this is an 
upstream package consumed by all distributions and old versions of Fedora/
RHEL,  what is the portable thing to do?

There's other things out there like pybind_dir which are probably messed up, 
too.

Should they be hardcoded to mean python2 in autotools and swig?

-Steve

[Charalampos Stratakis]

----- Original Message -----
>  From: "Steve Grubb" <sgrubb(a)redhat.com&gt;
>  To: "Development discussions related to Fedora"
> <devel(a)lists.fedoraproject.org&gt;
>  Sent: Thursday, August 1, 2019 12:03:01 AM
>  Subject: python2->python3 mass rebuild and auto tools
>  
>  Hello,
>  
>  I have a package that fails to build because libraries aren't where they are
>  suposed to be. I looked at the project page
>  
>  https://fedoraproject.org/wiki/Changes/Python_means_Python3
>  
>  and there is no mention of the effect on autotools.
>  
>  I have pyexec_PYTHON. What is supposed to be there? And since this is an
>  upstream package consumed by all distributions and old versions of Fedora/
>  RHEL,  what is the portable thing to do?
>  
>  There's other things out there like pybind_dir which are probably messed up,
>  too.
>  
>  Should they be hardcoded to mean python2 in autotools and swig?
>  
>  -Steve
>  
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct:
>  https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]
>   
Could you point out in which package the problem manifests?

I've dealt with numerous autotools issues before in respect to Python 3.8, and most of
them boil down to autotools (or the project using autotools) invoking python-config --libs
to embed python, which in order to achieve it in 3.8 the additional flag --embed needs to
be used. 

-- 
Regards,

Charalampos Stratakis
Software Engineer
Python Maintenance Team, Red Hat

[Charalampos Stratakis]

----- Original Message -----
>  From: "Charalampos Stratakis" <cstratak(a)redhat.com&gt;
>  To: "Development discussions related to Fedora"
> <devel(a)lists.fedoraproject.org&gt;
>  Sent: Thursday, August 1, 2019 1:03:51 AM
>  Subject: Re: python2->python3 mass rebuild and auto tools
>  
>  
>  
>  ----- Original Message -----
>  > From: "Steve Grubb" <sgrubb(a)redhat.com&gt;
>  > To: "Development discussions related to Fedora"
>  > <devel(a)lists.fedoraproject.org&gt;
>  > Sent: Thursday, August 1, 2019 12:03:01 AM
>  > Subject: python2->python3 mass rebuild and auto tools
>  > 
>  > Hello,
>  > 
>  > I have a package that fails to build because libraries aren't where they
>  > are
>  > suposed to be. I looked at the project page
>  > 
>  > https://fedoraproject.org/wiki/Changes/Python_means_Python3
>  > 
>  > and there is no mention of the effect on autotools.
>  > 
>  > I have pyexec_PYTHON. What is supposed to be there? And since this is an
>  > upstream package consumed by all distributions and old versions of Fedora/
>  > RHEL,  what is the portable thing to do?
>  > 
>  > There's other things out there like pybind_dir which are probably messed
>  > up,
>  > too.
>  > 
>  > Should they be hardcoded to mean python2 in autotools and swig?
>  > 
>  > -Steve
>  > 
>  > _______________________________________________
>  > devel mailing list -- devel(a)lists.fedoraproject.org
>  > To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  > Fedora Code of Conduct:
>  > https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  > List Archives:
>  > https://lists.fedoraproject.org/archives/list/[e-mail]
>  > 
>  
>  Could you point out in which package the problem manifests?
>  
>  I've dealt with numerous autotools issues before in respect to Python 3.8,
>  and most of them boil down to autotools (or the project using autotools)
>  invoking python-config --libs to embed python, which in order to achieve it
>  in 3.8 the additional flag --embed needs to be used.
>  
>  --
>  Regards,
>  
>  Charalampos Stratakis
>  Software Engineer
>  Python Maintenance Team, Red Hat
>   
And I've just realized I'm talking about a different problem here, not related to
this change, heh.

Could you still point out the package you are dealing with?

-- 
Regards,

Charalampos Stratakis
Software Engineer
Python Maintenance Team, Red Hat

[Steve Grubb]

On Wednesday, July 31, 2019 7:06:17 PM EDT Charalampos Stratakis wrote:
>  ----- Original Message -----
>  
>  > From: "Charalampos Stratakis" <cstratak(a)redhat.com&gt;
>  > To: "Development discussions related to Fedora"
>  > <devel(a)lists.fedoraproject.org&gt; 
 Sent: Thursday, August 1, 2019 1:03:51
>  > AM
>  > Subject: Re: python2->python3 mass rebuild and auto tools
>  > 
>  > 
>  > 
>  > ----- Original Message -----
>  > 
>  > > From: "Steve Grubb" <sgrubb(a)redhat.com&gt;
>  > > To: "Development discussions related to Fedora"
>  > > <devel(a)lists.fedoraproject.org&gt;
>  > > Sent: Thursday, August 1, 2019 12:03:01 AM
>  > > Subject: python2->python3 mass rebuild and auto tools
>  > > 
>  > > Hello,
>  > > 
>  > > I have a package that fails to build because libraries aren't where
>  > > they
>  > > are
>  > > suposed to be. I looked at the project page
>  > > 
>  > > https://fedoraproject.org/wiki/Changes/Python_means_Python3
>  > > 
>  > > and there is no mention of the effect on autotools.
>  > > 
>  > > I have pyexec_PYTHON. What is supposed to be there? And since this is
>  > > an
>  > > upstream package consumed by all distributions and old versions of
>  > > Fedora/ 
 RHEL,  what is the portable thing to do?
>  > > 
>  > > There's other things out there like pybind_dir which are probably
>  > > messed
>  > > up,
>  > > too.
>  > > 
>  > > Should they be hardcoded to mean python2 in autotools and swig?
>  > > 
>  > > -Steve
>  > > 
>  > > _______________________________________________
>  > > devel mailing list -- devel(a)lists.fedoraproject.org
>  > > To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  > > Fedora Code of Conduct:
>  > > https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  > > List Guidelines:
>  > > https://fedoraproject.org/wiki/Mailing_list_guidelines
>  > > List Archives:
>  > > https://lists.fedoraproject.org/archives/list/[e-mail]
>  > > .org
>  > Could you point out in which package the problem manifests?
>  > 
>  > I've dealt with numerous autotools issues before in respect to Python
>  > 3.8, 
 and most of them boil down to autotools (or the project using
>  > autotools) invoking python-config --libs to embed python, which
> in order
>  > to achieve it in 3.8 the additional flag --embed needs to be used.
>  > 
>  > --
>  And I've just realized I'm talking about a different problem here, not
>  related to this change, heh. 
 
>  Could you still point out the package you are dealing with? 
Audit

-Steve

[Miro Hrončok]

On 01. 08. 19 0:03, Steve Grubb wrote:
>  Hello, 
Hi Steve.

>  I have a package that fails to build because libraries aren't
> where they are
>  suposed to be. I looked at the project page
>  
>  https://fedoraproject.org/wiki/Changes/Python_means_Python3
>  
>  and there is no mention of the effect on autotools. 
Sorry a about not mentioning autotools specifically.

>  I have pyexec_PYTHON. What is supposed to be there? And since this is
> an
>  upstream package consumed by all distributions and old versions of Fedora/
>  RHEL,  what is the portable thing to do?
>  
>  There's other things out there like pybind_dir which are probably messed up,
>  too. 
Unfortunately, I don't know much about autoools, but I suspect that the same 
mechanics that were needed for Python 3 to work are now needed for Python 2.

What package exactly is failing? Is it audit? I'll have a look.

>  Should they be hardcoded to mean python2 in autotools and swig?
No. That was kinda the point of this change: "python" means Python 3.

Python 2 is deprecated and will be retired.

https://fedoraproject.org/wiki/Changes/RetirePython2

-- 
Miro Hrončok
--
Phone: [phone]
IRC: mhroncok

[Miro Hrončok]

On 01. 08. 19 1:08, Miro Hrončok wrote:
>  On 01. 08. 19 0:03, Steve Grubb wrote:
> > I have pyexec_PYTHON. What is supposed to be there? And since this is an
> > upstream package consumed by all distributions and old versions of Fedora/
> > RHEL,  what is the portable thing to do?
> >
> > There's other things out there like pybind_dir which are probably messed up,
> > too.
>  
>  Unfortunately, I don't know much about autoools, but I suspect that the same 
>  mechanics that were needed for Python 3 to work are now needed for Python 2.
>  
>  What package exactly is failing? Is it audit? I'll have a look. 
Here: https://src.fedoraproject.org/rpms/audit/pull-request/4

-- 
Miro Hrončok
--
Phone: [phone]
IRC: mhroncok

[Steve Grubb]

On Wednesday, July 31, 2019 7:08:57 PM EDT Miro Hrončok wrote:
>  On 01. 08. 19 0:03, Steve Grubb wrote:
>  > I have a package that fails to build because libraries aren't where they
>  > are suposed to be. I looked at the project page
>  > 
>  > https://fedoraproject.org/wiki/Changes/Python_means_Python3
>  > 
>  > and there is no mention of the effect on autotools.
>  
>  Sorry a about not mentioning autotools specifically.
>  
>  > I have pyexec_PYTHON. What is supposed to be there? And since this is an
>  > upstream package consumed by all distributions and old versions of
>  > Fedora/  RHEL,  what is the portable thing to do?
>  > 
>  > There's other things out there like pybind_dir which are probably messed
>  > up,  too.
>  
>  Unfortunately, I don't know much about autoools, but I suspect that the
>  same mechanics that were needed for Python 3 to work are now needed for
>  Python 2. What package exactly is failing? Is it audit? I'll have a look. 
Audit. But is seems that autotools shoul hard code the old sematics so that 
all packages do the right thing. It seems that python3 equivalents have been 
introduced. They do the right thing with the python migration. But there are 
things that are expectd to defaulto python 2.


>  > Should they be hardcoded to mean python2 in autotools and swig?
>  
>  
>  No. That was kinda the point of this change: "python" means Python 3.
>  
>  Python 2 is deprecated and will be retired. 
Of course. But there is a legacy pyexec_PYTHON and there is a py3exec_PYTHON. 
What this change means is that they are one in the same. I do not think that 
is the intention. Because...is there a py2exec_PYTHON? I do not find it 
grepping my system. And this would need to have been advanced years ago, not 
today.

Thanks,
-Steve

[Miro Hrončok]

On 01. 08. 19 4:43, Steve Grubb wrote:
>  Audit. But is seems that autotools shoul hard code the old sematics
> so that
>  all packages do the right thing. It seems that python3 equivalents have been
>  introduced. They do the right thing with the python migration. But there are
>  things that are expectd to defaulto python 2. 

https://src.fedoraproject.org/rpms/audit/pull-request/4

Autotools usually already does the right thing, aka choosing python2 if it 
cannot find python. Using "python" in Fedora packages is forbidden anyway.

> >> Should they be hardcoded to mean python2 in autotools and
> swig?
> >
> > No. That was kinda the point of this change: "python" means Python 3.
> >
> > Python 2 is deprecated and will be retired.
>  Of course. But there is a legacy pyexec_PYTHON and there is a py3exec_PYTHON.
>  What this change means is that they are one in the same. I do not think that
>  is the intention. Because...is there a py2exec_PYTHON? I do not find it
>  grepping my system. And this would need to have been advanced years ago, not
>  today. 
I don't know if there is py2exec_PYTHON. But autotools is no magic (although it 
might appear as such), it just uses the commands available. "python" command 
being forbidden in Fedora packages and "python" command meaning Python 3 have 
been advocated for years.

https://fedoraproject.org/wiki/FinalizingFedoraSwitchtoPython3

I'm sorry if that made you packaging work more complicated. But I just don't 
agree that we should make exceptions here. (Nevertheless I would have no idea 
how to do that, since every project ships their own pile of autotools that may 
or may not use the "python" command that way or another).

-- 
Miro Hrončok
--
Phone: [phone]
IRC: mhroncok

[Steve Grubb]

On Thursday, August 1, 2019 4:37:01 AM EDT Miro Hrončok wrote:
>  On 01. 08. 19 4:43, Steve Grubb wrote:
>  > Audit. But is seems that autotools shoul hard code the old sematics so
>  > that all packages do the right thing. It seems that python3 equivalents
>  > have been introduced. They do the right thing with the python migration.
>  > But there are things that are expectd to defaulto python 2.
>  
>  https://src.fedoraproject.org/rpms/audit/pull-request/4
>  
>  Autotools usually already does the right thing, aka choosing python2 if it
>  cannot find python. Using "python" in Fedora packages is forbidden anyway.
I applied your patch. Thanks.

But I am concerned that this is a bandaid because it patches the spec file and 
all distributions will have to do the same thing.

-Steve

>  >>> Should they be hardcoded to mean python2 in autotools
> and swig?
>  >> 
>  >> No. That was kinda the point of this change: "python" means Python 3.
>  >> 
>  >> Python 2 is deprecated and will be retired.
>  > 
>  > Of course. But there is a legacy pyexec_PYTHON and there is a
>  > py3exec_PYTHON. What this change means is that they are one in the same.
>  > I do not think that is the intention. Because...is there a
>  > py2exec_PYTHON? I do not find it grepping my system. And this would need
>  > to have been advanced years ago, not today.
>  
>  I don't know if there is py2exec_PYTHON. But autotools is no magic
>  (although it might appear as such), it just uses the commands available.
>  "python" command being forbidden in Fedora packages and "python"
> command
>  meaning Python 3 have been advocated for years.
>  
>  https://fedoraproject.org/wiki/FinalizingFedoraSwitchtoPython3
>  
>  I'm sorry if that made you packaging work more complicated. But I just
>  don't agree that we should make exceptions here. (Nevertheless I would
>  have no idea how to do that, since every project ships their own pile of
>  autotools that may or may not use the "python" command that way or
>  another).

[Miro Hrončok]

On 01. 08. 19 23:44, Steve Grubb wrote:
>  On Thursday, August 1, 2019 4:37:01 AM EDT Miro Hrončok wrote:
> > On 01. 08. 19 4:43, Steve Grubb wrote:
> >> Audit. But is seems that autotools shoul hard code the old sematics so
> >> that all packages do the right thing. It seems that python3 equivalents
> >> have been introduced. They do the right thing with the python migration.
> >> But there are things that are expectd to defaulto python 2.
> >
> > https://src.fedoraproject.org/rpms/audit/pull-request/4
> >
> > Autotools usually already does the right thing, aka choosing python2 if it
> > cannot find python. Using "python" in Fedora packages is forbidden anyway.
>  
>  I applied your patch. Thanks.
>  
>  But I am concerned that this is a bandaid because it patches the spec file and
>  all distributions will have to do the same thing. 
How does this Fedora impact other distributions?

-- 
Miro Hrončok
--
Phone: [phone]
IRC: mhroncok

[Steve Grubb]

On Thursday, August 1, 2019 6:37:47 PM EDT Miro Hrončok wrote:
>  On 01. 08. 19 23:44, Steve Grubb wrote:
>  > On Thursday, August 1, 2019 4:37:01 AM EDT Miro Hrončok wrote:
>  >> On 01. 08. 19 4:43, Steve Grubb wrote:
>  >>> Audit. But is seems that autotools shoul hard code the old sematics so
>  >>> that all packages do the right thing. It seems that python3 equivalents
>  >>> have been introduced. They do the right thing with the python
>  >>> migration.
>  >>> But there are things that are expectd to defaulto python 2.
>  >> 
>  >> https://src.fedoraproject.org/rpms/audit/pull-request/4
>  >> 
>  >> Autotools usually already does the right thing, aka choosing python2 if
>  >> it
>  >> cannot find python. Using "python" in Fedora packages is forbidden
>  >> anyway.
>  > 
>  > I applied your patch. Thanks.
>  > 
>  > But I am concerned that this is a bandaid because it patches the spec
>  > file and all distributions will have to do the same thing.
>  
>  How does this Fedora impact other distributions? 
Because eventually we all (distros) have to go through it. So, if autotools 
handled this gracefully, I and everyone else would have a configure.ac file and 
Makefile.am that just does the right thing. So, I am faced with making an 
announcement on my project's mail list that maintainer on other distros have 
to do this hack because autotools is incapable of doing the right thing. 
There is no migration plan for autotools. We could have got in front of this 
and helped make it smooth.

My package is now built in rawhide thanks to your patch. Fedora is OK. I just 
worry about bug reports rolling in on other distros upstream.

-Steve

[Miro Hrončok]

On 02. 08. 19 5:21, Steve Grubb wrote:
>  On Thursday, August 1, 2019 6:37:47 PM EDT Miro Hrončok wrote:
> > On 01. 08. 19 23:44, Steve Grubb wrote:
> >> On Thursday, August 1, 2019 4:37:01 AM EDT Miro Hrončok wrote:
> >>> On 01. 08. 19 4:43, Steve Grubb wrote:
> >>>> Audit. But is seems that autotools shoul hard code the old sematics so
> >>>> that all packages do the right thing. It seems that python3 equivalents
> >>>> have been introduced. They do the right thing with the python
> >>>> migration.
> >>>> But there are things that are expectd to defaulto python 2.
> >>>
> >>> https://src.fedoraproject.org/rpms/audit/pull-request/4
> >>>
> >>> Autotools usually already does the right thing, aka choosing python2 if
> >>> it
> >>> cannot find python. Using "python" in Fedora packages is forbidden
> >>> anyway.
> >>
> >> I applied your patch. Thanks.
> >>
> >> But I am concerned that this is a bandaid because it patches the spec
> >> file and all distributions will have to do the same thing.
> >
> > How does this Fedora impact other distributions?
>  
>  Because eventually we all (distros) have to go through it. So, if autotools
>  handled this gracefully, I and everyone else would have a configure.ac file and
>  Makefile.am that just does the right thing. So, I am faced with making an
>  announcement on my project's mail list that maintainer on other distros have
>  to do this hack because autotools is incapable of doing the right thing. 
Oh, now I understand. Sorry for the confusion.

I don't really understand autotools that much, but I'm quite confident that 
there is A Proper Way™ to have --with-python2 and --with-python3 (instead of 
--with-python). I've seen projects in Fedora have that.

Hopefully somebody more experienced in autotools can help here.

-- 
Miro Hrončok
--
Phone: [phone]
IRC: mhroncok

[Miro Hrončok]

On 02. 08. 19 10:57, Miro Hrončok wrote:
>  On 02. 08. 19 5:21, Steve Grubb wrote:
> > On Thursday, August 1, 2019 6:37:47 PM EDT Miro Hrončok wrote:
> >> On 01. 08. 19 23:44, Steve Grubb wrote:
> >>> On Thursday, August 1, 2019 4:37:01 AM EDT Miro Hrončok wrote:
> >>>> On 01. 08. 19 4:43, Steve Grubb wrote:
> >>>>> Audit. But is seems that autotools shoul hard code the old sematics
> so
> >>>>> that all packages do the right thing. It seems that python3
> equivalents
> >>>>> have been introduced. They do the right thing with the python
> >>>>> migration.
> >>>>> But there are things that are expectd to defaulto python 2.
> >>>>
> >>>> https://src.fedoraproject.org/rpms/audit/pull-request/4
> >>>>
> >>>> Autotools usually already does the right thing, aka choosing python2 if
> >>>> it
> >>>> cannot find python. Using "python" in Fedora packages is
> forbidden
> >>>> anyway.
> >>>
> >>> I applied your patch. Thanks.
> >>>
> >>> But I am concerned that this is a bandaid because it patches the spec
> >>> file and all distributions will have to do the same thing.
> >>
> >> How does this Fedora impact other distributions?
> >
> > Because eventually we all (distros) have to go through it. So, if autotools
> > handled this gracefully, I and everyone else would have a configure.ac file and
> > Makefile.am that just does the right thing. So, I am faced with making an
> > announcement on my project's mail list that maintainer on other distros have
> > to do this hack because autotools is incapable of doing the right thing.
>  
>  Oh, now I understand. Sorry for the confusion.
>  
>  I don't really understand autotools that much, but I'm quite confident that 
>  there is A Proper Way™ to have --with-python2 and --with-python3 (instead of 
>  --with-python). I've seen projects in Fedora have that.
>  
>  Hopefully somebody more experienced in autotools can help here. 
I've tried https://github.com/linux-audit/audit-userspace/pull/102 but honestly 
have no idea if I'm not reinventing the wheel.

-- 
Miro Hrončok
--
Phone: [phone]
IRC: mhroncok

[Miro Hrončok]

On 02. 08. 19 5:21, Steve Grubb wrote:
>  On Thursday, August 1, 2019 6:37:47 PM EDT Miro Hrončok wrote:
> > On 01. 08. 19 23:44, Steve Grubb wrote:
> >> On Thursday, August 1, 2019 4:37:01 AM EDT Miro Hrončok wrote:
> >>> On 01. 08. 19 4:43, Steve Grubb wrote:
> >>>> Audit. But is seems that autotools shoul hard code the old sematics so
> >>>> that all packages do the right thing. It seems that python3 equivalents
> >>>> have been introduced. They do the right thing with the python
> >>>> migration.
> >>>> But there are things that are expectd to defaulto python 2.
> >>>
> >>> https://src.fedoraproject.org/rpms/audit/pull-request/4
> >>>
> >>> Autotools usually already does the right thing, aka choosing python2 if
> >>> it
> >>> cannot find python. Using "python" in Fedora packages is forbidden
> >>> anyway.
> >>
> >> I applied your patch. Thanks.
> >>
> >> But I am concerned that this is a bandaid because it patches the spec
> >> file and all distributions will have to do the same thing.
> >
> > How does this Fedora impact other distributions?
>  
>  Because eventually we all (distros) have to go through it. So, if autotools
>  handled this gracefully, I and everyone else would have a configure.ac file and
>  Makefile.am that just does the right thing. So, I am faced with making an
>  announcement on my project's mail list that maintainer on other distros have
>  to do this hack because autotools is incapable of doing the right thing.
>  There is no migration plan for autotools. We could have got in front of this
>  and helped make it smooth.
>  
>  My package is now built in rawhide thanks to your patch. Fedora is OK. I just
>  worry about bug reports rolling in on other distros upstream. 
I've talked to my friend at SUSE, who I know is much better with autotools than 
me and also knows Python. He expressed one thing that I think is worth 
mentioning here:

There is no point to "fix" AM_PATH_PYTHON to search for Python 2 only, 
introducing AM_PATH_PYTHON3 etc. just for the remaining 5 months of Python 2 
lifetime.

Instead, please focus on getting rid of python2-audit (required bya udit-viewer 
and python2-policycoreutils (that is required only by dokuwiki-selinux and vdsm 
(broken since Fedora 28))).

-- 
Miro Hrončok
--
Phone: [phone]
IRC: mhroncok

[Steve Grubb]

On Friday, August 2, 2019 8:15:24 AM EDT Miro Hrončok wrote:
>  On 02. 08. 19 5:21, Steve Grubb wrote:
>  > On Thursday, August 1, 2019 6:37:47 PM EDT Miro Hrončok wrote:
>  >> On 01. 08. 19 23:44, Steve Grubb wrote:
>  >>> On Thursday, August 1, 2019 4:37:01 AM EDT Miro Hrončok wrote:
>  >>>> On 01. 08. 19 4:43, Steve Grubb wrote:
>  >>>>> Audit. But is seems that autotools shoul hard code the old sematics
>  >>>>> so
>  >>>>> that all packages do the right thing. It seems that python3
>  >>>>> equivalents
>  >>>>> have been introduced. They do the right thing with the python
>  >>>>> migration.
>  >>>>> But there are things that are expectd to defaulto python 2.
>  >>>> 
>  >>>> https://src.fedoraproject.org/rpms/audit/pull-request/4
>  >>>> 
>  >>>> Autotools usually already does the right thing, aka choosing python2
>  >>>> if
>  >>>> it
>  >>>> cannot find python. Using "python" in Fedora packages is
> forbidden
>  >>>> anyway.
>  >>> 
>  >>> I applied your patch. Thanks.
>  >>> 
>  >>> But I am concerned that this is a bandaid because it patches the spec
>  >>> file and all distributions will have to do the same thing.
>  >> 
>  >> How does this Fedora impact other distributions?
>  > 
>  > Because eventually we all (distros) have to go through it. So, if
>  > autotools handled this gracefully, I and everyone else would have a
>  > configure.ac file and Makefile.am that just does the right thing. So, I
>  > am faced with making an announcement on my project's mail list that
>  > maintainer on other distros have to do this hack because autotools is
>  > incapable of doing the right thing. There is no migration plan for
>  > autotools. We could have got in front of this and helped make it smooth.
>  > 
>  > My package is now built in rawhide thanks to your patch. Fedora is OK. I
>  > just worry about bug reports rolling in on other distros upstream.
>  
>  I've talked to my friend at SUSE, who I know is much better with autotools
>  than me and also knows Python. He expressed one thing that I think is
>  worth mentioning here:
>  
>  There is no point to "fix" AM_PATH_PYTHON to search for Python 2 only,
>  introducing AM_PATH_PYTHON3 etc. just for the remaining 5 months of Python
>  2 lifetime. 
Sure. I can understand that python2 is coming to an end. However, audit 
installs and runs fine on very old versions of Linux. As soon as I make it 
require only recent kernels/environments, I'll start getting issues opened 
asking to fix the breakage.
 
>  Instead, please focus on getting rid of python2-audit (required by
>  audit-viewer 
This ^^^ should go end of life asap.

>  and python2-policycoreutils (that is required only by
>  dokuwiki-selinux and vdsm (broken since Fedora 28))). 
I'd love to quit building the python2 extensions. I can't drop them any time 
soon upstream.

There must be other upstream developers that have these same issues/concerns.

-Steve

(very delayed list email) Re: python2->python3 mass rebuild and auto tools

[Samuel Sieb]

On 8/1/19 2:44 PM, Steve Grubb wrote:
[snip]

I don't know if any list admins are following this list, but this email 
was delayed almost two days.  From the headers (included below) it looks 
like it might have been stuck in a moderation queue.  Is that what 
happened?  I've noticed a few older emails showing up today.

Received: from mailman01.phx2.fedoraproject.org 
(mailman01.phx2.fedoraproject.org [10.5.126.36])
	by bastion01.phx2.fedoraproject.org (Postfix) with ESMTP id 10DD360B3E82;
	Sat,  3 Aug 2019 06:47:18 +0000 (UTC)
DKIM-Filter: OpenDKIM Filter v2.11.0 bastion01.phx2.fedoraproject.org 
10DD360B3E82
Received: from mailman01.phx2.fedoraproject.org (localhost [IPv6:::1])
	by mailman01.phx2.fedoraproject.org (Postfix) with ESMTP id 8FBF8586B06CB;
	Sat,  3 Aug 2019 06:47:17 +0000 (UTC)

--> right here on the same server

Received: by mailman01.phx2.fedoraproject.org (Postfix, from userid 991)
	id E6F9D586AF363; Thu,  1 Aug 2019 21:44:22 +0000 (UTC)
Received: from smtp-mm-ib01.fedoraproject.org 
(smtp-mm-ib01.vpn.fedoraproject.org [192.168.1.83])
	by mailman01.phx2.fedoraproject.org (Postfix) with ESMTP id AE50C586B06D9
	for <devel(a)lists.fedoraproject.org>; Thu,  1 Aug 2019 21:44:21 +0000 (UTC)

(very delayed list email) Re: python2->python3 mass rebuild and auto tools

[Kevin Fenzi]

On 8/3/19 12:01 AM, Samuel Sieb wrote:
>  On 8/1/19 2:44 PM, Steve Grubb wrote:
>  [snip]
>  
>  I don't know if any list admins are following this list, but this email
>  was delayed almost two days.  From the headers (included below) it looks
>  like it might have been stuck in a moderation queue.  Is that what
>  happened?  I've noticed a few older emails showing up today. 
No. It was delayed due to commits in epel8/epel8-playground branches.
There's about 70,000 (so far) commit emails there going to the
scm-commits list, so the list server has been swamped processing all those.

Hopefully it will catch up soon.

kevin

[Björn 'besser82' Esser]

Am Donnerstag, den 01.08.2019, 17:44 -0400 schrieb Steve Grubb:
>  On Thursday, August 1, 2019 4:37:01 AM EDT Miro Hrončok wrote:
>  > On 01. 08. 19 4:43, Steve Grubb wrote:
>  > > Audit. But is seems that autotools shoul hard code the old
>  > > sematics so
>  > > that all packages do the right thing. It seems that python3
>  > > equivalents
>  > > have been introduced. They do the right thing with the python
>  > > migration.
>  > > But there are things that are expectd to defaulto python 2.
>  > 
>  > https://src.fedoraproject.org/rpms/audit/pull-request/4
>  > 
>  > Autotools usually already does the right thing, aka choosing python2
>  > if it
>  > cannot find python. Using "python" in Fedora packages is forbidden
>  > anyway.
>  
>  I applied your patch. Thanks.
>  
>  But I am concerned that this is a bandaid because it patches the spec
>  file and 
>  all distributions will have to do the same thing. 

Just until the end of this year, as Python2 will be finally dead on
2020-01-01.

Anyways, there are easy ways in Autotools to archieve this, and I can
craft a patch, if you guide me to the canonical upstream of audit.

Björn

[Dridi Boukelmoune]

>  Anyways, there are easy ways in Autotools to archieve this, and I
> can
>  craft a patch, if you guide me to the canonical upstream of audit. 
And even if you don't patch it, the need to export [1] a PYTHON
variable in the environment is exactly what should be done.

The tragedy of autotools is that people end up expecting everything to
be handled _auto_matically. Where we sit downstream we should never
ever need to worry about configure.ac files coming from upstream. One
of the goals of autotools is to allow upstream to redistribute a
bootstrapped archive, one that has already gone through the various
auto* utilities (and the controversial libtool) with a simple `make
dist`.

In the case of this package, you already _don't_ run the configure
script vanilla:

    %configure --sbindir=/sbin --libdir=/%{_lib} --with-python=yes \
        --with-python3=yes \
        --enable-gssapi-krb5=yes --with-arm --with-aarch64 \
        --with-libcap-ng=yes --enable-zos-remote \
        --enable-systemd

And I assume you don't complain that audit's configure script doesn't
figure automagically that on Fedora it should --enable or build --with
exactly that configuration. Upstream gives you a choice regarding what
you may take from the build.

Auto-detection works the same, if it doesn't yield the results you
need downstream, you have an escape hatch:

    ./configure --help
    [...]
    Some influential environment variables:
      [...]
      PYTHON      the Python interpreter
    [...]

There is nothing worrying about having to help the configure script,
in some cases you have to to ensure a reproducible build.

It does suck when you need to apply a similar tweak everywhere, but it
could be something handled by the %configure macro if it starts
showing up in more than just the audit package, or simply as part of
the python-means-python3 change (except that here it's the other way
around, which we probably don't want).

Cheers,
Dridi

[1] preferred way for autoconf would be %configure ... PYTHON=python2