an update to automake-1.11?

[Owen Taylor]

I was rather surprised to see:

 https://admin.fedoraproject.org/updates/F9/FEDORA-2009-6661
 https://admin.fedoraproject.org/updates/F10/FEDORA-2009-6076
 https://admin.fedoraproject.org/updates/F11/FEDORA-2009-6370

Where the automake was upgraded to 1.11 for F9, F10, and F11.

In general automake hasn't had a very good track record of compatibility
between 1.x and 1.y, though this has been getting better recently.
I don't see any specific mentions of incompatible changes in a quick
scan of:

 http://lists.gnu.org/archive/html/automake/2009-05/msg00093.html

But it is also a pretty long release announcement so it wouldn't
surprise me if there were some subtle incompatibilities.

The only breakage I'm actually aware of in the gnome-common package;
gnome-common-2.26 and earlier doesn't know that automake-1.11 is 
a valid replacement when automake-1.10 is asked for.

So, we definitely need to release an update for gnome-common, or people
aren't going to be able to do GNOME development on F11.

But is this the type of upgrade that makes sense in general? It seems to
me that we should be very conservative in upgrading build tools,
especially in "maintenance mode" distributions like F9 and F10.

- Owen

[Braden McDaniel]

On 6/30/09 2:05 PM, Owen Taylor wrote:
>  I was rather surprised to see:
>
>    https://admin.fedoraproject.org/updates/F9/FEDORA-2009-6661
>    https://admin.fedoraproject.org/updates/F10/FEDORA-2009-6076
>    https://admin.fedoraproject.org/updates/F11/FEDORA-2009-6370
>
>  Where the automake was upgraded to 1.11 for F9, F10, and F11. 
Wow.  That *is* surprising.

>  But is this the type of upgrade that makes sense in general? 
No.

Even if Automake had a better track record of compatibility, upgrading a 
build tool to something that isn't simply a bugfix release does not make 
sense *in general* for released versions of Fedora.  Special 
circumstances that might prompt such a move can always be discussed, of 
course.  But this sort of thing incurs way too much risk.

-- 
Braden McDaniel                      e-mail: <braden(a)endoframe.com&gt;
<http://endoframe.com>               Jabber: <braden(a)jabber.org&gt;

[Daniel P. Berrange]

On Tue, Jun 30, 2009 at 02:05:57PM -0400, Owen Taylor wrote:
>  I was rather surprised to see:
>  
>   https://admin.fedoraproject.org/updates/F9/FEDORA-2009-6661
>   https://admin.fedoraproject.org/updates/F10/FEDORA-2009-6076
>   https://admin.fedoraproject.org/updates/F11/FEDORA-2009-6370
>  
>  Where the automake was upgraded to 1.11 for F9, F10, and F11.
>  
>  In general automake hasn't had a very good track record of compatibility
>  between 1.x and 1.y, though this has been getting better recently.
>  I don't see any specific mentions of incompatible changes in a quick
>  scan of:
>  
>   http://lists.gnu.org/archive/html/automake/2009-05/msg00093.html
>  
>  But it is also a pretty long release announcement so it wouldn't
>  surprise me if there were some subtle incompatibilities.
>  
>  The only breakage I'm actually aware of in the gnome-common package;
>  gnome-common-2.26 and earlier doesn't know that automake-1.11 is 
>  a valid replacement when automake-1.10 is asked for.
>  
>  So, we definitely need to release an update for gnome-common, or people
>  aren't going to be able to do GNOME development on F11.
>  
>  But is this the type of upgrade that makes sense in general? It seems to
>  me that we should be very conservative in upgrading build tools,
>  especially in "maintenance mode" distributions like F9 and F10. 
This is seriously dubious for F9, since if it causes a problem there
is next to no time in which to fix it before F9 updates are turned 
off.  In general I struggle to believe that there is a compelling
need to rebase automake versions in our stable releases.

Daniel
-- 
|: Red Hat, Engineering, London   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

[Kevin Kofler]

Daniel P. Berrange wrote:
>  This is seriously dubious for F9, since if it causes a problem there
>  is next to no time in which to fix it before F9 updates are turned
>  off.  In general I struggle to believe that there is a compelling
>  need to rebase automake versions in our stable releases. 
Some software may need the new version to build.

        Kevin Kofler

[Sam Varshavchik]

Kevin Kofler writes:

>  Daniel P. Berrange wrote:
> > This is seriously dubious for F9, since if it causes a problem there
> > is next to no time in which to fix it before F9 updates are turned
> > off.  In general I struggle to believe that there is a compelling
> > need to rebase automake versions in our stable releases.
>  
>  Some software may need the new version to build. 
Then, they need to be patched so that they would get built for F9, or they 
should not be built for F9 altogether.

[Stepan Kasal]

Hello,

On Tue, Jun 30, 2009 at 06:58:39PM -0400, Sam Varshavchik wrote:
>  Kevin Kofler writes:
> > Some software may need the new version to build.
>
>  Then, they need to be patched so that they would get built for F9, or 
>  they should not be built for F9 altogether. 
I'm afraid the answer shows you do not fully understand the context.
Sorry,

Stepan

[Sam Varshavchik]

Stepan Kasal writes:

>  Hello,
>  
>  On Tue, Jun 30, 2009 at 06:58:39PM -0400, Sam Varshavchik wrote:
> > Kevin Kofler writes:
> >> Some software may need the new version to build.
> >
> > Then, they need to be patched so that they would get built for F9, or 
> > they should not be built for F9 altogether.
>  
>  I'm afraid the answer shows you do not fully understand the context.
>  Sorry, 
I understand the concept very well, and have done precisely that numerous 
times before.

[Ralf Corsepius]

Kevin Kofler wrote:
>  Daniel P. Berrange wrote:
> > This is seriously dubious for F9, since if it causes a problem there
> > is next to no time in which to fix it before F9 updates are turned
> > off.  In general I struggle to believe that there is a compelling
> > need to rebase automake versions in our stable releases.
>  
>  Some software may need the new version to build. 
Very unlikely.


However, this upgrade may break rebuilding some of the packages whose 
packagers refuse to accept that running the autotools during builts is 
harmful.

Anyway, the changes between automake-1.10 and automake-1.11 have been 
comparatively harmless.

Ralf

[Stepan Kasal]

Hello,

On Wed, Jul 01, 2009 at 07:30:37AM +0200, Ralf Corsepius wrote:
>  Kevin Kofler wrote:
> > Some software may need the new version to build.
>
>  Very unlikely. 
There are people using the new features, like Jim Mayering, the
coreutils maintainer, and others.  Building checkouts or even
tarballs of their projects on F9 might be a problem.

>  However, this upgrade may break rebuilding some of the packages whose
>  
>  packagers refuse to accept that running the autotools during builts is  
>  harmful. 
But this is not too likely either.  Only a minority of Fedora
packages does run autotools in their spec files.

>  Anyway, the changes between automake-1.10 and automake-1.11 have
>  been  comparatively harmless. 
Thank you for saying that.  (Yes, I know this statment does not mean
you support the updates and I do not intend to imply that.)

Stepan

[Ralf Corsepius]

Stepan Kasal wrote:
>  Hello,
>  
>  On Wed, Jul 01, 2009 at 07:30:37AM +0200, Ralf Corsepius wrote:
> > Kevin Kofler wrote:
> >> Some software may need the new version to build.
> > Very unlikely.
>  
>  There are people using the new features, like Jim Mayering, the
>  coreutils maintainer, and others.  Building checkouts or even
>  tarballs of their projects on F9 might be a problem. 
No, not if they bundle the
generated auto* files with their tarballs, as 
they are supposed to do.

> > However, this upgrade may break rebuilding some of the packages
> whose  
> > packagers refuse to accept that running the autotools during builts is  
> > harmful.
>  
>  But this is not too likely either. 
Right, it is unlikely to happen with packages
which already use 
automake-1.10 or versions next to it.

>   Only a minority of Fedora
>  packages does run autotools in their spec files. 
Well, with Fedora having attracted
new packagers, who are not aware 
about the issues lurking, the number of such packages is increasing, 
once again!

> > Anyway, the changes between automake-1.10 and automake-1.11 have
> > been  comparatively harmless.
>  
>  Thank you for saying that.  (Yes, I know this statment does not mean
>  you support the updates and I do not intend to imply that.) 
Oh, actually I welcome
Fedora shipping automake-1.11, because
a) it will cause some moderate stir-up to those packages whose upstreams 
are still abusing the autotools.
b) it will force packagers who are running the autotools during builds 
to review their packages, and to reconsider their habits.
c) I (as a developer) am using automake-1.11 for my own works already.


The new features having been introduced to automake-1.11, however are 
not welcomed by me. Neither do I find them useful nor do I find them 
helpful - They better should never have been added to automake!

Ralf

[Kevin Kofler]

Ralf Corsepius wrote:
>  a) it will cause some moderate stir-up to those packages whose
> upstreams
>  are still abusing the autotools. 
s/ab// ;-)

Why can't we just move to a better build system with higher focus on
backwards compatibility? (And FYI, I'm "against autotools" more than
I'm "for CMake". It's just that in the past I could just say
"don't use
autotools", now I have an actual alternative to suggest.)

        Kevin Kofler

[Ralf Corsepius]

Kevin Kofler wrote:
>  Ralf Corsepius wrote:
> > a) it will cause some moderate stir-up to those packages whose upstreams
> > are still abusing the autotools.
>  
>  s/ab// ;-)
>  
>  Why can't we just move to a better build system with higher focus on
>  backwards compatibility? 
Because
a) the autotools are not as bad as you in your want to make them appear.
b) it's not "we" (Fedora) who chooses the tools, it's the
upstreams's 
choice.

>  (And FYI, I'm "against autotools" more than
>  I'm "for CMake". It's just that in the past I could just say
> "don't use
>  autotools", now I have an actual alternative to suggest.) 
Sigh, ... Kevin, please refrain from spreading foul and silly propaganda 
on topic you apparently have insufficient knowledge about.

[Matthew Woehlke]

Ralf Corsepius wrote:
>  Kevin Kofler wrote:
> > Ralf Corsepius wrote:
> >> a) it will cause some moderate stir-up to those packages whose upstreams
> >> are still abusing the autotools.
> >
> > s/ab// ;-)
> >
> > Why can't we just move to a better build system with higher focus on
> > backwards compatibility?
>  
>  Because
>  a) the autotools are not as bad as you in your want to make them appear. 
Sorry, but any build system where you can't patch the build system 
without risking the sky falling *is* that bad.

Why is it that to build a cmake-based project, it is a given that I run 
cmake, but to build an autotools-based project, I am expected to fear 
and dread and avoid-at-all-costs running autotools? Do you /really/, 
honestly not see anything wrong with that?

As Conrad noted, "[the] phobia of regenerating an auto-generated script 
just goes to show how completely broken autotools is."

-- 
Matthew
Please do not quote my e-mail address unobfuscated in message bodies.
-- 
It is training and experience that gives us the ability to abstract 
problems, remain objective, use previous knowledge, interact with users, 
and herd cats.
   -- Celeste Lyn Paul, on Usability Experts

[Braden McDaniel]

On 7/6/09 6:29 PM, Matthew Woehlke wrote:
>  Ralf Corsepius wrote:
> > Kevin Kofler wrote:
> >> Ralf Corsepius wrote:
> >>> a) it will cause some moderate stir-up to those packages whose
> >>> upstreams
> >>> are still abusing the autotools.
> >>
> >> s/ab// ;-)
> >>
> >> Why can't we just move to a better build system with higher focus on
> >> backwards compatibility?
> >
> > Because
> > a) the autotools are not as bad as you in your want to make them appear.
>
>  Sorry, but any build system where you can't patch the build system
>  without risking the sky falling *is* that bad.
>
>  Why is it that to build a cmake-based project, it is a given that I run
>  cmake, but to build an autotools-based project, I am expected to fear
>  and dread and avoid-at-all-costs running autotools? Do you /really/,
>  honestly not see anything wrong with that?
>
>  As Conrad noted, "[the] phobia of regenerating an auto-generated script
>  just goes to show how completely broken autotools is." 
The problem is not the sky falling, which is quite unlikely.  The 
problem is that when you regenerate configure and Makefile.in, you are 
changing files that are part of the upstream deliverable.  And you're 
doing so in a less-than-surgical way that may have unintended consequences.

The situation is similar to regenerating documentation (that was already 
included in the upstream distribution).  Unless you are certain that you 
are using the same versions of tools used by upstream, it is awfully 
hard to be confident that your doc generation toolchain is 
bugwards-compatible with upstream's.  You probably won't get any error 
messages--but you cannot be confident that the output has been rendered 
as upstream intended without some very careful inspection of the output.

The difference between regenerating the build system and regenerating 
documentation is that the former is irrelevant once you've successfully 
built the package.  If you get to that point, the way you got there 
should have no impact on end users.

The number of people chiming in on this thread to the effect, "I've 
regenerated configure/Makefile.in for years and I've never had a 
problem," is testament to the fact that backward compatibility of 
autotools releases has gotten a lot better in recent years.  The 
autotools are hardly unique in experiencing such growing pains during 
maturation.  Where they do differ from a tool like cmake is that they 
insulate packages against future changes to the autotools themselves by 
avoiding any requirement that they (the autotools) be invoked when 
building the package.  However, it is quite a bit more difficult to 
insulate against package maintainers determined to defeat such measures.

Regenerating the build system is the antithesis of providing surgical 
patches to solve a problem.  More often than not in package maintenance, 
"nuke 'em from orbit" is *not* the *only* way to be sure.

-- 
Braden McDaniel                      e-mail: <braden(a)endoframe.com&gt;
<http://endoframe.com>               Jabber: <braden(a)jabber.org&gt;

[Kevin Kofler]

Braden McDaniel wrote:
>  The number of people chiming in on this thread to the effect,
> "I've
>  regenerated configure/Makefile.in for years and I've never had a
>  problem," is testament to the fact that backward compatibility of
>  autotools releases has gotten a lot better in recent years.  The
>  autotools are hardly unique in experiencing such growing pains during
>  maturation. 
Then how come CMake manages to provide near-100% backwards compatibility? Of 
course, no software is perfect, but CMake's design is to be completely bug-
for-bug (*) compatible with the original version the project used (see also 
the CMake policy system), whereas the autotools are doing incompatible 
changes in a way which require the sources to be changed.

(*) of course only where the bugfix actually causes compatibility issues! 
Otherwise they just fix it.

>  Where they do differ from a tool like cmake is that they insulate
> packages
>  against future changes to the autotools themselves by avoiding any
>  requirement that they (the autotools) be invoked when building the
>  package. 
And that's bad because there's no plan B: incompatible changes are made 
(even fairly recently, see libtool 2.1) without providing backwards source 
compatibility, relying entirely on tarballs shipping generated files for 
backwards compatibility. This is unhelpful because it doesn't help the 
developer (upstream developer, packager etc.) who needs to edit the actual 
source code (and this is the reason why we're having this discussion in the 
first place), it doesn't help for things like snapshot checkouts from 
repositories which don't carry generated files (but only generate them for 
release tarballs, a fairly common practice) and of course it doesn't help if 
upstream doesn't ship generated files at all (though the autotools 
discourage that).

I think it would be much better to ensure rerunning autoreconf will always 
just work, then upstreams wouldn't have to ship autogenerated files as 
"source code". But of course that'd turn a lot of the autotools' core
design 
decisions (e.g. the idea of generating shell scripts in the first place) 
into accidents of history. So I'm sorry (and I know you'll probably never be 
convinced), but I don't think the autotools can be fixed and still be the 
autotools, the whole concept is flawed.

What I see is tarballs getting littered with generated files which are 1. 
unnecessary, as they can just be regenerated and 2. contain generated 
snippets which get old quickly. If I fix a bug in CMake, it'll automatically 
fix the issue for all subsequent builds of CMake-using software (unless my 
fix is incompatible and I have to introduce a "policy" for it, then they 
need to opt in to the fix). If I fix a bug in the autotools, some software 
may never pick it up, and the one that will may take years to pick it up! 
How is that an advantage?

        Kevin Kofler

[Braden McDaniel]

On Tue, 2009-07-07 at 02:02 +0200, Kevin Kofler wrote:
>  Braden McDaniel wrote:
>  > The number of people chiming in on this thread to the effect, "I've
>  > regenerated configure/Makefile.in for years and I've never had a
>  > problem," is testament to the fact that backward compatibility of
>  > autotools releases has gotten a lot better in recent years.  The
>  > autotools are hardly unique in experiencing such growing pains during
>  > maturation.
>  
>  Then how come CMake manages to provide near-100% backwards compatibility? Of 
>  course, no software is perfect, but CMake's design is to be completely bug-
>  for-bug (*) compatible with the original version the project used (see also 
>  the CMake policy system), whereas the autotools are doing incompatible 
>  changes in a way which require the sources to be changed.
>  
>  (*) of course only where the bugfix actually causes compatibility issues! 
>  Otherwise they just fix it. 
Breaking compatibility with previous versions of automake, autoconf, or
libtool has no impact on released tarballs made using those tools; they
continue to work as intended because they do not depend on the presence
of these tools.  As such, I imagine the autotools maintainers do not
feel so great an obligation to backward compatibility as the CMake
maintainers might.

>  > Where they do differ from a tool like cmake is that they
> insulate packages
>  > against future changes to the autotools themselves by avoiding any
>  > requirement that they (the autotools) be invoked when building the
>  > package.
>  
>  And that's bad because there's no plan B: incompatible changes are made 
>  (even fairly recently, see libtool 2.1) without providing backwards source 
>  compatibility, relying entirely on tarballs shipping generated files for 
>  backwards compatibility. 
That is the use case for the tools.  As with most software, little to no
support is provided for those who misuse it.  This is not especially
surprising.

>   This is unhelpful because it doesn't help the 
>  developer (upstream developer, packager etc.) who needs to edit the actual 
>  source code (and this is the reason why we're having this discussion in the 
>  first place), it doesn't help for things like snapshot checkouts from 
>  repositories which don't carry generated files (but only generate them for 
>  release tarballs, a fairly common practice) and of course it doesn't help if 
>  upstream doesn't ship generated files at all (though the autotools 
>  discourage that). 
Nor is it any hindrance in these endeavors.

The autotools are well known not to make tea, either.  And astoundingly,
I know of no plans to correct this.

-- 
Braden McDaniel <braden(a)endoframe.com&gt;

[Kevin Kofler]

Braden McDaniel wrote:
>  Breaking compatibility with previous versions of automake, autoconf,
> or
>  libtool has no impact on released tarballs made using those tools; they
>  continue to work as intended because they do not depend on the presence
>  of these tools.  As such, I imagine the autotools maintainers do not
>  feel so great an obligation to backward compatibility as the CMake
>  maintainers might. 
And that's exactly what I'm complaining about.

You eschewed my question about what the advantage of this way of working is, 
in face of the obvious drawbacks, i.e.:
* changing the actual source code (and yes, it's necessary for upstream to 
change the actual source code) may require other, unrelated changes to 
account for incompatible autotools changes. Especially if upstream is using 
a fast-moving distribution like Fedora. Even if updates like that automake 
1.11 update wouldn't get pushed, they'd still have to upgrade to a new 
Fedora with new autotools at least once a year. It's unrealistic to expect 
upstreams to have a self-installed custom copy of specific autotools 
versions. A few projects, such as GCC, do that, but most upstreams just use 
whatever autotools their distro ships, which is usually not even the same 
for all upstream developers (so configure scripts pingpong between different 
autotools versions as they're regenerated by different developers).
* any bugs in the shared template snippets get copied into all packages 
using the snippet, so you have to patch lots of packages to fix something. 
The infamous lib64 rpath bug is one example (and compounded by the fact that 
upstream refuses to fix it – one upstream developer claimed it fixed in 
upstream libtool 2.1, but it actually isn't).

For me, this design is inherently broken, just like bundling libraries is 
(and Fedora has policies banning the latter in most cases), for the same 
reasons (bugs getting copied to all packages), and I fail to see any 
advantage of that way of working.

        Kevin Kofler

[Braden McDaniel]

On Tue, 2009-07-07 at 14:01 +0200, Kevin Kofler wrote:
>  Braden McDaniel wrote:
>  > Breaking compatibility with previous versions of automake, autoconf, or
>  > libtool has no impact on released tarballs made using those tools; they
>  > continue to work as intended because they do not depend on the presence
>  > of these tools.  As such, I imagine the autotools maintainers do not
>  > feel so great an obligation to backward compatibility as the CMake
>  > maintainers might.
>  
>  And that's exactly what I'm complaining about. 
Why in this forum? What do you think you can accomplish by doing that?

>  You eschewed my question about what the advantage of this way of
> working is, 
>  in face of the obvious drawbacks, i.e.: 
The benefits of using software as its authors intend and support are, I
hope, obvious.

I don't understand the objective of your continued rambling outside
those parameters.  Your estimation of the build systems used by various
packages is completely irrelevant here.  Fedora is downstream.  If you
have issues with the upstream implementation of a package, take it
upstream.

-- 
Braden McDaniel <braden(a)endoframe.com&gt;

[Matthew Woehlke]

Braden McDaniel wrote:
>  Breaking compatibility with previous versions of automake, autoconf,
> or
>  libtool has no impact on released tarballs made using those tools; they
>  continue to work as intended because they do not depend on the presence
>  of these tools. 
...but they depend on a slew of *other* things, like a POSIX shell and 
many POSIX tools. And IIRC I've run into problems when those weren't the 
/GNU/ versions thereof.

There's a lot to be said for /one/ tool that - while, granted, it needs 
to be installed - not only allows you to build, but also do full 
development without having to hunt down some precise version thereof, 
have several versions installed at once, etc.

-- 
Matthew
Please do not quote my e-mail address unobfuscated in message bodies.
-- 
You're on your own for the pony. -- Richard Hughes, on feature requests

[Kevin Kofler]

Matthew Woehlke wrote:
>  ...but they depend on a slew of *other* things, like a POSIX shell
> and
>  many POSIX tools. 
Right. Assuming POSIX in a tool which is supposed to be a portability tool 
is completely nonsensical and anachronistic, considering the most popular 
operating system is a proprietary system which does not support POSIX out of 
the box. For the people on that inferior operating system, installing a 
POSIX environment is actually harder than installing a simple tool like 
CMake. And for those of us on a sane operating system, CMake is just one 
"yum install cmake" away. So what's the point of generating POSIX shell code

into each and every tarball?

        Kevin Kofler

[Richard W.M. Jones]

On Wed, Jul 01, 2009 at 12:40:44PM +0200, Ralf Corsepius wrote:
>  No, not if they bundle the generated auto* files with their tarballs,
> as  
>  they are supposed to do. 
They're not "supposed to do" that.  Don't make stuff up.

Rich.

-- 
Richard Jones, Emerging Technologies, Red Hat  http://et.redhat.com/~rjones
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into Xen guests.
http://et.redhat.com/~rjones/virt-p2v

[Toshio Kuratomi]

On 07/04/2009 03:22 PM, Richard W.M. Jones wrote:
>  On Wed, Jul 01, 2009 at 12:40:44PM +0200, Ralf Corsepius wrote:
> > No, not if they bundle the generated auto* files with their tarballs, as  
> > they are supposed to do.
>  
>  They're not "supposed to do" that.  Don't make stuff up.
>   
It's true there are no literal files matching the wildcard "auto*" that
are generated for inclusion in the tarballs.  But I think Ralf is
talking about the files generated by the auto-tools (autoconf, automake,
and libtool).  Those are supposed to be bundled with the tarballs.

-Toshio

[Sam Varshavchik]

Toshio Kuratomi writes:

>  On 07/04/2009 03:22 PM, Richard W.M. Jones wrote:
> > On Wed, Jul 01, 2009 at 12:40:44PM +0200, Ralf Corsepius wrote:
> >> No, not if they bundle the generated auto* files with their tarballs, as  
> >> they are supposed to do.
> > 
> > They're not "supposed to do" that.  Don't make stuff up.
> > 
>  
>  It's true there are no literal files matching the wildcard "auto*" that
>  are generated for inclusion in the tarballs.  But I think Ralf is
>  talking about the files generated by the auto-tools (autoconf, automake,
>  and libtool).  Those are supposed to be bundled with the tarballs. 
And, they are.

So, the automake update should not really have any impact on rebuilding any 
existing well-made rpm package. The only possible impact would be to those 
packages that rerun automake or autoconf, for some reason.

Although I do believe that there's a small number of rpms whose spec script 
does that, I really think that this is not correct, and the packaging 
guidelines should really prohibit that. If the configure script needs 
patching, make a patch against the configure script, and/or Makefile.in; 
rather than patching configure.in and Makefile.am, and rerun all the auto 
scripts.

[Richard W.M. Jones]

On Sat, Jul 04, 2009 at 10:01:48PM -0400, Sam Varshavchik wrote:
>  Although I do believe that there's a small number of rpms whose
> spec 
>  script does that, I really think that this is not correct, and the 
>  packaging guidelines should really prohibit that. If the configure script 
>  needs patching, make a patch against the configure script, and/or 
>  Makefile.in; rather than patching configure.in and Makefile.am, and rerun 
>  all the auto scripts. 
There's been lots of previous discussion of this silly idea of
patching generated code.  You end up carrying enormous patches
containing just line number changes that often can't be applied
upstream, and can't be carried forward to new upstream releases --
what on earth use is that?  And still no one has explained coherently
why the sky will fall if we patch configure.ac and Makefile.am and
just rerun autoconf/automake in the specfile.

Earlier thread here if you have the stomach for it:
http://www.redhat.com/archives/fedora-devel-list/2008-October/thread.html...

Rich.

-- 
Richard Jones, Emerging Technologies, Red Hat  http://et.redhat.com/~rjones
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into Xen guests.
http://et.redhat.com/~rjones/virt-p2v

[Till Maas]

On Sun July 5 2009, Richard W.M. Jones wrote:

>  There's been lots of previous discussion of this silly idea of
>  patching generated code.  You end up carrying enormous patches
>  containing just line number changes that often can't be applied
>  upstream, and can't be carried forward to new upstream releases --
>  what on earth use is that?  And still no one has explained coherently
>  why the sky will fall if we patch configure.ac and Makefile.am and
>  just rerun autoconf/automake in the specfile. 
There is also the third alternative to patch configure.ac and Makefile.am, 
send the patches upstream, then run autoconf/automake once to get a patch for 
the upstream tarball and use this patch inside the spec. The patch in the spec 
may still be big, but it does not hurt anyone afaics.

Regards
Till

[Richard W.M. Jones]

On Sun, Jul 05, 2009 at 04:37:02PM +0200, Till Maas wrote:
>  On Sun July 5 2009, Richard W.M. Jones wrote:
>  
>  > There's been lots of previous discussion of this silly idea of
>  > patching generated code.  You end up carrying enormous patches
>  > containing just line number changes that often can't be applied
>  > upstream, and can't be carried forward to new upstream releases --
>  > what on earth use is that?  And still no one has explained coherently
>  > why the sky will fall if we patch configure.ac and Makefile.am and
>  > just rerun autoconf/automake in the specfile.
>
>  There is also the third alternative to patch configure.ac and
>  Makefile.am, send the patches upstream, then run autoconf/automake
>  once to get a patch for the upstream tarball and use this patch
>  inside the spec. The patch in the spec may still be big, but it does
>  not hurt anyone afaics. 
But WHY!??!!!

Why is it bad to patch configure.ac and rerun the autotools stuff?  I
do this all the time and it doesn't fail, even when we upgrade
autotools mid-release.

Please someone explain why this is bad.  It's totally stupid to go to
all this extra effort and carry huge patches against what are
essentially binary files, unless there's a really _really_ good reason
for it.

Rich.

-- 
Richard Jones, Emerging Technologies, Red Hat  http://et.redhat.com/~rjones
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://et.redhat.com/~rjones/virt-df/

[Mark McLoughlin]

On Sun, 2009-07-05 at 22:17 +0100, Richard W.M. Jones wrote:

>  Why is it bad to patch configure.ac and rerun the autotools stuff?
I used to avoid re-running autotools in rpm builds because I worried
that a future autotools update would subtly screw up the build - e.g.
disabling a previously enabled feature in the built package.

These were in the days that upstream maintainers distributing tarballs
had to be very careful what versions of autotools they used - e.g. I
recall something like GNOME folks using Fedora's version of autoconf
2.52 because upstream 2.52 had broken utf-8 support[1].

Perhaps those days are gone now and autotools are much more reliable. If
upstream people who run "make dist" don't pay much attention to what
autotools versions they are using, then why should Fedora packagers care
either?

Cheers,
Mark.

[1] - I could be totally wrong on the details, but I do remember Owen
(who started this thread) pointing out the problem to GNOME maintainers

[Sam Varshavchik]

Richard W.M. Jones writes:

>  There's been lots of previous discussion of this silly idea of
>  patching generated code.  You end up carrying enormous patches
>  containing just line number changes that often can't be applied
>  upstream, and can't be carried forward to new upstream releases -- 
What line number changes? You cut a patch against configure, and you're 
done. That's it.

When a later release rolls down the pike, the patch gets rebased, and fixed 
up, against a newer version.

I fail to see any substantive difference between that, and patching any 
other file in the source tarball. With a subsequent release, you'll still 
have to rebase your existing patch, if the new release did not fix the 
original bug. As I understand, rpm's default settings now reject fuzz in 
patch files, so you'll just have to do it, now. And since the likelyhood of 
configure changing in a new release is no different than any other source 
file getting changed, on average, believing that some work can be saved just 
by choosing to patch a different file, then the one that really needs to be 
patched, is somewhat naive.

>  what on earth use is that?  And still no one has explained
> coherently
>  why the sky will fall if we patch configure.ac and Makefile.am and
>  just rerun autoconf/automake in the specfile. 
Because autoconf and automake are going to change a lot more than just what 
the patch was intended to patch. It's fairly likely that the upstream is 
using a different version of autoconf and automake, so this ends up 
producing a brand new configure and makefile. If I were to do that, then 
I would find it necessary to spend additional time testing the new configure 
script, running it an eyeballing all of its voluminous output, watching for 
something that falls out, as a result of the new configure script.

Dunno, it just seems much easier to me to just patch a single line in 
configure, adding or fixing some directory's name, then to do all that. I 
get the impression that there's a meme going around that patching configure 
is some kind of a herculean, impossible task, and that's its easier to patch 
configure.in, then run autoconf to generate a brand new configure script.

I suppose that there may be some situations where rebuilding the configure 
script is the right thing to do, but I wouldn't expect to have this happen 
very often.

[Conrad Meyer]

On Sunday 05 July 2009 07:45:46 am Sam Varshavchik wrote:
>  *snip*
>
>  With a subsequent release, you'll still
>  have to rebase your existing patch, if the new release did not fix the
>  original bug. As I understand, rpm's default settings now reject fuzz in
>  patch files, so you'll just have to do it, now. And since the likelyhood of
>  configure changing in a new release is no different than any other source
>  file getting changed, on average, believing that some work can be saved
>  just by choosing to patch a different file, then the one that really needs
>  to be patched, is somewhat naive. 
The problem is that configure scripts are not written by a human, but 
generated by autoconf. It is easy to make small changes to configure.ac and 
generate large changes in configure. This makes it easier to rebase patches 
against configure.ac.

Regards,
-- 
Conrad Meyer <cemeyer(a)u.washington.edu&gt;

[Sam Varshavchik]

Conrad Meyer writes:

>  On Sunday 05 July 2009 07:45:46 am Sam Varshavchik wrote:
> > *snip*
> >
> > With a subsequent release, you'll still
> > have to rebase your existing patch, if the new release did not fix the
> > original bug. As I understand, rpm's default settings now reject fuzz in
> > patch files, so you'll just have to do it, now. And since the likelyhood of
> > configure changing in a new release is no different than any other source
> > file getting changed, on average, believing that some work can be saved
> > just by choosing to patch a different file, then the one that really needs
> > to be patched, is somewhat naive.
>  
>  The problem is that configure scripts are not written by a human, but 
>  generated by autoconf. It is easy to make small changes to configure.ac and 
>  generate large changes in configure. This makes it easier to rebase patches 
>  against configure.ac. 
The macros in configure.ac generally expand out to canned shell script 
fragments, with the macro's parameters substituted somewhere. Changing a 
parameter in configure.ac usually results in an equivalent substitution in 
configure.

Generally, only when one adds or removes entire macros from configure.ac, 
that's when this results in wholesale changes to configure.

In my experience, the overwhelming majority of fixups to configure scripts 
involve nothing more than adjusting someone's pathname, or compiler flags. 
For this kind of scope, rebuilding the entire configure script is overkill, 
and I wouldn't do it unless I audit it and verify whether or not upstream is 
relying on some specific behavior in the specific version of autoconf that 
was used to build the original configure script. Patching the configure 
script is much safer than patching configure.ac, then have autoconf grok all 
.m4 macros and rebuild the whole thing, likely ending up with a completely 
different beast, that not only includes your changes but who knows what 
else.

[Conrad Meyer]

On Sunday 05 July 2009 11:24:43 am Sam Varshavchik wrote:
>  Conrad Meyer writes:
>  > On Sunday 05 July 2009 07:45:46 am Sam Varshavchik wrote:
>  >> *snip*
>  >>
>  >> With a subsequent release, you'll still
>  >> have to rebase your existing patch, if the new release did not fix the
>  >> original bug. As I understand, rpm's default settings now reject fuzz in
>  >> patch files, so you'll just have to do it, now. And since the likelyhood
>  >> of configure changing in a new release is no different than any other
>  >> source file getting changed, on average, believing that some work can be
>  >> saved just by choosing to patch a different file, then the one that
>  >> really needs to be patched, is somewhat naive.
>  >
>  > The problem is that configure scripts are not written by a human, but
>  > generated by autoconf. It is easy to make small changes to configure.ac
>  > and generate large changes in configure. This makes it easier to rebase
>  > patches against configure.ac.
>
>  The macros in configure.ac generally expand out to canned shell script
>  fragments, with the macro's parameters substituted somewhere. Changing a
>  parameter in configure.ac usually results in an equivalent substitution in
>  configure. 
Right. Still, it makes more sense to patch the original source than the 
generated result, I think.

>  Generally, only when one adds or removes entire macros from
> configure.ac,
>  that's when this results in wholesale changes to configure.
>
>  In my experience, the overwhelming majority of fixups to configure scripts
>  involve nothing more than adjusting someone's pathname, or compiler flags.
>  For this kind of scope, rebuilding the entire configure script is overkill,
>  and I wouldn't do it unless I audit it and verify whether or not upstream
>  is relying on some specific behavior in the specific version of autoconf
>  that was used to build the original configure script. Patching the
>  configure script is much safer than patching configure.ac, then have
>  autoconf grok all .m4 macros and rebuild the whole thing, likely ending up
>  with a completely different beast, that not only includes your changes but
>  who knows what else. 
Unrelated, but I think this sort of phobia of regenerating an auto-generated 
script just goes to show how completely broken autotools is.

Regards,
-- 
Conrad Meyer <cemeyer(a)u.washington.edu&gt;

[Kevin Kofler]

Conrad Meyer wrote:
>  Unrelated, but I think this sort of phobia of regenerating an
>  auto-generated script just goes to show how completely broken autotools
>  is. 
+1, "auto-generated source code" is an oxymoron, this design is really
broken.

        Kevin Kofler

[Richard W.M. Jones]

On Sun, Jul 05, 2009 at 02:24:43PM -0400, Sam Varshavchik wrote:
>  For this kind of scope, rebuilding the entire configure 
>  script is overkill, and I wouldn't do it unless I audit it and verify 
>  whether or not upstream is relying on some specific behavior in the 
>  specific version of autoconf that was used to build the original 
>  configure script. 
So to make this patch, I've got to track down the exact version
of all the autotools that upstream happens to be using.  Great.

Rich.

-- 
Richard Jones, Emerging Technologies, Red Hat  http://et.redhat.com/~rjones
New in Fedora 11: Fedora Windows cross-compiler. Compile Windows
programs, test, and build Windows installers. Over 70 libraries supprt'd
http://fedoraproject.org/wiki/MinGW http://www.annexia.org/fedora_mingw

[Sam Varshavchik]

Richard W.M. Jones writes:

>  On Sun, Jul 05, 2009 at 02:24:43PM -0400, Sam Varshavchik wrote:
> > For this kind of scope, rebuilding the entire configure 
> > script is overkill, and I wouldn't do it unless I audit it and verify 
> > whether or not upstream is relying on some specific behavior in the 
> > specific version of autoconf that was used to build the original 
> > configure script.
>  
>  So to make this patch, I've got to track down the exact version
>  of all the autotools that upstream happens to be using.  Great. 
If you want to patch configure.ac, and then rerun autoconf to generate a 
brand new configure script, then, yes, that's what due diligence indicates.

But that's what /you/ want to do, not me. Me, I'll just apply a patch to the 
configure script, directly.

[Kevin Kofler]

Sam Varshavchik wrote:
>  But that's what /you/ want to do, not me. Me, I'll just apply
> a patch to
>  the configure script, directly. 
And you'll be violating the GPL (unless you're talking about a
BSD-style-licensed software or configure.ac is explicitly marked with
special permissions). The GPL requires you to edit the preferred form for
modification, which is definitely NOT a generated file.

        Kevin Kofler

[Chris Ball]

Hi,

>  And you'll be violating the GPL (unless you're talking about
> a
>  BSD-style-licensed software or configure.ac is explicitly marked
>  with special permissions). The GPL requires you to edit the
>  preferred form for modification, which is definitely NOT a
>  generated file. 
[citation needed]

I think this clause talks about *my* preferred form for modification,
not yours -- it's saying that I have to distribute my changes in the
form I created them in.

If you write a program in Perl, and I port it to Python and release
it, I haven't "violated the preferred form for modification" by not
using Perl.

- Chris.
-- 
Chris Ball   <cjb(a)laptop.org&gt;

[Sam Varshavchik]

Kevin Kofler writes:

>  Sam Varshavchik wrote:
> > But that's what /you/ want to do, not me. Me, I'll just apply a patch to
> > the configure script, directly.
>  
>  And you'll be violating the GPL (unless you're talking about a
>  BSD-style-licensed software or configure.ac is explicitly marked with
>  special permissions). The GPL requires you to edit the preferred form for
>  modification, which is definitely NOT a generated file. 
You do not understand the GPL.

[Ville Skyttä]

On Monday 06 July 2009, Kevin Kofler wrote:
>  Sam Varshavchik wrote:
>  > But that's what /you/ want to do, not me. Me, I'll just apply a patch to
>  > the configure script, directly.
>
>  And you'll be violating the GPL (unless you're talking about a
>  BSD-style-licensed software or configure.ac is explicitly marked with
>  special permissions). 
The FSF seems to disagree with that.

http://www.gnu.org/software/autoconf/manual/html_node/Distributing.html#D...

$ sed -ne '4,8 p' configure # this one was generated by autoconf 2.63
#
# Copyright (C) 1992, 1993, 1994, 1995, 1996, 1998, 1999, 2000, 2001,
# 2002, 2003, 2004, 2005, 2006, 2007, 2008 Free Software Foundation, Inc.
# This configure script is free software; the Free Software Foundation
# gives unlimited permission to copy, distribute and modify it.

[Kevin Kofler]

Ville Skyttä wrote:
>  The FSF seems to disagree with that.
>  
>  http://www.gnu.org/software/autoconf/manual/html_node/Distributing.html#D...
That applies to the automatically copied shell code, but not necessarily to
the code from the original configure.ac.

        Kevin Kofler

[Ville Skyttä]

On Tuesday 07 July 2009, Kevin Kofler wrote:
>  Ville Skyttä wrote:
>  > The FSF seems to disagree with that.
>  >
>  > http://www.gnu.org/software/autoconf/manual/html_node/Distributing.html#D
>  >istributing
>
>  That applies to the automatically copied shell code, but not necessarily to
>  the code from the original configure.ac. 
Well, the copyright notice at the top of configure (included in my previous 
mail) pretty clearly tells me what I can do with the script, and who to 
contact in case I'd disagree or have any questions.

[John5342]

2009/7/7 Ville Skyttä <ville.skytta(a)iki.fi&gt;:
>  On Tuesday 07 July 2009, Kevin Kofler wrote:
> > Ville Skyttä wrote:
> > > The FSF seems to disagree with that.
> > >
> > > http://www.gnu.org/software/autoconf/manual/html_node/Distributing.html#D
> > >istributing
> >
> > That applies to the automatically copied shell code, but not necessarily to
> > the code from the original configure.ac.
>
>  Well, the copyright notice at the top of configure (included in my previous
>  mail) pretty clearly tells me what I can do with the script, and who to
>  contact in case I'd disagree or have any questions. 
That might be true but although the header in configure doesn't
mention it it is generated based on whats in configure.ac and
therefore it could be considered a derivative work. End result is if
the package is e.g GPLv2 then configure.ac probably is and it then
follows that the generated configure is potentially also GPLv2.

-- 
There are 10 kinds of people in the world: Those who understand binary
and those who don't...

[Kevin Kofler]

Ville Skyttä wrote:
>  Well, the copyright notice at the top of configure (included in my
>  previous mail) pretty clearly tells me what I can do with the script, and
>  who to contact in case I'd disagree or have any questions. 
The FSF cannot claim copyright over the configure.ac code I or whoever else 
wrote and which ends up in the configure script, nor grant a license to use 
it. That copyright notice and license grant only applies to the code 
portions which are actually under FSF copyright.

        Kevin Kofler

[Orcan Ogetbil]

On Sun, Jul 5, 2009 at 2:24 PM, Sam Varshavchik wrote:
>  [cut]
>  Patching the configure
>  script is much safer than patching configure.ac, then have autoconf grok all
>  .m4 macros and rebuild the whole thing, likely ending up with a completely
>  different beast, that not only includes your changes but who knows what
>  else.
>  
What else? You and some other people are defending this from the
beginning of this thread but no one explained what else might change.
If I patch configure.ac and Makefile.am, then run autotools and build
the RPM package that way, what else might go in unnoticed?

Please back up your claims. I do not have much knowledge to make
claims in either direction but I am willing to learn.

Orcan

[Sam Varshavchik]

Orcan Ogetbil writes:

>  On Sun, Jul 5, 2009 at 2:24 PM, Sam Varshavchik wrote:
> > [cut]
> > Patching the configure
> > script is much safer than patching configure.ac, then have autoconf grok all
> > .m4 macros and rebuild the whole thing, likely ending up with a completely
> > different beast, that not only includes your changes but who knows what
> > else.
> >
>  
>  What else? You and some other people are defending this from the
>  beginning of this thread but no one explained what else might change.
>  If I patch configure.ac and Makefile.am, then run autotools and build
>  the RPM package that way, what else might go in unnoticed? 
Why are you asking me? I'm the one arguing against patching configure.ac and 
Makefile.am and rerunning autotools.

>  Please back up your claims. I do not have much knowledge to make
>  claims in either direction but I am willing to learn. 
You can start by reading this thread, again.

[Orcan Ogetbil]

On Sun, Jul 5, 2009 at 10:00 PM, Sam Varshavchik wrote:
>  Orcan Ogetbil writes:
>
> > On Sun, Jul 5, 2009 at 2:24 PM, Sam Varshavchik wrote:
> >>
> >> [cut]
> >> Patching the configure
> >> script is much safer than patching configure.ac, then have autoconf grok
> >> all
> >> .m4 macros and rebuild the whole thing, likely ending up with a
> >> completely
> >> different beast, that not only includes your changes but who knows what
> >> else.
> >>
> >
> > What else? You and some other people are defending this from the
> > beginning of this thread but no one explained what else might change.
> > If I patch configure.ac and Makefile.am, then run autotools and build
> > the RPM package that way, what else might go in unnoticed?
>
>  Why are you asking me? I'm the one arguing against patching configure.ac and
>  Makefile.am and rerunning autotools.
>
> > Please back up your claims. I do not have much knowledge to make
> > claims in either direction but I am willing to learn.
>
>  You can start by reading this thread, again.
>
>  
No, I believe that you are the one who needs to read again. :)

Read my post. I know that you are against patching configure.ac and I
ask you what might go wrong unnoticed if you do patch configure.ac.
Example cases? please!

Orcan

[Richard W.M. Jones]

On Sun, Jul 05, 2009 at 10:45:46AM -0400, Sam Varshavchik wrote:
>  What line number changes? You cut a patch against configure, and
> you're  
>  done. That's it. 
And you get a big patch containing line numbers.  Here's a single line
change to configure.ac, and the corresponding patch that generates:

http://annexia.org/tmp/configure.patch

Note: I'm not even changing the version of autoconf, because really
what you mean is I have to track down the exact same version of all
autotools that the upstream author used.  If you don't do that, then
you get a _real_ big patch.

>  I fail to see any substantive difference between that, and patching
> any  
>  other file in the source tarball. With a subsequent release, you'll still 
>  have to rebase your existing patch, if the new release did not fix the  
>  original bug. 
No you don't - often patches continue to work over versions.  Not in
the case where you're patching a binary of course ...

Rich.

-- 
Richard Jones, Emerging Technologies, Red Hat  http://et.redhat.com/~rjones
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://et.redhat.com/~rjones/virt-df/

[Sam Varshavchik]

Richard W.M. Jones writes:

>  On Sun, Jul 05, 2009 at 10:45:46AM -0400, Sam Varshavchik wrote:
> > What line number changes? You cut a patch against configure, and you're  
> > done. That's it.
>  
>  And you get a big patch containing line numbers.  Here's a single line
>  change to configure.ac, and the corresponding patch that generates: 
 
======================

Who said anything about patching configure.ac? The cited link is not a patch 
to the configure script, it's a result of a patch to configure.ac.

I'll repeat again: patch configure, not configure.ac.

I said "patch configure", and you reply, "well, it won't work because
if you 
patch configure.ac, run autoconf, then generate the patch between the 
original configure, and the new configure, I get a big hairball". Duh.

I've been reading configure scripts long enough to know that your 
pseudo-patch is the result of adding AC_PATH_PROG(PERL, perl) to 
configure.ac.

Now, why don't you explain why, and we'll go from there. Presuming that this 
is needed because some script in $srcdir uses the PERL environment variable, 
or a later part of the configure script itself (it assumes that PERL is set 
in the environment) then I would just patch in:

PERL=/usr/bin/perl
export PERL

instead, to configure. Or, have the spec file set PERL itself, before 
running configure. If the reason for the patch is that there's some *.in in 
src with a @PERL@ placeholder, I'd just patch that *.in file directly, 
instead.

No need to uselessly screw around with configure, when I don't even need to 
do it.

Like I said earlier, there seems to be a meme going around that making a 
minor fix to configure is an impossible task. It can't be done, the only 
option is to fix configure.ac, and rerun autoconf. configure itself is 
untouchable. You can't patch it, you have to patch configure.ac, and then 
regenerate it.

[Kevin Kofler]

Sam Varshavchik wrote:
>  Now, why don't you explain why, and we'll go from there.
> Presuming that
>  this is needed because some script in $srcdir uses the PERL environment
>  variable, or a later part of the configure script itself (it assumes that
>  PERL is set in the environment) then I would just patch in:
>  
>  PERL=/usr/bin/perl
>  export PERL
>  
>  instead, to configure. 
This is a really dirty hack and cannot be upstreamed. Fixing configure.ac is
the clean fix.

>  Like I said earlier, there seems to be a meme going around that
> making a
>  minor fix to configure is an impossible task. It can't be done, the only
>  option is to fix configure.ac, and rerun autoconf. configure itself is
>  untouchable. You can't patch it, you have to patch configure.ac, and then
>  regenerate it. 
That "meme" goes around for a reason, patching a generated file is dirty,
can be hard to do depending on what you want to patch, leads to patches
which can't be upstreamed and, if the source code is GPLed, violates the
GPL.

        Kevin Kofler

[Sam Varshavchik]

Kevin Kofler writes:

>  Sam Varshavchik wrote:
> > Now, why don't you explain why, and we'll go from there. Presuming that
> > this is needed because some script in $srcdir uses the PERL environment
> > variable, or a later part of the configure script itself (it assumes that
> > PERL is set in the environment) then I would just patch in:
> > 
> > PERL=/usr/bin/perl
> > export PERL
> > 
> > instead, to configure.
>  
>  This is a really dirty hack and cannot be upstreamed. 
I see. A surgical patch that addresses the build failure directly is dirty. 
It's much cleaner to patch a different file that's one step removed, then 
run a tool to regenerate the script where the original problem was, and hope 
that there are not going to be any side effects.

And I don't recall ever claiming that this should be sent directly 
upstream. Upstrem can be informed in parallel, so that the proper fix can be 
put in place.

>                                                        Fixing
> configure.ac is
>  the clean fix. 
Fixing configure.ac accomplishes absolutely nothing. One has to run 
autotools to propagate the fix, and that's the messy part. Like I said, due 
diligence would require you to track which specific version of autotools the 
upstream used to generate the original scripts, and whether or not there's 
any code in configure.ac that's specific to that version of autotools, so 
that any side effects can be properly vetted, when the entire build system 
gets regenerated by autotools.

>  That "meme" goes around for a reason, patching a generated
> file is dirty, 
> From the viewpoint of a package builder it is not a generated file. It
> comes  
right out of the tarball, as is.

>  can be hard to do depending on what you want to patch, leads to
> patches
>  which can't be upstreamed and, if the source code is GPLed, violates the
>  GPL. 
How exactly would that violate the GPL?

[Kevin Kofler]

Sam Varshavchik wrote:
>  How exactly would that violate the GPL? 
You aren't patching the actual source code.

        Kevin Kofler

[Adam Jackson]

On Mon, 2009-07-06 at 14:22 +0200, Kevin Kofler wrote:
>  Sam Varshavchik wrote:
>  > How exactly would that violate the GPL?
>  
>  You aren't patching the actual source code. 
Assuming GPLv2, the term in the license that you're referring to is
"preferred form".  There is clearly some difference of opinion as to
what the preferred form is here.  In a strict construction sense, the
preferred form for modification is whatever the modifier opted to
modify.

More concretely, the source code on offer in section 3 is the
"corresponding source", meaning, the code and changes _you_ used to
produce the binary.  If you changed the generated source, well, that's a
thing you can do, and it means you have to distribute those changes.  If
you change the metasource, that's also a thing you can do, and you have
to distribute the recipe for creating the generated source.

In other words: no.

- ajax

[Sam Varshavchik]

Kevin Kofler writes:

>  Sam Varshavchik wrote:
> > How exactly would that violate the GPL?
>  
>  You aren't patching the actual source code. 
Oh, no!  You mean, the tarball I downloaded from upstream, labeled "source 
code", did not actually contain the source code?

Looks like I've been snookered.

[Kevin Kofler]

Sam Varshavchik wrote:
>  Oh, no!  You mean, the tarball I downloaded from upstream, labeled
> "source
>  code", did not actually contain the source code? 
It contains both the actual source code and some unreadable generated 
gibberish which is NOT source code and which is being passed off as such 
(which is why the autotools are broken by design: it's a mistake to 
encourage shipping generated files in a source tarball).

        Kevin Kofler

[Sam Varshavchik]

Kevin Kofler writes:

>  Sam Varshavchik wrote:
> > Oh, no!  You mean, the tarball I downloaded from upstream, labeled "source
> > code", did not actually contain the source code?
>  
>  It contains both the actual source code and some unreadable generated 
>  gibberish which is NOT source code and which is being passed off as such  
Just because you can't read it, it's not gibberish.

Besides, Merriam-Webster defines "source code" as:

http://www.merriam-webster.com/dictionary/source%20code

"a computer program in its original programming language (as FORTRAN or C) 
before translation into object code usually by a compiler"

You learn something new about configure scripts every day. I didn't know 
that gets translated into object code, before execution.

>  (which is why the autotools are broken by design: it's a mistake
> to 
>  encourage shipping generated files in a source tarball). 
Oh, ok. Good luck with your quest to change the mind of everyone who uses 
autoconf, to do it your way. Perhaps you'd like to show everyone how it 
should be done. Pick just one moderately popular package, convince them to 
let you own release management, then start releasing tarballs without a 
configure script. Let us know how it works out, but kindly give advance 
warning. I want to stock up on earplugs.

[Orcan Ogetbil]

Wow! 78 messages and still, no one gave solid examples of what might
go wrong unnoticed if one uses autotools in a specfile.

"Using autotools in a specfile is bad" started to sound like an urban
legend to me.

I'll keep reading.

Orcan

[Sam Varshavchik]

Orcan Ogetbil writes:

>  Wow! 78 messages and still, no one gave solid examples of what might
>  go wrong unnoticed if one uses autotools in a specfile. 
I already did, several times. You just ignored it.

[Orcan Ogetbil]

On Mon, Jul 6, 2009 at 7:22 PM, Sam Varshavchik wrote:
>  Orcan Ogetbil writes:
>
> > Wow! 78 messages and still, no one gave solid examples of what might
> > go wrong unnoticed if one uses autotools in a specfile.
>
>  I already did, several times. You just ignored it.
>  
Would you kindly give quotes or links to these examples? I read all
your messages for the 5th time and I still can't find your examples.

Many thanks!
Orcan

[Sam Varshavchik]

Orcan Ogetbil writes:

>  On Mon, Jul 6, 2009 at 7:22 PM, Sam Varshavchik wrote:
> > Orcan Ogetbil writes:
> >
> >> Wow! 78 messages and still, no one gave solid examples of what might
> >> go wrong unnoticed if one uses autotools in a specfile.
> >
> > I already did, several times. You just ignored it.
> >
>  
>  Would you kindly give quotes or links to these examples? I read all
>  your messages for the 5th time and I still can't find your examples. 
# Message-ID: <cone.1246920650.559785.28501.500(a)commodore.email-scan.com&gt;

# I guess it all comes down to what's easier: vetting the impact of your 
# minimalist changes to configure, versus vetting a freshly minted configure 
# script for any unintended side effects from regenerating it using a -- 
# very likely -- different version of autoconf than the upstream used 
# originally.

I specifically cited the potential danger from rebuilding configure that 
came out of a different version of autoconf than what the upstream used -- 
and I explicitly stated this three or four times.

It just fits into your blind spot so nicely -- because you are firmly 
convinced that there is never any downside, you completely ignore everytime 
someone brings up an obvious one.

Tell me what -- every time you choose to rebuild an upstream's configure -- 
do you always notice which specific version of autoconf the upstream used 
originally? Well, unless you always do so, it's very easy for something to 
go "unnoticed" by you.

Thank you for playing.

[Orcan Ogetbil]

On Mon, Jul 6, 2009 at 9:13 PM, Sam Varshavchik wrote:
>  Orcan Ogetbil writes:
>
> > On Mon, Jul 6, 2009 at 7:22 PM, Sam Varshavchik wrote:
> >>
> >> Orcan Ogetbil writes:
> >>
> >>> Wow! 78 messages and still, no one gave solid examples of what might
> >>> go wrong unnoticed if one uses autotools in a specfile.
> >>
> >> I already did, several times. You just ignored it.
> >>
> >
> > Would you kindly give quotes or links to these examples? I read all
> > your messages for the 5th time and I still can't find your examples.
>
>  # Message-ID: <cone.1246920650.559785.28501.500(a)commodore.email-scan.com&gt;
>
>  # I guess it all comes down to what's easier: vetting the impact of your #
>  minimalist changes to configure, versus vetting a freshly minted configure #
>  script for any unintended side effects from regenerating it using a -- #
>  very likely -- different version of autoconf than the upstream used #
>  originally.
>
>  I specifically cited the potential danger from rebuilding configure that
>  came out of a different version of autoconf than what the upstream used --
>  and I explicitly stated this three or four times.
>  
Yes you did say that it is dangerous a few times. But you never said
what the consequences would be, what the dangers actually are.

The only one closest example was given by Mark McLoughlin:

>  I used to avoid re-running autotools in rpm builds because I worried
>  that a future autotools update would subtly screw up the build - e.g.
>  disabling a previously enabled feature in the built package. 
but this will hardly go unnoticed.

>  It just fits into your blind spot so nicely -- because you are
> firmly
>  convinced that there is never any downside, you completely ignore everytime
>  someone brings up an obvious one.
>  
Hold on there. I am not ignoring. I am curiously reading because, as I
said, I'm willing to learn.

I am completely neutral about this issue. You don't need to fight me
:) Just show me some evidence so I'll get convinced into your side.

>  Tell me what -- every time you choose to rebuild an upstream's
> configure --
>  do you always notice which specific version of autoconf the upstream used
>  originally? Well, unless you always do so, it's very easy for something to
>  go "unnoticed" by you.
>  
What is this "something"? I am begging you to give me one (1) example.
I am not sarcastic at all. I am very sincere in this statement.

No, I don't really check what version of autotools upstream used. But
I look at the build logs and check the resulting binary.  If
everything looks reasonable I send it to updates-testing. I keep it
there for about 2 weeks (sometimes longer but most usually not
shorter). If there are no complaints I then push it to stable.

So, let us start from the beginning: Let's say I modify configure.ac
and use automake/autoconf during building my package. The package
builds and seems to work "fine". In what step can I miss "something"?
What will this "something" be?

Please stop the fight and help me. I need help. Thanks,
Orcan

[Sam Varshavchik]

Orcan Ogetbil writes:

>  On Mon, Jul 6, 2009 at 9:13 PM, Sam Varshavchik wrote: 
> > I specifically cited the potential danger from rebuilding
> configure that
> > came out of a different version of autoconf than what the upstream used --
> > and I explicitly stated this three or four times.
> >
>  
>  Yes you did say that it is dangerous a few times. But you never said
>  what the consequences would be, what the dangers actually are. 
I see. You want me to explain the possible consequences of a broken build?

>  Hold on there. I am not ignoring. I am curiously reading because, as
> I
>  said, I'm willing to learn.
>  
>  I am completely neutral about this issue. You don't need to fight me
>  :) Just show me some evidence so I'll get convinced into your side. 
There is no universal consequence of a broken build, and there is no tell 
tale symptom to watch for.

One example from memory occured about four years ago. Someone else, at 
$dayjob$, was trying to bootstrap the entire toolchain, including gcc. On 
multiple platforms.

Everything seemed to build, and work. Yet, when it was time to build other 
applications, gcc on x86_64 was barfing every time it tried to compile a 64 
bit shared library. Only on x86_64. An error was coming out of the linker. 
Just a typical undecipherable ld error message, that seems to be written in 
English words, but in a language other than English.

Anyway, to make a long story short, I ended up running some sample code 
through both the native (red hat) gcc compiler, and the bootstrapped 
compiler, producing .s assembly files.  Of course, I know x86_64 machine 
language as much as I know Latin, but there were differences. By staring at 
what ended up in the .s files, and some Googling, I eventually traced it 
down to an obscure breakage in one of the configure scripts. It was frobbing 
the version of ld, and, based on the results, making some decision as to 
what features to enable or disable. The breakage was that it was picking up 
the wrong ld (the native one, rather than the one being bootstrapped).

This is just one example of the results of a broken configure script. 
Everything would seem to build, apparently, but with the wrong options for 
the given platform, and you wouldn't realize that something was broken until 
a lot of other people already wasted a lot of time.

>  What is this "something"? I am begging you to give me one
> (1) example.
>  I am not sarcastic at all. I am very sincere in this statement. 
All you really have to observe is that configure does not run with the -e 
flag set, so if something in the massive shell script fails with a non-zero 
exit code, the rest of the configure script continues to plod along, with 
just a diagnostic message on stderr, which is easily lost amidst all the 
other noisy output produced by configure.

>  No, I don't really check what version of autotools upstream used.
> But
>  I look at the build logs and check the resulting binary.  If 
Unfortunately, it's not that easy. You may not very well realize that, on 
your platform, the result of some specific "checking for foo" should be 
"yes", instead of "no" (or the other way around), but breakage
introduced by 
a rebuild by a different version of autoconf flipped that setting.

>  So, let us start from the beginning: Let's say I modify
> configure.ac
>  and use automake/autoconf during building my package. The package
>  builds and seems to work "fine". In what step can I miss "something"?
You want me to give you a universal answer how to detect a problem caused by 
breakage due to a rebuild by a different version of autoconf?

As if this can only fail in one specific way, no matter what the break is. 
You believe that a single, specific, light bulb goes on, when configure 
produces wrong results, no matter what those wrong results are.

Riiiiight.

>  What will this "something" be? 
Could be anything. There is no single error message that is a tell-tale 
indication that it's broke.

And the chances of happening that as a result of making a targeted patch to 
configure, with well defined consequences, are orders of magnitude less than 
patching configure.ac, and then generating a brand. new. script.

[Ben Boeckel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Sam Varshavchik wrote:
<snip>

Frankly, all of this would scare me away from the autotools. If 
all that can happen from patching a small file rather than a 
monster file...

Forcing version x.y of some tool and then shipping the results of 
using said tool and then slapping the hands of those who try to 
move forward with a new version seems to me that something is 
wrong.

- --Ben
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (GNU/Linux)
>
> iEYEARECAAYFAkpVI30ACgkQiPi+MRHG3qTD8wCdEDKzia6Sig67u5TFuQGnDmSe
> xDEAoK+oUfvYZAHzuEb+Z4NbBdEcnuzK
> =Cuty
> -----END PGP SIGNATURE-----

[Mark McLoughlin]

On Mon, 2009-07-06 at 21:13 -0400, Sam Varshavchik wrote:

>  It just fits into your blind spot so nicely -- because you are firmly
>
>  convinced that there is never any downside, you completely ignore everytime 
>  someone brings up an obvious one. 
Have a look at http://live.gnome.org/CodeOfConduct

e.g. "assume people mean well" - IMHO, Orcan is just looking for
specific examples of things breaking in the past because he genuinely
wants to understand the issue. And there hasn't been specific examples
on this thread yet.

>  Tell me what -- every time you choose to rebuild an upstream's
> configure -- 
>  do you always notice which specific version of autoconf the upstream used 
>  originally? Well, unless you always do so, it's very easy for something to 
>  go "unnoticed" by you. 
A counter point is how many upstream maintainers look at what version of
autoconf they're building tarballs with?

My guess is that these days very, very few do and it's not causing
problems - perhaps autoconf isn't so broken anymore?

(Bear in mind, I used to advocate what you're advocating, but it's
looking like the situation has changed)

Cheers,
Mark.

[Kevin Kofler]

Sam Varshavchik wrote:
>  Just because you can't read it, it's not gibberish. 
It's not that *I* can't read it, it's that it is just plain hard to read, 
especially because it contains workarounds for bazillions of broken 
proprietary *nix shells (trying to use Bourne-style shell code as a portable 
language is a major design failure of its own, there are tons of subtly 
different dialects and megatons of plain bugs).

Try reading that 1.1 MB (!) shell script:
http://svn.calcforge.org/viewvc/emu-
tigcc/trunk/configure?revision=2861&root=emu-tigcc

It's generated from a 9.7 KB configure.ac:
http://svn.calcforge.org/viewvc/emu-
tigcc/trunk/configure.ac?revision=2847&root=emu-tigcc

It uses the stock KDE 3 acinclude.m4 and stock macros fetched by aclocal, no 
special magic.

Needless to say, getting that project off the autotools is a high-priority 
item for me…

>  Besides, Merriam-Webster defines "source code" as:
>  
>  http://www.merriam-webster.com/dictionary/source%20code
>  
>  "a computer program in its original programming language (as FORTRAN or C) 
The original programming language is the configure.ac language.

>  before translation into object code usually by a compiler"
And the object code is the shell code.

>  You learn something new about configure scripts every day. I
> didn't know
>  that gets translated into object code, before execution. 
They get translated into shell code which is interpreted by the shell (which 
is used as a kind of VM). But the shell code is clearly NOT the ORIGINAL 
programming language, it's generated.

>  Oh, ok. Good luck with your quest to change the mind of everyone who
> uses
>  autoconf, to do it your way. 
Actually they should just stop using autoconf because it is not designed for 
doing things right.

And in fact KDE did just that (they moved to CMake and nobody at KDE is 
missing the autotools, quite the opposite!) and several other projects 
followed suit.

        Kevin Kofler

[Sam Varshavchik]

Kevin Kofler writes:

>  Sam Varshavchik wrote:
> > Just because you can't read it, it's not gibberish.
>  
>  It's not that *I* can't read it, it's that it is just plain hard to read, 
>  especially because it contains workarounds for bazillions of broken 
>  proprietary *nix shells (trying to use Bourne-style shell code as a portable 
>  language is a major design failure of its own, there are tons of subtly 
>  different dialects and megatons of plain bugs).
>  
>  Try reading that 1.1 MB (!) shell script:
>  http://svn.calcforge.org/viewvc/emu-
>  tigcc/trunk/configure?revision=2861&root=emu-tigcc
>  
>  It's generated from a 9.7 KB configure.ac:
>  http://svn.calcforge.org/viewvc/emu-
>  tigcc/trunk/configure.ac?revision=2847&root=emu-tigcc 
Sure, why not. It just so happens that, not too long ago, I was in an 
analogous position where I had some other package that also built against 
zlib, for $dayjob$. At $dayjob$ we also package free software using a 
scripted reproducible build. Not RPMs, but an analogous process, and at 
$dayjob$, for reasons that are irrelevant, zlib was installed into a 
nonstandard location.

The fix for that, and the analogous fix here, were that to be the case, was 
to stick

CPPFLAGS="-I <path> $CPPFLAGS"

right after the following line in configure, grep for it:

# Check for zlib

I'm of course, skipping over some other details (similar thing needs to be 
done with LDFLAGS).

So, you see -- it's not really complicated. Not at all. And, this is a 
typical reason why one might need to fix up the configure script. In at 
least 99%+ of the time

Perhaps if it's necessary to do major surgery on some package's configure 
script, for some reason -- if wholesale changes are required -- one might 
have an argument for patching configure.ac (or configure.in, for us 
old-timers), and rebuilding configure. But for 99% of the actual use case 
situations out there, it's like driving in a 1" nail with a sledgehammer. 
Too much risk for collateral damage.

>  And in fact KDE did just that (they moved to CMake and nobody at KDE
> is 
>  missing the autotools, quite the opposite!) and several other projects 
>  followed suit. 
Yes, well, that might be one of the reasons why KDE is sweeping over the 
Linux desktop, and Gnome is just a fading memory for most.

[OT] Re: an update to automake-1.11?

[Peter Gordon]

On Mon, 2009-07-06 at 21:24 -0400, Sam Varshavchik wrote:
>  Yes, well, that might be one of the reasons why KDE is sweeping over
> the 
>  Linux desktop, and Gnome is just a fading memory for most. 
Please don't claim such obviously fallacious things. Like it or not,
GNOME has been - and continues to be - the default desktop environment
on a great many major GNU/Linux distributions (including Fedora). 
-- 
Peter Gordon (codergeek42) <peter(a)thecodergeek.com&gt;
Who am I? :: http://thecodergeek.com/about-me

[Sam Varshavchik]

Peter Gordon writes:

>  On Mon, 2009-07-06 at 21:24 -0400, Sam Varshavchik wrote:
> > Yes, well, that might be one of the reasons why KDE is sweeping over the 
> > Linux desktop, and Gnome is just a fading memory for most.
>  
>  Please don't claim such obviously fallacious things. Like it or not,
>  GNOME has been - and continues to be - the default desktop environment
>  on a great many major GNU/Linux distributions (including Fedora).  
Yes -- I plead guilty. I often forget to put explicit <sarcasm> tags in the 
right places. Perhaps you might want to reread my entire message.

[Kevin Kofler]

Sam Varshavchik wrote:
>  Sure, why not. It just so happens that, not too long ago, I was in
> an
>  analogous position where I had some other package that also built against
>  zlib, for $dayjob$. At $dayjob$ we also package free software using a
>  scripted reproducible build. Not RPMs, but an analogous process, and at
>  $dayjob$, for reasons that are irrelevant, zlib was installed into a
>  nonstandard location. 
In CMake, in such a case, you just have to use:
-DCMAKE_PREFIX_PATH="<path>"
It can also be exported as an environment variable. Either way, no changes 
to the scripts needed at all.

        Kevin Kofler

[Sam Varshavchik]

Kevin Kofler writes:

>  Sam Varshavchik wrote:
> > Sure, why not. It just so happens that, not too long ago, I was in an
> > analogous position where I had some other package that also built against
> > zlib, for $dayjob$. At $dayjob$ we also package free software using a
> > scripted reproducible build. Not RPMs, but an analogous process, and at
> > $dayjob$, for reasons that are irrelevant, zlib was installed into a
> > nonstandard location.
>  
>  In CMake, in such a case, you just have to use:
>  -DCMAKE_PREFIX_PATH="<path>"
>  It can also be exported as an environment variable. Either way, no changes 
>  to the scripts needed at all. 
That's great, and if this discussion was about cmake, then this would be a 
valid point. But, this thread is not about cmake.

[Kevin Kofler]

Sam Varshavchik wrote:
>  That's great, and if this discussion was about cmake, then this
> would be a
>  valid point. But, this thread is not about cmake. 
That CMake has this feature implies that the autotools suck for not having 
it and forcing you to patch the configure script for your usecase.

        Kevin Kofler

[Adam Jackson]

On Sun, 2009-07-05 at 18:50 -0400, Sam Varshavchik wrote:
>  Richard W.M. Jones writes:
>  > On Sun, Jul 05, 2009 at 10:45:46AM -0400, Sam Varshavchik wrote:
>  >> What line number changes? You cut a patch against configure, and you're  
>  >> done. That's it.
>  > 
>  > And you get a big patch containing line numbers.  Here's a single line
>  > change to configure.ac, and the corresponding patch that generates:
>    ======================
>  
>  Who said anything about patching configure.ac? The cited link is not a patch 
>  to the configure script, it's a result of a patch to configure.ac.
>  
>  I'll repeat again: patch configure, not configure.ac.
>  
>  I said "patch configure", and you reply, "well, it won't work because
> if you 
>  patch configure.ac, run autoconf, then generate the patch between the 
>  original configure, and the new configure, I get a big hairball". Duh. 
The fundamental problem with patching configure instead of configure.ac
(or Makefile instead of Makefile.am) is that it's changing the wrong
semantic level.  As a bad example (because sed would be a better tool),
imagine a patch to Makefile that does basically s/-O2/-Os/g.  Now
upstream makes a new release, and adds a new build target.  Your patch
might still apply, but it'll miss the CFLAGS emitted for the new target,
and your patch will no longer _mean_ the same thing it used to.

This is the same reason we patch .c files, and not the intermediate .i
or .S.  Don't let the fact that you never see the intermediate files in
the tarball confuse you.  You don't see them because they're not
abstraction levels humans should have to deal with; and neither is the
complete expanded configure script.

- ajax

[Sam Varshavchik]

Adam Jackson writes:

>  On Sun, 2009-07-05 at 18:50 -0400, Sam Varshavchik wrote:
> > Richard W.M. Jones writes:
> > > On Sun, Jul 05, 2009 at 10:45:46AM -0400, Sam Varshavchik wrote:
> > >> What line number changes? You cut a patch against configure, and you're 
>
> > >> done. That's it.
> > > 
> > > And you get a big patch containing line numbers.  Here's a single line
> > > change to configure.ac, and the corresponding patch that generates:
> >   ======================
> > 
> > Who said anything about patching configure.ac? The cited link is not a patch 
> > to the configure script, it's a result of a patch to configure.ac.
> > 
> > I'll repeat again: patch configure, not configure.ac.
> > 
> > I said "patch configure", and you reply, "well, it won't work
> because if you 
> > patch configure.ac, run autoconf, then generate the patch between the 
> > original configure, and the new configure, I get a big hairball". Duh.
>  
>  The fundamental problem with patching configure instead of configure.ac
>  (or Makefile instead of Makefile.am) is that it's changing the wrong
>  semantic level. 
As was discussed previously in this thread, when creating packages the 
objective is not to patch the correct semantic level. If there's a problem 
that prevents the source from getting built properly, it's my understanding 
that the objective is to fix it in the way that absolutely minimizes the 
chance of accidentally introducing a side effect that creates a new problem 
that did not exist before.

Whatever you would like the upstream to do for the next release, is a 
separate task. It may or may not be the same thing.

So, the choices are, once it's identified where configure goes wrong are:

1) Fix the configure script, with shellcode whose contents are well 
understood

2) Patch configure.ac, and feed it to a code generator that spits out a 
brand new configure script.

Your turn. Of course, if you take #2, you would, of course, verify which 
specific version of autoconf the upstream used, and whether the differences 
between your's and upstream's autoconf does not have any other impacts on 
the configure script.

[Adam Jackson]

On Mon, 2009-07-06 at 17:53 -0400, Sam Varshavchik wrote:

>  So, the choices are, once it's identified where configure goes
> wrong are:
>  
>  1) Fix the configure script, with shellcode whose contents are well 
>  understood
>  
>  2) Patch configure.ac, and feed it to a code generator that spits out a 
>  brand new configure script.
>  
>  Your turn. Of course, if you take #2, you would, of course, verify which 
>  specific version of autoconf the upstream used, and whether the differences 
>  between your's and upstream's autoconf does not have any other impacts on 
>  the configure script. 
I suppose it depends whether you consider the initial act of package
creation, or the continued maintenance of that packaging, to be more
time consuming.  All I know is that rediffing patches to configure.ac
takes way less time than rediffing against configure, and that as a
practice that hasn't (non-trivially) bitten me once in over three years,
where configure patches have.

- ajax

[Sam Varshavchik]

Adam Jackson writes:

>  On Mon, 2009-07-06 at 17:53 -0400, Sam Varshavchik wrote:
>  
> > So, the choices are, once it's identified where configure goes wrong are:
> > 
> > 1) Fix the configure script, with shellcode whose contents are well 
> > understood
> > 
> > 2) Patch configure.ac, and feed it to a code generator that spits out a 
> > brand new configure script.
> > 
> > Your turn. Of course, if you take #2, you would, of course, verify which 
> > specific version of autoconf the upstream used, and whether the differences 
> > between your's and upstream's autoconf does not have any other impacts on 
> > the configure script.
>  
>  I suppose it depends whether you consider the initial act of package
>  creation, or the continued maintenance of that packaging, to be more
>  time consuming.  All I know is that rediffing patches to configure.ac
>  takes way less time than rediffing against configure, and that as a 
Gee, I didn't know that rediffing is a mandatory step. Here I was, just 
fixing configure by opening it in emacs, adding one or two lines, or 
changing a variable setting, saving it the diffing the results against the 
original configure file, producing a tiny patch. And I was doing it wrong 
way all along…

[Kevin Kofler]

Sam Varshavchik wrote:
>  Gee, I didn't know that rediffing is a mandatory step. 
It is when your patch no longer applies after you upgraded the package to a 
new upstream version.

        Kevin Kofler

[Sam Varshavchik]

Kevin Kofler writes:

>  Sam Varshavchik wrote:
> > Gee, I didn't know that rediffing is a mandatory step.
>  
>  It is when your patch no longer applies after you upgraded the package to a 
>  new upstream version. 
Which, as I pointed out, is still the case if you were to patch configure.ac 
instead.

But, go ahead and ignore this inconvenient fact, too.

[Kevin Kofler]

Sam Varshavchik wrote:
>  Which, as I pointed out, is still the case if you were to patch
>  configure.ac instead.
>  
>  But, go ahead and ignore this inconvenient fact, too. 
As I explained (and you ignored), configure.ac tends to change a lot less 
between upstream releases, especially with a lot fewer irrelevant changes 
like line number changes or changes in aclocal snippets (because upstream 
WILL regenerate the file, not surgically edit it!), so it's a lot less 
likely to produce fuzz.

        Kevin Kofler

[Sam Varshavchik]

Kevin Kofler writes:

>  Sam Varshavchik wrote:
> > Which, as I pointed out, is still the case if you were to patch
> > configure.ac instead.
> > 
> > But, go ahead and ignore this inconvenient fact, too.
>  
>  As I explained (and you ignored), configure.ac tends to change a lot less 
>  between upstream releases, especially with a lot fewer irrelevant changes  
This may come as a shock to some, but configure does not often change unless 
configure.ac changes too.

So, I'm not sure what does the frequency of changes to configure.ac has to 
do with anything.

>  like line number changes or changes in aclocal snippets (because
> upstream 
>  WILL regenerate the file, not surgically edit it!), so it's a lot less 
>  likely to produce fuzz. 
99% of the time when configure changes, it's due to changes in configure.ac.

If someone thinks that by patching configure.ac, instead of configure, one 
achieves tremendous savings in the frequency of needing to rebase one's 
patches, they're in a desperate need for a reality check.

Furthermore, changes to configure.ac will more likely to result in a more 
frequent manual rebases, where the corresponding changes in configure are 
far more likely to result in much simpler rebasing that's limited only to 
eliminating the fuzz. If one patches a line or two in configure.ac, then if 
the upstream futzes around with a neighboring line, the patch is going to 
get kicked out, and must be manually rebased. On the other hand, if a 
corresponding patch was against configure, instead, (and by that I mean a 
real patch, and not a patch to configure.ac followed by autoconf followed by 
a diff of the new configure against the old, I hate to explicitly say this, 
but some folks around here have a mental block on this subject, and can't 
wrap their brains around the concept of patching configure directly), the 
corresponding differences in configure would've ended up being hundred of 
lines apart, resulting in nothing more than some fuzz, that can be 
automatically updated.

[Kevin Kofler]

Sam Varshavchik wrote:
>  This may come as a shock to some, but configure does not often
> change
>  unless configure.ac changes too.
>  
>  So, I'm not sure what does the frequency of changes to configure.ac has to
>  do with anything. 
Where your argument falls apart is that patch fuzz is a local concept! It's 
irrelevant that the file is not byte-identical. What matters is whether the 
context lines directly above and below the line(s) you're patching changed. 
And that's a lot more likely to happen for configure than for configure.ac.

>  If someone thinks that by patching configure.ac, instead of
> configure, one
>  achieves tremendous savings in the frequency of needing to rebase one's
>  patches, they're in a desperate need for a reality check. 
No, they're just more familiar with how patch(1) works than you.

>  Furthermore, changes to configure.ac will more likely to result in a
> more
>  frequent manual rebases, where the corresponding changes in configure are
>  far more likely to result in much simpler rebasing that's limited only to
>  eliminating the fuzz. If one patches a line or two in configure.ac, then
>  if the upstream futzes around with a neighboring line, the patch is going
>  to get kicked out, and must be manually rebased. On the other hand, if a
>  corresponding patch was against configure, instead, (and by that I mean a
>  real patch, and not a patch to configure.ac followed by autoconf followed
>  by a diff of the new configure against the old, I hate to explicitly say
>  this, but some folks around here have a mental block on this subject, and
>  can't wrap their brains around the concept of patching configure
>  directly), the corresponding differences in configure would've ended up
>  being hundred of lines apart, resulting in nothing more than some fuzz,
>  that can be automatically updated. 
Huh? It's quite the opposite, as those "hundred (sic) of lines" which are 
inserted automatically (i.e. which do NOT come from configure.ac) are the 
ones most likely to change, it just takes upstream using a different 
autoconf (and they WILL do that, it's only in your dream world that all 
upstreams use a fixed autoconf binary and never change it, only a few 
projects like GCC do that, and even there, the version changes from time to 
time) and those lines will be completely different.

        Kevin Kofler

[Sam Varshavchik]

Kevin Kofler writes:

>  Sam Varshavchik wrote:
> > This may come as a shock to some, but configure does not often change
> > unless configure.ac changes too.
> > 
> > So, I'm not sure what does the frequency of changes to configure.ac has to
> > do with anything.
>  
>  Where your argument falls apart is that patch fuzz is a local concept! It's 
>  irrelevant that the file is not byte-identical. What matters is whether the 
>  context lines directly above and below the line(s) you're patching changed. 
>  And that's a lot more likely to happen for configure than for configure.ac. 
Wrong, as usual.

Since each autoconf macro typically expands out to hundreds lines of 
shellcode, with the autoconf macro's parameter embedded somewhere in the 
middle of all that stuff, were you to change a parameter to an autoconf 
macro in configure.ac, and upstream changes the parameter in the next line, 
your patch gets broken.

If the corresponding line in configure ended up getting patched, instead, 
your patch would still apply, with fuzz, since the results of upstream's 
changes would be hundreds of shellcode lines away.

The patch would still apply.

Even if the upstream deleted the preceding, or the succeeding, macro 
altogether, your patch to configure would still apply, since it patches one 
line in the middle of several dozens of lines of shellcode, and patch(1) is 
likely to search for the necessary fuzz.

> > If someone thinks that by patching configure.ac, instead of
> configure, one
> > achieves tremendous savings in the frequency of needing to rebase one's
> > patches, they're in a desperate need for a reality check.
>  
>  No, they're just more familiar with how patch(1) works than you. 
Yes, tell me again how conflicting patches to neighboring lines in 
configure.ac "works", while the equivalent two patches hundreds of lines 
apart in configure do not.


>  
> > Furthermore, changes to configure.ac will more likely to result in a more
> > frequent manual rebases, where the corresponding changes in configure are
> > far more likely to result in much simpler rebasing that's limited only to
> > eliminating the fuzz. If one patches a line or two in configure.ac, then
> > if the upstream futzes around with a neighboring line, the patch is going
> > to get kicked out, and must be manually rebased. On the other hand, if a
> > corresponding patch was against configure, instead, (and by that I mean a
> > real patch, and not a patch to configure.ac followed by autoconf followed
> > by a diff of the new configure against the old, I hate to explicitly say
> > this, but some folks around here have a mental block on this subject, and
> > can't wrap their brains around the concept of patching configure
> > directly), the corresponding differences in configure would've ended up
> > being hundred of lines apart, resulting in nothing more than some fuzz,
> > that can be automatically updated.
>  
>  Huh? It's quite the opposite, as those "hundred (sic) of lines" which are 
>  inserted automatically (i.e. which do NOT come from configure.ac) are the 
>  ones most likely to change, 
Perhaps you should actually familiarize yourself with how most autoconf 
macros work, before you attempt to engage in a deep philosophical 
discussions on their merits.

Stuff like AC_PATH_PROG produces several dozens lines of canned shellcode, 
with the arguments to AC_PATH_PROG appearing once, in the middle of them.

Changing the parameter to AC_PATH_PROG, for example, does not change 
hundreds of lines of shellcode. This may come as a shock to you, but it does 
not.

[Richard W.M. Jones]

On Tue, Jul 07, 2009 at 06:05:47PM -0400, Sam Varshavchik wrote:
>  If someone thinks that by patching configure.ac, instead of
> configure, 
>  one achieves tremendous savings in the frequency of needing to rebase 
>  one's patches, they're in a desperate need for a reality check. 
No, you are.  Please find out how patch works.

Rich.

-- 
Richard Jones, Emerging Technologies, Red Hat  http://et.redhat.com/~rjones
New in Fedora 11: Fedora Windows cross-compiler. Compile Windows
programs, test, and build Windows installers. Over 70 libraries supprt'd
http://fedoraproject.org/wiki/MinGW http://www.annexia.org/fedora_mingw

[Sam Varshavchik]

Richard W.M. Jones writes:

>  On Tue, Jul 07, 2009 at 06:05:47PM -0400, Sam Varshavchik wrote:
> > If someone thinks that by patching configure.ac, instead of configure, 
> > one achieves tremendous savings in the frequency of needing to rebase 
> > one's patches, they're in a desperate need for a reality check.
>  
>  No, you are.  Please find out how patch works. 
I do. Which is why you cannot hold your end of the debate.

[Toshio Kuratomi]

On 07/06/2009 02:53 PM, Sam Varshavchik wrote:
>  Adam Jackson writes:
>  
> > On Sun, 2009-07-05 at 18:50 -0400, Sam Varshavchik wrote:
> >> Richard W.M. Jones writes:
> >> > On Sun, Jul 05, 2009 at 10:45:46AM -0400, Sam Varshavchik wrote:
> >> >> What line number changes? You cut a patch against configure, and
> >> you're  >> done. That's it.
> >> > > And you get a big patch containing line numbers.  Here's a single
> >> line
> >> > change to configure.ac, and the corresponding patch that generates:
> >>   ======================
> >>
> >> Who said anything about patching configure.ac? The cited link is not
> >> a patch to the configure script, it's a result of a patch to
> >> configure.ac.
> >>
> >> I'll repeat again: patch configure, not configure.ac.
> >>
> >> I said "patch configure", and you reply, "well, it won't work
> because
> >> if you patch configure.ac, run autoconf, then generate the patch
> >> between the original configure, and the new configure, I get a big
> >> hairball". Duh.
> >
> > The fundamental problem with patching configure instead of configure.ac
> > (or Makefile instead of Makefile.am) is that it's changing the wrong
> > semantic level.
>  
>  As was discussed previously in this thread, when creating packages the
>  objective is not to patch the correct semantic level. 
Actually, in Fedora, it is.  We work closely with upstream.  If you
patch the correct semantic level, you can send the patch back to
upstream for incorporation.  If you only patch the configure script you
aren't helping upstream to improve their code.

Also, patching the configure script, while easier fixing the things the
majority of times that you've had to fix the build scripts is just
anecdotal.  My own anecdotes are skewed in the other direction -- I
can't recall one time that I've needed to patch the configure.ac script
where it would have been easier to patch the configure script directly
instead.  Introducing side-effects is something to watch out for but
patching configure instead of the true source is a short term fix, not a
long term solution.

-Toshio

[Sam Varshavchik]

Toshio Kuratomi writes:

>  On 07/06/2009 02:53 PM, Sam Varshavchik wrote:
> > As was discussed previously in this thread, when creating packages the
> > objective is not to patch the correct semantic level.
>  
>  Actually, in Fedora, it is.  We work closely with upstream.  If you
>  patch the correct semantic level, you can send the patch back to
>  upstream for incorporation.  If you only patch the configure script you
>  aren't helping upstream to improve their code. 
Right. And what exactly is difficult about still sending the ultimate patch 
upstream, but using a minimalist patch to configure, for the actual package, 
for the interim?

I guess it all comes down to what's easier: vetting the impact of your 
minimalist changes to configure, versus vetting a freshly minted configure 
script for any unintended side effects from regenerating it using a -- very 
likely -- different version of autoconf than the upstream used originally.

I know which one I'll choose. But, if some feel that vetting the entire 
configure script, whatever floats their boat. Although, I suspect, that 99% 
of the time everyone ignores it, hoping that the new configure script works 
as before, sans the patch. Basically cross your fingers, ignore it, and hope 
that nothing ends up broken.

[Braden McDaniel]

On 7/6/09 6:10 PM, Toshio Kuratomi wrote:

[snip]

>  Introducing side-effects is something to watch out for but
>  patching configure instead of the true source is a short term fix, not a
>  long term solution. 
*Any* patch should be viewed as a short-term fix.  A patch that needs to 
persist indefinitely suggests broken maintainership somewhere along the 
line--either upstream, of the Fedora package in question, or elsewhere 
in Fedora's infrastructure.

-- 
Braden McDaniel                      e-mail: <braden(a)endoframe.com&gt;
<http://endoframe.com>               Jabber: <braden(a)jabber.org&gt;

[Toshio Kuratomi]

On 07/06/2009 03:57 PM, Braden McDaniel wrote:
>  On 7/6/09 6:10 PM, Toshio Kuratomi wrote:
>  
>  [snip]
>  
> > Introducing side-effects is something to watch out for but
> > patching configure instead of the true source is a short term fix, not a
> > long term solution.
>  
>  *Any* patch should be viewed as a short-term fix.  A patch that needs to
>  persist indefinitely suggests broken maintainership somewhere along the
>  line--either upstream, of the Fedora package in question, or elsewhere
>  in Fedora's infrastructure.
>   
<nod> But one of those patches is upstreamable and the other is not.
The upstreamable patch is a step on the road to the long term fix.  The
non-upstreamable one is a dead-end.

-Toshio

[Braden McDaniel]

On Mon, 2009-07-06 at 16:36 -0700, Toshio Kuratomi wrote:
>  On 07/06/2009 03:57 PM, Braden McDaniel wrote:
>  > On 7/6/09 6:10 PM, Toshio Kuratomi wrote:
>  > 
>  > [snip]
>  > 
>  >> Introducing side-effects is something to watch out for but
>  >> patching configure instead of the true source is a short term fix, not a
>  >> long term solution.
>  > 
>  > *Any* patch should be viewed as a short-term fix.  A patch that needs to
>  > persist indefinitely suggests broken maintainership somewhere along the
>  > line--either upstream, of the Fedora package in question, or elsewhere
>  > in Fedora's infrastructure.
>  > 
>  <nod> But one of those patches is upstreamable and the other is not.
>  The upstreamable patch is a step on the road to the long term fix.  The
>  non-upstreamable one is a dead-end. 
Creating a patch to configure/Makefile.in in no way precludes a package
maintainer from sending an analogous patch to configure.ac/Makefile.am
upstream.  So, yes, it's a "dead end" that:

     1. reduces the size of the changeset between the upstream package
        and the one Fedora actually builds and
     2. improves the resiliency of the package build to changes to
        Fedora's autotools chain.

-- 
Braden McDaniel <braden(a)endoframe.com&gt;

[Toshio Kuratomi]

On 07/06/2009 08:09 PM, Braden McDaniel wrote:
>  On Mon, 2009-07-06 at 16:36 -0700, Toshio Kuratomi wrote:
> > On 07/06/2009 03:57 PM, Braden McDaniel wrote:
> >> On 7/6/09 6:10 PM, Toshio Kuratomi wrote:
> >>
> >> [snip]
> >>
> >>> Introducing side-effects is something to watch out for but
> >>> patching configure instead of the true source is a short term fix, not a
> >>> long term solution.
> >>
> >> *Any* patch should be viewed as a short-term fix.  A patch that needs to
> >> persist indefinitely suggests broken maintainership somewhere along the
> >> line--either upstream, of the Fedora package in question, or elsewhere
> >> in Fedora's infrastructure.
> >>
> > <nod> But one of those patches is upstreamable and the other is not.
> > The upstreamable patch is a step on the road to the long term fix.  The
> > non-upstreamable one is a dead-end.
>  
>  Creating a patch to configure/Makefile.in in no way precludes a package
>  maintainer from sending an analogous patch to configure.ac/Makefile.am
>  upstream.  So, yes, it's a "dead end" that:
>  
>       1. reduces the size of the changeset between the upstream package
>          and the one Fedora actually builds and
>       2. improves the resiliency of the package build to changes to
>          Fedora's autotools chain.
>   
Perhaps but it doesn't decrease the work that the maintainer has to do.
 The package maintainer needs to patch configure.ac so they have
something upstreamable.  Then they need to test that their changes make
the changes they think and don't break things via unintended
side-effects.  If they notice discrepancies, they have to find out what
is going on (whether it's a version difference in automake or a bug
introduced by their patch) and fix it.  Then they can submit that
upstream.  At that point, the reasons for creating a separate, even just
a few lines different code that directly changes configure is extraneous
work.

Do all maintainers do this kind of work when they make a patch for
configure.ac?  Perhaps not but they are doing their duties better if
they do.  Do all maintainers make upstreamable patches when they
directly fix a configure script?  I think the answer is the same.

-Toshio

[Braden McDaniel]

On Tue, 2009-07-07 at 01:17 -0700, Toshio Kuratomi wrote:
>  On 07/06/2009 08:09 PM, Braden McDaniel wrote:
>  > On Mon, 2009-07-06 at 16:36 -0700, Toshio Kuratomi wrote:
>  >> On 07/06/2009 03:57 PM, Braden McDaniel wrote:
>  >>> On 7/6/09 6:10 PM, Toshio Kuratomi wrote:
>  >>>
>  >>> [snip]
>  >>>
>  >>>> Introducing side-effects is something to watch out for but
>  >>>> patching configure instead of the true source is a short term fix, not
> a
>  >>>> long term solution.
>  >>>
>  >>> *Any* patch should be viewed as a short-term fix.  A patch that needs to
>  >>> persist indefinitely suggests broken maintainership somewhere along the
>  >>> line--either upstream, of the Fedora package in question, or elsewhere
>  >>> in Fedora's infrastructure.
>  >>>
>  >> <nod> But one of those patches is upstreamable and the other is not.
>  >> The upstreamable patch is a step on the road to the long term fix.  The
>  >> non-upstreamable one is a dead-end.
>  > 
>  > Creating a patch to configure/Makefile.in in no way precludes a package
>  > maintainer from sending an analogous patch to configure.ac/Makefile.am
>  > upstream.  So, yes, it's a "dead end" that:
>  > 
>  >      1. reduces the size of the changeset between the upstream package
>  >         and the one Fedora actually builds and
>  >      2. improves the resiliency of the package build to changes to
>  >         Fedora's autotools chain.
>  > 
>  Perhaps but it doesn't decrease the work that the maintainer has to do. 
It very well might if Fedora upgrades to a new autoconf, automake, or
libtool that is not 100% backward compatible with the previous version.

Obviously there is a class of Fedora package maintainers who are
comfortable incurring that risk and prefer simply to pick up the pieces
when such breakage occurs.

And then there are those of us who don't mind doing 5-15 minutes of work
for the insurance that updates to Fedora's autotools will have no impact
on our package's build.

-- 
Braden McDaniel <braden(a)endoframe.com&gt;

[Toshio Kuratomi]

On 07/07/2009 09:45 AM, Braden McDaniel wrote:
>  On Tue, 2009-07-07 at 01:17 -0700, Toshio Kuratomi wrote: 
> > Perhaps but it doesn't decrease the work that the maintainer
> has to do.
>  
>  It very well might if Fedora upgrades to a new autoconf, automake, or
>  libtool that is not 100% backward compatible with the previous version.
>   
As opposed to having to repatch the configure script everytime upstream
makes a new release? And as opposed to specifying BuildRequires:
automake10?  And as opposed to needing to know that the build breaks so
that you can update the patch that you sent to upstream?

>  Obviously there is a class of Fedora package maintainers who are
>  comfortable incurring that risk and prefer simply to pick up the pieces
>  when such breakage occurs.
>  
>  And then there are those of us who don't mind doing 5-15 minutes of work
>  for the insurance that updates to Fedora's autotools will have no impact
>  on our package's build.
>   
<nod> we're arguing over which of these outlooks is correct now because
we have different priorities for helping upstream improve their build
scripts vs making sure that the Fedora package builds.

-Toshio

[yersinia]

On Tue, Jul 7, 2009 at 6:45 PM, Braden McDaniel <braden(a)endoframe.com&gt;wrote:

>  On Tue, 2009-07-07 at 01:17 -0700, Toshio Kuratomi wrote:
>  > On 07/06/2009 08:09 PM, Braden McDaniel wrote:
>  > > On Mon, 2009-07-06 at 16:36 -0700, Toshio Kuratomi wrote:
>  > >> On 07/06/2009 03:57 PM, Braden McDaniel wrote:
>  > >>> On 7/6/09 6:10 PM, Toshio Kuratomi wrote:
>  > >>>
>  > >>> [snip]
>  > >>>
>  > >>>> Introducing side-effects is something to watch out for but
>  > >>>> patching configure instead of the true source is a short term fix,
>  not a
>  > >>>> long term solution.
>  > >>>
>  > >>> *Any* patch should be viewed as a short-term fix.  A patch that needs
>  to
>  > >>> persist indefinitely suggests broken maintainership somewhere along
>  the
>  > >>> line--either upstream, of the Fedora package in question, or
>  elsewhere
>  > >>> in Fedora's infrastructure.
>  > >>>
>  > >> <nod> But one of those patches is upstreamable and the other is not.
>  > >> The upstreamable patch is a step on the road to the long term fix.
>   The
>  > >> non-upstreamable one is a dead-end.
>  > >
>  > > Creating a patch to configure/Makefile.in in no way precludes a package
>  > > maintainer from sending an analogous patch to configure.ac/Makefile.am
>  > > upstream.  So, yes, it's a "dead end" that:
>  > >
>  > >      1. reduces the size of the changeset between the upstream package
>  > >         and the one Fedora actually builds and
>  > >      2. improves the resiliency of the package build to changes to
>  > >         Fedora's autotools chain.
>  > >
>  > Perhaps but it doesn't decrease the work that the maintainer has to do.
>
>  It very well might if Fedora upgrades to a new autoconf, automake, or
>  libtool that is not 100% backward compatible with the previous version.
>  
It is not hard at all to have ALL the version of autotool installed. Simply
pick one
(for example for automake) version for the default (for example 1.10 ) and
call
this package automake. If you want also automake 1.11 package this as
automake-1.11 rpm
and, if the developer want, it is its choice, use this instead of the
default via the autotool env var. I do this in RHEL
also for libtool 2.2 ecc.

[yersinia]

On Thu, Jul 9, 2009 at 9:47 AM, yersinia <yersinia.spiros(a)gmail.com&gt; wrote:

>  On Tue, Jul 7, 2009 at 6:45 PM, Braden McDaniel
> <braden(a)endoframe.com&gt;wrote:
>
> > On Tue, 2009-07-07 at 01:17 -0700, Toshio Kuratomi wrote:
> > > On 07/06/2009 08:09 PM, Braden McDaniel wrote:
> > > > On Mon, 2009-07-06 at 16:36 -0700, Toshio Kuratomi wrote:
> > > >> On 07/06/2009 03:57 PM, Braden McDaniel wrote:
> > > >>> On 7/6/09 6:10 PM, Toshio Kuratomi wrote:
> > > >>>
> > > >>> [snip]
> > > >>>
> > > >>>> Introducing side-effects is something to watch out for but
> > > >>>> patching configure instead of the true source is a short term
> fix,
> > not a
> > > >>>> long term solution.
> > > >>>
> > > >>> *Any* patch should be viewed as a short-term fix.  A patch that
> > needs to
> > > >>> persist indefinitely suggests broken maintainership somewhere
> along
> > the
> > > >>> line--either upstream, of the Fedora package in question, or
> > elsewhere
> > > >>> in Fedora's infrastructure.
> > > >>>
> > > >> <nod> But one of those patches is upstreamable and the other is
> not.
> > > >> The upstreamable patch is a step on the road to the long term fix.
> >  The
> > > >> non-upstreamable one is a dead-end.
> > > >
> > > > Creating a patch to configure/Makefile.in in no way precludes a
> > package
> > > > maintainer from sending an analogous patch to
> > configure.ac/Makefile.am
> > > > upstream.  So, yes, it's a "dead end" that:
> > > >
> > > >      1. reduces the size of the changeset between the upstream package
> > > >         and the one Fedora actually builds and
> > > >      2. improves the resiliency of the package build to changes to
> > > >         Fedora's autotools chain.
> > > >
> > > Perhaps but it doesn't decrease the work that the maintainer has to do.
> >
> > It very well might if Fedora upgrades to a new autoconf, automake, or
> > libtool that is not 100% backward compatible with the previous version.
> >
>
>  It is not hard at all to have ALL the version of autotool installed. Simply
>  pick one
>  (for example for automake) version for the default (for example 1.10 ) and
>  call
>  this package automake. If you want also automake 1.11 package this as
>  automake-1.11 rpm
>  and, if the developer want, it is its choice, use this instead of the
>  default via the autotool env var. I do this in RHEL
>  also for libtool 2.2 ecc.
>
>  And for gcc ecc. - so not only "compat" package but "upstream"
> package - 
without support as it is but if my user want, why not ?

Regards

[Richard W.M. Jones]

On Mon, Jul 06, 2009 at 11:09:51PM -0400, Braden McDaniel wrote:
>       2. improves the resiliency of the package build to changes to
>          Fedora's autotools chain. 
Many projects come with public source repositories, and those don't
include the binary configure/Makefile.in files.  You usually build
those locally with a script like 'autogen.sh'.  Projects that depend
on precise versions of the autotools are just going to break under
those conditions.

libguestfs is a case in point - the Debian maintainer builds it from
git using some unknown version of autoconf, and I build it on RHEL and
Fedora using other versions of autoconf.  If there was a bug reported
to me that configure.ac didn't work on (eg.) Debian's autoconf, I'd
consider it a bug in the project itself, and I wouldn't tell the
Debian maintainer to install a specific autoconf.

It's better in the long term to fix the configure.ac so that it can
work with any version of autotools.  If it requires some specific
version or a narrow range of versions, I'd consider it broken.

Rich.

-- 
Richard Jones, Emerging Technologies, Red Hat  http://et.redhat.com/~rjones
Read my programming blog: http://rwmj.wordpress.com
Fedora now supports 75 OCaml packages (the OPEN alternative to F#)
http://cocan.org/getting_started_with_ocaml_on_red_hat_and_fedora

[Sam Varshavchik]

Richard W.M. Jones writes:

>  On Mon, Jul 06, 2009 at 11:09:51PM -0400, Braden McDaniel wrote:
> >      2. improves the resiliency of the package build to changes to
> >         Fedora's autotools chain.
>  
>  Many projects come with public source repositories, and those don't
>  include the binary configure/Makefile.in files.  You usually build
>  those locally with a script like 'autogen.sh'.  Projects that depend
>  on precise versions of the autotools are just going to break under
>  those conditions. 
Bingo. Which is why this is a rather strange (not my first, or the second, 
or the nth choice, but I had to spend a few minutes here picking the correct 
adjective that expresses the general idea, but is still somewhat diplomatic) 
way to publish source. And, which is why this is somewhat of a rarity, and a 
novelty.

>  libguestfs is a case in point - the Debian maintainer builds it from
>  git using some unknown version of autoconf, and I build it on RHEL and 
This is a rare exception. For each project you can cite that releases their 
sources this way, I'll be happy to cite twenty others who don't.

Feel free to come up with your largest list. I'll just go through 
Sourceforge, and grab the first x*20 projects, in response.

Given that automake's "make dist" automatically rolls Makefile.in, and 
configure into the tarball (together with a bunch of other stuff), one has 
to go out of their way to leave them out of the tarball.

Bizarre.

[Mark McLoughlin]

On Tue, 2009-07-07 at 07:14 -0400, Sam Varshavchik wrote:
>  > libguestfs is a case in point - the Debian maintainer builds it
> from
>  > git using some unknown version of autoconf, and I build it on RHEL and
>  
>  This is a rare exception. 
No, it's a rare exception for project to keep autotools generated files
in version control.

Yet people still build lots of projects from version control on a
variety of different distros using different versions of autotools.

I'm also making the point that maintainers build tarballs without paying
much attention to the versions of autotools they're using.

>  For each project you can cite that releases their 
>  sources this way, I'll be happy to cite twenty others who don't.
>
>  Feel free to come up with your largest list. I'll just go through 
>  Sourceforge, and grab the first x*20 projects, in response. 
Please tone down the hyperbole.

I'd be very interested to hear of a specific case where a recent
autotools update has broken old tarball builds. If that was in fact
common, and we had some examples, I'd agree with you.

>  Given that automake's "make dist" automatically rolls
> Makefile.in, and 
>  configure into the tarball (together with a bunch of other stuff), one has 
>  to go out of their way to leave them out of the tarball. 
Yes, that autotools generated files are distributed in tarballs is a
clear autotools design decision. But why? Is it:

  a) because the autotools maintainers feel it is unreliable to have
     people building from the tarball to re-run autotools or

  b) because the autotools maintainers feel it is inconvenient to 
     require people build from tarballs to have autotools installed

I suspect (b) is primary reason, especially in recent times.

Cheers,
Mark.

[Sam Varshavchik]

Mark McLoughlin writes:

>  On Tue, 2009-07-07 at 07:14 -0400, Sam Varshavchik wrote:
> > > libguestfs is a case in point - the Debian maintainer builds it from
> > > git using some unknown version of autoconf, and I build it on RHEL and
> > 
> > This is a rare exception.
>  
>  No, it's a rare exception for project to keep autotools generated files
>  in version control.
>  
>  Yet people still build lots of projects from version control on a
>  variety of different distros using different versions of autotools. 
I'm sure that there are some folks who do that. But, the overwhelming 
majority of folks want to compile some stable result, rather than the 
greatest and the latest, which may not even compile at all.

Presumably, those who want to build straight out of the upstream repo, have 
sufficient skills and experience to deal with autotools.

>  I'm also making the point that maintainers build tarballs without
> paying
>  much attention to the versions of autotools they're using. 
I don't get that impression. When I end up upgrading, as a result of the 
entire distro upgrade, or otherwise, to a new autotools, I make sure that I 
go through my existing configure scripts with a fine-toothed comb. Every 
time this happens I always end up tweaking something, making sure to replace 
obsoleted macros with their replacements, etc… But I can only speak for 
myself.

[Kevin Kofler]

Sam Varshavchik wrote:
>  I don't get that impression. When I end up upgrading, as a result
> of the
>  entire distro upgrade, or otherwise, to a new autotools, I make sure that
>  I go through my existing configure scripts with a fine-toothed comb. Every
>  time this happens I always end up tweaking something, making sure to
>  replace obsoleted macros with their replacements, etc… But I can only
>  speak for myself. 
Indeed. I can also only speak for myself, but I just run "autoreconf -i -f" 
and ignore all the warnings on the autotools-using projects I inherited.

        Kevin Kofler

[Sam Varshavchik]

Kevin Kofler writes:

>  Sam Varshavchik wrote:
> > I don't get that impression. When I end up upgrading, as a result of the
> > entire distro upgrade, or otherwise, to a new autotools, I make sure that
> > I go through my existing configure scripts with a fine-toothed comb. Every
> > time this happens I always end up tweaking something, making sure to
> > replace obsoleted macros with their replacements, etc… But I can only
> > speak for myself.
>  
>  Indeed. I can also only speak for myself, but I just run "autoreconf -i -f" 
>  and ignore all the warnings on the autotools-using projects I inherited. 
That's definitely a useful tip -- always ignore warnings. They are 
always completely meaningless.

You don't need to worry about them.

[Kevin Kofler]

Sam Varshavchik wrote:
>  That's definitely a useful tip -- always ignore warnings. They
> are
>  always completely meaningless.
>  
>  You don't need to worry about them. 
Quit the sarcasm. The thing is: this is how things work in the real world, 
whether it's a good idea or not. The autotools spit out tons of 
incomprehensible warnings (e.g. "`foo' should be called before `bar'",
but 
both "foo" and "bar" are called by some canned snippets, not by your
own 
code) for things which previous versions accepted just fine, it's no 
surprise that maintainers aren't motivated to fix them.

        Kevin Kofler

[Jesse Keating]

On Jul 7, 2009, at 4:14, Sam Varshavchik <mrsam(a)courier-mta.com&gt; wrote:

>  Richard W.M. Jones writes:
>
> > On Mon, Jul 06, 2009 at 11:09:51PM -0400, Braden McDaniel wrote:
> >>     2. improves the resiliency of the package build to changes to
> >>        Fedora's autotools chain.
> > Many projects come with public source repositories, and those don't
> > include the binary configure/Makefile.in files.  You usually build
> > those locally with a script like 'autogen.sh'.  Projects that depend
> > on precise versions of the autotools are just going to break under
> > those conditions.
>
>  Bingo. Which is why this is a rather strange (not my first, or the  
>  second, or the nth choice, but I had to spend a few minutes here  
>  picking the correct adjective that expresses the general idea, but  
>  is still somewhat diplomatic) way to publish source. And, which is  
>  why this is somewhat of a rarity, and a novelty.
>
> > libguestfs is a case in point - the Debian maintainer builds it from
> > git using some unknown version of autoconf, and I build it on RHEL  
> > and
>
>  This is a rare exception. For each project you can cite that  
>  releases their sources this way, I'll be happy to cite twenty others  
>  who don't.
>
>  Feel free to come up with your largest list. I'll just go through  
>  Sourceforge, and grab the first x*20 projects, in response.
>
>  Given that automake's "make dist" automatically rolls Makefile.in,  
>  and configure into the tarball (together with a bunch of other  
>  stuff), one has to go out of their way to leave them out of the  
>  tarball.
>
>  Bizarre. 

These days distributing via tarball is bizarre.  Distributed source  
control is changing the way that projects work and release. Sure there  
are plenty of projects out here that don't work this way but more and  
more are headed in this direction.

--
Jes

[Seth Vidal]

On Tue, 7 Jul 2009, Jesse Keating wrote:

>  
>  These days distributing via tarball is bizarre.  Distributed source control is changing
> the way that projects work and
>  release. Sure there are plenty of projects out here that don't work this way but more
> and more are headed in this
>  direction. 
>  
I disagree. I think the discrete tarball snapshot of a release will 
continue for quite some time and I've not seen anyone moving away from 
that in their public software releases.

-sv

[Colin Walters]

On Tue, Jul 7, 2009 at 10:23 AM, Seth Vidal<skvidal(a)fedoraproject.org> wrote:
>  I disagree. I think the discrete tarball snapshot of a release will continue
>  for quite some time and I've not seen anyone moving away from that in their
>  public software releases. 
It's not so black and white; for example, it often makes sense to pull
git snapshots of a project during rawhide.  If our tooling made it
easy to run a command and pull from git I'd definitely use it.  Not
for all projects, and not all the time, but it'd be useful.

[Seth Vidal]

On Tue, 7 Jul 2009, Colin Walters wrote:

>  On Tue, Jul 7, 2009 at 10:23 AM, Seth
> Vidal<skvidal(a)fedoraproject.org> wrote:
> >
> > I disagree. I think the discrete tarball snapshot of a release will continue
> > for quite some time and I've not seen anyone moving away from that in their
> > public software releases.
>
>  It's not so black and white; for example, it often makes sense to pull
>  git snapshots of a project during rawhide.  If our tooling made it
>  easy to run a command and pull from git I'd definitely use it.  Not
>  for all projects, and not all the time, but it'd be useful.
>  
umm - I don't think I said tarballs were the only way to do things. I just 
said that I didn't think they were as disused for releases as jesse 
suggested.


-sv

[Sam Varshavchik]

Jesse Keating writes:

>  These days distributing via tarball is bizarre.  Distributed source 
>  control is changing the way that projects work and release. Sure there are 
>  plenty of projects out here that don't work this way but more and more are 
>  headed in this direction.  
Yes. If I get the desire to build a new library, or something, I always want 
to just find the URL to the upstream's source control repo, then check out 
whatever I find in there, and run with it. I'm confident that I will never 
have any problems building it, and it won't have any issues.

This is the best and the most reliable mechanism of downloading stable code.














<sarcasm>

[Jesse Keating]

On Tue, 2009-07-07 at 18:16 -0400, Sam Varshavchik wrote:
>  Jesse Keating writes:
>  
>  > These days distributing via tarball is bizarre.  Distributed source 
>  > control is changing the way that projects work and release. Sure there are 
>  > plenty of projects out here that don't work this way but more and more are 
>  > headed in this direction. 
>  
>  Yes. If I get the desire to build a new library, or something, I always want 
>  to just find the URL to the upstream's source control repo, then check out 
>  whatever I find in there, and run with it. I'm confident that I will never 
>  have any problems building it, and it won't have any issues.
>  
>  This is the best and the most reliable mechanism of downloading stable code.
>   
Ahem.  Tagged releases in source code.  Just skipping the step of
extracting that into a tarball to pass around.

-- 
Jesse Keating
Fedora -- Freedom² is a feature!
identi.ca: http://identi.ca/jkeating

[Kevin Kofler]

Sam Varshavchik wrote:
>  As I understand, rpm's default settings now reject fuzz in patch
> files, so
>  you'll just have to do it, now. 
"%define _default_patch_fuzz 2" (or even 3) can do miracles. :-p

>  And since the likelyhood of configure changing in a new release is
> no
>  different than any other source file getting changed 
This is a wrong assertion to start from: configure tends to change a lot
since configure.ac gets changed at least once per version (to bump the
version) and upstream will regenerate configure with the then-current
configure.ac on their then-current distro. Sometimes, it's not even the
same person regenerating configure each time, so it can get regenerated on
completely different distros. Plus, as even small patches to configure.ac
can change a lot in configure (e.g. line numbers all over the place) and as
upstream WILL regenerate configure, not patch it directly, fuzz is a lot
more likely in configure than configure.ac.

>  on average, believing that some work can be saved just by choosing
> to
>  patch a different file, then the one that really needs to be patched, is
>  somewhat naive. 
It's believing it can't which is naive. All evidence points towards the fact
that patching configure.ac is indeed less work for changes.

>  Because autoconf and automake are going to change a lot more than
> just
>  what the patch was intended to patch. It's fairly likely that the upstream
>  is using a different version of autoconf and automake, so this ends up
>  producing a brand new configure and makefile. If I were to do that, then
>  I would find it necessary to spend additional time testing the new
>  configure script, running it an eyeballing all of its voluminous output,
>  watching for something that falls out, as a result of the new configure
>  script. 
And you think upstream does that? On the autotools-using stuff I'm now
upstream for (no, it wasn't my decision to use autotools, and yes, moving
to CMake is a high-priority todo item), I just run "autoreconf -i -f" with
the latest autotools updates of the version of Fedora I'm currently running
and ignore all the warnings.

        Kevin Kofler

[Ralf Corsepius]

Richard W.M. Jones wrote:
>  On Wed, Jul 01, 2009 at 12:40:44PM +0200, Ralf Corsepius wrote:
> > No, not if they bundle the generated auto* files with their tarballs, as  
> > they are supposed to do.
>  
>  They're not "supposed to do" that.  
Wrong.

>  Don't make stuff up. 
Read the manuals, then come back.

[Mark McLoughlin]

On Tue, 2009-06-30 at 14:05 -0400, Owen Taylor wrote:

>  But is this the type of upgrade that makes sense in general? It seems
> to
>  me that we should be very conservative in upgrading build tools,
>  especially in "maintenance mode" distributions like F9 and F10. 
Agreed, and if there was a truly compelling reason for the upgrade, one
would expect it to be mentioned in the update description.

See also https://fedoraproject.org/wiki/Package_update_guidelines

Cheers,
Mark.

[Brian Pepple]

On Tue, 2009-06-30 at 14:05 -0400, Owen Taylor wrote:
>  But is this the type of upgrade that makes sense in general? It seems
> to
>  me that we should be very conservative in upgrading build tools,
>  especially in "maintenance mode" distributions like F9 and F10. 
Totally agree with you.  We shouldn't be pushing updates to build tools
in our stable releases unless there is a really strong reason for it.

Later,
/B
-- 
Brian Pepple <bpepple(a)fedoraproject.org&gt;

identi.ca: http://identi.ca/bpepple
gpg --keyserver pgp.mit.edu --recv-keys 810CC15E
BD5E 6F9E 8688 E668 8F5B  CBDE 326A E936 810C C15E

[Orcan Ogetbil]

By the way,

On Tue, Jun 30, 2009 at 2:05 PM, Owen Taylor wrote:
>  I was rather surprised to see:
>
>   https://admin.fedoraproject.org/updates/F9/FEDORA-2009-6661
>   https://admin.fedoraproject.org/updates/F10/FEDORA-2009-6076
>   https://admin.fedoraproject.org/updates/F11/FEDORA-2009-6370
>
>  Where the automake was upgraded to 1.11 for F9, F10, and F11.
>  
why is there no update information about these builds? There is a big
"Notes" box on bodhi which is there for a reason.

Orcan

[Stepan Kasal]

Hi,

On Tue, Jun 30, 2009 at 08:24:53PM -0400, Orcan Ogetbil wrote:
>  why is there no update information about these builds? There is a
> big
>  "Notes" box on bodhi which is there for a reason. 
I apologize for that.  I got bored writing three-word nonsenses so I
tried the null string.  I will do better now when I know that it
might be read by someone in certain cases.

Stepan

[Michael Cronenworth]

Stepan Kasal on 07/01/2009 05:05 AM wrote:
>  
>  I apologize for that.  I got bored writing three-word nonsenses so I
>  tried the null string.  I will do better now when I know that it
>  might be read by someone in certain cases.
>  
Fedora 11 brings a PackageKit that actually promotes and accentuates the
notes for each package. Not having any actually looks bad this time
around. It was "normal" to not have them in F9 or F10 but not anymore.

[Ondřej Vašík]

Owen Taylor wrote:
>  I was rather surprised to see:
>  
>   https://admin.fedoraproject.org/updates/F9/FEDORA-2009-6661
>   https://admin.fedoraproject.org/updates/F10/FEDORA-2009-6076
>   https://admin.fedoraproject.org/updates/F11/FEDORA-2009-6370
>  
>  Where the automake was upgraded to 1.11 for F9, F10, and F11. 
Upgrade on F-11 (and F-10) was requested because there are some projects
(like gnulib/coreutils) which really need automake 1.11 for build in
latest stable versions.
Anyway I'm a bit surprised by F-9 update - as we argumented by more
possible projects with need for automake-1.11 during F-10/F-11
lifecycle. Packages automake/autoconf are special - as you need them not
only for packaging, but for building not packaged projects from
git/whatever upstream repos. 
Yes, you could always compile latest autotools as well and use them, but
automake 1.10 is ~2 years old software. AFAIK there were no
incompatibility changes in automake, so the only problem could be if the
program relies on exact automake version - which is imho wrong.

>  In general automake hasn't had a very good track record of
> compatibility
>  between 1.x and 1.y, though this has been getting better recently. 
Previous automake 1.10 is more than 2 years old, automake 1.10b and beta
release were used at least for building testing gnulib and coreutils
project and there were no troubles. Ralf Wildenhues is very conservative
with changes - which is good for projects like automake/autoconf.
Therefore I don't expect incompatibility issues.
Anyone experienced troubles with latest automake in rawhide (other than
exact automake version check)? If so - it's still in testing, so it
still could (and for F-9 imho anyway should) be unpushed...

Greetings,
          Ondřej Vašík

[Pádraig Brady]

Ondřej Vašík wrote:
>  Owen Taylor wrote:
> > I was rather surprised to see:
> >
> >  https://admin.fedoraproject.org/updates/F9/FEDORA-2009-6661
> >  https://admin.fedoraproject.org/updates/F10/FEDORA-2009-6076
> >  https://admin.fedoraproject.org/updates/F11/FEDORA-2009-6370
> >
> > Where the automake was upgraded to 1.11 for F9, F10, and F11.
>  
>  Upgrade on F-11 (and F-10) was requested because there are some projects
>  (like gnulib/coreutils) which really need automake 1.11 for build in
>  latest stable versions. 
To clarify, coreutils needs automake only for development,
but it is nice not to have to build it as described here:
http://git.savannah.gnu.org/gitweb/?p=coreutils.git;a=blob;f=README-prere...

I am surprised it was pused to F9 & F10

cheers,
Pádraig.

[Mark McLoughlin]

On Wed, 2009-07-01 at 09:02 +0200, Ondřej Vašík wrote:
>  Owen Taylor wrote:
>  > I was rather surprised to see:
>  > 
>  >  https://admin.fedoraproject.org/updates/F9/FEDORA-2009-6661
>  >  https://admin.fedoraproject.org/updates/F10/FEDORA-2009-6076
>  >  https://admin.fedoraproject.org/updates/F11/FEDORA-2009-6370
>  > 
>  > Where the automake was upgraded to 1.11 for F9, F10, and F11.
>  
>  Upgrade on F-11 (and F-10) was requested because there are some projects
>  (like gnulib/coreutils) which really need automake 1.11 for build in
>  latest stable versions. 
Is there a bug report with details of this gnulib/coreutils request?

Thanks,
Mark.

[Ondřej Vašík]

Mark McLoughlin wrote:
>  On Wed, 2009-07-01 at 09:02 +0200, Ondřej Vašík wrote:
>  > Owen Taylor wrote:
>  > > I was rather surprised to see:
>  > > 
>  > >  https://admin.fedoraproject.org/updates/F9/FEDORA-2009-6661
>  > >  https://admin.fedoraproject.org/updates/F10/FEDORA-2009-6076
>  > >  https://admin.fedoraproject.org/updates/F11/FEDORA-2009-6370
>  > > 
>  > > Where the automake was upgraded to 1.11 for F9, F10, and F11.
>  > 
>  > Upgrade on F-11 (and F-10) was requested because there are some projects
>  > (like gnulib/coreutils) which really need automake 1.11 for build in
>  > latest stable versions.
>  
>  Is there a bug report with details of this gnulib/coreutils request? 
Not really, it was just direct irl/irc/mail communication with
automake/autoconf fedora maintainers&comaintainers. First request was
only about 1.10b in rawhide (after f-12 split) - as I needed at least
1.10b to build coreutils-7.4 there (otherwise only with an ugly hack).

Greetings,
         Ondřej Vašík

[Ralf Corsepius]

Ondřej Vašík wrote:
>  Mark McLoughlin wrote:
> > On Wed, 2009-07-01 at 09:02 +0200, Ondřej Vašík wrote:
> >> Owen Taylor wrote:
> >>> I was rather surprised to see:
> >>>
> >>>  https://admin.fedoraproject.org/updates/F9/FEDORA-2009-6661
> >>>  https://admin.fedoraproject.org/updates/F10/FEDORA-2009-6076
> >>>  https://admin.fedoraproject.org/updates/F11/FEDORA-2009-6370
> >>>
> >>> Where the automake was upgraded to 1.11 for F9, F10, and F11.
> >> Upgrade on F-11 (and F-10) was requested because there are some projects
> >> (like gnulib/coreutils) which really need automake 1.11 for build in
> >> latest stable versions.
> > Is there a bug report with details of this gnulib/coreutils request?
>  
>  Not really, it was just direct irl/irc/mail communication with
>  automake/autoconf fedora maintainers&comaintainers. First request was
>  only about 1.10b in rawhide (after f-12 split) - as I needed at least
>  1.10b to build coreutils-7.4 there (otherwise only with an ugly hack). 
And? This should not be a problem to you.

Simply install the original GNU versions of these autotools in parallel 
(using a different --prefix) to the vendor supplied versions and use 
them for your packages (here core-utils).

Ralf

[Ondřej Vašík]

Ralf Corsepius wrote:
>  Ondřej Vašík wrote:
>  > Mark McLoughlin wrote:
>  >> On Wed, 2009-07-01 at 09:02 +0200, Ondřej Vašík wrote:
>  >>> Owen Taylor wrote:
>  >>>> I was rather surprised to see:
>  >>>>
>  >>>>  https://admin.fedoraproject.org/updates/F9/FEDORA-2009-6661
>  >>>>  https://admin.fedoraproject.org/updates/F10/FEDORA-2009-6076
>  >>>>  https://admin.fedoraproject.org/updates/F11/FEDORA-2009-6370
>  >>>>
>  >>>> Where the automake was upgraded to 1.11 for F9, F10, and F11.
>  >>> Upgrade on F-11 (and F-10) was requested because there are some projects
>  >>> (like gnulib/coreutils) which really need automake 1.11 for build in
>  >>> latest stable versions.
>  >> Is there a bug report with details of this gnulib/coreutils request?
>  > 
>  > Not really, it was just direct irl/irc/mail communication with
>  > automake/autoconf fedora maintainers&comaintainers. First request was
>  > only about 1.10b in rawhide (after f-12 split) - as I needed at least
>  > 1.10b to build coreutils-7.4 there (otherwise only with an ugly hack).
>  
>  And? This should not be a problem to you. 
This is not problem for me on my machine - I had 1.10a/1.10b/1.11
already on my machine for quite a long time those days - but to build
them in koji it required hacky solution (e.g. temporarily
reverting/disabling things which do need automake 1.10a+ or some even
more ugly things). I was talking about first request - just to add 1.10b
to rawhide, initiative for updating F-10/F-11 to F-11 came later from
Jim Meyering side. Anyway - I like F-11 update of autotools, F-10 update
is still ok for me, but imho F-9 update should not make it to stable.

Greetings,
         Ondřej

[Mark McLoughlin]

On Wed, 2009-07-01 at 12:50 +0200, Ondřej Vašík wrote:
>  Mark McLoughlin wrote:
>  > On Wed, 2009-07-01 at 09:02 +0200, Ondřej Vašík wrote:
>  > > Owen Taylor wrote:
>  > > > I was rather surprised to see:
>  > > > 
>  > > >  https://admin.fedoraproject.org/updates/F9/FEDORA-2009-6661
>  > > >  https://admin.fedoraproject.org/updates/F10/FEDORA-2009-6076
>  > > >  https://admin.fedoraproject.org/updates/F11/FEDORA-2009-6370
>  > > > 
>  > > > Where the automake was upgraded to 1.11 for F9, F10, and F11.
>  > > 
>  > > Upgrade on F-11 (and F-10) was requested because there are some projects
>  > > (like gnulib/coreutils) which really need automake 1.11 for build in
>  > > latest stable versions.
>  > 
>  > Is there a bug report with details of this gnulib/coreutils request?
>  
>  Not really, it was just direct irl/irc/mail communication with
>  automake/autoconf fedora maintainers&comaintainers. First request was
>  only about 1.10b in rawhide (after f-12 split) - as I needed at least
>  1.10b to build coreutils-7.4 there (otherwise only with an ugly hack). 
Okay, but what exactly are we talking about here? What does gnulib or
coreutils need that 1.10 doesn't have?

A rebase of an important package in three stable releases, which is
expected to break rebuilds of some packages, should surely have more
justification than an empty update description, no associated bugzilla
and claims that Jim Meyering needs some unspecified new features.

Cheers,
Mark.

[Ralf Corsepius]

Mark McLoughlin wrote:
>  On Wed, 2009-07-01 at 12:50 +0200, Ondřej Vašík wrote:
> > Mark McLoughlin wrote:
> >> On Wed, 2009-07-01 at 09:02 +0200, Ondřej Vašík wrote:
> >>> Owen Taylor wrote:
> >>>> I was rather surprised to see:
> >>>>
> >>>>  https://admin.fedoraproject.org/updates/F9/FEDORA-2009-6661
> >>>>  https://admin.fedoraproject.org/updates/F10/FEDORA-2009-6076
> >>>>  https://admin.fedoraproject.org/updates/F11/FEDORA-2009-6370
> >>>>
> >>>> Where the automake was upgraded to 1.11 for F9, F10, and F11.
> >>> Upgrade on F-11 (and F-10) was requested because there are some projects
> >>> (like gnulib/coreutils) which really need automake 1.11 for build in
> >>> latest stable versions.
> >> Is there a bug report with details of this gnulib/coreutils request?
> > Not really, it was just direct irl/irc/mail communication with
> > automake/autoconf fedora maintainers&comaintainers. First request was
> > only about 1.10b in rawhide (after f-12 split) - as I needed at least
> > 1.10b to build coreutils-7.4 there (otherwise only with an ugly hack).
>  
>  Okay, but what exactly are we talking about here? What does gnulib or
>  coreutils need that 1.10 doesn't have? 
coreutils' automake configuration
uses features which are not available 
before automake-1.11 (rsp. post automake-1.10 devel snapshots)

>  A rebase of an important package in three stable releases, which is
>  expected to break rebuilds of some packages, 
No, it doesn't. It's simply
that developing coreutils now requires a 
specific version of the autotools.

This is not much different from demands of many other projects, which 
also require specific versions.

The only difference is coreutils requiring a very recent automake, due 
to it exploiting a new feature and not a specific, older version due to 
suffering from compatibility issues, like most other such cases do.


>  should surely have more
>  justification than an empty update description, no associated bugzilla
>  and claims that Jim Meyering needs some unspecified new features. 
Well, these new
features have been introduced to automake, primarily due 
to Jim Meyering's initiative/pressure.

I guess hardly anybody but him is currently using these features.


Ralf

[Jim Meyering]

Mark McLoughlin wrote:
>  On Wed, 2009-07-01 at 12:50 +0200, Ondřej Vašík wrote:
> > Mark McLoughlin wrote:
> > > On Wed, 2009-07-01 at 09:02 +0200, Ondřej Vašík wrote:
> > > > Owen Taylor wrote:
> > > > > I was rather surprised to see:
> > > > >
> > > > >  https://admin.fedoraproject.org/updates/F9/FEDORA-2009-6661
> > > > >  https://admin.fedoraproject.org/updates/F10/FEDORA-2009-6076
> > > > >  https://admin.fedoraproject.org/updates/F11/FEDORA-2009-6370
> > > > >
> > > > > Where the automake was upgraded to 1.11 for F9, F10, and F11.
> > > >
> > > > Upgrade on F-11 (and F-10) was requested because there are some projects
> > > > (like gnulib/coreutils) which really need automake 1.11 for build in
> > > > latest stable versions.
> > >
> > > Is there a bug report with details of this gnulib/coreutils request?
> >
> > Not really, it was just direct irl/irc/mail communication with
> > automake/autoconf fedora maintainers&comaintainers. First request was
> > only about 1.10b in rawhide (after f-12 split) - as I needed at least
> > 1.10b to build coreutils-7.4 there (otherwise only with an ugly hack).
>
>  Okay, but what exactly are we talking about here? What does gnulib or
>  coreutils need that 1.10 doesn't have? 
Hi Mark,

I think it's great that automake-1.11 made it into F11 and F10.
Even for F9, it's seems worthwhile.

The features in automake-1.11 that I've found worthwhile
(in addition to 3 years worth of improved robustness,
portability and performance, fewer bugs, etc.)
are enabled by these two lines from coreutils' configure.ac:

  AM_INIT_AUTOMAKE([1.11 dist-xz color-tests parallel-tests])
  AM_SILENT_RULES([yes])

The silent-rules option makes it so build output by default
no longer includes many compile/link/etc command invocations.
Instead, you get very brief lines like these:

  ...
  CC     tee.o
  CC     test.o
  CC     timeout.o
  CC     true.o
  CC     truncate.o
  CC     tty.o
  CC     whoami.o
  CC     yes.o
  CC     base64.o
  CC     setuidgid.o
  CC     getlimits.o
  CC     libstdbuf_so-libstdbuf.o
  CC     su.o
  AR     libver.a
  CCLD   chroot
  CCLD   uname
  CCLD   hostid
  CCLD   nice
  CCLD   who
  CCLD   users
  CCLD   pinky
  CCLD   uptime
  CCLD   stty
  ...

Run "make V=1" if you want the verbose output you're used to.  Note that
I prefer the behavior shown above.  Remove the ([yes]) if you (as package
maintainer) want to provide the option, but with V=1 as default.

That may look trivial, but the reduced clutter makes surprising output
stand out more than it used to.  This has helped me find at least two
minor problems in coreutils and gnulib already.

parallel-tests is well worth it any time I run "make check"
on a multi-core system.  On a quad-core system, with many, fine-grained
tests, it cuts test run time by 70% or more.  Of course, it helps to
start the longest-running tests early enough so that they have a better
chance to complete before other cores go idle.

> From automake's NEWS: this was an important improvement for
projects that install many files into the same directory,
especially on systems with SELinux enabled (in some extreme
cases, this change resulted in a 30-X(!) speed-up):

  - The targets `install' and `uninstall' are more efficient now, in that
    for example multiple files from one Automake variable such as
    `bin_SCRIPTS' are copied in one `install' (or `libtool --mode=install')
    invocation if they do not have to be renamed.

color-tests is no big deal.
It gives you e.g., green "PASS" and red "FAIL" highlighting
in the output of "make check".

I like dist-xz because the compressed tarballs are so much
smaller than bzip2-compressed ones.  xz is the successor
to LZMA (http://tukaani.org/xz/).  I install it from source.

FYI, I do not make changes like these lightly.  I follow automake
development very closely and even contribute once in a while.
The standard of quality there is very high.  When I discovered that
some tools could compress tarballs 10-35% better than bzip2, I poked
and prodded the contenders.  Lzma-utils stood out for its quality of
implementation and it adherence to the de-facto gzip/bzip2 standards.
Due to a significant format change, the name has changed too, and now
the tool is called xz.

There are many more improvements and bug fixes in 1.11
that were not in any previous version.  See the NEWS file for
the complete list:

    http://git.sv.gnu.org/cgit/automake.git/plain/NEWS

>  A rebase of an important package in three stable releases, which is
>  expected to break rebuilds of some packages, should surely have more
>  justification than an empty update description, no associated bugzilla
>  and claims that Jim Meyering needs some unspecified new features. 
I've been lamenting the 3-year-old version of automake in Fedora for
years, and worse, having to jump through hoops for projects like qpid,
parted, corosync, openais, etc. because I refuse to waste time working
around bugs/misfeatures that were fixed in upstream automake years before.
I've been helping people build from source and install their own
versions of these tools for use on Fedora-based systems.  This script
was essential before the upgrade to automake-1.11:

    http://et.redhat.com/~meyering/autotools-install

Distributing such an old version of automake (with no newer alternative)
was doing a disservice to Fedora developers everywhere.  That old version
of automake would inject its older, inferior infrastructure into every
Makefile.in and aclocal.m4, and from there into every package's configure
script.  People using stock Fedora were thus stuck with some 3-year-old bugs,
and could not take advantage of the numerous improvements made during
that period.  Fedora is better than that.

Thanks again to the Fedora automake maintainers.

Jim

[Kevin Kofler]

Jim Meyering wrote:
>  The silent-rules option makes it so build output by default
>  no longer includes many compile/link/etc command invocations.
>  Instead, you get very brief lines like these:
>  
>    ...
>    CC     tee.o 
[etc.]

FWIW, that's what CMake does by default. (CMake's implementation is better
though: it also gives the progress percentage and uses color-coding so you
can immediately recognize the different kinds of operations and so the
operations are visually separated from the stderr output.) It is frowned
upon in Fedora because it doesn't allow you to easily check that
RPM_OPT_FLAGS are being used and so our %cmake and %cmake_kde4 macros
include:
-DCMAKE_VERBOSE_MAKEFILE=ON
which overrides this behavior and shows the full command lines by default.
(It's also possible for CMake-using packages to do this at make time with
make VERBOSE=ON which is what was the previous recommendation.)

>  Run "make V=1" if you want the verbose output you're
> used to. 
This will be REQUIRED in Fedora for packages using this feature (unless this
can somehow be set at configure time, in which case our %configure macro
could do it similarly to how our %cmake and %cmake_kde4 macros do).

By the way, it's pretty sad that automake had to reinvent the wheel instead
of using the established VERBOSE=ON syntax.

>  That may look trivial, but the reduced clutter makes surprising
> output
>  stand out more than it used to.  
I agree (and I like how this has been the default in CMake since forever),
but we were told that this is no good for Fedora because we should have the
full command lines in build.log so we can check for the optflags.

        Kevin Kofler

[Mark McLoughlin]

On Thu, 2009-07-02 at 00:38 +0200, Kevin Kofler wrote:

>  > Run "make V=1" if you want the verbose output
> you're used to.
>  
>  This will be REQUIRED in Fedora for packages using this feature  
Yes, it's a good idea for packages to do this, as it makes the koji logs
much more useful. We do this for qemu builds, for example.

(The syntax was copied from the kernel AFAIK)

Cheers,
Mark.

[Mark McLoughlin]

Hi Jim,

On Wed, 2009-07-01 at 23:49 +0200, Jim Meyering wrote:
>  Mark McLoughlin wrote:
>  > On Wed, 2009-07-01 at 12:50 +0200, Ondřej Vašík wrote:
>  >> Mark McLoughlin wrote:
>  >> > On Wed, 2009-07-01 at 09:02 +0200, Ondřej Vašík wrote:
>  >> > > Owen Taylor wrote:
>  >> > > > I was rather surprised to see:
>  >> > > >
>  >> > > >  https://admin.fedoraproject.org/updates/F9/FEDORA-2009-6661
>  >> > > >  https://admin.fedoraproject.org/updates/F10/FEDORA-2009-6076
>  >> > > >  https://admin.fedoraproject.org/updates/F11/FEDORA-2009-6370
>  >> > > >
>  >> > > > Where the automake was upgraded to 1.11 for F9, F10, and F11.
>  >> > >
>  >> > > Upgrade on F-11 (and F-10) was requested because there are some
> projects
>  >> > > (like gnulib/coreutils) which really need automake 1.11 for build in
>  >> > > latest stable versions.
>  >> >
>  >> > Is there a bug report with details of this gnulib/coreutils request?
>  >>
>  >> Not really, it was just direct irl/irc/mail communication with
>  >> automake/autoconf fedora maintainers&comaintainers. First request was
>  >> only about 1.10b in rawhide (after f-12 split) - as I needed at least
>  >> 1.10b to build coreutils-7.4 there (otherwise only with an ugly hack).
>  >
>  > Okay, but what exactly are we talking about here? What does gnulib or
>  > coreutils need that 1.10 doesn't have?
>  
>  Hi Mark,
>  
>  I think it's great that automake-1.11 made it into F11 and F10.
>  Even for F9, it's seems worthwhile.
>  
>  The features in automake-1.11 that I've found worthwhile
>  (in addition to 3 years worth of improved robustness,
>  portability and performance, fewer bugs, etc.)
>  are enabled by these two lines from coreutils' configure.ac:
>  
>    AM_INIT_AUTOMAKE([1.11 dist-xz color-tests parallel-tests])
>    AM_SILENT_RULES([yes]) 
Thanks for the details, that's exactly the kind of information I was
trying to squeeze out of this discussion :-)

IMHO, if this had been in a bug report and referenced in the update
description, this discussion would purely be about whether it makes
sense to do this in F-9 (and F-10, maybe).

The issue here is weighing up the benefit of a 1.11 update to developers
using F-9 and F-10 versus the risk of breaking existing working builds.
It sounds automake has improved its level of compatibility between
releases, so the risk is relatively low. Even still, I'd be inclined to
say that developers who want 0.11 should install it themselves or update
to F-11.

>  > A rebase of an important package in three stable releases, which
> is
>  > expected to break rebuilds of some packages, should surely have more
>  > justification than an empty update description, no associated bugzilla
>  > and claims that Jim Meyering needs some unspecified new features.
>  
>  I've been lamenting the 3-year-old version of automake in Fedora for
>  years, 
This "3 year old" issue is upstream's fault for taking that long to ship
a new release. 0.11 is 6 weeks old, so Fedora hadn't much choice in the
matter.

The timing vis-a-vis F-11 looks like it was unfortunate. Could we have
included a 0.10b in F-11 GA and update to 0.11 in an update? That would
have given the new code more Fedora testing.

Thanks again,
Mark.

[Jim Meyering]

Mark McLoughlin wrote:
...
>  The issue here is weighing up the benefit of a 1.11 update to
> developers
>  using F-9 and F-10 versus the risk of breaking existing working builds.
>  It sounds automake has improved its level of compatibility between
>  releases, so the risk is relatively low. Even still, I'd be inclined to
>  say that developers who want 0.11 should install it themselves or update
>  to F-11. 
That would make it inconvenient to use the new features in any project
that does development (runs automake) on the still-very-useful F10.
It is precisely this barrier-to-adoption that we want to overcome.
These days, there is no excuse not to provide the very latest stable
releases of automake and autoconf on F10 and F11.  Yes, they really
are that stable and robust.  Besides, these are developer-only tools.
They are not like libraries.  They typically aren't even installed on
end-user systems.  The only potential penalty is a failure (not to build
from source, but) to rebuild after rerunning autoconf.  Pretty minor,
considering all of the good reasons to upgrade, for both the developer
and the eventual user.

I try to accommodate progressiveness, when the benefit appears to outweigh
the risk.  Here, I see little risk.  So far, I know of no incompatibility
that would require any manual work.  But even if there are a few corner
cases, anyone who cannot find the time for whatever small changes are
needed to update to automake-1.11 can install an older version and
use that.

By the way, has anyone reported a problem that can be attributed
to this new version of automake?  (we won't count the one involving
gnome-common-2.26 ;-)

It's like replacement functions in gnulib: the target systems with
working functions pays no penalty at all, but a system with a missing
or broken function has to endure the cost of the replacement or wrapper.

[Ralf Corsepius]

Jim Meyering wrote:

>  I try to accommodate progressiveness, when the benefit appears to
>  outweigh the risk. 
ACK. The risk of an automake-1.10->automake-1.11 upgrade on
Fedora is 
close to zero and outweigh the effects of bug fixes having gone into 
automake-1.11.

>  So far, I know of no incompatibility
>  that would require any manual work. 
I am not aware of any, either.

The latest major breakage with autoconf/automake, I encountered was 
introduction of AC_ENABLE/DISABLE_OPTION_CHECKING, which causes 
irritating warnings when not being treated manually.

However this was introduced by autoconf, not automake, and is more a 
nuissance but a regression.

>  But even if there are a few corner
>  cases, anyone who cannot find the time for whatever small changes are
>  needed to update to automake-1.11 can install an older version and
>  use that. 
The fact "make V=..." uses an unprefixed/non-namespace safe
environment 
variable is a candidate to cause problems. Admitted, the likelihood of 
older packages tripping over this is close to null.

Ralf

[Stepan Kasal]

Hi,

On Tue, Jun 30, 2009 at 02:05:57PM -0400, Owen Taylor wrote:
>  In general automake hasn't had a very good track record of
> compatibility
>  between 1.x and 1.y, though this has been getting better recently. 
Yeah, there were some serious problems with the redisign in 2001.

Recently = since 1.8, at least.  So we are observing more than  5 years of
good track.

>  But is this the type of upgrade that makes sense in general? It seems
> to
>  me that we should be very conservative in upgrading build tools,
>  especially in "maintenance mode" distributions like F9 and F10. 
Well, in a sense, Automake is not a build tool, it's more a
maintainer tool.  In the typical case, it is run interactively by
the maintainer of a package to create the tarball.

Yes, there are many cases where automake is run from the spec file
and these are in danger.  If there were a backward compatibility bug,
these may not rebuild.  But, AFAIK, this is not anything that would
prevent people from using Fedora.

OTOH, people might want to use Fedora for software development.  And
building new versions of software packages might require new features
or rely on Automake bugs being fixed.

IOW, what's Fedora good for after its EOL?  If it is a museum
artifact, then I'm spoiling the game.  If it is to be used in real
life, then update to Automake 1.11 is beneficial for the developers
using it and harmless for the non-developer uses (office, proxy,
etc.)

>  But it is also a pretty long release announcement so it wouldn't
>  surprise me if there were some subtle incompatibilities. 
The Automake maintainer is very careful.  So it would not surprise me
if the amount of incompatibilities would be surprisingly small.

>  The only breakage I'm actually aware of in the gnome-common
> package; 
Yeah, that is a very clumsy package.

>  gnome-common-2.26 and earlier doesn't know that automake-1.11 is
>
>  a valid replacement when automake-1.10 is asked for. 
Consider telling it that 1.12 is going to be a valid replacement for
both of these as well.

Stepan

[Nalin Dahyabhai]

On Wed, Jul 01, 2009 at 11:52:20AM +0200, Stepan Kasal wrote:
>  IOW, what's Fedora good for after its EOL?  If it is a museum
>  artifact, then I'm spoiling the game.  If it is to be used in real
>  life, then update to Automake 1.11 is beneficial for the developers
>  using it and harmless for the non-developer uses (office, proxy,
>  etc.) 
After a release goes EOL, we don't push updates for it.  That means we
don't push security updates for it.  To me, that means people shouldn't
use it.

That's my short answer anyway.

Here's the long one:

The above having been said, you obviously *can*, but outside of making
sure things build on it -- a development use case which upgrading the
developer tools and libraries actually makes less useful for me -- I
strongly recommend against doing so.

Of course, I can't deny that there are people who continue to run EOL
releases in production.  But in doing so they take on the responsibility
of ensuring that things which might need updates (in particular,
security updates) either get them, or are only deployed in such a way
that they're not affected by whatever issues would normally require
updates.

That cumulative list of issues typically grows for a given release as
time passes (and as each successive release reaches EOL with an
ever-larger set of packages it may grow at a faster rate), so the amount
of work also grows as time passes.

I worry that some of the people who run EOL releases don't even know
that they've taken on this burden, that a release going EOL means that
we're not doing this work any more, and that means that *they* have to
do the work.  And I worry that that's partly due to us not always being
emphatic about warning them of it.  Sometimes it feels like failure.

Cheers,

Nalin

[Kevin Kofler]

Nalin Dahyabhai wrote:
>  I worry that some of the people who run EOL releases don't even
> know
>  that they've taken on this burden, that a release going EOL means that
>  we're not doing this work any more, and that means that *they* have to
>  do the work.  And I worry that that's partly due to us not always being
>  emphatic about warning them of it.  Sometimes it feels like failure. 
Maybe the last update to any EOL distro should be a firstboot-like tool
which just gives 3 options:
* Run preupgrade
* Wipe all hard disks
* Turn off computer

        Kevin Kofler

[Seth Vidal]

On Wed, 1 Jul 2009, Kevin Kofler wrote:

>  Nalin Dahyabhai wrote:
> > I worry that some of the people who run EOL releases don't even know
> > that they've taken on this burden, that a release going EOL means that
> > we're not doing this work any more, and that means that *they* have to
> > do the work.  And I worry that that's partly due to us not always being
> > emphatic about warning them of it.  Sometimes it feels like failure.
>
>  Maybe the last update to any EOL distro should be a firstboot-like tool
>  which just gives 3 options:
>  * Run preupgrade
>  * Wipe all hard disks
>  * Turn off computer 
yum install system-autodeath

-sv

[Kevin Kofler]

Seth Vidal wrote:
>  yum install system-autodeath 
That just turns off networking (so then how do you preupgrade from there?
And it lets people keep running their obsolete stuff forever in their
closet) and it has to be explicitly installed.

        Kevin Kofler

[Seth Vidal]

On Wed, 1 Jul 2009, Kevin Kofler wrote:

>  Seth Vidal wrote:
> > yum install system-autodeath
>
>  That just turns off networking (so then how do you preupgrade from there?
>  And it lets people keep running their obsolete stuff forever in their
>  closet) and it has to be explicitly installed.
>  
yum install sense-of-humor

-sv

[Adam Jackson]

On Wed, 2009-07-01 at 11:51 -0400, Seth Vidal wrote:
>  On Wed, 1 Jul 2009, Kevin Kofler wrote:
>  > Seth Vidal wrote:
>  >> yum install system-autodeath
>  >
>  > That just turns off networking (so then how do you preupgrade from there?
>  > And it lets people keep running their obsolete stuff forever in their
>  > closet) and it has to be explicitly installed.
>  
>  yum install sense-of-humor 
He can't, KDE doesn't support that yet.

- ajax

[Peter Jones]

On 07/01/2009 01:22 PM, Adam Jackson wrote:
>  On Wed, 2009-07-01 at 11:51 -0400, Seth Vidal wrote:
> > On Wed, 1 Jul 2009, Kevin Kofler wrote:
> >> Seth Vidal wrote:
> >>> yum install system-autodeath
> >> That just turns off networking (so then how do you preupgrade from there?
> >> And it lets people keep running their obsolete stuff forever in their
> >> closet) and it has to be explicitly installed.
> > yum install sense-of-humor
>  
>  He can't, KDE doesn't support that yet. 
Did you inform them that they needed to?

-- 
        Peter

The trouble with the global village are all the global village idiots.
		-- Paul Ginsparg

[José Abílio Matos]

On Wednesday 01 July 2009 18:22:52 Adam Jackson wrote:
>  He can't, KDE doesn't support that yet.
>
>  - ajax 
Neither does gnome apparently: :-)

$  yum search sense-of-humor
Loaded plugins: dellsysidplugin2, presto, refresh-packagekit
Warning: No matches found for: sense-of-humor
No Matches found

In any case I would recommend the better form:

yum install sense-of-humor --skip-broken

as you never know what can be broken if you do that. ;-)

-- 
José Abílio