Old Flex version in FC5?

[Benjamin Thery]

Hello,

I use Fedora Core 5 to test Mobile IPv6 on Linux.
In a recent version of the mobility daemon source code a change made 
to a lex file make the build failson FC5. This changes introduced some 
flex options to reduce warnings output during compilation.

To my surprise these options are not supported by the current flex 
version shipped with FC5. I found it strange because FC5 is very 
recent. Then, googling a bit, I discovered that flex state seems a bit 
confusing:

- Fedora ships 2.5.4a (released in 1997)
- Gentoo has 2.5.33
- Debian has two different packages flex-old (2.5.4a) and flex (2.5.33)

My question is: why does Fedora ships flex 2.5.4a and not a more 
recent one?
What are the mysteries behind flex versioning?

Thanks,

Benjamin

[Rex Dieter]

Benjamin Thery wrote:
> My question is: why does Fedora ships flex 2.5.4a and not a more recent 
> one?

Probably at least 1 of:
* If it aint broke, don't fix it
* No one has asked

File an update request against Fedora Core/flex at 
http://bugzilla.redhat.com/

-- Rex

[Ralf Corsepius]

On Thu, 2006-05-04 at 07:33 -0500, Rex Dieter wrote:
> Benjamin Thery wrote:
> > My question is: why does Fedora ships flex 2.5.4a and not a more recent 
> > one?
>
> Probably at least 1 of:
> If it aint broke, don't fix it
> No one has asked

* Many flex versions > 2.5.4a had been too broken to be usable
* Many flex versions > 2.5.4a had been too incompatible to flex-2.5.x
* Many major packages have not been prepared for flex > 2.5.x
> File an update request against Fedora Core/flex at 
> http://bugzilla.redhat.com/

Ralf

[Rex Dieter]

Ralf Corsepius wrote:
> On Thu, 2006-05-04 at 07:33 -0500, Rex Dieter wrote:
> > Benjamin Thery wrote:
> > > My question is: why does Fedora ships flex 2.5.4a and not a more recent 
> > > one?
> >
> > Probably at least 1 of:
> > If it aint broke, don't fix it
> > No one has asked
> Many flex versions > 2.5.4a had been too broken to be usable
> Many flex versions > 2.5.4a had been too incompatible to flex-2.5.x
> Many major packages have not been prepared for flex > 2.5.x

Probably explains why Benjamin noticed other distros with 2 flex 
packages (one old, one new).  Perhaps someone could review possibilities 
of dually-installed flex pkgs, and submit the "new" one to Extras. (:

-- Rex

[Benjamin Thery]

Rex Dieter wrote:
> Ralf Corsepius wrote:
> > On Thu, 2006-05-04 at 07:33 -0500, Rex Dieter wrote:
> > > Benjamin Thery wrote:
> > > > My question is: why does Fedora ships flex 2.5.4a and not a more 
> > > > recent one?
> > >
> > > Probably at least 1 of:
> > > If it aint broke, don't fix it
> > > No one has asked
> > Many flex versions > 2.5.4a had been too broken to be usable
> > Many flex versions > 2.5.4a had been too incompatible to flex-2.5.x
> > Many major packages have not been prepared for flex > 2.5.x
>
> Probably explains why Benjamin noticed other distros with 2 flex 
> packages (one old, one new).  
> Perhaps someone could review possibilities 
> of dually-installed flex pkgs, and submit the "new" one to Extras. (:

This sounds like a good compromise. One of the first thing I did when 
I discovered Debian has two packages was: heading to Extras to see if 
there was also an "updated-experimental" flex package there :)

I've opened a bugzilla to track this update request: 190673.

Thanks for your answers.
Benjamin

[Michael A. Peters]

On Thu, 2006-05-04 at 09:30 -0500, Rex Dieter wrote:
> Probably explains why Benjamin noticed other distros with 2 flex 
> packages (one old, one new).  Perhaps someone could review possibilities 
> of dually-installed flex pkgs, and submit the "new" one to Extras. (:

Does Debian install them parallel installable?

I remember this being discussed before in an Extras review - where
someone was submitting a package that needed the new flex.

If I remember, the problem was that the new flex would have to be
installed in such a way that it was never used by mistake, and no one
had submitted a package yet. The package ended up being patched to not
need the new flex, and nothing more came of it.

How many software packages out there do not build on Fedora Core for
lack of the new flex?

[Benjamin Thery]

Michael A. Peters wrote:
> On Thu, 2006-05-04 at 09:30 -0500, Rex Dieter wrote:
> > Probably explains why Benjamin noticed other distros with 2 flex 
> > packages (one old, one new).  Perhaps someone could review possibilities 
> > of dually-installed flex pkgs, and submit the "new" one to Extras. (:
>
> Does Debian install them parallel installable?

Doesn't seem so.
The files overwrite each others:
http://packages.debian.org/cgi-bin/search_contents.pl?searchmode=filelist&am...
> I remember this being discussed before in an Extras review - where
> someone was submitting a package that needed the new flex.
>
> If I remember, the problem was that the new flex would have to be
> installed in such a way that it was never used by mistake, and no one
> had submitted a package yet. The package ended up being patched to not
> need the new flex, and nothing more came of it.
>
> How many software packages out there do not build on Fedora Core for
> lack of the new flex?

[Ralf Ertzinger]

Hi.

On Thu, 04 May 2006 14:40:22 +0200, Ralf Corsepius wrote:
> Many flex versions > 2.5.4a had been too broken to be usable
> Many flex versions > 2.5.4a had been too incompatible to flex-2.5.x
> Many major packages have not been prepared for flex > 2.5.x

It can't be that bad if Debian stable uses 2.5.31.

[Ralf Corsepius]

On Thu, 2006-05-04 at 16:40 +0200, Ralf Ertzinger wrote:
> Hi.
>
> On Thu, 04 May 2006 14:40:22 +0200, Ralf Corsepius wrote:
> > Many flex versions > 2.5.4a had been too broken to be usable
> > Many flex versions > 2.5.4a had been too incompatible to flex-2.5.x
> > Many major packages have not been prepared for flex > 2.5.x
>
> It can't be that bad if Debian stable uses 2.5.31.

Well, it's worse than that ... I guess you are aware which major
packages I am referring to and what all might break because of a flex
upgrade?

Ralf

[Ralf Ertzinger]

Hi.

On Thu, 04 May 2006 17:05:15 +0200, Ralf Corsepius wrote:
> Well, it's worse than that ... I guess you are aware which major
> packages I am referring to and what all might break because of a flex
> upgrade?

I am not.

[Ralf Corsepius]

On Thu, 2006-05-04 at 17:07 +0200, Ralf Ertzinger wrote:
> Hi.
>
> On Thu, 04 May 2006 17:05:15 +0200, Ralf Corsepius wrote:
> > Well, it's worse than that ... I guess you are aware which major
> > packages I am referring to and what all might break because of a flex
> > upgrade?
>
> I am not.

The very core of the distro: Binutils, gdb, GCC.

Ralf

[John Ellson]

Ralf Ertzinger wrote:
> Hi.
>
> On Thu, 04 May 2006 14:40:22 +0200, Ralf Corsepius wrote:
> > Many flex versions > 2.5.4a had been too broken to be usable
> > Many flex versions > 2.5.4a had been too incompatible to flex-2.5.x
> > Many major packages have not been prepared for flex > 2.5.x
>
> It can't be that bad if Debian stable uses 2.5.31.

flex-2.5..31 on Debian definitely broke graphviz builds.

I just tried flex-2.5.33 from  Sourceforge on an x86_64 Rawhide box
and that particular problem seems to be fixed,  but I would have to
test all platform builds to be more confident.

Problems with flex and bison can be particularly obscure and result in
strange breakages in stable code.    Just this week I ran into a problem
in bison that only showed up when bison products generated on FC5
were compiled on Windows. (Fixed promptly by Roland McGrath.  Thanks 
again.).

These tools need extra care when updating.

John

[Bojan Smojver]

John Ellson <ellson <at> research.att.com> writes:
> These tools need extra care when updating.

Absolutely.

Would there be interest in having a flex-reentrant (based on 2.5.33) package in
Extras (in parallel with existing flex) that would install itself in /opt/flex,
as per:

http://www.pathname.com/fhs/pub/fhs-2.3.html#OPTADDONAPPLICATIONSOFTWAREPACK...

With such approach, both pieces of software could exist on the system at the
same time without, with the old, proven flex being the default. The reentrant
flex would be completely invisible in terms of binary paths, libraries, man
pages etc., until explicity referenced by other software needing this kind of
functionality.

PS. I'm aware that /opt isn't the most popular choice (*cough* is completely
against policy *cough*) for Fedora [Extras] packages, but in this particular
case it may give us some optional (pun intended :-) functionality that we
otherwise wouldn't have.

--
Bojan

[Bojan Smojver]

Bojan Smojver <bojan <at> rexursive.com> writes:
> Would there be interest in having a flex-reentrant (based on 2.5.33) package in
> Extras

Something like this:

ftp://ftp.rexursive.com/pub/flex/flex-reentrant.spec

--
Bojan

Update to flex 2.5.33

[Petr Machata]

On Thu, 2006-05-11 at 04:15 +0000, Bojan Smojver wrote:
> Bojan Smojver <bojan <at> rexursive.com> writes:
> > Would there be interest in having a flex-reentrant (based on 2.5.33) package in
> > Extras
>
> Something like this:
>
> ftp://ftp.rexursive.com/pub/flex/flex-reentrant.spec

Ok.  Having flex 2.5.3x in fedora was my original goal when I became
maintainer of flex.  However old version is heavy with patches, and new
version went more or less it's own way.  So I'd rather do it only after
certain confidence grows.

So... what about creating flex-reentrant, that would be drop-in
replacement for flex?  People willing to experiment would install it
instead of flex:
  Name: flex-reentrant
  Version: 2.5.33
  Obsoletes: flex < 2.5.33
  Provides: flex = %{version}

All packages requiring flex would then pick up flex-reentrant
automatically, without the need to pass configure flags or to setup
environment variables.  I think this way, we would get the greatest
impact, yet the users would be free to choose if they want to install
reentrant flex.

When new flex turns out to be just fine, original flex package will be
updated, obsoleting reentrant.

The downside obviously is that you will have either reentrant, or old
flex installed.  This issue doesn't arise for Bojan's package, which
installs flex to /opt.  However I'm afraid people would find it
cumbersome to have to explicitly pick the right binary.

Thoughts?
> Bojan

PM

Update to flex 2.5.33

[Ralf Corsepius]

On Thu, 2006-06-15 at 19:26 +0200, Petr Machata wrote:
> On Thu, 2006-05-11 at 04:15 +0000, Bojan Smojver wrote:
> > Bojan Smojver <bojan <at> rexursive.com> writes:
> > > Would there be interest in having a flex-reentrant (based on 2.5.33) package in
> > > Extras
> >
> > Something like this:
> >
> > ftp://ftp.rexursive.com/pub/flex/flex-reentrant.spec
>
> Ok.  Having flex 2.5.3x in fedora was my original goal when I became
> maintainer of flex.  However old version is heavy with patches, and new
> version went more or less it's own way.  So I'd rather do it only after
> certain confidence grows.
>
> So... what about creating flex-reentrant,

Naming it flex-reentrant is a mis-nomer.

Reentrancy in flex code is a feature of a scanner's sources, not a
feature of the scanner-generator (flex). It had been possible to write
reentrant scanners even with old flex for many years.

Ralf

Update to flex 2.5.33

[Petr Machata]

Ralf Corsepius wrote:
> On Thu, 2006-06-15 at 19:26 +0200, Petr Machata wrote:
> > On Thu, 2006-05-11 at 04:15 +0000, Bojan Smojver wrote:
> > > Bojan Smojver <bojan <at> rexursive.com> writes:
> > > > Would there be interest in having a flex-reentrant (based on 2.5.33) package in
> > > > Extras
> > >
> > > Something like this:
> > >
> > > ftp://ftp.rexursive.com/pub/flex/flex-reentrant.spec
> >
> > Ok.  Having flex 2.5.3x in fedora was my original goal when I became
> > maintainer of flex.  However old version is heavy with patches, and new
> > version went more or less it's own way.  So I'd rather do it only after
> > certain confidence grows.
> >
> > So... what about creating flex-reentrant,
>
> Naming it flex-reentrant is a mis-nomer.
>
> Reentrancy in flex code is a feature of a scanner's sources, not a
> feature of the scanner-generator (flex). It had been possible to write

Yes.
> reentrant scanners even with old flex for many years.

Possibly.  The name refers to -R switch in 2.5.3x, which promises to
build reentrant lexer.  With this switch, built lexers store all
variables in a structure passed via parameter, using no global
variables.  Afaik, flex was doing similar thing in c++ mode for a long
time now.

Anyway, we can as well call is flex-new.
> Ralf

PM