Handling Multiple Version Scenarios

[T.C. Hollingsworth]

The "inherits" module has a new breaking version out, and some modules
are now using both versions.  Jamie and I ran into something like this
with uglify-js, and we just hacked around it for then, but doing that
for this situation would probably take longer than just finding a nice
way to make this easier generally, so lets do that.  ;-)

Here's what I have in mind:

Packages in this situation should install in version-specific
directories under %{nodejs_sitelib}, e.g.:
uglify-js (2.x) -> %{nodejs_sitelib}/uglify-js@2
uglify-js1 (1.x) -> %{nodejs_sitelib}/uglify-js@1

The @ sign is special to npm.  To get uglify-js 1 instead of 2 with
npm you'd run `npm install uglify-js@1`, so by using it here that
means you can also `npm link uglify-js@1` to bring in the RPM version.
 It also prevents clashes with module names that happen to contain
numbers, and opens the door to patches to npm in the future to be
smarter here.  (e.g. so even `npm link uglify-js(a)1.3` will do the
right thing as `npm install uglify-js(a)1.3` does.)

The main uglify-js package in this example would also contain a
%{nodejs_sitelib}/uglify-js -> uglify-js@2 symlink so `npm link
uglify-js` continues to work.

Now that we have a way to install them, we need a way to tell
%nodejs_symlink_deps about these special modules.  There are two ways
I can think of:

1.) Make it check %{nodejs_sitelib} at build time for the existence of
these name@version directories.  This requires that every package that
have dependencies experiencing this issue to BuildRequire those
dependencies, but doesn't require keeping track of these centrally as
with 2.

2.) Just keep a text file in "nodejs-packaging" with a list of these
special modules.  That means updating nodejs-packaging and keeping
track of these special modules, but nothing special for package
maintainers unless they need to BuildRequire them for %check.  This
text file would just be a simple newline-separated list of the npm
names of the affected modules.

There's also technically 3.) Just ship everything in versioned
directories so we don't have to worry about it.  But that adds a bunch
of complexity for a 1% case and requires changing every package, so I
think that one is a no go.

I'm leaning towards 2 because that prevents packagers from having to
deal with any of this in most cases.  The only one where they'd need
to be aware of it is for BuildRequires when you need the right version
of the module, in which case they'd just have to make sure they drag
in the right version, preferably with something like:
BuildRequires: npm(uglify-js) < 2

Note that the right RPM is already brought in properly regardless
thanks to the "npm(foo)" virtual provides, so that part of the problem
is already solved.  The RPM name will remain irrelevant, but should
follow the standard Fedora policy for this regardless (which seems to
be just appending the major part of the version to the name).

Also, I'm only worrying about major versions here, because all npm
modules are supposed to follow the semantic versioning spec [1] which
requires a major version bump for this sort of thing to happen anyway,
so I don't see the point in overengineering this at this time.

I'm curious if anyone has any other suggestions or a recommendation
here.  Thanks!

-T.C.

[1] http://semver.org/

[Stanislav Ochotnicky]

Quoting T.C. Hollingsworth (2013-06-05 03:56:24)
...
>  Now that we have a way to install them, we need a way to tell
>  %nodejs_symlink_deps about these special modules.  There are two ways
>  I can think of:
>  
>  1.) Make it check %{nodejs_sitelib} at build time for the existence of
>  these name@version directories.  This requires that every package that
>  have dependencies experiencing this issue to BuildRequire those
>  dependencies, but doesn't require keeping track of these centrally as
>  with 2. 
You mean so that even package that only requires version @2 would need to BR @1?
I am probably misunderstanding something here.

>  
>  2.) Just keep a text file in "nodejs-packaging" with a list of these
>  special modules.  That means updating nodejs-packaging and keeping
>  track of these special modules, but nothing special for package
>  maintainers unless they need to BuildRequire them for %check.  This
>  text file would just be a simple newline-separated list of the npm
>  names of the affected modules. 
We used to have something similar in Java world (maven2-common-poms) which
contained mappings between upstream version and fedora JAR. For reasons that
likely wouldn't affect Node.js this approach was quite bad (and got worse over
time). If you believe the list can be easily updated, pruned and just maintained
then sure, no problem.


>  There's also technically 3.) Just ship everything in versioned
>  directories so we don't have to worry about it.  But that adds a bunch
>  of complexity for a 1% case and requires changing every package, so I
>  think that one is a no go. 
Nah

>  I'm leaning towards 2 because that prevents packagers from having
> to
>  deal with any of this in most cases.  The only one where they'd need
>  to be aware of it is for BuildRequires when you need the right version
>  of the module, in which case they'd just have to make sure they drag
>  in the right version, preferably with something like:
>  BuildRequires: npm(uglify-js) < 2
>  
>  Note that the right RPM is already brought in properly regardless
>  thanks to the "npm(foo)" virtual provides, so that part of the problem
>  is already solved.  The RPM name will remain irrelevant, but should
>  follow the standard Fedora policy for this regardless (which seems to
>  be just appending the major part of the version to the name). 
Well...I have one request WRT naming: Always make the latest-greatest version be
without version suffix. When you need to package a new version that breaks stuff
do it like this:

    1. Create new package that is a copy of current version (just append major
       version number to the name)
    2. Update original version to latest/greatest. 

Automatic RPM requires should help here, and you will prevent explosion of
higher and higher version numbers in package names. I've seen it first hand,
it's ugly.

eS

-- 
Stanislav Ochotnicky <sochotnicky(a)redhat.com&gt;
Software Engineer - Developer Experience

PGP: 7B087241
Red Hat Inc.                               http://cz.redhat.com

[T.C. Hollingsworth]

On Wed, Jun 5, 2013 at 12:49 AM, Stanislav Ochotnicky
<sochotnicky(a)redhat.com&gt; wrote:
>  Quoting T.C. Hollingsworth (2013-06-05 03:56:24)
>  ...
> > Now that we have a way to install them, we need a way to tell
> > %nodejs_symlink_deps about these special modules.  There are two ways
> > I can think of:
> >
> > 1.) Make it check %{nodejs_sitelib} at build time for the existence of
> > these name@version directories.  This requires that every package that
> > have dependencies experiencing this issue to BuildRequire those
> > dependencies, but doesn't require keeping track of these centrally as
> > with 2.
>
>  You mean so that even package that only requires version @2 would need to BR @1? 
No, they could actually BuildRequire any arbitrary version of the
module.  (Although obviously it would be preferable to BuildRequire
the one you're using.  ;-)  I'd want to keep the test simple, just
checking for any directory that startswith(npm_name + '@').

>  I am probably misunderstanding something here. 
My fault, that sentence doesn't parse like I meant it to.

> > 2.) Just keep a text file in "nodejs-packaging" with a
> list of these
> > special modules.  That means updating nodejs-packaging and keeping
> > track of these special modules, but nothing special for package
> > maintainers unless they need to BuildRequire them for %check.  This
> > text file would just be a simple newline-separated list of the npm
> > names of the affected modules.
>
>  We used to have something similar in Java world (maven2-common-poms) which
>  contained mappings between upstream version and fedora JAR. For reasons that
>  likely wouldn't affect Node.js this approach was quite bad (and got worse over
>  time). If you believe the list can be easily updated, pruned and just maintained
>  then sure, no problem. 
If for some reason this file is inaccurate, %nodejs_symlink_deps will
do the wrong thing and create a dangling symlink, the end result being
a backtrace upon trying to require() the module.  Hopefully this would
be enough incentive to keep the file updated.  ;-)

> > There's also technically 3.) Just ship everything in
> versioned
> > directories so we don't have to worry about it.  But that adds a bunch
> > of complexity for a 1% case and requires changing every package, so I
> > think that one is a no go.
>
>  Nah
>
> > I'm leaning towards 2 because that prevents packagers from having to
> > deal with any of this in most cases.  The only one where they'd need
> > to be aware of it is for BuildRequires when you need the right version
> > of the module, in which case they'd just have to make sure they drag
> > in the right version, preferably with something like:
> > BuildRequires: npm(uglify-js) < 2
> >
> > Note that the right RPM is already brought in properly regardless
> > thanks to the "npm(foo)" virtual provides, so that part of the problem
> > is already solved.  The RPM name will remain irrelevant, but should
> > follow the standard Fedora policy for this regardless (which seems to
> > be just appending the major part of the version to the name).
>
>  Well...I have one request WRT naming: Always make the latest-greatest version be
>  without version suffix. When you need to package a new version that breaks stuff
>  do it like this:
>
>      1. Create new package that is a copy of current version (just append major
>         version number to the name)
>      2. Update original version to latest/greatest. 
This is exactly what I had in mind, what I did for uglify-js, and what
I intend to do for nodejs-inherits.  :-)

>  Automatic RPM requires should help here, and you will prevent
> explosion of
>  higher and higher version numbers in package names. I've seen it first hand,
>  it's ugly. 
I really hope this stays a rare instance, but that's part of the
reason why I want to keep this limited to only the major version.
(And luckily, that's exactly how the npm versioning spec is supposed
to work.)

-T.C.