8:48 a.m.
I've started working on packaging scuttlebot for Fedora. I see that we now have a
Fedora package for every nodejs module. This makes it easy to map directories in
node_modules to package names - however, it means submitting hundreds of packages to ever
get scuttlebot submitted.
I'm wondering if there is a better way. A node module typically corresponds to a .o
file in a C library (with exceptions like libsodium). It is like having a separate
package for every function in glibc. Suppose we did this:
1) a nodejs-stdlib that includes all the common modules (a list to be argued over at
length :-) ). There is no penalty other than a small amount of disk space for unused
modules - just like with a C library.
2) other multi-module systems are combined - usually including all modules with the same
first word. For instance, all the pull stream modules begin with 'pull':
pull-abortable, pull-box-stream, pull-cat, pull-cont, etc. This would become nodejs-pull,
and include all the pull modules.
When the package name matches the first word of the module name, then determining the
package is still easy. When that is not the case, as with the proposed nodejs-stdlib,
then dnf can still search for npm(...)
9:42 a.m.
On Fri, Aug 4, 2017 at 6:48 AM, Stuart D Gathman <stuart(a)gathman.org> wrote:
I've started working on packaging scuttlebot for Fedora. I see
that we now have a Fedora package for every nodejs module. This makes it easy to map
directories in node_modules to package names - however, it means submitting hundreds of
packages to ever get scuttlebot submitted.
I'm wondering if there is a better way. A node module typically corresponds to a .o
file in a C library (with exceptions like libsodium). It is like having a separate
package for every function in glibc. Suppose we did this:
1) a nodejs-stdlib that includes all the common modules (a list to be argued over at
length :-) ). There is no penalty other than a small amount of disk space for unused
modules - just like with a C library.
2) other multi-module systems are combined - usually including all modules with the same
first word. For instance, all the pull stream modules begin with 'pull':
pull-abortable, pull-box-stream, pull-cat, pull-cont, etc. This would become nodejs-pull,
and include all the pull modules.
When the package name matches the first word of the module name, then determining the
package is still easy. When that is not the case, as with the proposed nodejs-stdlib,
then dnf can still search for npm(...)
Although this looks tempting, it's only looking at half the problem.
Versions
Nodejs modules get updated all the time, at different rates for
different packages.
If you had just one package for many modules, it would be getting
updated at an alarming rate.
Troy
7:06 a.m.
On Fri, Aug 4, 2017 at 10:43 AM Troy Dawson <tdawson(a)redhat.com> wrote:
On Fri, Aug 4, 2017 at 6:48 AM, Stuart D Gathman
<stuart(a)gathman.org>
wrote:
> I've started working on packaging scuttlebot for Fedora. I see that we
now have a Fedora package for every nodejs module. This makes it easy to
map directories in node_modules to package names - however, it means
submitting hundreds of packages to ever get scuttlebot submitted.
>
> I'm wondering if there is a better way. A node module typically
corresponds to a .o file in a C library (with exceptions like libsodium).
It is like having a separate package for every function in glibc. Suppose
we did this:
>
> 1) a nodejs-stdlib that includes all the common modules (a list to be
argued over at length :-) ). There is no penalty other than a small amount
of disk space for unused modules - just like with a C library.
>
> 2) other multi-module systems are combined - usually including all
modules with the same first word. For instance, all the pull stream
modules begin with 'pull': pull-abortable, pull-box-stream, pull-cat,
pull-cont, etc. This would become nodejs-pull, and include all the pull
modules.
>
> When the package name matches the first word of the module name, then
determining the package is still easy. When that is not the case, as with
the proposed nodejs-stdlib, then dnf can still search for npm(...)
Although this looks tempting, it's only looking at half the problem.
Versions
Nodejs modules get updated all the time, at different rates for
different packages.
If you had just one package for many modules, it would be getting
updated at an alarming rate.
I think the better approach is carefully-controlled bundling in Fedora. As
of a couple years ago, it is now permissible to bundle software together in
Fedora if it meets certain conditions:
1) If the dependency is already packaged in Fedora and this software is
compatible with that version, then this software must link against the
unbundled version.
2) If the dependency is not yet packaged in Fedora but is likely to be
useful to large amounts of Fedora software, it is strongly encouraged that
it be packaged separately.
3) Other dependencies MAY be carried internally by the package that needs
them, but that package MUST include `Provides: bundled(npm(modulename)) =
version`
Rule 3) is so that if there is a security vulnerability in npm(modulename),
we can find any and all software that is required to be updated.
I haven't had any time to work on it, but I'd very much like to develop an
automatic RPM dependency generator that will recurse down the node_modules
directories, read their package.json files and automatically create those
Provides. If anyone else wants to take a crack at doing that, it would be
an immense help.
9:09 p.m.
On 08/09/2017 08:06 AM, Stephen Gallagher wrote:
On Fri, Aug 4, 2017 at 10:43 AM Troy Dawson <tdawson(a)redhat.com
<mailto:tdawson@redhat.com>> wrote:
Although this looks tempting, it's only looking at half the problem.
Versions
Nodejs modules get updated all the time, at different rates for
different packages.
If you had just one package for many modules, it would be getting
updated at an alarming rate.
Point taken.
I think the better approach is carefully-controlled bundling in
Fedora. As of a couple years ago, it is now permissible to bundle
software together in Fedora if it meets certain conditions:
1) If the dependency is already packaged in Fedora and this software
is compatible with that version, then this software must link against
the unbundled version.
2) If the dependency is not yet packaged in Fedora but is likely to be
useful to large amounts of Fedora software, it is strongly encouraged
that it be packaged separately.
3) Other dependencies MAY be carried internally by the package that
needs them, but that package MUST include `Provides:
bundled(npm(modulename)) = version`
Rule 3) is so that if there is a security vulnerability in
npm(modulename), we can find any and all software that is required to
be updated.
I haven't had any time to work on it, but I'd very much like to
develop an automatic RPM dependency generator that will recurse down
the node_modules directories, read their package.json files and
automatically create those Provides. If anyone else wants to take a
crack at doing that, it would be an immense help.
That sounds workable. Thanks!
There are 502 npm dependencies of
scuttlebot, 196 of which are already in Fedora. I shudder to think of
the Requires: list and the Provides: list!!
11:45 a.m.
On Wed, Aug 9, 2017 at 8:06 AM Stephen Gallagher <sgallagh(a)redhat.com>
wrote:
I haven't had any time to work on it, but I'd very much like
to develop an
automatic RPM dependency generator that will recurse down the node_modules
directories, read their package.json files and automatically create those
Provides. If anyone else wants to take a crack at doing that, it would be
an immense help.
A big thank-you needs to go to Tomas Tomacek for
https://pagure.io/nodejs-packaging/pull-request/2 which is now in effect on
Fedora 28 (Rawhide) builds.
So now any Node.js package that is built in Fedora will automatically have
`Provides: bundled(nodejs-foo)` added automatically.
3:59 a.m.
I was wondering, could we use bundling and clean up the mess that is
current nodejs stack in Fedora?
To be more precise, there are currently over 2000 packages and over 500
nodejs-* is available bugs,
that nobody has time to fix and/or feels like fixing, because updating one
module might/will break several others.
For the sake of maintainability, I'd like to leave as little packages as
possible.
On Thu, Sep 14, 2017 at 6:45 PM, Stephen Gallagher <sgallagh(a)redhat.com>
wrote:
On Wed, Aug 9, 2017 at 8:06 AM Stephen Gallagher <sgallagh(a)redhat.com>
wrote:
> I haven't had any time to work on it, but I'd very much like to develop
> an automatic RPM dependency generator that will recurse down the
> node_modules directories, read their package.json files and automatically
> create those Provides. If anyone else wants to take a crack at doing that,
> it would be an immense help.
>
A big thank-you needs to go to Tomas Tomacek for
https://pagure.io/nodejs-packaging/pull-request/2 which is now in effect
on Fedora 28 (Rawhide) builds.
So now any Node.js package that is built in Fedora will automatically have
`Provides: bundled(nodejs-foo)` added automatically.
_______________________________________________
nodejs mailing list -- nodejs(a)lists.fedoraproject.org
To unsubscribe send an email to nodejs-leave(a)lists.fedoraproject.org
5:11 a.m.
That sounds like a Change Proposal for F28.
So your idea would be to mass-retire all but the most common or
security-sensitive NPMs from Fedora and rely on bundling in most cases?
That will also mean coming up with a strategy for removing those unbundled
packages on updates. Thoughts on that?
On Wed, Sep 20, 2017, 4:59 AM Zuzana Svetlikova <zsvetlik(a)redhat.com> wrote:
I was wondering, could we use bundling and clean up the mess that is
current nodejs stack in Fedora?
To be more precise, there are currently over 2000 packages and over 500
nodejs-* is available bugs,
that nobody has time to fix and/or feels like fixing, because updating one
module might/will break several others.
For the sake of maintainability, I'd like to leave as little packages as
possible.
On Thu, Sep 14, 2017 at 6:45 PM, Stephen Gallagher <sgallagh(a)redhat.com>
wrote:
>
>
> On Wed, Aug 9, 2017 at 8:06 AM Stephen Gallagher <sgallagh(a)redhat.com>
> wrote:
>
>> I haven't had any time to work on it, but I'd very much like to develop
>> an automatic RPM dependency generator that will recurse down the
>> node_modules directories, read their package.json files and automatically
>> create those Provides. If anyone else wants to take a crack at doing that,
>> it would be an immense help.
>>
>
> A big thank-you needs to go to Tomas Tomacek for
> https://pagure.io/nodejs-packaging/pull-request/2 which is now in effect
> on Fedora 28 (Rawhide) builds.
>
> So now any Node.js package that is built in Fedora will automatically
> have `Provides: bundled(nodejs-foo)` added automatically.
>
> _______________________________________________
> nodejs mailing list -- nodejs(a)lists.fedoraproject.org
> To unsubscribe send an email to nodejs-leave(a)lists.fedoraproject.org
>
> _______________________________________________
nodejs mailing list -- nodejs(a)lists.fedoraproject.org
To unsubscribe send an email to nodejs-leave(a)lists.fedoraproject.org
5:34 a.m.
Binary compiled ones probably also need to be kept.
Presumably this also means identifying "top level" packages that are an
end goal in themselves and need to be kept.
Tom
On 20/09/17 11:11, Stephen Gallagher wrote:
That sounds like a Change Proposal for F28.
So your idea would be to mass-retire all but the most common or
security-sensitive NPMs from Fedora and rely on bundling in most cases?
That will also mean coming up with a strategy for removing those
unbundled packages on updates. Thoughts on that?
On Wed, Sep 20, 2017, 4:59 AM Zuzana Svetlikova <zsvetlik(a)redhat.com
<mailto:zsvetlik@redhat.com>> wrote:
I was wondering, could we use bundling and clean up the mess that is
current nodejs stack in Fedora?
To be more precise, there are currently over 2000 packages and over
500 nodejs-* is available bugs,
that nobody has time to fix and/or feels like fixing, because
updating one module might/will break several others.
For the sake of maintainability, I'd like to leave as little
packages as possible.
On Thu, Sep 14, 2017 at 6:45 PM, Stephen Gallagher
<sgallagh(a)redhat.com <mailto:sgallagh@redhat.com>> wrote:
On Wed, Aug 9, 2017 at 8:06 AM Stephen Gallagher
<sgallagh(a)redhat.com <mailto:sgallagh@redhat.com>> wrote:
I haven't had any time to work on it, but I'd very much like
to develop an automatic RPM dependency generator that will
recurse down the node_modules directories, read their
package.json files and automatically create those Provides.
If anyone else wants to take a crack at doing that, it would
be an immense help.
A big thank-you needs to go to Tomas Tomacek for
https://pagure.io/nodejs-packaging/pull-request/2 which is now
in effect on Fedora 28 (Rawhide) builds.
So now any Node.js package that is built in Fedora will
automatically have `Provides: bundled(nodejs-foo)` added
automatically.
_______________________________________________
nodejs mailing list -- nodejs(a)lists.fedoraproject.org
<mailto:nodejs@lists.fedoraproject.org>
To unsubscribe send an email to
nodejs-leave(a)lists.fedoraproject.org
<mailto:nodejs-leave@lists.fedoraproject.org>
_______________________________________________
nodejs mailing list -- nodejs(a)lists.fedoraproject.org
<mailto:nodejs@lists.fedoraproject.org>
To unsubscribe send an email to nodejs-leave(a)lists.fedoraproject.org
<mailto:nodejs-leave@lists.fedoraproject.org>
_______________________________________________
nodejs mailing list -- nodejs(a)lists.fedoraproject.org
To unsubscribe send an email to nodejs-leave(a)lists.fedoraproject.org
--
Tom Hughes (tom(a)compton.nu)
http://compton.nu/
2408
days inactive
2455
days old
nodejs@lists.fedoraproject.org
7 comments
6 participants
participants (6)
-
Stephen Gallagher -
Stuart D Gathman -
Stuart Gathman
-
Tom Hughes -
Troy Dawson -
Zuzana Svetlikova