jquery bundling for new packages

[Jens-Ulrik Petersen]

TIL that there are around a thousand packages
in Fedora that include a file named jquery.js... [1]

Anyway it is good to see efforts to tackle these kinds of problems
with the Javascript and Web Assets Guidelines. :)

I want to ask if the packaging of grunt and jquery
blocks current package reviews of packages that bundle jquery.js?
Or can they proceed for now until jquery is actually packaged in Fedora?

Jens

ps It is going to be a long cleanup process to fix all those packages with bundled files.

[1] https://fedoraproject.org/wiki/Changes/Web_Assets#Web_application_packagers

[Mattias Ellert]

tor 2013-09-19 klockan 05:35 -0400 skrev Jens Petersen:
> TIL that there are around a thousand packages
> in Fedora that include a file named jquery.js... [1]
>
> Anyway it is good to see efforts to tackle these kinds of problems
> with the Javascript and Web Assets Guidelines. :)
>
> I want to ask if the packaging of grunt and jquery
> blocks current package reviews of packages that bundle jquery.js?
> Or can they proceed for now until jquery is actually packaged in Fedora?
>
> Jens
>
> ps It is going to be a long cleanup process to fix all those packages with bundled files.
>
> [1] https://fedoraproject.org/wiki/Changes/Web_Assets#Web_application_packagers

I am especially interested in how to deal with doxygen generated
documentation. I maintain quite a lot such packages. Will every package
using doxygen have to replace the jquery.js installed by doxygen with a
symlink in %install, or will doxygen be patched to install a symlink
instead of a bundled copy?

Mattias

[T.C. Hollingsworth]

On Thu, Sep 19, 2013 at 3:10 AM, Mattias Ellert
 wrote:
> I am especially interested in how to deal with doxygen generated
> documentation. I maintain quite a lot such packages. Will every package
> using doxygen have to replace the jquery.js installed by doxygen with a
> symlink in %install, or will doxygen be patched to install a symlink
> instead of a bundled copy?

Definitely something like the latter.  Patching every doxygen-using spec file
would be madness.  :-)

-T.C.

[Christopher Meng]

Just a question: How to deal with the incompatibility when patching all
jquery js to the latest one?

I have some web apps which don't need the latest version of jquery, how
should I do if I want step further?

[T.C. Hollingsworth]

On Sat, Sep 21, 2013 at 2:34 AM, Christopher Meng  wrote:
> Just a question: How to deal with the incompatibility when patching all
> jquery js to the latest one?
>
> I have some web apps which don't need the latest version of jquery, how
> should I do if I want step further?

My plan is to have two parallel installable jquery packages: one for the 2.x
branch and one for the 1.x branch.  As much as I'd like to skip the latter,
older IE versions will continue to be the bane of web developers' existence for
some time to come.  We'll continue to support the 1.x series in Fedora as long
as jQuery upstream supports it.

For stuff that needs older jQuery versions, ideally we can patch them to work
with the latest version.  But for a quick fix, we can get most of them working
with 1.10 using jquery-migrate [1] with minimal effort.  (All you need is an
extra <script> tag in most cases.  :-)

For stuff that uses really old jQuery versions where even jquery-migrate isn't
an option (e.g. < 1.6), my first concern is whether upstream is alive in the
first place.  Because seriously, that stuff is OLD.  We really shouldn't be
shipping code that was deprecated three years ago!  If upstream is actively
maintained, then they really need to get with the program.

That being said, our de facto policy regarding bundling is to grandfather old
packages in, which means they can bundle those ancient versions of jQuery till
the end of days if that's what they want.  Though I might have to ask FPC if I
can add Provides: security-nightmare-waiting-to-happen to their spec files.  ;-)

Where migration is possible, I fully intend to have everything unbundled for
F21, wielding my shiny new provenpackager hammer wherever necessary.

Going forward we'll treat jQuery just like any other library in the
distribution.  When a new version comes out, ideally we'll migrate everything by
the next Fedora release, or else provide a parallel-installable backward compat
package only when absolutely necessary.

If you want to help get ready, I suggest working with your upstreams to migrate
to jQuery 1.10, using jquery-migrate where necessary.  That'll make our lives
much easier when we get ready to unbundle.

BTW, I intend to write up a couple of blog posts outlining the general plan and
a list of tips for packagers who want to get a head start within the next month.
But right now I'm busy working on some awesome tools to further automate nodejs
packaging, since we're going to need a lot of nodejs-foo packages as
BuildRequires for the various js-foo packages we want to introduce.

-T.C.

[1] https://github.com/jquery/jquery-migrate/

[Dridi Boukelmoune]

Hi,

My 2 cents,

On Mon, Sep 23, 2013 at 2:50 AM, T.C. Hollingsworth
 wrote:
> My plan is to have two parallel installable jquery packages: one for the 2.x
> branch and one for the 1.x branch.  As much as I'd like to skip the latter,
> older IE versions will continue to be the bane of web developers' existence for
> some time to come.  We'll continue to support the 1.x series in Fedora as long
> as jQuery upstream supports it.

JQuery is a library that "often" breaks compatibility (compared to
what I'm used to). Having one version of each branch sounds a bit
restrictive.
> For stuff that needs older jQuery versions, ideally we can patch them to work
> with the latest version.  But for a quick fix, we can get most of them working
> with 1.10 using jquery-migrate [1] with minimal effort.  (All you need is an
> extra <script> tag in most cases.  :-)

This is javascript. The language is too dynamic and permissive for me
to feel comfortable with an automated migration. What if the project
uses some eval magic ? Does it work with both dotted and bracketed
notations ..?
> For stuff that uses really old jQuery versions where even jquery-migrate isn't
> an option (e.g. < 1.6), my first concern is whether upstream is alive in the
> first place.  Because seriously, that stuff is OLD.  We really shouldn't be
> shipping code that was deprecated three years ago!  If upstream is actively
> maintained, then they really need to get with the program.

While I agree with you, I also understand people who don't want to
migrate "just because" when the application is stable. Especially with
a library like jQuery that tends to rapidly deprecate and remove
stuff.
> That being said, our de facto policy regarding bundling is to grandfather old
> packages in, which means they can bundle those ancient versions of jQuery till
> the end of days if that's what they want.  Though I might have to ask FPC if I
> can add Provides: security-nightmare-waiting-to-happen to their spec files.  ;-)
>
> Where migration is possible, I fully intend to have everything unbundled for
> F21, wielding my shiny new provenpackager hammer wherever necessary.
>
> Going forward we'll treat jQuery just like any other library in the
> distribution.  When a new version comes out, ideally we'll migrate everything by
> the next Fedora release, or else provide a parallel-installable backward compat
> package only when absolutely necessary.
>
> If you want to help get ready, I suggest working with your upstreams to migrate
> to jQuery 1.10, using jquery-migrate where necessary.  That'll make our lives
> much easier when we get ready to unbundle.
>
> BTW, I intend to write up a couple of blog posts outlining the general plan and
> a list of tips for packagers who want to get a head start within the next month.
> But right now I'm busy working on some awesome tools to further automate nodejs
> packaging, since we're going to need a lot of nodejs-foo packages as
> BuildRequires for the various js-foo packages we want to introduce.
>
> -T.C.
> [1] https://github.com/jquery/jquery-migrate/
>
> packaging mailing list
> [e-mail]
> https://admin.fedoraproject.org/mailman/listinfo/packaging

[T.C. Hollingsworth]

On Sun, Sep 22, 2013 at 10:57 PM, Dridi Boukelmoune
 wrote:
> Hi,
>
> My 2 cents,
>
> JQuery is a library that "often" breaks compatibility (compared to
> what I'm used to). Having one version of each branch sounds a bit
> restrictive.

It's not as bad as it used to be when jQuery was younger.  It's moving along
much more stably nowadays, and I'm sure upstream will provide migration
strategies like jquery-migrate when/if it becomes necessary in the future.

Again, parallel-installable compat versions are an option where absolutely
necessary, but they're rightfully strongly discouraged.
> This is javascript. The language is too dynamic and permissive for me
> to feel comfortable with an automated migration. What if the project
> uses some eval magic ? Does it work with both dotted and bracketed
> notations ..?

It's not an automated migration, python 2to3-style, if that's what you were
thinking.  It uses the standard jQuery plugin interface to add back the various
interfaces that were removed, and works everywhere jQuery itself works.  This
is JavaScript, you can do a lot of crazy stuff with prototypes.  ;-)

But, keep in mind that also means that it'll happily add back dangerous features
that were removed for good reason, like stuff that makes XSS as easy as PHP used
to make SQL injection. :-P  This is why it's a crutch and porting is strongly
recommended where possible.

Also, I forgot to mention:  you can also add jquery-migrate to any page using
jquery < 1.9 and it'll emit warnings to help you port your code.
> While I agree with you, I also understand people who don't want to
> migrate "just because" when the application is stable. Especially with
> a library like jQuery that tends to rapidly deprecate and remove
> stuff.

This boils down to the whole reason we ban bundled libraries in the first place:
https://fedoraproject.org/wiki/Packaging:No_Bundled_Libraries#Old_Code

Limping along with old code because "it just works" even though it's
unmaintained has never been acceptable in Fedora.  I see no compelling reason
why JS should be any different and lots of good reasons why it should.

To pick on PHP again, there's an example of something that rapidly iterated in
response to security/design flaws in often backwards-incompatible ways.  Nobody
would have ever dreamed about keeping an old version of PHP around just because
some application relied on bad ideas like register_globals.

-T.C.

[T.C. Hollingsworth]

On Thu, Sep 19, 2013 at 2:35 AM, Jens Petersen  wrote:
> TIL that there are around a thousand packages
> in Fedora that include a file named jquery.js... [1]

For added fun, there are about 100 more files named jquery.js in the distro than
there are packages containing that file, so some of those packages have several
copies of it.  Or maybe one has a hundred?  The mind boggles.  O_o
> Anyway it is good to see efforts to tackle these kinds of problems
> with the Javascript and Web Assets Guidelines. :)
>
> I want to ask if the packaging of grunt and jquery
> blocks current package reviews of packages that bundle jquery.js?
> Or can they proceed for now until jquery is actually packaged in Fedora?

Toshio at least seemed to be of the opinion that they have to wait now:
https://lists.fedoraproject.org/pipermail/devel/2013-August/187836.html

Personally, I'd rather put it off until F21 branch time or something like that.
We're not quite ready to kick this off yet (most of the Change got punted off
to F21 thanks to scope creep and me not having time to get what needed to be
done done before change freeze).

Honestly, a few more copies of jquery.js really aren't going to kill us at this
point.  As long as it's not 100 in one package, anyway.  ;-)

I'll try and catch FPC during their meeting in a few hours and see if they have
an opinion on this if I can, but I'm not sure I can be on IRC at the time.  Feel
free to bring it up there yourself if you'd like.
> Jens
>
> ps It is going to be a long cleanup process to fix all those packages with bundled files.

Sure is.  :-/

-T.C.

[Orion Poplawski]

On 09/19/2013 05:52 AM, T.C. Hollingsworth wrote:
> On Thu, Sep 19, 2013 at 2:35 AM, Jens Petersen  wrote:
> > TIL that there are around a thousand packages
> > in Fedora that include a file named jquery.js... [1]
>
> For added fun, there are about 100 more files named jquery.js in the distro than
> there are packages containing that file, so some of those packages have several
> copies of it.  Or maybe one has a hundred?  The mind boggles.  O_o
> > Anyway it is good to see efforts to tackle these kinds of problems
> > with the Javascript and Web Assets Guidelines. :)
> >
> > I want to ask if the packaging of grunt and jquery
> > blocks current package reviews of packages that bundle jquery.js?
> > Or can they proceed for now until jquery is actually packaged in Fedora?
>
> Toshio at least seemed to be of the opinion that they have to wait now:
> https://lists.fedoraproject.org/pipermail/devel/2013-August/187836.html
>
> Personally, I'd rather put it off until F21 branch time or something like that.
> We're not quite ready to kick this off yet (most of the Change got punted off
> to F21 thanks to scope creep and me not having time to get what needed to be
> done done before change freeze).
>
> Honestly, a few more copies of jquery.js really aren't going to kill us at this
> point.  As long as it's not 100 in one package, anyway.  ;-)
>
> I'll try and catch FPC during their meeting in a few hours and see if they have
> an opinion on this if I can, but I'm not sure I can be on IRC at the time.  Feel
> free to bring it up there yourself if you'd like.
> > Jens
> >
> > ps It is going to be a long cleanup process to fix all those packages with bundled files.
>
> Sure is.  :-/
> -T.C.
>
> packaging mailing list
> [e-mail]
> https://admin.fedoraproject.org/mailman/listinfo/packaging

Are we any farther on this now?  I've got a new review with query
(https://bugzilla.redhat.com/show_bug.cgi?id=734248)