nokogiri vs mini-portile

[Aleksandar Kostadinov]

Hello,

I understood recently that we modify some gem files in fedora in a 
non-compatible way [1].

Namely nokogiri is stripped off mini-portile which is not needed for 
building in Fedora. Unfortunately this change also affects ability of an 
app with Gemfile.lock to be installed in a system different from the 
development system. e.g. build Gemfile.lock on Fedora 26 but installed 
in some other OS.

IMO this is similar to forking the upstream package mgmt system and puts 
developers on fedora into surprising situations.

My proposal is to stop doing such modifications to gem files. We can 
ship mini-portile and still build against system libraries. This will 
allow seamless `bundler` package management no matter where Gemfile.lock 
was created.

If we find that some upstream gem is not well packaged, then we should 
rather work with the upstream to have that fixed. This would align with 
the usual strategy of Red Hat and Fedora to avoid shipping non-upstream 
patches (unless they are critical importance).

Regards,
Aleksandar

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1482641

[Vít Ondruch]

Hi Aleksandar,

You touched just tip of the iceberg unfortunately. This is not just
about nokogiri vs mini-portile. In this case, it would lead just to
installation of build-time package (useless for binary distribution),
but there are other similar cases I can think of:

1) Platform specific dependencies, such as in rubygem-listen:

https://src.fedoraproject.org/rpms/rubygem-listen/blob/master/f/rubygem-list...

there is now just "rb-fsevent" dependency, which is mac specific
library, but in previous versions of listen, there used to be dependency
on "rb-kqueue" which is BSD specific stuff and there used to be
dependency even on some Windows library. We don't want to have useles
libraries on Fedora and install useless dependencies.

2) Again from rubygem-listen:

https://src.fedoraproject.org/rpms/rubygem-listen/blob/master/f/rubygem-list...

This depends on ruby_dep. Again, this not concern of Fedora. On specific
Fedora is specific version of Ruby and what version of Ruby is available
on Fedora is not concern of any application running on Fedora.

Actually look at the naive code:

https://github.com/e2/ruby_dep/blob/master/lib/ruby_dep/ruby_version.rb#L27

It says something about some version of Ruby, but it does not every
version and it cannot know if our version of Ruby is vulnerable or not,
unless you are going to suggest at the time we should stop patching
vulnerabilities in Ruby.

3) And lets take look at rubygem-sdoc. This package changes dependency
on json:

https://src.fedoraproject.org/rpms/rubygem-sdoc/blob/master/f/rubygem-sdoc.s...

This was even accepted upstream, but was not released yet. If you are
using upstream sdoc, you would be forcing Fedora to ship old vulnerable
version of json. Would you like that?

All in all, on these three examples I tried to explain you that changes
in our packages are inevitable. We are not doing them lightheartedly and
I don't think we are happy about them. Also I hope that every time, they
were brought upstream and some of theme were resolved while the other
were not. But they are really necessary sometimes.

To your original issue, I'll personally be unhappy if anything changes
in rubygem-nokogiri in regard to miniportile ...

Vít

Dne 9.11.2017 v 12:55 Aleksandar Kostadinov napsal(a):
> Hello,
>
> I understood recently that we modify some gem files in fedora in a
> non-compatible way [1].
>
> Namely nokogiri is stripped off mini-portile which is not needed for
> building in Fedora. Unfortunately this change also affects ability of
> an app with Gemfile.lock to be installed in a system different from
> the development system. e.g. build Gemfile.lock on Fedora 26 but
> installed in some other OS.
>
> IMO this is similar to forking the upstream package mgmt system and
> puts developers on fedora into surprising situations.
>
> My proposal is to stop doing such modifications to gem files. We can
> ship mini-portile and still build against system libraries. This will
> allow seamless `bundler` package management no matter where
> Gemfile.lock was created.
>
> If we find that some upstream gem is not well packaged, then we should
> rather work with the upstream to have that fixed. This would align
> with the usual strategy of Red Hat and Fedora to avoid shipping
> non-upstream patches (unless they are critical importance).
>
> Regards,
> Aleksandar
>
> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1482641
> _______________________________________________
> ruby-sig mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]

[Vít Ondruch]

Actually there is 4th case.

4) Take look at rubygem-fog:

https://src.fedoraproject.org/rpms/rubygem-fog/blob/master/f/rubygem-fog.spe...

There are removed several dependencies because they are not available on
Fedora yet. And in fact, nobody has missed them. At least not enough to
open ticket demanding them ...

Vít

Dne 9.11.2017 v 15:05 Vít Ondruch napsal(a):
> Hi Aleksandar,
>
> You touched just tip of the iceberg unfortunately. This is not just
> about nokogiri vs mini-portile. In this case, it would lead just to
> installation of build-time package (useless for binary distribution),
> but there are other similar cases I can think of:
> Platform specific dependencies, such as in rubygem-listen:
>
> https://src.fedoraproject.org/rpms/rubygem-listen/blob/master/f/rubygem-list...
>
> there is now just "rb-fsevent" dependency, which is mac specific
> library, but in previous versions of listen, there used to be dependency
> on "rb-kqueue" which is BSD specific stuff and there used to be
> dependency even on some Windows library. We don't want to have useles
> libraries on Fedora and install useless dependencies.
> Again from rubygem-listen:
>
> https://src.fedoraproject.org/rpms/rubygem-listen/blob/master/f/rubygem-list...
>
> This depends on ruby_dep. Again, this not concern of Fedora. On specific
> Fedora is specific version of Ruby and what version of Ruby is available
> on Fedora is not concern of any application running on Fedora.
>
> Actually look at the naive code:
>
> https://github.com/e2/ruby_dep/blob/master/lib/ruby_dep/ruby_version.rb#L27
>
> It says something about some version of Ruby, but it does not every
> version and it cannot know if our version of Ruby is vulnerable or not,
> unless you are going to suggest at the time we should stop patching
> vulnerabilities in Ruby.
> And lets take look at rubygem-sdoc. This package changes dependency
>
> on json:
>
> https://src.fedoraproject.org/rpms/rubygem-sdoc/blob/master/f/rubygem-sdoc.s...
>
> This was even accepted upstream, but was not released yet. If you are
> using upstream sdoc, you would be forcing Fedora to ship old vulnerable
> version of json. Would you like that?
>
> All in all, on these three examples I tried to explain you that changes
> in our packages are inevitable. We are not doing them lightheartedly and
> I don't think we are happy about them. Also I hope that every time, they
> were brought upstream and some of theme were resolved while the other
> were not. But they are really necessary sometimes.
>
> To your original issue, I'll personally be unhappy if anything changes
> in rubygem-nokogiri in regard to miniportile ...
>
> Vít
>
> Dne 9.11.2017 v 12:55 Aleksandar Kostadinov napsal(a):
> > Hello,
> >
> > I understood recently that we modify some gem files in fedora in a
> > non-compatible way [1].
> >
> > Namely nokogiri is stripped off mini-portile which is not needed for
> > building in Fedora. Unfortunately this change also affects ability of
> > an app with Gemfile.lock to be installed in a system different from
> > the development system. e.g. build Gemfile.lock on Fedora 26 but
> > installed in some other OS.
> >
> > IMO this is similar to forking the upstream package mgmt system and
> > puts developers on fedora into surprising situations.
> >
> > My proposal is to stop doing such modifications to gem files. We can
> > ship mini-portile and still build against system libraries. This will
> > allow seamless `bundler` package management no matter where
> > Gemfile.lock was created.
> >
> > If we find that some upstream gem is not well packaged, then we should
> > rather work with the upstream to have that fixed. This would align
> > with the usual strategy of Red Hat and Fedora to avoid shipping
> > non-upstream patches (unless they are critical importance).
> >
> > Regards,
> > Aleksandar
> >
> > [1] https://bugzilla.redhat.com/show_bug.cgi?id=1482641
> > _______________________________________________
> > ruby-sig mailing list -- [e-mail]
> > To unsubscribe send an email to [e-mail]
>
> ruby-sig mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]

[Aleksandar Kostadinov]

I've read over the internet the recommendation to not use fedora 
packaged deps and go for the upstream ones to avoid issues. I don't have 
a wide view over the ecosystem but provided I can't rely on 
compatibility of my Gemfile.lock, then I'd rather also use upstream deps 
too.

This way packaged fedora libs become less relevant. There are very few 
people that would file a bug report of any kind. And nobody will persist 
when project is unwilling to accept certain changes.

So I don't think nobody missed the removed deps. Probably there are very 
few people trying to use the packaged gems and being motivated to report 
issues after they figure out upstream bits work already.

To make it clear, I'm not against compatible modifications. I'm against 
incompatible modifications where we claim that the gem is with name "x" 
and version "y" but in fact this is some modification of the actual gem.

It is like installing RPM with same name and version on different 
machines and getting different incompatible bits.

I thought that idea behind Fedora is to be leading edge dev platform. 
Then I develop an simple rails app, try to deploy it on OpenShift and 
that fails (OpenShift has rhscl as a base for the ruby s2i image).

If we want to disable certain libraries that don't apply to our 
platform, then it makes sense to add them as some kind of mocks. They 
would be listed and still useless code will not be distributed.

Or we could patch our bundler implementation + rubygem implementation to 
ignore certain deps and still include them in Gemfile.lock.

Maybe an approach like above will also break some use cases, hopefully 
some minor ones only.

At the end though, if I can't rely on my (simple) app to install on 
anything but *same Fedora version*, then packaged deps are of little use 
to me. Even worse for projects with multiple collaborators who very 
often use different OS, at least different versions.

Did I get wrong what use cases we try to cover in Fedora?
Did I get wrong who are our target users?
Is there any other supported way to develop ruby apps under fedora to 
keep them compatible with non-exactly-the-same-fedora-version?

Vít Ondruch wrote on 11/09/17 16:15:
> Actually there is 4th case.
> Take look at rubygem-fog:
>
> https://src.fedoraproject.org/rpms/rubygem-fog/blob/master/f/rubygem-fog.spe...
>
> There are removed several dependencies because they are not available on
> Fedora yet. And in fact, nobody has missed them. At least not enough to
> open ticket demanding them ...
>
> Vít
>
> Dne 9.11.2017 v 15:05 Vít Ondruch napsal(a):
> > Hi Aleksandar,
> >
> > You touched just tip of the iceberg unfortunately. This is not just
> > about nokogiri vs mini-portile. In this case, it would lead just to
> > installation of build-time package (useless for binary distribution),
> > but there are other similar cases I can think of:
> > Platform specific dependencies, such as in rubygem-listen:
> >
> > https://src.fedoraproject.org/rpms/rubygem-listen/blob/master/f/rubygem-list...
> >
> > there is now just "rb-fsevent" dependency, which is mac specific
> > library, but in previous versions of listen, there used to be dependency
> > on "rb-kqueue" which is BSD specific stuff and there used to be
> > dependency even on some Windows library. We don't want to have useles
> > libraries on Fedora and install useless dependencies.
> > Again from rubygem-listen:
> >
> > https://src.fedoraproject.org/rpms/rubygem-listen/blob/master/f/rubygem-list...
> >
> > This depends on ruby_dep. Again, this not concern of Fedora. On specific
> > Fedora is specific version of Ruby and what version of Ruby is available
> > on Fedora is not concern of any application running on Fedora.
> >
> > Actually look at the naive code:
> >
> > https://github.com/e2/ruby_dep/blob/master/lib/ruby_dep/ruby_version.rb#L27
> >
> > It says something about some version of Ruby, but it does not every
> > version and it cannot know if our version of Ruby is vulnerable or not,
> > unless you are going to suggest at the time we should stop patching
> > vulnerabilities in Ruby.
> > And lets take look at rubygem-sdoc. This package changes dependency
> >
> > on json:
> >
> > https://src.fedoraproject.org/rpms/rubygem-sdoc/blob/master/f/rubygem-sdoc.s...
> >
> > This was even accepted upstream, but was not released yet. If you are
> > using upstream sdoc, you would be forcing Fedora to ship old vulnerable
> > version of json. Would you like that?
> >
> > All in all, on these three examples I tried to explain you that changes
> > in our packages are inevitable. We are not doing them lightheartedly and
> > I don't think we are happy about them. Also I hope that every time, they
> > were brought upstream and some of theme were resolved while the other
> > were not. But they are really necessary sometimes.
> >
> > To your original issue, I'll personally be unhappy if anything changes
> > in rubygem-nokogiri in regard to miniportile ...
> >
> > Vít
> >
> > Dne 9.11.2017 v 12:55 Aleksandar Kostadinov napsal(a):
> > > Hello,
> > >
> > > I understood recently that we modify some gem files in fedora in a
> > > non-compatible way [1].
> > >
> > > Namely nokogiri is stripped off mini-portile which is not needed for
> > > building in Fedora. Unfortunately this change also affects ability of
> > > an app with Gemfile.lock to be installed in a system different from
> > > the development system. e.g. build Gemfile.lock on Fedora 26 but
> > > installed in some other OS.
> > >
> > > IMO this is similar to forking the upstream package mgmt system and
> > > puts developers on fedora into surprising situations.
> > >
> > > My proposal is to stop doing such modifications to gem files. We can
> > > ship mini-portile and still build against system libraries. This will
> > > allow seamless `bundler` package management no matter where
> > > Gemfile.lock was created.
> > >
> > > If we find that some upstream gem is not well packaged, then we should
> > > rather work with the upstream to have that fixed. This would align
> > > with the usual strategy of Red Hat and Fedora to avoid shipping
> > > non-upstream patches (unless they are critical importance).
> > >
> > > Regards,
> > > Aleksandar
> > >
> > > [1] https://bugzilla.redhat.com/show_bug.cgi?id=1482641
> > > _______________________________________________
> > > ruby-sig mailing list -- [e-mail]
> > > To unsubscribe send an email to [e-mail]
> >
> > ruby-sig mailing list -- [e-mail]
> > To unsubscribe send an email to [e-mail]
>
> ruby-sig mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]