[Fedora-packaging] New packaging guidelines for Ruby

Vít Ondruch vondruch at redhat.com
Fri Mar 2 08:40:41 UTC 2012


Toshio,

This discussion begins to be tiring. I'll go last time through this 
topic trying to explain every concern you risen in your answer and I 
hope not to be very personal :)

Yes, I agree with you that #3 would be perfect in ideal world, but since 
we are not living in ideal world, I prefer #2. Since I am maintaining 
more than 80 rubygems out of 290 currently available in Rawhide, since I 
work with Ruby almost exclusively for more than 5 years, since the 
guidelines draft was gone through review of Ruby-SIG members, I believe 
you could respect more this opinion.

Dne 1.3.2012 17:21, Toshio Kuratomi napsal(a):
> On Thu, Mar 01, 2012 at 10:05:58AM +0100, Vít Ondruch wrote:
>> Dne 29.2.2012 17:53, Toshio Kuratomi napsal(a):
>>
>>
>>      I've been thinking about this for a week and also waiting for people to
>>      bring forth packages that would be broken by it to see if there's any actual
>>      problems with the strategy.  No packages beyond the original one which is
>>      a bug from upstream has been presented.  Without that sort of example, I'm
>>      going to vote for #3.
>>
>>
>> Well it is hard to bring you some example, because there are non.
>>
> If there are no problems, then there's no reason not to use it.

There are no problem with approach #3, since nobody uses it. I am not 
gonna to convert any of my packages just to prove you that you are 
wrong. Prove me that you are right, that rubygem-idn is the only 
problematic package and I'll believe you.

>
>> There are no
>> experiences with the #3 except with rubygem-idn, and in this case the simple
>> approach described in guidelines fails. Try to remove the following lines from
>> the spec:
>>
>> # Avoid "cert_chain must not be nil" error.
>> sed -i -e "10d" %{gem_name}.gemspec
>>
>> and you'll see what I am talking about. Please do not ignore these custom lines
>> I had to come up with to let the #3 work.
>>
> I have not ignored this.  In the ticket, I stated that this looks like an
> upstream bug which, as packagers, it's our job to detect and fix.
> cert_chain is being set to nil by the upstream packaging.  This is not
> supported by the current versions of the build tools (rubygems).  We update
> so the build script to work with the current version of the tools (and
> hopefully submit the change to upstream).
>
> When gcc updates and code stops working, we patch the upstream code to work
> with the new gcc.  When python updates, we patch upstream code of third
> party modules so that they work.  There's nothing out of the ordinary with
> this at all.
>
>>
>>      #2 is more complex as people have to keep in mind two sets of instructions.
>>      When a bug fix is needed, it will not be simple to apply it as you'd have to
>>      change the whole structure of the spec file in order to apply the patch and
>>      then revert those changes when the patch is no longer needed.  The
>>      temptation for the packager in that situation will be to not apply bug fixes
>>      until after they've made it into an upstream gem release.
>>
>>
>> Yes, it will not be simple to apply changes, therefore we are moving from
>> approach #1 which was valid up until now and proposing the approach #2, which
>> works well for all packages except rubygem-idn
>>
> No.  You are proposing approach #2 plus approach #3.  So your proposal is
> twice as complex as approach #3 as there's twice as many recipes for
> building rubygems to learn.

You don't have to care about #3 as long as you don't need it. That is 
the beauty. You are learning gradually. We agreed that from 290 rubygems 
in Fedora you need #3 for one case, yet everybody should know it.

Nobody who comes from Ruby land and want to package his gem for Fedora 
knows anything about "gem spec" command. Yes, that is nice that you want 
to learn them this command right from beginning, but may be they will go 
away, since they would be confused what is going on.

>
>>
>>      #2 does not fix the problem with rpmbuild --short-circuit.
>>
>>
>> To be honest, I do not understand what is the point with the --short-circuit.
>> According to documentation "--short-circuit skip straight to specified stage
>> (only for c,i)". I do not know how are you using it, but it implies that you
>> have to do "rpmbuild -bp" to be able to do "rpmbuild -bc --short-circuit". In
>> that case, what does not work for you? What do you want to see in this step if
>> it is empty?
>>
> rpmbuild -bp kernel.spec
> cd kernel-3.0.0
> # edit some source files, making backups w/ extension .myfix
> cd ..
> rpmbuild -bc --short-circuit kernel.spec
> # If failure, edit source some more
> # If success,: gendiff kernel-3.0.0 .myfix>  myfix.patch and add it to spec
> # file

Yes, that is good example. I never used it though and will never used it 
even if we went with #3.

It is not useful for RubyGems, since I would never build/install the gem 
just to try if binary extension builds. I would go to ext dir and fired 
"make" (optionally preceded by "ruby extconf.rb"). Never ever "rpmbuild 
-bc --shortcircuit".

>
>>
>>      #3 was compared in the ticket with requiring "Fedora users to always rebuild
>>      each RPM locally, because there might be sometimes broken dependency or
>>      other error."  This is a wrong comparison.  A packager's job is to deal with
>>      broken dependencies and other errors, not a users.  #3 requires packagers to
>>      deal with these issues, not users.
>>
>>
>> You should understand that RubyGems is packaging system and if I am packaging
>> the gem for Fedora, I am user of the packaging system from point of view of
>> RubyGems and I am package maintainer form point of RPM. No need to argue about
>> "packager's job". I just wanted to find something what would give you better
>> feeling what is it about and I gave better, more detailed, example along the
>> lines somewhere in this thread to Rex.
>>
> Yes, you are the user of the upstream build system.  But it's not
> upstream's build system that the end users of Fedora see.  So the comparison
> that drags in end users is a wrong comparison.

No, it is not. I don't need to know more about RubyGems then necessary 
(although it might be beneficial). If you go to package first gem for 
Fedora, you are RubyGems user at best and may be you are not RubyGems 
user at all. You just want to have something packaged.

>
> Upstream build systems are sometimes easily adaptable to the way that
> software should be packaged in rpms but at other times it is something that
> we have to work around.  Take a look at packaging java code where upstream
> has bundled multiple versions of jar files just for their package build as
> an example of something even more invasive than what we're talking about
> here.

Yes, we are unbundling other gems from gems, or even Java packages, but 
that is different topic. And it has nothing to do with packaging system.

>
>>      I'm also seeing the argument made for #2 that it's less work because
>>      a single command exists to do all the steps.  Therefore we should use it.
>>      This is not a compelling argument because the same can be said of other
>>      build systems.  For most python code, for instance:
>>
>>      %setup -n %{srcname}-%{version}
>>      python setup.py install --root %{buildroot}
>>
>>      even "make install" is valid under this argument.
>>
>>
>> I am not familiar with python, but you cannot compare "make install" with "gem
>> install". It is something really different. Moreover, "make install" doesn't
>> necessarily mean that the "configure&&  make" will be run before.
> The argument I'm seeing is that "Upstream provides me with a command to run
> that upstream has told me to use to build and install the package.  It does
> that in one step instead of spread over three steps like I want.  I could
> do it a different way using the standard tools and then it will do things in
> the proper three steps but I want to do what upstream has asked instead of
> what is proper."  For that argument, the same can indeed be said about "make
> install".

I was forced by you to learn something about packaging of Python 
libraries and there is really nothing in common with RubyGems. If you 
want to compare gem, you must compare it with RPM, not with *tarball*, 
typically used by Python libraries.

There is no "--skip-build" flag available for "gem install" command 
which would allow us simple move to #3.

>
>
>>      #3 is also (barring someone giving us examples where packages are actually
>>      broken) only a small one-time cost.  Changing the rubygem spec files that
>>      are using the old guidelines to use the new guidelines.  As stated earlier,
>>      the FPC has always known that the Rubygem guidelines needed to be changed
>>      but was unaware that the time had come when the old guidelines were no
>>      longer needed (lutter hasn't been on the FPC for years).  Since you're
>>      proposing changes to the guidelines anyway, you can hardly complain about
>>      a one-time cost.  In terms of an ongoing cost, until you show that there's
>>      a non-bug reason that unpacking the source and rebuilding the gem is wrong,
>>      there's no extra cost in maintaining this that package maintainers shouldn't
>>      be responsible for anyhow.
>>
>>
>> Yes, you see just script which do conversion, but you do not count the
>> exceptions. Once more, I used #3 for rubygem-idn, the only gem which required
>> it and in this one case it immediately failed. So for me it is a *100% failure*
>> of your conversion script.
> I've told you earlier why this is not an impressive statistic.  Even if
> *all* rubygem packages were to fail because they're using features of
> rubygems that have been removed in the current version of rubygems, it would
> be beneficial that we had detected the problem and submitted patches to the
> builds to upstream.  So in a way the fact that this bug in rubygem-idn would
> have passed unnoticed if you didn't have to fix something else is a failure
> in your proposed guidelines as it's not being detected until unrelated
> changes are made.

Beneficial for whom? Upstream will care about their .gemspec file in 
repository they are using to build the gem. They will care about their 
rake taks which can build gem, because they are using them. But they 
will not care about any .gemspec provided by "gem spec" command.

  You do not understand that during the life of the library, there is 
several forms of .gemspec. There is probably the upstream .gemspec file 
(but not necessarily, it could be generated from code), which is 
actually executable Ruby code. Then there is some metadata in .gem file, 
however at this point, it was converted to YAML, hence the the "gem 
spec" command without additional parameters provide YAML output. Even if 
you used the "--ruby" flag, the .gemspec is already different then the 
original file used by upstream. After installing the gem by "gem 
install", You will get in the gems/specifications third form of the 
.gemspec file.

>
>>  From last versions of packaging guidelines we moved from #1 to #2 which solves
>> practical problems. Move from #2 to #3 solves some artificial problems.
>>
> That is incorrect.  #2 is incomplete by itself.  You must include #3 in
> order to have a workable guideline.  OTOH, #3 is complete in and of itself.
> So leaving off #2 halves the number of specfile variations they need to know
> if they want to package only rubygems.

I never said that #2 is complete. I said that it is practical and 
satisfactory in majority of cases. #3 is exceptional case and should be 
handled as all other exceptions. If you want some stats, it is likely 
1/290, i.e. 0.35 % chance that you will need #3 for speaking about 
current Rawhide, hence it is exception.

>
>> Yes, I agree with you that if differs a bit from other packages. Yes, I agree
>> with you that it could be better. Yes, I proposed that I will work with
>> upstream to allow better way of patching C extensions, without need to move to
>> #3. Neither of this seems to satisfy you.
>>
> That is correct.  A workflow that looks like #3 is the correct way to
> package software.  You are not proposing to work with upstream to make that
> possible but instead working with upstream to kludge their current broken
> process to do one more thing (patching C extensions) in a broken manner.

Oh my, how you can say that other packaging system has broken process? 
They had different design goals and I am sure that their design goal was 
not compatibility with RPM nor mimic RPM. If it was so broken, then 
nobody would used it, but the reality is opposite. Everybody is 
wondering why not to stick with rubygems, why use RPM. And I see that 
sometimes it would be really easier.

>
> The proposed #3 is the workflow that the package build should have.  If
> you're unsatisfied with that, you should be talking to upstream about how to
> adapt their build system to meet that workflow in all cases.

The proposed #3 is the workflow RPM has and RubyGems does not have. 
There is nothing wrong with that.

>
>> Please note that this [1] was the last proposal coming from me and Ruby-SIG. I
>> am not against formal changes of the guidelines, as was done by Toshio, if FPC
>> believes they will be better aligned with other guidelines. I never said
>> nothing against. I totally support that effort. When I was asked to come up
>> with the way how to patch the C extensions and I did that. However I am
>> strongly against mandatory #3.
>>
> But you have given no examples that show why it would be wrong.  And without
> that, you're just spreading fear that a mandatory #3 will cause you issues.

Come on, you gave no example that it will work for majority of cases, so 
how you can propose it??? If Ruby-SIG believed it is that important, it 
would be already mandatory in the proposal.

>
> If you're so sure that you can get close to a 100% failure rate out of #3
> that is due to problems with the procedure rather than bugs in upstream
> packaging, you should be able to pick a random rubygem, adapt it to #3 and
> then expose some new type of bug that we can examine and determine why it's
> unworkable.  Failing to do that, the argument that it won't work is just
> speculation, not fact.

As same as from your side. Please don't try to manipulate us.


Vit


>
> -Toshio
>
>
> --
> packaging mailing list
> packaging at lists.fedoraproject.org
> https://admin.fedoraproject.org/mailman/listinfo/packaging

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.fedoraproject.org/pipermail/ruby-sig/attachments/20120302/982af062/attachment-0001.html>


More information about the ruby-sig mailing list