New packaging guidelines for Ruby
by Vít Ondruch
Hi,
since I was not subscribed to this mailing list, I starting new thread
in Hope to move forward with the packaging guidelines. There are still
some outstanding issues.
= Mandatory rebuilding gems =
Yes, Ruby SIG is still against it, since there is known just one gem ATM
which needs such treatment. Now I list several pros/cons:
Pros:
* It would allow ruby packages to follow the same steps as other packages.
Cons:
* More overhead for maintainers.
* More confusion for new-commers, since this approach is not know in
Ruby community and there is no best way how to achieve it.
* There is only one known gem in Fedora, which needs this treatment ATM.
* If you need patch binary part of gem, it is sign that the gem is not
well maintained by upstream, otherwise it would not be needed.
= Vendorlib =
It is not good idea to move vendorlib out of the Ruby directory
structure. Actually it is pair of directories, vendorlib and
vendorarchlib. These directories are typically used by libraries Ruby
bindings, such as geos, subversion, etc. This platform dependent
bindings has no meaning to other Ruby implementations, such as JRuby.
= ruby(name) vs rubygems(name) =
Although we want to see as much libraries as possible provided in a gem
form, there still be some libraries which are not gemified. However,
Gemifies library *should not* always provide also ruby(name) virtual
provide, since these are not always simply interchangeable. Gem caries
with itself its metadata. When gem is loaded to Ruby's library path,
this metadata are processed as well and it might put some other
dependent libraries to Ruby's library path as well. In contrary, the
ruby(name) provide will mean that it is not gemified library, so I would
prefer to stay with distinction between ruby(name) and rubygem(name).
Gem would provide the ruby(name) only in case it obsoletes previously
available ruby library.
These are 3 concerns I remember was the most controversial. Please feel
free to share your thoughts.
Vít
12 years, 1 month
Re: [Fedora-packaging] New packaging guidelines for Ruby
by Vít Ondruch
Dne 28.2.2012 18:36, Michal Fojtik napsal(a):
> On Feb 28, 2012, at 6:25 PM, Vít Ondruch wrote:
>
>> Dne 28.2.2012 18:05, Rex Dieter napsal(a):
>>> On 02/28/2012 10:56 AM, Vít Ondruch wrote:
>>>
>>>> All three versions provide the same output, unless I did some mistake,
>>>> since I did not tested it (actually the middle one was taken directly
>>>> from rubygem-POpen4.spec). So which version you prefer? Please note that
>>>> the "gem install" will always "unpack" the gem with some additional, for
>>>> our case unimportant, steps. We do not distribute the .gem file anywhere.
>>> You didn't exactly (directly) answer my question. Pretend I don't know much about ruby... (not far from the truth).
>>>
>>> So, for rubygem packages that include native (C or otherwise code), how and when is this compiled? If it is always done so during either version of these guidelines, please do accept my apologies for being ignorant.
>> It is compiled during the "gem install" step. So "gem install" is doing %prep, %build, %install in one step. So yes, it is always done for either version of guidelines.
>>
>>> However, you do say you do not distribute the .gem, though I'm curious why this seem to contradict you:
>>>
>>> rpm -qlp rubygem-POpen4-0.1.4-3.fc17.noarch.rpm
>>> /usr/share/gems/cache/POpen4-0.1.4.gem
>>> ...
>>>
>> I knew you will point it out :).
>>
>> It is cached version of the original gem, which RubyGems may use to restore the gem into its original state (and may be other unknown purposes). However, it is not used in runtime, nor it is good idea to restore the gems maintained by RPM by gem command. Moreover, even though the gem would not be available in the cache dir, RubyGems will download it. Hence we add new clause into the packaging guidelines:
> gem help commands
>
> pristine Restores installed gems to pristine condition from files
> located in the gem cache
> unpack Unpack an installed gem to the current directory
>
> AFAIK this is the reason gem files are kept in cache dir. I barely understand how 'pristine' command
> could be dangerous to RPM (personally I think it can be useful when you make some dirty modification
> to installed gem directly as root (trying to debug/solve some issue)).
>
> If the case is that we don't want to keep these files on filesystem, then these two commands should
> be 'patched' out from the 'gem' command or some decent warning to user should be provided.
Since the gems managed by RPM are no longer in GEM_HOME, neither the
"gem pristine" nor "gem unpack" can touch them. If you want to
reinstall RPM nanaged gem, you should use something like "yum reinstall
rubygem-foo" or alternatively "rpm -i --force rubygem-foo.rpm".
"gem pristine" and "gem unpack" will work no matter if the .gem file is
kept in cache or not. In the rare case you need them, the .gem files
will be downloaded from rubygems.org. That is a beauty of cache, you
can loose it and it will not hurt.
Vit
12 years, 1 month
Scripting language dependencies
by Jon Masters
Greetings,
I hope everyone is having a good day. Here in the Fedora ARM project,
we're having a great time overall, but we've run up against a familiar
foe in our preparation for Fedora 17 and rawhide: superfluous deps in
SPEC files. Today's example is a shell-escaped call to Ruby just to
determine some gcc flags for an unrelated package build, but that's just
today's example. There are many others, so some standard would help.
Why is this a problem? The most obvious issue is the superfluous
dependencies, pulling in ruby (and all of its deps), for example. But
the bigger problem for the distribution as a whole is that we are
becoming less bootstrappable[0] over time, and we are suffering from
dependency creep that makes life difficult for secondary architectures
(or even for the primary architecture if a bunch of rebuilds were to
fail and there were no deps to satisfy a circular path).
Eventually, the issue of dependency creep is going to bite the broader
distribution. In the meantime, can we please impose some kind of rule
surrounding which superfluous build deps are allowed? I'm not advocating
for the removal of Ruby, Lua, and so on and a return to old fashioned
shell, but I am suggesting that we cap where we are and try to avoid
introducing any more bizzare build dependencies in the future. For
example, say a package grew a SPEC file need for a call to an INTERCAL
script just to determine gcc flags? It's not really all that different
from calling Ruby to expand gcc flags and maybe I happen to like using
INTERCAL over Ruby? Or why not have a hardware compiled state machine
implement some custom logic to determine gcc flags? Maybe I find that
easier than writing some good old fashioned shell, etc.
In all seriousness, can we please have some direction? If this is
already implied or codified, could someone remind packagers to refrain
from adding build dependencies like this? I'd recommend that, in
general, if a one line shell escape can be used, that ought to be the
preference over adding a one-line build dep on $random scripting lang.
Thanks!
Jon.
[0] Rebuilding the entire distribution from first principles, using
nothing more than a toolchain to begin the process. We did this for
Fedora 15 on ARMv7. I eventually would like it that Fedora be able to
bootstrap itself on any architecture trivially, and I know Debian (and
other distributions) are similarly keen to return to that world.
12 years, 1 month
Systemd conversion trigger question
by Jason L Tibbitts III
This came up on IRC the other day, and as I'm finishing up the
conversion of my packages I realize that I don't know the proper answer.
Packages converting to systemd are mandated by
https://fedoraproject.org/wiki/Packaging:ScriptletSnippets#Systemd to
include a trigger:
%triggerun -- zoneminder < 1.25.0-4
Assume that F16 is at 1.25.0-3.fc16 and F17 is at 1.25.0-4.fc17
Now, there's a caveat that any update to F16 keep a ver-rel less than
1.25.0-4. Which is fine if you just need to tweak the package, since
you can use 1.25.0-3.fc16.1.
I don't, however, recall any discussion about what the packager is
supposed to do if 1.25.1 comes out (and for fun, assume it fixes a huge
security hole and is absolutely essential). What does F16 get? The
options seem to me to be:
* Just update F16 to 1.25.1-1.fc16 and deal with the fact that the
trigger doesn't get run.
* Update to 1.25.1 but lie and call it 1.25.0 in the rpm version.
* Freeze the F16 version at 1.25.0 and ignore bug reports.
The only thing that makes any sense to me (perhaps because this whole
trigger and systemd-sysv-convert thing seems entirely pointless) is to
just do the first option. Is there anything else to be done? I suppose
it should be theoretically possible to do the second option and encode
the real package version in the release somehow
(1.25.0-3.fc16.1.1.25.1.fc16 or something crazy like that) but that
seems even less palatable than just dropping the trigger and requiring
admins to do systemctl enable zoneminder.service. Which, not
coincidentally, is something they pretty much have to do anyway.
- J<
12 years, 1 month
Guideline change request: make PIE 'must' instead of 'should'
by Bill Nottingham
https://fedorahosted.org/fpc/ticket/144
The guidelines currently read:
...
If your package meets the following critera you should consider enabling the
PIE compiler flags:
Your package is long running. This means it's likely to be started and
keep running until the machine is rebooted, not start on demand and quit on
idle.
Your package has suid binaries, or binaries with capabilities.
Your package runs as root.
Your package accepts/processes untrusted input.
...
I'd like to change this from 'should' to 'must'. I do not believe there are
convincing performance reasons that we have found that would require not
using it for packages that fit this criteria. The only concern would be
programs that don't work as PIE ... those can be treated as exceptions.
Comments?
Bill
12 years, 1 month
Re: [Fedora-packaging] Packaging guidelines - mandatory rebuilding gems
by Toshio Kuratomi
Sending to packaging(a)lists.fp.o where this belongs.
On Tue, Feb 14, 2012 at 05:27:06PM -0500, Mo Morsi wrote:
>
> Thoughts:
>
> - I am also not a fan of the current approach of always repackaging the gem,
> seems like extra steps / added complexity which is unneeded, what is the
> justification for it? In the majority of cases, gem install works out of the
> box and the gem doesn't need any modifications.
>
There's a lot of justification in the ticket. There's various pieces.
* The historical ruby/gem guidelines are a huge hack that FPC approved because
it seemed that gems wouldn't work with the normal separation of %prep
%build %install. The recent draft showed that this is no longer the case.
* Proper separation allows rpmbuild -b[pci] --shortcircuit to work properly.
* The previous guidelines actually didn't allow patching. The original
proposed draft guidelines ended up being overly complex as they had
separate cases for gem, gem w/ C extension, patched gem, patched gem w/
C extension, patched gem w/ C extension that fails to build. This
strategy has a single way to handle all of this. The cost is three extra
lines which are easy to conceptualize and explain. (unpacking the code
from an archive, then assembling the code into a form that can be built
from).
* Proper separation of the steps allows people new to ruby packaging but old
hands at rpm packaging (a large portion of reviewers) to come to
understand the basics of packaging rubygems.
> - what is the rationality behind having a gem package provide ruby
> (RUBYLIBRARY)? Seems like the non-gem and gem package distinction is more
> explicit when the gems provide rubygem(gemname) and non-gems provide ruby
> (libraryname). Since the dependency needs to be represented anyways in any
> other packages that depend on the gem, an explicit dependency on rubygem
> (gemname) might not be a bad idea.
>
In the old guidelines, rubygems were distinct from plain ruby packages.
This was because at the time they were written:
require 'rubylibrary'
imported a plain ruby library and:
gem_require 'rubylibrary'
The had wholly different syntax at the source code level. The piece of
vondruch's draft that stated that non-gem subpackages were no longer needed
for gem packages lead me to find that the rubygems package now sets up the
ruby library path so that "require 'rubylibrary'" works for both gems and
nongems. With that change, we don't need to package all ruby libraries in
a non-gem version and the need to have separate requires and provides
lessens. (The caveat here is that ruby code would need to have require
'rubygems' somewhere early in its code but if that's considered sufficiently
compatible for one of these cases, then it's sufficient for both cases)
When might you absolutely need to use Require: rubygem()? I know of the
specific case when you are requiring a specific version of code via::
gem 'rubylibrary'
I also imagine that there's ways that ruby code can make use of the metadata
that gems provide but I don't know how to detect those. In either of those
cases, the rubygem() require would seem to be most appropriate.
When might you want to use the Require ruby() version? When a package's
code uses plain "require 'rubylibrary'", it would work whether the rpm
package is shipping a rubygem or a plain ruby library. Therefore, if the
packager dosn't want to guess whether we're shipping the relevant library as
a rubygem or as a non-gem ruby library, they should be able to do a simple
"Requires: ruby(library)". If the packager knows that we package that
pacticular library as a rubygem, they can use the rubygem() version of the
requires instead but they are no longer required to in this case.
So I think answering your question, whether the dependency is satisfied by
a gem or non-gem library is no longer of interest in the simple case of
a script or piece of code using "require 'rubylibrary'". For places where
it does matter, the Guideline should require using Requires: rubygem() (if
you have a list of other things to look for, that could be added to flesh
out that section).
> - Binary Extensions - gem install then recompile doesn't make alot of sense
> when the other alternative, gem unpack, patch, rebuild, and them install would
> work in both cases. Why is this split out like this?
>
I'm not sure I understand this. Are you referring to the draft guideline
that vondruch originally proposed with the many separate pages or the
changes that I have been making to have only a single case? The single case
does the gem unpack, patch, rebuild (via gem install), and finally install
via mkdir/cp/mv.
I've forked this to this page:
https://fedoraproject.org/wiki/User:Toshio/RubyPackagingDraft
for now so that people can better see what that proposal looks like. I'll
have to keep the two pages in sync for a bit until someone does or does not
bring up a rubygem where the reduced-complexity guidelines are as direly
broken as people are claiming.
> - the rspec package will soon finally be updated to rspec2 so the BR:
> rspec-core in the test guidelines can be changes to a BR: rspec
Nice. Can you specify what versions of Fedora that will be operable on?
It's nice to note things like that in the Guidelines.
>
>
> Answers to specific questions expressed in the draft:
>
> - Interpreter independence - seems reasonable, how we address supporting these
> libraries on multiple interpreters needs to be flushed out, abliet not
> necessarily for this release as time is short
>
> - Move text about interpreter independence to here - seems reasonable to me
>
So -- vondruch seems to have issues with this. Code doesn't seem to be
entirely interpreter independent? Or rather, only certain code is? The FPC
really needs to understand this portion of the issue better as it affects
things like where the directories belong in the FHS, whether it makes sense
to share rubygems across interpreters and whether it makes sense to share
vendorlib or vendorarch across interpreters. Example srpms are the best way
to make clear what's happening here :-).
Note that when you talk about this being possibly something for later
releases -- unfortunately, that seems to be an underlying assumption of the
draft guidelines that we were presented with so we need to be able to get
correct the parts that make sense.
> - Confirm change (remove ruby(rubygems) dep) - also seems reasonable to me
>
I've finalized this.
> - Give examples? and Do we want to tell what the arguments to gem install do? -
> both would be appreciated
>
heh :-) Unfortunately, I don't believe we've had a ruby person on the FPC
since lutter (who came up with the original guidelines). So if examples and
explanations of arguments are desired, I'll need to recieve them from people
who use ruby.
> - Replacement instructions - as discussed not a huge fan of this
>
Unless someone can present examples where this doesn't work that aren't
flaws with upstream code, this is likely to become the way to do things.
> - Library placement - the bit about 'foo' is somewhat confusing, perhaps
> replacing it with something like "[ext|lib]" or similar would work instead?
>
Yeah -- this was in the portion of the guidelines to be replaced. In my
draft, I've simply removed that section (as it's already in the previous
section where I renamed it $REQUIRE_PATHS with a comment about where
$REQUIRE_PATHS comes from)
-Toshio
12 years, 1 month
Time for Lua packaging guidelines?
by Michel Alexandre Salim
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi all,
As we start accumulating more Lua packages in our repository, is it
perhaps time to start working on Lua packaging guidelines (similar to
one for Python, Ocaml etc.).
There are ongoing reviews for lutok and datalog, in particular, where
a guideline on package naming would be useful. A rule saying "use lua-
prefix unless package starts with lu / lua", similar to what we have
for Python, would cover lutok, for example (but not datalog).
Has anyone started on something like this? Searching the wiki for Lua
does not turn up any draft packaging guidelines. I can start one if
people are interested.
Thanks,
- --
Michel Alexandre Salim
Fedora Project Contributor: http://fedoraproject.org/
Email: salimma(a)fedoraproject.org | GPG key ID: A36A937A
Jabber: hircus(a)jabber.ccc.de | IRC: hircus(a)irc.freenode.net
() ascii ribbon campaign - against html e-mail
/\ www.asciiribbon.org - against proprietary attachments
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQEcBAEBAgAGBQJPKpfsAAoJEEr1VKujapN6eV0H/1dCY3OSI8u6C/okk/EPgKrJ
y4JqG6FEsdvVBr/jTBpiX23xvLRvP/U598TWNFYhSUR05FYzAwy7D4/RQsaiuHQH
arG/9YDhWFHRBx3rmfPeUr3CzgLTkMz/7PpU6fQ5EQrZ9OBbOqb3kLQKvITQqMtm
iWndeASfyUaql+K4Xue8J+3HL+Sql1cJUxyiObqof7CEE7PEHRe9iyb/Z21VVgvC
LwvfpQSx3vbwPFi8THNoN/JeoNuPPE2U4naAXtF74xuUQUrFDHm6SNnaqUKfMRt9
priS70KiOX7N0Z103pcE42cUvTVspUwK02EfalCCHEfJRGnuQFrnf+G2w4//vPM=
=t8eN
-----END PGP SIGNATURE-----
12 years, 2 months
