Gemfile.lock questions

Jason Guiditta jason.guiditta at gmail.com
Tue Apr 24 01:26:48 UTC 2012


On 19/04/12 08:02 +0200, Vít Ondruch wrote:
>Hi Ken,
>
>Dne 19.4.2012 03:43, Ken Dreyer napsal(a):
>>I have two more Ruby newbie questions :)
>>
>>I see a lot of Gems contain a "Gemfile.lock" file. From looking at the
>>conventions in Fedora's packages, it's a good idea to not include the
>>Gemfile.lock in RPMs. I'm trying to understand how Gemfile.lock
>>relates to packaging.
>>
>>General question: Is Gemfile.lock only intended to be a convenience
>>for Ruby Gem developers? I thought Gemfile.lock was only for
>>development, but it seems like its contents can also affect an
>>application's runtime...?
>
>Regarding the Gemfile.lock, I would suggest you to read [1].
>
>Otherwise, there is not straight forward answer. Rails applications 
>definitely uses Bundler, which by default reads the Gemfile.lock. For 
>other applications, it depends. My suggestion is not to hardly depend 
>on Bundler, i.e. do not use "require 'bundler/setup'" or similar in 
>your app. You can always run "bundle exec yourapp" instead.
>

I can see why you would come to this conclusion/suggestion, but I have
another thought on how to approach this, below.  My concern here is if
fedora (and other distros) start having to make such changes to
libraries, there is going to both be a higher risk of error or
something not being loaded properly, and higher cost of maintenance
for the packager.

>>
>>Packaging question: In preparation for packaging up Gitorious, I've
>>found that I need to delete some entries from Gemfile.lock in order to
>>use alternate Gem versions. Is it always safe to override the version
>>numbers in Gemfile.lock like this?
>
>I would suggest to remove Bundler dependency at all. Bundler will 
>always use/create the Gemfile.lock. If you provide it by yourself and 
>some component will get updated, the application fail. If you don't 
>provide it, the application will try to create it but this will (at 
>least should ;) fail, due to limited rights. The version resolution 
>which Bundler provides is not needed, since the dependencies are 
>determined by RPM (and occasionally they may differ from original 
>dependencies in upstream). The only thing you will miss without 
>Bundler is initialized and restricted environment, e.g. at the start 
>of the application, Bundler reads the Gemfile.lock and place 
>immediately all the gems into Ruby's load path and there is not 
>possible to load other gem from your application in the future.
>
>I hope that somebody from Aeolus will chime in with their experience 
>with Bundler and how they use it.
>
Here I am :)  Sorry for the delay, was off for a couple days.  So, I
have been trying different approaches to this over the last $time, and
have had varying levels of sucess thus far.  As others have said, the
main problem is bundler and rpm/yum have very different views of the
world.  Unlike others on this list. however, I do not see the bundler
view as incorrect.  In fact, I think it really makes a lot of sense,
as it guarantees the same _full dependency chain_ for a given app or
library.  If you install one 5 different machines at any different
time, so long as you have the same Gemfile.lock, you will have the
same versions of everything your app needs.  The place where I think
bundler falls down is lacking a built-in way to support our use case
for rpm, deb, or or package managed systems that do not use purely
rubygems.

So, first a quick summary of what we are doing _right this second_ on
aeolus[1], then a map of where we plan to go with it shortly.  Right
now, in simple form, we just plain don't package our
Gemfile/Gemfile.lock into the rpm.  If rails/cucumber/rspec don't see
a file, they just work as you would expect.  Unfortuntely, part of
what you have to expect for rails apps, is that it then doesn't know
what libraries your app needs if there is no Gemfile to refer to.  For
now, we have gotten around this by adding a conditional block in our
application.rb, which duplicates the libraries listed in the Gemfile.
Obviously, this is far from ideal, but does work, and keep packaging
for rpms simpler, at the cost of making it harder for upstream ruby
developers, and easier to miss adding a dependency in all the correct
spots.

This was always intended to be a temporary solution.  It has seemed to
me from the beginning that we should be using the same Gemfile whether
we choose to have bundler involved or not, and that there must be some
hook that we could use to make this happen.  After much poking and
thinking, I came up with something I think could potentially work not
only for us, but all of ruby on fedora (and by extension, and system
that relies on a package manager of some kind).  Our next step doesn't
quite get us to that point, but the second phase should do it.  I ran
this by someone from openshift (another red hat cloud project), and
they had come to some similar conclusions, so I am hopeful of
convincing more of the community, and eventually the bundler upstream,
of buying into this idea.

Phase one is simply getting rid of our duplication of dependencies
between the Gemfile and application.rb by extending bundler with a
Bundler::Ext.system_require method.  This simple levergaes some
bundler internals to parse the Gemfile and read in the deps there
without creating or touching the lock file.  In order to work, this
step requires we continue to not have a Gemfile in the rpm, so we will
go with something more like Gemfile.in.

Phase two expands this concept by making the core of bundler itself
aware of an environent variable (let's call it USE_BUNDLER for now),
which if set, causes it to not attempt to create or read a lock file,
and to just load the dependeincies based on whatever the system has
installed.  This is meant _only_ for pure rpm/deb style systems where
the gem can reasonably be expected to only have one version installed,
otherwise you are back to the situation bundler solves for us.  I
realize this sounds incredibly simple, and it is for the most part.
If we can get upstream bundler to accept that not everyone who uses
bundler is deploying things in the scenario they were attempting to
solve, we may be able to coax it into working for everyone.  The
remaining detail I see is where to set this env variable system-wide,
so it will not only work for an app like ours (which can easily set it
in the initscript), but for _any_ app or lib on the system.  I am
thinking /etc/$something, but vit might have some more ideas there
once we get that far (and assuming he were to get on board with this
plan).

Long as this email was, I realize I may not have provided all the
detail needed, so feel free to follow up with questions on this line
of thinking, and maybe we can finally solve this thing once and for
all.

-j

>
>Note that there was already some discussion about Bundler and 
>packaging guidelines [2], but at the end, we did not found sound 
>statement how to handle it correctly. So if you succeed with Gitorius, 
>your solution might become standard ;)
>
>Vit
>
>
>
>[1] http://yehudakatz.com/2011/05/30/gem-versioning-and-bundler-doing-it-right/
>[2] http://lists.fedoraproject.org/pipermail/ruby-sig/2012-January/000759.html
>_______________________________________________
>ruby-sig mailing list
>ruby-sig at lists.fedoraproject.org
>https://admin.fedoraproject.org/mailman/listinfo/ruby-sig


More information about the ruby-sig mailing list