Hello all!
I've copied Tom Enebo on this reply...he may want to get on the list too.
From: "Bohuslav Kabrda" bkabrda@redhat.com
Hi all, since we have MRI Ruby integrated pretty nicely into Fedora, I started working on JRuby's new upcoming release (1.7) - I'm planning to put it into Fedora 19.
Excellent!
Work to do: Most importantly, I'd like to work on integrating JRuby's RubyGems with our system RubyGems, so that JRuby doesn't include (at least in Fedora) its own slightly modified copy. My idea (please shout if you don't like it!) is to hook JRuby into our RubyGems concept [4]:
- All the behaviour mentioned in [4] will stay.
- JRuby and MRI will load non-platform-specific Gems from the same locations, only the extensions will be placed in different directories.
If this is possible, I don't see an issue. JRuby currently "supports" C extensions, but we're deprecating that...so in general the only gems you'll get on JRuby are non-extensions (no platform specified in the gem) and -java gems (containing compiled classes or jar files; e.g. nokogiri, hpricot, json).
As long as you can have nokogiri-java and nokogiri C ext installed without JRuby *ever* loading the C ext, this should be fine.
How to achieve that:
- Ideally push JRuby's changes to RubyGems upstream (not sure if they'd accept them), or just apply them to our Fedora RubyGems downstream for the time being - shouldn't break anything, AFAICS
We have a standing bug to get all of our changes pushed upstream. Most of them are just one-off patches, but there's also our maven support (which, I'm sad to say, never really worked like we wanted it to). Ideally we could ship stock RubyGems for JRuby 1.7 final.
- Make JRuby work with our custom operating_system.rb [5] - this would probably require the JRuby's RbConfig::CONFIG to somehow change it's values closer to MRI's
File an issue and we can look into it. We largely simulate RbConfig since we don't actually have a per-platform build time to populate it.
- Figure out the packaging changes around it:
-- Naming Gems that are only for JRuby
Any gem that's -java platform will only work on JRuby. There may be non -java platform gems that use JRuby-specific features (like Java integration) too, however.
-- Placement of JRuby's extensions -- Creating packages for Gems that have extensions for both JRuby and MRI -- How to work with ruby(abi) virtual provides - both implementations should have them, but yum currently resolves virtual provides "randomly" (it chooses the provider that has less dependencies, if I'm correct)
So this is basically allowing JRuby to "provide" "ruby" as a dependency, right? Yeah ideally we should be able to do that, but of course the C extension thing makes it hard for us to replace them everywhere (and the Java extension thing makes it hard for them to replace us everywhere.
I'm cc'ing Charles Nutter in hope that he will join this discussion on ruby-sig :)
I'm on the list now :)
FYI, the JRuby 1.7 schedule has us doing a final release around the end of this month. Ideally we'd have everything we need changed wrapped up ASAP.
- Charlie
Couple more details for y'all.
On Tue, Sep 11, 2012 at 8:02 PM, Charles Oliver Nutter headius@headius.com wrote:
- Ideally push JRuby's changes to RubyGems upstream (not sure if they'd accept them), or just apply them to our Fedora RubyGems downstream for the time being - shouldn't break anything, AFAICS
We have a standing bug to get all of our changes pushed upstream. Most of them are just one-off patches, but there's also our maven support (which, I'm sad to say, never really worked like we wanted it to). Ideally we could ship stock RubyGems for JRuby 1.7 final.
We maintain a forked git repo for our RubyGems patches. Everything is handled as branches that correspond to the main RubyGems' repo's tags, e.g. the v1.8.24 tag/release led to the following branch in JRuby's fork:
https://github.com/jruby/rubygems/tree/jruby-rubygems_1_8_24
All the changes should be diffable and blameable for easy investigation.
We are debating just removing the maven support, since it has never really worked quite right.
- Charlie
Dne 12.9.2012 03:02, Charles Oliver Nutter napsal(a):
- Figure out the packaging changes around it:
-- Naming Gems that are only for JRuby
Any gem that's -java platform will only work on JRuby. There may be non -java platform gems that use JRuby-specific features (like Java integration) too, however.
This was more or less about conventions for package naming in Fedora, I guess.
Currently, every rubygem has the rubygem- prefix, e.g. for nokogiri, the rubygem-nokogiri is the name of gem in Fedora. However, if we are going to support more Ruby implementations, we will need to ship the versions for MRI and JRuby and may be others. This means that (unless we decide to keep everything in one package) we need to distinguish between packages which are for MRI and packages for JRuby. There also might be packages, which are useful just for JRuby.
I think, that for the platform specific bits, we should use subpackages and somehow follow the RubyGems conventions, i.e. use -java for JRuby packages. This is clear. Unfortunately, I am not sure how to distinguish between C extensions for MRI and Rubinius for example (neither I really explored, what are the possibilities, but I am afraid, that it will not be that easy :/).
Vit
On Wed, Sep 12, 2012 at 2:27 AM, Vít Ondruch vondruch@redhat.com wrote:
I think, that for the platform specific bits, we should use subpackages and somehow follow the RubyGems conventions, i.e. use -java for JRuby packages. This is clear. Unfortunately, I am not sure how to distinguish between C extensions for MRI and Rubinius for example (neither I really explored, what are the possibilities, but I am afraid, that it will not be that easy :/).
My understanding is that the *sources* of the extensions should be the same for both MRI and Rubinius, but the *binaries* are definitely different. For example, a number of things MRI defines as macros directly into MRI-specific struct layout are actually functions in Rubinius...so an extension build for one definitely will not work on the other.
- Charlie
Dne 12.9.2012 09:54, Charles Oliver Nutter napsal(a):
On Wed, Sep 12, 2012 at 2:27 AM, Vít Ondruch vondruch@redhat.com wrote:
I think, that for the platform specific bits, we should use subpackages and somehow follow the RubyGems conventions, i.e. use -java for JRuby packages. This is clear. Unfortunately, I am not sure how to distinguish between C extensions for MRI and Rubinius for example (neither I really explored, what are the possibilities, but I am afraid, that it will not be that easy :/).
My understanding is that the *sources* of the extensions should be the same for both MRI and Rubinius, but the *binaries* are definitely different. For example, a number of things MRI defines as macros directly into MRI-specific struct layout are actually functions in Rubinius...so an extension build for one definitely will not work on the other.
Exactly.
What I am afraid that RubyGems can't distinguish between gems for Rubinius and MRI as it can't differentiate between binary gems built for MRI 1.8 or MRI 1.9. For example, for therubyracer gem [1], you can get prebuilt gems, but I doubt they can work for Ruby 1.8 and 1.9 as well. There are "fat binary" gems for Windows (e.g. nokogiri), but that is more or less workaround, which doesn't scale. RubyGems should support more than platform to choose the correct gem version.
Vit
[1] https://rubygems.org/gems/therubyracer
- Charlie
ruby-sig mailing list ruby-sig@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/ruby-sig
Hi Charles, thanks for getting in loop :)
----- Original Message -----
Hello all!
I've copied Tom Enebo on this reply...he may want to get on the list too.
From: "Bohuslav Kabrda" bkabrda@redhat.com
Hi all, since we have MRI Ruby integrated pretty nicely into Fedora, I started working on JRuby's new upcoming release (1.7) - I'm planning to put it into Fedora 19.
Excellent!
Work to do: Most importantly, I'd like to work on integrating JRuby's RubyGems with our system RubyGems, so that JRuby doesn't include (at least in Fedora) its own slightly modified copy. My idea (please shout if you don't like it!) is to hook JRuby into our RubyGems concept [4]:
- All the behaviour mentioned in [4] will stay.
- JRuby and MRI will load non-platform-specific Gems from the same
locations, only the extensions will be placed in different directories.
If this is possible, I don't see an issue. JRuby currently "supports" C extensions, but we're deprecating that...so in general the only gems you'll get on JRuby are non-extensions (no platform specified in the gem) and -java gems (containing compiled classes or jar files; e.g. nokogiri, hpricot, json).
As long as you can have nokogiri-java and nokogiri C ext installed without JRuby *ever* loading the C ext, this should be fine.
Yep, that is the intention.
How to achieve that:
- Ideally push JRuby's changes to RubyGems upstream (not sure if
they'd accept them), or just apply them to our Fedora RubyGems downstream for the time being - shouldn't break anything, AFAICS
We have a standing bug to get all of our changes pushed upstream. Most of them are just one-off patches, but there's also our maven support (which, I'm sad to say, never really worked like we wanted it to). Ideally we could ship stock RubyGems for JRuby 1.7 final.
Great, cool, superb :) That is exactly what Fedora needs. I will definitely try how that will work with our system rubygems, once that is done.
- Make JRuby work with our custom operating_system.rb [5] - this
would probably require the JRuby's RbConfig::CONFIG to somehow change it's values closer to MRI's
File an issue and we can look into it. We largely simulate RbConfig since we don't actually have a per-platform build time to populate it.
I did: [1], with a patch, too :)
- Figure out the packaging changes around it:
-- Naming Gems that are only for JRuby
Any gem that's -java platform will only work on JRuby. There may be non -java platform gems that use JRuby-specific features (like Java integration) too, however.
As Vit has mentioned, this is basically packaging/naming issue. Still, we will have to be very careful using same gems for MRI and JRuby.
-- Placement of JRuby's extensions -- Creating packages for Gems that have extensions for both JRuby and MRI -- How to work with ruby(abi) virtual provides - both implementations should have them, but yum currently resolves virtual provides "randomly" (it chooses the provider that has less dependencies, if I'm correct)
So this is basically allowing JRuby to "provide" "ruby" as a dependency, right? Yeah ideally we should be able to do that, but of course the C extension thing makes it hard for us to replace them everywhere (and the Java extension thing makes it hard for them to replace us everywhere.
I'm cc'ing Charles Nutter in hope that he will join this discussion on ruby-sig :)
I'm on the list now :)
FYI, the JRuby 1.7 schedule has us doing a final release around the end of this month. Ideally we'd have everything we need changed wrapped up ASAP.
- Charlie
I've been working on and experimenting with the path-rework patch for last two days. As I mentioned, using the default rubygems and doing the path rework as proposed in [1] should do most of the work. Then we will probably have to adjust our custom operating_system.rb and that should be all.
Thanks for joining the discussion and welcome to Fedora Ruby-sig.
Slavek.
On Thu, Sep 13, 2012 at 2:16 AM, Bohuslav Kabrda bkabrda@redhat.com wrote:
We have a standing bug to get all of our changes pushed upstream. Most of them are just one-off patches, but there's also our maven support (which, I'm sad to say, never really worked like we wanted it to). Ideally we could ship stock RubyGems for JRuby 1.7 final.
Great, cool, superb :) That is exactly what Fedora needs. I will definitely try how that will work with our system rubygems, once that is done.
We have opted to remove Maven support from our fork of RubyGems, which left only two diffs from upstream. I've submitted pull requests for both of them; the second could use a little cleanup and love, probably.
https://github.com/rubygems/rubygems/pull/370 https://github.com/rubygems/rubygems/pull/371
I've pushed the Maven removal to our fork. It will be nice to finally ditch the fork soon.
- Charlie
ruby-sig@lists.fedoraproject.org
-
Bohuslav Kabrda -
Charles Oliver Nutter -
Vít Ondruch