[Fedora-packaging] purpose of ruby(abi), python(abi), etc

Tue Jan 8 09:17:31 UTC 2013

Dne 7.1.2013 19:58, Toshio Kuratomi napsal(a):
> On Mon, Jan 07, 2013 at 03:33:58AM -0500, Bohuslav Kabrda wrote:
>>      Second, when you talk about minor feature enhancements, that includes the
>>      things that I mentioned? (Changes to the ruby language, api additions,
>>      deletions, and changes to the ruby stdlib, changes to the filesystem layout
>>      that have visibility to other ruby packages or ruby users).  Is it policy
>>      that those types of changes are allowed or do they just slip in once in a
>>      while because no one realized that a particular change would have that sort
>>      of effect?
>>
>> Well, when I say "abi", I mean the C library, but when I say "api", I mean the
>> pure Ruby language, which can be interpreted by any Ruby interpreter. I think
>> we need to distinguish these two - ruby(api) for pure Ruby libraries and ruby
>> (abi) or jruby(???) for those that contain an extension.
>> As for the second point, AFAIK they "slip once in a while".
>>
> [snip]
>>      Okay.  That's not how fedora traditionally used these virtual provides as
>>      that would be redundant with the requires on the shared library.  The
>>      virtual provides were used on the things that I mentioned instead (it
>>      sounds like those things might be called "compat modes" by ruby people.  Is
>>      that correct?)
>>
>> Yep, I think so.
>>
> So problem 1)  The Ruby SIG finds ruby(abi) hard to understand.  We can
> simply rename it to ruby(compat-mode) to resolve this.

We are thinking about following virtual provide names:

ruby(release)
ruby(version)
ruby(compat)
ruby(compat-mode)

Currently, we prefer to call this virtual provide ruby(release), which 
fits best IMO. I wouldn't go with any "compat" in virtual provide name, 
since it might be confused with -compat packages.

>    There's no need to
> make a new ruby(abi) virtual provide for the C extensions because the
> automatic library dep finding should take care of that.

Exactly.

>    Requires
> a guideline update (discussing here) and a mass rebuild of the packages that
> provide and require ruby(abi).

Yes, we will need mass rebuild anyway, even if we just change ruby(abi) 
version. Moreover, we are going to do minor adjustment of RubyGems 
filesystem layout for further compatibility with alternative Ruby 
implementations (mainly targeted at Rubinius ATM)

>
> The mass rebuild should be doable at this point in the cycle.  If we
> run into issues deciding on the rest of the proposal we should decide on
> whether to make this rename of virtual provides separately (and first) so
> that it can go into a mass rebuild.  (mass rebuilds are usually done several
> weeks (up to a month) before alpha).

We are planing to ask rel-eng for separate buildroot tag. This would 
ease the revert in case things go awry

> Now on to the harder parts of the proposal...
>
>>      > JRuby always supports certain version(s) - compat modes - of Ruby. Right
>>      now, it supports 1.9 and 1.8 (and new features from next version of cRuby
>>      continuously land in the latest compat mode, so e.g. the 1.9 compat mode of
>>      JRuby may be in fact a bit different from cRuby 1.9).
>>
>>      This sounds like a version of a compat mode isn't finalized until the last
>>      cruby release of that version is released?
>>
>> What do you mean when saying "finalized"? If you mean that e.g. features from
>> 1.8.x to 1.8.y land continuously in the JRuby's 1.8 compat mode, then the
>> answer to your question is yes.
>>
> With "finalized" I was talking about cRuby but I may have misunderstood what
> you wrote earlier (with "new features from the next version of cRuby").
> I was trying to find out if cRuby continued to add things to the MAJOR.MINOR
> compat mode after the release of the first MAJOR.MINOR:
>
> For cRuby MAJOR.MINOR (example: 1.8), is it cRuby policy that 1.8.1,
> 1.8.2, 1.8.3, etc do not have changes that affect compat mode?  No new
> features or changes to existing ones that mean that code written on 1.8.0
> will/will not run on 1.8.3?  What about the opposite: is there policy about
> adding and changing features so that code written on 1.8.3 will run on
> 1.8.0?  Is this policy frequently broken by accidents or rarely?

Let me explain a bit.

1.8.7, 1.9.0, 1.9.1, 1.9.2, 1.9.3, 2.0.0, all these are major versions. 
These major versions are typically released every 18 months. They may or 
may not break things. However, you cannot know that from version. What 
is clear is that patch releases should not break anything (although for 
example fixing CVE-2011-4815 [1] broke some compatibility, due to change 
in natural ordering of hashes, but that was always wrong assumption). 
They are typically safe to update.

For example, as you probably know, the ABI was kept compatible in 
between 1.9.1..1.9.3 versions where there will be ABI breakage for Ruby 
2.0.0. However, in between 1.9.1 and 1.9.2, there was polished Unicode 
support, which of course might break something. It depends how lucky you 
are. For example, between 1.9.3 and 2.0.0, there is going to be change 
in default encoding of Ruby scripts from US-ASCII to UTF-8. I am not 
sure if you can foresee every possible issue with this change, I cannot.

To conclude, there is not yet know what will be next version and what it 
will break. This will be known later, when its development begins. And 
it will be clearly declared and communicated [2].

I can assure you that there is no unnoticed feature creep.

>
> If I re-read what you wrote to apply to jRuby instead of cRuby, then I have
> another question:  how do compat modes line up with jRuby releases?  Does
> jRuby only ship releases when a compat mode is 100%?  (It sounds like the
> answer is no since you mention that no release of jRuby support fork(), for
> instance).  Is there some compat-mode-completion criteria for shipping
> a release of jRuby?

There does not exist nothing like 100% compatibility, and that is not 
just about fork, but about C Ruby extensions and so on. The JRuby 
compatibility is based on best effort. I.e. it should work in most of 
the cases and if it does not work in your case, JRuby team will try hard 
to solve the issues.

The compatibility of Ruby implementations is assured/measured by 
RubySpec project [3]. The JRuby is trying hard to use original 
unmodified cRuby standard library, this can give you overview how 
compatible these implementations are.

There is also expected that for example you will start to develop you 
Ruby on Rails application using cRuby but you will run it by JRuby in 
production.

>
> If we don't have 100% compatibility the present structure of virtual
> proivides is not well adapted to a shared compat-mode virtual provide.
> (Similar dangers exist in python where, for instance pypy is close to
> compatible with python2 but has a few corner cases:
> http://pypy.readthedocs.org/en/latest/cpython_differences.html ).  I don't
> believe we have precedent as to "how compatible" something would need to be
> to share this type of virtual provide but in general, the right thing to do
> is to err on the side of caution in order to protect the end user.

Obviously, there is no universal answer to compatibility nor what 
interpreter is the best. Therefore we want to allow to use every Ruby 
implementation in the world as easy as possible, providing sensible 
defaults to those who don't care.

> Debian has a method that they use for their python stack that is pretty
> complex and a little hacky that might be usable.  It requires packagers to
> manually mark their package as being compatible with certain versions of
> python and then the needed module files are symlinked into the library paths
> for the compatible python versions when the package is installed.  Something
> along these lines is a lot more work to setup but may be another
> possibility.

The problem with this approach is that even if the code is compatible 
with certain version/implementation and maintainer does not provide the 
symlinks, it will not work. We would like to go the opposite way, be 
proactive. I.e. we expect that everything works and if not, either fix 
it to really work or disable it to not be broken.

Vít

>
> -Toshio
>

[1] https://bugzilla.redhat.com/show_bug.cgi?id=CVE-2011-4815
[2] http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/50443
[3] http://rubyspec.org/