[Fedora-packaging] purpose of ruby(abi), python(abi), etc

[Vít Ondruch]

Dne 9.1.2013 01:05, Toshio Kuratomi napsal(a):
> On Tue, Jan 08, 2013 at 10:17:31AM +0100, Vít Ondruch wrote:
>> Dne 7.1.2013 19:58, Toshio Kuratomi napsal(a):
>>
>>> I was trying to find out if cRuby continued to add things to the MAJOR.MINOR
>>> compat mode after the release of the first MAJOR.MINOR:
>>>
>>> For cRuby MAJOR.MINOR (example: 1.8), is it cRuby policy that 1.8.1,
>>> 1.8.2, 1.8.3, etc do not have changes that affect compat mode?  No new
>>> features or changes to existing ones that mean that code written on 1.8.0
>>> will/will not run on 1.8.3?  What about the opposite: is there policy about
>>> adding and changing features so that code written on 1.8.3 will run on
>>> 1.8.0?  Is this policy frequently broken by accidents or rarely?
>> Let me explain a bit.
>>
>> 1.8.7, 1.9.0, 1.9.1, 1.9.2, 1.9.3, 2.0.0, all these are major
>> versions. These major versions are typically released every 18
>> months. They may or may not break things. However, you cannot know
>> that from version. What is clear is that patch releases should not
>> break anything (although for example fixing CVE-2011-4815 [1] broke
>> some compatibility, due to change in natural ordering of hashes, but
>> that was always wrong assumption). They are typically safe to update.
>>
> Okay.  So this is why the rub(abi) virtual provide has a version of
> MAJOR.MINOR.MICRO.  Thre's no knowing by the version number whether a MICRO
> version changed things relevant to the compat-mode or not so the virtual
> provide has to go to that level of detail.
>
>> For example, as you probably know, the ABI was kept compatible in
>> between 1.9.1..1.9.3 versions where there will be ABI breakage for
>> Ruby 2.0.0.
> To remain clear here, when you say ABI, are you talking about the elf
> library abi or the compat-mode criteria  (ruby's stdlib, keywords, and other
> features discussed earlier)?  I'm reading it as elf library abi but I want
> to be sure since we aren't talking about elf library abi in the rest of this
> discussion so I'm not sure if I'm misinterpreting or there's a reason for
> bringing it up that I'm not seeing.

I am always talking about elf library ABI.

>> However, in between 1.9.1 and 1.9.2, there was polished
>> Unicode support, which of course might break something. It depends
>> how lucky you are. For example, between 1.9.3 and 2.0.0, there is
>> going to be change in default encoding of Ruby scripts from US-ASCII
>> to UTF-8. I am not sure if you can foresee every possible issue with
>> this change, I cannot.
>>
>> To conclude, there is not yet know what will be next version and what
>> it will break. This will be known later, when its development begins.
>> And it will be clearly declared and communicated [2].
>>
>> I can assure you that there is no unnoticed feature creep.
>>
> I'm only incidentally looking for unnoticed compat-mode relevant changes
> Although, some of my questions were to know if you thought that upstream was
> on top of that, I'd certainly hope that you were on top of it even if they
> weren't.  The real question, though is where the compat-mode relevant
> changes are entering in and how that affects the version.  From your
> message, it sounds like compat-mode relevant changes can enter in at any
> time (in relation to the version numbers that we're given).

once in 18 months is not exactly anytime ;)

>
>>> If I re-read what you wrote to apply to jRuby instead of cRuby, then I have
>>> another question:  how do compat modes line up with jRuby releases?  Does
>>> jRuby only ship releases when a compat mode is 100%?  (It sounds like the
>>> answer is no since you mention that no release of jRuby support fork(), for
>>> instance).  Is there some compat-mode-completion criteria for shipping
>>> a release of jRuby?
>> There does not exist nothing like 100% compatibility, and that is not
>> just about fork, but about C Ruby extensions and so on. The JRuby
>> compatibility is based on best effort. I.e. it should work in most of
>> the cases and if it does not work in your case, JRuby team will try
>> hard to solve the issues.
>>
>> The compatibility of Ruby implementations is assured/measured by
>> RubySpec project [3]. The JRuby is trying hard to use original
>> unmodified cRuby standard library, this can give you overview how
>> compatible these implementations are.
>>
>> There is also expected that for example you will start to develop you
>> Ruby on Rails application using cRuby but you will run it by JRuby in
>> production.
>>
>>
>>> If we don't have 100% compatibility the present structure of virtual
>>> proivides is not well adapted to a shared compat-mode virtual provide.
>>> (Similar dangers exist in python where, for instance pypy is close to
>>> compatible with python2 but has a few corner cases:
>>> http://pypy.readthedocs.org/en/latest/cpython_differences.html  ).  I don't
>>> believe we have precedent as to "how compatible" something would need to be
>>> to share this type of virtual provide but in general, the right thing to do
>>> is to err on the side of caution in order to protect the end user.
>> Obviously, there is no universal answer to compatibility nor what
>> interpreter is the best. Therefore we want to allow to use every Ruby
>> implementation in the world as easy as possible, providing sensible
>> defaults to those who don't care.
>>
>>> Debian has a method that they use for their python stack that is pretty
>>> complex and a little hacky that might be usable.  It requires packagers to
>>> manually mark their package as being compatible with certain versions of
>>> python and then the needed module files are symlinked into the library paths
>>> for the compatible python versions when the package is installed.  Something
>>> along these lines is a lot more work to setup but may be another
>>> possibility.
>> The problem with this approach is that even if the code is compatible
>> with certain version/implementation and maintainer does not provide
>> the symlinks, it will not work. We would like to go the opposite way,
>> be proactive. I.e. we expect that everything works and if not, either
>> fix it to really work or disable it to not be broken.
>>
>  From what I'm reading so far, I don't think that your approach is going to
> be able to pass as a guideline update.  You essentially are asking to remove
> the version from the virtual provide and instead let all packages containing
> ruby code to merely specify that they need "Ruby"... not ruby-1.8.1 vs
> ruby-2.0.0, etc.  This is too loose a coupling for proper package management
> as it allows users to install incompatible combinations of packages because
> the dependencies aren't specified precisely enough.

Not exactly. The virtual provide should be versioned. So the ruby-libs 
should provide:

Provides: ruby(release) = 2.0.0

But the require side would do nonversioned Requires: ruby(release). If 
there would be some reason, e.g. library which is working just with some 
specific version of Ruby, it could use Requires: ruby(release) >= 2.0.

JRuby 1.7 could then provide:

Provides: ruby(release) = 1.9
Provides: ruby(release) = 1.8

To claim compatibility with older releases.



BTW, it is interesting that rpmlint complains about multiple virtual 
provides with different versions:

ruby-libs.x86_64: E: useless-provides ruby(release)
This package provides 2 times the same capacity. It should only provide it
once.

while it works as expected in reality.

> OTOH, although all the guidelines I looked at do tell you how to require the
> langugage itself, not all of them make it mandatory:
>
> Language  Virtual Provide                           Required/Optional
> java      java >= specific_version                  Required
> lisp      None but see note (1)
> mono      None but mono is compiled and automatic deps are extracted that
>            require mono
> ocaml     ocaml(runtime) = MAJOR.MINOR.MICRO        Required for libraries
> perl      perl(:MODULE_COMPAT_MAJOR.MINOR.MICRO)    Required
> python    python(abi) = MAJOR.MINOR                 Required  (auto-filled)
> php       php(language) >= VERSION                  Optional (only if it
>                                                      needs a specific version)
> R         R-Core                                    Unknown (2)
> ruby      ruby(abi) = MAJOR.MINOR.MICRO             Required
> tcl       Requires: tcl(abi) = MAJOR.MINOR          Required
>
> (1) lisp doesn't have a Virtual Provide at all but the lisp language isn't
> making rapid changes so the assumption seems to be that all lisp source is
> compatible.  The compiled files, which are incompatible between lisp
> interpreters and perhaps versions of those interpreters is generated after
> the package is installed which is similar in that respect to what Debian is
> doing for python.
>
> (2) R-Core is in the spec templates.  It is unversioned there.  Nothing says
> whether a versioned virtual provide should be used or not and some
> R packages use a versioned Requirement on R-Core while others use an
> unversioned one.
>
> Looking at this list, it seems that the major languages do have required
> versioned Requires.  It may be that the ones where it's optional (php, lisp)
> or don't specify a version (R) should be updated to make that a mandatory
> requirement.

 From that list I would say that Perl is the closest one, although I do 
not understand why they don't version the virtual provide, as I am 
proposing above. But this was not working in previous versions of RPM I 
guess.


Vít