[Fedora-packaging] purpose of ruby(abi), python(abi), etc

Wed Jan 9 00:05:07 UTC 2013

On Tue, Jan 08, 2013 at 10:17:31AM +0100, Vít Ondruch wrote:
> Dne 7.1.2013 19:58, Toshio Kuratomi napsal(a):
> 
> >I was trying to find out if cRuby continued to add things to the MAJOR.MINOR
> >compat mode after the release of the first MAJOR.MINOR:
> >
> >For cRuby MAJOR.MINOR (example: 1.8), is it cRuby policy that 1.8.1,
> >1.8.2, 1.8.3, etc do not have changes that affect compat mode?  No new
> >features or changes to existing ones that mean that code written on 1.8.0
> >will/will not run on 1.8.3?  What about the opposite: is there policy about
> >adding and changing features so that code written on 1.8.3 will run on
> >1.8.0?  Is this policy frequently broken by accidents or rarely?
> 
> Let me explain a bit.
> 
> 1.8.7, 1.9.0, 1.9.1, 1.9.2, 1.9.3, 2.0.0, all these are major
> versions. These major versions are typically released every 18
> months. They may or may not break things. However, you cannot know
> that from version. What is clear is that patch releases should not
> break anything (although for example fixing CVE-2011-4815 [1] broke
> some compatibility, due to change in natural ordering of hashes, but
> that was always wrong assumption). They are typically safe to update.
> 
Okay.  So this is why the rub(abi) virtual provide has a version of
MAJOR.MINOR.MICRO.  Thre's no knowing by the version number whether a MICRO
version changed things relevant to the compat-mode or not so the virtual
provide has to go to that level of detail.

> For example, as you probably know, the ABI was kept compatible in
> between 1.9.1..1.9.3 versions where there will be ABI breakage for
> Ruby 2.0.0.

To remain clear here, when you say ABI, are you talking about the elf
library abi or the compat-mode criteria  (ruby's stdlib, keywords, and other
features discussed earlier)?  I'm reading it as elf library abi but I want
to be sure since we aren't talking about elf library abi in the rest of this
discussion so I'm not sure if I'm misinterpreting or there's a reason for
bringing it up that I'm not seeing.

> However, in between 1.9.1 and 1.9.2, there was polished
> Unicode support, which of course might break something. It depends
> how lucky you are. For example, between 1.9.3 and 2.0.0, there is
> going to be change in default encoding of Ruby scripts from US-ASCII
> to UTF-8. I am not sure if you can foresee every possible issue with
> this change, I cannot.
> 
> To conclude, there is not yet know what will be next version and what
> it will break. This will be known later, when its development begins.
> And it will be clearly declared and communicated [2].
> 
> I can assure you that there is no unnoticed feature creep.
> 

I'm only incidentally looking for unnoticed compat-mode relevant changes
Although, some of my questions were to know if you thought that upstream was
on top of that, I'd certainly hope that you were on top of it even if they
weren't.  The real question, though is where the compat-mode relevant
changes are entering in and how that affects the version.  From your
message, it sounds like compat-mode relevant changes can enter in at any
time (in relation to the version numbers that we're given).

> >
> >If I re-read what you wrote to apply to jRuby instead of cRuby, then I have
> >another question:  how do compat modes line up with jRuby releases?  Does
> >jRuby only ship releases when a compat mode is 100%?  (It sounds like the
> >answer is no since you mention that no release of jRuby support fork(), for
> >instance).  Is there some compat-mode-completion criteria for shipping
> >a release of jRuby?
> 
> There does not exist nothing like 100% compatibility, and that is not
> just about fork, but about C Ruby extensions and so on. The JRuby
> compatibility is based on best effort. I.e. it should work in most of
> the cases and if it does not work in your case, JRuby team will try
> hard to solve the issues.
> 
> The compatibility of Ruby implementations is assured/measured by
> RubySpec project [3]. The JRuby is trying hard to use original
> unmodified cRuby standard library, this can give you overview how
> compatible these implementations are.
> 
> There is also expected that for example you will start to develop you
> Ruby on Rails application using cRuby but you will run it by JRuby in
> production.
> 
> 
> >
> >If we don't have 100% compatibility the present structure of virtual
> >proivides is not well adapted to a shared compat-mode virtual provide.
> >(Similar dangers exist in python where, for instance pypy is close to
> >compatible with python2 but has a few corner cases:
> >http://pypy.readthedocs.org/en/latest/cpython_differences.html ).  I don't
> >believe we have precedent as to "how compatible" something would need to be
> >to share this type of virtual provide but in general, the right thing to do
> >is to err on the side of caution in order to protect the end user.
> 
> Obviously, there is no universal answer to compatibility nor what
> interpreter is the best. Therefore we want to allow to use every Ruby
> implementation in the world as easy as possible, providing sensible
> defaults to those who don't care.
> 
> >Debian has a method that they use for their python stack that is pretty
> >complex and a little hacky that might be usable.  It requires packagers to
> >manually mark their package as being compatible with certain versions of
> >python and then the needed module files are symlinked into the library paths
> >for the compatible python versions when the package is installed.  Something
> >along these lines is a lot more work to setup but may be another
> >possibility.
> 
> The problem with this approach is that even if the code is compatible
> with certain version/implementation and maintainer does not provide
> the symlinks, it will not work. We would like to go the opposite way,
> be proactive. I.e. we expect that everything works and if not, either
> fix it to really work or disable it to not be broken.
> 

From what I'm reading so far, I don't think that your approach is going to
be able to pass as a guideline update.  You essentially are asking to remove
the version from the virtual provide and instead let all packages containing
ruby code to merely specify that they need "Ruby"... not ruby-1.8.1 vs
ruby-2.0.0, etc.  This is too loose a coupling for proper package management
as it allows users to install incompatible combinations of packages because
the dependencies aren't specified precisely enough.

OTOH, although all the guidelines I looked at do tell you how to require the
langugage itself, not all of them make it mandatory:

Language  Virtual Provide                           Required/Optional
java      java >= specific_version                  Required
lisp      None but see note (1)
mono      None but mono is compiled and automatic deps are extracted that
          require mono
ocaml     ocaml(runtime) = MAJOR.MINOR.MICRO        Required for libraries
perl      perl(:MODULE_COMPAT_MAJOR.MINOR.MICRO)    Required
python    python(abi) = MAJOR.MINOR                 Required  (auto-filled)
php       php(language) >= VERSION                  Optional (only if it
                                                    needs a specific version)
R         R-Core                                    Unknown (2)
ruby      ruby(abi) = MAJOR.MINOR.MICRO             Required
tcl       Requires: tcl(abi) = MAJOR.MINOR          Required

(1) lisp doesn't have a Virtual Provide at all but the lisp language isn't
making rapid changes so the assumption seems to be that all lisp source is
compatible.  The compiled files, which are incompatible between lisp
interpreters and perhaps versions of those interpreters is generated after
the package is installed which is similar in that respect to what Debian is
doing for python.

(2) R-Core is in the spec templates.  It is unversioned there.  Nothing says
whether a versioned virtual provide should be used or not and some
R packages use a versioned Requirement on R-Core while others use an
unversioned one.

Looking at this list, it seems that the major languages do have required
versioned Requires.  It may be that the ones where it's optional (php, lisp)
or don't specify a version (R) should be updated to make that a mandatory
requirement.

-Toshio
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.fedoraproject.org/pipermail/packaging/attachments/20130108/b205a15d/attachment.sig>