EPEL Questions about SCL during meeting

[Nick Coghlan]

On 09/16/2014 12:48 AM, Honza Horak wrote:
> Right, devtoolset is one of the examples where I see the point of
> non-SCL package depended on SCL.
> 
> Dynamic languages are a bit different story, since (with a bit of
> abstraction) depended packages are often libraries that extend
> functionality of the language ecosystem and thus we want to install them
> directly into the ecosystem.

I think for those use cases, it's better to focus on *switching
packaging systems* above the language runtime layer. Java devs are going
to want to use Maven, Ruby devs are going to want to use gems, Node.js
devs are going to want to use npm, Python devs are going to want to use
pip, Python data analysts are going to want to use conda.

At that point, dependency management, security updates, etc are *not the
platform provider's problem any more*. The platform has done its job in
providing us with a language runtime and core operating system
infrastructure, dealing with anything above that is our job as
application developers. We need the OS to just *get out of our way* at
that layer, rather than trying to impose new platform specific rules
that we end up having to work around.

> But if some package is created strictly above the ecosystem, it does not
> have to care if the interpreter comes from SCL package or not and it
> will work fine as soon as the interpreter matches the required version.
>
> Or, on the other hand, requiring package always as SCL for all
> SCL-depended packages does not seem to make things easier.

Right, I don't think that's useful either. However, I also don't believe
"depending on an SCL" is sufficiently specific to figure out what the
user experience should look like. For that, we need to look at more
concrete dependency management use cases.

1. Applications directly integrated into Fedora

- needs parallel installation support at the individual package layer
- may need full parallel stacks (e.g. Py2 vs Py3)
- may need to be resolved at runtime (e.g. CherryPy2 vs CherryPy3)
- may need language level support (e.g. pkg_resources)
- not an SCL problem

2. Independent web applications/network services

- switch to language specific tools above the runtime layer
- use containers for easy deployment
- not an SCL problem

3. Independent GUI applications

- could adopt the same model as web applications
- sandboxed silos with controlled access to platform services (ala Android)
- not an SCL problem (except perhaps to provide GUI frameworks as SCLs)

4. Independent command line applications

- container images are too bloated for simple scripts
- annoying being locked into old runtimes due to CentOS/RHEL 6
- package level parallel installs make dependency management painful

Accordingly, I think a good motivating use case to help guide the design
of what it means for an ordinary package to depend on an SCL is "single
file Python/Ruby/whatever script depending on a suitable language
runtime version".

So, for example, I might want to use the py33 SCL to run my script,
regardless of whether it was running on Fedora, CentOS or RHEL.
Addressing that only requires defining two things:

1. What do I put in the Requires field in my spec file?
2. How do I get the shebang line in my script(s) set correctly?

For example, this would let us upgrade our client app for
beaker-project.org to always use Python 2.7 (or even Python 3) without
abandoning support for CentOS/RHEL 6. On Fedora, the benefit would be
that we could upgrade to a new version of the language runtime when it
suited *us*, rather than when the default version in Fedora changed
(this isn't a problem with Python 2 any more, but it will be a factor
again once we switch our command line Python apps to Python 3. It's also
a problem for command line clients written in Ruby).

Regards,
Nick.

P.S. For some more radical long-term visionary ideas in this space,
Lennart's recent post is worth a read:
http://0pointer.net/blog/revisiting-how-we-put-together-linux-systems.html
(I don't necessarily agree with the suggested *solutions* in that piece,
but as an overview of what's still broken, I think it's great)

-- 
Nick Coghlan
Red Hat Hosted & Shared Services
Software Engineering & Development, Brisbane

HSS Provisioning Architect