[Fedora-packaging] SCL Notes and Questions

[Bohuslav Kabrda]

First of all, thanks for doing so thorough review/comment summary. I really appreciate the amount of work you've put into this.
In this mail, I'm going to respond just to the parts about the draft itself. I think Marcela covered the other parts pretty well and there is nothing I want to add there.
I've done some changes in the draft according to your comments... please see the wiki page history to see the specific adjustments.

> Comments on the SCL Draft:
> https://fedoraproject.org/wiki/User:Bkabrda/SCLGuidelinesDraft
> 
> * Can we assume that the implementation can be changed to address
> problems, add features, or follow different conventions?  Can be done
> if backwards compat is possible?  Or is implementation set in stone so
> we'd have to kludge around any shortcomings?
> 

So, this is a very general question, so my answer is: Depends. If we come up with a sane proposal, I think the scl-utils upstream will accept, but I can't really say that in general.

> * One of the goals of SCLs is isolation from the rest of the system,
> correct?  It seems, though, that once you run the enable script, that
> shell would have a different PATH with the possibility of a different
> language interpreter.  As long as scripts can use /usr/bin/env as a
> shebang or scripts are using PATH-based commands these will end up
> using programs from the SCL.  Should FPC revisit banning these?  Is it
> okay to run with SCLs even without banning?  Just something to
> document?
> 

So, I think we've already mentioned this during our Python-3-shift-discussion. My opinion on this is, that if you're packaging a script for system, it should run with system interpreter, regardless of current $PATH.

> === Specific comments on the draft ===
>  * Move template to the bottom of the draft.  Had many questions after
> reading the template that were eventually answered in other parts of
> the draft.

Do you mean the metapackage template?

>  * Style guide is not to have a period at the end of Summary

Fixed, sorry :)

>  * Description in template should probably be more clear that
> packagers should fill in a proper description

I added a line with "Provide some useful info about this SCL." into the description.

>  * Template skips %prep and %build.  Are these empty?  If so, just
> include the empty %prep and %build headers.

Yikes! The %prep section actually has to be there, because macros.scl use %{buildsubdir}, which is defined during %prep... fixed.

>  * "The -build subpackage should include Requires: scl-utils-build."
> Why a should and not a must?

I guess I can take this one out, assuming that scl-utils-build package is always tagged in buildroot. The reason why I included the line about that Requires is, that each <scl>-build subpackage needs scl-utils-build to work (in the sense that having only <scl>-build without scl-utils-build will fail). So... some explicitness, maybe unnecessary.

>  * Content about the enable scriptlet implies that the usual path
> macros (_bindir etc) are changed in the scl package.  Need to make it
> clear what standard macros are being overridden to what. Perhaps
> consider "tables" like
> http://fedoraproject.org/wiki/Packaging:MinGW#Filesystem_location_macros
> but with slightly different columns, e,g, macro | override? | Normally
> Modified? | example | description

I don't think we'd need a table for this, since all the macros are just prefixed with "/opt/<provider>/<scl>/root/". I mentioned this in the draft.

>  * "The macros.%{scl}-config in %{scl}-build package contains the %scl
> macro definition (name of the SCL). If you need to use some more
> macros for this SCL, add them into this file." <= but there's nothing
> in the spec file that obviously creates this.  Need instructions on
> how to create this file for manual use and then how to include the
> file in the spec file and not have whatever automated macro normally
> creates it not overwrite it.

This macro file is created by %scl_install and it's location is %{buildroot}%{_root_sysconfdir}/rpm/macros.%{scl}-config. I mentioned that in the draft.

>  * "essential for its runtime". "Essential" needs to be explained and
> hopefully it is just a link to the earlier part of the doc.

Rephrased.

>  * -runtime package must be arch specific => it contains
> %{scl_prefix}%{_libdir} so it's arch dependent. (and this means the
> main package must be arch specific too)
> 

Is this unclear in the draft? It seems pretty clear to me. Do you have a specific wording for the draft in mind?

>  * Creating the enable file in %install -- would rather see macros
> like:  - %scl_add_path %scl_add_ld_library_path %scl_add_manpath than
> the HERE-doc style thing that will cut and paste to most specs.

That's actually an interesting idea and should probably be proposed to upstream. I'm afraid that I can't do anything about this until upstream includes such macros.

>  * "set of macros that contain the values of the original RPM macros"
> should be expanded on.  naming of these could be better => establish a
> common prefix for anything that is being overridden instead of
> %pkg_name vs %_root_.  also pkg_name probably conflicts with some
> existing packages.  Maybe something like %_prescl_name,
> %_prescl_libdir, etc.  [can change the recommendation from: packagers
> should define %pkg_name to packagers should define a macro that is the
> value of pkg_name => %{?scl: %global srcname %pkg_name and %{?!scl:
> %global srcname %name}

- I included an example of the original macro and redefined _root_* macro.
The problem with the %pkg_name part is, that it has to be %pkg_name (unless upstream changes that), so that you're able to use one macro throughout your specfile.
The name of the original macros are upstream thing. I myself am ok with them, but feel free to propose your idea to scl-utils upstream.

>  * Little more guidance needed for Requires. Maybe making the examples
> more real-life would clarify things.  ("Since ifconfig is a binary
> we're going to use the system version of, do not use scl_prefix with
> it. rubygem-foo, otoh, has to be in the path of the ruby interpreter
> which this scl is to be a part of. Therefore it needs to use
> scl_prefix"). Also, an example of a "Versioned Requires" would be
> helpful.

Done.

>  * Need more wording on Requires: of non-scl packages as they can act
> as conflicts.  Something like: Requires: foo = 1.2.3 … where the scl
> package is the only thing keeping foo back at that version.  So you
> can't update "the OS" without updating the scl.  wording might be
> something like "yes, you can require non-scl stuff … but you have to
> be very general in your requirements, and if you need something that
> might be updated you have to either scl bundle it or be willing to
> rebuild the scl when it updates".

I added a note about this.

>  * Should we list somewhere the scl-modified deps scripts?  Otherwise
> how will someone creating a new ocaml-scl know whether an ocaml-scl
> dep script has already been written?

Yep, we can probably add a section like this when we actually gather scripts like these.

>  * Please elaborate on the function of %scl_package_override()

I put a detailed explanation in admon/tip frame.

> * In the dealing with macro files section: I think that the "Now
> consider a situation[...]" starts a whole new scenario?  Correct?  If
> so, needs to be better phrased right now it's associating with the
> previous examples but it doesn't seem to make sense in that context.

Correct. I placed a subsubsubsection header before that - it should clearly separate the two scenarios.

> * So If I've understood everything correctly, building in a standard
> buildroot will get you a non-scl'd package.  Building with the
> $scl-build package installed will build the scl version?
>   * Should say that in the first line of Building Packages

Yes, that is assuming you do everything right :) Done.

> * In Building Packages: Don't need to include scl-utils-build
> explicitly in the config_opts line (brought in via dependencies),
> correct?
> 

Not really. Without scl-utils-build, you won't be able to build metapackage in Koji, as it will fail on composing SRPM, because it won't be able to parse specfile (Unknown tag: %scl_package).

> == Conversion to SCLs ==
>  * Parts of this section are more "tips & tricks"? Should it be on a
> non-guideline page?

Yes, it can be, I don't really mind. I just find it handy to be at the same place.

>    * Probably best to vette the advice anyway and then move what makes
> sense to a separate page afterwards.
>  * Add information about why /usr/bin/env is not a generally good
> thing in SCLs.  If this is more general than SCLs, does FPC need to
> revisit a ban on /usr/bin/env in shebangs?

As mentioned above, I think this should be more general.

>  * sed line should limit to the first line of the file as well as the
> shebang pattern.

Limited.

>  * how  does scl deal with compiled-in paths inside of binaries?  Does
> it  depend on %configure to do the right thing with %{_libdir} etc?
> If so  we should point out to people that old-school builds which
> aren't  configure driven may need patches (actually, this applies to
> install  location as well as compiled-in paths and to scripts that
> have file  paths embedded as well as binaries)

SCL builds depend on not-hardcoded paths, so some amount of patching may be needed. I added an admon/caution about this to guidelines.

>  * How do auto provides and requires for elf libraries work?
> 

Problematically :) Filters for these (prepending "%{scl}-") have to be provided by collection packagers). I know that there was a scl-utils bug for this, but I can't find it right now. I'll put this on my TODO list and find it out.

> == Inter-SCL Dependencies ==
> * Example of using this with versioned Requires?

Added.

> * The %scl_require example just doesn't seem right somehow.... The
> ruby193 scl brings in rails-3.2.3?  That seems like poor naming (if
> the metapackage brings in rails, then it should be named rails.  OTOH,
> if the package purpose is to provide ruby193 then rails should be
> optional not essential.) or poor dependency management (Which would be
> part of the higher level questions of what can be an SCL and what can
> be inside an SCL).
> 

Poor naming. I changed the scl name in the example to rails323.

> * Are non-scl packages allowed to dep on scl packages?  If so, how do
> they do so?
> 

Generally, we should keep dependencies of non-scl packages on scl packages at minimum, although it does make sense in some cases - for example, user just wants to install OpenStack and he doesn't care if it depends on a collection or not, so it makes perfect sense to have "openstack" package depend on SCL. On the other hand, this can quickly become a terrible mess, if not done cautiously. I'll try to think about some wording when this is allowed and when not.

> * The filters don't work in EL6 thing.... people seem more excited
> about SCLs in EPEL than in fedora so we probably need to document how
> to filter the provides and requires for EL6.  However, we can point to
> the EPEL6 Differences page from here and document the behaviour there.
> 

Yes, that is a good idea. I see that Marcela has already added a reference to a bug that explains that filters don't work in epel. I added link to the wiki page that describes filtering on epel.

> == General Comments ==
>  * No definition of a SCL appears in the document
>    * Example text: Software Collections (SCLs) give rpm packagers a
> means of packaging multiple versions of software for a single
> distribution.  The version from the software collection does not
> interact with the system version. They can provide backwards and
> forwards compat versions of a package compared to what's on the
> system.
>  * Include "definitions" at the beginning of the document, some
> example things that need definition (and suggestions)
>    * SCL -> whole software collection? <- originally the term was
> "dynamic software collection" for the "whole thing" but it has fallen
> in to disuse... we could bring it back
>    * SCL defining package -> the srpm-level package that defines the scl.
>    * SCL metapackage -> built from the SCL defining package.  defines
> the essential packages you get when you install the scl.
>    * SCL runtime package -> built from the SCL defining package.  It's
> the setup of the system for this SCL including registering the SCL
> with the system and setting up the filesystem.
>    * SCL build package -> built from the SCL defining package.  It
> contains macro definitions needed to build an SCL.
>    * SCL package -> any package which can be built for an SCL.
>  * Elaborate on what "belongs" in the "-runtime" package
>    * Extracting one of these from the sample repository seems like
> it's a filesystem package directory tree inside of the scl?  Anything
> else?

I added a general section that contains some definitions and explanations.

> -Toshio

Thanks,
Slavek.

-- 
Regards,
Bohuslav "Slavek" Kabrda.