Build-essential packages

Tue Jun 16 14:50:48 UTC 2015

On 2015-06-12 22:32, Dennis Gilmore wrote:
> On Friday, June 12, 2015 02:21:14 PM Carlos O'Donell wrote:
>> On 06/12/2015 12:11 PM, Dennis Gilmore wrote:
>>> On Thursday, June 11, 2015 08:36:38 AM Florian Weimer wrote:
>>>> On 05/21/2015 10:11 PM, Jason L Tibbitts III wrote:
>>>>> The BuildRequires section of the guidelines has been revised; the
>>>>> exceptions list is gone.  The release engineering folks are free to
>>>>> define the buildroot and rpm is free to change its dependency list.
>>>>>
>>>>>   * https://fedoraproject.org/wiki/Packaging:Guidelines#BuildRequires_2
>>>>>   *
>>>>>   https://fedoraproject.org/w/index.php?title=Packaging%3AGuidelines&diff
>>>>>   =
>>>>>   413629&oldid=409506 * https://fedorahosted.org/fpc/ticket/497
>>>>
>>>> Can we get a build-essential package instead that requires everything
>>>> that is needed to get a working C and C++ compiler, and run most
>>>> autoconf/automake/libtool-generated scripts (but not the autotools
>>>> themselves)?
>>>
>>> Can you please help me to understand the problem you are trying to solve?
>>> what is different to "dnf install @buildsys-build" other than a package
>>> vs a comps group
>>
>> The recent policy changes mean that developers have to take action to fix
>> broken spec files. Comments like those above are essentially a request,
>> and the request from our developers is going to be:
>>
>> "Now that the buildroot can contain almost nothing, what do I need to
>> require in order to build C or C++ applications?"
>>
>> "Do I have to figure out every possible command that autoconf might call
>> and include it in my BuildRequires or is there some magic meta prodives
>> I can use?"
>>
>> To answer this question for C and C++ development I have filed this FPC:
>>
>> https://fedorahosted.org/fpc/ticket/540
>>
>> And while the pedantic answer is "BuildRequire and Require on whatever
>> you need", that is in my opinion insufficient guidance for Fedora packagers.
>
> Okay thanks, with my releng hat on I had no idea this was coming and would
> have suggested to the FPC to not change anything, at the least giving releng a
> heads up saying that they were going to make us the gatekeepers would have
> been appreciated. There is certainly no immediate plan to change anything from
> status quo. I guess this was triggered by the request recently to remove c,
> c++, and make from the minimal buildroot.
>
> in my mind metapackages while they could serve a useful purpose could quite
> easily be abused and lead to buildroot bloat.
>
> "koji list-groups f23-build" shows us currently pulling in
> build  [f23]
>    bash: None, default  [f23]
>    bzip2: None, default  [f23]
>    coreutils: None, default  [f23]
>    cpio: None, default  [f23]
>    diffutils: None, default  [f23]
>    fedora-release: None, default  [f23]
>    findutils: None, default  [f23]
>    gawk: None, default  [f23]
>    gcc: None, default  [f23]
>    gcc-c++: None, default  [f23]
>    grep: None, default  [f23]
>    gzip: None, default  [f23]
>    info: None, default  [f23]
>    make: None, default  [f23]
>    patch: None, default  [f23]
>    redhat-rpm-config: None, default  [f23]
>    rpm-build: None, default  [f23]
>    sed: None, default  [f23]
>    shadow-utils: None, default  [f23]
>    tar: None, default  [f23]
>    unzip: None, default  [f23]
>    util-linux: None, default  [f23]
>    which: None, default  [f23]
>    xz: None, default  [f23]
>
> of those the only things I think we would consider removing are gcc and gcc-
> c++ I think make is wide enough used that it should not be removed.  I would
> also proposed that there would be no random changes to what is installed in
> the minimal buildroot during a releases life. so the earliest that there would
> be any change is in rawhide when we branch f23 off.  so what builds today in
> rawhide will not have the buildroot changed under it for the life of f23.  but
> once we branch and rawhide targets f24 as part of the branching process would
> be the only time we change the package set used for the minimal buildroot.  it
> would require that we coordinate the changes with comps, so that you can test
> locally.
>
>>>> In my opinion, it is a bad use of developer time to track what programs
>>>> exactly are called from ./configure, and how these programs match
>>>> sed/grep/coreutils.  It would also  give us a central place where we can
>>>> fix breakage due to missing packages in build roots because a
>>>> significant fraction of packages got a build-required package through an
>>>> indirect dependency.
>>>
>>> can you describe the issues and breakages you are talking about, or point
>>> me at some examples.  the buildroot does not often break.  but in the
>>> scenario you are talking about fixing may be difficult without being able
>>> to build the package that has the fix.
>>
>> At present there is no breakage, but as proactive Fedora packagers Florian
>> and I have kicked off the conversation to say "What will happen?" "What do
>> packagers need to do to keep their spec files building?"
>
> Okay that is not at all a bad thing. I think that we will need to be really
> careful and as part of when we announce that the branch event has happened we
> list any changes to the minimal buildroot.  would that be sufficient to allay
> your fears?
>
>>> I am trying to understand what you see as broken and how this would fix
>>> it. as opposed to how things are currently done.
>>
>> What is broken is that we have no standard set of meta requires to say
>> things like "I need a C development environment" e.g. BuildRequires:
>> c-devel.
> at least as we define it in the minimal buildroot today it would be
> BuildRequires gcc  perhaps we could look at some virtual provides to allow for
> some transition to a different compiler down the road. But that kind of thing
> can get really messy.
>
>> Worse is when it comes to autoconf and determining exactly all the commands
>> possibly run by the upstream configure. We need a tool that in some ways
>> helps compute the set of tools required from a build.
>
> automated tooling to parse a source tree and figure out the BuildRequires
> would be a nice thing here. very useful if we cut the minimal buildroot to
> enough to run rpmbuild.

If there is a tool that can compute BuildRequires then maybe the 
packages could be installed on-demand during the build. We as Java-Maven 
packagers face the problem that the set of required packages cannot be 
reliably determined before the package is actually being built as the 
project object model is generated from multiple sources on the fly. As a 
solution we developed a mock plugin called "pm_request", which is now 
part of upstream mock (available since v. 1.2.9, disabled by default). 
It listens on a socket within the buildroot and can fulfill request for 
package management commands (usualy installation) from the buildsystem 
(or other tool) currently running within the buildroot. So when we want 
to package a new maven-built library, we only specify BuildRequires: 
maven-local and the rest is installed automatically during local 
mockbuild. The plugin prints what was installed, so we can use the 
output as BuildRequires for subsequent build in Koji, which doesn't 
enable the plugin. It is currently used only by Maven stack, but there's 
nothing maven-specific and I can imagine it could also be used by 
autoconf based builds (it still needs to have some tool to make the 
requests). So my question is - if more language stacks use it, would it 
be possible to permit using it in Fedora Koji? We already generate 
Requires, why not BuildRequires?

>
> Though here i can see a world where we say a npm buildroot for building
> node.js natively and then wrapping the output in a rpm for shipping at the
> end. we could then look at doing the same for rubyjems, maven etc. build
> everything in its native format and wrap the results as rpms to ship. we could
> additionally ship a maven repo etc also.  then the software we build could
> extend beyond the Fedora ecosystem.
>
>
>>> right now we have the build group defined in koji and buildsys-build in
>>> comps. the packages defined in the comps groups define the minimal
>>> buildroot, which is what gets installed when you do "mock -r <config>
>>> --init"  BuildRequires get installed on top of it.  then the build is
>>> done
>>
>> That's not at issue. What's at issue is how do packagers keep their spec
>> files building given the policy change enables the buildroot to contain
>> almost nothing.
> I think the answer here is just they we be very clear and have a checkpoint in
> the schedule where we can make any changes.
>
> Dennis

Michael Simacek