[PATCH] Add anaconda-runtime to the package list for buildinstall

[Mark McLoughlin]

With e.g.:

  repo --name=rawhide --baseurl=http://foo

  %packages
  %end

You get:

  OSError: Got an error from /usr/lib/anaconda-runtime/buildinstall:

and trying out the buildinstall command directly, you see:

  Running buildinstall...
  No Match for argument anaconda-runtime
  Nothing to download

The issue here is that one of the first things buildinstall
tries to do is install anaconda-runtime from the supplied
repository, so the user must add anaconda-runtime to the
package list or pungi fails.

Automatically add it to the package list rather than
requiring the user to do it.

Signed-off-by: Mark McLoughlin <markmc(a)redhat.com&gt;
---
 src/bin/pungi.py |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/src/bin/pungi.py b/src/bin/pungi.py
index 91ca403..bcbe6fd 100755
--- a/src/bin/pungi.py
+++ b/src/bin/pungi.py
@@ -77,6 +77,10 @@ def main():
 
     # Actually do work.
     if not opts.sourceisos:
+        if opts.do_all or opts.do_buildinstall:
+            # buildinstall requires anaconda-runtime (rh #443371)
+            ksparser.handler.packages.add(["anaconda-runtime"])
+
         if opts.do_all or opts.do_gather:
             mygather = pypungi.gather.Gather(config, ksparser)
             mygather.getPackageObjects()
-- 
1.5.4.5

[Jeroen van Meeuwen]

Mark McLoughlin wrote:
>  With e.g.:
>  
>    repo --name=rawhide --baseurl=http://foo
>  
>    %packages
>    %end
>  
>  You get:
>  
>    OSError: Got an error from /usr/lib/anaconda-runtime/buildinstall:
>  
>  and trying out the buildinstall command directly, you see:
>  
>    Running buildinstall...
>    No Match for argument anaconda-runtime
>    Nothing to download
>  
>  The issue here is that one of the first things buildinstall
>  tries to do is install anaconda-runtime from the supplied
>  repository, so the user must add anaconda-runtime to the
>  package list or pungi fails.
>  
>  Automatically add it to the package list rather than
>  requiring the user to do it.
>   
Because pungi doesn't pull in *any* "required" packages, I was reluctant 
to submit this patch... It seems that once you go down this road you 
will want to catch *all* required packages, like we do in Revisor (and 
creates a very, very long list[1]).

Kind regards,

Jeroen van Meeuwen
-kanarip

[1] http://git.fedorahosted.org/git/?p=revisor;a=blob;f=revisor/cfg.py#l1165

[Jesse Keating]

On Mon, 2008-04-21 at 10:54 +0200, Jeroen van Meeuwen wrote:
>  
>  Because pungi doesn't pull in *any* "required" packages, I was reluctant 
>  to submit this patch... It seems that once you go down this road you 
>  will want to catch *all* required packages, like we do in Revisor (and 
>  creates a very, very long list[1]). 
Yep.  I'll be fixing this proper after F9 goes out by removing the
dependency of having these compose needs in the manifest.  It'll allow
fully graphical installers for tiny package sets too.

-- 
Jesse Keating
Fedora -- All my bits are free, are yours?

[Jeroen van Meeuwen]

Jesse Keating wrote:
>  On Mon, 2008-04-21 at 10:54 +0200, Jeroen van Meeuwen wrote:
> > Because pungi doesn't pull in *any* "required" packages, I was
> reluctant 
> > to submit this patch... It seems that once you go down this road you 
> > will want to catch *all* required packages, like we do in Revisor (and 
> > creates a very, very long list[1]).
>  
>  Yep.  I'll be fixing this proper after F9 goes out by removing the
>  dependency of having these compose needs in the manifest.  It'll allow
>  fully graphical installers for tiny package sets too.
>   
I might be looking at doing the same -given that I may misunderstand 
what you're going to do. Does look similar to:

1. Pull in the requested packages (depsolve, etc) to become the final 
payload,

2. Create another repo with the required packages and pull in the 
required packages as well as their dependencies, build the installer 
images with the joined repo configuration,

3. Delete all overhead

Or were you thinking something else?

Kind regards,

Jeroen van Meeuwen
-kanarip

[Jesse Keating]

On Mon, 2008-04-21 at 13:44 +0200, Jeroen van Meeuwen wrote:
>  Or were you thinking something else? 
I was.  buildinstall recently changed how it works.  You no longer point
it at a directory of packages, you point it at a yum repo.  You can also
have it output to a different directory.  So my plan, which will take
some modification to createrepo (Seth is already aware of this and has
it planned), is to use pungi to wire up all the repos configured in the
kickstart file and generate a "reference" repo.  This repo would be
"local" to the file system, use a combined comps grouping (for all the
repos that you wish to consider for groupdata (another change coming for
pykickstart)), but the repodata itself would have hrefs to the packages
locations in the other repos.  You get local repodata without having to
mirror all the packages.  Buildinstall would be pointed at that repo,
and it would download (insert caching dances here) the packages it needs
in order to make install images.  Then (well, still not sure of the step
orders) pungi would use it's manifest to create the package trees and
insert the produced images into the trees.

That's the fairly rough overview.

-- 
Jesse Keating
Fedora -- All my bits are free, are yours?

[Jeroen van Meeuwen]

Jesse Keating wrote:
>  On Mon, 2008-04-21 at 13:44 +0200, Jeroen van Meeuwen wrote:
> > Or were you thinking something else?
>   
[...snip...]
>  That's the fairly rough overview.
>   
Revisiting this topic;

I've got this implemented differently right now; I'm pointing 
buildinstall to one or more of the enabled repositories configured, as 
it's just using yum with a configuration file that can perfectly point 
at baseurls and mirrorlists. This does require some modifications to 
buildinstall though, and I'm not sure whether the anaconda guys would 
even consider it.

Attached is a the modified buildinstall script taking multiple --repo 
parameters (currently only taking baseurls though).

Kind regards,

Jeroen van Meeuwen
-kanarip

#!/bin/bash
#
# buildinstall
#
# Copyright (C) 2007  Red Hat, Inc.  All rights reserved.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program.  If not, see <http://www.gnu.org/licenses/>;.
#

usage() {
    echo "Usage: buildinstall --version <version> --product <product>
--release <comment> [--output outputdir] [--discs <discstring>]
<root>" >&2
    exit 1
}

PRODUCTPATH="anaconda"

declare -a repositories

while [ $# -gt 0 ]; do
    case $1 in
    # general options affecting how we build things
    --nogr)
        NOGRSTR="--nogr"
        shift
        ;;
    --debug)
        DEBUGSTR="--debug"
        shift
        ;;

    # release information
    --version)
        VERSION=$2
        shift; shift
        ;;
    --release)
        RELEASESTR=$2
        shift; shift
        ;;
    --product)
        PRODUCTSTR=$2
        shift; shift
        ;;
    --variant)
        VARIANT=$2
        shift; shift
        ;;
    --bugurl)
        BUGURL=$2
        shift; shift
        ;;

    --output)
        OUTPUT=$2
        shift; shift
        ;;
    --updates)
        UPDATES=$2
        shift; shift
        ;;
    --repo)
        repositories[${#repositories[@]}]=$2
        shift; shift
        ;;
    *)
        REPO=$1
        shift
        ;;
    esac
done

if [ -z "$PRODUCTSTR" ]; then
    usage
fi

if [ -z "$VERSION" ]; then
    usage
fi

if [ -z "$REPO" ]; then
    usage
fi

if [ -z "$RELEASESTR" ]; then
    usage
fi

if [ -z "$BUGURL" ]; then
    BUGURL="your distribution provided bug reporting tool."
fi

if [[ "$REPO" =~ ^/ ]]; then
    [ -n "$OUTPUT" ] || OUTPUT=$REPO
    REPO="file://$REPO"
fi

if [ -z "$OUTPUT" ]; then
    usage
fi

if [ ! -d "$OUTPUT" ]; then
    mkdir -p $OUTPUT
fi

yumconf=$(mktemp /tmp/yum.conf.XXXXXX)
cat > $yumconf <<EOF
[main]
keepcache=0
gpgcheck=0
plugins=0
reposdir=
tsflags=nodocs
EOF

if [ ${#repositories[@]} -lt 1 ]; then
    cat >> $yumconf <<EOF
[anacondarepo]
name=anaconda repo
baseurl=$REPO
enabled=1
EOF
else
    i=0
    while [ $i -lt ${#repositories[@]} ]; do
        cat >> $yumconf <<EOF
[repo-$i]
name=anaconda repo $i
baseurl=${repositories[$i]}
enabled=1
EOF
        i=$[ $i + 1 ]
    done
fi

BUILDINSTDIR=`mktemp -d ${TMPDIR:-/tmp}/buildinstall.tree.XXXXXX`
TREEDIR=`mktemp -d ${TMPDIR:-/tmp}/treedir.XXXXXX`

echo "Running buildinstall..."

pushd $BUILDINSTDIR
BUILDARCH=`repoquery -c $yumconf --qf "%{ARCH}\n" anaconda-runtime`
yumdownloader -c $yumconf anaconda-runtime || exit 1
rpm2cpio anaconda-runtime*rpm | cpio --quiet -iumd './usr*'
rm -f anaconda-runtime*rpm
popd

UPD_INSTROOT=./upd-instroot
MK_IMAGES=./mk-images
MK_TREEINFO=./maketreeinfo.py
MK_STAMP=./makestamp.py
BUILDINSTALL=./buildinstall

for f in $UPD_INSTROOT $MK_IMAGES $MK_STAMP $MK_TREEINFO $BUILDINSTALL; do
    if [ -n "$UPDATES" -a -f $UPDATES/usr/lib/anaconda-runtime/$f ]; then
    cp -a $UPDATES/usr/lib/anaconda-runtime/$f* $BUILDINSTDIR/
    elif [ ! -f $f ]; then
    cp -a $BUILDINSTDIR/usr/lib/anaconda-runtime/$f* $BUILDINSTDIR/
    else
    cp -a $f* $BUILDINSTDIR/
    fi
done

UPD_INSTROOT=$BUILDINSTDIR/upd-instroot
MK_IMAGES=$BUILDINSTDIR/mk-images
MK_TREEINFO=$BUILDINSTDIR/maketreeinfo.py
MK_STAMP=$BUILDINSTDIR/makestamp.py
BUILDINSTALL=$BUILDINSTDIR/buildinstall

if [ -n "$UPDATES" ]; then UPDATES="--updates $UPDATES"; fi

echo "Building images..."
$UPD_INSTROOT $DEBUGSTR $NOGRSTR --arch $BUILDARCH $UPDATES --mindir $TREEDIR/minstg2
--stg2dir $TREEDIR/stage2 $yumconf

echo "Writing .treeinfo file..."
$MK_TREEINFO --family="$PRODUCTSTR" ${VARIANT:+--variant="$VARIANT"}
--version=$VERSION --arch=$BUILDARCH --outfile=$OUTPUT/.treeinfo

# FIXME: need to update mk-images to take the yumconf
echo "Making images..."
$MK_IMAGES $DEBUGSTR $NOGRSTR --mindir $TREEDIR/minstg2 --stg2dir $TREEDIR/stage2 --arch
$BUILDARCH --product "$PRODUCTSTR" --version $VERSION --bugurl
"$BUGURL" --output $OUTPUT $yumconf

echo "Writing .discinfo file"
$MK_STAMP --releasestr="$RELEASESTR" --arch=$BUILDARCH --discNum="ALL"
 --outfile=$OUTPUT/.discinfo

rm -rf $TREEDIR $BUILDINSTDIR
rm -f $yumconf

[Jesse Keating]

On Mon, 2008-05-12 at 22:54 +0200, Jeroen van Meeuwen wrote:
>  Revisiting this topic;
>  
>  I've got this implemented differently right now; I'm pointing 
>  buildinstall to one or more of the enabled repositories configured, as 
>  it's just using yum with a configuration file that can perfectly point 
>  at baseurls and mirrorlists. This does require some modifications to 
>  buildinstall though, and I'm not sure whether the anaconda guys would 
>  even consider it.
>  
>  Attached is a the modified buildinstall script taking multiple --repo 
>  parameters (currently only taking baseurls though). 
Didn't Will Woods submit a patch shortly after I posted this that
allowed for passing multiple repos to buildinstall?

-- 
Jesse Keating
Fedora -- Freedom² is a feature!

[Jeroen van Meeuwen]

Jesse Keating wrote:
>  On Mon, 2008-05-12 at 22:54 +0200, Jeroen van Meeuwen wrote:
> > Revisiting this topic;
> >
> > I've got this implemented differently right now; I'm pointing 
> > buildinstall to one or more of the enabled repositories configured, as 
> > it's just using yum with a configuration file that can perfectly point 
> > at baseurls and mirrorlists. This does require some modifications to 
> > buildinstall though, and I'm not sure whether the anaconda guys would 
> > even consider it.
> >
> > Attached is a the modified buildinstall script taking multiple --repo 
> > parameters (currently only taking baseurls though).
>  
>  Didn't Will Woods submit a patch shortly after I posted this that
>  allowed for passing multiple repos to buildinstall?
>   
You're right, he did, on the anaconda-devel list. I've missed it ;-)

Kind regards,

Jeroen van Meeuwen
-kanarip

[Jeremy Katz]

On Mon, 2008-05-12 at 23:12 +0200, Jeroen van Meeuwen wrote:
>  Jesse Keating wrote:
>  > On Mon, 2008-05-12 at 22:54 +0200, Jeroen van Meeuwen wrote:
>  >> Revisiting this topic;
>  >>
>  >> I've got this implemented differently right now; I'm pointing 
>  >> buildinstall to one or more of the enabled repositories configured, as 
>  >> it's just using yum with a configuration file that can perfectly point 
>  >> at baseurls and mirrorlists. This does require some modifications to 
>  >> buildinstall though, and I'm not sure whether the anaconda guys would 
>  >> even consider it.
>  >>
>  >> Attached is a the modified buildinstall script taking multiple --repo 
>  >> parameters (currently only taking baseurls though).
>  > 
>  > Didn't Will Woods submit a patch shortly after I posted this that
>  > allowed for passing multiple repos to buildinstall?
>  > 
>  
>  You're right, he did, on the anaconda-devel list. I've missed it ;-) 
And fwiw, I applied the patch to anaconda master about 30 or 45 minutes
ago :)

Jeremy