fedup: boot image not found

Mon Dec 15 23:21:11 UTC 2014

On Mon, 2014-12-15 at 18:14 +0000, Brian McGrew wrote:
> > On Dec 15, 2014, at 9:12 AM, Adam Williamson <
> > adamwill at fedoraproject.org> wrote:
> > 
> > On Mon, 2014-12-15 at 16:40 +0000, Brian McGrew wrote:
> > > Good morning all,
> > > 
> > > Running fedup and getting the following on a stock install of 
> > > F20:
> > > 
> > > [admin at knotts ~]$ sudo fedup --network 21 --product=workstation 
> > > [sudo] password for admin:
> > > setting up repos...
> > > getting boot images...
> > > 
> > > Downloading failed: couldn't get boot images: No more mirrors to 
> > > try. Last error was: [Errno 14] HTTP Error 404 - Not Found
> > > 
> > > 
> > > [admin at knotts ~]$ sudo fedup --network rawhide --
> > > product=workstation setting up repos...
> > > getting boot images...
> > > 
> > > Downloading failed: couldn't get boot images: No more mirrors to 
> > > try. Last error was: [Errno 14] HTTP Error 404 - Not Found
> > > [admin at knotts ~]$
> > 
> > Can't immediately see a good reason it isn't working, does it work 
> > with --instrepo= 
> > https://dl.fedoraproject.org/pub/fedora/linux/releases/21/Server/x86_64/os/ ? (assuming x86_64, 
> > of
> > course)
> 
> Of course x86_64!  That appears to be working.  Odd!  Will let you 
> know if it doesn’t finish.  I’ll get to 21 and then to rawhide and 
> see what happens.

fedup is rarely tested prior to Alpha, FWIW. You will need to manually 
provide a --instrepo parameter again. Here, I may as well explain 
what's actually going on there.

What fedup gets from the 'installation repository' is a kernel and a 
special initramfs. The initramfs contains the bits that drive the 
actual upgrade step of the fedup process - they're in the package 
'fedup-dracut', and part of the Fedora compose process involves 
building the special initramfs with those bits in it.

This special initramfs is called upgrade.img. When you pass fedup a --
instrepo parameter, it looks for the file .treeinfo.signed in the 
location you specify. So, for that one above, it found:

https://dl.fedoraproject.org/pub/fedora/linux/releases/21/Server/x86_64/os/.treeinfo.signed

you can go look at that for yourself, and you'll see what's in it - 
the name is apt, it provides 'info'rmation on the 'tree'. The bit 
fedup cares about is that it tells it where to find its special 
upgrade initramfs:

[images-x86_64]
kernel = images/pxeboot/vmlinuz
initrd = images/pxeboot/initrd.img
upgrade = images/pxeboot/upgrade.img
boot.iso = images/boot.iso

fedup goes and grabs the 'kernel' and 'upgrade' files (the kernel is 
just the kernel file that's in the kernel package in the same tree), 
and that's what it adds to the bootloader config, the environment you 
boot into after the first fedup phase has downloaded all the packages 
and told you to reboot to continue.

If you *don't* pass a --instrepo parameter, what fedup does is it asks 
mirrormanager for the location: it asks mirrormanager for a mirror 
list for the 'fedora-install' repository for the release you specified 
you wanted to upgrade to, and the appropriate arch. You can do the 
same thing through mirror manager's web interface:

https://mirrors.fedoraproject.org/mirrorlist?repo=fedora-install-21&arch=x86_64

So if you just say 'fedup --network 21' on an x86_64 system, fedup 
goes and asks mirrormanager for that list of repos, then picks one and 
uses it as the 'installation repository'. (When I read your mail the 
first thing I did was check that URL and make sure the list it 
produced was sane; it does seem to be, so I'm not quite sure why you 
got a 404. It's possible you hit a bad mirror which didn't have the 
files in place, I guess?)

So now let's see what happens if you try and fedup to 22 without 
giving it any help:

https://mirrors.fedoraproject.org/mirrorlist?repo=fedora-install-22&arch=x86_64

"# repo = fedora-install-22 arch = x86_64 error: invalid repo or arch"

Ooops. Well, let's try Rawhide:

https://mirrors.fedoraproject.org/mirrorlist?repo=fedora-install-rawhide&arch=x86_64

OK, that's a proper list, so you could try 'fedup --network 
rawhide'...but wait! What does fedup do with the instrepo? It looks 
for .treeinfo.signed. Let's check that's available on a mirror from 
the list:

http://fedora.mirror.nexicom.net/linux/development/rawhide/x86_64/os/.treeinfo.signed ... 
404.

Oh dear. First problem here is, Rawhide trees generally aren't signed. 
So, you'd also have to pass --nogpgcheck; this will make it look for 
the un-signed version of the file instead, which is just .treeinfo :

http://fedora.mirror.nexicom.net/linux/development/rawhide/x86_64/os/.treeinfo ... 
404.

Well pants! Why, you wonder? Because the nightly compose attempt 
failed. There's an automated attempt to 'compose' the Rawhide (and 
Branched, when it exists) tree nightly. That's the process that 
generates all the images you see mentioned in the .treeinfo file - the 
upgrade initramfs, but also boot.iso and various other bits and 
pieces. It doesn't always work. When it doesn't work, there are no 
images, and no .treeinfo. You can look in the current tree and see the 
place where the upgrade.img ought to be - 
http://fedora.mirror.nexicom.net/linux/development/rawhide/x86_64/os
/images/pxeboot - doesn't exist, in fact, the whole /images sub-
directory doesn't exist. That's what a tree which didn't get composed 
looks like. (Note: this may only be true today, of course. Tomorrow, 
or whenever you read this mail, Future Reader, the files may exist, 
because that day's compose succeeded. Just pretend it didn't. Or, stop 
reading, and read this mail again on a day when it didn't!)

TCs and RCs are, of course, always 'composed', so as soon as we hit 
Alpha TC1, you know you can at least find an upgrade.img in the most 
recent TC/RC tree. But prior to Alpha TC1, it can be a bit trickier.

Now something I only recently learned as part of the whole 'let's do 
testing earlier' effort I've been working on is that Rawhide and 
Branched nightly trees are kept around for a while in a location with 
predictable URLs. That's what makes the nightly testing pages kinda 
viable - we can pick a tree which did compose successfully, declare it 
'open for testing', and have stable links to its network install 
images, as 
https://fedoraproject.org/wiki/Test_Results:Fedora_22_20141208_Installation does. So you can, at least, fairly reliably expect to find a .treeinfo and 
an upgrade.img in the last nightly tree 'nominated' for install 
testing, and use that.

(It occurs to me that I can probably do some wiki template magic in 
the fedup test case instructions so they will provide the correct 
instrepo parameter for the most recent nightly compose or TC/RC; I'll 
have to look at that later.)

Soooo, after that long digression, bottom line - if you want to test 
fedup to F22, you should probably run:

fedup --network 22 --nogpgcheck --instrepo 
http://kojipkgs.fedoraproject.org/mash/rawhide-20141208/rawhide/x86_64/os/

(attentive readers will note that 
http://kojipkgs.fedoraproject.org/mash/rawhide-20141208/rawhide/x86_64/os/.treeinfo exists but 

http://kojipkgs.fedoraproject.org/mash/rawhide-20141208/rawhide/x86_64/os/.treeinfo.signed does not, as I explained above, hence the --
nogpgcheck).

but don't come crying to me if it explodes ;) You might well be the 
first to test it for F22. Exciting, eh?
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net