Manipulate yum repo files in %post

[Robin Bowes]

Hi,

I'd like to manipulate yum repo files in the %post section of my 
kickstarts, specifically to disable repos that are being provided 
through local mirrors.

I've knocked together some python code that does this:

http://robinbowes.com/article.php/20081026162228424

Sample usage:

initool --file /etc/yum.repos.d/epel.repo --section epel --option enabled=0
initool --file /etc/yum.repos.d/epel.repo --section epel --del priority

My question: what's the best way to make this available to a kickstart? 
Is there anything cobbler can do, or do I need to do something like:

echo > /tmp/initool << \EOF
# script goes here
...
EOF
chmod +x /tmp/initool

R.

[James Cammarata]

Robin Bowes wrote:
>  Hi,
>  
>  I'd like to manipulate yum repo files in the %post section of my 
>  kickstarts, specifically to disable repos that are being provided 
>  through local mirrors.
>  
>  I've knocked together some python code that does this:
>  
>  http://robinbowes.com/article.php/20081026162228424
>  
>  Sample usage:
>  
>  initool --file /etc/yum.repos.d/epel.repo --section epel --option enabled=0
>  initool --file /etc/yum.repos.d/epel.repo --section epel --del priority
>  
>  My question: what's the best way to make this available to a kickstart? 
>  Is there anything cobbler can do, or do I need to do something like: 
You could do it that way, seems like a bit much though when doing this works for me:

sed -i 's/enabled=1/enabled=0/' file.repo

or 

perl -npe 's/enabled=1/enabled=0/' -i file.repo

or to remove a priority line completely:

perl -npe 's/^priority=.*\n$//' -i file.repo

Of course, that will affect all repos in that file (if more than one is listed), but
generally I'm betting that'd be fine since you'd probably want to disable a
similar group all at once.  You could come up with some fancy regex to target only one
repo entry (but that's kind of a PITA to be honest), but this is what I would do.

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

[Robin Bowes]

James Cammarata wrote:
>  Robin Bowes wrote:
> > Hi,
> > 
> > I'd like to manipulate yum repo files in the %post section of my 
> > kickstarts, specifically to disable repos that are being provided 
> > through local mirrors.
> > 
> > I've knocked together some python code that does this:
> > 
> > http://robinbowes.com/article.php/20081026162228424
> > 
> > Sample usage:
> > 
> > initool --file /etc/yum.repos.d/epel.repo --section epel --option
> > enabled=0 initool --file /etc/yum.repos.d/epel.repo --section epel
> > --del priority
> > 
> > My question: what's the best way to make this available to a
> > kickstart? Is there anything cobbler can do, or do I need to do
> > something like:
>  
>  You could do it that way, seems like a bit much though when doing
>  this works for me:
>  
>  sed -i 's/enabled=1/enabled=0/' file.repo
>  
>  or
>  
>  perl -npe 's/enabled=1/enabled=0/' -i file.repo
>  
>  or to remove a priority line completely:
>  
>  perl -npe 's/^priority=.*\n$//' -i file.repo
>  
>  Of course, that will affect all repos in that file (if more than one
>  is listed), but generally I'm betting that'd be fine since you'd
>  probably want to disable a similar group all at once.  You could come
>  up with some fancy regex to target only one repo entry (but that's
>  kind of a PITA to be honest), but this is what I would do. 
I've used the one-liner solution in the past (and still do for simple 
substitutions), and come up with this solution precisely because I want 
to be able to selectively target one repo entry in a file containing 
more than one.

My main question was around making this code available for the kickstart 
process. I want to know if writing it to /tmp/ is my only option (or 
/usr/local/bin, possibly).

R.

[Jeroen van Meeuwen]

Robin Bowes wrote:
>  Hi,
>  
>  I'd like to manipulate yum repo files in the %post section of my 
>  kickstarts, specifically to disable repos that are being provided 
>  through local mirrors.
>  
>  I've knocked together some python code that does this:
>  
>  http://robinbowes.com/article.php/20081026162228424
>  
>  Sample usage:
>  
>  initool --file /etc/yum.repos.d/epel.repo --section epel --option enabled=0
>  initool --file /etc/yum.repos.d/epel.repo --section epel --del priority
>  
>  My question: what's the best way to make this available to a kickstart? 
>  Is there anything cobbler can do, or do I need to do something like:
>   
Package initool and submit it to <insert-distribution-here> so it's 
included in the repositories used, and then make sure it's installed on 
the system by including it in the %packages section.

-Jeroen

[Robin Bowes]

Jeroen van Meeuwen wrote:
>  Robin Bowes wrote:
> > Hi,
> >
> > I'd like to manipulate yum repo files in the %post section of my 
> > kickstarts, specifically to disable repos that are being provided 
> > through local mirrors.
> >
> > I've knocked together some python code that does this:
> >
> > http://robinbowes.com/article.php/20081026162228424
> >
> > Sample usage:
> >
> > initool --file /etc/yum.repos.d/epel.repo --section epel --option enabled=0
> > initool --file /etc/yum.repos.d/epel.repo --section epel --del priority
> >
> > My question: what's the best way to make this available to a kickstart? 
> > Is there anything cobbler can do, or do I need to do something like:
> >
>  
>  Package initool and submit it to <insert-distribution-here> so it's 
>  included in the repositories used, and then make sure it's installed on 
>  the system by including it in the %packages section. 
Yes, I could do that. I could also package it and add it to my private 
yum repo.

I want to know how best to make it available *without* doing that.

R.

[Chris O'Regan]

>  I want to know how best to make it available *without* doing that.
You can drop it on a web server somewhere and wget the file. Or you can 
export a file system via NFS and mount it on your client.


Chris

[Robin Bowes]

Chris O'Regan wrote:
> > I want to know how best to make it available *without* doing
> that.
>  
>  You can drop it on a web server somewhere and wget the file. 
Hmmm, a possibility; is wget guaranteed to be available in the kickstart 
environment?

>  Or you can
>  export a file system via NFS and mount it on your client. 
Possible, but I'd rather not go down this route.

I'll try using wget in a snippet; failing that, I'll just do the echo thing.

Thx,

R.

[Seth Vidal]

On Mon, 2008-10-27 at 21:36 +0000, Robin Bowes wrote:
>  Chris O'Regan wrote:
>  >> I want to know how best to make it available *without* doing that.
>  > 
>  > You can drop it on a web server somewhere and wget the file.
>  
>  Hmmm, a possibility; is wget guaranteed to be available in the kickstart 
>  environment? 
no but urlgrabber is there - it has to be for yum to work.


-sv

[Robin Bowes]

seth vidal wrote:
>  On Mon, 2008-10-27 at 21:36 +0000, Robin Bowes wrote:
> > Chris O'Regan wrote:
> >>> I want to know how best to make it available *without* doing that.
> >> You can drop it on a web server somewhere and wget the file.
> > Hmmm, a possibility; is wget guaranteed to be available in the kickstart 
> > environment?
>  
>  no but urlgrabber is there - it has to be for yum to work. 
Ah, OK. That will work.

So, I can put initool on the cobbler server, e.g. in 
http://example.com/cobbler/tools/ (yes, I created the "tools" directory) 
then create a cobbler snippet:

urlgrabber http://cobbler.example.com/cobbler/tools/initool /tmp/initool
chmod +x /tmp/initool

initool will then be available for use by the rest of the kickstart script.

Thanks for all the input.

R.

[Ronald J. Yacketta]

>  seth vidal wrote:
> > On Mon, 2008-10-27 at 21:36 +0000, Robin Bowes wrote:
> >> Chris O'Regan wrote:
> >>>> I want to know how best to make it available *without* doing that.
> >>> You can drop it on a web server somewhere and wget the file.
> >> Hmmm, a possibility; is wget guaranteed to be available in the
> >> kickstart
> >> environment?
> >
> > no but urlgrabber is there - it has to be for yum to work.
>
>  Ah, OK. That will work.
>
>  So, I can put initool on the cobbler server, e.g. in
>  http://example.com/cobbler/tools/ (yes, I created the "tools" directory)
>  then create a cobbler snippet:
>
>  urlgrabber http://cobbler.example.com/cobbler/tools/initool /tmp/initool
>  chmod +x /tmp/initool
>
>  initool will then be available for use by the rest of the kickstart
>  script.
>
>  Thanks for all the input.
>
>  R.
>
>  _______________________________________________
>  cobbler mailing list
>  cobbler(a)lists.fedorahosted.org
>  https://fedorahosted.org/mailman/listinfo/cobbler
>  

IIRC the cobbler dir under /var/www/cobbler (think that is the default) is
recreated every time you run a cobbler sync. I ran into this when I placed
some files under cobbler/dir to use wget and noticed that the dir was
deleted after a syn.

I ended up creating a /var/www/html/scripts directory to hold my %post fun


-Ron

[Robin Bowes]

Ronald J Yacketta wrote:
>  IIRC the cobbler dir under /var/www/cobbler (think that is the
> default) is
>  recreated every time you run a cobbler sync. I ran into this when I placed
>  some files under cobbler/dir to use wget and noticed that the dir was
>  deleted after a syn.
>  
>  I ended up creating a /var/www/html/scripts directory to hold my %post fun 
Well, blow me - so it does.

Why does cobbler do this? And it does *explicitly* delete everything but 
a set of dirs that "should" exist - the code is in action_sync.py:

             if os.path.isdir(path):
                 if not x in ["web", "webui", 
"localmirror","repo_mirror","ks_mirror","images","links","repo_profile","repo_system","svc","rendered"]

:
                     # delete directories that shouldn't exist
                     utils.rmtree(path)
                 if x in 
["kickstarts","kickstarts_sys","images","systems","distros","profiles","repo_profile","repo_system","rendered"]:
                     # clean out directory contents
                     utils.rmtree_contents(path)



This seems a little draconian to me - why not just clean up the 
directories that cobbler manages, i.e. remove the first clause that 
deletes anything not in the list? What harm is there in leaving them there?

R.

[Robin Bowes]

Robin Bowes wrote:
>  seth vidal wrote:
> > On Mon, 2008-10-27 at 21:36 +0000, Robin Bowes wrote:
> >> Chris O'Regan wrote:
> >>>> I want to know how best to make it available *without* doing that.
> >>> You can drop it on a web server somewhere and wget the file.
> >> Hmmm, a possibility; is wget guaranteed to be available in the kickstart 
> >> environment?
> > no but urlgrabber is there - it has to be for yum to work.
>  
>  Ah, OK. That will work.
>  
>  So, I can put initool on the cobbler server, e.g. in 
>  http://example.com/cobbler/tools/ (yes, I created the "tools" directory) 
>  then create a cobbler snippet:
>  
>  urlgrabber http://cobbler.example.com/cobbler/tools/initool /tmp/initool
>  chmod +x /tmp/initool
>  
>  initool will then be available for use by the rest of the kickstart script. 
Hmmm, or so one would think...

For some reason, this is not working.

I've put the initool script elsewhere on the cobbler server, and have 
switched to use wget, so my %pre snippet looks like this:

# Copy initool to use in post
echo "in tools_pre" > /root/tools_pre
wget "http://dist.robinbowes.com/tools/initool" -O /usr/bin/initool
chmod +x /usr/bin/initool

Then, in %post, I have these blocks of code (both loaded from snippets):

# disable repos that are mirrored by cobbler
/usr/bin/initool --file /etc/yum.repos.d/CentOS-Base.repo --section all 
--option enabled=0
/usr/bin/initool --file /etc/yum.repos.d/epel.repo --section all 
--option enabled=0
/usr/bin/initool --file /etc/yum.repos.d/epel-testing.repo --section all 
--option enabled=0
/usr/bin/initool --file /etc/yum.repos.d/rpmforge.repo --section all 
--option enabled=0

# set repo priority
/usr/bin/initool --file /etc/yum.repos.d/cobbler-config.repo --section 
core-0 --option priority=1
/usr/bin/initool --file /etc/yum.repos.d/cobbler-config.repo --section 
CentOS-5-x86_64-Updates --option priority=10
/usr/bin/initool --file /etc/yum.repos.d/cobbler-config.repo --section 
RPMForge-RHEL5-x86_64 --option priority=99
/usr/bin/initool --file /etc/yum.repos.d/cobbler-config.repo --section 
epel-5-x86_64 --option priority=20


Neither of these blocks of code works - the repo files are not changed.

initool does not exist in /usr/bin, and /root/tools_pre does not exist.

In a flash of inspiration (!) I moved the snippet that copies the 
initool script into %post, cunningly changing "pre" to "post" in the 
process:

# Copy initool to use in post
echo "in tools_post" > /root/tools_post
urlgrabber http://dist.robinbowes.com/tools/initool /usr/bin/initool
chmod +x /usr/bin/initool

This now works!

Anyone got any idea why it didn't work in %pre?

R.

[Daniel Hawker]

Anyone got any idea why it didn't work in %pre?

R.

--

I would think because the %post part of any kickstart actually happens in a chroot jail of
the installed os. %pre and %post -nochroot happen outside of the chroot.

Basically...
%pre and %post -nochroot happen before/after Anaconda is running. Hence things happen on
the mounted initrd based install image.

When %post gets called, it actually chroot logins to the mounted dir structure that
Anaconda installs to (ottomh, /mnt/sysimage/blah or similar). Hence anything in %post
happens within the chroot. Everything outside of the chroot structure is not accessible.
In your case the tools you wgetted.

I've hit this issue in the past when trying to do something similar (not using
Cobbler, just Kickstart and PXE). You just have to be a bit *imaginative* about your %post
scripts :)

Either (as you have done) wget the files from within the chroot or wget in %pre but run
the scripts in %post -nochroot. I'd do the former, the nochroot environment can get
awfully confusing and has a limited toolset welded in the Anaconda initrd image.

HTH

Dan

--

[Robin Bowes]

Daniel Hawker wrote:
>  I would think because the %post part of any kickstart actually
>  happens in a chroot jail of the installed os. %pre and %post
>  -nochroot happen outside of the chroot. 
Ahhhh, that makes sense.

Thanks.

R.

[Chris O'Regan]

>  Anyone got any idea why it didn't work in %pre? 
The %pre section runs after the kickstart file is read but before it is 
fully parsed. The main purpose for it is to fine-tune the kickstart 
file. For example, before cobbler, I use it to set a random root 
password, to generate a static network configuration for my servers, and 
to specify a partition layout (using %include).

The %post section happens after the system has been installed, and is an 
ideal place to customize the system configuration before rebooting.


Chris

[Chris O'Regan]

> > [...] is wget guaranteed to be available in the kickstart 
> > environment?
>  
>  no but urlgrabber is there - it has to be for yum to work. 
Well, cobbler expects wget to be there; many of the built-in snippets 
use it. Just because yum requires urlgrabber does not mean that it is 
guaranteed to be there; not all distros use yum. IMHO, wget is a safer bet.


Chris

[Seth Vidal]

On Tue, 2008-10-28 at 10:08 -0400, Chris O'Regan wrote:
>  >> [...] is wget guaranteed to be available in the kickstart 
>  >> environment?
>  > 
>  > no but urlgrabber is there - it has to be for yum to work.
>  
>  Well, cobbler expects wget to be there; many of the built-in snippets 
>  use it. Just because yum requires urlgrabber does not mean that it is 
>  guaranteed to be there; not all distros use yum. IMHO, wget is a safer bet.
>   
He was talking about kickstart.
kickstart is a mode of anaconda
anaconda requires yum
yum requires urlgrabber

For a fedora/rhel/centos install yum is going to be there, so urlgrabber
will be there.

I do not dispute that wget is a fine choice, i was just trying to help
his specific issue.

no harm intended.
-sv

[Chris O'Regan]

>  He was talking about kickstart.
>  kickstart is a mode of anaconda
>  anaconda requires yum
>  yum requires urlgrabber 
True for modern versions of Red Hat-based distros. Anything prior to 
RHEL5 and CentOS5, for example, do not use yum and will not have 
urlgrabber. (Not sure when yum was introduced to Fedora.)

Other people reading this list might find this topic (fetching scripts 
from the web server) interesting. I was merely pointing out that 'wget' 
is a safer choice, and what the author of cobbler himself has chosen to 
use when communicating with cobbler via the web server.

For a moment I was about to reconfigure my scripts to use urlgrabber, 
too, but then I reminded myself that I still have to support RHEL3 for a 
little while longer. I felt it was worth pointing this out if only to 
save one person reading this list from hours worth of debugging 
wondering why his kickstart script is failing.


Chris

[R P Herrold]

On Tue, 28 Oct 2008, Chris O'Regan wrote:

> > He was talking about kickstart.
> > kickstart is a mode of anaconda
> > anaconda requires yum
> > yum requires urlgrabber
>
>  True for modern versions of Red Hat-based distros. Anything prior to
>  RHEL5 and CentOS5, for example, do not use yum and will not have
>  urlgrabber. (Not sure when yum was introduced to Fedora.) 
ehh?  CentOS has used yum from its inception, and in ** all **
released versions; before that, yum was in use, pushed back to 
RHL 7 series by me locally.

-- Russ herrold

[Seth Vidal]

On Tue, 2008-10-28 at 11:37 -0400, R P Herrold wrote:
>  On Tue, 28 Oct 2008, Chris O'Regan wrote:
>  
>  >> He was talking about kickstart.
>  >> kickstart is a mode of anaconda
>  >> anaconda requires yum
>  >> yum requires urlgrabber
>  >
>  > True for modern versions of Red Hat-based distros. Anything prior to
>  > RHEL5 and CentOS5, for example, do not use yum and will not have
>  > urlgrabber. (Not sure when yum was introduced to Fedora.)
>  
>  ehh?  CentOS has used yum from its inception, and in ** all **
>  released versions; before that, yum was in use, pushed back to 
>  RHL 7 series by me locally. 
Agreed - but if they're using rhel from rhn or what not (and not getting
anything from epel or centos like a HUGE number of people do) then they
won't have it by default.

-sv

[Chris O'Regan]

>  ehh?  CentOS has used yum from its inception, and in ** all **
>  released versions; before that, yum was in use, pushed back to 
>  RHL 7 series by me locally. 
Apologies, I only starting using CentOS at v5 and just assumed that it 
suffered from the same yumlessness as RHEL4 and less.


Chris