[PATCH] Don't mess up ctime during cobbler reposync

[Robert Vogelgesang]

Hello,

cobbler trac issue #579 is now resolved, but as I've noted there,
cobbler does mess up the ctime timestamps of a mirrored repo during
the reposync action, which could mislead "createrepo --update --C".

To get this resolved, too, I've created the attached patch.  The
patch limits the ctime changing actions to the bare minimum.
This was confirmed by the logging I've added for debugging #579.

But there is a futher bug, and as I'm by no means a real python
programmer, I'm currently unable to tell if there's something
wrong with my patch, or if it's a bug in the logging support.

The problem: The commands are executed as written, but only the parts
of the commands after the pipe symbol are logged; i. e. for cmd1 only
"xargs -0 chown root:apache" is logged, and for cmd2 only
"xargs -0 chmod a+rX".  I've done a cursory search in the standard
Python docs for any special meaning of the pipe symbol, but found
nothing.

So I'm asking here for help, to resolve this strange logging problem.
Maybe this is an issue with cobbler's use of python's logging?

	Robert

PS.: The patch changes next to nothing with regard to what other
packages are required for cobbler; the findutils package is
required by the initscripts and man packages, and restorecon
comes with policycoreutils which are required for SELinux.

[Scott Henson]

Excerpts from Robert Vogelgesang's message of Fri Apr 30 04:08:59 -0400 2010:
>  Hello,
>  
>  cobbler trac issue #579 is now resolved, but as I've noted there,
>  cobbler does mess up the ctime timestamps of a mirrored repo during
>  the reposync action, which could mislead "createrepo --update --C".
>  
>  To get this resolved, too, I've created the attached patch.  The
>  patch limits the ctime changing actions to the bare minimum.
>  This was confirmed by the logging I've added for debugging #579.
>  
>  But there is a futher bug, and as I'm by no means a real python
>  programmer, I'm currently unable to tell if there's something
>  wrong with my patch, or if it's a bug in the logging support.
>  
>  The problem: The commands are executed as written, but only the parts
>  of the commands after the pipe symbol are logged; i. e. for cmd1 only
>  "xargs -0 chown root:apache" is logged, and for cmd2 only
>  "xargs -0 chmod a+rX".  I've done a cursory search in the standard
>  Python docs for any special meaning of the pipe symbol, but found
>  nothing.
>  
>  So I'm asking here for help, to resolve this strange logging problem.
>  Maybe this is an issue with cobbler's use of python's logging? 
Cobbler uses subprocess and if you look at utils.subprocess_call, you
will see it calling Popen.  Basically this won't honor pipes in the
output.  You have to chain together calls to subprocess.Popen to do
pipes.

http://docs.python.org/library/subprocess.html#replacing-shell-pipeline

Makes it a little difficult to do with cobbler's implementation.  I'm
thinking we make call_subprocess smart enough to pipe things together
when its given a list, and just do the normal thing when given a
string?
-- 
Scott Henson
Red Hat CIS Operator
WVU Alum BSAE/BSME

[Benjamin Riggs]

On 4/30/2010 12:00 PM, Scott Henson wrote:
>  Excerpts from Robert Vogelgesang's message of Fri Apr 30 04:08:59
> -0400 2010:
>     
> > Hello,
> >
> > cobbler trac issue #579 is now resolved, but as I've noted there,
> > cobbler does mess up the ctime timestamps of a mirrored repo during
> > the reposync action, which could mislead "createrepo --update --C".
> >
> > To get this resolved, too, I've created the attached patch.  The
> > patch limits the ctime changing actions to the bare minimum.
> > This was confirmed by the logging I've added for debugging #579.
> >
> > But there is a futher bug, and as I'm by no means a real python
> > programmer, I'm currently unable to tell if there's something
> > wrong with my patch, or if it's a bug in the logging support.
> >
> > The problem: The commands are executed as written, but only the parts
> > of the commands after the pipe symbol are logged; i. e. for cmd1 only
> > "xargs -0 chown root:apache" is logged, and for cmd2 only
> > "xargs -0 chmod a+rX".  I've done a cursory search in the standard
> > Python docs for any special meaning of the pipe symbol, but found
> > nothing.
> >
> > So I'm asking here for help, to resolve this strange logging problem.
> > Maybe this is an issue with cobbler's use of python's logging?
> >      
>  Cobbler uses subprocess and if you look at utils.subprocess_call, you
>  will see it calling Popen.  Basically this won't honor pipes in the
>  output.  You have to chain together calls to subprocess.Popen to do
>  pipes.
>
>  http://docs.python.org/library/subprocess.html#replacing-shell-pipeline
>
>  Makes it a little difficult to do with cobbler's implementation.  I'm
>  thinking we make call_subprocess smart enough to pipe things together
>  when its given a list, and just do the normal thing when given a
>  string?
>      
Perhaps a more elegant (and robust) solution would be to use os.chown 
and os.path.walk.

-- 
Benjamin Riggs
System Staff
School of Mathematics
University of Minnesota
11 Vincent Hall
612-625-4895

[Robert Vogelgesang]

On Fri, Apr 30, 2010 at 01:00:02PM -0400, Scott Henson wrote:
>  Excerpts from Robert Vogelgesang's message of Fri Apr 30 04:08:59
> -0400 2010:
>  > 
>  > The problem: The commands are executed as written, but only the parts
>  > of the commands after the pipe symbol are logged; i. e. for cmd1 only
>  > "xargs -0 chown root:apache" is logged, and for cmd2 only
>  > "xargs -0 chmod a+rX".  I've done a cursory search in the standard
>  > Python docs for any special meaning of the pipe symbol, but found
>  > nothing.
>  > 
>  > So I'm asking here for help, to resolve this strange logging problem.
>  > Maybe this is an issue with cobbler's use of python's logging?
>  
>  Cobbler uses subprocess and if you look at utils.subprocess_call, you
>  will see it calling Popen.  Basically this won't honor pipes in the
>  output.  You have to chain together calls to subprocess.Popen to do
>  pipes.
>  
>  http://docs.python.org/library/subprocess.html#replacing-shell-pipeline
>  
>  Makes it a little difficult to do with cobbler's implementation.  I'm
>  thinking we make call_subprocess smart enough to pipe things together
>  when its given a list, and just do the normal thing when given a
>  string? 
subprocess.Popen optionally uses the shell to execute the command
string, and util.subprocess_call does already enable the shell when
calling subprocess.Popen.

This is why the commands *are* doing what they are supposed to do,
as I wrote in my initial mail.

The problem with my previous patch is just that the "left hand side"
of the pipe does not get logged.  A possible, but less efficient,
solution is to use find(1)'s -exec action instead of a pipe.
I've attached a patch that implements this.

	Robert

>  -- 
>  Scott Henson
>  Red Hat CIS Operator
>  WVU Alum BSAE/BSME
>  _______________________________________________
>  cobbler-devel mailing list
>  cobbler-devel(a)lists.fedorahosted.org
>  https://fedorahosted.org/mailman/listinfo/cobbler-devel