FW: [Fedora Infrastructure] #1719: MM hotfix: crawler incorrectly creating dir entries for not up-to-date mirrors

[Ricky Zhou]

On 2009-10-05 10:40:31 AM, Matt_Domsch at Dell.com wrote:
> Requesting +1s to apply this hotfix, which only affects the MM crawler.
> 
> --
> Matt Domsch
> Technology Strategist, Dell Office of the CTO
> linux.dell.com & www.dell.com/linux
> 
> 
> 
> 
> -----Original Message-----
> From: Fedora Infrastructure [mailto:trac at fedorahosted.org]
> Sent: Mon 10/5/2009 10:38 AM
> To: mdomsch at fedoraproject.org
> Cc: ricky at fedoraproject.org; nb at fedoraproject.org
> Subject: [Fedora Infrastructure] #1719: MM hotfix: crawler incorrectly creating dir entries for not up-to-date mirrors
>  
> #1719: MM hotfix: crawler incorrectly creating dir entries for not up-to-date
> mirrors
> -----------------------------+----------------------------------------------
>  Reporter:  mdomsch          |       Owner:  mdomsch   
>      Type:  bug              |      Status:  new       
>  Priority:  major            |   Milestone:  Fedora 12 
> Component:  Web Application  |     Version:  Production
>  Severity:  Normal           |    Keywords:  hotfix    
> -----------------------------+----------------------------------------------
>  = phenomenon =
>  nb reported that running report_mirror would report hundreds of
>  directories deleted on each run.
> 
>  = reason =
>  crawler is creating HostCategoryDir entries for not up-to-date directories
>  (in fact, dirs that nb has excluded on his mirror).  report_mirror then
>  deletes these entries.
> 
> 
>  = recommendation =
>  commit 20986e503db481d4760e6e3ea74b07863a8e1cf9
>  Author: Matt Domsch <matt at domsch.com>
>  Date:   Sun Oct 4 22:25:15 2009 -0500
> 
>      crawler: don't create HCDs for directories which aren't up2date
> 
>  diff --git a/server/crawler_perhost b/server/crawler_perhost
>  index 9553fb8..8964c7e 100755
>  --- a/server/crawler_perhost
>  +++ b/server/crawler_perhost
>  @@ -303,6 +303,9 @@ def sync_hcds(host, host_category_dirs):
>           if hcd.count() > 0:
>               hcd = hcd[0]
>           else:
>  +            # don't create HCDs for directories which aren't up2date on
>  the mirror
>  +            # chances are the mirror is excluding that directory
>  +            if not up2date: continue
>               hcd = HostCategoryDir(host_category=hc, path=path,
>  directory=d)
> 
>           if hcd.directory is None:
> 
> -- 
+1

Thanks,
Ricky
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
Url : http://lists.fedoraproject.org/pipermail/infrastructure/attachments/20091005/504eee15/attachment.bin