[PATCH] avoid \xZZ in mirrorlist urls

[Matt Domsch]

On Wed, May 12, 2010 at 08:59:18PM -0500, Matt Domsch wrote:
> On Wed, May 12, 2010 at 08:56:12PM -0500, Matt Domsch wrote:
> > > Hey is there a way to test this on staging first to make sure it grabs
> > > the URLs. RewriteRules's make my head hurt and I get things backwards
> > > all the time.
> > 
> > In staging, I can test whether or not \x gets caught.  I can't test
> > whether it'll catch the ones that are causing the problems from
> > outside users though.  (If I could, I would have been able to find and
> > fix the root cause, which clearly this isn't).
> > 
> > I'm open to other ideas.  Perhaps what I see in the URL logs isn't
> > what's actually being sent?  I don't know...
> 
> Here's the symptom.  admin.fp.o/status/app01
> 
> 16-0					    30536   0/1113/1113	W   18.42   11020  0   0.0   10.07  10.07	192.168.1.7		     app01.phx2.fedoraproject.org		     GET /mirrorlist?repo=\xc2\xadfedora-8&arch=i386 HTTP/1.0
> 63-0					    30713   0/2151/2151	W   38.47   3819 0 0.0 21.13 21.13  192.168.1.7 app01.phx2.fedoraproject.org GET /mirrorlist?repo=\xc2\xadfedora-8&arch=i386 HTTP/1.0
> 
> In both cases, the jobs are in W state from apache's POV (so
> responding to the client request), see how long they've been sitting
> there (11020 and 3819 seconds respectively), and the format of the
> query args.  I send the same thing via wget, and it succeeds, no failure.

OK, having spent a few more hours, I don't need the RewriteRule. Yea!

I just need to convert to unicode, and leave it in unicode.  Malformed
requests fail lookups as would be expected and return the "you asked
for an invalid repo or arch" message.  Good requests return the
mirrorlist.  All is swimmingly.


commit 9251a20e8ff8239bae2c74e64ad20a4645423ae9
Author: Matt Domsch <Matt_Domsch at dell.com>
Date:   Wed May 12 22:59:46 2010 -0500

    mirrorlist_client: leave query params as utf8

diff --git a/mirrorlist-server/mirrorlist_client.wsgi
b/mirrorlist-server/mirrorlist_client.wsgi
index 3508f19..36077a1 100755
--- a/mirrorlist-server/mirrorlist_client.wsgi
+++ b/mirrorlist-server/mirrorlist_client.wsgi
@@ -93,7 +93,7 @@ def request_setup(request):
 
     for k, v in d.iteritems():
         try:
-            d[k] = unicode(v, 'utf8', 'ignore').encode('utf8')
+            d[k] = unicode(v, 'utf8', 'replace')
         except:
             pass
     return d



-- 
Matt Domsch
Technology Strategist
Dell | Office of the CTO