problem with https://fedoraproject.org/en/verify

[Richard Z]

Hi,

sorry to use this contact address, was not sure where else to direct this.

The procedure in this page has one significant stumbling point - the *-CHECKSUM
file is a gzipped file - at least when it is downloaded using wget.
( I have tested it with https://fedoraproject.org/static/checksums/Fedora-12-i386-CHECKSUM
)

Obviously it needs to unzipped - which is not quickly obvious for a bunch of reasons,
eg it has no ".gz" suffix and many editors/viewers will unzip it automatically
so the
unsuspecting user is left scratching his head why it doesnt work when the file looks
perfectly ok when viewed with "less"

The result are rather cryptic error messages like
# gpg --verify *-CHECKSUM
gpg: no valid OpenPGP data found.
gpg: the signature could not be verified.
Please remember that the signature file (.sig or .asc)
should be the first file given on the command line.

Regards
Richard

[Ricky Zhou]

On 2009-12-31 10:24:53 PM, Richard Zidlicky wrote:
>  sorry to use this contact address, was not sure where else to direct
>
>  this. 
No problem at all, this is the right place to report website issues like 
this.

>  The procedure in this page has one significant stumbling point - the
> *-CHECKSUM
>  file is a gzipped file - at least when it is downloaded using wget.
>  ( I have tested it with
> https://fedoraproject.org/static/checksums/Fedora-12-i386-CHECKSUM ) 
That definitely
shouldn't be a gzipped file - does the file command say 
that it is one?  This is what the file should look like:

[ricky@gamma ~]$ wget https://fedoraproject.org/static/checksums/Fedora-12-i386-CHECKSUM
--2009-12-31 16:34:26-- 
https://fedoraproject.org/static/checksums/Fedora-12-i386-CHECKSUM
Resolving fedoraproject.org... 152.46.7.221, 66.35.62.162, 80.239.156.214, ...
Connecting to fedoraproject.org|152.46.7.221|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1511 (1.5K) [text/plain]
Saving to: “Fedora-12-i386-CHECKSUM”

100%[======================================================>] 1,511       --.-K/s   in
0s

2009-12-31 16:34:27 (5.54 MB/s) - “Fedora-12-i386-CHECKSUM” saved [1511/1511]

[ricky@gamma ~]$ cat Fedora-12-i386-CHECKSUM
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

f0ad929cd259957e160ea442eb80986b5f01daaffdbcc7e5a1840a666c4447c7 *Fedora-12-i386-DVD.iso
2f548ce50c459a0270e85a7d63b2383c55239bf6aead9314a0f887f3623ddace
*Fedora-12-i386-disc1.iso
ce77d16d1b3362859aaa856f1f29c7197db69264d8ce6b9f8111dcee4d5e9ef7
*Fedora-12-i386-disc2.iso
8c39cb9e3c1583948dcad21f9fdbe48a3ff6a8d1b536462188d47747c2640b36
*Fedora-12-i386-disc3.iso
07f03f67d23331e8c7a37ad19e9a99062a4584a3e028beb40c49923bb5c70c6b
*Fedora-12-i386-disc4.iso
dff8c478fb73452a8799016deeecccde3097d40a0b756d681bfe6be2e56bb9eb
*Fedora-12-i386-disc5.iso
128112527bdd4036ec82d678b5d5362aa7a11ac15a73647afd743d7a325f7df9
*Fedora-12-i386-netinst.iso
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.5 (GNU/Linux)
>
> iQIVAwUBSvurkZ0cw0hXu8y6AQIdQw//WuT1eE5LUzN3tBnBJzMsvD90/gz1kM0A
> 4qtM+SSRjrx0MwkVkP5spO/xfkk7sncTE51Bl88lDAvpC/00b+u3MQEya9aApZyT
> CmggKB/bmozQyX3C7HbXwUIMrCRmNVkYCkgQKLQd/MK+r73dXCuHNpyfeBSuZGsy
> iCpX003Wu6U92jlwljBkgU+FrgJwAmr6b7hEurQaf2fqmN1d4Nh+llwqOEIykd5A
> Ci1ApI05NBEX/z9KG+WR+YtCuRqUwD6U5SrjBSQD86NGLcsJ49gBrbu1um3cUvlC
> YRvCjT4zDBn32au+pBKXjlQf4TrCt3SooYnmf0D+1iefrN0Sijpft+bQ26poSjkp
> pj+wnVkUg2shfm+0imiPIGos6cJRmj0o4w3CzyDs6sOIcIcYB4ohyFasczsjYT40
> LSCcKBFZXNEw8OogcoPZpp79Yr7iX0C0JQ45xgzPrDegKSLVkTvpXyHCbmd21Zkz
> oPu2kFoR+tEVPfESVFqSqnYJC/TtwokEHbaVCUEpP44L3PpGiVTqK/uZnReQRbLM
> ZuMtXRa2j3i0iSlEKfAS0L+9mvWzGzp8UOQzH7UyZgb0RKfVRYcHW0oXpfMqFD9C
> IA/0pgDQNnQRq3OPxnjHfNKAtezfNBaaU45xA9gA2olzzVrhzgXKjn3MRK2tyrlA
> XpaHoVKUVFU=
> =HttN
> -----END PGP SIGNATURE----- 
Thanks,
Ricky

[Richard Z]

On Thu, Dec 31, 2009 at 04:35:21PM -0500, Ricky Zhou wrote:
>  On 2009-12-31 10:24:53 PM, Richard Zidlicky wrote:
>  > sorry to use this contact address, was not sure where else to direct 
>  > this.
>  No problem at all, this is the right place to report website issues like 
>  this.
>  
>  > The procedure in this page has one significant stumbling point - the *-CHECKSUM
>  > file is a gzipped file - at least when it is downloaded using wget.
>  > ( I have tested it with
> https://fedoraproject.org/static/checksums/Fedora-12-i386-CHECKSUM )
>  That definitely shouldn't be a gzipped file - does the file command say 
>  that it is one?  This is what the file should look like: 
thats what happens here:

[rz@rz rai]$ wget -c --proxy=off
https://fedoraproject.org/static/checksums/Fedora-12-i386-CHECKSUM
--2009-12-31 22:58:03-- 
https://fedoraproject.org/static/checksums/Fedora-12-i386-CHECKSUM
Resolving fedoraproject.org... 80.239.156.214, 152.46.7.221, 66.35.62.162, ...
Connecting to fedoraproject.org|80.239.156.214|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1078 (1.1K) [text/plain]
Saving to: “Fedora-12-i386-CHECKSUM”

100%[======================================>] 1,078       --.-K/s   in 0.1s    

2009-12-31 22:58:10 (8.79 KB/s) - “Fedora-12-i386-CHECKSUM” saved [1078/1078]


[rz@rz rai]$ file Fedora-12-i386-CHECKSUM 
Fedora-12-i386-CHECKSUM: gzip compressed data, from Unix

[rz@rz rai]$ od -c Fedora-12-i386-CHECKSUM |head
0000000 037 213  \b  \0  \0  \0  \0  \0  \0 003   m 223   K 227 252   8
0000020 024   F 347 374  \n 207 335 355 252   k   H   B 036   w 255   ;
0000040  \0 304   G 251 224 210   X 352   , 344   a   !   *  \b 242   %
0000060 277 276 271 325   = 253 233   i 262 366   9 337 316   9   /   /
0000100 335 361 202 361   4 354   - 307 313   ^   < 035 207 301 260 267
0000120  \b 342 330 035  \a 277 357   ^ 254 211 250   ?   ~ 366 342 211
0000140   k   [ 226 001   B   q 310 245 202 016 347 016 325   6 001   Z
0000160   `  \f   u 312  \0   g   $   u  \f 260 225 020 306 250   T   J
0000200 252 035   a   3  \f 004   !   D   b 214 251 244 275 177   F   Z
0000220 025 225   x 261 341   K 206 030   y 031   n 206   ? 262 272 260


Richard

[Ricky Zhou]

On 2009-12-31 11:01:02 PM, Richard Zidlicky wrote:
>  thats what happens here:
>  
>  [rz@rz rai]$ wget -c --proxy=off
> https://fedoraproject.org/static/checksums/Fedora-12-i386-CHECKSUM
>  --2009-12-31 22:58:03-- 
> https://fedoraproject.org/static/checksums/Fedora-12-i386-CHECKSUM
>  Resolving fedoraproject.org... 80.239.156.214, 152.46.7.221, 66.35.62.162, ...
>  Connecting to fedoraproject.org|80.239.156.214|:443... connected.
>  HTTP request sent, awaiting response... 200 OK
>  Length: 1078 (1.1K) [text/plain]
>  Saving to: “Fedora-12-i386-CHECKSUM”
>  
>  100%[======================================>] 1,078       --.-K/s   in 0.1s    
>  
>  2009-12-31 22:58:10 (8.79 KB/s) - “Fedora-12-i386-CHECKSUM” saved [1078/1078]
>  
>  
>  [rz@rz rai]$ file Fedora-12-i386-CHECKSUM 
>  Fedora-12-i386-CHECKSUM: gzip compressed data, from Unix 
Is there anything between
you and th eserver that could be affecting 
this?  Also, does this happen as well when you view it in a browser?

Just out of curiosity, what does this file decompress to, if anything?

Thanks,
Ricky

[Richard Z]

On Thu, Dec 31, 2009 at 05:07:30PM -0500, Ricky Zhou wrote:

>  > [rz@rz rai]$ file Fedora-12-i386-CHECKSUM 
>  > Fedora-12-i386-CHECKSUM: gzip compressed data, from Unix
>  Is there anything between you and th eserver that could be affecting 
>  this?  Also, does this happen as well when you view it in a browser? 
there is the NAT firewall of my umts provider (O2-Germany). Actually one of 
the less intrusive providers, even allows VoIP and everything else.

The browser displays the expected content in cleartext. Size 1078 bytes, quirks 
mode rendering, verified by equifax "site does not provide ownership
information"
AES-256, CN=fedoraproject.org.

>  Just out of curiosity, what does this file decompress to, if
> anything? 
it does decompress to the exact same as the browser displays and gpg-verify
as well as sha256sum give the expected results.

So I do not think O2 would be running MIM attacks, it could be something in
wget-1.12-2.fc10 headers that causes a misunderstanding with the server regarding
to compression?

Just checked curl with the same URL and it gives the cleartext content to 
stdout. Wget the compressed stuff again..

Can you look into your logs for 82.113.121.184, should be firefox, curl and wget
accesses. Its a NAT so you may get many more.

Richard

[Ricky Zhou]

>  Just checked curl with the same URL and it gives the cleartext
> content 
>  to stdout. Wget the compressed stuff again..
>  
>  Can you look into your logs for 82.113.121.184, should be firefox, curl and wget
>  accesses. Its a NAT so you may get many more. 
I couldn't find anything useful
in the logs, but I was able to reproduce 
the issue using:

wget --header="Accept-Encoding: gzip"
https://fedoraproject.org/static/checksums/Fedora-12-i386-CHECKSUM

I'll try to look more into why wget isn't decompressing the result 
properly.

Thanks,
Ricky

[Ricky Zhou]

On 2009-12-31 06:25:05 PM, Ricky Zhou wrote:
>  wget --header="Accept-Encoding: gzip" 
>  https://fedoraproject.org/static/checksums/Fedora-12-i386-CHECKSUM 
Looking a bit
more into this, I found 
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=150766, which says that 
wget does not support Content-Encoding: gzip, which is an HTTP 1.1 
feature.  Can you give me the output of:

wget -d https://fedoraproject.org/static/checksums/Fedora-12-i386-CHECKSUM

I'm curious to see if Accept-Encoding: gzip is being sent or 
Content-Encoding: gzip is coming back.

Thanks,
Ricky

[Richard Z]

On Thu, Dec 31, 2009 at 06:30:05PM -0500, Ricky Zhou wrote:
>  On 2009-12-31 06:25:05 PM, Ricky Zhou wrote:
>  > wget --header="Accept-Encoding: gzip" 
>  > https://fedoraproject.org/static/checksums/Fedora-12-i386-CHECKSUM
>  Looking a bit more into this, I found 
>  http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=150766, which says that 
>  wget does not support Content-Encoding: gzip, which is an HTTP 1.1 
>  feature.  Can you give me the output of:
>  
>  wget -d https://fedoraproject.org/static/checksums/Fedora-12-i386-CHECKSUM 
I should kick my butt - have completely forgotten that I set user agent and other headers
in .wgetrc to emulate dillo, which did accept gzip encoding.

There is probably nothing you could do to help users like this, not even sure it 
is worth considering this as wget bug. 

Happy New Year

>  I'm curious to see if Accept-Encoding: gzip is being sent or 
>  Content-Encoding: gzip is coming back. 
both

---request begin---
GET /static/checksums/Fedora-12-i386-CHECKSUM HTTP/1.0
User-Agent: Dillo/0.8.4-i18n-misc
Accept: */*
Host: fedoraproject.org
Connection: Keep-Alive
Accept-Encoding: gzip
Accept-Charset: utf-8,ISO-8859-1,ISO-8859-2,ISO-8859-9,ISO-8859-15

---request end---
HTTP request sent, awaiting response... 
---response begin---
HTTP/1.1 200 OK
Date: Fri, 01 Jan 2010 17:27:32 GMT
Server: Apache/2.2.3 (Red Hat)
Last-Modified: Fri, 01 Jan 2010 17:04:43 GMT
Accept-Ranges: bytes
Cache-Control: max-age=604800
Expires: Fri, 08 Jan 2010 17:27:32 GMT
Vary: Accept-Encoding,User-Agent
Content-Encoding: gzip
ProxyTime: D=728
ProxyServer: proxy3.fedoraproject.org
Content-Length: 1078
Keep-Alive: timeout=15, max=500
Connection: Keep-Alive
Content-Type: text/plain; charset=utf-8

---response end---
200 OK
Registered socket 3 for persistent reuse.

[Ricky Zhou]

On 2010-01-01 06:39:19 PM, Richard Zidlicky wrote:
>  I should kick my butt - have completely forgotten that I set user 
>  agent and other headers
>  in .wgetrc to emulate dillo, which did accept gzip encoding.
>  
>  There is probably nothing you could do to help users like this, not even sure it 
>  is worth considering this as wget bug.  
Ah, that explains why I couldn't find
any entries for wget in the logs - 
thanks for getting this cleared up.

>  Happy New Year 
Same to you!

Thanks,
Ricky