[RETRY #2] Web100 kernel patch on fedora stock kernels

[Angelo]

Quoting myself from an year ago: (19 October 2004)
--------------------------------------------------------
Hi all,
Linux TCP/IP protocol stack is well built but it lacks a bit in
performance.

To give you an idea, my box has a 10MBit connection to the internet, and
if I try to transfer a file to a little lagged but high bandwidth host
(50ms / 100MBit) this lack of optimization will begin showing.

Even if my link will permit to transfer from this host at about
900-1000KB/s I will get only 600-650KB/s. Even using windows tuned to
broadband connection I will get 900KB/s.

This seems to be related to the tcp window scaling, and even if i try to
tune the kernel related parameters (receive/send buffer) logging the
traffic the tcp window seems to be left too small for this transfers.

Web100 (www.web100.org) is a project that has solved this issue creating
a patch that will tune every tcp/ip connection to get the best performance.

I hope this patch will be inserted in the fedora stock kernels.

Regards,
Angelo
-----------------------------------------------------

In a year not a single reply.. i'm still hoping to get this in

Any idea, comment, reason to disagree, something ?

Regards,
Angelo

[Alan Cox]

On Wed, Oct 12, 2005 at 06:08:06PM +0200, Angelo wrote:
>  In a year not a single reply.. i'm still hoping to get this in
>  Any idea, comment, reason to disagree, something ? 
Try netdev(a)oss.sgi.com. Fedora tracks the base kernel.

[Angelo]

Alan Cox wrote:

> Try netdev(a)oss.sgi.com. Fedora tracks the base kernel.
>  
i'll try.. i hoped that it could be inserted in the long list of already 
added patches to the vanilla kernel

regards,
Angelo

[Rahul Sundaram]

Angelo wrote:

>  Alan Cox wrote:
>
> > Try netdev(a)oss.sgi.com. Fedora tracks the base kernel.
> >
>  i'll try.. i hoped that it could be inserted in the long list of 
>  already added patches to the vanilla kernel
>
>  regards,
>  Angelo 
If you look at the existing patches generally do not add new features 
but contain bug fixes in order to fix the problem sooner and usually are 
later merged into the upstream kernel. In general, working with the 
upstream projects would mean that there is better integration, testing 
and review within the projects and less deviation and maintenance issues 
at the distribution level.  If the patch list in the kernel is long then 
we need to work on fixing that rather than use it as a justification to 
make it even worse.

regards
Rahul

[Kenneth Porter]

--On Thursday, October 13, 2005 4:47 AM +0530 Rahul Sundaram 
<sundaram(a)redhat.com&gt; wrote:

>  If you look at the existing patches generally do not add new features
> but
>  contain bug fixes in order to fix the problem sooner and usually are
>  later merged into the upstream kernel. 
Is there some way to tell for each patch where it came from and what it's 
status is upstream? Like a kernel.org bugzilla entry (or whatever they 
use). How does the kernel maintainer know when it's time to pull a patch 
out of the RPM?

[Dave Jones]

On Wed, Oct 12, 2005 at 04:20:22PM -0700, Kenneth Porter wrote:

>  >If you look at the existing patches generally do not add new
> features but
>  >contain bug fixes in order to fix the problem sooner and usually are
>  >later merged into the upstream kernel.
>  
>  Is there some way to tell for each patch where it came from and what it's 
>  status is upstream? Like a kernel.org bugzilla entry (or whatever they 
>  use). How does the kernel maintainer know when it's time to pull a patch 
>  out of the RPM? 
It's usually time to pull it when it no longer applies to the latest upstream :-)
The actual patchfiles contain info at the top of the file (or at least a
pointer to the bugzilla they're fixing) in most cases, though some of the
'legacy' patches don't have any description yet.

		Dave

[Rahul Sundaram]

Dave Jones wrote:

> On Wed, Oct 12, 2005 at 04:20:22PM -0700, Kenneth Porter wrote:
>
>  > >If you look at the existing patches generally do not add new features but
>  > >contain bug fixes in order to fix the problem sooner and usually are
>  > >later merged into the upstream kernel.
>  > 
>  > Is there some way to tell for each patch where it came from and what it's 
>  > status is upstream? Like a kernel.org bugzilla entry (or whatever they 
>  > use). How does the kernel maintainer know when it's time to pull a patch 
>  > out of the RPM?
>
> It's usually time to pull it when it no longer applies to the latest upstream :-)
> The actual patchfiles contain info at the top of the file (or at least a
> pointer to the bugzilla they're fixing) in most cases, though some of the
> 'legacy' patches don't have any description yet.
>
> 		Dave
>
>   
>  
It would be useful to add description to those legacy patches too and 
maintain the current status in a URL and kill the Fedora kernels are 
heavily patched myth while providing technically capable or merely 
curious users more information on the patches.

regards
Rahil

[Dave Jones]

On Thu, Oct 13, 2005 at 05:11:03AM +0530, Rahul Sundaram wrote:

>  >The actual patchfiles contain info at the top of the file (or at
> least a
>  >pointer to the bugzilla they're fixing) in most cases, though some of the
>  >'legacy' patches don't have any description yet.
>  >
>  It would be useful to add description to those legacy patches too  
Sure. I'm getting to them one by one, but I've more important things
to fix tbh.

>  and maintain the current status in a URL and kill the Fedora kernels
> are 
>  heavily patched myth while providing technically capable or merely 
>  curious users more information on the patches. 
My http://people.redhat.com/davej/kernels/Fedora/*/ pages have a list
of patches in each kernel, though they are somewhat out of date now.

At some point the patches in CVS will all be documented, and
I can run some script nightly that auto-generates those text files.

		Dave

[Rahul Sundaram]

Dave Jones wrote:

> On Thu, Oct 13, 2005 at 05:11:03AM +0530, Rahul Sundaram wrote:
>
>  > >The actual patchfiles contain info at the top of the file (or at least a
>  > >pointer to the bugzilla they're fixing) in most cases, though some of the
>  > >'legacy' patches don't have any description yet.
>  > >
>  > It would be useful to add description to those legacy patches too 
>
> Sure. I'm getting to them one by one, but I've more important things
> to fix tbh.
>
>   
>  
Right. This is just making the information better documented and  
transparent to end users. Has lower priority to fixing any actual 
problems obviously.

>  > and maintain the current status in a URL and kill the Fedora
> kernels are 
>  > heavily patched myth while providing technically capable or merely 
>  > curious users more information on the patches.
>
> My http://people.redhat.com/davej/kernels/Fedora/*/ pages have a list
> of patches in each kernel, though they are somewhat out of date now.
>
> At some point the patches in CVS will all be documented, and
> I can run some script nightly that auto-generates those text files.
>   
>  
That would be great. Thanks

regards
Rahul

[Dimi Paun]

From: "Dave Jones" <davej(a)redhat.com&gt;
>  My http://people.redhat.com/davej/kernels/Fedora/*/ pages have a
> list
>  of patches in each kernel, though they are somewhat out of date now. 
Looking at:

http://people.redhat.com/davej/kernels/Fedora/FC4/kernel-2.6.13-1.1529_FC...

I see this:
+ echo 'Patch #1021 (linux-2.6-debug-sleep-in-irq-warning.patch):'
+ echo 'Patch #1900 (linux-2.6-obsolete-idescsi-warning.patch):'
+ echo 'Patch #1901 (linux-2.6-obsolete-oss-warning.patch):'
+ echo 'Patch #801 (linux-2.6-build-userspace-headers-warning.patch):'
/usr/include/asm/byteorder.h:6:2: warning: #warning using private kernel
header; include <endian.h> instead!
/usr/lib/bison.simple:164:5: warning: "YYMAXDEPTH" is not defined
Patch #1021 (linux-2.6-debug-sleep-in-irq-warning.patch):
Patch #1900 (linux-2.6-obsolete-idescsi-warning.patch):
Patch #1901 (linux-2.6-obsolete-oss-warning.patch):
Patch #801 (linux-2.6-build-userspace-headers-warning.patch):

Does this mean that 2.6.13-1.1529 in FC4 has only 4 patches on top of
the upstream kernel?

>  At some point the patches in CVS will all be documented, and
>  I can run some script nightly that auto-generates those text files. 
That would be very cool indeed, I was just trying to convince a
friend that FC4 kernels are rather standard. It would have come
in handy :)

-- 
Dimi Paun <dimi(a)lattica.com&gt;
Lattica, Inc.

[Jason L Tibbitts III]

> >>>> "DP" == Dimi Paun <dimi(a)lattica.com&gt;
> writes: 
DP> Does this mean that 2.6.13-1.1529 in FC4 has only 4 patches on top
DP> of the upstream kernel?

I count 101 applied patches.  You can check the current kernel
packages out out Fedora CVS and see for yourself:

fedora-cvs/kernel/FC-4> grep '^%patch' kernel-2.6.spec |wc -l
101

Current rawhide has only 81.  As I understand it, the plan is to tend
towards zero but of course that will not be possible in general.

 - J<

[Rahul Sundaram]

Jason L Tibbitts III wrote:

> >>>>>"DP" == Dimi Paun
> <dimi(a)lattica.com&gt; writes:
> >>>>>            
> >>>>>
>
> DP> Does this mean that 2.6.13-1.1529 in FC4 has only 4 patches on top
> DP> of the upstream kernel?
>
> I count 101 applied patches.  You can check the current kernel
> packages out out Fedora CVS and see for yourself:
>
> fedora-cvs/kernel/FC-4> grep '^%patch' kernel-2.6.spec |wc -l
> 101
>  
Counting the number of occurances of the word is misleading since it 
appears in the spec comments several times. You need to look into the 
source rpm or http://cvs.fedora.redhat.com and count the actual number 
of patches.

regards
Rahul

[Jason L Tibbitts III]

> >>>> "RS" == Rahul Sundaram
> <sundaram(a)redhat.com&gt; writes: 
RS> Counting the number of occurances of the word is misleading since
RS> it appears in the spec comments several times.

I don't believe it can be both in a comment and at the beginning of a
line, since the spec file format does not have block comments.  I
could not find a single match that wasn't the direct application of a
patch.  If you have evidence to the contrary, I invite you to
enlighten me.

RS> You need to look into the source rpm or
RS> http://cvs.fedora.redhat.com and count the actual number of
RS> patches.

As I said, I am looking at a current CVS checkout.  However, if you
just count the patch files, you may be counting some that aren't
applied in the current .spec, such as
linux-2.6-debug-detect-softlockups.patch, which appears in a current
CVS checkout and in the SRPM but is not actually applied by the
current spec.

Anyway, it's a trivial matter so I apologize for spamming.  But I
relly did try to engage my brain before posting, and I do have at
least a bit of experience doing this.

 - J<

[Shahms King]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jason L Tibbitts III wrote:
|>>>>>"RS" == Rahul Sundaram <sundaram(a)redhat.com&gt; writes:
|
|
| RS> Counting the number of occurances of the word is misleading since
| RS> it appears in the spec comments several times.
|
| I don't believe it can be both in a comment and at the beginning of a
| line, since the spec file format does not have block comments.  I
| could not find a single match that wasn't the direct application of a
| patch.  If you have evidence to the contrary, I invite you to
| enlighten me.

It won't count patches that are in comments, but spec files do support
various forms of conditional statements.  The last time I looked at the
kernel spec (which was a while ago, but still FC4) there are a number of
patches that are not used at all (inside the equivalent of an "if false"
block) and several that are architecture specific, so the count may be
off some, but certainly not by much.

- --Shahms
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.1 (GNU/Linux)
> Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org
>
> iD8DBQFDTb7eRsWY28vsX4ARAidmAKCgYty6HrKwKMb81GKktNNms3QIogCfQ8SE
> GeF4ovMVpMnWArPpLHePZtw=
> =Qhkn
> -----END PGP SIGNATURE-----

[Jason L Tibbitts III]

> >>>> "SEK" == Shahms E King
> <shahms(a)shahms.com&gt; writes: 
SEK> The last time I looked at the kernel spec (which was a while ago,
SEK> but still FC4) there are a number of patches that are not used at
SEK> all (inside the equivalent of an "if false" block) and several
SEK> that are architecture specific, so the count may be off some, but
SEK> certainly not by much.

In the current development kernel I count seven patches that are
conditionally applied, all wrapped in

%if %{includexen}
...
%endif

Those patches are:

Patch700: linux-2.6.12-xen.patch
Patch701: linux-2.6.12-xen-additional.patch
Patch702: linux-2.6.9-xen-compile.patch
Patch812: linux-2.6.8-execshield-xen.patch
Patch814: linux-2.6.12rc3-xen-vdso-note.patch
Patch1051: linux-2.6.8-devmem-xen.patch
Patch1061: linux-2.6.10-crash-xen.patch

The current specfile is very clean, significantly better than the 2.4
specs. I suggest that folks interested in this discussion take a look
at it:
http://cvs.fedora.redhat.com/viewcvs/devel/kernel/kernel-2.6.spec

 - J<

[Dave Jones]

On Wed, Oct 12, 2005 at 07:53:09PM -0400, Dimi Paun wrote:
>  From: "Dave Jones" <davej(a)redhat.com&gt;
>  > My http://people.redhat.com/davej/kernels/Fedora/*/ pages have a list
>  > of patches in each kernel, though they are somewhat out of date now.
>  
>  Looking at:
>  
>
> http://people.redhat.com/davej/kernels/Fedora/FC4/kernel-2.6.13-1.1529_FC...
>  
>  I see this:
>  + echo 'Patch #1021 (linux-2.6-debug-sleep-in-irq-warning.patch):'
>  + echo 'Patch #1900 (linux-2.6-obsolete-idescsi-warning.patch):'
>  + echo 'Patch #1901 (linux-2.6-obsolete-oss-warning.patch):'
>  + echo 'Patch #801 (linux-2.6-build-userspace-headers-warning.patch):'
>  /usr/include/asm/byteorder.h:6:2: warning: #warning using private kernel
>  header; include <endian.h> instead!
>  /usr/lib/bison.simple:164:5: warning: "YYMAXDEPTH" is not defined
>  Patch #1021 (linux-2.6-debug-sleep-in-irq-warning.patch):
>  Patch #1900 (linux-2.6-obsolete-idescsi-warning.patch):
>  Patch #1901 (linux-2.6-obsolete-oss-warning.patch):
>  Patch #801 (linux-2.6-build-userspace-headers-warning.patch):
>  
>  Does this mean that 2.6.13-1.1529 in FC4 has only 4 patches on top of
>  the upstream kernel? 
No, that's a side effect of the output of the build log being fed
through sort -u before grepping for warnings.

		Dave

[Kenneth Porter]

--On Wednesday, October 12, 2005 2:54 PM -0400 Alan Cox <alan(a)redhat.com&gt; 
wrote:

>  On Wed, Oct 12, 2005 at 06:08:06PM +0200, Angelo wrote:
> > In a year not a single reply.. i'm still hoping to get this in
> > Any idea, comment, reason to disagree, something ?
>
>  Try netdev(a)oss.sgi.com. Fedora tracks the base kernel. 
A search of their archives for "web100" turned up this plea for a kernel 
insider to help clean up the patch, back in 2003:

<http://oss.sgi.com/archives/netdev/2003-08/msg00100.html>

That was the highest-scoring match. The rest of the initial ones mention 
web100 in proposing performance patches.

The patch on the Web100 page seems relatively up-to-date but the site 
itself and the white papers are pretty old (in web time). The Web100 
mailing list looks like it's been pretty quiet for a year. Hard to get a 
read on what's happening with the project, if anything.