Re: Suitability of Python for daemon processes
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Mike McGrath wrote:
On Mon, 26 Oct 2009, Chuck Anderson wrote:
> On Sun, Oct 25, 2009 at 06:46:05PM -0400, Ben Boeckel wrote:
> > Seeing as it is a mirroring daemon, the network is the bottleneck. If it
isn't
> > then either you're sitting next door, our implementation
is bad, or the
> > hardware shouldn't be a mirror in the first place.
>
> Speaking from experience, the network isn't always the bottleneck.
> I/O performance is often a performance problem, especially when
> walking the directory tree to build filelists. CPU performance can
> come into play if you are performing hashes or compression of the data
> to be transferred. I suggest you post your message to the Fedora
> mirror-list-d where I'm sure you'll get lots of feedback.
>
Very true, if this behaves similarly to rsync. Reading over a large
change set to transmit only small changes is very resource intensive
everywhere but the network.
-Mike
The way we are planning to handle updates should keep this small. The network
side knows nothing of the on-disk file structure and the tree-creator part
knows nothing of the network other than "it's where the things I need come
from", so updates won't be "what's changed in this directory?" so
much as "what
do I not have that is on my shopping list?" which is then asked for of the
network.
- --Ben
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
iQIcBAEBCAAGBQJK5mVtAAoJEKaxavVX4C1XJVQP/RgFr9vXeqFv+fbASL6HWAr4
rUYsltcI7U5+9j7jmsbh5K+8FvOJOmdc2erEaE+hmOowdv7jvCw46ywLjcU/ZbkY
VyKMFeDsfmfd8geaujhQZULYbiDjmpFS6+uqX45ONBvIXk1L7f2zrnN9sTciSx7Z
XOmT4V9NwKnHEoIKWNaYnb7dt4Q3/8xgJ53ZdF4JNB3tJ23IsRtzCRZOofPvfy4Y
NdiAOxz5V0HY/tP4mmMH147W4QCL2lpCBLstLwcT/7rc5z3pDknPR4p8iQN4xvAn
7Q+y/12ewqxcGn8O6hKTdlG0CBt6YVeYegonZYp8V+BJ74ozaqDiR+IKVIXbJBZt
VwTzdEhiBQfAb/BVZz+Vg4YW03WxitKAKlMQ7oGKjtHuTC5j/3EVX6Gio1Lo4EHX
4DsW9XRGqDEPwqwwgKsIXMkjfNXRkdKQQzamFKVO6XTzsyJaJ+Hsv1Az0+De4SQ+
lLEgrCdJ5JLrj3+yoWoH0sk7AJDLpyrxGSaYEs0wvz+AepJK5pXVd++OivTuGdWk
lvXuSB15HG5tzKzDT45yFOR3vBhY1Bsa2j5uNBKrHUdjOuMZL/n/lfxdDH0e+GxF
RE3b8/P/LV6qGDZNyWIEVgZraqpbDohKOI98M3Tdo1kiLZtIY+/ulKtg/EuIUEra
hpc7/IiTPVQqe82zvbYR
=2gfL
-----END PGP SIGNATURE-----