Varnish

[Darren VanBuren]

On Wed, Jul 28, 2010 at 03:36, Jeroen van Meeuwen <kanarip at kanarip.com> wrote:
> Mike McGrath wrote:
>> On Fri, 23 Jul 2010, Jeroen van Meeuwen wrote:
>>
>> > Mike McGrath wrote:
>> > > I've been looking at a better proxy solution.  I initially pushed back
>> > > against varnish because it would complicate the environment, and this
> will
>> > > but since apache isn't cutting it I figured a slow incremental change is
>> > > the best approach.  So what I'm proposing is this:
>> > >
>> > > httpd(proxy) -> varnish(proxy) -> haproxy(proxy) -> httpd(app)
>> > >
>> > > So a couple of reasons why I'm choosing to do design, especially since,
> in
>> > > theory, varnish can completely replace both httpd and haproxy in that
>> > > picture.
>> > >
>> >
>> > I do not have all that much positive experience wrt. Varnish's efficiency.
>> > Have you researched any other alternatives?
>> >
>>
>> Define efficiency, load times, disk space, cpu usage?
>>
>
> Admittedly I'm not an expert in Varnish, but we run it over at Fedora Unity
I also, have only slightly touched Varnish, just getting it to cache
things, I cached one site that was rather slow to respond for
everyone, that ended up being taken down, and I saw some fair
improvement with the cached version. The stack was Apache up front
with Varnish at the backend (at least speaking in terms of my
machine).
> (behind apache, as a caching engine for plone/zope). Jonathan Steffan has a
> lot more expertise on Varnish - I let him know about this thread in case he
> would be willing to allow us to pick his brain. Pages remained presenting
> their cached version after editing, article locks are shown where in fact
> there were no locks, that sorta thing.
>
> I suppose sinking one's teeth into it would suffice so resolve such problems,
> but for our deployment Varnish bloated the system more then we had in mind in
> terms or reducing the load on the backend application. This, in our case, had
> to do with memory consumption more so then with anything else.
Varnish can be told not to use memory for caching, and that's how I've
used it, 1GB doesn't go a long way when you've got 64-bit Apache
HTTPd.

>> > In other architecture designs (large or heavily loaded rails deployments
> for
>> > example), the concept of assets is introduced. Would such functionality
> serve
>> > purpose?
>> >
>>
>> I've not used assets before, what's the scoop?
>>
>
> With mod_passenger for Apache, *everything* that you hit on the VirtualHost
> goes through the Passenger stack. It has learned to not parse the contents of
I didn't know that. Then again, my rails apps don't get traffic at all.
> /images/, sorta speak, yet the client locks a thread and makes the the rather
> bloated httpd+passenger run in circles even for just a little while -per
> "asset".
>
> With assets -and I'm not sure that term is accurate "across the board"-, a
> bunch of smart Ruby-on-Rails webservers (with mod_rails/mod_rack and such) no
> longer serve any content that is 1) static or 2) generated to be cached, and
> not subject to any Access Control. The latter would, in the case of Ruby on
> Rails, apply to haml/sass content, but in the case of Fedora would (maybe)
> apply to the mirror lists. These assets can then be hosted on a light-weight,
> stupid and efficient webserver -even without caching such would off-load the
> backend application servers considerably.
This definition of assets is most definitely correct. I tend to think
of them as anything that is used frequently, and changed rarely, like
HTML generated by cron jobs or changed manually by people, CSS,
JavaScript, images of any kind.

Darren VanBuren
==================
http://theoks.net/