From dave at cheney.net  Tue Dec  4 09:08:11 2007
From: dave at cheney.net (Dave Cheney)
Date: Tue, 4 Dec 2007 20:08:11 +1100
Subject: Looking for information on balanced backends
Message-ID: <890DB058-BC98-4DC4-B7F6-334958780AA4@cheney.net>

Hi,

	We use a pair of varnish servers behind a load balancer provided by  
our hosting center. For various technical reasons the load balancer  
can't be used to balance the servers that varnish sits in front of so  
I would like to use the balanced backend functionality (introduced  
somewhere around changeset 1940). I've been looking in the source for / 
trunk and brances/1.2 but I can't find any documentation for  
constructing a VLC script for balanced backends. Can anyone give any  
sample configuration ?

Cheers

Dave


From caleb.anthony at gmail.com  Tue Dec  4 15:58:30 2007
From: caleb.anthony at gmail.com (Caleb Anthony)
Date: Tue, 4 Dec 2007 08:58:30 -0700
Subject: Looking for information on balanced backends
In-Reply-To: <890DB058-BC98-4DC4-B7F6-334958780AA4@cheney.net>
References: <890DB058-BC98-4DC4-B7F6-334958780AA4@cheney.net>
Message-ID: <33afbbe70712040758yfe547cdt1104bfbd5f75887c@mail.gmail.com>

This should help.

http://varnish.projects.linpro.no/wiki/Backends

Right now there are two load balancing methods: round robin and
weighted round robin.

backend_round_robin rr {
                set backend.set = {
                        { "foo1.bar.com", "http" }
                        { "foo2.bar.com", "http" }
                        { "foo3.bar.com", "http" }
                };
        }

backend_random rrr {
                set backend.set = {
                        { "foo1.bar.com", "http", 0.3 }
                        { "foo2.bar.com", "http", 0.6 }
                        { "foo3.bar.com", "http", 0.1 }
                };
        }

Once you have that in your VCL, just select which backend to use like
you normally would:

sub vcl_recv {
                if {req.http.host ~ "foo"} {
                        req.backend = rr;
                } elseif {req.http.host ~ "bar"} {
                        req.backend = rr;
                } else {
                        req.backend = rrr;
                }
        }


BTW, the regular round robin works pretty good for us.

On 12/4/07, Dave Cheney <dave at cheney.net> wrote:
> Hi,
>
>        We use a pair of varnish servers behind a load balancer provided by
> our hosting center. For various technical reasons the load balancer
> can't be used to balance the servers that varnish sits in front of so
> I would like to use the balanced backend functionality (introduced
> somewhere around changeset 1940). I've been looking in the source for /
> trunk and brances/1.2 but I can't find any documentation for
> constructing a VLC script for balanced backends. Can anyone give any
> sample configuration ?
>
> Cheers
>
> Dave
> _______________________________________________
> varnish-dev mailing list
> varnish-dev at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-dev
>


From jodok at lovelysystems.com  Wed Dec 12 10:27:17 2007
From: jodok at lovelysystems.com (Jodok Batlogg)
Date: Wed, 12 Dec 2007 11:27:17 +0100
Subject: varnish using 100% cpu
Message-ID: <6473FBF9-0F08-43DC-9A72-BA6D442D9CBD@lovelysystems.com>

hi,

we're using varnish in production and have roughly every 10 days the  
problem that varnish is running at 100% CPU (but requests are beeing  
served "normally").

today i've been able to attach gdb to the running process, look a  
little bit around and generate a core dump.
if someone wants to help getting this bug solved - the core dump is at http://download.lovelysystems.com/public/core.1237

thanks

jodok


From phk at phk.freebsd.dk  Wed Dec 12 18:35:56 2007
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Wed, 12 Dec 2007 18:35:56 +0000
Subject: varnish using 100% cpu 
In-Reply-To: Your message of "Wed, 12 Dec 2007 11:27:17 +0100."
	<6473FBF9-0F08-43DC-9A72-BA6D442D9CBD@lovelysystems.com> 
Message-ID: <64163.1197484556@critter.freebsd.dk>

In message <6473FBF9-0F08-43DC-9A72-BA6D442D9CBD at lovelysystems.com>, Jodok Batlogg writes:
>hi,
>
>we're using varnish in production and have roughly every 10 days the  
>problem that varnish is running at 100% CPU (but requests are beeing  
>served "normally").
>
>today i've been able to attach gdb to the running process, look a  
>little bit around and generate a core dump.
>if someone wants to help getting this bug solved - the core dump is at http://download.lovelysystems.com/public/core.1237

Which version are you running ?

I fixed a cpu-eating bug in the telnet/CLI code a few days ago, do you have
the patch for that ?

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From jodok at lovelysystems.com  Wed Dec 12 19:05:33 2007
From: jodok at lovelysystems.com (Jodok Batlogg)
Date: Wed, 12 Dec 2007 20:05:33 +0100
Subject: varnish using 100% cpu 
In-Reply-To: <64163.1197484556@critter.freebsd.dk>
References: <64163.1197484556@critter.freebsd.dk>
Message-ID: <347B8E62-5986-4A95-9923-7947993ECAE6@lovelysystems.com>

On 12.12.2007, at 19:35, Poul-Henning Kamp wrote:

> In message <6473FBF9-0F08-43DC-9A72-BA6D442D9CBD at lovelysystems.com>,  
> Jodok Batlogg writes:
>> hi,
>>
>> we're using varnish in production and have roughly every 10 days the
>> problem that varnish is running at 100% CPU (but requests are beeing
>> served "normally").
>>
>> today i've been able to attach gdb to the running process, look a
>> little bit around and generate a core dump.
>> if someone wants to help getting this bug solved - the core dump is  
>> at http://download.lovelysystems.com/public/core.1237
>
> Which version are you running ?

sorry forgot it :)
i'm using 1.1.1

> I fixed a cpu-eating bug in the telnet/CLI code a few days ago, do  
> you have
> the patch for that ?

not yet, where can i find it?

i'll try and give feedback in 10 days the latest :)

jodok

>
>
> -- 
> Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
> phk at FreeBSD.ORG         | TCP/IP since RFC 956
> FreeBSD committer       | BSD since 4.3-tahoe
> Never attribute to malice what can adequately be explained by  
> incompetence.



From phk at phk.freebsd.dk  Wed Dec 12 20:14:55 2007
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Wed, 12 Dec 2007 20:14:55 +0000
Subject: varnish using 100% cpu 
In-Reply-To: Your message of "Wed, 12 Dec 2007 20:05:33 +0100."
	<347B8E62-5986-4A95-9923-7947993ECAE6@lovelysystems.com> 
Message-ID: <64763.1197490495@critter.freebsd.dk>

In message <347B8E62-5986-4A95-9923-7947993ECAE6 at lovelysystems.com>, Jodok Batlogg writes:

>not yet, where can i find it?

Try trunk.


-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From jesus at omniti.com  Fri Dec 21 05:07:23 2007
From: jesus at omniti.com (Theo Schlossnagle)
Date: Fri, 21 Dec 2007 00:07:23 -0500
Subject: Solaris support
Message-ID: <63AB202A-A1CE-486D-8231-7F61588F84D1@omniti.com>

Hello all,

We've been running this for a while on Solaris.  Works really well.

We have some minor modifications to the source tree to make it work,  
but there is one minor design change to make things work well with our  
ports cache_acceptor.

What we need is the function in cache_acceptor.c:

void vca_return_session(struct sess *sp);

to be delegated to the acceptor implementations.  Solaris allows for  
very efficient user initiated notifications across its event system  
and that does that by talking directly to the port and not using the  
vca_pipes.

I'd like to have a return_session element added to the acceptor  
structure that is void (*return_session)(struct sess *);

And the vca_return_session function be:

void
vca_return_session(struct sess *sp)
{
   vca->return_session(sp);
}

I see this as a cleaner mechanism regardless as there is no reason for  
the generic cache_acceptor to care about int vca_pipes[2]; -- it's an  
implementation detail.  How's that sound?

I'd like to see this change in trunk before I submit my patch as it  
will pretty solidly affect how the return session stuff is handled in  
our changeset.

My current patch supports:
   sendfile and sendfilev on solaris
   using fcntl() when flock() is unavailable
   A cache_acceptor_ports.c that embraces Solaris' port eventing system

--
Theo Schlossnagle
Esoteric Curio -- http://lethargy.org/
OmniTI Computer Consulting, Inc. -- http://omniti.com/



From theo at omniti.com  Mon Dec 31 21:26:16 2007
From: theo at omniti.com (Theo Schlossnagle)
Date: Mon, 31 Dec 2007 16:26:16 -0500
Subject: development efforts on the Solaris side.
Message-ID: <762EF0EC-5C51-41BA-A38F-65DDCD4F9249@omniti.com>

Hi guys,

I'd really like to be able to contribute some of the improvements  
we've made to varnish back.  Is there a way I can get access to  
commit.  I'd be happy to stay in my own branch.  My current patch set  
is unwieldy and I'm very tempted to just start my own repos... That,  
of course, seems silly.  I've fixed up (removed) some of the gccism in  
favor or more portability (#include over -include).  I've fixed a few  
bugs, made the VCC line a but smarter and more accepting of non gcc  
compiler, I've added a portfs acceptor and built a storage_umem  
allocator facility that rides on Solaris' excellent libumem (highly  
scalable allocator) which we also ported to run on Linux and FreeBSD  
(and Mac OS X): https://labs.omniti.com/trac/portableumem

Next steps?

Best regards,

Theo

--
Theo Schlossnagle
Principal/CTO
OmniTI Computer Consulting, Inc.
W: http://omniti.com
P: +1.443.325.1357 x201
F: +1.410.872.4911