From ssm at linpro.no  Tue Sep 19 07:48:03 2006
From: ssm at linpro.no (Stig Sandbeck Mathisen)
Date: Tue, 19 Sep 2006 09:48:03 +0200
Subject: Varnish blogged...
Message-ID: <7xu034e1qk.fsf@linpro.no>


http://www.mnot.net/blog/2006/08/21/caching_performance

-- 
Stig Sandbeck Mathisen, Linpro


From trondmm-varnish at crusaders.no  Wed Sep 20 13:22:40 2006
From: trondmm-varnish at crusaders.no (Trond Michelsen)
Date: Wed, 20 Sep 2006 15:22:40 +0200
Subject: VCL documentation
Message-ID: <20060920132240.GG13253@crusaders.no>

Hi. 

I've just downloaded and installed varnish, and I was just wondering
if the VCL config language is documented somewhere, or if there are
any examples that I could use for inspiration.

-- 
Trond Michelsen



From phk at phk.freebsd.dk  Wed Sep 20 14:43:15 2006
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Wed, 20 Sep 2006 14:43:15 +0000
Subject: VCL documentation 
In-Reply-To: Your message of "Wed, 20 Sep 2006 15:22:40 +0200."
	<20060920132240.GG13253@crusaders.no> 
Message-ID: <33545.1158763395@critter.freebsd.dk>

In message <20060920132240.GG13253 at crusaders.no>, Trond Michelsen writes:
>Hi. 
>
>I've just downloaded and installed varnish, and I was just wondering
>if the VCL config language is documented somewhere, or if there are
>any examples that I could use for inspiration.

The main example right now is the default VCL code which you will
find in the source file bin/varnishd/mgt_vcc.c.

Also, this is the code we run at VG right now:

backend default {
        set backend.host = "10.0.2.1";
        set backend.port = "80";
}

acl purge {
        "localhost";
        "10.0.0.1";
}

sub vcl_recv {
        if (req.request == "PURGE") {
                if (!client.ip ~ purge) {
                        error 405 "Not allowed.";
                }
                lookup;
        }
        if (req.request == "GET" && req.http.cookie) {
                lookup;
        }
}

sub vcl_hit {
        if (req.request == "PURGE") {
                set obj.ttl = 0s;
                error 200 "Purged.";
        }
}

sub vcl_miss {
        if (req.request == "PURGE") {
                error 404 "Not in cache.";
        }
}


The important thing to know which is not obvious, is that if
you do not hit an "action" in for instance vcl_miss(), the
default vcl_miss() function will be executed and will determine
the action.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From des at linpro.no  Wed Sep 20 16:33:09 2006
From: des at linpro.no (Dag-Erling =?iso-8859-1?Q?Sm=F8rgrav?=)
Date: Wed, 20 Sep 2006 18:33:09 +0200
Subject: VCL documentation
References: <20060920132240.GG13253@crusaders.no>
Message-ID: <ujrodtapkfu.fsf@cat.linpro.no>

Trond Michelsen <trondmm-varnish at crusaders.no> writes:
> I've just downloaded and installed varnish, and I was just wondering
> if the VCL config language is documented somewhere, or if there are
> any examples that I could use for inspiration.

I am working on a vcl(7) manual page, which will hopefully be included
along with a few bug fixes in a 1.0.1 release tomorrow or Friday.

Here's a wiki page with a config I've used to test how well a stock
Varnish install handles the major Norwegian online news outlets:

http://varnish.projects.linpro.no/wiki/NorskeNettaviser

(Varnish is very flexible, but we want it to run well "out of the box"
so it will be useful in an emergency when you don't have time to read
the docs and write a config file)

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


From trondmm-varnish at crusaders.no  Wed Sep 20 19:44:10 2006
From: trondmm-varnish at crusaders.no (Trond Michelsen)
Date: Wed, 20 Sep 2006 21:44:10 +0200
Subject: VCL documentation
In-Reply-To: <33545.1158763395@critter.freebsd.dk>
References: <20060920132240.GG13253@crusaders.no>
	<33545.1158763395@critter.freebsd.dk>
Message-ID: <20060920194410.GB19935@crusaders.no>

On Wed, Sep 20, 2006 at 02:43:15PM +0000, Poul-Henning Kamp wrote:
> In message <20060920132240.GG13253 at crusaders.no>, Trond Michelsen writes:
>> I've just downloaded and installed varnish, and I was just wondering
>> if the VCL config language is documented somewhere, or if there are
>> any examples that I could use for inspiration.
> The main example right now is the default VCL code which you will
> find in the source file bin/varnishd/mgt_vcc.c.

Thanks.

So, if I want to cache absolutely everything that is requested through
the cache, no matter what the headers might say, I could write
something like this:

sub vcl_fetch {
    insert;
}

Would that work?

> The important thing to know which is not obvious, is that if
> you do not hit an "action" in for instance vcl_miss(), the
> default vcl_miss() function will be executed and will determine
> the action.

Ah, OK. 

-- 
  // Trond Michelsen
\X/  mike at crusaders.no


From phk at phk.freebsd.dk  Wed Sep 20 20:37:31 2006
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Wed, 20 Sep 2006 20:37:31 +0000
Subject: VCL documentation 
In-Reply-To: Your message of "Wed, 20 Sep 2006 21:44:10 +0200."
	<20060920194410.GB19935@crusaders.no> 
Message-ID: <54953.1158784651@critter.freebsd.dk>

In message <20060920194410.GB19935 at crusaders.no>, Trond Michelsen writes:
>On Wed, Sep 20, 2006 at 02:43:15PM +0000, Poul-Henning Kamp wrote:
>> In message <20060920132240.GG13253 at crusaders.no>, Trond Michelsen writes:
>>> I've just downloaded and installed varnish, and I was just wondering
>>> if the VCL config language is documented somewhere, or if there are
>>> any examples that I could use for inspiration.
>> The main example right now is the default VCL code which you will
>> find in the source file bin/varnishd/mgt_vcc.c.
>
>Thanks.
>
>So, if I want to cache absolutely everything that is requested through
>the cache, no matter what the headers might say, I could write
>something like this:
>
>sub vcl_fetch {
>    insert;
>}
>
>Would that work?

You should probably also do:

	sub vcl_recv {
		lookup;
	}


-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From ltning at anduin.net  Thu Sep 21 06:32:11 2006
From: ltning at anduin.net (=?ISO-8859-1?Q?Eirik_=D8verby?=)
Date: Thu, 21 Sep 2006 08:32:11 +0200
Subject: Caching efficiency, VCL
Message-ID: <B511F2D4-8028-4954-8CB1-A6FD2A37917E@anduin.net>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi all,

installed Varnish during the presentation yesterday, and was playing  
with it for most of the evening (together with Marcus, who has  
already reared his head on the -dev list). Given the nature of the  
site we're trying to serve, there's not much that can be cached  
unless we do some serious customizations on the VCL side. That,  
combined with some adjustments of the site code, and we should see  
some pretty good improvements.

Questions:
- - Others on this list have asked for VCL documentation.  I seel the  
same.
- - There's also some lack in the documentation of (runtime) parameters  
listed in the varnishd man page.
- - Can I control access to the management interface through VCL?
- - Due to the fact that webserver and varnish are currently running on  
the same box, I am using a pf ruleset like the following:
no rdr on $int_if proto tcp from $my_ip to $my_ip port 80
rdr on $int_if proto tcp from any to $my_ip port 80 -> $my_ip port 8080
Is this going to limit my performance in any significant way? Given  
that I'm running on hardware comparable to what the devs have been  
testing on, and the site I'm serving sees only a tiny fraction of the  
traffic, I wouldn't think so.
- - The cmdline allows me to specify a minimum ttl for objects -- I'm  
missing a way to specify max ttl. Is this something I could do in  
VCL, and in that case, why is there an option for the minimum ttl?

I guess that's it for now. Hoping to get a grip of this beast soon!

Thanks,
/Eirik
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFFEjHsDRlfnc8VQWcRAqgkAJ9Id6zN9xcaulN8Q27bJIuhL1U95wCgmwrj
gC4fwotnyxTrN6ZjRkKP49g=
=oyBw
-----END PGP SIGNATURE-----


From des at linpro.no  Thu Sep 21 10:05:37 2006
From: des at linpro.no (Dag-Erling =?iso-8859-1?Q?Sm=F8rgrav?=)
Date: Thu, 21 Sep 2006 12:05:37 +0200
Subject: Caching efficiency, VCL
References: <B511F2D4-8028-4954-8CB1-A6FD2A37917E@anduin.net>
Message-ID: <ujrmz8tmt5a.fsf@cat.linpro.no>

Eirik ?verby <ltning at anduin.net> writes:
> - Others on this list have asked for VCL documentation.  I seel the
> same.

I'm working on it.

> - There's also some lack in the documentation of (runtime)
> parameters listed in the varnishd man page.

'param.show -l' in the management interface will give you a complete
list with fairly detailed descriptions.

> - Can I control access to the management interface through VCL?

No.

> - Due to the fact that webserver and varnish are currently running
> on the same box, I am using a pf ruleset like the following:
> no rdr on $int_if proto tcp from $my_ip to $my_ip port 80
> rdr on $int_if proto tcp from any to $my_ip port 80 -> $my_ip port 8080
> Is this going to limit my performance in any significant way? Given
> that I'm running on hardware comparable to what the devs have been
> testing on, and the site I'm serving sees only a tiny fraction of the
> traffic, I wouldn't think so.

Can't you make your web server run on port 8080?

> - The cmdline allows me to specify a minimum ttl for objects -- I'm
> missing a way to specify max ttl. Is this something I could do in
> VCL, and in that case, why is there an option for the minimum ttl?

This is a documentation error - the option actually controls the
*default* ttl if none is provided by the backend server.

You can inspect and modify the TTL in VCL; vcl_fetch would be a good
place to do that...  something like:

sub vcl_fetch {
    if (obj.ttl > 120s) {
        set obj.ttl = 120s;
    }
}

(not tested, so caveat emptor)

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


From ltning at anduin.net  Thu Sep 21 11:03:55 2006
From: ltning at anduin.net (=?ISO-8859-1?Q?Eirik_=D8verby?=)
Date: Thu, 21 Sep 2006 13:03:55 +0200
Subject: Caching efficiency, VCL
In-Reply-To: <ujrmz8tmt5a.fsf@cat.linpro.no>
References: <B511F2D4-8028-4954-8CB1-A6FD2A37917E@anduin.net>
	<ujrmz8tmt5a.fsf@cat.linpro.no>
Message-ID: <D14829C5-D135-4FCA-8C41-5559E1708CCB@anduin.net>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

- -----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sep 21, 2006, at 12:05, Dag-Erling Sm?rgrav wrote:

> Eirik ?verby <ltning at anduin.net> writes:
>> - Others on this list have asked for VCL documentation.  I seel the
>> same.
>
> I'm working on it.

Good =)

>> - There's also some lack in the documentation of (runtime)
>> parameters listed in the varnishd man page.
>
> 'param.show -l' in the management interface will give you a complete
> list with fairly detailed descriptions.

Ok, my bad. Thanks.

>> - Can I control access to the management interface through VCL?
>
> No.
>
>> - Due to the fact that webserver and varnish are currently running
>> on the same box, I am using a pf ruleset like the following:
>> no rdr on $int_if proto tcp from $my_ip to $my_ip port 80
>> rdr on $int_if proto tcp from any to $my_ip port 80 -> $my_ip port  
>> 8080
>> Is this going to limit my performance in any significant way? Given
>> that I'm running on hardware comparable to what the devs have been
>> testing on, and the site I'm serving sees only a tiny fraction of the
>> traffic, I wouldn't think so.
>
> Can't you make your web server run on port 8080?

Yes I can, however as Marcus stated elsewhere, Varnish does not add a  
X-Origin-something to the request to the back-end, therefore the back- 
end thinks it is in fact the front-end -- and all links etc. are  
generated with :8080. Sure this could be fixed, but my pf approach  
was the faster way right now.

Plus, the pf approach allows me to have a "failover" - if Varnish is  
not running, it will allow the client to connect directly to the back- 
end (with some modifications). This is not interesting in all cases,  
but in ours it is, and gives us some level of freedom to play around  
with Varnish ;)

>> - The cmdline allows me to specify a minimum ttl for objects -- I'm
>> missing a way to specify max ttl. Is this something I could do in
>> VCL, and in that case, why is there an option for the minimum ttl?
>
> This is a documentation error - the option actually controls the
> *default* ttl if none is provided by the backend server.
>
> You can inspect and modify the TTL in VCL; vcl_fetch would be a good
> place to do that...  something like:
>
> sub vcl_fetch {
>     if (obj.ttl > 120s) {
>         set obj.ttl = 120s;
>     }
> }
>
> (not tested, so caveat emptor)

Brilliant.

/Eirik



- -----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFFEnGYDRlfnc8VQWcRAsuFAJ4jUFeh6Pa7ZdIvHNV5aK0ShSK9QACfcJTS
dEQtvYDpmoNtICcJTOeWCzQ=
=WC0t
- -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFFEnGbDRlfnc8VQWcRAiT9AKCcc9ULWUT5WuUr6jYi8ZOARyOdogCdEiJM
wZUWP7OcaADi0ewWHxRLu64=
=uNKB
-----END PGP SIGNATURE-----


From ingvar at linpro.no  Fri Sep 22 21:31:22 2006
From: ingvar at linpro.no (Ingvar Hagelund)
Date: 22 Sep 2006 23:31:22 +0200
Subject: varnish-1.0.1 rpm packages for RHEL4
Message-ID: <83k63vd1w5.fsf@uname.e.linpro.no>

Inspired by the lack of precompiled packages, I rolled a set of i386,
x86_64 and src packages for RHEL4. Included is a specfile, a simple
test config and a RedHat compliant init script. Please consider
including them in the upstream source.

Packages available at http://users.linpro.no/ingvar/varnish/

Building binary packkages for other Red Hat derived distributions like
Fedora or CentOS, should be as simple as rebuilding the source rpm.

Regards,
Ingvar Hagelund

-- 
Blogs are the new usenet
(Mike of UserFriendly.org)



From ingvar at linpro.no  Fri Sep 22 21:36:26 2006
From: ingvar at linpro.no (Ingvar Hagelund)
Date: 22 Sep 2006 23:36:26 +0200
Subject: 1.0.1 building bugs
Message-ID: <83fyejd1np.fsf@uname.e.linpro.no>

Building varnish on RHEL4, I found that the 1.0.1 version has at least
two small build bugs:

The build does not consider '--sbindir' from the configure
script. Using it should make varnishd go into the defined target
directory. Now, it doesn't.

The build does not symlink from libvcl.so.0.0.0 to libvcl.so.0

Regards,
Ingvar Hagelund
Linpro AS

-- 
Blogs are the new usenet
(Mike of UserFriendly.org)



From trondmm-varnish at crusaders.no  Mon Sep 25 07:59:16 2006
From: trondmm-varnish at crusaders.no (Trond Michelsen)
Date: Mon, 25 Sep 2006 09:59:16 +0200
Subject: Altering response based on content-type
Message-ID: <20060925075916.GK13253@crusaders.no>

Hi.

I'm using Varnish as a cache for a WMS service, and this service will
occasionally time out on a request. If it does it will return either
an image with an error message, or an XML-file with the
error-message. Unfortunately, both responses return the status code
200 OK.

The webclient doesn't handle the XML-response very well, and I don't
want Varnish to cache the images containing error messages.

So - is it possible to get Varnish to return a default image if the
response from the server is an XML-file?

-- 
Trond Michelsen



From phk at phk.freebsd.dk  Mon Sep 25 08:17:19 2006
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Mon, 25 Sep 2006 08:17:19 +0000
Subject: Altering response based on content-type 
In-Reply-To: Your message of "Mon, 25 Sep 2006 09:59:16 +0200."
	<20060925075916.GK13253@crusaders.no> 
Message-ID: <4954.1159172239@critter.freebsd.dk>

In message <20060925075916.GK13253 at crusaders.no>, Trond Michelsen writes:
>Hi.
>
>I'm using Varnish as a cache for a WMS service, and this service will
>occasionally time out on a request. If it does it will return either
>an image with an error message, or an XML-file with the
>error-message. Unfortunately, both responses return the status code
>200 OK.
>
>The webclient doesn't handle the XML-response very well, and I don't
>want Varnish to cache the images containing error messages.
>
>So - is it possible to get Varnish to return a default image if the
>response from the server is an XML-file?

There are no headers we can detect instead ?

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From trondmm-varnish at crusaders.no  Mon Sep 25 08:51:02 2006
From: trondmm-varnish at crusaders.no (Trond Michelsen)
Date: Mon, 25 Sep 2006 10:51:02 +0200
Subject: Altering response based on content-type
In-Reply-To: <4954.1159172239@critter.freebsd.dk>
References: <20060925075916.GK13253@crusaders.no>
	<4954.1159172239@critter.freebsd.dk>
Message-ID: <20060925085102.GL13253@crusaders.no>

On Mon, Sep 25, 2006 at 08:17:19AM +0000, Poul-Henning Kamp wrote:
> In message <20060925075916.GK13253 at crusaders.no>, Trond Michelsen writes:
>> I'm using Varnish as a cache for a WMS service, and this service will
>> occasionally time out on a request. If it does it will return either
>> an image with an error message, or an XML-file with the
>> error-message. Unfortunately, both responses return the status code
>> 200 OK.
>> The webclient doesn't handle the XML-response very well, and I don't
>> want Varnish to cache the images containing error messages.
>> So - is it possible to get Varnish to return a default image if the
>> response from the server is an XML-file?
> There are no headers we can detect instead ?

If the requests asks for errors to be inline, then no. The response
headers are exactly the same for a successful request as for an
unsuccessful one. In fact the program returns only the bare minimum of
headers. In both a successful and an unsuccsessful request, headers
look like this:

HTTP/1.1 200 OK
Date: Mon, 25 Sep 2006 08:42:46 GMT
Server: Apache/1.3.33 (Debian GNU/Linux) AxKit/1.62 mod_perl/1.29
Transfer-Encoding: chunked
Content-Type: image/png

However, the client can ask for error messages to be returned in one
of three formats:

- application/vnd.ogc.se_xml

In this case, the content-type of the response will be
"application/vnd.ogc.se_xml", so this should be detectable.

- application/vnd.ogc.se_inimage

This will return an image with the error message
embedded. Content-type will be image/png (or whatever format the
client requested).

- application/vnd.ogc.se_blank

This one is actually a bit interesting. I didn't know about it until
right now, and this image will always be of the same size (142 bytes
for PNG), and initial tests indicates that it is different from any of
the blank tiles generated from the mapserver. 

So that would give me another option, which probably is simpler. Set
obj.ttl to 0s if its size is 142 bytes. Is that possible? 

-- 
Trond Michelsen



From phk at phk.freebsd.dk  Mon Sep 25 18:41:19 2006
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Mon, 25 Sep 2006 18:41:19 +0000
Subject: Altering response based on content-type 
In-Reply-To: Your message of "Mon, 25 Sep 2006 10:51:02 +0200."
	<20060925085102.GL13253@crusaders.no> 
Message-ID: <3002.1159209679@critter.freebsd.dk>

In message <20060925085102.GL13253 at crusaders.no>, Trond Michelsen writes:

>So that would give me another option, which probably is simpler. Set
>obj.ttl to 0s if its size is 142 bytes. Is that possible? 

If your backend includes a Content-Length: header you can test on,
yes.

If it uses chunked encoding we won't know the size until after the
vcl_fetch() has been called.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From superjoe at gmail.com  Sat Sep 30 18:13:20 2006
From: superjoe at gmail.com (Joseph McDonald)
Date: Sat, 30 Sep 2006 11:13:20 -0700
Subject: send hostname?
Message-ID: <73096a160609301113k2991a01fo1211a434e414ec27@mail.gmail.com>

Hi,

I'm just starting with varnish, I point my backend to a name based
server, but the name doesn't seem to get sent down to the backend
server.  Is there some switch I need to get it to send the hostname
down with the request?

thanks,
-joe