From des at linpro.no  Mon Oct  2 14:03:52 2006
From: des at linpro.no (Dag-Erling =?iso-8859-1?Q?Sm=F8rgrav?=)
Date: Mon, 02 Oct 2006 16:03:52 +0200
Subject: send hostname?
In-Reply-To: <73096a160609301113k2991a01fo1211a434e414ec27@mail.gmail.com>
	(Joseph McDonald's message of "Sat, 30 Sep 2006 11:13:20 -0700")
References: <73096a160609301113k2991a01fo1211a434e414ec27@mail.gmail.com>
Message-ID: <ujrlknyerw7.fsf@cat.linpro.no>

"Joseph McDonald" <superjoe at gmail.com> writes:
> I'm just starting with varnish, I point my backend to a name based
> server, but the name doesn't seem to get sent down to the backend
> server.  Is there some switch I need to get it to send the hostname
> down with the request?

Varnish will send a Host: header to the backend if it got one from the
client.

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


From phk at phk.freebsd.dk  Mon Oct  2 14:09:14 2006
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Mon, 02 Oct 2006 14:09:14 +0000
Subject: send hostname? 
In-Reply-To: Your message of "Mon, 02 Oct 2006 16:03:52 +0200."
	<ujrlknyerw7.fsf@cat.linpro.no> 
Message-ID: <60339.1159798154@critter.freebsd.dk>

In message <ujrlknyerw7.fsf at cat.linpro.no>, Dag-Erling =?iso-8859-1?Q?Sm=F8rgrav?= 
writes:
>"Joseph McDonald" <superjoe at gmail.com> writes:
>> I'm just starting with varnish, I point my backend to a name based
>> server, but the name doesn't seem to get sent down to the backend
>> server.  Is there some switch I need to get it to send the hostname
>> down with the request?
>
>Varnish will send a Host: header to the backend if it got one from the
>client.

sorry about the delay, I've been occupied with other stuff.

Actually, Varnish will insert a Host: if there isn't one, but only
in the "fetch" case.  Similar code was missing in the pass & pipe cases.

Fixed.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From phk at phk.freebsd.dk  Mon Oct  2 19:55:34 2006
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Mon, 02 Oct 2006 19:55:34 +0000
Subject: Varnish has been podcast..
Message-ID: <61716.1159818934@critter.freebsd.dk>


http://bsdtalk.blogspot.com.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.


From benny.kjellgren at trig.com  Wed Oct  4 14:39:06 2006
From: benny.kjellgren at trig.com (Benny Kjellgren (TRIG/adocca))
Date: Wed, 4 Oct 2006 16:39:06 +0200
Subject: chroot or/and none root user
Message-ID: <20061004143906.GA11126@trig.com>

Hi,

I have started to test Varnish on Debian (and Archlinux) and the
first thing I was looking for is to how to run Varnish as a none
root user (eg www-data) and/or put it in a jail (chroot).

Is it possible to do that ?

-- 
Benny Kjellgren


From phk at phk.freebsd.dk  Wed Oct  4 20:20:37 2006
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Wed, 04 Oct 2006 20:20:37 +0000
Subject: chroot or/and none root user 
In-Reply-To: Your message of "Wed, 04 Oct 2006 16:39:06 +0200."
	<20061004143906.GA11126@trig.com> 
Message-ID: <96645.1159993237@critter.freebsd.dk>

In message <20061004143906.GA11126 at trig.com>, "Benny Kjellgren (TRIG/adocca)" write
s:
>Hi,
>
>I have started to test Varnish on Debian (and Archlinux) and the
>first thing I was looking for is to how to run Varnish as a none
>root user (eg www-data) and/or put it in a jail (chroot).
>
>Is it possible to do that ?

If you don't use a priviledged listen port, it should just work.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From ingvar at linpro.no  Thu Oct  5 07:27:42 2006
From: ingvar at linpro.no (Ingvar Hagelund)
Date: Thu, 05 Oct 2006 09:27:42 +0200
Subject: chroot or/and none root user
In-Reply-To: <96645.1159993237@critter.freebsd.dk>
References: <96645.1159993237@critter.freebsd.dk>
Message-ID: <4524B3EE.3060505@linpro.no>

* Benny Kjellgren
>> I have started to test Varnish on Debian (and Archlinux) and the
>> first thing I was looking for is to how to run Varnish as a none
>> root user (eg www-data) and/or put it in a jail (chroot).
>>
>> Is it possible to do that ?

* Poul-Henning Kamp
> If you don't use a priviledged listen port, it should just work.

Any plans to use the model used by eg apache or squid, with a
dispatching process running in priviledged mode, and process children
running with a non privilegded user?

One might think that this model makes privilege escalation harder, or at
least, less intrusive, in case there should be any exploitable bugs in
varnish (but of course, there would never be any such thing 8-) .

Ingvar

-- 
When everything else fails: Symlink


From benny.kjellgren at trig.com  Thu Oct  5 07:53:54 2006
From: benny.kjellgren at trig.com (Benny Kjellgren (TRIG/adocca))
Date: Thu, 5 Oct 2006 09:53:54 +0200
Subject: chroot or/and none root user
In-Reply-To: <4524B3EE.3060505@linpro.no>
References: <96645.1159993237@critter.freebsd.dk> <4524B3EE.3060505@linpro.no>
Message-ID: <20061005075354.GA2811@trig.com>

> Any plans to use the model used by eg apache or squid, with a
> dispatching process running in priviledged mode, and process children
> running with a non privilegded user?

An other example is Pound
http://www.apsis.ch/pound/
In Pound you can set "User", "Group" and "RootJail".

--
Benny Kjellgren



From benny.kjellgren at trig.com  Thu Oct  5 09:18:34 2006
From: benny.kjellgren at trig.com (Benny Kjellgren (TRIG/adocca))
Date: Thu, 5 Oct 2006 11:18:34 +0200
Subject: chroot or/and none root user
In-Reply-To: <4524B3EE.3060505@linpro.no>
References: <96645.1159993237@critter.freebsd.dk> <4524B3EE.3060505@linpro.no>
Message-ID: <20061005091834.GB2811@trig.com>

> If you don't use a priviledged listen port, it should just work.

FYI
I solved it by running varnishd as user="varnish" and listen
on port 8080 and added a iptables NAT rule that redirect
port 80 to port 8080. Works fine. (Debian)

-- 
Benny Kjellgren


From superjoe at gmail.com  Wed Oct  4 22:06:47 2006
From: superjoe at gmail.com (Joseph McDonald)
Date: Wed, 4 Oct 2006 15:06:47 -0700
Subject: Content-encoding: gzip
Message-ID: <73096a160610041506w374b315fy32ea1ba4f6826c5@mail.gmail.com>

Hi,

my backend does gzip compression for clients that support it.  However
it appears that varnish caches and sends out whatever it got first.
That is, if a client supports gzip,  than it caches the gzip version.
If the next client does not support gzip, varnish still sends out the
gzip version.  and vice versa.

Do I need to configure something for varnish to respect the
Content-encoding header?

thanks,
-joe


From phk at phk.freebsd.dk  Thu Oct  5 09:35:41 2006
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Thu, 05 Oct 2006 09:35:41 +0000
Subject: Content-encoding: gzip 
In-Reply-To: Your message of "Wed, 04 Oct 2006 15:06:47 MST."
	<73096a160610041506w374b315fy32ea1ba4f6826c5@mail.gmail.com> 
Message-ID: <292.1160040941@critter.freebsd.dk>

In message <73096a160610041506w374b315fy32ea1ba4f6826c5 at mail.gmail.com>, "Joseph Mc
Donald" writes:
>Hi,
>
>my backend does gzip compression for clients that support it.  However
>it appears that varnish caches and sends out whatever it got first.
>That is, if a client supports gzip,  than it caches the gzip version.
>If the next client does not support gzip, varnish still sends out the
>gzip version.  and vice versa.
>
>Do I need to configure something for varnish to respect the
>Content-encoding header?

We havn't implemented the Vary: header yet, so we are not able
to do this trick yet.


-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From phk at phk.freebsd.dk  Thu Oct  5 09:38:05 2006
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Thu, 05 Oct 2006 09:38:05 +0000
Subject: chroot or/and none root user 
In-Reply-To: Your message of "Thu, 05 Oct 2006 09:27:42 +0200."
	<4524B3EE.3060505@linpro.no> 
Message-ID: <354.1160041085@critter.freebsd.dk>

In message <4524B3EE.3060505 at linpro.no>, Ingvar Hagelund writes:
>* Benny Kjellgren
>>> I have started to test Varnish on Debian (and Archlinux) and the
>>> first thing I was looking for is to how to run Varnish as a none
>>> root user (eg www-data) and/or put it in a jail (chroot).
>>>
>>> Is it possible to do that ?
>
>* Poul-Henning Kamp
>> If you don't use a priviledged listen port, it should just work.
>
>Any plans to use the model used by eg apache or squid, with a
>dispatching process running in priviledged mode, and process children
>running with a non privilegded user?

We could do that, having the management process run as root and
he child as some other user.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From Anders.Berg at vg.no  Thu Oct  5 09:40:52 2006
From: Anders.Berg at vg.no (Anders Berg)
Date: Thu, 5 Oct 2006 11:40:52 +0200
Subject: SV: chroot or/and none root user 
Message-ID: <EFA0DD92845D8749A63720971210CE08143C50@VG-EXC-VIR-1.Akersgt.local>

If you feel it makes sense Poul-Henning, I certainly do.

Anders Berg

> -----Opprinnelig melding-----
> Fra: varnish-misc-bounces at projects.linpro.no 
> [mailto:varnish-misc-bounces at projects.linpro.no] P? vegne av 
> Poul-Henning Kamp
> Sendt: 5. oktober 2006 11:38
> Til: varnish-misc at projects.linpro.no
> Emne: Re: chroot or/and none root user 
> 
> In message <4524B3EE.3060505 at linpro.no>, Ingvar Hagelund writes:
> >* Benny Kjellgren
> >>> I have started to test Varnish on Debian (and Archlinux) and the 
> >>> first thing I was looking for is to how to run Varnish as a none 
> >>> root user (eg www-data) and/or put it in a jail (chroot).
> >>>
> >>> Is it possible to do that ?
> >
> >* Poul-Henning Kamp
> >> If you don't use a priviledged listen port, it should just work.
> >
> >Any plans to use the model used by eg apache or squid, with a 
> >dispatching process running in priviledged mode, and process 
> children 
> >running with a non privilegded user?
> 
> We could do that, having the management process run as root 
> and he child as some other user.
> 
> -- 
> Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
> phk at FreeBSD.ORG         | TCP/IP since RFC 956
> FreeBSD committer       | BSD since 4.3-tahoe    
> Never attribute to malice what can adequately be explained by 
> incompetence.
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-misc
> 

*****************************************************************
Denne fotnoten bekrefter at denne e-postmeldingen ble
skannet av MailSweeper og funnet fri for virus.
*****************************************************************
This footnote confirms that this email message has been swept by
MailSweeper for the presence of computer viruses.
*****************************************************************


From benny.kjellgren at trig.com  Thu Oct  5 12:11:43 2006
From: benny.kjellgren at trig.com (Benny Kjellgren (TRIG/adocca))
Date: Thu, 5 Oct 2006 14:11:43 +0200
Subject: Originating IP address to backend
Message-ID: <20061005121143.GC2811@trig.com>

Hi,

My backend need to determine originating IP address.
Is it a way to make Varnish to set HTTP_CLIENT_IP and/or
HTTP_X_FORWARDED_FOR ?

-- 
Benny Kjellgren


From phk at phk.freebsd.dk  Thu Oct  5 20:29:16 2006
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Thu, 05 Oct 2006 20:29:16 +0000
Subject: Originating IP address to backend 
In-Reply-To: Your message of "Thu, 05 Oct 2006 14:11:43 +0200."
	<20061005121143.GC2811@trig.com> 
Message-ID: <2663.1160080156@critter.freebsd.dk>

In message <20061005121143.GC2811 at trig.com>, "Benny Kjellgren (TRIG/adocca)" wr
ites:
>Hi,
>
>My backend need to determine originating IP address.
>Is it a way to make Varnish to set HTTP_CLIENT_IP and/or
>HTTP_X_FORWARDED_FOR ?

I already added code for X-Forwarded-For after the release.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From stonorn at giraffen.dk  Thu Oct  5 21:21:58 2006
From: stonorn at giraffen.dk (Anton Stonor)
Date: Thu, 05 Oct 2006 23:21:58 +0200
Subject: Prefetching a page from backend after timeout/purge
Message-ID: <45257776.7060002@giraffen.dk>

Varnish looks very promissing--thank you guys for making it happen.

A small question: I've been playing around with VCL and would like 
Varnish to prefetch a fresh page from the backend after a timeout. The 
idea is to spare the end user from waiting for a slow backend the first 
time a page gets fetched after a timeout/purge.

Should'nt this ...

sub vcl_timeout {
     fetch;
}

... do the trick? It doesn't seem to do what I expect.

The same issue when it comes to PURGE. I was hoping you could sneak a 
"fetch" in here, but it doesn't work.

sub vcl_hit {
         if (req.request == "PURGE") {
                 set obj.ttl = 0s;
                 fetch;
                 error 200 "Purged.";
         }
}

Is there a way to let Varnish fetch a fresh page immediately after a PURGE?


/Anton


From phk at phk.freebsd.dk  Thu Oct  5 22:27:21 2006
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Thu, 05 Oct 2006 22:27:21 +0000
Subject: Prefetching a page from backend after timeout/purge 
In-Reply-To: Your message of "Thu, 05 Oct 2006 23:21:58 +0200."
	<45257776.7060002@giraffen.dk> 
Message-ID: <3073.1160087241@critter.freebsd.dk>

In message <45257776.7060002 at giraffen.dk>, Anton Stonor writes:
>Varnish looks very promissing--thank you guys for making it happen.
>
>A small question: I've been playing around with VCL and would like 
>Varnish to prefetch a fresh page from the backend after a timeout. The 
>idea is to spare the end user from waiting for a slow backend the first 
>time a page gets fetched after a timeout/purge.
>
>Should'nt this ...
>
>sub vcl_timeout {
>     fetch;
>}
>
>... do the trick? It doesn't seem to do what I expect.

Yes, but it's not implemented yet.

>The same issue when it comes to PURGE. I was hoping you could sneak a 
>"fetch" in here, but it doesn't work.

No, fetch is not a legal action in vcl_hit.

Maybe it should be, but it isn't right now.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From stonorn at giraffen.dk  Fri Oct  6 04:48:06 2006
From: stonorn at giraffen.dk (Anton Stonor)
Date: Fri, 06 Oct 2006 06:48:06 +0200
Subject: Prefetching a page from backend after timeout/purge
In-Reply-To: <3073.1160087241@critter.freebsd.dk>
References: <3073.1160087241@critter.freebsd.dk>
Message-ID: <4525E006.5030705@giraffen.dk>

Poul-Henning Kamp wrote:
>> Should'nt this ...
>>
>> sub vcl_timeout {
>>     fetch;
>> }

> Yes, but it's not implemented yet.

OK, thanks. Looking forward to it.

/Anton


From rolf.sommerhalder at alumni.ethz.ch  Sun Oct  8 08:41:00 2006
From: rolf.sommerhalder at alumni.ethz.ch (Rolf Sommerhalder)
Date: Sun, 8 Oct 2006 10:41:00 +0200
Subject: OpebBSD port of Varnish
Message-ID: <9893390f0610080141i5fe57f54k83d2402fb6fb262a@mail.gmail.com>

Hello list

while attempting to build Varnish on OpenBSD-current I stumbled across
one tiny and a larger issue at ompile time:

1. varnish-1.0.1/lib/libvarnish/cli_common.c requires an additional
   #include <sys/types.h>
before
   #include <sys/uio.h>

With this tiny issue out of the way, the next one appears to be more
substantial:

2. varnish-1.0.1/lbin/varnishd/cache_acceptor_kqueue.c makes use of
the kevent filter EVFILT_TIMER which is apparently not supported in
OpenBSD.
Unfortunately, I am not fluent enough a kernel programmer to
paraphrase this kevent into an OpenBSD equivalent without considerable
effort up front.

Do you intend to increase portability and add support beyond Linux and
FreeBSD anyway?

I am looking forward to test Varnish under OpenBSD on i386, sparc and
sparc64 archs, and eventually to maintain an OpenBSD port/package.
Thank you for making Varnish available.

Regards,
Rolf


From des at linpro.no  Mon Oct  9 10:12:08 2006
From: des at linpro.no (Dag-Erling =?iso-8859-1?Q?Sm=F8rgrav?=)
Date: Mon, 09 Oct 2006 12:12:08 +0200
Subject: OpebBSD port of Varnish
In-Reply-To: <9893390f0610080141i5fe57f54k83d2402fb6fb262a@mail.gmail.com>
	(Rolf Sommerhalder's message of "Sun, 8 Oct 2006 10:41:00 +0200")
References: <9893390f0610080141i5fe57f54k83d2402fb6fb262a@mail.gmail.com>
Message-ID: <ujrwt794x3b.fsf@cat.linpro.no>

"Rolf Sommerhalder" <rolf.sommerhalder at alumni.ethz.ch> writes:
> 2. varnish-1.0.1/lbin/varnishd/cache_acceptor_kqueue.c makes use of
> the kevent filter EVFILT_TIMER which is apparently not supported in
> OpenBSD.
> Unfortunately, I am not fluent enough a kernel programmer to
> paraphrase this kevent into an OpenBSD equivalent without considerable
> effort up front.

Without EVFILT_TIMER, much of the point of using kqueue is gone.  I'd
suggest you try to lobby the OpenBSD developers to update their kqueue
implementation (FreeBSD has had EVFILT_TIMER for over five years).

> Do you intend to increase portability and add support beyond Linux and
> FreeBSD anyway?

I'm afraid it isn't high on our priority list.  We primarily target
the systems we use ourselves (FreeBSD 6 and 7, Ubuntu, RHEL / CentOS)

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


From chris+varnish-misc at schug.net  Tue Oct 10 08:18:24 2006
From: chris+varnish-misc at schug.net (Christoph Schug)
Date: Tue, 10 Oct 2006 10:18:24 +0200
Subject: PID file; VCL code generation
Message-ID: <20061010081824.GA4005@voodoo.schug.net>

Hi,

I just stumbled over Varnish and started playing around with it. While
I'm not a coder I've got two issues from the sysadmin's point of view.
I know, it's not good style being new on the list to place feature
requests in the very first post. Nevertheless, those are the very first
things I encountered on my initial deployment.

o PID file
  It would be very nice to have a PID file which can be used in some
  start/stop scripts. IMHO some killall(1) approch is not very nice,
  especially when running multiple instances of Varnish on the very same
  machine (e.g. release version and current SVN version).

o VCL code generation
  As far as I can see, lib/libvcl/vcc_compile.c just picks up some
  cc(1) for VCL code generation. As I have boxes with different C
  compilers installed I'm currently doing sed(1) substs at build
  time. Nevertheless I guess having a configurable path here would be
  benefical.

Thanks in advance,
-cs



From tabary at efrei.fr  Tue Oct 10 18:06:19 2006
From: tabary at efrei.fr (Cedric Tabary)
Date: Tue, 10 Oct 2006 20:06:19 +0200
Subject: caching php
Message-ID: <20061010180619.GA34666@efrei.fr>

Hi,

I installed varnish (from trunk) lauched with :
/usr/local/varnish/sbin/varnishd -a '0.0.0.0:80' -b'127.0.0.1:8080' -T
127.0.0.1:1664 -sfile,/tmp/varnish,1g -t 1200

But I have a problem with php scripts being cached, which is really annoying.

Normally a http response containing ETag or Last-Modified can be
cached but other should not !
Is it a wanted behavior of varnish not to respect this ? is it a bug ?

Any help would be welcome to help me not cache php.

Thanks


From stonorn at giraffen.dk  Wed Oct 11 09:03:43 2006
From: stonorn at giraffen.dk (Anton Stonor)
Date: Wed, 11 Oct 2006 11:03:43 +0200
Subject: caching php
In-Reply-To: <20061010180619.GA34666@efrei.fr>
References: <20061010180619.GA34666@efrei.fr>
Message-ID: <452CB36F.4070906@giraffen.dk>

Cedric Tabary wrote:

> I installed varnish (from trunk) lauched with :
> /usr/local/varnish/sbin/varnishd -a '0.0.0.0:80' -b'127.0.0.1:8080' -T
> 127.0.0.1:1664 -sfile,/tmp/varnish,1g -t 1200
> 
> But I have a problem with php scripts being cached, which is really annoying.

You might want to loose the minimum caching time parameter (-t 1200).


/Anton


From tabary at efrei.fr  Wed Oct 11 09:31:45 2006
From: tabary at efrei.fr (Cedric Tabary)
Date: Wed, 11 Oct 2006 11:31:45 +0200
Subject: caching php
In-Reply-To: <452CB36F.4070906@giraffen.dk>
References: <20061010180619.GA34666@efrei.fr> <452CB36F.4070906@giraffen.dk>
Message-ID: <20061011093145.GB34666@efrei.fr>

On 11/10/2006 11:03, Anton Stonor wrote:
> Cedric Tabary wrote:
> 
> >I installed varnish (from trunk) lauched with :
> >/usr/local/varnish/sbin/varnishd -a '0.0.0.0:80' -b'127.0.0.1:8080' -T
> >127.0.0.1:1664 -sfile,/tmp/varnish,1g -t 1200
> >
> >But I have a problem with php scripts being cached, which is really 
> >annoying.
> 
> You might want to loose the minimum caching time parameter (-t 1200).

I removed the -t 1200, but my php is still cached !

(I also tried -t 0 but nothing is cached then)

C?dric


From stonorn at giraffen.dk  Wed Oct 11 14:17:09 2006
From: stonorn at giraffen.dk (Anton Stonor)
Date: Wed, 11 Oct 2006 16:17:09 +0200
Subject: caching php
In-Reply-To: <20061011093145.GB34666@efrei.fr>
References: <20061010180619.GA34666@efrei.fr> <452CB36F.4070906@giraffen.dk>
	<20061011093145.GB34666@efrei.fr>
Message-ID: <452CFCE5.4000209@giraffen.dk>

Cedric Tabary wrote:

> I removed the -t 1200, but my php is still cached !

Well, maybe the HTTP headers from the backend invites Varnish to cache. 
Or there _could_ be a bug in Varnish.

Could you post an example HTTP header of an object you expect Varnish 
not to cache?

http://www.web-caching.com/ is a great resource about HTTP headers and 
caching, btw.

/Anton


From tabary at efrei.fr  Wed Oct 11 14:29:15 2006
From: tabary at efrei.fr (Cedric Tabary)
Date: Wed, 11 Oct 2006 16:29:15 +0200
Subject: caching php
In-Reply-To: <452CFCE5.4000209@giraffen.dk>
References: <20061010180619.GA34666@efrei.fr> <452CB36F.4070906@giraffen.dk>
	<20061011093145.GB34666@efrei.fr> <452CFCE5.4000209@giraffen.dk>
Message-ID: <20061011142915.GC34666@efrei.fr>

On 11/10/2006 16:17, Anton Stonor wrote:
> Cedric Tabary wrote:
> 
> >I removed the -t 1200, but my php is still cached !
> 
> Well, maybe the HTTP headers from the backend invites Varnish to cache. 
> Or there _could_ be a bug in Varnish.
> 
> Could you post an example HTTP header of an object you expect Varnish 
> not to cache?

I have a simple php script as example :
time.php:
<?
echo time();
?>

Here are the headers from the backend directly :
root at php10:~/varnish-cache% HEAD http://127.0.0.1:8080/time.php
200 OK
Connection: close
Date: Wed, 11 Oct 2006 14:20:08 GMT
Server: Apache/1.3.34 (Unix) PHP/4.4.2 mod_ssl/2.8.25 OpenSSL/0.9.7g
Content-Type: text/html; charset=iso-8859-1
Client-Date: Wed, 11 Oct 2006 14:20:08 GMT
Client-Peer: 127.0.0.1:8080
Client-Response-Num: 1
X-Powered-By: PHP/4.4.2

And the headers through varnish :
root at php10:~/varnish-cache% HEAD http://127.0.0.1/time.php
200 OK
Connection: close
Date: Wed, 11 Oct 2006 14:22:42 GMT
Via: 1.1 varnish
Age: 5
Server: Apache/1.3.34 (Unix) PHP/4.4.2 mod_ssl/2.8.25 OpenSSL/0.9.7g
Content-Length: 10
Content-Type: text/html; charset=iso-8859-1
Client-Date: Wed, 11 Oct 2006 14:22:47 GMT
Client-Peer: 127.0.0.1:80
Client-Response-Num: 1
X-Powered-By: PHP/4.4.2
X-Varnish: 1281745475 1281745470


I also managed to get a log of varnish inserting the php script in cache :

   12 SessionOpen  c 127.0.0.1 36672
   12 ReqStart     c 127.0.0.1 36672 1281745566
   12 RxRequest    c HEAD
   12 RxURL        c /time.php
   12 RxProtocol   c HTTP/1.1
   12 RxHeader     c Connection: close
   12 RxHeader     c Host: 127.0.0.1
   12 RxHeader     c User-Agent: lwp-request/2.06
   12 VCL_call     c recv
   12 VCL_return   c lookup
   12 VCL_call     c miss
   12 VCL_return   c fetch
   15 BackendOpen  b default 127.0.0.1 24543 127.0.0.1 8080
   15 BackendXID   b 1281745566
   12 Backend      c 15 default
   15 TxRequest    b GET
   15 TxURL        b /time.php
   15 TxProtocol   b HTTP/1.1
   15 TxHeader     b Host: 127.0.0.1
   15 TxHeader     b User-Agent: lwp-request/2.06
   15 TxHeader     b X-Varnish: 1281745566
   15 TxHeader     b X-Forwarded-for: 127.0.0.1
   15 RxProtocol   b HTTP/1.1
   15 RxStatus     b 200
   15 RxResponse   b OK
   15 RxHeader     b Date: Wed, 11 Oct 2006 14:29:19 GMT
   15 RxHeader     b Server: Apache/1.3.34 (Unix) PHP/4.4.2
mod_ssl/2.8.25 OpenSSL/0.9.7g
   15 RxHeader     b X-Powered-By: PHP/4.4.2
   15 RxHeader     b Connection: close
   15 RxHeader     b Transfer-Encoding: chunked
   15 RxHeader     b Content-Type: text/html; charset=iso-8859-1
   12 TTL          c 1281745566 RFC 120 1160576959 1160576959 0 0 0
   12 VCL_call     c fetch
   12 VCL_return   c insert
   12 ObjProtocol  c HTTP/1.1
   12 ObjStatus    c 200
   12 ObjResponse  c OK
   12 ObjHeader    c Date: Wed, 11 Oct 2006 14:29:19 GMT
   12 ObjHeader    c Server: Apache/1.3.34 (Unix) PHP/4.4.2
mod_ssl/2.8.25 OpenSSL/0.9.7g
   12 ObjHeader    c X-Powered-By: PHP/4.4.2
   12 ObjHeader    c Content-Type: text/html; charset=iso-8859-1
   12 ObjHeader    c Content-Length: 10
   15 BackendClose b default
   12 Length       c 10
   12 TxProtocol   c HTTP/1.1
   12 TxStatus     c 200
   12 TxResponse   c OK
   12 TxHeader     c Date: Wed, 11 Oct 2006 14:29:19 GMT
   12 TxHeader     c Server: Apache/1.3.34 (Unix) PHP/4.4.2
mod_ssl/2.8.25 OpenSSL/0.9.7g
   12 TxHeader     c X-Powered-By: PHP/4.4.2
   12 TxHeader     c Content-Type: text/html; charset=iso-8859-1
   12 TxHeader     c Content-Length: 10
   12 TxHeader     c X-Varnish: 1281745566
   12 TxHeader     c Age: 0
   12 TxHeader     c Via: 1.1 varnish
   12 TxHeader     c Connection: close
   12 ReqEnd       c 1281745566 1160576959.257762000
1160576959.258505000 0.002817000 0.000721000 0.000022000
    0 StatAddr       127.0.0.1 0 397 11 11 0 0 3 3218 0
   12 SessionClose c Connection: close
   12 StatSess     c 127.0.0.1 36672 0 1 1 0 0 1 284 0


C?dric


From phk at phk.freebsd.dk  Wed Oct 11 14:37:55 2006
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Wed, 11 Oct 2006 14:37:55 +0000
Subject: caching php 
In-Reply-To: Your message of "Wed, 11 Oct 2006 16:29:15 +0200."
	<20061011142915.GC34666@efrei.fr> 
Message-ID: <1069.1160577475@critter.freebsd.dk>

In message <20061011142915.GC34666 at efrei.fr>, Cedric Tabary writes:
>On 11/10/2006 16:17, Anton Stonor wrote:
>> Cedric Tabary wrote:
>> 
>> >I removed the -t 1200, but my php is still cached !
>> 
>> Well, maybe the HTTP headers from the backend invites Varnish to cache. 
>> Or there _could_ be a bug in Varnish.
>> 
>> Could you post an example HTTP header of an object you expect Varnish 
>> not to cache?

Varnish has a "default TTL" which you can specify on the command line
or modify from the CLI.

Absent any instruction from the backend, Varnish will cache objects
that long.

Default is 2 minutes.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From tabary at efrei.fr  Wed Oct 11 14:51:29 2006
From: tabary at efrei.fr (Cedric Tabary)
Date: Wed, 11 Oct 2006 16:51:29 +0200
Subject: caching php
In-Reply-To: <1069.1160577475@critter.freebsd.dk>
References: <20061011142915.GC34666@efrei.fr>
	<1069.1160577475@critter.freebsd.dk>
Message-ID: <20061011145129.GD34666@efrei.fr>

On 11/10/2006 14:37, Poul-Henning Kamp wrote:
> In message <20061011142915.GC34666 at efrei.fr>, Cedric Tabary writes:
> >On 11/10/2006 16:17, Anton Stonor wrote:
> >> Cedric Tabary wrote:
> >> 
> >> >I removed the -t 1200, but my php is still cached !
> >> 
> >> Well, maybe the HTTP headers from the backend invites Varnish to cache. 
> >> Or there _could_ be a bug in Varnish.
> >> 
> >> Could you post an example HTTP header of an object you expect Varnish 
> >> not to cache?
> 
> Varnish has a "default TTL" which you can specify on the command line
> or modify from the CLI.
> 
> Absent any instruction from the backend, Varnish will cache objects
> that long.

Ok so I have to put -t 0 and specify a Cache-Control with mod_expires
on static content ?

And if I put a Cache-Control header in a specific php script I
can also make it cacheable ?

this sounds good to me :)
I will try this...

C?dric


From phk at phk.freebsd.dk  Wed Oct 11 15:05:56 2006
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Wed, 11 Oct 2006 15:05:56 +0000
Subject: caching php 
In-Reply-To: Your message of "Wed, 11 Oct 2006 16:51:29 +0200."
	<20061011145129.GD34666@efrei.fr> 
Message-ID: <1327.1160579156@critter.freebsd.dk>

In message <20061011145129.GD34666 at efrei.fr>, Cedric Tabary writes:

>> Varnish has a "default TTL" which you can specify on the command line
>> or modify from the CLI.
>> 
>> Absent any instruction from the backend, Varnish will cache objects
>> that long.
>
>Ok so I have to put -t 0 and specify a Cache-Control with mod_expires
>on static content ?

Yes.

>And if I put a Cache-Control header in a specific php script I
>can also make it cacheable ?

Yes.

>this sounds good to me :)

I hope so :-)

The 120 second default is sort of a stab at a good default
"anti-slashdotting" setting, but obviously it's open for discussion
if ot proves to be a Bad Thing.


-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From tabary at efrei.fr  Thu Oct 12 11:19:09 2006
From: tabary at efrei.fr (Cedric Tabary)
Date: Thu, 12 Oct 2006 13:19:09 +0200
Subject: Cookie and X-FOrwarded-For
Message-ID: <20061012111909.GE34666@efrei.fr>

Thanks for your help on caching php, but now I have another problem.
When I do a request with a cookie the X-Forwarded-For is not set.

C?dric


From des at linpro.no  Fri Oct 13 08:50:35 2006
From: des at linpro.no (Dag-Erling =?iso-8859-1?Q?Sm=F8rgrav?=)
Date: Fri, 13 Oct 2006 10:50:35 +0200
Subject: Cookie and X-FOrwarded-For
In-Reply-To: <20061012111909.GE34666@efrei.fr> (Cedric Tabary's message of
	"Thu, 12 Oct 2006 13:19:09 +0200")
References: <20061012111909.GE34666@efrei.fr>
Message-ID: <ujrmz80y4ys.fsf@cat.linpro.no>

Cedric Tabary <tabary at efrei.fr> writes:
> Thanks for your help on caching php, but now I have another problem.
> When I do a request with a cookie the X-Forwarded-For is not set.

If you're running 1.0.1, it's a known bug that has been fixed in
trunk.

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


From des at linpro.no  Fri Oct 13 08:52:08 2006
From: des at linpro.no (Dag-Erling =?iso-8859-1?Q?Sm=F8rgrav?=)
Date: Fri, 13 Oct 2006 10:52:08 +0200
Subject: PID file; VCL code generation
In-Reply-To: <20061010081824.GA4005@voodoo.schug.net> (Christoph Schug's
	message of "Tue, 10 Oct 2006 10:18:24 +0200")
References: <20061010081824.GA4005@voodoo.schug.net>
Message-ID: <ujririoy4w7.fsf@cat.linpro.no>

Christoph Schug <chris+varnish-misc at schug.net> writes:
> o PID file
>   It would be very nice to have a PID file which can be used in some
>   start/stop scripts. IMHO some killall(1) approch is not very nice,
>   especially when running multiple instances of Varnish on the very same
>   machine (e.g. release version and current SVN version).

Yes, that would be a good idea.

> o VCL code generation
>   As far as I can see, lib/libvcl/vcc_compile.c just picks up some
>   cc(1) for VCL code generation. As I have boxes with different C
>   compilers installed I'm currently doing sed(1) substs at build
>   time. Nevertheless I guess having a configurable path here would be
>   benefical.

Ideally, this would be farmed out to a helper script to allow easy
modification of the compiler command and parameters.

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


From tabary at efrei.fr  Fri Oct 13 09:51:34 2006
From: tabary at efrei.fr (Cedric Tabary)
Date: Fri, 13 Oct 2006 11:51:34 +0200
Subject: Cookie and X-FOrwarded-For
In-Reply-To: <ujrmz80y4ys.fsf@cat.linpro.no>
References: <20061012111909.GE34666@efrei.fr> <ujrmz80y4ys.fsf@cat.linpro.no>
Message-ID: <20061013095134.GF34666@efrei.fr>

On 13/10/2006 10:50, Dag-Erling Smrgrav wrote:
> Cedric Tabary <tabary at efrei.fr> writes:
> > Thanks for your help on caching php, but now I have another problem.
> > When I do a request with a cookie the X-Forwarded-For is not set.
> 
> If you're running 1.0.1, it's a known bug that has been fixed in
> trunk.

Strange because I was running trunk.

/usr/local/varnish/sbin/varnishd -a '0.0.0.0:80' -b'127.0.0.1:8080' -T
127.0.0.1:1664 -sfile,/tmp/varnish,1g -t 0

I added this in PassSession() in cache_pass.c
    http_PrintfHeader(w, vc->fd, vc->http,
            "X-Forwarded-for: %s", sp->addr);

I don't know if it is the right place but it worked for me :)

C?dric



From chris+varnish-misc at schug.net  Fri Oct 13 12:24:33 2006
From: chris+varnish-misc at schug.net (Christoph Schug)
Date: Fri, 13 Oct 2006 14:24:33 +0200
Subject: PID file; VCL code generation
In-Reply-To: <ujririoy4w7.fsf@cat.linpro.no>
References: <20061010081824.GA4005@voodoo.schug.net>
	<ujririoy4w7.fsf@cat.linpro.no>
Message-ID: <20061013122433.GE30977@voodoo.schug.net>

On Fri, Oct 13, 2006, Dag-Erling Sm?rgrav wrote:

> Christoph Schug <chris+varnish-misc at schug.net> writes:
> > o PID file
[...]
> 
> Yes, that would be a good idea.

In the meantime I packaged Varnish as OpenPKG package using this patch
against 1.0.1 by setting -DOPKG_PIDFILE=/foo/bar/varnishd.pid in
CPPFLAGS. Not elegant, but works for me :)

Index: bin/varnishd/mgt_child.c
--- bin/varnishd/mgt_child.c.orig	2006-09-20 19:18:51 +0200
+++ bin/varnishd/mgt_child.c	2006-10-11 22:42:04 +0200
@@ -172,6 +172,21 @@
 
 	fprintf(stderr, "start child pid %d\n", i);
 
+#ifdef OPKG_PIDFILE
+	{
+		pid_t pid;
+		FILE *fp;
+
+		pid = getpid();
+		if ((fp = fopen(OPKG_PIDFILE, "w")) == NULL) {
+			fprintf(stderr, "varnishd:ERROR: cannot write PID file '%s'\n", OPKG_PIDFILE);
+			exit(1);
+		}
+		fprintf(fp, "%ld\n", (long)pid);
+		fclose(fp);
+	}
+#endif
+
 	AZ(close(child_fds[1]));
 	child_fds[1] = -1;

-cs


From phk at phk.freebsd.dk  Fri Oct 13 12:40:01 2006
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Fri, 13 Oct 2006 12:40:01 +0000
Subject: PID file; VCL code generation 
In-Reply-To: Your message of "Fri, 13 Oct 2006 10:52:08 +0200."
	<ujririoy4w7.fsf@cat.linpro.no> 
Message-ID: <1036.1160743201@critter.freebsd.dk>

In message <ujririoy4w7.fsf at cat.linpro.no>, Dag-Erling =?iso-8859-1?Q?Sm=F8rgra
v?= writes:

>> o VCL code generation
>>   As far as I can see, lib/libvcl/vcc_compile.c just picks up some
>>   cc(1) for VCL code generation. As I have boxes with different C
>>   compilers installed I'm currently doing sed(1) substs at build
>>   time. Nevertheless I guess having a configurable path here would be
>>   benefical.
>
>Ideally, this would be farmed out to a helper script to allow easy
>modification of the compiler command and parameters.

No, it should just be a parameter.  We don't want to make
Varnish dependent on a lot of auxillary files.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From des at linpro.no  Fri Oct 13 13:03:41 2006
From: des at linpro.no (Dag-Erling =?iso-8859-1?Q?Sm=F8rgrav?=)
Date: Fri, 13 Oct 2006 15:03:41 +0200
Subject: PID file; VCL code generation
In-Reply-To: <1036.1160743201@critter.freebsd.dk> (Poul-Henning Kamp's message
	of "Fri, 13 Oct 2006 12:40:01 +0000")
References: <1036.1160743201@critter.freebsd.dk>
Message-ID: <ujrfydsmkpe.fsf@cat.linpro.no>

"Poul-Henning Kamp" <phk at phk.freebsd.dk> writes:
> "Dag-Erling Sm?rgrav" <des at des.no> writes:
> > Ideally, this would be farmed out to a helper script to allow easy
> > modification of the compiler command and parameters.
> No, it should just be a parameter.  We don't want to make
> Varnish dependent on a lot of auxillary files.

Some platforms may require more than a single command to produce a
loadable object; for instance, the object file generated by the
compiler may need processing with objcopy or ld or some similar tool.
Letting a helper script handle this is by far the easiest solution.

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


From phk at phk.freebsd.dk  Fri Oct 13 13:07:43 2006
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Fri, 13 Oct 2006 13:07:43 +0000
Subject: PID file; VCL code generation 
In-Reply-To: Your message of "Fri, 13 Oct 2006 15:03:41 +0200."
	<ujrfydsmkpe.fsf@cat.linpro.no> 
Message-ID: <1274.1160744863@critter.freebsd.dk>

In message <ujrfydsmkpe.fsf at cat.linpro.no>, Dag-Erling =?iso-8859-1?Q?Sm=F8rgra
v?= writes:
>"Poul-Henning Kamp" <phk at phk.freebsd.dk> writes:
>> "Dag-Erling Sm=F8rgrav" <des at des.no> writes:
>> > Ideally, this would be farmed out to a helper script to allow easy
>> > modification of the compiler command and parameters.
>> No, it should just be a parameter.  We don't want to make
>> Varnish dependent on a lot of auxillary files.
>
>Some platforms may require more than a single command to produce a
>loadable object; for instance, the object file generated by the
>compiler may need processing with objcopy or ld or some similar tool.
>Letting a helper script handle this is by far the easiest solution.

Nothing you cannot do with a couple of semicolons and "sh -c $command"

I really really really do not want to see Varnish sprout a ton
of weird support files which people have to think about.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From des at linpro.no  Fri Oct 13 13:10:35 2006
From: des at linpro.no (Dag-Erling =?iso-8859-1?Q?Sm=F8rgrav?=)
Date: Fri, 13 Oct 2006 15:10:35 +0200
Subject: PID file; VCL code generation
In-Reply-To: <1274.1160744863@critter.freebsd.dk> (Poul-Henning Kamp's message
	of "Fri, 13 Oct 2006 13:07:43 +0000")
References: <1274.1160744863@critter.freebsd.dk>
Message-ID: <ujrbqogmkdw.fsf@cat.linpro.no>

"Poul-Henning Kamp" <phk at phk.freebsd.dk> writes:
> I really really really do not want to see Varnish sprout a ton
> of weird support files which people have to think about.

Hopefully, the only person who needs to think about the compiler
helper script is the person doing the packaging for your favorite
distribution...

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


From chris+varnish-misc at schug.net  Fri Oct 13 13:15:08 2006
From: chris+varnish-misc at schug.net (Christoph Schug)
Date: Fri, 13 Oct 2006 15:15:08 +0200
Subject: PID file; VCL code generation
In-Reply-To: <ujrbqogmkdw.fsf@cat.linpro.no>
References: <1274.1160744863@critter.freebsd.dk>
	<ujrbqogmkdw.fsf@cat.linpro.no>
Message-ID: <20061013131508.GG30977@voodoo.schug.net>

On Fri, Oct 13, 2006, Dag-Erling Sm?rgrav wrote:

> "Poul-Henning Kamp" <phk at phk.freebsd.dk> writes:
> > I really really really do not want to see Varnish sprout a ton
> > of weird support files which people have to think about.
> 
> Hopefully, the only person who needs to think about the compiler
> helper script is the person doing the packaging for your favorite
> distribution...

I don't see a difference, whether the program called is a real C
compiler or just a shell script which sorts out platform-specific stuff
(whoever this one likes to maintain). In case it is a user configurable
path with default to the current setting it simply doen't matter, does
it?

-cs


From phk at phk.freebsd.dk  Fri Oct 13 13:23:07 2006
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Fri, 13 Oct 2006 13:23:07 +0000
Subject: PID file; VCL code generation 
In-Reply-To: Your message of "Fri, 13 Oct 2006 15:15:08 +0200."
	<20061013131508.GG30977@voodoo.schug.net> 
Message-ID: <1333.1160745787@critter.freebsd.dk>

In message <20061013131508.GG30977 at voodoo.schug.net>, Christoph Schug writes:
>On Fri, Oct 13, 2006, Dag-Erling Sm?rgrav wrote:
>
>> "Poul-Henning Kamp" <phk at phk.freebsd.dk> writes:
>> > I really really really do not want to see Varnish sprout a ton
>> > of weird support files which people have to think about.
>> 
>> Hopefully, the only person who needs to think about the compiler
>> helper script is the person doing the packaging for your favorite
>> distribution...
>
>I don't see a difference, whether the program called is a real C
>compiler or just a shell script which sorts out platform-specific stuff
>(whoever this one likes to maintain). In case it is a user configurable
>path with default to the current setting it simply doen't matter, does
>it?

One of the design goals of Varnish was "one single file".  If you
have ever had an emergency at 2:05AM, you know that to be a very
important feature.

I have yet to see one single computer where the command to build a
shared object could not be encoded in a single command line and
therefore I see no reason to invalidate one of our goals for
that reason.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From phk at phk.freebsd.dk  Fri Oct 13 13:23:48 2006
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Fri, 13 Oct 2006 13:23:48 +0000
Subject: PID file; VCL code generation 
In-Reply-To: Your message of "Fri, 13 Oct 2006 15:10:35 +0200."
	<ujrbqogmkdw.fsf@cat.linpro.no> 
Message-ID: <1344.1160745828@critter.freebsd.dk>

In message <ujrbqogmkdw.fsf at cat.linpro.no>, Dag-Erling =?iso-8859-1?Q?Sm=F8rgra
v?= writes:
>"Poul-Henning Kamp" <phk at phk.freebsd.dk> writes:
>> I really really really do not want to see Varnish sprout a ton
>> of weird support files which people have to think about.
>
>Hopefully, the only person who needs to think about the compiler
>helper script is the person doing the packaging for your favorite
>distribution...

As I said, we'll make the string a run time parameter, and if
people insist on polluting their system with pointless scripts,
that allows them to do so.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From lists at tomster.org  Fri Oct 13 13:29:16 2006
From: lists at tomster.org (Tom Lazar)
Date: Fri, 13 Oct 2006 15:29:16 +0200
Subject: PID file; VCL code generation 
In-Reply-To: <1333.1160745787@critter.freebsd.dk>
References: <1333.1160745787@critter.freebsd.dk>
Message-ID: <F8E9317E-A0CF-4B92-80EE-5D0D8358C0FE@tomster.org>

On Oct 13, 2006, at 3:23 PM, Poul-Henning Kamp wrote:

> One of the design goals of Varnish was "one single file".  If you
> have ever had an emergency at 2:05AM, you know that to be a very
> important feature.

FWIW:

+1 (!!)

at any rate, version 1.0.2 doesn't seem like a good candidate to  
break with that design goal already ;-)

>
> I have yet to see one single computer where the command to build a
> shared object could not be encoded in a single command line and
> therefore I see no reason to invalidate one of our goals for
> that reason.
>
> -- 
> Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
> phk at FreeBSD.ORG         | TCP/IP since RFC 956
> FreeBSD committer       | BSD since 4.3-tahoe
> Never attribute to malice what can adequately be explained by  
> incompetence.
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-misc
>



From chris+varnish-misc at schug.net  Fri Oct 13 13:37:30 2006
From: chris+varnish-misc at schug.net (Christoph Schug)
Date: Fri, 13 Oct 2006 15:37:30 +0200
Subject: PID file; VCL code generation
In-Reply-To: <1333.1160745787@critter.freebsd.dk>
References: <20061013131508.GG30977@voodoo.schug.net>
	<1333.1160745787@critter.freebsd.dk>
Message-ID: <20061013133730.GH30977@voodoo.schug.net>

On Fri, Oct 13, 2006, Poul-Henning Kamp wrote:

> I have yet to see one single computer where the command to build a
> shared object could not be encoded in a single command line and
> therefore I see no reason to invalidate one of our goals for
> that reason.

I fully agree with you. What I was trying to say, if we have a facility
to call user-defineable external commands, it simply doesn't matter
whether this command is single line command or a somewhat more complex
command, which itself calls 'cc', etc. As long as there's a safe
default, just everybody can stick with it. Those (like me) who'd like to
define some non-default value can shoot in their feet with every flavor
they want. One of the nice features of a swiss army knife is that you
can retract the blades before putting it the pocket. But you don't have
too ;-)

-cs


From gaute at pht.no  Mon Oct 30 08:13:18 2006
From: gaute at pht.no (Gaute Amundsen)
Date: Mon, 30 Oct 2006 09:13:18 +0100
Subject: Rewriting urls?
Message-ID: <200610300913.19421.gaute@pht.no>

Got a "Warning: could not send message for past 4 hours" on this message, s? 
I'm trying again.. 
--

Hi.

I am looking into using varnish as a front for Zope.
I have only skimmed the docs so far, but I could find nothing obvious at 
least.

In apache what I need to do looks like this:

RewriteRule ^/(.*) 
http://localhost:8080/web/MMBase/http/www.website.no:80/customers/cust_name/MMRoot/$1
[P]

Assignment is not possible to req.url it seems, and none of the examples I 
have found works...


set req.url = "foo" + req.url
rewrite        s/foo$/bar/
rewrite req.url.host "cnn.no" "vg.no"


Regards

Gaute Amundsen




From gaute at pht.no  Sat Oct 28 13:47:05 2006
From: gaute at pht.no (Gaute Amundsen)
Date: Sat, 28 Oct 2006 15:47:05 +0200
Subject: Rewriting urls?
Message-ID: <200610281547.05719.gaute@pht.no>

Hi.

I am looking into using varnish as a front for Zope.
I have only skimmed the docs so far, but I could find nothing obvious at 
least.

In apache what I need to do looks like this:

RewriteRule ^/(.*) 
http://localhost:8080/web/MMBase/http/www.website.no:80/customers/cust_name/MMRoot/$1
[P]

Assignment is not possible to req.url it seems, and none of the examples I 
have found works...


set req.url = "foo" + req.url
rewrite        s/foo$/bar/
rewrite req.url.host "cnn.no" "vg.no"


Regards

Gaute Amundsen




From Allan.J at cobsen.dk  Mon Oct 30 12:38:31 2006
From: Allan.J at cobsen.dk (Allan Jacobsen)
Date: Mon, 30 Oct 2006 13:38:31 +0100 (CET)
Subject: Problem with varnish
Message-ID: <5ap9vqln.1162211911.4277360.allan@localhost>


Hi

I wanted to try out varnish, so I downloaded the RHES4 rpm, and have not
had any problem installing it, but when I try to run with it, it does
not seem to cache anything.....

I am running siege from my desktop PC at 192.168.200.17 to the RHES
server on 192.168.200.93, and the apache server is on a third server at
192.168.200.91.
The /etc/varnish/default.net.vcl is like this:

backend webserver {
        set backend.host = "192.168.200.91";
        set backend.port = "80";
}

sub vcl_recv {
        if (req.http.host ~ "") {
                set req.backend = webserver;
        } else {
                error 404 "You requested a document from an unknown
virtual host.";
        }
}

But every request seems to be sent directly on to 192.168.200.91. I am
looking at varnishstat on another console and the "Client requests
received" and "Backend connections initiated" are identical.
Am i missing something obvious ?

Best regards
Allan Jacobsen


From phk at phk.freebsd.dk  Tue Oct 31 18:17:37 2006
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Tue, 31 Oct 2006 18:17:37 +0000
Subject: Problem with varnish 
In-Reply-To: Your message of "Mon, 30 Oct 2006 13:38:31 +0100."
	<5ap9vqln.1162211911.4277360.allan@localhost> 
Message-ID: <10758.1162318657@critter.freebsd.dk>

In message <5ap9vqln.1162211911.4277360.allan at localhost>, "Allan Jacobsen" writ
es:

>sub vcl_recv {
>        if (req.http.host ~ "") {
>                set req.backend = webserver;

What exactly is the test for here ?  That there isn't a host: header ?

In that case, try:

	if (!req.http.host) {

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From phk at phk.freebsd.dk  Tue Oct 31 18:18:34 2006
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Tue, 31 Oct 2006 18:18:34 +0000
Subject: Rewriting urls? 
In-Reply-To: Your message of "Sat, 28 Oct 2006 15:47:05 +0200."
	<200610281547.05719.gaute@pht.no> 
Message-ID: <10772.1162318714@critter.freebsd.dk>

In message <200610281547.05719.gaute at pht.no>, Gaute Amundsen writes:

>I am looking into using varnish as a front for Zope.
>I have only skimmed the docs so far, but I could find nothing obvious at 
>least.

We are not as far with the VCL functions as I had hoped, and amongst
the things you can't do yet is rewrites of that order.

Stay tuned, it will come.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From ingvar at linpro.no  Tue Oct 31 18:26:29 2006
From: ingvar at linpro.no (Ingvar Hagelund)
Date: Tue, 31 Oct 2006 19:26:29 +0100
Subject: Problem with varnish
In-Reply-To: <5ap9vqln.1162211911.4277360.allan@localhost>
References: <5ap9vqln.1162211911.4277360.allan@localhost>
Message-ID: <1162319189.3816.11.camel@re.e37>

* Allan Jacobsen:
> I wanted to try out varnish, so I downloaded the RHES4 rpm, and have not
> had any problem installing it, but when I try to run with it, it does
> not seem to cache anything.....

There are some issues with varnish 1.0.1. A bugfix release 1.0.2 which
probably will fix at least your initial problems is in the cooking, and
will probably hit the sf mirrors "real soon".

For those who can't wait, consider building varnish from subversion, and
have a look at http://varnish.projects.linpro.no/wiki/Repository .

To build a rpm from a svn checkout, have a look at
http://varnish.projects.linpro.no/browser/trunk/varnish-cache/redhat/README.redhat 

Best regards, and thanks for trying varnish,
Ingvar Hagelund

-- 



From allan.j at cobsen.dk  Tue Oct 31 19:04:28 2006
From: allan.j at cobsen.dk (Allan Jacobsen)
Date: Tue, 31 Oct 2006 20:04:28 +0100
Subject: Problem with varnish
In-Reply-To: <10758.1162318657@critter.freebsd.dk>
References: <10758.1162318657@critter.freebsd.dk>
Message-ID: <200610312004.30938.allan.j@cobsen.dk>

On Tuesday 31 October 2006 19:17, Poul-Henning Kamp wrote:
> In message <5ap9vqln.1162211911.4277360.allan at localhost>, "Allan Jacobsen" 
writ
> es:
> 
> >sub vcl_recv {
> >        if (req.http.host ~ "") {
> >                set req.backend = webserver;
> 
> What exactly is the test for here ?  That there isn't a host: header ?
> 
> In that case, try:
> 
> 	if (!req.http.host) {

Actually i have no idea what i am doing, it is just the default file from the 
rpm package, I have just changed the backend server settings.

Best regards
Allan Jacobsen




From lists at tomster.org  Sat Oct 28 17:10:16 2006
From: lists at tomster.org (Tom Lazar)
Date: Sat, 28 Oct 2006 19:10:16 +0200
Subject: FYI: Installation instructions updated to include Mac OS X
Message-ID: <83581A1D-E1BD-4377-A97C-C0F75D3F72E8@tomster.org>

I just installed varnish successfully on my OSX boxen and edited the  
installation instructions[1] accordingly.

it's just two additional steps:

--snip--
1. install `automake`, i.e. via [http://darwinports.opendarwin.org/  
Darwin Ports]: `sudo port install automake` as the version of  
`automake` that ships with OS X is too old.

2. patch the `autogen.sh` file:

{{{
10a11,15
 > if [ -e /opt/local/bin/automake ] ; then
 >       PATH=/opt/local/bin:${PATH}
 >       export PATH
 > fi
 >
30c35
< libtoolize --copy --force
---
 > glibtoolize --copy --force
}}}

''n.b. this assumes you are using the abovementioned [http:// 
darwinports.opendarwin.org/ Darwin Ports]. The key point is to insert  
the path to your newer version of `automake` '''before''' the system  
path in order to 'hide' the old version from `autogen.sh`.''

--snap--


cheers,

tom

[1] http://varnish.projects.linpro.no/wiki/Installation


From ltning at anduin.net  Tue Oct 31 19:39:44 2006
From: ltning at anduin.net (=?ISO-8859-1?Q?Eirik_=D8verby?=)
Date: Tue, 31 Oct 2006 20:39:44 +0100
Subject: FYI: Installation instructions updated to include Mac OS X
In-Reply-To: <83581A1D-E1BD-4377-A97C-C0F75D3F72E8@tomster.org>
References: <83581A1D-E1BD-4377-A97C-C0F75D3F72E8@tomster.org>
Message-ID: <9CDE3BE2-853F-4C02-924D-279FBBDA53DA@anduin.net>

On Oct 28, 2006, at 7:10 PM, Tom Lazar wrote:

> I just installed varnish successfully on my OSX boxen and edited  
> the installation instructions[1] accordingly.

Just curious, what happened to those missing time functions on OSX?
/Eirik

>
> it's just two additional steps:
>
> --snip--
> 1. install `automake`, i.e. via [http://darwinports.opendarwin.org/  
> Darwin Ports]: `sudo port install automake` as the version of  
> `automake` that ships with OS X is too old.
>
> 2. patch the `autogen.sh` file:
>
> {{{
> 10a11,15
> > if [ -e /opt/local/bin/automake ] ; then
> >       PATH=/opt/local/bin:${PATH}
> >       export PATH
> > fi
> >
> 30c35
> < libtoolize --copy --force
> ---
> > glibtoolize --copy --force
> }}}
>
> ''n.b. this assumes you are using the abovementioned [http:// 
> darwinports.opendarwin.org/ Darwin Ports]. The key point is to  
> insert the path to your newer version of `automake` '''before'''  
> the system path in order to 'hide' the old version from  
> `autogen.sh`.''
>
> --snap--
>
>
> cheers,
>
> tom
>
> [1] http://varnish.projects.linpro.no/wiki/Installation
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-misc
>
>



From phk at phk.freebsd.dk  Tue Oct 31 19:41:09 2006
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Tue, 31 Oct 2006 19:41:09 +0000
Subject: Varnish version 2 and beyond
Message-ID: <11162.1162323669@critter.freebsd.dk>



Varnish version 1 has been out for some time now, and it is time
to talk about varnish version 2 and the future in general.

I told Anders that I wanted a bit of vacation before I even
contemplated the future.  That is why I have been relatively silent
and inactive since the release.

But I have also talked to a number of early Varnish users and given
a couple of talks about Varnish in various settings and the reception
have been very warm and postive.

And then everybody asks me "What happens next ?  What about version 2 ?"

So lets me open the discussion about version 2 with a loooong email:


What ?
======

The features I have heard people ask for so far are:

    Things we already know how to do:
	Bugfixes
	Docs
	More VCL
	Simple Server side includes.
	Vary: header processing (requirement for compression etc.)
	Compression.
	tar storage method for static/emergency contents
	The cluster controller

    Tasks which can be delegated:
	Tcpdump-like specification to varnishlog(1) etc.
	RRD statistics
	GUI / Dashboard

    Things we need to think more about how to do:
	Prefetching
	backend loadbalancing

    Things we need to think a lot about how to do:
	Cookie/Auth handling
	URL-scrubber functionality (exploits etc)

As a programmer I want to add, high on the list:

	Regression test driver
	Regression test cases


How ?
=====

The plan Anders talked about for version 2, was to go through a
process setting up a user-group with a governing board, writing
specification documents for Varnish version 2, produce time estimates,
write a budget and pass the hat around and see what happened.

Even though I am not very interested in the administrative side of
the project, I would strongly advice setting up some kind of
"core-team" or governing board, well ahead of the time where it
becomes necessary to have one.


Who ?
=====

Some of the tasks above are probably best left in my hands while
others would be better off delegated.  As good first approximation
is that if it is in the Varnishd binary it is probably "my turf"
if it is outside it can delegated.

If you guys get a better offer from somebody else or decide you
can't afford my services, I won't disappear from Varnish in toto.

Chances are I might still land a contract or two with people who
need help with Varnish, but my involvement would be a sparetime
activity and obviously a lot less intense than otherwise.


How Much ?
==========

As you know VG.no sponsored version 1.0 and I don't know if the
amount they spent is really secret or not.  I trust Anders will
disclose it if he feels this is necessary for the discussion.

(Currently linpro.no runs the project servers, I don't know under
what arrangement that happens.  If they cannot find enough Varnish
work in Norway, we may have to pay them to do so, or migrate it to
sourceforge or whatever.)

For my own part, I feed my kids by working as an independent
contractor and as such I have to send invoices to somebody every
so often.  I have plenty of uncommitted hours in 2007 still, and
if people want to pay me to continue on Varnish, I'm game.  (I
do have a little bit of available time left in 2006 also).

For reasons of personal development I prefer to not get nailed down
in just one area of computing.  I also have a number of faithful
customers which keep coming back with interesting stuff for me to
work on.

So I try to only put half of my time up for any one customer when
it comes to long term contracts.  I work better that way too, it
gives me more time to think about the issues in the background.

For the magnitude of work we are talking about here, my listprice
is DKR610/h (USD105/h) which after my 10% discount on BSD licensed
Open Source, becomes DKR550/h (USD94/h) (For danish customers sales
tax would be on top of that).

If you guys prefer to go through the formal excercise and define
tasks, milestones etc. etc, then it's fine for me.  The only two
conditions I set is that you guys set up a governing board so I
have a competent and authoritative management to work with on all
the paper and that you tell me where to send my invoices.  This
last bit could easily be the most tricky.

But Seeing the domainnames we get email from, I get the impression
that finding funding for future Varnish development may not be as
troublesome as we anticipated when we began, so a much simpler
alternative is also an option:

If you guys are willing to pay me DKR50K/month (USD8600/month) and
agree to a 3 month notice of termination, I will commit to using
half of my time on Varnish in whatever direction we together decide
to take it.  In that scenario I'm willing to send invoices directly
to whoever the donor of the month is.  That way we don't have to
wait for a formal user organization to get the financial side
hashed out.

The main downside to this way of doing things is that it does
exactly take away alot of the incentive to get a solid user
organization set up.

My advice is that you first decide the organizational aspect(s)
before you start the technical bits and prioritzation of the wishlist.

I think that was what I had to say right now and I think you guys
should talk/email/whisper in smokefilled rooms/fight it out with
EMACS macros or however else you want to settle between yourselves
how you want to progress from here.


Poul-Henning

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.