From nagita.k at gmail.com  Wed Jan  3 17:08:28 2007
From: nagita.k at gmail.com (Nagita Karunaratne)
Date: Wed, 3 Jan 2007 09:08:28 -0800
Subject: Urchin
Message-ID: <b2990fb00701030908h7fd26381o84dbd9c276364c34@mail.gmail.com>

Hello,

I am not a true sysadmin but I would like to run Varnish with a open source
CMS hosting sites in a virtual hosting environment (apache/mysql).

If your customer are using Urchin to track visitors will it be able to work
correctly if some of the pages are cached with Varnish?

Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20070103/3ccde00f/attachment.html>

From dwetzel at nerim.net  Wed Jan  3 17:38:16 2007
From: dwetzel at nerim.net (Damien Wetzel)
Date: Wed, 3 Jan 2007 18:38:16 +0100
Subject: Urchin
In-Reply-To: <b2990fb00701030908h7fd26381o84dbd9c276364c34@mail.gmail.com>
References: <b2990fb00701030908h7fd26381o84dbd9c276364c34@mail.gmail.com>
Message-ID: <17819.59912.294217.253336@dwetzel@nerim.net>

Hello Nagita,
I think there is no reasons why urchin wouldn't work.
any hit to a cached or not page would result in a hit to urchin.
Damien,
Nagita Karunaratne writes:
 > Hello,
 > 
 > I am not a true sysadmin but I would like to run Varnish with a open source
 > CMS hosting sites in a virtual hosting environment (apache/mysql).
 > 
 > If your customer are using Urchin to track visitors will it be able to work
 > correctly if some of the pages are cached with Varnish?
 > 
 > Thanks
 > Hello,<br><br>I am not a true sysadmin but I would like to run Varnish with a open source CMS hosting sites in a virtual hosting environment (apache/mysql).<br><br>If your customer are using Urchin to track visitors will it be able to work correctly if some of the pages are cached with Varnish?
 > <br><br>Thanks<br>
 > _______________________________________________
 > varnish-misc mailing list
 > varnish-misc at projects.linpro.no
 > http://projects.linpro.no/mailman/listinfo/varnish-misc



From phk at phk.freebsd.dk  Wed Jan  3 17:10:13 2007
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Wed, 03 Jan 2007 17:10:13 +0000
Subject: Urchin 
In-Reply-To: Your message of "Wed, 03 Jan 2007 09:08:28 PST."
	<b2990fb00701030908h7fd26381o84dbd9c276364c34@mail.gmail.com> 
Message-ID: <13889.1167844213@critter.freebsd.dk>

In message <b2990fb00701030908h7fd26381o84dbd9c276364c34 at mail.gmail.com>, "Nagi
ta Karunaratne" writes:

>I am not a true sysadmin but I would like to run Varnish with a open source
>CMS hosting sites in a virtual hosting environment (apache/mysql).
>
>If your customer are using Urchin to track visitors will it be able to work
>correctly if some of the pages are cached with Varnish?

Good question.

I'm not aware of how Urchin works so I can't really give you an answer.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From xi at borderworlds.dk  Thu Jan  4 21:32:08 2007
From: xi at borderworlds.dk (Christian Laursen)
Date: Thu, 04 Jan 2007 22:32:08 +0100
Subject: Urchin
In-Reply-To: <13889.1167844213@critter.freebsd.dk> (Poul-Henning Kamp's
	message of "Wed, 03 Jan 2007 17:10:13 +0000")
References: <13889.1167844213@critter.freebsd.dk>
Message-ID: <ygfy7oi32cn.fsf@dominion.borderworlds.dk>

"Poul-Henning Kamp" <phk at phk.freebsd.dk> writes:

> In message <b2990fb00701030908h7fd26381o84dbd9c276364c34 at mail.gmail.com>, "Nagi
> ta Karunaratne" writes:
>
>>If your customer are using Urchin to track visitors will it be able to work
>>correctly if some of the pages are cached with Varnish?
>
> Good question.
>
> I'm not aware of how Urchin works so I can't really give you an answer.

Urchin (aka Google Analytics) is pure client side and works by requesting some
javascript from their servers from the web pages.

-- 
Christian Laursen


From nagita.k at gmail.com  Fri Jan  5 05:30:51 2007
From: nagita.k at gmail.com (Nagita Karunaratne)
Date: Thu, 4 Jan 2007 21:30:51 -0800
Subject: Urchin
In-Reply-To: <ygfy7oi32cn.fsf@dominion.borderworlds.dk>
References: <13889.1167844213@critter.freebsd.dk>
	<ygfy7oi32cn.fsf@dominion.borderworlds.dk>
Message-ID: <b2990fb00701042130l7ea2e353vcfaf04e7252eb33d@mail.gmail.com>

On 1/4/07, Christian Laursen <xi at borderworlds.dk> wrote:
>
>
> Urchin (aka Google Analytics) is pure client side and works by requesting
> some
> javascript from their servers from the web pages.



Actually I was referring to Urchin 5 not the Google version.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20070104/2a9783d3/attachment.html>

From nagita.k at gmail.com  Sun Jan 14 06:09:36 2007
From: nagita.k at gmail.com (Nagita Karunaratne)
Date: Sat, 13 Jan 2007 22:09:36 -0800
Subject: does varnish write objects to disk?
Message-ID: <b2990fb00701132209y2c38c1bemc96f45d587d9a7ae@mail.gmail.com>

Varnish caches objects into *memory* and not disk right?

So there is no point in assigning a disk drive solely to varnish for
caching objects in an effort to improve performance?

Can anyone give a very high level of how Varnish works? I've read the
presentations and most of the archives but not very clear on this
topic.

Thanks,
Nagita


From phk at phk.freebsd.dk  Mon Jan 15 07:35:39 2007
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Mon, 15 Jan 2007 07:35:39 +0000
Subject: does varnish write objects to disk? 
In-Reply-To: Your message of "Sat, 13 Jan 2007 22:09:36 PST."
	<b2990fb00701132209y2c38c1bemc96f45d587d9a7ae@mail.gmail.com> 
Message-ID: <22422.1168846539@critter.freebsd.dk>

In message <b2990fb00701132209y2c38c1bemc96f45d587d9a7ae at mail.gmail.com>, "Nagi
ta Karunaratne" writes:

>Varnish caches objects into *memory* and not disk right?

No, varnish caches objects into "virtual memory" which means that
RAM acts as a cache for the disk.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From hema at tv2.dk  Fri Jan 12 08:59:06 2007
From: hema at tv2.dk (Henrik Nicolaisen)
Date: Fri, 12 Jan 2007 09:59:06 +0100
Subject: Correct no cache headers ?
Message-ID: <5AD6C93C-0E2B-4748-BD22-16A28F8261A8@tv2.dk>

Hi

What are the correct cache headers to set if I need to tell varnish  
not to cache a file.

Right now I use :

Pragma: no-cache
Cache-Control: s-maxage=0, max-age=0, must-revalidate, no-cache

But sometimes I get some other users cached file, and it seems to be  
when I try to get a file in the exact same moment as som eother user.

In vcl i have the following code to allow caching af some of the  
files even though a user is logged in.

         # force lookup even when cookies are present
         if (req.request == "GET" && req.http.cookie) {
                 lookup;
         }

And when i request the file I get the following in varnishlog

    15 VCL_call     c recv
    15 VCL_return   c lookup
    15 VCL_call     c miss
    15 VCL_return   c fetch

    16 RxHeader     b Pragma: no-cache
    16 RxHeader     b Cache-Control: s-maxage=0, max-age=0, must- 
revalidate, no-cache
    16 RxHeader     b Expires: Fri, 05 Jan 2007 08:57:03 GMT
    16 RxHeader     b Content-Length: 1555
    16 RxHeader     b Connection: close
    16 RxHeader     b Content-Type: text/javascript
    15 TTL          c 277100755 RFC 0 1168592154 1168592223  
1167987423 0 0
    15 VCL_call     c fetch
    15 VCL_return   c insert

Could it be the insert that is made by varnish that is the problem ?


Regards,

Henrik Nicolaisen
TV 2 Interaktiv


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20070112/f4702f2e/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 1931 bytes
Desc: not available
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20070112/f4702f2e/attachment.bin>

From phk at phk.freebsd.dk  Mon Jan 15 09:54:55 2007
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Mon, 15 Jan 2007 09:54:55 +0000
Subject: Correct no cache headers ? 
In-Reply-To: Your message of "Fri, 12 Jan 2007 09:59:06 +0100."
	<5AD6C93C-0E2B-4748-BD22-16A28F8261A8@tv2.dk> 
Message-ID: <23994.1168854895@critter.freebsd.dk>

In message <5AD6C93C-0E2B-4748-BD22-16A28F8261A8 at tv2.dk>, Henrik Nicolaisen wri
tes:

>But sometimes I get some other users cached file, and it seems to be  
>when I try to get a file in the exact same moment as som eother user.

Hmm, yes, that could possibly happen with the current code.

Can I get you to file a bug report in the trac system ?

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From ff at partyticket.net  Mon Jan 15 18:30:29 2007
From: ff at partyticket.net (Flemming Frandsen)
Date: Mon, 15 Jan 2007 19:30:29 +0100
Subject: does varnish write objects to disk?
In-Reply-To: <22422.1168846539@critter.freebsd.dk>
References: <22422.1168846539@critter.freebsd.dk>
Message-ID: <45ABC845.8010202@partyticket.net>

Poul-Henning Kamp wrote:
>>Varnish caches objects into *memory* and not disk right?
> 
> No, varnish caches objects into "virtual memory" which means that
> RAM acts as a cache for the disk.

I guess the interesting detail is how eager the kernel is to write the 
mapped memory to disk.

It would be a shame to spend time and IO bandwidth to write a 1G cache 
out to disk if the box has 4G of RAM and would never need the RAM for 
anything other than the cache, woudn't it?

-- 
  Regards Flemming Frandsen - http://dion.swamp.dk - YAPH



From phk at phk.freebsd.dk  Mon Jan 15 18:33:29 2007
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Mon, 15 Jan 2007 18:33:29 +0000
Subject: does varnish write objects to disk? 
In-Reply-To: Your message of "Mon, 15 Jan 2007 19:30:29 +0100."
	<45ABC845.8010202@partyticket.net> 
Message-ID: <26916.1168886009@critter.freebsd.dk>

In message <45ABC845.8010202 at partyticket.net>, Flemming Frandsen writes:
>Poul-Henning Kamp wrote:
>>>Varnish caches objects into *memory* and not disk right?
>> 
>> No, varnish caches objects into "virtual memory" which means that
>> RAM acts as a cache for the disk.
>
>I guess the interesting detail is how eager the kernel is to write the 
>mapped memory to disk.
>
>It would be a shame to spend time and IO bandwidth to write a 1G cache 
>out to disk if the box has 4G of RAM and would never need the RAM for 
>anything other than the cache, woudn't it?

The kernels I know of can be described as distinctively reluctant
to write things out.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From nagita.k at gmail.com  Mon Jan 15 18:55:42 2007
From: nagita.k at gmail.com (Nagita Karunaratne)
Date: Mon, 15 Jan 2007 10:55:42 -0800
Subject: does varnish write objects to disk?
In-Reply-To: <26916.1168886009@critter.freebsd.dk>
References: <45ABC845.8010202@partyticket.net>
	<26916.1168886009@critter.freebsd.dk>
Message-ID: <b2990fb00701151055v76e4983bma935611bebc8a7e0@mail.gmail.com>

Thanks for your help. I take it you let the kernel manage as much as possible.

But I really want to know how I can improve performance in a
multi-cpu, multi-core processor systems running multiple cms-websites
in a virtual hosting configuration outside the scope of the kernel.

For example, a specific RAID configuration that will make it easier
for varnish. Or do I leave it all up to the kernel.

Is FreeBSD and Linux2.6 equally capable with respect to varnish?

Thanks,
Nagita


On 1/15/07, Poul-Henning Kamp <phk at phk.freebsd.dk> wrote:
> In message <45ABC845.8010202 at partyticket.net>, Flemming Frandsen writes:
> >Poul-Henning Kamp wrote:
> >>>Varnish caches objects into *memory* and not disk right?
> >>
> >> No, varnish caches objects into "virtual memory" which means that
> >> RAM acts as a cache for the disk.
> >
> >I guess the interesting detail is how eager the kernel is to write the
> >mapped memory to disk.
> >
> >It would be a shame to spend time and IO bandwidth to write a 1G cache
> >out to disk if the box has 4G of RAM and would never need the RAM for
> >anything other than the cache, woudn't it?
>
> The kernels I know of can be described as distinctively reluctant
> to write things out.
>
> --
> Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
> phk at FreeBSD.ORG         | TCP/IP since RFC 956
> FreeBSD committer       | BSD since 4.3-tahoe
> Never attribute to malice what can adequately be explained by incompetence.
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-misc
>


From phk at phk.freebsd.dk  Mon Jan 15 18:59:57 2007
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Mon, 15 Jan 2007 18:59:57 +0000
Subject: does varnish write objects to disk? 
In-Reply-To: Your message of "Mon, 15 Jan 2007 10:55:42 PST."
	<b2990fb00701151055v76e4983bma935611bebc8a7e0@mail.gmail.com> 
Message-ID: <27039.1168887597@critter.freebsd.dk>

In message <b2990fb00701151055v76e4983bma935611bebc8a7e0 at mail.gmail.com>, "Nagi
ta Karunaratne" writes:

>But I really want to know how I can improve performance in a
>multi-cpu, multi-core processor systems running multiple cms-websites
>in a virtual hosting configuration outside the scope of the kernel.
>
>For example, a specific RAID configuration that will make it easier
>for varnish. Or do I leave it all up to the kernel.

We have very little experience with these kinds of questions so far,
mainly because nobody has told usmuch about their experiences yet.

(This would be a good time for our users to chime in :-)

>Is FreeBSD and Linux2.6 equally capable with respect to varnish?

Until now I have not heard of measurable differences.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From Kenneth.Rorvik at hio.no  Wed Jan 17 08:08:11 2007
From: Kenneth.Rorvik at hio.no (=?ISO-8859-1?Q?Kenneth_R=F8rvik?=)
Date: Wed, 17 Jan 2007 09:08:11 +0100
Subject: Moving a redirect from backend to varnish
Message-ID: <45ADD96B.5080503@hio.no>

Hi there. I am currently responsible for a reasonably large cms 
(http://www.hio.no). Now, the problem is, my predecessor has *not* 
implemented a cache for this site, and the server is getting loaded down 
quite heavily due to many mysql-requests.

Enter varnish (1.0.2-7 on Red Hat AS4) .... but, there is a problem: 
poorly documented vcl ;)

One of the most important aspects about the site is that we have an 
apache-rewrite on "/" for all http-requests from our OWN networks when 
there is no Referer header set. The reason is we (or, our information 
department.... ;) want all internal requests to / to end up on the news 
page on first open, and subsequent clicks on links to / to actually take 
you to /.

Now, a proxy will certainly mess up this apache rewrite. The apache 
rewrite itself looks like this (truncated for brevity):

RewriteCond %{REMOTE_ADDR}  ^128\.39\.75\.*  [OR]
RewriteCond %{REMOTE_ADDR}  ^128\.39\.89\.*  [OR]
RewriteCond %{REMOTE_ADDR}  ^2001:700:700:\.*
RewriteCond %{HTTP_REFERER} !^http://www\.hio\.no
RewriteRule ^/$ /content/view/full/27 [R=permanent,L]

A rather simple redirect.

The problem is, with varnish in place, this will of course not have the 
desired effect. the vcl man page tells us a few things, but there have 
been some problems implementing this redirect.

At the moment, I am playing with using an acl to check if the client is 
on an internal network, and then see if Referer is set and the page 
being requested is /, and then act accordingly.

There are two problems in this. The first is making the test act 
accordingly, the second is the acl syntax.. first, the test. I am 
currently using something like:

if (client ~ internal && req.url == "/" && req.http.referer) {...}

The acl uses a syntax which I've noticed in the list archives for this 
list is possibly bugged for checking IP's (?):

acl internal {
         "158.36.161.1"/24;
         "158.36.78.1"/23;
	...
	...
}

Now, the man page uses .1 for the IPs, but .0 should be more intuitive? 
Also the /24 should probably be placed inside the quotes?

Now, provided I get the right test in place - the next question is how 
to handle the redirect itself. You could either send the request on to 
the backend (but this is not good caching... ;), using pipe (?), or we 
could tell the client directly that the document is moved, and what the 
correct url is. But how could I get varnish to simply say "beeeep, no, 
go to THIS page instead" using something like a http 301?

Or is there another solution that the current documentation does not expose?

Oh, and is there any need for credits to the varnishproject if I use it 
on http://www.hio.no?

-- 
Kenneth R?rvik, IT HiO
Tlf 22 45 20 83
Kenneth.Rorvik at hio.no


From ltning at anduin.net  Wed Jan 17 08:29:36 2007
From: ltning at anduin.net (=?ISO-8859-1?Q?Eirik_=D8verby?=)
Date: Wed, 17 Jan 2007 09:29:36 +0100
Subject: Moving a redirect from backend to varnish
In-Reply-To: <45ADD96B.5080503@hio.no>
References: <45ADD96B.5080503@hio.no>
Message-ID: <973DB3CB-4ADE-40B9-B151-BDC35ED99386@anduin.net>


On Jan 17, 2007, at 09:08, Kenneth R?rvik wrote:

> Hi there. I am currently responsible for a reasonably large cms  
> (http://www.hio.no). Now, the problem is, my predecessor has *not*  
> implemented a cache for this site, and the server is getting loaded  
> down quite heavily due to many mysql-requests.
>
> Enter varnish (1.0.2-7 on Red Hat AS4) .... but, there is a  
> problem: poorly documented vcl ;)
>
> One of the most important aspects about the site is that we have an  
> apache-rewrite on "/" for all http-requests from our OWN networks  
> when there is no Referer header set. The reason is we (or, our  
> information department.... ;) want all internal requests to / to  
> end up on the news page on first open, and subsequent clicks on  
> links to / to actually take you to /.
>
> Now, a proxy will certainly mess up this apache rewrite. The apache  
> rewrite itself looks like this (truncated for brevity):
>
> RewriteCond %{REMOTE_ADDR}  ^128\.39\.75\.*  [OR]
> RewriteCond %{REMOTE_ADDR}  ^128\.39\.89\.*  [OR]
> RewriteCond %{REMOTE_ADDR}  ^2001:700:700:\.*
> RewriteCond %{HTTP_REFERER} !^http://www\.hio\.no
> RewriteRule ^/$ /content/view/full/27 [R=permanent,L]
>
> A rather simple redirect.

I'm not going to solve your problem here, but I have a question: Why  
are you using a permanent redirect here?? It is, from my  
understanding, not at all a permanent redirect, it's a conditional  
one -- and you have no intention to blocking access to the / page  
from internal users; you just want them to get there following a link  
somewhere in the www.hio.no hive.
 From my limited knowledge of HTTP, the way you are doing it here  
should be causing trouble with browsers who do what they are allowed  
to - i.e. cache the redirect (as the browser will never see any of  
the conditions anyway). Not to mention all the caches you could be  
poisoning, if you have any such internally (and with hio having a  
presumably rather large network, I wouldn't automatically assume that  
your cache would be the only one, authorized or not).

Just curious -- and looking forward to some helpful answers to your  
questions ;)

/Eirik

>
> The problem is, with varnish in place, this will of course not have  
> the desired effect. the vcl man page tells us a few things, but  
> there have been some problems implementing this redirect.
>
> At the moment, I am playing with using an acl to check if the  
> client is on an internal network, and then see if Referer is set  
> and the page being requested is /, and then act accordingly.
>
> There are two problems in this. The first is making the test act  
> accordingly, the second is the acl syntax.. first, the test. I am  
> currently using something like:
>
> if (client ~ internal && req.url == "/" && req.http.referer) {...}
>
> The acl uses a syntax which I've noticed in the list archives for  
> this list is possibly bugged for checking IP's (?):
>
> acl internal {
>         "158.36.161.1"/24;
>         "158.36.78.1"/23;
> 	...
> 	...
> }
>
> Now, the man page uses .1 for the IPs, but .0 should be more  
> intuitive? Also the /24 should probably be placed inside the quotes?
>
> Now, provided I get the right test in place - the next question is  
> how to handle the redirect itself. You could either send the  
> request on to the backend (but this is not good caching... ;),  
> using pipe (?), or we could tell the client directly that the  
> document is moved, and what the correct url is. But how could I get  
> varnish to simply say "beeeep, no, go to THIS page instead" using  
> something like a http 301?
>
> Or is there another solution that the current documentation does  
> not expose?
>
> Oh, and is there any need for credits to the varnishproject if I  
> use it on http://www.hio.no?
>
> -- 
> Kenneth R?rvik, IT HiO
> Tlf 22 45 20 83
> Kenneth.Rorvik at hio.no
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-misc
>
>



From Kenneth.Rorvik at hio.no  Wed Jan 17 09:54:41 2007
From: Kenneth.Rorvik at hio.no (=?ISO-8859-1?Q?Kenneth_R=F8rvik?=)
Date: Wed, 17 Jan 2007 10:54:41 +0100
Subject: Moving a redirect from backend to varnish
In-Reply-To: <973DB3CB-4ADE-40B9-B151-BDC35ED99386@anduin.net>
References: <45ADD96B.5080503@hio.no>
	<973DB3CB-4ADE-40B9-B151-BDC35ED99386@anduin.net>
Message-ID: <45ADF261.3070507@hio.no>

Eirik ?verby wrote:

> I'm not going to solve your problem here, but I have a question: Why are 
> you using a permanent redirect here?? It is, from my understanding, not 
> at all a permanent redirect, it's a conditional one -- and you have no 
> intention to blocking access to the / page from internal users; you just 
> want them to get there following a link somewhere in the www.hio.no hive.
>  From my limited knowledge of HTTP, the way you are doing it here should 
> be causing trouble with browsers who do what they are allowed to - i.e. 
> cache the redirect (as the browser will never see any of the conditions 
> anyway). Not to mention all the caches you could be poisoning, if you 
> have any such internally (and with hio having a presumably rather large 
> network, I wouldn't automatically assume that your cache would be the 
> only one, authorized or not).

Hmm, thinking about this, the client IS getting a http 301 and a correct 
URL for subsequent use, so I don't think this would be poisoning caches?


-- 
Kenneth R?rvik, IT HiO
Tlf 22 45 20 83
Kenneth.Rorvik at hio.no


From Kenneth.Rorvik at hio.no  Wed Jan 17 11:53:26 2007
From: Kenneth.Rorvik at hio.no (=?ISO-8859-1?Q?Kenneth_R=F8rvik?=)
Date: Wed, 17 Jan 2007 12:53:26 +0100
Subject: Moving a redirect from backend to varnish
In-Reply-To: <45ADD96B.5080503@hio.no>
References: <45ADD96B.5080503@hio.no>
Message-ID: <45AE0E36.30403@hio.no>

Kenneth R?rvik wrote:

> Now, provided I get the right test in place - the next question is how 
> to handle the redirect itself. You could either send the request on to 

Looks like the test is OK now. Next problem: piping to the server now 
makes it impossible for the server to distinguish clients (since they 
all come from varnish), so the next Q is:

How can I have varnish itself send a http 301 or 307 to the client with 
the correct URL? Cannot find this documented, or on google for that matter.

-- 
Kenneth R?rvik, IT HiO
Tlf 22 45 20 83
Kenneth.Rorvik at hio.no


From Kenneth.Rorvik at hio.no  Wed Jan 17 11:58:55 2007
From: Kenneth.Rorvik at hio.no (=?ISO-8859-1?Q?Kenneth_R=F8rvik?=)
Date: Wed, 17 Jan 2007 12:58:55 +0100
Subject: Moving a redirect from backend to varnish
In-Reply-To: <45AE0E36.30403@hio.no>
References: <45ADD96B.5080503@hio.no> <45AE0E36.30403@hio.no>
Message-ID: <45AE0F7F.2030604@hio.no>

> How can I have varnish itself send a http 301 or 307 to the client with 
> the correct URL? Cannot find this documented, or on google for that matter.

Hmm, a bit quick:

http://projects.linpro.no/pipermail/varnish-misc/2006-October/000104.html

Any thoughts on when this can be expected?

-- 
Kenneth R?rvik, IT HiO
Tlf 22 45 20 83
Kenneth.Rorvik at hio.no


From ssm at linpro.no  Wed Jan 17 12:08:13 2007
From: ssm at linpro.no (Stig Sandbeck Mathisen)
Date: Wed, 17 Jan 2007 13:08:13 +0100
Subject: Moving a redirect from backend to varnish
In-Reply-To: <45AE0E36.30403@hio.no> (Kenneth =?utf-8?Q?R=C3=B8rvik's?=
	message of "Wed\, 17 Jan 2007 12\:53\:26 +0100")
References: <45ADD96B.5080503@hio.no> <45AE0E36.30403@hio.no>
Message-ID: <7xtzypn9ea.fsf@linpro.no>

Kenneth R?rvik <Kenneth.Rorvik at hio.no> writes:

> Kenneth R?rvik wrote:
>
>> Now, provided I get the right test in place - the next question is
>> how to handle the redirect itself. You could either send the request
>> on to 
>
> Looks like the test is OK now. Next problem: piping to the server now
> makes it impossible for the server to distinguish clients (since they
> all come from varnish), so the next Q is:
>
> How can I have varnish itself send a http 301 or 307 to the client
> with the correct URL? Cannot find this documented, or on google for
> that matter.

Varnish adds (or modifies an existing?) X-Forwarded-For header,
containing a path of IP addresses of the client and additional
proxies.

You should be able to use the server-variable %{HTTP_FORWARDED}, or
perhaps %{HTTP_X_FORWARDED_FROM}, for matching the client address with
RewriteCond. 

Note that it is possible for the user to add information to this
header when sending a request.

-- 
Stig Sandbeck Mathisen, Linpro


From patrick at zill.net  Tue Jan 30 06:16:48 2007
From: patrick at zill.net (Patrick Giagnocavo)
Date: Tue, 30 Jan 2007 01:16:48 -0500
Subject: VCL help needed
Message-ID: <65c91d15180d6f5e92000bd1931e8db8@zill.net>

I have been using varnish for a particular web blog/forum app, so far 
it is working well, except that forum users get confused when they 
delete or add a post and do not see the changed page immediately; so 
they either double-click (meaning a duplicate post) or try to delete 
something that has already been deleted.

In any case, the first rule is "don't confuse the user", so since I 
have trained the users to expect one kind of behavior I can't expect 
them to automatically adjust to a different behavior.

I ended up commenting out the vcl_fetch routine in the default vcl.conf 
and setting default_ttl to 15 seconds on the command line.  Basically 
this is almost disabling the cache (hits are 10% or so).

Based on my reading, what I think will work best, is to set default_ttl 
to a very low value like 5 seconds, then use a regexp to specify 
explicitly what should be cached.

This works because what I want to cache are the .css, .gif, .jpg, .swf, 
.mp3 files - since virtually all the actual HTML is dynamically 
generated out of a database, simply off-loading the delivery of other 
files is the key thing I want varnish to do.

e.g. "if filename ends in .gif|.jpg|.css|.mp3 add to cache and set 
timeout to 600 seconds" ; otherwise default_ttl at 5 seconds will 
expire the page and eliminate double-clicks and double-deletes.

Can I put a regexp in the vcl_fetch routine?  Just add something like 
this to the stock vcl.conf:

sub vcl_fetch {
        if (req.url  - "/.css|.gif|.jpg|.mp3|.swf/") {

              # force minimum ttl of 600 seconds (mod from orig. 180s)

		      if (obj.ttl < 600s) {
                   set obj.ttl = 600s;
                   }
              }
}

I am sure I have the syntax wrong, and the regular expression is wrong; 
but I think I am clear on what I am trying to do.

Cordially
Patrick Giagnocavo
patrick at zill.net



From phk at phk.freebsd.dk  Tue Jan 30 09:18:51 2007
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Tue, 30 Jan 2007 09:18:51 +0000
Subject: VCL help needed 
In-Reply-To: Your message of "Tue, 30 Jan 2007 01:16:48 EST."
	<65c91d15180d6f5e92000bd1931e8db8@zill.net> 
Message-ID: <34447.1170148731@critter.freebsd.dk>

In message <65c91d15180d6f5e92000bd1931e8db8 at zill.net>, Patrick Giagnocavo writ
es:

>Can I put a regexp in the vcl_fetch routine?  Just add something like 
>this to the stock vcl.conf:
>
>sub vcl_fetch {
>        if (req.url  - "/.css|.gif|.jpg|.mp3|.swf/") {

	if (req.url ~ "\.css|\.gif|\.jpg|\.mp3|\.svf") {

If there are no tracker suffixes, it will be faster if you
anchor the regexps to the tail end of the url:

	if (req.url ~ "\.css$|\.gif$|\.jpg$|\.mp3$|\.svf$") {

Other than that, yes, it should work just fine.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From ff at suse.de  Tue Jan 30 10:43:29 2007
From: ff at suse.de (ff at suse.de)
Date: Tue, 30 Jan 2007 11:43:29 +0100 (CET)
Subject: VCL help needed
In-Reply-To: <34447.1170148731@critter.freebsd.dk>
References: Your message of "Tue, 30 Jan 2007 01:16:48 EST."
	<65c91d15180d6f5e92000bd1931e8db8@zill.net>
	<34447.1170148731@critter.freebsd.dk>
Message-ID: <8591.217.28.163.19.1170153809.squirrel@localhost>

> 	if (req.url ~ "\.css$|\.gif$|\.jpg$|\.mp3$|\.svf$") {

or with less typage, right?:
 	if (req.url ~ "\.(css|gif|jpg|mp3|swf)$") {




From phk at phk.freebsd.dk  Tue Jan 30 11:22:24 2007
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Tue, 30 Jan 2007 11:22:24 +0000
Subject: VCL help needed 
In-Reply-To: Your message of "Tue, 30 Jan 2007 11:43:29 +0100."
	<8591.217.28.163.19.1170153809.squirrel@localhost> 
Message-ID: <35118.1170156144@critter.freebsd.dk>

In message <8591.217.28.163.19.1170153809.squirrel at localhost>, ff at suse.de write
s:
>> 	if (req.url ~ "\.css$|\.gif$|\.jpg$|\.mp3$|\.svf$") {
>
>or with less typage, right?:
> 	if (req.url ~ "\.(css|gif|jpg|mp3|swf)$") {

Probably yes, I'm not a RE wizard either :-)

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From ask at develooper.com  Wed Jan 31 03:24:42 2007
From: ask at develooper.com (=?ISO-8859-1?Q?Ask_Bj=F8rn_Hansen?=)
Date: Tue, 30 Jan 2007 19:24:42 -0800
Subject: VCL help needed
In-Reply-To: <65c91d15180d6f5e92000bd1931e8db8@zill.net>
References: <65c91d15180d6f5e92000bd1931e8db8@zill.net>
Message-ID: <D0FE5446-CFB2-4131-AF67-46DD20334412@develooper.com>


On Jan 29, 2007, at 22:16, Patrick Giagnocavo wrote:

> This works because what I want to cache are  
> the .css, .gif, .jpg, .swf, .mp3 files - since virtually all the  
> actual HTML is dynamically generated out of a database, simply off- 
> loading the delivery of other files is the key thing I want varnish  
> to do.

Why not just serve the static media from a separate hostname?

On a related note, Cal Henderson wrote a good article about making  
css, javascript and image files extra cacheable.

	http://www.thinkvitamin.com/features/webapps/serving-javascript-fast


  - ask

-- 
http://develooper.com/ - http://askask.com/




From patrick at zill.net  Wed Jan 31 04:03:12 2007
From: patrick at zill.net (Patrick Giagnocavo)
Date: Tue, 30 Jan 2007 23:03:12 -0500
Subject: VCL help needed
In-Reply-To: <D0FE5446-CFB2-4131-AF67-46DD20334412@develooper.com>
References: <65c91d15180d6f5e92000bd1931e8db8@zill.net>
	<D0FE5446-CFB2-4131-AF67-46DD20334412@develooper.com>
Message-ID: <ca592761c965bbaba4cc779bca019f96@zill.net>


On Jan 30, 2007, at 10:24 PM, Ask Bj?rn Hansen wrote:

>
> On Jan 29, 2007, at 22:16, Patrick Giagnocavo wrote:
>
>> This works because what I want to cache are the .css, .gif, .jpg, 
>> .swf, .mp3 files - since virtually all the actual HTML is dynamically 
>> generated out of a database, simply off-loading the delivery of other 
>> files is the key thing I want varnish to do.
>
> Why not just serve the static media from a separate hostname?
>

The client would rather pay me than have to change all the hard-coded 
information in all the places.  And there are some other issues with 
the way the site is coded that might make it difficult (each user has a 
URL of the form username.domain.com) .

Besides we seem to be saving some CPU overhead just in having Varnish 
handle the sometimes-slow TCP connections with the users (not all are 
on high speed connections).

> On a related note, Cal Henderson wrote a good article about making 
> css, javascript and image files extra cacheable.
>
> 	http://www.thinkvitamin.com/features/webapps/serving-javascript-fast
>

thanks for the link!

--Patrick