From stam at stone-it.com  Tue Feb  1 22:00:03 2011
From: stam at stone-it.com (Erwin Stam|Stone-IT)
Date: Tue, 01 Feb 2011 23:00:03 +0100
Subject: max open files
Message-ID: <4D488263.7070305@stone-it.com>

Hi All,

just witnessed a website crumbling due to Varnish hitting the 
max_open_files limit (ulimit -n). After that I had the following 
hypothesis I want to check with you.

When Varnish hits the _max_open_file limit it starts to put servers in 
unhealthy state due to it not being able to open network sockets since 
network sockets are also open files.

Is that a correct assumption?


regards,

Erwin


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110201/46b9cac7/attachment.html>

From moseleymark at gmail.com  Wed Feb  2 01:28:25 2011
From: moseleymark at gmail.com (Mark Moseley)
Date: Tue, 1 Feb 2011 17:28:25 -0800
Subject: Please test 2.1.5
In-Reply-To: <20110126070847.GA2097@freud>
References: <20110125161505.GC21248@freud> <20110126014233.GA27828@freud>
	<20110126070847.GA2097@freud>
Message-ID: <AANLkTimjWBruBdmW9inmsiyqnGvev6-791Rc4hmnMv+Q@mail.gmail.com>

On Tue, Jan 25, 2011 at 11:08 PM, Kristian Lyngstol
<kristian at varnish-software.com> wrote:
> On Wed, Jan 26, 2011 at 02:42:33AM +0100, Kristian Lyngstol wrote:
>> Oh, and still no-go on 32-bit rpms, I think I will have to re-think my
>> strategy tomorrow... But nobody here uses 32-bit anyway, right? ;)
>
> There we go. Hooray for just looping over rpmbuild for 9 hours until a few
> i386 rpms pop out in the other end. (all due to timing issues with the
> regression tests on a virtualized i386 box).
>
> So now all the packages are available.
>
> - Kristian

Been running your 2.1.5 debs for a handful of days now. Seems to be
running just fine. I've had a couple of errors like below (when
someone was downloading a 1.5 gig zip file, another with an 890 meg
flv -- backend is 2 13gb SSD partitions) but they're nothing new. I
got them in 2.1.4 as well. Presumably just ran out of space.


varnishd[30949]: Child (28124) died signal=6
varnishd[30949]: Child (28124) Panic message: Assert error in
STV_alloc(), stevedore.c line 192:
Condition((st): != NULL) not true.
thread: = (cache-worker)
ident: = Linux,2.6.36.2,x86_64,-sfile,-hcritbit,epoll
Backtrace:
0x4267c3: /usr/sbin/varnishd [0x4267c3]
0x43cb05: /usr/sbin/varnishd(STV_alloc+0x125) [0x43cb05]
0x41dce6: /usr/sbin/varnishd(FetchBody+0x616) [0x41dce6]
0x414445: /usr/sbin/varnishd [0x414445]
0x415a5d: /usr/sbin/varnishd(CNT_Session+0x35d) [0x415a5d]
0x428c63: /usr/sbin/varnishd [0x428c63]
0x427f5e: /usr/sbin/varnishd [0x427f5e]
0x32160920fc7: /lib/libpthread.so.0 [0x32160920fc7]
0x321601fb64d: /lib/libc.so.6(clone+0x6d) [0x321601fb64d]
sp: = 0x31bbca74008 {
fd: = 12, id = 12, xid = 764793922,
client: = 192.168.151.40 40613,
step: = STP_FETCH,
handling: = pass,
err_code: = 200, err_reason = (null),
restarts: = 0, esis = 0
ws: = 0x31bbca74080 {
id: = "sess",
{s,f,r,e}: = {0x31bbca74cd8,+592,(nil),+65536},
},:
http[req]: = {
ws: = 0x31bbca74080[sess]



From david at firechaser.com  Wed Feb  2 08:55:26 2011
From: david at firechaser.com (David Murphy)
Date: Wed, 2 Feb 2011 08:55:26 +0000
Subject: Caching only images, no cookies
Message-ID: <AANLkTi=rSRGteygdk2XFx+f-7h3EMhJ4366BRfKctpR5@mail.gmail.com>

Hello

I would like to use Varnish to only cache images and to strictly never
cache any cookies. Previous attempts at writing the .vcl file led to
situations where if an image was missing from the server, the
underlying CMS would try to generate a new images (using Imagecache)
and this led to session swapping because cookies were cached and the
passed onto other users.

So, I'm looking to have a very simply, reliable script that only cares
about images and cannot in any circumstances cache cookies.

Would something like this do the trick?

sub vcl_recv {
 if (req.url ~ "\.(png|gif|jpg)$") {
    lookup;
 }
}

# strip the cookie before insertion
sub vcl_fetch {
 if (req.url ~ "\.(png|gif|jpg)$") {
   unset beresp.http.set-cookie;
}

Thanks, David



From perbu at varnish-software.com  Wed Feb  2 09:02:37 2011
From: perbu at varnish-software.com (Per Buer)
Date: Wed, 2 Feb 2011 10:02:37 +0100
Subject: Caching only images, no cookies
In-Reply-To: <AANLkTi=rSRGteygdk2XFx+f-7h3EMhJ4366BRfKctpR5@mail.gmail.com>
References: <AANLkTi=rSRGteygdk2XFx+f-7h3EMhJ4366BRfKctpR5@mail.gmail.com>
Message-ID: <AANLkTincsN34O=6Q4KmDRTf9tV=atfU7kjYy2-oe+rvn@mail.gmail.com>

Hi David,


On Wed, Feb 2, 2011 at 9:55 AM, David Murphy <david at firechaser.com> wrote:
> Hello
>
> I would like to use Varnish to only cache images and to strictly never
> cache any cookies. Previous attempts at writing the .vcl file led to
> situations where if an image was missing from the server, the
> underlying CMS would try to generate a new images (using Imagecache)
> and this led to session swapping because cookies were cached and the
> passed onto other users.

Right. Sound bad.


> So, I'm looking to have a very simply, reliable script that only cares
> about images and cannot in any circumstances cache cookies.
>
> Would something like this do the trick?
>
> sub vcl_recv {
> ?if (req.url ~ "\.(png|gif|jpg)$") {
> ? ?lookup;
> ?}
> }

No. Just drop the relevant cookie in vcl_recv. If you strip away the
cookies the image will be cached by the built in logic. Easy and safe.

-- 
Per Buer,?Varnish Software
Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer
Varnish makes websites fly!
Want to learn more about Varnish? http://www.varnish-software.com/whitepapers



From david at firechaser.com  Wed Feb  2 09:14:26 2011
From: david at firechaser.com (David Murphy)
Date: Wed, 2 Feb 2011 09:14:26 +0000
Subject: Caching only images, no cookies
In-Reply-To: <AANLkTincsN34O=6Q4KmDRTf9tV=atfU7kjYy2-oe+rvn@mail.gmail.com>
References: <AANLkTi=rSRGteygdk2XFx+f-7h3EMhJ4366BRfKctpR5@mail.gmail.com>
	<AANLkTincsN34O=6Q4KmDRTf9tV=atfU7kjYy2-oe+rvn@mail.gmail.com>
Message-ID: <AANLkTikkBt63SQYB3aawrajrgn0R-Kud8Ea34mtBUzCg@mail.gmail.com>

Morning Per

So the default behaviour of Varnish (only extended if I start writing
my own .vcl) would make a smart choice about what should be cached?
That sounds good. So provided that I strip the cookies for the image
file types, then Varnish will spot that these request do not have any
cookies and will safely cache them, ignoring all other requests (since
I've not stripped their cookies).

Something as simple as

sub vcl_recv {
   if (req.url ~ "\.(png|gif|jpg)$") {
        remove req.http.Cookie;
    }
}

...and nothing else in .vcl file?

Thanks, David


On Wed, Feb 2, 2011 at 9:02 AM, Per Buer <perbu at varnish-software.com> wrote:
> Hi David,
>
>
> On Wed, Feb 2, 2011 at 9:55 AM, David Murphy <david at firechaser.com> wrote:
>> Hello
>>
>> I would like to use Varnish to only cache images and to strictly never
>> cache any cookies. Previous attempts at writing the .vcl file led to
>> situations where if an image was missing from the server, the
>> underlying CMS would try to generate a new images (using Imagecache)
>> and this led to session swapping because cookies were cached and the
>> passed onto other users.
>
> Right. Sound bad.
>
>
>> So, I'm looking to have a very simply, reliable script that only cares
>> about images and cannot in any circumstances cache cookies.
>>
>> Would something like this do the trick?
>>
>> sub vcl_recv {
>> ?if (req.url ~ "\.(png|gif|jpg)$") {
>> ? ?lookup;
>> ?}
>> }
>
> No. Just drop the relevant cookie in vcl_recv. If you strip away the
> cookies the image will be cached by the built in logic. Easy and safe.
>
> --
> Per Buer,?Varnish Software
> Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer
> Varnish makes websites fly!
> Want to learn more about Varnish? http://www.varnish-software.com/whitepapers
>



From perbu at varnish-software.com  Wed Feb  2 09:49:02 2011
From: perbu at varnish-software.com (Per Buer)
Date: Wed, 2 Feb 2011 10:49:02 +0100
Subject: Caching only images, no cookies
In-Reply-To: <AANLkTikkBt63SQYB3aawrajrgn0R-Kud8Ea34mtBUzCg@mail.gmail.com>
References: <AANLkTi=rSRGteygdk2XFx+f-7h3EMhJ4366BRfKctpR5@mail.gmail.com>
	<AANLkTincsN34O=6Q4KmDRTf9tV=atfU7kjYy2-oe+rvn@mail.gmail.com>
	<AANLkTikkBt63SQYB3aawrajrgn0R-Kud8Ea34mtBUzCg@mail.gmail.com>
Message-ID: <AANLkTikOf0kYAymJeZCpYQBdRwBTkNuynAHUCCXGwgki@mail.gmail.com>

David,

On Wed, Feb 2, 2011 at 10:14 AM, David Murphy <david at firechaser.com> wrote:
> Morning Per
>
> So the default behaviour of Varnish (only extended if I start writing
> my own .vcl) would make a smart choice about what should be cached?
> That sounds good. So provided that I strip the cookies for the image
> file types, then Varnish will spot that these request do not have any
> cookies and will safely cache them, ignoring all other requests (since
> I've not stripped their cookies).

Right.

> Something as simple as
>
> sub vcl_recv {
> ? if (req.url ~ "\.(png|gif|jpg)$") {
> ? ? ? ?remove req.http.Cookie;
> ? ?}
> }

Right. Provided your backend produces sane cache-control headers. If
there is no information on how long to cache it Varnish will cache for
a default period of 120 seconds. You can override the TTL -
http://www.varnish-cache.org/docs/trunk/tutorial/increasing_your_hitrate.html#overriding-the-time-to-live-ttl

> ...and nothing else in .vcl file?

No. That should be sufficient.


-- 
Per Buer,?Varnish Software
Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer
Varnish makes websites fly!
Want to learn more about Varnish? http://www.varnish-software.com/whitepapers



From tfheen at varnish-software.com  Wed Feb  2 10:58:49 2011
From: tfheen at varnish-software.com (Tollef Fog Heen)
Date: Wed, 02 Feb 2011 11:58:49 +0100
Subject: Rewriting URL's or Content inside a req object using regsub or
	regsuball
In-Reply-To: <AANLkTikwi+qSZiVqJ=vg6DiBNiR1+A8L5qtMeRcsvChj@mail.gmail.com>
	(Harry Yeh's message of "Thu, 6 Jan 2011 21:28:24 -0800")
References: <AANLkTimAOxdrwxr7Y+q5OE1GOVZT9NRinE6N3w=VEXPH@mail.gmail.com>
	<AANLkTikwi+qSZiVqJ=vg6DiBNiR1+A8L5qtMeRcsvChj@mail.gmail.com>
Message-ID: <87ipx24s9i.fsf@qurzaw.varnish-software.com>

]] Harry Yeh 

Hi,

sorry about the late response.

| I am currently having some success with the Reverse Proxying features of
| Varnish, and the only thing left that I need to be able to do is essentially
| rewrite some of the URL's in the body.

Varnish doesn't rewrite bodies, so there's no way to do this today.  If
you use apache on your backends, you can use mod_proxy_html.

Regards,
-- 
Tollef Fog Heen
Varnish Software
t: +47 21 98 92 64



From kristian at varnish-software.com  Wed Feb  2 11:11:02 2011
From: kristian at varnish-software.com (Kristian Lyngstol)
Date: Wed, 2 Feb 2011 12:11:02 +0100
Subject: Please test 2.1.5
In-Reply-To: <AANLkTimjWBruBdmW9inmsiyqnGvev6-791Rc4hmnMv+Q@mail.gmail.com>
References: <20110125161505.GC21248@freud> <20110126014233.GA27828@freud>
	<20110126070847.GA2097@freud>
	<AANLkTimjWBruBdmW9inmsiyqnGvev6-791Rc4hmnMv+Q@mail.gmail.com>
Message-ID: <20110202111102.GA2251@freud>

On Tue, Feb 01, 2011 at 05:28:25PM -0800, Mark Moseley wrote:
> Been running your 2.1.5 debs for a handful of days now. Seems to be
> running just fine. I've had a couple of errors like below (when
> someone was downloading a 1.5 gig zip file, another with an 890 meg
> flv -- backend is 2 13gb SSD partitions) but they're nothing new. I
> got them in 2.1.4 as well. Presumably just ran out of space.
>
> varnishd[30949]: Child (28124) Panic message: Assert error in
> STV_alloc(), stevedore.c line 192:
> Condition((st): != NULL) not true.

Thanks for the feedback!

Are you by any chance using multiple -s options? This assert will happen
after Varnish gives up finding a stevedore for storage after trying 50
times (IIRC), and typically happens due to either: A. Having lots of small
files then fetching a big one - 50 small files removed still wont fit the
big file. or B. Multiple -s options, one which is tiny and Varnish - by
random chance - picks the small one 50 times in a row (since this is done
concurrently, you can have many threads using the algorithm at the same
time, so while it is normally round-robin selection, the "other" threads
might get the big one.... It's not common, but we've seen it happen.).

Either way, this just goes to show that we need to deal better with files
of vastly different sizes. In your case, I'd recommend trying to pipe() the
big files.

- Kristian



From david at firechaser.com  Wed Feb  2 16:40:48 2011
From: david at firechaser.com (David Murphy)
Date: Wed, 2 Feb 2011 16:40:48 +0000
Subject: Caching only images, no cookies
In-Reply-To: <AANLkTikOf0kYAymJeZCpYQBdRwBTkNuynAHUCCXGwgki@mail.gmail.com>
References: <AANLkTi=rSRGteygdk2XFx+f-7h3EMhJ4366BRfKctpR5@mail.gmail.com>
	<AANLkTincsN34O=6Q4KmDRTf9tV=atfU7kjYy2-oe+rvn@mail.gmail.com>
	<AANLkTikkBt63SQYB3aawrajrgn0R-Kud8Ea34mtBUzCg@mail.gmail.com>
	<AANLkTikOf0kYAymJeZCpYQBdRwBTkNuynAHUCCXGwgki@mail.gmail.com>
Message-ID: <AANLkTi=Lzi=NRiV8L7eReCiQG1-5dBF7X+pFxowmt=VX@mail.gmail.com>

Thanks for this Per.

Following your adbvise I have got the following in my .vcl ( I use
sudo /usr/local/sbin/varnishd -a :80 -f
/usr/local/etc/varnish/default.vcl -s malloc,1G to start Varnish)

//START
backend default {
.host = "127.0.0.1";
.port = "8080"; //Apache listening
.connect_timeout = 600s;
.first_byte_timeout = 600s;
.between_bytes_timeout = 600s;
}

sub vcl_recv {
  if (req.url ~ "\.(png|gif|jpg)$") {
       remove req.http.Cookie;
   }
}
sub vcl_deliver {
        if (obj.hits > 0) {
                set resp.http.X-Cache = "HIT";
        } else {
                set resp.http.X-Cache = "MISS";
        }
}

sub vcl_error {
  set obj.http.Content-Type = "text/html; charset=utf-8";
  synthetic {"
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
<head></head><body>error message goes here</body></html>
   "};
}
//END

When I visit a page (of a Drupal powered site) that is delivered via
Varnish, the XHTML is not cached (good) and when I view an image in
separate browser tab and check firebug I get

X-Varnish	1355426855 1355414030
Age	1135
Via	1.1 varnish
Connection	keep-alive
X-Cache	HIT

Which looks good too :).

 I can see cookie data in the request headers, but no cookie data in
the response headers. Example page:
http://web4.v1.southbank.signoff.info/visitor-info

So given the above, can I be confident that the only thing Varnish
will cache is anything that does not have a cookie ( i.e. what is
caches sensibly by default, plus the images inclusion I've added)?

I'm going to start playing with the CLI now to understand a bit more
about what's actually in the cache. Is there an easy way to see if the
cache contains anything with a cookie? That would be a useful sanity
check for me to run from time to time as I develop the .vcl.

Many thanks - am really pleased with how easily and quickly I was able
to get this running.

Best, David




On Wed, Feb 2, 2011 at 9:49 AM, Per Buer <perbu at varnish-software.com> wrote:
> David,
>
> On Wed, Feb 2, 2011 at 10:14 AM, David Murphy <david at firechaser.com> wrote:
>> Morning Per
>>
>> So the default behaviour of Varnish (only extended if I start writing
>> my own .vcl) would make a smart choice about what should be cached?
>> That sounds good. So provided that I strip the cookies for the image
>> file types, then Varnish will spot that these request do not have any
>> cookies and will safely cache them, ignoring all other requests (since
>> I've not stripped their cookies).
>
> Right.
>
>> Something as simple as
>>
>> sub vcl_recv {
>> ? if (req.url ~ "\.(png|gif|jpg)$") {
>> ? ? ? ?remove req.http.Cookie;
>> ? ?}
>> }
>
> Right. Provided your backend produces sane cache-control headers. If
> there is no information on how long to cache it Varnish will cache for
> a default period of 120 seconds. You can override the TTL -
> http://www.varnish-cache.org/docs/trunk/tutorial/increasing_your_hitrate.html#overriding-the-time-to-live-ttl
>
>> ...and nothing else in .vcl file?
>
> No. That should be sufficient.
>
>
> --
> Per Buer,?Varnish Software
> Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer
> Varnish makes websites fly!
> Want to learn more about Varnish? http://www.varnish-software.com/whitepapers
>



From moseleymark at gmail.com  Wed Feb  2 18:13:35 2011
From: moseleymark at gmail.com (Mark Moseley)
Date: Wed, 2 Feb 2011 10:13:35 -0800
Subject: Please test 2.1.5
In-Reply-To: <20110202111102.GA2251@freud>
References: <20110125161505.GC21248@freud> <20110126014233.GA27828@freud>
	<20110126070847.GA2097@freud>
	<AANLkTimjWBruBdmW9inmsiyqnGvev6-791Rc4hmnMv+Q@mail.gmail.com>
	<20110202111102.GA2251@freud>
Message-ID: <AANLkTimx+D2iBfrgeVyJiP-XiWPOZeqLvBJjkC=wdLvM@mail.gmail.com>

On Wed, Feb 2, 2011 at 3:11 AM, Kristian Lyngstol
<kristian at varnish-software.com> wrote:
> On Tue, Feb 01, 2011 at 05:28:25PM -0800, Mark Moseley wrote:
>> Been running your 2.1.5 debs for a handful of days now. Seems to be
>> running just fine. I've had a couple of errors like below (when
>> someone was downloading a 1.5 gig zip file, another with an 890 meg
>> flv -- backend is 2 13gb SSD partitions) but they're nothing new. I
>> got them in 2.1.4 as well. Presumably just ran out of space.
>>
>> varnishd[30949]: Child (28124) Panic message: Assert error in
>> STV_alloc(), stevedore.c line 192:
>> Condition((st): != NULL) not true.
>
> Thanks for the feedback!
>
> Are you by any chance using multiple -s options? This assert will happen
> after Varnish gives up finding a stevedore for storage after trying 50
> times (IIRC), and typically happens due to either: A. Having lots of small
> files then fetching a big one - 50 small files removed still wont fit the
> big file. or B. Multiple -s options, one which is tiny and Varnish - by
> random chance - picks the small one 50 times in a row (since this is done
> concurrently, you can have many threads using the algorithm at the same
> time, so while it is normally round-robin selection, the "other" threads
> might get the big one.... It's not common, but we've seen it happen.).

I see it both in single -s instances and instances with 2 -s
filestores. Since it's almost always gigantic files (in the hundreds
of megs to gigs range), I'd imagine "A" is the more likely choice. I
only just started using 2 -s args to test out the new OCZ Revodrives
(which are insane) we got. The single instance filestores are always
>15G, and this case with the 2 filestores was 2 13G filestores, so
pretty decent-sized.

> Either way, this just goes to show that we need to deal better with files
> of vastly different sizes. In your case, I'd recommend trying to pipe() the
> big files.

Yup, we do pipe them. Based on a thread I'd started the other day
(10/28-ish) about dealing with huge files, it seems like even going to
pipe won't save the file from being put into the backend and forcing
out other files. Originally, we were doing this in vcl_fetch:

if ( beresp.http.Content-Length ~
"[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]" ) {
        return ( pass );
}

Based on Tollef's advice, we switched that to:

(in vcl_fetch)

        if ( beresp.http.Content-Length ~ "[0-9]{9,}" ) {
                set req.http.X-EN-PP = "1";
                restart;
        }

and then in vcl_recv does a check for "req.restarts > 0" and calls
pipe. This works nicely but huge files that end up getting piped still
seem to get stored (albeit very temporarily) on that first
pre-restarted, request.



From Klapproth.M at sportresult.com  Thu Feb  3 10:30:05 2011
From: Klapproth.M at sportresult.com (Klapproth, Martin)
Date: Thu, 3 Feb 2011 11:30:05 +0100
Subject: max open files
In-Reply-To: <4D488263.7070305@stone-it.com>
References: <4D488263.7070305@stone-it.com>
Message-ID: <4BC50911656FBB49858C6ADB1B8D223F038D96FC@mail.wigemic.lpz>

Hi,

Yes thats right. To increase the maximum number of file descriptors
add/edit the line in /etc/security/limits.conf:

*              -       nofile         8192

I've set the amount to 8192 (8*1024). I'm monitoring the file
descriptors of my varnish. Max was 2500.

Regards,

Martin

________________________________________
From: varnish-misc-bounces at varnish-cache.org
[mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Erwin
Stam|Stone-IT
Sent: Dienstag, 1. Februar 2011 23:00
To: varnish-misc at varnish-cache.org
Subject: max open files

Hi All,

just witnessed a website crumbling due to Varnish hitting the
max_open_files limit (ulimit -n). After that I had the following
hypothesis I want to check with you.

When Varnish hits the _max_open_file limit it starts to put servers in
unhealthy state due to it not being able to open network sockets since
network sockets are also open files.

Is that a correct assumption?


regards,

Erwin




From armdan20 at gmail.com  Thu Feb  3 14:12:44 2011
From: armdan20 at gmail.com (andan andan)
Date: Thu, 3 Feb 2011 15:12:44 +0100
Subject: Too much TIME_WAIT connections on backends after 2.1.5 upgrade
Message-ID: <AANLkTikbYsNzsUR95tWz3Qqa5z7Bvxc20SUs6xqenfvn@mail.gmail.com>

Hi.

This morning, we have upgraded to 2.1.5 (from 2.1.4) and now we have
too much TIME_WAIT connections on backends.

Before the upgrade on one backend we can saw over 90 TIME_WAIT
connections and just after the
upgrade the TIME_WAIT connections goes to more than 400 and several
hours after, the number is stable.

This is not an issue perse, we can reduce the TIME_WAIT connections
tunning several kernel parameters, but I'm curious about this new
behaviour.

Thanks.
Kind Regards.



From johnson at nmr.mgh.harvard.edu  Thu Feb  3 20:42:25 2011
From: johnson at nmr.mgh.harvard.edu (Chris Johnson)
Date: Thu, 3 Feb 2011 15:42:25 -0500 (EST)
Subject: Changing the fetch for virtual hosts.
In-Reply-To: <AANLkTikbYsNzsUR95tWz3Qqa5z7Bvxc20SUs6xqenfvn@mail.gmail.com>
References: <AANLkTikbYsNzsUR95tWz3Qqa5z7Bvxc20SUs6xqenfvn@mail.gmail.com>
Message-ID: <alpine.LRH.2.00.1102031529190.10090@maequeses.nmr.mgh.harvard.edu>

      Hi,

      Very new to varnish.  Still re-re-re-reading the  documentaion.
Read is one thing undestanding is another.

      I have one Apache server serving multiple virtual hosts.  I'm
trying to phase out one host and point people to another virtual host
on the same server serviced by the same varnish.  For various reasons
I can't just cut one off.

      IF (big IF) I'm reading things right here, it looks like I can
rewrite the host name in vcl_recv after the request is received but
before the fetch.  If I do this in vcl_recv;

if (req.http.host ~ "^old.host.edu") {
set req.http.host = "new.host.edu";
}

will the request go to the new host?  The idea is to stop varnish from
basically cache everything twice since both new.host and old.host
point to the same set of pages.  I don't want varnish doing double duty
here.  All varish knows is that the URLs are different and if I
understand correctly will treat them as different sets of pages
caching both.

      Thnak you.

-------------------------------------------------------------------------------
Chris Johnson               |Internet: johnson at nmr.mgh.harvard.edu
Systems Administrator       |Web:      http://www.nmr.mgh.harvard.edu/~johnson
NMR Center                  |Voice:    617.726.0949
Mass. General Hospital      |FAX:      617.726.7422
149 (2301) 13th Street      |An object will fall so as to do the most damage.
Charlestown, MA., 02129 USA | The law of selective gravity.
-------------------------------------------------------------------------------


The information in this e-mail is intended only for the person to whom it is
addressed. If you believe this e-mail was sent to you in error and the e-mail
contains patient information, please contact the Partners Compliance HelpLine at
http://www.partners.org/complianceline . If the e-mail was sent to you in error
but does not contain patient information, please contact the sender and properly
dispose of the e-mail.




From armdan20 at gmail.com  Wed Feb  2 12:22:50 2011
From: armdan20 at gmail.com (andan andan)
Date: Wed, 2 Feb 2011 13:22:50 +0100
Subject: Too much TIME_WAIT connections after 2.1.5 upgrade
Message-ID: <AANLkTinBuBo00b1fEY1yjJKTEC=mO+CEEBgY_C31bfRz@mail.gmail.com>

Hi.

This morning, we have upgraded to 2.1.5 (from 2.1.4) and now we have
too much TIME_WAIT connections on backends.

Before the upgrade on one backend we can saw over 90 TIME_WAIT
connections and just after the
upgrade the TIME_WAIT connections goes to more than 400 and several
hours after, the number is stable.

This is not an issue perse, we can reduce the TIME_WAIT connections
tunning several kernel parameters, but I'm curious about this new
behaviour.

Thanks.
Kind Regards.



From perbu at varnish-software.com  Fri Feb  4 09:04:01 2011
From: perbu at varnish-software.com (Per Buer)
Date: Fri, 4 Feb 2011 10:04:01 +0100
Subject: Changing the fetch for virtual hosts.
In-Reply-To: <alpine.LRH.2.00.1102031529190.10090@maequeses.nmr.mgh.harvard.edu>
References: <AANLkTikbYsNzsUR95tWz3Qqa5z7Bvxc20SUs6xqenfvn@mail.gmail.com>
	<alpine.LRH.2.00.1102031529190.10090@maequeses.nmr.mgh.harvard.edu>
Message-ID: <AANLkTi=+dWrYH7e9z37yeCw7kp+qk04Vt9CxhOxERhoQ@mail.gmail.com>

Hi Chris,

On Thu, Feb 3, 2011 at 9:42 PM, Chris Johnson
<johnson at nmr.mgh.harvard.edu> wrote:
> ? ? IF (big IF) I'm reading things right here, it looks like I can
> rewrite the host name in vcl_recv after the request is received but
> before the fetch. ?If I do this in vcl_recv;
>
> if (req.http.host ~ "^old.host.edu") {
> set req.http.host = "new.host.edu";
> }
>
> will the request go to the new host?

Not unless you explicitly set the backend later. The only thing you do
here is to set a new Host: header, which makes sense, btw.

>?The idea is to stop varnish from basically cache everything twice since
> both new.host and old.host point to the same set of pages.
> I don't want varnish doing double duty
> here. ?All varish knows is that the URLs are different and if I
> understand correctly will treat them as different sets of pages
> caching both.

You assume correctly. The default is that varnish uses the Host and
the URL as key, you can check out the vcl_hash subroutine in the
default VCL.



-- 
Per Buer,?Varnish Software
Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer
Varnish makes websites fly!
Want to learn more about Varnish? http://www.varnish-software.com/whitepapers



From Ray.Bacion at oa.mo.gov  Thu Feb  3 19:00:56 2011
From: Ray.Bacion at oa.mo.gov (Bacion, Ray)
Date: Thu, 3 Feb 2011 13:00:56 -0600
Subject: Varnish not completely loading page from IE 7 browser
Message-ID: <20C7CC58ED808E429A2B575E3CB585EB33ED0172@EXCHVS3A.mx.state.mo.us>

Hello, we have installed Varnish on one of our RedHat 5 Linux machines.   I have pasted the configuration below.    I did remove the real hostnames and IP addresses from the configuration.

The configuration works fine with Mozilla for sites I have defined, but on IE 7 the "hostname1" site does not completely load.   I see the favicon.ico and the title of the page, but the site continues to load without completing or sometimes completing after a long time (minute plus) to load.   The second site "shared_73_165" loads just fine.

Please note a week ago I had the same sites defined, but IE had problems loading the "shared_73_165" site but loaded the "hostname1" site just fine.   I assume there is some slight difference in the configuration between this week and last to make this change occur, but I don't know what it would be.    Mozilla has always loaded both sites fine.


I am having trouble identifying the root cause of the problem.   Can anyone help me with trying to determine why IE 7 is taking so long to complete the loading of the page if it completes at all?

I can also add that a when I look at the varnishncsa output I see that with IE 7 that not all of the elements on the HTML page are being requested, but through Mozilla I do see all the requests.   I can also tell you both sites work fine if I bypass Varnish.   It also does appear that IE 8 has the same issue although I have not tested with it as much as I have IE 7.

Any help that someone can provide would be appreciated?



backend hostname1
{
  .host="xxx.xxx.xxx.xxx";.port="80";.max_connections=200;.connect_timeout=10s;.first_byte_timeout=10s;
  .between_bytes_timeout=10s;
}
backend shared_73_165
{
  .host="xxx.xxx.xxx.xxx";.port="80";.max_connections=200;.connect_timeout=10s;.first_byte_timeout=10s;
  .between_bytes_timeout=10s;
}

sub vcl_recv
{
  # Set Backend Server
  if (req.http.host ~ "^hostname1.com"     || req.http.host ~ "^test.hostname1.com")              { set req.backend = hostname1;  }
  else                                                                                                                                                                       { set req.backend = shared_73_165; }

  # Set HTTP Header NS-Client-IP so Apache can log client's IP address.
  set req.http.NS-Client-IP = client.ip;

  # If HTTP Header Cache-Contro="no-cache" then purge any cache page that may exist for that URL and pass request on.
  if (req.http.Cache-Control ~ "no-cache") { purge_url(req.url); return(pass); }

  # If request is not a valid HTTP Method pipe it to the backend it is not cacheable.
  if (req.request!="GET"    && req.request!="HEAD"    && req.request!="PUT"     && req.request!="POST" &&
      req.request!="TRACE"  && req.request!="OPTIONS" && req.request!="DELETE")
  { return (pipe); }  /*Non-RFC2616*/

  # If request is not a GET or a HEAD then pass it through since its not cacheable.
  if (req.request!="GET" || req.request!="HEAD") { return (pass); }

  # Unset cookies on Drupal specific files that should be cached.
  if (req.url ~ "^/(files|misc|sites|themes|modules)/" || req.url ~ "\.(txt|ico)$") { unset req.http.Cookie; }

  # If request requires Authentication or has a Cookie then it is not cacheable.
  if (req.http.Authorization || req.http.Cookie) { return (pass); }

  # Normalize and compress content if Encoding starts with gzip or defalte but do not compress binary files.
  # @see:  http://varnish.projects.linpro.no/wiki/FAQ/Compression
  if (req.http.Accept-Encoding) {
    if    (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") { remove req.http.Accept-Encoding; }
    elsif (req.http.Accept-Encoding ~ "gzip")   {set req.http.Accept-Encoding = "gzip"; }
    elsif (req.http.Accept-Encoding ~ "deflate"){set req.http.Accept-Encoding = "deflate"; }
    else                                        {unset req.http.Accept-Encoding; }
  }

  # If requested page is currently being processed on another thread to a backend server,  server cached page if its not older than 30s.
  set req.grace = 30s;

  # Fall back to the default vcl_recv which will call the return(lookup);
}

sub vcl_pipe
{
  set req.http.connection = "close";
  # Fall back to the default vcl_pipe which will call return(pipe);
}

sub vcl_fetch
{
  # If the response from the backend is not a 200, 203, 300, 301, 302, 404 or 410.
  if (!beresp.cacheable) { return (pass); }
}
--------------------------------------------------------
Raymond Bacion

ITSD Operating System Group
Phone: (573) 522-1468
Fax :    (573) 522-1384
Email:  Ray.Bacion at oa.mo.gov

-------------------------------------------------------



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110203/4062437d/attachment.html>

From johnson at nmr.mgh.harvard.edu  Fri Feb  4 14:24:06 2011
From: johnson at nmr.mgh.harvard.edu (Chris Johnson)
Date: Fri, 4 Feb 2011 09:24:06 -0500 (EST)
Subject: Changing the fetch for virtual hosts.
In-Reply-To: <AANLkTi=+dWrYH7e9z37yeCw7kp+qk04Vt9CxhOxERhoQ@mail.gmail.com>
References: <AANLkTikbYsNzsUR95tWz3Qqa5z7Bvxc20SUs6xqenfvn@mail.gmail.com>
	<alpine.LRH.2.00.1102031529190.10090@maequeses.nmr.mgh.harvard.edu>
	<AANLkTi=+dWrYH7e9z37yeCw7kp+qk04Vt9CxhOxERhoQ@mail.gmail.com>
Message-ID: <alpine.LRH.2.00.1102040907030.3092@maequeses.nmr.mgh.harvard.edu>

> Hi Chris,
>
>> ? ? IF (big IF) I'm reading things right here, it looks like I can
>> rewrite the host name in vcl_recv after the request is received but
>> before the fetch. ?If I do this in vcl_recv;
>>
>> if (req.http.host ~ "^old.host.edu") {
>> set req.http.host = "new.host.edu";
>> }
>>
>> will the request go to the new host?
>
> Not unless you explicitly set the backend later. The only thing you do
> here is to set a new Host: header, which makes sense, btw.

      Not clear on what you mean by that first sentence.  Can you
elaborate?

>
>> ?The idea is to stop varnish from basically cache everything twice since
>> both new.host and old.host point to the same set of pages.
>> I don't want varnish doing double duty
>> here. ?All varish knows is that the URLs are different and if I
>> understand correctly will treat them as different sets of pages
>> caching both.
>
> You assume correctly. The default is that varnish uses the Host and
> the URL as key, you can check out the vcl_hash subroutine in the
> default VCL.
>
>

      I was looking through the example snippets to see if there was
anything to prevent redundant caching.  I've found two exampes that
look interesting.  They are

      o  Redirecting the browser to another hostname

      o  Using static regular expressions to munge an incoming HTTP
         request to become something very different. Very useful if
         you want to apply the same varnish config to all sites on
         a server hosting many, many small sites

      The last sounds closer to what I'm trying to do to prevent
redundant caching.

      Has anyone had/solved this problem?  If these examples exist in
the wiki here one would take a guess that it or something very similar
has.  I'd very much like to see an example.  What I'd like to do is
keep what varnish calls the URL and change the host name where the
fetch actually happens. so everything gets fetch from the one virtual
hosst.

-------------------------------------------------------------------------------
Chris Johnson               |Internet: johnson at nmr.mgh.harvard.edu
Systems Administrator       |Web:      http://www.nmr.mgh.harvard.edu/~johnson
NMR Center                  |Voice:    617.726.0949
Mass. General Hospital      |FAX:      617.726.7422
149 (2301) 13th Street      |Survival, all by it self, isn't worth it.
Charlestown, MA., 02129 USA |                 Me
-------------------------------------------------------------------------------


The information in this e-mail is intended only for the person to whom it is
addressed. If you believe this e-mail was sent to you in error and the e-mail
contains patient information, please contact the Partners Compliance HelpLine at
http://www.partners.org/complianceline . If the e-mail was sent to you in error
but does not contain patient information, please contact the sender and properly
dispose of the e-mail.

From perbu at varnish-software.com  Fri Feb  4 15:04:26 2011
From: perbu at varnish-software.com (Per Buer)
Date: Fri, 4 Feb 2011 16:04:26 +0100
Subject: Changing the fetch for virtual hosts.
In-Reply-To: <alpine.LRH.2.00.1102040907030.3092@maequeses.nmr.mgh.harvard.edu>
References: <AANLkTikbYsNzsUR95tWz3Qqa5z7Bvxc20SUs6xqenfvn@mail.gmail.com>
	<alpine.LRH.2.00.1102031529190.10090@maequeses.nmr.mgh.harvard.edu>
	<AANLkTi=+dWrYH7e9z37yeCw7kp+qk04Vt9CxhOxERhoQ@mail.gmail.com>
	<alpine.LRH.2.00.1102040907030.3092@maequeses.nmr.mgh.harvard.edu>
Message-ID: <AANLkTinbZ8bv5gAe4w3J3VoV=UVfxsNw43LxfugxENuF@mail.gmail.com>

On Fri, Feb 4, 2011 at 3:24 PM, Chris Johnson
<johnson at nmr.mgh.harvard.edu> wrote:
>>
>> Not unless you explicitly set the backend later. The only thing you do
>> here is to set a new Host: header, which makes sense, btw.
>
> ? ? Not clear on what you mean by that first sentence. ?Can you
> elaborate?

Sure. You need to set what backend your using something like this at some point:

       if (req.http.host ~ "^www.example.com") {
         set req.backend = foo;
       }

If you already have set req.http.host before reaching this things will be fine.

> ? ? I was looking through the example snippets to see if there was
> anything to prevent redundant caching. ?I've found two exampes that
> look interesting. ?They are
>
> ? ? o ?Redirecting the browser to another hostname
>
> ? ? o ?Using static regular expressions to munge an incoming HTTP
> ? ? ? ?request to become something very different. Very useful if
> ? ? ? ?you want to apply the same varnish config to all sites on
> ? ? ? ?a server hosting many, many small sites
>
> ? ? The last sounds closer to what I'm trying to do to prevent
> redundant caching.
>
> ? ? Has anyone had/solved this problem?

I think your approach is fine.


-- 
Per Buer,?Varnish Software
Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer
Varnish makes websites fly!
Want to learn more about Varnish? http://www.varnish-software.com/whitepapers



From johnson at nmr.mgh.harvard.edu  Fri Feb  4 16:17:30 2011
From: johnson at nmr.mgh.harvard.edu (Chris Johnson)
Date: Fri, 4 Feb 2011 11:17:30 -0500 (EST)
Subject: Changing the fetch for virtual hosts.
In-Reply-To: <AANLkTinbZ8bv5gAe4w3J3VoV=UVfxsNw43LxfugxENuF@mail.gmail.com>
References: <AANLkTikbYsNzsUR95tWz3Qqa5z7Bvxc20SUs6xqenfvn@mail.gmail.com>
	<alpine.LRH.2.00.1102031529190.10090@maequeses.nmr.mgh.harvard.edu>
	<AANLkTi=+dWrYH7e9z37yeCw7kp+qk04Vt9CxhOxERhoQ@mail.gmail.com>
	<alpine.LRH.2.00.1102040907030.3092@maequeses.nmr.mgh.harvard.edu>
	<AANLkTinbZ8bv5gAe4w3J3VoV=UVfxsNw43LxfugxENuF@mail.gmail.com>
Message-ID: <alpine.LRH.2.00.1102041113330.3092@maequeses.nmr.mgh.harvard.edu>

On Fri, 4 Feb 2011, Per Buer wrote:

> On Fri, Feb 4, 2011 at 3:24 PM, Chris Johnson
> <johnson at nmr.mgh.harvard.edu> wrote:
>>>
>>> Not unless you explicitly set the backend later. The only thing you do
>>> here is to set a new Host: header, which makes sense, btw.
>>
>> ? ? Not clear on what you mean by that first sentence. ?Can you
>> elaborate?
>
> Sure. You need to set what backend your using something like this at some point:
>
>       if (req.http.host ~ "^www.example.com") {
>         set req.backend = foo;
>       }

      Which goes in vcl_recv and in his case foo is a backend declaration
function that sets up .host and .port for the virtual host I want, yes?
>
> If you already have set req.http.host before reaching this things will be fine.
>
>> ? ? I was looking through the example snippets to see if there was
>> anything to prevent redundant caching. ?I've found two exampes that
>> look interesting. ?They are
>>
>> ? ? o ?Redirecting the browser to another hostname
>>
>> ? ? o ?Using static regular expressions to munge an incoming HTTP
>> ? ? ? ?request to become something very different. Very useful if
>> ? ? ? ?you want to apply the same varnish config to all sites on
>> ? ? ? ?a server hosting many, many small sites
>>
>> ? ? The last sounds closer to what I'm trying to do to prevent
>> redundant caching.
>>
>> ? ? Has anyone had/solved this problem?
>
> I think your approach is fine.
>
>
> -- 
> Per Buer,?Varnish Software
> Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer
> Varnish makes websites fly!
> Want to learn more about Varnish? http://www.varnish-software.com/whitepapers
>
>
>

--------------------------------------------------------------------------------
Chris Johnson               |Internet: johnson at nmr.mgh.harvard.edu
Systems Administrator       |Web:      http://www.nmr.mgh.harvard.edu/~johnson
NMR Center                  |Voice:    617.726.0949
Mass. General Hospital      |FAX:      617.726.7422
149 (2301) 13th Street      |God stand between you and harm in all the empty
Charlestown, MA., 02129 USA |places where you must walk. old Egyptian blessing
--------------------------------------------------------------------------------


The information in this e-mail is intended only for the person to whom it is
addressed. If you believe this e-mail was sent to you in error and the e-mail
contains patient information, please contact the Partners Compliance HelpLine at
http://www.partners.org/complianceline . If the e-mail was sent to you in error
but does not contain patient information, please contact the sender and properly
dispose of the e-mail.

From scaunter at topscms.com  Fri Feb  4 16:58:29 2011
From: scaunter at topscms.com (Caunter, Stefan)
Date: Fri, 4 Feb 2011 11:58:29 -0500
Subject: Varnish not completely loading page from IE 7 browser
In-Reply-To: <20C7CC58ED808E429A2B575E3CB585EB33ED0172@EXCHVS3A.mx.state.mo.us>
References: <20C7CC58ED808E429A2B575E3CB585EB33ED0172@EXCHVS3A.mx.state.mo.us>
Message-ID: <7F0AA702B8A85A4A967C4C8EBAD6902CDE6E93@TMG-EVS02.torstar.net>

We need to know which version of varnish.

 

Which  exact version of IE7? Does the page call external components?
Anything https? Are the back ends the same? Is there a load balancer?

 

Stefan Caunter 

Operations

Torstar Digital

m: (416) 561-4871

 

 

From: varnish-misc-bounces at varnish-cache.org
[mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Bacion, Ray
Sent: February-03-11 2:01 PM
To: 'varnish-misc at varnish-cache.org'
Cc: OA.Lampadmins
Subject: Varnish not completely loading page from IE 7 browser

 

Hello, we have installed Varnish on one of our RedHat 5 Linux machines.
I have pasted the configuration below.    I did remove the real
hostnames and IP addresses from the configuration.

 

The configuration works fine with Mozilla for sites I have defined, but
on IE 7 the "hostname1" site does not completely load.   I see the
favicon.ico and the title of the page, but the site continues to load
without completing or sometimes completing after a long time (minute
plus) to load.   The second site "shared_73_165" loads just fine.  

 

Please note a week ago I had the same sites defined, but IE had problems
loading the "shared_73_165" site but loaded the "hostname1" site just
fine.   I assume there is some slight difference in the configuration
between this week and last to make this change occur, but I don't know
what it would be.    Mozilla has always loaded both sites fine.

 

 

I am having trouble identifying the root cause of the problem.   Can
anyone help me with trying to determine why IE 7 is taking so long to
complete the loading of the page if it completes at all?   

 

I can also add that a when I look at the varnishncsa output I see that
with IE 7 that not all of the elements on the HTML page are being
requested, but through Mozilla I do see all the requests.   I can also
tell you both sites work fine if I bypass Varnish.   It also does appear
that IE 8 has the same issue although I have not tested with it as much
as I have IE 7.

 

Any help that someone can provide would be appreciated?

 

 

 

backend hostname1

{

 
.host="xxx.xxx.xxx.xxx";.port="80";.max_connections=200;.connect_timeout
=10s;.first_byte_timeout=10s;

  .between_bytes_timeout=10s;

}

backend shared_73_165

{

 
.host="xxx.xxx.xxx.xxx";.port="80";.max_connections=200;.connect_timeout
=10s;.first_byte_timeout=10s;

  .between_bytes_timeout=10s;

}

 

sub vcl_recv

{

  # Set Backend Server

  if (req.http.host ~ "^hostname1.com"     || req.http.host ~
"^test.hostname1.com")              { set req.backend = hostname1;  }

  else
{ set req.backend = shared_73_165; }

 

  # Set HTTP Header NS-Client-IP so Apache can log client's IP address.

  set req.http.NS-Client-IP = client.ip;

 

  # If HTTP Header Cache-Contro="no-cache" then purge any cache page
that may exist for that URL and pass request on.

  if (req.http.Cache-Control ~ "no-cache") { purge_url(req.url);
return(pass); }

 

  # If request is not a valid HTTP Method pipe it to the backend it is
not cacheable.

  if (req.request!="GET"    && req.request!="HEAD"    &&
req.request!="PUT"     && req.request!="POST" &&

      req.request!="TRACE"  && req.request!="OPTIONS" &&
req.request!="DELETE")

  { return (pipe); }  /*Non-RFC2616*/

 

  # If request is not a GET or a HEAD then pass it through since its not
cacheable.

  if (req.request!="GET" || req.request!="HEAD") { return (pass); }

 

  # Unset cookies on Drupal specific files that should be cached.

  if (req.url ~ "^/(files|misc|sites|themes|modules)/" || req.url ~
"\.(txt|ico)$") { unset req.http.Cookie; }

 

  # If request requires Authentication or has a Cookie then it is not
cacheable.

  if (req.http.Authorization || req.http.Cookie) { return (pass); }

 

  # Normalize and compress content if Encoding starts with gzip or
defalte but do not compress binary files.

  # @see:  http://varnish.projects.linpro.no/wiki/FAQ/Compression

  if (req.http.Accept-Encoding) {

    if    (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") { remove
req.http.Accept-Encoding; }

    elsif (req.http.Accept-Encoding ~ "gzip")   {set
req.http.Accept-Encoding = "gzip"; }

    elsif (req.http.Accept-Encoding ~ "deflate"){set
req.http.Accept-Encoding = "deflate"; }

    else                                        {unset
req.http.Accept-Encoding; }

  }

 

  # If requested page is currently being processed on another thread to
a backend server,  server cached page if its not older than 30s.

  set req.grace = 30s;

 

  # Fall back to the default vcl_recv which will call the
return(lookup);

}

 

sub vcl_pipe

{

  set req.http.connection = "close";

  # Fall back to the default vcl_pipe which will call return(pipe);

}

 

sub vcl_fetch

{

  # If the response from the backend is not a 200, 203, 300, 301, 302,
404 or 410.

  if (!beresp.cacheable) { return (pass); }

}

-------------------------------------------------------- 
Raymond Bacion 

 

ITSD Operating System Group 
Phone: (573) 522-1468 
Fax :    (573) 522-1384 
Email:  Ray.Bacion at oa.mo.gov 

 

------------------------------------------------------- 

 

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110204/0cfde53b/attachment.html>

From perbu at varnish-software.com  Fri Feb  4 17:12:14 2011
From: perbu at varnish-software.com (Per Buer)
Date: Fri, 4 Feb 2011 18:12:14 +0100
Subject: Changing the fetch for virtual hosts.
In-Reply-To: <alpine.LRH.2.00.1102041113330.3092@maequeses.nmr.mgh.harvard.edu>
References: <AANLkTikbYsNzsUR95tWz3Qqa5z7Bvxc20SUs6xqenfvn@mail.gmail.com>
	<alpine.LRH.2.00.1102031529190.10090@maequeses.nmr.mgh.harvard.edu>
	<AANLkTi=+dWrYH7e9z37yeCw7kp+qk04Vt9CxhOxERhoQ@mail.gmail.com>
	<alpine.LRH.2.00.1102040907030.3092@maequeses.nmr.mgh.harvard.edu>
	<AANLkTinbZ8bv5gAe4w3J3VoV=UVfxsNw43LxfugxENuF@mail.gmail.com>
	<alpine.LRH.2.00.1102041113330.3092@maequeses.nmr.mgh.harvard.edu>
Message-ID: <AANLkTikCtPK0pJarEh+_VBbL+Pz89byQfETGV6+Y+YQr@mail.gmail.com>

On Fri, Feb 4, 2011 at 5:17 PM, Chris Johnson
<johnson at nmr.mgh.harvard.edu> wrote:
> On Fri, 4 Feb 2011, Per Buer wrote:
>>
>> ? ? ?if (req.http.host ~ "^www.example.com") {
>> ? ? ? ?set req.backend = foo;
>> ? ? ?}
>
> ? ? Which goes in vcl_recv and in his case foo is a backend declaration
> function that sets up .host and .port for the virtual host I want, yes?

Almost. foo is here a declared backend - declared with the keyword
"backend" - http://www.varnish-cache.org/docs/trunk/tutorial/backend_servers.html





-- 
Per Buer,?Varnish Software
Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer
Varnish makes websites fly!
Want to learn more about Varnish? http://www.varnish-software.com/whitepapers



From drais at icantclick.org  Fri Feb  4 22:33:14 2011
From: drais at icantclick.org (david raistrick)
Date: Fri, 4 Feb 2011 17:33:14 -0500 (EST)
Subject: update req.backend to contain .host
Message-ID: <alpine.BSF.2.00.1102041731340.54349@murf.icantclick.org>


re #varnish discussion about req.backend (when being set in a http header) 
only contains the director (if it's a director) and doesn't allow any way 
to see the actual backend.

<Mithrandir> keen: maybe the string representation of the various backends
              should include the .host field.



thanks. :)

--
david raistrick        http://www.netmeister.org/news/learn2quote.html
drais at icantclick.org             http://www.expita.com/nomime.html




From Ray.Bacion at oa.mo.gov  Fri Feb  4 18:05:02 2011
From: Ray.Bacion at oa.mo.gov (Bacion, Ray)
Date: Fri, 4 Feb 2011 12:05:02 -0600
Subject: Varnish not completely loading page from IE 7 browser
In-Reply-To: <7F0AA702B8A85A4A967C4C8EBAD6902CDE6E93@TMG-EVS02.torstar.net>
References: <20C7CC58ED808E429A2B575E3CB585EB33ED0172@EXCHVS3A.mx.state.mo.us>
	<7F0AA702B8A85A4A967C4C8EBAD6902CDE6E93@TMG-EVS02.torstar.net>
Message-ID: <20C7CC58ED808E429A2B575E3CB585EB33ED0186@EXCHVS3A.mx.state.mo.us>

Thank you for your quick response.

The version of Varnish is 2.1.4-2.el5
The IE version is 7.0.5370.13
Nothing is HTTPS.
The backends are on the same box as Varnish.  They do point to two different sites we are testing.
There is a load balancer a CITRIX NetScaler that is in front of Varnish.
I'm not sure if these sites use external components as I did not develop the site.  If this becomes an important issue, I will ask the developers.

One other note of interest.   In retesting everything before sending this email, I learned that everything is working using IE if I go directly to Varnish, however, again one of the two sites is not responding properly when I use IE and go through the CITRIX load balancer to Varnish.   I also had a co-worker test on a different machine.   He was using the same IE version.   He had the same results, however, for whatever reason he had trouble getting to the second website, but got to the first website just fine.   Essentially he is having trouble getting to the opposite site that I was having trouble getting to when testing through the CITRIX load balancer to Varnish.   Both sites do fine when connecting through CITRIX and make a direct connection to the web servers.

Is there any specific concerns with running Varnish behind a CITRIX load balancer?

Again, any help you can provide on troubleshooting this problem would be appreciated.


From: Caunter, Stefan [mailto:scaunter at topscms.com]
Sent: Friday, February 04, 2011 10:58 AM
To: Bacion, Ray; varnish-misc at varnish-cache.org
Cc: OA.Lampadmins
Subject: RE: Varnish not completely loading page from IE 7 browser

We need to know which version of varnish.

Which  exact version of IE7? Does the page call external components? Anything https? Are the back ends the same? Is there a load balancer?

Stefan Caunter
Operations
Torstar Digital
m: (416) 561-4871


From: varnish-misc-bounces at varnish-cache.org [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Bacion, Ray
Sent: February-03-11 2:01 PM
To: 'varnish-misc at varnish-cache.org'
Cc: OA.Lampadmins
Subject: Varnish not completely loading page from IE 7 browser

Hello, we have installed Varnish on one of our RedHat 5 Linux machines.   I have pasted the configuration below.    I did remove the real hostnames and IP addresses from the configuration.

The configuration works fine with Mozilla for sites I have defined, but on IE 7 the "hostname1" site does not completely load.   I see the favicon.ico and the title of the page, but the site continues to load without completing or sometimes completing after a long time (minute plus) to load.   The second site "shared_73_165" loads just fine.

Please note a week ago I had the same sites defined, but IE had problems loading the "shared_73_165" site but loaded the "hostname1" site just fine.   I assume there is some slight difference in the configuration between this week and last to make this change occur, but I don't know what it would be.    Mozilla has always loaded both sites fine.


I am having trouble identifying the root cause of the problem.   Can anyone help me with trying to determine why IE 7 is taking so long to complete the loading of the page if it completes at all?

I can also add that a when I look at the varnishncsa output I see that with IE 7 that not all of the elements on the HTML page are being requested, but through Mozilla I do see all the requests.   I can also tell you both sites work fine if I bypass Varnish.   It also does appear that IE 8 has the same issue although I have not tested with it as much as I have IE 7.

Any help that someone can provide would be appreciated?



backend hostname1
{
  .host="xxx.xxx.xxx.xxx";.port="80";.max_connections=200;.connect_timeout=10s;.first_byte_timeout=10s;
  .between_bytes_timeout=10s;
}
backend shared_73_165
{
  .host="xxx.xxx.xxx.xxx";.port="80";.max_connections=200;.connect_timeout=10s;.first_byte_timeout=10s;
  .between_bytes_timeout=10s;
}

sub vcl_recv
{
  # Set Backend Server
  if (req.http.host ~ "^hostname1.com"     || req.http.host ~ "^test.hostname1.com")              { set req.backend = hostname1;  }
  else                                                                                                                                                                       { set req.backend = shared_73_165; }

  # Set HTTP Header NS-Client-IP so Apache can log client's IP address.
  set req.http.NS-Client-IP = client.ip;

  # If HTTP Header Cache-Contro="no-cache" then purge any cache page that may exist for that URL and pass request on.
  if (req.http.Cache-Control ~ "no-cache") { purge_url(req.url); return(pass); }

  # If request is not a valid HTTP Method pipe it to the backend it is not cacheable.
  if (req.request!="GET"    && req.request!="HEAD"    && req.request!="PUT"     && req.request!="POST" &&
      req.request!="TRACE"  && req.request!="OPTIONS" && req.request!="DELETE")
  { return (pipe); }  /*Non-RFC2616*/

  # If request is not a GET or a HEAD then pass it through since its not cacheable.
  if (req.request!="GET" || req.request!="HEAD") { return (pass); }

  # Unset cookies on Drupal specific files that should be cached.
  if (req.url ~ "^/(files|misc|sites|themes|modules)/" || req.url ~ "\.(txt|ico)$") { unset req.http.Cookie; }

  # If request requires Authentication or has a Cookie then it is not cacheable.
  if (req.http.Authorization || req.http.Cookie) { return (pass); }

  # Normalize and compress content if Encoding starts with gzip or defalte but do not compress binary files.
  # @see:  http://varnish.projects.linpro.no/wiki/FAQ/Compression
  if (req.http.Accept-Encoding) {
    if    (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") { remove req.http.Accept-Encoding; }
    elsif (req.http.Accept-Encoding ~ "gzip")   {set req.http.Accept-Encoding = "gzip"; }
    elsif (req.http.Accept-Encoding ~ "deflate"){set req.http.Accept-Encoding = "deflate"; }
    else                                        {unset req.http.Accept-Encoding; }
  }

  # If requested page is currently being processed on another thread to a backend server,  server cached page if its not older than 30s.
  set req.grace = 30s;

  # Fall back to the default vcl_recv which will call the return(lookup);
}

sub vcl_pipe
{
  set req.http.connection = "close";
  # Fall back to the default vcl_pipe which will call return(pipe);
}

sub vcl_fetch
{
  # If the response from the backend is not a 200, 203, 300, 301, 302, 404 or 410.
  if (!beresp.cacheable) { return (pass); }
}
--------------------------------------------------------
Raymond Bacion

ITSD Operating System Group
Phone: (573) 522-1468
Fax :    (573) 522-1384
Email:  Ray.Bacion at oa.mo.gov

-------------------------------------------------------



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110204/af876690/attachment.html>

From johnson at nmr.mgh.harvard.edu  Mon Feb  7 13:49:26 2011
From: johnson at nmr.mgh.harvard.edu (Chris Johnson)
Date: Mon, 7 Feb 2011 08:49:26 -0500 (EST)
Subject: Changing the fetch for virtual hosts.
In-Reply-To: <AANLkTikCtPK0pJarEh+_VBbL+Pz89byQfETGV6+Y+YQr@mail.gmail.com>
References: <AANLkTikbYsNzsUR95tWz3Qqa5z7Bvxc20SUs6xqenfvn@mail.gmail.com>
	<alpine.LRH.2.00.1102031529190.10090@maequeses.nmr.mgh.harvard.edu>
	<AANLkTi=+dWrYH7e9z37yeCw7kp+qk04Vt9CxhOxERhoQ@mail.gmail.com>
	<alpine.LRH.2.00.1102040907030.3092@maequeses.nmr.mgh.harvard.edu>
	<AANLkTinbZ8bv5gAe4w3J3VoV=UVfxsNw43LxfugxENuF@mail.gmail.com>
	<alpine.LRH.2.00.1102041113330.3092@maequeses.nmr.mgh.harvard.edu>
	<AANLkTikCtPK0pJarEh+_VBbL+Pz89byQfETGV6+Y+YQr@mail.gmail.com>
Message-ID: <alpine.LRH.2.00.1102070838580.556@maequeses.nmr.mgh.harvard.edu>

      Huh.  Well, I think I've done everything everyone said was right.
Here's a snippet of what I have in varnish's default.vcl;

backend freesurfer {
       .host = "127.0.0.1";
       .port = "8080";
}

which happens to be the same place the default backend points to
because this is a different virtual host on the same server.

And in vcl_recv I have;

     if (req.http.host ~ "^sufer.nmr.mgh.harvard.edu") {
       set req.http.host = "www.freesurfer.net";
       set req.backend = freesurfer;
     }

which referenes my backend declaration and rewrites the header host
name.

      How do I tell if this is working?  Remember, I'm trying to make
the old virtual host references to sufer.nmr.mgh.harvard.edu be keyed
on and fetch from the new virtual host www.freesurfer.net so varnish
doesn't cache everything twice.

      Varnich didn't complain when I restarted it and seems to be
caching.  I'm just not sure what at this point.

> On Fri, Feb 4, 2011 at 5:17 PM, Chris Johnson
> <johnson at nmr.mgh.harvard.edu> wrote:
>> On Fri, 4 Feb 2011, Per Buer wrote:
>>>
>>> ? ? ?if (req.http.host ~ "^www.example.com") {
>>> ? ? ? ?set req.backend = foo;
>>> ? ? ?}
>>
>> ? ? Which goes in vcl_recv and in his case foo is a backend declaration
>> function that sets up .host and .port for the virtual host I want, yes?
>
> Almost. foo is here a declared backend - declared with the keyword
> "backend" - http://www.varnish-cache.org/docs/trunk/tutorial/backend_servers.html
>
>
>
>
>
> -- 
> Per Buer,?Varnish Software
> Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer
> Varnish makes websites fly!
> Want to learn more about Varnish? http://www.varnish-software.com/whitepapers
>
>
>

-------------------------------------------------------------------------------
Chris Johnson               |Internet: johnson at nmr.mgh.harvard.edu
Systems Administrator       |Web:      http://www.nmr.mgh.harvard.edu/~johnson
NMR Center                  |Voice:    617.726.0949
Mass. General Hospital      |FAX:      617.726.7422
149 (2301) 13th Street      |Man's a kind of missing link
Charlestown, MA., 02129 USA |fondly thinking he can think.  Piet Hein
-------------------------------------------------------------------------------


The information in this e-mail is intended only for the person to whom it is
addressed. If you believe this e-mail was sent to you in error and the e-mail
contains patient information, please contact the Partners Compliance HelpLine at
http://www.partners.org/complianceline . If the e-mail was sent to you in error
but does not contain patient information, please contact the sender and properly
dispose of the e-mail.

From dhelkowski at sbgnet.com  Mon Feb  7 15:02:39 2011
From: dhelkowski at sbgnet.com (David Helkowski)
Date: Mon, 07 Feb 2011 10:02:39 -0500
Subject: ESI includes don't account for manually timed out files
Message-ID: <4D50098F.6070303@sbgnet.com>

I have noticed what seems to be a bug in varnish. If anyone has 
experience with using ESI and timing things out from the cache manually 
at the same time, please take a look at this and, if it is not a bug, 
take a moment to explain what I am missing.

I have ESI setup in varnish. I have a file 'a.ehtml' that includes 
'b.html' via an ESI include. I have my hit function setup to do 'set 
obj.ttl=0s' when a 'no-cache' request is sent in.

If I make a request to b.html to reset it via 'no-cache' ( shift-refresh 
from my browser ), 'a.ehtml' still continues to be cached for the 
original ttl of 'b.html' before I set it to 0.

That is:
1. b.html contains '1'
2. a.ehtml shows '1' in the browser ( it includes b )
3. change the contents of b.html to '2'
4. shift-refresh b.html to force it to update in the cache
5. b.html now shows '2' in a browser
6. a.ehtml continues to show '1' ( until time length of original timeout 
of b.html passes )

The relevant portion of vcl_hit:
if( req.http.Cache-Control == "no-cache" ) {
   if (! (req.http.User-Agent ~ "bot|MSIE") ) {
     set obj.ttl = 0s; return(restart);
   }
}

The revelant portion of vcl_fetch:
if( req.url ~ "\.ehtml$" || beresp.http.Content-Type == "text/ehtml" ) {
   set beresp.http.Content-Type = "text/html";
   esi;
}




From phk at phk.freebsd.dk  Mon Feb  7 17:00:32 2011
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Mon, 07 Feb 2011 17:00:32 +0000
Subject: Stupid memory leak in -trunk fixed...
Message-ID: <9568.1297098032@critter.freebsd.dk>


It's always easier to see once you have found the bug, and now I did.

For some time varnish-trunk has been leaking all the malloc'ed
memory used for pass requests.

Sorry about that, fixed now.


-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.



From rtshilston at gmail.com  Mon Feb  7 22:18:14 2011
From: rtshilston at gmail.com (Robert Shilston)
Date: Mon, 7 Feb 2011 22:18:14 +0000
Subject: Bad HTTP request
Message-ID: <7DFB3F91-82B6-4808-B08A-C5690E36D381@gmail.com>

Hi,

For various reasons we're seeing a request such as:

  276 RxRequest    c POST
  276 RxURL        c /abc
  276 RxProtocol   c HTTP/1.1
  276 RxHeader     c X-Forwarded-For: 1.2.3.4
  276 RxHeader     c x-param-1: 13
  276 RxHeader     c x-param-2: 12sdf
  276 RxHeader     c x-param-3
  276 RxHeader     c x-param-4: DEF
  276 RxHeader     c Host: example.com

The problem is that the HTTP client is sending simply "x-param-3" if the value for this header is null.  When Varnish passes this on to Apache, Apache rejects it as an invalid request.     I can't seem to edit this header in VCL, either by doing an 'if (req.http.x-param-3 == "")...', or by simply 'set req.http.x-param-3 = "foo";'.

Does anyone have a suggestion as to how to deal with this bad HTTP client?  I appreciate the best thing to do is fix the client, but that'll take a few weeks to get redeployed, and I'm hoping to fix this issue in Varnish.


Rob


From phk at phk.freebsd.dk  Mon Feb  7 22:21:54 2011
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Mon, 07 Feb 2011 22:21:54 +0000
Subject: Bad HTTP request 
In-Reply-To: Your message of "Mon, 07 Feb 2011 22:18:14 GMT."
	<7DFB3F91-82B6-4808-B08A-C5690E36D381@gmail.com> 
Message-ID: <28504.1297117314@critter.freebsd.dk>

In message <7DFB3F91-82B6-4808-B08A-C5690E36D381 at gmail.com>, Robert Shilston wr
ites:

>The problem is that the HTTP client is sending simply "x-param-3"
>if the value for this header is null.

It is an invalid request.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.



From straightflush at gmail.com  Tue Feb  8 01:13:00 2011
From: straightflush at gmail.com (AD)
Date: Mon, 7 Feb 2011 20:13:00 -0500
Subject: Bad HTTP request
In-Reply-To: <28504.1297117314@critter.freebsd.dk>
References: <7DFB3F91-82B6-4808-B08A-C5690E36D381@gmail.com>
	<28504.1297117314@critter.freebsd.dk>
Message-ID: <AANLkTimvt1-eFp1ex3Df1xcwAQOWugkBnVOdZs74V9kO@mail.gmail.com>

why doesnt

 set req.http.x-param-3 = "foo";

work ?  What does varnishlog show?



On Mon, Feb 7, 2011 at 5:21 PM, Poul-Henning Kamp <phk at phk.freebsd.dk>wrote:

> In message <7DFB3F91-82B6-4808-B08A-C5690E36D381 at gmail.com>, Robert
> Shilston wr
> ites:
>
> >The problem is that the HTTP client is sending simply "x-param-3"
> >if the value for this header is null.
>
> It is an invalid request.
>
> --
> Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
> phk at FreeBSD.ORG         | TCP/IP since RFC 956
> FreeBSD committer       | BSD since 4.3-tahoe
> Never attribute to malice what can adequately be explained by incompetence.
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110207/aa9195e6/attachment.html>

From j.begumisa at gmail.com  Tue Feb  8 01:22:35 2011
From: j.begumisa at gmail.com (Joseph Begumisa)
Date: Mon, 7 Feb 2011 17:22:35 -0800
Subject: Issue with varnish passing absolute url to backend
Message-ID: <AANLkTinpjs-92yzHhRmVq5uSaK5AoLENmr4O_gU4_DTT@mail.gmail.com>

Hi,

Just installed varnish and I have an issue with varnish sending GET
requests to the backend server.  If I try to access
http://test.example.com/username, varnish sends GET /username to the
backend server running zeus web server.

The problem with this is I have a couple of rewrites on the zeus web
server that rewrite the url http://test.example.com/username to
http://test.example.com/profile/username.  So the GET request sent by
varnish does not match the rewrite rule and I get back a 404 Page Not
found error.

I can change the rewrite rule, however, is this by design or am I
missing something in my configuration?  I'd like varnish to pass the
GET request as the absolute url i.e http://test.example.com/username

I tried to keep my default.vcl configuration as simple as possible for
now before building it up.


### Custom Configuration

backend web1 {
     .host = "10.10.26.73";
     .port = "80";
}

I've run a couple of searches, gone through the faqs and
documentations but I don't seem to see anything that would point to
why this happens.  Any help would be appreciated.  Thank you.

Best Regards,

Joseph



From varnish at mm.quex.org  Tue Feb  8 04:09:31 2011
From: varnish at mm.quex.org (Michael Alger)
Date: Tue, 8 Feb 2011 12:09:31 +0800
Subject: Issue with varnish passing absolute url to backend
In-Reply-To: <AANLkTinpjs-92yzHhRmVq5uSaK5AoLENmr4O_gU4_DTT@mail.gmail.com>
References: <AANLkTinpjs-92yzHhRmVq5uSaK5AoLENmr4O_gU4_DTT@mail.gmail.com>
Message-ID: <20110208040931.GA12290@grum.quex.org>

On Mon, Feb 07, 2011 at 05:22:35PM -0800, Joseph Begumisa wrote:
> 
> Just installed varnish and I have an issue with varnish sending GET
> requests to the backend server.  If I try to access
> http://test.example.com/username, varnish sends GET /username to the
> backend server running zeus web server.

This is the normal format for an HTTP request. The most basic level
looks like:

GET /username HTTP/1.0
Host: test.example.com

> The problem with this is I have a couple of rewrites on the zeus web
> server that rewrite the url http://test.example.com/username to
> http://test.example.com/profile/username.  So the GET request sent by
> varnish does not match the rewrite rule and I get back a 404 Page Not
> found error.

You will only see requests in the form of

GET http://test.example.com/username HTTP/1.0
Host: test.example.com

if the request is being made via a proxy, i.e. if the user-agent
making the request believes it is speaking to a proxy, rather than
directly to a web server.

Since Varnish is expecting to be speaking directly to an origin
server, it's not going to make proxy-style requests. Neither will
the client make proxy-style requests to Varnish, since from the
client's perspective, Varnish is the origin server.

> I can change the rewrite rule, however, is this by design or am I
> missing something in my configuration?  I'd like varnish to pass the
> GET request as the absolute url i.e http://test.example.com/username

It's definitely by design. You might be able to force Varnish to
send the absolute by doing something like:

set req.url = "http://test.example.com" req.url;

in vcl_recv, but ... I'd be hesitant to even try and see if that
approach "works".  It probably won't cause Varnish to explode, but
it's certainly an odd way of doing things.

A better solution would be to configure the web server properly.
Normally if you care about the hostname of the server, it's because
you're doing name-based virtual hosting. In this case, the web
server will receive the request, look at the Host: header that was
sent with it, and match that against a particular website. That
website can then have its own configuration, whereby it rewrites
requests of the form /username to /profile/username.

If this is purely for internal use, you might be able to work around
it by configuring Varnish as a proxy in your web browser. That still
seems like a hack though. Varnish is designed to behave like an
origin server, speaking to backends which also behave like origin
servers.



From TFigueiro at au.westfield.com  Tue Feb  8 04:27:02 2011
From: TFigueiro at au.westfield.com (Thiago Figueiro)
Date: Tue, 8 Feb 2011 15:27:02 +1100
Subject: Issue with varnish passing absolute url to backend
In-Reply-To: <20110208040931.GA12290@grum.quex.org>
References: <AANLkTinpjs-92yzHhRmVq5uSaK5AoLENmr4O_gU4_DTT@mail.gmail.com>
	<20110208040931.GA12290@grum.quex.org>
Message-ID: <64E73E81AAC26A49AC9EA28CBE65365107EF89BD@AUSYDEVS01.au.ad.westfield.com>

>From Michael Alger:

> This is the normal format for an HTTP request. The most basic level
> looks like:
>
> GET /username HTTP/1.0
> Host: test.example.com
>
> You will only see requests in the form of
>
> GET http://test.example.com/username HTTP/1.0
> Host: test.example.com
>
> if the request is being made via a proxy, i.e. if the user-agent

It looks to me like your suggestion to set req.url will do what the OP
needs, however your statement above is not accurate.

The "Host" header is part of the HTTP/1.1 implementation.  All HTTP/1.0
requests will be in the above format, made via a proxy or not.

I'm not sure if Varnish can be configured to talk HTTP 1.0 to a
backends.


Cheers,
Thiago.


______________________________________________________
    CONFIDENTIALITY NOTICE    
This electronic mail message, including any and/or all attachments, is for the sole use of the intended recipient(s), and may contain confidential and/or privileged information, pertaining to business conducted under the direction and supervision of the sending organization. All electronic mail messages, which may have been established as expressed views and/or opinions (stated either within the electronic mail message or any of its attachments), are left to the sole responsibility of that of the sender, and are not necessarily attributed to the sending organization. Unauthorized interception, review, use, disclosure or distribution of any such information contained within this electronic mail message and/or its attachment(s), is (are) strictly prohibited. If you are not the intended recipient, please contact the sender by replying to this electronic mail message, along with the destruction all copies of the original electronic mail message (along with any attachments).
______________________________________________________



From varnish at mm.quex.org  Tue Feb  8 04:40:32 2011
From: varnish at mm.quex.org (Michael Alger)
Date: Tue, 8 Feb 2011 12:40:32 +0800
Subject: Issue with varnish passing absolute url to backend
In-Reply-To: <64E73E81AAC26A49AC9EA28CBE65365107EF89BD@AUSYDEVS01.au.ad.westfield.com>
References: <AANLkTinpjs-92yzHhRmVq5uSaK5AoLENmr4O_gU4_DTT@mail.gmail.com>
	<20110208040931.GA12290@grum.quex.org>
	<64E73E81AAC26A49AC9EA28CBE65365107EF89BD@AUSYDEVS01.au.ad.westfield.com>
Message-ID: <20110208044032.GA13454@grum.quex.org>

On Tue, Feb 08, 2011 at 03:27:02PM +1100, Thiago Figueiro wrote:
> From Michael Alger:
> 
> > This is the normal format for an HTTP request. The most basic level
> > looks like:
> >
> > GET /username HTTP/1.0
> > Host: test.example.com
> >
> > You will only see requests in the form of
> >
> > GET http://test.example.com/username HTTP/1.0
> > Host: test.example.com
> >
> > if the request is being made via a proxy, i.e. if the user-agent
> 
> The "Host" header is part of the HTTP/1.1 implementation.

Most current UAs which send HTTP/1.0 requests include the Host
header, although I believe you're right in that it's only actually
required in HTTP/1.1. But there's lots of services that rely on the
Host header, because that's the only available method for having
multiple sites on the same IP/port combination.

I did consider mentioning that the host header was optional but
figured it would just confuse things, and requests without the host
header are rarely seen in the wild to the best of my knowledge.
(Doesn't mean you shouldn't be prepared to handle such requests, of
course.)

> All HTTP/1.0 requests will be in the above format, made via a
> proxy or not.

http://www.w3.org/Protocols/HTTP/1.0/spec.html#Request-URI

    Request-URI    = absoluteURI | abs_path

  The two options for Request-URI are dependent on the nature of the
  request. 

  The absoluteURI form is only allowed when the request is being made
  to a proxy.

The Host header was added to the spec precisely because the origin
server had no way of determining what host the client thought it was
speaking to.



From j.begumisa at gmail.com  Tue Feb  8 05:00:23 2011
From: j.begumisa at gmail.com (Joseph Begumisa)
Date: Mon, 7 Feb 2011 21:00:23 -0800
Subject: Issue with varnish passing absolute url to backend
In-Reply-To: <20110208040931.GA12290@grum.quex.org>
References: <AANLkTinpjs-92yzHhRmVq5uSaK5AoLENmr4O_gU4_DTT@mail.gmail.com>
	<20110208040931.GA12290@grum.quex.org>
Message-ID: <AANLkTimN3w2R3ebk2ARF+J9RCNey5YmU_y=7_Wp3soJG@mail.gmail.com>

On Mon, Feb 7, 2011 at 8:09 PM, Michael Alger <varnish at mm.quex.org> wrote:

> Since Varnish is expecting to be speaking directly to an origin
> server, it's not going to make proxy-style requests. Neither will
> the client make proxy-style requests to Varnish, since from the
> client's perspective, Varnish is the origin server.
>
>
> It's definitely by design.

Thanks for clarifying that it is by design.  I will just edit my
rewrite rules on the web server and the issue should be solved.

Joseph



From TFigueiro at au.westfield.com  Tue Feb  8 05:47:49 2011
From: TFigueiro at au.westfield.com (Thiago Figueiro)
Date: Tue, 8 Feb 2011 16:47:49 +1100
Subject: Issue with varnish passing absolute url to backend
In-Reply-To: <20110208044032.GA13454@grum.quex.org>
References: <AANLkTinpjs-92yzHhRmVq5uSaK5AoLENmr4O_gU4_DTT@mail.gmail.com><20110208040931.GA12290@grum.quex.org><64E73E81AAC26A49AC9EA28CBE65365107EF89BD@AUSYDEVS01.au.ad.westfield.com>
	<20110208044032.GA13454@grum.quex.org>
Message-ID: <64E73E81AAC26A49AC9EA28CBE65365107EF8A14@AUSYDEVS01.au.ad.westfield.com>

>  The absoluteURI form is only allowed when the request is being made
>  to a proxy.

Looks like I missed that line in the RFC (and so did a few client
developers :-) ).  Thanks for pointing it out.



______________________________________________________
    CONFIDENTIALITY NOTICE    
This electronic mail message, including any and/or all attachments, is for the sole use of the intended recipient(s), and may contain confidential and/or privileged information, pertaining to business conducted under the direction and supervision of the sending organization. All electronic mail messages, which may have been established as expressed views and/or opinions (stated either within the electronic mail message or any of its attachments), are left to the sole responsibility of that of the sender, and are not necessarily attributed to the sending organization. Unauthorized interception, review, use, disclosure or distribution of any such information contained within this electronic mail message and/or its attachment(s), is (are) strictly prohibited. If you are not the intended recipient, please contact the sender by replying to this electronic mail message, along with the destruction all copies of the original electronic mail message (along with any attachments).
______________________________________________________



From dhelkowski at sbgnet.com  Mon Feb  7 14:56:47 2011
From: dhelkowski at sbgnet.com (David Helkowski)
Date: Mon, 07 Feb 2011 09:56:47 -0500
Subject: ESI includes don't account for manually timed out files
Message-ID: <4D50082F.8090008@sbgnet.com>

I have noticed what seems to be a bug in varnish. If anyone has 
experience with using ESI and timing things out from the cache manually 
at the same time, please take a look at this and, if it is not a bug, 
take a moment to explain what I am missing.

I have ESI setup in varnish. I have a file 'a.ehtml' that includes 
'b.html' via an ESI include. I have my hit function setup to do 'set 
obj.ttl=0s' when a 'no-cache' request is sent in.

If I make a request to b.html to reset it via 'no-cache' ( shift-refresh 
from my browser ), 'a.ehtml' still continues to be cached for the 
original ttl of 'b.html' before I set it to 0.

That is:
1. b.html contains '1'
2. a.ehtml shows '1' in the browser ( it includes b )
3. change the contents of b.html to '2'
4. shift-refresh b.html to force it to update in the cache
5. b.html now shows '2' in a browser
6. a.ehtml continues to show '1' ( until time length of original timeout 
of b.html passes )

The relevant portion of vcl_hit:
  if( req.http.Cache-Control == "no-cache" ) {

    if (! (req.http.User-Agent ~ "bot|MSIE") ) {

        set obj.ttl = 0s; return(restart);

    }
    }

The revelant portion of vcl_fetch:

if( req.url ~ "\.ehtml$" || beresp.http.Content-Type == "text/ehtml" ) {

    set beresp.http.Content-Type = "text/html";
    esi;

    }




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110207/c82ad808/attachment.html>

From adam at dberg.org  Mon Feb  7 15:39:55 2011
From: adam at dberg.org (Adam Denenberg)
Date: Mon, 7 Feb 2011 10:39:55 -0500
Subject: Changing the fetch for virtual hosts.
In-Reply-To: <alpine.LRH.2.00.1102070838580.556@maequeses.nmr.mgh.harvard.edu>
References: <AANLkTikbYsNzsUR95tWz3Qqa5z7Bvxc20SUs6xqenfvn@mail.gmail.com>
	<alpine.LRH.2.00.1102031529190.10090@maequeses.nmr.mgh.harvard.edu>
	<AANLkTi=+dWrYH7e9z37yeCw7kp+qk04Vt9CxhOxERhoQ@mail.gmail.com>
	<alpine.LRH.2.00.1102040907030.3092@maequeses.nmr.mgh.harvard.edu>
	<AANLkTinbZ8bv5gAe4w3J3VoV=UVfxsNw43LxfugxENuF@mail.gmail.com>
	<alpine.LRH.2.00.1102041113330.3092@maequeses.nmr.mgh.harvard.edu>
	<AANLkTikCtPK0pJarEh+_VBbL+Pz89byQfETGV6+Y+YQr@mail.gmail.com>
	<alpine.LRH.2.00.1102070838580.556@maequeses.nmr.mgh.harvard.edu>
Message-ID: <AANLkTikJe_CV4VocFi1wEwGd6Y-736cWbL3hUmy3RKOG@mail.gmail.com>

 Run varnishlog against the request and confirm a TCP_HIT when making the
second request to the different hostname.  Also confirm you do not see the
request hit your backend webserver logfile



On Mon, Feb 7, 2011 at 8:49 AM, Chris Johnson
<johnson at nmr.mgh.harvard.edu>wrote:

>     Huh.  Well, I think I've done everything everyone said was right.
> Here's a snippet of what I have in varnish's default.vcl;
>
> backend freesurfer {
>      .host = "127.0.0.1";
>      .port = "8080";
> }
>
> which happens to be the same place the default backend points to
> because this is a different virtual host on the same server.
>
> And in vcl_recv I have;
>
>    if (req.http.host ~ "^sufer.nmr.mgh.harvard.edu") {
>      set req.http.host = "www.freesurfer.net";
>      set req.backend = freesurfer;
>    }
>
> which referenes my backend declaration and rewrites the header host
> name.
>
>     How do I tell if this is working?  Remember, I'm trying to make
> the old virtual host references to sufer.nmr.mgh.harvard.edu be keyed
> on and fetch from the new virtual host www.freesurfer.net so varnish
> doesn't cache everything twice.
>
>     Varnich didn't complain when I restarted it and seems to be
> caching.  I'm just not sure what at this point.
>
>
>  On Fri, Feb 4, 2011 at 5:17 PM, Chris Johnson
>> <johnson at nmr.mgh.harvard.edu> wrote:
>>
>>> On Fri, 4 Feb 2011, Per Buer wrote:
>>>
>>>>
>>>>      if (req.http.host ~ "^www.example.com") {
>>>>        set req.backend = foo;
>>>>      }
>>>>
>>>
>>>     Which goes in vcl_recv and in his case foo is a backend declaration
>>> function that sets up .host and .port for the virtual host I want, yes?
>>>
>>
>> Almost. foo is here a declared backend - declared with the keyword
>> "backend" -
>> http://www.varnish-cache.org/docs/trunk/tutorial/backend_servers.html
>>
>>
>>
>>
>>
>> --
>> Per Buer, Varnish Software
>> Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer
>> Varnish makes websites fly!
>> Want to learn more about Varnish?
>> http://www.varnish-software.com/whitepapers
>>
>>
>>
>>
>
> -------------------------------------------------------------------------------
> Chris Johnson               |Internet: johnson at nmr.mgh.harvard.edu
> Systems Administrator       |Web:
> http://www.nmr.mgh.harvard.edu/~johnson
> NMR Center                  |Voice:    617.726.0949
> Mass. General Hospital      |FAX:      617.726.7422
> 149 (2301) 13th Street      |Man's a kind of missing link
> Charlestown, MA., 02129 USA |fondly thinking he can think.  Piet Hein
>
>
> -------------------------------------------------------------------------------
>
>
> The information in this e-mail is intended only for the person to whom it
> is
> addressed. If you believe this e-mail was sent to you in error and the
> e-mail
> contains patient information, please contact the Partners Compliance
> HelpLine at
> http://www.partners.org/complianceline . If the e-mail was sent to you in
> error
> but does not contain patient information, please contact the sender and
> properly
> dispose of the e-mail.
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110207/cdb68365/attachment.html>

From B.Dodd at comicrelief.com  Tue Feb  8 23:05:01 2011
From: B.Dodd at comicrelief.com (Ben Dodd)
Date: Tue, 8 Feb 2011 23:05:01 +0000
Subject: Dual server Varnish setup 
Message-ID: <0E7D9950-E2A1-4B1C-B619-A0CDE528C8B1@comicrelief.com>

Given this functionality, does anyone know what the VCL is required to implement this dual server Varnish setup scenario?
http://varnish-cache.org/trac/wiki/VCLExampleHashIgnoreBusy

Thanks, Ben

Comic Relief  1st Floor  89 Albert Embankment  London SE1 7TP  Tel: 020 7820 2000  Fax: 020 7820 2222  red at comicrelief.com  www.comicrelief.com

Comic Relief is the operating name of Charity Projects, a company limited by guarantee and registered in England no. 1806414; registered charity 326568 (England & Wales) and SC039730 (Scotland). Comic Relief Ltd is a subsidiary of Charity Projects and registered in England no. 1967154. Registered offices: Hanover House, 14 Hanover Square, London W1S 1HP. VAT no. 773865187.

This email (and any attachment) may contain confidential and/or privileged information. If you are not the intended addressee, you must not use, disclose, copy or rely on anything in this email and should contact the sender and delete it immediately. The views of the author are not necessarily those of Comic Relief. We cannot guarantee that this email (and any attachment) is virus free or has not been intercepted and amended, so do not accept liability for any damage resulting from software viruses. You should carry out your own virus checks.





From david at firechaser.com  Tue Feb  8 23:14:05 2011
From: david at firechaser.com (David Murphy)
Date: Tue, 8 Feb 2011 23:14:05 +0000
Subject: Using varnishtop to show only cached objects
Message-ID: <AANLkTikZgyD9huFsE0LpfDRbgX8QXrGymzN9v9bR0sSn@mail.gmail.com>

Hello

Is it possible to show what is being added to the Varnish cache in
realtime, similar to how I can see backend requests using varnishtop
-i TxURL ?

I want to be able to study what is actually ending up on the cache as
I tweak VCL.

Thanks, David



From david at firechaser.com  Tue Feb  8 23:29:25 2011
From: david at firechaser.com (David Murphy)
Date: Tue, 8 Feb 2011 23:29:25 +0000
Subject: Segfault for varnishtop running latest Varnish on Ubuntu 10.10
Message-ID: <AANLkTincHnCTULfgcdk2KH__V9boETbyEsCihEc5JFq8@mail.gmail.com>

Hello

I've installed (as of 8th Feb 2011) the latest Varnish on my 32bit
Ubuntu 10.10 (macbook pro) using following:

curl http://repo.varnish-cache.org/debian/GPG-key.txt | apt-key add -
echo "deb http://repo.varnish-cache.org/debian/ $(lsb_release -s -c)
varnish-2.1" >> /etc/apt/sources.list
apt-get update
apt-get install varnish

Varnish runs fine but when I run

varnishtop -i rxurl

I get a segfault as soon as I load a web page to pass data through Varnish.

I've checked numerous threads and got the impression that this issue
was fixed (I recall having the same problem on Ubunutu 6 months ago) -
any way I can solve this?

Thanks, David



From sinkeyteck at yahoo.com  Wed Feb  9 06:50:40 2011
From: sinkeyteck at yahoo.com (KiT Sin)
Date: Wed, 9 Feb 2011 14:50:40 +0800
Subject: nested directors
Message-ID: <20110209065040.GB45946@passion.hsins.com>

is it possible to define a director using a set of directors?

i'm hoping to use hash director in varnish with a configuration like
below, so that whenever a backend squid is down, cache miss can be
avoided when varnish redistributes traffic away from the failed server.

e.g.

director rr1 round-robin {
    {
	.backend = squid1;
    }
    {
	.backend = squid2;
    }
}
	
director rr2 round-robin {
    {
	.backend = squid3;
    }
    {
	.backend = squid4;
    }
}
	
director rr3 round-robin {
    {
	.backend = squid5;
    }
    {
	.backend = squid6;
    }
}
	
director layer7 hash {
    {
	.backend = rr1;
    }
    {
	.backend = rr2;
    }
    {
	.backend = rr3;
    }
}

thanks
kit






From pwlazy at gmail.com  Wed Feb  9 06:43:29 2011
From: pwlazy at gmail.com (=?GB2312?B?xe3OsA==?=)
Date: Wed, 9 Feb 2011 14:43:29 +0800
Subject: install problem in Red Hat Enterprise Linux
Message-ID: <AANLkTi=g1UFSkMM1iAFTJhECFHfizXaX3U4oZoYcTT=p@mail.gmail.com>

hi all:

     i install varnish using the source code "varnish-2.1.4.tar.gz" in
ubuntu10.4,  command lines as follows:

     wget http://repo.varnish-cache.org/source/varnish-2.1.4.tar.gz
     tar xzvf varnish-2.1.4.tar.gz
     cd varnish-2.1.4/
    ./configure --prefix=/me/dev/varnish
     make && make install

     in the "/me/dev/varnish/bin"  directory, there are 9 tools as follows
     "varnishadm  varnishhist  varnishlog  varnishncsa  varnishreplay
varnishsizes  varnishstat  varnishtest  varnishtop"

     it's ok!

    but when i use the same install steps in "Red Hat Enterprise Linux
Server release 5.3 (Tikanga)", the problem appeared

    in the "/me/dev/varnish/bin"  directory, there are only 5 tools as
follows
    "varnishadm  varnishlog  varnishncsa  varnishreplay  varnishtest"

    serveral import tools such varnishstat aren't created, please help me!
thanks !



  Regards!
  pwlazy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110209/74083f80/attachment.html>

From bjorn at ruberg.no  Wed Feb  9 07:56:04 2011
From: bjorn at ruberg.no (=?UTF-8?B?QmrDuHJuIFJ1YmVyZw==?=)
Date: Wed, 09 Feb 2011 08:56:04 +0100
Subject: install problem in Red Hat Enterprise Linux
In-Reply-To: <AANLkTi=g1UFSkMM1iAFTJhECFHfizXaX3U4oZoYcTT=p@mail.gmail.com>
References: <AANLkTi=g1UFSkMM1iAFTJhECFHfizXaX3U4oZoYcTT=p@mail.gmail.com>
Message-ID: <4D524894.2030204@ruberg.no>

On 02/09/2011 07:43 AM, ?? wrote:
> hi all:
>
>       i install varnish using the source code "varnish-2.1.4.tar.gz" in
> ubuntu10.4,  command lines as follows:
>
>       wget http://repo.varnish-cache.org/source/varnish-2.1.4.tar.gz
>       tar xzvf varnish-2.1.4.tar.gz
>       cd varnish-2.1.4/
>      ./configure --prefix=/me/dev/varnish
>       make && make install
>
>       in the "/me/dev/varnish/bin"  directory, there are 9 tools as follows
> "varnishadm  varnishhist  varnishlog  varnishncsa  varnishreplay
> varnishsizes  varnishstat  varnishtest  varnishtop"
>
>       it's ok!
>
>      but when i use the same install steps in "Red Hat Enterprise Linux
> Server release 5.3 (Tikanga)", the problem appeared
>
>      in the "/me/dev/varnish/bin"  directory, there are only 5 tools as
> follows
> "varnishadm  varnishlog  varnishncsa  varnishreplay  varnishtest"
>
>      serveral import tools such varnishstat aren't created, please help
> me! thanks !

This indicates that you did not install the ncurses development 
libraries on your RHEL system.

-- 
Bj?rn




From pom at dmsp.de  Wed Feb  9 10:47:00 2011
From: pom at dmsp.de (Stefan Pommerening)
Date: Wed, 09 Feb 2011 11:47:00 +0100
Subject: How to find out a single failing backend from a round-robin director?
	(Inline-C)
Message-ID: <4D5270A4.5040104@dmsp.de>

Hi all,

I'm working with a set of varnish-caches (2.1.3) in front of some 
round-robin director configured backends.
Sometimes I experience a 503 error due to some unforseeable 
circumstances like network hiccups, content server problems and other 
magic...  ;-)
Nevertheless all backends are healthy regarding the large majority of 
other requests (and watchdog of course), we deal with sporadic failures 
here.

Obviously I don't want those 503's to go through to the user.

I inserted the following code in my vcl_fetch:

if (beresp.status == 503) {
    set beresp.saintmode = 2s;
    restart;
}

This works quite well, the failed (503 delivering) request is forwarded 
to the next backend in the director and is successfully served then.

What I am missing is the exact backend information which backend did not 
serve the request or better said: where did the 503 come from?
I already included some inline-C in order to send info to syslog, but I 
am missing some VRT_(r_)something to find out the failing backend 
without searching the backend's logfiles directly.

My current syslog()-msg is:

syslog (LOG_LOCAL7 | LOG_INFO, "VarnishLog(vcl_fetch): %s/%s[%s] 
[REST=%02d] %s http://%s%s",
    VRT_backend_string(sp), VRT_r_server_hostname(sp), 
VRT_IP_string(sp,VRT_r_server_ip(sp)),
    VRT_r_req_restarts(sp), VRT_r_req_request(sp), VRT_GetHdr(sp, 
HDR_REQ, "\005host:"), VRT_r_req_url(sp));

I managed to get the backend definition, but not the specific backend 
machine name (or IP) from the backend director.

... and: is there some sort of overview on which VRT_magic fields (for 
use with inline-C) exists?

Thanks in advance,
Stefan

-- 

*Dipl.-Inform. Stefan Pommerening
Informatik-B?ro: IT-Dienste & Projekte, Consulting & Coaching*
http://www.dmsp.de


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110209/658bb622/attachment.html>

From david at firechaser.com  Wed Feb  9 12:47:46 2011
From: david at firechaser.com (David Murphy)
Date: Wed, 9 Feb 2011 12:47:46 +0000
Subject: Segfault for varnishtop running latest Varnish on Ubuntu 10.10
In-Reply-To: <AANLkTincHnCTULfgcdk2KH__V9boETbyEsCihEc5JFq8@mail.gmail.com>
References: <AANLkTincHnCTULfgcdk2KH__V9boETbyEsCihEc5JFq8@mail.gmail.com>
Message-ID: <AANLkTi=L83TFgJ5h96kDPOOj6HeR55h4oG+M1iH_p8gr@mail.gmail.com>

I managed to fix this.

I removed my Varnish install

apt-get remove varnish

replaced "deb http://repo.varnish-cache.org/debian/ maverick
varnish-2.1" with "deb http://gb.archive.ubuntu.com/ubuntu/
maverick-proposed restricted main multiverse universe"  in source.list

apt-get update
apt-get install varnish

varnishtop now seems to be working fine on 32bit, (am now using
varnish_2.1.3-7ubuntu0.1_i386.deb and
libvarnish1_2.1.3-7ubuntu0.1_i386.deb).

Best, David

On Tue, Feb 8, 2011 at 11:29 PM, David Murphy <david at firechaser.com> wrote:
> Hello
>
> I've installed (as of 8th Feb 2011) the latest Varnish on my 32bit
> Ubuntu 10.10 (macbook pro) using following:
>
> curl http://repo.varnish-cache.org/debian/GPG-key.txt | apt-key add -
> echo "deb http://repo.varnish-cache.org/debian/ $(lsb_release -s -c)
> varnish-2.1" >> /etc/apt/sources.list
> apt-get update
> apt-get install varnish
>
> Varnish runs fine but when I run
>
> varnishtop -i rxurl
>
> I get a segfault as soon as I load a web page to pass data through Varnish.
>
> I've checked numerous threads and got the impression that this issue
> was fixed (I recall having the same problem on Ubunutu 6 months ago) -
> any way I can solve this?
>
> Thanks, David
>



From tfheen at varnish-software.com  Wed Feb  9 13:06:36 2011
From: tfheen at varnish-software.com (Tollef Fog Heen)
Date: Wed, 09 Feb 2011 14:06:36 +0100
Subject: Segfault for varnishtop running latest Varnish on Ubuntu 10.10
In-Reply-To: <AANLkTi=L83TFgJ5h96kDPOOj6HeR55h4oG+M1iH_p8gr@mail.gmail.com>
	(David Murphy's message of "Wed, 9 Feb 2011 12:47:46 +0000")
References: <AANLkTincHnCTULfgcdk2KH__V9boETbyEsCihEc5JFq8@mail.gmail.com>
	<AANLkTi=L83TFgJ5h96kDPOOj6HeR55h4oG+M1iH_p8gr@mail.gmail.com>
Message-ID: <87d3n19x2b.fsf@qurzaw.varnish-software.com>

]] David Murphy 

| apt-get remove varnish
| 
| replaced "deb http://repo.varnish-cache.org/debian/ maverick
| varnish-2.1" with "deb http://gb.archive.ubuntu.com/ubuntu/
| maverick-proposed restricted main multiverse universe"  in source.list
| 
| apt-get update
| apt-get install varnish
| 
| varnishtop now seems to be working fine on 32bit, (am now using
| varnish_2.1.3-7ubuntu0.1_i386.deb and
| libvarnish1_2.1.3-7ubuntu0.1_i386.deb).

We don't support maverick on repo.varnish-cache.org, only the LTS
releases are supported so this might explain it.  I guess we should
document that clearly on the wiki page.

-- 
Tollef Fog Heen
Varnish Software
t: +47 21 98 92 64



From david at firechaser.com  Wed Feb  9 14:22:17 2011
From: david at firechaser.com (David Murphy)
Date: Wed, 9 Feb 2011 14:22:17 +0000
Subject: Using varnishtop to show only cached objects
In-Reply-To: <AANLkTikZgyD9huFsE0LpfDRbgX8QXrGymzN9v9bR0sSn@mail.gmail.com>
References: <AANLkTikZgyD9huFsE0LpfDRbgX8QXrGymzN9v9bR0sSn@mail.gmail.com>
Message-ID: <AANLkTimBAXFd-7S60+zHvA4eKUvoKTTNP=ZS8WxkoUsM@mail.gmail.com>

Here's what I've tried so far:

1. Catching headers that have 'X-Cache: HIT | MISS' header, using

varnishtop -C -I \^X-Cache

This doesn't return any results, even though they are in the logs e.g.

19.27    TxHeader        X-Cache: HIT

I assumed it was a regex syntax problem but the following returns results fine:

varnishtop -C -I \^User-Agent

Anyway, my actual aim in all of this is to be able to see what is
actually being stored in / added to the cache. I can see all the
request that reach the backend no problem ( using varnishtop -i txurl
) but am a bit stuck on how to display only the items that are
actually stored in the cache.

Thanks, David






On Tue, Feb 8, 2011 at 11:14 PM, David Murphy <david at firechaser.com> wrote:
> Hello
>
> Is it possible to show what is being added to the Varnish cache in
> realtime, similar to how I can see backend requests using varnishtop
> -i TxURL ?
>
> I want to be able to study what is actually ending up on the cache as
> I tweak VCL.
>
> Thanks, David
>



From david at firechaser.com  Wed Feb  9 15:14:24 2011
From: david at firechaser.com (David Murphy)
Date: Wed, 9 Feb 2011 15:14:24 +0000
Subject: Caching only images, no cookies
In-Reply-To: <AANLkTikOf0kYAymJeZCpYQBdRwBTkNuynAHUCCXGwgki@mail.gmail.com>
References: <AANLkTi=rSRGteygdk2XFx+f-7h3EMhJ4366BRfKctpR5@mail.gmail.com>
	<AANLkTincsN34O=6Q4KmDRTf9tV=atfU7kjYy2-oe+rvn@mail.gmail.com>
	<AANLkTikkBt63SQYB3aawrajrgn0R-Kud8Ea34mtBUzCg@mail.gmail.com>
	<AANLkTikOf0kYAymJeZCpYQBdRwBTkNuynAHUCCXGwgki@mail.gmail.com>
Message-ID: <AANLkTikNYag=LQk=qDxjr1sdG284cfU8e6MP0i4+63mi@mail.gmail.com>

Hello

I've been testing removing cookies from images/css/js and am a little
unclear on the difference between:


//start ==========
 sub vcl_recv {
  if (req.url ~ "\.(png|gif|jpg|swf|css|js)$") {
        remove req.http.Cookie;
   }
//end==========


and...


//start =========
sub vcl_recv {
  if (req.url ~ "\.(png|gif|jpg|swf|css|js)$") {
    unset req.http.cookie;
  }
}

sub vcl_fetch {
  if (req.url ~ "\.(png|gif|jpg|swf|css|js)$") {
    unset beresp.http.set-cookie;
  }
}
//end =========


The first example (as suggested by Per below) removes the cookie from
the incoming requests for these files so that these objects are
cacheable.

What I'm a bit unclear on is why the 2nd example uses 'unset' in both
vcl_recv and also when the object has been retrieved from the backend
(vcl_fetch).

I need to be 100% sure that for all image/css objects, there are no
cookies stored anywhere. Is this what the first example does, whereas
the second example 'ignores' the cookie?

Thanks for any help.

Best, David


On Wed, Feb 2, 2011 at 9:49 AM, Per Buer <perbu at varnish-software.com> wrote:
> David,
>
> On Wed, Feb 2, 2011 at 10:14 AM, David Murphy <david at firechaser.com> wrote:
>> Morning Per
>>
>> So the default behaviour of Varnish (only extended if I start writing
>> my own .vcl) would make a smart choice about what should be cached?
>> That sounds good. So provided that I strip the cookies for the image
>> file types, then Varnish will spot that these request do not have any
>> cookies and will safely cache them, ignoring all other requests (since
>> I've not stripped their cookies).
>
> Right.
>
>> Something as simple as
>>
>> sub vcl_recv {
>> ? if (req.url ~ "\.(png|gif|jpg)$") {
>> ? ? ? ?remove req.http.Cookie;
>> ? ?}
>> }
>
> Right. Provided your backend produces sane cache-control headers. If
> there is no information on how long to cache it Varnish will cache for
> a default period of 120 seconds. You can override the TTL -
> http://www.varnish-cache.org/docs/trunk/tutorial/increasing_your_hitrate.html#overriding-the-time-to-live-ttl
>
>> ...and nothing else in .vcl file?
>
> No. That should be sufficient.
>
>
> --
> Per Buer,?Varnish Software
> Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer
> Varnish makes websites fly!
> Want to learn more about Varnish? http://www.varnish-software.com/whitepapers
>



From ruben at varnish-software.com  Wed Feb  9 17:12:10 2011
From: ruben at varnish-software.com (=?ISO-8859-1?Q?Rub=E9n_Romero?=)
Date: Wed, 9 Feb 2011 18:12:10 +0100
Subject: Reminder - VUG3 in Amsterdam - February 14th-15th 2011
Message-ID: <AANLkTinTSLztADv6UVKLngsfrvVmQ0kPNBsXwhdDKzKY@mail.gmail.com>

Hei,

Just sending a reminder for the Varnish User Meeting in Amsterdam next week.

Details here: http://varnish-cache.org/trac/wiki/VUG3

A big thanks go to TomTom that is hosting the event!

All the best,
-
Rub?n Romero
Varnish Software
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110209/1aeab97e/attachment.html>

From lin.support at gmail.com  Wed Feb  9 19:28:01 2011
From: lin.support at gmail.com (linuxsupport)
Date: Thu, 10 Feb 2011 00:58:01 +0530
Subject: Varnish 2.1.4 issue with ISA proxy
Message-ID: <AANLkTi=U=-+WSxL4zhG58qToipXUFWb1O=9veXvPfTJm@mail.gmail.com>

Hi,

I have varnish 2.1.4 running for last 3 months, everything was working well,
suddenly I got complaints from multiple users that they can not open the
sites.

users are using ISA porxy to reach to internet, it seems issue when ISA and
Varnish both are used, users made some test and reported as below.

1. If they do not use ISA proxy sites open perfectly (bypass ISA proxy)
2. If they do not use Varnish sites works well (bypass Varnish)

On firefox half of the page open, and on Chrome it says duplicate
content-length error.

I am not sure why this happens when both ISA and Varnish are used.

Please let me know if anyone else encountered the same issue.

Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110210/1c7b8d23/attachment.html>

From a.hongens at netmatch.nl  Wed Feb  9 19:36:25 2011
From: a.hongens at netmatch.nl (=?UTF-8?B?QW5nZWxvIEjDtm5nZW5z?=)
Date: Wed, 09 Feb 2011 20:36:25 +0100
Subject: Varnish 2.1.4 issue with ISA proxy
In-Reply-To: <AANLkTi=U=-+WSxL4zhG58qToipXUFWb1O=9veXvPfTJm@mail.gmail.com>
References: <AANLkTi=U=-+WSxL4zhG58qToipXUFWb1O=9veXvPfTJm@mail.gmail.com>
Message-ID: <4D52ECB9.7080702@netmatch.nl>

On 9-2-2011 20:28, linuxsupport wrote:
> Hi,
> 
> I have varnish 2.1.4 running for last 3 months, everything was working
> well, suddenly I got complaints from multiple users that they can not
> open the sites.
> 
> users are using ISA porxy to reach to internet, it seems issue when ISA
> and Varnish both are used, users made some test and reported as below.
> 
> 1. If they do not use ISA proxy sites open perfectly (bypass ISA proxy)
> 2. If they do not use Varnish sites works well (bypass Varnish)
> 
> On firefox half of the page open, and on Chrome it says duplicate
> content-length error.
> 
> I am not sure why this happens when both ISA and Varnish are used.
> 
> Please let me know if anyone else encountered the same issue.

I saw some posts a while back on this list about a bug in varnish that
caused it to send (or pass?) duplicate content-length headers in the
response. For most proxies and/or browsers it's not a problem, but
apparently ISA has a problem with it.

Search on the list, and you will find more details about it.

-- 


With kind regards,


Angelo H?ngens
systems administrator

MCSE on Windows 2003
MCSE on Windows 2000
MS Small Business Specialist
------------------------------------------
NetMatch
tourism internet software solutions

Ringbaan Oost 2b
5013 CA Tilburg
+31 (0)13 5811088
+31 (0)13 5821239

A.Hongens at netmatch.nl
www.netmatch.nl
------------------------------------------





From TFigueiro at au.westfield.com  Wed Feb  9 21:55:19 2011
From: TFigueiro at au.westfield.com (Thiago Figueiro)
Date: Thu, 10 Feb 2011 08:55:19 +1100
Subject: How to find out a single failing backend from a round-robin
	director?(Inline-C)
In-Reply-To: <4D5270A4.5040104@dmsp.de>
References: <4D5270A4.5040104@dmsp.de>
Message-ID: <64E73E81AAC26A49AC9EA28CBE65365107EF8CA7@AUSYDEVS01.au.ad.westfield.com>

(sorry for top-posting)

 

I believe varnishlog has all you need:

 

   54 BackendOpen  b backend_name source_ip source_port dest_ip dest_port

 

Just track the id on the left (54 in this case) for the request and response and you should be able to find the culprit.  

 

 

From: varnish-misc-bounces at varnish-cache.org [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Stefan Pommerening
Sent: Wednesday, 9 February 2011 9:47 PM
To: varnish-misc at varnish-cache.org
Subject: How to find out a single failing backend from a round-robin director?(Inline-C)

 

Hi all,

I'm working with a set of varnish-caches (2.1.3) in front of some round-robin director configured backends.
Sometimes I experience a 503 error due to some unforseeable circumstances like network hiccups, content server problems and other magic...  ;-)
Nevertheless all backends are healthy regarding the large majority of other requests (and watchdog of course), we deal with sporadic failures here.

Obviously I don't want those 503's to go through to the user.

I inserted the following code in my vcl_fetch:

if (beresp.status == 503) {
   set beresp.saintmode = 2s;
   restart;
}

This works quite well, the failed (503 delivering) request is forwarded to the next backend in the director and is successfully served then.

What I am missing is the exact backend information which backend did not serve the request or better said: where did the 503 come from?
I already included some inline-C in order to send info to syslog, but I am missing some VRT_(r_)something to find out the failing backend without searching the backend's logfiles directly.

My current syslog()-msg is:

syslog (LOG_LOCAL7 | LOG_INFO, "VarnishLog(vcl_fetch): %s/%s[%s] [REST=%02d] %s http://%s%s <http://%25s%25s> ", 
   VRT_backend_string(sp), VRT_r_server_hostname(sp), VRT_IP_string(sp,VRT_r_server_ip(sp)), 
   VRT_r_req_restarts(sp), VRT_r_req_request(sp), VRT_GetHdr(sp, HDR_REQ, "\005host:"), VRT_r_req_url(sp));

I managed to get the backend definition, but not the specific backend machine name (or IP) from the backend director.

... and: is there some sort of overview on which VRT_magic fields (for use with inline-C) exists?

Thanks in advance,
Stefan

-- 

Dipl.-Inform. Stefan Pommerening
Informatik-B?ro: IT-Dienste & Projekte, Consulting & Coaching
http://www.dmsp.de

 


______________________________________________________
    CONFIDENTIALITY NOTICE    
This electronic mail message, including any and/or all attachments, is for the sole use of the intended recipient(s), and may contain confidential and/or privileged information, pertaining to business conducted under the direction and supervision of the sending organization. All electronic mail messages, which may have been established as expressed views and/or opinions (stated either within the electronic mail message or any of its attachments), are left to the sole responsibility of that of the sender, and are not necessarily attributed to the sending organization. Unauthorized interception, review, use, disclosure or distribution of any such information contained within this electronic mail message and/or its attachment(s), is (are) strictly prohibited. If you are not the intended recipient, please contact the sender by replying to this electronic mail message, along with the destruction all copies of the original electronic mail message (along with any attachments).
______________________________________________________
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110210/66e4720c/attachment.html>

From pom at dmsp.de  Thu Feb 10 09:33:39 2011
From: pom at dmsp.de (Stefan Pommerening)
Date: Thu, 10 Feb 2011 10:33:39 +0100
Subject: How to find out a single failing backend from a round-robin
	director?(Inline-C)
In-Reply-To: <64E73E81AAC26A49AC9EA28CBE65365107EF8CA7@AUSYDEVS01.au.ad.westfield.com>
References: <4D5270A4.5040104@dmsp.de>
	<64E73E81AAC26A49AC9EA28CBE65365107EF8CA7@AUSYDEVS01.au.ad.westfield.com>
Message-ID: <4D53B0F3.2060805@dmsp.de>

Hi Thiago,

Am 09.02.2011 22:55, schrieb Thiago Figueiro:

 > I believe varnishlog has all you need:
 >
 >    54 BackendOpen  b backend_name source_ip source_port dest_ip dest_port
 >
 > Just track the id on the left (54 in this case) for the request and 
response and you should be able to find the culprit.


That's correct - in my case the following (example):

   417 VCL_call     c recv
   417 VCL_return   c lookup
   417 VCL_call     c hash
   417 VCL_return   c hash
   417 VCL_call     c miss
   417 VCL_return   c fetch
    31 BackendClose - cmsweb5f3cae1
    31 BackendOpen  b cmsweb5f3cae1 10.40.XXX.YYY 37607 10.40.XXX.ZZZ 80
   417 Backend      c 31 group5cae1 cmsweb5f3cae1
    31 TxRequest    b GET


Unfortunately the inline-C expression VRT_backend_string(sp) gives me 
only the director name, which is 'group5cae1'.

What I need to get in my inline C is the specific backend choosen from 
the director, 'cmsweb5f3cae1' in this example.
How to get this? Any VRT_something for this?

-- stefan


 > From: varnish-misc-bounces at varnish-cache.org 
[mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Stefan 
Pommerening
 > Sent: Wednesday, 9 February 2011 9:47 PM
 > To: varnish-misc at varnish-cache.org
 > Subject: How to find out a single failing backend from a round-robin 
director?(Inline-C)
 >
 >
 >
 > Hi all,
 >
 > I'm working with a set of varnish-caches (2.1.3) in front of some 
round-robin director configured backends.
 > Sometimes I experience a 503 error due to some unforseeable 
circumstances like network hiccups, content server problems and other 
magic...  ;-)
 > Nevertheless all backends are healthy regarding the large majority of 
other requests (and watchdog of course), we deal with sporadic failures 
here.
 >
 > Obviously I don't want those 503's to go through to the user.
 >
 > I inserted the following code in my vcl_fetch:
 >
 > if (beresp.status == 503) {
 >    set beresp.saintmode = 2s;
 >    restart;
 > }
 >
 > This works quite well, the failed (503 delivering) request is 
forwarded to the next backend in the director and is successfully served 
then.
 >
 > What I am missing is the exact backend information which backend did 
not serve the request or better said: where did the 503 come from?
 > I already included some inline-C in order to send info to syslog, but 
I am missing some VRT_(r_)something to find out the failing backend 
without searching the backend's logfiles directly.
 >
 > My current syslog()-msg is:
 >
 > syslog (LOG_LOCAL7 | LOG_INFO, "VarnishLog(vcl_fetch): %s/%s[%s] 
[REST=%02d] %s http://%s%s",
 >    VRT_backend_string(sp), VRT_r_server_hostname(sp), 
VRT_IP_string(sp,VRT_r_server_ip(sp)),
 >    VRT_r_req_restarts(sp), VRT_r_req_request(sp), VRT_GetHdr(sp, 
HDR_REQ, "\005host:"), VRT_r_req_url(sp));
 >
 > I managed to get the backend definition, but not the specific backend 
machine name (or IP) from the backend director.
 >
 > ... and: is there some sort of overview on which VRT_magic fields 
(for use with inline-C) exists?
 >
 > Thanks in advance,
 > Stefan


-- 

Dipl.-Inform. Stefan Pommerening
Informatik-B?ro: IT-Dienste & Projekte, Consulting & Coaching
http://www.dmsp.de





From l at lrowe.co.uk  Thu Feb 10 13:46:40 2011
From: l at lrowe.co.uk (Laurence Rowe)
Date: Thu, 10 Feb 2011 13:46:40 +0000
Subject: Dual server Varnish setup
In-Reply-To: <0E7D9950-E2A1-4B1C-B619-A0CDE528C8B1@comicrelief.com>
References: <0E7D9950-E2A1-4B1C-B619-A0CDE528C8B1@comicrelief.com>
Message-ID: <AANLkTikWwqzV2Lo0a9DdndrKA2V9ut2YwVPrqzq_xROQ@mail.gmail.com>

On 8 February 2011 23:05, Ben Dodd <B.Dodd at comicrelief.com> wrote:
> Given this functionality, does anyone know what the VCL is required to implement this dual server Varnish setup scenario?
> http://varnish-cache.org/trac/wiki/VCLExampleHashIgnoreBusy

Something like this should work:


backend appserver { ... }
backend other_varnish { ... }

# Server 1
sub vcl_recv {
    set req.backend = other_varnish;
    if (req.http.X-Via-Varnish || !req.backend.healthy) {
        # Request is coming from the other server.
        # Ignore busy objects to avoid race condition.
        set req.hash_ignore_busy = 1;
        set req.backend = appserver;
    } else {
        set req.http.X-Via-Varnish = "true";
    }
}

# Server 2
sub vcl_recv {
    set req.backend = other_varnish;
    if (req.http.X-Via-Varnish || !req.backend.healthy) {
        # Request is coming from the other server.
        set req.backend = appserver;
    } else {
        set req.http.X-Via-Varnish = "true";
    }
}

Laurence



From B.Dodd at comicrelief.com  Thu Feb 10 16:00:21 2011
From: B.Dodd at comicrelief.com (Ben Dodd)
Date: Thu, 10 Feb 2011 16:00:21 +0000
Subject: Dual server Varnish setup
In-Reply-To: <4D53F2C5.4090502@bizztravel.nl>
References: <0E7D9950-E2A1-4B1C-B619-A0CDE528C8B1@comicrelief.com>
	<AANLkTikWwqzV2Lo0a9DdndrKA2V9ut2YwVPrqzq_xROQ@mail.gmail.com>
	<4D53F2C5.4090502@bizztravel.nl>
Message-ID: <A5983459-1EAC-43A2-953F-DD3A3D95478D@comicrelief.com>

As I have 8 backends, and potentially 4 varnish servers, I was going to group them using directors...is this valid? Or is it best to have them all in 1 large director? Any ideas how this would look?

Thanks for all your help, Ben

On 10 Feb 2011, at 14:14, Martin Boer wrote:

> I would also add weights so that each varnish prefers to ask its sibling
> say 250 times more often than asking the app-server.
> This will ensure that both varnishes will be as filled and as equal as
> possible.
> I don't know if 250 could be replaced by a larger number but this works
> for me.
>
> Regards,
> Martin
>
> On 02/10/2011 02:46 PM, Laurence Rowe wrote:
>> On 8 February 2011 23:05, Ben Dodd<B.Dodd at comicrelief.com>  wrote:
>>> Given this functionality, does anyone know what the VCL is required to implement this dual server Varnish setup scenario?
>>> http://varnish-cache.org/trac/wiki/VCLExampleHashIgnoreBusy
>> Something like this should work:
>>
>>
>> backend appserver { ... }
>> backend other_varnish { ... }
>>
>> # Server 1
>> sub vcl_recv {
>>     set req.backend = other_varnish;
>>     if (req.http.X-Via-Varnish || !req.backend.healthy) {
>>         # Request is coming from the other server.
>>         # Ignore busy objects to avoid race condition.
>>         set req.hash_ignore_busy = 1;
>>         set req.backend = appserver;
>>     } else {
>>         set req.http.X-Via-Varnish = "true";
>>     }
>> }
>>
>> # Server 2
>> sub vcl_recv {
>>     set req.backend = other_varnish;
>>     if (req.http.X-Via-Varnish || !req.backend.healthy) {
>>         # Request is coming from the other server.
>>         set req.backend = appserver;
>>     } else {
>>         set req.http.X-Via-Varnish = "true";
>>     }
>> }
>>
>> Laurence
>>
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at varnish-cache.org
>> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>
>>
>
>
> ______________________________________________________________________
> This email has been scanned by the MessageLabs Email Security System.
> For more information please visit http://www.messagelabs.com/email
> ______________________________________________________________________


Comic Relief  1st Floor  89 Albert Embankment  London SE1 7TP  Tel: 020 7820 2000  Fax: 020 7820 2222  red at comicrelief.com  www.comicrelief.com

Comic Relief is the operating name of Charity Projects, a company limited by guarantee and registered in England no. 1806414; registered charity 326568 (England & Wales) and SC039730 (Scotland). Comic Relief Ltd is a subsidiary of Charity Projects and registered in England no. 1967154. Registered offices: Hanover House, 14 Hanover Square, London W1S 1HP. VAT no. 773865187.

This email (and any attachment) may contain confidential and/or privileged information. If you are not the intended addressee, you must not use, disclose, copy or rely on anything in this email and should contact the sender and delete it immediately. The views of the author are not necessarily those of Comic Relief. We cannot guarantee that this email (and any attachment) is virus free or has not been intercepted and amended, so do not accept liability for any damage resulting from software viruses. You should carry out your own virus checks.



From B.Dodd at comicrelief.com  Thu Feb 10 16:37:52 2011
From: B.Dodd at comicrelief.com (Ben Dodd)
Date: Thu, 10 Feb 2011 16:37:52 +0000
Subject: Dual server Varnish setup
In-Reply-To: <4D53F2C5.4090502@bizztravel.nl>
References: <0E7D9950-E2A1-4B1C-B619-A0CDE528C8B1@comicrelief.com>
	<AANLkTikWwqzV2Lo0a9DdndrKA2V9ut2YwVPrqzq_xROQ@mail.gmail.com>
	<4D53F2C5.4090502@bizztravel.nl>
Message-ID: <7D4E01A8-3273-4C61-B72B-E3DAF0C0F8A4@comicrelief.com>

Something like this:

director cr-backends round-robin {
  .retries = 5;

  {
    .backend...
  }

  # Repeat the above entry for each web server.
}

director cr-varnish round-robin {
  .retries = 4;

  {
    .backend...
  }

  # Repeat the above entry for each varnish server.
}

sub vcl_recv {

  set req.backend = cr-varnish;

  if (req.http.X-Via-Varnish || req.restarts > 3) {
    # Request is coming from the other server.
    # Ignore busy objects to avoid race condition.
    set req.hash_ignore_busy = 1;
    set req.backend = cr-backends;
  } else {
    set req.http.X-Via-Varnish = "true";
  }
}

Thanks, Ben

On 10 Feb 2011, at 14:14, Martin Boer wrote:

> I would also add weights so that each varnish prefers to ask its sibling
> say 250 times more often than asking the app-server.
> This will ensure that both varnishes will be as filled and as equal as
> possible.
> I don't know if 250 could be replaced by a larger number but this works
> for me.
>
> Regards,
> Martin
>
> On 02/10/2011 02:46 PM, Laurence Rowe wrote:
>> On 8 February 2011 23:05, Ben Dodd<B.Dodd at comicrelief.com>  wrote:
>>> Given this functionality, does anyone know what the VCL is required to implement this dual server Varnish setup scenario?
>>> http://varnish-cache.org/trac/wiki/VCLExampleHashIgnoreBusy
>> Something like this should work:
>>
>>
>> backend appserver { ... }
>> backend other_varnish { ... }
>>
>> # Server 1
>> sub vcl_recv {
>>     set req.backend = other_varnish;
>>     if (req.http.X-Via-Varnish || !req.backend.healthy) {
>>         # Request is coming from the other server.
>>         # Ignore busy objects to avoid race condition.
>>         set req.hash_ignore_busy = 1;
>>         set req.backend = appserver;
>>     } else {
>>         set req.http.X-Via-Varnish = "true";
>>     }
>> }
>>
>> # Server 2
>> sub vcl_recv {
>>     set req.backend = other_varnish;
>>     if (req.http.X-Via-Varnish || !req.backend.healthy) {
>>         # Request is coming from the other server.
>>         set req.backend = appserver;
>>     } else {
>>         set req.http.X-Via-Varnish = "true";
>>     }
>> }
>>
>> Laurence
>>
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at varnish-cache.org
>> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>
>>
>
>
> ______________________________________________________________________
> This email has been scanned by the MessageLabs Email Security System.
> For more information please visit http://www.messagelabs.com/email
> ______________________________________________________________________


Comic Relief  1st Floor  89 Albert Embankment  London SE1 7TP  Tel: 020 7820 2000  Fax: 020 7820 2222  red at comicrelief.com  www.comicrelief.com

Comic Relief is the operating name of Charity Projects, a company limited by guarantee and registered in England no. 1806414; registered charity 326568 (England & Wales) and SC039730 (Scotland). Comic Relief Ltd is a subsidiary of Charity Projects and registered in England no. 1967154. Registered offices: Hanover House, 14 Hanover Square, London W1S 1HP. VAT no. 773865187.

This email (and any attachment) may contain confidential and/or privileged information. If you are not the intended addressee, you must not use, disclose, copy or rely on anything in this email and should contact the sender and delete it immediately. The views of the author are not necessarily those of Comic Relief. We cannot guarantee that this email (and any attachment) is virus free or has not been intercepted and amended, so do not accept liability for any damage resulting from software viruses. You should carry out your own virus checks.



From l at lrowe.co.uk  Thu Feb 10 17:01:11 2011
From: l at lrowe.co.uk (Laurence Rowe)
Date: Thu, 10 Feb 2011 17:01:11 +0000
Subject: Dual server Varnish setup
In-Reply-To: <7D4E01A8-3273-4C61-B72B-E3DAF0C0F8A4@comicrelief.com>
References: <0E7D9950-E2A1-4B1C-B619-A0CDE528C8B1@comicrelief.com>
	<AANLkTikWwqzV2Lo0a9DdndrKA2V9ut2YwVPrqzq_xROQ@mail.gmail.com>
	<4D53F2C5.4090502@bizztravel.nl>
	<7D4E01A8-3273-4C61-B72B-E3DAF0C0F8A4@comicrelief.com>
Message-ID: <AANLkTi=LSHkxWFGPK77zkDcr-=R90db67v0Xa3jVQULv@mail.gmail.com>

You'd be better off using the random director for cr-backends as each
varnish defines the director.

For the varnishes, you probably don't want to use a director, but
check the next varnish in the ring. Requests to another varnish should
be much cheaper than going to the backend, and you want to fill up
each varnish as you go around to minimise backend requests.

sub vcl_recv {
   set req.backend = next_varnish;
   # Keep a unary count of how many varnishes this request has been through
   if (!req.http.X-Via-Varnish) {
       set req.http.X-Via-Varnish = "1";
   } else {
       set req.http.X-Via-Varnish = req.http.X-Via-Varnish "1";
   }
   if (req.http.X-Via-Varnish ~ "1111" || !req.backend.healthy) {
       # Request is coming from the other server.
       set req.backend = cr-backends;
   } else {
       set req.http.X-Via-Varnish = "true";
   }
}

This is more complex than a pair though, so not sure what you want to
do about the hash_ignore_busy here.

Laurence

On 10 February 2011 16:37, Ben Dodd <B.Dodd at comicrelief.com> wrote:
> Something like this:
>
> director cr-backends round-robin {
> ?.retries = 5;
>
> ?{
> ? ?.backend...
> ?}
>
> ?# Repeat the above entry for each web server.
> }
>
> director cr-varnish round-robin {
> ?.retries = 4;
>
> ?{
> ? ?.backend...
> ?}
>
> ?# Repeat the above entry for each varnish server.
> }
>
> sub vcl_recv {
>
> ?set req.backend = cr-varnish;
>
> ?if (req.http.X-Via-Varnish || req.restarts > 3) {
> ? ?# Request is coming from the other server.
> ? ?# Ignore busy objects to avoid race condition.
> ? ?set req.hash_ignore_busy = 1;
> ? ?set req.backend = cr-backends;
> ?} else {
> ? ?set req.http.X-Via-Varnish = "true";
> ?}
> }
>
> Thanks, Ben
>
> On 10 Feb 2011, at 14:14, Martin Boer wrote:
>
>> I would also add weights so that each varnish prefers to ask its sibling
>> say 250 times more often than asking the app-server.
>> This will ensure that both varnishes will be as filled and as equal as
>> possible.
>> I don't know if 250 could be replaced by a larger number but this works
>> for me.
>>
>> Regards,
>> Martin
>>
>> On 02/10/2011 02:46 PM, Laurence Rowe wrote:
>>> On 8 February 2011 23:05, Ben Dodd<B.Dodd at comicrelief.com> ?wrote:
>>>> Given this functionality, does anyone know what the VCL is required to implement this dual server Varnish setup scenario?
>>>> http://varnish-cache.org/trac/wiki/VCLExampleHashIgnoreBusy
>>> Something like this should work:
>>>
>>>
>>> backend appserver { ... }
>>> backend other_varnish { ... }
>>>
>>> # Server 1
>>> sub vcl_recv {
>>> ? ? set req.backend = other_varnish;
>>> ? ? if (req.http.X-Via-Varnish || !req.backend.healthy) {
>>> ? ? ? ? # Request is coming from the other server.
>>> ? ? ? ? # Ignore busy objects to avoid race condition.
>>> ? ? ? ? set req.hash_ignore_busy = 1;
>>> ? ? ? ? set req.backend = appserver;
>>> ? ? } else {
>>> ? ? ? ? set req.http.X-Via-Varnish = "true";
>>> ? ? }
>>> }
>>>
>>> # Server 2
>>> sub vcl_recv {
>>> ? ? set req.backend = other_varnish;
>>> ? ? if (req.http.X-Via-Varnish || !req.backend.healthy) {
>>> ? ? ? ? # Request is coming from the other server.
>>> ? ? ? ? set req.backend = appserver;
>>> ? ? } else {
>>> ? ? ? ? set req.http.X-Via-Varnish = "true";
>>> ? ? }
>>> }
>>>
>>> Laurence
>>>
>>> _______________________________________________
>>> varnish-misc mailing list
>>> varnish-misc at varnish-cache.org
>>> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>>
>>>
>>
>>
>> ______________________________________________________________________
>> This email has been scanned by the MessageLabs Email Security System.
>> For more information please visit http://www.messagelabs.com/email
>> ______________________________________________________________________
>
>
> Comic Relief ?1st Floor ?89 Albert Embankment ?London SE1 7TP ?Tel: 020 7820 2000 ?Fax: 020 7820 2222 ?red at comicrelief.com ?www.comicrelief.com
>
> Comic Relief is the operating name of Charity Projects, a company limited by guarantee and registered in England no. 1806414; registered charity 326568 (England & Wales) and SC039730 (Scotland). Comic Relief Ltd is a subsidiary of Charity Projects and registered in England no. 1967154. Registered offices: Hanover House, 14 Hanover Square, London W1S 1HP. VAT no. 773865187.
>
> This email (and any attachment) may contain confidential and/or privileged information. If you are not the intended addressee, you must not use, disclose, copy or rely on anything in this email and should contact the sender and delete it immediately. The views of the author are not necessarily those of Comic Relief. We cannot guarantee that this email (and any attachment) is virus free or has not been intercepted and amended, so do not accept liability for any damage resulting from software viruses. You should carry out your own virus checks.
>



From johnson at nmr.mgh.harvard.edu  Thu Feb 10 18:03:57 2011
From: johnson at nmr.mgh.harvard.edu (Chris Johnson)
Date: Thu, 10 Feb 2011 13:03:57 -0500 (EST)
Subject: Changing the fetch for virtual hosts.
In-Reply-To: <AANLkTikJe_CV4VocFi1wEwGd6Y-736cWbL3hUmy3RKOG@mail.gmail.com>
References: <AANLkTikbYsNzsUR95tWz3Qqa5z7Bvxc20SUs6xqenfvn@mail.gmail.com>
	<alpine.LRH.2.00.1102031529190.10090@maequeses.nmr.mgh.harvard.edu>
	<AANLkTi=+dWrYH7e9z37yeCw7kp+qk04Vt9CxhOxERhoQ@mail.gmail.com>
	<alpine.LRH.2.00.1102040907030.3092@maequeses.nmr.mgh.harvard.edu>
	<AANLkTinbZ8bv5gAe4w3J3VoV=UVfxsNw43LxfugxENuF@mail.gmail.com>
	<alpine.LRH.2.00.1102041113330.3092@maequeses.nmr.mgh.harvard.edu>
	<AANLkTikCtPK0pJarEh+_VBbL+Pz89byQfETGV6+Y+YQr@mail.gmail.com>
	<alpine.LRH.2.00.1102070838580.556@maequeses.nmr.mgh.harvard.edu>
	<AANLkTikJe_CV4VocFi1wEwGd6Y-736cWbL3hUmy3RKOG@mail.gmail.com>
Message-ID: <alpine.LRH.2.00.1102101259080.15712@maequeses.nmr.mgh.harvard.edu>

      I'm running 2.0.5.  Where would I look in the varnishlog for
TCP_HIT?  I never see anything like that in here.  And I know it's
caching because moinmoin is a time pig and the fetchs don't show in
access_log.

> ?Run varnishlog against the request and confirm a TCP_HIT when making the
> second request to the different hostname. ?Also confirm you do not see the
> request hit your backend webserver logfile
> 
> 
> On Mon, Feb 7, 2011 at 8:49 AM, Chris Johnson <johnson at nmr.mgh.harvard.edu>
> wrote:
>       ? ? Huh. ?Well, I think I've done everything everyone said was
>       right.
>       Here's a snippet of what I have in varnish's default.vcl;
>
>       backend freesurfer {
>       ? ? ?.host = "127.0.0.1";
>       ? ? ?.port = "8080";
>       }
>
>       which happens to be the same place the default backend points to
>       because this is a different virtual host on the same server.
>
>       And in vcl_recv I have;
>
>       ? ?if (req.http.host ~ "^sufer.nmr.mgh.harvard.edu") {
>       ? ? ?set req.http.host = "www.freesurfer.net";
>       ? ? ?set req.backend = freesurfer;
>       ? ?}
>
>       which referenes my backend declaration and rewrites the header
>       host
>       name.
>
>       ? ? How do I tell if this is working? ?Remember, I'm trying to
>       make
>       the old virtual host references to sufer.nmr.mgh.harvard.edu be
>       keyed
>       on and fetch from the new virtual host www.freesurfer.net so
>       varnish
>       doesn't cache everything twice.
>
>       ? ? Varnich didn't complain when I restarted it and seems to be
>       caching. ?I'm just not sure what at this point.
>
>             On Fri, Feb 4, 2011 at 5:17 PM, Chris Johnson
>             <johnson at nmr.mgh.harvard.edu> wrote:
>                   On Fri, 4 Feb 2011, Per Buer wrote:
>
>                         ? ? ?if (req.http.host ~
>                         "^www.example.com") {
>                         ? ? ? ?set req.backend =
>                         foo;
>                         ? ? ?}
> 
>
>                   ? ? Which goes in vcl_recv and in his
>                   case foo is a backend declaration
>                   function that sets up .host and .port
>                   for the virtual host I want, yes?
> 
>
>             Almost. foo is here a declared backend - declared
>             with the keyword
>             "backend" -
>             http://www.varnish-cache.org/docs/trunk/tutorial/backend_servers.html
> 
> 
> 
> 
>
>             --
>             Per Buer,?Varnish Software
>             Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 /
>             Skype: per.buer
>             Varnish makes websites fly!
>             Want to learn more about Varnish?
>             http://www.varnish-software.com/whitepapers
> 
> 
> 
> 
> ---------------------------------------------------------------------------
> ----
> Chris Johnson ? ? ? ? ? ? ? |Internet: johnson at nmr.mgh.harvard.edu
> Systems Administrator ? ? ? |Web: ? ?
> ?http://www.nmr.mgh.harvard.edu/~johnson
> NMR Center ? ? ? ? ? ? ? ? ?|Voice: ? ?617.726.0949
> Mass. General Hospital ? ? ?|FAX: ? ? ?617.726.7422
> 149 (2301) 13th Street ? ? ?|Man's a kind of missing link
> Charlestown, MA., 02129 USA |fondly thinking he can think. ?Piet Hein
> 
> ---------------------------------------------------------------------------
> ----
> 
> 
> The information in this e-mail is intended only for the person to whom
> it is
> addressed. If you believe this e-mail was sent to you in error and the
> e-mail
> contains patient information, please contact the Partners Compliance
> HelpLine at
> http://www.partners.org/complianceline . If the e-mail was sent to you
> in error
> but does not contain patient information, please contact the sender
> and properly
> dispose of the e-mail.
> 
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
> 
> 
> 
>

-------------------------------------------------------------------------------
Chris Johnson               |Internet: johnson at nmr.mgh.harvard.edu
Systems Administrator       |Web:      http://www.nmr.mgh.harvard.edu/~johnson
NMR Center                  |Voice:    617.726.0949
Mass. General Hospital      |FAX:      617.726.7422
149 (2301) 13th Street      |Life n, something from which nobody escapes
Charlestown, MA., 02129 USA |alive.  Me
-------------------------------------------------------------------------------

From kongfranon at gmail.com  Thu Feb 10 20:50:42 2011
From: kongfranon at gmail.com (Mike Franon)
Date: Thu, 10 Feb 2011 15:50:42 -0500
Subject: varnish 2.15 - default set to cache nothing except two urls?
Message-ID: <AANLkTimh8ej1q1=Nmq1SGgh5yT_onqGod8-dNRP6StBY@mail.gmail.com>

HI,

I am fairly new to varnish, and was curious, is there a way to have all
traffic go through varnish to the backend without caching unless for
specific pages.  For example we only want two pages to be cached, our home,
and a specific page under the home as seen below?

Here is my default.vcl

Thanks



backend default {
  .host = "127.0.0.1";
  .port = "80";
}


sub vcl_recv {

    # for the default home page
    if (req.url ~ "^/") {
        unset req.http.cookie;
    }


   # if they go directly to /samplepage
    if (req.url ~ "^/samplepage") {
        unset req.http.cookie;
    }

}

sub vcl_fetch {
    if (req.url ~ "^/") {
        unset beresp.http.set-cookie;
    }

    if (req.url ~ "^/samplepage") {
        unset beresp.http.set-cookie;
    }


    #override cache headers from backend to keep element in cache for 5
minutes
    if (beresp.ttl < 120s) {
       set beresp.ttl = 360s;
  }

}
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110210/2a9caac5/attachment.html>

From scaunter at topscms.com  Thu Feb 10 21:16:27 2011
From: scaunter at topscms.com (Caunter, Stefan)
Date: Thu, 10 Feb 2011 16:16:27 -0500
Subject: varnish 2.15 - default set to cache nothing except two urls?
In-Reply-To: <AANLkTimh8ej1q1=Nmq1SGgh5yT_onqGod8-dNRP6StBY@mail.gmail.com>
References: <AANLkTimh8ej1q1=Nmq1SGgh5yT_onqGod8-dNRP6StBY@mail.gmail.com>
Message-ID: <7F0AA702B8A85A4A967C4C8EBAD6902CEB689D@TMG-EVS02.torstar.net>

You need in vcl_recv:

 

if (req.url ~ "/blah"){

                unset req.http.cookie;

                return(lookup);

}

else {

                return(pass);

}

 

Setting 360s for beresp.ttl gets you 6 minutes J

 

Resetting it to 360 if it goes under 120 seems odd.

 

Use something like this in vcl_fetch:

 

if (req.url ~ "/blah"){

                set beresp.ttl = 300s;

                unset req.http.set-cookie;

                set beresp.http.cache-control = "public, max-age = 300";

                set beresp.http.X-CacheReason = "Blah";

                return(deliver);

}

 

Stefan Caunter 

Operations

Torstar Digital

m: (416) 561-4871

 

 

From: varnish-misc-bounces at varnish-cache.org
[mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Mike Franon
Sent: February-10-11 3:51 PM
To: varnish-misc at varnish-cache.org
Subject: varnish 2.15 - default set to cache nothing except two urls?

 

HI,

 

I am fairly new to varnish, and was curious, is there a way to have all
traffic go through varnish to the backend without caching unless for
specific pages.  For example we only want two pages to be cached, our
home, and a specific page under the home as seen below?

 

Here is my default.vcl 

 

Thanks

 

 

 

backend default {

  .host = "127.0.0.1";   

  .port = "80";

}

 

 

sub vcl_recv {

 

    # for the default home page

    if (req.url ~ "^/") {

        unset req.http.cookie;

    }

 

    

   # if they go directly to /samplepage

    if (req.url ~ "^/samplepage") {

        unset req.http.cookie;

    }

 

}

 

sub vcl_fetch {

    if (req.url ~ "^/") {

        unset beresp.http.set-cookie;

    }

    

    if (req.url ~ "^/samplepage") {

        unset beresp.http.set-cookie;

    }

    

    

    #override cache headers from backend to keep element in cache for 5
minutes

    if (beresp.ttl < 120s) {

       set beresp.ttl = 360s;

  }

 

}

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110210/760e08b9/attachment.html>

From kongfranon at gmail.com  Thu Feb 10 21:50:30 2011
From: kongfranon at gmail.com (Mike Franon)
Date: Thu, 10 Feb 2011 16:50:30 -0500
Subject: varnish 2.15 - default set to cache nothing except two urls?
In-Reply-To: <7F0AA702B8A85A4A967C4C8EBAD6902CEB689D@TMG-EVS02.torstar.net>
References: <AANLkTimh8ej1q1=Nmq1SGgh5yT_onqGod8-dNRP6StBY@mail.gmail.com>
	<7F0AA702B8A85A4A967C4C8EBAD6902CEB689D@TMG-EVS02.torstar.net>
Message-ID: <AANLkTikrit_x7BxvWeOiMHWNmLNPtYFpLPF6Z-TwXDnj@mail.gmail.com>

Thank you for the help!

Quick question:

for this block

if (req.url ~ "/blah"){

                set beresp.ttl = 300s;

                *unset req.http.set-cookie;*

                set beresp.http.cache-control = "public, max-age = 300";

                set beresp.http.X-CacheReason = "Blah";

                return(deliver);

}



I thought it was unset beresp.http.set-cookie;


Is there a difference?



Thanks again.


On Thu, Feb 10, 2011 at 4:16 PM, Caunter, Stefan <scaunter at topscms.com>wrote:

> You need in vcl_recv:
>
>
>
> if (req.url ~ ?/blah?){
>
>                 unset req.http.cookie;
>
>                 return(lookup);
>
> }
>
> else {
>
>                 return(pass);
>
> }
>
>
>
> Setting 360s for beresp.ttl gets you 6 minutes J
>
>
>
> Resetting it to 360 if it goes under 120 seems odd.
>
>
>
> Use something like this in vcl_fetch:
>
>
>
> if (req.url ~ "/blah"){
>
>                 set beresp.ttl = 300s;
>
>                 unset req.http.set-cookie;
>
>                 set beresp.http.cache-control = "public, max-age = 300";
>
>                 set beresp.http.X-CacheReason = "Blah";
>
>                 return(deliver);
>
> }
>
>
>
> Stefan Caunter
>
> Operations
>
> Torstar Digital
>
> m: (416) 561-4871
>
>
>
>
>
> *From:* varnish-misc-bounces at varnish-cache.org [mailto:
> varnish-misc-bounces at varnish-cache.org] *On Behalf Of *Mike Franon
> *Sent:* February-10-11 3:51 PM
> *To:* varnish-misc at varnish-cache.org
> *Subject:* varnish 2.15 - default set to cache nothing except two urls?
>
>
>
> HI,
>
>
>
> I am fairly new to varnish, and was curious, is there a way to have all
> traffic go through varnish to the backend without caching unless for
> specific pages.  For example we only want two pages to be cached, our home,
> and a specific page under the home as seen below?
>
>
>
> Here is my default.vcl
>
>
>
> Thanks
>
>
>
>
>
>
>
> backend default {
>
>   .host = "127.0.0.1";
>
>   .port = "80";
>
> }
>
>
>
>
>
> sub vcl_recv {
>
>
>
>     # for the default home page
>
>     if (req.url ~ "^/") {
>
>         unset req.http.cookie;
>
>     }
>
>
>
>
>
>    # if they go directly to /samplepage
>
>     if (req.url ~ "^/samplepage") {
>
>         unset req.http.cookie;
>
>     }
>
>
>
> }
>
>
>
> sub vcl_fetch {
>
>     if (req.url ~ "^/") {
>
>         unset beresp.http.set-cookie;
>
>     }
>
>
>
>     if (req.url ~ "^/samplepage") {
>
>         unset beresp.http.set-cookie;
>
>     }
>
>
>
>
>
>     #override cache headers from backend to keep element in cache for 5
> minutes
>
>     if (beresp.ttl < 120s) {
>
>        set beresp.ttl = 360s;
>
>   }
>
>
>
> }
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110210/367ee906/attachment.html>

From B.Dodd at comicrelief.com  Thu Feb 10 22:16:54 2011
From: B.Dodd at comicrelief.com (Ben Dodd)
Date: Thu, 10 Feb 2011 22:16:54 +0000
Subject: Tracking requests through Varnish servers
Message-ID: <EE7E8030-CEF7-452C-8E14-F93647C74CBA@comicrelief.com>

Hello,

I'd like to be able to track the progress of a request as it passes through multiple Varnish servers (using each other as backends).

What is the best way to do this? I've tried using +=, but this is not acceptable for strings (failed to compile). Currently I have multiple headers as follows but is there a way of concatenating them into a single string?

set resp.http.X-Cache-v1 = "MISS"
set resp.http.X-Cache-v2 = "HIT"

Thanks, Ben

Comic Relief  1st Floor  89 Albert Embankment  London SE1 7TP  Tel: 020 7820 2000  Fax: 020 7820 2222  red at comicrelief.com  www.comicrelief.com

Comic Relief is the operating name of Charity Projects, a company limited by guarantee and registered in England no. 1806414; registered charity 326568 (England & Wales) and SC039730 (Scotland). Comic Relief Ltd is a subsidiary of Charity Projects and registered in England no. 1967154. Registered offices: Hanover House, 14 Hanover Square, London W1S 1HP. VAT no. 773865187.

This email (and any attachment) may contain confidential and/or privileged information. If you are not the intended addressee, you must not use, disclose, copy or rely on anything in this email and should contact the sender and delete it immediately. The views of the author are not necessarily those of Comic Relief. We cannot guarantee that this email (and any attachment) is virus free or has not been intercepted and amended, so do not accept liability for any damage resulting from software viruses. You should carry out your own virus checks.



From samcrawford at gmail.com  Thu Feb 10 22:23:05 2011
From: samcrawford at gmail.com (samcrawford at gmail.com)
Date: Thu, 10 Feb 2011 22:23:05 +0000
Subject: Tracking requests through Varnish servers
In-Reply-To: <EE7E8030-CEF7-452C-8E14-F93647C74CBA@comicrelief.com>
References: <EE7E8030-CEF7-452C-8E14-F93647C74CBA@comicrelief.com>
Message-ID: <666457628-1297376585-cardhu_decombobulator_blackberry.rim.net-1027443638-@b17.c2.bise7.blackberry>

Hi Ben,

Do a search for Varnish string concatenation, it's (briefly) documented in the VCL guide.

The following should do the trick though:

set resp.http.X-Cache = "MISS " resp.http.X-Cache

Thanks,

Sam

Sent from my BlackBerry? wireless device

-----Original Message-----
From: Ben Dodd <B.Dodd at comicrelief.com>
Sender: varnish-misc-bounces at varnish-cache.org
Date: Thu, 10 Feb 2011 22:16:54 
To: varnish-misc at varnish-cache.org<varnish-misc at varnish-cache.org>
Subject: Tracking requests through Varnish servers

Hello,

I'd like to be able to track the progress of a request as it passes through multiple Varnish servers (using each other as backends).

What is the best way to do this? I've tried using +=, but this is not acceptable for strings (failed to compile). Currently I have multiple headers as follows but is there a way of concatenating them into a single string?

set resp.http.X-Cache-v1 = "MISS"
set resp.http.X-Cache-v2 = "HIT"

Thanks, Ben

Comic Relief  1st Floor  89 Albert Embankment  London SE1 7TP  Tel: 020 7820 2000  Fax: 020 7820 2222  red at comicrelief.com  www.comicrelief.com

Comic Relief is the operating name of Charity Projects, a company limited by guarantee and registered in England no. 1806414; registered charity 326568 (England & Wales) and SC039730 (Scotland). Comic Relief Ltd is a subsidiary of Charity Projects and registered in England no. 1967154. Registered offices: Hanover House, 14 Hanover Square, London W1S 1HP. VAT no. 773865187.

This email (and any attachment) may contain confidential and/or privileged information. If you are not the intended addressee, you must not use, disclose, copy or rely on anything in this email and should contact the sender and delete it immediately. The views of the author are not necessarily those of Comic Relief. We cannot guarantee that this email (and any attachment) is virus free or has not been intercepted and amended, so do not accept liability for any damage resulting from software viruses. You should carry out your own virus checks.

_______________________________________________
varnish-misc mailing list
varnish-misc at varnish-cache.org
http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

From david at firechaser.com  Fri Feb 11 00:48:47 2011
From: david at firechaser.com (David Murphy)
Date: Fri, 11 Feb 2011 01:48:47 +0100
Subject: Do I need to unset cookie in vcl_fetch if there is remove
	req.http.Cookie in vcl_recv?
Message-ID: <AANLkTim6YoJ6uz=XJqZCYSo90vZAQvscA4JFDT=F=9dZ@mail.gmail.com>

Hello

I've been testing removing cookies from images/css/js and am a little
unclear on the difference between:


//start ==========
 sub vcl_recv {
 if (req.url ~ "\.(png|gif|jpg|swf|css|js)$") {
       remove req.http.Cookie;
  }
//end==========


and...


//start =========
sub vcl_recv {
 if (req.url ~ "\.(png|gif|jpg|swf|css|js)$") {
   unset req.http.cookie;
 }
}

sub vcl_fetch {
 if (req.url ~ "\.(png|gif|jpg|swf|css|js)$") {
   unset beresp.http.set-cookie;
 }
}
//end =========


The first example removes the cookie from
the incoming requests for these files so that these objects are
cacheable.

What I'm a bit unclear on is why the 2nd example uses 'unset' in both
vcl_recv and also when the object has been retrieved from the backend
(vcl_fetch).

I need to be 100% sure that for all image/css objects, there are no
cookies stored anywhere. Is this what the first example does, whereas
the second example 'ignores' the cookie?

Thanks for any help.

Best, David



From perbu at varnish-software.com  Fri Feb 11 02:45:07 2011
From: perbu at varnish-software.com (Per Buer)
Date: Fri, 11 Feb 2011 03:45:07 +0100
Subject: Do I need to unset cookie in vcl_fetch if there is remove
	req.http.Cookie in vcl_recv?
In-Reply-To: <AANLkTim6YoJ6uz=XJqZCYSo90vZAQvscA4JFDT=F=9dZ@mail.gmail.com>
References: <AANLkTim6YoJ6uz=XJqZCYSo90vZAQvscA4JFDT=F=9dZ@mail.gmail.com>
Message-ID: <AANLkTimfFbJtqcsKOronPxcEg_L4Ez+Bz84824LtDtuP@mail.gmail.com>

On Fri, Feb 11, 2011 at 1:48 AM, David Murphy <david at firechaser.com> wrote:
> Hello
>
> I've been testing removing cookies from images/css/js and am a little
> unclear on the difference between:
> (..)
> ? ? ? remove req.http.Cookie;
> (..)
> ? unset req.http.cookie;
> (..)

>From "man vcl":

You can remove headers with the remove or unset keywords, which are synonym.

-- 
Per Buer,?Varnish Software
Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer
Varnish makes websites fly!
Want to learn more about Varnish? http://www.varnish-software.com/whitepapers



From varnish at mm.quex.org  Fri Feb 11 04:01:25 2011
From: varnish at mm.quex.org (Michael Alger)
Date: Fri, 11 Feb 2011 12:01:25 +0800
Subject: Do I need to unset cookie in vcl_fetch if there is remove
	req.http.Cookie in vcl_recv?
In-Reply-To: <AANLkTim6YoJ6uz=XJqZCYSo90vZAQvscA4JFDT=F=9dZ@mail.gmail.com>
References: <AANLkTim6YoJ6uz=XJqZCYSo90vZAQvscA4JFDT=F=9dZ@mail.gmail.com>
Message-ID: <20110211040125.GA5644@grum.quex.org>

On Fri, Feb 11, 2011 at 01:48:47AM +0100, David Murphy wrote:
> Hello
> 
> I've been testing removing cookies from images/css/js and am a
> little unclear on the difference between:
> 
> 
> //start ==========
>  sub vcl_recv {
>  if (req.url ~ "\.(png|gif|jpg|swf|css|js)$") {
>        remove req.http.Cookie;
>   }
> //end==========
> 
> 
> and...
> 
> 
> //start =========
> sub vcl_recv {
>  if (req.url ~ "\.(png|gif|jpg|swf|css|js)$") {
>    unset req.http.cookie;
>  }
> }
> 
> sub vcl_fetch {
>  if (req.url ~ "\.(png|gif|jpg|swf|css|js)$") {
>    unset beresp.http.set-cookie;
>  }
> }
> //end =========
> 
> 
> The first example removes the cookie from the incoming requests
> for these files so that these objects are cacheable.

Kind of. I think it's more accurate to say that removing the cookie
from the request allows Varnish to look up the requested URI in its
cache. The default VCL will pass any request that includes a cookie
to the backend, on the assumption that the cookie may matter to the
website (for example, the cookie may indicate the user is logged in
which allows the website to decline to serve resources to users who
aren't logged in).

The above example assumes that any request matching that pattern is
not dependent on cookies, and therefore the cookie can be discarded
from the request altogether.

If you didn't remove the cookie from the client's request, but also
didn't consider the cookie as part of the hash value when trying to
fetch the object from cache, you'd achieve much the same thing. The
difference is that, if the object isn't in cache, when Varnish goes
to fetch it from the backend, that request will include any cookies
that that particular client had sent.

Probably that won't matter, but if the server is actually sending a
different response to different cookies, the cached object will end
up being served to other clients who should have received something
different because they had a different cookie.

If the server's response doesn't vary based on the cookies, then it
doesn't matter if you send an object that was fetched with cookie A
to a client who has cookie B; but it doesn't make any sense to send
the server cookie values it's going to ignore. Stripping the cookie
early in the request gives you a simpler configuration and achieves
the same result.

> What I'm a bit unclear on is why the 2nd example uses 'unset' in
> both vcl_recv and also when the object has been retrieved from the
> backend (vcl_fetch).

The line in vcl_recv removes any cookie that the client may have in
its /request/. In vcl_fetch you're removing the "Set-Cookie" header
from the /response/ from the server (which assumes a cache miss and
therefore a backend request to fetch the object).

Responses that set a cookie are usually not cacheable, since it can
often indicate some client-specific activity is taking place, which
is why the default VCL will refuse to cache such responses.

Some session-tracking mechanisms may try to set a cookie with every
response, without regard for whether doing so actually makes sense.
Removing the Set-Cookie header in responses to certain object types
provides a workaround for that situation.

Without the line in vcl_fetch, if your server is actually sending a
Set-Cookie header with responses for those resources, you would end
up either never caching them in the first place, or you'd cache the
response along with the Set-Cookie. Subsequent cache hits would see
every client receiving the same cookie. If it actually is a session
tracking cookie, that could be pretty disastrous.

> I need to be 100% sure that for all image/css objects, there are
> no cookies stored anywhere. Is this what the first example does,
> whereas the second example 'ignores' the cookie?

Varnish doesn't really store cookies, though it can be set up so as
to add them to the hash used to lookup cached entities. Both of the
examples are ignoring the cookies, but that first example will only
ignore the cookies sent by the client when requesting resources. In
the second example, you also ignore any cookies that are set by the
server when responding to the requests.

You could consider this to be preventing the browser from receiving
cookies, however that's not quite correct since other components of
the site may set cookies, and it's unlikely their path restrictions
will prevent them from being "valid" for images/css/etc. That's why
you still want to strip them from the incoming request, even though
you're also stripping them from certain responses.



From tfheen at varnish-software.com  Fri Feb 11 06:41:20 2011
From: tfheen at varnish-software.com (Tollef Fog Heen)
Date: Fri, 11 Feb 2011 07:41:20 +0100
Subject: How to find out a single failing backend from a round-robin
	director?(Inline-C)
In-Reply-To: <4D53B0F3.2060805@dmsp.de> (Stefan Pommerening's message of "Thu, 
	10 Feb 2011 10:33:39 +0100")
References: <4D5270A4.5040104@dmsp.de>
	<64E73E81AAC26A49AC9EA28CBE65365107EF8CA7@AUSYDEVS01.au.ad.westfield.com>
	<4D53B0F3.2060805@dmsp.de>
Message-ID: <87r5bf5azz.fsf@qurzaw.varnish-software.com>

]] Stefan Pommerening 

| What I need to get in my inline C is the specific backend choosen from
| the director, 'cmsweb5f3cae1' in this example.
| How to get this? Any VRT_something for this?

I don't believe that is exposed anywhere, I'm afraid.

-- 
Tollef Fog Heen
Varnish Software
t: +47 21 98 92 64



From martin.boer at bizztravel.nl  Thu Feb 10 14:14:29 2011
From: martin.boer at bizztravel.nl (Martin Boer)
Date: Thu, 10 Feb 2011 15:14:29 +0100
Subject: Dual server Varnish setup
In-Reply-To: <AANLkTikWwqzV2Lo0a9DdndrKA2V9ut2YwVPrqzq_xROQ@mail.gmail.com>
References: <0E7D9950-E2A1-4B1C-B619-A0CDE528C8B1@comicrelief.com>
	<AANLkTikWwqzV2Lo0a9DdndrKA2V9ut2YwVPrqzq_xROQ@mail.gmail.com>
Message-ID: <4D53F2C5.4090502@bizztravel.nl>

I would also add weights so that each varnish prefers to ask its sibling 
say 250 times more often than asking the app-server.
This will ensure that both varnishes will be as filled and as equal as 
possible.
I don't know if 250 could be replaced by a larger number but this works 
for me.

Regards,
Martin

On 02/10/2011 02:46 PM, Laurence Rowe wrote:
> On 8 February 2011 23:05, Ben Dodd<B.Dodd at comicrelief.com>  wrote:
>> Given this functionality, does anyone know what the VCL is required to implement this dual server Varnish setup scenario?
>> http://varnish-cache.org/trac/wiki/VCLExampleHashIgnoreBusy
> Something like this should work:
>
>
> backend appserver { ... }
> backend other_varnish { ... }
>
> # Server 1
> sub vcl_recv {
>      set req.backend = other_varnish;
>      if (req.http.X-Via-Varnish || !req.backend.healthy) {
>          # Request is coming from the other server.
>          # Ignore busy objects to avoid race condition.
>          set req.hash_ignore_busy = 1;
>          set req.backend = appserver;
>      } else {
>          set req.http.X-Via-Varnish = "true";
>      }
> }
>
> # Server 2
> sub vcl_recv {
>      set req.backend = other_varnish;
>      if (req.http.X-Via-Varnish || !req.backend.healthy) {
>          # Request is coming from the other server.
>          set req.backend = appserver;
>      } else {
>          set req.http.X-Via-Varnish = "true";
>      }
> }
>
> Laurence
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
>




From scaunter at topscms.com  Fri Feb 11 14:29:14 2011
From: scaunter at topscms.com (Caunter, Stefan)
Date: Fri, 11 Feb 2011 09:29:14 -0500
Subject: varnish 2.15 - default set to cache nothing except two urls?
In-Reply-To: <AANLkTikrit_x7BxvWeOiMHWNmLNPtYFpLPF6Z-TwXDnj@mail.gmail.com>
References: <AANLkTimh8ej1q1=Nmq1SGgh5yT_onqGod8-dNRP6StBY@mail.gmail.com><7F0AA702B8A85A4A967C4C8EBAD6902CEB689D@TMG-EVS02.torstar.net>
	<AANLkTikrit_x7BxvWeOiMHWNmLNPtYFpLPF6Z-TwXDnj@mail.gmail.com>
Message-ID: <7F0AA702B8A85A4A967C4C8EBAD6902CEB6924@TMG-EVS02.torstar.net>

The difference is that in recv it is a "req" from the client. Unsetting
it "strips" the cookie, and makes the request cacheable.

 

In fetch it is indeed "beresp" (back end response), so you are
removing/changing HTTP headers from your web server. I've added a
max-age to control client browser behaviour in the example too.

 

Stefan Caunter 

Operations

Torstar Digital

m: (416) 561-4871

 

 

From: varnish-misc-bounces at varnish-cache.org
[mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Mike Franon
Sent: February-10-11 4:51 PM
To: varnish-misc at varnish-cache.org
Subject: Re: varnish 2.15 - default set to cache nothing except two
urls?

 

Thank you for the help!

Quick question:

for this block

if (req.url ~ "/blah"){

                set beresp.ttl = 300s;

                unset req.http.set-cookie;

                set beresp.http.cache-control = "public, max-age = 300";

                set beresp.http.X-CacheReason = "Blah";

                return(deliver);

}

 

 

I thought it was unset beresp.http.set-cookie;

 

Is there a difference?

 

 

Thanks again.

 

On Thu, Feb 10, 2011 at 4:16 PM, Caunter, Stefan <scaunter at topscms.com>
wrote:

You need in vcl_recv:

 

if (req.url ~ "/blah"){

                unset req.http.cookie;

                return(lookup);

}

else {

                return(pass);

}

 

Setting 360s for beresp.ttl gets you 6 minutes J

 

Resetting it to 360 if it goes under 120 seems odd.

 

Use something like this in vcl_fetch:

 

if (req.url ~ "/blah"){

                set beresp.ttl = 300s;

                unset req.http.set-cookie;

                set beresp.http.cache-control = "public, max-age = 300";

                set beresp.http.X-CacheReason = "Blah";

                return(deliver);

}

 

Stefan Caunter 

Operations

Torstar Digital

m: (416) 561-4871

 

 

From: varnish-misc-bounces at varnish-cache.org
[mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Mike Franon
Sent: February-10-11 3:51 PM
To: varnish-misc at varnish-cache.org
Subject: varnish 2.15 - default set to cache nothing except two urls?

 

HI,

 

I am fairly new to varnish, and was curious, is there a way to have all
traffic go through varnish to the backend without caching unless for
specific pages.  For example we only want two pages to be cached, our
home, and a specific page under the home as seen below?

 

Here is my default.vcl 

 

Thanks

 

 

 

backend default {

  .host = "127.0.0.1";   

  .port = "80";

}

 

 

sub vcl_recv {

 

    # for the default home page

    if (req.url ~ "^/") {

        unset req.http.cookie;

    }

 

    

   # if they go directly to /samplepage

    if (req.url ~ "^/samplepage") {

        unset req.http.cookie;

    }

 

}

 

sub vcl_fetch {

    if (req.url ~ "^/") {

        unset beresp.http.set-cookie;

    }

    

    if (req.url ~ "^/samplepage") {

        unset beresp.http.set-cookie;

    }

    

    

    #override cache headers from backend to keep element in cache for 5
minutes

    if (beresp.ttl < 120s) {

       set beresp.ttl = 360s;

  }

 

}

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110211/05ea20f3/attachment.html>

From kongfranon at gmail.com  Fri Feb 11 14:45:00 2011
From: kongfranon at gmail.com (Mike Franon)
Date: Fri, 11 Feb 2011 09:45:00 -0500
Subject: varnish 2.15 - default set to cache nothing except two urls?
In-Reply-To: <7F0AA702B8A85A4A967C4C8EBAD6902CEB6924@TMG-EVS02.torstar.net>
References: <AANLkTimh8ej1q1=Nmq1SGgh5yT_onqGod8-dNRP6StBY@mail.gmail.com>
	<7F0AA702B8A85A4A967C4C8EBAD6902CEB689D@TMG-EVS02.torstar.net>
	<AANLkTikrit_x7BxvWeOiMHWNmLNPtYFpLPF6Z-TwXDnj@mail.gmail.com>
	<7F0AA702B8A85A4A967C4C8EBAD6902CEB6924@TMG-EVS02.torstar.net>
Message-ID: <AANLkTim=w=WMaohR1h3+4HEg5k---3bkn5WFbsxg-H=D@mail.gmail.com>

Great thanks,

got one more question  If I do this it caches everything,  it never gets to
the else return(pass), the reason I have the if set that way is I want it to
cache the home page but if the client clicks on anything else once they get
to home page, not to cache and send the cookie.

Thanks again.

:

sub vcl_recv {
if (req.url ~ "/"){
                unset req.http.cookie;
                return(lookup);
}

else {
                return(pass);
}

}
sub vcl_fetch {
    if (req.url ~ "/"){
        set beresp.ttl = 300s;
        set beresp.http.cache-control = "public, max-age = 300";
        set beresp.http.X-CacheReason = "varnishcache";
        unset beresp.http.set-cookie;
        return(deliver);
    }
}





On Fri, Feb 11, 2011 at 9:29 AM, Caunter, Stefan <scaunter at topscms.com>wrote:

> The difference is that in recv it is a ?req? from the client. Unsetting it
> ?strips? the cookie, and makes the request cacheable.
>
>
>
> In fetch it is indeed ?beresp? (back end response), so you are
> removing/changing HTTP headers from your web server. I?ve added a max-age to
> control client browser behaviour in the example too.
>
>
>
> Stefan Caunter
>
> Operations
>
> Torstar Digital
>
> m: (416) 561-4871
>
>
>
>
>
> *From:* varnish-misc-bounces at varnish-cache.org [mailto:
> varnish-misc-bounces at varnish-cache.org] *On Behalf Of *Mike Franon
>
> *Sent:* February-10-11 4:51 PM
> *To:* varnish-misc at varnish-cache.org
> *Subject:* Re: varnish 2.15 - default set to cache nothing except two
> urls?
>
>
>
> Thank you for the help!
>
> Quick question:
>
> for this block
>
> if (req.url ~ "/blah"){
>
>                 set beresp.ttl = 300s;
>
>                 *unset req.http.set-cookie;*
>
>                 set beresp.http.cache-control = "public, max-age = 300";
>
>                 set beresp.http.X-CacheReason = "Blah";
>
>                 return(deliver);
>
> }
>
>
>
>
>
> I thought it was unset beresp.http.set-cookie;
>
>
>
> Is there a difference?
>
>
>
>
>
> Thanks again.
>
>
>
> On Thu, Feb 10, 2011 at 4:16 PM, Caunter, Stefan <scaunter at topscms.com>
> wrote:
>
> You need in vcl_recv:
>
>
>
> if (req.url ~ ?/blah?){
>
>                 unset req.http.cookie;
>
>                 return(lookup);
>
> }
>
> else {
>
>                 return(pass);
>
> }
>
>
>
> Setting 360s for beresp.ttl gets you 6 minutes J
>
>
>
> Resetting it to 360 if it goes under 120 seems odd.
>
>
>
> Use something like this in vcl_fetch:
>
>
>
> if (req.url ~ "/blah"){
>
>                 set beresp.ttl = 300s;
>
>                 unset req.http.set-cookie;
>
>                 set beresp.http.cache-control = "public, max-age = 300";
>
>                 set beresp.http.X-CacheReason = "Blah";
>
>                 return(deliver);
>
> }
>
>
>
> Stefan Caunter
>
> Operations
>
> Torstar Digital
>
> m: (416) 561-4871
>
>
>
>
>
> *From:* varnish-misc-bounces at varnish-cache.org [mailto:
> varnish-misc-bounces at varnish-cache.org] *On Behalf Of *Mike Franon
> *Sent:* February-10-11 3:51 PM
> *To:* varnish-misc at varnish-cache.org
> *Subject:* varnish 2.15 - default set to cache nothing except two urls?
>
>
>
> HI,
>
>
>
> I am fairly new to varnish, and was curious, is there a way to have all
> traffic go through varnish to the backend without caching unless for
> specific pages.  For example we only want two pages to be cached, our home,
> and a specific page under the home as seen below?
>
>
>
> Here is my default.vcl
>
>
>
> Thanks
>
>
>
>
>
>
>
> backend default {
>
>   .host = "127.0.0.1";
>
>   .port = "80";
>
> }
>
>
>
>
>
> sub vcl_recv {
>
>
>
>     # for the default home page
>
>     if (req.url ~ "^/") {
>
>         unset req.http.cookie;
>
>     }
>
>
>
>
>
>    # if they go directly to /samplepage
>
>     if (req.url ~ "^/samplepage") {
>
>         unset req.http.cookie;
>
>     }
>
>
>
> }
>
>
>
> sub vcl_fetch {
>
>     if (req.url ~ "^/") {
>
>         unset beresp.http.set-cookie;
>
>     }
>
>
>
>     if (req.url ~ "^/samplepage") {
>
>         unset beresp.http.set-cookie;
>
>     }
>
>
>
>
>
>     #override cache headers from backend to keep element in cache for 5
> minutes
>
>     if (beresp.ttl < 120s) {
>
>        set beresp.ttl = 360s;
>
>   }
>
>
>
> }
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110211/7e2118c2/attachment.html>

From scaunter at topscms.com  Fri Feb 11 14:49:23 2011
From: scaunter at topscms.com (Caunter, Stefan)
Date: Fri, 11 Feb 2011 09:49:23 -0500
Subject: varnish 2.15 - default set to cache nothing except two urls?
In-Reply-To: <AANLkTim=w=WMaohR1h3+4HEg5k---3bkn5WFbsxg-H=D@mail.gmail.com>
References: <AANLkTimh8ej1q1=Nmq1SGgh5yT_onqGod8-dNRP6StBY@mail.gmail.com><7F0AA702B8A85A4A967C4C8EBAD6902CEB689D@TMG-EVS02.torstar.net><AANLkTikrit_x7BxvWeOiMHWNmLNPtYFpLPF6Z-TwXDnj@mail.gmail.com><7F0AA702B8A85A4A967C4C8EBAD6902CEB6924@TMG-EVS02.torstar.net>
	<AANLkTim=w=WMaohR1h3+4HEg5k---3bkn5WFbsxg-H=D@mail.gmail.com>
Message-ID: <7F0AA702B8A85A4A967C4C8EBAD6902CEB6937@TMG-EVS02.torstar.net>

It's matching everything, the regex is unanchored.

 

You need

 

if (req.url ~ "^/$"){

 

Stefan Caunter 

Operations

Torstar Digital

m: (416) 561-4871

 

 

From: varnish-misc-bounces at varnish-cache.org
[mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Mike Franon
Sent: February-11-11 9:45 AM
To: varnish-misc at varnish-cache.org
Subject: Re: varnish 2.15 - default set to cache nothing except two
urls?

 

Great thanks,

got one more question  If I do this it caches everything,  it never gets
to the else return(pass), the reason I have the if set that way is I
want it to cache the home page but if the client clicks on anything else
once they get to home page, not to cache and send the cookie.  

Thanks again.

:

sub vcl_recv {
if (req.url ~ "/"){
                unset req.http.cookie;
                return(lookup);
}

else {
                return(pass);
}

}
sub vcl_fetch {
    if (req.url ~ "/"){
        set beresp.ttl = 300s;
        set beresp.http.cache-control = "public, max-age = 300";
        set beresp.http.X-CacheReason = "varnishcache";
        unset beresp.http.set-cookie;
        return(deliver);
    }
}






On Fri, Feb 11, 2011 at 9:29 AM, Caunter, Stefan <scaunter at topscms.com>
wrote:

The difference is that in recv it is a "req" from the client. Unsetting
it "strips" the cookie, and makes the request cacheable.

 

In fetch it is indeed "beresp" (back end response), so you are
removing/changing HTTP headers from your web server. I've added a
max-age to control client browser behaviour in the example too.

 

Stefan Caunter 

Operations

Torstar Digital

m: (416) 561-4871

 

 

From: varnish-misc-bounces at varnish-cache.org
[mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Mike Franon


Sent: February-10-11 4:51 PM
To: varnish-misc at varnish-cache.org

Subject: Re: varnish 2.15 - default set to cache nothing except two
urls?

 

Thank you for the help!

Quick question:

for this block

if (req.url ~ "/blah"){

                set beresp.ttl = 300s;

                unset req.http.set-cookie;

                set beresp.http.cache-control = "public, max-age = 300";

                set beresp.http.X-CacheReason = "Blah";

                return(deliver);

}

 

 

I thought it was unset beresp.http.set-cookie;

 

Is there a difference?

 

 

Thanks again.

 

On Thu, Feb 10, 2011 at 4:16 PM, Caunter, Stefan <scaunter at topscms.com>
wrote:

You need in vcl_recv:

 

if (req.url ~ "/blah"){

                unset req.http.cookie;

                return(lookup);

}

else {

                return(pass);

}

 

Setting 360s for beresp.ttl gets you 6 minutes J

 

Resetting it to 360 if it goes under 120 seems odd.

 

Use something like this in vcl_fetch:

 

if (req.url ~ "/blah"){

                set beresp.ttl = 300s;

                unset req.http.set-cookie;

                set beresp.http.cache-control = "public, max-age = 300";

                set beresp.http.X-CacheReason = "Blah";

                return(deliver);

}

 

Stefan Caunter 

Operations

Torstar Digital

m: (416) 561-4871

 

 

From: varnish-misc-bounces at varnish-cache.org
[mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Mike Franon
Sent: February-10-11 3:51 PM
To: varnish-misc at varnish-cache.org
Subject: varnish 2.15 - default set to cache nothing except two urls?

 

HI,

 

I am fairly new to varnish, and was curious, is there a way to have all
traffic go through varnish to the backend without caching unless for
specific pages.  For example we only want two pages to be cached, our
home, and a specific page under the home as seen below?

 

Here is my default.vcl 

 

Thanks

 

 

 

backend default {

  .host = "127.0.0.1";   

  .port = "80";

}

 

 

sub vcl_recv {

 

    # for the default home page

    if (req.url ~ "^/") {

        unset req.http.cookie;

    }

 

    

   # if they go directly to /samplepage

    if (req.url ~ "^/samplepage") {

        unset req.http.cookie;

    }

 

}

 

sub vcl_fetch {

    if (req.url ~ "^/") {

        unset beresp.http.set-cookie;

    }

    

    if (req.url ~ "^/samplepage") {

        unset beresp.http.set-cookie;

    }

    

    

    #override cache headers from backend to keep element in cache for 5
minutes

    if (beresp.ttl < 120s) {

       set beresp.ttl = 360s;

  }

 

}

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110211/9437fd9d/attachment.html>

From kongfranon at gmail.com  Fri Feb 11 14:59:19 2011
From: kongfranon at gmail.com (Mike Franon)
Date: Fri, 11 Feb 2011 09:59:19 -0500
Subject: varnish 2.15 - default set to cache nothing except two urls?
In-Reply-To: <7F0AA702B8A85A4A967C4C8EBAD6902CEB6937@TMG-EVS02.torstar.net>
References: <AANLkTimh8ej1q1=Nmq1SGgh5yT_onqGod8-dNRP6StBY@mail.gmail.com>
	<7F0AA702B8A85A4A967C4C8EBAD6902CEB689D@TMG-EVS02.torstar.net>
	<AANLkTikrit_x7BxvWeOiMHWNmLNPtYFpLPF6Z-TwXDnj@mail.gmail.com>
	<7F0AA702B8A85A4A967C4C8EBAD6902CEB6924@TMG-EVS02.torstar.net>
	<AANLkTim=w=WMaohR1h3+4HEg5k---3bkn5WFbsxg-H=D@mail.gmail.com>
	<7F0AA702B8A85A4A967C4C8EBAD6902CEB6937@TMG-EVS02.torstar.net>
Message-ID: <AANLkTikJ9HyX+7g-t3A3sGRe18Df94yatisznZXWRxaJ@mail.gmail.com>

Thank you!  I forgot about that...

so now my vcl looks like this and it works!!


Thanks again for the help.

sub vcl_recv {
if (req.url ~ "^/$" || req.url ~ "^/sample"){
                unset req.http.cookie;
                return(lookup);
}

else {
                return(pass);
}

}
sub vcl_fetch {
    if (req.url ~ "^/$" || req.url ~ "^/sample"){
        set beresp.ttl = 300s;
        set beresp.http.cache-control = "public, max-age = 300";
        set beresp.http.X-CacheReason = "varnishcache";
        unset beresp.http.set-cookie;
        return(deliver);
    }
}


On Fri, Feb 11, 2011 at 9:49 AM, Caunter, Stefan <scaunter at topscms.com>wrote:

> It?s matching everything, the regex is unanchored.
>
>
>
> You need
>
>
>
> if (req.url ~ ?^/$?){
>
>
>
> Stefan Caunter
>
> Operations
>
> Torstar Digital
>
> m: (416) 561-4871
>
>
>
>
>
> *From:* varnish-misc-bounces at varnish-cache.org [mailto:
> varnish-misc-bounces at varnish-cache.org] *On Behalf Of *Mike Franon
> *Sent:* February-11-11 9:45 AM
>
> *To:* varnish-misc at varnish-cache.org
> *Subject:* Re: varnish 2.15 - default set to cache nothing except two
> urls?
>
>
>
> Great thanks,
>
> got one more question  If I do this it caches everything,  it never gets to
> the else return(pass), the reason I have the if set that way is I want it to
> cache the home page but if the client clicks on anything else once they get
> to home page, not to cache and send the cookie.
>
> Thanks again.
>
> :
>
> sub vcl_recv {
> if (req.url ~ "/"){
>                 unset req.http.cookie;
>                 return(lookup);
> }
>
> else {
>                 return(pass);
> }
>
> }
> sub vcl_fetch {
>     if (req.url ~ "/"){
>         set beresp.ttl = 300s;
>         set beresp.http.cache-control = "public, max-age = 300";
>         set beresp.http.X-CacheReason = "varnishcache";
>         unset beresp.http.set-cookie;
>         return(deliver);
>     }
> }
>
>
>
>
> On Fri, Feb 11, 2011 at 9:29 AM, Caunter, Stefan <scaunter at topscms.com>
> wrote:
>
> The difference is that in recv it is a ?req? from the client. Unsetting it
> ?strips? the cookie, and makes the request cacheable.
>
>
>
> In fetch it is indeed ?beresp? (back end response), so you are
> removing/changing HTTP headers from your web server. I?ve added a max-age to
> control client browser behaviour in the example too.
>
>
>
> Stefan Caunter
>
> Operations
>
> Torstar Digital
>
> m: (416) 561-4871
>
>
>
>
>
> *From:* varnish-misc-bounces at varnish-cache.org [mailto:
> varnish-misc-bounces at varnish-cache.org] *On Behalf Of *Mike Franon
>
>
> *Sent:* February-10-11 4:51 PM
> *To:* varnish-misc at varnish-cache.org
>
> *Subject:* Re: varnish 2.15 - default set to cache nothing except two
> urls?
>
>
>
> Thank you for the help!
>
> Quick question:
>
> for this block
>
> if (req.url ~ "/blah"){
>
>                 set beresp.ttl = 300s;
>
>                 *unset req.http.set-cookie;*
>
>                 set beresp.http.cache-control = "public, max-age = 300";
>
>                 set beresp.http.X-CacheReason = "Blah";
>
>                 return(deliver);
>
> }
>
>
>
>
>
> I thought it was unset beresp.http.set-cookie;
>
>
>
> Is there a difference?
>
>
>
>
>
> Thanks again.
>
>
>
> On Thu, Feb 10, 2011 at 4:16 PM, Caunter, Stefan <scaunter at topscms.com>
> wrote:
>
> You need in vcl_recv:
>
>
>
> if (req.url ~ ?/blah?){
>
>                 unset req.http.cookie;
>
>                 return(lookup);
>
> }
>
> else {
>
>                 return(pass);
>
> }
>
>
>
> Setting 360s for beresp.ttl gets you 6 minutes J
>
>
>
> Resetting it to 360 if it goes under 120 seems odd.
>
>
>
> Use something like this in vcl_fetch:
>
>
>
> if (req.url ~ "/blah"){
>
>                 set beresp.ttl = 300s;
>
>                 unset req.http.set-cookie;
>
>                 set beresp.http.cache-control = "public, max-age = 300";
>
>                 set beresp.http.X-CacheReason = "Blah";
>
>                 return(deliver);
>
> }
>
>
>
> Stefan Caunter
>
> Operations
>
> Torstar Digital
>
> m: (416) 561-4871
>
>
>
>
>
> *From:* varnish-misc-bounces at varnish-cache.org [mailto:
> varnish-misc-bounces at varnish-cache.org] *On Behalf Of *Mike Franon
> *Sent:* February-10-11 3:51 PM
> *To:* varnish-misc at varnish-cache.org
> *Subject:* varnish 2.15 - default set to cache nothing except two urls?
>
>
>
> HI,
>
>
>
> I am fairly new to varnish, and was curious, is there a way to have all
> traffic go through varnish to the backend without caching unless for
> specific pages.  For example we only want two pages to be cached, our home,
> and a specific page under the home as seen below?
>
>
>
> Here is my default.vcl
>
>
>
> Thanks
>
>
>
>
>
>
>
> backend default {
>
>   .host = "127.0.0.1";
>
>   .port = "80";
>
> }
>
>
>
>
>
> sub vcl_recv {
>
>
>
>     # for the default home page
>
>     if (req.url ~ "^/") {
>
>         unset req.http.cookie;
>
>     }
>
>
>
>
>
>    # if they go directly to /samplepage
>
>     if (req.url ~ "^/samplepage") {
>
>         unset req.http.cookie;
>
>     }
>
>
>
> }
>
>
>
> sub vcl_fetch {
>
>     if (req.url ~ "^/") {
>
>         unset beresp.http.set-cookie;
>
>     }
>
>
>
>     if (req.url ~ "^/samplepage") {
>
>         unset beresp.http.set-cookie;
>
>     }
>
>
>
>
>
>     #override cache headers from backend to keep element in cache for 5
> minutes
>
>     if (beresp.ttl < 120s) {
>
>        set beresp.ttl = 360s;
>
>   }
>
>
>
> }
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110211/b9009b42/attachment.html>

From j.begumisa at gmail.com  Fri Feb 11 23:12:06 2011
From: j.begumisa at gmail.com (Joseph Begumisa)
Date: Fri, 11 Feb 2011 15:12:06 -0800
Subject: Varnish High CPU Usage / High Load Average
Message-ID: <AANLkTinGLb-Fj4hKJ6wks0zewEs3HMCyPGbYaDtuonkY@mail.gmail.com>

Hi,

I'm having an issue where when I start varnish, CPU usage rises in
about 3minutes to about 95% and load average rises to about 150.  I
then have to stop varnish and restart it before the system becomes
unresponsive.

Specs of the server i'm using are: Dual Quad-Core Intel Xeon CPU E5440
@ 2.83GHz with 16GB RAM.

I'm running version 2.1.5 and varnishd is started with the following options:

DAEMON_OPTS="-a 10.0.6.6:80
             -T localhost:6082 \
             -f /opt/varnish/etc/varnish/default.vcl \
             -p thread_pools=8 \
             -p thread_pool_min=100 \
             -p thread_pool_max=2000 \
             -p thread_pool_add_delay=2 \
             -p cli_timeout=25 \
             -p session_linger=50/100/150 \
             -h classic,30007 \
             -p listen_depth=4096 \
             -P /var/run/varnish/varnish.pid \
             -u varnish -g varnish \
             -s malloc,4G"

my default.vcl file is pretty basic for now:

backend web1 {
     .host = "10.0.6.66";
     .port = "80";
}

I was running varnishstat and obtained these stats immediately after I
stopped the varnish service since cpu usage was approaching 100%:

0+00:03:03
Hitrate ratio:       10      100      145
Hitrate avg:     0.9814   0.9796   0.9794

       40090         0.00       219.07 Client connections accepted
      143712         0.00       785.31 Client requests received
       95786         0.00       523.42 Cache hits
       11834         0.00        64.67 Cache hits for pass
        2321         0.00        12.68 Cache misses
        3096         0.00        16.92 Backend conn. success
       44665         0.00       244.07 Backend conn. reuses
           6         0.00         0.03 Backend conn. was closed
       45827         0.00       250.42 Backend conn. recycles
          87         0.00         0.48 Fetch head
       29726         0.00       162.44 Fetch with Length
        3339         0.00        18.25 Fetch chunked
          52         0.00         0.28 Fetch wanted close
           4         0.00         0.02 Fetch failed
        1583          .            .   N struct sess_mem
        1376          .            .   N struct sess
        2314          .            .   N struct object
        3037          .            .   N struct objectcore
        3037          .            .   N struct objecthead
         267          .            .   N struct vbe_conn
         800          .            .   N worker threads
         800         0.00         4.37 N worker threads created
           1          .            .   N backends
       11612          .            .   N LRU moved objects
       60661         0.00       331.48 Objects sent with write
       39967         0.00       218.40 Total Sessions
      143712         0.00       785.31 Total Requests
        1399         0.00         7.64 Total pipe
       44178         0.00       241.41 Total pass
       46203         0.00       252.48 Total fetch
    51252168         0.00    280066.49 Total header bytes
   795295815         0.00   4345878.77 Total body bytes
        7931         0.00        43.34 Session Closed


Any insights into what would cause that / changes I need to make would
be helpful.  Thanks.


Joseph



From j.begumisa at gmail.com  Sat Feb 12 02:19:14 2011
From: j.begumisa at gmail.com (Joseph Begumisa)
Date: Fri, 11 Feb 2011 18:19:14 -0800
Subject: Varnish High CPU Usage / High Load Average
In-Reply-To: <AANLkTinGLb-Fj4hKJ6wks0zewEs3HMCyPGbYaDtuonkY@mail.gmail.com>
References: <AANLkTinGLb-Fj4hKJ6wks0zewEs3HMCyPGbYaDtuonkY@mail.gmail.com>
Message-ID: <AANLkTin9w94SZLkGo_AbV=SySP5wB7Cf=qqGMj82YYA8@mail.gmail.com>

Please disregard.   After using only a basic vcl file, I noticed that
cpu usage and load average was back to normal.  So I went through the
customized default.vcl and noticed my purging syntax just might be the
cause of the problem.  After correcting it and running more tests, the
issue of high cpu usage and high load average no longer existed.

Best Regards,

Joseph



On Fri, Feb 11, 2011 at 3:12 PM, Joseph Begumisa <j.begumisa at gmail.com> wrote:
> Hi,
>
> I'm having an issue where when I start varnish, CPU usage rises in
> about 3minutes to about 95% and load average rises to about 150. ?I
> then have to stop varnish and restart it before the system becomes
> unresponsive.
>
> Specs of the server i'm using are: Dual Quad-Core Intel Xeon CPU E5440
> @ 2.83GHz with 16GB RAM.
>
> I'm running version 2.1.5 and varnishd is started with the following options:
>
> DAEMON_OPTS="-a 10.0.6.6:80
> ? ? ? ? ? ? -T localhost:6082 \
> ? ? ? ? ? ? -f /opt/varnish/etc/varnish/default.vcl \
> ? ? ? ? ? ? -p thread_pools=8 \
> ? ? ? ? ? ? -p thread_pool_min=100 \
> ? ? ? ? ? ? -p thread_pool_max=2000 \
> ? ? ? ? ? ? -p thread_pool_add_delay=2 \
> ? ? ? ? ? ? -p cli_timeout=25 \
> ? ? ? ? ? ? -p session_linger=50/100/150 \
> ? ? ? ? ? ? -h classic,30007 \
> ? ? ? ? ? ? -p listen_depth=4096 \
> ? ? ? ? ? ? -P /var/run/varnish/varnish.pid \
> ? ? ? ? ? ? -u varnish -g varnish \
> ? ? ? ? ? ? -s malloc,4G"
>
> my default.vcl file is pretty basic for now:
>
> backend web1 {
> ? ? .host = "10.0.6.66";
> ? ? .port = "80";
> }
>
> I was running varnishstat and obtained these stats immediately after I
> stopped the varnish service since cpu usage was approaching 100%:
>
> 0+00:03:03
> Hitrate ratio: ? ? ? 10 ? ? ?100 ? ? ?145
> Hitrate avg: ? ? 0.9814 ? 0.9796 ? 0.9794
>
> ? ? ? 40090 ? ? ? ? 0.00 ? ? ? 219.07 Client connections accepted
> ? ? ?143712 ? ? ? ? 0.00 ? ? ? 785.31 Client requests received
> ? ? ? 95786 ? ? ? ? 0.00 ? ? ? 523.42 Cache hits
> ? ? ? 11834 ? ? ? ? 0.00 ? ? ? ?64.67 Cache hits for pass
> ? ? ? ?2321 ? ? ? ? 0.00 ? ? ? ?12.68 Cache misses
> ? ? ? ?3096 ? ? ? ? 0.00 ? ? ? ?16.92 Backend conn. success
> ? ? ? 44665 ? ? ? ? 0.00 ? ? ? 244.07 Backend conn. reuses
> ? ? ? ? ? 6 ? ? ? ? 0.00 ? ? ? ? 0.03 Backend conn. was closed
> ? ? ? 45827 ? ? ? ? 0.00 ? ? ? 250.42 Backend conn. recycles
> ? ? ? ? ?87 ? ? ? ? 0.00 ? ? ? ? 0.48 Fetch head
> ? ? ? 29726 ? ? ? ? 0.00 ? ? ? 162.44 Fetch with Length
> ? ? ? ?3339 ? ? ? ? 0.00 ? ? ? ?18.25 Fetch chunked
> ? ? ? ? ?52 ? ? ? ? 0.00 ? ? ? ? 0.28 Fetch wanted close
> ? ? ? ? ? 4 ? ? ? ? 0.00 ? ? ? ? 0.02 Fetch failed
> ? ? ? ?1583 ? ? ? ? ?. ? ? ? ? ? ?. ? N struct sess_mem
> ? ? ? ?1376 ? ? ? ? ?. ? ? ? ? ? ?. ? N struct sess
> ? ? ? ?2314 ? ? ? ? ?. ? ? ? ? ? ?. ? N struct object
> ? ? ? ?3037 ? ? ? ? ?. ? ? ? ? ? ?. ? N struct objectcore
> ? ? ? ?3037 ? ? ? ? ?. ? ? ? ? ? ?. ? N struct objecthead
> ? ? ? ? 267 ? ? ? ? ?. ? ? ? ? ? ?. ? N struct vbe_conn
> ? ? ? ? 800 ? ? ? ? ?. ? ? ? ? ? ?. ? N worker threads
> ? ? ? ? 800 ? ? ? ? 0.00 ? ? ? ? 4.37 N worker threads created
> ? ? ? ? ? 1 ? ? ? ? ?. ? ? ? ? ? ?. ? N backends
> ? ? ? 11612 ? ? ? ? ?. ? ? ? ? ? ?. ? N LRU moved objects
> ? ? ? 60661 ? ? ? ? 0.00 ? ? ? 331.48 Objects sent with write
> ? ? ? 39967 ? ? ? ? 0.00 ? ? ? 218.40 Total Sessions
> ? ? ?143712 ? ? ? ? 0.00 ? ? ? 785.31 Total Requests
> ? ? ? ?1399 ? ? ? ? 0.00 ? ? ? ? 7.64 Total pipe
> ? ? ? 44178 ? ? ? ? 0.00 ? ? ? 241.41 Total pass
> ? ? ? 46203 ? ? ? ? 0.00 ? ? ? 252.48 Total fetch
> ? ?51252168 ? ? ? ? 0.00 ? ?280066.49 Total header bytes
> ? 795295815 ? ? ? ? 0.00 ? 4345878.77 Total body bytes
> ? ? ? ?7931 ? ? ? ? 0.00 ? ? ? ?43.34 Session Closed
>
>
> Any insights into what would cause that / changes I need to make would
> be helpful. ?Thanks.
>
>
> Joseph
>

From B.Dodd at comicrelief.com  Sat Feb 12 11:13:58 2011
From: B.Dodd at comicrelief.com (Ben Dodd)
Date: Sat, 12 Feb 2011 11:13:58 +0000
Subject: Determining hash variations
Message-ID: <356FD748-D91A-4CFE-BD6F-992841A1CC7F@comicrelief.com>

Hello,

Is there anyway to determine the number of hash variations a URL has in the cache? So if a page has multiple  versions based on cookies, accept-encoding etc.

Kind regards, Ben

Comic Relief  1st Floor  89 Albert Embankment  London SE1 7TP  Tel: 020 7820 2000  Fax: 020 7820 2222  red at comicrelief.com  www.comicrelief.com

Comic Relief is the operating name of Charity Projects, a company limited by guarantee and registered in England no. 1806414; registered charity 326568 (England & Wales) and SC039730 (Scotland). Comic Relief Ltd is a subsidiary of Charity Projects and registered in England no. 1967154. Registered offices: Hanover House, 14 Hanover Square, London W1S 1HP. VAT no. 773865187.

This email (and any attachment) may contain confidential and/or privileged information. If you are not the intended addressee, you must not use, disclose, copy or rely on anything in this email and should contact the sender and delete it immediately. The views of the author are not necessarily those of Comic Relief. We cannot guarantee that this email (and any attachment) is virus free or has not been intercepted and amended, so do not accept liability for any damage resulting from software viruses. You should carry out your own virus checks.



From stewsnooze at gmail.com  Sat Feb 12 18:22:03 2011
From: stewsnooze at gmail.com (Stewart Robinson)
Date: Sat, 12 Feb 2011 18:22:03 +0000
Subject: Determining hash variations
In-Reply-To: <356FD748-D91A-4CFE-BD6F-992841A1CC7F@comicrelief.com>
References: <356FD748-D91A-4CFE-BD6F-992841A1CC7F@comicrelief.com>
Message-ID: <AANLkTinL=9Niy__2CNNW-toGYTUO3KX1K-+EN4_eMXC9@mail.gmail.com>

Hi,

I think that you won't be able to interrogate the cache to find out
what is in it. You might be able to construct some script that watches
varnishlog to build up a list of these URL variations but I wouldn't
do that in production at comic reliefs scale.

Stew

On 12 February 2011 11:13, Ben Dodd <B.Dodd at comicrelief.com> wrote:
> Hello,
>
> Is there anyway to determine the number of hash variations a URL has in the cache? So if a page has multiple ?versions based on cookies, accept-encoding etc.
>
> Kind regards, Ben
>
> Comic Relief ?1st Floor ?89 Albert Embankment ?London SE1 7TP ?Tel: 020 7820 2000 ?Fax: 020 7820 2222 ?red at comicrelief.com ?www.comicrelief.com
>
> Comic Relief is the operating name of Charity Projects, a company limited by guarantee and registered in England no. 1806414; registered charity 326568 (England & Wales) and SC039730 (Scotland). Comic Relief Ltd is a subsidiary of Charity Projects and registered in England no. 1967154. Registered offices: Hanover House, 14 Hanover Square, London W1S 1HP. VAT no. 773865187.
>
> This email (and any attachment) may contain confidential and/or privileged information. If you are not the intended addressee, you must not use, disclose, copy or rely on anything in this email and should contact the sender and delete it immediately. The views of the author are not necessarily those of Comic Relief. We cannot guarantee that this email (and any attachment) is virus free or has not been intercepted and amended, so do not accept liability for any damage resulting from software viruses. You should carry out your own virus checks.
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>



From scaunter at topscms.com  Mon Feb 14 13:08:00 2011
From: scaunter at topscms.com (Caunter, Stefan)
Date: Mon, 14 Feb 2011 08:08:00 -0500
Subject: Determining hash variations
In-Reply-To: <AANLkTinL=9Niy__2CNNW-toGYTUO3KX1K-+EN4_eMXC9@mail.gmail.com>
References: <356FD748-D91A-4CFE-BD6F-992841A1CC7F@comicrelief.com>
	<AANLkTinL=9Niy__2CNNW-toGYTUO3KX1K-+EN4_eMXC9@mail.gmail.com>
Message-ID: <37702739-8784-4EE2-9541-E2EE38AAC1B6@topscms.com>

The interaction between the webapp and vcl is how you know what is in the cache. 
Do you normalize AE? If you are hashing based on cookie you will see low hit rate. Add debug headers to observe what is pulled from cache with different clients. 

Stefan Caunter
Operations
TorstarDigital
416.561.4871

On 2011-02-12, at 1:23 PM, "Stewart Robinson" <stewsnooze at gmail.com> wrote:

> Hi,
> 
> I think that you won't be able to interrogate the cache to find out
> what is in it. You might be able to construct some script that watches
> varnishlog to build up a list of these URL variations but I wouldn't
> do that in production at comic reliefs scale.
> 
> Stew
> 
> On 12 February 2011 11:13, Ben Dodd <B.Dodd at comicrelief.com> wrote:
>> Hello,
>> 
>> Is there anyway to determine the number of hash variations a URL has in the cache? So if a page has multiple  versions based on cookies, accept-encoding etc.
>> 
>> Kind regards, Ben
>> 
>> Comic Relief  1st Floor  89 Albert Embankment  London SE1 7TP  Tel: 020 7820 2000  Fax: 020 7820 2222  red at comicrelief.com  www.comicrelief.com
>> 
>> Comic Relief is the operating name of Charity Projects, a company limited by guarantee and registered in England no. 1806414; registered charity 326568 (England & Wales) and SC039730 (Scotland). Comic Relief Ltd is a subsidiary of Charity Projects and registered in England no. 1967154. Registered offices: Hanover House, 14 Hanover Square, London W1S 1HP. VAT no. 773865187.
>> 
>> This email (and any attachment) may contain confidential and/or privileged information. If you are not the intended addressee, you must not use, disclose, copy or rely on anything in this email and should contact the sender and delete it immediately. The views of the author are not necessarily those of Comic Relief. We cannot guarantee that this email (and any attachment) is virus free or has not been intercepted and amended, so do not accept liability for any damage resulting from software viruses. You should carry out your own virus checks.
>> 
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at varnish-cache.org
>> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>> 
> 
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc



From johnson at nmr.mgh.harvard.edu  Mon Feb 14 14:42:29 2011
From: johnson at nmr.mgh.harvard.edu (Chris Johnson)
Date: Mon, 14 Feb 2011 09:42:29 -0500 (EST)
Subject: virtual host remapping.
In-Reply-To: <AANLkTinL=9Niy__2CNNW-toGYTUO3KX1K-+EN4_eMXC9@mail.gmail.com>
References: <356FD748-D91A-4CFE-BD6F-992841A1CC7F@comicrelief.com>
	<AANLkTinL=9Niy__2CNNW-toGYTUO3KX1K-+EN4_eMXC9@mail.gmail.com>
Message-ID: <alpine.LRH.2.00.1102140925180.1249@maequeses.nmr.mgh.harvard.edu>

      Hi,

      A quick restatement of the problem.  I have virtual hosts on my
server and I want to phase one out and move to another but don't
want varnish double caching both sites as they are of course the same.

      Following is the top chunk of default.vcl.  The only things I've
added are the freesurfer backend definition

backend freesurfer {
       .host = "127.0.0.1";
       .port = "8080";
}

and the second if

     if (req.http.host ~ "^sufer.nmr.mgh.harvard.edu") {
       set req.http.host = "www.freesurfer.net";
       set req.backend = freesurfer;
     }

>From what I've been able to figure out the "if" looks for the old
virtual host and set's below it rewrite the header to look at the new
virtual host.  The last line in the vcl_recv function is a
return(_lookup) which I hope catches the request and handles.  If I
should be doing something else yet I don't know what.

      But I don't think it's working. I've looked at varnishlog output
and the backend fetch seems to be still using the old address.  And I
see no references to the new virtual host when I try to access the
pages via the old address in httpd's  access_log

      What am I doing wrong?  Tnx.

#This is a basic VCL configuration file for varnish.  See the vcl(7)
#man page for details on VCL syntax and semantics.
#
#Default backend definition.  Set this to point to your content
#server.
#
backend default {
   .host = "127.0.0.1";
   .port = "8080";
}

# local definition backend

backend freesurfer {
       .host = "127.0.0.1";
       .port = "8080";
}

#
#Below is a commented-out copy of the default VCL logic.  If you
#redefine any of these subroutines, the built-in logic will be
#appended to your code.
#
sub vcl_recv {
     if (req.request != "GET" &&
       req.request != "HEAD" &&
       req.request != "PUT" &&
       req.request != "POST" &&
       req.request != "TRACE" &&
       req.request != "OPTIONS" &&
       req.request != "DELETE") {
         /* Non-RFC2616 or CONNECT which is weird. */
         return (pipe);
     }
     if (req.http.host ~ "^sufer.nmr.mgh.harvard.edu") {
       set req.http.host = "www.freesurfer.net";
       set req.backend = freesurfer;
     }
     if (req.url ~ "^/fswiki") {
         remove req.http.Cookie;
     }
     if (req.url ~ "^/wiki/fswiki_htdocs") {
         remove req.http.Cookie;
     }
     if (req.request != "GET" && req.request != "HEAD") {
         /* We only deal with GET and HEAD by default */
         return (pass);
     }
     if (req.http.Authorization || req.http.Cookie) {
         /* Not cacheable by default */
         return (pass);
     }
     return (lookup);
}

-------------------------------------------------------------------------------
Chris Johnson               |Internet: johnson at nmr.mgh.harvard.edu
Systems Administrator       |Web:      http://www.nmr.mgh.harvard.edu/~johnson
NMR Center                  |Voice:    617.726.0949
Mass. General Hospital      |FAX:      617.726.7422
149 (2301) 13th Street      |"A good engineer never reinvents the wheel when
Charlestown, MA., 02129 USA |an existing one with modifications will do." Me
-------------------------------------------------------------------------------


The information in this e-mail is intended only for the person to whom it is
addressed. If you believe this e-mail was sent to you in error and the e-mail
contains patient information, please contact the Partners Compliance HelpLine at
http://www.partners.org/complianceline . If the e-mail was sent to you in error
but does not contain patient information, please contact the sender and properly
dispose of the e-mail.




From perbu at varnish-software.com  Mon Feb 14 15:37:31 2011
From: perbu at varnish-software.com (Per Buer)
Date: Mon, 14 Feb 2011 16:37:31 +0100
Subject: virtual host remapping.
In-Reply-To: <alpine.LRH.2.00.1102140925180.1249@maequeses.nmr.mgh.harvard.edu>
References: <356FD748-D91A-4CFE-BD6F-992841A1CC7F@comicrelief.com>
	<AANLkTinL=9Niy__2CNNW-toGYTUO3KX1K-+EN4_eMXC9@mail.gmail.com>
	<alpine.LRH.2.00.1102140925180.1249@maequeses.nmr.mgh.harvard.edu>
Message-ID: <AANLkTinAfJ307rSU4Ebvcvc-q=zjJqraQJ4QcJ9o5nU6@mail.gmail.com>

Hi,

Without a relevant log segment from varnishlog it's difficult to see,
but let me guess:

On Mon, Feb 14, 2011 at 3:42 PM, Chris Johnson
<johnson at nmr.mgh.harvard.edu> wrote:
> (..)
> backend freesurfer {
> ? ? ?.host = "127.0.0.1";
> ? ? ?.port = "8080";
> }

Right. Freesurfer.

>
> and the second if
>
> ? ?if (req.http.host ~ "^sufer.nmr.mgh.harvard.edu") {

sufer? Spelling error?



-- 
Per Buer,?Varnish Software
Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer
Varnish makes websites fly!
Want to learn more about Varnish? http://www.varnish-software.com/whitepapers



From johnson at nmr.mgh.harvard.edu  Mon Feb 14 15:43:53 2011
From: johnson at nmr.mgh.harvard.edu (Chris Johnson)
Date: Mon, 14 Feb 2011 10:43:53 -0500 (EST)
Subject: virtual host remapping.
In-Reply-To: <AANLkTinAfJ307rSU4Ebvcvc-q=zjJqraQJ4QcJ9o5nU6@mail.gmail.com>
References: <356FD748-D91A-4CFE-BD6F-992841A1CC7F@comicrelief.com>
	<AANLkTinL=9Niy__2CNNW-toGYTUO3KX1K-+EN4_eMXC9@mail.gmail.com>
	<alpine.LRH.2.00.1102140925180.1249@maequeses.nmr.mgh.harvard.edu>
	<AANLkTinAfJ307rSU4Ebvcvc-q=zjJqraQJ4QcJ9o5nU6@mail.gmail.com>
Message-ID: <alpine.LRH.2.00.1102141041500.1249@maequeses.nmr.mgh.harvard.edu>

      SIGH.  I've only ever had one keyboard I ever liked the action
on.  It was a C-ITHO 101 but those are long gone.  Thank you.  But
otherwise this should work, yes?

> Hi,
>
> Without a relevant log segment from varnishlog it's difficult to see,
> but let me guess:
>
> On Mon, Feb 14, 2011 at 3:42 PM, Chris Johnson
> <johnson at nmr.mgh.harvard.edu> wrote:
>> (..)
>> backend freesurfer {
>> ? ? ?.host = "127.0.0.1";
>> ? ? ?.port = "8080";
>> }
>
> Right. Freesurfer.
>
>>
>> and the second if
>>
>> ? ?if (req.http.host ~ "^sufer.nmr.mgh.harvard.edu") {
>
> sufer? Spelling error?
>
>
>
> -- 
> Per Buer,?Varnish Software
> Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer
> Varnish makes websites fly!
> Want to learn more about Varnish? http://www.varnish-software.com/whitepapers
>
>
>

-------------------------------------------------------------------------------
Chris Johnson               |Internet: johnson at nmr.mgh.harvard.edu
Systems Administrator       |Web:      http://www.nmr.mgh.harvard.edu/~johnson
NMR Center                  |Voice:    617.726.0949
Mass. General Hospital      |FAX:      617.726.7422
149 (2301) 13th Street      |Prediction is difficult, especially when it comes
Charlestown, MA., 02129 USA |to the future.  Yogi Berra.
-------------------------------------------------------------------------------


The information in this e-mail is intended only for the person to whom it is
addressed. If you believe this e-mail was sent to you in error and the e-mail
contains patient information, please contact the Partners Compliance HelpLine at
http://www.partners.org/complianceline . If the e-mail was sent to you in error
but does not contain patient information, please contact the sender and properly
dispose of the e-mail.

From ask at develooper.com  Mon Feb 14 18:18:29 2011
From: ask at develooper.com (=?iso-8859-1?Q?Ask_Bj=F8rn_Hansen?=)
Date: Mon, 14 Feb 2011 10:18:29 -0800
Subject: virtual host remapping.
In-Reply-To: <alpine.LRH.2.00.1102140925180.1249@maequeses.nmr.mgh.harvard.edu>
References: <356FD748-D91A-4CFE-BD6F-992841A1CC7F@comicrelief.com>
	<AANLkTinL=9Niy__2CNNW-toGYTUO3KX1K-+EN4_eMXC9@mail.gmail.com>
	<alpine.LRH.2.00.1102140925180.1249@maequeses.nmr.mgh.harvard.edu>
Message-ID: <C62C880F-C827-45D2-B5B9-2FF4690DD654@develooper.com>


On Feb 14, 2011, at 6:42, Chris Johnson wrote:

At a glance your VCL looks right.  That being said, I'd suggest you just forward the old site with 301 redirects to the new URL.


 - ask


From johnson at nmr.mgh.harvard.edu  Mon Feb 14 18:34:20 2011
From: johnson at nmr.mgh.harvard.edu (Chris Johnson)
Date: Mon, 14 Feb 2011 13:34:20 -0500 (EST)
Subject: virtual host remapping.
In-Reply-To: <C62C880F-C827-45D2-B5B9-2FF4690DD654@develooper.com>
References: <356FD748-D91A-4CFE-BD6F-992841A1CC7F@comicrelief.com>
	<AANLkTinL=9Niy__2CNNW-toGYTUO3KX1K-+EN4_eMXC9@mail.gmail.com>
	<alpine.LRH.2.00.1102140925180.1249@maequeses.nmr.mgh.harvard.edu>
	<C62C880F-C827-45D2-B5B9-2FF4690DD654@develooper.com>
Message-ID: <alpine.LRH.2.00.1102141331070.1249@maequeses.nmr.mgh.harvard.edu>

       Thank you VERY much.  I completely missed that.  I don't even
really need to muck with varnish this way.  It will help but time
should take care of most of the problem either way.

>
> On Feb 14, 2011, at 6:42, Chris Johnson wrote:
>
> At a glance your VCL looks right.  That being said, I'd suggest you just forward the old site with 301 redirects to the new URL.
>
>
> - ask
>
>
>

-------------------------------------------------------------------------------
Chris Johnson               |Internet: johnson at nmr.mgh.harvard.edu
Systems Administrator       |Web:      http://www.nmr.mgh.harvard.edu/~johnson
NMR Center                  |Voice:    617.726.0949
Mass. General Hospital      |FAX:      617.726.7422
149 (2301) 13th Street      |If God has a plan for my life then I want to have
Charlestown, MA., 02129 USA |a little chat with the architect.  Me
-------------------------------------------------------------------------------


The information in this e-mail is intended only for the person to whom it is
addressed. If you believe this e-mail was sent to you in error and the e-mail
contains patient information, please contact the Partners Compliance HelpLine at
http://www.partners.org/complianceline . If the e-mail was sent to you in error
but does not contain patient information, please contact the sender and properly
dispose of the e-mail.




From cooltechemail at gmail.com  Tue Feb 15 13:07:01 2011
From: cooltechemail at gmail.com (Vince)
Date: Tue, 15 Feb 2011 05:07:01 -0800
Subject: Dynamically Extend Grace Period
Message-ID: <AANLkTik5oKyEc_6uw4dgxB-7LwFK_QYCZuUu6YtnC2rG@mail.gmail.com>

Hi All,

I am wondering if I can dynamically extend grace period. For example,
currently I have beresp.grace = 10m. If something bad happens to the backend
and it needs more than 10 minutes to recover, 10m grace period is not
enough. Is there anyway to dynamically increase the grace period, say 2
hours, so varnish can continue to serve old content until the backend is
available again.

I tried to modify the grace period in the vcl and then reload the vcl.
However, it doesn't work. What's even worse is that reload the vcl will
remove all old content stored in varnish.

Thanks,

Vince
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110215/25e0da54/attachment.html>

From perbu at varnish-software.com  Tue Feb 15 13:53:31 2011
From: perbu at varnish-software.com (Per Buer)
Date: Tue, 15 Feb 2011 14:53:31 +0100
Subject: Dynamically Extend Grace Period
In-Reply-To: <AANLkTik5oKyEc_6uw4dgxB-7LwFK_QYCZuUu6YtnC2rG@mail.gmail.com>
References: <AANLkTik5oKyEc_6uw4dgxB-7LwFK_QYCZuUu6YtnC2rG@mail.gmail.com>
Message-ID: <AANLkTikbidTjoYtKyXGu5EonsFoEsBpA+Jd5-Oduhha2@mail.gmail.com>

Hi,

On Tue, Feb 15, 2011 at 2:07 PM, Vince <cooltechemail at gmail.com> wrote:
> Hi All,
>
> I am wondering if I can dynamically extend grace period. For example,
> currently I have beresp.grace = 10m. If something bad happens to the backend
> and it needs more than 10 minutes to recover, 10m grace period is not
> enough. Is there anyway to dynamically increase the grace period, say 2
> hours, so varnish can continue to serve old content until the backend is
> available again.

Sort of. You could set the grace period to, say 2 hours, on the beresp
object and then set the req.grace period to 10min. Then, when
something bad hits the fan you increase the grace period in vcl_recv.


-- 
Per Buer,?Varnish Software
Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer
Varnish makes websites fly!
Want to learn more about Varnish? http://www.varnish-software.com/whitepapers



From alexus at gmail.com  Tue Feb 15 18:15:57 2011
From: alexus at gmail.com (alexus)
Date: Tue, 15 Feb 2011 13:15:57 -0500
Subject: Block IP with Varnish
Message-ID: <AANLkTinya1HnYysFWSc+KNCjeu9bF65=FduV3pCfJUQF@mail.gmail.com>

How do I block IP address and/or range of IP address(es) using Varnish?

# varnishd -V
varnishd (varnish-2.1.3 SVN )
Copyright (c) 2006-2009 Linpro AS / Verdens Gang AS
# uname -a
Linux foo.bar.com 2.6.18-238.1.1.el5 #1 SMP Tue Jan 4 13:32:19 EST
2011 x86_64 x86_64 x86_64 GNU/Linux
#


-- 
http://alexus.org/



From crcook at gmail.com  Tue Feb 15 19:03:33 2011
From: crcook at gmail.com (Chris Cook)
Date: Tue, 15 Feb 2011 14:03:33 -0500
Subject: Block IP with Varnish
In-Reply-To: <AANLkTinya1HnYysFWSc+KNCjeu9bF65=FduV3pCfJUQF@mail.gmail.com>
References: <AANLkTinya1HnYysFWSc+KNCjeu9bF65=FduV3pCfJUQF@mail.gmail.com>
Message-ID: <0224DE2D-D17A-433A-A672-EB769B216892@gmail.com>

Wouldn't it be easier (and more appropriate) to use iptables or some  
other appropriate FW program?

On Feb 15, 2011, at 1:15 PM, alexus wrote:

> How do I block IP address and/or range of IP address(es) using  
> Varnish?
>
> # varnishd -V
> varnishd (varnish-2.1.3 SVN )
> Copyright (c) 2006-2009 Linpro AS / Verdens Gang AS
> # uname -a
> Linux foo.bar.com 2.6.18-238.1.1.el5 #1 SMP Tue Jan 4 13:32:19 EST
> 2011 x86_64 x86_64 x86_64 GNU/Linux
> #
>
>
> -- 
> http://alexus.org/
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc




From alexus at gmail.com  Tue Feb 15 19:53:36 2011
From: alexus at gmail.com (alexus)
Date: Tue, 15 Feb 2011 14:53:36 -0500
Subject: Block IP with Varnish
In-Reply-To: <0224DE2D-D17A-433A-A672-EB769B216892@gmail.com>
References: <AANLkTinya1HnYysFWSc+KNCjeu9bF65=FduV3pCfJUQF@mail.gmail.com>
	<0224DE2D-D17A-433A-A672-EB769B216892@gmail.com>
Message-ID: <AANLkTikZQK5eL=D-jt6GLpjLUQ0kYJsp-pCiou=6yQue@mail.gmail.com>

probably would... but I'm at that point where I want to block certain
agents and not just necessarily IP, so Varnish gives me much more
flexibility vs iptables

On Tue, Feb 15, 2011 at 2:03 PM, Chris Cook <crcook at gmail.com> wrote:
> Wouldn't it be easier (and more appropriate) to use iptables or some other
> appropriate FW program?
>
> On Feb 15, 2011, at 1:15 PM, alexus wrote:
>
>> How do I block IP address and/or range of IP address(es) using Varnish?
>>
>> # varnishd -V
>> varnishd (varnish-2.1.3 SVN )
>> Copyright (c) 2006-2009 Linpro AS / Verdens Gang AS
>> # uname -a
>> Linux foo.bar.com 2.6.18-238.1.1.el5 #1 SMP Tue Jan 4 13:32:19 EST
>> 2011 x86_64 x86_64 x86_64 GNU/Linux
>> #
>>
>>
>> --
>> http://alexus.org/
>>
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at varnish-cache.org
>> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
>



-- 
http://alexus.org/



From j.begumisa at gmail.com  Tue Feb 15 20:14:47 2011
From: j.begumisa at gmail.com (Joseph Begumisa)
Date: Tue, 15 Feb 2011 12:14:47 -0800
Subject: Block IP with Varnish
In-Reply-To: <AANLkTikZQK5eL=D-jt6GLpjLUQ0kYJsp-pCiou=6yQue@mail.gmail.com>
References: <AANLkTinya1HnYysFWSc+KNCjeu9bF65=FduV3pCfJUQF@mail.gmail.com>
	<0224DE2D-D17A-433A-A672-EB769B216892@gmail.com>
	<AANLkTikZQK5eL=D-jt6GLpjLUQ0kYJsp-pCiou=6yQue@mail.gmail.com>
Message-ID: <AANLkTik0J4OHJCbkqPSNiP=Y9ptpUzmB3621VsPhSa98@mail.gmail.com>

On Tue, Feb 15, 2011 at 11:53 AM, alexus <alexus at gmail.com> wrote:
> probably would... but I'm at that point where I want to block certain
> agents and not just necessarily IP, so Varnish gives me much more
> flexibility vs iptables
>

Create the acls containing the list of ip addresses or networks:

acl forbidden {
        "192.168.168.0"/24;
        "10.10.10.0"/24;
}
acl useragents {
        "192.168.200.0"/24;
        "10.10.9.0"/24;
}

Then add the following / modify as appropriate under sub vcl_recv

sub vcl_recv {

       # Block access from these hosts
        if (client.ip ~ forbidden) {
        error 403 "Forbidden";
        }

        # Block Access from these user agents
        if (req.http.User-Agent ~ "Wget" {
                        error 403 "Forbidden";
        }
                return (lookup);
 }


Joseph



From scott at cebotics.com  Wed Feb 16 02:55:22 2011
From: scott at cebotics.com (Scott Leonard)
Date: Tue, 15 Feb 2011 21:55:22 -0500
Subject: Top IP addresses
Message-ID: <400F771396314592BD26B139071D6909@XPS9100>

Is there anyway to use varnishtop or varnishlog to display a list of aggregated counts of (top) request IP addresses much in the same way that you can do for referrers or user agents (e.g. varnishtop -i RxHeader -C -I ^Referer)?

I don?t see ?IP? as a tag in varnish top to use for this and also can?t seem to persuade varnishlog to generate the list.

Thanks,
Scott
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110215/308c856d/attachment.html>

From wido at widodh.nl  Wed Feb 16 09:22:38 2011
From: wido at widodh.nl (Wido den Hollander)
Date: Wed, 16 Feb 2011 10:22:38 +0100
Subject: Top IP addresses
In-Reply-To: <400F771396314592BD26B139071D6909@XPS9100>
References: <400F771396314592BD26B139071D6909@XPS9100>
Message-ID: <1297848158.2603.1.camel@wido-laptop.pcextreme.nl>

Hi Scott,

On Tue, 2011-02-15 at 21:55 -0500, Scott Leonard wrote:
> the same way that you can do for referrers or user agents (e.g.
> varnishtop -i RxHeader -C -I ^Referer)?

You could try: varnishlog -i SessionOpen

With some magic with awk, sort and uniq you should be able to generate a
top-list of IP-Addresses.

Wido




From Antony.A.Xynidakis at team.telstra.com  Tue Feb 15 21:57:47 2011
From: Antony.A.Xynidakis at team.telstra.com (Xynidakis, Tony A)
Date: Wed, 16 Feb 2011 08:57:47 +1100
Subject: caching behaviour for root domains (homepage)
Message-ID: <4ADDD2718038414E91589285477BA11C44D85D134E@WSMSG3152V.srv.dir.telstra.com>

Hi all
This is my first post and still learning Varnish

in the configs for varnish, we set up the caching rules not to cache .aspx pages.

our home page is the root domain  http://v8supercars.com.au<http://v8supercars.com.au>

in the varnish trace the request is 'GET /' which makes sense but without the .aspx extension it's doesn't get fall into the .aspx rule that we have.

is there additional rules that we need to add into the configs.

Kind regards

Tony

Tony Xynidakis
Content and Media - BigPond
Phone *: (03) 9632 8161
Fax *: (03) 9600 1235
Mobile *: 0419 110 409
Email *: txynidakis at team.telstra.com<mailto:txynidakis at team.telstra.com>
Address : Level 32, 300 Latrobe St, Melbourne, Vic, 3000

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110216/b8411d6b/attachment.html>

From david at firechaser.com  Wed Feb 16 13:37:59 2011
From: david at firechaser.com (David Murphy)
Date: Wed, 16 Feb 2011 13:37:59 +0000
Subject: caching behaviour for root domains (homepage)
In-Reply-To: <4ADDD2718038414E91589285477BA11C44D85D134E@WSMSG3152V.srv.dir.telstra.com>
References: <4ADDD2718038414E91589285477BA11C44D85D134E@WSMSG3152V.srv.dir.telstra.com>
Message-ID: <AANLkTikFrVWf=CDZppev8pGb35Uq6EebEWuDXz3fLCUz@mail.gmail.com>

 I'm new too :) , so not sure if this is the best way, but something
like this might work?

sub vcl_recv {
   if (  req.url ~ "\.(aspx)$"  || req.url ~ "^/$" ) {
      # don't lookup cache for .aspx pages or root /
      return(pass);
   } else {
     return(lookup);
   }
}

Best, David

On Tue, Feb 15, 2011 at 9:57 PM, Xynidakis, Tony A
<Antony.A.Xynidakis at team.telstra.com> wrote:
> Hi all
>
> This is my first post and still learning Varnish
>
> in the configs for varnish, we set up the caching rules not to cache .aspx
> pages.
>
> our home page is the root domain ?http://v8supercars.com.au
>
> in the varnish trace the request is 'GET /' which makes sense but without
> the .aspx extension it's doesn't get fall into the .aspx rule that we have.
>
> is there additional rules that we need to add into the configs.
>
> Kind regards
>
> Tony
>
>
>
> Tony Xynidakis
> Content and Media - BigPond
> Phone (: (03) 9632 8161
> Fax 1: (03) 9600 1235
> Mobile (: 0419 110 409
> Email *: txynidakis at team.telstra.com
> Address : Level 32, 300 Latrobe St, Melbourne, Vic, 3000
>
>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>



From scaunter at topscms.com  Wed Feb 16 15:49:33 2011
From: scaunter at topscms.com (Caunter, Stefan)
Date: Wed, 16 Feb 2011 10:49:33 -0500
Subject: caching behaviour for root domains (homepage)
In-Reply-To: <4ADDD2718038414E91589285477BA11C44D85D134E@WSMSG3152V.srv.dir.telstra.com>
References: <4ADDD2718038414E91589285477BA11C44D85D134E@WSMSG3152V.srv.dir.telstra.com>
Message-ID: <7F0AA702B8A85A4A967C4C8EBAD6902CEB70AE@TMG-EVS02.torstar.net>

You need in vcl_recv

 

if (req.url ~ "^/$"){

                set req.url = "default.aspx";

}

 

and then handle it the way you want.

 

Stefan Caunter 

Operations

Torstar Digital

m: (416) 561-4871

 

 

From: varnish-misc-bounces at varnish-cache.org
[mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Xynidakis,
Tony A
Sent: February-15-11 4:58 PM
To: varnish-misc at varnish-cache.org
Subject: caching behaviour for root domains (homepage)

 

Hi all 

This is my first post and still learning Varnish

in the configs for varnish, we set up the caching rules not to cache
.aspx pages. 

our home page is the root domain  http://v8supercars.com.au
<http://v8supercars.com.au>  

in the varnish trace the request is 'GET /' which makes sense but
without the .aspx extension it's doesn't get fall into the .aspx rule
that we have. 

is there additional rules that we need to add into the configs. 

Kind regards 

Tony 

 

Tony Xynidakis
Content and Media - BigPond
Phone (: (03) 9632 8161
Fax 1: (03) 9600 1235
Mobile (: 0419 110 409
Email *: txynidakis at team.telstra.com
<mailto:txynidakis at team.telstra.com> 
Address : Level 32, 300 Latrobe St, Melbourne, Vic, 3000 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110216/0e4ef2e0/attachment.html>

From alexus at gmail.com  Wed Feb 16 20:37:25 2011
From: alexus at gmail.com (alexus)
Date: Wed, 16 Feb 2011 15:37:25 -0500
Subject: DDOS
Message-ID: <AANLkTimyMj0Ymyrdp7_X5J1NvZnoamhWQ_O38LHAT4u-@mail.gmail.com>

How does Varnish handles DDOS?

here is my problem, in our environment varnish servers static pages
but dynamic pages it passed to application server (apache).
so every onces in a while we have some attacker(s) who start doing
some sort of attacks against us, and apache hits very high load on
server and about to go down...
so I look up an IP address and I block it on Varnish .vcl file, and
load goes back to normal right away...
is there something that can be done automatically? or is there
anything can be done at all to somehow handle this type of issue?

please advise

-- 
http://alexus.org/



From frank at huddler-inc.com  Wed Feb 16 21:31:20 2011
From: frank at huddler-inc.com (Frank Farmer)
Date: Wed, 16 Feb 2011 13:31:20 -0800
Subject: Probes stop for minutes once backend is decared unhealty?
Message-ID: <AANLkTikr5qa-OF-4xTcttSc_ettX5H7-hApV3wbboc7t@mail.gmail.com>

I'm having an issue where my backends fall behind for a little while
(maybe 30 seconds or so), and then varnish stops probing entirely for
minutes at a time, even though the host has long since recovered.  I'm
near capacity, currently, so I can't afford to lose a backend for
minutes -- the extra traffic tends to back up my other backends, which
then also end up taken out of the pool for minutes, even though they
recover in seconds.

Is there anything I can do to control this interval?  I'd love to have
varnish never wait more than, say, 30 seconds between probes, even at
the worst of times.

Varnish version:

# varnishd -V
varnishd (varnish-2.1.3 SVN 5049:5055)
Copyright (c) 2006-2009 Linpro AS / Verdens Gang AS

Excerpt of Apache access_log showing probes received by one of my backends:

# ... NORMAL PROBE FREQUENCY
app005 - - - [16/Feb/2011:18:44:00 +0000] "GET /health.html HTTP/1.1"
200 24 "-" "-"
app005 - - - [16/Feb/2011:18:44:01 +0000] "GET /health.html HTTP/1.1"
200 24 "-" "-"
app005 - - - [16/Feb/2011:18:44:02 +0000] "GET /health.html HTTP/1.1"
200 24 "-" "-"
app005 - - - [16/Feb/2011:18:44:04 +0000] "GET /health.html HTTP/1.1"
200 24 "-" "-"
# PROBES STOP FOR 2 MINUTES
app005 - - - [16/Feb/2011:18:46:07 +0000] "GET /health.html HTTP/1.1"
200 24 "-" "-"
app005 - - - [16/Feb/2011:18:46:07 +0000] "GET /health.html HTTP/1.1"
200 24 "-" "-"
app005 - - - [16/Feb/2011:18:46:07 +0000] "GET /health.html HTTP/1.1"
200 24 "-" "-"
app005 - - - [16/Feb/2011:18:46:07 +0000] "GET /health.html HTTP/1.1"
200 24 "-" "-"
app005 - - - [16/Feb/2011:18:46:07 +0000] "GET /health.html HTTP/1.1"
200 24 "-" "-"
app005 - - - [16/Feb/2011:18:46:09 +0000] "GET /health.html HTTP/1.1"
200 24 "-" "-"
app005 - - - [16/Feb/2011:18:46:09 +0000] "GET /health.html HTTP/1.1"
200 24 "-" "-"
app005 - - - [16/Feb/2011:18:46:09 +0000] "GET /health.html HTTP/1.1"
200 24 "-" "-"
app005 - - - [16/Feb/2011:18:46:09 +0000] "GET /health.html HTTP/1.1"
200 24 "-" "-"
app005 - - - [16/Feb/2011:18:46:10 +0000] "GET /health.html HTTP/1.1"
200 24 "-" "-"
app005 - - - [16/Feb/2011:18:46:10 +0000] "GET /health.html HTTP/1.1"
200 24 "-" "-"
app005 - - - [16/Feb/2011:18:46:11 +0000] "GET /health.html HTTP/1.1"
200 24 "-" "-"
app005 - - - [16/Feb/2011:18:46:12 +0000] "GET /health.html HTTP/1.1"
200 24 "-" "-"
app005 - - - [16/Feb/2011:18:46:13 +0000] "GET /health.html HTTP/1.1"
200 24 "-" "-"
app005 - - - [16/Feb/2011:18:46:14 +0000] "GET /health.html HTTP/1.1"
200 24 "-" "-"
app005 - - - [16/Feb/2011:18:46:15 +0000] "GET /health.html HTTP/1.1"
200 24 "-" "-"
app005 - - - [16/Feb/2011:18:46:16 +0000] "GET /health.html HTTP/1.1"
200 24 "-" "-"
app005 - - - [16/Feb/2011:18:46:17 +0000] "GET /health.html HTTP/1.1"
200 24 "-" "-"
app005 - - - [16/Feb/2011:18:46:18 +0000] "GET /health.html HTTP/1.1"
200 24 "-" "-"
app005 - - - [16/Feb/2011:18:46:19 +0000] "GET /health.html HTTP/1.1"
200 24 "-" "-"
app005 - - - [16/Feb/2011:18:46:22 +0000] "GET /health.html HTTP/1.1"
200 24 "-" "-"
# PROBES STOP FOR 2 MINUTES
app005 - - - [16/Feb/2011:18:48:23 +0000] "GET /health.html HTTP/1.1"
200 24 "-" "-"
app005 - - - [16/Feb/2011:18:48:23 +0000] "GET /health.html HTTP/1.1"
200 24 "-" "-"
# ... SNIP SEVERAL MINUTES OF NORMAL PROBING
app005 - - - [16/Feb/2011:18:54:33 +0000] "GET /health.html HTTP/1.1"
200 24 "-" "-"
app005 - - - [16/Feb/2011:18:54:33 +0000] "GET /health.html HTTP/1.1"
200 24 "-" "-"
app005 - - - [16/Feb/2011:18:54:33 +0000] "GET /health.html HTTP/1.1"
200 24 "-" "-"
# PROBES STOP FOR 7 MINUTES
app005 - - - [16/Feb/2011:19:01:33 +0000] "GET /health.html HTTP/1.1"
200 24 "-" "-"
app005 - - - [16/Feb/2011:19:01:45 +0000] "GET /health.html HTTP/1.1"
200 24 "-" "-"
app005 - - - [16/Feb/2011:19:01:46 +0000] "GET /health.html HTTP/1.1"
200 24 "-" "-"
# ...PROBES OPERATE NORMALLY FOR HOURS...


Backend config:


backend app001 {
  .host = "app001-private";
  .port = "8880";
  .probe = {
    .url = "/health.html";
    .timeout = 2s;
    .interval = 1s;
    .window = 10;
    .threshold = 8;
  }
  .connect_timeout = 2s;
}

backend app002 {
  .host = "app002-private";
  .port = "8880";
  .probe = {
    .url = "/health.html";
    .timeout = 2s;
    .interval = 1s;
    .window = 10;
    .threshold = 8;
  }
  .connect_timeout = 2s;
}

backend app003 {
  .host = "app003-private";
  .port = "8880";
  .probe = {
    .url = "/health.html";
    .timeout = 2s;
    .interval = 1s;
    .window = 10;
    .threshold = 8;
  }
  .connect_timeout = 2s;
}

backend app005 {
  .host = "app005-private";
  .port = "8880";
  .probe = {
    .url = "/health.html";
    .timeout = 2s;
    .interval = 1s;
    .window = 10;
    .threshold = 5;
    .initial = 10;
  }
  .connect_timeout = 2s;
}

backend app006 {
  .host = "app006-private";
  .port = "8880";
  .probe = {
    .url = "/health.html";
    .timeout = 2s;
    .interval = 1s;
    .window = 10;
    .threshold = 5;
    .initial = 10;
  }
  .connect_timeout = 2s;
}

director app_servers random {
	{
		.backend = app001;
		.weight = 10;
	}
	{
		.backend = app002;
		.weight = 100;
	}
	{
		.backend = app003;
		.weight = 75;
	}
	{
		.backend = app005;
		.weight = 300;
	}
	{
		.backend = app006;
		.weight = 300;
	}
}



From stig at zedge.net  Wed Feb 16 22:20:39 2011
From: stig at zedge.net (Stig Bakken)
Date: Wed, 16 Feb 2011 23:20:39 +0100
Subject: DDOS
In-Reply-To: <AANLkTimyMj0Ymyrdp7_X5J1NvZnoamhWQ_O38LHAT4u-@mail.gmail.com>
References: <AANLkTimyMj0Ymyrdp7_X5J1NvZnoamhWQ_O38LHAT4u-@mail.gmail.com>
Message-ID: <AANLkTikeWF69YhfTc2MoUFCJgTQMPcnok1iaCPx5o-mq@mail.gmail.com>

On Wed, Feb 16, 2011 at 9:37 PM, alexus <alexus at gmail.com> wrote:

> How does Varnish handles DDOS?
>
> here is my problem, in our environment varnish servers static pages
> but dynamic pages it passed to application server (apache).
> so every onces in a while we have some attacker(s) who start doing
> some sort of attacks against us, and apache hits very high load on
> server and about to go down...
> so I look up an IP address and I block it on Varnish .vcl file, and
> load goes back to normal right away...
> is there something that can be done automatically? or is there
> anything can be done at all to somehow handle this type of issue?
>
> please advise


Usually you would handle this in your firewall.  Varnish is a fine hammer,
but for some things screwdrivers still rule.
(Btw, if blocking IP addresses does the trick for you, it's just a DOS, not
a DDOS.)

 - Stig



-- 
Stig Bakken
CTO, Zedge.net - free your phone!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110216/5ed54592/attachment.html>

From alexus at gmail.com  Wed Feb 16 23:01:06 2011
From: alexus at gmail.com (alexus)
Date: Wed, 16 Feb 2011 18:01:06 -0500
Subject: DDOS
In-Reply-To: <AANLkTikeWF69YhfTc2MoUFCJgTQMPcnok1iaCPx5o-mq@mail.gmail.com>
References: <AANLkTimyMj0Ymyrdp7_X5J1NvZnoamhWQ_O38LHAT4u-@mail.gmail.com>
	<AANLkTikeWF69YhfTc2MoUFCJgTQMPcnok1iaCPx5o-mq@mail.gmail.com>
Message-ID: <AANLkTin9K_5KQLe6KJhWF+85arGtXU6ssVaixfB=xJmq@mail.gmail.com>

right, in this case it was a single IP ... well maybe not single but
not that many to consider that as DDOS...
my question how does Varnish handles this kind or problems at all...

On Wed, Feb 16, 2011 at 5:20 PM, Stig Bakken <stig at zedge.net> wrote:
> On Wed, Feb 16, 2011 at 9:37 PM, alexus <alexus at gmail.com> wrote:
>>
>> How does Varnish handles DDOS?
>>
>> here is my problem, in our environment varnish servers static pages
>> but dynamic pages it passed to application server (apache).
>> so every onces in a while we have some attacker(s) who start doing
>> some sort of attacks against us, and apache hits very high load on
>> server and about to go down...
>> so I look up an IP address and I block it on Varnish .vcl file, and
>> load goes back to normal right away...
>> is there something that can be done automatically? or is there
>> anything can be done at all to somehow handle this type of issue?
>>
>> please advise
>
> Usually you would handle this in your firewall. ?Varnish is a fine hammer,
> but for some things screwdrivers still rule.
> (Btw, if blocking IP addresses does the trick for you, it's just a DOS, not
> a DDOS.)
>
> ?- Stig
>
> --
> Stig Bakken
> CTO, Zedge.net - free your phone!
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>



-- 
http://alexus.org/



From Nicholas_Maesepp at scee.net  Wed Feb 16 23:13:16 2011
From: Nicholas_Maesepp at scee.net (Nicholas_Maesepp at scee.net)
Date: Thu, 17 Feb 2011 09:13:16 +1000
Subject: DDOS
In-Reply-To: <AANLkTimyMj0Ymyrdp7_X5J1NvZnoamhWQ_O38LHAT4u-@mail.gmail.com>
References: <AANLkTimyMj0Ymyrdp7_X5J1NvZnoamhWQ_O38LHAT4u-@mail.gmail.com>
Message-ID: <OF8A51CAF6.87572723-ONCA257839.007E7BF1-CA257839.007FA542@scee.net>

As someone else said best to use a firewall, if you want to use a simple 
one just run iptables, since this is only DoS you best to use rate limit 
connections per IP, depending on the number of objects etc you should be 
able to get away with keeping connections to a fairly low number and not 
hampering browsing experience at all. It would need to move to being DDoS 
for it to bring it down. 

Only cheaty way I can think of doing it in the vcl is to define multiple 
backends one with no max_connections and one with a fairly low 
max_connections. Define some ip acl's using netblocks. If you feel all of 
the hammering comes from china and very little legit web traffic you could 
find their netblocks and force them to use the one with a set limit on 
.max_connections while others dont have such limits. Vice versa, if you 
expect 90% of traffic to be within your own country allow those to connect 
to the backend with higher max_connections and have everyone else connect 
with a low number. If someone tries to hammer your page it will just 
starve those outside your target audience but keep your backend healthy 
and your regular visitors fine. Lots of caveats in that and wouldn't 
recommend it, purely for science. Definitely use a firewall or other 
tools.

Nick




From:   alexus <alexus at gmail.com>
To:     varnish-misc at varnish-cache.org
Date:   02/17/2011 07:37 AM
Subject:        DDOS
Sent by:        varnish-misc-bounces at varnish-cache.org



How does Varnish handles DDOS?

here is my problem, in our environment varnish servers static pages
but dynamic pages it passed to application server (apache).
so every onces in a while we have some attacker(s) who start doing
some sort of attacks against us, and apache hits very high load on
server and about to go down...
so I look up an IP address and I block it on Varnish .vcl file, and
load goes back to normal right away...
is there something that can be done automatically? or is there
anything can be done at all to somehow handle this type of issue?

please advise

-- 
http://alexus.org/

_______________________________________________
varnish-misc mailing list
varnish-misc at varnish-cache.org
http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc




Sony Computer Entertainment Australia Pty Ltd
Level 1, 63-73 Ann Street Surry Hills NSW 2010
P.O. Box 5023 Darlinghurst NSW 2010
ph: +61 (0)2 9324 9500 fax: +61 (0)2 9324 9558
http://au.playstation.com
http://www.facebook.com/PlayStationAU


New tools, new games, new fun.
Sackboy returns in LittleBigPlanet 2
Out now and exclusive to PlayStation 3




**********************************************************************
This email and any files transmitted with it are confidential and intended 
solely for the use of the individual or entity to whom they are addressed. 
If you have received this email in error please notify postmaster at scee.net
This footnote also confirms that this email message has been checked for 
all known viruses.
Sony Computer Entertainment Australia Pty. Limited
Registered Office: Level 1, 63-73 Ann Street, Surry Hills, NSW 2010 
Australia
Registered in Australia: 077 583 183
**********************************************************************

P Please consider the environment before printing this e-mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110217/90ad53d7/attachment.html>

From scaunter at topscms.com  Thu Feb 17 03:41:27 2011
From: scaunter at topscms.com (Caunter, Stefan)
Date: Wed, 16 Feb 2011 22:41:27 -0500
Subject: DDOS
In-Reply-To: <OF8A51CAF6.87572723-ONCA257839.007E7BF1-CA257839.007FA542@scee.net>
References: <AANLkTimyMj0Ymyrdp7_X5J1NvZnoamhWQ_O38LHAT4u-@mail.gmail.com>
	<OF8A51CAF6.87572723-ONCA257839.007E7BF1-CA257839.007FA542@scee.net>
Message-ID: <BE2E0B80-E286-425B-9FDA-909C5F9F5286@topscms.com>

Increment a header and send an error if limit is reached from client.ip

Stefan Caunter
Operations
TorstarDigital
416.561.4871

On 2011-02-16, at 6:14 PM, "Nicholas_Maesepp at scee.net" <Nicholas_Maesepp at scee.net> wrote:

> As someone else said best to use a firewall, if you want to use a simple one just run iptables, since this is only DoS you best to use rate limit connections per IP, depending on the number of objects etc you should be able to get away with keeping connections to a fairly low number and not hampering browsing experience at all. It would need to move to being DDoS for it to bring it down. 
> 
> Only cheaty way I can think of doing it in the vcl is to define multiple backends one with no max_connections and one with a fairly low max_connections. Define some ip acl's using netblocks. If you feel all of the hammering comes from china and very little legit web traffic you could find their netblocks and force them to use the one with a set limit on .max_connections while others dont have such limits. Vice versa, if you expect 90% of traffic to be within your own country allow those to connect to the backend with higher max_connections and have everyone else connect with a low number. If someone tries to hammer your page it will just starve those outside your target audience but keep your backend healthy and your regular visitors fine. Lots of caveats in that and wouldn't recommend it, purely for science. Definitely use a firewall or other tools. 
> 
> Nick 
> 
> 
> 
> 
> From:        alexus <alexus at gmail.com> 
> To:        varnish-misc at varnish-cache.org 
> Date:        02/17/2011 07:37 AM 
> Subject:        DDOS 
> Sent by:        varnish-misc-bounces at varnish-cache.org 
> 
> 
> 
> How does Varnish handles DDOS?
> 
> here is my problem, in our environment varnish servers static pages
> but dynamic pages it passed to application server (apache).
> so every onces in a while we have some attacker(s) who start doing
> some sort of attacks against us, and apache hits very high load on
> server and about to go down...
> so I look up an IP address and I block it on Varnish .vcl file, and
> load goes back to normal right away...
> is there something that can be done automatically? or is there
> anything can be done at all to somehow handle this type of issue?
> 
> please advise
> 
> -- 
> http://alexus.org/
> 
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
> 
> 
> 
> Sony Computer Entertainment Australia Pty Ltd
> Level 1, 63-73 Ann Street Surry Hills NSW 2010
> P.O. Box 5023 Darlinghurst NSW 2010
> ph: +61 (0)2 9324 9500 fax: +61 (0)2 9324 9558
> 
> http://au.playstation.com
> http://www.facebook.com/PlayStationAU
> 
> 
> 
> New tools, new games, new fun.
> Sackboy returns in LittleBigPlanet 2
> Out now and exclusive to PlayStation 3
> 
> 
> 
> 
> **********************************************************************
> This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify postmaster at scee.net
> This footnote also confirms that this email message has been checked for all known viruses.
> Sony Computer Entertainment Australia Pty. Limited
> Registered Office: Level 1, 63-73 Ann Street, Surry Hills, NSW 2010 Australia
> Registered in Australia: 077 583 183
> **********************************************************************
> 
> P Please consider the environment before printing this e-mail
> 
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110216/79199b2a/attachment.html>

From scaunter at topscms.com  Thu Feb 17 03:45:36 2011
From: scaunter at topscms.com (Caunter, Stefan)
Date: Wed, 16 Feb 2011 22:45:36 -0500
Subject: Probes stop for minutes once backend is decared unhealty?
In-Reply-To: <AANLkTikr5qa-OF-4xTcttSc_ettX5H7-hApV3wbboc7t@mail.gmail.com>
References: <AANLkTikr5qa-OF-4xTcttSc_ettX5H7-hApV3wbboc7t@mail.gmail.com>
Message-ID: <027A837F-1AD8-4349-8851-3862E332744D@topscms.com>

Your probes are very impatient to my eye. 2 sec timeout is aggressive. You can set a more patient view of your backends in another director, and restart if a request failed, sending to the slower but more likely to be available backends. Use 60 sec timeout on the patient view. 

Stefan Caunter
Operations
TorstarDigital
416.561.4871

On 2011-02-16, at 4:31 PM, "Frank Farmer" <frank at huddler-inc.com> wrote:

> I'm having an issue where my backends fall behind for a little while
> (maybe 30 seconds or so), and then varnish stops probing entirely for
> minutes at a time, even though the host has long since recovered.  I'm
> near capacity, currently, so I can't afford to lose a backend for
> minutes -- the extra traffic tends to back up my other backends, which
> then also end up taken out of the pool for minutes, even though they
> recover in seconds.
> 
> Is there anything I can do to control this interval?  I'd love to have
> varnish never wait more than, say, 30 seconds between probes, even at
> the worst of times.
> 
> Varnish version:
> 
> # varnishd -V
> varnishd (varnish-2.1.3 SVN 5049:5055)
> Copyright (c) 2006-2009 Linpro AS / Verdens Gang AS
> 
> Excerpt of Apache access_log showing probes received by one of my backends:
> 
> # ... NORMAL PROBE FREQUENCY
> app005 - - - [16/Feb/2011:18:44:00 +0000] "GET /health.html HTTP/1.1"
> 200 24 "-" "-"
> app005 - - - [16/Feb/2011:18:44:01 +0000] "GET /health.html HTTP/1.1"
> 200 24 "-" "-"
> app005 - - - [16/Feb/2011:18:44:02 +0000] "GET /health.html HTTP/1.1"
> 200 24 "-" "-"
> app005 - - - [16/Feb/2011:18:44:04 +0000] "GET /health.html HTTP/1.1"
> 200 24 "-" "-"
> # PROBES STOP FOR 2 MINUTES
> app005 - - - [16/Feb/2011:18:46:07 +0000] "GET /health.html HTTP/1.1"
> 200 24 "-" "-"
> app005 - - - [16/Feb/2011:18:46:07 +0000] "GET /health.html HTTP/1.1"
> 200 24 "-" "-"
> app005 - - - [16/Feb/2011:18:46:07 +0000] "GET /health.html HTTP/1.1"
> 200 24 "-" "-"
> app005 - - - [16/Feb/2011:18:46:07 +0000] "GET /health.html HTTP/1.1"
> 200 24 "-" "-"
> app005 - - - [16/Feb/2011:18:46:07 +0000] "GET /health.html HTTP/1.1"
> 200 24 "-" "-"
> app005 - - - [16/Feb/2011:18:46:09 +0000] "GET /health.html HTTP/1.1"
> 200 24 "-" "-"
> app005 - - - [16/Feb/2011:18:46:09 +0000] "GET /health.html HTTP/1.1"
> 200 24 "-" "-"
> app005 - - - [16/Feb/2011:18:46:09 +0000] "GET /health.html HTTP/1.1"
> 200 24 "-" "-"
> app005 - - - [16/Feb/2011:18:46:09 +0000] "GET /health.html HTTP/1.1"
> 200 24 "-" "-"
> app005 - - - [16/Feb/2011:18:46:10 +0000] "GET /health.html HTTP/1.1"
> 200 24 "-" "-"
> app005 - - - [16/Feb/2011:18:46:10 +0000] "GET /health.html HTTP/1.1"
> 200 24 "-" "-"
> app005 - - - [16/Feb/2011:18:46:11 +0000] "GET /health.html HTTP/1.1"
> 200 24 "-" "-"
> app005 - - - [16/Feb/2011:18:46:12 +0000] "GET /health.html HTTP/1.1"
> 200 24 "-" "-"
> app005 - - - [16/Feb/2011:18:46:13 +0000] "GET /health.html HTTP/1.1"
> 200 24 "-" "-"
> app005 - - - [16/Feb/2011:18:46:14 +0000] "GET /health.html HTTP/1.1"
> 200 24 "-" "-"
> app005 - - - [16/Feb/2011:18:46:15 +0000] "GET /health.html HTTP/1.1"
> 200 24 "-" "-"
> app005 - - - [16/Feb/2011:18:46:16 +0000] "GET /health.html HTTP/1.1"
> 200 24 "-" "-"
> app005 - - - [16/Feb/2011:18:46:17 +0000] "GET /health.html HTTP/1.1"
> 200 24 "-" "-"
> app005 - - - [16/Feb/2011:18:46:18 +0000] "GET /health.html HTTP/1.1"
> 200 24 "-" "-"
> app005 - - - [16/Feb/2011:18:46:19 +0000] "GET /health.html HTTP/1.1"
> 200 24 "-" "-"
> app005 - - - [16/Feb/2011:18:46:22 +0000] "GET /health.html HTTP/1.1"
> 200 24 "-" "-"
> # PROBES STOP FOR 2 MINUTES
> app005 - - - [16/Feb/2011:18:48:23 +0000] "GET /health.html HTTP/1.1"
> 200 24 "-" "-"
> app005 - - - [16/Feb/2011:18:48:23 +0000] "GET /health.html HTTP/1.1"
> 200 24 "-" "-"
> # ... SNIP SEVERAL MINUTES OF NORMAL PROBING
> app005 - - - [16/Feb/2011:18:54:33 +0000] "GET /health.html HTTP/1.1"
> 200 24 "-" "-"
> app005 - - - [16/Feb/2011:18:54:33 +0000] "GET /health.html HTTP/1.1"
> 200 24 "-" "-"
> app005 - - - [16/Feb/2011:18:54:33 +0000] "GET /health.html HTTP/1.1"
> 200 24 "-" "-"
> # PROBES STOP FOR 7 MINUTES
> app005 - - - [16/Feb/2011:19:01:33 +0000] "GET /health.html HTTP/1.1"
> 200 24 "-" "-"
> app005 - - - [16/Feb/2011:19:01:45 +0000] "GET /health.html HTTP/1.1"
> 200 24 "-" "-"
> app005 - - - [16/Feb/2011:19:01:46 +0000] "GET /health.html HTTP/1.1"
> 200 24 "-" "-"
> # ...PROBES OPERATE NORMALLY FOR HOURS...
> 
> 
> Backend config:
> 
> 
> backend app001 {
>  .host = "app001-private";
>  .port = "8880";
>  .probe = {
>    .url = "/health.html";
>    .timeout = 2s;
>    .interval = 1s;
>    .window = 10;
>    .threshold = 8;
>  }
>  .connect_timeout = 2s;
> }
> 
> backend app002 {
>  .host = "app002-private";
>  .port = "8880";
>  .probe = {
>    .url = "/health.html";
>    .timeout = 2s;
>    .interval = 1s;
>    .window = 10;
>    .threshold = 8;
>  }
>  .connect_timeout = 2s;
> }
> 
> backend app003 {
>  .host = "app003-private";
>  .port = "8880";
>  .probe = {
>    .url = "/health.html";
>    .timeout = 2s;
>    .interval = 1s;
>    .window = 10;
>    .threshold = 8;
>  }
>  .connect_timeout = 2s;
> }
> 
> backend app005 {
>  .host = "app005-private";
>  .port = "8880";
>  .probe = {
>    .url = "/health.html";
>    .timeout = 2s;
>    .interval = 1s;
>    .window = 10;
>    .threshold = 5;
>    .initial = 10;
>  }
>  .connect_timeout = 2s;
> }
> 
> backend app006 {
>  .host = "app006-private";
>  .port = "8880";
>  .probe = {
>    .url = "/health.html";
>    .timeout = 2s;
>    .interval = 1s;
>    .window = 10;
>    .threshold = 5;
>    .initial = 10;
>  }
>  .connect_timeout = 2s;
> }
> 
> director app_servers random {
>    {
>        .backend = app001;
>        .weight = 10;
>    }
>    {
>        .backend = app002;
>        .weight = 100;
>    }
>    {
>        .backend = app003;
>        .weight = 75;
>    }
>    {
>        .backend = app005;
>        .weight = 300;
>    }
>    {
>        .backend = app006;
>        .weight = 300;
>    }
> }
> 
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc



From phk at phk.freebsd.dk  Thu Feb 17 09:09:07 2011
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Thu, 17 Feb 2011 09:09:07 +0000
Subject: Varnish-development-light
Message-ID: <76021.1297933747@critter.freebsd.dk>


Hi Gang,

Thanks to all who came to the VUG3, it was wonderful to meet you
and to get your input and bugreports.

I made it home safely, but my laptop developed a fever during the
trip and now shut down with overtemperature, whenever I run the
varnish regression tests.  I will drop it off for service today and
it may be an entire week before I have it again.

In the meantime my "backup" workstation is a Samsung N150 Plus
netbook and this will cramp my usual style a bit, but hopefully not
too much.

Until I get Xorg running on it in a different mode than black pixels
on black background, I'll be in console-mode only and therefore not
in IRC.  (Should any of you have a working xorg.conf for this machine
I would like a copy)

Send email to get hold of me...

Poul-Henning

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.



From perbu at varnish-software.com  Thu Feb 17 09:32:05 2011
From: perbu at varnish-software.com (Per Buer)
Date: Thu, 17 Feb 2011 10:32:05 +0100
Subject: DDOS
In-Reply-To: <AANLkTimyMj0Ymyrdp7_X5J1NvZnoamhWQ_O38LHAT4u-@mail.gmail.com>
References: <AANLkTimyMj0Ymyrdp7_X5J1NvZnoamhWQ_O38LHAT4u-@mail.gmail.com>
Message-ID: <AANLkTikagkBsZ+GGvDeP6nYJVe1ruagJTtAQ=_Y7iXgj@mail.gmail.com>

On Wed, Feb 16, 2011 at 9:37 PM, alexus <alexus at gmail.com> wrote:
> How does Varnish handles DDOS?

I've interviewed a couple of sysadmins that have been hit with DOS and
DDOS attacks a couple of times. I've write an article that is
available for download here:
http://www.varnish-software.com/products-services/whitepapers (please
note: registration required).

A lot of the time you can get much better results blocking stuff with
Varnish because you can block on the HTTP level, so you can block
specific user agents, accept-encoding strings or on any combination of
headers.

-- 
Per Buer,?Varnish Software
Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer
Varnish makes websites fly!
Want to learn more about Varnish? http://www.varnish-software.com/whitepapers



From martin.boer at bizztravel.nl  Thu Feb 17 09:50:19 2011
From: martin.boer at bizztravel.nl (Martin Boer)
Date: Thu, 17 Feb 2011 10:50:19 +0100
Subject: DDOS
In-Reply-To: <AANLkTikagkBsZ+GGvDeP6nYJVe1ruagJTtAQ=_Y7iXgj@mail.gmail.com>
References: <AANLkTimyMj0Ymyrdp7_X5J1NvZnoamhWQ_O38LHAT4u-@mail.gmail.com>
	<AANLkTikagkBsZ+GGvDeP6nYJVe1ruagJTtAQ=_Y7iXgj@mail.gmail.com>
Message-ID: <4D5CEF5B.3030702@bizztravel.nl>

Our experience with DOS attacks is that Varnish will happily deliver 
more pages/second then the receiver can swallow.
Our uplink provider blocks the DDOS attacks.

On 02/17/2011 10:32 AM, Per Buer wrote:
> On Wed, Feb 16, 2011 at 9:37 PM, alexus<alexus at gmail.com>  wrote:
>> How does Varnish handles DDOS?
> I've interviewed a couple of sysadmins that have been hit with DOS and
> DDOS attacks a couple of times. I've write an article that is
> available for download here:
> http://www.varnish-software.com/products-services/whitepapers (please
> note: registration required).
>
> A lot of the time you can get much better results blocking stuff with
> Varnish because you can block on the HTTP level, so you can block
> specific user agents, accept-encoding strings or on any combination of
> headers.
>




From alexus at gmail.com  Thu Feb 17 17:34:56 2011
From: alexus at gmail.com (alexus)
Date: Thu, 17 Feb 2011 12:34:56 -0500
Subject: DDOS
In-Reply-To: <BE2E0B80-E286-425B-9FDA-909C5F9F5286@topscms.com>
References: <AANLkTimyMj0Ymyrdp7_X5J1NvZnoamhWQ_O38LHAT4u-@mail.gmail.com>
	<OF8A51CAF6.87572723-ONCA257839.007E7BF1-CA257839.007FA542@scee.net>
	<BE2E0B80-E286-425B-9FDA-909C5F9F5286@topscms.com>
Message-ID: <AANLkTimpt90D8CmtU_F++9kw4SaVxGjgB2RFzdOgE+mW@mail.gmail.com>

can you please explain more? I'm not sure if I follow it to full extend

On Wed, Feb 16, 2011 at 10:41 PM, Caunter, Stefan <scaunter at topscms.com>wrote:

> Increment a header and send an error if limit is reached from client.ip
>
> Stefan Caunter
> Operations
> TorstarDigital
> 416.561.4871
>
> On 2011-02-16, at 6:14 PM, "Nicholas_Maesepp at scee.net" <
> Nicholas_Maesepp at scee.net> wrote:
>
> As someone else said best to use a firewall, if you want to use a simple
> one just run iptables, since this is only DoS you best to use rate limit
> connections per IP, depending on the number of objects etc you should be
> able to get away with keeping connections to a fairly low number and not
> hampering browsing experience at all. It would need to move to being DDoS
> for it to bring it down.
>
> Only cheaty way I can think of doing it in the vcl is to define multiple
> backends one with no max_connections and one with a fairly low
> max_connections. Define some ip acl's using netblocks. If you feel all of
> the hammering comes from china and very little legit web traffic you could
> find their netblocks and force them to use the one with a set limit on
> .max_connections while others dont have such limits. Vice versa, if you
> expect 90% of traffic to be within your own country allow those to connect
> to the backend with higher max_connections and have everyone else connect
> with a low number. If someone tries to hammer your page it will just starve
> those outside your target audience but keep your backend healthy and your
> regular visitors fine. Lots of caveats in that and wouldn't recommend it,
> purely for science. Definitely use a firewall or other tools.
>
> Nick
>
>
>
>
> From:        alexus <alexus at gmail.com>
> To:        varnish-misc at varnish-cache.org
> Date:        02/17/2011 07:37 AM
> Subject:        DDOS
> Sent by:        varnish-misc-bounces at varnish-cache.org
> ------------------------------
>
>
>
> How does Varnish handles DDOS?
>
> here is my problem, in our environment varnish servers static pages
> but dynamic pages it passed to application server (apache).
> so every onces in a while we have some attacker(s) who start doing
> some sort of attacks against us, and apache hits very high load on
> server and about to go down...
> so I look up an IP address and I block it on Varnish .vcl file, and
> load goes back to normal right away...
> is there something that can be done automatically? or is there
> anything can be done at all to somehow handle this type of issue?
>
> please advise
>
> --
> http://alexus.org/
>
> _______________________________________________
> varnish-misc mailing list
>  <varnish-misc at varnish-cache.org>varnish-misc at varnish-cache.org
> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
> *
> *
>
> *
> Sony Computer Entertainment Australia Pty Ltd*
> Level 1, 63-73 Ann Street Surry Hills NSW 2010
> P.O. Box 5023 Darlinghurst NSW 2010
> ph: +61 (0)2 9324 9500 fax: +61 (0)2 9324 9558
>
> *http://au.playstation.com* <http://au.playstation.com/>*
> **http://www.facebook.com/PlayStationAU*<http://www.facebook.com/PlaystationAU>
>
>
> *
> New tools, new games, new fun.
> Sackboy returns in LittleBigPlanet 2
> Out now and exclusive to PlayStation 3*
>
> *
> *[image: LBP2] <http://www.littlebigplanet.com/>
>
> **********************************************************************
> This email and any files transmitted with it are confidential and intended
> solely for the use of the individual or entity to whom they are addressed.
> If you have received this email in error please notify
> <postmaster at scee.net>postmaster at scee.net
> This footnote also confirms that this email message has been checked for
> all known viruses.
> Sony Computer Entertainment Australia Pty. Limited
> Registered Office: Level 1, 63-73 Ann Street, Surry Hills, NSW 2010
> Australia
> Registered in Australia: 077 583 183
> **********************************************************************
>
> P* Please consider the environment before printing this e-mail*
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>



-- 
http://alexus.org/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110217/71acfbe2/attachment.html>

From alexus at gmail.com  Thu Feb 17 17:35:28 2011
From: alexus at gmail.com (alexus)
Date: Thu, 17 Feb 2011 12:35:28 -0500
Subject: DDOS
In-Reply-To: <OF8A51CAF6.87572723-ONCA257839.007E7BF1-CA257839.007FA542@scee.net>
References: <AANLkTimyMj0Ymyrdp7_X5J1NvZnoamhWQ_O38LHAT4u-@mail.gmail.com>
	<OF8A51CAF6.87572723-ONCA257839.007E7BF1-CA257839.007FA542@scee.net>
Message-ID: <AANLkTi=UBbEK95jJU_96e8aVgiE+oiy1qe1MB62Vpcvo@mail.gmail.com>

I like your idea, I'll look into that ...

On Wed, Feb 16, 2011 at 6:13 PM, <Nicholas_Maesepp at scee.net> wrote:

> As someone else said best to use a firewall, if you want to use a simple
> one just run iptables, since this is only DoS you best to use rate limit
> connections per IP, depending on the number of objects etc you should be
> able to get away with keeping connections to a fairly low number and not
> hampering browsing experience at all. It would need to move to being DDoS
> for it to bring it down.
>
> Only cheaty way I can think of doing it in the vcl is to define multiple
> backends one with no max_connections and one with a fairly low
> max_connections. Define some ip acl's using netblocks. If you feel all of
> the hammering comes from china and very little legit web traffic you could
> find their netblocks and force them to use the one with a set limit on
> .max_connections while others dont have such limits. Vice versa, if you
> expect 90% of traffic to be within your own country allow those to connect
> to the backend with higher max_connections and have everyone else connect
> with a low number. If someone tries to hammer your page it will just starve
> those outside your target audience but keep your backend healthy and your
> regular visitors fine. Lots of caveats in that and wouldn't recommend it,
> purely for science. Definitely use a firewall or other tools.
>
> Nick
>
>
>
>
> From:        alexus <alexus at gmail.com>
> To:        varnish-misc at varnish-cache.org
> Date:        02/17/2011 07:37 AM
> Subject:        DDOS
> Sent by:        varnish-misc-bounces at varnish-cache.org
> ------------------------------
>
>
>
> How does Varnish handles DDOS?
>
> here is my problem, in our environment varnish servers static pages
> but dynamic pages it passed to application server (apache).
> so every onces in a while we have some attacker(s) who start doing
> some sort of attacks against us, and apache hits very high load on
> server and about to go down...
> so I look up an IP address and I block it on Varnish .vcl file, and
> load goes back to normal right away...
> is there something that can be done automatically? or is there
> anything can be done at all to somehow handle this type of issue?
>
> please advise
>
> --
> http://alexus.org/
>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
> *
> *
>
> *
> Sony Computer Entertainment Australia Pty Ltd*
> Level 1, 63-73 Ann Street Surry Hills NSW 2010
> P.O. Box 5023 Darlinghurst NSW 2010
> ph: +61 (0)2 9324 9500 fax: +61 (0)2 9324 9558
>
> *http://au.playstation.com* <http://au.playstation.com/>*
> **http://www.facebook.com/PlayStationAU*<http://www.facebook.com/PlaystationAU>
>
>
> *
> New tools, new games, new fun.
> Sackboy returns in LittleBigPlanet 2
> Out now and exclusive to PlayStation 3*
>
> *
> *[image: LBP2] <http://www.littlebigplanet.com/>
>
> **********************************************************************
> This email and any files transmitted with it are confidential and intended
> solely for the use of the individual or entity to whom they are addressed.
> If you have received this email in error please notify postmaster at scee.net
> This footnote also confirms that this email message has been checked for
> all known viruses.
> Sony Computer Entertainment Australia Pty. Limited
> Registered Office: Level 1, 63-73 Ann Street, Surry Hills, NSW 2010
> Australia
> Registered in Australia: 077 583 183
> **********************************************************************
>
> P* Please consider the environment before printing this e-mail*
>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>



-- 
http://alexus.org/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110217/e424ed2a/attachment.html>

From alexus at gmail.com  Thu Feb 17 17:37:13 2011
From: alexus at gmail.com (alexus)
Date: Thu, 17 Feb 2011 12:37:13 -0500
Subject: DDOS
In-Reply-To: <AANLkTikagkBsZ+GGvDeP6nYJVe1ruagJTtAQ=_Y7iXgj@mail.gmail.com>
References: <AANLkTimyMj0Ymyrdp7_X5J1NvZnoamhWQ_O38LHAT4u-@mail.gmail.com>
	<AANLkTikagkBsZ+GGvDeP6nYJVe1ruagJTtAQ=_Y7iXgj@mail.gmail.com>
Message-ID: <AANLkTik0uANqXcvZsWiZ4sb+1Wg7hr7NQ5=cMZMai2E3@mail.gmail.com>

i just finished reading, in your whitepage ddos was easily match to a
single header (user-agent) so it's easy to block that ...
but regardless i found some interesting stuff to look into, so thanks
alot for writing it :)

On Thu, Feb 17, 2011 at 4:32 AM, Per Buer <perbu at varnish-software.com> wrote:
> On Wed, Feb 16, 2011 at 9:37 PM, alexus <alexus at gmail.com> wrote:
>> How does Varnish handles DDOS?
>
> I've interviewed a couple of sysadmins that have been hit with DOS and
> DDOS attacks a couple of times. I've write an article that is
> available for download here:
> http://www.varnish-software.com/products-services/whitepapers (please
> note: registration required).
>
> A lot of the time you can get much better results blocking stuff with
> Varnish because you can block on the HTTP level, so you can block
> specific user agents, accept-encoding strings or on any combination of
> headers.
>
> --
> Per Buer,?Varnish Software
> Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer
> Varnish makes websites fly!
> Want to learn more about Varnish? http://www.varnish-software.com/whitepapers
>



-- 
http://alexus.org/



From adam.mcmanus at gmail.com  Thu Feb 17 21:02:50 2011
From: adam.mcmanus at gmail.com (Adam McManus)
Date: Thu, 17 Feb 2011 15:02:50 -0600
Subject: 503 returning FetchError c http first read error: -1 0
Message-ID: <AANLkTimCTvOHodzWuz8YYn0mkqFUztJ+T=C2qiu7Kd+i@mail.gmail.com>

I could use some help troubleshooting why users sometimes receive a 503
error from Varnish. It does not happen every time, but it is fairly
reproducible on the below example (http://gowalla.com/users/keeg). The
backends are all configured as follows:

backend server1 {
  .host = "<internal_ip>";
  .port = "<internal_port>";
  .connect_timeout = 30s;
  .first_byte_timeout = 30s;
  .between_bytes_timeout = 30s;
}

The director configuration is:

director rr_director round-robin {
  { .backend = server1; }
  { .backend = server2; }
  { .backend = server3; }
  { .backend = server4; }
  { .backend = server5; }
  { .backend = server6; }
  { .backend = server7; }
  { .backend = server8; }
  { .backend = server9; }
}

Here is the varnishlog output of a 503 response. I've searched everywhere
for information on the FetchError provided, but I can't find anything that
seems to apply. Also, the request is not hitting any of the timeouts listed
in the backend configuration. The 503 response usually returns in less than
a second.

   27 SessionOpen  c 127.0.0.1 33235 :8080
   27 ReqStart     c 127.0.0.1 33235 1983217687
   27 RxRequest    c GET
   27 RxURL        c /users/keeg
   27 RxProtocol   c HTTP/1.0
   27 RxHeader     c X-Real-IP: 24.153.226.54
   27 RxHeader     c X-Forwarded-For: 24.153.226.54
   27 RxHeader     c Host: gowalla.com
   27 RxHeader     c Connection: close
   27 RxHeader     c Referer: http://gowalla.com/users/keeg
   27 RxHeader     c Cache-Control: max-age=0
   27 RxHeader     c Accept:
application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
   27 RxHeader     c User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X
10_6_6; en-US) AppleWebKit/534.13 (KHTML, like Gecko) Chrome/9.0.597.102
Safari/534.13
   27 RxHeader     c Accept-Encoding: gzip,deflate,sdch
   27 RxHeader     c Accept-Language: en-US,en;q=0.8
   27 RxHeader     c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
   27 RxHeader     c If-None-Match: "bf9753b17d60d033e7654ea22abfcbd8"
   27 VCL_call     c recv pass
   27 VCL_call     c hash hash
   27 VCL_call     c pass pass
   27 Backend      c 29 rr_director server9
   27 FetchError   c http first read error: -1 0 (Success)
   27 VCL_call     c error deliver
   27 VCL_call     c deliver deliver
   27 TxProtocol   c HTTP/1.1
   27 TxStatus     c 503
   27 TxResponse   c Service Unavailable
   27 TxHeader     c Server: Varnish
   27 TxHeader     c Retry-After: 0
   27 TxHeader     c Content-Type: text/html; charset=utf-8
   27 TxHeader     c Content-Length: 419
   27 TxHeader     c Date: Thu, 17 Feb 2011 20:44:21 GMT
   27 TxHeader     c X-Varnish: 1983217687
   27 TxHeader     c Age: 1
   27 TxHeader     c Via: 1.1 varnish
   27 TxHeader     c Connection: close
   27 Length       c 419
   27 ReqEnd       c 1983217687 1297975461.143035889 1297975461.773808956
0.000031948 0.630686998 0.000086069

I would appreciate any insights anyone might have.

Thanks,

--
Adam McManus
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110217/5bb05602/attachment.html>

From phk at phk.freebsd.dk  Thu Feb 17 21:28:53 2011
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Thu, 17 Feb 2011 21:28:53 +0000
Subject: 503 returning FetchError c http first read error: -1 0 
In-Reply-To: Your message of "Thu, 17 Feb 2011 15:02:50 CST."
	<AANLkTimCTvOHodzWuz8YYn0mkqFUztJ+T=C2qiu7Kd+i@mail.gmail.com> 
Message-ID: <97507.1297978133@critter.freebsd.dk>

In message <AANLkTimCTvOHodzWuz8YYn0mkqFUztJ+T=C2qiu7Kd+i at mail.gmail.com>, Adam
 McManus writes:

>   27 VCL_call     c pass pass
>   27 Backend      c 29 rr_director server9
>   27 FetchError   c http first read error: -1 0 (Success)
>   27 VCL_call     c error deliver

To me this looks like your backend does not answer inside the
"first_byte" timeout.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.



From adam.mcmanus at gmail.com  Thu Feb 17 22:05:32 2011
From: adam.mcmanus at gmail.com (Adam McManus)
Date: Thu, 17 Feb 2011 16:05:32 -0600
Subject: 503 returning FetchError c http first read error: -1 0
In-Reply-To: <97507.1297978133@critter.freebsd.dk>
References: <AANLkTimCTvOHodzWuz8YYn0mkqFUztJ+T=C2qiu7Kd+i@mail.gmail.com>
	<97507.1297978133@critter.freebsd.dk>
Message-ID: <AANLkTi=Tqzz1_SykDAmgtmzFN_SUKxdYcTC6c=XwRdhH@mail.gmail.com>

> To me this looks like your backend does not answer inside the
> "first_byte" timeout.

That's what I initially thought, but the 503 is being generated within
1-2 seconds max of initiating the request, not the 30 seconds being
defined in each backend. Am I misunderstanding the first_byte timeout
setting?

--

Adam McManus
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110217/1d90da24/attachment.html>

From phk at phk.freebsd.dk  Thu Feb 17 23:19:15 2011
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Thu, 17 Feb 2011 23:19:15 +0000
Subject: 503 returning FetchError c http first read error: -1 0 
In-Reply-To: Your message of "Thu, 17 Feb 2011 16:05:32 CST."
	<AANLkTi=Tqzz1_SykDAmgtmzFN_SUKxdYcTC6c=XwRdhH@mail.gmail.com> 
Message-ID: <20161.1297984755@critter.freebsd.dk>

In message <AANLkTi=Tqzz1_SykDAmgtmzFN_SUKxdYcTC6c=XwRdhH at mail.gmail.com>, Adam
 McManus writes:

>> To me this looks like your backend does not answer inside the
>> "first_byte" timeout.
>
>That's what I initially thought, but the 503 is being generated within
>1-2 seconds max of initiating the request, not the 30 seconds being
>defined in each backend. Am I misunderstanding the first_byte timeout
>setting?

Probably not.

Are you 100% sure your backend is not closing the connection ?

Use tcpdump or similar to find out...

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.



From rdelsalle at gmail.com  Fri Feb 18 14:19:22 2011
From: rdelsalle at gmail.com (Roch Delsalle)
Date: Fri, 18 Feb 2011 15:19:22 +0100
Subject: Varnish : how to add an exception for dynamic pages with cookies
Message-ID: <AANLkTim8FMN8wM+JXEfwjhBCCrR5wp3ENnvGjaeWSo4=@mail.gmail.com>

Hi,

I would like to know what's the right way to avoid caching "some pages" of a
website using Varnish and cache all the others.

This is what I have tried to do with the vcl conf:

sub vcl_fetch {
        #set beresp.ttl = 1d;
        if (!(req.url ~ "/page1withauth") ||
            !(req.url ~ "/page2withauth")) {
           unset beresp.http.set-cookie;
        }
        if (!beresp.cacheable) {
            return (pass);
        }
        if (beresp.http.Set-Cookie) {
            return (pass);
        }
        return (deliver);
}

Thanks

--
D-Ro.ch
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110218/7b3692a0/attachment.html>

From kongfranon at gmail.com  Fri Feb 18 14:56:02 2011
From: kongfranon at gmail.com (Mike Franon)
Date: Fri, 18 Feb 2011 09:56:02 -0500
Subject: varnish 2.15 - bypass caching for mobile devices?
Message-ID: <AANLkTikpeHQ17CVXuHB6NH6wAoYEdWydK8PJaM+3Mmev@mail.gmail.com>

Hi,

Right now everything has been working great with this config, but we
discovered an issue with our mobile devices, not ever getting to the
apache redirect we setup to go to our mobile site, since it gets the
cached version of our home page.  Is there a way to say if a mobile
device by agent to bypass caching all together and let it pass through
to the apache redirect which then goes to another site all together?


Thanks,
Mike


Here is my default.vcl


sub vcl_recv {
if (req.url ~ "^/$" || req.url ~ "^/samples"){
                unset req.http.cookie;
                return(lookup);
}

else {
                return(pass);
}

}
sub vcl_fetch {
    if (req.url ~ "^/$" || req.url ~ "^/samples"){
        set beresp.ttl = 300s;
        set beresp.http.cache-control = "public, max-age = 300";
        set beresp.http.X-CacheReason = "varnishcache";
        unset beresp.http.set-cookie;
        return(deliver);
    }
}



From pom at dmsp.de  Fri Feb 18 15:30:47 2011
From: pom at dmsp.de (Stefan Pommerening)
Date: Fri, 18 Feb 2011 16:30:47 +0100
Subject: ulimit -v on SLES10/11
Message-ID: <4D5E90A7.5070508@dmsp.de>

Hi all,

I am currently hunting down a strange configuration problem on SLES 
machines.
And I could need some help if someone of you guys is also running SLES.

I had a strange problem with varnish not coming up which was result of 
virtual memory on newly installed SLES 11 machines was configured the 
same size as real memory was. On our 'older' SLES 10 machines the value 
of ulimit -v was always 'unlimited' which allows varnish to fetch as 
much virtual memory it needs. This took me some time to find out and now 
I wonder what the default configuration on SLES 10 / 11 is. As I have no 
clue where to look for it I'd like to ask if someone could send me the 
values he finds on his default SLES machine (ulimit -v).

Need not be on the mailing list - might be direct personal mail. I will 
send out a summary if this happens to be an error in the default SLES 11 
configuration as this might be helpful for others, too.

Have a nice week-end,
Stefan

-- 

*Dipl.-Inform. Stefan Pommerening
Informatik-B?ro: IT-Dienste & Projekte, Consulting & Coaching*
http://www.dmsp.de <http://www.dmsp.de/>





From josh at schulzone.org  Fri Feb 18 15:31:00 2011
From: josh at schulzone.org (Josh)
Date: Fri, 18 Feb 2011 08:31:00 -0700
Subject: Varnish : how to add an exception for dynamic pages with cookies
In-Reply-To: <AANLkTim8FMN8wM+JXEfwjhBCCrR5wp3ENnvGjaeWSo4=@mail.gmail.com>
References: <AANLkTim8FMN8wM+JXEfwjhBCCrR5wp3ENnvGjaeWSo4=@mail.gmail.com>
Message-ID: <AANLkTinx74763JtYMc326zk5tGdgCyduQVwrYZy6fwZn@mail.gmail.com>

On Fri, Feb 18, 2011 at 7:19 AM, Roch Delsalle <rdelsalle at gmail.com> wrote:
> Hi,
>
> I would like to know what's the right way to avoid caching "some pages" of a
> website using Varnish and cache all the others.
>
> This is what I have tried to do with the vcl conf:
>
> sub vcl_fetch {
> ? ? ? ? #set beresp.ttl = 1d;
> ? ? ? ? if (!(req.url ~ "/page1withauth") ||
> ? ? ? ? ? ? !(req.url ~ "/page2withauth")) {
> ? ? ? ? ? ?unset beresp.http.set-cookie;
> ? ? ? ? }
> ? ? ? ? if (!beresp.cacheable) {
> ? ? ? ? ? ? return (pass);
> ? ? ? ? }
> ? ? ? ? if (beresp.http.Set-Cookie) {
> ? ? ? ? ? ? return (pass);
> ? ? ? ? }
> ? ? ? ? return (deliver);
> }
>
> Thanks
>
> --
> D-Ro.ch
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>



-- 
josh
@schulz
http://schulzone.org



From stig at zedge.net  Fri Feb 18 15:37:16 2011
From: stig at zedge.net (Stig Bakken)
Date: Fri, 18 Feb 2011 16:37:16 +0100
Subject: varnish 2.15 - bypass caching for mobile devices?
In-Reply-To: <AANLkTikpeHQ17CVXuHB6NH6wAoYEdWydK8PJaM+3Mmev@mail.gmail.com>
References: <AANLkTikpeHQ17CVXuHB6NH6wAoYEdWydK8PJaM+3Mmev@mail.gmail.com>
Message-ID: <AANLkTikLnbkhgsLnrH+tU-6TwmOa4Pvo1CNY4eL9JC1Z@mail.gmail.com>

Browser recognition is a royal pain to begin with, and even more so with
mobile browsers (their user agents may differ between requests in some
cases).

I would suggest you move all of the browser/device detection logic to your
backend, but store the detected device in a cookie.  If varnish sees a
request without that cookie, it redirects the user to a browser/device
detection backend (using wurfl or deviceatlas for detection), which again
redirects to the original page with an extra parameter in the URL which
makes the final destination set the cookie.

>From there on, you can extract the device cookie with varnish and append it
to your varnish hash, as per
http://www.varnish-cache.org/trac/wiki/VCLExampleCacheCookies.

Of course, the browser detection backend URL must not be cached. :)

 - Stig

On Fri, Feb 18, 2011 at 3:56 PM, Mike Franon <kongfranon at gmail.com> wrote:

> Hi,
>
> Right now everything has been working great with this config, but we
> discovered an issue with our mobile devices, not ever getting to the
> apache redirect we setup to go to our mobile site, since it gets the
> cached version of our home page.  Is there a way to say if a mobile
> device by agent to bypass caching all together and let it pass through
> to the apache redirect which then goes to another site all together?
>
>
> Thanks,
> Mike
>
>
> Here is my default.vcl
>
>
> sub vcl_recv {
> if (req.url ~ "^/$" || req.url ~ "^/samples"){
>                unset req.http.cookie;
>                return(lookup);
> }
>
> else {
>                return(pass);
> }
>
> }
> sub vcl_fetch {
>    if (req.url ~ "^/$" || req.url ~ "^/samples"){
>        set beresp.ttl = 300s;
>        set beresp.http.cache-control = "public, max-age = 300";
>        set beresp.http.X-CacheReason = "varnishcache";
>        unset beresp.http.set-cookie;
>        return(deliver);
>    }
> }
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>



-- 
Stig Bakken
CTO, Zedge.net - free your phone!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110218/3234f61a/attachment.html>

From rdelsalle at gmail.com  Fri Feb 18 15:58:26 2011
From: rdelsalle at gmail.com (Roch Delsalle)
Date: Fri, 18 Feb 2011 16:58:26 +0100
Subject: Varnish : how to add an exception for dynamic pages with cookies
In-Reply-To: <AANLkTinx74763JtYMc326zk5tGdgCyduQVwrYZy6fwZn@mail.gmail.com>
References: <AANLkTim8FMN8wM+JXEfwjhBCCrR5wp3ENnvGjaeWSo4=@mail.gmail.com>
	<AANLkTinx74763JtYMc326zk5tGdgCyduQVwrYZy6fwZn@mail.gmail.com>
Message-ID: <AANLkTikhhU4KesYaNOJ-dBfjfSZz6uQHmkO2Ksi6hqOy@mail.gmail.com>

Thank you for your answer, I'm going to manage it from the Cache-Control
headers then,
All my pages are containing cookies, is it going to be a problem or do I
need to use "unset beresp.http.set-cookie;" somewhere?

On Fri, Feb 18, 2011 at 4:31 PM, Josh <josh at schulzone.org> wrote:

> On Fri, Feb 18, 2011 at 7:19 AM, Roch Delsalle <rdelsalle at gmail.com>
> wrote:
> > Hi,
> >
> > I would like to know what's the right way to avoid caching "some pages"
> of a
> > website using Varnish and cache all the others.
> >
> > This is what I have tried to do with the vcl conf:
> >
> > sub vcl_fetch {
> >         #set beresp.ttl = 1d;
> >         if (!(req.url ~ "/page1withauth") ||
> >             !(req.url ~ "/page2withauth")) {
> >            unset beresp.http.set-cookie;
> >         }
> >         if (!beresp.cacheable) {
> >             return (pass);
> >         }
> >         if (beresp.http.Set-Cookie) {
> >             return (pass);
> >         }
> >         return (deliver);
> > }
> >
> > Thanks
> >
> > --
> > D-Ro.ch
> > _______________________________________________
> > varnish-misc mailing list
> > varnish-misc at varnish-cache.org
> > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
> >
>
>
>
> --
> josh
> @schulz
> http://schulzone.org
>



-- 
D-Ro.ch
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110218/4a016459/attachment.html>

From josh at schulzone.org  Fri Feb 18 16:26:20 2011
From: josh at schulzone.org (Josh)
Date: Fri, 18 Feb 2011 09:26:20 -0700
Subject: Varnish : how to add an exception for dynamic pages with cookies
In-Reply-To: <AANLkTikhhU4KesYaNOJ-dBfjfSZz6uQHmkO2Ksi6hqOy@mail.gmail.com>
References: <AANLkTim8FMN8wM+JXEfwjhBCCrR5wp3ENnvGjaeWSo4=@mail.gmail.com>
	<AANLkTinx74763JtYMc326zk5tGdgCyduQVwrYZy6fwZn@mail.gmail.com>
	<AANLkTikhhU4KesYaNOJ-dBfjfSZz6uQHmkO2Ksi6hqOy@mail.gmail.com>
Message-ID: <AANLkTikZorXt-rHyL5SrFnkTZasTmTuAjoDh6cW2nKBw@mail.gmail.com>

It depends what you use the cookies for.  If they interact with code
to personalize the page then you can't cache them, but if they're
analytics cookies (for example) you can strip them out individually.
Here's the relevant line from my vcl:

set req.http.Cookie = regsuball(req.http.Cookie,
"(^|;\s*)(mbox|s_cc|s_cc|s_sq|s_vi)=[^;]+;?", "");

That pulls all the analytics cookies out from the back end request.
That's a better idea than just nuking the cookies, on the chance that
your code really is looking for one.  The server doesn't care about
any of those cookies, it's on page javascript elements that use them.

On Fri, Feb 18, 2011 at 8:58 AM, Roch Delsalle <rdelsalle at gmail.com> wrote:
> Thank you for your answer, I'm going to manage it from the?Cache-Control
> headers then,
> All my pages are containing cookies, is it going to be a problem or do I
> need to use "unset beresp.http.set-cookie;" somewhere?
> On Fri, Feb 18, 2011 at 4:31 PM, Josh <josh at schulzone.org> wrote:
>>
>> On Fri, Feb 18, 2011 at 7:19 AM, Roch Delsalle <rdelsalle at gmail.com>
>> wrote:
>> > Hi,
>> >
>> > I would like to know what's the right way to avoid caching "some pages"
>> > of a
>> > website using Varnish and cache all the others.
>> >
>> > This is what I have tried to do with the vcl conf:
>> >
>> > sub vcl_fetch {
>> > ? ? ? ? #set beresp.ttl = 1d;
>> > ? ? ? ? if (!(req.url ~ "/page1withauth") ||
>> > ? ? ? ? ? ? !(req.url ~ "/page2withauth")) {
>> > ? ? ? ? ? ?unset beresp.http.set-cookie;
>> > ? ? ? ? }
>> > ? ? ? ? if (!beresp.cacheable) {
>> > ? ? ? ? ? ? return (pass);
>> > ? ? ? ? }
>> > ? ? ? ? if (beresp.http.Set-Cookie) {
>> > ? ? ? ? ? ? return (pass);
>> > ? ? ? ? }
>> > ? ? ? ? return (deliver);
>> > }
>> >
>> > Thanks
>> >
>> > --
>> > D-Ro.ch
>> > _______________________________________________
>> > varnish-misc mailing list
>> > varnish-misc at varnish-cache.org
>> > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>> >
>>
>>
>>
>> --
>> josh
>> @schulz
>> http://schulzone.org
>
>
>
> --
> D-Ro.ch



-- 
josh
@schulz
http://schulzone.org



From scaunter at topscms.com  Fri Feb 18 16:33:23 2011
From: scaunter at topscms.com (Caunter, Stefan)
Date: Fri, 18 Feb 2011 11:33:23 -0500
Subject: varnish 2.15 - bypass caching for mobile devices?
In-Reply-To: <AANLkTikLnbkhgsLnrH+tU-6TwmOa4Pvo1CNY4eL9JC1Z@mail.gmail.com>
References: <AANLkTikpeHQ17CVXuHB6NH6wAoYEdWydK8PJaM+3Mmev@mail.gmail.com>
	<AANLkTikLnbkhgsLnrH+tU-6TwmOa4Pvo1CNY4eL9JC1Z@mail.gmail.com>
Message-ID: <7F0AA702B8A85A4A967C4C8EBAD6902CF7C057@TMG-EVS02.torstar.net>

We use the vcl_error redirect technique with success:

 

You can edit the example to suit different UAs and mobile views. The ?no referrer? is important, to prevent continuous redirects once we land on the target site. 

 

in vcl_recv

 

                if (!req.http.referer && req.http.host ~ "www.example.ca" && req.http.user-agent ~ "[aA]ndroid|[bB]lack[Bb]erry|

DoCoMo|[iI][Pp]hone|NetFront|Opera Mini|PalmOS|PalmSource|Symbian OS|webOS|Ipod|lg9700|nokiae|nokian|blackberry9|htc") {

                        error 751 "Moved Temporarily";

                }

 

in vcl_error

 

if (obj.status == 751) {

                set obj.http.Location = "http://richmobile.example.ca/";

                set obj.status = 302;

                return(deliver);

        }

 

Stefan Caunter 

Operations

Torstar Digital

m: (416) 561-4871

 

 

From: varnish-misc-bounces at varnish-cache.org [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Stig Bakken
Sent: February-18-11 10:37 AM
To: Mike Franon
Cc: varnish-misc at varnish-cache.org
Subject: Re: varnish 2.15 - bypass caching for mobile devices?

 

Browser recognition is a royal pain to begin with, and even more so with mobile browsers (their user agents may differ between requests in some cases).

 

I would suggest you move all of the browser/device detection logic to your backend, but store the detected device in a cookie.  If varnish sees a request without that cookie, it redirects the user to a browser/device detection backend (using wurfl or deviceatlas for detection), which again redirects to the original page with an extra parameter in the URL which makes the final destination set the cookie.

 

From there on, you can extract the device cookie with varnish and append it to your varnish hash, as per

http://www.varnish-cache.org/trac/wiki/VCLExampleCacheCookies.

 

Of course, the browser detection backend URL must not be cached. :)

 

 - Stig

 

On Fri, Feb 18, 2011 at 3:56 PM, Mike Franon <kongfranon at gmail.com> wrote:

Hi,

Right now everything has been working great with this config, but we
discovered an issue with our mobile devices, not ever getting to the
apache redirect we setup to go to our mobile site, since it gets the
cached version of our home page.  Is there a way to say if a mobile
device by agent to bypass caching all together and let it pass through
to the apache redirect which then goes to another site all together?


Thanks,
Mike


Here is my default.vcl


sub vcl_recv {
if (req.url ~ "^/$" || req.url ~ "^/samples"){
               unset req.http.cookie;
               return(lookup);
}

else {
               return(pass);
}

}
sub vcl_fetch {
   if (req.url ~ "^/$" || req.url ~ "^/samples"){
       set beresp.ttl = 300s;
       set beresp.http.cache-control = "public, max-age = 300";
       set beresp.http.X-CacheReason = "varnishcache";
       unset beresp.http.set-cookie;
       return(deliver);
   }
}

_______________________________________________
varnish-misc mailing list
varnish-misc at varnish-cache.org
http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc




-- 
Stig Bakken
CTO, Zedge.net - free your phone!

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110218/7b5bc033/attachment.html>

From scaunter at topscms.com  Fri Feb 18 16:40:09 2011
From: scaunter at topscms.com (Caunter, Stefan)
Date: Fri, 18 Feb 2011 11:40:09 -0500
Subject: varnish 2.15 - bypass caching for mobile devices?
In-Reply-To: <7F0AA702B8A85A4A967C4C8EBAD6902CF7C057@TMG-EVS02.torstar.net>
References: <AANLkTikpeHQ17CVXuHB6NH6wAoYEdWydK8PJaM+3Mmev@mail.gmail.com><AANLkTikLnbkhgsLnrH+tU-6TwmOa4Pvo1CNY4eL9JC1Z@mail.gmail.com>
	<7F0AA702B8A85A4A967C4C8EBAD6902CF7C057@TMG-EVS02.torstar.net>
Message-ID: <7F0AA702B8A85A4A967C4C8EBAD6902CF7C062@TMG-EVS02.torstar.net>

My bad, old example. Here?s one with (?i) case insensitivity and a rewrite rule.

 

 

  if (req.url ~ "^/(.*)" && req.http.host ~ "(?i)www.example.ca" && req.http.user-agent ~ "(?i)iphone|htc|android|

ipod|lg9700|blackberry9|nokiae|nokian|docomo|netfront|palmos|ipod|lg9700|htc|opera mini|palmsource|symbian OS|psp" && !req.http.referer ~ "(?i)example.ca") {

                        set req.http.newhost = regsub(req.url, "^/(.*)", "http://richmobile.example.ca/\1");

                        error 753 req.http.newhost;

                }

if (obj.status == 753) {

                set obj.http.Location = obj.response;

                set obj.status = 302;

                return(deliver);

        }

 

 

Stefan Caunter 

Operations

Torstar Digital

m: (416) 561-4871

 

 

From: varnish-misc-bounces at varnish-cache.org [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Caunter, Stefan
Sent: February-18-11 11:33 AM
To: Stig Bakken; Mike Franon
Cc: varnish-misc at varnish-cache.org
Subject: RE: varnish 2.15 - bypass caching for mobile devices?

 

We use the vcl_error redirect technique with success:

 

You can edit the example to suit different UAs and mobile views. The ?no referrer? is important, to prevent continuous redirects once we land on the target site. 

 

in vcl_recv

 

                if (!req.http.referer && req.http.host ~ "www.example.ca" && req.http.user-agent ~ "[aA]ndroid|[bB]lack[Bb]erry|

DoCoMo|[iI][Pp]hone|NetFront|Opera Mini|PalmOS|PalmSource|Symbian OS|webOS|Ipod|lg9700|nokiae|nokian|blackberry9|htc") {

                        error 751 "Moved Temporarily";

                }

 

in vcl_error

 

if (obj.status == 751) {

                set obj.http.Location = "http://richmobile.example.ca/";

                set obj.status = 302;

                return(deliver);

        }

 

Stefan Caunter 

Operations

Torstar Digital

m: (416) 561-4871

 

 

From: varnish-misc-bounces at varnish-cache.org [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Stig Bakken
Sent: February-18-11 10:37 AM
To: Mike Franon
Cc: varnish-misc at varnish-cache.org
Subject: Re: varnish 2.15 - bypass caching for mobile devices?

 

Browser recognition is a royal pain to begin with, and even more so with mobile browsers (their user agents may differ between requests in some cases).

 

I would suggest you move all of the browser/device detection logic to your backend, but store the detected device in a cookie.  If varnish sees a request without that cookie, it redirects the user to a browser/device detection backend (using wurfl or deviceatlas for detection), which again redirects to the original page with an extra parameter in the URL which makes the final destination set the cookie.

 

From there on, you can extract the device cookie with varnish and append it to your varnish hash, as per

http://www.varnish-cache.org/trac/wiki/VCLExampleCacheCookies.

 

Of course, the browser detection backend URL must not be cached. :)

 

 - Stig

 

On Fri, Feb 18, 2011 at 3:56 PM, Mike Franon <kongfranon at gmail.com> wrote:

Hi,

Right now everything has been working great with this config, but we
discovered an issue with our mobile devices, not ever getting to the
apache redirect we setup to go to our mobile site, since it gets the
cached version of our home page.  Is there a way to say if a mobile
device by agent to bypass caching all together and let it pass through
to the apache redirect which then goes to another site all together?


Thanks,
Mike


Here is my default.vcl


sub vcl_recv {
if (req.url ~ "^/$" || req.url ~ "^/samples"){
               unset req.http.cookie;
               return(lookup);
}

else {
               return(pass);
}

}
sub vcl_fetch {
   if (req.url ~ "^/$" || req.url ~ "^/samples"){
       set beresp.ttl = 300s;
       set beresp.http.cache-control = "public, max-age = 300";
       set beresp.http.X-CacheReason = "varnishcache";
       unset beresp.http.set-cookie;
       return(deliver);
   }
}

_______________________________________________
varnish-misc mailing list
varnish-misc at varnish-cache.org
http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc




-- 
Stig Bakken
CTO, Zedge.net - free your phone!

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110218/705ff248/attachment.html>

From j73mor at gmail.com  Sun Feb 20 14:24:20 2011
From: j73mor at gmail.com (Jesper Mortensen)
Date: Sun, 20 Feb 2011 15:24:20 +0100
Subject: Feature consideration: SSL via HAProxy's "Proxy" protocol and Stunnel
Message-ID: <AANLkTik0oi_kBuj2nw6Mbt7NVGJQ7xWDdsHe08Z0bwDf@mail.gmail.com>

Dear all,

Have you guys seen HAProxy's "Proxy Protocol"? The gist of it is a
patch for Stunnel 4.33 and smaller change to HAProxy. Used together
these permit flexible SSL handling with HAProxy.

http://haproxy.1wt.eu/download/1.5/doc/proxy-protocol.txt
http://haproxy.1wt.eu/
(paragraph below heading "Quick News" - "Nov 11th, 2010")

I'm no expert, but this seems to be a very low-friction way of adding
good SSL support, and should be applicable to Varnish as well? SSL
support is on the post-2.1 roadmap; is the above helpful in
prioritizing this feature?  :-)

Thanks,


-- 
Best Regards,

Jesper Mortensen



From cosimo at streppone.it  Mon Feb 21 06:20:20 2011
From: cosimo at streppone.it (Cosimo Streppone)
Date: Mon, 21 Feb 2011 17:20:20 +1100
Subject: Do I need to unset cookie in vcl_fetch if there is remove
	req.http.Cookie in vcl_recv?
In-Reply-To: <AANLkTim6YoJ6uz=XJqZCYSo90vZAQvscA4JFDT=F=9dZ@mail.gmail.com>
References: <AANLkTim6YoJ6uz=XJqZCYSo90vZAQvscA4JFDT=F=9dZ@mail.gmail.com>
Message-ID: <op.vq8o76o2s5ttvb@cn03>

On Fri, 11 Feb 2011 11:48:47 +1100, David Murphy <david at firechaser.com>  
wrote:

> I've been testing removing cookies from images/css/js and am a little
> unclear on the difference between:
>
> //start ==========
>  sub vcl_recv {
>  if (req.url ~ "\.(png|gif|jpg|swf|css|js)$") {
>        remove req.http.Cookie;
>   }

vcl_recv is run when varnish receives the request from the client.

"req.http.Cookie" is the Cookie header that the client
sends in its request. If you remove that, your backends
won't see any cookies and varnish will, by default, cache the
object.

> sub vcl_fetch {
>  if (req.url ~ "\.(png|gif|jpg|swf|css|js)$") {
>    unset beresp.http.set-cookie;
>  }

This is vcl_fetch, that, I believe, it's run when
varnish has got the object from the backend already.

"beresp.http.set-cookie" is the Set-Cookie header
of your backend's response, so if you remove it that means
you want to strip any cookies that *your backend* might
want to send to the client.

I'm not sure if you set any cookies when serving images
(someone does), but maybe it's just an additional
"safety measure"?

Hope this helps.

-- 
Cosimo



From cosimo at streppone.it  Mon Feb 21 06:20:27 2011
From: cosimo at streppone.it (Cosimo Streppone)
Date: Mon, 21 Feb 2011 17:20:27 +1100
Subject: Dynamically Extend Grace Period
In-Reply-To: <AANLkTikbidTjoYtKyXGu5EonsFoEsBpA+Jd5-Oduhha2@mail.gmail.com>
References: <AANLkTik5oKyEc_6uw4dgxB-7LwFK_QYCZuUu6YtnC2rG@mail.gmail.com>
	<AANLkTikbidTjoYtKyXGu5EonsFoEsBpA+Jd5-Oduhha2@mail.gmail.com>
Message-ID: <op.vq8o8diks5ttvb@cn03>

On Wed, 16 Feb 2011 00:53:31 +1100, Per Buer <perbu at varnish-software.com>  
wrote:

> On Tue, Feb 15, 2011 at 2:07 PM, Vince <cooltechemail at gmail.com> wrote:
>>
>> I am wondering if I can dynamically extend grace period. For example,
>> currently I have beresp.grace = 10m. If something bad happens to the  
>> backend
>
> [...]
>
> Sort of. You could set the grace period to, say 2 hours, on the beresp
> object and then set the req.grace period to 10min. Then, when
> something bad hits the fan you increase the grace period in vcl_recv.

Assuming 2.1+, would the proper (for some meaning of proper)
way to do this to use saint mode?

-- 
Cosimo



From tfheen at varnish-software.com  Tue Feb 22 09:53:43 2011
From: tfheen at varnish-software.com (Tollef Fog Heen)
Date: Tue, 22 Feb 2011 10:53:43 +0100
Subject: Determining hash variations
In-Reply-To: <356FD748-D91A-4CFE-BD6F-992841A1CC7F@comicrelief.com> (Ben
	Dodd's message of "Sat, 12 Feb 2011 11:13:58 +0000")
References: <356FD748-D91A-4CFE-BD6F-992841A1CC7F@comicrelief.com>
Message-ID: <87y658jsyg.fsf@qurzaw.varnish-software.com>

]] Ben Dodd 

| Is there anyway to determine the number of hash variations a URL has
| in the cache? So if a page has multiple versions based on cookies,
| accept-encoding etc.

Assuming you mean variants coming from Vary, look at the difference
between objects and object heads.  There's no way to know the number for
a given object without resorting to inline C.

-- 
Tollef Fog Heen
Varnish Software
t: +47 21 98 92 64



From junxian.yan at gmail.com  Tue Feb 22 10:10:40 2011
From: junxian.yan at gmail.com (Junxian Yan)
Date: Tue, 22 Feb 2011 02:10:40 -0800
Subject: inspect the response header to find cached request
Message-ID: <AANLkTimCsSJRBYOE=LpPvApj2A00QB_e649O23RVo1=S@mail.gmail.com>

Is there any parameters to determine if some request has been cached by
varnish when inspect the response header

Junxian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110222/43157f42/attachment.html>

From stewsnooze at gmail.com  Tue Feb 22 10:18:17 2011
From: stewsnooze at gmail.com (Stewart Robinson)
Date: Tue, 22 Feb 2011 10:18:17 +0000
Subject: inspect the response header to find cached request
In-Reply-To: <AANLkTimCsSJRBYOE=LpPvApj2A00QB_e649O23RVo1=S@mail.gmail.com>
References: <AANLkTimCsSJRBYOE=LpPvApj2A00QB_e649O23RVo1=S@mail.gmail.com>
Message-ID: <AANLkTikKZC5hqLQZkBJeJWifDym=Vy9A9r6jQCzSEanY@mail.gmail.com>

Hi,

You could change your vcl_deliver function in your vcl to have
something like this.

sub vcl_deliver {
	if (obj.hits > 0) {
		set resp.http.X-Cache-Action = "HIT";
		set resp.http.X-Cache-Hits = obj.hits;
	} else {
		set resp.http.X-Cache-Action = "MISS";
	}
}


This works on the 2.0x series. Can't remember off the top of my head
whether vcl_deliver has renamed obj to something else.

Stewart Robinson
@stewsnooze

http://fullfatthings.com/


On 22 February 2011 10:10, Junxian Yan <junxian.yan at gmail.com> wrote:
> Is there any parameters to determine if some request has been cached by
> varnish when inspect the response header
> Junxian
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>



From shib4u at gmail.com  Tue Feb 22 11:56:44 2011
From: shib4u at gmail.com (Shibashish)
Date: Tue, 22 Feb 2011 17:26:44 +0530
Subject: Force cache a php page
Message-ID: <AANLkTikExenMMu4npzKHRDOt_SQaWBxwb6Cppk4GccWe@mail.gmail.com>

Hi All,

I have a website under varnish-2.1.5-1. I have a php page (test.php) which
is kinda static and I can cache it for some time (5 mins). The URLs hitting
this page are like the following due to which the varnish cache is MISSed
and the apache webserver is being hit. How do I do a regex to overcome this?
My trial config is mentioned in the end....

     1.00 TxURL
 /test.php?id=219488&TabID=3&Res=2&dt=866845858823555200
     1.00 TxURL
 /test.php?id=253114&TabID=3&Res=2&dt=846642321042837100
     1.00 TxURL
 /test.php?id=104326&TabID=3&Res=2&dt=575570316149238400
     1.00 TxURL
 /test.php?id=30946&TabID=3&Res=2&dt=109061349055833870
     1.00 TxURL
 /test.php?id=109331&TabID=3&Res=2&dt=284968385895093150
     1.00 TxURL
 /test.php?id=147630&TabID=3&Res=2&dt=182582005324904030
     1.00 TxURL
 /test.php?id=226618&TabID=3&Res=2&dt=567666382104328500
     1.00 TxURL
 /test.php?id=44438&TabID=3&Res=2&dt=894643067109792400
     1.00 TxURL
 /test.php?id=219029&TabID=3&Res=2&dt=120583019713746690
     1.00 TxURL
 /test.php?id=92700&TabID=3&Res=2&dt=812672457300415500
     1.00 TxURL
 /test.php?id=244364&TabID=3&Res=2&dt=543154670573941800
     1.00 TxURL
 /test.php?id=204543&TabID=3&Res=2&dt=108553982023531000
     1.00 TxURL
 /test.php?id=155579&TabID=3&Res=2&dt=609157805933082500
     1.00 TxURL
 /test.php?id=100248&TabID=3&Res=2&dt=329215175928002100
     1.00 TxURL
 /test.php?id=155347&TabID=3&Res=2&dt=324280933690657500
     1.00 TxURL
 /test.php?id=153167&TabID=3&Res=2&dt=262366359106033180
     1.00 TxURL
 /test.php?id=175035&TabID=3&Res=2&dt=748310387497392400
     1.00 TxURL
 /test.php?id=122942&TabID=3&Res=2&dt=89373125154077630
     1.00 TxURL
 /test.php?id=157531&TabID=3&Res=2&dt=164945409132294180
     1.00 TxURL
 /test.php?id=100248&TabID=3&Res=2&dt=381019014643790400
     1.00 TxURL
 /test.php?id=103053&TabID=3&Res=2&dt=597790616600820500


vcl_recv

 if (req.url ~ "\.phg(\?.*)?$") {
  return(lookup);}

Thanks.

--
Shib
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110222/8baebea8/attachment.html>

From traian.bratucu at eea.europa.eu  Tue Feb 22 12:01:47 2011
From: traian.bratucu at eea.europa.eu (Traian Bratucu)
Date: Tue, 22 Feb 2011 13:01:47 +0100
Subject: Force cache a php page
In-Reply-To: <AANLkTikExenMMu4npzKHRDOt_SQaWBxwb6Cppk4GccWe@mail.gmail.com>
References: <AANLkTikExenMMu4npzKHRDOt_SQaWBxwb6Cppk4GccWe@mail.gmail.com>
Message-ID: <E7097BFB283E614C96AE10D5307726E301AE8907A0BC@SLOTHMAIL.eea.eu.int>

Just a note: please be aware that by default PHP will issue some headers that will not permit caching (Cache-Control). Please also verify php.ini.
Regarding your regex... you wrote "phg"...

Traian

From: varnish-misc-bounces at varnish-cache.org [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Shibashish
Sent: Tuesday, February 22, 2011 12:57 PM
To: varnish-misc at varnish-cache.org
Subject: Force cache a php page

Hi All,

I have a website under varnish-2.1.5-1. I have a php page (test.php) which is kinda static and I can cache it for some time (5 mins). The URLs hitting this page are like the following due to which the varnish cache is MISSed and the apache webserver is being hit. How do I do a regex to overcome this? My trial config is mentioned in the end....

     1.00 TxURL          /test.php?id=219488&TabID=3&Res=2&dt=866845858823555200
     1.00 TxURL          /test.php?id=253114&TabID=3&Res=2&dt=846642321042837100
     1.00 TxURL          /test.php?id=104326&TabID=3&Res=2&dt=575570316149238400
     1.00 TxURL          /test.php?id=30946&TabID=3&Res=2&dt=109061349055833870
     1.00 TxURL          /test.php?id=109331&TabID=3&Res=2&dt=284968385895093150
     1.00 TxURL          /test.php?id=147630&TabID=3&Res=2&dt=182582005324904030
     1.00 TxURL          /test.php?id=226618&TabID=3&Res=2&dt=567666382104328500
     1.00 TxURL          /test.php?id=44438&TabID=3&Res=2&dt=894643067109792400
     1.00 TxURL          /test.php?id=219029&TabID=3&Res=2&dt=120583019713746690
     1.00 TxURL          /test.php?id=92700&TabID=3&Res=2&dt=812672457300415500
     1.00 TxURL          /test.php?id=244364&TabID=3&Res=2&dt=543154670573941800
     1.00 TxURL          /test.php?id=204543&TabID=3&Res=2&dt=108553982023531000
     1.00 TxURL          /test.php?id=155579&TabID=3&Res=2&dt=609157805933082500
     1.00 TxURL          /test.php?id=100248&TabID=3&Res=2&dt=329215175928002100
     1.00 TxURL          /test.php?id=155347&TabID=3&Res=2&dt=324280933690657500
     1.00 TxURL          /test.php?id=153167&TabID=3&Res=2&dt=262366359106033180
     1.00 TxURL          /test.php?id=175035&TabID=3&Res=2&dt=748310387497392400
     1.00 TxURL          /test.php?id=122942&TabID=3&Res=2&dt=89373125154077630
     1.00 TxURL          /test.php?id=157531&TabID=3&Res=2&dt=164945409132294180
     1.00 TxURL          /test.php?id=100248&TabID=3&Res=2&dt=381019014643790400
     1.00 TxURL          /test.php?id=103053&TabID=3&Res=2&dt=597790616600820500


vcl_recv

 if (req.url ~ "\.phg(\?.*)?$") {
  return(lookup);}

Thanks.

--
Shib
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110222/92cb70a1/attachment.html>

From d.storm at drecomm.nl  Tue Feb 22 12:03:41 2011
From: d.storm at drecomm.nl (Dennis Storm - Drecomm BV)
Date: Tue, 22 Feb 2011 13:03:41 +0100
Subject: Force cache a php page
In-Reply-To: <AANLkTikExenMMu4npzKHRDOt_SQaWBxwb6Cppk4GccWe@mail.gmail.com>
References: <AANLkTikExenMMu4npzKHRDOt_SQaWBxwb6Cppk4GccWe@mail.gmail.com>
Message-ID: <4D63A61D.1020106@drecomm.nl>

Looks like you have a typo in the vcl config. Try using php instead of 
phg :)

--- bad ---
vcl_recv

  if (req.url ~ "\.phg(\?.*)?$") {
   return(lookup);}

--- better ---
vcl_recv

  if (req.url ~ "\.php(\?.*)?$") {
   return(lookup);}

?

Kind regards.

Dennis Storm
Senior Systeembeheerder

------------------------------------------------------------------------

E d.storm at drecomm.nl

T +31 (0)50 - 577 58 22 (doorkiesnummer: 9701)

M +31 (0)6 - 414 101 81

	drecomm

------------------------------------------------------------------------


  Vestiging Groningen

Hoendiep 208 	T +31 (0)50 - 577 58 22 	Bekijk onze actuele projecten
9745 ED  Groningen 	F +31 (0)50 - 577 58 23 	www.drecomm.nl 
<http://www.drecomm.nl>

------------------------------------------------------------------------

On 02/22/2011 12:56 PM, Shibashish wrote:
> Hi All,
>
> I have a website under varnish-2.1.5-1. I have a php page (test.php) 
> which is kinda static and I can cache it for some time (5 mins). The 
> URLs hitting this page are like the following due to which the varnish 
> cache is MISSed and the apache webserver is being hit. How do I do a 
> regex to overcome this? My trial config is mentioned in the end....
>
>      1.00 TxURL         
>  /test.php?id=219488&TabID=3&Res=2&dt=866845858823555200
>      1.00 TxURL         
>  /test.php?id=253114&TabID=3&Res=2&dt=846642321042837100
>      1.00 TxURL         
>  /test.php?id=104326&TabID=3&Res=2&dt=575570316149238400
>      1.00 TxURL         
>  /test.php?id=30946&TabID=3&Res=2&dt=109061349055833870
>      1.00 TxURL         
>  /test.php?id=109331&TabID=3&Res=2&dt=284968385895093150
>      1.00 TxURL         
>  /test.php?id=147630&TabID=3&Res=2&dt=182582005324904030
>      1.00 TxURL         
>  /test.php?id=226618&TabID=3&Res=2&dt=567666382104328500
>      1.00 TxURL         
>  /test.php?id=44438&TabID=3&Res=2&dt=894643067109792400
>      1.00 TxURL         
>  /test.php?id=219029&TabID=3&Res=2&dt=120583019713746690
>      1.00 TxURL         
>  /test.php?id=92700&TabID=3&Res=2&dt=812672457300415500
>      1.00 TxURL         
>  /test.php?id=244364&TabID=3&Res=2&dt=543154670573941800
>      1.00 TxURL         
>  /test.php?id=204543&TabID=3&Res=2&dt=108553982023531000
>      1.00 TxURL         
>  /test.php?id=155579&TabID=3&Res=2&dt=609157805933082500
>      1.00 TxURL         
>  /test.php?id=100248&TabID=3&Res=2&dt=329215175928002100
>      1.00 TxURL         
>  /test.php?id=155347&TabID=3&Res=2&dt=324280933690657500
>      1.00 TxURL         
>  /test.php?id=153167&TabID=3&Res=2&dt=262366359106033180
>      1.00 TxURL         
>  /test.php?id=175035&TabID=3&Res=2&dt=748310387497392400
>      1.00 TxURL         
>  /test.php?id=122942&TabID=3&Res=2&dt=89373125154077630
>      1.00 TxURL         
>  /test.php?id=157531&TabID=3&Res=2&dt=164945409132294180
>      1.00 TxURL         
>  /test.php?id=100248&TabID=3&Res=2&dt=381019014643790400
>      1.00 TxURL         
>  /test.php?id=103053&TabID=3&Res=2&dt=597790616600820500
>
>
> vcl_recv
>
>  if (req.url ~ "\.phg(\?.*)?$") {
>   return(lookup);}
>
> Thanks.
>
> --
> Shib
>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110222/6d28d2c8/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: top_logo.jpg
Type: image/jpeg
Size: 24234 bytes
Desc: not available
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110222/6d28d2c8/attachment.jpg>

From shib4u at gmail.com  Tue Feb 22 12:15:15 2011
From: shib4u at gmail.com (Shibashish)
Date: Tue, 22 Feb 2011 17:45:15 +0530
Subject: Force cache a php page
In-Reply-To: <E7097BFB283E614C96AE10D5307726E301AE8907A0BC@SLOTHMAIL.eea.eu.int>
References: <AANLkTikExenMMu4npzKHRDOt_SQaWBxwb6Cppk4GccWe@mail.gmail.com>
	<E7097BFB283E614C96AE10D5307726E301AE8907A0BC@SLOTHMAIL.eea.eu.int>
Message-ID: <AANLkTim6tR+KOidRWcw8RP6fb2SNU59xY0QcwajHYN=S@mail.gmail.com>

On Tue, Feb 22, 2011 at 5:31 PM, Traian Bratucu <
traian.bratucu at eea.europa.eu> wrote:

> Just a note: please be aware that by default PHP will issue some headers
> that will not permit caching (Cache-Control). Please also verify php.ini.
>
> Regarding your regex... you wrote ?phg?...
>
>
>
> Traian
>
>
>
> *From:* varnish-misc-bounces at varnish-cache.org [mailto:
> varnish-misc-bounces at varnish-cache.org] *On Behalf Of *Shibashish
> *Sent:* Tuesday, February 22, 2011 12:57 PM
> *To:* varnish-misc at varnish-cache.org
> *Subject:* Force cache a php page
>
>
>
> Hi All,
>
>
>
> I have a website under varnish-2.1.5-1. I have a php page (test.php) which
> is kinda static and I can cache it for some time (5 mins). The URLs hitting
> this page are like the following due to which the varnish cache is MISSed
> and the apache webserver is being hit. How do I do a regex to overcome this?
> My trial config is mentioned in the end....
>
>
>
>      1.00 TxURL
>  /test.php?id=219488&TabID=3&Res=2&dt=866845858823555200
>
>      1.00 TxURL
>  /test.php?id=253114&TabID=3&Res=2&dt=846642321042837100
>
>      1.00 TxURL
>  /test.php?id=104326&TabID=3&Res=2&dt=575570316149238400
>
>      1.00 TxURL
>  /test.php?id=30946&TabID=3&Res=2&dt=109061349055833870
>
>      1.00 TxURL
>  /test.php?id=109331&TabID=3&Res=2&dt=284968385895093150
>
>      1.00 TxURL
>  /test.php?id=147630&TabID=3&Res=2&dt=182582005324904030
>
>      1.00 TxURL
>  /test.php?id=226618&TabID=3&Res=2&dt=567666382104328500
>
>      1.00 TxURL
>  /test.php?id=44438&TabID=3&Res=2&dt=894643067109792400
>
>      1.00 TxURL
>  /test.php?id=219029&TabID=3&Res=2&dt=120583019713746690
>
>      1.00 TxURL
>  /test.php?id=92700&TabID=3&Res=2&dt=812672457300415500
>
>      1.00 TxURL
>  /test.php?id=244364&TabID=3&Res=2&dt=543154670573941800
>
>      1.00 TxURL
>  /test.php?id=204543&TabID=3&Res=2&dt=108553982023531000
>
>      1.00 TxURL
>  /test.php?id=155579&TabID=3&Res=2&dt=609157805933082500
>
>      1.00 TxURL
>  /test.php?id=100248&TabID=3&Res=2&dt=329215175928002100
>
>      1.00 TxURL
>  /test.php?id=155347&TabID=3&Res=2&dt=324280933690657500
>
>      1.00 TxURL
>  /test.php?id=153167&TabID=3&Res=2&dt=262366359106033180
>
>      1.00 TxURL
>  /test.php?id=175035&TabID=3&Res=2&dt=748310387497392400
>
>      1.00 TxURL
>  /test.php?id=122942&TabID=3&Res=2&dt=89373125154077630
>
>      1.00 TxURL
>  /test.php?id=157531&TabID=3&Res=2&dt=164945409132294180
>
>      1.00 TxURL
>  /test.php?id=100248&TabID=3&Res=2&dt=381019014643790400
>
>      1.00 TxURL
>  /test.php?id=103053&TabID=3&Res=2&dt=597790616600820500
>
>
>
>
>
> vcl_recv
>
>
>
>  if (req.url ~ "\.phg(\?.*)?$") {
>
>   return(lookup);}
>
>
>
> Thanks.
>
>
>
> --
>
> Shib
>

Thanks for your reply. The header has....

Cache-Control max-age=600, public

The regex has been corrected. Just to let you know that any subsequent
requests to the same url, i.e. /test.php?abc, if hit for a second or 3rd
time or more, shows a cache HIT.

ShiB.
while ( ! ( succeed = try() ) );
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110222/916ca30b/attachment.html>

From kongfranon at gmail.com  Tue Feb 22 14:03:04 2011
From: kongfranon at gmail.com (Mike Franon)
Date: Tue, 22 Feb 2011 09:03:04 -0500
Subject: varnish 2.15 - bypass caching for mobile devices?
In-Reply-To: <7F0AA702B8A85A4A967C4C8EBAD6902CF7C062@TMG-EVS02.torstar.net>
References: <AANLkTikpeHQ17CVXuHB6NH6wAoYEdWydK8PJaM+3Mmev@mail.gmail.com>
	<AANLkTikLnbkhgsLnrH+tU-6TwmOa4Pvo1CNY4eL9JC1Z@mail.gmail.com>
	<7F0AA702B8A85A4A967C4C8EBAD6902CF7C057@TMG-EVS02.torstar.net>
	<7F0AA702B8A85A4A967C4C8EBAD6902CF7C062@TMG-EVS02.torstar.net>
Message-ID: <AANLkTimKTQy0s1z-R=3bdp3NQVSB0aC1-xSntZ+OvNS3@mail.gmail.com>

Thank you for everyone's help.

What I did was utlize the F5 load balance iRules, and put in a rule if
a mobile agent to redirect to a different pool, and it resolved it.

Thanks

On Fri, Feb 18, 2011 at 11:40 AM, Caunter, Stefan <scaunter at topscms.com> wrote:
> My bad, old example. Here?s one with (?i) case insensitivity and a rewrite
> rule.
>
>
>
>
>
> ? if (req.url ~ "^/(.*)" && req.http.host ~ "(?i)www.example.ca" &&
> req.http.user-agent ~ "(?i)iphone|htc|android|
>
> ipod|lg9700|blackberry9|nokiae|nokian|docomo|netfront|palmos|ipod|lg9700|htc|opera
> mini|palmsource|symbian OS|psp" && !req.http.referer ~ "(?i)example.ca") {
>
> ??????????????????????? set req.http.newhost = regsub(req.url, "^/(.*)",
> "http://richmobile.example.ca/\1");
>
> ??????????????????????? error 753 req.http.newhost;
>
> ??????????????? }
>
> if (obj.status == 753) {
>
> ??????? ????????set obj.http.Location = obj.response;
>
> ??????????????? set obj.status = 302;
>
> ??????????????? return(deliver);
>
> ??????? }
>
>
>
>
>
> Stefan Caunter
>
> Operations
>
> Torstar Digital
>
> m: (416) 561-4871
>
>
>
>
>
> From: varnish-misc-bounces at varnish-cache.org
> [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Caunter, Stefan
> Sent: February-18-11 11:33 AM
> To: Stig Bakken; Mike Franon
> Cc: varnish-misc at varnish-cache.org
> Subject: RE: varnish 2.15 - bypass caching for mobile devices?
>
>
>
> We use the vcl_error redirect technique with success:
>
>
>
> You can edit the example to suit different UAs and mobile views. The ?no
> referrer? is important, to prevent continuous redirects once we land on the
> target site.
>
>
>
> in vcl_recv
>
>
>
> ??????????????? if (!req.http.referer && req.http.host ~ "www.example.ca" &&
> req.http.user-agent ~ "[aA]ndroid|[bB]lack[Bb]erry|
>
> DoCoMo|[iI][Pp]hone|NetFront|Opera Mini|PalmOS|PalmSource|Symbian
> OS|webOS|Ipod|lg9700|nokiae|nokian|blackberry9|htc") {
>
> ??????????????????????? error 751 "Moved Temporarily";
>
> ? ??????????????}
>
>
>
> in vcl_error
>
>
>
> if (obj.status == 751) {
>
> ??????????????? set obj.http.Location = "http://richmobile.example.ca/";
>
> ??????????????? set obj.status = 302;
>
> ??????????????? return(deliver);
>
> ??????? }
>
>
>
> Stefan Caunter
>
> Operations
>
> Torstar Digital
>
> m: (416) 561-4871
>
>
>
>
>
> From: varnish-misc-bounces at varnish-cache.org
> [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Stig Bakken
> Sent: February-18-11 10:37 AM
> To: Mike Franon
> Cc: varnish-misc at varnish-cache.org
> Subject: Re: varnish 2.15 - bypass caching for mobile devices?
>
>
>
> Browser recognition is a royal pain to begin with, and even more so with
> mobile browsers?(their user agents may differ between requests in some
> cases).
>
>
>
> I would suggest you move all of the browser/device detection logic to your
> backend, but store the detected device in a cookie. ?If varnish sees a
> request without that cookie, it redirects the user to a browser/device
> detection backend (using wurfl or deviceatlas for detection), which again
> redirects to the original page with an extra parameter in the URL which
> makes the final destination set the cookie.
>
>
>
> From there on, you can extract the device cookie with varnish and append it
> to your varnish hash, as per
>
> http://www.varnish-cache.org/trac/wiki/VCLExampleCacheCookies.
>
>
>
> Of course, the browser detection backend URL must not be cached. :)
>
>
>
> ?- Stig
>
>
>
> On Fri, Feb 18, 2011 at 3:56 PM, Mike Franon <kongfranon at gmail.com> wrote:
>
> Hi,
>
> Right now everything has been working great with this config, but we
> discovered an issue with our mobile devices, not ever getting to the
> apache redirect we setup to go to our mobile site, since it gets the
> cached version of our home page. ?Is there a way to say if a mobile
> device by agent to bypass caching all together and let it pass through
> to the apache redirect which then goes to another site all together?
>
>
> Thanks,
> Mike
>
>
> Here is my default.vcl
>
>
> sub vcl_recv {
> if (req.url ~ "^/$" || req.url ~ "^/samples"){
> ? ? ? ? ? ? ? ?unset req.http.cookie;
> ? ? ? ? ? ? ? ?return(lookup);
> }
>
> else {
> ? ? ? ? ? ? ? ?return(pass);
> }
>
> }
> sub vcl_fetch {
> ? ?if (req.url ~ "^/$" || req.url ~ "^/samples"){
> ? ? ? ?set beresp.ttl = 300s;
> ? ? ? ?set beresp.http.cache-control = "public, max-age = 300";
> ? ? ? ?set beresp.http.X-CacheReason = "varnishcache";
> ? ? ? ?unset beresp.http.set-cookie;
> ? ? ? ?return(deliver);
> ? ?}
> }
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
>
> --
> Stig Bakken
> CTO, Zedge.net - free your phone!



From kongfranon at gmail.com  Tue Feb 22 14:10:50 2011
From: kongfranon at gmail.com (Mike Franon)
Date: Tue, 22 Feb 2011 09:10:50 -0500
Subject: varnish 2.15 - possible security exploit?
Message-ID: <AANLkTimzDZXpY=OXb-g3uVj=FurbWpjHweJzLChqrBLg@mail.gmail.com>

HI,

I was curious does anyone know of any serious security exploits that
can use varnish as an open proxy?


The reason I ask is we just put up 3 varnish servers, and about 4 days
after we started to get a DDOS attack.

If anything the varnish servers really helped offload it.  We are
still unders attack for at least 12 hours +

The reason why I am thinking that some sort of exploit might be going
on is, looking at the varnish logs I was seeing some url's for domains
we do not even own.  I am not sure how get requests are coming through
for not our own domain's?  Majority of get are for us, but 10% or so
are not

It could have been just a coicendence that we got a DDOS attack a few
days after, and glad I had varnish in, becuase we were getting over
3,000 req/sec, which our web servers definitely could not handle.



Thanks for any suggestions.



From perbu at varnish-software.com  Tue Feb 22 14:15:16 2011
From: perbu at varnish-software.com (Per Buer)
Date: Tue, 22 Feb 2011 15:15:16 +0100
Subject: varnish 2.15 - possible security exploit?
In-Reply-To: <AANLkTimzDZXpY=OXb-g3uVj=FurbWpjHweJzLChqrBLg@mail.gmail.com>
References: <AANLkTimzDZXpY=OXb-g3uVj=FurbWpjHweJzLChqrBLg@mail.gmail.com>
Message-ID: <AANLkTim5n6nHHED+dOQotM6j7yL0njex_7FL9zexQUbz@mail.gmail.com>

Hi Mike,

On Tue, Feb 22, 2011 at 3:10 PM, Mike Franon <kongfranon at gmail.com> wrote:
>
> I was curious does anyone know of any serious security exploits that
> can use varnish as an open proxy?

No. It is impossible to configure Varnish as an open proxy, since
every backend has to be predefined.

> The reason I ask is we just put up 3 varnish servers, and about 4 days
> after we started to get a DDOS attack.
>
> If anything the varnish servers really helped offload it. ?We are
> still unders attack for at least 12 hours +
>
> The reason why I am thinking that some sort of exploit might be going
> on is, looking at the varnish logs I was seeing some url's for domains
> we do not even own. ?I am not sure how get requests are coming through
> for not our own domain's? ?Majority of get are for us, but 10% or so
> are not

Just the DOS client which hammers you with random URLs, including host
names that are not configured at your site. You can probably just
whitelist the ones you're using and disregard the rest.


-- 
Per Buer,?Varnish Software
Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer
Varnish makes websites fly!
Want to learn more about Varnish? http://www.varnish-software.com/whitepapers



From pom at dmsp.de  Tue Feb 22 14:22:27 2011
From: pom at dmsp.de (Stefan Pommerening)
Date: Tue, 22 Feb 2011 15:22:27 +0100
Subject: varnish 2.15 - possible security exploit?
In-Reply-To: <AANLkTimzDZXpY=OXb-g3uVj=FurbWpjHweJzLChqrBLg@mail.gmail.com>
References: <AANLkTimzDZXpY=OXb-g3uVj=FurbWpjHweJzLChqrBLg@mail.gmail.com>
Message-ID: <4D63C6A3.5010006@dmsp.de>

Am 22.02.2011 15:10, schrieb Mike Franon:
> The reason why I am thinking that some sort of exploit might be going
> on is, looking at the varnish logs I was seeing some url's for domains
> we do not even own.  I am not sure how get requests are coming through
> for not our own domain's?  Majority of get are for us, but 10% or so
> are not

Varnish is generally only logging the host header of the http requests. 
You can easily connect to some server using its ip address and transfer 
some random host header for the http request itself. This can be easily 
done by using wget or telnet for example. I am using this regularly for 
testing purposes when updating some configuration on vhosts or stuff.

Therefore the strange domain names have nothing to do with some security 
exploit, but this is simply another layer of connectiviy.

Stefan

-- 

*Dipl.-Inform. Stefan Pommerening
Informatik-B?ro: IT-Dienste & Projekte, Consulting & Coaching*
http://www.dmsp.de <http://www.dmsp.de/>





From kongfranon at gmail.com  Tue Feb 22 15:01:48 2011
From: kongfranon at gmail.com (Mike Franon)
Date: Tue, 22 Feb 2011 10:01:48 -0500
Subject: varnish 2.15 - possible security exploit?
In-Reply-To: <4D63C6A3.5010006@dmsp.de>
References: <AANLkTimzDZXpY=OXb-g3uVj=FurbWpjHweJzLChqrBLg@mail.gmail.com>
	<4D63C6A3.5010006@dmsp.de>
Message-ID: <AANLkTinTtUjiAFhXbSvbHbmmqoXjbvCV3ym-jt63Xa=c@mail.gmail.com>

Thank you Per Buer, and Stefan,


That is what I thought but just wanted to double-check.

At least I know the varnish servers can handle the load to help offset
a DDOS attack.  Before varnish, if we ever got above 100 req/sec our
apache servers would not respond.  But we are still trying to figure
out ways of stopping it, which is really tough, since they are hitting
our home page directly, and we cannot block that request, we have
blocked at the F5 level, top 20 ip address, hostname and user-agents.

Thanks again,
Mike


On Tue, Feb 22, 2011 at 9:22 AM, Stefan Pommerening <pom at dmsp.de> wrote:
> Am 22.02.2011 15:10, schrieb Mike Franon:
>>
>> The reason why I am thinking that some sort of exploit might be going
>> on is, looking at the varnish logs I was seeing some url's for domains
>> we do not even own. ?I am not sure how get requests are coming through
>> for not our own domain's? ?Majority of get are for us, but 10% or so
>> are not
>
> Varnish is generally only logging the host header of the http requests. You
> can easily connect to some server using its ip address and transfer some
> random host header for the http request itself. This can be easily done by
> using wget or telnet for example. I am using this regularly for testing
> purposes when updating some configuration on vhosts or stuff.
>
> Therefore the strange domain names have nothing to do with some security
> exploit, but this is simply another layer of connectiviy.
>
> Stefan
>
> --
>
> *Dipl.-Inform. Stefan Pommerening
> Informatik-B?ro: IT-Dienste & Projekte, Consulting & Coaching*
> http://www.dmsp.de <http://www.dmsp.de/>
>
>
>



From phk at phk.freebsd.dk  Tue Feb 22 14:46:45 2011
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Tue, 22 Feb 2011 14:46:45 +0000
Subject: varnish 2.15 - possible security exploit? 
In-Reply-To: Your message of "Tue, 22 Feb 2011 09:10:50 EST."
	<AANLkTimzDZXpY=OXb-g3uVj=FurbWpjHweJzLChqrBLg@mail.gmail.com> 
Message-ID: <2946.1298386005@critter.freebsd.dk>

In message <AANLkTimzDZXpY=OXb-g3uVj=FurbWpjHweJzLChqrBLg at mail.gmail.com>, Mike
 Franon writes:
>HI,
>
>I was curious does anyone know of any serious security exploits that
>can use varnish as an open proxy?

Only if they can reload the Varnish VCL somehow.  Varnish has the
backends hardcoded in VCL.

>The reason why I am thinking that some sort of exploit might be going
>on is, looking at the varnish logs I was seeing some url's for domains
>we do not even own.

And what does the log says happen to them ?

You can probably do something like:

	if (req.http.host !~ "<regexp matching your domains") {
		error(755);	/* No need to be civilized here */
	}

To prevent them from reaching your backend.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.



From scaunter at topscms.com  Tue Feb 22 18:03:35 2011
From: scaunter at topscms.com (Caunter, Stefan)
Date: Tue, 22 Feb 2011 13:03:35 -0500
Subject: varnish 2.15 - possible security exploit? 
In-Reply-To: <2946.1298386005@critter.freebsd.dk>
References: Your message of "Tue, 22 Feb 2011 09:10:50
	EST."<AANLkTimzDZXpY=OXb-g3uVj=FurbWpjHweJzLChqrBLg@mail.gmail.com>
	<2946.1298386005@critter.freebsd.dk>
Message-ID: <7F0AA702B8A85A4A967C4C8EBAD6902CF7C23B@TMG-EVS02.torstar.net>

>In message
<AANLkTimzDZXpY=OXb-g3uVj=FurbWpjHweJzLChqrBLg at mail.gmail.com>, Mike
 Franon writes:
>>I was curious does anyone know of any serious security exploits that
>>can use varnish as an open proxy?

>Only if they can reload the Varnish VCL somehow.  Varnish has the
>backends hardcoded in VCL.

>>The reason why I am thinking that some sort of exploit might be going
>>on is, looking at the varnish logs I was seeing some url's for domains
>>we do not even own.

>And what does the log says happen to them ?

>You can probably do something like:

>	if (req.http.host !~ "<regexp matching your domains") {
>		error(755);	/* No need to be civilized here */
>	}

>To prevent them from reaching your backend.


Sure, but maybe we have a non-host specific config for a farm, where if
DNS sends you to varnish, it doesn't check the host header, it just
selects a backend. A regexp matching many domains is avoided in this
case. 

Lets you put varnish in front of many sites without a lot of fuss.

If there's an invalid host, we can simply cache the "don't got" page.

Potential for DoS attack, but hardly specific to varnish.

SC



From joao.lisanti at locaweb.com.br  Tue Feb 22 21:05:11 2011
From: joao.lisanti at locaweb.com.br (=?iso-8859-1?Q?Jo=E3o_Gabriel?=)
Date: Tue, 22 Feb 2011 18:05:11 -0300
Subject: 503 returning FetchError c http first read error: -1 0 
In-Reply-To: <20161.1297984755@critter.freebsd.dk>
References: <20161.1297984755@critter.freebsd.dk>
Message-ID: <C462D1EF-FC43-4033-8670-D6C43432FD9E@locaweb.com.br>

Probably the backend is closing  connection, I'm having the same problem here, for some reason the backend is  
closing. I'm looking for the reason

Jo?o Gabriel
CT-Linux




On Feb 17, 2011, at 9:19 PM, Poul-Henning Kamp wrote:

> In message <AANLkTi=Tqzz1_SykDAmgtmzFN_SUKxdYcTC6c=XwRdhH at mail.gmail.com>, Adam
> McManus writes:
> 
>>> To me this looks like your backend does not answer inside the
>>> "first_byte" timeout.
>> 
>> That's what I initially thought, but the 503 is being generated within
>> 1-2 seconds max of initiating the request, not the 30 seconds being
>> defined in each backend. Am I misunderstanding the first_byte timeout
>> setting?
> 
> Probably not.
> 
> Are you 100% sure your backend is not closing the connection ?
> 
> Use tcpdump or similar to find out...
> 
> -- 
> Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
> phk at FreeBSD.ORG         | TCP/IP since RFC 956
> FreeBSD committer       | BSD since 4.3-tahoe    
> Never attribute to malice what can adequately be explained by incompetence.
> 
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc




From joao.lisanti at locaweb.com.br  Wed Feb 23 02:54:14 2011
From: joao.lisanti at locaweb.com.br (=?iso-8859-1?Q?Jo=E3o_Gabriel?=)
Date: Tue, 22 Feb 2011 23:54:14 -0300
Subject: 503 returning FetchError c http first read error: -1 0 
In-Reply-To: <C462D1EF-FC43-4033-8670-D6C43432FD9E@locaweb.com.br>
References: <20161.1297984755@critter.freebsd.dk>
	<C462D1EF-FC43-4033-8670-D6C43432FD9E@locaweb.com.br>
Message-ID: <78E52E34-3E10-47E4-A4DB-5ACDE3D77F08@locaweb.com.br>

The reason that was generating the error to us was the mod_remoteip, if using
Apache, try disabling the modules and test.

Jo?o Gabriel
CT-Linux




On Feb 22, 2011, at 6:05 PM, Jo?o Gabriel wrote:

> Probably the backend is closing  connection, I'm having the same problem here, for some reason the backend is  
> closing. I'm looking for the reason
> 
> Jo?o Gabriel
> CT-Linux
> 
> 
> 
> 
> On Feb 17, 2011, at 9:19 PM, Poul-Henning Kamp wrote:
> 
>> In message <AANLkTi=Tqzz1_SykDAmgtmzFN_SUKxdYcTC6c=XwRdhH at mail.gmail.com>, Adam
>> McManus writes:
>> 
>>>> To me this looks like your backend does not answer inside the
>>>> "first_byte" timeout.
>>> 
>>> That's what I initially thought, but the 503 is being generated within
>>> 1-2 seconds max of initiating the request, not the 30 seconds being
>>> defined in each backend. Am I misunderstanding the first_byte timeout
>>> setting?
>> 
>> Probably not.
>> 
>> Are you 100% sure your backend is not closing the connection ?
>> 
>> Use tcpdump or similar to find out...
>> 
>> -- 
>> Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
>> phk at FreeBSD.ORG         | TCP/IP since RFC 956
>> FreeBSD committer       | BSD since 4.3-tahoe    
>> Never attribute to malice what can adequately be explained by incompetence.
>> 
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at varnish-cache.org
>> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
> 
> 
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc




From adi at netstyle.ch  Wed Feb 23 14:28:32 2011
From: adi at netstyle.ch (Adrian Lienhard)
Date: Wed, 23 Feb 2011 15:28:32 +0100
Subject: HTTP/304 response from backend takes 15s to be returned to client
Message-ID: <863BA4F8-3476-4219-A313-0DE8767FD860@netstyle.ch>

Hi,

I'm experiencing a problem with passing conditional client requests that are not cached. We use Varnish to cache HTML documents. Static files like CSS are not cached (pass).

The relevant bit of vcl_recv trivially is:

sub vcl_recv {
	...
	if (req.url ~ "\.(css|js|jpg|jpeg|png|gif|mp3|ogg|flv|swf|pdf)$") {
	    return(pass);
	}
	...

If the client sends a conditional request (with If-Modified-Since header) the backend (Apache) responds with a HTTP/304 as expected.

The problem is that in this case it takes Varnish 15 seconds to send the 304 response to the client (this is consistently reproducible and always 15s plus a few milliseconds). If, however, the client request is not a conditional request, everything works as expected.

I've also tested with return(lookup) n vcl_recv, which always misses, but the problem is exactly the same. However if I return(pipe), it works OK.

I post the varnishlog of a failing 304 and of a succeeding 200 below (I've indicated where the 15s delay is). Maybe interesting to know is that I see the request being logged in Apache immediately. Also Apache works just fine without Varnish in front.

Versions:
- varnish-2.1.4 SVN on Debian Squeeze
- Apache (running on another host): 2.2.9

Any help appreciated.

Cheers,
Adrian

BTW: Should I anyway rather do pipe for static resources that are never cached?



Conditional request (FAIL)
==========================
HTTP/304 response takes 15s
-----------------------
   12 SessionOpen  c 212.103.66.219 6147 :80
   12 ReqStart     c 212.103.66.219 6147 391596406
   12 RxRequest    c GET
   12 RxURL        c /_cmsbox_2.2.13_85/layout/default.css
   12 RxProtocol   c HTTP/1.1
   12 RxHeader     c User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3
   12 RxHeader     c Host: adrian.cmsbox.biz
   12 RxHeader     c Accept: */*
   12 RxHeader     c If-Modified-Since: Wed, 16 Feb 2011 18:05:26 GMT
   12 VCL_call     c recv
   12 VCL_return   c pass
   12 VCL_call     c hash
   12 VCL_return   c hash
   12 VCL_call     c pass
   12 VCL_return   c pass
   14 BackendClose - cmsbox
   14 BackendOpen  b cmsbox 212.103.66.208 55801 212.103.66.211 80
   12 Backend      c 14 cmsbox cmsbox
   14 TxRequest    b GET
   14 TxURL        b /_cmsbox_2.2.13_85/layout/default.css
   14 TxProtocol   b HTTP/1.1
   14 TxHeader     b User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3
   14 TxHeader     b Host: adrian.cmsbox.biz
   14 TxHeader     b Accept: */*
   14 TxHeader     b If-Modified-Since: Wed, 16 Feb 2011 18:05:26 GMT
   14 TxHeader     b X-Forwarded-For: 212.103.66.219
   14 TxHeader     b X-Varnish: 391596406
=============== Waits 15s here / backend loggs request immediately  ===============
   14 RxProtocol   b HTTP/1.1
   14 RxStatus     b 304
   14 RxResponse   b Not Modified
   14 RxHeader     b Date: Wed, 23 Feb 2011 13:55:00 GMT
   14 RxHeader     b Server: Apache/2.2.9 (Debian)
   14 RxHeader     b ETag: "c46450-8120-49c6a201bc580"
   14 RxHeader     b Expires: Thu, 23 Feb 2012 13:55:00 GMT
   14 RxHeader     b Cache-Control: max-age=31536000
   14 RxHeader     b Vary: Accept-Encoding
   12 TTL          c 391596406 RFC 31536000 1298469297 0 0 31536000 0
   12 VCL_call     c fetch
   12 VCL_return   c pass
   12 ObjProtocol  c HTTP/1.1
   12 ObjStatus    c 304
   12 ObjResponse  c Not Modified
   12 ObjHeader    c Date: Wed, 23 Feb 2011 13:55:00 GMT
   12 ObjHeader    c Server: Apache/2.2.9 (Debian)
   12 ObjHeader    c ETag: "c46450-8120-49c6a201bc580"
   12 ObjHeader    c Expires: Thu, 23 Feb 2012 13:55:00 GMT
   12 ObjHeader    c Vary: Accept-Encoding
   14 Length       b 0
   14 BackendClose b cmsbox
   12 VCL_call     c deliver
   12 VCL_return   c deliver
   12 TxProtocol   c HTTP/1.1
   12 TxStatus     c 304
   12 TxResponse   c Not Modified
   12 TxHeader     c Server: Apache/2.2.9 (Debian)
   12 TxHeader     c Expires: Thu, 23 Feb 2012 13:55:00 GMT
   12 TxHeader     c Vary: Accept-Encoding
   12 TxHeader     c Content-Length: 0
   12 TxHeader     c Date: Wed, 23 Feb 2011 13:55:12 GMT
   12 TxHeader     c X-Varnish: 391596406
   12 TxHeader     c Age: 15
   12 TxHeader     c Via: 1.1 varnish
   12 TxHeader     c Connection: keep-alive
   12 TxHeader     c X-Cache-Hits: 0
   12 Length       c 0
   12 ReqEnd       c 391596406 1298469297.322397470 1298469312.322380066 0.000116587 14.999900818 0.000081778
   12 SessionClose c EOF
   12 StatSess     c 212.103.66.219 6147 15 1 1 0 1 1 269 0



Non-conditional request (OK)
============================
HTTP/200 response within 100ms or less

------------------------
   12 SessionOpen  c 212.103.66.219 23833 :80
   12 ReqStart     c 212.103.66.219 23833 2103978089
   12 RxRequest    c HEAD
   12 RxURL        c /_cmsbox_2.2.13_85/layout/default.css
   12 RxProtocol   c HTTP/1.1
   12 RxHeader     c User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3
   12 RxHeader     c Host: adrian.cmsbox.biz
   12 RxHeader     c Accept: */*
   12 VCL_call     c recv
   12 VCL_return   c lookup
   12 VCL_call     c hash
   12 VCL_return   c hash
   12 HitPass      c 2103978045
   12 VCL_call     c pass
   12 VCL_return   c pass
   15 BackendClose b cmsbox
   15 BackendOpen  b cmsbox 212.103.66.208 50035 212.103.66.211 80
   12 Backend      c 15 cmsbox cmsbox
   15 TxRequest    b HEAD
   15 TxURL        b /_cmsbox_2.2.13_85/layout/default.css
   15 TxProtocol   b HTTP/1.1
   15 TxHeader     b User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3
   15 TxHeader     b Host: adrian.cmsbox.biz
   15 TxHeader     b Accept: */*
   15 TxHeader     b X-Forwarded-For: 212.103.66.219
   15 TxHeader     b X-Varnish: 2103978089
   15 RxProtocol   b HTTP/1.1
   15 RxStatus     b 200
   15 RxResponse   b OK
   15 RxHeader     b Date: Wed, 23 Feb 2011 09:39:32 GMT
   15 RxHeader     b Server: Apache/2.2.9 (Debian)
   15 RxHeader     b Last-Modified: Wed, 16 Feb 2011 18:05:26 GMT
   15 RxHeader     b ETag: "c46450-8120-49c6a201bc580"
   15 RxHeader     b Accept-Ranges: bytes
   15 RxHeader     b Content-Length: 33056
   15 RxHeader     b Cache-Control: max-age=31536000
   15 RxHeader     b Expires: Thu, 23 Feb 2012 09:39:32 GMT
   15 RxHeader     b Vary: Accept-Encoding
   15 RxHeader     b Content-Type: text/css
   12 TTL          c 2103978089 RFC 31536000 1298453970 0 0 31536000 0
   12 VCL_call     c fetch
   12 VCL_return   c pass
   12 ObjProtocol  c HTTP/1.1
   12 ObjStatus    c 200
   12 ObjResponse  c OK
   12 ObjHeader    c Date: Wed, 23 Feb 2011 09:39:32 GMT
   12 ObjHeader    c Server: Apache/2.2.9 (Debian)
   12 ObjHeader    c Last-Modified: Wed, 16 Feb 2011 18:05:26 GMT
   12 ObjHeader    c ETag: "c46450-8120-49c6a201bc580"
   12 ObjHeader    c Content-Length: 33056
   12 ObjHeader    c Expires: Thu, 23 Feb 2012 09:39:32 GMT
   12 ObjHeader    c Vary: Accept-Encoding
   12 ObjHeader    c Content-Type: text/css
   15 Length       b 0
   15 BackendReuse b cmsbox
   12 VCL_call     c deliver
   12 VCL_return   c deliver
   12 TxProtocol   c HTTP/1.1
   12 TxStatus     c 200
   12 TxResponse   c OK
   12 TxHeader     c Server: Apache/2.2.9 (Debian)
   12 TxHeader     c Last-Modified: Wed, 16 Feb 2011 18:05:26 GMT
   12 TxHeader     c Content-Length: 33056
   12 TxHeader     c Expires: Thu, 23 Feb 2012 09:39:32 GMT
   12 TxHeader     c Vary: Accept-Encoding
   12 TxHeader     c Content-Type: text/css
   12 TxHeader     c Date: Wed, 23 Feb 2011 09:39:30 GMT
   12 TxHeader     c X-Varnish: 2103978089
   12 TxHeader     c Age: 0
   12 TxHeader     c Via: 1.1 varnish
   12 TxHeader     c Connection: keep-alive
   12 TxHeader     c X-Cache-Hits: 0
   12 Length       c 0
   12 ReqEnd       c 2103978089 1298453970.929079056 1298453970.930070400 0.000118494 0.000920773 0.000070572
   12 Debug        c "herding"
   12 SessionClose c no request
   12 StatSess     c 212.103.66.219 23833 0 1 1 0 1 1 333 0
    0 ExpKill      - 2103978084 -10


From wido at widodh.nl  Wed Feb 23 14:35:01 2011
From: wido at widodh.nl (Wido den Hollander)
Date: Wed, 23 Feb 2011 15:35:01 +0100
Subject: HTTP/304 response from backend takes 15s to be returned to client
In-Reply-To: <863BA4F8-3476-4219-A313-0DE8767FD860@netstyle.ch>
References: <863BA4F8-3476-4219-A313-0DE8767FD860@netstyle.ch>
Message-ID: <1298471701.2392.31.camel@wido-desktop>

Hi,

On Wed, 2011-02-23 at 15:28 +0100, Adrian Lienhard wrote:
> Hi,
> 
> I'm experiencing a problem with passing conditional client requests that are not cached. We use Varnish to cache HTML documents. Static files like CSS are not cached (pass).
> 
> The relevant bit of vcl_recv trivially is:
> 
> sub vcl_recv {
> 	...
> 	if (req.url ~ "\.(css|js|jpg|jpeg|png|gif|mp3|ogg|flv|swf|pdf)$") {
> 	    return(pass);
> 	}
> 	...
> 
> If the client sends a conditional request (with If-Modified-Since header) the backend (Apache) responds with a HTTP/304 as expected.
> 
> The problem is that in this case it takes Varnish 15 seconds to send the 304 response to the client (this is consistently reproducible and always 15s plus a few milliseconds). If, however, the client request is not a conditional request, everything works as expected.
> 
> I've also tested with return(lookup) n vcl_recv, which always misses, but the problem is exactly the same. However if I return(pipe), it works OK.
> 
> I post the varnishlog of a failing 304 and of a succeeding 200 below (I've indicated where the 15s delay is). Maybe interesting to know is that I see the request being logged in Apache immediately. Also Apache works just fine without Varnish in front.
> 
> Versions:
> - varnish-2.1.4 SVN on Debian Squeeze
> - Apache (running on another host): 2.2.9
> 

There was a fix regarding this in 2.1.5, I suggest you upgrade to that
version.

Wido

> Any help appreciated.
> 
> Cheers,
> Adrian
> 
> BTW: Should I anyway rather do pipe for static resources that are never cached?

I think pass is fine?

> 
> 
> 
> Conditional request (FAIL)
> ==========================
> HTTP/304 response takes 15s
> -----------------------
>    12 SessionOpen  c 212.103.66.219 6147 :80
>    12 ReqStart     c 212.103.66.219 6147 391596406
>    12 RxRequest    c GET
>    12 RxURL        c /_cmsbox_2.2.13_85/layout/default.css
>    12 RxProtocol   c HTTP/1.1
>    12 RxHeader     c User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3
>    12 RxHeader     c Host: adrian.cmsbox.biz
>    12 RxHeader     c Accept: */*
>    12 RxHeader     c If-Modified-Since: Wed, 16 Feb 2011 18:05:26 GMT
>    12 VCL_call     c recv
>    12 VCL_return   c pass
>    12 VCL_call     c hash
>    12 VCL_return   c hash
>    12 VCL_call     c pass
>    12 VCL_return   c pass
>    14 BackendClose - cmsbox
>    14 BackendOpen  b cmsbox 212.103.66.208 55801 212.103.66.211 80
>    12 Backend      c 14 cmsbox cmsbox
>    14 TxRequest    b GET
>    14 TxURL        b /_cmsbox_2.2.13_85/layout/default.css
>    14 TxProtocol   b HTTP/1.1
>    14 TxHeader     b User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3
>    14 TxHeader     b Host: adrian.cmsbox.biz
>    14 TxHeader     b Accept: */*
>    14 TxHeader     b If-Modified-Since: Wed, 16 Feb 2011 18:05:26 GMT
>    14 TxHeader     b X-Forwarded-For: 212.103.66.219
>    14 TxHeader     b X-Varnish: 391596406
> =============== Waits 15s here / backend loggs request immediately  ===============
>    14 RxProtocol   b HTTP/1.1
>    14 RxStatus     b 304
>    14 RxResponse   b Not Modified
>    14 RxHeader     b Date: Wed, 23 Feb 2011 13:55:00 GMT
>    14 RxHeader     b Server: Apache/2.2.9 (Debian)
>    14 RxHeader     b ETag: "c46450-8120-49c6a201bc580"
>    14 RxHeader     b Expires: Thu, 23 Feb 2012 13:55:00 GMT
>    14 RxHeader     b Cache-Control: max-age=31536000
>    14 RxHeader     b Vary: Accept-Encoding
>    12 TTL          c 391596406 RFC 31536000 1298469297 0 0 31536000 0
>    12 VCL_call     c fetch
>    12 VCL_return   c pass
>    12 ObjProtocol  c HTTP/1.1
>    12 ObjStatus    c 304
>    12 ObjResponse  c Not Modified
>    12 ObjHeader    c Date: Wed, 23 Feb 2011 13:55:00 GMT
>    12 ObjHeader    c Server: Apache/2.2.9 (Debian)
>    12 ObjHeader    c ETag: "c46450-8120-49c6a201bc580"
>    12 ObjHeader    c Expires: Thu, 23 Feb 2012 13:55:00 GMT
>    12 ObjHeader    c Vary: Accept-Encoding
>    14 Length       b 0
>    14 BackendClose b cmsbox
>    12 VCL_call     c deliver
>    12 VCL_return   c deliver
>    12 TxProtocol   c HTTP/1.1
>    12 TxStatus     c 304
>    12 TxResponse   c Not Modified
>    12 TxHeader     c Server: Apache/2.2.9 (Debian)
>    12 TxHeader     c Expires: Thu, 23 Feb 2012 13:55:00 GMT
>    12 TxHeader     c Vary: Accept-Encoding
>    12 TxHeader     c Content-Length: 0
>    12 TxHeader     c Date: Wed, 23 Feb 2011 13:55:12 GMT
>    12 TxHeader     c X-Varnish: 391596406
>    12 TxHeader     c Age: 15
>    12 TxHeader     c Via: 1.1 varnish
>    12 TxHeader     c Connection: keep-alive
>    12 TxHeader     c X-Cache-Hits: 0
>    12 Length       c 0
>    12 ReqEnd       c 391596406 1298469297.322397470 1298469312.322380066 0.000116587 14.999900818 0.000081778
>    12 SessionClose c EOF
>    12 StatSess     c 212.103.66.219 6147 15 1 1 0 1 1 269 0
> 
> 
> 
> Non-conditional request (OK)
> ============================
> HTTP/200 response within 100ms or less
> 
> ------------------------
>    12 SessionOpen  c 212.103.66.219 23833 :80
>    12 ReqStart     c 212.103.66.219 23833 2103978089
>    12 RxRequest    c HEAD
>    12 RxURL        c /_cmsbox_2.2.13_85/layout/default.css
>    12 RxProtocol   c HTTP/1.1
>    12 RxHeader     c User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3
>    12 RxHeader     c Host: adrian.cmsbox.biz
>    12 RxHeader     c Accept: */*
>    12 VCL_call     c recv
>    12 VCL_return   c lookup
>    12 VCL_call     c hash
>    12 VCL_return   c hash
>    12 HitPass      c 2103978045
>    12 VCL_call     c pass
>    12 VCL_return   c pass
>    15 BackendClose b cmsbox
>    15 BackendOpen  b cmsbox 212.103.66.208 50035 212.103.66.211 80
>    12 Backend      c 15 cmsbox cmsbox
>    15 TxRequest    b HEAD
>    15 TxURL        b /_cmsbox_2.2.13_85/layout/default.css
>    15 TxProtocol   b HTTP/1.1
>    15 TxHeader     b User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3
>    15 TxHeader     b Host: adrian.cmsbox.biz
>    15 TxHeader     b Accept: */*
>    15 TxHeader     b X-Forwarded-For: 212.103.66.219
>    15 TxHeader     b X-Varnish: 2103978089
>    15 RxProtocol   b HTTP/1.1
>    15 RxStatus     b 200
>    15 RxResponse   b OK
>    15 RxHeader     b Date: Wed, 23 Feb 2011 09:39:32 GMT
>    15 RxHeader     b Server: Apache/2.2.9 (Debian)
>    15 RxHeader     b Last-Modified: Wed, 16 Feb 2011 18:05:26 GMT
>    15 RxHeader     b ETag: "c46450-8120-49c6a201bc580"
>    15 RxHeader     b Accept-Ranges: bytes
>    15 RxHeader     b Content-Length: 33056
>    15 RxHeader     b Cache-Control: max-age=31536000
>    15 RxHeader     b Expires: Thu, 23 Feb 2012 09:39:32 GMT
>    15 RxHeader     b Vary: Accept-Encoding
>    15 RxHeader     b Content-Type: text/css
>    12 TTL          c 2103978089 RFC 31536000 1298453970 0 0 31536000 0
>    12 VCL_call     c fetch
>    12 VCL_return   c pass
>    12 ObjProtocol  c HTTP/1.1
>    12 ObjStatus    c 200
>    12 ObjResponse  c OK
>    12 ObjHeader    c Date: Wed, 23 Feb 2011 09:39:32 GMT
>    12 ObjHeader    c Server: Apache/2.2.9 (Debian)
>    12 ObjHeader    c Last-Modified: Wed, 16 Feb 2011 18:05:26 GMT
>    12 ObjHeader    c ETag: "c46450-8120-49c6a201bc580"
>    12 ObjHeader    c Content-Length: 33056
>    12 ObjHeader    c Expires: Thu, 23 Feb 2012 09:39:32 GMT
>    12 ObjHeader    c Vary: Accept-Encoding
>    12 ObjHeader    c Content-Type: text/css
>    15 Length       b 0
>    15 BackendReuse b cmsbox
>    12 VCL_call     c deliver
>    12 VCL_return   c deliver
>    12 TxProtocol   c HTTP/1.1
>    12 TxStatus     c 200
>    12 TxResponse   c OK
>    12 TxHeader     c Server: Apache/2.2.9 (Debian)
>    12 TxHeader     c Last-Modified: Wed, 16 Feb 2011 18:05:26 GMT
>    12 TxHeader     c Content-Length: 33056
>    12 TxHeader     c Expires: Thu, 23 Feb 2012 09:39:32 GMT
>    12 TxHeader     c Vary: Accept-Encoding
>    12 TxHeader     c Content-Type: text/css
>    12 TxHeader     c Date: Wed, 23 Feb 2011 09:39:30 GMT
>    12 TxHeader     c X-Varnish: 2103978089
>    12 TxHeader     c Age: 0
>    12 TxHeader     c Via: 1.1 varnish
>    12 TxHeader     c Connection: keep-alive
>    12 TxHeader     c X-Cache-Hits: 0
>    12 Length       c 0
>    12 ReqEnd       c 2103978089 1298453970.929079056 1298453970.930070400 0.000118494 0.000920773 0.000070572
>    12 Debug        c "herding"
>    12 SessionClose c no request
>    12 StatSess     c 212.103.66.219 23833 0 1 1 0 1 1 333 0
>     0 ExpKill      - 2103978084 -10
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc




From rdelsalle at gmail.com  Wed Feb 23 14:43:05 2011
From: rdelsalle at gmail.com (Roch Delsalle)
Date: Wed, 23 Feb 2011 15:43:05 +0100
Subject: Varnish 2.0.6 & unset beresp.http.set-cookie;
Message-ID: <AANLkTi=YaipdCc7OpmM6MNbHB52i_BnK4Tj93nK-2BTq@mail.gmail.com>

Hi,

I'm trying to use the function *unset beresp.http.set-cookie*;  in my
configuration but it doesn't seem to be recognized with Varnish 2.0.6,
Is there any alternative ?

Thanks

-- 
D-Ro.ch
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110223/28e5b00b/attachment.html>

From mhettwer at team.mobile.de  Wed Feb 23 14:47:40 2011
From: mhettwer at team.mobile.de (Hettwer, Marian)
Date: Wed, 23 Feb 2011 14:47:40 +0000
Subject: Varnish 2.0.6 & unset beresp.http.set-cookie;
In-Reply-To: <AANLkTi=YaipdCc7OpmM6MNbHB52i_BnK4Tj93nK-2BTq@mail.gmail.com>
Message-ID: <C98ADC81.698C%mhettwer@team.mobile.de>



On 23.02.11 15:43, "Roch Delsalle" <rdelsalle at gmail.com> wrote:

>Hi,
>I'm trying to use the function unset beresp.http.set-cookie;  in my
>configuration but it doesn't seem to be recognized with Varnish 2.0.6,
>Is there any alternative ?
Wild guess in the dark: Upgrade to varnish 2.1.5!

HTH,
Marian




From bjorn at ruberg.no  Wed Feb 23 14:47:57 2011
From: bjorn at ruberg.no (=?ISO-8859-1?Q?Bj=F8rn_Ruberg?=)
Date: Wed, 23 Feb 2011 15:47:57 +0100
Subject: Varnish 2.0.6 & unset beresp.http.set-cookie;
In-Reply-To: <AANLkTi=YaipdCc7OpmM6MNbHB52i_BnK4Tj93nK-2BTq@mail.gmail.com>
References: <AANLkTi=YaipdCc7OpmM6MNbHB52i_BnK4Tj93nK-2BTq@mail.gmail.com>
Message-ID: <4D651E1D.4060405@ruberg.no>

On 02/23/2011 03:43 PM, Roch Delsalle wrote:
> Hi,
>
> I'm trying to use the function /*unset beresp.http.set-cookie*/;  in my
> configuration but it doesn't seem to be recognized with Varnish 2.0.6,
> Is there any alternative ?

Alternative #1: Upgrade to Varnish 2.1, where beresp exists

Alternative #2: Use obj. instead of beresp.

-- 
Bj?rn




From pom at dmsp.de  Wed Feb 23 14:48:46 2011
From: pom at dmsp.de (Stefan Pommerening)
Date: Wed, 23 Feb 2011 15:48:46 +0100
Subject: Varnish 2.0.6 & unset beresp.http.set-cookie;
In-Reply-To: <AANLkTi=YaipdCc7OpmM6MNbHB52i_BnK4Tj93nK-2BTq@mail.gmail.com>
References: <AANLkTi=YaipdCc7OpmM6MNbHB52i_BnK4Tj93nK-2BTq@mail.gmail.com>
Message-ID: <4D651E4E.8010809@dmsp.de>

Am 23.02.2011 15:43, schrieb Roch Delsalle:
> I'm trying to use the function /*unset beresp.http.set-cookie*/;  in 
> my configuration but it doesn't seem to be recognized with Varnish 2.0.6,
> Is there any alternative ?

Yes, please use 'unset obj.http.set-cookie' instead.

Before varnish 2.1.x this was the usual syntax. Starting with 2.1.x 
obj.* was changed to beresp.*

> Thanks
>
> -- 
> D-Ro.ch <http://D-Ro.ch>
>

-- 

*Dipl.-Inform. Stefan Pommerening
Informatik-B?ro: IT-Dienste & Projekte, Consulting & Coaching*
http://www.dmsp.de <http://www.dmsp.de/>





From rdelsalle at gmail.com  Wed Feb 23 14:53:14 2011
From: rdelsalle at gmail.com (Roch Delsalle)
Date: Wed, 23 Feb 2011 15:53:14 +0100
Subject: Varnish 2.0.6 & unset beresp.http.set-cookie;
In-Reply-To: <4D651E4E.8010809@dmsp.de>
References: <AANLkTi=YaipdCc7OpmM6MNbHB52i_BnK4Tj93nK-2BTq@mail.gmail.com>
	<4D651E4E.8010809@dmsp.de>
Message-ID: <AANLkTimY8g_K9VJyempqrfTXr1PVqchu0E81ZO+WiYo-@mail.gmail.com>

Thanks for your answer, unfortunately it looks like we can't use it inside
vcl_fetch

Message from VCC-compiler:
Variable 'obj.http.set-cookie' not accessible in method 'vcl_fetch'.


On Wed, Feb 23, 2011 at 3:48 PM, Stefan Pommerening <pom at dmsp.de> wrote:

> Am 23.02.2011 15:43, schrieb Roch Delsalle:
>
>  I'm trying to use the function /*unset beresp.http.set-cookie*/;  in my
>> configuration but it doesn't seem to be recognized with Varnish 2.0.6,
>> Is there any alternative ?
>>
>
> Yes, please use 'unset obj.http.set-cookie' instead.
>
> Before varnish 2.1.x this was the usual syntax. Starting with 2.1.x obj.*
> was changed to beresp.*
>
>  Thanks
>>
>> --
>> D-Ro.ch <http://D-Ro.ch>
>>
>>
> --
>
> *Dipl.-Inform. Stefan Pommerening
> Informatik-B?ro: IT-Dienste & Projekte, Consulting & Coaching*
> http://www.dmsp.de <http://www.dmsp.de/>
>
>
>


-- 
Roch Delsalle
? +33 6 99 11 14 22
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110223/56ef14db/attachment.html>

From stewsnooze at gmail.com  Wed Feb 23 17:40:48 2011
From: stewsnooze at gmail.com (Stewart Robinson)
Date: Wed, 23 Feb 2011 17:40:48 +0000
Subject: Varnish 2.0.6 & unset beresp.http.set-cookie;
In-Reply-To: <AANLkTi=YaipdCc7OpmM6MNbHB52i_BnK4Tj93nK-2BTq@mail.gmail.com>
References: <AANLkTi=YaipdCc7OpmM6MNbHB52i_BnK4Tj93nK-2BTq@mail.gmail.com>
Message-ID: <1259575287341211151@unknownmsgid>

On 23 Feb 2011, at 14:43, Roch Delsalle <rdelsalle at gmail.com> wrote:

Hi,

I'm trying to use the function *unset beresp.http.set-cookie*;  in my
configuration but it doesn't seem to be recognized with Varnish 2.0.6,
Is there any alternative ?

Thanks

-- 
D-Ro.ch

_______________________________________________
varnish-misc mailing list
varnish-misc at varnish-cache.org
http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

Use obj instead of beresp

Cheers
Stew

Sent from my iPhone
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110223/daa8b878/attachment.html>

From rdelsalle at gmail.com  Wed Feb 23 17:45:44 2011
From: rdelsalle at gmail.com (Roch Delsalle)
Date: Wed, 23 Feb 2011 18:45:44 +0100
Subject: Varnish 2.0.6 & unset beresp.http.set-cookie;
In-Reply-To: <1259575287341211151@unknownmsgid>
References: <AANLkTi=YaipdCc7OpmM6MNbHB52i_BnK4Tj93nK-2BTq@mail.gmail.com>
	<1259575287341211151@unknownmsgid>
Message-ID: <AANLkTinZ_FjakvoeA0wxKLAUxFuaBA_uJV55+Az7C5mT@mail.gmail.com>

Thanks, I have updated to the last vestion.

On Wed, Feb 23, 2011 at 6:40 PM, Stewart Robinson <stewsnooze at gmail.com>wrote:

>
>
> On 23 Feb 2011, at 14:43, Roch Delsalle <rdelsalle at gmail.com> wrote:
>
> Hi,
>
> I'm trying to use the function *unset beresp.http.set-cookie*;  in my
> configuration but it doesn't seem to be recognized with Varnish 2.0.6,
> Is there any alternative ?
>
> Thanks
>
> --
> <http://D-Ro.ch>D-Ro.ch
>
> _______________________________________________
> varnish-misc mailing list
>
> varnish-misc at varnish-cache.org
> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
> Use obj instead of beresp
>
> Cheers
> Stew
>
> Sent from my iPhone
>



-- 
D-Ro.ch
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110223/fec5a862/attachment.html>

From justinp at distribion.com  Wed Feb 23 19:13:41 2011
From: justinp at distribion.com (Justin Pasher)
Date: Wed, 23 Feb 2011 13:13:41 -0600
Subject: git branch for 2.1.5
Message-ID: <4D655C65.1050401@distribion.com>

Hello all,

I'm trying to checkout the latest version (2.1.5) from the repository. 
I'm new to git, so I'm fumbling around a little bit. When I do a "git 
branch -r", the latest tag I see is origin/tags/varnish-2.1.4. It 
doesn't look like a tag exists for 2.1.5. What is the best way to check 
out the 2.1.5 branch without grabbing the latest revisions after that 
release? Thanks.

-- 
Justin Pasher



From ketil at froyn.name  Wed Feb 23 19:34:55 2011
From: ketil at froyn.name (Ketil Froyn)
Date: Wed, 23 Feb 2011 20:34:55 +0100
Subject: git branch for 2.1.5
In-Reply-To: <4D655C65.1050401@distribion.com>
References: <4D655C65.1050401@distribion.com>
Message-ID: <AANLkTik-9=zwYTVL6O7o7nBQTzkLRwWC0qOVyed-viWC@mail.gmail.com>

On Wed, Feb 23, 2011 at 8:13 PM, Justin Pasher <justinp at distribion.com> wrote:
> Hello all,
>
> I'm trying to checkout the latest version (2.1.5) from the repository. I'm
> new to git, so I'm fumbling around a little bit. When I do a "git branch
> -r", the latest tag I see is origin/tags/varnish-2.1.4. It doesn't look like
> a tag exists for 2.1.5. What is the best way to check out the 2.1.5 branch
> without grabbing the latest revisions after that release? Thanks.

There's a tag called varnish-2.1.5 (and a tag for all the other
versions too), so you can do:

git checkout varnish-2.1.5

I suspect the reason is the recent conversion from subversion to git,
and how git handles tags in subversion when importing. Subversion
doesn't know the difference between branches and tags, it's more a
matter of convention there.

Cheers, Ketil



From ccastro at altavoz.net  Wed Feb 23 20:50:39 2011
From: ccastro at altavoz.net (Claudio Castro)
Date: Wed, 23 Feb 2011 17:50:39 -0300
Subject: Partial content problem with facebook
Message-ID: <4D65731F.9070508@altavoz.net>

Hi everybody,

I'm having trouble integrating one of my host domains with facebook 
"share" (http://www.facebook.com/sharer.php), i already fix it to 
another site changing http_headers.h on 2.1.3, but it doesn't work for 
this one, then i try upgrading varnish to 2.1.5 but it fails too, it 
only works directly to the backend.

Any advice?


-- 
Claudio Castro N.
Jefe de Plataforma
AltaVoz S.A.
  http://www.altavoz.net
Vi?a del Mar:
  2 Poniente 355 of 53
  +56 32 276 8060
Santiago:
  Guardia Vieja 255, oficina 612
  +56 2 585 4264




From stewsnooze at gmail.com  Wed Feb 23 21:21:41 2011
From: stewsnooze at gmail.com (Stewart Robinson)
Date: Wed, 23 Feb 2011 21:21:41 +0000
Subject: Partial content problem with facebook
In-Reply-To: <4D65731F.9070508@altavoz.net>
References: <4D65731F.9070508@altavoz.net>
Message-ID: <-2154442578084817260@unknownmsgid>

On 23 Feb 2011, at 20:54, Claudio Castro <ccastro at altavoz.net> wrote:

> Hi everybody,
>
> I'm having trouble integrating one of my host domains with facebook "share" (http://www.facebook.com/sharer.php), i already fix it to another site changing http_headers.h on 2.1.3, but it doesn't work for this one, then i try upgrading varnish to 2.1.5 but it fails too, it only works directly to the backend.
>
> Any advice?
>
>
> --
> Claudio Castro N.
> Jefe de Plataforma
> AltaVoz S.A.
> http://www.altavoz.net
> Vi?a del Mar:
> 2 Poniente 355 of 53
> +56 32 276 8060
> Santiago:
> Guardia Vieja 255, oficina 612
> +56 2 585 4264
>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

Hi,
Are you using Facebook.com as a varnish backend for some calls from
your application?

If not I don't see how varnish is involved with a http request to Facebook?

Stew



From ccastro at altavoz.net  Wed Feb 23 21:37:47 2011
From: ccastro at altavoz.net (Claudio Castro)
Date: Wed, 23 Feb 2011 18:37:47 -0300
Subject: Partial content problem with facebook
In-Reply-To: <-2154442578084817260@unknownmsgid>
References: <4D65731F.9070508@altavoz.net> <-2154442578084817260@unknownmsgid>
Message-ID: <4D657E2B.5070307@altavoz.net>

Thanks for the quick answer i gonna try to explain it in a better way.

In the page im hosting exist a button to share the page on facebook, 
this button call a facebook php: http://www.facebook.com/sharer.php? 
with the URL of the page you want to share as a argument, then facebook 
calls my webserver page and insert an "article" on the visitor fb page, 
but when we try to do this trough varnish its return 206 partial 
content, it happened before with 2.1.3 on other host and it was fix 
editing include/http_headers.h and recompiling but for this host in 
particular that doesn't work, neither upgrading to 2.1.5.

varnishlog:


   18 SessionOpen  c 66.220.156.248 59158 200.91.41.13:80
   18 ReqStart     c 66.220.156.248 59158 465084232
   18 RxRequest    c GET
   18 RxURL        c /fi/fa/fo/fu.html
   18 RxProtocol   c HTTP/1.1
   18 RxHeader     c User-Agent: facebookexternalhit/1.1 
(+http://www.facebook.com/externalhit_uatext.php)
   18 RxHeader     c Host: www.xx.cl
   18 RxHeader     c Accept: */*
   18 RxHeader     c Accept-Encoding: gzip
   18 RxHeader     c Range: bytes=0-40960
   18 VCL_call     c recv pass
   18 VCL_call     c hash hash
   18 VCL_call     c pass pass
   18 Backend      c 44 default default
   18 TTL          c 465084232 RFC 120 1298496205 0 0 0 0
   18 VCL_call     c fetch restart
   18 ObjProtocol  c HTTP/1.1
   18 ObjStatus    c 206
   18 ObjResponse  c Partial Content
   18 ObjHeader    c Date: Wed, 23 Feb 2011 21:23:26 GMT
   18 ObjHeader    c Server: Apache
   18 ObjHeader    c Vary: Accept-Encoding
   18 ObjHeader    c Content-Encoding: gzip
   18 ObjHeader    c Content-Range: bytes 0-13900/13901
   18 ObjHeader    c Content-Length: 13901
   18 ObjHeader    c Content-Type: text/html; charset=UTF-8
   18 VCL_call     c recv pass
   18 VCL_call     c hash hash
   18 VCL_call     c pass pass
   18 Backend      c 44 default default
   18 TTL          c 465084232 RFC 120 1298496205 0 0 0 0
   18 VCL_call     c fetch restart                        <------ This 
restart is because i use a inexistent page to capture the log, and i
                                                                                      
have a restart on 404 in my vcl
   18 ObjProtocol  c HTTP/1.1
   18 ObjStatus    c 206
   18 ObjResponse  c Partial Content
   18 ObjHeader    c Date: Wed, 23 Feb 2011 21:23:26 GMT
   18 ObjHeader    c Server: Apache
   18 ObjHeader    c Vary: Accept-Encoding
   18 ObjHeader    c Content-Encoding: gzip
   18 ObjHeader    c Content-Range: bytes 0-13900/13901
   18 ObjHeader    c Content-Length: 13901
   18 ObjHeader    c Content-Type: text/html; charset=UTF-8
   18 VCL_call     c recv pass
   18 VCL_call     c hash hash
   18 VCL_call     c pass pass
   18 Backend      c 44 default default
   18 TTL          c 465084232 RFC 120 1298496205 0 0 0 0
   18 VCL_call     c fetch restart
   18 ObjProtocol  c HTTP/1.1
   18 ObjStatus    c 206
   18 ObjResponse  c Partial Content
   18 ObjHeader    c Date: Wed, 23 Feb 2011 21:23:26 GMT
   18 ObjHeader    c Server: Apache
   18 ObjHeader    c Vary: Accept-Encoding
   18 ObjHeader    c Content-Encoding: gzip
   18 ObjHeader    c Content-Range: bytes 0-13900/13901
   18 ObjHeader    c Content-Length: 13901
   18 ObjHeader    c Content-Type: text/html; charset=UTF-8
   18 VCL_call     c recv pass
   18 VCL_call     c hash hash
   18 VCL_call     c pass pass
   18 Backend      c 44 default default
   18 TTL          c 465084232 RFC 120 1298496205 0 0 0 0
   18 VCL_call     c fetch restart
   18 ObjProtocol  c HTTP/1.1
   18 ObjStatus    c 206
   18 ObjResponse  c Partial Content
   18 ObjHeader    c Date: Wed, 23 Feb 2011 21:23:26 GMT
   18 ObjHeader    c Server: Apache
   18 ObjHeader    c Vary: Accept-Encoding
   18 ObjHeader    c Content-Encoding: gzip
   18 ObjHeader    c Content-Range: bytes 0-13900/13901
   18 ObjHeader    c Content-Length: 13901
   18 ObjHeader    c Content-Type: text/html; charset=UTF-8
   18 VCL_call     c recv pass
   18 VCL_call     c error deliver
   18 VCL_call     c deliver deliver
   18 TxProtocol   c HTTP/1.1
   18 TxStatus     c 206
   18 TxResponse   c Partial Content
   18 TxHeader     c Server: Varnish
   18 TxHeader     c Retry-After: 0
   18 TxHeader     c Content-Type: text/html; charset=utf-8
   18 TxHeader     c Content-Length: 315
   18 TxHeader     c Date: Wed, 23 Feb 2011 21:23:25 GMT
   18 TxHeader     c X-Varnish: 465084232
   18 TxHeader     c Age: 0
   18 TxHeader     c Via: 1.1 varnish
   18 TxHeader     c Connection: close
   18 Length       c 315
   18 ReqEnd       c 465084232 1298496205.030656099 1298496205.104776859 
0.000021458 0.074101925 0.000018835

Stewart Robinson wrote:
> On 23 Feb 2011, at 20:54, Claudio Castro <ccastro at altavoz.net> wrote:
>
>   
>> Hi everybody,
>>
>> I'm having trouble integrating one of my host domains with facebook "share" (http://www.facebook.com/sharer.php), i already fix it to another site changing http_headers.h on 2.1.3, but it doesn't work for this one, then i try upgrading varnish to 2.1.5 but it fails too, it only works directly to the backend.
>>
>> Any advice?
>>
>>
>> --
>> Claudio Castro N.
>> Jefe de Plataforma
>> AltaVoz S.A.
>> http://www.altavoz.net
>> Vi?a del Mar:
>> 2 Poniente 355 of 53
>> +56 32 276 8060
>> Santiago:
>> Guardia Vieja 255, oficina 612
>> +56 2 585 4264
>>
>>
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at varnish-cache.org
>> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>     
>
> Hi,
> Are you using Facebook.com as a varnish backend for some calls from
> your application?
>
> If not I don't see how varnish is involved with a http request to Facebook?
>
> Stew
>   


-- 
Claudio Castro N.
Jefe de Plataforma
AltaVoz S.A.
  http://www.altavoz.net
Vi?a del Mar:
  2 Poniente 355 of 53
  +56 32 276 8060
Santiago:
  Guardia Vieja 255, oficina 612
  +56 2 585 4264




From simon at darkmere.gen.nz  Wed Feb 23 22:06:11 2011
From: simon at darkmere.gen.nz (Simon Lyall)
Date: Thu, 24 Feb 2011 11:06:11 +1300 (NZDT)
Subject: Something funny with HEAD ? 
Message-ID: <alpine.DEB.2.00.1102241054490.14929@green.darkmere.gen.nz>


I was testing out using a HEAD request to pre-load my cache with newly 
created content (which it does, good) and I noticed a weird timeout.

When I use curl it takes a few seconds before timing out. This is even 
the case when the body is only a few bytes.


# time curl -v -X HEAD http://www.varnish-cache.org

* About to connect() to www.varnish-cache.org port 80
*   Trying 87.238.37.6... connected
* Connected to www.varnish-cache.org (87.238.37.6) port 80
> HEAD / HTTP/1.1
> User-Agent: curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 
OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5
> Host: www.varnish-cache.org
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: Apache/2.2.14 (Ubuntu)
< X-Powered-By: PHP/5.3.2-1ubuntu4.7
< Cache-Control: public, max-age=43200
< Last-Modified: Wed, 23 Feb 2011 11:20:32 +0000
< Expires: Sun, 11 Mar 1984 12:00:00 GMT
< Vary: Cookie,Accept-Encoding
< ETag: "1298460032"
< Content-Type: text/html; charset=utf-8
< Content-Length: 23470
< Date: Wed, 23 Feb 2011 21:58:26 GMT
< X-Varnish: 864838647 864663774
< Age: 38274
< Via: 1.1 varnish
< Connection: keep-alive
* transfer closed with 23470 bytes remaining to read
* Closing connection #0
curl: (18) transfer closed with 23470 bytes remaining to read

real    0m5.833s
user    0m0.002s
sys     0m0.002s


Compare this to a normal GET


# time curl -v http://www.varnish-cache.org > /dev/null
* About to connect() to www.varnish-cache.org port 80
*   Trying 87.238.37.6... connected
* Connected to www.varnish-cache.org (87.238.37.6) port 80
> GET / HTTP/1.1
> User-Agent: curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 
OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5
> Host: www.varnish-cache.org
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: Apache/2.2.14 (Ubuntu)
< X-Powered-By: PHP/5.3.2-1ubuntu4.7
< Cache-Control: public, max-age=43200
< Last-Modified: Wed, 23 Feb 2011 11:20:32 +0000
< Expires: Sun, 11 Mar 1984 12:00:00 GMT
< Vary: Cookie,Accept-Encoding
< ETag: "1298460032"
< Content-Type: text/html; charset=utf-8
< Content-Length: 23470
< Date: Wed, 23 Feb 2011 22:00:22 GMT
< X-Varnish: 864838987 864663774
< Age: 38390
< Via: 1.1 varnish
< Connection: keep-alive
   % Total    % Received % Xferd  Average Speed   Time    Time     Time 
Current
                                  Dload  Upload   Total   Spent    Left 
Speed
100 23470  100 23470    0     0  39348      0 --:--:-- --:--:-- --:--:-- 
72243* Connection #0 to host www.varnish-cache.org left intact

* Closing connection #0

real    0m0.602s
user    0m0.001s
sys     0m0.004s





-- 
Simon Lyall  |  Very Busy  |  Web: http://www.darkmere.gen.nz/
"To stay awake all night adds a day to your life" - Stilgar | eMT.




From ccastro at altavoz.net  Wed Feb 23 22:10:34 2011
From: ccastro at altavoz.net (Claudio Castro)
Date: Wed, 23 Feb 2011 19:10:34 -0300
Subject: Partial content problem with facebook
In-Reply-To: <4D657E2B.5070307@altavoz.net>
References: <4D65731F.9070508@altavoz.net> <-2154442578084817260@unknownmsgid>
	<4D657E2B.5070307@altavoz.net>
Message-ID: <4D6585DA.9070509@altavoz.net>

Meanwhile we put a pipe on useragent ~ facebook, and it works.

maybe someone can find it useful:

sub vcl_recv {

if (req.http.user-agent ~ "facebookexternalhit")
        {
        return(pipe);
        }



Saludos,



Claudio Castro wrote:
> Thanks for the quick answer i gonna try to explain it in a better way.
>
> In the page im hosting exist a button to share the page on facebook, 
> this button call a facebook php: http://www.facebook.com/sharer.php? 
> with the URL of the page you want to share as a argument, then 
> facebook calls my webserver page and insert an "article" on the 
> visitor fb page, but when we try to do this trough varnish its return 
> 206 partial content, it happened before with 2.1.3 on other host and 
> it was fix editing include/http_headers.h and recompiling but for this 
> host in particular that doesn't work, neither upgrading to 2.1.5.
>
> varnishlog:
>
>
>   18 SessionOpen  c 66.220.156.248 59158 200.91.41.13:80
>   18 ReqStart     c 66.220.156.248 59158 465084232
>   18 RxRequest    c GET
>   18 RxURL        c /fi/fa/fo/fu.html
>   18 RxProtocol   c HTTP/1.1
>   18 RxHeader     c User-Agent: facebookexternalhit/1.1 
> (+http://www.facebook.com/externalhit_uatext.php)
>   18 RxHeader     c Host: www.xx.cl
>   18 RxHeader     c Accept: */*
>   18 RxHeader     c Accept-Encoding: gzip
>   18 RxHeader     c Range: bytes=0-40960
>   18 VCL_call     c recv pass
>   18 VCL_call     c hash hash
>   18 VCL_call     c pass pass
>   18 Backend      c 44 default default
>   18 TTL          c 465084232 RFC 120 1298496205 0 0 0 0
>   18 VCL_call     c fetch restart
>   18 ObjProtocol  c HTTP/1.1
>   18 ObjStatus    c 206
>   18 ObjResponse  c Partial Content
>   18 ObjHeader    c Date: Wed, 23 Feb 2011 21:23:26 GMT
>   18 ObjHeader    c Server: Apache
>   18 ObjHeader    c Vary: Accept-Encoding
>   18 ObjHeader    c Content-Encoding: gzip
>   18 ObjHeader    c Content-Range: bytes 0-13900/13901
>   18 ObjHeader    c Content-Length: 13901
>   18 ObjHeader    c Content-Type: text/html; charset=UTF-8
>   18 VCL_call     c recv pass
>   18 VCL_call     c hash hash
>   18 VCL_call     c pass pass
>   18 Backend      c 44 default default
>   18 TTL          c 465084232 RFC 120 1298496205 0 0 0 0
>   18 VCL_call     c fetch restart                        <------ This 
> restart is because i use a inexistent page to capture the log, and i
>                                                                                      
> have a restart on 404 in my vcl
>   18 ObjProtocol  c HTTP/1.1
>   18 ObjStatus    c 206
>   18 ObjResponse  c Partial Content
>   18 ObjHeader    c Date: Wed, 23 Feb 2011 21:23:26 GMT
>   18 ObjHeader    c Server: Apache
>   18 ObjHeader    c Vary: Accept-Encoding
>   18 ObjHeader    c Content-Encoding: gzip
>   18 ObjHeader    c Content-Range: bytes 0-13900/13901
>   18 ObjHeader    c Content-Length: 13901
>   18 ObjHeader    c Content-Type: text/html; charset=UTF-8
>   18 VCL_call     c recv pass
>   18 VCL_call     c hash hash
>   18 VCL_call     c pass pass
>   18 Backend      c 44 default default
>   18 TTL          c 465084232 RFC 120 1298496205 0 0 0 0
>   18 VCL_call     c fetch restart
>   18 ObjProtocol  c HTTP/1.1
>   18 ObjStatus    c 206
>   18 ObjResponse  c Partial Content
>   18 ObjHeader    c Date: Wed, 23 Feb 2011 21:23:26 GMT
>   18 ObjHeader    c Server: Apache
>   18 ObjHeader    c Vary: Accept-Encoding
>   18 ObjHeader    c Content-Encoding: gzip
>   18 ObjHeader    c Content-Range: bytes 0-13900/13901
>   18 ObjHeader    c Content-Length: 13901
>   18 ObjHeader    c Content-Type: text/html; charset=UTF-8
>   18 VCL_call     c recv pass
>   18 VCL_call     c hash hash
>   18 VCL_call     c pass pass
>   18 Backend      c 44 default default
>   18 TTL          c 465084232 RFC 120 1298496205 0 0 0 0
>   18 VCL_call     c fetch restart
>   18 ObjProtocol  c HTTP/1.1
>   18 ObjStatus    c 206
>   18 ObjResponse  c Partial Content
>   18 ObjHeader    c Date: Wed, 23 Feb 2011 21:23:26 GMT
>   18 ObjHeader    c Server: Apache
>   18 ObjHeader    c Vary: Accept-Encoding
>   18 ObjHeader    c Content-Encoding: gzip
>   18 ObjHeader    c Content-Range: bytes 0-13900/13901
>   18 ObjHeader    c Content-Length: 13901
>   18 ObjHeader    c Content-Type: text/html; charset=UTF-8
>   18 VCL_call     c recv pass
>   18 VCL_call     c error deliver
>   18 VCL_call     c deliver deliver
>   18 TxProtocol   c HTTP/1.1
>   18 TxStatus     c 206
>   18 TxResponse   c Partial Content
>   18 TxHeader     c Server: Varnish
>   18 TxHeader     c Retry-After: 0
>   18 TxHeader     c Content-Type: text/html; charset=utf-8
>   18 TxHeader     c Content-Length: 315
>   18 TxHeader     c Date: Wed, 23 Feb 2011 21:23:25 GMT
>   18 TxHeader     c X-Varnish: 465084232
>   18 TxHeader     c Age: 0
>   18 TxHeader     c Via: 1.1 varnish
>   18 TxHeader     c Connection: close
>   18 Length       c 315
>   18 ReqEnd       c 465084232 1298496205.030656099 
> 1298496205.104776859 0.000021458 0.074101925 0.000018835
>
> Stewart Robinson wrote:
>> On 23 Feb 2011, at 20:54, Claudio Castro <ccastro at altavoz.net> wrote:
>>
>>  
>>> Hi everybody,
>>>
>>> I'm having trouble integrating one of my host domains with facebook 
>>> "share" (http://www.facebook.com/sharer.php), i already fix it to 
>>> another site changing http_headers.h on 2.1.3, but it doesn't work 
>>> for this one, then i try upgrading varnish to 2.1.5 but it fails 
>>> too, it only works directly to the backend.
>>>
>>> Any advice?
>>>
>>>
>>> -- 
>>> Claudio Castro N.
>>> Jefe de Plataforma
>>> AltaVoz S.A.
>>> http://www.altavoz.net
>>> Vi?a del Mar:
>>> 2 Poniente 355 of 53
>>> +56 32 276 8060
>>> Santiago:
>>> Guardia Vieja 255, oficina 612
>>> +56 2 585 4264
>>>
>>>
>>> _______________________________________________
>>> varnish-misc mailing list
>>> varnish-misc at varnish-cache.org
>>> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>>     
>>
>> Hi,
>> Are you using Facebook.com as a varnish backend for some calls from
>> your application?
>>
>> If not I don't see how varnish is involved with a http request to 
>> Facebook?
>>
>> Stew
>>   
>
>


-- 
Claudio Castro N.
Jefe de Plataforma
AltaVoz S.A.
  http://www.altavoz.net
Vi?a del Mar:
  2 Poniente 355 of 53
  +56 32 276 8060
Santiago:
  Guardia Vieja 255, oficina 612
  +56 2 585 4264




From gbell at teksystems.com  Wed Feb 23 22:46:56 2011
From: gbell at teksystems.com (Bell, Garrett)
Date: Wed, 23 Feb 2011 17:46:56 -0500
Subject: Networking - San Antonio, TX
Message-ID: <6B70F76238CBCA44968B3322F169CD256E01A2B79D@EXCH-MBX04.allegisgroup.com>

To whom it may concern,

I am a recruiter with TEKsystems in San Antonio, TX. I am currently networking with organizations to help find resources for positions with our local clients. I am wanting to network with your group and see if anyone would be able to give me some advice on sourcing for an opportunity in San Antonio, TX.

If anyone can point me in the right direction where I might be able to find professionals who have a strong Linux background and have also been working with Varnish recently please let me know; I would really appreciate any help.

I can be reached at 210-384-3125 or gbell at teksystems.com<mailto:gbell at teksystems.com>

Thank you,
Garrett Bell

________________________________
 Garrett Bell Technical Recruiter
1100 NW Loop 410  Suite 210, San Antonio, TX 78213
Toll-Free 877.856.2509
Office 210.384.3125
Fax 210.384.3050
[http://media03.linkedin.com/media/p/3/000/01a/024/153746b.png]
TEKsystems is the # 1 IT and Communications Staffing Company in the U.S. for the 10th consecutive year.
________________________________
Are you on LinkedIn? If so I would like to be connected to your network.   www.linkedin.com/in/garrettbell<http://www.linkedin.com/in/garrettbell>


________________________________
This electronic mail (including any attachments) may contain information that is privileged, confidential, and/or otherwise protected from disclosure to anyone other than its intended recipient(s). Any dissemination or use of this electronic mail or its contents (including any attachments) by persons other than the intended recipient(s) is strictly prohibited. If you have received this message in error, please notify us immediately by reply e-mail so that we may correct our internal records. Please then delete the original message (including any attachments) in its entirety. Thank you.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110223/2115ed94/attachment.html>

From tfheen at varnish-software.com  Thu Feb 24 09:49:33 2011
From: tfheen at varnish-software.com (Tollef Fog Heen)
Date: Thu, 24 Feb 2011 10:49:33 +0100
Subject: git branch for 2.1.5
In-Reply-To: <AANLkTik-9=zwYTVL6O7o7nBQTzkLRwWC0qOVyed-viWC@mail.gmail.com>
	(Ketil Froyn's message of "Wed, 23 Feb 2011 20:34:55 +0100")
References: <4D655C65.1050401@distribion.com>
	<AANLkTik-9=zwYTVL6O7o7nBQTzkLRwWC0qOVyed-viWC@mail.gmail.com>
Message-ID: <87bp21oj82.fsf@qurzaw.varnish-software.com>

]] Ketil Froyn 

| On Wed, Feb 23, 2011 at 8:13 PM, Justin Pasher <justinp at distribion.com> wrote:
| > Hello all,
| >
| > I'm trying to checkout the latest version (2.1.5) from the repository. I'm
| > new to git, so I'm fumbling around a little bit. When I do a "git branch
| > -r", the latest tag I see is origin/tags/varnish-2.1.4. It doesn't look like
| > a tag exists for 2.1.5. What is the best way to check out the 2.1.5 branch
| > without grabbing the latest revisions after that release? Thanks.
| 
| There's a tag called varnish-2.1.5 (and a tag for all the other
| versions too), so you can do:
| 
| git checkout varnish-2.1.5

Correct.

| I suspect the reason is the recent conversion from subversion to git,
| and how git handles tags in subversion when importing. Subversion
| doesn't know the difference between branches and tags, it's more a
| matter of convention there.

Again, correct.

-- 
Tollef Fog Heen
Varnish Software
t: +47 21 98 92 64



From indranilc at rediff-inc.com  Fri Feb 25 12:08:07 2011
From: indranilc at rediff-inc.com (Indranil Chakravorty)
Date: 25 Feb 2011 12:08:07 -0000
Subject: =?utf-8?B?VmFybmlzaCB3aXRoIHNxdWlkIGJhY2tlbmQ=?=
Message-ID: <1298634277.S.15143.H.TkFuYW5kIFNoYWgAdmFybmlzaCBtYWlsaW5nbGlzdA__.51528.pro-237-108.old.1298635687.587@webmail.rediffmail.com>

Hi,  I am using squid for byte range support for playing videos.My initial lands up on varnish and then I send a call to squid which acts as myorigin for content delivery. I am getting below errors every time I try to callsquid and serve that file.      13 TxHeader     c Expires:Wed, 22 Feb 2012 03:26:22 GMT   13 TxHeader     cX-SO-Time: Tue Feb 22 8:56:22 IST 2011   13 TxHeader     cContent-Type: video/flv   13 TxHeader     cLast-Modified: Fri, 25 Feb 2011 11:30:20 GMT   13 TxHeader     cCache-Control: max-age=31536000   13 TxHeader     cContent-Length: 76445689   13 TxHeader     c Date: Fri,25 Feb 2011 11:30:29 GMT   13 TxHeader     cConnection: keep-alive   13 TxHeader     c X-Cache:TCP_MISS   13Debug        c "Write error, len =68400/76446042, errno = Success"  Analysis: Session traced from Client to Varnish to squid and back.  Client ? varnish ? squid then Squid ? varnish ? client  Everything goes well.  Squid sends complete data tovarnish but it inturn does not return it back to client which is holding thesession.  I tried modifying some parameters   Param                           Default                         Modified fetch_chunksize           128 [kilobytes]               512send_timeout              600 [seconds]                1800sess_timeout              5 [seconds]                    600between_bytes_timeout     60.000000 [s]           120  But did not succeed; is there anything which makes varnishirresponsive; since the second request where varnish has cached that data iswell responded.   Is there any way where I can make this work on firstrequest?   Regards,     Anand
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110225/3cda3bbe/attachment.html>

From l at lrowe.co.uk  Sat Feb 26 11:27:59 2011
From: l at lrowe.co.uk (Laurence Rowe)
Date: Sat, 26 Feb 2011 11:27:59 +0000
Subject: Varnish with squid backend
In-Reply-To: <1298634277.S.15143.H.TkFuYW5kIFNoYWgAdmFybmlzaCBtYWlsaW5nbGlzdA__.51528.pro-237-108.old.1298635687.587@webmail.rediffmail.com>
References: <1298634277.S.15143.H.TkFuYW5kIFNoYWgAdmFybmlzaCBtYWlsaW5nbGlzdA__.51528.pro-237-108.old.1298635687.587@webmail.rediffmail.com>
Message-ID: <AANLkTind9AxgpVH6dDnhMaOiSOzHECRCaSygGN2jQJmr@mail.gmail.com>

On 25 February 2011 12:08, Indranil Chakravorty
<indranilc at rediff-inc.com> wrote:
> Hi,
>
> I am using squid for byte range support for playing videos.My initial lands
> up on varnish and then I send a call to squid which acts as myorigin for
> content delivery. I am getting below errors every time I try to callsquid
> and serve that file.
>
> 13 TxHeader c Expires:Wed, 22 Feb 2012 03:26:22 GMT
>
> 13 TxHeader cX-SO-Time: Tue Feb 22 8:56:22 IST 2011
>
> 13 TxHeader cContent-Type: video/flv
>
> 13 TxHeader cLast-Modified: Fri, 25 Feb 2011 11:30:20 GMT
>
> 13 TxHeader cCache-Control: max-age=31536000
>
> 13 TxHeader cContent-Length: 76445689
>
> 13 TxHeader c Date: Fri,25 Feb 2011 11:30:29 GMT
>
> 13 TxHeader cConnection: keep-alive
>
> 13 TxHeader c X-Cache:TCP_MISS
>
> 13Debug c "Write error, len =68400/76446042, errno = Success"
>
> Analysis:
>
> Session traced from Client to Varnish to squid and back.
>
> Client ? varnish ? squid then
>
> Squid ? varnish ? client
>
> Everything goes well. Squid sends complete data tovarnish but it inturn does
> not return it back to client which is holding thesession.
>
> I tried modifying some parameters
>
> Param Default Modified
>
> fetch_chunksize 128 [kilobytes] 512
>
> send_timeout 600 [seconds] 1800
>
> sess_timeout 5 [seconds] 600
>
> between_bytes_timeout 60.000000 [s] 120
>
> But did not succeed; is there anything which makes varnishirresponsive;
> since the second request where varnish has cached that data iswell
> responded.
>
> Is there any way where I can make this work on firstrequest?

You need to provide information about the version of Varnish and the
VCL that is being used for this request.

FWIW, I've found Varnish to be a poor fit for caching large media
files which get requested only occasionally as it must read the entire
file in before returning data to the client, so for these I just pipe
the request to the backend nginx server.

Laurence



From simon at darkmere.gen.nz  Mon Feb 28 02:23:57 2011
From: simon at darkmere.gen.nz (Simon Lyall)
Date: Mon, 28 Feb 2011 15:23:57 +1300 (NZDT)
Subject: Possible bug with Varnish + mp4 vs iphone/ipad
Message-ID: <alpine.DEB.2.00.1102281506570.25029@green.darkmere.gen.nz>


There is some sort of bug ( probably at the apple end ) with iphones and 
ipads downloading mp4 files off varnish. No problems with firefox and IE 
grabbing the same files. Not sure if it was affecting desktop safari.

The symptom is that the ipad just says:

" Cannot play remote file "

Everything looks okay in the logs and varnishlog and I couldn't make sense 
of the tcpdump. In the end we found:

https://bugs.launchpad.net/pantheon/+bug/674492

if (req.url ~ "\.mp4") {
     return (pipe);
}

Which works, although obviously not in a good way.

Running 2.1.3


Just like to say that over the last week varnish has really saved our 
bacon, we have had a huge traffic increase and accelerating our varnish 
deployment got us through it.

-- 
Simon Lyall  |  Very Busy  |  Web: http://www.darkmere.gen.nz/
"To stay awake all night adds a day to your life" - Stilgar | eMT.




From tfheen at varnish-software.com  Mon Feb 28 07:55:29 2011
From: tfheen at varnish-software.com (Tollef Fog Heen)
Date: Mon, 28 Feb 2011 08:55:29 +0100
Subject: Possible bug with Varnish + mp4 vs iphone/ipad
In-Reply-To: <alpine.DEB.2.00.1102281506570.25029@green.darkmere.gen.nz>
	(Simon Lyall's message of "Mon, 28 Feb 2011 15:23:57 +1300 (NZDT)")
References: <alpine.DEB.2.00.1102281506570.25029@green.darkmere.gen.nz>
Message-ID: <87sjv8twy6.fsf@qurzaw.varnish-software.com>

]] Simon Lyall 

| Which works, although obviously not in a good way.
| 
| Running 2.1.3

I believe it requires Range support.  You might want to upgrade to 2.1.5
and enable range and see if that fixes it.

Regards,
-- 
Tollef Fog Heen
Varnish Software
t: +47 21 98 92 64



From kpettijohn at dailyinsightgroup.com  Fri Feb 25 18:55:03 2011
From: kpettijohn at dailyinsightgroup.com (Kevin Pettijohn)
Date: Fri, 25 Feb 2011 18:55:03 +0000
Subject: Odd varnish generated 503 issue
Message-ID: <8FE20CEA-2842-4F6F-BB3E-D86A4AFB1B98@tarot.com>

Hello everyone,

I'm having some issues using varnish and pressflow. I am currently running varnish-2.1.5 SVN 0843d7a with pressflow 6.17. The problem is a small corner case and I have not been having much luck figuring it out. If anyone can point me in the right direction that would be much appreciated. The issue is that when a logged in user with administrative privileges tries to view our /forum page it displays the varnish generated error page for a 503 after the 60 second first_byte_timeout.

On the other hand if the user is logged in but does not have administrative privileges the /forum page loads just fine and the same goes for logged out users.

I tried setting a pass in vcl but it didn't seem to help:

// Pass on forum pages
  if (req.url ~ "^/forum$") {
       return (pass);
  }

Here are is the varnishlog output for the failed request (logged in user with admin):

   25 SessionOpen  c 75.146.54.221 53347 :80
   25 ReqStart     c 75.146.54.221 53347 366998573
   25 RxRequest    c GET
   25 RxURL        c /forum
   25 RxProtocol   c HTTP/1.1
   25 RxHeader     c Host: www.example.com<http://www.example.com/>
   25 RxHeader     c Connection: keep-alive
   25 RxHeader     c Referer: http://www.example.com/forum
   25 RxHeader     c Cache-Control: max-age=0
   25 RxHeader     c Accept: application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
   25 RxHeader     c User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_6; en-US) AppleWebKit/534.10 (KHTML, like Gecko) Chrome/8.0.552.215 Safari/534.10
   25 RxHeader     c Accept-Encoding: gzip,deflate,sdch
   25 RxHeader     c Accept-Language: en-US,en;q=0.8
   25 RxHeader     c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
   25 RxHeader     c Cookie: __qca=P0-2045062115-1295000757408; PHPSESSID=hrvrdh7k9qva2544rfcrqm4u91; BCKPHPSESSID=hrvrdh7k9qva2544rfcrqm4u91; __utmz=240455574.1298425082.43.4.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=http://example.com/; SESS26a2330bc4f50e2
   25 VCL_call     c recv pass
   25 VCL_call     c hash hash
   25 VCL_call     c pass pass
   25 Backend      c 21 dh web02
   25 FetchError   c http first read error: -1 11 (Resource temporarily unavailable)
   25 VCL_call     c error deliver
   25 VCL_call     c deliver deliver
   25 TxProtocol   c HTTP/1.1
   25 TxStatus     c 503
   25 TxResponse   c Service Unavailable
   25 TxHeader     c Server: Varnish
   25 TxHeader     c Retry-After: 0
   25 TxHeader     c Content-Type: text/html; charset=UTF-8
   25 TxHeader     c Content-Length: 2385
   25 TxHeader     c Date: Thu, 24 Feb 2011 23:34:12 GMT
   25 TxHeader     c X-Varnish: 366998573
   25 TxHeader     c Age: 60
   25 TxHeader     c Via: 1.1 varnish
   25 TxHeader     c Connection: close
   25 TxHeader     c X-Cache: MISS
   25 Length       c 2385
   25 ReqEnd       c 366998573 1298590392.349307060 1298590452.349412918 0.000030041 60.000077963 0.000027895

    0 ExpKill      - 366993383 -300
    0 ExpKill      - 366993387 -300



Here is the log for a successful load (logged out user):



   32 SessionOpen  c 75.146.54.241 61453 :80
   32 ReqStart     c 75.146.54.241 61453 367412524
   32 RxRequest    c GET
   32 RxURL        c /forum
   32 RxProtocol   c HTTP/1.1
   32 RxHeader     c Host: www.example.com<http://www.example.com/>
   32 RxHeader     c Accept-Encoding: gzip, deflate
   32 RxHeader     c Accept-Language: en-us
   32 RxHeader     c User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_6; en-us) AppleWebKit/533.19.4 (KHTML, like Gecko) Version/5.0.3 Safari/533.19.4
   32 RxHeader     c Accept: application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
   32 RxHeader     c Cookie: __utmc=240455574; __utma=240455574.1048652604.1298005414.1298005414.1298425351.2; __utmz=240455574.1298005414.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)
   32 RxHeader     c Connection: keep-alive
   32 VCL_call     c recv pass
   32 VCL_call     c hash hash
   32 VCL_call     c pass pass
   32 Backend      c 9 dh web03
   32 TTL          c 367412524 RFC 600 1298659484 0 0 600 0
   32 VCL_call     c fetch pass
   32 ObjProtocol  c HTTP/1.1
   32 ObjStatus    c 200
   32 ObjResponse  c OK
   32 ObjHeader    c Date: Fri, 25 Feb 2011 18:45:29 GMT
   32 ObjHeader    c Server: Apache/2.2.8 (Unix) PHP/5.2.5 mod_ssl/2.2.8 OpenSSL/0.9.7a
   32 ObjHeader    c X-Powered-By: PHP/5.2.5
   32 ObjHeader    c Cache-Control: public, max-age=600
   32 ObjHeader    c Last-Modified: Fri, 25 Feb 2011 18:45:29 +0000
   32 ObjHeader    c Expires: Sun, 11 Mar 1984 12:00:00 GMT
   32 ObjHeader    c Vary: Cookie
   32 ObjHeader    c ETag: "1298659529"
   32 ObjHeader    c Content-Type: text/html; charset=utf-8
   32 VCL_call     c deliver deliver
   32 TxProtocol   c HTTP/1.1
   32 TxStatus     c 200
   32 TxResponse   c OK
   32 TxHeader     c Server: Apache/2.2.8 (Unix) PHP/5.2.5 mod_ssl/2.2.8 OpenSSL/0.9.7a
   32 TxHeader     c X-Powered-By: PHP/5.2.5
   32 TxHeader     c Cache-Control: public, max-age=600
   32 TxHeader     c Last-Modified: Fri, 25 Feb 2011 18:45:29 +0000
   32 TxHeader     c Expires: Sun, 11 Mar 1984 12:00:00 GMT
   32 TxHeader     c Vary: Cookie
   32 TxHeader     c ETag: "1298659529"
   32 TxHeader     c Content-Type: text/html; charset=utf-8
   32 TxHeader     c Content-Length: 83552
   32 TxHeader     c Date: Fri, 25 Feb 2011 18:44:45 GMT
   32 TxHeader     c X-Varnish: 367412524
   32 TxHeader     c Age: 0
   32 TxHeader     c Via: 1.1 varnish
   32 TxHeader     c Connection: keep-alive
   32 TxHeader     c X-Cache: MISS
   32 Length       c 83552
   32 ReqEnd       c 367412524 1298659483.560426950 1298659485.242687941 0.000025988 1.473248959 0.209012032


If anyone can give me some insight It would be greatly appreciated
__________________
Kevin Pettijohn
503 927 3585
Operations & IT
The Daily Insight Group
kpettijohn at tarot.com<mailto:kpettijohn at tarot.com>





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110225/1b34fffd/attachment.html>

From stewsnooze at gmail.com  Mon Feb 28 08:19:44 2011
From: stewsnooze at gmail.com (Stewart Robinson)
Date: Mon, 28 Feb 2011 08:19:44 +0000
Subject: Odd varnish generated 503 issue
In-Reply-To: <8FE20CEA-2842-4F6F-BB3E-D86A4AFB1B98@tarot.com>
References: <8FE20CEA-2842-4F6F-BB3E-D86A4AFB1B98@tarot.com>
Message-ID: <885562469490214145@unknownmsgid>

On 28 Feb 2011, at 08:12, Kevin Pettijohn <kpettijohn at dailyinsightgroup.com>
wrote:

Hello everyone,

I'm having some issues using varnish and pressflow. I am currently
running varnish-2.1.5 SVN 0843d7a with pressflow 6.17. The problem is a
small corner case and I have not been having much luck figuring it out. If
anyone can point me in the right direction that would be much appreciated.
The issue is that when a logged in user with administrative privileges tries
to view our /forum page it displays the varnish generated error page for a
503 after the 60 second first_byte_timeout.

On the other hand if the user is logged in but does not have administrative
privileges the /forum page loads just fine and the same goes for logged out
users.

I tried setting a pass in vcl but it didn't seem to help:

// Pass on forum pages
  if (req.url ~ "^/forum$") {
       return (pass);
  }

Here are is the varnishlog output for the failed request (logged in user
with admin):

   25 SessionOpen  c 75.146.54.221 53347 :80
   25 ReqStart     c 75.146.54.221 53347 366998573
   25 RxRequest    c GET
   25 RxURL        c /forum
   25 RxProtocol   c HTTP/1.1
   25 RxHeader     c Host: www.example.com
   25 RxHeader     c Connection: keep-alive
   25 RxHeader     c Referer: http://www.example.com/forum
   25 RxHeader     c Cache-Control: max-age=0
   25 RxHeader     c Accept:
application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
   25 RxHeader     c User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X
10_6_6; en-US) AppleWebKit/534.10 (KHTML, like Gecko) Chrome/8.0.552.215
Safari/534.10
   25 RxHeader     c Accept-Encoding: gzip,deflate,sdch
   25 RxHeader     c Accept-Language: en-US,en;q=0.8
   25 RxHeader     c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
   25 RxHeader     c Cookie: __qca=P0-2045062115-1295000757408;
PHPSESSID=hrvrdh7k9qva2544rfcrqm4u91;
BCKPHPSESSID=hrvrdh7k9qva2544rfcrqm4u91;
__utmz=240455574.1298425082.43.4.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=
http://example.com/; SESS26a2330bc4f50e2
   25 VCL_call     c recv pass
   25 VCL_call     c hash hash
   25 VCL_call     c pass pass
   25 Backend      c 21 dh web02
   25 FetchError   c http first read error: -1 11 (Resource temporarily
unavailable)
   25 VCL_call     c error deliver
   25 VCL_call     c deliver deliver
   25 TxProtocol   c HTTP/1.1
   25 TxStatus     c 503
   25 TxResponse   c Service Unavailable
   25 TxHeader     c Server: Varnish
   25 TxHeader     c Retry-After: 0
   25 TxHeader     c Content-Type: text/html; charset=UTF-8
   25 TxHeader     c Content-Length: 2385
   25 TxHeader     c Date: Thu, 24 Feb 2011 23:34:12 GMT
   25 TxHeader     c X-Varnish: 366998573
   25 TxHeader     c Age: 60
   25 TxHeader     c Via: 1.1 varnish
   25 TxHeader     c Connection: close
   25 TxHeader     c X-Cache: MISS
   25 Length       c 2385
   25 ReqEnd       c 366998573 1298590392.349307060 1298590452.349412918
0.000030041 60.000077963 0.000027895

    0 ExpKill      - 366993383 -300
    0 ExpKill      - 366993387 -300



Here is the log for a successful load (logged out user):



   32 SessionOpen  c 75.146.54.241 61453 :80
   32 ReqStart     c 75.146.54.241 61453 367412524
   32 RxRequest    c GET
   32 RxURL        c /forum
   32 RxProtocol   c HTTP/1.1
   32 RxHeader     c Host: www.example.com
   32 RxHeader     c Accept-Encoding: gzip, deflate
   32 RxHeader     c Accept-Language: en-us
   32 RxHeader     c User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X
10_6_6; en-us) AppleWebKit/533.19.4 (KHTML, like Gecko) Version/5.0.3
Safari/533.19.4
   32 RxHeader     c Accept:
application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
   32 RxHeader     c Cookie: __utmc=240455574;
__utma=240455574.1048652604.1298005414.1298005414.1298425351.2;
__utmz=240455574.1298005414.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)
   32 RxHeader     c Connection: keep-alive
   32 VCL_call     c recv pass
   32 VCL_call     c hash hash
   32 VCL_call     c pass pass
   32 Backend      c 9 dh web03
   32 TTL          c 367412524 RFC 600 1298659484 0 0 600 0
   32 VCL_call     c fetch pass
   32 ObjProtocol  c HTTP/1.1
   32 ObjStatus    c 200
   32 ObjResponse  c OK
   32 ObjHeader    c Date: Fri, 25 Feb 2011 18:45:29 GMT
   32 ObjHeader    c Server: Apache/2.2.8 (Unix) PHP/5.2.5 mod_ssl/2.2.8
OpenSSL/0.9.7a
   32 ObjHeader    c X-Powered-By: PHP/5.2.5
   32 ObjHeader    c Cache-Control: public, max-age=600
   32 ObjHeader    c Last-Modified: Fri, 25 Feb 2011 18:45:29 +0000
   32 ObjHeader    c Expires: Sun, 11 Mar 1984 12:00:00 GMT
   32 ObjHeader    c Vary: Cookie
   32 ObjHeader    c ETag: "1298659529"
   32 ObjHeader    c Content-Type: text/html; charset=utf-8
   32 VCL_call     c deliver deliver
   32 TxProtocol   c HTTP/1.1
   32 TxStatus     c 200
   32 TxResponse   c OK
   32 TxHeader     c Server: Apache/2.2.8 (Unix) PHP/5.2.5 mod_ssl/2.2.8
OpenSSL/0.9.7a
   32 TxHeader     c X-Powered-By: PHP/5.2.5
   32 TxHeader     c Cache-Control: public, max-age=600
   32 TxHeader     c Last-Modified: Fri, 25 Feb 2011 18:45:29 +0000
   32 TxHeader     c Expires: Sun, 11 Mar 1984 12:00:00 GMT
   32 TxHeader     c Vary: Cookie
   32 TxHeader     c ETag: "1298659529"
   32 TxHeader     c Content-Type: text/html; charset=utf-8
   32 TxHeader     c Content-Length: 83552
   32 TxHeader     c Date: Fri, 25 Feb 2011 18:44:45 GMT
   32 TxHeader     c X-Varnish: 367412524
   32 TxHeader     c Age: 0
   32 TxHeader     c Via: 1.1 varnish
   32 TxHeader     c Connection: keep-alive
   32 TxHeader     c X-Cache: MISS
   32 Length       c 83552
   32 ReqEnd       c 367412524 1298659483.560426950 1298659485.242687941
0.000025988 1.473248959 0.209012032


If anyone can give me some insight It would be greatly appreciated
__________________
Kevin Pettijohn
503 927 3585
Operations & IT
The Daily Insight Group
kpettijohn at tarot.com





_______________________________________________
varnish-misc mailing list
varnish-misc at varnish-cache.org
http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

Does the forum page load for the admin user if you point your browser
directly at the backend? When logged in the forum page will do lot of extra
queries to determine which forums need to show new counts e.t.c.

It could be that the page is simply timing out

Stew

Sent from my iPhone
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20110228/342a4569/attachment.html>

From marko at oktober.nl  Mon Feb 28 12:24:38 2011
From: marko at oktober.nl (Marko Kruijer)
Date: Mon, 28 Feb 2011 13:24:38 +0100
Subject: set cookies with ESI includes possible?
Message-ID: <4D6B9406.6000305@oktober.nl>

I'm currently using varnish for a fairly large webshop. My ESI includes 
generate the dynamic content on various pages.

The problem I'm having here is that I need a few cookies to be set for 
every user, this functionality is done by the ESI includes.

However they are not being set if the page is cached. I know that 
Varnish generates the complete page (by fetching the cache and the ESI 
stuff and putting it all together) and thus set cookie headers from the 
ESI are not sent through to the client. Is it possible to still do this 
somehow?

I can also resort to setting cookies by javascript, generated by ESI and 
whatnot, but I'd rather try the 'non-javascript' approach first.

Regards,
Marko




From adi at netstyle.ch  Mon Feb 28 16:37:30 2011
From: adi at netstyle.ch (Adrian Lienhard)
Date: Mon, 28 Feb 2011 17:37:30 +0100
Subject: HTTP/304 response from backend takes 15s to be returned to client
In-Reply-To: <1298471701.2392.31.camel@wido-desktop>
References: <863BA4F8-3476-4219-A313-0DE8767FD860@netstyle.ch>
	<1298471701.2392.31.camel@wido-desktop>
Message-ID: <06A6970D-AAA7-4AEC-9B5B-02649040CA61@netstyle.ch>

Hi Wido,

Thanks for the response. Upgrading to 2.1.5 indeed fixed the problem.

Cheers,
Adrian



On Feb 23, 2011, at 15:35 , Wido den Hollander wrote:

> Hi,
> 
> On Wed, 2011-02-23 at 15:28 +0100, Adrian Lienhard wrote:
>> Hi,
>> 
>> I'm experiencing a problem with passing conditional client requests that are not cached. We use Varnish to cache HTML documents. Static files like CSS are not cached (pass).
>> 
>> The relevant bit of vcl_recv trivially is:
>> 
>> sub vcl_recv {
>> 	...
>> 	if (req.url ~ "\.(css|js|jpg|jpeg|png|gif|mp3|ogg|flv|swf|pdf)$") {
>> 	    return(pass);
>> 	}
>> 	...
>> 
>> If the client sends a conditional request (with If-Modified-Since header) the backend (Apache) responds with a HTTP/304 as expected.
>> 
>> The problem is that in this case it takes Varnish 15 seconds to send the 304 response to the client (this is consistently reproducible and always 15s plus a few milliseconds). If, however, the client request is not a conditional request, everything works as expected.
>> 
>> I've also tested with return(lookup) n vcl_recv, which always misses, but the problem is exactly the same. However if I return(pipe), it works OK.
>> 
>> I post the varnishlog of a failing 304 and of a succeeding 200 below (I've indicated where the 15s delay is). Maybe interesting to know is that I see the request being logged in Apache immediately. Also Apache works just fine without Varnish in front.
>> 
>> Versions:
>> - varnish-2.1.4 SVN on Debian Squeeze
>> - Apache (running on another host): 2.2.9
>> 
> 
> There was a fix regarding this in 2.1.5, I suggest you upgrade to that
> version.
> 
> Wido
> 
>> Any help appreciated.
>> 
>> Cheers,
>> Adrian
>> 
>> BTW: Should I anyway rather do pipe for static resources that are never cached?
> 
> I think pass is fine?
> 
>> 
>> 
>> 
>> Conditional request (FAIL)
>> ==========================
>> HTTP/304 response takes 15s
>> -----------------------
>>   12 SessionOpen  c 212.103.66.219 6147 :80
>>   12 ReqStart     c 212.103.66.219 6147 391596406
>>   12 RxRequest    c GET
>>   12 RxURL        c /_cmsbox_2.2.13_85/layout/default.css
>>   12 RxProtocol   c HTTP/1.1
>>   12 RxHeader     c User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3
>>   12 RxHeader     c Host: adrian.cmsbox.biz
>>   12 RxHeader     c Accept: */*
>>   12 RxHeader     c If-Modified-Since: Wed, 16 Feb 2011 18:05:26 GMT
>>   12 VCL_call     c recv
>>   12 VCL_return   c pass
>>   12 VCL_call     c hash
>>   12 VCL_return   c hash
>>   12 VCL_call     c pass
>>   12 VCL_return   c pass
>>   14 BackendClose - cmsbox
>>   14 BackendOpen  b cmsbox 212.103.66.208 55801 212.103.66.211 80
>>   12 Backend      c 14 cmsbox cmsbox
>>   14 TxRequest    b GET
>>   14 TxURL        b /_cmsbox_2.2.13_85/layout/default.css
>>   14 TxProtocol   b HTTP/1.1
>>   14 TxHeader     b User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3
>>   14 TxHeader     b Host: adrian.cmsbox.biz
>>   14 TxHeader     b Accept: */*
>>   14 TxHeader     b If-Modified-Since: Wed, 16 Feb 2011 18:05:26 GMT
>>   14 TxHeader     b X-Forwarded-For: 212.103.66.219
>>   14 TxHeader     b X-Varnish: 391596406
>> =============== Waits 15s here / backend loggs request immediately  ===============
>>   14 RxProtocol   b HTTP/1.1
>>   14 RxStatus     b 304
>>   14 RxResponse   b Not Modified
>>   14 RxHeader     b Date: Wed, 23 Feb 2011 13:55:00 GMT
>>   14 RxHeader     b Server: Apache/2.2.9 (Debian)
>>   14 RxHeader     b ETag: "c46450-8120-49c6a201bc580"
>>   14 RxHeader     b Expires: Thu, 23 Feb 2012 13:55:00 GMT
>>   14 RxHeader     b Cache-Control: max-age=31536000
>>   14 RxHeader     b Vary: Accept-Encoding
>>   12 TTL          c 391596406 RFC 31536000 1298469297 0 0 31536000 0
>>   12 VCL_call     c fetch
>>   12 VCL_return   c pass
>>   12 ObjProtocol  c HTTP/1.1
>>   12 ObjStatus    c 304
>>   12 ObjResponse  c Not Modified
>>   12 ObjHeader    c Date: Wed, 23 Feb 2011 13:55:00 GMT
>>   12 ObjHeader    c Server: Apache/2.2.9 (Debian)
>>   12 ObjHeader    c ETag: "c46450-8120-49c6a201bc580"
>>   12 ObjHeader    c Expires: Thu, 23 Feb 2012 13:55:00 GMT
>>   12 ObjHeader    c Vary: Accept-Encoding
>>   14 Length       b 0
>>   14 BackendClose b cmsbox
>>   12 VCL_call     c deliver
>>   12 VCL_return   c deliver
>>   12 TxProtocol   c HTTP/1.1
>>   12 TxStatus     c 304
>>   12 TxResponse   c Not Modified
>>   12 TxHeader     c Server: Apache/2.2.9 (Debian)
>>   12 TxHeader     c Expires: Thu, 23 Feb 2012 13:55:00 GMT
>>   12 TxHeader     c Vary: Accept-Encoding
>>   12 TxHeader     c Content-Length: 0
>>   12 TxHeader     c Date: Wed, 23 Feb 2011 13:55:12 GMT
>>   12 TxHeader     c X-Varnish: 391596406
>>   12 TxHeader     c Age: 15
>>   12 TxHeader     c Via: 1.1 varnish
>>   12 TxHeader     c Connection: keep-alive
>>   12 TxHeader     c X-Cache-Hits: 0
>>   12 Length       c 0
>>   12 ReqEnd       c 391596406 1298469297.322397470 1298469312.322380066 0.000116587 14.999900818 0.000081778
>>   12 SessionClose c EOF
>>   12 StatSess     c 212.103.66.219 6147 15 1 1 0 1 1 269 0
>> 
>> 
>> 
>> Non-conditional request (OK)
>> ============================
>> HTTP/200 response within 100ms or less
>> 
>> ------------------------
>>   12 SessionOpen  c 212.103.66.219 23833 :80
>>   12 ReqStart     c 212.103.66.219 23833 2103978089
>>   12 RxRequest    c HEAD
>>   12 RxURL        c /_cmsbox_2.2.13_85/layout/default.css
>>   12 RxProtocol   c HTTP/1.1
>>   12 RxHeader     c User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3
>>   12 RxHeader     c Host: adrian.cmsbox.biz
>>   12 RxHeader     c Accept: */*
>>   12 VCL_call     c recv
>>   12 VCL_return   c lookup
>>   12 VCL_call     c hash
>>   12 VCL_return   c hash
>>   12 HitPass      c 2103978045
>>   12 VCL_call     c pass
>>   12 VCL_return   c pass
>>   15 BackendClose b cmsbox
>>   15 BackendOpen  b cmsbox 212.103.66.208 50035 212.103.66.211 80
>>   12 Backend      c 15 cmsbox cmsbox
>>   15 TxRequest    b HEAD
>>   15 TxURL        b /_cmsbox_2.2.13_85/layout/default.css
>>   15 TxProtocol   b HTTP/1.1
>>   15 TxHeader     b User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3
>>   15 TxHeader     b Host: adrian.cmsbox.biz
>>   15 TxHeader     b Accept: */*
>>   15 TxHeader     b X-Forwarded-For: 212.103.66.219
>>   15 TxHeader     b X-Varnish: 2103978089
>>   15 RxProtocol   b HTTP/1.1
>>   15 RxStatus     b 200
>>   15 RxResponse   b OK
>>   15 RxHeader     b Date: Wed, 23 Feb 2011 09:39:32 GMT
>>   15 RxHeader     b Server: Apache/2.2.9 (Debian)
>>   15 RxHeader     b Last-Modified: Wed, 16 Feb 2011 18:05:26 GMT
>>   15 RxHeader     b ETag: "c46450-8120-49c6a201bc580"
>>   15 RxHeader     b Accept-Ranges: bytes
>>   15 RxHeader     b Content-Length: 33056
>>   15 RxHeader     b Cache-Control: max-age=31536000
>>   15 RxHeader     b Expires: Thu, 23 Feb 2012 09:39:32 GMT
>>   15 RxHeader     b Vary: Accept-Encoding
>>   15 RxHeader     b Content-Type: text/css
>>   12 TTL          c 2103978089 RFC 31536000 1298453970 0 0 31536000 0
>>   12 VCL_call     c fetch
>>   12 VCL_return   c pass
>>   12 ObjProtocol  c HTTP/1.1
>>   12 ObjStatus    c 200
>>   12 ObjResponse  c OK
>>   12 ObjHeader    c Date: Wed, 23 Feb 2011 09:39:32 GMT
>>   12 ObjHeader    c Server: Apache/2.2.9 (Debian)
>>   12 ObjHeader    c Last-Modified: Wed, 16 Feb 2011 18:05:26 GMT
>>   12 ObjHeader    c ETag: "c46450-8120-49c6a201bc580"
>>   12 ObjHeader    c Content-Length: 33056
>>   12 ObjHeader    c Expires: Thu, 23 Feb 2012 09:39:32 GMT
>>   12 ObjHeader    c Vary: Accept-Encoding
>>   12 ObjHeader    c Content-Type: text/css
>>   15 Length       b 0
>>   15 BackendReuse b cmsbox
>>   12 VCL_call     c deliver
>>   12 VCL_return   c deliver
>>   12 TxProtocol   c HTTP/1.1
>>   12 TxStatus     c 200
>>   12 TxResponse   c OK
>>   12 TxHeader     c Server: Apache/2.2.9 (Debian)
>>   12 TxHeader     c Last-Modified: Wed, 16 Feb 2011 18:05:26 GMT
>>   12 TxHeader     c Content-Length: 33056
>>   12 TxHeader     c Expires: Thu, 23 Feb 2012 09:39:32 GMT
>>   12 TxHeader     c Vary: Accept-Encoding
>>   12 TxHeader     c Content-Type: text/css
>>   12 TxHeader     c Date: Wed, 23 Feb 2011 09:39:30 GMT
>>   12 TxHeader     c X-Varnish: 2103978089
>>   12 TxHeader     c Age: 0
>>   12 TxHeader     c Via: 1.1 varnish
>>   12 TxHeader     c Connection: keep-alive
>>   12 TxHeader     c X-Cache-Hits: 0
>>   12 Length       c 0
>>   12 ReqEnd       c 2103978089 1298453970.929079056 1298453970.930070400 0.000118494 0.000920773 0.000070572
>>   12 Debug        c "herding"
>>   12 SessionClose c no request
>>   12 StatSess     c 212.103.66.219 23833 0 1 1 0 1 1 333 0
>>    0 ExpKill      - 2103978084 -10
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at varnish-cache.org
>> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>