From behzadaltaf at gmail.com  Thu Jan  1 06:14:30 2015
From: behzadaltaf at gmail.com (Behzad Altaf)
Date: Thu, 1 Jan 2015 11:44:30 +0530
Subject: Varnish Clustering
In-Reply-To: <3EE228D3-4022-4F4D-B642-7F614015F924@ultra-secure.de>
References: <CAJpkvFgj5wD37ptMwFtSq+TYfqFGw9vPdxvx5Nd6naDBi6-qAw@mail.gmail.com>
 <3EE228D3-4022-4F4D-B642-7F614015F924@ultra-secure.de>
Message-ID: <CAJpkvFiyJqqstzCs4_rjQ3wmX-8kRoHnvEuCM6kxh-LRPqe8og@mail.gmail.com>

Thanks so much for your reply. 

On Wed, Dec 31, 2014 at 10:28 PM, Rainer Duffner <rainer at ultra-secure.de>
wrote:

>
> Am 31.12.2014 um 08:48 schrieb Behzad Altaf <behzadaltaf at gmail.com>:
>
> Hi All,
>
> Would anyone please help me with the exact steps that are needed for
> putting varnish boxes into clustering and self replication mode.
>
> I have two boxes on which Varnish is running with a backend webserver.
>
> I looked at the following but it does not have the steps
>
>
> https://www.varnish-software.com/blog/introducing-varnish-high-availability-2
>
>
>
>
> This is for the commercial, subscription-only varnish plus (and then you
> probably need the Gold Subscription).
>
> OSS varnish does not have a cache-replication feature.
>
> Can't you just load-balance between the two varnish-instances?
> Both will warm-up slower, but at least you'd have two warm caches.
>
> Of course, your application should then do cache-invalidation on both
> instances - which is usually the trickier part and probably one reason why
> people fork-over the money for the subscription...
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150101/ab468aaf/attachment.html>

From mailman at haukebruno.de  Fri Jan  2 14:24:26 2015
From: mailman at haukebruno.de (Hauke)
Date: Fri, 02 Jan 2015 15:24:26 +0100
Subject: Varnish Clustering
In-Reply-To: <CAJpkvFiyJqqstzCs4_rjQ3wmX-8kRoHnvEuCM6kxh-LRPqe8og@mail.gmail.com>
References: <CAJpkvFgj5wD37ptMwFtSq+TYfqFGw9vPdxvx5Nd6naDBi6-qAw@mail.gmail.com>
 <3EE228D3-4022-4F4D-B642-7F614015F924@ultra-secure.de>
 <CAJpkvFiyJqqstzCs4_rjQ3wmX-8kRoHnvEuCM6kxh-LRPqe8og@mail.gmail.com>
Message-ID: <2772861.esxOgumPm0@notenbuch.haukebruno.de>

Hey,

I personally use varnish in a simple pacemaker/corosync setting for high 
availability.

cheers,
hauke

-- 
Best regards,
Hauke
On Thursday, January 01, 2015 11:44:30 AM Behzad Altaf wrote:
> Thanks so much for your reply.
> 
> On Wed, Dec 31, 2014 at 10:28 PM, Rainer Duffner <rainer at ultra-secure.de>
> 
> wrote:
> > Am 31.12.2014 um 08:48 schrieb Behzad Altaf <behzadaltaf at gmail.com>:
> > 
> > Hi All,
> > 
> > Would anyone please help me with the exact steps that are needed for
> > putting varnish boxes into clustering and self replication mode.
> > 
> > I have two boxes on which Varnish is running with a backend webserver.
> > 
> > I looked at the following but it does not have the steps
> > 
> > 
> > https://www.varnish-software.com/blog/introducing-varnish-high-availabilit
> > y-2
> > 
> > 
> > 
> > 
> > This is for the commercial, subscription-only varnish plus (and then you
> > probably need the Gold Subscription).
> > 
> > OSS varnish does not have a cache-replication feature.
> > 
> > Can't you just load-balance between the two varnish-instances?
> > Both will warm-up slower, but at least you'd have two warm caches.
> > 
> > Of course, your application should then do cache-invalidation on both
> > instances - which is usually the trickier part and probably one reason why
> > people fork-over the money for the subscription...




From phk at phk.freebsd.dk  Mon Jan  5 09:18:16 2015
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Mon, 05 Jan 2015 09:18:16 +0000
Subject: hit-for-pass ignores Vary header
In-Reply-To: <CAKsTx7AnGoGURk6hQR9om4K6mAQohxQRi+HX0Ya-LqAX0cC2fg@mail.gmail.com>
References: <CAKsTx7AnGoGURk6hQR9om4K6mAQohxQRi+HX0Ya-LqAX0cC2fg@mail.gmail.com>
Message-ID: <11602.1420449496@critter.freebsd.dk>

--------
In message <CAKsTx7AnGoGURk6hQR9om4K6mAQohxQRi+HX0Ya-LqAX0cC2fg at mail.gmail.com>
, Jonathan Matthews writes:

Hi Jonathan,

Sorry about the late reply.  (And happy new year while we're at it :-)

>What I believe is happening is that the resulting hit-for-pass object
>that gets generated and cached seems to ignore the Vary header's
>presence. All subsequent requests for that Host & URI combination get
>hit-for-pass'ed, regardless of the previously valid object sitting in
>the cache for the Host & URI & Vary-header-pointer combination.

I tried this scenario out on -trunk using the varnishtest below and
it does what I expect it to.

We did change something in this particular space recently-ish, so
4.0 may act differently in this corner-case.

Poul-Henning

-------------------------
varnishtest "vary hit-for-pass"

server s1 {
	fatal
	rxreq
	expect req.http.foo == "1"
	txresp -hdr "Vary: foo" -bodylen 1

	rxreq
	expect req.http.foo == "2"
	txresp -status 401 -hdr "Vary: foo" -bodylen 2

	rxreq
	expect req.http.foo == "2"
	txresp -status 402 -hdr "Vary: foo" -bodylen 3
} -start

varnish v1 -vcl+backend {
	sub vcl_hash {
		hash_data(req.http.foo);
	}
} -start

client c1 {
	txreq -hdr "foo: 1"
	rxresp
	expect resp.status == 200
	expect resp.bodylen == 1
	delay .1

	txreq -hdr "foo: 2"
	rxresp
	expect resp.status == 401
	expect resp.bodylen == 2
	delay .1

	txreq -hdr "foo: 2"
	rxresp
	expect resp.status == 402
	expect resp.bodylen == 3
	delay .1

	txreq -hdr "foo: 1"
	rxresp
	expect resp.status == 200
	expect resp.bodylen == 1
	delay .1

} -run

-------------------------

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.



From dridi.boukelmoune at zenika.com  Mon Jan  5 11:09:15 2015
From: dridi.boukelmoune at zenika.com (Dridi Boukelmoune)
Date: Mon, 5 Jan 2015 12:09:15 +0100
Subject: hit-for-pass ignores Vary header
In-Reply-To: <11602.1420449496@critter.freebsd.dk>
References: <CAKsTx7AnGoGURk6hQR9om4K6mAQohxQRi+HX0Ya-LqAX0cC2fg@mail.gmail.com>
 <11602.1420449496@critter.freebsd.dk>
Message-ID: <CABtDKm4LzCQCdBD0V-P47oDa75LWn7oooxdr-52NuL_tiJrDAw@mail.gmail.com>

On Mon, Jan 5, 2015 at 10:18 AM, Poul-Henning Kamp <phk at phk.freebsd.dk> wrote:
>>What I believe is happening is that the resulting hit-for-pass object
>>that gets generated and cached seems to ignore the Vary header's
>>presence. All subsequent requests for that Host & URI combination get
>>hit-for-pass'ed, regardless of the previously valid object sitting in
>>the cache for the Host & URI & Vary-header-pointer combination.

Hi Jonathan,

Since variants are different representations of a *same* resource,
invalidating the resource *could* invalidate all its representations.

AFAICT, this is undefined behavior.

> I tried this scenario out on -trunk using the varnishtest below and
> it does what I expect it to.

Hi PHK,

The scenario works fine, but it doesn't match Jonathan's description
of his problem. Adding the header to the cache key is his workaround,
but your test passes even with just the built-in VCL on both the
current trunk and the 4.0.2 tag.

Happy New Year,
Dridi



From phk at phk.freebsd.dk  Mon Jan  5 11:43:46 2015
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Mon, 05 Jan 2015 11:43:46 +0000
Subject: hit-for-pass ignores Vary header
In-Reply-To: <CABtDKm4LzCQCdBD0V-P47oDa75LWn7oooxdr-52NuL_tiJrDAw@mail.gmail.com>
References: <CAKsTx7AnGoGURk6hQR9om4K6mAQohxQRi+HX0Ya-LqAX0cC2fg@mail.gmail.com>
 <11602.1420449496@critter.freebsd.dk>
 <CABtDKm4LzCQCdBD0V-P47oDa75LWn7oooxdr-52NuL_tiJrDAw@mail.gmail.com>
Message-ID: <12227.1420458226@critter.freebsd.dk>

--------
In message <CABtDKm4LzCQCdBD0V-P47oDa75LWn7oooxdr-52NuL_tiJrDAw at mail.gmail.com>
, Dridi Boukelmoune writes:

>The scenario works fine, but it doesn't match Jonathan's description
>of his problem. Adding the header to the cache key is his workaround,
>but your test passes even with just the built-in VCL on both the
>current trunk and the 4.0.2 tag.

I missed the bit about vcl_hash being a workaround.

If somebody can hack the vtc to show the problem that would be helpful.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.



From dridi.boukelmoune at zenika.com  Mon Jan  5 16:43:51 2015
From: dridi.boukelmoune at zenika.com (Dridi Boukelmoune)
Date: Mon, 5 Jan 2015 17:43:51 +0100
Subject: hit-for-pass ignores Vary header
In-Reply-To: <12227.1420458226@critter.freebsd.dk>
References: <CAKsTx7AnGoGURk6hQR9om4K6mAQohxQRi+HX0Ya-LqAX0cC2fg@mail.gmail.com>
 <11602.1420449496@critter.freebsd.dk>
 <CABtDKm4LzCQCdBD0V-P47oDa75LWn7oooxdr-52NuL_tiJrDAw@mail.gmail.com>
 <12227.1420458226@critter.freebsd.dk>
Message-ID: <CABtDKm7oamAcmQBhvs9qB630fdHBNGVgV8s8os+R=k5HaZrdGA@mail.gmail.com>

On Mon, Jan 5, 2015 at 12:43 PM, Poul-Henning Kamp <phk at phk.freebsd.dk> wrote:
> If somebody can hack the vtc to show the problem that would be helpful.

Jonathan is actually running Varnish 3.0.5, and again, your test case
runs fine (only tested with the built-in VCL) for both varnish 3.0.5, 3.0.6,
and the current 3.0 branch.

Dridi



From behzadaltaf at gmail.com  Tue Jan  6 07:03:28 2015
From: behzadaltaf at gmail.com (Behzad Altaf)
Date: Tue, 6 Jan 2015 12:33:28 +0530
Subject: Varnish GET/POST cache Invalidation
Message-ID: <CAJpkvFhFuMjsgS6v=9gDBPMm7oc_mt4Oyob3f_ubmA6U-Bjz4A@mail.gmail.com>

Hi All,

I am trying to understand varnish Automatic Cache invalidation strategy
using HTTP VERBS. i.e.

I do a GET on a URL - VARNISH fetches from backend.


I do a GET on same URL - VARNISH delivers from CACHE

I do a POST on the same URL - VARNISH passes to back end

I do a GET on the same URL - VARNISH delivers from CACHE (why?)
(Expected behaviour Varnish invalidates with last POST and fetches from
backend)

Would anybody please help me understand why VARNISH is NOT INVALIDATING the
cache on the POST of the same URL.

What am I missing?

Thanks and Regards,
BA
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150106/aa9fb89b/attachment.html>

From thomas.lecomte at virtual-expo.com  Tue Jan  6 07:14:09 2015
From: thomas.lecomte at virtual-expo.com (Thomas Lecomte)
Date: Tue, 6 Jan 2015 08:14:09 +0100
Subject: Varnish GET/POST cache Invalidation
In-Reply-To: <CAJpkvFhFuMjsgS6v=9gDBPMm7oc_mt4Oyob3f_ubmA6U-Bjz4A@mail.gmail.com>
References: <CAJpkvFhFuMjsgS6v=9gDBPMm7oc_mt4Oyob3f_ubmA6U-Bjz4A@mail.gmail.com>
Message-ID: <CAJban9ARh03yRGKOFdNQ8QGHEM=fjzG0+LoA_+WtXJakuX4ExA@mail.gmail.com>

On Tue, Jan 6, 2015 at 8:03 AM, Behzad Altaf <behzadaltaf at gmail.com> wrote:
>
> Would anybody please help me understand why VARNISH is NOT INVALIDATING the
> cache on the POST of the same URL.
>
> What am I missing?

Hello,

Why would be varnish supposed to invalidate content when you use the
POST method?

The default VCL doesn't do that. The behaviour you are experiencing is
normal. Could you post your VCL?

Regards,

-- 
Thomas Lecomte / Sysadmin
+33 4 86 13 48 65 / Virtual Expo / Marseille, France



From thomas.lecomte at virtual-expo.com  Tue Jan  6 07:36:50 2015
From: thomas.lecomte at virtual-expo.com (Thomas Lecomte)
Date: Tue, 6 Jan 2015 08:36:50 +0100
Subject: Varnish GET/POST cache Invalidation
In-Reply-To: <CAJpkvFjFyM+WQQt=pP29_-i+YKTyxN9_Q8cyCCHRREqPfad4iQ@mail.gmail.com>
References: <CAJpkvFhFuMjsgS6v=9gDBPMm7oc_mt4Oyob3f_ubmA6U-Bjz4A@mail.gmail.com>
 <CAJban9ARh03yRGKOFdNQ8QGHEM=fjzG0+LoA_+WtXJakuX4ExA@mail.gmail.com>
 <CAJpkvFjFyM+WQQt=pP29_-i+YKTyxN9_Q8cyCCHRREqPfad4iQ@mail.gmail.com>
Message-ID: <20150106073650.GA4608@wks140.directindustry.com>

On Tue, Jan 06, 2015 at 12:57:06PM +0530, Behzad Altaf wrote:
> Thank you so much for your reply Thomas.
> 
> I have not made any changes in the default VCL.
> 
> Is the expectation wrong?
> 
> GET on URL  --> CACHES
> POST/PUT on same URL --> Invalidates

Yes, your expectation is wrong. The default VCL will pass POST/PUT
requests to the backend, but it won't invalidate the cache.

To invalidate the cache using an HTTP method, you need to implement that
in the VCL.

Have a look at the following documentation:

https://www.varnish-cache.org/docs/3.0/tutorial/purging.html#http-purges

Regards,

-- 
Thomas Lecomte / Sysadmin
Virtual Expo  / Marseille, France



From hubert at htop.pl  Tue Jan  6 18:24:43 2015
From: hubert at htop.pl (hubert at htop.pl)
Date: Tue, 06 Jan 2015 18:24:43 +0000
Subject: IP input and output
Message-ID: <7c219d2aace88f9aac78682db7db12ec@htop.pl>

Welcome. My varnish listen on many IPs. For example 127.0.0.1 127.0.0.2 and 127.0.0.3 on port 80. How to configure to relay/forward request to backend in this format:
 INPUT IP TO VARNISH | IP OF BACKEND
 127.0.0.1???????????????????????? 127.0.0.1
 127.0.0.2 ??????????????????????? 127.0.0.2
 127.0.0.3 ??????????????????????? 127.0.0.3
 Of course, the backend listens on other ports.

 --
 Thank you.
 Hubert Nodzak-Pluta
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150106/44ec77b3/attachment.html>

From yamakasi.014 at gmail.com  Wed Jan  7 12:55:17 2015
From: yamakasi.014 at gmail.com (Matt .)
Date: Wed, 7 Jan 2015 13:55:17 +0100
Subject: Rewrite foo.com to bla.com/$1 ?
Message-ID: <CAPNQp07sawk3zrcd4KZt0REiSNUU-njaqS1Sbpvy-CGaxpvorg@mail.gmail.com>

Hi All,

In the past I tested some scripts on on varnish for rewriting a domainname.

My idea would be foo.com/* to bla.com/$1

Can I accomplish this with just something like:

if (req.http.host ~ "foo.com") {
set req.http.host = "bla.com/$1";
}

I'm trying to find examples but I only find much of the same like


foo.com/bla -> foo.com/bleh


Cheers,

Matt



From perbu at varnish-software.com  Wed Jan  7 13:03:36 2015
From: perbu at varnish-software.com (Per Buer)
Date: Wed, 7 Jan 2015 14:03:36 +0100
Subject: Rewrite foo.com to bla.com/$1 ?
In-Reply-To: <CAPNQp07sawk3zrcd4KZt0REiSNUU-njaqS1Sbpvy-CGaxpvorg@mail.gmail.com>
References: <CAPNQp07sawk3zrcd4KZt0REiSNUU-njaqS1Sbpvy-CGaxpvorg@mail.gmail.com>
Message-ID: <CAOXZevA3e71VwMgPeh5L4JwwQ_xRjnmC4rdGaNQcFzj0Gxd9Sg@mail.gmail.com>

On Wed, Jan 7, 2015 at 1:55 PM, Matt . <yamakasi.014 at gmail.com> wrote:

> Hi All,
>
> In the past I tested some scripts on on varnish for rewriting a domainname.
>
> My idea would be foo.com/* to bla.com/$1
>
> Can I accomplish this with just something like:
>
> if (req.http.host ~ "foo.com") {
> set req.http.host = "bla.com/$1";
> }
>

Unless I missed something essential you could just do

if (req.http.host ~ "foo.com") {
      set req.http.host = "bar.com";
}

req.url will stay untouched. Remember that the full URL consists of
req.http.host and req.url.

-- 
*Per Buer*
CTO | Varnish Software AS
Cell: +47 95839117
We Make Websites Fly!
www.varnish-software.com
[image: Register now]
<http://info.varnish-software.com/varnish-summits-autumn-2014-registration>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150107/aa0e0e95/attachment.html>

From yamakasi.014 at gmail.com  Wed Jan  7 14:04:15 2015
From: yamakasi.014 at gmail.com (Matt .)
Date: Wed, 7 Jan 2015 15:04:15 +0100
Subject: Rewrite foo.com to bla.com/$1 ?
In-Reply-To: <CAOXZevA3e71VwMgPeh5L4JwwQ_xRjnmC4rdGaNQcFzj0Gxd9Sg@mail.gmail.com>
References: <CAPNQp07sawk3zrcd4KZt0REiSNUU-njaqS1Sbpvy-CGaxpvorg@mail.gmail.com>
 <CAOXZevA3e71VwMgPeh5L4JwwQ_xRjnmC4rdGaNQcFzj0Gxd9Sg@mail.gmail.com>
Message-ID: <CAPNQp04w-VEvftme1AZnVPioMv8gtA8XegVeA5YswoDY81FeGA@mail.gmail.com>

Hi,

Thanks, I was just not sure anymore, too long ago.

But you say that the full URL consists of req.http.host and req.url, does
that really matter in this case as req.url stays untouched ?

2015-01-07 14:03 GMT+01:00 Per Buer <perbu at varnish-software.com>:

> On Wed, Jan 7, 2015 at 1:55 PM, Matt . <yamakasi.014 at gmail.com> wrote:
>
>> Hi All,
>>
>> In the past I tested some scripts on on varnish for rewriting a
>> domainname.
>>
>> My idea would be foo.com/* to bla.com/$1
>>
>> Can I accomplish this with just something like:
>>
>> if (req.http.host ~ "foo.com") {
>> set req.http.host = "bla.com/$1";
>> }
>>
>
> Unless I missed something essential you could just do
>
> if (req.http.host ~ "foo.com") {
>       set req.http.host = "bar.com";
> }
>
> req.url will stay untouched. Remember that the full URL consists of
> req.http.host and req.url.
>
> --
> *Per Buer*
> CTO | Varnish Software AS
> Cell: +47 95839117
> We Make Websites Fly!
> www.varnish-software.com
> [image: Register now]
> <http://info.varnish-software.com/varnish-summits-autumn-2014-registration>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150107/de519161/attachment.html>

From perbu at varnish-software.com  Wed Jan  7 15:01:17 2015
From: perbu at varnish-software.com (Per Buer)
Date: Wed, 7 Jan 2015 16:01:17 +0100
Subject: Rewrite foo.com to bla.com/$1 ?
In-Reply-To: <CAPNQp04w-VEvftme1AZnVPioMv8gtA8XegVeA5YswoDY81FeGA@mail.gmail.com>
References: <CAPNQp07sawk3zrcd4KZt0REiSNUU-njaqS1Sbpvy-CGaxpvorg@mail.gmail.com>
 <CAOXZevA3e71VwMgPeh5L4JwwQ_xRjnmC4rdGaNQcFzj0Gxd9Sg@mail.gmail.com>
 <CAPNQp04w-VEvftme1AZnVPioMv8gtA8XegVeA5YswoDY81FeGA@mail.gmail.com>
Message-ID: <CAOXZevAPt-gQMxokTf-Ya2MV_YnBOYe-Hy87pzNZd6ZiyV--Vg@mail.gmail.com>

On Wed, Jan 7, 2015 at 3:04 PM, Matt . <yamakasi.014 at gmail.com> wrote:

> Hi,
>
> Thanks, I was just not sure anymore, too long ago.
>
> But you say that the full URL consists of req.http.host and req.url, does
> that really matter in this case as req.url stays untouched ?
>

Well, I was just telling you this because you tried to stick parts of the
url-part into the host header. So I think you might need some enlightenment
on that part. :-)

-- 
*Per Buer*
CTO | Varnish Software AS
Cell: +47 95839117
We Make Websites Fly!
www.varnish-software.com
[image: Register now]
<http://info.varnish-software.com/varnish-summits-autumn-2014-registration>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150107/22fd08c1/attachment.html>

From yamakasi.014 at gmail.com  Wed Jan  7 15:19:22 2015
From: yamakasi.014 at gmail.com (Matt .)
Date: Wed, 7 Jan 2015 16:19:22 +0100
Subject: Rewrite foo.com to bla.com/$1 ?
In-Reply-To: <CAOXZevAPt-gQMxokTf-Ya2MV_YnBOYe-Hy87pzNZd6ZiyV--Vg@mail.gmail.com>
References: <CAPNQp07sawk3zrcd4KZt0REiSNUU-njaqS1Sbpvy-CGaxpvorg@mail.gmail.com>
 <CAOXZevA3e71VwMgPeh5L4JwwQ_xRjnmC4rdGaNQcFzj0Gxd9Sg@mail.gmail.com>
 <CAPNQp04w-VEvftme1AZnVPioMv8gtA8XegVeA5YswoDY81FeGA@mail.gmail.com>
 <CAOXZevAPt-gQMxokTf-Ya2MV_YnBOYe-Hy87pzNZd6ZiyV--Vg@mail.gmail.com>
Message-ID: <CAPNQp07_u2ey7goDNfxaRK0Sq3qhbJ-YjEEx2t-XVXha9TA01A@mail.gmail.com>

Yes, very true!

Thank you.

2015-01-07 16:01 GMT+01:00 Per Buer <perbu at varnish-software.com>:

> On Wed, Jan 7, 2015 at 3:04 PM, Matt . <yamakasi.014 at gmail.com> wrote:
>
>> Hi,
>>
>> Thanks, I was just not sure anymore, too long ago.
>>
>> But you say that the full URL consists of req.http.host and req.url, does
>> that really matter in this case as req.url stays untouched ?
>>
>
> Well, I was just telling you this because you tried to stick parts of the
> url-part into the host header. So I think you might need some enlightenment
> on that part. :-)
>
> --
> *Per Buer*
> CTO | Varnish Software AS
> Cell: +47 95839117
> We Make Websites Fly!
> www.varnish-software.com
> [image: Register now]
> <http://info.varnish-software.com/varnish-summits-autumn-2014-registration>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150107/2f446455/attachment.html>

From hugo.cisneiros at gmail.com  Wed Jan  7 16:37:29 2015
From: hugo.cisneiros at gmail.com (Hugo Cisneiros (Eitch))
Date: Wed, 7 Jan 2015 14:37:29 -0200
Subject: Varnish 4 Request Flow
Message-ID: <CA+KACLkV=yg0QdZDWc4xmQ+7ewPc481o=8MEO-P+FuS-2uboAQ@mail.gmail.com>

Hi!

I'm beginning to use Varnish 4 a lot, and it's working great. I used to
print the Varnish 3 Request Flow into my desk for reference when writing
lots and lots of VCL code. Is there an "official" request flow for Varnish4?

The request flow for Varnish 3 that I used is in the Varnish Book:

https://www.varnish-software.com/static/book/VCL_Basics.html

I also found a "non-official" request flow for Varnish 4:

http://www.emanuelis.eu/2014/09/25/varnish-4-vcl-request-flow/

Is it right? Or we have an "official" one?

Thanks!

-- 
[]'s
Hugo
www.devin.com.br
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150107/9016f449/attachment.html>

From apj at mutt.dk  Wed Jan  7 17:30:04 2015
From: apj at mutt.dk (Andreas Plesner Jacobsen)
Date: Wed, 7 Jan 2015 18:30:04 +0100
Subject: Varnish 4 Request Flow
In-Reply-To: <CA+KACLkV=yg0QdZDWc4xmQ+7ewPc481o=8MEO-P+FuS-2uboAQ@mail.gmail.com>
References: <CA+KACLkV=yg0QdZDWc4xmQ+7ewPc481o=8MEO-P+FuS-2uboAQ@mail.gmail.com>
Message-ID: <20150107173004.GH19870@nerd.dk>

On Wed, Jan 07, 2015 at 02:37:29PM -0200, Hugo Cisneiros (Eitch) wrote:
> 
> I'm beginning to use Varnish 4 a lot, and it's working great. I used to
> print the Varnish 3 Request Flow into my desk for reference when writing
> lots and lots of VCL code. Is there an "official" request flow for Varnish4?

Nils added a graph to the docs recently: https://www.varnish-cache.org/trac/changeset/a48147aae7157fbc2d7de19b9e541ffa18fc9fa3

https://www.varnish-cache.org/docs/trunk/reference/states.html

-- 
Andreas



From hugo.cisneiros at gmail.com  Wed Jan  7 18:23:09 2015
From: hugo.cisneiros at gmail.com (Hugo Cisneiros (Eitch))
Date: Wed, 7 Jan 2015 16:23:09 -0200
Subject: Varnish 4 Request Flow
In-Reply-To: <20150107173004.GH19870@nerd.dk>
References: <CA+KACLkV=yg0QdZDWc4xmQ+7ewPc481o=8MEO-P+FuS-2uboAQ@mail.gmail.com>
 <20150107173004.GH19870@nerd.dk>
Message-ID: <CA+KACLmBJRggG_4gnkGK-A0Agz1o7dW8fEyHPcvzSG_7PigOfA@mail.gmail.com>

On Wed, Jan 7, 2015 at 3:30 PM, Andreas Plesner Jacobsen <apj at mutt.dk> wrote:
> > I'm beginning to use Varnish 4 a lot, and it's working great. I used to
> > print the Varnish 3 Request Flow into my desk for reference when writing
> > lots and lots of VCL code. Is there an "official" request flow for Varnish4?
>
> Nils added a graph to the docs recently: https://www.varnish-cache.org/trac/changeset/a48147aae7157fbc2d7de19b9e541ffa18fc9fa3
>
> https://www.varnish-cache.org/docs/trunk/reference/states.html

That's exactly what I was looking for! Thanks a lot :-)

-- 
[]'s
Hugo
www.devin.com.br



From dridi.boukelmoune at zenika.com  Wed Jan  7 21:14:04 2015
From: dridi.boukelmoune at zenika.com (Dridi Boukelmoune)
Date: Wed, 7 Jan 2015 22:14:04 +0100
Subject: IP input and output
In-Reply-To: <7c219d2aace88f9aac78682db7db12ec@htop.pl>
References: <7c219d2aace88f9aac78682db7db12ec@htop.pl>
Message-ID: <CABtDKm7ho3KbET2GrWXq8RfqhK7N1DrhCORb0dn7yTgO=76yOg@mail.gmail.com>

Hi,

There is no simple way to do what you want, but it is possible using
the `server.ip` variable. You can then pick backends according to this
variable but you cannot automatically map a backend to a listen
address. Have a look at the manual, vcl(7).

You might also want to be careful with your hashes if you go down that road.

Please find attached a working test case to illustrate how you can
pick your backend, you can run it with varnishtest(1).

Dridi

On Tue, Jan 6, 2015 at 7:24 PM,  <hubert at htop.pl> wrote:
> Welcome. My varnish listen on many IPs. For example 127.0.0.1 127.0.0.2 and
> 127.0.0.3 on port 80. How to configure to relay/forward request to backend
> in this format:
> INPUT IP TO VARNISH | IP OF BACKEND
> 127.0.0.1                         127.0.0.1
> 127.0.0.2                         127.0.0.2
> 127.0.0.3                         127.0.0.3
> Of course, the backend listens on other ports.
>
> --
> Thank you.
> Hubert Nodzak-Pluta
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
-------------- next part --------------
A non-text attachment was scrubbed...
Name: server_ip.vtc
Type: application/octet-stream
Size: 597 bytes
Desc: not available
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150107/de2a8f1b/attachment.obj>

From gkaragiannidis at dolnet.gr  Sun Jan 11 15:32:40 2015
From: gkaragiannidis at dolnet.gr (=?utf-8?B?zpPOuc6szr3Ovc63z4IgzprOsc+BzrHOs865zrHOvc69zq/OtM63z4I=?=)
Date: Sun, 11 Jan 2015 15:32:40 +0000
Subject: Source code for 3.0.6 missing from Debian repo
Message-ID: <DB3PR02MB026A32099A863598902B1E9DF420@DB3PR02MB026.eurprd02.prod.outlook.com>

Hi, 

I can't seem to download the source for Varnish 3.06 using the Wheezy APT repo. 
I need to fetch the source in order to compile libvmod-shield against it.

My sources.list entry: 

deb http://repo.varnish-cache.org/debian/ wheezy varnish-3.0 
deb-src http://repo.varnish-cache.org/debian/ wheezy varnish-3.0 

My current install of 3.0.6:


Any ideas?

Regards
Yiannis

From andrew.langhorn at digital.cabinet-office.gov.uk  Sun Jan 11 17:13:29 2015
From: andrew.langhorn at digital.cabinet-office.gov.uk (Andrew Langhorn)
Date: Sun, 11 Jan 2015 17:13:29 +0000
Subject: Source code for 3.0.6 missing from Debian repo
In-Reply-To: <DB3PR02MB026A32099A863598902B1E9DF420@DB3PR02MB026.eurprd02.prod.outlook.com>
References: <DB3PR02MB026A32099A863598902B1E9DF420@DB3PR02MB026.eurprd02.prod.outlook.com>
Message-ID: <CANCa_WZucD5KLa1Edc7i5C-PG=mecS=mc6FaQp31kNT7oNEUNQ@mail.gmail.com>

Yiannis,

Have you run apt-get update?

Andrew

On Sunday, 11 January 2015, ??????? ????????????? <gkaragiannidis at dolnet.gr>
wrote:

> Hi,
>
> I can't seem to download the source for Varnish 3.06 using the Wheezy APT
> repo.
> I need to fetch the source in order to compile libvmod-shield against it.
>
> My sources.list entry:
>
> deb http://repo.varnish-cache.org/debian/ wheezy varnish-3.0
> deb-src http://repo.varnish-cache.org/debian/ wheezy varnish-3.0
>
> My current install of 3.0.6:
>
>
> Any ideas?
>
> Regards
> Yiannis
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org <javascript:;>
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>


--
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150111/d33a4ca0/attachment.html>

From gkaragiannidis at dolnet.gr  Mon Jan 12 09:27:01 2015
From: gkaragiannidis at dolnet.gr (=?utf-8?B?zpPOuc6szr3Ovc63z4IgzprOsc+BzrHOs865zrHOvc69zq/OtM63z4I=?=)
Date: Mon, 12 Jan 2015 09:27:01 +0000
Subject: Source code for 3.0.6 missing from Debian repo 
Message-ID: <DB3PR02MB02607BED4B77EAEF9CCD82FDF430@DB3PR02MB026.eurprd02.prod.outlook.com>

Hi Andrew,
Of course I ?ve run apt-get update
Have you managed to download the varnish source?

Regards
Yiannis

From: Andrew Langhorn [mailto:andrew.langhorn at digital.cabinet-office.gov.uk]
Sent: Sunday, January 11, 2015 7:13 PM
To: ??????? ?????????????
Cc: varnish-misc at varnish-cache.org
Subject: Re: Source code for 3.0.6 missing from Debian repo

Yiannis,

Have you run apt-get update?

Andrew

On Sunday, 11 January 2015, ??????? ????????????? <gkaragiannidis at dolnet.gr<mailto:gkaragiannidis at dolnet.gr>> wrote:
Hi,

I can't seem to download the source for Varnish 3.06 using the Wheezy APT repo.
I need to fetch the source in order to compile libvmod-shield against it.

My sources.list entry:

deb http://repo.varnish-cache.org/debian/ wheezy varnish-3.0
deb-src http://repo.varnish-cache.org/debian/ wheezy varnish-3.0

My current install of 3.0.6:


Any ideas?

Regards
Yiannis
_______________________________________________
varnish-misc mailing list
varnish-misc at varnish-cache.org<javascript:;>
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc


--
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150112/1ccb068e/attachment.html>

From carlos.abalde at gmail.com  Mon Jan 12 09:35:35 2015
From: carlos.abalde at gmail.com (Carlos Abalde)
Date: Mon, 12 Jan 2015 10:35:35 +0100
Subject: Source code for 3.0.6 missing from Debian repo
In-Reply-To: <DB3PR02MB02607BED4B77EAEF9CCD82FDF430@DB3PR02MB026.eurprd02.prod.outlook.com>
References: <DB3PR02MB02607BED4B77EAEF9CCD82FDF430@DB3PR02MB026.eurprd02.prod.outlook.com>
Message-ID: <4783C7AA-7EB7-4D56-99ED-22D533E7C054@gmail.com>


> On 12 Jan 2015, at 10:27, ??????? ????????????? <gkaragiannidis at dolnet.gr> wrote:
> 
> Hi Andrew,
> Of course I ?ve run apt-get update
> Have you managed to download the varnish source?
>  

Hi,

I?ve experienced the same problem both in Debian and Ubuntu. You can download sources from https://repo.varnish-cache.org/debian/pool/varnish-3.0/v/varnish/varnish_3.0.6.orig.tar.gz

Cheers,

--
Carlos Abalde


From razvanphp at yahoo.com  Mon Jan 12 11:31:24 2015
From: razvanphp at yahoo.com (Razvan Grigore)
Date: Mon, 12 Jan 2015 12:31:24 +0100
Subject: Error at boot with varnishncsa
References: <mailman.7423.1421061924.2337.varnish-misc@varnish-cache.org>
Message-ID: <9EF2387A-20A2-40D6-BAD1-203CBA99CC7B@yahoo.com>

Hi.

We have the same problem with varnish-4.0.2, I see that the patch by @lkarsten is already there but the problem is still not solved.

We could run dpkg --configure -a successfully only after adding sleep 10 at the beginning of the init script of varnishncsa as suggested before.

Can anybody take a look at this? Currently it prevents smooth apt updates.

Thank you!
Razvan

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150112/228d14d5/attachment.html>

From ruben at varnish-software.com  Mon Jan 12 12:42:30 2015
From: ruben at varnish-software.com (=?UTF-8?Q?Rub=C3=A9n_Romero?=)
Date: Mon, 12 Jan 2015 13:42:30 +0100
Subject: Varnish 4 Request Flow
In-Reply-To: <CA+KACLmBJRggG_4gnkGK-A0Agz1o7dW8fEyHPcvzSG_7PigOfA@mail.gmail.com>
References: <CA+KACLkV=yg0QdZDWc4xmQ+7ewPc481o=8MEO-P+FuS-2uboAQ@mail.gmail.com>
 <20150107173004.GH19870@nerd.dk>
 <CA+KACLmBJRggG_4gnkGK-A0Agz1o7dW8fEyHPcvzSG_7PigOfA@mail.gmail.com>
Message-ID: <CAEfnkDbBMvWt7oo+7bvW6NBkfJHMAnAejx_wtyuTWFrcomLvmQ@mail.gmail.com>

On Wed, Jan 7, 2015 at 7:23 PM, Hugo Cisneiros (Eitch) <
hugo.cisneiros at gmail.com> wrote:

> On Wed, Jan 7, 2015 at 3:30 PM, Andreas Plesner Jacobsen <apj at mutt.dk>
> wrote:
> > > I'm beginning to use Varnish 4 a lot, and it's working great. I used to
> > > print the Varnish 3 Request Flow into my desk for reference when
> writing
> > > lots and lots of VCL code. Is there an "official" request flow for
> Varnish4?
> >
> > Nils added a graph to the docs recently:
> https://www.varnish-cache.org/trac/changeset/a48147aae7157fbc2d7de19b9e541ffa18fc9fa3
> >
> > https://www.varnish-cache.org/docs/trunk/reference/states.html
>
> That's exactly what I was looking for! Thanks a lot :-)


Hi Hugo,

Also worthwhile checking is the flow graph and other things in the new
version of the Varnish book which is in alpha now and focuses on 4.0:
https://github.com/varnish/Varnish-Book/blob/Varnish-Book-v4/varnish_book.rst

I believe we made it from source ourselves, so it is more or less the same
graph Andreas pointed you to.


> --
> []'s
> Hugo
> www.devin.com.br


Best,
-- 
<http://www.varnish-software.com/>*Rub?n Romero*
Varnish Software AS
Cell: +47 95964088 / Office: +47 21989260
Skype, Twitter & IRC: ruben_varnish
We Make Websites Fly!

[image: Varnish Summits Autumn 2014]
<http://info.varnish-software.com/varnish-summits-autumn-2014-registration>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150112/60bc2098/attachment.html>

From max.clark at gmail.com  Tue Jan 13 18:25:31 2015
From: max.clark at gmail.com (Max Clark)
Date: Tue, 13 Jan 2015 10:25:31 -0800
Subject: PURGE Regardless of Vary/Hash
Message-ID: <CAAaeqWeMTNOaDPSYeBT1xeBUuTj-rnP6dWe+c7gCt_7OUvbvMg@mail.gmail.com>

Hello,

I am trying to configure a consistent behavior for the PURGE with Varnish.
As has been pointed out in IRC there are several behaviors with Vary that
would impact the actual object being cached, plus we've added a cookie to
the hash data for unique page displays based on language. So my question,
if we wanted to PURGE everything for this URL:

www.foo.com/directory/article/

Regardless of any Vary, or Cookie hash data what needs to happen with the
VCL?

Thanks,
Max
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150113/de37c2dd/attachment.html>

From thomas.lecomte at virtual-expo.com  Tue Jan 13 19:01:18 2015
From: thomas.lecomte at virtual-expo.com (Thomas Lecomte)
Date: Tue, 13 Jan 2015 20:01:18 +0100
Subject: PURGE Regardless of Vary/Hash
In-Reply-To: <CAAaeqWeMTNOaDPSYeBT1xeBUuTj-rnP6dWe+c7gCt_7OUvbvMg@mail.gmail.com>
References: <CAAaeqWeMTNOaDPSYeBT1xeBUuTj-rnP6dWe+c7gCt_7OUvbvMg@mail.gmail.com>
Message-ID: <CAJban9BRXqP3Zj==abkUX4_9=k6vm5CWQKboh0sXLAEPwWXgAA@mail.gmail.com>

On Tue, Jan 13, 2015 at 7:25 PM, Max Clark <max.clark at gmail.com> wrote:
> Regardless of any Vary, or Cookie hash data what needs to happen with the
> VCL?

Hello Max,

I think you need to perform one PURGE call per variant, with the
appropriate data in the headers so that the VCL can compute the hash
of every single object you want to purge from the cache. I don't think
you can purge multiple objects at once, unlike with bans.

If I am wrong, I would love to be corrected because I had the very
same issue and had to went with bans instead :-)

/Thomas

-- 
Thomas Lecomte | Sysadmin
Virtual Expo, Marseille, France



From dridi.boukelmoune at zenika.com  Tue Jan 13 19:15:32 2015
From: dridi.boukelmoune at zenika.com (Dridi Boukelmoune)
Date: Tue, 13 Jan 2015 20:15:32 +0100
Subject: PURGE Regardless of Vary/Hash
In-Reply-To: <CAJban9BRXqP3Zj==abkUX4_9=k6vm5CWQKboh0sXLAEPwWXgAA@mail.gmail.com>
References: <CAAaeqWeMTNOaDPSYeBT1xeBUuTj-rnP6dWe+c7gCt_7OUvbvMg@mail.gmail.com>
 <CAJban9BRXqP3Zj==abkUX4_9=k6vm5CWQKboh0sXLAEPwWXgAA@mail.gmail.com>
Message-ID: <CABtDKm4CiabBkhZGz4mWwgggxzK7wdHp9HtvRrJE0yzHxOAaKA@mail.gmail.com>

On Tue, Jan 13, 2015 at 8:01 PM, Thomas Lecomte
<thomas.lecomte at virtual-expo.com> wrote:
> If I am wrong, I would love to be corrected because I had the very
> same issue and had to went with bans instead :-)

As a matter of fact, a purge happens for all variants behind a hash.

The default hash is computed from the host header (fallback to the
interface ip) and the url. Unless you do funky stuff in vcl_recv, it
should be a no brainer. My rule of thumb is to treat PURGE requests
like GET requests (include any modification that may happen on cache
keys before the lookup).

Don't forget to purge on a cache miss, just in case there's a variant
sitting in your cache.

For ban vs purge, you need to ask yourselves more questions, like
whether you'd need to keep a stale object in case of a backend
failure.

Dridi



From max.clark at gmail.com  Tue Jan 13 19:32:03 2015
From: max.clark at gmail.com (Max Clark)
Date: Tue, 13 Jan 2015 11:32:03 -0800
Subject: PURGE Regardless of Vary/Hash
In-Reply-To: <CABtDKm4CiabBkhZGz4mWwgggxzK7wdHp9HtvRrJE0yzHxOAaKA@mail.gmail.com>
References: <CAAaeqWeMTNOaDPSYeBT1xeBUuTj-rnP6dWe+c7gCt_7OUvbvMg@mail.gmail.com>
 <CAJban9BRXqP3Zj==abkUX4_9=k6vm5CWQKboh0sXLAEPwWXgAA@mail.gmail.com>
 <CABtDKm4CiabBkhZGz4mWwgggxzK7wdHp9HtvRrJE0yzHxOAaKA@mail.gmail.com>
Message-ID: <CAAaeqWcLDxb3eeA3uqWycP6hx3Y6vpvn+Q9TAi6GwkY6mvUefw@mail.gmail.com>

On Tue, Jan 13, 2015 at 11:15 AM, Dridi Boukelmoune <
dridi.boukelmoune at zenika.com> wrote:

> For ban vs purge, you need to ask yourselves more questions, like
> whether you'd need to keep a stale object in case of a backend
> failure.


Is that the only difference between BAN and PURGE?

Here's the recv, hash, hit and miss logic:

https://dpaste.de/V6d4

With this do I need to iterate over the different possible hash_data or
will the purge in the miss take care of it?

Thanks!
Max
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150113/f589f67f/attachment.html>

From dridi.boukelmoune at zenika.com  Tue Jan 13 19:52:02 2015
From: dridi.boukelmoune at zenika.com (Dridi Boukelmoune)
Date: Tue, 13 Jan 2015 20:52:02 +0100
Subject: PURGE Regardless of Vary/Hash
In-Reply-To: <CAAaeqWcLDxb3eeA3uqWycP6hx3Y6vpvn+Q9TAi6GwkY6mvUefw@mail.gmail.com>
References: <CAAaeqWeMTNOaDPSYeBT1xeBUuTj-rnP6dWe+c7gCt_7OUvbvMg@mail.gmail.com>
 <CAJban9BRXqP3Zj==abkUX4_9=k6vm5CWQKboh0sXLAEPwWXgAA@mail.gmail.com>
 <CABtDKm4CiabBkhZGz4mWwgggxzK7wdHp9HtvRrJE0yzHxOAaKA@mail.gmail.com>
 <CAAaeqWcLDxb3eeA3uqWycP6hx3Y6vpvn+Q9TAi6GwkY6mvUefw@mail.gmail.com>
Message-ID: <CABtDKm5TF0kzm91dDkFegajQQvPh3K0Fh+PxJc+KqY02yuqpVw@mail.gmail.com>

On Tue, Jan 13, 2015 at 8:32 PM, Max Clark <max.clark at gmail.com> wrote:
> Is that the only difference between BAN and PURGE?

Nope, both are fundamentally different!

> Here's the recv, hash, hit and miss logic:
>
> https://dpaste.de/V6d4
>
> With this do I need to iterate over the different possible hash_data or will
> the purge in the miss take care of it?

Like I said, a purge picks objects from a hash. So in your case if you
want to purge www.foo.com/directory/article/ you need to handle
X-Edition headers in your purge request(s).

> Thanks!
> Max
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc



From max.clark at gmail.com  Tue Jan 13 19:55:33 2015
From: max.clark at gmail.com (Max Clark)
Date: Tue, 13 Jan 2015 11:55:33 -0800
Subject: PURGE Regardless of Vary/Hash
In-Reply-To: <CABtDKm5TF0kzm91dDkFegajQQvPh3K0Fh+PxJc+KqY02yuqpVw@mail.gmail.com>
References: <CAAaeqWeMTNOaDPSYeBT1xeBUuTj-rnP6dWe+c7gCt_7OUvbvMg@mail.gmail.com>
 <CAJban9BRXqP3Zj==abkUX4_9=k6vm5CWQKboh0sXLAEPwWXgAA@mail.gmail.com>
 <CABtDKm4CiabBkhZGz4mWwgggxzK7wdHp9HtvRrJE0yzHxOAaKA@mail.gmail.com>
 <CAAaeqWcLDxb3eeA3uqWycP6hx3Y6vpvn+Q9TAi6GwkY6mvUefw@mail.gmail.com>
 <CABtDKm5TF0kzm91dDkFegajQQvPh3K0Fh+PxJc+KqY02yuqpVw@mail.gmail.com>
Message-ID: <CAAaeqWcZARrbopzGcLi3wEsbmrU8jsJNY4vxAb9s7Uddxbj-WQ@mail.gmail.com>

On Tue, Jan 13, 2015 at 11:52 AM, Dridi Boukelmoune <
dridi.boukelmoune at zenika.com> wrote:

> Like I said, a purge picks objects from a hash. So in your case if you
> want to purge www.foo.com/directory/article/ you need to handle
> X-Edition headers in your purge request(s).
>

Is there a way to handle this solely in the Varnish VCL?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150113/625dc8f0/attachment.html>

From dridi.boukelmoune at zenika.com  Tue Jan 13 20:23:19 2015
From: dridi.boukelmoune at zenika.com (Dridi Boukelmoune)
Date: Tue, 13 Jan 2015 21:23:19 +0100
Subject: PURGE Regardless of Vary/Hash
In-Reply-To: <CAAaeqWcZARrbopzGcLi3wEsbmrU8jsJNY4vxAb9s7Uddxbj-WQ@mail.gmail.com>
References: <CAAaeqWeMTNOaDPSYeBT1xeBUuTj-rnP6dWe+c7gCt_7OUvbvMg@mail.gmail.com>
 <CAJban9BRXqP3Zj==abkUX4_9=k6vm5CWQKboh0sXLAEPwWXgAA@mail.gmail.com>
 <CABtDKm4CiabBkhZGz4mWwgggxzK7wdHp9HtvRrJE0yzHxOAaKA@mail.gmail.com>
 <CAAaeqWcLDxb3eeA3uqWycP6hx3Y6vpvn+Q9TAi6GwkY6mvUefw@mail.gmail.com>
 <CABtDKm5TF0kzm91dDkFegajQQvPh3K0Fh+PxJc+KqY02yuqpVw@mail.gmail.com>
 <CAAaeqWcZARrbopzGcLi3wEsbmrU8jsJNY4vxAb9s7Uddxbj-WQ@mail.gmail.com>
Message-ID: <CABtDKm7LjiqaiHrpYkoe54ZWesmwwr5R1+JJmTy1DPs_g+fvLg@mail.gmail.com>

On Tue, Jan 13, 2015 at 8:55 PM, Max Clark <max.clark at gmail.com> wrote:
>
> On Tue, Jan 13, 2015 at 11:52 AM, Dridi Boukelmoune
> <dridi.boukelmoune at zenika.com> wrote:
>>
>> Like I said, a purge picks objects from a hash. So in your case if you
>> want to purge www.foo.com/directory/article/ you need to handle
>> X-Edition headers in your purge request(s).
>
>
> Is there a way to handle this solely in the Varnish VCL?

The header is part of the request, it is user input and shouldn't even
be trusted in the first place. Most of the time I'd reject a request
without a host header for instance.

And also I don't even know X-Edition's purpose, so I can't answer your
question with just VCL. That being said, the answer is most likely no
anyway.



From raymond.jennings at nytimes.com  Thu Jan 15 14:36:46 2015
From: raymond.jennings at nytimes.com (Jennings III, Raymond)
Date: Thu, 15 Jan 2015 09:36:46 -0500
Subject: Cannot figure out why I am getting a miss on a static URL
Message-ID: <CADFjDbO2mE8502A9KYhbZqKFcLU0mk7XY15zrmayTmZZ4N_0Vg@mail.gmail.com>

I have a static URL that I set a TTL for 24h but yet every so many minutes
I get a "miss" (as shown in my varnishncsa.)

I purge this URL every 24 hours.

1 - The URL never has any parameters.
2 - I do not hash over the hostname only the URI
3 - Nobody is issuing any PURGE or BAN requests
4 - n_lru_nuked is always 0
5 - There are no 500 or 400 responses

What is causing this URL to get a miss when everything I see looks good?
If the cache size has been exceeded I would have expected to see the nuked
value be something other than 0.

vcl_fetch is basically just:

set beresp.ttl = 24h;
return(deliver);

Any reason why Varnish seems to be taking it upon itself to cause this
miss?  In 24 hours I get about 12000 requests and out of those 120 misses
and I should see 2 miss requests at most.

The backends are healthy.

This is varnish-3.0.6-1.el6.x86_64

Thanks.



Raymond Jennings III
*nytimes.com <http://nytimes.com/>*
*Office: 212.556.7786 <212-556-7786>*

*iPhone: 914.330.5074 <914-330-5074>E-mail: Raymond.Jennings at nytimes.com
<Raymond.Jennings at nytimes.com>FaceTime: Raymond.Jennings at nytimes.com
<Raymond.Jennings at nytimes.com>*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150115/076d52f4/attachment.html>

From dridi.boukelmoune at zenika.com  Fri Jan 16 15:01:10 2015
From: dridi.boukelmoune at zenika.com (Dridi Boukelmoune)
Date: Fri, 16 Jan 2015 16:01:10 +0100
Subject: Cannot figure out why I am getting a miss on a static URL
In-Reply-To: <CADFjDbO2mE8502A9KYhbZqKFcLU0mk7XY15zrmayTmZZ4N_0Vg@mail.gmail.com>
References: <CADFjDbO2mE8502A9KYhbZqKFcLU0mk7XY15zrmayTmZZ4N_0Vg@mail.gmail.com>
Message-ID: <CABtDKm4q2gv64PAhdCHAjjZK3FYNXWC9czBcCQnf2TKFxHXcXw@mail.gmail.com>

Hi Raymond,

Have you looked at the logs?

Your description of the issue is too vague, I don't think anyone can
help you with just that.

Cheers,
Dridi

On Thu, Jan 15, 2015 at 3:36 PM, Jennings III, Raymond
<raymond.jennings at nytimes.com> wrote:
> I have a static URL that I set a TTL for 24h but yet every so many minutes I
> get a "miss" (as shown in my varnishncsa.)
>
> I purge this URL every 24 hours.
>
> 1 - The URL never has any parameters.
> 2 - I do not hash over the hostname only the URI
> 3 - Nobody is issuing any PURGE or BAN requests
> 4 - n_lru_nuked is always 0
> 5 - There are no 500 or 400 responses
>
> What is causing this URL to get a miss when everything I see looks good?  If
> the cache size has been exceeded I would have expected to see the nuked
> value be something other than 0.
>
> vcl_fetch is basically just:
>
> set beresp.ttl = 24h;
> return(deliver);
>
> Any reason why Varnish seems to be taking it upon itself to cause this miss?
> In 24 hours I get about 12000 requests and out of those 120 misses and I
> should see 2 miss requests at most.
>
> The backends are healthy.
>
> This is varnish-3.0.6-1.el6.x86_64
>
> Thanks.
>
>
>
> Raymond Jennings III
> nytimes.com
> Office: 212.556.7786
> iPhone: 914.330.5074
> E-mail: Raymond.Jennings at nytimes.com
> FaceTime: Raymond.Jennings at nytimes.com
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc



From raymond.jennings at nytimes.com  Fri Jan 16 15:07:07 2015
From: raymond.jennings at nytimes.com (Jennings III, Raymond)
Date: Fri, 16 Jan 2015 10:07:07 -0500
Subject: Cannot figure out why I am getting a miss on a static URL
In-Reply-To: <CABtDKm4q2gv64PAhdCHAjjZK3FYNXWC9czBcCQnf2TKFxHXcXw@mail.gmail.com>
References: <CADFjDbO2mE8502A9KYhbZqKFcLU0mk7XY15zrmayTmZZ4N_0Vg@mail.gmail.com>
 <CABtDKm4q2gv64PAhdCHAjjZK3FYNXWC9czBcCQnf2TKFxHXcXw@mail.gmail.com>
Message-ID: <CADFjDbPDUi39gEJjRU-7N8M0WxK90Da5NbU11=jOtqkq+c8z-g@mail.gmail.com>

varnishncsa clearly shows hits and misses within a 24 hour period despite
clearly setting the TTL to 24 hours.  Varnishlog clearly shows the TTL
being set to 24 hours.  I have NO lru_nuked objects as clearly shown by the
varnish stats.

I'm not sure what is vague.  I am setting an object for a TTL of 24 hours
yet during that 24 hour period it is still getting miss requests.  The URLs
in question have no GET parameters.  I am hashing only on the URL and not
the hostname.

So what does this tell me?  It tells me that my cache size is big enough.

My guess is there is some type of contention from locking of some type
given other requests coming through.  There are quite a few PUT and POST
requests that pass through this Varnish server.

Ummmm, I hardly think that "anyone can't help me."  I think most definitely
"someone" that has some insight about the internals could shed some light
on this problem.



Raymond Jennings III
*nytimes.com <http://nytimes.com/>*
*Office: 212.556.7786 <212-556-7786>*

*iPhone: 914.330.5074 <914-330-5074>E-mail: Raymond.Jennings at nytimes.com
<Raymond.Jennings at nytimes.com>FaceTime: Raymond.Jennings at nytimes.com
<Raymond.Jennings at nytimes.com>*

On Fri, Jan 16, 2015 at 10:01 AM, Dridi Boukelmoune <
dridi.boukelmoune at zenika.com> wrote:

> Hi Raymond,
>
> Have you looked at the logs?
>
> Your description of the issue is too vague, I don't think anyone can
> help you with just that.
>
> Cheers,
> Dridi
>
> On Thu, Jan 15, 2015 at 3:36 PM, Jennings III, Raymond
> <raymond.jennings at nytimes.com> wrote:
> > I have a static URL that I set a TTL for 24h but yet every so many
> minutes I
> > get a "miss" (as shown in my varnishncsa.)
> >
> > I purge this URL every 24 hours.
> >
> > 1 - The URL never has any parameters.
> > 2 - I do not hash over the hostname only the URI
> > 3 - Nobody is issuing any PURGE or BAN requests
> > 4 - n_lru_nuked is always 0
> > 5 - There are no 500 or 400 responses
> >
> > What is causing this URL to get a miss when everything I see looks
> good?  If
> > the cache size has been exceeded I would have expected to see the nuked
> > value be something other than 0.
> >
> > vcl_fetch is basically just:
> >
> > set beresp.ttl = 24h;
> > return(deliver);
> >
> > Any reason why Varnish seems to be taking it upon itself to cause this
> miss?
> > In 24 hours I get about 12000 requests and out of those 120 misses and I
> > should see 2 miss requests at most.
> >
> > The backends are healthy.
> >
> > This is varnish-3.0.6-1.el6.x86_64
> >
> > Thanks.
> >
> >
> >
> > Raymond Jennings III
> > nytimes.com
> > Office: 212.556.7786
> > iPhone: 914.330.5074
> > E-mail: Raymond.Jennings at nytimes.com
> > FaceTime: Raymond.Jennings at nytimes.com
> >
> > _______________________________________________
> > varnish-misc mailing list
> > varnish-misc at varnish-cache.org
> > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150116/191fba98/attachment.html>

From viktor.gunnarson at ericsson.com  Fri Jan 16 15:21:11 2015
From: viktor.gunnarson at ericsson.com (Viktor Gunnarson)
Date: Fri, 16 Jan 2015 15:21:11 +0000
Subject: Cannot figure out why I am getting a miss on a static URL
In-Reply-To: <CADFjDbPDUi39gEJjRU-7N8M0WxK90Da5NbU11=jOtqkq+c8z-g@mail.gmail.com>
References: <CADFjDbO2mE8502A9KYhbZqKFcLU0mk7XY15zrmayTmZZ4N_0Vg@mail.gmail.com>
 <CABtDKm4q2gv64PAhdCHAjjZK3FYNXWC9czBcCQnf2TKFxHXcXw@mail.gmail.com>
 <CADFjDbPDUi39gEJjRU-7N8M0WxK90Da5NbU11=jOtqkq+c8z-g@mail.gmail.com>
Message-ID: <CA6977DCB73E5F45977D63AB5E00C9E7010B1EA5@ESESSMB109.ericsson.se>

Have you tried logging the requests that get misses? Perhaps that could give some additional info.

varnishlog -q "VCL_call ~ 'MISS'"

Also you could provide the vcl because without that it?s really hard to know what happens...

Best regards,
Viktor


From: varnish-misc-bounces+viktor.gunnarson=ericsson.com at varnish-cache.org [mailto:varnish-misc-bounces+viktor.gunnarson=ericsson.com at varnish-cache.org] On Behalf Of Jennings III, Raymond
Sent: den 16 januari 2015 16:07
To: Dridi Boukelmoune
Cc: varnish-misc
Subject: Re: Cannot figure out why I am getting a miss on a static URL

varnishncsa clearly shows hits and misses within a 24 hour period despite clearly setting the TTL to 24 hours.  Varnishlog clearly shows the TTL being set to 24 hours.  I have NO lru_nuked objects as clearly shown by the varnish stats.

I'm not sure what is vague.  I am setting an object for a TTL of 24 hours yet during that 24 hour period it is still getting miss requests.  The URLs in question have no GET parameters.  I am hashing only on the URL and not the hostname.

So what does this tell me?  It tells me that my cache size is big enough.

My guess is there is some type of contention from locking of some type given other requests coming through.  There are quite a few PUT and POST requests that pass through this Varnish server.

Ummmm, I hardly think that "anyone can't help me."  I think most definitely "someone" that has some insight about the internals could shed some light on this problem.



Raymond Jennings III
nytimes.com<http://nytimes.com/>
Office: 212.556.7786<tel:212-556-7786>
iPhone: 914.330.5074<tel:914-330-5074>
E-mail: Raymond.Jennings at nytimes.com<mailto:Raymond.Jennings at nytimes.com>
FaceTime: Raymond.Jennings at nytimes.com<mailto:Raymond.Jennings at nytimes.com>

On Fri, Jan 16, 2015 at 10:01 AM, Dridi Boukelmoune <dridi.boukelmoune at zenika.com<mailto:dridi.boukelmoune at zenika.com>> wrote:
Hi Raymond,

Have you looked at the logs?

Your description of the issue is too vague, I don't think anyone can
help you with just that.

Cheers,
Dridi

On Thu, Jan 15, 2015 at 3:36 PM, Jennings III, Raymond
<raymond.jennings at nytimes.com<mailto:raymond.jennings at nytimes.com>> wrote:
> I have a static URL that I set a TTL for 24h but yet every so many minutes I
> get a "miss" (as shown in my varnishncsa.)
>
> I purge this URL every 24 hours.
>
> 1 - The URL never has any parameters.
> 2 - I do not hash over the hostname only the URI
> 3 - Nobody is issuing any PURGE or BAN requests
> 4 - n_lru_nuked is always 0
> 5 - There are no 500 or 400 responses
>
> What is causing this URL to get a miss when everything I see looks good?  If
> the cache size has been exceeded I would have expected to see the nuked
> value be something other than 0.
>
> vcl_fetch is basically just:
>
> set beresp.ttl = 24h;
> return(deliver);
>
> Any reason why Varnish seems to be taking it upon itself to cause this miss?
> In 24 hours I get about 12000 requests and out of those 120 misses and I
> should see 2 miss requests at most.
>
> The backends are healthy.
>
> This is varnish-3.0.6-1.el6.x86_64
>
> Thanks.
>
>
>
> Raymond Jennings III
> nytimes.com<http://nytimes.com>
> Office: 212.556.7786<tel:212.556.7786>
> iPhone: 914.330.5074<tel:914.330.5074>
> E-mail: Raymond.Jennings at nytimes.com<mailto:Raymond.Jennings at nytimes.com>
> FaceTime: Raymond.Jennings at nytimes.com<mailto:Raymond.Jennings at nytimes.com>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org<mailto:varnish-misc at varnish-cache.org>
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150116/084c083b/attachment.html>

From dridi.boukelmoune at zenika.com  Fri Jan 16 15:26:11 2015
From: dridi.boukelmoune at zenika.com (Dridi Boukelmoune)
Date: Fri, 16 Jan 2015 16:26:11 +0100
Subject: Cannot figure out why I am getting a miss on a static URL
In-Reply-To: <CADFjDbPDUi39gEJjRU-7N8M0WxK90Da5NbU11=jOtqkq+c8z-g@mail.gmail.com>
References: <CADFjDbO2mE8502A9KYhbZqKFcLU0mk7XY15zrmayTmZZ4N_0Vg@mail.gmail.com>
 <CABtDKm4q2gv64PAhdCHAjjZK3FYNXWC9czBcCQnf2TKFxHXcXw@mail.gmail.com>
 <CADFjDbPDUi39gEJjRU-7N8M0WxK90Da5NbU11=jOtqkq+c8z-g@mail.gmail.com>
Message-ID: <CABtDKm7KmuJfFeeSvBi33H=R=9_7V=TgYDDJEpfyMu_o-PjTdQ@mail.gmail.com>

What I mean is, post enough VCL to reproduce the issue, and maybe
varnishlog logs showing what's happening. If your vcl_fetch really
returns "deliver" unconditionally, you're probably doing something
wrong.

Dridi

On Fri, Jan 16, 2015 at 4:07 PM, Jennings III, Raymond
<raymond.jennings at nytimes.com> wrote:
> varnishncsa clearly shows hits and misses within a 24 hour period despite
> clearly setting the TTL to 24 hours.  Varnishlog clearly shows the TTL being
> set to 24 hours.  I have NO lru_nuked objects as clearly shown by the
> varnish stats.
>
> I'm not sure what is vague.  I am setting an object for a TTL of 24 hours
> yet during that 24 hour period it is still getting miss requests.  The URLs
> in question have no GET parameters.  I am hashing only on the URL and not
> the hostname.
>
> So what does this tell me?  It tells me that my cache size is big enough.
>
> My guess is there is some type of contention from locking of some type given
> other requests coming through.  There are quite a few PUT and POST requests
> that pass through this Varnish server.
>
> Ummmm, I hardly think that "anyone can't help me."  I think most definitely
> "someone" that has some insight about the internals could shed some light on
> this problem.
>
>
>
> Raymond Jennings III
> nytimes.com
> Office: 212.556.7786
> iPhone: 914.330.5074
> E-mail: Raymond.Jennings at nytimes.com
> FaceTime: Raymond.Jennings at nytimes.com
>
> On Fri, Jan 16, 2015 at 10:01 AM, Dridi Boukelmoune
> <dridi.boukelmoune at zenika.com> wrote:
>>
>> Hi Raymond,
>>
>> Have you looked at the logs?
>>
>> Your description of the issue is too vague, I don't think anyone can
>> help you with just that.
>>
>> Cheers,
>> Dridi
>>
>> On Thu, Jan 15, 2015 at 3:36 PM, Jennings III, Raymond
>> <raymond.jennings at nytimes.com> wrote:
>> > I have a static URL that I set a TTL for 24h but yet every so many
>> > minutes I
>> > get a "miss" (as shown in my varnishncsa.)
>> >
>> > I purge this URL every 24 hours.
>> >
>> > 1 - The URL never has any parameters.
>> > 2 - I do not hash over the hostname only the URI
>> > 3 - Nobody is issuing any PURGE or BAN requests
>> > 4 - n_lru_nuked is always 0
>> > 5 - There are no 500 or 400 responses
>> >
>> > What is causing this URL to get a miss when everything I see looks good?
>> > If
>> > the cache size has been exceeded I would have expected to see the nuked
>> > value be something other than 0.
>> >
>> > vcl_fetch is basically just:
>> >
>> > set beresp.ttl = 24h;
>> > return(deliver);
>> >
>> > Any reason why Varnish seems to be taking it upon itself to cause this
>> > miss?
>> > In 24 hours I get about 12000 requests and out of those 120 misses and I
>> > should see 2 miss requests at most.
>> >
>> > The backends are healthy.
>> >
>> > This is varnish-3.0.6-1.el6.x86_64
>> >
>> > Thanks.
>> >
>> >
>> >
>> > Raymond Jennings III
>> > nytimes.com
>> > Office: 212.556.7786
>> > iPhone: 914.330.5074
>> > E-mail: Raymond.Jennings at nytimes.com
>> > FaceTime: Raymond.Jennings at nytimes.com
>> >
>> > _______________________________________________
>> > varnish-misc mailing list
>> > varnish-misc at varnish-cache.org
>> > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
>



From dridi.boukelmoune at zenika.com  Fri Jan 16 15:27:11 2015
From: dridi.boukelmoune at zenika.com (Dridi Boukelmoune)
Date: Fri, 16 Jan 2015 16:27:11 +0100
Subject: Cannot figure out why I am getting a miss on a static URL
In-Reply-To: <CA6977DCB73E5F45977D63AB5E00C9E7010B1EA5@ESESSMB109.ericsson.se>
References: <CADFjDbO2mE8502A9KYhbZqKFcLU0mk7XY15zrmayTmZZ4N_0Vg@mail.gmail.com>
 <CABtDKm4q2gv64PAhdCHAjjZK3FYNXWC9czBcCQnf2TKFxHXcXw@mail.gmail.com>
 <CADFjDbPDUi39gEJjRU-7N8M0WxK90Da5NbU11=jOtqkq+c8z-g@mail.gmail.com>
 <CA6977DCB73E5F45977D63AB5E00C9E7010B1EA5@ESESSMB109.ericsson.se>
Message-ID: <CABtDKm45LpJr_uDaPWaBL+EmSaRDD9B2iKBo5LKKYB1YoEOwRQ@mail.gmail.com>

On Fri, Jan 16, 2015 at 4:21 PM, Viktor Gunnarson
<viktor.gunnarson at ericsson.com> wrote:
> Have you tried logging the requests that get misses? Perhaps that could give
> some additional info.
>
>
>
> varnishlog -q "VCL_call ~ 'MISS'"

Nope, he's using Varnish 3.

> Also you could provide the vcl because without that it?s really hard to know
> what happens...
>
>
>
> Best regards,
>
> Viktor
>
>
>
>
>
> From: varnish-misc-bounces+viktor.gunnarson=ericsson.com at varnish-cache.org
> [mailto:varnish-misc-bounces+viktor.gunnarson=ericsson.com at varnish-cache.org]
> On Behalf Of Jennings III, Raymond
> Sent: den 16 januari 2015 16:07
> To: Dridi Boukelmoune
> Cc: varnish-misc
> Subject: Re: Cannot figure out why I am getting a miss on a static URL
>
>
>
> varnishncsa clearly shows hits and misses within a 24 hour period despite
> clearly setting the TTL to 24 hours.  Varnishlog clearly shows the TTL being
> set to 24 hours.  I have NO lru_nuked objects as clearly shown by the
> varnish stats.
>
>
>
> I'm not sure what is vague.  I am setting an object for a TTL of 24 hours
> yet during that 24 hour period it is still getting miss requests.  The URLs
> in question have no GET parameters.  I am hashing only on the URL and not
> the hostname.
>
>
>
> So what does this tell me?  It tells me that my cache size is big enough.
>
>
>
> My guess is there is some type of contention from locking of some type given
> other requests coming through.  There are quite a few PUT and POST requests
> that pass through this Varnish server.
>
>
>
> Ummmm, I hardly think that "anyone can't help me."  I think most definitely
> "someone" that has some insight about the internals could shed some light on
> this problem.
>
>
>
>
>
>
> Raymond Jennings III
>
> nytimes.com
>
> Office: 212.556.7786
>
> iPhone: 914.330.5074
>
> E-mail: Raymond.Jennings at nytimes.com
>
> FaceTime: Raymond.Jennings at nytimes.com
>
>
>
> On Fri, Jan 16, 2015 at 10:01 AM, Dridi Boukelmoune
> <dridi.boukelmoune at zenika.com> wrote:
>
> Hi Raymond,
>
> Have you looked at the logs?
>
> Your description of the issue is too vague, I don't think anyone can
> help you with just that.
>
> Cheers,
> Dridi
>
>
> On Thu, Jan 15, 2015 at 3:36 PM, Jennings III, Raymond
> <raymond.jennings at nytimes.com> wrote:
>> I have a static URL that I set a TTL for 24h but yet every so many minutes
>> I
>> get a "miss" (as shown in my varnishncsa.)
>>
>> I purge this URL every 24 hours.
>>
>> 1 - The URL never has any parameters.
>> 2 - I do not hash over the hostname only the URI
>> 3 - Nobody is issuing any PURGE or BAN requests
>> 4 - n_lru_nuked is always 0
>> 5 - There are no 500 or 400 responses
>>
>> What is causing this URL to get a miss when everything I see looks good?
>> If
>> the cache size has been exceeded I would have expected to see the nuked
>> value be something other than 0.
>>
>> vcl_fetch is basically just:
>>
>> set beresp.ttl = 24h;
>> return(deliver);
>>
>> Any reason why Varnish seems to be taking it upon itself to cause this
>> miss?
>> In 24 hours I get about 12000 requests and out of those 120 misses and I
>> should see 2 miss requests at most.
>>
>> The backends are healthy.
>>
>> This is varnish-3.0.6-1.el6.x86_64
>>
>> Thanks.
>>
>>
>>
>> Raymond Jennings III
>> nytimes.com
>> Office: 212.556.7786
>> iPhone: 914.330.5074
>> E-mail: Raymond.Jennings at nytimes.com
>> FaceTime: Raymond.Jennings at nytimes.com
>>
>
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at varnish-cache.org
>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
>



From raymond.jennings at nytimes.com  Fri Jan 16 15:34:17 2015
From: raymond.jennings at nytimes.com (Jennings III, Raymond)
Date: Fri, 16 Jan 2015 10:34:17 -0500
Subject: Cannot figure out why I am getting a miss on a static URL
In-Reply-To: <CA6977DCB73E5F45977D63AB5E00C9E7010B1EA5@ESESSMB109.ericsson.se>
References: <CADFjDbO2mE8502A9KYhbZqKFcLU0mk7XY15zrmayTmZZ4N_0Vg@mail.gmail.com>
 <CABtDKm4q2gv64PAhdCHAjjZK3FYNXWC9czBcCQnf2TKFxHXcXw@mail.gmail.com>
 <CADFjDbPDUi39gEJjRU-7N8M0WxK90Da5NbU11=jOtqkq+c8z-g@mail.gmail.com>
 <CA6977DCB73E5F45977D63AB5E00C9E7010B1EA5@ESESSMB109.ericsson.se>
Message-ID: <CADFjDbOGigoBkq6CLq3kMAnoxY==uC=t=KZ+yvgJgKzbhF3=Lw@mail.gmail.com>

As far as I know there is no "-q" option on varnishlog for Varnish 3.0.6

The VCL is too complicated to post but I have added a ton of syslog
conditional calls to show that the right lines and blocks in the VCL are
being run.

but here is some:

sub vcl_hash {
 hash_data(req.url);
 return(hash);
}


sub vcl_fetch {
...

   if ( req.url ~ "^/my/url/.*" ) {
       set beresp.ttl = 24h;
C{
     syslog(LOG_ERR, "TTL was set to 24h");
}C
   }

   return(deliver);
}


sub vcl_deliver {

 return(deliver);
}


In my varnishncsa I print "hit" and "miss" with the option:
  %{Varnish:handling}x

That is where I am seeing the miss requests.


tail -f varnishncsa.access_log | grep "\/my\/url" | grep "miss"


Sometimes I get a miss within 2 minutes, other times it can go 20 minutes.


I also have a staging system with the exact same VCL and it never happens.
So it does not appear to be something forcing a cache clearing (There re no
PURGE or BAN requests) and I am currently ignoring all cache related
headers. and force a return(deliver) so that the default varnish handler
for vcl_fetch does not run.


Again, I see no lru_objects, I do see some lru_moved objects and I see
quite a few lock operations but I do not know what they mean.  So my guess
is that this problem has to do with some type of contention whether it be
threads or locks.



Raymond Jennings III
*nytimes.com <http://nytimes.com/>*
*Office: 212.556.7786 <212-556-7786>*

*iPhone: 914.330.5074 <914-330-5074>E-mail: Raymond.Jennings at nytimes.com
<Raymond.Jennings at nytimes.com>FaceTime: Raymond.Jennings at nytimes.com
<Raymond.Jennings at nytimes.com>*

On Fri, Jan 16, 2015 at 10:21 AM, Viktor Gunnarson <
viktor.gunnarson at ericsson.com> wrote:

>  Have you tried logging the requests that get misses? Perhaps that could
> give some additional info.
>
>
>
> varnishlog -q "VCL_call ~ 'MISS'"
>
>
>
> Also you could provide the vcl because without that it?s really hard to
> know what happens...
>
>
>
> Best regards,
>
> Viktor
>
>
>
>
>
> *From:* varnish-misc-bounces+viktor.gunnarson=
> ericsson.com at varnish-cache.org [mailto:
> varnish-misc-bounces+viktor.gunnarson=ericsson.com at varnish-cache.org] *
> On Behalf Of *Jennings III, Raymond
> *Sent:* den 16 januari 2015 16:07
> *To:* Dridi Boukelmoune
> *Cc:* varnish-misc
> *Subject:* Re: Cannot figure out why I am getting a miss on a static URL
>
>
>
> varnishncsa clearly shows hits and misses within a 24 hour period despite
> clearly setting the TTL to 24 hours.  Varnishlog clearly shows the TTL
> being set to 24 hours.  I have NO lru_nuked objects as clearly shown by the
> varnish stats.
>
>
>
> I'm not sure what is vague.  I am setting an object for a TTL of 24 hours
> yet during that 24 hour period it is still getting miss requests.  The URLs
> in question have no GET parameters.  I am hashing only on the URL and not
> the hostname.
>
>
>
> So what does this tell me?  It tells me that my cache size is big enough.
>
>
>
> My guess is there is some type of contention from locking of some type
> given other requests coming through.  There are quite a few PUT and POST
> requests that pass through this Varnish server.
>
>
>
> Ummmm, I hardly think that "anyone can't help me."  I think most
> definitely "someone" that has some insight about the internals could shed
> some light on this problem.
>
>
>
>
>
>
>       Raymond Jennings III
>
> *nytimes.com <http://nytimes.com/>*
>
> *Office:* 212.556.7786 <212-556-7786>
>
> *iPhone:* 914.330.5074 <914-330-5074>
>
> *E-mail: **Raymond.Jennings at nytimes.com <Raymond.Jennings at nytimes.com>*
>
> *FaceTime:* Raymond.Jennings at nytimes.com
>
>
>
> On Fri, Jan 16, 2015 at 10:01 AM, Dridi Boukelmoune <
> dridi.boukelmoune at zenika.com> wrote:
>
> Hi Raymond,
>
> Have you looked at the logs?
>
> Your description of the issue is too vague, I don't think anyone can
> help you with just that.
>
> Cheers,
> Dridi
>
>
> On Thu, Jan 15, 2015 at 3:36 PM, Jennings III, Raymond
> <raymond.jennings at nytimes.com> wrote:
> > I have a static URL that I set a TTL for 24h but yet every so many
> minutes I
> > get a "miss" (as shown in my varnishncsa.)
> >
> > I purge this URL every 24 hours.
> >
> > 1 - The URL never has any parameters.
> > 2 - I do not hash over the hostname only the URI
> > 3 - Nobody is issuing any PURGE or BAN requests
> > 4 - n_lru_nuked is always 0
> > 5 - There are no 500 or 400 responses
> >
> > What is causing this URL to get a miss when everything I see looks
> good?  If
> > the cache size has been exceeded I would have expected to see the nuked
> > value be something other than 0.
> >
> > vcl_fetch is basically just:
> >
> > set beresp.ttl = 24h;
> > return(deliver);
> >
> > Any reason why Varnish seems to be taking it upon itself to cause this
> miss?
> > In 24 hours I get about 12000 requests and out of those 120 misses and I
> > should see 2 miss requests at most.
> >
> > The backends are healthy.
> >
> > This is varnish-3.0.6-1.el6.x86_64
> >
> > Thanks.
> >
> >
> >
> > Raymond Jennings III
> > nytimes.com
> > Office: 212.556.7786
> > iPhone: 914.330.5074
> > E-mail: Raymond.Jennings at nytimes.com
> > FaceTime: Raymond.Jennings at nytimes.com
> >
>
> > _______________________________________________
> > varnish-misc mailing list
> > varnish-misc at varnish-cache.org
> > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150116/9fd45490/attachment.html>

From thierry.magnien at sfr.com  Fri Jan 16 15:52:37 2015
From: thierry.magnien at sfr.com (MAGNIEN, Thierry)
Date: Fri, 16 Jan 2015 15:52:37 +0000
Subject: Cannot figure out why I am getting a miss on a static URL
In-Reply-To: <CADFjDbOGigoBkq6CLq3kMAnoxY==uC=t=KZ+yvgJgKzbhF3=Lw@mail.gmail.com>
References: <CADFjDbO2mE8502A9KYhbZqKFcLU0mk7XY15zrmayTmZZ4N_0Vg@mail.gmail.com>
 <CABtDKm4q2gv64PAhdCHAjjZK3FYNXWC9czBcCQnf2TKFxHXcXw@mail.gmail.com>
 <CADFjDbPDUi39gEJjRU-7N8M0WxK90Da5NbU11=jOtqkq+c8z-g@mail.gmail.com>
 <CA6977DCB73E5F45977D63AB5E00C9E7010B1EA5@ESESSMB109.ericsson.se>
 <CADFjDbOGigoBkq6CLq3kMAnoxY==uC=t=KZ+yvgJgKzbhF3=Lw@mail.gmail.com>
Message-ID: <5D103CE839D50E4CBC62C9FD7B83287C010D6AF8F8@EXCN013.encara.local.ads>

Hi,

Please try something like this?:

varnishlog ?c ?m VCL_call:miss

This will show you requests leading to a miss. If this is always on the same URL, please try :

varnishlog ?c ?m RxURL:your_url

This will show what TTL is applied to your object.

Regards,
Thierry
---------------


De?: varnish-misc-bounces+thierry.magnien=sfr.com at varnish-cache.org [mailto:varnish-misc-bounces+thierry.magnien=sfr.com at varnish-cache.org] De la part de Jennings III, Raymond
Envoy??: vendredi 16 janvier 2015 16:34
??: Viktor Gunnarson
Cc?: varnish-misc
Objet?: Re: Cannot figure out why I am getting a miss on a static URL

As far as I know there is no "-q" option on varnishlog for Varnish 3.0.6

The VCL is too complicated to post but I have added a ton of syslog conditional calls to show that the right lines and blocks in the VCL are being run.

but here is some:

sub vcl_hash {
?hash_data(req.url);
?return(hash);
}


sub vcl_fetch {
...

? ?if ( req.url ~ "^/my/url/.*" ) {
? ? ? ?set beresp.ttl = 24h;
C{
? ? ?syslog(LOG_ERR, "TTL was set to 24h");
}C
? ?}

? ?return(deliver);
}


sub vcl_deliver {

?return(deliver);
}


In my varnishncsa I print "hit" and "miss" with the option: ??%{Varnish:handling}x

That is where I am seeing the miss requests.


tail -f varnishncsa.access_log | grep "\/my\/url" | grep "miss"


Sometimes I get a miss within 2 minutes, other times it can go 20 minutes.


I also have a staging system with the exact same VCL and it never happens.? So it does not appear to be something forcing a cache clearing (There re no PURGE or BAN requests) and I am currently ignoring all cache related headers. and force a return(deliver) so that the default varnish handler for vcl_fetch does not run.


Again, I see no lru_objects, I do see some lru_moved objects and I see quite a few lock operations but I do not know what they mean.? So my guess is that this problem has to do with some type of contention whether it be threads or locks.




Raymond Jennings III
nytimes.com
Office:?212.556.7786
iPhone:?914.330.5074
E-mail:?Raymond.Jennings at nytimes.com
FaceTime:?Raymond.Jennings at nytimes.com

On Fri, Jan 16, 2015 at 10:21 AM, Viktor Gunnarson <viktor.gunnarson at ericsson.com> wrote:
Have you tried logging the requests that get misses? Perhaps that could give some additional info.
?
varnishlog -q "VCL_call ~ 'MISS'"
?
Also you could provide the vcl because without that it?s really hard to know what happens...
?
Best regards,
Viktor
?
?
From: varnish-misc-bounces+viktor.gunnarson=ericsson.com at varnish-cache.org [mailto:varnish-misc-bounces+viktor.gunnarson=ericsson.com at varnish-cache.org] On Behalf Of Jennings III, Raymond
Sent: den 16 januari 2015 16:07
To: Dridi Boukelmoune
Cc: varnish-misc
Subject: Re: Cannot figure out why I am getting a miss on a static URL
?
varnishncsa clearly shows hits and misses within a 24 hour period despite clearly setting the TTL to 24 hours.? Varnishlog clearly shows the TTL being set to 24 hours.? I have NO lru_nuked objects as clearly shown by the varnish stats.
?
I'm not sure what is vague.? I am setting an object for a TTL of 24 hours yet during that 24 hour period it is still getting miss requests.? The URLs in question have no GET parameters.? I am hashing only on the URL and not the hostname.
?
So what does this tell me?? It tells me that my cache size is big enough.
?
My guess is there is some type of contention from locking of some type given other requests coming through.? There are quite a few PUT and POST requests that pass through this Varnish server.
?
Ummmm, I hardly think that "anyone can't help me." ?I think most definitely "someone" that has some insight about the internals could shed some light on this problem.
?
?


Raymond Jennings III
nytimes.com
Office:?212.556.7786
iPhone:?914.330.5074
E-mail:?Raymond.Jennings at nytimes.com
FaceTime:?Raymond.Jennings at nytimes.com
?
On Fri, Jan 16, 2015 at 10:01 AM, Dridi Boukelmoune <dridi.boukelmoune at zenika.com> wrote:
Hi Raymond,

Have you looked at the logs?

Your description of the issue is too vague, I don't think anyone can
help you with just that.

Cheers,
Dridi

On Thu, Jan 15, 2015 at 3:36 PM, Jennings III, Raymond
<raymond.jennings at nytimes.com> wrote:
> I have a static URL that I set a TTL for 24h but yet every so many minutes I
> get a "miss" (as shown in my varnishncsa.)
>
> I purge this URL every 24 hours.
>
> 1 - The URL never has any parameters.
> 2 - I do not hash over the hostname only the URI
> 3 - Nobody is issuing any PURGE or BAN requests
> 4 - n_lru_nuked is always 0
> 5 - There are no 500 or 400 responses
>
> What is causing this URL to get a miss when everything I see looks good?? If
> the cache size has been exceeded I would have expected to see the nuked
> value be something other than 0.
>
> vcl_fetch is basically just:
>
> set beresp.ttl = 24h;
> return(deliver);
>
> Any reason why Varnish seems to be taking it upon itself to cause this miss?
> In 24 hours I get about 12000 requests and out of those 120 misses and I
> should see 2 miss requests at most.
>
> The backends are healthy.
>
> This is varnish-3.0.6-1.el6.x86_64
>
> Thanks.
>
>
>
> Raymond Jennings III
> nytimes.com
> Office: 212.556.7786
> iPhone: 914.330.5074
> E-mail: Raymond.Jennings at nytimes.com
> FaceTime: Raymond.Jennings at nytimes.com
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
?


From raymond.jennings at nytimes.com  Tue Jan 20 15:57:17 2015
From: raymond.jennings at nytimes.com (Jennings III, Raymond)
Date: Tue, 20 Jan 2015 10:57:17 -0500
Subject: Trying to get Varnish to create a core dump
Message-ID: <CADFjDbNmOeJ1+dV6ffxfMqiCfUV2DQD6Sf=bnZemWk9aqmb33Q@mail.gmail.com>

I am using the production rpm of varnish (not building via the varnish
source) but my VCL has a lot of inline C and I also load a dynamic loadable
library in vcl_init which does has full symbol table, debug on.  I keep
getting segmentation faults but cannot get a core file from Varnish to save
my life.

/var/log/kern clearly shows varnish segfaulting

I have tried:

ulimit -c unlimited
echo 1 > /proc/sys/kernel/core_uses_pid
echo 2 > /proc/sys/fs/suid_dumpable
mkdir /mnt/cores
chmod 777 /mnt/cores
echo /mnt/cores/core > /proc/sys/kernel/core_pattern

Anything else that I am overlooking?

Thanks



*nytimes.com <http://nytimes.com/>*
*Office: 212.556.7786 <212-556-7786>*

*iPhone: 914.330.5074 <914-330-5074>E-mail: Raymond.Jennings at nytimes.com
<Raymond.Jennings at nytimes.com>FaceTime: Raymond.Jennings at nytimes.com
<Raymond.Jennings at nytimes.com>*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150120/e48593d5/attachment.html>

From info at massivescale.net  Tue Jan 20 17:18:43 2015
From: info at massivescale.net (Andrzej Godziuk)
Date: Tue, 20 Jan 2015 18:18:43 +0100
Subject: Trying to get Varnish to create a core dump
In-Reply-To: <CADFjDbNmOeJ1+dV6ffxfMqiCfUV2DQD6Sf=bnZemWk9aqmb33Q@mail.gmail.com>
References: <CADFjDbNmOeJ1+dV6ffxfMqiCfUV2DQD6Sf=bnZemWk9aqmb33Q@mail.gmail.com>
Message-ID: <20150120181843.0bbc0c04@gdr-desktop.gdr.name>

On Tue, 20 Jan 2015 10:57:17 -0500
"Jennings III, Raymond" <raymond.jennings at nytimes.com> wrote:

> I am using the production rpm of varnish (not building via the varnish
> source) but my VCL has a lot of inline C and I also load a dynamic
> loadable library in vcl_init which does has full symbol table, debug
> on.  I keep getting segmentation faults but cannot get a core file
> from Varnish to save my life.
> 
> /var/log/kern clearly shows varnish segfaulting
> 
> I have tried:
> 
> ulimit -c unlimited
> echo 1 > /proc/sys/kernel/core_uses_pid
> echo 2 > /proc/sys/fs/suid_dumpable
> mkdir /mnt/cores
> chmod 777 /mnt/cores
> echo /mnt/cores/core > /proc/sys/kernel/core_pattern
> 
> Anything else that I am overlooking?

Did you put "ulimit -c unlimited" in Varnish init script?

Also, if you're on Ubuntu, try the following:
rcapparmor stop

-- 
Andrzej Godziuk
http://massivescale.net/



From raymond.jennings at nytimes.com  Tue Jan 20 18:36:52 2015
From: raymond.jennings at nytimes.com (Jennings III, Raymond)
Date: Tue, 20 Jan 2015 13:36:52 -0500
Subject: Trying to get Varnish to create a core dump
In-Reply-To: <20150120181843.0bbc0c04@gdr-desktop.gdr.name>
References: <CADFjDbNmOeJ1+dV6ffxfMqiCfUV2DQD6Sf=bnZemWk9aqmb33Q@mail.gmail.com>
 <20150120181843.0bbc0c04@gdr-desktop.gdr.name>
Message-ID: <CADFjDbNN-Tc6EYBfptLQhch4vr7GYy-TQNXvZcN8+HD5St9cQA@mail.gmail.com>

Yes, I do have that in the init.d script

I am on centos.

Thanks.

Raymond Jennings III
*nytimes.com <http://nytimes.com/>*
*Office: 212.556.7786 <212-556-7786>*

*iPhone: 914.330.5074 <914-330-5074>E-mail: Raymond.Jennings at nytimes.com
<Raymond.Jennings at nytimes.com>FaceTime: Raymond.Jennings at nytimes.com
<Raymond.Jennings at nytimes.com>*

On Tue, Jan 20, 2015 at 12:18 PM, Andrzej Godziuk <info at massivescale.net>
wrote:

> On Tue, 20 Jan 2015 10:57:17 -0500
> "Jennings III, Raymond" <raymond.jennings at nytimes.com> wrote:
>
> > I am using the production rpm of varnish (not building via the varnish
> > source) but my VCL has a lot of inline C and I also load a dynamic
> > loadable library in vcl_init which does has full symbol table, debug
> > on.  I keep getting segmentation faults but cannot get a core file
> > from Varnish to save my life.
> >
> > /var/log/kern clearly shows varnish segfaulting
> >
> > I have tried:
> >
> > ulimit -c unlimited
> > echo 1 > /proc/sys/kernel/core_uses_pid
> > echo 2 > /proc/sys/fs/suid_dumpable
> > mkdir /mnt/cores
> > chmod 777 /mnt/cores
> > echo /mnt/cores/core > /proc/sys/kernel/core_pattern
> >
> > Anything else that I am overlooking?
>
> Did you put "ulimit -c unlimited" in Varnish init script?
>
> Also, if you're on Ubuntu, try the following:
> rcapparmor stop
>
> --
> Andrzej Godziuk
> http://massivescale.net/
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150120/dbf2dfba/attachment.html>

From raymond.jennings at nytimes.com  Fri Jan 23 17:36:17 2015
From: raymond.jennings at nytimes.com (Jennings III, Raymond)
Date: Fri, 23 Jan 2015 12:36:17 -0500
Subject: Small (hopefully helpful) script to monitor Varnish's uptime
Message-ID: <CADFjDbOSSy-nH_iVOaXHj7+R2BbFdk69RewzzYXefByVxh9c1A@mail.gmail.com>

I was having a lot of segmenation faults in Varnish (due to our own
mistakes) but unless you are monitoring syslog or /var/log/kern you might
not ever know it's happening given that the child process restarts very
quickly so I wrote this little cron job:


You first have to bootstrap this right after you first install and start
varnish with:

/usr/bin/varnishstat -1 | /bin/grep 'uptime' | /bin/awk '{ print $2}'
>/var/nyt/logs/varnish/varnish_uptime




The cron job is:


#!/bin/bash

new_varnish_uptime=`/usr/bin/varnishstat -1 | /bin/grep 'uptime' | /bin/awk
'{ print $2}'`
old_varnish_uptime=`cat /var/nyt/logs/varnish/varnish_uptime`

delta_time=$(echo "$new_varnish_uptime - $old_varnish_uptime" | /usr/bin/bc)

if [[ $delta_time -lt 0 ]]; then
   echo "Uh oh!!!  Varnish was restarted.  It's uptime is now
$new_varnish_uptime and it used to be $old_varnish_uptime." | /bin/mail -s
"Varnish Server Restarted!" "Raymond.Jennings at nytimes.com"
fi

echo $new_varnish_uptime >/var/nyt/logs/varnish/varnish_uptime



There is a race condition if Varnish is restarting faster than the
frequency of the cron job running.

Raymond Jennings III
*nytimes.com <http://nytimes.com/>*
*Office: 212.556.7786 <212-556-7786>*

*iPhone: 914.330.5074 <914-330-5074>E-mail: Raymond.Jennings at nytimes.com
<Raymond.Jennings at nytimes.com>FaceTime: Raymond.Jennings at nytimes.com
<Raymond.Jennings at nytimes.com>*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150123/795acfcd/attachment.html>

From jnerin at gmail.com  Fri Jan 23 18:05:33 2015
From: jnerin at gmail.com (Jorge)
Date: Fri, 23 Jan 2015 18:05:33 +0000
Subject: Small (hopefully helpful) script to monitor Varnish's uptime
References: <CADFjDbOSSy-nH_iVOaXHj7+R2BbFdk69RewzzYXefByVxh9c1A@mail.gmail.com>
Message-ID: <CAF-dNMZksSurdXdFMxKVjQt_wRrYG3Li=bnP55=q5mrKea7Z2g@mail.gmail.com>

Hi Jennings,

You could avoid the race by also checking the elapsed seconds
since varnish_uptime file was last written > new_varnish_uptime:

if [ $(($(date "+%s")-$(stat --printf="%Z" varnish_uptime))) -gt
$new_varnish_uptime ]; then

Note: completely untested code. And you can do integer arithmetic in bash
without spawning bc with $(( ))

El Fri Jan 23 2015 at 18:39:48, Jennings III, Raymond (<
raymond.jennings at nytimes.com>) escribi?:

> I was having a lot of segmenation faults in Varnish (due to our own
> mistakes) but unless you are monitoring syslog or /var/log/kern you might
> not ever know it's happening given that the child process restarts very
> quickly so I wrote this little cron job:
>
>
> You first have to bootstrap this right after you first install and start
> varnish with:
>
> /usr/bin/varnishstat -1 | /bin/grep 'uptime' | /bin/awk '{ print $2}'
> >/var/nyt/logs/varnish/varnish_uptime
>
>
>
>
> The cron job is:
>
>
> #!/bin/bash
>
> new_varnish_uptime=`/usr/bin/varnishstat -1 | /bin/grep 'uptime' |
> /bin/awk '{ print $2}'`
> old_varnish_uptime=`cat /var/nyt/logs/varnish/varnish_uptime`
>
> delta_time=$(echo "$new_varnish_uptime - $old_varnish_uptime" |
> /usr/bin/bc)
>
> if [[ $delta_time -lt 0 ]]; then
>    echo "Uh oh!!!  Varnish was restarted.  It's uptime is now
> $new_varnish_uptime and it used to be $old_varnish_uptime." | /bin/mail -s
> "Varnish Server Restarted!" "Raymond.Jennings at nytimes.com"
> fi
>
> echo $new_varnish_uptime >/var/nyt/logs/varnish/varnish_uptime
>
>
>
> There is a race condition if Varnish is restarting faster than the
> frequency of the cron job running.
>
> Raymond Jennings III
> *nytimes.com <http://nytimes.com/>*
> *Office: 212.556.7786 <212-556-7786>*
>
> *iPhone: 914.330.5074 <914-330-5074>E-mail: Raymond.Jennings at nytimes.com
> <Raymond.Jennings at nytimes.com>FaceTime: Raymond.Jennings at nytimes.com
> <Raymond.Jennings at nytimes.com>*
>  _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150123/a48a433a/attachment.html>

From em at nuel.is  Fri Jan 23 18:06:43 2015
From: em at nuel.is (Emanuelis)
Date: Fri, 23 Jan 2015 20:06:43 +0200
Subject: Small (hopefully helpful) script to monitor Varnish's uptime
In-Reply-To: <CADFjDbOSSy-nH_iVOaXHj7+R2BbFdk69RewzzYXefByVxh9c1A@mail.gmail.com>
References: <CADFjDbOSSy-nH_iVOaXHj7+R2BbFdk69RewzzYXefByVxh9c1A@mail.gmail.com>
Message-ID: <CADcsm30VRRxgHadJoshwS0yhsyapMZ=dyQPwnzJ6yaLYxgVv4Q@mail.gmail.com>

I like such info in graphics and without spamming my inbox. Cacti can
draw a nice graph with uptime (see attachment).
-------------- next part --------------
A non-text attachment was scrubbed...
Name: varnish_uptime.png
Type: image/png
Size: 28857 bytes
Desc: not available
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150123/dd4f070a/attachment.png>

From dridi.boukelmoune at zenika.com  Fri Jan 23 18:18:38 2015
From: dridi.boukelmoune at zenika.com (Dridi Boukelmoune)
Date: Fri, 23 Jan 2015 19:18:38 +0100
Subject: Small (hopefully helpful) script to monitor Varnish's uptime
In-Reply-To: <CADFjDbOSSy-nH_iVOaXHj7+R2BbFdk69RewzzYXefByVxh9c1A@mail.gmail.com>
References: <CADFjDbOSSy-nH_iVOaXHj7+R2BbFdk69RewzzYXefByVxh9c1A@mail.gmail.com>
Message-ID: <CABtDKm4HOyPHskwcfsE_UxqF4zYbZ7afKk_UMZ3HDoM99ssjdw@mail.gmail.com>

On Fri, Jan 23, 2015 at 6:36 PM, Jennings III, Raymond
<raymond.jennings at nytimes.com> wrote:
> /usr/bin/varnishstat -1 | /bin/grep 'uptime' | /bin/awk '{ print $2}'
>>/var/nyt/logs/varnish/varnish_uptime

You don't need grep!

varnishstat -1 -f uptime | awk '{print $2}'



From paul.wolstenholme at cbc.ca  Fri Jan 23 19:29:24 2015
From: paul.wolstenholme at cbc.ca (Paul Wolstenholme)
Date: Fri, 23 Jan 2015 11:29:24 -0800
Subject: varnish 4.0 libvmod-timers
Message-ID: <CAND1e16w+5GWRQ21DrCWnB2anaWx4qZ+M7jOUO+0EOK7CRNr3w@mail.gmail.com>

I'm in the process of starting the migration from 3.0 to 4.0. Most of the
VMODS we use seem to pass the varnishtests except libvmod-timers:

**** v1    0.1 CLI RX| Message from VCC-compiler:\n
**** v1    0.1 CLI RX| Not running as root, no priv-sep\n
**** v1    0.1 CLI RX| Operator > not possible on TIME\n
**** v1    0.1 CLI RX| ('input' Line 10 Pos 32)\n
**** v1    0.1 CLI RX|         if( timers.req_start() > 0 ) {\n
**** v1    0.1 CLI RX| -------------------------------#------\n
**** v1    0.1 CLI RX| \n
**** v1    0.1 CLI RX| Running VCC-compiler failed, exited with 2\n
**** v1    0.1 CLI RX| \n
**** v1    0.1 CLI RX| VCL compilation failed
---- v1    0.1 FAIL VCL does not compile


It looks like some 4.0 work has been done by pushrax (
https://github.com/pushrax/libvmod-timers) so I have tried that instead of
https://github.com/jib/libvmod-timers.

I'm just wondering if anyone is using timers successfully on 4.0. We like
to push stuff to graphite and would feel a bit uncomfortable without any
graphing/instrumentation.

If someone has an alternative method of instrumentation I'd like to hear
about that as well. I checked out
https://github.com/pbruna/Varnish-Agent-Dashboard. It seems broken for
vagent2 4.0.

Cheers,
Paul



**** v1    0.1 CLI RX| Message from VCC-compiler:\n
**** v1    0.1 CLI RX| Not running as root, no priv-sep\n
**** v1    0.1 CLI RX| Operator > not possible on TIME\n
**** v1    0.1 CLI RX| ('input' Line 10 Pos 32)\n
**** v1    0.1 CLI RX|         if( timers.req_start() > 0 ) {\n
**** v1    0.1 CLI RX| -------------------------------#------\n
**** v1    0.1 CLI RX| \n
**** v1    0.1 CLI RX| Running VCC-compiler failed, exited with 2\n
**** v1    0.1 CLI RX| \n
**** v1    0.1 CLI RX| VCL compilation failed
---- v1    0.1 FAIL VCL does not compile


-- 

Paul Wolstenholme
CBC Music - System Administrator
CBC  Vancouver
604.662.6632
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150123/b91990ce/attachment.html>

From infos at opendoc.net  Tue Jan 27 07:38:55 2015
From: infos at opendoc.net (Alexandre)
Date: Tue, 27 Jan 2015 08:38:55 +0100
Subject: best practice with varnish
Message-ID: <54C7408F.4020602@opendoc.net>

Hello everyone,

We currently have a single config file containing many regex and 
condition. Do you think it would be better to split into several small 
conf file for performance and flexibility?

Thank you.

Alexandre



From infos at opendoc.net  Tue Jan 27 08:10:10 2015
From: infos at opendoc.net (Alexandre)
Date: Tue, 27 Jan 2015 09:10:10 +0100
Subject: Varnish Clustering
In-Reply-To: <2772861.esxOgumPm0@notenbuch.haukebruno.de>
References: <CAJpkvFgj5wD37ptMwFtSq+TYfqFGw9vPdxvx5Nd6naDBi6-qAw@mail.gmail.com>
 <3EE228D3-4022-4F4D-B642-7F614015F924@ultra-secure.de>
 <CAJpkvFiyJqqstzCs4_rjQ3wmX-8kRoHnvEuCM6kxh-LRPqe8og@mail.gmail.com>
 <2772861.esxOgumPm0@notenbuch.haukebruno.de>
Message-ID: <54C747E2.6030205@opendoc.net>

Hi Hauke, using pacemaker, one node is active at a time? This is better 
than using a load balancer?

Sorry for my newbie question.

Alex.

On 02/01/15 15:24, Hauke wrote:
> Hey,
>
> I personally use varnish in a simple pacemaker/corosync setting for high
> availability.
>
> cheers,
> hauke
>



From redalert.commander at gmail.com  Tue Jan 27 09:50:05 2015
From: redalert.commander at gmail.com (Redalert Commander)
Date: Tue, 27 Jan 2015 10:50:05 +0100
Subject: See the offloading percentage of a varnish installation?
In-Reply-To: <CAMDu+mNN9HBwcixQXfbtASS-Wheq4YL8-dzSa4QshLShbORTvw@mail.gmail.com>
References: <CAMDu+mNN9HBwcixQXfbtASS-Wheq4YL8-dzSa4QshLShbORTvw@mail.gmail.com>
Message-ID: <CAMDu+mMkWvFRNu-i=kDkNbpgr2naT2LK0xPCibDRk5HCZqD=iQ@mail.gmail.com>

Hi,

I'm a system engineer with limited knowledge about varnish, I do know
it is used for caching.
One of our clients uses varnish on their website, but I'd like to know
the offload percentage.
How much traffic reaches the Apache server behind it, and how much is
handled only by Varnish's cache(s)? Is there a relative simple way to
detect/query this information?

We run varnish 3.0.x.

Regards,
Steven



From perbu at varnish-software.com  Tue Jan 27 09:56:10 2015
From: perbu at varnish-software.com (Per Buer)
Date: Tue, 27 Jan 2015 10:56:10 +0100
Subject: See the offloading percentage of a varnish installation?
In-Reply-To: <CAMDu+mMkWvFRNu-i=kDkNbpgr2naT2LK0xPCibDRk5HCZqD=iQ@mail.gmail.com>
References: <CAMDu+mNN9HBwcixQXfbtASS-Wheq4YL8-dzSa4QshLShbORTvw@mail.gmail.com>
 <CAMDu+mMkWvFRNu-i=kDkNbpgr2naT2LK0xPCibDRk5HCZqD=iQ@mail.gmail.com>
Message-ID: <CAOXZevAXY6G56wOd62XeWjQHr2bAyt-BEZbwL0ZQ0h-R_f0noQ@mail.gmail.com>

Hi Steven.

Just run varnishstat. It will show you the hitrate at the top. This is a
running average. To calculate the hitrate for the whole uptime look at the
cache_hits and cache_miss counters.

On Tue, Jan 27, 2015 at 10:50 AM, Redalert Commander <
redalert.commander at gmail.com> wrote:

> Hi,
>
> I'm a system engineer with limited knowledge about varnish, I do know
> it is used for caching.
> One of our clients uses varnish on their website, but I'd like to know
> the offload percentage.
> How much traffic reaches the Apache server behind it, and how much is
> handled only by Varnish's cache(s)? Is there a relative simple way to
> detect/query this information?
>
> We run varnish 3.0.x.
>
> Regards,
> Steven
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>



-- 
*Per Buer*
CTO | Varnish Software AS
Cell: +47 95839117
We Make Websites Fly!
www.varnish-software.com
[image: Register now]
<http://info.varnish-software.com/varnish-summits-autumn-2014-registration>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150127/2d2d0588/attachment.html>

From redalert.commander at gmail.com  Tue Jan 27 10:35:39 2015
From: redalert.commander at gmail.com (Redalert Commander)
Date: Tue, 27 Jan 2015 11:35:39 +0100
Subject: [solved] Re: See the offloading percentage of a varnish installation?
Message-ID: <CAMDu+mOPQYhJQMN8L4j2QRg2=BmQh=r6OyXw22QDmo6AEPo+Dw@mail.gmail.com>

Hi Per,

Thanks, that was very easy.
It shows me an offloading of roughly 45%, just looking at cache hits and misses.

Regards,
Steven

2015-01-27 10:56 GMT+01:00 Per Buer <perbu at varnish-software.com>:
> Hi Steven.
>
> Just run varnishstat. It will show you the hitrate at the top. This is a
> running average. To calculate the hitrate for the whole uptime look at the
> cache_hits and cache_miss counters.
>
> On Tue, Jan 27, 2015 at 10:50 AM, Redalert Commander
> <redalert.commander at gmail.com> wrote:
>>
>> Hi,
>>
>> I'm a system engineer with limited knowledge about varnish, I do know
>> it is used for caching.
>> One of our clients uses varnish on their website, but I'd like to know
>> the offload percentage.
>> How much traffic reaches the Apache server behind it, and how much is
>> handled only by Varnish's cache(s)? Is there a relative simple way to
>> detect/query this information?
>>
>> We run varnish 3.0.x.
>>
>> Regards,
>> Steven
>>
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at varnish-cache.org
>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
>
>
>
> --
> Per Buer
> CTO | Varnish Software AS
> Cell: +47 95839117
> We Make Websites Fly!
> www.varnish-software.com
>



From infos at opendoc.net  Wed Jan 28 20:04:09 2015
From: infos at opendoc.net (Alexandre)
Date: Wed, 28 Jan 2015 21:04:09 +0100
Subject: best practice with varnish
In-Reply-To: <54C7408F.4020602@opendoc.net>
References: <54C7408F.4020602@opendoc.net>
Message-ID: <54C940B9.4010901@opendoc.net>

Hello,

anyone have an idea?

Alex.

On 27/01/15 08:38, Alexandre wrote:
> Hello everyone,
>
> We currently have a single config file containing many regex and
> condition. Do you think it would be better to split into several small
> conf file for performance and flexibility?
>
> Thank you.
>
> Alexandre
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc



From thierry.magnien at sfr.com  Wed Jan 28 20:26:14 2015
From: thierry.magnien at sfr.com (MAGNIEN, Thierry)
Date: Wed, 28 Jan 2015 20:26:14 +0000
Subject: best practice with varnish
In-Reply-To: <54C940B9.4010901@opendoc.net>
References: <54C7408F.4020602@opendoc.net>,<54C940B9.4010901@opendoc.net>
Message-ID: <8uqnl725xjew7i9i519bp7sh.1422476770723@email.android.com>

Hi,

Don't take performance into consideration: your multiple files will be combined and compiled to C as if it were a single one.

Flexibility depends on how you prefer. I used both and I personally prefer a single large file, but that's my own view.

Thierry


-------- Message d'origine --------
De : Alexandre
Date :28/01/2015 21:07 (GMT+01:00)
A : varnish-misc at varnish-cache.org
Objet : Re: best practice with varnish

Hello,

anyone have an idea?

Alex.

On 27/01/15 08:38, Alexandre wrote:
> Hello everyone,
>
> We currently have a single config file containing many regex and
> condition. Do you think it would be better to split into several small
> conf file for performance and flexibility?
>
> Thank you.
>
> Alexandre
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

_______________________________________________
varnish-misc mailing list
varnish-misc at varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150128/3a12c20d/attachment.html>

From pprocacci at datapipe.com  Wed Jan 28 20:32:36 2015
From: pprocacci at datapipe.com (Paul A. Procacci)
Date: Wed, 28 Jan 2015 15:32:36 -0500
Subject: best practice with varnish
In-Reply-To: <54C940B9.4010901@opendoc.net>
References: <54C7408F.4020602@opendoc.net> <54C940B9.4010901@opendoc.net>
Message-ID: <20150128203235.GA5622@freenx.int.smq.datapipe.net>

> >We currently have a single config file containing many regex and
> >condition. Do you think it would be better to split into several small
> >conf file for performance and flexibility?

VCL gets compiled into an object file and then loaded into a running
varnishd process.  Splitting the file into seperate files and what have
you will have 0 impact on performance.

If however you feel that splitting the vcl source into multiple files
helps you maintain it, then that's a good reason to do so.

It's up to you.

~Paul



From max.clark at gmail.com  Wed Jan 28 20:56:01 2015
From: max.clark at gmail.com (Max Clark)
Date: Wed, 28 Jan 2015 12:56:01 -0800
Subject: best practice with varnish
In-Reply-To: <54C940B9.4010901@opendoc.net>
References: <54C7408F.4020602@opendoc.net>
	<54C940B9.4010901@opendoc.net>
Message-ID: <CAAaeqWfE5Jd+_2c2Wr8bgqfpAbN+S=u3RUi6PAReAJNfY+nYjg@mail.gmail.com>

I switched one install to multiple files which made editing easier.
Tracking down errors in the VCL when loading the configuration was
difficult. After a while merged everything back to a single file. HTH

On Wed, Jan 28, 2015 at 12:04 PM, Alexandre <infos at opendoc.net> wrote:

> Hello,
>
> anyone have an idea?
>
> Alex.
>
> On 27/01/15 08:38, Alexandre wrote:
>
>> Hello everyone,
>>
>> We currently have a single config file containing many regex and
>> condition. Do you think it would be better to split into several small
>> conf file for performance and flexibility?
>>
>> Thank you.
>>
>> Alexandre
>>
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at varnish-cache.org
>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150128/2e3d505d/attachment.html>

From jan at architechs.eu  Wed Jan 28 21:34:19 2015
From: jan at architechs.eu (=?utf-8?B?SmFuLUFhZ2UgRnJ5ZGVuYsO4LUJydXZvbGw=?=)
Date: Wed, 28 Jan 2015 21:34:19 +0000
Subject: best practice with varnish
In-Reply-To: <CAAaeqWfE5Jd+_2c2Wr8bgqfpAbN+S=u3RUi6PAReAJNfY+nYjg@mail.gmail.com>
References: <54C7408F.4020602@opendoc.net>	<54C940B9.4010901@opendoc.net>
 <CAAaeqWfE5Jd+_2c2Wr8bgqfpAbN+S=u3RUi6PAReAJNfY+nYjg@mail.gmail.com>
Message-ID: <AM3PR06MB226389709791C3E08A77060B9330@AM3PR06MB226.eurprd06.prod.outlook.com>

Hi,

We've split relatively static stuff like backends, health checks, etc into separate files in order to reduce the noise in the main VCL. The logic remains in a single file, though.

Best regards
Jan


From dridi.boukelmoune at zenika.com  Wed Jan 28 22:06:46 2015
From: dridi.boukelmoune at zenika.com (Dridi Boukelmoune)
Date: Wed, 28 Jan 2015 23:06:46 +0100
Subject: best practice with varnish
In-Reply-To: <AM3PR06MB226389709791C3E08A77060B9330@AM3PR06MB226.eurprd06.prod.outlook.com>
References: <54C7408F.4020602@opendoc.net> <54C940B9.4010901@opendoc.net>
 <CAAaeqWfE5Jd+_2c2Wr8bgqfpAbN+S=u3RUi6PAReAJNfY+nYjg@mail.gmail.com>
 <AM3PR06MB226389709791C3E08A77060B9330@AM3PR06MB226.eurprd06.prod.outlook.com>
Message-ID: <CABtDKm4c8kEJwXid-ouVd-ggeUu=x4Xfw-rSKMw2fFYFvkMwjA@mail.gmail.com>

Hi,

I usually move environment-specific stuff to a specific file, and keep
the cache policy in a single file. By environment, I mean dev,
staging, production etc.

What usually changes between environments is backends/directors, ACLs
and backend selection.

Cheers,
Dridi

On Wed, Jan 28, 2015 at 10:34 PM, Jan-Aage Frydenb?-Bruvoll
<jan at architechs.eu> wrote:
> Hi,
>
> We've split relatively static stuff like backends, health checks, etc into separate files in order to reduce the noise in the main VCL. The logic remains in a single file, though.
>
> Best regards
> Jan
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc



From max.clark at gmail.com  Thu Jan 29 15:35:49 2015
From: max.clark at gmail.com (Max Clark)
Date: Thu, 29 Jan 2015 07:35:49 -0800
Subject: GeoIP Detection and Country Specific Content
Message-ID: <CAAaeqWfdBJzhHTTj-ijLPqFiyH6rTk035ocR6qR_JCdKfFVbYA@mail.gmail.com>

Hello,

We have been working to enable region specific content for our web site.
The basic mechanism is already in place, specifically:

When a user visits the site and a cookie is not present the application
server will compare their IP address with the GeoIP database. Based on the
result the application server will generate the proper content for the user
(different logo images, text, etc...) and return this with a cookie
specifying the edition for future requests bypassing the GeoIP lookup. The
cookie will also allow the user to select a different region than what
their IP resolves to.

Again this is working in the application, and Varnish was extended to match
on the cookie value (named "edition") and to use the edition value as part
of the hash data so there is a unique cache for the different regions.

Here's the problem:

When a user visits the site for the first time they do not have a cookie in
the request. So Varnish has a blank value for the edition as part of the
hash data. However this request when it hits the application server does
resolve to a specific edition and return the region content on the reply.
This is being cached by Varnish and used for future requests by new users
with no cookie.

Put simply, if the page is not in cache, and a browser from the UK does not
have the edition cookie a page with UK content is generated and cached in
Varnish. If a browser from the US goes to the same page, Varnish will serve
the US user the UK cached page because this is what was cached based on the
hash data.

Without turning off caching for cookieless requests - how do we resolve
this?

Thanks in advance,
Max
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150129/6620227c/attachment.html>

From info at massivescale.net  Thu Jan 29 09:46:15 2015
From: info at massivescale.net (Andrzej Godziuk)
Date: Thu, 29 Jan 2015 10:46:15 +0100
Subject: Is there a recent security problem?
Message-ID: <20150129104615.56f15f4f@gdr-desktop.gdr.name>

Hello,

I'm administrating a few servers, some of them running Varnish, set up
and running in different companies by different people. 

In the recent days, unrelated servers which have been runing stable for
many months suddenly started not accepting connections on the Varnish
port. No system metrics such as accept() backlog, number of open files
or RAM, system load etc are on an unusually high level.

Restarting Varnish helps, but the problem returns in a few hours or
days, depending on traffic. Most systems have the Varnish port 80
exposed to the internet, one is behind a firewall and another is behind
Pound.

The only thing these systems have in common is that they're running
Varnish 3.0.6 on CentOS 6 - and that they have a mysterious problem
which is only solved by restarting Varnish. Configuration has not been
changed in any of them recently, except for standard CentOS updates (not
on all of them).

One of the systems has been running for 2 years without any
intervention and today, all of the sudden, I got 503 from Pound (which
was in front of Varnish). This is highly irregular.

Is somebody having problems like this recently? The only explanation I
have is a security problem that someone found and is exploiting. This
or sun spots or increased radiation above Europe :)

Andrzej Godziuk



From thierry.magnien at sfr.com  Thu Jan 29 16:46:54 2015
From: thierry.magnien at sfr.com (MAGNIEN, Thierry)
Date: Thu, 29 Jan 2015 16:46:54 +0000
Subject: GeoIP Detection and Country Specific Content
In-Reply-To: <CAAaeqWfdBJzhHTTj-ijLPqFiyH6rTk035ocR6qR_JCdKfFVbYA@mail.gmail.com>
References: <CAAaeqWfdBJzhHTTj-ijLPqFiyH6rTk035ocR6qR_JCdKfFVbYA@mail.gmail.com>
Message-ID: <jn08u759v166oxic5scohj4m.1422549906676@email.android.com>

Hi,

Best would be to move geoip lookup into varnish. That way you can set req.http.cookie value to the correct country before varnish looks up the object in its cache.

Regards,
Thierry


-------- Message d'origine --------
De : Max Clark
Date :29/01/2015 17:40 (GMT+01:00)
A : varnish-misc at varnish-cache.org
Objet : GeoIP Detection and Country Specific Content

Hello,

We have been working to enable region specific content for our web site. The basic mechanism is already in place, specifically:

When a user visits the site and a cookie is not present the application server will compare their IP address with the GeoIP database. Based on the result the application server will generate the proper content for the user (different logo images, text, etc...) and return this with a cookie specifying the edition for future requests bypassing the GeoIP lookup. The cookie will also allow the user to select a different region than what their IP resolves to.

Again this is working in the application, and Varnish was extended to match on the cookie value (named "edition") and to use the edition value as part of the hash data so there is a unique cache for the different regions.

Here's the problem:

When a user visits the site for the first time they do not have a cookie in the request. So Varnish has a blank value for the edition as part of the hash data. However this request when it hits the application server does resolve to a specific edition and return the region content on the reply. This is being cached by Varnish and used for future requests by new users with no cookie.

Put simply, if the page is not in cache, and a browser from the UK does not have the edition cookie a page with UK content is generated and cached in Varnish. If a browser from the US goes to the same page, Varnish will serve the US user the UK cached page because this is what was cached based on the hash data.

Without turning off caching for cookieless requests - how do we resolve this?

Thanks in advance,
Max
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150129/cb0d73eb/attachment.html>

From guillaume.quintard at smartjog.com  Thu Jan 29 16:46:14 2015
From: guillaume.quintard at smartjog.com (Guillaume Quintard)
Date: Thu, 29 Jan 2015 17:46:14 +0100
Subject: GeoIP Detection and Country Specific Content
In-Reply-To: <CAAaeqWfdBJzhHTTj-ijLPqFiyH6rTk035ocR6qR_JCdKfFVbYA@mail.gmail.com>
References: <CAAaeqWfdBJzhHTTj-ijLPqFiyH6rTk035ocR6qR_JCdKfFVbYA@mail.gmail.com>
Message-ID: <54CA63D6.4030708@smartjog.com>

On 01/29/2015 04:35 PM, Max Clark wrote:
> Hello,
>
> We have been working to enable region specific content for our web
> site. The basic mechanism is already in place, specifically:
>
> When a user visits the site and a cookie is not present the
> application server will compare their IP address with the GeoIP
> database. Based on the result the application server will generate the
> proper content for the user (different logo images, text, etc...) and
> return this with a cookie specifying the edition for future requests
> bypassing the GeoIP lookup. The cookie will also allow the user to
> select a different region than what their IP resolves to.
>
> Again this is working in the application, and Varnish was extended to
> match on the cookie value (named "edition") and to use the edition
> value as part of the hash data so there is a unique cache for the
> different regions.
>
> Here's the problem:
>
> When a user visits the site for the first time they do not have a
> cookie in the request. So Varnish has a blank value for the edition as
> part of the hash data. However this request when it hits the
> application server does resolve to a specific edition and return the
> region content on the reply. This is being cached by Varnish and used
> for future requests by new users with no cookie.
>
> Put simply, if the page is not in cache, and a browser from the
> UK does not have the edition cookie a page with UK content is
> generated and cached in Varnish. If a browser from the US goes to the
> same page, Varnish will serve the US user the UK cached page because
> this is what was cached based on the hash data.
>
> Without turning off caching for cookieless requests - how do we
> resolve this?
>

I may not have understood exactly what you need, but can't you hash the
country code instead of the cookie ?

-- 
Guillaume Quintard

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150129/a943f2cd/attachment.html>

From brettgfitzgerald at gmail.com  Thu Jan 29 18:24:10 2015
From: brettgfitzgerald at gmail.com (Brett Fitzgerald)
Date: Thu, 29 Jan 2015 18:24:10 +0000
Subject: GeoIP Detection and Country Specific Content
References: <CAAaeqWfdBJzhHTTj-ijLPqFiyH6rTk035ocR6qR_JCdKfFVbYA@mail.gmail.com>
 <54CA63D6.4030708@smartjog.com>
Message-ID: <CALUXoqLnL3k_pw6aaPJS0iX9EMaqQ=Q5T35kDvAJJPunWc=JSA@mail.gmail.com>

I recently went through a very similar situation. We solved it by using a
vmod to do a country (and timezone, in our case) lookup, set the values in
an http header, and then vary on that header.

On Thu Jan 29 2015 at 11:54:12 AM Guillaume Quintard <
guillaume.quintard at smartjog.com> wrote:

> On 01/29/2015 04:35 PM, Max Clark wrote:
>
> Hello,
>
>  We have been working to enable region specific content for our web site.
> The basic mechanism is already in place, specifically:
>
>  When a user visits the site and a cookie is not present the application
> server will compare their IP address with the GeoIP database. Based on the
> result the application server will generate the proper content for the user
> (different logo images, text, etc...) and return this with a cookie
> specifying the edition for future requests bypassing the GeoIP lookup. The
> cookie will also allow the user to select a different region than what
> their IP resolves to.
>
>  Again this is working in the application, and Varnish was extended to
> match on the cookie value (named "edition") and to use the edition value as
> part of the hash data so there is a unique cache for the different regions.
>
>  Here's the problem:
>
>  When a user visits the site for the first time they do not have a cookie
> in the request. So Varnish has a blank value for the edition as part of the
> hash data. However this request when it hits the application server does
> resolve to a specific edition and return the region content on the reply.
> This is being cached by Varnish and used for future requests by new users
> with no cookie.
>
>  Put simply, if the page is not in cache, and a browser from the UK does
> not have the edition cookie a page with UK content is generated and cached
> in Varnish. If a browser from the US goes to the same page, Varnish will
> serve the US user the UK cached page because this is what was cached based
> on the hash data.
>
>  Without turning off caching for cookieless requests - how do we resolve
> this?
>
>
> I may not have understood exactly what you need, but can't you hash the
> country code instead of the cookie ?
>
>
> --
> Guillaume Quintard
>
>  _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150129/105febd9/attachment.html>

From infos at opendoc.net  Thu Jan 29 07:54:01 2015
From: infos at opendoc.net (Alexandre)
Date: Thu, 29 Jan 2015 08:54:01 +0100
Subject: best practice with varnish
In-Reply-To: <CABtDKm4c8kEJwXid-ouVd-ggeUu=x4Xfw-rSKMw2fFYFvkMwjA@mail.gmail.com>
References: <54C7408F.4020602@opendoc.net> <54C940B9.4010901@opendoc.net>
 <CAAaeqWfE5Jd+_2c2Wr8bgqfpAbN+S=u3RUi6PAReAJNfY+nYjg@mail.gmail.com>
 <AM3PR06MB226389709791C3E08A77060B9330@AM3PR06MB226.eurprd06.prod.outlook.com>
 <CABtDKm4c8kEJwXid-ouVd-ggeUu=x4Xfw-rSKMw2fFYFvkMwjA@mail.gmail.com>
Message-ID: <54C9E719.1080700@opendoc.net>

Thank you all for your reactivity and your answers. I so I leave the 
configuration in a single file.

Thank you !

Alex.

On 28/01/15 23:06, Dridi Boukelmoune wrote:
> Hi,
>
> I usually move environment-specific stuff to a specific file, and keep
> the cache policy in a single file. By environment, I mean dev,
> staging, production etc.
>
> What usually changes between environments is backends/directors, ACLs
> and backend selection.
>
> Cheers,
> Dridi
>
> On Wed, Jan 28, 2015 at 10:34 PM, Jan-Aage Frydenb?-Bruvoll
> <jan at architechs.eu> wrote:
>> Hi,
>>
>> We've split relatively static stuff like backends, health checks, etc into separate files in order to reduce the noise in the main VCL. The logic remains in a single file, though.
>>
>> Best regards
>> Jan
>>
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at varnish-cache.org
>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>



From phk at phk.freebsd.dk  Thu Jan 29 22:01:44 2015
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Thu, 29 Jan 2015 22:01:44 +0000
Subject: Is there a recent security problem?
In-Reply-To: <20150129104615.56f15f4f@gdr-desktop.gdr.name>
References: <20150129104615.56f15f4f@gdr-desktop.gdr.name>
Message-ID: <67853.1422568904@critter.freebsd.dk>

--------
In message <20150129104615.56f15f4f at gdr-desktop.gdr.name>, Andrzej Godziuk writ
es:

>Is somebody having problems like this recently? The only explanation I
>have is a security problem that someone found and is exploiting. This
>or sun spots or increased radiation above Europe :)

I havn't had reports of any of those three things..

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.



From bluethundr at gmail.com  Fri Jan 30 18:35:10 2015
From: bluethundr at gmail.com (Tim Dunphy)
Date: Fri, 30 Jan 2015 13:35:10 -0500
Subject: forbid access to certain URLs in VCL
Message-ID: <CAOZy0ekSdcDB1GQ+ZBtDxACSWTiMx4Qxn=ZqWeMQp8R_DpXXKQ@mail.gmail.com>

Hey Sam,


 We have a request to block access to certain URLs from outside the
 network using Varnish. I came up with a way that I think will work. But
I?d like to bounce my method off you and see what you think. This is just
for staging right now. Production will probably be soon, but they haven?t
asked yet.

 I started by setting up an ACL that included all the IPs and networks that
I know of (so far) that constitute our internal network:


acl localnetwork {

  "xx.xx.xx.xx";

   "xx.xx.xx.xx"/24;

}


And then, down in vcl_recv I put the following:

 if(req.http.host ~ "^origin\.test-stage\.ourcompany\.com$" && req.url ~
"(^/user|^/user/|^/user/*)" && client.ip !~ localnetwork) {

           error 403 "Forbidden";

   }

Here, I?m basically saying if you?re hitting this http host and these
following URLs and you don?t belong to any of the IPs mentioned in the
localhost ACL, you?ll receive an 403 forbidden error.

I?m only unsure of a couple of things, that I?d like your opinion on. Would
I need to have the error handled by a call to a separate vcl_error ?

Also for the phrase  "^origin\.test-stage\.nbcuni\.com$?  will I need to
escape the dash in test-stage.nbcuni.com in order for this to work?

OR do you think what I have here will do the trick? I think it might be
good as is.

Thanks for your help!

Tim

-- 
GPG me!!

gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150130/ddb508b9/attachment.html>

From bluethundr at gmail.com  Fri Jan 30 22:42:32 2015
From: bluethundr at gmail.com (Tim Dunphy)
Date: Fri, 30 Jan 2015 17:42:32 -0500
Subject: error purging URLs
Message-ID: <CAOZy0e=DHHnae+LvC2f-5ce5TtW6v1uGk3Wn1OH41amjRvu4Mg@mail.gmail.com>

Hey guys,

 For some reason when I go to purge the varnish cache on my second varnish
node, I get this error:

[root at varnish2:~] #varnishadm ban "req.url ~ /"
Cannot open /var/lib/varnish/varnish2/_.vsm: No such file or directory
Could not open shared memory

Can someone please explain to me what this error means and how to correct
that?

Thanks!
Tim

-- 
GPG me!!

gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150130/7416d818/attachment.html>

From andrew.langhorn at digital.cabinet-office.gov.uk  Fri Jan 30 23:29:37 2015
From: andrew.langhorn at digital.cabinet-office.gov.uk (Andrew Langhorn)
Date: Fri, 30 Jan 2015 23:29:37 +0000
Subject: error purging URLs
In-Reply-To: <CAOZy0e=DHHnae+LvC2f-5ce5TtW6v1uGk3Wn1OH41amjRvu4Mg@mail.gmail.com>
References: <CAOZy0e=DHHnae+LvC2f-5ce5TtW6v1uGk3Wn1OH41amjRvu4Mg@mail.gmail.com>
Message-ID: <CANCa_WazwCEVJUJabveJ2Mj_QwRMYpz4Fn7YgeGSLcpk1n+rWQ@mail.gmail.com>

Sounds permissions-related. I'd start off with:

- `ps aux | grep varnishd` as root and noting the PID of varnishd
- `lsof -p $pid | grep vsm` to see where Varnish's shared memory log is
- checking that the Varnish user has read/write access to the location
returned above

Also, how are you using varnishd? Would be useful if you could post your
full varnishd command, including switches.

thanks

On 30 January 2015 at 22:42, Tim Dunphy <bluethundr at gmail.com> wrote:

> Hey guys,
>
>  For some reason when I go to purge the varnish cache on my second varnish
> node, I get this error:
>
> [root at varnish2:~] #varnishadm ban "req.url ~ /"
> Cannot open /var/lib/varnish/varnish2/_.vsm: No such file or directory
> Could not open shared memory
>
> Can someone please explain to me what this error means and how to correct
> that?
>
> Thanks!
> Tim
>
> --
> GPG me!!
>
> gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B
>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>



-- 
Andrew Langhorn
Web Operations
Government Digital Service

e: andrew.langhorn at digital.cabinet-office.gov.uk
t: +44 (0)7810 737375
a: 6th Floor, Aviation House, 125 Kingsway, London, WC2B 6NH
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20150130/4a996ccf/attachment.html>