From bott505 at gmail.com  Wed Jun 13 04:03:39 2007
From: bott505 at gmail.com (Brian Ott)
Date: Tue, 12 Jun 2007 21:03:39 -0700
Subject: question marks in the url
Message-ID: <1e30bbfe0706122103k7ae6d9e9w86431609c5d3b09f@mail.gmail.com>

We have urls: http://xxxx.yyy.com/test.jpg?tsp=11301981

the tsp gets updated when the picture is updated, its a timestamp.
this way the user doesnt have to refresh their browser to see new
pictures.

varnish doesnt cache urls with ? in them, it appears.

How can I make it cache this url?


From des at linpro.no  Wed Jun 13 08:37:34 2007
From: des at linpro.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=)
Date: Wed, 13 Jun 2007 10:37:34 +0200
Subject: question marks in the url
In-Reply-To: <1e30bbfe0706122103k7ae6d9e9w86431609c5d3b09f@mail.gmail.com>
	(Brian Ott's message of "Tue\, 12 Jun 2007 21\:03\:39 -0700")
References: <1e30bbfe0706122103k7ae6d9e9w86431609c5d3b09f@mail.gmail.com>
Message-ID: <87645sz1v5.fsf@des.linpro.no>

"Brian Ott" <bott505 at gmail.com> writes:
> We have urls: http://xxxx.yyy.com/test.jpg?tsp=11301981
>
> the tsp gets updated when the picture is updated, its a timestamp.
> this way the user doesnt have to refresh their browser to see new
> pictures.

This is a very strange way of doing things.  Far better to drop the
timestamp, set a long expiry time on the image and invalidate the URL in
Varnish when the picture is updated.

> varnish doesnt cache urls with ? in them, it appears.

Yes, it does.  There must be something else at play - most likely
cookies.  Since you didn't provide logs, there is no way to tell exactly
what is going on.

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


From phk at phk.freebsd.dk  Wed Jun 13 09:13:41 2007
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Wed, 13 Jun 2007 09:13:41 +0000
Subject: question marks in the url 
In-Reply-To: Your message of "Tue, 12 Jun 2007 21:03:39 MST."
	<1e30bbfe0706122103k7ae6d9e9w86431609c5d3b09f@mail.gmail.com> 
Message-ID: <47996.1181726021@critter.freebsd.dk>

In message <1e30bbfe0706122103k7ae6d9e9w86431609c5d3b09f at mail.gmail.com>, "Bria
n Ott" writes:
>We have urls: http://xxxx.yyy.com/test.jpg?tsp=11301981
>
>the tsp gets updated when the picture is updated, its a timestamp.
>this way the user doesnt have to refresh their browser to see new
>pictures.
>
>varnish doesnt cache urls with ? in them, it appears.

Varnish doesn't care about '?', but if you have cookies it will
not cache.


-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From jallspaw at yahoo.com  Thu Jun 14 23:02:28 2007
From: jallspaw at yahoo.com (john allspaw)
Date: Thu, 14 Jun 2007 16:02:28 -0700 (PDT)
Subject: quick LRU question
Message-ID: <34781.53827.qm@web31805.mail.mud.yahoo.com>

After taking a look at:

http://varnish.projects.linpro.no/wiki/ArchitectureLRU

and seeing:  
"
Obviously, if the cache/storage turnover is so high, that N becomes
a large fraction of it, this scheme breaks down.  Presumably you have other worries in that case."

I wonder:  will varnish be able to handle a cache that is constantly full and overflowing ?  We currently use squid for reverse-proxy caching for a working set of objects that is constantly growing, and for sure much larger than the caches will ever be. Efficient object eviction in this case is pretty important. :)

thanks!
john




       
____________________________________________________________________________________
Looking for a deal? Find great prices on flights and hotels with Yahoo! FareChase.
http://farechase.yahoo.com/


From des at linpro.no  Fri Jun 15 07:24:01 2007
From: des at linpro.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=)
Date: Fri, 15 Jun 2007 09:24:01 +0200
Subject: quick LRU question
In-Reply-To: <34781.53827.qm@web31805.mail.mud.yahoo.com> (john allspaw's
	message of "Thu\, 14 Jun 2007 16\:02\:28 -0700 \(PDT\)")
References: <34781.53827.qm@web31805.mail.mud.yahoo.com>
Message-ID: <87hcp9wui6.fsf@des.linpro.no>

john allspaw <jallspaw at yahoo.com> writes:
> I wonder: will varnish be able to handle a cache that is constantly
> full and overflowing ?  We currently use squid for reverse-proxy
> caching for a working set of objects that is constantly growing, and
> for sure much larger than the caches will ever be. Efficient object
> eviction in this case is pretty important. :)

We have to focus first of all on getting *some* kind of eviction
working, but rest assured that we will not settle for the simplest
implementation, at least not in the long run.

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


From vilhelm at aptoma.com  Wed Jun 20 14:36:17 2007
From: vilhelm at aptoma.com (=?ISO-8859-1?Q?=22Vilhelm_K=2E_Vard=F8y=22?=)
Date: Wed, 20 Jun 2007 16:36:17 +0200
Subject: Status of ticket #105?
Message-ID: <46793B61.8020304@aptoma.com>

Hi,

I'm wondering if there is any progress regarding the issue described in 
http://varnish.projects.linpro.no/ticket/105 ?

In production we currently run monit (a monitoring daemon) who checks if 
Varnish is alive, and starts it if it's down. This happens a lot once 
the url.purge commands is executed when there's traffic. This is 
somewhat a blocker since the short downtime is not that desirable.

We hope to use the same web-application at a larger scale for Verdens 
Gang AS, but we want this issue to be solved by then.

-- 
Best regards,
Vilhelm K. Vard?y
Programmer - APTOMA AS
Akersgata 55, 0170 Oslo, NORWAY
Phone     +47 24 14 50 71
Mobile   +47 46 66 23 96
http://www.aptoma.no


From des at linpro.no  Thu Jun 21 06:41:58 2007
From: des at linpro.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=)
Date: Thu, 21 Jun 2007 08:41:58 +0200
Subject: Status of ticket #105?
In-Reply-To: <46793B61.8020304@aptoma.com> ("Vilhelm K. =?utf-8?Q?Vard?=
	=?utf-8?Q?=C3=B8y=22's?= message of
	"Wed\, 20 Jun 2007 16\:36\:17 +0200")
References: <46793B61.8020304@aptoma.com>
Message-ID: <87tzt1lsg9.fsf@des.linpro.no>

"Vilhelm K. Vard?y" <vilhelm at aptoma.com> writes:
> I'm wondering if there is any progress regarding the issue described in
> http://varnish.projects.linpro.no/ticket/105 ?

Sorry, we haven't had time to look into it.

The first stack trace shows an assertion failure in EXP_TTLchange() and
is probably related  to url.purge; the second shows an assertion failure
in smf_alloc() and is most likely a result of Varnish running out of
storage space.

The attached patch should prevent the first case from reoccurring.  The
second is being worked on, and should be fixed in a couple of weeks.

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no

-------------- next part --------------
A non-text attachment was scrubbed...
Name: hsh_lookup.diff
Type: text/x-diff
Size: 472 bytes
Desc: not available
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20070621/60eb9131/attachment.bin>

From vilhelm at aptoma.com  Thu Jun 21 07:26:19 2007
From: vilhelm at aptoma.com (=?UTF-8?B?IlZpbGhlbG0gSy4gVmFyZMO4eSI=?=)
Date: Thu, 21 Jun 2007 09:26:19 +0200
Subject: Status of ticket #105?
In-Reply-To: <87tzt1lsg9.fsf@des.linpro.no>
References: <46793B61.8020304@aptoma.com> <87tzt1lsg9.fsf@des.linpro.no>
Message-ID: <467A281B.1050509@aptoma.com>

Dag-Erling Sm?rgrav wrote:
> The first stack trace shows an assertion failure in EXP_TTLchange() and
> is probably related  to url.purge [...]
> 
> The attached patch should prevent the first case from reoccurring.  

Thanks.   We'll apply the patch today, but there won't be any real 
traffic on the site until tonight.

-- 
Vilhelm.


From vilhelm at aptoma.com  Fri Jun 29 08:13:09 2007
From: vilhelm at aptoma.com (=?UTF-8?B?IlZpbGhlbG0gSy4gVmFyZMO4eSI=?=)
Date: Fri, 29 Jun 2007 10:13:09 +0200
Subject: Status of ticket #105?
In-Reply-To: <467A281B.1050509@aptoma.com>
References: <46793B61.8020304@aptoma.com> <87tzt1lsg9.fsf@des.linpro.no>
	<467A281B.1050509@aptoma.com>
Message-ID: <4684BF15.1060602@aptoma.com>

Vilhelm K. Vard?y wrote:
> Dag-Erling Sm?rgrav wrote:
>> The first stack trace shows an assertion failure in EXP_TTLchange() and
>> is probably related  to url.purge [...]
>>
>> The attached patch should prevent the first case from reoccurring.  
> 
> Thanks.   We'll apply the patch today, but there won't be any real 
> traffic on the site until tonight.
> 

The patch obviously fixed the one issue, thanks.  However, varnishd 
stilled died over 100 times within 2 hours on the last "run" (monit 
sends a email for each time it has to start varnish), this time I 
discovered another one:

(gdb) bt
#0  0x2817becb in kill () from /lib/libc.so.6
#1  0x280b32d2 in raise () from /usr/lib/libthr.so.2
#2  0x2817ab78 in abort () from /lib/libc.so.6
#3  0x2809eaf6 in lbv_assert (func=0x8068c49 "ev_schedule_one",
     file=0x8068cb2 "mgt_event.c", line=450,
     cond=0x8068de8 "evb->lpfd < evb->npfd", err=9) at assert.c:58
#4  0x0805e51c in ev_schedule_one (evb=0x8075040) at mgt_event.c:450
#5  0x0805e5c9 in ev_schedule (evb=0x8075040) at mgt_event.c:344
#6  0x0805c301 in mgt_run (dflag=0, T_arg=0xbfbfe7b0 "\001") at 
mgt_child.c:409
#7  0x08063dd2 in main (argc=17, argv=0xbfbfed3c) at varnishd.c:540


I'm not sure that this is the cause in all cases(/crashes), since the 
core-file gets overwritten for each time (I only have the core-file from 
the last time it crashed). Is there a simple way to keep all core-files?

-- 
Vilhelm.


From phk at phk.freebsd.dk  Fri Jun 29 08:22:00 2007
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Fri, 29 Jun 2007 08:22:00 +0000
Subject: Status of ticket #105? 
In-Reply-To: Your message of "Fri, 29 Jun 2007 10:13:09 +0200."
	<4684BF15.1060602@aptoma.com> 
Message-ID: <4558.1183105320@critter.freebsd.dk>


On FreeBSD you can set the corefile name with the sysctl "kern.corefile",
I belive there is a %something escape that substitutes PID number.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From vilhelm at aptoma.com  Fri Jun 29 09:06:04 2007
From: vilhelm at aptoma.com (=?ISO-8859-1?Q?=22Vilhelm_K=2E_Vard=F8y=22?=)
Date: Fri, 29 Jun 2007 11:06:04 +0200
Subject: Status of ticket #105?
In-Reply-To: <4558.1183105320@critter.freebsd.dk>
References: <4558.1183105320@critter.freebsd.dk>
Message-ID: <4684CB7C.9090608@aptoma.com>

Poul-Henning Kamp wrote:
> On FreeBSD you can set the corefile name with the sysctl "kern.corefile",
> I belive there is a %something escape that substitutes PID number.
> 

Thanks for pointing me in the right direction!

If it should be in someones interest: %P substitutes the pid number, and 
%N the filename.

-- 
Vilhelm.


From rudd-o at rudd-o.com  Fri Jun 29 20:04:49 2007
From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O))
Date: Fri, 29 Jun 2007 15:04:49 -0500
Subject: utma utmz
Message-ID: <1183147489.32019.23.camel@gabriela>

Cookie issue again,

My site always feeds the utma and utmz (google analytics) cookies to
requests.  How can I specify in the varnish configuration file
default.vcl that if *only* those two cookies are present, it should
supply files from cache, but if any other cookies are present, it should
contact the backend server (or use the appropriate caching policy)?

regards,


	Manuel Amador (Rudd-O) <rudd-o at rudd-o.com>
	The R Zone - http://rudd-o.com/
	GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/

Now playing, courtesy of Amarok: 
Someone whom you reject today, will reject you tomorrow.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20070629/3dc837e8/attachment.bin>

From rudd-o at rudd-o.com  Fri Jun 29 19:41:10 2007
From: rudd-o at rudd-o.com (Manuel Amador (Rudd-O))
Date: Fri, 29 Jun 2007 14:41:10 -0500
Subject: Problem with varnish and caching
Message-ID: <1183146070.32019.20.camel@gabriela>

As you might already have noted, I reported a bug on varnish caching
files indiscriminately.

I now have a different problem.

My page sets a few cookies.  That'd be okay and it should produce
dynamic pages, which Varnish is furnishing through my backend.  The
thing is, these cookies are sent along requests for CSS and PNG and JPG
and JS files, which causes varnish to contact the backend.  I don't want
that to happen (I'm happy with them being cached by Varnish 120
seconds).

How can I tell Varnish that requests with a response that includes ETag
(a discriminant for static files) should be forcibly cached?

The VCL man page makes no mention of any discriminant variable I could
use in any of varnish's subroutines that will let me forcibly have them
served from the cache.

Again: I want files that apache responded with an ETag to be forcibly
cached, no matter if my client sends cookies along or not.

luck,

	Manuel Amador (Rudd-O) <rudd-o at rudd-o.com>
	The R Zone - http://rudd-o.com/
	GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/

Now playing, courtesy of Amarok: 
You will engage in a profitable business activity.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20070629/ac540ae2/attachment.bin>

From des at linpro.no  Sat Jun 30 21:36:10 2007
From: des at linpro.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=)
Date: Sat, 30 Jun 2007 23:36:10 +0200
Subject: utma utmz
In-Reply-To: <1183147489.32019.23.camel@gabriela> (Manuel Amador's message of
	"Fri\, 29 Jun 2007 15\:04\:49 -0500")
References: <1183147489.32019.23.camel@gabriela>
Message-ID: <87ir9515x1.fsf@des.linpro.no>

[moved from -dev to -misc]

"Manuel Amador (Rudd-O)" <rudd-o at rudd-o.com> writes:
> My site always feeds the utma and utmz (google analytics) cookies to
> requests.  How can I specify in the varnish configuration file
> default.vcl that if *only* those two cookies are present, it should
> supply files from cache, but if any other cookies are present, it should
> contact the backend server (or use the appropriate caching policy)?

There's no easy way to do that with the current code base, but we expect
to have the required functionality in place within a couple of weeks.

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no


From des at linpro.no  Sat Jun 30 21:41:58 2007
From: des at linpro.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=)
Date: Sat, 30 Jun 2007 23:41:58 +0200
Subject: Problem with varnish and caching
In-Reply-To: <1183146070.32019.20.camel@gabriela> (Manuel Amador's message of
	"Fri\, 29 Jun 2007 14\:41\:10 -0500")
References: <1183146070.32019.20.camel@gabriela>
Message-ID: <87ejjt15nd.fsf@des.linpro.no>

[moved from -dev to -misc]

"Manuel Amador (Rudd-O)" <rudd-o at rudd-o.com> writes:
> As you might already have noted, I reported a bug on varnish caching
> files indiscriminately.

This is not a bug, it is a misunderstanding.  It appears you expect
Varnish to act like an RFC 2616 "shared cache" whereas it is in fact a
"surrogate" (see the "Edge Architecture Specification" by Oracle and
Akamai, although Varnish does not yet fully implement that specification
either)

> My page sets a few cookies.  That'd be okay and it should produce
> dynamic pages, which Varnish is furnishing through my backend.  The
> thing is, these cookies are sent along requests for CSS and PNG and JPG
> and JS files, which causes varnish to contact the backend.  I don't want
> that to happen (I'm happy with them being cached by Varnish 120
> seconds).
>
> How can I tell Varnish that requests with a response that includes ETag
> (a discriminant for static files) should be forcibly cached?

This is basically the same issue as in your previous email, and the
answer is the same.

DES
-- 
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no