From plfgoa at gmail.com  Fri Jan  2 04:37:00 2009
From: plfgoa at gmail.com (Paras Fadte)
Date: Fri, 2 Jan 2009 10:07:00 +0530
Subject: Panic message : Varnish restart
Message-ID: <75cf5800901012037u2d6d2a60q9d1088ea2f10d1@mail.gmail.com>

Hi,

I got a message like following in /var/log/messages and child
restarted. what could be the issue ?

varnishd[28876]: Child (31253) Panic message: Missing errorhandling
code in fetch_chunked(), cache_fetch.c line 113:   Condition(be >
 bp) not true.  thread = (cache-worker)sp = 0x2aabd2970008 {   fd =
302, id = 302, xid = 1800279969,

Varnish Version is :

varnishd (varnish-trunk)
Copyright (c) 2006-2008 Linpro AS / Verdens Gang AS

Thanks in advance.

-Paras


From perbu at linpro.no  Fri Jan  2 17:03:33 2009
From: perbu at linpro.no (Per Buer)
Date: Fri, 02 Jan 2009 18:03:33 +0100
Subject: Configuration problem, possibly a PEBCAK.
In-Reply-To: <495B5C1D.5010302@stillbilde.net>
References: <495B5C1D.5010302@stillbilde.net>
Message-ID: <495E48E5.1000003@linpro.no>

Svein Skogen (List Mail Account) wrote:

> Is there a way to configure varnish to start sending what it receives
> immediately, while still retaining a copy in the cache, or do I have to
> make a choice between caching, or forwarding?

Neither. Both passing and caching an object will fetch the whole object
into memory before passing it on to the client. "pipe" will do what you
want, but you probably don't want that as pipe has other side effects.

This "streaming" mode would be very handy and we are looking for someone
to sponsor this feature.

-- 
http://linpro.no/ | http://redpill.se/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 260 bytes
Desc: OpenPGP digital signature
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20090102/ea11fb78/attachment.bin>

From oliver.oli+0815 at gmail.com  Sun Jan  4 10:08:12 2009
From: oliver.oli+0815 at gmail.com (Oliver Oli)
Date: Sun, 4 Jan 2009 11:08:12 +0100
Subject: Configuration problem, possibly a PEBCAK.
In-Reply-To: <495E48E5.1000003@linpro.no>
References: <495B5C1D.5010302@stillbilde.net> <495E48E5.1000003@linpro.no>
Message-ID: <66f535320901040208s56518357v95d6492eed6373a6@mail.gmail.com>

On Fri, Jan 2, 2009 at 6:03 PM, Per Buer <perbu at linpro.no> wrote:

> Neither. Both passing and caching an object will fetch the whole object
> into memory before passing it on to the client. "pipe" will do what you
> want, but you probably don't want that as pipe has other side effects.

What kind of side effects?


From perbu at linpro.no  Sun Jan  4 11:28:29 2009
From: perbu at linpro.no (Per Buer)
Date: Sun, 04 Jan 2009 12:28:29 +0100
Subject: Configuration problem, possibly a PEBCAK.
In-Reply-To: <66f535320901040208s56518357v95d6492eed6373a6@mail.gmail.com>
References: <495B5C1D.5010302@stillbilde.net> <495E48E5.1000003@linpro.no>
	<66f535320901040208s56518357v95d6492eed6373a6@mail.gmail.com>
Message-ID: <49609D5D.8020008@linpro.no>

Oliver Oli wrote:
> On Fri, Jan 2, 2009 at 6:03 PM, Per Buer <perbu at linpro.no> wrote:
> 
>> Neither. Both passing and caching an object will fetch the whole object
>> into memory before passing it on to the client. "pipe" will do what you
>> want, but you probably don't want that as pipe has other side effects.
> 
> What kind of side effects?

pipe shuffles bytes back and forth. If pipelining is used another HTTP
request might show up on the same connection - and you want even notice.

-- 

http://linpro.no/ | http://redpill.se/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 260 bytes
Desc: OpenPGP digital signature
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20090104/1fa67dd6/attachment.bin>

From M.Mazurek at netsync.pl  Sun Jan  4 21:37:46 2009
From: M.Mazurek at netsync.pl (Marcin Mazurek)
Date: Sun, 4 Jan 2009 22:37:46 +0100
Subject: varnishtop - grouping by part of url
Message-ID: <20090104213746.GB13876@netsync.pl>

Hi,

I'm playing a bitwith varnishtop. Is it possible somehow to get a
statistics based on part of URL that go through varnish?

Eg:

/aaa/bbb/1/asd/dasdad/231/

/aaa/ccc/2/sdsad/gw/222


and I would like to get stats for:

- /aaa/bbb
- /aaa/ccc


Is it possible?

regards

-- 
Marcin Mazurek



From andrew at scoop.co.nz  Mon Jan  5 08:29:55 2009
From: andrew at scoop.co.nz (Andrew McNaughton)
Date: Mon, 05 Jan 2009 19:29:55 +1100
Subject: varnishtop - grouping by part of url
In-Reply-To: <20090104213746.GB13876@netsync.pl>
References: <20090104213746.GB13876@netsync.pl>
Message-ID: <4961C503.302@scoop.co.nz>

Use either varnishlog or varnishncsa, and go from there. If you need to
ask, then probably it's easiest to use varnishncsa to feed logs into an
off the shelf web log analysis package.

Andrew

Marcin Mazurek wrote:
> Hi,
>
> I'm playing a bitwith varnishtop. Is it possible somehow to get a
> statistics based on part of URL that go through varnish?
>
> Eg:
>
> /aaa/bbb/1/asd/dasdad/231/
>
> /aaa/ccc/2/sdsad/gw/222
>
>
> and I would like to get stats for:
>
> - /aaa/bbb
> - /aaa/ccc
>
>
> Is it possible?
>
> regards
>
>   



From phk at phk.freebsd.dk  Mon Jan  5 08:58:50 2009
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Mon, 05 Jan 2009 08:58:50 +0000
Subject: Happy 3 year Birthday Varnish
Message-ID: <81548.1231145930@critter.freebsd.dk>


Thanks to email archives, future historians will have an easy task,
provided they know what to grep for.

They will know that Varnish was discussed at the EuroBSDcon in Basel
in November 2005 and that the first meeting was held, not on 25th
of januaray 2006 as planned, but on 2nd of february instead, due
to the SAS personel strike.

There could however, be disagreement if Dag-Erlings initial creation
of the subversion tree structure on 1st of february, the meeting
at VG's downtown Oslo office, my email with architectural thoughts
from the 9th of february or the blackboard in Basel counts as the
official birtday of the project.

So for the email record, let me state that the official birth of
the project was New Years morning 2006, where I finally made up
my mind that this could be a fun thing to work on.

In other words, Varnish was three years old four days ago and I'm
quite happy with the progress our toddler is showing.

I promised to drop an email every so often to keep people in
the loop so you know what to expect for the next half year or so.

Before X-mas I added a new lookup algorithm called "critbit"
which hopefully will address some of the performance/memory 
shortcomings of the "classic hash" lookup.

Once we are happy with the stability of it, we will probably
roll another 2.0.x release, before I start to tear up the
tree for the big ticket item: persistent storage.

We hope to have a 2.1 release with you in early summer and the
major ticket item there is persistent storage, ie: not loosing
your cached content when varnish restarts.

There will probably also be a Web UI/management console application
in that release

As always we will try to hold the buglist in check, it has been
growing a bit over the x-mas holiday, but it will get due attention
now that we're back in the saddle.

On a personal note: some of you found my wishlist on amazon.com
and brigthened up my X-mas, thanks a lot, always appreciated :-)

Happy New Year everybody!

Poul-Henning
-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.


From phk at phk.freebsd.dk  Mon Jan  5 08:30:07 2009
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Mon, 05 Jan 2009 08:30:07 +0000
Subject: Configuration problem, possibly a PEBCAK. 
In-Reply-To: Your message of "Wed, 31 Dec 2008 12:48:45 +0100."
	<495B5C1D.5010302@stillbilde.net> 
Message-ID: <81389.1231144207@critter.freebsd.dk>

In message <495B5C1D.5010302 at stillbilde.net>, "Svein Skogen (List Mail Account)
" writes:

>Is there a way to configure varnish to start sending what it receives
>immediately, while still retaining a copy in the cache, or do I have to
>make a choice between caching, or forwarding?

Not yet.  It's on our wishlist.


-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From tfheen at linpro.no  Mon Jan  5 10:39:22 2009
From: tfheen at linpro.no (Tollef Fog Heen)
Date: Mon, 05 Jan 2009 11:39:22 +0100
Subject: varnishtop - grouping by part of url
In-Reply-To: <20090104213746.GB13876@netsync.pl> (Marcin Mazurek's message of
	"Sun, 4 Jan 2009 22:37:46 +0100")
References: <20090104213746.GB13876@netsync.pl>
Message-ID: <87ljtqkpet.fsf@qurzaw.linpro.no>

]] Marcin Mazurek 

| I'm playing a bitwith varnishtop. Is it possible somehow to get a
| statistics based on part of URL that go through varnish?
| 
| Eg:
| 
| /aaa/bbb/1/asd/dasdad/231/
| 
| /aaa/ccc/2/sdsad/gw/222
| 
| 
| and I would like to get stats for:
| 
| - /aaa/bbb
| - /aaa/ccc
| 
| 
| Is it possible?

Not at the moment, no.  It would be interesting to have the ability to
do so, though.

-- 
Tollef Fog Heen 
Redpill Linpro -- Changing the game!
t: +47 21 54 41 73


From naama at answers.com  Mon Jan  5 11:17:27 2009
From: naama at answers.com (Naama Bamberger)
Date: Mon, 5 Jan 2009 13:17:27 +0200
Subject: Is it possible to compare an ACL list to a specific header?
In-Reply-To: <6910a60812160102t12f04cf2l1a712a68436f3a1b@mail.gmail.com>
References: <6910a60812160102t12f04cf2l1a712a68436f3a1b@mail.gmail.com>
Message-ID: <749CACF29BDFB64E9F80189ECD778688043CD81C@jermail1.atomant.net>

I want to block some IPs, but cannot use
if (client.ip ~ blocked_ips), 
since all the requests go through a load balancer.

The original user IP is stored by the load balancer in a custom header.
I tried something like if (req.http.X-My-Custom-Header ~ blocked_ips),
but trying to compile it causes a segfault.

I also tried to write a C function like this:

sub client_check {
C{
        if (match_acl_named_blocked_ips(sp, VRT_GetHdr(sp, HDR_REQ,
"\021X-My-Custom-Header:"))) {
                VRT_error(sp, 403, "IP blocked - user denied");
                VRT_done(sp, VCL_RET_ERROR);
        }
}C
}

It compiled, but I get this on every request:

    7 SessionOpen  c 10.16.8.5 57600 :80
    0 WorkThread   - 0x42802c00 start
    0 WorkThread   - 0x43203c00 start
    0 WorkThread   - 0x43c04c00 start
    0 CLI          - Rd vcl.load boot ./vcl.1P9zoqAU.so
    0 CLI          - Wr 0 200 Loaded "./vcl.1P9zoqAU.so" as "boot"
    0 CLI          - Rd vcl.use boot
    0 CLI          - Wr 0 200
    0 CLI          - Rd start
    0 Debug        - "Acceptor is epoll"
    0 CLI          - Wr 0 200
    0 WorkThread   - 0x45a07c00 start

Thanks,
Naama Bamberger
Engineering, Director
Answers.com
naama at answers.com
http://www.answers.com
http://wiki.answers.com




From wangqidi at gmail.com  Mon Jan  5 11:33:32 2009
From: wangqidi at gmail.com (wangqidi)
Date: Mon, 5 Jan 2009 19:33:32 +0800
Subject: Varnish keep-alive problem
Message-ID: <200901051933295932843@gmail.com>

hi?there

Varnish is a very good web accelerator, and i find it support KeepAlive.
The default keep-alive is on. I want to turn off the keep-alive, but i don't know how to do it.
Please tell me how to turn off the keep-alive.

And would this have any adverse side-effects?

Grateful for your help. Thanks.

Timo
2009-01-05
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20090105/0a7c8b67/attachment.html>

From tfheen at linpro.no  Tue Jan  6 13:26:40 2009
From: tfheen at linpro.no (Tollef Fog Heen)
Date: Tue, 06 Jan 2009 14:26:40 +0100
Subject: Varnish keep-alive problem
In-Reply-To: <200901051933295932843@gmail.com> (wangqidi@gmail.com's message
	of "Mon, 5 Jan 2009 19:33:32 +0800")
References: <200901051933295932843@gmail.com>
Message-ID: <87aba4bm5r.fsf@qurzaw.linpro.no>

]] "wangqidi" 

| Varnish is a very good web accelerator, and i find it support KeepAlive.
| The default keep-alive is on. I want to turn off the keep-alive, but i don't know how to do it.
| Please tell me how to turn off the keep-alive.

I don't believe that's possible out of the box.  Why do you want to turn
off keepalive?

-- 
Tollef Fog Heen 
Redpill Linpro -- Changing the game!
t: +47 21 54 41 73


From nick at loman.net  Tue Jan  6 13:32:51 2009
From: nick at loman.net (Nick Loman)
Date: Tue, 06 Jan 2009 13:32:51 +0000
Subject: Varnish keep-alive problem
In-Reply-To: <87aba4bm5r.fsf@qurzaw.linpro.no>
References: <200901051933295932843@gmail.com> <87aba4bm5r.fsf@qurzaw.linpro.no>
Message-ID: <49635D83.7050806@loman.net>

Tollef Fog Heen wrote:

> | Varnish is a very good web accelerator, and i find it support KeepAlive.
> | The default keep-alive is on. I want to turn off the keep-alive, but i don't know how to do it.
> | Please tell me how to turn off the keep-alive.
> 
> I don't believe that's possible out of the box.  Why do you want to turn
> off keepalive?

You can turn it off on the backend web-server.

Alternatively, you could try adding: "Connection: close" to the returned 
headers in the VCL which should encourage the client to close the 
connection.

Cheers,

Nick.





From tfheen at linpro.no  Tue Jan  6 13:42:32 2009
From: tfheen at linpro.no (Tollef Fog Heen)
Date: Tue, 06 Jan 2009 14:42:32 +0100
Subject: next release?
In-Reply-To: <200812211455.11650.ottolski@web.de> (Sascha Ottolski's message
	of "Sun, 21 Dec 2008 14:55:11 +0100")
References: <200812211455.11650.ottolski@web.de>
Message-ID: <8763ksblfb.fsf@qurzaw.linpro.no>

]] Sascha Ottolski 

| I'm curious when the next stable (minor) release is planned. I'm 
| especially interested in
|
| "Date: 2008-11-14 01:19:33 +0100 (Fri, 14 Nov 2008)
| New Revision: 3390

[...]

| which seem to be in the trunk, but didn't made it into 2.0.2.

I don't think I'll let 3390 into 2.0.3.  It might be safe, but I would
like to play it conservative.

| If no release is planned, which trunk revision could be recommended for 
| productive environment?

Just cherry-picking 3390 should work, it seems fairly self-contained.

-- 
Tollef Fog Heen 
Redpill Linpro -- Changing the game!
t: +47 21 54 41 73


From tfheen at linpro.no  Tue Jan  6 13:45:33 2009
From: tfheen at linpro.no (Tollef Fog Heen)
Date: Tue, 06 Jan 2009 14:45:33 +0100
Subject: Configuration problem, possibly a PEBCAK.
In-Reply-To: <49609D5D.8020008@linpro.no> (Per Buer's message of "Sun, 04 Jan
	2009 12:28:29 +0100")
References: <495B5C1D.5010302@stillbilde.net> <495E48E5.1000003@linpro.no>
	<66f535320901040208s56518357v95d6492eed6373a6@mail.gmail.com>
	<49609D5D.8020008@linpro.no>
Message-ID: <871vvgblaa.fsf@qurzaw.linpro.no>

]] Per Buer 

| Oliver Oli wrote:
| > On Fri, Jan 2, 2009 at 6:03 PM, Per Buer <perbu at linpro.no> wrote:
| > 
| >> Neither. Both passing and caching an object will fetch the whole object
| >> into memory before passing it on to the client. "pipe" will do what you
| >> want, but you probably don't want that as pipe has other side effects.
| > 
| > What kind of side effects?
| 
| pipe shuffles bytes back and forth. If pipelining is used another HTTP
| request might show up on the same connection - and you want even notice.

This can be worked around by having set bereq.http.Connection = "close";
in vcl_pipe, at the cost of not having keepalives for those requests.

-- 
Tollef Fog Heen 
Redpill Linpro -- Changing the game!
t: +47 21 54 41 73


From roy at karlsbakk.net  Tue Jan  6 15:27:35 2009
From: roy at karlsbakk.net (Roy Sigurd Karlsbakk)
Date: Tue, 6 Jan 2009 16:27:35 +0100
Subject: question about configure warning
Message-ID: <04F06E3F-8148-43BD-A716-53746A98AA22@karlsbakk.net>

Hi all

I'm setting up an amd64 box (that is intel xeon x86_64) with ubuntu  
8.04 and varnish 2.0.2, and configure gives me a warning about sendfile

> configure: WARNING: won't look for sendfile() on x86_64-unknown- 
> linux-gnu

Does this mean it won't be using sendfile() on this system?

roy
--
Roy Sigurd Karlsbakk
(+47) 98013356
roy at karlsbakk.net
--
I all pedagogikk er det essensielt at pensum presenteres  
intelligibelt. Det er en element?rt imperativ for alle pedagoger ?  
unng? eksessiv anvendelse av idiomer med fremmed opprinnelse. I de  
fleste tilfeller eksisterer adekvate og relevante synonymer p? norsk.



From marcussmith at britarch.ac.uk  Tue Jan  6 15:42:14 2009
From: marcussmith at britarch.ac.uk (Marcus Smith)
Date: Tue, 06 Jan 2009 15:42:14 +0000
Subject: question about configure warning
In-Reply-To: <04F06E3F-8148-43BD-A716-53746A98AA22@karlsbakk.net>
References: <04F06E3F-8148-43BD-A716-53746A98AA22@karlsbakk.net>
Message-ID: <49637BD6.40809@britarch.ac.uk>

Roy Sigurd Karlsbakk wrote:

> Does this mean it won't be using sendfile() on this system?

If I recall correctly, it won't be using sendfile() on *almost all*
systems (except certain versions of BSD?).  From the porting pages:

"The build system will automatically detect the availability of epoll()
and build the corresponding cache_acceptor. It will also automatically
detect the availability of sendfile(), though its use is discouraged
(and disabled by default) due to a variety of non-Linux-specific issues."

(http://varnish.projects.linpro.no/wiki/Porting/Linux/2.6)

best wishes,
Marcus

-- 
Marcus Smith
Information Officer
The Council for British Archaeology


From michael at dynamine.net  Tue Jan  6 17:48:02 2009
From: michael at dynamine.net (Michael S. Fischer)
Date: Tue, 6 Jan 2009 09:48:02 -0800
Subject: question about configure warning
In-Reply-To: <49637BD6.40809@britarch.ac.uk>
References: <04F06E3F-8148-43BD-A716-53746A98AA22@karlsbakk.net>
	<49637BD6.40809@britarch.ac.uk>
Message-ID: <0673749E-7A79-4355-BEA4-DE2D04F4EEB8@dynamine.net>

On Jan 6, 2009, at 7:42 AM, Marcus Smith wrote:
> "The build system will automatically detect the availability of  
> epoll()
> and build the corresponding cache_acceptor. It will also automatically
> detect the availability of sendfile(), though its use is discouraged
> (and disabled by default) due to a variety of non-Linux-specific  
> issues."
>
> (http://varnish.projects.linpro.no/wiki/Porting/Linux/2.6)

What are these "non-Linux-specific issues" to which the document refers?

--Michael


From zredsox at gmail.com  Wed Jan  7 19:03:41 2009
From: zredsox at gmail.com (Scott McCullough)
Date: Wed, 7 Jan 2009 14:03:41 -0500
Subject: Varnish Auth Sessions (vBulletin)
Message-ID: <916960160901071103g53ae0530ta56b3abd3bf75b22@mail.gmail.com>

I run a vbulletin site with a lighttpd server and have been trying to
configure varnish as a front end to cache and serve dynamic pages.

I have looked through the mailing list and do not see any vBulletin specific
implementation advice. Doing google searches on the web leads me to some
information (mostly about plone) but with my limited skill set I have yet to
find a workable solution.

( I have successfully been able to get it to just cache images, css and
javascript.)

However, It would seem I am only able to get varnish to work in two modes
when it comes to php pages: Cache Everything, or Cache Nothing

I have tried using both:

sub vcl_recv {

     if (req.http.Authenticate || req.http.Authorization) {
             pass;
  }
}

and

sub vcl_fetch {

      if (req.http.Authorization && !obj.http.Cache-Control ~ "public") {
      pass;

      }
}

...but it still caches authenticated sessions and all the personalized page
portions that come with those sessions.


As an anonymous user heading to my site you are immediately presented with
the cookies - bblastvisit, bblastactivity, _utma, utmb, _utmc, utmcsr (and
Ad cookies; google analytics cookies) - so everyone gets a cookie for
everything but there are no special cookies for logged in users (unless they
choose persistent login which would add bbpassword and bbuserid.)

I have played around with:

unset req.http.Cookie;

...but I don't properly grasp how to use it in conjunction with the cookies
in play.



Any help or redirection would be appreciated.

Thanks.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20090107/042d9da3/attachment.html>

From zredsox at gmail.com  Thu Jan  8 01:56:57 2009
From: zredsox at gmail.com (Scott McCullough)
Date: Wed, 7 Jan 2009 20:56:57 -0500
Subject: Varnish Auth Sessions (vBulletin)
Message-ID: <916960160901071756jc669485ta0083d5dfce003e2@mail.gmail.com>

I ended up resolving my own issue with a little help from the irc channel.

In case anyone else is looking for a duck tape solution...

I created my own cookie (auth_user) that executes in the login.php under the
"start do login" post request

$value = 'access';
setcookie("auth_user", $value);
setcookie("auth_user", $value, time()+86400);
setcookie("auth_user", $value, time()+86400, "/", ".mysite.com", 1);

and then in default.vcl:

sub vcl_recv {
if (req.http.cookie && req.http.cookie ~ "auth_user") {
    pass;
   }
}

sub vcl_hash {
if (req.http.cookie && req.http.cookie ~ "auth_user") { set req.hash +=
"auth"; }
    set req.hash += req.url;

 hash;
}
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20090107/e815956a/attachment.html>

From tfheen at linpro.no  Thu Jan  8 10:12:08 2009
From: tfheen at linpro.no (Tollef Fog Heen)
Date: Thu, 08 Jan 2009 11:12:08 +0100
Subject: 2.0.3 planning
Message-ID: <87aba2gl8n.fsf@qurzaw.linpro.no>


Hi,

I'm evaluating what changes that should go into 2.0.3 and have come up
with the following list:

------------------------------------------------------------------------
r3433 | phk | 2008-11-25 09:37:34 +0100 (ti., 25 nov. 2008) | 6 lines

When we receive an If-Modified-Since on an ESI object, do not process the conditional
for the child object and pretend to send a 304 reply for them, if we have decided to
deliver the main object.

Fixes #386

------------------------------------------------------------------------
r3361 | tfheen | 2008-11-06 12:46:31 +0100 (to., 06 nov. 2008) | 7 lines

Fix up $N vs \N in man page

The VCL man page documented the capturing parentheses as using $N
rather than \N which is actually used.

Fixes #359

------------------------------------------------------------------------
r3362 | tfheen | 2008-11-06 12:57:05 +0100 (to., 06 nov. 2008) | 4 lines

Document the size parameter to -s malloc

Fixes #362

------------------------------------------------------------------------
r3366 | phk | 2008-11-10 10:29:52 +0100 (ma., 10 nov. 2008) | 12 lines

Add a debug CLI command to seed random(3).

This is a lot less useful than it could have been, as the Open Group
only mandates that:

        Like rand(), random() shall produce by default a sequence
        of numbers that can be duplicated by calling srandom() with
        1 as the seed.

But crucially leaves out *which* sequence of numbers.

------------------------------------------------------------------------
r3367 | phk | 2008-11-10 10:37:21 +0100 (ma., 10 nov. 2008) | 14 lines

Add a toplevel word which examines the sequence returned by
srandom(1) and stops the test if we do not get the same sequence
as we expect.

The Open Group does not define which deterministic sequence srandom(1)
should result in, on that it be deterministic, but I have high hopes
in the general sanity and expect that UNIX people across the board
have realized that for portability the same sequence should be
returned on all platforms.

At the very least FreeBSD and Linux/GLIBC, as seen on projects.linpro.no,
agree.

------------------------------------------------------------------------
r3368 | phk | 2008-11-10 10:40:39 +0100 (ma., 10 nov. 2008) | 7 lines

Add a test of the random director that uses actual randomness.

This will be skipped on platforms where srandom(1) does not result
in the same deterministic sequence of random numbers as on FreeBSD
and Linux.

------------------------------------------------------------------------
r3386 | phk | 2008-11-11 20:06:55 +0100 (ti., 11 nov. 2008) | 5 lines

Implement restart in vcl_hit.

Fixes #365

------------------------------------------------------------------------
r3387 | phk | 2008-11-11 20:07:30 +0100 (ti., 11 nov. 2008) | 3 lines

Regression test case for ticket 365: restart in hit.


------------------------------------------------------------------------
r3388 | phk | 2008-11-11 21:22:05 +0100 (ti., 11 nov. 2008) | 8 lines

Make sure that
        set obj.ttl = 0s
means that the object is not hit again by actually using "-1" instead.

This works around the rounding error which otherwise causes the object
to be inside TTL for up to one second - epsilon.

------------------------------------------------------------------------
r3401 | tfheen | 2008-11-18 14:29:34 +0100 (ti., 18 nov. 2008) | 5 lines

Make malloc print max storage size

storage_file prints the maximum storage size, make malloc do the same,
for consistency.

------------------------------------------------------------------------
r3402 | tfheen | 2008-11-18 21:53:03 +0100 (ti., 18 nov. 2008) | 5 lines

Document grace

Thanks to perbu for suggested documentation
Fixes 355

------------------------------------------------------------------------
r3407 | tfheen | 2008-11-19 17:26:16 +0100 (on., 19 nov. 2008) | 4 lines

Correct defaults in varnishd.1

The defaults for thread_pool_min and thread_pools were wrong; fixed.

------------------------------------------------------------------------
r3408 | phk | 2008-11-20 09:50:56 +0100 (to., 20 nov. 2008) | 3 lines

Check ECONNRESET

r3418 | tfheen | 2008-11-22 02:35:16 +0100 (l?., 22 nov. 2008) | 2 lines

Fix typo

------------------------------------------------------------------------
r3420 | phk | 2008-11-24 11:05:55 +0100 (ma., 24 nov. 2008) | 5 lines

Update license to remove the advertising clause, reflecting similar change
in the FreeBSD original.

Approved by:    des
------------------------------------------------------------------------
r3426 | tfheen | 2008-11-24 15:04:42 +0100 (ma., 24 nov. 2008) | 4 lines

Fix typo (s/timeout/interval/) in default parameters for backend health

Thanks to Jonny @ globo for noticing.

------------------------------------------------------------------------
r3463 | ingvar | 2008-12-18 10:47:05 +0100 (to., 18 des. 2008) | 1 line

Changed rpm summary string as requested by the Fedora project
------------------------------------------------------------------------
r3466 | phk | 2008-12-18 11:31:08 +0100 (to., 18 des. 2008) | 5 lines

Add a missing error check.

Fixes #409

------------------------------------------------------------------------
r3488 | phk | 2008-12-21 19:33:30 +0100 (s?., 21 des. 2008) | 3 lines

Assert fcntl(2) have not failed.

------------------------------------------------------------------------
r3490 | phk | 2008-12-21 19:34:02 +0100 (s?., 21 des. 2008) | 3 lines

Assert that realloc() did.

------------------------------------------------------------------------
r3492 | phk | 2008-12-21 19:35:24 +0100 (s?., 21 des. 2008) | 4 lines

Be more stringent about our timeout: fail even if we did get a
socket, but took to.

------------------------------------------------------------------------

I'm also wondering if we should merge the ?backend timeout patch?,
namely 3406, 3411, 3412:

r3406 | petter | 2008-11-19 15:13:57 +0100 (on., 19 nov. 2008) | 8 lines

Added support for setting read timeouts for backend requests
(first_byte_timeout and between_bytes_timeout), in addition to make the
connect_timeout available for the bereq object in vcl_miss and
vcl_fetch.

first_byte_timeout is a read timeout from the connection to the backend
is created to when the first byte arrives. It can be set as a parameter
to varnish, as a field in the backend declaration or as
bereq.first_byte_timeout in vcl_miss and vcl_pass.

between_bytes_timeout is a read timeout between each read from the
backend. It can be set as a parameter to varnish, as a field in the
backend declaration or as bereq.between_bytes_timeout in vcl_miss and
vcl_pass.

The time unit for these timeout values are seconds. NOTE: The
connect_timeout previously used milliseconds as time unit, so beware.

------------------------------------------------------------------------
r3411 | petter | 2008-11-20 12:01:26 +0100 (to., 20 nov. 2008) | 1 line

Added documentaiton on the timeouts
------------------------------------------------------------------------
r3412 | petter | 2008-11-21 08:21:13 +0100 (fr., 21 nov. 2008) | 2 lines

Soooorry.

Comments?  Thoughts?

-- 
Tollef Fog Heen 
Redpill Linpro -- Changing the game!
t: +47 21 54 41 73


From tfheen at linpro.no  Thu Jan  8 10:29:31 2009
From: tfheen at linpro.no (Tollef Fog Heen)
Date: Thu, 08 Jan 2009 11:29:31 +0100
Subject: 2.1 plans
Message-ID: <8763kqgkfo.fsf@qurzaw.linpro.no>


Hi,

a short while before Christmas, I wrote up a small document pointing to
what I would like to get into 2.1 and when I'd like milestones to
happen.  This is a suggestion, I'm open to ideas and comments on both
feature set as well as if my guesstimates for dates is completely off:

Varnish 2.1 release plan

The theme for Varnish 2.1 is "scalability", particularly trying to
address the needs of sites like finn.no which has a lot of objects and
where priming the cache takes a long time, leading to long periods of
higher load on the backend servers.

The main feature is persistent storage, see
http://varnish.projects.linpro.no/wiki/ArchitecturePersistentStorage
for design notes. Another important scalability feature is a new
lockless hash algorithm which scales much better than the current
one.  Poul-Henning already has an implementation of this in the tree,
but it's still fresh.

Minor features which would be nice to get in are:

* Web UI, showing pretty graphs as well as allowing easy configuration
  of a cluster of Varnish machines.

* Expiry randomisation.  This reduces the "lemmings" effect where you
  end up with a many objects with almost the same TTL (typically on
  startup) which then expire at the same time.  The feature will allow
  you to set the TTL to plus/minus X %.

* Dynamic, user-defined counters that can be read and written from VCL

* Forced purges, where a thread walks the list of purged objects and
  removes them.

The schedule
------------

Alphas:
 - 2009-01-15: New hash algorithm working
 - 2009-02-15: Web UI
 - 2009-03-15: Persistent storage
Beta:
 - 2009-04-01: Feature complete
Release
 - 2009-05-20: Release candidate
 - 2009-05-01: No release critical bugs left
 - 2009-05-10: Release

-- 
Tollef Fog Heen 
Redpill Linpro -- Changing the game!
t: +47 21 54 41 73


From jfbustarret at wat.tv  Thu Jan  8 15:57:42 2009
From: jfbustarret at wat.tv (BUSTARRET, Jean-francois)
Date: Thu, 8 Jan 2009 16:57:42 +0100
Subject: 2.1 plans
In-Reply-To: <8763kqgkfo.fsf@qurzaw.linpro.no>
References: <8763kqgkfo.fsf@qurzaw.linpro.no>
Message-ID: <53C652A09719C54DA24741D0157CB26902994692@TFPRDEXS1.tf1.groupetf1.fr>


Great news !
 
What about more ESI features (ie : cookie support), backend revalidation with conditional GETs or streaming fetches (http://varnish.projects.linpro.no/wiki/PostTwoShoppingList) ?

Jean-Fran?ois

> -----Message d'origine-----
> De : varnish-misc-bounces at projects.linpro.no 
> [mailto:varnish-misc-bounces at projects.linpro.no] De la part 
> de Tollef Fog Heen
> Envoy? : jeudi 8 janvier 2009 11:30
> ? : varnish-misc at projects.linpro.no
> Objet : 2.1 plans
> 
> 
> Hi,
> 
> a short while before Christmas, I wrote up a small document 
> pointing to
> what I would like to get into 2.1 and when I'd like milestones to
> happen.  This is a suggestion, I'm open to ideas and comments on both
> feature set as well as if my guesstimates for dates is completely off:
> 
> Varnish 2.1 release plan
> 
> The theme for Varnish 2.1 is "scalability", particularly trying to
> address the needs of sites like finn.no which has a lot of objects and
> where priming the cache takes a long time, leading to long periods of
> higher load on the backend servers.
> 
> The main feature is persistent storage, see
> http://varnish.projects.linpro.no/wiki/ArchitecturePersistentStorage
> for design notes. Another important scalability feature is a new
> lockless hash algorithm which scales much better than the current
> one.  Poul-Henning already has an implementation of this in the tree,
> but it's still fresh.
> 
> Minor features which would be nice to get in are:
> 
> * Web UI, showing pretty graphs as well as allowing easy configuration
>   of a cluster of Varnish machines.
> 
> * Expiry randomisation.  This reduces the "lemmings" effect where you
>   end up with a many objects with almost the same TTL (typically on
>   startup) which then expire at the same time.  The feature will allow
>   you to set the TTL to plus/minus X %.
> 
> * Dynamic, user-defined counters that can be read and written from VCL
> 
> * Forced purges, where a thread walks the list of purged objects and
>   removes them.
> 
> The schedule
> ------------
> 
> Alphas:
>  - 2009-01-15: New hash algorithm working
>  - 2009-02-15: Web UI
>  - 2009-03-15: Persistent storage
> Beta:
>  - 2009-04-01: Feature complete
> Release
>  - 2009-05-20: Release candidate
>  - 2009-05-01: No release critical bugs left
>  - 2009-05-10: Release
> 
> -- 
> Tollef Fog Heen 
> Redpill Linpro -- Changing the game!
> t: +47 21 54 41 73
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-misc
> 


From michael at dynamine.net  Thu Jan  8 16:51:01 2009
From: michael at dynamine.net (Michael S. Fischer)
Date: Thu, 8 Jan 2009 08:51:01 -0800
Subject: 2.1 plans
In-Reply-To: <8763kqgkfo.fsf@qurzaw.linpro.no>
References: <8763kqgkfo.fsf@qurzaw.linpro.no>
Message-ID: <A33CD9C2-83E0-4175-A02D-D1C7D7081FCE@dynamine.net>

What about CARP-like cache routing (i.e., where multiple cache servers  
themselves are hash buckets)?  This would go a LONG way towards  
scalability.

--Michael

On Jan 8, 2009, at 2:29 AM, Tollef Fog Heen wrote:

>
> Hi,
>
> a short while before Christmas, I wrote up a small document pointing  
> to
> what I would like to get into 2.1 and when I'd like milestones to
> happen.  This is a suggestion, I'm open to ideas and comments on both
> feature set as well as if my guesstimates for dates is completely off:
>
> Varnish 2.1 release plan
>
> The theme for Varnish 2.1 is "scalability", particularly trying to
> address the needs of sites like finn.no which has a lot of objects and
> where priming the cache takes a long time, leading to long periods of
> higher load on the backend servers.
>
> The main feature is persistent storage, see
> http://varnish.projects.linpro.no/wiki/ArchitecturePersistentStorage
> for design notes. Another important scalability feature is a new
> lockless hash algorithm which scales much better than the current
> one.  Poul-Henning already has an implementation of this in the tree,
> but it's still fresh.
>
> Minor features which would be nice to get in are:
>
> * Web UI, showing pretty graphs as well as allowing easy configuration
>  of a cluster of Varnish machines.
>
> * Expiry randomisation.  This reduces the "lemmings" effect where you
>  end up with a many objects with almost the same TTL (typically on
>  startup) which then expire at the same time.  The feature will allow
>  you to set the TTL to plus/minus X %.
>
> * Dynamic, user-defined counters that can be read and written from VCL
>
> * Forced purges, where a thread walks the list of purged objects and
>  removes them.
>
> The schedule
> ------------
>
> Alphas:
> - 2009-01-15: New hash algorithm working
> - 2009-02-15: Web UI
> - 2009-03-15: Persistent storage
> Beta:
> - 2009-04-01: Feature complete
> Release
> - 2009-05-20: Release candidate
> - 2009-05-01: No release critical bugs left
> - 2009-05-10: Release
>
> -- 
> Tollef Fog Heen
> Redpill Linpro -- Changing the game!
> t: +47 21 54 41 73
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-misc



From jeff at funnyordie.com  Thu Jan  8 18:33:05 2009
From: jeff at funnyordie.com (Jeff Anderson)
Date: Thu, 8 Jan 2009 10:33:05 -0800
Subject: 2.1 plans
In-Reply-To: <8763kqgkfo.fsf@qurzaw.linpro.no>
References: <8763kqgkfo.fsf@qurzaw.linpro.no>
Message-ID: <5B4E5CA0-25B0-42B0-831E-2BFCCE606FD7@funnyordie.com>

I'd like to see individual object request statistics and a method to  
prefetch objects from the backend that are most frequently requested.   
Perhaps also a way to prioritize objects into cache tiers based on  
frequency of requests.  So, for example, highly requested objects are  
maintained in RAM and less frequently requested objects are cached to  
disk.  If persistent storage is on its way maybe a method to assign  
priority to large disk cache volumes versus memory regions.  It might  
be nice to have a distributed and/or tiered cache model where a single  
master has a very large cache and potentially very long grace ability  
where objects can exist even if stale.  That master in turn could host  
frontend caches that communicate  efficiently to the master cache and  
also have a tiered internal object priority.

Thanks,
--Jeff


On Jan 8, 2009, at 2:29 AM, Tollef Fog Heen wrote:

>
> Hi,
>
> a short while before Christmas, I wrote up a small document pointing  
> to
> what I would like to get into 2.1 and when I'd like milestones to
> happen.  This is a suggestion, I'm open to ideas and comments on both
> feature set as well as if my guesstimates for dates is completely off:
>
> Varnish 2.1 release plan
>
> The theme for Varnish 2.1 is "scalability", particularly trying to
> address the needs of sites like finn.no which has a lot of objects and
> where priming the cache takes a long time, leading to long periods of
> higher load on the backend servers.
>
> The main feature is persistent storage, see
> http://varnish.projects.linpro.no/wiki/ArchitecturePersistentStorage
> for design notes. Another important scalability feature is a new
> lockless hash algorithm which scales much better than the current
> one.  Poul-Henning already has an implementation of this in the tree,
> but it's still fresh.
>
> Minor features which would be nice to get in are:
>
> * Web UI, showing pretty graphs as well as allowing easy configuration
>  of a cluster of Varnish machines.
>
> * Expiry randomisation.  This reduces the "lemmings" effect where you
>  end up with a many objects with almost the same TTL (typically on
>  startup) which then expire at the same time.  The feature will allow
>  you to set the TTL to plus/minus X %.
>
> * Dynamic, user-defined counters that can be read and written from VCL
>
> * Forced purges, where a thread walks the list of purged objects and
>  removes them.
>
> The schedule
> ------------
>
> Alphas:
> - 2009-01-15: New hash algorithm working
> - 2009-02-15: Web UI
> - 2009-03-15: Persistent storage
> Beta:
> - 2009-04-01: Feature complete
> Release
> - 2009-05-20: Release candidate
> - 2009-05-01: No release critical bugs left
> - 2009-05-10: Release
>
> -- 
> Tollef Fog Heen
> Redpill Linpro -- Changing the game!
> t: +47 21 54 41 73
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-misc

--Jeff
jeff at funnyordie.com





From tim at metaweb.com  Thu Jan  8 18:33:14 2009
From: tim at metaweb.com (Tim Kientzle)
Date: Thu, 8 Jan 2009 10:33:14 -0800
Subject: 2.0.3 planning
In-Reply-To: <87aba2gl8n.fsf@qurzaw.linpro.no>
References: <87aba2gl8n.fsf@qurzaw.linpro.no>
Message-ID: <328DC4EA-FA55-4583-9752-5086AE963A2F@metaweb.com>

This is a very strange comment.  If Varnish requires a
particular sequence, it should implement its own.  If
it requires particular statistical properties, it should
test for those, not test for a specific sequence.

Tim


On Jan 8, 2009, at 2:12 AM, Tollef Fog Heen wrote:

> ------------------------------------------------------------------------
> r3367 | phk | 2008-11-10 10:37:21 +0100 (ma., 10 nov. 2008) | 14 lines
>
> Add a toplevel word which examines the sequence returned by
> srandom(1) and stops the test if we do not get the same sequence
> as we expect.
>
> The Open Group does not define which deterministic sequence srandom(1)
> should result in, on that it be deterministic, but I have high hopes
> in the general sanity and expect that UNIX people across the board
> have realized that for portability the same sequence should be
> returned on all platforms.
>
> At the very least FreeBSD and Linux/GLIBC, as seen on  
> projects.linpro.no,
> agree.



From perbu at linpro.no  Thu Jan  8 18:54:16 2009
From: perbu at linpro.no (Per Buer)
Date: Thu, 08 Jan 2009 19:54:16 +0100
Subject: 2.1 plans
In-Reply-To: <5B4E5CA0-25B0-42B0-831E-2BFCCE606FD7@funnyordie.com>
References: <8763kqgkfo.fsf@qurzaw.linpro.no>
	<5B4E5CA0-25B0-42B0-831E-2BFCCE606FD7@funnyordie.com>
Message-ID: <49664BD8.1090506@linpro.no>

Jeff Anderson wrote:
> I'd like to see individual object request statistics and a method to  
> prefetch objects from the backend that are most frequently requested.   
> Perhaps also a way to prioritize objects into cache tiers based on  
> frequency of requests.  So, for example, highly requested objects are  
> maintained in RAM and less frequently requested objects are cached to  
> disk.  

Your operating system already does this today with Varnish. Squid tries
to maintain a two tier cache hierarchy without success.

> If persistent storage is on its way maybe a method to assign  
> priority to large disk cache volumes versus memory regions.

Noted.

> It might  be nice to have a distributed and/or tiered cache model where a single  
> master has a very large cache and potentially very long grace ability  
> where objects can exist even if stale.  That master in turn could host  
> frontend caches that communicate  efficiently to the master cache and  
> also have a tiered internal object priority.

I believe most of this can be achieved today. Stale objects will
hopefully reach the 2.0 series before the 2.1 revolutions - at least as
a patch, I hope.

> 
> Thanks,
> --Jeff
> 
> 
> On Jan 8, 2009, at 2:29 AM, Tollef Fog Heen wrote:
> 
>> Hi,
>>
>> a short while before Christmas, I wrote up a small document pointing  
>> to
>> what I would like to get into 2.1 and when I'd like milestones to
>> happen.  This is a suggestion, I'm open to ideas and comments on both
>> feature set as well as if my guesstimates for dates is completely off:
>>
>> Varnish 2.1 release plan
>>
>> The theme for Varnish 2.1 is "scalability", particularly trying to
>> address the needs of sites like finn.no which has a lot of objects and
>> where priming the cache takes a long time, leading to long periods of
>> higher load on the backend servers.
>>
>> The main feature is persistent storage, see
>> http://varnish.projects.linpro.no/wiki/ArchitecturePersistentStorage
>> for design notes. Another important scalability feature is a new
>> lockless hash algorithm which scales much better than the current
>> one.  Poul-Henning already has an implementation of this in the tree,
>> but it's still fresh.
>>
>> Minor features which would be nice to get in are:
>>
>> * Web UI, showing pretty graphs as well as allowing easy configuration
>>  of a cluster of Varnish machines.
>>
>> * Expiry randomisation.  This reduces the "lemmings" effect where you
>>  end up with a many objects with almost the same TTL (typically on
>>  startup) which then expire at the same time.  The feature will allow
>>  you to set the TTL to plus/minus X %.
>>
>> * Dynamic, user-defined counters that can be read and written from VCL
>>
>> * Forced purges, where a thread walks the list of purged objects and
>>  removes them.
>>
>> The schedule
>> ------------
>>
>> Alphas:
>> - 2009-01-15: New hash algorithm working
>> - 2009-02-15: Web UI
>> - 2009-03-15: Persistent storage
>> Beta:
>> - 2009-04-01: Feature complete
>> Release
>> - 2009-05-20: Release candidate
>> - 2009-05-01: No release critical bugs left
>> - 2009-05-10: Release
>>
>> -- 
>> Tollef Fog Heen
>> Redpill Linpro -- Changing the game!
>> t: +47 21 54 41 73
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at projects.linpro.no
>> http://projects.linpro.no/mailman/listinfo/varnish-misc
> 
> --Jeff
> jeff at funnyordie.com
> 
> 
> 
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-misc


-- 
Per Buer - Leder Infrastruktur og Drift - Redpill Linpro
Telefon: 21 54 41 21 - Mobil: 958 39 117
http://linpro.no/ | http://redpill.se/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 260 bytes
Desc: OpenPGP digital signature
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20090108/3721977f/attachment.bin>

From perbu at linpro.no  Thu Jan  8 19:00:21 2009
From: perbu at linpro.no (Per Buer)
Date: Thu, 08 Jan 2009 20:00:21 +0100
Subject: 2.1 plans
In-Reply-To: <8763kqgkfo.fsf@qurzaw.linpro.no>
References: <8763kqgkfo.fsf@qurzaw.linpro.no>
Message-ID: <49664D45.8050502@linpro.no>

Hi,

Tollef Fog Heen wrote:
> Hi,
> 
> a short while before Christmas, I wrote up a small document pointing to
> what I would like to get into 2.1 and when I'd like milestones to
> happen.  This is a suggestion, I'm open to ideas and comments on both
> feature set as well as if my guesstimates for dates is completely off:
> 
> Varnish 2.1 release plan
>  (..)

As the money man in the project I would like to say that if anybody
misses anything on the feature list we are happy to talk to sponsors
about it. Send me or varnish at linpro.no an email and we'll discuss it.
Your feature might not go into the mainline right away but we'll be
happy to support a patched version of varnish until we are able to put
the patch into a proper release.


Regards

Per.

-- 
http://linpro.no/ | http://redpill.se/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 260 bytes
Desc: OpenPGP digital signature
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20090108/70ce556b/attachment.bin>

From jeff at funnyordie.com  Thu Jan  8 19:25:14 2009
From: jeff at funnyordie.com (Jeff Anderson)
Date: Thu, 8 Jan 2009 11:25:14 -0800
Subject: 2.1 plans
In-Reply-To: <49664BD8.1090506@linpro.no>
References: <8763kqgkfo.fsf@qurzaw.linpro.no>
	<5B4E5CA0-25B0-42B0-831E-2BFCCE606FD7@funnyordie.com>
	<49664BD8.1090506@linpro.no>
Message-ID: <86E86AAD-FBED-4958-9EC7-62C69AE33219@funnyordie.com>

Thanks for the response.

I think inline page compression would be great too.  Store gzipped  
objects in the persistent cache and unzip if uncompressed objects are  
requested.




On Jan 8, 2009, at 10:54 AM, Per Buer wrote:

> Jeff Anderson wrote:
>> I'd like to see individual object request statistics and a method to
>> prefetch objects from the backend that are most frequently requested.
>> Perhaps also a way to prioritize objects into cache tiers based on
>> frequency of requests.  So, for example, highly requested objects are
>> maintained in RAM and less frequently requested objects are cached to
>> disk.
>
> Your operating system already does this today with Varnish. Squid  
> tries
> to maintain a two tier cache hierarchy without success.
>
>> If persistent storage is on its way maybe a method to assign
>> priority to large disk cache volumes versus memory regions.
>
> Noted.
>
>> It might  be nice to have a distributed and/or tiered cache model  
>> where a single
>> master has a very large cache and potentially very long grace ability
>> where objects can exist even if stale.  That master in turn could  
>> host
>> frontend caches that communicate  efficiently to the master cache and
>> also have a tiered internal object priority.
>
> I believe most of this can be achieved today. Stale objects will
> hopefully reach the 2.0 series before the 2.1 revolutions - at least  
> as
> a patch, I hope.
>
>>
>> Thanks,
>> --Jeff
>>
>>
>> On Jan 8, 2009, at 2:29 AM, Tollef Fog Heen wrote:
>>
>>> Hi,
>>>
>>> a short while before Christmas, I wrote up a small document pointing
>>> to
>>> what I would like to get into 2.1 and when I'd like milestones to
>>> happen.  This is a suggestion, I'm open to ideas and comments on  
>>> both
>>> feature set as well as if my guesstimates for dates is completely  
>>> off:
>>>
>>> Varnish 2.1 release plan
>>>
>>> The theme for Varnish 2.1 is "scalability", particularly trying to
>>> address the needs of sites like finn.no which has a lot of objects  
>>> and
>>> where priming the cache takes a long time, leading to long periods  
>>> of
>>> higher load on the backend servers.
>>>
>>> The main feature is persistent storage, see
>>> http://varnish.projects.linpro.no/wiki/ArchitecturePersistentStorage
>>> for design notes. Another important scalability feature is a new
>>> lockless hash algorithm which scales much better than the current
>>> one.  Poul-Henning already has an implementation of this in the  
>>> tree,
>>> but it's still fresh.
>>>
>>> Minor features which would be nice to get in are:
>>>
>>> * Web UI, showing pretty graphs as well as allowing easy  
>>> configuration
>>> of a cluster of Varnish machines.
>>>
>>> * Expiry randomisation.  This reduces the "lemmings" effect where  
>>> you
>>> end up with a many objects with almost the same TTL (typically on
>>> startup) which then expire at the same time.  The feature will allow
>>> you to set the TTL to plus/minus X %.
>>>
>>> * Dynamic, user-defined counters that can be read and written from  
>>> VCL
>>>
>>> * Forced purges, where a thread walks the list of purged objects and
>>> removes them.
>>>
>>> The schedule
>>> ------------
>>>
>>> Alphas:
>>> - 2009-01-15: New hash algorithm working
>>> - 2009-02-15: Web UI
>>> - 2009-03-15: Persistent storage
>>> Beta:
>>> - 2009-04-01: Feature complete
>>> Release
>>> - 2009-05-20: Release candidate
>>> - 2009-05-01: No release critical bugs left
>>> - 2009-05-10: Release
>>>
>>> -- 
>>> Tollef Fog Heen
>>> Redpill Linpro -- Changing the game!
>>> t: +47 21 54 41 73
>>> _______________________________________________
>>> varnish-misc mailing list
>>> varnish-misc at projects.linpro.no
>>> http://projects.linpro.no/mailman/listinfo/varnish-misc
>>
>> --Jeff
>> jeff at funnyordie.com
>>
>>
>>
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at projects.linpro.no
>> http://projects.linpro.no/mailman/listinfo/varnish-misc
>
>
> -- 
> Per Buer - Leder Infrastruktur og Drift - Redpill Linpro
> Telefon: 21 54 41 21 - Mobil: 958 39 117
> http://linpro.no/ | http://redpill.se/
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-misc

--Jeff
jeff at funnyordie.com





From michael at dynamine.net  Thu Jan  8 19:28:32 2009
From: michael at dynamine.net (Michael S. Fischer)
Date: Thu, 8 Jan 2009 11:28:32 -0800
Subject: 2.1 plans
In-Reply-To: <86E86AAD-FBED-4958-9EC7-62C69AE33219@funnyordie.com>
References: <8763kqgkfo.fsf@qurzaw.linpro.no>
	<5B4E5CA0-25B0-42B0-831E-2BFCCE606FD7@funnyordie.com>
	<49664BD8.1090506@linpro.no>
	<86E86AAD-FBED-4958-9EC7-62C69AE33219@funnyordie.com>
Message-ID: <3A5C2CC6-16C3-4195-8CE2-4A3284D3E22D@dynamine.net>

+1.  This is a very good idea for optimizing RAM utilization.

--Michael

On Jan 8, 2009, at 11:25 AM, Jeff Anderson wrote:

> Thanks for the response.
>
> I think inline page compression would be great too.  Store gzipped
> objects in the persistent cache and unzip if uncompressed objects are
> requested.
>
>
>
>
> On Jan 8, 2009, at 10:54 AM, Per Buer wrote:
>
>> Jeff Anderson wrote:
>>> I'd like to see individual object request statistics and a method to
>>> prefetch objects from the backend that are most frequently  
>>> requested.
>>> Perhaps also a way to prioritize objects into cache tiers based on
>>> frequency of requests.  So, for example, highly requested objects  
>>> are
>>> maintained in RAM and less frequently requested objects are cached  
>>> to
>>> disk.
>>
>> Your operating system already does this today with Varnish. Squid
>> tries
>> to maintain a two tier cache hierarchy without success.
>>
>>> If persistent storage is on its way maybe a method to assign
>>> priority to large disk cache volumes versus memory regions.
>>
>> Noted.
>>
>>> It might  be nice to have a distributed and/or tiered cache model
>>> where a single
>>> master has a very large cache and potentially very long grace  
>>> ability
>>> where objects can exist even if stale.  That master in turn could
>>> host
>>> frontend caches that communicate  efficiently to the master cache  
>>> and
>>> also have a tiered internal object priority.
>>
>> I believe most of this can be achieved today. Stale objects will
>> hopefully reach the 2.0 series before the 2.1 revolutions - at least
>> as
>> a patch, I hope.
>>
>>>
>>> Thanks,
>>> --Jeff
>>>
>>>
>>> On Jan 8, 2009, at 2:29 AM, Tollef Fog Heen wrote:
>>>
>>>> Hi,
>>>>
>>>> a short while before Christmas, I wrote up a small document  
>>>> pointing
>>>> to
>>>> what I would like to get into 2.1 and when I'd like milestones to
>>>> happen.  This is a suggestion, I'm open to ideas and comments on
>>>> both
>>>> feature set as well as if my guesstimates for dates is completely
>>>> off:
>>>>
>>>> Varnish 2.1 release plan
>>>>
>>>> The theme for Varnish 2.1 is "scalability", particularly trying to
>>>> address the needs of sites like finn.no which has a lot of objects
>>>> and
>>>> where priming the cache takes a long time, leading to long periods
>>>> of
>>>> higher load on the backend servers.
>>>>
>>>> The main feature is persistent storage, see
>>>> http://varnish.projects.linpro.no/wiki/ 
>>>> ArchitecturePersistentStorage
>>>> for design notes. Another important scalability feature is a new
>>>> lockless hash algorithm which scales much better than the current
>>>> one.  Poul-Henning already has an implementation of this in the
>>>> tree,
>>>> but it's still fresh.
>>>>
>>>> Minor features which would be nice to get in are:
>>>>
>>>> * Web UI, showing pretty graphs as well as allowing easy
>>>> configuration
>>>> of a cluster of Varnish machines.
>>>>
>>>> * Expiry randomisation.  This reduces the "lemmings" effect where
>>>> you
>>>> end up with a many objects with almost the same TTL (typically on
>>>> startup) which then expire at the same time.  The feature will  
>>>> allow
>>>> you to set the TTL to plus/minus X %.
>>>>
>>>> * Dynamic, user-defined counters that can be read and written from
>>>> VCL
>>>>
>>>> * Forced purges, where a thread walks the list of purged objects  
>>>> and
>>>> removes them.
>>>>
>>>> The schedule
>>>> ------------
>>>>
>>>> Alphas:
>>>> - 2009-01-15: New hash algorithm working
>>>> - 2009-02-15: Web UI
>>>> - 2009-03-15: Persistent storage
>>>> Beta:
>>>> - 2009-04-01: Feature complete
>>>> Release
>>>> - 2009-05-20: Release candidate
>>>> - 2009-05-01: No release critical bugs left
>>>> - 2009-05-10: Release
>>>>
>>>> -- 
>>>> Tollef Fog Heen
>>>> Redpill Linpro -- Changing the game!
>>>> t: +47 21 54 41 73
>>>> _______________________________________________
>>>> varnish-misc mailing list
>>>> varnish-misc at projects.linpro.no
>>>> http://projects.linpro.no/mailman/listinfo/varnish-misc
>>>
>>> --Jeff
>>> jeff at funnyordie.com
>>>
>>>
>>>
>>> _______________________________________________
>>> varnish-misc mailing list
>>> varnish-misc at projects.linpro.no
>>> http://projects.linpro.no/mailman/listinfo/varnish-misc
>>
>>
>> -- 
>> Per Buer - Leder Infrastruktur og Drift - Redpill Linpro
>> Telefon: 21 54 41 21 - Mobil: 958 39 117
>> http://linpro.no/ | http://redpill.se/
>>
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at projects.linpro.no
>> http://projects.linpro.no/mailman/listinfo/varnish-misc
>
> --Jeff
> jeff at funnyordie.com
>
>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-misc



From tfheen at linpro.no  Fri Jan  9 09:47:50 2009
From: tfheen at linpro.no (Tollef Fog Heen)
Date: Fri, 09 Jan 2009 10:47:50 +0100
Subject: 2.1 plans
In-Reply-To: <53C652A09719C54DA24741D0157CB26902994692@TFPRDEXS1.tf1.groupetf1.fr>
	(Jean-francois BUSTARRET's message of "Thu, 8 Jan 2009 16:57:42
	+0100")
References: <8763kqgkfo.fsf@qurzaw.linpro.no>
	<53C652A09719C54DA24741D0157CB26902994692@TFPRDEXS1.tf1.groupetf1.fr>
Message-ID: <87ocygx12x.fsf@qurzaw.linpro.no>

]] "BUSTARRET, Jean-francois" 

| What about more ESI features (ie : cookie support), backend
| revalidation with conditional GETs or streaming fetches
| (http://varnish.projects.linpro.no/wiki/PostTwoShoppingList) ?

More ESI support would be interesting too.  However, we have a limited
amount of manpower and in order for 2.1 to actually release sometime
this summer rather than next summer, I'd like us to limit the feature
list somewhat.  If patches show up, I'm sure we can find the time to
integrate those, though. :-)

-- 
Tollef Fog Heen 
Redpill Linpro -- Changing the game!
t: +47 21 54 41 73


From tfheen at linpro.no  Fri Jan  9 09:59:35 2009
From: tfheen at linpro.no (Tollef Fog Heen)
Date: Fri, 09 Jan 2009 10:59:35 +0100
Subject: 2.1 plans
In-Reply-To: <A33CD9C2-83E0-4175-A02D-D1C7D7081FCE@dynamine.net> (Michael
	S. Fischer's message of "Thu, 8 Jan 2009 08:51:01 -0800")
References: <8763kqgkfo.fsf@qurzaw.linpro.no>
	<A33CD9C2-83E0-4175-A02D-D1C7D7081FCE@dynamine.net>
Message-ID: <87k594x0jc.fsf@qurzaw.linpro.no>

]] "Michael S. Fischer" 

| What about CARP-like cache routing (i.e., where multiple cache servers
| themselves are hash buckets)?  This would go a LONG way towards
| scalability.

http://varnish.projects.linpro.no/wiki/PostTwoShoppingList second item
sounds like what you want?

-- 
Tollef Fog Heen 
Redpill Linpro -- Changing the game!
t: +47 21 54 41 73


From phk at phk.freebsd.dk  Fri Jan  9 11:40:59 2009
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Fri, 09 Jan 2009 11:40:59 +0000
Subject: 2.0.3 planning 
In-Reply-To: Your message of "Thu, 08 Jan 2009 11:12:08 +0100."
	<87aba2gl8n.fsf@qurzaw.linpro.no> 
Message-ID: <20870.1231501259@critter.freebsd.dk>


I hate to sound old-fashioned, but it would probably be faster for me
to look at the diff of stuff you do _not_ want to merge...

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From plfgoa at gmail.com  Fri Jan  9 12:22:58 2009
From: plfgoa at gmail.com (Paras Fadte)
Date: Fri, 9 Jan 2009 17:52:58 +0530
Subject: Fwd: Panic message : Varnish restart
In-Reply-To: <75cf5800901012037u2d6d2a60q9d1088ea2f10d1@mail.gmail.com>
References: <75cf5800901012037u2d6d2a60q9d1088ea2f10d1@mail.gmail.com>
Message-ID: <75cf5800901090422g7c28e7car4583ad1abdeeb103@mail.gmail.com>

Hi,

Can anybody please respond to the following  query of mine ?

Thank you.

-Paras

---------- Forwarded message ----------
From: Paras Fadte <plfgoa at gmail.com>
Date: Fri, Jan 2, 2009 at 10:07 AM
Subject: Panic message : Varnish restart
To: "varnish-misc at projects.linpro.no" <varnish-misc at projects.linpro.no>


Hi,

I got a message like following in /var/log/messages and child
restarted. what could be the issue ?

varnishd[28876]: Child (31253) Panic message: Missing errorhandling
code in fetch_chunked(), cache_fetch.c line 113:   Condition(be >
 bp) not true.  thread = (cache-worker)sp = 0x2aabd2970008 {   fd =
302, id = 302, xid = 1800279969,

Varnish Version is :

varnishd (varnish-trunk)
Copyright (c) 2006-2008 Linpro AS / Verdens Gang AS

Thanks in advance.

-Paras


From tfheen at linpro.no  Fri Jan  9 12:37:26 2009
From: tfheen at linpro.no (Tollef Fog Heen)
Date: Fri, 09 Jan 2009 13:37:26 +0100
Subject: Dropping some mailing lists?
Message-ID: <87y6xkvent.fsf@qurzaw.linpro.no>


(Please reply only to -misc so we don't end up with a zillion copies of
this all over the lists.)

Hi,

we currently have very many mailing lists:

varnish-announce  	Release announcements and other important news.
varnish-bugs 	Bug reports go here.
varnish-commit 	Commit messages go here.
varnish-dev 	Discussions regarding Varnish development.
varnish-dist 	Discussions regarding Varnish releases and packaging
varnish-misc 	Discussions on miscellaneous topics relating to Varnish
varnish-test 	For testing purposes.

At the same time, we don't have that much traffic, so having a total of
seven mailing lists seem a bit excessive.

I suggest we get rid of -dev, -dist and -test.  -announce and -misc will
be kept as discussion lists.  -bugs and -commit will continue as
trac-to-email lists (i.e. not discussion lists).

Comments?

-- 
Tollef Fog Heen 
Redpill Linpro -- Changing the game!
t: +47 21 54 41 73


From f.engelhardt at 21torr.com  Fri Jan  9 12:37:40 2009
From: f.engelhardt at 21torr.com (f.engelhardt at 21torr.com)
Date: Fri, 9 Jan 2009 13:37:40 +0100
Subject: Dropping some mailing lists?
Message-ID: <B0068700293@mail.21torr.com>

Vielen Dank fuer Ihre Nachricht. Ich bin vom 23. Dezember 2008 bis zum 11. Januar 2009 au?er Haus und habe keinen Zugang zu meinen E-Mails. Wenden Sie sich daher in dringenden Faellen bitte an Jochen Hild:
j.hild at 21torr.com
Telefon +49 (0) 7121 / 3 48 - 220


Thank you for your message. I am out of office from the 23. Dezember 2008 until the 11. January 2009 and do not have e-mail access. In urgent cases please contact Jochen Hild:
j.hild at 21torr.com
fon: +49 (0) 7121 / 3 48 - 220

-- 
florian engelhardt

21TORR AGENCY gmbh
engineering unit

heinestrasse 72
d-72762 reutlingen
phone +49-7121-348-281
fax +49-7121-348-259
f.engelhardt at 21torr.com

21TORR - die neue medien leidenschaft
www.21torr.com




From michael at dynamine.net  Fri Jan  9 16:49:15 2009
From: michael at dynamine.net (Michael S. Fischer)
Date: Fri, 9 Jan 2009 08:49:15 -0800
Subject: 2.1 plans
In-Reply-To: <87k594x0jc.fsf@qurzaw.linpro.no>
References: <8763kqgkfo.fsf@qurzaw.linpro.no>
	<A33CD9C2-83E0-4175-A02D-D1C7D7081FCE@dynamine.net>
	<87k594x0jc.fsf@qurzaw.linpro.no>
Message-ID: <53C6394F-29DC-4D44-8606-2FB8416A3DEA@dynamine.net>

On Jan 9, 2009, at 1:59 AM, Tollef Fog Heen wrote:

> | What about CARP-like cache routing (i.e., where multiple cache  
> servers
> | themselves are hash buckets)?  This would go a LONG way towards
> | scalability.
>
> http://varnish.projects.linpro.no/wiki/PostTwoShoppingList second item
> sounds like what you want?

Yup, sounds like what I proposed about a year ago :-)

--Michael


From phk at phk.freebsd.dk  Sun Jan 11 11:25:55 2009
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Sun, 11 Jan 2009 11:25:55 +0000
Subject: Fwd: Panic message : Varnish restart 
In-Reply-To: Your message of "Fri, 09 Jan 2009 17:52:58 +0530."
	<75cf5800901090422g7c28e7car4583ad1abdeeb103@mail.gmail.com> 
Message-ID: <5562.1231673155@critter.freebsd.dk>

In message <75cf5800901090422g7c28e7car4583ad1abdeeb103 at mail.gmail.com>, "Paras
 Fadte" writes:

>Can anybody please respond to the following  query of mine ?

I belive I fixed that yesterday in r3500 (ticket #387).

Sorry about the delay.

Poul-Henning


>Thank you.
>
>-Paras
>
>---------- Forwarded message ----------
>From: Paras Fadte <plfgoa at gmail.com>
>Date: Fri, Jan 2, 2009 at 10:07 AM
>Subject: Panic message : Varnish restart
>To: "varnish-misc at projects.linpro.no" <varnish-misc at projects.linpro.no>
>
>
>Hi,
>
>I got a message like following in /var/log/messages and child
>restarted. what could be the issue ?
>
>varnishd[28876]: Child (31253) Panic message: Missing errorhandling
>code in fetch_chunked(), cache_fetch.c line 113:   Condition(be >
> bp) not true.  thread = (cache-worker)sp = 0x2aabd2970008 {   fd =
>302, id = 302, xid = 1800279969,
>
>Varnish Version is :
>
>varnishd (varnish-trunk)
>Copyright (c) 2006-2008 Linpro AS / Verdens Gang AS
>
>Thanks in advance.
>
>-Paras
>_______________________________________________
>varnish-misc mailing list
>varnish-misc at projects.linpro.no
>http://projects.linpro.no/mailman/listinfo/varnish-misc
>

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From timball at gmail.com  Mon Jan 12 04:35:16 2009
From: timball at gmail.com (Timothy Ball)
Date: Sun, 11 Jan 2009 23:35:16 -0500
Subject: flushing cache doesn't seem to make varnish know about less
Message-ID: <f36957c80901112035u4fe8dd74p9f97e33d9b4d8cc8@mail.gmail.com>

a programming error caused varnish to think there were billions of
pages it had to know about. bug is quashed but varnish doesn't seem to
know

# this is a line from top
 3596  0.2 28.3g  28g 3808  256  57m 1916   12    0 S  20   0    0 varnishd
# this is a line from ps auxwww
nobody    3596  0.1  0.1 29681936 3968 ?       Sl   04:24   0:00
/opt/varnish/sbin/varnishd -a 10.13.37.1:80 -h classic -f
/etc/varnish/default.vcl -T 127.0.0.1:6082 -t 120 -w 1,500,20

i tried doing url.purge ".*" and url.purge "." to no avail.

how do i make this thing forget?

--timball

-- 
        GPG key available on pgpkeys.mit.edu
pub  1024D/511FBD54 2001-07-23 Timothy Lu Hu Ball <timball at tux.org>
Key fingerprint = B579 29B0 F6C8 C7AA 3840  E053 FE02 BB97 511F BD54


From ottolski at web.de  Mon Jan 12 07:39:26 2009
From: ottolski at web.de (Sascha Ottolski)
Date: Mon, 12 Jan 2009 08:39:26 +0100
Subject: flushing cache doesn't seem to make varnish know about less
In-Reply-To: <f36957c80901112035u4fe8dd74p9f97e33d9b4d8cc8@mail.gmail.com>
References: <f36957c80901112035u4fe8dd74p9f97e33d9b4d8cc8@mail.gmail.com>
Message-ID: <200901120839.26853.ottolski@web.de>

Am Montag 12 Januar 2009 05:35:16 schrieb Timothy Ball:
> a programming error caused varnish to think there were billions of
> pages it had to know about. bug is quashed but varnish doesn't seem
> to know
>
> # this is a line from top
>  3596  0.2 28.3g  28g 3808  256  57m 1916   12    0 S  20   0    0
> varnishd # this is a line from ps auxwww
> nobody    3596  0.1  0.1 29681936 3968 ?       Sl   04:24   0:00
> /opt/varnish/sbin/varnishd -a 10.13.37.1:80 -h classic -f
> /etc/varnish/default.vcl -T 127.0.0.1:6082 -t 120 -w 1,500,20
>
> i tried doing url.purge ".*" and url.purge "." to no avail.
>
> how do i make this thing forget?
>
> --timball

why not just restarting? it's so quick that it shouldn't hurt your 
service very much.


Cheers, Sascha


From tfheen at linpro.no  Mon Jan 12 10:06:38 2009
From: tfheen at linpro.no (Tollef Fog Heen)
Date: Mon, 12 Jan 2009 11:06:38 +0100
Subject: 2.1 plans
In-Reply-To: <496B133E.4000007@linpro.no> (Ingvar Hagelund's message of "Mon, 
	12 Jan 2009 10:54:06 +0100")
References: <8763kqgkfo.fsf@qurzaw.linpro.no> <496B133E.4000007@linpro.no>
Message-ID: <874p043kk1.fsf@qurzaw.linpro.no>

]] Ingvar Hagelund 

| * Tollef Fog Heen
| > a short while before Christmas, I wrote up a small document pointing to
| > what I would like to get into 2.1 (...)
| > Varnish 2.1 release plan
| > (...)
| 
| Will 2.1 be backwards compatible with vcl and configuration settings
| from 2.0?
|
| ie, will I be able to just upgrade a binary pre-built package and
| restart the server?

Yes, I don't see any need to break backwards compatibility for 2.1.

-- 
Tollef Fog Heen 
Redpill Linpro -- Changing the game!
t: +47 21 54 41 73


From perbu at linpro.no  Mon Jan 12 10:19:09 2009
From: perbu at linpro.no (Per Buer)
Date: Mon, 12 Jan 2009 11:19:09 +0100
Subject: flushing cache doesn't seem to make varnish know about less
In-Reply-To: <f36957c80901112035u4fe8dd74p9f97e33d9b4d8cc8@mail.gmail.com>
References: <f36957c80901112035u4fe8dd74p9f97e33d9b4d8cc8@mail.gmail.com>
Message-ID: <496B191D.8030803@linpro.no>

Timothy Ball wrote:
> a programming error caused varnish to think there were billions of
> pages it had to know about. bug is quashed but varnish doesn't seem to
> know
> (..)
> 
> i tried doing url.purge ".*" and url.purge "." to no avail.

"url.purge .*" will only invalidate the pages. They are not actually
evicted from cache. HTTP PURGE or TTL timeout are the only ways to
actually drop a page from cache, except restarting of course.

> how do i make this thing forget?

I guess the easiest would be a restart.
-- 
Per Buer - Leder Infrastruktur og Drift - Redpill Linpro
Telefon: 21 54 41 21 - Mobil: 958 39 117
http://linpro.no/ | http://redpill.se/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 260 bytes
Desc: OpenPGP digital signature
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20090112/7d07f6bc/attachment.bin>

From tfheen at linpro.no  Mon Jan 12 10:34:40 2009
From: tfheen at linpro.no (Tollef Fog Heen)
Date: Mon, 12 Jan 2009 11:34:40 +0100
Subject: flushing cache doesn't seem to make varnish know about less
In-Reply-To: <f36957c80901112035u4fe8dd74p9f97e33d9b4d8cc8@mail.gmail.com>
	(Timothy Ball's message of "Sun, 11 Jan 2009 23:35:16 -0500")
References: <f36957c80901112035u4fe8dd74p9f97e33d9b4d8cc8@mail.gmail.com>
Message-ID: <87zlhw24ov.fsf@qurzaw.linpro.no>

]] Timothy Ball 

| a programming error caused varnish to think there were billions of
| pages it had to know about. bug is quashed but varnish doesn't seem to
| know
| 
| # this is a line from top
|  3596  0.2 28.3g  28g 3808  256  57m 1916   12    0 S  20   0    0 varnishd
| # this is a line from ps auxwww
| nobody    3596  0.1  0.1 29681936 3968 ?       Sl   04:24   0:00
| /opt/varnish/sbin/varnishd -a 10.13.37.1:80 -h classic -f
| /etc/varnish/default.vcl -T 127.0.0.1:6082 -t 120 -w 1,500,20
| 
| i tried doing url.purge ".*" and url.purge "." to no avail.

url.purge does not actively go out and remove the objects; they'll be
picked up by the LRU cleaner after a while.

If you need to recover the memory, you can do a quick ?stop? and then
?start? in the CLI, but this will naturally nuke your complete cache, so
it can't be used if you just wanted to remove some bits.

-- 
Tollef Fog Heen 
Redpill Linpro -- Changing the game!
t: +47 21 54 41 73


From malevo at gmail.com  Mon Jan 12 13:26:07 2009
From: malevo at gmail.com (=?ISO-8859-1?Q?Pablo_Garc=EDa?=)
Date: Mon, 12 Jan 2009 11:26:07 -0200
Subject: Dropping some mailing lists?
In-Reply-To: <87y6xkvent.fsf@qurzaw.linpro.no>
References: <87y6xkvent.fsf@qurzaw.linpro.no>
Message-ID: <4ec3c3f70901120526k42a93073id14e621ba112f105@mail.gmail.com>

Agree

On Fri, Jan 9, 2009 at 10:37 AM, Tollef Fog Heen <tfheen at linpro.no> wrote:

>
> (Please reply only to -misc so we don't end up with a zillion copies of
> this all over the lists.)
>
> Hi,
>
> we currently have very many mailing lists:
>
> varnish-announce        Release announcements and other important news.
> varnish-bugs    Bug reports go here.
> varnish-commit  Commit messages go here.
> varnish-dev     Discussions regarding Varnish development.
> varnish-dist    Discussions regarding Varnish releases and packaging
> varnish-misc    Discussions on miscellaneous topics relating to Varnish
> varnish-test    For testing purposes.
>
> At the same time, we don't have that much traffic, so having a total of
> seven mailing lists seem a bit excessive.
>
> I suggest we get rid of -dev, -dist and -test.  -announce and -misc will
> be kept as discussion lists.  -bugs and -commit will continue as
> trac-to-email lists (i.e. not discussion lists).
>
> Comments?
>
> --
> Tollef Fog Heen
> Redpill Linpro -- Changing the game!
> t: +47 21 54 41 73
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-misc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20090112/022a0991/attachment.html>

From phk at phk.freebsd.dk  Mon Jan 12 14:01:43 2009
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Mon, 12 Jan 2009 14:01:43 +0000
Subject: Purge change...
Message-ID: <85190.1231768903@critter.freebsd.dk>


Right now we spend CPU&RAM to keep the built hash-string around, in case
we want to hit it with purge_hash later on.

purge_hash was made so that you could purge content on a particular
vhost without affecting other vhosts, but due to the wierd syntax,
it is not very convenient for this purpose.

The proposed change is to add a new purge primitive that takes a
list of (header, regexp) pairs, for instance:

	purge Host host3 URL "\.jpg$"

But once you think about it, you can do really wierd things:

	purge STATUS 404

	purge Set-Cookie "USER=9348395"

Comments welcome...

Poul-Henning

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From des at des.no  Mon Jan 12 15:16:33 2009
From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=)
Date: Mon, 12 Jan 2009 16:16:33 +0100
Subject: question about configure warning
In-Reply-To: <0673749E-7A79-4355-BEA4-DE2D04F4EEB8@dynamine.net> (Michael
	S. Fischer's message of "Tue, 6 Jan 2009 09:48:02 -0800")
References: <04F06E3F-8148-43BD-A716-53746A98AA22@karlsbakk.net>
	<49637BD6.40809@britarch.ac.uk>
	<0673749E-7A79-4355-BEA4-DE2D04F4EEB8@dynamine.net>
Message-ID: <86vdsk5zce.fsf@ds4.des.no>

"Michael S. Fischer" <michael at dynamine.net> writes:
> What are these "non-Linux-specific issues" to which the document refers?

The lack of a completion indicator + various implementation bugs.  The
only OS I know of that has a usable implementation is FreeBSD 8.

DES
-- 
Dag-Erling Sm?rgrav - des at des.no


From perbu at linpro.no  Mon Jan 12 16:12:11 2009
From: perbu at linpro.no (Per Buer)
Date: Mon, 12 Jan 2009 17:12:11 +0100
Subject: Purge change...
In-Reply-To: <85190.1231768903@critter.freebsd.dk>
References: <85190.1231768903@critter.freebsd.dk>
Message-ID: <496B6BDB.4070209@linpro.no>

Poul-Henning Kamp wrote:

> The proposed change is to add a new purge primitive that takes a
> list of (header, regexp) pairs, for instance:
> 
> 	purge Host host3 URL "\.jpg$"
> 
> But once you think about it, you can do really wierd things:
> 
> 	purge STATUS 404

This looks really useful. It can be used to implement "cache channels" -
don't worry, we'll do the RSS and other stuff outside of Varnish. :-)

One thing bothers me, however. This will mean we will accumulate a
longer and longer list or active purges. AFAIK this can be helped by two
means

1) The nuke thread. One threads which walks the heap and kills of purged
objects and shortens the list of purges.

2) Setting a TTL on a purge statement. If you have a maximum TTL of
24hrs there is no point in keeping the purge statement around and
checking incomming URIs against it after 24hrs.

2) might even make sense today.
-- 
http://linpro.no/ | http://redpill.se/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 260 bytes
Desc: OpenPGP digital signature
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20090112/24d07f4b/attachment.bin>

From naama at answers.com  Thu Jan 15 07:08:50 2009
From: naama at answers.com (Naama Bamberger)
Date: Thu, 15 Jan 2009 09:08:50 +0200
Subject: 503 errors using Citrix Netscaler
Message-ID: <749CACF29BDFB64E9F80189ECD778688043CD857@jermail1.atomant.net>

We're using Varnish 2.0.1 on our site, with Citrix Netscaler between
Varnish and our Apache Web machines.

We're getting hundreds of sporadic 503 errors an hour.

When I let Varnish round-robin between our web machines, bypassing the
load balancer, the problem disappears.

(We don't want to use this as a permanent solution, though, for failover
and maintenability reasons).

 

Are there any known issues regarding Varnish and Citrix Netscaler?

 

Thanks,

Naama

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20090115/5beb9ee6/attachment.html>

From phk at phk.freebsd.dk  Thu Jan 15 08:37:17 2009
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Thu, 15 Jan 2009 08:37:17 +0000
Subject: Is it possible to compare an ACL list to a specific header? 
In-Reply-To: Your message of "Mon, 05 Jan 2009 13:17:27 +0200."
	<749CACF29BDFB64E9F80189ECD778688043CD81C@jermail1.atomant.net> 
Message-ID: <2127.1232008637@critter.freebsd.dk>

In message <749CACF29BDFB64E9F80189ECD778688043CD81C at jermail1.atomant.net>, "Na
ama Bamberger" writes:
>I want to block some IPs, but cannot use
>if (client.ip ~ blocked_ips), 
>since all the requests go through a load balancer.
>
>The original user IP is stored by the load balancer in a custom header.
>I tried something like if (req.http.X-My-Custom-Header ~ blocked_ips),
>but trying to compile it causes a segfault.

A segfault would be a bug, but I can't reproduce that.

The problem is twofold, the ACL comparison only works on objects
of type IP and none of the http headers are that.

The other problem is that obviouly, we should be able to pick
the client IP from a header but currently we have no way to.


-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From plfgoa at gmail.com  Thu Jan 15 14:05:09 2009
From: plfgoa at gmail.com (Paras Fadte)
Date: Thu, 15 Jan 2009 19:35:09 +0530
Subject: Fwd: Panic message : Varnish restart
In-Reply-To: <5562.1231673155@critter.freebsd.dk>
References: <75cf5800901090422g7c28e7car4583ad1abdeeb103@mail.gmail.com>
	<5562.1231673155@critter.freebsd.dk>
Message-ID: <75cf5800901150605n5fbbd9f5tccc6028cece8c356@mail.gmail.com>

Thanks Poul.

On Sun, Jan 11, 2009 at 4:55 PM, Poul-Henning Kamp <phk at phk.freebsd.dk> wrote:
> In message <75cf5800901090422g7c28e7car4583ad1abdeeb103 at mail.gmail.com>, "Paras
>  Fadte" writes:
>
>>Can anybody please respond to the following  query of mine ?
>
> I belive I fixed that yesterday in r3500 (ticket #387).
>
> Sorry about the delay.
>
> Poul-Henning
>
>
>>Thank you.
>>
>>-Paras
>>
>>---------- Forwarded message ----------
>>From: Paras Fadte <plfgoa at gmail.com>
>>Date: Fri, Jan 2, 2009 at 10:07 AM
>>Subject: Panic message : Varnish restart
>>To: "varnish-misc at projects.linpro.no" <varnish-misc at projects.linpro.no>
>>
>>
>>Hi,
>>
>>I got a message like following in /var/log/messages and child
>>restarted. what could be the issue ?
>>
>>varnishd[28876]: Child (31253) Panic message: Missing errorhandling
>>code in fetch_chunked(), cache_fetch.c line 113:   Condition(be >
>> bp) not true.  thread = (cache-worker)sp = 0x2aabd2970008 {   fd =
>>302, id = 302, xid = 1800279969,
>>
>>Varnish Version is :
>>
>>varnishd (varnish-trunk)
>>Copyright (c) 2006-2008 Linpro AS / Verdens Gang AS
>>
>>Thanks in advance.
>>
>>-Paras
>>_______________________________________________
>>varnish-misc mailing list
>>varnish-misc at projects.linpro.no
>>http://projects.linpro.no/mailman/listinfo/varnish-misc
>>
>
> --
> Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
> phk at FreeBSD.ORG         | TCP/IP since RFC 956
> FreeBSD committer       | BSD since 4.3-tahoe
> Never attribute to malice what can adequately be explained by incompetence.
>


From malevo at gmail.com  Thu Jan 15 14:30:26 2009
From: malevo at gmail.com (=?ISO-8859-1?Q?Pablo_Garc=EDa?=)
Date: Thu, 15 Jan 2009 12:30:26 -0200
Subject: 503 errors using Citrix Netscaler
In-Reply-To: <749CACF29BDFB64E9F80189ECD778688043CD857@jermail1.atomant.net>
References: <749CACF29BDFB64E9F80189ECD778688043CD857@jermail1.atomant.net>
Message-ID: <4ec3c3f70901150630kc4be61l4d259b86e3bde4f8@mail.gmail.com>

Look at the logs in the netscaler for service or vserver down, it seems to
me that the vserver you're pointing the varnish to, is going up and down and
that's why the 503 response from the netscaler.

Regards, Pablo


On Thu, Jan 15, 2009 at 5:08 AM, Naama Bamberger <naama at answers.com> wrote:

>  We're using Varnish 2.0.1 on our site, with Citrix Netscaler between
> Varnish and our Apache Web machines.
>
> We're getting hundreds of sporadic 503 errors an hour.
>
> When I let Varnish round-robin between our web machines, bypassing the load
> balancer, the problem disappears.
>
> (We don't want to use this as a permanent solution, though, for failover
> and maintenability reasons).
>
>
>
> Are there any known issues regarding Varnish and Citrix Netscaler?
>
>
>
> Thanks,
>
> Naama
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-misc
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20090115/a9458e5e/attachment.html>

From ml at tinwong.com  Mon Jan 19 14:25:01 2009
From: ml at tinwong.com (M L)
Date: Mon, 19 Jan 2009 22:25:01 +0800
Subject: Backend connections failures
Message-ID: <113d871c0901190625k63264e30r17aed2859875b43a@mail.gmail.com>

Hi list


My varnish and backend server on different site , i guess sometime varnish
to my backend server network connections hiccough

have any way to setup varnish to retry when backend connections failures?
can i modify varnish to backend server connect timeout ?   .connect_timeout
= 1s; ?



       92184         7.00        10.78 Client connections accepted
      415513       124.98        48.60 Client requests received
      358635       117.98        41.95 Cache hits
       32184         3.00         3.76 Cache hits for pass
       20622         3.00         2.41 Cache misses
       56668         7.00         6.63 Backend connections success
           0         0.00         0.00 Backend connections not attempted
           0         0.00         0.00 Backend connections too many
         194         0.00         0.02 Backend connections failures
       11893         1.00         1.39 Backend connections reuses
       53314         5.00         6.24 Backend connections recycles
           0         0.00         0.00 Backend connections unused


box setup , varnish 2.0.2 / centos 5.2/64


Thanks all

Tin W
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20090119/b7f65350/attachment.html>

From anders at fupp.net  Tue Jan 20 11:05:00 2009
From: anders at fupp.net (Anders Nordby)
Date: Tue, 20 Jan 2009 12:05:00 +0100
Subject: Syslog with inline C
Message-ID: <20090120110500.GA33398@fupp.net>

Hi,

I want to use syslog() two places in vcl_recv.

If I put this one two places:

C{
	#include <syslog.h>
	syslog(LOG_ERR, "Bogus request: %s/%s", VRT_GetHdr(sp, HDR_REQ, "\005host:"), VRT_r_req_url(sp));
}C

I get this error (also if I use that block with #include <syslog.h> only
in the first block):

C-compiler said: ./vcl.eOj4OB3l.c: In function 'VGC_function_vcl_recv':
C-compiler said: ./vcl.eOj4OB3l.c:922: error: incompatible implicit
declaration of function 'syslog'
C-compiler said: /usr/include/syslog.h:195: error: previous implicit
declaration of 'syslog' was here
mgt_run_cc(): Compiler failed, exit 1VCL compilation failed

If I make a separate block with #include <syslog.h> before the ones with
syslog(), I get:

C-compiler said: ./vcl.yBWJKX4u.c: In function 'VGC_function_vcl_recv':
C-compiler said: ./vcl.yBWJKX4u.c:905: error: incompatible implicit
declaration of function 'syslog'
C-compiler said: /usr/include/syslog.h:195: error: previous implicit
declaration of 'syslog' was here
C-compiler said: ./vcl.yBWJKX4u.c:923: error: incompatible implicit
declaration of function 'syslog'
C-compiler said: /usr/include/syslog.h:195: error: previous implicit
declaration of 'syslog' was here
mgt_run_cc(): Compiler failed, exit 1VCL compilation failed

If I simply drop having #include <syslog.h> at all, I get:

C-compiler said: ./vcl.Qwyqx8g0.c: In function 'VGC_function_vcl_recv':
C-compiler said: ./vcl.Qwyqx8g0.c:902: error: 'LOG_ERR' undeclared
(first use in this function)
C-compiler said: ./vcl.Qwyqx8g0.c:902: error: (Each undeclared
identifier is reported only once
C-compiler said: ./vcl.Qwyqx8g0.c:902: error: for each function it
appears in.)
mgt_run_cc(): Compiler failed, exit 1VCL compilation failed

How can I use syslog() several places in one VCL function (like
vcl_recv)?

I wish we have syslog() in VCL one day. :)

Bye,

-- 
Anders.


From phk at phk.freebsd.dk  Tue Jan 20 12:06:00 2009
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Tue, 20 Jan 2009 12:06:00 +0000
Subject: Syslog with inline C 
In-Reply-To: Your message of "Tue, 20 Jan 2009 12:05:00 +0100."
	<20090120110500.GA33398@fupp.net> 
Message-ID: <11911.1232453160@critter.freebsd.dk>

In message <20090120110500.GA33398 at fupp.net>, Anders Nordby writes:
>Hi,
>
>I want to use syslog() two places in vcl_recv.
>
>If I put this one two places:
>
>C{
>	#include <syslog.h>
>	syslog(LOG_ERR, "Bogus request: %s/%s", VRT_GetHdr(sp, HDR_REQ, "\005host:"), VRT_r_req_url(sp));
>}C

Outside of any subroutine, you must do the includes, and the calls obviously
from inside:

	C{
	#include <syslog.h>
	}C

	sub vcl_recv {
		C{
		syslog(LOG_ERR, "foobar");
		}C
	}

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From plfgoa at gmail.com  Wed Jan 21 11:02:12 2009
From: plfgoa at gmail.com (Paras Fadte)
Date: Wed, 21 Jan 2009 16:32:12 +0530
Subject: Purging contents
Message-ID: <75cf5800901210302m5fc06acbh2334b0d90df7a54b@mail.gmail.com>

Hi,

When purging object from varnish cache from command line it always
tends to show 200 0 as the status code even if the object was not
there in cache ? What could be the reason ?

Thank you.

-Paras


From phk at phk.freebsd.dk  Wed Jan 21 11:07:15 2009
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Wed, 21 Jan 2009 11:07:15 +0000
Subject: Purging contents 
In-Reply-To: Your message of "Wed, 21 Jan 2009 16:32:12 +0530."
	<75cf5800901210302m5fc06acbh2334b0d90df7a54b@mail.gmail.com> 
Message-ID: <34066.1232536035@critter.freebsd.dk>

In message <75cf5800901210302m5fc06acbh2334b0d90df7a54b at mail.gmail.com>, Paras 
Fadte writes:
>Hi,
>
>When purging object from varnish cache from command line it always
>tends to show 200 0 as the status code even if the object was not
>there in cache ? What could be the reason ?

200 means the purge went well, and 0 that there is no resulting text.

purges don't happen until the object is looked up.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From plfgoa at gmail.com  Wed Jan 21 11:19:43 2009
From: plfgoa at gmail.com (Paras Fadte)
Date: Wed, 21 Jan 2009 16:49:43 +0530
Subject: Purging contents
In-Reply-To: <34066.1232536035@critter.freebsd.dk>
References: <75cf5800901210302m5fc06acbh2334b0d90df7a54b@mail.gmail.com>
	<34066.1232536035@critter.freebsd.dk>
Message-ID: <75cf5800901210319j1a2741bdp171bdffad920b6df@mail.gmail.com>

Does  it mean that 200 code doesn't have any relation to the object
being  or not being there  in the cache ? Does it only mean then that
the purge command was successfully executed ?

Thanks .

-Paras

On Wed, Jan 21, 2009 at 4:37 PM, Poul-Henning Kamp <phk at phk.freebsd.dk> wrote:
> In message <75cf5800901210302m5fc06acbh2334b0d90df7a54b at mail.gmail.com>, Paras
> Fadte writes:
>>Hi,
>>
>>When purging object from varnish cache from command line it always
>>tends to show 200 0 as the status code even if the object was not
>>there in cache ? What could be the reason ?
>
> 200 means the purge went well, and 0 that there is no resulting text.
>
> purges don't happen until the object is looked up.
>
> --
> Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
> phk at FreeBSD.ORG         | TCP/IP since RFC 956
> FreeBSD committer       | BSD since 4.3-tahoe
> Never attribute to malice what can adequately be explained by incompetence.
>


From phk at phk.freebsd.dk  Wed Jan 21 11:38:05 2009
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Wed, 21 Jan 2009 11:38:05 +0000
Subject: Purging contents 
In-Reply-To: Your message of "Wed, 21 Jan 2009 16:49:43 +0530."
	<75cf5800901210319j1a2741bdp171bdffad920b6df@mail.gmail.com> 
Message-ID: <36679.1232537885@critter.freebsd.dk>

In message <75cf5800901210319j1a2741bdp171bdffad920b6df at mail.gmail.com>, Paras 
Fadte writes:

>Does  it mean that 200 code doesn't have any relation to the object
>being  or not being there  in the cache ? Does it only mean then that
>the purge command was successfully executed ?

It only means that the purge was succesfully added to the list of
purges.

The way purges work in varnish is that we have a list of purge
commands which gets checked against the objects on demand, in
other words, when you have a cache hit, the purges that have 
come in since the last time this object was hit will be checked
to make sure we can still deliver it.

We do not start walking, the possibly several million, objects
in the cache every time you use the purge command.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From audun at ytterdal.net  Wed Jan 21 13:08:29 2009
From: audun at ytterdal.net (Audun Ytterdal)
Date: Wed, 21 Jan 2009 14:08:29 +0100
Subject: Purging contents
In-Reply-To: <36679.1232537885@critter.freebsd.dk>
References: <75cf5800901210319j1a2741bdp171bdffad920b6df@mail.gmail.com>
	<36679.1232537885@critter.freebsd.dk>
Message-ID: <8318f61f0901210508i25fb1385w91573182d8cd247@mail.gmail.com>

On Wed, Jan 21, 2009 at 12:38 PM, Poul-Henning Kamp <phk at phk.freebsd.dk> wrote:
> In message <75cf5800901210319j1a2741bdp171bdffad920b6df at mail.gmail.com>, Paras
> Fadte writes:
>
>>Does  it mean that 200 code doesn't have any relation to the object
>>being  or not being there  in the cache ? Does it only mean then that
>>the purge command was successfully executed ?
>
> It only means that the purge was succesfully added to the list of
> purges.
>
> The way purges work in varnish is that we have a list of purge
> commands which gets checked against the objects on demand, in
> other words, when you have a cache hit, the purges that have
> come in since the last time this object was hit will be checked
> to make sure we can still deliver it.
>
> We do not start walking, the possibly several million, objects
> in the cache every time you use the purge command.

Does that mean that vcl_miss in this part of the documentation never
would be run?

http://varnish.projects.linpro.no/wiki/VCLExamplePurging
-- 
Audun Ytterdal
http://audun.ytterdal.net


From phk at phk.freebsd.dk  Wed Jan 21 14:15:32 2009
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Wed, 21 Jan 2009 14:15:32 +0000
Subject: Purging contents 
In-Reply-To: Your message of "Wed, 21 Jan 2009 14:08:29 +0100."
	<8318f61f0901210508i25fb1385w91573182d8cd247@mail.gmail.com> 
Message-ID: <37200.1232547332@critter.freebsd.dk>

In message <8318f61f0901210508i25fb1385w91573182d8cd247 at mail.gmail.com>, Audun 
Ytterdal writes:
>On Wed, Jan 21, 2009 at 12:38 PM, Poul-Henning Kamp <phk at phk.freebsd.dk> wrote:
>> In message <75cf5800901210319j1a2741bdp171bdffad920b6df at mail.gmail.com>, Paras Fadte writes:


>Does that mean that vcl_miss in this part of the documentation never
>would be run?
>
>http://varnish.projects.linpro.no/wiki/VCLExamplePurging

Well, this is the other way to do purges: look up the single object
you want, and blast it away.

That method does not involve any regular expressions, and consequently
you have to be 100% precise to find the single object you want to
get rid off.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From plfgoa at gmail.com  Wed Jan 21 14:52:05 2009
From: plfgoa at gmail.com (Paras Fadte)
Date: Wed, 21 Jan 2009 20:22:05 +0530
Subject: Purging contents
In-Reply-To: <37200.1232547332@critter.freebsd.dk>
References: <8318f61f0901210508i25fb1385w91573182d8cd247@mail.gmail.com>
	<37200.1232547332@critter.freebsd.dk>
Message-ID: <75cf5800901210652ldba35e1l812778115e51834a@mail.gmail.com>

Hi Poul,

Can one specify host  when purging using command line ?

Thank you.

-Paras

On Wed, Jan 21, 2009 at 7:45 PM, Poul-Henning Kamp <phk at phk.freebsd.dk> wrote:
> In message <8318f61f0901210508i25fb1385w91573182d8cd247 at mail.gmail.com>, Audun
> Ytterdal writes:
>>On Wed, Jan 21, 2009 at 12:38 PM, Poul-Henning Kamp <phk at phk.freebsd.dk> wrote:
>>> In message <75cf5800901210319j1a2741bdp171bdffad920b6df at mail.gmail.com>, Paras Fadte writes:
>
>
>>Does that mean that vcl_miss in this part of the documentation never
>>would be run?
>>
>>http://varnish.projects.linpro.no/wiki/VCLExamplePurging
>
> Well, this is the other way to do purges: look up the single object
> you want, and blast it away.
>
> That method does not involve any regular expressions, and consequently
> you have to be 100% precise to find the single object you want to
> get rid off.
>
> --
> Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
> phk at FreeBSD.ORG         | TCP/IP since RFC 956
> FreeBSD committer       | BSD since 4.3-tahoe
> Never attribute to malice what can adequately be explained by incompetence.
>


From store+varnish at k9tfk.com  Wed Jan 21 15:58:54 2009
From: store+varnish at k9tfk.com (Nathan Lenz)
Date: Wed, 21 Jan 2009 09:58:54 -0600
Subject: Am I reading the varnishlog correctly?
Message-ID: <3cc6f8100901210758u384af1d4k8091c903614f50b@mail.gmail.com>

Hello,
I have a weird case of Varnish returning a hit with a Length of 0.  It
happens when I clear my browser cache and then hit refresh.  Varnish returns
a 304 status and my browser shows a blank white page.  Here is a snippet of
the varnishlog output:

   10 RxHeader     c If-Modified-Since: Mon, 08 Dec 2008 17:28:41 GMT
   10 RxHeader     c Cache-Control: max-age=0
   10 VCL_call     c recv
   10 VCL_return   c lookup
   10 VCL_call     c hash
   10 VCL_return   c hash
   10 Hit          c 1043648757
   10 VCL_call     c hit
   10 VCL_return   c deliver
  * 10 Length       c 0*
   10 VCL_call     c deliver
   10 VCL_return   c deliver
   10 TxProtocol   c HTTP/1.1
   10 TxStatus     c 304
   10 TxResponse   c Not Modified
   10 TxHeader     c Date: Wed, 21 Jan 2009 15:16:20 GMT
   10 TxHeader     c Via: 1.1 varnish
   10 TxHeader     c X-Varnish: 1043648775
   10 TxHeader     c Last-Modified: Mon, 08 Dec 2008 17:28:41 GMT

My default.vcl is pretty much the default, with a few minor changes that I
copied from the wiki.  (Normalizing Accept-Encoding and allowing purge on
Shift-Refresh).

I'm reading this as saying:

1. Browser asks varnish for a new copy, with a max age of 0 from Mon, 08 Dec
2008 17:28:41 GMT or newer.
2. vcl_recv decides to do a lookup.
3. It's found that the item exists in the cache (hit!) but that the file
hasn't been modified since Mon, 08 Dec 2008 17:28:41 GMT.
4. vcl_deliver decides to send a 304 not modified instead of the actual
response.

I wonder why my browser is sending an If-Modified-Since header when it
doesn't really have a copy?  I've been running Varnish on a heavily used
testing server for the past week and I get this from time to time in both
Firefox and Safari but I can't reproduce it consistently.  Anyone experience
anything like this?

-- 
Nathan <nathan.lenz at gmail.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20090121/4abc64f7/attachment.html>

From phk at phk.freebsd.dk  Wed Jan 21 16:06:08 2009
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Wed, 21 Jan 2009 16:06:08 +0000
Subject: Am I reading the varnishlog correctly? 
In-Reply-To: Your message of "Wed, 21 Jan 2009 09:58:54 CST."
	<3cc6f8100901210758u384af1d4k8091c903614f50b@mail.gmail.com> 
Message-ID: <37590.1232553968@critter.freebsd.dk>

In message <3cc6f8100901210758u384af1d4k8091c903614f50b at mail.gmail.com>, Nathan
 Lenz writes:

>I wonder why my browser is sending an If-Modified-Since header when it
>doesn't really have a copy?  I've been running Varnish on a heavily used
>testing server for the past week and I get this from time to time in both
>Firefox and Safari but I can't reproduce it consistently.  Anyone experience
>anything like this?

Maybe your browser think it has a valid copy, of length zero ?

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From cfarinella at appropriatesolutions.com  Wed Jan 21 21:50:17 2009
From: cfarinella at appropriatesolutions.com (Charlie Farinella)
Date: Wed, 21 Jan 2009 16:50:17 -0500
Subject: Varnish, Plone and Apache2
Message-ID: <200901211650.17560.cfarinella@appropriatesolutions.com>

I have one site running Plone with lighttpd and Varnish that I set up as 
documented here:
http://bitubique.com/content/accelerate-plone-varnish

I have now been asked to set up others substituting Apache2 for lighttpd 
by the developers, but haven't been able to find such detailed 
instructions for Apache2.  I believe I just need to find the Apache 
equivalent for this line from lighttpd.conf:

proxy.server = ( "/VirtualHostBase/" => (
        ( "host" => "127.0.0.1" , "port" => 6081 ) )
)

To my understanding something has to listen on port 80, send the request 
to Varnish, which then either serves from the cache or sends the request 
on to the Zope (Plone) port.

If anyone knows offhand or has some experience with this I'd like to hear 
from you.  Is Apache a bad choice for this? 

-- 
------------------------------------------------------------------------
Charles Farinella 
Appropriate Solutions, Inc. (www.AppropriateSolutions.com)
cfarinella at AppropriateSolutions.com
voice: 603.924.6079   fax: 603.924.8668



From tim at metaweb.com  Wed Jan 21 21:51:19 2009
From: tim at metaweb.com (Tim Kientzle)
Date: Wed, 21 Jan 2009 13:51:19 -0800
Subject: Breaking Varnish
Message-ID: <6545783F-B1A7-4FDA-94D8-8439A2D13A19@metaweb.com>

We're evaluating Varnish as a possible replacement for our
installed Squid servers.  Performance-wise, Varnish is very
impressive, and we're pretty pleased with the configuration
flexibility.

But...

Under heavy load, we're seeing a lot of segfaults and
assertion failures.  I've pasted an excerpt below of
two of the issues we've seen using Varnish 2.0.2 on Linux
2.6.21 kernel with the default VCL (using command-line options
to set the listen address and the addresses of the two back-end
servers).

We're going to repeat these tests and see if we can get
more detail, possibly including core dumps.  What other
information would be useful in diagnosing and fixing
these issues?

Cheers,

Tim Kientzle

==========================================================

1) Varnish repeatedly died due to SIGSEGV:

child (2816) Started
Child (2816) said Closed fds: 4 7 8 10 11
Child (2816) said Child starts
Child (2816) said managed to mmap 49392648192 bytes of 49392648192
Child (2816) said Ready
Child (2816) died signal=11
Child cleanup complete

2) Varnish repeatedly died due to SIGABRT:

child (3017) Started
Child (3017) said Closed fds: 4 7 8 10 11
Child (3017) said Child starts
Child (3017) said managed to mmap 49392648192 bytes of 49392648192
Child (3017) said Ready
Child (3017) died signal=6
Child (3017) Panic message: Assert error in cnt_lookup(),  
cache_center.c line 625:
   Condition(sp->objhead != NULL) not true. thread = (cache-worker)sp  
= 0x2afee0fb3008 {
   fd = -1, id = 15, xid = 0,
   client = 10.2.8.27:45430,
   step = STP_DONE,
   handling = DELIVER,
   ws = 0x2afee0fb3078 {
     id = "sess",
     {s,f,r,e} = {0x2afee0fb37b0,,+587,(nil),+8192},
   },
}, 


From isharov at yandex-team.ru  Wed Jan 21 22:01:59 2009
From: isharov at yandex-team.ru (Iliya Sharov)
Date: Thu, 22 Jan 2009 01:01:59 +0300
Subject: Breaking Varnish
In-Reply-To: <6545783F-B1A7-4FDA-94D8-8439A2D13A19@metaweb.com>
References: <6545783F-B1A7-4FDA-94D8-8439A2D13A19@metaweb.com>
Message-ID: <49779B57.70401@yandex-team.ru>

amd64 or i386 architecture?

Tim Kientzle ?????:
> We're evaluating Varnish as a possible replacement for our
> installed Squid servers.  Performance-wise, Varnish is very
> impressive, and we're pretty pleased with the configuration
> flexibility.
>
> But...
>
> Under heavy load, we're seeing a lot of segfaults and
> assertion failures.  I've pasted an excerpt below of
> two of the issues we've seen using Varnish 2.0.2 on Linux
> 2.6.21 kernel with the default VCL (using command-line options
> to set the listen address and the addresses of the two back-end
> servers).
>
> We're going to repeat these tests and see if we can get
> more detail, possibly including core dumps.  What other
> information would be useful in diagnosing and fixing
> these issues?
>
> Cheers,
>
> Tim Kientzle
>
> ==========================================================
>
> 1) Varnish repeatedly died due to SIGSEGV:
>
> child (2816) Started
> Child (2816) said Closed fds: 4 7 8 10 11
> Child (2816) said Child starts
> Child (2816) said managed to mmap 49392648192 bytes of 49392648192
> Child (2816) said Ready
> Child (2816) died signal=11
> Child cleanup complete
>
> 2) Varnish repeatedly died due to SIGABRT:
>
> child (3017) Started
> Child (3017) said Closed fds: 4 7 8 10 11
> Child (3017) said Child starts
> Child (3017) said managed to mmap 49392648192 bytes of 49392648192
> Child (3017) said Ready
> Child (3017) died signal=6
> Child (3017) Panic message: Assert error in cnt_lookup(),  
> cache_center.c line 625:
>    Condition(sp->objhead != NULL) not true. thread = (cache-worker)sp  
> = 0x2afee0fb3008 {
>    fd = -1, id = 15, xid = 0,
>    client = 10.2.8.27:45430,
>    step = STP_DONE,
>    handling = DELIVER,
>    ws = 0x2afee0fb3078 {
>      id = "sess",
>      {s,f,r,e} = {0x2afee0fb37b0,,+587,(nil),+8192},
>    },
> }, 
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-misc
>   



From phk at phk.freebsd.dk  Wed Jan 21 22:02:18 2009
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Wed, 21 Jan 2009 22:02:18 +0000
Subject: Breaking Varnish 
In-Reply-To: Your message of "Wed, 21 Jan 2009 13:51:19 PST."
	<6545783F-B1A7-4FDA-94D8-8439A2D13A19@metaweb.com> 
Message-ID: <39049.1232575338@critter.freebsd.dk>

In message <6545783F-B1A7-4FDA-94D8-8439A2D13A19 at metaweb.com>, Tim Kientzle wri
tes:

>Under heavy load, we're seeing a lot of segfaults and
>assertion failures.  I've pasted an excerpt below of
>two of the issues we've seen using Varnish 2.0.2 on Linux
>2.6.21 kernel with the default VCL (using command-line options
>to set the listen address and the addresses of the two back-end
>servers).

Hi Tim,

Can I get you to take -trunk for a spin ?

At least the second of the problems you pasted I'm pretty sure I
have nailed recently and the first one could easily be the same one
in a different disguise.

Poul-Henning

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From tim at metaweb.com  Wed Jan 21 22:12:06 2009
From: tim at metaweb.com (Tim Kientzle)
Date: Wed, 21 Jan 2009 14:12:06 -0800
Subject: Breaking Varnish
In-Reply-To: <49779B57.70401@yandex-team.ru>
References: <6545783F-B1A7-4FDA-94D8-8439A2D13A19@metaweb.com>
	<49779B57.70401@yandex-team.ru>
Message-ID: <AFC86B8E-3542-4E1F-8FF8-333229B4F7C5@metaweb.com>

Dual-core AMD processor using the x86_64 kernel.  Uname shows:

Linux 2.6.21.5 #9 SMP Thu Aug 16 17:21:29 UTC 2007 x86_64 AMD  
Opteron(tm) Processor 248 AuthenticAMD GNU/Linux


On Jan 21, 2009, at 2:01 PM, Iliya Sharov wrote:
> amd64 or i386 architecture?
>
> Tim Kientzle ?????:
>> We're evaluating Varnish as a possible replacement for our
>> installed Squid servers.  Performance-wise, Varnish is very
>> impressive, and we're pretty pleased with the configuration
>> flexibility.
>>
>> But...
>>
>> Under heavy load, we're seeing a lot of segfaults and
>> assertion failures.  I've pasted an excerpt below of
>> two of the issues we've seen using Varnish 2.0.2 on Linux
>> 2.6.21 kernel with the default VCL (using command-line options
>> to set the listen address and the addresses of the two back-end
>> servers).
>>
>> We're going to repeat these tests and see if we can get
>> more detail, possibly including core dumps.  What other
>> information would be useful in diagnosing and fixing
>> these issues?
>>
>> Cheers,
>>
>> Tim Kientzle
>>
>> ==========================================================
>>
>> 1) Varnish repeatedly died due to SIGSEGV:
>>
>> child (2816) Started
>> Child (2816) said Closed fds: 4 7 8 10 11
>> Child (2816) said Child starts
>> Child (2816) said managed to mmap 49392648192 bytes of 49392648192
>> Child (2816) said Ready
>> Child (2816) died signal=11
>> Child cleanup complete
>>
>> 2) Varnish repeatedly died due to SIGABRT:
>>
>> child (3017) Started
>> Child (3017) said Closed fds: 4 7 8 10 11
>> Child (3017) said Child starts
>> Child (3017) said managed to mmap 49392648192 bytes of 49392648192
>> Child (3017) said Ready
>> Child (3017) died signal=6
>> Child (3017) Panic message: Assert error in cnt_lookup(),
>> cache_center.c line 625:
>>   Condition(sp->objhead != NULL) not true. thread = (cache-worker)sp
>> = 0x2afee0fb3008 {
>>   fd = -1, id = 15, xid = 0,
>>   client = 10.2.8.27:45430,
>>   step = STP_DONE,
>>   handling = DELIVER,
>>   ws = 0x2afee0fb3078 {
>>     id = "sess",
>>     {s,f,r,e} = {0x2afee0fb37b0,,+587,(nil),+8192},
>>   },
>> },
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at projects.linpro.no
>> http://projects.linpro.no/mailman/listinfo/varnish-misc
>>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-misc



From ric at digitalmarbles.com  Wed Jan 21 22:22:25 2009
From: ric at digitalmarbles.com (Ricardo Newbery)
Date: Wed, 21 Jan 2009 14:22:25 -0800
Subject: [varnish] Varnish, Plone and Apache2
In-Reply-To: <200901211650.17560.cfarinella@appropriatesolutions.com>
References: <200901211650.17560.cfarinella@appropriatesolutions.com>
Message-ID: <7BBFD15A-9091-4527-AB6C-C233B3BF6A3D@digitalmarbles.com>


On Jan 21, 2009, at 1:50 PM, Charlie Farinella wrote:

> I have one site running Plone with lighttpd and Varnish that I set  
> up as
> documented here:
> http://bitubique.com/content/accelerate-plone-varnish


IMHO, the vcl generated by the plone.recipe.varnish recipe is superior  
to the one on that page.



> I have now been asked to set up others substituting Apache2 for  
> lighttpd
> by the developers, but haven't been able to find such detailed
> instructions for Apache2.  I believe I just need to find the Apache
> equivalent for this line from lighttpd.conf:
>
> proxy.server = ( "/VirtualHostBase/" => (
>        ( "host" => "127.0.0.1" , "port" => 6081 ) )
> )
>
> To my understanding something has to listen on port 80, send the  
> request
> to Varnish, which then either serves from the cache or sends the  
> request
> on to the Zope (Plone) port.
>
> If anyone knows offhand or has some experience with this I'd like to  
> hear
> from you.  Is Apache a bad choice for this?


Apache is not necessarily a bad choice.

You will need to use ProxyPass or RewriteRule directives.  The Apache  
setup isn't really that much different than the standard Zope/Apache  
config.  Plone.org has plenty of docs on this:
http://plone.org/documentation/tutorial/plone-apache
http://plone.org/documentation/how-to/plone-with-apache

You might also want to look into the CacheFu product:
http://plone.org/products/cachefu

Ric




From plfgoa at gmail.com  Thu Jan 22 05:21:34 2009
From: plfgoa at gmail.com (Paras Fadte)
Date: Thu, 22 Jan 2009 10:51:34 +0530
Subject: Purging contents
In-Reply-To: <75cf5800901210652ldba35e1l812778115e51834a@mail.gmail.com>
References: <8318f61f0901210508i25fb1385w91573182d8cd247@mail.gmail.com>
	<37200.1232547332@critter.freebsd.dk>
	<75cf5800901210652ldba35e1l812778115e51834a@mail.gmail.com>
Message-ID: <75cf5800901212121k6647747bg31877e8b1d51280b@mail.gmail.com>

Hi Poul,


For applying HTTP purge the VCL code specified on
http://varnish.projects.linpro.no/wiki/VCLExamplePurging shows as
following:




sub vcl_recv {

       if (req.request == "PURGE") {
                if (!client.ip ~ purge) {
                        error 405 "Not allowed.";
                }
                lookup;
        }


should it have "else" so that it becomes

sub vcl_recv {

       if (req.request == "PURGE") {
                if (!client.ip ~ purge) {
                        error 405 "Not allowed.";
                }

else {
                lookup;
        }

}


Also when I telnet to varnish listen port (not the management port) it
seems to close the connection after about 4-5 seconds automatically.
what could be the issue ? Telnetting to management port works fines.


Thanks.

-Paras


On Wed, Jan 21, 2009 at 8:22 PM, Paras Fadte <plfgoa at gmail.com> wrote:
> Hi Poul,
>
> Can one specify host  when purging using command line ?
>
> Thank you.
>
> -Paras
>
> On Wed, Jan 21, 2009 at 7:45 PM, Poul-Henning Kamp <phk at phk.freebsd.dk> wrote:
>> In message <8318f61f0901210508i25fb1385w91573182d8cd247 at mail.gmail.com>, Audun
>> Ytterdal writes:
>>>On Wed, Jan 21, 2009 at 12:38 PM, Poul-Henning Kamp <phk at phk.freebsd.dk> wrote:
>>>> In message <75cf5800901210319j1a2741bdp171bdffad920b6df at mail.gmail.com>, Paras Fadte writes:
>>
>>
>>>Does that mean that vcl_miss in this part of the documentation never
>>>would be run?
>>>
>>>http://varnish.projects.linpro.no/wiki/VCLExamplePurging
>>
>> Well, this is the other way to do purges: look up the single object
>> you want, and blast it away.
>>
>> That method does not involve any regular expressions, and consequently
>> you have to be 100% precise to find the single object you want to
>> get rid off.
>>
>> --
>> Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
>> phk at FreeBSD.ORG         | TCP/IP since RFC 956
>> FreeBSD committer       | BSD since 4.3-tahoe
>> Never attribute to malice what can adequately be explained by incompetence.
>>
>


From phk at phk.freebsd.dk  Thu Jan 22 07:35:55 2009
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Thu, 22 Jan 2009 07:35:55 +0000
Subject: Purging contents 
In-Reply-To: Your message of "Thu, 22 Jan 2009 10:51:34 +0530."
	<75cf5800901212121k6647747bg31877e8b1d51280b@mail.gmail.com> 
Message-ID: <58555.1232609755@critter.freebsd.dk>

In message <75cf5800901212121k6647747bg31877e8b1d51280b at mail.gmail.com>, Paras 
Fadte writes:

>should it have "else" so that it becomes
>
>sub vcl_recv {
>
>       if (req.request == "PURGE") {
>                if (!client.ip ~ purge) {
>                        error 405 "Not allowed.";
>                }
>
>else {
>                lookup;
>        }
>
>}

That is not necessary, "error" is a terminating action.

>Also when I telnet to varnish listen port (not the management port) it
>seems to close the connection after about 4-5 seconds automatically.
>what could be the issue ? Telnetting to management port works fines.

That is intentional, if clients don't send a request, we don't want
to waste resources on them.

The timeout is configurable with param sess_timeout.


-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From plfgoa at gmail.com  Thu Jan 22 07:37:33 2009
From: plfgoa at gmail.com (Paras Fadte)
Date: Thu, 22 Jan 2009 13:07:33 +0530
Subject: Purging contents
In-Reply-To: <58555.1232609755@critter.freebsd.dk>
References: <75cf5800901212121k6647747bg31877e8b1d51280b@mail.gmail.com>
	<58555.1232609755@critter.freebsd.dk>
Message-ID: <75cf5800901212337m7678ba95t89a1017be490274d@mail.gmail.com>

Thanks Poul.

On Thu, Jan 22, 2009 at 1:05 PM, Poul-Henning Kamp <phk at phk.freebsd.dk> wrote:
> In message <75cf5800901212121k6647747bg31877e8b1d51280b at mail.gmail.com>, Paras
> Fadte writes:
>
>>should it have "else" so that it becomes
>>
>>sub vcl_recv {
>>
>>       if (req.request == "PURGE") {
>>                if (!client.ip ~ purge) {
>>                        error 405 "Not allowed.";
>>                }
>>
>>else {
>>                lookup;
>>        }
>>
>>}
>
> That is not necessary, "error" is a terminating action.
>
>>Also when I telnet to varnish listen port (not the management port) it
>>seems to close the connection after about 4-5 seconds automatically.
>>what could be the issue ? Telnetting to management port works fines.
>
> That is intentional, if clients don't send a request, we don't want
> to waste resources on them.
>
> The timeout is configurable with param sess_timeout.
>
>
> --
> Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
> phk at FreeBSD.ORG         | TCP/IP since RFC 956
> FreeBSD committer       | BSD since 4.3-tahoe
> Never attribute to malice what can adequately be explained by incompetence.
>


From abhishek.rhce at gmail.com  Fri Jan 23 11:03:30 2009
From: abhishek.rhce at gmail.com (Abhishek Singh)
Date: Fri, 23 Jan 2009 16:33:30 +0530
Subject: What is Age header in varnish
Message-ID: <25daf17a0901230303n57d21c49tfb2d43998acbec63@mail.gmail.com>

Hi,

I have configured varnish for caching static content as well as my home page
that is index.php. my config is like this for caching index.php, i trying to
set cache time 1 week for index.php but i am confused how to make sure that
index.php is cached for one week while varnish is serving from cache that is
sure i checked varnish headers.




sub vcl_recv {

if(req.request == "GET" && req.http.cookie && req.http.host ~ "^
www.example.com$" && req.url ~ "^/index\.(php)") {
        remove req.http.Set-Cookie;
        lookup;
  }
}

sub vcl_fetch {
if (req.request == "GET" && req.http.host ~ "^www.example.com$" && req.url ~
"^/index\.(php)") {

        unset obj.http.set-cookie;
        set obj.ttl = 1w;
      }
}


Can anyone please explain that why Age header is changing on every refresh
while i set "set obj.ttl" for 1 week and is the use of Age header. Is there
anything that i have to modify in my varnish config file to cache index.php.
please suggest something.

X-Varnish

473060033 473059746

Age

8713

Via

1.1 varnish


X-Varnish

473060081 473059746

Age

10141

Via

1.1 varnish



-- 
Abhishek Kumar Singh
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20090123/327ff3b0/attachment.html>

From phk at phk.freebsd.dk  Fri Jan 23 11:10:36 2009
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Fri, 23 Jan 2009 11:10:36 +0000
Subject: What is Age header in varnish 
In-Reply-To: Your message of "Fri, 23 Jan 2009 16:33:30 +0530."
	<25daf17a0901230303n57d21c49tfb2d43998acbec63@mail.gmail.com> 
Message-ID: <3991.1232709036@critter.freebsd.dk>

In message <25daf17a0901230303n57d21c49tfb2d43998acbec63 at mail.gmail.com>, Abhis
hek Singh writes:

>Can anyone please explain that why Age header is changing [...]

The Age header is defined in RFC2616 as:

| 14.6 Age
| 
|       The Age response-header field conveys the sender's estimate of the
|       amount of time since the response (or its revalidation) was
|       generated at the origin server.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From niallo at metaweb.com  Fri Jan 23 22:29:47 2009
From: niallo at metaweb.com (Niall O'Higgins)
Date: Fri, 23 Jan 2009 14:29:47 -0800
Subject: Breaking Varnish
In-Reply-To: <08FB770A-648E-499B-B729-5DBC59C451FE@metaweb.com>
References: <39049.1232575338@critter.freebsd.dk>
	<sfid-H20090121-140702-+021.61-1@spamfilter.osbf.lua>
	<08FB770A-648E-499B-B729-5DBC59C451FE@metaweb.com>
Message-ID: <20090123222947.GB28138@digdug.corp.631h.metaweb.com>

Hi,

Regarding:

On Wed, Jan 21, 2009 at 02:05:55PM -0800, Tim Kientzle wrote:
> On Jan 21, 2009, at 2:02 PM, Poul-Henning Kamp wrote:
>>> Under heavy load, we're seeing a lot of segfaults and
>>> assertion failures.  I've pasted an excerpt below of
>>> two of the issues we've seen using Varnish 2.0.2 on Linux
>>> 2.6.21 kernel with the default VCL (using command-line options
>>> to set the listen address and the addresses of the two back-end
>>> servers).
>>
>> Hi Tim,
>>
>> Can I get you to take -trunk for a spin ?
>>
>> At least the second of the problems you pasted I'm pretty sure I
>> have nailed recently and the first one could easily be the same one
>> in a different disguise.

I've re-run the load test against varnish-trunk.  Trunk is better
behaved, but I now get output like this over and over:

child (19731) Started
Child (19731) said Closed fds: 4 7 8 10 11
Child (19731) said Child starts
Child (19731) said managed to mmap 49929912320 bytes of 49929912320
Child (19731) said Ready
Child (19731) not responding to ping, killing it.
Child (19731) not responding to ping, killing it.
Child (19731) not responding to ping, killing it.
Child (19731) died signal=3
Child cleanup complete

And varnish eventually exits with this message:

child (19773) Started
Pushing vcls failed: CLI communication error

I am running varnishd like so:

sbin/varnishd -f etc/varnish/default.vcl -F -a'0.0.0.0:8101'

My configuration file contains:

director www_director round-robin {
        { .backend = { .host = "appserver1"; .port = "8105"; } }
        { .backend = { .host = "appserver2"; .port = "8105"; } }
}

sub vcl_recv {
        if (req.http.host ~ "^varnishserver$") {
                        set req.backend = www_director;
        }
}

If there are other details which might help diagnose this, let me know and I'll
try to provide them.

-- 
Niall O'Higgins
Software Engineer
Metaweb Technologies, Inc.


From varnish-misc at projects.linpro.no  Sun Jan 25 14:25:23 2009
From: varnish-misc at projects.linpro.no (varnish-misc at projects.linpro.no)
Date: Sun, 25 Jan 2009 15:25:23 +0100 (CET)
Subject: Canadian Pharmacy Message 90111634
Message-ID: <20090125142523.7CB811F747D@projects.linpro.no>

An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20090125/bff45405/attachment.html>

From ottolski at web.de  Mon Jan 26 08:17:44 2009
From: ottolski at web.de (Sascha Ottolski)
Date: Mon, 26 Jan 2009 09:17:44 +0100
Subject: Panic message: Assert error in exp_timer(), cache_expire.c line 303
Message-ID: <200901260917.45223.ottolski@web.de>

Dear list,

after introducing a little change to my VCL, we now see crashes like 
this frequently:

Jan 25 01:49:43 localhost varnishd[6039]: Child (25613) not responding 
to ping, killing it.
Jan 25 01:49:43 localhost varnishd[6039]: Child (25613) died signal=6
Jan 25 01:49:43 localhost varnishd[6039]: Child (25613) Panic message: 
Assert error in exp_timer(), cache_expire.c line 303:   
Condition(oe2->timer_when >= oe->timer_when) not true.  thread = 
(cache-timeout)
Jan 25 01:49:43 localhost varnishd[6039]: Child cleanup complete
Jan 25 01:49:43 localhost varnishd[6039]: child (2663) Started

(I've also seen such crashes without the "not responding" messages 
before the Panic).

It probably has a close tie to this part in the VCL, as the crashes 
didn't happen without it, and the crashes seem to happen after roughly 
24 hours of runtime (but strange enough, didn't happen last night after 
it does some days in a row):

sub vcl_hit {
    if (req.request == "PURGE") {
        set obj.ttl = 0s;
        error 200 "Purged.";
    }
    # this line seems to trigger the crash
    set obj.ttl = 1d;
}

At the same time, varnish is started with "-t 86400". (BTW, it's a bit 
inconsistent that it doesn't seem to be allowed to say "-t 1d" like in 
the VCL).

I found a thread from december on the -dev list about a similar crash, 
but it doesn't appear to have been solved by now.

It all happens with trunk, r3497.

I'd appreciate any pointers to get around this. Is there anything I 
should see in the varnishlog about the crash reason? What to look for?


Thanks in advance,

Sascha


From lists at rebert.name  Mon Jan 26 13:19:18 2009
From: lists at rebert.name (lists at rebert.name)
Date: Mon, 26 Jan 2009 14:19:18 +0100
Subject: Varnish doesnt start ...
Message-ID: <497DB856.9000002@rebert.name>

Hi,
I have installed varnish with apt-get on a Debian i386 lenny ...

Does anyone know from where comes the problem ?

e003-xx:/home/etudiant# varnishd -f /etc/varnish/vcl.conf  -a 
0.0.0.0:80  -T 127.0.0.1:6969 -t 600 -w 1,4000,5 -h classic,500009  -s 
malloc,500M  -s file,/home/etudiant/VARNISH_CACHE/,1G
/usr/bin/ld: crti.o: No such file: No such file or directory
collect2: ld returned 1 exit status
pclose=256
Internal error: GCC returned 0x0100

Thanks !

----
REBERT Luc
Engineer student ESAIP Saint-Barthelemy d'Anjou
Networks enigneering
Website: www.luc.rebert.name



From allan.j at cobsen.dk  Mon Jan 26 14:10:57 2009
From: allan.j at cobsen.dk (Allan Jacobsen)
Date: Mon, 26 Jan 2009 15:10:57 +0100 (CET)
Subject: Varnish doesnt start ...
In-Reply-To: <497DB856.9000002@rebert.name>
References: <497DB856.9000002@rebert.name>
Message-ID: <21851.195.249.17.5.1232979057.squirrel@tn070422.typomedia.dk>

On Man, Januar 26, 2009 14:19, lists at rebert.name wrote:
> Hi,
> I have installed varnish with apt-get on a Debian i386 lenny ...
>
>
> Does anyone know from where comes the problem ?
>
>
> e003-xx:/home/etudiant# varnishd -f /etc/varnish/vcl.conf  -a
> 0.0.0.0:80  -T 127.0.0.1:6969 -t 600 -w 1,4000,5 -h classic,500009  -s
> malloc,500M  -s file,/home/etudiant/VARNISH_CACHE/,1G /usr/bin/ld: crti.o:
> No such file: No such file or directory
> collect2: ld returned 1 exit status
> pclose=256 Internal error: GCC returned 0x0100
>
It could look like you dont have a working compiler on the server ?

Best regards
Allan Jacobsen



From stonorn at giraffen.dk  Tue Jan 27 13:54:44 2009
From: stonorn at giraffen.dk (Anton Stonor)
Date: Tue, 27 Jan 2009 14:54:44 +0100
Subject: Cacheability - changed in Varnish 2? 
Message-ID: <497F1224.5060907@giraffen.dk>

Hi there,

Is it just me, my setup or has "cacheable" classification changed 
between Varnish 1 and Varnish 2?

In the old days you could just say e.g.

sub vcl_fetch {
         set obj.ttl = 1d;
}

and Varnish would cache everything for one day.

I Varnish 2 it looks like the objects must have either "Cache-Control" 
or "Expires" to be cached at all -- and then you can tamper with obj.ttl 
afterwards. If the backend doesn't set those headers, "set obj.ttl" 
doesn't seem have any effect.

It feels a bit limiting compared to the old days. If I want to cache 
more aggressive its much easier just to change a few lines in the 
Varnish conf than adding cache headers to the backend.

So is there a way to configure Varnish 2 to cache objects with no cache 
headers?

/Anton




From isharov at yandex-team.ru  Tue Jan 27 21:55:17 2009
From: isharov at yandex-team.ru (Iliya Sharov)
Date: Wed, 28 Jan 2009 00:55:17 +0300
Subject: epoll && linux 2.6
Message-ID: <497F82C5.7080407@yandex-team.ru>

Hello,

I have varnishd 2.0.1 in Debian Etch (2.6.22, 2.6.26 kernel)
I'm very strange, it doesn't useing epoll. 
That's the output of my configure and strace:

checking for epoll_ctl... yes
checking for poll... yes

writev(14, [{"200 19      \n", 13}, {"PONG 1233093165 1.0", 19}, {"\n", 1}], 3) = 33
poll([{fd=11, events=POLLIN, revents=POLLIN}], 1, -1) = 1
read(11, "ping\n", 8191)

In varnishlog -i "Debug" output i see 
Debug        - "Acceptor is epoll"

but really epoll don't enabled :(

With best regards,
Iliya




From phk at phk.freebsd.dk  Tue Jan 27 21:59:17 2009
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Tue, 27 Jan 2009 21:59:17 +0000
Subject: epoll && linux 2.6 
In-Reply-To: Your message of "Wed, 28 Jan 2009 00:55:17 +0300."
	<497F82C5.7080407@yandex-team.ru> 
Message-ID: <11027.1233093557@critter.freebsd.dk>

In message <497F82C5.7080407 at yandex-team.ru>, Iliya Sharov writes:
>Hello,
>
>I have varnishd 2.0.1 in Debian Etch (2.6.22, 2.6.26 kernel)
>I'm very strange, it doesn't useing epoll. 
>That's the output of my configure and strace:
>
>checking for epoll_ctl... yes
>checking for poll... yes
>
>writev(14, [{"200 19      \n", 13}, {"PONG 1233093165 1.0", 19}, {"\n", 1}], 3) = 33
>poll([{fd=11, events=POLLIN, revents=POLLIN}], 1, -1) = 1
>read(11, "ping\n", 8191)

There are a number of places in varnish where we need to wait for
a single or a couple of filedescriptors, a task poll(2) is perfectly
capable of handling.

The CLI communication is one of them, as shown in your trace.

The place we attempt to use kqueue or epoll is for monitoring
idle client connections for new requests, because of the
sheer number of filedescriptors, select or poll would take
a significant performance hit.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From des at des.no  Tue Jan 27 23:48:26 2009
From: des at des.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=)
Date: Wed, 28 Jan 2009 00:48:26 +0100
Subject: What is Age header in varnish
In-Reply-To: <25daf17a0901230303n57d21c49tfb2d43998acbec63@mail.gmail.com>
	(Abhishek Singh's message of "Fri, 23 Jan 2009 16:33:30 +0530")
References: <25daf17a0901230303n57d21c49tfb2d43998acbec63@mail.gmail.com>
Message-ID: <864ozkb99x.fsf@ds4.des.no>

Abhishek Singh <abhishek.rhce at gmail.com> writes:
> Can anyone please explain that why Age header is changing on every refresh
> while i set "set obj.ttl" for 1 week and is the use of Age header. Is there
> anything that i have to modify in my varnish config file to cache index.php.
> please suggest something.

I was born a little over 31 years ago.  I have an estimated TTL of ~45
years.  Every day, my age increases by 86,400 seconds.  That doesn't
mean there's anything wrong with me :P

DES
-- 
Dag-Erling Sm?rgrav - des at des.no


From tim at metaweb.com  Wed Jan 28 00:30:41 2009
From: tim at metaweb.com (Tim Kientzle)
Date: Tue, 27 Jan 2009 16:30:41 -0800
Subject: What is Age header in varnish
In-Reply-To: <864ozkb99x.fsf@ds4.des.no>
References: <25daf17a0901230303n57d21c49tfb2d43998acbec63@mail.gmail.com>
	<864ozkb99x.fsf@ds4.des.no>
Message-ID: <F0C32305-B909-43EB-9238-BD4658AB1FA0@metaweb.com>

> Abhishek Singh <abhishek.rhce at gmail.com> writes:
>> Can anyone please explain that why Age header is changing on every  
>> refresh
>> while i set "set obj.ttl" for 1 week and is the use of Age header.  
>> Is there
>> anything that i have to modify in my varnish config file to cache  
>> index.php.
>> please suggest something.

If I understand correctly, the "Age" header is set by Varnish
to indicate how long the file has been in the cache.  It is
supposed to change on each request.

Why do you think index.php is not being cached?
Have you checked your webserver logs?

Tim Kientzle



From phk at phk.freebsd.dk  Wed Jan 28 09:11:28 2009
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Wed, 28 Jan 2009 09:11:28 +0000
Subject: renaming varnish concepts...
Message-ID: <29849.1233133888@critter.freebsd.dk>


A couple of places in varnish I have clearly missed the boat when it
comes to naming.

And a couple of places, I should have thought more carefully.

I wonder if we should correct these things in 2.1, in order to not
confuse future users more than necessary, or if we already have
sufficient users that it does not make sense to change things like
this now.

2.1 is going to be a flag day anyway, because of the changes which
persistent storage will cause, so I'm thinking this might be a
good time for this sort of house-cleaning ?

Input welcome.

1.  Purge vs. Ban
-----------------

The CLI and VCL commands are named "purge", but they don't, they
add a ban to the list of bans.

I would actually like to rename purge to ban and add a real purge
function that gets rid of the current object (ie: one found in the
cache) and possibly its "Vary:" siblings.

Purge does sound like it will be gone, whereas "ban" better explains
what happens when we use the delayed regexp checks.

Obviously, if I co-opt "purge" to mean something different, backwards
compat is not possible, and all your purge scripts and VCLs with
purge facilities will break.


2.  CLI greeting
----------------

When you connect to CLI port (-T argument) it does not announce
itself.  This was probably a mistake, as it makes it harder to
do error detection in scripts that talk to the CLI.

The problem is that if I add the greeting, I will break all
existing scripts that talk to the CLI.


3.  Acceptor vs. waiter
-----------------------

The subfunction we today call "acceptors" are not really, they are
waiters that monitor client connections for new requests.

The real acceptor is a separate piece of code.

This one is only visible in a command-line argument/parameter which
users generally only fiddle as part of debugging.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From phk at phk.freebsd.dk  Wed Jan 28 09:14:58 2009
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Wed, 28 Jan 2009 09:14:58 +0000
Subject: Cacheability - changed in Varnish 2? 
In-Reply-To: Your message of "Tue, 27 Jan 2009 14:54:44 +0100."
	<497F1224.5060907@giraffen.dk> 
Message-ID: <29882.1233134098@critter.freebsd.dk>

In message <497F1224.5060907 at giraffen.dk>, Anton Stonor writes:

>I Varnish 2 it looks like the objects must have either "Cache-Control" 
>or "Expires" to be cached at all -- and then you can tamper with obj.ttl 
>afterwards. If the backend doesn't set those headers, "set obj.ttl" 
>doesn't seem have any effect.

Can you get me varnishlog of such a request ?  I'm not able to
tell from your description which path we're talking about.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From jfbustarret at wat.tv  Wed Jan 28 09:33:31 2009
From: jfbustarret at wat.tv (BUSTARRET, Jean-francois)
Date: Wed, 28 Jan 2009 10:33:31 +0100
Subject: =?iso-8859-1?Q?RE=A0=3A_renaming_varnish_concepts=2E=2E=2E?=
References: <29849.1233133888@critter.freebsd.dk>
Message-ID: <53C652A09719C54DA24741D0157CB269025B0D2F@TFPRDEXS1.tf1.groupetf1.fr>

+1 for some house-cleaning.

- Speaking of bans, is grace used for "banned" objets ?

If i "ban" an object, the first request will regenerate it from the back-end. Will following requests be getting the stale version during regeneration ?

- For the CLI, why not add a "compatible mode" (inactive by default) to remove the greeting ?

Jean-Fran?ois


-------- Message d'origine--------
De: varnish-misc-bounces at projects.linpro.no de la part de Poul-Henning Kamp
Date: mer. 28/01/2009 10:11
?: varnish-misc at projects.linpro.no
Objet : renaming varnish concepts...
 

A couple of places in varnish I have clearly missed the boat when it
comes to naming.

And a couple of places, I should have thought more carefully.

I wonder if we should correct these things in 2.1, in order to not
confuse future users more than necessary, or if we already have
sufficient users that it does not make sense to change things like
this now.

2.1 is going to be a flag day anyway, because of the changes which
persistent storage will cause, so I'm thinking this might be a
good time for this sort of house-cleaning ?

Input welcome.

1.  Purge vs. Ban
-----------------

The CLI and VCL commands are named "purge", but they don't, they
add a ban to the list of bans.

I would actually like to rename purge to ban and add a real purge
function that gets rid of the current object (ie: one found in the
cache) and possibly its "Vary:" siblings.

Purge does sound like it will be gone, whereas "ban" better explains
what happens when we use the delayed regexp checks.

Obviously, if I co-opt "purge" to mean something different, backwards
compat is not possible, and all your purge scripts and VCLs with
purge facilities will break.


2.  CLI greeting
----------------

When you connect to CLI port (-T argument) it does not announce
itself.  This was probably a mistake, as it makes it harder to
do error detection in scripts that talk to the CLI.

The problem is that if I add the greeting, I will break all
existing scripts that talk to the CLI.


3.  Acceptor vs. waiter
-----------------------

The subfunction we today call "acceptors" are not really, they are
waiters that monitor client connections for new requests.

The real acceptor is a separate piece of code.

This one is only visible in a command-line argument/parameter which
users generally only fiddle as part of debugging.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.
_______________________________________________
varnish-misc mailing list
varnish-misc at projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20090128/33450f1f/attachment.html>

From phk at phk.freebsd.dk  Wed Jan 28 09:52:47 2009
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Wed, 28 Jan 2009 09:52:47 +0000
Subject: Panic message: Assert error in exp_timer(),
	cache_expire.c line 303 
In-Reply-To: Your message of "Mon, 26 Jan 2009 09:17:44 +0100."
	<200901260917.45223.ottolski@web.de> 
Message-ID: <30248.1233136367@critter.freebsd.dk>

In message <200901260917.45223.ottolski at web.de>, Sascha Ottolski writes:

>Assert error in exp_timer(), cache_expire.c line 303:   
>Condition(oe2->timer_when >= oe->timer_when) not true.  thread = 
>(cache-timeout)
>
>[...]
>
>It all happens with trunk, r3497.

Fixed in r3547

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From phk at phk.freebsd.dk  Wed Jan 28 09:54:26 2009
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Wed, 28 Jan 2009 09:54:26 +0000
Subject: Breaking Varnish 
In-Reply-To: Your message of "Fri, 23 Jan 2009 14:29:47 PST."
	<20090123222947.GB28138@digdug.corp.631h.metaweb.com> 
Message-ID: <30269.1233136466@critter.freebsd.dk>

In message <20090123222947.GB28138 at digdug.corp.631h.metaweb.com>, Niall O'Higgi
ns writes:

>>> Hi Tim,
>>>
>>> Can I get you to take -trunk for a spin ?
>>>
>>> At least the second of the problems you pasted I'm pretty sure I
>>> have nailed recently and the first one could easily be the same one
>>> in a different disguise.
>
>I've re-run the load test against varnish-trunk.  Trunk is better
>behaved, but I now get output like this over and over:
>
>child (19731) Started
>Child (19731) said Closed fds: 4 7 8 10 11
>Child (19731) said Child starts
>Child (19731) said managed to mmap 49929912320 bytes of 49929912320
>Child (19731) said Ready
>Child (19731) not responding to ping, killing it.

This is a typical indication of raw overload, what levels of traffic
are you hitting it with ?


-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From phk at phk.freebsd.dk  Wed Jan 28 10:05:54 2009
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Wed, 28 Jan 2009 10:05:54 +0000
Subject: =?iso-8859-1?Q?RE=A0=3A_renaming_varnish_concepts=2E=2E=2E?= 
In-Reply-To: Your message of "Wed, 28 Jan 2009 10:33:31 +0100."
	<53C652A09719C54DA24741D0157CB269025B0D2F@TFPRDEXS1.tf1.groupetf1.fr> 
Message-ID: <30419.1233137154@critter.freebsd.dk>

In message <53C652A09719C54DA24741D0157CB269025B0D2F at TFPRDEXS1.tf1.groupetf1.fr
>, "BUSTARRET, Jean-francois" writes:

>+1 for some house-cleaning.
>
>- Speaking of bans, is grace used for "banned" objets ?
>
>If i "ban" an object, the first request will regenerate it from the =
>back-end. Will following requests be getting the stale version during =
>regeneration ?

That's another thing I'm looking at right now, I would like to add
a bit of grading to bans, so that they can take the sematics of
"mark expired" (so it will be used for grace) vs. "GONE!" where
it will never be seen again.

>- For the CLI, why not add a "compatible mode" (inactive by default) to =
>remove the greeting ?

Mostly because I generally hate that kind of pointless options :-)

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From stonorn at giraffen.dk  Wed Jan 28 10:23:08 2009
From: stonorn at giraffen.dk (Anton Stonor)
Date: Wed, 28 Jan 2009 11:23:08 +0100
Subject: Cacheability - changed in Varnish 2?
In-Reply-To: <29882.1233134098@critter.freebsd.dk>
References: <29882.1233134098@critter.freebsd.dk>
Message-ID: <4980320C.8030306@giraffen.dk>

Poul-Henning Kamp wrote:

> Can you get me varnishlog of such a request ?  I'm not able to
> tell from your description which path we're talking about.

Sure, just thought this was intended behavior so I wouldn't bore with 
details. Here we go.

Running Varnish 2.0.2 on Ubuntu, has tested on Debian Etch 64bit as well.

Failure:

     9 SessionOpen  c 127.0.0.1 41768 127.0.0.1:8000
     9 ReqStart     c 127.0.0.1 41768 1998529221
     9 RxRequest    c GET
     9 RxURL        c /test_varnish
     9 RxProtocol   c HTTP/1.1
     9 RxHeader     c Host: localhost:8000
     9 RxHeader     c User-Agent: Mozilla/5.0 (X11; U; Linux i686; 
en-US; rv:1.9.0.5) Gecko/2008121622 Ubuntu/8.10 (intrepid) Firefox/3.0.5
     9 RxHeader     c Accept: 
text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
     9 RxHeader     c Accept-Language: en-us,en;q=0.5
     9 RxHeader     c Accept-Encoding: gzip,deflate
     9 RxHeader     c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
     9 RxHeader     c Keep-Alive: 300
     9 RxHeader     c Connection: keep-alive
     9 VCL_call     c recv
     9 VCL_return   c lookup
     9 VCL_call     c hash
     9 VCL_return   c hash
     9 HitPass      c 1998529216
     9 VCL_call     c pass
     9 VCL_return   c pass
     9 Backend      c 10 backend_0 backend_0
    10 TxRequest    - GET
    10 TxURL        - /test_varnish
    10 TxProtocol   - HTTP/1.1
    10 TxHeader     - Host: localhost:8000
    10 TxHeader     - User-Agent: Mozilla/5.0 (X11; U; Linux i686; 
en-US; rv:1.9.0.5) Gecko/2008121622 Ubuntu/8.10 (intrepid) Firefox/3.0.5
    10 TxHeader     - Accept: 
text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
    10 TxHeader     - Accept-Language: en-us,en;q=0.5
    10 TxHeader     - Accept-Encoding: gzip,deflate
    10 TxHeader     - Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
    10 TxHeader     - X-Varnish: 1998529221
    10 TxHeader     - X-Forwarded-For: 127.0.0.1
    10 RxProtocol   - HTTP/1.1
    10 RxStatus     - 200
    10 RxResponse   - OK
    10 RxHeader     - Server: Zope/(Zope 2.10.6-final, python 2.4.5, 
linux2) ZServer/1.1 Plone/3.1.5.1
    10 RxHeader     - Date: Wed, 28 Jan 2009 10:13:23 GMT
    10 RxHeader     - Content-Length: 4
    10 RxHeader     - Content-Type: text/plain; charset=utf-8
     9 ObjProtocol  c HTTP/1.1
     9 ObjStatus    c 200
     9 ObjResponse  c OK
     9 ObjHeader    c Server: Zope/(Zope 2.10.6-final, python 2.4.5, 
linux2) ZServer/1.1 Plone/3.1.5.1
     9 ObjHeader    c Date: Wed, 28 Jan 2009 10:13:23 GMT
     9 ObjHeader    c Content-Type: text/plain; charset=utf-8
    10 BackendReuse - backend_0
     9 TTL          c 1998529221 RFC 0 1233137603 0 0 0 0
     9 VCL_call     c fetch
     9 TTL          c 1998529221 VCL 86400 1233137604
     9 VCL_return   c pass
     9 Length       c 4
     9 VCL_call     c deliver
     9 VCL_return   c deliver
     9 TxProtocol   c HTTP/1.1
     9 TxStatus     c 200
     9 TxResponse   c OK
     9 TxHeader     c Server: Zope/(Zope 2.10.6-final, python 2.4.5, 
linux2) ZServer/1.1 Plone/3.1.5.1
     9 TxHeader     c Content-Type: text/plain; charset=utf-8
     9 TxHeader     c Content-Length: 4
     9 TxHeader     c X-Varnish-Action: FETCH (pass - not cacheable)
     9 TxHeader     c Date: Wed, 28 Jan 2009 10:13:23 GMT
     9 TxHeader     c X-Varnish: 1998529221
     9 TxHeader     c Age: 0
     9 TxHeader     c Via: 1.1 varnish
     9 TxHeader     c Connection: keep-alive
     9 ReqEnd       c 1998529221 1233137603.757895708 
1233137603.761220932 0.000055552 0.003257036 0.000068188
     0 StatAddr     - 127.0.0.1 0 760 6 6 0 5 6 1970 24


Same backend header, just added a cache-control request key. Succes:


    9 SessionOpen  c 127.0.0.1 41774 127.0.0.1:8000
     9 ReqStart     c 127.0.0.1 41774 1998529224
     9 RxRequest    c GET
     9 RxURL        c /test_varnish
     9 RxProtocol   c HTTP/1.1
     9 RxHeader     c Host: localhost:8000
     9 RxHeader     c User-Agent: Mozilla/5.0 (X11; U; Linux i686; 
en-US; rv:1.9.0.5) Gecko/2008121622 Ubuntu/8.10 (intrepid) Firefox/3.0.5
     9 RxHeader     c Accept: 
text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
     9 RxHeader     c Accept-Language: en-us,en;q=0.5
     9 RxHeader     c Accept-Encoding: gzip,deflate
     9 RxHeader     c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
     9 RxHeader     c Keep-Alive: 300
     9 RxHeader     c Connection: keep-alive
     9 VCL_call     c recv
     9 VCL_return   c lookup
     9 VCL_call     c hash
     9 VCL_return   c hash
     9 HitPass      c 1998529216
     9 VCL_call     c pass
     9 VCL_return   c pass
     9 Backend      c 10 backend_0 backend_0
    10 TxRequest    - GET
    10 TxURL        - /test_varnish
    10 TxProtocol   - HTTP/1.1
    10 TxHeader     - Host: localhost:8000
    10 TxHeader     - User-Agent: Mozilla/5.0 (X11; U; Linux i686; 
en-US; rv:1.9.0.5) Gecko/2008121622 Ubuntu/8.10 (intrepid) Firefox/3.0.5
    10 TxHeader     - Accept: 
text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
    10 TxHeader     - Accept-Language: en-us,en;q=0.5
    10 TxHeader     - Accept-Encoding: gzip,deflate
    10 TxHeader     - Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
    10 TxHeader     - X-Varnish: 1998529224
    10 TxHeader     - X-Forwarded-For: 127.0.0.1
    10 RxProtocol   - HTTP/1.1
    10 RxStatus     - 200
    10 RxResponse   - OK
    10 RxHeader     - Server: Zope/(Zope 2.10.6-final, python 2.4.5, 
linux2) ZServer/1.1 Plone/3.1.5.1
    10 RxHeader     - Date: Wed, 28 Jan 2009 10:15:21 GMT
    10 RxHeader     - Content-Length: 4
    10 RxHeader     - Content-Type: text/plain; charset=utf-8
    10 RxHeader     - Cache-Control: max-age=1
     9 ObjProtocol  c HTTP/1.1
     9 ObjStatus    c 200
     9 ObjResponse  c OK
     9 ObjHeader    c Server: Zope/(Zope 2.10.6-final, python 2.4.5, 
linux2) ZServer/1.1 Plone/3.1.5.1
     9 ObjHeader    c Date: Wed, 28 Jan 2009 10:15:21 GMT
     9 ObjHeader    c Content-Type: text/plain; charset=utf-8
     9 ObjHeader    c Cache-Control: max-age=1
    10 BackendReuse - backend_0
     9 TTL          c 1998529224 RFC 1 1233137721 0 0 1 0
     9 VCL_call     c fetch
     9 TTL          c 1998529224 VCL 86400 1233137721
     9 VCL_return   c deliver
     9 Length       c 4
     9 VCL_call     c deliver
     9 VCL_return   c deliver
     9 TxProtocol   c HTTP/1.1
     9 TxStatus     c 200
     9 TxResponse   c OK
     9 TxHeader     c Server: Zope/(Zope 2.10.6-final, python 2.4.5, 
linux2) ZServer/1.1 Plone/3.1.5.1
     9 TxHeader     c Content-Type: text/plain; charset=utf-8
     9 TxHeader     c Cache-Control: max-age=1
     9 TxHeader     c Content-Length: 4
     9 TxHeader     c X-Varnish-Action: FETCH (insert)
     9 TxHeader     c Date: Wed, 28 Jan 2009 10:15:21 GMT
     9 TxHeader     c X-Varnish: 1998529224
     9 TxHeader     c Age: 0
     9 TxHeader     c Via: 1.1 varnish
     9 TxHeader     c Connection: keep-alive
     9 ReqEnd       c 1998529224 1233137721.165423393 
1233137721.168730259 0.000118971 0.003242731 0.000064135

And now the config file:


backend backend_0 {
	.host = "127.0.0.1";
	.port = "8080";
}


sub vcl_recv {
	set req.grace = 120s;
	set req.backend = backend_0;

}


sub vcl_hit {
	if (req.request == "PURGE") {
		purge_url(req.url);
		error 200 "Purged";
	}

	if (!obj.cacheable) {
         set obj.http.X-Varnish-Action = "PASS (not cacheable - hit)";
		pass;
	}
     set obj.http.X-Varnish-Action = "HIT (deliver - from cache)";
}


sub vcl_fetch {
	set obj.grace = 120s;
         set obj.ttl = 1d;


	if (!obj.cacheable) {
         set obj.http.X-Varnish-Action = "FETCH (pass - not cacheable)";
		pass;
	}
	if (obj.http.Set-Cookie) {
         set obj.http.X-Varnish-Action = "FETCH (pass - response sets 
cookie)";
		pass;
	}
	if (obj.http.Cache-Control ~ "(private|no-cache|no-store)") {
         set obj.http.X-Varnish-Action = "FETCH (pass - cache control 
disallows)";
		pass;
	}
	if (req.http.Authorization && !obj.http.Cache-Control ~ "public") {
         set obj.http.X-Varnish-Action = "FETCH (pass - authorized and 
no public cache control)";
		pass;
	}
	
     set obj.http.X-Varnish-Action = "FETCH (insert)";




/Anton


From ric at digitalmarbles.com  Wed Jan 28 10:28:47 2009
From: ric at digitalmarbles.com (Ricardo Newbery)
Date: Wed, 28 Jan 2009 02:28:47 -0800
Subject: [varnish] renaming varnish concepts...
In-Reply-To: <29849.1233133888@critter.freebsd.dk>
References: <29849.1233133888@critter.freebsd.dk>
Message-ID: <D0D631E2-5DC8-4B89-B496-47DE2BEA88E0@digitalmarbles.com>


On Jan 28, 2009, at 1:11 AM, Poul-Henning Kamp wrote:

> 1.  Purge vs. Ban
> -----------------
>
> The CLI and VCL commands are named "purge", but they don't, they
> add a ban to the list of bans.
>
> I would actually like to rename purge to ban and add a real purge
> function that gets rid of the current object (ie: one found in the
> cache) and possibly its "Vary:" siblings.
>
> Purge does sound like it will be gone, whereas "ban" better explains
> what happens when we use the delayed regexp checks.
>
> Obviously, if I co-opt "purge" to mean something different, backwards
> compat is not possible, and all your purge scripts and VCLs with
> purge facilities will break.



Can you explain what is the effective difference between a real purge  
and a ban?  Would both still kill all the Vary siblings?  Other than  
possibly releasing the memory quicker, I'm not sure why I should  
care   :-)

Ric




From phk at phk.freebsd.dk  Wed Jan 28 10:44:28 2009
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Wed, 28 Jan 2009 10:44:28 +0000
Subject: [varnish] renaming varnish concepts... 
In-Reply-To: Your message of "Wed, 28 Jan 2009 02:28:47 PST."
	<D0D631E2-5DC8-4B89-B496-47DE2BEA88E0@digitalmarbles.com> 
Message-ID: <41138.1233139468@critter.freebsd.dk>

In message <D0D631E2-5DC8-4B89-B496-47DE2BEA88E0 at digitalmarbles.com>, Ricardo N
ewbery writes:

>> 1.  Purge vs. Ban
>> -----------------
>
>Can you explain what is the effective difference between a real purge  
>and a ban?  Would both still kill all the Vary siblings?  Other than  
>possibly releasing the memory quicker, I'm not sure why I should  
>care   :-)

The sematics I would like to have:

expire:
	Can be done to an object found as cache hit.

	The object is marked as expired at this point in time, but
	may still be eligble for grace service.

purge:
	Can be done to an object found as cache hit.

	The object is gone as of this moment, and will never be
	seen again.

ban:
	Put a ban on an object, which takes effect if it is subject
	to a cache hit later on.


I would like expire & purge to have an option of also acting on all
other "Vary:" versions of an object.

I would also like a bans to have an option as to what they do to
banned objects: expire, purge (and option for taking all Varys.

expire & purge would only be available from VCL.

ban would be available from VCL or CLI.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From phk at phk.freebsd.dk  Wed Jan 28 10:55:56 2009
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Wed, 28 Jan 2009 10:55:56 +0000
Subject: Cacheability - changed in Varnish 2? 
In-Reply-To: Your message of "Wed, 28 Jan 2009 11:23:08 +0100."
	<4980320C.8030306@giraffen.dk> 
Message-ID: <57637.1233140156@critter.freebsd.dk>

In message <4980320C.8030306 at giraffen.dk>, Anton Stonor writes:
>Poul-Henning Kamp wrote:
>
>> Can you get me varnishlog of such a request ?  I'm not able to
>> tell from your description which path we're talking about.
>
>Sure, just thought this was intended behavior so I wouldn't bore with 
>details. Here we go.

Well, it may be intentional, I can't tell without the details :-)

>Running Varnish 2.0.2 on Ubuntu, has tested on Debian Etch 64bit as well.

>     9 VCL_call     c hash
>     9 VCL_return   c hash
>     9 HitPass      c 1998529216
>     9 VCL_call     c pass
>     9 VCL_return   c pass

This is the important bit, you have a cached object that says this
object cannot be cached, but should be passed.

As long as that object exists, you will always hit pass.

These "magic" objects are created when vcl_fetch() takes "pass" as
action.

>Same backend header, just added a cache-control request key. Succes:

Not really, you are still doing a pass:

>     9 VCL_call     c hash
>     9 VCL_return   c hash
>     9 HitPass      c 1998529216
>     9 VCL_call     c pass
>     9 VCL_return   c pass

If you have your varnish-log, the XID 1998529216 is the one to
look for, that created the "hit-for-pass" object for some
reason.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From ric at digitalmarbles.com  Wed Jan 28 11:15:51 2009
From: ric at digitalmarbles.com (Ricardo Newbery)
Date: Wed, 28 Jan 2009 03:15:51 -0800
Subject: [varnish] Re: Cacheability - changed in Varnish 2?
In-Reply-To: <4980320C.8030306@giraffen.dk>
References: <29882.1233134098@critter.freebsd.dk>
	<4980320C.8030306@giraffen.dk>
Message-ID: <1223EE2D-589D-4283-8D81-1D883F096CE0@digitalmarbles.com>


On Jan 28, 2009, at 2:23 AM, Anton Stonor wrote:

> sub vcl_recv {
> 	set req.grace = 120s;
> 	set req.backend = backend_0;
>
> }
>


Is this truly all you have in vcl_recv?  This will mean that any  
cookied requests will get passed.  Is this intentional?

Ric




From ric at digitalmarbles.com  Wed Jan 28 11:25:56 2009
From: ric at digitalmarbles.com (Ricardo Newbery)
Date: Wed, 28 Jan 2009 03:25:56 -0800
Subject: [varnish] Re: [varnish] renaming varnish concepts... 
In-Reply-To: <41138.1233139468@critter.freebsd.dk>
References: <41138.1233139468@critter.freebsd.dk>
Message-ID: <79A2CE8A-A135-4542-9C8C-1F4327FED5F2@digitalmarbles.com>


On Jan 28, 2009, at 2:44 AM, Poul-Henning Kamp wrote:

> In message, Ricardo Newbery writes:
>
>>> 1.  Purge vs. Ban
>>> -----------------
>>
>> Can you explain what is the effective difference between a real purge
>> and a ban?  Would both still kill all the Vary siblings?  Other than
>> possibly releasing the memory quicker, I'm not sure why I should
>> care   :-)
>
> The sematics I would like to have:
>
> expire:
> 	Can be done to an object found as cache hit.
>
> 	The object is marked as expired at this point in time, but
> 	may still be eligble for grace service.
>
> purge:
> 	Can be done to an object found as cache hit.
>
> 	The object is gone as of this moment, and will never be
> 	seen again.
>
> ban:
> 	Put a ban on an object, which takes effect if it is subject
> 	to a cache hit later on.
>
>
> I would like expire & purge to have an option of also acting on all
> other "Vary:" versions of an object.
>
> I would also like a bans to have an option as to what they do to
> banned objects: expire, purge (and option for taking all Varys.
>
> expire & purge would only be available from VCL.
>
> ban would be available from VCL or CLI.



Sorry, I'm still unclear...

Right now, doesn't purge_url also "ban" all Varys?

If so, then why would it matter whether a PURGE request resulted in a  
real "purge" or a "ban"?

Ric




From phk at phk.freebsd.dk  Wed Jan 28 11:31:55 2009
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Wed, 28 Jan 2009 11:31:55 +0000
Subject: [varnish] Re: [varnish] renaming varnish concepts... 
In-Reply-To: Your message of "Wed, 28 Jan 2009 03:25:56 PST."
	<79A2CE8A-A135-4542-9C8C-1F4327FED5F2@digitalmarbles.com> 
Message-ID: <67046.1233142315@critter.freebsd.dk>

In message <79A2CE8A-A135-4542-9C8C-1F4327FED5F2 at digitalmarbles.com>, Ricardo N
ewbery writes:

>Sorry, I'm still unclear...
>
>Right now, doesn't purge_url also "ban" all Varys?

Yes, but they won't be dealt with until they take a catch-hit.  The
idea is to deal with them all once we find the first one.

>If so, then why would it matter whether a PURGE request resulted in a  
>real "purge" or a "ban"?

It would get things out of the system faster.

This may not make a big difference to most sites, but very interactive
sites can have a LOT of purges going on.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From ric at digitalmarbles.com  Wed Jan 28 11:55:00 2009
From: ric at digitalmarbles.com (Ricardo Newbery)
Date: Wed, 28 Jan 2009 03:55:00 -0800
Subject: [varnish] Re: [varnish] Re: [varnish] renaming varnish
	concepts... 
In-Reply-To: <67046.1233142315@critter.freebsd.dk>
References: <67046.1233142315@critter.freebsd.dk>
Message-ID: <768C2A99-6D24-4D6F-B324-25E13CFBE0F7@digitalmarbles.com>


On Jan 28, 2009, at 3:31 AM, Poul-Henning Kamp wrote:

> In message <79A2CE8A- 
> A135-4542-9C8C-1F4327FED5F2 at digitalmarbles.com>, Ricardo N
> ewbery writes:
>
>> Sorry, I'm still unclear...
>>
>> Right now, doesn't purge_url also "ban" all Varys?
>
> Yes, but they won't be dealt with until they take a catch-hit.  The
> idea is to deal with them all once we find the first one.
>
>> If so, then why would it matter whether a PURGE request resulted in a
>> real "purge" or a "ban"?
>
> It would get things out of the system faster.
>
> This may not make a big difference to most sites, but very interactive
> sites can have a LOT of purges going on.


Cool... so why do you figure that backwards compatibility is not  
possible?  If my old purge scripts now start "purging" rather than  
"banning", why should anything break?

Ric




From phk at phk.freebsd.dk  Wed Jan 28 12:19:35 2009
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Wed, 28 Jan 2009 12:19:35 +0000
Subject: [varnish] Re: [varnish] Re: [varnish] renaming varnish
	concepts... 
In-Reply-To: Your message of "Wed, 28 Jan 2009 03:55:00 PST."
	<768C2A99-6D24-4D6F-B324-25E13CFBE0F7@digitalmarbles.com> 
Message-ID: <3497.1233145175@critter.freebsd.dk>

In message <768C2A99-6D24-4D6F-B324-25E13CFBE0F7 at digitalmarbles.com>, Ricardo N
ewbery writes:
>
>On Jan 28, 2009, at 3:31 AM, Poul-Henning Kamp wrote:
>
>> In message <79A2CE8A- 
>> A135-4542-9C8C-1F4327FED5F2 at digitalmarbles.com>, Ricardo N
>> ewbery writes:
>>
>>> Sorry, I'm still unclear...
>>>
>>> Right now, doesn't purge_url also "ban" all Varys?
>>
>> Yes, but they won't be dealt with until they take a catch-hit.  The
>> idea is to deal with them all once we find the first one.
>>
>>> If so, then why would it matter whether a PURGE request resulted in a
>>> real "purge" or a "ban"?
>>
>> It would get things out of the system faster.
>>
>> This may not make a big difference to most sites, but very interactive
>> sites can have a LOT of purges going on.
>
>
>Cool... so why do you figure that backwards compatibility is not  
>possible?  If my old purge scripts now start "purging" rather than  
>"banning", why should anything break?

Purge wouldn't be a CLI command

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From ric at digitalmarbles.com  Wed Jan 28 12:25:59 2009
From: ric at digitalmarbles.com (Ricardo Newbery)
Date: Wed, 28 Jan 2009 04:25:59 -0800
Subject: [varnish] renaming varnish 	concepts... 
In-Reply-To: <3497.1233145175@critter.freebsd.dk>
References: <3497.1233145175@critter.freebsd.dk>
Message-ID: <E9DD9378-1DAE-4077-A0C6-149DBAC0C49F@digitalmarbles.com>


On Jan 28, 2009, at 4:19 AM, Poul-Henning Kamp wrote:

> In message <768C2A99-6D24-4D6F- 
> B324-25E13CFBE0F7 at digitalmarbles.com>, Ricardo N
> ewbery writes:
>> Cool... so why do you figure that backwards compatibility is not
>> possible?  If my old purge scripts now start "purging" rather than
>> "banning", why should anything break?
>
> Purge wouldn't be a CLI command


Ah, okay...  why not?

Ric




From phk at phk.freebsd.dk  Wed Jan 28 12:30:58 2009
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Wed, 28 Jan 2009 12:30:58 +0000
Subject: [varnish] renaming varnish concepts... 
In-Reply-To: Your message of "Wed, 28 Jan 2009 04:25:59 PST."
	<E9DD9378-1DAE-4077-A0C6-149DBAC0C49F@digitalmarbles.com> 
Message-ID: <3576.1233145858@critter.freebsd.dk>

In message <E9DD9378-1DAE-4077-A0C6-149DBAC0C49F at digitalmarbles.com>, Ricardo N
ewbery writes:
>
>On Jan 28, 2009, at 4:19 AM, Poul-Henning Kamp wrote:
>
>> In message <768C2A99-6D24-4D6F- 
>> B324-25E13CFBE0F7 at digitalmarbles.com>, Ricardo N
>> ewbery writes:
>>> Cool... so why do you figure that backwards compatibility is not
>>> possible?  If my old purge scripts now start "purging" rather than
>>> "banning", why should anything break?
>>
>> Purge wouldn't be a CLI command
>
>Ah, okay...  why not?

Because you don't have a cached object at hand to purge, all you can
do from the CLI is to add bans that will deal with the objects when
they are found in the cache later on.

Your question is -exactly- why I want the rename:  purge sounds like
something happens to the object right now, and that is not possible
from the CLI context.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From ric at digitalmarbles.com  Wed Jan 28 13:10:37 2009
From: ric at digitalmarbles.com (Ricardo Newbery)
Date: Wed, 28 Jan 2009 05:10:37 -0800
Subject: [varnish] renaming varnish concepts... 
In-Reply-To: <3576.1233145858@critter.freebsd.dk>
References: <3576.1233145858@critter.freebsd.dk>
Message-ID: <E9E201B6-733C-48DD-B083-8BB94A6F4EBE@digitalmarbles.com>


On Jan 28, 2009, at 4:30 AM, Poul-Henning Kamp wrote:

> In message, Ricardo Newbery writes:
>>
>> On Jan 28, 2009, at 4:19 AM, Poul-Henning Kamp wrote:
>>>
>>> Purge wouldn't be a CLI command
>>
>> Ah, okay...  why not?
>
> Because you don't have a cached object at hand to purge, all you can
> do from the CLI is to add bans that will deal with the objects when
> they are found in the cache later on.
>
> Your question is -exactly- why I want the rename:  purge sounds like
> something happens to the object right now, and that is not possible
> from the CLI context.


Sure, I understand the motivation.  But FWIW, I already knew what  
purge meant in the varnish context.  I'm just trying to understand the  
implications of the proposed change.

I'm unclear on why we can't acquire the cached object from the CLI to  
do the purge.  I imagine the most common usecase is to purge based on  
a known url, and with the url don't we have enough information to get  
at the cache and purge all variants?

Ric




From plfgoa at gmail.com  Wed Jan 28 16:25:36 2009
From: plfgoa at gmail.com (Paras Fadte)
Date: Wed, 28 Jan 2009 21:55:36 +0530
Subject: [varnish] renaming varnish concepts...
In-Reply-To: <E9E201B6-733C-48DD-B083-8BB94A6F4EBE@digitalmarbles.com>
References: <3576.1233145858@critter.freebsd.dk>
	<E9E201B6-733C-48DD-B083-8BB94A6F4EBE@digitalmarbles.com>
Message-ID: <75cf5800901280825w50d09916r12ece84c832eb27b@mail.gmail.com>

Why is it not possible to purge a URL from CLI for a particular host ?

-Paras

On Wed, Jan 28, 2009 at 6:40 PM, Ricardo Newbery <ric at digitalmarbles.com> wrote:
>
> On Jan 28, 2009, at 4:30 AM, Poul-Henning Kamp wrote:
>
>> In message, Ricardo Newbery writes:
>>>
>>> On Jan 28, 2009, at 4:19 AM, Poul-Henning Kamp wrote:
>>>>
>>>> Purge wouldn't be a CLI command
>>>
>>> Ah, okay...  why not?
>>
>> Because you don't have a cached object at hand to purge, all you can
>> do from the CLI is to add bans that will deal with the objects when
>> they are found in the cache later on.
>>
>> Your question is -exactly- why I want the rename:  purge sounds like
>> something happens to the object right now, and that is not possible
>> from the CLI context.
>
>
> Sure, I understand the motivation.  But FWIW, I already knew what
> purge meant in the varnish context.  I'm just trying to understand the
> implications of the proposed change.
>
> I'm unclear on why we can't acquire the cached object from the CLI to
> do the purge.  I imagine the most common usecase is to purge based on
> a known url, and with the url don't we have enough information to get
> at the cache and purge all variants?
>
> Ric
>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-misc
>


From michael at dynamine.net  Wed Jan 28 17:29:21 2009
From: michael at dynamine.net (Michael S. Fischer)
Date: Wed, 28 Jan 2009 09:29:21 -0800
Subject: [varnish] renaming varnish concepts... 
In-Reply-To: <3576.1233145858@critter.freebsd.dk>
References: <3576.1233145858@critter.freebsd.dk>
Message-ID: <F988DE00-17BE-4DB9-8BE0-DDBCA2B4AB9A@dynamine.net>

On Jan 28, 2009, at 4:30 AM, Poul-Henning Kamp wrote:
> Your question is -exactly- why I want the rename:  purge sounds like
> something happens to the object right now, and that is not possible
> from the CLI context.

How about 'qpurge' ?

--Michael


From niallo at metaweb.com  Wed Jan 28 18:04:48 2009
From: niallo at metaweb.com (Niall O'Higgins)
Date: Wed, 28 Jan 2009 10:04:48 -0800
Subject: [+] Re: Breaking Varnish
In-Reply-To: <30269.1233136466@critter.freebsd.dk>
References: <sfid-+20090128-015501-+006.10-1@spamfilter.osbf.lua>
	<30269.1233136466@critter.freebsd.dk>
Message-ID: <20090128180448.GD28138@digdug.corp.631h.metaweb.com>

On Wed, Jan 28, 2009 at 09:54:26AM +0000, Poul-Henning Kamp wrote:
> >I've re-run the load test against varnish-trunk.  Trunk is better
> >behaved, but I now get output like this over and over:
> >
> >child (19731) Started
> >Child (19731) said Closed fds: 4 7 8 10 11
> >Child (19731) said Child starts
> >Child (19731) said managed to mmap 49929912320 bytes of 49929912320
> >Child (19731) said Ready
> >Child (19731) not responding to ping, killing it.
> 
> This is a typical indication of raw overload, what levels of traffic
> are you hitting it with ?

This kind of thing:

Transaction rate:            3776.65 trans/sec
Throughput:                     1.68 MB/sec
Concurrency:                   28.28

Does the parent process give up on restarting the child after a
certain number of failures?  I was surprised by the eventual complete
exit of varnishd with the message:

Pushing vcls failed: CLI communication error

Also, Varnish seems to be able to handle up to about double that load
for a while (we got up to 6701 t/sec), then it dies as above.  Seems
like it takes around 2-3 hours for the varnishd parent process
to die.

> 
> -- 
> Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
> phk at FreeBSD.ORG         | TCP/IP since RFC 956
> FreeBSD committer       | BSD since 4.3-tahoe    
> Never attribute to malice what can adequately be explained by incompetence.
> 

-- 
Niall O'Higgins
Software Engineer
Metaweb Technologies, Inc.


From phk at phk.freebsd.dk  Wed Jan 28 18:09:37 2009
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Wed, 28 Jan 2009 18:09:37 +0000
Subject: [+] Re: Breaking Varnish 
In-Reply-To: Your message of "Wed, 28 Jan 2009 10:04:48 PST."
	<20090128180448.GD28138@digdug.corp.631h.metaweb.com> 
Message-ID: <5214.1233166177@critter.freebsd.dk>

In message <20090128180448.GD28138 at digdug.corp.631h.metaweb.com>, Niall O'Higgi
ns writes:

>Transaction rate:            3776.65 trans/sec
>Throughput:                     1.68 MB/sec
>Concurrency:                   28.28
>
>Does the parent process give up on restarting the child after a
>certain number of failures?  I was surprised by the eventual complete
>exit of varnishd with the message:
>
>Pushing vcls failed: CLI communication error

It shouldn't do that, it should be able to restart it forever.

>Also, Varnish seems to be able to handle up to about double that load
>for a while (we got up to 6701 t/sec), then it dies as above.  Seems
>like it takes around 2-3 hours for the varnishd parent process
>to die.

Once you get to that level of load, the ability of the scheduler
to not do something stupid is paramount for survival.

Try to increase the "cli_timeout" parameter, it is probably set a
bit on the aggresive side by default.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From michael at dynamine.net  Wed Jan 28 18:18:48 2009
From: michael at dynamine.net (Michael S. Fischer)
Date: Wed, 28 Jan 2009 10:18:48 -0800
Subject: [+] Re: Breaking Varnish
In-Reply-To: <20090128180448.GD28138@digdug.corp.631h.metaweb.com>
References: <sfid-+20090128-015501-+006.10-1@spamfilter.osbf.lua>
	<30269.1233136466@critter.freebsd.dk>
	<20090128180448.GD28138@digdug.corp.631h.metaweb.com>
Message-ID: <136D469D-02D6-4144-AD1C-6A9E48F3D5E8@dynamine.net>

On Jan 28, 2009, at 10:04 AM, Niall O'Higgins wrote:
>> This is a typical indication of raw overload, what levels of traffic
>> are you hitting it with ?
>
> This kind of thing:
>
> Transaction rate:            3776.65 trans/sec
> Throughput:                     1.68 MB/sec
> Concurrency:                   28.28

That doesn't seem that high.  What OS/# CPUs are you using?

--Michael


From niallo at metaweb.com  Wed Jan 28 18:36:18 2009
From: niallo at metaweb.com (Niall O'Higgins)
Date: Wed, 28 Jan 2009 10:36:18 -0800
Subject: [+] Re: Breaking Varnish
In-Reply-To: <136D469D-02D6-4144-AD1C-6A9E48F3D5E8@dynamine.net>
References: <30269.1233136466@critter.freebsd.dk>
	<20090128180448.GD28138@digdug.corp.631h.metaweb.com>
	<sfid-H20090128-101902-+019.71-1@spamfilter.osbf.lua>
	<136D469D-02D6-4144-AD1C-6A9E48F3D5E8@dynamine.net>
Message-ID: <20090128183618.GE28138@digdug.corp.631h.metaweb.com>

On Wed, Jan 28, 2009 at 10:18:48AM -0800, Michael S. Fischer wrote:
> On Jan 28, 2009, at 10:04 AM, Niall O'Higgins wrote:
>>> This is a typical indication of raw overload, what levels of traffic
>>> are you hitting it with ?
>>
>> This kind of thing:
>>
>> Transaction rate:            3776.65 trans/sec
>> Throughput:                     1.68 MB/sec
>> Concurrency:                   28.28
>
> That doesn't seem that high.  What OS/# CPUs are you using?

Varnish is running on a dual CPU (amd64) Linux 2.6 machine.  We have
pushed it up to 6701 t/sec with multiple load-generation machines.  We
see the same child-restart behaviour whether we use a single
load-generation machine, or three.

>
> --Michael
>

-- 
Niall O'Higgins
Software Engineer
Metaweb Technologies, Inc.


From phk at phk.freebsd.dk  Wed Jan 28 18:52:00 2009
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Wed, 28 Jan 2009 18:52:00 +0000
Subject: [+] Re: Breaking Varnish 
In-Reply-To: Your message of "Wed, 28 Jan 2009 10:36:18 PST."
	<20090128183618.GE28138@digdug.corp.631h.metaweb.com> 
Message-ID: <5390.1233168720@critter.freebsd.dk>

In message <20090128183618.GE28138 at digdug.corp.631h.metaweb.com>, Niall O'Higgi
ns writes:
>On Wed, Jan 28, 2009 at 10:18:48AM -0800, Michael S. Fischer wrote:
>> On Jan 28, 2009, at 10:04 AM, Niall O'Higgins wrote:
>
>Varnish is running on a dual CPU (amd64) Linux 2.6 machine.  We have
>pushed it up to 6701 t/sec with multiple load-generation machines.  We
>see the same child-restart behaviour whether we use a single
>load-generation machine, or three.

As I said, increase the cli_timeout parameter, it is probably
to short for that kind of scenario.

Also, you should probably set srcaddr_ttl to zero, to disable
the (effectively unused) per source-IP statistics.


-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From tim at metaweb.com  Wed Jan 28 19:11:48 2009
From: tim at metaweb.com (Tim Kientzle)
Date: Wed, 28 Jan 2009 11:11:48 -0800
Subject: Breaking Varnish 
In-Reply-To: <30269.1233136466@critter.freebsd.dk>
References: <30269.1233136466@critter.freebsd.dk>
Message-ID: <F628E190-8DBB-46F9-8973-D9C0B0F67C9F@metaweb.com>

On Jan 28, 2009, at 1:54 AM, Poul-Henning Kamp wrote:
> In message <20090123222947.GB28138 at digdug.corp.631h.metaweb.com>,  
> Niall O'Higgi
> ns writes:
>>>> Can I get you to take -trunk for a spin ?
>>>>
>>>> At least the second of the problems you pasted I'm pretty sure I
>>>> have nailed recently and the first one could easily be the same one
>>>> in a different disguise.
>>
>> I've re-run the load test against varnish-trunk.  Trunk is better
>> behaved, but I now get output like this over and over:
>>
>> child (19731) Started
>> Child (19731) said Closed fds: 4 7 8 10 11
>> Child (19731) said Child starts
>> Child (19731) said managed to mmap 49929912320 bytes of 49929912320
>> Child (19731) said Ready
>> Child (19731) not responding to ping, killing it.
>
> This is a typical indication of raw overload, what levels of traffic
> are you hitting it with ?

Pretty heavy.  We put together a test workload that saturated
Squid at around 1500 req/s on a dual-core dev systems.  The symptoms
above appeared somewhere above 6000 req/s on the same hardware
and workload.

The test has two goals:
  1) To try to find bugs in Varnish that might prevent us from
     switching to Varnish from Squid.
  2) To understand how Varnish behaves when it becomes
     saturated.

When testing Squid in this fashion, we found no bugs.  Under
heavy load, Squid did become very slow but recovered cleanly
and went back into normal operation as soon as the load was
removed.

Varnish didn't fare quite so well.  We did find bugs, as you
know.  Fortunately, those seem to be fixed in trunk.  (When do
you expect the next point release?)

It also appears that Varnish eventually exits completely
if placed under high load.  I'm okay with that as long as it's
intentional behavior; we have a standard nanny that we use in
production to restart crashed services anyway.  Of course,
I understand that killing the child and starting a new one
will also lose the cache, which is obviously not particularly
desirable under heavy load.

Cheers,

Tim



From phk at phk.freebsd.dk  Wed Jan 28 20:34:06 2009
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Wed, 28 Jan 2009 20:34:06 +0000
Subject: Breaking Varnish 
In-Reply-To: Your message of "Wed, 28 Jan 2009 11:11:48 PST."
	<F628E190-8DBB-46F9-8973-D9C0B0F67C9F@metaweb.com> 
Message-ID: <5923.1233174846@critter.freebsd.dk>

In message <F628E190-8DBB-46F9-8973-D9C0B0F67C9F at metaweb.com>, Tim Kientzle wri
tes:

>It also appears that Varnish eventually exits completely
>if placed under high load.  I'm okay with that as long as it's
>intentional behavior; 

It is not intentional.

The entire point about the two-process trick is to not ever throw
in the towel if we can avoid it.

That said, there are classes of bugs for which we have no hope,
if for instance the manager process cannot fork or allocate
memory, then we are hosed top and bottom.

>Of course,
>I understand that killing the child and starting a new one
>will also lose the cache, which is obviously not particularly
>desirable under heavy load.

Persistent storage coming up in version 2.1 :-)

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From phk at phk.freebsd.dk  Wed Jan 28 20:36:19 2009
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Wed, 28 Jan 2009 20:36:19 +0000
Subject: [varnish] renaming varnish concepts... 
In-Reply-To: Your message of "Wed, 28 Jan 2009 21:55:36 +0530."
	<75cf5800901280825w50d09916r12ece84c832eb27b@mail.gmail.com> 
Message-ID: <5946.1233174979@critter.freebsd.dk>

In message <75cf5800901280825w50d09916r12ece84c832eb27b at mail.gmail.com>, Paras 
Fadte writes:

>Why is it not possible to purge a URL from CLI for a particular host ?

You can enter a ban for it.

Prior to the new code in -trunk, you need to use "purge.hash" and
construct an appropriate regexp.

With the new code, which will possibly be in 2.0.3, you can express
it sensibly:

	purge req.url ~ "some_regexp" && req.http.host ~ "some other regexp"

But we cannot do an object lookup, so we can not reclaim the storage
of those objects immediately.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From stonorn at giraffen.dk  Thu Jan 29 00:27:04 2009
From: stonorn at giraffen.dk (Anton Stonor)
Date: Thu, 29 Jan 2009 01:27:04 +0100
Subject: Cacheability - changed in Varnish 2?
In-Reply-To: <57637.1233140156@critter.freebsd.dk>
References: <57637.1233140156@critter.freebsd.dk>
Message-ID: <4980F7D8.8090405@giraffen.dk>

Hi Poul-Henning,


>>     9 VCL_call     c hash
>>     9 VCL_return   c hash
>>     9 HitPass      c 1998529216
>>     9 VCL_call     c pass
>>     9 VCL_return   c pass
> 
> This is the important bit, you have a cached object that says this
> object cannot be cached, but should be passed.

Thanks for digging into this and pointing out the relevant part. I see 
that my log-snippets wasn't that useful to nail the issue.

New try. First, a request with no expire or cache-control header.

     9 RxURL        c /test_varnish_noheader
     9 RxProtocol   c HTTP/1.1
     9 RxHeader     c Host: localhost:8000
     9 RxHeader     c User-Agent: Mozilla/5.0 (X11; U; Linux i686; 
en-US; rv:1.9.0.5) Gecko/2008121622 Ubuntu/8.10 (intrepid) Firefox/3.0.5
     9 RxHeader     c Accept: 
text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
     9 RxHeader     c Accept-Language: en-us,en;q=0.5
     9 RxHeader     c Accept-Encoding: gzip,deflate
     9 RxHeader     c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
     9 RxHeader     c Keep-Alive: 300
     9 RxHeader     c Connection: keep-alive
     9 VCL_call     c recv
     9 VCL_return   c lookup
     9 VCL_call     c hash
     9 VCL_return   c hash
     9 VCL_call     c miss
     9 VCL_return   c fetch
    10 BackendOpen  b backend_0 127.0.0.1 36975 127.0.0.1 8080
     9 Backend      c 10 backend_0 backend_0
    10 TxRequest    b GET
    10 TxURL        b /test_varnish_noheader
    10 TxProtocol   b HTTP/1.1
    10 TxHeader     b Host: localhost:8000
    10 TxHeader     b User-Agent: Mozilla/5.0 (X11; U; Linux i686; 
en-US; rv:1.9.0.5) Gecko/2008121622 Ubuntu/8.10 (intrepid) Firefox/3.0.5
    10 TxHeader     b Accept: 
text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
    10 TxHeader     b Accept-Language: en-us,en;q=0.5
    10 TxHeader     b Accept-Encoding: gzip,deflate
    10 TxHeader     b Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
    10 TxHeader     b X-Varnish: 1495399095
    10 TxHeader     b X-Forwarded-For: 127.0.0.1
    10 RxProtocol   b HTTP/1.1
    10 RxStatus     b 200
    10 RxResponse   b OK
    10 RxHeader     b Server: Zope/(Zope 2.10.6-final, python 2.4.5, 
linux2) ZServer/1.1 Plone/3.1.5.1
    10 RxHeader     b Date: Thu, 29 Jan 2009 00:10:40 GMT
    10 RxHeader     b Content-Length: 4
    10 RxHeader     b Content-Type: text/plain; charset=utf-8
     9 ObjProtocol  c HTTP/1.1
     9 ObjStatus    c 200
     9 ObjResponse  c OK
     9 ObjHeader    c Server: Zope/(Zope 2.10.6-final, python 2.4.5, 
linux2) ZServer/1.1 Plone/3.1.5.1
     9 ObjHeader    c Date: Thu, 29 Jan 2009 00:10:40 GMT
     9 ObjHeader    c Content-Type: text/plain; charset=utf-8
    10 BackendReuse b backend_0
     9 TTL          c 1495399095 RFC 0 1233187840 0 0 0 0
     9 VCL_call     c fetch
     9 TTL          c 1495399095 VCL 86400 1233187841
     9 VCL_return   c pass
     9 Length       c 4
     9 VCL_call     c deliver
     9 VCL_return   c deliver


"pass" is called inside vcl_fetch because the object is classified as 
!cachable. Subsequent requests gives same result as in my first mail:

     9 VCL_call     c recv
     9 VCL_return   c lookup
     9 VCL_call     c hash
     9 VCL_return   c hash
     9 HitPass      c 1495399095
     9 VCL_call     c pass
     9 VCL_return   c pass
     9 Backend


Then, restart Varnish, same vcl file, same backend reponse except that 
we are now adding a cache-control header:


   9 RxRequest    c GET
     9 RxURL        c /test_varnish
     9 RxProtocol   c HTTP/1.1
     9 RxHeader     c Host: localhost:8000
     9 RxHeader     c User-Agent: Mozilla/5.0 (X11; U; Linux i686; 
en-US; rv:1.9.0.5) Gecko/2008121622 Ubuntu/8.10 (intrepid) Firefox/3.0.5
     9 RxHeader     c Accept: 
text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
     9 RxHeader     c Accept-Language: en-us,en;q=0.5
     9 RxHeader     c Accept-Encoding: gzip,deflate
     9 RxHeader     c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
     9 RxHeader     c Keep-Alive: 300
     9 RxHeader     c Connection: keep-alive
     9 VCL_call     c recv
     9 VCL_return   c lookup
     9 VCL_call     c hash
     9 VCL_return   c hash
     9 VCL_call     c miss
     9 VCL_return   c fetch
    10 BackendOpen  b backend_0 127.0.0.1 38871 127.0.0.1 8080
     9 Backend      c 10 backend_0 backend_0
    10 TxRequest    b GET
    10 TxURL        b /test_varnish
    10 TxProtocol   b HTTP/1.1
    10 TxHeader     b Host: localhost:8000
    10 TxHeader     b User-Agent: Mozilla/5.0 (X11; U; Linux i686; 
en-US; rv:1.9.0.5) Gecko/2008121622 Ubuntu/8.10 (intrepid) Firefox/3.0.5
    10 TxHeader     b Accept: 
text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
    10 TxHeader     b Accept-Language: en-us,en;q=0.5
    10 TxHeader     b Accept-Encoding: gzip,deflate
    10 TxHeader     b Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
    10 TxHeader     b X-Varnish: 1198489031
    10 TxHeader     b X-Forwarded-For: 127.0.0.1
    10 RxProtocol   b HTTP/1.1
    10 RxStatus     b 200
    10 RxResponse   b OK
    10 RxHeader     b Server: Zope/(Zope 2.10.6-final, python 2.4.5, 
linux2) ZServer/1.1 Plone/3.1.5.1
    10 RxHeader     b Date: Thu, 29 Jan 2009 00:20:42 GMT
    10 RxHeader     b Content-Length: 4
    10 RxHeader     b Content-Type: text/plain; charset=utf-8
    10 RxHeader     b Cache-Control: max-age=1
     9 ObjProtocol  c HTTP/1.1
     9 ObjStatus    c 200
     9 ObjResponse  c OK
     9 ObjHeader    c Server: Zope/(Zope 2.10.6-final, python 2.4.5, 
linux2) ZServer/1.1 Plone/3.1.5.1
     9 ObjHeader    c Date: Thu, 29 Jan 2009 00:20:42 GMT
     9 ObjHeader    c Content-Type: text/plain; charset=utf-8
     9 ObjHeader    c Cache-Control: max-age=1
    10 BackendReuse b backend_0
     9 TTL          c 1198489031 RFC 1 1233188442 0 0 1 0
     9 VCL_call     c fetch
     9 TTL          c 1198489031 VCL 86400 1233188443
     9 VCL_return   c deliver
     9 Length       c 4
     9 VCL_call     c deliver
     9 VCL_return   c deliver
     9 TxProtocol   c HTTP/1.1
     9 TxStatus     c 200
     9 TxResponse   c OK
     9 TxHeader     c Server: Zope/(Zope 2.10.6-final, python 2.4.5, 
linux2) ZServer/1.1 Plone/3.1.5.1
     9 TxHeader     c Content-Type: text/plain; charset=utf-8
     9 TxHeader     c Cache-Control: max-age=1
     9 TxHeader     c Content-Length: 4
     9 TxHeader     c X-Varnish-Action: FETCH (insert)
     9 TxHeader     c Date: Thu, 29 Jan 2009 00:20:42 GMT
     9 TxHeader     c X-Varnish: 1198489031
     9 TxHeader     c Age: 0
     9 TxHeader     c Via: 1.1 varnish


Subsequent requests gives a cache hit (and respects my obj.tt value):

    9 RxRequest    c GET
     9 RxURL        c /test_varnish
     9 RxProtocol   c HTTP/1.1
     9 RxHeader     c Host: localhost:8000
     9 RxHeader     c User-Agent: Mozilla/5.0 (X11; U; Linux i686; 
en-US; rv:1.9.0.5) Gecko/2008121622 Ubuntu/8.10 (intrepid) Firefox/3.0.5
     9 RxHeader     c Accept: 
text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
     9 RxHeader     c Accept-Language: en-us,en;q=0.5
     9 RxHeader     c Accept-Encoding: gzip,deflate
     9 RxHeader     c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
     9 RxHeader     c Keep-Alive: 300
     9 RxHeader     c Connection: keep-alive
     9 VCL_call     c recv
     9 VCL_return   c lookup
     9 VCL_call     c hash
     9 VCL_return   c hash
     9 Hit          c 1198489031
     9 VCL_call     c hit
     9 VCL_return   c deliver
     9 Length       c 4
     9 VCL_call     c deliver
     9 VCL_return   c deliver
     9 TxProtocol   c HTTP/1.1
     9 TxStatus     c 200
     9 TxResponse   c OK
     9 TxHeader     c Server: Zope/(Zope 2.10.6-final, python 2.4.5, 
linux2) ZServer/1.1 Plone/3.1.5.1
     9 TxHeader     c Content-Type: text/plain; charset=utf-8
     9 TxHeader     c Cache-Control: max-age=1
     9 TxHeader     c Content-Length: 4
     9 TxHeader     c X-Varnish-Action: HIT (deliver - from cache)
     9 TxHeader     c Date: Thu, 29 Jan 2009 00:22:26 GMT
     9 TxHeader     c X-Varnish: 1198489032 1198489031
     9 TxHeader     c Age: 104



I haven't had any luck to get objects without proper "cache-control" or 
"expires" to cache. Hope there is a workaround.

/Anton


From stonorn at giraffen.dk  Thu Jan 29 00:29:56 2009
From: stonorn at giraffen.dk (Anton Stonor)
Date: Thu, 29 Jan 2009 01:29:56 +0100
Subject: [varnish] Re: Cacheability - changed in Varnish 2?
In-Reply-To: <1223EE2D-589D-4283-8D81-1D883F096CE0@digitalmarbles.com>
References: <29882.1233134098@critter.freebsd.dk>
	<4980320C.8030306@giraffen.dk>
	<1223EE2D-589D-4283-8D81-1D883F096CE0@digitalmarbles.com>
Message-ID: <4980F884.9090904@giraffen.dk>

Ricardo Newbery skrev:

>> sub vcl_recv {
>>     set req.grace = 120s;
>>     set req.backend = backend_0;
>>
>> }

> Is this truly all you have in vcl_recv?  This will mean that any cookied 
> requests will get passed.  Is this intentional?

No, this is not a production setup. My problem is not that I cache too 
much, but the opposite.

And yep, I know about the cookie issue: 
http://markmail.org/message/pfpx7lanicpumsdg

Thanks for noticing.

/Anton




From ric at digitalmarbles.com  Thu Jan 29 00:48:36 2009
From: ric at digitalmarbles.com (Ricardo Newbery)
Date: Wed, 28 Jan 2009 16:48:36 -0800
Subject: [varnish] Re: Cacheability - changed in Varnish 2?
In-Reply-To: <4980F884.9090904@giraffen.dk>
References: <29882.1233134098@critter.freebsd.dk>
	<4980320C.8030306@giraffen.dk>
	<1223EE2D-589D-4283-8D81-1D883F096CE0@digitalmarbles.com>
	<4980F884.9090904@giraffen.dk>
Message-ID: <CCA2614A-1CB0-4FEF-BA28-B6980EB7C2C7@digitalmarbles.com>


On Jan 28, 2009, at 4:29 PM, Anton Stonor wrote:

> Ricardo Newbery skrev:
>
>>> sub vcl_recv {
>>>    set req.grace = 120s;
>>>    set req.backend = backend_0;
>>>
>>> }
>
>> Is this truly all you have in vcl_recv?  This will mean that any  
>> cookied
>> requests will get passed.  Is this intentional?
>
> No, this is not a production setup. My problem is not that I cache too
> much, but the opposite.
>
> And yep, I know about the cookie issue:
> http://markmail.org/message/pfpx7lanicpumsdg
>
> Thanks for noticing.
>
> /Anton


Sorry, I'm confused.  Maybe I'm misunderstanding what you're saying  
here, but the cookie issue will mean that you will cache too little,  
not too much.

Ric




From phk at phk.freebsd.dk  Thu Jan 29 08:34:41 2009
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Thu, 29 Jan 2009 08:34:41 +0000
Subject: Cacheability - changed in Varnish 2? 
In-Reply-To: Your message of "Thu, 29 Jan 2009 01:27:04 +0100."
	<4980F7D8.8090405@giraffen.dk> 
Message-ID: <8731.1233218081@critter.freebsd.dk>

In message <4980F7D8.8090405 at giraffen.dk>, Anton Stonor writes:

>New try. First, a request with no expire or cache-control header.

>    10 RxProtocol   b HTTP/1.1
>    10 RxStatus     b 200
>    10 RxResponse   b OK
>    10 RxHeader     b Server: Zope/(Zope 2.10.6-final, python 2.4.5, 
>linux2) ZServer/1.1 Plone/3.1.5.1
>    10 RxHeader     b Date: Thu, 29 Jan 2009 00:10:40 GMT
>    10 RxHeader     b Content-Length: 4
>    10 RxHeader     b Content-Type: text/plain; charset=utf-8
>     9 ObjProtocol  c HTTP/1.1
>     9 ObjStatus    c 200
>     9 ObjResponse  c OK
>     9 ObjHeader    c Server: Zope/(Zope 2.10.6-final, python 2.4.5, 
>linux2) ZServer/1.1 Plone/3.1.5.1
>     9 ObjHeader    c Date: Thu, 29 Jan 2009 00:10:40 GMT
>     9 ObjHeader    c Content-Type: text/plain; charset=utf-8
>    10 BackendReuse b backend_0
>     9 TTL          c 1495399095 RFC 0 1233187840 0 0 0 0


As far as I can tell, a zero TTL (number after "RFC") can only
happen here if your default_ttl parameter is set to zero, OR
if there is clock-skew between the varnish machine and the
backend machine.

Make sure both machines run NTP.

You can test that they agree by running
	ntpdate -d $backend
on the varnish machine (or vice versa).

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From ric at digitalmarbles.com  Thu Jan 29 09:07:36 2009
From: ric at digitalmarbles.com (Ricardo Newbery)
Date: Thu, 29 Jan 2009 01:07:36 -0800
Subject: [varnish] Re: Cacheability - changed in Varnish 2? 
In-Reply-To: <8731.1233218081@critter.freebsd.dk>
References: <8731.1233218081@critter.freebsd.dk>
Message-ID: <9E3C9108-EB3A-485C-ADC0-948860F4B29B@digitalmarbles.com>


On Jan 29, 2009, at 12:34 AM, Poul-Henning Kamp wrote:

> In message <4980F7D8.8090405 at giraffen.dk>, Anton Stonor writes:
>
>> New try. First, a request with no expire or cache-control header.
>
>>   10 RxProtocol   b HTTP/1.1
>>   10 RxStatus     b 200
>>   10 RxResponse   b OK
>>   10 RxHeader     b Server: Zope/(Zope 2.10.6-final, python 2.4.5,
>> linux2) ZServer/1.1 Plone/3.1.5.1
>>   10 RxHeader     b Date: Thu, 29 Jan 2009 00:10:40 GMT
>>   10 RxHeader     b Content-Length: 4
>>   10 RxHeader     b Content-Type: text/plain; charset=utf-8
>>    9 ObjProtocol  c HTTP/1.1
>>    9 ObjStatus    c 200
>>    9 ObjResponse  c OK
>>    9 ObjHeader    c Server: Zope/(Zope 2.10.6-final, python 2.4.5,
>> linux2) ZServer/1.1 Plone/3.1.5.1
>>    9 ObjHeader    c Date: Thu, 29 Jan 2009 00:10:40 GMT
>>    9 ObjHeader    c Content-Type: text/plain; charset=utf-8
>>   10 BackendReuse b backend_0
>>    9 TTL          c 1495399095 RFC 0 1233187840 0 0 0 0
>
>
> As far as I can tell, a zero TTL (number after "RFC") can only
> happen here if your default_ttl parameter is set to zero, OR
> if there is clock-skew between the varnish machine and the
> backend machine.
>
> Make sure both machines run NTP.
>
> You can test that they agree by running
> 	ntpdate -d $backend
> on the varnish machine (or vice versa).


But would this matter since he is resetting the obj.ttl to 1 day in  
vcl_fetch?

Ric




From phk at phk.freebsd.dk  Thu Jan 29 09:37:30 2009
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Thu, 29 Jan 2009 09:37:30 +0000
Subject: [varnish] Re: Cacheability - changed in Varnish 2? 
In-Reply-To: Your message of "Thu, 29 Jan 2009 01:07:36 PST."
	<9E3C9108-EB3A-485C-ADC0-948860F4B29B@digitalmarbles.com> 
Message-ID: <25392.1233221850@critter.freebsd.dk>

In message <9E3C9108-EB3A-485C-ADC0-948860F4B29B at digitalmarbles.com>, Ricardo N
ewbery writes:

>>>    9 TTL          c 1495399095 RFC 0 1233187840 0 0 0 0
>
>But would this matter since he is resetting the obj.ttl to 1 day in  
>vcl_fetch?

Yes, the RFC2616 based TTL is calculated before vcl_fetch is run.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From stonorn at giraffen.dk  Thu Jan 29 09:56:44 2009
From: stonorn at giraffen.dk (Anton Stonor)
Date: Thu, 29 Jan 2009 10:56:44 +0100
Subject: Cacheability - changed in Varnish 2?
In-Reply-To: <8731.1233218081@critter.freebsd.dk>
References: <8731.1233218081@critter.freebsd.dk>
Message-ID: <49817D5C.10807@giraffen.dk>

Poul-Henning Kamp wrote:

> As far as I can tell, a zero TTL (number after "RFC") can only
> happen here if your default_ttl parameter is set to zero, OR
> if there is clock-skew between the varnish machine and the
> backend machine.

Right on!

Backend is running on the same box as Varnish for this test. But: you 
are right about default TTL being 0.

After setting default TTL to 60, everything runs smooth. So this issue 
was not about Varnish versions, but different default TTL configs.

Thanks alot.

/Anton


From phk at phk.freebsd.dk  Thu Jan 29 09:57:48 2009
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Thu, 29 Jan 2009 09:57:48 +0000
Subject: Cacheability - changed in Varnish 2? 
In-Reply-To: Your message of "Thu, 29 Jan 2009 10:56:44 +0100."
	<49817D5C.10807@giraffen.dk> 
Message-ID: <25456.1233223068@critter.freebsd.dk>

In message <49817D5C.10807 at giraffen.dk>, Anton Stonor writes:
>Poul-Henning Kamp wrote:
>
>> As far as I can tell, a zero TTL (number after "RFC") can only
>> happen here if your default_ttl parameter is set to zero, OR
>> if there is clock-skew between the varnish machine and the
>> backend machine.
>
>Right on!
>
>Backend is running on the same box as Varnish for this test. But: you 
>are right about default TTL being 0.

I hope we don't ship varnish that way on any platforms, the default
ttl should be 120 seconds...

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From stonorn at giraffen.dk  Thu Jan 29 10:26:56 2009
From: stonorn at giraffen.dk (Anton Stonor)
Date: Thu, 29 Jan 2009 11:26:56 +0100
Subject: Cacheability - changed in Varnish 2?
In-Reply-To: <25456.1233223068@critter.freebsd.dk>
References: <25456.1233223068@critter.freebsd.dk>
Message-ID: <49818470.10606@giraffen.dk>

Poul-Henning Kamp wrote:

> I hope we don't ship varnish that way on any platforms, the default
> ttl should be 120 seconds...

No, the "0" default ttl comes from this one:

http://pypi.python.org/pypi/plone.recipe.varnish

 From the changelog: "Add a default_ttl of zero seconds to the Varnish 
runner to avoid a Varnish bug with the handling of an Expires header 
with a date in the past."

/Anton


From ric at digitalmarbles.com  Thu Jan 29 10:37:30 2009
From: ric at digitalmarbles.com (Ricardo Newbery)
Date: Thu, 29 Jan 2009 02:37:30 -0800
Subject: [varnish] Re: Cacheability - changed in Varnish 2? 
In-Reply-To: <25456.1233223068@critter.freebsd.dk>
References: <25456.1233223068@critter.freebsd.dk>
Message-ID: <6526687E-A945-4BD3-B496-6FF474C8513A@digitalmarbles.com>


On Jan 29, 2009, at 1:57 AM, Poul-Henning Kamp wrote:

> In message <49817D5C.10807 at giraffen.dk>, Anton Stonor writes:
>> Poul-Henning Kamp wrote:
>>
>>> As far as I can tell, a zero TTL (number after "RFC") can only
>>> happen here if your default_ttl parameter is set to zero, OR
>>> if there is clock-skew between the varnish machine and the
>>> backend machine.
>>
>> Right on!
>>
>> Backend is running on the same box as Varnish for this test. But: you
>> are right about default TTL being 0.
>
> I hope we don't ship varnish that way on any platforms, the default
> ttl should be 120 seconds...


I'm guessing he was using the wrapper provided by plone.recipe.varnish  
which sets a zero default ttl upon startup.  It does this to route  
around the default Varnish behavior when seeing an Expires date in the  
past.  This is discussed in more detail in this thread...

http://projects.linpro.no/pipermail/varnish-misc/2008-April/thread.html#1689

But again, I'm unclear why the obj.ttl he set in the vcl doesn't just  
override the default.

Ric




From phk at phk.freebsd.dk  Thu Jan 29 10:47:34 2009
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Thu, 29 Jan 2009 10:47:34 +0000
Subject: Cacheability - changed in Varnish 2? 
In-Reply-To: Your message of "Thu, 29 Jan 2009 11:26:56 +0100."
	<49818470.10606@giraffen.dk> 
Message-ID: <25933.1233226054@critter.freebsd.dk>

In message <49818470.10606 at giraffen.dk>, Anton Stonor writes:
>Poul-Henning Kamp wrote:
>
>> I hope we don't ship varnish that way on any platforms, the default
>> ttl should be 120 seconds...
>
>No, the "0" default ttl comes from this one:
>
>http://pypi.python.org/pypi/plone.recipe.varnish
>
> From the changelog: "Add a default_ttl of zero seconds to the Varnish 
>runner to avoid a Varnish bug with the handling of an Expires header 
>with a date in the past."

That's been fixed, but I can't remember if it made it back to 2.0.x

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From ric at digitalmarbles.com  Thu Jan 29 10:55:19 2009
From: ric at digitalmarbles.com (Ricardo Newbery)
Date: Thu, 29 Jan 2009 02:55:19 -0800
Subject: [varnish] Re: Cacheability - changed in Varnish 2? 
In-Reply-To: <25933.1233226054@critter.freebsd.dk>
References: <25933.1233226054@critter.freebsd.dk>
Message-ID: <CA6B52FB-6866-4200-96CC-09CF2CD1DFD5@digitalmarbles.com>


On Jan 29, 2009, at 2:47 AM, Poul-Henning Kamp wrote:

> In message <49818470.10606 at giraffen.dk>, Anton Stonor writes:
>> Poul-Henning Kamp wrote:
>>
>>> I hope we don't ship varnish that way on any platforms, the default
>>> ttl should be 120 seconds...
>>
>> No, the "0" default ttl comes from this one:
>>
>> http://pypi.python.org/pypi/plone.recipe.varnish
>>
>> From the changelog: "Add a default_ttl of zero seconds to the Varnish
>> runner to avoid a Varnish bug with the handling of an Expires header
>> with a date in the past."
>
> That's been fixed, but I can't remember if it made it back to 2.0.x


Ahh, cool.  Fixed starting with which version?

Ric




From phk at phk.freebsd.dk  Thu Jan 29 11:09:00 2009
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Thu, 29 Jan 2009 11:09:00 +0000
Subject: [varnish] Re: Cacheability - changed in Varnish 2? 
In-Reply-To: Your message of "Thu, 29 Jan 2009 02:55:19 PST."
	<CA6B52FB-6866-4200-96CC-09CF2CD1DFD5@digitalmarbles.com> 
Message-ID: <26036.1233227340@critter.freebsd.dk>

In message <CA6B52FB-6866-4200-96CC-09CF2CD1DFD5 at digitalmarbles.com>, Ricardo N
ewbery writes:

>>> From the changelog: "Add a default_ttl of zero seconds to the Varnish
>>> runner to avoid a Varnish bug with the handling of an Expires header
>>> with a date in the past."
>>
>> That's been fixed, but I can't remember if it made it back to 2.0.x
>
>Ahh, cool.  Fixed starting with which version?

http://varnish.projects.linpro.no/changeset/3019

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.


From tinwong at gmail.com  Mon Jan 19 13:58:30 2009
From: tinwong at gmail.com (Tin Wong)
Date: Mon, 19 Jan 2009 21:58:30 +0800
Subject: Backend connections failures
Message-ID: <68c741b0901190558m1fe3c5b2t7d43afa716e6b139@mail.gmail.com>

Hi list


My varnish and backend server on different site , i guess sometime varnish
to my backend server network connections hiccough

have any way to setup varnish to retry when backend connections failures?
can i modify varnish to backend server connect timeout ?   .connect_timeout
= 1s; ?



       92184         7.00        10.78 Client connections accepted
      415513       124.98        48.60 Client requests received
      358635       117.98        41.95 Cache hits
       32184         3.00         3.76 Cache hits for pass
       20622         3.00         2.41 Cache misses
       56668         7.00         6.63 Backend connections success
           0         0.00         0.00 Backend connections not attempted
           0         0.00         0.00 Backend connections too many
         194         0.00         0.02 Backend connections failures
       11893         1.00         1.39 Backend connections reuses
       53314         5.00         6.24 Backend connections recycles
           0         0.00         0.00 Backend connections unused


box setup , varnish 2.0.2 / centos 5.2/64


Thanks all

Tin W
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-misc/attachments/20090119/b17932c1/attachment.html>

From tfheen at linpro.no  Fri Jan 30 09:14:55 2009
From: tfheen at linpro.no (Tollef Fog Heen)
Date: Fri, 30 Jan 2009 10:14:55 +0100
Subject: Backend connections failures
In-Reply-To: <113d871c0901190625k63264e30r17aed2859875b43a@mail.gmail.com>
	(M. L.'s message of "Mon, 19 Jan 2009 22:25:01 +0800")
References: <113d871c0901190625k63264e30r17aed2859875b43a@mail.gmail.com>
Message-ID: <87ocxpi29c.fsf@qurzaw.linpro.no>

]] "M L" 

| have any way to setup varnish to retry when backend connections failures?
| can i modify varnish to backend server connect timeout ?   .connect_timeout
| = 1s; ?

Not at the moment, no.  There's some interest in getting ?restart? from
vcl_error working which would allow you to do this, but I don't think
anybody has actually done the work necessary.

-- 
Tollef Fog Heen 
Redpill Linpro -- Changing the game!
t: +47 21 54 41 73


From tfheen at linpro.no  Fri Jan 30 09:58:07 2009
From: tfheen at linpro.no (Tollef Fog Heen)
Date: Fri, 30 Jan 2009 10:58:07 +0100
Subject: Varnish doesnt start ...
In-Reply-To: <497DB856.9000002@rebert.name> (lists@rebert.name's message of
	"Mon, 26 Jan 2009 14:19:18 +0100")
References: <497DB856.9000002@rebert.name>
Message-ID: <87k58di09c.fsf@qurzaw.linpro.no>

]] 

| Hi,
| I have installed varnish with apt-get on a Debian i386 lenny ...
| 
| Does anyone know from where comes the problem ?
| 
| e003-xx:/home/etudiant# varnishd -f /etc/varnish/vcl.conf  -a 
| 0.0.0.0:80  -T 127.0.0.1:6969 -t 600 -w 1,4000,5 -h classic,500009  -s 
| malloc,500M  -s file,/home/etudiant/VARNISH_CACHE/,1G
| /usr/bin/ld: crti.o: No such file: No such file or directory
| collect2: ld returned 1 exit status
| pclose=256
| Internal error: GCC returned 0x0100

Make sure to install build-essential, not just gcc.

-- 
Tollef Fog Heen 
Redpill Linpro -- Changing the game!
t: +47 21 54 41 73


From tfheen at linpro.no  Fri Jan 30 11:22:53 2009
From: tfheen at linpro.no (Tollef Fog Heen)
Date: Fri, 30 Jan 2009 12:22:53 +0100
Subject: worker threads limited
In-Reply-To: <25daf17a0901300234w7aa340e5yead7951bd41e9cfc@mail.gmail.com>
	(Abhishek Singh's message of "Fri, 30 Jan 2009 16:04:15 +0530")
References: <25daf17a0901300234w7aa340e5yead7951bd41e9cfc@mail.gmail.com>
Message-ID: <87fxj1hwc2.fsf@qurzaw.linpro.no>

]] Abhishek Singh 

| can any please tell me what is "worker threads limited" , is it number of
| max connection for varnish. I am confused with this parameter because i have
| two servers with same OS and HW but when i am starting varnish on both the
| server then on one server i am getting vale  0 for "worker threads limited"
| and on another server this value is 756 while i am starting varnishd with
| following command.........

You are running into the maximum number of threads allowed.  As long as
you don't have dropped connections too, this should not be a problem.

-- 
Tollef Fog Heen 
Redpill Linpro -- Changing the game!
t: +47 21 54 41 73