From phk at phk.freebsd.dk Sun Dec 3 23:58:36 2006 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Sun, 03 Dec 2006 23:58:36 +0000 Subject: Varnish: the outlook and X-mas and all that. Message-ID: <20345.1165190316@critter.freebsd.dk> I thought I'd give you guys a status. I've been giving my Varnish talk five times in the last couple of weeks from Milan to Stockholm and response is good. Next time is at the UKUUG spring conference in Manchester. Version 2 seems to be a go, funding seems to be in place and development will start early january (after the hangovers subside). The featurelist is not yet nailed down, so if you have any last second wishes, now would be a good time to mail them to me. I am also looking for intelligent suggestions to how to handle cookies and authentication in Varnish, if you have experience or ideas in this space, mail me your thoughts. I have been looking at the bug reports and I will tackle them asap, but have been a bit hampered by travel, various other activities and a persistent and potent murphy field in my lab which has so far claimed one full machine and one of the CPUs in another. Finally a number of private emails have asked me what you guys can do for varnish and a few have asked about my X-mas wishes as well. Without going as far as Tobi ?tiker has gone with autoconf for RRD and MRTG, I do in fact have an amazon wishlist which you guys are more than welcome to peruse: http://www.amazon.com/exec/obidos/wishlist/1XSZNDWXKIAS0 Merry X-mas Poul-Henning -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From ff at partyticket.net Mon Dec 4 22:19:13 2006 From: ff at partyticket.net (Flemming Frandsen) Date: Mon, 04 Dec 2006 23:19:13 +0100 Subject: Feature request: Babysitter of slow clients Message-ID: <45749EE1.3070006@partyticket.net> I have a web application that doesn't generate any cachable content, but varnish could help in one respect; The apache processes use gobs of memory, so maxclients has to be low, but once a page has been generated they have to wait around for the slow clients to fetch all of the content before they can get back to producing new content. Varnish could help in this situation by quickly slurping up the content from apache, so it can return to work and passing the content to the client at the clients pace. -- Regards Flemming Frandsen - http://dion.swamp.dk - YAPH From ask at develooper.com Tue Dec 5 02:03:24 2006 From: ask at develooper.com (=?ISO-8859-1?Q?Ask_Bj=F8rn_Hansen?=) Date: Mon, 4 Dec 2006 18:03:24 -0800 Subject: Feature request: Babysitter of slow clients In-Reply-To: <45749EE1.3070006@partyticket.net> References: <45749EE1.3070006@partyticket.net> Message-ID: <46F73D5B-A56F-4A33-90C0-9E33852DDF00@develooper.com> On Dec 4, 2006, at 14:19, Flemming Frandsen wrote: > Varnish could help in this situation by quickly slurping up the > content from apache, so it can return to work and passing the > content to the client at the clients pace. Doesn't it already? (I haven't tried Varnish yet, but that would be one of the Must Have features). I am at the Web Builder conference and I'm just told a bunch of people in my talk to use Varnish for this in 2007. :^) (The slides are at http://develooper.com/talks/ ) If you don't need caching, then perlbal has some of the same properties as Varnish and it does do the "slow feeding to clients" thing. (And Keep-Alive connections in both directions). http://danga.com/perlbal/ - ask -- http://develooper.com/ - http://askask.com/ From phk at phk.freebsd.dk Tue Dec 5 07:06:27 2006 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Tue, 05 Dec 2006 07:06:27 +0000 Subject: Feature request: Babysitter of slow clients In-Reply-To: Your message of "Mon, 04 Dec 2006 18:03:24 PST." <46F73D5B-A56F-4A33-90C0-9E33852DDF00@develooper.com> Message-ID: <27151.1165302387@critter.freebsd.dk> In message <46F73D5B-A56F-4A33-90C0-9E33852DDF00 at develooper.com>, =?ISO-8859-1? Q?Ask_Bj=F8rn_Hansen?= writes: > >On Dec 4, 2006, at 14:19, Flemming Frandsen wrote: > >> Varnish could help in this situation by quickly slurping up the >> content from apache, so it can return to work and passing the >> content to the client at the clients pace. > >Doesn't it already? Only when it caches, in pass mode it doesn't (yet). -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From dwetzel at nerim.net Tue Dec 5 11:56:18 2006 From: dwetzel at nerim.net (Damien Wetzel) Date: Tue, 5 Dec 2006 12:56:18 +0100 Subject: Varnish: the outlook and X-mas and all that. In-Reply-To: <20345.1165190316@critter.freebsd.dk> References: <20345.1165190316@critter.freebsd.dk> Message-ID: <17781.24162.594558.914339@dwetzel@nerim.net> Poul-Henning Kamp writes: Hello, > > I am also looking for intelligent suggestions to how to handle > cookies and authentication in Varnish, if you have experience or > ideas in this space, mail me your thoughts. Having worked for multiple CDN providers here are some ideas about how to deal with cookies and authentication : The URLs that matches Cookies or Authentication rules are staticaly defined (regexp) in Varnish or could be dynamically advertized by the origin server via a specific HTTP header. 1) COOKIE check presence mode: check if the named cookie is present on the requests from the end user, and make a request to the origin if the cookie is missing. All headers provided with the request are passed on the origin. Any Set-Cookie received is then added to the headers of the response varnish is doing to the enduser. 2) Authentication mode If an URL matches this mode, then before serving the cached ressource, a GET or HEAD request is done to the origin, passing the headers of the request. if response is : 200: Varnish delivers the cached object 302: Varnish returns it to the end user who attempts to retrieve it 401: varnish returns it to the end user. Damien, > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > phk at FreeBSD.ORG | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetence. > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc From stonorn at giraffen.dk Wed Dec 6 20:29:48 2006 From: stonorn at giraffen.dk (Anton Stonor) Date: Wed, 06 Dec 2006 21:29:48 +0100 Subject: Varnish: the outlook and X-mas and all that. In-Reply-To: <20345.1165190316@critter.freebsd.dk> References: <20345.1165190316@critter.freebsd.dk> Message-ID: <4577283C.4070604@giraffen.dk> Poul-Henning Kamp wrote: > Version 2 seems to be a go, funding seems to be in place and > development will start early january (after the hangovers subside). This is excellent news! I was wondering what would happen after your attempt to start a discussion of how to proceed with Varnish. What model did you end up with? > I do in fact have an amazon wishlist which you guys > are more than welcome to peruse: > > http://www.amazon.com/exec/obidos/wishlist/1XSZNDWXKIAS0 Lots of books about the atom bomb. I hope you don't have a hobby that we will hear alot about some day :-) /Anton From phk at phk.freebsd.dk Wed Dec 6 20:39:28 2006 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Wed, 06 Dec 2006 20:39:28 +0000 Subject: Varnish: the outlook and X-mas and all that. In-Reply-To: Your message of "Wed, 06 Dec 2006 21:29:48 +0100." <4577283C.4070604@giraffen.dk> Message-ID: <82159.1165437568@critter.freebsd.dk> In message <4577283C.4070604 at giraffen.dk>, Anton Stonor writes: >> http://www.amazon.com/exec/obidos/wishlist/1XSZNDWXKIAS0 > >Lots of books about the atom bomb. I hope you don't have a hobby that we >will hear alot about some day :-) Don't worry, it's a purely theoretical hobby of mine :-) I tend to notice missing stuff, things which are not said and things which are not talked about where they would naturally be. The cold war and atomic bomb history is littered with such 'holes'. Maybe some day I will write a book about all those holes. The public of a lot of democratic countries have been seriously mislead over the years. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From red5 at klaus.dk Tue Dec 12 18:06:38 2006 From: red5 at klaus.dk (Klaus) Date: Tue, 12 Dec 2006 19:06:38 +0100 Subject: Serve old content if backed is slow. Message-ID: <457EEFAE.9050605@klaus.dk> Hello! Is varnish able to serve old content if backend is slow ? I fact, the ideal way would be if content has expired, it would still serve the old content until new content is retrieved from backend. Regards, Klaus From phk at phk.freebsd.dk Wed Dec 13 10:21:57 2006 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Wed, 13 Dec 2006 10:21:57 +0000 Subject: Serve old content if backed is slow. In-Reply-To: Your message of "Tue, 12 Dec 2006 19:06:38 +0100." <457EEFAE.9050605@klaus.dk> Message-ID: <12139.1166005317@critter.freebsd.dk> In message <457EEFAE.9050605 at klaus.dk>, Klaus writes: >Hello! >Is varnish able to serve old content if backend is slow ? > >I fact, the ideal way would be if content has expired, it would still >serve the old content until new content is retrieved from backend. The "prefetch" feature we are planning will allow you to do something like that. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From stonorn at giraffen.dk Wed Dec 13 10:23:42 2006 From: stonorn at giraffen.dk (Anton Stonor) Date: Wed, 13 Dec 2006 11:23:42 +0100 Subject: Serve old content if backed is slow. In-Reply-To: <457EEFAE.9050605@klaus.dk> References: <457EEFAE.9050605@klaus.dk> Message-ID: <457FD4AE.1030207@giraffen.dk> Klaus wrote: > I fact, the ideal way would be if content has expired, it would still > serve the old content until new content is retrieved from backend. I believe "pre fetching" is on the drawing board and hopefully scheduled for version 2. It will probably make it possible to set up rules such as: Fetch a fresh copy from the backend 30 seconds before the object times out. /Anton From dwetzel at nerim.net Wed Dec 13 10:39:47 2006 From: dwetzel at nerim.net (Damien Wetzel) Date: Wed, 13 Dec 2006 11:39:47 +0100 Subject: Serve old content if backed is slow. In-Reply-To: <457FD4AE.1030207@giraffen.dk> References: <457EEFAE.9050605@klaus.dk> <457FD4AE.1030207@giraffen.dk> Message-ID: <17791.55411.360816.575367@dwetzel@nerim.net> Hello, Something that could be usefull in a subsequent release would be to return in case of unavailibily of the backend : 1) a stale copy for any non html object or a null sized object if that object is not in cache. 2) a stale copy of an html url or a user defined static html page if that html page is not in cache. Damien, Anton Stonor writes: > Klaus wrote: > > > I fact, the ideal way would be if content has expired, it would still > > serve the old content until new content is retrieved from backend. > > I believe "pre fetching" is on the drawing board and hopefully scheduled > for version 2. > > It will probably make it possible to set up rules such as: Fetch a fresh > copy from the backend 30 seconds before the object times out. > > /Anton > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc From phk at phk.freebsd.dk Wed Dec 13 10:41:49 2006 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Wed, 13 Dec 2006 10:41:49 +0000 Subject: Serve old content if backed is slow. In-Reply-To: Your message of "Wed, 13 Dec 2006 11:39:47 +0100." <17791.55411.360816.575367@dwetzel@nerim.net> Message-ID: <12387.1166006509@critter.freebsd.dk> In message <17791.55411.360816.575367 at dwetzel@nerim.net>, Damien Wetzel writes: >Hello, >Something that could be usefull in a subsequent release would be >to return in case of unavailibily of the backend : >1) a stale copy for any non html object or a null sized object if that object > is not in cache. >2) a stale copy of an html url or a user defined static html page > if that html page is not in cache. The plan is that you will have full control over what you do in the VCL code, including the case where the backend does not reply. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence.