From thomas.woinke at holtzbrinck.com Wed Dec 1 13:07:41 2010 From: thomas.woinke at holtzbrinck.com (Thomas Woinke) Date: Wed, 01 Dec 2010 14:07:41 +0100 Subject: backend max_connections Message-ID: <4CF6489D.9030306@holtzbrinck.com> Hi all, documentation is a bit terse about the max_connection parameter for backend declarations. I can't seem to find anything about what exactly is going to happen at the frontend side when concurrent backend connections are maxed out. My guess is that the client will either receive a 503 error or a stale object will be served, depending on the grace time. Am I right or wrong? Regards, Thomas From cosimo at streppone.it Wed Dec 1 13:57:51 2010 From: cosimo at streppone.it (Cosimo Streppone) Date: Wed, 01 Dec 2010 14:57:51 +0100 Subject: Maintenance mode, or multiple backends but just one used Message-ID: Hi, This is varnish 2.0.4. I have a VCL where I define a backend like: backend nginx { .host = "127.0.0.1"; .port = "8080"; } and then later in vcl_recv: set req.backend = nginx; Then I modified the VCL to add another backend, to test it as downtime/maintenance service, so that all requests coming in will just display a "we're upgrading/we're offline" static page: backend oops { .host = "1.2.3.4"; .port = "80"; } At the same time I modified req.backend to be "oops". Now, I'm using only one backend. If the backend that gets to be used in req.backend is the last defined one, then everything's fine. If the used backend is any other than the last one, I get this VCC error: Message from VCC-compiler: Unused backend oopsy, defined: (input Line 24 Pos 9) backend oops { --------#####-- Running VCC-compiler failed, exit 1 VCL compilation failed This came as a surprise... Is there a way around it? I'll probably setup a different "maintenance.vcl" and alternate the VCL configs between "main.vcl" and "maintenance.vcl" when I need to. Any better alternatives? -- Cosimo From michael.bu at gmail.com Wed Dec 1 19:16:28 2010 From: michael.bu at gmail.com (Michael Bu) Date: Thu, 2 Dec 2010 03:16:28 +0800 Subject: client.identity and client director -- How? Message-ID: Hi All, I want to know how to setup a client director and make use of client.identity to force the request always going to the same backend instance (ie. sticky). Here's my setup: backend a { .host = "192.168.1.4"; .port = "8880"; } backend b { .host = "192.168.1.5"; .port = "8880"; } director tomcat client { { .backend = a; .weight = 1; } { .backend = b; .weight = 1; } } sub vcl_recv { ## set sticky bit for backend director ## set client.identity = regsub(req.http.Cookie, ".*JSESSIONID=.*\.(\w).*", "\1"); set req.backend = tomcat; } Unfortunately the above setup didn't work as expected. Could some one please give me some advice? Thanks in advance! Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: From scaunter at topscms.com Wed Dec 1 19:23:24 2010 From: scaunter at topscms.com (Caunter, Stefan) Date: Wed, 1 Dec 2010 14:23:24 -0500 Subject: client.identity and client director -- How? In-Reply-To: References: Message-ID: <7F0AA702B8A85A4A967C4C8EBAD6902C9B6313@TMG-EVS02.torstar.net> Use an if in vcl_recv. If you need to stick based on cookie, use that header; I use X-F-F for a load-balancer setup. sub vcl_recv { # set client.identity based on x-f-f header, use that to stick to a backend server if (req.http.x-forwarded-for) { set client.identity = req.http.x-forwarded-for; set req.backend = tomcat; } # more stuff goes here return(lookup); } Stefan Caunter :: Senior Systems Administrator :: TOPS e: scaunter at topscms.com :: m: (416) 561-4871 www.thestar.com www.topscms.com From: varnish-misc-bounces at varnish-cache.org [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Michael Bu Sent: December-01-10 2:16 PM To: varnish-misc at varnish-cache.org Subject: client.identity and client director -- How? Hi All, I want to know how to setup a client director and make use of client.identity to force the request always going to the same backend instance (ie. sticky). Here's my setup: backend a { .host = "192.168.1.4"; .port = "8880"; } backend b { .host = "192.168.1.5"; .port = "8880"; } director tomcat client { { .backend = a; .weight = 1; } { .backend = b; .weight = 1; } } sub vcl_recv { ## set sticky bit for backend director ## set client.identity = regsub(req.http.Cookie, ".*JSESSIONID=.*\.(\w).*", "\1"); set req.backend = tomcat; } Unfortunately the above setup didn't work as expected. Could some one please give me some advice? Thanks in advance! Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.bu at gmail.com Wed Dec 1 19:48:04 2010 From: michael.bu at gmail.com (Michael Bu) Date: Thu, 2 Dec 2010 03:48:04 +0800 Subject: client.identity and client director -- How? In-Reply-To: <7F0AA702B8A85A4A967C4C8EBAD6902C9B6313@TMG-EVS02.torstar.net> References: <7F0AA702B8A85A4A967C4C8EBAD6902C9B6313@TMG-EVS02.torstar.net> Message-ID: Thanks for your reply. I could of course hard-coded the intended behavior with an "if" clause. But it would be nice to take advantage of the new "Client Director" feature which unfortunately wasn't documented... Thanks, Michael On Thu, Dec 2, 2010 at 3:23 AM, Caunter, Stefan wrote: > Use an if in vcl_recv. If you need to stick based on cookie, use that > header; I use X-F-F for a load-balancer setup. > > > > sub vcl_recv { > > # set client.identity based on x-f-f header, use that to stick to a backend > server > > > > if (req.http.x-forwarded-for) { > > set client.identity = req.http.x-forwarded-for; > > set req.backend = tomcat; > > } > > # more stuff goes here > > return(lookup); > > } > > > > Stefan Caunter :: Senior Systems Administrator :: TOPS > > e: scaunter at topscms.com :: m: (416) 561-4871 > > www.thestar.com www.topscms.com > > > > *From:* varnish-misc-bounces at varnish-cache.org [mailto: > varnish-misc-bounces at varnish-cache.org] *On Behalf Of *Michael Bu > *Sent:* December-01-10 2:16 PM > *To:* varnish-misc at varnish-cache.org > *Subject:* client.identity and client director -- How? > > > > Hi All, > > > I want to know how to setup a client director and make use of > client.identity to force the request always going to the same backend > instance (ie. sticky). > > Here's my setup: > > backend a { > .host = "192.168.1.4"; > .port = "8880"; > } > backend b { > .host = "192.168.1.5"; > .port = "8880"; > } > director tomcat client { > { > .backend = a; > .weight = 1; > } > { > .backend = b; > .weight = 1; > } > } > sub vcl_recv { > ## set sticky bit for backend director ## > set client.identity = regsub(req.http.Cookie, > ".*JSESSIONID=.*\.(\w).*", "\1"); > set req.backend = tomcat; > } > > Unfortunately the above setup didn't work as expected. > > Could some one please give me some advice? > > Thanks in advance! > Michael > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vburshteyn at broadway.com Thu Dec 2 03:06:51 2010 From: vburshteyn at broadway.com (Vitaly Burshteyn) Date: Wed, 1 Dec 2010 22:06:51 -0500 Subject: normalicing accept headers Message-ID: <0F69574F9901D4459C6B75C9FF64FBC405284487@mxfl01.hollywoodmedia.corp> Hello, So I am still trying to figure out why I cant hit a cached page with a different browser other then the one it was cached with. I am pretty sure i got my vary statements right, they show up similar in the logs, so I know I am missing something. Can there be something other then Vary tags that can be causing this? Thank you. Vitaly Burshteyn Senior Network Engineer Broadway.com, Theatre Direct International 729 7th Avenue New York, New York 10019 Phone: 212.817.9117 Cell# 917-701-5732 ____________________________________ The information contained in this transmission may contain privileged and confidential information. It is intended only for the use of the person(s) named above. If you are not the intended recipient, you are hereby notified that any review, dissemination, distribution or duplication of this communication is strictly prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. -------------- next part -------------- An HTML attachment was scrubbed... URL: From stig at zedge.net Thu Dec 2 06:23:51 2010 From: stig at zedge.net (Stig Bakken) Date: Thu, 2 Dec 2010 07:23:51 +0100 Subject: normalicing accept headers In-Reply-To: <0F69574F9901D4459C6B75C9FF64FBC405284487@mxfl01.hollywoodmedia.corp> References: <0F69574F9901D4459C6B75C9FF64FBC405284487@mxfl01.hollywoodmedia.corp> Message-ID: Hi Vitaly, It would probably be easier to help if you posted a log dump of the same url in two browsers. - Stig On Thu, Dec 2, 2010 at 4:06 AM, Vitaly Burshteyn wrote: > Hello, > > > > So I am still trying to figure out why I cant hit a cached page with a > different browser other then the one it was cached with. I am pretty sure i > got my vary statements right, they show up similar in the logs, so I know I > am missing something. > > > > Can there be something other then Vary tags that can be causing this? > > > > Thank you. > > > > Vitaly Burshteyn > > Senior Network Engineer > > Broadway.com, Theatre Direct International > > 729 7th Avenue > > New York, New York 10019 > > Phone: 212.817.9117 > > Cell# 917-701-5732 > > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bedis9 at gmail.com Thu Dec 2 09:05:23 2010 From: bedis9 at gmail.com (Bedis 9) Date: Thu, 2 Dec 2010 10:05:23 +0100 Subject: Maintenance mode, or multiple backends but just one used In-Reply-To: References: Message-ID: Hi, First of all, your backend is labelled "oops" while in your error it says "Unused backend oopsy". (there is a Y at the end...), maybe a typo? Second, you can use "restarts" and "backend.healthy" properties of a request to detect a failure and switch to your "oops" backend. cheers On Wed, Dec 1, 2010 at 2:57 PM, Cosimo Streppone wrote: > Hi, > > This is varnish 2.0.4. I have a VCL where I define a backend like: > > ?backend nginx { > ? ?.host = "127.0.0.1"; > ? ?.port = "8080"; > ?} > > and then later in vcl_recv: > > ?set req.backend = nginx; > > Then I modified the VCL to add another backend, > to test it as downtime/maintenance service, so > that all requests coming in will just display a > "we're upgrading/we're offline" static page: > > ?backend oops { > ? ? .host = "1.2.3.4"; > ? ? .port = "80"; > ?} > > At the same time I modified req.backend to be "oops". > Now, I'm using only one backend. > If the backend that gets to be used in req.backend is > the last defined one, then everything's fine. > > If the used backend is any other than the last one, > I get this VCC error: > > ?Message from VCC-compiler: > ?Unused backend oopsy, defined: > ?(input Line 24 Pos 9) > ?backend oops { > ?--------#####-- > ?Running VCC-compiler failed, exit 1 > ?VCL compilation failed > > This came as a surprise... Is there a way around it? > > I'll probably setup a different "maintenance.vcl" > and alternate the VCL configs between "main.vcl" and "maintenance.vcl" > when I need to. > > Any better alternatives? > > -- > Cosimo > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > From cosimo at streppone.it Thu Dec 2 09:13:52 2010 From: cosimo at streppone.it (Cosimo Streppone) Date: Thu, 02 Dec 2010 10:13:52 +0100 Subject: Maintenance mode, or multiple backends but just one used In-Reply-To: References: Message-ID: On Thu, 02 Dec 2010 10:05:23 +0100, Bedis 9 wrote: > First of all, your backend is labelled "oops" while in your error it > says "Unused backend oopsy". (there is a Y at the end...), maybe a > typo? Sorry. The name is correct. I typed the log instead of copy/pasting. > Second, you can use "restarts" and "backend.healthy" properties of a > request to detect a failure and switch to your "oops" backend. I decided to not enable health checks yet, because 2.0.4 doesn't have ".initial", and this will cause a black out period when restarting the service. Yes, we do reload, but sometimes we want a full restart, and having a 5/10/15 seconds down is not really cool. So, IIUC, what you are saying that I need to actually use all the backends I define. -- Cosimo From tfheen at varnish-software.com Thu Dec 2 09:36:18 2010 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Thu, 02 Dec 2010 10:36:18 +0100 Subject: Maintenance mode, or multiple backends but just one used In-Reply-To: (Cosimo Streppone's message of "Thu, 02 Dec 2010 10:13:52 +0100") References: Message-ID: <87mxoowm7h.fsf@qurzaw.varnish-software.com> ]] "Cosimo Streppone" | So, IIUC, what you are saying that I need to actually use | all the backends I define. Yes. This can be turned into a warning in trunk. Note that doing if (0) { set req.backend = backend1 ; set req.backend = backend2;} and so on will work fine. -- Tollef Fog Heen Varnish Software t: +47 21 98 92 64 From lin.support at gmail.com Thu Dec 2 09:59:11 2010 From: lin.support at gmail.com (linuxsupport) Date: Thu, 2 Dec 2010 09:59:11 +0000 Subject: Varnish storage Message-ID: Hi, I am new to Varnish, I like the features, currently I am testing it on development environment. We have 800+ web site so huge contents. I have 500Gb storage and 24 GB memory on each server total 3 servers - Dual Quad Core Xeon. Can I use 20 GB of memory and 500 GB of disk as cache storage? (ie. -s malloc,20G -s file,/lv2/v_cache/varnish_storage.bin Does Both memory and file storage work together? Is it good idea to have single file of 500GB for cache? or should I split it as 500/3? ( In know Varnish does not pre allocate the space). Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From david at collaborativebusiness.co.uk Thu Dec 2 10:08:53 2010 From: david at collaborativebusiness.co.uk (David Turner) Date: Thu, 2 Dec 2010 10:08:53 +0000 Subject: Different cache policy for googlebot Message-ID: <490D0D32-8A55-4965-AEEE-1568906EDE6E@collaborativebusiness.co.uk> I have been digging around the documentation and wiki to see if this has been done before, it seems not so it might just be a bad idea... I'm working on a site that has a large number of dynamic pages. Googlebot is going to town spidering everything in sight and we need to get it under control in the short-term while we address the underlying performance. The content on the pages needs to be displayed to humans with a short cache time, but for Googlebot we wouldn't mind caching much more aggressively. So my thought was to manage the cache such that if anyone other than googlebot requested a page that we process it normally with a reasonable TTL and update the cache. But if Googlebot requests a page, determined by the agent string, we try to serve the page from the cache if it's available (even if it's stale) and otherwise fetch from the backend and update as normal. Aside from this maybe being a bad idea, I'm not sure how efficiently this could be implemented with Varnish. The reason for trying to handle all this in Varnish is that we can't easily make changes to the underlying CMS to handle this. Is this a good or bad idea? And at what point in the varnish pipeline is it most efficient to handle this? -- David M Turner Collaborative Business Services Ltd From wido at widodh.nl Thu Dec 2 10:16:07 2010 From: wido at widodh.nl (Wido den Hollander) Date: Thu, 02 Dec 2010 11:16:07 +0100 Subject: Varnish storage In-Reply-To: References: Message-ID: <1291284967.2900.2.camel@wido-desktop> Hi, I would use malloc: -s malloc,200G Then create a SWAP partition of 300GB on your 500GB disk and add it as SWAP. The kernel will then take care of the rest. Wido On Thu, 2010-12-02 at 09:59 +0000, linuxsupport wrote: > Hi, > > I am new to Varnish, I like the features, currently I am testing it on > development environment. > > We have 800+ web site so huge contents. I have 500Gb storage and 24 GB > memory on each server total 3 servers - Dual Quad Core Xeon. > > Can I use 20 GB of memory and 500 GB of disk as cache storage? (ie. -s > malloc,20G -s file,/lv2/v_cache/varnish_storage.bin > Does Both memory and file storage work together? > > Is it good idea to have single file of 500GB for cache? or should I > split it as 500/3? ( In know Varnish does not pre allocate the space). > > Thanks > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc From stewsnooze at gmail.com Thu Dec 2 10:22:33 2010 From: stewsnooze at gmail.com (Stewart Robinson) Date: Thu, 2 Dec 2010 10:22:33 +0000 Subject: Different cache policy for googlebot In-Reply-To: <490D0D32-8A55-4965-AEEE-1568906EDE6E@collaborativebusiness.co.uk> References: <490D0D32-8A55-4965-AEEE-1568906EDE6E@collaborativebusiness.co.uk> Message-ID: Hi, I think you could match on the Google bot string in vcl_hash and set a different hash key and with that set long cache times e.t.c but this would be essentially splitting your cache in half for google and not google and I really don't think that is a good idea as it lowers the number of items you can store. Isn't a better temporary option to log into Google Webmaster tools and slow the crawl down. It is valid for 90 days after setting so it should give you breathing room. I assume this is a Drupal site as I've seen you at Drupal events. Could you also make a special settings.php setting that enables boost just for Googlebot so it doesn't crawl old articles again at any load? Stewart Robinson. On 2 December 2010 10:08, David Turner wrote: > I have been digging around the documentation and wiki to see if this has > been done before, it seems not so it might just be a bad idea... > > I'm working on a site that has a large number of dynamic pages. Googlebot > is going to town spidering everything in sight and we need to get it under > control in the short-term while we address the underlying performance. > > The content on the pages needs to be displayed to humans with a short cache > time, but for Googlebot we wouldn't mind caching much more aggressively. > > So my thought was to manage the cache such that if anyone other than > googlebot requested a page that we process it normally with a reasonable TTL > and update the cache. But if Googlebot requests a page, determined by the > agent string, we try to serve the page from the cache if it's available > (even if it's stale) and otherwise fetch from the backend and update as > normal. > > Aside from this maybe being a bad idea, I'm not sure how efficiently this > could be implemented with Varnish. The reason for trying to handle all this > in Varnish is that we can't easily make changes to the underlying CMS to > handle this. > > Is this a good or bad idea? And at what point in the varnish pipeline is it > most efficient to handle this? > > > > -- > David M Turner > Collaborative Business Services Ltd > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From david at collaborativebusiness.co.uk Thu Dec 2 10:29:20 2010 From: david at collaborativebusiness.co.uk (David Turner) Date: Thu, 2 Dec 2010 10:29:20 +0000 Subject: Different cache policy for googlebot In-Reply-To: References: <490D0D32-8A55-4965-AEEE-1568906EDE6E@collaborativebusiness.co.uk> Message-ID: <8A96548A-C3BE-43B9-9EA1-BE80B49D56D9@collaborativebusiness.co.uk> I agree that it's not a good idea to split the cache. The way I see it, the human traffic will need to keep the cache updated so that pages can be returned from there when Googlebot comes. (Cache misses for Googlebot would still fetch from the backend.) We have slowed Googlebot down, and maybe that's the best solution at the end of the day. My worry with that is that the site is growing quickly so by slowing Google down, it has an older view of the whole site whereas my proposal is to let it keep coming but let it hold an older version of each page. Unfortunately not a Drupal site otherwise I'd have been looking there first! ;o) On 2 Dec 2010, at 10:22, Stewart Robinson wrote: > Hi, > > I think you could match on the Google bot string in vcl_hash and set a different hash key and with that set long cache times e.t.c but this would be essentially splitting your cache in half for google and not google and I really don't think that is a good idea as it lowers the number of items you can store. > > Isn't a better temporary option to log into Google Webmaster tools and slow the crawl down. It is valid for 90 days after setting so it should give you breathing room. I assume this is a Drupal site as I've seen you at Drupal events. Could you also make a special settings.php setting that enables boost just for Googlebot so it doesn't crawl old articles again at any load? > > Stewart Robinson. -- David M Turner Collaborative Business Services Ltd From vasile at cristescu.org Thu Dec 2 10:31:15 2010 From: vasile at cristescu.org (Vasile Cristescu) Date: Thu, 2 Dec 2010 12:31:15 +0200 Subject: Varnish + FreeBSD + Cpanel Message-ID: <201012021231.16209.vasile@cristescu.org> Hello, I have a FreeBSD box (7.2-RELEASE) on which I use Cpanel (WHM 11.26.20) to manage the domains. I'm trying to install Varnish (from ports) in front of Apache (Server version: Apache/2.2.13 Server built: Oct 12 2009 12:20:31 Cpanel::Easy::Apache v3.2.0 rev4793 ). I have change the port apache listens on to *:8080, and edited default.acl as following: backend default { .host = "127.0.0.1"; .port = "8080"; } sub vcl_recv { if (req.http.x-forwarded-for) { set req.http.X-Forwarded-For = req.http.X-Forwarded-For ", " client.ip; } else { set req.http.X-Forwarded-For = client.ip; } if (req.request != "GET" && req.request != "HEAD" && req.request != "PUT" && req.request != "POST" && req.request != "TRACE" && req.request != "OPTIONS" && req.request != "DELETE") { /* Non-RFC2616 or CONNECT which is weird. */ return (pipe); } if (req.request != "GET" && req.request != "HEAD") { /* We only deal with GET and HEAD by default */ return (pass); } if (req.http.Authorization || req.http.Cookie) { /* Not cacheable by default */ return (pass); } return (lookup); } I've restarted apache, started varnishd, but when I try to access a domain I get the default cpanel apache index (welcome to cpanel .. etc). Does anyone have any idea what am I doing wrong ? -- Best Regards, Vasile Cristescu From phk at phk.freebsd.dk Thu Dec 2 10:33:12 2010 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Thu, 02 Dec 2010 10:33:12 +0000 Subject: Different cache policy for googlebot In-Reply-To: Your message of "Thu, 02 Dec 2010 10:29:20 GMT." <8A96548A-C3BE-43B9-9EA1-BE80B49D56D9@collaborativebusiness.co.uk> Message-ID: <2263.1291285992@critter.freebsd.dk> In message <8A96548A-C3BE-43B9-9EA1-BE80B49D56D9 at collaborativebusiness.co.uk>, David Turner writes: >We have slowed Googlebot down, and maybe that's the best solution >at the end of the day. My worry with that is that the site is growing >quickly so by slowing Google down, it has an older view of the whole >site whereas my proposal is to let it keep coming but let it hold >an older version of each page. You only need to slow it on cache misses ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From lin.support at gmail.com Thu Dec 2 10:44:55 2010 From: lin.support at gmail.com (linuxsupport) Date: Thu, 2 Dec 2010 10:44:55 +0000 Subject: Varnish storage In-Reply-To: <1291284967.2900.2.camel@wido-desktop> References: <1291284967.2900.2.camel@wido-desktop> Message-ID: OK, Can I make the swap partition of 500G? I also want to confirm if using 500G swap partition for cache is good? or spiting is needed? Aniruddh On Thu, Dec 2, 2010 at 10:16 AM, Wido den Hollander wrote: > Hi, > > I would use malloc: -s malloc,200G > > Then create a SWAP partition of 300GB on your 500GB disk and add it as > SWAP. > > The kernel will then take care of the rest. > > Wido > > On Thu, 2010-12-02 at 09:59 +0000, linuxsupport wrote: > > Hi, > > > > I am new to Varnish, I like the features, currently I am testing it on > > development environment. > > > > We have 800+ web site so huge contents. I have 500Gb storage and 24 GB > > memory on each server total 3 servers - Dual Quad Core Xeon. > > > > Can I use 20 GB of memory and 500 GB of disk as cache storage? (ie. -s > > malloc,20G -s file,/lv2/v_cache/varnish_storage.bin > > Does Both memory and file storage work together? > > > > Is it good idea to have single file of 500GB for cache? or should I > > split it as 500/3? ( In know Varnish does not pre allocate the space). > > > > Thanks > > _______________________________________________ > > varnish-misc mailing list > > varnish-misc at varnish-cache.org > > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From david at collaborativebusiness.co.uk Thu Dec 2 10:45:48 2010 From: david at collaborativebusiness.co.uk (David Turner) Date: Thu, 2 Dec 2010 10:45:48 +0000 Subject: Different cache policy for googlebot In-Reply-To: <2263.1291285992@critter.freebsd.dk> References: <2263.1291285992@critter.freebsd.dk> Message-ID: <0C561017-9A55-4CD9-9D66-B31AC7B0CBD6@collaborativebusiness.co.uk> On 2 Dec 2010, at 10:33, Poul-Henning Kamp wrote: > You only need to slow it on cache misses ? The other way around. I don't mind it getting old data for a cache hit, but a cache miss should be serviced as required. Humans should get be getting lightly cached pages. This is currently being handled in the application but to keep the varnish cache up to date the human requests need to be fetched instead of passed straight through. -- David M Turner Collaborative Business Services Ltd Tel: +44 161 850 0586 Collaborative Business Services Ltd is a private limited company registered in England and Wales. Registered number: 6666362 Registered office: Floor 10, 3 Hardman Street, Manchester, M3 3HF From phk at phk.freebsd.dk Thu Dec 2 11:38:40 2010 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Thu, 02 Dec 2010 11:38:40 +0000 Subject: Different cache policy for googlebot In-Reply-To: Your message of "Thu, 02 Dec 2010 10:45:48 GMT." <0C561017-9A55-4CD9-9D66-B31AC7B0CBD6@collaborativebusiness.co.uk> Message-ID: <2971.1291289920@critter.freebsd.dk> In message <0C561017-9A55-4CD9-9D66-B31AC7B0CBD6 at collaborativebusiness.co.uk>, David Turner writes: >On 2 Dec 2010, at 10:33, Poul-Henning Kamp wrote: > >> You only need to slow it on cache misses ? > >The other way around. I don't mind it getting old data for a cache >hit, but a cache miss should be serviced as required. Hmm, I'm not sure I understand, but then again, I'm not a webmaster, I'm just a coder :-) Consider giving the google bot a longer req.grace than other clients ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From wido at widodh.nl Thu Dec 2 11:44:10 2010 From: wido at widodh.nl (Wido den Hollander) Date: Thu, 02 Dec 2010 12:44:10 +0100 Subject: Varnish storage In-Reply-To: References: <1291284967.2900.2.camel@wido-desktop> Message-ID: <1291290250.2900.12.camel@wido-desktop> Hi, On Thu, 2010-12-02 at 10:44 +0000, linuxsupport wrote: > OK, Can I make the swap partition of 500G? > Yes, no problem! > I also want to confirm if using 500G swap partition for cache is good? > or spiting is needed? It sounds fine, but if a singl disk is up to the task (in terms of IOps), I'm not sure about. But here is imho no need in splitting it up in smaller partitions. Wido > > Aniruddh > > On Thu, Dec 2, 2010 at 10:16 AM, Wido den Hollander > wrote: > Hi, > > I would use malloc: -s malloc,200G > > Then create a SWAP partition of 300GB on your 500GB disk and > add it as > SWAP. > > The kernel will then take care of the rest. > > Wido > > > On Thu, 2010-12-02 at 09:59 +0000, linuxsupport wrote: > > Hi, > > > > I am new to Varnish, I like the features, currently I am > testing it on > > development environment. > > > > We have 800+ web site so huge contents. I have 500Gb storage > and 24 GB > > memory on each server total 3 servers - Dual Quad Core Xeon. > > > > Can I use 20 GB of memory and 500 GB of disk as cache > storage? (ie. -s > > malloc,20G -s file,/lv2/v_cache/varnish_storage.bin > > Does Both memory and file storage work together? > > > > Is it good idea to have single file of 500GB for cache? or > should I > > split it as 500/3? ( In know Varnish does not pre allocate > the space). > > > > Thanks > > > _______________________________________________ > > varnish-misc mailing list > > varnish-misc at varnish-cache.org > > > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > From vburshteyn at broadway.com Thu Dec 2 11:52:18 2010 From: vburshteyn at broadway.com (Vitaly Burshteyn) Date: Thu, 2 Dec 2010 06:52:18 -0500 Subject: normalicing accept headers In-Reply-To: References: <0F69574F9901D4459C6B75C9FF64FBC405284487@mxfl01.hollywoodmedia.corp> Message-ID: <0F69574F9901D4459C6B75C9FF64FBC405284488@mxfl01.hollywoodmedia.corp> Stig, Ok so here is the log output from crome: 4 TxHeader b Host: stagingvarnish.broadway.com 4 TxHeader b Referer: http://stagingvarnish.broadway.com/shows/jersey-boys/ 4 TxHeader b Accept: application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 4 TxHeader b User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.7 (KHTML, like Gecko) Chrome/7.0.517.44 Safari/534.7 4 TxHeader b Accept-Language: en-US,en;q=0.8 4 TxHeader b Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3 4 TxHeader b Accept-Encoding: gzip 4 TxHeader b Cookie: omni=bwy; LMD_18157499=0; affi=old; s_cc=true; s_sq=broadwayprod%3D%2526pid%253DShow%25253A%252520Jersey%252520Boys%25253A%252520Overview%2526pidt%253D1%2526oid%253Dhttp%25253A//stagingvarnish.broadway.com/%2526ot%253DA%26broadwayqa%3D%2526pid%253 4 TxHeader b X-Varnish: 548007645 0 Backend_health - s1 Still healthy 4--X-RH 10 5 10 0.110571 0.069738 HTTP/1.1 200 OK 0 Backend_health - s2 Still healthy 4--X-RH 10 5 10 0.054721 0.053967 HTTP/1.1 200 OK 0 CLI - Rd ping 0 CLI - Wr 200 PONG 1291290621 1.0 0 Backend_health - s1 Still healthy 4--X-RH 10 5 10 0.061056 0.067568 HTTP/1.1 200 OK 4 RxProtocol b HTTP/1.1 4 RxStatus b 200 4 RxResponse b OK 4 RxHeader b Date: Thu, 02 Dec 2010 11:50:19 GMT 4 RxHeader b Server: Apache/2.2.9 (Fedora) 4 RxHeader b Vary: User-Agent,Cookie,Accept-Encoding 4 RxHeader b Content-Encoding: gzip 4 RxHeader b Connection: close 4 RxHeader b Transfer-Encoding: chunked 4 RxHeader b Content-Type: text/html; charset=utf-8 3 TTL c 548007645 RFC 120 1291290622 0 0 0 0 3 VCL_call c fetch 3 TTL c 548007645 VCL 86400 1291290620 3 TTL c 548007645 VCL 240 1291290620 3 VCL_return c deliver 3 ObjProtocol c HTTP/1.1 3 ObjStatus c 200 3 ObjResponse c OK 3 ObjHeader c Date: Thu, 02 Dec 2010 11:50:19 GMT 3 ObjHeader c Server: Apache/2.2.9 (Fedora) 3 ObjHeader c Vary: User-Agent,Cookie,Accept-Encoding 3 ObjHeader c Content-Type: text/html; charset=utf-8 3 ObjHeader c Content-Encoding: gzip 3 ObjHeader c Cache-Control: max-age=3600 3 ObjHeader c magicmarker: 1 4 BackendClose b s1 3 Length c 12314 3 VCL_call c deliver 3 VCL_return c deliver 3 TxProtocol c HTTP/1.1 3 TxStatus c 200 3 TxResponse c OK 3 TxHeader c Server: Apache/2.2.9 (Fedora) 3 TxHeader c Vary: User-Agent,Cookie,Accept-Encoding 3 TxHeader c Content-Type: text/html; charset=utf-8 3 TxHeader c Content-Encoding: gzip 3 TxHeader c Cache-Control: max-age=3600 3 TxHeader c magicmarker: 1 3 TxHeader c Content-Length: 12314 3 TxHeader c Date: Thu, 02 Dec 2010 11:50:22 GMT 3 TxHeader c X-Varnish: 548007645 3 TxHeader c Age: 0 3 TxHeader c Via: 1.1 varnish 3 TxHeader c Connection: keep-alive 3 ReqEnd c 548007645 1291290619.872500896 1291290622.596549034 0.000085831 2.723987103 0.000061035 3 Debug c "herding" 0 Backend_health - s2 Still healthy 4--X-RH 10 5 10 0.053767 0.053917 HTTP/1.1 200 OK 0 Backend_health - s1 Still healthy 4--X-RH 10 5 10 0.053378 0.064020 HTTP/1.1 200 OK 0 CLI - Rd ping 0 CLI - Wr 200 PONG 1291290624 1.0 0 Backend_health - s2 Still healthy 4--X-RH 10 5 10 0.053538 0.053822 HTTP/1.1 200 OK 0 Backend_health - s1 Still healthy 4--X-RH 10 5 10 0.054416 0.061619 HTTP/1.1 200 OK 0 Backend_health - s2 Still healthy 4--X-RH 10 5 10 0.053792 0.053815 HTTP/1.1 200 OK And this is from firefox ? same pc different browser and it does not hit cache. 4 TxHeader b X-Varnish: 548007646 0 Backend_health - s1 Still healthy 4--X-RH 10 5 10 0.163856 0.081809 HTTP/1.1 200 OK 0 Backend_health - s2 Still healthy 4--X-RH 10 5 10 0.058637 0.062411 HTTP/1.1 200 OK 0 Backend_health - s1 Still healthy 4--X-RH 10 5 10 0.054320 0.074936 HTTP/1.1 200 OK 0 CLI - Rd ping 0 CLI - Wr 200 PONG 1291290714 1.0 0 Backend_health - s2 Still healthy 4--X-RH 10 5 10 0.074913 0.065536 HTTP/1.1 200 OK 4 RxProtocol b HTTP/1.1 4 RxStatus b 200 4 RxResponse b OK 4 RxHeader b Date: Thu, 02 Dec 2010 11:51:52 GMT 4 RxHeader b Server: Apache/2.2.9 (Fedora) 4 RxHeader b Vary: User-Agent,Cookie,Accept-Encoding 4 RxHeader b Content-Encoding: gzip 4 RxHeader b Connection: close 4 RxHeader b Transfer-Encoding: chunked 4 RxHeader b Content-Type: text/html; charset=utf-8 3 TTL c 548007646 RFC 120 1291290715 0 0 0 0 3 VCL_call c fetch 3 TTL c 548007646 VCL 86400 1291290712 3 TTL c 548007646 VCL 240 1291290712 3 VCL_return c deliver 3 ObjProtocol c HTTP/1.1 3 ObjStatus c 200 3 ObjResponse c OK 3 ObjHeader c Date: Thu, 02 Dec 2010 11:51:52 GMT 3 ObjHeader c Server: Apache/2.2.9 (Fedora) 3 ObjHeader c Vary: User-Agent,Cookie,Accept-Encoding 3 ObjHeader c Content-Type: text/html; charset=utf-8 3 ObjHeader c Content-Encoding: gzip 3 ObjHeader c Cache-Control: max-age=3600 3 ObjHeader c magicmarker: 1 4 BackendClose b s2 3 Length c 12314 3 VCL_call c deliver 3 VCL_return c deliver 3 TxProtocol c HTTP/1.1 3 TxStatus c 200 3 TxResponse c OK 3 TxHeader c Server: Apache/2.2.9 (Fedora) 3 TxHeader c Vary: User-Agent,Cookie,Accept-Encoding 3 TxHeader c Content-Type: text/html; charset=utf-8 3 TxHeader c Content-Encoding: gzip 3 TxHeader c Cache-Control: max-age=3600 3 TxHeader c magicmarker: 1 3 TxHeader c Content-Length: 12314 3 TxHeader c Date: Thu, 02 Dec 2010 11:51:55 GMT 3 TxHeader c X-Varnish: 548007646 3 TxHeader c Age: 0 3 TxHeader c Via: 1.1 varnish 3 TxHeader c Connection: keep-alive 3 ReqEnd c 548007646 1291290712.451340914 1291290715.647519112 0.000088930 3.196125031 0.000053167 3 Debug c "herding" 0 Backend_health - s1 Still healthy 4--X-RH 10 5 10 0.053839 0.069662 HTTP/1.1 200 OK 0 Backend_health - s2 Still healthy 4--X-RH 10 5 10 0.054015 0.062656 HTTP/1.1 200 OK Vitaly Burshteyn Senior Network Engineer Broadway.com, Theatre Direct International 729 7th Avenue New York, New York 10019 Phone: 212.817.9117 Cell# 917-701-5732 From: Stig Bakken [mailto:stig at zedge.net] Sent: Thursday, December 02, 2010 1:24 AM To: Vitaly Burshteyn Cc: varnish-misc at varnish-cache.org Subject: Re: normalicing accept headers Hi Vitaly, It would probably be easier to help if you posted a log dump of the same url in two browsers. - Stig On Thu, Dec 2, 2010 at 4:06 AM, Vitaly Burshteyn wrote: Hello, So I am still trying to figure out why I cant hit a cached page with a different browser other then the one it was cached with. I am pretty sure i got my vary statements right, they show up similar in the logs, so I know I am missing something. Can there be something other then Vary tags that can be causing this? Thank you. Vitaly Burshteyn Senior Network Engineer Broadway.com, Theatre Direct International 729 7th Avenue New York, New York 10019 Phone: 212.817.9117 Cell# 917-701-5732 _______________________________________________ varnish-misc mailing list varnish-misc at varnish-cache.org http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc ____________________________________ The information contained in this transmission may contain privileged and confidential information. It is intended only for the use of the person(s) named above. If you are not the intended recipient, you are hereby notified that any review, dissemination, distribution or duplication of this communication is strictly prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ionathan at gmail.com Thu Dec 2 11:58:45 2010 From: ionathan at gmail.com (Jonathan Leibiusky) Date: Thu, 2 Dec 2010 08:58:45 -0300 Subject: normalicing accept headers In-Reply-To: <0F69574F9901D4459C6B75C9FF64FBC405284488@mxfl01.hollywoodmedia.corp> References: <0F69574F9901D4459C6B75C9FF64FBC405284487@mxfl01.hollywoodmedia.corp> <0F69574F9901D4459C6B75C9FF64FBC405284488@mxfl01.hollywoodmedia.corp> Message-ID: Hi, I can see you are Varying by User-Agent and Cookie, it might be that. Varying by User-Agent will make varnish to hit again the backend if it doesn't have a cached version already. And varying by cookie will hit again even if you have a cached version for the browser, unless you always send the same cookie. I think your problem is with the Vary header. On Thu, Dec 2, 2010 at 8:52 AM, Vitaly Burshteyn wrote: > Stig, > > > > Ok so here is the log output from crome: > > > > > > 4 TxHeader b Host: stagingvarnish.broadway.com > > 4 TxHeader b Referer: > http://stagingvarnish.broadway.com/shows/jersey-boys/ > > 4 TxHeader b Accept: > application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 > > 4 TxHeader b User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; > en-US) AppleWebKit/534.7 (KHTML, like Gecko) Chrome/7.0.517.44 Safari/534.7 > > 4 TxHeader b Accept-Language: en-US,en;q=0.8 > > 4 TxHeader b Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3 > > 4 TxHeader b Accept-Encoding: gzip > > 4 TxHeader b Cookie: omni=bwy; LMD_18157499=0; affi=old; s_cc=true; > s_sq=broadwayprod%3D%2526pid%253DShow%25253A%252520Jersey%252520Boys%25253A%252520Overview%2526pidt%253D1%2526oid%253Dhttp%25253A// > stagingvarnish.broadway.com/%2526ot%253DA%26broadwayqa%3D%2526pid%253 > > 4 TxHeader b X-Varnish: 548007645 > > 0 Backend_health - s1 Still healthy 4--X-RH 10 5 10 0.110571 0.069738 > HTTP/1.1 200 OK > > 0 Backend_health - s2 Still healthy 4--X-RH 10 5 10 0.054721 0.053967 > HTTP/1.1 200 OK > > 0 CLI - Rd ping > > 0 CLI - Wr 200 PONG 1291290621 1.0 > > 0 Backend_health - s1 Still healthy 4--X-RH 10 5 10 0.061056 0.067568 > HTTP/1.1 200 OK > > 4 RxProtocol b HTTP/1.1 > > 4 RxStatus b 200 > > 4 RxResponse b OK > > 4 RxHeader b Date: Thu, 02 Dec 2010 11:50:19 GMT > > 4 RxHeader b Server: Apache/2.2.9 (Fedora) > > 4 RxHeader b Vary: User-Agent,Cookie,Accept-Encoding > > 4 RxHeader b Content-Encoding: gzip > > 4 RxHeader b Connection: close > > 4 RxHeader b Transfer-Encoding: chunked > > 4 RxHeader b Content-Type: text/html; charset=utf-8 > > 3 TTL c 548007645 RFC 120 1291290622 0 0 0 0 > > 3 VCL_call c fetch > > 3 TTL c 548007645 VCL 86400 1291290620 > > 3 TTL c 548007645 VCL 240 1291290620 > > 3 VCL_return c deliver > > 3 ObjProtocol c HTTP/1.1 > > 3 ObjStatus c 200 > > 3 ObjResponse c OK > > 3 ObjHeader c Date: Thu, 02 Dec 2010 11:50:19 GMT > > 3 ObjHeader c Server: Apache/2.2.9 (Fedora) > > 3 ObjHeader c Vary: User-Agent,Cookie,Accept-Encoding > > 3 ObjHeader c Content-Type: text/html; charset=utf-8 > > 3 ObjHeader c Content-Encoding: gzip > > 3 ObjHeader c Cache-Control: max-age=3600 > > 3 ObjHeader c magicmarker: 1 > > 4 BackendClose b s1 > > 3 Length c 12314 > > 3 VCL_call c deliver > > 3 VCL_return c deliver > > 3 TxProtocol c HTTP/1.1 > > 3 TxStatus c 200 > > 3 TxResponse c OK > > 3 TxHeader c Server: Apache/2.2.9 (Fedora) > > 3 TxHeader c Vary: User-Agent,Cookie,Accept-Encoding > > 3 TxHeader c Content-Type: text/html; charset=utf-8 > > 3 TxHeader c Content-Encoding: gzip > > 3 TxHeader c Cache-Control: max-age=3600 > > 3 TxHeader c magicmarker: 1 > > 3 TxHeader c Content-Length: 12314 > > 3 TxHeader c Date: Thu, 02 Dec 2010 11:50:22 GMT > > 3 TxHeader c X-Varnish: 548007645 > > 3 TxHeader c Age: 0 > > 3 TxHeader c Via: 1.1 varnish > > 3 TxHeader c Connection: keep-alive > > 3 ReqEnd c 548007645 1291290619.872500896 1291290622.596549034 > 0.000085831 2.723987103 0.000061035 > > 3 Debug c "herding" > > 0 Backend_health - s2 Still healthy 4--X-RH 10 5 10 0.053767 0.053917 > HTTP/1.1 200 OK > > 0 Backend_health - s1 Still healthy 4--X-RH 10 5 10 0.053378 0.064020 > HTTP/1.1 200 OK > > 0 CLI - Rd ping > > 0 CLI - Wr 200 PONG 1291290624 1.0 > > 0 Backend_health - s2 Still healthy 4--X-RH 10 5 10 0.053538 0.053822 > HTTP/1.1 200 OK > > 0 Backend_health - s1 Still healthy 4--X-RH 10 5 10 0.054416 0.061619 > HTTP/1.1 200 OK > > 0 Backend_health - s2 Still healthy 4--X-RH 10 5 10 0.053792 0.053815 > HTTP/1.1 200 OK > > > > > > > > > > > > And this is from firefox ? same pc different browser and it does not hit > cache. > > > > 4 TxHeader b X-Varnish: 548007646 > > 0 Backend_health - s1 Still healthy 4--X-RH 10 5 10 0.163856 0.081809 > HTTP/1.1 200 OK > > 0 Backend_health - s2 Still healthy 4--X-RH 10 5 10 0.058637 0.062411 > HTTP/1.1 200 OK > > 0 Backend_health - s1 Still healthy 4--X-RH 10 5 10 0.054320 0.074936 > HTTP/1.1 200 OK > > 0 CLI - Rd ping > > 0 CLI - Wr 200 PONG 1291290714 1.0 > > 0 Backend_health - s2 Still healthy 4--X-RH 10 5 10 0.074913 0.065536 > HTTP/1.1 200 OK > > 4 RxProtocol b HTTP/1.1 > > 4 RxStatus b 200 > > 4 RxResponse b OK > > 4 RxHeader b Date: Thu, 02 Dec 2010 11:51:52 GMT > > 4 RxHeader b Server: Apache/2.2.9 (Fedora) > > 4 RxHeader b Vary: User-Agent,Cookie,Accept-Encoding > > 4 RxHeader b Content-Encoding: gzip > > 4 RxHeader b Connection: close > > 4 RxHeader b Transfer-Encoding: chunked > > 4 RxHeader b Content-Type: text/html; charset=utf-8 > > 3 TTL c 548007646 RFC 120 1291290715 0 0 0 0 > > 3 VCL_call c fetch > > 3 TTL c 548007646 VCL 86400 1291290712 > > 3 TTL c 548007646 VCL 240 1291290712 > > 3 VCL_return c deliver > > 3 ObjProtocol c HTTP/1.1 > > 3 ObjStatus c 200 > > 3 ObjResponse c OK > > 3 ObjHeader c Date: Thu, 02 Dec 2010 11:51:52 GMT > > 3 ObjHeader c Server: Apache/2.2.9 (Fedora) > > 3 ObjHeader c Vary: User-Agent,Cookie,Accept-Encoding > > 3 ObjHeader c Content-Type: text/html; charset=utf-8 > > 3 ObjHeader c Content-Encoding: gzip > > 3 ObjHeader c Cache-Control: max-age=3600 > > 3 ObjHeader c magicmarker: 1 > > 4 BackendClose b s2 > > 3 Length c 12314 > > 3 VCL_call c deliver > > 3 VCL_return c deliver > > 3 TxProtocol c HTTP/1.1 > > 3 TxStatus c 200 > > 3 TxResponse c OK > > 3 TxHeader c Server: Apache/2.2.9 (Fedora) > > 3 TxHeader c Vary: User-Agent,Cookie,Accept-Encoding > > 3 TxHeader c Content-Type: text/html; charset=utf-8 > > 3 TxHeader c Content-Encoding: gzip > > 3 TxHeader c Cache-Control: max-age=3600 > > 3 TxHeader c magicmarker: 1 > > 3 TxHeader c Content-Length: 12314 > > 3 TxHeader c Date: Thu, 02 Dec 2010 11:51:55 GMT > > 3 TxHeader c X-Varnish: 548007646 > > 3 TxHeader c Age: 0 > > 3 TxHeader c Via: 1.1 varnish > > 3 TxHeader c Connection: keep-alive > > 3 ReqEnd c 548007646 1291290712.451340914 1291290715.647519112 > 0.000088930 3.196125031 0.000053167 > > 3 Debug c "herding" > > 0 Backend_health - s1 Still healthy 4--X-RH 10 5 10 0.053839 0.069662 > HTTP/1.1 200 OK > > 0 Backend_health - s2 Still healthy 4--X-RH 10 5 10 0.054015 0.062656 > HTTP/1.1 200 OK > > > > > > > > > > > > > > Vitaly Burshteyn > > Senior Network Engineer > > Broadway.com, Theatre Direct International > > 729 7th Avenue > > New York, New York 10019 > > Phone: 212.817.9117 > > Cell# 917-701-5732 > > > > *From:* Stig Bakken [mailto:stig at zedge.net] > *Sent:* Thursday, December 02, 2010 1:24 AM > *To:* Vitaly Burshteyn > *Cc:* varnish-misc at varnish-cache.org > *Subject:* Re: normalicing accept headers > > > > Hi Vitaly, > > > > It would probably be easier to help if you posted a log dump of the same > url in two browsers. > > > > - Stig > > > > On Thu, Dec 2, 2010 at 4:06 AM, Vitaly Burshteyn > wrote: > > Hello, > > > > So I am still trying to figure out why I cant hit a cached page with a > different browser other then the one it was cached with. I am pretty sure i > got my vary statements right, they show up similar in the logs, so I know I > am missing something. > > > > Can there be something other then Vary tags that can be causing this? > > > > Thank you. > > > > Vitaly Burshteyn > > Senior Network Engineer > > Broadway.com, Theatre Direct International > > 729 7th Avenue > > New York, New York 10019 > > Phone: 212.817.9117 > > Cell# 917-701-5732 > > > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From perbu at varnish-software.com Thu Dec 2 12:00:48 2010 From: perbu at varnish-software.com (Per Buer) Date: Thu, 2 Dec 2010 13:00:48 +0100 Subject: normalicing accept headers In-Reply-To: <0F69574F9901D4459C6B75C9FF64FBC405284488@mxfl01.hollywoodmedia.corp> References: <0F69574F9901D4459C6B75C9FF64FBC405284487@mxfl01.hollywoodmedia.corp> <0F69574F9901D4459C6B75C9FF64FBC405284488@mxfl01.hollywoodmedia.corp> Message-ID: Vitaly, Your vary header is: Vary: User-Agent,Cookie,Accept-Encoding So you'll more or less have every page cached separately for every user. I would drop the User-Agent from Vary and normalize Accept-Encoding in vcl_recv. Also, you need take special care of your cookies if you're not doing this already. Per. On Thu, Dec 2, 2010 at 12:52 PM, Vitaly Burshteyn wrote: > > Stig, > > > > Ok so here is the log output from crome: > > > > > > ?? 4 TxHeader???? b Host: stagingvarnish.broadway.com > > ??? 4 TxHeader???? b Referer: http://stagingvarnish.broadway.com/shows/jersey-boys/ > > ??? 4 TxHeader??? ?b Accept: application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 > > ??? 4 TxHeader???? b User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.7 (KHTML, like Gecko) Chrome/7.0.517.44 Safari/534.7 > > ??? 4 TxHeader???? b Accept-Language: en-US,en;q=0.8 > > ??? 4 TxHeader???? b Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3 > > ??? 4 TxHeader???? b Accept-Encoding: gzip > > ??? 4 TxHeader???? b Cookie: omni=bwy; LMD_18157499=0; affi=old; s_cc=true; s_sq=broadwayprod%3D%2526pid%253DShow%25253A%252520Jersey%252520Boys%25253A%252520Overview%2526pidt%253D1%2526oid%253Dhttp%25253A//stagingvarnish.broadway.com/%2526ot%253DA%26broadwayqa%3D%2526pid%253 > > ??? 4 TxHeader???? b X-Varnish: 548007645 > > ??? 0 Backend_health - s1 Still healthy 4--X-RH 10 5 10 0.110571 0.069738 HTTP/1.1 200 OK > > ??? 0 Backend_health - s2 Still healthy 4--X-RH 10 5 10 0.054721 0.053967 HTTP/1.1 200 OK > > ??? 0 CLI????????? - Rd ping > > ??? 0 CLI????????? - Wr 200 PONG 1291290621 1.0 > > ??? 0 Backend_health - s1 Still healthy 4--X-RH 10 5 10 0.061056 0.067568 HTTP/1.1 200 OK > > ??? 4 RxProtocol?? b HTTP/1.1 > > ??? 4 RxStatus???? b 200 > > ??? 4 RxResponse?? b OK > > ??? 4 RxHeader???? b Date: Thu, 02 Dec 2010 11:50:19 GMT > > ??? 4 RxHeader???? b Server: Apache/2.2.9 (Fedora) > > ??? 4 RxHeader???? b Vary: User-Agent,Cookie,Accept-Encoding > > ??? 4 RxHeader???? b Content-Encoding: gzip > > ??? 4 RxHeader???? b Connection: close > > ??? 4 RxHeader???? b Transfer-Encoding: chunked > > ??? 4 RxHeader???? b Content-Type: text/html; charset=utf-8 > > ??? 3 TTL????????? c 548007645 RFC 120 1291290622 0 0 0 0 > > ??? 3 VCL_call???? c fetch > > ??? 3 TTL????????? c 548007645 VCL 86400 1291290620 > > ??? 3 TTL????????? c 548007645 VCL 240 1291290620 > > ??? 3 VCL_return?? c deliver > > ??? 3 ObjProtocol? c HTTP/1.1 > > ??? 3 ObjStatus??? c 200 > > ??? 3 ObjResponse? c OK > > ??? 3 ObjHeader??? c Date: Thu, 02 Dec 2010 11:50:19 GMT > > ??? 3 ObjHeader??? c Server: Apache/2.2.9 (Fedora) > > ??? 3 ObjHeader??? c Vary: User-Agent,Cookie,Accept-Encoding > > ??? 3 ObjHeader??? c Content-Type: text/html; charset=utf-8 > > ??? 3 ObjHeader??? c Content-Encoding: gzip > > ??? 3 ObjHeader??? c Cache-Control: max-age=3600 > > ??? 3 ObjHeader??? c magicmarker: 1 > > ??? 4 BackendClose b s1 > > ??? 3 Length?????? c 12314 > > ??? 3 VCL_call???? c deliver > > ??? 3 VCL_return?? c deliver > > ??? 3 TxProtocol?? c HTTP/1.1 > > ??? 3 TxStatus???? c 200 > > ??? 3 TxResponse?? c OK > > ??? 3 TxHeader???? c Server: Apache/2.2.9 (Fedora) > > ??? 3 TxHeader???? c Vary: User-Agent,Cookie,Accept-Encoding > > ??? 3 TxHeader???? c Content-Type: text/html; charset=utf-8 > > ??? 3 TxHeader???? c Content-Encoding: gzip > > ??? 3 TxHeader???? c Cache-Control: max-age=3600 > > ??? 3 TxHeader???? c magicmarker: 1 > > ??? 3 TxHeader???? c Content-Length: 12314 > > ??? 3 TxHeader???? c Date: Thu, 02 Dec 2010 11:50:22 GMT > > ??? 3 TxHeader???? c X-Varnish: 548007645 > > ??? 3 TxHeader???? c Age: 0 > > ??? 3 TxHeader???? c Via: 1.1 varnish > > ??? 3 TxHeader???? c Connection: keep-alive > > ??? 3 ReqEnd?????? c 548007645 1291290619.872500896 1291290622.596549034 0.000085831 2.723987103 0.000061035 > > ??? 3 Debug??????? c "herding" > > ??? 0 Backend_health - s2 Still healthy 4--X-RH 10 5 10 0.053767 0.053917 HTTP/1.1 200 OK > > ??? 0 Backend_health - s1 Still healthy 4--X-RH 10 5 10 0.053378 0.064020 HTTP/1.1 200 OK > > ??? 0 CLI????????? - Rd ping > > ??? 0 CLI????????? - Wr 200 PONG 1291290624 1.0 > > ??? 0 Backend_health - s2 Still healthy 4--X-RH 10 5 10 0.053538 0.053822 HTTP/1.1 200 OK > > ??? 0 Backend_health - s1 Still healthy 4--X-RH 10 5 10 0.054416 0.061619 HTTP/1.1 200 OK > > ??? 0 Backend_health - s2 Still healthy 4--X-RH 10 5 10 0.053792 0.053815 HTTP/1.1 200 OK > > > > > > > > > > > > And this is from firefox ? same pc different browser and it does not hit cache. > > > > ? 4 TxHeader???? b X-Varnish: 548007646 > > ??? 0 Backend_health - s1 Still healthy 4--X-RH 10 5 10 0.163856 0.081809 HTTP/1.1 200 OK > > ??? 0 Backend_health - s2 Still healthy 4--X-RH 10 5 10 0.058637 0.062411 HTTP/1.1 200 OK > > ??? 0 Backend_health - s1 Still healthy 4--X-RH 10 5 10 0.054320 0.074936 HTTP/1.1 200 OK > > ??? 0 CLI????????? - Rd ping > > ??? 0 CLI????????? - Wr 200 PONG 1291290714 1.0 > > ??? 0 Backend_health - s2 Still healthy 4--X-RH 10 5 10 0.074913 0.065536 HTTP/1.1 200 OK > > ??? 4 RxProtocol?? b HTTP/1.1 > > ??? 4 RxStatus???? b 200 > > ??? 4 RxResponse?? b OK > > ??? 4 RxHeader???? b Date: Thu, 02 Dec 2010 11:51:52 GMT > > ??? 4 RxHeader???? b Server: Apache/2.2.9 (Fedora) > > ??? 4 RxHeader???? b Vary: User-Agent,Cookie,Accept-Encoding > > ??? 4 RxHeader???? b Content-Encoding: gzip > > ??? 4 RxHeader???? b Connection: close > > ??? 4 RxHeader???? b Transfer-Encoding: chunked > > ??? 4 RxHeader???? b Content-Type: text/html; charset=utf-8 > > ??? 3 TTL????????? c 548007646 RFC 120 1291290715 0 0 0 0 > > ??? 3 VCL_call ????c fetch > > ??? 3 TTL????????? c 548007646 VCL 86400 1291290712 > > ??? 3 TTL????????? c 548007646 VCL 240 1291290712 > > ??? 3 VCL_return?? c deliver > > ??? 3 ObjProtocol? c HTTP/1.1 > > ??? 3 ObjStatus??? c 200 > > ??? 3 ObjResponse? c OK > > ??? 3 ObjHeader??? c Date: Thu, 02 Dec 2010 11:51:52 GMT > > ??? 3 ObjHeader??? c Server: Apache/2.2.9 (Fedora) > > ??? 3 ObjHeader??? c Vary: User-Agent,Cookie,Accept-Encoding > > ??? 3 ObjHeader??? c Content-Type: text/html; charset=utf-8 > > ??? 3 ObjHeader??? c Content-Encoding: gzip > > ??? 3 ObjHeader? ??c Cache-Control: max-age=3600 > > ??? 3 ObjHeader??? c magicmarker: 1 > > ??? 4 BackendClose b s2 > > ??? 3 Length?????? c 12314 > > ??? 3 VCL_call???? c deliver > > ??? 3 VCL_return?? c deliver > > ??? 3 TxProtocol?? c HTTP/1.1 > > ??? 3 TxStatus???? c 200 > > ??? 3 TxResponse?? c OK > > ??? 3 TxHeader???? c Server: Apache/2.2.9 (Fedora) > > ??? 3 TxHeader???? c Vary: User-Agent,Cookie,Accept-Encoding > > ??? 3 TxHeader???? c Content-Type: text/html; charset=utf-8 > > ??? 3 TxHeader???? c Content-Encoding: gzip > > ??? 3 TxHeader???? c Cache-Control: max-age=3600 > > ??? 3 TxHeader???? c magicmarker: 1 > > ??? 3 TxHeader???? c Content-Length: 12314 > > ??? 3 TxHeader???? c Date: Thu, 02 Dec 2010 11:51:55 GMT > > ??? 3 TxHeader???? c X-Varnish: 548007646 > > ??? 3 TxHeader???? c Age: 0 > > ??? 3 TxHeader???? c Via: 1.1 varnish > > ?? ?3 TxHeader???? c Connection: keep-alive > > ??? 3 ReqEnd?????? c 548007646 1291290712.451340914 1291290715.647519112 0.000088930 3.196125031 0.000053167 > > ??? 3 Debug??????? c "herding" > > ??? 0 Backend_health - s1 Still healthy 4--X-RH 10 5 10 0.053839 0.069662 HTTP/1.1 200 OK > > ??? 0 Backend_health - s2 Still healthy 4--X-RH 10 5 10 0.054015 0.062656 HTTP/1.1 200 OK > > > > > > > > > > > > > > Vitaly Burshteyn > > Senior Network Engineer > > Broadway.com, Theatre Direct International > > 729? 7th Avenue > > New York, New York 10019 > > Phone: 212.817.9117 > > Cell# 917-701-5732 > > > > From: Stig Bakken [mailto:stig at zedge.net] > Sent: Thursday, December 02, 2010 1:24 AM > To: Vitaly Burshteyn > Cc: varnish-misc at varnish-cache.org > Subject: Re: normalicing accept headers > > > > Hi Vitaly, > > > > It would probably be easier to help if you posted a log dump of the same url in two browsers. > > > > ?- Stig > > > > On Thu, Dec 2, 2010 at 4:06 AM, Vitaly Burshteyn wrote: > > Hello, > > > > So I am still trying to figure out why I cant hit a cached page with a different browser other then the one it was cached with.? I am pretty sure i got my vary statements right, they show up similar in the logs, so I know I am missing something. > > > > Can there be something other then Vary? tags that can be causing this? > > > > Thank you. > > > > Vitaly Burshteyn > > Senior Network Engineer > > Broadway.com, Theatre Direct International > > 729? 7th Avenue > > New York, New York 10019 > > Phone: 212.817.9117 > > Cell# 917-701-5732 > > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc -- Per Buer,?Varnish Software Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer Varnish makes websites fly! From Klapproth.M at sportresult.com Thu Dec 2 12:32:35 2010 From: Klapproth.M at sportresult.com (Klapproth, Martin) Date: Thu, 2 Dec 2010 13:32:35 +0100 Subject: Individual Cache for different Browsers? Message-ID: <4BC50911656FBB49858C6ADB1B8D223F038D96CB@mail.wigemic.lpz> Hello, I have requested one page with different browsers (firefox, chrome). If I make a second request with the same browser I get the object from the varnish cache. But if the first browser makes the first request and the second browser makes the second one I get a miss. In my configuration the hash will be determined only by the url and host header (nothing else). Client side Cache-Control or Pragma headers were totally ignored. Here a snippet out of my vcl config: sub vcl_recv { unset req.http.Pragma; unset req.http.Cache-Control; ... } sub vcl_hash { set req.hash += req.url; if (req.url == "/") { set req.hash += req.http.host; } return (hash); } sub vcl_deliver { if (obj.hits > 0) { set resp.http.X-Cache = "HIT"; } else { set resp.http.X-Cache = "MISS"; } unset resp.http.X-Varnish; } Here the 2 browser requests: Request 1 by Firefox: Host: XXXXX User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.12) Gecko/20101026 Firefox/3.6.12 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 115 Connection: keep-alive Referer: XXXXX Cookie: __utma=31143592.1898076621.1291293810.1291291810.1291291810.1; __utmb=31173592; __utmc=31543592; __utmz=31143592.1291291810.1.1.utmccn=(direct)|utmcsr=(direct)|utmcmd=(n one); __utma=61459198.633465754.1291291861.1291241861.1291291861.1; __utmb=6145912.2.10.1291291861; __utmc=61459198; __utmz=61459198.1291881861.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(n one Response 1 (Request made by Firefox) Server: nginx Date: Thu, 02 Dec 2010 12:11:26 GMT Content-Type: text/html; Charset=utf-8 Connection: keep-alive Keep-Alive: timeout=20 Cache-Control: public, max-age=600 Expires: Thu, 02 Dec 2010 12:21:26 GMT Last-Modified: Thu, 02 Dec 2010 12:11:26 GMT CacheInfo: 600 Content-Encoding: gzip X-Cache: MISS // that's ok, cause the object wasn't cached before Content-Length: 7978 Age: 0 Request 2 by Chrome Accept: application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,i mage/png,*/*;q=0.5 User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.7 (KHTML, like Gecko) Chrome/7.0.517.44 Safari/534.7 Response 2 by Chrome Age: 0 Cache-Control: public, maxage=600 CacheInfo: 600 Connection: keep-alive Content-Encoding: gzip Content-Length: 8004 Content-Type: text/html; Charset=utf-8 Date: Thu, 02 Dec 2010 12:11:56 GMT Expires: Thu, 02 Dec 2010 12:21:56 GMT Keep-Alive: timeout=20 Last-Modified: Thu, 02 Dec 2010 12:11:56 GMT Server: nginx X-Cache: MISS // That's not ok, cause the request was done 30 sec after the first one ant the max-age is 600 The nginx is only proxying the request to varnish and handles gzip. The object will be cached (can see it in varnishlog). Can you help me with my problem? Thanks Martin -------------- next part -------------- An HTML attachment was scrubbed... URL: From sime at sime.net.au Thu Dec 2 12:28:08 2010 From: sime at sime.net.au (Simon Males) Date: Thu, 2 Dec 2010 23:28:08 +1100 Subject: Chasing 503 Guru's Message-ID: Just wanted to share my understanding of tracking down 503 / Guru meditation errors. Steps: 1. Source the XID 2. Start varnishlog with -d 3. Search the output for the above mentioned XID 4. Hopefully the log entry will provide a clue Though how long do the shared memory logs exist? Or is it common practice to write varnishlog to disk in production environments? -- Simon Males From perbu at varnish-software.com Thu Dec 2 13:04:09 2010 From: perbu at varnish-software.com (Per Buer) Date: Thu, 2 Dec 2010 14:04:09 +0100 Subject: Chasing 503 Guru's In-Reply-To: References: Message-ID: On Thu, Dec 2, 2010 at 1:28 PM, Simon Males wrote: > Just wanted to share my understanding of tracking down 503 / Guru > meditation errors. > > Steps: > 1. Source the XID > 2. Start varnishlog with -d > 3. Search the output for the above mentioned XID > 4. Hopefully the log entry will provide a clue > > Though how long do the shared memory logs exist? Depends on how much traffic you have. On a busy site it should be a minute or so. Try logging to disk and you notice how long 80MB (the default) will last. It will give you an indication on how long the shmlog lasts. However, the shmlog has some fixed length records so it wastes a bit of space compared to the output to disk. > Or is it common practice to write varnishlog to disk in production environments? No. On Linux the vm get's slightly dazzled by all the IO and will behave strangely. You can instruct varnishlog to only log 503's. That might be ideal for you. -- Per Buer,?Varnish Software Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer Varnish makes websites fly! From tfheen at varnish-software.com Thu Dec 2 13:09:27 2010 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Thu, 02 Dec 2010 14:09:27 +0100 Subject: Different cache policy for googlebot In-Reply-To: <0C561017-9A55-4CD9-9D66-B31AC7B0CBD6@collaborativebusiness.co.uk> (David Turner's message of "Thu, 2 Dec 2010 10:45:48 +0000") References: <2263.1291285992@critter.freebsd.dk> <0C561017-9A55-4CD9-9D66-B31AC7B0CBD6@collaborativebusiness.co.uk> Message-ID: <87ipzcwcc8.fsf@qurzaw.varnish-software.com> ]] David Turner | On 2 Dec 2010, at 10:33, Poul-Henning Kamp wrote: | | > You only need to slow it on cache misses ? | | The other way around. I don't mind it getting old data for a cache | hit, but a cache miss should be serviced as required. | | Humans should get be getting lightly cached pages. This is currently | being handled in the application but to keep the varnish cache up to | date the human requests need to be fetched instead of passed straight | through. Sounds like you want to set obj.grace to a high value, and then set req.grace to low values for humans, and a higher value for robots and turn on saint mode for robots. At least I think that'll fix your use case. -- Tollef Fog Heen Varnish Software t: +47 21 98 92 64 From david at collaborativebusiness.co.uk Thu Dec 2 14:17:50 2010 From: david at collaborativebusiness.co.uk (David Turner) Date: Thu, 2 Dec 2010 14:17:50 +0000 Subject: Different cache policy for googlebot In-Reply-To: <87ipzcwcc8.fsf@qurzaw.varnish-software.com> References: <2263.1291285992@critter.freebsd.dk> <0C561017-9A55-4CD9-9D66-B31AC7B0CBD6@collaborativebusiness.co.uk> <87ipzcwcc8.fsf@qurzaw.varnish-software.com> Message-ID: On 2 Dec 2010, at 13:09, Tollef Fog Heen wrote: > Sounds like you want to set obj.grace to a high value, and then set > req.grace to low values for humans, and a higher value for robots and > turn on saint mode for robots. At least I think that'll fix your use > case. On your advice Tollef I tried the following test configuration and sent requests using lwp-request to change the user agent string. sub vcl_fetch { set beresp.ttl = 10s; set beresp.grace = 60s; } sub vcl_recv { if ( req.http.user-agent ~ "foo" ) { set req.grace = 30s; } else { set req.grace = 5s; } } But that's just expiring content for either user-agent case according to the ttl. I think it must be the saint mode, but I don't understand how to set that for a specific user agent, as I read the documentation I should use grace mode OR saintmode and the parameter is set on the beresp object. I missed something I'm sure... -- David M Turner Collaborative Business Services Ltd Tel: +44 161 850 0586 Collaborative Business Services Ltd is a private limited company registered in England and Wales. Registered number: 6666362 Registered office: Floor 10, 3 Hardman Street, Manchester, M3 3HF From sime at sime.net.au Thu Dec 2 14:30:01 2010 From: sime at sime.net.au (Simon Males) Date: Fri, 3 Dec 2010 01:30:01 +1100 Subject: Chasing 503 Guru's In-Reply-To: References: Message-ID: On Fri, Dec 3, 2010 at 12:04 AM, Per Buer wrote: > On Thu, Dec 2, 2010 at 1:28 PM, Simon Males wrote: >> Just wanted to share my understanding of tracking down 503 / Guru >> meditation errors. >> >> Steps: >> 1. Source the XID >> 2. Start varnishlog with -d >> 3. Search the output for the above mentioned XID >> 4. Hopefully the log entry will provide a clue >> >> Though how long do the shared memory logs exist? > > Depends on how much traffic you have. On a busy site it should be a > minute or so. Try logging to disk and you notice how long 80MB (the > default) will last. It will give you an indication on how long the > shmlog lasts. However, the shmlog has some fixed length records so it > wastes a bit of space compared to the output to disk. So `varnishlog -w /tmp/varnishd.log` is self managing/rotating? I performed the above with -d and it maintained at 58M. Without -d it's growing slowly then expected. 7M after 20 minutes of traffic. Where is logging to disk size specified? >> Or is it common practice to write varnishlog to disk in production environments? > > No. On Linux the vm get's slightly dazzled by all the IO and will > behave strangely. You can instruct varnishlog to only log 503's. That > might be ideal for you. I'm not sure how to do this. The closest I get is: varnishlog -o -i TxStatus -I 503 Though that doesn't return the whole entry. -- Simon Males From perbu at varnish-software.com Thu Dec 2 15:49:11 2010 From: perbu at varnish-software.com (Per Buer) Date: Thu, 2 Dec 2010 16:49:11 +0100 Subject: Chasing 503 Guru's In-Reply-To: References: Message-ID: On Thu, Dec 2, 2010 at 3:30 PM, Simon Males wrote: > On Fri, Dec 3, 2010 at 12:04 AM, Per Buer wrote: >> On Thu, Dec 2, 2010 at 1:28 PM, Simon Males wrote: >>> Just wanted to share my understanding of tracking down 503 / Guru >>> meditation errors. >>> >>> Steps: >>> 1. Source the XID >>> 2. Start varnishlog with -d >>> 3. Search the output for the above mentioned XID >>> 4. Hopefully the log entry will provide a clue >>> >>> Though how long do the shared memory logs exist? >> >> Depends on how much traffic you have. On a busy site it should be a >> minute or so. Try logging to disk and you notice how long 80MB (the >> default) will last. It will give you an indication on how long the >> shmlog lasts. However, the shmlog has some fixed length records so it >> wastes a bit of space compared to the output to disk. > > So `varnishlog -w /tmp/varnishd.log` is self managing/rotating? No. Not really. Varnish writes to a piece of shared memory. varnishlog _reads_ from this memory. The memory segment is written to in a circular buffer sort of way. So after a while the old content gets overwritten. > I performed the above with -d and it maintained at 58M. Without -d > it's growing slowly then expected. 7M after 20 minutes of traffic. > Where is logging to disk size specified? Nowhere. You have to run varnishlog as a deamon to log to disk i a permanent fasion. -- Per Buer,?Varnish Software Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer Varnish makes websites fly! From josh at schulzone.org Thu Dec 2 16:46:11 2010 From: josh at schulzone.org (Josh) Date: Thu, 2 Dec 2010 09:46:11 -0700 Subject: Individual Cache for different Browsers? In-Reply-To: <4BC50911656FBB49858C6ADB1B8D223F038D96CB@mail.wigemic.lpz> References: <4BC50911656FBB49858C6ADB1B8D223F038D96CB@mail.wigemic.lpz> Message-ID: My guess is that it's the cookies you're sending through. I had the same problem and it turned out to be analytics cookies. Those _ut* cookies look like the same thing. You can strip those out using something like this in vcl_recv set req.http.Cookie = regsuball(req.http.Cookie, "(^|;\s*)(__utma|__utms|etc...)=[^;]+;?", ""); and that should take care of it. On Thu, Dec 2, 2010 at 5:32 AM, Klapproth, Martin wrote: > Hello, > > > > I have requested one page with different browsers (firefox, chrome). If I > make a second request with the same browser I get the object from the > varnish cache. But if the first browser makes the first request and the > second browser makes the second one I get a miss. > > > > In my configuration the hash will be determined only by the url and host > header (nothing else). Client side Cache-Control or Pragma headers were > totally ignored. Here a snippet out of my vcl config: > > > > sub vcl_recv { > > ??????? unset req.http.Pragma; > > ??????? unset req.http.Cache-Control; > > ??????? ? > > } > > > > sub vcl_hash { > > ??? set req.hash += req.url; > > ??? if (req.url == "/") { > > ??????? set req.hash += req.http.host; > > ??? } > > ??? return (hash); > > } > > > > sub vcl_deliver { > > ?????????? if (obj.hits > 0) { > > ????????????? set resp.http.X-Cache = "HIT"; > > ?????????? } else { > > ??????????????? set resp.http.X-Cache = "MISS"; > > ?????????? } > > ?????????? unset resp.http.X-Varnish; > > } > > > > Here the 2 browser requests: > > > > Request 1 by Firefox: > > > > Host: XXXXX > > User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.12) > Gecko/20101026 Firefox/3.6.12 > > Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 > > Accept-Language: en-us,en;q=0.5 > > Accept-Encoding: gzip,deflate > > Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 > > Keep-Alive: 115 > > Connection: keep-alive > > Referer: XXXXX > > Cookie: __utma=31143592.1898076621.1291293810.1291291810.1291291810.1; > __utmb=31173592; __utmc=31543592; > __utmz=31143592.1291291810.1.1.utmccn=(direct)|utmcsr=(direct)|utmcmd=(none); > __utma=61459198.633465754.1291291861.1291241861.1291291861.1; > __utmb=6145912.2.10.1291291861; __utmc=61459198; > __utmz=61459198.1291881861.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none > > > > Response 1 (Request made by Firefox) > > > > Server: nginx > > Date: Thu, 02 Dec 2010 12:11:26 GMT > > Content-Type: text/html; Charset=utf-8 > > Connection: keep-alive > > Keep-Alive: timeout=20 > > Cache-Control: public, max-age=600 > > Expires: Thu, 02 Dec 2010 12:21:26 GMT > > Last-Modified: Thu, 02 Dec 2010 12:11:26 GMT > > CacheInfo: 600 > > Content-Encoding: gzip > > X-Cache: MISS???????? ??// that?s ok, cause the object wasn?t cached before > > Content-Length: 7978 > > Age: 0 > > > > Request 2 by Chrome > > > > Accept: > application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 > > User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) > AppleWebKit/534.7 (KHTML, like Gecko) Chrome/7.0.517.44 Safari/534.7 > > > > Response 2 by Chrome > > > > Age: 0 > > Cache-Control: public, maxage=600 > > CacheInfo: 600 > > Connection: keep-alive > > Content-Encoding: gzip > > Content-Length: 8004 > > Content-Type: text/html; Charset=utf-8 > > Date: Thu, 02 Dec 2010 12:11:56 GMT > > Expires: Thu, 02 Dec 2010 12:21:56 GMT > > Keep-Alive: timeout=20 > > Last-Modified: Thu, 02 Dec 2010 12:11:56 GMT > > Server: nginx > > X-Cache: MISS?? ??// That?s not ok, cause the request was done 30 sec after > the first one ant the max-age is 600 > > > > The nginx is only proxying the request to varnish and handles gzip. The > object will be cached (can see it in varnishlog). > > > > Can you help me with my problem? > > > > Thanks > > > > Martin > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -- josh @schulz http://schulzone.org From tfheen at varnish-software.com Fri Dec 3 09:50:24 2010 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Fri, 03 Dec 2010 10:50:24 +0100 Subject: 2.1.5 testing Message-ID: <87sjyfuqvz.fsf@qurzaw.varnish-software.com> Hi, We're getting close to 2.1.5 and I haven't received enough testing reports. If people could test the tar.gz available in http://varnish-cache.org/~tfheen/varnish-2.1.5pre1.tar.gz that would be most welcome. Report back to me privately or on the list. Cheers, -- Tollef Fog Heen Varnish Software t: +47 21 98 92 64 From amoiz.shine at gmail.com Fri Dec 3 10:09:01 2010 From: amoiz.shine at gmail.com (Sharl.Jimh.Tsin) Date: Fri, 3 Dec 2010 18:09:01 +0800 Subject: 2.1.5 testing In-Reply-To: <87sjyfuqvz.fsf@qurzaw.varnish-software.com> References: <87sjyfuqvz.fsf@qurzaw.varnish-software.com> Message-ID: OK,i will test it. Best regards, Sharl.Jimh.Tsin (From China **Obviously Taiwan INCLUDED**) 2010/12/3 Tollef Fog Heen : > > Hi, > > We're getting close to 2.1.5 and I haven't received enough testing > reports. ?If people could test the tar.gz available in > http://varnish-cache.org/~tfheen/varnish-2.1.5pre1.tar.gz that would be > most welcome. ?Report back to me privately or on the list. > > Cheers, > -- > Tollef Fog Heen > Varnish Software > t: +47 21 98 92 64 > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > From mhettwer at team.mobile.de Fri Dec 3 13:43:16 2010 From: mhettwer at team.mobile.de (Hettwer, Marian) Date: Fri, 3 Dec 2010 13:43:16 +0000 Subject: 2.1.5 testing In-Reply-To: <87sjyfuqvz.fsf@qurzaw.varnish-software.com> Message-ID: Hi, On 03.12.10 10:50, "Tollef Fog Heen" wrote: > >Hi, > >We're getting close to 2.1.5 and I haven't received enough testing >reports. If people could test the tar.gz available in >http://varnish-cache.org/~tfheen/varnish-2.1.5pre1.tar.gz that would be >most welcome. Report back to me privately or on the list. Any specific testing needed? Anything to watch out for? If not, I'll just give it a try. We have a pretty straight forward setup of varnish at work (2 directors, health checks, nothing fancy). Cheers, Marian From tfheen at varnish-software.com Fri Dec 3 14:10:38 2010 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Fri, 03 Dec 2010 15:10:38 +0100 Subject: 2.1.5 testing In-Reply-To: (Marian Hettwer's message of "Fri, 3 Dec 2010 13:43:16 +0000") References: Message-ID: <87k4jrueu9.fsf@qurzaw.varnish-software.com> ]] "Hettwer, Marian" | Hi, | | | On 03.12.10 10:50, "Tollef Fog Heen" wrote: | | > | >Hi, | > | >We're getting close to 2.1.5 and I haven't received enough testing | >reports. If people could test the tar.gz available in | >http://varnish-cache.org/~tfheen/varnish-2.1.5pre1.tar.gz that would be | >most welcome. Report back to me privately or on the list. | | Any specific testing needed? Anything to watch out for? The high-profile bugs from 2.1.4 that I want to make sure are fixed are requests with If-Modified-Since together with pass, that those don't generate any kinds of timeouts, as well as well as making sure if a backend sets Content-Length and you pass, we don't add our own. In addition, whatever you can throw at it in terms of normal and irregular traffic is welcome. | If not, I'll just give it a try. We have a pretty straight forward setup | of varnish at work (2 directors, health checks, nothing fancy). Thanks! -- Tollef Fog Heen Varnish Software t: +47 21 98 92 64 From alteriks at gmail.com Sat Dec 4 15:35:21 2010 From: alteriks at gmail.com (Krzysztof Dajka) Date: Sat, 4 Dec 2010 15:35:21 +0000 Subject: Problem with case-insensitive (?i) regex Message-ID: Hi, I'm trying to make case-insensitive regex to catch mobile browsers and redirect them to lightweight version of web page. I found this doc: http://www.varnish-cache.org/docs/2.1/faq/general.html?highlight=insensitive But using "?i" before regular expression makes VCL compiler complain: storage_malloc: max size 200 MB. Message from VCC-compiler: Regexp compilation error: nothing to repeat (/exp/config/rewrite_UA.vcl Line 2 Pos 30) if ( req.http.User-Agent ~ "?iOpera Mini" ) -----------------------------##############-- Running VCC-compiler failed, exit 1 VCL compilation failed Here is my vcl snippet: (If I remove "?i" in front of every regex compilation goes fine) sub vcl_recv { if ( req.http.User-Agent ~ "?iOpera Mini" || req.http.User-Agent ~ "?iSymbianOS" || req.http.User-Agent ~ "?iSymbian OS" || req.http.User-Agent ~ "?iWindows CE" || req.http.User-Agent ~ "?iBlackBerry" || req.http.User-Agent ~ "?iNetFront" || req.http.User-Agent ~ "?iPalm OS" || req.http.User-Agent ~ "?iBlazer" || req.http.User-Agent ~ "?iElaine" || req.http.User-Agent ~ "?i^WAP" || req.http.User-Agent ~ "?iPlucker" || req.http.User-Agent ~ "?iiPhone" || req.http.User-Agent ~ "?iiPod" || req.http.User-Agent ~ "?iAndroid" || req.http.User-Agent ~ "?iMIDP" ) { error 701; } } From flylm.ml at gmail.com Sat Dec 4 15:44:25 2010 From: flylm.ml at gmail.com (FabD [ML]) Date: Sat, 4 Dec 2010 16:44:25 +0100 Subject: About Varnish cache delivery Message-ID: Hi, I've a question about a case : 1) Imagine Varnish have a page in cache, but a page that is not compressed. 2) If I call this page with a client that supports gzip encoding, varnish ask backend for a gzip encoded page or send the non-encoded page from cache ? Actually, i notice sometime I receive some "non-encoded" page, I would like to understant why. ps : Accept-encoding header is normalized. Thanks, Fabien From cd at sentia.nl Sat Dec 4 16:29:06 2010 From: cd at sentia.nl (Camiel Dobbelaar) Date: Sat, 04 Dec 2010 17:29:06 +0100 Subject: Problem with case-insensitive (?i) regex In-Reply-To: References: Message-ID: <4CFA6C52.5080007@sentia.nl> On 4-12-2010 16:35, Krzysztof Dajka wrote: > I'm trying to make case-insensitive regex to catch mobile browsers and > redirect them to lightweight version of web page. > I found this doc: > http://www.varnish-cache.org/docs/2.1/faq/general.html?highlight=insensitive > > But using "?i" before regular expression makes VCL compiler complain: I fell into the same trap, because the example in the FAQ is wrong. The surrounding FAQ text has it correct. (shame on me for not reporting it) The brackets are mandatory, so you have to put "(?i)" at the start. -- Cam From ionathan at gmail.com Sat Dec 4 16:51:23 2010 From: ionathan at gmail.com (Jonathan Leibiusky) Date: Sat, 4 Dec 2010 13:51:23 -0300 Subject: About Varnish cache delivery In-Reply-To: References: Message-ID: If you have a Vary header like this (or something like this): Vary: Accept-Encoding And your request has a header like this: Accept-Encoding: gzip What will happen is that varnish won't find in the cache the resource you are looking for with this encoding, so it will go to the backend and ask for it. I believe you can play with vcl and change this behavior. Although you should be careful. Jonathan On Sat, Dec 4, 2010 at 12:44 PM, FabD [ML] wrote: > Hi, > > I've a question about a case : > > 1) Imagine Varnish have a page in cache, but a page that is not compressed. > 2) If I call this page with a client that supports gzip encoding, > varnish ask backend for a gzip encoded page or send the non-encoded > page from cache ? > > Actually, i notice sometime I receive some "non-encoded" page, I would > like to understant why. > > ps : Accept-encoding header is normalized. > > Thanks, > Fabien > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alteriks at gmail.com Sat Dec 4 19:24:04 2010 From: alteriks at gmail.com (Krzysztof Dajka) Date: Sat, 4 Dec 2010 19:24:04 +0000 Subject: Problem with case-insensitive (?i) regex In-Reply-To: <4CFA6C52.5080007@sentia.nl> References: <4CFA6C52.5080007@sentia.nl> Message-ID: Thanks a lot. It would be nice if information about usage of case-insensitive regex could be put into VCL(7) On Sat, Dec 4, 2010 at 4:29 PM, Camiel Dobbelaar wrote: > > On 4-12-2010 16:35, Krzysztof Dajka wrote: >> I'm trying to make case-insensitive regex to catch mobile browsers and >> redirect them to lightweight version of web page. >> I found this doc: >> http://www.varnish-cache.org/docs/2.1/faq/general.html?highlight=insensitive >> >> But using "?i" before regular expression makes VCL compiler complain: > > I fell into the same trap, because the example in the FAQ is wrong. ?The > surrounding FAQ text has it correct. ?(shame on me for not reporting it) > > The brackets are mandatory, so you have to put "(?i)" at the start. > > -- > Cam > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > From perbu at varnish-software.com Sat Dec 4 20:43:24 2010 From: perbu at varnish-software.com (Per Buer) Date: Sat, 4 Dec 2010 21:43:24 +0100 Subject: Problem with case-insensitive (?i) regex In-Reply-To: References: <4CFA6C52.5080007@sentia.nl> Message-ID: Something like this: I'm not really familiar with flags to the pcre engine so I need some help with this. Is this correct? --- doc/sphinx/reference/vcl.rst (revision 5639) +++ doc/sphinx/reference/vcl.rst (working copy) @@ -288,6 +288,22 @@ return (pipe); } +Regular Expressions +------------------- + +In Varnish 2.1.0 Varnish switched to using PCRE - Perl-compatible +regular expressions. For a complete description of PCRE please see the +PCRE(3) man page. + +To send flags to the PCRE engine, such as to turn on *case +insensitivity* add the flag within parens following a question mark, +like this::: + + if (req.http.host ~ "(?i)example.com$") { + ... + } + + On Sat, Dec 4, 2010 at 8:24 PM, Krzysztof Dajka wrote: > Thanks a lot. > It would be nice if ?information about usage of case-insensitive regex > could be put into VCL(7) > > On Sat, Dec 4, 2010 at 4:29 PM, Camiel Dobbelaar wrote: >> >> On 4-12-2010 16:35, Krzysztof Dajka wrote: >>> I'm trying to make case-insensitive regex to catch mobile browsers and >>> redirect them to lightweight version of web page. >>> I found this doc: >>> http://www.varnish-cache.org/docs/2.1/faq/general.html?highlight=insensitive >>> >>> But using "?i" before regular expression makes VCL compiler complain: >> >> I fell into the same trap, because the example in the FAQ is wrong. ?The >> surrounding FAQ text has it correct. ?(shame on me for not reporting it) >> >> The brackets are mandatory, so you have to put "(?i)" at the start. >> >> -- >> Cam >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc -- Per Buer,?Varnish Software Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer Varnish makes websites fly! From tfheen at varnish-software.com Mon Dec 6 07:34:25 2010 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Mon, 06 Dec 2010 08:34:25 +0100 Subject: Problem with case-insensitive (?i) regex In-Reply-To: (Per Buer's message of "Sat, 4 Dec 2010 21:43:24 +0100") References: <4CFA6C52.5080007@sentia.nl> Message-ID: <874oartkvy.fsf@qurzaw.varnish-software.com> ]] Per Buer Hi, | +In Varnish 2.1.0 Varnish switched to using PCRE - Perl-compatible | +regular expressions. For a complete description of PCRE please see the | +PCRE(3) man page. You probably want the pcresyntax(3) or pcrepattern(3) man page. | +To send flags to the PCRE engine, such as to turn on *case | +insensitivity* add the flag within parens following a question mark, | +like this::: | + | + if (req.http.host ~ "(?i)example.com$") { | + ... | + } Yup, looks good to me. -- Tollef Fog Heen Varnish Software t: +47 21 98 92 64 From ross at trademe.co.nz Tue Dec 7 02:10:21 2010 From: ross at trademe.co.nz (Ross Brown) Date: Tue, 7 Dec 2010 15:10:21 +1300 Subject: Chasing 503 Guru's In-Reply-To: References: Message-ID: <1FF67D7369ED1A45832180C7C1109BCA28BB9C7A90@tmmail0.trademe.local> -----Original Message----- From: varnish-misc-bounces at varnish-cache.org [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Simon Males Sent: Friday, 3 December 2010 3:30 a.m. To: Per Buer Cc: varnish-misc at projects.linpro.no Subject: Re: Chasing 503 Guru's >>> Or is it common practice to write varnishlog to disk in production environments? >> >> No. On Linux the vm get's slightly dazzled by all the IO and will >> behave strangely. You can instruct varnishlog to only log 503's. That >> might be ideal for you. > I'm not sure how to do this. The closest I get is: > varnishlog -o -i TxStatus -I 503 > > Though that doesn't return the whole entry. You could try something like this, which takes inspiration from something I found on Tollef's blog ( http://err.no/personal/blog/2008/Dec/17#2008-12-17-10-14_poor_mans_filtering_language ) varnishlog -o | perl -ne 'BEGIN { $/ = "";} print if (/TxStatus.*(50\d)/);' -------------- next part -------------- An HTML attachment was scrubbed... URL: From traian.bratucu at eea.europa.eu Tue Dec 7 10:36:58 2010 From: traian.bratucu at eea.europa.eu (Traian Bratucu) Date: Tue, 7 Dec 2010 11:36:58 +0100 Subject: varnish default retries on backend timeout? Message-ID: Hello, We have been using Varnish for a while and encountered a specific problem I don't seem to be able to figure out. Apparently on a POST request to the backend, for which the backend times out, the request is automatically retried by varnish a second time. Obviously that is a big problem since POST data gets submitted twice. I do not have any restart statements in the VCL so I guess this is some default Varnish behavior, hopefully controlled by some global parameter. Here is what I get logged: ------------- 32 Backend c 5 xxx_authenticated black_instance_1 32 FetchError c http first read error: -1 11 (Resource temporarily unavailable) 32 Backend c 87 xxx_authenticated gray_instance_2 32 FetchError c http first read error: -1 11 (Resource temporarily unavailable) 32 VCL_call c error deliver 32 VCL_call c deliver deliver 32 TxProtocol c HTTP/1.1 32 TxStatus c 503 ------------- Is there a way to avoid this double post? Thank you, Traian -------------- next part -------------- An HTML attachment was scrubbed... URL: From jelder at locamoda.com Wed Dec 8 01:43:21 2010 From: jelder at locamoda.com (Jacob Elder) Date: Tue, 7 Dec 2010 20:43:21 -0500 Subject: Display bug in varnishstat Message-ID: (Sorry if I'm not following protocol; I don't see where to submit this on the site.) The varnishstat hitrate average header will display "nan" instead of the expected information if the user resizes the terminal window, or if the user presses any of the arrow keys. I am running the official debs for 2.1.2-1~hardy1 on amd64 and running varnishstat over SSH from Terminal.app. -- Jacob Elder @jelder (646) 535-3379 -------------- next part -------------- An HTML attachment was scrubbed... URL: From lars at renoz.dk Wed Dec 8 13:24:37 2010 From: lars at renoz.dk (Lars Wolff) Date: Wed, 08 Dec 2010 14:24:37 +0100 Subject: Varnish stripping Cache-Control header Message-ID: <4CFF8715.4030509@renoz.dk> Hi, I have a problem with Varnish 2.1.4 stripping the Cache-Control header, which results in the client caching pages, that are not supposed to be cached (or to be more precise - locally storing them, allowing the back button to be served without a hit to the server). The header looks like this (should be standard PHP session Cache-control). Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0 I'm running FreeBSD, and this problem has been verified with the default.vcl. Steps: 1. Restart varnishd 2. Hit the page - this time the Cache-Control header will be served to the client 3. Hit the page again - this time the Cache-Control header will be stripped by varnish. From now on it will be stripped. varnishlog -b still reports: ... 10 RxHeader b Set-Cookie: PHPSESSID=ba69a435445f9d1b617951cb2ef00d35; path=/ 10 RxHeader b Expires: Thu, 19 Nov 1981 08:52:00 GMT 10 RxHeader b Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0 10 RxHeader b Pragma: no-cache ... But varnishlog -c reports: ... 9 TxHeader c Set-Cookie: PHPSESSID=1bf083ca11d321412cce8155e593fe86; path=/ 9 TxHeader c Expires: Thu, 19 Nov 1981 08:52:00 GMT 9 TxHeader c Pragma: no-cache ... The result is not cached in varnish due to the Cookie - which is exactly as it should be. First time after the restart varnishlog -c reports ... 9 VCL_call c recv 9 VCL_return c lookup 9 VCL_call c hash 9 VCL_return c hash 9 VCL_call c miss 9 VCL_return c fetch 9 Backend c 10 default default 9 TTL c 1643766043 RFC 0 1291814342 1291814342 375007920 0 0 9 VCL_call c fetch 9 VCL_return c pass ... 9 VCL_call c deliver 9 VCL_return c deliver ... But after the first requests it reports: ... 9 VCL_call c recv 9 VCL_return c lookup 9 VCL_call c hash 9 VCL_return c hash 9 HitPass c 1643766043 9 VCL_call c pass 9 VCL_return c pass ... 9 VCL_call c fetch 9 VCL_return c pass ... 9 VCL_call c deliver 9 VCL_return c deliver ... So its cached as a pass - fine, but this is where I dont understand whats happening. It fetches the page corrently from the server, and then removed the Cache-Control header. Why? This was not a problem in the past - I think it was version 2.1.2 or 2.1.3 when it worked, so I have a feeling that this might be a bug. Best regards Lars -------------- next part -------------- An HTML attachment was scrubbed... URL: From jelder at locamoda.com Wed Dec 8 23:23:34 2010 From: jelder at locamoda.com (Jacob Elder) Date: Wed, 8 Dec 2010 18:23:34 -0500 Subject: What is "herding?" Message-ID: I see this in varnishlog all the time and I've never been able to find any definitive explanation of what this means. 22 SessionOpen c 108.7.221.151 61656 :80 22 Debug c *herding* 22 Interrupted c ReqStart -- Jacob Elder @jelder (646) 535-3379 -------------- next part -------------- An HTML attachment was scrubbed... URL: From phk at phk.freebsd.dk Thu Dec 9 08:48:07 2010 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Thu, 09 Dec 2010 08:48:07 +0000 Subject: What is "herding?" In-Reply-To: Your message of "Wed, 08 Dec 2010 18:23:34 EST." Message-ID: <5882.1291884487@critter.freebsd.dk> In message , Jaco b Elder writes: > 22 SessionOpen c 108.7.221.151 61656 :80 > 22 Debug c *herding* > 22 Interrupted c ReqStart It's a debug message that I forgot to remove, it indicates that the thread pool sizes are being examined. Don't worry about it. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From albert at merge.nl Thu Dec 9 10:36:27 2010 From: albert at merge.nl (Albert Skibinski) Date: Thu, 9 Dec 2010 11:36:27 +0100 Subject: very long page load on chrome after POST Message-ID: I've got server running latest varnish 2.1.4 on debian/apache with mod-php and everything is working fine for sites who are using the reverse proxy. (in my case some drupal sites) Other drupal sites who are not using the cache work fine, but with two annoying glitches: 1. in chrome, a page load after a POST action takes forever (45 seconds average). This does not happen in other browsers. 2. A page refresh (F5) on the sites with no cache also takes about that time (in all browers) I did a lot of research and I'm pretty sure it's not the VCL. I'm using a vcl comparable with something like http://highervisibilitywebsites.com/example-varnish-vcl-drupal-pressflow-site I also found this http://www.gossamer-threads.com/lists/varnish/misc/16808 which looks related to #1? Maybe these are known issues which are fixed in 2.1.5? --- Albert Skibinski +31 (0) 76 - 888 00 31 albert at merge.nl -------------- next part -------------- An HTML attachment was scrubbed... URL: From traian.bratucu at eea.europa.eu Thu Dec 9 10:48:16 2010 From: traian.bratucu at eea.europa.eu (Traian Bratucu) Date: Thu, 9 Dec 2010 11:48:16 +0100 Subject: varnish default retries on backend timeout? In-Reply-To: References: Message-ID: I found the issue myself, perhaps it will help others. The backend servers were using keep-alive with varnish, and whenever varnish got a timeout from the backend, it would retry the request exactly once (hardcoded) because the initial backend was marked as "recycled". Disabling keep-alive on the backend servers resolved my issue. Good luck, Traian From: varnish-misc-bounces at varnish-cache.org [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Traian Bratucu Sent: Tuesday, December 07, 2010 11:37 AM To: 'varnish-misc at varnish-cache.org' Subject: varnish default retries on backend timeout? Hello, We have been using Varnish for a while and encountered a specific problem I don't seem to be able to figure out. Apparently on a POST request to the backend, for which the backend times out, the request is automatically retried by varnish a second time. Obviously that is a big problem since POST data gets submitted twice. I do not have any restart statements in the VCL so I guess this is some default Varnish behavior, hopefully controlled by some global parameter. Here is what I get logged: ------------- 32 Backend c 5 xxx_authenticated black_instance_1 32 FetchError c http first read error: -1 11 (Resource temporarily unavailable) 32 Backend c 87 xxx_authenticated gray_instance_2 32 FetchError c http first read error: -1 11 (Resource temporarily unavailable) 32 VCL_call c error deliver 32 VCL_call c deliver deliver 32 TxProtocol c HTTP/1.1 32 TxStatus c 503 ------------- Is there a way to avoid this double post? Thank you, Traian -------------- next part -------------- An HTML attachment was scrubbed... URL: From ask at develooper.com Thu Dec 9 15:29:36 2010 From: ask at develooper.com (=?iso-8859-1?Q?Ask_Bj=F8rn_Hansen?=) Date: Thu, 9 Dec 2010 07:29:36 -0800 Subject: very long page load on chrome after POST In-Reply-To: References: Message-ID: <578973CC-787B-4BCB-B02E-0FA4C9E2E156@develooper.com> On Dec 9, 2010, at 2:36, Albert Skibinski wrote: > I also found this http://www.gossamer-threads.com/lists/varnish/misc/16808 which looks related to #1? > Maybe these are known issues which are fixed in 2.1.5? Yeah, sounds similar. I'd try 2.1.5 first. :-) From albert at merge.nl Thu Dec 9 19:51:03 2010 From: albert at merge.nl (Albert Skibinski) Date: Thu, 9 Dec 2010 20:51:03 +0100 Subject: very long page load on chrome after POST In-Reply-To: <578973CC-787B-4BCB-B02E-0FA4C9E2E156@develooper.com> References: <578973CC-787B-4BCB-B02E-0FA4C9E2E156@develooper.com> Message-ID: Actually, I solved this problem today by disabling Keep-Alive in Apache (saw the tip in another mailing) It seems Varnish 2.1.4 has a conflict with Keep-Alive. Both issues where resolved by this. Cheers, --- Albert Skibinski 076 - 888 00 31 *www.merge.nl* 2010/12/9 Ask Bj?rn Hansen > > On Dec 9, 2010, at 2:36, Albert Skibinski wrote: > > > I also found this > http://www.gossamer-threads.com/lists/varnish/misc/16808 which looks > related to #1? > > Maybe these are known issues which are fixed in 2.1.5? > > Yeah, sounds similar. I'd try 2.1.5 first. :-) -------------- next part -------------- An HTML attachment was scrubbed... URL: From ash2kk at gmail.com Fri Dec 10 12:57:35 2010 From: ash2kk at gmail.com (Mikhail Mazursky) Date: Fri, 10 Dec 2010 17:57:35 +0500 Subject: Problem with case-insensitive (?i) regex In-Reply-To: References: <4CFA6C52.5080007@sentia.nl> Message-ID: 2010/12/5 Per Buer : > +insensitivity* add the flag within parens following a question mark, Hi, i think you misspelled "parenTs". From kristian at varnish-software.com Fri Dec 10 13:13:33 2010 From: kristian at varnish-software.com (Kristian Lyngstol) Date: Fri, 10 Dec 2010 14:13:33 +0100 Subject: Parents and parens (was: Problem with case-insensitive (?i) regex) In-Reply-To: References: <4CFA6C52.5080007@sentia.nl> Message-ID: <20101210131333.GG1913@freud> On Fri, Dec 10, 2010 at 05:57:35PM +0500, Mikhail Mazursky wrote: > 2010/12/5 Per Buer : > > +insensitivity* add the flag within parens following a question mark, > > Hi, i think you misspelled "parenTs". ?Parentheses (singular, parenthesis) ? also called simply brackets (UK), or round brackets, curved brackets, oval brackets, or, colloquially, parens.? or: Parenthesis \Pa*ren"the*sis\ (p[.a]*r[e^]n"th[-e]*s[i^]s), n.; pl. {Parentheses}. [NL., fr. Gr. pare`nqesis, fr. parentiqe`nai to put in beside, insert; para` beside + 'en in + tiqe`nai to put, place. See {Para-}, {En-}, 2, and {Thesis}.] [1913 Webster] oh, and while I'm at it: Colloquial \Col*lo"qui*al\, a. [See {Colloqui}.] Pertaining to, or used in, conversation, esp. common and familiar conversation; conversational; hence, unstudied; informal; as, colloquial intercourse; colloquial phrases; a colloquial style. -- {Col*lo"qui*al*ly}, adv. [1913 Webster] (sorry, couldn't help myself). - Kristian From cooltechemail at gmail.com Fri Dec 10 14:08:26 2010 From: cooltechemail at gmail.com (Vincent) Date: Fri, 10 Dec 2010 06:08:26 -0800 Subject: remove/unset set-cookies Message-ID: Hi, I am trying to remove all set-cookies from the backend but I still found Set-Cookie in the log (varnishlog -b). I have the following setup: sub vcl_fetch { ..... ..... unset beresp.http.set-cookie; } and varnishlog -b shows something like this: 16 RxHeader b Set-Cookie: name=xyc; domain=.example.com; path=/; expires=Sun, 09-Jan-2011 14:04:10 GMT I am wondering how to completely remove the Set-Cookie. Currently my hitrate is only 80%, I think the set-cookie header is the main reason. also what's the difference between unset beresp.http.set-cookie; and remove beresp.http.set-cookie; Thank you, Vincent -------------- next part -------------- An HTML attachment was scrubbed... URL: From traian.bratucu at eea.europa.eu Fri Dec 10 14:12:43 2010 From: traian.bratucu at eea.europa.eu (Traian Bratucu) Date: Fri, 10 Dec 2010 15:12:43 +0100 Subject: remove/unset set-cookies In-Reply-To: References: Message-ID: Hello Vincent, You make a small confusion. The log (varnishlog -b) is showing you that the backend is sending cookies, not that varnish is sending the cookies to the client. The cookies are removed from the object before they are sent to the client. This means that those cookies will not reach the varnish client (usually user browser). So: Client asks url -> varnish receives request, forwards to backend -> backend responds to varnish with set-cookie -> varnish strips cookie (unset beresp.http.set-cookie) and sends the response back to the client Hope it is clear now for you. Traian From: varnish-misc-bounces at varnish-cache.org [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Vincent Sent: Friday, December 10, 2010 3:08 PM To: varnish-misc at varnish-cache.org Subject: remove/unset set-cookies Hi, I am trying to remove all set-cookies from the backend but I still found Set-Cookie in the log (varnishlog -b). I have the following setup: sub vcl_fetch { ..... ..... unset beresp.http.set-cookie; } and varnishlog -b shows something like this: 16 RxHeader b Set-Cookie: name=xyc; domain=.example.com; path=/; expires=Sun, 09-Jan-2011 14:04:10 GMT I am wondering how to completely remove the Set-Cookie. Currently my hitrate is only 80%, I think the set-cookie header is the main reason. also what's the difference between unset beresp.http.set-cookie; and remove beresp.http.set-cookie; Thank you, Vincent -------------- next part -------------- An HTML attachment was scrubbed... URL: From cooltechemail at gmail.com Fri Dec 10 14:17:34 2010 From: cooltechemail at gmail.com (Vincent) Date: Fri, 10 Dec 2010 06:17:34 -0800 Subject: remove/unset set-cookies In-Reply-To: References: Message-ID: Hi Traian, Thank you!! Vincent On Fri, Dec 10, 2010 at 6:12 AM, Traian Bratucu < traian.bratucu at eea.europa.eu> wrote: > Hello Vincent, > > > > You make a small confusion. The log (varnishlog ?b) is showing you that the > backend is sending cookies, not that varnish is sending the cookies to the > client. > > The cookies are removed from the object before they are sent to the client. > This means that those cookies will not reach the varnish client (usually > user browser). > > > > So: > > > > Client asks url -> varnish receives request, forwards to backend -> backend > responds to varnish *with* set-cookie -> varnish strips cookie (unset > beresp.http.set-cookie) and sends the response back to the client > > > > Hope it is clear now for you. > > > > Traian > > > > *From:* varnish-misc-bounces at varnish-cache.org [mailto: > varnish-misc-bounces at varnish-cache.org] *On Behalf Of *Vincent > *Sent:* Friday, December 10, 2010 3:08 PM > *To:* varnish-misc at varnish-cache.org > *Subject:* remove/unset set-cookies > > > > Hi, > > I am trying to remove all set-cookies from the backend but I still found > Set-Cookie in the log (varnishlog -b). > > I have the following setup: > > sub vcl_fetch { > ..... > ..... > unset beresp.http.set-cookie; > } > > and varnishlog -b shows something like this: > > 16 RxHeader b Set-Cookie: name=xyc; domain=.example.com; path=/; > expires=Sun, 09-Jan-2011 14:04:10 GMT > > I am wondering how to completely remove the Set-Cookie. Currently my > hitrate is only 80%, I think the set-cookie header is the main reason. > > also what's the difference between > unset beresp.http.set-cookie; > and > remove beresp.http.set-cookie; > > > Thank you, > > Vincent > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vburshteyn at broadway.com Fri Dec 10 17:36:00 2010 From: vburshteyn at broadway.com (Vitaly Burshteyn) Date: Fri, 10 Dec 2010 12:36:00 -0500 Subject: cookies In-Reply-To: References: Message-ID: <0F69574F9901D4459C6B75C9FF64FBC40538F7EC@mxfl01.hollywoodmedia.corp> Hi folks, We use cookies to test if it's a returning visitor or not. Is there any way to force a lookup on varnish if a specific cookie is present? Thanks ____________________________________ The information contained in this transmission may contain privileged and confidential information. It is intended only for the use of the person(s) named above. If you are not the intended recipient, you are hereby notified that any review, dissemination, distribution or duplication of this communication is strictly prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. From armdan20 at gmail.com Mon Dec 13 08:46:39 2010 From: armdan20 at gmail.com (andan andan) Date: Mon, 13 Dec 2010 09:46:39 +0100 Subject: cookies In-Reply-To: <0F69574F9901D4459C6B75C9FF64FBC40538F7EC@mxfl01.hollywoodmedia.corp> References: <0F69574F9901D4459C6B75C9FF64FBC40538F7EC@mxfl01.hollywoodmedia.corp> Message-ID: 2010/12/10 Vitaly Burshteyn : > Hi folks, > > We use cookies to test if it's a returning visitor or not. ?Is there any > way to force a lookup on varnish if a specific cookie is present? http://www.varnish-cache.org/docs/2.1/reference/vcl.html Look at "..how to force Varnish to cache documents even when cookies are present::" From tom.coady at gmail.com Mon Dec 13 10:47:47 2010 From: tom.coady at gmail.com (Tom Coady) Date: Mon, 13 Dec 2010 10:47:47 +0000 Subject: Server Not Responding Message-ID: Hello I installed varnish using debian apt-get, checked etc/varnish/default.vcl, killed and restarted varnish, but I get no response on ports 8080 and 2000. I've been through the troubleshooting without finding anything; varnishlog returns 0 CLI Rd ping 0 CLI Wr 0 200 PONG 1292235717 but nothing when loading pages on 8080 I think I only have 760Mb memory, so wonder if using the default 1G is what's preventing it from working? Nothing much in var/log/messages or syslog. Tom -------------- next part -------------- An HTML attachment was scrubbed... URL: From traian.bratucu at eea.europa.eu Mon Dec 13 10:54:00 2010 From: traian.bratucu at eea.europa.eu (Traian Bratucu) Date: Mon, 13 Dec 2010 11:54:00 +0100 Subject: Server Not Responding In-Reply-To: References: Message-ID: Hi Tom, I think your varnish is on port 6081. Do a nestat or something to see. I don?t know exactly where the default startup conf is on debian, maybe /etc/default/varnish? You can check it out to change the port. Traian From: varnish-misc-bounces at varnish-cache.org [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Tom Coady Sent: Monday, December 13, 2010 11:48 AM To: varnish-misc at varnish-cache.org Subject: Server Not Responding Hello I installed varnish using debian apt-get, checked etc/varnish/default.vcl, killed and restarted varnish, but I get no response on ports 8080 and 2000. I've been through the troubleshooting without finding anything; varnishlog returns 0 CLI Rd ping 0 CLI Wr 0 200 PONG 1292235717 but nothing when loading pages on 8080 I think I only have 760Mb memory, so wonder if using the default 1G is what's preventing it from working? Nothing much in var/log/messages or syslog. Tom -------------- next part -------------- An HTML attachment was scrubbed... URL: From tom.coady at gmail.com Mon Dec 13 11:02:25 2010 From: tom.coady at gmail.com (Tom Coady) Date: Mon, 13 Dec 2010 11:02:25 +0000 Subject: Server Not Responding In-Reply-To: References: Message-ID: Thanks Traian! $ sudo varnishd -f /etc/varnish/default.vcl -s malloc,1G -T 127.0.0.1:2000-a 0.0.0.0:8080 Using old SHMFILE $ more /etc/varnish/default.vcl backend default { set backend.host = "127.0.0.1"; set backend.port = "8080"; } $ netstat Active Internet connections (w/o servers) Proto Recv-Q Send-Q Local Address Foreign Address State tcp 0 256 ip-10-48-59-119.eu-:ssh 78-105-154-24.zon:54973 ESTABLISHED tcp 0 0 ip-10-48-59-119.e:49212 subtshirt.com:www TIME_WAIT tcp 0 0 ip-10-48-59-119.e:49213 subtshirt.com:www TIME_WAIT tcp 0 0 ip-10-48-59-119.e:49214 subtshirt.com:www TIME_WAIT tcp 0 0 ip-10-48-59-119.e:49215 subtshirt.com:www TIME_WAIT tcp 0 0 ip-10-48-59-119.e:49208 subtshirt.com:www TIME_WAIT tcp 0 0 ip-10-48-59-119.e:49209 subtshirt.com:www TIME_WAIT tcp 0 0 ip-10-48-59-119.e:49210 subtshirt.com:www TIME_WAIT tcp 0 0 ip-10-48-59-119.e:49211 subtshirt.com:www TIME_WAIT tcp 0 0 ip-10-48-59-119.e:49204 subtshirt.com:www TIME_WAIT tcp 0 0 ip-10-48-59-119.e:49205 subtshirt.com:www TIME_WAIT tcp 0 0 ip-10-48-59-119.e:49206 subtshirt.com:www TIME_WAIT tcp 0 0 ip-10-48-59-119.e:49207 subtshirt.com:www TIME_WAIT tcp 0 0 ip-10-48-59-119.e:49200 subtshirt.com:www TIME_WAIT tcp 0 0 ip-10-48-59-119.e:49201 subtshirt.com:www TIME_WAIT tcp 0 0 ip-10-48-59-119.e:49202 subtshirt.com:www TIME_WAIT tcp 0 0 ip-10-48-59-119.e:49203 subtshirt.com:www TIME_WAIT tcp 0 32 ip-10-48-59-119.eu-:ssh customer-static-2:54665 ESTABLISHED tcp6 0 0 ip-10-48-59-119.eu-:www t332.tiggee.net:60435 TIME_WAIT Active UNIX domain sockets (w/o servers) Proto RefCnt Flags Type State I-Node Path unix 2 [ ] DGRAM 657 @/org/kernel/udev/udevd unix 4 [ ] DGRAM 846770 /dev/log unix 3 [ ] STREAM CONNECTED 913898 unix 3 [ ] STREAM CONNECTED 913897 unix 2 [ ] DGRAM 913067 unix 3 [ ] STREAM CONNECTED 880234 unix 3 [ ] STREAM CONNECTED 880233 unix 2 [ ] DGRAM 880232 unix 3 [ ] STREAM CONNECTED 678682 unix 3 [ ] STREAM CONNECTED 678681 unix 3 [ ] STREAM CONNECTED 678426 unix 3 [ ] STREAM CONNECTED 678425 unix 3 [ ] STREAM CONNECTED 678424 unix 3 [ ] STREAM CONNECTED 678423 unix 2 [ ] DGRAM 2553 On Mon, Dec 13, 2010 at 10:54, Traian Bratucu wrote: > Hi Tom, > > > > I think your varnish is on port 6081. Do a nestat or something to see. > > I don?t know exactly where the default startup conf is on debian, maybe > /etc/default/varnish? You can check it out to change the port. > > > > Traian > > > > *From:* varnish-misc-bounces at varnish-cache.org [mailto: > varnish-misc-bounces at varnish-cache.org] *On Behalf Of *Tom Coady > *Sent:* Monday, December 13, 2010 11:48 AM > *To:* varnish-misc at varnish-cache.org > *Subject:* Server Not Responding > > > > Hello > > I installed varnish using debian apt-get, checked etc/varnish/default.vcl, > killed and restarted varnish, but I get no response on ports 8080 and 2000. > > I've been through the troubleshooting without finding anything; varnishlog > returns > > 0 CLI Rd ping > > 0 CLI Wr 0 200 PONG 1292235717 > > but nothing when loading pages on 8080 > > > > I think I only have 760Mb memory, so wonder if using the default 1G is > what's preventing it from working? > > Nothing much in var/log/messages or syslog. > > Tom > -------------- next part -------------- An HTML attachment was scrubbed... URL: From perbu at varnish-software.com Mon Dec 13 11:12:04 2010 From: perbu at varnish-software.com (Per Buer) Date: Mon, 13 Dec 2010 12:12:04 +0100 Subject: Server Not Responding In-Reply-To: References: Message-ID: Hi. You're asking Varnish to connect to itself. Kind of a short circuit. You want to change your listening port to 80. Per. On Mon, Dec 13, 2010 at 12:02 PM, Tom Coady wrote: > Thanks Traian! > $ sudo varnishd -f /etc/varnish/default.vcl -s malloc,1G -T 127.0.0.1:2000 > -a 0.0.0.0:8080 > Using old SHMFILE > $ more?/etc/varnish/default.vcl > backend default { > set backend.host = "127.0.0.1"; > set backend.port = "8080"; > } > $ netstat > Active Internet connections (w/o servers) > Proto Recv-Q Send-Q Local Address ? ? ? ? ? Foreign Address ? ? ? ? State > > tcp ? ? ? ?0 ? ?256 ip-10-48-59-119.eu-:ssh 78-105-154-24.zon:54973 > ESTABLISHED > tcp ? ? ? ?0 ? ? ?0 ip-10-48-59-119.e:49212 subtshirt.com:www > TIME_WAIT > tcp ? ? ? ?0 ? ? ?0 ip-10-48-59-119.e:49213 subtshirt.com:www > TIME_WAIT > tcp ? ? ? ?0 ? ? ?0 ip-10-48-59-119.e:49214 subtshirt.com:www > TIME_WAIT > tcp ? ? ? ?0 ? ? ?0 ip-10-48-59-119.e:49215 subtshirt.com:www > TIME_WAIT > tcp ? ? ? ?0 ? ? ?0 ip-10-48-59-119.e:49208 subtshirt.com:www > TIME_WAIT > tcp ? ? ? ?0 ? ? ?0 ip-10-48-59-119.e:49209 subtshirt.com:www > TIME_WAIT > tcp ? ? ? ?0 ? ? ?0 ip-10-48-59-119.e:49210 subtshirt.com:www > TIME_WAIT > tcp ? ? ? ?0 ? ? ?0 ip-10-48-59-119.e:49211 subtshirt.com:www > TIME_WAIT > tcp ? ? ? ?0 ? ? ?0 ip-10-48-59-119.e:49204 subtshirt.com:www > TIME_WAIT > tcp ? ? ? ?0 ? ? ?0 ip-10-48-59-119.e:49205 subtshirt.com:www > TIME_WAIT > tcp ? ? ? ?0 ? ? ?0 ip-10-48-59-119.e:49206 subtshirt.com:www > TIME_WAIT > tcp ? ? ? ?0 ? ? ?0 ip-10-48-59-119.e:49207 subtshirt.com:www > TIME_WAIT > tcp ? ? ? ?0 ? ? ?0 ip-10-48-59-119.e:49200 subtshirt.com:www > TIME_WAIT > tcp ? ? ? ?0 ? ? ?0 ip-10-48-59-119.e:49201 subtshirt.com:www > TIME_WAIT > tcp ? ? ? ?0 ? ? ?0 ip-10-48-59-119.e:49202 subtshirt.com:www > TIME_WAIT > tcp ? ? ? ?0 ? ? ?0 ip-10-48-59-119.e:49203 subtshirt.com:www > TIME_WAIT > tcp ? ? ? ?0 ? ? 32 ip-10-48-59-119.eu-:ssh customer-static-2:54665 > ESTABLISHED > tcp6 ? ? ? 0 ? ? ?0 ip-10-48-59-119.eu-:www t332.tiggee.net:60435 > TIME_WAIT > Active UNIX domain sockets (w/o servers) > Proto RefCnt Flags ? ? ? Type ? ? ? State ? ? ? ? I-Node ? Path > unix ?2 ? ? ?[ ] ? ? ? ? DGRAM ? ? ? ? ? ? ? ? ? ?657 > ?@/org/kernel/udev/udevd > unix ?4 ? ? ?[ ] ? ? ? ? DGRAM ? ? ? ? ? ? ? ? ? ?846770 ? /dev/log > unix ?3 ? ? ?[ ] ? ? ? ? STREAM ? ? CONNECTED ? ? 913898 > unix ?3 ? ? ?[ ] ? ? ? ? STREAM ? ? CONNECTED ? ? 913897 > unix ?2 ? ? ?[ ] ? ? ? ? DGRAM ? ? ? ? ? ? ? ? ? ?913067 > unix ?3 ? ? ?[ ] ? ? ? ? STREAM ? ? CONNECTED ? ? 880234 > unix ?3 ? ? ?[ ] ? ? ? ? STREAM ? ? CONNECTED ? ? 880233 > unix ?2 ? ? ?[ ] ? ? ? ? DGRAM ? ? ? ? ? ? ? ? ? ?880232 > unix ?3 ? ? ?[ ] ? ? ? ? STREAM ? ? CONNECTED ? ? 678682 > unix ?3 ? ? ?[ ] ? ? ? ? STREAM ? ? CONNECTED ? ? 678681 > unix ?3 ? ? ?[ ] ? ? ? ? STREAM ? ? CONNECTED ? ? 678426 > unix ?3 ? ? ?[ ] ? ? ? ? STREAM ? ? CONNECTED ? ? 678425 > unix ?3 ? ? ?[ ] ? ? ? ? STREAM ? ? CONNECTED ? ? 678424 > unix ?3 ? ? ?[ ] ? ? ? ? STREAM ? ? CONNECTED ? ? 678423 > unix ?2 ? ? ?[ ] ? ? ? ? DGRAM ? ? ? ? ? ? ? ? ? ?2553 > On Mon, Dec 13, 2010 at 10:54, Traian Bratucu > wrote: >> >> Hi Tom, >> >> >> >> I think your varnish is on port 6081. Do a nestat or something to see. >> >> I don?t know exactly where the default startup conf is on debian, maybe >> /etc/default/varnish? You can check it out to change the port. >> >> >> >> Traian >> >> >> >> From: varnish-misc-bounces at varnish-cache.org >> [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Tom Coady >> Sent: Monday, December 13, 2010 11:48 AM >> To: varnish-misc at varnish-cache.org >> Subject: Server Not Responding >> >> >> >> Hello >> >> I installed varnish using debian apt-get, checked?etc/varnish/default.vcl, >> killed and restarted varnish, but I get no response on ports 8080 and 2000. >> >> I've been through the troubleshooting without finding anything; varnishlog >> returns >> >> ?? ?0 CLI ? ? ? ? ? ?Rd ping >> >> ?? ?0 CLI ? ? ? ? ? ?Wr 0 200 PONG 1292235717 >> >> but nothing when loading pages on 8080 >> >> >> >> I think I only have 760Mb memory, so wonder if using the default 1G is >> what's preventing it from working? >> >> Nothing much in var/log/messages or syslog. >> >> Tom > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -- Per Buer,?Varnish Software Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer Varnish makes websites fly! From kristian at varnish-software.com Mon Dec 13 11:15:22 2010 From: kristian at varnish-software.com (Kristian Lyngstol) Date: Mon, 13 Dec 2010 12:15:22 +0100 Subject: Server Not Responding In-Reply-To: References: Message-ID: <20101213111521.GB4000@freud> On Mon, Dec 13, 2010 at 11:02:25AM +0000, Tom Coady wrote: > $ sudo varnishd -f /etc/varnish/default.vcl -s malloc,1G -T 127.0.0.1:2000-a > 0.0.0.0:8080 You are missing a space between -T and -a. Also, you should be using the normal startup scripts instead of executing varnishd directly, then edit /etc/default/varnish (or /etc/sysconfig/varnish for redhat-like systems). > $ netstat You need to use -l (or better yet: -nlpt) to get anything useful for this scenario. Ie: netstat -nlpt. - Kristian PS: It's easier to follow your thread if you don't top-post. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: Digital signature URL: From pom at dmsp.de Mon Dec 13 11:38:31 2010 From: pom at dmsp.de (Stefan Pommerening) Date: Mon, 13 Dec 2010 12:38:31 +0100 Subject: Memory usage of varnish 2.1.3 Message-ID: <4D0605B7.5030309@dmsp.de> Hello, we are using varnish 2.1.3 in a production environment after upgrading from varnish 2.0.6. Varnish is running on several linux blades (suse 10.3) with more than 20 gb of ram each. The former configuration using varnish 2.0.6 was not using the -s parameter for memory configuration. Because it seems to be a bit complicated to find documentation exactly on this topic for different varnish releases I hope that someone from this list can point me in the right direction. Whilst we had absolutely no memory problems with varnish 2.0.6 the new varnish release (2.1.3) seems to behave different than 2.0.6. Currently (with 2.1.3) varnish is running without -s parameter like before (2.0.6). Nevertheless it creates its shared memory file in /usr/lib/varnish// and fills up more and more space in the filesystem. For now we are changing the configuration to use "-s malloc,5G" for each instance in the beginning and watch memory consumption and varnishstat (memory usage and of course nuked objects). My questions is: How is varnish behaving if I do NOT use the -s parameter at all (varnish 2.0.6 compared to 2.1.3 - any differences?) Thanks in advance, Stefan From tom.coady at gmail.com Mon Dec 13 12:00:41 2010 From: tom.coady at gmail.com (Tom Coady) Date: Mon, 13 Dec 2010 12:00:41 +0000 Subject: Server Not Responding In-Reply-To: <20101213111521.GB4000@freud> References: <20101213111521.GB4000@freud> Message-ID: On Mon, Dec 13, 2010 at 11:15, Kristian Lyngstol < kristian at varnish-software.com> wrote: > On Mon, Dec 13, 2010 at 11:02:25AM +0000, Tom Coady wrote: > > $ sudo varnishd -f /etc/varnish/default.vcl -s malloc,1G -T > 127.0.0.1:2000-a > > 0.0.0.0:8080 > > You are missing a space between -T and -a. > Sorry, I don't know why the spaces got knocked out, they're there in the history. > Also, you should be using the normal startup scripts instead of executing > varnishd directly, then edit /etc/default/varnish (or > /etc/sysconfig/varnish for redhat-like systems). > Isn't that what I did by calling -f /etc/varnish/default.vcl ? > > $ netstat > > You need to use -l (or better yet: -nlpt) to get anything useful for this > scenario. Ie: netstat -nlpt. Sorry, I was thinking it was lacking something useful! $ sudo netstat -nlptActive Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 127.0.0.1:3306 0.0.0.0:* LISTEN 1432/mysqld tcp 0 0 0.0.0.0:8080 0.0.0.0:* LISTEN 30261/varnishd tcp 0 0 127.0.0.1:2000 0.0.0.0:* LISTEN 30261/varnishd tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1355/sshd tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 1759/exim4 tcp6 0 0 :::80 :::* LISTEN 1824/apache2 tcp6 0 0 :::22 :::* LISTEN 1355/sshd tcp6 0 0 :::443 :::* LISTEN 1824/apache2 -------------- next part -------------- An HTML attachment was scrubbed... URL: From tom.coady at gmail.com Mon Dec 13 12:03:28 2010 From: tom.coady at gmail.com (Tom Coady) Date: Mon, 13 Dec 2010 12:03:28 +0000 Subject: Server Not Responding In-Reply-To: References: Message-ID: On Mon, Dec 13, 2010 at 11:12, Per Buer wrote: > You're asking Varnish to connect to itself. Kind of a short circuit. > You want to change your listening port to 80. > Thanks Per I was thinking I would test 8080 first as it's a working server :-| Are you saying that 0.0.0.0 is incorrect in $ sudo varnishd -f /etc/varnish/default.vcl -s malloc,1G -T 127.0.0.1:2000 -a 0.0.0.0:8080 ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From mhettwer at team.mobile.de Mon Dec 13 12:12:10 2010 From: mhettwer at team.mobile.de (Hettwer, Marian) Date: Mon, 13 Dec 2010 12:12:10 +0000 Subject: Server Not Responding In-Reply-To: Message-ID: Hi Tom, On 13.12.10 12:02, "Tom Coady" > wrote: Thanks Traian! $ sudo varnishd -f /etc/varnish/default.vcl -s malloc,1G -T 127.0.0.1:2000 -a 0.0.0.0:8080 This means, varnish should listen on all interfaces on tcp/8080. $ more /etc/varnish/default.vcl backend default { set backend.host = "127.0.0.1"; set backend.port = "8080"; } That doesn't make sense. You tell varnish to use the backend at localhost 8080, where as seen above, its varnish. More likely is that you want to set the backend.port to 80 (as in a later reply of yours I can see that your apache2 is running on 80). Change it to 80 and varnish will be able to process the request (if nothing else is wrong) ;) Cheers, Marian From tom.coady at gmail.com Mon Dec 13 12:24:32 2010 From: tom.coady at gmail.com (Tom Coady) Date: Mon, 13 Dec 2010 12:24:32 +0000 Subject: Server Not Responding In-Reply-To: References: Message-ID: On Mon, Dec 13, 2010 at 12:12, Hettwer, Marian wrote: > $ more /etc/varnish/default.vcl > backend default { > set backend.host = "127.0.0.1"; > set backend.port = "8080"; > } > That doesn't make sense. You tell varnish to use the backend at localhost > 8080, where as seen above, its varnish. > $ more /etc/default/varnish NFILES=131072 INSTANCE=$(uname -n) # DAEMON_OPTS="-a :6081 \ -T localhost:6082 \ -b localhost:8080 \ -u varnish -g varnish \ -s file,/var/lib/varnish/$INSTANCE/varnish_storage.bin,1G" Should I s/8080/80 ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From mhettwer at team.mobile.de Mon Dec 13 12:31:31 2010 From: mhettwer at team.mobile.de (Hettwer, Marian) Date: Mon, 13 Dec 2010 12:31:31 +0000 Subject: Server Not Responding In-Reply-To: Message-ID: On 13.12.10 13:24, "Tom Coady" wrote: >On Mon, Dec 13, 2010 at 12:12, Hettwer, Marian >wrote: > >$ more /etc/varnish/default.vcl >backend default { >set backend.host = "127.0.0.1"; >set backend.port = "8080"; >} > >That doesn't make sense. You tell varnish to use the backend at localhost >8080, where as seen above, its varnish. > > > >$ more /etc/default/varnish NFILES=131072 >INSTANCE=$(uname -n) ># >DAEMON_OPTS="-a :6081 \ > -T localhost:6082 \ > -b localhost:8080 \ > -u varnish -g varnish \ > -s file,/var/lib/varnish/$INSTANCE/varnish_storage.bin,1G" > >Should I s/8080/80 ? Erm. You are using debian here,right? I would kick out the option -b localhost:8080 completely. Then go to /etc/varnish/default.vcl and change "set backend.port = 8080" to "set backend.port = 80". Afterwards, start using the debian startup scripts, or if you must, start it manually with "sudo varnishd -f /etc/varnish/default.vcl -s malloc,1G -T 127.0.0.1:2000 -a 0.0.0.0:8080" regards, Marian PS.: Apologies to everyone using a decent MUA. Mine at work is Outlook for Mac and obviously it gets quoting completely wrong. Arrgh. What a pain. From tom.coady at gmail.com Mon Dec 13 12:38:49 2010 From: tom.coady at gmail.com (Tom Coady) Date: Mon, 13 Dec 2010 12:38:49 +0000 Subject: Server Not Responding In-Reply-To: References: Message-ID: On Mon, Dec 13, 2010 at 12:31, Hettwer, Marian wrote: > On 13.12.10 13:24, "Tom Coady" wrote: > Erm. You are using debian here,right? > yes > I would kick out the option -b localhost:8080 completely. > ok > Then go to /etc/varnish/default.vcl and change "set backend.port = 8080" > to "set backend.port = 80". > already done: backend default { set backend.host = "127.0.0.1"; set backend.port = "80"; } > > Afterwards, start using the debian startup scripts, or if you must, start > it manually with "sudo varnishd -f /etc/varnish/default.vcl -s malloc,1G > -T 127.0.0.1:2000 -a 0.0.0.0:8080" > $ sudo varnishd -f /etc/varnish/default.vcl -s malloc,1G -T 127.0.0.1:2000-a 0.0.0.0:8080 Using old SHMFILE $ varnishlog 0 CLI Rd ping 0 CLI Wr 0 200 PONG 1292243669 so still no sign of life while serving pages.. -------------- next part -------------- An HTML attachment was scrubbed... URL: From pom at dmsp.de Mon Dec 13 15:31:48 2010 From: pom at dmsp.de (Stefan Pommerening) Date: Mon, 13 Dec 2010 16:31:48 +0100 Subject: cookies In-Reply-To: References: <0F69574F9901D4459C6B75C9FF64FBC40538F7EC@mxfl01.hollywoodmedia.corp> Message-ID: <4D063C64.3090004@dmsp.de> Am 13.12.2010 09:46, schrieb andan andan: > 2010/12/10 Vitaly Burshteyn: >> Hi folks, >> >> We use cookies to test if it's a returning visitor or not. Is there any >> way to force a lookup on varnish if a specific cookie is present? > http://www.varnish-cache.org/docs/2.1/reference/vcl.html > > Look at "..how to force Varnish to cache documents even when cookies > are present::" Just to point you in the right direction (although reading the reference documents _is_ necessary, too): in vcl_recv: (example) if (req.request == "GET" && req.http.cookie && req.http.cookie ~ "") { [ normalize cookie for reasonable caching / reduce variations (lower number of objects in cache) ] return (lookup); } From armdan20 at gmail.com Mon Dec 13 16:08:31 2010 From: armdan20 at gmail.com (andan andan) Date: Mon, 13 Dec 2010 17:08:31 +0100 Subject: cookies In-Reply-To: <4D063C64.3090004@dmsp.de> References: <0F69574F9901D4459C6B75C9FF64FBC40538F7EC@mxfl01.hollywoodmedia.corp> <4D063C64.3090004@dmsp.de> Message-ID: 2010/12/13 Stefan Pommerening : > Am 13.12.2010 09:46, schrieb andan andan: >> >> 2010/12/10 Vitaly Burshteyn: >>> >>> Hi folks, >>> >>> We use cookies to test if it's a returning visitor or not. ?Is there any >>> way to force a lookup on varnish if a specific cookie is present? >> >> http://www.varnish-cache.org/docs/2.1/reference/vcl.html >> >> Look at "..how to force Varnish to cache documents even when cookies >> are present::" > > Just to point you in the right direction (although reading the reference > documents _is_ necessary, too): > > in vcl_recv: > (example) > if (req.request == "GET" && req.http.cookie && req.http.cookie ~ "") > { > ? [ normalize cookie for reasonable caching / reduce variations (lower > number of objects in cache) ] In the sake of good order, this piece of code doesn't affect the number of objects in cache. To vary the objects in cache in function of a cookie, you need to add the cookie to the "req.hash" in the "hash" section. Kind Regards. From jelder at locamoda.com Mon Dec 13 16:55:16 2010 From: jelder at locamoda.com (Jacob Elder) Date: Mon, 13 Dec 2010 11:55:16 -0500 Subject: Cache-control: no-cache Message-ID: Just posting this so it will make it into the mailing list archives, and hopefully spark a discussion: The default Varnish config does not honor "no-cache" in Cache-control headers. It is my personal opinion that it should, but it's easy enough to add this behavior if desired. sub vcl_fetch { if (beresp.http.cache-control ~ "no-cache") { return(pass); } } -- Jacob Elder @jelder (646) 535-3379 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jim.robinson at stanford.edu Mon Dec 13 17:08:19 2010 From: jim.robinson at stanford.edu (James A. Robinson) Date: Mon, 13 Dec 2010 09:08:19 -0800 Subject: Cache-control: no-cache In-Reply-To: References: Message-ID: On Mon, Dec 13, 2010 at 08:55, Jacob Elder wrote: > Just posting this so it will make it into the mailing list archives, and > hopefully spark a discussion: > The default Varnish config does not honor "no-cache" in Cache-control > headers. It is my personal opinion that it should, but it's easy enough to > add this behavior if desired. > sub vcl_fetch { > ?? ?if (beresp.http.cache-control ~ "no-cache") { > ?? ? ? ?return(pass); > ?? ?} > } I think it really depends on your application. As someone who deals with publications that change rarely (new content comes in frequently, but rarely changes), we have to deal with bots that crawl the site and specify no-cache -- it doesn't make sense to honor that request if we know the data hasn't actually changed. Jim From alexus at gmail.com Mon Dec 13 17:24:20 2010 From: alexus at gmail.com (alexus) Date: Mon, 13 Dec 2010 12:24:20 -0500 Subject: max-age Message-ID: how/where can I define a cache (max-age) in default.vcl ? I have few PDF files that being cached and 2 weeks seems a bit much in my opinion, I'd like to change that. -- http://alexus.org/ From pom at dmsp.de Mon Dec 13 18:10:46 2010 From: pom at dmsp.de (Stefan Pommerening) Date: Mon, 13 Dec 2010 19:10:46 +0100 Subject: max-age In-Reply-To: References: Message-ID: <4D0661A6.9070804@dmsp.de> Am 13.12.2010 18:24, schrieb alexus: > how/where can I define a cache (max-age) in default.vcl ? > I have few PDF files that being cached and 2 weeks seems a bit much in > my opinion, I'd like to change that. Add some code - similar to the following EXAMPLE - to vcl_fetch: if (beresp.http.content-type ~ "pdf") { unset beresp.http.expires; unset beresp.http.cache-control; set beresp.http.cache-control = "max-age = 1d"; set beresp.ttl = 86400s; } Stefan From jelder at locamoda.com Mon Dec 13 19:55:23 2010 From: jelder at locamoda.com (Jacob Elder) Date: Mon, 13 Dec 2010 14:55:23 -0500 Subject: Cache-control: no-cache In-Reply-To: References: Message-ID: On Mon, Dec 13, 2010 at 12:08 PM, James A. Robinson < jim.robinson at stanford.edu> wrote: > On Mon, Dec 13, 2010 at 08:55, Jacob Elder wrote: > > Just posting this so it will make it into the mailing list archives, and > > hopefully spark a discussion: > > The default Varnish config does not honor "no-cache" in Cache-control > > headers. It is my personal opinion that it should, but it's easy enough > to > > add this behavior if desired. > > sub vcl_fetch { > > if (beresp.http.cache-control ~ "no-cache") { > > return(pass); > > } > > } > > I think it really depends on your application. As someone who deals > with publications that change rarely (new content comes in frequently, > but rarely changes), we have to deal with bots that crawl the site and > specify no-cache -- it doesn't make sense to honor that request if we > know the data hasn't actually changed. > > > Jim > You are right but I should have been more clear. I believe that Varnish should honor no-cache from backends but not necessarily from clients. -- Jacob Elder @jelder (646) 535-3379 -------------- next part -------------- An HTML attachment was scrubbed... URL: From TFigueiro at au.westfield.com Mon Dec 13 21:37:00 2010 From: TFigueiro at au.westfield.com (Thiago Figueiro) Date: Tue, 14 Dec 2010 08:37:00 +1100 Subject: max-age In-Reply-To: References: Message-ID: <64E73E81AAC26A49AC9EA28CBE65365107A77771@AUSYDEVS01.au.ad.westfield.com> In my experience it's better if you're able to let Varnish trust the max-age specified by the backends. Otherwise, as time goes by, you will have so many rules in your VCLs that you will make it hard to understand. Most web servers allow you to configure cache headers based on the content type; I suggest you try that approach before hacking your Varnish configuration. Cheers, Thiago. -----Original Message----- From: varnish-misc-bounces at varnish-cache.org [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of alexus Sent: Tuesday, 14 December 2010 4:24 AM To: varnish-misc at varnish-cache.org Subject: max-age how/where can I define a cache (max-age) in default.vcl ? I have few PDF files that being cached and 2 weeks seems a bit much in my opinion, I'd like to change that. -- http://alexus.org/ _______________________________________________ varnish-misc mailing list varnish-misc at varnish-cache.org http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc ______________________________________________________ CONFIDENTIALITY NOTICE This electronic mail message, including any and/or all attachments, is for the sole use of the intended recipient(s), and may contain confidential and/or privileged information, pertaining to business conducted under the direction and supervision of the sending organization. All electronic mail messages, which may have been established as expressed views and/or opinions (stated either within the electronic mail message or any of its attachments), are left to the sole responsibility of that of the sender, and are not necessarily attributed to the sending organization. Unauthorized interception, review, use, disclosure or distribution of any such information contained within this electronic mail message and/or its attachment(s), is (are) strictly prohibited. If you are not the intended recipient, please contact the sender by replying to this electronic mail message, along with the destruction all copies of the original electronic mail message (along with any attachments). ______________________________________________________ From ibeginhere at gmail.com Tue Dec 14 02:30:48 2010 From: ibeginhere at gmail.com (ll) Date: Tue, 14 Dec 2010 10:30:48 +0800 Subject: New Version backend config Message-ID: <4D06D6D8.2090309@gmail.com> hi?all?Is there any change about the muti-backend config in the varnish-cache-2.1.4 version ? I set the backend like that backend abc { .host="www.abc.com"; .port="80"; } the www.abc.com DNS had change to my varnish server ip 192.168.1.1 and i had edit the varnish server 's /etc/hosts file ,add 192.168.1.2 www.abc.com but now, all the url which had set the backend to "abc" is not work.I use the same config in the 2.0.4 version is fine. I don't know how to fix it ? From kristian at varnish-software.com Tue Dec 14 08:04:54 2010 From: kristian at varnish-software.com (Kristian Lyngstol) Date: Tue, 14 Dec 2010 09:04:54 +0100 Subject: Cache-control: no-cache In-Reply-To: References: Message-ID: <20101214080339.GA2064@freud> On Mon, Dec 13, 2010 at 11:55:16AM -0500, Jacob Elder wrote: > The default Varnish config does not honor "no-cache" in Cache-control > headers. It is my personal opinion that it should, but it's easy enough to > add this behavior if desired. This is a trade-off of sorts. RFC2616 doesn't really have any real description of a reverse proxy, and thus we are left making best efforts. In most cases, we find ourself on the origin-side of the equation. The Cache-Control header is currently sent to the clients too and it is with those in mind that most web application developers design the Cache-Control header. Even using s-maxage/max-age is controversial enough. The difference is that s-maxage/max-age/expire require date-parsing, which isn't trivial in VCL at the moment. Check if no-cache or private is present is trivial. Personally, I'm leaning towards keeping the curent behavior, but it's a topic of frequent debate. It's very common to distinguish between web-clients and the reverse proxy - ideally we'd want our own set of headers... - Kristian From ITLJ at gyldendal.dk Tue Dec 14 08:24:14 2010 From: ITLJ at gyldendal.dk (=?iso-8859-1?Q?Lars_J=F8rgensen?=) Date: Tue, 14 Dec 2010 09:24:14 +0100 Subject: max-age In-Reply-To: <4D0661A6.9070804@dmsp.de> References: <4D0661A6.9070804@dmsp.de> Message-ID: <311386E60ED7E74C8716A6573A7740A01870F175DF@s-exch01-v.gyldendal.local> >if (beresp.http.content-type ~ "pdf") { > unset beresp.http.expires; > unset beresp.http.cache-control; > set beresp.http.cache-control = "max-age = 1d"; > set beresp.ttl = 86400s; >} And if you can't do seconds-to-days calculations in your head, you can specify it like this set beresp.ttl = 24h; or even set beresp.ttl = 14d; Lars From Klapproth.M at sportresult.com Tue Dec 14 08:24:54 2010 From: Klapproth.M at sportresult.com (Klapproth, Martin) Date: Tue, 14 Dec 2010 09:24:54 +0100 Subject: New Version backend config In-Reply-To: <4D06D6D8.2090309@gmail.com> Message-ID: <4BC50911656FBB49858C6ADB1B8D223F038D96D1@mail.wigemic.lpz> Have you tried to restart your varnishd. Varnish performs DNS lookups only at the startup, I think. -----Original Message----- From: varnish-misc-bounces at varnish-cache.org [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of ll Sent: Dienstag, 14. Dezember 2010 03:31 To: varnish-misc at varnish-cache.org Subject: New Version backend config hi?all?Is there any change about the muti-backend config in the varnish-cache-2.1.4 version ? I set the backend like that backend abc { .host="www.abc.com"; .port="80"; } the www.abc.com DNS had change to my varnish server ip 192.168.1.1 and i had edit the varnish server 's /etc/hosts file ,add 192.168.1.2 www.abc.com but now, all the url which had set the backend to "abc" is not work.I use the same config in the 2.0.4 version is fine. I don't know how to fix it ? _______________________________________________ varnish-misc mailing list varnish-misc at varnish-cache.org http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc From ITLJ at gyldendal.dk Tue Dec 14 08:27:13 2010 From: ITLJ at gyldendal.dk (=?utf-8?B?TGFycyBKw7hyZ2Vuc2Vu?=) Date: Tue, 14 Dec 2010 09:27:13 +0100 Subject: New Version backend config In-Reply-To: <4D06D6D8.2090309@gmail.com> References: <4D06D6D8.2090309@gmail.com> Message-ID: <311386E60ED7E74C8716A6573A7740A01870F175E0@s-exch01-v.gyldendal.local> Hi ll, Specify the ip address instead of the DNS name. That way you also cut the name lookups out of the equation. Lars -----Original Message----- From: varnish-misc-bounces at varnish-cache.org [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of ll Sent: Tuesday, December 14, 2010 3:31 AM To: varnish-misc at varnish-cache.org Subject: New Version backend config hi?all?Is there any change about the muti-backend config in the varnish-cache-2.1.4 version ? I set the backend like that backend abc { .host="www.abc.com"; .port="80"; } the www.abc.com DNS had change to my varnish server ip 192.168.1.1 and i had edit the varnish server 's /etc/hosts file ,add 192.168.1.2 www.abc.com but now, all the url which had set the backend to "abc" is not work.I use the same config in the 2.0.4 version is fine. I don't know how to fix it ? _______________________________________________ varnish-misc mailing list varnish-misc at varnish-cache.org http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc From tom.coady at gmail.com Tue Dec 14 11:34:32 2010 From: tom.coady at gmail.com (Tom Coady) Date: Tue, 14 Dec 2010 11:34:32 +0000 Subject: Server Not Responding In-Reply-To: References: Message-ID: On Mon, Dec 13, 2010 at 12:46, Hettwer, Marian wrote: > > HTTP/1.1 200 OK > > Server: Apache/2.2.9 (Debian) mod_fastcgi/2.4.6 > >Phusion_Passenger/2.2.15 PHP/5.2.6-1+lenny9 with Suhosin-Patch > >mod_ssl/2.2.9 OpenSSL/0.9.8g > > It's your Apache answering! Through... > Is that good or bad? > X-Varnish: 2019819826 > Varnish! > > Age: 0 > It's fresh and doesn't come from varnish's cache. Try again and you should > see an increased Age. > > Via: 1.1 varnish > Means http 1.1 and varnish. > I believe you are good to go. > By the way, you should have seen that request in varnishlog too. > Thanks Marian It's still showing age 0 and nothing special shows in varnishlog, so I assume it's still not working :( Does it help to know some of the domains on this server? -------------- next part -------------- An HTML attachment was scrubbed... URL: From tfheen at varnish-software.com Tue Dec 14 12:44:49 2010 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Tue, 14 Dec 2010 13:44:49 +0100 Subject: Cache-control: no-cache In-Reply-To: <20101214080339.GA2064@freud> (Kristian Lyngstol's message of "Tue, 14 Dec 2010 09:04:54 +0100") References: <20101214080339.GA2064@freud> Message-ID: <87vd2wa5ha.fsf@qurzaw.varnish-software.com> ]] Kristian Lyngstol | Personally, I'm leaning towards keeping the curent behavior, but it's a | topic of frequent debate. It's very common to distinguish between | web-clients and the reverse proxy - ideally we'd want our own set of | headers... This sounds very much like what Surrogate-Control was designed for. Not that it has seen much uptake, though. -- Tollef Fog Heen Varnish Software t: +47 21 98 92 64 From ruben at varnish-software.com Tue Dec 14 12:54:21 2010 From: ruben at varnish-software.com (=?ISO-8859-1?Q?Rub=E9n_Romero?=) Date: Tue, 14 Dec 2010 13:54:21 +0100 Subject: Varnish User Group meeting 3 in Amsterdam [VUG3] 14+15/02/2011 Message-ID: Hello to you all, It is again that time of the year! On February 14th and 15th 2011 the third Varnish User Group meeting will be held at the TomTom Offices by Rembrandtplein in Amsterdam. For general information and how to sign up please visit the following link: * http://www.varnish-cache.org/trac/wiki/VUG3 We have a maximum of 30 attendees. Please sign-up only if you are actually attending and do so as quickly as you possible can as seats might be taken. -- Best regards, -- Rub?n Romero Varnish Software e-mail ruben at varnish-software.com?/?skype: ruben_varnish P: +47 21 98 92?62 /?M: +47 95 96 40 88 Online Sales Chat: http://www.varnish-software.com/contact-us ====================== ?Varnish makes websites fly! ====================== www.varnish-software.com twitter.com/varnishsoftware linkedin.com/companies/varnish-software From jelder at locamoda.com Tue Dec 14 13:11:08 2010 From: jelder at locamoda.com (Jacob Elder) Date: Tue, 14 Dec 2010 08:11:08 -0500 Subject: Cache-control: no-cache In-Reply-To: <87vd2wa5ha.fsf@qurzaw.varnish-software.com> References: <20101214080339.GA2064@freud> <87vd2wa5ha.fsf@qurzaw.varnish-software.com> Message-ID: On Tue, Dec 14, 2010 at 7:44 AM, Tollef Fog Heen < tfheen at varnish-software.com> wrote: > ]] Kristian Lyngstol > > | Personally, I'm leaning towards keeping the curent behavior, but it's a > | topic of frequent debate. It's very common to distinguish between > | web-clients and the reverse proxy - ideally we'd want our own set of > | headers... > > This sounds very much like what Surrogate-Control was designed for. Not > that it has seen much uptake, though. > > Wow, I wish I had known about Surrogate-Control before inventing X-Cache-Alternate for internal use in my organization. It would probably catch on if Varnish was the reference implementation. > -- > Tollef Fog Heen > Varnish Software > t: +47 21 98 92 64 > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -- Jacob Elder @jelder (646) 535-3379 -------------- next part -------------- An HTML attachment was scrubbed... URL: From pom at dmsp.de Tue Dec 14 16:05:29 2010 From: pom at dmsp.de (Stefan Pommerening) Date: Tue, 14 Dec 2010 17:05:29 +0100 Subject: Listing of cache content possible? Message-ID: <4D0795C9.1000806@dmsp.de> Hi, is there any way or tool to find out what is stored in the cache at a given time? I know varnishstat gives me some insight (n_objecthead, n_objectcore, ...) but I'd like to get a list of host/url tupels (e.g. from n_objecthead) with the size of each object (core) in bytes. Somehow possible - or am I dreaming? ;-) Stefan -- http://dmsp.de From lin.support at gmail.com Tue Dec 14 16:28:41 2010 From: lin.support at gmail.com (linuxsupport) Date: Tue, 14 Dec 2010 16:28:41 +0000 Subject: Listing of cache content possible? In-Reply-To: <4D0795C9.1000806@dmsp.de> References: <4D0795C9.1000806@dmsp.de> Message-ID: It is not possible at the moment, future version may add this functionality. Regards, Aniruddh On Tue, Dec 14, 2010 at 4:05 PM, Stefan Pommerening wrote: > Hi, > > is there any way or tool to find out what is stored in the cache at a given > time? > > I know varnishstat gives me some insight (n_objecthead, n_objectcore, ...) > but I'd like to get a list of host/url tupels (e.g. from n_objecthead) with > the > size of each object (core) in bytes. > > Somehow possible - or am I dreaming? ;-) > > Stefan > > -- > http://dmsp.de > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From armdan20 at gmail.com Tue Dec 14 16:41:29 2010 From: armdan20 at gmail.com (andan andan) Date: Tue, 14 Dec 2010 17:41:29 +0100 Subject: Listing of cache content possible? In-Reply-To: <4D0795C9.1000806@dmsp.de> References: <4D0795C9.1000806@dmsp.de> Message-ID: 2010/12/14 Stefan Pommerening : > Hi, > > is there any way or tool to find out what is stored in the cache at a given > time? According to http://www.varnish-cache.org/docs/2.1/faq/general.html you can't. But always you can do a GET and look at X-Varnish header, hit (two numbers) or miss (one number), says to you if the object is in cache, of course only if the object is cacheable. And of course, if the object is cacheable and the object was not in cache before you do GET, now it's in cache :) In some scenarios this trick is not desirable, specially if you ONLY want to know if the object is in cache. Kind Regards. From ibeginhere at gmail.com Wed Dec 15 00:54:01 2010 From: ibeginhere at gmail.com (ll) Date: Wed, 15 Dec 2010 08:54:01 +0800 Subject: New Version backend config In-Reply-To: <311386E60ED7E74C8716A6573A7740A01870F175E0@s-exch01-v.gyldendal.local> References: <4D06D6D8.2090309@gmail.com> <311386E60ED7E74C8716A6573A7740A01870F175E0@s-exch01-v.gyldendal.local> Message-ID: <4D0811A9.3090108@gmail.com> what is different between the .host is domain and IP ? I thought the IP had multi domains ,so I set the .host exactly to a domain which I want to proxy . ? 2010-12-14 16:27, Lars J?rgensen ??: > Hi ll, > > Specify the ip address instead of the DNS name. That way you also cut the name lookups out of the equation. > > > Lars > > -----Original Message----- > From: varnish-misc-bounces at varnish-cache.org [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of ll > Sent: Tuesday, December 14, 2010 3:31 AM > To: varnish-misc at varnish-cache.org > Subject: New Version backend config > > hi?all?Is there any change about the muti-backend config in the varnish-cache-2.1.4 version ? > I set the backend like that > backend abc { > .host="www.abc.com"; > .port="80"; > } > the www.abc.com DNS had change to my varnish server ip 192.168.1.1 > and i had edit the varnish server 's /etc/hosts file ,add > 192.168.1.2 www.abc.com > > but now, all the url which had set the backend to "abc" is not work.I use the same config in the > 2.0.4 version is fine. I don't know how to fix it ? > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc From varnish at mm.quex.org Wed Dec 15 02:41:21 2010 From: varnish at mm.quex.org (Michael Alger) Date: Wed, 15 Dec 2010 10:41:21 +0800 Subject: New Version backend config In-Reply-To: <4D0811A9.3090108@gmail.com> References: <4D06D6D8.2090309@gmail.com> <311386E60ED7E74C8716A6573A7740A01870F175E0@s-exch01-v.gyldendal.local> <4D0811A9.3090108@gmail.com> Message-ID: <20101215024121.GA6422@grum.quex.org> On Wed, Dec 15, 2010 at 08:54:01AM +0800, ll wrote: > ? 2010-12-14 16:27, Lars J?rgensen ??: >> Hi ll, >> >> Specify the ip address instead of the DNS name. That way you also cut >> the name lookups out of the equation. > > what is different between the .host is domain and IP ? > > I thought the IP had multi domains ,so I set the .host exactly to a > domain which I want to proxy . The backend definition gives the address and port of a backend web server which you want to retrieve content from, in the event that the request cannot be satisfied by Varnish's cache. When you specify the backend .host attribute, you are just telling Varnish where to connect to. It doesn't alter anything about the request headers sent, including the Host header. It's purely used to specify an endpoint of a TCP/IP connection. The actual data which is sent over that connection is determined by the request and the VCL code. If you have multiple websites on different domain names hosted by the same server on the same IP address+port (i.e. name-based virtual hosting), then the web server is deciding which site to serve content for based on the "Host:" header in the HTTP request. That is, it reads the request, looks at the value of the Host: header, and sees if it has a site configured which matches that hostname. You can check/manipulate the Host header within VCL using req.http.Host. If you don't explicitly do anything with it, then it will remain set to whatever the client sent in its request. For the most reliable/easiest to understand use, I would recommend: - always specify the IP address for your backend servers. - if you have multiple backends for the same website, group them using a director; or use retries and some VCL logic to select a different backend if the first is no good. - it's a good idea to check the Host header of the incoming request to make sure it matches a site you actually host/proxy. if (! req.http.Host ~ "^(www\.abc\.com|www\.xyz\.net|alpha\.bet)$") { error 403 "Site not hosted here"; } - if you are serving multiple domains, make sure you're including req.http.Host in the hash when looking up items in the cache. - also consider rewriting it to a single form. e.g. rewrite a host header of "abc.com" to "www.abc.com" to avoid having two versions of the same object cached. if (req.http.Host == "abc.com") { set req.http.Host = "www.abc.com"; } - I'd also recommend doing something like: set req.http.Host = regsub(req.http.Host, ":80$", ""); to remove the port 80 specification from the Host header, which some clients send. - and consider what will/should happen if the client doesn't send a Host header at all. if (! req.http.Host) { error 500 "No host header"; } You'd want to implement these checks in the reverse of the order I listed them, for obvious reasons. :) >> -----Original Message----- >> From: varnish-misc-bounces at varnish-cache.org On Behalf Of ll >> Sent: Tuesday, December 14, 2010 3:31 AM >> To: varnish-misc at varnish-cache.org >> Subject: New Version backend config >> >> hi?all?Is there any change about the muti-backend config in the >> varnish-cache-2.1.4 version ? >> I set the backend like that >> backend abc { >> .host="www.abc.com"; >> .port="80"; >> } >> the www.abc.com DNS had change to my varnish server ip 192.168.1.1 >> and i had edit the varnish server 's /etc/hosts file ,add >> 192.168.1.2 www.abc.com >> >> but now, all the url which had set the backend to "abc" is not work.I >> use the same config in the 2.0.4 version is fine. I don't know how to >> fix it ? From cdgraff at gmail.com Wed Dec 15 02:58:28 2010 From: cdgraff at gmail.com (Alejandro) Date: Tue, 14 Dec 2010 23:58:28 -0300 Subject: Examples of Balancer setting? Message-ID: Hi Guys, Regards, from Argentina... Have someone an example of real world Balancer setting with health check? I have 4 nodes of Drupal and need do caching and balancing with Varnish. Thanks for any tips or example. Kind regards, Alejandro -------------- next part -------------- An HTML attachment was scrubbed... URL: From stewsnooze at gmail.com Wed Dec 15 07:29:35 2010 From: stewsnooze at gmail.com (Stewart Robinson) Date: Wed, 15 Dec 2010 07:29:35 +0000 Subject: Examples of Balancer setting? In-Reply-To: References: Message-ID: Hi, I assume you mean the Pressflow distribution of Drupal and not vanilla Drupal. As Drupal doesn't really set the right headers to be able to support Varying by cookie. This URL features a decent varnish-Drupal vcl. Pressflow doesn't need specific load balancing control so just make a config from this [age for your backends http://www.varnish-cache.org/trac/wiki/LoadBalancing You'll also need to make changes to Pressflow settings.... https://wiki.fourkitchens.com/display/PF/Using+Pressflow+behind+a+reverse+proxy+cache Stew @stewsnooze http://fullfatthings.com/ On 15 December 2010 02:58, Alejandro wrote: > Hi Guys, > > Regards, from Argentina... > > Have someone an example of real world Balancer setting with health check? > > I have 4 nodes of Drupal and need do caching and balancing with Varnish. > > Thanks for any tips or example. > > Kind regards, > Alejandro > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From andre.oien.langvand at aspiro.com Mon Dec 6 13:02:25 2010 From: andre.oien.langvand at aspiro.com (=?ISO-8859-1?Q?Andr=E9_=D8ien_Langvand?=) Date: Mon, 6 Dec 2010 14:02:25 +0100 Subject: Streaming pass/fetch Message-ID: Hi, In regards to the note http://www.varnish-cache.org/trac/wiki/PostTwoShoppingList#a12.Streamingpassfetch, are there any updates on when we could expect this to be implemented? We are looking for the best way to set up caching nodes for our streamed media (music), currently considering Varnish on boxes with 96gb ram and 300gb SSD for backing file. Seems to do the job extremely well, besides the fact that Varnish needs to fetch the whole file from backend before serving it. As far as I can see, nginx with ncache would be the best alternative as of today, however, as far as what me and my friend Google can tell, ncache doesn't seem stable enough for production. Andr? -------------- next part -------------- An HTML attachment was scrubbed... URL: From anand at rediff-inc.com Wed Dec 8 08:27:59 2010 From: anand at rediff-inc.com (Anand Shah) Date: Wed, 8 Dec 2010 13:57:59 +0530 Subject: ICP / cache_peer for varnish In-Reply-To: References: Message-ID: <5005DF89ACBF4B6CA4BCC418AA5ED127@rediff.net> HI, Is it possible to use the squid command "cache_peer" in varnish?? Does Varnish support ICP for inter-cache co-ordination. There is no cache peering in Varnish. What could be the best possible option to achieve it using varnish. ICP should be implemented in varnish to make it perform much faster. I have a multi datacenter environment and using around 40 varnish systems to cache. 1. Restarts/hangs remove all cache contents. (Cache persistency required) 2. Varnish cache masters and peers across different hubs. (Currently using squid for byte range support and cache peer) I want complete varnish architecture to make web faster for me. Please suggest. Regards, Anand _____ From: varnish-misc-bounces at varnish-cache.org [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Jacob Elder Sent: 08 December 2010 07:13 To: varnish-misc at varnish-cache.org Subject: Display bug in varnishstat (Sorry if I'm not following protocol; I don't see where to submit this on the site.) The varnishstat hitrate average header will display "nan" instead of the expected information if the user resizes the terminal window, or if the user presses any of the arrow keys. I am running the official debs for 2.1.2-1~hardy1 on amd64 and running varnishstat over SSH from Terminal.app. -- Jacob Elder @jelder (646) 535-3379 -------------- next part -------------- An HTML attachment was scrubbed... URL: From cdgraff at gmail.com Wed Dec 15 14:38:18 2010 From: cdgraff at gmail.com (Alejandro) Date: Wed, 15 Dec 2010 11:38:18 -0300 Subject: Examples of Balancer setting? In-Reply-To: References: Message-ID: Hi Stew, Thank for your response... Yes I'm using Pressflow, currently with CDN module... this is our current model Varnish (Static Files) using CDN module, only Images are redirected to static.domain.com The Drupals have 4 nodes, and all use load Balancer in front to them... Our idea is change this for Varnish do the balancing and the full caching not only the Static files with other URL. Thanks again for any tips. Regards, Alejandro 2010/12/15 Stewart Robinson > Hi, > > I assume you mean the Pressflow distribution of Drupal and not vanilla > Drupal. As Drupal doesn't really set the right headers to be able to support > Varying by cookie. > > This URL features a decent varnish-Drupal vcl. Pressflow doesn't need > specific load balancing control so just make a config from this [age for > your backends > > http://www.varnish-cache.org/trac/wiki/LoadBalancing > > You'll also need to make changes to Pressflow settings.... > https://wiki.fourkitchens.com/display/PF/Using+Pressflow+behind+a+reverse+proxy+cache > > Stew > > @stewsnooze > http://fullfatthings.com/ > > On 15 December 2010 02:58, Alejandro wrote: > >> Hi Guys, >> >> Regards, from Argentina... >> >> Have someone an example of real world Balancer setting with health check? >> >> I have 4 nodes of Drupal and need do caching and balancing with Varnish. >> >> Thanks for any tips or example. >> >> Kind regards, >> Alejandro >> >> _______________________________________________ >> varnish-misc mailing list >> >> varnish-misc at varnish-cache.org >> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lin.support at gmail.com Wed Dec 15 20:04:54 2010 From: lin.support at gmail.com (linuxsupport) Date: Thu, 16 Dec 2010 01:34:54 +0530 Subject: Varnishstate Message-ID: Hi, What is the difference between Client connections accepted and Client requests received? It shows me a big difference, so I am a bit concern about it. 1764965 7.99 3:34 Client connections accepted 5768448 28.96 10.91 Client requests received Also N LRU moved object is so high 917050 . . N LRU moved objects? Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From perbu at varnish-software.com Wed Dec 15 20:30:44 2010 From: perbu at varnish-software.com (Per Buer) Date: Wed, 15 Dec 2010 21:30:44 +0100 Subject: Varnishstate In-Reply-To: References: Message-ID: On Wed, Dec 15, 2010 at 9:04 PM, linuxsupport wrote: > Hi, > > What is the difference between Client connections accepted and Client > requests received? Several request are pushed over a tcp connection. -- Per Buer,?Varnish Software Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer Varnish makes websites fly! From pom at dmsp.de Wed Dec 15 21:39:17 2010 From: pom at dmsp.de (Stefan Pommerening) Date: Wed, 15 Dec 2010 22:39:17 +0100 Subject: Varnishstate In-Reply-To: References: Message-ID: <3248d478-d020-41f8-9b71-bd9a93991b16@email.android.com> With HTTP1.1 you can push several client requests over the same tcp connection without closing and reopening it which takes some load off the servers. But you can force varnish by vcl to close every connection after the first request. That is necessary when in need of a correct X-Forwarded-For header because only the first client request has a X-Forwarded-For header set. Stefan "Per Buer" schrieb: >On Wed, Dec 15, 2010 at 9:04 PM, linuxsupport >wrote: >> Hi, >> >> What is the difference between Client connections accepted and Client >> requests received? > >Several request are pushed over a tcp connection. > > >-- >Per Buer,?Varnish Software >Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer >Varnish makes websites fly! > >_______________________________________________ >varnish-misc mailing list >varnish-misc at varnish-cache.org >http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc -- Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail gesendet. From perbu at varnish-software.com Thu Dec 16 07:52:12 2010 From: perbu at varnish-software.com (Per Buer) Date: Thu, 16 Dec 2010 08:52:12 +0100 Subject: Varnishstate In-Reply-To: References: Message-ID: On Wed, Dec 15, 2010 at 9:04 PM, linuxsupport wrote: > > Also N LRU moved object is so high Don't worry about it. It's only maintaining the LRU list, quite cheap operations. -- Per Buer,?Varnish Software Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer Varnish makes websites fly! Want to learn more about Varnish? http://www.varnish-software.com/whitepapers From tfheen at varnish-software.com Thu Dec 16 08:57:25 2010 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Thu, 16 Dec 2010 09:57:25 +0100 Subject: Varnishstate In-Reply-To: <3248d478-d020-41f8-9b71-bd9a93991b16@email.android.com> (Stefan Pommerening's message of "Wed, 15 Dec 2010 22:39:17 +0100") References: <3248d478-d020-41f8-9b71-bd9a93991b16@email.android.com> Message-ID: <87y67q5c3u.fsf@qurzaw.varnish-software.com> ]] Stefan Pommerening Hi, | But you can force varnish by vcl to close every connection after the | first request. That is necessary when in need of a correct | X-Forwarded-For header because only the first client request has a | X-Forwarded-For header set. That is only true for piped requests, not cache misses or passed requests. Cheers, -- Tollef Fog Heen Varnish Software t: +47 21 98 92 64 From fhelmschrott at gmail.com Thu Dec 16 09:06:04 2010 From: fhelmschrott at gmail.com (Frank Helmschrott) Date: Thu, 16 Dec 2010 10:06:04 +0100 Subject: Realizing vhosts with varnish Message-ID: Hi list, here's my situation: I got one dedicated server to run varnish and multiple backend servers delivering websites. These Backend servers host several different web projects and the standard solution is the each project runs without any cache or anything else. Now i want to take some load off of the servers by using varnish and routing them through the cache. I thought about different solutions and searched around a bit and came across this blog post (http://www.varnish-software.com/blog/virtual-hosts-varnish) and also found the mailing list entry on that topic. I tried it out in my config but it didn't work. I think this is due to my missing knowledge so i thought i'd ask here. I currently have a default.vcl containing these lines: sub vcl_recv { if (!req.http.Host) { error 404 "Need a host header"; } set req.http.Host = regsub(req.http.Host, "^www\.", ""); set req.http.Host = regsub(req.http.Host, ":80$", ""); if (req.http.Host = "domain1.com") { include "/etc/varnish/vhost-domain1.com.vcl"; } elsif (req.http.Host = "domain2.com") { include "/etc/varnish/vhost-domain2.com.vcl"; } } The two vhost files contain a basically working configuration (it workes well when using it as the only config). With the default.vcl from above varnish doesn't start. Unfortunately on my server (using CentOS 5.5 and varnish from EPL) i don't get any error messages what may be wrong. Is there something missing that must be set before this if/else statement can be handled? I'm currently a bit confused and don't know where to continue. Any help is appreciated. I'm using the latest Version from EPEL which is 2.0.6 One additional question: When fetching the files from backend servers, doesn't varnish send a hostname? I tried putting some more vhosts on one IP on the backend and setup this IP in varnish as backend server but what varnish delivered was the default content from the backend instead of the vhosts content the hostname belonged to. Thanks for helping! -- Frank From perbu at varnish-software.com Thu Dec 16 09:15:00 2010 From: perbu at varnish-software.com (Per Buer) Date: Thu, 16 Dec 2010 10:15:00 +0100 Subject: Realizing vhosts with varnish In-Reply-To: References: Message-ID: Hi Frank. On Thu, Dec 16, 2010 at 10:06 AM, Frank Helmschrott wrote: > > I currently have a default.vcl containing these lines: I think your VCL looks sane. It's a lot easier to see what is going on if you could show us some varnishlog with a request going though. -- Per Buer,?Varnish Software Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer Varnish makes websites fly! Want to learn more about Varnish? http://www.varnish-software.com/whitepapers From pom at dmsp.de Thu Dec 16 09:38:26 2010 From: pom at dmsp.de (Stefan Pommerening) Date: Thu, 16 Dec 2010 10:38:26 +0100 Subject: Realizing vhosts with varnish In-Reply-To: References: Message-ID: <4D09DE12.2020201@dmsp.de> Please use double equation marks: if (req.http.Host == "domain1.com") { ..... } Stefan Am 16.12.2010 10:06, schrieb Frank Helmschrott: > Hi list, > > here's my situation: > > I got one dedicated server to run varnish and multiple backend servers > delivering websites. These Backend servers host several different web > projects and the standard solution is the each project runs without > any cache or anything else. > > Now i want to take some load off of the servers by using varnish and > routing them through the cache. I thought about different solutions > and searched around a bit and came across this blog post > (http://www.varnish-software.com/blog/virtual-hosts-varnish) and also > found the mailing list entry on that topic. I tried it out in my > config but it didn't work. I think this is due to my missing knowledge > so i thought i'd ask here. > > I currently have a default.vcl containing these lines: > > sub vcl_recv { > if (!req.http.Host) > { > error 404 "Need a host header"; > } > > set req.http.Host = regsub(req.http.Host, "^www\.", ""); > set req.http.Host = regsub(req.http.Host, ":80$", ""); > > if (req.http.Host = "domain1.com") > { > include "/etc/varnish/vhost-domain1.com.vcl"; > } > elsif (req.http.Host = "domain2.com") > { > include "/etc/varnish/vhost-domain2.com.vcl"; > } > } > > > The two vhost files contain a basically working configuration (it > workes well when using it as the only config). With the default.vcl > from above varnish doesn't start. Unfortunately on my server (using > CentOS 5.5 and varnish from EPL) i don't get any error messages what > may be wrong. Is there something missing that must be set before this > if/else statement can be handled? > > I'm currently a bit confused and don't know where to continue. Any > help is appreciated. I'm using the latest Version from EPEL which is > 2.0.6 > > One additional question: When fetching the files from backend servers, > doesn't varnish send a hostname? I tried putting some more vhosts on > one IP on the backend and setup this IP in varnish as backend server > but what varnish delivered was the default content from the backend > instead of the vhosts content the hostname belonged to. > > Thanks for helping! -- *Dipl.-Inform. Stefan Pommerening Informatik-B?ro: IT-Dienste & Projekte, Consulting & Coaching* An der Kappe 171 B, 13583 Berlin, Germany, Email: pom at dmsp.de Telephone(Office): +49-(0)30 / 375 84 888, Mobile: +49 - (0)179 / 394 80 72 Fax(Office): +49-(0)30 / 604 8594, http://www.dmsp.de Der Inhalt dieser E-Mail ist vertraulich und ausschlie?lich f?r den bezeichneten Adressaten bestimmt. Wenn Sie nicht der vorgesehene Adressat dieser E-Mail oder dessen Vertreter sein sollten, so beachten Sie bitte, dass jede Form der Kenntnisnahme, Ver?ffentlichung, Vervielf?ltigung oder Weitergabe des Inhalts dieser E-Mail unzul?ssig ist. Wir bitten Sie, sich in diesem Fall mit dem Absender der E-Mail in Verbindung zu setzen. -------------- next part -------------- An HTML attachment was scrubbed... URL: From kristian at varnish-software.com Thu Dec 16 09:41:56 2010 From: kristian at varnish-software.com (Kristian Lyngstol) Date: Thu, 16 Dec 2010 10:41:56 +0100 Subject: Realizing vhosts with varnish In-Reply-To: References: Message-ID: <20101216094155.GA1948@freud> On Thu, Dec 16, 2010 at 10:06:04AM +0100, Frank Helmschrott wrote: > if (req.http.Host = "domain1.com") That should be req.http.host == "domain1.com". (misisng a =). Other than that, we will need the rest of the vcl. - Kristian -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: Digital signature URL: From varnish at mm.quex.org Thu Dec 16 09:44:24 2010 From: varnish at mm.quex.org (Michael Alger) Date: Thu, 16 Dec 2010 17:44:24 +0800 Subject: Realizing vhosts with varnish In-Reply-To: References: Message-ID: <20101216094424.GA23867@grum.quex.org> On Thu, Dec 16, 2010 at 10:06:04AM +0100, Frank Helmschrott wrote: > I currently have a default.vcl containing these lines: > > sub vcl_recv { > > if (req.http.Host = "domain1.com") Is this verbatim? Should be == for comparison; with just an = you'll get an error: Message from VCC-compiler: Invalid condition '=' on IP number variable only '==', '!=' and '~' are legal (/etc/varnish/inc-serverips.vcl Line 2 Pos 15) if (server.ip = "10.23.190.21") --------------#---------------- Running VCC-compiler failed, exit 1 VCL compilation failed > The two vhost files contain a basically working configuration (it > workes well when using it as the only config). With the default.vcl > from above varnish doesn't start. Unfortunately on my server (using > CentOS 5.5 and varnish from EPL) i don't get any error messages what > may be wrong. Is there something missing that must be set before this > if/else statement can be handled? You can look in /var/log/syslog or similar to see if you get errors logged there. Alternatively, you can use the command-line interface to issue the reload command. # varnishadm -S /etc/varnish/secret -T :6082 vcl.load newconfig /etc/varnish/default.vcl vcl.use newconfig > One additional question: When fetching the files from backend servers, > doesn't varnish send a hostname? I tried putting some more vhosts on > one IP on the backend and setup this IP in varnish as backend server > but what varnish delivered was the default content from the backend > instead of the vhosts content the hostname belonged to. It'll send whatever hostname is set in the request, i.e. req.http.Host. At least, it should. You are stripping the "www." from the front - is the backend expecting that and configured to accept that? From kristian at varnish-software.com Thu Dec 16 09:59:13 2010 From: kristian at varnish-software.com (Kristian Lyngstol) Date: Thu, 16 Dec 2010 10:59:13 +0100 Subject: Display bug in varnishstat In-Reply-To: References: Message-ID: <20101216095912.GB1948@freud> On Tue, Dec 07, 2010 at 08:43:21PM -0500, Jacob Elder wrote: > (Sorry if I'm not following protocol; I don't see where to submit this on > the site.) Under trac, there is a "new ticket" tab after you log in. If you do not see it, you may need edit-bits. Drop me a mail with your trac user and I'll fix it (it's just an anti-spam measure). - Kristian -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: Digital signature URL: From fhelmschrott at gmail.com Thu Dec 16 12:33:57 2010 From: fhelmschrott at gmail.com (Frank Helmschrott) Date: Thu, 16 Dec 2010 13:33:57 +0100 Subject: Realizing vhosts with varnish In-Reply-To: <20101216094424.GA23867@grum.quex.org> References: <20101216094424.GA23867@grum.quex.org> Message-ID: > Is this verbatim? Should be == for comparison; with just an = you'll get > an error: > > Message from VCC-compiler: > Invalid condition '=' on IP number variable > ?only '==', '!=' and '~' are legal > (/etc/varnish/inc-serverips.vcl Line 2 Pos 15) > if (server.ip = "10.23.190.21") I corrected this but this doesn't change anything to the behaviour. All i get logged into /var/log/messages is this line: > Dec 16 20:28:46 cache01 varnishd[2787]: Manager got SIGINT Everything that is in the VCLs that get loaded by the part of the script that i posted here shouldn't make any problems as that part works perfectly without the "vhost selection" used here. So the problems seems to be inside this. Is it a problem to have more than one sub vcl_recv? There's more of that in the loaded files. Or does this if/else stuff have to go into vcl_recv anyway? I just copied over this solution from the URL given in my first post (http://www.varnish-software.com/blog/virtual-hosts-varnish) to have a good starting point. They don't tell anything about that sub function there but i thought it should be inside vcl_recv(). Besides from that that is everything i have in my defaul.vcl for now. Thanks for all your help. Frank From pom at dmsp.de Thu Dec 16 13:30:17 2010 From: pom at dmsp.de (Stefan Pommerening) Date: Thu, 16 Dec 2010 14:30:17 +0100 Subject: Realizing vhosts with varnish In-Reply-To: References: <20101216094424.GA23867@grum.quex.org> Message-ID: <4D0A1469.9050001@dmsp.de> Hi Frank, of couse you only need one single vcl_recv subroutine. This applies to all vcl subroutines. I usually start with grabbing the default vcl and then add everything I need to the existing (default) subroutines. If you use the include statement you have to keep in mind that inclusion is a textual substitution - therefore no repeated definition of vcl_recv (or even other subroutines) is allowed. I did not try this myself using your vcl fragment but if you don't get a reasonable error message this might be something to think about in upcoming varnish releases. Nevertheless I usually use the varnish CLI to load a new vcl in order to check the syntax when loading (vcl.load) and switch back and forth in case of errors (vcl.use). Usually the error message when getting a compilation error from a syntax error in the vcl is quite self-explanatory. Stefan -- http://dmsp.de Am 16.12.2010 13:33, schrieb Frank Helmschrott: >> Is this verbatim? Should be == for comparison; with just an = you'll get >> an error: >> >> Message from VCC-compiler: >> Invalid condition '=' on IP number variable >> only '==', '!=' and '~' are legal >> (/etc/varnish/inc-serverips.vcl Line 2 Pos 15) >> if (server.ip = "10.23.190.21") > I corrected this but this doesn't change anything to the behaviour. > > All i get logged into /var/log/messages is this line: > >> Dec 16 20:28:46 cache01 varnishd[2787]: Manager got SIGINT > Everything that is in the VCLs that get loaded by the part of the > script that i posted here shouldn't make any problems as that part > works perfectly without the "vhost selection" used here. So the > problems seems to be inside this. > > Is it a problem to have more than one sub vcl_recv? There's more of > that in the loaded files. Or does this if/else stuff have to go into > vcl_recv anyway? I just copied over this solution from the URL given > in my first post > (http://www.varnish-software.com/blog/virtual-hosts-varnish) to have a > good starting point. They don't tell anything about that sub function > there but i thought it should be inside vcl_recv(). > > Besides from that that is everything i have in my defaul.vcl for now. > > Thanks for all your help. > > Frank From marko at oktober.nl Thu Dec 16 15:06:09 2010 From: marko at oktober.nl (Marko Kruijer) Date: Thu, 16 Dec 2010 16:06:09 +0100 Subject: unset cookie in vcl_recv not working Message-ID: <4D0A2AE1.3010106@oktober.nl> I'm trying to set up varnish and I'm struggling with the vcl configuration. I took the default.vcl and modified the subroutine vcl_recv, my first goal was to cache pages that send cookies as well. At first I tried to remove a certain cookie, but after many failures I just put 'unset req.http.cookie;' in the vcl_fetch. If my website doesn't send cookies, everything is being cached fine (60 secs at the moment). But When sending cookies, even with my unset cookie config, Varnish does not cache the pages. What approach do I need to use to make this work or debug this? I think I'm missing something obvious, but what? Regards, Marko From traian.bratucu at eea.europa.eu Thu Dec 16 15:10:23 2010 From: traian.bratucu at eea.europa.eu (Traian Bratucu) Date: Thu, 16 Dec 2010 16:10:23 +0100 Subject: unset cookie in vcl_recv not working In-Reply-To: <4D0A2AE1.3010106@oktober.nl> References: <4D0A2AE1.3010106@oktober.nl> Message-ID: You are removing the request cookie there (req.). You should be removing the backend response cookie which is beresp.http.set-cookie. Please read documentation - http://www.varnish-cache.org/trac/wiki/VCLExampleCacheCookies -----Original Message----- From: varnish-misc-bounces at varnish-cache.org [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Marko Kruijer Sent: Thursday, December 16, 2010 4:06 PM To: varnish-misc at varnish-cache.org Subject: unset cookie in vcl_recv not working I'm trying to set up varnish and I'm struggling with the vcl configuration. I took the default.vcl and modified the subroutine vcl_recv, my first goal was to cache pages that send cookies as well. At first I tried to remove a certain cookie, but after many failures I just put 'unset req.http.cookie;' in the vcl_fetch. If my website doesn't send cookies, everything is being cached fine (60 secs at the moment). But When sending cookies, even with my unset cookie config, Varnish does not cache the pages. What approach do I need to use to make this work or debug this? I think I'm missing something obvious, but what? Regards, Marko _______________________________________________ varnish-misc mailing list varnish-misc at varnish-cache.org http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc From perbu at varnish-software.com Thu Dec 16 16:59:30 2010 From: perbu at varnish-software.com (Per Buer) Date: Thu, 16 Dec 2010 17:59:30 +0100 Subject: Realizing vhosts with varnish In-Reply-To: <4D0A1469.9050001@dmsp.de> References: <20101216094424.GA23867@grum.quex.org> <4D0A1469.9050001@dmsp.de> Message-ID: On Thu, Dec 16, 2010 at 2:30 PM, Stefan Pommerening wrote: > > of couse you only need one single vcl_recv subroutine. This applies to all > vcl subroutines. > I usually start with grabbing the default vcl and then add everything I > need to the existing (default) subroutines. > > If you use the include statement you have to keep in mind that inclusion is > a textual > substitution - therefore no repeated definition of vcl_recv (or even other > subroutines) > is allowed. > Actually, multiple definitions of the same subroutine is allowed. They are then concatenated, the flow being terminated only by the return statements. This is how the default VCL get overridden by your custom VCL. There is a good example in the man vcl. Of course, this might get very confusing when you have multiple vcl_recv subroutines spread over several included files. Caution advised. :-) -- Per Buer, Varnish Software Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer Varnish makes websites fly! Want to learn more about Varnish? http://www.varnish-software.com/whitepapers -------------- next part -------------- An HTML attachment was scrubbed... URL: From pom at dmsp.de Thu Dec 16 17:41:52 2010 From: pom at dmsp.de (Stefan Pommerening) Date: Thu, 16 Dec 2010 18:41:52 +0100 Subject: Realizing vhosts with varnish In-Reply-To: References: <20101216094424.GA23867@grum.quex.org> <4D0A1469.9050001@dmsp.de> Message-ID: <4D0A4F60.3030407@dmsp.de> Ok, very good statement - normally I know the override mechanism used by the custom VCL. Nevertheless my last email was incorrect - sorry for that. But: What happens when you include a vcl_recv INSIDE another vcl_recv subroutine? This is what Frank did. Stefan -- http://dmsp.de Am 16.12.2010 17:59, schrieb Per Buer: > On Thu, Dec 16, 2010 at 2:30 PM, Stefan Pommerening > wrote: > > > of couse you only need one single vcl_recv subroutine. This > applies to all vcl subroutines. > I usually start with grabbing the default vcl and then add > everything I need to the existing (default) subroutines. > > If you use the include statement you have to keep in mind that > inclusion is a textual > substitution - therefore no repeated definition of vcl_recv (or > even other subroutines) > is allowed. > > > Actually, multiple definitions of the same subroutine is allowed. They > are then concatenated, the flow being terminated only by the return > statements. This is how the default VCL get overridden by your custom > VCL. > > There is a good example in the man vcl. Of course, this might get very > confusing when you have multiple vcl_recv subroutines spread over > several included files. Caution advised. :-) -------------- next part -------------- An HTML attachment was scrubbed... URL: From perbu at varnish-software.com Thu Dec 16 17:53:11 2010 From: perbu at varnish-software.com (Per Buer) Date: Thu, 16 Dec 2010 18:53:11 +0100 Subject: Realizing vhosts with varnish In-Reply-To: <4D0A4F60.3030407@dmsp.de> References: <20101216094424.GA23867@grum.quex.org> <4D0A1469.9050001@dmsp.de> <4D0A4F60.3030407@dmsp.de> Message-ID: You get a compiler error. The sub keyword is not allowed within a sub. Per. On Thu, Dec 16, 2010 at 6:41 PM, Stefan Pommerening wrote: > Ok, very good statement - normally I know the override mechanism used by > the custom VCL. > Nevertheless my last email was incorrect - sorry for that. > > But: What happens when you include a vcl_recv INSIDE another vcl_recv > subroutine? > This is what Frank did. > > Stefan > -- > http://dmsp.de > > Am 16.12.2010 17:59, schrieb Per Buer: > > On Thu, Dec 16, 2010 at 2:30 PM, Stefan Pommerening wrote: > > >> of couse you only need one single vcl_recv subroutine. This applies to all >> vcl subroutines. >> I usually start with grabbing the default vcl and then add everything I >> need to the existing (default) subroutines. >> >> If you use the include statement you have to keep in mind that inclusion >> is a textual >> substitution - therefore no repeated definition of vcl_recv (or even other >> subroutines) >> is allowed. >> > > Actually, multiple definitions of the same subroutine is allowed. They > are then concatenated, the flow being terminated only by the return > statements. This is how the default VCL get overridden by your custom VCL. > > There is a good example in the man vcl. Of course, this might get very > confusing when you have multiple vcl_recv subroutines spread over several > included files. Caution advised. :-) > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -- Per Buer, Varnish Software Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer Varnish makes websites fly! Want to learn more about Varnish? http://www.varnish-software.com/whitepapers -------------- next part -------------- An HTML attachment was scrubbed... URL: From paulo at aliancaproject.com Thu Dec 16 20:09:01 2010 From: paulo at aliancaproject.com (Paulo Paracatu) Date: Thu, 16 Dec 2010 18:09:01 -0200 Subject: Refreshing modified content Message-ID: Hi. I'm trying Varnish on my servers, and so far I'm quite happy with it. But I have a little problem: I uploaded an image, hit it in my browser, then I modified it and reuploaded it, and tried to hit it again, and the cache serves the old image. It does not serve the new image until the ttl (120s) is expired. I tried the "VCLExampleEnableForceRefresh", but the varnish-cache is only refreshed with ctrl-f5/no-cache header. Is this the normal behavior of varnish, or is it possible to 'revalidate' the cached content with the backend? PS: My browser's cache is disabled. Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From thebog at gmail.com Thu Dec 16 20:23:15 2010 From: thebog at gmail.com (thebog) Date: Thu, 16 Dec 2010 21:23:15 +0100 Subject: Fwd: Refreshing modified content In-Reply-To: References: Message-ID: ---------- Forwarded message ---------- From: thebog Date: Thu, Dec 16, 2010 at 9:22 PM Subject: Re: Refreshing modified content To: Paulo Paracatu The "best" thing to do in this case is to purge the object from the cache. (http://www.varnish-cache.org/docs/trunk/tutorial/purging.html) Or change the URL :) YS Anders Berg On Thu, Dec 16, 2010 at 9:09 PM, Paulo Paracatu wrote: > Hi. I'm trying Varnish on my servers, and so far I'm quite happy with it. > But I have a little problem: I uploaded an image, hit it in my browser, then > I modified it and reuploaded it, and tried to hit it again, and the cache > serves the old image. It does not serve the new image until the ttl (120s) > is expired. > I tried the "VCLExampleEnableForceRefresh", but the varnish-cache is only > refreshed with ctrl-f5/no-cache header. Is this the normal behavior of > varnish, or is it possible to 'revalidate' the cached content with the > backend? > PS: My browser's cache is disabled. > Thanks > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > From paulo at aliancaproject.com Thu Dec 16 20:31:00 2010 From: paulo at aliancaproject.com (Paulo Paracatu) Date: Thu, 16 Dec 2010 18:31:00 -0200 Subject: Refreshing modified content In-Reply-To: References: Message-ID: If I understood it, the purging method isn't automatic, right? I'd need to purge the content everytime it is modified. This is kinda stupid... I host more than 10k sites, modifying files everytime. If I set a high TTL, the backend will be happy and the webmaster will be angry. If I set a low TTL, the webmaster will be happy, but the backend will die. Plus, there is no point using a cache if the TTL is low. 2010/12/16 thebog > ---------- Forwarded message ---------- > From: thebog > Date: Thu, Dec 16, 2010 at 9:22 PM > Subject: Re: Refreshing modified content > To: Paulo Paracatu > > > The "best" thing to do in this case is to purge the object from the > cache. (http://www.varnish-cache.org/docs/trunk/tutorial/purging.html) > > Or change the URL :) > > > YS > Anders Berg > > > On Thu, Dec 16, 2010 at 9:09 PM, Paulo Paracatu > wrote: > > Hi. I'm trying Varnish on my servers, and so far I'm quite happy with it. > > But I have a little problem: I uploaded an image, hit it in my browser, > then > > I modified it and reuploaded it, and tried to hit it again, and the cache > > serves the old image. It does not serve the new image until the ttl > (120s) > > is expired. > > I tried the "VCLExampleEnableForceRefresh", but the varnish-cache is only > > refreshed with ctrl-f5/no-cache header. Is this the normal behavior of > > varnish, or is it possible to 'revalidate' the cached content with the > > backend? > > PS: My browser's cache is disabled. > > Thanks > > _______________________________________________ > > varnish-misc mailing list > > varnish-misc at varnish-cache.org > > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From perbu at varnish-software.com Thu Dec 16 20:54:10 2010 From: perbu at varnish-software.com (Per Buer) Date: Thu, 16 Dec 2010 21:54:10 +0100 Subject: Refreshing modified content In-Reply-To: References: Message-ID: Hi, On Thu, Dec 16, 2010 at 9:31 PM, Paulo Paracatu wrote: > If I understood it, the purging method isn't automatic, right? I'd need to > purge the content everytime it is modified. Hi, Nobody purges the cache manually. You get your CMS to purge the cache for you. This is kinda stupid... I host more than 10k sites, modifying files > everytime. If I set a high TTL, the backend will be happy and the webmaster > will be angry. If I set a low TTL, the webmaster will be happy, but the > backend will die. Plus, there is no point using a cache if the TTL is low. > It isn't stupid, computers don't have intuition and you have to actually tell them when you update data. If you can propose another way of getting the cache to magically purge itself of stale content please share it with us. :-) -- Per Buer, Varnish Software Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer Varnish makes websites fly! Want to learn more about Varnish? http://www.varnish-software.com/whitepapers -------------- next part -------------- An HTML attachment was scrubbed... URL: From paulo at aliancaproject.com Thu Dec 16 21:26:28 2010 From: paulo at aliancaproject.com (Paulo Paracatu) Date: Thu, 16 Dec 2010 19:26:28 -0200 Subject: Refreshing modified content In-Reply-To: References: Message-ID: Hi, I read somewhere about backend conditional requests, or GET IMS. As I understand, that would be exactly what I need. I'd be able to set a low TTL and yet keep the BW beetween backend-varnish low. And sorry if I sound rude. :P 2010/12/16 Per Buer > Hi, > > On Thu, Dec 16, 2010 at 9:31 PM, Paulo Paracatu wrote: > >> If I understood it, the purging method isn't automatic, right? I'd need to >> purge the content everytime it is modified. > > > Hi, > Nobody purges the cache manually. You get your CMS to purge the cache for > you. > > This is kinda stupid... I host more than 10k sites, modifying files >> everytime. If I set a high TTL, the backend will be happy and the webmaster >> will be angry. If I set a low TTL, the webmaster will be happy, but the >> backend will die. Plus, there is no point using a cache if the TTL is low. >> > > It isn't stupid, computers don't have intuition and you have to actually > tell them when you update data. If you can propose another way of getting > the cache to magically purge itself of stale content please share it with > us. :-) > > -- > Per Buer, Varnish Software > Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer > Varnish makes websites fly! > Want to learn more about Varnish? > http://www.varnish-software.com/whitepapers > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ionathan at gmail.com Thu Dec 16 21:33:52 2010 From: ionathan at gmail.com (Jonathan Leibiusky) Date: Thu, 16 Dec 2010 18:33:52 -0300 Subject: Refreshing modified content In-Reply-To: References: Message-ID: How about using the If-Modified-Since header? If the backend server interpret this header and answers 304, maybe you could configure varnish to give back the cached resource. Of course your client should send that in the request. Or maybe there is a way to tell varnish to send internally a conditional request every once in a while? Jonathan On 12/16/10, Per Buer wrote: > Hi, > > On Thu, Dec 16, 2010 at 9:31 PM, Paulo Paracatu > wrote: > >> If I understood it, the purging method isn't automatic, right? I'd need to >> purge the content everytime it is modified. > > > Hi, > Nobody purges the cache manually. You get your CMS to purge the cache for > you. > > This is kinda stupid... I host more than 10k sites, modifying files >> everytime. If I set a high TTL, the backend will be happy and the >> webmaster >> will be angry. If I set a low TTL, the webmaster will be happy, but the >> backend will die. Plus, there is no point using a cache if the TTL is low. >> > > It isn't stupid, computers don't have intuition and you have to actually > tell them when you update data. If you can propose another way of getting > the cache to magically purge itself of stale content please share it with > us. :-) > > -- > Per Buer, Varnish Software > Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer > Varnish makes websites fly! > Want to learn more about Varnish? > http://www.varnish-software.com/whitepapers > From kova70 at gmail.com Thu Dec 16 23:03:16 2010 From: kova70 at gmail.com (Raymond Hall) Date: Thu, 16 Dec 2010 17:03:16 -0600 Subject: Refreshing modified content In-Reply-To: References: Message-ID: If you're using apache and mod_expires, you can cusomize you TTLs pretty specifically, per file type, mime and directory. With that you can settle for a set of compromise TTLs that suit everybody. Than again, you can use a proxy aware CMS like pressflow. regards. Ray On Thu, Dec 16, 2010 at 3:26 PM, Paulo Paracatu wrote: > Hi, > I read somewhere about backend conditional requests, or GET IMS. As I > understand, that would be exactly what I need. I'd be able to set a low TTL > and yet keep the BW beetween backend-varnish low. > And sorry if I sound rude. :P > > 2010/12/16 Per Buer >> >> Hi, >> On Thu, Dec 16, 2010 at 9:31 PM, Paulo Paracatu >> wrote: >>> >>> If I understood it, the purging method isn't automatic, right? I'd need >>> to purge the content everytime it is modified. >> >> Hi, >> Nobody purges the cache manually. You get your CMS to purge the cache for >> you. >>> >>> This is kinda stupid... I host more than 10k sites, modifying files >>> everytime. If I set a high TTL, the backend will be happy and the webmaster >>> will be angry. If I set a low TTL, the webmaster will be happy, but the >>> backend will die. Plus, there is no point using a cache if the TTL is low. >> >> It isn't stupid, computers don't have?intuition?and you have to actually >> tell them when you update data. If you can propose another way of getting >> the cache to magically purge itself of stale content please share it with >> us. :-) >> -- >> Per Buer,?Varnish Software >> Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer >> Varnish makes websites fly! >> Want to learn more about Varnish? >> http://www.varnish-software.com/whitepapers >> >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -- Knowingly entering a Ponzi scheme can be rational, in the economic sense, even at the last round of the scheme if a government will likely bail out those participating in the Ponzi scheme. The IPAB Posse From indranilc at rediff-inc.com Fri Dec 17 06:18:37 2010 From: indranilc at rediff-inc.com (Indranil Chakravorty) Date: 17 Dec 2010 06:18:37 -0000 Subject: =?utf-8?B?U2Vzc2lvbkNsb3NlLXRpbWVvdXQgZXJyb3Jz?= Message-ID: <1292495933.S.7813.H.TkFuYW5kIFNoYWgAdmFybmlzaCBtYWlsaW5nIGxpc3Q_.33761.pro-237-108.old.1292566717.28493@webmail.rediffmail.com> Hi All,      We have been noticing a number of such errors. Could anyone please let me know if it indicates a problem. Thanks. 2169 SessionClose - timeout 2169 StatSess - 117.192.68.128 50070 13 1 1 0 0 0 25775102154 SessionClose - timeout2154 StatSess - 59.95.73.47 1349 14 1 1 0 0 0 297 432111 SessionClose - timeout2111 StatSess - 210.212.160.101 35335 17 1 1 0 0 0 25723692173 SessionClose - timeout2173 StatSess - 59.180.73.179 2964 13 1 1 0 0 0 25754632186 SessionClose - timeout2186 StatSess - 59.162.19.3 56970 13 1 1 0 0 0 230 02085 SessionClose - timeout2085 StatSess - 59.95.84.71 65489 17 1 1 0 0 1 367779702179 SessionClose - timeout2179 StatSess - 59.92.166.95 4211 13 1 1 0 0 0 345 51812194 SessionClose - timeout2194 StatSess - 59.95.190.97 1580 13 1 1 0 0 0 257 19882181 SessionClose - timeout2181 StatSess - 59.92.166.95 4212 13 1 1 0 0 0 230 01227 SessionClose - timeout -------------- next part -------------- An HTML attachment was scrubbed... URL: From fhelmschrott at gmail.com Fri Dec 17 06:57:27 2010 From: fhelmschrott at gmail.com (Frank Helmschrott) Date: Fri, 17 Dec 2010 07:57:27 +0100 Subject: Trouble using varnishadm Message-ID: Hi, as said in my last thread i'm just in the beginning of exploring varnish. As i read about people mostly using varnishadm to check there new vcls and testing them before they activate it, i wanted to try myself and connect to varnish via varnishadm. Unfortunately this happens to be much more complicated than i thought. All i get back is the 'usage' line. Here's my current setup. Varnishd is started by the RHEL startup scripts with these daemon_opts: DAEMON_OPTS="-a :80 \ -T :6082 \ -f /etc/varnish/default.vcl \ -u varnish -g varnish \ -s file,/var/lib/varnish/varnish_storage.bin,64G" i've changed -T from localhost:6082 to :6082 as i thought this is needed to allow purging from other hosts. Is this true? ACL for purging is set in my VCL file. When i now try to connect with varnishadm i get this result: [root at cache01 varnish]# varnishadm -T localhost:6082 usage: varnishadm -T [address]:port command [...] And then back to the shell. From what i understood is that if i don't give a command it should connect and give an interactive in/out from STD OUT. That's at least what Docs for 2.1 say - i have 2.0.6 but i don't see any possible changes in the doc. Any help is appreciated. Think this is basic learning stuff i should know before starting over :) -- Frank From matthew.hoopes at gmail.com Fri Dec 17 04:28:36 2010 From: matthew.hoopes at gmail.com (Matthew Hoopes) Date: Thu, 16 Dec 2010 23:28:36 -0500 Subject: Refreshing modified content In-Reply-To: References: Message-ID: Although it does seem a little crazy, we set up a little job (with the proper ssh keys and such) to automatically call out to our proxy boxes and "manually" purge the cache on any image upload. It wasn't really huge overhead or anything, and it really worked like a champ. That way, you can set the TTL pretty high, and when the file is replaced, it shows up right away. I personally like auto-ssh to the proxy and use netcat to call to the admin interface instead of using the PURGE HTTP call. It gives you regex control, and you can control the hostname and path pretty specifically. YMMV, of course. On Thu, Dec 16, 2010 at 4:26 PM, Paulo Paracatu wrote: > Hi, > > I read somewhere about backend conditional requests, or GET IMS. As I > understand, that would be exactly what I need. I'd be able to set a low TTL > and yet keep the BW beetween backend-varnish low. > > And sorry if I sound rude. :P > > 2010/12/16 Per Buer > >> Hi, >> >> On Thu, Dec 16, 2010 at 9:31 PM, Paulo Paracatu > > wrote: >> >>> If I understood it, the purging method isn't automatic, right? I'd need >>> to purge the content everytime it is modified. >> >> >> Hi, >> Nobody purges the cache manually. You get your CMS to purge the cache for >> you. >> >> This is kinda stupid... I host more than 10k sites, modifying files >>> everytime. If I set a high TTL, the backend will be happy and the webmaster >>> will be angry. If I set a low TTL, the webmaster will be happy, but the >>> backend will die. Plus, there is no point using a cache if the TTL is low. >>> >> >> It isn't stupid, computers don't have intuition and you have to actually >> tell them when you update data. If you can propose another way of getting >> the cache to magically purge itself of stale content please share it with >> us. :-) >> >> -- >> Per Buer, Varnish Software >> Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer >> Varnish makes websites fly! >> Want to learn more about Varnish? >> http://www.varnish-software.com/whitepapers >> >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kristian at varnish-software.com Fri Dec 17 07:49:31 2010 From: kristian at varnish-software.com (Kristian Lyngstol) Date: Fri, 17 Dec 2010 08:49:31 +0100 Subject: Realizing vhosts with varnish In-Reply-To: <4D0A1469.9050001@dmsp.de> References: <20101216094424.GA23867@grum.quex.org> <4D0A1469.9050001@dmsp.de> Message-ID: <20101217074930.GA2178@freud> On Thu, Dec 16, 2010 at 02:30:17PM +0100, Stefan Pommerening wrote: > If you use the include statement you have to keep in mind that > inclusion is a textual substitution - therefore no repeated definition of > vcl_recv (or even other subroutines) is allowed. This is incorrect. You can define all routines as many times as you want. They are executed in the order they are included and - most importantly - they STOP executing at the first return(). The very last VCL "included" is the default VCL - that is why you do not need to copy the logic on the default VCL unless you are planning on return()ing in your custom VCL: unless you issue return(), VCL execution continues until all (including the default) variants of a subroutine is executed. If you define include inside of vcl_recv, then you can not have any sub's in the included file. The reason is simple: you can't define vcl_recv inside vcl_recv. - Kristian -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: Digital signature URL: From kristian at varnish-software.com Fri Dec 17 07:51:24 2010 From: kristian at varnish-software.com (Kristian Lyngstol) Date: Fri, 17 Dec 2010 08:51:24 +0100 Subject: Realizing vhosts with varnish In-Reply-To: <20101217074930.GA2178@freud> References: <20101216094424.GA23867@grum.quex.org> <4D0A1469.9050001@dmsp.de> <20101217074930.GA2178@freud> Message-ID: <20101217075123.GB2178@freud> On Fri, Dec 17, 2010 at 08:49:31AM +0100, Kristian Lyngstol wrote: > This is incorrect. I just realized Per already answered this - sorry about the duplicated response. (It seems to be popular in this thread). - Kristian -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: Digital signature URL: From fhelmschrott at gmail.com Fri Dec 17 07:56:50 2010 From: fhelmschrott at gmail.com (Frank Helmschrott) Date: Fri, 17 Dec 2010 08:56:50 +0100 Subject: Realizing vhosts with varnish In-Reply-To: <20101217075123.GB2178@freud> References: <20101216094424.GA23867@grum.quex.org> <4D0A1469.9050001@dmsp.de> <20101217074930.GA2178@freud> <20101217075123.GB2178@freud> Message-ID: I just found out that probably all i really need is to switch backends based on the req.http.Host called or based on IP (i gotta find out which is more useful for me). Most of the other stuff should in my case be nearly the same for alle projects handled by varnish. 2010/12/17 Kristian Lyngstol : > On Fri, Dec 17, 2010 at 08:49:31AM +0100, Kristian Lyngstol wrote: >> This is incorrect. > > I just realized Per already answered this - sorry about the duplicated > response. (It seems to be popular in this thread). > > - Kristian > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.10 (GNU/Linux) > > iQEcBAEBAgAGBQJNCxZ6AAoJEIC1IG5rXEfqy4IH/Rq+2qhfiI52QtSiB/BTepsV > cZogjfXU2unrtze9tyrG2TLpgJIQVh4X1CKsmsu0G5zRwfQhv7u4jHa3f2aao/Mq > q0r4p+PDqyaCfxuihxIrdi7a6bYARbTnKsmQjequBJBhhr9Gk9wJibrZvPxXvVIf > HxhtkhjWmWjETxBTOy91II2xn/e14ONAAk2PMCR5t5sO4FkerEqqXRMtmOLBJIFu > ETmwtATQdftANR6hI8TAcQKwdVzgCdmyBFzu/gPzOKufkdWmQ+GLXd2sSW90aQ7i > msbWqFlj3KSuVpTBUwv7JUBdlD3sskYbcyKDPSywJWp7jeRau8nm/dBeMQn5NGw= > =M5Rz > -----END PGP SIGNATURE----- > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -- Frank From bjorn at ruberg.no Fri Dec 17 08:02:46 2010 From: bjorn at ruberg.no (=?ISO-8859-1?Q?Bj=F8rn_Ruberg?=) Date: Fri, 17 Dec 2010 09:02:46 +0100 Subject: Trouble using varnishadm In-Reply-To: References: Message-ID: <4D0B1926.4030004@ruberg.no> [...] > When i now try to connect with > varnishadm i get this result: > > [root at cache01 varnish]# varnishadm -T localhost:6082 > usage: varnishadm -T [address]:port command [...] > > And then back to the shell. From what i understood is that if i don't > give a command it should connect and give an interactive in/out from > STD OUT. That's at least what Docs for 2.1 say - i have 2.0.6 but i > don't see any possible changes in the doc. If you had actually read the usage output you'd have seen that 'command' is not in square brackets, which means it's mandatory. Ergo, with your version of varnishadm, the command is required on the command line and interactive usage is probably not possible. You should not assume that whatever's working in 2.1 will apply to 2.0 just because it's not mentioned in the documentation for 2.1. You should use the documentation for your version. Lacking that, "man varnishadm" on your system would be quite useful. -- Bj?rn From kristian at varnish-software.com Fri Dec 17 08:04:36 2010 From: kristian at varnish-software.com (Kristian Lyngstol) Date: Fri, 17 Dec 2010 09:04:36 +0100 Subject: SessionClose-timeout errors In-Reply-To: <1292495933.S.7813.H.TkFuYW5kIFNoYWgAdmFybmlzaCBtYWlsaW5nIGxpc3Q_.33761.pro-237-108.old.1292566717.28493@webmail.rediffmail.com> References: <1292495933.S.7813.H.TkFuYW5kIFNoYWgAdmFybmlzaCBtYWlsaW5nIGxpc3Q_.33761.pro-237-108.old.1292566717.28493@webmail.rediffmail.com> Message-ID: <20101217080435.GC2178@freud> On Fri, Dec 17, 2010 at 06:18:37AM -0000, Indranil Chakravorty wrote: > Hi All,      We have been noticing a number of such > errors. Could anyone please let me know if it indicates a problem. > Thanks. It's not a problem. The clients keep connections open for a while to be able to re-use them (ie: keep-alive). If they have no further use of them, they will time out (default timeout is 4s). - Kristian PS: HTML mail isn't very practical... -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: Digital signature URL: From pom at dmsp.de Fri Dec 17 09:51:13 2010 From: pom at dmsp.de (Stefan Pommerening) Date: Fri, 17 Dec 2010 10:51:13 +0100 Subject: Varnish 2.1.3 nuking objects from cache Message-ID: <4D0B3291.1080506@dmsp.de> Hi, Until today I thought that when varnish (2.1.3) is nuking objects from cache (counter n_lru_nuked increasing) the reason must be that the cache is full. My strange situation is: - varnish 2.1.3 configured -s file,,20GB - monitoring using varnishstat -f sm_balloc, sm_bfree - ~ 15 GB cache used, ~ 5 GB cache free - counter n_lru_nuked increasing every now and then (not rapidly, but occasionally although cache usage is stable at about 75% now) What else (other than a full cache) can be the reason for objects being nuked from cache? (not expired) I've read that there was a bug in varnish 2.0.6 with some similar behaviour, but I am using 2.1.3. Stefan -- http://dmsp.de From ITLJ at gyldendal.dk Fri Dec 17 11:27:23 2010 From: ITLJ at gyldendal.dk (=?iso-8859-1?Q?Lars_J=F8rgensen?=) Date: Fri, 17 Dec 2010 12:27:23 +0100 Subject: Realizing vhosts with varnish In-Reply-To: References: <20101216094424.GA23867@grum.quex.org> <4D0A1469.9050001@dmsp.de> <20101217074930.GA2178@freud> <20101217075123.GB2178@freud> Message-ID: <311386E60ED7E74C8716A6573A7740A01870F17775@s-exch01-v.gyldendal.local> Hi Frank, I don't what you're trying to achieve on the differentiated caching. If you're simply looking to cache some virtual hosts and not others on the same physical host, why don't you just do it in DNS? The cached host names point to varnish, the uncached directly to the backend. Lars -----Original Message----- From: varnish-misc-bounces at varnish-cache.org [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Frank Helmschrott Sent: Friday, December 17, 2010 8:57 AM To: Stefan Pommerening; varnish-misc at varnish-cache.org Subject: Re: Realizing vhosts with varnish I just found out that probably all i really need is to switch backends based on the req.http.Host called or based on IP (i gotta find out which is more useful for me). Most of the other stuff should in my case be nearly the same for alle projects handled by varnish. 2010/12/17 Kristian Lyngstol : > On Fri, Dec 17, 2010 at 08:49:31AM +0100, Kristian Lyngstol wrote: >> This is incorrect. > > I just realized Per already answered this - sorry about the duplicated > response. (It seems to be popular in this thread). > > - Kristian > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.10 (GNU/Linux) > > iQEcBAEBAgAGBQJNCxZ6AAoJEIC1IG5rXEfqy4IH/Rq+2qhfiI52QtSiB/BTepsV > cZogjfXU2unrtze9tyrG2TLpgJIQVh4X1CKsmsu0G5zRwfQhv7u4jHa3f2aao/Mq > q0r4p+PDqyaCfxuihxIrdi7a6bYARbTnKsmQjequBJBhhr9Gk9wJibrZvPxXvVIf > HxhtkhjWmWjETxBTOy91II2xn/e14ONAAk2PMCR5t5sO4FkerEqqXRMtmOLBJIFu > ETmwtATQdftANR6hI8TAcQKwdVzgCdmyBFzu/gPzOKufkdWmQ+GLXd2sSW90aQ7i > msbWqFlj3KSuVpTBUwv7JUBdlD3sskYbcyKDPSywJWp7jeRau8nm/dBeMQn5NGw= > =M5Rz > -----END PGP SIGNATURE----- > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -- Frank _______________________________________________ varnish-misc mailing list varnish-misc at varnish-cache.org http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc From marko at oktober.nl Fri Dec 17 14:23:35 2010 From: marko at oktober.nl (Marko Kruijer) Date: Fri, 17 Dec 2010 15:23:35 +0100 Subject: different cache per user agent Message-ID: <4D0B7267.1030609@oktober.nl> I've got Varnish working after some struggling, for testing purposes I've set the objects ttl to 15 secs. I noticed that when calling a page from Firefox, and immediately after that from Internet Explorer, the served page is different. They are however cached for 15 secs. Is this default for Varnish? If I'm correct it is not, there are NO cookies set. What can cause this behaviour? Regards, Marko From pom at dmsp.de Fri Dec 17 14:39:03 2010 From: pom at dmsp.de (Stefan Pommerening) Date: Fri, 17 Dec 2010 15:39:03 +0100 Subject: different cache per user agent In-Reply-To: <4D0B7267.1030609@oktober.nl> References: <4D0B7267.1030609@oktober.nl> Message-ID: <4D0B7607.3060605@dmsp.de> Hi Marko, sounds like your web server sends a vary: header for user-agent. This might be the reason that you have different variants of the same object in cache. Look here: http://www.varnish-cache.org/docs/trunk/tutorial/vary.html Stefan -- http://dmsp.de Am 17.12.2010 15:23, schrieb Marko Kruijer: > I've got Varnish working after some struggling, for testing purposes > I've set the objects ttl to 15 secs. > > I noticed that when calling a page from Firefox, and immediately after > that from Internet Explorer, the served page is different. They are > however cached for 15 secs. > > Is this default for Varnish? If I'm correct it is not, there are NO > cookies set. What can cause this behaviour? > > Regards, > Marko From stig at zedge.net Fri Dec 17 14:40:35 2010 From: stig at zedge.net (Stig Bakken) Date: Fri, 17 Dec 2010 15:40:35 +0100 Subject: hanging connections Message-ID: Hi list, I am trying out varnish on some boxes mostly serving static files with Apache. It's almost working smoothly, but sometimes we have requests that seem to take an inordinate amount of time. The backend servers are running with keepalives disabled (since there was a bug related to keepalives in varnish 2.1.4). I have tried running 2.1.3, 2.1.4 and yesterday's trunk, all with the same result. I have a script processing varnishlog output to give me the serving latency based on the ReqEnd tag, and frequently see objects served from cache, size around 25kB, that take 5+ seconds to serve. At first I hypothesized that this included the time required to send enough packets to the client that the remaining data was in kernel buffers and varnish considered the request done. To test this, I wrote a test client that fired off a request to some cached object, slept 10 seconds, then read headers and body. From varnishlog's point of view, this request took >1ms to complete, so that was an incorrect assumption. So right now I'm out of ideas. Has anyone run into this before? - Stig /etc/sysconfig/varnish: NFILES=131072 MEMLOCK=82000 DAEMON_OPTS="\ -T localhost:6082 \ -f /etc/varnish/fs.vcl \ -u varnish \ -g varnish \ -S /etc/varnish/secret \ -p thread_pools=8 \ -p thread_pool_add_delay=2 \ -p thread_pool_min=100 \ -p thread_pool_max=20000 \ -p thread_pool_timeout=300 \ -t 86400 \ -s malloc,8G" VCL file: backend default { .host = "127.0.0.1"; .port = "8088"; } sub vcl_recv { if (req.request != "GET" && req.request != "HEAD" && req.request != "PUT" && req.request != "POST" && req.request != "TRACE" && req.request != "OPTIONS" && req.request != "DELETE") { /* Non-RFC2616 or CONNECT which is weird. */ return (pipe); } if (req.restarts == 0) { if (req.http.x-forwarded-for) { set req.http.X-Forwarded-For = req.http.X-Forwarded-For ", " client.ip; } else { set req.http.X-Forwarded-For = client.ip; } } if (req.request != "GET") { return (pass); } if (req.url ~ "-t\.(jpg|jpeg|gif|png)$" || req.url ~ "^/content-management\.php\?(zth|zcr)=" ) { remove req.http.Cookie; return (lookup); } if (req.url ~ "^/video-streamer\.php") { return (pipe); } return (pass); } sub vcl_pipe { set bereq.http.Connection = "close"; } sub vcl_fetch { remove beresp.http.ETag; } An in case any of this matters, sysctl stuff: net.core.rmem_max=16777216 net.core.wmem_max=16777216 net.ipv4.tcp_rmem=4096 87380 16777216 net.ipv4.tcp_wmem=4096 65536 16777216 net.ipv4.tcp_fin_timeout = 3 net.ipv4.tcp_tw_recycle = 1 net.core.netdev_max_backlog = 30000 net.ipv4.tcp_no_metrics_save=1 net.core.somaxconn = 262144 net.ipv4.tcp_syncookies = 1 net.ipv4.tcp_max_orphans = 262144 net.ipv4.tcp_max_syn_backlog = 262144 net.ipv4.tcp_synack_retries = 2 net.ipv4.tcp_syn_retries = 2 -- Stig Bakken CTO, Zedge.net - free your phone! -------------- next part -------------- An HTML attachment was scrubbed... URL: From josh at schulzone.org Fri Dec 17 15:23:24 2010 From: josh at schulzone.org (Josh) Date: Fri, 17 Dec 2010 08:23:24 -0700 Subject: different cache per user agent In-Reply-To: <4D0B7267.1030609@oktober.nl> References: <4D0B7267.1030609@oktober.nl> Message-ID: Check the vary-by coming back from the backend and verify that you're really not setting any cookies. I saw this same thing and realized it was cookies being set by analytics javascript on page. Watch requests in something like fiddler and see what you're getting back. On Fri, Dec 17, 2010 at 7:23 AM, Marko Kruijer wrote: > I've got Varnish working after some struggling, for testing purposes I've > set the objects ttl to 15 secs. > > I noticed that when calling a page from Firefox, and immediately after that > from Internet Explorer, the served page is different. They are however > cached for 15 secs. > > Is this default for Varnish? If I'm correct it is not, there are NO cookies > set. What can cause this behaviour? > > Regards, > Marko > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -- josh @schulz http://schulzone.org From marko at oktober.nl Fri Dec 17 15:36:37 2010 From: marko at oktober.nl (Marko Kruijer) Date: Fri, 17 Dec 2010 16:36:37 +0100 Subject: different cache per user agent In-Reply-To: References: <4D0B7267.1030609@oktober.nl> Message-ID: <4D0B8385.1050507@oktober.nl> I've found the problem already. My backend was gzipping content, this somehow made the browsers cache differently. On 17-12-2010 16:23, Josh wrote: > Check the vary-by coming back from the backend and verify that you're > really not setting any cookies. I saw this same thing and realized it > was cookies being set by analytics javascript on page. Watch requests > in something like fiddler and see what you're getting back. > > On Fri, Dec 17, 2010 at 7:23 AM, Marko Kruijer wrote: >> I've got Varnish working after some struggling, for testing purposes I've >> set the objects ttl to 15 secs. >> >> I noticed that when calling a page from Firefox, and immediately after that >> from Internet Explorer, the served page is different. They are however >> cached for 15 secs. >> >> Is this default for Varnish? If I'm correct it is not, there are NO cookies >> set. What can cause this behaviour? >> >> Regards, >> Marko >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> > > From jim.robinson at stanford.edu Fri Dec 17 15:42:34 2010 From: jim.robinson at stanford.edu (James A. Robinson) Date: Fri, 17 Dec 2010 07:42:34 -0800 Subject: Refreshing modified content In-Reply-To: References: Message-ID: On Thu, Dec 16, 2010 at 12:31, Paulo Paracatu wrote: > If I understood it, the purging method isn't automatic, right? I'd need to > purge the content everytime it is modified. > This is kinda stupid... I host more than 10k sites, modifying files > everytime. If I set a high TTL, the backend will be happy and the webmaster > will be angry. If I set a low TTL, the webmaster will be happy, but the > backend will die. Plus, there is no point using a cache if the TTL is low. In a later post you ask about whether or not varnish could be configured to send a conditional GET on every request Basically varnish would be looking up an item in its own cache, seeing if it had a Last-Modified or ETag header from the backend, and sending a conditional GET -- if it got an entity back it'd store that entity as the new version, otherwise serve the old. I'd be curious if anyone's put together VCL logic that is capable of that. It'd be good to know how to do it. If it's possible to due, this technique might work well when you are fronting a backend that is very fast at computing conditional GETs, e.g., static files that can be examined to see if its inode, size, or last modified time has changed. I imagine most people using varnish have slower backend servers, ones that build dynamic content and aren't able to respond to conditional GETs any more efficiently than they could respond to an unqualified GET. One of the places we use varnish at my dept is fronting a large (half terabyte of about six million files) static file server. Instead of using some form of conditional GET, what we use are cache channels. http://tools.ietf.org/id/draft-nottingham-http-cache-channels-01.txt When various programs are updating the backend server static files they POST the filepath to our cache channel server. We have another program running that monitors the cache channel once a minute for updates, and when it sees a new entry show up it turns around and sends a PURGE request to varnish. Jim - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - James A. Robinson jim.robinson at stanford.edu Stanford University HighWire Press http://highwire.stanford.edu/ +1 650 7237294 (Work) +1 650 7259335 (Fax) From ionathan at gmail.com Fri Dec 17 15:50:48 2010 From: ionathan at gmail.com (Jonathan Leibiusky) Date: Fri, 17 Dec 2010 12:50:48 -0300 Subject: Refreshing modified content In-Reply-To: References: Message-ID: I think that conditional GETs could be really useful here. I just don't know how varnish works in this scenario. Maybe someone from the varnish development team can shed some light here :) On Fri, Dec 17, 2010 at 12:42 PM, James A. Robinson < jim.robinson at stanford.edu> wrote: > On Thu, Dec 16, 2010 at 12:31, Paulo Paracatu > wrote: > > If I understood it, the purging method isn't automatic, right? I'd need > to > > purge the content everytime it is modified. > > This is kinda stupid... I host more than 10k sites, modifying files > > everytime. If I set a high TTL, the backend will be happy and the > webmaster > > will be angry. If I set a low TTL, the webmaster will be happy, but the > > backend will die. Plus, there is no point using a cache if the TTL is > low. > > In a later post you ask about whether or not varnish could be > configured to send a conditional GET on every request > > Basically varnish would be looking up an item in its own cache, seeing > if it had a Last-Modified or ETag header from the backend, and sending > a conditional GET -- if it got an entity back it'd store that entity > as the new version, otherwise serve the old. I'd be curious if > anyone's put together VCL logic that is capable of that. It'd be good > to know how to do it. > > If it's possible to due, this technique might work well when you are > fronting a backend that is very fast at computing conditional GETs, > e.g., static files that can be examined to see if its inode, size, or > last modified time has changed. I imagine most people using varnish > have slower backend servers, ones that build dynamic content and > aren't able to respond to conditional GETs any more efficiently than > they could respond to an unqualified GET. > > One of the places we use varnish at my dept is fronting a large (half > terabyte of about six million files) static file server. Instead of > using some form of conditional GET, what we use are cache channels. > > http://tools.ietf.org/id/draft-nottingham-http-cache-channels-01.txt > > When various programs are updating the backend server static files > they POST the filepath to our cache channel server. We have another > program running that monitors the cache channel once a minute for > updates, and when it sees a new entry show up it turns around and > sends a PURGE request to varnish. > > Jim > > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > James A. Robinson jim.robinson at stanford.edu > Stanford University HighWire Press http://highwire.stanford.edu/ > +1 650 7237294 (Work) +1 650 7259335 (Fax) > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From perbu at varnish-software.com Sat Dec 18 09:05:55 2010 From: perbu at varnish-software.com (Per Buer) Date: Sat, 18 Dec 2010 10:05:55 +0100 Subject: different cache per user agent In-Reply-To: <4D0B8385.1050507@oktober.nl> References: <4D0B7267.1030609@oktober.nl> <4D0B8385.1050507@oktober.nl> Message-ID: Check out "using varnish" in the docs. It has a recipe on normalizing accept-encoding. Per. On Fri, Dec 17, 2010 at 4:36 PM, Marko Kruijer wrote: > I've found the problem already. My backend was gzipping content, this > somehow made the browsers cache differently. > > On 17-12-2010 16:23, Josh wrote: >> >> Check the vary-by coming back from the backend and verify that you're >> really not setting any cookies. ?I saw this same thing and realized it >> was cookies being set by analytics javascript on page. ?Watch requests >> in something like fiddler and see what you're getting back. >> >> On Fri, Dec 17, 2010 at 7:23 AM, Marko Kruijer ?wrote: >>> >>> I've got Varnish working after some struggling, for testing purposes I've >>> set the objects ttl to 15 secs. >>> >>> I noticed that when calling a page from Firefox, and immediately after >>> that >>> from Internet Explorer, the served page is different. They are however >>> cached for 15 secs. >>> >>> Is this default for Varnish? If I'm correct it is not, there are NO >>> cookies >>> set. What can cause this behaviour? >>> >>> Regards, >>> Marko >>> >>> _______________________________________________ >>> varnish-misc mailing list >>> varnish-misc at varnish-cache.org >>> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >>> >> >> > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -- Per Buer,?Varnish Software Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer Varnish makes websites fly! From mrtopf at gmail.com Sun Dec 19 17:23:50 2010 From: mrtopf at gmail.com (Christian Scholz) Date: Sun, 19 Dec 2010 18:23:50 +0100 Subject: Problem with 304 responses from backend (results in 503 errors) Message-ID: Hi! I am using?varnishd (varnish-2.1.4 SVN 5447M) and I am having a problem if the backend returns a 304 error as this results in varnish hanging for 60s and then returning a 503. I put the logfile at https://gist.github.com/33a1dd74a352745d7cc2, and the relevant lines are this (done with varnishlog -o): 14 ObjProtocol c HTTP/1.1 14 ObjStatus c 304 14 ObjResponse c Not Modified 14 ObjHeader c Server: Zope/(2.12.11, python 2.6.5, linux2) ZServer/1.1 14 ObjHeader c Date: Sun, 19 Dec 2010 17:02:18 GMT 14 ObjHeader c X-Cache-Rule: product.simplelayout 14 ObjHeader c ETag: |Authenticated;Internal;Member|****|1292695386.0 14 ObjHeader c X-Cache-Operation: plone.app.caching.weakCaching 14 ObjHeader c X-Varnish-Action: FETCH (pass - not cacheable) 14 FetchError c eof read_error: 11 14 VCL_call c error deliver 14 VCL_call c deliver deliver 14 TxProtocol c HTTP/1.1 14 TxStatus c 503 14 TxResponse c Service Unavailable So the backend returns a 304 without a content-length header. You can see my varnish vcl here: https://gist.github.com/c3211681ce7dc0681bdc The result of this is that every second reload of a page results in a 503. Does somebody know if I am doing something wrong or if that's a bug either in varnish or Plone? Thanks in advance! Christian -- Christian Scholz ? ? ? ? ? ? ? ? ? ? ? ? ?Homepage: http://comlounge.net COM.lounge GmbH ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?http://mrtopf.de/blog Hanbrucher Str. 33 ? ? ? ? ? ? ? ? ? ? ? ? ? ? http://twitter.com/mrtopf 52064 Aachen ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? Skype: HerrTopf Tel: +49 241 400 730 0 ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?cs at comlounge.net Fax: +49 241 979 00 850 ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?IRC: MrTopf Podcasts: Der OpenWeb-Podcast (http://openwebpodcast.de) Data Without Borders (http://datawithoutborders.net) Politisches: http://politfunk.de/ From mrtopf at gmail.com Sun Dec 19 18:38:26 2010 From: mrtopf at gmail.com (Christian Scholz) Date: Sun, 19 Dec 2010 19:38:26 +0100 Subject: Problem with 304 responses from backend (results in 503 errors) In-Reply-To: References: Message-ID: Hi! Answering myself and thanks to nav on IRC this looks like my problem: http://www.varnish-cache.org/trac/changeset/5478 and http://www.varnish-cache.org/trac/ticket/803 I will try with a svn version of varnish tomorrow then and see if it fixes things. cheers, Christian 2010/12/19 Christian Scholz : > Hi! > > I am using?varnishd (varnish-2.1.4 SVN 5447M) and I am having a > problem if the backend returns a 304 error as this results in varnish > hanging for 60s and then returning a 503. > > I put the logfile at https://gist.github.com/33a1dd74a352745d7cc2, and > the relevant lines are this (done with varnishlog -o): > > 14 ObjProtocol ?c HTTP/1.1 > 14 ObjStatus ? ?c 304 > 14 ObjResponse ?c Not Modified > 14 ObjHeader ? ?c Server: Zope/(2.12.11, python 2.6.5, linux2) ZServer/1.1 > 14 ObjHeader ? ?c Date: Sun, 19 Dec 2010 17:02:18 GMT > 14 ObjHeader ? ?c X-Cache-Rule: product.simplelayout > 14 ObjHeader ? ?c ETag: |Authenticated;Internal;Member|****|1292695386.0 > 14 ObjHeader ? ?c X-Cache-Operation: plone.app.caching.weakCaching > 14 ObjHeader ? ?c X-Varnish-Action: FETCH (pass - not cacheable) > 14 FetchError ? c eof read_error: 11 > 14 VCL_call ? ? c error deliver > 14 VCL_call ? ? c deliver deliver > 14 TxProtocol ? c HTTP/1.1 > 14 TxStatus ? ? c 503 > 14 TxResponse ? c Service Unavailable > > So the backend returns a 304 without a content-length header. > > You can see my varnish vcl here: https://gist.github.com/c3211681ce7dc0681bdc > > The result of this is that every second reload of a page results in a 503. > > Does somebody know if I am doing something wrong or if that's a bug > either in varnish or Plone? > > Thanks in advance! > > Christian > > > > -- > Christian Scholz ? ? ? ? ? ? ? ? ? ? ? ? ?Homepage: http://comlounge.net > COM.lounge GmbH ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?http://mrtopf.de/blog > Hanbrucher Str. 33 ? ? ? ? ? ? ? ? ? ? ? ? ? ? http://twitter.com/mrtopf > 52064 Aachen ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? Skype: HerrTopf > Tel: +49 241 400 730 0 ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?cs at comlounge.net > Fax: +49 241 979 00 850 ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?IRC: MrTopf > > Podcasts: > Der OpenWeb-Podcast (http://openwebpodcast.de) > Data Without Borders (http://datawithoutborders.net) > Politisches: http://politfunk.de/ > -- Christian Scholz ? ? ? ? ? ? ? ? ? ? ? ? ?Homepage: http://comlounge.net COM.lounge GmbH ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?http://mrtopf.de/blog Hanbrucher Str. 33 ? ? ? ? ? ? ? ? ? ? ? ? ? ? http://twitter.com/mrtopf 52064 Aachen ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? Skype: HerrTopf Tel: +49 241 400 730 0 ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?cs at comlounge.net Fax: +49 241 979 00 850 ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?IRC: MrTopf Podcasts: Der OpenWeb-Podcast (http://openwebpodcast.de) Data Without Borders (http://datawithoutborders.net) Politisches: http://politfunk.de/ From A.Hongens at netmatch.nl Sun Dec 19 16:50:10 2010 From: A.Hongens at netmatch.nl (=?iso-8859-1?Q?Angelo_H=F6ngens?=) Date: Sun, 19 Dec 2010 16:50:10 +0000 Subject: varnishncsa Message-ID: <6A7ABA19243F1E4EADD8BB1563CDDCCB15034E@TIL-EXCH-05.netmatch.local> Hey, I noticed on some of my varnish machines that varnish was continuously restarting. In the logs I saw these errors: Dec 19 16:14:15 nmt-nlb-05 varnishd[4014]: Child (32358) died signal=6 Dec 19 16:14:15 nmt-nlb-05 varnishd[4014]: Child (32358) Panic message: Missing errorhandling code in sma_alloc(), storage_malloc.c line 81: Condition((sma->s.ptr) != 0) not true.errno = 12 (Cannot allocate memory) Using top, I saw the machine was using a lot of swap. I don't really know how to interpret the memory counters, but it looks like varnishnca was the culprit, using 10GB of virtual memory: --------------------------------------------------- Tasks: 149 total, 1 running, 148 sleeping, 0 stopped, 0 zombie Cpu(s): 3.5%us, 1.3%sy, 0.0%ni, 93.8%id, 0.2%wa, 0.1%hi, 1.0%si, 0.0%st Mem: 8173416k total, 8081768k used, 91648k free, 32860k buffers Swap: 4192888k total, 3900592k used, 292296k free, 1523580k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 3304 haproxy 15 0 62724 43m 440 S 21.8 0.5 461:28.79 haproxy 27159 varnish 18 0 1647m 232m 64m S 9.9 2.9 0:03.91 varnishd 3770 root 15 0 9743m 5.8g 80m S 2.0 74.8 35:42.34 varnishncsa 1 root 15 0 10352 588 552 S 0.0 0.0 0:01.17 init --------------------------------------------------- Looks like a memory leak to me? After I restarted varnishnca and had it running for 15 minutes (it's doing ~750 requests/sec), it still used 122MB, and the machine was happy (not using swap anymore). As a workaround I created a cron job to regularly restart the varnishncsa service, but perhaps this might need some looking into? About my environment: Running Varnish 2.1.4 on CentOS 5.5 x64. I downloaded the SRPM on a dedicated build box, applied Tollef's varnishncsa patch he wrote for our company, built the RPM's, and deployed these to our balancer machines. -- With kind regards, Angelo H?ngens Systems Administrator ------------------------------------------ NetMatch tourism internet software solutions Ringbaan Oost 2b 5013 CA Tilburg T: +31 (0)13 5811088 F: +31 (0)13 5821239 mailto:A.Hongens at netmatch.nl http://www.netmatch.nl ------------------------------------------ From fhelmschrott at gmail.com Mon Dec 20 07:04:33 2010 From: fhelmschrott at gmail.com (Frank Helmschrott) Date: Mon, 20 Dec 2010 08:04:33 +0100 Subject: Keepalive Sessions Message-ID: Good morning, i wonder how the thing is with keep alive sessions from apache when using it behind a varnish cache server. Doesn't keep alive make sense here as all the connection come from the varnish cache (one source) instead of the users (many sources). I don't know how these sessions work in detail but thats what came to my tired mind on early monday morning :) Probably anyone can give an idea here. -- Frank From kristian at varnish-software.com Mon Dec 20 08:19:01 2010 From: kristian at varnish-software.com (Kristian Lyngstol) Date: Mon, 20 Dec 2010 09:19:01 +0100 Subject: Keepalive Sessions In-Reply-To: References: Message-ID: <20101220081900.GA2445@freud> On Mon, Dec 20, 2010 at 08:04:33AM +0100, Frank Helmschrott wrote: > i wonder how the thing is with keep alive sessions from apache when > using it behind a varnish cache server. Doesn't keep alive make sense > here as all the connection come from the varnish cache (one source) > instead of the users (many sources). I don't know how these sessions > work in detail but thats what came to my tired mind on early monday > morning :) Varnish uses (and likes) keep-alive to the back-end. You want your backend to support keep-alive, since there's little point in re-establishing connections all the time. There's a "backend connections reused" counter that'll tell indicate how efficient Varnish is at taking advantage of keep-alive (and recycled, which is much the same thing). There's one exception, though: When you are using pipe (not pass), you want to set "connection: close" in vcl_pipe to disable keep-alive. This is because Varnish only sees the first request in a piped session - if the first request is piped, the next requests will be too. This is a problem if you need different backends for the requests; have cached the subsequent requests; or if you need the x-forwarded-for header or other varnish-manipulated headers on the subsequent requests. But other than pipe, you really want to use keep-alive. - Kristian -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: Digital signature URL: From kristian at varnish-software.com Mon Dec 20 08:25:05 2010 From: kristian at varnish-software.com (Kristian Lyngstol) Date: Mon, 20 Dec 2010 09:25:05 +0100 Subject: hanging connections In-Reply-To: References: Message-ID: <20101220082503.GB2445@freud> On Fri, Dec 17, 2010 at 03:40:35PM +0100, Stig Bakken wrote: > I have a script processing varnishlog output to give me the serving > latency based on the ReqEnd tag, and frequently see objects served from > cache, size around 25kB, that take 5+ seconds to serve. If there are two requests for an object, request A and request B, the following will happen: Time | Action 0s | A asks for /foo | Cache-miss - fetch from backend 0.5s | B asks for /foo | Cache hit on busy object - wait for it 5s | Web server is finished delivering /foo 5.1s | A gets /foo - cache miss 5.1s | B gets /foo - cache hit It's a bit difficult to spot this as it happens, it's been a matter of discussion recently (Arthur being the one who first became aware of the issue). It sounds like this might be what you are seeing. - Kristian -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: Digital signature URL: From alex at arcalpin.com Mon Dec 20 09:58:09 2010 From: alex at arcalpin.com (Modesto Alexandre) Date: Mon, 20 Dec 2010 10:58:09 +0100 Subject: 503 errors on POST Message-ID: <201012201058.10015.alex@arcalpin.com> Hello, I have a problem with Varnish, from time to time I get errors 503 on the "POST" Errors are infrequent, but annoying for users I can not identify the source of the problem My setup consists of: * A server Varnish * 3 going backends on the same server (3 routes) Here are the errors I have (I have masked the private information): http://demo.ovh.net/view/6ac24fbc5400039bc86fd3444556ef76/0.colored Do you have any leads? How to find the problem in production? Thank you, Alex. From traian.bratucu at eea.europa.eu Mon Dec 20 10:59:34 2010 From: traian.bratucu at eea.europa.eu (Traian Bratucu) Date: Mon, 20 Dec 2010 11:59:34 +0100 Subject: Keepalive Sessions In-Reply-To: <20101220081900.GA2445@freud> References: , <20101220081900.GA2445@freud> Message-ID: I must point out another "exception" you may want to be aware of. When using keep-alive between varnish and the backend, if the client sends a request which times out (backend does not respond in the specified timeout time) then varnish will retry the request (hardcoded logic) one more time. Even if you don't specify any restarts in the VCL. This is no problem normally, unless your request was a POST which the backend takes a long time to process. With several backends such a POST request which times out on the first try will be retried, and you will have yourself a double-post. In my case the backends were timing out on processing some long email lists and I was finding that all emails were sent twice. And that sucks :) On Mon, Dec 20, 2010 at 08:04:33AM +0100, Frank Helmschrott wrote: > i wonder how the thing is with keep alive sessions from apache when > using it behind a varnish cache server. Doesn't keep alive make sense > here as all the connection come from the varnish cache (one source) > instead of the users (many sources). I don't know how these sessions > work in detail but thats what came to my tired mind on early monday > morning :) Varnish uses (and likes) keep-alive to the back-end. You want your backend to support keep-alive, since there's little point in re-establishing connections all the time. There's a "backend connections reused" counter that'll tell indicate how efficient Varnish is at taking advantage of keep-alive (and recycled, which is much the same thing). There's one exception, though: When you are using pipe (not pass), you want to set "connection: close" in vcl_pipe to disable keep-alive. This is because Varnish only sees the first request in a piped session - if the first request is piped, the next requests will be too. This is a problem if you need different backends for the requests; have cached the subsequent requests; or if you need the x-forwarded-for header or other varnish-manipulated headers on the subsequent requests. But other than pipe, you really want to use keep-alive. - Kristian From dennis.hendriksen at kalooga.com Mon Dec 20 11:05:33 2010 From: dennis.hendriksen at kalooga.com (Dennis Hendriksen) Date: Mon, 20 Dec 2010 12:05:33 +0100 Subject: Varnish 2.0.6 thread pile-up Message-ID: <1292843133.20355.44.camel@kalooga-dennis> Hi, We are running Varnish 2.0.6 on two machines (dual quad core CPU, kernel 2.6.18-194.3.1.el5) through a load balancer. Both Varnishes are connected to load-balanced (fast responding) back-ends, each handling on average 500 requests/s (97+% hits). Each once and a while (in the range from once an hour to once a week) we see an instant worker thread pile-up with overflowed and sometimes dropped work requests. During these short periods (about 15min) everything returns to normal. The pile-ups occur at non regular intervals at both quiet and busy times. Pile-ups occur on both machines at the same time. During pile-ups Varnish responses are extremely slow and transfer rates drop disrupting our services. Furthermore we sometimes see the following in /var/log/messages: Dec 19 22:40:04 cache1 kernel: possible SYN flooding on port 80. Sending cookies. Dec 19 22:41:05 cache1 kernel: possible SYN flooding on port 80. Sending cookies. We've been looking closely at internal (back-ends) and external (network) factors but so far haven't found a cause of the problems described above. Anyone has a clue what could be the cause of this problem? Or recognizes this issue? Thanks for your time, Dennis --- Varnish: Varnish config (storage: malloc 10GB): accept_fd_holdoff 50 [ms] acceptor default (epoll, poll) auto_restart on [bool] backend_http11 on [bool] between_bytes_timeout 60.000000 [s] cache_vbe_conns off [bool] cc_command "exec cc -fpic -shared -Wl,-x -o %o %s" cli_buffer 8192 [bytes] cli_timeout 5 [seconds] client_http11 off [bool] clock_skew 10 [s] connect_timeout 0.400000 [s] default_grace 10 default_ttl 120 [seconds] diag_bitmap 0x0 [bitmap] err_ttl 0 [seconds] esi_syntax 0 [bitmap] fetch_chunksize 128 [kilobytes] first_byte_timeout 60.000000 [s] group varnish (103) listen_address :80 listen_depth 1024 [connections] log_hashstring off [bool] log_local_address off [bool] lru_interval 2 [seconds] max_esi_includes 5 [includes] max_restarts 4 [restarts] obj_workspace 8192 [bytes] overflow_max 100 [%] ping_interval 3 [seconds] pipe_timeout 60 [seconds] prefer_ipv6 off [bool] purge_dups on [bool] purge_hash on [bool] rush_exponent 3 [requests per request] send_timeout 600 [seconds] sess_timeout 5 [seconds] sess_workspace 16384 [bytes] session_linger 50 [ms] session_max 100000 [sessions] shm_reclen 255 [bytes] shm_workspace 8192 [bytes] srcaddr_hash 1049 [buckets] srcaddr_ttl 0 [seconds] thread_pool_add_delay 20 [milliseconds] thread_pool_add_threshold 2 [requests] thread_pool_fail_delay 200 [milliseconds] thread_pool_max 1000 [threads] thread_pool_min 15 [threads] thread_pool_purge_delay 1000 [milliseconds] thread_pool_stack unlimited [bytes] thread_pool_timeout 300 [seconds] thread_pools 8 [pools] user varnish (101) vcl_trace off [bool] From fla_torres at yahoo.com.br Mon Dec 20 14:01:09 2010 From: fla_torres at yahoo.com.br (Flavio Torres) Date: Mon, 20 Dec 2010 12:01:09 -0200 Subject: Varnish 2.0.6 thread pile-up In-Reply-To: <1292843133.20355.44.camel@kalooga-dennis> References: <1292843133.20355.44.camel@kalooga-dennis> Message-ID: <4D0F61A5.8030105@yahoo.com.br> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 12/20/2010 09:05 AM, Dennis Hendriksen wrote: > Furthermore we sometimes see the following in /var/log/messages: > Dec 19 22:40:04 cache1 kernel: possible SYN flooding on port 80. Sending > cookies. > Dec 19 22:41:05 cache1 kernel: possible SYN flooding on port 80. Sending > cookies. Try to change at your sysctl.conf the following value: net.ipv4.tcp_syncookies = 0 See: http://www.varnish-cache.org/trac/wiki/Performance hope this helps -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk0PYY0ACgkQNRQApncg2948vgCfaxCKQoCwPGG5GSNwXv45DcIg CIAAniRud/SV/vzBTaZb3xlcbIOKAPPz =fygR -----END PGP SIGNATURE----- From cooltechemail at gmail.com Mon Dec 20 14:04:53 2010 From: cooltechemail at gmail.com (Vincent) Date: Mon, 20 Dec 2010 06:04:53 -0800 Subject: child process of varnishd restarts automatically Message-ID: Hi All We are running varnish on a 32-bit centos 5 box and we have noticed that sometimes the child process of varnishd will restart automatically. /var/log/messages shows the following error message: varnishd[12707]: Child (12708) died signal=6 varnishd[12707]: Child (12708) Panic message: Missing errorhandling code in sma_alloc(), storage_malloc.c line 81: Condition((sma->s.ptr) != 0) not true.errno = 12 (Cannot allocate memory) thread = (cache-worker) ident = Linux,2.6.18-194.8.1.el5PAE,i686,-smalloc,-hcritbit,epoll Backtrace: 0x806cb6c: varnishd [0x806cb6c] 0x8088047: varnishd [0x8088047] 0x8085b3a: varnishd(STV_alloc+0xda) [0x8085b3a] 0x8063a91: varnishd(FetchBody+0x7c1) [0x8063a91] 0x8059f87: varnishd [0x8059f87] 0x805b93a: varnishd(CNT_Session+0x44a) [0x805b93a] 0x806f5ef: varnishd [0x806f5ef] 0x806e671: varnishd [0x806e671] 0x806eb6f: varnishd [0x806eb6f] 0xc65832: /lib/libpthread.so.0 [0xc65832] sp = 0xb7ec5004 { fd = 15, id = 15, xid = 608646521, client = 127.0.0.1:57125, step = STP_FETCH, handling = deliver, err_code = 200, err_reason = (null), restarts = 0, esis = 0 ws = 0xb7ec504c { id = "sess", {s,f,r,e} = {0xb7ec57dc,+956,(nil),+16384}, }, http[req] = { ws = 0x varnishd[12707]: child (32242) Started varnishd[12707]: Child (32242) said varnishd[12707]: Child (32242) said Child starts Varnishd was started with option "malloc,1.5G" and it runs without any problem for hours before the child process restarts. When the child process restarts, the system still have about 1.5G free memory so this is not a out of memory issue. Does anybody know how to solve this? Thanks, Vincent -------------- next part -------------- An HTML attachment was scrubbed... URL: From stig at zedge.net Mon Dec 20 14:09:42 2010 From: stig at zedge.net (Stig Bakken) Date: Mon, 20 Dec 2010 15:09:42 +0100 Subject: hanging connections In-Reply-To: <20101220082503.GB2445@freud> References: <20101220082503.GB2445@freud> Message-ID: On Mon, Dec 20, 2010 at 9:25 AM, Kristian Lyngstol < kristian at varnish-software.com> wrote: > On Fri, Dec 17, 2010 at 03:40:35PM +0100, Stig Bakken wrote: > > I have a script processing varnishlog output to give me the serving > > latency based on the ReqEnd tag, and frequently see objects served from > > cache, size around 25kB, that take 5+ seconds to serve. > > If there are two requests for an object, request A and request B, the > following will happen: > > Time | Action > 0s | A asks for /foo > | Cache-miss - fetch from backend > 0.5s | B asks for /foo > | Cache hit on busy object - wait for it > 5s | Web server is finished delivering /foo > 5.1s | A gets /foo - cache miss > 5.1s | B gets /foo - cache hit > > It's a bit difficult to spot this as it happens, it's been a matter of > discussion recently (Arthur being the one who first became aware of the > issue). > > It sounds like this might be what you are seeing. > > - Kristian Yeah, that could be it. A race condition that is to be expected. I'll test whether this explains the issue by comparing 95/99-percentile latency with cache hit rate over time. - Stig -- Stig Bakken CTO, Zedge.net - free your phone! -------------- next part -------------- An HTML attachment was scrubbed... URL: From traian.bratucu at eea.europa.eu Mon Dec 20 14:19:08 2010 From: traian.bratucu at eea.europa.eu (Traian Bratucu) Date: Mon, 20 Dec 2010 15:19:08 +0100 Subject: child process of varnishd restarts automatically In-Reply-To: References: Message-ID: It is possible Varnish is trying to fetch a very large object (hundreds of Mb, maybe >1Gb) and it tries to store it in the cache, failing because of lack of memory. At least that is what it looks like to me. Traian ________________________________________ Hi All We are running varnish on a 32-bit centos 5 box and we have noticed that sometimes the child process of varnishd will restart automatically. /var/log/messages shows the following error message: varnishd[12707]: Child (12708) died signal=6 varnishd[12707]: Child (12708) Panic message: Missing errorhandling code in sma_alloc(), storage_malloc.c line 81: Condition((sma->s.ptr) != 0) not true.errno = 12 (Cannot allocate memory) thread = (cache-worker) ident = Linux,2.6.18-194.8.1.el5PAE,i686,-smalloc,-hcritbit,epoll Backtrace: 0x806cb6c: varnishd [0x806cb6c] 0x8088047: varnishd [0x8088047] 0x8085b3a: varnishd(STV_alloc+0xda) [0x8085b3a] 0x8063a91: varnishd(FetchBody+0x7c1) [0x8063a91] 0x8059f87: varnishd [0x8059f87] 0x805b93a: varnishd(CNT_Session+0x44a) [0x805b93a] 0x806f5ef: varnishd [0x806f5ef] 0x806e671: varnishd [0x806e671] 0x806eb6f: varnishd [0x806eb6f] 0xc65832: /lib/libpthread.so.0 [0xc65832] sp = 0xb7ec5004 { fd = 15, id = 15, xid = 608646521, client = 127.0.0.1:57125, step = STP_FETCH, handling = deliver, err_code = 200, err_reason = (null), restarts = 0, esis = 0 ws = 0xb7ec504c { id = "sess", {s,f,r,e} = {0xb7ec57dc,+956,(nil),+16384}, }, http[req] = { ws = 0x varnishd[12707]: child (32242) Started varnishd[12707]: Child (32242) said varnishd[12707]: Child (32242) said Child starts Varnishd was started with option "malloc,1.5G" and it runs without any problem for hours before the child process restarts. When the child process restarts, the system still have about 1.5G free memory so this is not a out of memory issue. Does anybody know how to solve this? Thanks, Vincent From kristian at varnish-software.com Mon Dec 20 14:22:18 2010 From: kristian at varnish-software.com (Kristian Lyngstol) Date: Mon, 20 Dec 2010 15:22:18 +0100 Subject: child process of varnishd restarts automatically In-Reply-To: References: Message-ID: <20101220142217.GC2445@freud> Hi, On Mon, Dec 20, 2010 at 06:04:53AM -0800, Vincent wrote: > We are running varnish on a 32-bit centos 5 box and we have noticed that > sometimes the child process of varnishd will restart automatically. You should be using 64bit. > varnishd[12707]: Child (12708) Panic message: Missing errorhandling code in > sma_alloc(), storage_malloc.c line 81: Condition((sma->s.ptr) != 0) not > true.errno = 12 (Cannot allocate memory) thread = (cache-worker) ident = (...) > Varnishd was started with option "malloc,1.5G" and it runs without any > problem for hours before the child process restarts. When the child process > restarts, the system still have about 1.5G free memory so this is not a out > of memory issue. You are out of memory - possibly 32-bit-related issue since you have a artificially limited address space. That your system has 1.5GB "free" memory doesn't mean you can use it. That's why you want to run 64-bit systems: then you can actually use the memory too. An other possible culprit is disabling overcommit - but if you actually had free memory, it's unlikely that that's your problem. The proper solution to this problem is to reinstall your system using a 64-bit operating system. If you prefer wasting countless hours, you can try to estimate the overhead of the object count (roughly 1kB) and how that affects the total memory usage of varnishd, factor in stack size of threads, constant memory overhead and the available address space left to you - it's seriously not fun (or all that easy). - Kristian -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: Digital signature URL: From stig at zedge.net Mon Dec 20 15:14:57 2010 From: stig at zedge.net (Stig Bakken) Date: Mon, 20 Dec 2010 16:14:57 +0100 Subject: hanging connections In-Reply-To: <20101220082503.GB2445@freud> References: <20101220082503.GB2445@freud> Message-ID: On Mon, Dec 20, 2010 at 9:25 AM, Kristian Lyngstol < kristian at varnish-software.com> wrote: > On Fri, Dec 17, 2010 at 03:40:35PM +0100, Stig Bakken wrote: > > I have a script processing varnishlog output to give me the serving > > latency based on the ReqEnd tag, and frequently see objects served from > > cache, size around 25kB, that take 5+ seconds to serve. > > If there are two requests for an object, request A and request B, the > following will happen: > > Time | Action > 0s | A asks for /foo > | Cache-miss - fetch from backend > 0.5s | B asks for /foo > | Cache hit on busy object - wait for it > 5s | Web server is finished delivering /foo > 5.1s | A gets /foo - cache miss > 5.1s | B gets /foo - cache hit > > It's a bit difficult to spot this as it happens, it's been a matter of > discussion recently (Arthur being the one who first became aware of the > issue). > > It sounds like this might be what you are seeing. > Here are some more details. The attached file is an edited varnishlog dump, with elapsed time appended after each ReqEnd. Notice that one of the requests takes 137 seconds, during which time all of the others in the dump complete much faster. The Age: header is high, which rules out the cache injection race condition, right? This is with 2.1.4 installed from the official rpm on RHEL, btw. - Stig -- Stig Bakken CTO, Zedge.net - free your phone! -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- 190 SessionOpen c xx.xx.xx.xx 4400 :80 190 ReqStart c xx.xx.xx.xx 4400 2086897843 190 RxRequest c GET 190 RxURL c /content/7/0/8/0/18-16662853-708002298-t.jpg 190 RxProtocol c HTTP/1.1 190 RxHeader c Accept: */* 190 RxHeader c Referer: http://www.zedge.net/ 190 RxHeader c Accept-Language: en 190 RxHeader c User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; GTB6.6; InfoPath.1) 190 RxHeader c Accept-Encoding: gzip, deflate 190 RxHeader c Host: fsa.zedge.net 190 RxHeader c Connection: Keep-Alive 190 RxHeader c Cookie: xxxx 190 VCL_call c recv lookup 190 VCL_call c hash hash 190 Hit c 2067494928 190 VCL_call c hit deliver 190 VCL_call c deliver deliver 190 TxProtocol c HTTP/1.1 190 TxStatus c 200 190 TxResponse c OK 190 TxHeader c Server: Apache/2.2.14 (Unix) 190 TxHeader c Last-Modified: Sun, 12 Dec 2010 19:51:47 GMT 190 TxHeader c Cache-Control: max-age=2592000 190 TxHeader c Expires: Wed, 12 Jan 2011 10:01:04 GMT 190 TxHeader c Content-Type: image/jpeg 190 TxHeader c Content-Length: 30522 190 TxHeader c Date: Tue, 14 Dec 2010 08:07:52 GMT 190 TxHeader c X-Varnish: 2086897843 2067494928 190 TxHeader c Age: 79608 190 TxHeader c Via: 1.1 varnish 190 TxHeader c Connection: keep-alive 190 Length c 30522 190 ReqEnd c 2086897843 1292314072.912205935 1292314072.912265062 0.000041962 0.000025034 0.000034094 0.000059 123 ReqStart c xx.xx.xx.xx 46854 2086897839 123 RxRequest c GET 123 RxURL c /content/7/0/8/0/18-16662853-708002298-t.jpg 123 RxProtocol c HTTP/1.0 123 RxHeader c Host: fsa.zedge.net 123 RxHeader c User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 123 RxHeader c Accept: image/png,image/*;q=0.8,*/*;q=0.5 123 RxHeader c Accept-Language: en-us,en;q=0.5 123 RxHeader c Accept-Encoding: gzip,deflate 123 RxHeader c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 123 RxHeader c Keep-Alive: 115 123 RxHeader c Referer: http://www.zedge.net/ 123 RxHeader c Cookie: xxxx 123 RxHeader c Via: 1.1 mail.cmeri.res.in:3128 (squid/2.6.STABLE6) 123 RxHeader c X-Forwarded-For: xx 123 RxHeader c Cache-Control: max-age=259200 123 RxHeader c Connection: keep-alive 123 VCL_call c recv lookup 123 VCL_call c hash hash 123 Hit c 2067494928 123 VCL_call c hit deliver 123 VCL_call c deliver deliver 123 TxProtocol c HTTP/1.1 123 TxStatus c 200 123 TxResponse c OK 123 TxHeader c Server: Apache/2.2.14 (Unix) 123 TxHeader c Last-Modified: Sun, 12 Dec 2010 19:51:47 GMT 123 TxHeader c Cache-Control: max-age=2592000 123 TxHeader c Expires: Wed, 12 Jan 2011 10:01:04 GMT 123 TxHeader c Content-Type: image/jpeg 123 TxHeader c Content-Length: 30522 123 TxHeader c Date: Tue, 14 Dec 2010 08:07:52 GMT 123 TxHeader c X-Varnish: 2086897839 2067494928 123 TxHeader c Age: 79608 123 TxHeader c Via: 1.1 varnish 123 TxHeader c Connection: keep-alive 123 Length c 30522 123 ReqEnd c 2086897839 1292314072.893872976 1292314073.218530893 1.294373989 0.000030041 0.324627876 0.324658 355 ReqStart c xx.xx.xx.xx 1800 2086900244 355 RxRequest c GET 355 RxURL c /content/7/0/8/0/18-16662853-708002298-t.jpg 355 RxProtocol c HTTP/1.1 355 RxHeader c Accept: */* 355 RxHeader c Referer: http://www.zedge.net/themes/ 355 RxHeader c Accept-Language: en-us 355 RxHeader c User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0) 355 RxHeader c Accept-Encoding: gzip, deflate 355 RxHeader c Host: fsa.zedge.net 355 RxHeader c Connection: Keep-Alive 355 RxHeader c Cookie: xxxx 355 VCL_call c recv lookup 355 VCL_call c hash hash 355 Hit c 2067494928 355 VCL_call c hit deliver 355 VCL_call c deliver deliver 355 TxProtocol c HTTP/1.1 355 TxStatus c 200 355 TxResponse c OK 355 TxHeader c Server: Apache/2.2.14 (Unix) 355 TxHeader c Last-Modified: Sun, 12 Dec 2010 19:51:47 GMT 355 TxHeader c Cache-Control: max-age=2592000 355 TxHeader c Expires: Wed, 12 Jan 2011 10:01:04 GMT 355 TxHeader c Content-Type: image/jpeg 355 TxHeader c Content-Length: 30522 355 TxHeader c Date: Tue, 14 Dec 2010 08:08:10 GMT 355 TxHeader c X-Varnish: 2086900244 2067494928 355 TxHeader c Age: 79626 355 TxHeader c Via: 1.1 varnish 355 TxHeader c Connection: keep-alive 355 Length c 30522 355 ReqEnd c 2086900244 1292314090.363420963 1292314090.363485098 1.637055874 0.000021935 0.000042200 0.000064 583 SessionOpen c xx.xx.xx.xx 4942 :80 583 ReqStart c xx.xx.xx.xx 4942 2086900274 583 RxRequest c GET 583 RxURL c /content/7/0/8/0/18-16662853-708002298-t.jpg 583 RxProtocol c HTTP/1.1 583 RxHeader c Accept: */* 583 RxHeader c Referer: http://www.zedge.net/ 583 RxHeader c Accept-Language: en-us 583 RxHeader c User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727) 583 RxHeader c Accept-Encoding: gzip, deflate 583 RxHeader c Host: fsa.zedge.net 583 RxHeader c Connection: Keep-Alive 583 RxHeader c Cookie: xxxx 583 VCL_call c recv lookup 583 VCL_call c hash hash 583 Hit c 2067494928 583 VCL_call c hit deliver 583 VCL_call c deliver deliver 583 TxProtocol c HTTP/1.1 583 TxStatus c 200 583 TxResponse c OK 583 TxHeader c Server: Apache/2.2.14 (Unix) 583 TxHeader c Last-Modified: Sun, 12 Dec 2010 19:51:47 GMT 583 TxHeader c Cache-Control: max-age=2592000 583 TxHeader c Expires: Wed, 12 Jan 2011 10:01:04 GMT 583 TxHeader c Content-Type: image/jpeg 583 TxHeader c Content-Length: 30522 583 TxHeader c Date: Tue, 14 Dec 2010 08:08:10 GMT 583 TxHeader c X-Varnish: 2086900274 2067494928 583 TxHeader c Age: 79626 583 TxHeader c Via: 1.1 varnish 583 TxHeader c Connection: keep-alive 583 Length c 30522 583 ReqEnd c 2086900274 1292314090.479566097 1292314090.479645967 0.000112057 0.000029802 0.000050068 0.000080 277 SessionOpen c xx.xx.xx.xx 12475 :80 277 ReqStart c xx.xx.xx.xx 12475 2086900986 277 RxRequest c GET 277 RxURL c /content/7/0/8/0/18-16662853-708002298-t.jpg 277 RxProtocol c HTTP/1.1 277 RxHeader c Host: fsa.zedge.net 277 RxHeader c User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 277 RxHeader c Accept: image/png,image/*;q=0.8,*/*;q=0.5 277 RxHeader c Accept-Language: en-us,en;q=0.5 277 RxHeader c Accept-Encoding: gzip,deflate 277 RxHeader c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 277 RxHeader c Keep-Alive: 115 277 RxHeader c Connection: keep-alive 277 RxHeader c Referer: http://www.zedge.net/ 277 RxHeader c Cookie: xxxx 277 VCL_call c recv lookup 277 VCL_call c hash hash 277 Hit c 2067494928 277 VCL_call c hit deliver 277 VCL_call c deliver deliver 277 TxProtocol c HTTP/1.1 277 TxStatus c 200 277 TxResponse c OK 277 TxHeader c Server: Apache/2.2.14 (Unix) 277 TxHeader c Last-Modified: Sun, 12 Dec 2010 19:51:47 GMT 277 TxHeader c Cache-Control: max-age=2592000 277 TxHeader c Expires: Wed, 12 Jan 2011 10:01:04 GMT 277 TxHeader c Content-Type: image/jpeg 277 TxHeader c Content-Length: 30522 277 TxHeader c Date: Tue, 14 Dec 2010 08:08:15 GMT 277 TxHeader c X-Varnish: 2086900986 2067494928 277 TxHeader c Age: 79630 277 TxHeader c Via: 1.1 varnish 277 TxHeader c Connection: keep-alive 277 Length c 30522 277 ReqEnd c 2086900986 1292314095.218641043 1292314095.218713999 0.000039101 0.000028849 0.000044107 0.000073 450 SessionOpen c xx.xx.xx.xx 1605 :80 450 ReqStart c xx.xx.xx.xx 1605 2086901100 450 RxRequest c GET 450 RxURL c /content/7/0/8/0/18-16662853-708002298-t.jpg 450 RxProtocol c HTTP/1.1 450 RxHeader c Host: fsa.zedge.net 450 RxHeader c Connection: keep-alive 450 RxHeader c Referer: http://www.zedge.net/ 450 RxHeader c Accept: */* 450 RxHeader c User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.10 (KHTML, like Gecko) Chrome/8.0.552.215 Safari/534.10 450 RxHeader c Accept-Encoding: gzip,deflate,sdch 450 RxHeader c Accept-Language: es-ES,es;q=0.8 450 RxHeader c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3 450 RxHeader c Cookie: xxxx 450 VCL_call c recv lookup 450 VCL_call c hash hash 450 Hit c 2067494928 450 VCL_call c hit deliver 450 VCL_call c deliver deliver 450 TxProtocol c HTTP/1.1 450 TxStatus c 200 450 TxResponse c OK 450 TxHeader c Server: Apache/2.2.14 (Unix) 450 TxHeader c Last-Modified: Sun, 12 Dec 2010 19:51:47 GMT 450 TxHeader c Cache-Control: max-age=2592000 450 TxHeader c Expires: Wed, 12 Jan 2011 10:01:04 GMT 450 TxHeader c Content-Type: image/jpeg 450 TxHeader c Content-Length: 30522 450 TxHeader c Date: Tue, 14 Dec 2010 08:08:15 GMT 450 TxHeader c X-Varnish: 2086901100 2067494928 450 TxHeader c Age: 79631 450 TxHeader c Via: 1.1 varnish 450 TxHeader c Connection: keep-alive 450 Length c 30522 450 ReqEnd c 2086901100 1292314095.979444027 1292314095.979499102 0.000091076 0.000020981 0.000034094 0.000055 587 ReqStart c xx.xx.xx.xx 47084 2086901110 587 RxRequest c GET 587 RxURL c /content/7/0/8/0/18-16662853-708002298-t.jpg 587 RxProtocol c HTTP/1.1 587 RxHeader c User-Agent: Opera/9.80 (J2ME/MIDP; Opera Mini/5.1.22296/22.401; U; en) Presto/2.5.25 Version/10.54 587 RxHeader c Host: fsa.zedge.net 587 RxHeader c Accept: text/html, application/xml;q=0.9, application/xhtml+xml, image/png, image/jpeg, image/gif, image/x-xbitmap, */*;q=0.1 587 RxHeader c Accept-Language: en-GB,en;q=0.9 587 RxHeader c Accept-Charset: iso-8859-1, utf-8, utf-16, *;q=0.1 587 RxHeader c Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0 587 RxHeader c Referer: http://www.zedge.net/ 587 RxHeader c Cookie: xxxx 587 RxHeader c Cookie2: $Version=1 587 RxHeader c Connection: Keep-Alive, TE 587 RxHeader c TE: deflate, gzip, chunked, identity, trailers 587 RxHeader c X-OperaMini-Features: advanced, file_system, camera, touch, folding, routing 587 RxHeader c X-OperaMini-Phone: Generic # J2ME Midlet 587 RxHeader c x-forwarded-for: xx 587 RxHeader c X-OperaMini-Phone-UA: UNTRUSTED/1.0 587 VCL_call c recv lookup 587 VCL_call c hash hash 587 Hit c 2067494928 587 VCL_call c hit deliver 587 VCL_call c deliver deliver 587 TxProtocol c HTTP/1.1 587 TxStatus c 200 587 TxResponse c OK 587 TxHeader c Server: Apache/2.2.14 (Unix) 587 TxHeader c Last-Modified: Sun, 12 Dec 2010 19:51:47 GMT 587 TxHeader c Cache-Control: max-age=2592000 587 TxHeader c Expires: Wed, 12 Jan 2011 10:01:04 GMT 587 TxHeader c Content-Type: image/jpeg 587 TxHeader c Content-Length: 30522 587 TxHeader c Date: Tue, 14 Dec 2010 08:08:16 GMT 587 TxHeader c X-Varnish: 2086901110 2067494928 587 TxHeader c Age: 79631 587 TxHeader c Via: 1.1 varnish 587 TxHeader c Connection: keep-alive 587 Length c 30522 587 ReqEnd c 2086901110 1292314096.071873903 1292314096.211152077 0.139271975 0.000024080 0.139254093 0.139278 585 ReqStart c xx.xx.xx.xx 47085 2086901111 585 RxRequest c GET 585 RxURL c /content/7/0/8/0/18-16662853-708002298-t.jpg 585 RxProtocol c HTTP/1.1 585 RxHeader c User-Agent: Opera/9.80 (J2ME/MIDP; Opera Mini/5.1.22296/22.401; U; en) Presto/2.5.25 Version/10.54 585 RxHeader c Host: fsa.zedge.net 585 RxHeader c Accept: text/html, application/xml;q=0.9, application/xhtml+xml, image/png, image/jpeg, image/gif, image/x-xbitmap, */*;q=0.1 585 RxHeader c Accept-Language: en-GB,en;q=0.9 585 RxHeader c Accept-Charset: iso-8859-1, utf-8, utf-16, *;q=0.1 585 RxHeader c Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0 585 RxHeader c Referer: http://www.zedge.net/ 585 RxHeader c Cookie: xxxx 585 RxHeader c Cookie2: $Version=1 585 RxHeader c Connection: Keep-Alive, TE 585 RxHeader c TE: deflate, gzip, chunked, identity, trailers 585 RxHeader c X-OperaMini-Features: advanced, file_system, camera, touch, folding, routing 585 RxHeader c X-OperaMini-Phone: Generic # J2ME Midlet 585 RxHeader c x-forwarded-for: xx 585 RxHeader c X-OperaMini-Phone-UA: UNTRUSTED/1.0 585 VCL_call c recv lookup 585 VCL_call c hash hash 585 Hit c 2067494928 585 VCL_call c hit deliver 585 VCL_call c deliver deliver 585 TxProtocol c HTTP/1.1 585 TxStatus c 200 585 TxResponse c OK 585 TxHeader c Server: Apache/2.2.14 (Unix) 585 TxHeader c Last-Modified: Sun, 12 Dec 2010 19:51:47 GMT 585 TxHeader c Cache-Control: max-age=2592000 585 TxHeader c Expires: Wed, 12 Jan 2011 10:01:04 GMT 585 TxHeader c Content-Type: image/jpeg 585 TxHeader c Content-Length: 30522 585 TxHeader c Date: Tue, 14 Dec 2010 08:08:16 GMT 585 TxHeader c X-Varnish: 2086901111 2067494928 585 TxHeader c Age: 79631 585 TxHeader c Via: 1.1 varnish 585 TxHeader c Connection: keep-alive 585 Length c 30522 585 ReqEnd c 2086901111 1292314096.071985960 1292314096.211450100 0.279229879 0.000020027 0.139444113 0.139464 304 SessionOpen c xx.xx.xx.xx 2273 :80 304 ReqStart c xx.xx.xx.xx 2273 2086901659 304 RxRequest c GET 304 RxURL c /content/7/0/8/0/18-16662853-708002298-t.jpg 304 RxProtocol c HTTP/1.1 304 RxHeader c Accept: */* 304 RxHeader c Referer: http://www.zedge.net/ 304 RxHeader c Accept-Language: en-us 304 RxHeader c User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; GTB6.3; InfoPath.2) 304 RxHeader c Accept-Encoding: gzip, deflate 304 RxHeader c If-Modified-Since: Sun, 12 Dec 2010 19:51:47 GMT 304 RxHeader c Host: fsa.zedge.net 304 RxHeader c Connection: Keep-Alive 304 RxHeader c Cookie: xxxx 304 VCL_call c recv lookup 304 VCL_call c hash hash 304 Hit c 2067494928 304 VCL_call c hit deliver 304 VCL_call c deliver deliver 304 TxProtocol c HTTP/1.1 304 TxStatus c 304 304 TxResponse c Not Modified 304 TxHeader c Date: Tue, 14 Dec 2010 08:08:19 GMT 304 TxHeader c Via: 1.1 varnish 304 TxHeader c X-Varnish: 2086901659 304 TxHeader c Last-Modified: Sun, 12 Dec 2010 19:51:47 GMT 304 TxHeader c Cache-Control: max-age=2592000 304 TxHeader c Expires: Wed, 12 Jan 2011 10:01:04 GMT 304 TxHeader c Connection: keep-alive 304 Length c 0 304 ReqEnd c 2086901659 1292314099.982669115 1292314099.982716084 0.000020027 0.000021935 0.000025034 0.000047 781 SessionOpen c xx.xx.xx.xx 3590 :80 781 ReqStart c xx.xx.xx.xx 3590 2086901854 781 RxRequest c GET 781 RxURL c /content/7/0/8/0/18-16662853-708002298-t.jpg 781 RxProtocol c HTTP/1.1 781 RxHeader c Host: fsa.zedge.net 781 RxHeader c Connection: keep-alive 781 RxHeader c Referer: http://www.zedge.net/ 781 RxHeader c Accept: */* 781 RxHeader c User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.10 (KHTML, like Gecko) Chrome/8.0.552.224 Safari/534.10 781 RxHeader c Accept-Encoding: gzip,deflate,sdch 781 RxHeader c Accept-Language: en-GB,en-US;q=0.8,en;q=0.6 781 RxHeader c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3 781 RxHeader c Cookie: xxxx 781 VCL_call c recv lookup 781 VCL_call c hash hash 781 Hit c 2067494928 781 VCL_call c hit deliver 781 VCL_call c deliver deliver 781 TxProtocol c HTTP/1.1 781 TxStatus c 200 781 TxResponse c OK 781 TxHeader c Server: Apache/2.2.14 (Unix) 781 TxHeader c Last-Modified: Sun, 12 Dec 2010 19:51:47 GMT 781 TxHeader c Cache-Control: max-age=2592000 781 TxHeader c Expires: Wed, 12 Jan 2011 10:01:04 GMT 781 TxHeader c Content-Type: image/jpeg 781 TxHeader c Content-Length: 30522 781 TxHeader c Date: Tue, 14 Dec 2010 08:08:20 GMT 781 TxHeader c X-Varnish: 2086901854 2067494928 781 TxHeader c Age: 79636 781 TxHeader c Via: 1.1 varnish 781 TxHeader c Connection: keep-alive 781 Length c 30522 781 ReqEnd c 2086901854 1292314100.744549990 1292314100.744611025 0.000034094 0.000020027 0.000041008 0.000061 727 ReqStart c xx.xx.xx.xx 35523 2086901880 727 RxRequest c GET 727 RxURL c /content/7/0/8/0/18-16662853-708002298-t.jpg 727 RxProtocol c HTTP/1.1 727 RxHeader c User-Agent: Opera/9.80 (J2ME/MIDP; Opera Mini/5.1.21214/22.401; U; en) Presto/2.5.25 Version/10.54 727 RxHeader c Host: fsa.zedge.net 727 RxHeader c Accept: text/html, application/xml;q=0.9, application/xhtml+xml, image/png, image/jpeg, image/gif, image/x-xbitmap, */*;q=0.1 727 RxHeader c Accept-Language: en,en;q=0.9 727 RxHeader c Accept-Charset: iso-8859-1, utf-8, utf-16, *;q=0.1 727 RxHeader c Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0 727 RxHeader c Referer: http://www.zedge.net/ 727 RxHeader c Cookie: xxxx 727 RxHeader c Cookie2: $Version=1 727 RxHeader c Connection: Keep-Alive, TE 727 RxHeader c TE: deflate, gzip, chunked, identity, trailers 727 RxHeader c x-forwarded-for: xx 727 RxHeader c X-OperaMini-Phone: Generic # J2ME Midlet 727 RxHeader c X-OperaMini-Phone-UA: UNTRUSTED/1.0 727 RxHeader c X-OperaMini-Features: advanced, file_system, camera, folding 727 VCL_call c recv lookup 727 VCL_call c hash hash 727 Hit c 2067494928 727 VCL_call c hit deliver 727 VCL_call c deliver deliver 727 TxProtocol c HTTP/1.1 727 TxStatus c 200 727 TxResponse c OK 727 TxHeader c Server: Apache/2.2.14 (Unix) 727 TxHeader c Last-Modified: Sun, 12 Dec 2010 19:51:47 GMT 727 TxHeader c Cache-Control: max-age=2592000 727 TxHeader c Expires: Wed, 12 Jan 2011 10:01:04 GMT 727 TxHeader c Content-Type: image/jpeg 727 TxHeader c Content-Length: 30522 727 TxHeader c Date: Tue, 14 Dec 2010 08:08:20 GMT 727 TxHeader c X-Varnish: 2086901880 2067494928 727 TxHeader c Age: 79636 727 TxHeader c Via: 1.1 varnish 727 TxHeader c Connection: keep-alive 727 Length c 30522 727 ReqEnd c 2086901880 1292314100.869229078 1292314101.006513119 0.137564182 0.000025034 0.137259007 0.137284 290 ReqStart c xx.xx.xx.xx 35582 2086901865 290 RxRequest c GET 290 RxURL c /content/7/0/8/0/18-16662853-708002298-t.jpg 290 RxProtocol c HTTP/1.1 290 RxHeader c User-Agent: Opera/9.80 (J2ME/MIDP; Opera Mini/5.1.21214/22.401; U; en) Presto/2.5.25 Version/10.54 290 RxHeader c Host: fsa.zedge.net 290 RxHeader c Accept: text/html, application/xml;q=0.9, application/xhtml+xml, image/png, image/jpeg, image/gif, image/x-xbitmap, */*;q=0.1 290 RxHeader c Accept-Language: en,en;q=0.9 290 RxHeader c Accept-Charset: iso-8859-1, utf-8, utf-16, *;q=0.1 290 RxHeader c Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0 290 RxHeader c Referer: http://www.zedge.net/ 290 RxHeader c Cookie: xxxx 290 RxHeader c Cookie2: $Version=1 290 RxHeader c Connection: Keep-Alive, TE 290 RxHeader c TE: deflate, gzip, chunked, identity, trailers 290 RxHeader c x-forwarded-for: xx 290 RxHeader c X-OperaMini-Phone: Generic # J2ME Midlet 290 RxHeader c X-OperaMini-Phone-UA: UNTRUSTED/1.0 290 RxHeader c X-OperaMini-Features: advanced, file_system, camera, folding 290 VCL_call c recv lookup 290 VCL_call c hash hash 290 Hit c 2067494928 290 VCL_call c hit deliver 290 VCL_call c deliver deliver 290 TxProtocol c HTTP/1.1 290 TxStatus c 200 290 TxResponse c OK 290 TxHeader c Server: Apache/2.2.14 (Unix) 290 TxHeader c Last-Modified: Sun, 12 Dec 2010 19:51:47 GMT 290 TxHeader c Cache-Control: max-age=2592000 290 TxHeader c Expires: Wed, 12 Jan 2011 10:01:04 GMT 290 TxHeader c Content-Type: image/jpeg 290 TxHeader c Content-Length: 30522 290 TxHeader c Date: Tue, 14 Dec 2010 08:08:20 GMT 290 TxHeader c X-Varnish: 2086901865 2067494928 290 TxHeader c Age: 79636 290 TxHeader c Via: 1.1 varnish 290 TxHeader c Connection: keep-alive 290 Length c 30522 290 ReqEnd c 2086901865 1292314100.808101892 1292314101.083616018 0.137889862 0.000026226 0.275487900 0.275514 391 SessionOpen c xx.xx.xx.xx 5349 :80 391 ReqStart c xx.xx.xx.xx 5349 2086902309 391 RxRequest c GET 391 RxURL c /content/7/0/8/0/18-16662853-708002298-t.jpg 391 RxProtocol c HTTP/1.1 391 RxHeader c Host: fsa.zedge.net 391 RxHeader c User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 391 RxHeader c Accept: image/png,image/*;q=0.8,*/*;q=0.5 391 RxHeader c Accept-Language: en-us,en;q=0.5 391 RxHeader c Accept-Encoding: gzip,deflate 391 RxHeader c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 391 RxHeader c Keep-Alive: 115 391 RxHeader c Connection: keep-alive 391 RxHeader c Referer: http://www.zedge.net/ 391 RxHeader c Cookie: xxxx 391 VCL_call c recv lookup 391 VCL_call c hash hash 391 Hit c 2067494928 391 VCL_call c hit deliver 391 VCL_call c deliver deliver 391 TxProtocol c HTTP/1.1 391 TxStatus c 200 391 TxResponse c OK 391 TxHeader c Server: Apache/2.2.14 (Unix) 391 TxHeader c Last-Modified: Sun, 12 Dec 2010 19:51:47 GMT 391 TxHeader c Cache-Control: max-age=2592000 391 TxHeader c Expires: Wed, 12 Jan 2011 10:01:04 GMT 391 TxHeader c Content-Type: image/jpeg 391 TxHeader c Content-Length: 30522 391 TxHeader c Date: Tue, 14 Dec 2010 08:08:22 GMT 391 TxHeader c X-Varnish: 2086902309 2067494928 391 TxHeader c Age: 79638 391 TxHeader c Via: 1.1 varnish 391 TxHeader c Connection: keep-alive 391 Length c 30522 391 ReqEnd c 2086902309 1292314102.957597971 1292314102.957654953 0.000035048 0.000020981 0.000036001 0.000057 988 SessionOpen c xx.xx.xx.xx 63428 :80 988 ReqStart c xx.xx.xx.xx 63428 2086902494 988 RxRequest c GET 988 RxURL c /content/7/0/8/0/18-16662853-708002298-t.jpg 988 RxProtocol c HTTP/1.1 988 RxHeader c Host: fsa.zedge.net 988 RxHeader c User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9) Gecko/2008052906 Firefox/3.0 988 RxHeader c Accept: image/png,image/*;q=0.8,*/*;q=0.5 988 RxHeader c Accept-Language: en-us,en;q=0.5 988 RxHeader c Accept-Encoding: gzip,deflate 988 RxHeader c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 988 RxHeader c Keep-Alive: 300 988 RxHeader c Connection: keep-alive 988 RxHeader c Referer: http://www.zedge.net/themes// 988 RxHeader c Cookie: xxxx 988 VCL_call c recv lookup 988 VCL_call c hash hash 988 Hit c 2067494928 988 VCL_call c hit deliver 988 VCL_call c deliver deliver 988 TxProtocol c HTTP/1.1 988 TxStatus c 200 988 TxResponse c OK 988 TxHeader c Server: Apache/2.2.14 (Unix) 988 TxHeader c Last-Modified: Sun, 12 Dec 2010 19:51:47 GMT 988 TxHeader c Cache-Control: max-age=2592000 988 TxHeader c Expires: Wed, 12 Jan 2011 10:01:04 GMT 988 TxHeader c Content-Type: image/jpeg 988 TxHeader c Content-Length: 30522 988 TxHeader c Date: Tue, 14 Dec 2010 08:08:23 GMT 988 TxHeader c X-Varnish: 2086902494 2067494928 988 TxHeader c Age: 79639 988 TxHeader c Via: 1.1 varnish 988 TxHeader c Connection: keep-alive 988 Length c 30522 988 ReqEnd c 2086902494 1292314103.791788101 1292314103.791872025 0.000230074 0.000032902 0.000051022 0.000084 212 SessionOpen c xx.xx.xx.xx 41211 :80 212 ReqStart c xx.xx.xx.xx 41211 2086902374 212 RxRequest c GET 212 RxURL c /content/7/0/8/0/18-16662853-708002298-t.jpg 212 RxProtocol c HTTP/1.0 212 RxHeader c Host: fsa.zedge.net 212 RxHeader c User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 ( .NET CLR 3.5.30729) 212 RxHeader c Accept: image/png,image/*;q=0.8,*/*;q=0.5 212 RxHeader c Accept-Language: en-us,en;q=0.5 212 RxHeader c Accept-Encoding: gzip,deflate 212 RxHeader c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 212 RxHeader c Keep-Alive: 115 212 RxHeader c Connection: keep-alive 212 RxHeader c Referer: http://www.zedge.net/ 212 RxHeader c Via: 1.0 Webcat-Skein-192.168.2.1 (awarrenhttp/2.0.0.4.7) 212 VCL_call c recv lookup 212 VCL_call c hash hash 212 Hit c 2067494928 212 VCL_call c hit deliver 212 VCL_call c deliver deliver 212 TxProtocol c HTTP/1.1 212 TxStatus c 200 212 TxResponse c OK 212 TxHeader c Server: Apache/2.2.14 (Unix) 212 TxHeader c Last-Modified: Sun, 12 Dec 2010 19:51:47 GMT 212 TxHeader c Cache-Control: max-age=2592000 212 TxHeader c Expires: Wed, 12 Jan 2011 10:01:04 GMT 212 TxHeader c Content-Type: image/jpeg 212 TxHeader c Content-Length: 30522 212 TxHeader c Date: Tue, 14 Dec 2010 08:08:23 GMT 212 TxHeader c X-Varnish: 2086902374 2067494928 212 TxHeader c Age: 79639 212 TxHeader c Via: 1.1 varnish 212 TxHeader c Connection: keep-alive 212 Length c 30522 212 ReqEnd c 2086902374 1292314103.289890051 1292314103.909584045 0.000030041 0.000020981 0.619673014 0.619694 1069 ReqStart c xx.xx.xx.xx 1988 2086903151 1069 RxRequest c GET 1069 RxURL c /content/7/0/8/0/18-16662853-708002298-t.jpg 1069 RxProtocol c HTTP/1.1 1069 RxHeader c Accept: */* 1069 RxHeader c Referer: http://www.zedge.net/themes/ 1069 RxHeader c Accept-Language: en-us 1069 RxHeader c Accept-Encoding: gzip, deflate 1069 RxHeader c User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; InfoPath.2) 1069 RxHeader c Host: fsa.zedge.net 1069 RxHeader c Connection: Keep-Alive 1069 RxHeader c Cookie: xxxx 1069 VCL_call c recv lookup 1069 VCL_call c hash hash 1069 Hit c 2067494928 1069 VCL_call c hit deliver 1069 VCL_call c deliver deliver 1069 TxProtocol c HTTP/1.1 1069 TxStatus c 200 1069 TxResponse c OK 1069 TxHeader c Server: Apache/2.2.14 (Unix) 1069 TxHeader c Last-Modified: Sun, 12 Dec 2010 19:51:47 GMT 1069 TxHeader c Cache-Control: max-age=2592000 1069 TxHeader c Expires: Wed, 12 Jan 2011 10:01:04 GMT 1069 TxHeader c Content-Type: image/jpeg 1069 TxHeader c Content-Length: 30522 1069 TxHeader c Date: Tue, 14 Dec 2010 08:08:26 GMT 1069 TxHeader c X-Varnish: 2086903151 2067494928 1069 TxHeader c Age: 79642 1069 TxHeader c Via: 1.1 varnish 1069 TxHeader c Connection: keep-alive 1069 Length c 30522 1069 ReqEnd c 2086903151 1292314106.261312008 1292314106.261378050 0.340662956 0.000026941 0.000039101 0.000066 229 SessionOpen c xx.xx.xx.xx 28728 :80 229 ReqStart c xx.xx.xx.xx 28728 2086903365 229 RxRequest c GET 229 RxURL c /content/7/0/8/0/18-16662853-708002298-t.jpg 229 RxProtocol c HTTP/1.1 229 RxHeader c Host: fsa.zedge.net 229 RxHeader c User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:2.0b7) Gecko/20100101 Firefox/4.0b7 229 RxHeader c Accept: image/png,image/*;q=0.8,*/*;q=0.5 229 RxHeader c Accept-Language: en-us,en;q=0.5 229 RxHeader c Accept-Encoding: gzip, deflate 229 RxHeader c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 229 RxHeader c Keep-Alive: 115 229 RxHeader c Connection: keep-alive 229 RxHeader c Referer: http://www.zedge.net/themes// 229 RxHeader c Cookie: xxxx 229 RxHeader c If-Modified-Since: Sun, 12 Dec 2010 19:51:47 GMT 229 RxHeader c Cache-Control: max-age=0 229 VCL_call c recv lookup 229 VCL_call c hash hash 229 Hit c 2067494928 229 VCL_call c hit deliver 229 VCL_call c deliver deliver 229 TxProtocol c HTTP/1.1 229 TxStatus c 304 229 TxResponse c Not Modified 229 TxHeader c Date: Tue, 14 Dec 2010 08:08:27 GMT 229 TxHeader c Via: 1.1 varnish 229 TxHeader c X-Varnish: 2086903365 229 TxHeader c Last-Modified: Sun, 12 Dec 2010 19:51:47 GMT 229 TxHeader c Cache-Control: max-age=2592000 229 TxHeader c Expires: Wed, 12 Jan 2011 10:01:04 GMT 229 TxHeader c Connection: keep-alive 229 Length c 0 229 ReqEnd c 2086903365 1292314107.039563894 1292314107.039627075 0.000093937 0.000032187 0.000030994 0.000063 115 ReqStart c xx.xx.xx.xx 50325 2086893861 115 RxRequest c GET 115 RxURL c /content/7/0/8/0/18-16662853-708002298-t.jpg 115 RxProtocol c HTTP/1.1 115 RxHeader c User-Agent: Opera/9.80 (Windows NT 6.1; U; en) Presto/2.6.30 Version/10.63 115 RxHeader c Host: fsa.zedge.net 115 RxHeader c Accept: text/html, application/xml;q=0.9, application/xhtml+xml, image/png, image/jpeg, image/gif, image/x-xbitmap, */*;q=0.1 115 RxHeader c Accept-Language: en-US,en;q=0.9 115 RxHeader c Accept-Charset: iso-8859-1, utf-8, utf-16, *;q=0.1 115 RxHeader c Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0 115 RxHeader c Referer: http://www.zedge.net/ 115 RxHeader c Cookie: xxxx 115 RxHeader c Cookie2: $Version=1 115 RxHeader c Connection: Keep-Alive 115 VCL_call c recv lookup 115 VCL_call c hash hash 115 Hit c 2067494928 115 VCL_call c hit deliver 115 VCL_call c deliver deliver 115 TxProtocol c HTTP/1.1 115 TxStatus c 200 115 TxResponse c OK 115 TxHeader c Server: Apache/2.2.14 (Unix) 115 TxHeader c Last-Modified: Sun, 12 Dec 2010 19:51:47 GMT 115 TxHeader c Cache-Control: max-age=2592000 115 TxHeader c Expires: Wed, 12 Jan 2011 10:01:04 GMT 115 TxHeader c Content-Type: image/jpeg 115 TxHeader c Content-Length: 30522 115 TxHeader c Date: Tue, 14 Dec 2010 08:06:10 GMT 115 TxHeader c X-Varnish: 2086893861 2067494928 115 TxHeader c Age: 79506 115 TxHeader c Via: 1.1 varnish 115 TxHeader c Connection: keep-alive 115 Length c 30522 115 ReqEnd c 2086893861 1292313970.238029957 1292314107.295713902 0.000037909 0.000041962 137.057641983 137.057684 890 SessionOpen c xx.xx.xx.xx 26479 :80 890 ReqStart c xx.xx.xx.xx 26479 2086903723 890 RxRequest c GET 890 RxURL c /content/7/0/8/0/18-16662853-708002298-t.jpg 890 RxProtocol c HTTP/1.1 890 RxHeader c Host: fsa.zedge.net 890 RxHeader c User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.16) Gecko/20101130 Firefox/3.5.16 (.NET CLR 3.5.30729) 890 RxHeader c Accept: image/png,image/*;q=0.8,*/*;q=0.5 890 RxHeader c Accept-Language: en-us,en;q=0.5 890 RxHeader c Accept-Encoding: gzip,deflate 890 RxHeader c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 890 RxHeader c Keep-Alive: 300 890 RxHeader c Connection: keep-alive 890 RxHeader c Referer: http://www.zedge.net/ 890 RxHeader c Cookie: xxxx 890 VCL_call c recv lookup 890 VCL_call c hash hash 890 Hit c 2067494928 890 VCL_call c hit deliver 890 VCL_call c deliver deliver 890 TxProtocol c HTTP/1.1 890 TxStatus c 200 890 TxResponse c OK 890 TxHeader c Server: Apache/2.2.14 (Unix) 890 TxHeader c Last-Modified: Sun, 12 Dec 2010 19:51:47 GMT 890 TxHeader c Cache-Control: max-age=2592000 890 TxHeader c Expires: Wed, 12 Jan 2011 10:01:04 GMT 890 TxHeader c Content-Type: image/jpeg 890 TxHeader c Content-Length: 30522 890 TxHeader c Date: Tue, 14 Dec 2010 08:08:28 GMT 890 TxHeader c X-Varnish: 2086903723 2067494928 890 TxHeader c Age: 79644 890 TxHeader c Via: 1.1 varnish 890 TxHeader c Connection: keep-alive 890 Length c 30522 890 ReqEnd c 2086903723 1292314108.585319996 1292314108.585386038 0.000025988 0.000020981 0.000045061 0.000066 1009 SessionOpen c xx.xx.xx.xx 62329 :80 1009 ReqStart c xx.xx.xx.xx 62329 2086903984 1009 RxRequest c GET 1009 RxURL c /content/7/0/8/0/18-16662853-708002298-t.jpg 1009 RxProtocol c HTTP/1.1 1009 RxHeader c Host: fsa.zedge.net 1009 RxHeader c User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; ar; rv:1.9.2) Gecko/20100115 Firefox/3.6 1009 RxHeader c Accept: image/png,image/*;q=0.8,*/*;q=0.5 1009 RxHeader c Accept-Language: ar,en-us;q=0.7,en;q=0.3 1009 RxHeader c Accept-Encoding: gzip,deflate 1009 RxHeader c Accept-Charset: UTF-8,* 1009 RxHeader c Keep-Alive: 115 1009 RxHeader c Connection: keep-alive 1009 RxHeader c Referer: http://www.zedge.net/ 1009 RxHeader c Cookie: xxxx 1009 VCL_call c recv lookup 1009 VCL_call c hash hash 1009 Hit c 2067494928 1009 VCL_call c hit deliver 1009 VCL_call c deliver deliver 1009 TxProtocol c HTTP/1.1 1009 TxStatus c 200 1009 TxResponse c OK 1009 TxHeader c Server: Apache/2.2.14 (Unix) 1009 TxHeader c Last-Modified: Sun, 12 Dec 2010 19:51:47 GMT 1009 TxHeader c Cache-Control: max-age=2592000 1009 TxHeader c Expires: Wed, 12 Jan 2011 10:01:04 GMT 1009 TxHeader c Content-Type: image/jpeg 1009 TxHeader c Content-Length: 30522 1009 TxHeader c Date: Tue, 14 Dec 2010 08:08:29 GMT 1009 TxHeader c X-Varnish: 2086903984 2067494928 1009 TxHeader c Age: 79645 1009 TxHeader c Via: 1.1 varnish 1009 TxHeader c Connection: keep-alive 1009 Length c 30522 1009 ReqEnd c 2086903984 1292314109.441423893 1292314109.441485882 0.000111818 0.000025034 0.000036955 0.000062 434 SessionOpen c xx.xx.xx.xx 50555 :80 434 ReqStart c xx.xx.xx.xx 50555 2086903612 434 RxRequest c GET 434 RxURL c /content/7/0/8/0/18-16662853-708002298-t.jpg 434 RxProtocol c HTTP/1.1 434 RxHeader c Host: fsa.zedge.net 434 RxHeader c Connection: keep-alive 434 RxHeader c Referer: http://www.zedge.net/ 434 RxHeader c Accept: */* 434 RxHeader c User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.10 (KHTML, like Gecko) Chrome/8.0.552.215 Safari/534.10 434 RxHeader c Accept-Encoding: gzip,deflate,sdch 434 RxHeader c Accept-Language: en-US,en;q=0.8 434 RxHeader c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3 434 RxHeader c Cookie: xxxx 434 VCL_call c recv lookup 434 VCL_call c hash hash 434 Hit c 2067494928 434 VCL_call c hit deliver 434 VCL_call c deliver deliver 434 TxProtocol c HTTP/1.1 434 TxStatus c 200 434 TxResponse c OK 434 TxHeader c Server: Apache/2.2.14 (Unix) 434 TxHeader c Last-Modified: Sun, 12 Dec 2010 19:51:47 GMT 434 TxHeader c Cache-Control: max-age=2592000 434 TxHeader c Expires: Wed, 12 Jan 2011 10:01:04 GMT 434 TxHeader c Content-Type: image/jpeg 434 TxHeader c Content-Length: 30522 434 TxHeader c Date: Tue, 14 Dec 2010 08:08:28 GMT 434 TxHeader c X-Varnish: 2086903612 2067494928 434 TxHeader c Age: 79643 434 TxHeader c Via: 1.1 varnish 434 TxHeader c Connection: keep-alive 434 Length c 30522 434 ReqEnd c 2086903612 1292314108.215404987 1292314110.524404049 0.000029087 0.000020027 2.308979034 2.308999 805 ReqStart c xx.xx.xx.xx 15040 2086904276 805 RxRequest c GET 805 RxURL c /content/7/0/8/0/18-16662853-708002298-t.jpg 805 RxProtocol c HTTP/1.1 805 RxHeader c Host: fsa.zedge.net 805 RxHeader c User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 ( .NET CLR 3.5.30729; .NET4.0E) 805 RxHeader c Accept: image/png,image/*;q=0.8,*/*;q=0.5 805 RxHeader c Accept-Language: en-us,en;q=0.5 805 RxHeader c Accept-Encoding: gzip,deflate 805 RxHeader c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 805 RxHeader c Keep-Alive: 115 805 RxHeader c Connection: keep-alive 805 RxHeader c Referer: http://www.zedge.net/themes/ 805 RxHeader c Cookie: xxxx 805 VCL_call c recv lookup 805 VCL_call c hash hash 805 Hit c 2067494928 805 VCL_call c hit deliver 805 VCL_call c deliver deliver 805 TxProtocol c HTTP/1.1 805 TxStatus c 200 805 TxResponse c OK 805 TxHeader c Server: Apache/2.2.14 (Unix) 805 TxHeader c Last-Modified: Sun, 12 Dec 2010 19:51:47 GMT 805 TxHeader c Cache-Control: max-age=2592000 805 TxHeader c Expires: Wed, 12 Jan 2011 10:01:04 GMT 805 TxHeader c Content-Type: image/jpeg 805 TxHeader c Content-Length: 30522 805 TxHeader c Date: Tue, 14 Dec 2010 08:08:30 GMT 805 TxHeader c X-Varnish: 2086904276 2067494928 805 TxHeader c Age: 79646 805 TxHeader c Via: 1.1 varnish 805 TxHeader c Connection: keep-alive 805 Length c 30522 805 ReqEnd c 2086904276 1292314110.554608107 1292314110.554663897 0.000011206 0.000024796 0.000030994 0.000056 683 SessionOpen c xx.xx.xx.xx 1892 :80 683 ReqStart c xx.xx.xx.xx 1892 2086904292 683 RxRequest c GET 683 RxURL c /content/7/0/8/0/18-16662853-708002298-t.jpg 683 RxProtocol c HTTP/1.1 683 RxHeader c User-Agent: Opera/9.20 (Windows NT 5.1; U; en) 683 RxHeader c Host: fsa.zedge.net 683 RxHeader c Accept: text/html, application/xml;q=0.9, application/xhtml+xml, image/png, image/jpeg, image/gif, image/x-xbitmap, */*;q=0.1 683 RxHeader c Accept-Language: en-US,en;q=0.9 683 RxHeader c Accept-Charset: iso-8859-1, utf-8, utf-16, *;q=0.1 683 RxHeader c Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0 683 RxHeader c Referer: http://www.zedge.net/download/ 683 RxHeader c Cookie: xxxx 683 RxHeader c Cookie2: $Version=1 683 RxHeader c Connection: Keep-Alive, TE 683 RxHeader c TE: deflate, gzip, chunked, identity, trailers 683 VCL_call c recv lookup 683 VCL_call c hash hash 683 Hit c 2067494928 683 VCL_call c hit deliver 683 VCL_call c deliver deliver 683 TxProtocol c HTTP/1.1 683 TxStatus c 200 683 TxResponse c OK 683 TxHeader c Server: Apache/2.2.14 (Unix) 683 TxHeader c Last-Modified: Sun, 12 Dec 2010 19:51:47 GMT 683 TxHeader c Cache-Control: max-age=2592000 683 TxHeader c Expires: Wed, 12 Jan 2011 10:01:04 GMT 683 TxHeader c Content-Type: image/jpeg 683 TxHeader c Content-Length: 30522 683 TxHeader c Date: Tue, 14 Dec 2010 08:08:30 GMT 683 TxHeader c X-Varnish: 2086904292 2067494928 683 TxHeader c Age: 79646 683 TxHeader c Via: 1.1 varnish 683 TxHeader c Connection: keep-alive 683 Length c 30522 683 ReqEnd c 2086904292 1292314110.618022919 1292314110.618091106 0.000022888 0.000025034 0.000043154 0.000068 760 SessionOpen c xx.xx.xx.xx 2279 :80 760 ReqStart c xx.xx.xx.xx 2279 2086904299 760 RxRequest c GET 760 RxURL c /content/7/0/8/0/18-16662853-708002298-t.jpg 760 RxProtocol c HTTP/1.1 760 RxHeader c Host: fsa.zedge.net 760 RxHeader c User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 ( .NET CLR 3.5.30729) 760 RxHeader c Accept: image/png,image/*;q=0.8,*/*;q=0.5 760 RxHeader c Accept-Language: en-us,en;q=0.5 760 RxHeader c Accept-Encoding: gzip,deflate 760 RxHeader c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 760 RxHeader c Keep-Alive: 115 760 RxHeader c Connection: keep-alive 760 RxHeader c Referer: http://www.zedge.net/ 760 VCL_call c recv lookup 760 VCL_call c hash hash 760 Hit c 2067494928 760 VCL_call c hit deliver 760 VCL_call c deliver deliver 760 TxProtocol c HTTP/1.1 760 TxStatus c 200 760 TxResponse c OK 760 TxHeader c Server: Apache/2.2.14 (Unix) 760 TxHeader c Last-Modified: Sun, 12 Dec 2010 19:51:47 GMT 760 TxHeader c Cache-Control: max-age=2592000 760 TxHeader c Expires: Wed, 12 Jan 2011 10:01:04 GMT 760 TxHeader c Content-Type: image/jpeg 760 TxHeader c Content-Length: 30522 760 TxHeader c Date: Tue, 14 Dec 2010 08:08:30 GMT 760 TxHeader c X-Varnish: 2086904299 2067494928 760 TxHeader c Age: 79646 760 TxHeader c Via: 1.1 varnish 760 TxHeader c Connection: keep-alive 760 Length c 30522 760 ReqEnd c 2086904299 1292314110.634171963 1292314110.634222031 0.000025988 0.000018120 0.000031948 0.000050 From michaell at dazzit.com Mon Dec 20 21:07:30 2010 From: michaell at dazzit.com (Michael Lenaghan) Date: Mon, 20 Dec 2010 16:07:30 -0500 Subject: Incorrect example of blocking IE for deflate encoding Message-ID: The page at http://www.varnish-cache.org/trac/wiki/FAQ/Compression has this example: if (req.http.Accept-Encoding) { if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") { # No point in compressing these remove req.http.Accept-Encoding; } elsif (req.http.Accept-Encoding ~ "gzip") { set req.http.Accept-Encoding = "gzip"; } elsif (req.http.Accept-Encoding ~ "deflate" && req.http.user-agent !~ "Internet Explorer") { set req.http.Accept-Encoding = "deflate"; } else { # unkown algorithm remove req.http.Accept-Encoding; } } According to the list of user agent strings at http://www.useragentstring.com/pages/Internet%20Explorer/ IE never uses the value "Internet Explorer"; rather, it uses "MSIE". From fla_torres at yahoo.com.br Tue Dec 21 00:35:36 2010 From: fla_torres at yahoo.com.br (Flavio Torres) Date: Mon, 20 Dec 2010 22:35:36 -0200 Subject: 503 errors on POST In-Reply-To: <201012201058.10015.alex@arcalpin.com> References: <201012201058.10015.alex@arcalpin.com> Message-ID: <4D0FF658.60805@yahoo.com.br> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello, On 12/20/2010 07:58 AM, Modesto Alexandre wrote: > Here are the errors I have (I have masked the private information): > > http://demo.ovh.net/view/6ac24fbc5400039bc86fd3444556ef76/0.colored Your backend seems not be good. How is your backend_health [1] ? [1] - http://www.varnish-cache.org/trac/wiki/BackendPolling - -- /*---------------------------------------------------*/ /* * Flavio Torres - Administrador de Sistemas*/ /* /_\ */int y,z;main(x){x=(x>0?-9:x);z=((z=(x+5) /* o. .o */)>0?z:-z);printf(!x&&++y?"\n":z?z>y%3+y/ /* ((Y)) */3?" ":x<-5?"/":"\\":y?"|":"*");y-9?main( /* ()) (() */++x):puts(" _|_|_")&&puts(" \\___/");} /* ()^() A party animal isn't just for Christmas.*/ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) iEUEARECAAYFAk0P9lAACgkQNRQApncg295BxQCg3w1uUX8vy70tH2m/LpYS1Vfu 634AmJgNhgbw8drpddXmQ0u0JYveohM= =G0nB -----END PGP SIGNATURE----- From dennis.hendriksen at kalooga.com Tue Dec 21 07:01:56 2010 From: dennis.hendriksen at kalooga.com (Dennis Hendriksen) Date: Tue, 21 Dec 2010 08:01:56 +0100 Subject: Varnish 2.0.6 thread pile-up In-Reply-To: <4D0F61A5.8030105@yahoo.com.br> References: <1292843133.20355.44.camel@kalooga-dennis> <4D0F61A5.8030105@yahoo.com.br> Message-ID: <1292914916.2425.2.camel@kalooga-dennis> Hello Flavio, I've tried to disable syncookies, but it didn't help. Sending syncookies is most likely an effect and not the cause. Anyone else who has an idea? Greetings, Dennis On Mon, 2010-12-20 at 12:01 -0200, Flavio Torres wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On 12/20/2010 09:05 AM, Dennis Hendriksen wrote: > > Furthermore we sometimes see the following in /var/log/messages: > > Dec 19 22:40:04 cache1 kernel: possible SYN flooding on port 80. Sending > > cookies. > > Dec 19 22:41:05 cache1 kernel: possible SYN flooding on port 80. Sending > > cookies. > > Try to change at your sysctl.conf the following value: > > net.ipv4.tcp_syncookies = 0 > > > See: http://www.varnish-cache.org/trac/wiki/Performance > > > hope this helps > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.10 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iEYEARECAAYFAk0PYY0ACgkQNRQApncg2948vgCfaxCKQoCwPGG5GSNwXv45DcIg > CIAAniRud/SV/vzBTaZb3xlcbIOKAPPz > =fygR > -----END PGP SIGNATURE----- > From alex at arcalpin.com Tue Dec 21 07:48:07 2010 From: alex at arcalpin.com (Modesto Alexandre) Date: Tue, 21 Dec 2010 08:48:07 +0100 Subject: 503 errors on POST In-Reply-To: <4D0FF658.60805@yahoo.com.br> References: <201012201058.10015.alex@arcalpin.com> <4D0FF658.60805@yahoo.com.br> Message-ID: <201012210848.07720.alex@arcalpin.com> Le mardi 21 d?cembre 2010, Flavio Torres a ?crit : > Hello, > > On 12/20/2010 07:58 AM, Modesto Alexandre wrote: > > Here are the errors I have (I have masked the private information): > > > > http://demo.ovh.net/view/6ac24fbc5400039bc86fd3444556ef76/0.colored > > Your backend seems not be good. How is your backend_health [1] ? > > [1] - http://www.varnish-cache.org/trac/wiki/BackendPolling Backends look good : varnishadm -T localhost:6082 debug.health Backend backend1 is Healthy Current states good: 10 threshold: 8 window: 10 Average responsetime of good probes: 0.144168 Oldest Newest ================================================================ 4444444444444444444444444444444444444444444444444444444444444444 Good IPv4 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX Good Xmit RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR- Good Recv HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH- Happy Backend backend3 is Healthy Current states good: 9 threshold: 8 window: 10 Average responsetime of good probes: 0.106183 Oldest Newest ================================================================ 4444444444444444444444444444444444444444444444444444444444444444 Good IPv4 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX Good Xmit RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR-R Good Recv HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH-H Happy Backend backend2 is Healthy Current states good: 9 threshold: 8 window: 10 Average responsetime of good probes: 0.092774 Oldest Newest ================================================================ 4444444444444444444444444444444444444444444444444444444444444444 Good IPv4 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX Good Xmit RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR- Good Recv HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH- Happy Configuration of backend pooling : backend backend1 { .host = "ip.ip.ip.ip"; .port = "80"; .connect_timeout = 300s; .first_byte_timeout = 300s; .between_bytes_timeout = 300s; .probe = { .url = "/url.gif"; .timeout = 1500 ms; .interval = 5s; .window = 10; .threshold = 8; } } backend2 & backend3 have the same configuration any idea ? Alex. From stig at zedge.net Tue Dec 21 08:36:14 2010 From: stig at zedge.net (Stig Bakken) Date: Tue, 21 Dec 2010 09:36:14 +0100 Subject: hanging connections In-Reply-To: References: <20101220082503.GB2445@freud> Message-ID: Kristian, please let me know whether I should file a ticket for this. - Stig On Mon, Dec 20, 2010 at 4:14 PM, Stig Bakken wrote: > On Mon, Dec 20, 2010 at 9:25 AM, Kristian Lyngstol < > kristian at varnish-software.com> wrote: > >> On Fri, Dec 17, 2010 at 03:40:35PM +0100, Stig Bakken wrote: >> > I have a script processing varnishlog output to give me the serving >> > latency based on the ReqEnd tag, and frequently see objects served from >> > cache, size around 25kB, that take 5+ seconds to serve. >> >> If there are two requests for an object, request A and request B, the >> following will happen: >> >> Time | Action >> 0s | A asks for /foo >> | Cache-miss - fetch from backend >> 0.5s | B asks for /foo >> | Cache hit on busy object - wait for it >> 5s | Web server is finished delivering /foo >> 5.1s | A gets /foo - cache miss >> 5.1s | B gets /foo - cache hit >> >> It's a bit difficult to spot this as it happens, it's been a matter of >> discussion recently (Arthur being the one who first became aware of the >> issue). >> >> It sounds like this might be what you are seeing. >> > > Here are some more details. The attached file is an edited varnishlog > dump, with elapsed time appended after each ReqEnd. Notice that one of the > requests takes 137 seconds, during which time all of the others in the dump > complete much faster. The Age: header is high, which rules out the cache > injection race condition, right? > > This is with 2.1.4 installed from the official rpm on RHEL, btw. > > - Stig > > > -- > Stig Bakken > CTO, Zedge.net - free your phone! > -- Stig Bakken CTO, Zedge.net - free your phone! -------------- next part -------------- An HTML attachment was scrubbed... URL: From kristian at varnish-software.com Tue Dec 21 08:51:44 2010 From: kristian at varnish-software.com (Kristian Lyngstol) Date: Tue, 21 Dec 2010 09:51:44 +0100 Subject: hanging connections In-Reply-To: References: <20101220082503.GB2445@freud> Message-ID: <20101221085143.GA4680@freud> On Tue, Dec 21, 2010 at 09:36:14AM +0100, Stig Bakken wrote: > Kristian, please let me know whether I should file a ticket for this. So far I haven't seen anything that suggests a bug. A single slow response is all the logs show - for better or worse. That could just as well just be a slow client or packet loss. What I need to be able to determine if this is a bug in Varnish, network-issue or phase-of-moon glitch is: - Several occurrences of slow responses and the requests around it in varnishlog (you can obfuscate the IP - but I need to be able to tell which requests come from the same IP: ie: if there are two IP's, A.A.A.A and B.B.B.B are ok, but X.X.X.X for both are not). - 'varnishstat -1' output - 'sysctl -a' output - Network configuration on the machine(s), specially with regards to any load balancers - 'dmesg' - Complete VCL (I do believe I already have that) - All start-up parameters. Any other running services on the machines. If that doesn't reveal the nature issue, I'll need to see tcpdump-output of the incident to track it further - at that point, obfuscation is not practically possible without risking the integrity of the data. In short: all you can give me. The issue doesn't have an obvious explanation and there isn't really any reason why a cache hit would take any real amount of time when we rule out busy objects. I should also point out that I can only really spend time on this to rule out or confirm if it is a varnish bug or not. -- Kristian Lyngst?l Product Specialist Varnish Software -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: Digital signature URL: From stig at zedge.net Tue Dec 21 09:03:55 2010 From: stig at zedge.net (Stig Bakken) Date: Tue, 21 Dec 2010 10:03:55 +0100 Subject: hanging connections In-Reply-To: <20101221085143.GA4680@freud> References: <20101220082503.GB2445@freud> <20101221085143.GA4680@freud> Message-ID: Understood (of course). I'll assemble the information needed. - Stig On Tue, Dec 21, 2010 at 9:51 AM, Kristian Lyngstol < kristian at varnish-software.com> wrote: > On Tue, Dec 21, 2010 at 09:36:14AM +0100, Stig Bakken wrote: > > Kristian, please let me know whether I should file a ticket for this. > > So far I haven't seen anything that suggests a bug. > > A single slow response is all the logs show - for better or worse. That > could just as well just be a slow client or packet loss. > > What I need to be able to determine if this is a bug in Varnish, > network-issue or phase-of-moon glitch is: > > - Several occurrences of slow responses and the requests around it in > varnishlog (you can obfuscate the IP - but I need to be able to tell > which requests come from the same IP: ie: if there are two IP's, A.A.A.A > and B.B.B.B are ok, but X.X.X.X for both are not). > - 'varnishstat -1' output > - 'sysctl -a' output > - Network configuration on the machine(s), specially with regards to any > load balancers > - 'dmesg' > - Complete VCL (I do believe I already have that) > - All start-up parameters. Any other running services on the machines. > > If that doesn't reveal the nature issue, I'll need to see tcpdump-output of > the incident to track it further - at that point, obfuscation is not > practically possible without risking the integrity of the data. > > In short: all you can give me. > > The issue doesn't have an obvious explanation and there isn't really any > reason why a cache hit would take any real amount of time when we rule out > busy objects. I should also point out that I can only really spend time on > this to rule out or confirm if it is a varnish bug or not. > > -- > Kristian Lyngst?l > Product Specialist > Varnish Software > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.10 (GNU/Linux) > > iQEcBAEBAgAGBQJNEGqeAAoJEIC1IG5rXEfqLC4IAJm4o8ernD3XLfHvzHO0ui77 > dp8Dy+zXYr9mJ5Mjfyz1CPjHOvuloRBlV5Gt20z872mtcPTedVMuk8w9fWlUo1RU > lwWaMnZ4qHhwhWg5zWiDi/hqno5mpR6BPcVoT+pQyOJp2Z4jB/rRvkV3IkXt1Eb/ > TtLhKROxkiGUIa/uayzPGyb2KMjD+KrSn8Jny66vzrvetPpIUd3/A7txuJ4RHNMI > MWx4zed8O9Bm4FR9z109Nr3umr89Lw4AcheAa17m91ZTP1DcFx+DfRTn2IlpFeb0 > 6+fAB9svoC+K7+PJqnMo9XYp9COzCNWWN9fv34x30rIouH6DSRtOwOyJ9gihOGw= > =Jmur > -----END PGP SIGNATURE----- > > -- Stig Bakken CTO, Zedge.net - free your phone! -------------- next part -------------- An HTML attachment was scrubbed... URL: From havardf at met.no Tue Dec 21 10:46:05 2010 From: havardf at met.no (=?utf-8?Q?H=C3=A5vard_Futs=C3=A6ter?=) Date: Tue, 21 Dec 2010 10:46:05 +0000 (UTC) Subject: backend max_connections In-Reply-To: <4CF6489D.9030306@holtzbrinck.com> Message-ID: <821436202.649479.1292928365167.JavaMail.root@imap1b> I have been wondering about the same thing. Anyone who can help? ----- Original Message ----- > Hi all, > > documentation is a bit terse about the max_connection parameter for > backend declarations. > I can't seem to find anything about what exactly is going to happen at > the frontend side when concurrent backend connections are maxed out. > > My guess is that the client will either receive a 503 error or a stale > object will be served, depending on the grace time. > > Am I right or wrong? > > Regards, > Thomas > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc -- ------ Mvh, H?vard Futs?ter Telefon: +47 22 96 32 69 From perbu at varnish-software.com Tue Dec 21 11:38:35 2010 From: perbu at varnish-software.com (Per Buer) Date: Tue, 21 Dec 2010 12:38:35 +0100 Subject: backend max_connections In-Reply-To: <821436202.649479.1292928365167.JavaMail.root@imap1b> References: <4CF6489D.9030306@holtzbrinck.com> <821436202.649479.1292928365167.JavaMail.root@imap1b> Message-ID: Hi H?vard, Thomas, On Tue, Dec 21, 2010 at 11:46 AM, H?vard Futs?ter wrote: > I have been wondering about the same thing. Anyone who can help? > > ----- Original Message ----- > > Hi all, > > > > documentation is a bit terse about the max_connection parameter for > > backend declarations. > > I can't seem to find anything about what exactly is going to happen at > > the frontend side when concurrent backend connections are maxed out. > > > > My guess is that the client will either receive a 503 error or a stale > > object will be served, depending on the grace time. > > > > Am I right or wrong? > You get an error. You could handle it in vcl_error if you'd like to. If you're using saint mode a stale object might be served. Per. Top-quoting is, uhm, not good. -- Per Buer, Varnish Software Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer Varnish makes websites fly! Want to learn more about Varnish? http://www.varnish-software.com/whitepapers -------------- next part -------------- An HTML attachment was scrubbed... URL: From havardf at met.no Tue Dec 21 12:41:50 2010 From: havardf at met.no (=?utf-8?Q?H=C3=A5vard_Futs=C3=A6ter?=) Date: Tue, 21 Dec 2010 12:41:50 +0000 (UTC) Subject: backend max_connections In-Reply-To: Message-ID: <1371947421.651072.1292935310995.JavaMail.root@imap1b> ----- Original Message ----- > Hi H?vard, Thomas, > > > On Tue, Dec 21, 2010 at 11:46 AM, H?vard Futs?ter < havardf at met.no > > wrote: > > > I have been wondering about the same thing. Anyone who can help? > > > > > ----- Original Message ----- > > Hi all, > > > > documentation is a bit terse about the max_connection parameter for > > backend declarations. > > I can't seem to find anything about what exactly is going to happen > > at > > the frontend side when concurrent backend connections are maxed out. > > > > My guess is that the client will either receive a 503 error or a > > stale > > object will be served, depending on the grace time. > > > > Am I right or wrong? > > > > > You get an error. You could handle it in vcl_error if you'd like to. > If you're using saint mode a stale object might be served. Thanks! > > Per. > > > Top-quoting is, uhm, not good. Agreed, my bad. > -- > Per Buer, Varnish Software > Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer > Varnish makes websites fly! > Want to learn more about Varnish? > http://www.varnish-software.com/whitepapers > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc -- ------ Mvh, H?vard Futs?ter Telefon: +47 22 96 32 69 From cooltechemail at gmail.com Tue Dec 21 22:03:17 2010 From: cooltechemail at gmail.com (Vincent) Date: Tue, 21 Dec 2010 14:03:17 -0800 Subject: child process of varnishd restarts automatically In-Reply-To: <20101220142217.GC2445@freud> References: <20101220142217.GC2445@freud> Message-ID: Hi Kristian, Thank you for your reply. I think the problem is when varnish uses more than 4GB virtual memory on 32-bit, it will restart. I started varnish with "-s malloc,1.5G". Varnish will gradually increase memory usage and stop at about 1.7G. However, the virtual memory size is growing continuously and it doesn't seem to stop. We restarted varnish yesterday and currently the VM size is 2.7G, and it is still growing. Is this normal? Yes, I know we should have used 32-bit in the first place. The reason is that we have some other applications work on 32-bit only. Anyway, we will build a 64-bit box dedicated for Varnish. However, isn't it nice to have varnish run without any problems on 32-bit machines? Thank you again. Vincent On Mon, Dec 20, 2010 at 6:22 AM, Kristian Lyngstol < kristian at varnish-software.com> wrote: > Hi, > > On Mon, Dec 20, 2010 at 06:04:53AM -0800, Vincent wrote: > > We are running varnish on a 32-bit centos 5 box and we have noticed that > > sometimes the child process of varnishd will restart automatically. > > You should be using 64bit. > > > varnishd[12707]: Child (12708) Panic message: Missing errorhandling code > in > > sma_alloc(), storage_malloc.c line 81: Condition((sma->s.ptr) != 0) not > > true.errno = 12 (Cannot allocate memory) thread = (cache-worker) ident = > > (...) > > > Varnishd was started with option "malloc,1.5G" and it runs without any > > problem for hours before the child process restarts. When the child > process > > restarts, the system still have about 1.5G free memory so this is not a > out > > of memory issue. > > You are out of memory - possibly 32-bit-related issue since you have a > artificially limited address space. > > That your system has 1.5GB "free" memory doesn't mean you can use it. > That's why you want to run 64-bit systems: then you can actually use the > memory too. An other possible culprit is disabling overcommit - but if you > actually had free memory, it's unlikely that that's your problem. > > The proper solution to this problem is to reinstall your system using a > 64-bit operating system. If you prefer wasting countless hours, you can try > to estimate the overhead of the object count (roughly 1kB) and how that > affects the total memory usage of varnishd, factor in stack size of > threads, constant memory overhead and the available address space left to > you - it's seriously not fun (or all that easy). > > - Kristian > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.10 (GNU/Linux) > > iQEcBAEBAgAGBQJND2aYAAoJEIC1IG5rXEfqVFgH/0TeE1wpicXP8Y4ze4EYTNBp > 6/H+XFWs4YOukZkPING4r61oa5KPc0zDAjgsfrlbeY1f3stQtIXRXJ3zAI+i3XTp > zHELLKw91deQGOWBepvjojaUZxBEg6gcwIKMP1gSK17HI6T5OyLoxlunr0rvvbRd > Yonmqj5EOAXpZSMBsf8Qe31spybhEcwvG6LYsu/Oho7SxBk7aU7RcvvqbF2EW6OJ > Z60IrJK94qLYkY7dFT5/BguPp2G3CaEL6Ok3P6j8WSvWA/d4i0VXHnWeosG+dvoh > W3b8g+4ACeqJSI7uxepR0M2gFD6y2Ait4+VeV14JAWUkqyOZ2s7c92fANou6Jf4= > =7I7m > -----END PGP SIGNATURE----- > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kristian at varnish-software.com Wed Dec 22 00:13:05 2010 From: kristian at varnish-software.com (Kristian Lyngstol) Date: Wed, 22 Dec 2010 01:13:05 +0100 Subject: child process of varnishd restarts automatically In-Reply-To: References: <20101220142217.GC2445@freud> Message-ID: <20101222001305.GA23087@freud> Hi, On Tue, Dec 21, 2010 at 02:03:17PM -0800, Vincent wrote: > I think the problem is when varnish uses more than 4GB virtual memory on > 32-bit, it will restart. I started varnish with "-s malloc,1.5G". Varnish > will gradually increase memory usage and stop at about 1.7G. However, the > virtual memory size is growing continuously and it doesn't seem to stop. We > restarted varnish yesterday and currently the VM size is 2.7G, and it is > still growing. Is this normal? The virtual memory usage depends on a large number of factors, and when you are limited to 3GB of it (assuming a user/kernel split at 3G/1G) trying to determine if there's a leak or not is just not worth the effort. Varnish is a massively parallel application, each thread has an overhead, each session has an overhead, each VCL file has an overhead, each object cached has an overhead. And so forth. These components were all designed for a 64-bit architecture where address space is a non-issue. Over the years, we've monitored real memory usage to determine memory leaks and changes in memory overhead. This effort was, for example, the reason obj_workspace was removed. Every reported memory leak I've investigated to date has turned out to be a wild goose chase. It's taken us quite some time to determine a reasonable estimate for _real_ memory overhead on 64-bit as it changed from Varnish 2.0 to 2.1, and I don't think you'll see us try to do the same thing for address space just because it would be a funny little thing to have on a platform nobody really need. If it is of interest, the per-object overhead we are seeing is currently roughly 1kB for each object - but that's resident memory, not virtual. With a million objects, that means 1GB of overhead just for the objects - I wouldn't be surprised if it was twice that in virtual memory. If you really really need 32-bit for some silly application, spend 100$ buying a dedicated 64-bit-box for Varnish. Because Varnish really really needs 64-bit. You'll save yourself a great deal of pain and you'll be running on the platform Varnish was designed for. > Yes, I know we should have used 32-bit in the first place. The reason is > that we have some other applications work on 32-bit only. Anyway, we will > build a 64-bit box dedicated for Varnish. However, isn't it nice to have > varnish run without any problems on 32-bit machines? No, we don't care much about 32-bit. Use 64-bit. If you got patches for 32-bit, we'll take a look, but trying to fix a legacy-platform isn't time well spent. If you fix it, we can check your conclusions, but that's about it. And in this case, it doesn't look like there's anything to fix. If I were you, I'd investigate the "32-bit only" application holding you back, not the 64-bit application actually using your hardware to its potential. You are trying to tune your Ferrari for a race track in your back yard: it'll never work well in that scenario and you'll never be happy with it like that. I usually call 32-bit platforms "Nintendo platforms" in the context of Varnish (and compare it to Varnish running on an iPhone, for example): It's fun if it works, but we wont spend any considerably time tuning it for the platform. But that's not really a fair thing to say - after all, the Nintendo 64 came out in 1996. - Kristian From imanandshah at gmail.com Wed Dec 22 07:12:24 2010 From: imanandshah at gmail.com (Anand Shah) Date: Wed, 22 Dec 2010 12:42:24 +0530 Subject: Too many 503 backend errors Message-ID: Hi, I am running varnish on 64 bit Centos. Getting too many 503 errors recently and i tried debugging it with some help from Forums on mailinglist. All requests and responses given below. I am not able to reach any conclusion; please help me as what could be done to understand the exact issue. Attempts: 1. Varnish in debug mode 2. gdb 3. strace (linux system and lib calls) All outputs given below for reference: VARNISHLOGS: 180 ReqStart c 203.223.188.130 4147 583956431 180 RxRequest c GET 180 RxURL c /0/OasDefault/LO_TataNano_Kerala_01/nano_640x480.swf?click=http://ads.mydomain.com/media/ads/click_lx.ads/www.mydomainmail.com/logout.htm/L-29/562622447/Middle/OasDefault/LO_TataNano_Kerala_01/LO_TataNano_UP_0101.html/79392b38676b7a2b4548304141525538?URL 180 RxProtocol c HTTP/1.1 180 RxHeader c Host: imads.mydomain.com 180 RxHeader c User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.11) Gecko/20101012 Firefox/3.6.11 180 RxHeader c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 180 RxHeader c Accept-Language: en-us,en;q=0.5 180 RxHeader c Accept-Encoding: gzip,deflate 180 RxHeader c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 180 RxHeader c Keep-Alive: 115 180 RxHeader c Connection: keep-alive 180 RxHeader c Referer: http://login.mydomain.com/bn/logout.cgi?formname=general&function_name=logout&login=gopal.pgv&session_id=6L25PK1KWKlno6s8Ul4BgQBopj70kV10 180 RxHeader c Cookie: OAX=y9+8gkz+EH0AARU8; RLOC=%5F%5F4i1NtmpD2pc%5F%5FU9WPPAYrSWc%5F%5FtHonjGX8AnI%5F%5Find%5F%5F; RMFL=011PVICcU3018r0; RuW=1292401506681957; accounttype=77; Rp=g%3D2%26a%3D20%26c%3D10%26s%3D30%26cn%3D099%26z%3D000000%26p%3D031%26e%3D00%26d%3D_9_%26i 180 VCL_call c recv lookup 180 VCL_call c hash hash 180 VCL_call c miss fetch 180 FetchError c no backend connection 180 VCL_call c error deliver 180 Length c 487 180 VCL_call c deliver deliver 180 TxProtocol c HTTP/1.1 180 TxStatus c 503 180 TxResponse c Service Unavailable 180 TxHeader c Content-Type: text/html; charset=utf-8 180 TxHeader c Content-Length: 487 180 TxHeader c Date: Wed, 22 Dec 2010 06:39:13 GMT 180 TxHeader c Connection: close 180 TxHeader c Server: mydomain/2.0.6 180 TxHeader c X-Cache: TCP_MISS 180 ReqEnd c 583956431 1292999948.983546972 1292999953.983323097 0.000041008 4.999706984 0.000069141 MANUALLY SENT THE SAME REQUEST USING CURL: [root at mymachine ~]# curl --head --header host:imads.mydomain.com "http://localhost//0/OasDefault/IIPM_CPL_Nov10_B3_MW/iipm_winter02_300x600_10dec_03.gif" HTTP/1.1 200 OK Last-Modified: Thu, 02 Dec 2010 12:19:14 GMT P3P: CP="NON NID PSAa PSDa OUR IND UNI COM NAV STA",policyref="/w3c/p3p.xml" Content-Type: image/gif cache-control: max-age=604800 Content-Length: 4150 Date: Wed, 22 Dec 2010 06:39:26 GMT Connection: keep-alive H-Served-By: mymachine Server: mydomain/2.0.6 X-Cache: TCP_MISS [root at mymachine ~]# curl --head --header host:imads.mydomain.com "http://localhost//0/OasDefault/IIPM_CPL_Nov10_B3_MW/iipm_winter02_300x600_10dec_03.gif" HTTP/1.1 200 OK Last-Modified: Thu, 02 Dec 2010 12:19:14 GMT P3P: CP="NON NID PSAa PSDa OUR IND UNI COM NAV STA",policyref="/w3c/p3p.xml" Content-Type: image/gif cache-control: max-age=604800 Content-Length: 4150 Date: Wed, 22 Dec 2010 06:39:33 GMT Connection: keep-alive H-Served-By: mymachine Server: mydomain/2.0.6 X-Cache: TCP_HIT GDB Trace: [New Thread 0x40ccd940 (LWP 11258)] Loaded symbols for /lib64/libpthread.so.0 Reading symbols from /lib64/libnsl.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/libnsl.so.1 Reading symbols from /lib64/libm.so.6...(no debugging symbols found)...done. Loaded symbols for /lib64/libm.so.6 Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib64/libc.so.6 Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /lib64/librt.so.1... (no debugging symbols found)...done. Loaded symbols for /lib64/librt.so.1 Reading symbols from /lib64/libpcre.so.0...(no debugging symbols found)...done. Loaded symbols for /lib64/libpcre.so.0 Reading symbols from /lib64/libnss_files.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libnss_files.so.2 Symbol file not found for ./vcl.FANefPfn.so 0x0000003c31acae46 in poll () from /lib64/libc.so.6 (gdb) n Cannot find bounds of current function (gdb) bt #0 0x0000003c31acae46 in poll () from /lib64/libc.so.6 #1 0x00000035d58056aa in CLS_Poll () from /usr/lib64/libvarnish.so.1 #2 0x0000000000414b11 in CLI_Run () #3 0x00000000004212a3 in child_main () #4 0x000000000042f6d9 in ?? () #5 0x000000000042ff09 in ?? () #6 0x00000035d5807c47 in ?? () from /usr/lib64/libvarnish.so.1 #7 0x00000035d58082d8 in vev_schedule () from /usr/lib64/libvarnish.so.1 #8 0x000000000042f943 in MGT_Run () #9 0x000000000043deca in main () VARNISG DEBUG OUTPUT: Child (11257) not responding to ping, killing it. Child (11257) not responding to ping, killing it. Child (11257) not responding to ping, killing it. Child (11257) not responding to ping, killing it. Child (11257) not responding to ping, killing it. Child (11257) not responding to ping, killing it. Child (11257) not responding to ping, killing it. Child (11257) not responding to ping, killing it. Child (11257) not responding to ping, killing it. Child (11257) not responding to ping, killing it. Child (11257) not responding to ping, killing it. Child (11257) not responding to ping, killing it. Child (11257) not responding to ping, killing it. Regards, Anand From varnish at mm.quex.org Wed Dec 22 07:56:46 2010 From: varnish at mm.quex.org (Michael Alger) Date: Wed, 22 Dec 2010 15:56:46 +0800 Subject: Too many 503 backend errors In-Reply-To: References: Message-ID: <20101222075646.GA22736@grum.quex.org> On Wed, Dec 22, 2010 at 12:42:24PM +0530, Anand Shah wrote: > I am running varnish on 64 bit Centos. Getting too many 503 errors > recently and i tried debugging it with some help from Forums on > mailinglist. > VARNISHLOGS: > > 180 VCL_call c miss fetch > 180 FetchError c no backend connection > 180 VCL_call c error deliver In my experience, "no backend connection" indicates that whatever backend was selected to handle this request was unavailable due to the health check/probe returning an error. What was the backend health at the time? You can use the 'debug.health' command in the CLI to view health information, or varnishlog -i Backend_health to see the results of the health checks as they occur. If a backend is marked as 'sick' then Varnish won't send any requests to it, since it's unlikely to help the situation. From imanandshah at gmail.com Wed Dec 22 08:38:56 2010 From: imanandshah at gmail.com (Anand Shah) Date: Wed, 22 Dec 2010 14:08:56 +0530 Subject: varnish-misc Digest, Vol 57, Issue 42 In-Reply-To: References: Message-ID: I immediately fired a test call using CURL which indicated me a tcp_miss and i also got the response from origin. So backend was working and in healthy state. [root at mymachine ~]# curl --head --header host:imads.mydomain.com "http://localhost//0/OasDefault/IIPM_CPL_Nov10_B3_MW/iipm_winter02_300x600_10dec_03.gif" HTTP/1.1 200 OK Last-Modified: Thu, 02 Dec 2010 12:19:14 GMT P3P: CP="NON NID PSAa PSDa OUR IND UNI COM NAV STA",policyref="/w3c/p3p.xml" Content-Type: image/gif cache-control: max-age=604800 Content-Length: 4150 Date: Wed, 22 Dec 2010 06:39:26 GMT Connection: keep-alive H-Served-By: mymachine Server: mydomain/2.0.6 X-Cache: TCP_MISS [root at mymachine ~]# curl --head --header host:imads.mydomain.com "http://localhost//0/OasDefault/IIPM_CPL_Nov10_B3_MW/iipm_winter02_300x600_10dec_03.gif" HTTP/1.1 200 OK Last-Modified: Thu, 02 Dec 2010 12:19:14 GMT P3P: CP="NON NID PSAa PSDa OUR IND UNI COM NAV STA",policyref="/w3c/p3p.xml" Content-Type: image/gif cache-control: max-age=604800 Content-Length: 4150 Date: Wed, 22 Dec 2010 06:39:33 GMT Connection: keep-alive H-Served-By: mymachine Server: mydomain/2.0.6 X-Cache: TCP_HIT Regards, Anand On Wed, Dec 22, 2010 at 12:42:24PM +0530, Anand Shah wrote: > I am running varnish on 64 bit Centos. Getting too many 503 errors > recently and i tried debugging it with some help from Forums on > mailinglist. > VARNISHLOGS: > > 180 VCL_call c miss fetch > 180 FetchError c no backend connection > 180 VCL_call c error deliver In my experience, "no backend connection" indicates that whatever backend was selected to handle this request was unavailable due to the health check/probe returning an error. What was the backend health at the time? You can use the 'debug.health' command in the CLI to view health information, or varnishlog -i Backend_health to see the results of the health checks as they occur. If a backend is marked as 'sick' then Varnish won't send any requests to it, since it's unlikely to help the situation. _______________________________________________ varnish-misc mailing list varnish-misc at varnish-cache.org http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc From imanandshah at gmail.com Wed Dec 22 08:39:15 2010 From: imanandshah at gmail.com (Anand Shah) Date: Wed, 22 Dec 2010 14:09:15 +0530 Subject: Too many 503 backend errors Message-ID: I immediately fired a test call using CURL which indicated me a tcp_miss and i also got the response from origin. So backend was working and in healthy state. [root at mymachine ~]# curl --head --header host:imads.mydomain.com "http://localhost//0/OasDefault/IIPM_CPL_Nov10_B3_MW/iipm_winter02_300x600_10dec_03.gif" HTTP/1.1 200 OK Last-Modified: Thu, 02 Dec 2010 12:19:14 GMT P3P: CP="NON NID PSAa PSDa OUR IND UNI COM NAV STA",policyref="/w3c/p3p.xml" Content-Type: image/gif cache-control: max-age=604800 Content-Length: 4150 Date: Wed, 22 Dec 2010 06:39:26 GMT Connection: keep-alive H-Served-By: mymachine Server: mydomain/2.0.6 X-Cache: TCP_MISS [root at mymachine ~]# curl --head --header host:imads.mydomain.com "http://localhost//0/OasDefault/IIPM_CPL_Nov10_B3_MW/iipm_winter02_300x600_10dec_03.gif" HTTP/1.1 200 OK Last-Modified: Thu, 02 Dec 2010 12:19:14 GMT P3P: CP="NON NID PSAa PSDa OUR IND UNI COM NAV STA",policyref="/w3c/p3p.xml" Content-Type: image/gif cache-control: max-age=604800 Content-Length: 4150 Date: Wed, 22 Dec 2010 06:39:33 GMT Connection: keep-alive H-Served-By: mymachine Server: mydomain/2.0.6 X-Cache: TCP_HIT Regards, Anand On Wed, Dec 22, 2010 at 12:42:24PM +0530, Anand Shah wrote: > I am running varnish on 64 bit Centos. Getting too many 503 errors > recently and i tried debugging it with some help from Forums on > mailinglist. > VARNISHLOGS: > > 180 VCL_call c miss fetch > 180 FetchError c no backend connection > 180 VCL_call c error deliver In my experience, "no backend connection" indicates that whatever backend was selected to handle this request was unavailable due to the health check/probe returning an error. What was the backend health at the time? You can use the 'debug.health' command in the CLI to view health information, or varnishlog -i Backend_health to see the results of the health checks as they occur. If a backend is marked as 'sick' then Varnish won't send any requests to it, since it's unlikely to help the situation. _______________________________________________ varnish-misc mailing list varnish-misc at varnish-cache.org http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc From traian.bratucu at eea.europa.eu Wed Dec 22 08:51:17 2010 From: traian.bratucu at eea.europa.eu (Traian Bratucu) Date: Wed, 22 Dec 2010 09:51:17 +0100 Subject: Too many 503 backend errors In-Reply-To: References: Message-ID: Your backend is probably incorrectly defined in the vcl file. Please attach the vcl file or paste the backend definition, it's most likely wrong. Of course you could also have some sort of firewall problem and should check for that too. ________________________________________ 180 FetchError c no backend connection 180 VCL_call c error deliver From rtshilston at gmail.com Wed Dec 22 08:54:02 2010 From: rtshilston at gmail.com (Robert Shilston) Date: Wed, 22 Dec 2010 08:54:02 +0000 Subject: Too many 503 backend errors In-Reply-To: References: Message-ID: <648AB4F2-DD23-45D3-A62F-538D9AB7C986@gmail.com> Anand, Check that vcl_recv is actually setting a backend. It might be that there's a route through the VCL that doesn't cause a backend to be specified. Rob On 22 Dec 2010, at 08:39, Anand Shah wrote: > I immediately fired a test call using CURL which indicated me a > tcp_miss and i also got the response from origin. > > So backend was working and in healthy state. > > > > [root at mymachine ~]# curl --head --header host:imads.mydomain.com > "http://localhost//0/OasDefault/IIPM_CPL_Nov10_B3_MW/iipm_winter02_300x600_10dec_03.gif" > HTTP/1.1 200 OK > Last-Modified: Thu, 02 Dec 2010 12:19:14 GMT > P3P: CP="NON NID PSAa PSDa OUR IND UNI COM NAV STA",policyref="/w3c/p3p.xml" > Content-Type: image/gif > cache-control: max-age=604800 > Content-Length: 4150 > Date: Wed, 22 Dec 2010 06:39:26 GMT > Connection: keep-alive > H-Served-By: mymachine > Server: mydomain/2.0.6 > X-Cache: TCP_MISS > > [root at mymachine ~]# curl --head --header host:imads.mydomain.com > "http://localhost//0/OasDefault/IIPM_CPL_Nov10_B3_MW/iipm_winter02_300x600_10dec_03.gif" > HTTP/1.1 200 OK > Last-Modified: Thu, 02 Dec 2010 12:19:14 GMT > P3P: CP="NON NID PSAa PSDa OUR IND UNI COM NAV STA",policyref="/w3c/p3p.xml" > Content-Type: image/gif > cache-control: max-age=604800 > Content-Length: 4150 > Date: Wed, 22 Dec 2010 06:39:33 GMT > Connection: keep-alive > H-Served-By: mymachine > Server: mydomain/2.0.6 > X-Cache: TCP_HIT > > > > > Regards, > Anand > > On Wed, Dec 22, 2010 at 12:42:24PM +0530, Anand Shah wrote: >> I am running varnish on 64 bit Centos. Getting too many 503 errors >> recently and i tried debugging it with some help from Forums on >> mailinglist. > >> VARNISHLOGS: >> >> 180 VCL_call c miss fetch >> 180 FetchError c no backend connection >> 180 VCL_call c error deliver > > In my experience, "no backend connection" indicates that whatever > backend was selected to handle this request was unavailable due to the > health check/probe returning an error. > > What was the backend health at the time? You can use the 'debug.health' > command in the CLI to view health information, or > > varnishlog -i Backend_health > > to see the results of the health checks as they occur. If a backend is > marked as 'sick' then Varnish won't send any requests to it, since > it's unlikely to help the situation. > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc From fhelmschrott at gmail.com Wed Dec 22 09:23:04 2010 From: fhelmschrott at gmail.com (Frank Helmschrott) Date: Wed, 22 Dec 2010 10:23:04 +0100 Subject: Problems translating 2.0.x vcl to 2.1.x Message-ID: Hi, i manage upgrading my varnishd to the latest from the varnish RHEL repos and now trying to translate my vcl to the newest syntax. I already changed some things which worked but i still get this error: Message from VCC-compiler: Expected an action, 'if', '{' or '}' (input Line 204 Pos 5) discard; ----#######- Running VCC-compiler failed, exit 1 VCL compilation failed here's the line from vcl: sub vcl_timeout { discard; } I have to state that i copied over a vcl that i found to fit my needs for now and didn't write it myself. I couldn't find any changes with the section from 2.0.x to 2.1.x so i'm a bit confused for now. Thanks for helping! -- Frank From andreas.haase at evolver.de Wed Dec 22 09:28:47 2010 From: andreas.haase at evolver.de (Andreas Haase) Date: Wed, 22 Dec 2010 10:28:47 +0100 Subject: Problems translating 2.0.x vcl to 2.1.x In-Reply-To: References: Message-ID: <1293010127.3082.8.camel@andreas-haase-linux.evolver.de> Hello, Am Mittwoch, den 22.12.2010, 10:23 +0100 schrieb Frank Helmschrott: > sub vcl_timeout { > discard; > } try it this way: sub vcl_timeout { return(discard); } From imanandshah at gmail.com Wed Dec 22 10:52:51 2010 From: imanandshah at gmail.com (Anand Shah) Date: Wed, 22 Dec 2010 16:22:51 +0530 Subject: Too many 503 backend errors In-Reply-To: <648AB4F2-DD23-45D3-A62F-538D9AB7C986@gmail.com> References: <648AB4F2-DD23-45D3-A62F-538D9AB7C986@gmail.com> Message-ID: It works for 6 out of 10 requests. I have also found that varnish polls backend before sending a GET request. Since i never received any request on my apache server for this dropped requests i am in doubt as if what makes varnish feel the backend is in unhealthy state. Is there any way that i can troubleshoot? Anand On 12/22/10, Robert Shilston wrote: > Anand, > > Check that vcl_recv is actually setting a backend. It might be that there's > a route through the VCL that doesn't cause a backend to be specified. > > Rob > > On 22 Dec 2010, at 08:39, Anand Shah wrote: > >> I immediately fired a test call using CURL which indicated me a >> tcp_miss and i also got the response from origin. >> >> So backend was working and in healthy state. >> >> >> >> [root at mymachine ~]# curl --head --header host:imads.mydomain.com >> "http://localhost//0/OasDefault/IIPM_CPL_Nov10_B3_MW/iipm_winter02_300x600_10dec_03.gif" >> HTTP/1.1 200 OK >> Last-Modified: Thu, 02 Dec 2010 12:19:14 GMT >> P3P: CP="NON NID PSAa PSDa OUR IND UNI COM NAV >> STA",policyref="/w3c/p3p.xml" >> Content-Type: image/gif >> cache-control: max-age=604800 >> Content-Length: 4150 >> Date: Wed, 22 Dec 2010 06:39:26 GMT >> Connection: keep-alive >> H-Served-By: mymachine >> Server: mydomain/2.0.6 >> X-Cache: TCP_MISS >> >> [root at mymachine ~]# curl --head --header host:imads.mydomain.com >> "http://localhost//0/OasDefault/IIPM_CPL_Nov10_B3_MW/iipm_winter02_300x600_10dec_03.gif" >> HTTP/1.1 200 OK >> Last-Modified: Thu, 02 Dec 2010 12:19:14 GMT >> P3P: CP="NON NID PSAa PSDa OUR IND UNI COM NAV >> STA",policyref="/w3c/p3p.xml" >> Content-Type: image/gif >> cache-control: max-age=604800 >> Content-Length: 4150 >> Date: Wed, 22 Dec 2010 06:39:33 GMT >> Connection: keep-alive >> H-Served-By: mymachine >> Server: mydomain/2.0.6 >> X-Cache: TCP_HIT >> >> >> >> >> Regards, >> Anand >> >> On Wed, Dec 22, 2010 at 12:42:24PM +0530, Anand Shah wrote: >>> I am running varnish on 64 bit Centos. Getting too many 503 errors >>> recently and i tried debugging it with some help from Forums on >>> mailinglist. >> >>> VARNISHLOGS: >>> >>> 180 VCL_call c miss fetch >>> 180 FetchError c no backend connection >>> 180 VCL_call c error deliver >> >> In my experience, "no backend connection" indicates that whatever >> backend was selected to handle this request was unavailable due to the >> health check/probe returning an error. >> >> What was the backend health at the time? You can use the 'debug.health' >> command in the CLI to view health information, or >> >> varnishlog -i Backend_health >> >> to see the results of the health checks as they occur. If a backend is >> marked as 'sick' then Varnish won't send any requests to it, since >> it's unlikely to help the situation. >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > From rtshilston at gmail.com Wed Dec 22 10:58:41 2010 From: rtshilston at gmail.com (Robert Shilston) Date: Wed, 22 Dec 2010 10:58:41 +0000 Subject: Too many 503 backend errors In-Reply-To: References: <648AB4F2-DD23-45D3-A62F-538D9AB7C986@gmail.com> Message-ID: Anand, I've simplified our VCL as much as practical to demonstrate how we use the backends. > backend server2 { > .host = "a.b.c.12"; > .port = "80"; > .probe = { > .request = > "GET /healthforvarnish HTTP/1.1" > "Host: mysite.com" > "User-agent: Varnish-probe" > "Connection: close"; > .interval = 5s; > .timeout = 2500ms; > .window = 10; > .threshold = 8; > } > } > backend server3 { > .host = "a.b.c.13"; > .port = "80"; > .probe = { > .request = > "GET /healthforvarnish HTTP/1.1" > "Host: mysite.com" > "User-agent: Varnish-probe" > "Connection: close"; > .interval = 5s; > .timeout = 2500ms; > .window = 10; > .threshold = 8; > } > } > ... > backend server2failsafe { .host = "a.b.c.12"; .port="80"; } > backend server3failsafe { .host = "a.b.c.13"; .port="80"; } > ... > > director workingpool random { > { .backend = server2; .weight = 1; } > { .backend = server3; .weight = 1; } > ... > } > > director failsafepool random { > { .backend = server2failsafe; .weight = 1; } > { .backend = server3failsafe; .weight = 1; } > ... > } > > ... > > sub vcl_recv { > set req.backend = workingpool; > > if (!req.backend.healthy) { > set req.backend = failsafepool; > } > If you implement something like this, then run varnishncsa, you'll see which backends are being selected, and should deliver responses for 100% of your requests. If it does, then you'll need to debug the health checks to see why your backends might be being considered unhealthy. Rob On 22 Dec 2010, at 10:52, Anand Shah wrote: > It works for 6 out of 10 requests. I have also found that varnish > polls backend before sending a GET request. Since i never received any > request on my apache server for this dropped requests i am in doubt as > if what makes varnish feel the backend is in unhealthy state. > > Is there any way that i can troubleshoot? > > > > Anand > > On 12/22/10, Robert Shilston wrote: >> Anand, >> >> Check that vcl_recv is actually setting a backend. It might be that there's >> a route through the VCL that doesn't cause a backend to be specified. >> >> Rob >> >> On 22 Dec 2010, at 08:39, Anand Shah wrote: >> >>> I immediately fired a test call using CURL which indicated me a >>> tcp_miss and i also got the response from origin. >>> >>> So backend was working and in healthy state. >>> >>> >>> >>> [root at mymachine ~]# curl --head --header host:imads.mydomain.com >>> "http://localhost//0/OasDefault/IIPM_CPL_Nov10_B3_MW/iipm_winter02_300x600_10dec_03.gif" >>> HTTP/1.1 200 OK >>> Last-Modified: Thu, 02 Dec 2010 12:19:14 GMT >>> P3P: CP="NON NID PSAa PSDa OUR IND UNI COM NAV >>> STA",policyref="/w3c/p3p.xml" >>> Content-Type: image/gif >>> cache-control: max-age=604800 >>> Content-Length: 4150 >>> Date: Wed, 22 Dec 2010 06:39:26 GMT >>> Connection: keep-alive >>> H-Served-By: mymachine >>> Server: mydomain/2.0.6 >>> X-Cache: TCP_MISS >>> >>> [root at mymachine ~]# curl --head --header host:imads.mydomain.com >>> "http://localhost//0/OasDefault/IIPM_CPL_Nov10_B3_MW/iipm_winter02_300x600_10dec_03.gif" >>> HTTP/1.1 200 OK >>> Last-Modified: Thu, 02 Dec 2010 12:19:14 GMT >>> P3P: CP="NON NID PSAa PSDa OUR IND UNI COM NAV >>> STA",policyref="/w3c/p3p.xml" >>> Content-Type: image/gif >>> cache-control: max-age=604800 >>> Content-Length: 4150 >>> Date: Wed, 22 Dec 2010 06:39:33 GMT >>> Connection: keep-alive >>> H-Served-By: mymachine >>> Server: mydomain/2.0.6 >>> X-Cache: TCP_HIT >>> >>> >>> >>> >>> Regards, >>> Anand >>> >>> On Wed, Dec 22, 2010 at 12:42:24PM +0530, Anand Shah wrote: >>>> I am running varnish on 64 bit Centos. Getting too many 503 errors >>>> recently and i tried debugging it with some help from Forums on >>>> mailinglist. >>> >>>> VARNISHLOGS: >>>> >>>> 180 VCL_call c miss fetch >>>> 180 FetchError c no backend connection >>>> 180 VCL_call c error deliver >>> >>> In my experience, "no backend connection" indicates that whatever >>> backend was selected to handle this request was unavailable due to the >>> health check/probe returning an error. >>> >>> What was the backend health at the time? You can use the 'debug.health' >>> command in the CLI to view health information, or >>> >>> varnishlog -i Backend_health >>> >>> to see the results of the health checks as they occur. If a backend is >>> marked as 'sick' then Varnish won't send any requests to it, since >>> it's unlikely to help the situation. >>> >>> _______________________________________________ >>> varnish-misc mailing list >>> varnish-misc at varnish-cache.org >>> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >>> >>> _______________________________________________ >>> varnish-misc mailing list >>> varnish-misc at varnish-cache.org >>> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> >> From kristian at varnish-software.com Wed Dec 22 12:07:16 2010 From: kristian at varnish-software.com (Kristian Lyngstol) Date: Wed, 22 Dec 2010 13:07:16 +0100 Subject: Problems translating 2.0.x vcl to 2.1.x In-Reply-To: <1293010127.3082.8.camel@andreas-haase-linux.evolver.de> References: <1293010127.3082.8.camel@andreas-haase-linux.evolver.de> Message-ID: <20101222120716.GA31622@freud> On Wed, Dec 22, 2010 at 10:28:47AM +0100, Andreas Haase wrote: > Am Mittwoch, den 22.12.2010, 10:23 +0100 schrieb Frank Helmschrott: > > sub vcl_timeout { > > discard; > > } > > try it this way: Or: # sub vcl_timeout { # return(discard); # Discard isn't actually usable, thus commented out. # } ;-) - Kristian -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: Digital signature URL: From A.Hongens at netmatch.nl Wed Dec 22 12:22:53 2010 From: A.Hongens at netmatch.nl (=?iso-8859-1?Q?Angelo_H=F6ngens?=) Date: Wed, 22 Dec 2010 12:22:53 +0000 Subject: blocking not working Message-ID: <6A7ABA19243F1E4EADD8BB1563CDDCCB15F398@TIL-EXCH-05.netmatch.local> I just added another Bad Guy to my balancer's block list, and I want them to see a 403 access denied instead of content. It worked in the past (at least back in the 2.0.x age), but now it does not seem to work anymore. Perhaps some syntax changed in 2.1.x regarding this? I'm running 2.1.4 on CentOS 5.5 x64. Here's some of my VCL: acl block { "a.b.91.19"; /* 20100301 making dummy requests */ "c.d.40.34"; /* 20100618 There are quite many invalid requests to our RSS */ "e.f.195.11"; /* 20101221 scraping */ } sub vcl_recv { if ( client.ip ~ block ) { error 403 "Access denied"; } # Add a unique header containing the client address remove req.http.X-Forwarded-For; set req.http.X-Forwarded-For = client.ip; .. } But when I look in my varnishncsa log, I still see successful requests being passed: e.f.195.11 - - [22/Dec/2010:13:19:02 +0100] "GET http://www.example.com/accommodation/accoinfo.aspx?accommodationId=197473 HTTP/1.1" 301 0 "-" "Mozilla/4.0 (compatible; Win32; WinHttp.WinHttpRequest.5)" 0.007832050 miss e.f.195.11 - - [22/Dec/2010:13:19:04 +0100] "GET http://www.example.com/verenigde-staten-van-amerika/south-carolina/charleston/charleston-marriott/hotel/informatie HTTP/1.1" 200 176281 "-" "Mozilla/4.0 (compatible; Win32; WinHttp.WinHttpRequest.5)" 0.665360928 miss e.f.195.11 - - [22/Dec/2010:13:19:04 +0100] "GET http://www.example.com/accommodation/accoinfo.aspx?accommodationId=197474 HTTP/1.1" 301 0 "-" "Mozilla/4.0 (compatible; Win32; WinHttp.WinHttpRequest.5)" 0.006700993 miss e.f.195.11 - - [22/Dec/2010:13:19:05 +0100] "GET http://www.example.com/verenigde-staten-van-amerika/south-carolina/charleston/church-street-inn/appartement/informatie HTTP/1.1" 200 163794 "-" "Mozilla/4.0 (compatible; Win32; WinHttp.WinHttpRequest.5)" 0.241801977 miss e.f.195.11 - - [22/Dec/2010:13:19:05 +0100] "GET http://www.example.com/accommodation/accoinfo.aspx?accommodationId=197475 HTTP/1.1" 301 0 "-" "Mozilla/4.0 (compatible; Win32; WinHttp.WinHttpRequest.5)" 0.006364822 miss e.f.195.11 - - [22/Dec/2010:13:19:06 +0100] "GET http://www.example.com/verenigde-staten-van-amerika/south-carolina/north-charleston/comfort-inn-coliseum/hotel/informatie HTTP/1.1" 200 171431 "-" "Mozilla/4.0 (compatible; Win32; WinHttp.WinHttpRequest.5)" 0.625963926 miss -- With kind regards, Angelo H?ngens Systems Administrator ------------------------------------------ NetMatch tourism internet software solutions Ringbaan Oost 2b 5013 CA Tilburg T: +31 (0)13 5811088 F: +31 (0)13 5821239 mailto:A.Hongens at netmatch.nl http://www.netmatch.nl ------------------------------------------ From ccastro at altavoz.net Wed Dec 22 14:20:26 2010 From: ccastro at altavoz.net (Claudio Castro) Date: Wed, 22 Dec 2010 11:20:26 -0300 Subject: cookie issue Message-ID: <4D12092A.6050108@altavoz.net> hi, My site uses cookies to authenticate users, but I have problems when I use varnish: from the backend side, all the clients come from the same cookie (i.e. varnish), and not from several origins (customers), any ideas? Thanks in advance. -- Claudio Castro N. Ingeniero Jefe Area de Plataforma AltaVoz S.A. http://www.altavoz.net Vi?a del Mar: 2 Poniente 355 of 53 +56 32 276 8060 Santiago: Pedro de Valdivia 555 of 315 +56 2 585 4264 From scaunter at topscms.com Wed Dec 22 15:25:12 2010 From: scaunter at topscms.com (Caunter, Stefan) Date: Wed, 22 Dec 2010 10:25:12 -0500 Subject: blocking not working In-Reply-To: <6A7ABA19243F1E4EADD8BB1563CDDCCB15F398@TIL-EXCH-05.netmatch.local> References: <6A7ABA19243F1E4EADD8BB1563CDDCCB15F398@TIL-EXCH-05.netmatch.local> Message-ID: <7F0AA702B8A85A4A967C4C8EBAD6902CB6523E@TMG-EVS02.torstar.net> I'm seeing 301 responses, do you have a redirect rule that is getting executed first? Stefan Caunter :: Senior Systems Administrator :: TOPS e: scaunter at topscms.com :: m: (416) 561-4871 www.thestar.com www.topscms.com -----Original Message----- From: varnish-misc-bounces at varnish-cache.org [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Angelo H?ngens Sent: December-22-10 7:23 AM To: 'varnish-misc at varnish-cache.org' Subject: blocking not working I just added another Bad Guy to my balancer's block list, and I want them to see a 403 access denied instead of content. It worked in the past (at least back in the 2.0.x age), but now it does not seem to work anymore. Perhaps some syntax changed in 2.1.x regarding this? I'm running 2.1.4 on CentOS 5.5 x64. Here's some of my VCL: acl block { "a.b.91.19"; /* 20100301 making dummy requests */ "c.d.40.34"; /* 20100618 There are quite many invalid requests to our RSS */ "e.f.195.11"; /* 20101221 scraping */ } sub vcl_recv { if ( client.ip ~ block ) { error 403 "Access denied"; } # Add a unique header containing the client address remove req.http.X-Forwarded-For; set req.http.X-Forwarded-For = client.ip; .. } But when I look in my varnishncsa log, I still see successful requests being passed: e.f.195.11 - - [22/Dec/2010:13:19:02 +0100] "GET http://www.example.com/accommodation/accoinfo.aspx?accommodationId=197473 HTTP/1.1" 301 0 "-" "Mozilla/4.0 (compatible; Win32; WinHttp.WinHttpRequest.5)" 0.007832050 miss e.f.195.11 - - [22/Dec/2010:13:19:04 +0100] "GET http://www.example.com/verenigde-staten-van-amerika/south-carolina/charleston/charleston-marriott/hotel/informatie HTTP/1.1" 200 176281 "-" "Mozilla/4.0 (compatible; Win32; WinHttp.WinHttpRequest.5)" 0.665360928 miss e.f.195.11 - - [22/Dec/2010:13:19:04 +0100] "GET http://www.example.com/accommodation/accoinfo.aspx?accommodationId=197474 HTTP/1.1" 301 0 "-" "Mozilla/4.0 (compatible; Win32; WinHttp.WinHttpRequest.5)" 0.006700993 miss e.f.195.11 - - [22/Dec/2010:13:19:05 +0100] "GET http://www.example.com/verenigde-staten-van-amerika/south-carolina/charleston/church-street-inn/appartement/informatie HTTP/1.1" 200 163794 "-" "Mozilla/4.0 (compatible; Win32; WinHttp.WinHttpRequest.5)" 0.241801977 miss e.f.195.11 - - [22/Dec/2010:13:19:05 +0100] "GET http://www.example.com/accommodation/accoinfo.aspx?accommodationId=197475 HTTP/1.1" 301 0 "-" "Mozilla/4.0 (compatible; Win32; WinHttp.WinHttpRequest.5)" 0.006364822 miss e.f.195.11 - - [22/Dec/2010:13:19:06 +0100] "GET http://www.example.com/verenigde-staten-van-amerika/south-carolina/north-charleston/comfort-inn-coliseum/hotel/informatie HTTP/1.1" 200 171431 "-" "Mozilla/4.0 (compatible; Win32; WinHttp.WinHttpRequest.5)" 0.625963926 miss -- With kind regards, Angelo H?ngens Systems Administrator ------------------------------------------ NetMatch tourism internet software solutions Ringbaan Oost 2b 5013 CA Tilburg T: +31 (0)13 5811088 F: +31 (0)13 5821239 mailto:A.Hongens at netmatch.nl http://www.netmatch.nl ------------------------------------------ From thebog at gmail.com Wed Dec 22 20:57:49 2010 From: thebog at gmail.com (thebog) Date: Wed, 22 Dec 2010 21:57:49 +0100 Subject: Fwd: cookie issue In-Reply-To: References: <4D12092A.6050108@altavoz.net> Message-ID: ---------- Forwarded message ---------- From: thebog Date: Wed, Dec 22, 2010 at 9:57 PM Subject: Re: cookie issue To: Claudio Castro Yes, to read the documentation :) http://www.varnish-cache.org/docs/2.1/tutorial/increasing_your_hitrate.html?highlight=cookie Ab On Wed, Dec 22, 2010 at 3:20 PM, Claudio Castro wrote: > hi, > > My site uses cookies to authenticate users, but I have problems when I use > varnish: from the backend side, all the clients come from the same cookie > (i.e. varnish), and not from several origins (customers), any ideas? > > Thanks in advance. > > -- > Claudio Castro N. > Ingeniero Jefe Area de Plataforma > AltaVoz S.A. > ?http://www.altavoz.net > Vi?a del Mar: > ?2 Poniente 355 of 53 > ?+56 32 276 8060 > Santiago: > ?Pedro de Valdivia 555 of 315 > ?+56 2 585 4264 > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > From imanandshah at gmail.com Thu Dec 23 05:26:21 2010 From: imanandshah at gmail.com (Anand Shah) Date: Thu, 23 Dec 2010 10:56:21 +0530 Subject: Too many 503 backend errors In-Reply-To: References: <648AB4F2-DD23-45D3-A62F-538D9AB7C986@gmail.com> Message-ID: Thanks robert, But i have already tried all this stuff's. The problem with probing is that if your back-end is unhealthy for a second and if varnish declares it unhealthy due to some reason than it shows backend unhealthy for next 5 mins or so whereas the interval is only 5s. So if i m running on one back-end all the requests for this backend fails. The one i m using is a very domain with 3000 req/s. so cannot afford a second of unhealthy network or unreachability. Regards, Anand On Wed, Dec 22, 2010 at 4:28 PM, Robert Shilston wrote: > Anand, > > I've simplified our VCL as much as practical to demonstrate how we use the > backends. > > > backend server2 { > > .host = "a.b.c.12"; > > .port = "80"; > > .probe = { > > .request = > > "GET /healthforvarnish HTTP/1.1" > > "Host: mysite.com" > > "User-agent: Varnish-probe" > > "Connection: close"; > > .interval = 5s; > > .timeout = 2500ms; > > .window = 10; > > .threshold = 8; > > } > > } > > backend server3 { > > .host = "a.b.c.13"; > > .port = "80"; > > .probe = { > > .request = > > "GET /healthforvarnish HTTP/1.1" > > "Host: mysite.com" > > "User-agent: Varnish-probe" > > "Connection: close"; > > .interval = 5s; > > .timeout = 2500ms; > > .window = 10; > > .threshold = 8; > > } > > } > > ... > > backend server2failsafe { .host = "a.b.c.12"; .port="80"; } > > backend server3failsafe { .host = "a.b.c.13"; .port="80"; } > > ... > > > > director workingpool random { > > { .backend = server2; .weight = 1; } > > { .backend = server3; .weight = 1; } > > ... > > } > > > > director failsafepool random { > > { .backend = server2failsafe; .weight = 1; } > > { .backend = server3failsafe; .weight = 1; } > > ... > > } > > > > ... > > > > sub vcl_recv { > > set req.backend = workingpool; > > > > if (!req.backend.healthy) { > > set req.backend = failsafepool; > > } > > > > > > If you implement something like this, then run varnishncsa, you'll see > which backends are being selected, and should deliver responses for 100% of > your requests. If it does, then you'll need to debug the health checks to > see why your backends might be being considered unhealthy. > > Rob > > > On 22 Dec 2010, at 10:52, Anand Shah wrote: > > > It works for 6 out of 10 requests. I have also found that varnish > > polls backend before sending a GET request. Since i never received any > > request on my apache server for this dropped requests i am in doubt as > > if what makes varnish feel the backend is in unhealthy state. > > > > Is there any way that i can troubleshoot? > > > > > > > > Anand > > > > On 12/22/10, Robert Shilston wrote: > >> Anand, > >> > >> Check that vcl_recv is actually setting a backend. It might be that > there's > >> a route through the VCL that doesn't cause a backend to be specified. > >> > >> Rob > >> > >> On 22 Dec 2010, at 08:39, Anand Shah wrote: > >> > >>> I immediately fired a test call using CURL which indicated me a > >>> tcp_miss and i also got the response from origin. > >>> > >>> So backend was working and in healthy state. > >>> > >>> > >>> > >>> [root at mymachine ~]# curl --head --header host:imads.mydomain.com > >>> " > http://localhost//0/OasDefault/IIPM_CPL_Nov10_B3_MW/iipm_winter02_300x600_10dec_03.gif > " > >>> HTTP/1.1 200 OK > >>> Last-Modified: Thu, 02 Dec 2010 12:19:14 GMT > >>> P3P: CP="NON NID PSAa PSDa OUR IND UNI COM NAV > >>> STA",policyref="/w3c/p3p.xml" > >>> Content-Type: image/gif > >>> cache-control: max-age=604800 > >>> Content-Length: 4150 > >>> Date: Wed, 22 Dec 2010 06:39:26 GMT > >>> Connection: keep-alive > >>> H-Served-By: mymachine > >>> Server: mydomain/2.0.6 > >>> X-Cache: TCP_MISS > >>> > >>> [root at mymachine ~]# curl --head --header host:imads.mydomain.com > >>> " > http://localhost//0/OasDefault/IIPM_CPL_Nov10_B3_MW/iipm_winter02_300x600_10dec_03.gif > " > >>> HTTP/1.1 200 OK > >>> Last-Modified: Thu, 02 Dec 2010 12:19:14 GMT > >>> P3P: CP="NON NID PSAa PSDa OUR IND UNI COM NAV > >>> STA",policyref="/w3c/p3p.xml" > >>> Content-Type: image/gif > >>> cache-control: max-age=604800 > >>> Content-Length: 4150 > >>> Date: Wed, 22 Dec 2010 06:39:33 GMT > >>> Connection: keep-alive > >>> H-Served-By: mymachine > >>> Server: mydomain/2.0.6 > >>> X-Cache: TCP_HIT > >>> > >>> > >>> > >>> > >>> Regards, > >>> Anand > >>> > >>> On Wed, Dec 22, 2010 at 12:42:24PM +0530, Anand Shah wrote: > >>>> I am running varnish on 64 bit Centos. Getting too many 503 errors > >>>> recently and i tried debugging it with some help from Forums on > >>>> mailinglist. > >>> > >>>> VARNISHLOGS: > >>>> > >>>> 180 VCL_call c miss fetch > >>>> 180 FetchError c no backend connection > >>>> 180 VCL_call c error deliver > >>> > >>> In my experience, "no backend connection" indicates that whatever > >>> backend was selected to handle this request was unavailable due to the > >>> health check/probe returning an error. > >>> > >>> What was the backend health at the time? You can use the 'debug.health' > >>> command in the CLI to view health information, or > >>> > >>> varnishlog -i Backend_health > >>> > >>> to see the results of the health checks as they occur. If a backend is > >>> marked as 'sick' then Varnish won't send any requests to it, since > >>> it's unlikely to help the situation. > >>> > >>> _______________________________________________ > >>> varnish-misc mailing list > >>> varnish-misc at varnish-cache.org > >>> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > >>> > >>> _______________________________________________ > >>> varnish-misc mailing list > >>> varnish-misc at varnish-cache.org > >>> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > >> > >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From varnish at mm.quex.org Thu Dec 23 05:37:36 2010 From: varnish at mm.quex.org (Michael Alger) Date: Thu, 23 Dec 2010 13:37:36 +0800 Subject: Too many 503 backend errors In-Reply-To: References: <648AB4F2-DD23-45D3-A62F-538D9AB7C986@gmail.com> Message-ID: <20101223053736.GA27224@grum.quex.org> On Thu, Dec 23, 2010 at 10:56:21AM +0530, Anand Shah wrote: > But i have already tried all this stuff's. The problem with probing is > that if your back-end is unhealthy for a second and if varnish > declares it unhealthy due to some reason than it shows backend > unhealthy for next 5 mins or so whereas the interval is only 5s. So if > i m running on one back-end all the requests for this backend fails. That depends on how you set the probe parameters. The .interval setting controls how frequently the requests are made. The .window setting controls how many responses (healthy/sick) are remembered. The .threshold setting controls how many of the last '.window' responses must be 'healthy' in order for the backend to be considered healthy. So if you set, for example, .window to 10 and .threshold to 1, then so long as at least 1 of the last 10 probes to that backend was successful, Varnish will consider the backend healthy. If your .interval is 5 seconds, then that means the probe must fail 10 times in a row for 50 seconds. That's unlikely to occur unless there's something seriously wrong with the backend. Simple settings, but they give you a lot of flexibility in deciding when to stop sending additional traffic to a backend that is having problems. This can be good even if you only have one backend, as if it's failing to respond to requests because it's overloaded, sending it more requests isn't going to help the situation, while backing off might allow it to recover quicker. From traian.bratucu at eea.europa.eu Thu Dec 23 08:05:56 2010 From: traian.bratucu at eea.europa.eu (Traian Bratucu) Date: Thu, 23 Dec 2010 09:05:56 +0100 Subject: Too many 503 backend errors In-Reply-To: References: <648AB4F2-DD23-45D3-A62F-538D9AB7C986@gmail.com> , Message-ID: Anand, You may want to look at grace mode, it could resolve your issue. When the backend is too busy you can serve stale content for a minute or something. Have a look: http://www.varnish-cache.org/docs/trunk/tutorial/handling_misbehaving_servers.html Traian ________________________________________ From: varnish-misc-bounces at varnish-cache.org [varnish-misc-bounces at varnish-cache.org] On Behalf Of Anand Shah [imanandshah at gmail.com] Sent: Thursday, December 23, 2010 6:26 AM To: Robert Shilston Cc: varnish-misc at varnish-cache.org Subject: Re: Too many 503 backend errors Thanks robert, But i have already tried all this stuff's. The problem with probing is that if your back-end is unhealthy for a second and if varnish declares it unhealthy due to some reason than it shows backend unhealthy for next 5 mins or so whereas the interval is only 5s. So if i m running on one back-end all the requests for this backend fails. The one i m using is a very domain with 3000 req/s. so cannot afford a second of unhealthy network or unreachability. Regards, Anand From rtshilston at gmail.com Thu Dec 23 09:32:37 2010 From: rtshilston at gmail.com (Robert Shilston) Date: Thu, 23 Dec 2010 09:32:37 +0000 Subject: Too many 503 backend errors In-Reply-To: References: <648AB4F2-DD23-45D3-A62F-538D9AB7C986@gmail.com> , Message-ID: Anand, My VCL extract coped with a backend failure - if all backends fail on their monitoring probe (which, for example, might monitor any database replication lag etc), then Varnish will fail over to sending requests to any backend, regardless of its perceived health. This allows for the possibility of service continuing in a worst case scenario, whilst still behaving well when there's just a partial degradation of backends. Rob On 23 Dec 2010, at 08:05, Traian Bratucu wrote: > Anand, > > You may want to look at grace mode, it could resolve your issue. When the backend is too busy you can serve stale content for a minute or something. > > Have a look: http://www.varnish-cache.org/docs/trunk/tutorial/handling_misbehaving_servers.html > > Traian > ________________________________________ > From: varnish-misc-bounces at varnish-cache.org [varnish-misc-bounces at varnish-cache.org] On Behalf Of Anand Shah [imanandshah at gmail.com] > Sent: Thursday, December 23, 2010 6:26 AM > To: Robert Shilston > Cc: varnish-misc at varnish-cache.org > Subject: Re: Too many 503 backend errors > > Thanks robert, > > But i have already tried all this stuff's. The problem with probing is that if your back-end is unhealthy for a second and if varnish declares it unhealthy due to some reason than it shows backend unhealthy for next 5 mins or so whereas the interval is only 5s. So if i m running on one back-end all the requests for this backend fails. > > The one i m using is a very domain with 3000 req/s. > > so cannot afford a second of unhealthy network or unreachability. > > Regards, > Anand > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc From imanandshah at gmail.com Thu Dec 23 10:24:24 2010 From: imanandshah at gmail.com (Anand Shah) Date: Thu, 23 Dec 2010 15:54:24 +0530 Subject: Too many 503 backend errors In-Reply-To: References: <648AB4F2-DD23-45D3-A62F-538D9AB7C986@gmail.com> Message-ID: Thanks rob, I tried saint mode to handle 503 requests, but m facing write errors now; is there any way i can efficiently. if (beresp.status == 503) { set beresp.saintmode = 10s; restart; } set beresp.grace = 5m; I have a couple of questions regarding this issue: 1. It seems to me that Varnish fails when it tries to write the request to the backend -- is this correct? 2. The integer value '11' reported in "backend write error: 11" -- is it the value of ``errno`` for the failed call to `write`? 3. If above is correct: In the platform where Varnish is running, 11 seems to correspond to EAGAIN: $ python -c 'import errno; print errno.EAGAIN' 11 $ Is there any way of telling Varnish to try the write(2) again? 834 FetchError c backend write error: 11 834 VCL_call c error deliver 834 Length c 488 834 VCL_call c deliver deliver 834 TxProtocol c HTTP/1.1 834 TxStatus c 503 834 TxResponse c Service Unavailable 834 TxHeader c Content-Type: text/html; charset=utf-8 834 TxHeader c Content-Length: 488 834 TxHeader c Date: Thu, 23 Dec 2010 09:59:50 GMT 834 TxHeader c Connection: close Regards, Anand On Thu, Dec 23, 2010 at 3:02 PM, Robert Shilston wrote: > Anand, > > My VCL extract coped with a backend failure - if all backends fail on their > monitoring probe (which, for example, might monitor any database replication > lag etc), then Varnish will fail over to sending requests to any backend, > regardless of its perceived health. This allows for the possibility of > service continuing in a worst case scenario, whilst still behaving well when > there's just a partial degradation of backends. > > Rob > > On 23 Dec 2010, at 08:05, Traian Bratucu wrote: > > > Anand, > > > > You may want to look at grace mode, it could resolve your issue. When the > backend is too busy you can serve stale content for a minute or something. > > > > Have a look: > http://www.varnish-cache.org/docs/trunk/tutorial/handling_misbehaving_servers.html > > > > Traian > > ________________________________________ > > From: varnish-misc-bounces at varnish-cache.org [ > varnish-misc-bounces at varnish-cache.org] On Behalf Of Anand Shah [ > imanandshah at gmail.com] > > Sent: Thursday, December 23, 2010 6:26 AM > > To: Robert Shilston > > Cc: varnish-misc at varnish-cache.org > > Subject: Re: Too many 503 backend errors > > > > Thanks robert, > > > > But i have already tried all this stuff's. The problem with probing is > that if your back-end is unhealthy for a second and if varnish declares it > unhealthy due to some reason than it shows backend unhealthy for next 5 mins > or so whereas the interval is only 5s. So if i m running on one back-end all > the requests for this backend fails. > > > > The one i m using is a very domain with 3000 req/s. > > > > so cannot afford a second of unhealthy network or unreachability. > > > > Regards, > > Anand > > > > > > _______________________________________________ > > varnish-misc mailing list > > varnish-misc at varnish-cache.org > > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From A.Hongens at netmatch.nl Thu Dec 23 14:27:48 2010 From: A.Hongens at netmatch.nl (=?iso-8859-1?Q?Angelo_H=F6ngens?=) Date: Thu, 23 Dec 2010 14:27:48 +0000 Subject: blocking not working In-Reply-To: <6A7ABA19243F1E4EADD8BB1563CDDCCB15F398@TIL-EXCH-05.netmatch.local> References: <6A7ABA19243F1E4EADD8BB1563CDDCCB15F398@TIL-EXCH-05.netmatch.local> Message-ID: <6A7ABA19243F1E4EADD8BB1563CDDCCB160F4E@TIL-EXCH-05.netmatch.local> Reply to my own post: I was being stupid. The vcl was correct, but my script that loads the vcl into varnish was not working correctly, and I did not read my script output as I should have :( -- With kind regards, Angelo H?ngens Systems Administrator ------------------------------------------ NetMatch tourism internet software solutions Ringbaan Oost 2b 5013 CA Tilburg T: +31 (0)13 5811088 F: +31 (0)13 5821239 mailto:A.Hongens at netmatch.nl http://www.netmatch.nl ------------------------------------------ > -----Original Message----- > From: varnish-misc-bounces at varnish-cache.org [mailto:varnish-misc- > bounces at varnish-cache.org] On Behalf Of Angelo H?ngens > Sent: woensdag 22 december 2010 13:23 > To: 'varnish-misc at varnish-cache.org' > Subject: blocking not working > > > I just added another Bad Guy to my balancer's block list, and I want > them to see a 403 access denied instead of content. It worked in the > past (at least back in the 2.0.x age), but now it does not seem to work > anymore. > > Perhaps some syntax changed in 2.1.x regarding this? > > I'm running 2.1.4 on CentOS 5.5 x64. > > Here's some of my VCL: > > > acl block { > "a.b.91.19"; /* 20100301 making dummy requests */ > "c.d.40.34"; /* 20100618 There are quite many invalid requests to our > RSS */ > "e.f.195.11"; /* 20101221 scraping */ > } > > sub vcl_recv { > if ( client.ip ~ block ) { > error 403 "Access denied"; > } > # Add a unique header containing the client address > remove req.http.X-Forwarded-For; > set req.http.X-Forwarded-For = client.ip; > .. > } > > > But when I look in my varnishncsa log, I still see successful requests > being passed: > > e.f.195.11 - - [22/Dec/2010:13:19:02 +0100] "GET > http://www.example.com/accommodation/accoinfo.aspx?accommodationId=1974 > 73 HTTP/1.1" 301 0 "-" "Mozilla/4.0 (compatible; Win32; > WinHttp.WinHttpRequest.5)" 0.007832050 miss > e.f.195.11 - - [22/Dec/2010:13:19:04 +0100] "GET > http://www.example.com/verenigde-staten-van-amerika/south- > carolina/charleston/charleston-marriott/hotel/informatie HTTP/1.1" 200 > 176281 "-" "Mozilla/4.0 (compatible; Win32; WinHttp.WinHttpRequest.5)" > 0.665360928 miss > e.f.195.11 - - [22/Dec/2010:13:19:04 +0100] "GET > http://www.example.com/accommodation/accoinfo.aspx?accommodationId=1974 > 74 HTTP/1.1" 301 0 "-" "Mozilla/4.0 (compatible; Win32; > WinHttp.WinHttpRequest.5)" 0.006700993 miss > e.f.195.11 - - [22/Dec/2010:13:19:05 +0100] "GET > http://www.example.com/verenigde-staten-van-amerika/south- > carolina/charleston/church-street-inn/appartement/informatie HTTP/1.1" > 200 163794 "-" "Mozilla/4.0 (compatible; Win32; > WinHttp.WinHttpRequest.5)" 0.241801977 miss > e.f.195.11 - - [22/Dec/2010:13:19:05 +0100] "GET > http://www.example.com/accommodation/accoinfo.aspx?accommodationId=1974 > 75 HTTP/1.1" 301 0 "-" "Mozilla/4.0 (compatible; Win32; > WinHttp.WinHttpRequest.5)" 0.006364822 miss > e.f.195.11 - - [22/Dec/2010:13:19:06 +0100] "GET > http://www.example.com/verenigde-staten-van-amerika/south- > carolina/north-charleston/comfort-inn-coliseum/hotel/informatie > HTTP/1.1" 200 171431 "-" "Mozilla/4.0 (compatible; Win32; > WinHttp.WinHttpRequest.5)" 0.625963926 miss > > -- > > > With kind regards, > > > Angelo H?ngens > > Systems Administrator > > ------------------------------------------ > NetMatch > tourism internet software solutions > > Ringbaan Oost 2b > 5013 CA Tilburg > T: +31 (0)13 5811088 > F: +31 (0)13 5821239 > > mailto:A.Hongens at netmatch.nl > http://www.netmatch.nl > ------------------------------------------ > > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc From steamboatid at gmail.com Fri Dec 24 04:09:19 2010 From: steamboatid at gmail.com (dwi kristianto) Date: Fri, 24 Dec 2010 11:09:19 +0700 Subject: limit concurrent connection Message-ID: hello all, i'm new in here and also, new varnish user. i've tried to seek around the archieve but can not find what i'm looking for. is there any search form that i can use? btw, i just installed newest varnish for my production box, is there any way in the vcl configs to limit concurrent connections at one time? maybe just like apache server_limit. regards, dwi From indranilc at rediff-inc.com Fri Dec 24 04:53:04 2010 From: indranilc at rediff-inc.com (Indranil Chakravorty) Date: 24 Dec 2010 04:53:04 -0000 Subject: =?utf-8?B?VXNlIHNlc3MgKnNwIGZyb20gdmNs?= Message-ID: <20101224045304.51213.qmail@pro236-131.mxout.rediffmailpro.com> Hi,   Is is possible to use the sp available from vcl and which is used as a parameter for the compiled vcl C functions? I am especially interested in using sp->t_req the time of receiving the request for the session pointer. Please let me know. Thanks. Thanks, Neel -------------- next part -------------- An HTML attachment was scrubbed... URL: From cosimo at streppone.it Fri Dec 24 08:10:23 2010 From: cosimo at streppone.it (Cosimo Streppone) Date: Fri, 24 Dec 2010 09:10:23 +0100 Subject: Use sess *sp from vcl In-Reply-To: <20101224045304.51213.qmail@pro236-131.mxout.rediffmailpro.com> References: <20101224045304.51213.qmail@pro236-131.mxout.rediffmailpro.com> Message-ID: In data 24 dicembre 2010 alle ore 05:53:04, Indranil Chakravorty ha scritto: > Hi, >   Is is possible to use the sp available from vcl and which is used > as a parameter for the compiled vcl C functions? It's definitely possible to use "sp" in your C functions: https://github.com/cosimo/varnish-accept-language/blob/master/examples/sample.vcl#L11 the vcl_rewrite_accept_language() function is defined as: https://github.com/cosimo/varnish-accept-language/blob/master/examples/accept-language.vcl#L156 -- Cosimo From steamboatid at gmail.com Sat Dec 25 09:10:00 2010 From: steamboatid at gmail.com (dwi kristianto) Date: Sat, 25 Dec 2010 16:10:00 +0700 Subject: missing cache-control http respoonse Message-ID: helllo.. i've double check the backend server and it return "Cache-Control" header accordingly. but somehow, varnishd trim it. what could be wrong? many thanks in advance. regards, dwi From slink at schokola.de Tue Dec 28 11:55:21 2010 From: slink at schokola.de (Nils Goroll) Date: Tue, 28 Dec 2010 12:55:21 +0100 Subject: missing cache-control http respoonse In-Reply-To: References: Message-ID: <4D19D029.3030000@schokola.de> > i've double check the backend server and it return "Cache-Control" > header accordingly. but somehow, varnishd trim it. > what could be wrong? If you have a hit-for-pass case, this should fix the issue: http://www.varnish-cache.org/trac/changeset/5625 From slink at schokola.de Tue Dec 28 11:56:44 2010 From: slink at schokola.de (Nils Goroll) Date: Tue, 28 Dec 2010 12:56:44 +0100 Subject: Varnish stripping Cache-Control header In-Reply-To: <4CFF8715.4030509@renoz.dk> References: <4CFF8715.4030509@renoz.dk> Message-ID: <4D19D07C.9070509@schokola.de> Hi Lars and All, > I have a problem with Varnish 2.1.4 stripping the Cache-Control header, I've seen exactly the same behavior with 2.1.4 any applying this patch has fixed the issue: http://www.varnish-cache.org/trac/changeset/5625 Nils From gmoniey at gmail.com Tue Dec 28 23:27:08 2010 From: gmoniey at gmail.com (.) Date: Tue, 28 Dec 2010 15:27:08 -0800 Subject: Trouble understanding Varnishlog Message-ID: I setup a simple rule in the my VCL to cache requests to a url, however, according to the X-Varnish header, the requests are not being cached. So I fired up varnishlog, and it confused me to see that it said there was a 'hit'. Can someone explain why the object is not being returned as a cached object? I'm confused around the fact that it says there is a 'hit', but the X-Varnish header only has 1 field. I read that if the Set-Cookie header is sent, then varnish does not cache the object, but I'm pretty sure thats not being sent. Is there another rule I'm missing? Thanks. Here is an excerpt of the log: 15 SessionOpen c 10.0.19.23 59326 :80 15 ReqStart c 10.0.19.23 59326 1001168302 15 RxRequest c GET 15 RxURL c /items?xzy=true 15 RxProtocol c HTTP/1.1 15 RxHeader c Host: XYZ 15 RxHeader c User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 GTB7.1 15 RxHeader c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 15 RxHeader c Accept-Language: en-us,en;q=0.5 15 RxHeader c Accept-Encoding: gzip,deflate 15 RxHeader c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 15 RxHeader c Keep-Alive: 115 15 RxHeader c Connection: keep-alive 15 RxHeader c Cookie: __qca=P0-707762352-1283409801091; __utma=182775871.333345873.1284063320.1293103561.1293171223.24; __utmz=182775871.1284063320.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); km_lv=x; km_ai=T7R0VpZshcfDBJiF4n8Edt1vrrI; km_uq=; __utmv=182775871.p 15 RxHeader c If-None-Match: "4ee85970d12afd8992d3dc1651f07b9d" 15 VCL_call c recv 15 VCL_return c lookup 15 VCL_call c hash 15 VCL_return c hash 15 Hit c 1001168004 15 VCL_call c hit 15 VCL_return c deliver 15 VCL_call c deliver 15 VCL_return c deliver 15 TxProtocol c HTTP/1.1 15 TxStatus c 304 15 TxResponse c Not Modified 15 TxHeader c Date: Tue, 28 Dec 2010 23:19:53 GMT 15 TxHeader c Via: 1.1 varnish 15 TxHeader c X-Varnish: 1001168302 15 TxHeader c Cache-Control: private, max-age=0, must-revalidate 15 TxHeader c ETag: "4ee85970d12afd8992d3dc1651f07b9d" 15 TxHeader c Connection: keep-alive 15 Length c 0 15 ReqEnd c 1001168302 1293578393.040635109 1293578393.040782928 0.000112057 0.000097990 0.000049829 15 Debug c "herding" -------------- next part -------------- An HTML attachment was scrubbed... URL: From steamboatid at gmail.com Wed Dec 29 05:26:39 2010 From: steamboatid at gmail.com (dwi kristianto) Date: Wed, 29 Dec 2010 12:26:39 +0700 Subject: modify Retry-after http response Message-ID: hi all, i came from http://www.varnish-cache.org/trac/wiki/VCL#vcl_error and dont see any documentation about it. for now, i'm suffering from some bots that abusing my web server. using varnish and iptables it may cut down the abuse. despite of that, for other good bots and SEO purposes, it's good to return "Retry-After" header when error happened. how can i do that? should i use vcl_error sub? regards, dwi. From bedis9 at gmail.com Wed Dec 29 09:59:01 2010 From: bedis9 at gmail.com (Bedis 9) Date: Wed, 29 Dec 2010 10:59:01 +0100 Subject: Refreshing modified content In-Reply-To: References: Message-ID: On Fri, Dec 17, 2010 at 4:50 PM, Jonathan Leibiusky wrote: > I think that conditional GETs could be really useful here. I just don't know > how varnish works in this scenario. Maybe someone from the varnish > development team can shed some light here :) > > On Fri, Dec 17, 2010 at 12:42 PM, James A. Robinson > wrote: >> >> On Thu, Dec 16, 2010 at 12:31, Paulo Paracatu >> wrote: >> > If I understood it, the purging method isn't automatic, right? I'd need >> > to >> > purge the content everytime it is modified. >> > This is kinda stupid... I host more than 10k sites, modifying files >> > everytime. If I set a high TTL, the backend will be happy and the >> > webmaster >> > will be angry. If I set a low TTL, the webmaster will be happy, but the >> > backend will die. Plus, there is no point using a cache if the TTL is >> > low. >> >> In a later post you ask about whether or not varnish could be >> configured to send a conditional GET on every request >> >> Basically varnish would be looking up an item in its own cache, seeing >> if it had a Last-Modified or ETag header from the backend, and sending >> a conditional GET -- if it got an entity back it'd store that entity >> as the new version, otherwise serve the old. ?I'd be curious if >> anyone's put together VCL logic that is capable of that. ?It'd be good >> to know how to do it. >> >> If it's possible to due, this technique might work well when you are >> fronting a backend that is very fast at computing conditional GETs, >> e.g., static files that can be examined to see if its inode, size, or >> last modified time has changed. ?I imagine most people using varnish >> have slower backend servers, ones that build dynamic content and >> aren't able to respond to conditional GETs any more efficiently than >> they could respond to an unqualified GET. >> >> One of the places we use varnish at my dept is fronting a large (half >> terabyte of about six million files) static file server. ?Instead of >> using some form of conditional GET, what we use are cache channels. >> >> ?http://tools.ietf.org/id/draft-nottingham-http-cache-channels-01.txt >> >> When various programs are updating the backend server static files >> they POST the filepath to our cache channel server. ?We have another >> program running that monitors the cache channel once a minute for >> updates, and when it sees a new entry show up it turns around and >> sends a PURGE request to varnish. >> >> Jim >> >> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - >> James A. Robinson ? ? ? ? ? ? ? ? ? ? ? jim.robinson at stanford.edu >> Stanford University HighWire Press ? ? ?http://highwire.stanford.edu/ >> +1 650 7237294 (Work) ? ? ? ? ? ? ? ? ? +1 650 7259335 (Fax) >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > Hi, Maybe you can have a long TTL in your Cache-Control headers and force a lower one into your VCL for Varnish itself. my 2 cents :) cheers From nicholas.tang at livestream.com Wed Dec 29 22:59:15 2010 From: nicholas.tang at livestream.com (Nicholas Tang) Date: Wed, 29 Dec 2010 17:59:15 -0500 Subject: Load balancing streaming (rtsp) servers Message-ID: Question: is it possible to load balance rtsp servers using Varnish? They'd need to "stick" based on client ip. My thought was to try something like this: backend mobile-1 { .host = ""; include "/usr/local/varnish/etc/varnish/backend.vcl"; } backend mobile-2 { .host = ""; include "/usr/local/varnish/etc/varnish/backend.vcl"; } backend mobile-3 { .host = ""; include "/usr/local/varnish/etc/varnish/backend.vcl"; } backend mobile-4 { .host = ""; include "/usr/local/varnish/etc/varnish/backend.vcl"; } director mobile_rtsp client { { .backend = mobile-1; } { .backend = mobile-2; } { .backend = mobile-3; } { .backend = mobile-4; } } sub vcl_recv { set req.backend = mobile_rtsp; set client.identity = client.ip; return (pipe); } sub vcl_pipe { # close backend connection after each pipe - this prevents requests from stepping on each other # http://www.varnish-cache.org/trac/wiki/VCLExamplePipe set bereq.http.connection = "close"; } Thanks, Nicholas *Nicholas Tang**:* VP, Dev Ops nicholas.tang at livestream.com | t: +1 (646) 495 9707 | m: +1 (347) 410 6066 | 111 8th Avenue, Floor 15, New York, NY 10011 [image: www.livestream.com] -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto.fernandezcrisial at gmail.com Thu Dec 30 15:47:36 2010 From: roberto.fernandezcrisial at gmail.com (=?ISO-8859-1?Q?Roberto_O=2E_Fern=E1ndez_Crisial?=) Date: Thu, 30 Dec 2010 12:47:36 -0300 Subject: Memory usage Message-ID: Hello! I am trying to find out how many memory is being used and how many memory is free from varnish cache. I have two servers running varnish (varnish-2.1.3 SVN), and both started with "-s malloc,28G" option. I've tried with varnishstat, looking for sma's values (I think "sma_balloc + sma_bfree = 28G"), but only in one server shows "correct" information: sma_balloc 1641487715428 . SMA bytes allocated sma_bfree 1611596371850 . SMA bytes free The other server shows: sma_balloc 239065444117 . SMA bytes allocated sma_bfree 226776048576 . SMA bytes free What should I do to see the memory used and memory free from varnish cache? Thank you. Regards, Roberto. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jhalfmoon at milksnot.com Thu Dec 30 16:37:42 2010 From: jhalfmoon at milksnot.com (Johnny Halfmoon) Date: Thu, 30 Dec 2010 17:37:42 +0100 Subject: Use sess *sp from vcl Message-ID: <20101230173742.5sa0r7byso48k8cs@webmail.milksnot.com> >> Hi, >> Is is possible to use the sp available from vcl and which is used as a >> parameter for the compiled vcl C functions? I am especially interested in >> using sp->t_req the time of receiving the request for the session pointer. Hi Neel, doing stuff like sp->t_req is not going to work and there is currently no VRT function that gives you access to that parameter. The way I worked around this is to use VRT_r_now(sp). The example below is an example of how I solved this issue. Cheers, Johnny //--- Example code follows just about now... ---// C{ double g_ReqStartTime, g_FetchStartTime, g_FetchEndTime; }C sub vcl_recv { C{ g_ReqStartTime = VRT_r_now(sp); g_FetchStartTime = 0; }C } sub vcl_miss { C{ g_FetchStartTime = VRT_r_now(sp); }C } sub vcl_pass { C{ g_FetchStartTime = VRT_r_now(sp); }C } sub vcl_fetch { C{ g_FetchEndTime = VRT_r_now(sp); }C } sub vcl_deliver { C{ double ProcessTimeDelta, BackendTimeDelta, TotalTimeDelta; double CurrentTime = VRT_r_now(sp); TotalTimeDelta = (CurrentTime - g_ReqStartTime); if (g_FetchStartTime == 0) { ProcessTimeDelta = TotalTimeDelta; BackendTimeDelta = 0; } else { ProcessTimeDelta = (g_FetchStartTime - g_ReqStartTime) + (CurrentTime - g_FetchEndTime); BackendTimeDelta = (g_FetchEndTime - g_FetchStartTime); } syslog(LOG_INFO, "Example: t0:%.6f t1:%.6f t2:%.6f", TotalTimeDelta, BackendTimeDelta, ProcessTimeDelta); }C } //--- EOF --// From ratz at hotmail.com Wed Dec 22 08:37:48 2010 From: ratz at hotmail.com (Aaron .) Date: Wed, 22 Dec 2010 16:37:48 +0800 Subject: Ubuntu installation guide page installs outdated Varnish cache Message-ID: Hello, tried contacting varnish contact person, but couldn't find any ways to do so - thus the mailing list. Sorry if I'm causing others any inconvenience ;-) Just a heads up for those using the Ubuntu installation guide: it installs an outdated varnish: root at xen7:/etc/apache2# varnishd -Vvarnishd (varnish-1.0.3)Copyright (c) 2006 Linpro AS / Verdens Gang ASroot at xen7:/etc/apache2# Here's what I did (rather than re-compliing from source to get 2.1.4 (am getting 2.1.2 on Ubuntu Hardy) 1. >From http://repo.varnish-cache.org/debian/dists/hardy/varnish-2.1/binary-amd64/: wget http://repo.varnish-cache.org/debian/dists/hardy/varnish-2.1/binary-amd64/varnish_2.1.2-1~hardy1_amd64.deb 2.To resolve dependency issues: apt-get install libvarnish1 3. dpkg -i varnish_2.1.2-1~hardy1_amd64.deb After installation: root at xen7:/home/bsg/download# varnishd -Vvarnishd (varnish-2.1.2 SVN )Copyright (c) 2006-2009 Linpro AS / Verdens Gang ASroot at xen7:/home/bsg/download# Hope this helps :-) Keep up the great work guys! Cheers, Aaron -------------- next part -------------- An HTML attachment was scrubbed... URL: