From rafailow at gmail.com Sun May 2 22:03:51 2010 From: rafailow at gmail.com (rafailowski) Date: Mon, 3 May 2010 00:03:51 +0200 Subject: Graphing varnishtop Message-ID: Hi list, I would like to graph TxStatus ans RxStatus (typically with munin/cacti). varnishtop -1 -i TxStatus works well but the varnishtop values are frequently re-initialised (related to the shm logsize?). Here's a test with varnish 2.1.1 while true; do varnishtop -1 -i TxStatus; sleep 300; done 50685.00 TxStatus 200 2495.00 TxStatus 200 62179.00 TxStatus 200 26260.00 TxStatus 200 So munin graphs wrongs results, Have you find a solution to graphs with varnishtop? Thx in advance, Rafa. From sime at sime.net.au Mon May 3 05:13:33 2010 From: sime at sime.net.au (Simon Males) Date: Mon, 3 May 2010 15:13:33 +1000 Subject: Varnish fails on boot with Amazon S3 backend Message-ID: Hello, I'm using Varnish 2.0.6 on Ubuntu 9.04 and found that Varnish cannot be started when the backend is unreachable (obviously). I believe Varnish is being started before some degree of networking is available (DNS?). It particularly struggles on Amazon S3, as its the only backend which doesn't have an IP. If I attempt to start it again manually it works fine. Below I have output the error file, and the VCL with the Amazon backend. # cat /tmp/*.varnish storage_malloc: max size 2048 MB. Message from VCC-compiler: Backend host '"site.s3.amazonaws.com"': Name or service not known (/etc/varnish/backends.vcl Line 65 Pos 17) .host = "site.s3.amazonaws.com"; ----------------#######################- In backend specification starting at: (/etc/varnish/backends.vcl Line 64 Pos 1) backend s3 { #######------ Running VCC-compiler failed, exit 1 VCL compilation failed backend s3 { .host = "site.s3.amazonaws.com"; .port = "80"; .connect_timeout = 5s; .first_byte_timeout = 20s; .between_bytes_timeout = 15s; .probe = { .url = "/status.txt"; .timeout = 2s; .window = 6; .threshold = 3; .interval = 10s; } } -- Simon Males From phk at phk.freebsd.dk Mon May 3 06:44:10 2010 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Mon, 03 May 2010 06:44:10 +0000 Subject: Varnish fails on boot with Amazon S3 backend In-Reply-To: Your message of "Mon, 03 May 2010 15:13:33 +1000." Message-ID: <74195.1272869050@critter.freebsd.dk> In message , Si mon Males writes: >Below I have output the error file, and the VCL with the Amazon backend. > ># cat /tmp/*.varnish >storage_malloc: max size 2048 MB. >Message from VCC-compiler: >Backend host '"site.s3.amazonaws.com"': Name or service not known >(/etc/varnish/backends.vcl Line 65 Pos 17) > .host = "site.s3.amazonaws.com"; >----------------#######################- That name returns a CNAME dns record here, you need to use something that returns a A record. Poul-Henning -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From bendj095124367913213465 at gmail.com Wed May 5 01:13:41 2010 From: bendj095124367913213465 at gmail.com (Ben DJ) Date: Tue, 4 May 2010 18:13:41 -0700 Subject: Need some help understanding these headers -- *is* my file being cached, or not? Message-ID: I've got varnish setup to cache JS inbetween an nginx front-end (gzip compression only) & an apache2 backed serving content. On loading my page, here's the LiveHTTPHeader result for one particular file: https://my.domain.com/srv/js/minified.1273021105.js GET /srv/js/minified.1273021105.js HTTP/1.1 Host: my.domain.com User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.4) Gecko/20100417 Accept: */* Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 115 Connection: keep-alive Referer: https://my.domain.com/ > Pragma: no-cache > Cache-Control: no-cache HTTP/1.1 200 OK Server: nginx/0.8.35 Date: Wed, 05 May 2010 01:02:36 GMT Content-Type: text/x-js Transfer-Encoding: chunked Connection: keep-alive Vary: Accept-Encoding Last-Modified: Wed, 05 May 2010 00:58:30 GMT Etag: "19f3b-c911-485ce530d9980" > X-Cacheable: NO: !obj.cacheable X-Varnish: 199766821 Age: 0 Via: 1.1 varnish X-Cache: MISS Expires: Sat, 08 May 2010 01:02:36 GMT Cache-Control: max-age=259200 Content-Encoding: gzip The three lines I've marked, > Pragma: no-cache > Cache-Control: no-cache > X-Cacheable: NO: !obj.cacheable Have me confused ... IIUC, and not sure I do, they seem to indicate the file's NOT being cached. I'm happy to provide my vcl.conf if helpful/necessary. b4 I go tromping though it again, can someone clarify what is, in fact, happening to that file, according to the headers above? I'm clearly not 'fluent' enough in header-speak, yet ... Thanks, Ben From bedis9 at gmail.com Wed May 5 03:45:39 2010 From: bedis9 at gmail.com (Bedis 9) Date: Wed, 5 May 2010 05:45:39 +0200 Subject: Need some help understanding these headers -- *is* my file being cached, or not? In-Reply-To: References: Message-ID: Hi, Your browser sends the "Cache-Control: no-cache", by default Varnish will follow it and bypass the cache. You can force caching that by using: # Ignore Cache-Control header from the request if (req.http.Cache-Control ~ "no-cache") { unset req.http.Cache-Control; } if (req.http.Pragma ~ "no-cache") { unset req.http.Pragma; } in vcl_recv rgs On Wed, May 5, 2010 at 3:13 AM, Ben DJ wrote: > I've got varnish setup to cache JS inbetween an nginx front-end (gzip > compression only) & an apache2 backed serving content. > > On loading my page, here's the LiveHTTPHeader result for one particular file: > > ? ? ? ? ? ? ? ?https://my.domain.com/srv/js/minified.1273021105.js > > > > ? ? ? ? ? ? ? ?GET /srv/js/minified.1273021105.js HTTP/1.1 > > ? ? ? ? ? ? ? ?Host: my.domain.com > > ? ? ? ? ? ? ? ?User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.4) > Gecko/20100417 > > ? ? ? ? ? ? ? ?Accept: */* > > ? ? ? ? ? ? ? ?Accept-Language: en-us,en;q=0.5 > > ? ? ? ? ? ? ? ?Accept-Encoding: gzip,deflate > > ? ? ? ? ? ? ? ?Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 > > ? ? ? ? ? ? ? ?Keep-Alive: 115 > > ? ? ? ? ? ? ? ?Connection: keep-alive > > ? ? ? ? ? ? ? ?Referer: https://my.domain.com/ > >> ? ? ? ? ? ? ? Pragma: no-cache > >> ? ? ? ? ? ? ? Cache-Control: no-cache > > > > ? ? ? ? ? ? ? ?HTTP/1.1 200 OK > > ? ? ? ? ? ? ? ?Server: nginx/0.8.35 > > ? ? ? ? ? ? ? ?Date: Wed, 05 May 2010 01:02:36 GMT > > ? ? ? ? ? ? ? ?Content-Type: text/x-js > > ? ? ? ? ? ? ? ?Transfer-Encoding: chunked > > ? ? ? ? ? ? ? ?Connection: keep-alive > > ? ? ? ? ? ? ? ?Vary: Accept-Encoding > > ? ? ? ? ? ? ? ?Last-Modified: Wed, 05 May 2010 00:58:30 GMT > > ? ? ? ? ? ? ? ?Etag: "19f3b-c911-485ce530d9980" > >> ? ? ? ? ? ? ? X-Cacheable: NO: !obj.cacheable > > ? ? ? ? ? ? ? ?X-Varnish: 199766821 > > ? ? ? ? ? ? ? ?Age: 0 > > ? ? ? ? ? ? ? ?Via: 1.1 varnish > > ? ? ? ? ? ? ? ?X-Cache: MISS > > ? ? ? ? ? ? ? ?Expires: Sat, 08 May 2010 01:02:36 GMT > > ? ? ? ? ? ? ? ?Cache-Control: max-age=259200 > > ? ? ? ? ? ? ? ?Content-Encoding: gzip > > > The three lines I've marked, > > ? ? ? ?> ? ? ? ? ? ? ? Pragma: no-cache > > ? ? ? ?> ? ? ? ? ? ? ? Cache-Control: no-cache > > ? ? ? ?> ? ? ? ? ? ? ? X-Cacheable: NO: !obj.cacheable > > > > Have me confused ... IIUC, and not sure I do, they seem to indicate > the file's NOT being cached. > > I'm happy to provide my vcl.conf if helpful/necessary. b4 I go > tromping though it again, can someone clarify what is, in fact, > happening to that file, according to the headers above? ?I'm clearly > not 'fluent' enough in header-speak, yet ... > > > Thanks, > > Ben > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://lists.varnish-cache.org/mailman/listinfo/varnish-misc > From tfheen at varnish-software.com Wed May 5 07:15:22 2010 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Wed, 05 May 2010 09:15:22 +0200 Subject: Need some help understanding these headers -- *is* my file being cached, or not? In-Reply-To: (Ben DJ's message of "Tue, 4 May 2010 18:13:41 -0700") References: Message-ID: <87aasebz05.fsf@qurzaw.linpro.no> ]] Ben DJ | I've got varnish setup to cache JS inbetween an nginx front-end (gzip | compression only) & an apache2 backed serving content. | | On loading my page, here's the LiveHTTPHeader result for one particular file: [...] | > Pragma: no-cache | > Cache-Control: no-cache These are client headers, we don't care about them. | Have me confused ... IIUC, and not sure I do, they seem to indicate | the file's NOT being cached. Indeed, it looks like that way. | I'm happy to provide my vcl.conf if helpful/necessary. b4 I go | tromping though it again, can someone clarify what is, in fact, | happening to that file, according to the headers above? I'm clearly | not 'fluent' enough in header-speak, yet ... A copy of the VCL + what varnishlog -o output when you run the request would be useful. -- Tollef Fog Heen Varnish Software t: +47 21 54 41 73 From andreas.haase at evolver.de Thu May 6 09:12:27 2010 From: andreas.haase at evolver.de (Andreas Haase - evolver group) Date: Thu, 6 May 2010 11:12:27 +0200 (CEST) Subject: Sticky clients and varnish Message-ID: Hello, I'd like to use a director with sticky client configured. But all documentation I found only mentions random and round-robin as possible options. Is there any recent docu for this feature? -- Mit freundlichen Gruessen Andreas Haase Administration und Technik evolver services GmbH Fon +49 / (0)3 71 / 4 00 03 727 Fax +49 / (0)3 71 / 4 00 03 79 E-Mail andreas.haase at evolver.de Web http://www.evolver.de Sitz der Gesellschaft: Chemnitz Handelsregister: Amtsgericht Chemnitz, HRB 22649 Geschaeftsfuehrer: Dirk Neubauer From phk at phk.freebsd.dk Thu May 6 09:35:23 2010 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Thu, 06 May 2010 09:35:23 +0000 Subject: Sticky clients and varnish In-Reply-To: Your message of "Thu, 06 May 2010 11:12:27 +0200." Message-ID: <82048.1273138523@critter.freebsd.dk> In message , Andreas Haase - evolver group writes: >Hello, > >I'd like to use a director with sticky client configured. But all >documentation I found only mentions random and round-robin as possible >options. Is there any recent docu for this feature? We're working on the docs, but if you make a "client" director (syntax otherwise like the "random" director) it will distribute based on the client's IP#. This does not (yet) take X-Forwarded-For headers into account. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From martin.boer at netclever.nl Thu May 6 09:55:04 2010 From: martin.boer at netclever.nl (Martin Boer) Date: Thu, 06 May 2010 11:55:04 +0200 Subject: Sticky clients and varnish In-Reply-To: <82048.1273138523@critter.freebsd.dk> References: <82048.1273138523@critter.freebsd.dk> Message-ID: <4BE291F8.9000304@netclever.nl> Poul-Henning Kamp wrote: > In message , > Andreas Haase - evolver group writes: > >> Hello, >> >> I'd like to use a director with sticky client configured. But all >> documentation I found only mentions random and round-robin as possible >> options. Is there any recent docu for this feature? >> > > We're working on the docs, but if you make a "client" director > (syntax otherwise like the "random" director) it will distribute > based on the client's IP#. > > This does not (yet) take X-Forwarded-For headers into account. > > Just out of curiosity, wouldn't that give very weird results when several people use the same outbound IP-address ? I would assume the purpose of stickyness is to give returning browsers a headstart in something. But I could be wrong on both accounts. :) From phk at phk.freebsd.dk Thu May 6 10:07:13 2010 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Thu, 06 May 2010 10:07:13 +0000 Subject: Sticky clients and varnish In-Reply-To: Your message of "Thu, 06 May 2010 11:55:04 +0200." <4BE291F8.9000304@netclever.nl> Message-ID: <85391.1273140433@critter.freebsd.dk> In message <4BE291F8.9000304 at netclever.nl>, Martin Boer writes: >Poul-Henning Kamp wrote: >Just out of curiosity, wouldn't that give very weird results when >several people use the same outbound IP-address ? No, they just all end up on the same backend. Depending on your site, this may indeed be a good idea, since it optimizes for the "Hey, try this URL...." scenario in workplaces. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From martin.boer at netclever.nl Thu May 6 10:15:18 2010 From: martin.boer at netclever.nl (Martin Boer) Date: Thu, 06 May 2010 12:15:18 +0200 Subject: Sticky clients and varnish In-Reply-To: <85391.1273140433@critter.freebsd.dk> References: <85391.1273140433@critter.freebsd.dk> Message-ID: <4BE296B6.9020804@netclever.nl> Poul-Henning Kamp wrote: > In message <4BE291F8.9000304 at netclever.nl>, Martin Boer writes: > >> Poul-Henning Kamp wrote: >> > > >> Just out of curiosity, wouldn't that give very weird results when >> several people use the same outbound IP-address ? >> > > No, they just all end up on the same backend. > > Depending on your site, this may indeed be a good idea, since it > optimizes for the "Hey, try this URL...." scenario in workplaces. > That's true, but trying to do fancy stuff with user sessions based on IP addresses would give very strange results. From phk at phk.freebsd.dk Thu May 6 10:30:15 2010 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Thu, 06 May 2010 10:30:15 +0000 Subject: Sticky clients and varnish In-Reply-To: Your message of "Thu, 06 May 2010 12:15:18 +0200." <4BE296B6.9020804@netclever.nl> Message-ID: <85500.1273141815@critter.freebsd.dk> In message <4BE296B6.9020804 at netclever.nl>, Martin Boer writes: >That's true, but trying to do fancy stuff with user sessions based on IP >addresses would give very strange results. What "fancy stuff" are you thinking about ? All we do is pick a backend... -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From fehwalker at gmail.com Thu May 6 20:01:57 2010 From: fehwalker at gmail.com (Bryan Fullerton) Date: Thu, 6 May 2010 16:01:57 -0400 Subject: Sticky Load Balancing with Varnish In-Reply-To: <1987.1271353395@critter.freebsd.dk> References: <1271329847.6165.37.camel@app-srv-debian-amdmp2.idni> <1987.1271353395@critter.freebsd.dk> Message-ID: On Thu, Apr 15, 2010 at 1:43 PM, Poul-Henning Kamp wrote: > In message <1271329847.6165.37.camel at app-srv-debian-amdmp2.idni>, Niklas Norber > g writes: > >>Ok, but it would be nice if this load-balancing supports backends that >>uses session-cookies. And that the "control" can be choosen to look at a >>LB-cookie. > > You mean: > > ? ? ? ?sub vcl_recv { > ? ? ? ? ? ? ? ?set client.id = req.cookie[SESSION_BLAH]; > ? ? ? ?} > > Yes, that's the idea :-) This doesn't appear to be in 2.1.1 or 2.1.2 (at least I don't see it in the changelog). Will it be coming soon? We really need this capability so we can send established sessions to the same backend based on session cookie. Thanks, Bryan From phk at phk.freebsd.dk Fri May 7 08:27:59 2010 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Fri, 07 May 2010 08:27:59 +0000 Subject: Sticky Load Balancing with Varnish In-Reply-To: Your message of "Thu, 06 May 2010 16:01:57 -0400." Message-ID: <19516.1273220879@critter.freebsd.dk> In message , Brya n Fullerton writes: >This doesn't appear to be in 2.1.1 or 2.1.2 (at least I don't see it >in the changelog). Will it be coming soon? We really need this >capability so we can send established sessions to the same backend >based on session cookie. It's on my list, but it is stuck behind som renovations to the VCL compiler that are not quite there yet. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From dtownrobbrown at gmail.com Tue May 11 17:54:08 2010 From: dtownrobbrown at gmail.com (Rob Brown) Date: Tue, 11 May 2010 10:54:08 -0700 Subject: manipulating page content Message-ID: Hi, I just signed up for the list because I am considering using Varnish for an upcoming project. I'm not very familiar with it's capabilities yet, and haven't been able to quickly find an answer to a specific question I have: Is it possible for Varnish to "manipulate" the pages that it serves out of cache "on the way out" (post-cache)? More specifically: I have an app that generates pages with a session key in them that I need to strip out to make cacheable, but then put back in based on a session key parameter sent in the query string. If this is something Varnish can do, i'd like to know before getting to far down the road. Comments welcome. Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From phk at phk.freebsd.dk Tue May 11 18:06:13 2010 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Tue, 11 May 2010 18:06:13 +0000 Subject: manipulating page content In-Reply-To: Your message of "Tue, 11 May 2010 10:54:08 MST." Message-ID: <43533.1273601173@critter.freebsd.dk> In message , Rob Brown writes: >Is it possible for Varnish to "manipulate" the pages that it serves out of >cache "on the way out" (post-cache)? >More specifically: I have an app that generates pages with a session key in >them that I need to strip out to make cacheable, but then put back in based >on a session key parameter sent in the query string. If you are talking about headers (cookies ?) then it is possible. If we are talking about stuffin the body of the object, the anwer is no. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From tfheen at varnish-software.com Wed May 12 06:42:06 2010 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Wed, 12 May 2010 08:42:06 +0200 Subject: manipulating page content In-Reply-To: <43533.1273601173@critter.freebsd.dk> (Poul-Henning Kamp's message of "Tue, 11 May 2010 18:06:13 +0000") References: <43533.1273601173@critter.freebsd.dk> Message-ID: <87pr11liyp.fsf@qurzaw.linpro.no> ]] "Poul-Henning Kamp" | In message , Rob | Brown writes: | | >Is it possible for Varnish to "manipulate" the pages that it serves | >out of cache "on the way out" (post-cache)? More specifically: I have | >an app that generates pages with a session key in them that I need to | >strip out to make cacheable, but then put back in based on a session | >key parameter sent in the query string. | | If you are talking about headers (cookies ?) then it is possible. | | If we are talking about stuffin the body of the object, the anwer is | no. You could always use ESI for it, but apart from that, there's currently no way to manipulate the content of a page. -- Tollef Fog Heen Varnish Software t: +47 21 54 41 73 From dtownrobbrown at gmail.com Wed May 12 07:40:36 2010 From: dtownrobbrown at gmail.com (Rob Brown) Date: Wed, 12 May 2010 00:40:36 -0700 Subject: manipulating page content In-Reply-To: <87pr11liyp.fsf@qurzaw.linpro.no> References: <43533.1273601173@critter.freebsd.dk> <87pr11liyp.fsf@qurzaw.linpro.no> Message-ID: > > If we are talking about stuff in the body of the object, the anwer is > no. > yes, unfortunately that is what I am looking for. I am trying to see if varnish can "replicate" the functionality of a commercial cache appliance from a "well-known" vendor. > You could always use ESI for it, but apart from that, there's currently > no way to manipulate the content of a page. > ESI would probably work and be much easier, but reading the wiki ( http://varnish-cache.org/wiki/ESIfeatures) it seems like Varnish doesn't yet support the "Content substitution based on variables" functionality. I am not well versed in ESI, but I think would be looking to do something like this: get page In the wiki, it says "For now we have deemed this feature uninteresting, but adding it is just a matter of programming." What are the chances of getting support for variables in a future release? -------------- next part -------------- An HTML attachment was scrubbed... URL: From ebe at dmi.dk Wed May 12 10:25:40 2010 From: ebe at dmi.dk (Eivind Bengtsson) Date: Wed, 12 May 2010 12:25:40 +0200 Subject: Chield dies Message-ID: <4BEA8224.2090409@dmi.dk> Hi guys I've just had an issue on one of our 4 varnishservers - that I cannot explain.. The varnishd-child just dies. I have a guess though - Might it have something to do with me (a check on my behalf) telnetting to the serviceport to check the backend.health? Is my perl-fu not strong enough? After the child is dead, I do not see the alarm until 10 mins later ... and startup varnishd - I have no varnishlog from the incident but as I start up varnish everything works again... Every 5 minutes my nagios does four checks on the varnishmachines, to check health of the backend. command[check_varnish_backends]=/usr/lib/nagios/plugins/check_varnish_backend_health -all command[check_varnish_backend_euros08]=/usr/lib/nagios/plugins/check_varnish_backend_health -h euros08 command[check_varnish_backend_euros12]=/usr/lib/nagios/plugins/check_varnish_backend_health -h euros12 command[check_varnish_backend_carbon]=/usr/lib/nagios/plugins/check_varnish_backend_health -h carbon I've inlined the .pl doing the nagioscheck and the syslog output from varnishd. Does anyone have a clue ? Sorry in advance for the rathe long mail .. Regards /Eivind # varnishd -V varnishd (varnish-2.0.6) Copyright (c) 2006-2009 Linpro AS / Verdens Gang A ______________________________________ SYSLOG: May 12 10:05:54 varnish1 varnishd[19448]: CLI 7 open from telnet 127.0.0.1:46759 127.0.0.1:6082 May 12 10:05:54 varnish1 varnishd[19448]: CLI 7 result 200 "debug.health" May 12 10:05:54 varnish1 varnishd[19448]: CLI 7 closed May 12 10:05:54 varnish1 varnishd[19448]: CLI 7 open from telnet 127.0.0.1:46761 127.0.0.1:6082 May 12 10:05:54 varnish1 varnishd[19448]: CLI 7 result 200 "debug.health" May 12 10:05:54 varnish1 varnishd[19448]: CLI 7 closed May 12 10:05:54 varnish1 varnishd[19448]: CLI 7 open from telnet 127.0.0.1:46762 127.0.0.1:6082 May 12 10:05:54 varnish1 varnishd[19448]: CLI 7 result 200 "debug.health" May 12 10:05:54 varnish1 varnishd[19448]: CLI 7 closed May 12 10:05:54 varnish1 varnishd[19448]: CLI 7 open from telnet 127.0.0.1:46764 127.0.0.1:6082 May 12 10:05:54 varnish1 varnishd[19448]: CLI 7 result 200 "debug.health" May 12 10:05:54 varnish1 varnishd[19448]: CLI 7 closed .... May 12 10:07:58 varnish1 varnishd[19448]: Child (19449) not responding to ping, killing it. May 12 10:08:00 varnish1 varnishd[19448]: Child (19449) not responding to ping, killing it. May 12 10:08:00 varnish1 varnishd[19448]: Child (19449) died signal=3 May 12 10:08:02 varnish1 varnishd[19448]: Child cleanup complete May 12 10:08:02 varnish1 varnishd[19448]: child (18174) Started May 12 10:08:04 varnish1 kernel: [6214551.451581] Inbound IN=eth0 OUT= MAC=00:14:22:1e:ba:b8:00:0e:84:3b:ec:c0:08:00 SRC=66.151.125.8 DST=130.225.244.83 LEN=32 TOS=0x00 PREC=0x00 TTL=1 ID=1802 PROTO=UDP SPT=11057 DPT=33442 LEN=12 May 12 10:08:06 varnish1 kernel: [6214553.449867] Inbound IN=eth0 OUT= MAC=00:14:22:1e:ba:b8:00:0e:84:3b:ec:c0:08:00 SRC=66.151.125.8 DST=130.225.244.83 LEN=32 TOS=0x00 PREC=0x00 TTL=1 ID=1802 PROTO=UDP SPT=11057 DPT=33442 LEN=12 May 12 10:08:07 varnish1 varnishd[19448]: Pushing vcls failed: CLI communication error May 12 10:08:07 varnish1 varnishd[19448]: Stopping Child May 12 10:08:07 varnish1 varnishd[19448]: Child (18174) said Closed fds: 4 5 6 10 11 13 14 May 12 10:08:07 varnish1 varnishd[19448]: Child (18174) said Child starts May 12 10:08:07 varnish1 varnishd[19448]: Child (18174) said managed to mmap 2147483648 bytes of 2147483648 May 12 10:08:07 varnish1 varnishd[19448]: Child (18174) said Ready .... May 12 10:08:07 varnish1 varnishd[19448]: Pushing vcls failed: CLI communication error May 12 10:08:07 varnish1 varnishd[19448]: Stopping Child May 12 10:08:07 varnish1 varnishd[19448]: Child (18174) said Closed fds: 4 5 6 10 11 13 14 May 12 10:08:07 varnish1 varnishd[19448]: Child (18174) said Child starts May 12 10:08:07 varnish1 varnishd[19448]: Child (18174) said managed to mmap 2147483648 bytes of 2147483648 May 12 10:08:07 varnish1 varnishd[19448]: Child (18174) said Ready .... May 12 10:08:08 varnish1 varnishd[19448]: Child (18174) said Probe("GET /vandstand/servlet/ImageServlet HTTP/1.1 May 12 10:08:08 varnish1 varnishd[19448]: Child (18174) said May 12 10:08:08 varnish1 varnishd[19448]: Child (18174) said Host: carbon.dmi.dk May 12 10:08:08 varnish1 varnishd[19448]: Child (18174) said May 12 10:08:08 varnish1 varnishd[19448]: Child (18174) said Connection: close May 12 10:08:08 varnish1 varnishd[19448]: Child (18174) said May 12 10:08:08 varnish1 varnishd[19448]: last message repeated 2 times May 12 10:08:08 varnish1 varnishd[19448]: Child (18174) said ", 3, 10) May 12 10:08:08 varnish1 varnishd[19448]: Child (18174) said Probe("GET / HTTP/1.1 May 12 10:08:08 varnish1 varnishd[19448]: Child (18174) said May 12 10:08:08 varnish1 varnishd[19448]: Child (18174) said Host: localhost May 12 10:08:08 varnish1 varnishd[19448]: Child (18174) said May 12 10:08:08 varnish1 varnishd[19448]: Child (18174) said Connection: close May 12 10:08:08 varnish1 varnishd[19448]: Child (18174) said May 12 10:08:08 varnish1 varnishd[19448]: last message repeated 2 times May 12 10:08:08 varnish1 varnishd[19448]: Child (18174) said ", 3, 5) May 12 10:08:08 varnish1 varnishd[19448]: Child (18174) said Probe("GET /dmi/dmi.css HTTP/1.1 May 12 10:08:08 varnish1 varnishd[19448]: Child (18174) said May 12 10:08:08 varnish1 varnishd[19448]: Child (18174) said Host: euros08.dmi.dk May 12 10:08:08 varnish1 varnishd[19448]: Child (18174) said May 12 10:08:08 varnish1 varnishd[19448]: Child (18174) said Connection: close May 12 10:08:08 varnish1 varnishd[19448]: Child (18174) said May 12 10:08:08 varnish1 varnishd[19448]: last message repeated 2 times May 12 10:08:08 varnish1 varnishd[19448]: Child (18174) said ", 2, 10) May 12 10:08:08 varnish1 varnishd[19448]: Child (18174) ended May 12 10:08:08 varnish1 varnishd[19448]: Child (-1) said Probe("GET /dmi/dmi.css HTTP/1.1 May 12 10:08:08 varnish1 varnishd[19448]: Child (-1) said May 12 10:08:08 varnish1 varnishd[19448]: Child (-1) said Host: euros12.dmi.dk May 12 10:08:08 varnish1 varnishd[19448]: Child (-1) said May 12 10:08:08 varnish1 varnishd[19448]: Child (-1) said Connection: close May 12 10:08:08 varnish1 varnishd[19448]: Child (-1) said May 12 10:08:08 varnish1 varnishd[19448]: last message repeated 2 times May 12 10:08:08 varnish1 varnishd[19448]: Child (-1) said ", 2, 10) May 12 10:08:08 varnish1 varnishd[19448]: Child cleanup complete .... May 12 10:21:42 varnish1 varnishd[19448]: Manager got SIGINT May 12 10:21:43 varnish1 varnishd[19184]: child (19185) Started May 12 10:21:43 varnish1 varnishd[19184]: Child (19185) said Closed fds: 4 5 6 10 11 13 14 May 12 10:21:43 varnish1 varnishd[19184]: Child (19185) said Child starts May 12 10:21:43 varnish1 varnishd[19184]: Child (19185) said managed to mmap 2147483648 bytes of 2147483648 May 12 10:21:43 varnish1 varnishd[19184]: Child (19185) said Ready May 12 10:21:43 varnish1 varnishd[19184]: Child (19185) said Probe("GET /vandstand/servlet/ImageServlet HTTP/1.1 May 12 10:21:43 varnish1 varnishd[19184]: Child (19185) said May 12 10:21:43 varnish1 varnishd[19184]: Child (19185) said Host: carbon.dmi.dk May 12 10:21:43 varnish1 varnishd[19184]: Child (19185) said May 12 10:21:43 varnish1 varnishd[19184]: Child (19185) said Connection: close May 12 10:21:43 varnish1 varnishd[19184]: Child (19185) said ...And so on.... _________________________________ # cat /usr/lib/nagios/plugins/check_varnish_backend_health #!/usr/bin/perl use strict; use Net::Telnet (); use Getopt::Long; my $version = ''; my $help = ''; my $result = ''; my $status = ''; my $host = ''; my $all = ''; my $verbose = ''; my $handle = new Net::Telnet ( Port=> '6082', Timeout => 8, prompt => '/\.\n|\n\n/' ); ## Print Help sub usage { my $msg = shift; if (defined $msg) { print "$msg\n"; } print << "END"; Usage: check_varnish_backend_health option Options: --help -h|--host hostname --all --verbose --version NB! Only one option at the time! :-) END exit(3); } #Get options from command line $result = GetOptions( "host|h=s" => \$host, "all" => \$all, "version" => \$version, "verbose" => \$verbose, "help" => \$help ); if ($help) {usage();} if ($version) {usage('Version is 0.9_EBE');} if ($all) {if ($host){usage('Either --all _or_ --host hostname')};} if ($verbose) {if ($host){usage('Either --verbose _or_ --host hostname')};} my $ip = "127.0.0.1"; $handle->open("$ip"); my @lines = $handle->cmd("debug\.health"); $handle->close; my %backends_state_hash; my @sick_backends; foreach (@lines) { if (/Backend/) { my @line = split(' ',$_); $status = pop(@line); my $be = @line[1]; if ($verbose) { print "Host: $be => $status\n"; } if ($status eq 'Sick'){ push(@sick_backends,$be); } $backends_state_hash{$be} = $status; } } if ($host) { if ( $backends_state_hash{$host} eq 'Healthy'){ print "OK: Host $host is Healthy"; exit 0; } elsif ( $backends_state_hash{$host} eq 'Sick'){ print "Critical: Host $host is Sick"; exit 2; } else { print "Warning: Host $host is not Healthy, nor is it Sick"; exit 1; } } my $number_of_sick_backends=@sick_backends; my $number_of_backends = keys %backends_state_hash; if ( $number_of_backends == 0 ) { print "Warning: No backends are probed"; exit 1; } if ( $number_of_sick_backends == 0 ) { print "OK: All hosts are Healthy"; exit 0; } if ( $number_of_sick_backends == $number_of_backends ) { print "Critical: All hosts are Sick - @sick_backends"; exit 2; } else { print "Warning: Some hosts are sick: @sick_backends"; exit 1; } ______________________________________ -- Eivind Bengtsson Systemadministrator - Cand.merc.(dat) Danmarks Meteorologiske Institut Lyngbyvej 100 2100 K?benhavn ? Direkte tlf. : 39157544 Email: ebe at dmi.dk echo 'This is not a pipe.' | cat -> /dev/tty From phk at phk.freebsd.dk Wed May 12 20:00:16 2010 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Wed, 12 May 2010 20:00:16 +0000 Subject: manipulating page content In-Reply-To: Your message of "Wed, 12 May 2010 00:40:36 MST." Message-ID: <3319.1273694416@critter.freebsd.dk> In message , Rob Brown writes: >ESI would probably work and be much easier, but reading the wiki ( >http://varnish-cache.org/wiki/ESIfeatures) it seems like Varnish doesn't yet >support the "Content substitution based on variables" functionality. Correct, with limited developer bandwidth, choices have to be made. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From phk at phk.freebsd.dk Wed May 12 20:04:31 2010 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Wed, 12 May 2010 20:04:31 +0000 Subject: Chield dies In-Reply-To: Your message of "Wed, 12 May 2010 12:25:40 +0200." <4BEA8224.2090409@dmi.dk> Message-ID: <3333.1273694671@critter.freebsd.dk> In message <4BEA8224.2090409 at dmi.dk>, Eivind Bengtsson writes: >on my behalf) telnetting to the serviceport to check the backend.health? >Is my perl-fu not strong enough? In general, the debug.* commands should be used in production, I have named them thusly for a reason. I do realize that we need a much better way to report and monitor backend state/health/stats, and I'm working on that. One thing to be aware of, is that the CLI is singlethreaded, so a CLI command taking a long time could potentially get in the way of the ping/pong check. Try increasing the cli_timeout parameter to make the master process less facist in this aspect. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From l at lrowe.co.uk Wed May 12 20:41:01 2010 From: l at lrowe.co.uk (Laurence Rowe) Date: Wed, 12 May 2010 21:41:01 +0100 Subject: manipulating page content In-Reply-To: References: <43533.1273601173@critter.freebsd.dk> <87pr11liyp.fsf@qurzaw.linpro.no> Message-ID: On 12 May 2010 08:40, Rob Brown wrote: >> If we are talking about stuff in the body of the object, the anwer is >> no. > > > yes, unfortunately that is what I am looking for. I am trying to see if > varnish can "replicate" the functionality of a commercial cache appliance > from a "well-known" vendor. > >> >> You could always use ESI for it, but apart from that, there's currently >> no way to manipulate the content of a page. > > ESI would probably work and be much easier, but reading the wiki > (http://varnish-cache.org/wiki/ESIfeatures)?it seems like Varnish doesn't > yet support the "Content substitution based on variables" functionality. > I am not well versed in ESI, but I think would be looking to do something > like this: > > href="http://www.example.com/getpage?sessionKey=$(QUERY_STRING{'sessionKey'})"/>get > page > > In the wiki, it says "For now we have deemed this feature uninteresting, but > adding it is just a matter of programming." What are the chances of getting > support for variables in a future release? It might be possible to make this work with just ESI includes. Assuming that the ESI subrequests reuse the same req object as the parent request (I'm not sure about this point), all you need do is generate the key as a synthetic response in vcl_error. Something like: sub vcl_recv { if (req.url == "/_esi/sessionKey") { error 701; } else { set req.http.X-Session-Key = # regexp to extract session key from url } } sub vcl_error { if (obj.status == 701) { synthetic { "get page" }; } } And use in your page: Laurence From dtownrobbrown at gmail.com Wed May 12 22:52:21 2010 From: dtownrobbrown at gmail.com (dtownrobbrown at gmail.com) Date: Wed, 12 May 2010 22:52:21 +0000 Subject: manipulating page content In-Reply-To: Message-ID: <00163646b670a886d204866d7e61@google.com> On May 12, 2010 1:41pm, Laurence Rowe wrote: > On 12 May 2010 08:40, Rob Brown dtownrobbrown at gmail.com> wrote: > >> If we are talking about stuff in the body of the object, the anwer is > >> no. > > > > > > yes, unfortunately that is what I am looking for. I am trying to see if > > varnish can "replicate" the functionality of a commercial cache > appliance > > from a "well-known" vendor. > > > >> > >> You could always use ESI for it, but apart from that, there's currently > >> no way to manipulate the content of a page. > > > > ESI would probably work and be much easier, but reading the wiki > > (http://varnish-cache.org/wiki/ESIfeatures) it seems like Varnish > doesn't > > yet support the "Content substitution based on variables" functionality. > > I am not well versed in ESI, but I think would be looking to do > something > > like this: > > > > > > > href="http://www.example.com/getpage?sessionKey=$(QUERY_STRING{'sessionKey'})"/>get > > page > > > > In the wiki, it says "For now we have deemed this feature > uninteresting, but > > adding it is just a matter of programming." What are the chances of > getting > > support for variables in a future release? > It might be possible to make this work with just ESI includes. > Assuming that the ESI subrequests reuse the same req object as the > parent request (I'm not sure about this point), all you need do is > generate the key as a synthetic response in vcl_error. Something like: > sub vcl_recv { > if (req.url == "/_esi/sessionKey") { > error 701; > } else { > set req.http.X-Session-Key = # regexp to extract session key from url > } > } > sub vcl_error { > if (obj.status == 701) { > synthetic { > "http://www.example.com/getpage?sessionKey=" > req.http.X-Session-Key "%34>get page" > }; > } > } > And use in your page: > > Laurence That's pretty cool... Will have to try it! -------------- next part -------------- An HTML attachment was scrubbed... URL: From plfgoa at gmail.com Fri May 14 10:37:54 2010 From: plfgoa at gmail.com (Paras Fadte) Date: Fri, 14 May 2010 16:07:54 +0530 Subject: Changes in vcl Message-ID: Hi , Have changes been made in vcl such that "obj.cacheable" is now referred to as "beresp.cacheable" . Is this the only change ? Is this change applicable in all functions in vcl configuration file or just limited to some functions . example only in vcl_fetch "obj" is replaced by "beresp". Thank you. -Paras -------------- next part -------------- An HTML attachment was scrubbed... URL: From tfheen at varnish-software.com Fri May 14 14:49:36 2010 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Fri, 14 May 2010 16:49:36 +0200 Subject: vim syntax In-Reply-To: <201004051958.53424.glen@delfi.ee> ("Elan =?utf-8?Q?Ruusam?= =?utf-8?Q?=C3=A4e=22's?= message of "Mon, 5 Apr 2010 19:58:53 +0300") References: <200911241957.54742.glen@delfi.ee> <20100122140158.GA9672@kjeks.varnish-cache.com> <201004051958.53424.glen@delfi.ee> Message-ID: <877hn6a67z.fsf@qurzaw.linpro.no> ]] Elan Ruusam?e | publishing url of my efforts: | http://cvs.pld-linux.org/cgi-bin/cvsweb.cgi/packages/vim-syntax-vcl/vcl.vim | | i'd love to get feedback and improvements on it :) Are you happy with us pulling this into varnish-tools in the svn repository? That'll make sure it stays around. -- Tollef Fog Heen Varnish Software t: +47 21 54 41 73 From tfheen at varnish-software.com Fri May 14 14:51:22 2010 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Fri, 14 May 2010 16:51:22 +0200 Subject: Changes in vcl In-Reply-To: (Paras Fadte's message of "Fri, 14 May 2010 16:07:54 +0530") References: Message-ID: <8739xua651.fsf@qurzaw.linpro.no> ]] Paras Fadte | Have changes been made in vcl such that "obj.cacheable" is now | referred to as "beresp.cacheable" . obj.* was renamed to beresp.* with the change to version 2.1, yes. | Is this the only change ? I recommend reading the changelog, it should all be documented there. :-) -- Tollef Fog Heen Varnish Software t: +47 21 54 41 73 From dtownrobbrown at gmail.com Fri May 14 17:50:51 2010 From: dtownrobbrown at gmail.com (Rob Brown) Date: Fri, 14 May 2010 10:50:51 -0700 Subject: Changes in vcl In-Reply-To: References: Message-ID: On Fri, May 14, 2010 at 3:37 AM, Paras Fadte wrote: > Hi , > > Have changes been made in vcl such that "obj.cacheable" is now referred to > as "beresp.cacheable" . Is this the only change ? Is this change applicable > in all functions in vcl configuration file or just limited to some functions > . example only in vcl_fetch "obj" is replaced by "beresp". > > Thank you. > > -Paras > Ah, yes... I ran into this same issue because I had originally set up Varnish on a test Ubuntu workstation and just installed the package from the repository, which happened to be v.2.0.3. I had my config all dialed in and working there, and then built a new box with CentOS and installed the 2.1 RPM from sourceforge. When I copied my "working" config over and tried to start varnishd, it just "died". No error message. Nothing. Took me a good chunk of time of debugging before I finally ran across the note in the changelog and was like "ah ha!". It would be nice if: a) There was some syntax checking on startup that would tell you that obj is not valid in vcl_fetch (anymore). b) Put a note on WIKI pages that have "example" configs that will only work on 2.0. (I'll volunteer for b if someone wants to send me the code to edit wiki). -------------- next part -------------- An HTML attachment was scrubbed... URL: From tfheen at varnish-software.com Fri May 14 21:41:39 2010 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Fri, 14 May 2010 23:41:39 +0200 Subject: Changes in vcl In-Reply-To: (Rob Brown's message of "Fri, 14 May 2010 10:50:51 -0700") References: Message-ID: <874oia88ks.fsf@qurzaw.linpro.no> ]] Rob Brown | a) There was some syntax checking on startup that would tell you that obj is | not valid in vcl_fetch (anymore). There is, but I believe the init scripts on RH don't communicate the startup failure back to the user properly. | b) Put a note on WIKI pages that have "example" configs that will only work | on 2.0. | (I'll volunteer for b if someone wants to send me the code to edit wiki). What is your wiki user name? I'm happy to give you the edit bit. (Please just update them to work with 2.1 rather than marking them as 2.0-only, thought). -- Tollef Fog Heen Varnish Software t: +47 21 54 41 73 From glen at delfi.ee Sat May 15 12:37:46 2010 From: glen at delfi.ee (Elan =?utf-8?q?Ruusam=C3=A4e?=) Date: Sat, 15 May 2010 15:37:46 +0300 Subject: vim syntax In-Reply-To: <877hn6a67z.fsf@qurzaw.linpro.no> References: <200911241957.54742.glen@delfi.ee> <201004051958.53424.glen@delfi.ee> <877hn6a67z.fsf@qurzaw.linpro.no> Message-ID: <201005151537.46648.glen@delfi.ee> On Friday 14 May 2010 17:49:36 Tollef Fog Heen wrote: > ]] Elan Ruusam?e > > | publishing url of my efforts: > | http://cvs.pld-linux.org/cgi-bin/cvsweb.cgi/packages/vim-syntax-vcl/vcl.v > |im > | > | i'd love to get feedback and improvements on it :) > > Are you happy with us pulling this into varnish-tools in the svn > repository? That'll make sure it stays around. absolutely yes, there's no question of that :) -- glen From samcrawford at gmail.com Tue May 18 08:47:42 2010 From: samcrawford at gmail.com (Sam Crawford) Date: Tue, 18 May 2010 09:47:42 +0100 Subject: Varnish 2.06 running away with memory Message-ID: Morning all, A bit of a newbie question I fear... We've been successfully running Varnish 2.04 for over a year now on our intranet, and have found in the past two days that our instances in one particular region are eating up a lot of system memory. We've upgraded to 2.06 a couple of days ago, and are still experiencing the same issue. [root at webserver ~]# ps aux USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND .... root 32637 0.0 0.0 97268 1096 ? Ss 05:13 0:00 /local/0/opt/varnish/sbin/varnishd -P /var/run/varnish.pid -a 10.131.100.36:8080 -T 127.0.0.1:8 082 -n internal -f /local/0/opt/varnish/etc/varnish/internal.vcl -s file,/local/0/opt/varnish/cache/internal.cache,512M nobody 32638 23.9 81.7 14641440 13418764 ? Sl 05:13 177:35 /local/0/opt/varnish/sbin/varnishd -P /var/run/varnish.pid -a 10.131.100.36:8080 -T 127.0.0.1:8 082 -n internal -f /local/0/opt/varnish/etc/varnish/internal.vcl -s file,/local/0/opt/varnish/cache/internal.cache,512M See above that PID 32638 is using 14GB of RAM. We only have a relatively small number of items (in the order of 10's of thousands), but they're hit very frequently. Therefore a cache size of 512M to 1GB is sufficient for our needs. Below is the output from varnishstat: [root at webserver bin]# ./varnishstat -n internal -1 uptime 45005 . Child uptime client_conn 411358 9.14 Client connections accepted client_drop 0 0.00 Connection dropped, no sess client_req 1513928 33.64 Client requests received cache_hit 222603 4.95 Cache hits cache_hitpass 59894 1.33 Cache hits for pass cache_miss 15566 0.35 Cache misses backend_conn 39422 0.88 Backend conn. success backend_unhealthy 0 0.00 Backend conn. not attempted backend_busy 0 0.00 Backend conn. too many backend_fail 0 0.00 Backend conn. failures backend_reuse 1251666 27.81 Backend conn. reuses backend_toolate 25078 0.56 Backend conn. was closed backend_recycle 1276976 28.37 Backend conn. recycles backend_unused 0 0.00 Backend conn. unused fetch_head 354 0.01 Fetch head fetch_length 594722 13.21 Fetch with Length fetch_chunked 631083 14.02 Fetch chunked fetch_eof 0 0.00 Fetch EOF fetch_bad 0 0.00 Fetch had bad headers fetch_close 395 0.01 Fetch wanted close fetch_oldhttp 0 0.00 Fetch pre HTTP/1.1 closed fetch_zero 64637 1.44 Fetch zero len fetch_failed 0 0.00 Fetch failed n_srcaddr 0 . N struct srcaddr n_srcaddr_act 0 . N active struct srcaddr n_sess_mem 447 . N struct sess_mem n_sess 94 . N struct sess n_object 1460 . N struct object n_objecthead 2370 . N struct objecthead n_smf 3034 . N struct smf n_smf_frag 144 . N small free smf n_smf_large 13 . N large free smf n_vbe_conn 11 . N struct vbe_conn n_bereq 299 . N struct bereq n_wrk 43 . N worker threads n_wrk_create 975 0.02 N worker threads created n_wrk_failed 0 0.00 N worker threads not created n_wrk_max 0 0.00 N worker threads limited n_wrk_queue 0 0.00 N queued work requests n_wrk_overflow 2857 0.06 N overflowed work requests n_wrk_drop 0 0.00 N dropped work requests n_backend 1 . N backends n_expired 14134 . N expired objects n_lru_nuked 0 . N LRU nuked objects n_lru_saved 0 . N LRU saved objects n_lru_moved 184533 . N LRU moved objects n_deathrow 0 . N objects on deathrow losthdr 0 0.00 HTTP header overflows n_objsendfile 0 0.00 Objects sent with sendfile n_objwrite 1262710 28.06 Objects sent with write n_objoverflow 0 0.00 Objects overflowing workspace s_sess 411349 9.14 Total Sessions s_req 1513941 33.64 Total Requests s_pipe 0 0.00 Total pipe s_pass 1275725 28.35 Total pass s_fetch 1291202 28.69 Total fetch s_hdrbytes 559253905 12426.48 Total header bytes s_bodybytes 6015944416 133672.80 Total body bytes sess_closed 58624 1.30 Session Closed sess_pipeline 0 0.00 Session Pipeline sess_readahead 0 0.00 Session Read Ahead sess_linger 1483437 32.96 Session Linger sess_herd 1207272 26.83 Session herd shm_records 106754812 2372.07 SHM records shm_writes 5312192 118.04 SHM writes shm_flushes 240 0.01 SHM flushes due to overflow shm_cont 2364 0.05 SHM MTX contention shm_cycles 50 0.00 SHM cycles through buffer sm_nreq 2519582 55.98 allocator requests sm_nobj 2877 . outstanding allocations sm_balloc 24608768 . bytes allocated sm_bfree 512262144 . bytes free sma_nreq 0 0.00 SMA allocator requests sma_nobj 0 . SMA outstanding allocations sma_nbytes 0 . SMA outstanding bytes sma_balloc 0 . SMA bytes allocated sma_bfree 0 . SMA bytes free sms_nreq 89 0.00 SMS allocator requests sms_nobj 0 . SMS outstanding allocations sms_nbytes 0 . SMS outstanding bytes sms_balloc 43343 . SMS bytes allocated sms_bfree 43343 . SMS bytes freed backend_req 1291192 28.69 Backend requests made n_vcl 1 0.00 N vcl total n_vcl_avail 1 0.00 N vcl available n_vcl_discard 0 0.00 N vcl discarded n_purge 246879 . N total active purges n_purge_add 252285 5.61 N new purges added n_purge_retire 5406 0.12 N old purges deleted n_purge_obj_test 245731 5.46 N objects tested n_purge_re_test 214813964 4773.11 N regexps tested against n_purge_dups 3227 0.07 N duplicate purges removed hcb_nolock 0 0.00 HCB Lookups without lock hcb_lock 0 0.00 HCB Lookups with lock hcb_insert 0 0.00 HCB Inserts esi_parse 0 0.00 Objects ESI parsed (unlock) esi_errors 0 0.00 ESI parse errors (unlock) Are we just missing a simple parameter to varnishd, or is there something else here? Thanks, Sam From scott at idealist.org Tue May 18 14:19:04 2010 From: scott at idealist.org (Scott Wilson) Date: Tue, 18 May 2010 16:19:04 +0200 Subject: Varnish 2.06 running away with memory In-Reply-To: References: Message-ID: Hi Sam, On 18 May 2010 10:47, Sam Crawford wrote: > Morning all, > > A bit of a newbie question I fear... We've been successfully running > Varnish 2.04 for over a year now on our intranet, and have found in > the past two days that our instances in one particular region are > eating up a lot of system memory. We've upgraded to 2.06 a couple of > days ago, and are still experiencing the same issue. > > n_purge ? ? ? ? ? ? ? ?246879 ? ? ? ? ?. ? N total active purges > n_purge_add ? ? ? ? ? ?252285 ? ? ? ? 5.61 N new purges added > n_purge_retire ? ? ? ? ? 5406 ? ? ? ? 0.12 N old purges deleted > n_purge_obj_test ? ? ? 245731 ? ? ? ? 5.46 N objects tested > n_purge_re_test ? ? 214813964 ? ? ?4773.11 N regexps tested against > n_purge_dups ? ? ? ? ? ? 3227 ? ? ? ? 0.07 N duplicate purges removed > hcb_nolock ? ? ? ? ? ? ? ? ?0 ? ? ? ? 0.00 HCB Lookups without lock > hcb_lock ? ? ? ? ? ? ? ? ? ?0 ? ? ? ? 0.00 HCB Lookups with lock > hcb_insert ? ? ? ? ? ? ? ? ?0 ? ? ? ? 0.00 HCB Inserts > esi_parse ? ? ? ? ? ? ? ? ? 0 ? ? ? ? 0.00 Objects ESI parsed (unlock) > esi_errors ? ? ? ? ? ? ? ? ?0 ? ? ? ? 0.00 ESI parse errors (unlock) > It looks to me like you're generating an awful lot of purge list entries. We struggled for a long time with this problem before abandoning purge in favour of setting object TTLs to 0. See this FAQ entry: http://varnish-cache.org/wiki/VCLExamplePurging The problem we were having was that we were using purge to implement force-refresh functionality, but there are so many badly behaved crawlers and other clients out there that send always send the NoCache header that the purge list would swamp all available resources. Hope that helps! cheers, scott From samcrawford at gmail.com Tue May 18 14:41:36 2010 From: samcrawford at gmail.com (Sam Crawford) Date: Tue, 18 May 2010 15:41:36 +0100 Subject: Varnish 2.06 running away with memory In-Reply-To: References: Message-ID: Scott, Thanks for that, I'll experiment with that and see how we fare! Regards, Sam On 18 May 2010 15:19, Scott Wilson wrote: > Hi Sam, > > On 18 May 2010 10:47, Sam Crawford wrote: >> Morning all, >> >> A bit of a newbie question I fear... We've been successfully running >> Varnish 2.04 for over a year now on our intranet, and have found in >> the past two days that our instances in one particular region are >> eating up a lot of system memory. We've upgraded to 2.06 a couple of >> days ago, and are still experiencing the same issue. >> > >> n_purge ? ? ? ? ? ? ? ?246879 ? ? ? ? ?. ? N total active purges >> n_purge_add ? ? ? ? ? ?252285 ? ? ? ? 5.61 N new purges added >> n_purge_retire ? ? ? ? ? 5406 ? ? ? ? 0.12 N old purges deleted >> n_purge_obj_test ? ? ? 245731 ? ? ? ? 5.46 N objects tested >> n_purge_re_test ? ? 214813964 ? ? ?4773.11 N regexps tested against >> n_purge_dups ? ? ? ? ? ? 3227 ? ? ? ? 0.07 N duplicate purges removed >> hcb_nolock ? ? ? ? ? ? ? ? ?0 ? ? ? ? 0.00 HCB Lookups without lock >> hcb_lock ? ? ? ? ? ? ? ? ? ?0 ? ? ? ? 0.00 HCB Lookups with lock >> hcb_insert ? ? ? ? ? ? ? ? ?0 ? ? ? ? 0.00 HCB Inserts >> esi_parse ? ? ? ? ? ? ? ? ? 0 ? ? ? ? 0.00 Objects ESI parsed (unlock) >> esi_errors ? ? ? ? ? ? ? ? ?0 ? ? ? ? 0.00 ESI parse errors (unlock) >> > > It looks to me like you're generating an awful lot of purge list > entries. ?We struggled for a long time with this problem before > abandoning purge in favour of setting object TTLs to 0. ?See this FAQ > entry: > > http://varnish-cache.org/wiki/VCLExamplePurging > > The problem we were having was that we were using purge to implement > force-refresh functionality, but there are so many badly behaved > crawlers and other clients out there that send always send the NoCache > header that the purge list would swamp all available resources. > > Hope that helps! ?cheers, > scott > From l at lrowe.co.uk Tue May 18 22:57:14 2010 From: l at lrowe.co.uk (Laurence Rowe) Date: Tue, 18 May 2010 23:57:14 +0100 Subject: Varnish 2.06 running away with memory In-Reply-To: References: Message-ID: Be aware that if you use varies, you must purge all potential combinations (for example, variations on Accept-Encoding) individually when using `set obj.ttl = 0s`. I believe Varnish 2.1 now has a thread to process the purge list and prevent it growing out of hand. Laurence On 18 May 2010 15:41, Sam Crawford wrote: > Scott, > > Thanks for that, I'll experiment with that and see how we fare! > > Regards, > > Sam > > > On 18 May 2010 15:19, Scott Wilson wrote: >> Hi Sam, >> >> On 18 May 2010 10:47, Sam Crawford wrote: >>> Morning all, >>> >>> A bit of a newbie question I fear... We've been successfully running >>> Varnish 2.04 for over a year now on our intranet, and have found in >>> the past two days that our instances in one particular region are >>> eating up a lot of system memory. We've upgraded to 2.06 a couple of >>> days ago, and are still experiencing the same issue. >>> >> >>> n_purge ? ? ? ? ? ? ? ?246879 ? ? ? ? ?. ? N total active purges >>> n_purge_add ? ? ? ? ? ?252285 ? ? ? ? 5.61 N new purges added >>> n_purge_retire ? ? ? ? ? 5406 ? ? ? ? 0.12 N old purges deleted >>> n_purge_obj_test ? ? ? 245731 ? ? ? ? 5.46 N objects tested >>> n_purge_re_test ? ? 214813964 ? ? ?4773.11 N regexps tested against >>> n_purge_dups ? ? ? ? ? ? 3227 ? ? ? ? 0.07 N duplicate purges removed >>> hcb_nolock ? ? ? ? ? ? ? ? ?0 ? ? ? ? 0.00 HCB Lookups without lock >>> hcb_lock ? ? ? ? ? ? ? ? ? ?0 ? ? ? ? 0.00 HCB Lookups with lock >>> hcb_insert ? ? ? ? ? ? ? ? ?0 ? ? ? ? 0.00 HCB Inserts >>> esi_parse ? ? ? ? ? ? ? ? ? 0 ? ? ? ? 0.00 Objects ESI parsed (unlock) >>> esi_errors ? ? ? ? ? ? ? ? ?0 ? ? ? ? 0.00 ESI parse errors (unlock) >>> >> >> It looks to me like you're generating an awful lot of purge list >> entries. ?We struggled for a long time with this problem before >> abandoning purge in favour of setting object TTLs to 0. ?See this FAQ >> entry: >> >> http://varnish-cache.org/wiki/VCLExamplePurging >> >> The problem we were having was that we were using purge to implement >> force-refresh functionality, but there are so many badly behaved >> crawlers and other clients out there that send always send the NoCache >> header that the purge list would swamp all available resources. >> >> Hope that helps! ?cheers, >> scott >> > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://lists.varnish-cache.org/mailman/listinfo/varnish-misc > From samcrawford at gmail.com Wed May 19 07:51:59 2010 From: samcrawford at gmail.com (Sam Crawford) Date: Wed, 19 May 2010 08:51:59 +0100 Subject: Varnish 2.06 running away with memory In-Reply-To: References: Message-ID: Thanks Laurence. To be clear, does that mean with Varnish 2.1.x I can just use purge(...) and not see the same memory/CPU hit as on Varnish 2.0.x ? Thanks, Sam On 18 May 2010 23:57, Laurence Rowe wrote: > Be aware that if you use varies, you must purge all potential > combinations (for example, variations on Accept-Encoding) individually > when using `set obj.ttl = 0s`. I believe Varnish 2.1 now has a thread > to process the purge list and prevent it growing out of hand. > > Laurence > > On 18 May 2010 15:41, Sam Crawford wrote: >> Scott, >> >> Thanks for that, I'll experiment with that and see how we fare! >> >> Regards, >> >> Sam >> >> >> On 18 May 2010 15:19, Scott Wilson wrote: >>> Hi Sam, >>> >>> On 18 May 2010 10:47, Sam Crawford wrote: >>>> Morning all, >>>> >>>> A bit of a newbie question I fear... We've been successfully running >>>> Varnish 2.04 for over a year now on our intranet, and have found in >>>> the past two days that our instances in one particular region are >>>> eating up a lot of system memory. We've upgraded to 2.06 a couple of >>>> days ago, and are still experiencing the same issue. >>>> >>> >>>> n_purge ? ? ? ? ? ? ? ?246879 ? ? ? ? ?. ? N total active purges >>>> n_purge_add ? ? ? ? ? ?252285 ? ? ? ? 5.61 N new purges added >>>> n_purge_retire ? ? ? ? ? 5406 ? ? ? ? 0.12 N old purges deleted >>>> n_purge_obj_test ? ? ? 245731 ? ? ? ? 5.46 N objects tested >>>> n_purge_re_test ? ? 214813964 ? ? ?4773.11 N regexps tested against >>>> n_purge_dups ? ? ? ? ? ? 3227 ? ? ? ? 0.07 N duplicate purges removed >>>> hcb_nolock ? ? ? ? ? ? ? ? ?0 ? ? ? ? 0.00 HCB Lookups without lock >>>> hcb_lock ? ? ? ? ? ? ? ? ? ?0 ? ? ? ? 0.00 HCB Lookups with lock >>>> hcb_insert ? ? ? ? ? ? ? ? ?0 ? ? ? ? 0.00 HCB Inserts >>>> esi_parse ? ? ? ? ? ? ? ? ? 0 ? ? ? ? 0.00 Objects ESI parsed (unlock) >>>> esi_errors ? ? ? ? ? ? ? ? ?0 ? ? ? ? 0.00 ESI parse errors (unlock) >>>> >>> >>> It looks to me like you're generating an awful lot of purge list >>> entries. ?We struggled for a long time with this problem before >>> abandoning purge in favour of setting object TTLs to 0. ?See this FAQ >>> entry: >>> >>> http://varnish-cache.org/wiki/VCLExamplePurging >>> >>> The problem we were having was that we were using purge to implement >>> force-refresh functionality, but there are so many badly behaved >>> crawlers and other clients out there that send always send the NoCache >>> header that the purge list would swamp all available resources. >>> >>> Hope that helps! ?cheers, >>> scott >>> >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc >> > From perbu at varnish-software.com Wed May 19 08:09:45 2010 From: perbu at varnish-software.com (Per Buer) Date: Wed, 19 May 2010 10:09:45 +0200 Subject: Varnish 2.06 running away with memory In-Reply-To: References: Message-ID: No. 2.1 still uses memory for keeping around 'bans'. Note that Varnish will discard bans older then the oldest object in the cache. If you keep your TTLs somewhat short memory usage will be lower. In 2.1 Varnish can 'nuke' all variants of a page through in a ttl=0 manner. Per. On Wed, May 19, 2010 at 9:51 AM, Sam Crawford wrote: > Thanks Laurence. To be clear, does that mean with Varnish 2.1.x I can > just use purge(...) and not see the same memory/CPU hit as on Varnish > 2.0.x ? > > Thanks, > > Sam > > > On 18 May 2010 23:57, Laurence Rowe wrote: >> Be aware that if you use varies, you must purge all potential >> combinations (for example, variations on Accept-Encoding) individually >> when using `set obj.ttl = 0s`. I believe Varnish 2.1 now has a thread >> to process the purge list and prevent it growing out of hand. >> >> Laurence >> >> On 18 May 2010 15:41, Sam Crawford wrote: >>> Scott, >>> >>> Thanks for that, I'll experiment with that and see how we fare! >>> >>> Regards, >>> >>> Sam >>> >>> >>> On 18 May 2010 15:19, Scott Wilson wrote: >>>> Hi Sam, >>>> >>>> On 18 May 2010 10:47, Sam Crawford wrote: >>>>> Morning all, >>>>> >>>>> A bit of a newbie question I fear... We've been successfully running >>>>> Varnish 2.04 for over a year now on our intranet, and have found in >>>>> the past two days that our instances in one particular region are >>>>> eating up a lot of system memory. We've upgraded to 2.06 a couple of >>>>> days ago, and are still experiencing the same issue. >>>>> >>>> >>>>> n_purge ? ? ? ? ? ? ? ?246879 ? ? ? ? ?. ? N total active purges >>>>> n_purge_add ? ? ? ? ? ?252285 ? ? ? ? 5.61 N new purges added >>>>> n_purge_retire ? ? ? ? ? 5406 ? ? ? ? 0.12 N old purges deleted >>>>> n_purge_obj_test ? ? ? 245731 ? ? ? ? 5.46 N objects tested >>>>> n_purge_re_test ? ? 214813964 ? ? ?4773.11 N regexps tested against >>>>> n_purge_dups ? ? ? ? ? ? 3227 ? ? ? ? 0.07 N duplicate purges removed >>>>> hcb_nolock ? ? ? ? ? ? ? ? ?0 ? ? ? ? 0.00 HCB Lookups without lock >>>>> hcb_lock ? ? ? ? ? ? ? ? ? ?0 ? ? ? ? 0.00 HCB Lookups with lock >>>>> hcb_insert ? ? ? ? ? ? ? ? ?0 ? ? ? ? 0.00 HCB Inserts >>>>> esi_parse ? ? ? ? ? ? ? ? ? 0 ? ? ? ? 0.00 Objects ESI parsed (unlock) >>>>> esi_errors ? ? ? ? ? ? ? ? ?0 ? ? ? ? 0.00 ESI parse errors (unlock) >>>>> >>>> >>>> It looks to me like you're generating an awful lot of purge list >>>> entries. ?We struggled for a long time with this problem before >>>> abandoning purge in favour of setting object TTLs to 0. ?See this FAQ >>>> entry: >>>> >>>> http://varnish-cache.org/wiki/VCLExamplePurging >>>> >>>> The problem we were having was that we were using purge to implement >>>> force-refresh functionality, but there are so many badly behaved >>>> crawlers and other clients out there that send always send the NoCache >>>> header that the purge list would swamp all available resources. >>>> >>>> Hope that helps! ?cheers, >>>> scott >>>> >>> >>> _______________________________________________ >>> varnish-misc mailing list >>> varnish-misc at varnish-cache.org >>> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc >>> >> > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://lists.varnish-cache.org/mailman/listinfo/varnish-misc > -- Per Buer, Varnish Software Phone: +47 21 54 41 21 / Mobile: +47 958 39 117 / skype: per.buer From tfheen at varnish-software.com Wed May 19 10:04:09 2010 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Wed, 19 May 2010 12:04:09 +0200 Subject: Varnish 2.06 running away with memory In-Reply-To: (Sam Crawford's message of "Wed, 19 May 2010 08:51:59 +0100") References: Message-ID: <87pr0s438m.fsf@qurzaw.linpro.no> ]] Sam Crawford | Thanks Laurence. To be clear, does that mean with Varnish 2.1.x I can | just use purge(...) and not see the same memory/CPU hit as on Varnish | 2.0.x ? As long as the purge thread is active and your bans only rely on obj.* (and not req.*), Varnish will try to keep it under control by actively checking bans against objects. As an alternative, you can use the nuke support Per's mail refers to. That is more suitable for getting rid of single objects rather than, say, a whole hierarchy. -- Tollef Fog Heen Varnish Software t: +47 21 54 41 73 From l at lrowe.co.uk Wed May 19 10:09:51 2010 From: l at lrowe.co.uk (Laurence Rowe) Date: Wed, 19 May 2010 11:09:51 +0100 Subject: Varnish 2.06 running away with memory In-Reply-To: <87pr0s438m.fsf@qurzaw.linpro.no> References: <87pr0s438m.fsf@qurzaw.linpro.no> Message-ID: Is there an example of using nuke? I'm particularly interested in seeing how to use it in vcl_miss (for when there is nothing cached for the particular vary from the PURGE request). Laurence On 19 May 2010 11:04, Tollef Fog Heen wrote: > ]] Sam Crawford > > | Thanks Laurence. To be clear, does that mean with Varnish 2.1.x I can > | just use purge(...) and not see the same memory/CPU hit as on Varnish > | 2.0.x ? > > As long as the purge thread is active and your bans only rely on obj.* > (and not req.*), Varnish will try to keep it under control by actively > checking bans against objects. > > As an alternative, you can use the nuke support Per's mail refers to. > That is more suitable for getting rid of single objects rather than, > say, a whole hierarchy. > > -- > Tollef Fog Heen > Varnish Software > t: +47 21 54 41 73 > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://lists.varnish-cache.org/mailman/listinfo/varnish-misc > From armdan20 at gmail.com Wed May 19 13:13:52 2010 From: armdan20 at gmail.com (andan andan) Date: Wed, 19 May 2010 15:13:52 +0200 Subject: Varnish 2.06 running away with memory In-Reply-To: References: Message-ID: 2010/5/19 Per Buer : > No. 2.1 still uses memory for keeping around 'bans'. Note that Varnish > will discard bans older then the oldest object in the cache. If you > keep your TTLs somewhat short memory usage will be lower. > > In 2.1 Varnish can 'nuke' all variants of a page through in a ttl=0 manner. Please, can you give more details on this point or give a link to the documentation. Thanks in advance. From perbu at varnish-software.com Wed May 19 14:40:29 2010 From: perbu at varnish-software.com (Per Buer) Date: Wed, 19 May 2010 16:40:29 +0200 Subject: Varnish 2.06 running away with memory In-Reply-To: References: Message-ID: On Wed, May 19, 2010 at 3:13 PM, andan andan wrote: > 2010/5/19 Per Buer : >> No. 2.1 still uses memory for keeping around 'bans'. Note that Varnish >> will discard bans older then the oldest object in the cache. If you >> keep your TTLs somewhat short memory usage will be lower. >> >> In 2.1 Varnish can 'nuke' all variants of a page through in a ttl=0 manner. > > Please, can you give more details on this point or give a link to the > documentation. I'm sorry to say that this requires inline C at the moment and is somewhat unavailable to me. Arthur or Poul should know how to invoke it. Per. -- Per Buer, Varnish Software Phone: +47 21 54 41 21 / Mobile: +47 958 39 117 / skype: per.buer From augusto at jadedpixel.com Thu May 20 22:45:25 2010 From: augusto at jadedpixel.com (Augusto Becciu) Date: Thu, 20 May 2010 19:45:25 -0300 Subject: Varnish 2.1 + kswapd0 freak out Message-ID: Hey guys, I'm running Varnish 2.1 in two m2.xlarge ec2 instances (17G of RAM + linux kernel 2.6.21.7-2.fc8xen-ec2-v1.0). Those two servers have been running for 2 months now almost without trouble. But I've noticed some crazy spikes in cpu usage (mostly in kernel land) once in a while. A few days ago I saw kswapd0 consuming 100% of a cpu core and varnishd consuming 100% of the other cpu core. I could strace varnish for a few seconds and everything looked normal, but then it crashed and left a zombie process eating most of the cpu so I had to restart the server. Today, exactly the same thing happened in the other server, and this is starting to scare me out. We're running varnish with the following params: varnishd -P /var/run/varnishd.pid -a 0.0.0.0:2000 -T 127.0.0.1:6082 -w 200,2000 -s malloc,12G -p lru_interval=20 -f /etc/varnish/varnish.vcl We don't have swap enabled on these servers. Here's varnishstat -1 when varnish was freaking out: client_conn 9592723 7.82 Client connections accepted client_drop 0 0.00 Connection dropped, no sess/wrk client_req 67302765 54.84 Client requests received cache_hit 50571130 41.20 Cache hits cache_hitpass 0 0.00 Cache hits for pass cache_miss 16050808 13.08 Cache misses backend_conn 16029200 13.06 Backend conn. success backend_unhealthy 0 0.00 Backend conn. not attempted backend_busy 0 0.00 Backend conn. too many backend_fail 20649 0.02 Backend conn. failures backend_reuse 12352 0.01 Backend conn. reuses backend_toolate 0 0.00 Backend conn. was closed backend_recycle 12352 0.01 Backend conn. recycles backend_unused 0 0.00 Backend conn. unused fetch_head 0 0.00 Fetch head fetch_length 12764170 10.40 Fetch with Length fetch_chunked 3272791 2.67 Fetch chunked fetch_eof 0 0.00 Fetch EOF fetch_bad 0 0.00 Fetch had bad headers fetch_close 49 0.00 Fetch wanted close fetch_oldhttp 0 0.00 Fetch pre HTTP/1.1 closed fetch_zero 0 0.00 Fetch zero len fetch_failed 3272895 2.67 Fetch failed n_sess_mem 587 . N struct sess_mem n_sess 465 . N struct sess n_object 659083 . N struct object n_vampireobject 0 . N unresurrected objects n_objectcore 659439 . N struct objectcore n_objecthead 907405 . N struct objecthead n_smf 0 . N struct smf n_smf_frag 0 . N small free smf n_smf_large 0 . N large free smf n_vbe_conn 277 . N struct vbe_conn n_wrk 400 . N worker threads n_wrk_create 458 0.00 N worker threads created n_wrk_failed 0 0.00 N worker threads not created n_wrk_max 0 0.00 N worker threads limited n_wrk_queue 0 0.00 N queued work requests n_wrk_overflow 874 0.00 N overflowed work requests n_wrk_drop 0 0.00 N dropped work requests n_backend 2 . N backends n_expired 112662 . N expired objects n_lru_nuked 11954429 . N LRU nuked objects n_lru_saved 0 . N LRU saved objects n_lru_moved 46618517 . N LRU moved objects n_deathrow 0 . N objects on deathrow losthdr 2 0.00 HTTP header overflows n_objsendfile 0 0.00 Objects sent with sendfile n_objwrite 60192420 49.04 Objects sent with write n_objoverflow 0 0.00 Objects overflowing workspace s_sess 9592577 7.82 Total Sessions s_req 67302765 54.84 Total Requests s_pipe 110 0.00 Total pipe s_pass 1691 0.00 Total pass s_fetch 12764115 10.40 Total fetch s_hdrbytes 21558035591 17564.97 Total header bytes s_bodybytes 1162454990977 947140.58 Total body bytes sess_closed 7687689 6.26 Session Closed sess_pipeline 0 0.00 Session Pipeline sess_readahead 0 0.00 Session Read Ahead sess_linger 61236267 49.89 Session Linger sess_herd 16659649 13.57 Session herd shm_records 3395953253 2766.94 SHM records shm_writes 131371160 107.04 SHM writes shm_flushes 661 0.00 SHM flushes due to overflow shm_cont 114836 0.09 SHM MTX contention shm_cycles 1378 0.00 SHM cycles through buffer sm_nreq 0 0.00 allocator requests sm_nobj 0 . outstanding allocations sm_balloc 0 . bytes allocated sm_bfree 0 . bytes free sma_nreq 37442974 30.51 SMA allocator requests sma_nobj 1318091 . SMA outstanding allocations sma_nbytes 12884892751 . SMA outstanding bytes sma_balloc 250925494011 . SMA bytes allocated sma_bfree 238040601260 . SMA bytes free sms_nreq 3967048 3.23 SMS allocator requests sms_nobj 0 . SMS outstanding allocations sms_nbytes 18446744073709527064 . SMS outstanding bytes sms_balloc 1895595320 . SMS bytes allocated sms_bfree 1895619352 . SMS bytes freed backend_req 16043889 13.07 Backend requests made n_vcl 1 0.00 N vcl total n_vcl_avail 1 0.00 N vcl available n_vcl_discard 0 0.00 N vcl discarded n_purge 26155 . N total active purges n_purge_add 678663 0.55 N new purges added n_purge_retire 652508 0.53 N old purges deleted n_purge_obj_test 47484518 38.69 N objects tested n_purge_re_test 41413683761 33742.88 N regexps tested against n_purge_dups 485656 0.40 N duplicate purges removed hcb_nolock 50605455 41.23 HCB Lookups without lock hcb_lock 566 0.00 HCB Lookups with lock hcb_insert 16016509 13.05 HCB Inserts esi_parse 0 0.00 Objects ESI parsed (unlock) esi_errors 0 0.00 ESI parse errors (unlock) accept_fail 0 0.00 Accept failures client_drop_late 0 0.00 Connection dropped late uptime 1227331 1.00 Client uptime Have anyone experienced something similar? Thanks, Augusto From ingvar at redpill-linpro.com Fri May 21 08:50:07 2010 From: ingvar at redpill-linpro.com (Ingvar Hagelund) Date: Fri, 21 May 2010 10:50:07 +0200 Subject: Changes in vcl In-Reply-To: <874oia88ks.fsf@qurzaw.linpro.no> References: <874oia88ks.fsf@qurzaw.linpro.no> Message-ID: <1274431807.15932.21.camel@yum.linpro.no> * Rob Brown > | a) There was some syntax checking on startup that would tell you that obj is > | not valid in vcl_fetch (anymore). * Tollef Fog Heen > There is, but I believe the init scripts on RH don't communicate the > startup failure back to the user properly. I'll file a bug against the EPEL package, and ask the package maintainer to fix that, if possible. On Red Hat, It's costume to redirect stdout and stderr to /dev/null, since most daemons have their own logfiles. Any chance to get that error message visible in the syslog? Ingvar From kb+varnish at slide.com Fri May 21 22:30:30 2010 From: kb+varnish at slide.com (Ken Brownfield) Date: Fri, 21 May 2010 15:30:30 -0700 Subject: Varnish 2.1 + kswapd0 freak out In-Reply-To: References: Message-ID: <48F06F15-AF6B-4EB3-AF30-6558D52C91F1@slide.com> This happens when there is significant memory pressure (heavy network or block I/O exacerbates it) when you have swap disabled. This is a long-standing kernel behavior issue, from my experience. I've seen it with MySQL and other servers, in addition to Varnish. That being said, perhaps Varnish is initiating a huge object expiry pass that's triggering this problem. If you increase the cli_timeout, it may help prevent the child from being reaped, to probably dubious benefit. Increasing /proc/sys/vm/min_free_kbytes (by 2-3x) may also help, but it's a dangerous beast. I use 131072 for all of my Varnish instances, for this reason (16-64GB boxes). You may also find that adding a few hundred meg of swap is the lesser of two evils. Hope it helps, -- Ken On May 20, 2010, at 3:45 PM, Augusto Becciu wrote: > We don't have swap enabled on these servers. From michael at dynamine.net Fri May 21 23:19:35 2010 From: michael at dynamine.net (Michael Fischer) Date: Fri, 21 May 2010 16:19:35 -0700 Subject: Varnish 2.1 + kswapd0 freak out In-Reply-To: <48F06F15-AF6B-4EB3-AF30-6558D52C91F1@slide.com> References: <48F06F15-AF6B-4EB3-AF30-6558D52C91F1@slide.com> Message-ID: +1 Add some swap. And be sure to watch your swap paging activity. --Michael On Fri, May 21, 2010 at 3:30 PM, Ken Brownfield wrote: > This happens when there is significant memory pressure (heavy network or block I/O exacerbates it) when you have swap disabled. ?This is a long-standing kernel behavior issue, from my experience. ?I've seen it with MySQL and other servers, in addition to Varnish. > > That being said, perhaps Varnish is initiating a huge object expiry pass that's triggering this problem. ?If you increase the cli_timeout, it may help prevent the child from being reaped, to probably dubious benefit. > > Increasing /proc/sys/vm/min_free_kbytes (by 2-3x) may also help, but it's a dangerous beast. ?I use 131072 for all of my Varnish instances, for this reason (16-64GB boxes). > > You may also find that adding a few hundred meg of swap is the lesser of two evils. > > Hope it helps, > -- > Ken > > On May 20, 2010, at 3:45 PM, Augusto Becciu wrote: > >> We don't have swap enabled on these servers. > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://lists.varnish-cache.org/mailman/listinfo/varnish-misc > From angie.tawfik at gmail.com Sun May 23 13:17:28 2010 From: angie.tawfik at gmail.com (Angie T. Muhammad) Date: Sun, 23 May 2010 16:17:28 +0300 Subject: Parsing non-English URLs Message-ID: Hello Varnish team I have varnish v. 2.1.2 on production and test servers . We are running a bilingual news website. On my test server I am trying to parse non-English URLs like follows: ....................... else if (req.url == "/*??????*") { set beresp.http.X-Cacheable = "Yes"; set beresp.ttl = 60m; return(deliver); } ....................... The word in *bold red* is in Arabic and it is a right-to-left language. The link can not be made in English and has no English equivalent. In case you are wondering, the word means "reports". My sole problem now is that varnish applies all other if-statements with full English URLs but not this one with Arabiv. Even if I try regex say: *req.url ~ "^/??????" *instead of the == sign, it starts with no errors but does not apply the rule. I tried the following: 1- Reversing the letters of the arabic word, so ?????? would be ?????? but it did not work 2- Copying the link directly into /etc/varnish/default.vcl, it produces something like: %D9%88%D8%B3%D9%88%D9%85%D8%A7%D8%AA Such html address handling prevents varnish from starting Any ideas? Your help is really appreciated. -- All the best, Angie -------------- next part -------------- An HTML attachment was scrubbed... URL: From samcrawford at gmail.com Mon May 24 14:51:34 2010 From: samcrawford at gmail.com (Sam Crawford) Date: Mon, 24 May 2010 15:51:34 +0100 Subject: Parsing non-English URLs In-Reply-To: References: Message-ID: It's not one that I'm familiar with, but if it were me, I'd try running varnishlog whilst putting a request for one of these URLs through. See how varnish prints it out in the RxURL field. This might give you some clues as how to specify it in the rules. Thanks, Sam 2010/5/23 Angie T. Muhammad : > Hello Varnish team > > I have varnish v. 2.1.2 on production and test servers . We are running a > bilingual news website. > On my test server I am trying to parse non-English URLs like follows: > > ....................... > ? else if (req.url == "/??????") { > ????? set beresp.http.X-Cacheable = "Yes"; > ????? set beresp.ttl = 60m; > ????? return(deliver); > ???? } > ?....................... > > The word in bold red is in Arabic and it is a right-to-left language. The > link can not be made in English and has no English equivalent. In case you > are wondering, the word means "reports". My sole problem now is that varnish > applies all other if-statements with full English URLs but not this one with > Arabiv. Even if I try regex say: req.url ~ "^/??????" instead of the == > sign, it starts with no errors but does not apply the rule. > > I tried the following: > 1- Reversing the letters of the arabic word, so ??????? would be ?????? but > it did not work > 2- Copying the link directly into /etc/varnish/default.vcl, it produces > something like: %D9%88%D8%B3%D9%88%D9%85%D8%A7%D8%AA > ???? Such html address handling prevents varnish from starting > > Any ideas? Your help is really appreciated. > > > -- > All the best, > Angie > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://lists.varnish-cache.org/mailman/listinfo/varnish-misc > From angie.tawfik at gmail.com Tue May 25 10:08:26 2010 From: angie.tawfik at gmail.com (Angie T. Muhammad) Date: Tue, 25 May 2010 13:08:26 +0300 Subject: Parsing non-English URLs In-Reply-To: References: Message-ID: Thank you Sam for your response. I already logged requests to cached Arabic URLs and here is the result of one request: =========================================================================================== Cookie: SESScfc90a62c81b7bfc6f292320b1d0b8ca=t7t650vu5qu02916unbtil9o66; SESS50745c6a3729e7f46278f7d281511580=qjc658f7cthp6dvj65rt6a8c64; SESS8348e9a0e0f6133hash*%ntrol: max-age=0%c9c2n9td5uuvj0hp73; SESSb323fb39997d18c5bde4c32f7bc0ffe1=0r5ve4k3i2ubmqu ???: 0  ?: ????-?????   806   =========================================================================================== I tried opening the log file with less, vim, and tail but all what am getting is either binary (less) or stuff like above (tail). I even tried limiting the accepted charset header sent by the browser to UTF-8 but failed. Here is my config for limiting the charset under sub vcl_rcv { } : ====================================== if (req.http.Accept-Charset) { remove req.http.Accept-Charset; set req.http.Accept-Charset = "utf-8"; } ====================================== I also tried including C header files as follows: =================================== C{ #include #include #include #include #include }C =================================== but it did not give me any result. I am thinking of recompiling with ncurses wchar enabled. Any ideas? 2010/5/24 Sam Crawford > It's not one that I'm familiar with, but if it were me, I'd try > running varnishlog whilst putting a request for one of these URLs > through. See how varnish prints it out in the RxURL field. This might > give you some clues as how to specify it in the rules. > > Thanks, > > Sam > > > 2010/5/23 Angie T. Muhammad : > > Hello Varnish team > > > > I have varnish v. 2.1.2 on production and test servers . We are running a > > bilingual news website. > > On my test server I am trying to parse non-English URLs like follows: > > > > ....................... > > else if (req.url == "/??????") { > > set beresp.http.X-Cacheable = "Yes"; > > set beresp.ttl = 60m; > > return(deliver); > > } > > ....................... > > > > The word in bold red is in Arabic and it is a right-to-left language. The > > link can not be made in English and has no English equivalent. In case > you > > are wondering, the word means "reports". My sole problem now is that > varnish > > applies all other if-statements with full English URLs but not this one > with > > Arabiv. Even if I try regex say: req.url ~ "^/??????" instead of the == > > sign, it starts with no errors but does not apply the rule. > > > > I tried the following: > > 1- Reversing the letters of the arabic word, so ?????? would be ?????? > but > > it did not work > > 2- Copying the link directly into /etc/varnish/default.vcl, it produces > > something like: %D9%88%D8%B3%D9%88%D9%85%D8%A7%D8%AA > > Such html address handling prevents varnish from starting > > > > Any ideas? Your help is really appreciated. > > > > > > -- > > All the best, > > Angie > > > > _______________________________________________ > > varnish-misc mailing list > > varnish-misc at varnish-cache.org > > http://lists.varnish-cache.org/mailman/listinfo/varnish-misc > > > -- All the best, Angie -------------- next part -------------- An HTML attachment was scrubbed... URL: From felix at seconddrawer.com.au Tue May 25 10:53:14 2010 From: felix at seconddrawer.com.au (Felix) Date: Tue, 25 May 2010 20:53:14 +1000 Subject: Parsing non-English URLs In-Reply-To: References: Message-ID: <20100525105314.GA4533@thinkpad> Just to add to the discussion, I have Varnish running in front of a couple of Thai language sites. The URL '/?????????????' corresponds to the following entry in varnishlog in Varnish 2.1.1: 13 RxURL c /%E0%B8%81%E0%B8%A5%E0%B8%B2%E0%B8%87%E0%B8%9B%E0%B8%A3%E0%B8%B0%E0%B9%80%E0%B8%97%E0%B8%A8%E0%B9%84%E0%B8%97%E0%B8%A2 which is just all the high bits escaped as %nn sequences. This is actually a result of the browser (in this case Chrome) doing the conversion. This is confirmed by a netcat session. I am not sure if all browsers do the same conversion. Some more details might be gleaned from here: http://code.google.com/p/browsersec/wiki/Part1#Unicode_in_URLs But obviously varnish needs to be able to cope with these conversions. -felix On Tue, May 25, 2010 at 01:08:26PM +0300, Angie T. Muhammad wrote: > Thank you Sam for your response. I already logged requests to cached Arabic > URLs and here is the result of one request: > =========================================================================================== > Cookie: SESScfc90a62c81b7bfc6f292320b1d0b8ca=t7t650vu5qu02916unbtil9o66; > SESS50745c6a3729e7f46278f7d281511580=qjc658f7cthp6dvj65rt6a8c64; > SESS8348e9a0e0f6133hash*%ntrol: max-age=0%c9c2n9td5uuvj0hp73; > SESSb323fb39997d18c5bde4c32f7bc0ffe1=0r5ve4k3i2ubmqu > ???: 0  ?: ????-?????   806   > =========================================================================================== > > I tried opening the log file with less, vim, and tail but all what am > getting is either binary (less) or stuff like above (tail). > I even tried limiting the accepted charset header sent by the browser to > UTF-8 but failed. Here is my config for limiting the charset under sub > vcl_rcv { } : > ====================================== > if (req.http.Accept-Charset) { > remove req.http.Accept-Charset; > set req.http.Accept-Charset = "utf-8"; > } > ====================================== > > I also tried including C header files as follows: > =================================== > C{ > #include > #include > #include > #include > #include > }C > =================================== > but it did not give me any result. > > I am thinking of recompiling with ncurses wchar enabled. Any ideas? > > > 2010/5/24 Sam Crawford > > > It's not one that I'm familiar with, but if it were me, I'd try > > running varnishlog whilst putting a request for one of these URLs > > through. See how varnish prints it out in the RxURL field. This might > > give you some clues as how to specify it in the rules. > > > > Thanks, > > > > Sam > > > > > > 2010/5/23 Angie T. Muhammad : > > > Hello Varnish team > > > > > > I have varnish v. 2.1.2 on production and test servers . We are running a > > > bilingual news website. > > > On my test server I am trying to parse non-English URLs like follows: > > > > > > ....................... > > > else if (req.url == "/??????") { > > > set beresp.http.X-Cacheable = "Yes"; > > > set beresp.ttl = 60m; > > > return(deliver); > > > } > > > ....................... > > > > > > The word in bold red is in Arabic and it is a right-to-left language. The > > > link can not be made in English and has no English equivalent. In case > > you > > > are wondering, the word means "reports". My sole problem now is that > > varnish > > > applies all other if-statements with full English URLs but not this one > > with > > > Arabiv. Even if I try regex say: req.url ~ "^/??????" instead of the == > > > sign, it starts with no errors but does not apply the rule. > > > > > > I tried the following: > > > 1- Reversing the letters of the arabic word, so ?????? would be ?????? > > but > > > it did not work > > > 2- Copying the link directly into /etc/varnish/default.vcl, it produces > > > something like: %D9%88%D8%B3%D9%88%D9%85%D8%A7%D8%AA > > > Such html address handling prevents varnish from starting > > > > > > Any ideas? Your help is really appreciated. > > > > > > > > > -- > > > All the best, > > > Angie > > > > > > _______________________________________________ > > > varnish-misc mailing list > > > varnish-misc at varnish-cache.org > > > http://lists.varnish-cache.org/mailman/listinfo/varnish-misc > > > > > > > > > -- > All the best, > Angie > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://lists.varnish-cache.org/mailman/listinfo/varnish-misc -- email: felix at seconddrawer.com.au web: http://seconddrawer.com.au/ gpg: E6FC 5BC6 268D B874 E546 8F6F A2BB 220B D5F6 92E3 Please don't send me Word or PowerPoint attachments. See http://www.gnu.org/philosophy/no-word-attachments.html From plfgoa at gmail.com Tue May 25 12:19:15 2010 From: plfgoa at gmail.com (Paras Fadte) Date: Tue, 25 May 2010 17:49:15 +0530 Subject: Increase in CPU usage Message-ID: Hi, I see increase in CPU usage upon upgrading from 2.0.5 to 2.1.2 ? Has anyone experienced the same ? The traffic, no.of requests , config file ( except for changes from obj to beresp in vcl_fetch) are same. What could be the reason ? Varnish version : varnishd (varnish-2.1.2 SVN 4769:4772) Thank you. -Paras -------------- next part -------------- An HTML attachment was scrubbed... URL: From phk at phk.freebsd.dk Tue May 25 12:28:41 2010 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Tue, 25 May 2010 12:28:41 +0000 Subject: Requesting real-world stats, please Message-ID: <57685.1274790521@critter.freebsd.dk> I may have "misunderestimated" the dynamics of how objects live in Varnish caches and thus taken a wrong optimization decision with the critbit hasher. I am looking for data to figure this out, specifically the values of the two varnishstat counters: "N struct objecthead." and "N struct object" Along with information about which version of varnish, and which, if any -h argument you use. If you happen to have this data for both -hcritbit and -hclassic it would be doubly-plus-wonderful. Thanks in advance, Poul-Henning -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From augusto at jadedpixel.com Tue May 25 14:02:34 2010 From: augusto at jadedpixel.com (Augusto Becciu) Date: Tue, 25 May 2010 11:02:34 -0300 Subject: Varnish 2.1 + kswapd0 freak out In-Reply-To: References: <48F06F15-AF6B-4EB3-AF30-6558D52C91F1@slide.com> Message-ID: Awesome, thanks guys! On Fri, May 21, 2010 at 8:19 PM, Michael Fischer wrote: > +1 > > Add some swap. ?And be sure to watch your swap paging activity. > > --Michael > > On Fri, May 21, 2010 at 3:30 PM, Ken Brownfield wrote: >> This happens when there is significant memory pressure (heavy network or block I/O exacerbates it) when you have swap disabled. ?This is a long-standing kernel behavior issue, from my experience. ?I've seen it with MySQL and other servers, in addition to Varnish. >> >> That being said, perhaps Varnish is initiating a huge object expiry pass that's triggering this problem. ?If you increase the cli_timeout, it may help prevent the child from being reaped, to probably dubious benefit. >> >> Increasing /proc/sys/vm/min_free_kbytes (by 2-3x) may also help, but it's a dangerous beast. ?I use 131072 for all of my Varnish instances, for this reason (16-64GB boxes). >> >> You may also find that adding a few hundred meg of swap is the lesser of two evils. >> >> Hope it helps, >> -- >> Ken >> >> On May 20, 2010, at 3:45 PM, Augusto Becciu wrote: >> >>> We don't have swap enabled on these servers. >> >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc >> > From jfrias at gmail.com Tue May 25 15:41:14 2010 From: jfrias at gmail.com (Javier Frias) Date: Tue, 25 May 2010 11:41:14 -0400 Subject: Requesting real-world stats, please In-Reply-To: <57685.1274790521@critter.freebsd.dk> References: <57685.1274790521@critter.freebsd.dk> Message-ID: Two varnish instances, they each do about 7million requests per day. 798639 . . N struct object 798918 . . N struct objectcore 658115 . . N struct objecthead 800850 . . N struct object 801108 . . N struct objectcore 659982 . . N struct objecthead Varnish 2.1 -h classic,250007 -Javier On Tue, May 25, 2010 at 8:28 AM, Poul-Henning Kamp wrote: > > I may have "misunderestimated" the dynamics of how objects live in > Varnish caches and thus taken a wrong optimization decision with > the critbit hasher. > > I am looking for data to figure this out, specifically the values of > the two varnishstat counters: > > ? ? ? ?"N struct objecthead." > and > ? ? ? ?"N struct object" > > Along with information about which version of varnish, and which, if > any -h argument you use. > > If you happen to have this data for both -hcritbit and -hclassic it > would be doubly-plus-wonderful. > > Thanks in advance, > > Poul-Henning > > -- > Poul-Henning Kamp ? ? ? | UNIX since Zilog Zeus 3.20 > phk at FreeBSD.ORG ? ? ? ? | TCP/IP since RFC 956 > FreeBSD committer ? ? ? | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetence. > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://lists.varnish-cache.org/mailman/listinfo/varnish-misc > From cosimo at streppone.it Tue May 25 15:42:56 2010 From: cosimo at streppone.it (Cosimo Streppone) Date: Tue, 25 May 2010 17:42:56 +0200 Subject: Requesting real-world stats, please In-Reply-To: <57685.1274790521@critter.freebsd.dk> References: <57685.1274790521@critter.freebsd.dk> Message-ID: On Tue, 25 May 2010 14:28:41 +0200, Poul-Henning Kamp wrote: > > I may have "misunderestimated" the dynamics of how objects live in > Varnish caches and thus taken a wrong optimization decision with > the critbit hasher. > > I am looking for data to figure this out, specifically the values of > the two varnishstat counters: > > "N struct objecthead." > and > "N struct object" > > Along with information about which version of varnish, and which, if > any -h argument you use. I don't have any critbit info, since we're running on 2.0.4, ii libvarnish1 2.0.4-5~bpo50+1 shared libraries for Varnish ii varnish 2.0.4-5~bpo50+1 a state-of-the-art, high-performance HTTP accelerator Here's the counters from one installation, hash default settings serving ~100 req/s of very small objects, 2 twin machines, n_object 100585 . N struct object n_objecthead 103700 . N struct objecthead n_object 102270 . N struct object n_objecthead 102294 . N struct objecthead Maybe I should use -h with a higher number? :) This other here is a different installation, ~500-1000 req/s of medium/big files/pics and longer ttl (10 days default), again 2 twin machines, n_object 303979 . N struct object n_objecthead 299199 . N struct objecthead n_object 506522 . N struct object n_objecthead 504143 . N struct objecthead Here I'm using '-h classic,500009' -- Cosimo From moseleymark at gmail.com Tue May 25 20:30:18 2010 From: moseleymark at gmail.com (Mark Moseley) Date: Tue, 25 May 2010 13:30:18 -0700 Subject: Requesting real-world stats, please In-Reply-To: <57685.1274790521@critter.freebsd.dk> References: <57685.1274790521@critter.freebsd.dk> Message-ID: On Tue, May 25, 2010 at 5:28 AM, Poul-Henning Kamp wrote: > > I may have "misunderestimated" the dynamics of how objects live in > Varnish caches and thus taken a wrong optimization decision with > the critbit hasher. > > I am looking for data to figure this out, specifically the values of > the two varnishstat counters: > > ? ? ? ?"N struct objecthead." > and > ? ? ? ?"N struct object" > > Along with information about which version of varnish, and which, if > any -h argument you use. > > If you happen to have this data for both -hcritbit and -hclassic it > would be doubly-plus-wonderful. Here are 10 2.1.2 64-bit servers using critbit (no -h arg) and 10 using classic (-h classic -- no bucket # arg). I restarted them all at the same time a couple of hours ago. Using critbit n_object 62228 . N struct object n_objecthead 63142 . N struct objecthead n_object 38113 . N struct object n_objecthead 39805 . N struct objecthead n_object 53140 . N struct object n_objecthead 53970 . N struct objecthead n_object 79585 . N struct object n_objecthead 80546 . N struct objecthead n_object 61286 . N struct object n_objecthead 62108 . N struct objecthead n_object 58180 . N struct object n_objecthead 59079 . N struct objecthead n_object 59765 . N struct object n_objecthead 60620 . N struct objecthead n_object 52693 . N struct object n_objecthead 53492 . N struct objecthead n_object 5132 . N struct object n_objecthead 5538 . N struct objecthead n_object 55557 . N struct object n_objecthead 56351 . N struct objecthead ================================================================== Using classic n_object 65614 . N struct object n_objecthead 65629 . N struct objecthead n_object 52666 . N struct object n_objecthead 52673 . N struct objecthead n_object 48054 . N struct object n_objecthead 47778 . N struct objecthead n_object 73213 . N struct object n_objecthead 73202 . N struct objecthead n_object 60705 . N struct object n_objecthead 60746 . N struct objecthead n_object 61087 . N struct object n_objecthead 61113 . N struct objecthead n_object 46570 . N struct object n_objecthead 46718 . N struct objecthead n_object 54921 . N struct object n_objecthead 54954 . N struct objecthead n_object 39509 . N struct object n_objecthead 39525 . N struct objecthead n_object 45367 . N struct object n_objecthead 45408 . N struct objecthead From audun at ytterdal.net Tue May 25 22:18:38 2010 From: audun at ytterdal.net (Audun Ytterdal) Date: Wed, 26 May 2010 00:18:38 +0200 Subject: Requesting real-world stats, please In-Reply-To: <57685.1274790521@critter.freebsd.dk> References: <57685.1274790521@critter.freebsd.dk> Message-ID: On Tue, May 25, 2010 at 2:28 PM, Poul-Henning Kamp wrote: > > I may have "misunderestimated" the dynamics of how objects live in > Varnish caches and thus taken a wrong optimization decision with > the critbit hasher. > > I am looking for data to figure this out, specifically the values of > the two varnishstat counters: > > ? ? ? ?"N struct objecthead." > and > ? ? ? ?"N struct object" > > Along with information about which version of varnish, and which, if > any -h argument you use. One varnish doing about 4320 million req/day, peaks at 8K reqs/s [root at dexter ~]# rpm -qa |grep varnish varnish-2.1.0-2.vg varnish-libs-2.1.0-2.vg [root at dexter ~]# ps axu |grep varnish varnish 1239 54.8 74.1 14895928 4526520 ? Sl May24 1053:21 /usr/sbin/varnishd -P /var/run/varnish.pid -a :80 -f /etc/varnish/vg.vcl -T 127.0.0.1:82 -t 600 -u varnish -g varnish -p thread_pool_add_delay 5 -p thread_pool_timeout 120 -p thread_pools 4 -w 200,3000,120 -s malloc,5G [root at dexter ~]# varnishstat -1 -f n_object,n_objecthead n_object 129723 . N struct object n_objecthead 163562 . N struct objecthead From angie.tawfik at gmail.com Wed May 26 09:22:22 2010 From: angie.tawfik at gmail.com (Angie T. Muhammad) Date: Wed, 26 May 2010 12:22:22 +0300 Subject: Requesting real-world stats, please In-Reply-To: References: <57685.1274790521@critter.freebsd.dk> Message-ID: Adun, may I ask about apache's configuration running behind this varnish installation and HW specs too if you don't mind ? Thank you :) On Wed, May 26, 2010 at 1:18 AM, Audun Ytterdal wrote: > On Tue, May 25, 2010 at 2:28 PM, Poul-Henning Kamp > wrote: > > > > I may have "misunderestimated" the dynamics of how objects live in > > Varnish caches and thus taken a wrong optimization decision with > > the critbit hasher. > > > > I am looking for data to figure this out, specifically the values of > > the two varnishstat counters: > > > > "N struct objecthead." > > and > > "N struct object" > > > > Along with information about which version of varnish, and which, if > > any -h argument you use. > > One varnish doing about 4320 million req/day, peaks at 8K reqs/s > > [root at dexter ~]# rpm -qa |grep varnish > varnish-2.1.0-2.vg > varnish-libs-2.1.0-2.vg > > [root at dexter ~]# ps axu |grep varnish > varnish 1239 54.8 74.1 14895928 4526520 ? Sl May24 1053:21 > /usr/sbin/varnishd -P /var/run/varnish.pid -a :80 -f > /etc/varnish/vg.vcl -T 127.0.0.1:82 -t 600 -u varnish -g varnish -p > thread_pool_add_delay 5 -p thread_pool_timeout 120 -p thread_pools 4 > -w 200,3000,120 -s malloc,5G > > [root at dexter ~]# varnishstat -1 -f n_object,n_objecthead > n_object 129723 . N struct object > n_objecthead 163562 . N struct objecthead > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://lists.varnish-cache.org/mailman/listinfo/varnish-misc > -- All the best, Angie -------------- next part -------------- An HTML attachment was scrubbed... URL: From michal.taborsky at nrholding.com Wed May 26 09:55:34 2010 From: michal.taborsky at nrholding.com (Michal Taborsky - Netretail Holding) Date: Wed, 26 May 2010 11:55:34 +0200 Subject: Requesting real-world stats, please In-Reply-To: <57685.1274790521@critter.freebsd.dk> References: <57685.1274790521@critter.freebsd.dk> Message-ID: <4BFCF016.7030005@nrholding.com> Two instances, version 2.0.6, no -h parameter, each serving about 7M requests per day, with peaks of about 250 req/s, stats are about the same for both: n_object 541521 . N struct object n_objecthead 543118 . N struct objecthead Most objects are small images, CSS, JS etc., very long TTL (practically forever). Michal Dne 25.5.2010 14:28, Poul-Henning Kamp napsal(a): > I may have "misunderestimated" the dynamics of how objects live in > Varnish caches and thus taken a wrong optimization decision with > the critbit hasher. > > I am looking for data to figure this out, specifically the values of > the two varnishstat counters: > > "N struct objecthead." > and > "N struct object" > > Along with information about which version of varnish, and which, if > any -h argument you use. > > If you happen to have this data for both -hcritbit and -hclassic it > would be doubly-plus-wonderful. > > Thanks in advance, > > Poul-Henning > > -- Michal T?borsk? chief systems architect Netretail Holding, B.V. http://www.nrholding.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From angie.tawfik at gmail.com Wed May 26 10:22:50 2010 From: angie.tawfik at gmail.com (Angie T. Muhammad) Date: Wed, 26 May 2010 13:22:50 +0300 Subject: Requesting real-world stats, please In-Reply-To: References: <57685.1274790521@critter.freebsd.dk> Message-ID: # ps aux | grep varnish varnish 3772 0.1 1.6 21146720 411640 ? Sl May08 42:28 /usr/sbin/varnishd -P /var/run/varnish.pid -a :80 -T localhost:6082 -f /etc/varnish/default.vcl -u varnish -g varnish -p thread_pools 4 -p thread_pool_min 500 -p thread_pool_max 4000 -p thread_pool_add_delay 2 -p listen_depth 4096 -p lru_interval 60 -p session_linger 100 -h classic,169313 -p sess_workspace 262144 -p log_hashstring off -p sess_timeout 10 -p shm_workspace 32768 -p connect_timeout 600 -p max_restarts 6 -s malloc,2G # varnishstat -1 -f n_object,n_objecthead n_object 9596 . N struct object n_objecthead 8216 . N struct objecthead # rpm -qa | grep varnish varnish-libs-2.1.2-1 varnish-2.1.2-1 serving around 70,000 requests / day with a peak of 12 req / sec On Wed, May 26, 2010 at 12:22 PM, Angie T. Muhammad wrote: > Adun, may I ask about apache's configuration running behind this varnish > installation and HW specs too if you don't mind ? > Thank you :) > > On Wed, May 26, 2010 at 1:18 AM, Audun Ytterdal wrote: > >> On Tue, May 25, 2010 at 2:28 PM, Poul-Henning Kamp >> wrote: >> > >> > I may have "misunderestimated" the dynamics of how objects live in >> > Varnish caches and thus taken a wrong optimization decision with >> > the critbit hasher. >> > >> > I am looking for data to figure this out, specifically the values of >> > the two varnishstat counters: >> > >> > "N struct objecthead." >> > and >> > "N struct object" >> > >> > Along with information about which version of varnish, and which, if >> > any -h argument you use. >> >> One varnish doing about 4320 million req/day, peaks at 8K reqs/s >> >> [root at dexter ~]# rpm -qa |grep varnish >> varnish-2.1.0-2.vg >> varnish-libs-2.1.0-2.vg >> >> [root at dexter ~]# ps axu |grep varnish >> varnish 1239 54.8 74.1 14895928 4526520 ? Sl May24 1053:21 >> /usr/sbin/varnishd -P /var/run/varnish.pid -a :80 -f >> /etc/varnish/vg.vcl -T 127.0.0.1:82 -t 600 -u varnish -g varnish -p >> thread_pool_add_delay 5 -p thread_pool_timeout 120 -p thread_pools 4 >> -w 200,3000,120 -s malloc,5G >> >> [root at dexter ~]# varnishstat -1 -f n_object,n_objecthead >> n_object 129723 . N struct object >> n_objecthead 163562 . N struct objecthead >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc >> > > > > -- > All the best, > Angie > -- All the best, Angie -------------- next part -------------- An HTML attachment was scrubbed... URL: From afassl at progis.de Wed May 26 14:15:09 2010 From: afassl at progis.de (Andreas Fassl) Date: Wed, 26 May 2010 16:15:09 +0200 Subject: Varnish: Re: Requesting real-world stats, please In-Reply-To: References: <57685.1274790521@critter.freebsd.dk> Message-ID: <4BFD2CED.7040902@progis.de> # varnishstat -V varnishstat (varnish-2.0.5) Copyright (c) 2006-2009 Linpro AS / Verdens Gang AS # varnishstat -1 -f n_object,n_objecthead n_object 441847 . N struct object n_objecthead 203761 . N struct objecthead # ps aux | grep varnish root 9418 0.0 0.0 110724 744 ? Ss May11 0:10 /usr/sbin/varnishd -P /var/run/varnishd.pid -a :80 -f /etc/varnish/prod/master.vcl -T 127.0.0.1:6082 -t 120 -w 200,4000,120 -s malloc,10G nobody 9419 5.7 83.4 20961460 10289712 ? Sl May11 1265:05 /usr/sbin/varnishd -P /var/run/varnishd.pid -a :80 -f /etc/varnish/prod/master.vcl -T 127.0.0.1:6082 -t 120 -w 200,4000,120 -s malloc,10G ------------------------------------- # varnishstat -V varnishstat (varnish-2.1.1 SVN ) Copyright (c) 2006-2009 Linpro AS / Verdens Gang AS # varnishstat -1 -f n_object,n_objecthead n_object 594432 . N struct object n_objecthead 364887 . N struct objecthead # ps aux | grep varnish root 1500 0.0 0.0 112948 1212 ? Ss May06 0:02 /usr/sbin/varnishd -P /var/run/varnishd.pid -a :80 -f /etc/varnish/prod/master.vcl -T 127.0.0.1:6082 -t 120 -w 200,4000,120 -s malloc,9G nobody 1501 21.1 91.2 23466228 11252380 ? Sl May06 6098:29 /usr/sbin/varnishd -P /var/run/varnishd.pid -a :80 -f /etc/varnish/prod/master.vcl -T 127.0.0.1:6082 -t 120 -w 200,4000,120 -s malloc,9G Angie T. Muhammad wrote: > # ps aux | grep varnish > varnish 3772 0.1 1.6 21146720 411640 ? Sl May08 42:28 > /usr/sbin/varnishd -P /var/run/varnish.pid -a :80 -T localhost:6082 -f > /etc/varnish/default.vcl -u varnish -g varnish -p thread_pools 4 -p > thread_pool_min 500 -p thread_pool_max 4000 -p thread_pool_add_delay 2 > -p listen_depth 4096 -p lru_interval 60 -p session_linger 100 -h > classic,169313 -p sess_workspace 262144 -p log_hashstring off -p > sess_timeout 10 -p shm_workspace 32768 -p connect_timeout 600 -p > max_restarts 6 -s malloc,2G > > # varnishstat -1 -f n_object,n_objecthead > n_object 9596 . N struct object > n_objecthead 8216 . N struct objecthead > > # rpm -qa | grep varnish > varnish-libs-2.1.2-1 > varnish-2.1.2-1 > > serving around 70,000 requests / day with a peak of 12 req / sec > > > On Wed, May 26, 2010 at 12:22 PM, Angie T. Muhammad > > wrote: > > Adun, may I ask about apache's configuration running behind this > varnish installation and HW specs too if you don't mind ? > Thank you :) > > On Wed, May 26, 2010 at 1:18 AM, Audun Ytterdal > > wrote: > > On Tue, May 25, 2010 at 2:28 PM, Poul-Henning Kamp > > wrote: > > > > I may have "misunderestimated" the dynamics of how objects > live in > > Varnish caches and thus taken a wrong optimization decision with > > the critbit hasher. > > > > I am looking for data to figure this out, specifically the > values of > > the two varnishstat counters: > > > > "N struct objecthead." > > and > > "N struct object" > > > > Along with information about which version of varnish, and > which, if > > any -h argument you use. > > One varnish doing about 4320 million req/day, peaks at 8K reqs/s > > [root at dexter ~]# rpm -qa |grep varnish > varnish-2.1.0-2.vg > varnish-libs-2.1.0-2.vg > > [root at dexter ~]# ps axu |grep varnish > varnish 1239 54.8 74.1 14895928 4526520 ? Sl May24 1053:21 > /usr/sbin/varnishd -P /var/run/varnish.pid -a :80 -f > /etc/varnish/vg.vcl -T 127.0.0.1:82 -t > 600 -u varnish -g varnish -p > thread_pool_add_delay 5 -p thread_pool_timeout 120 -p > thread_pools 4 > -w 200,3000,120 -s malloc,5G > > [root at dexter ~]# varnishstat -1 -f n_object,n_objecthead > n_object 129723 . N struct object > n_objecthead 163562 . N struct objecthead > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > > http://lists.varnish-cache.org/mailman/listinfo/varnish-misc > > > > > -- > All the best, > Angie > > > > > -- > All the best, > Angie > ------------------------------------------------------------------------ > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://lists.varnish-cache.org/mailman/listinfo/varnish-misc -------------- next part -------------- An HTML attachment was scrubbed... URL: From angie.tawfik at gmail.com Thu May 27 08:24:30 2010 From: angie.tawfik at gmail.com (Angie T. Muhammad) Date: Thu, 27 May 2010 11:24:30 +0300 Subject: Does varnish support URL UTF-8 encoding? Message-ID: Hello Varnish people I wonder does Varnish support URL utf-8 encoding? Non-ASCII urls (Arabic ones in my case) are not understood by Varnish 2.1.2. To get an idea of what I mean: 1- Try http://www.hypergurl.com/urlencode.html 2- Type non-ASCII characters (Arabic, Chinese, Hebrew, any non English language) 3- Choose UTF-8 character encoding 4- Generate the encoded URL When trying either plain Arabic (non ASCII) or encoded UTF-8 URL in default.vcl: A) Plain Arabic URL > Varnish ignores the rule !!! B) UTF-8 encoded URL > Varnish fails to start up !!! I am so sad because many people around the globe uses Varnish, yet it doesn't support URLs in their own languages :( -- All the best, Angie From audun at ytterdal.net Thu May 27 09:12:11 2010 From: audun at ytterdal.net (Audun Ytterdal) Date: Thu, 27 May 2010 11:12:11 +0200 Subject: Requesting real-world stats, please In-Reply-To: References: <57685.1274790521@critter.freebsd.dk> Message-ID: Most people have directors with several identical backends. I've got it the other way around: Several standalone apache, lighty and nginx bokses serving different stuff according to host and url. This particular varnishbox is a HP Proliant BL460c G1 2XDualcore 2.66 Ghz with 6 GB of RAM. Running Centos5.3. Only tuning is /var/lib/varnish on tmpfs The most active backend is a vmware-image running apache and 2GB of RAM. ServerLimit 256 MaxClients 256 MaxRequestsPerChild 4000 Serving approx 25-50 hits/sec On Wed, May 26, 2010 at 11:22 AM, Angie T. Muhammad wrote: > Adun, may I ask about apache's configuration running behind this varnish > installation and HW specs too if you don't mind ? > Thank you :) > > On Wed, May 26, 2010 at 1:18 AM, Audun Ytterdal wrote: >> >> On Tue, May 25, 2010 at 2:28 PM, Poul-Henning Kamp >> wrote: >> > >> > I may have "misunderestimated" the dynamics of how objects live in >> > Varnish caches and thus taken a wrong optimization decision with >> > the critbit hasher. >> > >> > I am looking for data to figure this out, specifically the values of >> > the two varnishstat counters: >> > >> > ? ? ? ?"N struct objecthead." >> > and >> > ? ? ? ?"N struct object" >> > >> > Along with information about which version of varnish, and which, if >> > any -h argument you use. >> >> One varnish doing about 4320 million req/day, peaks at 8K reqs/s >> >> [root at dexter ~]# rpm -qa |grep varnish >> varnish-2.1.0-2.vg >> varnish-libs-2.1.0-2.vg >> >> [root at dexter ~]# ps axu |grep varnish >> varnish ? 1239 54.8 74.1 14895928 4526520 ? ? ?Sl ? May24 1053:21 >> /usr/sbin/varnishd -P /var/run/varnish.pid -a :80 -f >> /etc/varnish/vg.vcl -T 127.0.0.1:82 -t 600 -u varnish -g varnish -p >> thread_pool_add_delay 5 -p thread_pool_timeout 120 -p thread_pools 4 >> -w 200,3000,120 -s malloc,5G >> >> [root at dexter ~]# varnishstat -1 -f n_object,n_objecthead >> n_object ? ? ? ? ? ? ? 129723 ? ? ? ? ?. ? N struct object >> n_objecthead ? ? ? ? ? 163562 ? ? ? ? ?. ? N struct objecthead >> -- Audun Ytterdal http://audun.ytterdal.net From angie.tawfik at gmail.com Thu May 27 09:32:09 2010 From: angie.tawfik at gmail.com (Angie T. Muhammad) Date: Thu, 27 May 2010 12:32:09 +0300 Subject: Requesting real-world stats, please In-Reply-To: References: <57685.1274790521@critter.freebsd.dk> Message-ID: Thank you so much Audun :) .. I am experimenting with different apache configs and different varnish thread configs to get the best performance. My page load time is around 7 seconds (home page) which is way too slow compared to HW and network capabilities of the server. Thanks again On 5/27/10, Audun Ytterdal wrote: > Most people have directors with several identical backends. I've got > it the other way around: Several standalone apache, lighty and nginx > bokses serving different stuff according to host and url. > > This particular varnishbox is a HP Proliant BL460c G1 2XDualcore 2.66 > Ghz with 6 GB of RAM. > Running Centos5.3. Only tuning is /var/lib/varnish on tmpfs > > The most active backend is a vmware-image running apache and 2GB of RAM. > ServerLimit 256 > MaxClients 256 > MaxRequestsPerChild 4000 > > Serving approx 25-50 hits/sec > > On Wed, May 26, 2010 at 11:22 AM, Angie T. Muhammad > wrote: >> Adun, may I ask about apache's configuration running behind this varnish >> installation and HW specs too if you don't mind ? >> Thank you :) >> >> On Wed, May 26, 2010 at 1:18 AM, Audun Ytterdal >> wrote: >>> >>> On Tue, May 25, 2010 at 2:28 PM, Poul-Henning Kamp >>> wrote: >>> > >>> > I may have "misunderestimated" the dynamics of how objects live in >>> > Varnish caches and thus taken a wrong optimization decision with >>> > the critbit hasher. >>> > >>> > I am looking for data to figure this out, specifically the values of >>> > the two varnishstat counters: >>> > >>> > ? ? ? ?"N struct objecthead." >>> > and >>> > ? ? ? ?"N struct object" >>> > >>> > Along with information about which version of varnish, and which, if >>> > any -h argument you use. >>> >>> One varnish doing about 4320 million req/day, peaks at 8K reqs/s >>> >>> [root at dexter ~]# rpm -qa |grep varnish >>> varnish-2.1.0-2.vg >>> varnish-libs-2.1.0-2.vg >>> >>> [root at dexter ~]# ps axu |grep varnish >>> varnish ? 1239 54.8 74.1 14895928 4526520 ? ? ?Sl ? May24 1053:21 >>> /usr/sbin/varnishd -P /var/run/varnish.pid -a :80 -f >>> /etc/varnish/vg.vcl -T 127.0.0.1:82 -t 600 -u varnish -g varnish -p >>> thread_pool_add_delay 5 -p thread_pool_timeout 120 -p thread_pools 4 >>> -w 200,3000,120 -s malloc,5G >>> >>> [root at dexter ~]# varnishstat -1 -f n_object,n_objecthead >>> n_object ? ? ? ? ? ? ? 129723 ? ? ? ? ?. ? N struct object >>> n_objecthead ? ? ? ? ? 163562 ? ? ? ? ?. ? N struct objecthead >>> > > -- > Audun Ytterdal > http://audun.ytterdal.net > -- All the best, Angie From alex at acasa.ro Thu May 27 10:15:38 2010 From: alex at acasa.ro (Alex Florescu) Date: Thu, 27 May 2010 13:15:38 +0300 Subject: two varnishd fails Message-ID: <20100527101538.692042F7674@smtp1.acasa.ro> Hello all, I am a varnish-cache user and most of the time everything goes smoothly, but I have encountered 2 serious problems with it. I have two deployments of varnish on different web clusters: - number1 web cluster - number2 web cluster Both have pretty much the same vcl. The first problem was last week with number1 web cluster. /var/log/messages: May 22 11:13:03 kernel: varnishd[21817] general protection rip:3497e00b79 rsp:62ed6970 error:0 May 22 11:13:32 kernel: varnishd[21958] general protection rip:3497e00b79 rsp:721f5970 error:0 May 22 11:14:48 kernel: varnishd[22115] general protection rip:3497e00b79 rsp:42fe2970 error:0 May 22 11:15:21 kernel: varnishd[22476] general protection rip:3497e00b79 rsp:7264e970 error:0 May 22 11:15:32 kernel: varnishd[22575] general protection rip:3497e00b79 rsp:5b135970 error:0 May 22 11:16:06 kernel: varnishd[22611]: segfault at 0000000000000010 rip 0000003497e00b6a rsp 0000000045d40970 error 4 May 22 11:16:21 kernel: varnishd[22746] general protection rip:3497e00b79 rsp:4c5bd970 error:0 May 22 11:16:33 kernel: varnishd[22876] general protection rip:3497e00b79 rsp:5b6ad970 error:0 May 22 11:16:39 kernel: varnishd[22889] general protection rip:3497e00b79 rsp:485a5970 error:0 May 22 11:16:51 kernel: varnishd[22969] general protection rip:3497e00b79 rsp:5d4c9970 error:0 When trying to access any website hosted on number1 cluster, I would get Connection refused. Based on what the log says, it seems that I get these protection faults everyday. The second problem was also on the same day, with number2 web cluster. /var/log/messages: May 22 16:17:16 varnishd[2451]: Child (25232) not responding to ping, killing it. May 22 16:17:18 varnishd[2451]: Child (25232) not responding to ping, killing it. May 22 16:17:18 varnishd[2451]: Child (25232) died signal=3 May 22 16:17:18 varnishd[2451]: child (22499) Started May 22 16:17:23 varnishd[2451]: Pushing vcls failed: CLI communication error May 22 16:17:30 varnishd[2451]: Child (22499) said Closed fds: 4 5 6 11 12 14 15 May 22 16:17:30 varnishd[2451]: Child (22499) said Child starts May 22 16:17:34 varnishd[2451]: Child (22499) said managed to mmap 2147483648 bytes of 2147483648 May 22 16:17:34 varnishd[2451]: Child (22499) said Ready May 22 16:17:36 varnishd[2451]: Child (22499) ended Here also I got Connection refused. What I noticed in the second log was that the hours were stuck somehow. It failed at exactly 19:18, and I restarted varnish around at 19:24, but the logs matching the restart were also "in-the-past". May 22 16:23:05 varnishd[2451]: Manager got SIGINT May 22 16:23:07 varnishd[23228]: child (23229) Started May 22 16:23:07 varnishd[23228]: Child (23229) said Closed fds: 4 5 6 10 11 13 14 May 22 16:23:07 varnishd[23228]: Child (23229) said Child starts May 22 16:23:07 varnishd[23228]: Child (23229) said managed to mmap 2147483648 bytes of 2147483648 May 22 16:23:07 varnishd[23228]: Child (23229) said Ready Everything else was logged at the correct time (I am using ntp), so I am dismissing the fact that the server was not in sync. Also, it is very strange that two completely unrelated varnish clusters would fail in the same day, at 9 hours distance. Using CentOs 5.4, 2.6.18-164.15.1.el5 and varnishd (varnish-2.0.5) on both machines. Any help on debugging these problems is kindly appreciated! From andrea.campi at zephirworks.com Fri May 28 13:15:23 2010 From: andrea.campi at zephirworks.com (andrea.campi at zephirworks.com) Date: Fri, 28 May 2010 15:15:23 +0200 Subject: Ignoring refresh / force-refresh from the client Message-ID: <20100528151523.203559uowjszt88w@webmail.zephirmail.net> Hi guys, I'm working with a client who has an emergency (a backend webapp misbehaving because of sluggishness under load), so I deployed varnish in front to relieve the load. The input from the customer is that the underlaying data doesn't change more often that 4 hours, so it would be good to be able to not send requests for a given URL more often than that. The basic setup was a piece of cake, but I'd like to also "lie" to users that hit refresh (and even forced refresh, if possible) and still serve the request from the cache, as long as the page is fresh. However in my tests a refresh from e.g. Firefox (with a "Cache-Control: max-age=0") causes the URL to be purged from the cache and fetched from the backend. AFAICT obj.cacheable is still true, so I'm not sure why is that happening. I've tried unsetting/removing req.http.cache-control and had no success. I've also tried sending Age: 0, also no success. Any suggestion, or pointer to documentation/blog posts/source code/whatever? In case it matters: the backend is sending an Expires header 4 hours in the future, no ETag. I'm also sending a max-age. Thanks in advance, bye, Andrea ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. From r at roze.lv Fri May 28 14:37:16 2010 From: r at roze.lv (Reinis Rozitis) Date: Fri, 28 May 2010 17:37:16 +0300 Subject: Ignoring refresh / force-refresh from the client References: <20100528151523.203559uowjszt88w@webmail.zephirmail.net> Message-ID: <101117E663A44C6F81870F8B7CB806FF@KD5> To ignore clients headers regarding cache this should be enough imo: sub vcl_recv { unset req.http.Cache-Control; } You can also override the backends cache settings. For example because of some performance concerns (in case of broken image/resource the backend wont be hammered by the client againt) we do something like this (in case the backend server "decides" to return something with less than 10 minutes cache timeout): sub vcl_fetch { if (beresp.ttl < 10m) { set beresp.ttl = 10m; } return (deliver); } rr ----- Original Message ----- From: To: Sent: Friday, May 28, 2010 4:15 PM Subject: Ignoring refresh / force-refresh from the client > Hi guys, > > I'm working with a client who has an emergency (a backend webapp misbehaving because of sluggishness under load), so I deployed > varnish in front to relieve the load. > > The input from the customer is that the underlaying data doesn't change more often that 4 hours, so it would be good to be able > to not send requests for a given URL more often than that. > > The basic setup was a piece of cake, but I'd like to also "lie" to users that hit refresh (and even forced refresh, if possible) > and still serve the request from the cache, as long as the page is fresh. > > However in my tests a refresh from e.g. Firefox (with a "Cache-Control: max-age=0") causes the URL to be purged from the cache > and fetched from the backend. > AFAICT obj.cacheable is still true, so I'm not sure why is that happening. > > I've tried unsetting/removing req.http.cache-control and had no success. > I've also tried sending Age: 0, also no success. > > Any suggestion, or pointer to documentation/blog posts/source code/whatever? > > In case it matters: the backend is sending an Expires header 4 hours in the future, no ETag. I'm also sending a max-age. > > Thanks in advance, bye, > Andrea > > ---------------------------------------------------------------- > This message was sent using IMP, the Internet Messaging Program. > > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://lists.varnish-cache.org/mailman/listinfo/varnish-misc > From andrea.campi at zephirworks.com Fri May 28 15:54:41 2010 From: andrea.campi at zephirworks.com (andrea.campi at zephirworks.com) Date: Fri, 28 May 2010 17:54:41 +0200 Subject: Ignoring refresh / force-refresh from the client In-Reply-To: <101117E663A44C6F81870F8B7CB806FF@KD5> References: <20100528151523.203559uowjszt88w@webmail.zephirmail.net> <101117E663A44C6F81870F8B7CB806FF@KD5> Message-ID: <20100528175441.11086cl7hhen1g4c@webmail.zephirmail.net> Reinis, Quoting Reinis Rozitis : > To ignore clients headers regarding cache this should be enough imo: > > sub vcl_recv { > unset req.http.Cache-Control; > } that's what I thought, and I have that as the first line in vcl_recv (more stuff after that). I still see this in the logs: 14 SessionOpen c 10.18.7.7 57244 0.0.0.0:80 14 ReqStart c 10.18.7.7 57244 865712614 14 RxRequest c GET 14 RxURL c /jimi-hendrix.html 14 RxProtocol c HTTP/1.1 14 RxHeader c Host: argomenti.ilsole24ore.com 14 RxHeader c User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_3; en-us) AppleWebKit/531.22.7 (KHTML, like Gecko) Version/4.0.5 Safari/531.22.7 14 RxHeader c Cache-Control: max-age=0 14 RxHeader c Accept: application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 14 RxHeader c Accept-Language: en-us 14 RxHeader c Accept-Encoding: gzip, deflate 14 RxHeader c Cookie: __qca=P0-697534472-1274625433878; base_domain_b16f823282571770f76c41b0015dfd67=ilsole24ore.com; pd_nw=4-275; s_cc=true; s_cm_NW=undefinedlocalhost%3A8080localhost%3A8080; s_lastvisit=1275061830533; s_nr=1275061830532-Repeat; s_sq=%5B%5BB%5D%5D; s_ 14 RxHeader c Connection: keep-alive 14 VCL_call c recv 14 VCL_return c lookup 14 VCL_call c hash 14 VCL_return c hash 14 VCL_call c discard 14 VCL_return c discard 0 ExpKill - 865712580 LRU 14 VCL_call c miss 14 VCL_return c fetch 14 Backend c 13 cq cq Any idea? Andrea ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. From scaunter at topscms.com Fri May 28 19:32:36 2010 From: scaunter at topscms.com (Caunter, Stefan) Date: Fri, 28 May 2010 15:32:36 -0400 Subject: varnish too fast for Captcha image with IIS backend Message-ID: <064FF286FD17EC418BFB74629578BED11975D763@tmg-mail4.torstar.net> Greetings: We have an email article pop-up with a captcha that works, when hit directly on a relatively slow IIS7 backend, since it has "time" to generate and get served as part of the pop-up. I observe that when we accelerate with varnish, the pop-up loads too quickly, and the captcha image is not built in time, so varnish serves a 404 to the client, and the image is not there. Upon refresh it has been built and gets served. I am using client director and pipe (necessarily) to ensure we hit the same backend for the correct captcha and corresponding transactions. Is an individual 404 something to which we can introduce a small delay, or a restart? Can we potentially build a delay into vcl for the individual captcha requests to compensate for this slowness since these are not cacheable? Has anyone implemented a "patience" subroutine? Stefan Caunter :: Senior Systems Administrator :: TOPS e: scaunter at topscms.com :: m: (416) 561-4871 www.thestar.com www.topscms.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From perbu at varnish-software.com Sat May 29 07:04:04 2010 From: perbu at varnish-software.com (Per Buer) Date: Sat, 29 May 2010 09:04:04 +0200 Subject: Ignoring refresh / force-refresh from the client In-Reply-To: <20100528175441.11086cl7hhen1g4c@webmail.zephirmail.net> References: <20100528151523.203559uowjszt88w@webmail.zephirmail.net> <101117E663A44C6F81870F8B7CB806FF@KD5> <20100528175441.11086cl7hhen1g4c@webmail.zephirmail.net> Message-ID: On Fri, May 28, 2010 at 5:54 PM, wrote: > > I still see this in the logs: > .. I see a cookie header. Varnish won't cache if your client sends cookie headers. I've tried to write something on the topic in the upcoming tutorial; http://varnish-cache.org/static/perbu/dirhtml/tutorial/increasing_your_hitrate/#cookies -- Per Buer, Varnish Software Phone: +47 21 54 41 21 / Mobile: +47 958 39 117 / skype: per.buer From andrea.campi at zephirworks.com Sat May 29 09:13:53 2010 From: andrea.campi at zephirworks.com (Andrea Campi) Date: Sat, 29 May 2010 11:13:53 +0200 Subject: Ignoring refresh / force-refresh from the client In-Reply-To: References: <20100528151523.203559uowjszt88w@webmail.zephirmail.net> <101117E663A44C6F81870F8B7CB806FF@KD5> <20100528175441.11086cl7hhen1g4c@webmail.zephirmail.net> Message-ID: <4C00DAD1.6030804@zephirworks.com> Per Buer wrote: > On Fri, May 28, 2010 at 5:54 PM, wrote: > >> I still see this in the logs: >> .. >> > > I see a cookie header. Varnish won't cache if your client sends cookie > headers. I've tried to write something on the topic in the upcoming > tutorial; http://varnish-cache.org/static/perbu/dirhtml/tutorial/increasing_your_hitrate/#cookies > > I do unset cookies. BTW sorry for sending only excerpts of the config file, I should make time to sanitize it from anything confidential and send it entirely. That said, AFAIU if the cookie was the matter, the cache would be bypassed completely, wouldn't it? Instead, from the log I see a lookup followed by a discard, even though I don't have anything like that in the config: 14 VCL_call c recv 14 VCL_return c lookup 14 VCL_call c hash 14 VCL_return c hash 14 VCL_call c discard 14 VCL_return c discard 0 ExpKill - 865712580 LRU Moreover, as long as I don't hit refresh in the browser the page is served from the cache--I confirmed that by adding custom headers. What is weirdest of all is that if I send the exact same headers with curl, I get a cache hit. Anyway, no point in wasting your time without the full config (and I need to be back on site for that). Have a nice weekend, Andrea -------------- next part -------------- An HTML attachment was scrubbed... URL: From bedis9 at gmail.com Sat May 29 10:19:16 2010 From: bedis9 at gmail.com (Bedis 9) Date: Sat, 29 May 2010 12:19:16 +0200 Subject: varnish too fast for Captcha image with IIS backend In-Reply-To: <064FF286FD17EC418BFB74629578BED11975D763@tmg-mail4.torstar.net> References: <064FF286FD17EC418BFB74629578BED11975D763@tmg-mail4.torstar.net> Message-ID: Hi, You can try to force a restart of the path of the captcha if the beresp.status is 404. But might be not slow enough... Maybe adding a small INLINE C loop which waits for a few milliseconds would help... cheers On Fri, May 28, 2010 at 9:32 PM, Caunter, Stefan wrote: > Greetings: > > > > We have an email article pop-up with a captcha that works, when hit directly > on a relatively slow IIS7 backend, since it has ?time? to generate and get > served as part of the pop-up. > > > > I observe that when we accelerate with varnish, the pop-up loads too > quickly, and the captcha image is not built in time, so varnish serves a 404 > to the client, and the image is not there. Upon refresh it has been built > and gets served. I am using client director and pipe (necessarily) to ensure > we hit the same backend for the correct captcha and corresponding > transactions. > > > > Is an individual 404 something to which we can introduce a small delay, or a > restart? > > > > Can we potentially build a delay into vcl for the individual captcha > requests to compensate for this slowness since these are not cacheable? Has > anyone implemented a ?patience? subroutine? > > > > Stefan Caunter :: Senior Systems Administrator :: TOPS > > e: scaunter at topscms.com? ::? m: (416) 561-4871 > > www.thestar.com www.topscms.com > > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > http://lists.varnish-cache.org/mailman/listinfo/varnish-misc > From ask at develooper.com Sun May 30 03:41:30 2010 From: ask at develooper.com (=?iso-8859-1?Q?Ask_Bj=F8rn_Hansen?=) Date: Sat, 29 May 2010 20:41:30 -0700 Subject: varnish too fast for Captcha image with IIS backend In-Reply-To: <064FF286FD17EC418BFB74629578BED11975D763@tmg-mail4.torstar.net> References: <064FF286FD17EC418BFB74629578BED11975D763@tmg-mail4.torstar.net> Message-ID: On May 28, 2010, at 12:32, Caunter, Stefan wrote: > Can we potentially build a delay into vcl for the individual captcha requests to compensate for this slowness since these are not cacheable? Has anyone implemented a ?patience? subroutine? Just make the backend build the captcha image on demand (rather than create it in one place and serve it from another, as it sounds like is what you are doing). - ask From tfheen at varnish-software.com Mon May 31 06:13:45 2010 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Mon, 31 May 2010 08:13:45 +0200 Subject: Parsing non-English URLs In-Reply-To: (Angie T. Muhammad's message of "Tue, 25 May 2010 13:08:26 +0300") References: Message-ID: <877hmk61k6.fsf@qurzaw.linpro.no> ]] "Angie T. Muhammad" | I tried opening the log file with less, vim, and tail but all what am | getting is either binary (less) or stuff like above (tail). Use varnishlog(1) to read the varnishlog. tail and less won't do the right thing as the log file is a circular buffer and they have no idea where varnish is currently writing. -- Tollef Fog Heen Varnish Software t: +47 21 54 41 73 From tfheen at varnish-software.com Mon May 31 06:16:28 2010 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Mon, 31 May 2010 08:16:28 +0200 Subject: Requesting real-world stats, please In-Reply-To: (Cosimo Streppone's message of "Tue, 25 May 2010 17:42:56 +0200") References: <57685.1274790521@critter.freebsd.dk> Message-ID: <8739x861fn.fsf@qurzaw.linpro.no> ]] "Cosimo Streppone" | n_object 100585 . N struct object | n_objecthead 103700 . N struct objecthead | | n_object 102270 . N struct object | n_objecthead 102294 . N struct objecthead | | Maybe I should use -h with a higher number? :) If anything, with ~100k objects, you should choose a smaller number. The rule of thumb is the hash size should be about 10% of the number of objects, so a prime number around 10k would be about right here. Not that I think it matters that much, you waste a bit of memory, nothing more. -- Tollef Fog Heen Varnish Software t: +47 21 54 41 73 From tfheen at varnish-software.com Mon May 31 06:26:03 2010 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Mon, 31 May 2010 08:26:03 +0200 Subject: Ignoring refresh / force-refresh from the client In-Reply-To: <20100528175441.11086cl7hhen1g4c@webmail.zephirmail.net> (andrea campi's message of "Fri, 28 May 2010 17:54:41 +0200") References: <20100528151523.203559uowjszt88w@webmail.zephirmail.net> <101117E663A44C6F81870F8B7CB806FF@KD5> <20100528175441.11086cl7hhen1g4c@webmail.zephirmail.net> Message-ID: <87y6f04mf8.fsf@qurzaw.linpro.no> ]] | Reinis, | | Quoting Reinis Rozitis : | | > To ignore clients headers regarding cache this should be enough imo: | > | > sub vcl_recv { | > unset req.http.Cache-Control; | > } | | that's what I thought, and I have that as the first line in vcl_recv | (more stuff after that). We don't pay attention to cache-control from clients, so unless you do that in your VCL, that's not what's going on. | 14 RxHeader c Connection: keep-alive | 14 VCL_call c recv | 14 VCL_return c lookup | 14 VCL_call c hash | 14 VCL_return c hash | 14 VCL_call c discard | 14 VCL_return c discard | 0 ExpKill - 865712580 LRU | 14 VCL_call c miss | 14 VCL_return c fetch | 14 Backend c 13 cq cq Looks like a completely normal cache miss with a full cache where the last item on the LRU list is kicked out to make space for mr. Hendrix. -- Tollef Fog Heen Varnish Software t: +47 21 54 41 73 From andrea.campi at zephirworks.com Mon May 31 07:18:26 2010 From: andrea.campi at zephirworks.com (Andrea Campi) Date: Mon, 31 May 2010 09:18:26 +0200 Subject: Ignoring refresh / force-refresh from the client In-Reply-To: <87y6f04mf8.fsf@qurzaw.linpro.no> References: <20100528151523.203559uowjszt88w@webmail.zephirmail.net> <101117E663A44C6F81870F8B7CB806FF@KD5> <20100528175441.11086cl7hhen1g4c@webmail.zephirmail.net> <87y6f04mf8.fsf@qurzaw.linpro.no> Message-ID: <4C0362C2.6060107@zephirworks.com> Tollef Fog Heen wrote: > | 14 RxHeader c Connection: keep-alive > | 14 VCL_call c recv > | 14 VCL_return c lookup > | 14 VCL_call c hash > | 14 VCL_return c hash > | 14 VCL_call c discard > | 14 VCL_return c discard > | 0 ExpKill - 865712580 LRU > | 14 VCL_call c miss > | 14 VCL_return c fetch > | 14 Backend c 13 cq cq > > Looks like a completely normal cache miss with a full cache where the > last item on the LRU list is kicked out to make space for mr. Hendrix. > Makes sense. I will double check, and make sure I have the full config available before asking again :) Andrea From alex.florescu at acasa.ro Sat May 22 16:49:56 2010 From: alex.florescu at acasa.ro (Alex Florescu) Date: Sat, 22 May 2010 16:49:56 -0000 Subject: two varnishd fails Message-ID: <4BF80B26.2070902@acasa.ro> Hello all, I am a varnish-cache user and most of the time everything goes smoothly, but today I have encountered 2 problems with it. I have two deployments of varnish, on different web clusters: - number1 web cluster - number2 web cluster Both have pretty much the same vcl. The first problem was today with number1 web cluster. /var/log/messages: May 22 11:13:03 kernel: varnishd[21817] general protection rip:3497e00b79 rsp:62ed6970 error:0 May 22 11:13:32 kernel: varnishd[21958] general protection rip:3497e00b79 rsp:721f5970 error:0 May 22 11:14:48 kernel: varnishd[22115] general protection rip:3497e00b79 rsp:42fe2970 error:0 May 22 11:15:21 kernel: varnishd[22476] general protection rip:3497e00b79 rsp:7264e970 error:0 May 22 11:15:32 kernel: varnishd[22575] general protection rip:3497e00b79 rsp:5b135970 error:0 May 22 11:16:06 kernel: varnishd[22611]: segfault at 0000000000000010 rip 0000003497e00b6a rsp 0000000045d40970 error 4 May 22 11:16:21 kernel: varnishd[22746] general protection rip:3497e00b79 rsp:4c5bd970 error:0 May 22 11:16:33 kernel: varnishd[22876] general protection rip:3497e00b79 rsp:5b6ad970 error:0 May 22 11:16:39 kernel: varnishd[22889] general protection rip:3497e00b79 rsp:485a5970 error:0 May 22 11:16:51 kernel: varnishd[22969] general protection rip:3497e00b79 rsp:5d4c9970 error:0 When trying to access any website hosted on number1 cluster, I would get Connection refused. The second problem was also today, with number2 web cluster. /var/log/messages: May 22 16:17:16 varnishd[2451]: Child (25232) not responding to ping, killing it. May 22 16:17:18 varnishd[2451]: Child (25232) not responding to ping, killing it. May 22 16:17:18 varnishd[2451]: Child (25232) died signal=3 May 22 16:17:18 varnishd[2451]: child (22499) Started May 22 16:17:23 varnishd[2451]: Pushing vcls failed: CLI communication error May 22 16:17:30 varnishd[2451]: Child (22499) said Closed fds: 4 5 6 11 12 14 15 May 22 16:17:30 varnishd[2451]: Child (22499) said Child starts May 22 16:17:34 varnishd[2451]: Child (22499) said managed to mmap 2147483648 bytes of 2147483648 May 22 16:17:34 varnishd[2451]: Child (22499) said Ready May 22 16:17:36 varnishd[2451]: Child (22499) ended Here also I got Connection refused. What I noticed in the second log was that the hours were stuck somehow. It failed at exactly 19:18, and I restarted varnish around at 19:24, but the logs matching the restart were also "in-the-past". May 22 16:23:05 varnishd[2451]: Manager got SIGINT May 22 16:23:07 varnishd[23228]: child (23229) Started May 22 16:23:07 varnishd[23228]: Child (23229) said Closed fds: 4 5 6 10 11 13 14 May 22 16:23:07 varnishd[23228]: Child (23229) said Child starts May 22 16:23:07 varnishd[23228]: Child (23229) said managed to mmap 2147483648 bytes of 2147483648 May 22 16:23:07 varnishd[23228]: Child (23229) said Ready Everything else was logged at the correct hours (I am using ntp), so I am dismissing the fact that the server's time was not synched. Also, it is very strange that two completely unrelated varnish clusters would fail in the same day, at 9 hours distance. Using CentOs 5.4, 2.6.18-164.15.1.el5 and varnishd (varnish-2.0.5) on both machines. Any help on debugging these problems is kindly appreciated! From xose.vazquez at gmail.com Wed May 26 14:12:58 2010 From: xose.vazquez at gmail.com (Xose Vazquez Perez) Date: Wed, 26 May 2010 14:12:58 -0000 Subject: jemalloc in varnish Message-ID: hi, There are newer versions of jemalloc at: http://www.canonware.com/jemalloc/ -thanks- -- ?All? muevan feroz guerra, ciegos reyes por un palmo m?s de tierra; que yo aqu? tengo por m?o cuanto abarca el mar brav?o, a quien nadie impuso leyes. Y no hay playa, sea cualquiera, ni bandera de esplendor, que no sienta mi derecho y d? pecho a mi valor.? From vburshteyn at broadway.com Sat May 29 03:27:55 2010 From: vburshteyn at broadway.com (Vitaly Burshteyn) Date: Sat, 29 May 2010 03:27:55 -0000 Subject: not serving from Cache Message-ID: <0F69574F9901D4459C6B75C9FF64FBC404B4D962@mxfl01.hollywoodmedia.corp> Hi folks, thanks in advance. So I have varnish running on 2.1.1 with the load balance feature that works great. Everything works, exept that if I shut down the backend I get 503 erros. Aka nothing is served from cache.. Any ideas as to what I am going wrong in my vlc file? Thanks, backend server1 { .host = ""; } backend server2 { .host = ""; } director bw round-robin { { .backend = server1; } { .backend = server2; }} sub vcl_recv { set req.backend = bw; if (req.url ~ "/") { unset req.http.cookie; } if (req.url ~ "\.(png|gif|jpg|swf|css|js)$") { return(lookup); } # Redirect requests to /admin/ to /oops/ if (req.url ~ "^/admin/"){ set req.url = regsub(req.url, "^/admin/", "/oops/"); } # Never cache this path if (req.url ~ "/affiliates/visa-signature/") { return(pass); } if (req.url ~ "^/r/\d+/") { return(pass); } # Don't cache if specific HTTP headers exist if (req.http.Expect) { return(pass); } if (req.http.Authorization) { return(pass); } set req.grace = 60m; } sub vcl_fetch { set beresp.ttl = 120s; set beresp.grace = 200s; if ( beresp.status == 500) { set beresp.grace = 60s; restart; } if (req.url ~ "/") { unset beresp.http.set-cookie; } if (req.url ~ "\.(png|gif|jpg|swf|css|js)$") { unset beresp.http.set-cookie; } # Don't cache error pages if (beresp.status == 503) { return(pass); } # Varnish will dutifully serve different objects for different values # of "Cookie: foo" if "Vary:" contains "Cookie". We don't want this. if (beresp.http.Vary == "Cookie") { remove beresp.http.Vary; } # Anti-dogpiling feature - serve a stale request until a new one # can be fetched, for up to 30s if (!beresp.cacheable) { return(deliver); } return(deliver); } sub vcl_hit { remove obj.http.Set-Cookie; } ____________________________________ The information contained in this transmission may contain privileged and confidential information. It is intended only for the use of the person(s) named above. If you are not the intended recipient, you are hereby notified that any review, dissemination, distribution or duplication of this communication is strictly prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. -------------- next part -------------- An HTML attachment was scrubbed... URL: