From h.paulissen at qbell.nl Thu Oct 1 07:31:16 2009 From: h.paulissen at qbell.nl (Henry Paulissen) Date: Thu, 1 Oct 2009 09:31:16 +0200 Subject: SMA outstanding allocations Message-ID: <003201ca4269$297408b0$7c5c1a10$@paulissen@qbell.nl> I would like to have some info about SMA outstanding allocations. What is the meaning of it? What does it mean if that number is high / increasing with the second without decreasing over time? What are the configuration options regarding to this item? My guess is that it is how many objects there are in a temporary table (between fetch and the lru), in waiting state to be written to the lru. If this is true: What does it mean? Is it that my lru is locked most of the time and therefore cant be written? Does the maximum storage option (-s malloc,5G) also affects this storage or isn't this storage checked for size? Does the cleanup processes (duplicate content check, remove expired content, etc) also check this list? Regards, Henry Paulissen From phk at phk.freebsd.dk Thu Oct 1 12:56:11 2009 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Thu, 01 Oct 2009 12:56:11 +0000 Subject: SMA outstanding allocations In-Reply-To: Your message of "Thu, 01 Oct 2009 09:31:16 +0200." <003201ca4269$297408b0$7c5c1a10$@paulissen@qbell.nl> Message-ID: <87164.1254401771@critter.freebsd.dk> In message <003201ca4269$297408b0$7c5c1a10$@paulissen at qbell.nl>, "Henry Pauliss en" writes: >I would like to have some info about SMA outstanding allocations. That is number of allocations made with the -smalloc stevedore. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From h.paulissen at qbell.nl Thu Oct 1 16:41:38 2009 From: h.paulissen at qbell.nl (Henry Paulissen) Date: Thu, 1 Oct 2009 18:41:38 +0200 Subject: SMA outstanding allocations In-Reply-To: <87164.1254401771@critter.freebsd.dk> References: Your message of "Thu, 01 Oct 2009 09:31:16 +0200." <003201ca4269$297408b0$7c5c1a10$@paulissen@qbell.nl> <87164.1254401771@critter.freebsd.dk> Message-ID: <003901ca42b6$0bc3c1a0$234b44e0$@paulissen@qbell.nl> Is this bad when the number is high and increasing every second? -----Oorspronkelijk bericht----- Van: phk at critter.freebsd.dk [mailto:phk at critter.freebsd.dk] Namens Poul-Henning Kamp Verzonden: donderdag 1 oktober 2009 14:56 Aan: Henry Paulissen CC: varnish-misc at projects.linpro.no Onderwerp: Re: SMA outstanding allocations In message <003201ca4269$297408b0$7c5c1a10$@paulissen at qbell.nl>, "Henry Pauliss en" writes: >I would like to have some info about SMA outstanding allocations. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From lists at pauldowman.com Fri Oct 2 20:25:58 2009 From: lists at pauldowman.com (Paul Dowman) Date: Fri, 2 Oct 2009 16:25:58 -0400 Subject: Excluding url's from varnishhist Message-ID: Hi, I'm having trouble figuring out how to exclude certain URL's from varnishhist. I want to exclude static files, e.g. urls that match a pattern like /\.png|\.gif|\.js|\.css|\.ico/ (because these don't cause much load on the back-end, I want to see only the requests that would hit my app servers). I know about the -X regex argument, but it doesn't seem to do what I want. Actually I don't really understand what it does, the man page says that it excludes "log entries" that match a pattern, but as far as I can tell it doesn't match URL's. What's the right way to do this? BTW, it's almost impossible to search the mailing list archives here because it seems to search all lists and my query matches hundreds of messages from the varnish-commit list, so I apologize if this has been answered before. Thanks, Paul -------------- next part -------------- An HTML attachment was scrubbed... URL: From niallo at metaweb.com Fri Oct 2 21:26:08 2009 From: niallo at metaweb.com (Niall O'Higgins) Date: Fri, 2 Oct 2009 14:26:08 -0700 Subject: POST body logging Message-ID: <20091002212608.GD19984@digdug.corp.631h.metaweb.com> Hi all, I'm trying to get Varnish to log POST body contents. It looks like I will want to modify FetchReqBody() in cache_fetch.c to write the data into a new until its complete, then log it. Does this sound like a good approach, or is there a better way? Thanks. -- Niall O'Higgins Software Engineer Metaweb Technologies, Inc. From phk at phk.freebsd.dk Sun Oct 4 07:00:03 2009 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Sun, 04 Oct 2009 07:00:03 +0000 Subject: POST body logging In-Reply-To: Your message of "Fri, 02 Oct 2009 14:26:08 MST." <20091002212608.GD19984@digdug.corp.631h.metaweb.com> Message-ID: <1865.1254639603@critter.freebsd.dk> In message <20091002212608.GD19984 at digdug.corp.631h.metaweb.com>, Niall O'Higgi ns writes: >Hi all, > >I'm trying to get Varnish to log POST body contents. Sorry, we have no code to support that presently. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From cosimo at streppone.it Mon Oct 5 14:20:45 2009 From: cosimo at streppone.it (Cosimo Streppone) Date: Mon, 05 Oct 2009 16:20:45 +0200 Subject: Cookies being sent back with a cached response Message-ID: Hi everyone, thanks for Varnish! I've been trying to understand how to control it (2.0.4) during these last few weeks. :) It seems I'm experiencing an issue with unexpected content being delivered to clients. Full story: I tried to configure varnish to accept only one kind of request for now, where the URL matches "/avatar/12345" where 12345 is the userid. That's a static picture. Actually is a redirect (301) that points to a static picture, so I'm caching the redirect response from the backend. So far, so good. Our frontend apaches only send Varnish this type of request, nothing else. In no whatsoever way these requests can issue cookies (Set-Cookie header), but we've had reports of users getting other people's pictures, but how can that be since URLs are clearly user-based? So, say: /avatar/12345 is avatar of user whose id is 12345 and it can be requested by anyone without being logged-in. Sometimes it seems like varnish serves the wrong content (?). Then looking at varnishlog output, and grepping for "Cookie", I saw this (unrelated?): 90 TxHeader b Cookie: session=bf6ac86feaed6b23bef66a8f44097eac42cf24 ... How can this ever happen? The backends that serve the request never output cookies, and even if they did, it would be "Set-Cookie:" anyway... I'm confused. I also wrote a perl test script that runs against a Varnish instance and reports results back, with expected cookies or cache hit/miss, and that runs and passes all tests. It's only when I put it on production load that this seems to happen... The following is the VCL I'm using. I tried to add some "debug" information in the headers. I'm pretty sure there's something awfully broken, but I have no idea of what it is :) # Add a special header to mark objects as hit/miss sub vcl_deliver { if (obj.hits > 0) { set resp.http.X-Varnish-Status = "hit"; } else { set resp.http.X-Varnish-Status = "miss"; } } sub vcl_recv { set req.backend = mybackends; if (req.request != "GET") { pass; } if (req.url ~ "^/avatar/.+$") { unset req.http.Cookie; unset req.http.Authorization; lookup; } pass; } sub vcl_fetch { set obj.http.X-Varnish-URL = req.url; if (!obj.cacheable) { set obj.http.X-Varnish-Cacheable = "no, not cacheable"; pass; } if (req.http.Cache-Control ~ "no-cache") { set obj.http.X-Varnish-Cacheable = "no, cache-control=no-cache"; purge_url(req.url); pass; } if (req.http.Cache-Control ~ "private") { set obj.http.X-Varnish-Cacheable = "no, cache-control=private"; pass; } if (obj.http.Set-Cookie) { set obj.http.X-Varnish-Cacheable = "no, sets a cookie"; pass; } if (req.request != "GET") { set obj.http.X-Varnish-Cacheable = "no, !GET"; pass; } if (req.url ~ "^/avatar/.+$") { #//// XXX This shouldn't be necessary? unset obj.http.Set-Cookie; set obj.http.X-Varnish-Cacheable = "yes, allowed by URL"; set obj.ttl = 1h; deliver; } # Check for cookie only after always-cache URLs if (req.http.Cookie ~ "session") { set obj.http.X-Varnish-Cacheable = "no, session cookie"; pass; } set obj.http.X-Varnish-Cacheable = "no, URL didn't match"; pass; } Thank you for reading through this, I would appreciate any help from you or hints on how to further debug this. -- Cosimo From rtshilston at gmail.com Mon Oct 5 16:39:12 2009 From: rtshilston at gmail.com (Rob S) Date: Mon, 05 Oct 2009 17:39:12 +0100 Subject: Cookies being sent back with a cached response In-Reply-To: References: Message-ID: <4ACA2130.3000005@gmail.com> Cosimo Streppone wrote: > ... > > Thank you for reading through this, I would > appreciate any help from you or hints on how to further > debug this. > Cosimo, I'd suggest you add the varnish ID in the Apache log files (or similar), then wait for a user to report the problem. Once they've reported it, work out what their user id should be, find their access in the varnish logs, and play back the log file to see the back end request. Hopefully this'll narrow it down. If the user id is available in a cookie, then you could write some javascript to detect the situation that the image URL doesn't match the user id in the cookie, and then log extra information into a logging script. You'll end up kicking yourself once you find the problem.... we certainly have on occasion. Just a few ideas. Rob From cosimo at streppone.it Tue Oct 6 07:35:59 2009 From: cosimo at streppone.it (Cosimo Streppone) Date: Tue, 06 Oct 2009 09:35:59 +0200 Subject: Cookies being sent back with a cached response In-Reply-To: <4ACA2130.3000005@gmail.com> References: <4ACA2130.3000005@gmail.com> Message-ID: Rob wrote: > I'd suggest you add the varnish ID in the Apache log files (or similar), > then wait for a user to report the problem. That's interesting. I didn't think of cross-referencing apache logs. > work out what their user id should be, find their access in the varnish > logs, and play back the log file to see the back end request. Are you referring to the varnishlog output, or the varnishncsa output? What's the common practice? Archiving varnishlog or varnishncsa (or something else) output? > You'll end up kicking yourself once you find the problem.... we > certainly have on occasion. I hope so :) Thanks for your ideas. -- Cosimo From rtshilston at gmail.com Tue Oct 6 10:02:20 2009 From: rtshilston at gmail.com (Rob S) Date: Tue, 06 Oct 2009 11:02:20 +0100 Subject: Cookies being sent back with a cached response In-Reply-To: References: <4ACA2130.3000005@gmail.com> Message-ID: <4ACB15AC.2010901@gmail.com> Cosimo Streppone wrote: > Rob wrote: > >> I'd suggest you add the varnish ID in the Apache log files (or >> similar), then wait for a user to report the problem. > > That's interesting. > I didn't think of cross-referencing apache logs. You can log the varnish request in the Apache logs with something like: LogFormat "%h %l %u %t \"%r\" %s %b \"%{Referer}i\" \"%{User-Agent}i\" %D %{Host}i %A \"%{X-Varnish}i\"" combined > >> work out what their user id should be, find their access in the >> varnish logs, and play back the log file to see the back end request. > > Are you referring to the varnishlog output, or the varnishncsa output? > What's the common practice? Archiving varnishlog or varnishncsa (or > something else) output? I was referring to the varnishlog output. You can replay a request with something like: varnishlog -r /var/log/varnish/varnish.log -c -o TxHeader 264247211 As for storage, all I can tell you is what we do - store both, rotate daily or weekly, and keep historically as appropriate. > >> You'll end up kicking yourself once you find the problem.... we >> certainly have on occasion. > > I hope so :) > Thanks for your ideas. > From thomas at fragstein.de Thu Oct 8 06:44:52 2009 From: thomas at fragstein.de (Thomas Fragstein) Date: Thu, 8 Oct 2009 08:44:52 +0200 Subject: Varnish Listen problem ipv6 and ipv4 Message-ID: <0363C39C-113D-4C59-834D-D4BCE45BD1DA@fragstein.de> Hallo, i have a problem with Varnish 2.0.4 when i will start varnish with "-A :80" then varnish will only Listen on ipv6. have anybody a idee who i can win round varnish? Thomas From tfheen at redpill-linpro.com Thu Oct 8 07:31:28 2009 From: tfheen at redpill-linpro.com (Tollef Fog Heen) Date: Thu, 08 Oct 2009 09:31:28 +0200 Subject: Varnish 503 Service unavailable error In-Reply-To: <75cf5800908130243k3c0e7db6i2d01788a42ab352e@mail.gmail.com> (Paras Fadte's message of "Thu, 13 Aug 2009 15:13:43 +0530") References: <75cf5800908130119o5b10f41bn85e7d76fcb7ebc96@mail.gmail.com> <87fxbw5atk.fsf@qurzaw.linpro.no> <75cf5800908130243k3c0e7db6i2d01788a42ab352e@mail.gmail.com> Message-ID: <87k4z61gvj.fsf@qurzaw.linpro.no> ]] Paras Fadte | Following is the output of varnishstat -1 . Also ,do I have to restart | varnish when I change the "connect_timeout" parameter ? No, no need to restart varnish. | backend_fail 37154 0.59 Backend connections failures As you can see here, you have quite a few backend connection failures. Is your backend overloaded? -- Tollef Fog Heen Redpill Linpro -- Changing the game! t: +47 21 54 41 73 From tfheen at redpill-linpro.com Thu Oct 8 07:49:26 2009 From: tfheen at redpill-linpro.com (Tollef Fog Heen) Date: Thu, 08 Oct 2009 09:49:26 +0200 Subject: varnish redundancy In-Reply-To: (Laurence Rowe's message of "Mon, 14 Sep 2009 17:49:38 +0100") References: <19494.1251975715@critter.freebsd.dk> Message-ID: <87fx9u1g1l.fsf@qurzaw.linpro.no> ]] Laurence Rowe | Is there an advantage in using a director here instead of the following? | | sub vcl_recv { | set req.backend = haproxy01; | if (!req.backend.healthy) { | set req.backend = haproxy02; | } If haproxy01 is marked as healthy, but refuses the connection for whatever reason, you'll end up in vcl_error here. A director will give you a connection to the other one. -- Tollef Fog Heen Redpill Linpro -- Changing the game! t: +47 21 54 41 73 From tfheen at redpill-linpro.com Thu Oct 8 08:03:32 2009 From: tfheen at redpill-linpro.com (Tollef Fog Heen) Date: Thu, 08 Oct 2009 10:03:32 +0200 Subject: performance scalability of a multi-core In-Reply-To: <5C379B2C-12AF-415D-A80A-FAC0F0D38422@slide.com> (Ken Brownfield's message of "Thu, 10 Sep 2009 09:32:46 -0700") References: <75856.1252513828@critter.freebsd.dk> <1D5E4E47-4B6B-4811-A290-4128246D552A@slide.com> <4AA8B754.1080706@1art.cz> <5C379B2C-12AF-415D-A80A-FAC0F0D38422@slide.com> Message-ID: <878wfm1fe3.fsf@qurzaw.linpro.no> ]] Ken Brownfield | My weapon of choice there would be oprofile, run something like this | under high load and/or when you have a lot of threads active: latencytop (on newer linux kernels) could also be useful. | At the bottom you'll see the most costly calls and any obvious stand- | outs. My runs on Varnish show an interestingly high amount of | timezone stuff in libc, but no real low-hanging fruit. Interesting. I am guessing this is strftime being silly. Citing a friend of mine, Steinar H. Gunderson, discovered this gem: It seems that by POSIX, strftime needs to call tzset, which initializes the timezone information in glibc. tzset needs to read the environment to see if the environment variable TZ has changed. The environment is essentially a global variable, so tzset needs to take a lock and then scan linearly through it (which can take some time if it's large). Furthermore, since TZ points to a file, tzset needs to stat it to check that it hasn't changed -- and it seems to do so under the lock, for good measure. I think we might want to roll our own strftime in TIM_format rather than relying on the madness that is described above and that is POSIX. -- Tollef Fog Heen Redpill Linpro -- Changing the game! t: +47 21 54 41 73 From tfheen at redpill-linpro.com Thu Oct 8 08:12:12 2009 From: tfheen at redpill-linpro.com (Tollef Fog Heen) Date: Thu, 08 Oct 2009 10:12:12 +0200 Subject: Sub-second granularity logs In-Reply-To: <20090910215439.GX6403@digdug.corp.631h.metaweb.com> (Niall O'Higgins's message of "Thu, 10 Sep 2009 14:54:39 -0700") References: <1499.1252618389@critter.freebsd.dk> <20090910215439.GX6403@digdug.corp.631h.metaweb.com> Message-ID: <874oqa1ezn.fsf@qurzaw.linpro.no> ]] Niall O'Higgins | > Which shmrecord are we talking about ? | | I have only started looking through the existing shmlog tags to see | exactly what corresponds to our needs, but basically we are looking | for millisecond accuracy at each of these points: | | - When the cache received the request from the client | - When the request header was forwarded by the cache | - When the first byte of the response header was recieved back | - When the end of the response header was received or detected | - When the first byte of the response body was received or detected | - When the cache finished sending the response back to the client At least some of those are already available, as explained in http://varnish.projects.linpro.no/wiki/Varnishlog -- Tollef Fog Heen Redpill Linpro -- Changing the game! t: +47 21 54 41 73 From tfheen at redpill-linpro.com Thu Oct 8 08:21:48 2009 From: tfheen at redpill-linpro.com (Tollef Fog Heen) Date: Thu, 08 Oct 2009 10:21:48 +0200 Subject: Cookies being sent back with a cached response In-Reply-To: (Cosimo Streppone's message of "Mon, 05 Oct 2009 16:20:45 +0200") References: Message-ID: <87zl82z46b.fsf@qurzaw.linpro.no> ]] "Cosimo Streppone" [...] | Then looking at varnishlog output, and grepping for "Cookie", | I saw this (unrelated?): | | 90 TxHeader b Cookie: session=bf6ac86feaed6b23bef66a8f44097eac42cf24 | ... | | How can this ever happen? | The backends that serve the request never output cookies, and even if they | did, it would be "Set-Cookie:" anyway... I'm confused. This means Varnish is sending a header (TxHeader) to the backend (b), with a Cookie: session=? set. Anything received from a backend is RxHeader and b. I would guess at you having something, somewhere setting an erronous domain-wide cookie. Possibly in Javascript. -- Tollef Fog Heen Redpill Linpro -- Changing the game! t: +47 21 54 41 73 From tfheen at redpill-linpro.com Thu Oct 8 08:36:05 2009 From: tfheen at redpill-linpro.com (Tollef Fog Heen) Date: Thu, 08 Oct 2009 10:36:05 +0200 Subject: Excluding url's from varnishhist In-Reply-To: (Paul Dowman's message of "Fri, 2 Oct 2009 16:25:58 -0400") References: Message-ID: <87vdiqz3ii.fsf@qurzaw.linpro.no> ]] Paul Dowman | Hi, | I'm having trouble figuring out how to exclude certain URL's from | varnishhist. I want to exclude static files, e.g. urls that match a pattern | like /\.png|\.gif|\.js|\.css|\.ico/ (because these don't cause much load on | the back-end, I want to see only the requests that would hit my app | servers). | | I know about the -X regex argument, but it doesn't seem to do what I want. | Actually I don't really understand what it does, the man page says that it | excludes "log entries" that match a pattern, but as far as I can tell it | doesn't match URL's. | | What's the right way to do this? There's currently no way to do this, I'm afraid. varnishtop -b -i TxURL gives you the URLs, but not any timing information. -- Tollef Fog Heen Redpill Linpro -- Changing the game! t: +47 21 54 41 73 From tfheen at redpill-linpro.com Thu Oct 8 08:45:58 2009 From: tfheen at redpill-linpro.com (Tollef Fog Heen) Date: Thu, 08 Oct 2009 10:45:58 +0200 Subject: SMA outstanding allocations In-Reply-To: <003901ca42b6$0bc3c1a0$234b44e0$@paulissen@qbell.nl> (Henry Paulissen's message of "Thu, 1 Oct 2009 18:41:38 +0200") References: <003201ca4269$297408b0$7c5c1a10$@paulissen@qbell.nl> <87164.1254401771@critter.freebsd.dk> <003901ca42b6$0bc3c1a0$234b44e0$@paulissen@qbell.nl> Message-ID: <87r5tez321.fsf@qurzaw.linpro.no> ]] "Henry Paulissen" | Is this bad when the number is high and increasing every second? It's normal during startup, it just means it's adding more objects to the storage. Once you reach stable state, it probably won't move too much. -- Tollef Fog Heen Redpill Linpro -- Changing the game! t: +47 21 54 41 73 From tfheen at redpill-linpro.com Thu Oct 8 08:48:20 2009 From: tfheen at redpill-linpro.com (Tollef Fog Heen) Date: Thu, 08 Oct 2009 10:48:20 +0200 Subject: Varnish Listen problem ipv6 and ipv4 In-Reply-To: <0363C39C-113D-4C59-834D-D4BCE45BD1DA@fragstein.de> (Thomas Fragstein's message of "Thu, 8 Oct 2009 08:44:52 +0200") References: <0363C39C-113D-4C59-834D-D4BCE45BD1DA@fragstein.de> Message-ID: <87my42z2y3.fsf@qurzaw.linpro.no> ]] Thomas Fragstein Hi, | when i will start varnish with "-A :80" then varnish will only Listen | on ipv6. It's -a, not -A, but I'm guessing you're using that already. | have anybody a idee who i can win round varnish? Make sure you don't have anything else listening on ipv4 port 80? Also, which platform are you on? -- Tollef Fog Heen Redpill Linpro -- Changing the game! t: +47 21 54 41 73 From rtshilston at gmail.com Thu Oct 8 08:59:15 2009 From: rtshilston at gmail.com (Rob S) Date: Thu, 08 Oct 2009 09:59:15 +0100 Subject: Excluding url's from varnishhist In-Reply-To: <87vdiqz3ii.fsf@qurzaw.linpro.no> References: <87vdiqz3ii.fsf@qurzaw.linpro.no> Message-ID: <4ACDA9E3.1090000@gmail.com> Tollef Fog Heen wrote: > ]] Paul Dowman > > | Hi, > | I'm having trouble figuring out how to exclude certain URL's from > | varnishhist. I want to exclude static files, e.g. urls that match a pattern > | like /\.png|\.gif|\.js|\.css|\.ico/ (because these don't cause much load on > | the back-end, I want to see only the requests that would hit my app > | servers). > | > | I know about the -X regex argument, but it doesn't seem to do what I want. > | Actually I don't really understand what it does, the man page says that it > | excludes "log entries" that match a pattern, but as far as I can tell it > | doesn't match URL's. > | > | What's the right way to do this? > > There's currently no way to do this, I'm afraid. varnishtop -b -i TxURL > gives you the URLs, but not any timing information. > I realise it'd produce slightly different information, but you could create a separate backend director for files that match this regex, then look at varnishncsa for those particular backends... From cosimo at streppone.it Thu Oct 8 10:02:05 2009 From: cosimo at streppone.it (Cosimo Streppone) Date: Thu, 08 Oct 2009 12:02:05 +0200 Subject: Cookies being sent back with a cached response In-Reply-To: <87zl82z46b.fsf@qurzaw.linpro.no> References: <87zl82z46b.fsf@qurzaw.linpro.no> Message-ID: On Thu, 08 Oct 2009 10:21:48 +0200, Tollef Fog Heen wrote: > | Then looking at varnishlog output, and grepping for "Cookie", > | I saw this (unrelated?): > | > | 90 TxHeader b Cookie: > session=bf6ac86feaed6b23bef66a8f44097eac42cf24 > | ... > | > | How can this ever happen? > | The backends that serve the request never output cookies, and even if > they > | did, it would be "Set-Cookie:" anyway... I'm confused. > > This means Varnish is sending a header (TxHeader) to the backend (b), > with a Cookie: session=? set. Anything received from a backend is > RxHeader and b. Ok, it starts to make sense. Thanks for you clarification. -- Cosimo From tfheen at redpill-linpro.com Thu Oct 8 11:44:18 2009 From: tfheen at redpill-linpro.com (Tollef Fog Heen) Date: Thu, 08 Oct 2009 13:44:18 +0200 Subject: if-none-match status In-Reply-To: <20090910120356.27ae58af@der-dieb> (Joe Williams's message of "Thu, 10 Sep 2009 12:03:56 -0700") References: <20090910120356.27ae58af@der-dieb> Message-ID: <87eipeyust.fsf@qurzaw.linpro.no> ]] Joe Williams | I am just curious in the status of if-none-match support from the | commit logs and this mailing list email | (http://www.mail-archive.com/varnish-misc at projects.linpro.no/msg02738.html) | it looks like its been committed. As was asked in the | aforementioned email are there any examples of how to use/configure/etc | this new feature? It's used by clients, there are no bits to configure. -- Tollef Fog Heen Redpill Linpro -- Changing the game! t: +47 21 54 41 73 From stockrt at gmail.com Thu Oct 8 14:35:07 2009 From: stockrt at gmail.com (=?ISO-8859-1?Q?Rog=E9rio_Schneider?=) Date: Thu, 8 Oct 2009 11:35:07 -0300 Subject: performance scalability of a multi-core In-Reply-To: <878wfm1fe3.fsf@qurzaw.linpro.no> References: <75856.1252513828@critter.freebsd.dk> <1D5E4E47-4B6B-4811-A290-4128246D552A@slide.com> <4AA8B754.1080706@1art.cz> <5C379B2C-12AF-415D-A80A-FAC0F0D38422@slide.com> <878wfm1fe3.fsf@qurzaw.linpro.no> Message-ID: <100657c90910080735y2ff32adcqefaa7bb75999dd8e@mail.gmail.com> > I think we might want to roll our own strftime in TIM_format rather than > relying on the madness that is described above and that is POSIX. madness: Now, that was funny :) -- Rog?rio Schneider MSN: stockrt at hotmail.com GTalk: stockrt at gmail.com Skype: stockrt http://stockrt.github.com From thomas at fragstein.de Thu Oct 8 14:43:34 2009 From: thomas at fragstein.de (Thomas Fragstein) Date: Thu, 08 Oct 2009 16:43:34 +0200 Subject: Varnish Listen problem ipv6 and ipv4 In-Reply-To: <87my42z2y3.fsf@qurzaw.linpro.no> References: <0363C39C-113D-4C59-834D-D4BCE45BD1DA@fragstein.de> <87my42z2y3.fsf@qurzaw.linpro.no> Message-ID: <4ACDFA96.1090500@fragstein.de> Hi, oh sorry, yes is "-a" not the upper A and yes no other processes listen on port 80. ok i have only one process that listen ob 127.0.01:80. The Plattform is Debian 5.0 amd64 when i start the varnish with an ipv4 address it will bind to this (0.0.0.0:80 is also ok) but i start -a :80 then varnish only bind to :::80 and the protocol is ipv6 it will not listen ob ipv4 thx for help Thomas Tollef Fog Heen schrieb: > ]] Thomas Fragstein > > Hi, > > | when i will start varnish with "-A :80" then varnish will only Listen > | on ipv6. > > It's -a, not -A, but I'm guessing you're using that already. > > | have anybody a idee who i can win round varnish? > > Make sure you don't have anything else listening on ipv4 port 80? Also, > which platform are you on? > > From thomas at fragstein.de Thu Oct 8 14:52:46 2009 From: thomas at fragstein.de (Thomas Fragstein) Date: Thu, 08 Oct 2009 16:52:46 +0200 Subject: Varnish Listen problem ipv6 and ipv4 In-Reply-To: <4ACDFA96.1090500@fragstein.de> References: <0363C39C-113D-4C59-834D-D4BCE45BD1DA@fragstein.de> <87my42z2y3.fsf@qurzaw.linpro.no> <4ACDFA96.1090500@fragstein.de> Message-ID: <4ACDFCBE.2090606@fragstein.de> Hallo Volks, thx for food for thought the problem was the process on 127.0.0.1:80 bye Thomas Thomas Fragstein schrieb: > Hi, > > oh sorry, yes is "-a" not the upper A and yes no other processes listen > on port 80. > ok i have only one process that listen ob 127.0.01:80. > > The Plattform is Debian 5.0 amd64 > > when i start the varnish with an ipv4 address it will bind to this > (0.0.0.0:80 is also ok) > but i start -a :80 then varnish only bind to :::80 and the protocol is > ipv6 it will not listen ob ipv4 > > thx for help > Thomas > > Tollef Fog Heen schrieb: > >> ]] Thomas Fragstein >> >> Hi, >> >> | when i will start varnish with "-A :80" then varnish will only Listen >> | on ipv6. >> >> It's -a, not -A, but I'm guessing you're using that already. >> >> | have anybody a idee who i can win round varnish? >> >> Make sure you don't have anything else listening on ipv4 port 80? Also, >> which platform are you on? >> >> >> > > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc > From ppragin at solutionset.com Thu Oct 8 18:34:50 2009 From: ppragin at solutionset.com (Pavel Pragin) Date: Thu, 8 Oct 2009 11:34:50 -0700 Subject: Panic message: Assert error in VSL(), shmlog.c line 157 Message-ID: Varnish was running fine for months. It no longer starts and I see these errors on the systems logs. This was compiled by me. Thanks Version: [root at web2 varnish-2.0.4]# /usr/local/sbin/varnishd -V varnishd (varnish-2.0.4) Copyright (c) 2006-2009 Linpro AS / Verdens Gang AS Platform: [root at web2 varnish-2.0.4]# uname -a Linux web2.trunity.net 2.6.18-128.7.1.el5 #1 SMP Mon Aug 24 08:21:56 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux Sep 17 22:37:52 web2 varnishd[27525]: Child (27526) Panic message: Assert error in VSL(), shmlog.c line 157: Condition(loghead->ptr < loghead->size) not true. thread = (cache-worker)sp = 0x2aaaeb37b008 { fd = 13, id = 13, xid = 0, client = 10.4.4.5:34351, step = STP_FIRST, handling = error, ws = 0x2aaaeb37b078 { id = "sess", {s,f,r,e} = {0x2aaaeb37b808,,+15,(nil),+16384}, }, worker = 0x49de8be0 { }, }, Sep 17 22:37:52 web2 varnishd[27630]: Child (27631) died signal=6 Sep 17 22:37:52 web2 varnishd[27630]: Child (27631) Panic message: Assert error in VSL(), shmlog.c line 157: Condition(loghead->ptr < loghead->size) not true. thread = (cache-worker)sp = 0x2aaaeb37b008 { fd = 13, id = 13, xid = 0, client = 10.4.4.5:34351, step = STP_FIRST, handling = error, ws = 0x2aaaeb37b078 { id = "sess", {s,f,r,e} = {0x2aaaeb37b808,,+15,(nil),+16384}, }, worker = 0x49de8be0 { }, }, Sep 17 22:37:52 web2 varnishd[27525]: child (2021) Started Sep 17 22:37:52 web2 varnishd[27630]: child (2022) Started Sep 17 22:37:52 web2 varnishd[27525]: Pushing vcls failed: CLI communication error Sep 17 22:37:52 web2 varnishd[27525]: Child (2021) said Closed fds: 4 5 6 10 11 13 14 Sep 17 22:37:52 web2 varnishd[27525]: Child (2021) said Child starts Sep 17 22:37:52 web2 varnishd[27630]: Pushing vcls failed: CLI communication error Sep 17 22:37:52 web2 varnishd[27630]: Child (2022) said Closed fds: 4 5 6 10 11 13 14 Sep 17 22:37:52 web2 varnishd[27630]: Child (2022) said Child starts Sep 17 22:37:52 web2 varnishd[27681]: Child (27682) died signal=6 Sep 17 22:37:52 web2 varnishd[27681]: Child (27682) Panic message: Assert error in VSL(), shmlog.c line 157: Condition(loghead->ptr < loghead->size) not true. thread = (cache-worker)sp = 0x2aaaeb303008 { fd = 10, id = 10, xid = 0, client = 10.4.4.5:44416, step = STP_FIRST, handling = error, ws = 0x2aaaeb303078 { id = "sess", {s,f,r,e} = {0x2aaaeb303808,,+15,(nil),+16384}, }, worker = 0x4a141be0 { }, }, PAVEL PRAGIN ppragin at solutionset.com T > 650.328.3900 M > 408.806.8621 F > 650.328.3901 SolutionSet The Brand Technology Company http://www.SolutionSet.com PA > 275 Alma Street, Palo Alto, CA 94301 SF > 85 Second St., San Francisco, CA 94105 -------------- next part -------------- An HTML attachment was scrubbed... URL: From phk at phk.freebsd.dk Thu Oct 8 21:37:24 2009 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Thu, 08 Oct 2009 21:37:24 +0000 Subject: Panic message: Assert error in VSL(), shmlog.c line 157 In-Reply-To: Your message of "Thu, 08 Oct 2009 11:34:50 MST." Message-ID: <7838.1255037844@critter.freebsd.dk> Remove the shared memory file, it must have become corrupted. Varnish will recreate it on startup. The file is named _.vsl and is located wherever your varnish has its workdirectory (-n argument or whatever is default on your platform) Poul-henning In message , Pavel Pragin writes: >--===============1052186233== >Content-Language: en-US >Content-Type: multipart/alternative; > boundary="_000_F1B9797980995749A47520A7F9136E48103DAAE5E7MVLMAILBOXsol_" > >--_000_F1B9797980995749A47520A7F9136E48103DAAE5E7MVLMAILBOXsol_ >Content-Type: text/plain; charset="us-ascii" >Content-Transfer-Encoding: quoted-printable > >Varnish was running fine for months. It no longer starts and I see these er= >rors on the systems logs. This was compiled by me. >Thanks > >Version: >[root at web2 varnish-2.0.4]# /usr/local/sbin/varnishd -V >varnishd (varnish-2.0.4) >Copyright (c) 2006-2009 Linpro AS / Verdens Gang AS > >Platform: >[root at web2 varnish-2.0.4]# uname -a >Linux web2.trunity.net 2.6.18-128.7.1.el5 #1 SMP Mon Aug 24 08:21:56 EDT 20= >09 x86_64 x86_64 x86_64 GNU/Linux > >Sep 17 22:37:52 web2 varnishd[27525]: Child (27526) Panic message: Assert e= >rror in VSL(), shmlog.c line 157: Condition(loghead->ptr < loghead->size)= > not true. thread =3D (cache-worker)sp =3D 0x2aaaeb37b008 { fd =3D 13, i= >d =3D 13, xid =3D 0, client =3D 10.4.4.5:34351, step =3D STP_FIRST, h= >andling =3D error, ws =3D 0x2aaaeb37b078 { id =3D "sess", {s,f,r= >,e} =3D {0x2aaaeb37b808,,+15,(nil),+16384}, }, worker =3D 0x49de8be0 = >{ }, }, >Sep 17 22:37:52 web2 varnishd[27630]: Child (27631) died signal=3D6 >Sep 17 22:37:52 web2 varnishd[27630]: Child (27631) Panic message: Assert e= >rror in VSL(), shmlog.c line 157: Condition(loghead->ptr < loghead->size)= > not true. thread =3D (cache-worker)sp =3D 0x2aaaeb37b008 { fd =3D 13, i= >d =3D 13, xid =3D 0, client =3D 10.4.4.5:34351, step =3D STP_FIRST, h= >andling =3D error, ws =3D 0x2aaaeb37b078 { id =3D "sess", {s,f,r= >,e} =3D {0x2aaaeb37b808,,+15,(nil),+16384}, }, worker =3D 0x49de8be0 = >{ }, }, >Sep 17 22:37:52 web2 varnishd[27525]: child (2021) Started >Sep 17 22:37:52 web2 varnishd[27630]: child (2022) Started >Sep 17 22:37:52 web2 varnishd[27525]: Pushing vcls failed: CLI communicatio= >n error >Sep 17 22:37:52 web2 varnishd[27525]: Child (2021) said Closed fds: 4 5 6 1= >0 11 13 14 >Sep 17 22:37:52 web2 varnishd[27525]: Child (2021) said Child starts >Sep 17 22:37:52 web2 varnishd[27630]: Pushing vcls failed: CLI communicatio= >n error >Sep 17 22:37:52 web2 varnishd[27630]: Child (2022) said Closed fds: 4 5 6 1= >0 11 13 14 >Sep 17 22:37:52 web2 varnishd[27630]: Child (2022) said Child starts >Sep 17 22:37:52 web2 varnishd[27681]: Child (27682) died signal=3D6 >Sep 17 22:37:52 web2 varnishd[27681]: Child (27682) Panic message: Assert e= >rror in VSL(), shmlog.c line 157: Condition(loghead->ptr < loghead->size)= > not true. thread =3D (cache-worker)sp =3D 0x2aaaeb303008 { fd =3D 10, i= >d =3D 10, xid =3D 0, client =3D 10.4.4.5:44416, step =3D STP_FIRST, h= >andling =3D error, ws =3D 0x2aaaeb303078 { id =3D "sess", {s,f,r= >,e} =3D {0x2aaaeb303808,,+15,(nil),+16384}, }, worker =3D 0x4a141be0 = >{ }, }, > >PAVEL PRAGIN >ppragin at solutionset.com > >T > 650.328.3900 >M > 408.806.8621 >F > 650.328.3901 > >SolutionSet >The Brand Technology Company >http://www.SolutionSet.com > >PA > 275 Alma Street, Palo Alto, CA 94301 >SF > 85 Second St., San Francisco, CA 94105 > > >--_000_F1B9797980995749A47520A7F9136E48103DAAE5E7MVLMAILBOXsol_ >Content-Type: text/html; charset="us-ascii" >Content-Transfer-Encoding: quoted-printable > >osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" = >xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:= >//www.w3.org/TR/REC-html40"> > > > > > > > > > > >
> >

Varnish was running fine for months. It no longer star= >ts and >I see these errors on the systems logs. This was compiled by me.= >

> >

Thanks

> >

 

> >

Version:

> >

[root at web2 varnish-2.0.4]# /usr/local/sbin/varnishd -V= >

> >

varnishd (varnish-2.0.4)

> >

Copyright (c) 2006-2009 Linpro AS / Verdens Gang ASp>

> >

 

> >

Platform:

> >

[root at web2 varnish-2.0.4]# uname -a

> >

Linux web2.trunity.net 2.6.18-128.7.1.el5 #1 SMP Mon A= >ug 24 >08:21:56 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux

> >

 

> >

Sep 17 22:37:52 web2 varnishd[27525]: Child (27526) Pa= >nic >message: Assert error in VSL(), shmlog.c line 157:   >Condition(loghead->ptr < loghead->size) not true.  thread =3D >(cache-worker)sp =3D 0x2aaaeb37b008 {   fd =3D 13, id =3D 13, xid= > =3D >0,   client =3D 10.4.4.5:34351,   step =3D >STP_FIRST,   handling =3D error,   ws =3D 0x2aaaeb37b07= >8 >{      id =3D "sess",   &= >nbsp; >{s,f,r,e} =3D {0x2aaaeb37b808,,+15,(nil),+16384},   >},     worker =3D 0x49de8be0 {     = >}, },

> >

Sep 17 22:37:52 web2 varnishd[27630]: Child (27631) di= >ed >signal=3D6

> >

Sep 17 22:37:52 web2 varnishd[27630]: Child (27631) Pa= >nic >message: Assert error in VSL(), shmlog.c line 157:   >Condition(loghead->ptr < loghead->size) not true.  thread =3D >(cache-worker)sp =3D 0x2aaaeb37b008 {   fd =3D 13, id =3D 13, xid= > =3D >0,   client =3D 10.4.4.5:34351,   step =3D >STP_FIRST,   handling =3D error,   ws =3D 0x2aaaeb37b07= >8 >{      id =3D "sess",   &= >nbsp; >{s,f,r,e} =3D {0x2aaaeb37b808,,+15,(nil),+16384},   >},     worker =3D 0x49de8be0 {     = >}, },

> >

Sep 17 22:37:52 web2 varnishd[27525]: child (2021) Sta= >rted

> >

Sep 17 22:37:52 web2 varnishd[27630]: child (2022) Sta= >rted

> >

Sep 17 22:37:52 web2 varnishd[27525]: Pushing vcls fai= >led: >CLI communication error

> >

Sep 17 22:37:52 web2 varnishd[27525]: Child (2021) sai= >d Closed >fds: 4 5 6 10 11 13 14

> >

Sep 17 22:37:52 web2 varnishd[27525]: Child (2021) sai= >d >Child starts

> >

Sep 17 22:37:52 web2 varnishd[27630]: Pushing vcls fai= >led: >CLI communication error

> >

Sep 17 22:37:52 web2 varnishd[27630]: Child (2022) sai= >d >Closed fds: 4 5 6 10 11 13 14

> >

Sep 17 22:37:52 web2 varnishd[27630]: Child (2022) sai= >d >Child starts

> >

Sep 17 22:37:52 web2 varnishd[27681]: Child (27682) di= >ed >signal=3D6

> >

Sep 17 22:37:52 web2 varnishd[27681]: Child (27682) Pa= >nic >message: Assert error in VSL(), shmlog.c line 157:   Condition(lo= >ghead->ptr >< loghead->size) not true.  thread =3D (cache-worker)sp =3D >0x2aaaeb303008 {   fd =3D 10, id =3D 10, xid =3D 0,   c= >lient =3D >10.4.4.5:44416,   step =3D STP_FIRST,   handling =3D >error,   ws =3D 0x2aaaeb303078 {      id= > =3D >"sess",     {s,f,r,e} =3D {0x2aaaeb303808,,+1= >5,(nil),+16384},   >},     worker =3D 0x4a141be0 {     = >}, },

> >

 

> >

","sans-serif"; >color:black'>PAVEL PRAGINily: >"Verdana","sans-serif";color:black'>
>pp= >ragin at solutionset.com >
>
>; >color:black'>T >erdana","sans-serif"; >color:black'> sans-serif"; >color:#5AC100'> 650.328.3900font-family:"Verdana","sans-serif";color:black'>
>; >color:black'>M >erdana","sans-serif"; >color:black'> sans-serif"; >color:#5AC100'>408.806.8621y: >"Verdana","sans-serif";color:black'>
>; >color:black'>F >erdana","sans-serif"; >color:black'> sans-serif"; >color:#5AC100'> 650.328.3901font-family:"Verdana","sans-serif";color:black'>
>
>; >color:black'>SolutionSetly: >"Verdana","sans-serif";color:black'>
>The Brand Technology Company
>http://w= >ww.SolutionSet.com >
>
>; >color:black'>PA >Verdana","sans-serif"; >color:#5AC100'> 275 Alma Street, Palo Alto, CA 94301style=3D'font-size:7.5pt;font-family:"Verdana","sans-serif";color:black'>r> >; >color:black'>SF >Verdana","sans-serif"; >color:#5AC100'> 85 Second St., San Francisco, CA 94105style=3D'font-size:7.5pt;font-family:"Verdana","sans-serif";color:black'> <= >o:p>

> >

 

> >
> > > > > >--_000_F1B9797980995749A47520A7F9136E48103DAAE5E7MVLMAILBOXsol_-- > >--===============1052186233== >Content-Type: text/plain; charset="us-ascii" >MIME-Version: 1.0 >Content-Transfer-Encoding: 7bit >Content-Disposition: inline > >_______________________________________________ >varnish-misc mailing list >varnish-misc at projects.linpro.no >http://projects.linpro.no/mailman/listinfo/varnish-misc > >--===============1052186233==-- > -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From david.birdsong at gmail.com Thu Oct 8 23:06:22 2009 From: david.birdsong at gmail.com (David Birdsong) Date: Thu, 8 Oct 2009 16:06:22 -0700 Subject: backend health checks and 204 Message-ID: I couldn't find this anywhere, so apologies if it's already been hashed out. Would it be possible for the backend health checks to be altered to consider a 204 response from a backend as healthy? In my case, nginx will only return a 200 if it sends something so I'll use it's in memory gif, but I sure would like a 204 ;) to be a valid 'healty' response. From ketil at froyn.name Fri Oct 9 12:10:24 2009 From: ketil at froyn.name (Ketil Froyn) Date: Fri, 9 Oct 2009 14:10:24 +0200 Subject: Varnish timeout after 10 minutes Message-ID: <8585e2640910090510se9d40bdra7516ccdffb4efd0@mail.gmail.com> I deployed varnish on a webserver that also runs PHP to serve larger files of up to about 1GB. NOTE, I am using the stock varnish shipped with Debian 5, which is varnish version 1.1.2. After running this setup for a while, I got some reports of constantly interrupting downloads. After some poking around, I realised that downloads of were being interrupted if they took more that 10 minutes. Here's the output from a couple of tests with Varnish: $ time wget -t 1 --header "Host: hostname" http://server:6081/download.php?args=here ... 2009-10-09 12:31:35 (303 KB/s) - Connection closed at byte 186251297. Giving up. real 10m9.089s user 0m4.030s sys 0m11.061s $ time wget -t 1 --header "Host: hostname" http://server:6081/download.php?args=here ... 2009-10-09 12:11:55 (314 KB/s) - Connection closed at byte 192727409. Giving up. real 10m8.255s user 0m3.296s sys 0m10.937s Testing the webserver directly, it works fine. Is there a time limit like this in newer versions of varnish? Is there a workaround I can use? It seems "pass"ing these requests doesn't help, but "pipe"ing them appears to help. Is there a better solution than hardcoding "pipe"s of all requests that could take more than 10 minutes? By the way, I also noticed that even though varnish times out and cuts the connection, varnishncsa logs the full size of the original file, not the amount of data actually transferred. Regards, Ketil From stockrt at gmail.com Fri Oct 9 13:16:09 2009 From: stockrt at gmail.com (=?ISO-8859-1?Q?Rog=E9rio_Schneider?=) Date: Fri, 9 Oct 2009 10:16:09 -0300 Subject: Varnish timeout after 10 minutes In-Reply-To: <8585e2640910090510se9d40bdra7516ccdffb4efd0@mail.gmail.com> References: <8585e2640910090510se9d40bdra7516ccdffb4efd0@mail.gmail.com> Message-ID: <100657c90910090616y1ed37a7exce5a7ccad362e617@mail.gmail.com> > I deployed varnish on a webserver that also runs PHP to serve larger > files of up to about 1GB. NOTE, I am using the stock varnish shipped > with Debian 5, which is varnish version 1.1.2. We (everyone agrees, right?) recommend using a newer one :) > Is there a time limit like this in newer versions of varnish? Is there > a workaround I can use? It seems "pass"ing these requests doesn't > help, but "pipe"ing them appears to help. Is there a better solution > than hardcoding "pipe"s of all requests that could take more than 10 > minutes? Take a look at the startup option: "send_timeout" http://varnish.projects.linpro.no/browser/tags/varnish-1.1.2/bin/varnishd/mgt_param.c Regards, -- Rog?rio Schneider MSN: stockrt at hotmail.com GTalk: stockrt at gmail.com Skype: stockrt http://stockrt.github.com From scaunter at topscms.com Fri Oct 9 13:39:27 2009 From: scaunter at topscms.com (Caunter, Stefan) Date: Fri, 9 Oct 2009 09:39:27 -0400 Subject: Varnish timeout after 10 minutes In-Reply-To: <100657c90910090616y1ed37a7exce5a7ccad362e617@mail.gmail.com> References: <8585e2640910090510se9d40bdra7516ccdffb4efd0@mail.gmail.com> <100657c90910090616y1ed37a7exce5a7ccad362e617@mail.gmail.com> Message-ID: <064FF286FD17EC418BFB74629578BED1143E0293@tmg-mail4.torstar.net> I run 2.04, but this is default (ten minutes) - perhaps version 1 did not detect client session like version 2 does. param.show send_timeout send_timeout 600 [seconds] Default is 600 Send timeout for client connections. If no data has been sent to the client in this many seconds, the session is closed. See setsockopt(2) under SO_SNDTIMEO for more information. NB: This parameter may take quite some time to take (full) effect. Stefan Caunter Skype: stefancaunter -----Original Message----- From: varnish-misc-bounces at projects.linpro.no [mailto:varnish-misc-bounces at projects.linpro.no] On Behalf Of Rog?rio Schneider Sent: October-09-09 9:16 AM To: Ketil Froyn Cc: varnish-misc at projects.linpro.no Subject: Re: Varnish timeout after 10 minutes > I deployed varnish on a webserver that also runs PHP to serve larger > files of up to about 1GB. NOTE, I am using the stock varnish shipped > with Debian 5, which is varnish version 1.1.2. We (everyone agrees, right?) recommend using a newer one :) > Is there a time limit like this in newer versions of varnish? Is there > a workaround I can use? It seems "pass"ing these requests doesn't > help, but "pipe"ing them appears to help. Is there a better solution > than hardcoding "pipe"s of all requests that could take more than 10 > minutes? Take a look at the startup option: "send_timeout" http://varnish.projects.linpro.no/browser/tags/varnish-1.1.2/bin/varnishd/mgt_param.c Regards, -- Rog?rio Schneider MSN: stockrt at hotmail.com GTalk: stockrt at gmail.com Skype: stockrt http://stockrt.github.com _______________________________________________ varnish-misc mailing list varnish-misc at projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc From ketil at froyn.name Fri Oct 9 14:06:09 2009 From: ketil at froyn.name (Ketil Froyn) Date: Fri, 9 Oct 2009 16:06:09 +0200 Subject: Varnish timeout after 10 minutes In-Reply-To: <100657c90910090616y1ed37a7exce5a7ccad362e617@mail.gmail.com> References: <8585e2640910090510se9d40bdra7516ccdffb4efd0@mail.gmail.com> <100657c90910090616y1ed37a7exce5a7ccad362e617@mail.gmail.com> Message-ID: <8585e2640910090706i5a769610k31f62b173056e780@mail.gmail.com> Sorry for the private answer, Rog?rio. 2009/10/9 Rog?rio Schneider : >> I deployed varnish on a webserver that also runs PHP to serve larger >> files of up to about 1GB. NOTE, I am using the stock varnish shipped >> with Debian 5, which is varnish version 1.1.2. > > We (everyone agrees, right?) recommend using a newer one :) That would be best, of course. :) But varnish isn't core business for me, so I'm relying on distributions to keep this updated/secure. >> Is there a time limit like this in newer versions of varnish? Is there >> a workaround I can use? It seems "pass"ing these requests doesn't >> help, but "pipe"ing them appears to help. Is there a better solution >> than hardcoding "pipe"s of all requests that could take more than 10 >> minutes? > > Take a look at the startup option: "send_timeout" > > http://varnish.projects.linpro.no/browser/tags/varnish-1.1.2/bin/varnishd/mgt_param.c Thanks. The way I read this option, it is intended to take care of the situation where the client isn't receiving data. However, I have confirmed that send_timeout was in fact guilty as charged, by setting it to 300 seconds and observing that the connection was closed after 5 minutes instead of 10. Is this fixed in varnish 2.0.4? Looked at this page: http://varnish.projects.linpro.no/browser/tags/varnish-2.0.4/varnish-cache/bin/varnishd/mgt_param.c and I'm not any wiser yet. :-) Regards, Ketil From stockrt at gmail.com Fri Oct 9 14:46:59 2009 From: stockrt at gmail.com (=?ISO-8859-1?Q?Rog=E9rio_Schneider?=) Date: Fri, 9 Oct 2009 11:46:59 -0300 Subject: Varnish timeout after 10 minutes In-Reply-To: <8585e2640910090706i5a769610k31f62b173056e780@mail.gmail.com> References: <8585e2640910090510se9d40bdra7516ccdffb4efd0@mail.gmail.com> <100657c90910090616y1ed37a7exce5a7ccad362e617@mail.gmail.com> <8585e2640910090706i5a769610k31f62b173056e780@mail.gmail.com> Message-ID: <100657c90910090746g2348e8c8k1600547fc5dffb7b@mail.gmail.com> On Fri, Oct 9, 2009 at 11:06 AM, Ketil Froyn wrote: > Thanks. The way I read this option, it is intended to take care of the > situation where the client isn't receiving data. However, I have > confirmed that send_timeout was in fact guilty as charged, by setting > it to 300 seconds and observing that the connection was closed after 5 > minutes instead of 10. >From trunk: 534 { "send_timeout", tweak_timeout, &master.send_timeout, 0, 0, 535 "Send timeout for client connections. " 536 "If no data has been sent to the client in this many seconds, " 537 "the session is closed.\n" 538 "See setsockopt(2) under SO_SNDTIMEO for more information.", 539 DELAYED_EFFECT, 540 "600", "seconds" }, Yep, the description SHOULD be fixed some day :) The thing is, the connection **will be** timeouted "send_timeout" seconds after it has been set, with or without real network activity. It is, even if the user is receiving data, the connection will be closed in "send_timeout" seconds. From: SOCKET(7) Linux Programmer?s Manual SOCKET(7) SO_RCVTIMEO and SO_SNDTIMEO Specify the receiving or sending timeouts until reporting an error. The argument is a struct timeval. If an input or output function blocks for this period of time, and data has been sent or received, the return value of that function will be the amount of data transferred; if no data has been transferred and the timeout has been reached then -1 is returned with errno set to EAGAIN or EWOULDBLOCK just as if the socket was specified to be non-blocking. If the timeout is set to zero (the default) then the operation will never timeout. Timeouts only have effect for system calls that perform socket I/O (e.g., read(2), recvmsg(2), send(2), sendmsg(2)); timeouts have no effect for select(2), poll(2), epoll_wait(2), etc. Regards, -- Rog?rio Schneider MSN: stockrt at hotmail.com GTalk: stockrt at gmail.com Skype: stockrt http://stockrt.github.com From ketil at froyn.name Fri Oct 9 16:20:10 2009 From: ketil at froyn.name (Ketil Froyn) Date: Fri, 9 Oct 2009 18:20:10 +0200 Subject: Varnish timeout after 10 minutes In-Reply-To: <100657c90910090746g2348e8c8k1600547fc5dffb7b@mail.gmail.com> References: <8585e2640910090510se9d40bdra7516ccdffb4efd0@mail.gmail.com> <100657c90910090616y1ed37a7exce5a7ccad362e617@mail.gmail.com> <8585e2640910090706i5a769610k31f62b173056e780@mail.gmail.com> <100657c90910090746g2348e8c8k1600547fc5dffb7b@mail.gmail.com> Message-ID: <8585e2640910090920o604684fanbb06a9f68df05bad@mail.gmail.com> 2009/10/9 Rog?rio Schneider : > > The thing is, the connection **will be** timeouted "send_timeout" > seconds after it has been set, with or without real network activity. > It is, even if the user is receiving data, the connection will be > closed in "send_timeout" seconds. Ok, no point in upgrading over this issue, then. :-) I'll increase send_timeout and/or "pipe" necessary connections. Thanks for the help! Regards, Ketil From joe at joetify.com Fri Oct 9 18:35:35 2009 From: joe at joetify.com (Joe Williams) Date: Fri, 9 Oct 2009 11:35:35 -0700 Subject: if-none-match status In-Reply-To: <20090921102952.5277153a@der-dieb> References: <20090910120356.27ae58af@der-dieb> <20090913191412.4a30b791@der-dieb> <20090921102952.5277153a@der-dieb> Message-ID: <20091009113535.1f07bfb0@der-dieb> Anyone have any info? -Joe On Mon, 21 Sep 2009 10:29:52 -0700 Joe Williams wrote: > > Anything? > > -Joe > > > On Sun, 13 Sep 2009 19:14:12 -0700 > Joe Williams wrote: > > > > > Anyone tried this out yet? > > > > > > Devs, > > > > Have any info on how to use it? > > > > > > -Joe > > > > > > On Thu, 10 Sep 2009 12:03:56 -0700 > > Joe Williams wrote: > > > > > > > > I am just curious in the status of if-none-match support from the > > > commit logs and this mailing list email > > > (http://www.mail-archive.com/varnish-misc at projects.linpro.no/msg02738.html) > > > it looks like its been committed. As was asked in the > > > aforementioned email are there any examples of how to > > > use/configure/etc this new feature? > > > > > > Thanks. > > > > > > -Joe > > > > > > > > > > > > -- Name: Joseph A. Williams Email: joe at joetify.com Blog: http://www.joeandmotorboat.com/ From stockrt at gmail.com Fri Oct 9 18:40:14 2009 From: stockrt at gmail.com (=?ISO-8859-1?Q?Rog=E9rio_Schneider?=) Date: Fri, 9 Oct 2009 15:40:14 -0300 Subject: if-none-match status In-Reply-To: <20091009113535.1f07bfb0@der-dieb> References: <20090910120356.27ae58af@der-dieb> <20090913191412.4a30b791@der-dieb> <20090921102952.5277153a@der-dieb> <20091009113535.1f07bfb0@der-dieb> Message-ID: <100657c90910091140j19a06451obcbb34bf06b1d37d@mail.gmail.com> > Anyone have any info? What more do you need besides that already answered by Tollef? http://projects.linpro.no/pipermail/varnish-misc/2009-October/003174.html Regards, -- Rog?rio Schneider MSN: stockrt at hotmail.com GTalk: stockrt at gmail.com Skype: stockrt http://stockrt.github.com From joe at joetify.com Fri Oct 9 18:53:59 2009 From: joe at joetify.com (Joe Williams) Date: Fri, 9 Oct 2009 11:53:59 -0700 Subject: if-none-match status In-Reply-To: <100657c90910091140j19a06451obcbb34bf06b1d37d@mail.gmail.com> References: <20090910120356.27ae58af@der-dieb> <20090913191412.4a30b791@der-dieb> <20090921102952.5277153a@der-dieb> <20091009113535.1f07bfb0@der-dieb> <100657c90910091140j19a06451obcbb34bf06b1d37d@mail.gmail.com> Message-ID: <20091009115359.52dbc7ba@der-dieb> Oops, looks like I missed that reply. So it's just a matter of making sure clients use that header and varnish will decide to serve a 304 or an updated object? -Joe On Fri, 9 Oct 2009 15:40:14 -0300 Rog?rio Schneider wrote: > > Anyone have any info? > > What more do you need besides that already answered by Tollef? > > http://projects.linpro.no/pipermail/varnish-misc/2009-October/003174.html > > Regards, -- Name: Joseph A. Williams Email: joe at joetify.com Blog: http://www.joeandmotorboat.com/ From stockrt at gmail.com Fri Oct 9 19:26:14 2009 From: stockrt at gmail.com (=?ISO-8859-1?Q?Rog=E9rio_Schneider?=) Date: Fri, 9 Oct 2009 16:26:14 -0300 Subject: if-none-match status In-Reply-To: <20091009115359.52dbc7ba@der-dieb> References: <20090910120356.27ae58af@der-dieb> <20090913191412.4a30b791@der-dieb> <20090921102952.5277153a@der-dieb> <20091009113535.1f07bfb0@der-dieb> <100657c90910091140j19a06451obcbb34bf06b1d37d@mail.gmail.com> <20091009115359.52dbc7ba@der-dieb> Message-ID: <100657c90910091226q31be9664ya8fd50025a478a17@mail.gmail.com> 2009/10/9 Joe Williams : > > Oops, looks like I missed that reply. So it's just a matter of making > sure clients use that header and varnish will decide to serve a 304 or > an updated object? It is, indeed, a matter of "making sure you deliver ETag: response header from the backend" and everything is set. From now one the clients should "see" this ETag: header in the response, and then start issuing If-None-Match: request headers in the next "conditional GETs" they made. Then, you have an improved chance of delivering 304 instead of 200 + full content. Good luck, -- Rog?rio Schneider MSN: stockrt at hotmail.com GTalk: stockrt at gmail.com Skype: stockrt http://stockrt.github.com From joe at joetify.com Fri Oct 9 21:41:14 2009 From: joe at joetify.com (Joe Williams) Date: Fri, 9 Oct 2009 14:41:14 -0700 Subject: if-none-match status In-Reply-To: <100657c90910091226q31be9664ya8fd50025a478a17@mail.gmail.com> References: <20090910120356.27ae58af@der-dieb> <20090913191412.4a30b791@der-dieb> <20090921102952.5277153a@der-dieb> <20091009113535.1f07bfb0@der-dieb> <100657c90910091140j19a06451obcbb34bf06b1d37d@mail.gmail.com> <20091009115359.52dbc7ba@der-dieb> <100657c90910091226q31be9664ya8fd50025a478a17@mail.gmail.com> Message-ID: <20091009144114.7dfbbce1@der-dieb> Has is been considered doing this with the backend servers rather than the client? Basically to re-validate objects in the cache? A request hits varnish, varnish requests the backend with an if-none-match header and the backend replies with either a 200 or 304. This way the client does not need to know anything of etags, the cache is always up to date for commonly requested objects and the backend won't need to produce the content if the object in the cache is the most recent. Thanks. -Joe On Fri, 9 Oct 2009 16:26:14 -0300 Rog?rio Schneider wrote: > 2009/10/9 Joe Williams : > > > > Oops, looks like I missed that reply. So it's just a matter of > > making sure clients use that header and varnish will decide to > > serve a 304 or an updated object? > > It is, indeed, a matter of "making sure you deliver ETag: response > header from the backend" and everything is set. From now one the > clients should "see" this ETag: header in the response, and then start > issuing If-None-Match: request headers in the next "conditional GETs" > they made. Then, you have an improved chance of delivering 304 instead > of 200 + full content. > > Good luck, -- Name: Joseph A. Williams Email: joe at joetify.com Blog: http://www.joeandmotorboat.com/ From stockrt at gmail.com Fri Oct 9 22:06:55 2009 From: stockrt at gmail.com (=?ISO-8859-1?Q?Rog=E9rio_Schneider?=) Date: Fri, 9 Oct 2009 19:06:55 -0300 Subject: if-none-match status In-Reply-To: <20091009144114.7dfbbce1@der-dieb> References: <20090910120356.27ae58af@der-dieb> <20090913191412.4a30b791@der-dieb> <20090921102952.5277153a@der-dieb> <20091009113535.1f07bfb0@der-dieb> <100657c90910091140j19a06451obcbb34bf06b1d37d@mail.gmail.com> <20091009115359.52dbc7ba@der-dieb> <100657c90910091226q31be9664ya8fd50025a478a17@mail.gmail.com> <20091009144114.7dfbbce1@der-dieb> Message-ID: <100657c90910091506g746940aej4fe5ce67a67904b6@mail.gmail.com> 2009/10/9 Joe Williams : > > Has is been considered doing this with the backend servers rather than > the client? Basically to re-validate objects in the cache? A request > hits varnish, varnish requests the backend with an if-none-match header > and the backend replies with either a 200 or 304. This way the client > does not need to know anything of etags, the cache is always up to > date for commonly requested objects and the backend won't need to > produce the content if the object in the cache is the most recent. Take a look at: http://varnish.projects.linpro.no/changeset/4294 The change was made to the "client <-> varnish" communication, not "varnish <-> backend". This later communication is, AFAIK, always done without conditionals. Regards, -- Rog?rio Schneider MSN: stockrt at hotmail.com GTalk: stockrt at gmail.com Skype: stockrt http://stockrt.github.com From kb+varnish at slide.com Fri Oct 9 22:33:58 2009 From: kb+varnish at slide.com (Ken Brownfield) Date: Fri, 9 Oct 2009 15:33:58 -0700 Subject: Persistence Message-ID: <80F87A0F-EF4D-4D11-9B16-8A05657ED70B@slide.com> I've started playing with persistence a bit in trunk, and it seems like the storage is persistent across restarts of the child, but /not/ the parent. For a small working set, having any persistence at all is somewhat optional. For large working sets, you really want persistence across parent restarts, crashes, power loss, etc. Are there any plans in place for this, or hopefully I'm missing something? Thanks, -- Ken From phk at phk.freebsd.dk Fri Oct 9 22:36:35 2009 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Fri, 09 Oct 2009 22:36:35 +0000 Subject: Persistence In-Reply-To: Your message of "Fri, 09 Oct 2009 15:33:58 MST." <80F87A0F-EF4D-4D11-9B16-8A05657ED70B@slide.com> Message-ID: <13744.1255127795@critter.freebsd.dk> In message <80F87A0F-EF4D-4D11-9B16-8A05657ED70B at slide.com>, Ken Brownfield wri tes: >I've started playing with persistence a bit in trunk, and it seems >like the storage is persistent across restarts of the child, but /not/ >the parent. Hmm, that would be a bug if so, I'll have to check that. Poul-Henning -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From joe at joetify.com Fri Oct 9 23:52:11 2009 From: joe at joetify.com (Joe Williams) Date: Fri, 9 Oct 2009 16:52:11 -0700 Subject: if-none-match status In-Reply-To: <100657c90910091506g746940aej4fe5ce67a67904b6@mail.gmail.com> References: <20090910120356.27ae58af@der-dieb> <20090913191412.4a30b791@der-dieb> <20090921102952.5277153a@der-dieb> <20091009113535.1f07bfb0@der-dieb> <100657c90910091140j19a06451obcbb34bf06b1d37d@mail.gmail.com> <20091009115359.52dbc7ba@der-dieb> <100657c90910091226q31be9664ya8fd50025a478a17@mail.gmail.com> <20091009144114.7dfbbce1@der-dieb> <100657c90910091506g746940aej4fe5ce67a67904b6@mail.gmail.com> Message-ID: <20091009165211.32e3a96e@der-dieb> Thanks Rog?rio, I am totally aware that it currently is "client <-> varnish" communication that deals with if-none-match. My question is whether it was considered making an option for doing this on "varnish <-> backend" communication as well since it seems like a reasonable way to validate objects in the cache during lookups. Does the varnish project have a (official or unofficial) policy regarding doing backend communication with conditions? Thanks. -Joe On Fri, 9 Oct 2009 19:06:55 -0300 Rog?rio Schneider wrote: > 2009/10/9 Joe Williams : > > > > Has is been considered doing this with the backend servers rather > > than the client? Basically to re-validate objects in the cache? A > > request hits varnish, varnish requests the backend with an > > if-none-match header and the backend replies with either a 200 or > > 304. This way the client does not need to know anything of etags, > > the cache is always up to date for commonly requested objects and > > the backend won't need to produce the content if the object in the > > cache is the most recent. > > Take a look at: > > http://varnish.projects.linpro.no/changeset/4294 > > The change was made to the "client <-> varnish" communication, not > "varnish <-> backend". This later communication is, AFAIK, always done > without conditionals. > > Regards, -- Name: Joseph A. Williams Email: joe at joetify.com Blog: http://www.joeandmotorboat.com/ From stockrt at gmail.com Sat Oct 10 17:41:14 2009 From: stockrt at gmail.com (=?ISO-8859-1?Q?Rog=E9rio_Schneider?=) Date: Sat, 10 Oct 2009 14:41:14 -0300 Subject: if-none-match status In-Reply-To: <20091009165211.32e3a96e@der-dieb> References: <20090910120356.27ae58af@der-dieb> <20090913191412.4a30b791@der-dieb> <20090921102952.5277153a@der-dieb> <20091009113535.1f07bfb0@der-dieb> <100657c90910091140j19a06451obcbb34bf06b1d37d@mail.gmail.com> <20091009115359.52dbc7ba@der-dieb> <100657c90910091226q31be9664ya8fd50025a478a17@mail.gmail.com> <20091009144114.7dfbbce1@der-dieb> <100657c90910091506g746940aej4fe5ce67a67904b6@mail.gmail.com> <20091009165211.32e3a96e@der-dieb> Message-ID: <100657c90910101041p166acd60k1cd00a58f0af69ec@mail.gmail.com> 2009/10/9 Joe Williams : > > Thanks Rog?rio, I am totally aware that it currently is "client <-> > varnish" communication that deals with if-none-match. My question is ah, ok :) > whether it was considered making an option for doing this on "varnish > <-> backend" communication as well since it seems like a reasonable way > to validate objects in the cache during lookups. Does the varnish > project have a (official or unofficial) policy regarding doing > backend communication with conditions? Now that is a question I think others could answer (tollef, sky, phk?). I really do not know. I agree that this could be interesting for, say, really big files cache re-validation, without all the transfer from the backend to Varnish, and even worth if we are talking about using Varnish for CDN architectures (for better geolocation when delivering, with distant backends, in many cases over continents). Needless to say, this could be valid and very useful. Regards, -- Rog?rio Schneider MSN: stockrt at hotmail.com GTalk: stockrt at gmail.com Skype: stockrt http://stockrt.github.com From joe at joetify.com Sat Oct 10 18:21:21 2009 From: joe at joetify.com (Joe Williams) Date: Sat, 10 Oct 2009 11:21:21 -0700 Subject: if-none-match status In-Reply-To: <100657c90910101041p166acd60k1cd00a58f0af69ec@mail.gmail.com> References: <20090910120356.27ae58af@der-dieb> <20090913191412.4a30b791@der-dieb> <20090921102952.5277153a@der-dieb> <20091009113535.1f07bfb0@der-dieb> <100657c90910091140j19a06451obcbb34bf06b1d37d@mail.gmail.com> <20091009115359.52dbc7ba@der-dieb> <100657c90910091226q31be9664ya8fd50025a478a17@mail.gmail.com> <20091009144114.7dfbbce1@der-dieb> <100657c90910091506g746940aej4fe5ce67a67904b6@mail.gmail.com> <20091009165211.32e3a96e@der-dieb> <100657c90910101041p166acd60k1cd00a58f0af69ec@mail.gmail.com> Message-ID: <20091010112121.677f8852@der-dieb> Agreed, I think this would be a very cool feature. -Joe On Sat, 10 Oct 2009 14:41:14 -0300 Rog?rio Schneider wrote: > 2009/10/9 Joe Williams : > > > > Thanks Rog?rio, I am totally aware that it currently is "client <-> > > varnish" communication that deals with if-none-match. My question is > > ah, ok :) > > > whether it was considered making an option for doing this on > > "varnish <-> backend" communication as well since it seems like a > > reasonable way to validate objects in the cache during lookups. > > Does the varnish project have a (official or unofficial) policy > > regarding doing backend communication with conditions? > > Now that is a question I think others could answer (tollef, sky, > phk?). I really do not know. > > I agree that this could be interesting for, say, really big files > cache re-validation, without all the transfer from the backend to > Varnish, and even worth if we are talking about using Varnish for CDN > architectures (for better geolocation when delivering, with distant > backends, in many cases over continents). > > Needless to say, this could be valid and very useful. > > Regards, -- Name: Joseph A. Williams Email: joe at joetify.com Blog: http://www.joeandmotorboat.com/ From quasirob at googlemail.com Wed Oct 14 12:06:18 2009 From: quasirob at googlemail.com (Rob Ayres) Date: Wed, 14 Oct 2009 13:06:18 +0100 Subject: hitrate at 1.00 but backend still getting traffic Message-ID: Hi, Doing some testing and have got the hitrate average to 1.00. Cache misses are reported as 0.00 but we are still seeing reqests in the backends apache logs. The Total Pass line (see the varnishstat output below) seems to show about the right number of requests to account for this. But if there is still traffic going to the backend I dont understand why the hitrate is 1.00 The vcl file is very simple, backend default { .host = "barbican-z3"; .port = "80"; } sub vcl_fetch { if (obj.cacheable) { set obj.ttl = 24h; } } Also, is there a document detailing what each line of varnishstat means? Thanks, Rob Hitrate ratio: 10 100 1000 Hitrate avg: 1.0000 1.0000 0.9898 3404 0.00 0.69 Client connections accepted 474002 290.17 95.51 Client requests received 346436 271.16 69.80 Cache hits 55518 0.00 11.19 Cache misses 126781 19.01 25.55 Backend connections success 757 0.00 0.15 Backend connections failures 126778 20.01 25.54 Backend connections recycles 1 . . N struct srcaddr 1 . . N active struct srcaddr 27 . . N struct sess_mem 14 . . N struct sess 54903 . . N struct object 54906 . . N struct objecthead 114219 . . N struct smf 4215 . . N small free smf 1 . . N large free smf 3 . . N struct vbe_conn 10 . . N struct bereq 14 . . N worker threads 14 0.00 0.00 N worker threads created 0 0.00 0.00 N queued work requests 563 0.00 0.11 N overflowed work requests 1 . . N backends 194279 . . N LRU moved objects 481532 303.18 97.02 Objects sent with write 3404 0.00 0.69 Total Sessions 474019 291.18 95.51 Total Requests 72023 20.01 14.51 Total pass 126778 20.01 25.54 Total fetch 101827536 63514.30 20517.34 Total header bytes 5814577615 5316785.81 1171585.25 Total body bytes 1246 0.00 0.25 Session Closed 473212 292.18 95.35 Session herd 17724459 9138.51 3571.32 SHM records 1092786 605.37 220.19 SHM writes 4361 1.00 0.88 SHM MTX contention 5 0.00 0.00 SHM cycles through buffer 256109 40.02 51.60 allocator requests 110004 . . outstanding allocations 803733504 . . bytes allocated 7786201088 . . bytes free 764 0.00 0.15 SMS allocator requests 355236 . . SMS bytes allocated 355236 . . SMS bytes freed 126786 19.01 25.55 Backend requests made 1 0.00 0.00 N vcl total 1 0.00 0.00 N vcl available 1 . . N total active purges 1 0.00 0.00 N new purges added -------------- next part -------------- An HTML attachment was scrubbed... URL: From jfbustarret at tf1.fr Wed Oct 14 12:50:58 2009 From: jfbustarret at tf1.fr (BUSTARRET, Jean-francois) Date: Wed, 14 Oct 2009 14:50:58 +0200 Subject: hitrate at 1.00 but backend still getting traffic In-Reply-To: Message-ID: <53C652A09719C54DA24741D0157CB269042F461C@TFPRDEXS1.tf1.groupetf1.fr> > But if there is still traffic going to the backend I dont understand why the hitrate is 1.00 Hitrate = number of hits/number of *cacheable* requests The requests you see going to the backend are non-cacheable requests. By default, varnish does not cache if a request contains a cookie. You might want to add some rule to cache those requests (when possible). Jean-Fran?ois From mail at danielbruessler.de Wed Oct 14 13:33:27 2009 From: mail at danielbruessler.de (Daniel Bruessler) Date: Wed, 14 Oct 2009 15:33:27 +0200 Subject: call a sub function and return Message-ID: <4AD5D327.9040108@danielbruessler.de> Hi! I'd like to call a sub function and use the return-function. But is there a return-function? #called from vcl_fetch: call vcl_custom_cachingtime; sub vcl_custom_cachingtimes { if (req.url ~ "^/abc/*" { set obj.ttl = 10m; #return (0); } if (req.url ~ "^/def/*" { set obj.ttl = 30s; } } thanks in advance and cheers! Daniel :: Daniel Bruessler - Emilienstr. 10 - 90489 Nuernberg From phk at phk.freebsd.dk Wed Oct 14 14:09:10 2009 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Wed, 14 Oct 2009 14:09:10 +0000 Subject: call a sub function and return In-Reply-To: Your message of "Wed, 14 Oct 2009 15:33:27 +0200." <4AD5D327.9040108@danielbruessler.de> Message-ID: <32046.1255529350@critter.freebsd.dk> In message <4AD5D327.9040108 at danielbruessler.de>, Daniel Bruessler writes: >Hi! > >I'd like to call a sub function and use the return-function. But is >there a return-function? > >#called from vcl_fetch: call vcl_custom_cachingtime; >sub vcl_custom_cachingtimes { > > if (req.url ~ "^/abc/*" { > set obj.ttl = 10m; > #return (0); > } > > if (req.url ~ "^/def/*" { > set obj.ttl = 30s; > } > >} It's not clear from you example what you would use the return for, but you can simulate it, but putting you return value in a header instead -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From mail at danielbruessler.de Wed Oct 14 14:39:55 2009 From: mail at danielbruessler.de (Daniel Bruessler) Date: Wed, 14 Oct 2009 16:39:55 +0200 Subject: call a sub function and return In-Reply-To: <32046.1255529350@critter.freebsd.dk> References: <32046.1255529350@critter.freebsd.dk> Message-ID: <4AD5E2BB.2020807@danielbruessler.de> Hello, I don't need a "return" to return a value, I just want to jump out before the end of the function. I posted a simplified version of my function, the real one has several if-clauses. Cheers! Daniel >> Hi! >> >> I'd like to call a sub function and use the return-function. But is >> there a return-function? >> >> #called from vcl_fetch: call vcl_custom_cachingtime; >> sub vcl_custom_cachingtimes { >> >> if (req.url ~ "^/abc/*" { >> set obj.ttl = 10m; >> #return (0); >> } >> >> if (req.url ~ "^/def/*" { >> set obj.ttl = 30s; >> } >> >> } > > It's not clear from you example what you would use the return for, > but you can simulate it, but putting you return value in a header > instead > From phk at phk.freebsd.dk Wed Oct 14 14:42:00 2009 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Wed, 14 Oct 2009 14:42:00 +0000 Subject: call a sub function and return In-Reply-To: Your message of "Wed, 14 Oct 2009 16:39:55 +0200." <4AD5E2BB.2020807@danielbruessler.de> Message-ID: <32161.1255531320@critter.freebsd.dk> In message <4AD5E2BB.2020807 at danielbruessler.de>, Daniel Bruessler writes: >Hello, > >I don't need a "return" to return a value, I just want to jump out >before the end of the function. I posted a simplified version of my >function, the real one has several if-clauses. Use elseif ? if (foo) { } elseif (bar) { } elseif (foomble) { } else { } -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From lists at pauldowman.com Wed Oct 14 20:49:20 2009 From: lists at pauldowman.com (Paul Dowman) Date: Wed, 14 Oct 2009 16:49:20 -0400 Subject: Excluding url's from varnishhist In-Reply-To: <4ACDA9E3.1090000@gmail.com> References: <87vdiqz3ii.fsf@qurzaw.linpro.no> <4ACDA9E3.1090000@gmail.com> Message-ID: On Thu, Oct 8, 2009 at 4:59 AM, Rob S wrote: > Tollef Fog Heen wrote: > > ]] Paul Dowman > > > > | Hi, > > | I'm having trouble figuring out how to exclude certain URL's from > > | varnishhist. I want to exclude static files, e.g. urls that match a > pattern > > | like /\.png|\.gif|\.js|\.css|\.ico/ (because these don't cause much > load on > > | the back-end, I want to see only the requests that would hit my app > > | servers). > > | > > | I know about the -X regex argument, but it doesn't seem to do what I > want. > > | Actually I don't really understand what it does, the man page says that > it > > | excludes "log entries" that match a pattern, but as far as I can tell > it > > | doesn't match URL's. > > | > > | What's the right way to do this? > > > > There's currently no way to do this, I'm afraid. varnishtop -b -i TxURL > > gives you the URLs, but not any timing information. > > > I realise it'd produce slightly different information, but you could > create a separate backend director for files that match this regex, then > look at varnishncsa for those particular backends... Thanks for the suggestion, I'll consider that. varnishlog has everything I need to know (request time and hit/miss), just not all on one line, so I think what I'll do is write a little script that converts the multi-line varnishlog output into a single line with the few things that I want to see (URL, response time and hit/miss). I'll share that if/when I get around to doing it in case it's useful for anyone else. Paul -------------- next part -------------- An HTML attachment was scrubbed... URL: From mail at danielbruessler.de Thu Oct 15 13:40:57 2009 From: mail at danielbruessler.de (Daniel Bruessler) Date: Thu, 15 Oct 2009 15:40:57 +0200 Subject: setting .connect_timeout and param.set Message-ID: <4AD72669.6040603@danielbruessler.de> Hi, It works, when I set a param via telnet example param set connect_timeout 0.3 but it's not working in the default.vcl example backend default { .host = "127.0.0.1"; .port = "8080"; .connect_timeout = 0.3s; .max_connections = 800; } (for the last 2 params the compiler doesn't allow apostrophes) what am I doing wrong? Cheers! Daniel :: Daniel Bruessler - Emilienstr. 10 - 90489 Nuernberg From tfheen at redpill-linpro.com Fri Oct 16 06:42:09 2009 From: tfheen at redpill-linpro.com (Tollef Fog Heen) Date: Fri, 16 Oct 2009 08:42:09 +0200 Subject: hitrate at 1.00 but backend still getting traffic In-Reply-To: (Rob Ayres's message of "Wed, 14 Oct 2009 13:06:18 +0100") References: Message-ID: <87eip396wu.fsf@qurzaw.linpro.no> ]] Rob Ayres | Also, is there a document detailing what each line of varnishstat | means? http://varnish.projects.linpro.no/wiki/StatsExplained is a start, but is not complete. -- Tollef Fog Heen Redpill Linpro -- Changing the game! t: +47 21 54 41 73 From mail at danielbruessler.de Fri Oct 16 09:20:12 2009 From: mail at danielbruessler.de (Daniel Bruessler) Date: Fri, 16 Oct 2009 11:20:12 +0200 Subject: setting .connect_timeout and param.set In-Reply-To: <4AD72669.6040603@danielbruessler.de> References: <4AD72669.6040603@danielbruessler.de> Message-ID: <4AD83ACC.9060308@danielbruessler.de> Hello, sorry, I forgot to tell the version-number: I'm using varnish 2.0.4 Cheers! Daniel > Hi, > > It works, when I set a param via telnet example > param set connect_timeout 0.3 > > but it's not working in the default.vcl example > backend default { > .host = "127.0.0.1"; > .port = "8080"; > .connect_timeout = 0.3s; > .max_connections = 800; > } > > (for the last 2 params the compiler doesn't allow apostrophes) > > what am I doing wrong? > > Cheers! > Daniel > > :: Daniel Bruessler - Emilienstr. 10 - 90489 Nuernberg > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc > From stockrt at gmail.com Fri Oct 16 14:28:55 2009 From: stockrt at gmail.com (=?ISO-8859-1?Q?Rog=E9rio_Schneider?=) Date: Fri, 16 Oct 2009 11:28:55 -0300 Subject: hitrate at 1.00 but backend still getting traffic In-Reply-To: <53C652A09719C54DA24741D0157CB269042F461C@TFPRDEXS1.tf1.groupetf1.fr> References: <53C652A09719C54DA24741D0157CB269042F461C@TFPRDEXS1.tf1.groupetf1.fr> Message-ID: <100657c90910160728u7b5a3238xa9f54a45ec38e003@mail.gmail.com> On Wed, Oct 14, 2009 at 9:50 AM, BUSTARRET, Jean-francois wrote: > > By default, varnish does not cache if a request contains a cookie. You might want to add some rule to cache those requests (when possible). I believe that, by default, Varnish does not cache a *response* that contains Set-Cookie:, not a request containing Cookie:, as you said, right? Regards, -- Rog?rio Schneider MSN: stockrt at hotmail.com GTalk: stockrt at gmail.com Skype: stockrt http://stockrt.github.com From jfbustarret at tf1.fr Mon Oct 19 07:41:35 2009 From: jfbustarret at tf1.fr (BUSTARRET, Jean-francois) Date: Mon, 19 Oct 2009 09:41:35 +0200 Subject: hitrate at 1.00 but backend still getting traffic In-Reply-To: <100657c90910160728u7b5a3238xa9f54a45ec38e003@mail.gmail.com> Message-ID: <53C652A09719C54DA24741D0157CB269042F4F0E@TFPRDEXS1.tf1.groupetf1.fr> > -----Message d'origine----- > I believe that, by default, Varnish does not cache a *response* that > contains Set-Cookie:, not a request containing Cookie:, as you said, > right? Both are not cacheable. Here is what's in the default vlc_recv : 57 if (req.http.Authorization || req.http.Cookie) { 58 /* Not cacheable by default */ 59 return (pass); 60 } >From : http://varnish.projects.linpro.no/browser/branches/2.0/varnish-cache/bin/varnishd/default.vcl Jean-Fran?ois From itisgany at gmail.com Tue Oct 20 19:26:30 2009 From: itisgany at gmail.com (GaneshKumar Natarajan) Date: Tue, 20 Oct 2009 15:26:30 -0400 Subject: varnish 2.0.4 questions - no IMS, no persistence cache - please help Message-ID: I have used squid for last 2 years and i wanted to move to varnish mainly because of the performance. I have the following observations/questions in varnish 2.0.4. kindly help with my questions. Regards Ganesh 1. varnish doesn't send IMS request, when the object expires. detail: When a object is set with maxage=1 min, after 1 min, varnsish calls vcl_timeout which calls vcl_discard and the object is discarded. any request after 1 min ( a normal GET request ), varnish downloads the complete object from origin server. Squid sends If-Modified-Since, If-None-Match requests to origin, which results in 304 response, thereby origin server is not loaded. Question: could varnish just mark the object expired and on the next GET request to the object, send a If-Modified-Since request to origin server? if its not in architecture, can we do it with vcl_xxx routines to achieve the same? let me knw how to achieve that? -------------------------------- 2. Varnish cache is not persistent. detail: when i stop and restart varnish, all the objects in the cache are lost. Varnish has to rebuild the cache from beginning, which load the origin server. I used mmap storage option, with the assumption that it will have a backup copy of cached objects. Question: Is my observation true in 2.0.4? If so, is there any plans to include cache persistence in future releases? ----------------------------------------------------- 3. mmap storage : max i can configure is 340 GB. detail:: I was able to use only 340 GB of cache. any size after this, i would get error. child (25790) Started Pushing vcls failed: dlopen(./vcl.1P9zoqAU.so): ./vcl.1P9zoqAU.so: failed to map segment from shared object: Cannot allocate memory currently my system has 1.5 TB of storage. squid used the entire storage. Question: Why there is a limitation on cache storage? ---------------------------------------------------------- 4. I tried malloc storage option. Question: i have system of 32 GB physical ram and my swap storage is 65 GB. harddisk storage - 1.5 TB ( distribued in 5 cache directoreys /cach0,.../cache1 ). What is the best optimal malloc storage i can use to achive greater performance? -s malloc, [ size ] ??? ---------------------------------------------- 5. what is the difference between using malloc or mmap? Help me to move onto varnish world and convince others in my team. Ganesh From kristian at redpill-linpro.com Wed Oct 21 10:50:03 2009 From: kristian at redpill-linpro.com (Kristian Lyngstol) Date: Wed, 21 Oct 2009 12:50:03 +0200 Subject: rewrite redirects In-Reply-To: <4A8EA68C.7020403@datamatix.at> References: <4A8EA68C.7020403@datamatix.at> Message-ID: <20091021105003.GD4527@kjeks.linpro.no> On Fri, Aug 21, 2009 at 03:52:12PM +0200, Guenter Wildmann wrote: > Hello! > > Can varnish rewrite redirects from the backend (equivalent to > ProxyPassReverse in apache)? > If so, how is it done? Assuming the redirects are 302/301s; yes. Check for them in vcl_fetch, then do the magic there, but you may have to elaborate as to what precisely you want. -- Kristian Lyngst?l Redpill Linpro AS Tlf: +47 21544179 Mob: +47 99014497 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From kristian at redpill-linpro.com Wed Oct 21 11:34:26 2009 From: kristian at redpill-linpro.com (Kristian Lyngstol) Date: Wed, 21 Oct 2009 13:34:26 +0200 Subject: Varnish virtual memory usage In-Reply-To: <533E1F3F0E92C246A5129A1CF8F6F694F73D77@brain.office.mc> References: <533E1F3F0E92C246A5129A1CF8F6F694F73D77@brain.office.mc> Message-ID: <20091021113424.GE4527@kjeks.linpro.no> On Mon, Sep 21, 2009 at 02:55:07PM +0300, Roi Avinoam wrote: > At Metacafe we're testing the integration with Varnish, and I was tasked > with benchmarking our Varnish setup. I intentionally over-flooded the > server with requests, in an attempt to see how the system will behave > under extensive traffic. Surprisingly, the server ran out of swap and > crashed. That seems mighty strange. What sort of tests did you do? > In out configuration, "-s file,/var/lib/varnish/varnish_storage.bin,1G". > Does it mean Varnish shouldn't use more than 1GB of the virtual memory? > Is there any other way to limit the memory/storage usage? If you are using -s file and you have 4GB of memory, you are telling Varnish to create a _file_ of 1GB, and it's up to the kernel what it keeps in memory or not. If you actually run out of memory with this setup, you've either hit a bug (need more details first), or you're doing something strange like having the mmaped file (/var/lib/varnish/) in tmpfs with a sizelimit less than 1GB or something along those lines. But I need more details to say anything for certain. -- Kristian Lyngst?l Redpill Linpro AS Tlf: +47 21544179 Mob: +47 99014497 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From roi at metacafe.com Wed Oct 21 13:07:00 2009 From: roi at metacafe.com (Roi Avinoam) Date: Wed, 21 Oct 2009 15:07:00 +0200 Subject: Varnish virtual memory usage In-Reply-To: <20091021113424.GE4527@kjeks.linpro.no> References: <533E1F3F0E92C246A5129A1CF8F6F694F73D77@brain.office.mc> <20091021113424.GE4527@kjeks.linpro.no> Message-ID: <533E1F3F0E92C246A5129A1CF8F6F6940108A8B6@brain.office.mc> Thanks for your reply. 1. What I did was create 100 simultaneous processes, and each process requested the same page (with 'curl'): a. Once with the exact same URL - which resulted in a 99.9% hit-ratio and VERY high performance on varnish. b. And once with a random key that changes the URL (something like 'index.php?rand=193837364'), thus forcing Varnish to hit the backend, and store the multiple objects in memory. c. A combination of the two - in an attempt to maintain a 60-70% hit-ratio. What we saw is that the kernel simply filled all of the RAM *and* the swap until it crashed. 2. I'm sorry, but I'm still confused about the mmaped file. If it's limited to 1G, Varnish shouldn't use more than 1G of virtual memory, correct? Or in our setup - 1G of RAM? Thanks again :) -- Roi -----Original Message----- From: Kristian Lyngstol [mailto:kristian at redpill-linpro.com] Sent: Wednesday, October 21, 2009 1:34 PM To: Roi Avinoam Cc: varnish-misc at projects.linpro.no Subject: Re: Varnish virtual memory usage On Mon, Sep 21, 2009 at 02:55:07PM +0300, Roi Avinoam wrote: > At Metacafe we're testing the integration with Varnish, and I was > tasked with benchmarking our Varnish setup. I intentionally > over-flooded the server with requests, in an attempt to see how the > system will behave under extensive traffic. Surprisingly, the server > ran out of swap and crashed. That seems mighty strange. What sort of tests did you do? > In out configuration, "-s file,/var/lib/varnish/varnish_storage.bin,1G". > Does it mean Varnish shouldn't use more than 1GB of the virtual memory? > Is there any other way to limit the memory/storage usage? If you are using -s file and you have 4GB of memory, you are telling Varnish to create a _file_ of 1GB, and it's up to the kernel what it keeps in memory or not. If you actually run out of memory with this setup, you've either hit a bug (need more details first), or you're doing something strange like having the mmaped file (/var/lib/varnish/) in tmpfs with a sizelimit less than 1GB or something along those lines. But I need more details to say anything for certain. -- Kristian Lyngst?l Redpill Linpro AS Tlf: +47 21544179 Mob: +47 99014497 From h.paulissen at qbell.nl Wed Oct 21 13:22:48 2009 From: h.paulissen at qbell.nl (Henry Paulissen) Date: Wed, 21 Oct 2009 15:22:48 +0200 Subject: Varnish virtual memory usage Message-ID: <007401ca5251$9560d420$c0227c60$@paulissen@qbell.nl> We encounter the same problem. Its seems to occur only on x64 platforms. We decided to take a different approach and installed vmware to the machine. Next we did a setup of 6 guests with x32 PAE software. No strange memory leaks occurred since then at the price of small storage (3.5G max) and limited worker threads (256 max). Opened a ticket for the problem, but the wont listen until I buy a support contract (? ?8K). Seems they don?t want to know there is some kind of memory issue in their software. Anyway... Varnish is running stable now with some few tricks. Regards, -----Original Message----- From: varnish-misc-bounces at projects.linpro.no [mailto:varnish-misc-bounces at projects.linpro.no] On Behalf Of Kristian Lyngstol Sent: woensdag 21 oktober 2009 13:34 To: Roi Avinoam Cc: varnish-misc at projects.linpro.no Subject: Re: Varnish virtual memory usage On Mon, Sep 21, 2009 at 02:55:07PM +0300, Roi Avinoam wrote: > At Metacafe we're testing the integration with Varnish, and I was > tasked with benchmarking our Varnish setup. I intentionally > over-flooded the server with requests, in an attempt to see how the > system will behave under extensive traffic. Surprisingly, the server > ran out of swap and crashed. That seems mighty strange. What sort of tests did you do? > In out configuration, "-s file,/var/lib/varnish/varnish_storage.bin,1G". > Does it mean Varnish shouldn't use more than 1GB of the virtual memory? > Is there any other way to limit the memory/storage usage? If you are using -s file and you have 4GB of memory, you are telling Varnish to create a _file_ of 1GB, and it's up to the kernel what it keeps in memory or not. If you actually run out of memory with this setup, you've either hit a bug (need more details first), or you're doing something strange like having the mmaped file (/var/lib/varnish/) in tmpfs with a sizelimit less than 1GB or something along those lines. But I need more details to say anything for certain. -- Kristian Lyngst?l Redpill Linpro AS Tlf: +47 21544179 Mob: +47 99014497 From kristian at redpill-linpro.com Wed Oct 21 13:27:59 2009 From: kristian at redpill-linpro.com (Kristian Lyngstol) Date: Wed, 21 Oct 2009 15:27:59 +0200 Subject: Varnish virtual memory usage In-Reply-To: <533E1F3F0E92C246A5129A1CF8F6F6940108A8B6@brain.office.mc> References: <533E1F3F0E92C246A5129A1CF8F6F694F73D77@brain.office.mc> <20091021113424.GE4527@kjeks.linpro.no> <533E1F3F0E92C246A5129A1CF8F6F6940108A8B6@brain.office.mc> Message-ID: <20091021132759.GA3942@kjeks.linpro.no> On Wed, Oct 21, 2009 at 03:07:00PM +0200, Roi Avinoam wrote: > Thanks for your reply. > > 1. What I did was create 100 simultaneous processes, and each process requested the same page (with 'curl'): > a. Once with the exact same URL - which resulted in a 99.9% hit-ratio and VERY high performance on varnish. > b. And once with a random key that changes the URL (something like 'index.php?rand=193837364'), thus forcing Varnish to hit the backend, and store the multiple objects in memory. > c. A combination of the two - in an attempt to maintain a 60-70% hit-ratio. > > What we saw is that the kernel simply filled all of the RAM *and* the swap until it crashed. When did it fill up and how fast? > 2. I'm sorry, but I'm still confused about the mmaped file. If it's > limited to 1G, Varnish shouldn't use more than 1G of virtual memory, > correct? Or in our setup - 1G of RAM? The -s parameter only applies to storage, which is to say that it's an approximiation and there will be some additional overhead which is not stored there, ie datastructures for threads, sessions, backends etc. So with -s ..., 1G, you should expect a little bit over 1GB, but not nearly as much as you describe. -- Kristian Lyngst?l Redpill Linpro AS Tlf: +47 21544179 Mob: +47 99014497 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From kristian at redpill-linpro.com Wed Oct 21 13:45:04 2009 From: kristian at redpill-linpro.com ('Kristian Lyngstol') Date: Wed, 21 Oct 2009 15:45:04 +0200 Subject: Varnish virtual memory usage In-Reply-To: <20091021134300.GB3942@kjeks.linpro.no> References: <533E1F3F0E92C246A5129A1CF8F6F694F73D77@brain.office.mc> <20091021113424.GE4527@kjeks.linpro.no> <007101ca524e$0f4bc0f0$2de342d0$@paulissen@qbell.nl> <20091021134300.GB3942@kjeks.linpro.no> Message-ID: <20091021134504.GC3942@kjeks.linpro.no> ("forwarded" to -misc since I just notice the original went there too) - Kristian On Wed, Oct 21, 2009 at 03:43:00PM +0200, 'Kristian Lyngstol' wrote: > (ticket #546) > > On Wed, Oct 21, 2009 at 02:57:34PM +0200, Henry Paulissen wrote: > > Opened a ticket for the problem, but the wont listen until I buy a > > support contract (? ?8K). Seems they don?t want to know there is some > > kind of memory issue in their software. > > The ticket is not closed, we have, however, not been able to reproduce this > as we point out in the ticket. Until we can either reproduce this ourself > or get more data (more reports of the same issue, for instance), there > really isn't much we can do. > > For a service agreement customer, we would most likely use their system to > reproduce the issue and take it from there. You will understand if we do > not log into your system for free to solve a problem which so far has been > reported by two people. We do take the issue seriously, but memory leaks > that only occur on a specific setup that we do not have access to is nearly > impossible to track down. > > We could read through and verify our code for a year and still not find the > bug. > > Service agreements help sponsor the development of Varnish, in return you > get priority on bugs - even the ones that are difficult to track down. We > do not require anyone to pay for our service agreements to use Varnish or > report bugs, and we do not ignore bug reports from non-paying Varnish > users. > > As you may notice, we (Tollef, Poul-Henning and myself) offer a great deal > of support for free on the mailing lists and on IRC, so I think it's a bit > unfair to state that we do not care unless you pay for a service agreement, > even if we weren't able to help in your specific case. The offer of a > service agreement in the ticket was not meant to be an entry-fee to the bug > tracker, but rather a means to make us prioritize a complicated bug that > we would otherwise have to put on hold. I'm sorry if that didn't come > across clearly. > > -- > Kristian Lyngst?l > Redpill Linpro AS > Tlf: +47 21544179 > Mob: +47 99014497 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From h.paulissen at qbell.nl Wed Oct 21 14:22:55 2009 From: h.paulissen at qbell.nl (Henry Paulissen) Date: Wed, 21 Oct 2009 16:22:55 +0200 Subject: Varnish virtual memory usage Message-ID: <00a601ca5259$fb6cf7f0$f246e7d0$@paulissen@qbell.nl> Maybe it was a bit rough to say indeed... My apologies for that one. What im saying a longer time for now (also sayed to Paul by phone). I'm simply not making enough profit for ?8k, but im not saying I don?t want to pay anything for service / support. It?s simply not within my reach and for me it's cheaper to run 6 guests in a big vmware server as paying ?8K and get the problem (maybe) solved. Anyway, this is way to offtopic for this thread. I shared my experiences and solutions Roi can consider it as a solution for him. Regards, Henry Paulissen -----Original Message----- From: 'Kristian Lyngstol' [mailto:kristian at redpill-linpro.com] Sent: woensdag 21 oktober 2009 15:43 To: Henry Paulissen Cc: varnish at redpill-linpro.com Subject: Re: Varnish virtual memory usage (ticket #546) On Wed, Oct 21, 2009 at 02:57:34PM +0200, Henry Paulissen wrote: > Opened a ticket for the problem, but the wont listen until I buy a > support contract (? ?8K). Seems they don?t want to know there is some > kind of memory issue in their software. The ticket is not closed, we have, however, not been able to reproduce this as we point out in the ticket. Until we can either reproduce this ourself or get more data (more reports of the same issue, for instance), there really isn't much we can do. For a service agreement customer, we would most likely use their system to reproduce the issue and take it from there. You will understand if we do not log into your system for free to solve a problem which so far has been reported by two people. We do take the issue seriously, but memory leaks that only occur on a specific setup that we do not have access to is nearly impossible to track down. We could read through and verify our code for a year and still not find the bug. Service agreements help sponsor the development of Varnish, in return you get priority on bugs - even the ones that are difficult to track down. We do not require anyone to pay for our service agreements to use Varnish or report bugs, and we do not ignore bug reports from non-paying Varnish users. As you may notice, we (Tollef, Poul-Henning and myself) offer a great deal of support for free on the mailing lists and on IRC, so I think it's a bit unfair to state that we do not care unless you pay for a service agreement, even if we weren't able to help in your specific case. The offer of a service agreement in the ticket was not meant to be an entry-fee to the bug tracker, but rather a means to make us prioritize a complicated bug that we would otherwise have to put on hold. I'm sorry if that didn't come across clearly. -- Kristian Lyngst?l Redpill Linpro AS Tlf: +47 21544179 Mob: +47 99014497 From roi at metacafe.com Wed Oct 21 17:27:01 2009 From: roi at metacafe.com (Roi Avinoam) Date: Wed, 21 Oct 2009 19:27:01 +0200 Subject: Varnish virtual memory usage In-Reply-To: <20091021132759.GA3942@kjeks.linpro.no> References: <533E1F3F0E92C246A5129A1CF8F6F694F73D77@brain.office.mc> <20091021113424.GE4527@kjeks.linpro.no> <533E1F3F0E92C246A5129A1CF8F6F6940108A8B6@brain.office.mc> <20091021132759.GA3942@kjeks.linpro.no> Message-ID: <533E1F3F0E92C246A5129A1CF8F6F6940108A952@brain.office.mc> I don't remember how long it took exactly, but it filled up rather quickly. Probably a couple of hours. I also noticed the LRU-moved stat was pretty constant all along, no large spikes towards the end of the swap. Anyways, it was WAY over 1GB, we managed to fill more around 6-7GB. Just for kicks, I tried changing the storage to malloc, and limit it to 1GB, to see the difference. We have live traffic now, so I don't wanna run the load testing scripts, but we should hit the 1GB limit within a day or two. -----Original Message----- From: Kristian Lyngstol [mailto:kristian at redpill-linpro.com] Sent: Wednesday, October 21, 2009 3:28 PM To: Roi Avinoam Cc: varnish-misc at projects.linpro.no Subject: Re: Varnish virtual memory usage On Wed, Oct 21, 2009 at 03:07:00PM +0200, Roi Avinoam wrote: > Thanks for your reply. > > 1. What I did was create 100 simultaneous processes, and each process requested the same page (with 'curl'): > a. Once with the exact same URL - which resulted in a 99.9% hit-ratio and VERY high performance on varnish. > b. And once with a random key that changes the URL (something like 'index.php?rand=193837364'), thus forcing Varnish to hit the backend, and store the multiple objects in memory. > c. A combination of the two - in an attempt to maintain a 60-70% hit-ratio. > > What we saw is that the kernel simply filled all of the RAM *and* the swap until it crashed. When did it fill up and how fast? > 2. I'm sorry, but I'm still confused about the mmaped file. If it's > limited to 1G, Varnish shouldn't use more than 1G of virtual memory, > correct? Or in our setup - 1G of RAM? The -s parameter only applies to storage, which is to say that it's an approximiation and there will be some additional overhead which is not stored there, ie datastructures for threads, sessions, backends etc. So with -s ..., 1G, you should expect a little bit over 1GB, but not nearly as much as you describe. -- Kristian Lyngst?l Redpill Linpro AS Tlf: +47 21544179 Mob: +47 99014497 From darryl.dixon at winterhouseconsulting.com Wed Oct 21 21:09:56 2009 From: darryl.dixon at winterhouseconsulting.com (Darryl Dixon - Winterhouse Consulting) Date: Thu, 22 Oct 2009 10:09:56 +1300 (NZDT) Subject: Varnish virtual memory usage In-Reply-To: <533E1F3F0E92C246A5129A1CF8F6F6940108A952@brain.office.mc> References: <533E1F3F0E92C246A5129A1CF8F6F694F73D77@brain.office.mc> <20091021113424.GE4527@kjeks.linpro.no> <533E1F3F0E92C246A5129A1CF8F6F6940108A8B6@brain.office.mc> <20091021132759.GA3942@kjeks.linpro.no> <533E1F3F0E92C246A5129A1CF8F6F6940108A952@brain.office.mc> Message-ID: <57502.58.28.124.90.1256159396.squirrel@services.directender.co.nz> Can you paste your VCL please? Do you use the purge_url function in it? regards, Darryl Dixon Winterhouse Consulting Ltd http://www.winterhouseconsulting.com > I don't remember how long it took exactly, but it filled up rather > quickly. Probably a couple of hours. I also noticed the LRU-moved stat was > pretty constant all along, no large spikes towards the end of the swap. > > Anyways, it was WAY over 1GB, we managed to fill more around 6-7GB. > > Just for kicks, I tried changing the storage to malloc, and limit it to > 1GB, to see the difference. We have live traffic now, so I don't wanna run > the load testing scripts, but we should hit the 1GB limit within a day or > two. > > -----Original Message----- > From: Kristian Lyngstol [mailto:kristian at redpill-linpro.com] > Sent: Wednesday, October 21, 2009 3:28 PM > To: Roi Avinoam > Cc: varnish-misc at projects.linpro.no > Subject: Re: Varnish virtual memory usage > > On Wed, Oct 21, 2009 at 03:07:00PM +0200, Roi Avinoam wrote: >> Thanks for your reply. >> >> 1. What I did was create 100 simultaneous processes, and each process >> requested the same page (with 'curl'): >> a. Once with the exact same URL - which resulted in a 99.9% hit-ratio >> and VERY high performance on varnish. >> b. And once with a random key that changes the URL (something like >> 'index.php?rand=193837364'), thus forcing Varnish to hit the backend, >> and store the multiple objects in memory. >> c. A combination of the two - in an attempt to maintain a 60-70% >> hit-ratio. >> >> What we saw is that the kernel simply filled all of the RAM *and* the >> swap until it crashed. > > When did it fill up and how fast? > >> 2. I'm sorry, but I'm still confused about the mmaped file. If it's >> limited to 1G, Varnish shouldn't use more than 1G of virtual memory, >> correct? Or in our setup - 1G of RAM? > > The -s parameter only applies to storage, which is to say that it's an > approximiation and there will be some additional overhead which is not > stored there, ie datastructures for threads, sessions, backends etc. > > So with -s ..., 1G, you should expect a little bit over 1GB, but not > nearly as much as you describe. > > -- > Kristian Lyngst?l > Redpill Linpro AS > Tlf: +47 21544179 > Mob: +47 99014497 > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc > From h.paulissen at qbell.nl Wed Oct 21 22:55:03 2009 From: h.paulissen at qbell.nl (Henry Paulissen) Date: Thu, 22 Oct 2009 00:55:03 +0200 Subject: Varnish virtual memory usage In-Reply-To: References: <007401ca5251$9560d420$c0227c60$@paulissen@qbell.nl> Message-ID: <000001ca52a1$86da5bb0$948f1310$@paulissen@qbell.nl> We ran CentOS 5.3 X64 when we noticed this strange behavior. Later on we moved to Fedora core 11 X64. But we where still noticing the same memory allocation problems. Later on we reinstalled the server with vmware to run a couple of (half live a.k.a. beta) tests and noticed it isn?t happening under fedora core 11 x32. We do about 3000 connections/sec for static content (smaller images). For large images (> 200kb), javascript and css we have another instances running (all having the same issues, but im going to tell you about the static content instance). Hitrate is close to the 100% (99-100%). Server core's: 16 Memory: 24GB (VM host server is upgraded to 64GB ram and only doing varnish guests on malloc, so I doubt there's a real performance impact) Tried changing the number of thread_pools and workers, nothing helped. Did the sysctl recommended settings. Disabled conntrack filter in iptables. All incoming requests are with the "connection: close" close header (we have a high availability server above it, who doesn?t allow keep-alive connections. So he transforms every connection to close). Both storage type's where used. I did noticed something when I changed the lru_interval to 60. The reserved memory was keeping within his limits (before this changing this setting it grow way above max limit). But virtual memory is still way above memory the limit. If we didn?t restart varnish every few hours it grow above the physical memory limit and starts using the swap space. If the varnish server was restarted it freed up the memory. Tried both stable and svn versions. My VCL for static: ################################################################################################# ################################################################################################# director staticbackend round-robin { { .backend = { .host = "192.168.x.x"; .port = "x"; .connect_timeout = 2s; .first_byte_timeout = 5s; .between_bytes_timeout = 2s; } } { .backend = { .host = "192.168.x.x"; .port = "x"; .connect_timeout = 2s; .first_byte_timeout = 5s; .between_bytes_timeout = 2s; } } } sub vcl_recv { set req.backend = staticbackend; if (req.request != "GET" && req.request != "HEAD" && req.request != "PUT" && req.request != "POST" && req.request != "TRACE" && req.request != "OPTIO NS" && req.request != "DELETE") { /* Non-RFC2616 or CONNECT which is weird. Shoot this client, but first go in pipeline to the webserver. Maybe he knows what to do with this request. */ return (pipe); } remove req.http.X-Forwarded-For; remove req.http.Accept-Encoding; remove req.http.Accept-Charset; remove req.http.Accept-Language; remove req.http.Referer; remove req.http.Accept; remove req.http.Cookie; return (lookup); } sub vcl_pipe { set req.http.connection = "close"; return (pipe); } sub vcl_pass { return (pass); } sub vcl_hash { set req.hash += req.url; if (req.http.Cache-Control == "no-cache" || req.http.Pragma == "no-cache") { purge_url(req.url); } return (hash); } sub vcl_hit { if (!obj.cacheable) { return(pass); } return (deliver); } sub vcl_miss { return (fetch); } sub vcl_fetch { /* I hate it when varnish cashes my redirects. Some of them are dynamic. */ if (beresp.status == 302 || beresp.status == 301) { return (pass); } remove beresp.http.Set-Cookie; set beresp.grace = 2m; return (deliver); } sub vcl_deliver { remove resp.http.Via; remove resp.http.X-Varnish; if (obj.hits > 0) { set resp.http.X-Cache = "HIT"; } else { set resp.http.X-Cache = "MISS"; } set resp.http.Server = "static"; return (deliver); } sub vcl_error { set obj.http.Content-Type = "text/html; charset=utf-8"; synthetic {" "} obj.status " " obj.response {"

Error "} obj.status " " obj.response {"

"} obj.response {"

Guru Meditation:

XID: "} req.xid {"

"}; return (deliver); } ################################################################################################# ################################################################################################# For further details see my ticket: http://varnish.projects.linpro.no/ticket/546 @Kristian: When the programmers / engineers have some spare time over, they are always welcome to see it in live action. -----Oorspronkelijk bericht----- Van: Ken Brownfield [mailto:kb at slide.com] Verzonden: woensdag 21 oktober 2009 21:57 Aan: Henry Paulissen CC: varnish at projects.linpro.no Onderwerp: Re: Varnish virtual memory usage Small comments: 1) We're running Linux x86_64 exclusively here under significant load, with no memory issues. 2) Why don't you compile a 32-bit version of Varnish; wouldn't this have the same effect without the RAM and performance hit of VMs? 3) Do you make heavy use of purges? -- kb On Oct 21, 2009, at 6:22 AM, Henry Paulissen wrote: > We encounter the same problem. > > Its seems to occur only on x64 platforms. > We decided to take a different approach and installed vmware to the > machine. > Next we did a setup of 6 guests with x32 PAE software. > > No strange memory leaks occurred since then at the price of small > storage (3.5G max) and limited worker threads (256 max). > > Opened a ticket for the problem, but the wont listen until I buy a > support contract (? ?8K). > Seems they don?t want to know there is some kind of memory issue in > their software. > > Anyway... > Varnish is running stable now with some few tricks. > > > Regards, > > -----Original Message----- > From: varnish-misc-bounces at projects.linpro.no [mailto:varnish-misc- > bounces at projects.linpro.no] On Behalf Of Kristian Lyngstol > Sent: woensdag 21 oktober 2009 13:34 > To: Roi Avinoam > Cc: varnish-misc at projects.linpro.no > Subject: Re: Varnish virtual memory usage > > On Mon, Sep 21, 2009 at 02:55:07PM +0300, Roi Avinoam wrote: >> At Metacafe we're testing the integration with Varnish, and I was >> tasked with benchmarking our Varnish setup. I intentionally >> over-flooded the server with requests, in an attempt to see how the >> system will behave under extensive traffic. Surprisingly, the server >> ran out of swap and crashed. > > That seems mighty strange. What sort of tests did you do? > >> In out configuration, "-s file,/var/lib/varnish/varnish_storage.bin, >> 1G". >> Does it mean Varnish shouldn't use more than 1GB of the virtual >> memory? >> Is there any other way to limit the memory/storage usage? > > If you are using -s file and you have 4GB of memory, you are telling > Varnish to create a _file_ of 1GB, and it's up to the kernel what it > keeps in memory or not. If you actually run out of memory with this > setup, you've either hit a bug (need more details first), or you're > doing something strange like having the mmaped file (/var/lib/ > varnish/) in tmpfs with a sizelimit less than 1GB or something along > those lines. But I need more details to say anything for certain. > > -- > Kristian Lyngst?l > Redpill Linpro AS > Tlf: +47 21544179 > Mob: +47 99014497 > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc From scott at idealist.org Thu Oct 22 06:52:03 2009 From: scott at idealist.org (Scott Wilson) Date: Thu, 22 Oct 2009 08:52:03 +0200 Subject: Varnish virtual memory usage In-Reply-To: <-1252439508831938500@unknownmsgid> References: <-1252439508831938500@unknownmsgid> Message-ID: <2e63e54c0910212352r1c822bfekcad8914f79b65d1e@mail.gmail.com> We had a similar problem where varnish would fill all swap and crash every couple of weeks. The trick that seems to have solved the problem was to remove purge.url from our VCL (a lot of badly behaved clients send a lot more no-cache headers than necessary). We replaced purge.url with an approach that sets the object's ttl to zero and restarts the request. The details are here: http://varnish.projects.linpro.no/wiki/VCLExampleEnableForceRefresh In our case we're using FreeBSD 7.2 64-bit. All that said, it doesn't seem that this solution jives with Roi's random url test unless purge.url figured in his vcl / testing script. cheers, scott 2009/10/22 Henry Paulissen : > We ran CentOS 5.3 X64 when we noticed this strange behavior. Later on we moved to Fedora core 11 X64. But we where still noticing the same memory allocation problems. Later on we reinstalled the server with vmware to run a couple of (half live a.k.a. beta) tests and noticed it isn?t happening under fedora core 11 x32. > > We do about 3000 connections/sec for static content (smaller images). For large images (> 200kb), javascript and css we have another instances running (all having the same issues, but im going to tell you about the static content instance). > > Hitrate is close to the 100% (99-100%). > Server core's: 16 > Memory: 24GB (VM host server is upgraded to 64GB ram and only doing varnish guests on malloc, so I doubt there's a real performance impact) > > > Tried changing the number of thread_pools and workers, nothing helped. > Did the sysctl recommended settings. Disabled conntrack filter in iptables. > > All incoming requests are with the "connection: close" close header (we have a high availability server above it, who doesn?t allow keep-alive connections. So he transforms every connection to close). > > Both storage type's where used. > > I did noticed something when I changed the lru_interval to 60. The reserved memory was keeping within his limits (before this changing this setting it grow way above max limit). But virtual memory is still way above memory the limit. > > If we didn?t restart varnish every few hours it grow above the physical memory limit and starts using the swap space. If the varnish server was restarted it freed up the memory. > > Tried both stable and svn versions. > > > My VCL for static: > > ################################################################################################# > ################################################################################################# > > director staticbackend round-robin { > ? ? ? ?{ > ? ? ? ? ? ? ? ?.backend = { > ? ? ? ? ? ? ? ? ? ? ? ?.host = "192.168.x.x"; > ? ? ? ? ? ? ? ? ? ? ? ?.port = "x"; > ? ? ? ? ? ? ? ? ? ? ? ?.connect_timeout = 2s; > ? ? ? ? ? ? ? ? ? ? ? ?.first_byte_timeout = 5s; > ? ? ? ? ? ? ? ? ? ? ? ?.between_bytes_timeout = 2s; > ? ? ? ? ? ? ? ?} > ? ? ? ?} > ? ? ? ?{ > ? ? ? ? ? ? ? ?.backend = { > ? ? ? ? ? ? ? ? ? ? ? ?.host = "192.168.x.x"; > ? ? ? ? ? ? ? ? ? ? ? ?.port = "x"; > ? ? ? ? ? ? ? ? ? ? ? ?.connect_timeout = 2s; > ? ? ? ? ? ? ? ? ? ? ? ?.first_byte_timeout = 5s; > ? ? ? ? ? ? ? ? ? ? ? ?.between_bytes_timeout = 2s; > ? ? ? ? ? ? ? ?} > ? ? ? ?} > } > > sub vcl_recv { > ? ? ? ?set req.backend = staticbackend; > > ? ? ? ?if (req.request != "GET" && req.request != "HEAD" && req.request != "PUT" && req.request != "POST" && req.request != "TRACE" && req.request != "OPTIO > NS" && req.request != "DELETE") { > ? ? ? ? ? ? ? ?/* > ? ? ? ? ? ? ? ? ? ? ? ?Non-RFC2616 or CONNECT which is weird. > ? ? ? ? ? ? ? ? ? ? ? ?Shoot this client, but first go in pipeline to the webserver. > ? ? ? ? ? ? ? ? ? ? ? ?Maybe he knows what to do with this request. > ? ? ? ? ? ? ? ?*/ > > ? ? ? ? ? ? ? ?return (pipe); > ? ? ? ?} > > ? ? ? ?remove req.http.X-Forwarded-For; > ? ? ? ?remove req.http.Accept-Encoding; > ? ? ? ?remove req.http.Accept-Charset; > ? ? ? ?remove req.http.Accept-Language; > ? ? ? ?remove req.http.Referer; > ? ? ? ?remove req.http.Accept; > ? ? ? ?remove req.http.Cookie; > > ? ? ? ?return (lookup); > } > > sub vcl_pipe { > ? ? ? ?set req.http.connection = "close"; > ? ? ? ?return (pipe); > } > > sub vcl_pass { > ? ? ? ?return (pass); > } > > sub vcl_hash { > ? ? ? ?set req.hash += req.url; > > ? ? ? ?if (req.http.Cache-Control == "no-cache" || req.http.Pragma == "no-cache") { > ? ? ? ? ? ? ? ?purge_url(req.url); > ? ? ? ?} > > ? ? ? ?return (hash); > } > > sub vcl_hit { > ? ? ? ?if (!obj.cacheable) { > ? ? ? ? ? ? ? ?return(pass); > ? ? ? ?} > > ? ? ? ?return (deliver); > } > > sub vcl_miss { > ? ? ? ?return (fetch); > } > > sub vcl_fetch { > ? ? ? ?/* > ? ? ? ? ? ?I hate it when varnish cashes my redirects. > ? ? ? ? ? ?Some of them are dynamic. > ? ? ? ?*/ > ? ? ? ?if (beresp.status == 302 || beresp.status == 301) { > ? ? ? ? ? ? ? ?return (pass); > ? ? ? ?} > > ? ? ? ?remove beresp.http.Set-Cookie; > ? ? ? ?set beresp.grace = 2m; > > ? ? ? ?return (deliver); > } > > sub vcl_deliver { > ? ? ? ?remove resp.http.Via; > ? ? ? ?remove resp.http.X-Varnish; > > ? ? ? ?if (obj.hits > 0) { > ? ? ? ? ? ? ? ?set resp.http.X-Cache = "HIT"; > ? ? ? ?} else { > ? ? ? ? ? ? ? ?set resp.http.X-Cache = "MISS"; > ? ? ? ?} > ? ? ? ?set resp.http.Server = "static"; > > ? ? ? ?return (deliver); > } > > sub vcl_error { > ? ? ? ?set obj.http.Content-Type = "text/html; charset=utf-8"; > > ? ? ? ?synthetic {" > > > > ? ? ? ? > ? ? ? ? ? ? ? ?"} obj.status " " obj.response {" > ? ? ? ? > ? ? ? ? > ? ? ? ? ? ? ? ?

Error "} obj.status " " obj.response {"

> ? ? ? ? ? ? ? ?

"} obj.response {"

> ? ? ? ? ? ? ? ?

Guru Meditation:

> ? ? ? ? ? ? ? ?

XID: "} req.xid {"

> ? ? ? ? > > "}; > > ? ? ? ?return (deliver); > } > > ################################################################################################# > ################################################################################################# > > > For further details see my ticket: http://varnish.projects.linpro.no/ticket/546 > > > @Kristian: > When the programmers / engineers have some spare time over, they are always welcome to see it in live action. > > > -----Oorspronkelijk bericht----- > Van: Ken Brownfield [mailto:kb at slide.com] > Verzonden: woensdag 21 oktober 2009 21:57 > Aan: Henry Paulissen > CC: varnish at projects.linpro.no > Onderwerp: Re: Varnish virtual memory usage > > Small comments: > > 1) We're running Linux x86_64 exclusively here under significant load, > with no memory issues. > 2) Why don't you compile a 32-bit version of Varnish; wouldn't this > have the same effect without the RAM and performance hit of VMs? > 3) Do you make heavy use of purges? > -- > kb > > On Oct 21, 2009, at 6:22 AM, Henry Paulissen wrote: > >> We encounter the same problem. >> >> Its seems to occur only on x64 platforms. >> We decided to take a different approach and installed vmware to the >> machine. >> Next we did a setup of 6 guests with x32 PAE software. >> >> No strange memory leaks occurred since then at the price of small >> storage (3.5G max) and limited worker threads (256 max). >> >> Opened a ticket for the problem, but the wont listen until I buy a >> support contract (? ?8K). >> Seems they don?t want to know there is some kind of memory issue in >> their software. >> >> Anyway... >> Varnish is running stable now with some few tricks. >> >> >> Regards, >> >> -----Original Message----- >> From: varnish-misc-bounces at projects.linpro.no [mailto:varnish-misc- >> bounces at projects.linpro.no] On Behalf Of Kristian Lyngstol >> Sent: woensdag 21 oktober 2009 13:34 >> To: Roi Avinoam >> Cc: varnish-misc at projects.linpro.no >> Subject: Re: Varnish virtual memory usage >> >> On Mon, Sep 21, 2009 at 02:55:07PM +0300, Roi Avinoam wrote: >>> At Metacafe we're testing the integration with Varnish, and I was >>> tasked with benchmarking our Varnish setup. I intentionally >>> over-flooded the server with requests, in an attempt to see how the >>> system will behave under extensive traffic. Surprisingly, the server >>> ran out of swap and crashed. >> >> That seems mighty strange. What sort of tests did you do? >> >>> In out configuration, "-s file,/var/lib/varnish/varnish_storage.bin, >>> 1G". >>> Does it mean Varnish shouldn't use more than 1GB of the virtual >>> memory? >>> Is there any other way to limit the memory/storage usage? >> >> If you are using -s file and you have 4GB of memory, you are telling >> Varnish to create a _file_ of 1GB, and it's up to the kernel what it >> keeps in memory or not. If you actually run out of memory with this >> setup, you've either hit a bug (need more details first), or you're >> doing something strange like having the mmaped file (/var/lib/ >> varnish/) in tmpfs with a sizelimit less than 1GB or something along >> those lines. But I need more details to say anything for certain. >> >> -- >> Kristian Lyngst?l >> Redpill Linpro AS >> Tlf: +47 21544179 >> Mob: +47 99014497 >> >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at projects.linpro.no >> http://projects.linpro.no/mailman/listinfo/varnish-misc > > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc > From h.paulissen at qbell.nl Thu Oct 22 08:04:39 2009 From: h.paulissen at qbell.nl (Henry Paulissen) Date: Thu, 22 Oct 2009 10:04:39 +0200 Subject: Varnish virtual memory usage In-Reply-To: <2e63e54c0910212352r1c822bfekcad8914f79b65d1e@mail.gmail.com> References: <-1252439508831938500@unknownmsgid> <2e63e54c0910212352r1c822bfekcad8914f79b65d1e@mail.gmail.com> Message-ID: <00b501ca52ee$4df9a1a0$e9ece4e0$@paulissen@qbell.nl> Awh, Thank you for your comment. I'll make a test case of it tomorrow (or else after the weekend). I will report back. -----Original Message----- From: Scott Wilson [mailto:scott at idealist.org] Sent: donderdag 22 oktober 2009 8:52 To: Henry Paulissen Cc: varnish-misc at projects.linpro.no; kb at slide.com Subject: Re: Varnish virtual memory usage We had a similar problem where varnish would fill all swap and crash every couple of weeks. The trick that seems to have solved the problem was to remove purge.url from our VCL (a lot of badly behaved clients send a lot more no-cache headers than necessary). We replaced purge.url with an approach that sets the object's ttl to zero and restarts the request. The details are here: http://varnish.projects.linpro.no/wiki/VCLExampleEnableForceRefresh In our case we're using FreeBSD 7.2 64-bit. All that said, it doesn't seem that this solution jives with Roi's random url test unless purge.url figured in his vcl / testing script. cheers, scott 2009/10/22 Henry Paulissen : > We ran CentOS 5.3 X64 when we noticed this strange behavior. Later on we moved to Fedora core 11 X64. But we where still noticing the same memory allocation problems. Later on we reinstalled the server with vmware to run a couple of (half live a.k.a. beta) tests and noticed it isn?t happening under fedora core 11 x32. > > We do about 3000 connections/sec for static content (smaller images). For large images (> 200kb), javascript and css we have another instances running (all having the same issues, but im going to tell you about the static content instance). > > Hitrate is close to the 100% (99-100%). > Server core's: 16 > Memory: 24GB (VM host server is upgraded to 64GB ram and only doing varnish guests on malloc, so I doubt there's a real performance impact) > > > Tried changing the number of thread_pools and workers, nothing helped. > Did the sysctl recommended settings. Disabled conntrack filter in iptables. > > All incoming requests are with the "connection: close" close header (we have a high availability server above it, who doesn?t allow keep-alive connections. So he transforms every connection to close). > > Both storage type's where used. > > I did noticed something when I changed the lru_interval to 60. The reserved memory was keeping within his limits (before this changing this setting it grow way above max limit). But virtual memory is still way above memory the limit. > > If we didn?t restart varnish every few hours it grow above the physical memory limit and starts using the swap space. If the varnish server was restarted it freed up the memory. > > Tried both stable and svn versions. > > > My VCL for static: > > ################################################################################################# > ################################################################################################# > > director staticbackend round-robin { > { > .backend = { > .host = "192.168.x.x"; > .port = "x"; > .connect_timeout = 2s; > .first_byte_timeout = 5s; > .between_bytes_timeout = 2s; > } > } > { > .backend = { > .host = "192.168.x.x"; > .port = "x"; > .connect_timeout = 2s; > .first_byte_timeout = 5s; > .between_bytes_timeout = 2s; > } > } > } > > sub vcl_recv { > set req.backend = staticbackend; > > if (req.request != "GET" && req.request != "HEAD" && req.request != "PUT" && req.request != "POST" && req.request != "TRACE" && req.request != "OPTIO > NS" && req.request != "DELETE") { > /* > Non-RFC2616 or CONNECT which is weird. > Shoot this client, but first go in pipeline to the webserver. > Maybe he knows what to do with this request. > */ > > return (pipe); > } > > remove req.http.X-Forwarded-For; > remove req.http.Accept-Encoding; > remove req.http.Accept-Charset; > remove req.http.Accept-Language; > remove req.http.Referer; > remove req.http.Accept; > remove req.http.Cookie; > > return (lookup); > } > > sub vcl_pipe { > set req.http.connection = "close"; > return (pipe); > } > > sub vcl_pass { > return (pass); > } > > sub vcl_hash { > set req.hash += req.url; > > if (req.http.Cache-Control == "no-cache" || req.http.Pragma == "no-cache") { > purge_url(req.url); > } > > return (hash); > } > > sub vcl_hit { > if (!obj.cacheable) { > return(pass); > } > > return (deliver); > } > > sub vcl_miss { > return (fetch); > } > > sub vcl_fetch { > /* > I hate it when varnish cashes my redirects. > Some of them are dynamic. > */ > if (beresp.status == 302 || beresp.status == 301) { > return (pass); > } > > remove beresp.http.Set-Cookie; > set beresp.grace = 2m; > > return (deliver); > } > > sub vcl_deliver { > remove resp.http.Via; > remove resp.http.X-Varnish; > > if (obj.hits > 0) { > set resp.http.X-Cache = "HIT"; > } else { > set resp.http.X-Cache = "MISS"; > } > set resp.http.Server = "static"; > > return (deliver); > } > > sub vcl_error { > set obj.http.Content-Type = "text/html; charset=utf-8"; > > synthetic {" > > > > > "} obj.status " " obj.response {" > > >

Error "} obj.status " " obj.response {"

>

"} obj.response {"

>

Guru Meditation:

>

XID: "} req.xid {"

> > > "}; > > return (deliver); > } > > ################################################################################################# > ################################################################################################# > > > For further details see my ticket: http://varnish.projects.linpro.no/ticket/546 > > > @Kristian: > When the programmers / engineers have some spare time over, they are always welcome to see it in live action. > > > -----Oorspronkelijk bericht----- > Van: Ken Brownfield [mailto:kb at slide.com] > Verzonden: woensdag 21 oktober 2009 21:57 > Aan: Henry Paulissen > CC: varnish at projects.linpro.no > Onderwerp: Re: Varnish virtual memory usage > > Small comments: > > 1) We're running Linux x86_64 exclusively here under significant load, > with no memory issues. > 2) Why don't you compile a 32-bit version of Varnish; wouldn't this > have the same effect without the RAM and performance hit of VMs? > 3) Do you make heavy use of purges? > -- > kb > > On Oct 21, 2009, at 6:22 AM, Henry Paulissen wrote: > >> We encounter the same problem. >> >> Its seems to occur only on x64 platforms. >> We decided to take a different approach and installed vmware to the >> machine. >> Next we did a setup of 6 guests with x32 PAE software. >> >> No strange memory leaks occurred since then at the price of small >> storage (3.5G max) and limited worker threads (256 max). >> >> Opened a ticket for the problem, but the wont listen until I buy a >> support contract (? ?8K). >> Seems they don?t want to know there is some kind of memory issue in >> their software. >> >> Anyway... >> Varnish is running stable now with some few tricks. >> >> >> Regards, >> >> -----Original Message----- >> From: varnish-misc-bounces at projects.linpro.no [mailto:varnish-misc- >> bounces at projects.linpro.no] On Behalf Of Kristian Lyngstol >> Sent: woensdag 21 oktober 2009 13:34 >> To: Roi Avinoam >> Cc: varnish-misc at projects.linpro.no >> Subject: Re: Varnish virtual memory usage >> >> On Mon, Sep 21, 2009 at 02:55:07PM +0300, Roi Avinoam wrote: >>> At Metacafe we're testing the integration with Varnish, and I was >>> tasked with benchmarking our Varnish setup. I intentionally >>> over-flooded the server with requests, in an attempt to see how the >>> system will behave under extensive traffic. Surprisingly, the server >>> ran out of swap and crashed. >> >> That seems mighty strange. What sort of tests did you do? >> >>> In out configuration, "-s file,/var/lib/varnish/varnish_storage.bin, >>> 1G". >>> Does it mean Varnish shouldn't use more than 1GB of the virtual >>> memory? >>> Is there any other way to limit the memory/storage usage? >> >> If you are using -s file and you have 4GB of memory, you are telling >> Varnish to create a _file_ of 1GB, and it's up to the kernel what it >> keeps in memory or not. If you actually run out of memory with this >> setup, you've either hit a bug (need more details first), or you're >> doing something strange like having the mmaped file (/var/lib/ >> varnish/) in tmpfs with a sizelimit less than 1GB or something along >> those lines. But I need more details to say anything for certain. >> >> -- >> Kristian Lyngst?l >> Redpill Linpro AS >> Tlf: +47 21544179 >> Mob: +47 99014497 >> >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at projects.linpro.no >> http://projects.linpro.no/mailman/listinfo/varnish-misc > > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc > From tfheen at redpill-linpro.com Fri Oct 23 06:00:57 2009 From: tfheen at redpill-linpro.com (Tollef Fog Heen) Date: Fri, 23 Oct 2009 08:00:57 +0200 Subject: varnish 2.0.4 questions - no IMS, no persistence cache - please help In-Reply-To: (GaneshKumar Natarajan's message of "Tue, 20 Oct 2009 15:26:30 -0400") References: Message-ID: <87hbtqhco6.fsf@qurzaw.linpro.no> ]] GaneshKumar Natarajan | 1. varnish doesn't send IMS request, when the object expires. Correct. This is somewhat hard to do, but we see the request coming in regularly so it's something we might want to change at some point. I don't believe there is any way to achieve this using VCL. | 2. Varnish cache is not persistent. Correct, this is being fixed for 2.1. (It's the major feature for 2.1.) | 3. mmap storage : max i can configure is 340 GB. | | detail:: | I was able to use only 340 GB of cache. any size after this, i would get error. | child (25790) Started | Pushing vcls failed: dlopen(./vcl.1P9zoqAU.so): ./vcl.1P9zoqAU.so: | failed to map segment from shared object: Cannot allocate memory This looks weird. Can you tell us something about what OS and kernel versions you are running? | 4. I tried malloc storage option. | Question: | i have system of 32 GB physical ram and my swap storage is 65 GB. | harddisk storage - 1.5 TB ( distribued in 5 cache directoreys | /cach0,.../cache1 ). | What is the best optimal malloc storage i can use to achive greater performance? | -s malloc, [ size ] ??? It depends on your setup and your traffic patterns, so it's usually best to try some different ones. | 5. what is the difference between using malloc or mmap? | | Help me to move onto varnish world and convince others in my team. malloc is, well, using malloc to allocate the memory, letting the system swap handle any page-outs. mmap is just mmap-ing a large file, which still leaves it to the system to page out the unused bits, but often somewhat different code paths. Which one is better depends on OS, traffic pattern, amount of memory, etc. -- Tollef Fog Heen Redpill Linpro -- Changing the game! t: +47 21 54 41 73 From h.paulissen at qbell.nl Fri Oct 23 07:49:39 2009 From: h.paulissen at qbell.nl (Henry Paulissen) Date: Fri, 23 Oct 2009 09:49:39 +0200 Subject: Varnish virtual memory usage Message-ID: <000301ca53b5$5fea7f70$1fbf7e50$@paulissen@qbell.nl> Didnt did the trick though. ############################################################################ ##################### ############################################################################ ##################### uptime 13662 . Child uptime client_conn 959755 70.25 Client connections accepted client_drop 0 0.00 Connection dropped, no sess client_req 959747 70.25 Client requests received cache_hit 470881 34.47 Cache hits cache_hitpass 0 0.00 Cache hits for pass cache_miss 91401 6.69 Cache misses backend_conn 402367 29.45 Backend conn. success backend_unhealthy 0 0.00 Backend conn. not attempted backend_busy 0 0.00 Backend conn. too many backend_fail 15 0.00 Backend conn. failures backend_reuse 87098 6.38 Backend conn. reuses backend_toolate 7263 0.53 Backend conn. was closed backend_recycle 94363 6.91 Backend conn. recycles backend_unused 0 0.00 Backend conn. unused fetch_head 0 0.00 Fetch head fetch_length 475447 34.80 Fetch with Length fetch_chunked 5867 0.43 Fetch chunked fetch_eof 0 0.00 Fetch EOF fetch_bad 0 0.00 Fetch had bad headers fetch_close 0 0.00 Fetch wanted close fetch_oldhttp 0 0.00 Fetch pre HTTP/1.1 closed fetch_zero 0 0.00 Fetch zero len fetch_failed 1 0.00 Fetch failed n_sess_mem 495 . N struct sess_mem n_sess 46 . N struct sess n_object 41675 . N struct object n_vampireobject 0 . N unresurrected objects n_objectcore 41902 . N struct objectcore n_objecthead 35695 . N struct objecthead n_smf 0 . N struct smf n_smf_frag 0 . N small free smf n_smf_large 0 . N large free smf n_vbe_conn 8 . N struct vbe_conn n_wrk 1600 . N worker threads n_wrk_create 1600 0.12 N worker threads created n_wrk_failed 0 0.00 N worker threads not created n_wrk_max 0 0.00 N worker threads limited n_wrk_queue 0 0.00 N queued work requests n_wrk_overflow 809 0.06 N overflowed work requests n_wrk_drop 0 0.00 N dropped work requests n_backend 5 . N backends n_expired 49304 . N expired objects n_lru_nuked 0 . N LRU nuked objects n_lru_saved 0 . N LRU saved objects n_lru_moved 198381 . N LRU moved objects n_deathrow 0 . N objects on deathrow losthdr 21 0.00 HTTP header overflows n_objsendfile 0 0.00 Objects sent with sendfile n_objwrite 954443 69.86 Objects sent with write n_objoverflow 0 0.00 Objects overflowing workspace s_sess 959749 70.25 Total Sessions s_req 959747 70.25 Total Requests s_pipe 0 0.00 Total pipe s_pass 398061 29.14 Total pass s_fetch 481313 35.23 Total fetch s_hdrbytes 327272320 23954.93 Total header bytes s_bodybytes 1551538833 113566.01 Total body bytes sess_closed 959740 70.25 Session Closed sess_pipeline 0 0.00 Session Pipeline sess_readahead 0 0.00 Session Read Ahead sess_linger 0 0.00 Session Linger sess_herd 9 0.00 Session herd shm_records 64046389 4687.92 SHM records shm_writes 4351501 318.51 SHM writes shm_flushes 0 0.00 SHM flushes due to overflow shm_cont 4212 0.31 SHM MTX contention shm_cycles 26 0.00 SHM cycles through buffer sm_nreq 0 0.00 allocator requests sm_nobj 0 . outstanding allocations sm_balloc 0 . bytes allocated sm_bfree 0 . bytes free sma_nreq 572071 41.87 SMA allocator requests sma_nobj 83345 . SMA outstanding allocations sma_nbytes 499243475 . SMA outstanding bytes sma_balloc 1781391844 . SMA bytes allocated sma_bfree 1282148369 . SMA bytes free sms_nreq 8170 0.60 SMS allocator requests sms_nobj 0 . SMS outstanding allocations sms_nbytes 0 . SMS outstanding bytes sms_balloc 4770776 . SMS bytes allocated sms_bfree 4770776 . SMS bytes freed backend_req 486293 35.59 Backend requests made n_vcl 1 0.00 N vcl total n_vcl_avail 1 0.00 N vcl available n_vcl_discard 0 0.00 N vcl discarded n_purge 1 . N total active purges n_purge_add 1 0.00 N new purges added n_purge_retire 0 0.00 N old purges deleted n_purge_obj_test 0 0.00 N objects tested n_purge_re_test 0 0.00 N regexps tested against n_purge_dups 0 0.00 N duplicate purges removed hcb_nolock 0 0.00 HCB Lookups without lock hcb_lock 0 0.00 HCB Lookups with lock hcb_insert 0 0.00 HCB Inserts esi_parse 0 0.00 Objects ESI parsed (unlock) esi_errors 0 0.00 ESI parse errors (unlock) ############################################################################ ##################### ############################################################################ ##################### ############################################################################ ##################### ############################################################################ ##################### top - 03:35:50 up 23 days, 17:24, 1 user, load average: 0.44, 0.43, 0.43 Tasks: 377 total, 2 running, 374 sleeping, 0 stopped, 1 zombie Cpu0 : 0.0%us, 0.3%sy, 0.0%ni, 99.7%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Cpu1 : 0.6%us, 0.6%sy, 0.0%ni, 98.1%id, 0.0%wa, 0.0%hi, 0.6%si, 0.0%st Cpu2 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Cpu3 : 1.2%us, 0.9%sy, 0.0%ni, 96.9%id, 0.0%wa, 0.0%hi, 0.9%si, 0.0%st Cpu4 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Cpu5 : 4.4%us, 0.9%sy, 0.0%ni, 94.4%id, 0.0%wa, 0.0%hi, 0.3%si, 0.0%st Cpu6 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Cpu7 : 0.3%us, 0.0%sy, 0.0%ni, 99.4%id, 0.3%wa, 0.0%hi, 0.0%si, 0.0%st Cpu8 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Cpu9 : 2.5%us, 0.9%sy, 0.0%ni, 96.0%id, 0.0%wa, 0.0%hi, 0.6%si, 0.0%st Cpu10 : 8.4%us, 0.9%sy, 0.0%ni, 90.7%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Cpu11 : 2.2%us, 0.9%sy, 0.0%ni, 95.6%id, 0.3%wa, 0.0%hi, 0.9%si, 0.0%st Cpu12 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Cpu13 : 2.2%us, 0.9%sy, 0.0%ni, 96.0%id, 0.0%wa, 0.0%hi, 0.9%si, 0.0%st Cpu14 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Cpu15 : 0.6%us, 0.3%sy, 0.0%ni, 98.8%id, 0.0%wa, 0.0%hi, 0.3%si, 0.0%st Mem: 24675740k total, 20720232k used, 3955508k free, 792060k buffers Swap: 16779884k total, 70380k used, 16709504k free, 13771132k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 7634 nobody 15 0 5929m 2.3g 81m S 2.2 9.6 4:56.76 /usr/local/varnish/sbin/varnishd -P /var/run/varnish_xxx.pid -a 0.0.0.0:607 -f /usr/local/varnish/etc/varnish/xxx.xxx.xxx.vcl -T 0.0.0.0:707 -s malloc,10G -i xxx -n /usr/local/varnish/var/va 29144 nobody 15 0 2620m 486m 81m S 1.2 2.0 3:19.36 /usr/local/varnish/sbin/varnishd -P /var/run/varnish_yyy.pid -a 0.0.0.0:604 -f /usr/local/varnish/etc/varnish/yyy.yyy.yyy.vcl -T 0.0.0.0:704 -s malloc,1G -i yyy -n /u 11229 nobody 15 0 2008m 221m 80m S 0.9 0.9 117:35.14 /usr/local/varnish/sbin/varnishd -P /var/run/varnish_zzz.pid -a 0.0.0.0:606 -f /usr/local/varnish/etc/varnish/zzz.zzz.zzz.vcl -T 0.0.0.0:706 -s malloc,512m -i zzz -n /usr/loca 29308 nobody 15 0 1520m 195m 150m S 0.6 0.8 2:23.04 /usr/local/varnish/sbin/varnishd -P /var/run/varnish_aaa.pid -a 0.0.0.0:605 -f /usr/local/varnish/etc/varnish/aaa.aaa.aaa.vcl -T 0.0.0.0:705 -s file,/usr/local/varnish/var/varnish/flv/cache. 7646 root 15 0 116m 96m 80m S 0.0 0.4 0:00.00 /usr/local/varnish/bin/varnishncsa -afcD -n www -w /usr/local/varnish/var/varnish/xxx/access_log -P /var/run/varnishnsca_xxx.pid 29312 root 15 0 101m 81m 80m S 0.0 0.3 0:00.00 /usr/local/varnish/bin/varnishncsa -afcD -n flv -w /usr/local/varnish/var/varnish/yyy/access_log -P /var/run/varnishnsca_yyy.pid 29150 root 15 0 101m 81m 80m S 0.0 0.3 0:00.00 /usr/local/varnish/bin/varnishncsa -afcD -n photos_html -w /usr/local/varnish/var/varnish/zzz/access_log -P /var/run/varnishnsca_zzz.pid ############################################################################ ##################### ############################################################################ ##################### See varnish_yyy that one is configured without the purge and malloc is set to use 1G max. I could except a usage from 1.5G to 2G max. But it`s using 2.6G within 4 hours and will not decrease over time (it will only grow). My startup command: ############################################################################ ##################### ############################################################################ ##################### $exec -P $pidfile -a 0.0.0.0:${proxy_port} -f ${configfile} -T 0.0.0.0:${management_port} -s ${cachefile} -i ${INSTANCE_NAME} -n ${programdir}/var/varnish/${INSTANCE_NAME} -p obj_workspace=8192 -p sess_workspace=262144 -p listen_depth=8192 -p lru_interval=60 -p sess_timeout=10 -p shm_workspace=32768 -p ping_interval=2 -p thread_pools=16 -p thread_pool_min=100 -p thread_pool_max=4000 -p esi_syntax=1 -p overflow_max=10000 > /dev/null 2>&1 ############################################################################ ##################### ############################################################################ ##################### Note that adding the '-p lru_interval=60' did change the memory problem. The memory usage shifted from reserved memory to the virtual memory balloon. Anyway, it's a good tip to use the ttl=0 and restart instead of purge. Regards, From tfheen at redpill-linpro.com Fri Oct 23 08:28:41 2009 From: tfheen at redpill-linpro.com (Tollef Fog Heen) Date: Fri, 23 Oct 2009 10:28:41 +0200 Subject: if-none-match status In-Reply-To: <100657c90910101041p166acd60k1cd00a58f0af69ec@mail.gmail.com> (=?utf-8?Q?=22Rog=C3=A9rio?= Schneider"'s message of "Sat, 10 Oct 2009 14:41:14 -0300") References: <20090910120356.27ae58af@der-dieb> <20090913191412.4a30b791@der-dieb> <20090921102952.5277153a@der-dieb> <20091009113535.1f07bfb0@der-dieb> <100657c90910091140j19a06451obcbb34bf06b1d37d@mail.gmail.com> <20091009115359.52dbc7ba@der-dieb> <100657c90910091226q31be9664ya8fd50025a478a17@mail.gmail.com> <20091009144114.7dfbbce1@der-dieb> <100657c90910091506g746940aej4fe5ce67a67904b6@mail.gmail.com> <20091009165211.32e3a96e@der-dieb> <100657c90910101041p166acd60k1cd00a58f0af69ec@mail.gmail.com> Message-ID: <8763a6h5ty.fsf@qurzaw.linpro.no> ]] Rog?rio Schneider | 2009/10/9 Joe Williams : | > | > Thanks Rog?rio, I am totally aware that it currently is "client <-> | > varnish" communication that deals with if-none-match. My question is | | ah, ok :) | | > whether it was considered making an option for doing this on "varnish | > <-> backend" communication as well since it seems like a reasonable way | > to validate objects in the cache during lookups. Does the varnish | > project have a (official or unofficial) policy regarding doing | > backend communication with conditions? | | Now that is a question I think others could answer (tollef, sky, | phk?). I really do not know. It's somewhat hard to do and not really on the list for now. If somebody shows up with a patch or money to fund the development, we can take a look at it, but for now, it's nowhere near the top of list of features we want to implement. [...] | Needless to say, this could be valid and very useful. Absolutely, it's a completely valid feature request. -- Tollef Fog Heen Redpill Linpro -- Changing the game! t: +47 21 54 41 73 From tfheen at redpill-linpro.com Fri Oct 23 09:51:14 2009 From: tfheen at redpill-linpro.com (Tollef Fog Heen) Date: Fri, 23 Oct 2009 11:51:14 +0200 Subject: backend health checks and 204 In-Reply-To: (David Birdsong's message of "Thu, 8 Oct 2009 16:06:22 -0700") References: Message-ID: <87y6n2fnfx.fsf@qurzaw.linpro.no> ]] David Birdsong | I couldn't find this anywhere, so apologies if it's already been hashed out. | | Would it be possible for the backend health checks to be altered to | consider a 204 response from a backend as healthy? The attached patch does that. It's against trunk. phk, please review and ack/nack. (It needs documentation and possibly a test case.) -- Tollef Fog Heen Redpill Linpro -- Changing the game! t: +47 21 54 41 73 -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Add-support-for-setting-the-expected-response-for-ba.patch Type: text/x-diff Size: 3346 bytes Desc: not available URL: From christer.solskogen at gmail.com Sun Oct 25 01:54:25 2009 From: christer.solskogen at gmail.com (Christer Solskogen) Date: Sun, 25 Oct 2009 02:54:25 +0100 Subject: varnish in a freebsd jail Message-ID: I'm setting up a FreeBSD jail to run varnish in. And when I start varnish I get this: "Notice: locking SHMFILE in core failed: Operation not permitted" - I've seen this before, but that was when varnish was not running in a jail. Clues anyone? -- chs From phk at phk.freebsd.dk Sun Oct 25 08:18:19 2009 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Sun, 25 Oct 2009 08:18:19 +0000 Subject: varnish in a freebsd jail In-Reply-To: Your message of "Sun, 25 Oct 2009 02:54:25 +0100." Message-ID: <5830.1256458699@critter.freebsd.dk> In message , Christer Solskogen writes: >I'm setting up a FreeBSD jail to run varnish in. And when I start >varnish I get this: "Notice: locking SHMFILE in core failed: Operation >not permitted" - I've seen this before, but that was when varnish was >not running in a jail. It's not important, don't worry about it. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From itisgany at gmail.com Mon Oct 26 19:55:12 2009 From: itisgany at gmail.com (GaneshKumar Natarajan) Date: Mon, 26 Oct 2009 15:55:12 -0400 Subject: varnish 2.0.4 questions - no IMS, no persistence cache - please help In-Reply-To: <87hbtqhco6.fsf@qurzaw.linpro.no> References: <87hbtqhco6.fsf@qurzaw.linpro.no> Message-ID: Thanks Tollef Fog Heen for the response. > | 3. mmap storage : max i can configure is 340 GB. > This looks weird. Can you tell us something about what OS and kernel > versions you are running? Please find the details below. Let me know what i could try to increase mmap storage from my current limitation of 340 G. details: OS: Red Hat Enterprise Linux AS release 4 (Nahant Update 8) uname -a Linux 2.6.9-89.ELsmp #1 SMP Mon Apr 20 10:33:05 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux disk: /dev/mapper/vg0-cache0 173 1 164 1% /squid/cache0 /dev/mapper/vg1-cache1 275 6 255 3% /squid/cache1 /dev/mapper/vg2-cache2 275 6 255 3% /squid/cache2 /dev/mapper/vg3-cache3 275 6 255 3% /squid/cache3 /dev/mapper/vg4-cache4 275 6 255 3% /squid/cache4 /dev/mapper/vg5-cache5 275 6 255 3% /squid/cache5 [icds at mdtnj120sics ~]$ cat /proc/meminfo MemTotal: 32939168 kB MemFree: 154296 kB Buffers: 476836 kB Cached: 16745252 kB SwapCached: 0 kB Active: 28386124 kB Inactive: 1939176 kB HighTotal: 0 kB HighFree: 0 kB LowTotal: 32939168 kB LowFree: 154296 kB SwapTotal: 67108856 kB SwapFree: 67108616 kB Dirty: 0 kB Writeback: 0 kB Mapped: 13124212 kB Slab: 2374916 kB CommitLimit: 83578440 kB Committed_AS: 14006428 kB PageTables: 28520 kB VmallocTotal: 536870911 kB VmallocUsed: 295936 kB VmallocChunk: 536574591 kB HugePages_Total: 0 HugePages_Free: 0 Hugepagesize: 2048 kB > | 3. mmap storage : max i can configure is 340 GB. > | > | detail:: > | I was able to use only 340 GB of cache. any size after this, i would get error. > | child (25790) Started > | Pushing vcls failed: dlopen(./vcl.1P9zoqAU.so): ./vcl.1P9zoqAU.so: > | failed to map segment from shared object: Cannot allocate memory > > This looks weird. Can you tell us something about what OS and kernel > versions you are running? On Fri, Oct 23, 2009 at 2:00 AM, Tollef Fog Heen wrote: > ]] GaneshKumar Natarajan > > | 1. varnish doesn't send IMS request, when the object expires. > > Correct. ?This is somewhat hard to do, but we see the request coming in > regularly so it's something we might want to change at some point. ?I > don't believe there is any way to achieve this using VCL. > > | 2. Varnish cache is not persistent. > > Correct, this is being fixed for 2.1. ?(It's the major feature for 2.1.) > > | 3. mmap storage : max i can configure is 340 GB. > | > | detail:: > | I was able to use only 340 GB of cache. any size after this, i would get error. > | child (25790) Started > | Pushing vcls failed: dlopen(./vcl.1P9zoqAU.so): ./vcl.1P9zoqAU.so: > | failed to map segment from shared object: Cannot allocate memory > > This looks weird. ?Can you tell us something about what OS and kernel > versions you are running? > > | 4. I tried malloc storage option. > | Question: > | i have system of 32 GB physical ram and my swap storage is 65 GB. > | harddisk storage - 1.5 TB ( distribued in 5 cache directoreys > | /cach0,.../cache1 ). > | What is the best optimal malloc storage i can use to achive greater performance? > | -s malloc, [ size ] ??? > > It depends on your setup and your traffic patterns, so it's usually best > to try some different ones. > > | 5. what is the difference between using malloc or mmap? > | > | Help me to move onto varnish world and convince others in my team. > > malloc is, well, using malloc to allocate the memory, letting the system > swap handle any page-outs. ?mmap is just mmap-ing a large file, which > still leaves it to the system to page out the unused bits, but often > somewhat different code paths. ?Which one is better depends on OS, > traffic pattern, amount of memory, etc. > > -- > Tollef Fog Heen > Redpill Linpro -- Changing the game! > t: +47 21 54 41 73 > -- Regards, Gany From tfheen at redpill-linpro.com Tue Oct 27 08:12:07 2009 From: tfheen at redpill-linpro.com (Tollef Fog Heen) Date: Tue, 27 Oct 2009 09:12:07 +0100 Subject: varnish 2.0.4 questions - no IMS, no persistence cache - please help In-Reply-To: (GaneshKumar Natarajan's message of "Mon, 26 Oct 2009 15:55:12 -0400") References: <87hbtqhco6.fsf@qurzaw.linpro.no> Message-ID: <87y6mxuug8.fsf@qurzaw.linpro.no> ]] GaneshKumar Natarajan | Thanks Tollef Fog Heen for the response. | > | 3. mmap storage : max i can configure is 340 GB. | > This looks weird. Can you tell us something about what OS and kernel | > versions you are running? | | Please find the details below. Let me know what i could try to | increase mmap storage from my current limitation of 340 G. | details: Does it work if you run it with multiple -s arguments that are each smaller? Can you paste the output of ulimit -a? -- Tollef Fog Heen Redpill Linpro -- Changing the game! t: +47 21 54 41 73 From quasirob at googlemail.com Tue Oct 27 13:27:18 2009 From: quasirob at googlemail.com (Rob Ayres) Date: Tue, 27 Oct 2009 13:27:18 +0000 Subject: varnish with authirsed requests Message-ID: Hi, My client machine (actually another server generating requests) is creating POST requests to varnish which also include basic HTTP authentication. I've got varnish caching POST requests from my browser by changing the line in vcl_recv to this: if (req.request != "GET" && req.request != "HEAD" && req.request != "POST") So caching POSTs isnt a problem but I cant get it to cache the client requests from the client server because it is using authorisation (the authorisation is hard coded ) In vcl_recv I also commented out the "if (req.http.Authorization || req.http.Cookie) {" section and also added "unset req.http.Authorization;" but neither made any difference, every request shows up as a "Cache hits for pass". I'd be grateful for any suggestions anyone has. Rob -------------- next part -------------- An HTML attachment was scrubbed... URL: From itisgany at gmail.com Tue Oct 27 21:13:25 2009 From: itisgany at gmail.com (GaneshKumar Natarajan) Date: Tue, 27 Oct 2009 17:13:25 -0400 Subject: varnish 2.0.4 questions - no IMS, no persistence cache - please help In-Reply-To: <87y6mxuug8.fsf@qurzaw.linpro.no> References: <87hbtqhco6.fsf@qurzaw.linpro.no> <87y6mxuug8.fsf@qurzaw.linpro.no> Message-ID: > Does it work if you run it with multiple -s arguments that are each > smaller? Can you paste the output of ulimit -a? Yes, am able to start with multiple -s mmap option. -s file,/squid/cache1,60G \ -s file,/squid/cache2,60G \ -s file,/squid/cache3,60G \ -s file,/squid/cache4,60G \ if total exceeds 340G, issue is seen. ulimit -a core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited file size (blocks, -f) unlimited pending signals (-i) 1024 max locked memory (kbytes, -l) 32 max memory size (kbytes, -m) unlimited open files (-n) 65535 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 stack size (kbytes, -s) 10240 cpu time (seconds, -t) unlimited max user processes (-u) 278528 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited On Tue, Oct 27, 2009 at 4:12 AM, Tollef Fog Heen wrote: > ]] GaneshKumar Natarajan > > | Thanks ?Tollef Fog Heen for the response. > | > | 3. mmap storage : max i can configure is 340 GB. > | > This looks weird. ?Can you tell us something about what OS and kernel > | > versions you are running? > | > | Please find the details below. Let me know what i could try to > | increase mmap storage from my current limitation of 340 G. > | details: > > Does it work if you run it with multiple -s arguments that are each > smaller? ?Can you paste the output of ulimit -a? > > -- > Tollef Fog Heen > Redpill Linpro -- Changing the game! > t: +47 21 54 41 73 > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc > -- Regards, Gany From bernard at frit.net Thu Oct 29 12:49:33 2009 From: bernard at frit.net (Bernardf FRIT) Date: Thu, 29 Oct 2009 13:49:33 +0100 Subject: Trying to use X-Backend to select backend Message-ID: <4AE98F5D.2090300@frit.net> Hi, I'm mostly a newby. I downgraded varnish to v 2.0.3 in order to use the varnish-bereq-hosts.patch I want to : 1. Store the backend name into X-Backend custom header 2. Force each request with a X-Backend header to be directed to the stored backend name My .vcl file : sub vcl_recv { .../... if (req.http.host ~ "^(www.)?mysite.com$") { if (req.http.X-Backend) { set req.backend = req.http.X-Backend ; } else { set req.backend = www_director; } } ../.. sub vcl_fetch {. ../.. set obj.http.X-Backend = bereq.backend; ../.. And I got the following error : Message from VCC-compiler: Expected ID got 'req.http.X-Backend' (program line 233), at (input Line 54 Pos 27) set req.backend = req.http.X-Backend ; --------------------------##################-- I don't know how to transfer X-Backend value into req.backend. Can you help me please ? Best regards. -- Bernard FRIT -------------- next part -------------- An HTML attachment was scrubbed... URL: From v.bilek at 1art.cz Thu Oct 29 14:28:11 2009 From: v.bilek at 1art.cz (=?UTF-8?B?VsOhY2xhdiBCw61sZWs=?=) Date: Thu, 29 Oct 2009 15:28:11 +0100 Subject: Write error, len = 69696/260844, errno = Success Message-ID: <4AE9A67B.5000706@1art.cz> Hi We are solving strange problem with smol part of clients ( aprox 1%) on some objects they do not get all the content in varnishlog we see: Write error, len = 69696/260844, errno = Success in the archive of this mailing list i found that client ended connection before end of transfer... tcpdump we made confirms that... the question is WHY our investigation found no connection between OS, browser or anything else; the only thing we know is that straight get from apache works and get from varnish doesn't ( the network path in both cases is the same, no mather if it gets from cache or if it is passed ) When client tryes to reget the content it ends the same way (Write error, len = ...) in atachements is tcpdump of such connection and coresponding varnishlog output, Is there anyone who can point us where to look to find the problem? thanks for any respond Vaclav Bilek -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: proxy.log.1 URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: proxy1.tcpdump Type: application/octet-stream Size: 8816 bytes Desc: not available URL: From tfheen at redpill-linpro.com Fri Oct 30 09:02:54 2009 From: tfheen at redpill-linpro.com (Tollef Fog Heen) Date: Fri, 30 Oct 2009 10:02:54 +0100 Subject: Trying to use X-Backend to select backend In-Reply-To: <4AE98F5D.2090300@frit.net> (Bernardf FRIT's message of "Thu, 29 Oct 2009 13:49:33 +0100") References: <4AE98F5D.2090300@frit.net> Message-ID: <87y6mtw8xt.fsf@qurzaw.linpro.no> ]] Bernardf FRIT | I downgraded varnish to v 2.0.3 in order to use the | varnish-bereq-hosts.patch Which patch is this? | 1. Store the backend name into X-Backend custom header | 2. Force each request with a X-Backend header to be directed to the | stored backend name There is currently no way to look up backends by name, so you would have to write this as a series of if statements. -- Tollef Fog Heen Redpill Linpro -- Changing the game! t: +47 21 54 41 73 From olau at iola.dk Fri Oct 30 13:27:49 2009 From: olau at iola.dk (Ole Laursen) Date: Fri, 30 Oct 2009 13:27:49 +0000 (UTC) Subject: Varnish on shared virtual server with limited memory Message-ID: Hi fellow painters, I'm using Varnish on a shared hosting site on a virtual server. Has been working like a charm, however, I have for some time been wondering why ssh'ing into the box and loading seldomly loaded pages seems to have gotten significantly slower (maybe 10 seconds). If I look at the memory usage, Varnish has allocated about 1 GB virtual memory and 600 MB resident. I think there's about 700 MB memory available on the virtual server. So maybe Varnish is pushing some of the less frequently used applications into swap (or out of the file cache)? Is there some way to fix this other than to decrease the cache size? This is on a Linux server (2.6.9) with Varnish started with "-s file,/var/.../varnish_storage.bin,1G". Coming to think of it, it's a bit odd that there's 600 MB resident. I don't think the working set size for say a week is more than maybe 100 MB. The sites hosted on this server are quite small, and all static data has a "pass" rule so shouldn't enter the cache. Ole From bernard at frit.net Fri Oct 30 17:23:31 2009 From: bernard at frit.net (Bernardf FRIT) Date: Fri, 30 Oct 2009 18:23:31 +0100 Subject: Trying to use X-Backend to select backend In-Reply-To: <87y6mtw8xt.fsf@qurzaw.linpro.no> References: <4AE98F5D.2090300@frit.net> <87y6mtw8xt.fsf@qurzaw.linpro.no> Message-ID: <4AEB2113.1010808@frit.net> Tollef Fog Heen : Thanks for your answer > | I downgraded varnish to v 2.0.3 in order to use the > | varnish-bereq-hosts.patch > > Which patch is this? http://varnish.projects.linpro.no/attachment/ticket/481/ > | 1. Store the backend name into X-Backend custom header > | 2. Force each request with a X-Backend header to be directed to the > | stored backend name > > There is currently no way to look up backends by name, so you would have > to write this as a series of if statements Yes it worked but didn't do the job I expected. After first request to the site I want the browser to be always directed to the same backend (due to sessionId management). I think I have to use Cookies to achieve this or just serve static content with varnish and use haproxy as backend for dynamic content. I'm just wondering which solution is best : - varnish as frontend to haproxy - haproxy as frontend to varnish -- Bernard FRIT