From arianna.aondio at varnish-software.com Wed Nov 5 12:26:28 2014 From: arianna.aondio at varnish-software.com (Arianna Aondio) Date: Wed, 5 Nov 2014 13:26:28 +0100 Subject: New counters for VBE. Message-ID: Hi all, please find attached a patch that adds two per-backend counters to VSC_DO_VBE: - conn : gauge which represents the number of concurrent connections to backend. - req : counter which represents the number of requests sent. It has to be applied to the master branch. The change has been driven by costumer demand, the aim is to improve the backends' info and provide a better overview over backends performances. -- Arianna Aondio Software Developer | Varnish Software AS Mobile: +47 980 62 619 We Make Websites Fly www.varnish-software.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: New-counters-for-VBE.patch Type: text/x-patch Size: 2207 bytes Desc: not available URL: From fgsch at lodoss.net Wed Nov 5 13:23:20 2014 From: fgsch at lodoss.net (Federico Schwindt) Date: Wed, 5 Nov 2014 13:23:20 +0000 Subject: [PATCH] make bereq.uncacheable read-only Message-ID: Hi, I've been spending time writing tests for some other work I'm doing and I found bereq.uncacheable being writable, which doesn't make much sense to me. The same can be achieved setting beresp.uncacheable in vcl_backend_response or vcl_backend_error so this duplication seems unnecessary and having a single place to set whether the object is uncacheable is better. Comments? OK? -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001.bereq_uncacheable.patch Type: text/x-patch Size: 1988 bytes Desc: not available URL: From fgsch at lodoss.net Wed Nov 5 14:22:40 2014 From: fgsch at lodoss.net (Federico Schwindt) Date: Wed, 5 Nov 2014 14:22:40 +0000 Subject: [PATCH] make obj.uncacheable available in deliver Message-ID: Hi, Someone mentioned on irc that there is no to check whether the object was a pass (or hit-for-pass) short of adding headers in a few vcl subroutines. There is this obj.uncacheable attribute but it's only available in vcl_hit. This doesn't make much sense. vcl_hit will only be reached on hit, not on hit-for-pass. Oops! This patch makes obj.uncacheable available in deliver so you can find in a common place whether the request was a pass (or hit-for-pass) or hit. I've removed it from vcl_hit as always true. This patch builds upon my previous patch, btw. Comments? OK? -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001.move-obj_uncacheable.patch Type: text/x-patch Size: 1067 bytes Desc: not available URL: From slink at schokola.de Wed Nov 5 16:51:24 2014 From: slink at schokola.de (Nils Goroll) Date: Wed, 05 Nov 2014 17:51:24 +0100 Subject: [PATCH] make bereq.uncacheable read-only In-Reply-To: References: Message-ID: <545A558C.8020900@schokola.de> isn't the point of bereq.uncacheable to get a "late pass"? On 05/11/14 14:23, Federico Schwindt wrote: > > I've been spending time writing tests for some other work I'm doing and I found > bereq.uncacheable being writable, which doesn't make much sense to me. From daghf at varnish-software.com Tue Nov 11 16:41:00 2014 From: daghf at varnish-software.com (Dag Haavi Finstad) Date: Tue, 11 Nov 2014 17:41:00 +0100 Subject: PROXY protocol revisited Message-ID: Hi guys I've been looking into implementing the PROXY protocol[1] for Varnish 4. This has been discussed several times before, in particular it seems some consensus was found at VDD13Q4 in Berlin last year [2]. This was regarding the interface in terms of specifying the listening socket (-a) and the VCL *.ip bits. I was not present at this VDD myself, so I'd just like to discuss and perhaps get some verification as to what was agreed. As for the -a listening argument, this ties into how it should be handled for HTTP/2. From reading the notes, the agreed upon syntax was -a PROTO at IP:0 Is the PROTO@ part mandatory, or is there a fallback when it's left out (e.g. plain old '-a :80')? I think it makes most sense to have the fallback value be HTTP/1.1 that also supports HTTP/2 via Upgrade [3]. If we are also going to have a way of specifying the protocol to be exclusively HTTP/1 or HTTP/2 [4], we could use values 'http1' and 'http2' to denote that. Further, a PROXY protocol listen socket is specified like this: -a proxy at 192.168.1.10:8081 The PROXY implementation will hand over to the HTTP/1 FSM after processing the PROXY header. From my understanding the PROXY protocol is not specified for HTTP/2, so the connection here must stick with HTTP/1. Also, any incoming request on this interface not containing a valid PROXY header must be rejected. As for the VCL bits, I'm very happy with what was agreed upon at VDD13Q4 (local.ip, remote.ip, client.ip, server.ip). I don't see any mention of logging having been discussed, but I think it makes sense to have SessOpen use local.ip/remote.ip, while ReqStart should use server.ip/client.ip. varnishncsa will then use client.ip for logging the client host (%h). Opinions, input, comments? [1]: http://www.haproxy.org/download/1.5/doc/proxy-protocol.txt [2]: https://www.varnish-cache.org/trac/wiki/VDD13Q4#PROXY [3]: https://tools.ietf.org/html/draft-ietf-httpbis-http2-15#section-3.2 [4]: https://tools.ietf.org/html/draft-ietf-httpbis-http2-15#section-3.4 -- Dag Haavi Finstad Software Developer | Varnish Software Mobile: +47 476 64 134 We Make Websites Fly! From phk at phk.freebsd.dk Thu Nov 13 09:02:14 2014 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Thu, 13 Nov 2014 09:02:14 +0000 Subject: The stevedore API Message-ID: <83893.1415869334@critter.freebsd.dk> I've been mucking about with the stevedore api for some days in order to resolve a number of silly issues, including the vast overallocation for tiny objects when streaming. There are suprisingly many dead ends in this area. I think I have finally managed to come up with something that is usable, both from stevedore and varnishd side. The crucial insight is that we do not need to store the OA first since we now have accessor functions. Instead we can collect the OAs in the busyobj until we know their final size, and only then allocate space for them. But we still need to alert persistent stevedores before we make the first allocation for a new object, and we also (which is new!) want to give stevedores a chance to relocate segments once their size is known for sure. In that case it's the stevedores responsibility to keep track of streamers of the interrim segment before freeing it. Persistent stevedores also must be told what "magic key" it must present to resurrect an object. And finally we make it the cache_obj.c codes responsibility to free all allocated segments individually, so that the stevedore does not need to keep track of that. Seen from the stevedore: New objects are created with stv->newobj(). Arguments: busyobj total size estimate Returns: yes/no busyobj/objcore has private field(s) for the stevedores private use from here on. The stevedore can fail this if the object is undesirable, and another stevedore will be attempted, if nothing else, Transient. Storage segments are allocated with stv->alloc() Arguments: busyobj desired number of bytes Returns: interrim seg_id (uintptr_t) data pointer length Once the segment has been written to: stv->commit() Arguments: busyobj interrim seg_id length used Returns: final seg_id, may be different from interim seg_id Other threads may reference segments with stv->ref() Arguments: busyobj (possibly NULL) seg_id (interrim or final) Returns: data pointer length References are released with stv->rel() Arguments: busyobj (possibly NULL) seg_id (interrim or final) Returns: none Object is finalized with stv->final() Arguments: busyobj (not NULL) seg_id (key to resurrect persistent object) off_t (subkey to resurrect persistent object) Returns: none Objects are killed with stv->deleteobj() Arguments: objcore All allocated segments are individually freed with stv->free() Arguments: seg_id Seen from varnishd: stv->newobj() collect OA's in busyobj and access from there stv->alloc/commit until body is received all OA's defined. byte-stream encode OAs into last storage segment (including OA listing storage segment id's) (if space allows) else into separate segment stv->final(coords for OA bytestream) OA's now pulled out of stored object Comments most welcome... -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From martin at varnish-software.com Fri Nov 14 16:15:32 2014 From: martin at varnish-software.com (Martin Blix Grydeland) Date: Fri, 14 Nov 2014 17:15:32 +0100 Subject: The stevedore API In-Reply-To: <83893.1415869334@critter.freebsd.dk> References: <83893.1415869334@critter.freebsd.dk> Message-ID: Hi, This stevedore API looks great, and I believe it would work very well from the existing Varnish stevedore's point of view. But I have difficulties seeing this work well from a design that relies on double buffers. There might be details that I'm overlooking / not understanding fully at this time, so I'll try to outline my concerns. As I'm reading the API, Varnish will allow clients to stream also from the interim storage segments. In a double-buffered scenario, this means that the buffers can't be reused until all of the clients have finished with them. This is possible to do using refcounting in the ref()/rel() functions, but with a sufficiently popular object there will be slow clients pinning the buffers effectively doubling the memory pressure of a fetch. In my opinion the API needs to allow for the stevedore to be able to do double buffering using a single buffer that is being reused for each call to alloc(). This means that no streaming client should be allowed to touch any data until after the commit() call has been made, which should remove the need for the interrim seg_id's. During the IRC discussions on this, it was argued that it would be possible to have a buffer size returned from alloc() that is smaller than the size of the segment it buffers for. The real segment will then be appended to on each commit(), and the seg_id returned from commit() will then just be repeated for each append. I find this approach lacking. First, there will be no opportunity to coalesche the seg_id's. This can be handled in the stevedore during segment lookups (the ref() API call), by returning 0 lengths on all but the first ref() for the same seg_id by each client. This ofc then requires per client state in the stevedore. Also the seg_id list becomes unnecessarily big wasting space. Secondly, since the commit() function is supposed to be the point where trim() functionality happens, I fail to see how the code should be able to distinguish a commit() as an append from a commit() where a trim is needed. Based on this, I have an slightly modified suggestion for the API: New objects are created with stv->newobj(): * Arguments: * busyobj * total size estimate * Returns: * yes/no busyobj/objcore has private field(s) for the stevedores private use from here on. The stevedore can fail this if the object is undesirable, and another stevedore will be attempted, if nothing else, Transient. Storage segments are allocated with stv->alloc(): * Arguments: * busyobj * desired number of bytes * (uintptr_t) seg_id or 0. * Returns: * (void *) priv * data pointer * length The calling code can supply a previous seg_id that it would be OK to expand if the stevedore is able to (e.g. there is more room in the segment). This would typically be the previous seg_id received from commit(). Each call to alloc() needs to be followed by a commit(), passing the priv received. alloc() can fail, and indicates failure by returning a NULL data pointer. Once the segment has been written to: stv->commit() * Arguments: * busyobj * (void *) priv * length used * Returns: * (uintptr_t) seg_id This commits the storage and returns a seg_id(). This could be a new seg_id, or the same as the one passed to alloc() in which case it should not be added to the seg_id() list forming the object. commit() can fail, and indicates failure by returning a 0 seg_id. Trimming a segment: stv->trim() * Arguments: * busyobj * (uintptr_t) seg_id * Returns: * (uintptr_t) seg_id The calling code typically calls this at the end of a fetch operation. It can return a seg_id that is to replace the one passed, which the calling code should free() when it can ensure there are no readers (e.g. busyobj destruction). A trim that doesn't change the semantics (e.g. -sfile) will return 0 meaning no replace is necessary. Object is finalized with stv->final(): * Arguments: * busyobj (not NULL) * seg_id (key to resurrect persistent object) * off_t (subkey to resurrect persistent object) * Returns: none (failure point?) Objects are killd with stv->deleteobj(): * Arguments: * objcore All allocated segments are individually freed with stv->free(): * Arguments: * (uintptr_t) seg_id Regards, Martin Blix Grydeland On 13 November 2014 10:02, Poul-Henning Kamp wrote: > I've been mucking about with the stevedore api for some days in order > to resolve a number of silly issues, including the vast overallocation > for tiny objects when streaming. > > There are suprisingly many dead ends in this area. > > I think I have finally managed to come up with something that is usable, > both from stevedore and varnishd side. > > The crucial insight is that we do not need to store the OA first > since we now have accessor functions. Instead we can collect the > OAs in the busyobj until we know their final size, and only then > allocate space for them. > > But we still need to alert persistent stevedores before we make the > first allocation for a new object, and we also (which is new!) want > to give stevedores a chance to relocate segments once their size is > known for sure. In that case it's the stevedores responsibility > to keep track of streamers of the interrim segment before freeing it. > > Persistent stevedores also must be told what "magic key" it must > present to resurrect an object. > > And finally we make it the cache_obj.c codes responsibility to free > all allocated segments individually, so that the stevedore does not > need to keep track of that. > > Seen from the stevedore: > > New objects are created with stv->newobj(). > > Arguments: > busyobj > total size estimate > Returns: > yes/no > > busyobj/objcore has private field(s) for the stevedores > private use from here on. > > The stevedore can fail this if the object is undesirable, > and another stevedore will be attempted, if nothing else, > Transient. > > Storage segments are allocated with stv->alloc() > Arguments: > busyobj > desired number of bytes > Returns: > interrim seg_id (uintptr_t) > data pointer > length > > Once the segment has been written to: stv->commit() > Arguments: > busyobj > interrim seg_id > length used > Returns: > final seg_id, may be different from interim seg_id > > Other threads may reference segments with stv->ref() > Arguments: > busyobj (possibly NULL) > seg_id (interrim or final) > Returns: > data pointer > length > > References are released with stv->rel() > Arguments: > busyobj (possibly NULL) > seg_id (interrim or final) > Returns: > none > > Object is finalized with stv->final() > Arguments: > busyobj (not NULL) > seg_id (key to resurrect persistent object) > off_t (subkey to resurrect persistent object) > Returns: > none > > Objects are killed with stv->deleteobj() > Arguments: > objcore > > All allocated segments are individually freed with stv->free() > Arguments: > seg_id > > Seen from varnishd: > > stv->newobj() > collect OA's in busyobj and access from there > stv->alloc/commit until body is received > all OA's defined. > byte-stream encode OAs into last storage segment > (including OA listing storage segment id's) > (if space allows) else into separate segment > stv->final(coords for OA bytestream) > OA's now pulled out of stored object > > Comments most welcome... > > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > phk at FreeBSD.ORG | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetence. > > _______________________________________________ > varnish-dev mailing list > varnish-dev at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev > -- *Martin Blix Grydeland* Senior Developer | Varnish Software AS Mobile: +47 992 74 756 We Make Websites Fly! -------------- next part -------------- An HTML attachment was scrubbed... URL: From fgsch at lodoss.net Sun Nov 16 18:24:40 2014 From: fgsch at lodoss.net (Federico Schwindt) Date: Sun, 16 Nov 2014 18:24:40 +0000 Subject: PROXY protocol revisited In-Reply-To: References: Message-ID: Hi, I think it makes sense to make PROTO optional, falling back to current behavior if not present. Since proxy also supports IPv6 we should consider using brackets around the address for IPv6, e.g. proxy@[2001:67c:2804:1001::c21f:279b]:8081 Regarding logging, your suggestion makes sense to me. Salute. On Tue, Nov 11, 2014 at 4:41 PM, Dag Haavi Finstad < daghf at varnish-software.com> wrote: > Hi guys > > I've been looking into implementing the PROXY protocol[1] for Varnish > 4. This has been discussed several times before, in particular it > seems some consensus was found at VDD13Q4 in Berlin last year [2]. > This was regarding the interface in terms of specifying the listening > socket (-a) and the VCL *.ip bits. I was not present at this VDD > myself, so I'd just like to discuss and perhaps get some verification > as to what was agreed. > > As for the -a listening argument, this ties into how it should be > handled for HTTP/2. From reading the notes, the agreed upon syntax was > > -a PROTO at IP:0 > > Is the PROTO@ part mandatory, or is there a fallback when it's left > out (e.g. plain old '-a :80')? I think it makes most sense to have the > fallback value be HTTP/1.1 that also supports HTTP/2 via Upgrade [3]. > > If we are also going to have a way of specifying the protocol to be > exclusively HTTP/1 or HTTP/2 [4], we could use values 'http1' and > 'http2' to denote that. > > Further, a PROXY protocol listen socket is specified like this: > -a proxy at 192.168.1.10:8081 > > The PROXY implementation will hand over to the HTTP/1 FSM after > processing the PROXY header. From my understanding the PROXY protocol > is not specified for HTTP/2, so the connection here must stick with > HTTP/1. Also, any incoming request on this interface not containing a > valid PROXY header must be rejected. > > As for the VCL bits, I'm very happy with what was agreed upon at > VDD13Q4 (local.ip, remote.ip, client.ip, server.ip). > > I don't see any mention of logging having been discussed, but I think > it makes sense to have SessOpen use local.ip/remote.ip, while ReqStart > should use server.ip/client.ip. varnishncsa will then use client.ip > for logging the client host (%h). > > Opinions, input, comments? > > [1]: http://www.haproxy.org/download/1.5/doc/proxy-protocol.txt > [2]: https://www.varnish-cache.org/trac/wiki/VDD13Q4#PROXY > [3]: https://tools.ietf.org/html/draft-ietf-httpbis-http2-15#section-3.2 > [4]: https://tools.ietf.org/html/draft-ietf-httpbis-http2-15#section-3.4 > > -- > Dag Haavi Finstad > Software Developer | Varnish Software > Mobile: +47 476 64 134 > We Make Websites Fly! > > _______________________________________________ > varnish-dev mailing list > varnish-dev at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From anand at rediff-inc.com Mon Nov 17 13:05:22 2014 From: anand at rediff-inc.com (Anand Shah) Date: 17 Nov 2014 13:05:22 -0000 Subject: =?utf-8?B?U3dhcCBmbHVzaGVzIGFsbCBvYmplY3RzIGluIGNhY2hl?= Message-ID: <20141117130522.28448.qmail@pro237-61.mxout.rediffmailpro.com> Hello,When Linux pages outs data to disk; it destroys all cached objects which is contrary to the architecture published by PHK.When physical memory becomes scarce the Linux memory management subsystem must attempt to free physical pages. This task falls to the kernel swap daemon (kswapd). Linux does not swap out all of the swappable pages of the process that it has selected; instead it removes only a small number of pages.Pages cannot be swapped or discarded if they:are locked in memory,are shared; there is a separate, explicit, mechanism for swapping out these pages.Here we have been observing anything that swaps discards all the objects present in the cache and all items are disqualified from a successful HIT object. We have been extensively using Ganglia to observe the system behaviour. I am also attaching two images that describes the condition mentioned in the email. Here i am not certain if varnish should behave this way or not? Varnish version : varnishd (varnish-4.0.1 revision 4354e5e)Regards,Anand -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image1.jpg Type: image/jpeg Size: 391122 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image2.jpg Type: image/jpeg Size: 319331 bytes Desc: not available URL: From thierry.magnien at sfr.com Mon Nov 17 13:12:43 2014 From: thierry.magnien at sfr.com (MAGNIEN, Thierry) Date: Mon, 17 Nov 2014 13:12:43 +0000 Subject: Swap flushes all objects in cache In-Reply-To: <20141117130522.28448.qmail@pro237-61.mxout.rediffmailpro.com> References: <20141117130522.28448.qmail@pro237-61.mxout.rediffmailpro.com> Message-ID: <5D103CE839D50E4CBC62C9FD7B83287CC9C931EB@EXCN015.encara.local.ads> Hi, Are you sure you're not hitting a case where your kernel OOM kills varnish child ? In that case, obviously you lose all your cached content. Regards, Thierry --------------------- De?: varnish-dev-bounces+thierry.magnien=sfr.com at varnish-cache.org [mailto:varnish-dev-bounces+thierry.magnien=sfr.com at varnish-cache.org] De la part de Anand Shah Envoy??: lundi 17 novembre 2014 14:05 ??: varnish-dev at varnish-cache.org; phk Objet?: Swap flushes all objects in cache Hello, When Linux pages outs data to disk; it destroys all cached objects which is contrary to the architecture published by PHK. When physical memory becomes scarce the Linux memory management subsystem must attempt to free physical pages. This task falls to the kernel swap daemon (kswapd). Linux does not swap out all of the swappable pages of the process that it has selected; instead it removes only a small number of pages. Pages cannot be swapped or discarded if they: are locked in memory, are shared; there is a separate, explicit, mechanism for swapping out these pages. Here we have been observing anything that swaps discards all the objects present in the cache and all items are disqualified from a successful HIT object. We have been extensively using Ganglia to observe the system behaviour. I am also attaching two images that describes the condition mentioned in the email. Here i am not certain if varnish should behave this way or not?? Varnish version :?varnishd (varnish-4.0.1 revision 4354e5e) Regards, Anand From lkarsten at varnish-software.com Tue Nov 18 10:37:45 2014 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 18 Nov 2014 11:37:45 +0100 Subject: Swap flushes all objects in cache In-Reply-To: <20141117130522.28448.qmail@pro237-61.mxout.rediffmailpro.com> References: <20141117130522.28448.qmail@pro237-61.mxout.rediffmailpro.com> Message-ID: <20141118103744.GA8776@immer.varnish-software.com> On Mon, Nov 17, 2014 at 01:05:22PM -0000, Anand Shah wrote: > Hello,When Linux pages outs data to disk; it destroys all cached > objects which is contrary to the architecture published by PHK.When > physical memory becomes scarce the Linux memory management subsystem > must attempt to free physical pages. [cut] Hi. Please consider adding line feeds to your future emails, everything on one line is very hard to read. Based on the attached graphs I'd expect this to be caused by Varnish asserting/crashing, or that the operating system's out-of-memory handling killed off the process. More information on Varnish troubleshooting: https://www.varnish-cache.org/docs/4.0/users-guide/troubleshooting.html -- Lasse Karstensen Varnish Software AS From anand at rediff-inc.com Tue Nov 18 13:01:18 2014 From: anand at rediff-inc.com (Anand Shah) Date: 18 Nov 2014 13:01:18 -0000 Subject: =?utf-8?B?IHZhcm5pc2gtZGV2QHZhcm5pc2gtY2FjaGUub3JnOiBTd2FwIGZsdXNoZXMgYWxsIG9iamVjdHMgaW4gY2FjaGU=?= Message-ID: <1416310133.S.97416.autosave.drafts.old.1416315677.6731@webmail.rediffmail.com> I have noticed this everytime the memory offload happens. This is not a crash or segfault; coz the service does not go down and it keeps in running which is also evident in the graphs shared earlier. I can see the same behaviour everytime when operating system swaps. Can you confirm if this is a ideal behaviour with varnish?  Regards, Anand From: Lasse Karstensen <lkarsten at varnish-software.com> Sent: Tue, 18 Nov 2014 16:07:51 To: Anand Shah <anand at rediff-inc.com> Cc: varnish-dev at varnish-cache.org Subject: Re: Swap flushes all objects in cache On Mon, Nov 17, 2014 at 01:05:22PM -0000, Anand Shah wrote: > Hello,When Linux pages outs data to disk; it destroys all cached > objects which is contrary to the architecture published by PHK.When > physical memory becomes scarce the Linux memory management subsystem > must attempt to free physical pages. [cut] Hi. Please consider adding line feeds to your future emails, everything on one line is very hard to read. Based on the attached graphs I'd expect this to be caused by Varnish asserting/crashing, or that the operating system's out-of-memory handling killed off the process. More information on Varnish troubleshooting: https://www.varnish-cache.org/docs/4.0/users-guide/troubleshooting.html -- Lasse Karstensen Varnish Software AS -------------- next part -------------- An HTML attachment was scrubbed... URL: From perbu at varnish-software.com Tue Nov 18 17:40:26 2014 From: perbu at varnish-software.com (Per Buer) Date: Tue, 18 Nov 2014 18:40:26 +0100 Subject: varnish-dev@varnish-cache.org: Swap flushes all objects in cache In-Reply-To: <1416310133.S.97416.autosave.drafts.old.1416315677.6731@webmail.rediffmail.com> References: <1416310133.S.97416.autosave.drafts.old.1416315677.6731@webmail.rediffmail.com> Message-ID: Anand, On Tue, Nov 18, 2014 at 2:01 PM, Anand Shah wrote: > I have noticed this everytime the memory offload happens. This is not a > crash or segfault; coz the service does not go down and it keeps in running > which is also evident in the graphs shared earlier. I can see the same > behaviour everytime when operating system swaps. Can you confirm if this is > a ideal behaviour with varnish? Please keep user questions to the -misc list. This is most like misconfiguration causing the OOM killer to kill of the child process. It doesn't keep running. Look at the accepted sessions in the graph. It dives when this happens. -- *Per Buer* CTO | Varnish Software AS Cell: +47 95839117 We Make Websites Fly! www.varnish-software.com [image: Register now] -------------- next part -------------- An HTML attachment was scrubbed... URL: From Gauthier.Delacroix at coreye.fr Wed Nov 19 09:10:09 2014 From: Gauthier.Delacroix at coreye.fr (Delacroix, Gauthier) Date: Wed, 19 Nov 2014 09:10:09 +0000 Subject: [PATCH] Allow varnish_reload_vcl to discard old VCL after reload Message-ID: <5F530A9242E7F84F999DB40E0E268FBD45EB9CBA@mercalli.lild01.pictime.fr> This patch allows varnish_reload_vcl to discard old (available) VCL configurations after reload. DISCARD_OLD_VCL environment variable must be set to 1. It is set to 0 by default to avoid unexpected behavior change. --- redhat/varnish.params | 5 +++++ redhat/varnish.sysconfig | 5 +++++ redhat/varnish_reload_vcl | 15 ++++++++++++++- 3 files changed, 24 insertions(+), 1 deletion(-) diff --git a/redhat/varnish.params b/redhat/varnish.params index 636e975..09516c1 100644 --- a/redhat/varnish.params +++ b/redhat/varnish.params @@ -4,6 +4,11 @@ # Set this to 1 to make systemd reload try to switch vcl without restart. RELOAD_VCL=1 +# Set this to 1 to make init script reload try to discard old +# VCL configurations after reload. +# To make this work, you need to set RELOAD_VCL variable to 1. +DISCARD_ON_RELOAD=0 + # Main configuration file. You probably want to change it. VARNISH_VCL_CONF=/etc/varnish/default.vcl diff --git a/redhat/varnish.sysconfig b/redhat/varnish.sysconfig index 6aa2354..c532695 100644 --- a/redhat/varnish.sysconfig +++ b/redhat/varnish.sysconfig @@ -24,6 +24,11 @@ NPROCS="unlimited" # use Alternative 3, Advanced configuration, below RELOAD_VCL=1 +# Set this to 1 to make init script reload try to discard old +# VCL configurations after reload. +# To make this work, you need to set RELOAD_VCL variable to 1. +DISCARD_ON_RELOAD=0 + # This file contains 4 alternatives, please use only one. ## Alternative 1, Minimal configuration, no VCL diff --git a/redhat/varnish_reload_vcl b/redhat/varnish_reload_vcl index be3c21a..c686541 100755 --- a/redhat/varnish_reload_vcl +++ b/redhat/varnish_reload_vcl @@ -10,7 +10,7 @@ # The following environment variables have to be set: # RELOAD_VCL, VARNISH_VCL_CONF, VARNISH_ADMIN_LISTEN_PORT # The following are optional: -# VARNISH_SECRET_FILE, VARNISH_ADMIN_LISTEN_ADDRESS +# VARNISH_SECRET_FILE, VARNISH_ADMIN_LISTEN_ADDRESS, DISCARD_ON_RELOAD # # Requires GNU bash and GNU date # @@ -26,6 +26,7 @@ print_debug() { echo " Parsed configuration: RELOAD_VCL=\"$RELOAD_VCL\" +DISCARD_ON_RELOAD=\"$DISCARD_ON_RELOAD\" VARNISH_VCL_CONF=\"$VARNISH_VCL_CONF\" VARNISH_ADMIN_LISTEN_ADDRESS=\"$VARNISH_ADMIN_LISTEN_ADDRESS\" VARNISH_ADMIN_LISTEN_PORT=\"$VARNISH_ADMIN_LISTEN_PORT\" @@ -108,6 +109,18 @@ else echo "$VARNISHADM vcl.use failed" exit 1 fi + +if [ "$DISCARD_ON_RELOAD" = "1" ]; then + $VARNISHADM vcl.list | grep "^available" | awk '{ print $3 }' | while read old_config; do + if $VARNISHADM vcl.discard $old_config > /dev/null; then + echo "VCL '$old_config' discarded" + $debug && echo "$VARNISHADM vcl.discard succeded" + else + echo "$VARNISHADM vcl.discard failed" + exit 1 + fi + done +fi $VARNISHADM vcl.list echo Done exit 0 -- 1.8.3.1 From anand at rediff-inc.com Wed Nov 19 11:48:39 2014 From: anand at rediff-inc.com (Anand Shah) Date: 19 Nov 2014 11:48:39 -0000 Subject: =?utf-8?B?UmU6IHZhcm5pc2gtZGV2QHZhcm5pc2gtY2FjaGUub3JnOiBTd2FwIGZsdXNoZXMgYWxsIG9iamVjdHMgaW4gY2FjaGU=?= Message-ID: <1416377367.S.44274.autosave.drafts.1416397719.20567@webmail.rediffmail.com> Thank you Per for replying back. Under desperately low memory conditions, the out-of-memory (OOM) killer kicks in and picks up child process of varnish that holds up the virtual memory space (in my case 524785412 kB .i.e 500 GB) and kills it dropping all objects present in the vm space. I am not able to associate the child crash that happens and swapping that place is doing everything wrong here. Can you please re-consider the concerns and have someone look on the errors that are written while breaking.Nov 19 15:06:36 cdn abrt[11775]: Saved core dump of pid 10079 (/usr/sbin/varnishd) to /var/spool/abrt/ccpp-2014-11-19-15:06:33-10079 (381931520 bytes)Nov 19 15:06:36 cdn abrtd: Directory 'ccpp-2014-11-19-15:06:33-10079' creation detectedNov 19 15:06:36 cdn abrtd: Package 'varnish' isn't signed with proper keyNov 19 15:06:36 cdn abrtd: 'post-create' on '/var/spool/abrt/ccpp-2014-11-19-15:06:33-10079' exited with 1Nov 19 15:06:36 cdn abrtd: Deleting problem directory '/var/spool/abrt/ccpp-2014-11-19-15:06:33-10079'Nov 19 15:06:36 cdn varnishd[11921]: Child (10079) not responding to CLI, killing it.Nov 19 15:06:37 cdn varnishd[11921]: Child (10079) died signal=6 (core dumped)Nov 19 15:06:37 cdn varnishd[11921]: Child (10079) Panic message:#012Assert error in cnt_lookup(), cache/cache_req_fsm.c line 411:#012  Condition((oc->exp_flags & (1<<7)) == 0) not true.#012thread = (cache-worker)#012ident =Linux,2.6.32-431.el6.x86_64,x86_64,-sfile,-smalloc,-hclassic,epoll#012Backtrace:#012  0x43b03d: /usr/sbin/varnishd() [0x43b03d]#012  0x43b34d: /usr/sbin/varnishd() [0x43b34d]#012  0x440723: /usr/sbin/varnishd() [0x440723]#012  0x44288d: /usr/sbin/varnishd(CNT_Request+0x441) [0x44288d]#012  0x4339c0: /usr/sbin/varnishd(HTTP1_Session+0x429) [0x4339c0]#012  0x4458cb: /usr/sbin/varnishd() [0x4458cb]#012  0x43dfb5: /usr/sbin/varnishd(Pool_Work_Thread+0x4cb) [0x43dfb5]#012  0x456198: /usr/sbin/varnishd() [0x456198]#012  0x4562c1: /usr/sbin/varnishd(WRK_thread+0x27) [0x4562c1]#012  0x7f4027fc49d1: /lib64/libpthread.so.0(+0x79d1) [0x7f4027fc49d1]#012req = 0x7ec30d0e8020 {#012  sp = 0x7ec30e4257e0, vxid = 1074847970,  step = R_STP_LOOKUP,#012  req_body = R_BODY_NONE,#012  restarts = 0, esi_level = 0#012  sp = 0x7ec30e4257e0 {#012    fd = 75, vxid = 1106140,#012    client = 122.176.68.88 56100,#012    step = S_STP_WORKING,#012  },#012  worker = 0x7f4017709bf0 {#012    ws = 0x7f4017709e08 {#012      id = "wrk",#012      {s,f,r,e} = {0x7f40177093d0,0x7f40177093d0,(nil),+2048},#012    },#012  VCL::method = 0x0,#012  VCL::return = deliver,#012  },#012  ws = 0x7ec30d0e81b8 {#012    id = "req",#012    {s,f,r,e} = {0x7ec30d0ea010,+872,(nil),+57360},#012  },#012  http[req] = {#012    ws = 0x7ec30d0e81b8[req]#012      "GET",#012      "/thumbnail1/3a8bbd2bb61f4f3f.jpg",#012      "HTTP/1.1",#012      "Connection: keep-alive",#012      "Cache-Control: max-age=0",#012      "Accept: image/webp,*/*;q=0.8",#012      "If-Modified-Since: Mon, 08 Sep 2014 03:15:48 GMT",#012      "User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/38.0.2125.111 Safari/537.36",#012      "Referer: http://money.rediff.com/",#012      "Accept-Language: en-US,en;q=0.8",#012      "X-Forwarded-For: 122.176Regards,AnandFrom: Per Buer <perbu at varnish-software.com>Sent: Tue, 18 Nov 2014 23:10:50To: Anand_Shah <anand at rediff-inc.com>Subject: Re: varnish-dev at varnish-cache.org: Swap flushes all objects in cache Anand, On Tue, Nov 18, 2014 at 2:01 PM, Anand Shah <anand at rediff-inc.com> wrote: I have noticed this everytime the memory offload happens. This is not a crash or segfault; coz the service does not go down and it keeps in running which is also evident in the graphs shared earlier. I can see the same behaviour everytime when operating system swaps. Can you confirm if this is a ideal behaviour with varnish?    Please keep user questions to the -misc list.   This is most like misconfiguration causing the OOM killer to kill of the child process. It doesn't keep running. Look at the accepted sessions in the graph. It dives when this happens.    --  Per Buer CTO | Varnish Software AS Cell: +47 95839117 We Make Websites Fly! www.varnish-software.com   -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: nstruct1.jpg Type: image/jpeg Size: 344460 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: pageout1.jpg Type: image/jpeg Size: 338673 bytes Desc: not available URL: From daghf at varnish-software.com Thu Nov 20 08:07:17 2014 From: daghf at varnish-software.com (Dag Haavi Finstad) Date: Thu, 20 Nov 2014 09:07:17 +0100 Subject: PRIV_REQ/SESS in vcl_backend_* Message-ID: Hi Here's a suggestion (and a set of patches) for what PRIV_REQ/SESS could look like in vcl_backend_*, mostly based around IRC discussion. The idea is to make PRIV_REQ available also in vcl_backend_*, and then give a separate priv object from the one used on the client side. Realizing this could lead to confusion, it was suggested to rename it PRIV_XID. The priv object will however survive a VCL restart/retry, so the name is not completely honest in that it is not restricted to a single XID. I think this is the behavior that makes the most sense, but I'm not super excited about the name. A second point is regarding PRIV_SESS. A concern with PRIV_SESS is that the remote Varnish is talking to very often tends to be not the client itself, but rather a load balancer dispatching requests from a bunch of clients over the same persistent connections. So if you were to introduce a priv_sess vmod that handles some sort of client state in such a scenario, you will very quickly shoot yourself in the foot and leak state across clients unless you know what you are doing. I think the problems solved by PRIV_SESS are typically solved much safer via PRIV_REQ. A point brought up by Tollef on IRC is that PRIV_SESS lets you share state between a request and its ESI subrequests, something that has legitimate use cases. Patch #5 (attached) drops PRIV_SESS and introduces PRIV_ESI, which restricts the scope to a request and its client-side subrequests. Feedback/comments/complaints very welcome. :-) regards, Dag -- Dag Haavi Finstad Software Developer | Varnish Software Mobile: +47 476 64 134 We Make Websites Fly! -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Add-a-test-case-for-using-PRIV_-in-an-ESI-context-an.patch Type: text/x-patch Size: 2191 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0002-PRIV_REQ-PRIV_XID.-Renamed-to-make-the-situation-les.patch Type: text/x-patch Size: 5662 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0003-Make-the-VRTPRIV_-interface-slightly-more-generic-to.patch Type: text/x-patch Size: 6688 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0004-Introduce-PRIV_-XID-SESS-in-vcl_backend_.patch Type: text/x-patch Size: 5058 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0005-Drop-PRIV_SESS-and-welcome-PRIV_ESI.-Unlike-PRIV_SES.patch Type: text/x-patch Size: 8249 bytes Desc: not available URL: From lkarsten at varnish-software.com Fri Nov 21 15:48:37 2014 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Fri, 21 Nov 2014 16:48:37 +0100 Subject: Bugwash moved to Mondays at 13:00 CET Message-ID: <20141121154837.GA25724@immer.varnish-software.com> Hi all. As discussed on VDD14Q4, we're moving our weekly bugwash to Mondays at 13:00 CET. (from 12:00 CET) -- Lasse Karstensen Varnish Software AS From geoff at uplex.de Sat Nov 22 14:52:15 2014 From: geoff at uplex.de (Geoff Simmons) Date: Sat, 22 Nov 2014 15:52:15 +0100 Subject: [PATCH] Backport a fix for CLI protocol failure to 3.0.6 Message-ID: <5470A31F.8030901@uplex.de> Hello all, The patch in the attachment backports commit 2144dc78, a fix for mgt_cli.c, to version 3.0.6. The fix catches one of conditions for calling MGT_Child_Cli_Fail() that had previously been missed. The patch would have to be applied to the 3.0 branch. When the CLI between management and child processes gets out of sync, MGT_Child_Cli_Fail() causes the CLI connection to be closed and re-opened. We were seeing the effect (on both 3.0.3 and 3.0.6) that 'varnishadm vcl.list' occasionally returned the output of backend.list -- backend.list was being called by monitoring tools at the same time. The problem has gone away since we applied the patch. So if there's ever a 3.0.7, then I suggest that this fix be included. Best, Geoff -- ** * * UPLEX - Nils Goroll Systemoptimierung Scheffelstra?e 32 22301 Hamburg Tel +49 40 2880 5731 Mob +49 176 636 90917 Fax +49 40 42949753 http://uplex.de -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Use-better-criteria-for-determining-if-child-CLI-con.patch Type: text/x-patch Size: 1281 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From phk at phk.freebsd.dk Sun Nov 23 10:04:51 2014 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Sun, 23 Nov 2014 10:04:51 +0000 Subject: Bugwash moved to Mondays at 13:00 CET In-Reply-To: <20141121154837.GA25724@immer.varnish-software.com> References: <20141121154837.GA25724@immer.varnish-software.com> Message-ID: <64676.1416737091@critter.freebsd.dk> -------- In message <20141121154837.GA25724 at immer.varnish-software.com>, Lasse Karstense n writes: >As discussed on VDD14Q4, we're moving our weekly bugwash >to Mondays at 13:00 CET. (from 12:00 CET) And just so you don't think I didn't read this: Tomorrow I'll be attending a factory test for a customer and won't participate in the bugwash. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From daghf at varnish-software.com Tue Nov 25 13:49:49 2014 From: daghf at varnish-software.com (Dag Haavi Finstad) Date: Tue, 25 Nov 2014 14:49:49 +0100 Subject: PRIV_REQ/SESS in vcl_backend_* In-Reply-To: References: Message-ID: Hi again Here's a new set of patches, following the discussion at VDD14Q4 (https://www.varnish-cache.org/trac/wiki/VDD14Q4#Notestakenduringthemeeting) To summarize the changes, - PRIV_SESS is dropped. - PRIV_REQ is reintroduced as PRIV_TASK, which is also available as a separate priv pointer in vcl_backend_*. regards, Dag On Thu, Nov 20, 2014 at 9:07 AM, Dag Haavi Finstad wrote: > Hi > > Here's a suggestion (and a set of patches) for what PRIV_REQ/SESS > could look like in vcl_backend_*, mostly based around IRC discussion. > > The idea is to make PRIV_REQ available also in vcl_backend_*, and then > give a separate priv object from the one used on the client side. > Realizing this could lead to confusion, it was suggested to rename it > PRIV_XID. The priv object will however survive a VCL restart/retry, so > the name is not completely honest in that it is not restricted to a > single XID. I think this is the behavior that makes the most sense, > but I'm not super excited about the name. > > A second point is regarding PRIV_SESS. A concern with PRIV_SESS is > that the remote Varnish is talking to very often tends to be not the > client itself, but rather a load balancer dispatching requests from a > bunch of clients over the same persistent connections. So if you were > to introduce a priv_sess vmod that handles some sort of client state > in such a scenario, you will very quickly shoot yourself in the foot > and leak state across clients unless you know what you are doing. I > think the problems solved by PRIV_SESS are typically solved much safer > via PRIV_REQ. > > A point brought up by Tollef on IRC is that PRIV_SESS lets you share > state between a request and its ESI subrequests, something that has > legitimate use cases. Patch #5 (attached) drops PRIV_SESS and > introduces PRIV_ESI, which restricts the scope to a request and its > client-side subrequests. > > Feedback/comments/complaints very welcome. :-) > > regards, > Dag > > -- > Dag Haavi Finstad > Software Developer | Varnish Software > Mobile: +47 476 64 134 > We Make Websites Fly! -- Dag Haavi Finstad Software Developer | Varnish Software Mobile: +47 476 64 134 We Make Websites Fly! -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Drop-PRIV_SESS-as-agreed-on-VDD14Q4.patch Type: text/x-patch Size: 5425 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0002-Make-the-VRTPRIV_-interface-slightly-more-generic-to.patch Type: text/x-patch Size: 6037 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0003-Reintroduce-PRIV_REQ-as-PRIV_TASK.-PRIV_TASK-is-avai.patch Type: text/x-patch Size: 11984 bytes Desc: not available URL: From fgsch at lodoss.net Tue Nov 25 13:53:12 2014 From: fgsch at lodoss.net (Federico Schwindt) Date: Tue, 25 Nov 2014 13:53:12 +0000 Subject: varnish-dev@varnish-cache.org: Swap flushes all objects in cache In-Reply-To: <1416377367.S.44274.autosave.drafts.1416397719.20567@webmail.rediffmail.com> References: <1416377367.S.44274.autosave.drafts.1416397719.20567@webmail.rediffmail.com> Message-ID: Hi, This looks like https://www.varnish-cache.org/trac/ticket/1539. What varnish version are you running? On Wed, Nov 19, 2014 at 11:48 AM, Anand Shah wrote: > > > Thank you Per for replying back. > > > Under desperately low memory conditions, the out-of-memory (OOM) killer > kicks in and picks up child process of varnish that holds up the virtual > memory space (in my case 524785412 kB .i.e 500 GB) and kills it dropping > all objects present in the vm space. I am not able to associate the child > crash that happens and swapping that place is doing everything wrong here. > Can you please re-consider the concerns and have someone look on the errors > that are written while breaking. > > > > Nov 19 15:06:36 cdn abrt[11775]: Saved core dump of pid 10079 > (/usr/sbin/varnishd) to /var/spool/abrt/ccpp-2014-11-19-15:06:33-10079 > (381931520 bytes) > Nov 19 15:06:36 cdn abrtd: Directory 'ccpp-2014-11-19-15:06:33-10079' > creation detected > Nov 19 15:06:36 cdn abrtd: Package 'varnish' isn't signed with proper key > Nov 19 15:06:36 cdn abrtd: 'post-create' on > '/var/spool/abrt/ccpp-2014-11-19-15:06:33-10079' exited with 1 > Nov 19 15:06:36 cdn abrtd: Deleting problem directory > '/var/spool/abrt/ccpp-2014-11-19-15:06:33-10079' > Nov 19 15:06:36 cdn varnishd[11921]: Child (10079) not responding to CLI, > killing it. > Nov 19 15:06:37 cdn varnishd[11921]: Child (10079) died signal=6 (core > dumped) > > Nov 19 15:06:37 cdn varnishd[11921]: Child (10079) Panic > message:#012Assert error in cnt_lookup(), cache/cache_req_fsm.c line > 411:#012 Condition((oc->exp_flags & (1<<7)) == 0) not true.#012thread = > (cache-worker)#012ident = > Linux,2.6.32-431.el6.x86_64,x86_64,-sfile,-smalloc,-hclassic,epoll#012Backtrace:#012 > 0x43b03d: /usr/sbin/varnishd() [0x43b03d]#012 0x43b34d: > /usr/sbin/varnishd() [0x43b34d]#012 0x440723: /usr/sbin/varnishd() > [0x440723]#012 0x44288d: /usr/sbin/varnishd(CNT_Request+0x441) > [0x44288d]#012 0x4339c0: /usr/sbin/varnishd(HTTP1_Session+0x429) > [0x4339c0]#012 0x4458cb: /usr/sbin/varnishd() [0x4458cb]#012 0x43dfb5: > /usr/sbin/varnishd(Pool_Work_Thread+0x4cb) [0x43dfb5]#012 0x456198: > /usr/sbin/varnishd() [0x456198]#012 0x4562c1: > /usr/sbin/varnishd(WRK_thread+0x27) [0x4562c1]#012 0x7f4027fc49d1: > /lib64/libpthread.so.0(+0x79d1) [0x7f4027fc49d1]#012req = 0x7ec30d0e8020 > {#012 sp = 0x7ec30e4257e0, vxid = 1074847970, step = R_STP_LOOKUP,#012 > req_body = R_BODY_NONE,#012 restarts = 0, esi_level = 0#012 sp = > 0x7ec30e4257e0 {#012 fd = 75, vxid = 1106140,#012 client = > 122.176.68.88 56100,#012 step = S_STP_WORKING,#012 },#012 worker = > 0x7f4017709bf0 {#012 ws = 0x7f4017709e08 {#012 id = "wrk",#012 > {s,f,r,e} = {0x7f40177093d0,0x7f40177093d0,(nil),+2048},#012 },#012 > VCL::method = 0x0,#012 VCL::return = deliver,#012 },#012 ws = > 0x7ec30d0e81b8 {#012 id = "req",#012 {s,f,r,e} = > {0x7ec30d0ea010,+872,(nil),+57360},#012 },#012 http[req] = {#012 ws = > 0x7ec30d0e81b8[req]#012 "GET",#012 > "/thumbnail1/3a8bbd2bb61f4f3f.jpg",#012 "HTTP/1.1",#012 > "Connection: keep-alive",#012 "Cache-Control: max-age=0",#012 > "Accept: image/webp,*/*;q=0.8",#012 "If-Modified-Since: Mon, 08 Sep > 2014 03:15:48 GMT",#012 "User-Agent: Mozilla/5.0 (Windows NT 5.1) > AppleWebKit/537.36 (KHTML, like Gecko) Chrome/38.0.2125.111 > Safari/537.36",#012 "Referer: http://money.rediff.com/",#012 > "Accept-Language: en-US,en;q=0.8",#012 "X-Forwarded-For: 122.176 > > > > Regards, > Anand > > > > From: Per Buer > Sent: Tue, 18 Nov 2014 23:10:50 > To: Anand_Shah > Subject: Re: varnish-dev at varnish-cache.org: Swap flushes all objects in > cache > > > Anand, > > On Tue, Nov 18, 2014 at 2:01 PM, Anand Shah wrote: >> >> I have noticed this everytime the memory offload happens. This is not a >> crash or segfault; coz the service does not go down and it keeps in running >> which is also evident in the graphs shared earlier. I can see the same >> behaviour everytime when operating system swaps. Can you confirm if this is >> a ideal behaviour with varnish? > > > Please keep user questions to the -misc list. > > This is most like misconfiguration causing the OOM killer to kill of the > child process. It doesn't keep running. Look at the accepted sessions in > the graph. It dives when this happens. > > > -- > *Per Buer* > CTO | Varnish Software AS > Cell: +47 95839117 > We Make Websites Fly! > www.varnish-software.com > [image: Register now] > > > _______________________________________________ > varnish-dev mailing list > varnish-dev at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From phk at phk.freebsd.dk Thu Nov 27 15:52:36 2014 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Thu, 27 Nov 2014 15:52:36 +0000 Subject: PRIV_REQ/SESS in vcl_backend_* In-Reply-To: References: Message-ID: <14551.1417103556@critter.freebsd.dk> -------- In message , Dag Haavi Finstad writes: >Here's a new set of patches, following the discussion at VDD14Q4 >(https://www.varnish-cache.org/trac/wiki/VDD14Q4#Notestakenduringthemeeting) I've quickly eye-balled them and they look sane: commit please -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From daghf at varnish-software.com Thu Nov 27 16:11:30 2014 From: daghf at varnish-software.com (Dag Haavi Finstad) Date: Thu, 27 Nov 2014 17:11:30 +0100 Subject: PRIV_REQ/SESS in vcl_backend_* In-Reply-To: <14551.1417103556@critter.freebsd.dk> References: <14551.1417103556@critter.freebsd.dk> Message-ID: On Thu, Nov 27, 2014 at 4:52 PM, Poul-Henning Kamp wrote: > > In message > , Dag Haavi Finstad writes: > >>Here's a new set of patches, following the discussion at VDD14Q4 >>(https://www.varnish-cache.org/trac/wiki/VDD14Q4#Notestakenduringthemeeting) > > I've quickly eye-balled them and they look sane: commit please Thanks! Commited. -- Dag Haavi Finstad Software Developer | Varnish Software Mobile: +47 476 64 134 We Make Websites Fly! From fgsch at lodoss.net Thu Nov 27 18:44:36 2014 From: fgsch at lodoss.net (Federico Schwindt) Date: Thu, 27 Nov 2014 18:44:36 +0000 Subject: [PATCH] Make vrt_selecthttp public Message-ID: I found myself needing this for a vmod. OK? -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Make-vrt_selecthttp-public.patch Type: text/x-patch Size: 3053 bytes Desc: not available URL: From phk at phk.freebsd.dk Thu Nov 27 20:37:50 2014 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Thu, 27 Nov 2014 20:37:50 +0000 Subject: [PATCH] Make vrt_selecthttp public In-Reply-To: References: Message-ID: <54764.1417120670@critter.freebsd.dk> -------- In message , Federico Schwindt writes: >I found myself needing this for a vmod. > >OK? OK. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence.