From lasse.karstensen at gmail.com Fri Jan 4 14:37:19 2013 From: lasse.karstensen at gmail.com (Lasse Karstensen) Date: Fri, 4 Jan 2013 15:37:19 +0100 Subject: Soft purge support (patch) Message-ID: <20130104143716.GA8713@sierra.hyse.org> Attached is a small patch set that implement soft purge support for Varnish trunk. Soft purges are purges where the object TTL is reduced to zero, but grace is kept. In essence it adds a softpurge; statement in VCL that can be used where purge; ordinarily is. The main use case for soft purges are big Varnish setups with many backends and automated purging systems. When doing maintenance on one of the backends, you end up with automatically purging something that can't be recreated right now and Varnish starts sending 503 replies. With softpurge the grace handling will kick in and serve a stale object instead. Git branch is available here: https://github.com/lkarsten/Varnish-Cache/tree/softpurge I request that these patches are merged into trunk. -- Lasse Karstensen Varnish Software AS http://www.varnish-software.com/ -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Change-VRT_purge-format-to-support-soft-purge.patch Type: text/x-diff Size: 2428 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0002-Change-HSH_Purge-for-support-soft-purge.patch Type: text/x-diff Size: 3214 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0003-Add-test-case-for-soft-purge.patch Type: text/x-diff Size: 3137 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0004-Add-section-describing-soft-purge.patch Type: text/x-diff Size: 1426 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0005-Reference-documentation-for-purge-and-softpurge.patch Type: text/x-diff Size: 839 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0006-Only-check-the-time-if-it-is-a-soft-purge.patch Type: text/x-diff Size: 1107 bytes Desc: not available URL: From lasse.karstensen at gmail.com Fri Jan 4 16:16:59 2013 From: lasse.karstensen at gmail.com (Lasse Karstensen) Date: Fri, 4 Jan 2013 17:16:59 +0100 Subject: PATCH: (|be)req.method in VCL Message-ID: <20130104161658.GB8713@sierra.hyse.org> Hello. Attached is a small patch that adds the req.method and bereq.method attributes in VCL. These are aliased to the req.request and bereq.request attributes, to be more in harmony with the jargon used in HTTP in general. Please consider this for inclusion in trunk. -- Lasse Karstensen -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Add-req.method-and-bereq.method.patch Type: text/x-diff Size: 3974 bytes Desc: not available URL: From magnus at hagander.net Sat Jan 5 14:29:26 2013 From: magnus at hagander.net (Magnus Hagander) Date: Sat, 5 Jan 2013 15:29:26 +0100 Subject: Soft purge support (patch) In-Reply-To: <20130104143716.GA8713@sierra.hyse.org> References: <20130104143716.GA8713@sierra.hyse.org> Message-ID: On Fri, Jan 4, 2013 at 3:37 PM, Lasse Karstensen wrote: > Attached is a small patch set that implement soft purge support > for Varnish trunk. > > Soft purges are purges where the object TTL is reduced to zero, but grace is > kept. In essence it adds a softpurge; statement in VCL that can be used > where purge; ordinarily is. > > The main use case for soft purges are big Varnish setups with many > backends and automated purging systems. When doing maintenance on one of > the backends, you end up with automatically purging something that can't be > recreated right now and Varnish starts sending 503 replies. > With softpurge the grace handling will kick in and serve a stale object instead. Is there anything about this that actually becomes different in the "with many backends" scenario? I've been looking forward to this feature for many usecases with a single backend - when that backend is slow or unreliable. I don't see why the number of backends would make any difference, but am I missing something about the implementation? (Automated purging is exactly the usecase for me though) -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ From lasse.karstensen at gmail.com Sat Jan 5 20:55:58 2013 From: lasse.karstensen at gmail.com (Lasse Karstensen) Date: Sat, 5 Jan 2013 21:55:58 +0100 Subject: Soft purge support (patch) In-Reply-To: References: <20130104143716.GA8713@sierra.hyse.org> Message-ID: <20130105205557.GD8713@sierra.hyse.org> Magnus Hagander: > On Fri, Jan 4, 2013 at 3:37 PM, Lasse Karstensen > wrote: >> Attached is a small patch set that implement soft purge support >> for Varnish trunk. [..] >> The main use case for soft purges are big Varnish setups with many >> backends and automated purging systems. When doing maintenance on one of >> the backends, you end up with automatically purging something that can't be >> recreated right now and Varnish starts sending 503 replies. >> With softpurge the grace handling will kick in and serve a stale object instead. > Is there anything about this that actually becomes different in the > "with many backends" scenario? I've been looking forward to this > feature for many usecases with a single backend - when that backend is > slow or unreliable. I don't see why the number of backends would make > any difference, but am I missing something about the implementation? > (Automated purging is exactly the usecase for me though) No, there is nothing in there that is specific to how many backends you have. I was just trying to set the stage a bit, since it might not be entirely obvious when you need this. -- Lasse Karstensen Varnish Software AS http://www.varnish-software.com/ From geoff at uplex.de Mon Jan 7 13:30:03 2013 From: geoff at uplex.de (Geoff Simmons) Date: Mon, 07 Jan 2013 14:30:03 +0100 Subject: no varnishstat In-Reply-To: References: Message-ID: <50EACDDB.9000003@uplex.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 12/20/2012 04:26 AM, ??????? wrote: > varnish-3.0.2 no varnishstat command > why ?? That can happen if you don't have curses installed when you install Varnish. If you need to install curses/ncurses libraries, then installing Varnish again afterwards usually solves the problem. Best, Geoff - -- ** * * UPLEX - Nils Goroll Systemoptimierung Scheffelstra?e 32 (ACHTUNG: neue Adresse) 22301 Hamburg Tel +49 40 2880 5731 Mob +49 176 636 90917 Fax +49 40 42949753 http://uplex.de -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQIcBAEBCAAGBQJQ6s3bAAoJEOUwvh9pJNURoVAQALMpe8EiIjWLCPIg4ItImfPb 4By/8GO1KWOXQ9p2elMFFZOeMH7QmSyis0S/SN88aEXdkCBEemHw0F+xIXevxEhw I0iwcSiXss1Tt5jmXg20ErpZkM4ZUtyQRwN8xv/fPPIOyOONK7rtSf6ZsmrtQ6Do KrM1YAECZRix97JzeBuy5c6NWlJL34x6dy7HNTpQrXaW1KGpNZgyg/ZZ3BQJSupp gz+nhx8RNFbw/E8Ia/NLiOqdt7TmlHj6VynrPFZe66WRFDK7mTxlioYv9E0YuXQv LBCpDd7fuc1mriIljxypni/Hr+ST1EWAFdVV/WstH5sFBVYpM0LPoq5CI7+p6o6L H+TntF2NXWsQmp2ZboAvj5Z2b2Xhw1ixG6UxhbNvWw5S4/7mckxE6Ic2wkJd2p5o RqzA2am0LYIbT/oCZhITdAed19kc7FCWn0DS32N4NUfmU+RpY3b3JMOwQNqYzpag k4/T8YByUM4MJp6SEUy76rupJ4IlfiV5RhbKfmFiC5IGDa5B54bYTkAV5+GbQG+u wSyyl0TBEkPbjZZ8pAAt47Z1kODDIyyMaaux/oFh/QvNlaBDw/c0w90+JEYnRl5d Dh7MDnVrSoZCRIKD1fVQboxqbGZpQGPmIe7FWVhmJh6+11l4t1fAzaR8ByyfLNYD KcTzBaJSTn14K2AmZgqx =2GOV -----END PGP SIGNATURE----- From drwilco at drwilco.net Mon Jan 7 18:20:52 2013 From: drwilco at drwilco.net (Rogier 'DocWilco' Mulhuijzen) Date: Mon, 7 Jan 2013 10:20:52 -0800 Subject: Soft purge support (patch) In-Reply-To: <20130105205557.GD8713@sierra.hyse.org> References: <20130104143716.GA8713@sierra.hyse.org> <20130105205557.GD8713@sierra.hyse.org> Message-ID: How is this patch set different from the soft-purges that Varnish Software was working on? On Sat, Jan 5, 2013 at 12:55 PM, Lasse Karstensen < lasse.karstensen at gmail.com> wrote: > Magnus Hagander: > > On Fri, Jan 4, 2013 at 3:37 PM, Lasse Karstensen > > wrote: > >> Attached is a small patch set that implement soft purge support > >> for Varnish trunk. > [..] > >> The main use case for soft purges are big Varnish setups with many > >> backends and automated purging systems. When doing maintenance on one of > >> the backends, you end up with automatically purging something that > can't be > >> recreated right now and Varnish starts sending 503 replies. > >> With softpurge the grace handling will kick in and serve a stale object > instead. > > Is there anything about this that actually becomes different in the > > "with many backends" scenario? I've been looking forward to this > > feature for many usecases with a single backend - when that backend is > > slow or unreliable. I don't see why the number of backends would make > > any difference, but am I missing something about the implementation? > > (Automated purging is exactly the usecase for me though) > > No, there is nothing in there that is specific to how many backends you > have. > > I was just trying to set the stage a bit, since it might not be entirely > obvious when you need this. > > -- > Lasse Karstensen > Varnish Software AS > http://www.varnish-software.com/ > > _______________________________________________ > varnish-dev mailing list > varnish-dev at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From drwilco at drwilco.net Mon Jan 7 18:21:28 2013 From: drwilco at drwilco.net (Rogier 'DocWilco' Mulhuijzen) Date: Mon, 7 Jan 2013 10:21:28 -0800 Subject: Soft purge support (patch) In-Reply-To: References: <20130104143716.GA8713@sierra.hyse.org> <20130105205557.GD8713@sierra.hyse.org> Message-ID: Oops, just now spotted your signature. Is this just an iteration? On Mon, Jan 7, 2013 at 10:20 AM, Rogier 'DocWilco' Mulhuijzen < drwilco at drwilco.net> wrote: > How is this patch set different from the soft-purges that Varnish Software > was working on? > > > On Sat, Jan 5, 2013 at 12:55 PM, Lasse Karstensen < > lasse.karstensen at gmail.com> wrote: > >> Magnus Hagander: >> > On Fri, Jan 4, 2013 at 3:37 PM, Lasse Karstensen >> > wrote: >> >> Attached is a small patch set that implement soft purge support >> >> for Varnish trunk. >> [..] >> >> The main use case for soft purges are big Varnish setups with many >> >> backends and automated purging systems. When doing maintenance on one >> of >> >> the backends, you end up with automatically purging something that >> can't be >> >> recreated right now and Varnish starts sending 503 replies. >> >> With softpurge the grace handling will kick in and serve a stale >> object instead. >> > Is there anything about this that actually becomes different in the >> > "with many backends" scenario? I've been looking forward to this >> > feature for many usecases with a single backend - when that backend is >> > slow or unreliable. I don't see why the number of backends would make >> > any difference, but am I missing something about the implementation? >> > (Automated purging is exactly the usecase for me though) >> >> No, there is nothing in there that is specific to how many backends you >> have. >> >> I was just trying to set the stage a bit, since it might not be entirely >> obvious when you need this. >> >> -- >> Lasse Karstensen >> Varnish Software AS >> http://www.varnish-software.com/ >> >> _______________________________________________ >> varnish-dev mailing list >> varnish-dev at varnish-cache.org >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lasse.karstensen at gmail.com Tue Jan 8 10:17:59 2013 From: lasse.karstensen at gmail.com (Lasse Karstensen) Date: Tue, 8 Jan 2013 11:17:59 +0100 Subject: Soft purge support (patch) In-Reply-To: References: <20130104143716.GA8713@sierra.hyse.org> <20130105205557.GD8713@sierra.hyse.org> Message-ID: <20130108101758.GA27950@sierra.hyse.org> Rogier 'DocWilco' Mulhuijzen: > Oops, just now spotted your signature. Is this just an iteration? [..] Just the same feature ported to trunk. -- Lasse Karstensen Varnish Software AS http://www.varnish-software.com/ From phk at phk.freebsd.dk Wed Jan 9 10:17:17 2013 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Wed, 09 Jan 2013 10:17:17 +0000 Subject: V4 VCL ideas Message-ID: <2205.1357726637@critter.freebsd.dk> I'm still struggling with designing the V4 VCL language, and thought it was time to run my current sketch past the rest of you. One thing I'm particularly struggling with now, is how much we want to involve VCL in the mechanics of the protocol. Right now we are somewhat schizofrenic about this, for instance we handle incoming "Connection: xxx" in C-code, but allow VCL to set it on the way out. One possible consistent view is that _everything_ which might smell the least of policy, should be in VCL. The obvious downside is that VCL becomes quite complicated. Another possible consistent view is that we handle all protocol mechanics in the C-code, and only put "clean content policy" in VCL. That sounds like it could be a cleaner VCL language to me. As far as I can tell, the major argument for the first approach is trust management. For instance, can you trust the X-Forwarded-For header, can you not trust it, do you not care about it at all ? If it is handled in C-code, invisible to the VCL code, users may not realize that this is something they should think about in the first place. Right now I'm exploring a model where the VCL has a "core" part, which takes a HTTP request and responds to it, and a number of protocol specific parts, to enable people to get their hands on the low-level stuff. My current mock-up default.vcl presently looks like this, input, ideads and comments are most welcome. Don't hang yourself too much in specific names, those are just sort of place holders, it's more the overall structure I am interested in. ... and no, I still don't know what to do about vcl_error{}... Poul-Henning /*- * Copyright (c) 2006 Verdens Gang AS * Copyright (c) 2006-2013 Varnish Software AS * All rights reserved. * * Author: Poul-Henning Kamp * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * The default VCL code. * * NB! You do NOT need to copy & paste all of these functions into your * own vcl code, if you do not provide a definition of one of these * functions, the compiler will automatically fall back to the default * code from this file. * * This code will be prefixed with a backend declaration built from the * -b argument. */ sub vcl_http1_req { /* * V4: This is the new protocol specific "get-here-first" function * V4: in the future we may also have vcl_http2_req{}, vcl_spdy_req{} * V4: and so on. * V4: These functions are only invoked once, restart goes to vcl_req{} */ if (req.http.x-forwarded-for) { set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip; } else { set req.http.X-Forwarded-For = client.ip; } if (xxx_something_bad) { return (error); /* XXX V4: synthetic */ } return (req); } sub vcl_req { /* * V4 Formerly known as vcl_recv{} */ if (req.method != "GET" && req.method != "HEAD" && req.method != "PUT" && req.method != "POST" && req.method != "TRACE" && req.method != "OPTIONS" && req.method != "DELETE") { /* Non-RFC2616 or CONNECT which is weird. */ return (pipe); } if (req.method != "GET" && req.method != "HEAD") { /* We only deal with GET and HEAD by default */ return (pass); /* XXX V4: return(bereq) ? */ } if (req.http.Authorization || req.http.Cookie) { /* Not cacheable by default */ return (pass); /* XXX V4: return(bereq) ? */ } return (hash); } sub vcl_http1_pipe { /* * V4: Pipe is protocol specific as far as I can tell */ # Note that only the first request to the backend will have # X-Forwarded-For set. If you use X-Forwarded-For and want to # have it set for all requests, make sure to have: # set bereq.http.connection = "close"; # here. It is not set by default as it might break some broken web # applications, like IIS with NTLM authentication. return (pipe); } sub vcl_hash { hash_data(req.url); if (req.http.host) { hash_data(req.http.host); } else { hash_data(server.ip); hash_data(server.port); } return (lookup); } sub vcl_lookup { /* * V4: Formerly part of vcl_hit{}, vcl_miss{} and vcl_pass{} */ if (is_pass(obj)) { return (pass); /* XXX: V4: return(bereq) ? */ } if (obj.ttl > 0 s) { return (deliver); /* XXX: V4: return (resp) */ } background_fetch(obj); if (obj.grace > 0 s) { return (deliver); /* XXX: V4: return (resp) */ } return (fetch); } sub vcl_resp { /* * V4: Not obvious if this is/might be protocol specific */ return (deliver); } sub vcl_bereq { /* * V4: Formerly part of vcl_pass{} and vcl_miss{} * V4: Not obvious if this is/might be protocol specific */ } sub vcl_beresp { /* * V4: Formerly vcl_fetch{} * V4: Not obvious if this is/might be protocol specific */ if (beresp.ttl <= 0s || beresp.http.Set-Cookie || beresp.http.Vary == "*") { /* * Mark as "Hit-For-Pass" for the next 2 minutes */ set beresp.ttl = 120 s; set beresp.pass = true; } return (insert); } /* * We can come here "invisibly" with the following errors: 413, 417 & 503 */ sub vcl_error { set obj.http.Content-Type = "text/html; charset=utf-8"; set obj.http.Retry-After = "5"; synthetic {" "} + obj.status + " " + obj.response + {"

Error "} + obj.status + " " + obj.response + {"

"} + obj.response + {"

Guru Meditation:

XID: "} + req.xid + {"

Varnish cache server

"}; return (deliver); } sub vcl_init { return (ok); } sub vcl_fini { return (ok); } -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From Raul.Rangel at disney.com Wed Jan 9 16:38:57 2013 From: Raul.Rangel at disney.com (Rangel, Raul) Date: Wed, 9 Jan 2013 08:38:57 -0800 Subject: V4 VCL ideas In-Reply-To: <2205.1357726637@critter.freebsd.dk> References: <2205.1357726637@critter.freebsd.dk> Message-ID: <2465AAEEC8B8A242B26ED5F44BCA805F2605E3A997@SM-CALA-VXMB04A.swna.wdpr.disney.com> I like the idea of a vcl_http1_req method that runs before vcl_req but I don't like the idea of making it protocol specific. The reason I like it is that it gives me a place to do upfront processing for a request that I only want to do once. Like setting the X-Forwarded-For, stripping out cookies, geopip lookups, digest computations, etc. Then on a restart I know that those items have already been completed. The reason I don't like the idea of making it protocol specific is that then I have to add all that logic to each vcl_XXX_req method or move it down to vcl_req and check to see if it's the first time through vcl_req. I would propose using req.proto or something similar. sub vcl_pre_req { /* * V4: These functions are only invoked once, restart goes to vcl_req{} */ / * XXX V4: The user could add all the protocol specific code if they need it. */ if (req.proto == "HTTP1.0" || req.proto == "HTTP1.1") { ... } elseif (req.proto == "HTTP2.0") { ... } / * XXX V4: Set the X-Forwarded-For for all protocols. */ if (req.http.x-forwarded-for) { set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip; } else { set req.http.X-Forwarded-For = client.ip; } if (xxx_something_bad) { return (error); /* XXX V4: synthetic */ } / * XXX V4: Do some heavy lifting here. */ req.http.cookie = ...; req.http.X-Country = ...; req.http.X-Bucket = ...; return (req); } I would adopt the req.proto approach in the rest of the functions also. How does vcl_lookup work. Are we always guaranteed to have an obj, or do we have to check for existence? If it always exists how can we tell if the obj represents a cache miss? Keep up the great work! Raul -----Original Message----- From: varnish-dev-bounces at varnish-cache.org [mailto:varnish-dev-bounces at varnish-cache.org] On Behalf Of Poul-Henning Kamp Sent: Wednesday, January 09, 2013 3:17 AM To: varnish-dev at varnish-cache.org Subject: V4 VCL ideas I'm still struggling with designing the V4 VCL language, and thought it was time to run my current sketch past the rest of you. One thing I'm particularly struggling with now, is how much we want to involve VCL in the mechanics of the protocol. Right now we are somewhat schizofrenic about this, for instance we handle incoming "Connection: xxx" in C-code, but allow VCL to set it on the way out. One possible consistent view is that _everything_ which might smell the least of policy, should be in VCL. The obvious downside is that VCL becomes quite complicated. Another possible consistent view is that we handle all protocol mechanics in the C-code, and only put "clean content policy" in VCL. That sounds like it could be a cleaner VCL language to me. As far as I can tell, the major argument for the first approach is trust management. For instance, can you trust the X-Forwarded-For header, can you not trust it, do you not care about it at all ? If it is handled in C-code, invisible to the VCL code, users may not realize that this is something they should think about in the first place. Right now I'm exploring a model where the VCL has a "core" part, which takes a HTTP request and responds to it, and a number of protocol specific parts, to enable people to get their hands on the low-level stuff. My current mock-up default.vcl presently looks like this, input, ideads and comments are most welcome. Don't hang yourself too much in specific names, those are just sort of place holders, it's more the overall structure I am interested in. ... and no, I still don't know what to do about vcl_error{}... Poul-Henning /*- * Copyright (c) 2006 Verdens Gang AS * Copyright (c) 2006-2013 Varnish Software AS * All rights reserved. * * Author: Poul-Henning Kamp * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * The default VCL code. * * NB! You do NOT need to copy & paste all of these functions into your * own vcl code, if you do not provide a definition of one of these * functions, the compiler will automatically fall back to the default * code from this file. * * This code will be prefixed with a backend declaration built from the * -b argument. */ sub vcl_http1_req { /* * V4: This is the new protocol specific "get-here-first" function * V4: in the future we may also have vcl_http2_req{}, vcl_spdy_req{} * V4: and so on. * V4: These functions are only invoked once, restart goes to vcl_req{} */ if (req.http.x-forwarded-for) { set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip; } else { set req.http.X-Forwarded-For = client.ip; } if (xxx_something_bad) { return (error); /* XXX V4: synthetic */ } return (req); } sub vcl_req { /* * V4 Formerly known as vcl_recv{} */ if (req.method != "GET" && req.method != "HEAD" && req.method != "PUT" && req.method != "POST" && req.method != "TRACE" && req.method != "OPTIONS" && req.method != "DELETE") { /* Non-RFC2616 or CONNECT which is weird. */ return (pipe); } if (req.method != "GET" && req.method != "HEAD") { /* We only deal with GET and HEAD by default */ return (pass); /* XXX V4: return(bereq) ? */ } if (req.http.Authorization || req.http.Cookie) { /* Not cacheable by default */ return (pass); /* XXX V4: return(bereq) ? */ } return (hash); } sub vcl_http1_pipe { /* * V4: Pipe is protocol specific as far as I can tell */ # Note that only the first request to the backend will have # X-Forwarded-For set. If you use X-Forwarded-For and want to # have it set for all requests, make sure to have: # set bereq.http.connection = "close"; # here. It is not set by default as it might break some broken web # applications, like IIS with NTLM authentication. return (pipe); } sub vcl_hash { hash_data(req.url); if (req.http.host) { hash_data(req.http.host); } else { hash_data(server.ip); hash_data(server.port); } return (lookup); } sub vcl_lookup { /* * V4: Formerly part of vcl_hit{}, vcl_miss{} and vcl_pass{} */ if (is_pass(obj)) { return (pass); /* XXX: V4: return(bereq) ? */ } if (obj.ttl > 0 s) { return (deliver); /* XXX: V4: return (resp) */ } background_fetch(obj); if (obj.grace > 0 s) { return (deliver); /* XXX: V4: return (resp) */ } return (fetch); } sub vcl_resp { /* * V4: Not obvious if this is/might be protocol specific */ return (deliver); } sub vcl_bereq { /* * V4: Formerly part of vcl_pass{} and vcl_miss{} * V4: Not obvious if this is/might be protocol specific */ } sub vcl_beresp { /* * V4: Formerly vcl_fetch{} * V4: Not obvious if this is/might be protocol specific */ if (beresp.ttl <= 0s || beresp.http.Set-Cookie || beresp.http.Vary == "*") { /* * Mark as "Hit-For-Pass" for the next 2 minutes */ set beresp.ttl = 120 s; set beresp.pass = true; } return (insert); } /* * We can come here "invisibly" with the following errors: 413, 417 & 503 */ sub vcl_error { set obj.http.Content-Type = "text/html; charset=utf-8"; set obj.http.Retry-After = "5"; synthetic {" "} + obj.status + " " + obj.response + {"

Error "} + obj.status + " " + obj.response + {"

"} + obj.response + {"

Guru Meditation:

XID: "} + req.xid + {"

Varnish cache server

"}; return (deliver); } sub vcl_init { return (ok); } sub vcl_fini { return (ok); } -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. _______________________________________________ varnish-dev mailing list varnish-dev at varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev From slink at schokola.de Wed Jan 9 18:39:13 2013 From: slink at schokola.de (Nils Goroll) Date: Wed, 09 Jan 2013 19:39:13 +0100 Subject: [PATCH] portable cast from thread_id to (void *) Message-ID: <50EDB951.6090400@schokola.de> Fixes #932 ( actually a repost of Message-ID: <500801FD.2030403 at uplex.de> Date: Thu, 19 Jul 2012 14:47:57 +0200 From: Geoff Simmons Subject: [PATCHES] Fix build errors and sandbox bugs in the Solaris port ) -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: 0001-portable-cast-from-thread_id-to-void.patch URL: From slink at schokola.de Wed Jan 9 18:49:20 2013 From: slink at schokola.de (Nils Goroll) Date: Wed, 09 Jan 2013 19:49:20 +0100 Subject: [PATCH] make autocrap(tm_phk) tools happy Message-ID: <50EDBBB0.1050207@schokola.de> Reduce warning noise from autoconf Make automake happy Pull in current ld-version-script.m4 which fixes the issue documented here: https://lists.gnu.org/archive/html/bug-gnulib/2010-08/msg00198.html -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: 0002-make-autocrap-tm_phk-tools-happy.patch URL: From slink at schokola.de Wed Jan 9 19:49:41 2013 From: slink at schokola.de (Nils Goroll) Date: Wed, 09 Jan 2013 20:49:41 +0100 Subject: [PATCH] proper dependencies for vmod automake Message-ID: <50EDC9D5.2030808@schokola.de> parallel builds (make -jX) failed sporadically for the first compilation (before dependency tracking information was available) due to the race with python building vcc_if.h -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: 0003-proper-dependencies-for-vmod-automake.patch URL: From slink at schokola.de Wed Jan 9 20:01:00 2013 From: slink at schokola.de (Nils Goroll) Date: Wed, 09 Jan 2013 21:01:00 +0100 Subject: V4 VCL ideas In-Reply-To: <2465AAEEC8B8A242B26ED5F44BCA805F2605E3A997@SM-CALA-VXMB04A.swna.wdpr.disney.com> References: <2205.1357726637@critter.freebsd.dk> <2465AAEEC8B8A242B26ED5F44BCA805F2605E3A997@SM-CALA-VXMB04A.swna.wdpr.disney.com> Message-ID: <50EDCC7C.6000301@schokola.de> On 01/ 9/13 05:38 PM, Rangel, Raul wrote: > The reason I don't like the idea of making it protocol specific is that then I have to add all that logic to each vcl_XXX_req method IIUC in the SPDY case req.proto would refer to the SPDYfied HTTP version and not to SPDY itself. If you want to avoid code duplication, you could just call common vcl code. From slink at schokola.de Wed Jan 9 21:10:57 2013 From: slink at schokola.de (Nils Goroll) Date: Wed, 09 Jan 2013 22:10:57 +0100 Subject: V4 VCL ideas In-Reply-To: <2205.1357726637@critter.freebsd.dk> References: <2205.1357726637@critter.freebsd.dk> Message-ID: <50EDDCE1.9020000@schokola.de> On 01/ 9/13 11:17 AM, Poul-Henning Kamp wrote: > One possible consistent view is that _everything_ which might smell > the least of policy, should be in VCL. The obvious downside is > that VCL becomes quite complicated. I strongly believe that this is the right way to go, otherwise we won't get rid of magic knobs and the spirit of VCL as I perceive it is really to give control over all relevant aspects of Varnish in a manner as concise as possible and with high performance. My impression is that VCL already is too complicated for some (just consider regular expressions...), so to make novices' lives easier, having a config generator or good templates/examples would be the way to go, IMHO. > As far as I can tell, the major argument for the first approach > is trust management. I'd call it control or conciseness, but otherwise agree. > My current mock-up default.vcl presently looks like this, input, > ideads and comments are most welcome. see below. > ... and no, I still don't know what to do about vcl_error{}... I don't feel up-to-date on this, could you summarize the situation (again?) or point to the most up to date information on this? The latest I have is Tollefs Email as of 2012-01-12 and some traces from VUG6 in my bad memory. Other than that, I have proposed in https://www.varnish-cache.org/trac/wiki/Future_VCL to allow for vcl_error to not close the conn, which would be obsoleted if vcl_error was never called but for real (= varnish C-triggerd) errors. I am also referring to the wiki page for some of the following comments: * how would the always_miss and ignore_busy replacements look like? * how would purge code look like? * I think it would help to draw again the clear line between client and backend-side. which vcl callbacks live in which thread? DIUC that the line is above sub bereq ? > sub vcl_req { what aboot req.body and return(purge)? > sub vcl_lookup { > /* > * V4: Formerly part of vcl_hit{}, vcl_miss{} and vcl_pass{} > */ > if (is_pass(obj)) { So the hit-for-pass object would be called "pass object"? When I explain the current hit-for-pass logic, I most often use terms like "negative caching entry" or "no-caching marker object". So "pass (marker) object" seems to fill well. More concise than "hit for pass object" anyway. Yes, the C-code branch to hit miss and pass needs to go into lookup, definitely. sub vcl_bereq is nice because this is just what it does. Would I see a potential obj here (if lookup has found one)? I feel I should really revisit this again any try to port real VCL code to this concept. Nils From lkarsten at varnish-software.com Thu Jan 10 12:08:07 2013 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Thu, 10 Jan 2013 13:08:07 +0100 Subject: Split losthdr between client and backend Message-ID: <20130110120805.GA3581@immer.varnish-software.com> Hello. I'd like to propose an additional varnishstat counter that counts requests rejected (413) due to http_req_hdr_len overflow. When we do Varnish diagnostics for customers that are having problems, we often see increased losthdr. It would be very helpful to know if it was a broken client (usually loads of cookies) or a backend response that made this counter increase. In the case of a client you can increase http_req_hdr_(len|size), and in the backend case you probably want to fix your backend. Knowing which solution to recommend would be a big plus. I had a look at the code and adding this seems pretty simple to put into cache/cache_http1_fsm.c's http1_dissect(). (trivial patch here: http://pastie.org/5661064) However, I'm not convinced that this is the best place to do this, or that this is the best way of solving my initial problem. I'd appreciate any input on this. -- With regards, Lasse Karstensen Varnish Software AS From phk at phk.freebsd.dk Mon Jan 14 13:56:28 2013 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Mon, 14 Jan 2013 13:56:28 +0000 Subject: Split losthdr between client and backend In-Reply-To: <20130110120805.GA3581@immer.varnish-software.com> References: <20130110120805.GA3581@immer.varnish-software.com> Message-ID: <33829.1358171788@critter.freebsd.dk> Content-Type: text/plain; charset=ISO-8859-1 -------- In message <20130110120805.GA3581 at immer.varnish-software.com>, Lasse Karstensen writes: >I'd like to propose an additional varnishstat counter that counts requests >rejected (413) due to http_req_hdr_len overflow. I have mucked about and done something along these lines, let me know if it works for you. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From lkarsten at varnish-software.com Mon Jan 14 14:19:28 2013 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Mon, 14 Jan 2013 15:19:28 +0100 Subject: Split losthdr between client and backend In-Reply-To: <33829.1358171788@critter.freebsd.dk> References: <20130110120805.GA3581@immer.varnish-software.com> <33829.1358171788@critter.freebsd.dk> Message-ID: <20130114141927.GB10140@immer.varnish-software.com> On Mon, Jan 14, 2013 at 01:56:28PM +0000, Poul-Henning Kamp wrote: > In message <20130110120805.GA3581 at immer.varnish-software.com>, Lasse Karstensen > writes: > >I'd like to propose an additional varnishstat counter that counts requests > >rejected (413) due to http_req_hdr_len overflow. > I have mucked about and done something along these lines, let me know > if it works for you. >From what I can see it looks good. Thanks. -- With regards, Lasse Karstensen Varnish Software AS From jonathan.huot at thomsonreuters.com Tue Jan 15 12:56:52 2013 From: jonathan.huot at thomsonreuters.com (jonathan.huot at thomsonreuters.com) Date: Tue, 15 Jan 2013 12:56:52 +0000 Subject: varnishtest improvement for VMOD developers Message-ID: <8E656B642592B942AE317E2AFAE0ABA1305726DC@UK2P-ERFMMBX10.ERF.thomson.com> Hi everyone, I made custom changes in varnishtest sources (vtc_http.c) to get -gzipbody feature available within txreq command. I don't know if it is useful for varnish core because varnish usually don't care about requests body (POST), but it is useful for me as a VMOD developer. Does it make sense if I suggest patches to improve vtc_http.c in order to have identical features for txreq and txresp commands ? -- jonathan.huot at thomsonreuters.com This email was sent to you by Thomson Reuters, the global news and information company. Any views expressed in this message are those of the individual sender, except where the sender specifically states them to be the views of Thomson Reuters. From phk at phk.freebsd.dk Tue Jan 15 14:41:58 2013 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Tue, 15 Jan 2013 14:41:58 +0000 Subject: varnishtest improvement for VMOD developers In-Reply-To: <8E656B642592B942AE317E2AFAE0ABA1305726DC@UK2P-ERFMMBX10.ERF.thomson.com> References: <8E656B642592B942AE317E2AFAE0ABA1305726DC@UK2P-ERFMMBX10.ERF.thomson.com> Message-ID: <53955.1358260918@critter.freebsd.dk> Content-Type: text/plain; charset=ISO-8859-1 -------- In message <8E656B642592B942AE317E2AFAE0ABA1305726DC at UK2P-ERFMMBX10.ERF.thomson .com>, jonathan.huot at thomsonreuters.com writes: >I made custom changes in varnishtest sources (vtc_http.c) to get -gzipbody feature available within txreq command. >I don't know if it is useful for varnish core because varnish usually don't care about requests body (POST), but it is useful for me as a VMOD developer. Send patch :-) -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From jonathan.huot at thomsonreuters.com Mon Jan 21 15:21:31 2013 From: jonathan.huot at thomsonreuters.com (jonathan.huot at thomsonreuters.com) Date: Mon, 21 Jan 2013 15:21:31 +0000 Subject: varnishtest improvement for VMOD developers Message-ID: <8E656B642592B942AE317E2AFAE0ABA130573744@UK2P-ERFMMBX10.ERF.thomson.com> > -----Original Message----- > From: Poul-Henning Kamp [mailto:phk at phk.freebsd.dk] > >I made custom changes in varnishtest sources (vtc_http.c) to get -gzipbody > feature available within txreq command. > >I don't know if it is useful for varnish core because varnish usually don't care > about requests body (POST), but it is useful for me as a VMOD developer. > > Send patch :-) I was a little busy, but finally I send two patches. They're based on 3.0.2 but you can apply them on git-master with success. Feel free to comments. -- jonathan.huot at thomsonreuters.com This email was sent to you by Thomson Reuters, the global news and information company. Any views expressed in this message are those of the individual sender, except where the sender specifically states them to be the views of Thomson Reuters. -------------- next part -------------- A non-text attachment was scrubbed... Name: a00011.vtc.patch Type: application/octet-stream Size: 575 bytes Desc: a00011.vtc.patch URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: vtc_http.c.patch Type: application/octet-stream Size: 4727 bytes Desc: vtc_http.c.patch URL: From kristian at varnish-software.com Tue Jan 22 11:54:06 2013 From: kristian at varnish-software.com (Kristian Lyngstol) Date: Tue, 22 Jan 2013 12:54:06 +0100 Subject: Varnish Agent 2.0 released Message-ID: <20130122115406.GA11225@freud.kly.no> We just released the Varnish Agent 2.0. It's released under the same license as Varnish. The source is available at github [1] and I've set up a demo of it too [2]. Feel free to break the demo. Debian Squeeze/Wheezy and Ubuntu Precise packages have been built for convenience [3]. The Varnish Agent is a REST interface to the Varnish CLI and shmlog, adding a bit of persistence to allow remote control of Varnish. Version 2.0 is completely rewritten and can be considered a tech preview, as it doesn't support any means of authentication yet. There are a number of well known bugs, as is evident on the github issue tracker. We'll be relying on this for our proprietary Varnish Administration Console, but hopefully others can find it useful too. Any and all feedback is welcome. [1] http://github.com/varnish/vagent2 [2] http://84.209.194.23:6085/html/ [3] http://users.varnish-software.com/~kristian/agent/ - Kristian PS: I am not a front end developer! My javascript/frontend code is probably horrible. From slink at schokola.de Tue Jan 22 21:12:10 2013 From: slink at schokola.de (Nils Goroll) Date: Tue, 22 Jan 2013 22:12:10 +0100 Subject: [PATCH] remove invalid assertion in vws_thread() Message-ID: <50FF00AA.7080107@schokola.de> An embedded and charset-unspecified text was scrubbed... Name: 0001-remove-invalid-assertion-in-vws_thread.patch URL: From JSmall at daraco.com.au Wed Jan 23 02:26:19 2013 From: JSmall at daraco.com.au (Joshua Small) Date: Wed, 23 Jan 2013 02:26:19 +0000 Subject: Varnish on a pi Message-ID: <6223D6171835FF46A542643BF7A54A203C0D34B9@DARAEX01.business.ad> Hey guys, I've spent a lot of time working on this issue following my post on misc: https://www.varnish-cache.org/lists/pipermail/varnish-misc/2013-January/022706.html Two things have come out of this. The first is that, I've noted in multiple attempts at fixing this issue, every "make install" clobbers the default.vcl. I believe this is contrary to many standard applications, for example, a typical user might "make install" Apache and their httpd.conf is not clobbered. I've written a patch for this I would like to be considered. This is at the bottom of this email. Expanding on the bigger (and imo, more important) issue, I've had an independent pi user confirm the findings I posted to misc, which really feels like a development bug, hence this post. To avoid a full review of that posting, long story short is that on the pi hardware (armv61): Default configuration= segfault on startup Compile with -enable-diagnostics = successful operation I've spent a lot of time trying to track anything that this flag does. A grep for -DDIAGNOSTICS only seems to turn up the CFLAGS declaration in Makefiles, so I'm still working on what diagnostics it produces. Any assistance you can offer as to how I could debug this issue further would be appreciated. diff --git a/etc/Makefile.am b/etc/Makefile.am index e724822..909c35f 100644 --- a/etc/Makefile.am +++ b/etc/Makefile.am @@ -5,6 +5,16 @@ DISTCLEANFILES = default.vcl dist_varnishconf_DATA = default.vcl +install-exec-local: + @if test -f "$(dist_varnishconf_DATA)"; then \ + echo "$@ will not overwrite existing $(DESTDIR)$(dist_varnishcon + else \ + echo " $(MKDIR_P) '$(DESTDIR)$(varnishconfdir)'"; \ + $(MKDIR_P) "$(DESTDIR)$(varnishconfdir)" || exit 1; \ + echo " $(INSTALL_DATA) $$files '$(DESTDIR)$(varnishconfdir)'"; \ + $(INSTALL_DATA) $$files "$(DESTDIR)$(varnishconfdir)" || exit $$ + fi + default.vcl: $(top_srcdir)/bin/varnishd/default.vcl ( printf "This is a basic VCL configuration file for varnish. See the v man page for details on VCL syntax and semantics.\n\ -------------- next part -------------- An HTML attachment was scrubbed... URL: From jammy.linux at gmail.com Wed Jan 23 02:38:29 2013 From: jammy.linux at gmail.com (Jammy) Date: Wed, 23 Jan 2013 10:38:29 +0800 Subject: race condition issue? Message-ID: We're using varnish 3.0.3 in production, yesterday, we're riding on a heavy DDoS attack. And two of our varnish nodes crashed. it's very similar to https://www.varnish-cache.org/trac/ticket/516. Following is our stack traces. Any ideas? Thanks (gdb) info threads 999 Thread 0x7ff5eb7ff700 (LWP 3100) 0x00000031d000e54d in read () from /lib64/libpthread.so.0 998 Thread 0x7ff5ed9fd700 (LWP 3097) 0x00000031d000e54d in read () from /lib64/libpthread.so.0 ... ... 5 Thread 0x7ff88a8ff700 (LWP 30905) 0x00000031d000e54d in read () from /lib64/libpthread.so.0 4 Thread 0x7ff88cbfd700 (LWP 30903) 0x00000031d000ed2d in nanosleep () from /lib64/libpthread.so.0 3 Thread 0x7ff88dfff700 (LWP 30901) 0x00000031d000ed2d in nanosleep () from /lib64/libpthread.so.0 2 Thread 0x7ff7c81f2700 (LWP 31211) 0x00000031d000e054 in __lll_lock_wait () from /lib64/libpthread.so.0 * 1 Thread 0x7ff894cca780 (LWP 30898) 0x00000031d000e054 in __lll_lock_wait () from /lib64/libpthread.so.0 Program terminated with signal 3, Quit. #0 0x00000031d000e054 in __lll_lock_wait () from /lib64/libpthread.so.0 Missing separate debuginfos, use: debuginfo-install varnish-3.0.3-1.201301210212.production.x86_64 (gdb) bt #0 0x00000031d000e054 in __lll_lock_wait () from /lib64/libpthread.so.0 #1 0x00000031d0009388 in _L_lock_854 () from /lib64/libpthread.so.0 #2 0x00000031d0009257 in pthread_mutex_lock () from /lib64/libpthread.so.0 #3 0x0000000000433b90 in vsl_get () #4 0x0000000000433d48 in VSLR () #5 0x0000000000433f12 in VSL () #6 0x000000000041a18e in cli_cb_before () #7 0x00007ff89559be05 in cls_vlu2 (priv=0x7ff894b41540, av=0x7ff76d5b4080) at cli_serve.c:260 #8 0x00007ff89559c48b in cls_vlu (priv=0x7ff894b41540, p=0x2
) at cli_serve.c:339 #9 0x00007ff89559fe19 in LineUpProcess (l=0x7ff894b33580) at vlu.c:154 #10 0x00007ff89559ce8d in VCLS_Poll (cs=0x7ff894b03240, timeout=) at cli_serve.c:528 #11 0x000000000041a291 in CLI_Run () #12 0x000000000042e24c in child_main () #13 0x0000000000440d2c in start_child () #14 0x00000000004416b8 in MGT_Run () #15 0x000000000044fb4f in main () ---------------------------------- Best wishes, Jammy -------------- next part -------------- An HTML attachment was scrubbed... URL: From phk at phk.freebsd.dk Thu Jan 24 12:15:25 2013 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Thu, 24 Jan 2013 12:15:25 +0000 Subject: Varnish on a pi In-Reply-To: <6223D6171835FF46A542643BF7A54A203C0D34B9@DARAEX01.business.ad> References: <6223D6171835FF46A542643BF7A54A203C0D34B9@DARAEX01.business.ad> Message-ID: <1713.1359029725@critter.freebsd.dk> Content-Type: text/plain; charset=ISO-8859-1 -------- In message <6223D6171835FF46A542643BF7A54A203C0D34B9 at DARAEX01.business.ad>, Jos hua Small writes: Hi Joshua, I suspect the Arm problems are an alignment issue somewhere, but right now I don't have a strict-alignment machine available, so I have not had time to track it down. -enable-diagnostics may possibly change the -O and -g flags to the compiler, which could change the code generation enough to obscure the issue. Any kind of further information you can find, is more than welcome... -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From phk at phk.freebsd.dk Thu Jan 24 12:17:02 2013 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Thu, 24 Jan 2013 12:17:02 +0000 Subject: [PATCH] remove invalid assertion in vws_thread() In-Reply-To: <50FF00AA.7080107@schokola.de> References: <50FF00AA.7080107@schokola.de> Message-ID: <1732.1359029822@critter.freebsd.dk> Content-Type: text/plain; charset=ISO-8859-1 -------- Applied. In message <50FF00AA.7080107 at schokola.de>, Nils Goroll writes: >This is a multi-part message in MIME format. >--------------090507030006090308070204 >Content-Type: text/plain; charset=ISO-8859-1; format=flowed >Content-Transfer-Encoding: 7bit > > >--------------090507030006090308070204 >Content-Type: text/plain; > name="0001-remove-invalid-assertion-in-vws_thread.patch" >Content-Transfer-Encoding: 7bit >Content-Disposition: attachment; > filename="0001-remove-invalid-assertion-in-vws_thread.patch" > >>From 168a14c1dc01cfc360ea831e5e9e8104f46f37c4 Mon Sep 17 00:00:00 2001 >>From 168a14c1dc01cfc360ea831e5e9e8104f46f37c4 Mon Sep 17 00:00:00 2001 >From: Nils Goroll >Date: Tue, 22 Jan 2013 22:08:09 +0100 >Subject: [PATCH] remove invalid assertion in vws_thread() > >tmo can become negativ if timeout_idle gets changed on the fly >or when the rtc jumps >--- > bin/varnishd/waiter/cache_waiter_ports.c | 6 ------ > 1 files changed, 0 insertions(+), 6 deletions(-) > >diff --git a/bin/varnishd/waiter/cache_waiter_ports.c b/bin/varnishd/waiter/cache_waiter_ports.c >index aa3d766..3c07b95 100644 >--- a/bin/varnishd/waiter/cache_waiter_ports.c >+++ b/bin/varnishd/waiter/cache_waiter_ports.c >@@ -222,12 +222,6 @@ vws_thread(void *priv) > double tmo = > (sp->t_idle + cache_param->timeout_idle) - now; > >- /* >- * we should have removed all sps whose timeout >- * has passed >- */ >- assert(tmo > 0.0); >- > if (tmo < min_t) { > timeout = &min_ts; > } else if (tmo > max_t) { >-- >1.5.6.5 > > >--------------090507030006090308070204 >Content-Type: text/plain; charset="us-ascii" >MIME-Version: 1.0 >Content-Transfer-Encoding: 7bit >Content-Disposition: inline > >_______________________________________________ >varnish-dev mailing list >varnish-dev at varnish-cache.org >https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev >--------------090507030006090308070204-- > -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From phk at phk.freebsd.dk Fri Jan 25 08:47:38 2013 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Fri, 25 Jan 2013 08:47:38 +0000 Subject: varnishtest improvement for VMOD developers In-Reply-To: <8E656B642592B942AE317E2AFAE0ABA130573744@UK2P-ERFMMBX10.ERF.thomson.com> References: <8E656B642592B942AE317E2AFAE0ABA130573744@UK2P-ERFMMBX10.ERF.thomson.com> Message-ID: <57284.1359103658@critter.freebsd.dk> Content-Type: text/plain; charset=ISO-8859-1 -------- Applied. Poul-Henning In message <8E656B642592B942AE317E2AFAE0ABA130573744 at UK2P-ERFMMBX10.ERF.thomson .com>, jonathan.huot at thomsonreuters.com writes: >--_003_8E656B642592B942AE317E2AFAE0ABA130573744UK2PERFMMBX10ER_ >Content-Type: text/plain; charset=us-ascii >Content-Transfer-Encoding: 7bit > >> -----Original Message----- >> From: Poul-Henning Kamp [mailto:phk at phk.freebsd.dk] >> >I made custom changes in varnishtest sources (vtc_http.c) to get -gzipbody >> feature available within txreq command. >> >I don't know if it is useful for varnish core because varnish usually don't care >> about requests body (POST), but it is useful for me as a VMOD developer. >> >> Send patch :-) > >I was a little busy, but finally I send two patches. They're based on 3.0.2 but you can apply them on git-master with success. >Feel free to comments. > >-- >jonathan.huot at thomsonreuters.com -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From jammy.linux at gmail.com Tue Jan 29 04:34:13 2013 From: jammy.linux at gmail.com (Jammy) Date: Tue, 29 Jan 2013 12:34:13 +0800 Subject: Content-Length patch when streaming enabled Message-ID: <510958C9-05A5-4C9E-B6D1-7FF62B3D8466@gmail.com> From f4fde2d6247463621b3181396e7b14897f1be903 Mon Sep 17 00:00:00 2001 From: ijammy Date: Tue, 29 Jan 2013 12:31:08 +0800 Subject: [PATCH] when streaming enabled, if we need gzip or ungzip, won't set content-length --- bin/varnishd/cache_response.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bin/varnishd/cache_response.c b/bin/varnishd/cache_response.c index f22d48a..a16b6df 100644 --- a/bin/varnishd/cache_response.c +++ b/bin/varnishd/cache_response.c @@ -351,7 +351,7 @@ RES_StreamStart(struct sess *sp) if (sp->wrk->res_mode & RES_GUNZIP) http_Unset(sp->wrk->resp, H_Content_Encoding); - if (!(sp->wrk->res_mode & RES_CHUNKED) && + if ((sp->wrk->res_mode & RES_LEN) && sp->wrk->h_content_length != NULL) http_PrintfHeader(sp->wrk, sp->fd, sp->wrk->resp, "Content-Length: %s", sp->wrk->h_content_length); -- 1.7.10.2 (Apple Git-33) -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-when-streaming-enabled-if-we-need-gzip-or-ungzip-won.patch Type: application/octet-stream Size: 913 bytes Desc: not available URL: -------------- next part -------------- ---------------------------------- Best wishes, Jammy From klaus.brunner at gmail.com Thu Jan 31 07:56:16 2013 From: klaus.brunner at gmail.com (Klaus Brunner) Date: Thu, 31 Jan 2013 08:56:16 +0100 Subject: [PATCH, sort of] log "total time" in varnishncsa Message-ID: I've patched varnishncsa (as of 3.0.3) to add an option for logging the total time to answer a request. I needed this myself, and it seems to have been requested a couple of times by others (e.g. http://www.gossamer-threads.com/lists/varnish/misc/26408?#26408). The patched version has been used for well over a month now on a few clusters, without any apparent issues: https://gist.github.com/4175582#file-varnishncsa-request-time-patch I've also created a git fork to get this into trunk: https://github.com/KlausBrunner/Varnish-Cache However, I understand that varnishncsa will be broken on trunk as long as the API rewrite isn't finished. So I can't even test it there. Should I submit a patch against trunk anyway, or what's the preferred way of getting this into the next release (assuming that you accept it)? Thanks Klaus