From lcruzero at gmail.com Fri Mar 6 14:18:31 2015 From: lcruzero at gmail.com (L Cruzero) Date: Fri, 6 Mar 2015 09:18:31 -0500 Subject: how to Implement if-modified-since correctly for ESI Message-ID: Hi, I've enabled varnish ESI processing on a config that handles public domains passing through akamai, I've noticed that origin(varnish) is returning a 304 to akamai which then revalidates the object and passes stale content to the client, though ESI content has been modified, properly parsed and visible on the origin domains. running on varnish-3.0.6 revision 1899836 based on some tests, this not so elegant piece of vcl seems to solve the issue, setting defaul TTL to 2mins @ akamai and updating content served by ESI's, i've not seen any ESI content older than 2mins being served. if (beresp.status == 200 && beresp.http.X-ADI-NSAPP == "ssf" && beresp.http.Content-type ~ "html") { set beresp.http.Last-Modified = beresp.http.date; unset beresp.http.X-ADI-NSAPP; if i removed that piece of code from my config, my ESIs go stale. this is what my headers look like with and without the above piece of vcl *without *beresp.http.date *:* HTTP/1.1 200 OK Server: Apache *Last-Modified: Thu, 26 Feb 2015 14:21:06 GMT* ntCoent-Length: 7536 Cache-Control: max-age=1 Expires: Fri, 06 Mar 2015 13:41:28 GMT Content-Type: text/html; charset=UTF-8 X-ADI-NSAPP: ssf X-ADI-Backend: default Date: Fri, 06 Mar 2015 13:41:27 GMT X-Varnish: 1539216520 Age: 0 Via: 1.1 varnish Connection: keep-alive *with *beresp.http.date *:* HTTP/1.1 200 OK Server: Apache ntCoent-Length: 7536 Cache-Control: max-age=1 Expires: Fri, 06 Mar 2015 13:40:03 GMT Content-Type: text/html; charset=UTF-8 X-ADI-Backend: default *Last-Modified: Fri, 06 Mar 2015 13:40:02 GMT* Date: Fri, 06 Mar 2015 13:40:02 GMT X-Varnish: 1125382905 Age: 0 Via: 1.1 varnish Connection: keep-alive a helpful friend passed along this possible solution. https://www.varnish-cache.org/lists/pipermail/varnish-commit/2009-June/004322.html https://www.varnish-cache.org/trac/changeset/c64a6482f235a4f27e830d609c9a102c716d0bc6 this seems like a much more nicer way of handling ESI LM headers. any suggestions for properly handling LM for ESIs or a way to test and implement the above patch committed in 2009, would be most welcomed. kind regards -LC -------------- next part -------------- An HTML attachment was scrubbed... URL: From mhettwer at team.mobile.de Wed Mar 11 10:22:20 2015 From: mhettwer at team.mobile.de (Hettwer, Marian) Date: Wed, 11 Mar 2015 10:22:20 +0000 Subject: RespReason Service Unavailable but no backend connection attempt?! Message-ID: Hi Varnish Users, I?m seeing some weird behavior in a load test I?m doing. When stressing our backends up to the area of 2000 req/s, I?m seeing sporadic 503 responses from varnish. My understanding is, that this would happen if, for instance, no backend is available. I?m looking at my logs like that: varnishlog -n essearchvarnish -q "RespStatus >= 500 or BerespStatus >= 500? A sample 503 output is that one: * << Request >> 27632438 - Begin req 27630762 rxreq - Timestamp Start: 1426066874.412913 0.000000 0.000000 - Timestamp Req: 1426066874.412913 0.000000 0.000000 - ReqStart 10.47.28.251 49641 - ReqMethod GET - ReqURL /svc/somequery?string?documents=1 - ReqProtocol HTTP/1.1 - ReqHeader Accept-Language: de - ReqHeader User-Agent: Java1.8.0 - ReqHeader Host: essearch.local - ReqHeader Connection: Keep-Alive - ReqHeader Accept-Encoding: gzip - VCL_call RECV - VCL_return hash - VCL_call HASH - VCL_return lookup - Debug "XXXX HIT-FOR-PASS" - HitPass 2201620563 - VCL_call PASS - VCL_return fetch - Link bereq 27632439 pass - Timestamp Fetch: 1426066874.412967 0.000054 0.000054 - Timestamp Process: 1426066874.412979 0.000066 0.000012 - RespHeader Date: Wed, 11 Mar 2015 09:41:14 GMT - RespHeader Server: Varnish - RespHeader X-Varnish: 27632438 - RespProtocol HTTP/1.1 - RespStatus 503 - RespReason Service Unavailable - RespReason Service Unavailable - VCL_call SYNTH - RespHeader Content-Type: text/html; charset=utf-8 - RespHeader Retry-After: 5 - VCL_return deliver - RespHeader Content-Length: 282 - Debug "RES_MODE 2" - RespHeader Connection: keep-alive - Timestamp Resp: 1426066874.413033 0.000119 0.000053 - ReqAcct 454 0 454 212 282 494 - End What makes me wonder is, that this output is missing the actual backend request attempts. Also, printing "RespReason Service Unavailable? twice in a row?!? Why? So in a second shell, on the same varnish at the same time, I have this query running: varnishlog -b -n essearchvarnish -q "RespStatus >= 500 or BerespStatus >= 500? I added the -b parameter to see the backend requests. However, I get no output at all. So it seems varnish would throw the 503 without asking the backend? Or, hopefully more likely, I?m not using varnishlog correctly?! This is varnish 4.0.3-2~wheezy. How can I print client and backend connections in one go? Doing using the -c and -b parameter in parallel doesn?t produce any output (while in my other console, I still have the first query which produces the above output). I know that around 2000 req/s my backends are getting a bit slower ( up to 2 seconds for responses), but this should not hit any internal timeouts of varnish, right? I haven?t changed the default timeouts. Also the backends itself are healthy. I?m watching the backends (several tomcats) directly, and also from the varnish perspective via watch -n1 varnishadm -n essearchvarnish backend.list Everything looks healthy. Any help for how I could dig into this further would be appreciated :-) I guess it boils down to better use varnishlog on my side. Like getting client and backend connections for a request at the same time and not by using varnishlog in two shells differently. best regards, Marian From feld at feld.me Wed Mar 11 15:06:23 2015 From: feld at feld.me (Mark Felder) Date: Wed, 11 Mar 2015 10:06:23 -0500 Subject: vagent2 for varnish 4.0 Message-ID: <1426086383.970979.238926525.33CC5ED5@webmail.messagingengine.com> Hi all, Is there a release of vagent2 for Varnish 4.0 yet? I see a 4.0.0-RC2 tag in github[1] from May 2014, but nothing since. Thanks! [1] https://github.com/varnish/vagent2/releases From perbu at varnish-software.com Wed Mar 11 16:58:16 2015 From: perbu at varnish-software.com (Per Buer) Date: Wed, 11 Mar 2015 17:58:16 +0100 Subject: vagent2 for varnish 4.0 In-Reply-To: <1426086383.970979.238926525.33CC5ED5@webmail.messagingengine.com> References: <1426086383.970979.238926525.33CC5ED5@webmail.messagingengine.com> Message-ID: Hi Mark On Wed, Mar 11, 2015 at 4:06 PM, Mark Felder wrote: > Hi all, > > Is there a release of vagent2 for Varnish 4.0 yet? I see a 4.0.0-RC2 tag > in github[1] from May 2014, but nothing since. > Yes. The 4.0 branch supports Varnish 4 (and 3). We might not have tagged it properly, but build from the 4.0 branch and you should be fine. -- *Per Buer* CTO | Varnish Software AS Cell: +47 95839117 We Make Websites Fly! www.varnish-software.com [image: Register now] -------------- next part -------------- An HTML attachment was scrubbed... URL: From feld at feld.me Wed Mar 11 17:20:32 2015 From: feld at feld.me (Mark Felder) Date: Wed, 11 Mar 2015 12:20:32 -0500 Subject: vagent2 for varnish 4.0 In-Reply-To: References: <1426086383.970979.238926525.33CC5ED5@webmail.messagingengine.com> Message-ID: <1426094432.1400899.238993237.603BB3EF@webmail.messagingengine.com> On Wed, Mar 11, 2015, at 11:58, Per Buer wrote: > Hi Mark > > On Wed, Mar 11, 2015 at 4:06 PM, Mark Felder wrote: > > > Hi all, > > > > Is there a release of vagent2 for Varnish 4.0 yet? I see a 4.0.0-RC2 tag > > in github[1] from May 2014, but nothing since. > > > > Yes. The 4.0 branch supports Varnish 4 (and 3). > > We might not have tagged it properly, but build from the 4.0 branch and > you > should be fine. > Thanks for the info. If you manage to tag a release I will make a FreeBSD port/package so others can use it easily as well. From japrice at gmail.com Fri Mar 13 02:04:51 2015 From: japrice at gmail.com (Jason Price) Date: Thu, 12 Mar 2015 22:04:51 -0400 Subject: RespReason Service Unavailable but no backend connection attempt?! In-Reply-To: References: Message-ID: I'm not as familiar with varnish4 (I need vmods), but in varnish 3, backends need to be specified with timeouts: backend backend31 { .host = "10.24.14.112"; .port = "80"; .probe = backendcheck; .connect_timeout = 3s; .first_byte_timeout = 5s; .between_bytes_timeout = 2s; } If your backends are averaging 2+ seconds, then your outliers are probably 5-6 seconds... and that could easily trip the defaults (though I don't recall off-hand what they are). --Jason On Wed, Mar 11, 2015 at 6:22 AM, Hettwer, Marian wrote: > Hi Varnish Users, > > I?m seeing some weird behavior in a load test I?m doing. When stressing > our backends up to the area of 2000 req/s, I?m seeing sporadic 503 > responses from varnish. > My understanding is, that this would happen if, for instance, no backend > is available. > > I?m looking at my logs like that: > varnishlog -n essearchvarnish -q "RespStatus >= 500 or BerespStatus >= 500? > > A sample 503 output is that one: > > * << Request >> 27632438 > - Begin req 27630762 rxreq > - Timestamp Start: 1426066874.412913 0.000000 0.000000 > - Timestamp Req: 1426066874.412913 0.000000 0.000000 > - ReqStart 10.47.28.251 49641 > - ReqMethod GET > - ReqURL /svc/somequery?string?documents=1 > - ReqProtocol HTTP/1.1 > - ReqHeader Accept-Language: de > - ReqHeader User-Agent: Java1.8.0 > - ReqHeader Host: essearch.local > - ReqHeader Connection: Keep-Alive > - ReqHeader Accept-Encoding: gzip > - VCL_call RECV > - VCL_return hash > - VCL_call HASH > - VCL_return lookup > - Debug "XXXX HIT-FOR-PASS" > - HitPass 2201620563 > - VCL_call PASS > - VCL_return fetch > - Link bereq 27632439 pass > - Timestamp Fetch: 1426066874.412967 0.000054 0.000054 > - Timestamp Process: 1426066874.412979 0.000066 0.000012 > - RespHeader Date: Wed, 11 Mar 2015 09:41:14 GMT > - RespHeader Server: Varnish > - RespHeader X-Varnish: 27632438 > - RespProtocol HTTP/1.1 > - RespStatus 503 > - RespReason Service Unavailable > - RespReason Service Unavailable > - VCL_call SYNTH > - RespHeader Content-Type: text/html; charset=utf-8 > - RespHeader Retry-After: 5 > - VCL_return deliver > - RespHeader Content-Length: 282 > - Debug "RES_MODE 2" > - RespHeader Connection: keep-alive > - Timestamp Resp: 1426066874.413033 0.000119 0.000053 > - ReqAcct 454 0 454 212 282 494 > - End > > > > What makes me wonder is, that this output is missing the actual backend > request attempts. Also, printing "RespReason Service Unavailable? twice in > a row?!? Why? > So in a second shell, on the same varnish at the same time, I have this > query running: > > varnishlog -b -n essearchvarnish -q "RespStatus >= 500 or BerespStatus >= > 500? > I added the -b parameter to see the backend requests. > However, I get no output at all. So it seems varnish would throw the 503 > without asking the backend? > > Or, hopefully more likely, I?m not using varnishlog correctly?! > > This is varnish 4.0.3-2~wheezy. How can I print client and backend > connections in one go? > Doing using the -c and -b parameter in parallel doesn?t produce any output > (while in my other console, I still have the first query which produces > the above output). > > I know that around 2000 req/s my backends are getting a bit slower ( up to > 2 seconds for responses), but this should not hit any internal timeouts of > varnish, right? I haven?t changed the default timeouts. > Also the backends itself are healthy. I?m watching the backends (several > tomcats) directly, and also from the varnish perspective via > watch -n1 varnishadm -n essearchvarnish backend.list > Everything looks healthy. > > > Any help for how I could dig into this further would be appreciated :-) > I guess it boils down to better use varnishlog on my side. Like getting > client and backend connections for a request at the same time and not by > using varnishlog in two shells differently. > > best regards, > Marian > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From raymond.jennings at nytimes.com Thu Mar 19 14:07:01 2015 From: raymond.jennings at nytimes.com (Jennings III, Raymond) Date: Thu, 19 Mar 2015 10:07:01 -0400 Subject: Help on converting this V3 line to V4 Message-ID: (1) I changed vcl_error to vcl_backend_error but having a problem with the internal syntax. I used to have: sub vcl_backend_error { synthetic {"

"} + obj.response + {"

"}; return(deliver); } (2) I also had this line in my V3 vcl error 200 "BAN Set for " + req.url; Does this get converted to: set resp.status = 200; synthetic("BAN Set for " + req.url); return(deliver); The Upgrading page says this: error() is now synth() And you must explicitly return it: return (synth(999, "Response")); Not sure exactly what is going on here. Thanks for any insight. Ray -------------- next part -------------- An HTML attachment was scrubbed... URL: From raymond.jennings at nytimes.com Thu Mar 19 21:08:17 2015 From: raymond.jennings at nytimes.com (Jennings III, Raymond) Date: Thu, 19 Mar 2015 17:08:17 -0400 Subject: Can I tell if a PURGE was a hit or miss in Version 4 Message-ID: I used to be able to tell this in version 3 because vcl_miss and vcl_hit were called on a PURGE request. In Version 4 they are not so is there any way to tell if the PURGE actually purged anything from within vcl_purge? -------------- next part -------------- An HTML attachment was scrubbed... URL: From apj at mutt.dk Fri Mar 20 09:00:05 2015 From: apj at mutt.dk (Andreas Plesner) Date: Fri, 20 Mar 2015 10:00:05 +0100 Subject: Can I tell if a PURGE was a hit or miss in Version 4 In-Reply-To: References: Message-ID: <20150320090005.GG2656@nerd.dk> On Thu, Mar 19, 2015 at 05:08:17PM -0400, Jennings III, Raymond wrote: > I used to be able to tell this in version 3 because vcl_miss and vcl_hit > were called on a PURGE request. No you didn't. A vcl_miss could evict other vary'ants on the same hash. -- Andreas From raymond.jennings at nytimes.com Fri Mar 20 12:09:50 2015 From: raymond.jennings at nytimes.com (Raymond Jennings III) Date: Fri, 20 Mar 2015 08:09:50 -0400 Subject: Can I tell if a PURGE was a hit or miss in Version 4 In-Reply-To: <20150320090005.GG2656@nerd.dk> References: <20150320090005.GG2656@nerd.dk> Message-ID: <-4070119162332887698@unknownmsgid> vcl_miss and vcl_hit are both called on a purge in v3 and they are not in v4 > On Mar 20, 2015, at 5:02 AM, Andreas Plesner wrote: > >> On Thu, Mar 19, 2015 at 05:08:17PM -0400, Jennings III, Raymond wrote: >> >> I used to be able to tell this in version 3 because vcl_miss and vcl_hit >> were called on a PURGE request. > > No you didn't. A vcl_miss could evict other vary'ants on the same hash. > > -- > Andreas > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc From apj at mutt.dk Fri Mar 20 13:34:05 2015 From: apj at mutt.dk (Andreas Plesner) Date: Fri, 20 Mar 2015 14:34:05 +0100 Subject: Can I tell if a PURGE was a hit or miss in Version 4 In-Reply-To: <-4070119162332887698@unknownmsgid> References: <20150320090005.GG2656@nerd.dk> <-4070119162332887698@unknownmsgid> Message-ID: <20150320133405.GH2656@nerd.dk> On Fri, Mar 20, 2015 at 08:09:50AM -0400, Raymond Jennings III wrote: > vcl_miss and vcl_hit are both called on a purge in v3 and they are not in v4 I know. vcl_miss does not mean that nothing was/will be evicted. If you have multiple vary'ants on the same hash, you can have a miss if you don't have the particular vary'ant, that is requested, but all other vary'ants will still be evicted by the purge. -- Andreas From raymond.jennings at nytimes.com Fri Mar 20 17:42:55 2015 From: raymond.jennings at nytimes.com (Jennings III, Raymond) Date: Fri, 20 Mar 2015 13:42:55 -0400 Subject: Can I tell if a PURGE was a hit or miss in Version 4 In-Reply-To: <20150320133405.GH2656@nerd.dk> References: <20150320090005.GG2656@nerd.dk> <-4070119162332887698@unknownmsgid> <20150320133405.GH2656@nerd.dk> Message-ID: Okay - regardless. I would like to know on a PURGE request if "anything" was evicted. Even a true / false would be better than nothing. Raymond Jennings III *nytimes.com * *Office: 212.556.7786 <212-556-7786>* *iPhone: 914.330.5074 <914-330-5074>E-mail: Raymond.Jennings at nytimes.com FaceTime: Raymond.Jennings at nytimes.com * On Fri, Mar 20, 2015 at 9:34 AM, Andreas Plesner wrote: > On Fri, Mar 20, 2015 at 08:09:50AM -0400, Raymond Jennings III wrote: > > > vcl_miss and vcl_hit are both called on a purge in v3 and they are not > in v4 > > I know. vcl_miss does not mean that nothing was/will be evicted. If you > have > multiple vary'ants on the same hash, you can have a miss if you don't have > the > particular vary'ant, that is requested, but all other vary'ants will still > be > evicted by the purge. > > -- > Andreas > -------------- next part -------------- An HTML attachment was scrubbed... URL: From viktor.villafuerte at optusnet.com.au Sun Mar 22 22:43:14 2015 From: viktor.villafuerte at optusnet.com.au (Viktor Villafuerte) Date: Mon, 23 Mar 2015 09:43:14 +1100 Subject: Help on converting this V3 line to V4 In-Reply-To: References: Message-ID: <20150322224314.GA11848@optusnet.com.au> Hi Raymond, On Thu 19 Mar 2015 10:07:01, Jennings III, Raymond wrote: > (1) > I changed vcl_error to vcl_backend_error but having a problem with the > internal syntax. I used to have: > > sub vcl_backend_error { > > synthetic {"

"} + obj.response + {"

"}; > return(deliver); > } I'm using this like this: (this is from the migrate v3 to v4 page) synthetic( {"

"} + resp.http.response + {"

"} ); return(deliver); > > (2) > I also had this line in my V3 vcl > > error 200 "BAN Set for " + req.url; > > Does this get converted to: > > set resp.status = 200; > synthetic("BAN Set for " + req.url); > return(deliver); > > > The Upgrading page says this: > > error() is now synth() > > > And you must explicitly return it: > > return (synth(999, "Response")); > You need to do exactly what the page says return(synth(ERR_NO, "Error message")); Then in vcl_synth you can do smth like if (resp.status == ERR_NO) { do_smth_here; } > > Not sure exactly what is going on here. > > Thanks for any insight. > > Ray > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc -- Regards Viktor Villafuerte Optus Internet Engineering t: 02 808-25265 From miguel_3_gonzalez at yahoo.es Sun Mar 22 23:21:05 2015 From: miguel_3_gonzalez at yahoo.es (=?UTF-8?B?TWlndWVsIEdvbnrDoWxleg==?=) Date: Mon, 23 Mar 2015 00:21:05 +0100 Subject: Troubleshooting a Wordpress website Message-ID: <550F4E61.4080908@yahoo.es> Dear all, I have a Centos 6.6 running Cpanel with Apache 2.4 with PHP 5.5 running a Wordpress website. The first time it loads the website it loads fine, you move around the website and varnishncsa.log get the entries and apache too and seems to be fine. When I try to go to the Home section of the website, it doesn?t load completely. If I hit Ctrl + F5 I get in the apache logs the following entries: 127.0.0.1 - - [22/Mar/2015:23:54:13 +0100] "GET /wp-content/uploads/js_composer/custom.css HTTP/1.1" 200 - "http://mydomain.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0" 127.0.0.1 - - [22/Mar/2015:23:54:13 +0100] "GET /wp-content/themes/hazel/css/style_dynamic_responsive.php HTTP/1.1" 200 171 "http://mydomain.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0" 127.0.0.1 - - [22/Mar/2015:23:54:13 +0100] "GET /wp-content/themes/hazel/css/custom_css.php HTTP/1.1" 200 2 "http://mydomain.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0" 127.0.0.1 - - [22/Mar/2015:23:54:13 +0100] "GET /wp-content/themes/hazel/css/style_dynamic.php HTTP/1.1" 200 4698 "http://mydomain.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0" 127.0.0.1 - - [22/Mar/2015:23:54:15 +0100] "GET /wp-content/themes/hazel/js/default_dynamic.php HTTP/1.1" 200 1530 "http://mydomain.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0" 127.0.0.1 - - [22/Mar/2015:23:54:15 +0100] "GET /wp-content/themes/hazel/js/custom_js.php HTTP/1.1" 200 102 "http://mydomain.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0" 127.0.0.1 - - [22/Mar/2015:23:54:16 +0100] "GET /wp-content/themes/hazel/js/ajax.min.js HTTP/1.1" 200 3841 "http://mydomain.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0" which I believe should have been forwarded to the backend in the first place. I attach my default.vcl file (I have changed the IP address of the backend) Regards, Miguel -------------- next part -------------- # # This is an example VCL file for Varnish. # # It does not do anything by default, delegating control to the # builtin VCL. The builtin VCL is called when there is no explicit # return statement. # # See the VCL chapters in the Users Guide at https://www.varnish-cache.org/docs/ # and http://varnish-cache.org/trac/wiki/VCLExamples for more examples. # Marker to tell the VCL compiler that this VCL has been adapted to the # new 4.0 format. vcl 4.0; import std; # Default backend definition. Set this to point to your content server. backend default { .host = "192.168.1.10"; .port = "82"; } acl purge { "localhost"; } # This function is used when a request is send by a HTTP client (Browser) sub vcl_recv { # remove ?ver=xxxxx strings from urls so css and js files are cached. # Watch out when upgrading WordPress, need to restart Varnish or flush cache. set req.url = regsub(req.url, "\?ver=.*$", ""); # Remove "replytocom" from requests to make caching better. set req.url = regsub(req.url, "\?replytocom=.*$", ""); # We pass real IP to the backend #if (req.restarts == 0) { # if (req.http.X-Forwarded-For) { # set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip; # } else { # set req.http.X-Forwarded-For = client.ip; # } #} # Normalize the header, remove the port (in case you're testing this on various TCP ports) set req.http.Host = regsub(req.http.Host, ":[0-9]+", ""); # Remove has_js and CloudFlare/Google Analytics __* cookies. set req.http.Cookie = regsuball(req.http.Cookie, "(^|;\s*)(_[_a-z]+|has_js)=[^;]*", ""); # Remove a ";" prefix, if present. set req.http.Cookie = regsub(req.http.Cookie, "^;\s*", ""); # Allow purging from ACL if (req.method == "PURGE") { # If not allowed then a error 405 is returned if (!client.ip ~ purge) { return(synth(405, "This IP is not allowed to send PURGE requests.")); } # If allowed, do a cache_lookup -> vlc_hit() or vlc_miss() return (purge); } # Post requests will not be cached #if (req.http.Authorization || req.method == "POST") { # return (pass); #} # Pass anything other than GET and HEAD directly. if (req.method != "GET" && req.method != "HEAD") { return( pass ); } /* We only deal with GET and HEAD by default */ # --- WordPress specific configuration # Did not cache the admin and login pages if (req.url ~ "wp-(login|admin)" || req.url ~ "preview=true") { return (pass); } if (req.url ~ "(ajax|dynamic|custom)") { return(pass); } # Remove the "has_js" cookie set req.http.Cookie = regsuball(req.http.Cookie, "has_js=[^;]+(; )?", ""); # Remove any Google Analytics based cookies set req.http.Cookie = regsuball(req.http.Cookie, "__utm.=[^;]+(; )?", ""); # Remove the Quant Capital cookies (added by some plugin, all __qca) set req.http.Cookie = regsuball(req.http.Cookie, "__qc.=[^;]+(; )?", ""); # Remove the wp-settings-1 cookie set req.http.Cookie = regsuball(req.http.Cookie, "wp-settings-1=[^;]+(; )?", ""); # Remove the wp-settings-time-1 cookie set req.http.Cookie = regsuball(req.http.Cookie, "wp-settings-time-1=[^;]+(; )?", ""); # Remove the wp test cookie set req.http.Cookie = regsuball(req.http.Cookie, "wordpress_test_cookie=[^;]+(; )?", ""); # Are there cookies left with only spaces or that are empty? if (req.http.cookie ~ "^ *$") { unset req.http.cookie; } # Cache the following files extensions if (req.url ~ "\.(css|js|png|gif|jp(e)?g|swf|ico)") { unset req.http.cookie; } # Normalize Accept-Encoding header and compression # https://www.varnish-cache.org/docs/3.0/tutorial/vary.html if (req.http.Accept-Encoding) { # Do no compress compressed files... if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") { unset req.http.Accept-Encoding; } elsif (req.http.Accept-Encoding ~ "gzip") { set req.http.Accept-Encoding = "gzip"; } elsif (req.http.Accept-Encoding ~ "deflate") { set req.http.Accept-Encoding = "deflate"; } else { unset req.http.Accept-Encoding; } } # Check the cookies for wordpress-specific items if (req.http.Cookie ~ "wordpress_" || req.http.Cookie ~ "comment_") { return (pass); } if (!req.http.cookie) { unset req.http.cookie; } # --- End of WordPress specific configuration # Did not cache HTTP authentication and HTTP Cookie if (req.http.Authorization || req.http.Cookie) { # Not cacheable by default return (pass); } # Cache all others requests return (hash); } sub vcl_pipe { return (pipe); } sub vcl_pass { return (fetch); } # The data on which the hashing will take place sub vcl_hash { hash_data(req.url); if (req.http.host) { hash_data(req.http.host); } else { hash_data(server.ip); } # If the client supports compression, keep that in a different cache if (req.http.Accept-Encoding) { hash_data(req.http.Accept-Encoding); } return (lookup); } # This function is used when a request is sent by our backend (Nginx server) sub vcl_backend_response { # Remove some headers we never want to see unset beresp.http.Server; unset beresp.http.X-Powered-By; # For static content strip all backend cookies if (bereq.url ~ "\.(css|js|png|gif|jp(e?)g)|swf|ico") { unset beresp.http.cookie; } # Don't store backend if (bereq.url ~ "wp-(login|admin)" || bereq.url ~ "preview=true") { set beresp.uncacheable = true; set beresp.ttl = 30s; return (deliver); } # Only allow cookies to be set if we're in admin area if (!(bereq.url ~ "(wp-login|wp-admin|preview=true)")) { unset beresp.http.set-cookie; } # don't cache response to posted requests or those with basic auth if ( bereq.method == "POST" || bereq.http.Authorization ) { set beresp.uncacheable = true; set beresp.ttl = 120s; return (deliver); } # don't cache search results if ( bereq.url ~ "\?s=" ){ set beresp.uncacheable = true; set beresp.ttl = 120s; return (deliver); } # only cache status ok if ( beresp.status != 200 ) { set beresp.uncacheable = true; set beresp.ttl = 120s; return (deliver); } # A TTL of 2h set beresp.ttl = 2h; # Define the default grace period to serve cached content set beresp.grace = 30s; return (deliver); } # The routine when we deliver the HTTP request to the user # Last chance to modify headers that are sent to the client sub vcl_deliver { if (obj.hits > 0) { set resp.http.X-Cache = "cached"; } else { set resp.http.x-Cache = "uncached"; } # Remove some headers: PHP version unset resp.http.X-Powered-By; # Remove some headers: Apache version & OS unset resp.http.Server; # Remove some heanders: Varnish unset resp.http.Via; unset resp.http.X-Varnish; return (deliver); } sub vcl_init { return (ok); } sub vcl_fini { return (ok); } From apj at mutt.dk Mon Mar 23 07:22:13 2015 From: apj at mutt.dk (Andreas Plesner) Date: Mon, 23 Mar 2015 08:22:13 +0100 Subject: Can I tell if a PURGE was a hit or miss in Version 4 In-Reply-To: References: <20150320090005.GG2656@nerd.dk> <-4070119162332887698@unknownmsgid> <20150320133405.GH2656@nerd.dk> Message-ID: <20150323072213.GI2656@nerd.dk> On Fri, Mar 20, 2015 at 01:42:55PM -0400, Jennings III, Raymond wrote: > Okay - regardless. I would like to know on a PURGE request if "anything" > was evicted. Even a true / false would be better than nothing. I don't believe you can. I don't believe you ever could. You may be able to see something in the shmlog, but I'm not sure if you can connect those to the purge request. -- Andreas From raymond.jennings at nytimes.com Mon Mar 23 13:27:12 2015 From: raymond.jennings at nytimes.com (Jennings III, Raymond) Date: Mon, 23 Mar 2015 09:27:12 -0400 Subject: Can I tell if a PURGE was a hit or miss in Version 4 In-Reply-To: <20150323072213.GI2656@nerd.dk> References: <20150320090005.GG2656@nerd.dk> <-4070119162332887698@unknownmsgid> <20150320133405.GH2656@nerd.dk> <20150323072213.GI2656@nerd.dk> Message-ID: This is what I basically have been using for years with V3 and it is serving me exactly as I need and I would like to see if there is anything equivalent to this V4. In does not seem to be the case that vcl_hit or vcl_miss get called on a PURGE req in V4 - as they were in V3. sub vcl_hit { if (req.request == "PURGE") { C{ VRT_purge(sp, 0, 0); VRT_error(sp, 200, "Purged - Hit"); VRT_done(sp, VCL_RET_ERROR); }C } } sub vcl_miss { if (req.request == "PURGE") { C{ VRT_purge(sp, 0, 0); VRT_error(sp, 200, "Purged - Miss"); VRT_done(sp, VCL_RET_ERROR); }C } Raymond Jennings III *nytimes.com * *Office: 212.556.7786 <212-556-7786>* *iPhone: 914.330.5074 <914-330-5074>E-mail: Raymond.Jennings at nytimes.com FaceTime: Raymond.Jennings at nytimes.com * On Mon, Mar 23, 2015 at 3:22 AM, Andreas Plesner wrote: > On Fri, Mar 20, 2015 at 01:42:55PM -0400, Jennings III, Raymond wrote: > > > Okay - regardless. I would like to know on a PURGE request if "anything" > > was evicted. Even a true / false would be better than nothing. > > I don't believe you can. I don't believe you ever could. You may be able > to see > something in the shmlog, but I'm not sure if you can connect those to the > purge > request. > > -- > Andreas > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nick.tailor at gmail.com Mon Mar 23 15:42:11 2015 From: nick.tailor at gmail.com (nick tailor) Date: Mon, 23 Mar 2015 08:42:11 -0700 Subject: Troubleshooting a Wordpress website In-Reply-To: <550F4E61.4080908@yahoo.es> References: <550F4E61.4080908@yahoo.es> Message-ID: Hi Miguel, I'm just guessing based on the filename "style_dynamic_responsive.php" I don't think you can cache dynamic content ? Cheers Nick Tailor nicktailor.com On Sun, Mar 22, 2015 at 4:21 PM, Miguel Gonz?lez wrote: > Dear all, > > I have a Centos 6.6 running Cpanel with Apache 2.4 with PHP 5.5 > running a Wordpress website. > > The first time it loads the website it loads fine, you move around the > website and varnishncsa.log get the entries and apache too and seems to be > fine. > > When I try to go to the Home section of the website, it doesn?t load > completely. If I hit Ctrl + F5 I get in the apache logs the following > entries: > > 127.0.0.1 - - [22/Mar/2015:23:54:13 +0100] "GET /wp-content/uploads/js_composer/custom.css > HTTP/1.1" 200 - "http://mydomain.com/" "Mozilla/5.0 (Windows NT 6.1; > WOW64; rv:35.0) Gecko/20100101 Firefox/35.0" > 127.0.0.1 - - [22/Mar/2015:23:54:13 +0100] "GET > /wp-content/themes/hazel/css/style_dynamic_responsive.php HTTP/1.1" 200 > 171 "http://mydomain.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) > Gecko/20100101 Firefox/35.0" > 127.0.0.1 - - [22/Mar/2015:23:54:13 +0100] "GET > /wp-content/themes/hazel/css/custom_css.php HTTP/1.1" 200 2 " > http://mydomain.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) > Gecko/20100101 Firefox/35.0" > 127.0.0.1 - - [22/Mar/2015:23:54:13 +0100] "GET > /wp-content/themes/hazel/css/style_dynamic.php HTTP/1.1" 200 4698 " > http://mydomain.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) > Gecko/20100101 Firefox/35.0" > 127.0.0.1 - - [22/Mar/2015:23:54:15 +0100] "GET > /wp-content/themes/hazel/js/default_dynamic.php HTTP/1.1" 200 1530 " > http://mydomain.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) > Gecko/20100101 Firefox/35.0" > 127.0.0.1 - - [22/Mar/2015:23:54:15 +0100] "GET > /wp-content/themes/hazel/js/custom_js.php HTTP/1.1" 200 102 " > http://mydomain.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) > Gecko/20100101 Firefox/35.0" > 127.0.0.1 - - [22/Mar/2015:23:54:16 +0100] "GET > /wp-content/themes/hazel/js/ajax.min.js HTTP/1.1" 200 3841 " > http://mydomain.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) > Gecko/20100101 Firefox/35.0" > > which I believe should have been forwarded to the backend in the first > place. > > I attach my default.vcl file (I have changed the IP address of the > backend) > > Regards, > > Miguel > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -- Nick Tailor Senior Linux Systems Administrator BCLC, 2940 Virtual Way, Vancouver B.C. V5M 0A6 T 604 228 3046 C 778 388 1397 Connect with us: Twitter @BCLC | Twitter @BCLCGameSense | YouTube | Blog | bclc.com Last year, more than $1 billion generated by BCLC gambling activities went back into health care, education and community groups across B.C. -------------- next part -------------- An HTML attachment was scrubbed... URL: From miguel_3_gonzalez at yahoo.es Mon Mar 23 15:45:12 2015 From: miguel_3_gonzalez at yahoo.es (Miguel Gonzalez) Date: Mon, 23 Mar 2015 15:45:12 +0000 (UTC) Subject: Troubleshooting a Wordpress website In-Reply-To: References: Message-ID: <1368735167.1475528.1427125512782.JavaMail.yahoo@mail.yahoo.com> Dear Nick, ? Thanks for answering. I'm not trying to cache dynamic content. I'm trying to accomplish that Varnish sends to the backend those requests but It seems I'm not succeeding. ? How can I troubleshoot this? ? Many thanks, ? Miguel De: nick tailor Para: Miguel Gonz?lez CC: "varnish-misc at varnish-cache.org" Enviado: Lunes 23 de marzo de 2015 16:42 Asunto: Re: Troubleshooting a Wordpress website Hi Miguel, I'm just guessing based on the filename "style_dynamic_responsive.php"? I don't think you can cache dynamic content ? Cheers Nick Tailornicktailor.com On Sun, Mar 22, 2015 at 4:21 PM, Miguel Gonz?lez wrote: Dear all, ? ? I have a Centos 6.6 running Cpanel with Apache 2.4 with PHP 5.5 running a Wordpress website. ? ? The first time it loads the website it loads fine, you move around the website and varnishncsa.log get the entries and apache too and seems to be fine. ? ? When I try to go to the Home section of the website, it doesn?t load completely. If I hit? Ctrl + F5 I get in the apache logs the following entries: ? ? 127.0.0.1 - - [22/Mar/2015:23:54:13 +0100] "GET /wp-content/uploads/js_composer/custom.css HTTP/1.1" 200 - "http://mydomain.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0" 127.0.0.1 - - [22/Mar/2015:23:54:13 +0100] "GET /wp-content/themes/hazel/css/style_dynamic_responsive.php HTTP/1.1" 200 171 "http://mydomain.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0" 127.0.0.1 - - [22/Mar/2015:23:54:13 +0100] "GET /wp-content/themes/hazel/css/custom_css.php HTTP/1.1" 200 2 "http://mydomain.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0" 127.0.0.1 - - [22/Mar/2015:23:54:13 +0100] "GET /wp-content/themes/hazel/css/style_dynamic.php HTTP/1.1" 200 4698 "http://mydomain.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0" 127.0.0.1 - - [22/Mar/2015:23:54:15 +0100] "GET /wp-content/themes/hazel/js/default_dynamic.php HTTP/1.1" 200 1530 "http://mydomain.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0" 127.0.0.1 - - [22/Mar/2015:23:54:15 +0100] "GET /wp-content/themes/hazel/js/custom_js.php HTTP/1.1" 200 102 "http://mydomain.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0" 127.0.0.1 - - [22/Mar/2015:23:54:16 +0100] "GET /wp-content/themes/hazel/js/ajax.min.js HTTP/1.1" 200 3841 "http://mydomain.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0" ? ?which I believe should have been forwarded to the backend in the first place. ? ?I attach my default.vcl file (I have changed the IP address of the backend) ? ?Regards, ? ?Miguel _______________________________________________ varnish-misc mailing list varnish-misc at varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc -- Nick Tailor Senior Linux Systems Administrator BCLC, 2940 Virtual Way, Vancouver B.C. V5M 0A6 T?604?228 3046??C?778 388 1397 Connect with us: Twitter @BCLC|?Twitter @BCLCGameSense?|?YouTube?|?Blog?|?bclc.com?Last year, more than $1 billion generated by BCLC gambling activities went back into health care, education and community groups across B.C. -------------- next part -------------- An HTML attachment was scrubbed... URL: From raymond.jennings at nytimes.com Tue Mar 24 15:47:01 2015 From: raymond.jennings at nytimes.com (Jennings III, Raymond) Date: Tue, 24 Mar 2015 11:47:01 -0400 Subject: Do I have access to the URL in vcl_miss in version 4 Message-ID: I used to in version 3 but not sure about version 4 and how to reference it in the vcl. Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From raymond.jennings at nytimes.com Wed Mar 25 19:39:19 2015 From: raymond.jennings at nytimes.com (Jennings III, Raymond) Date: Wed, 25 Mar 2015 15:39:19 -0400 Subject: What happened to VRT_panic() in version 4 Message-ID: This C function was present in version 3 - is there anything like it in V4 ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlos.abalde at gmail.com Wed Mar 25 19:53:40 2015 From: carlos.abalde at gmail.com (Carlos Abalde) Date: Wed, 25 Mar 2015 20:53:40 +0100 Subject: What happened to VRT_panic() in version 4 In-Reply-To: References: Message-ID: <4A9BDD28-BFCB-4152-8272-AEF2F567CC2C@gmail.com> > On 25 Mar 2015, at 20:39, Jennings III, Raymond wrote: > > This C function was present in version 3 - is there anything like it in V4 ? Hi, I think that in V4 that implementation has been moved to the debug VMOD: debug.panic(...) [1] [1] https://www.varnish-cache.org/lists/pipermail/varnish-commit/2012-June/008975.html Cheers, -- Carlos Abalde From phk at phk.freebsd.dk Wed Mar 25 19:58:20 2015 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Wed, 25 Mar 2015 19:58:20 +0000 Subject: What happened to VRT_panic() in version 4 In-Reply-To: References: Message-ID: <53355.1427313500@critter.freebsd.dk> -------- In message , "Jennings II I, Raymond" writes: >This C function was present in version 3 - is there anything like it in V4 ? It was moved to VMOD_debug You can always cause a panic by doing an assert if you want... -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From viktor.villafuerte at optusnet.com.au Wed Mar 25 22:42:34 2015 From: viktor.villafuerte at optusnet.com.au (Viktor Villafuerte) Date: Thu, 26 Mar 2015 09:42:34 +1100 Subject: Help on converting this V3 line to V4 In-Reply-To: <20150322224314.GA11848@optusnet.com.au> References: <20150322224314.GA11848@optusnet.com.au> Message-ID: <20150325224234.GA9588@optusnet.com.au> Hi Raymond, > I get the following error on your first part: > > Message from VCC-compiler: > 'resp.http.response': Not available in method 'vcl_backend_error'. > At: ('input' Line 419 Pos 23) > synthetic( {"

"} + resp.http.response + {"

"} ); > ----------------------##################-------------- vcl_error is now vcl_synth that's where you can use resp.http.response. in your case it would be smth more like beresp.http.response Please see https://www.varnish-cache.org/docs/trunk/whats-new/upgrading.html for more details v On Mon 23 Mar 2015 09:43:14, Viktor Villafuerte wrote: > Hi Raymond, > > > On Thu 19 Mar 2015 10:07:01, Jennings III, Raymond wrote: > > (1) > > I changed vcl_error to vcl_backend_error but having a problem with the > > internal syntax. I used to have: > > > > sub vcl_backend_error { > > > > synthetic {"

"} + obj.response + {"

"}; > > return(deliver); > > } > > I'm using this like this: > (this is from the migrate v3 to v4 page) > > synthetic( {"

"} + resp.http.response + {"

"} ); > return(deliver); > > > > > > (2) > > I also had this line in my V3 vcl > > > > error 200 "BAN Set for " + req.url; > > > > Does this get converted to: > > > > set resp.status = 200; > > synthetic("BAN Set for " + req.url); > > return(deliver); > > > > > > The Upgrading page says this: > > > > error() is now synth() > > > > > > And you must explicitly return it: > > > > return (synth(999, "Response")); > > > > You need to do exactly what the page says > > return(synth(ERR_NO, "Error message")); > > Then in vcl_synth you can do smth like > > if (resp.status == ERR_NO) { > do_smth_here; > } > > > > > > Not sure exactly what is going on here. > > > > Thanks for any insight. > > > > Ray > > > _______________________________________________ > > varnish-misc mailing list > > varnish-misc at varnish-cache.org > > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > > -- > Regards > > Viktor Villafuerte > Optus Internet Engineering > t: 02 808-25265 > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc -- Regards Viktor Villafuerte Optus Internet Engineering t: 02 808-25265 From raymond.jennings at nytimes.com Thu Mar 26 13:51:27 2015 From: raymond.jennings at nytimes.com (Jennings III, Raymond) Date: Thu, 26 Mar 2015 09:51:27 -0400 Subject: What is accessible in vcl_purge Message-ID: Can I get access to anything - even the uri that is being purged or anything with the object being purged? -------------- next part -------------- An HTML attachment was scrubbed... URL: From raymond.jennings at nytimes.com Fri Mar 27 14:02:22 2015 From: raymond.jennings at nytimes.com (Jennings III, Raymond) Date: Fri, 27 Mar 2015 10:02:22 -0400 Subject: vcl_purge Message-ID: Can I get access to anything - that is being purged, the obj, if the purge was successful, anything at all? It sucks that vcl_miss and vcl_hit are off the request path (in the vcl anyway.) -------------- next part -------------- An HTML attachment was scrubbed... URL: From perbu at varnish-software.com Fri Mar 27 14:18:53 2015 From: perbu at varnish-software.com (Per Buer) Date: Fri, 27 Mar 2015 15:18:53 +0100 Subject: vcl_purge In-Reply-To: References: Message-ID: Hi Raymond. On Fri, Mar 27, 2015 at 3:02 PM, Jennings III, Raymond < raymond.jennings at nytimes.com> wrote: > Can I get access to anything - that is being purged, the obj, if the purge > was successful, anything at all? > > It sucks that vcl_miss and vcl_hit are off the request path (in the vcl > anyway.) > Nobody has ever expressed interest in this before. And Varnish 3 really didn't support it properly (due to Vary:-handling) so it isn't something we've ever supported properly. What is the use case? -- *Per Buer* CTO | Varnish Software AS Cell: +47 95839117 We Make Websites Fly! www.varnish-software.com [image: Register now] -------------- next part -------------- An HTML attachment was scrubbed... URL: From raymond.jennings at nytimes.com Fri Mar 27 14:27:32 2015 From: raymond.jennings at nytimes.com (Jennings III, Raymond) Date: Fri, 27 Mar 2015 10:27:32 -0400 Subject: vcl_purge In-Reply-To: References: Message-ID: When things get purged vcl_miss and vcl_hit were handling it just fine for me. I really don't care about the Vary issue. I just wanted to know if the hash on the uri for a purge - was their a match or not. "Nobody" - seems odd - I use it extensively. Raymond Jennings III *nytimes.com * *Office: 212.556.7786 <212-556-7786>* *iPhone: 914.330.5074 <914-330-5074>E-mail: Raymond.Jennings at nytimes.com FaceTime: Raymond.Jennings at nytimes.com * On Fri, Mar 27, 2015 at 10:18 AM, Per Buer wrote: > Hi Raymond. > > On Fri, Mar 27, 2015 at 3:02 PM, Jennings III, Raymond < > raymond.jennings at nytimes.com> wrote: > >> Can I get access to anything - that is being purged, the obj, if the >> purge was successful, anything at all? >> >> It sucks that vcl_miss and vcl_hit are off the request path (in the vcl >> anyway.) >> > > Nobody has ever expressed interest in this before. And Varnish 3 really > didn't support it properly (due to Vary:-handling) so it isn't something > we've ever supported properly. > > What is the use case? > > -- > *Per Buer* > CTO | Varnish Software AS > Cell: +47 95839117 > We Make Websites Fly! > www.varnish-software.com > [image: Register now] > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From perbu at varnish-software.com Fri Mar 27 14:43:55 2015 From: perbu at varnish-software.com (Per Buer) Date: Fri, 27 Mar 2015 15:43:55 +0100 Subject: vcl_purge In-Reply-To: References: Message-ID: On Fri, Mar 27, 2015 at 3:27 PM, Jennings III, Raymond < raymond.jennings at nytimes.com> wrote: > When things get purged vcl_miss and vcl_hit were handling it just fine for > me. I really don't care about the Vary issue. I just wanted to know if > the hash on the uri for a purge - was their a match or not. > Well. Knowing if there is a hash match rather than knowing if the object was purge doesn't sound very useful IMO. "Nobody" - seems odd - I use it extensively. > True. I've never heard anyone ask about this. Since Varnish is a cache people don't usually care about what potentially was there before a PURGE call. The only thing they care about is whether the purge succeeded or not. What do you use it for? Why do you care if the purge call purged something? > On Fri, Mar 27, 2015 at 10:18 AM, Per Buer > wrote: > >> Hi Raymond. >> >> On Fri, Mar 27, 2015 at 3:02 PM, Jennings III, Raymond < >> raymond.jennings at nytimes.com> wrote: >> >>> Can I get access to anything - that is being purged, the obj, if the >>> purge was successful, anything at all? >>> >>> It sucks that vcl_miss and vcl_hit are off the request path (in the vcl >>> anyway.) >>> >> >> Nobody has ever expressed interest in this before. And Varnish 3 really >> didn't support it properly (due to Vary:-handling) so it isn't something >> we've ever supported properly. >> >> What is the use case? >> >> -- >> *Per Buer* >> CTO | Varnish Software AS >> Cell: +47 95839117 >> We Make Websites Fly! >> www.varnish-software.com >> [image: Register now] >> >> > > -- *Per Buer* CTO | Varnish Software AS Cell: +47 95839117 We Make Websites Fly! www.varnish-software.com [image: Register now] -------------- next part -------------- An HTML attachment was scrubbed... URL: From toasty at dragondata.com Mon Mar 30 17:57:54 2015 From: toasty at dragondata.com (Kevin Day) Date: Mon, 30 Mar 2015 12:57:54 -0500 Subject: Large number of (dynamic?) backends Message-ID: I have a use case where I need to be able to take an incoming request like ?/XXX/filename? and translate that to fetching ?http://nodeXXX.example.com/filename? where there?d literally be hundreds of backend servers. i.e. XXX would be a varying 3 digit number. I saw this post here: https://www.varnish-software.com/blog/truly-dynamic-backend-definition-varnish From 2012, but I know a lot has changed since then. Is there an easy way this can be accomplished now? ? Kevin From contact at jpluscplusm.com Mon Mar 30 18:46:23 2015 From: contact at jpluscplusm.com (Jonathan Matthews) Date: Mon, 30 Mar 2015 19:46:23 +0100 Subject: Large number of (dynamic?) backends In-Reply-To: References: Message-ID: On 30 Mar 2015 18:59, "Kevin Day" wrote: > > I have a use case where I need to be able to take an incoming request like ?/XXX/filename? and translate that to fetching ? http://nodeXXX.example.com/filename? where there?d literally be hundreds of backend servers. i.e. XXX would be a varying 3 digit number. > > I saw this post here: > > https://www.varnish-software.com/blog/truly-dynamic-backend-definition-varnish > > From 2012, but I know a lot has changed since then. Is there an easy way this can be accomplished now? Modulo a significant advance in varnish dynamic backends I've missed, I'd do this with Varnish talking to one or more local or remote Nginx instances - it's trivial to implement that dynamic routing there. I wouldn't move to nginx alone however - varnish definitely does a better job of caching. IMHO & IME :-) Jonathan -------------- next part -------------- An HTML attachment was scrubbed... URL: From perbu at varnish-software.com Mon Mar 30 21:50:15 2015 From: perbu at varnish-software.com (Per Buer) Date: Mon, 30 Mar 2015 23:50:15 +0200 Subject: Large number of (dynamic?) backends In-Reply-To: References: Message-ID: On Mon, Mar 30, 2015 at 7:57 PM, Kevin Day wrote: > I have a use case where I need to be able to take an incoming request like > ?/XXX/filename? and translate that to fetching ? > http://nodeXXX.example.com/filename? where there?d literally be hundreds > of backend servers. i.e. XXX would be a varying 3 digit number. > > I saw this post here: > > > https://www.varnish-software.com/blog/truly-dynamic-backend-definition-varnish > > From 2012, but I know a lot has changed since then. Is there an easy way > this can be accomplished now? > I talked to PHK about this last thursday. There is code in the master branch to do this now. However, the VCL bit and a few minor things are still needed. This will be in the 4.1 release. -- *Per Buer* CTO | Varnish Software AS Cell: +47 95839117 We Make Websites Fly! www.varnish-software.com [image: Register now] -------------- next part -------------- An HTML attachment was scrubbed... URL: From bedis9 at gmail.com Mon Mar 30 22:12:27 2015 From: bedis9 at gmail.com (Baptiste) Date: Tue, 31 Mar 2015 00:12:27 +0200 Subject: Large number of (dynamic?) backends In-Reply-To: References: Message-ID: On Mon, Mar 30, 2015 at 8:46 PM, Jonathan Matthews wrote: > On 30 Mar 2015 18:59, "Kevin Day" wrote: >> >> I have a use case where I need to be able to take an incoming request like >> "/XXX/filename" and translate that to fetching >> "http://nodeXXX.example.com/filename" where there'd literally be hundreds of >> backend servers. i.e. XXX would be a varying 3 digit number. >> >> I saw this post here: >> >> >> https://www.varnish-software.com/blog/truly-dynamic-backend-definition-varnish >> >> From 2012, but I know a lot has changed since then. Is there an easy way >> this can be accomplished now? > > Modulo a significant advance in varnish dynamic backends I've missed, I'd do > this with Varnish talking to one or more local or remote Nginx instances - > it's trivial to implement that dynamic routing there. I wouldn't move to > nginx alone however - varnish definitely does a better job of caching. IMHO > & IME :-) Hi, Forget about nginx, you could easily do this with HAProxy. It requires a bit of rewriting then either a map or use-server rule with the freshly created Host header :) And it is processed at light speed without any latency. Baptiste From japrice at gmail.com Tue Mar 31 02:40:51 2015 From: japrice at gmail.com (Jason Price) Date: Mon, 30 Mar 2015 22:40:51 -0400 Subject: Large number of (dynamic?) backends In-Reply-To: References: Message-ID: I've written a custom program to generate backends based off DNS records (It's deeply custom to our use case, so sharing it probably won't help you too much). In short it is called by cron every minute. It wakes up, gathers all the IPs of all the various backends, and builds the backend definition for them, and wraps each backend up in a director [1]. If the resulting file is different from the current one, swap the file out, and replace it with the new one, then call 'service varnish reload'. Every hour another cron job wakes up and runs 'varnishadm vcl.discard '. This process works fairly well at high load, and doesn't flush the cache. It also relies on probe definitions requiring that new IPs pass one health check (at least) before traffic is sent to them. -Jason [1] Alternatively, in AWS it could wake up, scan the instances for various tag attributes and rely on those as backends. I've used that formula with success as well. On Mon, Mar 30, 2015 at 1:57 PM, Kevin Day wrote: > I have a use case where I need to be able to take an incoming request like > ?/XXX/filename? and translate that to fetching ? > http://nodeXXX.example.com/filename? where there?d literally be hundreds > of backend servers. i.e. XXX would be a varying 3 digit number. > > I saw this post here: > > > https://www.varnish-software.com/blog/truly-dynamic-backend-definition-varnish > > From 2012, but I know a lot has changed since then. Is there an easy way > this can be accomplished now? > > ? Kevin > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc -------------- next part -------------- An HTML attachment was scrubbed... URL: From dridi at varni.sh Tue Mar 31 10:20:48 2015 From: dridi at varni.sh (Dridi Boukelmoune) Date: Tue, 31 Mar 2015 12:20:48 +0200 Subject: Large number of (dynamic?) backends In-Reply-To: References: Message-ID: On Mon, Mar 30, 2015 at 11:50 PM, Per Buer wrote: > > I talked to PHK about this last thursday. There is code in the master branch to do this now. However, the VCL bit and a few minor things are still needed. This will be in the 4.1 release. The code is not enough for "dns director"-like dynamic backends, it would only work at VCL load/init time. Or maybe I'm wrong, but I didn't get an answer yet: https://www.varnish-cache.org/lists/pipermail/varnish-dev/2015-March/008259.html Cheers, Dridi