From david.theis at wetteronline.de Fri Jul 1 12:05:33 2011 From: david.theis at wetteronline.de (David Theis) Date: Fri, 01 Jul 2011 14:05:33 +0200 Subject: Question concerning rewriting browser address bar Message-ID: <4E0DB80D.5090209@wetteronline.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi everybody, i'm getting right to my question: if a user contacts one of our proxy servers using only the ip address i'd like to rewrite this ip address into a domain name, so that the user sees www.domainname.de instead of e.g 10.0.0.2 in his/her browser address bar. is this possible with varnishd ? We're using freebsd-7.4-RELEASE + varnish 2.1.5 ! mfg David -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk4NuA0ACgkQu/Z5Qo6690XlYACgj2or4DFsYSxMiAUPoTRjr9y0 xWQAoJSW112qam6BUixSZjwEibsLMLY0 =hw0m -----END PGP SIGNATURE----- From shib4u at gmail.com Fri Jul 1 12:12:42 2011 From: shib4u at gmail.com (Shibashish) Date: Fri, 1 Jul 2011 17:42:42 +0530 Subject: Question concerning rewriting browser address bar In-Reply-To: <4E0DB80D.5090209@wetteronline.de> References: <4E0DB80D.5090209@wetteronline.de> Message-ID: On Fri, Jul 1, 2011 at 5:35 PM, David Theis wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi everybody, > > i'm getting right to my question: > if a user contacts one of our proxy servers using only the ip address > i'd like to rewrite this ip address into a domain name, so that the user > sees www.domainname.de instead of e.g 10.0.0.2 in his/her browser > address bar. is this possible with varnishd ? > > We're using freebsd-7.4-RELEASE + varnish 2.1.5 ! > > mfg > David > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2.0.17 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iEYEARECAAYFAk4NuA0ACgkQu/Z5Qo6690XlYACgj2or4DFsYSxMiAUPoTRjr9y0 > xWQAoJSW112qam6BUixSZjwEibsLMLY0 > =hw0m > -----END PGP SIGNATURE----- > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > in vcl_recv if (req.http.host ~ "11.22.33.44"){ error 750 "http://www.domain.com"; } sub vcl_error { if (obj.status == 750) { set obj.http.Location = obj.response req.url; set obj.status = 301; return(deliver); } } -- Shib -------------- next part -------------- An HTML attachment was scrubbed... URL: From jonathan.hursey at adrevolution.com Fri Jul 1 13:43:13 2011 From: jonathan.hursey at adrevolution.com (Jonathan Hursey) Date: Fri, 1 Jul 2011 08:43:13 -0500 Subject: Question concerning rewriting browser address bar In-Reply-To: <4E0DB80D.5090209@wetteronline.de> References: <4E0DB80D.5090209@wetteronline.de> Message-ID: if you're just doing it for one: set req.http.host = regsub(req.http.host, "10.10.0.2", "www.domname.de"); if you want a universal catch all on that I'm not too sure. Sounds like you'd need to resolve the name. or mod_modrewrite on the backend. On Fri, Jul 1, 2011 at 7:05 AM, David Theis wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi everybody, > > i'm getting right to my question: > if a user contacts one of our proxy servers using only the ip address > i'd like to rewrite this ip address into a domain name, so that the user > sees www.domainname.de instead of e.g 10.0.0.2 in his/her browser > address bar. is this possible with varnishd ? > > We're using freebsd-7.4-RELEASE + varnish 2.1.5 ! > > mfg > David > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2.0.17 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iEYEARECAAYFAk4NuA0ACgkQu/Z5Qo6690XlYACgj2or4DFsYSxMiAUPoTRjr9y0 > xWQAoJSW112qam6BUixSZjwEibsLMLY0 > =hw0m > -----END PGP SIGNATURE----- > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -- *Jonathan M. Hursey* *Linux Systems Administrator* * * -------------- next part -------------- An HTML attachment was scrubbed... URL: From c.lukas at external.telekom.de Fri Jul 1 15:49:45 2011 From: c.lukas at external.telekom.de (Lukas, Christian (extern)) Date: Fri, 1 Jul 2011 17:49:45 +0200 Subject: round-robin director Message-ID: <94B46DAE54D73F4BB2E5D300892C13DD56043A23AA@QEO40072.de.t-online.corp> Hi everyone, I am new to the mailing list. If I do something wrong please let me know and I will change it in the future. We use the round-robin director and like to have one backend to be requested only every eight times. E.g. two backends (backend1, backend2). Backend1 should handle the first 7 request and backend2 one request ... backend1 handle 7 requests ... backend2 handle one request. Is it possible to do this configuration. Any help would be appreciated. Thanks, Christian Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto.fernandezcrisial at gmail.com Fri Jul 1 15:57:13 2011 From: roberto.fernandezcrisial at gmail.com (=?ISO-8859-1?Q?Roberto_O=2E_Fern=E1ndez_Crisial?=) Date: Fri, 1 Jul 2011 12:57:13 -0300 Subject: round-robin director In-Reply-To: <94B46DAE54D73F4BB2E5D300892C13DD56043A23AA@QEO40072.de.t-online.corp> References: <94B46DAE54D73F4BB2E5D300892C13DD56043A23AA@QEO40072.de.t-online.corp> Message-ID: Christian, I think you should use random director, so you can use weigth option and send traffic to a particular backend. Regards, Roberto O. Fern?ndez Crisial @rofc On Fri, Jul 1, 2011 at 12:49 PM, Lukas, Christian (extern) < c.lukas at external.telekom.de> wrote: > Hi everyone,**** > > ** ** > > I am new to the mailing list. If I do something wrong please let me know > and I will change it in the future.**** > > ** ** > > We use the round-robin director and like to have one backend to be > requested only every eight times.**** > > E.g. two backends (backend1, backend2). Backend1 should handle the first 7 > request and backend2 one request ? backend1 handle 7 requests ? backend2 > handle one request.**** > > ** ** > > Is it possible to do this configuration. Any help would be appreciated.*** > * > > ** ** > > Thanks,**** > > Christian Lukas**** > > ** ** > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lee at leetrout.com Fri Jul 1 16:02:57 2011 From: lee at leetrout.com (Lee Trout) Date: Fri, 1 Jul 2011 12:02:57 -0400 Subject: round-robin director In-Reply-To: References: <94B46DAE54D73F4BB2E5D300892C13DD56043A23AA@QEO40072.de.t-online.corp> Message-ID: Here's a quick example that should get you in the ballpark: http://pastebin.com/EUGWMthD 2011/7/1 Roberto O. Fern?ndez Crisial : > Christian, > I think you should use random director, so you can use weigth option and > send traffic to a particular backend. > Regards, > Roberto O. Fern?ndez Crisial > @rofc > > On Fri, Jul 1, 2011 at 12:49 PM, Lukas, Christian (extern) > wrote: >> >> Hi everyone, >> >> >> >> I am new to the mailing list. If I do something wrong please let me know >> and I will change it in the future. >> >> >> >> We use the round-robin director and like to have one backend to be >> requested only every eight times. >> >> E.g. two backends (backend1, backend2). Backend1 should handle the first 7 >> request and backend2 one request ? backend1 handle 7 requests ? backend2 >> handle one request. >> >> >> >> Is it possible to do this configuration. Any help would be appreciated. >> >> >> >> Thanks, >> >> ? Christian Lukas >> >> >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -- Lee Trout 606.483.1337 lee at leetrout.com http://www.leetrout.com From jonathan.hursey at adrevolution.com Fri Jul 1 19:32:33 2011 From: jonathan.hursey at adrevolution.com (Jonathan Hursey) Date: Fri, 1 Jul 2011 14:32:33 -0500 Subject: Varnishstat question Message-ID: Greetings fellow varnish enthusiasts. I was wondering if anyone could help me determine what would be considered good values for the follow fields in varnishstat output and what these fields indicate: 1 0.00 0.00 N vcl total 1 0.00 0.00 N vcl available 7 . . N total active purges 7 0.00 0.00 N new purges added 6112 0.00 0.02 N objects tested Thanks so much! -- *Jonathan M. Hursey* *Linux Systems Administrator* * * -------------- next part -------------- An HTML attachment was scrubbed... URL: From carrot at carrotis.com Fri Jul 1 22:32:53 2011 From: carrot at carrotis.com (Calvin Park) Date: Sat, 2 Jul 2011 07:32:53 +0900 Subject: same ip address for backend config. Message-ID: Hello Varnish user Can I add "Host header" using same ip address ? Thanks. eg. backend svr1 { .host = "1.2.3.4"; .header="www.abc.com"; .port = "80"; } backend svr2 { .host = "1.2.3.4"; .header="www.def.com"; .port = "80"; } From varnish at mm.quex.org Sat Jul 2 07:54:06 2011 From: varnish at mm.quex.org (Michael Alger) Date: Sat, 2 Jul 2011 15:54:06 +0800 Subject: same ip address for backend config. In-Reply-To: References: Message-ID: <20110702075406.GA24581@grum.quex.org> On Sat, Jul 02, 2011 at 07:32:53AM +0900, Calvin Park wrote: > > Can I add "Host header" using same ip address ? > > Thanks. > > eg. > > backend svr1 { > .host = "1.2.3.4"; > .header="www.abc.com"; > .port = "80"; > } Yes you can, but not like that. If your backend server is configured for name-based virtual hosting, then you don't actually _need_ to do anything. When the browser sends the request, it will include the Host: header saying which hostname it thinks it's talking to. Varnish will send the same request headers to your backend, which will then be able to select the correct virtual server. You only need to use multiple backend definitions if your site(s) content is served by multiple servers on different IP addresses and/or ports; or if you want to use different parameters for some requests (e.g. connection timeouts). Even though you don't need to do anything special, it's still a good idea to "normalize" the host header to prevent duplicate objects being cached. e.g. http://example.com/home and http://www.example.com/home probably refer to the same object, but are two different URLs and will be cached independently. To prevent that, in your vcl_fetch() you can look at the value of req.http.host (i.e. the Host: header sent by the client) and implement your own logic as required: if (req.http.host == "abc.com") { set req.http.host = "www.abc.com"; } else if (req.http.host == "def.com") { set req.http.host = "www.def.com"; } You can also use this method to implement different logic for different host headers, e.g. selecting a different backend (if you have multiple backend servers on different IPs), or using different caching strategies. Since some UAs may append the port number to the hostname even if it's making the request on port 80 (i.e. "Host: www.example.com:80") it's probably a good idea to use patterns of this form: if (req.http.host ~ "(?i)^(www\.)?abc\.com") { set req.http.host = "www.abc.com"; } else if (req.http.host ~ "(?i)^(www\.)?def\.com") { set req.http.host = "www.def.com"; } You might also want to have a final else clause to catch any requests which don't include a (recognised) Host header and send them an error page. Note that the above patterns aren't 100% explicit, and will match requests with hosts like "www.abc.company.com" as if they were for "www.abc.com", but generally that doesn't matter and won't cause any security risks. You might need to make the pattern a bit more explicit if you do happen to host other sites with hostnames which share the same prefix (or, put the longer one first in your comparisons). From mattias at nucleus.be Sun Jul 3 17:46:50 2011 From: mattias at nucleus.be (Mattias Geniar) Date: Sun, 3 Jul 2011 19:46:50 +0200 Subject: Drupal 7.x & Varnish 3.x In-Reply-To: References: Message-ID: <18834F5BEC10824891FB8B22AC821A5A01735072@nucleus-srv01.Nucleus.local> > Any ideas what's missing in my default.vcl to get Varnish 3.0 up & > running with Drupal 7? Sorry for the late reply, but it seems this hasn't gotten an answer yet. Have you succeeded in making Varnish cache your Drupal installation? >From the logs, can you see if any other cookies are being sent (perhaps unique to some kind of Drupal module)? Are you logged in to Drupal when performing these tests? Regards, Mattias Geniar From ildella at gmail.com Mon Jul 4 16:17:41 2011 From: ildella at gmail.com (Daniele Dellafiore) Date: Mon, 4 Jul 2011 18:17:41 +0200 Subject: Skip post Message-ID: Hi. I'm a new user. Well, actually just potential user :) I've installed varnish 2.1 on my machine with Ubuntu via apt-get. I've configured a backend server, keeping the name "default" so I guess it's used by default without having to set it. I did not change anything else on the default.vcl file I launched varnish with sudo varnishd -f /etc/varnish/default.vcl -s malloc,1G -T 127.0.0.1:2000 -a 0.0.0.0:8080 In fact the GET are working, but the POST are not. I was calling http://localhost:8181/sessions with a POST to sign users in, but now, with Varnish on 8080 http://localhost:8080/sessions I get a Service Unavailable There's something else I should do? I see in the sub vcl_recv that only GET and HEAD are processed so I expected the POST to be ignored. Any advice? Thanks. -- Daniele Dellafiore http://danieledellafiore.net From lee at leetrout.com Mon Jul 4 16:27:41 2011 From: lee at leetrout.com (Lee Trout) Date: Mon, 4 Jul 2011 12:27:41 -0400 Subject: Skip post Message-ID: I'm sure some one will offer more detail but you need to pass in your vcl if the request method is POST. http://www.varnish-cache.org/trac/wiki/Introduction sent from my phone... On Jul 4, 2011 12:18 PM, "Daniele Dellafiore" wrote: -------------- next part -------------- An HTML attachment was scrubbed... URL: From ildella at gmail.com Mon Jul 4 16:33:35 2011 From: ildella at gmail.com (Daniele Dellafiore) Date: Mon, 4 Jul 2011 18:33:35 +0200 Subject: Skip post In-Reply-To: References: Message-ID: I've tried both ways. In fact in the default.vcl there's this: # if (req.request != "GET" && req.request != "HEAD") { # /* We only deal with GET and HEAD by default */ # return (pass); # } that for me is the same, or even better, than: if (req.request == "POST") { pass; } that is suggested in the introduction document. Anyway, even adding the check on the POST, and of course uncommenting all the sub vcl_recv, the problem remains. On Mon, Jul 4, 2011 at 6:27 PM, Lee Trout wrote: > I'm sure some one will offer more detail but you need to pass in your vcl if > the request method is POST. > > http://www.varnish-cache.org/trac/wiki/Introduction > > sent from my phone... > > On Jul 4, 2011 12:18 PM, "Daniele Dellafiore" wrote: > -- Daniele Dellafiore http://danieledellafiore.net From andy at hexten.net Mon Jul 4 18:30:33 2011 From: andy at hexten.net (Andy Armstrong) Date: Mon, 4 Jul 2011 19:30:33 +0100 Subject: Processing logs Message-ID: <65AB3201-79EF-4B5D-B73A-C1024ECC166C@hexten.net> I'm trying to convert some varnish logs using varnishncsa -r log > log.ncsa It runs out of memory on a 4Gb box while processing a 4.7Mb log. That doesn't bode well for when it gets to the larger logs... Also I notice that varnishncsa won't run unless I have varnishd running on the same box - even when asked to read from a log file. Is there another way of extracting human readable / greppable logs? Thanks! -- Andy Armstrong, Hexten From jhayter at manta.com Tue Jul 5 15:30:44 2011 From: jhayter at manta.com (Jim Hayter) Date: Tue, 5 Jul 2011 15:30:44 +0000 Subject: Via header added to requests Message-ID: We are running varnish 2.0.5 in front of apache 2.2. I have been asked if it is possible to stop varnish from adding to the Via header when it sends a request to the backend. I suspect not after reading the RFCs ("The Via general-header field MUST be used by gateways and proxies to indicate the intermediate protocols and recipients between the user agent and the server on requests, and between the origin server and the client on responses."). I just wanted to verify my opinion. Thanks, Jim -- Jim Hayter From phk at phk.freebsd.dk Tue Jul 5 15:34:01 2011 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Tue, 05 Jul 2011 15:34:01 +0000 Subject: Via header added to requests In-Reply-To: Your message of "Tue, 05 Jul 2011 15:30:44 GMT." Message-ID: <4925.1309880041@critter.freebsd.dk> In message , Jim Hayter writ es: >We are running varnish 2.0.5 in front of apache 2.2. I have been >asked if it is possible to stop varnish from adding to the Via >header when it sends a request to the backend. I belive you can remove it in vcl_miss, vcl_pass and vcl_pipe. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From jhayter at manta.com Tue Jul 5 16:14:32 2011 From: jhayter at manta.com (Jim Hayter) Date: Tue, 5 Jul 2011 16:14:32 +0000 Subject: Via header added to requests In-Reply-To: <4925.1309880041@critter.freebsd.dk> References: Your message of "Tue, 05 Jul 2011 15:30:44 GMT." <4925.1309880041@critter.freebsd.dk> Message-ID: Thank you. That did indeed work. Jim -----Original Message----- From: phk at critter.freebsd.dk [mailto:phk at critter.freebsd.dk] On Behalf Of Poul-Henning Kamp Sent: Tuesday, July 05, 2011 11:34 AM To: Jim Hayter Cc: varnish-misc at varnish-cache.org Subject: Re: Via header added to requests In message , Jim Hayter writ es: >We are running varnish 2.0.5 in front of apache 2.2. I have been >asked if it is possible to stop varnish from adding to the Via >header when it sends a request to the backend. I belive you can remove it in vcl_miss, vcl_pass and vcl_pipe. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From samcrawford at gmail.com Tue Jul 5 16:28:37 2011 From: samcrawford at gmail.com (Sam Crawford) Date: Tue, 5 Jul 2011 17:28:37 +0100 Subject: Skip post In-Reply-To: References: Message-ID: What you have should be sufficient. Perhaps try running varnishlog whilst making a POST request and then pasting the (scrubbed) output here? Thanks, Sam On 4 July 2011 17:33, Daniele Dellafiore wrote: > I've tried both ways. > In fact in the default.vcl there's this: > > # ? ? if (req.request != "GET" && req.request != "HEAD") { > # ? ? ? ? /* We only deal with GET and HEAD by default */ > # ? ? ? ? return (pass); > # ? ? } > > that for me is the same, or even better, than: > > ?if (req.request == "POST") { > ? ? pass; > ?} > > that is suggested in the introduction document. > Anyway, even adding the check on the POST, and of course uncommenting > all the sub vcl_recv, the problem remains. > > On Mon, Jul 4, 2011 at 6:27 PM, Lee Trout wrote: >> I'm sure some one will offer more detail but you need to pass in your vcl if >> the request method is POST. >> >> http://www.varnish-cache.org/trac/wiki/Introduction >> >> sent from my phone... >> >> On Jul 4, 2011 12:18 PM, "Daniele Dellafiore" wrote: >> > > > > -- > Daniele Dellafiore > http://danieledellafiore.net > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > From flatcircle at hotmail.com Tue Jul 5 18:23:26 2011 From: flatcircle at hotmail.com (Flatcircle) Date: Tue, 5 Jul 2011 20:23:26 +0200 Subject: Drupal 7.x & Varnish 3.x In-Reply-To: <18834F5BEC10824891FB8B22AC821A5A01735072@nucleus-srv01.Nucleus.local> References: <18834F5BEC10824891FB8B22AC821A5A01735072@nucleus-srv01.Nucleus.local> Message-ID: An HTML attachment was scrubbed... URL: From btruitt at rackspace.com Tue Jul 5 20:56:46 2011 From: btruitt at rackspace.com (Ben Truitt) Date: Tue, 5 Jul 2011 20:56:46 +0000 Subject: HTCP CLR operation for purging clustered reverse proxy cache Message-ID: <2334798D-2C7A-497B-8DE6-5F8C4CBBB0DF@rackspace.com> Hi - Thanks for the great work on Varnish! My team at Rackspace is considering adopting Varnish for some RESTful services we're creating. We're planning to have a cluster of reverse proxy cache servers in front of our origin servers. One of the issues that is important to us is the ability to keep the cache consistent with the origin server's data store as much as possible. I realize that Varnish supports HTTP PURGE operation, but we'd prefer to use something like HTCP's CLR operation for a multicast purge. I came across a fairly old varnish-dev mailing list item related to this: http://www.mail-archive.com/varnish-dev at projects.linpro.no/msg00315.html That post indicates that HTCP CLR would be incorporated into Varnish, but I don't see that it has been. I'd love to know: 1) Are there plans to include HTCP CLR in Varnish any time soon? 2) Is there a plugin (such as the one Artur Bergman created) available that I can use today? Thanks in advance for your time! Best, Benjamin Truitt This email may include confidential information. If you received it in error, please delete it. From gilhildebrand at gmail.com Tue Jul 5 23:14:16 2011 From: gilhildebrand at gmail.com (Gil Hildebrand) Date: Tue, 5 Jul 2011 17:14:16 -0600 Subject: LostHeader in Varnish 3.0.0 Message-ID: I am testing upgrade from Varnish 2.0 to 3.0. In some cases, because of our advertising and analytics partners, cookies can be excessively long. This appears to be a problem in Varnish 3.0, as I get a 400 Bad Request response. When I look at varnishlog, I see a LostHeader for the Cookie header. I found some history on similar issues: http://comments.gmane.org/gmane.comp.web.varnish.misc/3236 https://www.varnish-cache.org/trac/ticket/455 https://www.varnish-cache.org/lists/pipermail/varnish-commit/2009-April/004227.html Initially I installed Varnish via rpm, so I downloaded the source to see if I could fix the problem via patch and compile. Unfortunately, I could not find HTTP_HDR_MAX_VAL anywhere in the source. Was it removed? Is there now a better way to increase the max number of headers? -------------- next part -------------- An HTML attachment was scrubbed... URL: From lee at leetrout.com Tue Jul 5 23:44:28 2011 From: lee at leetrout.com (Lee Trout) Date: Tue, 5 Jul 2011 19:44:28 -0400 Subject: LostHeader in Varnish 3.0.0 In-Reply-To: References: Message-ID: Great question! Just solved this problem this week... The docs are horribly outdated / missing info :( You need to use runtime parameters. For example, to allow 20k headers (default is 2k) use the following params when you start varnish: -p http_req_hdr_len=20000 -p http_resp_hdr_len=20000 Lee On Tue, Jul 5, 2011 at 7:14 PM, Gil Hildebrand wrote: > I am testing upgrade from Varnish 2.0 to 3.0. > > In some cases, because of our advertising and analytics partners, cookies > can be excessively long. This appears to be a problem in Varnish 3.0, as I > get a 400 Bad Request response. When I look at varnishlog, I see a > LostHeader for the Cookie header. > > I found some history on similar issues: > > http://comments.gmane.org/gmane.comp.web.varnish.misc/3236 > https://www.varnish-cache.org/trac/ticket/455 > > https://www.varnish-cache.org/lists/pipermail/varnish-commit/2009-April/004227.html > > Initially I installed Varnish via rpm, so I downloaded the source to see if > I could fix the problem via patch and compile. Unfortunately, I could not > find HTTP_HDR_MAX_VAL anywhere in the source. Was it removed? Is there now > a better way to increase the max number of headers? > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gilhildebrand at gmail.com Tue Jul 5 23:52:23 2011 From: gilhildebrand at gmail.com (Gil Hildebrand) Date: Tue, 5 Jul 2011 17:52:23 -0600 Subject: LostHeader in Varnish 3.0.0 In-Reply-To: References: Message-ID: Why thank you Lee! That was exactly what I needed. On Tue, Jul 5, 2011 at 5:44 PM, Lee Trout wrote: > Great question! > > Just solved this problem this week... The docs are horribly outdated / > missing info :( > > You need to use runtime parameters. For example, to allow 20k headers > (default is 2k) use the following params when you start varnish: > > -p http_req_hdr_len=20000 -p http_resp_hdr_len=20000 > > Lee > > On Tue, Jul 5, 2011 at 7:14 PM, Gil Hildebrand wrote: > >> I am testing upgrade from Varnish 2.0 to 3.0. >> >> In some cases, because of our advertising and analytics partners, cookies >> can be excessively long. This appears to be a problem in Varnish 3.0, as I >> get a 400 Bad Request response. When I look at varnishlog, I see a >> LostHeader for the Cookie header. >> >> I found some history on similar issues: >> >> http://comments.gmane.org/gmane.comp.web.varnish.misc/3236 >> https://www.varnish-cache.org/trac/ticket/455 >> >> https://www.varnish-cache.org/lists/pipermail/varnish-commit/2009-April/004227.html >> >> Initially I installed Varnish via rpm, so I downloaded the source to see >> if I could fix the problem via patch and compile. Unfortunately, I could not >> find HTTP_HDR_MAX_VAL anywhere in the source. Was it removed? Is there >> now a better way to increase the max number of headers? >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mattias at nucleus.be Wed Jul 6 06:38:16 2011 From: mattias at nucleus.be (Mattias Geniar) Date: Wed, 6 Jul 2011 08:38:16 +0200 Subject: Drupal 7.x & Varnish 3.x In-Reply-To: References: <18834F5BEC10824891FB8B22AC821A5A01735072@nucleus-srv01.Nucleus.local> Message-ID: <18834F5BEC10824891FB8B22AC821A5A01735124@nucleus-srv01.Nucleus.local> > Sadly, no... > > I tried several? configurations but still without success. > (The only 2 cookies with D7 are has_js and Drupal.toolbar.collapsed.) Perhaps you can have a look at these Drupal templates (_drupal_7-fetch/receive): https://github.com/mattiasgeniar/varnish-3.0-configuration-templates/tree/master/conf.d Or the one linked at drupal.org: http://drupal.org/files/issues/default.vcl_.txt These seem to yield great results for me, with a default Drupal 7 nearly all pages can be cached. Regards, Mattias Geniar From audun at ytterdal.net Wed Jul 6 07:42:19 2011 From: audun at ytterdal.net (Audun Ytterdal) Date: Wed, 6 Jul 2011 09:42:19 +0200 Subject: LostHeader in Varnish 3.0.0 In-Reply-To: References: Message-ID: I had to increase thread_pool_workspace and sess_workspace as well to prevent asserert-errors. On Wed, Jul 6, 2011 at 1:52 AM, Gil Hildebrand wrote: > Why thank you Lee! That was exactly what I needed. > > On Tue, Jul 5, 2011 at 5:44 PM, Lee Trout wrote: >> >> Great question! >> Just solved this problem this week... ?The docs are horribly outdated / >> missing info :( >> You need to use runtime parameters. ?For example, to allow 20k headers >> (default is 2k) use the following params when you start varnish: >> -p http_req_hdr_len=20000 -p http_resp_hdr_len=20000 >> Lee >> On Tue, Jul 5, 2011 at 7:14 PM, Gil Hildebrand >> wrote: >>> >>> I am testing upgrade from Varnish 2.0 to 3.0. >>> In some cases, because of our advertising and analytics partners, cookies >>> can be excessively long.?This appears to be a problem in Varnish 3.0, as I >>> get a 400 Bad Request response. When I look at varnishlog, I see a >>> LostHeader for the Cookie header. >>> I found some history on similar issues: >>> http://comments.gmane.org/gmane.comp.web.varnish.misc/3236 >>> https://www.varnish-cache.org/trac/ticket/455 >>> >>> https://www.varnish-cache.org/lists/pipermail/varnish-commit/2009-April/004227.html >>> Initially I installed Varnish via rpm, so I downloaded the source to see >>> if I could fix the problem via patch and compile. Unfortunately, I could not >>> find?HTTP_HDR_MAX_VAL anywhere in the source. Was it removed? Is there now a >>> better way to increase the max number of headers? >>> _______________________________________________ >>> varnish-misc mailing list >>> varnish-misc at varnish-cache.org >>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > From nfn at gmx.com Wed Jul 6 09:26:49 2011 From: nfn at gmx.com (Nuno Neves) Date: Wed, 06 Jul 2011 09:26:49 +0000 Subject: Purging on form submit Message-ID: <20110706092649.283130@gmx.com> Hello, I'm running a forum that has a ttl to expire content to guests only, but I would like to opitmize it to purge a page when we have new posts or after 24h (to update counters). My 1st question is how can I get the hiden input value I have that tell varnish that we have a new post? I was thinking of something like this: sub vcl_recv { if (req.request == "POST" && FORM.FIELD == "canpurge) { purge req.http.host == example.com && req.url ~ ^FORM.CANPURGE.VALUE.*$ } } This is an excertt of my actual configuration: ==================================================================================================== sub vcl_recv { ## == Mobile == if (req.http.User-Agent ~ "(iPad|iPhone|iPod|Android|SymbianOS|^BlackBerry|^SonyEricsson|^Nokia|^SAMSUNG|^LG)") { return(pass); } if (!((req.http.Cookie ~ "member_id=" && req.http.Cookie !~ "member_id=(0|-1)") || req.http.Cookie ~ "(guestSkinChoice|language)")) { if (req.url ~ "^/(public|forum|topic|gallery|blogs|members|user|calendar)/") { unset req.http.cookie; set req.grace = 15s; } if (req.url == "/" || req.url == "/index.php") { unset req.http.cookie; set req.grace = 15s; } } } sub vcl_fetch { if (!((req.http.Cookie ~ "member_id=" && req.http.Cookie !~ "member_id=(0|-1)") || req.http.Cookie ~ "(guestSkinChoice|language)")) { ## == INDEX == if (req.url == "/" || req.url == "/index.php") { unset beresp.http.set-cookie; set beresp.ttl = 300s; set beresp.grace = 30s; } ## == ESI == if (req.url ~ "^/(gallery/image|topic)/") { set beresp.do_esi = true; unset beresp.http.set-cookie; set beresp.ttl = 300s; set beresp.grace = 30s; ## Others } elseif (req.url ~ "^/(public|forum|gallery|blogs|members|user|calendar)/") { unset beresp.http.set-cookie; set beresp.ttl = 600s; set beresp.grace = 30s; } } } ==================================================================================================== Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From rtshilston at gmail.com Wed Jul 6 10:25:56 2011 From: rtshilston at gmail.com (Rob S) Date: Wed, 06 Jul 2011 11:25:56 +0100 Subject: Purging on form submit In-Reply-To: <20110706092649.283130@gmx.com> References: <20110706092649.283130@gmx.com> Message-ID: <4E143834.8030206@gmail.com> On 06/07/2011 10:26, Nuno Neves wrote: > how can I get the hiden input value I have that tell varnish that we > have a new post? > > > > I was thinking of something like this: > > sub vcl_recv { > if (req.request == "POST" && FORM.FIELD == "canpurge) { > purge req.http.host == example.com && req.url ~ > ^FORM.CANPURGE.VALUE.*$ > } > } Nuno: Varnish can't see the body of the response. However, if you're able to change the response, you can just add an HTTP header with this information. Then, in vcl_fetch (which is where the HTTP request is sent to the backend), you can probably add something like: sub vcl_fetch { if (beresp.http.x-purgepattern) { purge("obj.http.host == example.com && req.url ~ " beresp.http.x-purgepattern); } } Rob -------------- next part -------------- An HTML attachment was scrubbed... URL: From mattias at nucleus.be Wed Jul 6 10:40:21 2011 From: mattias at nucleus.be (Mattias Geniar) Date: Wed, 6 Jul 2011 12:40:21 +0200 Subject: Purging on form submit In-Reply-To: <4E143834.8030206@gmail.com> References: <20110706092649.283130@gmx.com> <4E143834.8030206@gmail.com> Message-ID: <18834F5BEC10824891FB8B22AC821A5A0173513E@nucleus-srv01.Nucleus.local> > [snip] >? However, if you're able to change the response, you can just add an HTTP header with this information.? Then, in vcl_fetch (which is where the HTTP request is sent to the backend), you can probably add something like: > If you can add extra headers for the request, you can just as easily use fsockopen() in PHP to open a socket over port 80 to your Varnish and issue a "PURGE" request. That way, your normal purging ACL's would apply. That's how the Wordpress Varnish plugin works, see lines 330+ in the following file: https://github.com/ojdupuis/wp-varnish/blob/608b9c5ff276fbb3c3bf61fc9c50b13e882ab2ad/wp-varnish.php Regards, Mattias Geniar From nfn at gmx.com Wed Jul 6 11:52:26 2011 From: nfn at gmx.com (Nuno Neves) Date: Wed, 06 Jul 2011 11:52:26 +0000 Subject: Purging on form submit Message-ID: <20110706115226.283110@gmx.com> Hello, I managed to add an new header: x-url: /topic/123-my-topic/ So I just need to add this bit? sub vcl_fetch { if (req.request == "POST" && beresp.http.x-url) { purge("obj.http.host == example.com && req.url ~ " beresp.http.x-url); } } Note that I need to purge /topic/123-my-topic/ and /topic/123-my-topic/all pages after slash. In this case can I use purge("obj.http.host == example.com && req.url ~ " beresp.http.x-url.*$);? Thanks ----- Original Message ----- From: Rob S Sent: 07/06/11 11:25 AM To: varnish-misc at varnish-cache.org Subject: Re: Purging on form submit On 06/07/2011 10:26, Nuno Neves wrote: how can I get the hiden input value I have that tell varnish that we have a new post? I was thinking of something like this: sub vcl_recv { if (req.request == "POST" && FORM.FIELD == "canpurge) { purge req.http.host == example.com && req.url ~ ^FORM.CANPURGE.VALUE.*$ } } Nuno: Varnish can't see the body of the response. However, if you're able to change the response, you can just add an HTTP header with this information. Then, in vcl_fetch (which is where the HTTP request is sent to the backend), you can probably add something like: sub vcl_fetch { if (beresp.http.x-purgepattern) { purge("obj.http.host == example.com && req.url ~ " beresp.http.x-purgepattern); } } Rob -------------- next part -------------- An HTML attachment was scrubbed... URL: From shib4u at gmail.com Wed Jul 6 12:11:27 2011 From: shib4u at gmail.com (Shibashish) Date: Wed, 6 Jul 2011 17:41:27 +0530 Subject: Question concerning rewriting browser address bar In-Reply-To: References: <4E0DB80D.5090209@wetteronline.de> Message-ID: On Fri, Jul 1, 2011 at 7:13 PM, Jonathan Hursey < jonathan.hursey at adrevolution.com> wrote: > if you're just doing it for one: > > set req.http.host = regsub(req.http.host, "10.10.0.2", "www.domname.de"); > > if you want a universal catch all on that I'm not too sure. Sounds like > you'd need to resolve the name. or mod_modrewrite on the backend. > > > On Fri, Jul 1, 2011 at 7:05 AM, David Theis wrote: > >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> Hi everybody, >> >> i'm getting right to my question: >> if a user contacts one of our proxy servers using only the ip address >> i'd like to rewrite this ip address into a domain name, so that the user >> sees www.domainname.de instead of e.g 10.0.0.2 in his/her browser >> address bar. is this possible with varnishd ? >> >> We're using freebsd-7.4-RELEASE + varnish 2.1.5 ! >> >> mfg >> David >> -----BEGIN PGP SIGNATURE----- >> Version: GnuPG v2.0.17 (GNU/Linux) >> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ >> >> iEYEARECAAYFAk4NuA0ACgkQu/Z5Qo6690XlYACgj2or4DFsYSxMiAUPoTRjr9y0 >> xWQAoJSW112qam6BUixSZjwEibsLMLY0 >> =hw0m >> -----END PGP SIGNATURE----- >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> > > > > -- > *Jonathan M. Hursey* > *Linux Systems Administrator* > > To catch all IP addresses, use... if (req.http.host ~ "^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])$") { error 750 "http://www.domain.com"; } sub vcl_error { if (obj.status == 750) { set obj.http.Location = obj.response req.url; set obj.status = 301; return(deliver); } } -------------- next part -------------- An HTML attachment was scrubbed... URL: From btruitt at rackspace.com Wed Jul 6 14:38:29 2011 From: btruitt at rackspace.com (Ben Truitt) Date: Wed, 6 Jul 2011 14:38:29 +0000 Subject: HTCP CLR operation for purging clustered reverse proxy cache In-Reply-To: <2334798D-2C7A-497B-8DE6-5F8C4CBBB0DF@rackspace.com> References: <2334798D-2C7A-497B-8DE6-5F8C4CBBB0DF@rackspace.com> Message-ID: <5DA70491-6E8F-4DEE-A4A2-D1F1D97CF60A@rackspace.com> I was able to find the location of the source code for the HTCP plugin I referred to in the below email (thanks, Artur Bergman): https://svn.wikia-code.com/utils/varnishhtcpd/ I'm still curious to know whether there are any plans to incorporate this or similar functionality for purging a clustered cache into Varnish going forward. Thanks, Benjamin On Jul 5, 2011, at 4:56 PM, wrote: > Hi - > Thanks for the great work on Varnish! > > My team at Rackspace is considering adopting Varnish for some RESTful services we're creating. We're planning to have a cluster of reverse proxy cache servers in front of our origin servers. One of the issues that is important to us is the ability to keep the cache consistent with the origin server's data store as much as possible. > > I realize that Varnish supports HTTP PURGE operation, but we'd prefer to use something like HTCP's CLR operation for a multicast purge. I came across a fairly old varnish-dev mailing list item related to this: > > http://www.mail-archive.com/varnish-dev at projects.linpro.no/msg00315.html > > That post indicates that HTCP CLR would be incorporated into Varnish, but I don't see that it has been. > > I'd love to know: > 1) Are there plans to include HTCP CLR in Varnish any time soon? > 2) Is there a plugin (such as the one Artur Bergman created) available that I can use today? > > Thanks in advance for your time! > > Best, > Benjamin Truitt This email may include confidential information. If you received it in error, please delete it. From perbu at varnish-software.com Wed Jul 6 14:44:08 2011 From: perbu at varnish-software.com (Per Buer) Date: Wed, 6 Jul 2011 16:44:08 +0200 Subject: HTCP CLR operation for purging clustered reverse proxy cache In-Reply-To: <5DA70491-6E8F-4DEE-A4A2-D1F1D97CF60A@rackspace.com> References: <2334798D-2C7A-497B-8DE6-5F8C4CBBB0DF@rackspace.com> <5DA70491-6E8F-4DEE-A4A2-D1F1D97CF60A@rackspace.com> Message-ID: Hi, Why should this be in Varnish directly? Whats wrong with having it in a separate program? Per. On Wed, Jul 6, 2011 at 4:38 PM, Ben Truitt wrote: > I was able to find the location of the source code for the HTCP plugin I > referred to in the below email (thanks, Artur Bergman): > > https://svn.wikia-code.com/utils/varnishhtcpd/ > > I'm still curious to know whether there are any plans to incorporate this > or similar functionality for purging a clustered cache into Varnish going > forward. > > Thanks, > Benjamin > > On Jul 5, 2011, at 4:56 PM, wrote: > > > Hi - > > Thanks for the great work on Varnish! > > > > My team at Rackspace is considering adopting Varnish for some RESTful > services we're creating. We're planning to have a cluster of reverse proxy > cache servers in front of our origin servers. One of the issues that is > important to us is the ability to keep the cache consistent with the origin > server's data store as much as possible. > > > > I realize that Varnish supports HTTP PURGE operation, but we'd prefer to > use something like HTCP's CLR operation for a multicast purge. I came > across a fairly old varnish-dev mailing list item related to this: > > > > http://www.mail-archive.com/varnish-dev at projects.linpro.no/msg00315.html > > > > That post indicates that HTCP CLR would be incorporated into Varnish, but > I don't see that it has been. > > > > I'd love to know: > > 1) Are there plans to include HTCP CLR in Varnish any time soon? > > 2) Is there a plugin (such as the one Artur Bergman created) available > that I can use today? > > > > Thanks in advance for your time! > > > > Best, > > Benjamin Truitt > > This email may include confidential information. If you received it in > error, please delete it. > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -- Per Buer, CEO Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer *Varnish makes websites fly!* Whitepapers | Video | Twitter -------------- next part -------------- An HTML attachment was scrubbed... URL: From btruitt at rackspace.com Wed Jul 6 15:37:28 2011 From: btruitt at rackspace.com (Ben Truitt) Date: Wed, 6 Jul 2011 15:37:28 +0000 Subject: HTCP CLR operation for purging clustered reverse proxy cache In-Reply-To: References: <2334798D-2C7A-497B-8DE6-5F8C4CBBB0DF@rackspace.com> <5DA70491-6E8F-4DEE-A4A2-D1F1D97CF60A@rackspace.com> Message-ID: <5D2722B0-56D4-4D58-AF47-6C3485B47043@rackspace.com> Good question! In my opinion, invalidating cache entries is an essential operation of a cache. Varnish already supports HTTP PURGE without a separate process, so invalidating entries is supported in Varnish directly. What is not supported directly is invalidating entries across a cluster. This requires listening on a separate port, but that doesn't seem like a very strong reason for breaking the functionality into a separate program. It also seems as though keeping the code for HTCP alongside the rest of Varnish will promote maintainability. Adding this to Varnish proper should lead to a good start for supporting the other operations in HTCP, specifically the ability for a server to query its peers to determine whether they have a cached representation available. Furthermore, Squid supports HTCP out of the box, so feature parity in this area will make it easier to gain market share. Thoughts? -Benjamin On Jul 6, 2011, at 10:44 AM, Per Buer wrote: Hi, Why should this be in Varnish directly? Whats wrong with having it in a separate program? Per. On Wed, Jul 6, 2011 at 4:38 PM, Ben Truitt > wrote: I was able to find the location of the source code for the HTCP plugin I referred to in the below email (thanks, Artur Bergman): https://svn.wikia-code.com/utils/varnishhtcpd/ I'm still curious to know whether there are any plans to incorporate this or similar functionality for purging a clustered cache into Varnish going forward. Thanks, Benjamin On Jul 5, 2011, at 4:56 PM, wrote: > Hi - > Thanks for the great work on Varnish! > > My team at Rackspace is considering adopting Varnish for some RESTful services we're creating. We're planning to have a cluster of reverse proxy cache servers in front of our origin servers. One of the issues that is important to us is the ability to keep the cache consistent with the origin server's data store as much as possible. > > I realize that Varnish supports HTTP PURGE operation, but we'd prefer to use something like HTCP's CLR operation for a multicast purge. I came across a fairly old varnish-dev mailing list item related to this: > > http://www.mail-archive.com/varnish-dev at projects.linpro.no/msg00315.html > > That post indicates that HTCP CLR would be incorporated into Varnish, but I don't see that it has been. > > I'd love to know: > 1) Are there plans to include HTCP CLR in Varnish any time soon? > 2) Is there a plugin (such as the one Artur Bergman created) available that I can use today? > > Thanks in advance for your time! > > Best, > Benjamin Truitt This email may include confidential information. If you received it in error, please delete it. _______________________________________________ varnish-misc mailing list varnish-misc at varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc -- [http://www.varnish-software.com/sites/default/files/varnishsoft_white_190x47.png] Per Buer, CEO Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer Varnish makes websites fly! Whitepapers | Video | Twitter This email may include confidential information. If you received it in error, please delete it. -------------- next part -------------- An HTML attachment was scrubbed... URL: From andy at hexten.net Wed Jul 6 17:06:02 2011 From: andy at hexten.net (Andy Armstrong) Date: Wed, 6 Jul 2011 18:06:02 +0100 Subject: Processing logs In-Reply-To: <65AB3201-79EF-4B5D-B73A-C1024ECC166C@hexten.net> References: <65AB3201-79EF-4B5D-B73A-C1024ECC166C@hexten.net> Message-ID: <63FE4315-F3AB-40B7-BAEA-A46022EE3EDD@hexten.net> On 4 Jul 2011, at 19:30, Andy Armstrong wrote: > I'm trying to convert some varnish logs using > > varnishncsa -r log > log.ncsa > > It runs out of memory on a 4Gb box while processing a 4.7Mb log. That doesn't bode well for when it gets to the larger logs... Sorry for the false alarm folks - turns out it was a problem with the log files. All sorted now. -- Andy Armstrong, Hexten From nfn at gmx.com Wed Jul 6 19:28:39 2011 From: nfn at gmx.com (Nuno Neves) Date: Wed, 6 Jul 2011 20:28:39 +0100 Subject: Purging on form submit In-Reply-To: <4E143834.8030206@gmail.com> References: <20110706092649.283130@gmx.com> <4E143834.8030206@gmail.com> Message-ID: <64B5A4B6-DFE1-4F31-BC6E-A0FED207FA3B@gmx.com> Hello, Using this code I get an error when starting varnish 3.0 The code: sub vcl_fetch { if (beresp.http.X-Purge) { purge("obj.http.host == example.com && req.url ~ " beresp.http.X-Purge); } } The error: * Starting HTTP accelerator varnishd ...fail! SMA.s0: max size 256 MB. Message from VCC-compiler: Expected ';' got '(' (program line 174), at ('input' Line 52 Pos 11) purge ("obj.http.host == example.com && req.url ~ " beresp.http.X-Purge); ----------#--------------------------------------------------------------------------- What do I need to change? Thanks Nuno A 2011/07/06, ?s 11:25, Rob S escreveu: > On 06/07/2011 10:26, Nuno Neves wrote: >> >> how can I get the hiden input value I have that tell varnish that we have a new post? >> >> >> >> I was thinking of something like this: >> >> sub vcl_recv { >> if (req.request == "POST" && FORM.FIELD == "canpurge) { >> purge req.http.host == example.com && req.url ~ ^FORM.CANPURGE.VALUE.*$ >> } >> } > > Nuno: > > Varnish can't see the body of the response. However, if you're able to change the response, you can just add an HTTP header with this information. Then, in vcl_fetch (which is where the HTTP request is sent to the backend), you can probably add something like: > > sub vcl_fetch { > if (beresp.http.x-purgepattern) { > purge("obj.http.host == example.com && req.url ~ " beresp.http.x-purgepattern); > } > } > > > Rob > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc -------------- next part -------------- An HTML attachment was scrubbed... URL: From btruitt at rackspace.com Wed Jul 6 21:16:01 2011 From: btruitt at rackspace.com (Ben Truitt) Date: Wed, 6 Jul 2011 21:16:01 +0000 Subject: HTCP CLR operation for purging clustered reverse proxy cache In-Reply-To: <5D2722B0-56D4-4D58-AF47-6C3485B47043@rackspace.com> References: <2334798D-2C7A-497B-8DE6-5F8C4CBBB0DF@rackspace.com> <5DA70491-6E8F-4DEE-A4A2-D1F1D97CF60A@rackspace.com> <5D2722B0-56D4-4D58-AF47-6C3485B47043@rackspace.com> Message-ID: Last comment on this, and then I'll leave it to you all to discuss as you see fit: In investigating Varnish more today, I was very surprised to learn that it does not respect section 13.10 of the HTTP/1.1 RFC (http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html#sec13), which states "Some HTTP methods MUST cause a cache to invalidate an entity. These methods are: PUT, DELETE, POST." I found this thread (http://www.gossamer-threads.com/lists/varnish/misc/19516), where Per Buer comments: "We won't do this because it would totally break if you have more then one Varnish servers. It's the backend job to notify the caches when the content actually changes." I disagree. According to the spec, this is the job of the cache. The spec doesn't address a clustered cache, but I would expect a clustered cache to perform clustered invalidation. HTCP CLR is designed for this purpose. Therefore, to me it makes total sense to incorporate this into Varnish proper. Thanks again for your time and attention! -Benjamin On Jul 6, 2011, at 11:37 AM, wrote: Good question! In my opinion, invalidating cache entries is an essential operation of a cache. Varnish already supports HTTP PURGE without a separate process, so invalidating entries is supported in Varnish directly. What is not supported directly is invalidating entries across a cluster. This requires listening on a separate port, but that doesn't seem like a very strong reason for breaking the functionality into a separate program. It also seems as though keeping the code for HTCP alongside the rest of Varnish will promote maintainability. Adding this to Varnish proper should lead to a good start for supporting the other operations in HTCP, specifically the ability for a server to query its peers to determine whether they have a cached representation available. Furthermore, Squid supports HTCP out of the box, so feature parity in this area will make it easier to gain market share. Thoughts? -Benjamin On Jul 6, 2011, at 10:44 AM, Per Buer wrote: Hi, Why should this be in Varnish directly? Whats wrong with having it in a separate program? Per. On Wed, Jul 6, 2011 at 4:38 PM, Ben Truitt > wrote: I was able to find the location of the source code for the HTCP plugin I referred to in the below email (thanks, Artur Bergman): https://svn.wikia-code.com/utils/varnishhtcpd/ I'm still curious to know whether there are any plans to incorporate this or similar functionality for purging a clustered cache into Varnish going forward. Thanks, Benjamin On Jul 5, 2011, at 4:56 PM, wrote: > Hi - > Thanks for the great work on Varnish! > > My team at Rackspace is considering adopting Varnish for some RESTful services we're creating. We're planning to have a cluster of reverse proxy cache servers in front of our origin servers. One of the issues that is important to us is the ability to keep the cache consistent with the origin server's data store as much as possible. > > I realize that Varnish supports HTTP PURGE operation, but we'd prefer to use something like HTCP's CLR operation for a multicast purge. I came across a fairly old varnish-dev mailing list item related to this: > > http://www.mail-archive.com/varnish-dev at projects.linpro.no/msg00315.html > > That post indicates that HTCP CLR would be incorporated into Varnish, but I don't see that it has been. > > I'd love to know: > 1) Are there plans to include HTCP CLR in Varnish any time soon? > 2) Is there a plugin (such as the one Artur Bergman created) available that I can use today? > > Thanks in advance for your time! > > Best, > Benjamin Truitt This email may include confidential information. If you received it in error, please delete it. _______________________________________________ varnish-misc mailing list varnish-misc at varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc -- [http://www.varnish-software.com/sites/default/files/varnishsoft_white_190x47.png] Per Buer, CEO Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer Varnish makes websites fly! Whitepapers | Video | Twitter This email may include confidential information. If you received it in error, please delete it. -------------- next part -------------- An HTML attachment was scrubbed... URL: From simon at darkmere.gen.nz Wed Jul 6 22:03:16 2011 From: simon at darkmere.gen.nz (Simon Lyall) Date: Thu, 7 Jul 2011 10:03:16 +1200 (NZST) Subject: HTCP CLR operation for purging clustered reverse proxy cache In-Reply-To: References: <2334798D-2C7A-497B-8DE6-5F8C4CBBB0DF@rackspace.com> <5DA70491-6E8F-4DEE-A4A2-D1F1D97CF60A@rackspace.com> <5D2722B0-56D4-4D58-AF47-6C3485B47043@rackspace.com> Message-ID: On Wed, 6 Jul 2011, Ben Truitt wrote: > Last comment on this, and then I'll leave it to you all to discuss as you see fit: > > In investigating Varnish more today, I was very surprised to learn that it does not respect?section 13.10 of the HTTP/1.1 RFC > (http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html#sec13), which states "Some HTTP methods MUST cause a cache to invalidate > an entity.?These methods are:?PUT,?DELETE,?POST."? > > I found this thread (http://www.gossamer-threads.com/lists/varnish/misc/19516), where Per Buer comments: > "We?won't?do?this?because?it?would?totally?break?if?you?have?more?then?one?Varnish?servers. It's the backend job to notify the > caches when the content?actually changes." > > I disagree. According to the spec, this is the job of the cache. The spec doesn't address a clustered cache, but I would expect > a clustered cache to perform clustered invalidation. HTCP CLR is designed for this purpose. Therefore, to me it makes total > sense to incorporate this into Varnish proper.? The thing is the RFC section is really written for caches that are running by the client (or their network provider) not those run by the website. The varnish cache is part of the website, it is inside the demarcation and from the point of view of the client is a webserver like apache. The main difference is that it is under the control of the website and the website can explicitly remove [and sometimes add] something from the cache. The cache in the spec is run by the client and the only way the server admin can talk to it is via headers added to http replies. If I send a PUT,DELETE or POST to your website do you remove entries for that page from your memcache, application cache and database cache? -- Simon Lyall | Very Busy | Web: http://www.darkmere.gen.nz/ "To stay awake all night adds a day to your life" - Stilgar | eMT. From l at lrowe.co.uk Wed Jul 6 22:10:42 2011 From: l at lrowe.co.uk (Laurence Rowe) Date: Wed, 6 Jul 2011 23:10:42 +0100 Subject: Purging on form submit In-Reply-To: <64B5A4B6-DFE1-4F31-BC6E-A0FED207FA3B@gmx.com> References: <20110706092649.283130@gmx.com> <4E143834.8030206@gmail.com> <64B5A4B6-DFE1-4F31-BC6E-A0FED207FA3B@gmx.com> Message-ID: In Varnish 3 it has been renamed to 'ban'. See https://www.varnish-cache.org/docs/3.0/tutorial/purging.html Laurence On 6 July 2011 20:28, Nuno Neves wrote: > Hello, > Using this code I get an error when starting varnish 3.0 > The code: > sub vcl_fetch { > ??if (beresp.http.X-Purge) { > ?? ?purge("obj.http.host == example.com && req.url ~ " beresp.http.X-Purge); > ??} > } > > The error: > ?* Starting HTTP accelerator varnishd > ?? ...fail! > SMA.s0: max size 256 MB. > Message from VCC-compiler: > Expected ';' got '(' > (program line 174), at > ('input' Line 52 Pos 11) > ?? ?purge ("obj.http.host == example.com && req.url ~ " > beresp.http.X-Purge); > ----------#--------------------------------------------------------------------------- > What do I need to change? > Thanks > Nuno > > > A 2011/07/06, ?s 11:25, Rob S escreveu: > > On 06/07/2011 10:26, Nuno Neves wrote: > > how can I get the hiden input value I have that tell varnish that we have a > new post? > > > > I was thinking of something like this: > > sub vcl_recv { > ? ? if (req.request == "POST" && FORM.FIELD == "canpurge) { > ? ? ? ? purge req.http.host == example.com && req.url ~ > ^FORM.CANPURGE.VALUE.*$ > ? ? } > } > > Nuno: > > Varnish can't see the body of the response.? However, if you're able to > change the response, you can just add an HTTP header with this information. > Then, in vcl_fetch (which is where the HTTP request is sent to the backend), > you can probably add something like: > > sub vcl_fetch { > ? if (beresp.http.x-purgepattern) { > ??? purge("obj.http.host == example.com && req.url ~ " > beresp.http.x-purgepattern); > ? } > } > > > Rob > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > From fdylist at gmail.com Thu Jul 7 02:58:24 2011 From: fdylist at gmail.com (Fibrizo) Date: Thu, 7 Jul 2011 10:58:24 +0800 Subject: Transient Storage memleak? Message-ID: <001a01cc3c51$c1253380$436f9a80$@gmail.com> I set -s malloc,4G but varnish eat all my mem(12G), and use swap! In varnishstat , I saw SMA.s0.nbytes is 4G is right, but SMA.Transient.nbytes is almost 6G! What is SMA.Transient and why it can't be free? And how can I control transient storage size? SMA.s0.nreq 2343385 74.98 Allocator requests SMA.s0.nobj 353562 . Outstanding allocations SMA.s0.nbytes 4294967257 . Outstanding bytes SMA.s0.balloc 23949971661 . Bytes allocated SMA.s0.bfree 19655004404 . Bytes free SMA.Transient.nreq 600545 19.21 Allocator requests SMA.Transient.nobj 487027 . Outstanding allocations SMA.Transient.nbytes 6709806097 . Outstanding bytes SMA.Transient.balloc 9203517551 . Bytes allocated SMA.Transient.bfree 2493711454 . Bytes free Like this issue: https://www.varnish-cache.org/trac/ticket/953 Thank you~ -------------- next part -------------- An HTML attachment was scrubbed... URL: From lajo at kb.dk Thu Jul 7 07:44:55 2011 From: lajo at kb.dk (=?iso-8859-1?Q?Lars_J=F8rgensen?=) Date: Thu, 7 Jul 2011 07:44:55 +0000 Subject: Changing backend timeout for specific urls? Message-ID: <6D2C830A0941EA40B6B483FE6EC98ADB1874A1EA@EXCHANGE-01.kb.dk> Hi, We have a specific path that is very slow to respond ("/cop/oai"). Would it be possible to change the backend timeout to 30 minutes for just this path? Lars -------------- next part -------------- An HTML attachment was scrubbed... URL: From lajo at kb.dk Thu Jul 7 10:18:15 2011 From: lajo at kb.dk (=?iso-8859-1?Q?Lars_J=F8rgensen?=) Date: Thu, 7 Jul 2011 10:18:15 +0000 Subject: Lifting a ban Message-ID: <6D2C830A0941EA40B6B483FE6EC98ADB1874E21C@EXCHANGE-01.kb.dk> Hi, When I issue a ban in varnishadm to invalidate a series of urls in the cache, when will this ban be lifted? Can I cancel it after a few hours or will it automatically expire at some point? I fear that over time I will accumulate quite a large list of bans, thus slowing Varnish down. Lars -------------- next part -------------- An HTML attachment was scrubbed... URL: From mls at pooteeweet.org Thu Jul 7 10:26:47 2011 From: mls at pooteeweet.org (Lukas Kahwe Smith) Date: Thu, 7 Jul 2011 12:26:47 +0200 Subject: Lifting a ban In-Reply-To: <6D2C830A0941EA40B6B483FE6EC98ADB1874E21C@EXCHANGE-01.kb.dk> References: <6D2C830A0941EA40B6B483FE6EC98ADB1874E21C@EXCHANGE-01.kb.dk> Message-ID: <264F8936-D8EC-431D-A2E8-60022C205C90@pooteeweet.org> afaik there is a separate thread that cleans up the ban list Lukas On 07.07.2011, at 12:18, Lars J?rgensen wrote: > Hi, > > > > When I issue a ban in varnishadm to invalidate a series of urls in the cache, when will this ban be lifted? Can I cancel it after a few hours or will it automatically expire at some point? > > > > I fear that over time I will accumulate quite a large list of bans, thus slowing Varnish down. > > > > > > Lars > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc -------------- next part -------------- An HTML attachment was scrubbed... URL: From phk at phk.freebsd.dk Thu Jul 7 10:35:10 2011 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Thu, 07 Jul 2011 10:35:10 +0000 Subject: Lifting a ban In-Reply-To: Your message of "Thu, 07 Jul 2011 10:18:15 GMT." <6D2C830A0941EA40B6B483FE6EC98ADB1874E21C@EXCHANGE-01.kb.dk> Message-ID: <5140.1310034910@critter.freebsd.dk> In message <6D2C830A0941EA40B6B483FE6EC98ADB1874E21C at EXCHANGE-01.kb.dk>, =?iso- 8859-1?Q?Lars_J=F8rgensen?= writes: >When I issue a ban in varnishadm to invalidate a series of urls in the cache, >when will this ban be lifted? When all objects in the cache at the time you issued the ban has either been tested against it, or expired. We have a "ban-lurker" thread which tries to speed this process up by testing objects against the bans, but since there is no request available, this thread can only test obj.* conditions, but not req.* conditions. One way to work around this limitation is to copy the fields you want to test from req into obj: sub vcl_fetch { // copy url to object fo ban lurker set beresp.http.XYZ-url = req.url; } sub vcl_deliver { // don't deliver url to client unset resp.http.XYZ-url; } ban obj.http.xyz-url ~ "*.jpg" -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From flatcircle at hotmail.com Thu Jul 7 15:29:44 2011 From: flatcircle at hotmail.com (Flatcircle) Date: Thu, 7 Jul 2011 17:29:44 +0200 Subject: Drupal 7.x & Varnish 3.x In-Reply-To: <18834F5BEC10824891FB8B22AC821A5A01735124@nucleus-srv01.Nucleus.local> References: <18834F5BEC10824891FB8B22AC821A5A01735072@nucleus-srv01.Nucleus.local> <18834F5BEC10824891FB8B22AC821A5A01735124@nucleus-srv01.Nucleus.local> Message-ID: An HTML attachment was scrubbed... URL: From perbu at varnish-software.com Thu Jul 7 17:13:35 2011 From: perbu at varnish-software.com (Per Buer) Date: Thu, 7 Jul 2011 19:13:35 +0200 Subject: Changing backend timeout for specific urls? In-Reply-To: <6D2C830A0941EA40B6B483FE6EC98ADB1874A1EA@EXCHANGE-01.kb.dk> References: <6D2C830A0941EA40B6B483FE6EC98ADB1874A1EA@EXCHANGE-01.kb.dk> Message-ID: Hi Lars, On Thu, Jul 7, 2011 at 9:44 AM, Lars J?rgensen wrote: We have a specific path that is very slow to respond ("/cop/oai"). Would it > be possible to change the backend timeout to 30 minutes for just this path? > Yes. Just define a new backend pointing to the same server(s) but with different timeouts. Then send requests to this backend in recv. Per. -- Per Buer, CEO Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer *Varnish makes websites fly!* Whitepapers | Video | Twitter -------------- next part -------------- An HTML attachment was scrubbed... URL: From isharov at yandex-team.ru Thu Jul 7 18:51:23 2011 From: isharov at yandex-team.ru (Iliya Sharov) Date: Thu, 07 Jul 2011 22:51:23 +0400 Subject: Transient Storage memleak? In-Reply-To: <001a01cc3c51$c1253380$436f9a80$@gmail.com> References: <001a01cc3c51$c1253380$436f9a80$@gmail.com> Message-ID: <4E16002B.7000208@yandex-team.ru> I can't use varnish3.0 due SMA.Transient eat all my server's memory too :-( Iliya Sharov 07.07.2011 06:58, Fibrizo ?????: > > I set --s malloc,4G but varnish eat all my mem(12G), and use swap! > > In varnishstat , I saw SMA.s0.nbytes is 4G is right, but > SMA.Transient.nbytes is almost 6G! > > What is SMA.Transient and why it can't be free? > > And how can I control transient storage size? > > SMA.s0.nreq 2343385 74.98 Allocator requests > > SMA.s0.nobj 353562 . Outstanding allocations > > SMA.s0.nbytes 4294967257 . Outstanding bytes > > SMA.s0.balloc 23949971661 . Bytes allocated > > SMA.s0.bfree 19655004404 . Bytes free > > SMA.Transient.nreq 600545 19.21 Allocator requests > > SMA.Transient.nobj 487027 . Outstanding allocations > > SMA.Transient.nbytes 6709806097 . Outstanding bytes > > SMA.Transient.balloc 9203517551 . Bytes allocated > > SMA.Transient.bfree 2493711454 . Bytes free > > Like this issue: https://www.varnish-cache.org/trac/ticket/953 > > Thank you~ > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc -------------- next part -------------- An HTML attachment was scrubbed... URL: From btruitt at rackspace.com Thu Jul 7 19:45:46 2011 From: btruitt at rackspace.com (Ben Truitt) Date: Thu, 7 Jul 2011 19:45:46 +0000 Subject: Lifting a ban In-Reply-To: <5140.1310034910@critter.freebsd.dk> References: <5140.1310034910@critter.freebsd.dk> Message-ID: Here is a more comprehensive explanation in case it is helpful: http://kristianlyng.wordpress.com/2010/07/28/smart-bans-with-varnish/ -Ben On Jul 7, 2011, at 6:35 AM, Poul-Henning Kamp wrote: > In message <6D2C830A0941EA40B6B483FE6EC98ADB1874E21C at EXCHANGE-01.kb.dk>, =?iso- > 8859-1?Q?Lars_J=F8rgensen?= writes: > > >> When I issue a ban in varnishadm to invalidate a series of urls in the cache, >> when will this ban be lifted? > > When all objects in the cache at the time you issued the ban has either > been tested against it, or expired. > > We have a "ban-lurker" thread which tries to speed this process up > by testing objects against the bans, but since there is no request > available, this thread can only test obj.* conditions, but not > req.* conditions. > > One way to work around this limitation is to copy the fields you > want to test from req into obj: > > sub vcl_fetch { > // copy url to object fo ban lurker > set beresp.http.XYZ-url = req.url; > } > > sub vcl_deliver { > // don't deliver url to client > unset resp.http.XYZ-url; > } > > > ban obj.http.xyz-url ~ "*.jpg" > > > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > phk at FreeBSD.ORG | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetence. > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc This email may include confidential information. If you received it in error, please delete it. From listas at kurtkraut.net Thu Jul 7 21:14:15 2011 From: listas at kurtkraut.net (Kurt Kraut) Date: Thu, 7 Jul 2011 18:14:15 -0300 Subject: Does Varnish uses ETag? Message-ID: Hi, ETag (http://en.wikipedia.org/wiki/HTTP_ETag) was created to allow the verification if the content has changed without even downloading it. It would be smart to a cache like Varnish to check the ETag of URLs at backend from time to time to check if they have changed and then the cached version and wait for a client to request it, providing a newer version of a JPEG or CSS for instance. Does Varnish used ETag to check cache freshness? - If yes, how does it behave? - If not, is it possible to achive through VCL? Thanks in advance. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mattias at nucleus.be Fri Jul 8 07:28:19 2011 From: mattias at nucleus.be (Mattias Geniar) Date: Fri, 8 Jul 2011 09:28:19 +0200 Subject: Drupal 7.x & Varnish 3.x In-Reply-To: References: <18834F5BEC10824891FB8B22AC821A5A01735072@nucleus-srv01.Nucleus.local> <18834F5BEC10824891FB8B22AC821A5A01735124@nucleus-srv01.Nucleus.local> Message-ID: <18834F5BEC10824891FB8B22AC821A5A017351D2@nucleus-srv01.Nucleus.local> > When I use the code below (based on the templates at https://github.com/mattiasgeniar/varnish-3.0-configuration-templates/tre e/master/conf.d), Drupal 7 + Varnish 3 'almost' works. > > Varnish is working perfectly (no backend calls, two Varnish timestamps in the header). > However, I can't log in in my Drupal site. Same behaviour with directories which are protected by .htaccess. The logon screen keeps popping up when I enter the correct credentials. Add the following at the bottom of the vcl receive (it's in the "default.vcl" file on github): if (req.http.Authorization || req.http.Cookie) { # Not cacheable by default return (pass); } Without it, you are indeed likely to run into problems. You may want to check the rest of the content in default.vcl. Regards, Mattias From geoff at uplex.de Fri Jul 8 10:23:54 2011 From: geoff at uplex.de (Geoff Simmons) Date: Fri, 08 Jul 2011 12:23:54 +0200 Subject: Does Varnish uses ETag? In-Reply-To: References: Message-ID: <4E16DABA.2050606@uplex.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 7/7/11 11:14 PM, Kurt Kraut wrote: > > Does Varnish used ETag to check cache freshness? Mainline Varnish doesn't, but there's an experimental branch 'experimental-ims' on the source repository in which it's implemented. > - If yes, how does it behave? http://www.varnish-cache.org/trac/wiki/BackendConditionalRequests As per the RFC, freshness checks against ETags are done with conditional requests to the backend using If-None-Match. > - If not, is it possible to achive through VCL? I don't think so, because standard VCL doesn't have a way to access an object in the cache that has an expired TTL, but otherwise matches the request, and hence could be refreshed. So you don't ordinarily know an ETag in VCL that could be used for the refresh, and even if you did, and got a 304 response from the backend, there's no way to tell Varnish to deliver the stale object after all. If you get a chance to try the IMS branch, please let us know how it goes. Best, Geoff - -- UPLEX Systemoptimierung Schwanenwik 24 22087 Hamburg http://uplex.de/ Mob: +49-176-63690917 -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.14 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQIcBAEBCAAGBQJOFtq5AAoJEOUwvh9pJNURIy4QAKPiZ05kblNGfw7RFJo5Z9fg 81feHHYKRy880yJrG9up8wNx02+9uku6gNlYkpkPcQqfP/fZu5PpUtjGK7pmCSWG Jv0bBEL+j5V2L0SaQtoxyyvBkQ5bjQcmrZnuxpYRKRdZeNpjIWY1BLHjMkC3z+sS 4KKPi1aiGTKYQH8O1FZHz+6dfP8OQseUCs7tLwYM2BYF2onh5H81ys/Adp6520lv WV2cJWKJO85MqH5C0qmyzGV3ovTFPaU3J4/4idzPxW/vbUDozr5/XjloG7IYzCi6 Hj5YrfoJlqBREqdAdZ46G/cxS4dKmvMWIO8XaJIYMBISDaXQysXfLq9vLiqODD5l /1/ijFfEWihwdoxJ/xi1ZgxhA8RYuxQQCMtStVch+QiHpp8ROw04GutXFmA67JeM aWIBt9WL/cXc/YSP2geSobY89/ROulnKcqV9n5xvxGe1L2bkwq1jQJTtINF6l1oC lBb9tZAA2JGEeJi8Fv9lcjcrMiLSPzoW16rbzpQNjxxu129Rq8ntteFkm5cqM4sD 2do8CvRO3YauIY6bUGhRVSjNHCvYf6HvvoZEoKvgJy5drziKyU3qx3WEq3TF+QQU q+Em6WhiYHQufj6OGnZCFJJ4SKDNUNzBkwzuoXay6jxvUtv3m1tGJyKMK0rYN7vv 24e8axxNjKucDr5uFx1L =PW5G -----END PGP SIGNATURE----- From everton at connectedinternet.co.uk Fri Jul 8 10:38:57 2011 From: everton at connectedinternet.co.uk (Everton Blair) Date: Fri, 8 Jul 2011 11:38:57 +0100 Subject: Fwd: [Wordpress]homepage not purging In-Reply-To: References: Message-ID: Hi I've just started using Varnish with Wordpress/W3TC, and all is ok, apart from my homepage not purging when an update is made e.g. a new post. I think this might be because my homepage is using a static page www.windows7news.com/magazine-homepage/ as it's homepage and is doing some magic within Wordpress to show this page when the page www.windows7news.com/is requested i.e. the server doesn't know that www.windows7news.com/magazine-homepage/ needs purging. I thought I'd temporariliy managed to fix this by adding: sub vcl_recv { if (req.url ~ "^/") { return(pass); } } But I only managed to turn the cache off completely. Is there anyway to tell Varnish to always purge the homepage if there's a change? Thanks in advance for any help. My full VCL is below. If anyone has any Wordpress specific changes they'd like to share then please send them my way! Regards EB ----------------------------------------------------------------------------------------------------------------------------------- backend b96_30_20_4 { .host = "96.30.20.4"; .port = "8080"; .probe = { .url = "/"; .interval = 5s; .timeout = 1 s; .window = 5; .threshold = 3; } // we include time outs so uploads don't time out .connect_timeout = 600s; .first_byte_timeout = 600s; .between_bytes_timeout = 600s; } backend b96_30_20_5 { .host = "96.30.20.5"; .port = "8080"; .probe = { .url = "/"; .interval = 5s; .timeout = 1 s; .window = 5; .threshold = 3; } // we include time outs so uploads don't time out .connect_timeout = 600s; .first_byte_timeout = 600s; .between_bytes_timeout = 600s; } backend b96_30_20_6 { .host = "96.30.20.6"; .port = "8080"; .probe = { .url = "/"; .interval = 5s; .timeout = 1 s; .window = 5; .threshold = 3; } // we include time outs so uploads don't time out .connect_timeout = 600s; .first_byte_timeout = 600s; .between_bytes_timeout = 600s; } backend b96_30_20_7 { .host = "96.30.20.7"; .port = "8080"; .probe = { .url = "/"; .interval = 5s; .timeout = 1 s; .window = 5; .threshold = 3; } // we include time outs so uploads don't time out .connect_timeout = 600s; .first_byte_timeout = 600s; .between_bytes_timeout = 600s; } backend b96_30_18_15 { .host = "96.30.18.15"; .port = "8080"; .probe = { .url = "/"; .interval = 5s; .timeout = 1 s; .window = 5; .threshold = 3; } // we include time outs so uploads don't time out .connect_timeout = 600s; .first_byte_timeout = 600s; .between_bytes_timeout = 600s; } acl a96_30_20_4 { "96.30.20.4"; } acl a96_30_20_5 { "96.30.20.5"; } acl a96_30_20_6 { "96.30.20.6"; } acl a96_30_20_7 { "96.30.20.7"; } acl a96_30_18_15 { "96.30.18.15"; } acl purge { "96.30.20.4"; "96.30.20.5"; "96.30.20.6"; "96.30.20.7"; "96.30.18.15"; } sub vcl_recv { if (server.ip ~ a96_30_20_4) { set req.backend = b96_30_20_4; } if (server.ip ~ a96_30_20_5) { set req.backend = b96_30_20_5; } if (server.ip ~ a96_30_20_6) { set req.backend = b96_30_20_6; } if (server.ip ~ a96_30_20_7) { set req.backend = b96_30_20_7; } if (server.ip ~ a96_30_18_15) { set req.backend = b96_30_18_15; } } sub vcl_recv { if (req.request == "PURGE") { if (!client.ip ~ purge) { error 405 "Not allowed."; } return(lookup); } if (req.url ~ "^/") { return(pass); } if (req.url ~ "^/forum/$") { return(pass); } if (req.url ~ "^/$") { unset req.http.cookie; } } sub vcl_hit { if (req.request == "PURGE") { set obj.ttl = 0s; error 200 "Purged."; } } sub vcl_miss { if (req.request == "PURGE") { error 404 "Not in cache."; } if (!(req.url ~ "wp-(login|admin)")) { unset req.http.cookie; } if (req.url ~ "^/[^?]+.(jpeg|jpg|png|gif|ico|js|css|txt|gz|zip|lzma|bz2|tgz|tbz|html|htm)(\?.|)$") { unset req.http.cookie; set req.url = regsub(req.url, "\?.$", ""); } if (req.url ~ "^/$") { unset req.http.cookie; } } sub vcl_fetch { if (req.url ~ "^/$") { unset beresp.http.set-cookie; } if (!(req.url ~ "wp-(login|admin)")) { unset beresp.http.set-cookie; } } -------------- next part -------------- An HTML attachment was scrubbed... URL: From tfheen at varnish-software.com Fri Jul 8 10:41:30 2011 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Fri, 08 Jul 2011 12:41:30 +0200 Subject: errno = Connection reset by peer In-Reply-To: (alexus@gmail.com's message of "Thu, 30 Jun 2011 00:13:15 -0400") References: Message-ID: <8739iht5ph.fsf@qurzaw.varnish-software.com> ]] alexus | 70 Debug c "Write error, retval = -1, len = 10212, errno = | Connection reset by peer" | | other then vcl_error(404, unknown virtual host) what that other message | really mean? That the client disconnected before we were done sending the object. -- Tollef Fog Heen Varnish Software t: +47 21 98 92 64 From tfheen at varnish-software.com Fri Jul 8 10:44:45 2011 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Fri, 08 Jul 2011 12:44:45 +0200 Subject: Varnishstat question In-Reply-To: (Jonathan Hursey's message of "Fri, 1 Jul 2011 14:32:33 -0500") References: Message-ID: <87y609rqzm.fsf@qurzaw.varnish-software.com> ]] Jonathan Hursey Hi, | Greetings fellow varnish enthusiasts. I was wondering if anyone could help | me determine what would be considered good values for the follow fields in | varnishstat output and what these fields indicate: | | 1 0.00 0.00 N vcl total | 1 0.00 0.00 N vcl available This just tells you that you have one VCL loaded. | 7 . . N total active purges | 7 0.00 0.00 N new purges added You have 7 bans added | 6112 0.00 0.02 N objects tested And 6112 objects tested against those bans in total. Regards, -- Tollef Fog Heen Varnish Software t: +47 21 98 92 64 From lee at leetrout.com Fri Jul 1 00:44:26 2011 From: lee at leetrout.com (Lee Trout) Date: Thu, 30 Jun 2011 20:44:26 -0400 Subject: Possible bug with very large headers in varnish-3.0.0 revision 3bd5997 Message-ID: Hello! Please excuse the interruption, but I wanted to see if I may have stumbled upon a bug or if there is a known issue or configuration option for requests with very large header values. Using varnish-3.0.0 revision 3bd5997 when a large request comes in (cookie header ~3k characters) the header is lost. I am seeing "LostHeader c Cookie:" in the logs. This paste includes the request and results comparing varnish-3.0.0 revision 3bd5997 and varnish-2.1.4 SVN 5447M http://pastebin.com/J2Qn0KGn Thank you in advance for your time and suggestions. Lee -- Lee Trout http://www.leetrout.com From dongyu at staff.sina.com.cn Thu Jul 7 02:57:48 2011 From: dongyu at staff.sina.com.cn (=?gb2312?B?t7+2q9Pq?=) Date: Thu, 07 Jul 2011 10:57:48 +0800 Subject: Transient Storage memleak? Message-ID: <001501cc3c51$a78bdb40$f6a391c0$@staff.sina.com.cn> I set ?Cs malloc,4G but varnish eat all my mem(12G), and use swap! In varnishstat , I saw SMA.s0.nbytes is 4G is right, but SMA.Transient.nbytes is almost 6G! What is SMA.Transient and why it can??t be free? And how can I control transient storage size? SMA.s0.nreq 2343385 74.98 Allocator requests SMA.s0.nobj 353562 . Outstanding allocations SMA.s0.nbytes 4294967257 . Outstanding bytes SMA.s0.balloc 23949971661 . Bytes allocated SMA.s0.bfree 19655004404 . Bytes free SMA.Transient.nreq 600545 19.21 Allocator requests SMA.Transient.nobj 487027 . Outstanding allocations SMA.Transient.nbytes 6709806097 . Outstanding bytes SMA.Transient.balloc 9203517551 . Bytes allocated SMA.Transient.bfree 2493711454 . Bytes free Like this issue: https://www.varnish-cache.org/trac/ticket/953 Thank you~ -------------- next part -------------- An HTML attachment was scrubbed... URL: From everton at windows7news.com Fri Jul 8 10:23:48 2011 From: everton at windows7news.com (Everton Blair) Date: Fri, 8 Jul 2011 11:23:48 +0100 Subject: [Wordpress]homepage not purging Message-ID: Hi I've just started using Varnish with Wordpress/W3TC, and all is ok, apart from my homepage not purging when an update is made e.g. a new post. I think this might be because my homepage is using a static page www.windows7news.com/magazine-homepage/ as it's homepage and is doing some magic within Wordpress to show this page when the page www.windows7news.com/is requested i.e. the server doesn't know that www.windows7news.com/magazine-homepage/ needs purging. I seem to have temporariliy managed to fix this by adding: sub vcl_recv { if (req.url ~ "^/") { return(pass); } } But this means my most trafficked page isn't getting cached. Is there anyway to tell Varnish to always purge the homepage if there's a change? Thanks in advance for any help. My full VCL is below. If anyone has any Wordpress specific changes they'd like to share then please send them my way! Regards EB ----------------------------------------------------------------------------------------------------------------------------------- backend b96_30_20_4 { .host = "96.30.20.4"; .port = "8080"; .probe = { .url = "/"; .interval = 5s; .timeout = 1 s; .window = 5; .threshold = 3; } // we include time outs so uploads don't time out .connect_timeout = 600s; .first_byte_timeout = 600s; .between_bytes_timeout = 600s; } backend b96_30_20_5 { .host = "96.30.20.5"; .port = "8080"; .probe = { .url = "/"; .interval = 5s; .timeout = 1 s; .window = 5; .threshold = 3; } // we include time outs so uploads don't time out .connect_timeout = 600s; .first_byte_timeout = 600s; .between_bytes_timeout = 600s; } backend b96_30_20_6 { .host = "96.30.20.6"; .port = "8080"; .probe = { .url = "/"; .interval = 5s; .timeout = 1 s; .window = 5; .threshold = 3; } // we include time outs so uploads don't time out .connect_timeout = 600s; .first_byte_timeout = 600s; .between_bytes_timeout = 600s; } backend b96_30_20_7 { .host = "96.30.20.7"; .port = "8080"; .probe = { .url = "/"; .interval = 5s; .timeout = 1 s; .window = 5; .threshold = 3; } // we include time outs so uploads don't time out .connect_timeout = 600s; .first_byte_timeout = 600s; .between_bytes_timeout = 600s; } backend b96_30_18_15 { .host = "96.30.18.15"; .port = "8080"; .probe = { .url = "/"; .interval = 5s; .timeout = 1 s; .window = 5; .threshold = 3; } // we include time outs so uploads don't time out .connect_timeout = 600s; .first_byte_timeout = 600s; .between_bytes_timeout = 600s; } acl a96_30_20_4 { "96.30.20.4"; } acl a96_30_20_5 { "96.30.20.5"; } acl a96_30_20_6 { "96.30.20.6"; } acl a96_30_20_7 { "96.30.20.7"; } acl a96_30_18_15 { "96.30.18.15"; } acl purge { "96.30.20.4"; "96.30.20.5"; "96.30.20.6"; "96.30.20.7"; "96.30.18.15"; } sub vcl_recv { if (server.ip ~ a96_30_20_4) { set req.backend = b96_30_20_4; } if (server.ip ~ a96_30_20_5) { set req.backend = b96_30_20_5; } if (server.ip ~ a96_30_20_6) { set req.backend = b96_30_20_6; } if (server.ip ~ a96_30_20_7) { set req.backend = b96_30_20_7; } if (server.ip ~ a96_30_18_15) { set req.backend = b96_30_18_15; } } sub vcl_recv { if (req.request == "PURGE") { if (!client.ip ~ purge) { error 405 "Not allowed."; } return(lookup); } if (req.url ~ "^/") { return(pass); } if (req.url ~ "^/forum/$") { return(pass); } if (req.url ~ "^/$") { unset req.http.cookie; } } sub vcl_hit { if (req.request == "PURGE") { set obj.ttl = 0s; error 200 "Purged."; } } sub vcl_miss { if (req.request == "PURGE") { error 404 "Not in cache."; } if (!(req.url ~ "wp-(login|admin)")) { unset req.http.cookie; } if (req.url ~ "^/[^?]+.(jpeg|jpg|png|gif|ico|js|css|txt|gz|zip|lzma|bz2|tgz|tbz|html|htm)(\?.|)$") { unset req.http.cookie; set req.url = regsub(req.url, "\?.$", ""); } if (req.url ~ "^/$") { unset req.http.cookie; } } sub vcl_fetch { if (req.url ~ "^/$") { unset beresp.http.set-cookie; } if (!(req.url ~ "wp-(login|admin)")) { unset beresp.http.set-cookie; } } -------------- next part -------------- An HTML attachment was scrubbed... URL: From tfheen at varnish-software.com Fri Jul 8 10:54:31 2011 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Fri, 08 Jul 2011 12:54:31 +0200 Subject: Changing backend timeout for specific urls? In-Reply-To: <6D2C830A0941EA40B6B483FE6EC98ADB1874A1EA@EXCHANGE-01.kb.dk> ("Lars =?utf-8?Q?J=C3=B8rgensen=22's?= message of "Thu, 7 Jul 2011 07:44:55 +0000") References: <6D2C830A0941EA40B6B483FE6EC98ADB1874A1EA@EXCHANGE-01.kb.dk> Message-ID: <87tyaxrqjc.fsf@qurzaw.varnish-software.com> ]] Lars J?rgensen | We have a specific path that is very slow to respond ("/cop/oai"). Would it be possible to change the backend timeout to 30 minutes for just this path? sub vcl_miss { if (req.url == "/cop/oai") { set bereq.first_byte_timeout = 31m; } } -- Tollef Fog Heen Varnish Software t: +47 21 98 92 64 From jonathan.hursey at adrevolution.com Fri Jul 8 13:36:42 2011 From: jonathan.hursey at adrevolution.com (Jonathan Hursey) Date: Fri, 8 Jul 2011 08:36:42 -0500 Subject: Varnishstat question In-Reply-To: <87y609rqzm.fsf@qurzaw.varnish-software.com> References: <87y609rqzm.fsf@qurzaw.varnish-software.com> Message-ID: Awesome thanks sooo much! On Fri, Jul 8, 2011 at 5:44 AM, Tollef Fog Heen wrote: > ]] Jonathan Hursey > > Hi, > > | Greetings fellow varnish enthusiasts. I was wondering if anyone could > help > | me determine what would be considered good values for the follow fields > in > | varnishstat output and what these fields indicate: > | > | 1 0.00 0.00 N vcl total > | 1 0.00 0.00 N vcl available > > This just tells you that you have one VCL loaded. > > | 7 . . N total active purges > | 7 0.00 0.00 N new purges added > > You have 7 bans added > > | 6112 0.00 0.02 N objects tested > > And 6112 objects tested against those bans in total. > > Regards, > -- > Tollef Fog Heen > Varnish Software > t: +47 21 98 92 64 > -- *Jonathan M. Hursey* *Linux Systems Administrator* * * -------------- next part -------------- An HTML attachment was scrubbed... URL: From flatcircle at hotmail.com Fri Jul 8 18:26:18 2011 From: flatcircle at hotmail.com (Flatcircle) Date: Fri, 8 Jul 2011 20:26:18 +0200 Subject: Drupal 7.x & Varnish 3.x In-Reply-To: <18834F5BEC10824891FB8B22AC821A5A017351D2@nucleus-srv01.Nucleus.local> References: <18834F5BEC10824891FB8B22AC821A5A01735072@nucleus-srv01.Nucleus.local> <18834F5BEC10824891FB8B22AC821A5A01735124@nucleus-srv01.Nucleus.local> <18834F5BEC10824891FB8B22AC821A5A017351D2@nucleus-srv01.Nucleus.local> Message-ID: An HTML attachment was scrubbed... URL: From scott at dor.ky Fri Jul 8 18:37:38 2011 From: scott at dor.ky (Scott Wilcox) Date: Fri, 8 Jul 2011 19:37:38 +0100 Subject: Varnish + Apache + SSL + Pound? Message-ID: <2F426C39-48D0-4D7A-A14C-4BBCA9391837@dor.ky> Hello folks, I've seen this and similar questions asked often and with varying replies. I'd like to have my own scenario looked at. I'm currently in the process of redesigning my work flow for one of my projects. The main web server that serves content is Apache. I plan to have Varnish caching this traffic where applicable. I would like to add a layer of redundancy in regards to alerting users of problems too. I also need to support SSL too. I'm looking at having pound accept connections which is then passed to varnish. Varnish then handles the traffic as normal. I've seen in v3 that basic HTML can be output for errors, but I'd like to include a stylesheet and logo too. I can host these off the site if needed, but I've considered having 'pretty' error messages which include stylesheets and images. I'd obviously have to host these off site which isn't a problem. Is it better to use nginx to accept both HTTP and HTTPS connections and serve the errors/content or does my original plan sound better? What do you use in production and why? Do you recommend pound or nginx? Many thanks for replies in advance. -- Scott Wilcox @dordotky | scott at dor.ky | http://dor.ky +44 (0) 7538 842418 | +1 (646) 827-0580 -------------- next part -------------- An HTML attachment was scrubbed... URL: From everton at connectedinternet.co.uk Fri Jul 8 19:04:40 2011 From: everton at connectedinternet.co.uk (Everton Blair) Date: Fri, 8 Jul 2011 20:04:40 +0100 Subject: [Wordpress]homepage not purging (Everton Blair) Message-ID: Can anyone help me with my problem please? It's been causing me major problems. Basically I'm using Wordpress and new posts have been forcing the homepage to purge, I think because I'm using a special template within wordpress to create the homepage and I think Varnish doesn't realise it needs to purge that page What I want to do is force varnish to purge my homepage when any purge requests come in to make sure it stays fresh and to be safe, but I can't get this to work: sub vcl_recv { if (req.request == "PURGE") { if (!client.ip ~ purge) { error 405 "Not allowed."; } if (req.url == "^windows7news.com/$") { purge(HOW DO I TELL VARNISH TO PURGE http://www.windows7news.com/magazine-homepage/ ????); } if (req.url == "^windows8news.com/$") { purge(HOW DO I TELL VARNISH TO PURGE http://www.windows8news.com/magazine-homepage/ ???); } return(lookup); } } Thanks for any help. On Fri, Jul 8, 2011 at 11:46 AM, wrote: > Send varnish-misc mailing list submissions to > varnish-misc at varnish-cache.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > or, via email, send a message with subject or body 'help' to > varnish-misc-request at varnish-cache.org > > You can reach the person managing the list at > varnish-misc-owner at varnish-cache.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of varnish-misc digest..." > > > Today's Topics: > > 1. Re: Does Varnish uses ETag? (Geoff Simmons) > 2. Fwd: [Wordpress]homepage not purging (Everton Blair) > 3. Re: errno = Connection reset by peer (Tollef Fog Heen) > 4. Re: Varnishstat question (Tollef Fog Heen) > 5. Possible bug with very large headers in varnish-3.0.0 > revision 3bd5997 (Lee Trout) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 08 Jul 2011 12:23:54 +0200 > From: Geoff Simmons > To: Kurt Kraut > Cc: varnish-misc at varnish-cache.org > Subject: Re: Does Varnish uses ETag? > Message-ID: <4E16DABA.2050606 at uplex.de> > Content-Type: text/plain; charset=ISO-8859-1 > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > On 7/7/11 11:14 PM, Kurt Kraut wrote: > > > > Does Varnish used ETag to check cache freshness? > > Mainline Varnish doesn't, but there's an experimental branch > 'experimental-ims' on the source repository in which it's implemented. > > > - If yes, how does it behave? > > http://www.varnish-cache.org/trac/wiki/BackendConditionalRequests > > As per the RFC, freshness checks against ETags are done with conditional > requests to the backend using If-None-Match. > > > - If not, is it possible to achive through VCL? > > I don't think so, because standard VCL doesn't have a way to access an > object in the cache that has an expired TTL, but otherwise matches the > request, and hence could be refreshed. > > So you don't ordinarily know an ETag in VCL that could be used for the > refresh, and even if you did, and got a 304 response from the backend, > there's no way to tell Varnish to deliver the stale object after all. > > If you get a chance to try the IMS branch, please let us know how it goes. > > > Best, > Geoff > - -- > UPLEX Systemoptimierung > Schwanenwik 24 > 22087 Hamburg > http://uplex.de/ > Mob: +49-176-63690917 > -----BEGIN PGP SIGNATURE----- > Version: GnuPG/MacGPG2 v2.0.14 (Darwin) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iQIcBAEBCAAGBQJOFtq5AAoJEOUwvh9pJNURIy4QAKPiZ05kblNGfw7RFJo5Z9fg > 81feHHYKRy880yJrG9up8wNx02+9uku6gNlYkpkPcQqfP/fZu5PpUtjGK7pmCSWG > Jv0bBEL+j5V2L0SaQtoxyyvBkQ5bjQcmrZnuxpYRKRdZeNpjIWY1BLHjMkC3z+sS > 4KKPi1aiGTKYQH8O1FZHz+6dfP8OQseUCs7tLwYM2BYF2onh5H81ys/Adp6520lv > WV2cJWKJO85MqH5C0qmyzGV3ovTFPaU3J4/4idzPxW/vbUDozr5/XjloG7IYzCi6 > Hj5YrfoJlqBREqdAdZ46G/cxS4dKmvMWIO8XaJIYMBISDaXQysXfLq9vLiqODD5l > /1/ijFfEWihwdoxJ/xi1ZgxhA8RYuxQQCMtStVch+QiHpp8ROw04GutXFmA67JeM > aWIBt9WL/cXc/YSP2geSobY89/ROulnKcqV9n5xvxGe1L2bkwq1jQJTtINF6l1oC > lBb9tZAA2JGEeJi8Fv9lcjcrMiLSPzoW16rbzpQNjxxu129Rq8ntteFkm5cqM4sD > 2do8CvRO3YauIY6bUGhRVSjNHCvYf6HvvoZEoKvgJy5drziKyU3qx3WEq3TF+QQU > q+Em6WhiYHQufj6OGnZCFJJ4SKDNUNzBkwzuoXay6jxvUtv3m1tGJyKMK0rYN7vv > 24e8axxNjKucDr5uFx1L > =PW5G > -----END PGP SIGNATURE----- > > > > ------------------------------ > > Message: 2 > Date: Fri, 8 Jul 2011 11:38:57 +0100 > From: Everton Blair > To: varnish-misc at varnish-cache.org > Subject: Fwd: [Wordpress]homepage not purging > Message-ID: > > > Content-Type: text/plain; charset="iso-8859-1" > > Hi > > I've just started using Varnish with Wordpress/W3TC, and all is ok, apart > from my homepage not purging when an update is made e.g. a new post. I > think this might be because my homepage is using a static page > www.windows7news.com/magazine-homepage/ as it's homepage and is doing some > magic within Wordpress to show this page when the page > www.windows7news.com/is requested i.e. the server doesn't know that > www.windows7news.com/magazine-homepage/ needs purging. > > I thought I'd temporariliy managed to fix this by adding: > > sub vcl_recv { > if (req.url ~ "^/") { > return(pass); > } > } > > But I only managed to turn the cache off completely. Is there anyway to > tell Varnish to always purge the homepage if there's a change? > > Thanks in advance for any help. > > My full VCL is below. If anyone has any Wordpress specific changes they'd > like to share then please send them my way! > > Regards > > EB > > > ----------------------------------------------------------------------------------------------------------------------------------- > > backend b96_30_20_4 { > .host = "96.30.20.4"; > .port = "8080"; > .probe = { > .url = "/"; > .interval = 5s; > .timeout = 1 s; > .window = 5; > .threshold = 3; > } > // we include time outs so uploads don't time out > .connect_timeout = 600s; > .first_byte_timeout = 600s; > .between_bytes_timeout = 600s; > } > > backend b96_30_20_5 { > .host = "96.30.20.5"; > .port = "8080"; > .probe = { > .url = "/"; > .interval = 5s; > .timeout = 1 s; > .window = 5; > .threshold = 3; > } > // we include time outs so uploads don't time out > .connect_timeout = 600s; > .first_byte_timeout = 600s; > .between_bytes_timeout = 600s; > } > > backend b96_30_20_6 { > .host = "96.30.20.6"; > .port = "8080"; > .probe = { > .url = "/"; > .interval = 5s; > .timeout = 1 s; > .window = 5; > .threshold = 3; > } > // we include time outs so uploads don't time out > .connect_timeout = 600s; > .first_byte_timeout = 600s; > .between_bytes_timeout = 600s; > } > > backend b96_30_20_7 { > .host = "96.30.20.7"; > .port = "8080"; > .probe = { > .url = "/"; > .interval = 5s; > .timeout = 1 s; > .window = 5; > .threshold = 3; > } > // we include time outs so uploads don't time out > .connect_timeout = 600s; > .first_byte_timeout = 600s; > .between_bytes_timeout = 600s; > } > > backend b96_30_18_15 { > .host = "96.30.18.15"; > .port = "8080"; > .probe = { > .url = "/"; > .interval = 5s; > .timeout = 1 s; > .window = 5; > .threshold = 3; > } > // we include time outs so uploads don't time out > .connect_timeout = 600s; > .first_byte_timeout = 600s; > .between_bytes_timeout = 600s; > } > > acl a96_30_20_4 { > "96.30.20.4"; > } > acl a96_30_20_5 { > "96.30.20.5"; > } > acl a96_30_20_6 { > "96.30.20.6"; > } > acl a96_30_20_7 { > "96.30.20.7"; > } > acl a96_30_18_15 { > "96.30.18.15"; > } > > acl purge { > "96.30.20.4"; > "96.30.20.5"; > "96.30.20.6"; > "96.30.20.7"; > "96.30.18.15"; > } > > sub vcl_recv { > if (server.ip ~ a96_30_20_4) { > set req.backend = b96_30_20_4; > } > if (server.ip ~ a96_30_20_5) { > set req.backend = b96_30_20_5; > } > if (server.ip ~ a96_30_20_6) { > set req.backend = b96_30_20_6; > } > if (server.ip ~ a96_30_20_7) { > set req.backend = b96_30_20_7; > } > if (server.ip ~ a96_30_18_15) { > set req.backend = b96_30_18_15; > } > } > > sub vcl_recv { > if (req.request == "PURGE") { > if (!client.ip ~ purge) { > error 405 "Not allowed."; > } > return(lookup); > } > if (req.url ~ "^/") { > return(pass); > } > if (req.url ~ "^/forum/$") { > return(pass); > } > if (req.url ~ "^/$") { > unset req.http.cookie; > } > } > sub vcl_hit { > if (req.request == "PURGE") { > set obj.ttl = 0s; > error 200 "Purged."; > } > } > sub vcl_miss { > if (req.request == "PURGE") { > error 404 "Not in cache."; > } > if (!(req.url ~ "wp-(login|admin)")) { > unset req.http.cookie; > } > if (req.url ~ > > "^/[^?]+.(jpeg|jpg|png|gif|ico|js|css|txt|gz|zip|lzma|bz2|tgz|tbz|html|htm)(\?.|)$") > { > unset req.http.cookie; > set req.url = regsub(req.url, "\?.$", ""); > } > if (req.url ~ "^/$") { > unset req.http.cookie; > } > } > sub vcl_fetch { > if (req.url ~ "^/$") { > unset beresp.http.set-cookie; > } > if (!(req.url ~ "wp-(login|admin)")) { > unset beresp.http.set-cookie; > } > } > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: < > https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20110708/bf4bd761/attachment-0001.html > > > > ------------------------------ > > Message: 3 > Date: Fri, 08 Jul 2011 12:41:30 +0200 > From: Tollef Fog Heen > To: google at alexus.org > Cc: varnish-misc at varnish-cache.org > Subject: Re: errno = Connection reset by peer > Message-ID: <8739iht5ph.fsf at qurzaw.varnish-software.com> > Content-Type: text/plain; charset=us-ascii > > ]] alexus > > | 70 Debug c "Write error, retval = -1, len = 10212, errno = > | Connection reset by peer" > | > | other then vcl_error(404, unknown virtual host) what that other message > | really mean? > > That the client disconnected before we were done sending the object. > > -- > Tollef Fog Heen > Varnish Software > t: +47 21 98 92 64 > > > > ------------------------------ > > Message: 4 > Date: Fri, 08 Jul 2011 12:44:45 +0200 > From: Tollef Fog Heen > To: Jonathan Hursey > Cc: varnish-misc at varnish-cache.org > Subject: Re: Varnishstat question > Message-ID: <87y609rqzm.fsf at qurzaw.varnish-software.com> > Content-Type: text/plain; charset=us-ascii > > ]] Jonathan Hursey > > Hi, > > | Greetings fellow varnish enthusiasts. I was wondering if anyone could > help > | me determine what would be considered good values for the follow fields > in > | varnishstat output and what these fields indicate: > | > | 1 0.00 0.00 N vcl total > | 1 0.00 0.00 N vcl available > > This just tells you that you have one VCL loaded. > > | 7 . . N total active purges > | 7 0.00 0.00 N new purges added > > You have 7 bans added > > | 6112 0.00 0.02 N objects tested > > And 6112 objects tested against those bans in total. > > Regards, > -- > Tollef Fog Heen > Varnish Software > t: +47 21 98 92 64 > > > > ------------------------------ > > Message: 5 > Date: Thu, 30 Jun 2011 20:44:26 -0400 > From: Lee Trout > To: varnish-misc at varnish-cache.org > Subject: Possible bug with very large headers in varnish-3.0.0 > revision 3bd5997 > Message-ID: > Content-Type: text/plain; charset=ISO-8859-1 > > Hello! > > Please excuse the interruption, but I wanted to see if I may have > stumbled upon a bug or if there is a known issue or configuration > option for requests with very large header values. > > Using varnish-3.0.0 revision 3bd5997 when a large request comes in > (cookie header ~3k characters) the header is lost. I am seeing > "LostHeader c Cookie:" in the logs. > > This paste includes the request and results comparing varnish-3.0.0 > revision 3bd5997 and varnish-2.1.4 SVN 5447M > http://pastebin.com/J2Qn0KGn > > Thank you in advance for your time and suggestions. > > Lee > -- > Lee Trout > http://www.leetrout.com > > > > ------------------------------ > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > End of varnish-misc Digest, Vol 64, Issue 13 > ******************************************** > -------------- next part -------------- An HTML attachment was scrubbed... URL: From everton at connectedinternet.co.uk Fri Jul 8 19:14:46 2011 From: everton at connectedinternet.co.uk (Everton Blair) Date: Fri, 8 Jul 2011 20:14:46 +0100 Subject: [Wordpress]homepage not purging (Everton Blair) Message-ID: sorry, I meant to say new posts ARE NOT forcing the homepage to purge/update, so I need a way to get the homepage to update whenever any purge request comes in. On Fri, Jul 8, 2011 at 8:05 PM, wrote: > Send varnish-misc mailing list submissions to > varnish-misc at varnish-cache.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > or, via email, send a message with subject or body 'help' to > varnish-misc-request at varnish-cache.org > > You can reach the person managing the list at > varnish-misc-owner at varnish-cache.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of varnish-misc digest..." > > > Today's Topics: > > 1. Re: Drupal 7.x & Varnish 3.x (Flatcircle) > 2. Varnish + Apache + SSL + Pound? (Scott Wilcox) > 3. Re:[Wordpress]homepage not purging (Everton Blair) (Everton Blair) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 8 Jul 2011 20:26:18 +0200 > From: Flatcircle > To: Mattias Geniar > Cc: varnish-misc at varnish-cache.org > Subject: Re: Drupal 7.x & Varnish 3.x > Message-ID: > Content-Type: text/plain; charset="us-ascii" > > An HTML attachment was scrubbed... > URL: < > https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20110708/7af63846/attachment-0001.html > > > > ------------------------------ > > Message: 2 > Date: Fri, 8 Jul 2011 19:37:38 +0100 > From: Scott Wilcox > To: > Subject: Varnish + Apache + SSL + Pound? > Message-ID: <2F426C39-48D0-4D7A-A14C-4BBCA9391837 at dor.ky> > Content-Type: text/plain; charset="us-ascii" > > Hello folks, > > I've seen this and similar questions asked often and with varying replies. > I'd like to have my own scenario looked at. I'm currently in the process of > redesigning my work flow for one of my projects. The main web server that > serves content is Apache. I plan to have Varnish caching this traffic where > applicable. I would like to add a layer of redundancy in regards to alerting > users of problems too. I also need to support SSL too. > > I'm looking at having pound accept connections which is then passed to > varnish. Varnish then handles the traffic as normal. > > I've seen in v3 that basic HTML can be output for errors, but I'd like to > include a stylesheet and logo too. I can host these off the site if needed, > but I've considered having 'pretty' error messages which include stylesheets > and images. I'd obviously have to host these off site which isn't a problem. > > Is it better to use nginx to accept both HTTP and HTTPS connections and > serve the errors/content or does my original plan sound better? > > What do you use in production and why? > > Do you recommend pound or nginx? > > Many thanks for replies in advance. > > -- > Scott Wilcox > > @dordotky | scott at dor.ky | http://dor.ky > +44 (0) 7538 842418 | +1 (646) 827-0580 > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: < > https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20110708/ced5f362/attachment-0001.html > > > > ------------------------------ > > Message: 3 > Date: Fri, 8 Jul 2011 20:04:40 +0100 > From: Everton Blair > To: varnish-misc at varnish-cache.org > Subject: Re:[Wordpress]homepage not purging (Everton Blair) > Message-ID: > > > Content-Type: text/plain; charset="iso-8859-1" > > Can anyone help me with my problem please? It's been causing me major > problems. Basically I'm using Wordpress and new posts have been forcing > the > homepage to purge, I think because I'm using a special template within > wordpress to create the homepage and I think Varnish doesn't realise it > needs to purge that page > > What I want to do is force varnish to purge my homepage when any purge > requests come in to make sure it stays fresh and to be safe, but I can't > get > this to work: > > sub vcl_recv { > if (req.request == "PURGE") { > if (!client.ip ~ purge) { > error 405 "Not allowed."; > } > if (req.url == "^windows7news.com/$") { > purge(HOW DO I TELL VARNISH TO PURGE > http://www.windows7news.com/magazine-homepage/ ????); > } > if (req.url == "^windows8news.com/$") { > purge(HOW DO I TELL VARNISH TO PURGE > http://www.windows8news.com/magazine-homepage/ ???); > } > return(lookup); > } > } > > Thanks for any help. > > > On Fri, Jul 8, 2011 at 11:46 AM, >wrote: > > > Send varnish-misc mailing list submissions to > > varnish-misc at varnish-cache.org > > > > To subscribe or unsubscribe via the World Wide Web, visit > > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > or, via email, send a message with subject or body 'help' to > > varnish-misc-request at varnish-cache.org > > > > You can reach the person managing the list at > > varnish-misc-owner at varnish-cache.org > > > > When replying, please edit your Subject line so it is more specific > > than "Re: Contents of varnish-misc digest..." > > > > > > Today's Topics: > > > > 1. Re: Does Varnish uses ETag? (Geoff Simmons) > > 2. Fwd: [Wordpress]homepage not purging (Everton Blair) > > 3. Re: errno = Connection reset by peer (Tollef Fog Heen) > > 4. Re: Varnishstat question (Tollef Fog Heen) > > 5. Possible bug with very large headers in varnish-3.0.0 > > revision 3bd5997 (Lee Trout) > > > > > > ---------------------------------------------------------------------- > > > > Message: 1 > > Date: Fri, 08 Jul 2011 12:23:54 +0200 > > From: Geoff Simmons > > To: Kurt Kraut > > Cc: varnish-misc at varnish-cache.org > > Subject: Re: Does Varnish uses ETag? > > Message-ID: <4E16DABA.2050606 at uplex.de> > > Content-Type: text/plain; charset=ISO-8859-1 > > > > -----BEGIN PGP SIGNED MESSAGE----- > > Hash: SHA256 > > > > On 7/7/11 11:14 PM, Kurt Kraut wrote: > > > > > > Does Varnish used ETag to check cache freshness? > > > > Mainline Varnish doesn't, but there's an experimental branch > > 'experimental-ims' on the source repository in which it's implemented. > > > > > - If yes, how does it behave? > > > > http://www.varnish-cache.org/trac/wiki/BackendConditionalRequests > > > > As per the RFC, freshness checks against ETags are done with conditional > > requests to the backend using If-None-Match. > > > > > - If not, is it possible to achive through VCL? > > > > I don't think so, because standard VCL doesn't have a way to access an > > object in the cache that has an expired TTL, but otherwise matches the > > request, and hence could be refreshed. > > > > So you don't ordinarily know an ETag in VCL that could be used for the > > refresh, and even if you did, and got a 304 response from the backend, > > there's no way to tell Varnish to deliver the stale object after all. > > > > If you get a chance to try the IMS branch, please let us know how it > goes. > > > > > > Best, > > Geoff > > - -- > > UPLEX Systemoptimierung > > Schwanenwik 24 > > 22087 Hamburg > > http://uplex.de/ > > Mob: +49-176-63690917 > > -----BEGIN PGP SIGNATURE----- > > Version: GnuPG/MacGPG2 v2.0.14 (Darwin) > > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > > > iQIcBAEBCAAGBQJOFtq5AAoJEOUwvh9pJNURIy4QAKPiZ05kblNGfw7RFJo5Z9fg > > 81feHHYKRy880yJrG9up8wNx02+9uku6gNlYkpkPcQqfP/fZu5PpUtjGK7pmCSWG > > Jv0bBEL+j5V2L0SaQtoxyyvBkQ5bjQcmrZnuxpYRKRdZeNpjIWY1BLHjMkC3z+sS > > 4KKPi1aiGTKYQH8O1FZHz+6dfP8OQseUCs7tLwYM2BYF2onh5H81ys/Adp6520lv > > WV2cJWKJO85MqH5C0qmyzGV3ovTFPaU3J4/4idzPxW/vbUDozr5/XjloG7IYzCi6 > > Hj5YrfoJlqBREqdAdZ46G/cxS4dKmvMWIO8XaJIYMBISDaXQysXfLq9vLiqODD5l > > /1/ijFfEWihwdoxJ/xi1ZgxhA8RYuxQQCMtStVch+QiHpp8ROw04GutXFmA67JeM > > aWIBt9WL/cXc/YSP2geSobY89/ROulnKcqV9n5xvxGe1L2bkwq1jQJTtINF6l1oC > > lBb9tZAA2JGEeJi8Fv9lcjcrMiLSPzoW16rbzpQNjxxu129Rq8ntteFkm5cqM4sD > > 2do8CvRO3YauIY6bUGhRVSjNHCvYf6HvvoZEoKvgJy5drziKyU3qx3WEq3TF+QQU > > q+Em6WhiYHQufj6OGnZCFJJ4SKDNUNzBkwzuoXay6jxvUtv3m1tGJyKMK0rYN7vv > > 24e8axxNjKucDr5uFx1L > > =PW5G > > -----END PGP SIGNATURE----- > > > > > > > > ------------------------------ > > > > Message: 2 > > Date: Fri, 8 Jul 2011 11:38:57 +0100 > > From: Everton Blair > > To: varnish-misc at varnish-cache.org > > Subject: Fwd: [Wordpress]homepage not purging > > Message-ID: > > CCDxm4Xw at mail.gmail.com > > > > > Content-Type: text/plain; charset="iso-8859-1" > > > > Hi > > > > I've just started using Varnish with Wordpress/W3TC, and all is ok, apart > > from my homepage not purging when an update is made e.g. a new post. I > > think this might be because my homepage is using a static page > > www.windows7news.com/magazine-homepage/ as it's homepage and is doing > some > > magic within Wordpress to show this page when the page > > www.windows7news.com/is requested i.e. the server doesn't know that > > www.windows7news.com/magazine-homepage/ needs purging. > > > > I thought I'd temporariliy managed to fix this by adding: > > > > sub vcl_recv { > > if (req.url ~ "^/") { > > return(pass); > > } > > } > > > > But I only managed to turn the cache off completely. Is there anyway to > > tell Varnish to always purge the homepage if there's a change? > > > > Thanks in advance for any help. > > > > My full VCL is below. If anyone has any Wordpress specific changes > they'd > > like to share then please send them my way! > > > > Regards > > > > EB > > > > > > > ----------------------------------------------------------------------------------------------------------------------------------- > > > > backend b96_30_20_4 { > > .host = "96.30.20.4"; > > .port = "8080"; > > .probe = { > > .url = "/"; > > .interval = 5s; > > .timeout = 1 s; > > .window = 5; > > .threshold = 3; > > } > > // we include time outs so uploads don't time out > > .connect_timeout = 600s; > > .first_byte_timeout = 600s; > > .between_bytes_timeout = 600s; > > } > > > > backend b96_30_20_5 { > > .host = "96.30.20.5"; > > .port = "8080"; > > .probe = { > > .url = "/"; > > .interval = 5s; > > .timeout = 1 s; > > .window = 5; > > .threshold = 3; > > } > > // we include time outs so uploads don't time out > > .connect_timeout = 600s; > > .first_byte_timeout = 600s; > > .between_bytes_timeout = 600s; > > } > > > > backend b96_30_20_6 { > > .host = "96.30.20.6"; > > .port = "8080"; > > .probe = { > > .url = "/"; > > .interval = 5s; > > .timeout = 1 s; > > .window = 5; > > .threshold = 3; > > } > > // we include time outs so uploads don't time out > > .connect_timeout = 600s; > > .first_byte_timeout = 600s; > > .between_bytes_timeout = 600s; > > } > > > > backend b96_30_20_7 { > > .host = "96.30.20.7"; > > .port = "8080"; > > .probe = { > > .url = "/"; > > .interval = 5s; > > .timeout = 1 s; > > .window = 5; > > .threshold = 3; > > } > > // we include time outs so uploads don't time out > > .connect_timeout = 600s; > > .first_byte_timeout = 600s; > > .between_bytes_timeout = 600s; > > } > > > > backend b96_30_18_15 { > > .host = "96.30.18.15"; > > .port = "8080"; > > .probe = { > > .url = "/"; > > .interval = 5s; > > .timeout = 1 s; > > .window = 5; > > .threshold = 3; > > } > > // we include time outs so uploads don't time out > > .connect_timeout = 600s; > > .first_byte_timeout = 600s; > > .between_bytes_timeout = 600s; > > } > > > > acl a96_30_20_4 { > > "96.30.20.4"; > > } > > acl a96_30_20_5 { > > "96.30.20.5"; > > } > > acl a96_30_20_6 { > > "96.30.20.6"; > > } > > acl a96_30_20_7 { > > "96.30.20.7"; > > } > > acl a96_30_18_15 { > > "96.30.18.15"; > > } > > > > acl purge { > > "96.30.20.4"; > > "96.30.20.5"; > > "96.30.20.6"; > > "96.30.20.7"; > > "96.30.18.15"; > > } > > > > sub vcl_recv { > > if (server.ip ~ a96_30_20_4) { > > set req.backend = b96_30_20_4; > > } > > if (server.ip ~ a96_30_20_5) { > > set req.backend = b96_30_20_5; > > } > > if (server.ip ~ a96_30_20_6) { > > set req.backend = b96_30_20_6; > > } > > if (server.ip ~ a96_30_20_7) { > > set req.backend = b96_30_20_7; > > } > > if (server.ip ~ a96_30_18_15) { > > set req.backend = b96_30_18_15; > > } > > } > > > > sub vcl_recv { > > if (req.request == "PURGE") { > > if (!client.ip ~ purge) { > > error 405 "Not allowed."; > > } > > return(lookup); > > } > > if (req.url ~ "^/") { > > return(pass); > > } > > if (req.url ~ "^/forum/$") { > > return(pass); > > } > > if (req.url ~ "^/$") { > > unset req.http.cookie; > > } > > } > > sub vcl_hit { > > if (req.request == "PURGE") { > > set obj.ttl = 0s; > > error 200 "Purged."; > > } > > } > > sub vcl_miss { > > if (req.request == "PURGE") { > > error 404 "Not in cache."; > > } > > if (!(req.url ~ "wp-(login|admin)")) { > > unset req.http.cookie; > > } > > if (req.url ~ > > > > > "^/[^?]+.(jpeg|jpg|png|gif|ico|js|css|txt|gz|zip|lzma|bz2|tgz|tbz|html|htm)(\?.|)$") > > { > > unset req.http.cookie; > > set req.url = regsub(req.url, "\?.$", ""); > > } > > if (req.url ~ "^/$") { > > unset req.http.cookie; > > } > > } > > sub vcl_fetch { > > if (req.url ~ "^/$") { > > unset beresp.http.set-cookie; > > } > > if (!(req.url ~ "wp-(login|admin)")) { > > unset beresp.http.set-cookie; > > } > > } > > -------------- next part -------------- > > An HTML attachment was scrubbed... > > URL: < > > > https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20110708/bf4bd761/attachment-0001.html > > > > > > > ------------------------------ > > > > Message: 3 > > Date: Fri, 08 Jul 2011 12:41:30 +0200 > > From: Tollef Fog Heen > > To: google at alexus.org > > Cc: varnish-misc at varnish-cache.org > > Subject: Re: errno = Connection reset by peer > > Message-ID: <8739iht5ph.fsf at qurzaw.varnish-software.com> > > Content-Type: text/plain; charset=us-ascii > > > > ]] alexus > > > > | 70 Debug c "Write error, retval = -1, len = 10212, errno = > > | Connection reset by peer" > > | > > | other then vcl_error(404, unknown virtual host) what that other message > > | really mean? > > > > That the client disconnected before we were done sending the object. > > > > -- > > Tollef Fog Heen > > Varnish Software > > t: +47 21 98 92 64 > > > > > > > > ------------------------------ > > > > Message: 4 > > Date: Fri, 08 Jul 2011 12:44:45 +0200 > > From: Tollef Fog Heen > > To: Jonathan Hursey > > Cc: varnish-misc at varnish-cache.org > > Subject: Re: Varnishstat question > > Message-ID: <87y609rqzm.fsf at qurzaw.varnish-software.com> > > Content-Type: text/plain; charset=us-ascii > > > > ]] Jonathan Hursey > > > > Hi, > > > > | Greetings fellow varnish enthusiasts. I was wondering if anyone could > > help > > | me determine what would be considered good values for the follow fields > > in > > | varnishstat output and what these fields indicate: > > | > > | 1 0.00 0.00 N vcl total > > | 1 0.00 0.00 N vcl available > > > > This just tells you that you have one VCL loaded. > > > > | 7 . . N total active purges > > | 7 0.00 0.00 N new purges added > > > > You have 7 bans added > > > > | 6112 0.00 0.02 N objects tested > > > > And 6112 objects tested against those bans in total. > > > > Regards, > > -- > > Tollef Fog Heen > > Varnish Software > > t: +47 21 98 92 64 > > > > > > > > ------------------------------ > > > > Message: 5 > > Date: Thu, 30 Jun 2011 20:44:26 -0400 > > From: Lee Trout > > To: varnish-misc at varnish-cache.org > > Subject: Possible bug with very large headers in varnish-3.0.0 > > revision 3bd5997 > > Message-ID: > > Content-Type: text/plain; charset=ISO-8859-1 > > > > Hello! > > > > Please excuse the interruption, but I wanted to see if I may have > > stumbled upon a bug or if there is a known issue or configuration > > option for requests with very large header values. > > > > Using varnish-3.0.0 revision 3bd5997 when a large request comes in > > (cookie header ~3k characters) the header is lost. I am seeing > > "LostHeader c Cookie:" in the logs. > > > > This paste includes the request and results comparing varnish-3.0.0 > > revision 3bd5997 and varnish-2.1.4 SVN 5447M > > http://pastebin.com/J2Qn0KGn > > > > Thank you in advance for your time and suggestions. > > > > Lee > > -- > > Lee Trout > > http://www.leetrout.com > > > > > > > > ------------------------------ > > > > _______________________________________________ > > varnish-misc mailing list > > varnish-misc at varnish-cache.org > > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > > > End of varnish-misc Digest, Vol 64, Issue 13 > > ******************************************** > > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: < > https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20110708/c40bf9ed/attachment.html > > > > ------------------------------ > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > End of varnish-misc Digest, Vol 64, Issue 15 > ******************************************** > -------------- next part -------------- An HTML attachment was scrubbed... URL: From stewsnooze at gmail.com Fri Jul 8 19:49:05 2011 From: stewsnooze at gmail.com (Stewart Robinson) Date: Fri, 8 Jul 2011 20:49:05 +0100 Subject: [Wordpress]homepage not purging (Everton Blair) In-Reply-To: References: Message-ID: <603466790792427215@unknownmsgid> On 8 Jul 2011, at 20:16, Everton Blair wrote: sorry, I meant to say new posts ARE NOT forcing the homepage to purge/update, so I need a way to get the homepage to update whenever any purge request comes in. On Fri, Jul 8, 2011 at 8:05 PM, wrote: > Send varnish-misc mailing list submissions to > varnish-misc at varnish-cache.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > or, via email, send a message with subject or body 'help' to > varnish-misc-request at varnish-cache.org > > You can reach the person managing the list at > varnish-misc-owner at varnish-cache.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of varnish-misc digest..." > > > Today's Topics: > > 1. Re: Drupal 7.x & Varnish 3.x (Flatcircle) > 2. Varnish + Apache + SSL + Pound? (Scott Wilcox) > 3. Re:[Wordpress]homepage not purging (Everton Blair) (Everton Blair) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 8 Jul 2011 20:26:18 +0200 > From: Flatcircle > To: Mattias Geniar > Cc: varnish-misc at varnish-cache.org > Subject: Re: Drupal 7.x & Varnish 3.x > Message-ID: > Content-Type: text/plain; charset="us-ascii" > > An HTML attachment was scrubbed... > URL: < > https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20110708/7af63846/attachment-0001.html > > > > ------------------------------ > > Message: 2 > Date: Fri, 8 Jul 2011 19:37:38 +0100 > From: Scott Wilcox > To: > Subject: Varnish + Apache + SSL + Pound? > Message-ID: <2F426C39-48D0-4D7A-A14C-4BBCA9391837 at dor.ky> > Content-Type: text/plain; charset="us-ascii" > > Hello folks, > > I've seen this and similar questions asked often and with varying replies. > I'd like to have my own scenario looked at. I'm currently in the process of > redesigning my work flow for one of my projects. The main web server that > serves content is Apache. I plan to have Varnish caching this traffic where > applicable. I would like to add a layer of redundancy in regards to alerting > users of problems too. I also need to support SSL too. > > I'm looking at having pound accept connections which is then passed to > varnish. Varnish then handles the traffic as normal. > > I've seen in v3 that basic HTML can be output for errors, but I'd like to > include a stylesheet and logo too. I can host these off the site if needed, > but I've considered having 'pretty' error messages which include stylesheets > and images. I'd obviously have to host these off site which isn't a problem. > > Is it better to use nginx to accept both HTTP and HTTPS connections and > serve the errors/content or does my original plan sound better? > > What do you use in production and why? > > Do you recommend pound or nginx? > > Many thanks for replies in advance. > > -- > Scott Wilcox > > @dordotky | scott at dor.ky | http://dor.ky > +44 (0) 7538 842418 | +1 (646) 827-0580 > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: < > https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20110708/ced5f362/attachment-0001.html > > > > ------------------------------ > > Message: 3 > Date: Fri, 8 Jul 2011 20:04:40 +0100 > From: Everton Blair > To: varnish-misc at varnish-cache.org > Subject: Re:[Wordpress]homepage not purging (Everton Blair) > Message-ID: > > > Content-Type: text/plain; charset="iso-8859-1" > > Can anyone help me with my problem please? It's been causing me major > problems. Basically I'm using Wordpress and new posts have been forcing > the > homepage to purge, I think because I'm using a special template within > wordpress to create the homepage and I think Varnish doesn't realise it > needs to purge that page > > What I want to do is force varnish to purge my homepage when any purge > requests come in to make sure it stays fresh and to be safe, but I can't > get > this to work: > > sub vcl_recv { > if (req.request == "PURGE") { > if (!client.ip ~ purge) { > error 405 "Not allowed."; > } > if (req.url == "^windows7news.com/$") { > purge(HOW DO I TELL VARNISH TO PURGE > http://www.windows7news.com/magazine-homepage/ ????); > } > if (req.url == "^windows8news.com/$") { > purge(HOW DO I TELL VARNISH TO PURGE > http://www.windows8news.com/magazine-homepage/ ???); > } > return(lookup); > } > } > > Thanks for any help. > > > On Fri, Jul 8, 2011 at 11:46 AM, >wrote: > > > Send varnish-misc mailing list submissions to > > varnish-misc at varnish-cache.org > > > > To subscribe or unsubscribe via the World Wide Web, visit > > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > or, via email, send a message with subject or body 'help' to > > varnish-misc-request at varnish-cache.org > > > > You can reach the person managing the list at > > varnish-misc-owner at varnish-cache.org > > > > When replying, please edit your Subject line so it is more specific > > than "Re: Contents of varnish-misc digest..." > > > > > > Today's Topics: > > > > 1. Re: Does Varnish uses ETag? (Geoff Simmons) > > 2. Fwd: [Wordpress]homepage not purging (Everton Blair) > > 3. Re: errno = Connection reset by peer (Tollef Fog Heen) > > 4. Re: Varnishstat question (Tollef Fog Heen) > > 5. Possible bug with very large headers in varnish-3.0.0 > > revision 3bd5997 (Lee Trout) > > > > > > ---------------------------------------------------------------------- > > > > Message: 1 > > Date: Fri, 08 Jul 2011 12:23:54 +0200 > > From: Geoff Simmons > > To: Kurt Kraut > > Cc: varnish-misc at varnish-cache.org > > Subject: Re: Does Varnish uses ETag? > > Message-ID: <4E16DABA.2050606 at uplex.de> > > Content-Type: text/plain; charset=ISO-8859-1 > > > > -----BEGIN PGP SIGNED MESSAGE----- > > Hash: SHA256 > > > > On 7/7/11 11:14 PM, Kurt Kraut wrote: > > > > > > Does Varnish used ETag to check cache freshness? > > > > Mainline Varnish doesn't, but there's an experimental branch > > 'experimental-ims' on the source repository in which it's implemented. > > > > > - If yes, how does it behave? > > > > http://www.varnish-cache.org/trac/wiki/BackendConditionalRequests > > > > As per the RFC, freshness checks against ETags are done with conditional > > requests to the backend using If-None-Match. > > > > > - If not, is it possible to achive through VCL? > > > > I don't think so, because standard VCL doesn't have a way to access an > > object in the cache that has an expired TTL, but otherwise matches the > > request, and hence could be refreshed. > > > > So you don't ordinarily know an ETag in VCL that could be used for the > > refresh, and even if you did, and got a 304 response from the backend, > > there's no way to tell Varnish to deliver the stale object after all. > > > > If you get a chance to try the IMS branch, please let us know how it > goes. > > > > > > Best, > > Geoff > > - -- > > UPLEX Systemoptimierung > > Schwanenwik 24 > > 22087 Hamburg > > http://uplex.de/ > > Mob: +49-176-63690917 > > -----BEGIN PGP SIGNATURE----- > > Version: GnuPG/MacGPG2 v2.0.14 (Darwin) > > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > > > iQIcBAEBCAAGBQJOFtq5AAoJEOUwvh9pJNURIy4QAKPiZ05kblNGfw7RFJo5Z9fg > > 81feHHYKRy880yJrG9up8wNx02+9uku6gNlYkpkPcQqfP/fZu5PpUtjGK7pmCSWG > > Jv0bBEL+j5V2L0SaQtoxyyvBkQ5bjQcmrZnuxpYRKRdZeNpjIWY1BLHjMkC3z+sS > > 4KKPi1aiGTKYQH8O1FZHz+6dfP8OQseUCs7tLwYM2BYF2onh5H81ys/Adp6520lv > > WV2cJWKJO85MqH5C0qmyzGV3ovTFPaU3J4/4idzPxW/vbUDozr5/XjloG7IYzCi6 > > Hj5YrfoJlqBREqdAdZ46G/cxS4dKmvMWIO8XaJIYMBISDaXQysXfLq9vLiqODD5l > > /1/ijFfEWihwdoxJ/xi1ZgxhA8RYuxQQCMtStVch+QiHpp8ROw04GutXFmA67JeM > > aWIBt9WL/cXc/YSP2geSobY89/ROulnKcqV9n5xvxGe1L2bkwq1jQJTtINF6l1oC > > lBb9tZAA2JGEeJi8Fv9lcjcrMiLSPzoW16rbzpQNjxxu129Rq8ntteFkm5cqM4sD > > 2do8CvRO3YauIY6bUGhRVSjNHCvYf6HvvoZEoKvgJy5drziKyU3qx3WEq3TF+QQU > > q+Em6WhiYHQufj6OGnZCFJJ4SKDNUNzBkwzuoXay6jxvUtv3m1tGJyKMK0rYN7vv > > 24e8axxNjKucDr5uFx1L > > =PW5G > > -----END PGP SIGNATURE----- > > > > > > > > ------------------------------ > > > > Message: 2 > > Date: Fri, 8 Jul 2011 11:38:57 +0100 > > From: Everton Blair > > To: varnish-misc at varnish-cache.org > > Subject: Fwd: [Wordpress]homepage not purging > > Message-ID: > > CCDxm4Xw at mail.gmail.com > > > > > Content-Type: text/plain; charset="iso-8859-1" > > > > Hi > > > > I've just started using Varnish with Wordpress/W3TC, and all is ok, apart > > from my homepage not purging when an update is made e.g. a new post. I > > think this might be because my homepage is using a static page > > www.windows7news.com/magazine-homepage/ as it's homepage and is doing > some > > magic within Wordpress to show this page when the page > > www.windows7news.com/is requested i.e. the server doesn't know that > > www.windows7news.com/magazine-homepage/ needs purging. > > > > I thought I'd temporariliy managed to fix this by adding: > > > > sub vcl_recv { > > if (req.url ~ "^/") { > > return(pass); > > } > > } > > > > But I only managed to turn the cache off completely. Is there anyway to > > tell Varnish to always purge the homepage if there's a change? > > > > Thanks in advance for any help. > > > > My full VCL is below. If anyone has any Wordpress specific changes > they'd > > like to share then please send them my way! > > > > Regards > > > > EB > > > > > > > ----------------------------------------------------------------------------------------------------------------------------------- > > > > backend b96_30_20_4 { > > .host = "96.30.20.4"; > > .port = "8080"; > > .probe = { > > .url = "/"; > > .interval = 5s; > > .timeout = 1 s; > > .window = 5; > > .threshold = 3; > > } > > // we include time outs so uploads don't time out > > .connect_timeout = 600s; > > .first_byte_timeout = 600s; > > .between_bytes_timeout = 600s; > > } > > > > backend b96_30_20_5 { > > .host = "96.30.20.5"; > > .port = "8080"; > > .probe = { > > .url = "/"; > > .interval = 5s; > > .timeout = 1 s; > > .window = 5; > > .threshold = 3; > > } > > // we include time outs so uploads don't time out > > .connect_timeout = 600s; > > .first_byte_timeout = 600s; > > .between_bytes_timeout = 600s; > > } > > > > backend b96_30_20_6 { > > .host = "96.30.20.6"; > > .port = "8080"; > > .probe = { > > .url = "/"; > > .interval = 5s; > > .timeout = 1 s; > > .window = 5; > > .threshold = 3; > > } > > // we include time outs so uploads don't time out > > .connect_timeout = 600s; > > .first_byte_timeout = 600s; > > .between_bytes_timeout = 600s; > > } > > > > backend b96_30_20_7 { > > .host = "96.30.20.7"; > > .port = "8080"; > > .probe = { > > .url = "/"; > > .interval = 5s; > > .timeout = 1 s; > > .window = 5; > > .threshold = 3; > > } > > // we include time outs so uploads don't time out > > .connect_timeout = 600s; > > .first_byte_timeout = 600s; > > .between_bytes_timeout = 600s; > > } > > > > backend b96_30_18_15 { > > .host = "96.30.18.15"; > > .port = "8080"; > > .probe = { > > .url = "/"; > > .interval = 5s; > > .timeout = 1 s; > > .window = 5; > > .threshold = 3; > > } > > // we include time outs so uploads don't time out > > .connect_timeout = 600s; > > .first_byte_timeout = 600s; > > .between_bytes_timeout = 600s; > > } > > > > acl a96_30_20_4 { > > "96.30.20.4"; > > } > > acl a96_30_20_5 { > > "96.30.20.5"; > > } > > acl a96_30_20_6 { > > "96.30.20.6"; > > } > > acl a96_30_20_7 { > > "96.30.20.7"; > > } > > acl a96_30_18_15 { > > "96.30.18.15"; > > } > > > > acl purge { > > "96.30.20.4"; > > "96.30.20.5"; > > "96.30.20.6"; > > "96.30.20.7"; > > "96.30.18.15"; > > } > > > > sub vcl_recv { > > if (server.ip ~ a96_30_20_4) { > > set req.backend = b96_30_20_4; > > } > > if (server.ip ~ a96_30_20_5) { > > set req.backend = b96_30_20_5; > > } > > if (server.ip ~ a96_30_20_6) { > > set req.backend = b96_30_20_6; > > } > > if (server.ip ~ a96_30_20_7) { > > set req.backend = b96_30_20_7; > > } > > if (server.ip ~ a96_30_18_15) { > > set req.backend = b96_30_18_15; > > } > > } > > > > sub vcl_recv { > > if (req.request == "PURGE") { > > if (!client.ip ~ purge) { > > error 405 "Not allowed."; > > } > > return(lookup); > > } > > if (req.url ~ "^/") { > > return(pass); > > } > > if (req.url ~ "^/forum/$") { > > return(pass); > > } > > if (req.url ~ "^/$") { > > unset req.http.cookie; > > } > > } > > sub vcl_hit { > > if (req.request == "PURGE") { > > set obj.ttl = 0s; > > error 200 "Purged."; > > } > > } > > sub vcl_miss { > > if (req.request == "PURGE") { > > error 404 "Not in cache."; > > } > > if (!(req.url ~ "wp-(login|admin)")) { > > unset req.http.cookie; > > } > > if (req.url ~ > > > > > "^/[^?]+.(jpeg|jpg|png|gif|ico|js|css|txt|gz|zip|lzma|bz2|tgz|tbz|html|htm)(\?.|)$") > > { > > unset req.http.cookie; > > set req.url = regsub(req.url, "\?.$", ""); > > } > > if (req.url ~ "^/$") { > > unset req.http.cookie; > > } > > } > > sub vcl_fetch { > > if (req.url ~ "^/$") { > > unset beresp.http.set-cookie; > > } > > if (!(req.url ~ "wp-(login|admin)")) { > > unset beresp.http.set-cookie; > > } > > } > > -------------- next part -------------- > > An HTML attachment was scrubbed... > > URL: < > > > https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20110708/bf4bd761/attachment-0001.html > > > > > > > ------------------------------ > > > > Message: 3 > > Date: Fri, 08 Jul 2011 12:41:30 +0200 > > From: Tollef Fog Heen > > To: google at alexus.org > > Cc: varnish-misc at varnish-cache.org > > Subject: Re: errno = Connection reset by peer > > Message-ID: <8739iht5ph.fsf at qurzaw.varnish-software.com> > > Content-Type: text/plain; charset=us-ascii > > > > ]] alexus > > > > | 70 Debug c "Write error, retval = -1, len = 10212, errno = > > | Connection reset by peer" > > | > > | other then vcl_error(404, unknown virtual host) what that other message > > | really mean? > > > > That the client disconnected before we were done sending the object. > > > > -- > > Tollef Fog Heen > > Varnish Software > > t: +47 21 98 92 64 > > > > > > > > ------------------------------ > > > > Message: 4 > > Date: Fri, 08 Jul 2011 12:44:45 +0200 > > From: Tollef Fog Heen > > To: Jonathan Hursey > > Cc: varnish-misc at varnish-cache.org > > Subject: Re: Varnishstat question > > Message-ID: <87y609rqzm.fsf at qurzaw.varnish-software.com> > > Content-Type: text/plain; charset=us-ascii > > > > ]] Jonathan Hursey > > > > Hi, > > > > | Greetings fellow varnish enthusiasts. I was wondering if anyone could > > help > > | me determine what would be considered good values for the follow fields > > in > > | varnishstat output and what these fields indicate: > > | > > | 1 0.00 0.00 N vcl total > > | 1 0.00 0.00 N vcl available > > > > This just tells you that you have one VCL loaded. > > > > | 7 . . N total active purges > > | 7 0.00 0.00 N new purges added > > > > You have 7 bans added > > > > | 6112 0.00 0.02 N objects tested > > > > And 6112 objects tested against those bans in total. > > > > Regards, > > -- > > Tollef Fog Heen > > Varnish Software > > t: +47 21 98 92 64 > > > > > > > > ------------------------------ > > > > Message: 5 > > Date: Thu, 30 Jun 2011 20:44:26 -0400 > > From: Lee Trout > > To: varnish-misc at varnish-cache.org > > Subject: Possible bug with very large headers in varnish-3.0.0 > > revision 3bd5997 > > Message-ID: > > Content-Type: text/plain; charset=ISO-8859-1 > > > > Hello! > > > > Please excuse the interruption, but I wanted to see if I may have > > stumbled upon a bug or if there is a known issue or configuration > > option for requests with very large header values. > > > > Using varnish-3.0.0 revision 3bd5997 when a large request comes in > > (cookie header ~3k characters) the header is lost. I am seeing > > "LostHeader c Cookie:" in the logs. > > > > This paste includes the request and results comparing varnish-3.0.0 > > revision 3bd5997 and varnish-2.1.4 SVN 5447M > > http://pastebin.com/J2Qn0KGn > > > > Thank you in advance for your time and suggestions. > > > > Lee > > -- > > Lee Trout > > http://www.leetrout.com > > > > > > > > ------------------------------ > > > > _______________________________________________ > > varnish-misc mailing list > > varnish-misc at varnish-cache.org > > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > > > End of varnish-misc Digest, Vol 64, Issue 13 > > ******************************************** > > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: < > https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20110708/c40bf9ed/attachment.html > > > > ------------------------------ > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > End of varnish-misc Digest, Vol 64, Issue 15 > ******************************************** > _______________________________________________ varnish-misc mailing list varnish-misc at varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc There is a separate tool called varnishadm that can regularly purge pages. Use that. Although you'll have to figure out how to trigger it on post. Alternatively can't you just cache the front page for only a minute or similar? Stew -------------- next part -------------- An HTML attachment was scrubbed... URL: From everton at connectedinternet.co.uk Fri Jul 8 20:11:07 2011 From: everton at connectedinternet.co.uk (Everton Blair) Date: Fri, 8 Jul 2011 21:11:07 +0100 Subject: [Wordpress]homepage not purging (Everton Blair) In-Reply-To: <603466790792427215@unknownmsgid> References: <603466790792427215@unknownmsgid> Message-ID: Hi Stewart Really appreciate you replying and trying to help as this has been driving me mad! Caching the homepage for a shorter period is a potentially a good idea, and I could I guess set it to purge say every 10-15 minutes, so it'd only be one 'unlucky' person who would see an uncached page every 10-15 mins. Not ideal, but it'd work. It'd be even better if I could purge the page and then somehow prime the cache again automatically with the page. On Fri, Jul 8, 2011 at 8:49 PM, Stewart Robinson wrote: > > > On 8 Jul 2011, at 20:16, Everton Blair > wrote: > > sorry, I meant to say new posts ARE NOT forcing the homepage to > purge/update, so I need a way to get the homepage to update whenever any > purge request comes in. > > On Fri, Jul 8, 2011 at 8:05 PM, < > varnish-misc-request at varnish-cache.org> wrote: > >> Send varnish-misc mailing list submissions to >> varnish-misc at varnish-cache.org >> >> To subscribe or unsubscribe via the World Wide Web, visit >> >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> or, via email, send a message with subject or body 'help' to >> >> varnish-misc-request at varnish-cache.org >> >> You can reach the person managing the list at >> >> varnish-misc-owner at varnish-cache.org >> >> When replying, please edit your Subject line so it is more specific >> than "Re: Contents of varnish-misc digest..." >> >> >> Today's Topics: >> >> 1. Re: Drupal 7.x & Varnish 3.x (Flatcircle) >> 2. Varnish + Apache + SSL + Pound? (Scott Wilcox) >> 3. Re:[Wordpress]homepage not purging (Everton Blair) (Everton Blair) >> >> >> ---------------------------------------------------------------------- >> >> Message: 1 >> Date: Fri, 8 Jul 2011 20:26:18 +0200 >> From: Flatcircle < flatcircle at hotmail.com> >> To: Mattias Geniar < mattias at nucleus.be> >> Cc: varnish-misc at varnish-cache.org >> Subject: Re: Drupal 7.x & Varnish 3.x >> Message-ID: >> Content-Type: text/plain; charset="us-ascii" >> >> An HTML attachment was scrubbed... >> URL: < >> https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20110708/7af63846/attachment-0001.html >> > >> >> ------------------------------ >> >> Message: 2 >> Date: Fri, 8 Jul 2011 19:37:38 +0100 >> From: Scott Wilcox < scott at dor.ky> >> To: < varnish-misc at varnish-cache.org> >> Subject: Varnish + Apache + SSL + Pound? >> Message-ID: < <2F426C39-48D0-4D7A-A14C-4BBCA9391837 at dor.ky> >> 2F426C39-48D0-4D7A-A14C-4BBCA9391837 at dor.ky> >> Content-Type: text/plain; charset="us-ascii" >> >> Hello folks, >> >> I've seen this and similar questions asked often and with varying replies. >> I'd like to have my own scenario looked at. I'm currently in the process of >> redesigning my work flow for one of my projects. The main web server that >> serves content is Apache. I plan to have Varnish caching this traffic where >> applicable. I would like to add a layer of redundancy in regards to alerting >> users of problems too. I also need to support SSL too. >> >> I'm looking at having pound accept connections which is then passed to >> varnish. Varnish then handles the traffic as normal. >> >> I've seen in v3 that basic HTML can be output for errors, but I'd like to >> include a stylesheet and logo too. I can host these off the site if needed, >> but I've considered having 'pretty' error messages which include stylesheets >> and images. I'd obviously have to host these off site which isn't a problem. >> >> Is it better to use nginx to accept both HTTP and HTTPS connections and >> serve the errors/content or does my original plan sound better? >> >> What do you use in production and why? >> >> Do you recommend pound or nginx? >> >> Many thanks for replies in advance. >> >> -- >> Scott Wilcox >> >> @dordotky | scott at dor.ky | http://dor.ky >> +44 (0) 7538 842418 | +1 (646) 827-0580 >> >> -------------- next part -------------- >> An HTML attachment was scrubbed... >> URL: < >> https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20110708/ced5f362/attachment-0001.html >> > >> >> ------------------------------ >> >> Message: 3 >> Date: Fri, 8 Jul 2011 20:04:40 +0100 >> From: Everton Blair < >> everton at connectedinternet.co.uk> >> To: varnish-misc at varnish-cache.org >> Subject: Re:[Wordpress]homepage not purging (Everton Blair) >> Message-ID: >> < >> CAOAN0RQEraiGv9e8K3jTsAuM3YOdgAyBK8iahHZBcZY1sfFTdw at mail.gmail.com> >> Content-Type: text/plain; charset="iso-8859-1" >> >> Can anyone help me with my problem please? It's been causing me major >> problems. Basically I'm using Wordpress and new posts have been forcing >> the >> homepage to purge, I think because I'm using a special template within >> wordpress to create the homepage and I think Varnish doesn't realise it >> needs to purge that page >> >> What I want to do is force varnish to purge my homepage when any purge >> requests come in to make sure it stays fresh and to be safe, but I can't >> get >> this to work: >> >> sub vcl_recv { >> if (req.request == "PURGE") { >> if (!client.ip ~ purge) { >> error 405 "Not allowed."; >> } >> if (req.url == "^ windows7news.com/$") { >> purge(HOW DO I TELL VARNISH TO PURGE >> >> http://www.windows7news.com/magazine-homepage/ ????); >> } >> if (req.url == "^ windows8news.com/$") { >> purge(HOW DO I TELL VARNISH TO PURGE >> >> http://www.windows8news.com/magazine-homepage/ ???); >> } >> return(lookup); >> } >> } >> >> Thanks for any help. >> >> >> On Fri, Jul 8, 2011 at 11:46 AM, < >> varnish-misc-request at varnish-cache.org>wrote: >> >> > Send varnish-misc mailing list submissions to >> > varnish-misc at varnish-cache.org >> > >> > To subscribe or unsubscribe via the World Wide Web, visit >> > >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> > or, via email, send a message with subject or body 'help' to >> > >> varnish-misc-request at varnish-cache.org >> > >> > You can reach the person managing the list at >> > >> varnish-misc-owner at varnish-cache.org >> > >> > When replying, please edit your Subject line so it is more specific >> > than "Re: Contents of varnish-misc digest..." >> > >> > >> > Today's Topics: >> > >> > 1. Re: Does Varnish uses ETag? (Geoff Simmons) >> > 2. Fwd: [Wordpress]homepage not purging (Everton Blair) >> > 3. Re: errno = Connection reset by peer (Tollef Fog Heen) >> > 4. Re: Varnishstat question (Tollef Fog Heen) >> > 5. Possible bug with very large headers in varnish-3.0.0 >> > revision 3bd5997 (Lee Trout) >> > >> > >> > ---------------------------------------------------------------------- >> > >> > Message: 1 >> > Date: Fri, 08 Jul 2011 12:23:54 +0200 >> > From: Geoff Simmons < geoff at uplex.de> >> > To: Kurt Kraut < listas at kurtkraut.net> >> > Cc: varnish-misc at varnish-cache.org >> > Subject: Re: Does Varnish uses ETag? >> > Message-ID: < <4E16DABA.2050606 at uplex.de>4E16DABA.2050606 at uplex.de> >> > Content-Type: text/plain; charset=ISO-8859-1 >> > >> > -----BEGIN PGP SIGNED MESSAGE----- >> > Hash: SHA256 >> > >> > On 7/7/11 11:14 PM, Kurt Kraut wrote: >> > > >> > > Does Varnish used ETag to check cache freshness? >> > >> > Mainline Varnish doesn't, but there's an experimental branch >> > 'experimental-ims' on the source repository in which it's implemented. >> > >> > > - If yes, how does it behave? >> > >> > >> http://www.varnish-cache.org/trac/wiki/BackendConditionalRequests >> > >> > As per the RFC, freshness checks against ETags are done with conditional >> > requests to the backend using If-None-Match. >> > >> > > - If not, is it possible to achive through VCL? >> > >> > I don't think so, because standard VCL doesn't have a way to access an >> > object in the cache that has an expired TTL, but otherwise matches the >> > request, and hence could be refreshed. >> > >> > So you don't ordinarily know an ETag in VCL that could be used for the >> > refresh, and even if you did, and got a 304 response from the backend, >> > there's no way to tell Varnish to deliver the stale object after all. >> > >> > If you get a chance to try the IMS branch, please let us know how it >> goes. >> > >> > >> > Best, >> > Geoff >> > - -- >> > UPLEX Systemoptimierung >> > Schwanenwik 24 >> > 22087 Hamburg >> > http://uplex.de/ >> > Mob: +49-176-63690917 >> > -----BEGIN PGP SIGNATURE----- >> > Version: GnuPG/MacGPG2 v2.0.14 (Darwin) >> > Comment: Using GnuPG with Mozilla - >> http://enigmail.mozdev.org/ >> > >> > iQIcBAEBCAAGBQJOFtq5AAoJEOUwvh9pJNURIy4QAKPiZ05kblNGfw7RFJo5Z9fg >> > 81feHHYKRy880yJrG9up8wNx02+9uku6gNlYkpkPcQqfP/fZu5PpUtjGK7pmCSWG >> > Jv0bBEL+j5V2L0SaQtoxyyvBkQ5bjQcmrZnuxpYRKRdZeNpjIWY1BLHjMkC3z+sS >> > 4KKPi1aiGTKYQH8O1FZHz+6dfP8OQseUCs7tLwYM2BYF2onh5H81ys/Adp6520lv >> > WV2cJWKJO85MqH5C0qmyzGV3ovTFPaU3J4/4idzPxW/vbUDozr5/XjloG7IYzCi6 >> > Hj5YrfoJlqBREqdAdZ46G/cxS4dKmvMWIO8XaJIYMBISDaXQysXfLq9vLiqODD5l >> > /1/ijFfEWihwdoxJ/xi1ZgxhA8RYuxQQCMtStVch+QiHpp8ROw04GutXFmA67JeM >> > aWIBt9WL/cXc/YSP2geSobY89/ROulnKcqV9n5xvxGe1L2bkwq1jQJTtINF6l1oC >> > lBb9tZAA2JGEeJi8Fv9lcjcrMiLSPzoW16rbzpQNjxxu129Rq8ntteFkm5cqM4sD >> > 2do8CvRO3YauIY6bUGhRVSjNHCvYf6HvvoZEoKvgJy5drziKyU3qx3WEq3TF+QQU >> > q+Em6WhiYHQufj6OGnZCFJJ4SKDNUNzBkwzuoXay6jxvUtv3m1tGJyKMK0rYN7vv >> > 24e8axxNjKucDr5uFx1L >> > =PW5G >> > -----END PGP SIGNATURE----- >> > >> > >> > >> > ------------------------------ >> > >> > Message: 2 >> > Date: Fri, 8 Jul 2011 11:38:57 +0100 >> > From: Everton Blair < >> everton at connectedinternet.co.uk> >> > To: varnish-misc at varnish-cache.org >> > Subject: Fwd: [Wordpress]homepage not purging >> > Message-ID: >> > >> CCDxm4Xw at mail.gmail.com >> > > >> > Content-Type: text/plain; charset="iso-8859-1" >> > >> > Hi >> > >> > I've just started using Varnish with Wordpress/W3TC, and all is ok, >> apart >> > from my homepage not purging when an update is made e.g. a new post. I >> > think this might be because my homepage is using a static page >> > >> www.windows7news.com/magazine-homepage/ as it's homepage and is doing >> some >> > magic within Wordpress to show this page when the page >> > www.windows7news.com/is requested i.e. >> the server doesn't know that >> > >> www.windows7news.com/magazine-homepage/ needs purging. >> > >> > I thought I'd temporariliy managed to fix this by adding: >> > >> > sub vcl_recv { >> > if (req.url ~ "^/") { >> > return(pass); >> > } >> > } >> > >> > But I only managed to turn the cache off completely. Is there anyway to >> > tell Varnish to always purge the homepage if there's a change? >> > >> > Thanks in advance for any help. >> > >> > My full VCL is below. If anyone has any Wordpress specific changes >> they'd >> > like to share then please send them my way! >> > >> > Regards >> > >> > EB >> > >> > >> > >> ----------------------------------------------------------------------------------------------------------------------------------- >> > >> > backend b96_30_20_4 { >> > .host = "96.30.20.4"; >> > .port = "8080"; >> > .probe = { >> > .url = "/"; >> > .interval = 5s; >> > .timeout = 1 s; >> > .window = 5; >> > .threshold = 3; >> > } >> > // we include time outs so uploads don't time out >> > .connect_timeout = 600s; >> > .first_byte_timeout = 600s; >> > .between_bytes_timeout = 600s; >> > } >> > >> > backend b96_30_20_5 { >> > .host = "96.30.20.5"; >> > .port = "8080"; >> > .probe = { >> > .url = "/"; >> > .interval = 5s; >> > .timeout = 1 s; >> > .window = 5; >> > .threshold = 3; >> > } >> > // we include time outs so uploads don't time out >> > .connect_timeout = 600s; >> > .first_byte_timeout = 600s; >> > .between_bytes_timeout = 600s; >> > } >> > >> > backend b96_30_20_6 { >> > .host = "96.30.20.6"; >> > .port = "8080"; >> > .probe = { >> > .url = "/"; >> > .interval = 5s; >> > .timeout = 1 s; >> > .window = 5; >> > .threshold = 3; >> > } >> > // we include time outs so uploads don't time out >> > .connect_timeout = 600s; >> > .first_byte_timeout = 600s; >> > .between_bytes_timeout = 600s; >> > } >> > >> > backend b96_30_20_7 { >> > .host = "96.30.20.7"; >> > .port = "8080"; >> > .probe = { >> > .url = "/"; >> > .interval = 5s; >> > .timeout = 1 s; >> > .window = 5; >> > .threshold = 3; >> > } >> > // we include time outs so uploads don't time out >> > .connect_timeout = 600s; >> > .first_byte_timeout = 600s; >> > .between_bytes_timeout = 600s; >> > } >> > >> > backend b96_30_18_15 { >> > .host = "96.30.18.15"; >> > .port = "8080"; >> > .probe = { >> > .url = "/"; >> > .interval = 5s; >> > .timeout = 1 s; >> > .window = 5; >> > .threshold = 3; >> > } >> > // we include time outs so uploads don't time out >> > .connect_timeout = 600s; >> > .first_byte_timeout = 600s; >> > .between_bytes_timeout = 600s; >> > } >> > >> > acl a96_30_20_4 { >> > "96.30.20.4"; >> > } >> > acl a96_30_20_5 { >> > "96.30.20.5"; >> > } >> > acl a96_30_20_6 { >> > "96.30.20.6"; >> > } >> > acl a96_30_20_7 { >> > "96.30.20.7"; >> > } >> > acl a96_30_18_15 { >> > "96.30.18.15"; >> > } >> > >> > acl purge { >> > "96.30.20.4"; >> > "96.30.20.5"; >> > "96.30.20.6"; >> > "96.30.20.7"; >> > "96.30.18.15"; >> > } >> > >> > sub vcl_recv { >> > if (server.ip ~ a96_30_20_4) { >> > set req.backend = b96_30_20_4; >> > } >> > if (server.ip ~ a96_30_20_5) { >> > set req.backend = b96_30_20_5; >> > } >> > if (server.ip ~ a96_30_20_6) { >> > set req.backend = b96_30_20_6; >> > } >> > if (server.ip ~ a96_30_20_7) { >> > set req.backend = b96_30_20_7; >> > } >> > if (server.ip ~ a96_30_18_15) { >> > set req.backend = b96_30_18_15; >> > } >> > } >> > >> > sub vcl_recv { >> > if (req.request == "PURGE") { >> > if (!client.ip ~ purge) { >> > error 405 "Not allowed."; >> > } >> > return(lookup); >> > } >> > if (req.url ~ "^/") { >> > return(pass); >> > } >> > if (req.url ~ "^/forum/$") { >> > return(pass); >> > } >> > if (req.url ~ "^/$") { >> > unset req.http.cookie; >> > } >> > } >> > sub vcl_hit { >> > if (req.request == "PURGE") { >> > set obj.ttl = 0s; >> > error 200 "Purged."; >> > } >> > } >> > sub vcl_miss { >> > if (req.request == "PURGE") { >> > error 404 "Not in cache."; >> > } >> > if (!(req.url ~ "wp-(login|admin)")) { >> > unset req.http.cookie; >> > } >> > if (req.url ~ >> > >> > >> "^/[^?]+.(jpeg|jpg|png|gif|ico|js|css|txt|gz|zip|lzma|bz2|tgz|tbz|html|htm)(\?.|)$") >> > { >> > unset req.http.cookie; >> > set req.url = regsub(req.url, "\?.$", ""); >> > } >> > if (req.url ~ "^/$") { >> > unset req.http.cookie; >> > } >> > } >> > sub vcl_fetch { >> > if (req.url ~ "^/$") { >> > unset beresp.http.set-cookie; >> > } >> > if (!(req.url ~ "wp-(login|admin)")) { >> > unset beresp.http.set-cookie; >> > } >> > } >> > -------------- next part -------------- >> > An HTML attachment was scrubbed... >> > URL: < >> > >> >> https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20110708/bf4bd761/attachment-0001.html >> > > >> > >> > ------------------------------ >> > >> > Message: 3 >> > Date: Fri, 08 Jul 2011 12:41:30 +0200 >> > From: Tollef Fog Heen < >> tfheen at varnish-software.com> >> > To: google at alexus.org >> > Cc: varnish-misc at varnish-cache.org >> > Subject: Re: errno = Connection reset by peer >> > Message-ID: < <8739iht5ph.fsf at qurzaw.varnish-software.com> >> 8739iht5ph.fsf at qurzaw.varnish-software.com> >> > Content-Type: text/plain; charset=us-ascii >> > >> > ]] alexus >> > >> > | 70 Debug c "Write error, retval = -1, len = 10212, errno = >> > | Connection reset by peer" >> > | >> > | other then vcl_error(404, unknown virtual host) what that other >> message >> > | really mean? >> > >> > That the client disconnected before we were done sending the object. >> > >> > -- >> > Tollef Fog Heen >> > Varnish Software >> > t: +47 21 98 92 64 >> > >> > >> > >> > ------------------------------ >> > >> > Message: 4 >> > Date: Fri, 08 Jul 2011 12:44:45 +0200 >> > From: Tollef Fog Heen < >> tfheen at varnish-software.com> >> > To: Jonathan Hursey < >> jonathan.hursey at adrevolution.com> >> > Cc: varnish-misc at varnish-cache.org >> > Subject: Re: Varnishstat question >> > Message-ID: < <87y609rqzm.fsf at qurzaw.varnish-software.com> >> 87y609rqzm.fsf at qurzaw.varnish-software.com> >> > Content-Type: text/plain; charset=us-ascii >> > >> > ]] Jonathan Hursey >> > >> > Hi, >> > >> > | Greetings fellow varnish enthusiasts. I was wondering if anyone could >> > help >> > | me determine what would be considered good values for the follow >> fields >> > in >> > | varnishstat output and what these fields indicate: >> > | >> > | 1 0.00 0.00 N vcl total >> > | 1 0.00 0.00 N vcl available >> > >> > This just tells you that you have one VCL loaded. >> > >> > | 7 . . N total active purges >> > | 7 0.00 0.00 N new purges added >> > >> > You have 7 bans added >> > >> > | 6112 0.00 0.02 N objects tested >> > >> > And 6112 objects tested against those bans in total. >> > >> > Regards, >> > -- >> > Tollef Fog Heen >> > Varnish Software >> > t: +47 21 98 92 64 >> > >> > >> > >> > ------------------------------ >> > >> > Message: 5 >> > Date: Thu, 30 Jun 2011 20:44:26 -0400 >> > From: Lee Trout < lee at leetrout.com> >> > To: varnish-misc at varnish-cache.org >> > Subject: Possible bug with very large headers in varnish-3.0.0 >> > revision 3bd5997 >> > Message-ID: >> eVPwg at mail.gmail.com> >> > Content-Type: text/plain; charset=ISO-8859-1 >> > >> > Hello! >> > >> > Please excuse the interruption, but I wanted to see if I may have >> > stumbled upon a bug or if there is a known issue or configuration >> > option for requests with very large header values. >> > >> > Using varnish-3.0.0 revision 3bd5997 when a large request comes in >> > (cookie header ~3k characters) the header is lost. I am seeing >> > "LostHeader c Cookie:" in the logs. >> > >> > This paste includes the request and results comparing varnish-3.0.0 >> > revision 3bd5997 and varnish-2.1.4 SVN 5447M >> > http://pastebin.com/J2Qn0KGn >> > >> > Thank you in advance for your time and suggestions. >> > >> > Lee >> > -- >> > Lee Trout >> > http://www.leetrout.com >> > >> > >> > >> > ------------------------------ >> > >> > _______________________________________________ >> > varnish-misc mailing list >> > varnish-misc at varnish-cache.org >> > >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> > >> > End of varnish-misc Digest, Vol 64, Issue 13 >> > ******************************************** >> > >> -------------- next part -------------- >> An HTML attachment was scrubbed... >> URL: < >> https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20110708/c40bf9ed/attachment.html >> > >> >> ------------------------------ >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> >> End of varnish-misc Digest, Vol 64, Issue 15 >> ******************************************** >> > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > > There is a separate tool called varnishadm that can regularly purge pages. > Use that. Although you'll have to figure out how to trigger it on post. > Alternatively can't you just cache the front page for only a minute or > similar? > > Stew > -------------- next part -------------- An HTML attachment was scrubbed... URL: From l at lrowe.co.uk Fri Jul 8 23:08:01 2011 From: l at lrowe.co.uk (Laurence Rowe) Date: Sat, 9 Jul 2011 00:08:01 +0100 Subject: [Wordpress]homepage not purging (Everton Blair) In-Reply-To: References: Message-ID: On 8 July 2011 20:04, Everton Blair wrote: > Can anyone help me with my problem please? ?It's been causing me major > problems. ?Basically I'm using Wordpress and new posts have been forcing the > homepage to purge, I think because I'm using a special template within > wordpress to create the homepage and I think Varnish doesn't realise it > needs to purge that page > What I want to do is force varnish to purge my homepage when any purge > requests come in to make sure it stays fresh and to be safe, but I can't get > this to work: > sub vcl_recv { > if (req.request == "PURGE") { > ? ? ? ? ? ? if (!client.ip ~ purge) { > ? ? ? ? ? ? ? ? error 405 "Not allowed."; > ? ? ? ? ? ? ? ? } > if (req.url == "^windows7news.com/$") { > purge(HOW DO I TELL VARNISH TO PURGE > http://www.windows7news.com/magazine-homepage/ ????); > } > if (req.url == "^windows8news.com/$") { > purge(HOW DO I TELL VARNISH TO PURGE > http://www.windows8news.com/magazine-homepage/ ???); > } > return(lookup); > ? ? ? ? } > } > Thanks for any help. Remember that req.url is only the path of the request, it does not include the entire hostname (except sometimes when you use curl, but just ignore that for now.) Assuming Varnish 2.1 you want something like: purge("req.host == " req.http.host " && req.url == /magazine-homepage/"); See: https://www.varnish-cache.org/trac/wiki/VCLExamplePurging Laurence From straightflush at gmail.com Fri Jul 8 23:25:05 2011 From: straightflush at gmail.com (AD) Date: Fri, 8 Jul 2011 19:25:05 -0400 Subject: ignore query param in req.hash Message-ID: what is the cleanest way to ignore a single query param in the cache key (req.hash) without having to modify req.url ? I still want the query param to go to the backend but i just want it to be ignored when generating the cache key in req.hash. Cheers, AD -------------- next part -------------- An HTML attachment was scrubbed... URL: From l at lrowe.co.uk Fri Jul 8 23:29:10 2011 From: l at lrowe.co.uk (Laurence Rowe) Date: Sat, 9 Jul 2011 00:29:10 +0100 Subject: [Wordpress]homepage not purging (Everton Blair) In-Reply-To: References: Message-ID: On 9 July 2011 00:16, Everton Blair wrote: > Hi Laurence > Thanks for the reply. ?I tried: > sub vcl_recv { > ? ? if (req.request == "PURGE") { > ? ? ? ? ? ? if (!client.ip ~ purge) { > ? ? ? ? ? ? ? ? error 405 "Not allowed."; > ? ? ? ? ? ? ? ? } > purge("req.host == " req.http.host " && req.url == /magazine-homepage/"); > return(lookup); > ? ? ? ? } > But I got this error: > Message from VCC-compiler: > Expected ';' got '(' > I'm using Varnish 3.0. In varnish 3.0 the syntax has changed to ban. See: https://www.varnish-cache.org/docs/3.0/tutorial/purging.html Laurence From phk at phk.freebsd.dk Sat Jul 9 05:39:16 2011 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Sat, 09 Jul 2011 05:39:16 +0000 Subject: ignore query param in req.hash In-Reply-To: Your message of "Fri, 08 Jul 2011 19:25:05 -0400." Message-ID: <7645.1310189956@critter.freebsd.dk> In message , AD writes: >what is the cleanest way to ignore a single query param in the cache key >(req.hash) without having to modify req.url ? Use regsub() to strip it, something like: hash_data(regsub(req.url, "[?].*", "")); -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From l at lrowe.co.uk Sat Jul 9 08:13:24 2011 From: l at lrowe.co.uk (Laurence Rowe) Date: Sat, 9 Jul 2011 09:13:24 +0100 Subject: [Wordpress]homepage not purging (Everton Blair) In-Reply-To: References: Message-ID: On 9 July 2011 07:25, Everton Blair wrote: > If I do: > sub vcl_recv { > ? ? if (req.request == "PURGE") { > ? ? ? ? ? ? if (!client.ip ~ purge) { > ? ? ? ? ? ? ? ? error 405 "Not allowed."; > ? ? ? ? ? ? ? ? } > ban("req.host == " req.http.host " && req.url == /magazine-homepage/"); > return(lookup); > ? ? ? ? } > Won't that stop that page ever being cached, or is a ban not a 'ban' and > just another type of flush/purge? VCL seems to = Very confusing language! > On Sat, Jul 9, 2011 at 12:29 AM, Laurence Rowe wrote: No, a 'ban' only affects objects already in the cache at the time it is set. Laurence P.S. Please stay on the mailing list rather than replying privately. From everton at connectedinternet.co.uk Sat Jul 9 09:13:32 2011 From: everton at connectedinternet.co.uk (Everton Blair) Date: Sat, 9 Jul 2011 10:13:32 +0100 Subject: [Wordpress]homepage not purging (Everton Blair) In-Reply-To: References: Message-ID: sorry about replying all. "No, a 'ban' only affects objects already in the cache at the time it is set." let's see if I understand this correctly: 1. homepage will be cached until a purge comes in 2. purge triggers the ban, which removes the homepage from the cache 3. when homepage is viewed again it is cached, and will be until another purge comes in? I think the use of the word 'ban' is what causes my confusion. I tried adding: purge("req.host == " req.http.host " && req.url == " ^/magazine-homepage/); and ban("req.host == " req.http.host " && req.url == " ^/magazine-homepage/); as I'm using 3.0, but I kept getting a syntax error. Can you tell me what I'm doing wrong please. Thanks On Sat, Jul 9, 2011 at 9:13 AM, Laurence Rowe wrote: > On 9 July 2011 07:25, Everton Blair > wrote: > > If I do: > > sub vcl_recv { > > if (req.request == "PURGE") { > > if (!client.ip ~ purge) { > > error 405 "Not allowed."; > > } > > ban("req.host == " req.http.host " && req.url == /magazine-homepage/"); > > return(lookup); > > } > > Won't that stop that page ever being cached, or is a ban not a 'ban' and > > just another type of flush/purge? VCL seems to = Very confusing language! > > On Sat, Jul 9, 2011 at 12:29 AM, Laurence Rowe wrote: > > No, a 'ban' only affects objects already in the cache at the time it is > set. > > Laurence > > P.S. Please stay on the mailing list rather than replying privately. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From enno at tty.nl Sat Jul 9 09:29:47 2011 From: enno at tty.nl (Enno van Amerongen) Date: Sat, 09 Jul 2011 11:29:47 +0200 Subject: [Wordpress]homepage not purging (Everton Blair) In-Reply-To: Message-ID: <20110709092947.fc00127d@groupware.tty.nl> use a far shorter ttl, 10-15 mins is way too much, use something like 10 seconds or even less. that will result in max 0.1 req/sec on your backend, I doubt that's a problem. Enno _____ From: Everton Blair [mailto:everton at connectedinternet.co.uk] To: Stewart Robinson [mailto:stewsnooze at gmail.com] Cc: varnish-misc at varnish-cache.org [mailto:varnish-misc at varnish-cache.org] Sent: Fri, 08 Jul 2011 22:11:07 +0200 Subject: Re: [Wordpress]homepage not purging (Everton Blair) Hi Stewart Really appreciate you replying and trying to help as this has been driving me mad! Caching the homepage for a shorter period is a potentially a good idea, and I could I guess set it to purge say every 10-15 minutes, so it'd only be one 'unlucky' person who would see an uncached page every 10-15 mins. Not ideal, but it'd work. It'd be even better if I could purge the page and then somehow prime the cache again automatically with the page. On Fri, Jul 8, 2011 at 8:49 PM, Stewart Robinson wrote: On 8 Jul 2011, at 20:16, Everton Blair wrote: sorry, I meant to say new posts ARE NOT forcing the homepage to purge/update, so I need a way to get the homepage to update whenever any purge request comes in. On Fri, Jul 8, 2011 at 8:05 PM, wrote: Send varnish-misc mailing list submissions to varnish-misc at varnish-cache.org To subscribe or unsubscribe via the World Wide Web, visit https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc or, via email, send a message with subject or body 'help' to varnish-misc-request at varnish-cache.org You can reach the person managing the list at varnish-misc-owner at varnish-cache.org When replying, please edit your Subject line so it is more specific than "Re: Contents of varnish-misc digest..." Today's Topics: 1. Re: Drupal 7.x & Varnish 3.x (Flatcircle) 2. Varnish + Apache + SSL + Pound? (Scott Wilcox) 3. Re:[Wordpress]homepage not purging (Everton Blair) (Everton Blair) ---------------------------------------------------------------------- Message: 1 Date: Fri, 8 Jul 2011 20:26:18 +0200 From: Flatcircle To: Mattias Geniar Cc: varnish-misc at varnish-cache.org Subject: Re: Drupal 7.x & Varnish 3.x Message-ID: Content-Type: text/plain; charset="us-ascii" An HTML attachment was scrubbed... URL: ------------------------------ Message: 2 Date: Fri, 8 Jul 2011 19:37:38 +0100 From: Scott Wilcox To: Subject: Varnish + Apache + SSL + Pound? Message-ID: <2F426C39-48D0-4D7A-A14C-4BBCA9391837 at dor.ky> Content-Type: text/plain; charset="us-ascii" Hello folks, I've seen this and similar questions asked often and with varying replies. I'd like to have my own scenario looked at. I'm currently in the process of redesigning my work flow for one of my projects. The main web server that serves content is Apache. I plan to have Varnish caching this traffic where applicable. I would like to add a layer of redundancy in regards to alerting users of problems too. I also need to support SSL too. I'm looking at having pound accept connections which is then passed to varnish. Varnish then handles the traffic as normal. I've seen in v3 that basic HTML can be output for errors, but I'd like to include a stylesheet and logo too. I can host these off the site if needed, but I've considered having 'pretty' error messages which include stylesheets and images. I'd obviously have to host these off site which isn't a problem. Is it better to use nginx to accept both HTTP and HTTPS connections and serve the errors/content or does my original plan sound better? What do you use in production and why? Do you recommend pound or nginx? Many thanks for replies in advance. -- Scott Wilcox @dordotky | scott at dor.ky | http://dor.ky +44 (0) 7538 842418 | +1 (646) 827-0580 -------------- next part -------------- An HTML attachment was scrubbed... URL: ------------------------------ Message: 3 Date: Fri, 8 Jul 2011 20:04:40 +0100 From: Everton Blair To: varnish-misc at varnish-cache.org Subject: Re:[Wordpress]homepage not purging (Everton Blair) Message-ID: Content-Type: text/plain; charset="iso-8859-1" Can anyone help me with my problem please? It's been causing me major problems. Basically I'm using Wordpress and new posts have been forcing the homepage to purge, I think because I'm using a special template within wordpress to create the homepage and I think Varnish doesn't realise it needs to purge that page What I want to do is force varnish to purge my homepage when any purge requests come in to make sure it stays fresh and to be safe, but I can't get this to work: sub vcl_recv { if (req.request == "PURGE") { if (!client.ip ~ purge) { error 405 "Not allowed."; } if (req.url == "^windows7news.com/$") { purge(HOW DO I TELL VARNISH TO PURGE http://www.windows7news.com/magazine-homepage/ ????); } if (req.url == "^windows8news.com/$") { purge(HOW DO I TELL VARNISH TO PURGE http://www.windows8news.com/magazine-homepage/ ???); } return(lookup); } } Thanks for any help. On Fri, Jul 8, 2011 at 11:46 AM, wrote: > Send varnish-misc mailing list submissions to > varnish-misc at varnish-cache.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > or, via email, send a message with subject or body 'help' to > varnish-misc-request at varnish-cache.org > > You can reach the person managing the list at > varnish-misc-owner at varnish-cache.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of varnish-misc digest..." > > > Today's Topics: > > 1. Re: Does Varnish uses ETag? (Geoff Simmons) > 2. Fwd: [Wordpress]homepage not purging (Everton Blair) > 3. Re: errno = Connection reset by peer (Tollef Fog Heen) > 4. Re: Varnishstat question (Tollef Fog Heen) > 5. Possible bug with very large headers in varnish-3.0.0 > revision 3bd5997 (Lee Trout) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 08 Jul 2011 12:23:54 +0200 > From: Geoff Simmons > To: Kurt Kraut > Cc: varnish-misc at varnish-cache.org > Subject: Re: Does Varnish uses ETag? > Message-ID: <4E16DABA.2050606 at uplex.de> > Content-Type: text/plain; charset=ISO-8859-1 > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > On 7/7/11 11:14 PM, Kurt Kraut wrote: > > > > Does Varnish used ETag to check cache freshness? > > Mainline Varnish doesn't, but there's an experimental branch > 'experimental-ims' on the source repository in which it's implemented. > > > - If yes, how does it behave? > > http://www.varnish-cache.org/trac/wiki/BackendConditionalRequests > > As per the RFC, freshness checks against ETags are done with conditional > requests to the backend using If-None-Match. > > > - If not, is it possible to achive through VCL? > > I don't think so, because standard VCL doesn't have a way to access an > object in the cache that has an expired TTL, but otherwise matches the > request, and hence could be refreshed. > > So you don't ordinarily know an ETag in VCL that could be used for the > refresh, and even if you did, and got a 304 response from the backend, > there's no way to tell Varnish to deliver the stale object after all. > > If you get a chance to try the IMS branch, please let us know how it goes. > > > Best, > Geoff > - -- > UPLEX Systemoptimierung > Schwanenwik 24 > 22087 Hamburg > http://uplex.de/ > Mob: +49-176-63690917 > -----BEGIN PGP SIGNATURE----- > Version: GnuPG/MacGPG2 v2.0.14 (Darwin) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iQIcBAEBCAAGBQJOFtq5AAoJEOUwvh9pJNURIy4QAKPiZ05kblNGfw7RFJo5Z9fg > 81feHHYKRy880yJrG9up8wNx02+9uku6gNlYkpkPcQqfP/fZu5PpUtjGK7pmCSWG > Jv0bBEL+j5V2L0SaQtoxyyvBkQ5bjQcmrZnuxpYRKRdZeNpjIWY1BLHjMkC3z+sS > 4KKPi1aiGTKYQH8O1FZHz+6dfP8OQseUCs7tLwYM2BYF2onh5H81ys/Adp6520lv > WV2cJWKJO85MqH5C0qmyzGV3ovTFPaU3J4/4idzPxW/vbUDozr5/XjloG7IYzCi6 > Hj5YrfoJlqBREqdAdZ46G/cxS4dKmvMWIO8XaJIYMBISDaXQysXfLq9vLiqODD5l > /1/ijFfEWihwdoxJ/xi1ZgxhA8RYuxQQCMtStVch+QiHpp8ROw04GutXFmA67JeM > aWIBt9WL/cXc/YSP2geSobY89/ROulnKcqV9n5xvxGe1L2bkwq1jQJTtINF6l1oC > lBb9tZAA2JGEeJi8Fv9lcjcrMiLSPzoW16rbzpQNjxxu129Rq8ntteFkm5cqM4sD > 2do8CvRO3YauIY6bUGhRVSjNHCvYf6HvvoZEoKvgJy5drziKyU3qx3WEq3TF+QQU > q+Em6WhiYHQufj6OGnZCFJJ4SKDNUNzBkwzuoXay6jxvUtv3m1tGJyKMK0rYN7vv > 24e8axxNjKucDr5uFx1L > =PW5G > -----END PGP SIGNATURE----- > > > > ------------------------------ > > Message: 2 > Date: Fri, 8 Jul 2011 11:38:57 +0100 > From: Everton Blair > To: varnish-misc at varnish-cache.org > Subject: Fwd: [Wordpress]homepage not purging > Message-ID: > > > Content-Type: text/plain; charset="iso-8859-1" > > Hi > > I've just started using Varnish with Wordpress/W3TC, and all is ok, apart > from my homepage not purging when an update is made e.g. a new post. I > think this might be because my homepage is using a static page > www.windows7news.com/magazine-homepage/ as it's homepage and is doing some > magic within Wordpress to show this page when the page > www.windows7news.com/is requested i.e. the server doesn't know that > www.windows7news.com/magazine-homepage/ needs purging. > > I thought I'd temporariliy managed to fix this by adding: > > sub vcl_recv { > if (req.url ~ "^/") { > return(pass); > } > } > > But I only managed to turn the cache off completely. Is there anyway to > tell Varnish to always purge the homepage if there's a change? > > Thanks in advance for any help. > > My full VCL is below. If anyone has any Wordpress specific changes they'd > like to share then please send them my way! > > Regards > > EB > > > ----------------------------------------------------------------------------------------------------------------------------------- > > backend b96_30_20_4 { > .host = "96.30.20.4"; > .port = "8080"; > .probe = { > .url = "/"; > .interval = 5s; > .timeout = 1 s; > .window = 5; > .threshold = 3; > } > // we include time outs so uploads don't time out > .connect_timeout = 600s; > .first_byte_timeout = 600s; > .between_bytes_timeout = 600s; > } > > backend b96_30_20_5 { > .host = "96.30.20.5"; > .port = "8080"; > .probe = { > .url = "/"; > .interval = 5s; > .timeout = 1 s; > .window = 5; > .threshold = 3; > } > // we include time outs so uploads don't time out > .connect_timeout = 600s; > .first_byte_timeout = 600s; > .between_bytes_timeout = 600s; > } > > backend b96_30_20_6 { > .host = "96.30.20.6"; > .port = "8080"; > .probe = { > .url = "/"; > .interval = 5s; > .timeout = 1 s; > .window = 5; > .threshold = 3; > } > // we include time outs so uploads don't time out > .connect_timeout = 600s; > .first_byte_timeout = 600s; > .between_bytes_timeout = 600s; > } > > backend b96_30_20_7 { > .host = "96.30.20.7"; > .port = "8080"; > .probe = { > .url = "/"; > .interval = 5s; > .timeout = 1 s; > .window = 5; > .threshold = 3; > } > // we include time outs so uploads don't time out > .connect_timeout = 600s; > .first_byte_timeout = 600s; > .between_bytes_timeout = 600s; > } > > backend b96_30_18_15 { > .host = "96.30.18.15"; > .port = "8080"; > .probe = { > .url = "/"; > .interval = 5s; > .timeout = 1 s; > .window = 5; > .threshold = 3; > } > // we include time outs so uploads don't time out > .connect_timeout = 600s; > .first_byte_timeout = 600s; > .between_bytes_timeout = 600s; > } > > acl a96_30_20_4 { > "96.30.20.4"; > } > acl a96_30_20_5 { > "96.30.20.5"; > } > acl a96_30_20_6 { > "96.30.20.6"; > } > acl a96_30_20_7 { > "96.30.20.7"; > } > acl a96_30_18_15 { > "96.30.18.15"; > } > > acl purge { > "96.30.20.4"; > "96.30.20.5"; > "96.30.20.6"; > "96.30.20.7"; > "96.30.18.15"; > } > > sub vcl_recv { > if (server.ip ~ a96_30_20_4) { > set req.backend = b96_30_20_4; > } > if (server.ip ~ a96_30_20_5) { > set req.backend = b96_30_20_5; > } > if (server.ip ~ a96_30_20_6) { > set req.backend = b96_30_20_6; > } > if (server.ip ~ a96_30_20_7) { > set req.backend = b96_30_20_7; > } > if (server.ip ~ a96_30_18_15) { > set req.backend = b96_30_18_15; > } > } > > sub vcl_recv { > if (req.request == "PURGE") { > if (!client.ip ~ purge) { > error 405 "Not allowed."; > } > return(lookup); > } > if (req.url ~ "^/") { > return(pass); > } > if (req.url ~ "^/forum/$") { > return(pass); > } > if (req.url ~ "^/$") { > unset req.http.cookie; > } > } > sub vcl_hit { > if (req.request == "PURGE") { > set obj.ttl = 0s; > error 200 "Purged."; > } > } > sub vcl_miss { > if (req.request == "PURGE") { > error 404 "Not in cache."; > } > if (!(req.url ~ "wp-(login|admin)")) { > unset req.http.cookie; > } > if (req.url ~ > > "^/[^?]+.(jpeg|jpg|png|gif|ico|js|css|txt|gz|zip|lzma|bz2|tgz|tbz|html|htm)(\?.|)$") > { > unset req.http.cookie; > set req.url = regsub(req.url, "\?.$", ""); > } > if (req.url ~ "^/$") { > unset req.http.cookie; > } > } > sub vcl_fetch { > if (req.url ~ "^/$") { > unset beresp.http.set-cookie; > } > if (!(req.url ~ "wp-(login|admin)")) { > unset beresp.http.set-cookie; > } > } > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: < > https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20110708/bf4bd761/attachment-0001.html > > > > ------------------------------ > > Message: 3 > Date: Fri, 08 Jul 2011 12:41:30 +0200 > From: Tollef Fog Heen > To: google at alexus.org > Cc: varnish-misc at varnish-cache.org > Subject: Re: errno = Connection reset by peer > Message-ID: <8739iht5ph.fsf at qurzaw.varnish-software.com> > Content-Type: text/plain; charset=us-ascii > > ]] alexus > > | 70 Debug c "Write error, retval = -1, len = 10212, errno = > | Connection reset by peer" > | > | other then vcl_error(404, unknown virtual host) what that other message > | really mean? > > That the client disconnected before we were done sending the object. > > -- > Tollef Fog Heen > Varnish Software > t: +47 21 98 92 64 > > > > ------------------------------ > > Message: 4 > Date: Fri, 08 Jul 2011 12:44:45 +0200 > From: Tollef Fog Heen > To: Jonathan Hursey > Cc: varnish-misc at varnish-cache.org > Subject: Re: Varnishstat question > Message-ID: <87y609rqzm.fsf at qurzaw.varnish-software.com> > Content-Type: text/plain; charset=us-ascii > > ]] Jonathan Hursey > > Hi, > > | Greetings fellow varnish enthusiasts. I was wondering if anyone could > help > | me determine what would be considered good values for the follow fields > in > | varnishstat output and what these fields indicate: > | > | 1 0.00 0.00 N vcl total > | 1 0.00 0.00 N vcl available > > This just tells you that you have one VCL loaded. > > | 7 . . N total active purges > | 7 0.00 0.00 N new purges added > > You have 7 bans added > > | 6112 0.00 0.02 N objects tested > > And 6112 objects tested against those bans in total. > > Regards, > -- > Tollef Fog Heen > Varnish Software > t: +47 21 98 92 64 > > > > ------------------------------ > > Message: 5 > Date: Thu, 30 Jun 2011 20:44:26 -0400 > From: Lee Trout > To: varnish-misc at varnish-cache.org > Subject: Possible bug with very large headers in varnish-3.0.0 > revision 3bd5997 > Message-ID: > Content-Type: text/plain; charset=ISO-8859-1 > > Hello! > > Please excuse the interruption, but I wanted to see if I may have > stumbled upon a bug or if there is a known issue or configuration > option for requests with very large header values. > > Using varnish-3.0.0 revision 3bd5997 when a large request comes in > (cookie header ~3k characters) the header is lost. I am seeing > "LostHeader c Cookie:" in the logs. > > This paste includes the request and results comparing varnish-3.0.0 > revision 3bd5997 and varnish-2.1.4 SVN 5447M > http://pastebin.com/J2Qn0KGn > > Thank you in advance for your time and suggestions. > > Lee > -- > Lee Trout > http://www.leetrout.com > > > > ------------------------------ > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > End of varnish-misc Digest, Vol 64, Issue 13 > ******************************************** > -------------- next part -------------- An HTML attachment was scrubbed... URL: ------------------------------ _______________________________________________ varnish-misc mailing list varnish-misc at varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc End of varnish-misc Digest, Vol 64, Issue 15 ******************************************** _______________________________________________ varnish-misc mailing list varnish-misc at varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc There is a separate tool called varnishadm that can regularly purge pages. Use that. Although you'll have to figure out how to trigger it on post. Alternatively can't you just cache the front page for only a minute or similar? Stew -------------- next part -------------- An HTML attachment was scrubbed... URL: From everton at connectedinternet.co.uk Sat Jul 9 11:39:40 2011 From: everton at connectedinternet.co.uk (Everton Blair) Date: Sat, 9 Jul 2011 12:39:40 +0100 Subject: [Wordpress]homepage not purging (Everton Blair) In-Reply-To: <20110709092947.fc00127d@groupware.tty.nl> References: <20110709092947.fc00127d@groupware.tty.nl> Message-ID: The problem is, won't that lead to the page never being cached? As it's my most trafficked page, I really want to cache it! On Sat, Jul 9, 2011 at 10:29 AM, Enno van Amerongen wrote: > ** > use a far shorter ttl, 10-15 mins is way too much, use something like 10 > seconds or even less. that will result in max 0.1 req/sec on your backend, I > doubt that's a problem. > > Enno > > ------------------------------ > *From:* Everton Blair [mailto:everton at connectedinternet.co.uk] > *To:* Stewart Robinson [mailto:stewsnooze at gmail.com] > *Cc:* varnish-misc at varnish-cache.org [mailto: > varnish-misc at varnish-cache.org] > *Sent:* Fri, 08 Jul 2011 22:11:07 +0200 > > *Subject:* Re: [Wordpress]homepage not purging (Everton Blair) > > Hi Stewart > > Really appreciate you replying and trying to help as this has been driving > me mad! > > Caching the homepage for a shorter period is a potentially a good idea, and > I could I guess set it to purge say every 10-15 minutes, so it'd only be one > 'unlucky' person who would see an uncached page every 10-15 mins. Not > ideal, but it'd work. It'd be even better if I could purge the page and > then somehow prime the cache again automatically with the page. > > > > On Fri, Jul 8, 2011 at 8:49 PM, Stewart Robinson wrote: > >> >> >> On 8 Jul 2011, at 20:16, Everton Blair >> wrote: >> >> sorry, I meant to say new posts ARE NOT forcing the homepage to >> purge/update, so I need a way to get the homepage to update whenever any >> purge request comes in. >> >> On Fri, Jul 8, 2011 at 8:05 PM, < >> varnish-misc-request at varnish-cache.org> wrote: >> >>> Send varnish-misc mailing list submissions to >>> varnish-misc at varnish-cache.org >>> >>> To subscribe or unsubscribe via the World Wide Web, visit >>> >>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >>> or, via email, send a message with subject or body 'help' to >>> >>> varnish-misc-request at varnish-cache.org >>> >>> You can reach the person managing the list at >>> >>> varnish-misc-owner at varnish-cache.org >>> >>> When replying, please edit your Subject line so it is more specific >>> than "Re: Contents of varnish-misc digest..." >>> >>> >>> Today's Topics: >>> >>> 1. Re: Drupal 7.x & Varnish 3.x (Flatcircle) >>> 2. Varnish + Apache + SSL + Pound? (Scott Wilcox) >>> 3. Re:[Wordpress]homepage not purging (Everton Blair) (Everton Blair) >>> >>> >>> ---------------------------------------------------------------------- >>> >>> Message: 1 >>> Date: Fri, 8 Jul 2011 20:26:18 +0200 >>> From: Flatcircle < flatcircle at hotmail.com> >>> To: Mattias Geniar < mattias at nucleus.be> >>> Cc: varnish-misc at varnish-cache.org >>> Subject: Re: Drupal 7.x & Varnish 3.x >>> Message-ID: >>> Content-Type: text/plain; charset="us-ascii" >>> >>> An HTML attachment was scrubbed... >>> URL: < >>> https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20110708/7af63846/attachment-0001.html >>> > >>> >>> ------------------------------ >>> >>> Message: 2 >>> Date: Fri, 8 Jul 2011 19:37:38 +0100 >>> From: Scott Wilcox < scott at dor.ky> >>> To: < varnish-misc at varnish-cache.org> >>> Subject: Varnish + Apache + SSL + Pound? >>> Message-ID: < <2F426C39-48D0-4D7A-A14C-4BBCA9391837 at dor.ky> >>> 2F426C39-48D0-4D7A-A14C-4BBCA9391837 at dor.ky> >>> Content-Type: text/plain; charset="us-ascii" >>> >>> Hello folks, >>> >>> I've seen this and similar questions asked often and with varying >>> replies. I'd like to have my own scenario looked at. I'm currently in the >>> process of redesigning my work flow for one of my projects. The main web >>> server that serves content is Apache. I plan to have Varnish caching this >>> traffic where applicable. I would like to add a layer of redundancy in >>> regards to alerting users of problems too. I also need to support SSL too. >>> >>> I'm looking at having pound accept connections which is then passed to >>> varnish. Varnish then handles the traffic as normal. >>> >>> I've seen in v3 that basic HTML can be output for errors, but I'd like to >>> include a stylesheet and logo too. I can host these off the site if needed, >>> but I've considered having 'pretty' error messages which include stylesheets >>> and images. I'd obviously have to host these off site which isn't a problem. >>> >>> Is it better to use nginx to accept both HTTP and HTTPS connections and >>> serve the errors/content or does my original plan sound better? >>> >>> What do you use in production and why? >>> >>> Do you recommend pound or nginx? >>> >>> Many thanks for replies in advance. >>> >>> -- >>> Scott Wilcox >>> >>> @dordotky | scott at dor.ky | http://dor.ky >>> +44 (0) 7538 842418 | +1 (646) 827-0580 >>> >>> -------------- next part -------------- >>> An HTML attachment was scrubbed... >>> URL: < >>> https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20110708/ced5f362/attachment-0001.html >>> > >>> >>> ------------------------------ >>> >>> Message: 3 >>> Date: Fri, 8 Jul 2011 20:04:40 +0100 >>> From: Everton Blair < >>> everton at connectedinternet.co.uk> >>> To: varnish-misc at varnish-cache.org >>> Subject: Re:[Wordpress]homepage not purging (Everton Blair) >>> Message-ID: >>> < >>> CAOAN0RQEraiGv9e8K3jTsAuM3YOdgAyBK8iahHZBcZY1sfFTdw at mail.gmail.com> >>> Content-Type: text/plain; charset="iso-8859-1" >>> >>> Can anyone help me with my problem please? It's been causing me major >>> problems. Basically I'm using Wordpress and new posts have been forcing >>> the >>> homepage to purge, I think because I'm using a special template within >>> wordpress to create the homepage and I think Varnish doesn't realise it >>> needs to purge that page >>> >>> What I want to do is force varnish to purge my homepage when any purge >>> requests come in to make sure it stays fresh and to be safe, but I can't >>> get >>> this to work: >>> >>> sub vcl_recv { >>> if (req.request == "PURGE") { >>> if (!client.ip ~ purge) { >>> error 405 "Not allowed."; >>> } >>> if (req.url == "^ windows7news.com/$") { >>> purge(HOW DO I TELL VARNISH TO PURGE >>> >>> http://www.windows7news.com/magazine-homepage/ ????); >>> } >>> if (req.url == "^ windows8news.com/$") { >>> purge(HOW DO I TELL VARNISH TO PURGE >>> >>> http://www.windows8news.com/magazine-homepage/ ???); >>> } >>> return(lookup); >>> } >>> } >>> >>> Thanks for any help. >>> >>> >>> On Fri, Jul 8, 2011 at 11:46 AM, < >>> varnish-misc-request at varnish-cache.org>wrote: >>> >>> > Send varnish-misc mailing list submissions to >>> > varnish-misc at varnish-cache.org >>> > >>> > To subscribe or unsubscribe via the World Wide Web, visit >>> > >>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >>> > or, via email, send a message with subject or body 'help' to >>> > >>> varnish-misc-request at varnish-cache.org >>> > >>> > You can reach the person managing the list at >>> > >>> varnish-misc-owner at varnish-cache.org >>> > >>> > When replying, please edit your Subject line so it is more specific >>> > than "Re: Contents of varnish-misc digest..." >>> > >>> > >>> > Today's Topics: >>> > >>> > 1. Re: Does Varnish uses ETag? (Geoff Simmons) >>> > 2. Fwd: [Wordpress]homepage not purging (Everton Blair) >>> > 3. Re: errno = Connection reset by peer (Tollef Fog Heen) >>> > 4. Re: Varnishstat question (Tollef Fog Heen) >>> > 5. Possible bug with very large headers in varnish-3.0.0 >>> > revision 3bd5997 (Lee Trout) >>> > >>> > >>> > ---------------------------------------------------------------------- >>> > >>> > Message: 1 >>> > Date: Fri, 08 Jul 2011 12:23:54 +0200 >>> > From: Geoff Simmons < geoff at uplex.de> >>> > To: Kurt Kraut < listas at kurtkraut.net> >>> > Cc: varnish-misc at varnish-cache.org >>> > Subject: Re: Does Varnish uses ETag? >>> > Message-ID: < <4E16DABA.2050606 at uplex.de>4E16DABA.2050606 at uplex.de> >>> > Content-Type: text/plain; charset=ISO-8859-1 >>> > >>> > -----BEGIN PGP SIGNED MESSAGE----- >>> > Hash: SHA256 >>> > >>> > On 7/7/11 11:14 PM, Kurt Kraut wrote: >>> > > >>> > > Does Varnish used ETag to check cache freshness? >>> > >>> > Mainline Varnish doesn't, but there's an experimental branch >>> > 'experimental-ims' on the source repository in which it's implemented. >>> > >>> > > - If yes, how does it behave? >>> > >>> > >>> http://www.varnish-cache.org/trac/wiki/BackendConditionalRequests >>> > >>> > As per the RFC, freshness checks against ETags are done with >>> conditional >>> > requests to the backend using If-None-Match. >>> > >>> > > - If not, is it possible to achive through VCL? >>> > >>> > I don't think so, because standard VCL doesn't have a way to access an >>> > object in the cache that has an expired TTL, but otherwise matches the >>> > request, and hence could be refreshed. >>> > >>> > So you don't ordinarily know an ETag in VCL that could be used for the >>> > refresh, and even if you did, and got a 304 response from the backend, >>> > there's no way to tell Varnish to deliver the stale object after all. >>> > >>> > If you get a chance to try the IMS branch, please let us know how it >>> goes. >>> > >>> > >>> > Best, >>> > Geoff >>> > - -- >>> > UPLEX Systemoptimierung >>> > Schwanenwik 24 >>> > 22087 Hamburg >>> > http://uplex.de/ >>> > Mob: +49-176-63690917 >>> > -----BEGIN PGP SIGNATURE----- >>> > Version: GnuPG/MacGPG2 v2.0.14 (Darwin) >>> > Comment: Using GnuPG with Mozilla - >>> http://enigmail.mozdev.org/ >>> > >>> > iQIcBAEBCAAGBQJOFtq5AAoJEOUwvh9pJNURIy4QAKPiZ05kblNGfw7RFJo5Z9fg >>> > 81feHHYKRy880yJrG9up8wNx02+9uku6gNlYkpkPcQqfP/fZu5PpUtjGK7pmCSWG >>> > Jv0bBEL+j5V2L0SaQtoxyyvBkQ5bjQcmrZnuxpYRKRdZeNpjIWY1BLHjMkC3z+sS >>> > 4KKPi1aiGTKYQH8O1FZHz+6dfP8OQseUCs7tLwYM2BYF2onh5H81ys/Adp6520lv >>> > WV2cJWKJO85MqH5C0qmyzGV3ovTFPaU3J4/4idzPxW/vbUDozr5/XjloG7IYzCi6 >>> > Hj5YrfoJlqBREqdAdZ46G/cxS4dKmvMWIO8XaJIYMBISDaXQysXfLq9vLiqODD5l >>> > /1/ijFfEWihwdoxJ/xi1ZgxhA8RYuxQQCMtStVch+QiHpp8ROw04GutXFmA67JeM >>> > aWIBt9WL/cXc/YSP2geSobY89/ROulnKcqV9n5xvxGe1L2bkwq1jQJTtINF6l1oC >>> > lBb9tZAA2JGEeJi8Fv9lcjcrMiLSPzoW16rbzpQNjxxu129Rq8ntteFkm5cqM4sD >>> > 2do8CvRO3YauIY6bUGhRVSjNHCvYf6HvvoZEoKvgJy5drziKyU3qx3WEq3TF+QQU >>> > q+Em6WhiYHQufj6OGnZCFJJ4SKDNUNzBkwzuoXay6jxvUtv3m1tGJyKMK0rYN7vv >>> > 24e8axxNjKucDr5uFx1L >>> > =PW5G >>> > -----END PGP SIGNATURE----- >>> > >>> > >>> > >>> > ------------------------------ >>> > >>> > Message: 2 >>> > Date: Fri, 8 Jul 2011 11:38:57 +0100 >>> > From: Everton Blair < >>> everton at connectedinternet.co.uk> >>> > To: varnish-misc at varnish-cache.org >>> > Subject: Fwd: [Wordpress]homepage not purging >>> > Message-ID: >>> > >>> CCDxm4Xw at mail.gmail.com >>> > > >>> > Content-Type: text/plain; charset="iso-8859-1" >>> > >>> > Hi >>> > >>> > I've just started using Varnish with Wordpress/W3TC, and all is ok, >>> apart >>> > from my homepage not purging when an update is made e.g. a new post. I >>> > think this might be because my homepage is using a static page >>> > >>> www.windows7news.com/magazine-homepage/ as it's homepage and is doing >>> some >>> > magic within Wordpress to show this page when the page >>> > www.windows7news.com/is requested i.e. >>> the server doesn't know that >>> > >>> www.windows7news.com/magazine-homepage/ needs purging. >>> > >>> > I thought I'd temporariliy managed to fix this by adding: >>> > >>> > sub vcl_recv { >>> > if (req.url ~ "^/") { >>> > return(pass); >>> > } >>> > } >>> > >>> > But I only managed to turn the cache off completely. Is there anyway >>> to >>> > tell Varnish to always purge the homepage if there's a change? >>> > >>> > Thanks in advance for any help. >>> > >>> > My full VCL is below. If anyone has any Wordpress specific changes >>> they'd >>> > like to share then please send them my way! >>> > >>> > Regards >>> > >>> > EB >>> > >>> > >>> > >>> ----------------------------------------------------------------------------------------------------------------------------------- >>> > >>> > backend b96_30_20_4 { >>> > .host = "96.30.20.4"; >>> > .port = "8080"; >>> > .probe = { >>> > .url = "/"; >>> > .interval = 5s; >>> > .timeout = 1 s; >>> > .window = 5; >>> > .threshold = 3; >>> > } >>> > // we include time outs so uploads don't time out >>> > .connect_timeout = 600s; >>> > .first_byte_timeout = 600s; >>> > .between_bytes_timeout = 600s; >>> > } >>> > >>> > backend b96_30_20_5 { >>> > .host = "96.30.20.5"; >>> > .port = "8080"; >>> > .probe = { >>> > .url = "/"; >>> > .interval = 5s; >>> > .timeout = 1 s; >>> > .window = 5; >>> > .threshold = 3; >>> > } >>> > // we include time outs so uploads don't time out >>> > .connect_timeout = 600s; >>> > .first_byte_timeout = 600s; >>> > .between_bytes_timeout = 600s; >>> > } >>> > >>> > backend b96_30_20_6 { >>> > .host = "96.30.20.6"; >>> > .port = "8080"; >>> > .probe = { >>> > .url = "/"; >>> > .interval = 5s; >>> > .timeout = 1 s; >>> > .window = 5; >>> > .threshold = 3; >>> > } >>> > // we include time outs so uploads don't time out >>> > .connect_timeout = 600s; >>> > .first_byte_timeout = 600s; >>> > .between_bytes_timeout = 600s; >>> > } >>> > >>> > backend b96_30_20_7 { >>> > .host = "96.30.20.7"; >>> > .port = "8080"; >>> > .probe = { >>> > .url = "/"; >>> > .interval = 5s; >>> > .timeout = 1 s; >>> > .window = 5; >>> > .threshold = 3; >>> > } >>> > // we include time outs so uploads don't time out >>> > .connect_timeout = 600s; >>> > .first_byte_timeout = 600s; >>> > .between_bytes_timeout = 600s; >>> > } >>> > >>> > backend b96_30_18_15 { >>> > .host = "96.30.18.15"; >>> > .port = "8080"; >>> > .probe = { >>> > .url = "/"; >>> > .interval = 5s; >>> > .timeout = 1 s; >>> > .window = 5; >>> > .threshold = 3; >>> > } >>> > // we include time outs so uploads don't time out >>> > .connect_timeout = 600s; >>> > .first_byte_timeout = 600s; >>> > .between_bytes_timeout = 600s; >>> > } >>> > >>> > acl a96_30_20_4 { >>> > "96.30.20.4"; >>> > } >>> > acl a96_30_20_5 { >>> > "96.30.20.5"; >>> > } >>> > acl a96_30_20_6 { >>> > "96.30.20.6"; >>> > } >>> > acl a96_30_20_7 { >>> > "96.30.20.7"; >>> > } >>> > acl a96_30_18_15 { >>> > "96.30.18.15"; >>> > } >>> > >>> > acl purge { >>> > "96.30.20.4"; >>> > "96.30.20.5"; >>> > "96.30.20.6"; >>> > "96.30.20.7"; >>> > "96.30.18.15"; >>> > } >>> > >>> > sub vcl_recv { >>> > if (server.ip ~ a96_30_20_4) { >>> > set req.backend = b96_30_20_4; >>> > } >>> > if (server.ip ~ a96_30_20_5) { >>> > set req.backend = b96_30_20_5; >>> > } >>> > if (server.ip ~ a96_30_20_6) { >>> > set req.backend = b96_30_20_6; >>> > } >>> > if (server.ip ~ a96_30_20_7) { >>> > set req.backend = b96_30_20_7; >>> > } >>> > if (server.ip ~ a96_30_18_15) { >>> > set req.backend = b96_30_18_15; >>> > } >>> > } >>> > >>> > sub vcl_recv { >>> > if (req.request == "PURGE") { >>> > if (!client.ip ~ purge) { >>> > error 405 "Not allowed."; >>> > } >>> > return(lookup); >>> > } >>> > if (req.url ~ "^/") { >>> > return(pass); >>> > } >>> > if (req.url ~ "^/forum/$") { >>> > return(pass); >>> > } >>> > if (req.url ~ "^/$") { >>> > unset req.http.cookie; >>> > } >>> > } >>> > sub vcl_hit { >>> > if (req.request == "PURGE") { >>> > set obj.ttl = 0s; >>> > error 200 "Purged."; >>> > } >>> > } >>> > sub vcl_miss { >>> > if (req.request == "PURGE") { >>> > error 404 "Not in cache."; >>> > } >>> > if (!(req.url ~ "wp-(login|admin)")) { >>> > unset req.http.cookie; >>> > } >>> > if (req.url ~ >>> > >>> > >>> "^/[^?]+.(jpeg|jpg|png|gif|ico|js|css|txt|gz|zip|lzma|bz2|tgz|tbz|html|htm)(\?.|)$") >>> > { >>> > unset req.http.cookie; >>> > set req.url = regsub(req.url, "\?.$", ""); >>> > } >>> > if (req.url ~ "^/$") { >>> > unset req.http.cookie; >>> > } >>> > } >>> > sub vcl_fetch { >>> > if (req.url ~ "^/$") { >>> > unset beresp.http.set-cookie; >>> > } >>> > if (!(req.url ~ "wp-(login|admin)")) { >>> > unset beresp.http.set-cookie; >>> > } >>> > } >>> > -------------- next part -------------- >>> > An HTML attachment was scrubbed... >>> > URL: < >>> > >>> >>> https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20110708/bf4bd761/attachment-0001.html >>> > > >>> > >>> > ------------------------------ >>> > >>> > Message: 3 >>> > Date: Fri, 08 Jul 2011 12:41:30 +0200 >>> > From: Tollef Fog Heen < >>> tfheen at varnish-software.com> >>> > To: google at alexus.org >>> > Cc: varnish-misc at varnish-cache.org >>> > Subject: Re: errno = Connection reset by peer >>> > Message-ID: < <8739iht5ph.fsf at qurzaw.varnish-software.com> >>> 8739iht5ph.fsf at qurzaw.varnish-software.com> >>> > Content-Type: text/plain; charset=us-ascii >>> > >>> > ]] alexus >>> > >>> > | 70 Debug c "Write error, retval = -1, len = 10212, errno = >>> > | Connection reset by peer" >>> > | >>> > | other then vcl_error(404, unknown virtual host) what that other >>> message >>> > | really mean? >>> > >>> > That the client disconnected before we were done sending the object. >>> > >>> > -- >>> > Tollef Fog Heen >>> > Varnish Software >>> > t: +47 21 98 92 64 >>> > >>> > >>> > >>> > ------------------------------ >>> > >>> > Message: 4 >>> > Date: Fri, 08 Jul 2011 12:44:45 +0200 >>> > From: Tollef Fog Heen < >>> tfheen at varnish-software.com> >>> > To: Jonathan Hursey < >>> jonathan.hursey at adrevolution.com> >>> > Cc: varnish-misc at varnish-cache.org >>> > Subject: Re: Varnishstat question >>> > Message-ID: < <87y609rqzm.fsf at qurzaw.varnish-software.com> >>> 87y609rqzm.fsf at qurzaw.varnish-software.com> >>> > Content-Type: text/plain; charset=us-ascii >>> > >>> > ]] Jonathan Hursey >>> > >>> > Hi, >>> > >>> > | Greetings fellow varnish enthusiasts. I was wondering if anyone could >>> > help >>> > | me determine what would be considered good values for the follow >>> fields >>> > in >>> > | varnishstat output and what these fields indicate: >>> > | >>> > | 1 0.00 0.00 N vcl total >>> > | 1 0.00 0.00 N vcl available >>> > >>> > This just tells you that you have one VCL loaded. >>> > >>> > | 7 . . N total active purges >>> > | 7 0.00 0.00 N new purges added >>> > >>> > You have 7 bans added >>> > >>> > | 6112 0.00 0.02 N objects tested >>> > >>> > And 6112 objects tested against those bans in total. >>> > >>> > Regards, >>> > -- >>> > Tollef Fog Heen >>> > Varnish Software >>> > t: +47 21 98 92 64 >>> > >>> > >>> > >>> > ------------------------------ >>> > >>> > Message: 5 >>> > Date: Thu, 30 Jun 2011 20:44:26 -0400 >>> > From: Lee Trout < lee at leetrout.com> >>> > To: varnish-misc at varnish-cache.org >>> > Subject: Possible bug with very large headers in varnish-3.0.0 >>> > revision 3bd5997 >>> > Message-ID: >>> eVPwg at mail.gmail.com> >>> > Content-Type: text/plain; charset=ISO-8859-1 >>> > >>> > Hello! >>> > >>> > Please excuse the interruption, but I wanted to see if I may have >>> > stumbled upon a bug or if there is a known issue or configuration >>> > option for requests with very large header values. >>> > >>> > Using varnish-3.0.0 revision 3bd5997 when a large request comes in >>> > (cookie header ~3k characters) the header is lost. I am seeing >>> > "LostHeader c Cookie:" in the logs. >>> > >>> > This paste includes the request and results comparing varnish-3.0.0 >>> > revision 3bd5997 and varnish-2.1.4 SVN 5447M >>> > http://pastebin.com/J2Qn0KGn >>> > >>> > Thank you in advance for your time and suggestions. >>> > >>> > Lee >>> > -- >>> > Lee Trout >>> > http://www.leetrout.com >>> > >>> > >>> > >>> > ------------------------------ >>> > >>> > _______________________________________________ >>> > varnish-misc mailing list >>> > varnish-misc at varnish-cache.org >>> > >>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >>> > >>> > End of varnish-misc Digest, Vol 64, Issue 13 >>> > ******************************************** >>> > >>> -------------- next part -------------- >>> An HTML attachment was scrubbed... >>> URL: < >>> https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20110708/c40bf9ed/attachment.html >>> > >>> >>> ------------------------------ >>> >>> _______________________________________________ >>> varnish-misc mailing list >>> varnish-misc at varnish-cache.org >>> >>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >>> >>> End of varnish-misc Digest, Vol 64, Issue 15 >>> ******************************************** >>> >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> >> >> There is a separate tool called varnishadm that can regularly purge pages. >> Use that. Although you'll have to figure out how to trigger it on post. >> Alternatively can't you just cache the front page for only a minute or >> similar? >> >> Stew >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From scott at dor.ky Sat Jul 9 21:21:57 2011 From: scott at dor.ky (Scott Wilcox) Date: Sat, 9 Jul 2011 22:21:57 +0100 Subject: A Few Questions Message-ID: 1) Can listening host/port be set in the VCL? 2) I want to do something similar to: sub vcl_hit { set obj.http.X-Cache-Hit = "true"; return (deliver); } sub vcl_miss { set obj.http.X-Cache-Hit = "false"; return (fetch); } for debugging purposes, however when I add that to a VCL, I get: Message from VCC-compiler: 'obj.http.X-Cache-Hit': cannot be set in method 'vcl_hit'. Thanks in advance. -- Scott Wilcox @dordotky | scott at dor.ky | http://dor.ky +44 (0) 7538 842418 | +1 (646) 827-0580 -------------- next part -------------- An HTML attachment was scrubbed... URL: From phk at phk.freebsd.dk Sat Jul 9 21:57:11 2011 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Sat, 09 Jul 2011 21:57:11 +0000 Subject: A Few Questions In-Reply-To: Your message of "Sat, 09 Jul 2011 22:21:57 +0100." Message-ID: <12079.1310248631@critter.freebsd.dk> In message , Scott Wilcox writes: >1) Can listening host/port be set in the VCL? No, but you can set it from the CLI. >sub vcl_hit { > set obj.http.X-Cache-Hit = "true"; > > [...] > >Message from VCC-compiler: >'obj.http.X-Cache-Hit': cannot be set in method 'vcl_hit'. And it's right: You don't have an object to modify at that point. Set req.http.X-Cache-Hit instead and copy that to the response in vcl_deliver{} -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From enno at tty.nl Sun Jul 10 11:19:45 2011 From: enno at tty.nl (Enno van Amerongen) Date: Sun, 10 Jul 2011 13:19:45 +0200 Subject: [Wordpress]homepage not purging (Everton Blair) In-Reply-To: Message-ID: <20110710111945.a139e4c2@groupware.tty.nl> it'll be cached for 10 seconds every time, so if we assume you get 20 homepage requests per second, u'll have a hitrate of 99.5%. The problem is, won't that lead to the page never being cached? As it's my most trafficked page, I really want to cache it! On Sat, Jul 9, 2011 at 10:29 AM, Enno van Amerongen wrote: use a far shorter ttl, 10-15 mins is way too much, use something like 10 seconds or even less. that will result in max 0.1 req/sec on your backend, I doubt that's a problem. Enno _____ From: Everton Blair [mailto:everton at connectedinternet.co.uk] To: Stewart Robinson [mailto:stewsnooze at gmail.com] Cc: varnish-misc at varnish-cache.org [mailto:varnish-misc at varnish-cache.org] Sent: Fri, 08 Jul 2011 22:11:07 +0200 Subject: Re: [Wordpress]homepage not purging (Everton Blair) Hi Stewart Really appreciate you replying and trying to help as this has been driving me mad! Caching the homepage for a shorter period is a potentially a good idea, and I could I guess set it to purge say every 10-15 minutes, so it'd only be one 'unlucky' person who would see an uncached page every 10-15 mins. Not ideal, but it'd work. It'd be even better if I could purge the page and then somehow prime the cache again automatically with the page. On Fri, Jul 8, 2011 at 8:49 PM, Stewart Robinson wrote: On 8 Jul 2011, at 20:16, Everton Blair wrote: sorry, I meant to say new posts ARE NOT forcing the homepage to purge/update, so I need a way to get the homepage to update whenever any purge request comes in. On Fri, Jul 8, 2011 at 8:05 PM, wrote: Send varnish-misc mailing list submissions to varnish-misc at varnish-cache.org To subscribe or unsubscribe via the World Wide Web, visit https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc or, via email, send a message with subject or body 'help' to varnish-misc-request at varnish-cache.org You can reach the person managing the list at varnish-misc-owner at varnish-cache.org When replying, please edit your Subject line so it is more specific than "Re: Contents of varnish-misc digest..." Today's Topics: 1. Re: Drupal 7.x & Varnish 3.x (Flatcircle) 2. Varnish + Apache + SSL + Pound? (Scott Wilcox) 3. Re:[Wordpress]homepage not purging (Everton Blair) (Everton Blair) ---------------------------------------------------------------------- Message: 1 Date: Fri, 8 Jul 2011 20:26:18 +0200 From: Flatcircle To: Mattias Geniar Cc: varnish-misc at varnish-cache.org Subject: Re: Drupal 7.x & Varnish 3.x Message-ID: Content-Type: text/plain; charset="us-ascii" An HTML attachment was scrubbed... URL: ------------------------------ Message: 2 Date: Fri, 8 Jul 2011 19:37:38 +0100 From: Scott Wilcox To: Subject: Varnish + Apache + SSL + Pound? Message-ID: <2F426C39-48D0-4D7A-A14C-4BBCA9391837 at dor.ky> Content-Type: text/plain; charset="us-ascii" Hello folks, I've seen this and similar questions asked often and with varying replies. I'd like to have my own scenario looked at. I'm currently in the process of redesigning my work flow for one of my projects. The main web server that serves content is Apache. I plan to have Varnish caching this traffic where applicable. I would like to add a layer of redundancy in regards to alerting users of problems too. I also need to support SSL too. I'm looking at having pound accept connections which is then passed to varnish. Varnish then handles the traffic as normal. I've seen in v3 that basic HTML can be output for errors, but I'd like to include a stylesheet and logo too. I can host these off the site if needed, but I've considered having 'pretty' error messages which include stylesheets and images. I'd obviously have to host these off site which isn't a problem. Is it better to use nginx to accept both HTTP and HTTPS connections and serve the errors/content or does my original plan sound better? What do you use in production and why? Do you recommend pound or nginx? Many thanks for replies in advance. -- Scott Wilcox @dordotky | scott at dor.ky | http://dor.ky +44 (0) 7538 842418 | +1 (646) 827-0580 -------------- next part -------------- An HTML attachment was scrubbed... URL: ------------------------------ Message: 3 Date: Fri, 8 Jul 2011 20:04:40 +0100 From: Everton Blair To: varnish-misc at varnish-cache.org Subject: Re:[Wordpress]homepage not purging (Everton Blair) Message-ID: Content-Type: text/plain; charset="iso-8859-1" Can anyone help me with my problem please? It's been causing me major problems. Basically I'm using Wordpress and new posts have been forcing the homepage to purge, I think because I'm using a special template within wordpress to create the homepage and I think Varnish doesn't realise it needs to purge that page What I want to do is force varnish to purge my homepage when any purge requests come in to make sure it stays fresh and to be safe, but I can't get this to work: sub vcl_recv { if (req.request == "PURGE") { if (!client.ip ~ purge) { error 405 "Not allowed."; } if (req.url == "^windows7news.com/$") { purge(HOW DO I TELL VARNISH TO PURGE http://www.windows7news.com/magazine-homepage/ ????); } if (req.url == "^windows8news.com/$") { purge(HOW DO I TELL VARNISH TO PURGE http://www.windows8news.com/magazine-homepage/ ???); } return(lookup); } } Thanks for any help. On Fri, Jul 8, 2011 at 11:46 AM, wrote: > Send varnish-misc mailing list submissions to > varnish-misc at varnish-cache.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > or, via email, send a message with subject or body 'help' to > varnish-misc-request at varnish-cache.org > > You can reach the person managing the list at > varnish-misc-owner at varnish-cache.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of varnish-misc digest..." > > > Today's Topics: > > 1. Re: Does Varnish uses ETag? (Geoff Simmons) > 2. Fwd: [Wordpress]homepage not purging (Everton Blair) > 3. Re: errno = Connection reset by peer (Tollef Fog Heen) > 4. Re: Varnishstat question (Tollef Fog Heen) > 5. Possible bug with very large headers in varnish-3.0.0 > revision 3bd5997 (Lee Trout) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 08 Jul 2011 12:23:54 +0200 > From: Geoff Simmons > To: Kurt Kraut > Cc: varnish-misc at varnish-cache.org > Subject: Re: Does Varnish uses ETag? > Message-ID: <4E16DABA.2050606 at uplex.de> > Content-Type: text/plain; charset=ISO-8859-1 > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > On 7/7/11 11:14 PM, Kurt Kraut wrote: > > > > Does Varnish used ETag to check cache freshness? > > Mainline Varnish doesn't, but there's an experimental branch > 'experimental-ims' on the source repository in which it's implemented. > > > - If yes, how does it behave? > > http://www.varnish-cache.org/trac/wiki/BackendConditionalRequests > > As per the RFC, freshness checks against ETags are done with conditional > requests to the backend using If-None-Match. > > > - If not, is it possible to achive through VCL? > > I don't think so, because standard VCL doesn't have a way to access an > object in the cache that has an expired TTL, but otherwise matches the > request, and hence could be refreshed. > > So you don't ordinarily know an ETag in VCL that could be used for the > refresh, and even if you did, and got a 304 response from the backend, > there's no way to tell Varnish to deliver the stale object after all. > > If you get a chance to try the IMS branch, please let us know how it goes. > > > Best, > Geoff > - -- > UPLEX Systemoptimierung > Schwanenwik 24 > 22087 Hamburg > http://uplex.de/ > Mob: +49-176-63690917 > -----BEGIN PGP SIGNATURE----- > Version: GnuPG/MacGPG2 v2.0.14 (Darwin) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iQIcBAEBCAAGBQJOFtq5AAoJEOUwvh9pJNURIy4QAKPiZ05kblNGfw7RFJo5Z9fg > 81feHHYKRy880yJrG9up8wNx02+9uku6gNlYkpkPcQqfP/fZu5PpUtjGK7pmCSWG > Jv0bBEL+j5V2L0SaQtoxyyvBkQ5bjQcmrZnuxpYRKRdZeNpjIWY1BLHjMkC3z+sS > 4KKPi1aiGTKYQH8O1FZHz+6dfP8OQseUCs7tLwYM2BYF2onh5H81ys/Adp6520lv > WV2cJWKJO85MqH5C0qmyzGV3ovTFPaU3J4/4idzPxW/vbUDozr5/XjloG7IYzCi6 > Hj5YrfoJlqBREqdAdZ46G/cxS4dKmvMWIO8XaJIYMBISDaXQysXfLq9vLiqODD5l > /1/ijFfEWihwdoxJ/xi1ZgxhA8RYuxQQCMtStVch+QiHpp8ROw04GutXFmA67JeM > aWIBt9WL/cXc/YSP2geSobY89/ROulnKcqV9n5xvxGe1L2bkwq1jQJTtINF6l1oC > lBb9tZAA2JGEeJi8Fv9lcjcrMiLSPzoW16rbzpQNjxxu129Rq8ntteFkm5cqM4sD > 2do8CvRO3YauIY6bUGhRVSjNHCvYf6HvvoZEoKvgJy5drziKyU3qx3WEq3TF+QQU > q+Em6WhiYHQufj6OGnZCFJJ4SKDNUNzBkwzuoXay6jxvUtv3m1tGJyKMK0rYN7vv > 24e8axxNjKucDr5uFx1L > =PW5G > -----END PGP SIGNATURE----- > > > > ------------------------------ > > Message: 2 > Date: Fri, 8 Jul 2011 11:38:57 +0100 > From: Everton Blair > To: varnish-misc at varnish-cache.org > Subject: Fwd: [Wordpress]homepage not purging > Message-ID: > > > Content-Type: text/plain; charset="iso-8859-1" > > Hi > > I've just started using Varnish with Wordpress/W3TC, and all is ok, apart > from my homepage not purging when an update is made e.g. a new post. I > think this might be because my homepage is using a static page > www.windows7news.com/magazine-homepage/ as it's homepage and is doing some > magic within Wordpress to show this page when the page > www.windows7news.com/is requested i.e. the server doesn't know that > www.windows7news.com/magazine-homepage/ needs purging. > > I thought I'd temporariliy managed to fix this by adding: > > sub vcl_recv { > if (req.url ~ "^/") { > return(pass); > } > } > > But I only managed to turn the cache off completely. Is there anyway to > tell Varnish to always purge the homepage if there's a change? > > Thanks in advance for any help. > > My full VCL is below. If anyone has any Wordpress specific changes they'd > like to share then please send them my way! > > Regards > > EB > > > ----------------------------------------------------------------------------------------------------------------------------------- > > backend b96_30_20_4 { > .host = "96.30.20.4"; > .port = "8080"; > .probe = { > .url = "/"; > .interval = 5s; > .timeout = 1 s; > .window = 5; > .threshold = 3; > } > // we include time outs so uploads don't time out > .connect_timeout = 600s; > .first_byte_timeout = 600s; > .between_bytes_timeout = 600s; > } > > backend b96_30_20_5 { > .host = "96.30.20.5"; > .port = "8080"; > .probe = { > .url = "/"; > .interval = 5s; > .timeout = 1 s; > .window = 5; > .threshold = 3; > } > // we include time outs so uploads don't time out > .connect_timeout = 600s; > .first_byte_timeout = 600s; > .between_bytes_timeout = 600s; > } > > backend b96_30_20_6 { > .host = "96.30.20.6"; > .port = "8080"; > .probe = { > .url = "/"; > .interval = 5s; > .timeout = 1 s; > .window = 5; > .threshold = 3; > } > // we include time outs so uploads don't time out > .connect_timeout = 600s; > .first_byte_timeout = 600s; > .between_bytes_timeout = 600s; > } > > backend b96_30_20_7 { > .host = "96.30.20.7"; > .port = "8080"; > .probe = { > .url = "/"; > .interval = 5s; > .timeout = 1 s; > .window = 5; > .threshold = 3; > } > // we include time outs so uploads don't time out > .connect_timeout = 600s; > .first_byte_timeout = 600s; > .between_bytes_timeout = 600s; > } > > backend b96_30_18_15 { > .host = "96.30.18.15"; > .port = "8080"; > .probe = { > .url = "/"; > .interval = 5s; > .timeout = 1 s; > .window = 5; > .threshold = 3; > } > // we include time outs so uploads don't time out > .connect_timeout = 600s; > .first_byte_timeout = 600s; > .between_bytes_timeout = 600s; > } > > acl a96_30_20_4 { > "96.30.20.4"; > } > acl a96_30_20_5 { > "96.30.20.5"; > } > acl a96_30_20_6 { > "96.30.20.6"; > } > acl a96_30_20_7 { > "96.30.20.7"; > } > acl a96_30_18_15 { > "96.30.18.15"; > } > > acl purge { > "96.30.20.4"; > "96.30.20.5"; > "96.30.20.6"; > "96.30.20.7"; > "96.30.18.15"; > } > > sub vcl_recv { > if (server.ip ~ a96_30_20_4) { > set req.backend = b96_30_20_4; > } > if (server.ip ~ a96_30_20_5) { > set req.backend = b96_30_20_5; > } > if (server.ip ~ a96_30_20_6) { > set req.backend = b96_30_20_6; > } > if (server.ip ~ a96_30_20_7) { > set req.backend = b96_30_20_7; > } > if (server.ip ~ a96_30_18_15) { > set req.backend = b96_30_18_15; > } > } > > sub vcl_recv { > if (req.request == "PURGE") { > if (!client.ip ~ purge) { > error 405 "Not allowed."; > } > return(lookup); > } > if (req.url ~ "^/") { > return(pass); > } > if (req.url ~ "^/forum/$") { > return(pass); > } > if (req.url ~ "^/$") { > unset req.http.cookie; > } > } > sub vcl_hit { > if (req.request == "PURGE") { > set obj.ttl = 0s; > error 200 "Purged."; > } > } > sub vcl_miss { > if (req.request == "PURGE") { > error 404 "Not in cache."; > } > if (!(req.url ~ "wp-(login|admin)")) { > unset req.http.cookie; > } > if (req.url ~ > > "^/[^?]+.(jpeg|jpg|png|gif|ico|js|css|txt|gz|zip|lzma|bz2|tgz|tbz|html|htm)(\?.|)$") > { > unset req.http.cookie; > set req.url = regsub(req.url, "\?.$", ""); > } > if (req.url ~ "^/$") { > unset req.http.cookie; > } > } > sub vcl_fetch { > if (req.url ~ "^/$") { > unset beresp.http.set-cookie; > } > if (!(req.url ~ "wp-(login|admin)")) { > unset beresp.http.set-cookie; > } > } > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: < > https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20110708/bf4bd761/attachment-0001.html > > > > ------------------------------ > > Message: 3 > Date: Fri, 08 Jul 2011 12:41:30 +0200 > From: Tollef Fog Heen > To: google at alexus.org > Cc: varnish-misc at varnish-cache.org > Subject: Re: errno = Connection reset by peer > Message-ID: <8739iht5ph.fsf at qurzaw.varnish-software.com> > Content-Type: text/plain; charset=us-ascii > > ]] alexus > > | 70 Debug c "Write error, retval = -1, len = 10212, errno = > | Connection reset by peer" > | > | other then vcl_error(404, unknown virtual host) what that other message > | really mean? > > That the client disconnected before we were done sending the object. > > -- > Tollef Fog Heen > Varnish Software > t: +47 21 98 92 64 > > > > ------------------------------ > > Message: 4 > Date: Fri, 08 Jul 2011 12:44:45 +0200 > From: Tollef Fog Heen > To: Jonathan Hursey > Cc: varnish-misc at varnish-cache.org > Subject: Re: Varnishstat question > Message-ID: <87y609rqzm.fsf at qurzaw.varnish-software.com> > Content-Type: text/plain; charset=us-ascii > > ]] Jonathan Hursey > > Hi, > > | Greetings fellow varnish enthusiasts. I was wondering if anyone could > help > | me determine what would be considered good values for the follow fields > in > | varnishstat output and what these fields indicate: > | > | 1 0.00 0.00 N vcl total > | 1 0.00 0.00 N vcl available > > This just tells you that you have one VCL loaded. > > | 7 . . N total active purges > | 7 0.00 0.00 N new purges added > > You have 7 bans added > > | 6112 0.00 0.02 N objects tested > > And 6112 objects tested against those bans in total. > > Regards, > -- > Tollef Fog Heen > Varnish Software > t: +47 21 98 92 64 > > > > ------------------------------ > > Message: 5 > Date: Thu, 30 Jun 2011 20:44:26 -0400 > From: Lee Trout > To: varnish-misc at varnish-cache.org > Subject: Possible bug with very large headers in varnish-3.0.0 > revision 3bd5997 > Message-ID: > Content-Type: text/plain; charset=ISO-8859-1 > > Hello! > > Please excuse the interruption, but I wanted to see if I may have > stumbled upon a bug or if there is a known issue or configuration > option for requests with very large header values. > > Using varnish-3.0.0 revision 3bd5997 when a large request comes in > (cookie header ~3k characters) the header is lost. I am seeing > "LostHeader c Cookie:" in the logs. > > This paste includes the request and results comparing varnish-3.0.0 > revision 3bd5997 and varnish-2.1.4 SVN 5447M > http://pastebin.com/J2Qn0KGn > > Thank you in advance for your time and suggestions. > > Lee > -- > Lee Trout > http://www.leetrout.com > > > > ------------------------------ > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > End of varnish-misc Digest, Vol 64, Issue 13 > ******************************************** > -------------- next part -------------- An HTML attachment was scrubbed... URL: ------------------------------ _______________________________________________ varnish-misc mailing list varnish-misc at varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc End of varnish-misc Digest, Vol 64, Issue 15 ******************************************** _______________________________________________ varnish-misc mailing list varnish-misc at varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc There is a separate tool called varnishadm that can regularly purge pages. Use that. Although you'll have to figure out how to trigger it on post. Alternatively can't you just cache the front page for only a minute or similar? Stew -------------- next part -------------- An HTML attachment was scrubbed... URL: From everton at connectedinternet.co.uk Sun Jul 10 11:41:45 2011 From: everton at connectedinternet.co.uk (Everton Blair) Date: Sun, 10 Jul 2011 12:41:45 +0100 Subject: [Wordpress]homepage not purging (Everton Blair) In-Reply-To: <20110709092947.fc00127d@groupware.tty.nl> References: <20110709092947.fc00127d@groupware.tty.nl> Message-ID: Hi I managed to fix this by using this VCL file https://github.com/mattiasgeniar/varnish-3.0-configuration-templates. Not sure how, but it worked!!! Thanks for all the help guys. Regards Everton On Sat, Jul 9, 2011 at 10:29 AM, Enno van Amerongen wrote: > ** > use a far shorter ttl, 10-15 mins is way too much, use something like 10 > seconds or even less. that will result in max 0.1 req/sec on your backend, I > doubt that's a problem. > > Enno > > ------------------------------ > *From:* Everton Blair [mailto:everton at connectedinternet.co.uk] > *To:* Stewart Robinson [mailto:stewsnooze at gmail.com] > *Cc:* varnish-misc at varnish-cache.org [mailto: > varnish-misc at varnish-cache.org] > *Sent:* Fri, 08 Jul 2011 22:11:07 +0200 > > *Subject:* Re: [Wordpress]homepage not purging (Everton Blair) > > Hi Stewart > > Really appreciate you replying and trying to help as this has been driving > me mad! > > Caching the homepage for a shorter period is a potentially a good idea, and > I could I guess set it to purge say every 10-15 minutes, so it'd only be one > 'unlucky' person who would see an uncached page every 10-15 mins. Not > ideal, but it'd work. It'd be even better if I could purge the page and > then somehow prime the cache again automatically with the page. > > > > On Fri, Jul 8, 2011 at 8:49 PM, Stewart Robinson wrote: > >> >> >> On 8 Jul 2011, at 20:16, Everton Blair >> wrote: >> >> sorry, I meant to say new posts ARE NOT forcing the homepage to >> purge/update, so I need a way to get the homepage to update whenever any >> purge request comes in. >> >> On Fri, Jul 8, 2011 at 8:05 PM, < >> varnish-misc-request at varnish-cache.org> wrote: >> >>> Send varnish-misc mailing list submissions to >>> varnish-misc at varnish-cache.org >>> >>> To subscribe or unsubscribe via the World Wide Web, visit >>> >>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >>> or, via email, send a message with subject or body 'help' to >>> >>> varnish-misc-request at varnish-cache.org >>> >>> You can reach the person managing the list at >>> >>> varnish-misc-owner at varnish-cache.org >>> >>> When replying, please edit your Subject line so it is more specific >>> than "Re: Contents of varnish-misc digest..." >>> >>> >>> Today's Topics: >>> >>> 1. Re: Drupal 7.x & Varnish 3.x (Flatcircle) >>> 2. Varnish + Apache + SSL + Pound? (Scott Wilcox) >>> 3. Re:[Wordpress]homepage not purging (Everton Blair) (Everton Blair) >>> >>> >>> ---------------------------------------------------------------------- >>> >>> Message: 1 >>> Date: Fri, 8 Jul 2011 20:26:18 +0200 >>> From: Flatcircle < flatcircle at hotmail.com> >>> To: Mattias Geniar < mattias at nucleus.be> >>> Cc: varnish-misc at varnish-cache.org >>> Subject: Re: Drupal 7.x & Varnish 3.x >>> Message-ID: >>> Content-Type: text/plain; charset="us-ascii" >>> >>> An HTML attachment was scrubbed... >>> URL: < >>> https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20110708/7af63846/attachment-0001.html >>> > >>> >>> ------------------------------ >>> >>> Message: 2 >>> Date: Fri, 8 Jul 2011 19:37:38 +0100 >>> From: Scott Wilcox < scott at dor.ky> >>> To: < varnish-misc at varnish-cache.org> >>> Subject: Varnish + Apache + SSL + Pound? >>> Message-ID: < <2F426C39-48D0-4D7A-A14C-4BBCA9391837 at dor.ky> >>> 2F426C39-48D0-4D7A-A14C-4BBCA9391837 at dor.ky> >>> Content-Type: text/plain; charset="us-ascii" >>> >>> Hello folks, >>> >>> I've seen this and similar questions asked often and with varying >>> replies. I'd like to have my own scenario looked at. I'm currently in the >>> process of redesigning my work flow for one of my projects. The main web >>> server that serves content is Apache. I plan to have Varnish caching this >>> traffic where applicable. I would like to add a layer of redundancy in >>> regards to alerting users of problems too. I also need to support SSL too. >>> >>> I'm looking at having pound accept connections which is then passed to >>> varnish. Varnish then handles the traffic as normal. >>> >>> I've seen in v3 that basic HTML can be output for errors, but I'd like to >>> include a stylesheet and logo too. I can host these off the site if needed, >>> but I've considered having 'pretty' error messages which include stylesheets >>> and images. I'd obviously have to host these off site which isn't a problem. >>> >>> Is it better to use nginx to accept both HTTP and HTTPS connections and >>> serve the errors/content or does my original plan sound better? >>> >>> What do you use in production and why? >>> >>> Do you recommend pound or nginx? >>> >>> Many thanks for replies in advance. >>> >>> -- >>> Scott Wilcox >>> >>> @dordotky | scott at dor.ky | http://dor.ky >>> +44 (0) 7538 842418 | +1 (646) 827-0580 >>> >>> -------------- next part -------------- >>> An HTML attachment was scrubbed... >>> URL: < >>> https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20110708/ced5f362/attachment-0001.html >>> > >>> >>> ------------------------------ >>> >>> Message: 3 >>> Date: Fri, 8 Jul 2011 20:04:40 +0100 >>> From: Everton Blair < >>> everton at connectedinternet.co.uk> >>> To: varnish-misc at varnish-cache.org >>> Subject: Re:[Wordpress]homepage not purging (Everton Blair) >>> Message-ID: >>> < >>> CAOAN0RQEraiGv9e8K3jTsAuM3YOdgAyBK8iahHZBcZY1sfFTdw at mail.gmail.com> >>> Content-Type: text/plain; charset="iso-8859-1" >>> >>> Can anyone help me with my problem please? It's been causing me major >>> problems. Basically I'm using Wordpress and new posts have been forcing >>> the >>> homepage to purge, I think because I'm using a special template within >>> wordpress to create the homepage and I think Varnish doesn't realise it >>> needs to purge that page >>> >>> What I want to do is force varnish to purge my homepage when any purge >>> requests come in to make sure it stays fresh and to be safe, but I can't >>> get >>> this to work: >>> >>> sub vcl_recv { >>> if (req.request == "PURGE") { >>> if (!client.ip ~ purge) { >>> error 405 "Not allowed."; >>> } >>> if (req.url == "^ windows7news.com/$") { >>> purge(HOW DO I TELL VARNISH TO PURGE >>> >>> http://www.windows7news.com/magazine-homepage/ ????); >>> } >>> if (req.url == "^ windows8news.com/$") { >>> purge(HOW DO I TELL VARNISH TO PURGE >>> >>> http://www.windows8news.com/magazine-homepage/ ???); >>> } >>> return(lookup); >>> } >>> } >>> >>> Thanks for any help. >>> >>> >>> On Fri, Jul 8, 2011 at 11:46 AM, < >>> varnish-misc-request at varnish-cache.org>wrote: >>> >>> > Send varnish-misc mailing list submissions to >>> > varnish-misc at varnish-cache.org >>> > >>> > To subscribe or unsubscribe via the World Wide Web, visit >>> > >>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >>> > or, via email, send a message with subject or body 'help' to >>> > >>> varnish-misc-request at varnish-cache.org >>> > >>> > You can reach the person managing the list at >>> > >>> varnish-misc-owner at varnish-cache.org >>> > >>> > When replying, please edit your Subject line so it is more specific >>> > than "Re: Contents of varnish-misc digest..." >>> > >>> > >>> > Today's Topics: >>> > >>> > 1. Re: Does Varnish uses ETag? (Geoff Simmons) >>> > 2. Fwd: [Wordpress]homepage not purging (Everton Blair) >>> > 3. Re: errno = Connection reset by peer (Tollef Fog Heen) >>> > 4. Re: Varnishstat question (Tollef Fog Heen) >>> > 5. Possible bug with very large headers in varnish-3.0.0 >>> > revision 3bd5997 (Lee Trout) >>> > >>> > >>> > ---------------------------------------------------------------------- >>> > >>> > Message: 1 >>> > Date: Fri, 08 Jul 2011 12:23:54 +0200 >>> > From: Geoff Simmons < geoff at uplex.de> >>> > To: Kurt Kraut < listas at kurtkraut.net> >>> > Cc: varnish-misc at varnish-cache.org >>> > Subject: Re: Does Varnish uses ETag? >>> > Message-ID: < <4E16DABA.2050606 at uplex.de>4E16DABA.2050606 at uplex.de> >>> > Content-Type: text/plain; charset=ISO-8859-1 >>> > >>> > -----BEGIN PGP SIGNED MESSAGE----- >>> > Hash: SHA256 >>> > >>> > On 7/7/11 11:14 PM, Kurt Kraut wrote: >>> > > >>> > > Does Varnish used ETag to check cache freshness? >>> > >>> > Mainline Varnish doesn't, but there's an experimental branch >>> > 'experimental-ims' on the source repository in which it's implemented. >>> > >>> > > - If yes, how does it behave? >>> > >>> > >>> http://www.varnish-cache.org/trac/wiki/BackendConditionalRequests >>> > >>> > As per the RFC, freshness checks against ETags are done with >>> conditional >>> > requests to the backend using If-None-Match. >>> > >>> > > - If not, is it possible to achive through VCL? >>> > >>> > I don't think so, because standard VCL doesn't have a way to access an >>> > object in the cache that has an expired TTL, but otherwise matches the >>> > request, and hence could be refreshed. >>> > >>> > So you don't ordinarily know an ETag in VCL that could be used for the >>> > refresh, and even if you did, and got a 304 response from the backend, >>> > there's no way to tell Varnish to deliver the stale object after all. >>> > >>> > If you get a chance to try the IMS branch, please let us know how it >>> goes. >>> > >>> > >>> > Best, >>> > Geoff >>> > - -- >>> > UPLEX Systemoptimierung >>> > Schwanenwik 24 >>> > 22087 Hamburg >>> > http://uplex.de/ >>> > Mob: +49-176-63690917 >>> > -----BEGIN PGP SIGNATURE----- >>> > Version: GnuPG/MacGPG2 v2.0.14 (Darwin) >>> > Comment: Using GnuPG with Mozilla - >>> http://enigmail.mozdev.org/ >>> > >>> > iQIcBAEBCAAGBQJOFtq5AAoJEOUwvh9pJNURIy4QAKPiZ05kblNGfw7RFJo5Z9fg >>> > 81feHHYKRy880yJrG9up8wNx02+9uku6gNlYkpkPcQqfP/fZu5PpUtjGK7pmCSWG >>> > Jv0bBEL+j5V2L0SaQtoxyyvBkQ5bjQcmrZnuxpYRKRdZeNpjIWY1BLHjMkC3z+sS >>> > 4KKPi1aiGTKYQH8O1FZHz+6dfP8OQseUCs7tLwYM2BYF2onh5H81ys/Adp6520lv >>> > WV2cJWKJO85MqH5C0qmyzGV3ovTFPaU3J4/4idzPxW/vbUDozr5/XjloG7IYzCi6 >>> > Hj5YrfoJlqBREqdAdZ46G/cxS4dKmvMWIO8XaJIYMBISDaXQysXfLq9vLiqODD5l >>> > /1/ijFfEWihwdoxJ/xi1ZgxhA8RYuxQQCMtStVch+QiHpp8ROw04GutXFmA67JeM >>> > aWIBt9WL/cXc/YSP2geSobY89/ROulnKcqV9n5xvxGe1L2bkwq1jQJTtINF6l1oC >>> > lBb9tZAA2JGEeJi8Fv9lcjcrMiLSPzoW16rbzpQNjxxu129Rq8ntteFkm5cqM4sD >>> > 2do8CvRO3YauIY6bUGhRVSjNHCvYf6HvvoZEoKvgJy5drziKyU3qx3WEq3TF+QQU >>> > q+Em6WhiYHQufj6OGnZCFJJ4SKDNUNzBkwzuoXay6jxvUtv3m1tGJyKMK0rYN7vv >>> > 24e8axxNjKucDr5uFx1L >>> > =PW5G >>> > -----END PGP SIGNATURE----- >>> > >>> > >>> > >>> > ------------------------------ >>> > >>> > Message: 2 >>> > Date: Fri, 8 Jul 2011 11:38:57 +0100 >>> > From: Everton Blair < >>> everton at connectedinternet.co.uk> >>> > To: varnish-misc at varnish-cache.org >>> > Subject: Fwd: [Wordpress]homepage not purging >>> > Message-ID: >>> > >>> CCDxm4Xw at mail.gmail.com >>> > > >>> > Content-Type: text/plain; charset="iso-8859-1" >>> > >>> > Hi >>> > >>> > I've just started using Varnish with Wordpress/W3TC, and all is ok, >>> apart >>> > from my homepage not purging when an update is made e.g. a new post. I >>> > think this might be because my homepage is using a static page >>> > >>> www.windows7news.com/magazine-homepage/ as it's homepage and is doing >>> some >>> > magic within Wordpress to show this page when the page >>> > www.windows7news.com/is requested i.e. >>> the server doesn't know that >>> > >>> www.windows7news.com/magazine-homepage/ needs purging. >>> > >>> > I thought I'd temporariliy managed to fix this by adding: >>> > >>> > sub vcl_recv { >>> > if (req.url ~ "^/") { >>> > return(pass); >>> > } >>> > } >>> > >>> > But I only managed to turn the cache off completely. Is there anyway >>> to >>> > tell Varnish to always purge the homepage if there's a change? >>> > >>> > Thanks in advance for any help. >>> > >>> > My full VCL is below. If anyone has any Wordpress specific changes >>> they'd >>> > like to share then please send them my way! >>> > >>> > Regards >>> > >>> > EB >>> > >>> > >>> > >>> ----------------------------------------------------------------------------------------------------------------------------------- >>> > >>> > backend b96_30_20_4 { >>> > .host = "96.30.20.4"; >>> > .port = "8080"; >>> > .probe = { >>> > .url = "/"; >>> > .interval = 5s; >>> > .timeout = 1 s; >>> > .window = 5; >>> > .threshold = 3; >>> > } >>> > // we include time outs so uploads don't time out >>> > .connect_timeout = 600s; >>> > .first_byte_timeout = 600s; >>> > .between_bytes_timeout = 600s; >>> > } >>> > >>> > backend b96_30_20_5 { >>> > .host = "96.30.20.5"; >>> > .port = "8080"; >>> > .probe = { >>> > .url = "/"; >>> > .interval = 5s; >>> > .timeout = 1 s; >>> > .window = 5; >>> > .threshold = 3; >>> > } >>> > // we include time outs so uploads don't time out >>> > .connect_timeout = 600s; >>> > .first_byte_timeout = 600s; >>> > .between_bytes_timeout = 600s; >>> > } >>> > >>> > backend b96_30_20_6 { >>> > .host = "96.30.20.6"; >>> > .port = "8080"; >>> > .probe = { >>> > .url = "/"; >>> > .interval = 5s; >>> > .timeout = 1 s; >>> > .window = 5; >>> > .threshold = 3; >>> > } >>> > // we include time outs so uploads don't time out >>> > .connect_timeout = 600s; >>> > .first_byte_timeout = 600s; >>> > .between_bytes_timeout = 600s; >>> > } >>> > >>> > backend b96_30_20_7 { >>> > .host = "96.30.20.7"; >>> > .port = "8080"; >>> > .probe = { >>> > .url = "/"; >>> > .interval = 5s; >>> > .timeout = 1 s; >>> > .window = 5; >>> > .threshold = 3; >>> > } >>> > // we include time outs so uploads don't time out >>> > .connect_timeout = 600s; >>> > .first_byte_timeout = 600s; >>> > .between_bytes_timeout = 600s; >>> > } >>> > >>> > backend b96_30_18_15 { >>> > .host = "96.30.18.15"; >>> > .port = "8080"; >>> > .probe = { >>> > .url = "/"; >>> > .interval = 5s; >>> > .timeout = 1 s; >>> > .window = 5; >>> > .threshold = 3; >>> > } >>> > // we include time outs so uploads don't time out >>> > .connect_timeout = 600s; >>> > .first_byte_timeout = 600s; >>> > .between_bytes_timeout = 600s; >>> > } >>> > >>> > acl a96_30_20_4 { >>> > "96.30.20.4"; >>> > } >>> > acl a96_30_20_5 { >>> > "96.30.20.5"; >>> > } >>> > acl a96_30_20_6 { >>> > "96.30.20.6"; >>> > } >>> > acl a96_30_20_7 { >>> > "96.30.20.7"; >>> > } >>> > acl a96_30_18_15 { >>> > "96.30.18.15"; >>> > } >>> > >>> > acl purge { >>> > "96.30.20.4"; >>> > "96.30.20.5"; >>> > "96.30.20.6"; >>> > "96.30.20.7"; >>> > "96.30.18.15"; >>> > } >>> > >>> > sub vcl_recv { >>> > if (server.ip ~ a96_30_20_4) { >>> > set req.backend = b96_30_20_4; >>> > } >>> > if (server.ip ~ a96_30_20_5) { >>> > set req.backend = b96_30_20_5; >>> > } >>> > if (server.ip ~ a96_30_20_6) { >>> > set req.backend = b96_30_20_6; >>> > } >>> > if (server.ip ~ a96_30_20_7) { >>> > set req.backend = b96_30_20_7; >>> > } >>> > if (server.ip ~ a96_30_18_15) { >>> > set req.backend = b96_30_18_15; >>> > } >>> > } >>> > >>> > sub vcl_recv { >>> > if (req.request == "PURGE") { >>> > if (!client.ip ~ purge) { >>> > error 405 "Not allowed."; >>> > } >>> > return(lookup); >>> > } >>> > if (req.url ~ "^/") { >>> > return(pass); >>> > } >>> > if (req.url ~ "^/forum/$") { >>> > return(pass); >>> > } >>> > if (req.url ~ "^/$") { >>> > unset req.http.cookie; >>> > } >>> > } >>> > sub vcl_hit { >>> > if (req.request == "PURGE") { >>> > set obj.ttl = 0s; >>> > error 200 "Purged."; >>> > } >>> > } >>> > sub vcl_miss { >>> > if (req.request == "PURGE") { >>> > error 404 "Not in cache."; >>> > } >>> > if (!(req.url ~ "wp-(login|admin)")) { >>> > unset req.http.cookie; >>> > } >>> > if (req.url ~ >>> > >>> > >>> "^/[^?]+.(jpeg|jpg|png|gif|ico|js|css|txt|gz|zip|lzma|bz2|tgz|tbz|html|htm)(\?.|)$") >>> > { >>> > unset req.http.cookie; >>> > set req.url = regsub(req.url, "\?.$", ""); >>> > } >>> > if (req.url ~ "^/$") { >>> > unset req.http.cookie; >>> > } >>> > } >>> > sub vcl_fetch { >>> > if (req.url ~ "^/$") { >>> > unset beresp.http.set-cookie; >>> > } >>> > if (!(req.url ~ "wp-(login|admin)")) { >>> > unset beresp.http.set-cookie; >>> > } >>> > } >>> > -------------- next part -------------- >>> > An HTML attachment was scrubbed... >>> > URL: < >>> > >>> >>> https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20110708/bf4bd761/attachment-0001.html >>> > > >>> > >>> > ------------------------------ >>> > >>> > Message: 3 >>> > Date: Fri, 08 Jul 2011 12:41:30 +0200 >>> > From: Tollef Fog Heen < >>> tfheen at varnish-software.com> >>> > To: google at alexus.org >>> > Cc: varnish-misc at varnish-cache.org >>> > Subject: Re: errno = Connection reset by peer >>> > Message-ID: < <8739iht5ph.fsf at qurzaw.varnish-software.com> >>> 8739iht5ph.fsf at qurzaw.varnish-software.com> >>> > Content-Type: text/plain; charset=us-ascii >>> > >>> > ]] alexus >>> > >>> > | 70 Debug c "Write error, retval = -1, len = 10212, errno = >>> > | Connection reset by peer" >>> > | >>> > | other then vcl_error(404, unknown virtual host) what that other >>> message >>> > | really mean? >>> > >>> > That the client disconnected before we were done sending the object. >>> > >>> > -- >>> > Tollef Fog Heen >>> > Varnish Software >>> > t: +47 21 98 92 64 >>> > >>> > >>> > >>> > ------------------------------ >>> > >>> > Message: 4 >>> > Date: Fri, 08 Jul 2011 12:44:45 +0200 >>> > From: Tollef Fog Heen < >>> tfheen at varnish-software.com> >>> > To: Jonathan Hursey < >>> jonathan.hursey at adrevolution.com> >>> > Cc: varnish-misc at varnish-cache.org >>> > Subject: Re: Varnishstat question >>> > Message-ID: < <87y609rqzm.fsf at qurzaw.varnish-software.com> >>> 87y609rqzm.fsf at qurzaw.varnish-software.com> >>> > Content-Type: text/plain; charset=us-ascii >>> > >>> > ]] Jonathan Hursey >>> > >>> > Hi, >>> > >>> > | Greetings fellow varnish enthusiasts. I was wondering if anyone could >>> > help >>> > | me determine what would be considered good values for the follow >>> fields >>> > in >>> > | varnishstat output and what these fields indicate: >>> > | >>> > | 1 0.00 0.00 N vcl total >>> > | 1 0.00 0.00 N vcl available >>> > >>> > This just tells you that you have one VCL loaded. >>> > >>> > | 7 . . N total active purges >>> > | 7 0.00 0.00 N new purges added >>> > >>> > You have 7 bans added >>> > >>> > | 6112 0.00 0.02 N objects tested >>> > >>> > And 6112 objects tested against those bans in total. >>> > >>> > Regards, >>> > -- >>> > Tollef Fog Heen >>> > Varnish Software >>> > t: +47 21 98 92 64 >>> > >>> > >>> > >>> > ------------------------------ >>> > >>> > Message: 5 >>> > Date: Thu, 30 Jun 2011 20:44:26 -0400 >>> > From: Lee Trout < lee at leetrout.com> >>> > To: varnish-misc at varnish-cache.org >>> > Subject: Possible bug with very large headers in varnish-3.0.0 >>> > revision 3bd5997 >>> > Message-ID: >>> eVPwg at mail.gmail.com> >>> > Content-Type: text/plain; charset=ISO-8859-1 >>> > >>> > Hello! >>> > >>> > Please excuse the interruption, but I wanted to see if I may have >>> > stumbled upon a bug or if there is a known issue or configuration >>> > option for requests with very large header values. >>> > >>> > Using varnish-3.0.0 revision 3bd5997 when a large request comes in >>> > (cookie header ~3k characters) the header is lost. I am seeing >>> > "LostHeader c Cookie:" in the logs. >>> > >>> > This paste includes the request and results comparing varnish-3.0.0 >>> > revision 3bd5997 and varnish-2.1.4 SVN 5447M >>> > http://pastebin.com/J2Qn0KGn >>> > >>> > Thank you in advance for your time and suggestions. >>> > >>> > Lee >>> > -- >>> > Lee Trout >>> > http://www.leetrout.com >>> > >>> > >>> > >>> > ------------------------------ >>> > >>> > _______________________________________________ >>> > varnish-misc mailing list >>> > varnish-misc at varnish-cache.org >>> > >>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >>> > >>> > End of varnish-misc Digest, Vol 64, Issue 13 >>> > ******************************************** >>> > >>> -------------- next part -------------- >>> An HTML attachment was scrubbed... >>> URL: < >>> https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20110708/c40bf9ed/attachment.html >>> > >>> >>> ------------------------------ >>> >>> _______________________________________________ >>> varnish-misc mailing list >>> varnish-misc at varnish-cache.org >>> >>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >>> >>> End of varnish-misc Digest, Vol 64, Issue 15 >>> ******************************************** >>> >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> >> >> There is a separate tool called varnishadm that can regularly purge pages. >> Use that. Although you'll have to figure out how to trigger it on post. >> Alternatively can't you just cache the front page for only a minute or >> similar? >> >> Stew >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From parsimail2001 at yahoo.se Sun Jul 10 12:51:31 2011 From: parsimail2001 at yahoo.se (Your Friend) Date: Sun, 10 Jul 2011 13:51:31 +0100 (BST) Subject: Problem with Varnish. Message-ID: <1310302291.40871.YahooMailClassic@web26301.mail.ukl.yahoo.com> Hey guys, I am fairly new to the Varnish, I installed Varnish as the frontend to nginx, everything works fine f?ruton a strange problem that I have never encountered before when I used only the nginx; I have an index page that links to the archives of this dress in two different ways: Link 1: website.com/blog/archive/index.php Link 2: website.com/blog/archive/ Here comes the problem, with Varnish does not link two work, the browser simply can not access the page and after a few seconds, nothing appears in the browser and the address bar shows the address that a page can not be reached, the address that appears in the address bar is: website.com:beckendport/blog/archive/ Configuration of the Web server is 127.0.0.1:beckendport and Varnish 80. I don't use htaccess. I would be grateful if you help me with this problem. Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: From shib4u at gmail.com Sun Jul 10 12:55:41 2011 From: shib4u at gmail.com (Shibashish) Date: Sun, 10 Jul 2011 18:25:41 +0530 Subject: Problem with Varnish. In-Reply-To: <1310302291.40871.YahooMailClassic@web26301.mail.ukl.yahoo.com> References: <1310302291.40871.YahooMailClassic@web26301.mail.ukl.yahoo.com> Message-ID: On Sun, Jul 10, 2011 at 6:21 PM, Your Friend wrote: > Hey guys, > > > I am fairly new to the Varnish, I installed Varnish as the frontend to > nginx, everything works fine f?ruton a strange problem that I have never > encountered before when I used only the nginx; > > I have an index page that links to the archives of this dress in two > different ways: > Link 1: > website.com/blog/archive/index.php > Link 2: > website.com/blog/archive/ > > Here comes the problem, with Varnish does not link two work, the browser > simply can not access the page and after a few seconds, nothing appears in > the browser and the address bar shows the address that a page can not be > reached, the address that appears in the address bar is: > > website.com:beckendport/blog/archive/ > > Configuration of the Web server is 127.0.0.1:beckendport and Varnish 80. > > I don't use htaccess. > > I would be grateful if you help me with this problem. > > Thanks! > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > Please post your vcl file. ShiB. while ( ! ( succeed = try() ) ); -------------- next part -------------- An HTML attachment was scrubbed... URL: From parsimail2001 at yahoo.se Sun Jul 10 13:21:42 2011 From: parsimail2001 at yahoo.se (Your Friend) Date: Sun, 10 Jul 2011 14:21:42 +0100 (BST) Subject: Problem with Varnish. In-Reply-To: Message-ID: <1310304102.49956.YahooMailClassic@web26308.mail.ukl.yahoo.com> Thanks for your answer, I'm new to varnish and don't use any script, could pls tell me in which file and where in the file I should put this code? --- Den s?n 2011-07-10 skrev Shibashish : Fr?n: Shibashish ?mne: Re: Problem with Varnish. Till: "Your Friend" Kopia: varnish-misc at varnish-cache.org Datum: s?ndag 10 juli 2011 01:55 On Sun, Jul 10, 2011 at 6:21 PM, Your Friend wrote: Hey guys, I am fairly new to the Varnish, I installed Varnish as the frontend to nginx, everything works fine f?ruton a strange problem that I have never encountered before when I used only the nginx; I have an index page that links to the archives of this dress in two different ways: Link 1: website.com/blog/archive/index.php Link 2: website.com/blog/archive/ Here comes the problem, with Varnish does not link two work, the browser simply can not access the page and after a few seconds, nothing appears in the browser and the address bar shows the address that a page can not be reached, the address that appears in the address bar is: website.com:beckendport/blog/archive/ Configuration of the Web server is 127.0.0.1:beckendport and Varnish 80. I don't use htaccess. I would be grateful if you help me with this problem. Thanks! _______________________________________________ varnish-misc mailing list varnish-misc at varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc Please post your vcl file. ShiB. while ( ! ( succeed = try() ) ); -------------- next part -------------- An HTML attachment was scrubbed... URL: From parsimail2001 at yahoo.se Sun Jul 10 13:38:12 2011 From: parsimail2001 at yahoo.se (Your Friend) Date: Sun, 10 Jul 2011 14:38:12 +0100 (BST) Subject: Problem with Varnish. In-Reply-To: Message-ID: <1310305092.8674.YahooMailClassic@web26303.mail.ukl.yahoo.com> hey, here is my vcl file: I commented out "sub vcl_fetch" because I got error while starting varnish.? backend default { .host = "127.0.0.1"; ? .port = "81"; } sub vcl_recv { # add a unique header containing the client IP address set req.http.X-Orig-Forwarded-For = client.ip; # we're only handling static content for now so remove any # Cookie that was sent in the request as that makes caching # impossible. if (req.url ~ "\.(jpe?g|png|gif|ico|js|css|zip|CSS|mp3|wma|wmv|avi)(\?.*|)$") { unset req.http.cookie; } } #sub vcl_fetch { # if the backend server adds a cookie to all responses, # remove it from static content so that it can be cached. # if (req.url ~ "\.(jpe?g|png|gif|ico|js|css|zip|CSS|mp3|wma|wmv|avi)(\?.*|)$") { # unset obj.http.set-cookie; # } #} --- Den s?n 2011-07-10 skrev Shibashish : Hey guys, I am fairly new to the Varnish, I installed Varnish as the frontend to nginx, everything works fine f?ruton a strange problem that I have never encountered before when I used only the nginx; I have an index page that links to the archives of this dress in two different ways: Link 1: website.com/blog/archive/index.php Link 2: website.com/blog/archive/ Here comes the problem, with Varnish does not link two work, the browser simply can not access the page and after a few seconds, nothing appears in the browser and the address bar shows the address that a page can not be reached, the address that appears in the address bar is: website.com:beckendport/blog/archive/ Configuration of the Web server is 127.0.0.1:beckendport and Varnish 80. I don't use htaccess. I would be grateful if you help me with this problem. Thanks! _______________________________________________ varnish-misc mailing list varnish-misc at varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc Please post your vcl file. ShiB. while ( ! ( succeed = try() ) ); -------------- next part -------------- An HTML attachment was scrubbed... URL: From goafter1981 at gmail.com Mon Jul 11 02:48:10 2011 From: goafter1981 at gmail.com (xwu) Date: Mon, 11 Jul 2011 10:48:10 +0800 Subject: Varnishlog will switch to debug mode automatically in varnish 3.0 In-Reply-To: References: Message-ID: <1310352490.3874.9.camel@xwu-ThinkPad-X201> Dear folks, When I run varnish for a while, varnishlog output will display too many Debug options. It may be bug. Some one can have comments on that? 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" 0 Debug - "lurker: 0x7f63dfc17f90 120 0" From tfheen at varnish-software.com Mon Jul 11 07:39:56 2011 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Mon, 11 Jul 2011 09:39:56 +0200 Subject: A Few Questions In-Reply-To: (Scott Wilcox's message of "Sat, 9 Jul 2011 22:21:57 +0100") References: Message-ID: <878vs5qn8z.fsf@qurzaw.varnish-software.com> ]] Scott Wilcox | 2) I want to do something similar to: | | sub vcl_hit { | set obj.http.X-Cache-Hit = "true"; | return (deliver); | } | sub vcl_miss { | set obj.http.X-Cache-Hit = "false"; | return (fetch); | } | | for debugging purposes, however when I add that to a VCL, I get: | | Message from VCC-compiler: | 'obj.http.X-Cache-Hit': cannot be set in method 'vcl_hit'. do something like: sub vcl_deliver { if (obj.hits > 1) { set resp.http.X-Cache-Hit = "true"; } else { set resp.http.X-Cache-Hit = "false"; } } Regards, -- Tollef Fog Heen Varnish Software t: +47 21 98 92 64 From jnerin+varnish at gmail.com Mon Jul 11 08:37:57 2011 From: jnerin+varnish at gmail.com (=?UTF-8?B?Sm9yZ2UgTmVyw61u?=) Date: Mon, 11 Jul 2011 10:37:57 +0200 Subject: A Few Questions In-Reply-To: <878vs5qn8z.fsf@qurzaw.varnish-software.com> References: <878vs5qn8z.fsf@qurzaw.varnish-software.com> Message-ID: On Mon, Jul 11, 2011 at 09:39, Tollef Fog Heen wrote: > ]] Scott Wilcox > > | 2) I want to do something similar to: > | > | sub vcl_hit { > | set obj.http.X-Cache-Hit = "true"; > | return (deliver); > | } > | sub vcl_miss { > | set obj.http.X-Cache-Hit = "false"; > | return (fetch); > | } > | > | for debugging purposes, however when I add that to a VCL, I get: > | > | Message from VCC-compiler: > | 'obj.http.X-Cache-Hit': cannot be set in method 'vcl_hit'. > > do something like: > > sub vcl_deliver { > if (obj.hits > 1) { > set resp.http.X-Cache-Hit = "true"; > } else { > set resp.http.X-Cache-Hit = "false"; > } > } > > Shouldn't it be obj.hits > 0? Scott read this about adding the header: https://www.varnish-cache.org/trac/wiki/VCLExampleHitMissHeader Greetings. > Regards, > -- > Tollef Fog Heen > Varnish Software > t: +47 21 98 92 64 > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -- Jorge Ner?n -------------- next part -------------- An HTML attachment was scrubbed... URL: From tfheen at varnish-software.com Mon Jul 11 09:03:29 2011 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Mon, 11 Jul 2011 11:03:29 +0200 Subject: A Few Questions In-Reply-To: ("Jorge =?utf-8?Q?Ner=C3=ADn=22's?= message of "Mon, 11 Jul 2011 10:37:57 +0200") References: <878vs5qn8z.fsf@qurzaw.varnish-software.com> Message-ID: <87vcv9p4ta.fsf@qurzaw.varnish-software.com> ]] Jorge Ner?n | > sub vcl_deliver { | > if (obj.hits > 1) { | > set resp.http.X-Cache-Hit = "true"; | > } else { | > set resp.http.X-Cache-Hit = "false"; | > } | > } | | Shouldn't it be obj.hits > 0? Yeah, it probably should, I thought we set it to 1 at first, but we don't. -- Tollef Fog Heen Varnish Software t: +47 21 98 92 64 From parsimail2001 at yahoo.se Mon Jul 11 11:16:50 2011 From: parsimail2001 at yahoo.se (Your Friend) Date: Mon, 11 Jul 2011 12:16:50 +0100 (BST) Subject: Problem with Varnish. In-Reply-To: Message-ID: <1310383010.81448.YahooMailClassic@web26304.mail.ukl.yahoo.com> Hey, I posted my vcl conf yesterday and still no response, is this forum active? shall I give up on varnish? Anyone who can help me with the issue? ? --- Den s?n 2011-07-10 skrev Shibashish : hey, here is my vcl file: I commented out "sub vcl_fetch" because I got error while starting varnish. backend default { .host = "127.0.0.1"; .port = "81"; } sub vcl_recv { # add a unique header containing the client IP address set req.http.X-Orig-Forwarded-For = client.ip; # we're only handling static content for now so remove any # Cookie that was sent in the request as that makes caching # impossible. if (req.url ~ "\.(jpe?g|png|gif|ico|js|css|zip|CSS|mp3|wma|wmv|avi)(\?.*|)$") { unset req.http.cookie; } } #sub vcl_fetch { # if the backend server adds a cookie to all responses, # remove it from static content so that it can be cached. # if (req.url ~ "\.(jpe?g|png|gif|ico|js|css|zip|CSS|mp3|wma|wmv|avi)(\?.*|)$") { # unset obj.http.set-cookie; # } #} Hey guys, I am fairly new to the Varnish, I installed Varnish as the frontend to nginx, everything works fine f?ruton a strange problem that I have never encountered before when I used only the nginx; I have an index page that links to the archives of this dress in two different ways: Link 1: website.com/blog/archive/index.php Link 2: website.com/blog/archive/ Here comes the problem, with Varnish does not link two work, the browser simply can not access the page and after a few seconds, nothing appears in the browser and the address bar shows the address that a page can not be reached, the address that appears in the address bar is: website.com:beckendport/blog/archive/ Configuration of the Web server is 127.0.0.1:beckendport and Varnish 80. I don't use htaccess. I would be grateful if you help me with this problem. Thanks! _______________________________________________ varnish-misc mailing list varnish-misc at varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc Please post your vcl file. ShiB. while ( ! ( succeed = try() ) ); -------------- next part -------------- An HTML attachment was scrubbed... URL: From traian.bratucu at eea.europa.eu Mon Jul 11 11:39:52 2011 From: traian.bratucu at eea.europa.eu (Traian Bratucu) Date: Mon, 11 Jul 2011 13:39:52 +0200 Subject: Problem with Varnish. In-Reply-To: <1310383010.81448.YahooMailClassic@web26304.mail.ukl.yahoo.com> References: <1310383010.81448.YahooMailClassic@web26304.mail.ukl.yahoo.com> Message-ID: Hello, >From what I can see, your problem might not be with varnish. As far as i know, varnish would return a 503 error on backend failure, you would never get redirected to website.com:backendport, as you say it is happening to you. Please try to use the command "varnishlog" to see exactly what is happening to the request. Also, posting which version of varnish (2.x or 3) you are using might also help. Regards, Traian From: varnish-misc-bounces at varnish-cache.org [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Your Friend Sent: Monday, July 11, 2011 1:17 PM To: Shibashish Cc: varnish-misc at varnish-cache.org Subject: Re: Problem with Varnish. Hey, I posted my vcl conf yesterday and still no response, is this forum active? shall I give up on varnish? Anyone who can help me with the issue? --- Den s?n 2011-07-10 skrev Shibashish >: hey, here is my vcl file: I commented out "sub vcl_fetch" because I got error while starting varnish. backend default { .host = "127.0.0.1"; .port = "81"; } sub vcl_recv { # add a unique header containing the client IP address set req.http.X-Orig-Forwarded-For = client.ip; # we're only handling static content for now so remove any # Cookie that was sent in the request as that makes caching # impossible. if (req.url ~ "\.(jpe?g|png|gif|ico|js|css|zip|CSS|mp3|wma|wmv|avi)(\?.*|)$") { unset req.http.cookie; } } #sub vcl_fetch { # if the backend server adds a cookie to all responses, # remove it from static content so that it can be cached. # if (req.url ~ "\.(jpe?g|png|gif|ico|js|css|zip|CSS|mp3|wma|wmv|avi)(\?.*|)$") { # unset obj.http.set-cookie; # } #} Hey guys, I am fairly new to the Varnish, I installed Varnish as the frontend to nginx, everything works fine f?ruton a strange problem that I have never encountered before when I used only the nginx; I have an index page that links to the archives of this dress in two different ways: Link 1: website.com/blog/archive/index.php Link 2: website.com/blog/archive/ Here comes the problem, with Varnish does not link two work, the browser simply can not access the page and after a few seconds, nothing appears in the browser and the address bar shows the address that a page can not be reached, the address that appears in the address bar is: website.com:beckendport/blog/archive/ Configuration of the Web server is 127.0.0.1:beckendport and Varnish 80. I don't use htaccess. I would be grateful if you help me with this problem. Thanks! _______________________________________________ varnish-misc mailing list varnish-misc at varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc Please post your vcl file. ShiB. while ( ! ( succeed = try() ) ); -------------- next part -------------- An HTML attachment was scrubbed... URL: From guly at luv.guly.org Mon Jul 11 11:42:27 2011 From: guly at luv.guly.org (Sandro guly Zaccarini) Date: Mon, 11 Jul 2011 13:42:27 +0200 Subject: Problem with Varnish. In-Reply-To: <1310383010.81448.YahooMailClassic@web26304.mail.ukl.yahoo.com> References: <1310383010.81448.YahooMailClassic@web26304.mail.ukl.yahoo.com> Message-ID: <20110711114227.GE369@shivaya.guly.org> On Mon, Jul 11, 2011 at 12:16:50PM +0100, Your Friend wrote: > Hey, > I posted my vcl conf yesterday and still no response, is this forum active? yes, actually this is a mailing list but it's active. i suggest you to ask for help, not to claim for it. > shall I give up on varnish? > Anyone who can help me with the issue? maybe your web developer or sysadmin can. i suppose that there is something wrong with your web stuff: your vcl doesn't rewrite any address so if your browser gives you a timeout on host:backendport/ it's because of the link points there. sz -- /"\ taste your favourite IT consultant \ / gpg public key http://www.guly.org/guly.asc X / \ From roberto.fernandezcrisial at gmail.com Mon Jul 11 12:26:39 2011 From: roberto.fernandezcrisial at gmail.com (=?utf-8?B?Um9iZXJ0byBPLiBGZXJuw6FuZGV6IENyaXNpYWw=?=) Date: Mon, 11 Jul 2011 12:26:39 +0000 Subject: Problem with Varnish. In-Reply-To: <1310383010.81448.YahooMailClassic@web26304.mail.ukl.yahoo.com> References: <1310383010.81448.YahooMailClassic@web26304.mail.ukl.yahoo.com> Message-ID: <1161331174-1310387201-cardhu_decombobulator_blackberry.rim.net-500200833-@b16.c27.bise6.blackberry> Do youy have any router or firewall dropping transfer packages? I mean, somethign that not allow varnish to retrieve or response from/to ngnix or internet? Regards, Roberto @rofc -----Original Message----- From: Your Friend Sender: varnish-misc-bounces at varnish-cache.org Date: Mon, 11 Jul 2011 12:16:50 To: Shibashish Cc: Subject: Re: Problem with Varnish. _______________________________________________ varnish-misc mailing list varnish-misc at varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc From parsimail2001 at yahoo.se Mon Jul 11 12:59:19 2011 From: parsimail2001 at yahoo.se (Your Friend) Date: Mon, 11 Jul 2011 13:59:19 +0100 (BST) Subject: Problem with Varnish. In-Reply-To: Message-ID: <1310389159.38909.YahooMailClassic@web26305.mail.ukl.yahoo.com> Hey, Thanks for your response. The version of Varnish is 2.1.3-8, latest stable version for Debian, I used APT. I tried to run varnishlog, than I clicked on the link 2, a few second later I got kicked out of server (ssh)! I think it's more like some security issue, I also checked server logs, I know it's firewall disconnecting client. But only if I use Link 2. maybe I wrote someting wrong in the conf file? --- Den m?n 2011-07-11 skrev Traian Bratucu : Hello, ?From what I can see, your problem might not be with varnish. As far as i know, varnish would return a 503 error on backend failure, you would never get redirected to website.com:backendport, as you say it is happening to you. ?Please try to use the command ?varnishlog? to see exactly what is happening to the request. Also, posting which version of varnish (2.x or 3) you are using might also help. ?Regards,Traian ?Hey, ?I posted my vcl conf yesterday and still no response, is this forum active? shall I give up on varnish? ?Anyone who can help me with the issue? ?? --- Den s?n 2011-07-10 skrev Shibashish :hey, here is my vcl file:I commented out "sub vcl_fetch" because I got error while starting varnish. ?backend default { .host = "127.0.0.1"; .port = "81"; } sub vcl_recv { # add a unique header containing the client IP address set req.http.X-Orig-Forwarded-For = client.ip; # we're only handling static content for now so remove any # Cookie that was sent in the request as that makes caching # impossible. if (req.url ~ "\.(jpe?g|png|gif|ico|js|css|zip|CSS|mp3|wma|wmv|avi)(\?.*|)$") { unset req.http.cookie; } } #sub vcl_fetch { # if the backend server adds a cookie to all responses, # remove it from static content so that it can be cached. # if (req.url ~ "\.(jpe?g|png|gif|ico|js|css|zip|CSS|mp3|wma|wmv|avi)(\?.*|)$") { # unset obj.http.set-cookie; # }#} ? ?Hey guys, ?I am fairly new to the Varnish, I installed Varnish as the frontend to nginx, everything works fine f?ruton a strange problem that I have never encountered before when I used only the nginx; I have an index page that links to the archives of this dress in two different ways: Link 1: website.com/blog/archive/index.php Link 2: website.com/blog/archive/ Here comes the problem, with Varnish does not link two work, the browser simply can not access the page and after a few seconds, nothing appears in the browser and the address bar shows the address that a page can not be reached, the address that appears in the address bar is: website.com:beckendport/blog/archive/ Configuration of the Web server is 127.0.0.1:beckendport and Varnish 80. I don't use htaccess. I would be grateful if you help me with this problem.Thanks! _______________________________________________ varnish-misc mailing list varnish-misc at varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc ?Please post your vcl file. ShiB. while ( ! ( succeed = try() ) ); ? -----Infogad bilaga f?ljer----- _______________________________________________ varnish-misc mailing list varnish-misc at varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc -------------- next part -------------- An HTML attachment was scrubbed... URL: From parsimail2001 at yahoo.se Mon Jul 11 13:05:41 2011 From: parsimail2001 at yahoo.se (Your Friend) Date: Mon, 11 Jul 2011 14:05:41 +0100 (BST) Subject: Problem with Varnish. In-Reply-To: <1161331174-1310387201-cardhu_decombobulator_blackberry.rim.net-500200833-@b16.c27.bise6.blackberry> Message-ID: <1310389541.80133.YahooMailClassic@web26301.mail.ukl.yahoo.com> Hey, Yes, iptables but in/outgoing for port 80 is open. nginx running on port 81 and varnish running on 80, beckend for varnish i set to "127.0.0.1", "81". I think it's because of firewall or sysctl... Do you know if there is anything in sysctl or firewall which can cause this problem?? thanks. --- Den m?n 2011-07-11 skrev Roberto O. Fern?ndez Crisial : Do youy have any router or firewall dropping transfer packages? I mean, somethign that not allow varnish to retrieve or response from/to ngnix or internet? Regards, Roberto @rofc _______________________________________________ varnish-misc mailing list varnish-misc at varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc -------------- next part -------------- An HTML attachment was scrubbed... URL: From mls at pooteeweet.org Wed Jul 13 06:28:09 2011 From: mls at pooteeweet.org (Lukas Kahwe Smith) Date: Wed, 13 Jul 2011 08:28:09 +0200 Subject: efficient strategy to cache warm multiple varnish servers Message-ID: Hi, If we ever have to restart our setup from scratch because of some big crash or because we are rolling out a new version of the site, I am wondering how to most efficiently warm all the varnish servers. Do I just warm one server and then rsync? Searching for "varnish cache warm" and didnt find anything very detailed. This link seems useful in the general context of cache warming scripts: https://www.varnish-cache.org/trac/wiki/VCLExampleHashAlwaysMiss This one however seems to sound like I can chain my varnish caches which could be what I need to do: https://www.varnish-cache.org/trac/wiki/VCLExampleHashIgnoreBusy Any other RTFM links much appreciated :) regards, Lukas Kahwe Smith mls at pooteeweet.org From lajo at kb.dk Wed Jul 13 14:09:16 2011 From: lajo at kb.dk (=?iso-8859-1?Q?Lars_J=F8rgensen?=) Date: Wed, 13 Jul 2011 14:09:16 +0000 Subject: Caching same object twice for different browsers Message-ID: <6D2C830A0941EA40B6B483FE6EC98ADB187656A3@EXCHANGE-02.kb.dk> Hi, I believe this has been touched upon before, but I can't seem to find anything useful about it. I have discovered that our Varnish caches the same object twice; once for MSIE and once for Firefox. This is from varnishlog: First Internet Explorer: 20 RxRequest c GET 20 RxURL c /erez4/cache/online_master_arkiv_webbilleder_DIA_kalender_Koncerter_diaens_mortenfranklarsen_tif_331510ad02b81dd2.jpg 20 RxProtocol c HTTP/1.1 20 RxHeader c Accept: text/html, application/xhtml+xml, */* 20 RxHeader c Accept-Language: da-DK 20 RxHeader c User-Agent: Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0) 20 RxHeader c Accept-Encoding: gzip, deflate 20 RxHeader c Host: www.kb.dk 20 RxHeader c Connection: Keep-Alive 20 RxHeader c Cache-Control: no-cache 20 RxHeader c Cookie: __utma=52637486.1667608039.1302159744.xxxxxxxxxx.1310564597.16; __utmz=xxxxxxxx.1307533937.4.2.utmccn=(referral)|utmcsr=intranet.kb.dk|utmcct=/ATO/Sider/Genveje.aspx|utmcmd=referral; __utmb=xxxxxx 20 VCL_call c recv lookup 20 VCL_call c hash 20 Hash c /erez4/cache/online_master_arkiv_webbilleder_DIA_kalender_Koncerter_diaens_mortenfranklarsen_tif_331510ad02b81dd2.jpg 20 Hash c www.kb.dk 20 VCL_return c hash 20 Hit c 1414499195 20 VCL_call c hit deliver 20 VCL_call c deliver deliver And then Firefox: 12 RxRequest c GET 12 RxURL c /erez4/cache/online%5Fmaster%5Farkiv%5Fwebbilleder%5FDIA%5Fkalender%5FKoncerter%5Fdiaens%5Fmortenfranklarsen%5Ftif%5F331510ad02b81dd2.jpg 12 RxProtocol c HTTP/1.1 12 RxHeader c Host: www.kb.dk 12 RxHeader c User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:5.0) Gecko/20100101 Firefox/5.0 12 RxHeader c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 12 RxHeader c Accept-Language: da,en-us;q=0.7,en;q=0.3 12 RxHeader c Accept-Encoding: gzip, deflate 12 RxHeader c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 12 RxHeader c Connection: keep-alive 12 RxHeader c Cookie: __switchTo5x=46; __unam=627126b-xxxxxxxxxx-90edcb2-3; __utma=52637486.497784643.xxxxxxxxxx310565486.1310565486.1; __utmb=52xxxx86; __utmc=52xxx86; __utmz=52637486.xxxxxxxxxxx.1.1.utmccn=(direct)|utmcsr=(direct)|utmcmd=(none) 12 RxHeader c If-Modified-Since: Wed, 13 Jul 2011 11:49:07 GMT 12 RxHeader c Cache-Control: max-age=0 12 VCL_call c recv lookup 12 VCL_call c hash 12 Hash c /erez4/cache/online%5Fmaster%5Farkiv%5Fwebbilleder%5FDIA%5Fkalender%5FKoncerter%5Fdiaens%5Fmortenfranklarsen%5Ftif%5F331510ad02b81dd2.jpg 12 Hash c www.kb.dk 12 VCL_return c hash 12 Hit c 1415348861 12 VCL_call c hit deliver 12 VCL_call c deliver deliver Don't worry about the cookies, they are all stripped before going to lookup. As you can see from the lines with "Hit", the two requests for the same object hits two different objects in the cache. I suspect I need to normalize some of the request headers, but which of them and how? -- Lars -------------- next part -------------- An HTML attachment was scrubbed... URL: From fla_torres at yahoo.com.br Wed Jul 13 14:21:50 2011 From: fla_torres at yahoo.com.br (Flavio Torres) Date: Wed, 13 Jul 2011 11:21:50 -0300 Subject: Caching same object twice for different browsers In-Reply-To: <6D2C830A0941EA40B6B483FE6EC98ADB187656A3@EXCHANGE-02.kb.dk> References: <6D2C830A0941EA40B6B483FE6EC98ADB187656A3@EXCHANGE-02.kb.dk> Message-ID: <4E1DA9FE.6090207@yahoo.com.br> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 07/13/2011 11:09 AM, Lars J?rgensen wrote: > I suspect I need to normalize some of the request headers, but which > of them and how? Hello Lars, We are using the following vcl (varnish 2.1): sub vcl_hash { if (req.http.user-agent ~ "iPhone|SymbianOS") { set req.hash += req.http.user-agent; } else { remove req.http.user-agent; } set req.hash += req.url; set req.hash += req.http.host; return (hash); } Hope this helps. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk4dqfgACgkQNRQApncg296jHACgo1TjX5/mlyaCJNqTGFN3xEng nBAAnRjtp3OSc3UJO8+P26EGMe++pkoM =he75 -----END PGP SIGNATURE----- From lajo at kb.dk Wed Jul 13 14:27:40 2011 From: lajo at kb.dk (=?iso-8859-1?Q?Lars_J=F8rgensen?=) Date: Wed, 13 Jul 2011 14:27:40 +0000 Subject: Caching same object twice for different browsers In-Reply-To: <4E1DA9FE.6090207@yahoo.com.br> References: <6D2C830A0941EA40B6B483FE6EC98ADB187656A3@EXCHANGE-02.kb.dk> <4E1DA9FE.6090207@yahoo.com.br> Message-ID: <6D2C830A0941EA40B6B483FE6EC98ADB187656BE@EXCHANGE-02.kb.dk> > We are using the following vcl (varnish 2.1): > > sub vcl_hash { > > if (req.http.user-agent ~ "iPhone|SymbianOS") { > set req.hash += req.http.user-agent; > } > else { > remove req.http.user-agent; > } > > set req.hash += req.url; > set req.hash += req.http.host; > > return (hash); > } > > > Hope this helps. It sure does, thank you. And if I don't care about user-agents at all (AFAIK the CMS always delivers the same page), I can do something like the code below? sub vcl_recv { ... remove req.http.user-agent; ... } If I do that, will varnishncsa still log user-agents? -- Lars From pmialon at free.fr Wed Jul 13 14:41:43 2011 From: pmialon at free.fr (Pierre-Gilles Mialon) Date: Wed, 13 Jul 2011 16:41:43 +0200 Subject: Persistent storage on varnish 3.0.0 Message-ID: <201107131641.52048.pmialon@free.fr> Hi, We try to use persistent storage with varnish 3.0.0, we run: varnishd -d -a :80 -f /etc/varnish/default.vcl -s persistent,/var/lib/varnish/instance/A.shm,10M On linux during the startup of varnishd the output mention: CHK(0x7f1bbd81d0a0 SILO 0x7f1bbc7e1000 SILO) = 3 And the persistent storage is reset. We try the same option with varnish 3.0.0 on amd64 arch with the package and the latest git sources: gcc-version | 4.2 | 4.4 | 4.6 | ------------------------------------------ FreeBSD 8.2 | OK | - | KO | Debian 6 | - | KO | KO | Debian sid | - | KO | KO | Fedora 15 | - | - | KO | ------------------------------------------ We test using the following procedure: . launch varnishd with the command below . start child . varnishreplay a log file . display the number of object (~1000): varnishstat -1 | grep '^n_objec' . stop child . start child . display the number of object . stop child . quit . relaunch the daemon . start child . display the number of object before quit command, varnish persistent storage work, if we quit, it only works on FreeBSD with gcc 4.2 Is it a known issue ? Thanks for your help. -- Pierre-Gilles Mialon -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part. URL: From parsimail2001 at yahoo.se Wed Jul 13 16:50:46 2011 From: parsimail2001 at yahoo.se (Your Friend) Date: Wed, 13 Jul 2011 17:50:46 +0100 (BST) Subject: Problem with Varnish. In-Reply-To: Message-ID: <1310575846.89298.YahooMailClassic@web26305.mail.ukl.yahoo.com> Thanks for your response! I put this code in vcl file and even tried to use it as else but it didn't solve the probelm, sub vcl_recv { ? set req.http.X-Orig-Forwarded-For = client.ip; ? if (req.url ~ "\.(jpe?g|png|gif|ico|js|css|zip|CSS|mp3|wma|wmv|avi)(\?.*|)$") { ????unset req.http.cookie; ? } #### backend definition for this request####? set req.backend = default; }and? sub vcl_recv { ? set req.http.X-Orig-Forwarded-For = client.ip; ? if (req.url ~ "\.(jpe?g|png|gif|ico|js|css|zip|CSS|mp3|wma|wmv|avi)(\?.*|)$") { ????unset req.http.cookie; ? } else{ #### backend definition for this request####? set req.backend = default; } }There is nothing in nginx error file, and before I put nginx behinde the varnish it worked perfectly. the request for mysite.com/blog is send to nginx and nginx send it to port 81 and i've blocked this port in fireall. nginx port config is 127.0.0.1:81; is there anything more we need to change in nginx? I have been trying to write a "URLfix" for the vcl_rcv to make varnish understand thattt mysite.com/* ? = ? mysite.com/*/ ? = ? mysite.com/*/index.php * = a name without any extension, alts? it's not a file but the name of directory. Thanks, --- Den tis 2011-07-12 skrev juan.salinas at mercurio.cl : It could have been you didn?t define the backend for the request in sub vcl_recv:sub vcl_recv { ? set req.http.X-Orig-Forwarded-For = client.ip; ? if (req.url ~ "\.(jpe?g|png|gif|ico|js|css|zip|CSS|mp3|wma|wmv|avi)(\?.*|)$") { ????unset req.http.cookie; ? } #### backend definition for this request####? set req.backend = default; } ?Juan Salinas Contreras ?De: varnish-misc-bounces at varnish-cache.org [mailto:varnish-misc-bounces at varnish-cache.org] En nombre de Your Friend Enviado el: domingo, 10 de julio de 2011 9:38 Para: Shibashish CC: varnish-misc at varnish-cache.org Asunto: Re: Problem with Varnish. ?hey, here is my vcl file: ?I commented out "sub vcl_fetch" because I got error while starting varnish.? ?backend default { .host = "127.0.0.1"; ? .port = "81"; } sub vcl_recv { # add a unique header containing the client IP address set req.http.X-Orig-Forwarded-For = client.ip; # we're only handling static content for now so remove any # Cookie that was sent in the request as that makes caching # impossible. if (req.url ~ "\.(jpe?g|png|gif|ico|js|css|zip|CSS|mp3|wma|wmv|avi)(\?.*|)$") { unset req.http.cookie; } } #sub vcl_fetch { # if the backend server adds a cookie to all responses, # remove it from static content so that it can be cached. # if (req.url ~ "\.(jpe?g|png|gif|ico|js|css|zip|CSS|mp3|wma|wmv|avi)(\?.*|)$") { # unset obj.http.set-cookie; # } #} ? ? --- Den s?n 2011-07-10 skrev Shibashish : Hey guys, ?I am fairly new to the Varnish, I installed Varnish as the frontend to nginx, everything works fine f?ruton a strange problem that I have never encountered before when I used only the nginx; I have an index page that links to the archives of this dress in two different ways: Link 1: website.com/blog/archive/index.php Link 2: website.com/blog/archive/ Here comes the problem, with Varnish does not link two work, the browser simply can not access the page and after a few seconds, nothing appears in the browser and the address bar shows the address that a page can not be reached, the address that appears in the address bar is: website.com:beckendport/blog/archive/ Configuration of the Web server is 127.0.0.1:beckendport and Varnish 80. I don't use htaccess. I would be grateful if you help me with this problem.Thanks! _______________________________________________ varnish-misc mailing list varnish-misc at varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc ?Please post your vcl file. ShiB. while ( ! ( succeed = try() ) ); ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From varnishlist at realvideosite.com Wed Jul 13 18:13:32 2011 From: varnishlist at realvideosite.com (Varnish List) Date: Wed, 13 Jul 2011 14:13:32 -0400 Subject: Varnish cache miss slower than no Varnish at all! Message-ID: Does anyone know if there a way to speed up cache misses when varnish goes to the back end? I have a two servers, one running varnish and one not. *Average over 24 hours:* server1(varnish): cache hit: 3ms cache miss: 52ms server2(no varnish): request: 24ms The miss on the Varnish server takes longer to process than the server that does not have varnish. Varnish misses take longer to process than no varnish at all. The servers are exactly the same and the Varnish server uses itself as a back end. However the back end in default.vcl is an ip address not localhost, but im not sure if that has anything to do with the processing time. Any suggestions on this one? Thanks :) -------------- next part -------------- An HTML attachment was scrubbed... URL: From lajo at kb.dk Thu Jul 14 07:19:52 2011 From: lajo at kb.dk (=?iso-8859-1?Q?Lars_J=F8rgensen?=) Date: Thu, 14 Jul 2011 07:19:52 +0000 Subject: Caching same object twice for different browsers In-Reply-To: <6D2C830A0941EA40B6B483FE6EC98ADB187656A3@EXCHANGE-02.kb.dk> References: <6D2C830A0941EA40B6B483FE6EC98ADB187656A3@EXCHANGE-02.kb.dk> Message-ID: <6D2C830A0941EA40B6B483FE6EC98ADB18765898@EXCHANGE-02.kb.dk> Hi, Sorry for top-quoting, but stuck with Outlook and fixed settings here at the office. Anyway, I found out that the web server does not use a Vary-header, so that wasn't the problem. After doing some manual tests and being unable the replicate the problem, I started to look harder at the two dumps below. Then it became pretty obvious. It seems that Explorer and Firefox requests urls in a different way. Explorer: /erez4/cache/online_master_arkiv_webbilleder_DIA_ Firefox: /erez4/cache/online%5Fmaster%5Farkiv%5Fwebbilleder%5FDIA Of course that means diffent objects in the cache since varnish hashes on req.url. Is there a know workaround to this problem? -- Lars From: varnish-misc-bounces at varnish-cache.org [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Lars J?rgensen Sent: Wednesday, July 13, 2011 4:09 PM To: varnish-misc at varnish-cache.org Subject: Caching same object twice for different browsers Hi, I believe this has been touched upon before, but I can't seem to find anything useful about it. I have discovered that our Varnish caches the same object twice; once for MSIE and once for Firefox. This is from varnishlog: First Internet Explorer: 20 RxRequest c GET 20 RxURL c /erez4/cache/online_master_arkiv_webbilleder_DIA_kalender_Koncerter_diaens_mortenfranklarsen_tif_331510ad02b81dd2.jpg 20 RxProtocol c HTTP/1.1 20 RxHeader c Accept: text/html, application/xhtml+xml, */* 20 RxHeader c Accept-Language: da-DK 20 RxHeader c User-Agent: Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0) 20 RxHeader c Accept-Encoding: gzip, deflate 20 RxHeader c Host: www.kb.dk 20 RxHeader c Connection: Keep-Alive 20 RxHeader c Cache-Control: no-cache 20 RxHeader c Cookie: __utma=52637486.1667608039.1302159744.xxxxxxxxxx.1310564597.16; __utmz=xxxxxxxx.1307533937.4.2.utmccn=(referral)|utmcsr=intranet.kb.dk|utmcct=/ATO/Sider/Genveje.aspx|utmcmd=referral; __utmb=xxxxxx 20 VCL_call c recv lookup 20 VCL_call c hash 20 Hash c /erez4/cache/online_master_arkiv_webbilleder_DIA_kalender_Koncerter_diaens_mortenfranklarsen_tif_331510ad02b81dd2.jpg 20 Hash c www.kb.dk 20 VCL_return c hash 20 Hit c 1414499195 20 VCL_call c hit deliver 20 VCL_call c deliver deliver And then Firefox: 12 RxRequest c GET 12 RxURL c /erez4/cache/online%5Fmaster%5Farkiv%5Fwebbilleder%5FDIA%5Fkalender%5FKoncerter%5Fdiaens%5Fmortenfranklarsen%5Ftif%5F331510ad02b81dd2.jpg 12 RxProtocol c HTTP/1.1 12 RxHeader c Host: www.kb.dk 12 RxHeader c User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:5.0) Gecko/20100101 Firefox/5.0 12 RxHeader c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 12 RxHeader c Accept-Language: da,en-us;q=0.7,en;q=0.3 12 RxHeader c Accept-Encoding: gzip, deflate 12 RxHeader c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 12 RxHeader c Connection: keep-alive 12 RxHeader c Cookie: __switchTo5x=46; __unam=627126b-xxxxxxxxxx-90edcb2-3; __utma=52637486.497784643.xxxxxxxxxx310565486.1310565486.1; __utmb=52xxxx86; __utmc=52xxx86; __utmz=52637486.xxxxxxxxxxx.1.1.utmccn=(direct)|utmcsr=(direct)|utmcmd=(none) 12 RxHeader c If-Modified-Since: Wed, 13 Jul 2011 11:49:07 GMT 12 RxHeader c Cache-Control: max-age=0 12 VCL_call c recv lookup 12 VCL_call c hash 12 Hash c /erez4/cache/online%5Fmaster%5Farkiv%5Fwebbilleder%5FDIA%5Fkalender%5FKoncerter%5Fdiaens%5Fmortenfranklarsen%5Ftif%5F331510ad02b81dd2.jpg 12 Hash c www.kb.dk 12 VCL_return c hash 12 Hit c 1415348861 12 VCL_call c hit deliver 12 VCL_call c deliver deliver Don't worry about the cookies, they are all stripped before going to lookup. As you can see from the lines with "Hit", the two requests for the same object hits two different objects in the cache. I suspect I need to normalize some of the request headers, but which of them and how? -- Lars -------------- next part -------------- An HTML attachment was scrubbed... URL: From juan.salinas at mercurio.cl Tue Jul 12 15:17:46 2011 From: juan.salinas at mercurio.cl (juan.salinas at mercurio.cl) Date: Tue, 12 Jul 2011 11:17:46 -0400 Subject: Problem with Varnish. In-Reply-To: <1310305092.8674.YahooMailClassic@web26303.mail.ukl.yahoo.com> References: <1310305092.8674.YahooMailClassic@web26303.mail.ukl.yahoo.com> Message-ID: It could have been you didn't define the backend for the request in sub vcl_recv: sub vcl_recv { set req.http.X-Orig-Forwarded-For = client.ip; if (req.url ~ "\.(jpe?g|png|gif|ico|js|css|zip|CSS|mp3|wma|wmv|avi)(\?.*|)$") { unset req.http.cookie; } #### backend definition for this request#### set req.backend = default; } Juan Salinas Contreras De: varnish-misc-bounces at varnish-cache.org [mailto:varnish-misc-bounces at varnish-cache.org] En nombre de Your Friend Enviado el: domingo, 10 de julio de 2011 9:38 Para: Shibashish CC: varnish-misc at varnish-cache.org Asunto: Re: Problem with Varnish. hey, here is my vcl file: I commented out "sub vcl_fetch" because I got error while starting varnish. backend default { .host = "127.0.0.1"; .port = "81"; } sub vcl_recv { # add a unique header containing the client IP address set req.http.X-Orig-Forwarded-For = client.ip; # we're only handling static content for now so remove any # Cookie that was sent in the request as that makes caching # impossible. if (req.url ~ "\.(jpe?g|png|gif|ico|js|css|zip|CSS|mp3|wma|wmv|avi)(\?.*|)$") { unset req.http.cookie; } } #sub vcl_fetch { # if the backend server adds a cookie to all responses, # remove it from static content so that it can be cached. # if (req.url ~ "\.(jpe?g|png|gif|ico|js|css|zip|CSS|mp3|wma|wmv|avi)(\?.*|)$") { # unset obj.http.set-cookie; # } #} --- Den s?n 2011-07-10 skrev Shibashish : Hey guys, I am fairly new to the Varnish, I installed Varnish as the frontend to nginx, everything works fine f?ruton a strange problem that I have never encountered before when I used only the nginx; I have an index page that links to the archives of this dress in two different ways: Link 1: website.com/blog/archive/index.php Link 2: website.com/blog/archive/ Here comes the problem, with Varnish does not link two work, the browser simply can not access the page and after a few seconds, nothing appears in the browser and the address bar shows the address that a page can not be reached, the address that appears in the address bar is: website.com:beckendport/blog/archive/ Configuration of the Web server is 127.0.0.1:beckendport and Varnish 80. I don't use htaccess. I would be grateful if you help me with this problem. Thanks! _______________________________________________ varnish-misc mailing list varnish-misc at varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc Please post your vcl file. ShiB. while ( ! ( succeed = try() ) ); -------------- next part -------------- An HTML attachment was scrubbed... URL: From tfheen at varnish-software.com Thu Jul 14 08:17:16 2011 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Thu, 14 Jul 2011 10:17:16 +0200 Subject: Caching same object twice for different browsers In-Reply-To: <6D2C830A0941EA40B6B483FE6EC98ADB18765898@EXCHANGE-02.kb.dk> ("Lars =?utf-8?Q?J=C3=B8rgensen=22's?= message of "Thu, 14 Jul 2011 07:19:52 +0000") References: <6D2C830A0941EA40B6B483FE6EC98ADB187656A3@EXCHANGE-02.kb.dk> <6D2C830A0941EA40B6B483FE6EC98ADB18765898@EXCHANGE-02.kb.dk> Message-ID: <87bowxjmyb.fsf@qurzaw.varnish-software.com> ]] Lars J?rgensen | Sorry for top-quoting, but stuck with Outlook and fixed settings here at the office. | | Anyway, I found out that the web server does not use a Vary-header, so that wasn't the problem. After doing some manual tests and being unable the replicate the problem, I started to look harder at the two dumps below. Then it became pretty obvious. | | It seems that Explorer and Firefox requests urls in a different way. | | Explorer: /erez4/cache/online_master_arkiv_webbilleder_DIA_ | Firefox: /erez4/cache/online%5Fmaster%5Farkiv%5Fwebbilleder%5FDIA | | Of course that means diffent objects in the cache since varnish hashes on req.url. Is there a know workaround to this problem? you could regsub it or with 3.0 I'd suggest having a vmod that just normalises URLs and removes extra % escapes. -- Tollef Fog Heen Varnish Software t: +47 21 98 92 64 From sime at sime.net.au Thu Jul 14 12:18:44 2011 From: sime at sime.net.au (Simon Males) Date: Thu, 14 Jul 2011 22:18:44 +1000 Subject: Reducing overflowed and dropped work requests Message-ID: Hello, The Varnish munin plugins have alerted me a number of times (say less then 5) that N dropped work requests (n_wrk_drop) is greater then 1. According to Kristian Lyngstol's post [1] this is represents when the request queue it full. Additionally I believe the overflowed work requests (n_wrk_overflow) will always be greater then n_wrk_drop. I think I should be trying to tackle two issues: Limit requests overflowing, followed by increasing the size of the request queue. Of the below startup parameters I believe thread_pools should match the number of cores the server has and increase the maximum number of threads to possibly double (from 25 to 50?). DAEMON_OPTS="-a :80 \ -T localhost:6082 \ -f /etc/varnish/default.vcl \ -S /etc/varnish/secret \ -w 10,25,120 \ -p listen_depth=8192 \ -p log_hashstring=off \ -p lru_interval=60 \ -p sess_timeout=10 \ -p shm_workspace=32768 \ -p ping_interval=1 \ -p thread_pools=4 \ -s malloc,2G" Varnish 2.1.5, x86_64, 4 GB, 8 or 16 cores. [1] http://kristianlyng.wordpress.com/2009/12/08/varnishstat-for-dummies/ -- Simon Males From mhettwer at team.mobile.de Thu Jul 14 12:52:18 2011 From: mhettwer at team.mobile.de (Hettwer, Marian) Date: Thu, 14 Jul 2011 13:52:18 +0100 Subject: rate limiting in varnish Message-ID: Hi All, Before we are going to try to write our own VMOD, does anybody accomplished to do rate limiting with varnish? As in: We like to rate limit some scrapers. They don't generate cache hits and will always punch right through to the backend. This is not good(tm). The thought here is to do rate limiting on client.ip base. If client.ip hits us more than n times / second, go to vcl_error and either throw him an error or probably even drop the connection. If bots are nasty to us, sometimes it's good to be nasty to them, eh? ;) Any thoughts/pointers on how one would implement this in varnish? Thanks in advance, Marian From isp at daviesinc.com Thu Jul 14 13:00:37 2011 From: isp at daviesinc.com (Chris Davies) Date: Thu, 14 Jul 2011 09:00:37 -0400 Subject: Detecting health of Backend in Director in VCL Message-ID: I've got a director set up with three nodes, and I can see how to check the health of the director, but, is there any method to check the health of a particular backend in the director? i.e. If our heavily weighted server is detected as down, I may want to handle some requests differently, but, while the director is up, it still reports healthy. From shib4u at gmail.com Thu Jul 14 13:05:38 2011 From: shib4u at gmail.com (Shibashish) Date: Thu, 14 Jul 2011 18:35:38 +0530 Subject: Detecting health of Backend in Director in VCL In-Reply-To: References: Message-ID: On Thu, Jul 14, 2011 at 6:30 PM, Chris Davies wrote: > I've got a director set up with three nodes, and I can see how to check the > health of the director, but, is there any method to check the health of a > particular backend in the director? > > i.e. If our heavily weighted server is detected as down, I may want to > handle some requests differently, but, while the director is up, it still > reports healthy. > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > Each backend host is checked with its own probe... .probe = { .url = "/probe.cgi"; .timeout = 34 ms; .interval = 1s; .window = 10; .threshold = 8; } More at https://www.varnish-cache.org/trac/wiki/BackendPolling ShiB. while ( ! ( succeed = try() ) ); -------------- next part -------------- An HTML attachment was scrubbed... URL: From isp at daviesinc.com Thu Jul 14 13:08:35 2011 From: isp at daviesinc.com (Chris Davies) Date: Thu, 14 Jul 2011 09:08:35 -0400 Subject: Detecting health of Backend in Director in VCL In-Reply-To: References: Message-ID: <472DD1C2-77BB-43C0-8A07-2F8DA9A9EB10@daviesinc.com> On Jul 14, 2011, at 9:05 AM, Shibashish wrote: > On Thu, Jul 14, 2011 at 6:30 PM, Chris Davies wrote: > I've got a director set up with three nodes, and I can see how to check the health of the director, but, is there any method to check the health of a particular backend in the director? > > Each backend host is checked with its own probe... > > .probe = { > .url = "/probe.cgi"; > .timeout = 34 ms; > .interval = 1s; > .window = 10; > .threshold = 8; > } > > More at https://www.varnish-cache.org/trac/wiki/BackendPolling And in VCL, how do I tell the health of the backend within a director? From stewsnooze at gmail.com Thu Jul 14 13:15:09 2011 From: stewsnooze at gmail.com (Stewart Robinson) Date: Thu, 14 Jul 2011 14:15:09 +0100 Subject: Detecting health of Backend in Director in VCL In-Reply-To: <472DD1C2-77BB-43C0-8A07-2F8DA9A9EB10@daviesinc.com> References: <472DD1C2-77BB-43C0-8A07-2F8DA9A9EB10@daviesinc.com> Message-ID: On 14 Jul 2011, at 14:08, Chris Davies wrote: > On Jul 14, 2011, at 9:05 AM, Shibashish wrote: >> On Thu, Jul 14, 2011 at 6:30 PM, Chris Davies wrote: >> I've got a director set up with three nodes, and I can see how to check the health of the director, but, is there any method to check the health of a particular backend in the director? >> >> Each backend host is checked with its own probe... >> >> .probe = { >> .url = "/probe.cgi"; >> .timeout = 34 ms; >> .interval = 1s; >> .window = 10; >> .threshold = 8; >> } >> >> More at https://www.varnish-cache.org/trac/wiki/BackendPolling > > And in VCL, how do I tell the health of the backend within a director? > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc You could set a header based on and then issue a restart then act on the header in vcl_recv. This example restarts to choose a new backend https://www.varnish-cache.org/trac/wiki/VCLExampleRestarts And I've previously done stuff like if (req.backend.healthy) { set req.grace = 60s; } else { set req.grace = 12h; } so you could combine to two to push some logic through Stew -------------- next part -------------- An HTML attachment was scrubbed... URL: From bedis9 at gmail.com Thu Jul 14 17:01:55 2011 From: bedis9 at gmail.com (Baptiste) Date: Thu, 14 Jul 2011 19:01:55 +0200 Subject: rate limiting in varnish In-Reply-To: References: Message-ID: Hi, You could easily do this with HAproxy in front of your Varnish... If all the requests are miss ones, you could also put your HAProxy between Varnish and your backend. cheers On Thu, Jul 14, 2011 at 2:52 PM, Hettwer, Marian wrote: > Hi All, > > Before we are going to try to write our own VMOD, does anybody > accomplished to do rate limiting with varnish? > As in: > > We like to rate limit some scrapers. They don't generate cache hits and > will always punch right through to the backend. This is not good(tm). > The thought here is to do rate limiting on client.ip base. > If client.ip hits us more than n times / second, go to vcl_error and > either throw him an error or probably even drop the connection. > If bots are nasty to us, sometimes it's good to be nasty to them, eh? ;) > > Any thoughts/pointers on how one would implement this in varnish? > > > Thanks in advance, > Marian > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > From sime at sime.net.au Fri Jul 15 01:15:27 2011 From: sime at sime.net.au (Simon Males) Date: Fri, 15 Jul 2011 11:15:27 +1000 Subject: Reducing overflowed and dropped work requests In-Reply-To: References: Message-ID: > The Varnish munin plugins have alerted me a number of times (say less > then 5) that N dropped work requests (n_wrk_drop) is greater then 1. > > According to Kristian Lyngstol's post [1] this is represents when the > request queue it full. Additionally I believe the overflowed work > requests (n_wrk_overflow) will always be greater then n_wrk_drop. > > I think I should be trying to tackle two issues: Limit requests > overflowing, followed by increasing the size of the request queue. > > Of the below startup parameters I believe thread_pools should match > the number of cores the server has and increase the maximum number of > threads to possibly double (from 25 to 50?). > > DAEMON_OPTS="-a :80 \ > ? ? ? ? ? ? -T localhost:6082 \ > ? ? ? ? ? ? -f /etc/varnish/default.vcl \ > ? ? ? ? ? ? -S /etc/varnish/secret \ > ? ? ? ? ? ? -w 10,25,120 \ > ? ? ? ? ? ? -p listen_depth=8192 \ > ? ? ? ? ? ? -p log_hashstring=off \ > ? ? ? ? ? ? -p lru_interval=60 \ > ? ? ? ? ? ? -p sess_timeout=10 \ > ? ? ? ? ? ? -p shm_workspace=32768 \ > ? ? ? ? ? ? -p ping_interval=1 \ > ? ? ? ? ? ? -p thread_pools=4 \ > ? ? ? ? ? ? -s malloc,2G" As overflow_max is default set to 100%, I believe I essentially need to increase the number of worker threads to increase the ratio. To increase the threads, I'll increase the thread_pools. -- Simon Males From jcpetit at syspark.com Fri Jul 15 02:57:33 2011 From: jcpetit at syspark.com (Jean-Christophe Petit) Date: Thu, 14 Jul 2011 22:57:33 -0400 Subject: rate limiting in varnish Message-ID: <4E1FAC9D.4030601@syspark.com> Hello, Dr Carter blog has a very interesting code for Varnish-2.1.x and memcached http://drcarter.info/2010/10/updating-anti-scraping-in-case-of-memcached-server-error/ It looks like porting the code to Varnish-3.0 shouldn't be difficult. Have a good day, JC -- Jean-Christophe Petit VP R&D et DSI GROUPE SYSPARK The Power of Internet Engineering / La Force du G?nie Internet 606 Cathcart, Suite 510 Montreal, Quebec H3B 1K9 - CANADA T: 1 514 875 8755 C: 1 514 575 5560 F: 1 514 875 8775 From tfheen at varnish-software.com Fri Jul 15 06:02:38 2011 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Fri, 15 Jul 2011 08:02:38 +0200 Subject: Detecting health of Backend in Director in VCL In-Reply-To: <472DD1C2-77BB-43C0-8A07-2F8DA9A9EB10@daviesinc.com> (Chris Davies's message of "Thu, 14 Jul 2011 09:08:35 -0400") References: <472DD1C2-77BB-43C0-8A07-2F8DA9A9EB10@daviesinc.com> Message-ID: <87y6005bep.fsf@qurzaw.varnish-software.com> ]] Chris Davies | And in VCL, how do I tell the health of the backend within a director? something like: backend foo { ? } director rr round-robin { { .backend = foo; } ? } sub vcl_recv { set req.backend = foo; if (!req.backend.healthy) { # Handle this } set req.backend = rr; } -- Tollef Fog Heen Varnish Software t: +47 21 98 92 64 From tfheen at varnish-software.com Fri Jul 15 06:35:11 2011 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Fri, 15 Jul 2011 08:35:11 +0200 Subject: RHEL 6 SElinux policy for 2.1.5 In-Reply-To: <22F96A574165AA4482FF290947347774025FAD@AUPDC00-MBX01P.au.ad.westfield.com> (Thiago Figueiro's message of "Tue, 21 Jun 2011 04:23:11 +0000") References: <22F96A574165AA4482FF290947347774025FAD@AUPDC00-MBX01P.au.ad.westfield.com> Message-ID: <87pqlc59wg.fsf@qurzaw.varnish-software.com> ]] Thiago Figueiro Hi, Thanks for spending the time writing the policy. | I'm not sure why varnishd needs to write to /etc/varnish but oh well... :-) It shouldn't need that. -- Tollef Fog Heen Varnish Software t: +47 21 98 92 64 From isp at daviesinc.com Fri Jul 15 11:37:18 2011 From: isp at daviesinc.com (Chris Davies) Date: Fri, 15 Jul 2011 07:37:18 -0400 Subject: Detecting health of Backend in Director in VCL In-Reply-To: <87y6005bep.fsf@qurzaw.varnish-software.com> References: <472DD1C2-77BB-43C0-8A07-2F8DA9A9EB10@daviesinc.com> <87y6005bep.fsf@qurzaw.varnish-software.com> Message-ID: On Jul 15, 2011, at 2:02 AM, Tollef Fog Heen wrote: > ]] Chris Davies > > | And in VCL, how do I tell the health of the backend within a director? > > something like: > > backend foo { > ? > } > > director rr round-robin { > { .backend = foo; } > ? > } > > sub vcl_recv { > set req.backend = foo; > if (!req.backend.healthy) { > # Handle this > } > set req.backend = rr; > } that is sort of what I started to figure out - Just seemed odd to have to set the backend, test, set the next backend. Thank you for the pointer in the right direction. From pmialon at free.fr Fri Jul 15 12:29:30 2011 From: pmialon at free.fr (Pierre-Gilles Mialon) Date: Fri, 15 Jul 2011 14:29:30 +0200 Subject: Persistent storage on varnish 3.0.0 Message-ID: <201107151429.31101.pmialon@free.fr> Le mercredi 13 juillet 2011 16:41:43, Pierre-Gilles Mialon a ?crit : > Hi, > > > We try to use persistent storage with varnish 3.0.0, > we run: > varnishd -d -a :80 -f /etc/varnish/default.vcl -s > persistent,/var/lib/varnish/instance/A.shm,10M > > On linux during the startup of varnishd the output mention: > CHK(0x7f1bbd81d0a0 SILO 0x7f1bbc7e1000 SILO) = 3 > > And the persistent storage is reset. I update this email, We found a workaround for Linux detailed in: https://www.varnish-cache.org/trac/ticket/962 > We try the same option with varnish 3.0.0 on amd64 arch > with the package and the latest git sources: > > gcc-version | 4.2 | 4.4 | 4.6 | > ------------------------------------------ > FreeBSD 8.2 | OK | - | KO | > Debian 6 | - | KO | KO | > Debian sid | - | KO | KO | > Fedora 15 | - | - | KO | > ------------------------------------------ > > We test using the following procedure: > . launch varnishd with the command below > . start child > . varnishreplay a log file > . display the number of object (~1000): > varnishstat -1 | grep '^n_objec' > . stop child > . start child > . display the number of object > . stop child > . quit > . relaunch the daemon > . start child > . display the number of object > > > before quit command, varnish persistent storage work, if we quit, > it only works on FreeBSD with gcc 4.2 > > > Is it a known issue ? > > Thanks for your help. -- Pierre-Gilles Mialon Yakaz -- Pierre-Gilles Mialon 15 rue du puits de l'Ermite 75005 Paris -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part. URL: From jhayter at manta.com Wed Jul 20 15:43:31 2011 From: jhayter at manta.com (Jim Hayter) Date: Wed, 20 Jul 2011 15:43:31 +0000 Subject: Varnish 2.0.5 - question on health status Message-ID: Running varnish 2.0.5 on Ubuntu 9.10. I have a question about how health checks are handled. As shown in the varnishlog entries below, the server stopped responding to the health checks. Once the count of successful checks dropped below 7 out of 10, it went sick. (Backend config for server also included below). What I find confusing is that immediately after that the health checks started succeeding but the count of successful checks out of the last 10 remained at 6 until 7 successive health checks succeeded and the count went to 7. It looks like being sick dropped the count to zero but didn't report that. Is this the way it should work? 0 Backend_health - server_abc Still healthy 4--X-S-RH 10 7 10 0.230660 0.097101 HTTP/1.1 200 OK 0 Backend_health - server_abc Still healthy 4--X-S--- 9 7 10 0.000000 0.097101 0 Backend_health - server_abc Still healthy 4--X-S--- 8 7 10 0.000000 0.097101 0 Backend_health - server_abc Still healthy 4--X-S--- 7 7 10 0.000000 0.097101 0 Backend_health - server_abc Went sick 4--X-S--- 6 7 10 0.000000 0.097101 0 Backend_health - server_abc Still sick 4--X-S-RH 6 7 10 0.694192 0.246374 HTTP/1.1 200 OK 0 Backend_health - server_abc Still sick 4--X-S-RH 6 7 10 0.025580 0.191176 HTTP/1.1 200 OK 0 Backend_health - server_abc Still sick 4--X-S-RH 6 7 10 0.022060 0.148897 HTTP/1.1 200 OK 0 Backend_health - server_abc Still sick 4--X-S-RH 6 7 10 0.018023 0.116178 HTTP/1.1 200 OK 0 Backend_health - server_abc Still sick 4--X-S-RH 6 7 10 0.019373 0.091977 HTTP/1.1 200 OK 0 Backend_health - server_abc Still sick 4--X-S-RH 6 7 10 0.073107 0.087259 HTTP/1.1 200 OK 0 Backend_health - server_abc Back healthy 4--X-S-RH 7 7 10 0.172980 0.108689 HTTP/1.1 200 OK 0 Backend_health - server_abc Still healthy 4--X-S-RH 8 7 10 0.076496 0.100641 HTTP/1.1 200 OK 0 Backend_health - server_abc Still healthy 4--X-S-RH 9 7 10 0.026741 0.082166 HTTP/1.1 200 OK 0 Backend_health - server_abc Still healthy 4--X-S-RH 10 7 10 0.020648 0.066786 HTTP/1.1 200 OK Backend server stanza in vcl: backend server_abc { .host = "server_abc"; .port = "8080"; .connect_timeout = 1s; .first_byte_timeout = 20s; .probe = { .request = "HEAD /favicon.ico HTTP/1.1" "Host: www." "User-Agent: varnish health probe" "Connection: close" ; .timeout = 1s; .interval = 2s; .window = 10; .threshold = 7; } } Thanks, Jim -- Jim Hayter From ken at simplyprecise.com Thu Jul 21 20:27:07 2011 From: ken at simplyprecise.com (ken at simplyprecise.com) Date: Thu, 21 Jul 2011 13:27:07 -0700 Subject: Configuration Questions & Assistance Message-ID: <20110721132707.144173gb7r4968wr@208.117.4.128> Hey All, I'm working on getting Varnish working with Apache 2.2.3 and PHP 5.3. I run multiple sites based off of Invision Power Board (eg: IP.Board), and they are configured in Apache as Name Virtual Hosts, and are all on the same IP Address. I've gotten Varnish to show the appropriate sites when visiting them, but keep coming across some issues. One of those such issues, is that it seems that when Varnish is enabled and I visit a page that is PHP based, but the PHP code starts with References: <20110721132707.144173gb7r4968wr@208.117.4.128> Message-ID: <20110721214619.GD2395@nat.myhome> > One of those such issues, is that it seems that when Varnish is > enabled and I visit a page that is PHP based, but the PHP code starts > with displayed on the browser. To me this sounds like your varnish isn't actually configured properly. Your web server is `in charge` of sending objects to varnish. If your php is configured to parse ' Another issue I'm having happens with javascript. We have a specific > Java Script based Ad that's displayed using Google Ad Manager. The > javascript file is hosted on our site, and called using Google Ad > Manager code. It seems that when varnish is enabled, that the > JavaScript code is being displayed rather than parsed as Java Script. This sounds much like the php problem/example. If your pages are being returned from the web server as plain text, then this would be a symptom. My suggestion is to double check your varnish config and make sure it points to a valid virtualhost. Double check your apache config to ensure you have a virtualhost listening on the ip address you told varnish to get its data. -- Paul Procacci Manager, UNIX Support Services Datapipe Managed Global IT Services 1.201.792.4847 (international) 1.888.749.5821 (toll free) ________________________________ This message may contain confidential or privileged information. If you are not the intended recipient, please advise us immediately and delete this message. See http://www.datapipe.com/about-us-legal-email-disclaimer.htm for further information on confidentiality and the risks of non-secure electronic communication. If you cannot access these links, please notify us by reply message and we will send the contents to you. From jacolmode at wp.pl Fri Jul 22 06:46:24 2011 From: jacolmode at wp.pl (Jacek) Date: Fri, 22 Jul 2011 08:46:24 +0200 Subject: Odp: streaming service In-Reply-To: <4e0b54d82cd799.04026933@wp.pl> References: <4e0b54d82cd799.04026933@wp.pl> Message-ID: <4e291cc053c7b1.43454733@wp.pl> Anyone? I would be very grateful for any suggestions. Dnia 29-06-2011 o godz. 18:37 Jacek Placek napisa?(a): > Hi, > > I'm planning on using Varnish in my project as a reverse proxy cache. > Mainly to cache dynamically created streaming content at the backend > side. The process is CPU intensive and takes a fair amount of time to > complete. > > I would like to ask you all which of the following requirements can be > accomplished by using the build-in functions, writing a dedicated module > (version 3.0) or changing the default functionality by patching the > source: > > 1) rate-limiting - a memcached set of counters, ex. ip address/time, > uri/time, ... > > 2) streaming - content generated via a backend should be streamed ASAP > to the client and saved to cache at the same time - ideally while the > content is generated it would be very good to have the ability to stream > to more then one client (non blocking object) > > 3) request manipulation - incoming request from a client is a HTTP GET > (domain/object_id) and should be transformed into HTTP POST to a backend > app extended with additional set of parameters (headers) taken from db > (memcache, mongodb, tokyo tyrant, mysql) > > 4) cache expire - based on popularity and size of an object - free space > should be reclaimed by purging least popular objects (amount of traffic > they generate) > > Thank you in advance, > > Jacek. > > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc From sumitraja at gmail.com Fri Jul 22 10:28:46 2011 From: sumitraja at gmail.com (Sumit Raja) Date: Fri, 22 Jul 2011 11:28:46 +0100 Subject: Auth before serving from cache/backend Message-ID: Hello, I am evaluating Varnish to put in front of an authenticated web application. Essentially I am trying to get Varnish to make a HEAD request to the back end for authentication before it serves from cache or passes the GET downstream. The web app uses basic auth. My first attempt seems to work, that is the HEAD request is made and content is cached correctly, until the HEAD to the back end results in a 401. From this point on Varnish always serves the 401 for that request, even after making the HEAD and getting a 2xx from the back end. I'm pretty sure I am missing something basic as this is my first attempt but some help would be appreciated with my config. The VCL is: sub vcl_recv { if (req.request != "GET" && . . //from default.vcl . . .// virtual host selection . if (req.restarts == 0) { . . // X-forwarded for from default.vcl . if (req.http.Authorization && req.backend == api) { return(pass); } } return (lookup); } sub vcl_pass { if (req.http.Authorization && req.backend == api && req.restarts == 0) { set bereq.request = "HEAD"; } return (pass); } sub vcl_fetch { if (req.http.Authorization && req.backend == api && req.restarts == 0) { if (beresp.status >= 200 && beresp.status < 400) { return(restart); } elsif (beresp.status != 401) { return(error); } else { error 401 "Not Authorised"; } } else { if (beresp.ttl <= 0s || beresp.http.Set-Cookie || beresp.http.Vary == "*") { set beresp.ttl = 120 s; return (hit_for_pass); } return (deliver); } } Thanks Sumit -------------- next part -------------- An HTML attachment was scrubbed... URL: From l at lrowe.co.uk Fri Jul 22 13:06:59 2011 From: l at lrowe.co.uk (Laurence Rowe) Date: Fri, 22 Jul 2011 14:06:59 +0100 Subject: streaming service In-Reply-To: <4e291cc053c7b1.43454733@wp.pl> References: <4e0b54d82cd799.04026933@wp.pl> <4e291cc053c7b1.43454733@wp.pl> Message-ID: On 22 July 2011 07:46, Jacek wrote: > Anyone? > > I would be very grateful for any suggestions. > > Dnia 29-06-2011 o godz. 18:37 Jacek Placek napisa?(a): >> Hi, >> >> I'm planning on using Varnish in my project as a reverse proxy cache. >> Mainly to cache dynamically created streaming content at the backend >> side. The process is CPU intensive and takes a fair amount of time to >> complete. >> >> I would like to ask you all which of the following requirements can be >> accomplished by using the build-in functions, writing a dedicated module >> (version 3.0) or changing the default functionality by patching the >> source: >> >> 1) rate-limiting - a memcached set of counters, ex. ip address/time, >> uri/time, ... >> >> 2) streaming - content generated via a backend should be streamed ASAP >> to the client and saved to cache at the same time - ideally while the >> content is generated it would be very good to have the ability to stream >> to more then one client (non blocking object) Except when using pipe mode (which means no caching or any further manipulation by Varnish), response objects are buffered into Varnish before being returned to the browser. This makes Varnish unsuitable for caching large numbers of videos / ISOs etc as whenever you hit an uncached object the entire large file will be read into Varnish before being returned to the user (even if all you want to do is manipulate some response headers.) >> 3) request manipulation - incoming request from a client is a HTTP GET >> (domain/object_id) and should be transformed into HTTP POST to a backend >> app extended with additional set of parameters (headers) taken from db >> (memcache, mongodb, tokyo tyrant, mysql) Varnish does not support changing the body of a request. >> 4) cache expire - based on popularity and size of an object - free space >> should be reclaimed by purging least popular objects (amount of traffic >> they generate) Yes, you can place a limit on the size of the Varnish cache and space is reclaimed along those lines. So I don't think Varnish will work for your use case. Laurence From l at lrowe.co.uk Fri Jul 22 13:17:23 2011 From: l at lrowe.co.uk (Laurence Rowe) Date: Fri, 22 Jul 2011 14:17:23 +0100 Subject: Auth before serving from cache/backend In-Reply-To: References: Message-ID: On 22 July 2011 11:28, Sumit Raja wrote: > Hello, > > I am evaluating Varnish to put in front of an authenticated web application. > Essentially I am trying to get Varnish to make a HEAD request to the back > end for authentication before it serves from cache or passes the GET > downstream. The web app uses basic auth. > > My first attempt seems to work, that is the HEAD request is made and content > is cached correctly, until the HEAD to the back end results in a 401. From > this point on Varnish always serves the 401 for that request, even after > making the HEAD and getting a 2xx from the back end. > > I'm pretty sure I am missing something basic as this is my first attempt but > some help would be appreciated with my config. The VCL is: > > sub vcl_recv { > ??? if (req.request != "GET" && > . > . //from default.vcl > . > . > .// virtual host selection > . > > ??? if (req.restarts == 0) { > ??????? . > ? ? ??? . // X-forwarded for from default.vcl > ??????? . > ??????? if (req.http.Authorization && req.backend == api) { > ??? ??????? return(pass); > ????? ? } > ??? } > ??? return (lookup); > } > > sub vcl_pass { > ?? if (req.http.Authorization && req.backend == api && req.restarts == 0) { > ?????? set bereq.request = "HEAD"; > ?? } > ?? return (pass); > } > > sub vcl_fetch { > > ??? if (req.http.Authorization && req.backend == api && req.restarts == 0) { > ??????? if (beresp.status >= 200 && beresp.status < 400) { > ??? ??????? return(restart); > ??????? } elsif (beresp.status != 401) { > ??? ??????? return(error); > ??????? } else { > ??? ??????? error 401 "Not Authorised"; > ?? ???? } > ? ? } else { > ??????? if (beresp.ttl <= 0s || > ??????????? beresp.http.Set-Cookie || > ??????????? beresp.http.Vary == "*") { > ??? ??????? set beresp.ttl = 120 s; > ??? ??????? return (hit_for_pass); > ??????? } > ??????? return (deliver); > ??? } > } > See this diagram of the flow in VCL: https://www.varnish-cache.org/trac/wiki/VCLExampleDefault You probably want something like this instead of your vcl_pass: sub vcl_hit { ?? if (req.http.Authorization && req.backend == api && req.restarts == 0) { ?????? set bereq.request = "HEAD"; return (pass); ?? } } Laurence From robertomoutinho at gmail.com Fri Jul 22 21:58:31 2011 From: robertomoutinho at gmail.com (Roberto Moutinho) Date: Fri, 22 Jul 2011 18:58:31 -0300 Subject: [Newbie] Need help with Inline C code. Message-ID: Hello to everyone, I've extended the C code on the http://blog.dansingerman.com/post/4604532761/how-to-block-rate-limited-traffic-with-varnishpage... But I have different needs from the original post since I'm serving a protected content all the time (Restfull API). Basically I have to authenticate a api key (received within a special header) on my database and store the key and theirs request limit value in memached. My main problem is that my C code works perfect when using gcc (ubuntu) to compile it but it does not compile together with Varnish. output from running varnishd * storage_malloc: max size 256 MB. Message from C-compiler: ./vcl.1P9zoqAU.c: In function ?API?: ./vcl.1P9zoqAU.c:602: error: invalid initializer Running C-compiler failed, exit 1 VCL compilation failed* Searching for this "invalid initializer" on Google have not helped me at all :( Best Regards Roberto Moutinho -------------- next part -------------- An HTML attachment was scrubbed... URL: From jonathan.hursey at adrevolution.com Fri Jul 22 22:06:14 2011 From: jonathan.hursey at adrevolution.com (Jonathan Hursey) Date: Fri, 22 Jul 2011 17:06:14 -0500 Subject: [Newbie] Need help with Inline C code. In-Reply-To: References: Message-ID: wish I could help with this... if you get an answer please forward it this way. I'm super curious. On Fri, Jul 22, 2011 at 4:58 PM, Roberto Moutinho wrote: > Hello to everyone, > > I've extended the C code on the > http://blog.dansingerman.com/post/4604532761/how-to-block-rate-limited-traffic-with-varnishpage... > But I have different needs from the original post since I'm serving a > protected content all the time (Restfull API). > > Basically I have to authenticate a api key (received within a special > header) on my database and store the key and theirs request limit value in > memached. > > My main problem is that my C code works perfect when using gcc (ubuntu) to > compile it but it does not compile together with Varnish. > > output from running varnishd > * > storage_malloc: max size 256 MB. > Message from C-compiler: > ./vcl.1P9zoqAU.c: In function ?API?: > ./vcl.1P9zoqAU.c:602: error: invalid initializer > Running C-compiler failed, exit 1 > VCL compilation failed* > > Searching for this "invalid initializer" on Google have not helped me at > all :( > > Best Regards > Roberto Moutinho > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kbrownfield at google.com Fri Jul 22 22:10:10 2011 From: kbrownfield at google.com (Ken Brownfield) Date: Fri, 22 Jul 2011 15:10:10 -0700 Subject: [Newbie] Need help with Inline C code. In-Reply-To: References: Message-ID: Hard to tell without seeing any code at all, and I "C" no code at that link. :) OT: I think scaling 100 million requests a month is a... dubious accomplishment -- 38 hits per second? Really? -- kb On Fri, Jul 22, 2011 at 14:58, Roberto Moutinho wrote: > Hello to everyone, > > I've extended the C code on the > http://blog.dansingerman.com/post/4604532761/how-to-block-rate-limited-traffic-with-varnishpage... > But I have different needs from the original post since I'm serving a > protected content all the time (Restfull API). > > Basically I have to authenticate a api key (received within a special > header) on my database and store the key and theirs request limit value in > memached. > > My main problem is that my C code works perfect when using gcc (ubuntu) to > compile it but it does not compile together with Varnish. > > output from running varnishd > * > storage_malloc: max size 256 MB. > Message from C-compiler: > ./vcl.1P9zoqAU.c: In function ?API?: > ./vcl.1P9zoqAU.c:602: error: invalid initializer > Running C-compiler failed, exit 1 > VCL compilation failed* > > Searching for this "invalid initializer" on Google have not helped me at > all :( > > Best Regards > Roberto Moutinho > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark at simplercomputing.net Fri Jul 22 22:42:42 2011 From: mark at simplercomputing.net (Mark E) Date: Fri, 22 Jul 2011 16:42:42 -0600 Subject: Varnish 3 serving same set of pages for different domains Message-ID: <4E29FCE2.8030305@simplercomputing.net> Hi, I have a strange issue that I need some help with. Here's the server config: - CentOS - Varnish 3 on port 80 installed in front of Apache on port 8080 - 6 web sites exist on the server, all with individual unique domains and or subdomains - All sites work fine via Apache When I add Varnish 3, pointing it at Apache as the backend, I see the same site content no matter what subdomain/domain I open in a browser. But the URL in the browser always appears correct. To troubleshoot this I've gone to a generic VCL with no rules at all, I only have the "backend" declaration to tell Varnish that Apache is on port 80 at x.x.x.x address - and to be clear I'm using the default.vcl that comes with Varnish 3 with the RedHat repository, which has no rules, etc., enabled. So to summarize this is what happens: - Enable Varnish - Open a.com and in a browser and it works fine, I see a.com content. - Open b.com in a browser and b.com is shown in the browser address bar, but the content I see is from a.com. - Disable Varnish, put Apache back on port 80, and both a.com and b.com load properly in a browser - I see the correct content for each site. Any ideas why this happens? Mark From robertomoutinho at gmail.com Fri Jul 22 23:24:33 2011 From: robertomoutinho at gmail.com (Roberto Moutinho) Date: Fri, 22 Jul 2011 20:24:33 -0300 Subject: [Newbie] Need help with Inline C code. In-Reply-To: References: Message-ID: My mistake... I've posted the wrong link. this is the one I was referring to: http://drcarter.info/2010/04/how-fighting-against-scraping-using-varnish-vcl-inline-c-memcached/ 2011/7/22 Ken Brownfield Hard to tell without seeing any code at all, and I "C" no code at that link. :) OT: I think scaling 100 million requests a month is a... dubious accomplishment -- 38 hits per second? Really? -- kb 2011/7/22 Jonathan Hursey > wish I could help with this... if you get an answer please forward it this > way. I'm super curious. > > > On Fri, Jul 22, 2011 at 4:58 PM, Roberto Moutinho < > robertomoutinho at gmail.com> wrote: > >> Hello to everyone, >> >> I've extended the C code on the >> http://blog.dansingerman.com/post/4604532761/how-to-block-rate-limited-traffic-with-varnishpage... >> But I have different needs from the original post since I'm serving a >> protected content all the time (Restfull API). >> >> Basically I have to authenticate a api key (received within a special >> header) on my database and store the key and theirs request limit value in >> memached. >> >> My main problem is that my C code works perfect when using gcc (ubuntu) to >> compile it but it does not compile together with Varnish. >> >> output from running varnishd >> * >> storage_malloc: max size 256 MB. >> Message from C-compiler: >> ./vcl.1P9zoqAU.c: In function ?API?: >> ./vcl.1P9zoqAU.c:602: error: invalid initializer >> Running C-compiler failed, exit 1 >> VCL compilation failed* >> >> Searching for this "invalid initializer" on Google have not helped me at >> all :( >> >> Best Regards >> Roberto Moutinho >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kbrownfield at google.com Sat Jul 23 00:07:25 2011 From: kbrownfield at google.com (Ken Brownfield) Date: Fri, 22 Jul 2011 17:07:25 -0700 Subject: [Newbie] Need help with Inline C code. In-Reply-To: References: Message-ID: Actually, more useful would be the output of "varnishd -C -f /path/to/your/VCL/file.vcl", in whole or primarily the area around line 602... Not sure how to get away without actually looking at the cause of the error. If it just said "syntax error", Google wouldn't be too helpful for that, either... :-/ -- kb On Fri, Jul 22, 2011 at 16:24, Roberto Moutinho wrote: > My mistake... I've posted the wrong link. > > this is the one I was referring to: > > http://drcarter.info/2010/04/how-fighting-against-scraping-using-varnish-vcl-inline-c-memcached/ > > > > 2011/7/22 Ken Brownfield > Hard to tell without seeing any code at all, and I "C" no code at that > link. > :) > > OT: I think scaling 100 million requests a month is a... dubious > accomplishment -- 38 hits per second? Really? > -- > kb > > 2011/7/22 Jonathan Hursey > >> wish I could help with this... if you get an answer please forward it this >> way. I'm super curious. >> >> >> On Fri, Jul 22, 2011 at 4:58 PM, Roberto Moutinho < >> robertomoutinho at gmail.com> wrote: >> >>> Hello to everyone, >>> >>> I've extended the C code on the >>> http://blog.dansingerman.com/post/4604532761/how-to-block-rate-limited-traffic-with-varnishpage... >>> But I have different needs from the original post since I'm serving a >>> protected content all the time (Restfull API). >>> >>> Basically I have to authenticate a api key (received within a special >>> header) on my database and store the key and theirs request limit value in >>> memached. >>> >>> My main problem is that my C code works perfect when using gcc (ubuntu) >>> to compile it but it does not compile together with Varnish. >>> >>> output from running varnishd >>> * >>> storage_malloc: max size 256 MB. >>> Message from C-compiler: >>> ./vcl.1P9zoqAU.c: In function ?API?: >>> ./vcl.1P9zoqAU.c:602: error: invalid initializer >>> Running C-compiler failed, exit 1 >>> VCL compilation failed* >>> >>> Searching for this "invalid initializer" on Google have not helped me at >>> all :( >>> >>> Best Regards >>> Roberto Moutinho >>> >>> _______________________________________________ >>> varnish-misc mailing list >>> varnish-misc at varnish-cache.org >>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >>> >> >> > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sime at sime.net.au Sat Jul 23 00:20:54 2011 From: sime at sime.net.au (Simon Males) Date: Sat, 23 Jul 2011 10:20:54 +1000 Subject: Varnish 3 serving same set of pages for different domains In-Reply-To: <4E29FCE2.8030305@simplercomputing.net> References: <4E29FCE2.8030305@simplercomputing.net> Message-ID: On Sat, Jul 23, 2011 at 8:42 AM, Mark E wrote: > Hi, > > I have a strange issue that I need some help with. Here's the server config: > > - CentOS > > - Varnish 3 on port 80 installed in front of Apache on port 8080 > > - 6 web sites exist on the server, all with individual unique domains and or > subdomains > > - All sites work fine via Apache > > When I add Varnish 3, pointing it at Apache as the backend, I see the same > site content no matter what subdomain/domain I open in a browser. But the > URL in the browser always appears correct. > > To troubleshoot this I've gone to a generic VCL with no rules at all, I only > have the "backend" declaration to tell Varnish that Apache is on port 80 at > x.x.x.x address - and to be clear I'm using the default.vcl that comes with > Varnish 3 with the RedHat repository, which has no rules, etc., enabled. Maybe confirm you are using the supplied VCL by dumping it: varnishadm -S /etc/varnish/secret -T localhost:6082 vcl.show boot -- Simon Males From apj at mutt.dk Sat Jul 23 07:46:50 2011 From: apj at mutt.dk (Andreas Plesner Jacobsen) Date: Sat, 23 Jul 2011 09:46:50 +0200 Subject: Varnish 3 serving same set of pages for different domains In-Reply-To: <4E29FCE2.8030305@simplercomputing.net> References: <4E29FCE2.8030305@simplercomputing.net> Message-ID: <20110723074650.GI4654@nerd.dk> On Fri, Jul 22, 2011 at 04:42:42PM -0600, Mark E wrote: > > - Varnish 3 on port 80 installed in front of Apache on port 8080 Did you test apache when running on port 8080? You may not have virtualhosts enabled when running as anything other than port 80. -- Andreas From robertomoutinho at gmail.com Mon Jul 25 13:36:18 2011 From: robertomoutinho at gmail.com (Roberto Moutinho) Date: Mon, 25 Jul 2011 10:36:18 -0300 Subject: [Newbie] Need help with Inline C code. In-Reply-To: References: Message-ID: Ken Brownfield, Using the command you mentioned (with the "-C" option on varnish daemon) I was able to find my mistake. I was initializing a variable with a "Varnish Run Time" value. Since this value is only accessible when a request is made, varnishd was not able to compile my code. The wrong code: * char xqbkey[32] = VRT_GetHdr(sp, HDR_REQ, "\012X-API-Key:");* And what I did after seeing my mistake: * char apikey[32]; sprintf(apikey, VRT_GetHdr(sp, HDR_REQ, "\012X-API-Key:"))* Thanks for the tip. Best Regards Roberto Moutinho Actually, more useful would be the output of "varnishd -C -f /path/to/your/VCL/file.vcl", in whole or primarily the area around line 602... Not sure how to get away without actually looking at the cause of the error. If it just said "syntax error", Google wouldn't be too helpful for that, either... :-/ -- kb On Fri, Jul 22, 2011 at 16:24, Roberto Moutinho >wrote: >* My mistake... I've posted the wrong link.*>**>* this is the one I was referring to:*>**>* http://drcarter.info/2010/04/how-fighting-against-scraping-using-varnish-vcl-inline-c-memcached/*>**>**>**>* 2011/7/22 Ken Brownfield >*>* Hard to tell without seeing any code at all, and I "C" no code at that*>* link.*>* :)*>**>* OT: I think scaling 100 million requests a month is a... dubious*>* accomplishment -- 38 hits per second? Really?*>* --*>* kb*>**>* 2011/7/22 Jonathan Hursey >*>**>>* wish I could help with this... if you get an answer please forward it this*>>* way. I'm super curious.*>>**>>**>>* On Fri, Jul 22, 2011 at 4:58 PM, Roberto Moutinho <*>>* robertomoutinho at gmail.com > wrote:*>>**>>>* Hello to everyone,*>>>**>>>* I've extended the C code on the*>>>* http://blog.dansingerman.com/post/4604532761/how-to-block-rate-limited-traffic-with-varnishpage...*>>>* But I have different needs from the original post since I'm serving a*>>>* protected content all the time (Restfull API).*>>>**>>>* Basically I have to authenticate a api key (received within a special*>>>* header) on my database and store the key and theirs request limit value in*>>>* memached.*>>>**>>>* My main problem is that my C code works perfect when using gcc (ubuntu)*>>>* to compile it but it does not compile together with Varnish.*>>>**>>>* output from running varnishd*>>>* **>>>* storage_malloc: max size 256 MB.*>>>* Message from C-compiler:*>>>* ./vcl.1P9zoqAU.c: In function ?API?:*>>>* ./vcl.1P9zoqAU.c:602: error: invalid initializer*>>>* Running C-compiler failed, exit 1*>>>* VCL compilation failed**>>>**>>>* Searching for this "invalid initializer" on Google have not helped me at*>>>* all :(*>>>**>>>* Best Regards*>>>* Roberto Moutinho*>>>**>>>* _______________________________________________*>>>* varnish-misc mailing list*>>>* varnish-misc at varnish-cache.org *>>>* https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc*>>>**>>**>>**>**>* _______________________________________________*>* varnish-misc mailing list*>* varnish-misc at varnish-cache.org *>* https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc*>**-------------- next part -------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From robertomoutinho at gmail.com Wed Jul 27 00:50:20 2011 From: robertomoutinho at gmail.com (Roberto Moutinho) Date: Tue, 26 Jul 2011 21:50:20 -0300 Subject: Using MySQL C API inside varnish Message-ID: Hello, As I said on an earlier thread... I'm trying to adapt varnish to authenticate users for my website API and I'm bumping into some troubles... I receive a header named x-api-key on every request made to the API and I need to authenticate those api keys before I start delivering cached content to my clients. I start with a variable that receive the value of *VRT_GetHdr(sp, HDR_REQ, "\012X-API-Key:")* and I would like to consult my database to find out if this key is valid. So I started writing a little C code to do it... *unsigned int num_rows; MYSQL *conn; MYSQL_RES *result; MYSQL_ROW row; conn = mysql_init(NULL); mysql_real_connect(conn, "Server IP", "username", "password", "database", 0, NULL, 0); mysql_query(conn, query); result = mysql_store_result(conn); num_rows = mysql_num_rows(result); row = mysql_fetch_row(result); char *limit = row[0]; if (num_rows == 0) { VRT_error(sp, 403, "Forbidden"); VRT_done(sp, VCL_RET_ERROR); } mysql_free_result(result); mysql_close(conn); }* If the databse return no rows for the key provided on the request that means that the key is invalid and I should return a 403 forbidden to the client. The C code works perfectly when compiled with GCC on ubuntu/centos and also compile inside the VCL. My daemon startup options are (staging server): *varnishd -s malloc,1G -T 127.0.0.1:2000 -P /www/varnishd.pid -f /etc/varnish/default.vcl -p 'cc_command=exec cc -fpic -shared -Wl,-x -L/usr/local/include/libmemcached/memcached.h -L/usr/lib64/mysql -lmysqlclient -lmemcached -o %o %s' -p 'cli_timeout=10' -p 'thread_pool_max=5000' -p 'thread_pools=4' -p 'thread_pool_min=200' -p 'thread_pool_add_delay=1ms' -p 'cli_timeout=1000s' -p 'ping_interval=1' -p 'cli_buffer=16384' -p 'session_linger=20ms' -p 'lru_interval=360s' -p 'listen_depth=8192' -h 'classic,500009';* and this is what I get from /var/log/messages: *Jul 25 19:59:09 api varnishd[6318]: child (6319) Started Jul 25 19:59:09 api varnishd[6318]: Child (6319) said Jul 25 19:59:09 api varnishd[6318]: Child (6319) said Child starts Jul 25 19:59:09 api varnishd[6318]: Child (6319) said managed to mmap 68230176768 bytes of 68230176768 Jul 25 16:59:16 api kernel: varnishd[6726]: segfault at 0000000000000000 rip 0000003839e6f0f1 rsp 00002abb50cb9d90 error 6 Jul 25 19:59:17 api varnishd[6318]: Child (6319) died signal=11 Jul 25 19:59:17 api varnishd[6318]: child (7128) Started Jul 25 19:59:17 api varnishd[6318]: Child (7128) said Jul 25 19:59:17 api varnishd[6318]: Child (7128) said Child starts Jul 25 19:59:17 api varnishd[6318]: Child (7128) said managed to mmap 68230176768 bytes of 68230176768* and if I start the daemon with a "-d -d" option I get this: *Varnish on Linux,2.6.18-194.el5,x86_64,-smalloc,-hcritbit 200 233 ----------------------------- Varnish Cache CLI 1.0 ----------------------------- Linux,2.6.18-194.el5,x86_64,-smalloc,-hcritbit Type 'help' for command list. Type 'quit' to close CLI session. Type 'start' to launch worker process. start child (13163) Started 200 0 Child (13184) died signal=11 Child cleanup complete child (13203) Started Child (13203) said Child (13203) said Child starts* I understand that signal 11 means that the program accessed a memory location that was not assigned...and probably died because of it. Is it really a bug in my code or am I doing something wrong in trying to make MySQL C API work with varnish ? ps: sorry for the long message. -------------- next part -------------- An HTML attachment was scrubbed... URL: From shib4u at gmail.com Wed Jul 27 11:34:17 2011 From: shib4u at gmail.com (Shibashish) Date: Wed, 27 Jul 2011 17:04:17 +0530 Subject: varnish backend redirection based on url Message-ID: Hi, I have a main website running which has now been translated to different languages. The different language sites are running on different backend servers. I want to redirect/distribute traffic to backends based on the url without rewriting the url/host. e.g www.domain.com/en -> backend1 www.domain.com/hi -> backend2 www.domain.com/ma -> backend3 www.domain.com/ch -> backend4 www.domain.com/ja -> backend5 The content should be served on the url www.domain.com/ja, i do not want to rewrite the domain as ja.domain.com. Is this possible in varnish or should i use mod_proxy/Apache ? TIA. ShiB. while ( ! ( succeed = try() ) ); -------------- next part -------------- An HTML attachment was scrubbed... URL: From mattias at nucleus.be Wed Jul 27 11:48:11 2011 From: mattias at nucleus.be (Mattias Geniar) Date: Wed, 27 Jul 2011 13:48:11 +0200 Subject: varnish backend redirection based on url In-Reply-To: References: Message-ID: <18834F5BEC10824891FB8B22AC821A5A017354B3@nucleus-srv01.Nucleus.local> > I have a main website running which has now been translated to different languages. The different language sites are running on different backend servers. I want to redirect/distribute traffic to backends based on the url without rewriting the url/host. That's possible, you can set a different backend based on a (part of the) URL. Have a look here: https://www.varnish-cache.org/docs/trunk/tutorial/advanced_backend_serve rs.html Specifically, you'd want something as: sub vcl_recv { if (req.url ~ "^/en/") { set req.backend = backend_en; } elseif (req.url ~ "^/hi/" { set req.backend = backend_hi; } } Regards, Mattias From ken at simplyprecise.com Wed Jul 27 17:19:05 2011 From: ken at simplyprecise.com (ken at simplyprecise.com) Date: Wed, 27 Jul 2011 10:19:05 -0700 Subject: VCL Seems to be ignored? Message-ID: <20110727101905.175339upeii4pj0p@208.117.4.128> Hello, in Varnish 3.0.0 I have the following in my VCL recv and VCL fetch functions: if (!((req.http.Cookie ~ "member_id=" && req.http.Cookie !~ "member_id=(0|-1)") || req.http.Cookie ~ "(guestSkinChoice|language)")) { Execute what's here.... } What I *think* this should do is only "Execute what's here" if: there is not a member_id cookie there is not a member_id cookie that equals 0 or -1 Am I incorrect? From matthew at matthewwilkes.co.uk Wed Jul 27 17:31:00 2011 From: matthew at matthewwilkes.co.uk (Matthew Wilkes) Date: Wed, 27 Jul 2011 18:31:00 +0100 Subject: VCL Seems to be ignored? In-Reply-To: <20110727101905.175339upeii4pj0p@208.117.4.128> References: <20110727101905.175339upeii4pj0p@208.117.4.128> Message-ID: <539E75AB-9662-4CCA-98AB-B2A14607640D@matthewwilkes.co.uk> On 2011-07-27, at 1819, ken at simplyprecise.com wrote: > > > if (!((req.http.Cookie ~ "member_id=" && req.http.Cookie !~ "member_id=(0|-1)") || req.http.Cookie ~ "(guestSkinChoice|language)")) { > Execute what's here.... > } > > What I *think* this should do is only "Execute what's here" if: > there is not a member_id cookie > there is not a member_id cookie that equals 0 or -1 > > Am I incorrect? Let's build this up bit by bit: (req.http.Cookie ~ "member_id=" && req.http.Cookie !~ "member_id=(0|-1)") There is a cookie with a name ending in 'member_id', and it doesn't with 0 or -1. We call this the member_id condition. > (!(member_id condition || req.http.Cookie ~ "(guestSkinChoice|language)")) The next bit is: req.http.Cookie ~ "(guestSkinChoice|language)" 'getSkinChoice' or 'language' exists in any cookie name or value. We call this one the preferences condition. > (!(member_id condition || preferences_condition)) So we end up with a NOT-ed OR, meaning neither. Hence, your condition means: There is not a cookie with a name ending in 'member_id', or there is one that starts with 0 or -1. In addition, neither 'getSkinChoice' nor 'language' exist in any cookie name or value. If you get confused by boolean logic it's best to write the conditions out as nested if statements. Matt From ken at simplyprecise.com Wed Jul 27 17:38:36 2011 From: ken at simplyprecise.com (ken at simplyprecise.com) Date: Wed, 27 Jul 2011 10:38:36 -0700 Subject: VCL Seems to be ignored? In-Reply-To: <539E75AB-9662-4CCA-98AB-B2A14607640D@matthewwilkes.co.uk> References: <20110727101905.175339upeii4pj0p@208.117.4.128> <539E75AB-9662-4CCA-98AB-B2A14607640D@matthewwilkes.co.uk> Message-ID: <20110727103836.40432qgc8fwzrg2k@208.117.4.128> Hi Matt, Thanks for the response. In the beginning of the if statement we have "if (!(" what does the exclamation point symbolize there? Also, here: req.http.Cookie !~ "member_id=(0|-1)" This is not the same as saying that the cookie member_id does not have the value of 0 or -1? Thanks again, I appreciate the assistance here. Quoting "Matthew Wilkes" : > > On 2011-07-27, at 1819, ken at simplyprecise.com wrote: > >> >> >> if (!((req.http.Cookie ~ "member_id=" && req.http.Cookie !~ >> "member_id=(0|-1)") || req.http.Cookie ~ >> "(guestSkinChoice|language)")) { >> Execute what's here.... >> } >> >> What I *think* this should do is only "Execute what's here" if: >> there is not a member_id cookie >> there is not a member_id cookie that equals 0 or -1 >> >> Am I incorrect? > > Let's build this up bit by bit: > > (req.http.Cookie ~ "member_id=" && req.http.Cookie !~ "member_id=(0|-1)") > > There is a cookie with a name ending in 'member_id', and it doesn't > with 0 or -1. We call this the member_id condition. > >> (!(member_id condition || req.http.Cookie ~ "(guestSkinChoice|language)")) > > > The next bit is: > > req.http.Cookie ~ "(guestSkinChoice|language)" > > 'getSkinChoice' or 'language' exists in any cookie name or value. > We call this one the preferences condition. > >> (!(member_id condition || preferences_condition)) > > So we end up with a NOT-ed OR, meaning neither. Hence, your condition means: > > There is not a cookie with a name ending in 'member_id', or there is > one that starts with 0 or -1. In addition, neither 'getSkinChoice' > nor 'language' exist in any cookie name or value. > > If you get confused by boolean logic it's best to write the > conditions out as nested if statements. > > Matt > > > From carsten.dietz at arejo.de Thu Jul 28 12:16:29 2011 From: carsten.dietz at arejo.de (Carsten Dietz) Date: Thu, 28 Jul 2011 14:16:29 +0200 Subject: Loadbalancing with 301 / 302 Message-ID: <002d01cc4d20$2e793790$8b6ba6b0$@dietz@arejo.de> Hi, I want to setup varnish as an loadbalancer for 4-5 Downloadserver. The best way to loadbalance with Failover is Roundrobin. backend img4 { .host = "img4.test.com"; .probe = { .url = "/"; .interval = 5s; .timeout = 1 s; .window = 5; .threshold = 3; } } backend img5 { .host = "img5.test.com "; .probe = { .url = "/"; .interval = 5s; .timeout = 1 s; .window = 5; .threshold = 3; } } backend img6 { .host = "img6.test.com "; .probe = { .url = "/"; .interval = 5s; .timeout = 1 s; .window = 5; .threshold = 3; } } backend img7 { .host = "img7.test.com "; .probe = { .url = "/"; .interval = 5s; .timeout = 1 s; .window = 5; .threshold = 3; } } director baz round-robin { { .backend = img4; } { .backend = img5; } { .backend = img6; } { .backend = img7; } } sub vcl_recv { set obj.http.Location = baz; set obj.status = 301; return(deliver); } I try to set other variables but it try to start with this gives an error. Has anybody an idea how it can works? Thanks Carsten -------------- next part -------------- An HTML attachment was scrubbed... URL: From enno at tty.nl Thu Jul 28 12:26:54 2011 From: enno at tty.nl (Enno van Amerongen) Date: Thu, 28 Jul 2011 14:26:54 +0200 Subject: Loadbalancing with 301 / 302 In-Reply-To: <002d01cc4d20$2e793790$8b6ba6b0$@dietz@arejo.de> References: <002d01cc4d20$2e793790$8b6ba6b0$@dietz@arejo.de> Message-ID: <4E31558E.4090405@tty.nl> Hi, First of all, if you get an error, tell us what it is. Second, you seem to have copypasted the healthchecks directly from the docs, not sure if it works or if its a typo in the docs, but I'd say: .timeout = 1s (without the space) Third, your vcl_recv is wrong, try: sub vcl_recv { set req.backend = baz; return (lookup); } good luck! Carsten Dietz wrote: > > Hi, > > > > I want to setup varnish as an loadbalancer for 4-5 Downloadserver. > > The best way to loadbalance with Failover is Roundrobin. > > > > > > backend img4 { > > .host = "img4.test.com"; > > .probe = { > > .url = "/"; > > .interval = 5s; > > .timeout = 1 s; > > .window = 5; > > .threshold = 3; > > } > > } > > > > backend img5 { > > .host = "img5.test.com "; > > .probe = { > > .url = "/"; > > .interval = 5s; > > .timeout = 1 s; > > .window = 5; > > .threshold = 3; > > } > > } > > > > > > backend img6 { > > .host = "img6.test.com "; > > .probe = { > > .url = "/"; > > .interval = 5s; > > .timeout = 1 s; > > .window = 5; > > .threshold = 3; > > } > > } > > > > backend img7 { > > .host = "img7.test.com "; > > .probe = { > > .url = "/"; > > .interval = 5s; > > .timeout = 1 s; > > .window = 5; > > .threshold = 3; > > } > > } > > > > director baz round-robin { > > { > > .backend = img4; > > } > > { > > .backend = img5; > > } > > { > > .backend = img6; > > } > > { > > .backend = img7; > > } > > > > } > > > > sub vcl_recv { > > set obj.http.Location = baz; > > set obj.status = 301; > > return(deliver); > > } > > > > I try to set other variables but it try to start with this gives an error. > > > > > Has anybody an idea how it can works? > > > > Thanks Carsten > > ------------------------------------------------------------------------ > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc From enno at tty.nl Thu Jul 28 13:45:23 2011 From: enno at tty.nl (Enno van Amerongen) Date: Thu, 28 Jul 2011 15:45:23 +0200 Subject: AW: Loadbalancing with 301 / 302 In-Reply-To: <004101cc4d26$a8da8150$fa8f83f0$@dietz@arejo.de> References: <002d01cc4d20$2e793790$8b6ba6b0$@dietz@arejo.de> <4E31558E.4090405@tty.nl> <004101cc4d26$a8da8150$fa8f83f0$@dietz@arejo.de> Message-ID: <4E3167F3.9060404@tty.nl> (please keep the discussion on the mailinglist, so others can benefit from it too) I'd say you're trying to use the wrong tool for the job. If its possible at all, it'll require some hefty vcl though, or inline C, or a vmod... what you want to do is: - abuse varnish backend polling, so you can pick the next healthy backend, and use it in a redirect - rewrite the request to use the hostname from varnish backend configuration and send a 301 what you dont want to do is: - use varnish as its meant to be: an http accelerating caching proxy - use caching - use proxying you can btw use errors to force a reply in varnish: sub vcl_recv { if (req.http.host ~ "^img\.test\.com$") { error 301; } } sub vcl_error { if (obj.status == 301) { # now, I dont know how you could get the current healthy backends and use the host to set: set obj.http.host = 'img2.test.com'; return (deliver); } } maybe someone else knows how to get the next backend, if possible at all... Enno Carsten Dietz wrote: > Hi Enno, > > yes I do copy past. > My problem is that I don?t have an idea how much I can do in varnish with > varnish code. > With lookup the varnish-server seems to download the requestet file and send > it to the "Requester". > But I want to give the "Requester" only a 301 / 302 with a new url, because > of traffic Limitations. > I can can handle this be an external script ok but this isn't the fastest > way. > > > > -----Urspr?ngliche Nachricht----- > Von: varnish-misc-bounces at varnish-cache.org > [mailto:varnish-misc-bounces at varnish-cache.org] Im Auftrag von Enno van > Amerongen > Gesendet: Donnerstag, 28. Juli 2011 14:27 > An: varnish-misc at varnish-cache.org > Betreff: Re: Loadbalancing with 301 / 302 > > Hi, > > First of all, if you get an error, tell us what it is. > > Second, you seem to have copypasted the healthchecks directly from the > docs, not sure if it works or if its a typo in the docs, but I'd say: > .timeout = 1s (without the space) > > Third, your vcl_recv is wrong, try: > > sub vcl_recv { > set req.backend = baz; > return (lookup); > } > > good luck! > > Carsten Dietz wrote: > >> Hi, >> >> >> >> I want to setup varnish as an loadbalancer for 4-5 Downloadserver. >> >> The best way to loadbalance with Failover is Roundrobin. >> >> >> >> >> >> backend img4 { >> >> .host = "img4.test.com"; >> >> .probe = { >> >> .url = "/"; >> >> .interval = 5s; >> >> .timeout = 1 s; >> >> .window = 5; >> >> .threshold = 3; >> >> } >> >> } >> >> >> >> backend img5 { >> >> .host = "img5.test.com "; >> >> .probe = { >> >> .url = "/"; >> >> .interval = 5s; >> >> .timeout = 1 s; >> >> .window = 5; >> >> .threshold = 3; >> >> } >> >> } >> >> >> >> >> >> backend img6 { >> >> .host = "img6.test.com "; >> >> .probe = { >> >> .url = "/"; >> >> .interval = 5s; >> >> .timeout = 1 s; >> >> .window = 5; >> >> .threshold = 3; >> >> } >> >> } >> >> >> >> backend img7 { >> >> .host = "img7.test.com "; >> >> .probe = { >> >> .url = "/"; >> >> .interval = 5s; >> >> .timeout = 1 s; >> >> .window = 5; >> >> .threshold = 3; >> >> } >> >> } >> >> >> >> director baz round-robin { >> >> { >> >> .backend = img4; >> >> } >> >> { >> >> .backend = img5; >> >> } >> >> { >> >> .backend = img6; >> >> } >> >> { >> >> .backend = img7; >> >> } >> >> >> >> } >> >> >> >> sub vcl_recv { >> >> set obj.http.Location = baz; >> >> set obj.status = 301; >> >> return(deliver); >> >> } >> >> >> >> I try to set other variables but it try to start with this gives an error. >> >> >> >> >> Has anybody an idea how it can works? >> >> >> >> Thanks Carsten >> >> ------------------------------------------------------------------------ >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> > > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > > From jhalfmoon at milksnot.com Thu Jul 28 15:37:38 2011 From: jhalfmoon at milksnot.com (Johnny Halfmoon) Date: Thu, 28 Jul 2011 17:37:38 +0200 Subject: code 206 - visible as '206' in varnishncsa and as '200' from VCL code In-Reply-To: <539E75AB-9662-4CCA-98AB-B2A14607640D@matthewwilkes.co.uk> References: <20110727101905.175339upeii4pj0p@208.117.4.128> <539E75AB-9662-4CCA-98AB-B2A14607640D@matthewwilkes.co.uk> Message-ID: <4E318242.9030301@milksnot.com> Hi there, as a long time user of Varnish I still have an annoying little issue on my todo list: It seems that when returning partial content, Varnishnsca properly logs the transfer with a code 206 'partial content', but logging the exact same transfer from VCL code, it returns code '200', which is incorrect. I have tried using VRT_r_obj_status, VRT_r_rest_status and VRT_r_beresp status to no avail. This is the case with both Varnish 2.1.5 and 3.0.0. Is this done on purpose by the code or can it be registered as a bug? This is how I test it (examples using Varnish 3.0.0): #=== The command to do the RANGE request (note the returncode 206 'partial content) === testproxy02 ~ # http_proxy=http://127.0.0.1:80 curl -v -r 500-1000 http://vi.nl . . > Proxy-Connection: Keep-Alive > < HTTP/1.1 206 Partial Content < Server: Apache . ...... #=== This is the output of varnishncsa (note the code '206')=== testproxy02 ~ # varnishncsa 127.0.0.1 - - [28/Jul/2011:17:17:23 +0200] "GET http://bla.nlhttp://bla.nl HTTP/1.1" 206 501 "-" "curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5" #=== This is the output I get in syslog (note the code '200') === testproxy02 ~ # tail -f /var/log/varnish/current code:200 127.0.0.1 - - "Thu, 28 Jul 2011 15:16:56 GMT" "GET http://bla.nl" 200 58944 "-" "curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5" X:616508938 B:bla_nl V:bla.nl h:1 #=== This is the bit of inline C code used to do the loggin from VCL === int mycode = VRT_r_resp_status(sp); if (mycode == NULL) { mycode = 0; } syslog(LOG_DEBUG, "code:%u ", mycode ); . ...... #=== EOF === Cheers, Johnny From roberto.fernandezcrisial at gmail.com Thu Jul 28 20:10:20 2011 From: roberto.fernandezcrisial at gmail.com (=?ISO-8859-1?Q?Roberto_O=2E_Fern=E1ndez_Crisial?=) Date: Thu, 28 Jul 2011 17:10:20 -0300 Subject: Am I going crazy? Message-ID: Hi guys, I think I'm going crazy. I'm trying to cache some object but I'm getting beresp.ttl as negative value. In vcl_fetch() I have: unset beresp.http.Cache-Control; unset beresp.http.Expires; But beresp.ttl continues being "-1.000". How is it possible? I'm using: varnishd (varnish-3.0.0 revision 3bd5997) Copyright (c) 2006 Verdens Gang AS Copyright (c) 2006-2011 Varnish Software AS Thank you, Roberto @rofc -------------- next part -------------- An HTML attachment was scrubbed... URL: From l at lrowe.co.uk Fri Jul 29 13:23:50 2011 From: l at lrowe.co.uk (Laurence Rowe) Date: Fri, 29 Jul 2011 14:23:50 +0100 Subject: code 206 - visible as '206' in varnishncsa and as '200' from VCL code In-Reply-To: <4E318242.9030301@milksnot.com> References: <20110727101905.175339upeii4pj0p@208.117.4.128> <539E75AB-9662-4CCA-98AB-B2A14607640D@matthewwilkes.co.uk> <4E318242.9030301@milksnot.com> Message-ID: On 28 July 2011 16:37, Johnny Halfmoon wrote: > Hi there, > > as a long time user of Varnish I still have an annoying little issue on my > todo list: It seems that when returning partial content, Varnishnsca > properly logs the transfer with a code 206 'partial content', but logging > the exact same transfer from VCL code, it returns code '200', which is > incorrect. I have tried using VRT_r_obj_status, VRT_r_rest_status and > VRT_r_beresp status to no avail. This is the case with both Varnish 2.1.5 > and 3.0.0. Is this done on purpose by the code or can it be registered as a > bug? > > This is how I test it (examples using Varnish 3.0.0): > > #=== The command to do the RANGE request (note the returncode 206 'partial > content) === > > testproxy02 ~ # http_proxy=http://127.0.0.1:80 curl -v -r 500-1000 > http://vi.nl > . > . >> >> Proxy-Connection: Keep-Alive >> > < HTTP/1.1 206 Partial Content > < Server: Apache > . > ...... > > #=== This is the output of varnishncsa (note the code '206')=== > > testproxy02 ~ # varnishncsa > 127.0.0.1 - - [28/Jul/2011:17:17:23 +0200] "GET http://bla.nlhttp://bla.nl > HTTP/1.1" 206 501 "-" "curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 > OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5" > > #=== This is the output I get in syslog (note the code '200') === > > testproxy02 ~ # tail -f /var/log/varnish/current > ?code:200 > ?127.0.0.1 - - "Thu, 28 Jul 2011 15:16:56 GMT" "GET http://bla.nl" 200 58944 > "-" "curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b > zlib/1.2.3 libidn/0.6.5" X:616508938 B:bla_nl V:bla.nl h:1 > > #=== This is the bit of inline C code used to do the loggin from VCL === > > int mycode = VRT_r_resp_status(sp); > if (mycode == NULL) { mycode = 0; } > syslog(LOG_DEBUG, "code:%u ", mycode ); > . > ...... > > #=== EOF === I believe Varnish makes the request to the backend without the Range header, so the result may still be cached. I'm not familiar enough with the VRT commands to know exactly what you are logging, but if it is the backend response code then you would expect that to be 200 (as it would not have seen the Range header.) Laurence From jhalfmoon at milksnot.com Fri Jul 29 13:56:29 2011 From: jhalfmoon at milksnot.com (Johnny Halfmoon) Date: Fri, 29 Jul 2011 15:56:29 +0200 Subject: code 206 - visible as '206' in varnishncsa and as '200' from VCL code In-Reply-To: References: <20110727101905.175339upeii4pj0p@208.117.4.128> <539E75AB-9662-4CCA-98AB-B2A14607640D@matthewwilkes.co.uk> <4E318242.9030301@milksnot.com> Message-ID: <20110729155629.h0y0e475s0c884sk@webmail.milksnot.com> ----- Message from l at lrowe.co.uk --------- Date: Fri, 29 Jul 2011 14:23:50 +0100 From: Laurence Rowe Reply-To: Laurence Rowe Subject: Re: code 206 - visible as '206' in varnishncsa and as '200' from VCL code To: Johnny Halfmoon Cc: "varnish-misc at varnish-cache.org" > On 28 July 2011 16:37, Johnny Halfmoon wrote: >> Hi there, >> >> as a long time user of Varnish I still have an annoying little issue on my >> todo list: It seems that when returning partial content, Varnishnsca >> properly logs the transfer with a code 206 'partial content', but logging >> the exact same transfer from VCL code, it returns code '200', which is >> incorrect. I have tried using VRT_r_obj_status, VRT_r_rest_status and >> VRT_r_beresp status to no avail. This is the case with both Varnish 2.1.5 >> and 3.0.0. Is this done on purpose by the code or can it be registered as a >> bug? >> >> This is how I test it (examples using Varnish 3.0.0): >> >> #=== The command to do the RANGE request (note the returncode 206 'partial >> content) === >> >> testproxy02 ~ # http_proxy=http://127.0.0.1:80 curl -v -r 500-1000 >> http://vi.nl >> . >> . >>> >>> Proxy-Connection: Keep-Alive >>> >> < HTTP/1.1 206 Partial Content >> < Server: Apache >> . >> ...... >> >> #=== This is the output of varnishncsa (note the code '206')=== >> >> testproxy02 ~ # varnishncsa >> 127.0.0.1 - - [28/Jul/2011:17:17:23 +0200] "GET http://bla.nlhttp://bla.nl >> HTTP/1.1" 206 501 "-" "curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 >> OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5" >> >> #=== This is the output I get in syslog (note the code '200') === >> >> testproxy02 ~ # tail -f /var/log/varnish/current >> ?code:200 >> ?127.0.0.1 - - "Thu, 28 Jul 2011 15:16:56 GMT" "GET http://bla.nl" 200 58944 >> "-" "curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b >> zlib/1.2.3 libidn/0.6.5" X:616508938 B:bla_nl V:bla.nl h:1 >> >> #=== This is the bit of inline C code used to do the loggin from VCL === >> >> int mycode = VRT_r_resp_status(sp); >> if (mycode == NULL) { mycode = 0; } >> syslog(LOG_DEBUG, "code:%u ", mycode ); >> . >> ...... >> >> #=== EOF === > > I believe Varnish makes the request to the backend without the Range > header, so the result may still be cached. I'm not familiar enough > with the VRT commands to know exactly what you are logging, but if it > is the backend response code then you would expect that to be 200 (as > it would not have seen the Range header.) > > Laurence > Hi Laurence, thank you for your reply. I agree that the backend response would be 200. But what I need to see is the actual response code Varnish sends to the frontend, which should be 206. In the example code I therefor explicitly read out VRT_r_resp_status , but still see code 200. Cheers, Johnny From andrzej at godziuk.pl Fri Jul 22 12:05:41 2011 From: andrzej at godziuk.pl (Andrzej Godziuk) Date: Fri, 22 Jul 2011 14:05:41 +0200 Subject: Configuration Questions & Assistance In-Reply-To: <20110721214619.GD2395@nat.myhome> References: <20110721132707.144173gb7r4968wr@208.117.4.128> <20110721214619.GD2395@nat.myhome> Message-ID: <201107221405.41672.andrzej@godziuk.pl> On Thursday, July 21, 2011 11:46:19 PM Paul A. Procacci wrote: > > One of those such issues, is that it seems that when Varnish is > > enabled and I visit a page that is PHP based, but the PHP code starts > > with > displayed on the browser. > > > This sounds much like the php problem/example. If your pages are being > returned from the web server as plain text, then this would be a symptom. > > My suggestion is to double check your varnish config and make sure it > points to a valid virtualhost. Double check your apache config to ensure > you have a virtualhost listening on the ip address you told varnish to get > its data. That's exactly what it sounds like. One more thing to check: maybe your Varnish has cached the unparsed response because Apache has been misconfigured for some time, and now it's still in the cache because of long TTL? Try cleaning your cache and like Paul mentioned, look into Apache configuration rather than Varnish. -- Andrzej Godziuk http://massivescale.net/