From japrice at gmail.com Tue Dec 1 00:35:08 2015 From: japrice at gmail.com (Jason Price) Date: Mon, 30 Nov 2015 19:35:08 -0500 Subject: Varnish and mod_deflate support In-Reply-To: <565C7C60.1020502@gmail.com> References: <565C7C60.1020502@gmail.com> Message-ID: First off, 3.0.7 is EOL. Move to 4.x as soon as you can Second, I'd leave the entire stanza out of VCL. The varnish defaults handle compression without issue in most cases. If the backend serves compressed data, it'll be cached as compressed, and served as compressed. Third, once you 'pipe' a request, varnish does nothing other than packet forwarding. It can't cache, it can't balance requests across backends, nothing. If the above doesn't resolve your problem, try to capture a request Client and Backend side transaction in varnishlog. That will help diagnose what the real problem is. -Jason On Mon, Nov 30, 2015 at 11:42 AM, georgi.int at gmail.com wrote: > Hello, > I have been using varnish 3.7 only as a proxy server for apache and have a > following lines in default.vcl which should handle the encodings: > > if (req.http.Accept-Encoding) { > if (req.http.Accept-Encoding ~ "gzip") { > # If the browser supports it, we'll use gzip. > #set req.http.Accept-Encoding = "gzip"; > unset req.http.Accept-Encoding; > } > else if (req.http.Accept-Encoding ~ "deflate") { > # Next, try deflate if it is supported. > set req.http.Accept-Encoding = "deflate"; > } > else { > # Unknown algorithm. Remove it and send unencoded. > unset req.http.Accept-Encoding; > } > } > > Although, customers which have mod_deflate rules in .htaccess file > experience the problem that their sites are not compressed. If I pipe the > site to apache site is compressed. SO, my question is what is the problem > with the deflate and my varnish configuration? Is it required to add > something other to varnish to work the deflate? I tried a couple of things > which I found in the net, but nothing worked. > > Thank you in advance for your answers! > > Best regards, > Georgi > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From georgi.int at gmail.com Tue Dec 1 07:47:08 2015 From: georgi.int at gmail.com (georgi.int at gmail.com) Date: Tue, 1 Dec 2015 09:47:08 +0200 Subject: Varnish and mod_deflate support In-Reply-To: References: <565C7C60.1020502@gmail.com> Message-ID: <565D507C.3020601@gmail.com> Thank you about your reply. 1. I understand, but the process for migrating is not so fast, because of compile all from source including all vmods, building new rpms and distributing them on many servers. Also, I am not sure that this is the only one solution :) 2. The pipe was only for the test to see if only on the backends will work. Now it's not piped, but the cache is stopped (it's all the time stopped, because I use it only as a firewall for apache). 3. I wouild like to ask you to see this peace of log, because I can't find anything anoying in it, but I am completely new to varnish so it's possible I miss something. I see that here it's accepting encoding gzip: 32 TxHeader b Host: mysite.com 32 TxHeader b User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:42.0) Gecko/20100101 Firefox/42.0 32 TxHeader b Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 32 TxHeader b Accept-Language: en-US,en;q=0.5 32 TxHeader b X-Country-Code: BG 32 TxHeader b X-Forwarded-For: IP, IP 32 TxHeader b X-Varnish: 1218787819 32 TxHeader b Accept-Encoding: gzip 32 RxProtocol b HTTP/1.1 32 RxStatus b 404 32 RxResponse b Not Found 32 RxHeader b Date: Tue, 01 Dec 2015 07:40:09 GMT 32 RxHeader b Server: Apache/2.2.27 (Unix) /5.0 mod_ssl/2.2.27 OpenSSL/1.0.1e-fips mod_bwlimited/1.4 mod_fastcgi/2.4.6 32 RxHeader b Vary: Accept-Encoding 32 RxHeader b Content-Encoding: gzip 32 RxHeader b Content-Length: 248 32 RxHeader b Connection: close 32 RxHeader b Content-Type: text/html; charset=iso-8859-1 32 Fetch_Body b 4(length) cls 0 mklen 1 32 Length b 248 32 BackendClose b default 30 SessionOpen c IP 59645 IP:80 30 ReqStart c IP 59645 1218787819 30 RxRequest c GET 30 RxURL c /favicon.ico 30 RxProtocol c HTTP/1.1 I read that it's possible to turn off completely mod_deflate from apache and use varnish built in compression, but at this time this is not solution for our company so I am trying to find another. Best regards, Georgi On 12/01/2015 02:35 AM, Jason Price wrote: > First off, 3.0.7 is EOL. Move to 4.x as soon as you can > > Second, I'd leave the entire stanza out of VCL. The varnish defaults > handle compression without issue in most cases. If the backend serves > compressed data, it'll be cached as compressed, and served as compressed. > > Third, once you 'pipe' a request, varnish does nothing other than > packet forwarding. It can't cache, it can't balance requests across > backends, nothing. > > If the above doesn't resolve your problem, try to capture a request > Client and Backend side transaction in varnishlog. That will help > diagnose what the real problem is. > > -Jason > > On Mon, Nov 30, 2015 at 11:42 AM, georgi.int at gmail.com > > wrote: > > Hello, > I have been using varnish 3.7 only as a proxy server for apache > and have a following lines in default.vcl which should handle the > encodings: > > if (req.http.Accept-Encoding) { > if (req.http.Accept-Encoding ~ "gzip") { > # If the browser supports it, we'll use gzip. > #set req.http.Accept-Encoding = "gzip"; > unset req.http.Accept-Encoding; > } > else if (req.http.Accept-Encoding ~ "deflate") { > # Next, try deflate if it is supported. > set req.http.Accept-Encoding = "deflate"; > } > else { > # Unknown algorithm. Remove it and send unencoded. > unset req.http.Accept-Encoding; > } > } > > Although, customers which have mod_deflate rules in .htaccess file > experience the problem that their sites are not compressed. If I > pipe the site to apache site is compressed. SO, my question is > what is the problem with the deflate and my varnish configuration? > Is it required to add something other to varnish to work the > deflate? I tried a couple of things which I found in the net, but > nothing worked. > > Thank you in advance for your answers! > > Best regards, > Georgi > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From japrice at gmail.com Wed Dec 2 01:58:31 2015 From: japrice at gmail.com (Jason Price) Date: Tue, 1 Dec 2015 20:58:31 -0500 Subject: Varnish and mod_deflate support In-Reply-To: <565D507C.3020601@gmail.com> References: <565C7C60.1020502@gmail.com> <565D507C.3020601@gmail.com> Message-ID: Well, you'll need to figure out why the backend served a 404 request, despite a valid looking Accept-Encoding header. The above gives you all the details about the request, so you can turn that into a curl request directly to the backend in question. That will give you a triage tool. When you figure out why the curl request fails, that will give you a hint as to what you need to change in your VCL (or apache) config. Could it be as simple as mod_deflate not accepting gzip, but preferring the previous compression? (I know nothing of mod_deflate) -Jason On Tue, Dec 1, 2015 at 2:47 AM, georgi.int at gmail.com wrote: > Thank you about your reply. > > 1. I understand, but the process for migrating is not so fast, because of > compile all from source including all vmods, building new rpms and > distributing them on many servers. Also, I am not sure that this is the > only one solution :) > > 2. The pipe was only for the test to see if only on the backends will > work. Now it's not piped, but the cache is stopped (it's all the time > stopped, because I use it only as a firewall for apache). > > 3. I wouild like to ask you to see this peace of log, because I can't find > anything anoying in it, but I am completely new to varnish so it's possible > I miss something. I see that here it's accepting encoding gzip: > > 32 TxHeader b Host: mysite.com > 32 TxHeader b User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux i686; > rv:42.0) Gecko/20100101 Firefox/42.0 > 32 TxHeader b Accept: > text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 > 32 TxHeader b Accept-Language: en-US,en;q=0.5 > 32 TxHeader b X-Country-Code: BG > 32 TxHeader b X-Forwarded-For: IP, IP > 32 TxHeader b X-Varnish: 1218787819 > 32 TxHeader b Accept-Encoding: gzip > 32 RxProtocol b HTTP/1.1 > 32 RxStatus b 404 > 32 RxResponse b Not Found > 32 RxHeader b Date: Tue, 01 Dec 2015 07:40:09 GMT > 32 RxHeader b Server: Apache/2.2.27 (Unix) /5.0 mod_ssl/2.2.27 > OpenSSL/1.0.1e-fips mod_bwlimited/1.4 mod_fastcgi/2.4.6 > 32 RxHeader b Vary: Accept-Encoding > 32 RxHeader b Content-Encoding: gzip > 32 RxHeader b Content-Length: 248 > 32 RxHeader b Connection: close > 32 RxHeader b Content-Type: text/html; charset=iso-8859-1 > 32 Fetch_Body b 4(length) cls 0 mklen 1 > 32 Length b 248 > 32 BackendClose b default > 30 SessionOpen c IP 59645 IP:80 > 30 ReqStart c IP 59645 1218787819 > 30 RxRequest c GET > 30 RxURL c /favicon.ico > 30 RxProtocol c HTTP/1.1 > > > I read that it's possible to turn off completely mod_deflate from apache > and use varnish built in compression, but at this time this is not solution > for our company so I am trying to find another. > > Best regards, > Georgi > > > On 12/01/2015 02:35 AM, Jason Price wrote: > > First off, 3.0.7 is EOL. Move to 4.x as soon as you can > > Second, I'd leave the entire stanza out of VCL. The varnish defaults > handle compression without issue in most cases. If the backend serves > compressed data, it'll be cached as compressed, and served as compressed. > > Third, once you 'pipe' a request, varnish does nothing other than packet > forwarding. It can't cache, it can't balance requests across backends, > nothing. > > If the above doesn't resolve your problem, try to capture a request Client > and Backend side transaction in varnishlog. That will help diagnose what > the real problem is. > > -Jason > > On Mon, Nov 30, 2015 at 11:42 AM, > georgi.int at gmail.com wrote: > >> Hello, >> I have been using varnish 3.7 only as a proxy server for apache and have >> a following lines in default.vcl which should handle the encodings: >> >> if (req.http.Accept-Encoding) { >> if (req.http.Accept-Encoding ~ "gzip") { >> # If the browser supports it, we'll use gzip. >> #set req.http.Accept-Encoding = "gzip"; >> unset req.http.Accept-Encoding; >> } >> else if (req.http.Accept-Encoding ~ "deflate") { >> # Next, try deflate if it is supported. >> set req.http.Accept-Encoding = "deflate"; >> } >> else { >> # Unknown algorithm. Remove it and send unencoded. >> unset req.http.Accept-Encoding; >> } >> } >> >> Although, customers which have mod_deflate rules in .htaccess file >> experience the problem that their sites are not compressed. If I pipe the >> site to apache site is compressed. SO, my question is what is the problem >> with the deflate and my varnish configuration? Is it required to add >> something other to varnish to work the deflate? I tried a couple of things >> which I found in the net, but nothing worked. >> >> Thank you in advance for your answers! >> >> Best regards, >> Georgi >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From georgi.int at gmail.com Wed Dec 2 10:44:31 2015 From: georgi.int at gmail.com (georgi.int at gmail.com) Date: Wed, 2 Dec 2015 12:44:31 +0200 Subject: Varnish and mod_deflate support In-Reply-To: References: <565C7C60.1020502@gmail.com> <565D507C.3020601@gmail.com> Message-ID: <565ECB8F.1000200@gmail.com> I figured out the problem and one stupid solution, but I am not sure if I break something this way so I would like yo confirm that to me :) In the following peace of code when I comment the unset everything works fine: if (req.http.Accept-Encoding) { if (req.http.Accept-Encoding ~ "gzip") { # If the browser supports it, we'll use gzip. #set req.http.Accept-Encoding = "gzip"; # unset req.http.Accept-Encoding; } else if (req.http.Accept-Encoding ~ "deflate") { # Next, try deflate if it is supported. set req.http.Accept-Encoding = "deflate"; } else { # Unknown algorithm. Remove it and send unencoded. unset req.http.Accept-Encoding; } } Why and am I break something this way? Can I remove it ? On 12/02/2015 03:58 AM, Jason Price wrote: > Well, you'll need to figure out why the backend served a 404 request, > despite a valid looking Accept-Encoding header. > > The above gives you all the details about the request, so you can turn > that into a curl request directly to the backend in question. That > will give you a triage tool. When you figure out why the curl request > fails, that will give you a hint as to what you need to change in your > VCL (or apache) config. > > Could it be as simple as mod_deflate not accepting gzip, but > preferring the previous compression? (I know nothing of mod_deflate) > > -Jason > > On Tue, Dec 1, 2015 at 2:47 AM, georgi.int at gmail.com > > wrote: > > Thank you about your reply. > > 1. I understand, but the process for migrating is not so fast, > because of compile all from source including all vmods, building > new rpms and distributing them on many servers. Also, I am not > sure that this is the only one solution :) > > 2. The pipe was only for the test to see if only on the backends > will work. Now it's not piped, but the cache is stopped (it's all > the time stopped, because I use it only as a firewall for apache). > > 3. I wouild like to ask you to see this peace of log, because I > can't find anything anoying in it, but I am completely new to > varnish so it's possible I miss something. I see that here it's > accepting encoding gzip: > > 32 TxHeader b Host: mysite.com > 32 TxHeader b User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux > i686; rv:42.0) Gecko/20100101 Firefox/42.0 > 32 TxHeader b Accept: > text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 > 32 TxHeader b Accept-Language: en-US,en;q=0.5 > 32 TxHeader b X-Country-Code: BG > 32 TxHeader b X-Forwarded-For: IP, IP > 32 TxHeader b X-Varnish: 1218787819 > 32 TxHeader b Accept-Encoding: gzip > 32 RxProtocol b HTTP/1.1 > 32 RxStatus b 404 > 32 RxResponse b Not Found > 32 RxHeader b Date: Tue, 01 Dec 2015 07:40:09 GMT > 32 RxHeader b Server: Apache/2.2.27 (Unix) /5.0 > mod_ssl/2.2.27 OpenSSL/1.0.1e-fips mod_bwlimited/1.4 mod_fastcgi/2.4.6 > 32 RxHeader b Vary: Accept-Encoding > 32 RxHeader b Content-Encoding: gzip > 32 RxHeader b Content-Length: 248 > 32 RxHeader b Connection: close > 32 RxHeader b Content-Type: text/html; charset=iso-8859-1 > 32 Fetch_Body b 4(length) cls 0 mklen 1 > 32 Length b 248 > 32 BackendClose b default > 30 SessionOpen c IP 59645 IP:80 > 30 ReqStart c IP 59645 1218787819 > 30 RxRequest c GET > 30 RxURL c /favicon.ico > 30 RxProtocol c HTTP/1.1 > > > I read that it's possible to turn off completely mod_deflate from > apache and use varnish built in compression, but at this time this > is not solution for our company so I am trying to find another. > > Best regards, > Georgi > > > On 12/01/2015 02:35 AM, Jason Price wrote: >> First off, 3.0.7 is EOL. Move to 4.x as soon as you can >> >> Second, I'd leave the entire stanza out of VCL. The varnish >> defaults handle compression without issue in most cases. If the >> backend serves compressed data, it'll be cached as compressed, >> and served as compressed. >> >> Third, once you 'pipe' a request, varnish does nothing other than >> packet forwarding. It can't cache, it can't balance requests >> across backends, nothing. >> >> If the above doesn't resolve your problem, try to capture a >> request Client and Backend side transaction in varnishlog. That >> will help diagnose what the real problem is. >> >> -Jason >> >> On Mon, Nov 30, 2015 at 11:42 AM, georgi.int at gmail.com >> > > wrote: >> >> Hello, >> I have been using varnish 3.7 only as a proxy server for >> apache and have a following lines in default.vcl which should >> handle the encodings: >> >> if (req.http.Accept-Encoding) { >> if (req.http.Accept-Encoding ~ "gzip") { >> # If the browser supports it, we'll use gzip. >> #set req.http.Accept-Encoding = "gzip"; >> unset req.http.Accept-Encoding; >> } >> else if (req.http.Accept-Encoding ~ "deflate") { >> # Next, try deflate if it is supported. >> set req.http.Accept-Encoding = "deflate"; >> } >> else { >> # Unknown algorithm. Remove it and send unencoded. >> unset req.http.Accept-Encoding; >> } >> } >> >> Although, customers which have mod_deflate rules in .htaccess >> file experience the problem that their sites are not >> compressed. If I pipe the site to apache site is compressed. >> SO, my question is what is the problem with the deflate and >> my varnish configuration? Is it required to add something >> other to varnish to work the deflate? I tried a couple of >> things which I found in the net, but nothing worked. >> >> Thank you in advance for your answers! >> >> Best regards, >> Georgi >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lkarsten at varnish-software.com Wed Dec 2 14:40:42 2015 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Wed, 2 Dec 2015 15:40:42 +0100 Subject: Update on vmods and packaging Message-ID: <20151202144041.GB19950@immer.varnish-software.com> Hi all. Varnish Software (VS) is writing and maintaining a fair amount of vmods for Varnish Cache. This includes major ones like vmod-header, which can be said to be borderline core Varnish functionality. We will be doing some changes on how these are maintained and distributed. Currently the normal way of using them (in the community) is to do a git checkout from Github, build it from source and run it. This is all very bleeding edge, and leaves something to be desired. We (VS) will start doing versioned tarball releases of the vmods. They will get changelogs, and the tarballs will be made available for mirroring just as Varnish Cache code is. This will allow distribution packagers to add the vmod packages on their platforms. I'm working with the Fedora/EPEL and Debian packagers to get it in place there. Additionally we will try to write down what the community can expect from VS in this regard, to help everyone make informed decisions on what to run. (on how long) The tarball releases will be adopted to the current latest Varnish Cache release. The development version will continue to live on Github, but will be moved to master branch in each one. Code for 4.0 and 3.0 will be deprecated. Implementation of this plan will be gradual over the next months. Further down the line it is likely that we will be combining the simple vmods (with no third party/library dependencies) into a single collection source package. Updates on that will come later. I'm happy to answer any questions on this, either on-list or privately. -- Lasse Karstensen Varnish Software (with VS hat on.) From idl0r at qasl.de Thu Dec 3 12:12:53 2015 From: idl0r at qasl.de (Christian Ruppert) Date: Thu, 03 Dec 2015 13:12:53 +0100 Subject: regsub(all?) input limit Message-ID: Hi, I just noticed that regsub and likely regsuball will just return the untouched input string if it exceeds 57118 bytes. Is it intended to do so, performance reasons perhaps? In this case it's a custom error page with some (eye candy) embedded images and in total a bit above the mentioned size. The regsub should actually just replace a pattern with the XID, in this case simply regsub(..., "bereq\.xid", bereq.xid). I've also a test case: VCL: import std; regsub(std.fileread("/etc/varnish/error.test"), "bereq\.xid", server.hostname + "-" + bereq.xid) File: cat <error.test XID: bereq.xid A e.g. custom error file that exceed a max. of 57118 bytes will be completely skipped by regsub() someembeddedimage:$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 56987 | head -n 1) EOF That can be reproduced with at least 4.0 and 4.1. -- Regards, Christian Ruppert From carlos.abalde at gmail.com Thu Dec 3 12:31:40 2015 From: carlos.abalde at gmail.com (Carlos Abalde) Date: Thu, 3 Dec 2015 13:31:40 +0100 Subject: regsub(all?) input limit In-Reply-To: References: Message-ID: <448DCB3B-CE5A-4338-89E9-8FD8778DDA6D@gmail.com> > On 03 Dec 2015, at 13:12, Christian Ruppert wrote: > > Hi, > > I just noticed that regsub and likely regsuball will just return the untouched input string if it exceeds 57118 bytes. Is it intended to do so, performance reasons perhaps? > In this case it's a custom error page with some (eye candy) embedded images and in total a bit above the mentioned size. The regsub should actually just replace a pattern with the XID, in this case simply regsub(..., "bereq\.xid", bereq.xid). Hi Christian, You're probably hitting the workspace_client (or workspace_backend) limit. Those are varnishd parameters [1] that default to 64k. I've not checked the regsub implementation, but I guess it is not able to allocate enough memory in the workspace and in that case it simply returns the original input string (the alternative would be returning NULL or killing the child process in an assertion). Try to increase those parameters and try your test again. [1] https://www.varnish-cache.org/docs/trunk/reference/varnishd.html#list-of-parameters Best, -- Carlos Abalde From idl0r at qasl.de Thu Dec 3 13:05:44 2015 From: idl0r at qasl.de (Christian Ruppert) Date: Thu, 03 Dec 2015 14:05:44 +0100 Subject: regsub(all?) input limit In-Reply-To: <448DCB3B-CE5A-4338-89E9-8FD8778DDA6D@gmail.com> References: <448DCB3B-CE5A-4338-89E9-8FD8778DDA6D@gmail.com> Message-ID: <3384d10123b66e883aac42058b4746ad@qasl.de> On 2015-12-03 13:31, Carlos Abalde wrote: >> On 03 Dec 2015, at 13:12, Christian Ruppert wrote: >> >> Hi, >> >> I just noticed that regsub and likely regsuball will just return the >> untouched input string if it exceeds 57118 bytes. Is it intended to do >> so, performance reasons perhaps? >> In this case it's a custom error page with some (eye candy) embedded >> images and in total a bit above the mentioned size. The regsub should >> actually just replace a pattern with the XID, in this case simply >> regsub(..., "bereq\.xid", bereq.xid). > > Hi Christian, > > You're probably hitting the workspace_client (or workspace_backend) > limit. Those are varnishd parameters [1] that default to 64k. I've not > checked the regsub implementation, but I guess it is not able to > allocate enough memory in the workspace and in that case it simply > returns the original input string (the alternative would be returning > NULL or killing the child process in an assertion). Try to increase > those parameters and try your test again. > > [1] > https://www.varnish-cache.org/docs/trunk/reference/varnishd.html#list-of-parameters > > Best, > > -- > Carlos Abalde Hi Carlos, thanks for your fast reply! :) You're right! Setting workspace_backend to at least 429k helps in this case. 64k vs. >429k is quite a lot tough. So that will be allocated for each backend req/resp? Is take from the space set by "-s" or additionally? Is there any better way to set the XID, instead of using smaller images? I'm not sure about the drawbacks when setting workspace_backend to e.g. 512k. -- Regards, Christian Ruppert From carlos.abalde at gmail.com Thu Dec 3 13:59:17 2015 From: carlos.abalde at gmail.com (Carlos Abalde) Date: Thu, 3 Dec 2015 14:59:17 +0100 Subject: regsub(all?) input limit In-Reply-To: <3384d10123b66e883aac42058b4746ad@qasl.de> References: <448DCB3B-CE5A-4338-89E9-8FD8778DDA6D@gmail.com> <3384d10123b66e883aac42058b4746ad@qasl.de> Message-ID: <380D2AF5-9B9D-4AF6-945D-633131EC77AE@gmail.com> > On 03 Dec 2015, at 14:05, Christian Ruppert wrote: > > thanks for your fast reply! :) > You're right! Setting workspace_backend to at least 429k helps in this case. 64k vs. >429k is quite a lot tough. So that will be allocated for each backend req/resp? Is take from the space set by "-s" or additionally? Is there any better way to set the XID, instead of using smaller images? I'm not sure about the drawbacks when setting workspace_backend to e.g. 512k. Hi, Workspaces are local to each Varnish thread. It's a common pattern used to reduce thread contention when they need to allocate memory. The amount of memory consumed by workspaces is additional to any storage defined with the -s parameter. Having a large workspace (e.g. 1MB) does not necessarily imply that each worker thread is going to consume 1MB. That's just virtual memory. If not really used by all worker threads, then that's not going to be a problem. Best, -- Carlos Abalde From idl0r at qasl.de Thu Dec 3 14:47:56 2015 From: idl0r at qasl.de (Christian Ruppert) Date: Thu, 03 Dec 2015 15:47:56 +0100 Subject: regsub(all?) input limit In-Reply-To: <380D2AF5-9B9D-4AF6-945D-633131EC77AE@gmail.com> References: <448DCB3B-CE5A-4338-89E9-8FD8778DDA6D@gmail.com> <3384d10123b66e883aac42058b4746ad@qasl.de> <380D2AF5-9B9D-4AF6-945D-633131EC77AE@gmail.com> Message-ID: <0d3ea2efd1982f6df4f4c1363b92adfd@qasl.de> On 2015-12-03 14:59, Carlos Abalde wrote: >> On 03 Dec 2015, at 14:05, Christian Ruppert wrote: >> >> thanks for your fast reply! :) >> You're right! Setting workspace_backend to at least 429k helps in this >> case. 64k vs. >429k is quite a lot tough. So that will be allocated >> for each backend req/resp? Is take from the space set by "-s" or >> additionally? Is there any better way to set the XID, instead of using >> smaller images? I'm not sure about the drawbacks when setting >> workspace_backend to e.g. 512k. > > Hi, > > Workspaces are local to each Varnish thread. It's a common pattern > used to reduce thread contention when they need to allocate memory. > The amount of memory consumed by workspaces is additional to any > storage defined with the -s parameter. Having a large workspace (e.g. > 1MB) does not necessarily imply that each worker thread is going to > consume 1MB. That's just virtual memory. If not really used by all > worker threads, then that's not going to be a problem. > > Best, > > -- > Carlos Abalde Thanks, Carlos! -- Regards, Christian Ruppert From Tom.Anheyer at Berlinonline.de Fri Dec 4 09:24:05 2015 From: Tom.Anheyer at Berlinonline.de (Tom Anheyer) Date: Fri, 4 Dec 2015 10:24:05 +0100 Subject: Client threads are dying after activating a new VCL Message-ID: <56615BB5.90100@Berlinonline.de> Hello, Varnish client threads are dying some time after activating a new VCL with segfault. This may the same issue like https://www.varnish-cache.org/trac/ticket/1802. Is there a workaround? A typical journal log looks like: Dec 04 08:52:57 varnish varnishd[13456]: CLI telnet 127.0.0.1 41273 127.0.0.1 6082 Rd auth 3f15e9186270a640b5f0958c35b1910e85cc7ab04e038021d983a67e6b0dfba0 Dec 04 08:52:57 varnish varnishd[13456]: CLI telnet 127.0.0.1 41273 127.0.0.1 6082 Wr 200 ----------------------------- Varnish Cache CLI 1.0 ----------------------------- Linux,3.16.7-29-desktop,x86_64,-junix,-smalloc,-smalloc,-hcritbit varnish-4.1.0 revision 3041728 Type 'help' for command list. Type 'quit' to close CLI session. Dec 04 08:52:57 varnish varnishd[13456]: CLI telnet 127.0.0.1 41273 127.0.0.1 6082 Rd ping Dec 04 08:52:57 varnish varnishd[13456]: CLI telnet 127.0.0.1 41273 127.0.0.1 6082 Wr 200 PONG 1449219177 1.0 Dec 04 08:52:57 varnish varnishd[13456]: CLI telnet 127.0.0.1 41273 127.0.0.1 6082 Rd vcl.load Test-2015-12-04-08:52:57 /etc/varnish/vcl.conf Dec 04 08:52:57 varnish varnishd[13456]: CLI telnet 127.0.0.1 41273 127.0.0.1 6082 Wr 200 VCL compiled. ? Dec 04 08:52:57 varnish varnishd[13456]: CLI telnet 127.0.0.1 41281 127.0.0.1 6082 Rd vcl.use Test-2015-12-04-08:52:57 Dec 04 08:52:57 varnish varnishd[13456]: CLI telnet 127.0.0.1 41281 127.0.0.1 6082 Wr 200 VCL 'Test-2015-12-04-08:52:57' now active ? Dec 04 08:52:57 varnish varnishd[13456]: CLI telnet 127.0.0.1 41285 127.0.0.1 6082 Rd vcl.list Dec 04 08:52:57 varnish varnishd[13456]: CLI telnet 127.0.0.1 41285 127.0.0.1 6082 Wr 200 available auto/warm 0 boot active auto/warm 0 Test-2015-12-04-08:52:57 ? Dec 04 09:02:59 varnish varnishd[13456]: Child (13466) not responding to CLI, killing it. Dec 04 09:02:59 varnish varnishd[13456]: Child (13466) not responding to CLI, killing it. Dec 04 09:02:59 varnish varnishd[13456]: Child (13466) died signal=11 Dec 04 09:02:59 varnish varnishd[13456]: Child (13466) Panic message: Assert error in child_sigsegv_handler(), mgt/mgt_child.c line 297: Condition(Segmentation fault by instruction at 0x5590) not true. thread = (cache-main) version = varnish-4.1.0 revision 3041728 ident = Linux,3.16.7-29-desktop,x86_64,-junix,-smalloc,-smalloc,-hcritbit,epoll Dec 04 09:02:59 varnish varnishd[13456]: Child (13466) died signal=11 Dec 04 09:02:59 varnish varnishd[13456]: Child (13466) Panic message: Dec 04 09:02:59 varnish varnishd[13456]: Assert error in child_sigsegv_handler(), mgt/mgt_child.c line 297: Dec 04 09:02:59 varnish varnishd[13456]: Condition(Segmentation fault by instruction at 0x5590) not true. Dec 04 09:02:59 varnish varnishd[13456]: thread = (cache-main) Dec 04 09:02:59 varnish varnishd[13456]: version = varnish-4.1.0 revision 3041728 Dec 04 09:02:59 varnish varnishd[13456]: ident = Linux,3.16.7-29-desktop,x86_64,-junix,-smalloc,-smalloc,-hcritbit,epoll Dec 04 09:02:59 varnish varnishd[13456]: Child cleanup complete Dec 04 09:02:59 varnish varnishd[13456]: Child cleanup complete Dec 04 09:02:59 varnish varnishd[13456]: child (13778) Started Dec 04 09:02:59 varnish varnishd[13456]: child (13778) Started Dec 04 09:03:00 varnish varnishd[13456]: Pushing vcls failed: Dec 04 09:03:00 varnish varnishd[13456]: Could not load compiled VCL. Dec 04 09:03:00 varnish varnishd[13456]: dlopen(vcl_boot/vgc.so) = vcl_boot/vgc.so: cannot open shared object file: No such file or directory Dec 04 09:03:00 varnish varnishd[13456]: Stopping Child Dec 04 09:03:00 varnish varnishd[13456]: Pushing vcls failed: Could not load compiled VCL. dlopen(vcl_boot/vgc.so) = vcl_boot/vgc.so: cannot open shared object file: No such file or directory Dec 04 09:03:00 varnish varnishd[13456]: Stopping Child Dec 04 09:03:01 varnish varnishd[13456]: Child (13778) ended Dec 04 09:03:01 varnish varnishd[13456]: Child (13778) ended Dec 04 09:03:01 varnish varnishd[13456]: Child (13778) said Child starts Dec 04 09:03:01 varnish varnishd[13456]: Child (13778) said Child dies Dec 04 09:03:01 varnish varnishd[13456]: Child (13778) said Child starts Dec 04 09:03:01 varnish varnishd[13456]: Child (13778) said Child dies Dec 04 09:03:01 varnish varnishd[13456]: Child cleanup complete Dec 04 09:03:01 varnish varnishd[13456]: Child cleanup complete ? System environment: opensuse 13.1 x86_64 tom -- Tom Anheyer Entwicklung & Technik Karl-Liebknecht-Stra?e 29 | 10178 Berlin | Germany Tel.: +49 (30) 23 27 - 52 10 Fax: +49 (30) 23 27 - 55 96 E-Mail: tom.anheyer at berlinonline.de Berlin.de | BerlinOnline.de | B2B-Deutschland.de | Deutschland-Reise.de | VisitBerlin.de Amtsgericht Berlin-Charlottenburg, HRA 31951 Sitz der Gesellschaft: Berlin, Deutschland Gesch?ftsf?hrer Olf Dziadek USt.-IdNr: DE219483549 pers?nlich haftender Gesellschafter: BerlinOnline Stadtportalbeteiligungsgesellschaft mbH Amtsgericht Berlin-Charlottenburg, HRB 79077 Sitz der Gesellschaft: Berlin, Deutschland Gesch?ftsf?hrer Olf Dziadek -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4266 bytes Desc: S/MIME Cryptographic Signature URL: From paulojjs at gmail.com Sat Dec 5 13:59:01 2015 From: paulojjs at gmail.com (Paulo Silva) Date: Sat, 5 Dec 2015 13:59:01 +0000 Subject: Purging behaviour Message-ID: Hi, I'm using varnish 4.1 and in my setup I'm using vcl_hash to save a different version in cache if a cookie is present, something like this: sub vcl_hash { if (req.http.cookie ~ "mycookie=") { hash_data("has_mycookie"); } } This bit is working very well but I was wondering how this will affect HTTP PURGE. Since the purge requests will only have the hostname and url, is purge smart enough to remove from cache both versions or will it remove only the object without the cookie? If only one version is removed is there some way to force varnish to remove both versions (without issuing a second PURGE request with the cookie)? Thanks -- Paulo Silva -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Sat Dec 5 14:22:47 2015 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Sat, 5 Dec 2015 15:22:47 +0100 Subject: Purging behaviour In-Reply-To: References: Message-ID: On Dec 5, 2015 15:07, "Paulo Silva" wrote: > Since the purge requests will only have the hostname and url, is purge smart enough to remove from cache both versions or will it remove only the object without the cookie? If only one version is removed is there some way to force varnish to remove both versions (without issuing a second PURGE request with the cookie)? Purge is not smart :-) it will purge a whole Objhead, ie. all objects with the same hash. Unfortunately, what you are doing stores the objects under different hashes. What you need here is the vary-header. I won't go into details now (you should have plenty of info in the varnish book), but basically, you can set req.http.mycookie = "yes" or "no" and beresp.http.vary = "mycookie" and varnish will do what you expect. -------------- next part -------------- An HTML attachment was scrubbed... URL: From paulojjs at gmail.com Sun Dec 6 10:54:43 2015 From: paulojjs at gmail.com (Paulo Silva) Date: Sun, 6 Dec 2015 10:54:43 +0000 Subject: Purging behaviour In-Reply-To: References: Message-ID: Thank you, that did the trick :-). 2015-12-05 14:22 GMT+00:00 Guillaume Quintard < guillaume at varnish-software.com>: > > On Dec 5, 2015 15:07, "Paulo Silva" wrote: > > Since the purge requests will only have the hostname and url, is purge > smart enough to remove from cache both versions or will it remove only the > object without the cookie? If only one version is removed is there some way > to force varnish to remove both versions (without issuing a second PURGE > request with the cookie)? > > Purge is not smart :-) it will purge a whole Objhead, ie. all objects with > the same hash. > > Unfortunately, what you are doing stores the objects under different > hashes. > > What you need here is the vary-header. I won't go into details now (you > should have plenty of info in the varnish book), but basically, you can set > req.http.mycookie = "yes" or "no" and beresp.http.vary = "mycookie" and > varnish will do what you expect. > -- Paulo Silva -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.lewis at nytimes.com Tue Dec 8 21:33:40 2015 From: eric.lewis at nytimes.com (Lewis, Eric) Date: Tue, 8 Dec 2015 16:33:40 -0500 Subject: Varnish and Docker Message-ID: Hey, we have recently open sourced a Dockerfile for creating a Varnish image, available in a GitHub repo as well as in a Docker Hub repository . I'd like to solicit feedback from you and invite the Varnish community to contribute to this effort. Most Dockerfiles install Varnish via a linux package manager, which makes it impossible to install VMODs because they need to be compiled against Varnish source. This image installs Varnish from source, and has straight-forward instructions and boilerplate for installing a VMOD. Not so far down the road, we can make a proposal to the Docker community to create an official repository for Docker. This would be a great step forward for easing Varnish adoption in this area. Thanks, Eric Lewis Web Developer, Interactive News The New York Times 620 Eighth Avenue, 2nd Floor New York, NY 10018 Office: (212) 556-2081 Cell: (610) 715-8560 -------------- next part -------------- An HTML attachment was scrubbed... URL: From geoff at uplex.de Wed Dec 9 09:57:02 2015 From: geoff at uplex.de (Geoff Simmons) Date: Wed, 9 Dec 2015 10:57:02 +0100 Subject: Varnish and Docker In-Reply-To: References: Message-ID: <5667FAEE.7090504@uplex.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 12/08/2015 10:33 PM, Lewis, Eric wrote: > > I'd like to solicit feedback from you and invite the Varnish > community to contribute to this effort. > > Most Dockerfiles install Varnish via a linux package manager, > which makes it impossible to install VMODs because they need to be > compiled against Varnish source. That's true of Varnish 3.x, but not since 4.0 -- VMODs are now built against a Varnish installation, which has all of the necessary headers and libs. It should be enough just to have pkg-config return the correct information for 'varnishapi', which will be the case without further intervention in a default Varnish installation. Not a big deal, since the Docker containers will presumably be for specific versions. It's just for the 4.x containers, the sources won't be necessary, at least not for building VMODs. Best, Geoff - -- ** * * UPLEX - Nils Goroll Systemoptimierung Scheffelstra?e 32 22301 Hamburg Tel +49 40 2880 5731 Mob +49 176 636 90917 Fax +49 40 42949753 http://uplex.de -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBCAAGBQJWZ/rjAAoJEOUwvh9pJNURoX4QAK9Ds9lgBlISF1CrDntwAChe ZX3cHXPcCaE+mUR8t+ESKF6I1t9ewf9jFVjrHi2oW6ZVHUAx+fQbdEvULi4PzfUj zumWrtzOA9kAx55ECBBCOxthZ7fLyDKSZxrJxeJVMpm5p4L46KyjEUUEDdm/FLZN cWUTIweHdfgroW+7McqxFlVRMiDDUEcoBXoAI9P23BXVl3doKBx/bAfyZUR6yn0y ESdZVKhlH2cdN1tFSuFO/v6Qd6YrhMPyG30d94UgZ1pUmzpqKNSQx4SkX1LU5jzx XJMXC9AcJzFTzMdGRfIOHFu0Opx2QIRQii5AOd58Bv3HmEjpZJFnJuwclfkSBxPK 4ASI3/Q/8LL3u5s2XGT7R4qNK095swP9kYPLDAclBMe/VDNyBpwdkbOm69yr3rhk j0u0ErkDUFOVucS5jXSt9LxS/7EX5WexDNIrQA+E8Uab2IbzaznMKS/xgKfrHcl1 WZSwTj/2in8QqaxIzYKAVjrXlRgEnn0QAR6+ZKe0+Str0I4BMw0zJ6Yb+DLJlw0t cKfBZ9rNLr+MqAsJ7YMyhHUgmuI9LIyHWr/hbhgqccM5LIURy7ZGISJ5MrAUyi1k 9ONKivNZc4Y+oAsQLxFhfwRWQ39ExerJMNjWIEx3SxpV+DEFk5iJ7RBT4KY3mfXe E4VJJ+ueBbfztEHaaFG2 =jo7N -----END PGP SIGNATURE----- From Tom.Anheyer at Berlinonline.de Wed Dec 9 10:55:16 2015 From: Tom.Anheyer at Berlinonline.de (Tom Anheyer) Date: Wed, 9 Dec 2015 11:55:16 +0100 Subject: V4.1 HTTP/1.0 responses without Content-Length header results in empty response Message-ID: <56680894.80904@Berlinonline.de> Hi, An old perl LWP script uses HTTP/1.0 posts to an apache backend. The generated backend response has no Content-Length header. Varnish 4.1.0 doesn't read the body (got a read error) and delivers a "Content-Length: 0" response back to the script. I know - http/1.0 its old and should/must be avoided and the problems are gone after updating the script to use http/1.1. But is this varnish behavior correct? best regards tom -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4266 bytes Desc: S/MIME Cryptographic Signature URL: From phk at phk.freebsd.dk Wed Dec 9 11:12:41 2015 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Wed, 09 Dec 2015 11:12:41 +0000 Subject: V4.1 HTTP/1.0 responses without Content-Length header results in empty response In-Reply-To: <56680894.80904@Berlinonline.de> References: <56680894.80904@Berlinonline.de> Message-ID: <3498.1449659561@critter.freebsd.dk> -------- In message <56680894.80904 at Berlinonline.de>, Tom Anheyer writes: >An old perl LWP script uses HTTP/1.0 posts to an apache backend. The >generated backend response has no Content-Length header. Varnish 4.1.0 >doesn't read the body (got a read error) and delivers a "Content-Length: 0" >response back to the script. I know - http/1.0 its old and >should/must be avoided and the problems are gone after updating the >script to use http/1.1. Please capture the varnishlog output so we can see exactly what happens. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From Tom.Anheyer at Berlinonline.de Wed Dec 9 12:39:12 2015 From: Tom.Anheyer at Berlinonline.de (Tom Anheyer) Date: Wed, 9 Dec 2015 13:39:12 +0100 Subject: V4.1 HTTP/1.0 responses without Content-Length header results in empty response In-Reply-To: <3498.1449659561@critter.freebsd.dk> References: <56680894.80904@Berlinonline.de> <3498.1449659561@critter.freebsd.dk> Message-ID: <566820F0.9090009@Berlinonline.de> Hello, the request to the backend using netcat: GET /tolle/api/url HTTP/1.0 User-Agent: location=XXX Content-Length: 0 Content-Type: application/x-www-form-urlencoded X-Remote-IP: 212.45.105.231 X-SSL: no X-Forwarded-For: 212.45.105.236 Host: service.berlin.de X-Portal: service.berlin.de X-Url: /tolle/api/url X-Varnish: 14205995" HTTP/1.1 200 OK Date: Wed, 09 Dec 2015 12:10:07 GMT Server: Apache/2.4.10 (Linux/SUSE) X-Powered-By: PHP/5.4.20 Expires: Thu, 19 Nov 1981 08:52:00 GMT Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Pragma: no-cache Strict-Transport-Security: max-age=3600; Set-Cookie: PHPSESSID=fjbpu6qtnhm0l6ev6hv9ievdeptgnvk0ejqeioaito6oeghqdhh1; path=/ X-Frame-Options: SAMEORIGIN Cache-Control: proxy-revalidate Connection: close Content-Type: text/html; charset=utf-8 -------- > In message <56680894.80904 at Berlinonline.de>, Tom Anheyer writes: > >> An old perl LWP script uses HTTP/1.0 posts to an apache backend. The >> generated backend response has no Content-Length header. Varnish 4.1.0 >> doesn't read the body (got a read error) and delivers a "Content-Length: 0" >> response back to the script. I know - http/1.0 its old and >> should/must be avoided and the problems are gone after updating the >> script to use http/1.1. > Please capture the varnishlog output so we can see exactly what > happens. > -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4266 bytes Desc: S/MIME Cryptographic Signature URL: From phk at phk.freebsd.dk Wed Dec 9 17:41:33 2015 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Wed, 09 Dec 2015 17:41:33 +0000 Subject: V4.1 HTTP/1.0 responses without Content-Length header results in empty response In-Reply-To: <566820F0.9090009@Berlinonline.de> References: <56680894.80904@Berlinonline.de> <3498.1449659561@critter.freebsd.dk> <566820F0.9090009@Berlinonline.de> Message-ID: <37349.1449682893@critter.freebsd.dk> -------- In message <566820F0.9090009 at Berlinonline.de>, Tom Anheyer writes: >the request to the backend using netcat: I think this is a bug we have already fixed. Try adding: sub vcl_backend_fetch { set bereq.proto = "HTTP/1.1"; } That should fix it. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From eric.lewis at nytimes.com Wed Dec 9 17:55:34 2015 From: eric.lewis at nytimes.com (Lewis, Eric) Date: Wed, 9 Dec 2015 12:55:34 -0500 Subject: Varnish and Docker In-Reply-To: <5667FAEE.7090504@uplex.de> References: <5667FAEE.7090504@uplex.de> Message-ID: > > Not a big deal, since the Docker containers will presumably be for specific > versions. It's just for the 4.x containers, the sources won't be > necessary, at least not for building VMODs. This is great to hear! I've put together a quick Dockerfile that uses Debian packages to install the latest stable (4.0.3). On top of this, I was able to install the example module with ease. However, other Varnish modules can't be built so easily on 4.0 yet . Module authors need to update their modules to build from the Varnish installation with 4.0 instead of from source. Perhaps we could reach out to the VMOD developer community about this. Until there's more support for this in modules, installing Varnish from source seems like the best default, so users don't get burned. Eric Lewis Web Developer, Interactive News The New York Times 620 Eighth Avenue, 2nd Floor New York, NY 10018 Office: (212) 556-2081 Cell: (610) 715-8560 On Wed, Dec 9, 2015 at 4:57 AM, Geoff Simmons wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > On 12/08/2015 10:33 PM, Lewis, Eric wrote: > > > > I'd like to solicit feedback from you and invite the Varnish > > community to contribute to this effort. > > > > Most Dockerfiles install Varnish via a linux package manager, > > which makes it impossible to install VMODs because they need to be > > compiled against Varnish source. > > That's true of Varnish 3.x, but not since 4.0 -- VMODs are now built > against a Varnish installation, which has all of the necessary headers > and libs. It should be enough just to have pkg-config return the > correct information for 'varnishapi', which will be the case without > further intervention in a default Varnish installation. > > Not a big deal, since the Docker containers will presumably be for > specific versions. It's just for the 4.x containers, the sources won't > be necessary, at least not for building VMODs. > > > Best, > Geoff > - -- > ** * * UPLEX - Nils Goroll Systemoptimierung > > Scheffelstra?e 32 > 22301 Hamburg > > Tel +49 40 2880 5731 > Mob +49 176 636 90917 > Fax +49 40 42949753 > > http://uplex.de > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1 > > iQIcBAEBCAAGBQJWZ/rjAAoJEOUwvh9pJNURoX4QAK9Ds9lgBlISF1CrDntwAChe > ZX3cHXPcCaE+mUR8t+ESKF6I1t9ewf9jFVjrHi2oW6ZVHUAx+fQbdEvULi4PzfUj > zumWrtzOA9kAx55ECBBCOxthZ7fLyDKSZxrJxeJVMpm5p4L46KyjEUUEDdm/FLZN > cWUTIweHdfgroW+7McqxFlVRMiDDUEcoBXoAI9P23BXVl3doKBx/bAfyZUR6yn0y > ESdZVKhlH2cdN1tFSuFO/v6Qd6YrhMPyG30d94UgZ1pUmzpqKNSQx4SkX1LU5jzx > XJMXC9AcJzFTzMdGRfIOHFu0Opx2QIRQii5AOd58Bv3HmEjpZJFnJuwclfkSBxPK > 4ASI3/Q/8LL3u5s2XGT7R4qNK095swP9kYPLDAclBMe/VDNyBpwdkbOm69yr3rhk > j0u0ErkDUFOVucS5jXSt9LxS/7EX5WexDNIrQA+E8Uab2IbzaznMKS/xgKfrHcl1 > WZSwTj/2in8QqaxIzYKAVjrXlRgEnn0QAR6+ZKe0+Str0I4BMw0zJ6Yb+DLJlw0t > cKfBZ9rNLr+MqAsJ7YMyhHUgmuI9LIyHWr/hbhgqccM5LIURy7ZGISJ5MrAUyi1k > 9ONKivNZc4Y+oAsQLxFhfwRWQ39ExerJMNjWIEx3SxpV+DEFk5iJ7RBT4KY3mfXe > E4VJJ+ueBbfztEHaaFG2 > =jo7N > -----END PGP SIGNATURE----- > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nicolas.fortier at quebecormedia.com Wed Dec 9 20:03:15 2015 From: nicolas.fortier at quebecormedia.com (Nicolas Fortier) Date: Wed, 9 Dec 2015 20:03:15 +0000 Subject: unable to view hash key in varnishlog (varnish 4.1) Message-ID: Hi, We upgraded to varnish 4.1 and now, We don't see hash key in varnishlog. How can I configure vcl_hash or deliver to view hash key? Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From nesta at iceburg.net Thu Dec 10 05:37:14 2015 From: nesta at iceburg.net (Brice Burgess) Date: Thu, 10 Dec 2015 00:37:14 -0500 Subject: Varnish and Docker In-Reply-To: References: <5667FAEE.7090504@uplex.de> Message-ID: <56690F8A.2030206@iceburg.net> Eric, The way I'm reading this dockerfile, you're using the official debian 8 (already compiled) varnish package? Perhaps a bit off-topic, but it's generally a good approach to build from source -- and using a build system to allow quick distribution of alternate versions (such as running a specific commit/release). So I think building from source does make sense. Docker plays very nicely here with with flexible version-tagging of images and 1.9 supports build arguments. Also helpful to offer more base images, and recommend alpine:latest as a lightweight alternative to debian:jessie (haven't personally compiled varnish on busybox/musl, although 4.1.0 exists as an alpine package). ~ Brice On 12/09/2015 12:55 PM, Lewis, Eric wrote: > > Not a big deal, since the Docker containers will presumably be for > specific versions. It's just for the 4.x containers, the sources > won't be necessary, at least not for building VMODs. > > > This is great to hear! I've put together a quick Dockerfile > that > uses Debian packages to install the latest stable (4.0.3). On top of > this, I was able to install the example module > with ease. > > However, other Varnish modules can't be built so easily on 4.0 yet > . Module > authors need to update their modules to build from the Varnish > installation with 4.0 instead of from source. Perhaps we could reach > out to the VMOD developer community about this. > > Until there's more support for this in modules, installing Varnish > from source seems like the best default, so users don't get burned. > > Eric Lewis > Web Developer, Interactive News > The New York Times > 620 Eighth Avenue, 2nd Floor > New York, NY 10018 > Office: (212) 556-2081 > Cell: (610) 715-8560 > > On Wed, Dec 9, 2015 at 4:57 AM, Geoff Simmons > wrote: > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > On 12/08/2015 10:33 PM, Lewis, Eric wrote: > > > > I'd like to solicit feedback from you and invite the Varnish > > community to contribute to this effort. > > > > Most Dockerfiles install Varnish via a linux package manager, > > which makes it impossible to install VMODs because they need to be > > compiled against Varnish source. > > That's true of Varnish 3.x, but not since 4.0 -- VMODs are now built > against a Varnish installation, which has all of the necessary headers > and libs. It should be enough just to have pkg-config return the > correct information for 'varnishapi', which will be the case without > further intervention in a default Varnish installation. > > Not a big deal, since the Docker containers will presumably be for > specific versions. It's just for the 4.x containers, the sources won't > be necessary, at least not for building VMODs. > > > Best, > Geoff > - -- > ** * * UPLEX - Nils Goroll Systemoptimierung > > Scheffelstra?e 32 > 22301 Hamburg > > Tel +49 40 2880 5731 > Mob +49 176 636 90917 > Fax +49 40 42949753 > > http://uplex.de > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1 > > iQIcBAEBCAAGBQJWZ/rjAAoJEOUwvh9pJNURoX4QAK9Ds9lgBlISF1CrDntwAChe > ZX3cHXPcCaE+mUR8t+ESKF6I1t9ewf9jFVjrHi2oW6ZVHUAx+fQbdEvULi4PzfUj > zumWrtzOA9kAx55ECBBCOxthZ7fLyDKSZxrJxeJVMpm5p4L46KyjEUUEDdm/FLZN > cWUTIweHdfgroW+7McqxFlVRMiDDUEcoBXoAI9P23BXVl3doKBx/bAfyZUR6yn0y > ESdZVKhlH2cdN1tFSuFO/v6Qd6YrhMPyG30d94UgZ1pUmzpqKNSQx4SkX1LU5jzx > XJMXC9AcJzFTzMdGRfIOHFu0Opx2QIRQii5AOd58Bv3HmEjpZJFnJuwclfkSBxPK > 4ASI3/Q/8LL3u5s2XGT7R4qNK095swP9kYPLDAclBMe/VDNyBpwdkbOm69yr3rhk > j0u0ErkDUFOVucS5jXSt9LxS/7EX5WexDNIrQA+E8Uab2IbzaznMKS/xgKfrHcl1 > WZSwTj/2in8QqaxIzYKAVjrXlRgEnn0QAR6+ZKe0+Str0I4BMw0zJ6Yb+DLJlw0t > cKfBZ9rNLr+MqAsJ7YMyhHUgmuI9LIyHWr/hbhgqccM5LIURy7ZGISJ5MrAUyi1k > 9ONKivNZc4Y+oAsQLxFhfwRWQ39ExerJMNjWIEx3SxpV+DEFk5iJ7RBT4KY3mfXe > E4VJJ+ueBbfztEHaaFG2 > =jo7N > -----END PGP SIGNATURE----- > > > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc -------------- next part -------------- An HTML attachment was scrubbed... URL: From Tom.Anheyer at Berlinonline.de Thu Dec 10 06:44:05 2015 From: Tom.Anheyer at Berlinonline.de (Tom Anheyer) Date: Thu, 10 Dec 2015 07:44:05 +0100 Subject: V4.1 HTTP/1.0 responses without Content-Length header results in empty response In-Reply-To: <37349.1449682893@critter.freebsd.dk> References: <56680894.80904@Berlinonline.de> <3498.1449659561@critter.freebsd.dk> <566820F0.9090009@Berlinonline.de> <37349.1449682893@critter.freebsd.dk> Message-ID: <56691F35.9020505@Berlinonline.de> Hello, thank you very much. It works. tom Am 09.12.2015 um 18:41 schrieb Poul-Henning Kamp: > set bereq.proto = "HTTP/1.1"; -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4266 bytes Desc: S/MIME Cryptographic Signature URL: From eric.lewis at nytimes.com Thu Dec 10 12:34:13 2015 From: eric.lewis at nytimes.com (Lewis, Eric) Date: Thu, 10 Dec 2015 07:34:13 -0500 Subject: Varnish and Docker In-Reply-To: <56690F8A.2030206@iceburg.net> References: <5667FAEE.7090504@uplex.de> <56690F8A.2030206@iceburg.net> Message-ID: Hi Brice, The way I'm reading this dockerfile, you're using the official debian 8 > (already compiled) varnish package? If you're looking at the Dockerfile from my last message, that was an example to show that it is possible to use a system package, but problematic because some Varnish Modules cannot be installed based on that package. The Dockerfile in our repo installs from source. Sorry if that was unclear! Perhaps a bit off-topic, but it's generally a good approach to build from > source -- and using a build system to allow quick distribution of alternate > versions (such as running a specific commit/release). So I think building > from source does make sense. Totally agree :) Also helpful to offer more base images, and recommend alpine:latest as a > lightweight alternative to debian:jessie Most official Dockerfiles are based on Debian, so we bend with this community practice. To move towards an official image repository, we should have one image. Thanks, Eric Lewis Web Developer, Interactive News The New York Times 620 Eighth Avenue, 2nd Floor New York, NY 10018 Office: (212) 556-2081 Cell: (610) 715-8560 On Thu, Dec 10, 2015 at 12:37 AM, Brice Burgess wrote: > Eric, > > The way I'm reading this dockerfile, you're using the official debian 8 > (already compiled) varnish package? > > Perhaps a bit off-topic, but it's generally a good approach to build from > source -- and using a build system to allow quick distribution of alternate > versions (such as running a specific commit/release). So I think building > from source does make sense. > > Docker plays very nicely here with with flexible version-tagging of images > and 1.9 supports build arguments. Also helpful to offer more base images, > and recommend alpine:latest as a lightweight alternative to debian:jessie > (haven't personally compiled varnish on busybox/musl, although 4.1.0 exists > as an alpine package). > > > ~ Brice > > > On 12/09/2015 12:55 PM, Lewis, Eric wrote: > > Not a big deal, since the Docker containers will presumably be for specific >> versions. It's just for the 4.x containers, the sources won't be >> necessary, at least not for building VMODs. > > > This is great to hear! I've put together a quick Dockerfile > that uses > Debian packages to install the latest stable (4.0.3). On top of this, I was > able to install the example module > with ease. > > However, other Varnish modules can't be built so easily on 4.0 yet > . Module authors > need to update their modules to build from the Varnish installation with > 4.0 instead of from source. Perhaps we could reach out to the VMOD > developer community about this. > > Until there's more support for this in modules, installing Varnish from > source seems like the best default, so users don't get burned. > > Eric Lewis > Web Developer, Interactive News > The New York Times > 620 Eighth Avenue, 2nd Floor > New York, NY 10018 > Office: (212) 556-2081 > Cell: (610) 715-8560 > > On Wed, Dec 9, 2015 at 4:57 AM, Geoff Simmons wrote: > >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA256 >> >> On 12/08/2015 10:33 PM, Lewis, Eric wrote: >> > >> > I'd like to solicit feedback from you and invite the Varnish >> > community to contribute to this effort. >> > >> > Most Dockerfiles install Varnish via a linux package manager, >> > which makes it impossible to install VMODs because they need to be >> > compiled against Varnish source. >> >> That's true of Varnish 3.x, but not since 4.0 -- VMODs are now built >> against a Varnish installation, which has all of the necessary headers >> and libs. It should be enough just to have pkg-config return the >> correct information for 'varnishapi', which will be the case without >> further intervention in a default Varnish installation. >> >> Not a big deal, since the Docker containers will presumably be for >> specific versions. It's just for the 4.x containers, the sources won't >> be necessary, at least not for building VMODs. >> >> >> Best, >> Geoff >> - -- >> ** * * UPLEX - Nils Goroll Systemoptimierung >> >> Scheffelstra?e 32 >> 22301 Hamburg >> >> Tel +49 40 2880 5731 <%2B49%2040%202880%205731> >> Mob +49 176 636 90917 >> Fax +49 40 42949753 <%2B49%2040%2042949753> >> >> http://uplex.de >> -----BEGIN PGP SIGNATURE----- >> Version: GnuPG v1 >> >> iQIcBAEBCAAGBQJWZ/rjAAoJEOUwvh9pJNURoX4QAK9Ds9lgBlISF1CrDntwAChe >> ZX3cHXPcCaE+mUR8t+ESKF6I1t9ewf9jFVjrHi2oW6ZVHUAx+fQbdEvULi4PzfUj >> zumWrtzOA9kAx55ECBBCOxthZ7fLyDKSZxrJxeJVMpm5p4L46KyjEUUEDdm/FLZN >> cWUTIweHdfgroW+7McqxFlVRMiDDUEcoBXoAI9P23BXVl3doKBx/bAfyZUR6yn0y >> ESdZVKhlH2cdN1tFSuFO/v6Qd6YrhMPyG30d94UgZ1pUmzpqKNSQx4SkX1LU5jzx >> XJMXC9AcJzFTzMdGRfIOHFu0Opx2QIRQii5AOd58Bv3HmEjpZJFnJuwclfkSBxPK >> 4ASI3/Q/8LL3u5s2XGT7R4qNK095swP9kYPLDAclBMe/VDNyBpwdkbOm69yr3rhk >> j0u0ErkDUFOVucS5jXSt9LxS/7EX5WexDNIrQA+E8Uab2IbzaznMKS/xgKfrHcl1 >> WZSwTj/2in8QqaxIzYKAVjrXlRgEnn0QAR6+ZKe0+Str0I4BMw0zJ6Yb+DLJlw0t >> cKfBZ9rNLr+MqAsJ7YMyhHUgmuI9LIyHWr/hbhgqccM5LIURy7ZGISJ5MrAUyi1k >> 9ONKivNZc4Y+oAsQLxFhfwRWQ39ExerJMNjWIEx3SxpV+DEFk5iJ7RBT4KY3mfXe >> E4VJJ+ueBbfztEHaaFG2 >> =jo7N >> -----END PGP SIGNATURE----- >> > > > > _______________________________________________ > varnish-misc mailing listvarnish-misc at varnish-cache.orghttps://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From viktor.villafuerte at optusnet.com.au Fri Dec 11 04:19:40 2015 From: viktor.villafuerte at optusnet.com.au (Viktor Villafuerte) Date: Fri, 11 Dec 2015 15:19:40 +1100 Subject: thread_pool_max Message-ID: <20151211041940.GB1490@optusnet.com.au> Ladies, gents In the Varnish on-line docs (section Tuning) there's a paragraph that states: "We rarely recommend running with more than 5000 threads. If you seem to need more than 5000 threads, it?s very likely that there is something not quite right about your setup.." I'm in a situtation where many many tens of thousands of keep-alive sessions can be established. Since every user needs two connections and I use 2 pools of 5000 threads, the max I can use is 10000 threads which does not sound enough to me. This is related to HLS streaming (TV channels) and I've currently increased thread_pool_max to 15000. This possibly can/will be not enough but better than 5000. The server that runs this is a beast with massive amount of memory and large number of CPUs capable of running of up to 35Gbps of traffic (4 x 10Gbps interfaces) with outlook of adding 2 more 10Gbps interfaces. So I do not face any underlying resource limitation (if that's part of reasoning to keep the number of threads at ~10k) >From my observations of current behaviour a connection does not mean a corresponding thread as I observe about 4:1 ratio of connections vs threads. I'd like to ask why the recommended number of threads in a pool is 5000? What would be the limit there and why? I assume this would be Varnish limit and may be serious problem if it cannot be overcome. Currently there are not that many users (cca 10000) but I'd like to find out about any such limitation before I actually hit it.. Please let me know or point me to any documentation/explanation of threds and threads pools I may have not found. thanks -- Regards Viktor Villafuerte Optus Internet Engineering t: +61 2 80825265 From perbu at varnish-software.com Fri Dec 11 07:36:06 2015 From: perbu at varnish-software.com (Per Buer) Date: Fri, 11 Dec 2015 08:36:06 +0100 Subject: thread_pool_max In-Reply-To: <20151211041940.GB1490@optusnet.com.au> References: <20151211041940.GB1490@optusnet.com.au> Message-ID: On Fri, Dec 11, 2015 at 5:19 AM, Viktor Villafuerte < viktor.villafuerte at optusnet.com.au> wrote: > Ladies, gents > > In the Varnish on-line docs (section Tuning) there's a paragraph that > states: > > "We rarely recommend running with more than 5000 threads. If you > seem to need more than 5000 threads, it?s very likely that there is > something not quite right about your setup.." > > Those are my words. 99% installation have a request rate under 2-3000 request per second and 5000 threads is more than enough. We've seen quite a few people adding tens of thousands of threads. I should probably replace with something like a formula. There are no well defined limits wrt capacity but in general people seem to struggle to push Varnish significantly beyond 15Gbps with real with traffic. I have not experienced this first hand and nobody has given me access to a live server with those kind of traffic numbers on it. Also, nobody get particularly specific wrt what breaks. I think it definitely should be possible to reach 20Gbps without any issue, but that is more an opinion than a fact. With synthetic traffic it is no problem reaching 80Gbps. -- *Per Buer* CTO | Varnish Software AS Cell: +47 95839117 We Make Websites Fly! www.varnish-software.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From joh.hendriks at gmail.com Fri Dec 11 19:16:14 2015 From: joh.hendriks at gmail.com (Johan Hendriks) Date: Fri, 11 Dec 2015 20:16:14 +0100 Subject: check vcl file for errors, kill's varnishd Message-ID: Hello all. I use varnish 4.1 on a FreeBSD 10.2 machine. All is running fine but when I want to make a change to the config and test the changes with the command varnishd -C - f /usr/local/etc/varnish/default.vcl, then I need to restart varnish because it does not respond anymore. I thought maybe this can not be done with the running config so I used a different file but it shows the same behavior. Is it something I am doing wrong, has things changed with 4.1 or is it something else. Thanks for your time. Regards Johan -------------- next part -------------- An HTML attachment was scrubbed... URL: From feld at feld.me Fri Dec 11 21:12:22 2015 From: feld at feld.me (Mark Felder) Date: Fri, 11 Dec 2015 15:12:22 -0600 Subject: check vcl file for errors, kill's varnishd In-Reply-To: References: Message-ID: <1449868342.3246100.465011425.720E2A3C@webmail.messagingengine.com> On Fri, Dec 11, 2015, at 13:16, Johan Hendriks wrote: > Hello all. > > I use varnish 4.1 on a FreeBSD 10.2 machine. All is running fine but when > I > want to make a change to the config and test the changes with the > command varnishd -C - f /usr/local/etc/varnish/default.vcl, then I need > to restart varnish because it does not respond anymore. > I thought maybe this can not be done with the running config so I used a > different file but it shows the same behavior. > > Is it something I am doing wrong, has things changed with 4.1 or is it > something else. > > Thanks for your time. > How did you install Varnish on FreeBSD? The lastest port (varnish4-4.1.0_2) has some nice features such as "service varnishd configtest" that will do the config test for you. In my testing I have not seen this cause varnish to stop responding. -- Mark Felder feld at feld.me From joh.hendriks at gmail.com Fri Dec 11 21:53:38 2015 From: joh.hendriks at gmail.com (Johan Hendriks) Date: Fri, 11 Dec 2015 22:53:38 +0100 Subject: check vcl file for errors, kill's varnishd In-Reply-To: <1449868342.3246100.465011425.720E2A3C@webmail.messagingengine.com> References: <1449868342.3246100.465011425.720E2A3C@webmail.messagingengine.com> Message-ID: <566B45E2.10200@gmail.com> I installed varnish with the ports system. The whole day I encountered this issue while we where testing, now I can use it without any problem? I can not replicate the problem anymore. Sorry for the noice. regards Johan Op 11/12/15 om 22:12 schreef Mark Felder: > > On Fri, Dec 11, 2015, at 13:16, Johan Hendriks wrote: >> Hello all. >> >> I use varnish 4.1 on a FreeBSD 10.2 machine. All is running fine but when >> I >> want to make a change to the config and test the changes with the >> command varnishd -C - f /usr/local/etc/varnish/default.vcl, then I need >> to restart varnish because it does not respond anymore. >> I thought maybe this can not be done with the running config so I used a >> different file but it shows the same behavior. >> >> Is it something I am doing wrong, has things changed with 4.1 or is it >> something else. >> >> Thanks for your time. >> > How did you install Varnish on FreeBSD? The lastest port > (varnish4-4.1.0_2) has some nice features such as "service varnishd > configtest" that will do the config test for you. > > In my testing I have not seen this cause varnish to stop responding. > From georgi.int at gmail.com Sat Dec 12 10:26:04 2015 From: georgi.int at gmail.com (georgi.int at gmail.com) Date: Sat, 12 Dec 2015 12:26:04 +0200 Subject: Why varnish caches as the cache is stopped? Message-ID: <566BF63C.3090802@gmail.com> Hello, I want to use varnish only as a proxy server, not to cache and have the following in the configuration which should stop the cache completely in vcl recv: if (req.http.Authorization || req.http.Cookie) { return (pass); } } the age header is 0, so it should not cache, but I notice that varnish_storage.bin is full although, so something is cached - right? On some servers with 32 GB ram29 is full and when I restart varnish the ram is no more used, but I am not sure it these two things are linked. Here is my /etc/sysconfig/varnish: # # The minimum number of worker threads to start VARNISH_MIN_THREADS=50 # # The Maximum number of worker threads to start VARNISH_MAX_THREADS=1000 # # Idle timeout for worker threads VARNISH_THREAD_TIMEOUT=120 # # Cache file location VARNISH_STORAGE_FILE=/usr/local/varnish/lib/varnish_storage.bin # # Cache file size: in bytes, optionally using k / M / G / T suffix, # # or in percentage of available disk space using the % suffix. VARNISH_STORAGE_SIZE=1G # # Backend storage specification VARNISH_STORAGE="file,${VARNISH_STORAGE_FILE},${VARNISH_STORAGE_SIZE}" # # Default TTL used when the backend does not specify one VARNISH_TTL=120 # # Varnish username VARNISHUSER=varnish # # Varnish group name VARNISHGROUP=varnish DAEMON_OPTS="-a ${VARNISH_LISTEN_ADDRESS}:${VARNISH_LISTEN_PORT} \ -T ${VARNISH_ADMIN_LISTEN_ADDRESS}:${VARNISH_ADMIN_LISTEN_PORT} \ -f ${VARNISH_VCL_CONF} \ -S ${VARNISH_SECRET_FILE} \ -s file,${VARNISH_STORAGE_FILE},${VARNISH_STORAGE_SIZE}" My questions are 1) does my conviguration stop completely the cache and why then storage.bin is full? 2) Why varnish eat so much ram - because of many requests, wrong configuration or? From colas.delmas at gmail.com Sun Dec 13 09:42:16 2015 From: colas.delmas at gmail.com (Nicolas Delmas) Date: Sun, 13 Dec 2015 10:42:16 +0100 Subject: Why varnish caches as the cache is stopped? In-Reply-To: <566BF63C.3090802@gmail.com> References: <566BF63C.3090802@gmail.com> Message-ID: Hi, If you don't need to modify header you could make a return(pipe) instead of ? return(pass). Maybe this could resolve your problem. Le 12 d?c. 2015 11:40 AM, "georgi.int at gmail.com" a ?crit : > Hello, > I want to use varnish only as a proxy server, not to cache and have the > following in the configuration which should stop the cache completely in > vcl recv: > > if (req.http.Authorization || req.http.Cookie) { > return (pass); > } > } > > the age header is 0, so it should not cache, but I notice that > varnish_storage.bin is full although, so something is cached - right? On > some servers with 32 GB ram29 is full and when I restart varnish the ram is > no more used, but I am not sure it these two things are linked. Here is my > /etc/sysconfig/varnish: > > # # The minimum number of worker threads to start > VARNISH_MIN_THREADS=50 > > # # The Maximum number of worker threads to start > VARNISH_MAX_THREADS=1000 > > # # Idle timeout for worker threads > VARNISH_THREAD_TIMEOUT=120 > > # # Cache file location > VARNISH_STORAGE_FILE=/usr/local/varnish/lib/varnish_storage.bin > > # # Cache file size: in bytes, optionally using k / M / G / T suffix, > # # or in percentage of available disk space using the % suffix. > VARNISH_STORAGE_SIZE=1G > > # # Backend storage specification > VARNISH_STORAGE="file,${VARNISH_STORAGE_FILE},${VARNISH_STORAGE_SIZE}" > > # # Default TTL used when the backend does not specify one > VARNISH_TTL=120 > > # # Varnish username > VARNISHUSER=varnish > > # # Varnish group name > VARNISHGROUP=varnish > > DAEMON_OPTS="-a ${VARNISH_LISTEN_ADDRESS}:${VARNISH_LISTEN_PORT} \ > -T > ${VARNISH_ADMIN_LISTEN_ADDRESS}:${VARNISH_ADMIN_LISTEN_PORT} \ > -f ${VARNISH_VCL_CONF} \ > -S ${VARNISH_SECRET_FILE} \ > -s file,${VARNISH_STORAGE_FILE},${VARNISH_STORAGE_SIZE}" > > > > My questions are 1) does my conviguration stop completely the cache and > why then storage.bin is full? > 2) Why varnish eat so much ram - because of many requests, wrong > configuration or? > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Sun Dec 13 10:24:26 2015 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Sun, 13 Dec 2015 11:24:26 +0100 Subject: Why varnish caches as the cache is stopped? In-Reply-To: <566BF63C.3090802@gmail.com> References: <566BF63C.3090802@gmail.com> Message-ID: On Dec 12, 2015 11:36, "georgi.int at gmail.com" wrote: > > Hello, > I want to use varnish only as a proxy server, not to cache and have the following in the configuration which should stop the cache completely in vcl recv: > > if (req.http.Authorization || req.http.Cookie) { > return (pass); > } > } > > the age header is 0, so it should not cache, but I notice that varnish_storage.bin is full although, so something is cached - right? On some servers with 32 GB ram29 is full and when I restart varnish the ram is no more used, but I am not sure it these two things are linked. Here is my /etc/sysconfig/varnish: > > # # The minimum number of worker threads to start > VARNISH_MIN_THREADS=50 > > # # The Maximum number of worker threads to start > VARNISH_MAX_THREADS=1000 > > # # Idle timeout for worker threads > VARNISH_THREAD_TIMEOUT=120 > > # # Cache file location > VARNISH_STORAGE_FILE=/usr/local/varnish/lib/varnish_storage.bin > > # # Cache file size: in bytes, optionally using k / M / G / T suffix, > # # or in percentage of available disk space using the % suffix. > VARNISH_STORAGE_SIZE=1G > > # # Backend storage specification > VARNISH_STORAGE="file,${VARNISH_STORAGE_FILE},${VARNISH_STORAGE_SIZE}" > > # # Default TTL used when the backend does not specify one > VARNISH_TTL=120 > > # # Varnish username > VARNISHUSER=varnish > > # # Varnish group name > VARNISHGROUP=varnish > > DAEMON_OPTS="-a ${VARNISH_LISTEN_ADDRESS}:${VARNISH_LISTEN_PORT} \ > -T ${VARNISH_ADMIN_LISTEN_ADDRESS}:${VARNISH_ADMIN_LISTEN_PORT} \ > -f ${VARNISH_VCL_CONF} \ > -S ${VARNISH_SECRET_FILE} \ > -s file,${VARNISH_STORAGE_FILE},${VARNISH_STORAGE_SIZE}" > > > > My questions are 1) does my conviguration stop completely the cache and why then storage.bin is full? > 2) Why varnish eat so much ram - because of many requests, wrong configuration or? > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc Hi, The best way to investigate here is to look at the varnishlog and check if indeed you end up in hit or miss instead of pass. -------------- next part -------------- An HTML attachment was scrubbed... URL: From Tom.Anheyer at Berlinonline.de Mon Dec 14 09:19:56 2015 From: Tom.Anheyer at Berlinonline.de (Tom Anheyer) Date: Mon, 14 Dec 2015 10:19:56 +0100 Subject: check vcl file for errors, kill's varnishd In-Reply-To: References: Message-ID: <566E89BC.7060201@Berlinonline.de> Hi, maybe you make the same mistake like me. Try: varnishd -C - f /usr/local/etc/varnish/default.vcl -n /tmp/ because varnishd -C ? deletes a n existing compiled boot config. regards tom Am 11.12.2015 um 20:16 schrieb Johan Hendriks: > Hello all. > > I use varnish 4.1 on a FreeBSD 10.2 machine. All is running fine but > when I want to make a change to the config and test the changes with > the command varnishd -C - f /usr/local/etc/varnish/default.vcl, > then I need to restart varnish because it does not respond anymore. > I thought maybe this can not be done with the running config so I used > a different file but it shows the same behavior. > > Is it something I am doing wrong, has things changed with 4.1 or is it > something else. > > Thanks for your time. > > Regards > Johan > > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4266 bytes Desc: S/MIME Cryptographic Signature URL: From georgi.int at gmail.com Mon Dec 14 16:47:28 2015 From: georgi.int at gmail.com (georgi.int at gmail.com) Date: Mon, 14 Dec 2015 18:47:28 +0200 Subject: Why varnish caches as the cache is stopped? In-Reply-To: References: <566BF63C.3090802@gmail.com> Message-ID: <566EF2A0.5040403@gmail.com> Logs didn't help me. Now I faced the same problem with php file which didn't refresh until varnish restart. After I put the following in vcl_recv { the problem was solved: if (req.url ~ "\.(png|gif|jp(e?)g|avi|flv|mp(e?)g|mp4|mp3|php|js)"){return(pass);} But why I should add this after I have : if (req.http.Authorization || req.http.Cookie) { return (pass); } } and is there better solution than the first one that wills top completely the cache? Also, I use the same config file on all servers and there isn't such a problem now, only on this customer. This is the log of the request befor I add the pass for certaing type of files: 58 TxResponse c OK 58 TxHeader c Server: Apache 58 TxHeader c Last-Modified: Mon, 12 Jul 2010 06:40:00 GMT 58 TxHeader c Content-Type: text/html 58 TxHeader c Content-Length: 5774 58 TxHeader c Accept-Ranges: bytes 58 TxHeader c Date: Mon, 14 Dec 2015 16:30:58 GMT 58 TxHeader c Connection: keep-alive 58 Length c 5774 58 ReqEnd c 2078285659 1450110658.014697075 1450110658.033298254 1.481907368 0.018534422 0.000066757 69 Debug c herding 69 SessionClose c timeout 69 StatSess c 212.233.149.2 54693 3 1 14 0 14 14 3823 211709 16 Debug c herding 16 Interrupted c ReqStart 16 ReqStart c 1.1.1.1 33259 2078285660 16 RxRequest c GET 16 RxURL c /testov.php 16 RxProtocol c HTTP/1.1 16 RxHeader c Host: siteofmycustomer.com 16 RxHeader c User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:42.0) Gecko/20100101 Firefox/42.0 16 RxHeader c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 16 RxHeader c Accept-Language: en-US,en;q=0.5 16 RxHeader c Accept-Encoding: gzip, deflate 16 RxHeader c Connection: keep-alive 16 RxHeader c Cache-Control: max-age=0 16 VCL_call c recv lookup 16 VCL_call c hash 16 Hash c /testov.php 16 Hash c termoproekt.bg 16 VCL_return c hash 16 Hit c 2078285077 16 VCL_call c hit deliver 16 VCL_call c deliver deliver 16 TxProtocol c HTTP/1.1 16 TxStatus c 200 16 TxResponse c OK 16 TxHeader c Server: Apache 16 TxHeader c Content-Type: text/html; charset=UTF-8 16 TxHeader c Content-Length: 0 16 TxHeader c Accept-Ranges: bytes 16 TxHeader c Date: Mon, 14 Dec 2015 16:30:58 GMT 16 TxHeader c Connection: keep-alive 16 Length c 0 On 12/13/2015 12:24 PM, Guillaume Quintard wrote: > > > On Dec 12, 2015 11:36, "georgi.int at gmail.com > " > wrote: > > > > Hello, > > I want to use varnish only as a proxy server, not to cache and have > the following in the configuration which should stop the cache > completely in vcl recv: > > > > if (req.http.Authorization || req.http.Cookie) { > > return (pass); > > } > > } > > > > the age header is 0, so it should not cache, but I notice that > varnish_storage.bin is full although, so something is cached - right? > On some servers with 32 GB ram29 is full and when I restart varnish > the ram is no more used, but I am not sure it these two things are > linked. Here is my /etc/sysconfig/varnish: > > > > # # The minimum number of worker threads to start > > VARNISH_MIN_THREADS=50 > > > > # # The Maximum number of worker threads to start > > VARNISH_MAX_THREADS=1000 > > > > # # Idle timeout for worker threads > > VARNISH_THREAD_TIMEOUT=120 > > > > # # Cache file location > > VARNISH_STORAGE_FILE=/usr/local/varnish/lib/varnish_storage.bin > > > > # # Cache file size: in bytes, optionally using k / M / G / T suffix, > > # # or in percentage of available disk space using the % suffix. > > VARNISH_STORAGE_SIZE=1G > > > > # # Backend storage specification > > VARNISH_STORAGE="file,${VARNISH_STORAGE_FILE},${VARNISH_STORAGE_SIZE}" > > > > # # Default TTL used when the backend does not specify one > > VARNISH_TTL=120 > > > > # # Varnish username > > VARNISHUSER=varnish > > > > # # Varnish group name > > VARNISHGROUP=varnish > > > > DAEMON_OPTS="-a ${VARNISH_LISTEN_ADDRESS}:${VARNISH_LISTEN_PORT} \ > > -T > ${VARNISH_ADMIN_LISTEN_ADDRESS}:${VARNISH_ADMIN_LISTEN_PORT} \ > > -f ${VARNISH_VCL_CONF} \ > > -S ${VARNISH_SECRET_FILE} \ > > -s file,${VARNISH_STORAGE_FILE},${VARNISH_STORAGE_SIZE}" > > > > > > > > My questions are 1) does my conviguration stop completely the cache > and why then storage.bin is full? > > 2) Why varnish eat so much ram - because of many requests, wrong > configuration or? > > > > _______________________________________________ > > varnish-misc mailing list > > varnish-misc at varnish-cache.org > > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > Hi, > > The best way to investigate here is to look at the varnishlog and > check if indeed you end up in hit or miss instead of pass. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Mon Dec 14 17:04:53 2015 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Mon, 14 Dec 2015 18:04:53 +0100 Subject: Why varnish caches as the cache is stopped? In-Reply-To: <566EF2A0.5040403@gmail.com> References: <566BF63C.3090802@gmail.com> <566EF2A0.5040403@gmail.com> Message-ID: On Dec 14, 2015 17:47, "georgi.int at gmail.com" wrote: > > Logs didn't help me. Now I faced the same problem with php file which didn't refresh until varnish restart. After I put the following in vcl_recv { the problem was solved: > > if (req.url ~ "\.(png|gif|jp(e?)g|avi|flv|mp(e?)g|mp4|mp3|php|js)"){return(pass);} > > But why I should add this after I have : > > > if (req.http.Authorization || req.http.Cookie) { > return (pass); > } > } > > and is there better solution than the first one that wills top completely the cache? Also, I use the same config file on all servers and there isn't such a problem now, only on this customer. > > This is the log of the request befor I add the pass for certaing type of files: > > 58 TxResponse c OK > 58 TxHeader c Server: Apache > 58 TxHeader c Last-Modified: Mon, 12 Jul 2010 06:40:00 GMT > 58 TxHeader c Content-Type: text/html > 58 TxHeader c Content-Length: 5774 > 58 TxHeader c Accept-Ranges: bytes > 58 TxHeader c Date: Mon, 14 Dec 2015 16:30:58 GMT > 58 TxHeader c Connection: keep-alive > 58 Length c 5774 > 58 ReqEnd c 2078285659 1450110658.014697075 1450110658.033298254 1.481907368 0.018534422 0.000066757 > 69 Debug c herding > 69 SessionClose c timeout > 69 StatSess c 212.233.149.2 54693 3 1 14 0 14 14 3823 211709 > 16 Debug c herding > 16 Interrupted c ReqStart > 16 ReqStart c 1.1.1.1 33259 2078285660 > 16 RxRequest c GET > 16 RxURL c /testov.php > 16 RxProtocol c HTTP/1.1 > 16 RxHeader c Host: siteofmycustomer.com > 16 RxHeader c User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:42.0) Gecko/20100101 Firefox/42.0 > 16 RxHeader c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 > 16 RxHeader c Accept-Language: en-US,en;q=0.5 > 16 RxHeader c Accept-Encoding: gzip, deflate > 16 RxHeader c Connection: keep-alive > 16 RxHeader c Cache-Control: max-age=0 > 16 VCL_call c recv lookup > 16 VCL_call c hash > 16 Hash c /testov.php > 16 Hash c termoproekt.bg > 16 VCL_return c hash > 16 Hit c 2078285077 > 16 VCL_call c hit deliver > 16 VCL_call c deliver deliver > 16 TxProtocol c HTTP/1.1 > 16 TxStatus c 200 > 16 TxResponse c OK > 16 TxHeader c Server: Apache > 16 TxHeader c Content-Type: text/html; charset=UTF-8 > 16 TxHeader c Content-Length: 0 > 16 TxHeader c Accept-Ranges: bytes > 16 TxHeader c Date: Mon, 14 Dec 2015 16:30:58 GMT > 16 TxHeader c Connection: keep-alive > 16 Length c 0 > Looking at the request (id 16), I see no reason for it to pass as it has no cookie or authorization header. Varnish seems to be doing the right thing. -------------- next part -------------- An HTML attachment was scrubbed... URL: From georgi.int at gmail.com Mon Dec 14 17:56:01 2015 From: georgi.int at gmail.com (georgi.int at gmail.com) Date: Mon, 14 Dec 2015 19:56:01 +0200 Subject: Why varnish caches as the cache is stopped? In-Reply-To: References: <566BF63C.3090802@gmail.com> <566EF2A0.5040403@gmail.com> Message-ID: <566F02B1.8070700@gmail.com> Yes, but the result were cached and can't understand why. While it's cached on port 80 where the varnish is, opening the page with :8080 was ok (on the apache side). Any other suggestions? On 12/14/2015 07:04 PM, Guillaume Quintard wrote: > Server: Apache > > 16 TxHeader c Content-Type: text/html; charset=UTF-8 > > 16 TxHeader c Content-Length: 0 > > 16 TxHeader c Accept-Ranges: by From georgi.int at gmail.com Mon Dec 14 18:06:17 2015 From: georgi.int at gmail.com (georgi.int at gmail.com) Date: Mon, 14 Dec 2015 20:06:17 +0200 Subject: Why varnish caches as the cache is stopped? In-Reply-To: <566F02B1.8070700@gmail.com> References: <566BF63C.3090802@gmail.com> <566EF2A0.5040403@gmail.com> <566F02B1.8070700@gmail.com> Message-ID: <566F0519.20600@gmail.com> My couleague told me that without restarting varnish when he had the same problem just wait around 5 minutes and the content is refreshed. I changed VARNISH_TTL=120 to 0, but not efect. I think that there is another ttl? On 12/14/2015 07:56 PM, georgi.int at gmail.com wrote: > Yes, but the result were cached and can't understand why. While it's > cached on port 80 where the varnish is, opening the page with :8080 > was ok (on the apache side). Any other suggestions? > > On 12/14/2015 07:04 PM, Guillaume Quintard wrote: >> Server: Apache >> > 16 TxHeader c Content-Type: text/html; charset=UTF-8 >> > 16 TxHeader c Content-Length: 0 >> > 16 TxHeader c Accept-Ranges: by > From viktor.villafuerte at optusnet.com.au Mon Dec 14 22:20:01 2015 From: viktor.villafuerte at optusnet.com.au (Viktor Villafuerte) Date: Tue, 15 Dec 2015 09:20:01 +1100 Subject: thread_pool_max In-Reply-To: References: <20151211041940.GB1490@optusnet.com.au> Message-ID: <20151214222001.GC1490@optusnet.com.au> Hi Per, On Fri 11 Dec 2015 08:36:06, Per Buer wrote: > On Fri, Dec 11, 2015 at 5:19 AM, Viktor Villafuerte < > viktor.villafuerte at optusnet.com.au> wrote: > > > Ladies, gents > > > > In the Varnish on-line docs (section Tuning) there's a paragraph that > > states: > > > > "We rarely recommend running with more than 5000 threads. If you > > seem to need more than 5000 threads, it?s very likely that there is > > something not quite right about your setup.." > > > > > Those are my words. 99% installation have a request rate under 2-3000 > request per second and 5000 threads is more than enough. We've seen quite a > few people adding tens of thousands of threads. I should probably replace > with something like a formula. > > There are no well defined limits wrt capacity but in general people seem to > struggle to push Varnish significantly beyond 15Gbps with real with > traffic. I have not experienced this first hand and nobody has given me > access to a live server with those kind of traffic numbers on it. Also, > nobody get particularly specific wrt what breaks. I think it definitely > should be possible to reach 20Gbps without any issue, but that is more an > opinion than a fact. Would you guys like to have some stats when we reach traffic levels over 15Gbps? If yes then what stats would you like? Currently we're running at 12Gbps very comfortably with 2 pools of 15k threads. > > With synthetic traffic it is no problem reaching 80Gbps. > > -- > *Per Buer* > CTO | Varnish Software AS > Cell: +47 95839117 > We Make Websites Fly! > www.varnish-software.com > -- Regards Viktor Villafuerte Optus Internet Engineering t: +61 2 80825265 From guillaume at varnish-software.com Tue Dec 15 08:58:44 2015 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Tue, 15 Dec 2015 09:58:44 +0100 Subject: Why varnish caches as the cache is stopped? In-Reply-To: <566F02B1.8070700@gmail.com> References: <566BF63C.3090802@gmail.com> <566EF2A0.5040403@gmail.com> <566F02B1.8070700@gmail.com> Message-ID: Why wouldn't it be cached? Sorry but I'm not getting the problem here. -- Guillaume Quintard On Mon, Dec 14, 2015 at 6:56 PM, georgi.int at gmail.com wrote: > Yes, but the result were cached and can't understand why. While it's > cached on port 80 where the varnish is, opening the page with :8080 was ok > (on the apache side). Any other suggestions? > > > On 12/14/2015 07:04 PM, Guillaume Quintard wrote: > >> Server: Apache >> > 16 TxHeader c Content-Type: text/html; charset=UTF-8 >> > 16 TxHeader c Content-Length: 0 >> > 16 TxHeader c Accept-Ranges: by >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From goncalo.silva at muzzley.com Tue Dec 15 15:57:19 2015 From: goncalo.silva at muzzley.com (=?UTF-8?Q?Gon=C3=A7alo_Silva?=) Date: Tue, 15 Dec 2015 15:57:19 +0000 Subject: Why Body cannot be fetched? Message-ID: Hello, There is one type of requests that Varnish (using version 4.1.0) only works if i put return(pipe) for them on the vcl_recv. If i change it to return(pass) i receive: - VCL_call BACKEND_FETCH - VCL_return fetch - BackendOpen 18 boot.server_api1 10.10.140.206 4050 10.10.140.250 43725 - Timestamp Bereq: 1450191336.453558 0.000820 0.000820 - Timestamp Beresp: 1450191336.461059 0.008320 0.007500 - BerespProtocol HTTP/1.1 - BerespStatus 200 - BerespReason OK - BerespHeader media-type: application/json;v=1 - BerespHeader content-type: application/json; charset=utf-8 - BerespHeader cache-control: no-cache - BerespHeader content-length: 698 - BerespHeader trailer: server-authorization - BerespHeader transfer-encoding: chunked - BerespHeader accept-ranges: bytes - BerespHeader Date: Tue, 15 Dec 2015 14:55:36 GMT - BerespHeader Connection: keep-alive - BackendClose 18 boot.server_api1 - Error Body cannot be fetched - Timestamp Error: 1450191336.461100 0.008361 0.000041 - BerespProtocol HTTP/1.1 - BerespStatus 503 If i do the request directly to the backend, i get this response: root at Reverse:/etc/varnish# curl -v -X 'GET' -H 'Authorization: Hawk id="xxx", ts="1450192889", nonce="J9wb_l", mac="HU+dyFe3PJQx7kEpQBUhDknUGQdqFPIlIoKokxxxxxx="' -H 'host: xxx.mydomain.com' -H 'User-agent: com.muzzley.stag/2.5.7-STAG (Android 4.4.2 ; WIKO GETAWAY)' -H 'content-type: application/json' ' http://10.10.140.206:4050/profiles/d1921c94-a4a3-4cf1-ade8-16a3041238dc' * Hostname was NOT found in DNS cache * Trying 10.10.140.206... * Connected to 10.10.140.206 (10.10.140.206) port 4050 (#0) > GET /profiles/d1921c94-a4a3-4cf1-ade8-16a3041238dc HTTP/1.1 > Accept: */* > Authorization: Hawk id="xxx", ts="1450192889", nonce="J9wb_l", mac="HU+dyFe3PJQx7kEpQBUhDknUGQdqFPIlIoKokxxxxx=" > host: xxx.mydomain.com > User-agent: com.muzzley.stag/2.5.7-STAG (Android 4.4.2 ; WIKO GETAWAY) > content-type: application/json > < HTTP/1.1 200 OK < media-type: application/json;v=1 < content-type: application/json; charset=utf-8 < cache-control: no-cache < content-length: 698 < trailer: server-authorization < transfer-encoding: chunked < accept-ranges: bytes < Date: Tue, 15 Dec 2015 15:21:52 GMT < Connection: keep-alive < * Connection #0 to host 10.10.140.206 left intact {"id":"5669b89707955b9996cxxxxx","uuid":"d1921c94-a4a3-4cf1-ade8-16a3041xxxxx","kind":"device","name":"mymanagertest","provider":"teste","resourceUrl":" http://demo-manager.office.mydomain.com/channels","authorizationUrl":" http://demo-manager.office.mydomain.com/authorization","subscriptionUrl":" http://demo-manager.office.mydomain.com/subscriptions","photoUrl":" http://demo-manager.office.mydomain.com/images/profile2.jpg ","channelPhotoUrl":"http://mydomain.com/teste.jpg ","interface":"baa1766d-d1a2-440b-8ed7-af8296fxxxxx","requiredCapability":"discovery-webview","requiredCapabilityVersion":1,"accessList":[],"auth":{"inbound":{},"outbound":{}},"globalManagerAccess":false,"openOauthInBrowser":false} I notice that the response arrives with the header "transfer-encoding: chunked", and i'm guessing this might be the problem. Do you have any clue what is happening, and how can i use return(pass) for these requests? Thanks Gon?alo Silva -------------- next part -------------- An HTML attachment was scrubbed... URL: From geoff at uplex.de Tue Dec 15 16:29:10 2015 From: geoff at uplex.de (Geoff Simmons) Date: Tue, 15 Dec 2015 17:29:10 +0100 Subject: Why Body cannot be fetched? In-Reply-To: References: Message-ID: <56703FD6.4050700@uplex.de> On 12/15/2015 04:57 PM, Gon?alo Silva wrote: >> > < HTTP/1.1 200 OK < media-type: application/json;v=1 < > content-type: application/json; charset=utf-8 < cache-control: > no-cache < content-length: 698 < trailer: server-authorization < > transfer-encoding: chunked < accept-ranges: bytes < Date: Tue, 15 > Dec 2015 15:21:52 GMT < Connection: keep-alive < * Connection #0 to > host 10.10.140.206 left intact > {"id":"5669b89707955b9996cxxxxx","uuid":"d1921c94-a4a3-4cf1-ade8-16a3041xxxxx","kind":"device","name":"mymanagertest","provider":"teste","resourceUrl":"http://demo-manager.office.mydomain.com/channels","authorizationUrl":"http://demo-manager.office.mydomain.com/authorization","subscriptionUrl":"http://demo-manager.office.mydomain.com/subscriptions","photoUrl":"http://demo-manager.office.mydomain.com/images/profile2.jpg","channelPhotoUrl":"http://mydomain.com/teste.jpg","interface":"baa1766d-d1a2-440b-8ed7-af8296fxxxxx","requiredCapability":"discovery-webview","requiredCapabilityVersion":1,"accessList":[],"auth":{"inbound":{},"outbound":{}},"globalManagerAccess":false,"openOauthInBrowser":false} > > > > I notice that the response arrives with the header > "transfer-encoding: chunked", and i'm guessing this might be the > problem. The response specifies both Content-Length *and* chunked encoding, but it has to be one or the other. In fact, the response body is not chunked-encoded, so it should have been only Content-Length. > ... how can i use return(pass) for these requests? For which requests? If you want pass for every URL that begins with "/profiles/", then: if (req.url ~ "^/profiles/") { return(pass); } But you really should fix the server, it evidently can't make up its mind about Content-Length or chunked encoding, and that can't ever be right. HTH, Geoff -- ** * * UPLEX - Nils Goroll Systemoptimierung Scheffelstra?e 32 22301 Hamburg Tel +49 40 2880 5731 Mob +49 176 636 90917 Fax +49 40 42949753 http://uplex.de -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From goncalo.silva at muzzley.com Tue Dec 15 16:59:23 2015 From: goncalo.silva at muzzley.com (=?UTF-8?Q?Gon=C3=A7alo_Silva?=) Date: Tue, 15 Dec 2015 16:59:23 +0000 Subject: Why Body cannot be fetched? In-Reply-To: <56703FD6.4050700@uplex.de> References: <56703FD6.4050700@uplex.de> Message-ID: I will inform the backend developers about the sending of the two incompatible headers. I already have the "if" filtering these requests, but returning pipe, now i know why return pass was not working Thank you On Tue, Dec 15, 2015 at 4:29 PM, Geoff Simmons wrote: > > The response specifies both Content-Length *and* chunked encoding, but > it has to be one or the other. In fact, the response body is not > chunked-encoded, so it should have been only Content-Length. > > > ... how can i use return(pass) for these requests? > > For which requests? If you want pass for every URL that begins with > "/profiles/", then: > > if (req.url ~ "^/profiles/") { > return(pass); > } > > But you really should fix the server, it evidently can't make up its > mind about Content-Length or chunked encoding, and that can't ever be > right. > > > HTH, > Geoff > -- > ** * * UPLEX - Nils Goroll Systemoptimierung > > Scheffelstra?e 32 > 22301 Hamburg > > Tel +49 40 2880 5731 > Mob +49 176 636 90917 > Fax +49 40 42949753 > > http://uplex.de > > -- Gon?alo Silva Head Security Portugal goncalo.silva at muzzley.com <%2B351%20%2896%29%20343-09-55> Rua do Ouro, n181, 1100-061 Lisboa http://www.muzzley.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlos.abalde at gmail.com Tue Dec 15 17:06:19 2015 From: carlos.abalde at gmail.com (Carlos Abalde) Date: Tue, 15 Dec 2015 18:06:19 +0100 Subject: How to iterate VMOD objects on VCL_EVENT_COLD? Message-ID: Hi all, I'm working in a VMOD that allows creating some objects, each one wrapping the access logic to a database. It looks like, vcl_init { new db1 = redis.db(...); new db2 = redis.db(...); ... } Each object implements a connection pool maintaining multiple opened connections in its internal state. I would like to be able to release those connections when the VCL using the VMOD is not active: - I think that's not doable in 4.0. The only alternative is waiting for the VCL to be discarded and then close connections when the __fini() callback is executed for each instance. - In 4.1 we have the VMOD event function. The VMOD could listen for VCL_EVENT_COLD events and then close established connections. In order to do that the full list of instances would be required. Is it possible to get access to that list somehow? Should it be manually implemented by the VMOD? Is there any plans to propagate VCL temperature events also to object instantiated by a VMOD? Thanks, -- Carlos Abalde From yakov.yarmo at gmail.com Tue Dec 15 18:22:13 2015 From: yakov.yarmo at gmail.com (=?UTF-8?B?15nXoten15Eg15nXqNee15XXnNeV15HXoden15k=?=) Date: Tue, 15 Dec 2015 20:22:13 +0200 Subject: I get these 3 FetchError once in a while when sending POST request Message-ID: This is from varnish log, any help will be appreciated since I have no clue how to debug it 15 BackendXID b 1576349933 > 15 TxRequest b POST > 15 TxURL b /node/add/teacher-profile > 15 TxProtocol b HTTP/1.1 > 15 TxHeader b Host: www.limudnaim.co.il > 15 TxHeader b Content-Length: 5677 > 15 TxHeader b Cache-Control: max-age=0 > 15 TxHeader b Accept: > text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 > 15 TxHeader b Origin: http://www.limudnaim.co.il > 15 TxHeader b Upgrade-Insecure-Requests: 1 > 15 TxHeader b User-Agent: Mozilla/5.0 (Windows NT 10.0) > AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.80 Safari/537.36 > 15 TxHeader b Content-Type: multipart/form-data; > boundary=----WebKitFormBoundaryAGCZTzttPHBR5JWG > 15 TxHeader b Referer: > http://www.limudnaim.co.il/node/add/teacher-profile > 15 TxHeader b Accept-Language: > en-US,en;q=0.8,he;q=0.6,yi;q=0.4,ro;q=0.2 > 15 TxHeader b DNT: 1 > 15 TxHeader b X-UA-Device: pc > 15 TxHeader b X-Full-Uri: www.limudnaim.co.il > %2Fnode%2Fadd%2Fteacher-profile > 15 TxHeader b X-Forwarded-For: 84.228.135.112 > 15 TxHeader b Accept-Encoding: gzip > 15 TxHeader b Cookie: > SESSdac9e0acd970131ed940ca8f937850da=89875f67a6f14ea8e5fbea001b308104 > 15 TxHeader b X-Varnish: 1576349933 > 15 BackendClose b default > 15 BackendOpen b default 82.80.17.31 51388 82.80.17.31 1012 > 15 BackendXID b 1576349933 > 15 TxRequest b POST > 15 TxURL b /node/add/teacher-profile > 15 TxProtocol b HTTP/1.1 > 15 TxHeader b Host: www.limudnaim.co.il > 15 TxHeader b Content-Length: 5677 > 15 TxHeader b Cache-Control: max-age=0 > 15 TxHeader b Accept: > text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 > 15 TxHeader b Origin: http://www.limudnaim.co.il > 15 TxHeader b Upgrade-Insecure-Requests: 1 > 15 TxHeader b User-Agent: Mozilla/5.0 (Windows NT 10.0) > AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.80 Safari/537.36 > 15 TxHeader b Content-Type: multipart/form-data; > boundary=----WebKitFormBoundaryAGCZTzttPHBR5JWG > 15 TxHeader b Referer: > http://www.limudnaim.co.il/node/add/teacher-profile > 15 TxHeader b Accept-Language: > en-US,en;q=0.8,he;q=0.6,yi;q=0.4,ro;q=0.2 > 15 TxHeader b DNT: 1 > 15 TxHeader b X-UA-Device: pc > 15 TxHeader b X-Full-Uri: www.limudnaim.co.il > %2Fnode%2Fadd%2Fteacher-profile > 15 TxHeader b X-Forwarded-For: 84.228.135.112 > 15 TxHeader b Accept-Encoding: gzip > 15 TxHeader b Cookie: > SESSdac9e0acd970131ed940ca8f937850da=89875f67a6f14ea8e5fbea001b308104 > 15 TxHeader b X-Varnish: 1576349933 > 15 BackendClose b default > 27 SessionOpen c 84.228.135.112 63279 :80 > 27 ReqStart c 84.228.135.112 63279 1576349933 > 27 RxRequest c POST > 27 RxURL c /node/add/teacher-profile > 27 RxProtocol c HTTP/1.1 > 27 RxHeader c Host: www.limudnaim.co.il > 27 RxHeader c Connection: keep-alive > 27 RxHeader c Content-Length: 5677 > 27 RxHeader c Cache-Control: max-age=0 > 27 RxHeader c Accept: > text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 > 27 RxHeader c Origin: http://www.limudnaim.co.il > 27 RxHeader c Upgrade-Insecure-Requests: 1 > 27 RxHeader c User-Agent: Mozilla/5.0 (Windows NT 10.0) > AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.80 Safari/537.36 > 27 RxHeader c Content-Type: multipart/form-data; > boundary=----WebKitFormBoundaryAGCZTzttPHBR5JWG > 27 RxHeader c Referer: > http://www.limudnaim.co.il/node/add/teacher-profile > 27 RxHeader c Accept-Encoding: gzip, deflate > 27 RxHeader c Accept-Language: > en-US,en;q=0.8,he;q=0.6,yi;q=0.4,ro;q=0.2 > 27 RxHeader c Cookie: > SESSdac9e0acd970131ed940ca8f937850da=89875f67a6f14ea8e5fbea001b308104; > __utmt=1; __utma=77876233.1831476979.1450191512.1450197638.1450200053.4; > __utmb=77876233.1.10.1450200053; __utmc=77876233; > __utmz=77876233.1450191512.1.1.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided) > 27 RxHeader c DNT: 1 > 27 VCL_call c recv pass > 27 VCL_call c hash > 27 Hash c /node/add/teacher-profile > 27 Hash c www.limudnaim.co.il > 27 VCL_return c hash > 27 VCL_call c pass pass > 27 Backend c 15 default default > 27 FetchError c http first read error: -1 0 (Success) > 27 Backend c 15 default default > 27 FetchError c Resource temporarily unavailable > 27 FetchError c backend write error: 11 (Resource temporarily > unavailable) > 27 VCL_call c error deliver > 27 VCL_call c deliver deliver > 27 TxProtocol c HTTP/1.1 > 27 TxStatus c 302 > 27 TxResponse c Service Unavailable > 27 TxHeader c Server: Varnish > 27 TxHeader c x-status: 503 > 27 TxHeader c Location: > /oops?path=/node/add/teacher-profile&obj_status=503&obj_response=Service > Unavailable&req_xid=1576349933 > 27 TxHeader c Accept-Ranges: bytes > 27 TxHeader c Date: Tue, 15 Dec 2015 17:21:24 GMT > 27 TxHeader c X-Varnish: 1576349933 > 27 TxHeader c Age: 5 > 27 TxHeader c Via: 1.1 varnish > 27 TxHeader c Connection: close > 27 TxHeader c X-Varnish-Cache: MISS > 27 Length c 0 > 27 ReqEnd c 1576349933 1450200079.217353344 > 1450200084.257194042 0.043095827 5.039769888 0.000070810 > 19 BackendXID b 1576349962 > 19 TxRequest b GET > 19 TxURL b > /oops?path=/node/add/teacher-profile&obj_status=503&obj_response=Service%20Unavailable&req_xid=1576349933 > 19 TxProtocol b HTTP/1.1 > 19 TxHeader b Host: www.limudnaim.co.il > 19 TxHeader b Cache-Control: max-age=0 > 19 TxHeader b Accept: > text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 > 19 TxHeader b Upgrade-Insecure-Requests: 1 > 19 TxHeader b User-Agent: Mozilla/5.0 (Windows NT 10.0) > AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.80 Safari/537.36 > 19 TxHeader b Referer: > http://www.limudnaim.co.il/node/add/teacher-profile > 19 TxHeader b Accept-Language: > en-US,en;q=0.8,he;q=0.6,yi;q=0.4,ro;q=0.2 > 19 TxHeader b DNT: 1 > 19 TxHeader b X-UA-Device: pc > 19 TxHeader b X-Full-Uri: www.limudnaim.co.il > %2Foops%3Fpath%3D%2Fnode%2Fadd%2Fteacher-profile%26obj_status%3D503%26obj_response%3DService%2520Unavailable%26req_xid%3D1576349933 > 19 TxHeader b X-Forwarded-For: 84.228.135.112 > 19 TxHeader b Accept-Encoding: gzip > 19 TxHeader b Cookie: > SESSdac9e0acd970131ed940ca8f937850da=89875f67a6f14ea8e5fbea001b308104 > 19 TxHeader b X-Varnish: 1576349962 > 19 RxProtocol b HTTP/1.1 > 19 RxStatus b 200 > 19 RxResponse b OK > 19 RxHeader b Date: Tue, 15 Dec 2015 17:21:27 GMT > 19 RxHeader b Server: Apache > 19 RxHeader b X-Powered-By: PHP/5.3.29 > 19 RxHeader b Last-Modified: Tue, 15 Dec 2015 17:21:27 +0000 > 19 RxHeader b Cache-Control: no-cache, must-revalidate, > post-check=0, pre-check=0 > 19 RxHeader b ETag: "1450200087" > 19 RxHeader b Vary: Accept-Encoding,User-Agent > 19 RxHeader b Content-Encoding: gzip > 19 RxHeader b X-UA-Compatible: IE=edge > 19 RxHeader b Content-Length: 4098 > 19 RxHeader b Content-Type: text/html; charset=utf-8 > 19 Fetch_Body b 4(length) cls 0 mklen 1 > 19 Length b 4098 > 19 BackendReuse b default > 19 BackendXID b 1576349963 > 19 TxRequest b GET > 19 TxURL b > /sites/default/files/advagg_css/css_abbccd52db10310c9aded8ac89cdeeb4_0.css > 19 TxProtocol b HTTP/1.1 > 19 TxHeader b Host: www.limudnaim.co.il > 19 TxHeader b Accept: text/css,*/*;q=0.1 > 19 TxHeader b User-Agent: Mozilla/5.0 (Windows NT 10.0) > AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.80 Safari/537.36 > 19 TxHeader b Referer: > http://www.limudnaim.co.il/oops?path=/node/add/teacher-profile&obj_status=503&obj_response=Service%20Unavailable&req_xid=1576349933 > 19 TxHeader b Accept-Language: > en-US,en;q=0.8,he;q=0.6,yi;q=0.4,ro;q=0.2 > 19 TxHeader b DNT: 1 > 19 TxHeader b X-UA-Device: pc > 19 TxHeader b X-Full-Uri: www.limudnaim.co.il > %2Fsites%2Fdefault%2Ffiles%2Fadvagg_css%2Fcss_abbccd52db10310c9aded8ac89cdeeb4_0.css > 19 TxHeader b X-Forwarded-For: 84.228.135.112, 84.228.135.112 > 19 TxHeader b X-Varnish: 1576349963 > 19 TxHeader b Accept-Encoding: gzip > 19 RxProtocol b HTTP/1.1 > 19 RxStatus b 200 > 19 RxResponse b OK > 19 RxHeader b Date: Tue, 15 Dec 2015 17:21:28 GMT > 19 RxHeader b Server: Apache > 19 RxHeader b Last-Modified: Wed, 20 Jan 1988 04:20:42 GMT > 19 RxHeader b Accept-Ranges: bytes > 19 RxHeader b Content-Length: 124 > 19 RxHeader b Cache-Control: max-age=290304000, no-transform, > public > 19 RxHeader b Expires: Tue, 20 Jan 2037 04:20:42 GMT > 19 RxHeader b X-UA-Compatible: IE=edge > 19 RxHeader b Content-Encoding: gzip > 19 RxHeader b Vary: Accept-Encoding,User-Agent > 19 RxHeader b Content-Type: text/css > 19 Fetch_Body b 4(length) cls 0 mklen 1 > 19 Length b 124 > 19 BackendReuse b default > 17 BackendOpen b default 82.80.17.31 51411 82.80.17.31 1012 > 17 BackendXID b 1576349967 > 17 TxRequest b GET > 17 TxURL b /node/add/teacher-profile > 17 TxProtocol b HTTP/1.1 > 17 TxHeader b Host: www.limudnaim.co.il > 17 TxHeader b Accept: > text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 > 17 TxHeader b Upgrade-Insecure-Requests: 1 > 17 TxHeader b User-Agent: Mozilla/5.0 (Windows NT 10.0) > AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.80 Safari/537.36 > 17 TxHeader b Referer: > http://www.limudnaim.co.il/oops?path=/node/add/teacher-profile&obj_status=503&obj_response=Service%20Unavailable&req_xid=1576349933 > 17 TxHeader b Accept-Language: > en-US,en;q=0.8,he;q=0.6,yi;q=0.4,ro;q=0.2 > 17 TxHeader b DNT: 1 > 17 TxHeader b X-UA-Device: pc > 17 TxHeader b X-Full-Uri: www.limudnaim.co.il > %2Fnode%2Fadd%2Fteacher-profile > 17 TxHeader b X-Forwarded-For: 84.228.135.112 > 17 TxHeader b Accept-Encoding: gzip > 17 TxHeader b Cookie: > SESSdac9e0acd970131ed940ca8f937850da=89875f67a6f14ea8e5fbea001b308104 > 17 TxHeader b X-Varnish: 1576349967 > 17 RxProtocol b HTTP/1.1 > 17 RxStatus b 200 > 17 RxResponse b OK > 17 RxHeader b Date: Tue, 15 Dec 2015 17:21:31 GMT > 17 RxHeader b Server: Apache > 17 RxHeader b X-Powered-By: PHP/5.3.29 > 17 RxHeader b Last-Modified: Tue, 15 Dec 2015 17:21:31 +0000 > 17 RxHeader b Cache-Control: no-cache, must-revalidate, > post-check=0, pre-check=0 > 17 RxHeader b ETag: "1450200091" > 17 RxHeader b Vary: Accept-Encoding,User-Agent > 17 RxHeader b Content-Encoding: gzip > 17 RxHeader b X-UA-Compatible: IE=edge > 17 RxHeader b Content-Length: 31691 > 17 RxHeader b Content-Type: text/html; charset=utf-8 > 17 Fetch_Body b 4(length) cls 0 mklen 1 > 17 Length b 31691 > 17 BackendReuse b default > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From joh.hendriks at gmail.com Tue Dec 15 18:27:04 2015 From: joh.hendriks at gmail.com (Johan Hendriks) Date: Tue, 15 Dec 2015 19:27:04 +0100 Subject: check vcl file for errors, kill's varnishd In-Reply-To: <566E89BC.7060201@Berlinonline.de> References: <566E89BC.7060201@Berlinonline.de> Message-ID: Thanks, I will use that in the future. Op 14 dec. 2015 10:36 schreef "Tom Anheyer" : > Hi, > > maybe you make the same mistake like me. Try: > > varnishd -C - f /usr/local/etc/varnish/default.vcl -n /tmp/ > > because varnishd -C ? deletes a n existing compiled boot config. > > regards > tom > > Am 11.12.2015 um 20:16 schrieb Johan Hendriks: > >> Hello all. >> >> I use varnish 4.1 on a FreeBSD 10.2 machine. All is running fine but when >> I want to make a change to the config and test the changes with the >> command varnishd -C - f /usr/local/etc/varnish/default.vcl, then I need to >> restart varnish because it does not respond anymore. >> I thought maybe this can not be done with the running config so I used a >> different file but it shows the same behavior. >> >> Is it something I am doing wrong, has things changed with 4.1 or is it >> something else. >> >> Thanks for your time. >> >> Regards >> Johan >> >> >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> > > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dridi at varni.sh Tue Dec 15 19:34:49 2015 From: dridi at varni.sh (Dridi Boukelmoune) Date: Tue, 15 Dec 2015 20:34:49 +0100 Subject: How to iterate VMOD objects on VCL_EVENT_COLD? In-Reply-To: References: Message-ID: Hi Carlos, > Each object implements a connection pool maintaining multiple opened connections in its internal state. I would like to be able to release those connections when the VCL using the VMOD is not active: > > - I think that's not doable in 4.0. The only alternative is waiting for the VCL to be discarded and then close connections when the __fini() callback is executed for each instance. In 4.0 you don't go through the __fini functions until the VCL is actually discarded. There's no way[1] to reliably release resources when the VCL is not active, only to release them when the VCL is discarded. > - In 4.1 we have the VMOD event function. The VMOD could listen for VCL_EVENT_COLD events and then close established connections. In order to do that the full list of instances would be required. Is it possible to get access to that list somehow? Should it be manually implemented by the VMOD? Is there any plans to propagate VCL temperature events also to object instantiated by a VMOD? This has been discussed and events are not propagated to VMOD objects so you need to keep track of them and do the event propagation yourself. From what I remember, there are no plans to send events to objects (they still follow an init/fini style) or other data structures such as private pointers (PRIV_VCL). Be aware that VCL temperature is racy in 4.1.0 and that some of the issues have been fixed in master and others are awaiting a review. During the review, we found new holes in the design that need to be discussed: what do we do with VCLs that fail to load or warm up when the child is started? What do we do if the active VCL fails? Cheers, Dridi [1] that I know of From stanhope at gmail.com Wed Dec 16 05:27:00 2015 From: stanhope at gmail.com (Phil Stanhope) Date: Tue, 15 Dec 2015 21:27:00 -0800 Subject: varnish-misc Digest, Vol 117, Issue 14 In-Reply-To: References: Message-ID: <3DBA555F-F4C4-41B7-9417-7CACD41C7F9D@gmail.com> Please share the stats for 150Gbs. And the config scenarios. That would be great to share. And the hardware configs. On metal? If not, how? -phil On Dec 15, 2015, at 03:00, varnish-misc-request at varnish-cache.org wrote: Send varnish-misc mailing list submissions to varnish-misc at varnish-cache.org To subscribe or unsubscribe via the World Wide Web, visit https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc or, via email, send a message with subject or body 'help' to varnish-misc-request at varnish-cache.org You can reach the person managing the list at varnish-misc-owner at varnish-cache.org When replying, please edit your Subject line so it is more specific than "Re: Contents of varnish-misc digest..." Today's Topics: 1. Re: thread_pool_max (Viktor Villafuerte) 2. Re: Why varnish caches as the cache is stopped? (Guillaume Quintard) ---------------------------------------------------------------------- Message: 1 Date: Tue, 15 Dec 2015 09:20:01 +1100 From: Viktor Villafuerte To: Per Buer Cc: Varnish Subject: Re: thread_pool_max Message-ID: <20151214222001.GC1490 at optusnet.com.au> Content-Type: text/plain; charset=utf-8 Hi Per, > On Fri 11 Dec 2015 08:36:06, Per Buer wrote: > On Fri, Dec 11, 2015 at 5:19 AM, Viktor Villafuerte < > viktor.villafuerte at optusnet.com.au> wrote: > >> Ladies, gents >> >> In the Varnish on-line docs (section Tuning) there's a paragraph that >> states: >> >> "We rarely recommend running with more than 5000 threads. If you >> seem to need more than 5000 threads, it?s very likely that there is >> something not quite right about your setup.." > Those are my words. 99% installation have a request rate under 2-3000 > request per second and 5000 threads is more than enough. We've seen quite a > few people adding tens of thousands of threads. I should probably replace > with something like a formula. > > There are no well defined limits wrt capacity but in general people seem to > struggle to push Varnish significantly beyond 15Gbps with real with > traffic. I have not experienced this first hand and nobody has given me > access to a live server with those kind of traffic numbers on it. Also, > nobody get particularly specific wrt what breaks. I think it definitely > should be possible to reach 20Gbps without any issue, but that is more an > opinion than a fact. Would you guys like to have some stats when we reach traffic levels over 15Gbps? If yes then what stats would you like? Currently we're running at 12Gbps very comfortably with 2 pools of 15k threads. > > With synthetic traffic it is no problem reaching 80Gbps. > > -- > *Per Buer* > CTO | Varnish Software AS > Cell: +47 95839117 > We Make Websites Fly! > www.varnish-software.com > -- Regards Viktor Villafuerte Optus Internet Engineering t: +61 2 80825265 ------------------------------ Message: 2 Date: Tue, 15 Dec 2015 09:58:44 +0100 From: Guillaume Quintard To: "georgi.int at gmail.com" Cc: varnish-misc at varnish-cache.org Subject: Re: Why varnish caches as the cache is stopped? Message-ID: Content-Type: text/plain; charset="utf-8" Why wouldn't it be cached? Sorry but I'm not getting the problem here. -- Guillaume Quintard On Mon, Dec 14, 2015 at 6:56 PM, georgi.int at gmail.com wrote: > Yes, but the result were cached and can't understand why. While it's > cached on port 80 where the varnish is, opening the page with :8080 was ok > (on the apache side). Any other suggestions? > > >> On 12/14/2015 07:04 PM, Guillaume Quintard wrote: >> >> Server: Apache >>> 16 TxHeader c Content-Type: text/html; charset=UTF-8 >>> 16 TxHeader c Content-Length: 0 >>> 16 TxHeader c Accept-Ranges: by -------------- next part -------------- An HTML attachment was scrubbed... URL: ------------------------------ _______________________________________________ varnish-misc mailing list varnish-misc at varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc End of varnish-misc Digest, Vol 117, Issue 14 ********************************************* From yakov.yarmo at gmail.com Wed Dec 16 06:06:38 2015 From: yakov.yarmo at gmail.com (=?UTF-8?B?15nXoten15Eg15nXqNee15XXnNeV15HXoden15k=?=) Date: Wed, 16 Dec 2015 08:06:38 +0200 Subject: I get these 3 FetchError once in a while when sending POST request Message-ID: I'm using varnish 3 and apache. This is from varnish log, any help will be appreciated since I have no clue how to debug it 15 BackendXID b 1576349933 15 TxRequest b POST 15 TxURL b /node/add/teacher-profile 15 TxProtocol b HTTP/1.1 15 TxHeader b Host: www.limudnaim.co.il 15 TxHeader b Content-Length: 5677 15 TxHeader b Cache-Control: max-age=0 15 TxHeader b Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 15 TxHeader b Origin: http://www.limudnaim.co.il 15 TxHeader b Upgrade-Insecure-Requests: 1 15 TxHeader b User-Agent: Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.80 Safari/537.36 15 TxHeader b Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryAGCZTzttPHBR5JWG 15 TxHeader b Referer: http://www.limudnaim.co.il/node/add/teacher-profile 15 TxHeader b Accept-Language: en-US,en;q=0.8,he;q=0.6,yi;q=0.4,ro;q=0.2 15 TxHeader b DNT: 1 15 TxHeader b X-UA-Device: pc 15 TxHeader b X-Full-Uri: www.limudnaim.co.il %2Fnode%2Fadd%2Fteacher-profile 15 TxHeader b X-Forwarded-For: 84.228.135.112 15 TxHeader b Accept-Encoding: gzip 15 TxHeader b Cookie: SESSdac9e0acd970131ed940ca8f937850da=89875f67a6f14ea8e5fbea001b308104 15 TxHeader b X-Varnish: 1576349933 15 BackendClose b default 15 BackendOpen b default 82.80.17.31 51388 82.80.17.31 1012 15 BackendXID b 1576349933 15 TxRequest b POST 15 TxURL b /node/add/teacher-profile 15 TxProtocol b HTTP/1.1 15 TxHeader b Host: www.limudnaim.co.il 15 TxHeader b Content-Length: 5677 15 TxHeader b Cache-Control: max-age=0 15 TxHeader b Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 15 TxHeader b Origin: http://www.limudnaim.co.il 15 TxHeader b Upgrade-Insecure-Requests: 1 15 TxHeader b User-Agent: Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.80 Safari/537.36 15 TxHeader b Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryAGCZTzttPHBR5JWG 15 TxHeader b Referer: http://www.limudnaim.co.il/node/add/teacher-profile 15 TxHeader b Accept-Language: en-US,en;q=0.8,he;q=0.6,yi;q=0.4,ro;q=0.2 15 TxHeader b DNT: 1 15 TxHeader b X-UA-Device: pc 15 TxHeader b X-Full-Uri: www.limudnaim.co.il %2Fnode%2Fadd%2Fteacher-profile 15 TxHeader b X-Forwarded-For: 84.228.135.112 15 TxHeader b Accept-Encoding: gzip 15 TxHeader b Cookie: SESSdac9e0acd970131ed940ca8f937850da=89875f67a6f14ea8e5fbea001b308104 15 TxHeader b X-Varnish: 1576349933 15 BackendClose b default 27 SessionOpen c 84.228.135.112 63279 :80 27 ReqStart c 84.228.135.112 63279 1576349933 27 RxRequest c POST 27 RxURL c /node/add/teacher-profile 27 RxProtocol c HTTP/1.1 27 RxHeader c Host: www.limudnaim.co.il 27 RxHeader c Connection: keep-alive 27 RxHeader c Content-Length: 5677 27 RxHeader c Cache-Control: max-age=0 27 RxHeader c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 27 RxHeader c Origin: http://www.limudnaim.co.il 27 RxHeader c Upgrade-Insecure-Requests: 1 27 RxHeader c User-Agent: Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.80 Safari/537.36 27 RxHeader c Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryAGCZTzttPHBR5JWG 27 RxHeader c Referer: http://www.limudnaim.co.il/node/add/teacher-profile 27 RxHeader c Accept-Encoding: gzip, deflate 27 RxHeader c Accept-Language: en-US,en;q=0.8,he;q=0.6,yi;q=0.4,ro;q=0.2 27 RxHeader c Cookie: SESSdac9e0acd970131ed940ca8f937850da=89875f67a6f14ea8e5fbea001b308104; __utmt=1; __utma=77876233.1831476979.1450191512.1450197638.1450200053.4; __utmb=77876233.1.10.1450200053; __utmc=77876233; __utmz=77876233.1450191512.1.1.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided) 27 RxHeader c DNT: 1 27 VCL_call c recv pass 27 VCL_call c hash 27 Hash c /node/add/teacher-profile 27 Hash c www.limudnaim.co.il 27 VCL_return c hash 27 VCL_call c pass pass 27 Backend c 15 default default 27 FetchError c http first read error: -1 0 (Success) 27 Backend c 15 default default 27 FetchError c Resource temporarily unavailable 27 FetchError c backend write error: 11 (Resource temporarily unavailable) 27 VCL_call c error deliver 27 VCL_call c deliver deliver 27 TxProtocol c HTTP/1.1 27 TxStatus c 302 27 TxResponse c Service Unavailable 27 TxHeader c Server: Varnish 27 TxHeader c x-status: 503 27 TxHeader c Location: /oops?path=/node/add/teacher-profile&obj_status=503&obj_response=Service Unavailable&req_xid=1576349933 27 TxHeader c Accept-Ranges: bytes 27 TxHeader c Date: Tue, 15 Dec 2015 17:21:24 GMT 27 TxHeader c X-Varnish: 1576349933 27 TxHeader c Age: 5 27 TxHeader c Via: 1.1 varnish 27 TxHeader c Connection: close 27 TxHeader c X-Varnish-Cache: MISS 27 Length c 0 27 ReqEnd c 1576349933 1450200079.217353344 1450200084.257194042 0.043095827 5.039769888 0.000070810 19 BackendXID b 1576349962 19 TxRequest b GET 19 TxURL b /oops?path=/node/add/teacher-profile&obj_status=503&obj_response=Service%20Unavailable&req_xid=1576349933 19 TxProtocol b HTTP/1.1 19 TxHeader b Host: www.limudnaim.co.il 19 TxHeader b Cache-Control: max-age=0 19 TxHeader b Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 19 TxHeader b Upgrade-Insecure-Requests: 1 19 TxHeader b User-Agent: Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.80 Safari/537.36 19 TxHeader b Referer: http://www.limudnaim.co.il/node/add/teacher-profile 19 TxHeader b Accept-Language: en-US,en;q=0.8,he;q=0.6,yi;q=0.4,ro;q=0.2 19 TxHeader b DNT: 1 19 TxHeader b X-UA-Device: pc 19 TxHeader b X-Full-Uri: www.limudnaim.co.il %2Foops%3Fpath%3D%2Fnode%2Fadd%2Fteacher-profile%26obj_status%3D503%26obj_response%3DService%2520Unavailable%26req_xid%3D1576349933 19 TxHeader b X-Forwarded-For: 84.228.135.112 19 TxHeader b Accept-Encoding: gzip 19 TxHeader b Cookie: SESSdac9e0acd970131ed940ca8f937850da=89875f67a6f14ea8e5fbea001b308104 19 TxHeader b X-Varnish: 1576349962 19 RxProtocol b HTTP/1.1 19 RxStatus b 200 19 RxResponse b OK 19 RxHeader b Date: Tue, 15 Dec 2015 17:21:27 GMT 19 RxHeader b Server: Apache 19 RxHeader b X-Powered-By: PHP/5.3.29 19 RxHeader b Last-Modified: Tue, 15 Dec 2015 17:21:27 +0000 19 RxHeader b Cache-Control: no-cache, must-revalidate, post-check=0, pre-check=0 19 RxHeader b ETag: "1450200087" 19 RxHeader b Vary: Accept-Encoding,User-Agent 19 RxHeader b Content-Encoding: gzip 19 RxHeader b X-UA-Compatible: IE=edge 19 RxHeader b Content-Length: 4098 19 RxHeader b Content-Type: text/html; charset=utf-8 19 Fetch_Body b 4(length) cls 0 mklen 1 19 Length b 4098 19 BackendReuse b default 19 BackendXID b 1576349963 19 TxRequest b GET 19 TxURL b /sites/default/files/advagg_css/css_abbccd52db10310c9aded8ac89cdeeb4_0.css 19 TxProtocol b HTTP/1.1 19 TxHeader b Host: www.limudnaim.co.il 19 TxHeader b Accept: text/css,*/*;q=0.1 19 TxHeader b User-Agent: Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.80 Safari/537.36 19 TxHeader b Referer: http://www.limudnaim.co.il/oops?path=/node/add/teacher-profile&obj_status=503&obj_response=Service%20Unavailable&req_xid=1576349933 19 TxHeader b Accept-Language: en-US,en;q=0.8,he;q=0.6,yi;q=0.4,ro;q=0.2 19 TxHeader b DNT: 1 19 TxHeader b X-UA-Device: pc 19 TxHeader b X-Full-Uri: www.limudnaim.co.il %2Fsites%2Fdefault%2Ffiles%2Fadvagg_css%2Fcss_abbccd52db10310c9aded8ac89cdeeb4_0.css 19 TxHeader b X-Forwarded-For: 84.228.135.112, 84.228.135.112 19 TxHeader b X-Varnish: 1576349963 19 TxHeader b Accept-Encoding: gzip 19 RxProtocol b HTTP/1.1 19 RxStatus b 200 19 RxResponse b OK 19 RxHeader b Date: Tue, 15 Dec 2015 17:21:28 GMT 19 RxHeader b Server: Apache 19 RxHeader b Last-Modified: Wed, 20 Jan 1988 04:20:42 GMT 19 RxHeader b Accept-Ranges: bytes 19 RxHeader b Content-Length: 124 19 RxHeader b Cache-Control: max-age=290304000, no-transform, public 19 RxHeader b Expires: Tue, 20 Jan 2037 04:20:42 GMT 19 RxHeader b X-UA-Compatible: IE=edge 19 RxHeader b Content-Encoding: gzip 19 RxHeader b Vary: Accept-Encoding,User-Agent 19 RxHeader b Content-Type: text/css 19 Fetch_Body b 4(length) cls 0 mklen 1 19 Length b 124 19 BackendReuse b default 17 BackendOpen b default 82.80.17.31 51411 82.80.17.31 1012 17 BackendXID b 1576349967 17 TxRequest b GET 17 TxURL b /node/add/teacher-profile 17 TxProtocol b HTTP/1.1 17 TxHeader b Host: www.limudnaim.co.il 17 TxHeader b Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 17 TxHeader b Upgrade-Insecure-Requests: 1 17 TxHeader b User-Agent: Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.80 Safari/537.36 17 TxHeader b Referer: http://www.limudnaim.co.il/oops?path=/node/add/teacher-profile&obj_status=503&obj_response=Service%20Unavailable&req_xid=1576349933 17 TxHeader b Accept-Language: en-US,en;q=0.8,he;q=0.6,yi;q=0.4,ro;q=0.2 17 TxHeader b DNT: 1 17 TxHeader b X-UA-Device: pc 17 TxHeader b X-Full-Uri: www.limudnaim.co.il %2Fnode%2Fadd%2Fteacher-profile 17 TxHeader b X-Forwarded-For: 84.228.135.112 17 TxHeader b Accept-Encoding: gzip 17 TxHeader b Cookie: SESSdac9e0acd970131ed940ca8f937850da=89875f67a6f14ea8e5fbea001b308104 17 TxHeader b X-Varnish: 1576349967 17 RxProtocol b HTTP/1.1 17 RxStatus b 200 17 RxResponse b OK 17 RxHeader b Date: Tue, 15 Dec 2015 17:21:31 GMT 17 RxHeader b Server: Apache 17 RxHeader b X-Powered-By: PHP/5.3.29 17 RxHeader b Last-Modified: Tue, 15 Dec 2015 17:21:31 +0000 17 RxHeader b Cache-Control: no-cache, must-revalidate, post-check=0, pre-check=0 17 RxHeader b ETag: "1450200091" 17 RxHeader b Vary: Accept-Encoding,User-Agent 17 RxHeader b Content-Encoding: gzip 17 RxHeader b X-UA-Compatible: IE=edge 17 RxHeader b Content-Length: 31691 17 RxHeader b Content-Type: text/html; charset=utf-8 17 Fetch_Body b 4(length) cls 0 mklen 1 17 Length b 31691 17 BackendReuse b default -------------- next part -------------- An HTML attachment was scrubbed... URL: From perbu at varnish-software.com Wed Dec 16 07:42:49 2015 From: perbu at varnish-software.com (Per Buer) Date: Wed, 16 Dec 2015 08:42:49 +0100 Subject: thread_pool_max In-Reply-To: <20151214222001.GC1490@optusnet.com.au> References: <20151211041940.GB1490@optusnet.com.au> <20151214222001.GC1490@optusnet.com.au> Message-ID: On Mon, Dec 14, 2015 at 11:20 PM, Viktor Villafuerte < viktor.villafuerte at optusnet.com.au> wrote: Would you guys like to have some stats when we reach traffic levels over > 15Gbps? If yes then what stats would you like? > > Currently we're running at 12Gbps very comfortably with 2 pools of 15k > threads. > varnishstat -1, hardware config and list of kernel tuneable changed would be awesome. And if you encounter it would be great to hear what breaks. So far the best I've heard is "varnish starts to act funny" and I have no idea what that means. -- *Per Buer* CTO | Varnish Software AS Cell: +47 95839117 We Make Websites Fly! www.varnish-software.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From yakov.yarmo at gmail.com Wed Dec 16 08:05:06 2015 From: yakov.yarmo at gmail.com (=?UTF-8?B?15nXoten15Eg15nXqNee15XXnNeV15HXoden15k=?=) Date: Wed, 16 Dec 2015 10:05:06 +0200 Subject: varnish 3 and FetchError http first read error: -1 0 (Success) Message-ID: On some (10%) post requsest i get status 503 (Service Unavailable). I really can't figure out why that is happening. And always when it happen I get additional FetchErrors: Resource temporarily unavailable backend write error: 11 (Resource temporarily unavailable) How can I debug it, or what is the cause? -------------- next part -------------- An HTML attachment was scrubbed... URL: From Tom.Anheyer at Berlinonline.de Wed Dec 16 12:39:28 2015 From: Tom.Anheyer at Berlinonline.de (Tom Anheyer) Date: Wed, 16 Dec 2015 13:39:28 +0100 Subject: How to use bereq.backend Message-ID: <56715B80.60802@Berlinonline.de> Hi, The following lines do not work in vcl_backend_error() bereq.backend == directorXYZ.backend() -> Backend not found: 'directorXYZ.backend' bereq.backend == directorXYZ -> Backend not found: 'directorTip' bereq.backend == "directorXYZ" -> Expected ID got '"directorXYZ"' Whats the right syntax? tom -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4266 bytes Desc: S/MIME Cryptographic Signature URL: From viktor.villafuerte at optusnet.com.au Wed Dec 16 22:12:31 2015 From: viktor.villafuerte at optusnet.com.au (Viktor Villafuerte) Date: Thu, 17 Dec 2015 09:12:31 +1100 Subject: varnish 3 and FetchError http first read error: -1 0 (Success) In-Reply-To: References: Message-ID: <20151216221231.GD1490@optusnet.com.au> Hi, this post https://www.varnish-cache.org/trac/ticket/968 talks about the exact problem and points to problems with the backend used by Varnish not Varnish itself. This could/should be worth investigating. v On Wed 16 Dec 2015 10:05:06, ???? ?????????? wrote: > On some (10%) post requsest i get status 503 (Service Unavailable). > I really can't figure out why that is happening. > And always when it happen I get additional FetchErrors: > > Resource temporarily unavailable > backend write error: 11 (Resource temporarily unavailable) > > How can I debug it, or what is the cause? > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc -- Regards Viktor Villafuerte Optus Internet Engineering t: +61 2 80825265 From yakov.yarmo at gmail.com Thu Dec 17 07:48:32 2015 From: yakov.yarmo at gmail.com (=?UTF-8?B?15nXoten15Eg15nXqNee15XXnNeV15HXoden15k=?=) Date: Thu, 17 Dec 2015 09:48:32 +0200 Subject: varnish 3 and FetchError http first read error: -1 0 (Success) In-Reply-To: <20151216221231.GD1490@optusnet.com.au> References: <20151216221231.GD1490@optusnet.com.au> Message-ID: Will do, thank you very much. On Thu, Dec 17, 2015 at 12:12 AM, Viktor Villafuerte < viktor.villafuerte at optusnet.com.au> wrote: > Hi, > > this post https://www.varnish-cache.org/trac/ticket/968 talks about the > exact problem and points to problems with the backend used by Varnish > not Varnish itself. > > This could/should be worth investigating. > > > v > > > On Wed 16 Dec 2015 10:05:06, ???? ?????????? wrote: > > On some (10%) post requsest i get status 503 (Service Unavailable). > > I really can't figure out why that is happening. > > And always when it happen I get additional FetchErrors: > > > > Resource temporarily unavailable > > backend write error: 11 (Resource temporarily unavailable) > > > > How can I debug it, or what is the cause? > > > _______________________________________________ > > varnish-misc mailing list > > varnish-misc at varnish-cache.org > > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > > -- > Regards > > Viktor Villafuerte > Optus Internet Engineering > t: +61 2 80825265 > -- *???? ??????????,???? ??? ????? ????www.limudnaim.co.il ??? ????? ?????? ??????* -------------- next part -------------- An HTML attachment was scrubbed... URL: From info at elsitar.com Thu Dec 17 14:49:49 2015 From: info at elsitar.com (Xavier Cardil Coll) Date: Thu, 17 Dec 2015 15:49:49 +0100 Subject: POST going through but returning cached page ? Message-ID: Hi, I am using this config for a wordpress-woocommerce with mod_pagespeed, but something weird happens on when I try to log in or register a user. I have made sure the /my-account/ is not cached and that the POST request to send the data at login / register is not cached, actually it works. Users login and register, but after one of those actions, the server returns the same form page with the form being empty, while it should return the "my account" page with the data of the user logged in. Can you please shed some light ? Here is the pasta of my config : http://pastebin.com/3NnW0Sbp Thanks ! -- ELSITAR -------------- next part -------------- An HTML attachment was scrubbed... URL: From tecnologia at unnoba.edu.ar Thu Dec 17 16:21:51 2015 From: tecnologia at unnoba.edu.ar (=?UTF-8?Q?Tecnolog=c3=ada_UNNOBA?=) Date: Thu, 17 Dec 2015 13:21:51 -0300 Subject: Varnish does not compile VCL after libc6 update Message-ID: <5672E11F.10100@unnoba.edu.ar> Hello! This morning I run "apt-get dist-upgrade" on Debian Stretch. [...] 2015-12-17 09:38:11 upgrade libc-dev-bin:amd64 2.19-22 2.21-4 2015-12-17 09:38:11 upgrade libc6-dev:amd64 2.19-22 2.21-4 2015-12-17 09:38:16 upgrade libc6:amd64 2.19-22 2.21-4 2015-12-17 09:38:25 upgrade libc-bin:amd64 2.19-22 2.21-4 2015-12-17 09:38:25 upgrade linux-libc-dev:amd64 4.2.6-1 4.2.6-3 [...] After that, when I restarted varnish service, I get this error: dic 17 11:50:30 cersey varnishd[4408]: Error: dic 17 11:50:30 cersey varnishd[4408]: Message from C-compiler: dic 17 11:50:30 cersey varnishd[4408]: /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crti.o: no se reconoce la direcci?n de reubicaci?n (0x2a) en la secci?n `.init' dic 17 11:50:30 cersey varnishd[4408]: /usr/bin/ld: fall? el enlace final: Valor err?neo dic 17 11:50:30 cersey varnishd[4408]: collect2: error: ld returned 1 exit status dic 17 11:50:30 cersey varnishd[4408]: Running C-compiler failed, exited with 1 dic 17 11:50:30 cersey varnishd[4408]: VCL compilation failed I have installed: varnish 4.1.0-2 libc6:amd64 2.21-4 Any suggestions? Anyone got something simmilar? Thanks in advance. From mark.staudinger at nyi.net Thu Dec 17 16:22:32 2015 From: mark.staudinger at nyi.net (Mark S.) Date: Thu, 17 Dec 2015 11:22:32 -0500 Subject: POST going through but returning cached page ? In-Reply-To: References: Message-ID: <5672E148.3090004@nyi.net> On 12/17/15 9:49 AM, Xavier Cardil Coll wrote: > Hi, I am using this config for a wordpress-woocommerce with > mod_pagespeed, but something weird happens on when I try to log in or > register a user. I have made sure the /my-account/ is not cached and > that the POST request to send the data at login / register is not > cached, actually it works. Users login and register, but after one of > those actions, the server returns the same form page with the form > being empty, while it should return the "my account" page with the > data of the user logged in. Can you please shed some light ? > > Here is the pasta of my config : http://pastebin.com/3NnW0Sbp > > Thanks ! > > -- Hi Xavier, Two things jump out at me. 1) You have several matching rules with extraneous '*' characters. For example, your vcl line 91 if(req.url ~ "^/(cart|my-account/*|checkout|addons|logout|lost-password|product/*)"){ Since we are using the "~" match, this might be better represented by if ( req.url ~ "^(cart|my-account/|checkout|addons|logout|lost-password|product/)" ) { This isn't a problem, but it may make the usage clearer. 2) Between lines 67 and 130 you have many test cases on when to pass. Following that, you have a section that removes cookies, including WordPress cookies, with no conditions. But I do not see any VCL above line 130 that issues a pass based on the presence of a WordPress cookie. It would help to see the varnishlog of what happens after the POST/login and the cached page, but one likely possibility is a page load that you aren't matching in lines 67-130 that is arriving without the proper cookies. I would expect to see something like this before you start un-setting cookies around line 140 if (req.url ~ "^[^?]*\.(mp[34]|rar|tar|tgz|wav|zip|bz2|xz|7z|avi|mov|ogm|mpe?g|mk[av])(\?.*)?$") { unset req.http.Cookie; return (hash); } if ( req.http.Cookie ~ "wordpress_" ) { return (pass); } This type of action might allow you to short-cut much of the "COOKIE MADNESS" section by wrapping in a " if req.http.Cookie " check. Cheers, -=Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From lucas at silk.co Mon Dec 21 14:44:41 2015 From: lucas at silk.co (Lucas Luitjes) Date: Mon, 21 Dec 2015 15:44:41 +0100 Subject: suspicious memory usage with 3.0.5 Message-ID: <56781059.6030707@silk.co> Hi, We're handling only 2-8 req/s and are using file based storage. Yet somehow varnish is using 1-2GB memory. RSS, not VM. I looked at the usual suspects, but the Transient storage is barely used, and there are only 33000 objects in cache so it can't be the object overhead either. I've read through two years worth on this list but wasn't able to find other possible causes. Can anybody point me in the right direction? I'm starting varnish with the following options: DAEMON_OPTS="-a :80 \ -T localhost:6082 \ -f /etc/varnish/default.vcl \ -S /etc/varnish/secret \ -p sess_timeout=30s \ -p shm_reclen=2048 \ -s file,/mnt/varnish-storage/varnish.bin,15000m\ -w 100,1000,120" ps output: USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 19382 0.0 0.1 126700 5200 ? Ss Dec13 0:15 /usr/sbin/varnishd -P /var/run/varnishd.pid -a :80 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -p sess_timeout=30s -p shm_reclen=2048 -s nobody 19386 0.4 31.5 4081716 1215100 ? Sl Dec13 46:45 /usr/sbin/varnishd -P /var/run/varnishd.pid -a :80 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -p sess_timeout=30s -p shm_reclen=2048 -s Param.show: varnish> param.show 200 acceptor_sleep_decay 0.900000 [] acceptor_sleep_incr 0.001000 [s] acceptor_sleep_max 0.050000 [s] auto_restart on [bool] ban_dups on [bool] ban_lurker_sleep 0.010000 [s] between_bytes_timeout 60.000000 [s] cc_command "exec gcc -std=gnu99 -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -pthread -fpic -shared -Wl,-x -o %o %s" cli_buffer 8192 [bytes] cli_timeout 10 [seconds] clock_skew 10 [s] connect_timeout 0.700000 [s] critbit_cooloff 180.000000 [s] default_grace 10.000000 [seconds] default_keep 0.000000 [seconds] default_ttl 120.000000 [seconds] diag_bitmap 0x0 [bitmap] esi_syntax 0 [bitmap] expiry_sleep 1.000000 [seconds] fetch_chunksize 128 [kilobytes] fetch_maxchunksize 262144 [kilobytes] first_byte_timeout 60.000000 [s] group nogroup (65534) gzip_level 6 [] gzip_memlevel 8 [] gzip_stack_buffer 32768 [Bytes] gzip_tmp_space 0 [] gzip_window 15 [] http_gzip_support on [bool] http_max_hdr 64 [header lines] http_range_support on [bool] http_req_hdr_len 8192 [bytes] http_req_size 32768 [bytes] http_resp_hdr_len 8192 [bytes] http_resp_size 32768 [bytes] idle_send_timeout 60 [seconds] listen_address :80 listen_depth 1024 [connections] log_hashstring on [bool] log_local_address off [bool] lru_interval 2 [seconds] max_esi_depth 5 [levels] max_restarts 4 [restarts] nuke_limit 50 [allocations] pcre_match_limit 10000 [] pcre_match_limit_recursion 10000 [] ping_interval 3 [seconds] pipe_timeout 60 [seconds] prefer_ipv6 off [bool] queue_max 100 [%] rush_exponent 3 [requests per request] saintmode_threshold 10 [objects] send_timeout 600 [seconds] sess_timeout 30 [seconds] sess_workspace 65536 [bytes] session_linger 50 [ms] session_max 100000 [sessions] shm_reclen 2048 [bytes] shm_workspace 8192 [bytes] shortlived 10.000000 [s] syslog_cli_traffic on [bool] thread_pool_add_delay 2 [milliseconds] thread_pool_add_threshold 2 [requests] thread_pool_fail_delay 200 [milliseconds] thread_pool_max 1000 [threads] thread_pool_min 100 [threads] thread_pool_purge_delay 1000 [milliseconds] thread_pool_stack unlimited [bytes] thread_pool_timeout 120 [seconds] thread_pool_workspace 65536 [bytes] thread_pools 2 [pools] thread_stats_rate 10 [requests] user nobody (65534) vcc_err_unref on [bool] vcl_dir /etc/varnish vcl_trace off [bool] vmod_dir /usr/lib/x86_64-linux-gnu/varnish/vmods waiter default (epoll, poll) Varnishstat: varnishstat -1 client_conn 1137599 1.63 Client connections accepted client_drop 0 0.00 Connection dropped, no sess/wrk client_req 2032210 2.91 Client requests received cache_hit 341943 0.49 Cache hits cache_hitpass 1842 0.00 Cache hits for pass cache_miss 708115 1.01 Cache misses backend_conn 89175 0.13 Backend conn. success backend_unhealthy 0 0.00 Backend conn. not attempted backend_busy 0 0.00 Backend conn. too many backend_fail 560 0.00 Backend conn. failures backend_reuse 1578521 2.26 Backend conn. reuses backend_toolate 13907 0.02 Backend conn. was closed backend_recycle 1592791 2.28 Backend conn. recycles backend_retry 3123 0.00 Backend conn. retry fetch_head 11776 0.02 Fetch head fetch_length 198636 0.28 Fetch with Length fetch_chunked 1387593 1.99 Fetch chunked fetch_eof 0 0.00 Fetch EOF fetch_bad 0 0.00 Fetch had bad headers fetch_close 371 0.00 Fetch wanted close fetch_oldhttp 0 0.00 Fetch pre HTTP/1.1 closed fetch_zero 0 0.00 Fetch zero len fetch_failed 23 0.00 Fetch failed fetch_1xx 0 0.00 Fetch no body (1xx) fetch_204 0 0.00 Fetch no body (204) fetch_304 854 0.00 Fetch no body (304) n_sess_mem 199 . N struct sess_mem n_sess 22 . N struct sess n_object 32945 . N struct object n_vampireobject 0 . N unresurrected objects n_objectcore 33006 . N struct objectcore n_objecthead 33434 . N struct objecthead n_waitinglist 175 . N struct waitinglist n_vbc 10 . N struct vbc n_wrk 200 . N worker threads n_wrk_create 200 0.00 N worker threads created n_wrk_failed 0 0.00 N worker threads not created n_wrk_max 0 0.00 N worker threads limited n_wrk_lqueue 0 0.00 work request queue length n_wrk_queued 2 0.00 N queued work requests n_wrk_drop 0 0.00 N dropped work requests n_backend 4 . N backends n_expired 274186 . N expired objects n_lru_nuked 394697 . N LRU nuked objects n_lru_moved 286567 . N LRU moved objects losthdr 8 0.00 HTTP header overflows n_objsendfile 0 0.00 Objects sent with sendfile n_objwrite 1993358 2.86 Objects sent with write n_objoverflow 0 0.00 Objects overflowing workspace s_sess 1137600 1.63 Total Sessions s_req 2032210 2.91 Total Requests s_pipe 62579 0.09 Total pipe s_pass 894827 1.28 Total pass s_fetch 1594496 2.28 Total fetch s_hdrbytes 944136821 1352.28 Total header bytes s_bodybytes 50610526058 72489.02 Total body bytes sess_closed 230365 0.33 Session Closed sess_pipeline 106 0.00 Session Pipeline sess_readahead 0 0.00 Session Read Ahead sess_linger 1857635 2.66 Session Linger sess_herd 1831395 2.62 Session herd shm_records 179189892 256.65 SHM records shm_writes 13094383 18.75 SHM writes shm_flushes 4218 0.01 SHM flushes due to overflow shm_cont 12594 0.02 SHM MTX contention shm_cycles 104 0.00 SHM cycles through buffer sms_nreq 11871 0.02 SMS allocator requests sms_nobj 0 . SMS outstanding allocations sms_nbytes 0 . SMS outstanding bytes sms_balloc 10070143 . SMS bytes allocated sms_bfree 10070143 . SMS bytes freed backend_req 1601778 2.29 Backend requests made n_vcl 1 0.00 N vcl total n_vcl_avail 1 0.00 N vcl available n_vcl_discard 0 0.00 N vcl discarded n_ban 45 . N total active bans n_ban_gone 42 . N total gone bans n_ban_add 173359 0.25 N new bans added n_ban_retire 173314 0.25 N old bans deleted n_ban_obj_test 65906808 94.40 N objects tested n_ban_re_test 233394372 334.29 N regexps tested against n_ban_dups 168535 0.24 N duplicate bans removed hcb_nolock 1051594 1.51 HCB Lookups without lock hcb_lock 705667 1.01 HCB Lookups with lock hcb_insert 705663 1.01 HCB Inserts esi_errors 0 0.00 ESI parse errors (unlock) esi_warnings 0 0.00 ESI parse warnings (unlock) accept_fail 0 0.00 Accept failures client_drop_late 0 0.00 Connection dropped late uptime 698182 1.00 Client uptime dir_dns_lookups 0 0.00 DNS director lookups dir_dns_failed 0 0.00 DNS director failed lookups dir_dns_hit 0 0.00 DNS director cached lookups hit dir_dns_cache_full 0 0.00 DNS director full dnscache vmods 5 . Loaded VMODs n_gzip 0 0.00 Gzip operations n_gunzip 666967 0.96 Gunzip operations sess_pipe_overflow 0 . Dropped sessions due to session pipe overflow LCK.sms.creat 1 0.00 Created locks LCK.sms.destroy 0 0.00 Destroyed locks LCK.sms.locks 35613 0.05 Lock Operations LCK.sms.colls 0 0.00 Collisions LCK.smp.creat 0 0.00 Created locks LCK.smp.destroy 0 0.00 Destroyed locks LCK.smp.locks 0 0.00 Lock Operations LCK.smp.colls 0 0.00 Collisions LCK.sma.creat 1 0.00 Created locks LCK.sma.destroy 0 0.00 Destroyed locks LCK.sma.locks 4808562 6.89 Lock Operations LCK.sma.colls 0 0.00 Collisions LCK.smf.creat 1 0.00 Created locks LCK.smf.destroy 0 0.00 Destroyed locks LCK.smf.locks 3799704 5.44 Lock Operations LCK.smf.colls 0 0.00 Collisions LCK.hsl.creat 0 0.00 Created locks LCK.hsl.destroy 0 0.00 Destroyed locks LCK.hsl.locks 0 0.00 Lock Operations LCK.hsl.colls 0 0.00 Collisions LCK.hcb.creat 1 0.00 Created locks LCK.hcb.destroy 0 0.00 Destroyed locks LCK.hcb.locks 1382264 1.98 Lock Operations LCK.hcb.colls 0 0.00 Collisions LCK.hcl.creat 0 0.00 Created locks LCK.hcl.destroy 0 0.00 Destroyed locks LCK.hcl.locks 0 0.00 Lock Operations LCK.hcl.colls 0 0.00 Collisions LCK.vcl.creat 1 0.00 Created locks LCK.vcl.destroy 0 0.00 Destroyed locks LCK.vcl.locks 2387 0.00 Lock Operations LCK.vcl.colls 0 0.00 Collisions LCK.stat.creat 1 0.00 Created locks LCK.stat.destroy 0 0.00 Destroyed locks LCK.stat.locks 1137777 1.63 Lock Operations LCK.stat.colls 0 0.00 Collisions LCK.sessmem.creat 1 0.00 Created locks LCK.sessmem.destroy 0 0.00 Destroyed locks LCK.sessmem.locks 1146538 1.64 Lock Operations LCK.sessmem.colls 0 0.00 Collisions LCK.wstat.creat 1 0.00 Created locks LCK.wstat.destroy 0 0.00 Destroyed locks LCK.wstat.locks 713721 1.02 Lock Operations LCK.wstat.colls 0 0.00 Collisions LCK.herder.creat 1 0.00 Created locks LCK.herder.destroy 0 0.00 Destroyed locks LCK.herder.locks 1 0.00 Lock Operations LCK.herder.colls 0 0.00 Collisions LCK.wq.creat 2 0.00 Created locks LCK.wq.destroy 0 0.00 Destroyed locks LCK.wq.locks 5522531 7.91 Lock Operations LCK.wq.colls 0 0.00 Collisions LCK.objhdr.creat 705475 1.01 Created locks LCK.objhdr.destroy 672290 0.96 Destroyed locks LCK.objhdr.locks 73461950 105.22 Lock Operations LCK.objhdr.colls 0 0.00 Collisions LCK.exp.creat 1 0.00 Created locks LCK.exp.destroy 0 0.00 Destroyed locks LCK.exp.locks 2312896 3.31 Lock Operations LCK.exp.colls 0 0.00 Collisions LCK.lru.creat 2 0.00 Created locks LCK.lru.destroy 0 0.00 Destroyed locks LCK.lru.locks 1340625 1.92 Lock Operations LCK.lru.colls 0 0.00 Collisions LCK.cli.creat 1 0.00 Created locks LCK.cli.destroy 0 0.00 Destroyed locks LCK.cli.locks 232608 0.33 Lock Operations LCK.cli.colls 0 0.00 Collisions LCK.ban.creat 1 0.00 Created locks LCK.ban.destroy 0 0.00 Destroyed locks LCK.ban.locks 135656520 194.30 Lock Operations LCK.ban.colls 0 0.00 Collisions LCK.vbp.creat 1 0.00 Created locks LCK.vbp.destroy 0 0.00 Destroyed locks LCK.vbp.locks 0 0.00 Lock Operations LCK.vbp.colls 0 0.00 Collisions LCK.vbe.creat 1 0.00 Created locks LCK.vbe.destroy 0 0.00 Destroyed locks LCK.vbe.locks 179544 0.26 Lock Operations LCK.vbe.colls 0 0.00 Collisions LCK.backend.creat 4 0.00 Created locks LCK.backend.destroy 0 0.00 Destroyed locks LCK.backend.locks 3454891 4.95 Lock Operations LCK.backend.colls 0 0.00 Collisions SMF.s0.c_req 1797552 2.57 Allocator requests SMF.s0.c_fail 395043 0.57 Allocator failures SMF.s0.c_bytes 95012720640 136085.89 Bytes allocated SMF.s0.c_freed 94071521280 134737.82 Bytes freed SMF.s0.g_alloc 67056 . Allocations outstanding SMF.s0.g_bytes 941199360 . Bytes outstanding SMF.s0.g_space 840527872 . Bytes available SMF.s0.g_smf 76469 . N struct smf SMF.s0.g_smf_frag 9402 . N small free smf SMF.s0.g_smf_large 11 . N large free smf SMA.Transient.c_req 1954222 2.80 Allocator requests SMA.Transient.c_fail 0 0.00 Allocator failures SMA.Transient.c_bytes 135146176063 193568.69 Bytes allocated SMA.Transient.c_freed 135146173687 193568.69 Bytes freed SMA.Transient.g_alloc 2 . Allocations outstanding SMA.Transient.g_bytes 2376 . Bytes outstanding SMA.Transient.g_space 0 . Bytes available VBE.prerender(10.0.0.194,,12000).vcls 1 . VCL references VBE.prerender(10.0.0.194,,12000).happy 0 . Happy health probes VBE.haproxy(127.0.0.1,,8080).vcls 1 . VCL references VBE.haproxy(127.0.0.1,,8080).happy 0 . Happy health probes VBE.tuna(127.0.0.1,,5555).vcls 1 . VCL references VBE.tuna(127.0.0.1,,5555).happy 0 . Happy health probes VBE.apache(127.0.0.1,,81).vcls 1 . VCL references VBE.apache(127.0.0.1,,81).happy 0 . Happy health probes Thanks! Lucas Luitjes From info at elsitar.com Wed Dec 23 11:39:20 2015 From: info at elsitar.com (Xavier Cardil Coll) Date: Wed, 23 Dec 2015 12:39:20 +0100 Subject: Bypass cache when add to cart ? Message-ID: On Nginx cache, you can do this to avoid keep on caching when you add something to the cart in Woocommerce : Skip cache for WooCommerce query string if ( $arg_add-to-cart != "" ) { set $skip_cache 1; } Skip cache when WooCommerce cart is not empty if ( $cookie_woocommerce_items_in_cart != "0" ) { set $skip_cache 1; } How can you do this in Varnish ? With this config, wich was made specifically for Wordpress + Woocommerce, when you add something to cart, cache is not disable store wide so you get a mix of cached and non cache pages, cart gets emptied : http://pastebin.com/TuYeMtUZ -- ELSITAR -------------- next part -------------- An HTML attachment was scrubbed... URL: From info at elsitar.com Wed Dec 23 12:15:06 2015 From: info at elsitar.com (Xavier Cardil Coll) Date: Wed, 23 Dec 2015 13:15:06 +0100 Subject: Bypass cache when add to cart ? In-Reply-To: References: Message-ID: Also, how to write a function to bypass cache entirely when a user logs in ? 2015-12-23 12:39 GMT+01:00 Xavier Cardil Coll : > On Nginx cache, you can do this to avoid keep on caching when you add > something to the cart in Woocommerce : > > Skip cache for WooCommerce query string > > if ( $arg_add-to-cart != "" ) { > set $skip_cache 1; > } > > Skip cache when WooCommerce cart is not empty > > if ( $cookie_woocommerce_items_in_cart != "0" ) { > set $skip_cache 1; > } > > How can you do this in Varnish ? With this config, wich was made > specifically for Wordpress + Woocommerce, when you add something to cart, > cache is not disable store wide so you get a mix of cached and non cache > pages, cart gets emptied : http://pastebin.com/TuYeMtUZ > > -- > > ELSITAR > > > -- ELSITAR -------------- next part -------------- An HTML attachment was scrubbed... URL: From info at elsitar.com Wed Dec 23 18:56:59 2015 From: info at elsitar.com (Xavier Cardil Coll) Date: Wed, 23 Dec 2015 19:56:59 +0100 Subject: Bypass cache when add to cart ? In-Reply-To: References: Message-ID: No one knows then ? Not even a hint or something ? 2015-12-23 13:15 GMT+01:00 Xavier Cardil Coll : > Also, how to write a function to bypass cache entirely when a user logs in > ? > > 2015-12-23 12:39 GMT+01:00 Xavier Cardil Coll : > >> On Nginx cache, you can do this to avoid keep on caching when you add >> something to the cart in Woocommerce : >> >> Skip cache for WooCommerce query string >> >> if ( $arg_add-to-cart != "" ) { >> set $skip_cache 1; >> } >> >> Skip cache when WooCommerce cart is not empty >> >> if ( $cookie_woocommerce_items_in_cart != "0" ) { >> set $skip_cache 1; >> } >> >> How can you do this in Varnish ? With this config, wich was made >> specifically for Wordpress + Woocommerce, when you add something to cart, >> cache is not disable store wide so you get a mix of cached and non cache >> pages, cart gets emptied : http://pastebin.com/TuYeMtUZ >> >> -- >> >> ELSITAR >> >> >> > > > -- > > ELSITAR > > > -- ELSITAR -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Wed Dec 23 20:05:18 2015 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Wed, 23 Dec 2015 21:05:18 +0100 Subject: Bypass cache when add to cart ? In-Reply-To: References: Message-ID: On Dec 23, 2015 20:51, "Xavier Cardil Coll" wrote: > > No one knows then ? Not even a hint or something ? > > 2015-12-23 13:15 GMT+01:00 Xavier Cardil Coll : >> >> Also, how to write a function to bypass cache entirely when a user logs in ? >> >> 2015-12-23 12:39 GMT+01:00 Xavier Cardil Coll : >>> >>> On Nginx cache, you can do this to avoid keep on caching when you add something to the cart in Woocommerce : >>> >>> Skip cache for WooCommerce query string >>> >>> if ( $arg_add-to-cart != "" ) { >>> set $skip_cache 1; >>> } >>> >>> Skip cache when WooCommerce cart is not empty >>> >>> if ( $cookie_woocommerce_items_in_cart != "0" ) { >>> set $skip_cache 1; >>> } >>> >>> How can you do this in Varnish ? With this config, wich was made specifically for Wordpress + Woocommerce, when you add something to cart, cache is not disable store wide so you get a mix of cached and non cache pages, cart gets emptied : http://pastebin.com/TuYeMtUZ >>> >>> >>> -- >>> >>> ELSITAR >>> >>> >> >> >> >> -- >> >> ELSITAR >> >> > > > > -- > > ELSITAR > > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc Chill out, most of the people are in vacation :-) So, in varnish, by default, if the request contains a cookie, the cache is bypassed. But you can also force a pass, for example if the request contains a "foo-bar" header like so : sub vcl_recv { if (req.http.foo-bar) { return (pass); } } For your needs, look at the query string and cookie vmods, it will allow to really fine-tune your workflow. And should you need more help, we are here :-) -------------- next part -------------- An HTML attachment was scrubbed... URL: From perbu at varnish-software.com Wed Dec 23 20:56:44 2015 From: perbu at varnish-software.com (Per Buer) Date: Wed, 23 Dec 2015 21:56:44 +0100 Subject: Bypass cache when add to cart ? In-Reply-To: References: Message-ID: Hi, Sure. Here is a hint. Read through the the relevant parts of the Varnish Book or varnish-cache.org's "Users Guide". What you're looking for is "passing" certain requests. You'll control this in vcl_recv and the part that tests on the query string will look something like this: if (req.url ~ "\?.*foo=bar") { return(pass); } You might want to have a look at the cookie vmod, especially if you struggle with regular expressions. Per. On Wed, Dec 23, 2015 at 7:56 PM, Xavier Cardil Coll wrote: > No one knows then ? Not even a hint or something ? > > 2015-12-23 13:15 GMT+01:00 Xavier Cardil Coll : > >> Also, how to write a function to bypass cache entirely when a user logs >> in ? >> >> 2015-12-23 12:39 GMT+01:00 Xavier Cardil Coll : >> >>> On Nginx cache, you can do this to avoid keep on caching when you add >>> something to the cart in Woocommerce : >>> >>> Skip cache for WooCommerce query string >>> >>> if ( $arg_add-to-cart != "" ) { >>> set $skip_cache 1; >>> } >>> >>> Skip cache when WooCommerce cart is not empty >>> >>> if ( $cookie_woocommerce_items_in_cart != "0" ) { >>> set $skip_cache 1; >>> } >>> >>> How can you do this in Varnish ? With this config, wich was made >>> specifically for Wordpress + Woocommerce, when you add something to cart, >>> cache is not disable store wide so you get a mix of cached and non cache >>> pages, cart gets emptied : http://pastebin.com/TuYeMtUZ >>> >>> -- >>> >>> ELSITAR >>> >>> >>> >> >> >> -- >> >> ELSITAR >> >> >> > > > -- > > ELSITAR > > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -- *Per Buer* CTO | Varnish Software AS Cell: +47 95839117 We Make Websites Fly! www.varnish-software.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From t.honacker at googlemail.com Wed Dec 23 21:35:44 2015 From: t.honacker at googlemail.com (Tobias Honacker) Date: Wed, 23 Dec 2015 22:35:44 +0100 Subject: Bypass cache when add to cart ? In-Reply-To: References: Message-ID: Hi Xavier, maybe this snippet will help you Cookie: https://github.com/lkarsten/libvmod-cookie Query String: https://github.com/Dridi/libvmod-querystring default.vcl: import cookie; import querystring; sub vcl_recv: if (req.url ~ "^/random/url") { cookie.parse(req.http.Cookie); cookie.filter_except("cookie_woocommerce"); if ((cookie.isset("cookie_woocommerce")) { return(pass); } else { return(hash); } } // to get the value out of the cookie -> cookie.get("woocommerce") != "false" and something like this set req.url = querystring.sort(req.url); set req.url = querystring.filter(req.url, "query_string"); ... do some magic 2015-12-23 21:05 GMT+01:00 Guillaume Quintard < guillaume at varnish-software.com>: > > On Dec 23, 2015 20:51, "Xavier Cardil Coll" wrote: > > > > No one knows then ? Not even a hint or something ? > > > > 2015-12-23 13:15 GMT+01:00 Xavier Cardil Coll : > >> > >> Also, how to write a function to bypass cache entirely when a user logs > in ? > >> > >> 2015-12-23 12:39 GMT+01:00 Xavier Cardil Coll : > >>> > >>> On Nginx cache, you can do this to avoid keep on caching when you add > something to the cart in Woocommerce : > >>> > >>> Skip cache for WooCommerce query string > >>> > >>> if ( $arg_add-to-cart != "" ) { > >>> set $skip_cache 1; > >>> } > >>> > >>> Skip cache when WooCommerce cart is not empty > >>> > >>> if ( $cookie_woocommerce_items_in_cart != "0" ) { > >>> set $skip_cache 1; > >>> } > >>> > >>> How can you do this in Varnish ? With this config, wich was made > specifically for Wordpress + Woocommerce, when you add something to cart, > cache is not disable store wide so you get a mix of cached and non cache > pages, cart gets emptied : http://pastebin.com/TuYeMtUZ > >>> > >>> > >>> -- > >>> > >>> ELSITAR > >>> > >>> > >> > >> > >> > >> -- > >> > >> ELSITAR > >> > >> > > > > > > > > -- > > > > ELSITAR > > > > > > > > _______________________________________________ > > varnish-misc mailing list > > varnish-misc at varnish-cache.org > > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > Chill out, most of the people are in vacation :-) > > So, in varnish, by default, if the request contains a cookie, the cache is > bypassed. > > But you can also force a pass, for example if the request contains a > "foo-bar" header like so : > > sub vcl_recv { > if (req.http.foo-bar) { > return (pass); > } > } > > For your needs, look at the query string and cookie vmods, it will allow > to really fine-tune your workflow. > > And should you need more help, we are here :-) > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From info at elsitar.com Wed Dec 23 23:24:43 2015 From: info at elsitar.com (Xavier Cardil Coll) Date: Thu, 24 Dec 2015 00:24:43 +0100 Subject: Bypass cache when add to cart ? In-Reply-To: References: Message-ID: thank you so much for this information, I will get hands down with it. 2015-12-23 22:35 GMT+01:00 Tobias Honacker : > Hi Xavier, > > maybe this snippet will help you > > Cookie: https://github.com/lkarsten/libvmod-cookie > Query String: https://github.com/Dridi/libvmod-querystring > > default.vcl: > > import cookie; > import querystring; > > sub vcl_recv: > > if (req.url ~ "^/random/url") { > cookie.parse(req.http.Cookie); > cookie.filter_except("cookie_woocommerce"); > if ((cookie.isset("cookie_woocommerce")) { > return(pass); > } else { > return(hash); > } > } // to get the value out of the cookie -> cookie.get("woocommerce") != > "false" > > and something like this > > set req.url = querystring.sort(req.url); > set req.url = querystring.filter(req.url, "query_string"); > ... do some magic > > 2015-12-23 21:05 GMT+01:00 Guillaume Quintard < > guillaume at varnish-software.com>: > >> >> On Dec 23, 2015 20:51, "Xavier Cardil Coll" wrote: >> > >> > No one knows then ? Not even a hint or something ? >> > >> > 2015-12-23 13:15 GMT+01:00 Xavier Cardil Coll : >> >> >> >> Also, how to write a function to bypass cache entirely when a user >> logs in ? >> >> >> >> 2015-12-23 12:39 GMT+01:00 Xavier Cardil Coll : >> >>> >> >>> On Nginx cache, you can do this to avoid keep on caching when you add >> something to the cart in Woocommerce : >> >>> >> >>> Skip cache for WooCommerce query string >> >>> >> >>> if ( $arg_add-to-cart != "" ) { >> >>> set $skip_cache 1; >> >>> } >> >>> >> >>> Skip cache when WooCommerce cart is not empty >> >>> >> >>> if ( $cookie_woocommerce_items_in_cart != "0" ) { >> >>> set $skip_cache 1; >> >>> } >> >>> >> >>> How can you do this in Varnish ? With this config, wich was made >> specifically for Wordpress + Woocommerce, when you add something to cart, >> cache is not disable store wide so you get a mix of cached and non cache >> pages, cart gets emptied : http://pastebin.com/TuYeMtUZ >> >>> >> >>> >> >>> -- >> >>> >> >>> ELSITAR >> >>> >> >>> >> >> >> >> >> >> >> >> -- >> >> >> >> ELSITAR >> >> >> >> >> > >> > >> > >> > -- >> > >> > ELSITAR >> > >> > >> > >> > _______________________________________________ >> > varnish-misc mailing list >> > varnish-misc at varnish-cache.org >> > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> >> Chill out, most of the people are in vacation :-) >> >> So, in varnish, by default, if the request contains a cookie, the cache >> is bypassed. >> >> But you can also force a pass, for example if the request contains a >> "foo-bar" header like so : >> >> sub vcl_recv { >> if (req.http.foo-bar) { >> return (pass); >> } >> } >> >> For your needs, look at the query string and cookie vmods, it will allow >> to really fine-tune your workflow. >> >> And should you need more help, we are here :-) >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> > > -- ELSITAR -------------- next part -------------- An HTML attachment was scrubbed... URL: From hernan at cmsmedios.com Sat Dec 26 15:27:15 2015 From: hernan at cmsmedios.com (=?UTF-8?Q?Hern=C3=A1n_Marsili?=) Date: Sat, 26 Dec 2015 15:27:15 +0000 Subject: using http/2 with varnish Message-ID: Hi, We are migrating a lot of customer to HTTPS to take advantage of HTTP/2. We are running VARNISH 3 and as far as we were able to determine, is not possible to run this scenario using Varnish. Can anyone point us on the right direction here? a) is there any way to run https on varnish 3? b) do we need to migrate to varnish 4? Regards Hern?n. -------------- next part -------------- An HTML attachment was scrubbed... URL: From yakov.yarmo at gmail.com Sun Dec 27 11:32:38 2015 From: yakov.yarmo at gmail.com (=?UTF-8?B?15nXoten15Eg15nXqNee15XXnNeV15HXoden15k=?=) Date: Sun, 27 Dec 2015 13:32:38 +0200 Subject: why i get response if it is a post request with pass? Message-ID: I checked my apache log and it responded with 200. 14 BackendXID b 2005611725 14 TxRequest b POST 14 TxURL b /node/16348/edit?destination=user/18407 14 TxProtocol b HTTP/1.1 14 TxHeader b Host: www.limudnaim.co.il 14 TxHeader b Content-Length: 7411 14 TxHeader b Cache-Control: max-age=0 14 TxHeader b Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 14 TxHeader b Origin: http://www.limudnaim.co.il 14 TxHeader b Upgrade-Insecure-Requests: 1 14 TxHeader b User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36 14 TxHeader b Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryHUWnuy0mKyJkVjht 14 TxHeader b Referer: http://www.limudnaim.co.il/node/16348/edit?destination=user/18407 14 TxHeader b Accept-Language: he-IL,he;q=0.8,en-US;q=0.6,en;q=0.4 14 TxHeader b X-UA-Device: pc 14 TxHeader b X-Full-Uri: www.limudnaim.co.il%2Fnode%2F16348%2Fedit%3Fdestination%3Duser%2F18407 14 TxHeader b X-Forwarded-For: 46.121.61.13 14 TxHeader b Accept-Encoding: gzip 14 TxHeader b Cookie: SESSdac9e0acd970131ed940ca8f937850da=cc448a1c9de0973a7878d6e0941536c9; NO_CACHE=Y 14 TxHeader b X-Varnish: 2005611725 14 BackendClose b default 14 BackendOpen b default 82.80.17.31 39482 82.80.17.31 1012 14 BackendXID b 2005611725 14 TxRequest b POST 14 TxURL b /node/16348/edit?destination=user/18407 14 TxProtocol b HTTP/1.1 14 TxHeader b Host: www.limudnaim.co.il 14 TxHeader b Content-Length: 7411 14 TxHeader b Cache-Control: max-age=0 14 TxHeader b Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 14 TxHeader b Origin: http://www.limudnaim.co.il 14 TxHeader b Upgrade-Insecure-Requests: 1 14 TxHeader b User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36 14 TxHeader b Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryHUWnuy0mKyJkVjht 14 TxHeader b Referer: http://www.limudnaim.co.il/node/16348/edit?destination=user/18407 14 TxHeader b Accept-Language: he-IL,he;q=0.8,en-US;q=0.6,en;q=0.4 14 TxHeader b X-UA-Device: pc 14 TxHeader b X-Full-Uri: www.limudnaim.co.il%2Fnode%2F16348%2Fedit%3Fdestination%3Duser%2F18407 14 TxHeader b X-Forwarded-For: 46.121.61.13 14 TxHeader b Accept-Encoding: gzip 14 TxHeader b Cookie: SESSdac9e0acd970131ed940ca8f937850da=cc448a1c9de0973a7878d6e0941536c9; NO_CACHE=Y 14 TxHeader b X-Varnish: 2005611725 14 BackendClose b default 17 SessionOpen c 46.121.61.13 4316 :80 17 ReqStart c 46.121.61.13 4316 2005611725 17 RxRequest c POST 17 RxURL c /node/16348/edit?destination=user/18407 17 RxProtocol c HTTP/1.1 17 RxHeader c Host: www.limudnaim.co.il 17 RxHeader c Connection: keep-alive 17 RxHeader c Content-Length: 7411 17 RxHeader c Cache-Control: max-age=0 17 RxHeader c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 17 RxHeader c Origin: http://www.limudnaim.co.il 17 RxHeader c Upgrade-Insecure-Requests: 1 17 RxHeader c User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36 17 RxHeader c Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryHUWnuy0mKyJkVjht 17 RxHeader c Referer: http://www.limudnaim.co.il/node/16348/edit?destination=user/18407 17 RxHeader c Accept-Encoding: gzip, deflate 17 RxHeader c Accept-Language: he-IL,he;q=0.8,en-US;q=0.6,en;q=0.4 17 RxHeader c Cookie: SESSdac9e0acd970131ed940ca8f937850da=cc448a1c9de0973a7878d6e0941536c9; __utmt=1; NO_CACHE=Y; __utma=77876233.313860119.1451212742.1451212742.1451212742.1; __utmb=77876233.22.9.1451213179574; __utmc=77876233; __utmz=77876233.1451212742.1.1.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided) 17 VCL_call c recv pass 17 VCL_call c hash 17 Hash c /node/16348/edit?destination=user/18407 17 Hash c www.limudnaim.co.il 17 VCL_return c hash 17 VCL_call c pass pass 17 Backend c 14 default default 17 FetchError c http first read error: -1 0 (Success) 17 Backend c 14 default default 17 FetchError c Resource temporarily unavailable 17 FetchError c backend write error: 11 (Resource temporarily unavailable) 17 VCL_call c error deliver 17 VCL_call c deliver deliver 17 TxProtocol c HTTP/1.1 17 TxStatus c 302 17 TxResponse c Service Unavailable 17 TxHeader c Server: Varnish 17 TxHeader c x-status: 503 17 TxHeader c Location: /oops?path=/node/16348/edit?destination=user/18407&obj_status=503&obj_response=Service Unavailable&req_xid=2005611725 17 TxHeader c Accept-Ranges: bytes 17 TxHeader c Date: Sun, 27 Dec 2015 10:47:37 GMT 17 TxHeader c X-Varnish: 2005611725 17 TxHeader c Age: 5 17 TxHeader c Via: 1.1 varnish 17 TxHeader c Connection: close 17 TxHeader c X-Varnish-Cache: MISS 17 Length c 0 17 ReqEnd c 2005611725 1451213252.184271336 1451213257.219164610 0.000056505 5.034824371 0.000068903 15 BackendOpen b default 82.80.17.31 39486 82.80.17.31 1012 15 BackendXID b 2005611727 15 TxRequest b GET 15 TxURL b /oops?path=/node/16348/edit?destination=user/18407&obj_status=503&obj_response=Service%20Unavailable&req_xid=2005611725 15 TxProtocol b HTTP/1.1 15 TxHeader b Host: www.limudnaim.co.il 15 TxHeader b Cache-Control: max-age=0 15 TxHeader b Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 15 TxHeader b Upgrade-Insecure-Requests: 1 15 TxHeader b User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36 15 TxHeader b Referer: http://www.limudnaim.co.il/node/16348/edit?destination=user/18407 15 TxHeader b Accept-Language: he-IL,he;q=0.8,en-US;q=0.6,en;q=0.4 15 TxHeader b X-UA-Device: pc 15 TxHeader b X-Full-Uri: www.limudnaim.co.il%2Foops%3Fpath%3D%2Fnode%2F16348%2Fedit%3Fdestination%3Duser%2F18407%26obj_status%3D503%26obj_response%3DService%2520Unavailable%26req_xid%3D2005611725 15 TxHeader b X-Forwarded-For: 46.121.61.13 15 TxHeader b Accept-Encoding: gzip 15 TxHeader b Cookie: SESSdac9e0acd970131ed940ca8f937850da=cc448a1c9de0973a7878d6e0941536c9; NO_CACHE=Y 15 TxHeader b X-Varnish: 2005611727 -------------- next part -------------- An HTML attachment was scrubbed... URL: From mattias at nucleus.be Sun Dec 27 20:27:53 2015 From: mattias at nucleus.be (Mattias Geniar) Date: Sun, 27 Dec 2015 20:27:53 +0000 Subject: using http/2 with varnish In-Reply-To: References: Message-ID: <4E272C03-196F-4010-95E3-088D1AF1F452@nucleus.be> > Can anyone point us on the right direction here? What you need is a reverse proxy in front of your Varnish instances: consider running a tool like Nginx (which has HTTP/2 support in its mainline repositories) that does all your TLS connections and proxies the request on to Varnish, to keep optimising the cache. Mattias From guillaume at varnish-software.com Mon Dec 28 08:48:33 2015 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Mon, 28 Dec 2015 09:48:33 +0100 Subject: using http/2 with varnish In-Reply-To: <4E272C03-196F-4010-95E3-088D1AF1F452@nucleus.be> References: <4E272C03-196F-4010-95E3-088D1AF1F452@nucleus.be> Message-ID: Hi, For a more detailed answer : we don't support H/2 in varnish yet (working on it!). So, if you really really want H/2, having nginx in front of varnish can be a solution. If you are only interested in https, however, varnish 4.1 and onward supports the proxy protocol. It will allow to use and SSL/TLS terminator such as hitch or haproxy that will handle the encryption for you. The advantage to using the proxy protocol is that varnish is aware of it. If you use nginx to proxy the requests, varnish will only see one client: nginx. This means you'll have to do some gymnastics with XFF headers if you want to filter by ip address for example. Plus, nginx is a bit overkill in terms of resources to just be a tls terminator. Migrating to varnish 4 requires a bit of work (not that much, really), but it's worth it, especially considering v3 is EOL. -- Guillaume Quintard On Sun, Dec 27, 2015 at 9:27 PM, Mattias Geniar wrote: > > Can anyone point us on the right direction here? > > What you need is a reverse proxy in front of your Varnish instances: > consider running a tool like Nginx (which has HTTP/2 support in its > mainline repositories) that does all your TLS connections and proxies the > request on to Varnish, to keep optimising the cache. > > Mattias > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Mon Dec 28 08:55:21 2015 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Mon, 28 Dec 2015 09:55:21 +0100 Subject: why i get response if it is a post request with pass? In-Reply-To: References: Message-ID: It looks like your backend failed to deliver ("http first read error", then "backend write error"), so you were taken to vcl_backend_error then to vcl_deliver. Could it be that your vcl has a vcl_backend_error that rewrites the error code? >From the varnish side, your backend failed, it may be a timeout problem (backend replied too late). -- Guillaume Quintard On Sun, Dec 27, 2015 at 12:32 PM, ???? ?????????? wrote: > I checked my apache log and it responded with 200. > > 14 BackendXID b 2005611725 > 14 TxRequest b POST > 14 TxURL b /node/16348/edit?destination=user/18407 > 14 TxProtocol b HTTP/1.1 > 14 TxHeader b Host: www.limudnaim.co.il > 14 TxHeader b Content-Length: 7411 > 14 TxHeader b Cache-Control: max-age=0 > 14 TxHeader b Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 > 14 TxHeader b Origin: http://www.limudnaim.co.il > 14 TxHeader b Upgrade-Insecure-Requests: 1 > 14 TxHeader b User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36 > 14 TxHeader b Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryHUWnuy0mKyJkVjht > 14 TxHeader b Referer: http://www.limudnaim.co.il/node/16348/edit?destination=user/18407 > 14 TxHeader b Accept-Language: he-IL,he;q=0.8,en-US;q=0.6,en;q=0.4 > 14 TxHeader b X-UA-Device: pc > 14 TxHeader b X-Full-Uri: www.limudnaim.co.il%2Fnode%2F16348%2Fedit%3Fdestination%3Duser%2F18407 > 14 TxHeader b X-Forwarded-For: 46.121.61.13 > 14 TxHeader b Accept-Encoding: gzip > 14 TxHeader b Cookie: SESSdac9e0acd970131ed940ca8f937850da=cc448a1c9de0973a7878d6e0941536c9; NO_CACHE=Y > 14 TxHeader b X-Varnish: 2005611725 > 14 BackendClose b default > 14 BackendOpen b default 82.80.17.31 39482 82.80.17.31 1012 > 14 BackendXID b 2005611725 > 14 TxRequest b POST > 14 TxURL b /node/16348/edit?destination=user/18407 > 14 TxProtocol b HTTP/1.1 > 14 TxHeader b Host: www.limudnaim.co.il > 14 TxHeader b Content-Length: 7411 > 14 TxHeader b Cache-Control: max-age=0 > 14 TxHeader b Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 > 14 TxHeader b Origin: http://www.limudnaim.co.il > 14 TxHeader b Upgrade-Insecure-Requests: 1 > 14 TxHeader b User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36 > 14 TxHeader b Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryHUWnuy0mKyJkVjht > 14 TxHeader b Referer: http://www.limudnaim.co.il/node/16348/edit?destination=user/18407 > 14 TxHeader b Accept-Language: he-IL,he;q=0.8,en-US;q=0.6,en;q=0.4 > 14 TxHeader b X-UA-Device: pc > 14 TxHeader b X-Full-Uri: www.limudnaim.co.il%2Fnode%2F16348%2Fedit%3Fdestination%3Duser%2F18407 > 14 TxHeader b X-Forwarded-For: 46.121.61.13 > 14 TxHeader b Accept-Encoding: gzip > 14 TxHeader b Cookie: SESSdac9e0acd970131ed940ca8f937850da=cc448a1c9de0973a7878d6e0941536c9; NO_CACHE=Y > 14 TxHeader b X-Varnish: 2005611725 > 14 BackendClose b default > 17 SessionOpen c 46.121.61.13 4316 :80 > 17 ReqStart c 46.121.61.13 4316 2005611725 > 17 RxRequest c POST > 17 RxURL c /node/16348/edit?destination=user/18407 > 17 RxProtocol c HTTP/1.1 > 17 RxHeader c Host: www.limudnaim.co.il > 17 RxHeader c Connection: keep-alive > 17 RxHeader c Content-Length: 7411 > 17 RxHeader c Cache-Control: max-age=0 > 17 RxHeader c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 > 17 RxHeader c Origin: http://www.limudnaim.co.il > 17 RxHeader c Upgrade-Insecure-Requests: 1 > 17 RxHeader c User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36 > 17 RxHeader c Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryHUWnuy0mKyJkVjht > 17 RxHeader c Referer: http://www.limudnaim.co.il/node/16348/edit?destination=user/18407 > 17 RxHeader c Accept-Encoding: gzip, deflate > 17 RxHeader c Accept-Language: he-IL,he;q=0.8,en-US;q=0.6,en;q=0.4 > 17 RxHeader c Cookie: SESSdac9e0acd970131ed940ca8f937850da=cc448a1c9de0973a7878d6e0941536c9; __utmt=1; NO_CACHE=Y; __utma=77876233.313860119.1451212742.1451212742.1451212742.1; __utmb=77876233.22.9.1451213179574; __utmc=77876233; __utmz=77876233.1451212742.1.1.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided) > 17 VCL_call c recv pass > 17 VCL_call c hash > 17 Hash c /node/16348/edit?destination=user/18407 > 17 Hash c www.limudnaim.co.il > 17 VCL_return c hash > 17 VCL_call c pass pass > 17 Backend c 14 default default > 17 FetchError c http first read error: -1 0 (Success) > 17 Backend c 14 default default > 17 FetchError c Resource temporarily unavailable > 17 FetchError c backend write error: 11 (Resource temporarily unavailable) > 17 VCL_call c error deliver > 17 VCL_call c deliver deliver > 17 TxProtocol c HTTP/1.1 > 17 TxStatus c 302 > 17 TxResponse c Service Unavailable > 17 TxHeader c Server: Varnish > 17 TxHeader c x-status: 503 > 17 TxHeader c Location: /oops?path=/node/16348/edit?destination=user/18407&obj_status=503&obj_response=Service Unavailable&req_xid=2005611725 > 17 TxHeader c Accept-Ranges: bytes > 17 TxHeader c Date: Sun, 27 Dec 2015 10:47:37 GMT > 17 TxHeader c X-Varnish: 2005611725 > 17 TxHeader c Age: 5 > 17 TxHeader c Via: 1.1 varnish > 17 TxHeader c Connection: close > 17 TxHeader c X-Varnish-Cache: MISS > 17 Length c 0 > 17 ReqEnd c 2005611725 1451213252.184271336 1451213257.219164610 0.000056505 5.034824371 0.000068903 > 15 BackendOpen b default 82.80.17.31 39486 82.80.17.31 1012 > 15 BackendXID b 2005611727 > 15 TxRequest b GET > 15 TxURL b /oops?path=/node/16348/edit?destination=user/18407&obj_status=503&obj_response=Service%20Unavailable&req_xid=2005611725 > 15 TxProtocol b HTTP/1.1 > 15 TxHeader b Host: www.limudnaim.co.il > 15 TxHeader b Cache-Control: max-age=0 > 15 TxHeader b Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 > 15 TxHeader b Upgrade-Insecure-Requests: 1 > 15 TxHeader b User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36 > 15 TxHeader b Referer: http://www.limudnaim.co.il/node/16348/edit?destination=user/18407 > 15 TxHeader b Accept-Language: he-IL,he;q=0.8,en-US;q=0.6,en;q=0.4 > 15 TxHeader b X-UA-Device: pc > 15 TxHeader b X-Full-Uri: www.limudnaim.co.il%2Foops%3Fpath%3D%2Fnode%2F16348%2Fedit%3Fdestination%3Duser%2F18407%26obj_status%3D503%26obj_response%3DService%2520Unavailable%26req_xid%3D2005611725 > 15 TxHeader b X-Forwarded-For: 46.121.61.13 > 15 TxHeader b Accept-Encoding: gzip > 15 TxHeader b Cookie: SESSdac9e0acd970131ed940ca8f937850da=cc448a1c9de0973a7878d6e0941536c9; NO_CACHE=Y > 15 TxHeader b X-Varnish: 2005611727 > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From yakov.yarmo at gmail.com Mon Dec 28 09:25:22 2015 From: yakov.yarmo at gmail.com (=?UTF-8?B?15nXoten15Eg15nXqNee15XXnNeV15HXoden15k=?=) Date: Mon, 28 Dec 2015 11:25:22 +0200 Subject: why i get response if it is a post request with pass? In-Reply-To: References: Message-ID: I'm using varnish 3 so I don't think that I have vcl_backend_error. As for the backend, I'm using Apache 2.2.31 and I have checked the logs but it seems that everything goes well there and I get status 200. Maybe I need to enable something in the Apache config? On Mon, Dec 28, 2015 at 10:55 AM, Guillaume Quintard < guillaume at varnish-software.com> wrote: > It looks like your backend failed to deliver ("http first read error", > then "backend write error"), so you were taken to vcl_backend_error then > to vcl_deliver. Could it be that your vcl has a vcl_backend_error that > rewrites the error code? > > From the varnish side, your backend failed, it may be a timeout problem > (backend replied too late). > > -- > Guillaume Quintard > > On Sun, Dec 27, 2015 at 12:32 PM, ???? ?????????? > wrote: > >> I checked my apache log and it responded with 200. >> >> 14 BackendXID b 2005611725 >> 14 TxRequest b POST >> 14 TxURL b /node/16348/edit?destination=user/18407 >> 14 TxProtocol b HTTP/1.1 >> 14 TxHeader b Host: www.limudnaim.co.il >> 14 TxHeader b Content-Length: 7411 >> 14 TxHeader b Cache-Control: max-age=0 >> 14 TxHeader b Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 >> 14 TxHeader b Origin: http://www.limudnaim.co.il >> 14 TxHeader b Upgrade-Insecure-Requests: 1 >> 14 TxHeader b User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36 >> 14 TxHeader b Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryHUWnuy0mKyJkVjht >> 14 TxHeader b Referer: http://www.limudnaim.co.il/node/16348/edit?destination=user/18407 >> 14 TxHeader b Accept-Language: he-IL,he;q=0.8,en-US;q=0.6,en;q=0.4 >> 14 TxHeader b X-UA-Device: pc >> 14 TxHeader b X-Full-Uri: www.limudnaim.co.il%2Fnode%2F16348%2Fedit%3Fdestination%3Duser%2F18407 >> 14 TxHeader b X-Forwarded-For: 46.121.61.13 >> 14 TxHeader b Accept-Encoding: gzip >> 14 TxHeader b Cookie: SESSdac9e0acd970131ed940ca8f937850da=cc448a1c9de0973a7878d6e0941536c9; NO_CACHE=Y >> 14 TxHeader b X-Varnish: 2005611725 >> 14 BackendClose b default >> 14 BackendOpen b default 82.80.17.31 39482 82.80.17.31 1012 >> 14 BackendXID b 2005611725 >> 14 TxRequest b POST >> 14 TxURL b /node/16348/edit?destination=user/18407 >> 14 TxProtocol b HTTP/1.1 >> 14 TxHeader b Host: www.limudnaim.co.il >> 14 TxHeader b Content-Length: 7411 >> 14 TxHeader b Cache-Control: max-age=0 >> 14 TxHeader b Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 >> 14 TxHeader b Origin: http://www.limudnaim.co.il >> 14 TxHeader b Upgrade-Insecure-Requests: 1 >> 14 TxHeader b User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36 >> 14 TxHeader b Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryHUWnuy0mKyJkVjht >> 14 TxHeader b Referer: http://www.limudnaim.co.il/node/16348/edit?destination=user/18407 >> 14 TxHeader b Accept-Language: he-IL,he;q=0.8,en-US;q=0.6,en;q=0.4 >> 14 TxHeader b X-UA-Device: pc >> 14 TxHeader b X-Full-Uri: www.limudnaim.co.il%2Fnode%2F16348%2Fedit%3Fdestination%3Duser%2F18407 >> 14 TxHeader b X-Forwarded-For: 46.121.61.13 >> 14 TxHeader b Accept-Encoding: gzip >> 14 TxHeader b Cookie: SESSdac9e0acd970131ed940ca8f937850da=cc448a1c9de0973a7878d6e0941536c9; NO_CACHE=Y >> 14 TxHeader b X-Varnish: 2005611725 >> 14 BackendClose b default >> 17 SessionOpen c 46.121.61.13 4316 :80 >> 17 ReqStart c 46.121.61.13 4316 2005611725 >> 17 RxRequest c POST >> 17 RxURL c /node/16348/edit?destination=user/18407 >> 17 RxProtocol c HTTP/1.1 >> 17 RxHeader c Host: www.limudnaim.co.il >> 17 RxHeader c Connection: keep-alive >> 17 RxHeader c Content-Length: 7411 >> 17 RxHeader c Cache-Control: max-age=0 >> 17 RxHeader c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 >> 17 RxHeader c Origin: http://www.limudnaim.co.il >> 17 RxHeader c Upgrade-Insecure-Requests: 1 >> 17 RxHeader c User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36 >> 17 RxHeader c Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryHUWnuy0mKyJkVjht >> 17 RxHeader c Referer: http://www.limudnaim.co.il/node/16348/edit?destination=user/18407 >> 17 RxHeader c Accept-Encoding: gzip, deflate >> 17 RxHeader c Accept-Language: he-IL,he;q=0.8,en-US;q=0.6,en;q=0.4 >> 17 RxHeader c Cookie: SESSdac9e0acd970131ed940ca8f937850da=cc448a1c9de0973a7878d6e0941536c9; __utmt=1; NO_CACHE=Y; __utma=77876233.313860119.1451212742.1451212742.1451212742.1; __utmb=77876233.22.9.1451213179574; __utmc=77876233; __utmz=77876233.1451212742.1.1.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided) >> 17 VCL_call c recv pass >> 17 VCL_call c hash >> 17 Hash c /node/16348/edit?destination=user/18407 >> 17 Hash c www.limudnaim.co.il >> 17 VCL_return c hash >> 17 VCL_call c pass pass >> 17 Backend c 14 default default >> 17 FetchError c http first read error: -1 0 (Success) >> 17 Backend c 14 default default >> 17 FetchError c Resource temporarily unavailable >> 17 FetchError c backend write error: 11 (Resource temporarily unavailable) >> 17 VCL_call c error deliver >> 17 VCL_call c deliver deliver >> 17 TxProtocol c HTTP/1.1 >> 17 TxStatus c 302 >> 17 TxResponse c Service Unavailable >> 17 TxHeader c Server: Varnish >> 17 TxHeader c x-status: 503 >> 17 TxHeader c Location: /oops?path=/node/16348/edit?destination=user/18407&obj_status=503&obj_response=Service Unavailable&req_xid=2005611725 >> 17 TxHeader c Accept-Ranges: bytes >> 17 TxHeader c Date: Sun, 27 Dec 2015 10:47:37 GMT >> 17 TxHeader c X-Varnish: 2005611725 >> 17 TxHeader c Age: 5 >> 17 TxHeader c Via: 1.1 varnish >> 17 TxHeader c Connection: close >> 17 TxHeader c X-Varnish-Cache: MISS >> 17 Length c 0 >> 17 ReqEnd c 2005611725 1451213252.184271336 1451213257.219164610 0.000056505 5.034824371 0.000068903 >> 15 BackendOpen b default 82.80.17.31 39486 82.80.17.31 1012 >> 15 BackendXID b 2005611727 >> 15 TxRequest b GET >> 15 TxURL b /oops?path=/node/16348/edit?destination=user/18407&obj_status=503&obj_response=Service%20Unavailable&req_xid=2005611725 >> 15 TxProtocol b HTTP/1.1 >> 15 TxHeader b Host: www.limudnaim.co.il >> 15 TxHeader b Cache-Control: max-age=0 >> 15 TxHeader b Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 >> 15 TxHeader b Upgrade-Insecure-Requests: 1 >> 15 TxHeader b User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36 >> 15 TxHeader b Referer: http://www.limudnaim.co.il/node/16348/edit?destination=user/18407 >> 15 TxHeader b Accept-Language: he-IL,he;q=0.8,en-US;q=0.6,en;q=0.4 >> 15 TxHeader b X-UA-Device: pc >> 15 TxHeader b X-Full-Uri: www.limudnaim.co.il%2Foops%3Fpath%3D%2Fnode%2F16348%2Fedit%3Fdestination%3Duser%2F18407%26obj_status%3D503%26obj_response%3DService%2520Unavailable%26req_xid%3D2005611725 >> 15 TxHeader b X-Forwarded-For: 46.121.61.13 >> 15 TxHeader b Accept-Encoding: gzip >> 15 TxHeader b Cookie: SESSdac9e0acd970131ed940ca8f937850da=cc448a1c9de0973a7878d6e0941536c9; NO_CACHE=Y >> 15 TxHeader b X-Varnish: 2005611727 >> >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> > > -- *???? ??????????,???? ??? ????? ????www.limudnaim.co.il ??? ????? ?????? ??????* -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Mon Dec 28 09:33:46 2015 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Mon, 28 Dec 2015 10:33:46 +0100 Subject: why i get response if it is a post request with pass? In-Reply-To: References: Message-ID: Well, duh. Sorry, I didn't even notice you were on v3. How much time does Apache take to answer? It might be just a tiny bit more than what varnish would like. Do you have a custom vcl_error? -- Guillaume Quintard On Mon, Dec 28, 2015 at 10:25 AM, ???? ?????????? wrote: > I'm using varnish 3 so I don't think that I have vcl_backend_error. > As for the backend, I'm using Apache 2.2.31 and I have checked the logs > but it seems that everything goes well there and I get status 200. > Maybe I need to enable something in the Apache config? > > On Mon, Dec 28, 2015 at 10:55 AM, Guillaume Quintard < > guillaume at varnish-software.com> wrote: > >> It looks like your backend failed to deliver ("http first read error", >> then "backend write error"), so you were taken to vcl_backend_error then >> to vcl_deliver. Could it be that your vcl has a vcl_backend_error that >> rewrites the error code? >> >> From the varnish side, your backend failed, it may be a timeout problem >> (backend replied too late). >> >> -- >> Guillaume Quintard >> >> On Sun, Dec 27, 2015 at 12:32 PM, ???? ?????????? >> wrote: >> >>> I checked my apache log and it responded with 200. >>> >>> 14 BackendXID b 2005611725 >>> 14 TxRequest b POST >>> 14 TxURL b /node/16348/edit?destination=user/18407 >>> 14 TxProtocol b HTTP/1.1 >>> 14 TxHeader b Host: www.limudnaim.co.il >>> 14 TxHeader b Content-Length: 7411 >>> 14 TxHeader b Cache-Control: max-age=0 >>> 14 TxHeader b Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 >>> 14 TxHeader b Origin: http://www.limudnaim.co.il >>> 14 TxHeader b Upgrade-Insecure-Requests: 1 >>> 14 TxHeader b User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36 >>> 14 TxHeader b Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryHUWnuy0mKyJkVjht >>> 14 TxHeader b Referer: http://www.limudnaim.co.il/node/16348/edit?destination=user/18407 >>> 14 TxHeader b Accept-Language: he-IL,he;q=0.8,en-US;q=0.6,en;q=0.4 >>> 14 TxHeader b X-UA-Device: pc >>> 14 TxHeader b X-Full-Uri: www.limudnaim.co.il%2Fnode%2F16348%2Fedit%3Fdestination%3Duser%2F18407 >>> 14 TxHeader b X-Forwarded-For: 46.121.61.13 >>> 14 TxHeader b Accept-Encoding: gzip >>> 14 TxHeader b Cookie: SESSdac9e0acd970131ed940ca8f937850da=cc448a1c9de0973a7878d6e0941536c9; NO_CACHE=Y >>> 14 TxHeader b X-Varnish: 2005611725 >>> 14 BackendClose b default >>> 14 BackendOpen b default 82.80.17.31 39482 82.80.17.31 1012 >>> 14 BackendXID b 2005611725 >>> 14 TxRequest b POST >>> 14 TxURL b /node/16348/edit?destination=user/18407 >>> 14 TxProtocol b HTTP/1.1 >>> 14 TxHeader b Host: www.limudnaim.co.il >>> 14 TxHeader b Content-Length: 7411 >>> 14 TxHeader b Cache-Control: max-age=0 >>> 14 TxHeader b Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 >>> 14 TxHeader b Origin: http://www.limudnaim.co.il >>> 14 TxHeader b Upgrade-Insecure-Requests: 1 >>> 14 TxHeader b User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36 >>> 14 TxHeader b Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryHUWnuy0mKyJkVjht >>> 14 TxHeader b Referer: http://www.limudnaim.co.il/node/16348/edit?destination=user/18407 >>> 14 TxHeader b Accept-Language: he-IL,he;q=0.8,en-US;q=0.6,en;q=0.4 >>> 14 TxHeader b X-UA-Device: pc >>> 14 TxHeader b X-Full-Uri: www.limudnaim.co.il%2Fnode%2F16348%2Fedit%3Fdestination%3Duser%2F18407 >>> 14 TxHeader b X-Forwarded-For: 46.121.61.13 >>> 14 TxHeader b Accept-Encoding: gzip >>> 14 TxHeader b Cookie: SESSdac9e0acd970131ed940ca8f937850da=cc448a1c9de0973a7878d6e0941536c9; NO_CACHE=Y >>> 14 TxHeader b X-Varnish: 2005611725 >>> 14 BackendClose b default >>> 17 SessionOpen c 46.121.61.13 4316 :80 >>> 17 ReqStart c 46.121.61.13 4316 2005611725 >>> 17 RxRequest c POST >>> 17 RxURL c /node/16348/edit?destination=user/18407 >>> 17 RxProtocol c HTTP/1.1 >>> 17 RxHeader c Host: www.limudnaim.co.il >>> 17 RxHeader c Connection: keep-alive >>> 17 RxHeader c Content-Length: 7411 >>> 17 RxHeader c Cache-Control: max-age=0 >>> 17 RxHeader c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 >>> 17 RxHeader c Origin: http://www.limudnaim.co.il >>> 17 RxHeader c Upgrade-Insecure-Requests: 1 >>> 17 RxHeader c User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36 >>> 17 RxHeader c Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryHUWnuy0mKyJkVjht >>> 17 RxHeader c Referer: http://www.limudnaim.co.il/node/16348/edit?destination=user/18407 >>> 17 RxHeader c Accept-Encoding: gzip, deflate >>> 17 RxHeader c Accept-Language: he-IL,he;q=0.8,en-US;q=0.6,en;q=0.4 >>> 17 RxHeader c Cookie: SESSdac9e0acd970131ed940ca8f937850da=cc448a1c9de0973a7878d6e0941536c9; __utmt=1; NO_CACHE=Y; __utma=77876233.313860119.1451212742.1451212742.1451212742.1; __utmb=77876233.22.9.1451213179574; __utmc=77876233; __utmz=77876233.1451212742.1.1.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided) >>> 17 VCL_call c recv pass >>> 17 VCL_call c hash >>> 17 Hash c /node/16348/edit?destination=user/18407 >>> 17 Hash c www.limudnaim.co.il >>> 17 VCL_return c hash >>> 17 VCL_call c pass pass >>> 17 Backend c 14 default default >>> 17 FetchError c http first read error: -1 0 (Success) >>> 17 Backend c 14 default default >>> 17 FetchError c Resource temporarily unavailable >>> 17 FetchError c backend write error: 11 (Resource temporarily unavailable) >>> 17 VCL_call c error deliver >>> 17 VCL_call c deliver deliver >>> 17 TxProtocol c HTTP/1.1 >>> 17 TxStatus c 302 >>> 17 TxResponse c Service Unavailable >>> 17 TxHeader c Server: Varnish >>> 17 TxHeader c x-status: 503 >>> 17 TxHeader c Location: /oops?path=/node/16348/edit?destination=user/18407&obj_status=503&obj_response=Service Unavailable&req_xid=2005611725 >>> 17 TxHeader c Accept-Ranges: bytes >>> 17 TxHeader c Date: Sun, 27 Dec 2015 10:47:37 GMT >>> 17 TxHeader c X-Varnish: 2005611725 >>> 17 TxHeader c Age: 5 >>> 17 TxHeader c Via: 1.1 varnish >>> 17 TxHeader c Connection: close >>> 17 TxHeader c X-Varnish-Cache: MISS >>> 17 Length c 0 >>> 17 ReqEnd c 2005611725 1451213252.184271336 1451213257.219164610 0.000056505 5.034824371 0.000068903 >>> 15 BackendOpen b default 82.80.17.31 39486 82.80.17.31 1012 >>> 15 BackendXID b 2005611727 >>> 15 TxRequest b GET >>> 15 TxURL b /oops?path=/node/16348/edit?destination=user/18407&obj_status=503&obj_response=Service%20Unavailable&req_xid=2005611725 >>> 15 TxProtocol b HTTP/1.1 >>> 15 TxHeader b Host: www.limudnaim.co.il >>> 15 TxHeader b Cache-Control: max-age=0 >>> 15 TxHeader b Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 >>> 15 TxHeader b Upgrade-Insecure-Requests: 1 >>> 15 TxHeader b User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36 >>> 15 TxHeader b Referer: http://www.limudnaim.co.il/node/16348/edit?destination=user/18407 >>> 15 TxHeader b Accept-Language: he-IL,he;q=0.8,en-US;q=0.6,en;q=0.4 >>> 15 TxHeader b X-UA-Device: pc >>> 15 TxHeader b X-Full-Uri: www.limudnaim.co.il%2Foops%3Fpath%3D%2Fnode%2F16348%2Fedit%3Fdestination%3Duser%2F18407%26obj_status%3D503%26obj_response%3DService%2520Unavailable%26req_xid%3D2005611725 >>> 15 TxHeader b X-Forwarded-For: 46.121.61.13 >>> 15 TxHeader b Accept-Encoding: gzip >>> 15 TxHeader b Cookie: SESSdac9e0acd970131ed940ca8f937850da=cc448a1c9de0973a7878d6e0941536c9; NO_CACHE=Y >>> 15 TxHeader b X-Varnish: 2005611727 >>> >>> >>> _______________________________________________ >>> varnish-misc mailing list >>> varnish-misc at varnish-cache.org >>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >>> >> >> > > > -- > > > > *???? ??????????,???? ??? ????? ????www.limudnaim.co.il > ??? ????? ?????? ??????* > -------------- next part -------------- An HTML attachment was scrubbed... URL: From yakov.yarmo at gmail.com Mon Dec 28 09:57:58 2015 From: yakov.yarmo at gmail.com (=?UTF-8?B?15nXoten15Eg15nXqNee15XXnNeV15HXoden15k=?=) Date: Mon, 28 Dec 2015 11:57:58 +0200 Subject: why i get response if it is a post request with pass? In-Reply-To: References: Message-ID: How can I check how much time Does Apache take to answer? And yes, I do have a custom vcl_error: sub vcl_error { if (obj.status == 750) { set obj.http.Location = obj.response; set obj.status = 302; return(deliver); } set obj.http.x-status = obj.status; set obj.status = 302; set obj.http.Location = "/oops?path=" + req.url + "&obj_status=" + obj.http.x-status + "&obj_response=" + obj.response + "&req_xid=" + req.xid; return (deliver); } On Mon, Dec 28, 2015 at 11:33 AM, Guillaume Quintard < guillaume at varnish-software.com> wrote: > Well, duh. Sorry, I didn't even notice you were on v3. > > How much time does Apache take to answer? It might be just a tiny bit more > than what varnish would like. > > Do you have a custom vcl_error? > > -- > Guillaume Quintard > > On Mon, Dec 28, 2015 at 10:25 AM, ???? ?????????? > wrote: > >> I'm using varnish 3 so I don't think that I have vcl_backend_error. >> As for the backend, I'm using Apache 2.2.31 and I have checked the logs >> but it seems that everything goes well there and I get status 200. >> Maybe I need to enable something in the Apache config? >> >> On Mon, Dec 28, 2015 at 10:55 AM, Guillaume Quintard < >> guillaume at varnish-software.com> wrote: >> >>> It looks like your backend failed to deliver ("http first read error", >>> then "backend write error"), so you were taken to vcl_backend_error >>> then to vcl_deliver. Could it be that your vcl has a vcl_backend_error that >>> rewrites the error code? >>> >>> From the varnish side, your backend failed, it may be a timeout problem >>> (backend replied too late). >>> >>> -- >>> Guillaume Quintard >>> >>> On Sun, Dec 27, 2015 at 12:32 PM, ???? ?????????? >> > wrote: >>> >>>> I checked my apache log and it responded with 200. >>>> >>>> 14 BackendXID b 2005611725 >>>> 14 TxRequest b POST >>>> 14 TxURL b /node/16348/edit?destination=user/18407 >>>> 14 TxProtocol b HTTP/1.1 >>>> 14 TxHeader b Host: www.limudnaim.co.il >>>> 14 TxHeader b Content-Length: 7411 >>>> 14 TxHeader b Cache-Control: max-age=0 >>>> 14 TxHeader b Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 >>>> 14 TxHeader b Origin: http://www.limudnaim.co.il >>>> 14 TxHeader b Upgrade-Insecure-Requests: 1 >>>> 14 TxHeader b User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36 >>>> 14 TxHeader b Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryHUWnuy0mKyJkVjht >>>> 14 TxHeader b Referer: http://www.limudnaim.co.il/node/16348/edit?destination=user/18407 >>>> 14 TxHeader b Accept-Language: he-IL,he;q=0.8,en-US;q=0.6,en;q=0.4 >>>> 14 TxHeader b X-UA-Device: pc >>>> 14 TxHeader b X-Full-Uri: www.limudnaim.co.il%2Fnode%2F16348%2Fedit%3Fdestination%3Duser%2F18407 >>>> 14 TxHeader b X-Forwarded-For: 46.121.61.13 >>>> 14 TxHeader b Accept-Encoding: gzip >>>> 14 TxHeader b Cookie: SESSdac9e0acd970131ed940ca8f937850da=cc448a1c9de0973a7878d6e0941536c9; NO_CACHE=Y >>>> 14 TxHeader b X-Varnish: 2005611725 >>>> 14 BackendClose b default >>>> 14 BackendOpen b default 82.80.17.31 39482 82.80.17.31 1012 >>>> 14 BackendXID b 2005611725 >>>> 14 TxRequest b POST >>>> 14 TxURL b /node/16348/edit?destination=user/18407 >>>> 14 TxProtocol b HTTP/1.1 >>>> 14 TxHeader b Host: www.limudnaim.co.il >>>> 14 TxHeader b Content-Length: 7411 >>>> 14 TxHeader b Cache-Control: max-age=0 >>>> 14 TxHeader b Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 >>>> 14 TxHeader b Origin: http://www.limudnaim.co.il >>>> 14 TxHeader b Upgrade-Insecure-Requests: 1 >>>> 14 TxHeader b User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36 >>>> 14 TxHeader b Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryHUWnuy0mKyJkVjht >>>> 14 TxHeader b Referer: http://www.limudnaim.co.il/node/16348/edit?destination=user/18407 >>>> 14 TxHeader b Accept-Language: he-IL,he;q=0.8,en-US;q=0.6,en;q=0.4 >>>> 14 TxHeader b X-UA-Device: pc >>>> 14 TxHeader b X-Full-Uri: www.limudnaim.co.il%2Fnode%2F16348%2Fedit%3Fdestination%3Duser%2F18407 >>>> 14 TxHeader b X-Forwarded-For: 46.121.61.13 >>>> 14 TxHeader b Accept-Encoding: gzip >>>> 14 TxHeader b Cookie: SESSdac9e0acd970131ed940ca8f937850da=cc448a1c9de0973a7878d6e0941536c9; NO_CACHE=Y >>>> 14 TxHeader b X-Varnish: 2005611725 >>>> 14 BackendClose b default >>>> 17 SessionOpen c 46.121.61.13 4316 :80 >>>> 17 ReqStart c 46.121.61.13 4316 2005611725 >>>> 17 RxRequest c POST >>>> 17 RxURL c /node/16348/edit?destination=user/18407 >>>> 17 RxProtocol c HTTP/1.1 >>>> 17 RxHeader c Host: www.limudnaim.co.il >>>> 17 RxHeader c Connection: keep-alive >>>> 17 RxHeader c Content-Length: 7411 >>>> 17 RxHeader c Cache-Control: max-age=0 >>>> 17 RxHeader c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 >>>> 17 RxHeader c Origin: http://www.limudnaim.co.il >>>> 17 RxHeader c Upgrade-Insecure-Requests: 1 >>>> 17 RxHeader c User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36 >>>> 17 RxHeader c Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryHUWnuy0mKyJkVjht >>>> 17 RxHeader c Referer: http://www.limudnaim.co.il/node/16348/edit?destination=user/18407 >>>> 17 RxHeader c Accept-Encoding: gzip, deflate >>>> 17 RxHeader c Accept-Language: he-IL,he;q=0.8,en-US;q=0.6,en;q=0.4 >>>> 17 RxHeader c Cookie: SESSdac9e0acd970131ed940ca8f937850da=cc448a1c9de0973a7878d6e0941536c9; __utmt=1; NO_CACHE=Y; __utma=77876233.313860119.1451212742.1451212742.1451212742.1; __utmb=77876233.22.9.1451213179574; __utmc=77876233; __utmz=77876233.1451212742.1.1.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided) >>>> 17 VCL_call c recv pass >>>> 17 VCL_call c hash >>>> 17 Hash c /node/16348/edit?destination=user/18407 >>>> 17 Hash c www.limudnaim.co.il >>>> 17 VCL_return c hash >>>> 17 VCL_call c pass pass >>>> 17 Backend c 14 default default >>>> 17 FetchError c http first read error: -1 0 (Success) >>>> 17 Backend c 14 default default >>>> 17 FetchError c Resource temporarily unavailable >>>> 17 FetchError c backend write error: 11 (Resource temporarily unavailable) >>>> 17 VCL_call c error deliver >>>> 17 VCL_call c deliver deliver >>>> 17 TxProtocol c HTTP/1.1 >>>> 17 TxStatus c 302 >>>> 17 TxResponse c Service Unavailable >>>> 17 TxHeader c Server: Varnish >>>> 17 TxHeader c x-status: 503 >>>> 17 TxHeader c Location: /oops?path=/node/16348/edit?destination=user/18407&obj_status=503&obj_response=Service Unavailable&req_xid=2005611725 >>>> 17 TxHeader c Accept-Ranges: bytes >>>> 17 TxHeader c Date: Sun, 27 Dec 2015 10:47:37 GMT >>>> 17 TxHeader c X-Varnish: 2005611725 >>>> 17 TxHeader c Age: 5 >>>> 17 TxHeader c Via: 1.1 varnish >>>> 17 TxHeader c Connection: close >>>> 17 TxHeader c X-Varnish-Cache: MISS >>>> 17 Length c 0 >>>> 17 ReqEnd c 2005611725 1451213252.184271336 1451213257.219164610 0.000056505 5.034824371 0.000068903 >>>> 15 BackendOpen b default 82.80.17.31 39486 82.80.17.31 1012 >>>> 15 BackendXID b 2005611727 >>>> 15 TxRequest b GET >>>> 15 TxURL b /oops?path=/node/16348/edit?destination=user/18407&obj_status=503&obj_response=Service%20Unavailable&req_xid=2005611725 >>>> 15 TxProtocol b HTTP/1.1 >>>> 15 TxHeader b Host: www.limudnaim.co.il >>>> 15 TxHeader b Cache-Control: max-age=0 >>>> 15 TxHeader b Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 >>>> 15 TxHeader b Upgrade-Insecure-Requests: 1 >>>> 15 TxHeader b User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36 >>>> 15 TxHeader b Referer: http://www.limudnaim.co.il/node/16348/edit?destination=user/18407 >>>> 15 TxHeader b Accept-Language: he-IL,he;q=0.8,en-US;q=0.6,en;q=0.4 >>>> 15 TxHeader b X-UA-Device: pc >>>> 15 TxHeader b X-Full-Uri: www.limudnaim.co.il%2Foops%3Fpath%3D%2Fnode%2F16348%2Fedit%3Fdestination%3Duser%2F18407%26obj_status%3D503%26obj_response%3DService%2520Unavailable%26req_xid%3D2005611725 >>>> 15 TxHeader b X-Forwarded-For: 46.121.61.13 >>>> 15 TxHeader b Accept-Encoding: gzip >>>> 15 TxHeader b Cookie: SESSdac9e0acd970131ed940ca8f937850da=cc448a1c9de0973a7878d6e0941536c9; NO_CACHE=Y >>>> 15 TxHeader b X-Varnish: 2005611727 >>>> >>>> >>>> _______________________________________________ >>>> varnish-misc mailing list >>>> varnish-misc at varnish-cache.org >>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >>>> >>> >>> >> >> >> -- >> >> >> >> *???? ??????????,???? ??? ????? ????www.limudnaim.co.il >> ??? ????? ?????? ??????* >> > > -- *???? ??????????,???? ??? ????? ????www.limudnaim.co.il ??? ????? ?????? ??????* -------------- next part -------------- An HTML attachment was scrubbed... URL: From geoff at uplex.de Mon Dec 28 10:17:20 2015 From: geoff at uplex.de (Geoff Simmons) Date: Mon, 28 Dec 2015 11:17:20 +0100 Subject: why i get response if it is a post request with pass? In-Reply-To: References: Message-ID: <56810C30.8040904@uplex.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 12/28/15 10:25 AM, ???? ?????????? wrote: > I'm using varnish 3 so I don't think that I have > vcl_backend_error. Right, but your VCL is evidently doing something in the form of error handling, since it reacted to the fetch error (which is seen in VCL as response code 503) by redirecting, with response code 302 and a Location header. The redirect goes to "/oops" with obj_status=503, obj_response=Service Unavailable and the request XID in the query string, which sounds like custom error handling. > As for the backend, I'm using Apache 2.2.31 and I have checked the > logs but it seems that everything goes well there and I get status > 200. Sure, your Apache might be getting 200, but it took over 5 seconds to do so, and Varnish didn't wait that long. Look here: ReqEnd c 2005611725 1451213252.184271336 1451213257.219164610 0.000056505 5.034824371 0.000068903 The second-to-last number tells you that Varnish waited just over 5 seconds for the response and gave up, causing: FetchError c http first read error: -1 0 (Success) That's a first byte timeout, which is very likely set to 5 seconds. Always look for FetchError records in cases like this. "http first read error" is very common, it means your backend was too slow. > Maybe I need to enable something in the Apache config? Impossible to say from a distance. It might help to tune your Apache, but it might be something else entirely, such as database fetches in your application that are taking too long. HTH, Geoff - -- UPLEX Systemoptimierung Scheffelstra?e 32 22301 Hamburg http://uplex.de/ Mob: +49-176-63690917 -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.14 (Darwin) iQIcBAEBCAAGBQJWgQwvAAoJEOUwvh9pJNURRIYP/2gYEcYC0RksxFGL7iPQtel/ 5b1zJrhpdIzA8qbJZoU+kq/hk9YcMosmZfP7EV2ZvwMSy7K3/gnzVrMW581u1+iZ FlHWCNAA4kr4RtW7PIuB3obTtKYnQhL122TdgRIvtz4CPijW35E5pVA4JeVPbkJO UEf0CitjXkCW5w5utcnxudAOLFJHVdXxrC7/HltRdb6BQYoO97UVlN+vhW46s/jl GDFYoDUhAb5CsJGzaVCIJN4woFg/1v7QviHSw2ruStbnbP4xbwO8jTCAQ5DwDH8p por5dcOuv+CQCoVtNtvxAGhqdUeWm1EG7Vf3cYVh/+bj2fmL1zKd+Dvzp1PUK3RO E3pOsI5nPVHhCz5na/YdRStZ9oZiTlhZCb9sP0Jkwc6S6PReq2v0M6LWX1Ol1JJb 049K+sZo3RJjZqLX0Bu3hEm+nhJ8r157DebuoMPZc/BekB8OLPHfJ5AV91Yite4B Nsl2RX/Antke+Jd02GwMLii2ctkwwV1Q5wbJRLMcbOOUEqBighFut3v8G4s5SKgX CC51gOPzzLipp334kaaFk8U/s6Uuv04A6iGe5uUVAvO/FARVYerK3cY3bYQlh5Tm gC9iouAImUpxmwZlAy6vHhhH3gqF/avmexHt3Q5kfxk/G19cZL4nYaY+XQPVoc4E dx5K2/lwtB67Y/yc7rEQ =7HHm -----END PGP SIGNATURE----- From yakov.yarmo at gmail.com Mon Dec 28 10:30:25 2015 From: yakov.yarmo at gmail.com (=?UTF-8?B?15nXoten15Eg15nXqNee15XXnNeV15HXoden15k=?=) Date: Mon, 28 Dec 2015 12:30:25 +0200 Subject: why i get response if it is a post request with pass? In-Reply-To: <56810C30.8040904@uplex.de> References: <56810C30.8040904@uplex.de> Message-ID: But if I have the following in my vcl file: .first_byte_timeout = 600s; .between_bytes_timeout = 600s; Doesn't varnish suppose to respect it? Where can I see how long the apache took to respond? On Mon, Dec 28, 2015 at 12:17 PM, Geoff Simmons wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > On 12/28/15 10:25 AM, ???? ?????????? wrote: > > I'm using varnish 3 so I don't think that I have > > vcl_backend_error. > > Right, but your VCL is evidently doing something in the form of error > handling, since it reacted to the fetch error (which is seen in VCL as > response code 503) by redirecting, with response code 302 and a > Location header. The redirect goes to "/oops" with obj_status=503, > obj_response=Service Unavailable and the request XID in the query > string, which sounds like custom error handling. > > > As for the backend, I'm using Apache 2.2.31 and I have checked the > > logs but it seems that everything goes well there and I get status > > 200. > > Sure, your Apache might be getting 200, but it took over 5 seconds to > do so, and Varnish didn't wait that long. Look here: > > ReqEnd c 2005611725 1451213252.184271336 1451213257.219164610 > 0.000056505 5.034824371 0.000068903 > > The second-to-last number tells you that Varnish waited just over 5 > seconds for the response and gave up, causing: > > FetchError c http first read error: -1 0 (Success) > > That's a first byte timeout, which is very likely set to 5 seconds. > > Always look for FetchError records in cases like this. "http first > read error" is very common, it means your backend was too slow. > > > Maybe I need to enable something in the Apache config? > > Impossible to say from a distance. It might help to tune your Apache, > but it might be something else entirely, such as database fetches in > your application that are taking too long. > > > HTH, > Geoff > - -- > UPLEX Systemoptimierung > Scheffelstra?e 32 > 22301 Hamburg > http://uplex.de/ > Mob: +49-176-63690917 > -----BEGIN PGP SIGNATURE----- > Version: GnuPG/MacGPG2 v2.0.14 (Darwin) > > iQIcBAEBCAAGBQJWgQwvAAoJEOUwvh9pJNURRIYP/2gYEcYC0RksxFGL7iPQtel/ > 5b1zJrhpdIzA8qbJZoU+kq/hk9YcMosmZfP7EV2ZvwMSy7K3/gnzVrMW581u1+iZ > FlHWCNAA4kr4RtW7PIuB3obTtKYnQhL122TdgRIvtz4CPijW35E5pVA4JeVPbkJO > UEf0CitjXkCW5w5utcnxudAOLFJHVdXxrC7/HltRdb6BQYoO97UVlN+vhW46s/jl > GDFYoDUhAb5CsJGzaVCIJN4woFg/1v7QviHSw2ruStbnbP4xbwO8jTCAQ5DwDH8p > por5dcOuv+CQCoVtNtvxAGhqdUeWm1EG7Vf3cYVh/+bj2fmL1zKd+Dvzp1PUK3RO > E3pOsI5nPVHhCz5na/YdRStZ9oZiTlhZCb9sP0Jkwc6S6PReq2v0M6LWX1Ol1JJb > 049K+sZo3RJjZqLX0Bu3hEm+nhJ8r157DebuoMPZc/BekB8OLPHfJ5AV91Yite4B > Nsl2RX/Antke+Jd02GwMLii2ctkwwV1Q5wbJRLMcbOOUEqBighFut3v8G4s5SKgX > CC51gOPzzLipp334kaaFk8U/s6Uuv04A6iGe5uUVAvO/FARVYerK3cY3bYQlh5Tm > gC9iouAImUpxmwZlAy6vHhhH3gqF/avmexHt3Q5kfxk/G19cZL4nYaY+XQPVoc4E > dx5K2/lwtB67Y/yc7rEQ > =7HHm > -----END PGP SIGNATURE----- > -- *???? ??????????,???? ??? ????? ????www.limudnaim.co.il ??? ????? ?????? ??????* -------------- next part -------------- An HTML attachment was scrubbed... URL: From geoff at uplex.de Mon Dec 28 10:57:28 2015 From: geoff at uplex.de (Geoff Simmons) Date: Mon, 28 Dec 2015 11:57:28 +0100 Subject: why i get response if it is a post request with pass? In-Reply-To: References: <56810C30.8040904@uplex.de> Message-ID: <56811598.9080203@uplex.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 12/28/15 11:30 AM, ???? ?????????? wrote: > But if I have the following in my vcl file: .first_byte_timeout = > 600s; .between_bytes_timeout = 600s; > > Doesn't varnish suppose to respect it? In that case it wasn't a first byte timeout. Possibly something is causing a network disconnect if there is no activity for 5 seconds. That could be something in your Apache config, or for example a network device such as a firewall. Some advice: don't set your Varnish timeouts to 10 minutes, that's astronomical, and will very likely be counterproductive. This is a typical point of contention between backend app developers and the Varnish admin -- if the apps get slow, they'll want Varnish to "wait forever". But if the apps take that long, one of your Varnish worker threads is blocked for up to 10 minutes. If the app is in serious trouble so that it takes that long for all requests, all of those threads will block, and eventually you'll hit the worker thread maximum and Varnish will start refusing connections. No amount of increasing the thread pool can save you when the timeouts are that long. If the apps are that slow, you have a catastrophe no matter what happens, and you're better off having Varnish time out within a few seconds, so that at least Varnish remains stable. And the developers will have to bite the bullet and fix the app. > Where can I see how long the apache took to respond? The Apache mod_log_config module has formatters for that, for use in the LogFormat directive; but that's no longer a Varnish question. The varnishlog ReqEnd record will tell you how long Varnish waited, look here: https://www.varnish-cache.org/trac/wiki/Varnishlog Best, Geoff - -- UPLEX Systemoptimierung Scheffelstra?e 32 22301 Hamburg http://uplex.de/ Mob: +49-176-63690917 -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.14 (Darwin) iQIcBAEBCAAGBQJWgRWWAAoJEOUwvh9pJNUR+OcQAIH/LY/4iHD7M3jx7U0RsCfS JJjNREkCGylWOC6aQfznUZpkhMTeyRrlPpCf4isEFi0cb7XJG1Y32g+fSPeXIC9K PrGs52uslW3eIbI+Ao7Y5ozs+yCRdbWRpobBS7HXa3aj9Ekc3mCrwIh7qAOkfJIY SGVyEPtsYZx/l8XiMhkaPBZkByHePgz+vYU4+EGf5CtAyE6GfGfLFtHd/9o2QuV2 rLPRmr8JmpDJg+Lg0rDBgQziiMfFjvAmAeuEJi9yNu1iNUhlkIPuS8eW0JWWDsDu 9A35gj1Fnip8XtUhZqrRqYX6+zFb6duCZvpj+EHRMMoaXFtuIbsQ/+gyUby8VDGG rAh7V0fOyHt1bGHvgLihZBY0htK1neP5IjiRBUn9DZr8yktfJN44/Ch6QHeTOrb5 PlgZMWKQ3Hgir0PnPhvmT9phR2e90OU6Qv9I8i8i2/crx1EFmrzUH4g0niaUhH/9 j3tiDjmRQJyf0bmcvdCaFW/30A5jsd7phRzDIL8FsmjT5yitSAIcgrjr2g0Cz4wu tIbf0WkJuynd+3cbODqc5MuxLAexlLOkfMZjm3vsxqI6Q24BifX4RUSSwaIsrfqs gHXhObHhlHkMAHqexvp6xVL6lguB4AXdDknfs6rZsPIRtimNzqtlL7V+69AKSG5d TXmnai99dOK6/AH8L8DR =+HvO -----END PGP SIGNATURE----- From yakov.yarmo at gmail.com Mon Dec 28 11:09:12 2015 From: yakov.yarmo at gmail.com (=?UTF-8?B?15nXoten15Eg15nXqNee15XXnNeV15HXoden15k=?=) Date: Mon, 28 Dec 2015 13:09:12 +0200 Subject: why i get response if it is a post request with pass? In-Reply-To: <56811598.9080203@uplex.de> References: <56810C30.8040904@uplex.de> <56811598.9080203@uplex.de> Message-ID: is there any other variable that controls the 5 seconds disconnection from varnish? Is it possible that there is a bug in varnish that ignores first_byte_timeout? On Mon, Dec 28, 2015 at 12:57 PM, Geoff Simmons wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > On 12/28/15 11:30 AM, ???? ?????????? wrote: > > But if I have the following in my vcl file: .first_byte_timeout = > > 600s; .between_bytes_timeout = 600s; > > > > Doesn't varnish suppose to respect it? > > In that case it wasn't a first byte timeout. Possibly something is > causing a network disconnect if there is no activity for 5 seconds. > That could be something in your Apache config, or for example a > network device such as a firewall. > > Some advice: don't set your Varnish timeouts to 10 minutes, that's > astronomical, and will very likely be counterproductive. This is a > typical point of contention between backend app developers and the > Varnish admin -- if the apps get slow, they'll want Varnish to "wait > forever". But if the apps take that long, one of your Varnish worker > threads is blocked for up to 10 minutes. If the app is in serious > trouble so that it takes that long for all requests, all of those > threads will block, and eventually you'll hit the worker thread > maximum and Varnish will start refusing connections. No amount of > increasing the thread pool can save you when the timeouts are that long. > > If the apps are that slow, you have a catastrophe no matter what > happens, and you're better off having Varnish time out within a few > seconds, so that at least Varnish remains stable. And the developers > will have to bite the bullet and fix the app. > > > Where can I see how long the apache took to respond? > > The Apache mod_log_config module has formatters for that, for use in > the LogFormat directive; but that's no longer a Varnish question. > > The varnishlog ReqEnd record will tell you how long Varnish waited, > look here: > > https://www.varnish-cache.org/trac/wiki/Varnishlog > > > Best, > Geoff > - -- > UPLEX Systemoptimierung > Scheffelstra?e 32 > 22301 Hamburg > http://uplex.de/ > Mob: +49-176-63690917 > -----BEGIN PGP SIGNATURE----- > Version: GnuPG/MacGPG2 v2.0.14 (Darwin) > > iQIcBAEBCAAGBQJWgRWWAAoJEOUwvh9pJNUR+OcQAIH/LY/4iHD7M3jx7U0RsCfS > JJjNREkCGylWOC6aQfznUZpkhMTeyRrlPpCf4isEFi0cb7XJG1Y32g+fSPeXIC9K > PrGs52uslW3eIbI+Ao7Y5ozs+yCRdbWRpobBS7HXa3aj9Ekc3mCrwIh7qAOkfJIY > SGVyEPtsYZx/l8XiMhkaPBZkByHePgz+vYU4+EGf5CtAyE6GfGfLFtHd/9o2QuV2 > rLPRmr8JmpDJg+Lg0rDBgQziiMfFjvAmAeuEJi9yNu1iNUhlkIPuS8eW0JWWDsDu > 9A35gj1Fnip8XtUhZqrRqYX6+zFb6duCZvpj+EHRMMoaXFtuIbsQ/+gyUby8VDGG > rAh7V0fOyHt1bGHvgLihZBY0htK1neP5IjiRBUn9DZr8yktfJN44/Ch6QHeTOrb5 > PlgZMWKQ3Hgir0PnPhvmT9phR2e90OU6Qv9I8i8i2/crx1EFmrzUH4g0niaUhH/9 > j3tiDjmRQJyf0bmcvdCaFW/30A5jsd7phRzDIL8FsmjT5yitSAIcgrjr2g0Cz4wu > tIbf0WkJuynd+3cbODqc5MuxLAexlLOkfMZjm3vsxqI6Q24BifX4RUSSwaIsrfqs > gHXhObHhlHkMAHqexvp6xVL6lguB4AXdDknfs6rZsPIRtimNzqtlL7V+69AKSG5d > TXmnai99dOK6/AH8L8DR > =+HvO > -----END PGP SIGNATURE----- > -- *???? ??????????,???? ??? ????? ????www.limudnaim.co.il ??? ????? ?????? ??????* -------------- next part -------------- An HTML attachment was scrubbed... URL: From hernan at cmsmedios.com Mon Dec 28 11:29:24 2015 From: hernan at cmsmedios.com (=?UTF-8?Q?Hern=C3=A1n_Marsili?=) Date: Mon, 28 Dec 2015 11:29:24 +0000 Subject: using http/2 with varnish In-Reply-To: References: <4E272C03-196F-4010-95E3-088D1AF1F452@nucleus.be> Message-ID: Thank you very much. Is there an estimate time for h2 support? (first quarter 2016, mid-year, etc?). Just to have a notion. Best regards, Hern?n. On Mon, Dec 28, 2015 at 5:48 AM Guillaume Quintard < guillaume at varnish-software.com> wrote: > Hi, > > For a more detailed answer : we don't support H/2 in varnish yet (working > on it!). So, if you really really want H/2, having nginx in front of > varnish can be a solution. > > If you are only interested in https, however, varnish 4.1 and onward > supports the proxy protocol. It will allow to use and SSL/TLS terminator > such as hitch or haproxy that will handle the encryption for you. > > The advantage to using the proxy protocol is that varnish is aware of it. > If you use nginx to proxy the requests, varnish will only see one client: > nginx. This means you'll have to do some gymnastics with XFF headers if you > want to filter by ip address for example. Plus, nginx is a bit overkill in > terms of resources to just be a tls terminator. > > Migrating to varnish 4 requires a bit of work (not that much, really), but > it's worth it, especially considering v3 is EOL. > > -- > Guillaume Quintard > > On Sun, Dec 27, 2015 at 9:27 PM, Mattias Geniar > wrote: > >> > Can anyone point us on the right direction here? >> >> What you need is a reverse proxy in front of your Varnish instances: >> consider running a tool like Nginx (which has HTTP/2 support in its >> mainline repositories) that does all your TLS connections and proxies the >> request on to Varnish, to keep optimising the cache. >> >> Mattias >> > _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Mon Dec 28 13:15:08 2015 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Mon, 28 Dec 2015 14:15:08 +0100 Subject: using http/2 with varnish In-Reply-To: References: <4E272C03-196F-4010-95E3-088D1AF1F452@nucleus.be> Message-ID: I think our goal is mid 2016, hopefully sooner, but don't take that as a promise. -- Guillaume Quintard On Mon, Dec 28, 2015 at 12:29 PM, Hern?n Marsili wrote: > Thank you very much. Is there an estimate time for h2 support? (first > quarter 2016, mid-year, etc?). Just to have a notion. > > Best regards, > Hern?n. > > > On Mon, Dec 28, 2015 at 5:48 AM Guillaume Quintard < > guillaume at varnish-software.com> wrote: > >> Hi, >> >> For a more detailed answer : we don't support H/2 in varnish yet (working >> on it!). So, if you really really want H/2, having nginx in front of >> varnish can be a solution. >> >> If you are only interested in https, however, varnish 4.1 and onward >> supports the proxy protocol. It will allow to use and SSL/TLS terminator >> such as hitch or haproxy that will handle the encryption for you. >> >> The advantage to using the proxy protocol is that varnish is aware of it. >> If you use nginx to proxy the requests, varnish will only see one client: >> nginx. This means you'll have to do some gymnastics with XFF headers if you >> want to filter by ip address for example. Plus, nginx is a bit overkill in >> terms of resources to just be a tls terminator. >> >> Migrating to varnish 4 requires a bit of work (not that much, really), >> but it's worth it, especially considering v3 is EOL. >> >> -- >> Guillaume Quintard >> >> On Sun, Dec 27, 2015 at 9:27 PM, Mattias Geniar >> wrote: >> >>> > Can anyone point us on the right direction here? >>> >>> What you need is a reverse proxy in front of your Varnish instances: >>> consider running a tool like Nginx (which has HTTP/2 support in its >>> mainline repositories) that does all your TLS connections and proxies the >>> request on to Varnish, to keep optimising the cache. >>> >>> Mattias >>> >> _______________________________________________ >>> varnish-misc mailing list >>> varnish-misc at varnish-cache.org >>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >>> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From yakov.yarmo at gmail.com Mon Dec 28 14:36:47 2015 From: yakov.yarmo at gmail.com (=?UTF-8?B?15nXoten15Eg15nXqNee15XXnNeV15HXoden15k=?=) Date: Mon, 28 Dec 2015 16:36:47 +0200 Subject: why i get response if it is a post request with pass? In-Reply-To: References: <56810C30.8040904@uplex.de> <56811598.9080203@uplex.de> Message-ID: Will increasing sess_timeout will help? On Mon, Dec 28, 2015 at 1:09 PM, ???? ?????????? wrote: > is there any other variable that controls the 5 seconds disconnection from > varnish? > Is it possible that there is a bug in varnish that ignores > first_byte_timeout? > > On Mon, Dec 28, 2015 at 12:57 PM, Geoff Simmons wrote: > >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA256 >> >> On 12/28/15 11:30 AM, ???? ?????????? wrote: >> > But if I have the following in my vcl file: .first_byte_timeout = >> > 600s; .between_bytes_timeout = 600s; >> > >> > Doesn't varnish suppose to respect it? >> >> In that case it wasn't a first byte timeout. Possibly something is >> causing a network disconnect if there is no activity for 5 seconds. >> That could be something in your Apache config, or for example a >> network device such as a firewall. >> >> Some advice: don't set your Varnish timeouts to 10 minutes, that's >> astronomical, and will very likely be counterproductive. This is a >> typical point of contention between backend app developers and the >> Varnish admin -- if the apps get slow, they'll want Varnish to "wait >> forever". But if the apps take that long, one of your Varnish worker >> threads is blocked for up to 10 minutes. If the app is in serious >> trouble so that it takes that long for all requests, all of those >> threads will block, and eventually you'll hit the worker thread >> maximum and Varnish will start refusing connections. No amount of >> increasing the thread pool can save you when the timeouts are that long. >> >> If the apps are that slow, you have a catastrophe no matter what >> happens, and you're better off having Varnish time out within a few >> seconds, so that at least Varnish remains stable. And the developers >> will have to bite the bullet and fix the app. >> >> > Where can I see how long the apache took to respond? >> >> The Apache mod_log_config module has formatters for that, for use in >> the LogFormat directive; but that's no longer a Varnish question. >> >> The varnishlog ReqEnd record will tell you how long Varnish waited, >> look here: >> >> https://www.varnish-cache.org/trac/wiki/Varnishlog >> >> >> Best, >> Geoff >> - -- >> UPLEX Systemoptimierung >> Scheffelstra?e 32 >> 22301 Hamburg >> http://uplex.de/ >> Mob: +49-176-63690917 >> -----BEGIN PGP SIGNATURE----- >> Version: GnuPG/MacGPG2 v2.0.14 (Darwin) >> >> iQIcBAEBCAAGBQJWgRWWAAoJEOUwvh9pJNUR+OcQAIH/LY/4iHD7M3jx7U0RsCfS >> JJjNREkCGylWOC6aQfznUZpkhMTeyRrlPpCf4isEFi0cb7XJG1Y32g+fSPeXIC9K >> PrGs52uslW3eIbI+Ao7Y5ozs+yCRdbWRpobBS7HXa3aj9Ekc3mCrwIh7qAOkfJIY >> SGVyEPtsYZx/l8XiMhkaPBZkByHePgz+vYU4+EGf5CtAyE6GfGfLFtHd/9o2QuV2 >> rLPRmr8JmpDJg+Lg0rDBgQziiMfFjvAmAeuEJi9yNu1iNUhlkIPuS8eW0JWWDsDu >> 9A35gj1Fnip8XtUhZqrRqYX6+zFb6duCZvpj+EHRMMoaXFtuIbsQ/+gyUby8VDGG >> rAh7V0fOyHt1bGHvgLihZBY0htK1neP5IjiRBUn9DZr8yktfJN44/Ch6QHeTOrb5 >> PlgZMWKQ3Hgir0PnPhvmT9phR2e90OU6Qv9I8i8i2/crx1EFmrzUH4g0niaUhH/9 >> j3tiDjmRQJyf0bmcvdCaFW/30A5jsd7phRzDIL8FsmjT5yitSAIcgrjr2g0Cz4wu >> tIbf0WkJuynd+3cbODqc5MuxLAexlLOkfMZjm3vsxqI6Q24BifX4RUSSwaIsrfqs >> gHXhObHhlHkMAHqexvp6xVL6lguB4AXdDknfs6rZsPIRtimNzqtlL7V+69AKSG5d >> TXmnai99dOK6/AH8L8DR >> =+HvO >> -----END PGP SIGNATURE----- >> > > > > -- > > > > *???? ??????????,???? ??? ????? ????www.limudnaim.co.il > ??? ????? ?????? ??????* > -- *???? ??????????,???? ??? ????? ????www.limudnaim.co.il ??? ????? ?????? ??????* -------------- next part -------------- An HTML attachment was scrubbed... URL: From subharaj.manna at gmail.com Tue Dec 29 06:29:20 2015 From: subharaj.manna at gmail.com (Debraj Manna) Date: Tue, 29 Dec 2015 11:59:20 +0530 Subject: Installing Varnis on Debian 7 Message-ID: Hi, I am trying to install Varnish 4 on Debian 7 as mentioned in this link . But on doing apt-get install varnish. I am getting the below error. root at SPK-D-0476:/home/debraj# apt-get install varnish Reading package lists... Done Building dependency tree Reading state information... Done Some packages could not be installed. This may mean that you have requested an impossible situation or if you are using the unstable distribution that some required packages have not yet been created or been moved out of Incoming. The following information may help to resolve the situation: The following packages have unmet dependencies: varnish : Depends: libc6 (>= 2.14) but 2.13-38+deb7u8 is to be installed Depends: libpcre3 (>= 1:8.35) but 1:8.30-5 is to be installed Depends: libvarnishapi1 (>= 4.1.0-1~jessie) but it is not going to be installed E: Unable to correct problems, you have held broken packages. I also tried aptitude but it is showing me the option to install varnish 3 but not varnish 4 root at SPK-D-0476:/home/debraj# aptitude install varnish The following NEW packages will be installed: varnish{b} 0 packages upgraded, 1 newly installed, 0 to remove and 40 not upgraded. Need to get 527 kB of archives. After unpacking 1,636 kB will be used. The following packages have unmet dependencies: varnish : Depends: libc6 (>= 2.14) but 2.13-38+deb7u8 is installed. Depends: libjemalloc1 (>= 2.1.1) but it is not going to be installed. Depends: libpcre3 (>= 1:8.35) but 1:8.30-5 is installed. Depends: libvarnishapi1 (>= 4.1.0-1~jessie) but it is not going to be installed. The following actions will resolve these dependencies: Keep the following packages at their current version: 1) varnish [Not Installed] Accept this solution? [Y/n/q/?] n The following actions will resolve these dependencies: Install the following packages: 1) libvarnishapi1 [3.0.2-2+deb7u1 (oldstable)] 2) varnish [3.0.2-2+deb7u1 (oldstable)] Accept this solution? [Y/n/q/?] Can some one let me know how can I install varnish 4 on Debian 7? Thanks, -------------- next part -------------- An HTML attachment was scrubbed... URL: From pprocacci at datapipe.com Tue Dec 29 10:12:56 2015 From: pprocacci at datapipe.com (Paul A. Procacci) Date: Tue, 29 Dec 2015 05:12:56 -0500 Subject: Installing Varnis on Debian 7 In-Reply-To: References: Message-ID: <20151229101256.GB3031@workvm.myhome> On Tue, Dec 29, 2015 at 11:59:20AM +0530, Debraj Manna wrote: > Hi, > I am trying to install Varnish 4 on Debian 7 as mentioned in this > [1]link. > But on doing apt-get install varnish. I am getting the below error. > root at SPK-D-0476:/home/debraj# apt-get install varnish > Reading package lists... Done > Building dependency tree > Reading state information... Done > Some packages could not be installed. This may mean that you have > requested an impossible situation or if you are using the unstable > distribution that some required packages have not yet been created > or been moved out of Incoming. > The following information may help to resolve the situation: > The following packages have unmet dependencies: > varnish : Depends: libc6 (>= 2.14) but 2.13-38+deb7u8 is to be > installed > Depends: libpcre3 (>= 1:8.35) but 1:8.30-5 is to be > installed > Depends: libvarnishapi1 (>= 4.1.0-1~jessie) but it is not > going to be installed > E: Unable to correct problems, you have held broken packages. > I also tried aptitude but it is showing me the option to install > varnish 3 but not varnish 4 > root at SPK-D-0476:/home/debraj# aptitude install varnish > The following NEW packages will be installed: > varnish{b} > 0 packages upgraded, 1 newly installed, 0 to remove and 40 not > upgraded. > Need to get 527 kB of archives. After unpacking 1,636 kB will be used. > The following packages have unmet dependencies: > varnish : Depends: libc6 (>= 2.14) but 2.13-38+deb7u8 is installed. > Depends: libjemalloc1 (>= 2.1.1) but it is not going to be > installed. > Depends: libpcre3 (>= 1:8.35) but 1:8.30-5 is installed. > Depends: libvarnishapi1 (>= 4.1.0-1~jessie) but it is not > going to be installed. > The following actions will resolve these dependencies: > Keep the following packages at their current version: > 1) varnish [Not Installed] > Accept this solution? [Y/n/q/?] n > The following actions will resolve these dependencies: > Install the following packages: > 1) libvarnishapi1 [3.0.2-2+deb7u1 (oldstable)] > 2) varnish [3.0.2-2+deb7u1 (oldstable)] > Accept this solution? [Y/n/q/?] > Can some one let me know how can I install varnish 4 on Debian 7? > Thanks, > Not sure about Debian, I'm a FreeBSD user myself, but when all else fails you can download the source and build your own rpm against the libs installed on your own system. ~Paul From magnus at hagander.net Tue Dec 29 10:18:55 2015 From: magnus at hagander.net (Magnus Hagander) Date: Tue, 29 Dec 2015 11:18:55 +0100 Subject: Installing Varnis on Debian 7 In-Reply-To: References: Message-ID: On Tue, Dec 29, 2015 at 7:29 AM, Debraj Manna wrote: > Hi, > > I am trying to install Varnish 4 on Debian 7 as mentioned in this link > . > As that link says, those instructions are for Debian 8, which is what causes dependency problems on Debian 7. You need to replace "jessie" with "wheezy" to install on Debian 7. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Tue Dec 29 10:37:24 2015 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Tue, 29 Dec 2015 11:37:24 +0100 Subject: Installing Varnis on Debian 7 In-Reply-To: <20151229101256.GB3031@workvm.myhome> References: <20151229101256.GB3031@workvm.myhome> Message-ID: Paul is onto something here (I'm on arch, but I still have some vague debian memories). You should be able to: apt-get build-dep varnish apt-get source varnish cd varnish dpkg-buildpackage Or something similar. -- Guillaume Quintard On Tue, Dec 29, 2015 at 11:12 AM, Paul A. Procacci wrote: > On Tue, Dec 29, 2015 at 11:59:20AM +0530, Debraj Manna wrote: > > Hi, > > I am trying to install Varnish 4 on Debian 7 as mentioned in this > > [1]link. > > But on doing apt-get install varnish. I am getting the below error. > > root at SPK-D-0476:/home/debraj# apt-get install varnish > > Reading package lists... Done > > Building dependency tree > > Reading state information... Done > > Some packages could not be installed. This may mean that you have > > requested an impossible situation or if you are using the unstable > > distribution that some required packages have not yet been created > > or been moved out of Incoming. > > The following information may help to resolve the situation: > > The following packages have unmet dependencies: > > varnish : Depends: libc6 (>= 2.14) but 2.13-38+deb7u8 is to be > > installed > > Depends: libpcre3 (>= 1:8.35) but 1:8.30-5 is to be > > installed > > Depends: libvarnishapi1 (>= 4.1.0-1~jessie) but it is not > > going to be installed > > E: Unable to correct problems, you have held broken packages. > > I also tried aptitude but it is showing me the option to install > > varnish 3 but not varnish 4 > > root at SPK-D-0476:/home/debraj# aptitude install varnish > > The following NEW packages will be installed: > > varnish{b} > > 0 packages upgraded, 1 newly installed, 0 to remove and 40 not > > upgraded. > > Need to get 527 kB of archives. After unpacking 1,636 kB will be used. > > The following packages have unmet dependencies: > > varnish : Depends: libc6 (>= 2.14) but 2.13-38+deb7u8 is installed. > > Depends: libjemalloc1 (>= 2.1.1) but it is not going to be > > installed. > > Depends: libpcre3 (>= 1:8.35) but 1:8.30-5 is installed. > > Depends: libvarnishapi1 (>= 4.1.0-1~jessie) but it is not > > going to be installed. > > The following actions will resolve these dependencies: > > Keep the following packages at their current version: > > 1) varnish [Not Installed] > > Accept this solution? [Y/n/q/?] n > > The following actions will resolve these dependencies: > > Install the following packages: > > 1) libvarnishapi1 [3.0.2-2+deb7u1 (oldstable)] > > 2) varnish [3.0.2-2+deb7u1 (oldstable)] > > Accept this solution? [Y/n/q/?] > > Can some one let me know how can I install varnish 4 on Debian 7? > > Thanks, > > > > Not sure about Debian, I'm a FreeBSD user myself, but when all else fails > you > can download the source and build your own rpm against the libs installed > on > your own system. > > ~Paul > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From yakov.yarmo at gmail.com Tue Dec 29 11:47:06 2015 From: yakov.yarmo at gmail.com (=?UTF-8?B?15nXoten15Eg15nXqNee15XXnNeV15HXoden15k=?=) Date: Tue, 29 Dec 2015 13:47:06 +0200 Subject: why i get response if it is a post request with pass? In-Reply-To: References: <56810C30.8040904@uplex.de> <56811598.9080203@uplex.de> Message-ID: I added to the apache log the the time taken to serve the request and it seems that the apache is serving the request but the varnish doesn't get it. Any clue why it can happen? On Mon, Dec 28, 2015 at 4:36 PM, ???? ?????????? wrote: > Will increasing sess_timeout will help? > > On Mon, Dec 28, 2015 at 1:09 PM, ???? ?????????? > wrote: > >> is there any other variable that controls the 5 seconds disconnection >> from varnish? >> Is it possible that there is a bug in varnish that ignores >> first_byte_timeout? >> >> On Mon, Dec 28, 2015 at 12:57 PM, Geoff Simmons wrote: >> >>> -----BEGIN PGP SIGNED MESSAGE----- >>> Hash: SHA256 >>> >>> On 12/28/15 11:30 AM, ???? ?????????? wrote: >>> > But if I have the following in my vcl file: .first_byte_timeout = >>> > 600s; .between_bytes_timeout = 600s; >>> > >>> > Doesn't varnish suppose to respect it? >>> >>> In that case it wasn't a first byte timeout. Possibly something is >>> causing a network disconnect if there is no activity for 5 seconds. >>> That could be something in your Apache config, or for example a >>> network device such as a firewall. >>> >>> Some advice: don't set your Varnish timeouts to 10 minutes, that's >>> astronomical, and will very likely be counterproductive. This is a >>> typical point of contention between backend app developers and the >>> Varnish admin -- if the apps get slow, they'll want Varnish to "wait >>> forever". But if the apps take that long, one of your Varnish worker >>> threads is blocked for up to 10 minutes. If the app is in serious >>> trouble so that it takes that long for all requests, all of those >>> threads will block, and eventually you'll hit the worker thread >>> maximum and Varnish will start refusing connections. No amount of >>> increasing the thread pool can save you when the timeouts are that long. >>> >>> If the apps are that slow, you have a catastrophe no matter what >>> happens, and you're better off having Varnish time out within a few >>> seconds, so that at least Varnish remains stable. And the developers >>> will have to bite the bullet and fix the app. >>> >>> > Where can I see how long the apache took to respond? >>> >>> The Apache mod_log_config module has formatters for that, for use in >>> the LogFormat directive; but that's no longer a Varnish question. >>> >>> The varnishlog ReqEnd record will tell you how long Varnish waited, >>> look here: >>> >>> https://www.varnish-cache.org/trac/wiki/Varnishlog >>> >>> >>> Best, >>> Geoff >>> - -- >>> UPLEX Systemoptimierung >>> Scheffelstra?e 32 >>> 22301 Hamburg >>> http://uplex.de/ >>> Mob: +49-176-63690917 >>> -----BEGIN PGP SIGNATURE----- >>> Version: GnuPG/MacGPG2 v2.0.14 (Darwin) >>> >>> iQIcBAEBCAAGBQJWgRWWAAoJEOUwvh9pJNUR+OcQAIH/LY/4iHD7M3jx7U0RsCfS >>> JJjNREkCGylWOC6aQfznUZpkhMTeyRrlPpCf4isEFi0cb7XJG1Y32g+fSPeXIC9K >>> PrGs52uslW3eIbI+Ao7Y5ozs+yCRdbWRpobBS7HXa3aj9Ekc3mCrwIh7qAOkfJIY >>> SGVyEPtsYZx/l8XiMhkaPBZkByHePgz+vYU4+EGf5CtAyE6GfGfLFtHd/9o2QuV2 >>> rLPRmr8JmpDJg+Lg0rDBgQziiMfFjvAmAeuEJi9yNu1iNUhlkIPuS8eW0JWWDsDu >>> 9A35gj1Fnip8XtUhZqrRqYX6+zFb6duCZvpj+EHRMMoaXFtuIbsQ/+gyUby8VDGG >>> rAh7V0fOyHt1bGHvgLihZBY0htK1neP5IjiRBUn9DZr8yktfJN44/Ch6QHeTOrb5 >>> PlgZMWKQ3Hgir0PnPhvmT9phR2e90OU6Qv9I8i8i2/crx1EFmrzUH4g0niaUhH/9 >>> j3tiDjmRQJyf0bmcvdCaFW/30A5jsd7phRzDIL8FsmjT5yitSAIcgrjr2g0Cz4wu >>> tIbf0WkJuynd+3cbODqc5MuxLAexlLOkfMZjm3vsxqI6Q24BifX4RUSSwaIsrfqs >>> gHXhObHhlHkMAHqexvp6xVL6lguB4AXdDknfs6rZsPIRtimNzqtlL7V+69AKSG5d >>> TXmnai99dOK6/AH8L8DR >>> =+HvO >>> -----END PGP SIGNATURE----- >>> >> >> >> >> -- >> >> >> >> *???? ??????????,???? ??? ????? ????www.limudnaim.co.il >> ??? ????? ?????? ??????* >> > > > > -- > > > > *???? ??????????,???? ??? ????? ????www.limudnaim.co.il > ??? ????? ?????? ??????* > -- *???? ??????????,???? ??? ????? ????www.limudnaim.co.il ??? ????? ?????? ??????* -------------- next part -------------- An HTML attachment was scrubbed... URL: From subharaj.manna at gmail.com Tue Dec 29 14:49:41 2015 From: subharaj.manna at gmail.com (Debraj Manna) Date: Tue, 29 Dec 2015 20:19:41 +0530 Subject: Installing Varnis on Debian 7 In-Reply-To: References: Message-ID: Thanks Magnus for pointing that out. On Dec 29, 2015 3:48 PM, "Magnus Hagander" wrote: > > > On Tue, Dec 29, 2015 at 7:29 AM, Debraj Manna > wrote: > >> Hi, >> >> I am trying to install Varnish 4 on Debian 7 as mentioned in this link >> . >> > > As that link says, those instructions are for Debian 8, which is what > causes dependency problems on Debian 7. You need to replace "jessie" with > "wheezy" to install on Debian 7. > > -- > Magnus Hagander > Me: http://www.hagander.net/ > Work: http://www.redpill-linpro.com/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From japrice at gmail.com Tue Dec 29 17:47:43 2015 From: japrice at gmail.com (Jason Price) Date: Tue, 29 Dec 2015 12:47:43 -0500 Subject: why i get response if it is a post request with pass? In-Reply-To: References: <56810C30.8040904@uplex.de> <56811598.9080203@uplex.de> Message-ID: sorry... forgot to include the list: Each backend definition has a setting for first_byte_timeout, between_byte_timeout and one other that I'm forgetting. IIRC, varnish defaults to a 5 second first byte timeout. This line indicates that Varnish waited 5 seconds before doing it's thing: 17 ReqEnd c 2005611725 1451213252.184271336 1451213257.219164610 0.000056505 *5.034824371* 0.000068903 Read the docs about backend timeout settings and go from there. On Tue, Dec 29, 2015 at 6:47 AM, ???? ?????????? wrote: > I added to the apache log the the time taken to serve the request and it > seems that the apache is serving the request but the varnish doesn't get it. > Any clue why it can happen? > > On Mon, Dec 28, 2015 at 4:36 PM, ???? ?????????? > wrote: > >> Will increasing sess_timeout will help? >> >> On Mon, Dec 28, 2015 at 1:09 PM, ???? ?????????? >> wrote: >> >>> is there any other variable that controls the 5 seconds disconnection >>> from varnish? >>> Is it possible that there is a bug in varnish that ignores >>> first_byte_timeout? >>> >>> On Mon, Dec 28, 2015 at 12:57 PM, Geoff Simmons wrote: >>> >>>> -----BEGIN PGP SIGNED MESSAGE----- >>>> Hash: SHA256 >>>> >>>> On 12/28/15 11:30 AM, ???? ?????????? wrote: >>>> > But if I have the following in my vcl file: .first_byte_timeout = >>>> > 600s; .between_bytes_timeout = 600s; >>>> > >>>> > Doesn't varnish suppose to respect it? >>>> >>>> In that case it wasn't a first byte timeout. Possibly something is >>>> causing a network disconnect if there is no activity for 5 seconds. >>>> That could be something in your Apache config, or for example a >>>> network device such as a firewall. >>>> >>>> Some advice: don't set your Varnish timeouts to 10 minutes, that's >>>> astronomical, and will very likely be counterproductive. This is a >>>> typical point of contention between backend app developers and the >>>> Varnish admin -- if the apps get slow, they'll want Varnish to "wait >>>> forever". But if the apps take that long, one of your Varnish worker >>>> threads is blocked for up to 10 minutes. If the app is in serious >>>> trouble so that it takes that long for all requests, all of those >>>> threads will block, and eventually you'll hit the worker thread >>>> maximum and Varnish will start refusing connections. No amount of >>>> increasing the thread pool can save you when the timeouts are that long. >>>> >>>> If the apps are that slow, you have a catastrophe no matter what >>>> happens, and you're better off having Varnish time out within a few >>>> seconds, so that at least Varnish remains stable. And the developers >>>> will have to bite the bullet and fix the app. >>>> >>>> > Where can I see how long the apache took to respond? >>>> >>>> The Apache mod_log_config module has formatters for that, for use in >>>> the LogFormat directive; but that's no longer a Varnish question. >>>> >>>> The varnishlog ReqEnd record will tell you how long Varnish waited, >>>> look here: >>>> >>>> https://www.varnish-cache.org/trac/wiki/Varnishlog >>>> >>>> >>>> Best, >>>> Geoff >>>> - -- >>>> UPLEX Systemoptimierung >>>> Scheffelstra?e 32 >>>> 22301 Hamburg >>>> http://uplex.de/ >>>> Mob: +49-176-63690917 >>>> -----BEGIN PGP SIGNATURE----- >>>> Version: GnuPG/MacGPG2 v2.0.14 (Darwin) >>>> >>>> iQIcBAEBCAAGBQJWgRWWAAoJEOUwvh9pJNUR+OcQAIH/LY/4iHD7M3jx7U0RsCfS >>>> JJjNREkCGylWOC6aQfznUZpkhMTeyRrlPpCf4isEFi0cb7XJG1Y32g+fSPeXIC9K >>>> PrGs52uslW3eIbI+Ao7Y5ozs+yCRdbWRpobBS7HXa3aj9Ekc3mCrwIh7qAOkfJIY >>>> SGVyEPtsYZx/l8XiMhkaPBZkByHePgz+vYU4+EGf5CtAyE6GfGfLFtHd/9o2QuV2 >>>> rLPRmr8JmpDJg+Lg0rDBgQziiMfFjvAmAeuEJi9yNu1iNUhlkIPuS8eW0JWWDsDu >>>> 9A35gj1Fnip8XtUhZqrRqYX6+zFb6duCZvpj+EHRMMoaXFtuIbsQ/+gyUby8VDGG >>>> rAh7V0fOyHt1bGHvgLihZBY0htK1neP5IjiRBUn9DZr8yktfJN44/Ch6QHeTOrb5 >>>> PlgZMWKQ3Hgir0PnPhvmT9phR2e90OU6Qv9I8i8i2/crx1EFmrzUH4g0niaUhH/9 >>>> j3tiDjmRQJyf0bmcvdCaFW/30A5jsd7phRzDIL8FsmjT5yitSAIcgrjr2g0Cz4wu >>>> tIbf0WkJuynd+3cbODqc5MuxLAexlLOkfMZjm3vsxqI6Q24BifX4RUSSwaIsrfqs >>>> gHXhObHhlHkMAHqexvp6xVL6lguB4AXdDknfs6rZsPIRtimNzqtlL7V+69AKSG5d >>>> TXmnai99dOK6/AH8L8DR >>>> =+HvO >>>> -----END PGP SIGNATURE----- >>>> >>> >>> >>> >>> -- >>> >>> >>> >>> *???? ??????????,???? ??? ????? ????www.limudnaim.co.il >>> ??? ????? ?????? ??????* >>> >> >> >> >> -- >> >> >> >> *???? ??????????,???? ??? ????? ????www.limudnaim.co.il >> ??? ????? ?????? ??????* >> > > > > -- > > > > *???? ??????????,???? ??? ????? ????www.limudnaim.co.il > ??? ????? ?????? ??????* > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From subharaj.manna at gmail.com Wed Dec 30 09:27:18 2015 From: subharaj.manna at gmail.com (Debraj Manna) Date: Wed, 30 Dec 2015 14:57:18 +0530 Subject: Varnish 500 Backend Fetch Failed when varnish is started as service Message-ID: Hi, I am new to varnish. Whenever I am trying to acess our backend via varnish I am getting the below error:- 503 Backend fetch failed

Error 503 Backend fetch failed

Backend fetch failed

Guru Meditation:

XID: 3

Varnish cache server

My vcl looks like below:- vcl 4.0; backend default { .host = "146.88.25.40"; .port = "8080"; } Directly requesting to the backend from the host in which varnish is installed works fine. I also tried setting .connect_timeout = 60s; first_byte_timeout = 60s; .between_bytes_timeout = 60s; But no luck. varnishlog looks something like below:- * << BeReq >> 3 - Begin bereq 2 fetch - Timestamp Start: 1451466805.152313 0.000000 0.000000 - BereqMethod GET - BereqURL /rde_server/res/feea8d0b3d50/recomm/home/sid/jb55653bbd980baaf37484ef70bcfcef9b84d7c680ad1b487aed1fb21ac6779025?home_tsFilter=&cat_tsFilter=%28conc.item.brand+not+in+%2836%2C4187%2C4201%2C3721%2C4103%2C4101%2C4199%2C4075%29%29%28conc.item.param1+in+%28+a - BereqProtocol HTTP/1.1 - BereqHeader User-Agent: curl/7.38.0 - BereqHeader Host: localhost:6081 - BereqHeader Accept: */* - BereqHeader X-Forwarded-For: ::1 - BereqHeader Accept-Encoding: gzip - BereqHeader X-Varnish: 3 - VCL_call BACKEND_FETCH - VCL_return fetch - FetchError no backend connection - Timestamp Beresp: 1451466805.152502 0.000189 0.000189 - Timestamp Error: 1451466805.152511 0.000198 0.000009 - BerespProtocol HTTP/1.1 - BerespStatus 503 - BerespReason Service Unavailable - BerespReason Backend fetch failed - BerespHeader Date: Wed, 30 Dec 2015 09:13:25 GMT - BerespHeader Server: Varnish - VCL_call BACKEND_ERROR - BerespHeader Content-Type: text/html; charset=utf-8 - BerespHeader Retry-After: 5 - VCL_return deliver - Storage malloc Transient - ObjProtocol HTTP/1.1 - ObjStatus 503 - ObjReason Backend fetch failed - ObjHeader Date: Wed, 30 Dec 2015 09:13:25 GMT - ObjHeader Server: Varnish - ObjHeader Content-Type: text/html; charset=utf-8 - ObjHeader Retry-After: 5 - Length 278 - BereqAcct 0 0 0 0 0 0 - End * << Request >> 2 - Begin req 1 rxreq - Timestamp Start: 1451466805.152189 0.000000 0.000000 - Timestamp Req: 1451466805.152189 0.000000 0.000000 - ReqStart ::1 44193 - ReqMethod GET - ReqURL /rde_server/res/feea8d0b3d50/recomm/home/sid/jb55653bbd980baaf37484ef70bcfcef9b84d7c680ad1b487aed1fb21ac6779025?home_tsFilter=&cat_tsFilter=%28conc.item.brand+not+in+%2836%2C4187%2C4201%2C3721%2C4103%2C4101%2C4199%2C4075%29%29%28conc.item.param1+in+%28+a - ReqProtocol HTTP/1.1 - ReqHeader User-Agent: curl/7.38.0 - ReqHeader Host: localhost:6081 - ReqHeader Accept: */* - ReqHeader X-Forwarded-For: ::1 - VCL_call RECV - VCL_return hash - VCL_call HASH - VCL_return lookup - VCL_call MISS - VCL_return fetch - Link bereq 3 fetch - Timestamp Fetch: 1451466805.152650 0.000461 0.000461 - RespProtocol HTTP/1.1 - RespStatus 503 - RespReason Backend fetch failed - RespHeader Date: Wed, 30 Dec 2015 09:13:25 GMT - RespHeader Server: Varnish - RespHeader Content-Type: text/html; charset=utf-8 - RespHeader Retry-After: 5 - RespHeader X-Varnish: 2 - RespHeader Age: 0 - RespHeader Via: 1.1 varnish-v4 - VCL_call DELIVER - VCL_return deliver - Timestamp Process: 1451466805.152718 0.000529 0.000068 - RespHeader Content-Length: 278 - Debug "RES_MODE 2" - RespHeader Connection: keep-alive - Timestamp Resp: 1451466805.152773 0.000584 0.000055 - ReqAcct 386 0 386 235 278 513 - End * << Session >> 1 - Begin sess 0 HTTP/1 - SessOpen ::1 44193 :6081 ::1 6081 1451466805.152117 14 - Link req 2 rxreq - SessClose REM_CLOSE 0.001 - End If I start varnish as below then also everything works fine. I am facing problem only when I start varnish via sudo service varnish start sudo varnishd -n foo -f /etc/varnish/prudsys.vcl -s malloc,1G -T localhost:6082 -a 0.0.0.0:6081 Can some one let me know what is going wrong? I am using varnish 4.1 on Debian 8. -------------- next part -------------- An HTML attachment was scrubbed... URL: From geoff at uplex.de Wed Dec 30 13:39:17 2015 From: geoff at uplex.de (Geoff Simmons) Date: Wed, 30 Dec 2015 14:39:17 +0100 Subject: Varnish 500 Backend Fetch Failed when varnish is started as service In-Reply-To: References: Message-ID: <5683DE85.3070703@uplex.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 12/30/15 10:27 AM, Debraj Manna wrote: > > I am new to varnish. Whenever I am trying to acess our backend via > varnish I am getting the below error:- Always look for the FetchError: > - FetchError no backend connection Varnish can't reach your backend. Since there's no 'Backend' record anywhere in your log, Varnish evidently doesn't know about any backend that it can connect to. > If I start varnish as below then also everything works fine. I am > facing problem only when I start varnish via sudo service varnish > start sudo varnishd -n foo -f /etc/varnish/prudsys.vcl -s malloc,1G > -T localhost:6082 -a 0.0.0.0:6081 Take a look at the start script invoked by 'service'. If you haven't changed anything about it, I would guess that it's not using the VCL file specified by -f in your command, but rather a "default" VCL that defines a backend that doesn't exist. HTH, Geoff - -- UPLEX Systemoptimierung Scheffelstra?e 32 22301 Hamburg http://uplex.de/ Mob: +49-176-63690917 -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.14 (Darwin) iQIcBAEBCAAGBQJWg96EAAoJEOUwvh9pJNURmREQAIbYDmd1RuPuGECv1VUFAEL3 HflMfEBaSPQINQh+7RSJ5uj8Qvvi1QRabb5eX4XpyU8+GjVPHpOfUKWWav42W2rH 9rnhaT51lR6kegtnxpPMqoIdTaygYd4Vcy2zMhVsjraVDRdN3JPpuK90P12aP3bO oQYrk+W6JXsG2DkkcpjLCGYOKD7RDX6pYpkMdYP3cBV21fobJs5o12CIuZckHMDe r2NeM08ofjhRSskRiBeBguAaQVwALPmTfwVMhZyCEpvFUGVIRJlm9C2YDba/NuEL et1Z1cyONh2v672f1QPohFCYZMkj8DpzJev2QxR0XOM5anEkpfCG7hrsNawAZOMk 9W3cwiZ7+ChOOnoMHA4jYQTbH9og6wJXe8vmy0TjNoDqLF4apTUoqoi27dgHIS4o wtNSLlC3rN0e8EPZlbQ0R4Xy4yUHDbDrXc0UrSPcPahAde49V/ZUf0RI+WGSa7nF C63q/rJV2OCN9NbvmGYTwRu24asgqmAl6gMwlAXMdI2G6Y4b36HdiwChfVOzUL/H Sgat8T3iIl/rkYaHP6Vfdw7+N7kV+/8RjEr62xRmw5ZiRDAv/wBknT1tTaSPWv33 bMlFYDujYeEkcGeKJdMLGuKn/S9I+GJEbSk6V6E7v1LfGKiU18wT/7VKKVEvawCb LodcXaEdHwAuWbuHjaOF =HYy+ -----END PGP SIGNATURE----- From subharaj.manna at gmail.com Wed Dec 30 16:57:32 2015 From: subharaj.manna at gmail.com (Debraj Manna) Date: Wed, 30 Dec 2015 22:27:32 +0530 Subject: Varnish 500 Backend Fetch Failed when varnish is started as service In-Reply-To: <5683DE85.3070703@uplex.de> References: <5683DE85.3070703@uplex.de> Message-ID: I have changed /etc/default/varnish. It looks something like below:- START=yes NFILES=131072 MEMLOCK=82000 DAEMON_OPTS="-a 0.0.0.0:6081 \ -T localhost:6082 \ -f /etc/varnish/prudsys.vcl \ -S /etc/varnish/secret \ -s malloc,1G" My prudsys.vcl looks like below:- vcl 4.0; backend default { .host = "146.88.25.40"; .port = "8080"; } I have also attached my /etc/init.d/varnish script On Wed, Dec 30, 2015 at 7:09 PM, Geoff Simmons wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > On 12/30/15 10:27 AM, Debraj Manna wrote: > > > > I am new to varnish. Whenever I am trying to acess our backend via > > varnish I am getting the below error:- > > Always look for the FetchError: > > > - FetchError no backend connection > > Varnish can't reach your backend. Since there's no 'Backend' record > anywhere in your log, Varnish evidently doesn't know about any backend > that it can connect to. > > > If I start varnish as below then also everything works fine. I am > > facing problem only when I start varnish via sudo service varnish > > start sudo varnishd -n foo -f /etc/varnish/prudsys.vcl -s malloc,1G > > -T localhost:6082 -a 0.0.0.0:6081 > > Take a look at the start script invoked by 'service'. If you haven't > changed anything about it, I would guess that it's not using the VCL > file specified by -f in your command, but rather a "default" VCL that > defines a backend that doesn't exist. > > > HTH, > Geoff > - -- > UPLEX Systemoptimierung > Scheffelstra?e 32 > 22301 Hamburg > http://uplex.de/ > Mob: +49-176-63690917 > -----BEGIN PGP SIGNATURE----- > Version: GnuPG/MacGPG2 v2.0.14 (Darwin) > > iQIcBAEBCAAGBQJWg96EAAoJEOUwvh9pJNURmREQAIbYDmd1RuPuGECv1VUFAEL3 > HflMfEBaSPQINQh+7RSJ5uj8Qvvi1QRabb5eX4XpyU8+GjVPHpOfUKWWav42W2rH > 9rnhaT51lR6kegtnxpPMqoIdTaygYd4Vcy2zMhVsjraVDRdN3JPpuK90P12aP3bO > oQYrk+W6JXsG2DkkcpjLCGYOKD7RDX6pYpkMdYP3cBV21fobJs5o12CIuZckHMDe > r2NeM08ofjhRSskRiBeBguAaQVwALPmTfwVMhZyCEpvFUGVIRJlm9C2YDba/NuEL > et1Z1cyONh2v672f1QPohFCYZMkj8DpzJev2QxR0XOM5anEkpfCG7hrsNawAZOMk > 9W3cwiZ7+ChOOnoMHA4jYQTbH9og6wJXe8vmy0TjNoDqLF4apTUoqoi27dgHIS4o > wtNSLlC3rN0e8EPZlbQ0R4Xy4yUHDbDrXc0UrSPcPahAde49V/ZUf0RI+WGSa7nF > C63q/rJV2OCN9NbvmGYTwRu24asgqmAl6gMwlAXMdI2G6Y4b36HdiwChfVOzUL/H > Sgat8T3iIl/rkYaHP6Vfdw7+N7kV+/8RjEr62xRmw5ZiRDAv/wBknT1tTaSPWv33 > bMlFYDujYeEkcGeKJdMLGuKn/S9I+GJEbSk6V6E7v1LfGKiU18wT/7VKKVEvawCb > LodcXaEdHwAuWbuHjaOF > =HYy+ > -----END PGP SIGNATURE----- > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: varnish Type: application/octet-stream Size: 3535 bytes Desc: not available URL: From subharaj.manna at gmail.com Wed Dec 30 17:08:17 2015 From: subharaj.manna at gmail.com (Debraj Manna) Date: Wed, 30 Dec 2015 22:38:17 +0530 Subject: Varnish 500 Backend Fetch Failed when varnish is started as service In-Reply-To: References: <5683DE85.3070703@uplex.de> Message-ID: Geoff, You are right when I added my backend info in the /etc/varnish/default.vcl . It is working correctly. Can you let me know what is going wrong even though I have mentioned by vcl, prudsys.vcl in /etc/default/varnish. It is still picking the default vcl. My /etc/default/varnish looks like below:- START=yes NFILES=131072 MEMLOCK=82000 DAEMON_OPTS="-a 0.0.0.0:6081 \ -T localhost:6082 \ -f /etc/varnish/prudsys.vcl \ -S /etc/varnish/secret \ -s malloc,1G" On Wed, Dec 30, 2015 at 10:27 PM, Debraj Manna wrote: > I have changed /etc/default/varnish. It looks something like below:- > > START=yes > NFILES=131072 > MEMLOCK=82000 > DAEMON_OPTS="-a 0.0.0.0:6081 \ > -T localhost:6082 \ > -f /etc/varnish/prudsys.vcl \ > -S /etc/varnish/secret \ > -s malloc,1G" > > My prudsys.vcl looks like below:- > > vcl 4.0; > backend default { > .host = "146.88.25.40"; > .port = "8080"; > } > > I have also attached my /etc/init.d/varnish script > > > On Wed, Dec 30, 2015 at 7:09 PM, Geoff Simmons wrote: > >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA256 >> >> On 12/30/15 10:27 AM, Debraj Manna wrote: >> > >> > I am new to varnish. Whenever I am trying to acess our backend via >> > varnish I am getting the below error:- >> >> Always look for the FetchError: >> >> > - FetchError no backend connection >> >> Varnish can't reach your backend. Since there's no 'Backend' record >> anywhere in your log, Varnish evidently doesn't know about any backend >> that it can connect to. >> >> > If I start varnish as below then also everything works fine. I am >> > facing problem only when I start varnish via sudo service varnish >> > start sudo varnishd -n foo -f /etc/varnish/prudsys.vcl -s malloc,1G >> > -T localhost:6082 -a 0.0.0.0:6081 >> >> Take a look at the start script invoked by 'service'. If you haven't >> changed anything about it, I would guess that it's not using the VCL >> file specified by -f in your command, but rather a "default" VCL that >> defines a backend that doesn't exist. >> >> >> HTH, >> Geoff >> - -- >> UPLEX Systemoptimierung >> Scheffelstra?e 32 >> 22301 Hamburg >> http://uplex.de/ >> Mob: +49-176-63690917 >> -----BEGIN PGP SIGNATURE----- >> Version: GnuPG/MacGPG2 v2.0.14 (Darwin) >> >> iQIcBAEBCAAGBQJWg96EAAoJEOUwvh9pJNURmREQAIbYDmd1RuPuGECv1VUFAEL3 >> HflMfEBaSPQINQh+7RSJ5uj8Qvvi1QRabb5eX4XpyU8+GjVPHpOfUKWWav42W2rH >> 9rnhaT51lR6kegtnxpPMqoIdTaygYd4Vcy2zMhVsjraVDRdN3JPpuK90P12aP3bO >> oQYrk+W6JXsG2DkkcpjLCGYOKD7RDX6pYpkMdYP3cBV21fobJs5o12CIuZckHMDe >> r2NeM08ofjhRSskRiBeBguAaQVwALPmTfwVMhZyCEpvFUGVIRJlm9C2YDba/NuEL >> et1Z1cyONh2v672f1QPohFCYZMkj8DpzJev2QxR0XOM5anEkpfCG7hrsNawAZOMk >> 9W3cwiZ7+ChOOnoMHA4jYQTbH9og6wJXe8vmy0TjNoDqLF4apTUoqoi27dgHIS4o >> wtNSLlC3rN0e8EPZlbQ0R4Xy4yUHDbDrXc0UrSPcPahAde49V/ZUf0RI+WGSa7nF >> C63q/rJV2OCN9NbvmGYTwRu24asgqmAl6gMwlAXMdI2G6Y4b36HdiwChfVOzUL/H >> Sgat8T3iIl/rkYaHP6Vfdw7+N7kV+/8RjEr62xRmw5ZiRDAv/wBknT1tTaSPWv33 >> bMlFYDujYeEkcGeKJdMLGuKn/S9I+GJEbSk6V6E7v1LfGKiU18wT/7VKKVEvawCb >> LodcXaEdHwAuWbuHjaOF >> =HYy+ >> -----END PGP SIGNATURE----- >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From subharaj.manna at gmail.com Thu Dec 31 06:00:44 2015 From: subharaj.manna at gmail.com (Debraj Manna) Date: Thu, 31 Dec 2015 11:30:44 +0530 Subject: Varnish 500 Backend Fetch Failed when varnish is started as service In-Reply-To: References: <5683DE85.3070703@uplex.de> Message-ID: I was facing the issue as mentioned in this blog post . This should be documented properly in varnish docs. On Wed, Dec 30, 2015 at 10:38 PM, Debraj Manna wrote: > Geoff, > > You are right when I added my backend info in the /etc/varnish/default.vcl > . It is working correctly. Can you let me know what is going wrong even > though I have mentioned by vcl, prudsys.vcl in /etc/default/varnish. It > is still picking the default vcl. > > My /etc/default/varnish looks like below:- > > START=yes > NFILES=131072 > MEMLOCK=82000 > DAEMON_OPTS="-a 0.0.0.0:6081 \ > -T localhost:6082 \ > -f /etc/varnish/prudsys.vcl \ > -S /etc/varnish/secret \ > -s malloc,1G" > > On Wed, Dec 30, 2015 at 10:27 PM, Debraj Manna > wrote: > >> I have changed /etc/default/varnish. It looks something like below:- >> >> START=yes >> NFILES=131072 >> MEMLOCK=82000 >> DAEMON_OPTS="-a 0.0.0.0:6081 \ >> -T localhost:6082 \ >> -f /etc/varnish/prudsys.vcl \ >> -S /etc/varnish/secret \ >> -s malloc,1G" >> >> My prudsys.vcl looks like below:- >> >> vcl 4.0; >> backend default { >> .host = "146.88.25.40"; >> .port = "8080"; >> } >> >> I have also attached my /etc/init.d/varnish script >> >> >> On Wed, Dec 30, 2015 at 7:09 PM, Geoff Simmons wrote: >> >>> -----BEGIN PGP SIGNED MESSAGE----- >>> Hash: SHA256 >>> >>> On 12/30/15 10:27 AM, Debraj Manna wrote: >>> > >>> > I am new to varnish. Whenever I am trying to acess our backend via >>> > varnish I am getting the below error:- >>> >>> Always look for the FetchError: >>> >>> > - FetchError no backend connection >>> >>> Varnish can't reach your backend. Since there's no 'Backend' record >>> anywhere in your log, Varnish evidently doesn't know about any backend >>> that it can connect to. >>> >>> > If I start varnish as below then also everything works fine. I am >>> > facing problem only when I start varnish via sudo service varnish >>> > start sudo varnishd -n foo -f /etc/varnish/prudsys.vcl -s malloc,1G >>> > -T localhost:6082 -a 0.0.0.0:6081 >>> >>> Take a look at the start script invoked by 'service'. If you haven't >>> changed anything about it, I would guess that it's not using the VCL >>> file specified by -f in your command, but rather a "default" VCL that >>> defines a backend that doesn't exist. >>> >>> >>> HTH, >>> Geoff >>> - -- >>> UPLEX Systemoptimierung >>> Scheffelstra?e 32 >>> 22301 Hamburg >>> http://uplex.de/ >>> Mob: +49-176-63690917 >>> -----BEGIN PGP SIGNATURE----- >>> Version: GnuPG/MacGPG2 v2.0.14 (Darwin) >>> >>> iQIcBAEBCAAGBQJWg96EAAoJEOUwvh9pJNURmREQAIbYDmd1RuPuGECv1VUFAEL3 >>> HflMfEBaSPQINQh+7RSJ5uj8Qvvi1QRabb5eX4XpyU8+GjVPHpOfUKWWav42W2rH >>> 9rnhaT51lR6kegtnxpPMqoIdTaygYd4Vcy2zMhVsjraVDRdN3JPpuK90P12aP3bO >>> oQYrk+W6JXsG2DkkcpjLCGYOKD7RDX6pYpkMdYP3cBV21fobJs5o12CIuZckHMDe >>> r2NeM08ofjhRSskRiBeBguAaQVwALPmTfwVMhZyCEpvFUGVIRJlm9C2YDba/NuEL >>> et1Z1cyONh2v672f1QPohFCYZMkj8DpzJev2QxR0XOM5anEkpfCG7hrsNawAZOMk >>> 9W3cwiZ7+ChOOnoMHA4jYQTbH9og6wJXe8vmy0TjNoDqLF4apTUoqoi27dgHIS4o >>> wtNSLlC3rN0e8EPZlbQ0R4Xy4yUHDbDrXc0UrSPcPahAde49V/ZUf0RI+WGSa7nF >>> C63q/rJV2OCN9NbvmGYTwRu24asgqmAl6gMwlAXMdI2G6Y4b36HdiwChfVOzUL/H >>> Sgat8T3iIl/rkYaHP6Vfdw7+N7kV+/8RjEr62xRmw5ZiRDAv/wBknT1tTaSPWv33 >>> bMlFYDujYeEkcGeKJdMLGuKn/S9I+GJEbSk6V6E7v1LfGKiU18wT/7VKKVEvawCb >>> LodcXaEdHwAuWbuHjaOF >>> =HYy+ >>> -----END PGP SIGNATURE----- >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.lewis at nytimes.com Thu Dec 31 15:11:10 2015 From: eric.lewis at nytimes.com (Lewis, Eric) Date: Thu, 31 Dec 2015 10:11:10 -0500 Subject: Varnish and Docker In-Reply-To: References: Message-ID: Hi, I've opened the proposal to include our Dockerfile for Varnish in the Official Docker Hub repository. Would anyone from the Varnish project be interested in maintaining the repository, or at least endorsing this effort? Eric Lewis Web Developer, Interactive News The New York Times 620 Eighth Avenue, 2nd Floor New York, NY 10018 Office: (212) 556-2081 Cell: (610) 715-8560 -------------- next part -------------- An HTML attachment was scrubbed... URL: