From jcpetit at syspark.com Tue Dec 1 03:59:11 2009 From: jcpetit at syspark.com (Jean-Christophe Petit) Date: Mon, 30 Nov 2009 22:59:11 -0500 Subject: still using cache when fetching content Message-ID: <4B14948F.90405@syspark.com> Hello, Is it possible to make Varnish sending the cache content at the same time it is fetching from the backend ? It will be more efficient for slow dynamic content ;) For example, I have a php page taking up to 5sec to run. If Varnish was able to get it while still sending the old cache page, it would be really great. No more unlucky visitor hitting it to update the cache.. Thanks, -- Jean-Christophe Petit VP R&D et DSI GROUPE SYSPARK From tfheen at redpill-linpro.com Tue Dec 1 07:23:40 2009 From: tfheen at redpill-linpro.com (Tollef Fog Heen) Date: Tue, 01 Dec 2009 08:23:40 +0100 Subject: Handling 304 and header refresh In-Reply-To: <95ff419c0911270534s35f02db8y9c0f49da8cb36e2d@mail.gmail.com> (Daniel Rodriguez's message of "Fri, 27 Nov 2009 14:34:44 +0100") References: <95ff419c0911270534s35f02db8y9c0f49da8cb36e2d@mail.gmail.com> Message-ID: <87ws17kvib.fsf@qurzaw.linpro.no> ]] Daniel Rodriguez | 13 TxHeader b Cookie: foo.comGlobal=0; cUser=nouser You need to strip the cookie before sending the request to the backend, else you will end up pass-ing the request. -- Tollef Fog Heen Redpill Linpro -- Changing the game! t: +47 21 54 41 73 From tfheen at redpill-linpro.com Tue Dec 1 07:27:48 2009 From: tfheen at redpill-linpro.com (Tollef Fog Heen) Date: Tue, 01 Dec 2009 08:27:48 +0100 Subject: still using cache when fetching content In-Reply-To: <4B14948F.90405@syspark.com> (Jean-Christophe Petit's message of "Mon, 30 Nov 2009 22:59:11 -0500") References: <4B14948F.90405@syspark.com> Message-ID: <87skbvkvbf.fsf@qurzaw.linpro.no> ]] Jean-Christophe Petit | Is it possible to make Varnish sending the cache content at the same | time it is fetching from the backend ? Not at the moment, no. | It will be more efficient for slow dynamic content ;) | For example, I have a php page taking up to 5sec to run. If Varnish was | able to get it while still sending the old cache page, it would be | really great. You could trick this by having a script that revisits any URLs that are slow every N minutes and set obj.ttl to 0s in vcl_hit if the user agent (or IP address) is what you expect for your script. Might not work if you have many different URLs, though. -- Tollef Fog Heen Redpill Linpro -- Changing the game! t: +47 21 54 41 73 From phk at phk.freebsd.dk Tue Dec 1 09:08:05 2009 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Tue, 01 Dec 2009 09:08:05 +0000 Subject: varnish bottleneck? In-Reply-To: Your message of "Fri, 27 Nov 2009 09:12:10 +0800." <4B0F276A.9000909@gmail.com> Message-ID: <31719.1259658485@critter.freebsd.dk> In message <4B0F276A.9000909 at gmail.com>, ll writes: >if (req.request == "POST"){ >pipe; >} Use pass, not pipe. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From coolbomb at gmail.com Tue Dec 1 12:39:23 2009 From: coolbomb at gmail.com (Daniel Rodriguez) Date: Tue, 1 Dec 2009 13:39:23 +0100 Subject: Handling 304 and header refresh In-Reply-To: <87ws17kvib.fsf@qurzaw.linpro.no> References: <95ff419c0911270534s35f02db8y9c0f49da8cb36e2d@mail.gmail.com> <87ws17kvib.fsf@qurzaw.linpro.no> Message-ID: <95ff419c0912010439u5646f427tc302ea36f17912a2@mail.gmail.com> Hi, I removed the Cookies, but as i imagined that isn't the problem. The original request got well cached, but after sending the 304 with the max-age modified, all the request end up been served by the backend and not from the cache. Lets walk this step by step. First request: GET /test/prueba.php HTTP/1.1 Host: www.foo.com User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.3) Gecko/2008092816 Iceweasel/3.0.3 (Debian-3.0.3-3) Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-us,en;q=0.5 Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 X-Forwarded-For: 192.168.90.41 Accept-Encoding: gzip X-Varnish: 484826736 X-Forwarded-For: 192.168.90.41 The server reply: HTTP/1.1 200 OK Date: Tue, 01 Dec 2009 10:48:32 GMT Server: Apache Cache-control: max-age=60 Vary: Accept-Encoding Last-Modified: Tue, 17 Nov 2009 19:10:17 GMT Connection: close Content-Type: text/html; charset=iso-8859-15 Content-Encoding: gzip Content-Length: 87 ---- This end up in the cache, and all request after this, got server by the varnish cache. (This can be verified in the varnishlog attached) In this next request, the apache server returns 304 but with a new max-age (401) ---- Second request: GET /test/prueba.php HTTP/1.1 Host: www.foo.com User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.3) Gecko/2008092816 Iceweasel/3.0.3 (Debian-3.0.3-3) Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-us,en;q=0.5 Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 X-Forwarded-For: 192.168.90.41 Accept-Encoding: gzip X-Varnish: 484826741 X-Forwarded-For: 192.168.90.41 The server reply: HTTP/1.1 304 Not Modified Date: Tue, 01 Dec 2009 10:50:33 GMT Server: Apache Connection: close Cache-control: max-age=401 --- This one does not go into the cache and all further attemps to the same object end up going to my backend --- Third request: GET /test/prueba.php HTTP/1.1 Host: www.foo.com User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.3) Gecko/2008092816 Iceweasel/3.0.3 (Debian-3.0.3-3) Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-us,en;q=0.5 Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 If-Modified-Since: Tue, 17 Nov 2009 19:10:17 GMT X-Forwarded-For: 192.168.90.41 Accept-Encoding: gzip X-Varnish: 484826742 X-Forwarded-For: 192.168.90.41 The server reply: HTTP/1.1 304 Not Modified Date: Tue, 01 Dec 2009 10:50:49 GMT Server: Apache Connection: close Cache-control: max-age=401 --- I'm attaching a new varnishlog output for demonstration. I have deleted some request because the log was very large but i think that everything needed to verify this is in there. Best regards, 12 SessionOpen c 192.168.90.41 42154 :80 12 ReqStart c 192.168.90.41 42154 484826736 12 RxRequest c GET 12 RxURL c /test/prueba.php 12 RxProtocol c HTTP/1.1 12 RxHeader c Host: www.foo.com 12 RxHeader c User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.3) Gecko/2008092816 Iceweasel/3.0.3 (Debian-3.0.3-3) 12 RxHeader c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 12 RxHeader c Accept-Language: en-us,en;q=0.5 12 RxHeader c Accept-Encoding: gzip,deflate 12 RxHeader c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 12 RxHeader c Keep-Alive: 300 12 RxHeader c Connection: keep-alive 12 RxHeader c If-Modified-Since: Tue, 17 Nov 2009 19:10:17 GMT 12 RxHeader c Cache-Control: max-age=0 12 VCL_call c recv 12 VCL_return c lookup 12 VCL_call c hash 12 VCL_return c hash 12 VCL_call c miss 12 VCL_return c fetch 13 BackendOpen b default 192.168.55.18 38237 192.168.9.158 80 12 Backend c 13 default default 13 TxRequest b GET 13 TxURL b /test/prueba.php 13 TxProtocol b HTTP/1.1 13 TxHeader b Host: www.foo.com 13 TxHeader b User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.3) Gecko/2008092816 Iceweasel/3.0.3 (Debian-3.0.3-3) 13 TxHeader b Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 13 TxHeader b Accept-Language: en-us,en;q=0.5 13 TxHeader b Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 13 TxHeader b X-Forwarded-For: 192.168.90.41 13 TxHeader b Accept-Encoding: gzip 13 TxHeader b X-Varnish: 484826736 13 TxHeader b X-Forwarded-For: 192.168.90.41 0 CLI - Rd ping 0 CLI - Wr 0 200 PONG 1259664512 1.0 0 CLI - Rd ping 0 CLI - Wr 0 200 PONG 1259664515 1.0 13 RxProtocol b HTTP/1.1 13 RxStatus b 200 13 RxResponse b OK 13 RxHeader b Date: Tue, 01 Dec 2009 10:48:32 GMT 13 RxHeader b Server: Apache 13 RxHeader b Cache-control: max-age=60 13 RxHeader b Vary: Accept-Encoding 13 RxHeader b Last-Modified: Tue, 17 Nov 2009 19:10:17 GMT 13 RxHeader b Connection: close 13 RxHeader b Content-Type: text/html; charset=iso-8859-15 13 RxHeader b Content-Encoding: gzip 13 RxHeader b Content-Length: 87 12 ObjProtocol c HTTP/1.1 12 ObjStatus c 200 12 ObjResponse c OK 12 ObjHeader c Date: Tue, 01 Dec 2009 10:48:32 GMT 12 ObjHeader c Server: Apache 12 ObjHeader c Cache-control: max-age=60 12 ObjHeader c Vary: Accept-Encoding 12 ObjHeader c Last-Modified: Tue, 17 Nov 2009 19:10:17 GMT 12 ObjHeader c Content-Type: text/html; charset=iso-8859-15 12 ObjHeader c Content-Encoding: gzip 13 BackendClose b default 12 TTL c 484826736 RFC 60 1259664516 0 0 60 0 12 VCL_call c fetch 12 VCL_return c deliver 12 Length c 0 12 VCL_call c deliver 12 VCL_return c deliver 12 TxProtocol c HTTP/1.1 12 TxStatus c 304 12 TxResponse c Not Modified 12 TxHeader c Date: Tue, 01 Dec 2009 10:48:31 GMT 12 TxHeader c Via: 1.1 varnish 12 TxHeader c X-Varnish: 484826736 12 TxHeader c Last-Modified: Tue, 17 Nov 2009 19:10:17 GMT 12 TxHeader c Cache-Control: max-age=60 12 TxHeader c Vary: Accept-Encoding 12 TxHeader c Connection: keep-alive 12 ReqEnd c 484826736 1259664511.749969006 1259664516.769968987 0.000000000 5.019999981 0.000000000 12 Debug c "herding" 0 CLI - Rd ping 0 CLI - Wr 0 200 PONG 1259664518 1.0 0 CLI - Rd ping 0 CLI - Wr 0 200 PONG 1259664521 1.0 12 SessionClose c timeout 12 StatSess c 192.168.90.41 42154 5 0 0 0 0 0 0 0 0 CLI - Rd ping 0 CLI - Wr 0 200 PONG 1259664524 1.0 12 SessionOpen c 192.168.90.41 42155 :80 12 ReqStart c 192.168.90.41 42155 484826737 12 RxRequest c GET 12 RxURL c /test/prueba.php 12 RxProtocol c HTTP/1.1 12 RxHeader c Host: www.foo.com 12 RxHeader c User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.3) Gecko/2008092816 Iceweasel/3.0.3 (Debian-3.0.3-3) 12 RxHeader c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 12 RxHeader c Accept-Language: en-us,en;q=0.5 12 RxHeader c Accept-Encoding: gzip,deflate 12 RxHeader c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 12 RxHeader c Keep-Alive: 300 12 RxHeader c Connection: keep-alive 12 RxHeader c If-Modified-Since: Tue, 17 Nov 2009 19:10:17 GMT 12 RxHeader c Cache-Control: max-age=0 12 VCL_call c recv 12 VCL_return c lookup 12 VCL_call c hash 12 VCL_return c hash 12 Hit c 484826736 12 VCL_call c hit 12 VCL_return c deliver 12 Length c 0 12 VCL_call c deliver 12 VCL_return c deliver 12 TxProtocol c HTTP/1.1 12 TxStatus c 304 12 TxResponse c Not Modified 12 TxHeader c Date: Tue, 01 Dec 2009 10:48:44 GMT 12 TxHeader c Via: 1.1 varnish 12 TxHeader c X-Varnish: 484826737 12 TxHeader c Last-Modified: Tue, 17 Nov 2009 19:10:17 GMT 12 TxHeader c Cache-Control: max-age=60 12 TxHeader c Vary: Accept-Encoding 12 TxHeader c Connection: keep-alive 12 ReqEnd c 484826737 1259664524.945969105 1259664524.945969105 0.004000187 0.000000000 0.000000000 12 Debug c "herding" 0 CLI - Rd ping 0 CLI - Wr 0 200 PONG 1259664527 1.0 12 ReqStart c 192.168.90.41 42155 484826738 12 RxRequest c GET 12 RxURL c /test/prueba.php 12 RxProtocol c HTTP/1.1 12 RxHeader c Host: www.foo.com 12 RxHeader c User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.3) Gecko/2008092816 Iceweasel/3.0.3 (Debian-3.0.3-3) 12 RxHeader c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 12 RxHeader c Accept-Language: en-us,en;q=0.5 12 RxHeader c Accept-Encoding: gzip,deflate 12 RxHeader c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 12 RxHeader c Keep-Alive: 300 12 RxHeader c Connection: keep-alive 12 RxHeader c If-Modified-Since: Tue, 17 Nov 2009 19:10:17 GMT 12 RxHeader c Cache-Control: max-age=0 12 VCL_call c recv 12 VCL_return c lookup 12 VCL_call c hash 12 VCL_return c hash 12 Hit c 484826736 12 VCL_call c hit 12 VCL_return c deliver 12 Length c 0 12 VCL_call c deliver 12 VCL_return c deliver 12 TxProtocol c HTTP/1.1 12 TxStatus c 304 12 TxResponse c Not Modified 12 TxHeader c Date: Tue, 01 Dec 2009 10:48:49 GMT 12 TxHeader c Via: 1.1 varnish 12 TxHeader c X-Varnish: 484826738 12 TxHeader c Last-Modified: Tue, 17 Nov 2009 19:10:17 GMT 12 TxHeader c Cache-Control: max-age=60 12 TxHeader c Vary: Accept-Encoding 12 TxHeader c Connection: keep-alive 12 ReqEnd c 484826738 1259664529.137969017 1259664529.137969017 4.191999912 0.000000000 0.000000000 12 Debug c "herding" 0 CLI - Rd ping 0 CLI - Wr 0 200 PONG 1259664530 1.0 0 CLI - Rd ping 0 CLI - Wr 0 200 PONG 1259664533 1.0 12 SessionClose c timeout 12 StatSess c 192.168.90.41 42155 4 0 0 0 0 0 0 0 0 CLI - Rd ping 0 CLI - Wr 0 200 PONG 1259664536 1.0 0 CLI - Rd ping 0 CLI - Wr 0 200 PONG 1259664539 1.0 0 CLI - Rd ping 0 CLI - Wr 0 200 PONG 1259664542 1.0 0 CLI - Rd ping 0 CLI - Wr 0 200 PONG 1259664545 1.0 12 SessionOpen c 192.168.90.41 37495 :80 -------------------------------------------------------------- I keep requesting the objet a lot (I delete that part from the log, there are only a few for demonstration purposes) all the request are server directly from the cache) At this time, apache will return 304 and send a new max-age=401 All request are now going to the backend. Not caching the object. --------------------------------------------------------------- 12 SessionOpen c 192.168.90.41 37499 :80 12 ReqStart c 192.168.90.41 37499 484826741 12 RxRequest c GET 12 RxURL c /test/prueba.php 12 RxProtocol c HTTP/1.1 12 RxHeader c Host: www.foo.com 12 RxHeader c User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.3) Gecko/2008092816 Iceweasel/3.0.3 (Debian-3.0.3-3) 12 RxHeader c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 12 RxHeader c Accept-Language: en-us,en;q=0.5 12 RxHeader c Accept-Encoding: gzip,deflate 12 RxHeader c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 12 RxHeader c Keep-Alive: 300 12 RxHeader c Connection: keep-alive 12 RxHeader c If-Modified-Since: Tue, 17 Nov 2009 19:10:17 GMT 12 RxHeader c Cache-Control: max-age=0 12 VCL_call c recv 12 VCL_return c lookup 12 VCL_call c hash 12 VCL_return c hash 12 VCL_call c miss 12 VCL_return c fetch 13 BackendOpen b default 192.168.55.18 38239 192.168.9.158 80 12 Backend c 13 default default 13 TxRequest b GET 13 TxURL b /test/prueba.php 13 TxProtocol b HTTP/1.1 13 TxHeader b Host: www.foo.com 13 TxHeader b User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.3) Gecko/2008092816 Iceweasel/3.0.3 (Debian-3.0.3-3) 13 TxHeader b Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 13 TxHeader b Accept-Language: en-us,en;q=0.5 13 TxHeader b Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 13 TxHeader b X-Forwarded-For: 192.168.90.41 13 TxHeader b Accept-Encoding: gzip 13 TxHeader b X-Varnish: 484826741 13 TxHeader b X-Forwarded-For: 192.168.90.41 13 RxProtocol b HTTP/1.1 13 RxStatus b 304 13 RxResponse b Not Modified 13 RxHeader b Date: Tue, 01 Dec 2009 10:50:33 GMT 13 RxHeader b Server: Apache 13 RxHeader b Connection: close 13 RxHeader b Cache-control: max-age=401 12 ObjProtocol c HTTP/1.1 12 ObjStatus c 304 12 ObjResponse c Not Modified 12 ObjHeader c Date: Tue, 01 Dec 2009 10:50:33 GMT 12 ObjHeader c Server: Apache 12 ObjHeader c Cache-control: max-age=401 13 BackendClose b default 12 TTL c 484826741 RFC 401 1259664631 0 0 401 0 12 VCL_call c fetch 12 VCL_return c pass 12 Length c 0 12 VCL_call c deliver 12 VCL_return c deliver 12 TxProtocol c HTTP/1.1 12 TxStatus c 304 12 TxResponse c Not Modified 12 TxHeader c Server: Apache 12 TxHeader c Cache-control: max-age=401 12 TxHeader c Content-Length: 0 12 TxHeader c Date: Tue, 01 Dec 2009 10:50:31 GMT 12 TxHeader c X-Varnish: 484826741 12 TxHeader c Age: 0 12 TxHeader c Via: 1.1 varnish 12 TxHeader c Connection: keep-alive 12 ReqEnd c 484826741 1259664631.945969105 1259664631.969969034 0.000000000 0.023999929 0.000000000 12 Debug c "herding" 0 CLI - Rd ping 0 CLI - Wr 0 200 PONG 1259664632 1.0 0 CLI - Rd ping 0 CLI - Wr 0 200 PONG 1259664635 1.0 12 SessionClose c timeout 12 StatSess c 192.168.90.41 37499 0 0 0 0 0 0 0 0 0 CLI - Rd ping 0 CLI - Wr 0 200 PONG 1259664638 1.0 0 CLI - Rd ping 0 CLI - Wr 0 200 PONG 1259664641 1.0 0 CLI - Rd ping 0 CLI - Wr 0 200 PONG 1259664644 1.0 0 CLI - Rd ping 0 CLI - Wr 0 200 PONG 1259664647 1.0 12 SessionOpen c 192.168.90.41 37500 :80 12 ReqStart c 192.168.90.41 37500 484826742 12 RxRequest c GET 12 RxURL c /test/prueba.php 12 RxProtocol c HTTP/1.1 12 RxHeader c Host: www.foo.com 12 RxHeader c User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.3) Gecko/2008092816 Iceweasel/3.0.3 (Debian-3.0.3-3) 12 RxHeader c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 12 RxHeader c Accept-Language: en-us,en;q=0.5 12 RxHeader c Accept-Encoding: gzip,deflate 12 RxHeader c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 12 RxHeader c Keep-Alive: 300 12 RxHeader c Connection: keep-alive 12 RxHeader c If-Modified-Since: Tue, 17 Nov 2009 19:10:17 GMT 12 RxHeader c Cache-Control: max-age=0 12 VCL_call c recv 12 VCL_return c lookup 12 VCL_call c hash 12 VCL_return c hash 12 HitPass c 484826741 12 VCL_call c pass 12 VCL_return c pass 13 BackendOpen b default 192.168.55.18 38240 192.168.9.158 80 12 Backend c 13 default default 13 TxRequest b GET 13 TxURL b /test/prueba.php 13 TxProtocol b HTTP/1.1 13 TxHeader b Host: www.foo.com 13 TxHeader b User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.3) Gecko/2008092816 Iceweasel/3.0.3 (Debian-3.0.3-3) 13 TxHeader b Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 13 TxHeader b Accept-Language: en-us,en;q=0.5 13 TxHeader b Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 13 TxHeader b If-Modified-Since: Tue, 17 Nov 2009 19:10:17 GMT 13 TxHeader b X-Forwarded-For: 192.168.90.41 13 TxHeader b Accept-Encoding: gzip 13 TxHeader b X-Varnish: 484826742 13 TxHeader b X-Forwarded-For: 192.168.90.41 13 RxProtocol b HTTP/1.1 13 RxStatus b 304 13 RxResponse b Not Modified 13 RxHeader b Date: Tue, 01 Dec 2009 10:50:49 GMT 13 RxHeader b Server: Apache 13 RxHeader b Connection: close 13 RxHeader b Cache-control: max-age=401 12 ObjProtocol c HTTP/1.1 12 ObjStatus c 304 12 ObjResponse c Not Modified 12 ObjHeader c Date: Tue, 01 Dec 2009 10:50:49 GMT 12 ObjHeader c Server: Apache 12 ObjHeader c Cache-control: max-age=401 13 BackendClose b default 12 TTL c 484826742 RFC 401 1259664647 0 0 401 0 12 VCL_call c fetch 12 VCL_return c pass 12 Length c 0 12 VCL_call c deliver 12 VCL_return c deliver 12 TxProtocol c HTTP/1.1 12 TxStatus c 304 12 TxResponse c Not Modified 12 TxHeader c Server: Apache 12 TxHeader c Cache-control: max-age=401 12 TxHeader c Content-Length: 0 12 TxHeader c Date: Tue, 01 Dec 2009 10:50:47 GMT 12 TxHeader c X-Varnish: 484826742 12 TxHeader c Age: 0 12 TxHeader c Via: 1.1 varnish 12 TxHeader c Connection: keep-alive 12 ReqEnd c 484826742 1259664647.989969015 1259664647.993968964 0.000000000 0.003999949 0.000000000 12 Debug c "herding" 0 CLI - Rd ping 0 CLI - Wr 0 200 PONG 1259664650 1.0 12 ReqStart c 192.168.90.41 37500 484826743 12 RxRequest c GET 12 RxURL c /test/prueba.php 12 RxProtocol c HTTP/1.1 12 RxHeader c Host: www.foo.com 12 RxHeader c User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.3) Gecko/2008092816 Iceweasel/3.0.3 (Debian-3.0.3-3) 12 RxHeader c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 12 RxHeader c Accept-Language: en-us,en;q=0.5 12 RxHeader c Accept-Encoding: gzip,deflate 12 RxHeader c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 12 RxHeader c Keep-Alive: 300 12 RxHeader c Connection: keep-alive 12 RxHeader c If-Modified-Since: Tue, 17 Nov 2009 19:10:17 GMT 12 RxHeader c Cache-Control: max-age=0 12 VCL_call c recv 12 VCL_return c lookup 12 VCL_call c hash 12 VCL_return c hash 12 HitPass c 484826741 12 VCL_call c pass 12 VCL_return c pass 13 BackendOpen b default 192.168.55.18 38241 192.168.9.158 80 12 Backend c 13 default default 13 TxRequest b GET 13 TxURL b /test/prueba.php 13 TxProtocol b HTTP/1.1 13 TxHeader b Host: www.foo.com 13 TxHeader b User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.3) Gecko/2008092816 Iceweasel/3.0.3 (Debian-3.0.3-3) 13 TxHeader b Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 13 TxHeader b Accept-Language: en-us,en;q=0.5 13 TxHeader b Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 13 TxHeader b If-Modified-Since: Tue, 17 Nov 2009 19:10:17 GMT 13 TxHeader b X-Forwarded-For: 192.168.90.41 13 TxHeader b Accept-Encoding: gzip 13 TxHeader b X-Varnish: 484826743 13 TxHeader b X-Forwarded-For: 192.168.90.41 13 RxProtocol b HTTP/1.1 13 RxStatus b 304 13 RxResponse b Not Modified 13 RxHeader b Date: Tue, 01 Dec 2009 10:50:53 GMT 13 RxHeader b Server: Apache 13 RxHeader b Connection: close 13 RxHeader b Cache-control: max-age=401 12 ObjProtocol c HTTP/1.1 12 ObjStatus c 304 12 ObjResponse c Not Modified 12 ObjHeader c Date: Tue, 01 Dec 2009 10:50:53 GMT 12 ObjHeader c Server: Apache 12 ObjHeader c Cache-control: max-age=401 13 BackendClose b default 12 TTL c 484826743 RFC 401 1259664652 0 0 401 0 12 VCL_call c fetch 12 VCL_return c pass 12 Length c 0 12 VCL_call c deliver 12 VCL_return c deliver 12 TxProtocol c HTTP/1.1 12 TxStatus c 304 12 TxResponse c Not Modified 12 TxHeader c Server: Apache 12 TxHeader c Cache-control: max-age=401 12 TxHeader c Content-Length: 0 12 TxHeader c Date: Tue, 01 Dec 2009 10:50:52 GMT 12 TxHeader c X-Varnish: 484826743 12 TxHeader c Age: 0 12 TxHeader c Via: 1.1 varnish 12 TxHeader c Connection: keep-alive 12 ReqEnd c 484826743 1259664652.677968979 1259664652.685969114 4.684000015 0.008000135 0.000000000 12 Debug c "herding" 0 CLI - Rd ping 0 CLI - Wr 0 200 PONG 1259664653 1.0 0 CLI - Rd ping 0 CLI - Wr 0 200 PONG 1259664656 1.0 On Tue, Dec 1, 2009 at 8:23 AM, Tollef Fog Heen wrote: > ]] Daniel Rodriguez > > | ? ?13 TxHeader ? ? b Cookie: foo.comGlobal=0; cUser=nouser > > You need to strip the cookie before sending the request to the backend, > else you will end up pass-ing the request. > > -- > Tollef Fog Heen > Redpill Linpro -- Changing the game! > t: +47 21 54 41 73 > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc > From bedis9 at gmail.com Tue Dec 1 14:44:50 2009 From: bedis9 at gmail.com (Bedis 9) Date: Tue, 1 Dec 2009 15:44:50 +0100 Subject: Handling 304 and header refresh In-Reply-To: <95ff419c0912010439u5646f427tc302ea36f17912a2@mail.gmail.com> References: <95ff419c0911270534s35f02db8y9c0f49da8cb36e2d@mail.gmail.com> <87ws17kvib.fsf@qurzaw.linpro.no> <95ff419c0912010439u5646f427tc302ea36f17912a2@mail.gmail.com> Message-ID: <30b80c9f0912010644i6c145a04u3a2d4a0fb1e377ea@mail.gmail.com> Hi, It sounds good that Varnish does not cache a 304 answer ;) In the second request, you have no "If-Modified-Since" headers in your request and your origin seems to answer you with a 304, which is not realy what varnish should expect! By the way, it's a very bad idea to setup a Cache-Control header with a max-age value on a 304 response. cheers On Tue, Dec 1, 2009 at 1:39 PM, Daniel Rodriguez wrote: > Hi, > > I removed the Cookies, but as i imagined that isn't the problem. The > original request got well cached, but after sending the 304 with the > max-age modified, all the request end up been served by the backend > and not from the cache. > > Lets walk this step by step. > > First request: > > GET /test/prueba.php HTTP/1.1 > Host: www.foo.com > User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.3) > Gecko/2008092816 Iceweasel/3.0.3 (Debian-3.0.3-3) > Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 > Accept-Language: en-us,en;q=0.5 > Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 > X-Forwarded-For: 192.168.90.41 > Accept-Encoding: gzip > X-Varnish: 484826736 > X-Forwarded-For: 192.168.90.41 > > The server reply: > > HTTP/1.1 200 OK > Date: Tue, 01 Dec 2009 10:48:32 GMT > Server: Apache > Cache-control: max-age=60 > Vary: Accept-Encoding > Last-Modified: Tue, 17 Nov 2009 19:10:17 GMT > Connection: close > Content-Type: text/html; charset=iso-8859-15 > Content-Encoding: gzip > Content-Length: 87 > > ---- > This end up in the cache, and all request after this, got server by > the varnish cache. (This can be verified in the varnishlog attached) > > In this next request, the apache server returns 304 but with a new max-age (401) > ---- > > Second request: > > GET /test/prueba.php HTTP/1.1 > Host: www.foo.com > User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.3) > Gecko/2008092816 Iceweasel/3.0.3 (Debian-3.0.3-3) > Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 > Accept-Language: en-us,en;q=0.5 > Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 > X-Forwarded-For: 192.168.90.41 > Accept-Encoding: gzip > X-Varnish: 484826741 > X-Forwarded-For: 192.168.90.41 > > The server reply: > > HTTP/1.1 304 Not Modified > Date: Tue, 01 Dec 2009 10:50:33 GMT > Server: Apache > Connection: close > Cache-control: max-age=401 > > --- > This one does not go into the cache and all further attemps to the > same object end up going to my backend > --- > > Third request: > > GET /test/prueba.php HTTP/1.1 > Host: www.foo.com > User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.3) > Gecko/2008092816 Iceweasel/3.0.3 (Debian-3.0.3-3) > Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 > Accept-Language: en-us,en;q=0.5 > Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 > If-Modified-Since: Tue, 17 Nov 2009 19:10:17 GMT > X-Forwarded-For: 192.168.90.41 > Accept-Encoding: gzip > X-Varnish: 484826742 > X-Forwarded-For: 192.168.90.41 > > The server reply: > > HTTP/1.1 304 Not Modified > Date: Tue, 01 Dec 2009 10:50:49 GMT > Server: Apache > Connection: close > Cache-control: max-age=401 > --- > > I'm attaching a new varnishlog output for demonstration. I have > deleted some request because the log was very large but i think that > everything needed to verify this is in there. > > Best regards, > > ? 12 SessionOpen ?c 192.168.90.41 42154 :80 > ? 12 ReqStart ? ? c 192.168.90.41 42154 484826736 > ? 12 RxRequest ? ?c GET > ? 12 RxURL ? ? ? ?c /test/prueba.php > ? 12 RxProtocol ? c HTTP/1.1 > ? 12 RxHeader ? ? c Host: www.foo.com > ? 12 RxHeader ? ? c User-Agent: Mozilla/5.0 (X11; U; Linux i686; > en-US; rv:1.9.0.3) Gecko/2008092816 Iceweasel/3.0.3 (Debian-3.0.3-3) > ? 12 RxHeader ? ? c Accept: > text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 > ? 12 RxHeader ? ? c Accept-Language: en-us,en;q=0.5 > ? 12 RxHeader ? ? c Accept-Encoding: gzip,deflate > ? 12 RxHeader ? ? c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 > ? 12 RxHeader ? ? c Keep-Alive: 300 > ? 12 RxHeader ? ? c Connection: keep-alive > ? 12 RxHeader ? ? c If-Modified-Since: Tue, 17 Nov 2009 19:10:17 GMT > ? 12 RxHeader ? ? c Cache-Control: max-age=0 > ? 12 VCL_call ? ? c recv > ? 12 VCL_return ? c lookup > ? 12 VCL_call ? ? c hash > ? 12 VCL_return ? c hash > ? 12 VCL_call ? ? c miss > ? 12 VCL_return ? c fetch > ? 13 BackendOpen ?b default 192.168.55.18 38237 192.168.9.158 80 > ? 12 Backend ? ? ?c 13 default default > ? 13 TxRequest ? ?b GET > ? 13 TxURL ? ? ? ?b /test/prueba.php > ? 13 TxProtocol ? b HTTP/1.1 > ? 13 TxHeader ? ? b Host: www.foo.com > ? 13 TxHeader ? ? b User-Agent: Mozilla/5.0 (X11; U; Linux i686; > en-US; rv:1.9.0.3) Gecko/2008092816 Iceweasel/3.0.3 (Debian-3.0.3-3) > ? 13 TxHeader ? ? b Accept: > text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 > ? 13 TxHeader ? ? b Accept-Language: en-us,en;q=0.5 > ? 13 TxHeader ? ? b Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 > ? 13 TxHeader ? ? b X-Forwarded-For: 192.168.90.41 > ? 13 TxHeader ? ? b Accept-Encoding: gzip > ? 13 TxHeader ? ? b X-Varnish: 484826736 > ? 13 TxHeader ? ? b X-Forwarded-For: 192.168.90.41 > ? ?0 CLI ? ? ? ? ?- Rd ping > ? ?0 CLI ? ? ? ? ?- Wr 0 200 PONG 1259664512 1.0 > ? ?0 CLI ? ? ? ? ?- Rd ping > ? ?0 CLI ? ? ? ? ?- Wr 0 200 PONG 1259664515 1.0 > ? 13 RxProtocol ? b HTTP/1.1 > ? 13 RxStatus ? ? b 200 > ? 13 RxResponse ? b OK > ? 13 RxHeader ? ? b Date: Tue, 01 Dec 2009 10:48:32 GMT > ? 13 RxHeader ? ? b Server: Apache > ? 13 RxHeader ? ? b Cache-control: max-age=60 > ? 13 RxHeader ? ? b Vary: Accept-Encoding > ? 13 RxHeader ? ? b Last-Modified: Tue, 17 Nov 2009 19:10:17 GMT > ? 13 RxHeader ? ? b Connection: close > ? 13 RxHeader ? ? b Content-Type: text/html; charset=iso-8859-15 > ? 13 RxHeader ? ? b Content-Encoding: gzip > ? 13 RxHeader ? ? b Content-Length: 87 > ? 12 ObjProtocol ?c HTTP/1.1 > ? 12 ObjStatus ? ?c 200 > ? 12 ObjResponse ?c OK > ? 12 ObjHeader ? ?c Date: Tue, 01 Dec 2009 10:48:32 GMT > ? 12 ObjHeader ? ?c Server: Apache > ? 12 ObjHeader ? ?c Cache-control: max-age=60 > ? 12 ObjHeader ? ?c Vary: Accept-Encoding > ? 12 ObjHeader ? ?c Last-Modified: Tue, 17 Nov 2009 19:10:17 GMT > ? 12 ObjHeader ? ?c Content-Type: text/html; charset=iso-8859-15 > ? 12 ObjHeader ? ?c Content-Encoding: gzip > ? 13 BackendClose b default > ? 12 TTL ? ? ? ? ?c 484826736 RFC 60 1259664516 0 0 60 0 > ? 12 VCL_call ? ? c fetch > ? 12 VCL_return ? c deliver > ? 12 Length ? ? ? c 0 > ? 12 VCL_call ? ? c deliver > ? 12 VCL_return ? c deliver > ? 12 TxProtocol ? c HTTP/1.1 > ? 12 TxStatus ? ? c 304 > ? 12 TxResponse ? c Not Modified > ? 12 TxHeader ? ? c Date: Tue, 01 Dec 2009 10:48:31 GMT > ? 12 TxHeader ? ? c Via: 1.1 varnish > ? 12 TxHeader ? ? c X-Varnish: 484826736 > ? 12 TxHeader ? ? c Last-Modified: Tue, 17 Nov 2009 19:10:17 GMT > ? 12 TxHeader ? ? c Cache-Control: max-age=60 > ? 12 TxHeader ? ? c Vary: Accept-Encoding > ? 12 TxHeader ? ? c Connection: keep-alive > ? 12 ReqEnd ? ? ? c 484826736 1259664511.749969006 > 1259664516.769968987 0.000000000 5.019999981 0.000000000 > ? 12 Debug ? ? ? ?c "herding" > ? ?0 CLI ? ? ? ? ?- Rd ping > ? ?0 CLI ? ? ? ? ?- Wr 0 200 PONG 1259664518 1.0 > ? ?0 CLI ? ? ? ? ?- Rd ping > ? ?0 CLI ? ? ? ? ?- Wr 0 200 PONG 1259664521 1.0 > ? 12 SessionClose c timeout > ? 12 StatSess ? ? c 192.168.90.41 42154 5 0 0 0 0 0 0 0 > ? ?0 CLI ? ? ? ? ?- Rd ping > ? ?0 CLI ? ? ? ? ?- Wr 0 200 PONG 1259664524 1.0 > ? 12 SessionOpen ?c 192.168.90.41 42155 :80 > ? 12 ReqStart ? ? c 192.168.90.41 42155 484826737 > ? 12 RxRequest ? ?c GET > ? 12 RxURL ? ? ? ?c /test/prueba.php > ? 12 RxProtocol ? c HTTP/1.1 > ? 12 RxHeader ? ? c Host: www.foo.com > ? 12 RxHeader ? ? c User-Agent: Mozilla/5.0 (X11; U; Linux i686; > en-US; rv:1.9.0.3) Gecko/2008092816 Iceweasel/3.0.3 (Debian-3.0.3-3) > ? 12 RxHeader ? ? c Accept: > text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 > ? 12 RxHeader ? ? c Accept-Language: en-us,en;q=0.5 > ? 12 RxHeader ? ? c Accept-Encoding: gzip,deflate > ? 12 RxHeader ? ? c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 > ? 12 RxHeader ? ? c Keep-Alive: 300 > ? 12 RxHeader ? ? c Connection: keep-alive > ? 12 RxHeader ? ? c If-Modified-Since: Tue, 17 Nov 2009 19:10:17 GMT > ? 12 RxHeader ? ? c Cache-Control: max-age=0 > ? 12 VCL_call ? ? c recv > ? 12 VCL_return ? c lookup > ? 12 VCL_call ? ? c hash > ? 12 VCL_return ? c hash > ? 12 Hit ? ? ? ? ?c 484826736 > ? 12 VCL_call ? ? c hit > ? 12 VCL_return ? c deliver > ? 12 Length ? ? ? c 0 > ? 12 VCL_call ? ? c deliver > ? 12 VCL_return ? c deliver > ? 12 TxProtocol ? c HTTP/1.1 > ? 12 TxStatus ? ? c 304 > ? 12 TxResponse ? c Not Modified > ? 12 TxHeader ? ? c Date: Tue, 01 Dec 2009 10:48:44 GMT > ? 12 TxHeader ? ? c Via: 1.1 varnish > ? 12 TxHeader ? ? c X-Varnish: 484826737 > ? 12 TxHeader ? ? c Last-Modified: Tue, 17 Nov 2009 19:10:17 GMT > ? 12 TxHeader ? ? c Cache-Control: max-age=60 > ? 12 TxHeader ? ? c Vary: Accept-Encoding > ? 12 TxHeader ? ? c Connection: keep-alive > ? 12 ReqEnd ? ? ? c 484826737 1259664524.945969105 > 1259664524.945969105 0.004000187 0.000000000 0.000000000 > ? 12 Debug ? ? ? ?c "herding" > ? ?0 CLI ? ? ? ? ?- Rd ping > ? ?0 CLI ? ? ? ? ?- Wr 0 200 PONG 1259664527 1.0 > ? 12 ReqStart ? ? c 192.168.90.41 42155 484826738 > ? 12 RxRequest ? ?c GET > ? 12 RxURL ? ? ? ?c /test/prueba.php > ? 12 RxProtocol ? c HTTP/1.1 > ? 12 RxHeader ? ? c Host: www.foo.com > ? 12 RxHeader ? ? c User-Agent: Mozilla/5.0 (X11; U; Linux i686; > en-US; rv:1.9.0.3) Gecko/2008092816 Iceweasel/3.0.3 (Debian-3.0.3-3) > ? 12 RxHeader ? ? c Accept: > text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 > ? 12 RxHeader ? ? c Accept-Language: en-us,en;q=0.5 > ? 12 RxHeader ? ? c Accept-Encoding: gzip,deflate > ? 12 RxHeader ? ? c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 > ? 12 RxHeader ? ? c Keep-Alive: 300 > ? 12 RxHeader ? ? c Connection: keep-alive > ? 12 RxHeader ? ? c If-Modified-Since: Tue, 17 Nov 2009 19:10:17 GMT > ? 12 RxHeader ? ? c Cache-Control: max-age=0 > ? 12 VCL_call ? ? c recv > ? 12 VCL_return ? c lookup > ? 12 VCL_call ? ? c hash > ? 12 VCL_return ? c hash > ? 12 Hit ? ? ? ? ?c 484826736 > ? 12 VCL_call ? ? c hit > ? 12 VCL_return ? c deliver > ? 12 Length ? ? ? c 0 > ? 12 VCL_call ? ? c deliver > ? 12 VCL_return ? c deliver > ? 12 TxProtocol ? c HTTP/1.1 > ? 12 TxStatus ? ? c 304 > ? 12 TxResponse ? c Not Modified > ? 12 TxHeader ? ? c Date: Tue, 01 Dec 2009 10:48:49 GMT > ? 12 TxHeader ? ? c Via: 1.1 varnish > ? 12 TxHeader ? ? c X-Varnish: 484826738 > ? 12 TxHeader ? ? c Last-Modified: Tue, 17 Nov 2009 19:10:17 GMT > ? 12 TxHeader ? ? c Cache-Control: max-age=60 > ? 12 TxHeader ? ? c Vary: Accept-Encoding > ? 12 TxHeader ? ? c Connection: keep-alive > ? 12 ReqEnd ? ? ? c 484826738 1259664529.137969017 > 1259664529.137969017 4.191999912 0.000000000 0.000000000 > ? 12 Debug ? ? ? ?c "herding" > ? ?0 CLI ? ? ? ? ?- Rd ping > ? ?0 CLI ? ? ? ? ?- Wr 0 200 PONG 1259664530 1.0 > ? ?0 CLI ? ? ? ? ?- Rd ping > ? ?0 CLI ? ? ? ? ?- Wr 0 200 PONG 1259664533 1.0 > ? 12 SessionClose c timeout > ? 12 StatSess ? ? c 192.168.90.41 42155 4 0 0 0 0 0 0 0 > ? ?0 CLI ? ? ? ? ?- Rd ping > ? ?0 CLI ? ? ? ? ?- Wr 0 200 PONG 1259664536 1.0 > ? ?0 CLI ? ? ? ? ?- Rd ping > ? ?0 CLI ? ? ? ? ?- Wr 0 200 PONG 1259664539 1.0 > ? ?0 CLI ? ? ? ? ?- Rd ping > ? ?0 CLI ? ? ? ? ?- Wr 0 200 PONG 1259664542 1.0 > ? ?0 CLI ? ? ? ? ?- Rd ping > ? ?0 CLI ? ? ? ? ?- Wr 0 200 PONG 1259664545 1.0 > ? 12 SessionOpen ?c 192.168.90.41 37495 :80 > -------------------------------------------------------------- > > I keep requesting the objet a lot (I delete that part from the log, > there are only a few for demonstration purposes) all the request are > server directly from the cache) > > At this time, apache will return 304 and send a new max-age=401 > > All request are now going to the backend. Not caching the object. > > --------------------------------------------------------------- > > ? 12 SessionOpen ?c 192.168.90.41 37499 :80 > ? 12 ReqStart ? ? c 192.168.90.41 37499 484826741 > ? 12 RxRequest ? ?c GET > ? 12 RxURL ? ? ? ?c /test/prueba.php > ? 12 RxProtocol ? c HTTP/1.1 > ? 12 RxHeader ? ? c Host: www.foo.com > ? 12 RxHeader ? ? c User-Agent: Mozilla/5.0 (X11; U; Linux i686; > en-US; rv:1.9.0.3) Gecko/2008092816 Iceweasel/3.0.3 (Debian-3.0.3-3) > ? 12 RxHeader ? ? c Accept: > text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 > ? 12 RxHeader ? ? c Accept-Language: en-us,en;q=0.5 > ? 12 RxHeader ? ? c Accept-Encoding: gzip,deflate > ? 12 RxHeader ? ? c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 > ? 12 RxHeader ? ? c Keep-Alive: 300 > ? 12 RxHeader ? ? c Connection: keep-alive > ? 12 RxHeader ? ? c If-Modified-Since: Tue, 17 Nov 2009 19:10:17 GMT > ? 12 RxHeader ? ? c Cache-Control: max-age=0 > ? 12 VCL_call ? ? c recv > ? 12 VCL_return ? c lookup > ? 12 VCL_call ? ? c hash > ? 12 VCL_return ? c hash > ? 12 VCL_call ? ? c miss > ? 12 VCL_return ? c fetch > ? 13 BackendOpen ?b default 192.168.55.18 38239 192.168.9.158 80 > ? 12 Backend ? ? ?c 13 default default > ? 13 TxRequest ? ?b GET > ? 13 TxURL ? ? ? ?b /test/prueba.php > ? 13 TxProtocol ? b HTTP/1.1 > ? 13 TxHeader ? ? b Host: www.foo.com > ? 13 TxHeader ? ? b User-Agent: Mozilla/5.0 (X11; U; Linux i686; > en-US; rv:1.9.0.3) Gecko/2008092816 Iceweasel/3.0.3 (Debian-3.0.3-3) > ? 13 TxHeader ? ? b Accept: > text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 > ? 13 TxHeader ? ? b Accept-Language: en-us,en;q=0.5 > ? 13 TxHeader ? ? b Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 > ? 13 TxHeader ? ? b X-Forwarded-For: 192.168.90.41 > ? 13 TxHeader ? ? b Accept-Encoding: gzip > ? 13 TxHeader ? ? b X-Varnish: 484826741 > ? 13 TxHeader ? ? b X-Forwarded-For: 192.168.90.41 > ? 13 RxProtocol ? b HTTP/1.1 > ? 13 RxStatus ? ? b 304 > ? 13 RxResponse ? b Not Modified > ? 13 RxHeader ? ? b Date: Tue, 01 Dec 2009 10:50:33 GMT > ? 13 RxHeader ? ? b Server: Apache > ? 13 RxHeader ? ? b Connection: close > ? 13 RxHeader ? ? b Cache-control: max-age=401 > ? 12 ObjProtocol ?c HTTP/1.1 > ? 12 ObjStatus ? ?c 304 > ? 12 ObjResponse ?c Not Modified > ? 12 ObjHeader ? ?c Date: Tue, 01 Dec 2009 10:50:33 GMT > ? 12 ObjHeader ? ?c Server: Apache > ? 12 ObjHeader ? ?c Cache-control: max-age=401 > ? 13 BackendClose b default > ? 12 TTL ? ? ? ? ?c 484826741 RFC 401 1259664631 0 0 401 0 > ? 12 VCL_call ? ? c fetch > ? 12 VCL_return ? c pass > ? 12 Length ? ? ? c 0 > ? 12 VCL_call ? ? c deliver > ? 12 VCL_return ? c deliver > ? 12 TxProtocol ? c HTTP/1.1 > ? 12 TxStatus ? ? c 304 > ? 12 TxResponse ? c Not Modified > ? 12 TxHeader ? ? c Server: Apache > ? 12 TxHeader ? ? c Cache-control: max-age=401 > ? 12 TxHeader ? ? c Content-Length: 0 > ? 12 TxHeader ? ? c Date: Tue, 01 Dec 2009 10:50:31 GMT > ? 12 TxHeader ? ? c X-Varnish: 484826741 > ? 12 TxHeader ? ? c Age: 0 > ? 12 TxHeader ? ? c Via: 1.1 varnish > ? 12 TxHeader ? ? c Connection: keep-alive > ? 12 ReqEnd ? ? ? c 484826741 1259664631.945969105 > 1259664631.969969034 0.000000000 0.023999929 0.000000000 > ? 12 Debug ? ? ? ?c "herding" > ? ?0 CLI ? ? ? ? ?- Rd ping > ? ?0 CLI ? ? ? ? ?- Wr 0 200 PONG 1259664632 1.0 > ? ?0 CLI ? ? ? ? ?- Rd ping > ? ?0 CLI ? ? ? ? ?- Wr 0 200 PONG 1259664635 1.0 > ? 12 SessionClose c timeout > ? 12 StatSess ? ? c 192.168.90.41 37499 0 0 0 0 0 0 0 0 > ? ?0 CLI ? ? ? ? ?- Rd ping > ? ?0 CLI ? ? ? ? ?- Wr 0 200 PONG 1259664638 1.0 > ? ?0 CLI ? ? ? ? ?- Rd ping > ? ?0 CLI ? ? ? ? ?- Wr 0 200 PONG 1259664641 1.0 > ? ?0 CLI ? ? ? ? ?- Rd ping > ? ?0 CLI ? ? ? ? ?- Wr 0 200 PONG 1259664644 1.0 > ? ?0 CLI ? ? ? ? ?- Rd ping > ? ?0 CLI ? ? ? ? ?- Wr 0 200 PONG 1259664647 1.0 > ? 12 SessionOpen ?c 192.168.90.41 37500 :80 > ? 12 ReqStart ? ? c 192.168.90.41 37500 484826742 > ? 12 RxRequest ? ?c GET > ? 12 RxURL ? ? ? ?c /test/prueba.php > ? 12 RxProtocol ? c HTTP/1.1 > ? 12 RxHeader ? ? c Host: www.foo.com > ? 12 RxHeader ? ? c User-Agent: Mozilla/5.0 (X11; U; Linux i686; > en-US; rv:1.9.0.3) Gecko/2008092816 Iceweasel/3.0.3 (Debian-3.0.3-3) > ? 12 RxHeader ? ? c Accept: > text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 > ? 12 RxHeader ? ? c Accept-Language: en-us,en;q=0.5 > ? 12 RxHeader ? ? c Accept-Encoding: gzip,deflate > ? 12 RxHeader ? ? c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 > ? 12 RxHeader ? ? c Keep-Alive: 300 > ? 12 RxHeader ? ? c Connection: keep-alive > ? 12 RxHeader ? ? c If-Modified-Since: Tue, 17 Nov 2009 19:10:17 GMT > ? 12 RxHeader ? ? c Cache-Control: max-age=0 > ? 12 VCL_call ? ? c recv > ? 12 VCL_return ? c lookup > ? 12 VCL_call ? ? c hash > ? 12 VCL_return ? c hash > ? 12 HitPass ? ? ?c 484826741 > ? 12 VCL_call ? ? c pass > ? 12 VCL_return ? c pass > ? 13 BackendOpen ?b default 192.168.55.18 38240 192.168.9.158 80 > ? 12 Backend ? ? ?c 13 default default > ? 13 TxRequest ? ?b GET > ? 13 TxURL ? ? ? ?b /test/prueba.php > ? 13 TxProtocol ? b HTTP/1.1 > ? 13 TxHeader ? ? b Host: www.foo.com > ? 13 TxHeader ? ? b User-Agent: Mozilla/5.0 (X11; U; Linux i686; > en-US; rv:1.9.0.3) Gecko/2008092816 Iceweasel/3.0.3 (Debian-3.0.3-3) > ? 13 TxHeader ? ? b Accept: > text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 > ? 13 TxHeader ? ? b Accept-Language: en-us,en;q=0.5 > ? 13 TxHeader ? ? b Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 > ? 13 TxHeader ? ? b If-Modified-Since: Tue, 17 Nov 2009 19:10:17 GMT > ? 13 TxHeader ? ? b X-Forwarded-For: 192.168.90.41 > ? 13 TxHeader ? ? b Accept-Encoding: gzip > ? 13 TxHeader ? ? b X-Varnish: 484826742 > ? 13 TxHeader ? ? b X-Forwarded-For: 192.168.90.41 > ? 13 RxProtocol ? b HTTP/1.1 > ? 13 RxStatus ? ? b 304 > ? 13 RxResponse ? b Not Modified > ? 13 RxHeader ? ? b Date: Tue, 01 Dec 2009 10:50:49 GMT > ? 13 RxHeader ? ? b Server: Apache > ? 13 RxHeader ? ? b Connection: close > ? 13 RxHeader ? ? b Cache-control: max-age=401 > ? 12 ObjProtocol ?c HTTP/1.1 > ? 12 ObjStatus ? ?c 304 > ? 12 ObjResponse ?c Not Modified > ? 12 ObjHeader ? ?c Date: Tue, 01 Dec 2009 10:50:49 GMT > ? 12 ObjHeader ? ?c Server: Apache > ? 12 ObjHeader ? ?c Cache-control: max-age=401 > ? 13 BackendClose b default > ? 12 TTL ? ? ? ? ?c 484826742 RFC 401 1259664647 0 0 401 0 > ? 12 VCL_call ? ? c fetch > ? 12 VCL_return ? c pass > ? 12 Length ? ? ? c 0 > ? 12 VCL_call ? ? c deliver > ? 12 VCL_return ? c deliver > ? 12 TxProtocol ? c HTTP/1.1 > ? 12 TxStatus ? ? c 304 > ? 12 TxResponse ? c Not Modified > ? 12 TxHeader ? ? c Server: Apache > ? 12 TxHeader ? ? c Cache-control: max-age=401 > ? 12 TxHeader ? ? c Content-Length: 0 > ? 12 TxHeader ? ? c Date: Tue, 01 Dec 2009 10:50:47 GMT > ? 12 TxHeader ? ? c X-Varnish: 484826742 > ? 12 TxHeader ? ? c Age: 0 > ? 12 TxHeader ? ? c Via: 1.1 varnish > ? 12 TxHeader ? ? c Connection: keep-alive > ? 12 ReqEnd ? ? ? c 484826742 1259664647.989969015 > 1259664647.993968964 0.000000000 0.003999949 0.000000000 > ? 12 Debug ? ? ? ?c "herding" > ? ?0 CLI ? ? ? ? ?- Rd ping > ? ?0 CLI ? ? ? ? ?- Wr 0 200 PONG 1259664650 1.0 > ? 12 ReqStart ? ? c 192.168.90.41 37500 484826743 > ? 12 RxRequest ? ?c GET > ? 12 RxURL ? ? ? ?c /test/prueba.php > ? 12 RxProtocol ? c HTTP/1.1 > ? 12 RxHeader ? ? c Host: www.foo.com > ? 12 RxHeader ? ? c User-Agent: Mozilla/5.0 (X11; U; Linux i686; > en-US; rv:1.9.0.3) Gecko/2008092816 Iceweasel/3.0.3 (Debian-3.0.3-3) > ? 12 RxHeader ? ? c Accept: > text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 > ? 12 RxHeader ? ? c Accept-Language: en-us,en;q=0.5 > ? 12 RxHeader ? ? c Accept-Encoding: gzip,deflate > ? 12 RxHeader ? ? c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 > ? 12 RxHeader ? ? c Keep-Alive: 300 > ? 12 RxHeader ? ? c Connection: keep-alive > ? 12 RxHeader ? ? c If-Modified-Since: Tue, 17 Nov 2009 19:10:17 GMT > ? 12 RxHeader ? ? c Cache-Control: max-age=0 > ? 12 VCL_call ? ? c recv > ? 12 VCL_return ? c lookup > ? 12 VCL_call ? ? c hash > ? 12 VCL_return ? c hash > ? 12 HitPass ? ? ?c 484826741 > ? 12 VCL_call ? ? c pass > ? 12 VCL_return ? c pass > ? 13 BackendOpen ?b default 192.168.55.18 38241 192.168.9.158 80 > ? 12 Backend ? ? ?c 13 default default > ? 13 TxRequest ? ?b GET > ? 13 TxURL ? ? ? ?b /test/prueba.php > ? 13 TxProtocol ? b HTTP/1.1 > ? 13 TxHeader ? ? b Host: www.foo.com > ? 13 TxHeader ? ? b User-Agent: Mozilla/5.0 (X11; U; Linux i686; > en-US; rv:1.9.0.3) Gecko/2008092816 Iceweasel/3.0.3 (Debian-3.0.3-3) > ? 13 TxHeader ? ? b Accept: > text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 > ? 13 TxHeader ? ? b Accept-Language: en-us,en;q=0.5 > ? 13 TxHeader ? ? b Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 > ? 13 TxHeader ? ? b If-Modified-Since: Tue, 17 Nov 2009 19:10:17 GMT > ? 13 TxHeader ? ? b X-Forwarded-For: 192.168.90.41 > ? 13 TxHeader ? ? b Accept-Encoding: gzip > ? 13 TxHeader ? ? b X-Varnish: 484826743 > ? 13 TxHeader ? ? b X-Forwarded-For: 192.168.90.41 > ? 13 RxProtocol ? b HTTP/1.1 > ? 13 RxStatus ? ? b 304 > ? 13 RxResponse ? b Not Modified > ? 13 RxHeader ? ? b Date: Tue, 01 Dec 2009 10:50:53 GMT > ? 13 RxHeader ? ? b Server: Apache > ? 13 RxHeader ? ? b Connection: close > ? 13 RxHeader ? ? b Cache-control: max-age=401 > ? 12 ObjProtocol ?c HTTP/1.1 > ? 12 ObjStatus ? ?c 304 > ? 12 ObjResponse ?c Not Modified > ? 12 ObjHeader ? ?c Date: Tue, 01 Dec 2009 10:50:53 GMT > ? 12 ObjHeader ? ?c Server: Apache > ? 12 ObjHeader ? ?c Cache-control: max-age=401 > ? 13 BackendClose b default > ? 12 TTL ? ? ? ? ?c 484826743 RFC 401 1259664652 0 0 401 0 > ? 12 VCL_call ? ? c fetch > ? 12 VCL_return ? c pass > ? 12 Length ? ? ? c 0 > ? 12 VCL_call ? ? c deliver > ? 12 VCL_return ? c deliver > ? 12 TxProtocol ? c HTTP/1.1 > ? 12 TxStatus ? ? c 304 > ? 12 TxResponse ? c Not Modified > ? 12 TxHeader ? ? c Server: Apache > ? 12 TxHeader ? ? c Cache-control: max-age=401 > ? 12 TxHeader ? ? c Content-Length: 0 > ? 12 TxHeader ? ? c Date: Tue, 01 Dec 2009 10:50:52 GMT > ? 12 TxHeader ? ? c X-Varnish: 484826743 > ? 12 TxHeader ? ? c Age: 0 > ? 12 TxHeader ? ? c Via: 1.1 varnish > ? 12 TxHeader ? ? c Connection: keep-alive > ? 12 ReqEnd ? ? ? c 484826743 1259664652.677968979 > 1259664652.685969114 4.684000015 0.008000135 0.000000000 > ? 12 Debug ? ? ? ?c "herding" > ? ?0 CLI ? ? ? ? ?- Rd ping > ? ?0 CLI ? ? ? ? ?- Wr 0 200 PONG 1259664653 1.0 > ? ?0 CLI ? ? ? ? ?- Rd ping > ? ?0 CLI ? ? ? ? ?- Wr 0 200 PONG 1259664656 1.0 > > On Tue, Dec 1, 2009 at 8:23 AM, Tollef Fog Heen > wrote: >> ]] Daniel Rodriguez >> >> | ? ?13 TxHeader ? ? b Cookie: foo.comGlobal=0; cUser=nouser >> >> You need to strip the cookie before sending the request to the backend, >> else you will end up pass-ing the request. >> >> -- >> Tollef Fog Heen >> Redpill Linpro -- Changing the game! >> t: +47 21 54 41 73 >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at projects.linpro.no >> http://projects.linpro.no/mailman/listinfo/varnish-misc >> > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc > From jcpetit at syspark.com Tue Dec 1 20:26:35 2009 From: jcpetit at syspark.com (Jean-Christophe Petit) Date: Tue, 01 Dec 2009 15:26:35 -0500 Subject: X-Forwarded-Forand 2 layers of Varnish Message-ID: <4B157BFB.2090401@syspark.com> Hello, we use 2 layers of Varnish and it proved to be highly scalable and efficient. But on the second layer, we are not able to get the X-Forwarded-For from the first layer to sent it to the Backend (Apache) In fact, on the second layer we have 2 X-Forwarded-For (Visitor + first Varnish layer) Is there something we can do on the first Varnish layer to only transmit the X-Forwarded-For of the visitor ? Thanks, -- Jean-Christophe Petit VP R&D et DSI GROUPE SYSPARK From kristian at redpill-linpro.com Wed Dec 2 12:04:29 2009 From: kristian at redpill-linpro.com (Kristian Lyngstol) Date: Wed, 2 Dec 2009 13:04:29 +0100 Subject: Saint mode In-Reply-To: <20091118135238.GA21288@fupp.net> References: <20091118135238.GA21288@fupp.net> Message-ID: <20091202120429.GB6614@kjeks.linpro.no> Hi Anders, On Wed, Nov 18, 2009 at 02:52:39PM +0100, Anders Nordby wrote: > We spoke briefly about in the Varnish meeting. Unfortunately I can not > remember the conclusion(s), so I ask again: > > 1) How to check in vcl_recv if a request was restarted saintmode? You > mentioned something about checking TTL? Does it matter if that request > is handled with pass or lookup? I want to avoid mixing up restarts due > to saint mode and restarts due to what should have been normal passes in > vcl_fetch (IMO) but which are restarts to handle it in vcl_recv to avoid > "hit for pass" problems. In vcl_recv we don't have a ttl, so you can't specifically check. You could do something simple like: sub vcl_fetch { .... if (beresp.status != 200) { set req.http.X-Saint-restart = 1; set beresp.saintmode = 1m; restart; } ... } That way you can easily check in recv. It's not perfect, but unless we have the TTL, that's the only way. In vcl_hit you can check if obj.ttl is negative, if it is, then you are using a graced object. > 2) How do use saint mode when backend fails to respond, covering > everything from no response/connection to backend just resetting the > connection? As far as my syslogging shows, connection failures are adressed > in vcl_error and easy/possible to catch in vcl_fetch? That's an excellent question. Implementation-wise saint-mode can handle this, but we loose a bit too much information before we reach vcl_error, it seems. Saint-mode uses both the backend and the objecthead(reference) to work, and we detach it a bit too soon. PHK: Do you have any idea how to solve this? We've talked about making more information available in vcl_error before, or just using vcl_fetch... > Cheers from Gyoda, Saitama, Japan. Cheers :) Just finished 2.5 weeks of various forms of training and traveling myself, trying to catch up on mail and whatnot. Apologies for the somewhat delayed response. -- Kristian Lyngst?l Redpill Linpro AS Tlf: +47 21544179 Mob: +47 99014497 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From stockrt at gmail.com Wed Dec 2 14:10:29 2009 From: stockrt at gmail.com (Rogerio Schneider) Date: Wed, 2 Dec 2009 12:10:29 -0200 Subject: Handling 304 and header refresh In-Reply-To: <30b80c9f0912010644i6c145a04u3a2d4a0fb1e377ea@mail.gmail.com> References: <95ff419c0911270534s35f02db8y9c0f49da8cb36e2d@mail.gmail.com> <87ws17kvib.fsf@qurzaw.linpro.no> <95ff419c0912010439u5646f427tc302ea36f17912a2@mail.gmail.com> <30b80c9f0912010644i6c145a04u3a2d4a0fb1e377ea@mail.gmail.com> Message-ID: <18D06E85-9F52-47D7-96F9-4B1ABAE74A75@gmail.com> Em 01/12/2009, ?s 12:44, Bedis 9 escreveu: > By the way, it's a very bad idea to setup a Cache-Control header with > a max-age value on a 304 response. Why is that bad? I believe the RFC says the opposite. Regards, Rogerio Schneider From stockrt at gmail.com Wed Dec 2 20:53:38 2009 From: stockrt at gmail.com (Rogerio Schneider) Date: Wed, 2 Dec 2009 18:53:38 -0200 Subject: still using cache when fetching content In-Reply-To: <87skbvkvbf.fsf@qurzaw.linpro.no> References: <4B14948F.90405@syspark.com> <87skbvkvbf.fsf@qurzaw.linpro.no> Message-ID: Wasn't him talking about serving graced objects? Att, Rogerio Schneider Em 01/12/2009, ?s 05:27, Tollef Fog Heen escreveu: > ]] Jean-Christophe Petit > > | Is it possible to make Varnish sending the cache content at the same > | time it is fetching from the backend ? > > Not at the moment, no. > > | It will be more efficient for slow dynamic content ;) > | For example, I have a php page taking up to 5sec to run. If > Varnish was > | able to get it while still sending the old cache page, it would be > | really great. > > You could trick this by having a script that revisits any URLs that > are > slow every N minutes and set obj.ttl to 0s in vcl_hit if the user > agent > (or IP address) is what you expect for your script. Might not work if > you have many different URLs, though. > > -- > Tollef Fog Heen > Redpill Linpro -- Changing the game! > t: +47 21 54 41 73 > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc From tfheen at redpill-linpro.com Thu Dec 3 08:04:22 2009 From: tfheen at redpill-linpro.com (Tollef Fog Heen) Date: Thu, 03 Dec 2009 09:04:22 +0100 Subject: Handling 304 and header refresh In-Reply-To: <95ff419c0912010439u5646f427tc302ea36f17912a2@mail.gmail.com> (Daniel Rodriguez's message of "Tue, 1 Dec 2009 13:39:23 +0100") References: <95ff419c0911270534s35f02db8y9c0f49da8cb36e2d@mail.gmail.com> <87ws17kvib.fsf@qurzaw.linpro.no> <95ff419c0912010439u5646f427tc302ea36f17912a2@mail.gmail.com> Message-ID: <874oo8h4ah.fsf@qurzaw.linpro.no> ]] Daniel Rodriguez | 13 BackendOpen b default 192.168.55.18 38239 192.168.9.158 80 | 12 Backend c 13 default default | 13 TxRequest b GET | 13 TxURL b /test/prueba.php | 13 TxProtocol b HTTP/1.1 | 13 TxHeader b Host: www.foo.com | 13 TxHeader b User-Agent: Mozilla/5.0 (X11; U; Linux i686; | en-US; rv:1.9.0.3) Gecko/2008092816 Iceweasel/3.0.3 (Debian-3.0.3-3) | 13 TxHeader b Accept: | text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 | 13 TxHeader b Accept-Language: en-us,en;q=0.5 | 13 TxHeader b Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 | 13 TxHeader b X-Forwarded-For: 192.168.90.41 | 13 TxHeader b Accept-Encoding: gzip | 13 TxHeader b X-Varnish: 484826741 | 13 TxHeader b X-Forwarded-For: 192.168.90.41 | 13 RxProtocol b HTTP/1.1 | 13 RxStatus b 304 | 13 RxResponse b Not Modified | 13 RxHeader b Date: Tue, 01 Dec 2009 10:50:33 GMT | 13 RxHeader b Server: Apache | 13 RxHeader b Connection: close | 13 RxHeader b Cache-control: max-age=401 Your backend seems to reply with 304 to an unconditional GET request, which is quite odd. -- Tollef Fog Heen Redpill Linpro -- Changing the game! t: +47 21 54 41 73 From joe at joetify.com Thu Dec 3 23:23:57 2009 From: joe at joetify.com (Joe Williams) Date: Thu, 03 Dec 2009 15:23:57 -0800 Subject: vcl_error synthetic escape chars Message-ID: <4B18488D.8030105@joetify.com> I am attempting to write my own error message in vcl_error and am running into an issue with escaping out "{" and "}". synthetic {"\{"error":"not_found","reason":"missing"\} This compiles and works but when I produce the error I get the slashes in the response: root at ubuntu:~# curl http://localhost:6081/test/adsf \{"error":"not_found","reason":"missing"\} I searched around and didn't find anything, is it possible to escape out the curly braces and keep them from showing up in the response? -Joe -- Name: Joseph A. Williams Email: joe at joetify.com Blog: http://www.joeandmotorboat.com/ From phk at phk.freebsd.dk Fri Dec 4 10:59:53 2009 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Fri, 04 Dec 2009 10:59:53 +0000 Subject: vcl_error synthetic escape chars In-Reply-To: Your message of "Thu, 03 Dec 2009 15:23:57 PST." <4B18488D.8030105@joetify.com> Message-ID: <14941.1259924393@critter.freebsd.dk> In message <4B18488D.8030105 at joetify.com>, Joe Williams writes: >I am attempting to write my own error message in vcl_error and am >running into an issue with escaping out "{" and "}". > > synthetic {"\{"error":"not_found","reason":"missing"\} > >This compiles and works but when I produce the error I get the slashes >in the response: > >root at ubuntu:~# curl http://localhost:6081/test/adsf >\{"error":"not_found","reason":"missing"\} > >I searched around and didn't find anything, is it possible to escape out >the curly braces and keep them from showing up in the response? Yes I belive %7b ... %7d will work. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From coolbomb at gmail.com Fri Dec 4 12:10:41 2009 From: coolbomb at gmail.com (Daniel Rodriguez) Date: Fri, 4 Dec 2009 13:10:41 +0100 Subject: Handling 304 and header refresh In-Reply-To: <30b80c9f0912010644i6c145a04u3a2d4a0fb1e377ea@mail.gmail.com> References: <95ff419c0911270534s35f02db8y9c0f49da8cb36e2d@mail.gmail.com> <87ws17kvib.fsf@qurzaw.linpro.no> <95ff419c0912010439u5646f427tc302ea36f17912a2@mail.gmail.com> <30b80c9f0912010644i6c145a04u3a2d4a0fb1e377ea@mail.gmail.com> Message-ID: <95ff419c0912040410m799de9b5v80b666bd9765e6bf@mail.gmail.com> Hi, Sorry for my late reply, but i had a couple of out-of-office days and i was not able to write. You are right, in the second request there is no "If-Modified-Since" in the communication between the varnish-backend, but between the client-varnish there is a "If-Modified-Since" (this can be verified in the varnish logs attached before), so varnish some how is asking for a unconditional GET, and ignoring the "If-Modified-Since", like Tollef also stated in a later reply to this thread On Thu, Dec 3, 2009 at 9:04 AM, Tollef Fog Heen wrote: > ]] Daniel Rodriguez [...] > Your backend seems to reply with 304 to an unconditional GET request, > which is quite odd. So there is no way to get this to work as as i want?. I'm also interested in the reason of why it's a bad idea to set a a max-age value on a 304 response, and open to any suggestion. Best Regards, On Tue, Dec 1, 2009 at 3:44 PM, Bedis 9 wrote: > Hi, > > It sounds good that Varnish does not cache a 304 answer ;) > In the second request, you have no "If-Modified-Since" headers in your > request and your origin seems to answer you with a 304, which is not > realy what varnish should expect! > > By the way, it's a very bad idea to setup a Cache-Control header with > a max-age value on a 304 response. > > cheers > > [...] From stockrt at gmail.com Fri Dec 4 13:04:49 2009 From: stockrt at gmail.com (Rogerio Schneider) Date: Fri, 4 Dec 2009 11:04:49 -0200 Subject: Handling 304 and header refresh In-Reply-To: <95ff419c0912040410m799de9b5v80b666bd9765e6bf@mail.gmail.com> References: <95ff419c0911270534s35f02db8y9c0f49da8cb36e2d@mail.gmail.com> <87ws17kvib.fsf@qurzaw.linpro.no> <95ff419c0912010439u5646f427tc302ea36f17912a2@mail.gmail.com> <30b80c9f0912010644i6c145a04u3a2d4a0fb1e377ea@mail.gmail.com> <95ff419c0912040410m799de9b5v80b666bd9765e6bf@mail.gmail.com> Message-ID: Em 04/12/2009, ?s 10:10, Daniel Rodriguez escreveu: > I'm also interested in the reason of why it's a bad idea to set a a > max-age value on a 304 response, and open to any suggestion. It is not a bad idea. It is, as a matter of fact, recommend by the RFC. It is just odd that your backend answer with 304 for an unconditional GET, as Tollef pointed out. > Att, Rogerio Schneider From joe at joetify.com Fri Dec 4 16:51:44 2009 From: joe at joetify.com (Joe Williams) Date: Fri, 04 Dec 2009 08:51:44 -0800 Subject: vcl_error synthetic escape chars In-Reply-To: <14941.1259924393@critter.freebsd.dk> References: <14941.1259924393@critter.freebsd.dk> Message-ID: <4B193E20.6030105@joetify.com> Thanks Poul but I think I'm still doing something incorrectly. root at ubuntu:~# curl http://localhost:6081/test/adsf %7b"error":"not_found","reason":"missing"%7d synthetic {"%7b"error":"not_found","reason":"missing"%7d "}; Do I need to do something differently with the quotations? -Joe On 12/4/09 2:59 AM, Poul-Henning Kamp wrote: > In message<4B18488D.8030105 at joetify.com>, Joe Williams writes: > >> I am attempting to write my own error message in vcl_error and am >> running into an issue with escaping out "{" and "}". >> >> synthetic {"\{"error":"not_found","reason":"missing"\} >> >> This compiles and works but when I produce the error I get the slashes >> in the response: >> >> root at ubuntu:~# curl http://localhost:6081/test/adsf >> \{"error":"not_found","reason":"missing"\} >> >> I searched around and didn't find anything, is it possible to escape out >> the curly braces and keep them from showing up in the response? >> > Yes I belive %7b ... %7d will work. > > -- Name: Joseph A. Williams Email: joe at joetify.com Blog: http://www.joeandmotorboat.com/ From l at lrowe.co.uk Fri Dec 4 17:05:14 2009 From: l at lrowe.co.uk (Laurence Rowe) Date: Fri, 4 Dec 2009 18:05:14 +0100 Subject: vcl_error synthetic escape chars In-Reply-To: <4B193E20.6030105@joetify.com> References: <14941.1259924393@critter.freebsd.dk> <4B193E20.6030105@joetify.com> Message-ID: According to http://varnish.projects.linpro.no/wiki/VCLSyntaxStrings you need to use 'basic' strings (not 'long' strings with the curly brackets) for %xx escapes to work. Laurence 2009/12/4 Joe Williams : > Thanks Poul but I think I'm still doing something incorrectly. > > root at ubuntu:~# curl http://localhost:6081/test/adsf > %7b"error":"not_found","reason":"missing"%7d > > > ? ? ? ? synthetic {"%7b"error":"not_found","reason":"missing"%7d > "}; > > Do I need to do something differently with the quotations? > > -Joe > > > On 12/4/09 2:59 AM, Poul-Henning Kamp wrote: >> In message<4B18488D.8030105 at joetify.com>, Joe Williams writes: >> >>> I am attempting to write my own error message in vcl_error and am >>> running into an issue with escaping out "{" and "}". >>> >>> ? ? ? ? ?synthetic {"\{"error":"not_found","reason":"missing"\} >>> >>> This compiles and works but when I produce the error I get the slashes >>> in the response: >>> >>> root at ubuntu:~# curl http://localhost:6081/test/adsf >>> \{"error":"not_found","reason":"missing"\} >>> >>> I searched around and didn't find anything, is it possible to escape out >>> the curly braces and keep them from showing up in the response? >>> >> Yes I belive %7b ... %7d will work. >> >> > > -- > Name: Joseph A. Williams > Email: joe at joetify.com > Blog: http://www.joeandmotorboat.com/ > > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc > From jcpetit at syspark.com Mon Dec 7 04:17:45 2009 From: jcpetit at syspark.com (Jean-Christophe Petit) Date: Sun, 06 Dec 2009 23:17:45 -0500 Subject: varnish-2.0.5 hangs In-Reply-To: <4B157BFB.2090401@syspark.com> References: <4B157BFB.2090401@syspark.com> Message-ID: <4B1C81E9.9060203@syspark.com> Hello, we have Varnish-2.0.5 which hangs for some clients Seems to happen a few minutes after a spike happened. Most of the clients don't have any problem but some do. Here is the log we got: 94 SessionOpen c 70.26.255.105 50686 :80 94 ReqStart c 70.26.255.105 50686 410619754 94 RxRequest c GET 94 RxURL c / 94 RxProtocol c HTTP/1.1 94 RxHeader c Host: www.beezid.com 94 RxHeader c User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 (.NET CLR 3.5.30729) 94 RxHeader c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 94 RxHeader c Accept-Language: en-us,en;q=0.5 94 RxHeader c Accept-Encoding: gzip,deflate 94 RxHeader c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 94 RxHeader c Keep-Alive: 300 94 RxHeader c Connection: keep-alive 94 RxHeader c Cookie: partner_uid=; __utma=253075994.1982649326.1256356542.1260054994.1260109935.52; __utmz=253075994.1260109935.52.3.utmcsr=twitter.com|utmccn=(referral)|utmcmd=referral|utmcct=/; PHPSESSID=iio7dukkjdkiibvuvkp4cjr8s4; __utmc=253075994; ysm_CK1VFNP4VFSV 94 VCL_call c recv 94 VCL_return c lookup 94 VCL_call c hash 94 VCL_return c hash 94 VCL_call c hash 94 VCL_return c hash 94 VCL_call c hash 94 VCL_return c hash 94 VCL_call c hash 94 VCL_return c hash 94 VCL_call c hash 94 VCL_return c hash 94 VCL_call c hash 94 VCL_return c hash 94 VCL_call c hash 94 VCL_return c hash 94 VCL_call c hash 94 VCL_return c hash 94 VCL_call c hash 94 VCL_return c hash 94 VCL_call c hash 94 VCL_return c hash 94 VCL_call c hash 94 VCL_return c hash 94 VCL_call c hash 94 VCL_return c hash 94 VCL_call c hash 94 VCL_return c hash 94 VCL_call c hash 94 VCL_return c hash 94 VCL_call c hash 94 VCL_return c hash 94 VCL_call c hash 94 VCL_return c hash 94 VCL_call c miss 94 VCL_return c fetch 94 Backend c 13 default default Is it something to do with the backend not accessible ? Thanks, -- Jean-Christophe Petit SYSPARK Inc. From tfheen at redpill-linpro.com Tue Dec 8 07:59:08 2009 From: tfheen at redpill-linpro.com (Tollef Fog Heen) Date: Tue, 08 Dec 2009 08:59:08 +0100 Subject: Handling 304 and header refresh In-Reply-To: <95ff419c0912040410m799de9b5v80b666bd9765e6bf@mail.gmail.com> (Daniel Rodriguez's message of "Fri, 4 Dec 2009 13:10:41 +0100") References: <95ff419c0911270534s35f02db8y9c0f49da8cb36e2d@mail.gmail.com> <87ws17kvib.fsf@qurzaw.linpro.no> <95ff419c0912010439u5646f427tc302ea36f17912a2@mail.gmail.com> <30b80c9f0912010644i6c145a04u3a2d4a0fb1e377ea@mail.gmail.com> <95ff419c0912040410m799de9b5v80b666bd9765e6bf@mail.gmail.com> Message-ID: <874oo1rj5f.fsf@qurzaw.linpro.no> ]] Daniel Rodriguez | You are right, in the second request there is no "If-Modified-Since" | in the communication between the varnish-backend, but between the | client-varnish there is a "If-Modified-Since" (this can be verified | in the varnish logs attached before), so varnish some how is asking | for a unconditional GET, and ignoring the "If-Modified-Since", like | Tollef also stated in a later reply to this thread Varnish doesn't do conditional GETs to the backend, so this is perfectly normal. | On Thu, Dec 3, 2009 at 9:04 AM, Tollef Fog Heen | wrote: | > ]] Daniel Rodriguez | [...] | > Your backend seems to reply with 304 to an unconditional GET request, | > which is quite odd. | | So there is no way to get this to work as as i want?. What do you want? A 304 response is not a valid response to an unconditional GET. | I'm also interested in the reason of why it's a bad idea to set a a | max-age value on a 304 response, and open to any suggestion. As Rogerio Schneider points out, it's not. -- Tollef Fog Heen Redpill Linpro -- Changing the game! t: +47 21 54 41 73 From schmidt at ze.tum.de Tue Dec 8 08:20:25 2009 From: schmidt at ze.tum.de (Gerhard Schmidt) Date: Tue, 08 Dec 2009 09:20:25 +0100 Subject: Why a second ReqEnd after SessionClose Message-ID: <4B1E0C49.1030003@ze.tum.de> Hi, I'm write a program that retrieves the log data directly from the shm of varnish. As I understand ReqEnd marks the end of a processed Request and SessionClose marks the close of a Client session and StatSess marks the end ob a Client session. Why is there a second ReqEnd between SessionClose and StatSess with Start and Endtime exactly the same time. 7 ReqEnd c 716311085 1260258453.362281322 1260258454.134468794 0.000075102 0.772091150 0.000096321 7 SessionClose c EOF 7 ReqEnd c 0 1260258454.135515213 1260258454.135515213 0.001046419 0.000000000 0.000000000 7 StatSess c 192.168.0.1 56958 1 1 1 0 0 1 504 76126 For ReqEnd I found a documentation but for the other LogEnties with multiple values like ReqStart, TTL, BackendOpen, StatSess etc I didn't find anything. Is there a documentation anywhere or can someone please tell me what the values mean. Regards Estartu -- ---------------------------------------------------------- Gerhard Schmidt | E-Mail: schmidt at ze.tum.de Technische Universit?t M?nchen | WWW & Online Services | Tel: +49 89 289-25270 | PGP-PublicKey Fax: +49 89 289-25257 | on request From fernando.papa at liveinteractive.net Tue Dec 8 17:01:49 2009 From: fernando.papa at liveinteractive.net (Fernando Papa) Date: Tue, 8 Dec 2009 14:01:49 -0300 Subject: only 503s after a few hours (trunk-4384) Message-ID: <001a01ca7828$246bf400$7802a8c0@interactive.live.net> Hello, After a few hours, Varnish trunk-4384 started replying with 503. New work-load, first day of use. Previously used Varnish only with internal traffic (eg no crazy headers, just carefully crafted requests). Facts: * Xeon 2.4 GHz (quad-core, 512KBx4 cache) server. CentOS 5.4. * Worked fine for ~9 hours, sending to backend 150+ K requests plus Varnish pings. * Backend is an application server. Payload is <1 KB plus minimal headers. * I can still access backend bypassing Varnish. * Varnish continues pinging backend, and receiving a timely 200 OK every time (backend logs). * req.backend.healthy = true. Tested with a codepath along the lines (req.request == "GET") && (req.url == "/") && (req.backend.healthy), that does an 'error 200 "OK"'. * varnishlog does not output anything at all. Not even the backend pings. * Tested every path within vcl_recv. Exiting with "error" works, but "lookup" results in a 503. * Very low CPU usage. Sorry, didn't check memory usage before restarting it. CFLAGS='-O2 -fomit-frame-pointer -march=pentium4 -mfpmath=sse' ./configure --prefix=$HOME/local --disable-largefile $HOME/local/sbin/varnishd -a *:80 -f $HOME/local/etc/varnish/xxx.vcl -p obj_workspace=4096 -s malloc,100M -p thread_pools=`grep -c 'model name' /proc/cpuinfo` backend fd1 { .host="localhost"; .port="85"; .connect_timeout = 1s; .first_byte_timeout = 20s; .between_bytes_timeout =2s; .probe = { .request = "GET /xxx HTTP/1.1" "Host: yyy" "Connection: close"; .timeout = 0.5 s; .window = 8; .threshold = 3; } } director fdcluster round-robin { { .backend = fd1; } } Restarted Varnish, now it works again. Any idea? Anything wrong in the command line? Thanks, Fernando. From plomer at boerse-go.de Wed Dec 9 10:46:05 2009 From: plomer at boerse-go.de (Mark Plomer) Date: Wed, 09 Dec 2009 11:46:05 +0100 Subject: Monitoring backend status Message-ID: <4B1F7FED.4020501@boerse-go.de> Hi, is it possible to retrieve some more details about the current backend status via the telnet interface? A list of all backends with current status (up/down) would be very helpful for monitoring. For the beginning, it would be enough to have the count of backends and the count of up-/down- backends in the "stats" output. Thanks Mark From phk at phk.freebsd.dk Wed Dec 9 10:56:24 2009 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Wed, 09 Dec 2009 10:56:24 +0000 Subject: Monitoring backend status In-Reply-To: Your message of "Wed, 09 Dec 2009 11:46:05 +0100." <4B1F7FED.4020501@boerse-go.de> Message-ID: <6057.1260356184@critter.freebsd.dk> In message <4B1F7FED.4020501 at boerse-go.de>, Mark Plomer writes: >Hi, >is it possible to retrieve some more details about the current backend >status via the telnet interface? >A list of all backends with current status (up/down) would be very >helpful for monitoring. >For the beginning, it would be enough to have the count of backends and >the count of up-/down- backends >in the "stats" output. There is an undocumented "debug.health" command you can try. Ideas for a final version of this are most welcom. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From rtshilston at gmail.com Wed Dec 9 11:31:28 2009 From: rtshilston at gmail.com (Rob S) Date: Wed, 09 Dec 2009 11:31:28 +0000 Subject: Monitoring backend status In-Reply-To: <6057.1260356184@critter.freebsd.dk> References: <6057.1260356184@critter.freebsd.dk> Message-ID: <4B1F8A90.7080003@gmail.com> Poul-Henning Kamp wrote: > In message <4B1F7FED.4020501 at boerse-go.de>, Mark Plomer writes: > >> Hi, >> is it possible to retrieve some more details about the current backend >> status via the telnet interface? >> A list of all backends with current status (up/down) would be very >> helpful for monitoring. >> For the beginning, it would be enough to have the count of backends and >> the count of up-/down- backends >> in the "stats" output. >> > > There is an undocumented "debug.health" command you can try. > > Ideas for a final version of this are most welcom. > > Brilliant. At the moment we tail the varnishlog, post process, and write out to disk for further monitoring. Here's the PHP we're using: #!/usr/local/bin/php Our monitoring then records "grep -c healthy /tmp/varnishbackendhealth" However, it looks like we can now use the far simpler: echo 'debug.health' | nc localhost 6082 |grep -c Healthy So - thanks for the change! Rob From fernando.papa at liveinteractive.net Wed Dec 9 13:51:04 2009 From: fernando.papa at liveinteractive.net (Fernando Papa) Date: Wed, 9 Dec 2009 10:51:04 -0300 Subject: only 503s after a few hours (trunk-4384) References: <001a01ca7828$246bf400$7802a8c0@interactive.live.net> Message-ID: <002b01ca78d6$a7de2b10$7802a8c0@interactive.live.net> Happened again. This time I left varnishlog running, and got... 989 TxStatus c 503 989 TxResponse c Service Unavailable 989 TxHeader c Server: Varnish 989 TxHeader c Date: Wed, 09 Dec 2009 00:38:07 GMT 989 TxHeader c Age: 0 989 TxHeader c Connection: close 989 ReqEnd c 1933950257 1260319087.004358053 1260319087.004553556 0.000067711 0.000121117 0.000074387 989 SessionClose c error 989 StatSess c 189.165.32.95 1480 0 1 1 0 0 0 117 0 5 Debug - "Too many open files when accept(2)ing. Sleeping." That "Too many..." debug message appears 709 times. First one: 1023 TxProtocol c HTTP/1.1 1023 TxStatus c 200 1023 TxResponse c OK 1023 TxHeader c Server: xxx 1023 TxHeader c Cache-Control: public, max-age=3600 1023 TxHeader c Content-Type: application/x-shockwave-flash 1023 TxHeader c Content-Length: 659 1023 TxHeader c Date: Wed, 09 Dec 2009 00:10:05 GMT 1023 TxHeader c Age: 972 1023 TxHeader c Connection: keep-alive 1023 ReqEnd c 1933931697 1260317405.392798424 1260317405.392909527 0.000062466 0.000048637 0.000062466 706 Debug c "herding" 695 Debug c "herding" 1007 Debug c "herding" 1023 Debug c "herding" 5 Debug - "Too many open files when accept(2)ing. Sleeping." 937 ReqStart c 190.13.3.173 51259 1933931698 937 RxRequest c GET 937 RxURL c xxx 937 RxProtocol c HTTP/1.1 937 RxHeader c Accept: */* 937 RxHeader c Referer: xxx 937 RxHeader c Accept-Language: es-co 937 RxHeader c UA-CPU: x86 937 RxHeader c Accept-Encoding: gzip, deflate 937 RxHeader c User-Agent: xxx 937 RxHeader c Host: xxx 937 RxHeader c Connection: Keep-Alive 937 RxHeader c Cookie: xxx=xxx 937 VCL_call c recv 937 VCL_return c lookup 937 VCL_call c hash 937 VCL_return c hash 937 VCL_call c miss 937 VCL_return c fetch 937 FetchError c no backend connection 937 VCL_call c error 937 VCL_return c deliver Varnish account does not have limits (ulimit returns unlimited). Memory usage (top): 403m/89m/81m. # ps -AF | grep varnish root 19223 1 0 21419 928 2 Dec08 ? 00:00:00 varnishd -a *:80 -f xxx.vcl -p obj_workspace 4096 -s malloc,100M -p thread_pools 4 nobody 19224 19223 0 103378 91244 3 Dec08 ? 00:06:52 varnishd -a *:80 -f xxx.vcl -p obj_workspace 4096 -s malloc,100M -p thread_pools 4 # kill 19224 # ps -AF | grep varnish root 19223 1 0 21419 960 2 Dec08 ? 00:00:00 varnishd -a *:80 -f xxx.vcl -p obj_workspace 4096 -s malloc,100M -p thread_pools 4 nobody 31319 19223 1 94673 3532 3 12:33 ? 00:00:00 varnishd -a *:80 -f xxx.vcl -p obj_workspace 4096 -s malloc,100M -p thread_pools 4 That made it work again --as in processing requests--, except for varnishlog that continues to show nothing. ----- Original Message ----- From: "Fernando Papa" To: Sent: Tuesday, December 08, 2009 2:01 PM Subject: only 503s after a few hours (trunk-4384) > Hello, > > After a few hours, Varnish trunk-4384 started replying with 503. > New work-load, first day of use. Previously used Varnish only with > internal > traffic (eg no crazy headers, just carefully crafted requests). > > Facts: > > * Xeon 2.4 GHz (quad-core, 512KBx4 cache) server. CentOS 5.4. > * Worked fine for ~9 hours, sending to backend 150+ K requests plus > Varnish > pings. > * Backend is an application server. Payload is <1 KB plus minimal > headers. > * I can still access backend bypassing Varnish. > * Varnish continues pinging backend, and receiving a timely 200 OK every > time (backend logs). > * req.backend.healthy = true. Tested with a codepath along the lines > (req.request == "GET") && (req.url == "/") && (req.backend.healthy), that > does an 'error 200 "OK"'. > * varnishlog does not output anything at all. Not even the backend pings. > * Tested every path within vcl_recv. Exiting with "error" works, but > "lookup" results in a 503. > * Very low CPU usage. Sorry, didn't check memory usage before restarting > it. > > CFLAGS='-O2 -fomit-frame-pointer -march=pentium4 -mfpmath=sse' > ./configure --prefix=$HOME/local --disable-largefile > > $HOME/local/sbin/varnishd -a *:80 -f $HOME/local/etc/varnish/xxx.vcl -p > obj_workspace=4096 -s malloc,100M -p thread_pools=`grep -c 'model name' > /proc/cpuinfo` > > backend fd1 > { > .host="localhost"; > .port="85"; > .connect_timeout = 1s; > .first_byte_timeout = 20s; > .between_bytes_timeout =2s; > .probe = { > .request = > "GET /xxx HTTP/1.1" > "Host: yyy" > "Connection: close"; > .timeout = 0.5 s; > .window = 8; > .threshold = 3; > } > } > > director fdcluster round-robin > { > { .backend = fd1; } > } > > Restarted Varnish, now it works again. > > Any idea? Anything wrong in the command line? > > Thanks, > Fernando. > From phk at phk.freebsd.dk Wed Dec 9 14:07:13 2009 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Wed, 09 Dec 2009 14:07:13 +0000 Subject: only 503s after a few hours (trunk-4384) In-Reply-To: Your message of "Wed, 09 Dec 2009 10:51:04 -0300." <002b01ca78d6$a7de2b10$7802a8c0@interactive.live.net> Message-ID: <6687.1260367633@critter.freebsd.dk> In message <002b01ca78d6$a7de2b10$7802a8c0 at interactive.live.net>, "Fernando Pap a" writes: >Happened again. This time I left varnishlog running, and got... > 5 Debug - "Too many open files when accept(2)ing. Sleeping." > That's your trouble: The kernel does not allow varnish to open as many filedescriptors as it needs. Use whatever means your kernel has to change this number, possibly ulimit(1) or similar. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From jhayter at manta.com Wed Dec 9 16:46:27 2009 From: jhayter at manta.com (Jim Hayter) Date: Wed, 9 Dec 2009 11:46:27 -0500 Subject: Need help understanding backend connection errors Message-ID: Hi, I'm looking at implementing varnish 2.0.5 as a front end to our apache web servers to cache static files. I'm currently testing and having issues with backend connections. Since this is a test system, it is not heavily used. I notice that if it goes without any requests for some time (over an hour or so), the next request received goes through the cycle below of 'FetchError c no backend connection' and restart until it hits the restart limit of 6. It also seems that if requests occur every few minutes that this is not an issue, so I suspect a timeout of some kind. I've been searching and reading but have not found an explanation for this behavior yet. More complete log of this can be seen at http://www.manta.com/jah/varnish_log.txt The command line I'm using to run varnish can be seen at http://www.manta.com/jah/varnish_start.txt Current vcl can be seen at http://www.manta.com/jah/varnish_vcl.txt I'd appreciate any insight. Thanks, Jim Varnish_log.txt excerpt: 4 VCL_call c recv 4 VCL_return c pass 4 VCL_call c pass 4 VCL_return c pass 31 BackendClose b default 5 BackendClose b default 4 FetchError c no backend connection 4 VCL_call c error 4 VCL_return c restart 4 VCL_call c recv 4 VCL_return c pass 4 VCL_call c pass 4 VCL_return c pass 4 FetchError c no backend connection 4 VCL_call c error 4 VCL_return c restart 4 VCL_call c recv 4 VCL_return c pass 4 VCL_call c pass 4 VCL_return c pass 4 FetchError c no backend connection 4 VCL_call c error 4 VCL_return c restart 4 VCL_call c recv 4 VCL_return c pass 4 VCL_call c pass 4 VCL_return c pass 4 FetchError c no backend connection 4 VCL_call c error 4 VCL_return c restart 4 VCL_call c recv 4 VCL_return c pass 4 VCL_call c pass 4 VCL_return c pass 4 FetchError c no backend connection 4 VCL_call c error 4 VCL_return c restart 4 VCL_call c recv 4 VCL_return c pass 4 VCL_call c pass 4 VCL_return c pass 4 FetchError c no backend connection 4 VCL_call c error 4 VCL_return c restart 4 VCL_call c recv 4 VCL_return c pass 4 VCL_call c error 4 VCL_return c deliver 4 Length c 488 4 VCL_call c deliver 4 VCL_return c deliver 4 TxProtocol c HTTP/1.1 4 TxStatus c 503 4 TxResponse c Service Unavailable 4 TxHeader c Server: Varnish 4 TxHeader c Retry-After: 0 4 TxHeader c Content-Type: text/html; charset=utf-8 4 TxHeader c Content-Length: 488 4 TxHeader c Date: Tue, 08 Dec 2009 21:37:50 GMT 4 TxHeader c X-Varnish: 1690688575 4 TxHeader c Age: 0 4 TxHeader c Via: 1.1 varnish 4 TxHeader c Connection: close 4 TxHeader c X-Served-By: ecnext39 4 TxHeader c X-Cache: MISS 4 ReqEnd c 1690688575 1260308270.019510984 1260308270.020113707 0.004284143 0.000570059 0.000032663 4 SessionClose c error 4 StatSess c 66.104.100.163 61534 0 1 1 0 6 0 273 488 From sten at blinkenlights.nl Thu Dec 10 08:13:01 2009 From: sten at blinkenlights.nl (Sten Spans) Date: Thu, 10 Dec 2009 09:13:01 +0100 (CET) Subject: only 503s after a few hours (trunk-4384) In-Reply-To: <002b01ca78d6$a7de2b10$7802a8c0@interactive.live.net> References: <001a01ca7828$246bf400$7802a8c0@interactive.live.net> <002b01ca78d6$a7de2b10$7802a8c0@interactive.live.net> Message-ID: On Wed, 9 Dec 2009, Fernando Papa wrote: > Varnish account does not have limits (ulimit returns unlimited). What does "ulimit -n" return? Most newer linux distro's set this to 1024 by default, which might not be enough for a busy varnish. -- Sten Spans "There is a crack in everything, that's how the light gets in." Leonard Cohen - Anthem From fernando.papa at liveinteractive.net Thu Dec 10 13:53:38 2009 From: fernando.papa at liveinteractive.net (Fernando Papa) Date: Thu, 10 Dec 2009 10:53:38 -0300 Subject: only 503s after a few hours (trunk-4384) References: <6687.1260367633@critter.freebsd.dk> Message-ID: <002501ca79a0$2dfe4fd0$7802a8c0@interactive.live.net> Hello, >>5 Debug - "Too many open files when accept(2)ing. Sleeping." >> > > That's your trouble: The kernel does not allow varnish to open > as many filedescriptors as it needs. > > Use whatever means your kernel has to change this number, possibly > ulimit(1) or similar. That certainly improves uptime. I believe it is leaking descriptors. It can't possible use so many with the kind of traffic (~7 req/s) it's receiving. I restarted it yesterday with 128K descriptors. # ps -AF | grep varn root 4726 1 0 21419 928 2 Dec09 ? 00:00:00 varnishd -a *:80 -f frontdoor.vcl -p obj_workspace 4096 -s malloc,100M -p thread_pools 4 nobody 4727 4726 0 140497 153232 3 Dec09 ? 00:06:48 varnishd -a *:80 -f frontdoor.vcl -p obj_workspace 4096 -s malloc,100M -p thread_pools 4 # ls -l /proc/4727/fd | wc -l 1253 It's currently using 1253 descriptors. # ls -lat /proc/4727/fd | grep "socket" | head -n 1 lrwx------ 1 nobody nobody 64 Dec 10 11:40 1244 -> socket:[17728046] # ls -lat /proc/4727/fd | grep "socket" | tail -n 5 lrwx------ 1 nobody nobody 64 Dec 9 15:27 23 -> socket:[16402456] lrwx------ 1 nobody nobody 64 Dec 9 15:27 24 -> socket:[16261656] lrwx------ 1 nobody nobody 64 Dec 9 15:27 25 -> socket:[16411051] lrwx------ 1 nobody nobody 64 Dec 9 15:27 4 -> socket:[16216281] lrwx------ 1 nobody nobody 64 Dec 9 15:27 5 -> socket:[16216282] # ls -lat /proc/4727/fd | grep "15:" | wc -l 76 It has 76 sockets that are 20 hours old. Listening and monitoring sockets might explain a few of these. [[ NOTE: Re-checked after an hour and it still has 76 ]] # lsof -l -R -V -Tqs | grep 16216282 varnishd 4727 4726 99 5u IPv4 16216282 TCP *:http (LISTEN QR=0 QS=0) OK. 16216281 also listens, but is IPv6. # lsof -l -R -V -Tqs | grep 16411051 varnishd 4727 4726 99 25u sock 0,5 16411051 can't identify protocol # grep 16411051 /proc/net/* # netstat --verbose --numeric --extend --all --notrim | grep 16411051 From plfgoa at gmail.com Thu Dec 17 12:17:06 2009 From: plfgoa at gmail.com (Paras Fadte) Date: Thu, 17 Dec 2009 17:47:06 +0530 Subject: Varnish 503 Service unavailable error In-Reply-To: <87fx80xqop.fsf@qurzaw.linpro.no> References: <75cf5800908130119o5b10f41bn85e7d76fcb7ebc96@mail.gmail.com> <87fxbw5atk.fsf@qurzaw.linpro.no> <75cf5800908130243k3c0e7db6i2d01788a42ab352e@mail.gmail.com> <87k4z61gvj.fsf@qurzaw.linpro.no> <75cf5800911110400h32542eek30946320ae108d8e@mail.gmail.com> <877htwdpex.fsf@qurzaw.linpro.no> <75cf5800911262046p6a5b286awf9b563065d5aaab9@mail.gmail.com> <87fx80xqop.fsf@qurzaw.linpro.no> Message-ID: <75cf5800912170417k19b6bccpb2c02fc640260834@mail.gmail.com> Hi, Sometimes I see following when using varnishlog *Interrupted b BackendOpen* What does it mean and under what condition(s) does it occur ? Thank you. -Paras On Fri, Nov 27, 2009 at 2:58 PM, Tollef Fog Heen wrote: > ]] Paras Fadte > > Hi, > > | FetchError showed " *no backend connection* " . Would playing > around > | with "backend probes" rectify this issue if backends are sometimes under > | load ? > | or would increasing the "connect_timeout" help ? > > ?no backend connection? means just that ? we failed to find a backend > connection in time, ran into the maximum number of connections or > similar. If the problem is connect_timeout, increasing that further > might help, yes. > > -- > Tollef Fog Heen > Redpill Linpro -- Changing the game! > t: +47 21 54 41 73 > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jhayter at manta.com Fri Dec 18 18:27:07 2009 From: jhayter at manta.com (Jim Hayter) Date: Fri, 18 Dec 2009 13:27:07 -0500 Subject: Connection issues on Solaris 10 (Accept failed: Bad file number) Message-ID: I've tried running both 2.0.5 and 2.0.6 on a Sun intel box running Solaris 10. Varnish works for a while and then starts generating log output like the examples below, reporting nothing but " Accept failed: Bad file number". I raised the process file descriptor limit, but that did not help. Is anyone using Varnish 2.0.x on a Sun intel box running Solaris 10? If so, do you have any suggestions? Jim Log excerpts: 254 TxHeader c X-Varnish: 1174982928 1174970257 254 TxHeader c Via: 1.1 varnish 254 TxHeader c Connection: keep-alive 254 TxHeader c X-Served-By: ecnext39 254 TxHeader c X-Cache: HIT 254 TxHeader c X-Cache-Hits: 146 254 TxHeader c age: 0 254 ReqEnd c 1174982928 1261082734.437414408 1261082734.437511683 0.8236 44638 0.000054836 0.000042439 21 Debug - "Accept failed: Bad file number" 21 Debug - "Accept failed: Bad file number" 21 Debug - "Accept failed: Bad file number" 21 Debug - "Accept failed: Bad file number" 21 Debug - "Accept failed: Bad file number" 21 Debug - "Accept failed: Bad file number" 21 Debug - "Accept failed: Bad file number" And this: 250 SessionOpen c 66.104.xxx.xxx 63163 216.12.xxx.xxx:3008 254 SessionOpen c 66.104.xxx.xxx 59936 216.12.xxx.xxx:3008 255 SessionOpen c 66.104.xxx.xxx 56010 216.12.xxx.xxx:3008 21 Debug - "Accept failed: Bad file number" 21 Debug - "Accept failed: Bad file number" 21 Debug - "Accept failed: Bad file number" And this: 252 TxHeader c X-Varnish: 815722009 815708939 252 TxHeader c Via: 1.1 varnish 252 TxHeader c Connection: keep-alive 252 TxHeader c X-Served-By: ecnext39 252 TxHeader c X-Cache: HIT 252 TxHeader c X-Cache-Hits: 134 252 TxHeader c age: 0 252 ReqEnd c 815722009 1261155693.774985075 1261155693.775098562 0.13373 6610 0.000053406 0.000060081 21 Debug - "Accept failed: Bad file number" 21 Debug - "Accept failed: Bad file number" 21 Debug - "Accept failed: Bad file number" 21 Debug - "Accept failed: Bad file number" 21 Debug - "Accept failed: Bad file number" From joe at joetify.com Tue Dec 22 17:12:24 2009 From: joe at joetify.com (Joe Williams) Date: Tue, 22 Dec 2009 09:12:24 -0800 Subject: Slow connections Message-ID: <4B30FDF8.9050806@joetify.com> I am seeing a good amount (1/100) of connections to varnish (from haproxy) taking 3 seconds. My first thought was the connection backlog but somaxconn and listen_depth are both set higher than the number of connections. Anyone have any suggestions on how to track down what is causing this or settings I can use to try to aleviate it? Thanks. -Joe -- Name: Joseph A. Williams Email: joe at joetify.com Blog: http://www.joeandmotorboat.com/ From h.paulissen at qbell.nl Tue Dec 22 22:58:24 2009 From: h.paulissen at qbell.nl (Henry Paulissen) Date: Tue, 22 Dec 2009 23:58:24 +0100 Subject: Slow connections In-Reply-To: <4B30FDF8.9050806@joetify.com> References: <4B30FDF8.9050806@joetify.com> Message-ID: <005601ca835a$44499a10$ccdcce30$@paulissen@qbell.nl> Have a look to the conntrack setting in the kernel (sysctl) on both sides. It could be that your conntrack is full (connectrack only exists if you use iptables with netfilter_conntrack). Regards, Henry -----Oorspronkelijk bericht----- Van: varnish-misc-bounces at projects.linpro.no [mailto:varnish-misc-bounces at projects.linpro.no] Namens Joe Williams Verzonden: dinsdag 22 december 2009 18:12 Aan: varnish-misc at projects.linpro.no Onderwerp: Slow connections I am seeing a good amount (1/100) of connections to varnish (from haproxy) taking 3 seconds. My first thought was the connection backlog but somaxconn and listen_depth are both set higher than the number of connections. Anyone have any suggestions on how to track down what is causing this or settings I can use to try to aleviate it? Thanks. -Joe -- Name: Joseph A. Williams Email: joe at joetify.com Blog: http://www.joeandmotorboat.com/ _______________________________________________ varnish-misc mailing list varnish-misc at projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc From joe at joetify.com Tue Dec 22 23:22:32 2009 From: joe at joetify.com (Joe Williams) Date: Tue, 22 Dec 2009 15:22:32 -0800 Subject: Slow connections In-Reply-To: <005601ca835a$44499a10$ccdcce30$@paulissen@qbell.nl> References: <4B30FDF8.9050806@joetify.com> <005601ca835a$44499a10$ccdcce30$@paulissen@qbell.nl> Message-ID: <4B3154B8.8000107@joetify.com> Thanks Henry, nf_conntrack_max is set high on both machines. I've had the full table issue before :P On 12/22/09 2:58 PM, Henry Paulissen wrote: > Have a look to the conntrack setting in the kernel (sysctl) on both sides. > It could be that your conntrack is full (connectrack only exists if you use > iptables with netfilter_conntrack). > > Regards, > Henry > > -----Oorspronkelijk bericht----- > Van: varnish-misc-bounces at projects.linpro.no > [mailto:varnish-misc-bounces at projects.linpro.no] Namens Joe Williams > Verzonden: dinsdag 22 december 2009 18:12 > Aan: varnish-misc at projects.linpro.no > Onderwerp: Slow connections > > > I am seeing a good amount (1/100) of connections to varnish (from > haproxy) taking 3 seconds. My first thought was the connection backlog > but somaxconn and listen_depth are both set higher than the number of > connections. Anyone have any suggestions on how to track down what is > causing this or settings I can use to try to aleviate it? > > Thanks. > > -Joe > > -- Name: Joseph A. Williams Email: joe at joetify.com Blog: http://www.joeandmotorboat.com/ From michael at dynamine.net Tue Dec 22 23:24:15 2009 From: michael at dynamine.net (Michael Fischer) Date: Tue, 22 Dec 2009 15:24:15 -0800 Subject: Slow connections In-Reply-To: <-1921896957570796580@unknownmsgid> References: <4B30FDF8.9050806@joetify.com> <-1921896957570796580@unknownmsgid> Message-ID: Are you seeing any increased TCP send queue lengths on the haproxy side, or TCP receive queue lengths on the Varnish side? (netstat -a) That might provide some clue as to what's going on. --Michael On Tue, Dec 22, 2009 at 2:58 PM, Henry Paulissen wrote: > Have a look to the conntrack setting in the kernel (sysctl) on both sides. > It could be that your conntrack is full (connectrack only exists if you use > iptables with netfilter_conntrack). > > Regards, > Henry > > -----Oorspronkelijk bericht----- > Van: varnish-misc-bounces at projects.linpro.no > [mailto:varnish-misc-bounces at projects.linpro.no] Namens Joe Williams > Verzonden: dinsdag 22 december 2009 18:12 > Aan: varnish-misc at projects.linpro.no > Onderwerp: Slow connections > > > I am seeing a good amount (1/100) of connections to varnish (from > haproxy) taking 3 seconds. My first thought was the connection backlog > but somaxconn and listen_depth are both set higher than the number of > connections. Anyone have any suggestions on how to track down what is > causing this or settings I can use to try to aleviate it? > > Thanks. > > -Joe > > -- > Name: Joseph A. Williams > Email: joe at joetify.com > Blog: http://www.joeandmotorboat.com/ > > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc > > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From h.paulissen at qbell.nl Tue Dec 22 23:41:27 2009 From: h.paulissen at qbell.nl (Henry Paulissen) Date: Wed, 23 Dec 2009 00:41:27 +0100 Subject: Slow connections In-Reply-To: <4B3154B8.8000107@joetify.com> References: <4B30FDF8.9050806@joetify.com> <005601ca835a$44499a10$ccdcce30$@paulissen@qbell.nl> <4B3154B8.8000107@joetify.com> Message-ID: <005a01ca8360$47d91ab0$d78b5010$@paulissen@qbell.nl> Next one. Did you tune the tcp fin timeout? (on both servers) Linux will standard holds all connection open till it hits the fin timeout length (tcp_fin and tcp_fin2). We decreased it to 3. HAProxy support: Do you forced a http connection close in haproxy? If all connections are in keep-alive your queue will fill up real quick. Henry -----Oorspronkelijk bericht----- Van: Joe Williams [mailto:joe at joetify.com] Verzonden: woensdag 23 december 2009 0:23 Aan: Henry Paulissen CC: varnish-misc at projects.linpro.no Onderwerp: Re: Slow connections Thanks Henry, nf_conntrack_max is set high on both machines. I've had the full table issue before :P On 12/22/09 2:58 PM, Henry Paulissen wrote: > Have a look to the conntrack setting in the kernel (sysctl) on both sides. > It could be that your conntrack is full (connectrack only exists if you use > iptables with netfilter_conntrack). > > Regards, > Henry > > -----Oorspronkelijk bericht----- > Van: varnish-misc-bounces at projects.linpro.no > [mailto:varnish-misc-bounces at projects.linpro.no] Namens Joe Williams > Verzonden: dinsdag 22 december 2009 18:12 > Aan: varnish-misc at projects.linpro.no > Onderwerp: Slow connections > > > I am seeing a good amount (1/100) of connections to varnish (from > haproxy) taking 3 seconds. My first thought was the connection backlog > but somaxconn and listen_depth are both set higher than the number of > connections. Anyone have any suggestions on how to track down what is > causing this or settings I can use to try to aleviate it? > > Thanks. > > -Joe > > -- Name: Joseph A. Williams Email: joe at joetify.com Blog: http://www.joeandmotorboat.com/ From michael at dynamine.net Wed Dec 23 00:12:01 2009 From: michael at dynamine.net (Michael Fischer) Date: Tue, 22 Dec 2009 16:12:01 -0800 Subject: Slow connections In-Reply-To: <4867769412916894740@unknownmsgid> References: <4B30FDF8.9050806@joetify.com> <4B3154B8.8000107@joetify.com> <4867769412916894740@unknownmsgid> Message-ID: haproxy has never supported keep-alive HTTP connections, to my knowledge. --Michael On Tue, Dec 22, 2009 at 3:41 PM, Henry Paulissen wrote: > Next one. > > Did you tune the tcp fin timeout? (on both servers) > Linux will standard holds all connection open till it hits the fin timeout > length (tcp_fin and tcp_fin2). > We decreased it to 3. > > HAProxy support: > Do you forced a http connection close in haproxy? > If all connections are in keep-alive your queue will fill up real quick. > > Henry > > -----Oorspronkelijk bericht----- > Van: Joe Williams [mailto:joe at joetify.com] > Verzonden: woensdag 23 december 2009 0:23 > Aan: Henry Paulissen > CC: varnish-misc at projects.linpro.no > Onderwerp: Re: Slow connections > > > Thanks Henry, nf_conntrack_max is set high on both machines. I've had > the full table issue before :P > > > > On 12/22/09 2:58 PM, Henry Paulissen wrote: > > Have a look to the conntrack setting in the kernel (sysctl) on both > sides. > > It could be that your conntrack is full (connectrack only exists if you > use > > iptables with netfilter_conntrack). > > > > Regards, > > Henry > > > > -----Oorspronkelijk bericht----- > > Van: varnish-misc-bounces at projects.linpro.no > > [mailto:varnish-misc-bounces at projects.linpro.no] Namens Joe Williams > > Verzonden: dinsdag 22 december 2009 18:12 > > Aan: varnish-misc at projects.linpro.no > > Onderwerp: Slow connections > > > > > > I am seeing a good amount (1/100) of connections to varnish (from > > haproxy) taking 3 seconds. My first thought was the connection backlog > > but somaxconn and listen_depth are both set higher than the number of > > connections. Anyone have any suggestions on how to track down what is > > causing this or settings I can use to try to aleviate it? > > > > Thanks. > > > > -Joe > > > > > > -- > Name: Joseph A. Williams > Email: joe at joetify.com > Blog: http://www.joeandmotorboat.com/ > > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From joe at joetify.com Wed Dec 23 00:28:23 2009 From: joe at joetify.com (Joe Williams) Date: Tue, 22 Dec 2009 16:28:23 -0800 Subject: Slow connections In-Reply-To: <005a01ca8360$47d91ab0$d78b5010$@paulissen@qbell.nl> References: <4B30FDF8.9050806@joetify.com> <005601ca835a$44499a10$ccdcce30$@paulissen@qbell.nl> <4B3154B8.8000107@joetify.com> <005a01ca8360$47d91ab0$d78b5010$@paulissen@qbell.nl> Message-ID: <4B316427.8050005@joetify.com> 1) Yes, net.ipv4.tcp_fin_timeout = 3 2) Sometimes, for some backends we do not use httpclose for others we do. -Joe On 12/22/09 3:41 PM, Henry Paulissen wrote: > Next one. > > Did you tune the tcp fin timeout? (on both servers) > Linux will standard holds all connection open till it hits the fin timeout > length (tcp_fin and tcp_fin2). > We decreased it to 3. > > HAProxy support: > Do you forced a http connection close in haproxy? > If all connections are in keep-alive your queue will fill up real quick. > > Henry > > -----Oorspronkelijk bericht----- > Van: Joe Williams [mailto:joe at joetify.com] > Verzonden: woensdag 23 december 2009 0:23 > Aan: Henry Paulissen > CC: varnish-misc at projects.linpro.no > Onderwerp: Re: Slow connections > > > Thanks Henry, nf_conntrack_max is set high on both machines. I've had > the full table issue before :P > > > > On 12/22/09 2:58 PM, Henry Paulissen wrote: > >> Have a look to the conntrack setting in the kernel (sysctl) on both sides. >> It could be that your conntrack is full (connectrack only exists if you >> > use > >> iptables with netfilter_conntrack). >> >> Regards, >> Henry >> >> -----Oorspronkelijk bericht----- >> Van: varnish-misc-bounces at projects.linpro.no >> [mailto:varnish-misc-bounces at projects.linpro.no] Namens Joe Williams >> Verzonden: dinsdag 22 december 2009 18:12 >> Aan: varnish-misc at projects.linpro.no >> Onderwerp: Slow connections >> >> >> I am seeing a good amount (1/100) of connections to varnish (from >> haproxy) taking 3 seconds. My first thought was the connection backlog >> but somaxconn and listen_depth are both set higher than the number of >> connections. Anyone have any suggestions on how to track down what is >> causing this or settings I can use to try to aleviate it? >> >> Thanks. >> >> -Joe >> >> >> > -- Name: Joseph A. Williams Email: joe at joetify.com Blog: http://www.joeandmotorboat.com/ From h.paulissen at qbell.nl Wed Dec 23 00:29:22 2009 From: h.paulissen at qbell.nl (Henry Paulissen) Date: Wed, 23 Dec 2009 01:29:22 +0100 Subject: Slow connections In-Reply-To: References: <4B30FDF8.9050806@joetify.com> <4B3154B8.8000107@joetify.com> <4867769412916894740@unknownmsgid> Message-ID: <005e01ca8366$f952b390$ebf81ab0$@paulissen@qbell.nl> True and false. When you dont define to close the connection it will do keep-alives. The problem with this is that only the first header of the stream will be checked against the acl's. If you use haproxy only to load balance between http servers and not doing routing based on url's (e.g. send \.(gif|jpg|png|jpeg) to static server and all else to processing cluster), you may use keep-alives. Henry Van: Michael Fischer [mailto:michael at dynamine.net] Verzonden: woensdag 23 december 2009 1:12 Aan: Henry Paulissen CC: Joe Williams; varnish-misc at projects.linpro.no Onderwerp: Re: Slow connections haproxy has never supported keep-alive HTTP connections, to my knowledge. --Michael On Tue, Dec 22, 2009 at 3:41 PM, Henry Paulissen wrote: Next one. Did you tune the tcp fin timeout? (on both servers) Linux will standard holds all connection open till it hits the fin timeout length (tcp_fin and tcp_fin2). We decreased it to 3. HAProxy support: Do you forced a http connection close in haproxy? If all connections are in keep-alive your queue will fill up real quick. Henry -----Oorspronkelijk bericht----- Van: Joe Williams [mailto:joe at joetify.com] Verzonden: woensdag 23 december 2009 0:23 Aan: Henry Paulissen CC: varnish-misc at projects.linpro.no Onderwerp: Re: Slow connections Thanks Henry, nf_conntrack_max is set high on both machines. I've had the full table issue before :P On 12/22/09 2:58 PM, Henry Paulissen wrote: > Have a look to the conntrack setting in the kernel (sysctl) on both sides. > It could be that your conntrack is full (connectrack only exists if you use > iptables with netfilter_conntrack). > > Regards, > Henry > > -----Oorspronkelijk bericht----- > Van: varnish-misc-bounces at projects.linpro.no > [mailto:varnish-misc-bounces at projects.linpro.no] Namens Joe Williams > Verzonden: dinsdag 22 december 2009 18:12 > Aan: varnish-misc at projects.linpro.no > Onderwerp: Slow connections > > > I am seeing a good amount (1/100) of connections to varnish (from > haproxy) taking 3 seconds. My first thought was the connection backlog > but somaxconn and listen_depth are both set higher than the number of > connections. Anyone have any suggestions on how to track down what is > causing this or settings I can use to try to aleviate it? > > Thanks. > > -Joe > > -- Name: Joseph A. Williams Email: joe at joetify.com Blog: http://www.joeandmotorboat.com/ _______________________________________________ varnish-misc mailing list varnish-misc at projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc -------------- next part -------------- An HTML attachment was scrubbed... URL: From joe at joetify.com Wed Dec 23 00:45:11 2009 From: joe at joetify.com (Joe Williams) Date: Tue, 22 Dec 2009 16:45:11 -0800 Subject: Slow connections In-Reply-To: References: <4B30FDF8.9050806@joetify.com> <-1921896957570796580@unknownmsgid> Message-ID: <4B316817.1010902@joetify.com> Michael, I've looked at it for a bit and see 0's across the board for the connections between the two machines. To that end it looks like I rarely have above 500 connections into varnish. -Joe On 12/22/09 3:24 PM, Michael Fischer wrote: > Are you seeing any increased TCP send queue lengths on the haproxy > side, or TCP receive queue lengths on the Varnish side? (netstat -a) > That might provide some clue as to what's going on. > > --Michael > > On Tue, Dec 22, 2009 at 2:58 PM, Henry Paulissen > wrote: > > Have a look to the conntrack setting in the kernel (sysctl) on > both sides. > It could be that your conntrack is full (connectrack only exists > if you use > iptables with netfilter_conntrack). > > Regards, > Henry > > -----Oorspronkelijk bericht----- > Van: varnish-misc-bounces at projects.linpro.no > > [mailto:varnish-misc-bounces at projects.linpro.no > ] Namens Joe Williams > Verzonden: dinsdag 22 december 2009 18:12 > Aan: varnish-misc at projects.linpro.no > > Onderwerp: Slow connections > > > I am seeing a good amount (1/100) of connections to varnish (from > haproxy) taking 3 seconds. My first thought was the connection backlog > but somaxconn and listen_depth are both set higher than the number of > connections. Anyone have any suggestions on how to track down what is > causing this or settings I can use to try to aleviate it? > > Thanks. > > -Joe > > -- > Name: Joseph A. Williams > Email: joe at joetify.com > Blog: http://www.joeandmotorboat.com/ > > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > > http://projects.linpro.no/mailman/listinfo/varnish-misc > > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > > http://projects.linpro.no/mailman/listinfo/varnish-misc > > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc > -- Name: Joseph A. Williams Email: joe at joetify.com Blog: http://www.joeandmotorboat.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From coolbomb at gmail.com Wed Dec 23 10:50:02 2009 From: coolbomb at gmail.com (Daniel Rodriguez) Date: Wed, 23 Dec 2009 11:50:02 +0100 Subject: Unconditional GETs and 304 from backend servers Message-ID: <95ff419c0912230250j401d314dwe33def2826e9d1b3@mail.gmail.com> Hi everyone, We are trying to solve a problem with our varnish implementation. The code of our web application, uses a "logic" to generate 304 messages to the requests made to some objects that requires some heavy processing on the server side. For example, some pages codes will check if some XML file or database information have changed since the last request made from the client (Varnish) and return a 304 if its not necessary to recreate the objects, and save us some process load on our poor web servers. The problem that we are currently having is that varnish will only ask unconditional GETs, so every time that varnish ask the web server for one of this objects after the age have expired (expecting a 200 and a full refresh) our servers have to recreate the whole objects (some of them taking a lot of CPU and time) even if this is not needed (in normal conditions with conditional GET they will reply 304 with less CPU usage). There is a possible solution to this ? maybe a workaround using VCL ? Thank you all, Best Regards, From plfgoa at gmail.com Wed Dec 23 10:59:05 2009 From: plfgoa at gmail.com (Paras Fadte) Date: Wed, 23 Dec 2009 16:29:05 +0530 Subject: Varnish 503 Service unavailable error In-Reply-To: <75cf5800912170417k19b6bccpb2c02fc640260834@mail.gmail.com> References: <75cf5800908130119o5b10f41bn85e7d76fcb7ebc96@mail.gmail.com> <87fxbw5atk.fsf@qurzaw.linpro.no> <75cf5800908130243k3c0e7db6i2d01788a42ab352e@mail.gmail.com> <87k4z61gvj.fsf@qurzaw.linpro.no> <75cf5800911110400h32542eek30946320ae108d8e@mail.gmail.com> <877htwdpex.fsf@qurzaw.linpro.no> <75cf5800911262046p6a5b286awf9b563065d5aaab9@mail.gmail.com> <87fx80xqop.fsf@qurzaw.linpro.no> <75cf5800912170417k19b6bccpb2c02fc640260834@mail.gmail.com> Message-ID: <75cf5800912230259gd0f8052paf4f9a3bf772670@mail.gmail.com> Can anybody respond to this ? On Thu, Dec 17, 2009 at 5:47 PM, Paras Fadte wrote: > Hi, > > Sometimes I see following when using varnishlog > > *Interrupted b BackendOpen* > > What does it mean and under what condition(s) does it occur ? > > Thank you. > > -Paras > > > On Fri, Nov 27, 2009 at 2:58 PM, Tollef Fog Heen < > tfheen at redpill-linpro.com> wrote: > >> ]] Paras Fadte >> >> Hi, >> >> | FetchError showed " *no backend connection* " . Would playing >> around >> | with "backend probes" rectify this issue if backends are sometimes under >> | load ? >> | or would increasing the "connect_timeout" help ? >> >> ?no backend connection? means just that ? we failed to find a backend >> connection in time, ran into the maximum number of connections or >> similar. If the problem is connect_timeout, increasing that further >> might help, yes. >> >> -- >> Tollef Fog Heen >> Redpill Linpro -- Changing the game! >> t: +47 21 54 41 73 >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at projects.linpro.no >> http://projects.linpro.no/mailman/listinfo/varnish-misc >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gaute at pht.no Mon Dec 28 00:21:01 2009 From: gaute at pht.no (Gaute Amundsen) Date: Mon, 28 Dec 2009 01:21:01 +0100 Subject: can backend trigger pipe? Message-ID: <1261959661.7567.1152.camel@totiki> Say I have site where an admin page takes a few minutes to complete, and it outputs text to notify about progress and keep the connection from timing out. Lets pretend I only have this one IP and port. Is there any other way to handle this than to hardcode "pipe" for those urls in vcl_recv? Any way for the script to return a header that can tell vcl_fetch to pipe? Or is that too late? Am I right to assume that "pass" is not intended to work in a case like this? running varnish-2.0.3 Regards Gaute Amundsen From david.birdsong at gmail.com Mon Dec 28 00:56:05 2009 From: david.birdsong at gmail.com (David Birdsong) Date: Sun, 27 Dec 2009 16:56:05 -0800 Subject: can backend trigger pipe? In-Reply-To: <1261959661.7567.1152.camel@totiki> References: <1261959661.7567.1152.camel@totiki> Message-ID: On Sun, Dec 27, 2009 at 4:21 PM, Gaute Amundsen wrote: > > Say I have site where an admin page takes a few minutes to complete, and > it outputs text to notify about progress and keep the connection from > timing out. > > Lets pretend I only have this one IP and port. > > Is there any other way to handle this than to hardcode "pipe" for those > urls in vcl_recv? > > Any way for the script to return a header that can tell vcl_fetch to > pipe? Or is that too late? > > Am I right to assume that "pass" is not intended to work in a case like > this? > I'm keen to hear a response on this too. I've been meaning to test out some layering where something behind varnish determines a high enough 'hit' count and sets a header indicating so. Without this header set varnish would simply pipe all responses in vcl_fetch. I've always assumed this was a valid thing to do in vcl_fetch, but I haven't gotten around to testing it. > running varnish-2.0.3 > > Regards > Gaute Amundsen > > > > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc > From joe at joetify.com Mon Dec 28 18:59:02 2009 From: joe at joetify.com (Joe Williams) Date: Mon, 28 Dec 2009 10:59:02 -0800 Subject: Slow connections In-Reply-To: <4B316817.1010902@joetify.com> References: <4B30FDF8.9050806@joetify.com> <-1921896957570796580@unknownmsgid> <4B316817.1010902@joetify.com> Message-ID: <4B38FFF6.1020001@joetify.com> Any other suggestions? Another analysis of the logs shows that varnish vs other backends (CouchDB) I see an order of magnitude higher percentage of 3 second connection times with varnish. Here's my varnish command options http://friendpaste.com/1Zvj1XkX75eKqdzrJQZcfK and my sysctl changes http://friendpaste.com/3lt3uhIm1NTNhrqGxvQEX6 the sysctl on haproxy is identical. If it makes a difference my VCL is pretty simple just with some header rewrites, round robin for 8 backend servers, exclusions for certain URI's and etc. Thanks again! -Joe On 12/22/09 4:45 PM, Joe Williams wrote: > Michael, > > I've looked at it for a bit and see 0's across the board for the > connections between the two machines. > > To that end it looks like I rarely have above 500 connections into > varnish. > > -Joe > > > On 12/22/09 3:24 PM, Michael Fischer wrote: >> Are you seeing any increased TCP send queue lengths on the haproxy >> side, or TCP receive queue lengths on the Varnish side? (netstat -a) >> That might provide some clue as to what's going on. >> >> --Michael >> >> On Tue, Dec 22, 2009 at 2:58 PM, Henry Paulissen >> > wrote: >> >> Have a look to the conntrack setting in the kernel (sysctl) on >> both sides. >> It could be that your conntrack is full (connectrack only exists >> if you use >> iptables with netfilter_conntrack). >> >> Regards, >> Henry >> >> -----Oorspronkelijk bericht----- >> Van: varnish-misc-bounces at projects.linpro.no >> >> [mailto:varnish-misc-bounces at projects.linpro.no >> ] Namens Joe Williams >> Verzonden: dinsdag 22 december 2009 18:12 >> Aan: varnish-misc at projects.linpro.no >> >> Onderwerp: Slow connections >> >> >> I am seeing a good amount (1/100) of connections to varnish (from >> haproxy) taking 3 seconds. My first thought was the connection >> backlog >> but somaxconn and listen_depth are both set higher than the number of >> connections. Anyone have any suggestions on how to track down what is >> causing this or settings I can use to try to aleviate it? >> >> Thanks. >> >> -Joe >> >> -- >> Name: Joseph A. Williams >> Email: joe at joetify.com >> Blog: http://www.joeandmotorboat.com/ >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at projects.linpro.no >> >> http://projects.linpro.no/mailman/listinfo/varnish-misc >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at projects.linpro.no >> >> http://projects.linpro.no/mailman/listinfo/varnish-misc >> >> >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at projects.linpro.no >> http://projects.linpro.no/mailman/listinfo/varnish-misc >> > > -- > Name: Joseph A. Williams > Email:joe at joetify.com > Blog:http://www.joeandmotorboat.com/ > > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc > -- Name: Joseph A. Williams Email: joe at joetify.com Blog: http://www.joeandmotorboat.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From cosimo at streppone.it Mon Dec 28 20:45:10 2009 From: cosimo at streppone.it (Cosimo Streppone) Date: Mon, 28 Dec 2009 21:45:10 +0100 Subject: Slow connections In-Reply-To: <4B38FFF6.1020001@joetify.com> References: <4B30FDF8.9050806@joetify.com> <-1921896957570796580@unknownmsgid> <4B316817.1010902@joetify.com> <4B38FFF6.1020001@joetify.com> Message-ID: On 28 Dec 2009 19:59:02, Joe Williams wrote: > Any other suggestions? Another analysis of the logs shows that varnish > vs other backends (CouchDB) I see an order of magnitude higher > percentage of 3 second connection times with varnish. > Here's my varnish command options > http://friendpaste.com/1Zvj1XkX75eKqdzrJQZcfK and my sysctl changes > http://friendpaste.com/3lt3uhIm1NTNhrqGxvQEX6 the sysctl on haproxy is > identical. Hi Joe, I didn't follow all the discussion, but a quick look made me think about my case. I practically did the same steps, with the same results as yours. Kernel tuning, somaxconn, listen_depth, etc... In my case I was experiencing dropped connections, no or very delayed syn-ack packets from varnish server, random varnish restarts (crashes?), and sudden system load spikes, even as high as 200/300. The solution to this problem was really simple. Switch from the "file" allocation to "malloc". In the config file I had: -s file,/var/lib/varnish/varnish.cache,20G and I changed it to: -s malloc,20G I see you have 400G, so this solution won't probably work for you?, but my suggestion would be to at least try it with as much RAM as you have, to see if the issue disappears. Then you can setup some swap partition maybe. Check this mail, http://www.mail-archive.com/varnish-misc at projects.linpro.no/msg01571.html -- Cosimo From joe at joetify.com Mon Dec 28 20:55:08 2009 From: joe at joetify.com (Joe Williams) Date: Mon, 28 Dec 2009 12:55:08 -0800 Subject: Slow connections In-Reply-To: References: <4B30FDF8.9050806@joetify.com> <-1921896957570796580@unknownmsgid> <4B316817.1010902@joetify.com> <4B38FFF6.1020001@joetify.com> Message-ID: <4B391B2C.2070108@joetify.com> Thanks Cosimo, I'll have to give that a shot. I use 400GB because I have a dedicated EC2 machine for varnish and figured I would use as much space as I have for it, looks like I'm actually only using around 8GB. -Joe On 12/28/09 12:45 PM, Cosimo Streppone wrote: > On 28 Dec 2009 19:59:02, Joe Williams wrote: > >> Any other suggestions? Another analysis of the logs shows that varnish >> vs other backends (CouchDB) I see an order of magnitude higher >> percentage of 3 second connection times with varnish. > >> Here's my varnish command options >> http://friendpaste.com/1Zvj1XkX75eKqdzrJQZcfK and my sysctl changes >> http://friendpaste.com/3lt3uhIm1NTNhrqGxvQEX6 the sysctl on haproxy is >> identical. > > Hi Joe, > > I didn't follow all the discussion, but a quick look made me > think about my case. > > I practically did the same steps, with the same results > as yours. Kernel tuning, somaxconn, listen_depth, etc... > > In my case I was experiencing dropped connections, no > or very delayed syn-ack packets from varnish server, random varnish > restarts (crashes?), and sudden system load spikes, > even as high as 200/300. > > The solution to this problem was really simple. > Switch from the "file" allocation to "malloc". > In the config file I had: > > -s file,/var/lib/varnish/varnish.cache,20G > > and I changed it to: > > -s malloc,20G > > I see you have 400G, so this solution won't probably work > for you?, but my suggestion would be to at least try it > with as much RAM as you have, to see if the issue disappears. > > Then you can setup some swap partition maybe. > Check this mail, > > http://www.mail-archive.com/varnish-misc at projects.linpro.no/msg01571.html > -- Name: Joseph A. Williams Email: joe at joetify.com Blog: http://www.joeandmotorboat.com/ From michael at dynamine.net Tue Dec 29 06:43:07 2009 From: michael at dynamine.net (Michael S. Fischer) Date: Mon, 28 Dec 2009 22:43:07 -0800 Subject: Slow connections In-Reply-To: <4B391B2C.2070108@joetify.com> References: <4B30FDF8.9050806@joetify.com> <-1921896957570796580@unknownmsgid> <4B316817.1010902@joetify.com> <4B38FFF6.1020001@joetify.com> <4B391B2C.2070108@joetify.com> Message-ID: That kind of VM overcommit (400GB on an 8GB box) is hazardous for performance anyway. I strongly advise configuring Varnish cache sizes at slightly under the actual RAM size of the box. If your working set size is larger, you need more boxes or more RAM anyway, as paging I/O will significantly impede your throughput. Another thing that would be useful to know are the TCP stats from the haproxy, e.g., netstat -s -t (Linux). Unfortunately you can't determine retransmits on a per-destination basis using netstat alone, but there might be open source tools out there that can help. Finally, sometimes I've seen these issues where the network interface or the switch were having speed/duplex autonegotiation issues or bad network cables; manually configuring the switch (e.g. to 1Gb full duplex) sometimes solved the issue. --Michael On Dec 28, 2009, at 12:55 PM, Joe Williams wrote: > Thanks Cosimo, I'll have to give that a shot. I use 400GB because I have > a dedicated EC2 machine for varnish and figured I would use as much > space as I have for it, looks like I'm actually only using around 8GB. > > -Joe > > > On 12/28/09 12:45 PM, Cosimo Streppone wrote: >> On 28 Dec 2009 19:59:02, Joe Williams wrote: >> >>> Any other suggestions? Another analysis of the logs shows that varnish >>> vs other backends (CouchDB) I see an order of magnitude higher >>> percentage of 3 second connection times with varnish. >> >>> Here's my varnish command options >>> http://friendpaste.com/1Zvj1XkX75eKqdzrJQZcfK and my sysctl changes >>> http://friendpaste.com/3lt3uhIm1NTNhrqGxvQEX6 the sysctl on haproxy is >>> identical. >> >> Hi Joe, >> >> I didn't follow all the discussion, but a quick look made me >> think about my case. >> >> I practically did the same steps, with the same results >> as yours. Kernel tuning, somaxconn, listen_depth, etc... >> >> In my case I was experiencing dropped connections, no >> or very delayed syn-ack packets from varnish server, random varnish >> restarts (crashes?), and sudden system load spikes, >> even as high as 200/300. >> >> The solution to this problem was really simple. >> Switch from the "file" allocation to "malloc". >> In the config file I had: >> >> -s file,/var/lib/varnish/varnish.cache,20G >> >> and I changed it to: >> >> -s malloc,20G >> >> I see you have 400G, so this solution won't probably work >> for you?, but my suggestion would be to at least try it >> with as much RAM as you have, to see if the issue disappears. >> >> Then you can setup some swap partition maybe. >> Check this mail, >> >> http://www.mail-archive.com/varnish-misc at projects.linpro.no/msg01571.html >> > > -- > Name: Joseph A. Williams > Email: joe at joetify.com > Blog: http://www.joeandmotorboat.com/ > > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc From ibeginhere at gmail.com Thu Dec 31 07:47:15 2009 From: ibeginhere at gmail.com (ll) Date: Thu, 31 Dec 2009 15:47:15 +0800 Subject: varnishtest Message-ID: <4B3C5703.8020002@gmail.com> Is there any manual about the command varnishtest ? I useing varnish 2.0.4 edition . and I want to have a script to test the backend whether the backend server is normal .I know the varnish can test it ,and configure some thing to change the backend when it test is failed.so I want to test it and send a mail or some other log can be trace .