From armdan20 at gmail.com Fri Oct 3 07:49:55 2008 From: armdan20 at gmail.com (andan andan) Date: Fri, 3 Oct 2008 09:49:55 +0200 Subject: ESI and clients HTTP/1.0 Message-ID: Hi all. Is there any manner to avoid Transfer-Encoding: chunked in responses of ESI pages to HTTP/1.0 clients. According to RFC: 14.41 Transfer-Encoding ... Many older HTTP/1.0 applications do not understand the Transfer- Encoding header. We have several issues with Squid2.5.x proxys (not newer versions). Unfortunately, this branch is still very commonly in productions environments. Thank you. BR. From phk at phk.freebsd.dk Fri Oct 3 07:57:42 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Fri, 03 Oct 2008 07:57:42 +0000 Subject: ESI and clients HTTP/1.0 In-Reply-To: Your message of "Fri, 03 Oct 2008 09:49:55 +0200." Message-ID: <4048.1223020662@critter.freebsd.dk> In message , "anda n andan" writes: >Hi all. > >Is there any manner to avoid Transfer-Encoding: chunked in responses >of ESI pages to HTTP/1.0 clients. Not at present, sorry. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From agalvani at gmail.com Fri Oct 3 11:49:11 2008 From: agalvani at gmail.com (Andre Galvani) Date: Fri, 3 Oct 2008 08:49:11 -0300 Subject: Blades Message-ID: Hello, someone has had experience running varnish on blades(DELL HW for example)? tks and regards, Andre Galvani -------------- next part -------------- An HTML attachment was scrubbed... URL: From perbu at linpro.no Fri Oct 3 12:30:07 2008 From: perbu at linpro.no (Per Buer) Date: Fri, 03 Oct 2008 14:30:07 +0200 Subject: Blades In-Reply-To: References: Message-ID: <48E6104F.9030001@linpro.no> Andre Galvani skrev: > Hello, > > someone has had experience running varnish on blades(DELL HW for example)? Yes. We run a lot of Varnish on blades. Runs smoothly. :-) -- Per Buer - Leder Infrastruktur og Drift - Redpill Linpro Telefon: 21 54 41 21 - Mobil: 958 39 117 http://linpro.no/ | http://redpill.se/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 252 bytes Desc: OpenPGP digital signature URL: From morten at startsiden.no Fri Oct 3 12:33:00 2008 From: morten at startsiden.no (Morten Bekkelund) Date: Fri, 03 Oct 2008 14:33:00 +0200 Subject: Cookie issue Message-ID: <48E610FC.7030607@startsiden.no> Hey guys. Here's the situation: We got some 3rd party apps that runs via javascript on our webpages. These apps sets cookies and therefore they are not cached by varnish by default. I'm looking for a good approach to force a "lookup" when these cookies are set, but still also force a "pass" when other cookies are set OR even do a "pass" for the entire session when the req.url matches ^/admin ...for instance. I've looked at the examples on the wiki, but couldn't find a suitable config for myself. Regards, Mortis From tfheen at linpro.no Fri Oct 3 13:41:36 2008 From: tfheen at linpro.no (Tollef Fog Heen) Date: Fri, 03 Oct 2008 15:41:36 +0200 Subject: Cookie issue In-Reply-To: <48E610FC.7030607@startsiden.no> (Morten Bekkelund's message of "Fri, 03 Oct 2008 14:33:00 +0200") References: <48E610FC.7030607@startsiden.no> Message-ID: <87abdleqy7.fsf@qurzaw.linpro.no> ]] Morten Bekkelund | I'm looking for a good approach to force a "lookup" when these cookies | are set, but still also force a "pass" when other cookies are set OR | even do a "pass" for the entire session when the req.url matches | ^/admin ...for instance. You probably want to do regex matches against req.http.cookie to ensure you only cache for particular pages. I'd also recommend reading through http://varnish.projects.linpro.no/wiki/VCLExampleCacheCookies if you haven't. Doing pass on ^/admin should be something like if (req.url ~ "^/admin") { pass; } in vcl_recv, so I presume that's not the problem you are running into? -- Tollef Fog Heen Redpill Linpro -- Changing the game! t: +47 21 54 41 73 From morten at startsiden.no Fri Oct 3 13:44:50 2008 From: morten at startsiden.no (Morten Bekkelund) Date: Fri, 03 Oct 2008 15:44:50 +0200 Subject: Cookie issue In-Reply-To: <87abdleqy7.fsf@qurzaw.linpro.no> References: <48E610FC.7030607@startsiden.no> <87abdleqy7.fsf@qurzaw.linpro.no> Message-ID: <48E621D2.1050809@startsiden.no> Tollef Fog Heen wrote: > ]] Morten Bekkelund > > | I'm looking for a good approach to force a "lookup" when these cookies > | are set, but still also force a "pass" when other cookies are set OR > | even do a "pass" for the entire session when the req.url matches > | ^/admin ...for instance. > > You probably want to do regex matches against req.http.cookie to ensure > you only cache for particular pages. I'd also recommend reading through > http://varnish.projects.linpro.no/wiki/VCLExampleCacheCookies if you > haven't. > > Doing pass on ^/admin should be something like > > if (req.url ~ "^/admin") { > pass; > } > > in vcl_recv, so I presume that's not the problem you are running into? > > Well, I've tried that. That passes the single request to ^/admin, but does a lookup on every other request since the stuff under admin uses a lot of stuff in other paths. So it's a bit tricky :) From ender at tuenti.com Fri Oct 3 15:54:09 2008 From: ender at tuenti.com (David =?utf-8?q?Mart=C3=ADnez_Moreno?=) Date: Fri, 3 Oct 2008 17:54:09 +0200 Subject: 400 return code on Content-length: 0. Message-ID: <200810031754.09644.ender@tuenti.com> I'm seeing HTTP/1.1 400 Bad Request when querying Varnish with Content-length: 0 headers. The point is that the RFC doesn't explicitly states which method forbids bodies so I guess it?s all right to have a zero valued Content-length. Thoughts? Ender. -- - You were at 4,000 degrees Kelvin! You were approaching supernova! - Sweet! - No, not "sweet"! That's the heat of the sun! - You could kill yourself, other people, and burn up the atmosphere, ending all human life as we know it. - Got it. Supernova bad. -- Susan and Johnny Storm, and Reed Richards (Fantastic Four). -- Responsable de sistemas tuenti.com From tfheen at linpro.no Mon Oct 6 07:30:02 2008 From: tfheen at linpro.no (Tollef Fog Heen) Date: Mon, 06 Oct 2008 09:30:02 +0200 Subject: Cookie issue In-Reply-To: <48E621D2.1050809@startsiden.no> (Morten Bekkelund's message of "Fri, 03 Oct 2008 15:44:50 +0200") References: <48E610FC.7030607@startsiden.no> <87abdleqy7.fsf@qurzaw.linpro.no> <48E621D2.1050809@startsiden.no> Message-ID: <87ljx2yydh.fsf@qurzaw.linpro.no> ]] Morten Bekkelund | Tollef Fog Heen wrote: | | > if (req.url ~ "^/admin") { | > pass; | > } | > | > in vcl_recv, so I presume that's not the problem you are running into? | | Well, I've tried that. That passes the single request to ^/admin, but | does a lookup on every other request since the stuff under admin uses | a lot of stuff in other paths. So it's a bit tricky :) >From the sound of it, it sounds like you are doing something very wrong here; why do you want to do pass on other bits than the page itself? I presume any other content would be images, CSS and so on? What is the problem you are trying to solve? If you really want to go your route, you could do something like setting a cookie on /admin and then checking for that in vcl_recv and if so doing pass. It sounds completely wrong and quite error-prone to me, though. -- Tollef Fog Heen Redpill Linpro -- Changing the game! t: +47 21 54 41 73 From phk at phk.freebsd.dk Mon Oct 6 07:53:45 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Mon, 06 Oct 2008 07:53:45 +0000 Subject: 400 return code on Content-length: 0. In-Reply-To: Your message of "Fri, 03 Oct 2008 17:54:09 +0200." <200810031754.09644.ender@tuenti.com> Message-ID: <30077.1223279625@critter.freebsd.dk> Which version of varnish ? Can you provide a varnishlog output for such a transaction ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From marcussmith at britarch.ac.uk Mon Oct 6 20:11:00 2008 From: marcussmith at britarch.ac.uk (Marcus Smith) Date: Mon, 06 Oct 2008 21:11:00 +0100 Subject: Regular expressions and trailing slashes Message-ID: <48EA70D4.8050703@britarch.ac.uk> I have Varnish (the Beta 2 release) sitting in front of two web servers. Some parts of the site live on one of the servers, the rest on the other, so my VCL code looks like this: backend www1 { .host = "192.168.100.1"; .port = "8080"; } backend www2 { .host = "192.168.100.2"; .port = "8080"; } sub vcl_recv { if (req.http.host ~ "^(www.)?example.(net|org|com)$") { /* Normalise domain names. */ set req.http.host = "www.example.com"; if (req.url ~ "^/foo/.*$" || req.url ~ "^/bar/.*$" || req.url ~ "^/baz/.*$" ) { set req.backend = www1; } else { set req.backend = www2; } } else { error 404 "Unknown virtual host!"; } } Unfortunately I can only seem to make this work correctly if the URL used to request "/foo", "/bar" etc. contains two slashes. The regexes above match "/foo/", "/foo/index.html", "/foo/bar/" etc, but *not* "/foo" (no trailing slash). I want them to match "/foo" as well... but *not* things like "/foobar/" (first three letters in common). I don't seem to be able to get Varnish to do this. I thought that simply changing the pattern to "^/foo(/.*)?$" would do what I wanted, but it does not match. That pattern *does* match as intended in Apache though, with e.g. mod_rewrite directives. Surprisingly, removing the trailing slash from the pattern does not work either: "^/foo.*$". Instead, in both cases, Varnish redirects to "http://www.example.com:8080/foo/" on the www2 backend, which itself then returns a 404: not only does it fail, but it fails in an unexpected manner. I have also tried "^/foo(/(.*))?$" and "^/foo(/(.)*)?$", more out of curiosity than any expectation that extra parentheses might significantly alter the behaviour (they don't). It seems pretty clear that I must be missing something here... do regexes in VCL take "." to mean something other than "match any one character"? The way it is used in the example code for normalising domains (both above and in the FAQ) suggests to me that they might. What is the correct way to match "/foo" and any sub-files and sub-folders of "/foo" ("/foo/", "/foo/bar", "/foo/index.html" etc) but *not* "/foobar"? Many thanks in advance, Marcus -- Marcus Smith Information Officer The Council for British Archaeology From espen at linpro.no Tue Oct 7 13:49:46 2008 From: espen at linpro.no (Espen Braastad) Date: Tue, 07 Oct 2008 15:49:46 +0200 Subject: Remove only selected cookies from requests? Message-ID: <48EB68FA.5000509@linpro.no> Hi, The question: Is it possible to match cookie names in each request to a regexp, and remove only specified cookies? Other cookies must not be removed. The scenario: - All requests to the site from the clients have cookies from Google Analytics set. We would like to remove these. - Some of the requests to the site from the clients have cookies with session information set. We would like to keep these. Requests with the Google Analytics cookies set (the cookie names are __utmz, __utma, __utmb, __utmc) should get these cookies removed, and inserted into cache if no other cookies are present. The requests with the session cookies must be passed (default varnish behaviour). Today we are using unset req.http.cookie on some file types defined in the vcl, but a more efficient method would be to remove all the google cookies making varnish able to cache even more of the requests. Thanks, -- mvh Espen Braastad, +47 21 54 41 37 espen at linpro.no Linpro AS - Ledende p? Linux From skye at F4.ca Tue Oct 7 21:46:22 2008 From: skye at F4.ca (Skye Poier Nott) Date: Tue, 7 Oct 2008 14:46:22 -0700 Subject: varnishncsa and liblogging Message-ID: Hello again, I'm currently working on adding liblogging (reliable syslog over BEEP) support to varnishncsa. http://www.liblogging.org/ Is this something that the project would be interested in adding to trunk when it's done? I presume it would need to be wrapped in a configure --with-liblogging option (default off) Thanks, Skye From phk at phk.freebsd.dk Tue Oct 7 21:51:59 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Tue, 07 Oct 2008 21:51:59 +0000 Subject: varnishncsa and liblogging In-Reply-To: Your message of "Tue, 07 Oct 2008 14:46:22 MST." Message-ID: <53419.1223416319@critter.freebsd.dk> In message , Skye Poier Nott writes : >Hello again, > >I'm currently working on adding liblogging (reliable syslog over BEEP) >support to varnishncsa. >http://www.liblogging.org/ > >Is this something that the project would be interested in adding to >trunk when it's done? I presume it would need to be wrapped in a >configure --with-liblogging option (default off) If it is not too intrusive, yes. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From ender at tuenti.com Wed Oct 8 01:51:59 2008 From: ender at tuenti.com (David =?utf-8?q?Mart=C3=ADnez_Moreno?=) Date: Wed, 8 Oct 2008 03:51:59 +0200 Subject: 400 return code on Content-length: 0. In-Reply-To: <30077.1223279625@critter.freebsd.dk> References: <30077.1223279625@critter.freebsd.dk> Message-ID: <200810080352.00087.ender@tuenti.com> El lunes, 6 de octubre de 2008, Poul-Henning Kamp escribi?: > Which version of varnish ? 2.0-beta1. > Can you provide a varnishlog output for such a transaction ? Of course: 0 CLI - Rd ping 0 CLI - Wr 0 200 PONG 1223430225 1.0 9 SessionOpen c 88.28.44.79 49601 0.0.0.0:90 0 CLI - Rd ping 0 CLI - Wr 0 200 PONG 1223430228 1.0 9 ReqStart c 88.28.44.79 49601 259527650 9 HttpGarbage c GET /j/foo.gif Host: foo.tuenti.com Content-length: 0 9 VCL_call c error 9 VCL_return c deliver 9 Length c 428 9 VCL_call c deliver 9 VCL_return c deliver 9 TxProtocol c HTTP/1.1 9 TxStatus c 400 9 TxResponse c Bad Request 9 TxHeader c Server: Varnish 9 TxHeader c Retry-After: 0 9 TxHeader c Content-Type: text/html; charset=utf-8 9 TxHeader c Content-Length: 428 9 TxHeader c Date: Wed, 08 Oct 2008 01:43:49 GMT 9 TxHeader c X-Varnish: 259527650 9 TxHeader c Age: 0 9 TxHeader c Via: 1.1 varnish 9 TxHeader c Connection: keep-alive 9 ReqEnd c 259527650 1223430229.306536674 1223430229.306603670 2.370194197 0.000043154 0.000023842 0 StatAddr - 88.28.44.79 0 19 2 1 0 0 0 231 0 0 CLI - Rd ping 0 CLI - Wr 0 200 PONG 1223430231 1.0 9 SessionClose c no request 9 StatSess c 88.28.44.79 49601 2 1 1 0 0 0 231 0 I've tried also to remove such header in vcl_fetch, as such header seemed not to be reachable in vcl_recv (varnishd told me when I tried): sub vcl_fetch { if (obj.http.Content-length == "0") { remove obj.http.Content-length; } if (!obj.cacheable) { ... That's the only deviation apart from the usual trick for serving content without checking with backend: # if (req.http.Authorization || req.http.Cookie) { if (req.http.Authorization) { from the shipped default.vcl file. Thanks in advance, Ender. -- - ?Cu?l es el futuro perfecto de AMAR? - ?Hijitos! -- Susanita (Mafalda). -- Responsable de sistemas tuenti.com From michael at dynamine.net Wed Oct 8 02:24:07 2008 From: michael at dynamine.net (Michael S. Fischer) Date: Tue, 7 Oct 2008 19:24:07 -0700 Subject: varnishncsa and liblogging In-Reply-To: References: Message-ID: <86db848d0810071924u49ce8571l5771b29178778fff@mail.gmail.com> I assume this is for logging daemon metadata/error conditions and not actual traffic? If this is for request/response logging, consider implementing a bridge daemon that reads from the SHM like varnishlog or varnishncsa does, and which then sends the output via liblogging. This will provide the fastest possible performance without bogging down Varnish, and makes it such that it can be updated or modified entirely independent of varnish itself. --Michael On Tue, Oct 7, 2008 at 2:46 PM, Skye Poier Nott wrote: > Hello again, > > I'm currently working on adding liblogging (reliable syslog over BEEP) > support to varnishncsa. > http://www.liblogging.org/ > > Is this something that the project would be interested in adding to > trunk when it's done? I presume it would need to be wrapped in a > configure --with-liblogging option (default off) > > Thanks, > Skye > > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc > > From phk at phk.freebsd.dk Wed Oct 8 07:22:44 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Wed, 08 Oct 2008 07:22:44 +0000 Subject: 400 return code on Content-length: 0. In-Reply-To: Your message of "Wed, 08 Oct 2008 03:51:59 +0200." <200810080352.00087.ender@tuenti.com> Message-ID: <55832.1223450564@critter.freebsd.dk> In message <200810080352.00087.ender at tuenti.com>, David =?utf-8?q?Mart=C3=ADnez _Moreno?= writes: >> Can you provide a varnishlog output for such a transaction ? > > Of course: > > 0 CLI - Rd ping > 0 CLI - Wr 0 200 PONG 1223430225 1.0 > 9 SessionOpen c 88.28.44.79 49601 0.0.0.0:90 > 0 CLI - Rd ping > 0 CLI - Wr 0 200 PONG 1223430228 1.0 > 9 ReqStart c 88.28.44.79 49601 259527650 > 9 HttpGarbage c GET /j/foo.gif >Host: foo.tuenti.com >Content-length: 0 > The trouble starts here: The HTTP request is not accepted as well formed by Varnish. Can you please repeat the test, but this time run varnishlog with the -w somefile argument, and then make that (binary!) file available to me ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From nick at loman.net Thu Oct 9 16:29:25 2008 From: nick at loman.net (Nick Loman) Date: Thu, 09 Oct 2008 17:29:25 +0100 Subject: Mac connection problem with Varnish 2.0-rc1 Message-ID: <48EE3165.50602@loman.net> Hi there, A client using Mac version of Safari complains of intermittent errors "kCFErrorDomainCFNetwork error 302", perhaps every other page, on dynamically generated pages. This is a stock install of varnish-2.0-rc1, with default.vcl and default configuration values. Everything works perfectly on Safari PC, Firefox on Mac/PC, etc. Downgrading to varnish 1.1.2 solves the problem with same hardware/software/config. I've attached the varnish log for the problematic part. Any help appreciated, Cheers, Nick. 9 SessionOpen c HIDDEN_IP_ADDRESS 30775 0.0.0.0:8080 0 CLI - Rd ping 0 CLI - Wr 0 200 PONG 1223567243 1.0 9 ReqStart c HIDDEN_IP_ADDRESS 30775 363533487 9 RxRequest c GET 9 RxURL c /.. 9 RxProtocol c HTTP/1.1 9 RxHeader c User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_3; en-us) AppleWebKit/525.18 (KHTML, like Gecko) Version/3.1.1 Safari/525.20 9 RxHeader c Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 9 RxHeader c Accept-Language: en-us 9 RxHeader c Accept-Encoding: gzip, deflate 9 RxHeader c Cookie: egtv_seen[5462]=yes; egtv_seen[5383]=yes; egtv_seen[5466]=yes; egtv_seen[5494]=yes; egtv_seen[5577]=yes; egtv_seen[5774]=yes; egtv_seen[5837]=yes; egtv_seen[5847]=yes; egtv_seen[5844]=yes; egtv_seen[5898]=yes; egtv_seen[5928]=yes; egtv_seen[5957]= 9 RxHeader c Connection: keep-alive 9 RxHeader c Host: www.myhost.com 9 VCL_call c recv 9 VCL_return c lookup 9 VCL_call c hash 9 VCL_return c hash 9 VCL_call c miss 9 VCL_return c fetch 10 BackendClose b default 10 BackendOpen b default 127.0.0.1 55130 127.0.0.1 80 9 Backend c 10 default default 10 TxRequest b GET 10 TxURL b /forum_threads.php?category_id=group-threads&view=groups 10 TxProtocol b HTTP/1.1 10 TxHeader b User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_3; en-us) AppleWebKit/525.18 (KHTML, like Gecko) Version/3.1.1 Safari/525.20 10 TxHeader b Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 10 TxHeader b Accept-Language: en-us 10 TxHeader b Accept-Encoding: gzip, deflate .. Cookie line deleted 10 TxHeader b Host: www.myhost.com 10 TxHeader b X-Varnish: 363533487 10 TxHeader b X-Forwarded-For: HIDDEN_IP_ADDRESS 10 RxProtocol b HTTP/1.1 10 RxStatus b 200 10 RxResponse b OK 10 RxHeader b Date: Thu, 09 Oct 2008 15:47:23 GMT 10 RxHeader b Server: Apache/2.2.9 (Unix) 10 RxHeader b X-Powered-By: PHP/5.2.6-pl7-gentoo 10 RxHeader b Expires: Mon, 26 Jul 1997 05:00:00 GMT 10 RxHeader b Cache-Control: max-age=0 10 RxHeader b Pragma: no-cache .. 12 Set-Cookie lines deleted 10 RxHeader b Last-Modified: Thu, 09 Oct 2008 15:47:24 GMT 10 RxHeader b Vary: Accept-Encoding 10 RxHeader b Content-Encoding: gzip 10 RxHeader b Content-Length: 8956 10 RxHeader b Content-Type: text/html; charset=ISO-8859-1 9 ObjProtocol c HTTP/1.1 9 ObjStatus c 200 9 ObjResponse c OK 9 ObjHeader c Date: Thu, 09 Oct 2008 15:47:23 GMT 9 ObjHeader c Server: Apache/2.2.9 (Unix) 9 ObjHeader c X-Powered-By: PHP/5.2.6-pl7-gentoo 9 ObjHeader c Expires: Mon, 26 Jul 1997 05:00:00 GMT 9 ObjHeader c Cache-Control: max-age=0 9 ObjHeader c Pragma: no-cache .. 12 Set-Cookie lines deleted 9 ObjHeader c Last-Modified: Thu, 09 Oct 2008 15:47:24 GMT 9 ObjHeader c Vary: Accept-Encoding 9 ObjHeader c Content-Encoding: gzip 9 ObjHeader c Content-Type: text/html; charset=ISO-8859-1 10 BackendReuse b default 9 TTL c 363533487 RFC 0 1223567244 0 0 0 0 9 VCL_call c fetch 9 VCL_return c pass 9 Length c 8956 9 VCL_call c deliver 9 VCL_return c deliver 9 TxProtocol c HTTP/1.1 9 TxStatus c 200 9 TxResponse c OK 9 TxHeader c Server: Apache/2.2.9 (Unix) 9 TxHeader c X-Powered-By: PHP/5.2.6-pl7-gentoo 9 TxHeader c Expires: Mon, 26 Jul 1997 05:00:00 GMT 9 TxHeader c Cache-Control: max-age=0 9 TxHeader c Pragma: no-cache 9 TxHeader c Last-Modified: Thu, 09 Oct 2008 15:47:24 GMT 9 TxHeader c Vary: Accept-Encoding 9 TxHeader c Content-Encoding: gzip 9 TxHeader c Content-Type: text/html; charset=ISO-8859-1 9 TxHeader c Content-Length: 8956 9 TxHeader c Date: Thu, 09 Oct 2008 15:47:24 GMT 9 TxHeader c X-Varnish: 363533487 9 TxHeader c Age: 0 9 TxHeader c Via: 1.1 varnish 9 TxHeader c Connection: keep-alive 9 ReqEnd c 363533487 1223567243.957491398 1223567244.345397711 0.053010225 0.387861967 0.000044346 0 StatAddr - HIDDEN_IP_ADDRESS 0 970 54 79 0 0 79 16775 106768 0 ExpPick - 363533487 ttl 0 VCL_call - timeout 0 VCL_return - discard 0 ExpKill - 363533487 -1223567244 12 SessionOpen c 10.0.0.219 47006 0.0.0.0:8080 12 SessionClose c error 12 ReqEnd c 0 1223567245.333011866 1223567245.333011866 0.000076771 0.000000000 0.000000000 12 StatSess c 10.0.0.219 47006 0 1 0 0 0 0 0 0 0 StatAddr - 10.0.0.219 0 1006 135 0 0 0 0 0 0 9 SessionClose c blast 9 StatSess c HIDDEN_IP_ADDRESS 30775 0 1 1 0 0 1 437 8956 9 SessionOpen c HIDDEN_IP_ADDRESS 30781 0.0.0.0:8080 9 SessionClose c blast 9 ReqEnd c 0 1223567245.963477135 1223567245.963477135 0.034464121 0.000000000 0.000000000 9 StatSess c HIDDEN_IP_ADDRESS 30781 0 1 0 0 0 0 0 0 0 StatAddr - HIDDEN_IP_ADDRESS 0 972 55 79 0 0 79 16775 106768 From kurt at arstechnica.com Thu Oct 9 12:38:33 2008 From: kurt at arstechnica.com (Kurt Mackey) Date: Thu, 9 Oct 2008 07:38:33 -0500 Subject: How much of the ESI spec is implemented? Message-ID: <114A665E8FCADD4199D425621C5FF9C6789F3662@34093-MBX-C02.mex07a.mlsrvr.com> So ESI looks interesting, but I can't quite figure out the extent to which Varnish supports it. For instance, the ESI expressions look ridiculously useful in conjunction with the include tag. Can you do things like this? I'm not good enough to completely understand the source, but it looks like that example may not be possible. It would be awesome to see expression support and support for the choose/when/otherwise tags as well. Is that the type of thing that someone could sponsor for development? -Kurt From nick at loman.net Sat Oct 11 14:37:11 2008 From: nick at loman.net (Nick Loman) Date: Sat, 11 Oct 2008 15:37:11 +0100 Subject: Ticket #250 (POST error when using Opera) Message-ID: <48F0BA17.7020602@loman.net> Hi there Just to update you on my experience with Varnish 1.1.2, I had exactly the same problem as reported by the user on this ticket: http://varnish.projects.linpro.no/ticket/250 In my case, the cause of the problem after some analysis of the log files was that the "Connection-Length" header was being removed, despite my VCL actually only amending headers, not deleting them. Hence the backend script could not process the POST request properly. The application of the 3110 patch fixed the problem. http://varnish.projects.linpro.no/changeset/3110 However, as the patch supplied is for Varnish 2.0, I needed to adapt it to the 1.1.2 source code, as follows: --- varnish-1.1.2/bin/varnishd/cache_http.c.orig 2008-10-11 13:22:23.0000 00000 +0100 +++ varnish-1.1.2/bin/varnishd/cache_http.c 2008-10-11 13:25:26.000000000 +0 100 @@ -767,6 +767,7 @@ if (to->nhd < HTTP_HDR_MAX) { to->hd[to->nhd].b = fm->hd[n].b; to->hd[to->nhd].e = fm->hd[n].e; + to->hdf[to->nhd] = 0; to->nhd++; } else { VSL_stats->losthdr++; @@ -913,10 +914,12 @@ WSL(w, SLT_LostHeader, fd, "%s", string); to->hd[field].b = NULL; to->hd[field].e = NULL; + to->hdf[field] = 0; } else { memcpy(p, string, l + 1); to->hd[field].b = p; to->hd[field].e = p + l; + to->hdf[field] = 0; } } @@ -962,6 +965,7 @@ } else { to->hd[to->nhd].b = to->ws->f; to->hd[to->nhd].e = to->ws->f + n; + to->hdf[to->nhd] = 0; WS_Release(to->ws, n + 1); to->nhd++; } @@ -976,8 +980,10 @@ for (v = u = HTTP_HDR_FIRST; u < hp->nhd; u++) { if (http_IsHdr(&hp->hd[u], hdr)) continue; - if (v != u) + if (v != u) { memcpy(&hp->hd[v], &hp->hd[u], sizeof hp->hd[v]); + memcpy(&hp->hdf[v], &hp->hdf[u], sizeof hp->hdf[v]); + } v++; } hp->nhd = v; Hope this helps anyone else struggling with strange issues on Varnish 1.1.2. I think the key point is that simply amending client headers in vcl_recv can also trigger missing headers being sent to the backend. Regards, Nick. From phk at phk.freebsd.dk Sat Oct 11 14:46:14 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Sat, 11 Oct 2008 14:46:14 +0000 Subject: Ticket #250 (POST error when using Opera) In-Reply-To: Your message of "Sat, 11 Oct 2008 15:37:11 +0100." <48F0BA17.7020602@loman.net> Message-ID: <33582.1223736374@critter.freebsd.dk> In message <48F0BA17.7020602 at loman.net>, Nick Loman writes: >Hi there > >Just to update you on my experience with Varnish 1.1.2, [...] At this point we _really_ urge everybody to upgrade to Varnish 2.0. The release candidate 1 was released this week, and we are ready to cut the release now. Poul-Henning PS: for those who can attend, there is a release party in Oslo on 2008-10-21, email varnish at linpro.no for details. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From nick at loman.net Sat Oct 11 14:48:59 2008 From: nick at loman.net (Nick Loman) Date: Sat, 11 Oct 2008 15:48:59 +0100 Subject: Ticket #250 (POST error when using Opera) In-Reply-To: <33582.1223736374@critter.freebsd.dk> References: <33582.1223736374@critter.freebsd.dk> Message-ID: <48F0BCDB.2020700@loman.net> Poul-Henning Kamp wrote: > In message <48F0BA17.7020602 at loman.net>, Nick Loman writes: >> Hi there >> >> Just to update you on my experience with Varnish 1.1.2, [...] > > At this point we _really_ urge everybody to upgrade to Varnish 2.0. > > The release candidate 1 was released this week, and we are ready to > cut the release now. Hi Poul-Henning, I started on Varnish 2.0-rc1, which was 99% great, but we experienced a problem which was unfortunately a show-stopper. See my post on the 9th October, "Mac connection problem with Varnish 2.0-rc1". If it wasn't for that, I'd be using Varnish 2.0 on our production sites right now. Regards, Nick. From phk at phk.freebsd.dk Sat Oct 11 14:54:04 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Sat, 11 Oct 2008 14:54:04 +0000 Subject: Ticket #250 (POST error when using Opera) In-Reply-To: Your message of "Sat, 11 Oct 2008 15:48:59 +0100." <48F0BCDB.2020700@loman.net> Message-ID: <33663.1223736844@critter.freebsd.dk> In message <48F0BCDB.2020700 at loman.net>, Nick Loman writes: >I started on Varnish 2.0-rc1, which was 99% great, but we experienced a >problem which was unfortunately a show-stopper. See my post on the 9th >October, "Mac connection problem with Varnish 2.0-rc1". If it wasn't for >that, I'd be using Varnish 2.0 on our production sites right now. I looked at that and didn't see anything in the varnishlog output that indicated a problem. I have no idea what the safari error you quoted in the message mean... -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From nick at loman.net Sat Oct 11 14:56:43 2008 From: nick at loman.net (Nick Loman) Date: Sat, 11 Oct 2008 15:56:43 +0100 Subject: Ticket #250 (POST error when using Opera) In-Reply-To: <33663.1223736844@critter.freebsd.dk> References: <33663.1223736844@critter.freebsd.dk> Message-ID: <48F0BEAB.7010605@loman.net> Poul-Henning Kamp wrote: > In message <48F0BCDB.2020700 at loman.net>, Nick Loman writes: > >> I started on Varnish 2.0-rc1, which was 99% great, but we experienced a >> problem which was unfortunately a show-stopper. See my post on the 9th >> October, "Mac connection problem with Varnish 2.0-rc1". If it wasn't for >> that, I'd be using Varnish 2.0 on our production sites right now. > > I looked at that and didn't see anything in the varnishlog output > that indicated a problem. Nor did I! ;) > I have no idea what the safari error you quoted in the message mean... No, nor do I. But it meant that a user couldn't use the site properly, and when I downgraded to Varnish 1.1.2 it worked absolutely fine. This was Varnish 2.0-rc1 with totally default settings, so I couldn't blame a dodgy VCL. Where should I go from here? I could try the latest trunk, I guess? Regards, Nick. From phk at phk.freebsd.dk Sat Oct 11 15:03:43 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Sat, 11 Oct 2008 15:03:43 +0000 Subject: Ticket #250 (POST error when using Opera) In-Reply-To: Your message of "Sat, 11 Oct 2008 15:56:43 +0100." <48F0BEAB.7010605@loman.net> Message-ID: <33781.1223737423@critter.freebsd.dk> In message <48F0BEAB.7010605 at loman.net>, Nick Loman writes: >Poul-Henning Kamp wrote: >> In message <48F0BCDB.2020700 at loman.net>, Nick Loman writes: >> >>> I started on Varnish 2.0-rc1, which was 99% great, but we experienced a >>> problem which was unfortunately a show-stopper. See my post on the 9th >>> October, "Mac connection problem with Varnish 2.0-rc1". If it wasn't for >>> that, I'd be using Varnish 2.0 on our production sites right now. >> >> I looked at that and didn't see anything in the varnishlog output >> that indicated a problem. > >Nor did I! ;) Are you sure that transaction was from one of the failing views ? >Where should I go from here? I could try the latest trunk, I guess? Make sure you get a failing transaction in the varnishlog. If that shows nothing abnormal, try getting a tcpdump of that transaction and see if there is anything abnormal in that. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From jeanmarc.pouchoulon at gmail.com Sat Oct 11 16:46:57 2008 From: jeanmarc.pouchoulon at gmail.com (jean-marc pouchoulon) Date: Sat, 11 Oct 2008 18:46:57 +0200 Subject: req.restarts in rc1 In-Reply-To: <33582.1223736374@critter.freebsd.dk> References: <33582.1223736374@critter.freebsd.dk> Message-ID: <48F0D881.2060405@gmail.com> Hi, Poul-Henning Kamp a ?crit : > > At this point we _really_ urge everybody to upgrade to Varnish 2.0. > Done for me in prod but I 've got a problem with this conf. If I stopped b3 backend got an error 503 and no failover .... backend b3 { .host = "....."; .port = "....."; .probe = { .request = "GET / HTTP/1.1" "Host: ..." "Connection: close"; .timeout = 1s; .interval = 5s; .window = 4; .threshold = 2; } } backend b4 { .host = "..."; .port = "..."; .probe = { .request = "GET / HTTP/1.1" "Host: ...." "Connection: close"; .timeout = 1s; .interval = 5s; .window = 4; .threshold = 2; } } sub vcl_fetch { set obj.grace = 30s; if (obj.ttl < 600s) { set obj.ttl = 600s; } if (obj.status != 200 && obj.status != 302 && obj.status != 403 && obj.status != 404 ) { restart; } } sub vcl_recv { set req.grace = 30s; if (req.url ~ "/dir") { if (req.restarts == 0) { set req.backend = b1; } else { set req.backend = b2; } } else { if (req.restarts == 0) { set req.backend = b3; } else { set req.backend = b4; } } 4 RxHeader c Cache-Control: no-cache, max-age=259200 4 RxHeader c Connection: keep-alive 4 VCL_call c recv 4 VCL_return c lookup 4 VCL_call c hash 4 VCL_return c hash 4 VCL_call c miss 4 VCL_return c fetch 4 VCL_call c error 4 VCL_return c deliver 4 Length c 451 4 VCL_call c deliver 4 VCL_return c deliver 4 TxProtocol c HTTP/1.1 4 TxStatus c 503 4 TxResponse c Service Unavailable 4 TxHeader c Server: Varnish 4 TxHeader c Retry-After: 0 4 TxHeader c Content-Type: text/html; charset=utf-8 4 TxHeader c Content-Length: 451 4 TxHeader c Date: Sat, 11 Oct 2008 16:28:46 GMT 4 TxHeader c X-Varnish: 43575121 4 TxHeader c Age: 0 4 TxHeader c Via: 1.1 varnish 4 TxHeader c Connection: close 4 ReqEnd c 43575121 1223742526.863905191 1223742526.864029646 0.001247644 0.000095606 0.000028849 4 SessionClose c error Problem or Bad conf ? I'm using director and Fail Over is working fine jean-marc > The release candidate 1 was released this week, and we are ready to > cut the release now. > > Poul-Henning > > PS: for those who can attend, there is a release party in Oslo > on 2008-10-21, email varnish at linpro.no for details. > > From sergehelly at yahoo.fr Sun Oct 12 12:07:42 2008 From: sergehelly at yahoo.fr (Serge Helly) Date: Sun, 12 Oct 2008 12:07:42 +0000 (GMT) Subject: Varnishreplay question Message-ID: <478514.24557.qm@web26501.mail.ukl.yahoo.com> Hi, I've been playing with Varnish for a couple of months now: impressive, really ! I would like to replay Apache logs. Is there a way to run "varnishreplay -r ncsa-file" ? If no, could that be consider as a future enhancement ? Thx by advance, Serge From duja at torlen.net Thu Oct 16 11:58:38 2008 From: duja at torlen.net (duja at torlen.net) Date: Thu, 16 Oct 2008 13:58:38 +0200 Subject: Great work on 2.0 Message-ID: Good job on the final release guys, you have done a great job ;) / E From ottolski at web.de Thu Oct 16 15:18:17 2008 From: ottolski at web.de (Sascha Ottolski) Date: Thu, 16 Oct 2008 17:18:17 +0200 Subject: on linux: only one cpu in use? In-Reply-To: <87ej33m9r3.fsf@qurzaw.linpro.no> References: <200809081111.28168.ottolski@web.de> <200809161050.59439.ottolski@web.de> <87ej33m9r3.fsf@qurzaw.linpro.no> Message-ID: <200810161718.17461.ottolski@web.de> Am Montag 29 September 2008 14:12:16 schrieb Tollef Fog Heen: > ]] Sascha Ottolski > > | Am Montag 15 September 2008 16:16:33 schrieb Tollef Fog Heen: > | > ]] Sascha Ottolski > | > > | > | it's debian etch with the stock 2.6.18 kernel. performance is > | > | ok, beside the fact that the load climbs up when the cache is > | > | about 40-50 % full (with a 517GB cache file); below 40 % the > | > | load stays below 3, but after that level it rises to over 7 and > | > | even 10 at peak times. the response times are still good, > | > | though. > | > > | > Is this with the mmap or the malloc backend? > | > | well, as the man page says about malloc "Not recommended for > | production use.", I use the file backend :-) > > Indeed. Fixed that just now. It seems like newer kernels has some > performance problems we are tickling. I haven't had the time to > track down which commit breaks us, but it's fairly noticeable on busy > sites. > > | In case it matters, it's with trunk r2640. > > This is fairly old, though. I will test the current version soon. What would be the advice when giving the malloc backend a try? We run varnish on a 32 GB machine with about the same amout of swap, and use up to 517 GB as swap file (experimenting with smaller cache files to see if it changes load or response time, but so far couldn't really see a difference). The wiki advices to limit the malloc to less then swap space, which would be probably way to low for our needs. Should add a big swapfile? Thanks, Sascha From pdy at ucla.edu Thu Oct 16 21:49:42 2008 From: pdy at ucla.edu (Paul Yuergens) Date: Thu, 16 Oct 2008 14:49:42 -0700 Subject: resp.http.Set-Cookie not accessible in method vcl_fetch Message-ID: <59F53E3C-2776-467A-BE49-868BD3415221@ucla.edu> Hello, I am using Varnish as a Caching Reverse Proxy with Plone/Zope on OS 10.5 and am always getting this silly error. I'm following directions from Martin Aspelli's Plone 3 book. I think it's some silly hostname thing but I can't figure out what it is. I have debug-mode set to "off" in zope. Here is the command and response paul$ varnishd -a localhost:8082 -f plone.vcl -s file,cache-storage,1G Variable 'resp.http.Set-Cookie' not accessible in method 'vcl_fetch'. At: (plone.vcl Line 68 Pos 9) if (resp.http.Set-Cookie) { --------####################--- Thanks for your time, Paul + + + + + + + + + + + + + + + + + + + + + + + p a u l y u e r g e n s + w e b m a s t e r + u c l a p s y c h o l o g y + pdy at ucla.edu + -------------- next part -------------- An HTML attachment was scrubbed... URL: From espen at linpro.no Thu Oct 16 22:18:51 2008 From: espen at linpro.no (Espen Braastad) Date: Fri, 17 Oct 2008 00:18:51 +0200 (CEST) Subject: Remove only selected cookies from requests? In-Reply-To: <48EB68FA.5000509@linpro.no> References: <48EB68FA.5000509@linpro.no> Message-ID: <12298774.1131224195531909.OPEN-XCHANGE.WebMail.tomcat@ox.linpro.no> On 2008-10-07 15:49, Espen Braastad wrote: >Hi, > >The question: >Is it possible to match cookie names in each request to a regexp, and >remove only specified cookies? Other cookies must not be removed. The answer: Yes. The following two lines in vcl_recv will remove the cookie named baz only: sub vcl_recv { # Is it the first one? set req.http.cookie = regsub(req.http.cookie, "baz=[^;]+(; )?", ""); # Or perhaps one in the middle or the last one? set req.http.cookie = regsub(req.http.cookie, "(; )?baz=[^;]+", ""); } (Thank you, Tollef) mvh Espen Braastad, +47 938 60 737 espen at linpro.no Linpro AS - Ledende p? Linux From pdy at ucla.edu Thu Oct 16 22:37:55 2008 From: pdy at ucla.edu (Paul Yuergens) Date: Thu, 16 Oct 2008 15:37:55 -0700 Subject: resp.http.Set-Cookie not accessible in method vcl_fetch In-Reply-To: <59F53E3C-2776-467A-BE49-868BD3415221@ucla.edu> References: <59F53E3C-2776-467A-BE49-868BD3415221@ucla.edu> Message-ID: <0A340DEE-D34C-452E-98B8-DE0CDFBDEAB8@ucla.edu> To any plone people getting this error. Look at CacheFu Extras available on the CacheFu products page. It has scripts on how to make a better vcl file then the one that comes in Martin's book -p On Oct 16, 2008, at 2:49 PM, Paul Yuergens wrote: > Hello, > > I am using Varnish as a Caching Reverse Proxy with Plone/Zope on OS > 10.5 and am always getting this silly error. I'm following > directions from Martin Aspelli's Plone 3 book. I think it's some > silly hostname thing but I can't figure out what it is. I have debug- > mode set to "off" in zope. Here is the command and response > > paul$ varnishd -a localhost:8082 -f plone.vcl -s file,cache-storage,1G > Variable 'resp.http.Set-Cookie' not accessible in method 'vcl_fetch'. > At: (plone.vcl Line 68 Pos 9) > if (resp.http.Set-Cookie) { > --------####################--- > > Thanks for your time, > > Paul > > + + + + + + + + + + + + + + + + + + + + + > + > + p a u l y u e r g e n s > + w e b m a s t e r > + u c l a p s y c h o l o g y > + pdy at ucla.edu > + > > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc + + + + + + + + + + + + + + + + + + + + + + + p a u l y u e r g e n s + w e b m a s t e r + u c l a p s y c h o l o g y + pdy at ucla.edu + -------------- next part -------------- An HTML attachment was scrubbed... URL: From tfheen at linpro.no Fri Oct 17 11:51:02 2008 From: tfheen at linpro.no (Tollef Fog Heen) Date: Fri, 17 Oct 2008 13:51:02 +0200 Subject: Remove only selected cookies from requests? In-Reply-To: <12298774.1131224195531909.OPEN-XCHANGE.WebMail.tomcat@ox.linpro.no> (Espen Braastad's message of "Fri, 17 Oct 2008 00:18:51 +0200 (CEST)") References: <48EB68FA.5000509@linpro.no> <12298774.1131224195531909.OPEN-XCHANGE.WebMail.tomcat@ox.linpro.no> Message-ID: <87wsg730ex.fsf@qurzaw.linpro.no> ]] Espen Braastad | On 2008-10-07 15:49, Espen Braastad wrote: | | >The question: | >Is it possible to match cookie names in each request to a regexp, and | >remove only specified cookies? Other cookies must not be removed. | | The answer: | Yes. The following two lines in vcl_recv will remove the cookie named | baz only: I've added this to the wiki, http://varnish.projects.linpro.no/wiki/VCLExampleRemovingSomeCookies -- Tollef Fog Heen Redpill Linpro -- Changing the game! t: +47 21 54 41 73 From phk at phk.freebsd.dk Sat Oct 18 10:52:05 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Sat, 18 Oct 2008 10:52:05 +0000 Subject: Varnish after 2.0.x - Your input needed Message-ID: <19290.1224327125@critter.freebsd.dk> Hi gang, With a little luck, things will now settle down after the 2.0 release frenzy, and in about a weeks time we will set our eyes on 2.1 and beyond. To help that effort, I need your input. The major feature of 2.1 is going to be persistent storage, and the aim is to release that before midsummer next year. What happens along with that and beyond that is largely a matter of finding time and/or sponsors. To get a sense of where people want Varnish to go and what to focus on, I would like if each of you please would take a moment to study: http://varnish.projects.linpro.no/wiki/PostTwoShoppingList I am mostly interested in your input with respect to the "Major workpackages" part of this page, so if you could drop me an (private) email where you rate them on a scale of: * This would be a mistake (please tell me why) * I would like this * I'm willing to (help) pay for this. The second part of the file contains a number of minor ideas, which I suspect we will get along the way, but if any of them are important, they can be elevated to the top list. I am of course also interested in things missing from the list, so feel free to suggest ideas (including your rating). Thanks in advance, Poul-Henning -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From m at ooh.dk Sat Oct 18 23:35:17 2008 From: m at ooh.dk (Mikkel =?unknown-8bit?B?SMO4Z2g=?=) Date: Sun, 19 Oct 2008 01:35:17 +0200 Subject: Filtering by specific cookie Message-ID: <20081018233517.GA3795@samson.jerusalem.local> Hi there, I'm trying to throw together a workaround for a tricky caching issue with the CMS/CMF I'm using, Drupal, and for that purpose, I need to take decisions about whether to serve cached content on the basis of the client not sending a particular cookie. In http://varnish.projects.linpro.no/wiki/VCLExampleRemovingSomeCookies, the example shows removing cookies with regular expressions via the regsub function which I can't seem to find documentation on. Is there a similar function just for checking whether there's a match, or even better, a way to check this that doesn't involve regular expressions? Kind regards, Mikkel H??gh From perbu at linpro.no Sun Oct 19 06:02:49 2008 From: perbu at linpro.no (Per Buer) Date: Sun, 19 Oct 2008 08:02:49 +0200 Subject: Filtering by specific cookie In-Reply-To: <20081018233517.GA3795@samson.jerusalem.local> References: <20081018233517.GA3795@samson.jerusalem.local> Message-ID: <48FACD89.7040303@linpro.no> Mikkel H?gh skrev: > In http://varnish.projects.linpro.no/wiki/VCLExampleRemovingSomeCookies, > the example shows removing cookies with regular expressions via the > regsub function which I can't seem to find documentation on. > Is there a similar function just for checking whether there's a match, > or even better, a way to check this that doesn't involve regular > expressions? in vcl_revc you can do stuff like: if (req.http.cookie ~ "foo.*bar|bar.*foo") { lookup; } -- Per Buer - Leder Infrastruktur og Drift - Redpill Linpro Telefon: 21 54 41 21 - Mobil: 958 39 117 http://linpro.no/ | http://redpill.se/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 252 bytes Desc: OpenPGP digital signature URL: From shahid.azeez at gmail.com Mon Oct 20 09:47:28 2008 From: shahid.azeez at gmail.com (Shahid P.A) Date: Mon, 20 Oct 2008 15:17:28 +0530 Subject: In Varnish how can I redirect a URL to another URL on a different host. Message-ID: <8f7dd1b0810200247u7464d152v5490105362c57bd@mail.gmail.com> Hello, How do I redirect a URL to another URL on a different host. Using 301/302 redirection from the host for example www.example.com/index.html to www.redirect.com/index.php -------------- next part -------------- An HTML attachment was scrubbed... URL: From tfheen at linpro.no Mon Oct 20 13:45:36 2008 From: tfheen at linpro.no (Tollef Fog Heen) Date: Mon, 20 Oct 2008 15:45:36 +0200 Subject: Solaris x86_64 with SunStudio In-Reply-To: <231965b00810200434g164c522cr996898587fd6c72@mail.gmail.com> (Kamil Radziszewski's message of "Mon, 20 Oct 2008 13:34:57 +0200") References: <231965b00810200434g164c522cr996898587fd6c72@mail.gmail.com> Message-ID: <87r66bqt1b.fsf@qurzaw.linpro.no> ]] "Kamil Radziszewski" | 1. LDFLAGS=-m64 CFLAGS=-m64 ./configure --prefix=/opt/varnish-2.0.1 | 2. make && make install You need to add VCC_CC="cc -m64 -Kpic -G -o %o %s" to the configure line. -- Tollef Fog Heen Redpill Linpro -- Changing the game! t: +47 21 54 41 73 From ottolski at web.de Mon Oct 20 14:59:37 2008 From: ottolski at web.de (Sascha Ottolski) Date: Mon, 20 Oct 2008 16:59:37 +0200 Subject: varnish 2.0.1 truncates pre-created cache file? Message-ID: <200810201659.37280.ottolski@web.de> Hi, bug or feature? With the new version, my cache file gets shrunk. I made it with dd with a size of ~259 GB, but after starting varnishd, only about 119 GB are left. I don't give a size parameter when starting, as in -s file,/var/cache/varnish/store.bin which worked as expected with older releases. Thanks, Sascha From ottolski at web.de Mon Oct 20 15:14:37 2008 From: ottolski at web.de (Sascha Ottolski) Date: Mon, 20 Oct 2008 17:14:37 +0200 Subject: varnish 2.0.1 truncates pre-created cache file? In-Reply-To: <200810201659.37280.ottolski@web.de> References: <200810201659.37280.ottolski@web.de> Message-ID: <200810201714.37295.ottolski@web.de> Am Montag 20 Oktober 2008 16:59:37 schrieb Sascha Ottolski: > Hi, > > bug or feature? With the new version, my cache file gets shrunk. I > made it with dd with a size of ~259 GB, but after starting varnishd, > only about 119 GB are left. I don't give a size parameter when > starting, as in > > -s file,/var/cache/varnish/store.bin > > which worked as expected with older releases. weird, after another reboot/restart, the cache file now has 189 GB ?! Cheers, Sascha From tfheen at linpro.no Tue Oct 21 06:27:18 2008 From: tfheen at linpro.no (Tollef Fog Heen) Date: Tue, 21 Oct 2008 08:27:18 +0200 Subject: varnish 2.0.1 truncates pre-created cache file? In-Reply-To: <200810201659.37280.ottolski@web.de> (Sascha Ottolski's message of "Mon, 20 Oct 2008 16:59:37 +0200") References: <200810201659.37280.ottolski@web.de> Message-ID: <878wsia2ex.fsf@qurzaw.linpro.no> ]] Sascha Ottolski | bug or feature? With the new version, my cache file gets shrunk. I made | it with dd with a size of ~259 GB, but after starting varnishd, only | about 119 GB are left. I don't give a size parameter when starting, as | in | | -s file,/var/cache/varnish/store.bin | | which worked as expected with older releases. I would guess at you running into us using 50% of the available space on the file system by default. If we don't use the existing size, I'd argue it's a bug, so please file one. -- Tollef Fog Heen Redpill Linpro -- Changing the game! t: +47 21 54 41 73 From anders at fupp.net Tue Oct 21 12:20:50 2008 From: anders at fupp.net (Anders Nordby) Date: Tue, 21 Oct 2008 14:20:50 +0200 Subject: In Varnish how can I redirect a URL to another URL on a different host. In-Reply-To: <8f7dd1b0810200247u7464d152v5490105362c57bd@mail.gmail.com> References: <8f7dd1b0810200247u7464d152v5490105362c57bd@mail.gmail.com> Message-ID: <20081021122050.GA77965@fupp.net> Hi, AFAIK, you can not (easily) do this. What you can do is to change req.url in vcl_recv to another URL that will do the redirect for you. On Mon, Oct 20, 2008 at 03:17:28PM +0530, Shahid P.A wrote: > Hello, > How do I redirect a URL to another URL on a different host. Using > 301/302 redirection from the host for example > [1]www.example.com/index.html to [2]www.redirect.com/index.php > > References > > 1. http://www.example.com/index.html > 2. http://www.redirect.com/index.php > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc -- Anders. From ottolski at web.de Tue Oct 21 13:03:22 2008 From: ottolski at web.de (Sascha Ottolski) Date: Tue, 21 Oct 2008 15:03:22 +0200 Subject: inline C-Code Message-ID: <200810211503.22923.ottolski@web.de> Hi, in one or the other presentation slide is a small example about inline C-Code; and there is a commit that only says "undocmented and unsupported" (http://varnish.projects.linpro.no/changeset/1301). Now I'm wondering, is there anyway at list ab bit of documentation somewhere, or could someone post some examples to the list? (may be a searchable archive of this list would reveal some?) Basically, I want to do a bit of calculation with the help of a (pseudo-)encrypted URL and the current time (which seems to be available as the "new" keyword), to decide if the request should be served, and if so, create the real URL that is to be send to the backend, and a hashkey. So probably I need something like the following seudo-code (my C knowledge is kinda limited :-(): sub vcl_recv { if (req.url ~ "/enc-prefix/") { C{ timestamp, real_url = decrypt(req.url); // valid for two hours if not (now + 2h < timestamp < now) { error XXX "page expired"; } req.url = real_url; }C lookup; } } So I guess the main questions are: - how to access existing vcl names/variables - how to set existing vcl names/variables - what if I need access to external C libraries, i.e. to use a custom "decrypt" function? Any pointers greatly appreciated, Sascha From b.perwenis at gmail.com Tue Oct 21 13:40:42 2008 From: b.perwenis at gmail.com (Bartek Perwenis) Date: Tue, 21 Oct 2008 15:40:42 +0200 Subject: varnish 2.0.1 under Solaris 10 Message-ID: <7f97c4a0810210640v31820ad2wf6290ace2a7e1450@mail.gmail.com> I want to migrate my varnish server from Linux to Solaris. I'm trying to compile varnish on a Solaris 10u5 x86 system. I'm using gcc provided by Sun with the installation cd... [root at varnish-pub-0-151 /]# which gcc /usr/sfw/bin/gcc [root at varnish-pub-0-151 /]# gcc -v Reading specs from /usr/sfw/lib/gcc/i386-pc-solaris2.10/3.4.3/specs Configured with: /builds/sfw10-gate/usr/src/cmd/gcc/gcc-3.4.3/configure --prefix=/usr/sfw --with-as=/usr/sfw/bin/gas --with-gnu-as --with-ld=/usr/ccs/bin/ld --without-gnu-ld --enable-languages=c,c++ --enable-shared Thread model: posix gcc version 3.4.3 (csl-sol210-3_4-branch+sol_rpath) ...and dependencies installed from sunfreeware.com (make-3.81, automake-1.10.1, autoconf-2.63, libtool-1.5.24)... [root at varnish-pub-0-151 /]# which make automake autoconf libtool /usr/local/bin/make /usr/local/bin/automake /usr/local/bin/autoconf /usr/local/bin/libtool [root at varnish-pub-0-151 /]# make -v GNU Make 3.81 Copyright (C) 2006 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. This program built for i386-pc-solaris2.10 ... and when I execute make, I have this error: (...) Making all in bin make[2]: Entering directory `/usr/share/src/varnish-2.0.1/bin' Making all in varnishadm make[3]: Entering directory `/usr/share/src/varnish-2.0.1/bin/varnishadm' gcc -DHAVE_CONFIG_H -I. -I../.. -I../../include -g -O2 -MT varnishadm.o -MD -MP -MF .deps/varnishadm.Tpo -c -o varnishadm.o varnishadm.c mv -f .deps/varnishadm.Tpo .deps/varnishadm.Po /bin/bash ../../libtool --tag=CC --mode=link gcc -g -O2 -o varnishadm varnishadm.o ../../lib/libvarnish/libvarnish.la ../../lib/libvarnishcompat/ libvarnishcompat.la -lnsl -lsocket -lumem -lsendfile mkdir .libs gcc -g -O2 -o .libs/varnishadm varnishadm.o ../../lib/libvarnish/.libs/libvarnish.so -lrt -lm ../../lib/libvarnishcompat/.libs/libvarnishcompat.so -lnsl -lsocket -lumem -lsendfile -R/opt/varnish-sfw/lib Undefined first referenced symbol in file isfinite ../../lib/libvarnish/.libs/libvarnish.so ld: fatal: Symbol referencing errors. No output written to .libs/varnishadm collect2: ld returned 1 exit status make[3]: *** [varnishadm] Error 1 make[3]: Leaving directory `/usr/share/src/varnish-2.0.1/bin/varnishadm' make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory `/usr/share/src/varnish-2.0.1/bin' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/usr/share/src/varnish-2.0.1' make: *** [all] Error 2 What am I missing? Any help from you guys? config.log is in the attachment. Any other informations I can provide? Best regards, Bartek. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: config.log.tar.gz Type: application/x-gzip Size: 13095 bytes Desc: not available URL: From kradziszewski at gmail.com Tue Oct 21 14:15:10 2008 From: kradziszewski at gmail.com (Kamil Radziszewski) Date: Tue, 21 Oct 2008 16:15:10 +0200 Subject: Solaris x86_64 with SunStudio In-Reply-To: <87r66bqt1b.fsf@qurzaw.linpro.no> References: <231965b00810200434g164c522cr996898587fd6c72@mail.gmail.com> <87r66bqt1b.fsf@qurzaw.linpro.no> Message-ID: <231965b00810210715h65bccab8j4715e90deb56c3c@mail.gmail.com> Hi, thanks for your reply. I didn't noticed that VCC_CC option in configure. But now I have another problem. When I try to GET a file through varnish the request is not sent to the backend and varnish reports 503 service unavailable. I'm running 2.0.1 varnish under Solaris 10 5/08 s10x_u5wos_10 X86 OS. Here is the output from the varnishlog: 5 SessionOpen c 172.21.0.59 54358 :80 5 ReqStart c 172.21.0.59 54358 1261771362 5 RxRequest c GET 5 RxURL c /02/995/2995913.2082909.jpg 5 RxProtocol c HTTP/1.1 5 RxHeader c Host: 172.21.250.47 5 RxHeader c User-Agent: Links (2.1pre37; Linux 2.6.26-1-amd64 x86_64; 156x59) 5 RxHeader c Accept: */* 5 RxHeader c Accept-Encoding: gzip, deflate 5 RxHeader c Accept-Charset: us-ascii, ISO-8859-1, ISO-8859-2, ISO-8859-3, ISO-8859-4, ISO-8859-5, ISO-8859-6, ISO-8859-7, ISO-8859-8, ISO-8859-9, ISO-8859-10, ISO-8859-13, ISO-8859-14, ISO-8859-15, ISO-8859-16, windows-1250, windows-1251, windows-1252, windows-1256, 5 RxHeader c Accept-Language: en, *;q=0.1 5 RxHeader c Connection: Keep-Alive 5 VCL_call c recv 5 VCL_return c lookup 5 VCL_call c hash 5 VCL_return c hash 5 VCL_call c miss 5 VCL_return c fetch 5 VCL_call c error 5 VCL_return c deliver 5 Length c 453 5 VCL_call c deliver 5 VCL_return c deliver 5 TxProtocol c HTTP/1.1 5 TxStatus c 503 5 TxResponse c Service Unavailable 5 TxHeader c Server: Varnish 5 TxHeader c Retry-After: 0 5 TxHeader c Content-Type: text/html; charset=utf-8 5 TxHeader c Content-Length: 453 5 TxHeader c Date: Tue, 21 Oct 2008 14:12:10 GMT 5 TxHeader c X-Varnish: 1261771362 5 TxHeader c Age: 0 5 TxHeader c Via: 1.1 varnish 5 TxHeader c Connection: close 5 ReqEnd c 1261771362 1224598330.323645830 1224598330.324286938 0.000258923 0.000586748 0.000054359 5 SessionClose c error 5 StatSess c 172.21.0.59 54358 0 1 1 0 0 0 235 453 0 StatAddr - 172.21.0.59 0 0 1 1 0 0 0 235 453 Best regards. Kamil -------------- next part -------------- An HTML attachment was scrubbed... URL: From m.marshall at imb.uq.edu.au Wed Oct 22 00:41:30 2008 From: m.marshall at imb.uq.edu.au (Mhairi Marshall) Date: Wed, 22 Oct 2008 10:41:30 +1000 Subject: Guru Meditation error Message-ID: <009501c933de$ecad0c90$ef756682@IMBPC.AD> Hi, I have been getting the Guru Meditation when using varnish (version 1.1.2) cache for Plone sites. I have seen from the mailing list that some people have been having the same problem but haven't seen any solutions posted. Is there any configuration for varnish or apache that will stop this error from happening? Thanks Mhairi -------------- next part -------------- An HTML attachment was scrubbed... URL: From jdouglas at bksnetworks.com Wed Oct 22 09:22:24 2008 From: jdouglas at bksnetworks.com (jdouglas) Date: Wed, 22 Oct 2008 05:22:24 -0400 Subject: Varnish 2 VCL Message-ID: <20081022092224.e1d8dd58@mx-1.bksnetworks.com> How do i get varnish to cache all images? I had varnish 1 working with this but it seems my vcl does not work at all with 2 Here is my VCL # # This is a basic VCL configuration file for varnish. See the vcl(7) # man page for details on VCL syntax and semantics. # # $Id: default.vcl 1818 2007-08-09 12:12:12Z des $ # # Default backend definition. Set this to point to your content # server. backend default { .host = "127.0.0.1"; .port = "80"; } acl purge { "localhost"; } sub vcl_recv { if (req.request == "PURGE") { if (!client.ip ~ purge) { error 405 "Not allowed."; } lookup; } if (req.request != "GET" && req.request != "HEAD") { pipe; } if (req.http.Expect) { pipe; } /* Always cache images and binaries */ if (req.url ~ "\.(jpg|jpeg|gif|png|tiff|tif|svg|swf|ico|vsd|doc|ppt|pps|xls|pdf|mp3|mp4|m4a|ogg|mov|avi|wmv|sxw|zip|gz|bz2|tgz|tar|rar|odc|odb|odf|odg|odi|odp|ods|odt|sxc|sxd|sxi|sxw|dmg|torrent|deb|msi|iso|rpm)$") { lookup; } /* Always cache CSS and javascript */ if (req.url ~ "\.(css|js)$") { lookup; } lookup; } /* Deal with purge requests */ sub vcl_hit { if (req.request == "PURGE") { set obj.ttl = 0s; error 200 "Purged"; } deliver; } sub vcl_miss { if (req.http.If-Modified-Since) { pass; } if (req.request == "PURGE") { error 404 "Not in cache"; } fetch; } sub vcl_fetch { if (!obj.valid) { error; } if (!obj.cacheable) { pass; } if (obj.http.Set-Cookie) { pass; } if(obj.http.Set-Cookie){ pass; } if(obj.http.Pragma ~ "no-cache" || obj.http.Cache-Control ~ "no-cache" || obj.http.Cache-Control ~ "private"){ pass; } if (req.url ~ "(errorPage?)") { set obj.ttl = 0s; } if (req.url ~ "(exceptionPage?)") { set obj.ttl = 0s; } if (req.url ~ "(unauthorizedPage?)") { set obj.ttl = 0s; } if (req.url ~ "(type=The+requested+Virtual+Kiosk+is+currently+not+available+from+%5Benter+Virtual+Kiosk+Sponsor+Name+here%5D.+Please+close+the+Virtual+Kiosk+and+try+again.)") { set obj.ttl = 0s; } insert; } sub vcl_deliver { remove resp.http.X-Varnish; remove resp.http.Via; } -------------- next part -------------- An HTML attachment was scrubbed... URL: From perbu at linpro.no Wed Oct 22 11:02:46 2008 From: perbu at linpro.no (Per Buer) Date: Wed, 22 Oct 2008 13:02:46 +0200 Subject: Guru Meditation error In-Reply-To: <009501c933de$ecad0c90$ef756682@IMBPC.AD> References: <009501c933de$ecad0c90$ef756682@IMBPC.AD> Message-ID: <48FF0856.1030300@linpro.no> Mhairi Marshall skrev: > > I have been getting the Guru Meditation when using varnish (version > 1.1.2) cache for Plone sites. I have seen from the mailing list that > some people have been having the same problem but haven?t seen any > solutions posted. > > > > Is there any configuration for varnish or apache that will stop this > error from happening? Its hard to say without any more information. Anyway, you should really upgrade to Varnish 2.0.1 (or 2.0.2 if its out). Varnish 2 is much more robust. This might be because Plone sometimes misbehavies - but try an upgrade and get back to us if you're still having trouble. -- Per Buer - Leder Infrastruktur og Drift - Redpill Linpro Telefon: 21 54 41 21 - Mobil: 958 39 117 http://linpro.no/ | http://redpill.se/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 252 bytes Desc: OpenPGP digital signature URL: From m.marshall at imb.uq.edu.au Fri Oct 24 01:41:34 2008 From: m.marshall at imb.uq.edu.au (Mhairi Marshall) Date: Fri, 24 Oct 2008 11:41:34 +1000 Subject: Guru Meditation error In-Reply-To: <48FF0856.1030300@linpro.no> References: <009501c933de$ecad0c90$ef756682@IMBPC.AD> <48FF0856.1030300@linpro.no> Message-ID: <005601c93579$a5815000$ef756682@IMBPC.AD> Hi, I have upgraded to varnish 2.0.1 and got the error again. 0 0.00 0.00 Backend connections not attempted 0 0.00 0.00 Backend connections too many 6 0.00 0.00 Backend connections failures 244 0.00 0.00 Backend connections reuses 246 0.00 0.00 Backend connections recycles 0 0.00 0.00 Backend connections unused It seems to be worse in the morning when the website hasn't been used overnight. Our network guys say it is an application problem as the application needs to send keep alives. Thanks Mhairi -----Original Message----- From: Per Buer [mailto:perbu at linpro.no] Sent: Wednesday, 22 October 2008 9:03 PM To: Mhairi Marshall Cc: varnish-misc at projects.linpro.no Subject: Re: Guru Meditation error Mhairi Marshall skrev: > > I have been getting the Guru Meditation when using varnish (version > 1.1.2) cache for Plone sites. I have seen from the mailing list that > some people have been having the same problem but haven't seen any > solutions posted. > > > > Is there any configuration for varnish or apache that will stop this > error from happening? Its hard to say without any more information. Anyway, you should really upgrade to Varnish 2.0.1 (or 2.0.2 if its out). Varnish 2 is much more robust. This might be because Plone sometimes misbehavies - but try an upgrade and get back to us if you're still having trouble. -- Per Buer - Leder Infrastruktur og Drift - Redpill Linpro Telefon: 21 54 41 21 - Mobil: 958 39 117 http://linpro.no/ | http://redpill.se/ From perbu at linpro.no Fri Oct 24 07:26:19 2008 From: perbu at linpro.no (Per Buer) Date: Fri, 24 Oct 2008 09:26:19 +0200 Subject: Guru Meditation error In-Reply-To: <005601c93579$a5815000$ef756682@IMBPC.AD> References: <009501c933de$ecad0c90$ef756682@IMBPC.AD> <48FF0856.1030300@linpro.no> <005601c93579$a5815000$ef756682@IMBPC.AD> Message-ID: <4901789B.9020506@linpro.no> Could you supply the complete varnishstat output? I _guess_ this is a misbehaving Plone installation. Per. Mhairi Marshall skrev: > Hi, > > I have upgraded to varnish 2.0.1 and got the error again. > > 0 0.00 0.00 Backend connections not attempted > 0 0.00 0.00 Backend connections too many > 6 0.00 0.00 Backend connections failures > 244 0.00 0.00 Backend connections reuses > 246 0.00 0.00 Backend connections recycles > 0 0.00 0.00 Backend connections unused > > It seems to be worse in the morning when the website hasn't been used > overnight. Our network guys say it is an application problem as the > application needs to send keep alives. > > Thanks > > Mhairi > > > > -----Original Message----- > From: Per Buer [mailto:perbu at linpro.no] > Sent: Wednesday, 22 October 2008 9:03 PM > To: Mhairi Marshall > Cc: varnish-misc at projects.linpro.no > Subject: Re: Guru Meditation error > > Mhairi Marshall skrev: >> I have been getting the Guru Meditation when using varnish (version >> 1.1.2) cache for Plone sites. I have seen from the mailing list that >> some people have been having the same problem but haven't seen any >> solutions posted. >> >> >> >> Is there any configuration for varnish or apache that will stop this >> error from happening? > > Its hard to say without any more information. Anyway, you should really > upgrade to Varnish 2.0.1 (or 2.0.2 if its out). Varnish 2 is much more > robust. > > This might be because Plone sometimes misbehavies - but try an upgrade > and get back to us if you're still having trouble. > -- Per Buer - Leder Infrastruktur og Drift - Redpill Linpro Telefon: 21 54 41 21 - Mobil: 958 39 117 http://linpro.no/ | http://redpill.se/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 252 bytes Desc: OpenPGP digital signature URL: From olau at iola.dk Fri Oct 31 17:02:14 2008 From: olau at iola.dk (Ole Laursen) Date: Fri, 31 Oct 2008 18:02:14 +0100 Subject: Wishlist: filtering for varnishhist Message-ID: Hi, Varnishhist is pretty cool. Unfortunately, most of my data comes from image files which are served by a well-functioning lighttpd instance. So I'm really only interested in the data from the Apache web server running Django. It would help if I could filter on arbitrary fields, and varnishhist would then ignore requests with matching fields. -X didn't help, and when looking at the source, it seems obvious why. :) -- Ole Laursen http://www.iola.dk/