From scott.stewart at nbtsolutions.com Sun Jun 3 15:00:59 2012 From: scott.stewart at nbtsolutions.com (Scott Stewart) Date: Sun, 3 Jun 2012 11:00:59 -0400 Subject: Basic Auth In-Reply-To: References: Message-ID: Hi Hugo, -I did as you suggested as added the 'pass' for requests asking for authentication. And sure enough, it started asking for the authentication--however it doesnt stop asking for the authentication--it never seems to authenticate. I know the auth works, because I can access it without going through varnish and it works fine. Any idea what could be the problem here? FWIW, The auth is not a typical basic auth, but a basic auth against an ldap using the apache and the mod_ldapns_auth On Wed, May 30, 2012 at 3:50 PM, Scott Stewart < scott.stewart at nbtsolutions.com> wrote: > Hi > > I have a newly minted varnish (and a newly minted varnish user myself) > install that has two backends, as spec'ed in this vcl (below). > > The "wms1" has no auth on it. > > The "default" backend is a Apache server with basic auth, but no one is > getting asked to authenticate. My understanding was the out of the box > config for varnish would not cache those authentication required pages, but > no one is being asked for a username/password > > What am I doing wrong here? The app works perfectly otherwise. > > > > # This is a basic VCL configuration file for varnish. See the vcl(7) > # man page for details on VCL syntax and semantics. > # > # Default backend definition. Set this to point to your content > # server. > # > backend default { > .host = "127.0.0.1"; > .port = "8081"; > } > > backend wms1 { > .host = "example.com"; > .port = "80"; > } > > sub vcl_recv { > > if (req.http.host ~ "^(mapsdev\.)example\.com" && req.url~ "^/wms") { > set req.http.host = "maps.example.com"; > set req.url = regsub(req.url, "^/wms", "/wms"); > set req.backend = wms1; > if (req.request == "GET" && req.http.cookie) > { > unset req.http.cookie; > } > } > if (req.http.host ~ "^(mapsdev\.)?example\.com" && req.url~ "^/app/") { > set req.url = regsub(req.url, "^/app/", "/flol/admin_2/public/"); > remove req.http.X-Forwarded-For; > set req.http.X-Forwarded-For = client.ip; > } > > sub vcl_fetch { > set beresp.ttl = 72h; // default ttl 72 hours > if (req.http.Authorization) > { > return(hit_for_pass); > } > } > > # > # Below is a commented-out copy of the default VCL logic. If you > # redefine any of these subroutines, the built-in logic will be > # appended to your code. > # sub vcl_recv { > # if (req.restarts == 0) { > # if (req.http.x-forwarded-for) { > # set req.http.X-Forwarded-For = > # req.http.X-Forwarded-For + ", " + client.ip; > # } else { > # set req.http.X-Forwarded-For = client.ip; > # } > # } > # if (req.request != "GET" && > # req.request != "HEAD" && > # req.request != "PUT" && > # req.request != "POST" && > # req.request != "TRACE" && > # req.request != "OPTIONS" && > # req.request != "DELETE") { > # /* Non-RFC2616 or CONNECT which is weird. */ > # return (pipe); > # } > # if (req.request != "GET" && req.request != "HEAD") { > # /* We only deal with GET and HEAD by default */ > # return (pass); > # } > # if (req.http.Authorization || req.http.Cookie) { > # /* Not cacheable by default */ > # return (pass); > # } > # return (lookup); > # } > # > # sub vcl_pipe { > # # Note that only the first request to the backend will have > # # X-Forwarded-For set. If you use X-Forwarded-For and want to > # # have it set for all requests, make sure to have: > # # set bereq.http.connection = "close"; > # # here. It is not set by default as it might break some broken web > # # applications, like IIS with NTLM authentication. > # return (pipe); > # } > # > # sub vcl_pass { > # return (pass); > # } > # > # sub vcl_hash { > # hash_data(req.url); > # if (req.http.host) { > # hash_data(req.http.host); > # } else { > # hash_data(server.ip); > # } > # return (hash); > # } > # > # sub vcl_hit { > # return (deliver); > # } > # > # sub vcl_miss { > # return (fetch); > # } > # > # sub vcl_fetch { > # if (beresp.ttl <= 0s || > # beresp.http.Set-Cookie || > # beresp.http.Vary == "*") { > # /* > # * Mark as "Hit-For-Pass" for the next 2 minutes > # */ > # set beresp.ttl = 120 s; > # return (hit_for_pass); > # } > # return (deliver); > # } > # > # sub vcl_deliver { > # return (deliver); > # } > # > # sub vcl_error { > # set obj.http.Content-Type = "text/html; charset=utf-8"; > # set obj.http.Retry-After = "5"; > # synthetic {" > # > # # "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> > # > # > # "} + obj.status + " " + obj.response + {" > # > # > #

Error "} + obj.status + " " + obj.response + {"

> #

"} + obj.response + {"

> #

Guru Meditation:

> #

XID: "} + req.xid + {"

> #
> #

Varnish cache server

> # > # > # "}; > # return (deliver); > # } > # > # sub vcl_init { > # return (ok); > # } > # > # sub vcl_fini { > # return (ok); > # } > > -- Scott Stewart Principal NBT Solutions LLC phone: (757)941-5110 email: scott.stewart at nbtsolutions.com website:www.nbtsolutions.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From tfheen at varnish-software.com Mon Jun 4 06:33:18 2012 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Mon, 4 Jun 2012 08:33:18 +0200 Subject: Varnish Cache 3.0.3 rc 1 Message-ID: <20120604063318.GA11089@err.no> Hi all, we're finally getting close to 3.0.3 (it's taken a while!), so if people could please try it, that would be most appreciated. Source can be downloaded from http://repo.varnish-cache.org/test/varnish-3.0.3-rc1.tar.gz, packages live in http://repo.varnish-cache.org/test/lucid/, http://repo.varnish-cache.org/test/squeeze/ and http://repo.varnish-cache.org/test/rpms/ Bugs to trac, please. Feedback most appreciated! -- Tollef Fog Heen Technical lead, Varnish Software t: +47 21 98 92 64 From tfheen at varnish-software.com Mon Jun 4 06:34:31 2012 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Mon, 4 Jun 2012 08:34:31 +0200 Subject: guro In-Reply-To: <22629abc65c8b84673389a658a2df293@mail.gmail.com> References: <22629abc65c8b84673389a658a2df293@mail.gmail.com> Message-ID: <20120604063431.GB11089@err.no> ]] Jon Hauge > Why don?t varnish not just serve the cached version? Because that's not what grace does. You probably want to look at saint mode for your use case. -- Tollef Fog Heen Technical lead, Varnish Software t: +47 21 98 92 64 From tfheen at varnish-software.com Mon Jun 4 06:44:15 2012 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Mon, 4 Jun 2012 08:44:15 +0200 Subject: how to debug varnish without compiler optimization In-Reply-To: References: Message-ID: <20120604064415.GC11089@err.no> ]] ?? > export CFLAGS=" -g -O0 " && ./configure && make && make install [?] > it seems strange, how to solve it? thanks for you answer! Seems like you might need a make clean in there too. -- Tollef Fog Heen Technical lead, Varnish Software t: +47 21 98 92 64 From tfheen at varnish-software.com Mon Jun 4 06:46:30 2012 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Mon, 4 Jun 2012 08:46:30 +0200 Subject: Varnish sending 304 not modified - unwanted In-Reply-To: References: Message-ID: <20120604064630.GD11089@err.no> ]] Shibashish > I have a site which has a FB login in it, login happens, but while logout, > varnish sends a 304 not modified, hence the page is not refreshed and > logout is not shown on the page. The cookies do get deleted as it should > happen. If i check from backend, everything seems ok. How can I rectify > this? unset If-Modified-Since and If-None-Match headers and it won't do the 304 handling. -- Tollef Fog Heen Technical lead, Varnish Software t: +47 21 98 92 64 From tfheen at varnish-software.com Mon Jun 4 06:50:13 2012 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Mon, 4 Jun 2012 08:50:13 +0200 Subject: Changing the Varnish Date header to UTC In-Reply-To: References: Message-ID: <20120604065013.GE11089@err.no> ]] Neil Saunders > Hopefully a quick one - Is there any way to get varnish to provide the date > header in UTC instead of GMT? No, it's the required format according to RFC2616. See section 3.3.1: HTTP-date = rfc1123-date | rfc850-date | asctime-date rfc1123-date = wkday "," SP date1 SP time SP "GMT" rfc850-date = weekday "," SP date2 SP time SP "GMT" -- Tollef Fog Heen Technical lead, Varnish Software t: +47 21 98 92 64 From jon.hauge at relationhouse.com Mon Jun 4 08:09:46 2012 From: jon.hauge at relationhouse.com (Jon Hauge) Date: Mon, 4 Jun 2012 10:09:46 +0200 Subject: SV: guro In-Reply-To: <20120604063431.GB11089@err.no> References: <22629abc65c8b84673389a658a2df293@mail.gmail.com> <20120604063431.GB11089@err.no> Message-ID: ]] Jon Hauge >> Why don?t varnish not just serve the cached version? ]] Tollef Fog Heen >Because that's not what grace does. You probably want to look at saint >mode for your use case. What does grace do then? From amoiz.shine at gmail.com Mon Jun 4 08:49:20 2012 From: amoiz.shine at gmail.com (Sharl Jimh Tsin) Date: Mon, 04 Jun 2012 16:49:20 +0800 Subject: Varnish Cache 3.0.3 rc 1 In-Reply-To: <20120604063318.GA11089@err.no> References: <20120604063318.GA11089@err.no> Message-ID: <4FCC7690.5080105@gmail.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 2012?06?04? 14:33, Tollef Fog Heen wrote: > Hi all, > > we're finally getting close to 3.0.3 (it's taken a while!), so if > people could please try it, that would be most appreciated. > > Source can be downloaded from > http://repo.varnish-cache.org/test/varnish-3.0.3-rc1.tar.gz, > packages live in http://repo.varnish-cache.org/test/lucid/, > http://repo.varnish-cache.org/test/squeeze/ and > http://repo.varnish-cache.org/test/rpms/ > > Bugs to trac, please. Feedback most appreciated! > OK,i will try it tonight on my VPS. - -- Best regards, Sharl.Jimh.Tsin (From China **Obviously Taiwan INCLUDED**) Using Gmail? Please read this important notice: http://www.fsf.org/campaigns/jstrap/gmail?10073. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJPzHaHAAoJEEYmNy4jisTjIu4IAIr43Jx+nmZ32IcHjYYctwpC N5tRhCHPZq3bl7Pra0eI6G+21RpntvIuXuXgSZnGlH6/riKIF8K1jelsbeonQpG5 2QCvngietPaf9hHExjJbYFEfqHnT8UWqBu+Rkr9JGAY2qgUnnjzU2qg1VlA+kIll 3a26XBieDq/eaSVFJNyG+q51kp1t9Jw+12MrlSjoGJsS/QtDVKPGfmiXs416rgxM fFiZb0GD5Z/wiH2TS3nFMfu2QcCYBV0pQbD7HxPBRsE1MOKKL6eXL5+eog9Z96bN ntmw7DY2LH7n3kJ2k+OiuHHDfI7LAhQrG87nb6txr0CJjZlt5Q3rQG/N8JDlF9I= =cLQK -----END PGP SIGNATURE----- From arundobriyaliitkgp at gmail.com Mon Jun 4 12:49:48 2012 From: arundobriyaliitkgp at gmail.com (Arun Dobriyal) Date: Mon, 4 Jun 2012 18:19:48 +0530 Subject: Stale-while-Validate for varnish Message-ID: Hi, 1. I want a state-while-revalidate feature(available in some reverse proxies like squid) for my varnish server. In a normal setting with the grace time, the varnish makes the first client to wait while the other clients are served stale content. But I want all clients to be served stale content, while the refereshing of the cache is done at the background asychronously (that way no one is kept at waiting) by varnish. Varnish doesnt support this as of now.. Is there any tweak to achieve it ? 2. I am planning a tweak for the above functionality which is as follows, for the first request, the varnish normally sends the request it to backend where its reported as a cache miss and then the request is validated (i used varnishlog to see that it reports vcl_miss for the first request) .. whereas for the other queued requests, they are just treated as a normal cache hit and delivered immediately.. now, I want to stop the first request to wait for the validation, so I am planning to somehow find if the current request is a stale request and within the grace time, if so, I will serve the stale response for it quickly, and since I want to validate the cache at the backend asynchronously, I willl send this url to an asynchronous queue which will cache this request whenever free.. now my question is, how can I find if the request is the first object and is stale but within the grace time ? If I am able to find this, I can probably create a Stale-while-Validate for varnish.. Thanks Arun -------------- next part -------------- An HTML attachment was scrubbed... URL: From rzuidhof at gmail.com Mon Jun 4 14:57:05 2012 From: rzuidhof at gmail.com (Richard) Date: Mon, 4 Jun 2012 16:57:05 +0200 Subject: Stale-while-Validate for varnish In-Reply-To: References: Message-ID: On Mon, Jun 4, 2012 at 2:49 PM, Arun Dobriyal wrote: > Hi, > > 1. I want a state-while-revalidate feature(available in some reverse > proxies like squid) for my varnish server. In a normal setting with the > grace time, the varnish > > makes the first client to wait while the other clients are served stale > content. But I want all clients to be served stale > > content, while the refereshing of the cache is done at the background > asychronously (that way no one is kept at waiting) by varnish. > > Varnish doesnt support this as of now.. Is there any tweak to achieve it ? > No, this is not supported. I have heard it is bit too complicated to implement and that the current functionality is good usually enough so there is not much request for it. > 2. I am planning a tweak for the above functionality which is as follows, > for the first request, the varnish normally sends the request it to backend > where its reported as a cache miss and then the request is validated (i > used varnishlog to see that it reports vcl_miss for the first request) .. > whereas for the other queued requests, they are just treated as a normal > cache hit and delivered immediately.. > > now, I want to stop the first request to wait for the validation, so I am > planning to somehow find if the current request is a stale request and > within the grace time, if so, I will serve the stale response for it > quickly, and since I want to validate the cache at the backend > asynchronously, I willl send this url to an asynchronous queue which will > cache this request whenever free.. > > now my question is, how can I find if the request is the first object and > is stale but within the grace time ? If I am able to find this, I can > probably create a Stale-while-Validate for varnish.. > You can do a trick like this with three backend definitions. First backend should have very low first_byte_timeout and between_byte_timeout. The second backend is a dummy (unhealthy) backend that will serve stale content if available. The third backend is a normal backend that can deliver requests that are generated by the backend server. The downside of this approach is that the backend server will receive more requests because the first request to the backend server will be cancelled and disregarded by Varnish. And this approach really needs a mechanism to asynchronously refresh content that is served from the stale cache. Otherwise the stale content will become too old. Kind regards, Richard Zuidhof -------------- next part -------------- An HTML attachment was scrubbed... URL: From perbu at varnish-software.com Mon Jun 4 15:10:29 2012 From: perbu at varnish-software.com (Per Buer) Date: Mon, 4 Jun 2012 17:10:29 +0200 Subject: Stale-while-Validate for varnish In-Reply-To: References: Message-ID: Hi, On Mon, Jun 4, 2012 at 4:57 PM, Richard wrote: > On Mon, Jun 4, 2012 at 2:49 PM, Arun Dobriyal < > arundobriyaliitkgp at gmail.com> wrote: > >> Hi, >> >> 1. I want a state-while-revalidate feature(available in some reverse >> proxies like squid) for my varnish server. In a normal setting with the >> grace time, the varnish >> >> makes the first client to wait while the other clients are served stale >> content. But I want all clients to be served stale >> >> content, while the refereshing of the cache is done at the background >> asychronously (that way no one is kept at waiting) by varnish. >> >> Varnish doesnt support this as of now.. Is there any tweak to achieve it ? >> > No, this is not supported. I have heard it is bit too complicated to > implement and that the current functionality is good usually enough so > there is not much request for it. > Current behavior in git master is to use a separate thread against the backend. This will make it easy, or at least much easier, to implement asynchronous requests like these. So, with a fair bit of patience we might implement something like this after the next major release. -- Per Buer Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer *Varnish makes websites fly!* Whitepapers | Video | Twitter -------------- next part -------------- An HTML attachment was scrubbed... URL: From victor.cheng at paperg.com Tue Jun 5 00:17:50 2012 From: victor.cheng at paperg.com (Victor Cheng) Date: Mon, 4 Jun 2012 17:17:50 -0700 Subject: Possible to cache a page for grace mode despite a TTL of 0? Message-ID: I'm not sure if this improper usage of Varnish or not, but I'd like to do the following for high availability purposes: I have a page that's dynamic in nature, but I want to be able to serve a cached copy of it if the backend servers happen to go down. Is it possible to do this while setting the beresp.ttl to 0? This seems to prevent grace mode and works fine if I set it to 1 (as well as if I set the relevant grace mode params), but showing people a cached page for even one second won't work right now because each load should save new data to a database. If the backend servers go down, it's okay if we aren't able to save the data -- we want to at least be able to display a cached copy of the page. That said, perhaps Varnish isn't meant for this -- any suggestions? Thanks! -Victor -------------- next part -------------- An HTML attachment was scrubbed... URL: From victor.cheng at paperg.com Tue Jun 5 18:50:16 2012 From: victor.cheng at paperg.com (Victor Cheng) Date: Tue, 5 Jun 2012 11:50:16 -0700 Subject: Possible to cache a page for grace mode despite a TTL of 0? In-Reply-To: References: Message-ID: Hi all, I was able to resolve this issue by taking a better look at the request flow here: https://www.varnish-software.com/static/book/VCL_Basics.html In case anyone runs into this issue, you just have to modify vcl_hit so that you pass if the backend is healthy (and if it's the particular request you want to be dynamic except when the backend goes down) Thanks, Victor On Mon, Jun 4, 2012 at 5:17 PM, Victor Cheng wrote: > I'm not sure if this improper usage of Varnish or not, but I'd like to do > the following for high availability purposes: > > I have a page that's dynamic in nature, but I want to be able to serve a > cached copy of it if the backend servers happen to go down. > > Is it possible to do this while setting the beresp.ttl to 0? This seems to > prevent grace mode and works fine if I set it to 1 (as well as if I set the > relevant grace mode params), but showing people a cached page for even one > second won't work right now because each load should save new data to a > database. If the backend servers go down, it's okay if we aren't able to > save the data -- we want to at least be able to display a cached copy of > the page. > > That said, perhaps Varnish isn't meant for this -- any suggestions? > > Thanks! > -Victor > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tfheen at varnish-software.com Wed Jun 6 06:38:15 2012 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Wed, 6 Jun 2012 08:38:15 +0200 Subject: SV: guro In-Reply-To: References: <22629abc65c8b84673389a658a2df293@mail.gmail.com> <20120604063431.GB11089@err.no> Message-ID: <20120606063815.GA3974@err.no> ]] Jon Hauge > ]] Jon Hauge > > >> Why don?t varnish not just serve the cached version? > > ]] Tollef Fog Heen > >Because that's not what grace does. You probably want to look at saint > >mode for your use case. > > What does grace do then? Serves the old object while we're busy fetching a new one, thereby preventing thread pileups. -- Tollef Fog Heen Technical lead, Varnish Software t: +47 21 98 92 64 From arundobriyaliitkgp at gmail.com Mon Jun 4 10:07:08 2012 From: arundobriyaliitkgp at gmail.com (Arun Dobriyal) Date: Mon, 4 Jun 2012 15:37:08 +0530 Subject: Stale while revalidate in Varnish Message-ID: Hi, I want a state-while-revalidate feature(available in some reverse proxies like squid) for my varnish server. In a normal setting with the grace time, the varnish makes the first client to wait while the other clients are served stale content. But I want all users to be served stale content, while the refereshing of the cache is done at the background asychronously (that way no one is kept at waiting) by varnish. Is somehow this possible in varnish ? Thanks AD -------------- next part -------------- An HTML attachment was scrubbed... URL: From yao.yang at autonavi.com Tue Jun 5 16:26:08 2012 From: yao.yang at autonavi.com (=?gb2312?B?0e7Sog==?=) Date: Tue, 5 Jun 2012 16:26:08 +0000 Subject: =?gb2312?B?VmFybmlzaCBoaWdoIHRocmVhZCBwcm9ibGVto6xhc2tpbmcgZm9yIGhlbHA=?= =?gb2312?B?o6E=?= Message-ID: <2012060600260055464367@autonavi.com> Hello?I have used Varnish3.0.2 for my work several months?It works fine until yesterday?here is some data statistics. I couldn't find some error, how to trace the error with tools which can be easy used. I need your help, thanks! I run varnish with default run parameters. /usr/local/varnish/sbin/varnishd -f /usr/local/varnish/etc/varnish/default.vcl -s malloc,36G -T 127.0.0.1:2000 -a 0.0.0.0:80 thread statistics of this week [cid:_Foxmail.0 at 9747951E-1276-4891-AD2C-B97EC26F4784] this is I used Varnishstat command for this log, you can see high thread below : client_conn 3004906465 149.74 Client connections accepted client_drop 15826 0.00 Connection dropped, no sess/wrk client_req 7248344267 361.19 Client requests received cache_hit 6890494918 343.36 Cache hits cache_hitpass 237554 0.01 Cache hits for pass cache_miss 14569361 0.73 Cache misses backend_conn 2376506 0.12 Backend conn. success backend_unhealthy 0 0.00 Backend conn. not attempted backend_busy 0 0.00 Backend conn. too many backend_fail 287 0.00 Backend conn. failures backend_reuse 355143346 17.70 Backend conn. reuses backend_toolate 220621 0.01 Backend conn. was closed backend_recycle 355381547 17.71 Backend conn. recycles backend_retry 1789 0.00 Backend conn. retry fetch_head 532 0.00 Fetch head fetch_length 127549059 6.36 Fetch with Length fetch_chunked 74737573 3.72 Fetch chunked fetch_eof 0 0.00 Fetch EOF fetch_bad 0 0.00 Fetch had bad headers fetch_close 398480 0.02 Fetch wanted close fetch_oldhttp 0 0.00 Fetch pre HTTP/1.1 closed fetch_zero 0 0.00 Fetch zero len fetch_failed 33 0.00 Fetch failed fetch_1xx 0 0.00 Fetch no body (1xx) fetch_204 0 0.00 Fetch no body (204) fetch_304 154791965 7.71 Fetch no body (304) n_sess_mem 100000 . N struct sess_mem n_sess 573047 . N struct sess n_object 52928 . N struct object n_vampireobject 0 . N unresurrected objects n_objectcore 53843 . N struct objectcore n_objecthead 53997 . N struct objecthead n_waitinglist 263224 . N struct waitinglist n_vbc 3 . N struct vbc n_wrk 919 . N worker threads n_wrk_create 207942 0.01 N worker threads created n_wrk_failed 0 0.00 N worker threads not created n_wrk_max 56590 0.00 N worker threads limited n_wrk_lqueue 0 0.00 work request queue length n_wrk_queued 2321505 0.12 N queued work requests n_wrk_drop 16055 0.00 N dropped work requests n_backend 2 . N backends n_expired 14516344 . N expired objects n_lru_nuked 0 . N LRU nuked objects n_lru_moved 1228756392 . N LRU moved objects losthdr 245059 0.01 HTTP header overflows n_objsendfile 0 0.00 Objects sent with sendfile n_objwrite 5263813707 262.30 Objects sent with write n_objoverflow 0 0.00 Objects overflowing workspace s_sess 3004890148 149.74 Total Sessions s_req 7248344267 361.19 Total Requests s_pipe 57317 0.00 Total pipe s_pass 342909497 17.09 Total pass s_fetch 357477576 17.81 Total fetch s_hdrbytes 2417774736372 120479.68 Total header bytes s_bodybytes 114294635264274 5695394.48 Total body bytes sess_closed 486804044 24.26 Session Closed sess_pipeline 1544047 0.08 Session Pipeline sess_readahead 379428 0.02 Session Read Ahead sess_linger 7012651760 349.45 Session Linger sess_herd 6537770899 325.78 Session herd shm_records 317672910554 15829.90 SHM records shm_writes 23635386981 1177.77 SHM writes shm_flushes 0 0.00 SHM flushes due to overflow shm_cont 401864660 20.03 SHM MTX contention shm_cycles 125523 0.01 SHM cycles through buffer sms_nreq 246351 0.01 SMS allocator requests sms_nobj 0 . SMS outstanding allocations sms_nbytes 0 . SMS outstanding bytes sms_balloc 97270490 . SMS bytes allocated sms_bfree 97270490 . SMS bytes freed backend_req 357478937 17.81 Backend requests made n_vcl 1 0.00 N vcl total n_vcl_avail 1 0.00 N vcl available n_vcl_discard 0 0.00 N vcl discarded n_ban 12 . N total active bans n_ban_add 13 0.00 N new bans added n_ban_retire 1 0.00 N old bans deleted n_ban_obj_test 51233 0.00 N objects tested n_ban_re_test 175003 0.01 N regexps tested against n_ban_dups 5 0.00 N duplicate bans removed hcb_nolock 6894873923 343.58 HCB Lookups without lock hcb_lock 12193302 0.61 HCB Lookups with lock hcb_insert 12192706 0.61 HCB Inserts esi_errors 0 0.00 ESI parse errors (unlock) esi_warnings 0 0.00 ESI parse warnings (unlock) accept_fail 0 0.00 Accept failures client_drop_late 229 0.00 Connection dropped late uptime 20067905 1.00 Client uptime dir_dns_lookups 0 0.00 DNS director lookups dir_dns_failed 0 0.00 DNS director failed lookups dir_dns_hit 0 0.00 DNS director cached lookups hit dir_dns_cache_full 0 0.00 DNS director full dnscache vmods 0 . Loaded VMODs n_gzip 32483891 1.62 Gzip operations n_gunzip 211871530 10.56 Gunzip operations LCK.sms.creat 1 0.00 Created locks LCK.sms.destroy 0 0.00 Destroyed locks LCK.sms.locks 739053 0.04 Lock Operations LCK.sms.colls 0 0.00 Collisions LCK.smp.creat 0 0.00 Created locks LCK.smp.destroy 0 0.00 Destroyed locks LCK.smp.locks 0 0.00 Lock Operations LCK.smp.colls 0 0.00 Collisions LCK.sma.creat 2 0.00 Created locks LCK.sma.destroy 0 0.00 Destroyed locks LCK.sma.locks 1228521290 61.22 Lock Operations LCK.sma.colls 0 0.00 Collisions LCK.smf.creat 0 0.00 Created locks LCK.smf.destroy 0 0.00 Destroyed locks LCK.smf.locks 0 0.00 Lock Operations LCK.smf.colls 0 0.00 Collisions LCK.hsl.creat 0 0.00 Created locks LCK.hsl.destroy 0 0.00 Destroyed locks LCK.hsl.locks 0 0.00 Lock Operations LCK.hsl.colls 0 0.00 Collisions LCK.hcb.creat 1 0.00 Created locks LCK.hcb.destroy 0 0.00 Destroyed locks LCK.hcb.locks 24444497 1.22 Lock Operations LCK.hcb.colls 0 0.00 Collisions LCK.hcl.creat 0 0.00 Created locks LCK.hcl.destroy 0 0.00 Destroyed locks LCK.hcl.locks 0 0.00 Lock Operations LCK.hcl.colls 0 0.00 Collisions LCK.vcl.creat 1 0.00 Created locks LCK.vcl.destroy 0 0.00 Destroyed locks LCK.vcl.locks 606285 0.03 Lock Operations LCK.vcl.colls 0 0.00 Collisions LCK.stat.creat 1 0.00 Created locks LCK.stat.destroy 0 0.00 Destroyed locks LCK.stat.locks 100000 0.00 Lock Operations LCK.stat.colls 0 0.00 Collisions LCK.sessmem.creat 1 0.00 Created locks LCK.sessmem.destroy 0 0.00 Destroyed locks LCK.sessmem.locks 3005069436 149.75 Lock Operations LCK.sessmem.colls 0 0.00 Collisions LCK.wstat.creat 1 0.00 Created locks LCK.wstat.destroy 0 0.00 Destroyed locks LCK.wstat.locks 3824514186 190.58 Lock Operations LCK.wstat.colls 0 0.00 Collisions LCK.herder.creat 1 0.00 Created locks LCK.herder.destroy 0 0.00 Destroyed locks LCK.herder.locks 2128507 0.11 Lock Operations LCK.herder.colls 0 0.00 Collisions LCK.wq.creat 2 0.00 Created locks LCK.wq.destroy 0 0.00 Destroyed locks LCK.wq.locks 14076329081 701.43 Lock Operations LCK.wq.colls 0 0.00 Collisions LCK.objhdr.creat 12399795 0.62 Created locks LCK.objhdr.destroy 12345878 0.62 Destroyed locks LCK.objhdr.locks 27753000910 1382.95 Lock Operations LCK.objhdr.colls 0 0.00 Collisions LCK.exp.creat 1 0.00 Created locks LCK.exp.destroy 0 0.00 Destroyed locks LCK.exp.locks 49133198 2.45 Lock Operations LCK.exp.colls 0 0.00 Collisions LCK.lru.creat 2 0.00 Created locks LCK.lru.destroy 0 0.00 Destroyed locks LCK.lru.locks 14570647 0.73 Lock Operations LCK.lru.colls 0 0.00 Collisions LCK.cli.creat 1 0.00 Created locks LCK.cli.destroy 0 0.00 Destroyed locks LCK.cli.locks 6689114 0.33 Lock Operations LCK.cli.colls 0 0.00 Collisions LCK.ban.creat 1 0.00 Created locks LCK.ban.destroy 0 0.00 Destroyed locks LCK.ban.locks 1839997818 91.69 Lock Operations LCK.ban.colls 0 0.00 Collisions LCK.vbp.creat 1 0.00 Created locks LCK.vbp.destroy 0 0.00 Destroyed locks LCK.vbp.locks 0 0.00 Lock Operations LCK.vbp.colls 0 0.00 Collisions LCK.vbe.creat 1 0.00 Created locks LCK.vbe.destroy 0 0.00 Destroyed locks LCK.vbe.locks 4754121 0.24 Lock Operations LCK.vbe.colls 0 0.00 Collisions LCK.backend.creat 2 0.00 Created locks LCK.backend.destroy 0 0.00 Destroyed locks LCK.backend.locks 720311744 35.89 Lock Operations LCK.backend.colls 0 0.00 Collisions SMA.s0.nreq 29419933 1.47 Allocator requests SMA.s0.nobj 107799 . Outstanding allocations SMA.s0.nbytes 1652918591 . Outstanding bytes SMA.s0.balloc 1249441221543 . Bytes allocated SMA.s0.bfree 1247788302952 . Bytes free SMA.Transient.nreq 531092059 26.46 Allocator requests SMA.Transient.nobj 0 . Outstanding allocations SMA.Transient.nbytes 0 . Outstanding bytes SMA.Transient.balloc13406920173393 . Bytes allocated SMA.Transient.bfree 13406920173393 . Bytes free VBE.default(127.0.0.1,,8080).vcls 1 . VCL references VBE.default(127.0.0.1,,8080).happy 0 . Happy health probes VBE.api_mapabc(127.0.0.1,,8080).vcls 1 . VCL references VBE.api_mapabc(127.0.0.1,,8080).happy 0 . Happy health probes ________________________________ [cid:image001.jpg at 01CC55C0.2961D160] ???-??? ?? Tel: 010-84107622 Mobile: 15011304105 MSN:spider_sea at 163.com Mail: yao.yang at autonavi.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Catch8D21.jpg Type: image/jpeg Size: 119210 bytes Desc: Catch8D21.jpg URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 2016_image002(12-22-11-54-48).jpg Type: image/jpeg Size: 2016 bytes Desc: 2016_image002(12-22-11-54-48).jpg URL: From luxiaochuang at yahoo.com.cn Wed Jun 6 06:47:10 2012 From: luxiaochuang at yahoo.com.cn (luchuang) Date: Wed, 6 Jun 2012 14:47:10 +0800 (CST) Subject: =?utf-8?B?6L2s5Y+R77yaIFdlbGNvbWUgdG8gdGhlICJ2YXJuaXNoLW1pc2MiIG1haWxp?= =?utf-8?B?bmcgbGlzdA==?= In-Reply-To: References: Message-ID: <1338965230.57860.YahooMailNeo@web15704.mail.cnb.yahoo.com> hello. ----- ????? ----- ???? "varnish-misc-request at varnish-cache.org" ???? luxiaochuang at yahoo.com.cn ????? 2012?6?6?, ???, ?? 2:39 ??: Welcome to the "varnish-misc" mailing list Welcome to the varnish-misc at varnish-cache.org mailing list! To post to this list, send your email to: ? varnish-misc at varnish-cache.org General information about the mailing list is at: ? https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc If you ever want to unsubscribe or change your options (eg, switch to or from digest mode, change your password, etc.), visit your subscription page at: ? https://www.varnish-cache.org/lists/mailman/options/varnish-misc/luxiaochuang%40yahoo.com.cn You can also make such adjustments via email by sending a message to: ? varnish-misc-request at varnish-cache.org with the word `help' in the subject or body (don't include the quotes), and you will get back a message with instructions. You must know your password to change your options (including changing the password, itself) or to unsubscribe.? It is: ? tony123456 Normally, Mailman will remind you of your varnish-cache.org mailing list passwords once every month, although you can disable this if you prefer.? This reminder will also include instructions on how to unsubscribe or change your account options.? There is also a button on your options page that will email your current password to you. -------------- next part -------------- An HTML attachment was scrubbed... URL: From perbu at varnish-software.com Wed Jun 6 09:21:49 2012 From: perbu at varnish-software.com (Per Buer) Date: Wed, 6 Jun 2012 11:21:49 +0200 Subject: Stale while revalidate in Varnish In-Reply-To: References: Message-ID: Hi Arun. On Mon, Jun 4, 2012 at 12:07 PM, Arun Dobriyal wrote: > Hi, > > I want a state-while-revalidate feature(available in some reverse > proxies like squid) for my varnish server. In a normal setting with the > grace time, the varnish > > makes the first client to wait while the other clients are served stale > content. But I want all users to be served stale > > content, while the refereshing of the cache is done at the background > asychronously (that way no one is kept at waiting) by varnish. > > Is somehow this possible in varnish ? > No. Sorry. Google it if you want more details. -- Per Buer Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer *Varnish makes websites fly!* Whitepapers | Video | Twitter -------------- next part -------------- An HTML attachment was scrubbed... URL: From Shirley.Harris at DOS.MyFlorida.com Wed Jun 6 19:49:53 2012 From: Shirley.Harris at DOS.MyFlorida.com (Harris, Shirley A.) Date: Wed, 6 Jun 2012 15:49:53 -0400 Subject: Varnish access log client IP's the same Message-ID: Hello, Everything was working then about a month ago the Varnish access log started showing 2.2.2.2 and 2.2.2.3 as almost all the client IP addresses. The client IP as Varnish sees it is now set up to show in Apache's access log, but the problem seems to be that Varnish still sees the IP addresses as 2.2.2.2 or 2.2.2.3. What could be causing this? This is Varnishncsa: 2.2.2.2 - - [06/Jun/2012:14:37:49 -0400] "GET http://XXXXX HTTP/1.1" 200 5793 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" 2.2.2.3 - - [06/Jun/2012:14:37:50 -0400] "GET http://XXXXX HTTP/1.1" 200 5171 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)" 2.2.2.3 - - [06/Jun/2012:14:37:50 -0400] "GET http://XXXXX HTTP/1.1" 200 7584 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" 2.2.2.3 - - [06/Jun/2012:14:37:50 -0400] "GET http://XXXXX HTTP/1.1" 200 5171 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)" 2.2.2.3 - - [06/Jun/2012:14:37:50 -0400] "GET http://XXXXX HTTP/1.1" 200 7584 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" This is the Apache access log with the Varnish client IP: 2.2.2.2, 2.2.2.2 - - [06/Jun/2012:14:34:33 -0400] "GET /Collections/folklife/audio.cfm HTTP/1.1" 301 29 "-" "Mozilla/5.0 (Windows NT 5.1; CT2228257_3.13.0.6; rv:12.0) Gecko/20100101 Firefox/12.0" 2.2.2.2, 2.2.2.2 - - [06/Jun/2012:14:34:34 -0400] "GET /audio/ HTTP/1.1" 200 8158 "-" "Mozilla/5.0 (Windows NT 5.1; CT2228257_3.13.0.6; rv:12.0) Gecko/20100101 Firefox/12.0" 2.2.2.3, 2.2.2.3 - - [06/Jun/2012:14:34:34 -0400] "POST /Collections/folklife/search_adv_002.cfm HTTP/1.1" 301 3 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)" 2.2.2.2, 2.2.2.2 - - [06/Jun/2012:14:34:34 -0400] "POST /Collections/SpanishLandGrants/index.cfm HTTP/1.1" 301 3 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)" 2.2.2.3, 2.2.2.3 - - [06/Jun/2012:14:34:34 -0400] "GET /collections/folklife/ HTTP/1.1" 200 19388 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)" Apache LogFormat: LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined Default.vcl: sub vcl_recv { remove req.http.X-Forwarded-For; set req.http.X-Forwarded-For = client.ip; if (req.restarts == 0) { if (req.http.x-forwarded-for) { set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip; } else { set req.http.X-Forwarded-For = client.ip; } } ... Thanks, Shirley Florida is headed in the right direction! View Florida's Jobs Growth Chart: http://www.flgov.com/photoview/jobcreationchart.jpg The Department of State is leading the commemoration of Florida's 500th anniversary in 2013. For more information, please go to www.fla500.com. The Department of State is committed to excellence. Please take our Customer Satisfaction Survey: http://survey.dos.state.fl.us/index.aspx?email=Shirley.Harris at DOS.MyFlorida.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: jobgrowthchart_sm.jpg Type: image/jpeg Size: 4544 bytes Desc: jobgrowthchart_sm.jpg URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: vivafl45.png Type: image/png Size: 1592 bytes Desc: vivafl45.png URL: From richard at academia.edu Wed Jun 6 22:29:07 2012 From: richard at academia.edu (Richard Price) Date: Wed, 6 Jun 2012 15:29:07 -0700 Subject: Varnish consultants Message-ID: Hi all, I'm the founder of Academia.edu, which is a platform for academics to share research papers. We have been using Varnish for about 18 months, and have recently been experiencing some issues where items are falling out of the cache after a few minutes, despite a ttl of 30 days, leading to a 50% hit rate. We've established that the pages are not being nuked by Varnish, nor are we purging them. We're looking to have an experienced eye look over our Varnish config to see if anything stands out. Does anyone on this list know of any Varnish consultants who we might be able to hire on an hourly basis? I've looked into the price plan at https://www.varnish-software.com/products-services/price-plan, but it looks like the minimum spend there is 3,300 euros, and we are just looking for someone to scan our config for an hour or two initially, so an hourly consulting rate would suit us better. Any recommendations or leads would be very helpful. We are based in San Francisco. Many thanks, Richard -- Richard Price Founder and CEO, Academia.edu * * *Academia.edu is a platform for academics to share research papers. Its mission is to accelerate the world's research. It is transforming the way scientists communicate.* * * *Read more on our blog , or watch a speechI gave on the future of science. * o: 415 829 2341 a: http://oxford.academia.edu/RichardPrice t: http://twitter.com/richardprice100 l: http://linkedin.com/in/RichardPrice -------------- next part -------------- An HTML attachment was scrubbed... URL: From richard at zuidhof.nl Thu Jun 7 07:59:31 2012 From: richard at zuidhof.nl (Richard Zuidhof) Date: Thu, 7 Jun 2012 09:59:31 +0200 Subject: Varnish Cache 3.0.3 rc 1 In-Reply-To: <20120604063318.GA11089@err.no> References: <20120604063318.GA11089@err.no> Message-ID: On Mon, Jun 4, 2012 at 8:33 AM, Tollef Fog Heen wrote: > Hi all, > > we're finally getting close to 3.0.3 (it's taken a while!), so if people > could please try it, that would be most appreciated. > Is there a change log available for this new version? https://www.varnish-cache.org/trac/browser/doc/changes.rst Richard -------------- next part -------------- An HTML attachment was scrubbed... URL: From ruben at varnish-software.com Thu Jun 7 09:01:28 2012 From: ruben at varnish-software.com (=?UTF-8?Q?Rub=C3=A9n_Romero?=) Date: Thu, 7 Jun 2012 11:01:28 +0200 Subject: Varnish consultants In-Reply-To: References: Message-ID: Hi Richard, What you need, sounds like we offer in the Diagnostic Package, which probably also has a price point closer to your budget (it is listed on the price plan). For more information and pricing, see: https://www.varnish-software.com/products-services/diagnostics-varnish-cache Also, Varnish Training is always a good option so you can fix such issues yourselves. We have a classroom session planned for San Francisco at the end of the month (after Velocity). Details and registration: https://www.varnish-software.com/products-services/training/varnish-administration-course Should you have any further questions, feel free to get in touch with me directly. On Thu, Jun 7, 2012 at 12:29 AM, Richard Price wrote: > Hi all, > > I'm the founder of Academia.edu, which is a platform for academics to > share research papers. We have been using Varnish for about 18 months, and > have recently been experiencing some issues where items are falling out of > the cache after a few minutes, despite a ttl of 30 days, leading to a 50% > hit rate. > > We've established that the pages are not being nuked by Varnish, nor are > we purging them. We're looking to have an experienced eye look over our > Varnish config to see if anything stands out. > > Does anyone on this list know of any Varnish consultants who we might be > able to hire on an hourly basis? I've looked into the price plan at > https://www.varnish-software.com/products-services/price-plan, but it > looks like the minimum spend there is 3,300 euros, and we are just looking > for someone to scan our config for an hour or two initially, so an hourly > consulting rate would suit us better. > > Any recommendations or leads would be very helpful. We are based in San > Francisco. Many thanks, > > Richard > > > > -- > Richard Price > Founder and CEO, Academia.edu > * > * > *Academia.edu is a platform for academics to share research papers. Its > mission is to accelerate the world's research. It is transforming the way > scientists communicate.* > * > * > *Read more on our blog , or watch a speechI gave on the future of science. > * > > o: 415 829 2341 > a: http://oxford.academia.edu/RichardPrice > t: http://twitter.com/richardprice100 > l: http://linkedin.com/in/RichardPrice > All the best, -- Rub?n Romero, Global Sales Executive Phone: +47 21 98 92 62 / Mobile: +47 959 64 088 / Skype&Twitter: ruben_varnish *Varnish makes websites fly!* Whitepapers | Video | Twitter | LinkedIn -------------- next part -------------- An HTML attachment was scrubbed... URL: From pwlazy at gmail.com Thu Jun 7 11:15:29 2012 From: pwlazy at gmail.com (=?GB2312?B?xe3OsA==?=) Date: Thu, 7 Jun 2012 19:15:29 +0800 Subject: how to get varnish thread dump Message-ID: hi: i use the "pstack" to get varnish child process thread dump, it's ok, but after i used "pstack" , the varnish child process disappear, it seems the varnish parent process killed the child process and forked a new child process so pstack will cause the child process killed, then is there a better way to get the thread dump thanks for answer -------------- next part -------------- An HTML attachment was scrubbed... URL: From arundobriyaliitkgp at gmail.com Fri Jun 8 06:17:24 2012 From: arundobriyaliitkgp at gmail.com (Arun Dobriyal) Date: Fri, 8 Jun 2012 11:47:24 +0530 Subject: Behavior of varnish with paging Message-ID: Hi, I am aspiring to use my disk for varnish caching. Since the OS does page caching in memory also, this would serve as a two layer of cache (one in memory and one in disk). Now my concern is to use almost whole of RAM for page caching (assuming that there is no swap space in the OS). How to configure so that I use my memory for caching specifically for varnish. Can I control the space used by varnish in RAM. Secondly, I am curious what will happen if I refresh the content of the cache in the disk and hence the content of the disk cache changes . If I use this, the content of the cache will be modified in the disk, would it reflect in the RAM as well (I want cache consistency across Page cache also, Does OS do it automatically, and if it does, will it write the content in the disk and delete the entry in the RAM)? And whether varnish will update the content in Swap also (In case I plan to use the swap space)? Thanks, Arun -------------- next part -------------- An HTML attachment was scrubbed... URL: From javier at casares.org Fri Jun 8 06:21:58 2012 From: javier at casares.org (Javier Casares) Date: Fri, 8 Jun 2012 08:21:58 +0200 Subject: Multiple storage with multiple backends In-Reply-To: References: Message-ID: As a sample: SYSCONFIG: -s memoria=malloc,128M \ -s disco=file,/var/lib/varnish/storage.bin,512M" VCL: if (req.url ~ "\.(7z|bmp|bz2|csv|doc|docx|epub|gif|gz|ico|jpeg|jpg|key|log|mp3|odp|ods|odt|ogg|pdf|png|pps|ppt|pptx|ps|psd|rar|rtf|svg|tar|tbz|tgz|tif|tiff|vcf|xls|xlsx|zip)(\?.*)?$") { set beresp.storage = "disco"; } elsif (req.url ~ "\.(css|js|sql|txt|xml)(\?.*)?$") { if(storage.memoria.free_space > 16MB) { set beresp.storage = "memoria"; } else { set beresp.storage = "disco"; } } else { if(storage.memoria.free_space > 16MB) { set beresp.storage = "memoria"; } else { set beresp.storage = "disco"; } } Javier Casares http://javiercasares.com/ From xhrfc at yahoo.com Mon Jun 11 17:42:25 2012 From: xhrfc at yahoo.com (X.H.R.F.C.) Date: Mon, 11 Jun 2012 10:42:25 -0700 (PDT) Subject: varnish exclude parts Message-ID: <1339436545.56268.YahooMailNeo@web43503.mail.sp1.yahoo.com> Hello, Is there any way to exclude some webpage parts from cache? For example i have an website where i auth users trough openid, provider steam and after i finish auth i show some avatar and a logout button - i use a cookie to log them in. The problem is that i have a lot of pages that will never change in the future, but this avatar is on every website page. How can i cache the entyre page, except this avatar portion? If you can provide some vcl code, i'll appreciate, i'm a complete beginner to varnish. Best Regards, -------------- next part -------------- An HTML attachment was scrubbed... URL: From javier at casares.org Mon Jun 11 17:50:24 2012 From: javier at casares.org (Javier Casares) Date: Mon, 11 Jun 2012 19:50:24 +0200 Subject: varnish exclude parts In-Reply-To: <1339436545.56268.YahooMailNeo@web43503.mail.sp1.yahoo.com> References: <1339436545.56268.YahooMailNeo@web43503.mail.sp1.yahoo.com> Message-ID: You must use ESI system https://www.varnish-cache.org/trac/wiki/ESIfeatures El lunes, 11 de junio de 2012, X.H.R.F.C. escribi?: > Hello, > > Is there any way to exclude some webpage parts from cache? > For example i have an website where i auth users trough openid, provider > steam and after i finish auth i show some avatar and a logout button - i > use a cookie to log them in. > The problem is that i have a lot of pages that will never change in the > future, but this avatar is on every website page. How can i cache the > entyre page, except this avatar portion? > If you can provide some vcl code, i'll appreciate, i'm a complete beginner > to varnish. > > > Best Regards, > > -- Javier Casares http://javiercasares.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From hugo.cisneiros at gmail.com Mon Jun 11 21:44:36 2012 From: hugo.cisneiros at gmail.com (Hugo Cisneiros (Eitch)) Date: Mon, 11 Jun 2012 18:44:36 -0300 Subject: Using varnish to cache remote content In-Reply-To: References: Message-ID: On Tue, May 29, 2012 at 6:51 AM, Ganesh Raut wrote: > How do i cache the external ?content in varnish...e.g js,css,images etc.... > > can you please send us example of varnish files. Hello, This information is the very basic of Varnish. The default configuration already is an example of this, and you can learn a lot more from the documentation pages: https://www.varnish-cache.org/docs/3.0/tutorial/index.html https://www.varnish-cache.org/docs/3.0/reference/vcl.html -- []'s Hugo www.devin.com.br From hugo.cisneiros at gmail.com Mon Jun 11 22:03:09 2012 From: hugo.cisneiros at gmail.com (Hugo Cisneiros (Eitch)) Date: Mon, 11 Jun 2012 19:03:09 -0300 Subject: Multi-language (cookie) site configuration In-Reply-To: References: Message-ID: On Wed, May 23, 2012 at 6:41 AM, wrote: > I'm trying to setup varnish for a multi-language vbulletin forum. The language is set through a cookie (bblanguageid). > > My current configuration is: http://pastebin.com/wCKZUcTX By the way, I don't think you can redefine VCL functions multiple times as you do in this configuration. It will consider only the first definition. > Now I've tried to cache a separate version if the bblanguageid=2 cookie is present: > > sub vcl_hash { > ? ? if (req.http.cookie ~ "bblanguageid=2") { > ? ? ? ? ? ?set req.hash += "deDE"; > ? ? } > } I do this very frequently (but with Varnish 3.0.2). The idea is the same, but I use hash_data to add the custom string "deDE". Here, the only thing you really need is this configuration. With this, varnish must be generating a different cache entry for this request with the cookie. To know exactly what is happening, you must use 'varnishlog' command and look at what varnish is doing, if it's adding the custom hash, and so on. -- []'s Hugo www.devin.com.br From hugo.cisneiros at gmail.com Mon Jun 11 22:09:22 2012 From: hugo.cisneiros at gmail.com (Hugo Cisneiros (Eitch)) Date: Mon, 11 Jun 2012 19:09:22 -0300 Subject: Varnish access log client IP's the same In-Reply-To: References: Message-ID: On Wed, Jun 6, 2012 at 4:49 PM, Harris, Shirley A. wrote: > Hello, > > Everything was working then about a month ago the Varnish access log started showing 2.2.2.2 and 2.2.2.3 as almost all the client IP addresses. > > The client IP as Varnish sees it is now set up to show in Apache?s access log, but the problem seems to be that Varnish still sees the IP addresses as 2.2.2.2 or 2.2.2.3. Since varnish gets the client IP from the network, probably this is not a varnish problem, but a networking one. Some other host must be getting the requests and passing to varnish, including the X-Forwarded-For HTTP header. I'd go with network analysis tool to discover what is happening (on linux, tcpdump or iptraf for example). -- []'s Hugo www.devin.com.br From travis.crowder at spechal.com Tue Jun 12 01:23:11 2012 From: travis.crowder at spechal.com (Travis Crowder) Date: Mon, 11 Jun 2012 20:23:11 -0500 Subject: Using varnish to cache remote content In-Reply-To: References: Message-ID: <4FD699FF.508@spechal.com> On 6/11/2012 4:44 PM, Hugo Cisneiros (Eitch) wrote: > On Tue, May 29, 2012 at 6:51 AM, Ganesh Raut wrote: >> How do i cache the external content in varnish...e.g js,css,images etc.... >> >> can you please send us example of varnish files. > Hello, > > This information is the very basic of Varnish. The default > configuration already is an example of this, and you can learn a lot > more from the documentation pages: > > https://www.varnish-cache.org/docs/3.0/tutorial/index.html > https://www.varnish-cache.org/docs/3.0/reference/vcl.html Short answer: You can't without a lot of effort. See https://www.varnish-cache.org/lists/pipermail/varnish-misc/2011-March/005952.html From cdevecchi at gmail.com Wed Jun 13 18:26:00 2012 From: cdevecchi at gmail.com (Claudio Devecchi) Date: Wed, 13 Jun 2012 15:26:00 -0300 Subject: Conditional Backend Request with stale objects Message-ID: Hello Everybody, I'm using varnish 3.0.2 and I need to do the following: For example: - I have a object with ttl 5m, after this time, when I receive a new request, it pass to backend and backend returns a 200 code, putting the object in cache again. My problem is: - I want to varnish to check in backend if the object was modified or not - if modified ok, I get a 200 and everything continues - if not modified, I want varnish to check the backend and receive a 304, keeping the object on cache like before. Somebody could help me with this? Tks Claudio -------------- next part -------------- An HTML attachment was scrubbed... URL: From paulo at aliancaproject.com Wed Jun 13 18:33:44 2012 From: paulo at aliancaproject.com (Paulo Paracatu) Date: Wed, 13 Jun 2012 15:33:44 -0300 Subject: Conditional Backend Request with stale objects In-Reply-To: References: Message-ID: There is a patch to do this: Backend Conditional Requests https://www.varnish-cache.org/trac/wiki/BackendConditionalRequests But I don't really know the status of it. I don't see anyone talking about it here since few months ago. 2012/6/13 Claudio Devecchi > Hello Everybody, > > I'm using varnish 3.0.2 and I need to do the following: > > For example: > > - I have a object with ttl 5m, after this time, when I receive a new > request, it pass to backend and backend returns a 200 code, putting the > object in cache again. > > My problem is: > > - I want to varnish to check in backend if the object was modified or not > - if modified ok, I get a 200 and everything continues > - if not modified, I want varnish to check the backend and receive a > 304, keeping the object on cache like before. > > Somebody could help me with this? > > Tks > Claudio > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From abraham.sustaita at buscacorp.com Wed Jun 13 20:24:42 2012 From: abraham.sustaita at buscacorp.com (Abraham Cruz Sustaita) Date: Wed, 13 Jun 2012 13:24:42 -0700 Subject: Varnish returns 503 error. Message-ID: I'm using varnish to cache the content of my websites. It is working as it supposed, but there is a problem. Randomly it returns an 503 error, it is really strange, since the app servers are ok and the load is under .8, also the database server its ok. Here is part of my configuration: backend app05 { .host = "app05.site.com"; .port = "80"; .connect_timeout = 0.7s; .first_byte_timeout = 30s; .between_bytes_timeout = 30s; .probe = { .url = "/"; .interval = 5s; .timeout = 1s; .window = 5; .threshold = 3; } } director app_director round-robin { { .backend = app01; } { .backend = app02; } { .backend = app03; } { .backend = app04; } { .backend = app05; } } sub vcl_fetch { # remove all cookies # unset beresp.http.set-cookie; # cache for 12 hours # set beresp.ttl = 2h; # Don't allow static files to set cookies. if (req.url ~ "(?i)\.(png|gif|jpeg|jpg|ico|swf|css|js|html|htm|mp4|flv)(\?[a-z0-9]+)?$") { unset beresp.http.set-cookie; set beresp.ttl = 12h; } else { set beresp.ttl = 30m; } # If the backend server doesn't return properly, don't send another connection to it # for 60s and try another backend via restart. # # https://www.varnish-cache.org/docs/trunk/tutorial/handling_misbehaving_servers.html # -- if(beresp.status == 500 || beresp.status == 503) { set beresp.saintmode = 5m; if (req.request != "POST") { * return(restart); # Here is the configuration to restart the connection if a server returns a 503 error, but it is not working* } else { error 500 "Failed"; } } # Allow items to be stale if needed. set beresp.grace = 1h; } -------------- next part -------------- An HTML attachment was scrubbed... URL: From rabeloo at gmail.com Wed Jun 13 20:41:17 2012 From: rabeloo at gmail.com (Raphael R. O.) Date: Wed, 13 Jun 2012 17:41:17 -0300 Subject: Automatic varnish cache Message-ID: Hi, There's any method on varnish to create the page cache before the first access on the website ? For example: I'm purge all url cached with varnishadm, after that, varnish can be access the URL and create the cache automatically, without that the first client access the page for create cache? *Thks!* *rabelo* -------------- next part -------------- An HTML attachment was scrubbed... URL: From hugo.cisneiros at gmail.com Wed Jun 13 20:44:35 2012 From: hugo.cisneiros at gmail.com (Hugo Cisneiros (Eitch)) Date: Wed, 13 Jun 2012 17:44:35 -0300 Subject: Varnish returns 503 error. In-Reply-To: References: Message-ID: On Wed, Jun 13, 2012 at 5:24 PM, Abraham Cruz Sustaita wrote: > I'm using varnish to cache the content of my websites. It is working as it > supposed, but there is a problem. Randomly it returns an 503 error, it is > really strange, since the app servers are ok and the load is under .8, also > the database server its ok. Here is part of my configuration: On Varnish, 503 means an error on the backend connection. Usually this happens when the web server takes more time than "first_byte_timeout" to reply to a request. But also, if for some reason the backend can't connect before the "connect_timeout" delay, it'll give you a 503. The proper way to debug these kind of things is to use varnishlog on everything related to these 503 errors. In your configuration, why you say that restart did not work? It goes to "error 500 failed"? What happens exactly? -- []'s Hugo www.devin.com.br From abraham.sustaita at buscacorp.com Wed Jun 13 20:51:14 2012 From: abraham.sustaita at buscacorp.com (Abraham Cruz Sustaita) Date: Wed, 13 Jun 2012 13:51:14 -0700 Subject: Varnish returns 503 error. In-Reply-To: References: Message-ID: I understand varnish have to restart the connection and try again in another app server (I have 5), but it allways returns the 503 error, so I understand that means the rule is being ignored. I have made this change to the vcl file: backend app05 { .host = "app05.site.com"; .port = "80"; .connect_timeout = *1.5s; # I do change this from 0.7 to 1.5 but it is still serving 503 errors...* .first_byte_timeout = 30s; .between_bytes_timeout = 30s; .probe = { .url = "/"; .interval = 5s; .timeout = 1s; .window = 5; .threshold = 3; } } Do I have to increase also the first_byte_timeout? Abraham Cruz Sustaita Leader Development Mail: abraham.sustaita at buscacorp.com Ph: +52 (664) 200.2213 F: +52 (664) 200.2234 M: +52 (664) 123.0312 Calle Brasil No. 8471. Col. Ju?rez, Tijuana Baja California, CP 22040 BuscaCorp.com | LevelUp.com | MetaTube.com | SanDiegoRed.com | Tarreo.com This e-mail message is intended only for the personal use of the recipient(s) named above. This message may be an attorney-client communication and as such privileged and confidential. If you are not an intended recipient,you may not review, copy or distribute this message. If you have received this communication in error, please notify us immediately by e-mail and delete the original message. Este mensaje es para uso personal de(los) destinatario(s) a cuyos nombres se dirige. Este mensaje puede ser comunicaci?n abogado-cliente y por tanto es privilegiado y confidencial. Si usted no es la persona a quien se intent? enviar este mensaje, no debe revisarlo, copiarlo o distribuirlo en forma alguna. Si ha recibido este mensaje por error, por favor notif?quelo inmediatamente por email y destruya el mensaje original. 2012/6/13 Hugo Cisneiros (Eitch) > On Wed, Jun 13, 2012 at 5:24 PM, Abraham Cruz Sustaita > wrote: > > I'm using varnish to cache the content of my websites. It is working as > it > > supposed, but there is a problem. Randomly it returns an 503 error, it is > > really strange, since the app servers are ok and the load is under .8, > also > > the database server its ok. Here is part of my configuration: > > On Varnish, 503 means an error on the backend connection. Usually this > happens when the web server takes more time than "first_byte_timeout" > to reply to a request. But also, if for some reason the backend can't > connect before the "connect_timeout" delay, it'll give you a 503. > > The proper way to debug these kind of things is to use varnishlog on > everything related to these 503 errors. > > In your configuration, why you say that restart did not work? It goes > to "error 500 failed"? What happens exactly? > > -- > []'s > Hugo > www.devin.com.br > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hugo.cisneiros at gmail.com Wed Jun 13 20:51:56 2012 From: hugo.cisneiros at gmail.com (Hugo Cisneiros (Eitch)) Date: Wed, 13 Jun 2012 17:51:56 -0300 Subject: Conditional Backend Request with stale objects In-Reply-To: References: Message-ID: On Wed, Jun 13, 2012 at 3:33 PM, Paulo Paracatu wrote: > There is a patch to do this: Backend Conditional Requests > https://www.varnish-cache.org/trac/wiki/BackendConditionalRequests > > But I don't?really?know the status of it. I don't see anyone talking about > it here since few months ago. I have tried this patch in the past and it worked very well. The only reason I didn't use in production yet is because it does not really help me because here Apache is generating the pages with some Server Side Includes. I did some tests and if the content of a SSI changes, Apache will not update the original etag since the static html never changed. But this is a limitation of Apache and not the patch. Give it a try! :) -- []'s Hugo www.devin.com.br From hugo.cisneiros at gmail.com Wed Jun 13 20:54:16 2012 From: hugo.cisneiros at gmail.com (Hugo Cisneiros (Eitch)) Date: Wed, 13 Jun 2012 17:54:16 -0300 Subject: Automatic varnish cache In-Reply-To: References: Message-ID: On Wed, Jun 13, 2012 at 5:41 PM, Raphael R. O. wrote: > There's any method on varnish to create the page cache before the first > access on the website ? The good way to do this is to create a bot/crawler with URLs and run it after the purges. This is called 'cache warming" and you can look at documentation / mailing list archive and you'll find much information. -- []'s Hugo www.devin.com.br From hugo.cisneiros at gmail.com Wed Jun 13 22:02:53 2012 From: hugo.cisneiros at gmail.com (Hugo Cisneiros (Eitch)) Date: Wed, 13 Jun 2012 19:02:53 -0300 Subject: Varnish returns 503 error. In-Reply-To: References: Message-ID: On Wed, Jun 13, 2012 at 5:51 PM, Abraham Cruz Sustaita wrote: > I understand varnish have to restart the connection and try again in another app server (I have 5), but it allways returns the 503 error, so I understand that means the rule is being ignored. The only way to be sure what is happening is to log details from the requests using varnishlog. The restart function will reset the request and it goes back to vcl_recv as if it is a new request. But since you're using saintmode, maybe varnish isn't really retrying to fetch this particular URL from the backend. You can do a if (req.restarts > 3) { set beresp.saintmode = 5m; } and it will restart at least 2 times before inserting the item on saintmode. > I have made this change to the vcl file: [...] > Do I have to increase also the first_byte_timeout? You need to increase only if the request isn't getting reply data in the first 30 seconds :) You can check this with varnishlog and varnishncsa to be sure. -- []'s Hugo www.devin.com.br From darvin.denmian at gmail.com Wed Jun 13 22:26:35 2012 From: darvin.denmian at gmail.com (Darvin Denmian) Date: Wed, 13 Jun 2012 19:26:35 -0300 Subject: Hiding internal backend requests Message-ID: Hi, That is my scenario: 1) Varnish (172.16.217.131:80), receives a request from a client, i.e: http://172.16.217.131:80/a.png 2) Request is forwarded to the Default Backend (127.0.0.1:8000) 3) Default backend receive the request and process it 4) That processing results in a new URL, i.e: http://172.16.217.132:80/a.png (**As you can see the IP has changed) 5) 172.16.217.132:80 is another backend in Varnish's config file 6) The new URL points to a resource that should be provided by Varnish (that resource generally is an image) My problem is: The client needs to execute 2 GETs to obtain the image. My question: How can I configure varnish to internally receive the response from the first backend(127.0.0.1:8000), and fetch data from the second backend (172.16.217.132:80), and after that, send the data to the client? Thanks. From abraham.sustaita at buscacorp.com Wed Jun 13 22:54:26 2012 From: abraham.sustaita at buscacorp.com (Abraham Cruz Sustaita) Date: Wed, 13 Jun 2012 15:54:26 -0700 Subject: Varnish returns 503 error. In-Reply-To: References: Message-ID: Thanks a lot. I have made the change you suggested me and it seems its working now. I'm still checking but I think its working very fine right now. Thanks a lot again. 2012/6/13 Hugo Cisneiros (Eitch) > On Wed, Jun 13, 2012 at 5:51 PM, Abraham Cruz Sustaita > wrote: > > I understand varnish have to restart the connection and try again in > another app server (I have 5), but it allways returns the 503 error, so I > understand that means the rule is being ignored. > > The only way to be sure what is happening is to log details from the > requests using varnishlog. > > The restart function will reset the request and it goes back to > vcl_recv as if it is a new request. But since you're using saintmode, > maybe varnish isn't really retrying to fetch this particular URL from > the backend. You can do a > > if (req.restarts > 3) { > set beresp.saintmode = 5m; > } > > and it will restart at least 2 times before inserting the item on > saintmode. > > > I have made this change to the vcl file: > [...] > > Do I have to increase also the first_byte_timeout? > > You need to increase only if the request isn't getting reply data in > the first 30 seconds :) You can check this with varnishlog and > varnishncsa to be sure. > > -- > []'s > Hugo > www.devin.com.br > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jc.bedier at gmail.com Thu Jun 14 07:44:00 2012 From: jc.bedier at gmail.com (Jean-Christian BEDIER) Date: Thu, 14 Jun 2012 09:44:00 +0200 Subject: Varnish returns 503 error. In-Reply-To: References: Message-ID: Hello, I have the same problem too, on random request i read it in my varnishlog: 11 FetchError c http first read error: -1 104 (Connection reset by peer) 13 BackendClose b farm1 11 VCL_call c error 11 VCL_return c deliver 11 VCL_call c deliver 11 VCL_return c deliver 11 TxProtocol c HTTP/1.1 11 TxStatus c 503 Did you have the same Abraham ? Regards, On Thu, Jun 14, 2012 at 12:54 AM, Abraham Cruz Sustaita wrote: > Thanks a lot. I have made the change you suggested me and it seems its > working now. > > I'm still checking but I think its working very fine right now. > > Thanks a lot again. > > > 2012/6/13 Hugo Cisneiros (Eitch) >> >> On Wed, Jun 13, 2012 at 5:51 PM, Abraham Cruz Sustaita >> >> wrote: >> > I understand varnish have to restart the connection and try again in >> > another app server (I have 5), but it allways returns the 503 error, so I >> > understand that means the rule is being ignored. >> >> The only way to be sure what is happening is to log details from the >> requests using varnishlog. >> >> The restart function will reset the request and it goes back to >> vcl_recv as if it is a new request. But since you're using saintmode, >> maybe varnish isn't really retrying to fetch this particular URL from >> the backend. You can do a >> >> if (req.restarts > 3) { >> ?set beresp.saintmode = 5m; >> } >> >> and it will restart at least 2 times before inserting the item on >> saintmode. >> >> > I have made this change to the vcl file: >> [...] >> > Do I have to increase also the first_byte_timeout? >> >> You need to increase only if the request isn't getting reply data in >> the first 30 seconds :) You can check this with varnishlog and >> varnishncsa to be sure. >> >> -- >> []'s >> Hugo >> www.devin.com.br >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc From cdevecchi at gmail.com Thu Jun 14 18:04:08 2012 From: cdevecchi at gmail.com (Claudio Devecchi) Date: Thu, 14 Jun 2012 15:04:08 -0300 Subject: Conditional Backend Request with stale objects In-Reply-To: References: Message-ID: Hi... I compiled the version following the instructions on https://www.varnish-cache.org/trac/wiki/BackendConditionalRequests, and using varnishlog I already see the headers If-Modified-Since and If-None-Match. But when the obj.ttl expires, varnish gets a 200 from backend and I was expecting 304. Somebody knows what could be wrong? Tks On Wed, Jun 13, 2012 at 5:51 PM, Hugo Cisneiros (Eitch) < hugo.cisneiros at gmail.com> wrote: > On Wed, Jun 13, 2012 at 3:33 PM, Paulo Paracatu > wrote: > > There is a patch to do this: Backend Conditional Requests > > https://www.varnish-cache.org/trac/wiki/BackendConditionalRequests > > > > But I don't really know the status of it. I don't see anyone talking > about > > it here since few months ago. > > I have tried this patch in the past and it worked very well. > > The only reason I didn't use in production yet is because it does not > really help me because here Apache is generating the pages with some > Server Side Includes. I did some tests and if the content of a SSI > changes, Apache will not update the original etag since the static > html never changed. > > But this is a limitation of Apache and not the patch. Give it a try! :) > > -- > []'s > Hugo > www.devin.com.br > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From abraham.sustaita at buscacorp.com Thu Jun 14 18:34:45 2012 From: abraham.sustaita at buscacorp.com (Abraham Cruz Sustaita) Date: Thu, 14 Jun 2012 11:34:45 -0700 Subject: Varnish returns 503 error. In-Reply-To: References: Message-ID: Actually I'm very newbbie in this varnish stuff, so I don't figure yet how to use the log at varnishlog, but I did what they suggested me. My vcl configuration file its this and it is working now. I have marked strong the changes I did in order to fix this: backend app01 { .host = "app01.site.com"; .port = "80"; *.connect_timeout = 1.5s;* * .first_byte_timeout = 45s;* .between_bytes_timeout = 30s; .probe = { .url = "/"; .interval = 5s; .timeout = 1s; .window = 5; .threshold = 3; } } backend app02 { .host = "app02.site.com"; .port = "80"; *.connect_timeout = 1.5s;* * .first_byte_timeout = 45s;* .between_bytes_timeout = 30s; .probe = { .url = "/"; .interval = 5s; .timeout = 1s; .window = 5; .threshold = 3; } } backend app03 { .host = "app03.site.com"; .port = "80"; *.connect_timeout = 1.5s;* * .first_byte_timeout = 45s;* .between_bytes_timeout = 30s; .probe = { .url = "/"; .interval = 5s; .timeout = 1s; .window = 5; .threshold = 3; } } backend app04 { .host = "app04.site.com"; .port = "80"; *.connect_timeout = 1.5s;* * .first_byte_timeout = 45s;* .between_bytes_timeout = 30s; .probe = { .url = "/"; .interval = 5s; .timeout = 1s; .window = 5; .threshold = 3; } } backend app05 { .host = "app05.site.com"; .port = "80"; *.connect_timeout = 1.5s;* * .first_byte_timeout = 45s;* .between_bytes_timeout = 30s; .probe = { .url = "/"; .interval = 5s; .timeout = 1s; .window = 5; .threshold = 3; } } director app_director round-robin { { .backend = app01; } { .backend = app02; } { .backend = app03; } { .backend = app04; } { .backend = app05; } } # Amazon S3 Backend # -- backend s3 { .host = "site.s3-website-us-east-1.amazonaws.com"; .port = "80"; } #acl internal { # "localhost"; # "127.0.0.1"; #} acl purge { "localhost"; "192.168.11.0"/24; } sub vcl_recv { if (req.request == "PURGE") { if (!client.ip ~ purge) { error 405 "Not allowed."; } ban("req.url = " + req.url); return (lookup); } # Normalize the host header # -- if (req.http.Host ~ "(?i)^(www.)?site.com") { set req.http.host = "www.site.com"; } set req.backend = app_director; if (req.http.Host == "s3.site-files.site.com" && req.url ~ "\.(flv|mp4|jpg)$") { unset req.http.cookie; unset req.http.cache-control; unset req.http.pragma; unset req.http.expires; unset req.http.etag; unset req.http.X-Forwarded-For; set req.backend = s3; set req.http.host = "site.s3-website-us-east-1.amazonaws.com"; return (lookup); } # Use anonymous, cached pages if all backends are down. if (! req.backend.healthy) { unset req.http.Cookie; set req.grace = 1h; } else { set req.grace = 30s; } # Add the X-Forwarded-For header # -- if (req.restarts == 0) { if (req.http.x-forwarded-for) { set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip; } else { set req.http.X-Forwarded-For = client.ip; } } # Sanely handle the request based on the type # -- if (req.request != "GET" && req.request != "HEAD" && req.request != "PUT" && req.request != "POST" && req.request != "TRACE" && req.request != "OPTIONS" && req.request != "DELETE") { /* Non-RFC2616 or CONNECT which is weird. */ return (pipe); } # Request by post needs no cache # -- if (req.request == "POST") { return (pass); } # Admin needs no cache # -- if (req.http.Host ~ "(?i)^(admin.)?site.com") { return (pass); } # Always cache the following file types for all users. if (req.url ~ "(?i)\.(png|gif|jpeg|jpg|ico|swf|flv|avi|mov|wmv|css|js|html|htm)(\?[a-z0-9]+)?$") { unset req.http.Cookie; return (lookup); } # Login and logout needs no cache # -- if (req.url ~ "^/usuario/salir" || req.url ~ "^/usuario/redirectWindow" || req.url ~ "^/usuario/al") { return (pass); } # Requests to /no-cache/ needs no cache # -- if (req.request == "GET" && req.url ~ "^/no-cache") { return (pass); } if (req.request == "GET" && req.url ~ "^/rss/siterss") { return (pass); } # subdomain.site.com and other.com/foro/ needs no cache # -- if (req.http.Host == "subdomain.site.com") { return (pass); } if (req.http.Host == "www.other.com" && req.url ~ "^/foro/" && req.http.Cookie ~ "(?i)(somecookie|othercookie)") { return (pass); } # Not sure what could match here, but it's in the default.vcl # -- if (req.request != "GET" && req.request != "HEAD") { /* We only deal with GET and HEAD by default */ return (pass); } # Handle compression correctly. Different browsers send different # "Accept-Encoding" headers, even though they mostly all support the same # compression mechanisms. By consolidating these compression headers into # a consistent format, we can reduce the size of the cache and get more hits.= # @see: http:// varnish.projects.linpro.no/wiki/FAQ/Compression if (req.http.Accept-Encoding) { if (req.http.Accept-Encoding ~ "gzip") { # If the browser supports it, we'll use gzip. set req.http.Accept-Encoding = "gzip"; } else if (req.http.Accept-Encoding ~ "deflate") { # Next, try deflate if it is supported. set req.http.Accept-Encoding = "deflate"; } else { # Unknown algorithm. Remove it and send unencoded. unset req.http.Accept-Encoding; } } # Cache all requests by default, overriding the standard Varnish behavior if (req.request == "GET" || req.request == "HEAD") { /* We only deal with GET and HEAD by default */ #unset req.http.Cookie; return (lookup); } # If there's a cookie left aft this point do not cache # or is this easier to keep in the cookie deletion above? # -- if (req.http.Authorization || req.http.Cookie) { /* Not cacheable by default */ return (pass); } # If we get here, try the cache # -- return (lookup); } sub vcl_pipe { return (pipe); } sub vcl_pass { return (pass); } sub vcl_hash { # Include cookie in cache hash. # This check is unnecessary because we already pass on all cookies. #if (req.http.Cookie) { # #set req.hash += req.http.Cookie; # hash_data(req.http.Cookie); #} } sub vcl_hit { if (req.request == "PURGE") { purge; error 200 "Purged"; } } sub vcl_miss { if (req.request == "PURGE") { purge; error 200 "Purged"; } } sub vcl_fetch { # S3 # -- if (req.http.Host == "s3.site-files.site.com" && req.url ~ "\.(flv|mp4|jpg)$") { remove beresp.http.x-amz-id-2; remove beresp.http.x-amz-request-id; set beresp.ttl = 1w; set beresp.grace = 30s; if (beresp.status != 200) { return (error); } } # remove all cookies # unset beresp.http.set-cookie; # cache for 12 hours # set beresp.ttl = 2h; # Don't allow static files to set cookies. if (req.url ~ "(?i)\.(png|gif|jpeg|jpg|ico|swf|css|js|html|htm|mp4|flv)(\?[a-z0-9]+)?$") { unset beresp.http.set-cookie; set beresp.ttl = 12h; } else { set beresp.ttl = 30m; } # If the backend server doesn't return properly, don't send another connection to it # for 60s and try another backend via restart. # # https://www.varnish-cache.org/docs/trunk/tutorial/handling_misbehaving_servers.html # -- if(beresp.status == 500 || beresp.status == 503) { *if (req.restarts > 3) {* * set beresp.saintmode = 5m;* * }* if (req.request != "POST") { return(restart); } else { error 500 "Failed"; } } # Allow items to be stale if needed. set beresp.grace = 1h; } sub vcl_deliver { # if (resp.http.magicmarker) { # unset resp.http.magicmarker; # set resp.http.age = "0"; # } if (obj.hits > 0) { set resp.http.X-Cache = "HIT"; } else { set resp.http.X-Cache = "MISS"; } return (deliver); } # In the event of an error, show friendlier messages. sub vcl_error { # Otherwise redirect to the homepage, which will likely be in the cache. set obj.http.Content-Type = "text/html; charset=utf-8"; synthetic {" Pagina no disponible temporalmente

Oops! Houston tenemos problemas!

La pagina que estas intentando ver por el momento no esta disponible..

Intenta recargando la pagina o puedes visitar Nuestra pagina principal

(Error "} + obj.status + " " + obj.response + {")
"}; return (deliver); } sub vcl_init { return (ok); } sub vcl_fini { return (ok); } 2012/6/14 Jean-Christian BEDIER > Hello, > > I have the same problem too, on random request i read it in my varnishlog: > > 11 FetchError c http first read error: -1 104 (Connection reset by > peer) > 13 BackendClose b farm1 > 11 VCL_call c error > 11 VCL_return c deliver > 11 VCL_call c deliver > 11 VCL_return c deliver > 11 TxProtocol c HTTP/1.1 > 11 TxStatus c 503 > > Did you have the same Abraham ? > > Regards, > > > > On Thu, Jun 14, 2012 at 12:54 AM, Abraham Cruz Sustaita > wrote: > > Thanks a lot. I have made the change you suggested me and it seems its > > working now. > > > > I'm still checking but I think its working very fine right now. > > > > Thanks a lot again. > > > > > > 2012/6/13 Hugo Cisneiros (Eitch) > >> > >> On Wed, Jun 13, 2012 at 5:51 PM, Abraham Cruz Sustaita > >> > >> wrote: > >> > I understand varnish have to restart the connection and try again in > >> > another app server (I have 5), but it allways returns the 503 error, > so I > >> > understand that means the rule is being ignored. > >> > >> The only way to be sure what is happening is to log details from the > >> requests using varnishlog. > >> > >> The restart function will reset the request and it goes back to > >> vcl_recv as if it is a new request. But since you're using saintmode, > >> maybe varnish isn't really retrying to fetch this particular URL from > >> the backend. You can do a > >> > >> if (req.restarts > 3) { > >> set beresp.saintmode = 5m; > >> } > >> > >> and it will restart at least 2 times before inserting the item on > >> saintmode. > >> > >> > I have made this change to the vcl file: > >> [...] > >> > Do I have to increase also the first_byte_timeout? > >> > >> You need to increase only if the request isn't getting reply data in > >> the first 30 seconds :) You can check this with varnishlog and > >> varnishncsa to be sure. > >> > >> -- > >> []'s > >> Hugo > >> www.devin.com.br > >> > >> _______________________________________________ > >> varnish-misc mailing list > >> varnish-misc at varnish-cache.org > >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > > > > > > > _______________________________________________ > > varnish-misc mailing list > > varnish-misc at varnish-cache.org > > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From abraham.sustaita at buscacorp.com Fri Jun 15 00:34:25 2012 From: abraham.sustaita at buscacorp.com (Abraham Cruz Sustaita) Date: Thu, 14 Jun 2012 17:34:25 -0700 Subject: Clear all the cache of a domain Message-ID: I use this class PHP (http://varnish-cache.org/wiki/CLI) to clean the Varnish cache, I only do this: $var01Obj = new Varnish('192.168.11.41', 6082, 3); $var01Obj->set_auth('XXXXXXXXXXXX' . "\n"); $ret = $var01Obj->connect(); if ($ret['status']) { print_r($var01Obj->purge_url($url)); } else { print_r($ret); } where $url is an url of the form ^/news/$ but I have a problem and I need to delete all the cache of a site, let say, I have www.site.com/news/ and the news are www.site.com/news/1/title/ and I have 5,000 news, but also I have other site www.other-site.com with the same friendly urls, so I only need to clean the cache for www.site.com not for www.other-site.com. If I make $url = '^www.site.com.*&'; will it work? I know ^/news.*$ works since I have some rules like that (for comments for example), but I have no idea if it will work using the full domain -------------- next part -------------- An HTML attachment was scrubbed... URL: From pprocacci at datapipe.com Fri Jun 15 01:14:33 2012 From: pprocacci at datapipe.com (Paul A. Procacci) Date: Thu, 14 Jun 2012 20:14:33 -0500 Subject: Clear all the cache of a domain In-Reply-To: References: Message-ID: <20120615011433.GB92856@nat.myhome> Not sure of the capabilities of the class you are using, but you can ban url's based on req.http.host and the passed url. ban req.http.host == "example.com" && req.url ~ "/news/.*" ^^ for example. ~Paul On Thu, Jun 14, 2012 at 05:34:25PM -0700, Abraham Cruz Sustaita wrote: > I use this class PHP ([1]http://varnish-cache.org/wiki/CLI) to clean > the Varnish cache, I only do this: > > $var01Obj = new Varnish('192.168.11.41', 6082, 3); > $var01Obj->set_auth('XXXXXXXXXXXX' . "\n"); > $ret = $var01Obj->connect(); > if ($ret['status']) { > print_r($var01Obj->purge_url($url)); > } else { > print_r($ret); > } > > where $url is an url of the form ^/news/$ but I have a problem and I > need to delete all the cache of a site, let say, I have > [2]www.site.com/news/ and the news are [3]www.site.com/news/1/title/ > and I have 5,000 news, but also I have other site [4]www.other-site.com > with the same friendly urls, so I only need to clean the cache for > [5]www.site.com not for [6]www.other-site.com. If I make > > $url = '^www.site.com.*&'; > > will it work? > > I know ^/news.*$ works since I have some rules like that (for comments > for example), but I have no idea if it will work using the full domain > > References > > 1. http://varnish-cache.org/wiki/CLI > 2. http://www.site.com/news/ > 3. http://www.site.com/news/1/title/ > 4. http://www.other-site.com/ > 5. http://www.site.com/ > 6. http://www.other-site.com/ > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc ________________________________ This message may contain confidential or privileged information. If you are not the intended recipient, please advise us immediately and delete this message. See http://www.datapipe.com/legal/email_disclaimer/ for further information on confidentiality and the risks of non-secure electronic communication. If you cannot access these links, please notify us by reply message and we will send the contents to you. From xhrfc at yahoo.com Fri Jun 15 09:34:11 2012 From: xhrfc at yahoo.com (X.H.R.F.C.) Date: Fri, 15 Jun 2012 02:34:11 -0700 (PDT) Subject: Error 503 Service Unavailable / Service Unavailable / Guru Meditation Message-ID: <1339752851.83512.YahooMailNeo@web43509.mail.sp1.yahoo.com> Hello, I have a wired problem with varnish, i get "Error 503 Service Unavailable / Service Unavailable / Guru Meditation". This is not happening all the time! If i restart varnishd all things are fine for random 5-10 mins. I checked varnishlog and i get : "9 FetchError?? c no backend connection". Strange thing because my 3 nginx servers are up and running just fine, checked error log on all of them and there is no problem :| I searched on google and from many results i'v read prev message from Abraham Cruz (https://www.varnish-cache.org/lists/pipermail/varnish-misc/2012-June/022127.html), i tryed to setup like he has in there but i receive same error. So here are some pastes from varnishlog and my vcls. Please let me know if i have some config erorr or something, i can't figure out what is wrong in here and why i get this. I'm using Varnish 3.0.0~beta1-1~squeeze2 on Debian 6 machine. # ---- start varnishlog 9 SessionOpen? c xxx.xxx.xxx.xxx 49940 website.com:80 ??? 9 ReqStart???? c xxx.xxx.xxx.xxx 49940 905690731 ??? 9 RxRequest??? c GET ??? 9 RxURL??????? c /auth/login ??? 9 RxProtocol?? c HTTP/1.1 ??? 9 RxHeader???? c Host: website.com ??? 9 RxHeader???? c User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20100101 Firefox/12.0 ??? 9 RxHeader???? c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 ??? 9 RxHeader???? c Accept-Language: en-us,en;q=0.5 ??? 9 RxHeader???? c Accept-Encoding: gzip, deflate ??? 9 RxHeader???? c Connection: keep-alive ??? 9 RxHeader???? c Cookie: __utma=163271977.1649200775.1339623409.1339678556.1339688677.4; AutoLogin=aa0cb1ba10b4f859d5ab6eab2fe8cb6f4be48867%7E7b40ad672393bf7738c6d065101f44283396ef11; __utmz=163271977.1339661812.2.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __un ??? 9 VCL_call???? c recv lookup ??? 9 VCL_call???? c hash ??? 9 Hash???????? c /auth/login ??? 9 Hash???????? c website.com ??? 9 VCL_return?? c hash ??? 9 VCL_call???? c miss fetch ??? 9 FetchError?? c no backend connection ??? 9 VCL_call???? c error deliver ??? 9 VCL_call???? c deliver deliver ??? 9 TxProtocol?? c HTTP/1.1 ??? 9 TxStatus???? c 503 ??? 9 TxResponse?? c Service Unavailable ??? 9 TxHeader???? c Server: Varnish ??? 9 TxHeader???? c Content-Type: text/html; charset=utf-8 ??? 9 TxHeader???? c Retry-After: 5 ??? 9 TxHeader???? c Content-Length: 418 ??? 9 TxHeader???? c Accept-Ranges: bytes ??? 9 TxHeader???? c Date: Fri, 15 Jun 2012 09:20:01 GMT ??? 9 TxHeader???? c X-Varnish: 905690731 ??? 9 TxHeader???? c Age: 0 ??? 9 TxHeader???? c Via: 1.1 varnish ??? 9 TxHeader???? c Connection: close ??? 9 Length?????? c 418 ??? 9 ReqEnd?????? c 905690731 1339752001.003274202 1339752001.003641129 0.000065327 0.000314951 0.000051975 ??? 9 SessionClose c error ??? 9 StatSess???? c xxx.xxx.xxx.xxx 49940 0 1 1 0 0 0 256 418 ??? 0 CLI????????? - Rd ping ??? 0 CLI????????? - Wr 200 19 PONG 1339752003 1.0 ??? 0 CLI????????? - Rd ping ??? 0 CLI????????? - Wr 200 19 PONG 1339752006 1.0 ??? 0 CLI????????? - Rd ping ??? 0 CLI????????? - Wr 200 19 PONG 1339752009 1.0 ??? 0 CLI????????? - Rd ping # ---- end varnish log # ---- start /etc/defaults/varnish START=yes NFILES=131072 MEMLOCK=82000 DAEMON_OPTS="-a website.com:80 \ ???????????? -T drr0003:6082 \ ???????????? -f /etc/varnish/website.vcl \ ???????????? -S /etc/varnish/secret \ ???????????? -s malloc,2G" # ---- end /etc/defaults/varnish # ---- start /etc/varnish/website.vcl # server 1 backend drr0001 { ??? .host = "drr0001"; ??? .port = "80"; ??? .probe = { ??????????????? .url = "/"; ??????????????? .interval = 10m; ??????????????? .timeout = 500s; ??????????????? .window = 5; ??????????????? .threshold = 3; ? } } # server 2 backend drr0002 { ??? .host = "drr0002"; ??? .port = "80"; ??? .probe = { ??????????????? .url = "/"; ??????????????? .interval = 10m; ??????????????? .timeout = 500s; ??????????????? .window = 5; ??????????????? .threshold = 3; ? } } # server 3 backend drr0003 { ??? .host = "drr0003"; ??? .port = "80"; ??? .probe = { ??????????????? .url = "/"; ??????????????? .interval = 10m; ??????????????? .timeout = 500s; ??????????????? .window = 5; ??????????????? .threshold = 3; ? } } # Round-Robin load balance director website round-robin { ??????? { ??????????????? .backend = drr0001; ??????? } ??????? { ??????????????? .backend = drr0002; ??????? } ??????? { ??????????????? .backend = drr0003; ??????? } } # ACL to allow cache Purge acl purge { ??? "drr0001"; ??? "drr0002"; ??? "drr0003"; } # Receive sub vcl_recv { ??? # check acl purge at the top and purges ? ??? if (req.request == "PURGE") ??? ??? { ??? ??? ??? ??? if (!client.ip ~ purge) { ????? ??? ??? ??? error 404 "Not Found"; ??? ??? ??? } ??? ??? ??? return (lookup); ? ??? } ? ??? if (! req.http.Host) ??? { ??? ??? error 404 "Your query need a host header !"; ??? } ??? if (req.http.Host ~ "website.com") ??? { ??? ??? set req.backend = website; ??? ??? include "/etc/varnish/website-esi-recv.vcl"; ??? } ??? else ??? { ??? ??? error 404 "No Varnish configuration for your host header."; ??? } } ## FETCH sub vcl_fetch { ??? if (req.http.Host ~ "test.website.com") ??? { ??? ??? include "/etc/varnish/website-esi-fetch.vcl"; ??? } ??? else ??? { ??? ??? return (deliver); ??? } } # HIT sub vcl_hit { ??? if (req.request == "PURGE") { ??? ??? purge; ??? ??? error 200 "Purged."; ??? } } # MISS sub vcl_miss { ??? if (req.request == "PURGE") { ??? ??? purge; ??? ??? error 200 "Purged."; ??? } } ## DELIVER sub vcl_deliver { ??? # Secure the header ??? # remove resp.http.Via; ??? # remove resp.http.X-Varnish; ??? # remove resp.http.Server; ??? # remove resp.http.X-Powered-By; } # ---- end /etc/varnish/website.vcl # ---- start /etc/varnish/website-esi-recv.vcl # Compatiblity with Apache log remove req.http.X-Forwarded-For; set??? req.http.X-Forwarded-For = client.ip; # Post requests will not be cached if (req.request == "POST") { ??? return (pass); } # Normalize encoding/compression if (req.http.Accept-Encoding) { ??? if (req.http.Accept-Encoding ~ "gzip") ??? { ??? ??? set req.http.Accept-Encoding = "gzip";???? ??? } ??? elsif (req.http.Accept-Encoding ~ "deflate") ??? { ??? ??? set req.http.Accept-Encoding = "deflate"; ??? } ??? else ??? { ??? ??? remove req.http.Accept-Encoding; ??? } } # Serve the page unset req.http.vary; if (req.url ~ "\.(jpeg|jpg|png|gif|ico|js|css|txt|gz|zip|lzma|bz2|tgz|tbz|html|htm)$") { ??? # Remove the cookie and make the request static ??? unset req.http.cookie; ??? return (lookup); } # Try to lookup in the cache return (lookup); # Cookie ? Not cacheable by default if (req.http.Authorization || req.http.Cookie) { ??? return (pass); } # ---- end /etc/varnish/website-esi-recv.vcl # ---- start /etc/varnish/website-esi-fetch.vcl # Enable ESI set beresp.do_esi = true; set beresp.ttl = 0s; # Defining TTL for specific URLs if (req.url ~ "\.(ico|js|css)$") { ??? set beresp.ttl = 24h; } if (req.url ~ "\/other1\/([0-9]+$)\/") { ??? set beresp.ttl = 10s; } if (req.url ~ "\/other2\/([0-9]+$)\/") { ??? set beresp.ttl = 24h; } return (deliver); # ---- end /etc/varnish/website-esi-fetch.vcl -------------- next part -------------- An HTML attachment was scrubbed... URL: From tobias.eichelbroenner at lamp-solutions.de Fri Jun 15 10:13:35 2012 From: tobias.eichelbroenner at lamp-solutions.de (=?ISO-8859-1?Q?Tobias_Eichelbr=F6nner?=) Date: Fri, 15 Jun 2012 12:13:35 +0200 Subject: Error 503 Service Unavailable / Service Unavailable / Guru Meditation In-Reply-To: <1339752851.83512.YahooMailNeo@web43509.mail.sp1.yahoo.com> References: <1339752851.83512.YahooMailNeo@web43509.mail.sp1.yahoo.com> Message-ID: <4FDB0ACF.8010405@lamp-solutions.de> Hello, > I have a wired problem with varnish, i get "Error 503 Service > Unavailable / Service Unavailable / Guru Meditation". This is not > happening all the time! If i restart varnishd all things are fine for > random 5-10 mins. you are probing url / on host:drr000x maybe this url is not configured, or the host unknown. Have you tried "lynx http://drr0001/" on the varnish machine? Sincerely From abraham.sustaita at buscacorp.com Fri Jun 15 16:49:49 2012 From: abraham.sustaita at buscacorp.com (Abraham Cruz Sustaita) Date: Fri, 15 Jun 2012 09:49:49 -0700 Subject: Error 503 Service Unavailable / Service Unavailable / Guru Meditation In-Reply-To: <4FDB0ACF.8010405@lamp-solutions.de> References: <1339752851.83512.YahooMailNeo@web43509.mail.sp1.yahoo.com> <4FDB0ACF.8010405@lamp-solutions.de> Message-ID: I could fix the error by configuring my vcl like this (I have marked bold the changes I had to make in order to fix it): backend app01 { .host = "app01.site.com"; .port = "80"; *.connect_timeout = 1.5s;* * .first_byte_timeout = 45s;* .between_bytes_timeout = 30s; .probe = { .url = "/"; .interval = 5s; .timeout = 1s; .window = 5; .threshold = 3; } } backend app02 { .host = "app02.site.com"; .port = "80"; *.connect_timeout = 1.5s;* * .first_byte_timeout = 45s;* .between_bytes_timeout = 30s; .probe = { .url = "/"; .interval = 5s; .timeout = 1s; .window = 5; .threshold = 3; } } backend app03 { .host = "app03.site.com"; .port = "80"; *.connect_timeout = 1.5s;* * .first_byte_timeout = 45s;* .between_bytes_timeout = 30s; .probe = { .url = "/"; .interval = 5s; .timeout = 1s; .window = 5; .threshold = 3; } } backend app04 { .host = "app04.site.com"; .port = "80"; *.connect_timeout = 1.5s;* * .first_byte_timeout = 45s;* .between_bytes_timeout = 30s; .probe = { .url = "/"; .interval = 5s; .timeout = 1s; .window = 5; .threshold = 3; } } backend app05 { .host = "app05.site.com"; .port = "80"; *.connect_timeout = 1.5s;* * .first_byte_timeout = 45s;* .between_bytes_timeout = 30s; .probe = { .url = "/"; .interval = 5s; .timeout = 1s; .window = 5; .threshold = 3; } } director app_director round-robin { { .backend = app01; } { .backend = app02; } { .backend = app03; } { .backend = app04; } { .backend = app05; } } # Amazon S3 Backend # -- backend s3 { .host = "site.s3-website-us-east-1.amazonaws.com"; .port = "80"; } #acl internal { # "localhost"; # "127.0.0.1"; #} acl purge { "localhost"; "192.168.11.0"/24; } sub vcl_recv { if (req.request == "PURGE") { if (!client.ip ~ purge) { error 405 "Not allowed."; } ban("req.url = " + req.url); return (lookup); } # Normalize the host header # -- if (req.http.Host ~ "(?i)^(www.)?site.com") { set req.http.host = "www.site.com"; } set req.backend = app_director; if (req.http.Host == "s3.site-files.site.com" && req.url ~ "\.(flv|mp4|jpg)$") { unset req.http.cookie; unset req.http.cache-control; unset req.http.pragma; unset req.http.expires; unset req.http.etag; unset req.http.X-Forwarded-For; set req.backend = s3; set req.http.host = "site.s3-website-us-east-1.amazonaws.com"; return (lookup); } # Use anonymous, cached pages if all backends are down. if (! req.backend.healthy) { unset req.http.Cookie; set req.grace = 1h; } else { set req.grace = 30s; } # Add the X-Forwarded-For header # -- if (req.restarts == 0) { if (req.http.x-forwarded-for) { set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip; } else { set req.http.X-Forwarded-For = client.ip; } } # Sanely handle the request based on the type # -- if (req.request != "GET" && req.request != "HEAD" && req.request != "PUT" && req.request != "POST" && req.request != "TRACE" && req.request != "OPTIONS" && req.request != "DELETE") { /* Non-RFC2616 or CONNECT which is weird. */ return (pipe); } # Request by post needs no cache # -- if (req.request == "POST") { return (pass); } # Admin needs no cache # -- if (req.http.Host ~ "(?i)^(admin.)?site.com") { return (pass); } # Always cache the following file types for all users. if (req.url ~ "(?i)\.(png|gif|jpeg|jpg|ico|swf|flv|avi|mov|wmv|css|js|html|htm)(\?[a-z0-9]+)?$") { unset req.http.Cookie; return (lookup); } # Login and logout needs no cache # -- if (req.url ~ "^/usuario/salir" || req.url ~ "^/usuario/redirectWindow" || req.url ~ "^/usuario/al") { return (pass); } # Requests to /no-cache/ needs no cache # -- if (req.request == "GET" && req.url ~ "^/no-cache") { return (pass); } if (req.request == "GET" && req.url ~ "^/rss/siterss") { return (pass); } # subdomain.site.com and other.com/foro/ needs no cache # -- if (req.http.Host == "subdomain.site.com") { return (pass); } if (req.http.Host == "www.other.com" && req.url ~ "^/foro/" && req.http.Cookie ~ "(?i)(somecookie|othercookie)") { return (pass); } # Not sure what could match here, but it's in the default.vcl # -- if (req.request != "GET" && req.request != "HEAD") { /* We only deal with GET and HEAD by default */ return (pass); } # Handle compression correctly. Different browsers send different # "Accept-Encoding" headers, even though they mostly all support the same # compression mechanisms. By consolidating these compression headers into # a consistent format, we can reduce the size of the cache and get more hits.= # @see: http:// varnish.projects.linpro.no/wiki/FAQ/Compression if (req.http.Accept-Encoding) { if (req.http.Accept-Encoding ~ "gzip") { # If the browser supports it, we'll use gzip. set req.http.Accept-Encoding = "gzip"; } else if (req.http.Accept-Encoding ~ "deflate") { # Next, try deflate if it is supported. set req.http.Accept-Encoding = "deflate"; } else { # Unknown algorithm. Remove it and send unencoded. unset req.http.Accept-Encoding; } } # Cache all requests by default, overriding the standard Varnish behavior if (req.request == "GET" || req.request == "HEAD") { /* We only deal with GET and HEAD by default */ #unset req.http.Cookie; return (lookup); } # If there's a cookie left aft this point do not cache # or is this easier to keep in the cookie deletion above? # -- if (req.http.Authorization || req.http.Cookie) { /* Not cacheable by default */ return (pass); } # If we get here, try the cache # -- return (lookup); } sub vcl_pipe { return (pipe); } sub vcl_pass { return (pass); } sub vcl_hash { # Include cookie in cache hash. # This check is unnecessary because we already pass on all cookies. #if (req.http.Cookie) { # #set req.hash += req.http.Cookie; # hash_data(req.http.Cookie); #} } sub vcl_hit { if (req.request == "PURGE") { purge; error 200 "Purged"; } } sub vcl_miss { if (req.request == "PURGE") { purge; error 200 "Purged"; } } sub vcl_fetch { # S3 # -- if (req.http.Host == "s3.site-files.site.com" && req.url ~ "\.(flv|mp4|jpg)$") { remove beresp.http.x-amz-id-2; remove beresp.http.x-amz-request-id; set beresp.ttl = 1w; set beresp.grace = 30s; if (beresp.status != 200) { return (error); } } # remove all cookies # unset beresp.http.set-cookie; # cache for 12 hours # set beresp.ttl = 2h; # Don't allow static files to set cookies. if (req.url ~ "(?i)\.(png|gif|jpeg|jpg|ico|swf|css|js|html|htm|mp4|flv)(\?[a-z0-9]+)?$") { unset beresp.http.set-cookie; set beresp.ttl = 12h; } else { set beresp.ttl = 30m; } # If the backend server doesn't return properly, don't send another connection to it # for 60s and try another backend via restart. # # https://www.varnish-cache.org/docs/trunk/tutorial/handling_misbehaving_servers.html # -- if(beresp.status == 500 || beresp.status == 503) { *if (req.restarts > 3) {* * set beresp.saintmode = 5m;* * }* if (req.request != "POST") { return(restart); } else { error 500 "Failed"; } } # Allow items to be stale if needed. set beresp.grace = 1h; } sub vcl_deliver { # if (resp.http.magicmarker) { # unset resp.http.magicmarker; # set resp.http.age = "0"; # } if (obj.hits > 0) { set resp.http.X-Cache = "HIT"; } else { set resp.http.X-Cache = "MISS"; } return (deliver); } # In the event of an error, show friendlier messages. sub vcl_error { # Otherwise redirect to the homepage, which will likely be in the cache. set obj.http.Content-Type = "text/html; charset=utf-8"; synthetic {" Pagina no disponible temporalmente

Oops! Houston tenemos problemas!

La pagina que estas intentando ver por el momento no esta disponible..

Intenta recargando la pagina o puedes visitar Nuestra pagina principal

(Error "} + obj.status + " " + obj.response + {")
"}; return (deliver); } sub vcl_init { return (ok); } sub vcl_fini { return (ok); } Abraham Cruz Sustaita Leader Development Mail: abraham.sustaita at buscacorp.com Ph: +52 (664) 200.2213 F: +52 (664) 200.2234 M: +52 (664) 123.0312 Calle Brasil No. 8471. Col. Ju?rez, Tijuana Baja California, CP 22040 BuscaCorp.com | LevelUp.com | MetaTube.com | SanDiegoRed.com | Tarreo.com This e-mail message is intended only for the personal use of the recipient(s) named above. This message may be an attorney-client communication and as such privileged and confidential. If you are not an intended recipient,you may not review, copy or distribute this message. If you have received this communication in error, please notify us immediately by e-mail and delete the original message. Este mensaje es para uso personal de(los) destinatario(s) a cuyos nombres se dirige. Este mensaje puede ser comunicaci?n abogado-cliente y por tanto es privilegiado y confidencial. Si usted no es la persona a quien se intent? enviar este mensaje, no debe revisarlo, copiarlo o distribuirlo en forma alguna. Si ha recibido este mensaje por error, por favor notif?quelo inmediatamente por email y destruya el mensaje original. 2012/6/15 Tobias Eichelbr?nner > Hello, > > > I have a wired problem with varnish, i get "Error 503 Service > > Unavailable / Service Unavailable / Guru Meditation". This is not > > happening all the time! If i restart varnishd all things are fine for > > random 5-10 mins. > > you are probing url / on host:drr000x maybe this url is not configured, > or the host unknown. > > Have you tried "lynx http://drr0001/" on the varnish machine? > > Sincerely > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Shirley.Harris at DOS.MyFlorida.com Mon Jun 18 12:16:10 2012 From: Shirley.Harris at DOS.MyFlorida.com (Harris, Shirley A.) Date: Mon, 18 Jun 2012 08:16:10 -0400 Subject: Varnish access log client IP's the same In-Reply-To: References: Message-ID: Yes, it was a networking problem. Thank you for helping. Florida is headed in the right direction! View Florida's Jobs Growth Chart: http://www.flgov.com/photoview/jobcreationchart.jpg The Department of State is leading the commemoration of Florida's 500th anniversary in 2013. For more information, please go to www.fla500.com. The Department of State is committed to excellence. Please take our Customer Satisfaction Survey: http://survey.dos.state.fl.us/index.aspx?email=Shirley.Harris at DOS.MyFlorida.com -----Original Message----- From: varnish-misc-bounces at varnish-cache.org [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Hugo Cisneiros (Eitch) Sent: Monday, June 11, 2012 6:09 PM To: varnish-misc at varnish-cache.org Subject: Re: Varnish access log client IP's the same On Wed, Jun 6, 2012 at 4:49 PM, Harris, Shirley A. wrote: > Hello, > > Everything was working then about a month ago the Varnish access log started showing 2.2.2.2 and 2.2.2.3 as almost all the client IP addresses. > > The client IP as Varnish sees it is now set up to show in Apache's access log, but the problem seems to be that Varnish still sees the IP addresses as 2.2.2.2 or 2.2.2.3. Since varnish gets the client IP from the network, probably this is not a varnish problem, but a networking one. Some other host must be getting the requests and passing to varnish, including the X-Forwarded-For HTTP header. I'd go with network analysis tool to discover what is happening (on linux, tcpdump or iptraf for example). -- []'s Hugo www.devin.com.br _______________________________________________ varnish-misc mailing list varnish-misc at varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc From martin.hewitt at gmail.com Mon Jun 18 15:18:54 2012 From: martin.hewitt at gmail.com (Martin Hewitt) Date: Mon, 18 Jun 2012 16:18:54 +0100 Subject: Purge object on POST or PUT Message-ID: Hi all, I'm looking at configuring Varnish to run infront of a CouchDB-based database cluster. These two, sadly, don't work perfectly together, but I'd like to know if it's possible to force a re-fetch of an object at a given URL after a POST or PUT has been sent to said URL. An example flow would be: 1. GET /database_name/ObjectId - Object is fetched from the backend and placed in cache 2. GET /database_name/ObjectId - Object is served from cache 3. POST /database_name/ObjectId - I'm currently using pass; to send through all POST and PUT requests, so the updated object now exists in the database - Here is where I'd like to purge the object in the cache 4. GET /database_name/ObjectId - Object is fetched from the backend and placed in cache 5. GET /database_name/ObjectId - Object is served from cache Currently, after the POST in [3], the subsequent GET passes the original, now stale, object from the cache, as the headers don't indicate that it should be updated. This header issue is why I'd like to force a purge based on my knowledge of what the system will do, instead of relying on the headers. I'm using Varnish 3, but can switch if a different version would make this doable! Many thanks, Martin From Larry.Hoffman at trivantis.com Mon Jun 18 18:50:33 2012 From: Larry.Hoffman at trivantis.com (Larry Hoffman) Date: Mon, 18 Jun 2012 14:50:33 -0400 Subject: SIze Limitation Problems???? Message-ID: Hey Guys, wanted to ask if you guys ran into this issue. A little background first: I'm running apache 2.2.14 (Ubuntu) Server with drupal 6.22. I installed varnish ( I believe it's version 1.1, not sure). Varnish seems to be working and caching the files. We have a download folder under the documents root with out exe products. These files are 885M or greater in size. These larger files are not be served by varnish. I uploaded an 18M exe file and that one downloads just fine. Any idea???? Thanks Larry From victor.cheng at paperg.com Mon Jun 18 19:11:02 2012 From: victor.cheng at paperg.com (Victor Cheng) Date: Mon, 18 Jun 2012 12:11:02 -0700 Subject: SIze Limitation Problems???? In-Reply-To: References: Message-ID: Have you checked your Varnish cache size? Perhaps it's too small to fit the full file (default in Ubuntu may be 512M) On Mon, Jun 18, 2012 at 11:50 AM, Larry Hoffman wrote: > Hey Guys, > > wanted to ask if you guys ran into this issue. > > A little background first: > > I'm running apache 2.2.14 (Ubuntu) Server with drupal 6.22. > > I installed varnish ( I believe it's version 1.1, not sure). > > Varnish seems to be working and caching the files. > > We have a download folder under the documents root with out exe products. > > These files are 885M or greater in size. > > These larger files are not be served by varnish. > > I uploaded an 18M exe file and that one downloads just fine. > > Any idea???? > > Thanks > Larry > > > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hugo.cisneiros at gmail.com Mon Jun 18 19:35:34 2012 From: hugo.cisneiros at gmail.com (Hugo Cisneiros (Eitch)) Date: Mon, 18 Jun 2012 16:35:34 -0300 Subject: SIze Limitation Problems???? In-Reply-To: References: Message-ID: On Mon, Jun 18, 2012 at 3:50 PM, Larry Hoffman wrote: [...] > We have a download folder under the documents root with out exe products. > These files are 885M or greater in size. > These larger files are not be served by varnish. > I uploaded an 18M exe file and that one downloads just fine. Why exactly your big files aren't served? With big files, you can have two major problems when serving them with Varnish. One could be the "send_timeout" parameter, that specifies a timeout serving files to the client. And the other can be the first_byte_timeout parameter, that specifies a timeout waiting for the data to come to begin serving. Tune up these parameters (on global scope or backend scope) and see if it helps. But IMHO, Varnish currently isn't efficient serving big, static files, since it has to download the entire file, put in the cache and serve to the client, and there's a huge overhead between getting from the server and serving to the user. This probably will be addressed when Varnish releases its streaming feature (without the limitations it has today) on future releases. At this time, I use here a direct web server to only serve big static files. -- []'s Hugo www.devin.com.br From Larry.Hoffman at trivantis.com Mon Jun 18 19:45:21 2012 From: Larry.Hoffman at trivantis.com (Larry Hoffman) Date: Mon, 18 Jun 2012 15:45:21 -0400 Subject: SIze Limitation Problems???? In-Reply-To: References: Message-ID: <0D09DBE2-61FC-47C0-A3A4-D829F52BB7F9@trivantis.com> Hey Victor, here is what I had in the varnish file: DAEMON_OPTS="-a :80 \ -T localhost:6082 \ -f /etc/varnish/default.vcl \ -S /etc/varnish/secret \ -s file,/var/lib/varnish/$INSTANCE/varnish_storage.bin,100G" Then I changed it too: DAEMON_OPTS="-a :80 \ -T localhost:6082 \ -f /etc/varnish/default.vcl \ -S /etc/varnish/secret \ -s file,/var/lib/varnish/$INSTANCE/varnish_storage.bin,900M" Is there another setting I need to set. Thanks Larry Larry Hoffman | Trivantis Corporation Direct: 513.852-6843 Larry.Hoffman at Trivantis.com | www.trivantis.com On Jun 18, 2012, at 3:11 PM, Victor Cheng wrote: > Have you checked your Varnish cache size? Perhaps it's too small to fit the full file (default in Ubuntu may be 512M) > > On Mon, Jun 18, 2012 at 11:50 AM, Larry Hoffman wrote: > Hey Guys, > > wanted to ask if you guys ran into this issue. > > A little background first: > > I'm running apache 2.2.14 (Ubuntu) Server with drupal 6.22. > > I installed varnish ( I believe it's version 1.1, not sure). > > Varnish seems to be working and caching the files. > > We have a download folder under the documents root with out exe products. > > These files are 885M or greater in size. > > These larger files are not be served by varnish. > > I uploaded an 18M exe file and that one downloads just fine. > > Any idea???? > > Thanks > Larry > > > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hugo.cisneiros at gmail.com Mon Jun 18 19:57:53 2012 From: hugo.cisneiros at gmail.com (Hugo Cisneiros (Eitch)) Date: Mon, 18 Jun 2012 16:57:53 -0300 Subject: Purge object on POST or PUT In-Reply-To: References: Message-ID: On Mon, Jun 18, 2012 at 12:18 PM, Martin Hewitt wrote: > I'm looking at configuring Varnish to run infront of a CouchDB-based > database cluster. These two, sadly, don't work perfectly together, but > I'd like to know if it's possible to force a re-fetch of an object at > a given URL after a POST or PUT has been sent to said URL. > > An example flow would be: > > 1. GET /database_name/ObjectId > - Object is fetched from the backend and placed in cache > 2. GET /database_name/ObjectId > - Object is served from cache > 3. POST /database_name/ObjectId > - I'm currently using pass; to send through all POST and PUT requests, > so the updated object now exists in the database > - Here is where I'd like to purge the object in the cache > 4. GET /database_name/ObjectId > - Object is fetched from the backend and placed in cache > 5. GET /database_name/ObjectId > - Object is served from cache > > Currently, after the POST in [3], the subsequent GET passes the > original, now stale, object from the cache, as the headers don't > indicate that it should be updated. This header issue is why I'd like > to force a purge based on my knowledge of what the system will do, > instead of relying on the headers. Have you tried using the PURGE/BAN method? For example, on vcl_recv you can do something like this (very generic and not tested): if (req.request == "POST" || req.request == "PUT") { ban("req.http.host == " + req.http.host + " && req.url == " + req.url); } This way, every post or put request will add the requested URL to be passed on the next request. See more on this issue in: https://www.varnish-cache.org/docs/trunk/tutorial/purging.html If you use the ban list, remember to tune the ban lurker parameter :) -- []'s Hugo www.devin.com.br From victor.cheng at paperg.com Mon Jun 18 20:13:01 2012 From: victor.cheng at paperg.com (Victor Cheng) Date: Mon, 18 Jun 2012 13:13:01 -0700 Subject: SIze Limitation Problems???? In-Reply-To: <0D09DBE2-61FC-47C0-A3A4-D829F52BB7F9@trivantis.com> References: <0D09DBE2-61FC-47C0-A3A4-D829F52BB7F9@trivantis.com> Message-ID: Along the lines of what Hugo said -- definitely check the parameters he mentioned. As for the change you made to adjust the size -- the 100G is probably more what you want in this particular case (assuming your hard drive is actually bigger than this), since you might have files over 900M? However, it doesn't especially make sense to cache these huge files unless you have multiple servers around the world and you need to have the cached files for latency reasons. Anyway, if it's still not working after tuning the parameters Hugo suggested, perhaps you could post more information about the exact issue. Like how can you tell the larger files are not being served by Varnish? Is it never making it into the object cache? Or are things timing out and the file not even being delivered etc. It would probably be most helpful to narrow down the point at which it actually fails. -Victor On Mon, Jun 18, 2012 at 12:45 PM, Larry Hoffman wrote: > Hey Victor, > > here is what I had in the varnish file: > > DAEMON_OPTS="-a :80 \ > -T localhost:6082 \ > -f /etc/varnish/default.vcl \ > -S /etc/varnish/secret \ > -s file,/var/lib/varnish/$INSTANCE/varnish_storage.bin,100G" > > Then I changed it too: > > DAEMON_OPTS="-a :80 \ > -T localhost:6082 \ > -f /etc/varnish/default.vcl \ > -S /etc/varnish/secret \ > -s file,/var/lib/varnish/$INSTANCE/varnish_storage.bin,900M" > > Is there another setting I need to set. > > Thanks > Larry > > > Larry Hoffman | > Trivantis Corporation > Direct: 513.852-6843 > Larry.Hoffman at Trivantis.com | www.trivantis.com > > On Jun 18, 2012, at 3:11 PM, Victor Cheng wrote: > > Have you checked your Varnish cache size? Perhaps it's too small to fit > the full file (default in Ubuntu may be 512M) > > On Mon, Jun 18, 2012 at 11:50 AM, Larry Hoffman < > Larry.Hoffman at trivantis.com> wrote: > >> Hey Guys, >> >> wanted to ask if you guys ran into this issue. >> >> A little background first: >> >> I'm running apache 2.2.14 (Ubuntu) Server with drupal 6.22. >> >> I installed varnish ( I believe it's version 1.1, not sure). >> >> Varnish seems to be working and caching the files. >> >> We have a download folder under the documents root with out exe products. >> >> These files are 885M or greater in size. >> >> These larger files are not be served by varnish. >> >> I uploaded an 18M exe file and that one downloads just fine. >> >> Any idea???? >> >> Thanks >> Larry >> >> >> >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hugo.cisneiros at gmail.com Mon Jun 18 20:19:13 2012 From: hugo.cisneiros at gmail.com (Hugo Cisneiros (Eitch)) Date: Mon, 18 Jun 2012 17:19:13 -0300 Subject: Conditional Backend Request with stale objects In-Reply-To: References: Message-ID: On Thu, Jun 14, 2012 at 3:04 PM, Claudio Devecchi wrote: > I compiled the version following the instructions on > https://www.varnish-cache.org/trac/wiki/BackendConditionalRequests, and > using varnishlog I already see the headers If-Modified-Since and > If-None-Match. But when the obj.ttl expires, varnish gets a 200 from backend > and I was expecting 304. Did you use the keep setting on the objects? And also check if you have something like "if (obj.ttl <= 0) { return (pass); }" along your VCL. Also, compare your varnishlog with the one from the page. If you still can't get the 304, post the varnishlog of the request :) -- []'s Hugo www.devin.com.br From hugo.cisneiros at gmail.com Mon Jun 18 20:26:04 2012 From: hugo.cisneiros at gmail.com (Hugo Cisneiros (Eitch)) Date: Mon, 18 Jun 2012 17:26:04 -0300 Subject: Hiding internal backend requests In-Reply-To: References: Message-ID: On Wed, Jun 13, 2012 at 7:26 PM, Darvin Denmian wrote: > That is my scenario: > > 1) Varnish (172.16.217.131:80), receives a request from a client, i.e: > http://172.16.217.131:80/a.png > 2) Request is forwarded to the Default Backend (127.0.0.1:8000) > 3) Default backend receive the request and process it > 4) That processing results in a new URL, i.e: > http://172.16.217.132:80/a.png (**As you can see the IP has changed) > 5) 172.16.217.132:80 is another backend in Varnish's config file > 6) The new URL points to a resource that should be provided by Varnish > (that resource generally is an image) > > My problem is: The client needs to execute 2 GETs to obtain the image. > > My question: ?How can I configure varnish to internally receive the > response from the first backend(127.0.0.1:8000), and fetch data from > the second backend (172.16.217.132:80), and after that, send the data > to the client? It's not simple, but it's possible. I use this approach with much success here. You will need something in the first backend response to differentiate the internal redirect from the successful request. For example, if the first backend needs to check another backend, you must write a HTTP header ("req.http.X-Internal-Redirect = True", for example). Then, on vcl_fetch, you use something like this: if (beresp.http.X-Internal-Redirect == "True") { set req.http.X-Internal-Redirect = "True"); return (restart); } With "return (restart)", your request will be restarted from scratch (entering vcl_recv), but with the additional header. Now you can set your other backend on vcl_recv based on this new header, like this on vcl_recv: if (req.http.X-Internal-Redirect == "True") { set req.backend = another_backend; } This new backend can't reply the additional header since this could create a restart loop. Good luck! :) -- []'s Hugo www.devin.com.br From darvin.denmian at gmail.com Mon Jun 18 20:41:37 2012 From: darvin.denmian at gmail.com (Darvin Denmian) Date: Mon, 18 Jun 2012 17:41:37 -0300 Subject: Hiding internal backend requests In-Reply-To: References: Message-ID: That's great! Thanks for your reply, I'll try it. On Mon, Jun 18, 2012 at 5:26 PM, Hugo Cisneiros (Eitch) wrote: > On Wed, Jun 13, 2012 at 7:26 PM, Darvin ?Denmian > wrote: >> That is my scenario: >> >> 1) Varnish (172.16.217.131:80), receives a request from a client, i.e: >> http://172.16.217.131:80/a.png >> 2) Request is forwarded to the Default Backend (127.0.0.1:8000) >> 3) Default backend receive the request and process it >> 4) That processing results in a new URL, i.e: >> http://172.16.217.132:80/a.png (**As you can see the IP has changed) >> 5) 172.16.217.132:80 is another backend in Varnish's config file >> 6) The new URL points to a resource that should be provided by Varnish >> (that resource generally is an image) >> >> My problem is: The client needs to execute 2 GETs to obtain the image. >> >> My question: ?How can I configure varnish to internally receive the >> response from the first backend(127.0.0.1:8000), and fetch data from >> the second backend (172.16.217.132:80), and after that, send the data >> to the client? > > It's not simple, but it's possible. I use this approach with much success here. > > You will need something in the first backend response to differentiate > the internal redirect from the successful request. For example, if the > first backend needs to check another backend, you must write a HTTP > header ("req.http.X-Internal-Redirect = True", for example). > > Then, on vcl_fetch, you use something like this: > > if (beresp.http.X-Internal-Redirect == "True") { > ? set req.http.X-Internal-Redirect = "True"); > ? return (restart); > } > > With "return (restart)", your request will be restarted from scratch > (entering vcl_recv), but with the additional header. Now you can set > your other backend on vcl_recv based on this new header, like this on > vcl_recv: > > if (req.http.X-Internal-Redirect == "True") { > ? set req.backend = another_backend; > } > > This new backend can't reply the additional header since this could > create a restart loop. > > Good luck! :) > > -- > []'s > Hugo > www.devin.com.br > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc From Larry.Hoffman at trivantis.com Mon Jun 18 21:03:26 2012 From: Larry.Hoffman at trivantis.com (Larry Hoffman) Date: Mon, 18 Jun 2012 17:03:26 -0400 Subject: SIze Limitation Problems???? In-Reply-To: References: <0D09DBE2-61FC-47C0-A3A4-D829F52BB7F9@trivantis.com> Message-ID: You bring up a good point about not caching these files. How would I do that? Thanks Larry On Jun 18, 2012, at 4:13 PM, Victor Cheng wrote: > Along the lines of what Hugo said -- definitely check the parameters he mentioned. > > As for the change you made to adjust the size -- the 100G is probably more what you want in this particular case (assuming your hard drive is actually bigger than this), since you might have files over 900M? However, it doesn't especially make sense to cache these huge files unless you have multiple servers around the world and you need to have the cached files for latency reasons. > > Anyway, if it's still not working after tuning the parameters Hugo suggested, perhaps you could post more information about the exact issue. Like how can you tell the larger files are not being served by Varnish? Is it never making it into the object cache? Or are things timing out and the file not even being delivered etc. It would probably be most helpful to narrow down the point at which it actually fails. > > -Victor > > On Mon, Jun 18, 2012 at 12:45 PM, Larry Hoffman wrote: > Hey Victor, > > here is what I had in the varnish file: > > DAEMON_OPTS="-a :80 \ > -T localhost:6082 \ > -f /etc/varnish/default.vcl \ > -S /etc/varnish/secret \ > -s file,/var/lib/varnish/$INSTANCE/varnish_storage.bin,100G" > > Then I changed it too: > > DAEMON_OPTS="-a :80 \ > -T localhost:6082 \ > -f /etc/varnish/default.vcl \ > -S /etc/varnish/secret \ > -s file,/var/lib/varnish/$INSTANCE/varnish_storage.bin,900M" > > Is there another setting I need to set. > > Thanks > Larry > > > Larry Hoffman | > Trivantis Corporation > Direct: 513.852-6843 > Larry.Hoffman at Trivantis.com | www.trivantis.com > > On Jun 18, 2012, at 3:11 PM, Victor Cheng wrote: > >> Have you checked your Varnish cache size? Perhaps it's too small to fit the full file (default in Ubuntu may be 512M) >> >> On Mon, Jun 18, 2012 at 11:50 AM, Larry Hoffman wrote: >> Hey Guys, >> >> wanted to ask if you guys ran into this issue. >> >> A little background first: >> >> I'm running apache 2.2.14 (Ubuntu) Server with drupal 6.22. >> >> I installed varnish ( I believe it's version 1.1, not sure). >> >> Varnish seems to be working and caching the files. >> >> We have a download folder under the documents root with out exe products. >> >> These files are 885M or greater in size. >> >> These larger files are not be served by varnish. >> >> I uploaded an 18M exe file and that one downloads just fine. >> >> Any idea???? >> >> Thanks >> Larry >> >> >> >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cdevecchi at gmail.com Mon Jun 18 21:37:58 2012 From: cdevecchi at gmail.com (Claudio Devecchi) Date: Mon, 18 Jun 2012 18:37:58 -0300 Subject: Conditional Backend Request with stale objects In-Reply-To: References: Message-ID: Hi Hugo, This problem I resolved today, my problem now is varnishncsa that is not logging. The log is coming this way 172.24.2.41 - - [15/Jun/2012:19:13:42 -0300] "- http://localhost-HTTP/1.0" 0 "-" "-" and the correct should be 172.22.2.41 - - [13/Jun/2012:16:06:00 -0300] "GET http://test.com/test1.jpgHTTP/1.1" 304 0 "-" "Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:12.0) Gecko/20100101 Firefox/12.0" Any idea? On Mon, Jun 18, 2012 at 5:19 PM, Hugo Cisneiros (Eitch) < hugo.cisneiros at gmail.com> wrote: > On Thu, Jun 14, 2012 at 3:04 PM, Claudio Devecchi > wrote: > > I compiled the version following the instructions on > > https://www.varnish-cache.org/trac/wiki/BackendConditionalRequests, and > > using varnishlog I already see the headers If-Modified-Since and > > If-None-Match. But when the obj.ttl expires, varnish gets a 200 from > backend > > and I was expecting 304. > > Did you use the keep setting on the objects? And also check if you > have something like "if (obj.ttl <= 0) { return (pass); }" along your > VCL. Also, compare your varnishlog with the one from the page. If you > still can't get the 304, post the varnishlog of the request :) > > -- > []'s > Hugo > www.devin.com.br > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rowan at jetboy.co.uk Mon Jun 18 23:03:38 2012 From: rowan at jetboy.co.uk (Rowan @ Jetboy) Date: Tue, 19 Jun 2012 00:03:38 +0100 Subject: Logging hit only in varnishncsa In-Reply-To: References: Message-ID: <004201cd4da6$991ce510$cb56af30$@co.uk> Varnish 3.0.2 / Ubuntu 12.04 Server x64 With Varnish sitting in front of Apache, am I right in saying that requests served from Varnish's cache will not get logged by Apache, and anything not served by Varnish will get logged both in Apache's logs and varnishncsa? If so, I'd like to only log Varnish hits in varnishncsa so I'm not double counting, as my intent is to aggregate all the logs for analysis. I can record whether it's a hit or a miss in Varnish's log by adding %{Varnish:hitmiss}x to the end of the log format. I was hoping to strip misses out with -X .*miss$ but it isn't working, and wouldn't be very elegant if it did. How do I get this working, or is there a better way? Ubuntu shows that varnishncsa has a [-i tag] argument, but this isn't listed in https://www.varnish-cache.org/docs/trunk/reference/varnishncsa.html I get back "-i is not valid for varnishncsa" regardless of what I try. https://www.varnish-cache.org/trac/ticket/646 suggests that there's an -o argument for this, but attempts with this return "invalid option -- 'o'" /etc/init.d/varnishncsa changes: LOG_FORMAT="%{Host}i %h %l %u %t \"%r\" %s %b \"%{Referer}i\" \"%{User-agent}i\" %{Varnish:hitmiss}x" DAEMON_OPTS="-a -c -D -P ${PIDFILE} -w /var/log/varnish/hollywood.varnish.log -X .*miss$" --chuid $USER --exec ${DAEMON} -- ${DAEMON_OPTS} -F "${LOG_FORMAT}" \ -------------- next part -------------- An HTML attachment was scrubbed... URL: From longvnit at gmail.com Tue Jun 19 14:28:05 2012 From: longvnit at gmail.com (Nguyen Thang Long) Date: Tue, 19 Jun 2012 21:28:05 +0700 Subject: How to stop varnish response when it's got bad request Message-ID: <084401cd4e27$c0d359a0$427a0ce0$@gmail.com> Hi, I have a problem with botnet, I can detect bad request to my server using varnish cache. Currently, I set varnish return error 403 when it's got bad request, but it's also got more bandwidth to responses (> 200Mbps) So, I want to varnish drop bad requests and not response to client. Help needed , Thanks ! -------------- next part -------------- An HTML attachment was scrubbed... URL: From perbu at varnish-software.com Tue Jun 19 16:26:27 2012 From: perbu at varnish-software.com (Per Buer) Date: Tue, 19 Jun 2012 18:26:27 +0200 Subject: How to stop varnish response when it's got bad request In-Reply-To: <084401cd4e27$c0d359a0$427a0ce0$@gmail.com> References: <084401cd4e27$c0d359a0$427a0ce0$@gmail.com> Message-ID: Hi Nguyen, On Tue, Jun 19, 2012 at 4:28 PM, Nguyen Thang Long wrote: > > I have a problem with botnet, I can detect bad request to my server using > varnish cache. > > ** > > Currently, I set varnish return error 403 when it?s got bad request, but > it?s also got more bandwidth to responses (> 200Mbps)**** > > So, I want to varnish drop bad requests and not response to client. > Martin wrote a VMOD that does conn_reset a while back. It is described here: https://www.varnish-software.com/blog/security-rejecting-offending-connections -- Per Buer Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer *Varnish makes websites fly!* Whitepapers | Video | Twitter -------------- next part -------------- An HTML attachment was scrubbed... URL: From fred at clift.org Tue Jun 19 17:27:35 2012 From: fred at clift.org (Fred Clift) Date: Tue, 19 Jun 2012 11:27:35 -0600 Subject: response body modifications - right place? Message-ID: I'm pretty sure based on my perusal of both the code and documentation (and mailing list archives) that Varnish currently has no provisions to inspect/modify response body data, right? If I were interested in doing some kind of modification/inspection of the response body, am I correct in assuming that i'd need to put a hook in at about line 776 of cache_center.c? This is right after the fetched content has been conditionally gunzipped. I've read that the 'varnish way' of doing stuff like this would be to modify the request (e.g. via header, or by munging the request URL) and then have the back end make customized/inspected content for me. Unfortunately, i dont have this luxury as I can't make (very much) change to the back end servers or content-generation. Are there any plans to put some content hooks in either the deliver or fetch phases? Ideally I'd like to have the opportunity to modify the response body just before an item is stuffed into the cache. Either in inline C/vcl or in a vmod. Thanks. Fred Clift From longvnit at gmail.com Wed Jun 20 02:10:26 2012 From: longvnit at gmail.com (Nguyen Long) Date: Wed, 20 Jun 2012 09:10:26 +0700 Subject: How to stop varnish response when it's got bad request In-Reply-To: References: <084401cd4e27$c0d359a0$427a0ce0$@gmail.com> Message-ID: Thanks Per Buer But, I can't build it, it's so hard I installed varnish from source successfully, next step, I installed vmods then got an error *You need rst2man installed to make dist* I Installed Python Docutils, but I can't fix it :( OS: CentOS 5.8- Linux localhost.localdomain 2.6.18-308.4.1.el5 #1 SMP Tue Apr 17 17:08:00 EDT 2012 x86_64 x86_64 x86_64 GNU/Linux Can anyone help me solve this problem ? Thank you so much :) @ Per Buer If I installed varnish from RPM package, is there a way to install vmods from another rpm package ? 2012/6/19 Per Buer > Hi Nguyen, > > On Tue, Jun 19, 2012 at 4:28 PM, Nguyen Thang Long wrote: > >> >> I have a problem with botnet, I can detect bad request to my server using >> varnish cache. >> >> ** >> >> Currently, I set varnish return error 403 when it?s got bad request, but >> it?s also got more bandwidth to responses (> 200Mbps)**** >> >> So, I want to varnish drop bad requests and not response to client. >> > > Martin wrote a VMOD that does conn_reset a while back. It is described > here: > > https://www.varnish-software.com/blog/security-rejecting-offending-connections > > > -- > Per Buer > Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer > *Varnish makes websites fly!* > Whitepapers | Video | > Twitter > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From info at massivescale.net Thu Jun 21 14:42:32 2012 From: info at massivescale.net (MassiveScale.net) Date: Thu, 21 Jun 2012 16:42:32 +0200 Subject: How to stop varnish response when it's got bad request In-Reply-To: References: <084401cd4e27$c0d359a0$427a0ce0$@gmail.com> Message-ID: <4FE332D8.8090807@massivescale.net> On 06/20/2012 04:10 AM, Nguyen Long wrote: > Thanks Per Buer > But, I can't build it, it's so hard > I installed varnish from source successfully, next step, I installed > vmods then got an error > *You need rst2man installed to make dist* > I Installed Python Docutils, but I can't fix it :( > > OS: CentOS 5.8- Linux localhost.localdomain 2.6.18-308.4.1.el5 #1 SMP > Tue Apr 17 17:08:00 EDT 2012 x86_64 x86_64 x86_64 GNU/Linux > > Can anyone help me solve this problem ? > Thank you so much :) It said that rst2man is missing. This package provides rst2man: http://pkgs.org/centos-5-rhel-5/repoforge-i386/python-docutils-0.6-1.el5.rf.noarch.rpm.html -- Andrzej Godziuk http://massivescale.net/ From xhrfc at yahoo.com Thu Jun 21 15:38:12 2012 From: xhrfc at yahoo.com (X.H.R.F.C.) Date: Thu, 21 Jun 2012 08:38:12 -0700 (PDT) Subject: varnish cache cookie page Message-ID: <1340293092.18718.YahooMailNeo@web43510.mail.sp1.yahoo.com> Hi all, Is there any way to cache a page even the client - browser - is sending cookie in header? My entyre website has a 'logon' status + avatar in the top of the page. I use esi to load avatar, user name ... in that section. It seems like varnish dosen't cache my specifyed pages, even they are using esi feature . Rule: if (req.http.Authorization || req.http.Cookie) { ? ? ? ? return (pass); } is there any way to make this possible? Regards, -------------- next part -------------- An HTML attachment was scrubbed... URL: From pom at dmsp.de Thu Jun 21 15:54:18 2012 From: pom at dmsp.de (Stefan Pommerening) Date: Thu, 21 Jun 2012 17:54:18 +0200 Subject: Varnishlog truncating cookie headers Message-ID: <4FE343AA.4070909@dmsp.de> Hi, varnishlog (using 3.0.2) seems to have a built-in limitation only displaying up to 255 characters in a single header line. Today I had a pretty hard time fiddling around with a bunch of cookie fragments and regular expressions in order to realize client director stickyness depending on a set of session conditions. Although I got it working finally it would have saved me some time if I could see the entire cookie header line in varnishlog. Is there any option I might have missed to enable displaying the complete headers - even when longer than 255 characters? Thanks a lot, Stefan Pommerening -- *Dipl.-Inform. Stefan Pommerening Informatik-B?ro: IT-Dienste & Projekte, Consulting & Coaching* http://www.dmsp.de -------------- next part -------------- An HTML attachment was scrubbed... URL: From hugo.cisneiros at gmail.com Thu Jun 21 16:13:43 2012 From: hugo.cisneiros at gmail.com (Hugo Cisneiros (Eitch)) Date: Thu, 21 Jun 2012 13:13:43 -0300 Subject: varnish cache cookie page In-Reply-To: <1340293092.18718.YahooMailNeo@web43510.mail.sp1.yahoo.com> References: <1340293092.18718.YahooMailNeo@web43510.mail.sp1.yahoo.com> Message-ID: On Thu, Jun 21, 2012 at 12:38 PM, X.H.R.F.C. wrote: > Is there any way to cache a page even the client - browser - is sending > cookie in header? > My entyre website has a 'logon' status + avatar in the top of the page. I > use esi to load avatar, user name ... in that section. > It seems like varnish dosen't cache my specifyed pages, even they are using > esi feature . > Rule: > > if (req.http.Authorization || req.http.Cookie) { > ? ? ? ? return (pass); > } Take out this rule to pass if there's a cookie on the request (req.http.Cookie) and do a lookup instead of pass. Then it will cache the page. But if your page changes because of a Cookie, every user that hits the cache will have the information about the first user. Since you use ESI, you could pass only the dynamic pages that changes, and keep all other on cache. That's the best way to do it. Another approach (not recommended because of the cache size and potential to overload it) is to create one cache per type of cookie, dealing in the vcl_hash section. For this, you can consult the mailing list archives and documentation and there will be plenty of information. -- []'s Hugo www.devin.com.br From perbu at varnish-software.com Fri Jun 22 07:02:11 2012 From: perbu at varnish-software.com (Per Buer) Date: Fri, 22 Jun 2012 09:02:11 +0200 Subject: Varnishlog truncating cookie headers In-Reply-To: <4FE343AA.4070909@dmsp.de> References: <4FE343AA.4070909@dmsp.de> Message-ID: Hi Stefan, On Thu, Jun 21, 2012 at 5:54 PM, Stefan Pommerening wrote: > Hi, > > varnishlog (using 3.0.2) seems to have a built-in limitation only > displaying up to 255 characters in a single header line. > Yes. This is controlled by the shm_reclen parameter. See "man varnishd". -- Per Buer Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer *Varnish makes websites fly!* Whitepapers | Video | Twitter -------------- next part -------------- An HTML attachment was scrubbed... URL: From javi at lavandeira.net Fri Jun 22 16:27:36 2012 From: javi at lavandeira.net (Javi Lavandeira) Date: Sat, 23 Jun 2012 01:27:36 +0900 Subject: varnishd working fine, but no output on varnishncsa/varnishlog/varnishstat Message-ID: Hello, I have Varnish 3.0.2 running on an OS X Lion Server machine (10.7.3), installed from MacPorts. Varnish works fine (it's serving my site), but I get no output when I run varnishncsa/varnishlog/varnishstat. Any idea on what I'm doing wrong? Here's my config file just in case, though there's nothing especial on it; it's just the typical Wordpress cookie/admin settings, purge support and X-Forwarded-For: header for the Apache backend: http://pastebin.com/tgrbYw26 Also, these are the related paths, binary versions and command line that I'm using: [root]cache:~ # which varnishstat /opt/local/bin/varnishstat [root]cache:~ # which varnishd /opt/local/sbin/varnishd [root]cache:~ # varnishd -V varnishd (varnish-3.0.2 revision 55e70a4) Copyright (c) 2006 Verdens Gang AS Copyright (c) 2006-2011 Varnish Software AS [root]cache:~ # varnishstat -V varnishstat (varnish-3.0.2 revision 55e70a4) Copyright (c) 2006 Verdens Gang AS Copyright (c) 2006-2011 Varnish Software AS [root]cache:~ # ps -ef |grep varnishd 0 19790 1 0 Tue11AM ?? 0:08.19 /opt/local/sbin/varnishd -f /opt/local/etc/varnish/default.vcl -s malloc,3G -T 127.0.0.1:2000 -a 0.0.0.0:80 -2 41275 19790 0 4:09AM ?? 0:33.84 /opt/local/sbin/varnishd -f /opt/local/etc/varnish/default.vcl -s malloc,3G -T 127.0.0.1:2000 -a 0.0.0.0:80 0 13266 13224 0 1:10AM ttys000 0:00.00 grep varnishd [root]cache:~ # system_profiler SPSoftwareDataType Software: System Software Overview: System Version: Mac OS X 10.7.3 (11D50) Kernel Version: Darwin 11.3.0 Boot Volume: HD1 Boot Mode: Normal Computer Name: cache User Name: System Administrator (root) Secure Virtual Memory: Enabled 64-bit Kernel and Extensions: Yes Time since boot: 5 days 44 minutes [root]cache:~ # uname -a Darwin cache.local 11.3.0 Darwin Kernel Version 11.3.0: Thu Jan 12 18:47:41 PST 2012; root:xnu-1699.24.23~1/RELEASE_X86_64 x86_64 [root]cache:~ # port info varnish Warning: port definitions are more than two weeks old, consider using selfupdate varnish @3.0.2 (www) Variants: universal Description: Varnish was written from the ground up to be a high performance caching reverse proxy. Homepage: http://www.varnish-cache.org Build Dependencies: pkgconfig Library Dependencies: pcre Platforms: darwin License: unknown Maintainers: pmq at macports.org, openmaintainer at macports.org Thanks in advance, -- Javi Lavandeira http://www.lavandeira.net -------------- next part -------------- An HTML attachment was scrubbed... URL: From xhrfc at yahoo.com Fri Jun 22 16:36:27 2012 From: xhrfc at yahoo.com (X.H.R.F.C.) Date: Fri, 22 Jun 2012 09:36:27 -0700 (PDT) Subject: varnish vcl syntax Message-ID: <1340382987.85063.YahooMailNeo@web43515.mail.sp1.yahoo.com> Hello,? Is there any way to get variable from regex and setup ttl ? example: application : PHP $this->response->headers(array('Content-Type' => 'text/html', 'esi-enabled' => 'true | 20m')); VARNISH if (beresp.http.esi-enabled ~ "true | [0-9a-z]+$") { ? ? ? ? set beresp.ttl = regsub(beresp.http.esi-enabled,"([0-9a-z]+$)","\0"); ? ? ? ? unset beresp.http.esi-enabled; } after varnish restart i get: Stopping HTTP accelerator: varnishd failed! Starting HTTP accelerator: varnishd failed! Message from VCC-compiler: Expression has type STRING, expected DURATION Regards -------------- next part -------------- An HTML attachment was scrubbed... URL: From javi at lavandeira.net Fri Jun 22 16:39:22 2012 From: javi at lavandeira.net (Javi Lavandeira) Date: Sat, 23 Jun 2012 01:39:22 +0900 Subject: [SOLVED] Re: varnishd working fine, but no output on varnishncsa/varnishlog/varnishstat In-Reply-To: References: Message-ID: <83C381AC-B706-4F6D-A90B-821521820FA7@lavandeira.net> Hello again, I found out what was wrong. I needed to add -F to the command line in the property list that launches Varnish from launchctl. This is the plist file that works, in case someone else has the same issue: [root]cache:~ # cat /Library/LaunchDaemons/org.macports.varnish.plist Label org.macports.varnish ProgramArguments /opt/local/sbin/varnishd -F -f /opt/local/etc/varnish/default.vcl -s malloc,3G -T 127.0.0.1:2000 -a 0.0.0.0:80 KeepAlive RunAtLoad WorkingDirectory /opt/local/sbin StandardErrorPath /var/log/varnish.log StandardOutPath /var/log/varnish.log Regards, -- Javi Lavandeira http://www.lavandeira.net On Jun 23, 2012, at 1:27 AM, Javi Lavandeira wrote: > Hello, > > I have Varnish 3.0.2 running on an OS X Lion Server machine (10.7.3), installed from MacPorts. Varnish works fine (it's serving my site), but I get no output when I run varnishncsa/varnishlog/varnishstat. > > Any idea on what I'm doing wrong? > > Here's my config file just in case, though there's nothing especial on it; it's just the typical Wordpress cookie/admin settings, purge support and X-Forwarded-For: header for the Apache backend: http://pastebin.com/tgrbYw26 > > Also, these are the related paths, binary versions and command line that I'm using: > > [root]cache:~ # which varnishstat > /opt/local/bin/varnishstat > [root]cache:~ # which varnishd > /opt/local/sbin/varnishd > [root]cache:~ # varnishd -V > varnishd (varnish-3.0.2 revision 55e70a4) > Copyright (c) 2006 Verdens Gang AS > Copyright (c) 2006-2011 Varnish Software AS > [root]cache:~ # varnishstat -V > varnishstat (varnish-3.0.2 revision 55e70a4) > Copyright (c) 2006 Verdens Gang AS > Copyright (c) 2006-2011 Varnish Software AS > > [root]cache:~ # ps -ef |grep varnishd > 0 19790 1 0 Tue11AM ?? 0:08.19 /opt/local/sbin/varnishd -f /opt/local/etc/varnish/default.vcl -s malloc,3G -T 127.0.0.1:2000 -a 0.0.0.0:80 > -2 41275 19790 0 4:09AM ?? 0:33.84 /opt/local/sbin/varnishd -f /opt/local/etc/varnish/default.vcl -s malloc,3G -T 127.0.0.1:2000 -a 0.0.0.0:80 > 0 13266 13224 0 1:10AM ttys000 0:00.00 grep varnishd > > [root]cache:~ # system_profiler SPSoftwareDataType > Software: > > System Software Overview: > > System Version: Mac OS X 10.7.3 (11D50) > Kernel Version: Darwin 11.3.0 > Boot Volume: HD1 > Boot Mode: Normal > Computer Name: cache > User Name: System Administrator (root) > Secure Virtual Memory: Enabled > 64-bit Kernel and Extensions: Yes > Time since boot: 5 days 44 minutes > > [root]cache:~ # uname -a > Darwin cache.local 11.3.0 Darwin Kernel Version 11.3.0: Thu Jan 12 18:47:41 PST 2012; root:xnu-1699.24.23~1/RELEASE_X86_64 x86_64 > > [root]cache:~ # port info varnish > Warning: port definitions are more than two weeks old, consider using selfupdate > varnish @3.0.2 (www) > Variants: universal > > Description: Varnish was written from the ground up to be a high performance caching reverse proxy. > Homepage: http://www.varnish-cache.org > > Build Dependencies: pkgconfig > Library Dependencies: pcre > Platforms: darwin > License: unknown > Maintainers: pmq at macports.org, openmaintainer at macports.org > > Thanks in advance, > > -- > Javi Lavandeira > http://www.lavandeira.net > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hugo.cisneiros at gmail.com Fri Jun 22 18:28:21 2012 From: hugo.cisneiros at gmail.com (Hugo Cisneiros (Eitch)) Date: Fri, 22 Jun 2012 15:28:21 -0300 Subject: varnish vcl syntax In-Reply-To: <1340382987.85063.YahooMailNeo@web43515.mail.sp1.yahoo.com> References: <1340382987.85063.YahooMailNeo@web43515.mail.sp1.yahoo.com> Message-ID: On Fri, Jun 22, 2012 at 1:36 PM, X.H.R.F.C. wrote: > Is there any way to get variable from regex and setup ttl ? [...] You could use the std vmod to convert from string to duration: https://www.varnish-cache.org/docs/trunk/reference/vmod_std.html#duration -- []'s Hugo www.devin.com.br From Travis.Crowder at penton.com Fri Jun 22 18:45:30 2012 From: Travis.Crowder at penton.com (Crowder, Travis) Date: Fri, 22 Jun 2012 18:45:30 +0000 Subject: Varnish using more RAM than allocated Message-ID: <61E9173DF681274790BA29C89A72200C0267EFFB@ksopexchm02.pme.penton.com> I recall seeing on the mailing list before a reason why Varnish would use more RAM than allocated due to storing something outside of the defined allocated memory. For example, I have a box that is allocated 6G of RAM, but it is using about 12G virtual memory. >top 9101 nobody 20 0 11.7g 6.8g 81m S 6.0 88.5 1270:21 varnishd Varnish is invoked via: /usr/local/sbin/varnishd -f /usr/local/etc/varnish/default.vcl -s malloc,6G -T xxx.xxx.xxx.xxx:2000 -a :80 Does anyone know what I am talking about? Thanks, Travis -------------- next part -------------- An HTML attachment was scrubbed... URL: From lee at leetrout.com Fri Jun 22 20:42:41 2012 From: lee at leetrout.com (Lee Trout) Date: Fri, 22 Jun 2012 16:42:41 -0400 Subject: Varnish using more RAM than allocated In-Reply-To: <61E9173DF681274790BA29C89A72200C0267EFFB@ksopexchm02.pme.penton.com> References: <61E9173DF681274790BA29C89A72200C0267EFFB@ksopexchm02.pme.penton.com> Message-ID: I generally don't worry about the high virtual memory usage. (and perhaps that will bite me some day ^_^) There are other memory requirements such as the threads (-s is only for the object cache). Beyond that, there is also transient storage for short lived requests which defaults to an unlimited malloc. Given that, I would say the resident memory size looks about right. https://www.varnish-cache.org/trac/wiki/Performance & https://www.varnish-cache.org/docs/3.0/reference/varnishd.html#transient-storage You could also dig in with pmap ;) On Fri, Jun 22, 2012 at 2:45 PM, Crowder, Travis wrote: > I recall seeing on the mailing list before a reason why Varnish would > use more RAM than allocated due to storing something outside of the defined > allocated memory.**** > > ** ** > > For example, I have a box that is allocated 6G of RAM, but it is using > about 12G virtual memory.**** > > ** ** > > >top**** > > 9101 nobody 20 0 11.7g 6.8g 81m S 6.0 88.5 1270:21 varnishd**** > > ** ** > > Varnish is invoked via:**** > > /usr/local/sbin/varnishd -f /usr/local/etc/varnish/default.vcl -s > malloc,6G -T xxx.xxx.xxx.xxx:2000 -a :80**** > > ** ** > > Does anyone know what I am talking about?**** > > ** ** > > Thanks,**** > > Travis**** > > ** ** > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From perbu at varnish-software.com Fri Jun 22 21:03:59 2012 From: perbu at varnish-software.com (Per Buer) Date: Fri, 22 Jun 2012 23:03:59 +0200 Subject: Varnish using more RAM than allocated In-Reply-To: <61E9173DF681274790BA29C89A72200C0267EFFB@ksopexchm02.pme.penton.com> References: <61E9173DF681274790BA29C89A72200C0267EFFB@ksopexchm02.pme.penton.com> Message-ID: On Fri, Jun 22, 2012 at 8:45 PM, Crowder, Travis wrote: > I recall seeing on the mailing list before a reason why Varnish would > use more RAM than allocated due to storing something outside of the defined > allocated memory.**** > > ** ** > > For example, I have a box that is allocated 6G of RAM, but it is using > about 12G virtual memory. > That doesn't make sense. Virtual memory is virtual. You're not using it, Varnish has mapped it into it's memory space. Just ignore it. Also, note that allocating memory to a backend (6G) does not include the amount of memory needed for the various internal structures - more or less 1k per object. -- Per Buer Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer *Varnish makes websites fly!* Whitepapers | Video | Twitter -------------- next part -------------- An HTML attachment was scrubbed... URL: From Travis.Crowder at penton.com Fri Jun 22 22:16:43 2012 From: Travis.Crowder at penton.com (Crowder, Travis) Date: Fri, 22 Jun 2012 22:16:43 +0000 Subject: Varnish using more RAM than allocated In-Reply-To: References: <61E9173DF681274790BA29C89A72200C0267EFFB@ksopexchm02.pme.penton.com>, Message-ID: <1816F2D8-C4EE-4B61-BCA0-24086CE0B09E@penton.com> Thank you. That about "1k per object for internal structures" is what I was looking for. Sent from my iPhone On Jun 22, 2012, at 4:04 PM, "Per Buer" > wrote: On Fri, Jun 22, 2012 at 8:45 PM, Crowder, Travis > wrote: I recall seeing on the mailing list before a reason why Varnish would use more RAM than allocated due to storing something outside of the defined allocated memory. For example, I have a box that is allocated 6G of RAM, but it is using about 12G virtual memory. That doesn't make sense. Virtual memory is virtual. You're not using it, Varnish has mapped it into it's memory space. Just ignore it. Also, note that allocating memory to a backend (6G) does not include the amount of memory needed for the various internal structures - more or less 1k per object. -- [http://www.varnish-software.com/sites/default/files/varnishsoft_white_190x47.png] Per Buer Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer Varnish makes websites fly! Whitepapers | Video | Twitter -------------- next part -------------- An HTML attachment was scrubbed... URL: From jon.hauge at relationhouse.com Sat Jun 23 13:38:50 2012 From: jon.hauge at relationhouse.com (Jon Hauge - Relation House) Date: Sat, 23 Jun 2012 15:38:50 +0200 Subject: how to prevent 503 guro Message-ID: We have a website where the backend sometimes do not behave. I do know that varnish do have a cached version of e.g. the frontpage. How do I make sure that varnish just serve that cached version no matter what - instead of throwing 503 guro? Sometimes the backend times out or varnish cannot establish a connection to the backend. I don?t use the probe because in most cases the probe thinks that the backend is ok ? but stil serves 503 goros. -------------- next part -------------- An HTML attachment was scrubbed... URL: From lasse.karstensen at gmail.com Sat Jun 23 15:37:40 2012 From: lasse.karstensen at gmail.com (Lasse Karstensen) Date: Sat, 23 Jun 2012 17:37:40 +0200 Subject: how to prevent 503 guro In-Reply-To: References: Message-ID: <20120623153739.GA25762@yankee.samfundet.no> Jon Hauge - Relation House: > We have a website where the backend sometimes do not behave. > I do know that varnish do have a cached version of e.g. the frontpage. > How do I make sure that varnish just serve that cached version no matter > what - instead of throwing 503 guro? Have a look at grace mode: https://www.varnish-cache.org/docs/trunk/tutorial/handling_misbehaving_servers.html -- Lasse Karstensen From abraham.sustaita at buscacorp.com Sun Jun 24 04:59:23 2012 From: abraham.sustaita at buscacorp.com (Abraham Cruz Sustaita) Date: Sat, 23 Jun 2012 21:59:23 -0700 Subject: Problem with regular expression. In-Reply-To: References: Message-ID: I have in my system this two kind of url's for files: http://site.com/img.jpg http://site.com/img.jpg?123 I have this regular expression: if (req.http.Host == "s3.files.fileserver.com" && req.url ~ "\.(flv|mp4|jpg|xml|png|gif)(\?[a-z0-9]+)$") { } but it only match the second one. How should I change it so it can match both urls? Thanks in advance. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jonathan.hursey at adrevolution.com Sun Jun 24 05:03:58 2012 From: jonathan.hursey at adrevolution.com (Jonathan Hursey) Date: Sun, 24 Jun 2012 00:03:58 -0500 Subject: Problem with regular expression. In-Reply-To: References: Message-ID: "\.(flv|mp4|jpg|xml|png|gif)(\?[a-z0-9]+)?" On Sat, Jun 23, 2012 at 11:59 PM, Abraham Cruz Sustaita < abraham.sustaita at buscacorp.com> wrote: > I have in my system this two kind of url's for files: > > http://site.com/img.jpg > http://site.com/img.jpg?123 > > I have this regular expression: > > if (req.http.Host == "s3.files.fileserver.com" && req.url ~ > "\.(flv|mp4|jpg|xml|png|gif)(\?[a-z0-9]+)$") { > } > > but it only match the second one. How should I change it so it can match > both urls? > > Thanks in advance. > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From abraham.sustaita at buscacorp.com Sun Jun 24 05:19:52 2012 From: abraham.sustaita at buscacorp.com (Abraham Cruz Sustaita) Date: Sat, 23 Jun 2012 22:19:52 -0700 Subject: Problem with regular expression. In-Reply-To: References: Message-ID: Thanks a lot. It works! 2012/6/23 Jonathan Hursey > "\.(flv|mp4|jpg|xml|png|gif)(\?[a-z0-9]+)?" > > > On Sat, Jun 23, 2012 at 11:59 PM, Abraham Cruz Sustaita < > abraham.sustaita at buscacorp.com> wrote: > >> I have in my system this two kind of url's for files: >> >> http://site.com/img.jpg >> http://site.com/img.jpg?123 >> >> I have this regular expression: >> >> if (req.http.Host == "s3.files.fileserver.com" && req.url ~ >> "\.(flv|mp4|jpg|xml|png|gif)(\?[a-z0-9]+)$") { >> } >> >> but it only match the second one. How should I change it so it can match >> both urls? >> >> Thanks in advance. >> >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jonathan.hursey at adrevolution.com Sun Jun 24 07:37:49 2012 From: jonathan.hursey at adrevolution.com (Jonathan Hursey) Date: Sun, 24 Jun 2012 02:37:49 -0500 Subject: Problem with regular expression. In-Reply-To: References: Message-ID: :) glad to help. On Sun, Jun 24, 2012 at 12:19 AM, Abraham Cruz Sustaita < abraham.sustaita at buscacorp.com> wrote: > Thanks a lot. It works! > > 2012/6/23 Jonathan Hursey > >> "\.(flv|mp4|jpg|xml|png|gif)(\?[a-z0-9]+)?" >> >> >> On Sat, Jun 23, 2012 at 11:59 PM, Abraham Cruz Sustaita < >> abraham.sustaita at buscacorp.com> wrote: >> >>> I have in my system this two kind of url's for files: >>> >>> http://site.com/img.jpg >>> http://site.com/img.jpg?123 >>> >>> I have this regular expression: >>> >>> if (req.http.Host == "s3.files.fileserver.com" && req.url ~ >>> "\.(flv|mp4|jpg|xml|png|gif)(\?[a-z0-9]+)$") { >>> } >>> >>> but it only match the second one. How should I change it so it can match >>> both urls? >>> >>> Thanks in advance. >>> >>> >>> _______________________________________________ >>> varnish-misc mailing list >>> varnish-misc at varnish-cache.org >>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From javier at casares.org Mon Jun 25 18:28:07 2012 From: javier at casares.org (Javier Casares) Date: Mon, 25 Jun 2012 20:28:07 +0200 Subject: Varnish Cache 3.0.3 rc 1 In-Reply-To: <20120604063318.GA11089@err.no> References: <20120604063318.GA11089@err.no> Message-ID: Some problems installing in CentOS 6.2 with "jemalloc", but everything works fine (by now) ;) fyi (this solved that): #rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/x86_64/jemalloc-3.0.0-2.el6.x86_64.rpm Javier Casares http://javiercasares.com/ 2012/6/4 Tollef Fog Heen : > Hi all, > > we're finally getting close to 3.0.3 (it's taken a while!), so if people > could please try it, that would be most appreciated. > > Source can be downloaded from > http://repo.varnish-cache.org/test/varnish-3.0.3-rc1.tar.gz, packages > live in http://repo.varnish-cache.org/test/lucid/, > http://repo.varnish-cache.org/test/squeeze/ and > http://repo.varnish-cache.org/test/rpms/ > > Bugs to trac, please. ?Feedback most appreciated! > > -- > Tollef Fog Heen > Technical lead, Varnish Software > t: +47 21 98 92 64 > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc From jeanpralo at gmail.com Tue Jun 26 12:09:19 2012 From: jeanpralo at gmail.com (Jean Praloran) Date: Tue, 26 Jun 2012 14:09:19 +0200 Subject: Handle timeout from a backend Message-ID: Hi there, I have a little issue with my differents varnish wich is probably due to a bad configuration. >From time to time I am having 503 error thrown by varnish probably due to a backend timeout which is not responding fast enough in regards of varnish timeouts. My question is there anyway to catch that kind of errors and serve a cached content instead of a 503 ? I'm running Debian 6.0.1 with varnishd (varnish-3.0.2 revision cbf1284) Regards, -- Praloran Jean -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeanpralo at gmail.com Tue Jun 26 13:37:52 2012 From: jeanpralo at gmail.com (Jean Praloran) Date: Tue, 26 Jun 2012 15:37:52 +0200 Subject: Handle timeout from a backend In-Reply-To: References: Message-ID: I didn't see the section Known limitations on grace and saint mode. Exactly what I dit and works like a charm :) Little modification to the backend so the timeout of the backend wouldn't be 60s and that's exactly what I was looking for, sorry for the useless mail :/ On Tue, Jun 26, 2012 at 2:09 PM, Jean Praloran wrote: > Hi there, > > > I have a little issue with my differents varnish wich is probably due to a > bad configuration. > > > From time to time I am having 503 error thrown by varnish probably due to > a backend timeout which is not responding fast enough in regards of varnish > timeouts. My question is there anyway to catch that kind of errors and > serve a cached content instead of a 503 ? > > > I'm running Debian 6.0.1 with varnishd (varnish-3.0.2 revision cbf1284) > > > Regards, > > > -- > Praloran Jean > -- Praloran Jean -------------- next part -------------- An HTML attachment was scrubbed... URL: From allan.brand at gmail.com Tue Jun 26 16:46:17 2012 From: allan.brand at gmail.com (Allan Brand) Date: Tue, 26 Jun 2012 11:46:17 -0500 Subject: Load Balancing and Varnish Message-ID: Greetings, I'm new to Varnish and I've seen a number of discussions where Varnish is used in conjunction with a load balancer in the following manner: [LB] -> [Varnish] -> [www 1-n] I'm guessing that Varnish is running locally on the www hosts? If not, Is there any reason why placing Varnish in front of the load balancers instead would not be ideal? [Varnish] -> [LB} -> [www 1-n] Thanks, Allan From javier at casares.org Tue Jun 26 16:58:05 2012 From: javier at casares.org (Javier Casares) Date: Tue, 26 Jun 2012 18:58:05 +0200 Subject: Load Balancing and Varnish In-Reply-To: References: Message-ID: ?Why don't you use Varnish as a LB? https://www.varnish-cache.org/trac/wiki/LoadBalancing Javier Casares http://javiercasares.com/ 2012/6/26 Allan Brand : > Greetings, > > I'm new to Varnish and I've seen a number of discussions where Varnish > is used in conjunction with a load balancer in the following manner: > [LB] -> [Varnish] -> [www 1-n] > > I'm guessing that Varnish is running locally on the www hosts? ?If > not, Is there any reason why placing Varnish in front of the load > balancers instead would not be ideal? > [Varnish] -> [LB} -> [www 1-n] > > Thanks, > Allan > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc From rowan at jetboy.co.uk Tue Jun 26 17:01:41 2012 From: rowan at jetboy.co.uk (Rowan @ Jetboy) Date: Tue, 26 Jun 2012 18:01:41 +0100 Subject: Syntax for piping varnish logs to rotatelogs In-Reply-To: <1340382987.85063.YahooMailNeo@web43515.mail.sp1.yahoo.com> References: <1340382987.85063.YahooMailNeo@web43515.mail.sp1.yahoo.com> Message-ID: <002701cd53bd$5c2efcc0$148cf640$@co.uk> Ubuntu 12.04 Server x64, Varnish 3.0.2 I'm trying to pipe varnishncsa's logs through Apache's rotatelogs, and running from the shell, things work fine: sudo varnishncsa -a -P /var/run/varnishncsa/varnishncsa.pid |/usr/sbin/rotatelogs /var/log/varnish/varnish.log.%Y%m%d%H 3600 creates a new logfile in /var/log/varnish, with rotation every hour (3600 seconds). However, I'm struggling to get things working the same way inside /etc/init.d/varnishncsa: PATH=/sbin:/bin:/usr/sbin:/usr/bin DAEMON=/usr/bin/$NAME PIDFILE=/var/run/$NAME/$NAME.pid LOGFILE=/var/log/varnish/varnishncsa.log USER=varnishlog DAEMON_OPTS="-a -P ${PIDFILE}" DAEMON_PIPE="|/usr/sbin/rotatelogs /var/log/varnish/varnish.log.%Y%m%d%H 3600" ... start_varnishncsa() { output=$(/bin/tempfile -s.varnish) log_daemon_msg "Starting $DESC" "$NAME" create_pid_directory if start-stop-daemon --start --verbose --pidfile ${PIDFILE} \ --chuid $USER --exec ${DAEMON} -- ${DAEMON_OPTS} \ > ${output} 2>&1; then log_end_msg 0 else log_end_msg 1 cat $output exit 1 fi rm $output } Where should I put DAEMON_PIPE in the above code? I've tried at the end of: if start-stop-daemon --start --verbose --pidfile ${PIDFILE} which is where additional command line parameters usually go, but it isn't creating a logfile. From allan.brand at gmail.com Tue Jun 26 17:31:49 2012 From: allan.brand at gmail.com (Allan Brand) Date: Tue, 26 Jun 2012 12:31:49 -0500 Subject: Load Balancing and Varnish In-Reply-To: References: Message-ID: Can Varnish load balance with algorithms other than round-robin or random? That's all I've been able to find. Thanks, Allan On Tue, Jun 26, 2012 at 11:58 AM, Javier Casares wrote: > ?Why don't you use Varnish as a LB? > > https://www.varnish-cache.org/trac/wiki/LoadBalancing > > > > Javier Casares > http://javiercasares.com/ > > > 2012/6/26 Allan Brand : >> Greetings, >> >> I'm new to Varnish and I've seen a number of discussions where Varnish >> is used in conjunction with a load balancer in the following manner: >> [LB] -> [Varnish] -> [www 1-n] >> >> I'm guessing that Varnish is running locally on the www hosts? ?If >> not, Is there any reason why placing Varnish in front of the load >> balancers instead would not be ideal? >> [Varnish] -> [LB} -> [www 1-n] >> >> Thanks, >> Allan >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc -- --------------------------------------- Allan (208) 977-6014 (eFax) From Travis.Crowder at penton.com Tue Jun 26 17:36:04 2012 From: Travis.Crowder at penton.com (Crowder, Travis) Date: Tue, 26 Jun 2012 17:36:04 +0000 Subject: Load Balancing and Varnish In-Reply-To: References: Message-ID: <61E9173DF681274790BA29C89A72200C02688DE7@ksopexchm02.pme.penton.com> Yes. See http://www.scribd.com/doc/92715754/Varnish-Book#page=122 -Travis -----Original Message----- From: varnish-misc-bounces at varnish-cache.org [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Allan Brand Sent: Tuesday, June 26, 2012 12:32 PM To: varnish-misc at varnish-cache.org Subject: Re: Load Balancing and Varnish Can Varnish load balance with algorithms other than round-robin or random? That's all I've been able to find. Thanks, Allan On Tue, Jun 26, 2012 at 11:58 AM, Javier Casares wrote: > ?Why don't you use Varnish as a LB? > > https://www.varnish-cache.org/trac/wiki/LoadBalancing > > > > Javier Casares > http://javiercasares.com/ > > > 2012/6/26 Allan Brand : >> Greetings, >> >> I'm new to Varnish and I've seen a number of discussions where >> Varnish is used in conjunction with a load balancer in the following manner: >> [LB] -> [Varnish] -> [www 1-n] >> >> I'm guessing that Varnish is running locally on the www hosts? ?If >> not, Is there any reason why placing Varnish in front of the load >> balancers instead would not be ideal? >> [Varnish] -> [LB} -> [www 1-n] >> >> Thanks, >> Allan >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc -- --------------------------------------- Allan (208) 977-6014 (eFax) _______________________________________________ varnish-misc mailing list varnish-misc at varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc From hugo.cisneiros at gmail.com Tue Jun 26 17:40:38 2012 From: hugo.cisneiros at gmail.com (Hugo Cisneiros (Eitch)) Date: Tue, 26 Jun 2012 14:40:38 -0300 Subject: Load Balancing and Varnish In-Reply-To: References: Message-ID: On Tue, Jun 26, 2012 at 1:46 PM, Allan Brand wrote: > I'm new to Varnish and I've seen a number of discussions where Varnish > is used in conjunction with a load balancer in the following manner: > [LB] -> [Varnish] -> [www 1-n] > > I'm guessing that Varnish is running locally on the www hosts? ?If > not, Is there any reason why placing Varnish in front of the load > balancers instead would not be ideal? > [Varnish] -> [LB} -> [www 1-n] Usually you use LB in front of varnish for high availability cases. Then, if a varnish fails, the other will take over all the work. But there's no problem using Varnish as a load balancer as it does a great job, as others said in this thread. But you must pay attention to the fail over/load balancing on the varnish as well :-) -- []'s Hugo www.devin.com.br From hugo.cisneiros at gmail.com Tue Jun 26 17:49:54 2012 From: hugo.cisneiros at gmail.com (Hugo Cisneiros (Eitch)) Date: Tue, 26 Jun 2012 14:49:54 -0300 Subject: Syntax for piping varnish logs to rotatelogs In-Reply-To: <002701cd53bd$5c2efcc0$148cf640$@co.uk> References: <1340382987.85063.YahooMailNeo@web43515.mail.sp1.yahoo.com> <002701cd53bd$5c2efcc0$148cf640$@co.uk> Message-ID: On Tue, Jun 26, 2012 at 2:01 PM, Rowan @ Jetboy wrote: > Ubuntu 12.04 Server x64, Varnish 3.0.2 > > I'm trying to pipe varnishncsa's logs through Apache's rotatelogs, and > running from the shell, things work fine: > > sudo varnishncsa -a -P /var/run/varnishncsa/varnishncsa.pid > |/usr/sbin/rotatelogs /var/log/varnish/varnish.log.%Y%m%d%H 3600 [...] I don't really know if this is the issue you're running, but maybe it its. Check this out: http://my.opera.com/cstrep/blog/2011/11/30/how-to-start-up-varnish-with-a-custom-cc-command-on-debian I had some problems adding the -F to varnishncsa daemon and it worked. Attaching the "bash -c" before the start-stop-daemon did the trick. Maybe the init script is ignoring your parameters for the same reason. -- []'s Hugo www.devin.com.br From allan.brand at gmail.com Tue Jun 26 17:57:44 2012 From: allan.brand at gmail.com (Allan Brand) Date: Tue, 26 Jun 2012 12:57:44 -0500 Subject: Load Balancing and Varnish In-Reply-To: References: Message-ID: Thanks! Does Varnish support least-connections or weighted least-connections load balancing? Also, in the scenario of placing a LB in front of Varnish, is the Varnish host a dedicated machine or is it running locally on the web servers? Allan On Tue, Jun 26, 2012 at 12:40 PM, Hugo Cisneiros (Eitch) wrote: > On Tue, Jun 26, 2012 at 1:46 PM, Allan Brand wrote: >> I'm new to Varnish and I've seen a number of discussions where Varnish >> is used in conjunction with a load balancer in the following manner: >> [LB] -> [Varnish] -> [www 1-n] >> >> I'm guessing that Varnish is running locally on the www hosts? ?If >> not, Is there any reason why placing Varnish in front of the load >> balancers instead would not be ideal? >> [Varnish] -> [LB} -> [www 1-n] > > Usually you use LB in front of varnish for high availability cases. > Then, if a varnish fails, the other will take over all the work. > > But there's no problem using Varnish as a load balancer as it does a > great job, as others said in this thread. But you must pay attention > to the fail over/load balancing on the varnish as well :-) > > -- > []'s > Hugo > www.devin.com.br > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc -- --------------------------------------- Allan (208) 977-6014 (eFax) From jc.bedier at gmail.com Tue Jun 26 20:24:54 2012 From: jc.bedier at gmail.com (Jean-Christian BEDIER) Date: Tue, 26 Jun 2012 22:24:54 +0200 Subject: Load Balancing and Varnish In-Reply-To: References: Message-ID: Real loadbalancer offer more functionality, like hash uri for sticky on specific varnish and reduce cache objet redundancy. On Tue, Jun 26, 2012 at 7:57 PM, Allan Brand wrote: > Thanks! ?Does Varnish support least-connections or weighted > least-connections load balancing? > > Also, in the scenario of placing a LB in front of Varnish, is the > Varnish host a dedicated machine or is it running locally on the web > servers? > > Allan > > On Tue, Jun 26, 2012 at 12:40 PM, Hugo Cisneiros (Eitch) > wrote: >> On Tue, Jun 26, 2012 at 1:46 PM, Allan Brand wrote: >>> I'm new to Varnish and I've seen a number of discussions where Varnish >>> is used in conjunction with a load balancer in the following manner: >>> [LB] -> [Varnish] -> [www 1-n] >>> >>> I'm guessing that Varnish is running locally on the www hosts? ?If >>> not, Is there any reason why placing Varnish in front of the load >>> balancers instead would not be ideal? >>> [Varnish] -> [LB} -> [www 1-n] >> >> Usually you use LB in front of varnish for high availability cases. >> Then, if a varnish fails, the other will take over all the work. >> >> But there's no problem using Varnish as a load balancer as it does a >> great job, as others said in this thread. But you must pay attention >> to the fail over/load balancing on the varnish as well :-) >> >> -- >> []'s >> Hugo >> www.devin.com.br >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > > > -- > --------------------------------------- > Allan > (208) 977-6014?(eFax) > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc From waplabs at gmail.com Tue Jun 26 21:45:34 2012 From: waplabs at gmail.com (Kamil) Date: Tue, 26 Jun 2012 23:45:34 +0200 Subject: Esi like mod_layout Message-ID: Hi I would like use Varnish like mod_layout but I can;t find how implement esi in all sites on my serwer automatically. It possible put in .vcl file or something else to include ads in all my websites? I can manually edit every public_html files on my sites to include it but I search something like mod_layout that that include it autmatically on all my new websites. Have you any idea? -- Pozdrawiam Kamil -------------- next part -------------- An HTML attachment was scrubbed... URL: From rowan at jetboy.co.uk Tue Jun 26 21:46:36 2012 From: rowan at jetboy.co.uk (Rowan @ Jetboy) Date: Tue, 26 Jun 2012 22:46:36 +0100 Subject: Syntax for piping varnish logs to rotatelogs In-Reply-To: References: <1340382987.85063.YahooMailNeo@web43515.mail.sp1.yahoo.com> <002701cd53bd$5c2efcc0$148cf640$@co.uk> Message-ID: <002e01cd53e5$297322c0$7c596840$@co.uk> Thanks Hugo, I think you've got it there. I've just had a response on Server Fault that seems to be based around the similar logic: http://serverfault.com/questions/402157/syntax-for-piping-varnish-logs-to-ro tatelogs/402506#402506 Am I right in thinking that this is the same limitation that prevents you passing arguments that contain spaces out from /etc/default/varnish and /etc/default/varnishncsa? Hacking init files seems like a bad idea, but there really doesn't seem to be another option. I'll give it a try and post back. > I'm trying to pipe varnishncsa's logs through Apache's rotatelogs, and > running from the shell, things work fine: > > sudo varnishncsa -a -P /var/run/varnishncsa/varnishncsa.pid > |/usr/sbin/rotatelogs /var/log/varnish/varnish.log.%Y%m%d%H 3600 [...] I don't really know if this is the issue you're running, but maybe it its. Check this out: http://my.opera.com/cstrep/blog/2011/11/30/how-to-start-up-varnish-with-a-cu stom-cc-command-on-debian I had some problems adding the -F to varnishncsa daemon and it worked. Attaching the "bash -c" before the start-stop-daemon did the trick. Maybe the init script is ignoring your parameters for the same reason. From hugo.cisneiros at gmail.com Tue Jun 26 21:50:10 2012 From: hugo.cisneiros at gmail.com (Hugo Cisneiros (Eitch)) Date: Tue, 26 Jun 2012 18:50:10 -0300 Subject: Syntax for piping varnish logs to rotatelogs In-Reply-To: <002e01cd53e5$297322c0$7c596840$@co.uk> References: <1340382987.85063.YahooMailNeo@web43515.mail.sp1.yahoo.com> <002701cd53bd$5c2efcc0$148cf640$@co.uk> <002e01cd53e5$297322c0$7c596840$@co.uk> Message-ID: On Tue, Jun 26, 2012 at 6:46 PM, Rowan @ Jetboy wrote: > Thanks Hugo, > > I think you've got it there. I've just had a response on Server Fault that > seems to be based around the similar logic: > > http://serverfault.com/questions/402157/syntax-for-piping-varnish-logs-to-ro > tatelogs/402506#402506 > > Am I right in thinking that this is the same limitation that prevents you > passing arguments that contain spaces out from /etc/default/varnish and > /etc/default/varnishncsa? Hacking init files seems like a bad idea, but > there really doesn't seem to be another option. I'll give it a try and post > back. Yes, it's the same bug. And since it's a bug, I think it's OK to hack the init script until the upstream fixes it (and hopefully they will fix :) -- []'s Hugo www.devin.com.br From abraham.sustaita at buscacorp.com Wed Jun 27 05:55:49 2012 From: abraham.sustaita at buscacorp.com (Abraham Cruz Sustaita) Date: Tue, 26 Jun 2012 22:55:49 -0700 Subject: Varnish Close the connection on long files Message-ID: We have a lot of videos. Some of them of 500Mb, and when the users want to dowload them, varnish close the connection before the file is fully downloaded. We know this is the problem cause everytime we receive the same file but with differents sizes. If the connection is slow, the file is smaller. I really can't figure where the problem is... This is the configuration I have: backend app01 { .host = "app01.xxxxxx.com"; .port = "80"; .connect_timeout = 1.5s; .first_byte_timeout = 45s; .between_bytes_timeout = 30s; .probe = { .url = "/"; .interval = 5s; .timeout = 1s; .window = 5; .threshold = 3; } } director app_director round-robin { { .backend = app01; } } #acl internal { # "localhost"; # "127.0.0.1"; #} acl purge { "localhost"; "192.168.11.0"/24; } sub vcl_recv { if (req.request == "PURGE") { if (!client.ip ~ purge) { error 405 "Not allowed."; } ban("req.url = " + req.url); return (lookup); } # Normalize the host header # -- if (req.http.Host ~ "(?i)^(www.)?xxxxx.com") { set req.http.host = "www.xxxxxxx.com"; } # Use anonymous, cached pages if all backends are down. if (! req.backend.healthy) { unset req.http.Cookie; set req.grace = 1h; } else { set req.grace = 30s; } # Add the X-Forwarded-For header # -- if (req.restarts == 0) { if (req.http.x-forwarded-for) { set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip; } else { set req.http.X-Forwarded-For = client.ip; } } # Sanely handle the request based on the type # -- if (req.request != "GET" && req.request != "HEAD" && req.request != "PUT" && req.request != "POST" && req.request != "TRACE" && req.request != "OPTIONS" && req.request != "DELETE") { /* Non-RFC2616 or CONNECT which is weird. */ return (pipe); } # Request by post needs no cache # -- if (req.request == "POST") { return (pass); } # Admin needs no cache # -- if (req.http.Host ~ "(?i)^(admin.)?xxxxxx.com") { return (pass); } # Always cache the following file types for all users. if (req.url ~ "(?i)\.(png|gif|jpeg|jpg|ico|swf|flv|avi|mov|wmv|css|js|html|htm)(\?[a-z0-9]+)?$") { unset req.http.Cookie; return (lookup); } # Login and logout needs no cache # -- if (req.url ~ "^/usuario/salir" || req.url ~ "^/usuario/redirectWindow" || req.url ~ "^/usuario/al") { return (pass); } # No save at cache the user profile if (req.url ~ "^/usuario/([0-9]+)/(.*?)/") { return (pass); } # Requests to /no-cache/ needs no cache # -- if (req.request == "GET" && req.url ~ "^/no-cache") { return (pass); } if (req.request == "GET" && req.url ~ "^/rss/xxxxx") { return (pass); } # Not sure what could match here, but it's in the default.vcl # -- if (req.request != "GET" && req.request != "HEAD") { /* We only deal with GET and HEAD by default */ return (pass); } # Handle compression correctly. Different browsers send different # "Accept-Encoding" headers, even though they mostly all support the same # compression mechanisms. By consolidating these compression headers into # a consistent format, we can reduce the size of the cache and get more hits.= # @see: http:// varnish.projects.linpro.no/wiki/FAQ/Compression if (req.http.Accept-Encoding) { if (req.http.Accept-Encoding ~ "gzip") { # If the browser supports it, we'll use gzip. set req.http.Accept-Encoding = "gzip"; } else if (req.http.Accept-Encoding ~ "deflate") { # Next, try deflate if it is supported. set req.http.Accept-Encoding = "deflate"; } else { # Unknown algorithm. Remove it and send unencoded. unset req.http.Accept-Encoding; } } # Cache all requests by default, overriding the standard Varnish behavior if (req.request == "GET" || req.request == "HEAD") { /* We only deal with GET and HEAD by default */ #unset req.http.Cookie; return (lookup); } # If there's a cookie left aft this point do not cache # or is this easier to keep in the cookie deletion above? # -- if (req.http.Authorization || req.http.Cookie) { /* Not cacheable by default */ return (pass); } # If we get here, try the cache # -- return (lookup); } sub vcl_pipe { return (pipe); } sub vcl_pass { return (pass); } sub vcl_hash { # Include cookie in cache hash. # This check is unnecessary because we already pass on all cookies. #if (req.http.Cookie) { # #set req.hash += req.http.Cookie; # hash_data(req.http.Cookie); #} } sub vcl_hit { if (req.request == "PURGE") { purge; error 200 "Purged"; } } sub vcl_miss { if (req.request == "PURGE") { purge; error 200 "Purged"; } } sub vcl_fetch { # remove all cookies # unset beresp.http.set-cookie; # cache for 12 hours # set beresp.ttl = 2h; # Don't allow static files to set cookies. if (req.url ~ "(?i)\.(png|gif|jpeg|jpg|ico|swf|css|js|html|htm|mp4|flv)(\?[a-z0-9]+)?$") { unset beresp.http.set-cookie; set beresp.ttl = 12h; } else { set beresp.ttl = 30m; } # If the backend server doesn't return properly, don't send another connection to it # for 60s and try another backend via restart. # # https://www.varnish-cache.org/docs/trunk/tutorial/handling_misbehaving_servers.html # -- if(beresp.status == 500 || beresp.status == 503) { if (req.restarts > 3) { set beresp.saintmode = 5m; } if (req.request != "POST") { return(restart); } else { error 500 "Failed"; } } # Allow items to be stale if needed. set beresp.grace = 1h; } sub vcl_deliver { # if (resp.http.magicmarker) { # unset resp.http.magicmarker; # set resp.http.age = "0"; # } if (obj.hits > 0) { set resp.http.X-Cache = "HIT"; } else { set resp.http.X-Cache = "MISS"; } return (deliver); } # In the event of an error, show friendlier messages. sub vcl_error { # Otherwise redirect to the homepage, which will likely be in the cache. set obj.http.Content-Type = "text/html; charset=utf-8"; synthetic {" ..:: XXXXX :: Por el momento estamos en mantenimiento ::..
Estamos experimentando problemas. Vuelve más tarde.
(Error "} + obj.status + " " + obj.response + {")
"}; return (deliver); } sub vcl_init { return (ok); } sub vcl_fini { return (ok); } -------------- next part -------------- An HTML attachment was scrubbed... URL: From bjorn.jungskar at motorola.com Wed Jun 27 06:19:21 2012 From: bjorn.jungskar at motorola.com (Bjorn Jungskar) Date: Wed, 27 Jun 2012 08:19:21 +0200 Subject: Referencing backend definition in one include file from another include file Message-ID: Hi, I have one Varnish VCL file that has two include statements, one that allows for custom backend definitions and one that allows for custom URL mappings. The problem is that I get compilation errors when referencing the backend defined in the included file from the other included file. Example: main.vcl: ------------ backend default { .host = "127.0.0.1"; .port = "8080"; } include "custom_backends.vcl"; sub vcl_recv { if ( req.url ~ "/someurl/" ) { req.backend = default; return (pass); } include "custom_mappings.vcl"; } custom_backends.vcl: -------------------------------- backend mybackend { .host = "127.0.0.1"; .port = "1234"; } custom_mappings.vcl: -------------------------------- if ( req.url ~ "/some_other_url/" ) { req.backend = mybackend; return (pass); } Also, it is not possible to define the custom backends directly in the custom_mappings file since a backend can not be defined inside a function. Is there any way to get around this problem? Regards, Bjorn From perbu at varnish-software.com Wed Jun 27 09:40:59 2012 From: perbu at varnish-software.com (Per Buer) Date: Wed, 27 Jun 2012 11:40:59 +0200 Subject: Load Balancing and Varnish In-Reply-To: References: Message-ID: Hi. Varnish can do that. VCL is a powerful thing. Use the client director and set client.identity to req.url. It might be that "client.identity" is a bad name. Maybe it should be renamed. Per. On Tue, Jun 26, 2012 at 10:24 PM, Jean-Christian BEDIER wrote: > Real loadbalancer offer more functionality, like hash uri for sticky > on specific varnish and reduce cache objet redundancy. > > On Tue, Jun 26, 2012 at 7:57 PM, Allan Brand > wrote: > > Thanks! Does Varnish support least-connections or weighted > > least-connections load balancing? > > > > Also, in the scenario of placing a LB in front of Varnish, is the > > Varnish host a dedicated machine or is it running locally on the web > > servers? > > > > Allan > > > > On Tue, Jun 26, 2012 at 12:40 PM, Hugo Cisneiros (Eitch) > > wrote: > >> On Tue, Jun 26, 2012 at 1:46 PM, Allan Brand > wrote: > >>> I'm new to Varnish and I've seen a number of discussions where Varnish > >>> is used in conjunction with a load balancer in the following manner: > >>> [LB] -> [Varnish] -> [www 1-n] > >>> > >>> I'm guessing that Varnish is running locally on the www hosts? If > >>> not, Is there any reason why placing Varnish in front of the load > >>> balancers instead would not be ideal? > >>> [Varnish] -> [LB} -> [www 1-n] > >> > >> Usually you use LB in front of varnish for high availability cases. > >> Then, if a varnish fails, the other will take over all the work. > >> > >> But there's no problem using Varnish as a load balancer as it does a > >> great job, as others said in this thread. But you must pay attention > >> to the fail over/load balancing on the varnish as well :-) > >> > >> -- > >> []'s > >> Hugo > >> www.devin.com.br > >> > >> _______________________________________________ > >> varnish-misc mailing list > >> varnish-misc at varnish-cache.org > >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > > > > > > > -- > > --------------------------------------- > > Allan > > (208) 977-6014 (eFax) > > > > _______________________________________________ > > varnish-misc mailing list > > varnish-misc at varnish-cache.org > > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -- Per Buer Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer *Varnish makes websites fly!* Whitepapers | Video | Twitter -------------- next part -------------- An HTML attachment was scrubbed... URL: From perbu at varnish-software.com Wed Jun 27 09:43:52 2012 From: perbu at varnish-software.com (Per Buer) Date: Wed, 27 Jun 2012 11:43:52 +0200 Subject: Referencing backend definition in one include file from another include file In-Reply-To: References: Message-ID: Hi Bjorn. On Wed, Jun 27, 2012 at 8:19 AM, Bjorn Jungskar wrote: > Hi, > > I have one Varnish VCL file that has two include statements, one that > allows for custom backend definitions and one that allows for custom > URL mappings. The problem is that I get compilation errors when > referencing the backend defined in the included file from the other > included file. > I think the parameter your looking for is vcc_err_unref. Setting it to "off" will result in a warning, not an error. -- Per Buer Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer *Varnish makes websites fly!* Whitepapers | Video | Twitter -------------- next part -------------- An HTML attachment was scrubbed... URL: From ghfg at co.sapo.pt Wed Jun 27 12:29:36 2012 From: ghfg at co.sapo.pt (Goncalo Garcia) Date: Wed, 27 Jun 2012 13:29:36 +0100 Subject: Varnish Close the connection on long files In-Reply-To: References: Message-ID: <20120627132936.2d2d80ce@valkyrie> On Tue, 26 Jun 2012 22:55:49 -0700 Abraham Cruz Sustaita wrote: > We have a lot of videos. Some of them of 500Mb, and when the users > want to dowload them, varnish close the connection before the file is > fully downloaded. We know this is the problem cause everytime we > receive the same file but with differents sizes. If the connection is > slow, the file is smaller. I really can't figure where the problem > is... Hi, have you looked at the send_timeout option in varnishd? % man varnishd "Send timeout for client connections. If the HTTP response hasn't been transmitted in this many seconds the session is closed. See setsockopt(2) under SO_SNDTIMEO for more information." We had a similar situation with some videos, after 60 seconds(default) the download was interrupted. ghfg > This is the configuration I have: > > backend app01 { > .host = "app01.xxxxxx.com"; > .port = "80"; > .connect_timeout = 1.5s; > .first_byte_timeout = 45s; > .between_bytes_timeout = 30s; > .probe = { > .url = "/"; > .interval = 5s; > .timeout = 1s; > .window = 5; > .threshold = 3; > } > } > director app_director round-robin { > { .backend = app01; } > } > > #acl internal { > # "localhost"; > # "127.0.0.1"; > #} > > acl purge { > "localhost"; > "192.168.11.0"/24; > } > > sub vcl_recv { > > if (req.request == "PURGE") { > if (!client.ip ~ purge) { > error 405 "Not allowed."; > } > ban("req.url = " + req.url); > return (lookup); > } > > # Normalize the host header > # -- > if (req.http.Host ~ "(?i)^(www.)?xxxxx.com") { > set req.http.host = "www.xxxxxxx.com"; > } > > # Use anonymous, cached pages if all backends are down. > if (! req.backend.healthy) { > unset req.http.Cookie; > set req.grace = 1h; > } else { > set req.grace = 30s; > } > > # Add the X-Forwarded-For header > # -- > if (req.restarts == 0) { > if (req.http.x-forwarded-for) { > set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + > client.ip; > } else { > set req.http.X-Forwarded-For = client.ip; > } > } > > # Sanely handle the request based on the type > # -- > if (req.request != "GET" && > req.request != "HEAD" && > req.request != "PUT" && > req.request != "POST" && > req.request != "TRACE" && > req.request != "OPTIONS" && > req.request != "DELETE") { > /* Non-RFC2616 or CONNECT which is weird. */ > return (pipe); > } > > # Request by post needs no cache > # -- > if (req.request == "POST") { > return (pass); > } > > # Admin needs no cache > # -- > if (req.http.Host ~ "(?i)^(admin.)?xxxxxx.com") { > return (pass); > } > > # Always cache the following file types for all users. > if (req.url ~ > "(?i)\.(png|gif|jpeg|jpg|ico|swf|flv|avi|mov|wmv|css|js|html|htm)(\?[a-z0-9]+)?$") > { > unset req.http.Cookie; > return (lookup); > } > > # Login and logout needs no cache > # -- > if (req.url ~ "^/usuario/salir" || req.url ~ > "^/usuario/redirectWindow" || req.url ~ "^/usuario/al") { > return (pass); > } > > # No save at cache the user profile > if (req.url ~ "^/usuario/([0-9]+)/(.*?)/") { > return (pass); > } > > # Requests to /no-cache/ needs no cache > # -- > if (req.request == "GET" && req.url ~ "^/no-cache") { > return (pass); > } > if (req.request == "GET" && req.url ~ "^/rss/xxxxx") { > return (pass); > } > > # Not sure what could match here, but it's in the default.vcl > # -- > if (req.request != "GET" && req.request != "HEAD") { > /* We only deal with GET and HEAD by default */ > return (pass); > } > > # Handle compression correctly. Different browsers send different > # "Accept-Encoding" headers, even though they mostly all support > the same # compression mechanisms. By consolidating these compression > headers into # a consistent format, we can reduce the size of the > cache and get more hits.= > # @see: http:// varnish.projects.linpro.no/wiki/FAQ/Compression > if (req.http.Accept-Encoding) { > if (req.http.Accept-Encoding ~ "gzip") { > # If the browser supports it, we'll use gzip. > set req.http.Accept-Encoding = "gzip"; > } > else if (req.http.Accept-Encoding ~ "deflate") { > # Next, try deflate if it is supported. > set req.http.Accept-Encoding = "deflate"; > } > else { > # Unknown algorithm. Remove it and send unencoded. > unset req.http.Accept-Encoding; > } > } > > # Cache all requests by default, overriding the standard Varnish > behavior if (req.request == "GET" || req.request == "HEAD") { > /* We only deal with GET and HEAD by default */ > #unset req.http.Cookie; > return (lookup); > } > > # If there's a cookie left aft this point do not cache > # or is this easier to keep in the cookie deletion above? > # -- > if (req.http.Authorization || req.http.Cookie) { > /* Not cacheable by default */ > return (pass); > } > > # If we get here, try the cache > # -- > return (lookup); > > } > > sub vcl_pipe { > return (pipe); > } > > sub vcl_pass { > return (pass); > } > > sub vcl_hash { > # Include cookie in cache hash. > # This check is unnecessary because we already pass on all cookies. > #if (req.http.Cookie) { > # #set req.hash += req.http.Cookie; > # hash_data(req.http.Cookie); > #} > } > > sub vcl_hit { > if (req.request == "PURGE") { > purge; > error 200 "Purged"; > } > } > > sub vcl_miss { > if (req.request == "PURGE") { > purge; > error 200 "Purged"; > } > } > > sub vcl_fetch { > > > # remove all cookies > # unset beresp.http.set-cookie; > > # cache for 12 hours > # set beresp.ttl = 2h; > > # Don't allow static files to set cookies. > if (req.url ~ > "(?i)\.(png|gif|jpeg|jpg|ico|swf|css|js|html|htm|mp4|flv)(\?[a-z0-9]+)?$") > { unset beresp.http.set-cookie; > set beresp.ttl = 12h; > } else { > set beresp.ttl = 30m; > } > > # If the backend server doesn't return properly, don't send another > connection to it > # for 60s and try another backend via restart. > # > # > https://www.varnish-cache.org/docs/trunk/tutorial/handling_misbehaving_servers.html > # -- > if(beresp.status == 500 || beresp.status == 503) { > if (req.restarts > 3) { > set beresp.saintmode = 5m; > } > if (req.request != "POST") { > return(restart); > } else { > error 500 "Failed"; > } > } > > # Allow items to be stale if needed. > set beresp.grace = 1h; > > } > > sub vcl_deliver { > # if (resp.http.magicmarker) { > # unset resp.http.magicmarker; > # set resp.http.age = "0"; > # } > > if (obj.hits > 0) { > set resp.http.X-Cache = "HIT"; > } else { > set resp.http.X-Cache = "MISS"; > } > return (deliver); > } > > # In the event of an error, show friendlier messages. > sub vcl_error { > # Otherwise redirect to the homepage, which will likely be in the > cache. set obj.http.Content-Type = "text/html; charset=utf-8"; > synthetic {" > http://www.w3.org/TR/html4/loose.dtd"> > > > > ..:: XXXXX :: Por el momento estamos en mantenimiento > ::.. > > > > > >
Estamos experimentando problemas. Vuelve > más tarde.
>
(Error "} + obj.status + " " + > obj.response > + {")
> > > "}; > return (deliver); > } > > sub vcl_init { > return (ok); > } > > sub vcl_fini { > return (ok); > } -- Chaos umpire sits, And by decision more embroils the fray By which he reigns: next him high arbiter Chance governs all. -- Milton, Paradise Lost -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: not available URL: From Travis.Crowder at penton.com Wed Jun 27 14:35:32 2012 From: Travis.Crowder at penton.com (Crowder, Travis) Date: Wed, 27 Jun 2012 14:35:32 +0000 Subject: Varnish send_timeout Message-ID: <61E9173DF681274790BA29C89A72200C0268A27F@ksopexchm02.pme.penton.com> This seems to be in reference to Ticker #402 (closed defect: fixed) The fix seems to be changing the documentation, but I don't know if that's proper. My version of varnish is "varnish-3.0.2 revision 55e70a4" I have been having users get incomplete/broken PDF files and have been having issues replicating the problem. However, I seem to be able to replicate the problem on a slow connection. With Varnish using the default 60s send_timeout, I get broken and reestablished connections. If I bump this to 600s, the problem goes away. I used wget with a 20k rate to download a 13M PDF file ( screenshot http://i46.tinypic.com/1zz6y4x.png ) to get the broken downloads. Is this expected behavior? Killing the connection despite being active? Am I just flat out doing it wrong? Thanks, Travis -------------- next part -------------- An HTML attachment was scrubbed... URL: From perbu at varnish-software.com Wed Jun 27 19:58:17 2012 From: perbu at varnish-software.com (Per Buer) Date: Wed, 27 Jun 2012 21:58:17 +0200 Subject: Varnish send_timeout In-Reply-To: <61E9173DF681274790BA29C89A72200C0268A27F@ksopexchm02.pme.penton.com> References: <61E9173DF681274790BA29C89A72200C0268A27F@ksopexchm02.pme.penton.com> Message-ID: Hi Travis. On Wed, Jun 27, 2012 at 4:35 PM, Crowder, Travis wrote: > > > ** > > With Varnish using the default 60s send_timeout, I get broken and > reestablished connections. If I bump this to 600s, the problem goes away. > Yes. This is a known bug. Please try 3.0.3 RC1 - it is fixed there, I believe. -- Per Buer Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer *Varnish makes websites fly!* Whitepapers | Video | Twitter -------------- next part -------------- An HTML attachment was scrubbed... URL: From fred at clift.org Wed Jun 27 22:36:14 2012 From: fred at clift.org (Fred Clift) Date: Wed, 27 Jun 2012 16:36:14 -0600 Subject: vcl newbie question Message-ID: I'm going to have many domains being proxied by varnish and I'd like to be able to invalidate by doing a PURGE on a different IP than the domain... At system startup, and as an emergency tool for system administrators, I BAN entire domains. For my whole-domain BANs my vcl looks like: # allow HOSTBAN from localhost and hosts in purge acl # HOSTBAN covers entire domain if (req.request == "HOSTBAN") { # Same ACL check as above: if (!client.ip ~ purge) { error 405 "Not allowed."; } std.log("trying to ban " + req.url); ban("req.http.host == " + req.url); # Throw a synthetic page so the # request won't go to the backend. error 200 "Ban added"; } and my python code does the following: def do_hostban(cachehost, domain): conn = httplib.HTTPConnection(cachehost) conn.request("HOSTBAN", domain) rl = conn.getresponse() conn.close() debugprint( "%s %s" % (rl.status, rl.reason)) return rl.status, rl.reason do_hostban("cacheIP", "www.bandomain.com") This does approximately the same as you can with curl, without all that nasty shell interpolation: /usr/bin/curl -s -X HOSTBAN "http://cacheIP/www.bandomain.com" I'd like to keep the cache host/ip separate from the domain(s) being hosted so that I could proxy or aggregate these at some central point (from several servers) without having to change much. So, my real problem is that I really only have a very superficial understanding of vcl. for purges I have: sub vcl_recv { # allow PURGE from localhost and hosts in purge acl # PURGE does a single, unique URL (+Vary-iants) if (req.request == "PURGE") { if (!client.ip ~ purge) { error 405 "Not allowed."; } return (lookup); } } sub vcl_hit { if (req.request == "PURGE") { purge; error 200 "Purged."; } } sub vcl_miss { if (req.request == "PURGE") { purge; error 200 "Purged."; } } How would I modify this code so that I could purge via approximately the following syntax? http://varnishIP/www.purgedomain.com/some/url.php I've stared at the docs for a while, and ended up going back t work on other parts of my project while hoping for enlightenment. Fred From abraham.sustaita at buscacorp.com Wed Jun 27 23:42:36 2012 From: abraham.sustaita at buscacorp.com (Abraham Cruz Sustaita) Date: Wed, 27 Jun 2012 16:42:36 -0700 Subject: Problem with Amazon S3 and varnish Message-ID: I have this configuration: # Amazon S3 Backend Metatube # -- backend s3 { .host = "metatube.s3-website-us-east-1.amazonaws.com"; .port = "80"; } But the amazon url sometimes resolves to more of one IP (I guess is a normal behaviour from Amazon), so I have to try several times to load the configuration file (via varnishadm) until it compiles correctly. The problem is I have a 5 different domains with amazon, so when finally one of them resolves to just one IP, the others resolve to more than one, so I can never compile the configuration file. Is there anyway to fix this via configuration? -------------- next part -------------- An HTML attachment was scrubbed... URL: From hugo.cisneiros at gmail.com Thu Jun 28 18:06:23 2012 From: hugo.cisneiros at gmail.com (Hugo Cisneiros (Eitch)) Date: Thu, 28 Jun 2012 15:06:23 -0300 Subject: Problem with Amazon S3 and varnish In-Reply-To: References: Message-ID: On Wed, Jun 27, 2012 at 8:42 PM, Abraham Cruz Sustaita wrote: > I have this configuration: > > # Amazon S3 Backend Metatube > # -- > backend s3 { > ? .host = "metatube.s3-website-us-east-1.amazonaws.com"; > ? .port = "80"; > } > > But the amazon url sometimes resolves to more of one IP (I guess is a normal > behaviour from Amazon), so I have to try several times to load the > configuration file (via varnishadm) until it compiles correctly. The problem > is I have a 5 different domains with amazon, so when finally one of them > resolves to just one IP, the others resolve to more than one, so I can never > compile the configuration file. Is there anyway to fix this via > configuration? Currently varnish does not support multiple IPs on one backend. There's no easy and native way to do this. Please see the mailing list archives for many discussions and some approaches about this issue :) -- []'s Hugo www.devin.com.br From xhrfc at yahoo.com Fri Jun 29 08:53:37 2012 From: xhrfc at yahoo.com (X.H.R.F.C.) Date: Fri, 29 Jun 2012 01:53:37 -0700 (PDT) Subject: varnish post multipart/form-data Message-ID: <1340960017.61719.YahooMailNeo@web43516.mail.sp1.yahoo.com> Is there any special setting to activate varnish post multipart/form-data ? all my posts are working, not when i'm uploading files. I get Error 503 Service Unavailable -------------- next part -------------- An HTML attachment was scrubbed... URL: From lusurf at foxmail.com Fri Jun 29 10:50:13 2012 From: lusurf at foxmail.com (Lusurf Lui) Date: Fri, 29 Jun 2012 18:50:13 +0800 Subject: High BackendReuse Rate Problem Message-ID: <4FED8865.1030606@foxmail.com> Hi, I'm using the Varnish-Cache-3.0.2 and have been encountered a very strange problem. I use a random director which is defined like this: ---------------------------------------------------- director randomlb random { { .backend = app1019; .weight = 15; } { .backend = app1018; .weight = 15; } { .backend = app1020; .weight = 15; } { .backend = app1021; .weight = 15; } } ---------------------------------------------------- The problem is that the BackendReuse rate of app1018 is significantly higher than the other backends. And this is a simple test case. ---------------------------------------------------- 526 BackendReuse app1018 ### 988 BackendReuse app1019 ### 561 BackendReuse app1020 573 BackendReuse app1021 ---------------------------------------------------- What should i do to gain a balanced backendreuse rate for all the backends? Thanks. --- Lusurf From javier at casares.org Fri Jun 29 10:56:47 2012 From: javier at casares.org (Javier Casares) Date: Fri, 29 Jun 2012 12:56:47 +0200 Subject: High BackendReuse Rate Problem In-Reply-To: <4FED8865.1030606@foxmail.com> References: <4FED8865.1030606@foxmail.com> Message-ID: I usually see the same... the first backend in a director taks a little bit more traffic that the others... I see it in 3.0.1 and 3.0.2, not tried in 3.0.3 Javier Casares http://javiercasares.com/ 2012/6/29 Lusurf Lui : > Hi, > > I'm using the Varnish-Cache-3.0.2 and have been encountered a very > strange problem. > > I use a random director which is defined like this: > ---------------------------------------------------- > director randomlb random { > ? ?{ > ? ? ? ?.backend = app1019; > ? ? ? ?.weight = 15; > ? ?} > ? ?{ > ? ? ? ?.backend = app1018; > ? ? ? ?.weight = 15; > ? ?} > ? ?{ > ? ? ? ?.backend = app1020; > ? ? ? ?.weight = 15; > ? ?} > ? ?{ > ? ? ? ?.backend = app1021; > ? ? ? ?.weight = 15; > ? ?} > } > ---------------------------------------------------- > The problem is that the BackendReuse rate of app1018 is significantly > higher than the other backends. > And this is a simple test case. > ---------------------------------------------------- > ? ?526 BackendReuse app1018 > ? ?### > ? ?988 BackendReuse app1019 > ? ?### > ? ?561 BackendReuse app1020 > ? ?573 BackendReuse app1021 > ---------------------------------------------------- > > What should i do to gain a balanced backendreuse rate for all the backends? > > Thanks. > > --- > Lusurf > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc From Travis.Crowder at penton.com Fri Jun 22 18:42:26 2012 From: Travis.Crowder at penton.com (Crowder, Travis) Date: Fri, 22 Jun 2012 18:42:26 -0000 Subject: RAM usage beyond specified amount Message-ID: <61E9173DF681274790BA29C89A72200C0267EF92@ksopexchm02.pme.penton.com> I recall seeing on the mailing list before a reason why Varnish would use more RAM than allocated due to storing something outside of the defined allocated memory. For example, I have a box that is allocated 6G of RAM, but it is using about 12G virtual memory. >top 9101 nobody 20 0 11.7g 6.8g 81m S 6.0 88.5 1270:21 varnishd Varnish is invoked via: /usr/local/sbin/varnishd -f /usr/local/etc/varnish/default.vcl -s malloc,6G -T xxx.xxx.xxx.xxx:2000 -a :80 Does anyone know what I am talking about? Thanks, Travis -------------- next part -------------- An HTML attachment was scrubbed... URL: