From varnish-cache at otoh.org Tue Jan 7 19:15:27 2014 From: varnish-cache at otoh.org (Paul Armstrong) Date: Tue, 7 Jan 2014 19:15:27 +0000 Subject: Disabling close in vcl_error leaks n_sess_mem Message-ID: <20140107191526.GU19190@suricate.otoh.org> Originally posted to varnish-dev, but varnish-misc was suggested as a better place. For our use case, we need to disable the default behavior of closing connections whenever vcl_error is called so that we can generate synthetic responses to various error conditions without causing connections to be constantly re-established (our use case is high speed traffic over connections which are kept alive for minutes at a time). We've applied the following patch in an effort to do this, but we keep on seeing leaks of n_sess_mem (it routinely bumps up against the 100,000 limit when we have high levels of traffic responded to synthetically). --- varnish-3.0.5/bin/varnishd/cache_center.c.orig 2013-12-02 +08:06:30.000000000 +0000 +++ varnish-3.0.5/bin/varnishd/cache_center.c 2013-12-31 +22:54:57.094893222 +0000 @@ -495,13 +495,18 @@ /* We always close when we take this path */ - sp->doclose = "error"; + // sp->doclose = "error"; sp->wantbody = 1; assert(sp->handling == VCL_RET_DELIVER); sp->err_code = 0; sp->err_reason = NULL; http_Setup(sp->wrk->bereq, NULL); + if (sp->obj->objcore != NULL) { + AN(sp->obj->objcore); + AN(sp->obj->objcore->ban); + HSH_Unbusy(sp); + } sp->step = STP_PREPRESP; return (0); } Obviously, we're missing some point in the cleanup process, but we're having trouble identifying it and some assistance would be appreciated. Anyone have ideas on what we're missing or where we should start in hunting this down? Thanks, Paul From timo.myyra at edita.fi Thu Jan 9 13:22:25 2014 From: timo.myyra at edita.fi (=?ISO-8859-1?Q?Myyr=E4=2C_Timo?=) Date: Thu, 9 Jan 2014 15:22:25 +0200 Subject: Problem setting cookie based client identity Message-ID: Hi, I'm having trouble setting up cookie based client identity for one of our sites. I'm planning on sanitising the Cookie value to only include customer specific session cookie and removing other values. Here's the relevant parts from vcl_recv: std.log("Cookie before sanitizing: " + req.http.Cookie); if (req.http.Cookie) { set req.http.Cookie = ";" + req.http.Cookie; set req.http.Cookie = regsuball(req.http.Cookie, "; +", ";"); set req.http.Cookie = regsuball(req.http.Cookie, ";([A-Z]+_SESSION|PHPSESSID)", "; \1"); set req.http.Cookie = regsuball(req.http.Cookie, ";[^ ][^;]*", ""); set req.http.Cookie = regsuball(req.http.Cookie, "^[; ]+|[; ]+$", ""); if (req.http.Cookie == "") { remove req.http.Cookie; } } std.log("Cookie after sanitizing: " + req.http.Cookie); if (req.http.Cookie) { set client.identity = req.http.Cookie; } else { set client.identity = client.ip; } std.log("Client identity set to " + client.identity); # Remove cookie from static resources if (req.url ~ "^assets") { remove req.http.Cookie; } Above config seems to work on when I make simple login and page tests via curl: $ curl --cookie "FOO_SESSION=ah5ai7qaej4a006o79s9500ao5; expires=Thu, 09-Jan-2014 20:45:01 GMT; path=/foo; domain=https://wwwdev.example.com/; secure; HttpOnly" -L -k --noproxy \* -i " https://wwwdev.example.com/foo/fi/product/100490" HTTP/1.1 200 OK Expires: Thu, 19 Nov 1981 08:52:00 GMT Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Pragma: no-cache Content-Type: text/html; charset=UTF-8 Content-Length: 22511 Accept-Ranges: bytes Date: Thu, 09 Jan 2014 13:09:34 GMT Age: 0 Connection: close X-Cache: MISS ... # varnishlog -m "RxHeader:(Linux|curl)" | egrep "(time|sanitizing|SESSION|identity)" 11 RxHeader c Cookie: FOO_SESSION=ah5ai7qaej4a006o79s9500ao5; expires=Thu, 09-Jan-2014 20:45:01 GMT; path=/foo; domain= https://wwwdev.example.com/; secure; HttpOnly 11 VCL_Log c Cookie before sanitizing: FOO_SESSION=ah5ai7qaej4a006o79s9500ao5; expires=Thu, 09-Jan-2014 20:45:01 GMT; path=/foo; domain=https://wwwdev.example.com/; secure; HttpOnly 11 VCL_Log c Cookie after sanitizing: FOO_SESSION=ah5ai7qaej4a006o79s9500ao5 11 VCL_Log c Client identity set to FOO_SESSION=ah5ai7qaej4a006o79s9500ao5 Problems is that when I test the site via Firefox I can login succesfully to site but when I click any link on the main page I get presented with the login page again. Seems like the varnish just droppes my cookie and wants to re-authenticate me. Here's snap of firefox varnishlog: 11 VCL_Log c Cookie before sanitizing: __utma=190219199.1951064551.1362571635.1387260530.1389248604.60; __utmz=190219199.1365759373.8... 11 VCL_Log c Cookie after sanitizing: 11 VCL_Log c Client identity set to 192.49.50.88 11 ObjHeader c Set-Cookie: FOO_SESSION=0ke75g55tvl8uglm3q95kjqf82; expires=Thu, 09-Jan-2014 21:14:38 GMT; path=/foo; domain= https://wwwdev.example.com/; secure; HttpOnly 11 TxHeader c Set-Cookie: FOO_SESSION=0ke75g55tvl8uglm3q95kjqf82; expires=Thu, 09-Jan-2014 21:14:38 GMT; path=/foo; domain= https://wwwdev.example.com/; secure; HttpOnly 11 VCL_Log c Cookie before sanitizing: __utma=190219199.1951064551.1362571635.1387260530.1389248604.60; __utmz=190219199.1365759373.8.2... 11 VCL_Log c Cookie after sanitizing: 11 VCL_Log c Client identity set to 192.49.50.88 As shown above, Varnish gets Set-Cookie from the application but doesn't seem to "stick" to cookie. The cookie just contains the google analytics stuff and not the FOO_SESSION as expected. I tried to remove the secure and HttpOnly flags from cookie but that didn't have any effect. Any idea what is happening here and how to get this working? Best regards, Timo Myyr? -------------- next part -------------- An HTML attachment was scrubbed... URL: From perbu at varnish-software.com Thu Jan 9 18:51:47 2014 From: perbu at varnish-software.com (Per Buer) Date: Thu, 9 Jan 2014 19:51:47 +0100 Subject: Problem setting cookie based client identity In-Reply-To: References: Message-ID: Hi Timo, On Thu, Jan 9, 2014 at 2:22 PM, Myyr?, Timo wrote: > Hi, > > I'm having trouble setting up cookie based client identity for one of our > sites. > I'm planning on sanitising the Cookie value to only include customer > specific session cookie and removing other values. > Ok. > > Here's the relevant parts from vcl_recv: > (..) > # Remove cookie from static resources > if (req.url ~ "^assets") { > remove req.http.Cookie; > } > This code probably won't be triggered as urls start with /. > Above config seems to work on when I make simple login and page tests via > curl: > $ curl --cookie "FOO_SESSION=ah5ai7qaej4a006o79s9500ao5; expires=Thu, > 09-Jan-2014 20:45:01 GMT; path=/foo; > Here you send to cookie with curl and all is fine. Problems is that when I test the site via Firefox I can login succesfully > to site but when I click any link on the main page I get presented with the > login page again. > Seems like the varnish just droppes my cookie and wants to re-authenticate > me. > There is one potential problem with the whole approach you are using. On the first request there is no cookie. Then the cookie is created and the session is created. The cookie is passed to the client through Varnish. Varnish doesn't know about the relationship between the session and the cookie. So Varnish will hash the cookie and might direct the client to another server, which might or might not have the session. Then there is this: Here's snap of firefox varnishlog: > 11 VCL_Log c Cookie before sanitizing: > __utma=190219199.1951064551.1362571635.1387260530.1389248604.60; > __utmz=190219199.1365759373.8... > 11 VCL_Log c Cookie after sanitizing: > 11 VCL_Log c Client identity set to 192.49.50.88 > 11 ObjHeader c Set-Cookie: FOO_SESSION=0ke75g55tvl8uglm3q95kjqf82; > expires=Thu, 09-Jan-2014 21:14:38 GMT; path=/foo; domain= > https://wwwdev.example.com/; secure; HttpOnly > 11 TxHeader c Set-Cookie: FOO_SESSION=0ke75g55tvl8uglm3q95kjqf82; > expires=Thu, 09-Jan-2014 21:14:38 GMT; path=/foo; domain= > https://wwwdev.example.com/; secure; HttpOnly > 11 VCL_Log c Cookie before sanitizing: > __utma=190219199.1951064551.1362571635.1387260530.1389248604.60; > __utmz=190219199.1365759373.8.2... > 11 VCL_Log c Cookie after sanitizing: > 11 VCL_Log c Client identity set to 192.49.50.88 > > As shown above, Varnish gets Set-Cookie from the application but doesn't > seem to "stick" to cookie. > Varnish doesn't know what Set-Cookie means. There is VCL that might trigger a hit-for-pass, but other than that there is no handling of Set-Cookie. Also, I think the syntax of your Set-Cookie is wrong. domain should look like domain=example.com - do not specify protocol here. So, Firefox, as it should throws the cookies away. Per. -- *Per Buer* CTO | Varnish Software AS Phone: +47 958 39 117 | Skype: per.buer We Make Websites Fly! Winner of the Red Herring Top 100 Europe Award 2013 -------------- next part -------------- An HTML attachment was scrubbed... URL: From timo.myyra at edita.fi Fri Jan 10 06:58:33 2014 From: timo.myyra at edita.fi (=?ISO-8859-1?Q?Myyr=E4=2C_Timo?=) Date: Fri, 10 Jan 2014 08:58:33 +0200 Subject: Problem setting cookie based client identity In-Reply-To: References: Message-ID: Ah, obvious issues yet again after they get pointed out :) Apparently it was firefox that discarded the cookie and obviously it doesn't anything about that to varnish logs. I modified the domain part to strip the protocol and trailing slash from it and things seem to work now. Gotta test a bit further to see varnish cache doesn't do anything unexpected. Now that I remove the cookie header from the /assets urls I think varnish caches those requests by default unless told otherwise. Need to verify they are common stuff and not something which would require authentication. Now that cookie is sanitized to include only my session cookie, I could probably add it to the hash function and tweak the varnish to cache pages even where cookies are set. This would give user-specific caching if I've understood the Varnish examples correctly. Thanks for your help, timo 2014/1/9 Per Buer > Hi Timo, > > On Thu, Jan 9, 2014 at 2:22 PM, Myyr?, Timo wrote: > >> Hi, >> >> I'm having trouble setting up cookie based client identity for one of our >> sites. >> I'm planning on sanitising the Cookie value to only include customer >> specific session cookie and removing other values. >> > > Ok. > >> >> Here's the relevant parts from vcl_recv: >> > (..) > > >> # Remove cookie from static resources >> if (req.url ~ "^assets") { >> remove req.http.Cookie; >> } >> > > This code probably won't be triggered as urls start with /. > > >> Above config seems to work on when I make simple login and page tests via >> curl: >> $ curl --cookie "FOO_SESSION=ah5ai7qaej4a006o79s9500ao5; expires=Thu, >> 09-Jan-2014 20:45:01 GMT; path=/foo; >> > > Here you send to cookie with curl and all is fine. > > > Problems is that when I test the site via Firefox I can login succesfully >> to site but when I click any link on the main page I get presented with the >> login page again. >> Seems like the varnish just droppes my cookie and wants to >> re-authenticate me. >> > > There is one potential problem with the whole approach you are using. On > the first request there is no cookie. Then the cookie is created and the > session is created. The cookie is passed to the client through Varnish. > > Varnish doesn't know about the relationship between the session and the > cookie. So Varnish will hash the cookie and might direct the client to > another server, which might or might not have the session. > > Then there is this: > > Here's snap of firefox varnishlog: >> 11 VCL_Log c Cookie before sanitizing: >> __utma=190219199.1951064551.1362571635.1387260530.1389248604.60; >> __utmz=190219199.1365759373.8... >> 11 VCL_Log c Cookie after sanitizing: >> 11 VCL_Log c Client identity set to 192.49.50.88 >> 11 ObjHeader c Set-Cookie: FOO_SESSION=0ke75g55tvl8uglm3q95kjqf82; >> expires=Thu, 09-Jan-2014 21:14:38 GMT; path=/foo; domain= >> https://wwwdev.example.com/; secure; HttpOnly >> 11 TxHeader c Set-Cookie: FOO_SESSION=0ke75g55tvl8uglm3q95kjqf82; >> expires=Thu, 09-Jan-2014 21:14:38 GMT; path=/foo; domain= >> https://wwwdev.example.com/; secure; HttpOnly >> 11 VCL_Log c Cookie before sanitizing: >> __utma=190219199.1951064551.1362571635.1387260530.1389248604.60; >> __utmz=190219199.1365759373.8.2... >> 11 VCL_Log c Cookie after sanitizing: >> 11 VCL_Log c Client identity set to 192.49.50.88 >> >> As shown above, Varnish gets Set-Cookie from the application but doesn't >> seem to "stick" to cookie. >> > > Varnish doesn't know what Set-Cookie means. There is VCL that might > trigger a hit-for-pass, but other than that there is no handling of > Set-Cookie. > > Also, I think the syntax of your Set-Cookie is wrong. domain should look > like domain=example.com - do not specify protocol here. So, Firefox, as > it should throws the cookies away. > > > Per. > > -- > *Per Buer* > CTO | Varnish Software AS > Phone: +47 958 39 117 | Skype: per.buer > We Make Websites Fly! > > Winner of the Red Herring Top 100 Europe Award 2013 > > > -- Parhain terveisin, Timo Myyr? IT-Asiantuntija Edita Prima Oy, Kehitysyksikk? Hakuninmaantie 2 PL 200 00043 Edita +358 40 860 2103 timo.myyra at edita.fi -------------- next part -------------- An HTML attachment was scrubbed... URL: From smwood4 at gmail.com Sat Jan 11 00:39:45 2014 From: smwood4 at gmail.com (Stephen Wood) Date: Fri, 10 Jan 2014 16:39:45 -0800 Subject: Does a return(restart) prevent first_byte_timeout from terminating a connection? Message-ID: I am experiencing some strange behavior. Periodically requests will take 2x the time specified in first_byte_timeout before they actually are killed and the server returns a 503. This is quite annoying and it means that our users receive a 504 gateway timeout instead of our error message. My gut tells me that it has something to do with the way return(restart) works. The response code returned from the backend is a 503. Let me describe my vcl config briefly. The backend definitions look like this: backend test1 { .host = "10.0.0.1"; .port = "80"; .connect_timeout = 1s; .first_byte_timeout = 65s; .between_bytes_timeout = 1s; .saintmode_threshold = 5; .probe = { .request = "GET /status HTTP/1.1" "Host: test.com" "Connection: close"; .interval = 60s; .timeout = 10s; .window = 5; .threshold = 3; } } (similar for backend test2) I'm using the round-robin directory. Under vcl_fetch I have this: sub vcl_fetch { if (beresp.status == 503) { set beresp.saintmode = 60s; if (req.request != "POST") { return(restart); } else { error 500 "Failed"; } } set beresp.grace = 10s; } Lastly, here's me catching the varnish server in the act of essentially making two requests and not timing out. 169 BackendXID b 1547011740 169 TxRequest b GET 169 TxURL b test? 169 TxProtocol b HTTP/1.1 169 TxHeader b User-Agent: curl/7.30.0 169 TxHeader b Host: xxx.xxx.xxx 169 TxHeader b Accept: */* 169 TxHeader b X-Varnish: 1547011740 169 TxHeader b Accept-Encoding: gzip 169 BackendClose b test1 107 BackendXID b 1547011740 107 TxRequest b GET 107 TxURL b /test? 107 TxProtocol b HTTP/1.1 107 TxHeader b User-Agent: curl/7.30.0 107 TxHeader b Host: xxx.xxx.xxx 107 TxHeader b Accept: */* 107 TxHeader b X-Varnish: 1547011740 107 TxHeader b Accept-Encoding: gzip 107 BackendClose b test2 47 SessionOpen c 162.208.42.130 59288 :80 47 ReqStart c 162.208.42.130 59288 1547011740 47 RxRequest c GET 47 RxURL c /test? 47 RxProtocol c HTTP/1.1 47 RxHeader c User-Agent: curl/7.30.0 47 RxHeader c Accept: */* 47 RxHeader c Connection: keep-alive 47 VCL_call c recv lookup 47 VCL_call c hash 47 Hash c 47 Hash c /test? 47 Hash c 47 VCL_return c hash 47 VCL_call c miss fetch 47 Backend c 169 main_cluster test1 47 FetchError c http first read error: -1 11 (Resource temporarily unavailable) 47 Backend c 107 main_cluster test2 47 FetchError c http first read error: -1 11 (Resource temporarily unavailable) 47 VCL_call c error deliver 47 VCL_call c deliver deliver 47 TxProtocol c HTTP/1.1 47 TxStatus c 503 47 TxResponse c Service Unavailable 47 TxHeader c Server: Varnish 47 TxHeader c Content-Type: application/json; charset=utf-8 47 TxHeader c Content-Length: 65 47 TxHeader c Accept-Ranges: bytes 47 TxHeader c Date: Fri, 10 Jan 2014 23:24:52 GMT 47 TxHeader c X-Varnish: 1547011740 47 TxHeader c Age: 130 47 TxHeader c Via: 1.1 varnish 47 TxHeader c Connection: close 47 Length c 65 47 ReqEnd c 1547011740 1389396162.194909096 1389396292.196509361 0.000040293 130.000771999 0.000828266 What it looks to me like what's happening is the first server is returning a bad status code after 60 seconds (the first_byte_timeout is 65), which satisfies the requirement of first_byte_timeout. The request triggers a return(restart) and it's tried again on another host, which takes another 60 or so seconds. Finally the request just dies by only after it's been hanging around for twice the time allotted in first_byte_timeout Does anyone know what I'm doing wrong? I do want to trigger a return(restart) for spurious status codes, but I want first_byte_timeout to serve as the time I'm willing to let a request go through. Is there a better parameter I can use than this one? It looks very similar to this bug but it looks like that's been fixed. I'm using varnish 3.0.5. Any help is greatly appreciated. -- Stephen Wood www.heystephenwood.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From io.chir0n at gmail.com Mon Jan 13 12:20:05 2014 From: io.chir0n at gmail.com (Chir0n) Date: Mon, 13 Jan 2014 10:20:05 -0200 Subject: 503 erros accessing xml generation page Message-ID: Hi, I'm getting 503 erros when trying to access a slow web page. This page generate a xml and sometimes it can take about 5 minutes to load. I changed many config options and anything worked. Recently I've put the timeout option in high values, but it didn't worked. An important information: Varnish is istalled on a different machine of Apache. This error don't happen on a test environment where Varnish is installed on the same machine of Apache. I'm running out of ideas about what to do. Can someone explain me why this is still happening? The important piece of my default.vcl are show below: (...) backend ws { .host ="x.x.x.x"; .port = "80"; .connect_timeout = 600s; .first_byte_timeout = 600s; .between_bytes_timeout = 600s; .send_timeout = 600s; .sess_timeout = 600s; .cli_timeout = 600s; } (...) } elseif (req.url ~ "^/xml/.*") { set req.backend = ws; return (pipe); } Also, this is the piece of log I get when the error occurs: 60 SessionOpen c x.x.x.x 35998 0.0.0.0:80 60 ReqStart c x.x.x.x 35998 1117143234 60 RxRequest c GET 60 RxURL c /xml/xxx 60 RxProtocol c HTTP/1.0 60 RxHeader c User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727) 60 RxHeader c Accept: */* 60 RxHeader c Host: www.xxxxxxxxx.com.br 60 RxHeader c Connection: Keep-Alive 60 VCL_call c recv pass 60 VCL_call c hash 60 Hash c /xml/xxx 60 Hash c www.xxxxxxxxx.com.br 60 VCL_return c hash 60 VCL_call c pass pass 60 Backend c 347 default default 60 FetchError c http first read error: -1 11 (Resource temporarily unavailable) 60 Backend c 294 default default 60 FetchError c http first read error: -1 11 (Resource temporarily unavailable) 60 VCL_call c error deliver 60 VCL_call c deliver deliver 60 TxProtocol c HTTP/1.1 60 TxStatus c 503 60 TxResponse c Service Unavailable 60 TxHeader c Content-Type: text/html; charset=utf-8 60 TxHeader c Retry-After: 5 60 TxHeader c Content-Length: 419 60 TxHeader c Accept-Ranges: bytes 60 TxHeader c Date: Mon, 13 Jan 2014 12:02:01 GMT 60 TxHeader c Age: 120 60 TxHeader c Connection: close 60 TxHeader c X-Cache: MISS 60 Length c 419 60 ReqEnd c 1117143234 1389614401.316947937 1389614521.319793940 0.000017881 120.002819061 0.000026941 All configs are shown below: /usr/local/varnish/bin/varnishadm param.show acceptor_sleep_decay 0.900000 [] acceptor_sleep_incr 0.001000 [s] acceptor_sleep_max 0.050000 [s] auto_restart on [bool] ban_dups on [bool] ban_lurker_sleep 0.010000 [s] between_bytes_timeout 60.000000 [s] cc_command "exec gcc -std=gnu99 -pthread -fpic -shared -Wl,-x -o %o %s" cli_buffer 8192 [bytes] cli_timeout 10 [seconds] clock_skew 10 [s] connect_timeout 360.000000 [s] critbit_cooloff 180.000000 [s] default_grace 10.000000 [seconds] default_keep 0.000000 [seconds] default_ttl 120.000000 [seconds] diag_bitmap 0x0 [bitmap] esi_syntax 0 [bitmap] expiry_sleep 1.000000 [seconds] fetch_chunksize 128 [kilobytes] fetch_maxchunksize 262144 [kilobytes] first_byte_timeout 60.000000 [s] group users (100) gzip_level 6 [] gzip_memlevel 8 [] gzip_stack_buffer 4096 [Bytes] gzip_tmp_space 0 [] gzip_window 15 [] http_gzip_support on [bool] http_max_hdr 64 [header lines] http_range_support on [bool] http_req_hdr_len 8192 [bytes] http_req_size 12288 [bytes] http_resp_hdr_len 8192 [bytes] http_resp_size 8192 [bytes] idle_send_timeout 60 [seconds] listen_address 0.0.0.0:80 listen_depth 1024 [connections] log_hashstring on [bool] log_local_address off [bool] lru_interval 2 [seconds] max_esi_depth 5 [levels] max_restarts 4 [restarts] nuke_limit 50 [allocations] pcre_match_limit 10000 [] pcre_match_limit_recursion 10000 [] ping_interval 3 [seconds] pipe_timeout 60 [seconds] prefer_ipv6 off [bool] queue_max 100 [%] rush_exponent 3 [requests per request] saintmode_threshold 10 [objects] send_timeout 1200 [seconds] sess_timeout 5 [seconds] sess_workspace 16384 [bytes] session_linger 50 [ms] session_max 100000 [sessions] shm_reclen 255 [bytes] shm_workspace 8192 [bytes] shortlived 10.000000 [s] syslog_cli_traffic on [bool] thread_pool_add_delay 2 [milliseconds] thread_pool_add_threshold 2 [requests] thread_pool_fail_delay 200 [milliseconds] thread_pool_max 500 [threads] thread_pool_min 5 [threads] thread_pool_purge_delay 1000 [milliseconds] thread_pool_stack 65536 [bytes] thread_pool_timeout 300 [seconds] thread_pool_workspace 16384 [bytes] thread_pools 2 [pools] thread_stats_rate 10 [requests] user varnish (1001) vcc_err_unref on [bool] vcl_dir /usr/local/varnish/etc/varnish vcl_trace off [bool] vmod_dir /usr/local/varnish/lib/varnish/vmods waiter default (epoll, poll) -------------- next part -------------- An HTML attachment was scrubbed... URL: From smwood4 at gmail.com Mon Jan 13 18:24:42 2014 From: smwood4 at gmail.com (Stephen Wood) Date: Mon, 13 Jan 2014 10:24:42 -0800 Subject: Does a return(restart) prevent first_byte_timeout from terminating a connection? In-Reply-To: References: Message-ID: Bump. Any idea what might be happening here? Otherwise can someone suggest to me how I can enforce a *real* timeout on a per-request basis? I'd like to keep the return(restart) part of my vcl, but not if it means that some request will take 2x longer than what is specific in .first_byte_timeout. On Fri, Jan 10, 2014 at 4:39 PM, Stephen Wood wrote: > I am experiencing some strange behavior. Periodically requests will take > 2x the time specified in first_byte_timeout before they actually are killed > and the server returns a 503. This is quite annoying and it means that our > users receive a 504 gateway timeout instead of our error message. > > My gut tells me that it has something to do with the way return(restart) > works. The response code returned from the backend is a 503. > > Let me describe my vcl config briefly. The backend definitions look like > this: > > backend test1 { > .host = "10.0.0.1"; > .port = "80"; > .connect_timeout = 1s; > .first_byte_timeout = 65s; > .between_bytes_timeout = 1s; > .saintmode_threshold = 5; > .probe = { > .request = > "GET /status HTTP/1.1" > "Host: test.com" > "Connection: close"; > .interval = 60s; > .timeout = 10s; > .window = 5; > .threshold = 3; > } > } > > (similar for backend test2) > > I'm using the round-robin directory. Under vcl_fetch I have this: > > sub vcl_fetch { > if (beresp.status == 503) { > set beresp.saintmode = 60s; > if (req.request != "POST") { > return(restart); > } else { > error 500 "Failed"; > } > } > > set beresp.grace = 10s; > } > > Lastly, here's me catching the varnish server in the act of essentially > making two requests and not timing out. > > 169 BackendXID b 1547011740 > 169 TxRequest b GET > 169 TxURL b test? > 169 TxProtocol b HTTP/1.1 > 169 TxHeader b User-Agent: curl/7.30.0 > 169 TxHeader b Host: xxx.xxx.xxx > 169 TxHeader b Accept: */* > 169 TxHeader b X-Varnish: 1547011740 > 169 TxHeader b Accept-Encoding: gzip > 169 BackendClose b test1 > > 107 BackendXID b 1547011740 > 107 TxRequest b GET > 107 TxURL b /test? > 107 TxProtocol b HTTP/1.1 > 107 TxHeader b User-Agent: curl/7.30.0 > 107 TxHeader b Host: xxx.xxx.xxx > 107 TxHeader b Accept: */* > 107 TxHeader b X-Varnish: 1547011740 > 107 TxHeader b Accept-Encoding: gzip > 107 BackendClose b test2 > > > 47 SessionOpen c 162.208.42.130 59288 :80 > 47 ReqStart c 162.208.42.130 59288 1547011740 > 47 RxRequest c GET > 47 RxURL c /test? > 47 RxProtocol c HTTP/1.1 > 47 RxHeader c User-Agent: curl/7.30.0 > 47 RxHeader c Accept: */* > 47 RxHeader c Connection: keep-alive > 47 VCL_call c recv lookup > 47 VCL_call c hash > 47 Hash c > 47 Hash c /test? > 47 Hash c > 47 VCL_return c hash > 47 VCL_call c miss fetch > 47 Backend c 169 main_cluster test1 > 47 FetchError c http first read error: -1 11 (Resource temporarily > unavailable) > 47 Backend c 107 main_cluster test2 > 47 FetchError c http first read error: -1 11 (Resource temporarily > unavailable) > 47 VCL_call c error deliver > 47 VCL_call c deliver deliver > 47 TxProtocol c HTTP/1.1 > 47 TxStatus c 503 > 47 TxResponse c Service Unavailable > 47 TxHeader c Server: Varnish > 47 TxHeader c Content-Type: application/json; charset=utf-8 > 47 TxHeader c Content-Length: 65 > 47 TxHeader c Accept-Ranges: bytes > 47 TxHeader c Date: Fri, 10 Jan 2014 23:24:52 GMT > 47 TxHeader c X-Varnish: 1547011740 > 47 TxHeader c Age: 130 > 47 TxHeader c Via: 1.1 varnish > 47 TxHeader c Connection: close > 47 Length c 65 > 47 ReqEnd c 1547011740 1389396162.194909096 1389396292.196509361 > 0.000040293 130.000771999 0.000828266 > > What it looks to me like what's happening is the first server is returning > a bad status code after 60 seconds (the first_byte_timeout is 65), which > satisfies the requirement of first_byte_timeout. The request triggers a > return(restart) and it's tried again on another host, which takes another > 60 or so seconds. Finally the request just dies by only after it's been > hanging around for twice the time allotted in first_byte_timeout > > Does anyone know what I'm doing wrong? I do want to trigger a > return(restart) for spurious status codes, but I want first_byte_timeout to > serve as the time I'm willing to let a request go through. Is there a > better parameter I can use than this one? > > It looks very similar to this bug but > it looks like that's been fixed. I'm using varnish 3.0.5. > > Any help is greatly appreciated. > -- > Stephen Wood > www.heystephenwood.com > -- Stephen Wood www.heystephenwood.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From thierry.magnien at sfr.com Tue Jan 14 08:13:39 2014 From: thierry.magnien at sfr.com (MAGNIEN, Thierry) Date: Tue, 14 Jan 2014 08:13:39 +0000 Subject: Does a return(restart) prevent first_byte_timeout from terminating a connection? In-Reply-To: References: Message-ID: <5D103CE839D50E4CBC62C9FD7B83287C274FF1B5@EXCN015.encara.local.ads> Hi, I think it's the normal behavior from Varnish point of view. Return(restart) restarts the whole request processing so first_byte_timeout is applied each time. Depending on what you want to achieve, either using half the value of first_byte_timeout or write a small patch for Varnish should do the trick. Regards, Thierry De : varnish-misc-bounces+thierry.magnien=sfr.com at varnish-cache.org [mailto:varnish-misc-bounces+thierry.magnien=sfr.com at varnish-cache.org] De la part de Stephen Wood Envoy? : lundi 13 janvier 2014 19:25 ? : varnish-misc at varnish-cache.org Objet : Re: Does a return(restart) prevent first_byte_timeout from terminating a connection? Bump. Any idea what might be happening here? Otherwise can someone suggest to me how I can enforce a real timeout on a per-request basis? I'd like to keep the return(restart) part of my vcl, but not if it means that some request will take 2x longer than what is specific in .first_byte_timeout. On Fri, Jan 10, 2014 at 4:39 PM, Stephen Wood > wrote: I am experiencing some strange behavior. Periodically requests will take 2x the time specified in first_byte_timeout before they actually are killed and the server returns a 503. This is quite annoying and it means that our users receive a 504 gateway timeout instead of our error message. My gut tells me that it has something to do with the way return(restart) works. The response code returned from the backend is a 503. Let me describe my vcl config briefly. The backend definitions look like this: backend test1 { .host = "10.0.0.1"; .port = "80"; .connect_timeout = 1s; .first_byte_timeout = 65s; .between_bytes_timeout = 1s; .saintmode_threshold = 5; .probe = { .request = "GET /status HTTP/1.1" "Host: test.com" "Connection: close"; .interval = 60s; .timeout = 10s; .window = 5; .threshold = 3; } } (similar for backend test2) I'm using the round-robin directory. Under vcl_fetch I have this: sub vcl_fetch { if (beresp.status == 503) { set beresp.saintmode = 60s; if (req.request != "POST") { return(restart); } else { error 500 "Failed"; } } set beresp.grace = 10s; } Lastly, here's me catching the varnish server in the act of essentially making two requests and not timing out. 169 BackendXID b 1547011740 169 TxRequest b GET 169 TxURL b test? 169 TxProtocol b HTTP/1.1 169 TxHeader b User-Agent: curl/7.30.0 169 TxHeader b Host: xxx.xxx.xxx 169 TxHeader b Accept: */* 169 TxHeader b X-Varnish: 1547011740 169 TxHeader b Accept-Encoding: gzip 169 BackendClose b test1 107 BackendXID b 1547011740 107 TxRequest b GET 107 TxURL b /test? 107 TxProtocol b HTTP/1.1 107 TxHeader b User-Agent: curl/7.30.0 107 TxHeader b Host: xxx.xxx.xxx 107 TxHeader b Accept: */* 107 TxHeader b X-Varnish: 1547011740 107 TxHeader b Accept-Encoding: gzip 107 BackendClose b test2 47 SessionOpen c 162.208.42.130 59288 :80 47 ReqStart c 162.208.42.130 59288 1547011740 47 RxRequest c GET 47 RxURL c /test? 47 RxProtocol c HTTP/1.1 47 RxHeader c User-Agent: curl/7.30.0 47 RxHeader c Accept: */* 47 RxHeader c Connection: keep-alive 47 VCL_call c recv lookup 47 VCL_call c hash 47 Hash c 47 Hash c /test? 47 Hash c 47 VCL_return c hash 47 VCL_call c miss fetch 47 Backend c 169 main_cluster test1 47 FetchError c http first read error: -1 11 (Resource temporarily unavailable) 47 Backend c 107 main_cluster test2 47 FetchError c http first read error: -1 11 (Resource temporarily unavailable) 47 VCL_call c error deliver 47 VCL_call c deliver deliver 47 TxProtocol c HTTP/1.1 47 TxStatus c 503 47 TxResponse c Service Unavailable 47 TxHeader c Server: Varnish 47 TxHeader c Content-Type: application/json; charset=utf-8 47 TxHeader c Content-Length: 65 47 TxHeader c Accept-Ranges: bytes 47 TxHeader c Date: Fri, 10 Jan 2014 23:24:52 GMT 47 TxHeader c X-Varnish: 1547011740 47 TxHeader c Age: 130 47 TxHeader c Via: 1.1 varnish 47 TxHeader c Connection: close 47 Length c 65 47 ReqEnd c 1547011740 1389396162.194909096 1389396292.196509361 0.000040293 130.000771999 0.000828266 What it looks to me like what's happening is the first server is returning a bad status code after 60 seconds (the first_byte_timeout is 65), which satisfies the requirement of first_byte_timeout. The request triggers a return(restart) and it's tried again on another host, which takes another 60 or so seconds. Finally the request just dies by only after it's been hanging around for twice the time allotted in first_byte_timeout Does anyone know what I'm doing wrong? I do want to trigger a return(restart) for spurious status codes, but I want first_byte_timeout to serve as the time I'm willing to let a request go through. Is there a better parameter I can use than this one? It looks very similar to this bug but it looks like that's been fixed. I'm using varnish 3.0.5. Any help is greatly appreciated. -- Stephen Wood www.heystephenwood.com -- Stephen Wood www.heystephenwood.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From straightflush at gmail.com Wed Jan 15 14:25:31 2014 From: straightflush at gmail.com (AD) Date: Wed, 15 Jan 2014 09:25:31 -0500 Subject: ScaleSimple - An Open Source CDN Message-ID: hey all, Just wanted to announce that we just open sourced ScaleSimple, an open source CDN platform based on varnish. Blog post here - http://blog.dberg.org/2014/01/scalesimple-open-source-cdn.html And feel free to upvote us on HN :) we are currently under newest https://news.ycombinator.com/newest Cheers Adam -------------- next part -------------- An HTML attachment was scrubbed... URL: From japrice at gmail.com Thu Jan 16 23:38:39 2014 From: japrice at gmail.com (Jason Price) Date: Thu, 16 Jan 2014 18:38:39 -0500 Subject: 503 Error, first byte timeout. Which Backend? Message-ID: So, I have backends which are occasionally slow. So varnish responds with a 503 error. How can I determine WHICH backend was used, which failed to respond in a timely manner? In varnishlog, I see errors like the following: 180 FetchError c http first read error: -1 11 (Resource temporarily unavailable) Reading the code, the -1 SEEMS to indicate that the backend closed the connection on this error. vcl_error doesn't give me access to beresp variables (like beresp.backend.name or IP, which would answer the question). This indicates that obj.* does NOT include a backend indication... Any way I can figure out which backend is failing me? --Jason -------------- next part -------------- An HTML attachment was scrubbed... URL: From smwood4 at gmail.com Fri Jan 17 00:38:26 2014 From: smwood4 at gmail.com (Stephen Wood) Date: Thu, 16 Jan 2014 16:38:26 -0800 Subject: 503 Error, first byte timeout. Which Backend? In-Reply-To: References: Message-ID: You should see this value as "Backend" in your varnishlog output. It should be right above the FetchError line you posted. On Thu, Jan 16, 2014 at 3:38 PM, Jason Price wrote: > So, I have backends which are occasionally slow. So varnish responds with > a 503 error. > > How can I determine WHICH backend was used, which failed to respond in a > timely manner? > > In varnishlog, I see errors like the following: > > 180 FetchError c http first read error: -1 11 (Resource temporarily > unavailable) > > Reading the code, the -1 SEEMS to indicate that the backend closed the > connection on this error. vcl_error doesn't give me access to beresp > variables (like beresp.backend.name or IP, which would answer the > question). This indicates that obj.* does NOT include a backend > indication... > > Any way I can figure out which backend is failing me? > > --Jason > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -- Stephen Wood www.heystephenwood.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From japrice at gmail.com Fri Jan 17 01:21:50 2014 From: japrice at gmail.com (Jason Price) Date: Thu, 16 Jan 2014 20:21:50 -0500 Subject: 503 Error, first byte timeout. Which Backend? In-Reply-To: References: Message-ID: I was doing that... Grep -B 1... But the thread index number (left most number) is different... Or is that different because it's doing back end processing? Jason <-- still figuring out varnishlog On Jan 16, 2014 7:39 PM, "Stephen Wood" wrote: > You should see this value as "Backend" in your varnishlog output. It > should be right above the FetchError line you posted. > > > On Thu, Jan 16, 2014 at 3:38 PM, Jason Price wrote: > >> So, I have backends which are occasionally slow. So varnish responds >> with a 503 error. >> >> How can I determine WHICH backend was used, which failed to respond in a >> timely manner? >> >> In varnishlog, I see errors like the following: >> >> 180 FetchError c http first read error: -1 11 (Resource temporarily >> unavailable) >> >> Reading the code, the -1 SEEMS to indicate that the backend closed the >> connection on this error. vcl_error doesn't give me access to beresp >> variables (like beresp.backend.name or IP, which would answer the >> question). This indicates that obj.* does NOT include a backend >> indication... >> >> Any way I can figure out which backend is failing me? >> >> --Jason >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> > > > > -- > Stephen Wood > www.heystephenwood.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From smwood4 at gmail.com Fri Jan 17 01:28:09 2014 From: smwood4 at gmail.com (Stephen Wood) Date: Thu, 16 Jan 2014 17:28:09 -0800 Subject: 503 Error, first byte timeout. Which Backend? In-Reply-To: References: Message-ID: Grepping won't really do it for you. You can use varnishlog to capture just 503 status codes by doing the following: varnishlog -m TxStatus:503 Leave that running and once you get a full output go ahead and paste it here and we'll take a look at it (remember to scrub any sensitive information). On Thu, Jan 16, 2014 at 5:21 PM, Jason Price wrote: > I was doing that... Grep -B 1... But the thread index number (left most > number) is different... Or is that different because it's doing back end > processing? > > Jason <-- still figuring out varnishlog > On Jan 16, 2014 7:39 PM, "Stephen Wood" wrote: > >> You should see this value as "Backend" in your varnishlog output. It >> should be right above the FetchError line you posted. >> >> >> On Thu, Jan 16, 2014 at 3:38 PM, Jason Price wrote: >> >>> So, I have backends which are occasionally slow. So varnish responds >>> with a 503 error. >>> >>> How can I determine WHICH backend was used, which failed to respond in a >>> timely manner? >>> >>> In varnishlog, I see errors like the following: >>> >>> 180 FetchError c http first read error: -1 11 (Resource temporarily >>> unavailable) >>> >>> Reading the code, the -1 SEEMS to indicate that the backend closed the >>> connection on this error. vcl_error doesn't give me access to beresp >>> variables (like beresp.backend.name or IP, which would answer the >>> question). This indicates that obj.* does NOT include a backend >>> indication... >>> >>> Any way I can figure out which backend is failing me? >>> >>> --Jason >>> >>> _______________________________________________ >>> varnish-misc mailing list >>> varnish-misc at varnish-cache.org >>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >>> >> >> >> >> -- >> Stephen Wood >> www.heystephenwood.com >> > -- Stephen Wood www.heystephenwood.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From japrice at gmail.com Tue Jan 21 02:00:38 2014 From: japrice at gmail.com (Jason Price) Date: Mon, 20 Jan 2014 21:00:38 -0500 Subject: beresp.ttl set to -1.000? Message-ID: I'm very confused. Despite backend headers instructing caching for 15 minutes, beresp.ttl get's overridden to be -1.000 immediately before vcl_fetch starts running. For the test, I restarted varnish, ran the test with an empty cache, and validated at the end that only one connection/request had come in via varnishstat. Backend response headers (via tcpdump/wireshark): Expires: Tue, 21 Jan 2014 01:35:54 GMT Cache-Control: max-age=900, s-maxage=900, public Vary: Accept-Encoding Vary: Accept-Encoding Content-Type: application/json Varnishlog output showing the two TTL lines. 10 TTL c 562972947 RFC 900 -1 -1 1390267254 0 1390267254 1390268154 900 10 VCL_call c fetch 10 TTL c 562972947 VCL 120 -1 -1 1390267254 -0 10 VCL_return c hit_for_pass So, vcl_fetch changes the ttl? Here's my modified vcl_fetch (faintly adapted from default.vcl for debugging): sub vcl_fetch { if (beresp.ttl <= 0s) { std.syslog((3*8)+5, "ttl fail"); } if (beresp.http.Set-Cookie) { std.syslog((3*8)+5, "cookie fail"); } if (beresp.http.Vary == "*") { std.syslog((3*8)+5, "Very fail"); } if (beresp.ttl <= 0s || beresp.http.Set-Cookie || beresp.http.Vary == "*") { if (beresp.status == 400) { set beresp.ttl = 5 s; return (deliver); } else if (beresp.status == 500) { set beresp.ttl = 5 s; std.syslog((3*8)+5, "beresp status code 500 for url request " + req.url); return (deliver); } else { /* * Mark as "Hit-For-Pass" for the next 2 minutes */ std.syslog((3 * 8) + 5, "what? beresp.ttl " + beresp.ttl + " beresp.proto " + beresp.proto + " beresp.status " + beresp.status + " beresp.response " + beresp.response + " beresp.backend.ip " + beresp.backend.ip + " beresp.http.Set-Cookie is " + beresp.http.Set-Cookie + " beresp.http.Vary is " + beresp.http.Vary + " req.url is " + req.url); set beresp.ttl = 120 s; return (hit_for_pass); } } return (deliver); } And indeed, the logs show: Jan 21 01:20:54 devvcache4x00 varnishd[4761]: *ttl fail* Jan 21 01:20:54 devvcache4x00 varnishd[4761]: what? *beresp.ttl -1.000*beresp.proto HTTP/1.1 beresp.status 200 beresp.response OK beresp.backend.ip 10.224.4.167 beresp.http.Set-Cookie is beresp.http.Vary is Accept-Encoding, Accept-Encoding req.url is Why is beresp.ttl getting set to -1.000 ? I can't find anything modifying beresp.ttl in my code (except what's above to set it to something positive). Varnish 3.0.4, with vmod_redis, vmod_digest Thank you for any help; Jason -------------- next part -------------- An HTML attachment was scrubbed... URL: From japrice at gmail.com Fri Jan 24 21:29:22 2014 From: japrice at gmail.com (Jason Price) Date: Fri, 24 Jan 2014 16:29:22 -0500 Subject: beresp.ttl set to -1.000? In-Reply-To: References: Message-ID: So, no one has seen anything like this? I'm watching varnish override the backend's given TTL... and I can't fathom what is going on here. On Mon, Jan 20, 2014 at 9:00 PM, Jason Price wrote: > I'm very confused. > > Despite backend headers instructing caching for 15 minutes, beresp.ttl > get's overridden to be -1.000 immediately before vcl_fetch starts running. > > For the test, I restarted varnish, ran the test with an empty cache, and > validated at the end that only one connection/request had come in via > varnishstat. > > Backend response headers (via tcpdump/wireshark): > > Expires: Tue, 21 Jan 2014 01:35:54 GMT > Cache-Control: max-age=900, s-maxage=900, public > Vary: Accept-Encoding > Vary: Accept-Encoding > Content-Type: application/json > > Varnishlog output showing the two TTL lines. > > 10 TTL c 562972947 RFC 900 -1 -1 1390267254 0 1390267254 > 1390268154 900 > 10 VCL_call c fetch > 10 TTL c 562972947 VCL 120 -1 -1 1390267254 -0 > 10 VCL_return c hit_for_pass > > So, vcl_fetch changes the ttl? > > Here's my modified vcl_fetch (faintly adapted from default.vcl for > debugging): > > sub vcl_fetch { > if (beresp.ttl <= 0s) { > std.syslog((3*8)+5, "ttl fail"); > } > if (beresp.http.Set-Cookie) { > std.syslog((3*8)+5, "cookie fail"); > } > if (beresp.http.Vary == "*") { > std.syslog((3*8)+5, "Very fail"); > } > if (beresp.ttl <= 0s || beresp.http.Set-Cookie || beresp.http.Vary == > "*") { > if (beresp.status == 400) { > set beresp.ttl = 5 s; > return (deliver); > } else if (beresp.status == 500) { > set beresp.ttl = 5 s; > std.syslog((3*8)+5, "beresp status code 500 for url request " + > req.url); > return (deliver); > } else { > /* > * Mark as "Hit-For-Pass" for the next 2 minutes > */ > std.syslog((3 * 8) + 5, "what? beresp.ttl " + beresp.ttl + > " beresp.proto " + beresp.proto + > " beresp.status " + beresp.status + > " beresp.response " + beresp.response + > " beresp.backend.ip " + beresp.backend.ip + > " beresp.http.Set-Cookie is " + beresp.http.Set-Cookie + > " beresp.http.Vary is " + beresp.http.Vary + > " req.url is " + req.url); > set beresp.ttl = 120 s; > return (hit_for_pass); > } > } > return (deliver); > } > > And indeed, the logs show: > > Jan 21 01:20:54 devvcache4x00 varnishd[4761]: *ttl fail* > Jan 21 01:20:54 devvcache4x00 varnishd[4761]: what? *beresp.ttl -1.000*beresp.proto HTTP/1.1 beresp.status 200 beresp.response OK > beresp.backend.ip 10.224.4.167 beresp.http.Set-Cookie is beresp.http.Vary > is Accept-Encoding, Accept-Encoding req.url is > > Why is beresp.ttl getting set to -1.000 ? I can't find anything > modifying beresp.ttl in my code (except what's above to set it to something > positive). > > Varnish 3.0.4, with vmod_redis, vmod_digest > > Thank you for any help; > Jason > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulex.ujjal at gmail.com Mon Jan 27 04:06:49 2014 From: ulex.ujjal at gmail.com (Ujjal Das) Date: Mon, 27 Jan 2014 09:36:49 +0530 Subject: Installing varnish Message-ID: Dear Sir I am a small ISP , I want to know the varnish server install where before the inventum server or after the inventum server also let me know the how many lan card use in this varnish server one or two, how to configure the lan card with ip.please let me know the details as soon as possible. -- Regards, Ujjal Das Sailesh Banerjee Road, 2NO.Sankari-Pukur,Burdwan, West-Bengal,India-713103, Mob : +919932006606 -------------- next part -------------- An HTML attachment was scrubbed... URL: From simon.lennon at oncommerce.co.uk Mon Jan 27 11:45:20 2014 From: simon.lennon at oncommerce.co.uk (Simon Lennon, OnCommerce Ltd.) Date: Mon, 27 Jan 2014 11:45:20 +0000 Subject: Processing Power Message-ID: <52E646D0.7060704@oncommerce.co.uk> Hi guys, I was wondering what kind of processing power varnish needs. I am setting up a dedicated environment with a custom built VCL which includes some regexp and the load balancing functionality. I was going to setup a dual core server with 96GB of ram, is this enough for a HIGH traffic site, says 10,000 hits a second with very high cache utilization? I will also be using the ban list to achieve a very high level of caching, with a daily restart to flush the archive, but I would expect several hundred regex patterns per day in the ban list. I know varnish is optimized for performance so I wasn't sure how much processing power it would need, thanks, -- *Simon Lennon* Technical Director OnCommerce Ltd. +44 (0) 1202 970 940 OnBay | Mr Snaps oncommerce *www.oncommerce.co.uk * Follow Us on Twitter Find Us On Facebook Find Us On Google+ onbuy *www.onbuy.co.uk * Follow Us on Twitter Find Us On Facebook Find Us On Google+ onrecycle *www.onrecycle.co.uk * Follow Us on Twitter Find Us On Facebook Find Us On Google+ OnBuy and OnRecycle are trading names of OnCommerce Ltd which is a registered company in England & Wales, Company Number: 08358927. Our trading address is: C/O Centerprise International, Hampshire International Business Park, Lime Tree Way, Basingstoke, Hampshire, RG24 8GQ. OnCommerce is a registered trademark no. 2540468. OnCommerce Ltd is a Centeprise International Group Company. This electronic message contains information from OnCommerce Ltd, which may be legally privileged and confidential. The information is intended to be for the use of the individual(s) or entity named above. If you are not the intended recipient, be aware that any disclosure, copying, distribution or use of the contents of this information is prohibited. If you have received this electronic message in error, please notify us by telephone (to the number above) or e-mail immediately. Any views, opinions or advice expressed in this electronic message are not necessarily the views, opinions or advice of OnCommerce Ltd. It is the responsibility of the recipient to ensure that any attachments are virus free and OnCommerce Ltd. bear no responsibility for any loss or damage arising in any way from the use thereof. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: onbay_email_senor_pinchy.png Type: image/png Size: 4332 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature2_12.jpg Type: image/jpeg Size: 1244 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: onbay_email_oncommerce_logo.png Type: image/png Size: 5758 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: onbay_email_footer_twitter.png Type: image/png Size: 3575 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: onbay_email_footer_facebook.png Type: image/png Size: 2802 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: onbay_email_footer_google.png Type: image/png Size: 2856 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: onbay_email_onbuy.png Type: image/png Size: 5216 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: onbay_email_onrecycle_logo.png Type: image/png Size: 6416 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature2_13.jpg Type: image/jpeg Size: 1298 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: onbay_email_centerprise_logo.png Type: image/png Size: 9852 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: spacer.gif Type: image/gif Size: 13126 bytes Desc: not available URL: From perbu at varnish-software.com Mon Jan 27 12:45:21 2014 From: perbu at varnish-software.com (Per Buer) Date: Mon, 27 Jan 2014 13:45:21 +0100 Subject: Processing Power In-Reply-To: <52E646D0.7060704@oncommerce.co.uk> References: <52E646D0.7060704@oncommerce.co.uk> Message-ID: Hi Simon, With regards to CPU, Varnish doesn't need that much. Typically, you'll have maybe 5 or 10% CPU utilisation on a modern machine running at 5000 requests a second. The memory is completely dependent on the site you are about to cache. Most websites are pretty small in terms of content size so 96GB should be more than sufficient to keep most of the website in memory, unless you have video content. Even then, 96GB is quite a lot. The bans will add a bit of CPU use to Varnish and might induce a bit of latency if you'll have a lot of them. The CPU usage is a factor of (no of bans) x number of objects in cache as every ban will have to be matched against every object in cache. CPUs are fast, so this might not be as bad as it sounds. Per. On Mon, Jan 27, 2014 at 12:45 PM, Simon Lennon, OnCommerce Ltd. < simon.lennon at oncommerce.co.uk> wrote: > Hi guys, I was wondering what kind of processing power varnish needs. I > am setting up a dedicated environment with a custom built VCL which > includes some regexp and the load balancing functionality. > > I was going to setup a dual core server with 96GB of ram, is this enough > for a HIGH traffic site, says 10,000 hits a second with very high cache > utilization? > > I will also be using the ban list to achieve a very high level of caching, > with a daily restart to flush the archive, but I would expect several > hundred regex patterns per day in the ban list. > > I know varnish is optimized for performance so I wasn't sure how much > processing power it would need, > > thanks, > -- > > *Simon Lennon* > Technical Director > > OnCommerce Ltd. > +44 (0) 1202 970 940 > > [image: OnBay | Mr Snaps] > > > > > > [image: oncommerce] > > *www.oncommerce.co.uk * > > [image: Follow Us on Twitter] [image: > Find Us On Facebook] [image: Find > Us On Google+] > > [image: onbuy] > > *www.onbuy.co.uk * > > [image: Follow Us on Twitter] [image: Find > Us On Facebook] [image: Find Us On > Google+] > > [image: onrecycle] > > *www.onrecycle.co.uk * > > [image: Follow Us on Twitter] [image: > Find Us On Facebook] [image: Find Us > On Google+] > > OnBuy and OnRecycle are trading names of OnCommerce Ltd which is > a registered company in England & Wales, Company Number: 08358927. Our > trading address is: C/O Centerprise International, Hampshire International > Business Park, Lime Tree Way, Basingstoke, Hampshire, RG24 8GQ. OnCommerce > is a registered trademark no. 2540468. > > OnCommerce Ltd is a Centeprise International Group Company. > > This electronic message contains information from OnCommerce Ltd, > which may be legally privileged and confidential. The information is > intended to be for the use of the individual(s) or entity named above. If > you are not the intended recipient, be aware that any disclosure, copying, > distribution or use of the contents of this information is prohibited. If > you have received this electronic message in error, please notify us by > telephone (to the number above) or e-mail immediately. Any views, opinions > or advice expressed in this electronic message are not necessarily the > views, opinions or advice of OnCommerce Ltd. It is the responsibility of > the recipient to ensure that any attachments are virus free and OnCommerce > Ltd. bear no responsibility for any loss or damage arising in any way from > the use thereof. > > > > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -- *Per Buer* CTO | Varnish Software Phone: +47 958 39 117 | Skype: per.buer We Make Websites Fly! Winner of the Red Herring Top 100 Global Award 2013 -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: onbay_email_oncommerce_logo.png Type: image/png Size: 5758 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: onbay_email_onbuy.png Type: image/png Size: 5216 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: onbay_email_footer_twitter.png Type: image/png Size: 3575 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: onbay_email_footer_facebook.png Type: image/png Size: 2802 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: onbay_email_onrecycle_logo.png Type: image/png Size: 6416 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: onbay_email_senor_pinchy.png Type: image/png Size: 4332 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: onbay_email_footer_google.png Type: image/png Size: 2856 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature2_13.jpg Type: image/jpeg Size: 1298 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature2_12.jpg Type: image/jpeg Size: 1244 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: spacer.gif Type: image/gif Size: 13126 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: onbay_email_centerprise_logo.png Type: image/png Size: 9852 bytes Desc: not available URL: From jan at architechs.eu Mon Jan 27 13:05:34 2014 From: jan at architechs.eu (=?iso-8859-1?Q?Jan-Aage_Frydenb=F8-Bruvoll?=) Date: Mon, 27 Jan 2014 13:05:34 +0000 Subject: Processing Power In-Reply-To: <52E646D0.7060704@oncommerce.co.uk> References: <52E646D0.7060704@oncommerce.co.uk> Message-ID: Hi, I am sure you have made your own calculations already and will know that the amount of RAM will be more dependent on the active data set and not so much dictate the potential speed of Varnish. To our experience we run out of file descriptors on the servers before we run out of steam on the Varnish side, and the former has so far only been because of misconfigurations on our part. We have distributed the load across a number of virtual servers, each taking in the range of 1000-2500 req/s - and that at a typical CPU load of 5-15% on our 2.5GHz clock CPUs. I am also sure you have done your considerations regarding the daily cache flush, but in case you have not, I would try to find ways around that, as it leaves you open to attack against your caches (or general overload) at the time after the flush, when a degree (unknown to me) of your cache has to be rebuilt for no reason. Best regards Jan From thierry.magnien at sfr.com Mon Jan 27 13:54:33 2014 From: thierry.magnien at sfr.com (MAGNIEN, Thierry) Date: Mon, 27 Jan 2014 13:54:33 +0000 Subject: beresp.ttl set to -1.000? In-Reply-To: References: Message-ID: <5D103CE839D50E4CBC62C9FD7B83287C2750BDD4@EXCN015.encara.local.ads> Hi Jason, Can you provide a full varnishlog output ? (not only -c nor -b) Thierry ----------------- De?: varnish-misc-bounces+thierry.magnien=sfr.com at varnish-cache.org [mailto:varnish-misc-bounces+thierry.magnien=sfr.com at varnish-cache.org] De la part de Jason Price Envoy??: vendredi 24 janvier 2014 22:29 ??: varnish-misc at varnish-cache.org Objet?: Re: beresp.ttl set to -1.000? So, no one has seen anything like this? I'm watching varnish override the backend's given TTL... and I can't fathom what is going on here. On Mon, Jan 20, 2014 at 9:00 PM, Jason Price wrote: I'm very confused.? Despite backend headers instructing caching for 15 minutes, beresp.ttl get's overridden to be -1.000 immediately before vcl_fetch starts running. For the test, I restarted varnish, ran the test with an empty cache, and validated at the end that only one connection/request had come in via varnishstat. Backend response headers (via tcpdump/wireshark): Expires: Tue, 21 Jan 2014 01:35:54 GMT Cache-Control: max-age=900, s-maxage=900, public Vary: Accept-Encoding Vary: Accept-Encoding Content-Type: application/json Varnishlog output showing the two TTL lines. ?? 10 TTL????????? c 562972947 RFC 900 -1 -1 1390267254 0 1390267254 1390268154 900 ?? 10 VCL_call???? c fetch ?? 10 TTL????????? c 562972947 VCL 120 -1 -1 1390267254 -0 ?? 10 VCL_return?? c hit_for_pass So, vcl_fetch changes the ttl? Here's my modified vcl_fetch (faintly adapted from default.vcl for debugging): sub vcl_fetch { ? if (beresp.ttl <= 0s) { ??? std.syslog((3*8)+5, "ttl fail"); ? }???? ? if (beresp.http.Set-Cookie) { ??? std.syslog((3*8)+5, "cookie fail"); ? }???? ? if (beresp.http.Vary == "*") { ??? std.syslog((3*8)+5, "Very fail"); ? }???? ? if (beresp.ttl <= 0s || beresp.http.Set-Cookie || beresp.http.Vary == "*") { ??? if (beresp.status == 400) { ????? set beresp.ttl = 5 s; ????? return (deliver); ??? } else if (beresp.status == 500) { ????? set beresp.ttl = 5 s; ????? std.syslog((3*8)+5, "beresp status code 500 for url request " + req.url); ????? return (deliver); ??? } else {??? ????? /*??????????????? ????? * Mark as "Hit-For-Pass" for the next 2 minutes ????? */??????????????? ????? std.syslog((3 * 8) + 5, "what? beresp.ttl " + beresp.ttl + ????????? " beresp.proto " + beresp.proto + ????????? " beresp.status " + beresp.status + ????????? " beresp.response " + beresp.response + ????????? " beresp.backend.ip " + beresp.backend.ip + ????????? " beresp.http.Set-Cookie is " + beresp.http.Set-Cookie +? ????????? " beresp.http.Vary is " + beresp.http.Vary + ????????? " req.url is " + req.url);??? ????? set beresp.ttl = 120 s; ????? return (hit_for_pass); ??? }?????????? ? }???? ? return (deliver); } And indeed, the logs show: Jan 21 01:20:54 devvcache4x00 varnishd[4761]: ttl fail Jan 21 01:20:54 devvcache4x00 varnishd[4761]: what? beresp.ttl -1.000 beresp.proto HTTP/1.1 beresp.status 200 beresp.response OK beresp.backend.ip 10.224.4.167 beresp.http.Set-Cookie is? beresp.http.Vary is Accept-Encoding, Accept-Encoding req.url is Why is beresp.ttl getting set to -1.000 ??? I can't find anything modifying beresp.ttl in my code (except what's above to set it to something positive). Varnish 3.0.4, with vmod_redis, vmod_digest Thank you for any help; Jason From james at ifixit.com Mon Jan 27 19:48:09 2014 From: james at ifixit.com (James Pearson) Date: Mon, 27 Jan 2014 11:48:09 -0800 Subject: Installing varnish In-Reply-To: References: Message-ID: <1390851768-sup-7138@geror.local> Excerpts from Ujjal Das's message of 2014-01-26 20:06:49 -0800: > I am a small ISP , I want to know the varnish server install where before the > inventum server or after the inventum server I don't know what an inventum server is, although a quick search seems to indicate inventum produces networking gear. In general, you want to put Varnish as far in front as possible, so you can return cache hits immediately. What are you planning on using Varnish for? It's normally used in front of web applications, not as a general caching proxy. Squid might be better suited for your purposes. > also let me know the how many lan card use in this varnish server one or > two, how to configure the lan card with ip.please let me know the details You know your workload better than we do (since we know nothing about it), and, really, if you're operating an ISP, you need to be able to handle this sort of thing on your own; if you can't, I don't see the future going very well for you. - P From james at ifixit.com Mon Jan 27 22:55:23 2014 From: james at ifixit.com (James Pearson) Date: Mon, 27 Jan 2014 14:55:23 -0800 Subject: Processing Power In-Reply-To: <52E646D0.7060704@oncommerce.co.uk> References: <52E646D0.7060704@oncommerce.co.uk> Message-ID: <1390862918-sup-5037@geror.local> Excerpts from Simon Lennon, OnCommerce Ltd.'s message of 2014-01-27 03:45:20 -0800: > Hi guys, I was wondering what kind of processing power varnish needs. I > am setting up a dedicated environment with a custom built VCL which > includes some regexp and the load balancing functionality. > > I was going to setup a dual core server with 96GB of ram, is this enough > for a HIGH traffic site, says 10,000 hits a second with very high cache > utilization? As always, testing yourself is the best way to find out. Although it's simplistic, I've found Bees with Machine Guns![0] to be useful in load-testing Varnish. One thing we noticed recently was that we were getting about 20% of our requests from clients that didn't support gzip. Turning off http_gzip_support[1] helped tick down our CPU usage a bit, although you have to be ok with Varnish keeping (potentially) two separate cache entries for each object. With that setting adjusted, in our testing varnish lookups were the only thing that caused any appreciable amount of load; our machine (an EC2 m1.large) could handle hits just fine up until the bandwidth capacity topped out. - P [0]: https://github.com/newsapps/beeswithmachineguns [1]: https://www.varnish-cache.org/docs/3.0/phk/gzip.html From japrice at gmail.com Mon Jan 27 23:37:47 2014 From: japrice at gmail.com (Jason Price) Date: Mon, 27 Jan 2014 18:37:47 -0500 Subject: beresp.ttl set to -1.000? In-Reply-To: <5D103CE839D50E4CBC62C9FD7B83287C2750BDD4@EXCN015.encara.local.ads> References: <5D103CE839D50E4CBC62C9FD7B83287C2750BDD4@EXCN015.encara.local.ads> Message-ID: Thank you for your help... In preparing the varnishlog output, I found my mistake. The default.vcl code was checking for an Authorization header, and was instructing all code to return(pass); due to that. --Jason On Mon, Jan 27, 2014 at 8:54 AM, MAGNIEN, Thierry wrote: > Hi Jason, > > Can you provide a full varnishlog output ? (not only -c nor -b) > > Thierry > > ----------------- > De : varnish-misc-bounces+thierry.magnien=sfr.com at varnish-cache.org[mailto: > varnish-misc-bounces+thierry.magnien=sfr.com at varnish-cache.org] De la > part de Jason Price > Envoy? : vendredi 24 janvier 2014 22:29 > ? : varnish-misc at varnish-cache.org > Objet : Re: beresp.ttl set to -1.000? > > So, no one has seen anything like this? > I'm watching varnish override the backend's given TTL... and I can't > fathom what is going on here. > > On Mon, Jan 20, 2014 at 9:00 PM, Jason Price wrote: > I'm very confused. > Despite backend headers instructing caching for 15 minutes, beresp.ttl > get's overridden to be -1.000 immediately before vcl_fetch starts running. > > For the test, I restarted varnish, ran the test with an empty cache, and > validated at the end that only one connection/request had come in via > varnishstat. > > Backend response headers (via tcpdump/wireshark): > > Expires: Tue, 21 Jan 2014 01:35:54 GMT > Cache-Control: max-age=900, s-maxage=900, public > Vary: Accept-Encoding > Vary: Accept-Encoding > Content-Type: application/json > > Varnishlog output showing the two TTL lines. > > 10 TTL c 562972947 RFC 900 -1 -1 1390267254 0 1390267254 > 1390268154 900 > 10 VCL_call c fetch > 10 TTL c 562972947 VCL 120 -1 -1 1390267254 -0 > 10 VCL_return c hit_for_pass > > So, vcl_fetch changes the ttl? > > Here's my modified vcl_fetch (faintly adapted from default.vcl for > debugging): > > sub vcl_fetch { > if (beresp.ttl <= 0s) { > std.syslog((3*8)+5, "ttl fail"); > } > if (beresp.http.Set-Cookie) { > std.syslog((3*8)+5, "cookie fail"); > } > if (beresp.http.Vary == "*") { > std.syslog((3*8)+5, "Very fail"); > } > if (beresp.ttl <= 0s || beresp.http.Set-Cookie || beresp.http.Vary == > "*") { > if (beresp.status == 400) { > set beresp.ttl = 5 s; > return (deliver); > } else if (beresp.status == 500) { > set beresp.ttl = 5 s; > std.syslog((3*8)+5, "beresp status code 500 for url request " + > req.url); > return (deliver); > } else { > /* > * Mark as "Hit-For-Pass" for the next 2 minutes > */ > std.syslog((3 * 8) + 5, "what? beresp.ttl " + beresp.ttl + > " beresp.proto " + beresp.proto + > " beresp.status " + beresp.status + > " beresp.response " + beresp.response + > " beresp.backend.ip " + beresp.backend.ip + > " beresp.http.Set-Cookie is " + beresp.http.Set-Cookie + > " beresp.http.Vary is " + beresp.http.Vary + > " req.url is " + req.url); > set beresp.ttl = 120 s; > return (hit_for_pass); > } > } > return (deliver); > } > And indeed, the logs show: > > Jan 21 01:20:54 devvcache4x00 varnishd[4761]: ttl fail > Jan 21 01:20:54 devvcache4x00 varnishd[4761]: what? beresp.ttl -1.000 > beresp.proto HTTP/1.1 beresp.status 200 beresp.response OK > beresp.backend.ip 10.224.4.167 beresp.http.Set-Cookie is beresp.http.Vary > is Accept-Encoding, Accept-Encoding req.url is > Why is beresp.ttl getting set to -1.000 ? I can't find anything > modifying beresp.ttl in my code (except what's above to set it to something > positive). > Varnish 3.0.4, with vmod_redis, vmod_digest > Thank you for any help; > Jason > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From simon.lennon at oncommerce.co.uk Tue Jan 28 09:42:11 2014 From: simon.lennon at oncommerce.co.uk (Simon) Date: Tue, 28 Jan 2014 09:42:11 +0000 Subject: Processing Power In-Reply-To: <52E646D0.7060704@oncommerce.co.uk> References: <52E646D0.7060704@oncommerce.co.uk> Message-ID: <52E77B73.9020407@oncommerce.co.uk> Thanks for the feedback on the processing power that was really helpful, has anyone combined varnish with nginx and used it as a load balancer / caching box. Does the performance requirements change much with nginx sitting in front of varnish? thanks, -- *Simon Lennon* Technical Director OnCommerce Ltd. +44 (0) 1202 970 940 OnBay | Mr Snaps oncommerce *www.oncommerce.co.uk * Follow Us on Twitter Find Us On Facebook Find Us On Google+ onbuy *www.onbuy.co.uk * Follow Us on Twitter Find Us On Facebook Find Us On Google+ onrecycle *www.onrecycle.co.uk * Follow Us on Twitter Find Us On Facebook Find Us On Google+ OnBuy and OnRecycle are trading names of OnCommerce Ltd which is a registered company in England & Wales, Company Number: 08358927. Our trading address is: C/O Centerprise International, Hampshire International Business Park, Lime Tree Way, Basingstoke, Hampshire, RG24 8GQ. OnCommerce is a registered trademark no. 2540468. OnCommerce Ltd is a Centeprise International Group Company. This electronic message contains information from OnCommerce Ltd, which may be legally privileged and confidential. The information is intended to be for the use of the individual(s) or entity named above. If you are not the intended recipient, be aware that any disclosure, copying, distribution or use of the contents of this information is prohibited. If you have received this electronic message in error, please notify us by telephone (to the number above) or e-mail immediately. Any views, opinions or advice expressed in this electronic message are not necessarily the views, opinions or advice of OnCommerce Ltd. It is the responsibility of the recipient to ensure that any attachments are virus free and OnCommerce Ltd. bear no responsibility for any loss or damage arising in any way from the use thereof. > Simon Lennon, OnCommerce Ltd. > 27 January 2014 11:45 > Hi guys, I was wondering what kind of processing power varnish needs. > I am setting up a dedicated environment with a custom built VCL which > includes some regexp and the load balancing functionality. > > I was going to setup a dual core server with 96GB of ram, is this > enough for a HIGH traffic site, says 10,000 hits a second with very > high cache utilization? > > I will also be using the ban list to achieve a very high level of > caching, with a daily restart to flush the archive, but I would expect > several hundred regex patterns per day in the ban list. > > I know varnish is optimized for performance so I wasn't sure how much > processing power it would need, > > thanks, > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: onbay_email_senor_pinchy.png Type: image/png Size: 4332 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature2_12.jpg Type: image/jpeg Size: 1244 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: onbay_email_oncommerce_logo.png Type: image/png Size: 5758 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: onbay_email_footer_twitter.png Type: image/png Size: 3575 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: onbay_email_footer_facebook.png Type: image/png Size: 2802 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: onbay_email_footer_google.png Type: image/png Size: 2856 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: onbay_email_onbuy.png Type: image/png Size: 5216 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: onbay_email_onrecycle_logo.png Type: image/png Size: 6416 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature2_13.jpg Type: image/jpeg Size: 1298 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: onbay_email_centerprise_logo.png Type: image/png Size: 9852 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: spacer.gif Type: image/gif Size: 13126 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: compose-unknown-contact.jpg Type: image/jpeg Size: 770 bytes Desc: not available URL: From jan at architechs.eu Tue Jan 28 10:47:05 2014 From: jan at architechs.eu (=?iso-8859-1?Q?Jan-Aage_Frydenb=F8-Bruvoll?=) Date: Tue, 28 Jan 2014 10:47:05 +0000 Subject: Processing Power In-Reply-To: <52E77B73.9020407@oncommerce.co.uk> References: <52E646D0.7060704@oncommerce.co.uk> <52E77B73.9020407@oncommerce.co.uk> Message-ID: Hi, We are using lighttpd (not nginx, granted, but the principle is the same) for handling SSL in front of some of our Varnish instances - it works fine from a performance perspective; it's just important to remember to strip and re-add any headers necessary for the SSL handling, so that you don't end up with twice the objects in Varnish. We could have used stunnel or haproxy here, though - this was done for familiarity reasons only. Otherwise we have found no reason for "hiding" Varnish behind other services, as Varnish's caching abilities far surpass any of the other solutions. Performance-wise there might not be so much in it vs a correctly configured nginx instance, but nginx's cache handling is hopeless compared to Varnish. Hope this helps. Best regards Jan From jledford at biltmore.com Tue Jan 28 12:28:43 2014 From: jledford at biltmore.com (Jason Ledford) Date: Tue, 28 Jan 2014 07:28:43 -0500 Subject: Processing Power Message-ID: <435CB3214F92FD4E8E5CEEB86A2044024292269257@MAILBOX.tbcnet.biltmore.com> >From a load balanced perspective, we have an f5 in front, and varnish is running on the same server as our web server, apache. So the web server is setup with varnish on the front end and apache right behind it, same machine. We then duplicated that web to give us our load balanced setup. Load testing blew out the water when we put varnish in front of it compared to just apache. From a scale standpoint you could look at adding components along the way and seeing where it breaks down instead of scaling out immediately. ~Jason Ledford www.biltmore.com -----Original Message----- From: Simon [simon.lennon at oncommerce.co.uk] Received: Tuesday, 28 Jan 2014, 5:49AM To: varnish-misc at varnish-cache.org [varnish-misc at varnish-cache.org] Subject: Re: Processing Power Thanks for the feedback on the processing power that was really helpful, has anyone combined varnish with nginx and used it as a load balancer / caching box. Does the performance requirements change much with nginx sitting in front of varnish? thanks, -- Simon Lennon Technical Director OnCommerce Ltd. +44 (0) 1202 970 940 [cid:part1.01060307.05070500 at oncommerce.co.uk] [cid:part3.06060503.01080309 at oncommerce.co.uk] www.oncommerce.co.uk [cid:part4.05060806.00020202 at oncommerce.co.uk][cid:part5.00080603.02040704 at oncommerce.co.uk][cid:part6.03070606.06090804 at oncommerce.co.uk] [cid:part7.01000102.06080906 at oncommerce.co.uk] www.onbuy.co.uk [cid:part4.05060806.00020202 at oncommerce.co.uk][cid:part5.00080603.02040704 at oncommerce.co.uk][cid:part6.03070606.06090804 at oncommerce.co.uk] [cid:part11.05000105.03060202 at oncommerce.co.uk] www.onrecycle.co.uk [cid:part4.05060806.00020202 at oncommerce.co.uk][cid:part5.00080603.02040704 at oncommerce.co.uk][cid:part6.03070606.06090804 at oncommerce.co.uk] [cid:part16.04060703.01080500 at oncommerce.co.uk] OnBuy and OnRecycle are trading names of OnCommerce Ltd which is a registered company in England & Wales, Company Number: 08358927. Our trading address is: C/O Centerprise International, Hampshire International Business Park, Lime Tree Way, Basingstoke, Hampshire, RG24 8GQ. OnCommerce is a registered trademark no. 2540468. OnCommerce Ltd is a Centeprise International Group Company. [cid:part17.05010105.06080100 at oncommerce.co.uk] This electronic message contains information from OnCommerce Ltd, which may be legally privileged and confidential. The information is intended to be for the use of the individual(s) or entity named above. If you are not the intended recipient, be aware that any disclosure, copying, distribution or use of the contents of this information is prohibited. If you have received this electronic message in error, please notify us by telephone (to the number above) or e-mail immediately. Any views, opinions or advice expressed in this electronic message are not necessarily the views, opinions or advice of OnCommerce Ltd. It is the responsibility of the recipient to ensure that any attachments are virus free and OnCommerce Ltd. bear no responsibility for any loss or damage arising in any way from the use thereof. [cid:part18.01040603.01030201 at oncommerce.co.uk] [cid:part18.01040603.01030201 at oncommerce.co.uk] [cid:part18.01040603.01030201 at oncommerce.co.uk] [cid:part18.01040603.01030201 at oncommerce.co.uk] [cid:part18.01040603.01030201 at oncommerce.co.uk] [cid:part18.01040603.01030201 at oncommerce.co.uk] [cid:part24.00010903.01030109 at oncommerce.co.uk] Simon Lennon, OnCommerce Ltd. 27 January 2014 11:45 Hi guys, I was wondering what kind of processing power varnish needs. I am setting up a dedicated environment with a custom built VCL which includes some regexp and the load balancing functionality. I was going to setup a dual core server with 96GB of ram, is this enough for a HIGH traffic site, says 10,000 hits a second with very high cache utilization? I will also be using the ban list to achieve a very high level of caching, with a daily restart to flush the archive, but I would expect several hundred regex patterns per day in the ban list. I know varnish is optimized for performance so I wasn't sure how much processing power it would need, thanks, _______________________________________________ varnish-misc mailing list varnish-misc at varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: onbay_email_senor_pinchy.png Type: image/png Size: 4332 bytes Desc: onbay_email_senor_pinchy.png URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature2_12.jpg Type: image/jpeg Size: 1244 bytes Desc: signature2_12.jpg URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: onbay_email_oncommerce_logo.png Type: image/png Size: 5758 bytes Desc: onbay_email_oncommerce_logo.png URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: onbay_email_footer_twitter.png Type: image/png Size: 3575 bytes Desc: onbay_email_footer_twitter.png URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: onbay_email_footer_facebook.png Type: image/png Size: 2802 bytes Desc: onbay_email_footer_facebook.png URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: onbay_email_footer_google.png Type: image/png Size: 2856 bytes Desc: onbay_email_footer_google.png URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: onbay_email_onbuy.png Type: image/png Size: 5216 bytes Desc: onbay_email_onbuy.png URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: onbay_email_onrecycle_logo.png Type: image/png Size: 6416 bytes Desc: onbay_email_onrecycle_logo.png URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature2_13.jpg Type: image/jpeg Size: 1298 bytes Desc: signature2_13.jpg URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: onbay_email_centerprise_logo.png Type: image/png Size: 9852 bytes Desc: onbay_email_centerprise_logo.png URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: spacer.gif Type: image/gif Size: 13126 bytes Desc: spacer.gif URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: compose-unknown-contact.jpg Type: image/jpeg Size: 770 bytes Desc: compose-unknown-contact.jpg URL: From seyhunak at gmail.com Wed Jan 29 08:08:13 2014 From: seyhunak at gmail.com (=?ISO-8859-1?Q?Seyhun_Aky=FCrek?=) Date: Wed, 29 Jan 2014 10:08:13 +0200 Subject: Varnish VCL problem - Rails application Message-ID: Hi, I have some questions; I installed and configured Varnish 3, on my Ubuntu 12.04 EC2 instance for Rails 3.2 application. Using Devise for authentication. But the problem is; If I try to login via login form, all redirects doesn't work anymore. I returns back to login after submitting. (sign_in path and 302 redirects) if pass. Varnish configuration: /etc/default/varnish[https://gist.github.com/seyhunak/8664423][1] /etc/varnish/default.vcl[https://gist.github.com/seyhunak/7943277][2] If I try to add this pass to, vcl_recv director, this time causes infinite loop. if(req.url ~ "sign_in" || req.url ~ "sign_out" || req.request == "POST" || req.request == "PUT" || req.request == "DELETE") { return (pass); } What is the workaround? Seyhun. -------------- next part -------------- An HTML attachment was scrubbed... URL: From carloalberto.scaglia at gmail.com Wed Jan 29 11:22:14 2014 From: carloalberto.scaglia at gmail.com (Carlo Alberto Scaglia) Date: Wed, 29 Jan 2014 12:22:14 +0100 Subject: Set Different TTL based on website Message-ID: Hi All, i'm trying to set different TTLs for different sites which are running on the same backend of my Varnish servers, this is how I tried to implement it: - Create different directors for sites (e.g Site1_PRD, Site2_PRD, Site1_STG, Site2_STG) - In sub choose_vhost I set the req.backend based on URL: if ((req.http.host ~ "(?i)^prd.site1.com$") |) ) { set req.backend = Site1_PRD; } - In sub vcl_fetch I inserted an IF statement to set TTLs: sub vcl_fetch { if ( req.backend == Site1_PRD ) { if (req.url !~ "\.aspx" && req.url !~ "\.htmx" && req.url !~ "\.ashx") { unset beresp.http.Set-Cookie; } ... } else { .... The problem is that it seems that this causes a lot of random 503 errors, while backend is working properly and it is also very difficult to manage because for every new site I need to nest another IF in the root else statement (and I think this cause an increased computational load). Is there a better way to configure different TTLs per site? Thanks for the help! Carlo Alberto Scaglia -------------- next part -------------- An HTML attachment was scrubbed... URL: From raymond.jennings at nytimes.com Wed Jan 29 19:56:56 2014 From: raymond.jennings at nytimes.com (Jennings, Raymond) Date: Wed, 29 Jan 2014 14:56:56 -0500 Subject: Varnish serves 503's for 10 second intervals a few times a day Message-ID: There has got to be a way I can drill down as to why Varnish does this. Looking at the varnishncsa output I am seeing 5 to 10 second intervals where all requests are returned as 503's When I look a the backend apache servers there are ZERO requests in these time periods. I see nothing in syslog with any varnishd messages or any other network error. I believe this is internal to Varnish but cannot find anything. Any clues how I can go about digging into this. This is a production system so I cannot modify code but I can insert VCL (C inlined is fine) For now I tried turning on varnishlog (already running varnishncsa.) -------------- next part -------------- An HTML attachment was scrubbed... URL: From smwood4 at gmail.com Wed Jan 29 20:18:18 2014 From: smwood4 at gmail.com (Stephen Wood) Date: Wed, 29 Jan 2014 12:18:18 -0800 Subject: Varnish serves 503's for 10 second intervals a few times a day In-Reply-To: References: Message-ID: How are health checks during and throughout this period? Are they all passing? I would make sure there aren't periods where your backend hosts aren't becoming unhealthy. On Wed, Jan 29, 2014 at 11:56 AM, Jennings, Raymond < raymond.jennings at nytimes.com> wrote: > There has got to be a way I can drill down as to why Varnish does this. > Looking at the varnishncsa output I am seeing 5 to 10 second intervals > where all requests are returned as 503's When I look a the backend apache > servers there are ZERO requests in these time periods. > > I see nothing in syslog with any varnishd messages or any other network > error. I believe this is internal to Varnish but cannot find anything. > Any clues how I can go about digging into this. This is a production > system so I cannot modify code but I can insert VCL (C inlined is fine) > For now I tried turning on varnishlog (already running varnishncsa.) > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -- Stephen Wood www.heystephenwood.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From james at ifixit.com Wed Jan 29 21:33:19 2014 From: james at ifixit.com (James Pearson) Date: Wed, 29 Jan 2014 13:33:19 -0800 Subject: Set Different TTL based on website In-Reply-To: References: Message-ID: <1391030941-sup-6098@geror.local> Excerpts from Carlo Alberto Scaglia's message of 2014-01-29 03:22:14 -0800: > Hi All, > > i'm trying to set different TTLs for different sites which are running on > the same backend of my Varnish servers, this is how I tried to implement it: > > - Create different directors for sites (e.g Site1_PRD, Site2_PRD, > Site1_STG, Site2_STG) > > - In sub choose_vhost I set the req.backend based on URL: > > if ((req.http.host ~ "(?i)^prd.site1.com$") |) ) { set > req.backend = Site1_PRD; } > > - In sub vcl_fetch I inserted an IF statement to set TTLs: > > sub vcl_fetch { > if ( req.backend == Site1_PRD ) { > if (req.url !~ "\.aspx" && req.url !~ "\.htmx" && req.url !~ "\.ashx") { > unset beresp.http.Set-Cookie; } > ... > } else { .... > > The problem is that it seems that this causes a lot of random 503 errors, > while backend is working properly and it is also very difficult to manage > because for every new site I need to nest another IF in the root else > statement (and I think this cause an increased computational load). > Is there a better way to configure different TTLs per site? The best way is to configure your applications to send back proper cache control headers, but I understand this isn't always feasible. Would it help to set the ttl in a variable (that is, a temporary header), then assign it later in one block? if site1: ttl = 5 else if site2: ttl = 10 else: ttl = 15 if long and complex: response.ttl = ttl - P From raymond.jennings at nytimes.com Wed Jan 29 22:06:22 2014 From: raymond.jennings at nytimes.com (Jennings, Raymond) Date: Wed, 29 Jan 2014 17:06:22 -0500 Subject: varnishd randomly gets a SIGINT Message-ID: I see that many people have had this over the years but there is usually one more additional piece of info in syslog I do not see anything other than the basics: Here is my syslog: Jan 20 11:43:12 ip-10-124-15-198 varnishd[32482]: Manager got SIGINT Jan 20 11:43:13 ip-10-124-15-198 varnishd[32592]: Platform: Linux,2.6.32-71.29.1.el6.x86_64,x86_64,-sfile,-smalloc,-hcritbit Jan 20 11:43:13 ip-10-124-15-198 varnishd[32592]: child (32596) Started Jan 20 11:43:13 ip-10-124-15-198 varnishd[32592]: Child (32596) said Child starts Jan 20 11:43:13 ip-10-124-15-198 varnishd[32592]: Child (32596) said SMF.s0 mmap'ed 9663676416 bytes of 9663676416 Here is varnishd running: root 3610 1 0 09:36 ? 00:00:00 /usr/sbin/varnishd -P /var/run/varnish.pid -a :80 -f /etc/varnish/default.vcl -T 10.124.15.198:6082 -t 2400 -w 1,1000,120 -p shm_reclen=1024 -p http_req_hdr_len=17408 -p idle_send_timeout=30 -u varnish -g varnish -S /etc/varnish/secret -s file,/var/lib/varnish/varnish_storage.bin,9G varnish 3614 3610 0 09:36 ? 00:00:37 /usr/sbin/varnishd -P /var/run/varnish.pid -a :80 -f /etc/varnish/default.vcl -T 10.124.15.198:6082 -t 2400 -w 1,1000,120 -p shm_reclen=1024 -p http_req_hdr_len=17408 -p idle_send_timeout=30 -u varnish -g varnish -S /etc/varnish/secret -s file,/var/lib/varnish/varnish_storage.bin,9G None of my filesystems are anywhere near full. None of my filesystems are anywhere near full: -rw------- 1 root root 9663676416 Jan 29 16:50 varnish_storage.bin /tmp is fine, etc. Is there anything I can use to see what is causing this? Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From james at ifixit.com Wed Jan 29 23:34:06 2014 From: james at ifixit.com (James Pearson) Date: Wed, 29 Jan 2014 15:34:06 -0800 Subject: Varnish VCL problem - Rails application In-Reply-To: References: Message-ID: <1391038228-sup-3247@geror.local> Excerpts from Seyhun Aky?rek's message of 2014-01-29 00:08:13 -0800: > Hi, I have some questions; > > I installed and configured Varnish 3, on my Ubuntu 12.04 EC2 instance for > Rails 3.2 application. Using Devise for authentication. > > But the problem is; > > If I try to login via login form, all redirects doesn't work anymore. I > returns back to login after submitting. (sign_in path and 302 redirects) if > pass. > > Varnish configuration: > > /etc/default/varnish[https://gist.github.com/seyhunak/8664423][1] > /etc/varnish/default.vcl[https://gist.github.com/seyhunak/7943277][2] I'm not quite sure what you're trying to acomplish with all the regexes on the Cookie header, but you probably want to have a rule something like this: # Don't cache requests from users who actually have a session cookie if (req.http.Cookie ~ "session=") { return (pass); } # Since we'll be caching this request and Cookies aren't part of the # hash key, don't even bother passing the cookies on to the backend. unset req.http.Cookie; - P