From apj at mutt.dk Fri Jul 1 07:16:04 2016 From: apj at mutt.dk (Andreas Plesner) Date: Fri, 1 Jul 2016 09:16:04 +0200 Subject: ban.list without expression In-Reply-To: References: Message-ID: <20160701071604.GG23724@nerd.dk> On Thu, Jun 30, 2016 at 02:18:17PM +0000, Dead wrote: > > for some strange reason my varnish setup is adding stuff to the ban list but without expressions. > > >?ban.list > 200 > Present bans: > 1467294760.786177 41873 C > 1467294023.391572 20347 C These are (C)ompleted bans. You'll have to catch them before they're completed and the ban expression is deleted. -- Andreas From colas.delmas at gmail.com Fri Jul 1 14:11:28 2016 From: colas.delmas at gmail.com (Nicolas Delmas) Date: Fri, 1 Jul 2016 16:11:28 +0200 Subject: random 503 + FetchError: straight insufficient bytes In-Reply-To: References: Message-ID: But how is it possible ? How the request could be stop between the header and the post data ? We call the URL like this : > var CF_log = {}; > CF_log.referrer = document.referrer; > CF_log.url = .... > > $.CF_ajax({ > type: 'POST', > dataType: 'json', > cache: false, > data: CF_log, > url: 'cf_log.php' > }); I get this when all works : > #### T 2016/06/28 11:34:36.529144 Client:11718 -> Varnish:80 [A] > POST /proxy.php?xdp_path=http%3A%2F*******Fcf_log.php HTTP/1.1..Host: > ******..Connection: keep-alive..Content-Length: 158..Accept: > application/json, text/javascript, */*; q=0.01..Origin:*****. > .X-Requested-With: > XMLHttpRequest..User-Agent: Mozilla/5.0 (X11; Linux x86_64) > AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.87 > Safari/537.36..Content-Type: application/x-www-form-urlencoded; > charset=UTF-8..Referer: ******..Accept-Encoding: gzip, > deflate..Accept-Language: en-US,en;q=0.8,fr;q=0.6..Cookie: > users_res=1920x1200; ************* > > ## > T 2016/06/28 11:34:36.529183 Client:11718 -> Varnish:80 [AP] > ********; awsV=1467102093324...._all_data_POST_ > ##### and this when it failed ## > T 2016/06/29 08:57:32.756481 Client:49781 -> Varnish:80 [AP] > POST /proxy.php?xdp_path=http%3A%2F%2F********2Fcf_log.php > HTTP/1.1..Accept: application/json, text/javascript, */*; > q=0.01..Content-Type: application/x-www-form-urlencoded; > charset=UTF-8..X-Requested-With: XMLHttpRequest..Referer: > ******************..Accept-Language: fr..Accept-Encoding: gzip, > deflate..User-Agent: Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.0; > Trident/5.0)..Host: *****..Content-Length: 434..Connection: > Keep-Alive..Cache-Control: no-cache..Cookie: ; __ybotn=1...._no_POST_data_ > ###### > T 2016/06/29 08:57:36.568554 92.155.255.170:49781 -> 85.116.36.59:80 [AR] > .. # > > Le 29 juin 2016 6:53 PM, "Guillaume Quintard" < guillaume at varnish-software.com> a ?crit : > Isn't it because the client stops writing that post request? > > -- > Guillaume Quintard > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dead_gardens_ at hotmail.com Mon Jul 4 08:27:34 2016 From: dead_gardens_ at hotmail.com (Dead) Date: Mon, 4 Jul 2016 08:27:34 +0000 Subject: ban.list without expression In-Reply-To: <20160701071604.GG23724@nerd.dk> References: , <20160701071604.GG23724@nerd.dk> Message-ID: >On Thu, Jun 30, 2016 at 02:18:17PM +0000, Dead wrote: >> >> for some strange reason my varnish setup is adding stuff to the ban list but without expressions. >> >> > ban.list >> 200 >> Present bans: >> 1467294760.786177 41873 C >> 1467294023.391572 20347 C > >These are (C)ompleted bans. You'll have to catch them before they're completed >and the ban expression is deleted. Thanks Andreas for answering, one extra question :) if these bans are completed, Why the count keep increasing? Ex: varnish> ban.list 200 Present bans: 1467620070.662158 14112 C 1467590933.392531 404731 C 1467561146.261999 603903 C 1467541539.007837 361451 C 1467533857.040905 128274 C 1467494869.660599 330760 C 1467477920.827571 185371 C 1467466291.265457 140189 C 1467464301.371105 21282 C 1467455809.468778 95382 C 1467454428.363314 12323 C 1467453373.069423 11182 C 1467451891.534789 15194 C 1467414299.709805 237760 C 1467408343.964311 57718 C 1467406430.565987 19998 C 1467404610.725681 19571 C 1467398394.284344 62284 C 1467381380.027632 147128 C 1467380528.035235 6380 C 1467369351.559516 81937 C 1467365690.557822 23165 C 1467360882.142809 28493 C varnish> ping 200 PONG 1467620531 1.0 varnish> ban.list 200 Present bans: 1467620070.662158 14366 C 1467590933.392531 404643 C 1467561146.261999 603866 C 1467541539.007837 361442 C 1467533857.040905 128268 C 1467494869.660599 330752 C 1467477920.827571 185369 C 1467466291.265457 140184 C 1467464301.371105 21278 C 1467455809.468778 95380 C 1467454428.363314 12323 C 1467453373.069423 11182 C 1467451891.534789 15193 C 1467414299.709805 237744 C 1467408343.964311 57715 C 1467406430.565987 19998 C 1467404610.725681 19570 C 1467398394.284344 62283 C 1467381380.027632 147126 C 1467380528.035235 6380 C 1467369351.559516 81933 C 1467365690.557822 23165 C 1467360882.142809 28491 C Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: From pinakee at waltzz.com Wed Jul 6 12:47:49 2016 From: pinakee at waltzz.com (Pinakee BIswas) Date: Wed, 6 Jul 2016 18:17:49 +0530 Subject: Cache invalidation is taking time - affecting product management and marketing Message-ID: <90b94a7d-e776-729b-8ec8-c82c47e985bc@waltzz.com> Hi, We are using PURGE or varnishadm ban to invalidate the cache but it's taking time for the same. Morever, it is not clear how long it takes. It is not clear which method is better and smoother - PURGE or ban. Is there a way to control the timing of cache clearing as it's affecting our product management and marketing? Thanks, Pinakee From apj at mutt.dk Wed Jul 6 13:47:40 2016 From: apj at mutt.dk (Andreas Plesner) Date: Wed, 6 Jul 2016 15:47:40 +0200 Subject: Cache invalidation is taking time - affecting product management and marketing In-Reply-To: <90b94a7d-e776-729b-8ec8-c82c47e985bc@waltzz.com> References: <90b94a7d-e776-729b-8ec8-c82c47e985bc@waltzz.com> Message-ID: <20160706134740.GL23724@nerd.dk> On Wed, Jul 06, 2016 at 06:17:49PM +0530, Pinakee BIswas wrote: > > We are using PURGE or varnishadm ban to invalidate the cache but it's taking > time for the same. Morever, it is not clear how long it takes. It is not > clear which method is better and smoother - PURGE or ban. > > Is there a way to control the timing of cache clearing as it's affecting our > product management and marketing? Both have immediate effect, so you need to go back and re-analyze your problem. -- Andreas From guillaume at varnish-software.com Wed Jul 6 20:35:30 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Wed, 6 Jul 2016 13:35:30 -0700 Subject: Question about variables In-Reply-To: References: Message-ID: I haven't checked the code, but it's probably because var.get returns "" when it can't find the variables. So the condition is true, there is a string, even though it's empty. Maybe try with : if (var.get(.....) ~ ".") just to make sure it's not empty You can also use sdt.log() to try and print what's inside various variables and debug that using varnishlog. -- Guillaume Quintard -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Fri Jul 8 14:25:54 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Fri, 8 Jul 2016 16:25:54 +0200 Subject: random 503 + FetchError: straight insufficient bytes In-Reply-To: References: Message-ID: The client may stop transmitting, for example. Or it could be that the POST request header are wrong/misleading. -- Guillaume Quintard -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Fri Jul 8 14:40:17 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Fri, 8 Jul 2016 16:40:17 +0200 Subject: Malicious redirection in Varnish? In-Reply-To: <1520549204.2727831.1465969921294.JavaMail.yahoo@mail.yahoo.com> References: <1520549204.2727831.1465969921294.JavaMail.yahoo.ref@mail.yahoo.com> <1520549204.2727831.1465969921294.JavaMail.yahoo@mail.yahoo.com> Message-ID: We actually need the bereq varnishlog to get an idea of what's going on. -- Guillaume Quintard -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Fri Jul 8 14:43:52 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Fri, 8 Jul 2016 16:43:52 +0200 Subject: Custom TTLs In-Reply-To: References: Message-ID: Hi, the attack header should be available in v_b_r as bereq.http.attack, as req is copied over as bereq and only slightly edited by Varnish (to remove the range-request header, for example). Just a note, if you are attacked, v_b_r is probably a bit late to respond, as the backend as already been bothered. YMWV, of course. -- Guillaume Quintard -------------- next part -------------- An HTML attachment was scrubbed... URL: From ianmac51 at gmail.com Sun Jul 10 07:25:42 2016 From: ianmac51 at gmail.com (Ian Macdonald) Date: Sun, 10 Jul 2016 08:25:42 +0100 Subject: Help with vcl file Message-ID: Hi All, New to varnish and this is probably something silly, but I have been looking and nothing jumps out. The below config is just me playing and learning, it works with ubuntu 16.04 and varnish 4.1.1 but not on CentOS 7 with varnish 4.1.3, if I request the / I do get a synth response, but if I request /index.html I get my custom 503 page, index.html is on the backend and will load if I use a basic config, any suggestions would be greatly appreciated. Best regards Mackyd # Playing with Varnish 4 # need to tell varnish what version the vcl is for vcl 4.0; # don't forget if you need a module function you need to import it import std; import vsthrottle; import tcp; backend default { .host = "192.168.1.71"; .port = "8081"; .max_connections = 3; } sub vcl_recv { # Strip any request cookies unset req.http.cookie; # this could be of use for testing sends tcp data to log file tcp.dump_info(); # Just playing with throttle here, practice modules usage # This works for client ip as expected. if (vsthrottle.is_denied(client.ip, 15, 10s)) { # Client has exceeded 15 reqs per 10s return (synth(429, "Too Many Requests")); } # respond HTTP 200 to / requests # LB Health check if (req.url ~ "^/$") { return (synth(750)); } if (req.url ~ "^/none.html$") { return (synth(760)); } # Make sure the LB Health check is passed through # to backend if (req.url ~ "^/search\?q=chutney$") { return (pass); } # backend pass through to test max_connections if (req.url ~ "^/bepass.html$") { return (pass); } } sub vcl_backend_response { if (bereq.url ~ "^/suggest\?") { set beresp.ttl = 360m; } else { set beresp.ttl = 2m; } } sub vcl_synth { if (resp.status == 750) { set resp.status = 200; # we probably don't need the synthetic here # if the load balancer is only looking for # status 200 but it is useful to know what # is being sent synthetic("Response to load balancer health probe"); return(deliver); } if (resp.status == 760) { set resp.status = 200; synthetic(std.fileread("/etc/varnish/500.html")); return(deliver); } } sub vcl_backend_error { if (beresp.status == 503) { set beresp.http.Content-Type = "text/html; charset=utf-8"; synthetic(std.fileread("/etc/varnish/500.html")); return (deliver); } /* The 'else' does not seem to get called and the 404 * is passed through from the backend, need to figure * out why at some point */ else { set beresp.http.Content-Type = "text/html; charset=utf-8"; synthetic( {" "} + beresp.status + " " + beresp.reason + {"

Error "} + beresp.status + " " + beresp.reason + {"

"} + beresp.reason + {"

Guru Meditation:

XID: "} + bereq.xid + {"

Varnish cache server error from else section vcl_backend_error

"} ); return (deliver); } } From dridi at varni.sh Mon Jul 11 13:11:01 2016 From: dridi at varni.sh (Dridi Boukelmoune) Date: Mon, 11 Jul 2016 15:11:01 +0200 Subject: Help with vcl file In-Reply-To: References: Message-ID: On Sun, Jul 10, 2016 at 9:25 AM, Ian Macdonald wrote: > Hi All, > > New to varnish and this is probably something silly, but I have been > looking and nothing jumps out. > > The below config is just me playing and learning, it works with ubuntu > 16.04 and varnish 4.1.1 but not on CentOS 7 with varnish 4.1.3, if I > request the / I do get a synth response, but if I request /index.html > I get my custom 503 page, index.html is on the backend and will load > if I use a basic config, any suggestions would be greatly appreciated. Hi, Is your backend healthy from Varnish's point of view both on Ubuntu and CentOS? varnishadm backend.list Dridi From ianmac51 at gmail.com Mon Jul 11 18:56:05 2016 From: ianmac51 at gmail.com (Ian Macdonald) Date: Mon, 11 Jul 2016 19:56:05 +0100 Subject: Help with vcl file In-Reply-To: References: Message-ID: Hi Dridi, I added a probe to my config and centos is showing sick with ubuntu showing healthy, they are the exact same config and the exact same backend I have disabled firewalld on centos and I can reach the backend using curl or httpie from the command line on centos All the best Mackyd On 11 July 2016 at 14:11, Dridi Boukelmoune wrote: > On Sun, Jul 10, 2016 at 9:25 AM, Ian Macdonald wrote: >> Hi All, >> >> New to varnish and this is probably something silly, but I have been >> looking and nothing jumps out. >> >> The below config is just me playing and learning, it works with ubuntu >> 16.04 and varnish 4.1.1 but not on CentOS 7 with varnish 4.1.3, if I >> request the / I do get a synth response, but if I request /index.html >> I get my custom 503 page, index.html is on the backend and will load >> if I use a basic config, any suggestions would be greatly appreciated. > > Hi, > > Is your backend healthy from Varnish's point of view both on Ubuntu and CentOS? > > varnishadm backend.list > > Dridi From dridi at varni.sh Tue Jul 12 09:51:37 2016 From: dridi at varni.sh (Dridi Boukelmoune) Date: Tue, 12 Jul 2016 11:51:37 +0200 Subject: Help with vcl file In-Reply-To: References: Message-ID: On Mon, Jul 11, 2016 at 8:56 PM, Ian Macdonald wrote: > Hi Dridi, > > I added a probe to my config and centos is showing sick with ubuntu > showing healthy, they are the exact same config and the exact same > backend I have disabled firewalld on centos and I can reach the > backend using curl or httpie from the command line on centos Hi Ian, Second random guess, is SELinux enabled? If so, does it work once disabled? Dridi From cello86 at gmail.com Tue Jul 12 10:09:04 2016 From: cello86 at gmail.com (Marcello Lorenzi) Date: Tue, 12 Jul 2016 12:09:04 +0200 Subject: Varnish 4.x and conditional request Message-ID: Hi All, we're checking the possibility to avoid the checking for an amount of time the forwarding of the conditional requests to a backend servers and use the object cached if it's present on varnish storage. Is it possible to force this caching via beresp.ttl and beresp.keep? Thanks, Marcello -------------- next part -------------- An HTML attachment was scrubbed... URL: From ianmac51 at gmail.com Tue Jul 12 10:50:17 2016 From: ianmac51 at gmail.com (Ian Macdonald) Date: Tue, 12 Jul 2016 11:50:17 +0100 Subject: Help with vcl file In-Reply-To: References: Message-ID: <03E6C1AD-9830-4EFE-9B6E-6E273133546F@gmail.com> Dridi Thank you very much, second guess was it, will have to look at selinux but for now just disabled it for test and it works Thanks again Mackyd > On 12 Jul 2016, at 10:51, Dridi Boukelmoune wrote: > >> On Mon, Jul 11, 2016 at 8:56 PM, Ian Macdonald wrote: >> Hi Dridi, >> >> I added a probe to my config and centos is showing sick with ubuntu >> showing healthy, they are the exact same config and the exact same >> backend I have disabled firewalld on centos and I can reach the >> backend using curl or httpie from the command line on centos > > Hi Ian, > > Second random guess, is SELinux enabled? If so, does it work once disabled? > > Dridi From dridi at varni.sh Tue Jul 12 12:33:11 2016 From: dridi at varni.sh (Dridi Boukelmoune) Date: Tue, 12 Jul 2016 14:33:11 +0200 Subject: Varnish 4.x and conditional request In-Reply-To: References: Message-ID: On Tue, Jul 12, 2016 at 12:09 PM, Marcello Lorenzi wrote: > Hi All, > we're checking the possibility to avoid the checking for an amount of time > the forwarding of the conditional requests to a backend servers and use the > object cached if it's present on varnish storage. You get this behavior by default if you use grace mode and have proper conditional headers. > Is it possible to force this caching via beresp.ttl and beresp.keep? Once you go beyond the ttl and grace periods, there's no way to serve the object from the cache. During the keep period, the object can only be used for a conditional backend request. Dridi From sujithnss at gmail.com Wed Jul 13 11:45:55 2016 From: sujithnss at gmail.com (sujith pv) Date: Wed, 13 Jul 2016 17:15:55 +0530 Subject: Varnich Memory in 90% holding continously Message-ID: Hi Team I have just started using Varnish in product environment. We did a load test (matching the Live load) in our application. Please find the below observations - Varnish with memalloc 8Gb - Varnish TTL is 2 hours - Varnish was idle just prior to test. But still memory usage was 30 % - Varnish log size was in few Kbs only - During the first 30 minutes of test, Varnish memory reached 90% . - Varnish log size reached 1 GB. - Immediately we stopped our test. After a week again we just checked the Varnish memory (until this Varnish was idle) , still it was showing up as 90% as it was during the test So I was totally confused in alaysing this issue. Could you guys share some thoughts around this please... Best Regards Sujith P V -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Wed Jul 13 12:02:25 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Wed, 13 Jul 2016 14:02:25 +0200 Subject: Varnich Memory in 90% holding continously In-Reply-To: References: Message-ID: Hi, How much RAM do you have on that machine? If the answer is 8GB, it means you allocated all the RAM, leaving nothing for the other applications. What do you mean by "varnish log size"? Do you redirect the output of varnishlog to a file? In that case, it's not suprising, varnishlog is extremely verbose, and you probably want to use varnishncsa instead. Best -- Guillaume Quintard -------------- next part -------------- An HTML attachment was scrubbed... URL: From sujithnss at gmail.com Wed Jul 13 12:27:49 2016 From: sujithnss at gmail.com (sujith pv) Date: Wed, 13 Jul 2016 17:57:49 +0530 Subject: Varnich Memory in 90% holding continously In-Reply-To: References: Message-ID: Thanks. Please find the stats attached.. On Wed, Jul 13, 2016 at 5:44 PM, Guillaume Quintard < guillaume at varnish-software.com> wrote: > You should keep that conversation o nthe mailing-list if you want people > smarter than me to answer you :-) > > Is that virtual or real memory? Can you show me the output of "ps -aux"? > > -- > Guillaume Quintard > > On Wed, Jul 13, 2016 at 2:08 PM, sujith pv wrote: > >> Hi >> >> I have 16GB RAM and only Varnish is the application running there. Yes we >> are writing the log to a flie which in turn managed via logrotate to clear >> it daily basis. >> But here the surprising fact is still after a week time, being idle why >> memory is showing up as still 90%. >> >> >> On Wed, Jul 13, 2016 at 5:32 PM, Guillaume Quintard < >> guillaume at varnish-software.com> wrote: >> >>> Hi, >>> >>> How much RAM do you have on that machine? If the answer is 8GB, it means >>> you allocated all the RAM, leaving nothing for the other applications. >>> >>> What do you mean by "varnish log size"? Do you redirect the output of >>> varnishlog to a file? In that case, it's not suprising, varnishlog is >>> extremely verbose, and you probably want to use varnishncsa instead. >>> >>> Best >>> >>> -- >>> Guillaume Quintard >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- $ ps -aux Warning: bad syntax, perhaps a bogus '-'? See /usr/share/doc/procps-3.2.8/FAQ USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 1 0.0 0.0 21452 1388 ? Ss Jan06 0:06 /sbin/init root 2 0.0 0.0 0 0 ? S Jan06 0:03 [kthreadd] root 3 0.0 0.0 0 0 ? S Jan06 0:16 [migration/0] root 4 0.0 0.0 0 0 ? S Jan06 0:24 [ksoftirqd/0] root 5 0.0 0.0 0 0 ? S Jan06 0:00 [migration/0] root 6 0.0 0.0 0 0 ? S Jan06 0:26 [watchdog/0] root 7 0.0 0.0 0 0 ? S Jan06 0:17 [migration/1] root 8 0.0 0.0 0 0 ? S Jan06 0:00 [migration/1] root 9 0.0 0.0 0 0 ? S Jan06 0:15 [ksoftirqd/1] root 10 0.0 0.0 0 0 ? S Jan06 0:28 [watchdog/1] root 11 0.0 0.0 0 0 ? S Jan06 0:17 [migration/2] root 12 0.0 0.0 0 0 ? S Jan06 0:00 [migration/2] root 13 0.0 0.0 0 0 ? S Jan06 0:10 [ksoftirqd/2] root 14 0.0 0.0 0 0 ? S Jan06 0:27 [watchdog/2] root 15 0.0 0.0 0 0 ? S Jan06 0:16 [migration/3] root 16 0.0 0.0 0 0 ? S Jan06 0:00 [migration/3] root 17 0.0 0.0 0 0 ? S Jan06 0:10 [ksoftirqd/3] root 18 0.0 0.0 0 0 ? S Jan06 0:28 [watchdog/3] root 19 0.0 0.0 0 0 ? S Jan06 0:16 [migration/4] root 20 0.0 0.0 0 0 ? S Jan06 0:00 [migration/4] root 21 0.0 0.0 0 0 ? S Jan06 0:08 [ksoftirqd/4] root 22 0.0 0.0 0 0 ? S Jan06 0:28 [watchdog/4] root 23 0.0 0.0 0 0 ? S Jan06 0:16 [migration/5] root 24 0.0 0.0 0 0 ? S Jan06 0:00 [migration/5] root 25 0.0 0.0 0 0 ? S Jan06 0:08 [ksoftirqd/5] root 26 0.0 0.0 0 0 ? S Jan06 0:28 [watchdog/5] root 27 0.0 0.0 0 0 ? S Jan06 0:16 [migration/6] root 28 0.0 0.0 0 0 ? S Jan06 0:00 [migration/6] root 29 0.0 0.0 0 0 ? S Jan06 0:08 [ksoftirqd/6] root 30 0.0 0.0 0 0 ? S Jan06 0:26 [watchdog/6] root 31 0.0 0.0 0 0 ? S Jan06 0:16 [migration/7] root 32 0.0 0.0 0 0 ? S Jan06 0:00 [migration/7] root 33 0.0 0.0 0 0 ? S Jan06 0:07 [ksoftirqd/7] root 34 0.0 0.0 0 0 ? S Jan06 0:26 [watchdog/7] root 35 0.0 0.0 0 0 ? S Jan06 17:54 [events/0] root 36 0.0 0.0 0 0 ? S Jan06 10:13 [events/1] root 37 0.0 0.0 0 0 ? S Jan06 10:41 [events/2] root 38 0.0 0.0 0 0 ? S Jan06 11:02 [events/3] root 39 0.0 0.0 0 0 ? S Jan06 16:44 [events/4] root 40 0.0 0.0 0 0 ? S Jan06 12:18 [events/5] root 41 0.0 0.0 0 0 ? S Jan06 12:34 [events/6] root 42 0.0 0.0 0 0 ? S Jan06 15:39 [events/7] root 43 0.0 0.0 0 0 ? S Jan06 0:00 [cgroup] root 44 0.0 0.0 0 0 ? S Jan06 0:03 [khelper] root 45 0.0 0.0 0 0 ? S Jan06 0:00 [netns] root 46 0.0 0.0 0 0 ? S Jan06 0:00 [async/mgr] root 47 0.0 0.0 0 0 ? S Jan06 0:00 [pm] root 48 0.0 0.0 0 0 ? S Jan06 0:00 [xenwatch] root 49 0.0 0.0 0 0 ? S Jan06 0:50 [xenbus] root 50 0.0 0.0 0 0 ? S Jan06 1:13 [sync_supers] root 51 0.0 0.0 0 0 ? S Jan06 1:17 [bdi-default] root 52 0.0 0.0 0 0 ? S Jan06 0:00 [kintegrityd/0] root 53 0.0 0.0 0 0 ? S Jan06 0:00 [kintegrityd/1] root 54 0.0 0.0 0 0 ? S Jan06 0:00 [kintegrityd/2] root 55 0.0 0.0 0 0 ? S Jan06 0:00 [kintegrityd/3] root 56 0.0 0.0 0 0 ? S Jan06 0:00 [kintegrityd/4] root 57 0.0 0.0 0 0 ? S Jan06 0:00 [kintegrityd/5] root 58 0.0 0.0 0 0 ? S Jan06 0:00 [kintegrityd/6] root 59 0.0 0.0 0 0 ? S Jan06 0:00 [kintegrityd/7] root 60 0.0 0.0 0 0 ? S Jan06 2:13 [kblockd/0] root 61 0.0 0.0 0 0 ? S Jan06 0:00 [kblockd/1] root 62 0.0 0.0 0 0 ? S Jan06 0:00 [kblockd/2] root 63 0.0 0.0 0 0 ? S Jan06 0:00 [kblockd/3] root 64 0.0 0.0 0 0 ? S Jan06 0:00 [kblockd/4] root 65 0.0 0.0 0 0 ? S Jan06 0:00 [kblockd/5] root 66 0.0 0.0 0 0 ? S Jan06 0:00 [kblockd/6] root 67 0.0 0.0 0 0 ? S Jan06 0:00 [kblockd/7] root 68 0.0 0.0 0 0 ? S Jan06 0:00 [ata_aux] root 69 0.0 0.0 0 0 ? S Jan06 0:00 [ata_sff/0] root 70 0.0 0.0 0 0 ? S Jan06 0:00 [ata_sff/1] root 71 0.0 0.0 0 0 ? S Jan06 0:00 [ata_sff/2] root 72 0.0 0.0 0 0 ? S Jan06 0:00 [ata_sff/3] root 73 0.0 0.0 0 0 ? S Jan06 0:00 [ata_sff/4] root 74 0.0 0.0 0 0 ? S Jan06 0:00 [ata_sff/5] root 75 0.0 0.0 0 0 ? S Jan06 0:00 [ata_sff/6] root 76 0.0 0.0 0 0 ? S Jan06 0:00 [ata_sff/7] root 77 0.0 0.0 0 0 ? S Jan06 0:00 [ksuspend_usbd] root 78 0.0 0.0 0 0 ? S Jan06 0:00 [khubd] root 79 0.0 0.0 0 0 ? S Jan06 0:00 [kseriod] root 80 0.0 0.0 0 0 ? S Jan06 0:00 [md/0] root 81 0.0 0.0 0 0 ? S Jan06 0:00 [md/1] root 82 0.0 0.0 0 0 ? S Jan06 0:00 [md/2] root 83 0.0 0.0 0 0 ? S Jan06 0:00 [md/3] root 84 0.0 0.0 0 0 ? S Jan06 0:00 [md/4] root 85 0.0 0.0 0 0 ? S Jan06 0:00 [md/5] root 86 0.0 0.0 0 0 ? S Jan06 0:00 [md/6] root 87 0.0 0.0 0 0 ? S Jan06 0:00 [md/7] root 88 0.0 0.0 0 0 ? S Jan06 0:00 [md_misc/0] root 89 0.0 0.0 0 0 ? S Jan06 0:00 [md_misc/1] root 90 0.0 0.0 0 0 ? S Jan06 0:00 [md_misc/2] root 91 0.0 0.0 0 0 ? S Jan06 0:00 [md_misc/3] root 92 0.0 0.0 0 0 ? S Jan06 0:00 [md_misc/4] root 93 0.0 0.0 0 0 ? S Jan06 0:00 [md_misc/5] root 94 0.0 0.0 0 0 ? S Jan06 0:00 [md_misc/6] root 95 0.0 0.0 0 0 ? S Jan06 0:00 [md_misc/7] root 96 0.0 0.0 0 0 ? S Jan06 0:00 [linkwatch] root 97 0.0 0.0 0 0 ? S Jan06 0:12 [khungtaskd] root 98 0.0 0.0 0 0 ? S Jan06 0:05 [kswapd0] root 99 0.0 0.0 0 0 ? SN Jan06 0:00 [ksmd] root 100 0.0 0.0 0 0 ? S Jan06 0:00 [aio/0] root 101 0.0 0.0 0 0 ? S Jan06 0:00 [aio/1] root 102 0.0 0.0 0 0 ? S Jan06 0:00 [aio/2] root 103 0.0 0.0 0 0 ? S Jan06 0:00 [aio/3] root 104 0.0 0.0 0 0 ? S Jan06 0:00 [aio/4] root 105 0.0 0.0 0 0 ? S Jan06 0:00 [aio/5] root 106 0.0 0.0 0 0 ? S Jan06 0:00 [aio/6] root 107 0.0 0.0 0 0 ? S Jan06 0:00 [aio/7] root 108 0.0 0.0 0 0 ? S Jan06 0:00 [crypto/0] root 109 0.0 0.0 0 0 ? S Jan06 0:00 [crypto/1] root 110 0.0 0.0 0 0 ? S Jan06 0:00 [crypto/2] root 111 0.0 0.0 0 0 ? S Jan06 0:00 [crypto/3] root 112 0.0 0.0 0 0 ? S Jan06 0:00 [crypto/4] root 113 0.0 0.0 0 0 ? S Jan06 0:00 [crypto/5] root 114 0.0 0.0 0 0 ? S Jan06 0:00 [crypto/6] root 115 0.0 0.0 0 0 ? S Jan06 0:00 [crypto/7] root 120 0.0 0.0 0 0 ? S Jan06 0:00 [kthrotld/0] root 121 0.0 0.0 0 0 ? S Jan06 0:00 [kthrotld/1] root 122 0.0 0.0 0 0 ? S Jan06 0:00 [kthrotld/2] root 123 0.0 0.0 0 0 ? S Jan06 0:00 [kthrotld/3] root 124 0.0 0.0 0 0 ? S Jan06 0:00 [kthrotld/4] root 125 0.0 0.0 0 0 ? S Jan06 0:00 [kthrotld/5] root 126 0.0 0.0 0 0 ? S Jan06 0:00 [kthrotld/6] root 127 0.0 0.0 0 0 ? S Jan06 0:00 [kthrotld/7] root 129 0.0 0.0 0 0 ? S Jan06 0:00 [khvcd] root 130 0.0 0.0 0 0 ? S Jan06 0:00 [kpsmoused] root 131 0.0 0.0 0 0 ? S Jan06 0:00 [usbhid_resume] root 162 0.0 0.0 0 0 ? S Jan06 0:00 [kstriped] root 336 0.0 0.0 0 0 ? S Jan06 0:00 [kdmflush] root 338 0.0 0.0 0 0 ? S Jan06 0:00 [kdmflush] root 355 0.0 0.0 0 0 ? S Jan06 0:10 [jbd2/dm-0-8] root 356 0.0 0.0 0 0 ? S Jan06 0:00 [ext4-dio-unwr] root 452 0.0 0.0 11212 328 ? S References: Message-ID: "ps aux > foo" to get the full width of the output collected in the foo file. We need the full command lines used too. -- Guillaume Quintard -------------- next part -------------- An HTML attachment was scrubbed... URL: From sujithnss at gmail.com Wed Jul 13 12:37:57 2016 From: sujithnss at gmail.com (sujith pv) Date: Wed, 13 Jul 2016 18:07:57 +0530 Subject: Varnich Memory in 90% holding continously In-Reply-To: References: Message-ID: Looks like even in foo file its the same result as I posted earlier. On Wed, Jul 13, 2016 at 6:04 PM, Guillaume Quintard < guillaume at varnish-software.com> wrote: > "ps aux > foo" to get the full width of the output collected in the foo > file. We need the full command lines used too. > > -- > Guillaume Quintard > -------------- next part -------------- An HTML attachment was scrubbed... URL: From arnall2009 at gmail.com Wed Jul 13 13:52:47 2016 From: arnall2009 at gmail.com (Arnall) Date: Wed, 13 Jul 2016 15:52:47 +0200 Subject: Varnish 4.1 : std.healthy and saint mode Message-ID: <4c202f80-cc5f-6ffe-8a49-ec2ad78463b0@gmail.com> Hello everyone, do you know if there's a way to make std.healthy(req.backend_hint) work with saint mode ? Actually it returns true even if the backend is blacklisted by saint mode for a particular request. So i can't make a difference in vcl_hit : if (std.healthy(req.backend_hint)) { if (obj.ttl + obj.grace > 0s) { #stale-while-revalidate return (deliver); } } else { if (obj.ttl + obj.grace + obj.keep > 0s) { #stale-if-error return (deliver); } } to simulate stale-while-revalidate and stale-if-error. (works well if the backend is really down) Thanks. From sujithnss at gmail.com Wed Jul 13 17:21:23 2016 From: sujithnss at gmail.com (sujith pv) Date: Wed, 13 Jul 2016 22:51:23 +0530 Subject: Varnich Memory in 90% holding continously In-Reply-To: References: Message-ID: Any help... On Wed, Jul 13, 2016 at 6:07 PM, sujith pv wrote: > Looks like even in foo file its the same result as I posted earlier. > > On Wed, Jul 13, 2016 at 6:04 PM, Guillaume Quintard < > guillaume at varnish-software.com> wrote: > >> "ps aux > foo" to get the full width of the output collected in the foo >> file. We need the full command lines used too. >> >> -- >> Guillaume Quintard >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From apj at mutt.dk Wed Jul 13 17:56:53 2016 From: apj at mutt.dk (Andreas Plesner) Date: Wed, 13 Jul 2016 19:56:53 +0200 Subject: Varnich Memory in 90% holding continously In-Reply-To: References: Message-ID: <20160713175653.GO23724@nerd.dk> On Wed, Jul 13, 2016 at 05:15:55PM +0530, sujith pv wrote: > > I have just started using Varnish in product environment. We did a load > test (matching the Live load) in our application. Please find the below > observations > > - Varnish with memalloc 8Gb > - Varnish TTL is 2 hours > - Varnish was idle just prior to test. But still memory usage was 30 % 30% of what? Measured how? > After a week again we just checked the Varnish memory (until this Varnish > was idle) , still it was showing up as 90% as it was during the test > So I was totally confused in alaysing this issue. Could you guys share some > thoughts around this please... Show us your vcl and the output of "varnishstat -1" -- Andreas From sujithnss at gmail.com Thu Jul 14 06:29:40 2016 From: sujithnss at gmail.com (sujith pv) Date: Thu, 14 Jul 2016 07:29:40 +0100 Subject: Varnich Memory in 90% holding continously In-Reply-To: <20160713175653.GO23724@nerd.dk> References: <20160713175653.GO23724@nerd.dk> Message-ID: Hi Andreas Plesner Please find the attached. Here in above mail 30% memory usage at that time prior to test. On Wed, Jul 13, 2016 at 6:56 PM, Andreas Plesner wrote: > On Wed, Jul 13, 2016 at 05:15:55PM +0530, sujith pv wrote: > > > > I have just started using Varnish in product environment. We did a load > > test (matching the Live load) in our application. Please find the below > > observations > > > > - Varnish with memalloc 8Gb > > - Varnish TTL is 2 hours > > - Varnish was idle just prior to test. But still memory usage was 30 % > > 30% of what? Measured how? > > > After a week again we just checked the Varnish memory (until this Varnish > > was idle) , still it was showing up as 90% as it was during the test > > So I was totally confused in alaysing this issue. Could you guys share > some > > thoughts around this please... > > Show us your vcl and the output of "varnishstat -1" > > -- > Andreas > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- # # This is an example VCL file for Varnish. # # It does not do anything by default, delegating control to the # builtin VCL. The builtin VCL is called when there is no explicit # return statement. # # See the VCL chapters in the Users Guide at https://www.varnish-cache.org/docs/ # and http://varnish-cache.org/trac/wiki/VCLExamples for more examples. # Marker to tell the VCL compiler that this VCL has been adapted to the # new 4.0 format. vcl 4.0; # Default backend definition. Set this to point to your content server. backend default { #Endeca-VIP .host = "172.17.204.254"; #MFEN01 #.host = "172.25.33.125"; #.host = "localhost"; .port = "15001"; #.port = ""; } sub vcl_recv { # Happens before we check if we have this in cache already. # # Typically you clean up the request here, removing cookies you don't need, # rewriting the request, etc. } sub vcl_backend_response { # Happens after we have read the response headers from the backend. # # Here you clean the response headers, removing silly Set-Cookie headers # and other mistakes your backend does. } sub vcl_deliver { # Happens when we have all the pieces we need, and are about to send the # response to the client. # # You can do accounting or modifying the final object here. } -------------- next part -------------- MEMPOOL.req0.randry 98 0.00 Pool ran dry MEMPOOL.sess0.live 0 . In use MEMPOOL.sess0.pool 10 . In Pool MEMPOOL.sess0.sz_wanted 384 . Size requested MEMPOOL.sess0.sz_needed 416 . Size allocated MEMPOOL.sess0.allocs 516396 0.07 Allocations MEMPOOL.sess0.frees 516396 0.07 Frees MEMPOOL.sess0.recycle 516283 0.07 Recycled from pool MEMPOOL.sess0.timeout 23605 0.00 Timed out from pool MEMPOOL.sess0.toosmall 0 0.00 Too small to recycle MEMPOOL.sess0.surplus 5 0.00 Too many for pool MEMPOOL.sess0.randry 113 0.00 Pool ran dry MEMPOOL.req1.live 0 . In use MEMPOOL.req1.pool 10 . In Pool MEMPOOL.req1.sz_wanted 65536 . Size requested MEMPOOL.req1.sz_needed 65568 . Size allocated MEMPOOL.req1.allocs 735530 0.09 Allocations MEMPOOL.req1.frees 735530 0.09 Frees MEMPOOL.req1.recycle 735436 0.09 Recycled from pool MEMPOOL.req1.timeout 22093 0.00 Timed out from pool MEMPOOL.req1.toosmall 0 0.00 Too small to recycle MEMPOOL.req1.surplus 26 0.00 Too many for pool MEMPOOL.req1.randry 94 0.00 Pool ran dry MEMPOOL.sess1.live 0 . In use MEMPOOL.sess1.pool 10 . In Pool MEMPOOL.sess1.sz_wanted 384 . Size requested MEMPOOL.sess1.sz_needed 416 . Size allocated MEMPOOL.sess1.allocs 516408 0.07 Allocations MEMPOOL.sess1.frees 516408 0.07 Frees MEMPOOL.sess1.recycle 516307 0.07 Recycled from pool MEMPOOL.sess1.timeout 23373 0.00 Timed out from pool MEMPOOL.sess1.toosmall 0 0.00 Too small to recycle MEMPOOL.sess1.surplus 13 0.00 Too many for pool MEMPOOL.sess1.randry 101 0.00 Pool ran dry SMA.s0.c_req 1280967 0.16 Allocator requests SMA.s0.c_fail 281299 0.04 Allocator failures SMA.s0.c_bytes 24345264927 3103.61 Bytes allocated SMA.s0.c_freed 24345264355 3103.61 Bytes freed SMA.s0.g_alloc 2 . Allocations outstanding SMA.s0.g_bytes 572 . Bytes outstanding SMA.s0.g_space 8589934020 . Bytes available SMA.Transient.c_req 1326 0.00 Allocator requests SMA.Transient.c_fail 0 0.00 Allocator failures SMA.Transient.c_bytes 560017 0.07 Bytes allocated SMA.Transient.c_freed 560017 0.07 Bytes freed SMA.Transient.g_alloc 0 . Allocations outstanding SMA.Transient.g_bytes 0 . Bytes outstanding SMA.Transient.g_space 0 . Bytes available VBE.default(172.17.204.254,,15001).vcls 1 . VCL references VBE.default(172.17.204.254,,15001).happy 0 . Happy health probes VBE.default(172.17.204.254,,15001).bereq_hdrbytes 552916838 70.49 Request header bytes VBE.default(172.17.204.254,,15001).bereq_bodybytes 0 0.00 Request body bytes VBE.default(172.17.204.254,,15001).beresp_hdrbytes 25039794 3.19 Response header bytes VBE.default(172.17.204.254,,15001).beresp_bodybytes 23799812406 3034.07 Response body bytes VBE.default(172.17.204.254,,15001).pipe_hdrbytes 0 0.00 Pipe request header bytes VBE.default(172.17.204.254,,15001).pipe_out 0 0.00 Piped bytes to backend VBE.default(172.17.204.254,,15001).pipe_in 0 0.00 Piped bytes from backend LCK.sms.creat 0 0.00 Created locks LCK.sms.destroy 0 0.00 Destroyed locks LCK.sms.locks 0 0.00 Lock Operations LCK.smp.creat 0 0.00 Created locks LCK.smp.destroy 0 0.00 Destroyed locks LCK.smp.locks 0 0.00 Lock Operations LCK.sma.creat 2 0.00 Created locks LCK.sma.destroy 0 0.00 Destroyed locks LCK.sma.locks 2283285 0.29 Lock Operations LCK.smf.creat 0 0.00 Created locks LCK.smf.destroy 0 0.00 Destroyed locks LCK.smf.locks 0 0.00 Lock Operations LCK.hsl.creat 0 0.00 Created locks LCK.hsl.destroy 0 0.00 Destroyed locks LCK.hsl.locks 0 0.00 Lock Operations LCK.hcb.creat 1 0.00 Created locks LCK.hcb.destroy 0 0.00 Destroyed locks LCK.hcb.locks 788894 0.10 Lock Operations LCK.hcl.creat 0 0.00 Created locks LCK.hcl.destroy 0 0.00 Destroyed locks LCK.hcl.locks 0 0.00 Lock Operations LCK.vcl.creat 1 0.00 Created locks LCK.vcl.destroy 0 0.00 Destroyed locks LCK.vcl.locks 752318 0.10 Lock Operations LCK.sessmem.creat 0 0.00 Created locks LCK.sessmem.destroy 0 0.00 Destroyed locks LCK.sessmem.locks 0 0.00 Lock Operations LCK.sess.creat 1032804 0.13 Created locks LCK.sess.destroy 1032804 0.13 Destroyed locks LCK.sess.locks 0 0.00 Lock Operations LCK.wstat.creat 1 0.00 Created locks LCK.wstat.destroy 0 0.00 Destroyed locks LCK.wstat.locks 2557060 0.33 Lock Operations LCK.herder.creat 0 0.00 Created locks LCK.herder.destroy 0 0.00 Destroyed locks LCK.herder.locks 0 0.00 Lock Operations LCK.wq.creat 3 0.00 Created locks LCK.wq.destroy 0 0.00 Destroyed locks LCK.wq.locks 8605252 1.10 Lock Operations LCK.objhdr.creat 372890 0.05 Created locks LCK.objhdr.destroy 372850 0.05 Destroyed locks LCK.objhdr.locks 11971479 1.53 Lock Operations LCK.exp.creat 1 0.00 Created locks LCK.exp.destroy 0 0.00 Destroyed locks LCK.exp.locks 2415859 0.31 Lock Operations LCK.lru.creat 2 0.00 Created locks LCK.lru.destroy 0 0.00 Destroyed locks LCK.lru.locks 2319636 0.30 Lock Operations LCK.cli.creat 1 0.00 Created locks LCK.cli.destroy 0 0.00 Destroyed locks LCK.cli.locks 2614227 0.33 Lock Operations LCK.ban.creat 1 0.00 Created locks LCK.ban.destroy 0 0.00 Destroyed locks LCK.ban.locks 26502316 3.38 Lock Operations LCK.vbp.creat 1 0.00 Created locks LCK.vbp.destroy 0 0.00 Destroyed locks LCK.vbp.locks 0 0.00 Lock Operations LCK.backend.creat 1 0.00 Created locks LCK.backend.destroy 0 0.00 Destroyed locks LCK.backend.locks 1120863 0.14 Lock Operations LCK.vcapace.creat 1 0.00 Created locks LCK.vcapace.destroy 0 0.00 Destroyed locks LCK.vcapace.locks 0 0.00 Lock Operations LCK.nbusyobj.creat 0 0.00 Created locks LCK.nbusyobj.destroy 0 0.00 Destroyed locks LCK.nbusyobj.locks 0 0.00 Lock Operations LCK.busyobj.creat 373621 0.05 Created locks LCK.busyobj.destroy 373621 0.05 Destroyed locks LCK.busyobj.locks 9753513 1.24 Lock Operations LCK.mempool.creat 6 0.00 Created locks LCK.mempool.destroy 0 0.00 Destroyed locks LCK.mempool.locks 48259482 6.15 Lock Operations LCK.vxid.creat 1 0.00 Created locks LCK.vxid.destroy 0 0.00 Destroyed locks LCK.vxid.locks 778 0.00 Lock Operations LCK.pipestat.creat 1 0.00 Created locks LCK.pipestat.destroy 0 0.00 Destroyed locks LCK.pipestat.locks 0 0.00 Lock Operations From apj at mutt.dk Thu Jul 14 07:44:06 2016 From: apj at mutt.dk (Andreas Plesner) Date: Thu, 14 Jul 2016 09:44:06 +0200 Subject: Varnich Memory in 90% holding continously In-Reply-To: References: <20160713175653.GO23724@nerd.dk> Message-ID: <20160714074406.GP23724@nerd.dk> On Thu, Jul 14, 2016 at 07:29:40AM +0100, sujith pv wrote: > > Please find the attached. Here in above mail 30% memory usage at that time > prior to test. Please answer all the questions: 30% of what? How did you measure this? -- Andreas From sujithnss at gmail.com Thu Jul 14 09:03:14 2016 From: sujithnss at gmail.com (sujith pv) Date: Thu, 14 Jul 2016 14:33:14 +0530 Subject: Varnich Memory in 90% holding continously In-Reply-To: <20160714074406.GP23724@nerd.dk> References: <20160713175653.GO23724@nerd.dk> <20160714074406.GP23724@nerd.dk> Message-ID: Hi I measured it using the linux command to get the highly utilized processes info. So that time it was 30% of memory. On Thu, Jul 14, 2016 at 1:14 PM, Andreas Plesner wrote: > On Thu, Jul 14, 2016 at 07:29:40AM +0100, sujith pv wrote: > > > > Please find the attached. Here in above mail 30% memory usage at that > time > > prior to test. > > Please answer all the questions: 30% of what? How did you measure this? > > -- > Andreas > -------------- next part -------------- An HTML attachment was scrubbed... URL: From apj at mutt.dk Thu Jul 14 10:50:37 2016 From: apj at mutt.dk (Andreas Plesner) Date: Thu, 14 Jul 2016 12:50:37 +0200 Subject: Varnich Memory in 90% holding continously In-Reply-To: References: <20160713175653.GO23724@nerd.dk> <20160714074406.GP23724@nerd.dk> Message-ID: <20160714105037.GQ23724@nerd.dk> On Thu, Jul 14, 2016 at 02:33:14PM +0530, sujith pv wrote: > > I measured it using the linux command to get the highly utilized processes > info. So that time it was 30% of memory. If you're talking about the VIRT column in "top", that is not a measure of real memory usage, and I'm going to assume that there really is no issue. If not, ask again, but be specific about which numbers you have measured, and what the 30% and 90% numbers are derived from. -- Andreas From sujithnss at gmail.com Thu Jul 14 11:12:04 2016 From: sujithnss at gmail.com (sujith pv) Date: Thu, 14 Jul 2016 16:42:04 +0530 Subject: Varnich Memory in 90% holding continously In-Reply-To: <20160714105037.GQ23724@nerd.dk> References: <20160713175653.GO23724@nerd.dk> <20160714074406.GP23724@nerd.dk> <20160714105037.GQ23724@nerd.dk> Message-ID: Hi I have used the below command *ps -eo pmem,pcpu,vsize,pid,cmd | sort -k 1 -nr | head -5* And I have got the below output. Also I have a monitoring tool UIM which also states still memory is holding up at 90% eventhough Varnish server is literally idle 90.9 0.1 17561316 12953 /u01/varnishapp/sbin/varnishd -P /u01/varnishapp/varnish_pid.pid -a :15001 -f /u01/varnishapp/etc/default.vcl -T 127.0.0.1:6082 -t 7200 -p thread_pool_min=50 -p thread_pool_max=1000 -p thread_pool_timeout=120 -u svc_varnishghs -g admin_uktil09_onlineservices_varnishghs -S /u01/varnishapp/etc/secret -s malloc,8G 0.5 0.3 100840 29277 bin/varnishlog 0.5 0.0 114156 12949 /u01/varnishapp/sbin/varnishd -P /u01/varnishapp/varnish_pid.pid -a :15001 -f /u01/varnishapp/etc/default.vcl -T 127.0.0.1:6082 -t 7200 -p thread_pool_min=50 -p thread_pool_max=1000 -p thread_pool_timeout=120 -u svc_varnishghs -g admin_uktil09_onlineservices_varnishghs -S /u01/varnishapp/etc/secret -s malloc,8G 0.2 0.1 282492 26519 splunkd -p 8089 start %MEM %CPU VSZ PID CMD On Thu, Jul 14, 2016 at 4:20 PM, Andreas Plesner wrote: > On Thu, Jul 14, 2016 at 02:33:14PM +0530, sujith pv wrote: > > > > I measured it using the linux command to get the highly utilized > processes > > info. So that time it was 30% of memory. > > If you're talking about the VIRT column in "top", that is not a measure of > real memory usage, and I'm going to assume that there really is no issue. > > If not, ask again, but be specific about which numbers you have measured, > and > what the 30% and 90% numbers are derived from. > > -- > Andreas > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From apj at mutt.dk Thu Jul 14 12:15:16 2016 From: apj at mutt.dk (Andreas Plesner) Date: Thu, 14 Jul 2016 14:15:16 +0200 Subject: Varnich Memory in 90% holding continously In-Reply-To: References: <20160713175653.GO23724@nerd.dk> <20160714074406.GP23724@nerd.dk> <20160714105037.GQ23724@nerd.dk> Message-ID: <20160714121516.GR23724@nerd.dk> On Thu, Jul 14, 2016 at 04:42:04PM +0530, sujith pv wrote: > > And I have got the below output. Also I have a monitoring tool UIM which > also states still memory is holding up at 90% eventhough Varnish server is > literally idle Did you compile varnish yourself? Is it linked to jemalloc? If you compiled it yourself without jemalloc available, you'll be using standard libc malloc, which will probably fragment badly. Use the official packages or at least make sure that jemalloc headers are available when compiling. -- Andreas From adam.schumacher at flightaware.com Thu Jul 14 12:25:30 2016 From: adam.schumacher at flightaware.com (Adam Schumacher) Date: Thu, 14 Jul 2016 12:25:30 +0000 Subject: Varnich Memory in 90% holding continously Message-ID: <2BBE80BC-ABD3-4621-BD6D-CCEF2B911ABD@flightaware.com> >>> And I have got the below output. Also I have a monitoring tool UIM which also states still memory is holding up at 90% eventhough Varnish server is literally idle 90% of what? How much RAM is installed on the host? You have malloc set at 8G. If you only have 10G of ram, this would be expected and you should lower your malloc setting, or adjust your monitoring parameters. Note that there is nothing inherently wrong with using most of your RAM ? that just means you are getting your money?s worth out of it. ::Adam -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4694 bytes Desc: not available URL: From sujithnss at gmail.com Thu Jul 14 12:36:50 2016 From: sujithnss at gmail.com (sujith pv) Date: Thu, 14 Jul 2016 18:06:50 +0530 Subject: Varnich Memory in 90% holding continously In-Reply-To: <2BBE80BC-ABD3-4621-BD6D-CCEF2B911ABD@flightaware.com> References: <2BBE80BC-ABD3-4621-BD6D-CCEF2B911ABD@flightaware.com> Message-ID: Thanks Adam. I assume we have compiled it ourself I need to look into as it was done earlier. Also could you please let me know whats the fastest way to clear the Varnish cache. Also here the surprising issue is that still today being Varnish idle, why it is holding up the memory 90%. Whats the way to release it manually. On Thu, Jul 14, 2016 at 5:55 PM, Adam Schumacher < adam.schumacher at flightaware.com> wrote: > >>> And I have got the below output. Also I have a monitoring tool UIM > which also states still memory is holding up at 90% eventhough Varnish > server is literally idle > > 90% of what? How much RAM is installed on the host? You have malloc set > at 8G. If you only have 10G of ram, this would be expected and you should > lower your malloc setting, or adjust your monitoring parameters. Note that > there is nothing inherently wrong with using most of your RAM ? that just > means you are getting your money?s worth out of it. > > ::Adam > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From yianniska at gmail.com Thu Jul 14 22:19:12 2016 From: yianniska at gmail.com (Yiannis Karayiannidis) Date: Fri, 15 Jul 2016 00:19:12 +0200 Subject: Varnich Memory in 90% holding continously In-Reply-To: References: <2BBE80BC-ABD3-4621-BD6D-CCEF2B911ABD@flightaware.com> Message-ID: Hi all, I've got a "strange problem" with varnish 4.1.3 with error 503 after a backend response with 304 I believe my mistake is that i try to cache a URL for which the backend send 304 response and that is something unallowed. Is that correct? If not was where is the problem? Regards Yiannis p.s I do not have the same problem with varnish 4.0.2 The details from varnishlog * << Request >> 721018 - Begin req 721017 rxreq - Timestamp Start: 1468531337.961047 0.000000 0.000000 - Timestamp Req: 1468531337.961047 0.000000 0.000000 - ReqStart 192.168.50.80 55518 - ReqMethod GET - ReqURL /event/diff?id=707658&rev=925&country=ES&LANG=en&xfarm=2&_=1468531333090 - ReqProtocol HTTP/1.1 - ReqHeader Host: www.xxx.ge - ReqHeader X-Forwarded-For: 192.168.55.145 - ReqHeader X-Forwarded-Proto: https - ReqHeader Connection: close - ReqHeader Accept: */* - ReqHeader X-Requested-With: XMLHttpRequest - ReqHeader User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36 - ReqHeader Referer: https://www.xxx.ge/event/index/707658 - ReqHeader Accept-Encoding: gzip, deflate, sdch, br - ReqHeader Accept-Language: en-US,en;q=0.8,el;q=0.6 - ReqHeader Cookie: __cfduid=d6091c5456123ad4d97ceda2ab91445248766; LANG=en; walkthrough=1; LANG=en; _cb_ls=1; optimizelyEndUserId=oeu1459345286386r0.9749153622397837; optimizelySegments=%7B%225389070408%22%3A%22false%22%2C%225372042113%22%3A%22gc%22%2C%225391 - ReqUnset X-Forwarded-For: 192.168.55.145 - ReqHeader X-Forwarded-For: 192.168.55.145, 192.168.50.80 - VCL_call RECV - ReqUnset Cookie: __cfduid=d6091c5456123ad4d97ceda2ab91445248766; LANG=en; walkthrough=1; LANG=en; _cb_ls=1; optimizelyEndUserId=oeu1459345286386r0.9749153622397837; optimizelySegments=%7B%225389070408%22%3A%22false%22%2C%225372042113%22%3A%22gc%22%2C%225391 - ReqHeader Cookie: __cfduid=d6091c5456123ad4d97ceda2ab91445248766; LANG=en; walkthrough=1; LANG=en; _cb_ls=1; optimizelyEndUserId=oeu1459345286386r0.9749153622397837; optimizelySegments=%7B%225389070408%22%3A%22false%22%2C%225372042113%22%3A%22gc%22%2C%225391 - ReqURL /event/diff?id=707658&rev=925&country=ES&LANG=en&xfarm=2 - ReqUnset Cookie: __cfduid=d6091c5456123ad4d97ceda2ab91445248766; LANG=en; walkthrough=1; LANG=en; _cb_ls=1; optimizelyEndUserId=oeu1459345286386r0.9749153622397837; optimizelySegments=%7B%225389070408%22%3A%22false%22%2C%225372042113%22%3A%22gc%22%2C%225391 - VCL_return hash - ReqUnset Accept-Encoding: gzip, deflate, sdch, br - ReqHeader Accept-Encoding: gzip - VCL_call HASH - ReqHeader X-defHash: /event/diff?id=707658&rev=925&country=ES&LANG=en&xfarm=2 - ReqUnset X-defHash: /event/diff?id=707658&rev=925&country=ES&LANG=en&xfarm=2 - ReqHeader X-defHash: /event/diff?id=707658&rev=925&country=ES&LANG=en&xfarm=2 + www.xxx.ge - VCL_return lookup - VCL_call MISS - VCL_return fetch - Link bereq 721019 fetch - Timestamp Fetch: 1468531337.963143 0.002097 0.002097 - Timestamp Process: 1468531337.963165 0.002118 0.000021 - RespHeader Date: Thu, 14 Jul 2016 21:22:17 GMT - RespHeader Server: Varnish - RespHeader X-Varnish: 721018 - RespProtocol HTTP/1.1 - RespStatus 503 - RespReason Service Unavailable - RespReason Service Unavailable - VCL_call SYNTH - RespHeader Content-Type: text/html; charset=utf-8 - RespHeader Retry-After: 5 - VCL_return deliver - RespHeader Content-Length: 280 - Storage malloc Transient - Debug "RES_MODE 2" - RespHeader Connection: close - Timestamp Resp: 1468531337.963247 0.002200 0.000082 - ReqAcct 1077 0 1077 205 280 485 - End * << BeReq >> 721019 - Begin bereq 721018 fetch - Timestamp Start: 1468531337.961169 0.000000 0.000000 - BereqMethod GET - BereqURL /event/diff?id=707658&rev=925&country=ES&LANG=en&xfarm=2& - BereqProtocol HTTP/1.1 - BereqHeader Host: www.xxx.ge - BereqHeader X-Forwarded-Proto: https - BereqHeader Accept: */* - BereqHeader X-Requested-With: XMLHttpRequest - BereqHeader User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36 - BereqHeader Referer: https://www.xxx.ge/event/index/707658 - BereqHeader Accept-Language: en-US,en;q=0.8,el;q=0.6 - BereqHeader X-Forwarded-For: 192.168.55.145, 172.16.50.80 - BereqHeader Accept-Encoding: gzip - BereqHeader X-defHash: /event/diff?id=707658&rev=925&country=ES&LANG=en&xfarm=2 + www.xxx.ge - BereqHeader X-Varnish: 721019 - VCL_call BACKEND_FETCH - VCL_return fetch - BackendOpen 28 boot.wb2_mob_stx_spo_gr 172.16.50.132 80 172.16.50.80 48508 - BackendStart 192.168.50.132 80 - Timestamp Bereq: 1468531337.961247 0.000078 0.000078 - Timestamp Beresp: 1468531337.963066 0.001897 0.001819 - BerespProtocol HTTP/1.1 - BerespStatus 304 - BerespReason Not Modified - BerespHeader Cache-Control: private - BerespHeader Server: Microsoft-IIS/8.5 - BerespHeader Set-Cookie: LANG=en; expires=Fri, 14-Jul-2017 21:22:17 GMT; path=/; secure; HttpOnly - BerespHeader X-AspNetMvc-Version: 5.2 - BerespHeader X-AspNet-Version: 4.0.30319 - BerespHeader X-Powered-By: ASP.NET - BerespHeader X-Farm: 12 - BerespHeader Date: Thu, 14 Jul 2016 21:22:17 GMT - TTL RFC 120 10 -1 1468531338 1468531338 1468531337 0 0 - Error 304 response but not conditional fetch - BackendClose 28 boot.wb2_mob_stx_spo_gr - BereqAcct 574 0 574 293 0 293 - End -------------- next part -------------- An HTML attachment was scrubbed... URL: From yianniska at gmail.com Thu Jul 14 22:24:04 2016 From: yianniska at gmail.com (Yiannis Karayiannidis) Date: Fri, 15 Jul 2016 00:24:04 +0200 Subject: 503 after a 304 backend response. Message-ID: Hi all, I've got a "problem" with varnish 4.1.3 with error 503 after a backend response with 304 I believe my mistake is that i try to cache a URL for which the backend send 304 response and that is something unallowed. Is that correct? If not was where is the problem? Regards Yiannis p.s I do not have the same problem with varnish 4.0.2 p.s.2 Sorry for sending the previous mail to the wrong thread.... The details from varnishlog * << Request >> 721018 - Begin req 721017 rxreq - Timestamp Start: 1468531337.961047 0.000000 0.000000 - Timestamp Req: 1468531337.961047 0.000000 0.000000 - ReqStart 192.168.50.80 55518 - ReqMethod GET - ReqURL /event/diff?id=707658&rev=925&country=ES&LANG=en&xfarm=2&_=1468531333090 - ReqProtocol HTTP/1.1 - ReqHeader Host: www.xxx.ge - ReqHeader X-Forwarded-For: 192.168.55.145 - ReqHeader X-Forwarded-Proto: https - ReqHeader Connection: close - ReqHeader Accept: */* - ReqHeader X-Requested-With: XMLHttpRequest - ReqHeader User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36 - ReqHeader Referer: https://www.xxx.ge/event/index/707658 - ReqHeader Accept-Encoding: gzip, deflate, sdch, br - ReqHeader Accept-Language: en-US,en;q=0.8,el;q=0.6 - ReqHeader Cookie: __cfduid=d6091c5456123ad4d97ceda2ab91445248766; LANG=en; walkthrough=1; LANG=en; _cb_ls=1; optimizelyEndUserId=oeu1459345286386r0.9749153622397837; optimizelySegments=%7B%225389070408%22%3A%22false%22%2C%225372042113%22%3A%22gc%22%2C%225391 - ReqUnset X-Forwarded-For: 192.168.55.145 - ReqHeader X-Forwarded-For: 192.168.55.145, 192.168.50.80 - VCL_call RECV - ReqUnset Cookie: __cfduid=d6091c5456123ad4d97ceda2ab91445248766; LANG=en; walkthrough=1; LANG=en; _cb_ls=1; optimizelyEndUserId=oeu1459345286386r0.9749153622397837; optimizelySegments=%7B%225389070408%22%3A%22false%22%2C%225372042113%22%3A%22gc%22%2C%225391 - ReqHeader Cookie: __cfduid=d6091c5456123ad4d97ceda2ab91445248766; LANG=en; walkthrough=1; LANG=en; _cb_ls=1; optimizelyEndUserId=oeu1459345286386r0.9749153622397837; optimizelySegments=%7B%225389070408%22%3A%22false%22%2C%225372042113%22%3A%22gc%22%2C%225391 - ReqURL /event/diff?id=707658&rev=925&country=ES&LANG=en&xfarm=2 - ReqUnset Cookie: __cfduid=d6091c5456123ad4d97ceda2ab91445248766; LANG=en; walkthrough=1; LANG=en; _cb_ls=1; optimizelyEndUserId=oeu1459345286386r0.9749153622397837; optimizelySegments=%7B%225389070408%22%3A%22false%22%2C%225372042113%22%3A%22gc%22%2C%225391 - VCL_return hash - ReqUnset Accept-Encoding: gzip, deflate, sdch, br - ReqHeader Accept-Encoding: gzip - VCL_call HASH - ReqHeader X-defHash: /event/diff?id=707658&rev=925&country=ES&LANG=en&xfarm=2 - ReqUnset X-defHash: /event/diff?id=707658&rev=925&country=ES&LANG=en&xfarm=2 - ReqHeader X-defHash: /event/diff?id=707658&rev=925&country=ES&LANG=en&xfarm=2 + www.xxx.ge - VCL_return lookup - VCL_call MISS - VCL_return fetch - Link bereq 721019 fetch - Timestamp Fetch: 1468531337.963143 0.002097 0.002097 - Timestamp Process: 1468531337.963165 0.002118 0.000021 - RespHeader Date: Thu, 14 Jul 2016 21:22:17 GMT - RespHeader Server: Varnish - RespHeader X-Varnish: 721018 - RespProtocol HTTP/1.1 - RespStatus 503 - RespReason Service Unavailable - RespReason Service Unavailable - VCL_call SYNTH - RespHeader Content-Type: text/html; charset=utf-8 - RespHeader Retry-After: 5 - VCL_return deliver - RespHeader Content-Length: 280 - Storage malloc Transient - Debug "RES_MODE 2" - RespHeader Connection: close - Timestamp Resp: 1468531337.963247 0.002200 0.000082 - ReqAcct 1077 0 1077 205 280 485 - End * << BeReq >> 721019 - Begin bereq 721018 fetch - Timestamp Start: 1468531337.961169 0.000000 0.000000 - BereqMethod GET - BereqURL /event/diff?id=707658&rev=925&country=ES&LANG=en&xfarm=2& - BereqProtocol HTTP/1.1 - BereqHeader Host: www.xxx.ge - BereqHeader X-Forwarded-Proto: https - BereqHeader Accept: */* - BereqHeader X-Requested-With: XMLHttpRequest - BereqHeader User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36 - BereqHeader Referer: https://www.xxx.ge/event/index/707658 - BereqHeader Accept-Language: en-US,en;q=0.8,el;q=0.6 - BereqHeader X-Forwarded-For: 192.168.55.145, 172.16.50.80 - BereqHeader Accept-Encoding: gzip - BereqHeader X-defHash: /event/diff?id=707658&rev=925&country=ES&LANG=en&xfarm=2 + www.xxx.ge - BereqHeader X-Varnish: 721019 - VCL_call BACKEND_FETCH - VCL_return fetch - BackendOpen 28 boot.wb2_mob_stx_spo_gr 172.16.50.132 80 172.16.50.80 48508 - BackendStart 192.168.50.132 80 - Timestamp Bereq: 1468531337.961247 0.000078 0.000078 - Timestamp Beresp: 1468531337.963066 0.001897 0.001819 - BerespProtocol HTTP/1.1 - BerespStatus 304 - BerespReason Not Modified - BerespHeader Cache-Control: private - BerespHeader Server: Microsoft-IIS/8.5 - BerespHeader Set-Cookie: LANG=en; expires=Fri, 14-Jul-2017 21:22:17 GMT; path=/; secure; HttpOnly - BerespHeader X-AspNetMvc-Version: 5.2 - BerespHeader X-AspNet-Version: 4.0.30319 - BerespHeader X-Powered-By: ASP.NET - BerespHeader X-Farm: 12 - BerespHeader Date: Thu, 14 Jul 2016 21:22:17 GMT - TTL RFC 120 10 -1 1468531338 1468531338 1468531337 0 0 - Error 304 response but not conditional fetch - BackendClose 28 boot.wb2_mob_stx_spo_gr - BereqAcct 574 0 574 293 0 293 - End -------------- next part -------------- An HTML attachment was scrubbed... URL: From apj at mutt.dk Fri Jul 15 07:42:07 2016 From: apj at mutt.dk (Andreas Plesner) Date: Fri, 15 Jul 2016 09:42:07 +0200 Subject: 503 after a 304 backend response. In-Reply-To: References: Message-ID: <20160715074207.GS23724@nerd.dk> On Fri, Jul 15, 2016 at 12:24:04AM +0200, Yiannis Karayiannidis wrote: > I've got a "problem" with varnish 4.1.3 with error 503 after a backend > response with 304 > > I believe my mistake is that i try to cache a URL for which the backend > send 304 response and that is something unallowed. > > Is that correct? > If not was where is the problem? Varnish did not do a conditional request, so a HTTP 304 is not a valid response. See RFC 7232. -- Andreas From pablolibo at gmail.com Sat Jul 16 13:48:00 2016 From: pablolibo at gmail.com (Pablo J. Villarruel) Date: Sat, 16 Jul 2016 10:48:00 -0300 Subject: Mode: pass all requests In-Reply-To: References: Message-ID: Hi guys, is there a way put the varnish-cache on mode pass all requests with varnishadm or another admin command? I need to skip the hits from vcl config file for debug Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From maillist-varnish at iamafreeman.com Sat Jul 16 20:24:21 2016 From: maillist-varnish at iamafreeman.com (varnish list) Date: Sat, 16 Jul 2016 21:24:21 +0100 Subject: varnish using as a proxy interfering with get requests Message-ID: Hello I'm attempting to use varnish in a somewhat unusual manner, but with fairly good reason I think. browser -> varnish server :80 -> apache proxy (localhost:3128) -> any site the idea being to use varnish to fire off statd timing info to get metrics on operations against an number of systems, some we run, some out sourced, in a standard manner issue is varnish 4.1.3, but I've tried other 4's, interferes with the users request I'm comparing the request using tshark and varnishlog proxy request can look like GET http://website.dom/url I see that hit varnish from the browser what apache gets is GET /url I can't figure out what is stripping the http://website.dom. My default.vcl is practically empty vcl 4.0; import std; import statsd; import timers; backend default { .host = "127.0.0.1"; .port = "3128"; } sub vcl_recv { return (pass); } sub vcl_backend_response { } sub vcl_deliver { if (req.url ~ "/REST/UI/Content/CheckOut") { statsd.incr("website.checkout"); } } sub vcl_init { statsd.server( "statdcollector.dom", "8125" ); statsd.prefix( "proxy.test." ); } varnish 3.0.5 didn't do this, but its statd and timing support isn't great and old hat, so I'm none too interesting in sticking with 3 Any clues where I find and disable this interference? Thanks Neil -------------- next part -------------- An HTML attachment was scrubbed... URL: From jpotter-varnish at codepuppy.com Sun Jul 17 14:24:28 2016 From: jpotter-varnish at codepuppy.com (Jeff Potter) Date: Sun, 17 Jul 2016 10:24:28 -0400 Subject: varnish using as a proxy interfering with get requests In-Reply-To: References: Message-ID: Is apache getting a Host header as well? I would expect a correct request to look like: GET /url Host: website.com -J > On Jul 16, 2016, at 4:24 PM, varnish list wrote: > > Hello > > I'm attempting to use varnish in a somewhat unusual manner, but with fairly good reason I think. > > browser -> varnish server :80 -> apache proxy (localhost:3128) -> any site > > the idea being to use varnish to fire off statd timing info to get metrics on operations against an number of systems, some we run, some out sourced, in a standard manner > > issue is varnish 4.1.3, but I've tried other 4's, interferes with the users request > > I'm comparing the request using tshark and varnishlog > > proxy request can look like > > GET http://website.dom/url > I see that hit varnish from the browser > > what apache gets is > GET /url > > I can't figure out what is stripping the http://website.dom . My default.vcl is practically empty > > vcl 4.0; > import std; > import statsd; > import timers; > > backend default { > .host = "127.0.0.1"; > .port = "3128"; > } > > sub vcl_recv { > return (pass); > } > > sub vcl_backend_response { > } > > sub vcl_deliver { > if (req.url ~ "/REST/UI/Content/CheckOut") { > statsd.incr("website.checkout"); > } > } > > sub vcl_init { > statsd.server( "statdcollector.dom", "8125" ); > statsd.prefix( "proxy.test." ); > } > > varnish 3.0.5 didn't do this, but its statd and timing support isn't great and old hat, so I'm none too interesting in sticking with 3 > > Any clues where I find and disable this interference? > > Thanks > > Neil > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Sun Jul 17 15:09:07 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Sun, 17 Jul 2016 17:09:07 +0200 Subject: Mode: pass all requests In-Reply-To: References: Message-ID: Hello Pablo, I would use vmod-var, and decide if I must pass or not, depending on the value of one of its variables. And I'd have some logic in vcl, protected by acl to allow me to change that value. Does it make sense? -------------- next part -------------- An HTML attachment was scrubbed... URL: From lagged at gmail.com Mon Jul 18 09:58:22 2016 From: lagged at gmail.com (Andrei) Date: Mon, 18 Jul 2016 12:58:22 +0300 Subject: varnish using as a proxy interfering with get requests In-Reply-To: References: Message-ID: Also, you'll want to enable ProxyPreserveHost in Apache. http://httpd.apache.org/docs/2.2/mod/mod_proxy.html#proxypreservehost http://httpd.apache.org/docs/2.4/mod/mod_proxy.html#proxypreservehost On Jul 17, 2016 17:51, "Jeff Potter" wrote: > > Is apache getting a Host header as well? I would expect a correct request > to look like: > > GET /url > Host: website.com > > -J > > On Jul 16, 2016, at 4:24 PM, varnish list < > maillist-varnish at iamafreeman.com> wrote: > > Hello > > I'm attempting to use varnish in a somewhat unusual manner, but with > fairly good reason I think. > > browser -> varnish server :80 -> apache proxy (localhost:3128) -> any site > > the idea being to use varnish to fire off statd timing info to get metrics > on operations against an number of systems, some we run, some out sourced, > in a standard manner > > issue is varnish 4.1.3, but I've tried other 4's, interferes with the > users request > > I'm comparing the request using tshark and varnishlog > > proxy request can look like > > GET http://website.dom/url > I see that hit varnish from the browser > > what apache gets is > GET /url > > I can't figure out what is stripping the http://website.dom. My > default.vcl is practically empty > > vcl 4.0; > import std; > import statsd; > import timers; > > backend default { > .host = "127.0.0.1"; > .port = "3128"; > } > > sub vcl_recv { > return (pass); > } > > sub vcl_backend_response { > } > > sub vcl_deliver { > if (req.url ~ "/REST/UI/Content/CheckOut") { > statsd.incr("website.checkout"); > } > } > > sub vcl_init { > statsd.server( "statdcollector.dom", "8125" ); > statsd.prefix( "proxy.test." ); > } > > varnish 3.0.5 didn't do this, but its statd and timing support isn't great > and old hat, so I'm none too interesting in sticking with 3 > > Any clues where I find and disable this interference? > > Thanks > > Neil > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From me at kelvinloke.com Tue Jul 19 16:00:16 2016 From: me at kelvinloke.com (Kelvin Loke) Date: Tue, 19 Jul 2016 16:00:16 +0000 Subject: Passing Backend App Errors to End Users Message-ID: I have a tough time digging this and none of my tries would work. The error messages from my backend contains useful details such as: HTTP 400 Bad Request Body: Missing fromDate. The fromDate must be a date in ISO 8601 format. HTTP 400 Bad Request Body: Missing required parameter: queueList The messages is very useful for the end users to know what was wrong with their request. I was trying hard to figure how Varnish could pass the backend app error to end users, not the Varnish's wrapped error message (Guru Mediation...) Does anyone have any experience in doing this? Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: From apj at mutt.dk Wed Jul 20 07:32:18 2016 From: apj at mutt.dk (Andreas Plesner) Date: Wed, 20 Jul 2016 09:32:18 +0200 Subject: Passing Backend App Errors to End Users In-Reply-To: References: Message-ID: <20160720073218.GT23724@nerd.dk> On Tue, Jul 19, 2016 at 04:00:16PM +0000, Kelvin Loke wrote: > I have a tough time digging this and none of my tries would work. > > The error messages from my backend contains useful details such as: > > HTTP 400 Bad Request > Body: Missing required parameter: queueList > > The messages is very useful for the end users to know what was wrong with > their request. > > I was trying hard to figure how Varnish could pass the backend app error to > end users, not the Varnish's wrapped error message (Guru Mediation...) Varnish will do that by default. If you get a 503 from Varnish, it is because your VCL returns that or because your backend violates the HTTP protocol. As always: Show us varnishlog and we will be able to tell. -- Andreas From maillist-varnish at iamafreeman.com Wed Jul 20 19:24:59 2016 From: maillist-varnish at iamafreeman.com (varnish list) Date: Wed, 20 Jul 2016 20:24:59 +0100 Subject: varnish using as a proxy interfering with get requests In-Reply-To: References: Message-ID: Thanks for the tips. I'll get to apache when I've got varnish passing the request on correctly I'll bisect the releases to find the change that stopped it working. it might be a 3 to 4 thing though. Neil On 18 Jul 2016 10:58, "Andrei" wrote: Also, you'll want to enable ProxyPreserveHost in Apache. http://httpd.apache.org/docs/2.2/mod/mod_proxy.html#proxypreservehost http://httpd.apache.org/docs/2.4/mod/mod_proxy.html#proxypreservehost On Jul 17, 2016 17:51, "Jeff Potter" wrote: > > Is apache getting a Host header as well? I would expect a correct request > to look like: > > GET /url > Host: website.com > > -J > > On Jul 16, 2016, at 4:24 PM, varnish list < > maillist-varnish at iamafreeman.com> wrote: > > Hello > > I'm attempting to use varnish in a somewhat unusual manner, but with > fairly good reason I think. > > browser -> varnish server :80 -> apache proxy (localhost:3128) -> any site > > the idea being to use varnish to fire off statd timing info to get metrics > on operations against an number of systems, some we run, some out sourced, > in a standard manner > > issue is varnish 4.1.3, but I've tried other 4's, interferes with the > users request > > I'm comparing the request using tshark and varnishlog > > proxy request can look like > > GET http://website.dom/url > I see that hit varnish from the browser > > what apache gets is > GET /url > > I can't figure out what is stripping the http://website.dom. My > default.vcl is practically empty > > vcl 4.0; > import std; > import statsd; > import timers; > > backend default { > .host = "127.0.0.1"; > .port = "3128"; > } > > sub vcl_recv { > return (pass); > } > > sub vcl_backend_response { > } > > sub vcl_deliver { > if (req.url ~ "/REST/UI/Content/CheckOut") { > statsd.incr("website.checkout"); > } > } > > sub vcl_init { > statsd.server( "statdcollector.dom", "8125" ); > statsd.prefix( "proxy.test." ); > } > > varnish 3.0.5 didn't do this, but its statd and timing support isn't great > and old hat, so I'm none too interesting in sticking with 3 > > Any clues where I find and disable this interference? > > Thanks > > Neil > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From me at kelvinloke.com Thu Jul 21 00:55:47 2016 From: me at kelvinloke.com (Kelvin Loke) Date: Thu, 21 Jul 2016 00:55:47 +0000 Subject: Passing Backend App Errors to End Users In-Reply-To: <20160720073218.GT23724@nerd.dk> References: <20160720073218.GT23724@nerd.dk> Message-ID: > > Varnish will do that by default. If you get a 503 from Varnish, it is > because your VCL returns that or because your backend violates the HTTP > protocol. > > As always: Show us varnishlog and we will be able to tell. > > -- > Andreas > Thanks for responding, let me give you a live example so you can understand clearer : ) 1. User -> Backend (b.kelvinloke.com). Backend returns HTTP 400 with body "Missing fromDate query string.", which is good. 2. User -> Varnish (v.kelvinloke.com) -> Backend. Varnish removes Backend's error body, replace with Varnish's generic error body. My goal is to make Varnish to show Backend's error body, not Varnish's generic error body, which I have a difficult time to figure it out. I actually tried below but Varnish doesn't allow me to reload the config, with error "Running VCC-compiler failed, exited with 2" on resp.body variable. sub vcl_synth { if (resp.status == 400) { set resp.body = {" "} + resp.body + {" "}; return (deliver); } } Here you go for the varnishlog if it helps. - Begin bereq 34174473 pass - Timestamp Start: 1469062408.399294 0.000000 0.000000 - BereqMethod GET - BereqURL / - BereqProtocol HTTP/1.1 - BereqHeader host: v.kelvinloke.com - BereqHeader Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 - BereqHeader Accept-Language: en-US,en;q=0.8 - BereqHeader Cache-Control: no-cache - BereqHeader Pragma: no-cache - BereqHeader Upgrade-Insecure-Requests: 1 - BereqHeader User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36 - BereqHeader X-Forwarded-Port: 80 - BereqHeader X-Forwarded-Proto: http - BereqHeader X-Forwarded-For: 190.112.230.244, 172.31.1.229 - BereqHeader True-Client-IP: 190.112.230.244 - BereqHeader Accept-Encoding: gzip - BereqHeader X-Varnish: 34174474 - VCL_call BACKEND_FETCH - VCL_return fetch - BackendOpen 40 928d2c0c-8e82-4cdc-a194-b6ad0f14e051.kelvin 54.65.82.174 80 172.31.8.52 40796 - BackendStart 54.65.82.174 80 - Timestamp Bereq: 1469062408.401388 0.002093 0.002093 - Timestamp Beresp: 1469062408.403494 0.004200 0.002107 - BerespProtocol HTTP/1.1 - BerespStatus 400 - BerespReason Bad Request - BerespHeader Server: nginx/1.6.2 - BerespHeader Date: Thu, 21 Jul 2016 00:43:01 GMT - BerespHeader Content-Type: text/html - BerespHeader Content-Length: 31 - BerespHeader Connection: close - BerespHeader ETag: "579014ce-1f" - TTL RFC -1 10 -1 1469062408 1469062408 1469061781 0 0 - VCL_call BACKEND_RESPONSE - BerespUnset ETag: "579014ce-1f" - TTL VCL 120 10 0 1469062408 - VCL_return deliver - Storage malloc Transient - ObjProtocol HTTP/1.1 - ObjStatus 400 - ObjReason Bad Request - ObjHeader Server: nginx/1.6.2 - ObjHeader Date: Thu, 21 Jul 2016 00:43:01 GMT - ObjHeader Content-Type: text/html - ObjHeader Content-Length: 31 - Fetch_Body 3 length stream - BackendClose 40 928d2c0c-8e82-4cdc-a194-b6ad0f14e051.kelvin - Timestamp BerespBody: 1469062408.403543 0.004249 0.000049 - Length 31 - BereqAcct 528 0 528 171 31 202 - End * << Request >> 34174473 - Begin req 34174464 rxreq - Timestamp Start: 1469062408.399251 0.000000 0.000000 - Timestamp Req: 1469062408.399251 0.000000 0.000000 - ReqStart 172.31.1.229 32033 - ReqMethod GET - ReqURL / - ReqProtocol HTTP/1.1 - ReqHeader host: v.kelvinloke.com - ReqHeader Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 - ReqHeader Accept-Encoding: gzip, deflate, sdch - ReqHeader Accept-Language: en-US,en;q=0.8 - ReqHeader Cache-Control: no-cache - ReqHeader Pragma: no-cache - ReqHeader Upgrade-Insecure-Requests: 1 - ReqHeader User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36 - ReqHeader X-Forwarded-For: 190.112.230.244 - ReqHeader X-Forwarded-Port: 80 - ReqHeader X-Forwarded-Proto: http - ReqHeader Connection: keep-alive - ReqUnset X-Forwarded-For: 190.112.230.244 - ReqHeader X-Forwarded-For: 190.112.230.244, 172.31.1.229 - VCL_call RECV - ReqHeader True-Client-IP: 190.112.230.244 - ReqUnset Accept-Encoding: gzip, deflate, sdch - ReqHeader Accept-Encoding: gzip - VCL_return pass - VCL_call HASH - VCL_return lookup - VCL_call PASS - VCL_return fetch - Link bereq 34174474 pass - Timestamp Fetch: 1469062408.403541 0.004290 0.004290 - RespProtocol HTTP/1.1 - RespStatus 400 - RespReason Bad Request - RespHeader Server: nginx/1.6.2 - RespHeader Date: Thu, 21 Jul 2016 00:43:01 GMT - RespHeader Content-Type: text/html - RespHeader Content-Length: 31 - RespHeader X-Varnish: 34174473 - RespHeader Age: 0 - RespHeader Via: 1.1 varnish-v4 - VCL_call DELIVER - RespUnset Via: 1.1 varnish-v4 - RespUnset Server: nginx/1.6.2 - RespUnset X-Varnish: 34174473 - VCL_return synth - Timestamp Process: 1469062408.403549 0.004297 0.000008 - Timestamp Process: 1469062408.403551 0.004300 0.000002 - RespHeader Date: Thu, 21 Jul 2016 00:53:28 GMT - RespHeader Server: Varnish - RespHeader X-Varnish: 34174473 - RespProtocol HTTP/1.1 - RespStatus 400 - RespReason Bad Request - RespReason Bad Request - VCL_call SYNTH - RespHeader Content-Type: text/html; charset=utf-8 - RespHeader Retry-After: 5 - VCL_return deliver - RespHeader Content-Length: 258 - Storage malloc Transient - Debug "RES_MODE 2" - RespHeader Connection: keep-alive - Timestamp Resp: 1469062408.403587 0.004336 0.000036 - ReqAcct 499 0 499 204 258 462 - End -------------- next part -------------- An HTML attachment was scrubbed... URL: From apj at mutt.dk Thu Jul 21 08:03:56 2016 From: apj at mutt.dk (Andreas Plesner) Date: Thu, 21 Jul 2016 10:03:56 +0200 Subject: Passing Backend App Errors to End Users In-Reply-To: References: <20160720073218.GT23724@nerd.dk> Message-ID: <20160721080356.GU23724@nerd.dk> On Thu, Jul 21, 2016 at 12:55:47AM +0000, Kelvin Loke wrote: > > My goal is to make Varnish to show Backend's error body, not Varnish's > generic error body, which I have a difficult time to figure it out. I Again: Varnish will do that by default. You don't have to do anything. You return synth in vcl_deliver. Don't do that if you don't want a synthetic response. > - VCL_call DELIVER > - RespUnset Via: 1.1 varnish-v4 > - RespUnset Server: nginx/1.6.2 > - RespUnset X-Varnish: 34174473 > - VCL_return synth -- Andreas From kelvin1111111 at gmail.com Thu Jul 21 16:32:28 2016 From: kelvin1111111 at gmail.com (Kelvin Loke) Date: Thu, 21 Jul 2016 16:32:28 +0000 Subject: Passing Backend App Errors to End Users In-Reply-To: <20160721080356.GU23724@nerd.dk> References: <20160720073218.GT23724@nerd.dk> <20160721080356.GU23724@nerd.dk> Message-ID: > > Again: Varnish will do that by default. You don't have to do anything. > > You return synth in vcl_deliver. Don't do that if you don't want a > synthetic > response. > > > - VCL_call DELIVER > > - RespUnset Via: 1.1 varnish-v4 > > - RespUnset Server: nginx/1.6.2 > > - RespUnset X-Varnish: 34174473 > > - VCL_return synth > You are totally right, I just realized that I have some "return synth()" logic in vcl_deliver, my bad. Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: From mic at strg.at Mon Jul 25 09:17:42 2016 From: mic at strg.at (Michael Dosser) Date: Mon, 25 Jul 2016 11:17:42 +0200 Subject: Varnish threads growing until no response Message-ID: Hi all, we are encountering a strange problem with one of our setups. We use Varnish since 2.1.5 (now 4.1.3) - the first time we use ESI heavily. Problem description: Randomly (sometimes after 20 days, but most of the time every 1-2 days) threads are growing from the default 400 until Varnish cannot create more (2000 is max configured) and the site is not responding. The site has an average of about 23 reqs/s and normal daily peak of 70 reqs/s. What we also see is an increased counter on MAIN.busy_sleep - I don?t know if this is relevant. I?m not sure if we are hitting a bug or if we have a vcl misconfiguration on our side. Here is some information about our setup: 1.) Startup varnishd -f /etc/varnish/fuf.vcl \ -h critbit \ -a 127.0.0.1:6081 \ -T 127.0.0.1:6082 \ -t 120 \ -S /etc/varnish/secret \ -s malloc,12G \ -p thread_pool_min=100 \ -p thread_pool_max=500 \ -p thread_pools=4 \ -p thread_pool_add_delay=2 \ -p vcc_allow_inline_c=on \ -p feature=+esi_disable_xml_check \ -p feature=+esi_ignore_other_elements \ -p feature=+esi_ignore_https" 2.) Varnish configuration: - fuf.vcl (main configuration, includes other files) - http://pastebin.com/bVMH6E1t - fuf-acl.vcl - http://pastebin.com/Y2RdLchK - fuf-error.vcl - http://pastebin.com/ypr1SBGX - fuf-extended_cache_control.vcl - http://pastebin.com/grB9qaPB - fuf-hash.vcl - http://pastebin.com/ZXVQJaz3 - fuf-local.vcl - http://pastebin.com/uhBucRti 3.) sysctl.conf net.ipv6.conf.all.disable_ipv6=1 net.ipv6.conf.default.disable_ipv6=1 net.ipv6.conf.lo.disable_ipv6=1 net.ipv6.conf.eth0.disable_ipv6=1 net.ipv4.conf.default.rp_filter=1 net.ipv4.conf.all.rp_filter=1 net.ipv4.conf.all.accept_redirects = 0 net.ipv4.conf.all.send_redirects = 0 net.ipv4.conf.all.accept_source_route = 0 vm.swappiness=10 net.ipv4.tcp_fin_timeout=15 net.ipv4.tcp_tw_recycle=0 net.ipv4.tcp_tw_reuse=0 net.ipv4.tcp_syncookies=0 net.ipv4.tcp_max_syn_backlog=60000 net.core.somaxconn=60000 net.ipv4.tcp_max_orphans=262144 net.ipv4.tcp_fin_timeout=15 4.) Various debugging files before we restarted Varnish (before hitting 2000 threads) - netstat -an (public IPs deleted) - http://pastebin.com/HunafJpG - varnishstat -1 output - http://pastebin.com/FDVDvssp - strace on Varnish child process - available but quite big (68 MB). I can provide it if necessary! There are obviously a lot of clone() syscalls ? 5.) Setup: Single physical server, 32 GB RAM. No swapping, no I/O wait. /var/lib/varnish mounted on tmpfs Pound on Port 80,443 (HTTPS termination) -> Varnish on 127.0.0.1:6081 -> Nginx on 127.0.0.1:8080 -> UWSGI -> Score application 6.) Used software: - OS: Debian 8.5, firewall enabled, ports 80 and 443 open to the public on INPUT chain, OUTPUT rule ACCEPT - Pound - Varnish 4.1.3 from Varnish repository - Nginx (Gzip compression turned off, all timeout settings are default) - UWSGI - Score (application) - Postgresql - Elasticsearch If you need more information please tell me what you need! Thanks a lot for your help - we are desperate in finding the misconfiguration ? Michael Dosser -- strg.at gmbh michael.dosser at strg.at gumpendorferstrasse 132, top 9, 1060 wien tel +43 (1) 526 56 29 mobile +43 699 1 7777 164 From Drew.AJ at principal.com Mon Jul 25 17:43:00 2016 From: Drew.AJ at principal.com (Drew, AJ) Date: Mon, 25 Jul 2016 17:43:00 +0000 Subject: Passing Cookies to Drupal Backend Message-ID: <626016bee4594904af755291c0ea645c@PFGDSMMSG001.principalusa.corp.principal.com> Hello, I have a question I need confirmation on from the group. I have been asked if it is possible to somehow retain cookies so that they are available to the Drupal PHP code, but not have them used for caching within Varnish. I could not think of a way to perform this, but thought I would ask those with more experience. Does anyone know if this is possible? The requirements would be to do this for all unauthenticated and authenticated traffic for a site. A couple of cookies will be used in caching, but most will just need to be passed through and ignored for caching, but all added back on for the return. Thanks! A J -----Message Disclaimer----- This e-mail message is intended only for the use of the individual or entity to which it is addressed, and may contain information that is privileged, confidential and exempt from disclosure under applicable law. If you are not the intended recipient, any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by reply email to Connect at principal.com and delete or destroy all copies of the original message and attachments thereto. Email sent to or from the Principal Financial Group or any of its member companies may be retained as required by law or regulation. Nothing in this message is intended to constitute an Electronic signature for purposes of the Uniform Electronic Transactions Act (UETA) or the Electronic Signatures in Global and National Commerce Act ("E-Sign") unless a specific statement to the contrary is included in this message. If you no longer wish to receive any further solicitation from the Principal Financial Group you may unsubscribe at https://www.principal.com/do-not-contact-form any time. If you are a Canadian resident and no longer wish to receive commercial electronic messages you may unsubscribe at https://www.principal.com/do-not-email-request-canadian-residents any time. -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Wed Jul 27 14:01:51 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Wed, 27 Jul 2016 16:01:51 +0200 Subject: Varnish threads growing until no response In-Reply-To: References: Message-ID: Sorry, TL;DR, but you have at least one problem: thread_pool_add_delay Value is: 0.000 [seconds] (default) Minimum is: 0.000 Wait at least this long after creating a thread. Some (buggy) systems may need a short (sub-second) delay between creating threads. Set this to a few milliseconds if you see the 'threads_failed' counter grow too much. Setting this too high results in insuffient worker threads. NB: We do not know yet if it is a good idea to change this parameter, or if the default value is even sensible. Caution is advised, and feedback is most welcome. This parameter is now in seconds, that has been know to bite some users. (but thanks to you, I'm only now noticing a typo in the description, thanks!) Can you set thread_pool_add_delay to 0 and let us know how it goes? -- Guillaume Quintard -------------- next part -------------- An HTML attachment was scrubbed... URL: From mic at strg.at Wed Jul 27 14:34:12 2016 From: mic at strg.at (Michael Dosser) Date: Wed, 27 Jul 2016 16:34:12 +0200 Subject: Varnish threads growing until no response In-Reply-To: References: Message-ID: <90B82C21-4A94-4AAE-95BB-CF9623DD78F8@strg.at> Hi, > Am 27.07.2016 um 16:01 schrieb Guillaume Quintard : > > Sorry, TL;DR, but you have at least one problem: > > thread_pool_add_delay [...] > This parameter is now in seconds, that has been know to bite some users. > > (but thanks to you, I'm only now noticing a typo in the description, thanks!) > > Can you set thread_pool_add_delay to 0 and let us know how it goes? I?ve set it to 0 and will let you know if this solves the problem, thx! Michael Dosser -- strg.at gmbh michael.dosser at strg.at gumpendorferstrasse 132, top 9, 1060 wien tel +43 (1) 526 56 29 mobile +43 699 1 7777 164 From guillaume at varnish-software.com Wed Jul 27 15:02:39 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Wed, 27 Jul 2016 17:02:39 +0200 Subject: Passing Cookies to Drupal Backend In-Reply-To: <626016bee4594904af755291c0ea645c@PFGDSMMSG001.principalusa.corp.principal.com> References: <626016bee4594904af755291c0ea645c@PFGDSMMSG001.principalusa.corp.principal.com> Message-ID: Hi, By default, Varnish will not cache if the request contains a cookie headers, or if the response contains a set-cookie header, this is totally configurable in vcl. Have a look at the builtin.vcl (mine can be found, in commented form at /usr/share/doc/varnish/builtin.vcl) : vcl_recv contains: if (req.http.Authorization || req.http.Cookie) { /* Not cacheable by default */ return (pass); } and vcl_backend_response: if (beresp.ttl <= 0s || beresp.http.Set-Cookie || beresp.http.Surrogate-control ~ "no-store" || (!beresp.http.Surrogate-Control && beresp.http.Cache-Control ~ "no-cache|no-store|private") || beresp.http.Vary == "*") { /* * Mark as "Hit-For-Pass" for the next 2 minutes */ set beresp.ttl = 120s; set beresp.uncacheable = true; } If you don't wish to go through this, you'll have to do a return in your own vcl, otherwise the builtin.vcl portion of the code gets executed. To cache part of cookies, check out the vmod-cookies, and in vcl_hash(), choose the interesting parts for caching. Does that help? P.S. : you want to reconsider adding a confidentiality warning that's longer than your actual message when you post to a public mailing list :-) -- Guillaume Quintard -------------- next part -------------- An HTML attachment was scrubbed... URL: From r at roze.lv Wed Jul 27 15:17:34 2016 From: r at roze.lv (Reinis Rozitis) Date: Wed, 27 Jul 2016 18:17:34 +0300 Subject: Passing Cookies to Drupal Backend In-Reply-To: <626016bee4594904af755291c0ea645c@PFGDSMMSG001.principalusa.corp.principal.com> References: <626016bee4594904af755291c0ea645c@PFGDSMMSG001.principalusa.corp.principal.com> Message-ID: <7772034CFB874322BFB6103194BEFF0E@MezhRoze> > I have been asked if it is possible to somehow retain cookies so that they > are available to the Drupal PHP code, but not have them used for caching > within Varnish. > I could not think of a way to perform this, but thought I would ask those > with more experience. > Does anyone know if this is possible? In general yes it should be possible. You can look for example at this article https://www.fastly.com/blog/vcl-cookie-monster (it's for the 3.x varnish but you should get the idea). For recent 4.x varnish the filtering should be even more simple with the Cookie vmod https://github.com/varnish/varnish-modules/blob/master/docs/vmod_cookie.rst and maybe the variable mod to store the initial cookies (rather than the old way in extra headers). rr From ayberk.kimsesiz at gmail.com Wed Jul 27 17:35:44 2016 From: ayberk.kimsesiz at gmail.com (Ayberk Kimsesiz) Date: Wed, 27 Jul 2016 20:35:44 +0300 Subject: Varnish CPU Usage Message-ID: Hello all, I have a problem with Varnish. Even though when i leave Default.vcl file with its DEFAULT settings or as Wordpress optimized, Apache's CPU usage still increases to 70 - 90%.. But if i disable Varnish, CPU usage returns to normal. I also cannot access to the website when i set the port setting as 127.0.0.1. Therefore this makes me have to use Server IP instead. What is your suggestions on these matters? Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From nick.tailor at gmail.com Wed Jul 27 17:58:31 2016 From: nick.tailor at gmail.com (nick tailor) Date: Wed, 27 Jul 2016 10:58:31 -0700 Subject: Varnish CPU Usage In-Reply-To: References: Message-ID: Im not sure you can set a port setting to an ip. Shouldnt it be a port number like 80? Cheers Nick Tailor Nicktailor.com On Jul 27, 2016 10:42 AM, "Ayberk Kimsesiz" wrote: > Hello all, > > I have a problem with Varnish. Even though when i leave Default.vcl file > with its DEFAULT settings or as Wordpress optimized, Apache's CPU usage > still increases to 70 - 90%.. But if i disable Varnish, CPU usage returns > to normal. > > I also cannot access to the website when i set the port setting as > 127.0.0.1. Therefore this makes me have to use Server IP instead. What is > your suggestions on these matters? > > Thanks. > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ayberk.kimsesiz at gmail.com Wed Jul 27 18:16:25 2016 From: ayberk.kimsesiz at gmail.com (Ayberk Kimsesiz) Date: Wed, 27 Jul 2016 21:16:25 +0300 Subject: Varnish CPU Usage In-Reply-To: References: Message-ID: Hi, Default.vcl: backend default { .host = "MY SERVER IP"; .port = "8080"; httpd.conf : Listen 8080 2016-07-27 20:58 GMT+03:00 nick tailor : > Im not sure you can set a port setting to an ip. Shouldnt it be a port > number like 80? > > Cheers > > Nick Tailor > Nicktailor.com > > On Jul 27, 2016 10:42 AM, "Ayberk Kimsesiz" > wrote: > >> Hello all, >> >> I have a problem with Varnish. Even though when i leave Default.vcl file >> with its DEFAULT settings or as Wordpress optimized, Apache's CPU usage >> still increases to 70 - 90%.. But if i disable Varnish, CPU usage returns >> to normal. >> >> I also cannot access to the website when i set the port setting as >> 127.0.0.1. Therefore this makes me have to use Server IP instead. What is >> your suggestions on these matters? >> >> Thanks. >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mic at strg.at Thu Jul 28 13:49:54 2016 From: mic at strg.at (Michael Dosser) Date: Thu, 28 Jul 2016 15:49:54 +0200 Subject: Varnish threads growing until no response In-Reply-To: <90B82C21-4A94-4AAE-95BB-CF9623DD78F8@strg.at> References: <90B82C21-4A94-4AAE-95BB-CF9623DD78F8@strg.at> Message-ID: <441BC562-AF7B-4103-81C3-82325B1B8E49@strg.at> Hi, > Am 27.07.2016 um 16:34 schrieb Michael Dosser : [...] >> Can you set thread_pool_add_delay to 0 and let us know how it goes? > > I?ve set it to 0 and will let you know if this solves the problem, thx! unfortunately this setting did not fix the problem. Again the Thread-Count did rise up very fast (within 30 seconds +40 threads). Varnish is restarted by a script before we hit 2000 threads and the script dumps varnishstat -1. At that time: MAIN.threads 441 . Total number of threads MAIN.threads_created 441 0.00 Threads created MAIN.sess_queued 439 0.00 Sessions queued for thread sess_queued is normally 0 ? only during the ?pike? it rises up. It seems something is blocking. How can I debug this further? Thx, Michael Dosser -- strg.at gmbh michael.dosser at strg.at gumpendorferstrasse 132, top 9, 1060 wien tel +43 (1) 526 56 29 mobile +43 699 1 7777 164 From ayberk.kimsesiz at gmail.com Thu Jul 28 15:11:20 2016 From: ayberk.kimsesiz at gmail.com (Ayberk Kimsesiz) Date: Thu, 28 Jul 2016 18:11:20 +0300 Subject: Varnish CPU Usage In-Reply-To: References: Message-ID: Is there no one to help? 2016-07-27 21:16 GMT+03:00 Ayberk Kimsesiz : > Hi, > > Default.vcl: > > backend default { > .host = "MY SERVER IP"; > .port = "8080"; > > httpd.conf : > > Listen 8080 > > 2016-07-27 20:58 GMT+03:00 nick tailor : > >> Im not sure you can set a port setting to an ip. Shouldnt it be a port >> number like 80? >> >> Cheers >> >> Nick Tailor >> Nicktailor.com >> >> On Jul 27, 2016 10:42 AM, "Ayberk Kimsesiz" >> wrote: >> >>> Hello all, >>> >>> I have a problem with Varnish. Even though when i leave Default.vcl file >>> with its DEFAULT settings or as Wordpress optimized, Apache's CPU usage >>> still increases to 70 - 90%.. But if i disable Varnish, CPU usage returns >>> to normal. >>> >>> I also cannot access to the website when i set the port setting as >>> 127.0.0.1. Therefore this makes me have to use Server IP instead. What is >>> your suggestions on these matters? >>> >>> Thanks. >>> >>> _______________________________________________ >>> varnish-misc mailing list >>> varnish-misc at varnish-cache.org >>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Thu Jul 28 16:08:07 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Thu, 28 Jul 2016 18:08:07 +0200 Subject: Varnish CPU Usage In-Reply-To: References: Message-ID: can you pastebin your vcl? and maybe a varnishlog? -- Guillaume Quintard -------------- next part -------------- An HTML attachment was scrubbed... URL: From ayberk.kimsesiz at gmail.com Thu Jul 28 16:26:01 2016 From: ayberk.kimsesiz at gmail.com (Ayberk Kimsesiz) Date: Thu, 28 Jul 2016 19:26:01 +0300 Subject: Varnish CPU Usage In-Reply-To: References: Message-ID: Hi, *Varnishlog* * Begin req 11316006 rxreq* *- Timestamp Start: 1469722432.394488 0.000000 0.000000* *- Timestamp Req: 1469722432.394488 0.000000 0.000000* *- ReqStart 37.155.54.14 44531* *- ReqMethod GET* *- ReqURL /wp-content/cache/minify/000000/M9QvyC8oLdBNqtQtLi0oriwuyUzWzyrWT84vSgUA.js* *- ReqProtocol HTTP/1.1* *- ReqHeader Host: ******.com* *- ReqHeader Connection: keep-alive* *- ReqHeader Accept: */** *- ReqHeader User-Agent: Mozilla/5.0 (iPhone; CPU iPhone OS 9_3_3 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 Mobile/13G34 Safari/601.1* *- ReqHeader Accept-Language: tr-tr* *- ReqHeader Referer: http://******.com/2015/05/28/iphone-sim/* *- ReqHeader Accept-Encoding: gzip, deflate* *- ReqHeader X-Forwarded-For: 37.155.54.14* *- VCL_call RECV* *- ReqHeader X-Actual-IP: 37.155.54.14* *- ReqUnset X-Forwarded-For: 37.155.54.14* *- ReqHeader X-Forwarded-For: 37.155.54.14, 37.155.54.14* *- ReqHeader Cookie:* *- ReqUnset Cookie:* *- ReqHeader Cookie:* *- ReqUnset X-Forwarded-For: 37.155.54.14, 37.155.54.14* *- ReqHeader X-Forwarded-For: 37.155.54.14, 37.155.54.14, 37.155.54.14* *- ReqUnset Accept-Encoding: gzip, deflate* *- ReqHeader Accept-Encoding: gzip* *- ReqUnset Cookie:* *- VCL_return hash* *- VCL_call HASH* *- VCL_return lookup* *- Hit 2159905671* *- VCL_call HIT* *- VCL_return deliver* *- RespProtocol HTTP/1.1* *- RespStatus 200* *- RespReason OK* *- RespHeader Date: Thu, 28 Jul 2016 15:39:44 GMT* *- RespHeader Server: Apache/2* *- RespHeader Last-Modified: Thu, 28 Jul 2016 07:01:56 GMT* *- RespHeader ETag: "2086-538acb5a886de-gzip"* *- RespHeader Content-Encoding: gzip* *- RespHeader Cache-Control: public, must-revalidate, proxy-revalidate* *- RespHeader Expires: Fri, 05 Aug 2016 16:21:56 GMT* *- RespHeader X-Powered-By: W3 Total Cache/0.9.4.1 * *- RespHeader Pragma: public* *- RespHeader Content-Length: 2648* *- RespHeader Content-Type: application/x-javascript* *- RespHeader Vary: Accept-Encoding* *- RespHeader X-Varnish: 10925524 12422023* *- RespHeader Age: 2048* *- RespHeader Via: 1.1 varnish-v4* *- VCL_call DELIVER* *- RespHeader X-Cache: HIT* *- VCL_return deliver* *- Timestamp Process: 1469722432.394522 0.000034 0.000034* *- Debug "RES_MODE 2"* *- RespHeader Connection: keep-alive* *- RespHeader Accept-Ranges: bytes* *- Timestamp Resp: 1469722432.394532 0.000044 0.000009* *- Debug "XXX REF 2"* *- ReqAcct 437 0 437 536 2648 3184* *- End* *Varnishstat* *Uptime mgt: 7+16:11:24 Hitrate n: 10 15 15* *Uptime child: 1+05:05:54 avg(n): 0.9377 0.9307 0.9307* * NAME CURRENT CHANGE AVERAGE AVG_10 AVG_100 AVG_1000* *MAIN.uptime 104754 1.00 1.00 1.00 1.00 1.00* *MAIN.sess_conn 945720 10.98 9.00 8.95 8.85 8.85* *MAIN.client_req 3194781 54.91 30.00 71.19 72.71 72.71* *MAIN.cache_hit 2958915 50.92 28.00 69.23 70.78 70.78* *MAIN.cache_miss 38356 1.00 . 0.17 0.14 0.14* *MAIN.backend_conn 53745 0.00 0.50 0.50 0.50* *MAIN.backend_fail 1 0.00 0.00 0.00 0.00* *MAIN.backend_reuse 181395 2.00 1.00 1.20 1.14 1.14* *MAIN.backend_toolate 51712 0.00 . 0.62 0.64 0.64* *MAIN.backend_recycle 233110 3.00 2.00 2.04 2.00 2.00* *MAIN.backend_retry 44 0.00 . 0.00 0.00 0.00* *MAIN.fetch_head 16 0.00 . 0.00 0.00 0.00* *MAIN.fetch_length 216684 3.00 2.00 2.04 2.00 2.00* *MAIN.fetch_chunked 17377 0.00 . 0.00 0.00 0.00* *MAIN.fetch_close 57 0.00 . 0.00 0.00 0.00* *MAIN.fetch_304 591 0.00 . 0.00 0.00 0.00* *MAIN.pools 2 0.00 . 2.00 2.00 2.00* *MAIN.threads 100 0.00 . 100.00 100.00 100.00* *MAIN.threads_limited 1 0.00 . 0.00 0.00 0.00* *MAIN.threads_created 402 0.00 . 0.00 0.00 0.00* *MAIN.threads_destroyed 302 0.00 . 0.00 0.00 0.00* *MAIN.busy_sleep 280 0.00 . 0.00 0.00 0.00* *MAIN.busy_wakeup 280 0.00 . 0.00 0.00 0.00* *MAIN.sess_queued 307 0.00 . 0.00 0.00 0.00* *MAIN.n_object 38125 0.00 . 38122.79 38122.60 38122.60* *Default.vcl* *vcl 4.0;* *import std;* *backend default {* * .host = "MY SERVER IP";* * .port = "8080";* *Varnish* *# Configuration file for varnish* *#* *# /etc/init.d/varnish expects the variable $DAEMON_OPTS to be set from this* *# shell script fragment.* *#* *# Maximum number of open files (for ulimit -n)* *NFILES=131072* *# Locked shared memory (for ulimit -l)* *# Default log size is 82MB + header* *MEMLOCK=82000* *# Maximum number of threads (for ulimit -u)* *NPROCS="unlimited"* *# Maximum size of corefile (for ulimit -c). Default in Fedora is 0* *# DAEMON_COREFILE_LIMIT="unlimited"* *# Set this to 1 to make init script reload try to switch vcl without restart.* *# To make this work, you need to set the following variables* *# explicit: VARNISH_VCL_CONF, VARNISH_ADMIN_LISTEN_ADDRESS,* *# VARNISH_ADMIN_LISTEN_PORT, VARNISH_SECRET_FILE, or in short,* *# use Alternative 3, Advanced configuration, below* *RELOAD_VCL=1* *# This file contains 4 alternatives, please use only one.* *## Alternative 1, Minimal configuration, no VCL* *#* *# Listen on port 6081, administration on localhost:6082, and forward to* *# content server on localhost:8080. Use a fixed-size cache file.* *#* *#DAEMON_OPTS="-a :6081 \* *# -T localhost:6082 \* *# -b localhost:8080 \* *# -u varnish -g varnish \* *# -s file,/var/lib/varnish/varnish_storage.bin,1G"* *## Alternative 2, Configuration with VCL* *#* *# Listen on port 6081, administration on localhost:6082, and forward to* *# one content server selected by the vcl file, based on the request. Use a* *# fixed-size cache file.* *#* *#DAEMON_OPTS="-a :80 \* *# -T localhost:6082 \* *# -f /etc/varnish/default.vcl \* *# -S /etc/varnish/secret \* *# -s malloc,2G"* *## Alternative 3, Advanced configuration* *#* *# See varnishd(1) for more information.* *#* *# # Main configuration file. You probably want to change it :)* *VARNISH_VCL_CONF=/etc/varnish/default.vcl* *#* *# # Default address and port to bind to* *# # Blank address means all IPv4 and IPv6 interfaces, otherwise specify* *# # a host name, an IPv4 dotted quad, or an IPv6 address in brackets.* *# VARNISH_LISTEN_ADDRESS=* *VARNISH_LISTEN_PORT=80* *#* *# # Telnet admin interface listen address and port* *VARNISH_ADMIN_LISTEN_ADDRESS=127.0.0.1* *VARNISH_ADMIN_LISTEN_PORT=6082* *#* *# # Shared secret file for admin interface* *VARNISH_SECRET_FILE=/etc/varnish/secret* *#* *# # The minimum number of worker threads to start* *VARNISH_MIN_THREADS=50* *#* *# # The Maximum number of worker threads to start* *VARNISH_MAX_THREADS=1000* *#* *# # Idle timeout for worker threads* *VARNISH_THREAD_TIMEOUT=120* *#* *# # Cache file size: in bytes, optionally using k / M / G / T suffix,* *# # or in percentage of available disk space using the % suffix.* *VARNISH_STORAGE_SIZE=4G* *#* *# # Backend storage specification* *VARNISH_STORAGE="malloc,${VARNISH_STORAGE_SIZE}"* *#* *# # Default TTL used when the backend does not specify one* *VARNISH_TTL=120* *#* *# # DAEMON_OPTS is used by the init script. If you add or remove options, make* *# # sure you update this section, too.* *DAEMON_OPTS="-a ${VARNISH_LISTEN_ADDRESS}:${VARNISH_LISTEN_PORT} \* * -f ${VARNISH_VCL_CONF} \* * -T ${VARNISH_ADMIN_LISTEN_ADDRESS}:${VARNISH_ADMIN_LISTEN_PORT} \* * -t ${VARNISH_TTL} \* * -p thread_pool_min=${VARNISH_MIN_THREADS} \* * -p thread_pool_max=${VARNISH_MAX_THREADS} \* * -p thread_pool_timeout=${VARNISH_THREAD_TIMEOUT} \* * -u varnish -g varnish \* * -S ${VARNISH_SECRET_FILE} \* * -s ${VARNISH_STORAGE}"* *#* *## Alternative 4, Do It Yourself. See varnishd(1) for more information.* *#* *# DAEMON_OPTS=""* 2016-07-28 19:08 GMT+03:00 Guillaume Quintard < guillaume at varnish-software.com>: > can you pastebin your vcl? and maybe a varnishlog? > > -- > Guillaume Quintard > -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Thu Jul 28 16:31:55 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Thu, 28 Jul 2016 18:31:55 +0200 Subject: Varnish CPU Usage In-Reply-To: References: Message-ID: You seem to have a good hit ratio, are you seeing anything on the apache logs that would explain the cpu usage? -- Guillaume Quintard -------------- next part -------------- An HTML attachment was scrubbed... URL: From ayberk.kimsesiz at gmail.com Thu Jul 28 16:57:25 2016 From: ayberk.kimsesiz at gmail.com (Ayberk Kimsesiz) Date: Thu, 28 Jul 2016 19:57:25 +0300 Subject: Varnish CPU Usage In-Reply-To: References: Message-ID: Hi, *CPU Monitor: * http://i.imgur.com/5KT1xRu.jpg *Apache status:* SrvPIDAccMCPUSSReqConnChildSlotClientProtocolVHostRequest *0-0* - 0/0/9766 . 134.59 37 0 0.0 0.00 64.40 ::1 http/1.1 ns1.***com:8080 OPTIONS * HTTP/1.0 *1-0* 14612 0/16/9058 _ 17.83 13 1498 0.0 0.02 53.29 176.***.10 http/1.1 www.***.com:8080 POST /wp-admin/admin-ajax.php HTTP/1.1 *2-0* 10863 0/179/9795 _ 185.14 6 1424 0.0 0.58 60.32 176.***.10 http/1.1 www.***.com:8080 POST /wp-admin/admin-ajax.php HTTP/1.1 *3-0* 13127 0/127/9435 _ 119.80 4 1419 0.0 0.42 56.51 176.***.10 http/1.1 www.***.com:8080 POST /wp-admin/admin-ajax.php HTTP/1.1 *4-0* - 0/0/9187 . 0.00 50 0 0.0 0.00 56.60 ::1 http/1.1 ns1.***.com:8080 OPTIONS * HTTP/1.0 *5-0* 14851 0/9/8761 _ 8.95 13 1559 0.0 0.01 57.90 176.***.10 http/1.1 www.***.com:8080 POST /wp-admin/admin-ajax.php HTTP/1.1 *6-0* 14852 0/6/8130 _ 6.67 4 1482 0.0 0.01 51.88 176.***.10 http/1.1 www.***.com:8080 POST /wp-admin/admin-ajax.php HTTP/1.1 *7-0* 14192 11/57/8355 *K* 72.73 0 1363 106.6 0.44 52.79 176.***.10 http/1.1 www.***.com:8080 POST /wp-admin/admin-ajax.php HTTP/1.1 *8-0* 13067 0/125/7795 _ 121.19 13 1759 0.0 0.65 68.27 176.***.10 http/1.1 www.***.com:8080 POST /wp-admin/admin-ajax.php HTTP/1.1 Apache error logs don't show anything about CPU. 2016-07-28 19:31 GMT+03:00 Guillaume Quintard < guillaume at varnish-software.com>: > You seem to have a good hit ratio, are you seeing anything on the apache > logs that would explain the cpu usage? > > -- > Guillaume Quintard > -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Fri Jul 29 12:20:14 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Fri, 29 Jul 2016 14:20:14 +0200 Subject: Varnish threads growing until no response In-Reply-To: <441BC562-AF7B-4103-81C3-82325B1B8E49@strg.at> References: <90B82C21-4A94-4AAE-95BB-CF9623DD78F8@strg.at> <441BC562-AF7B-4103-81C3-82325B1B8E49@strg.at> Message-ID: Silly question, but have you tried having more threads? And at the same time, thread_pool should be set to 2, unless you have a very valid reason to do otherwise. -- Guillaume Quintard -------------- next part -------------- An HTML attachment was scrubbed... URL: From mic at strg.at Fri Jul 29 12:44:47 2016 From: mic at strg.at (Michael Dosser) Date: Fri, 29 Jul 2016 14:44:47 +0200 Subject: Varnish threads growing until no response In-Reply-To: References: <90B82C21-4A94-4AAE-95BB-CF9623DD78F8@strg.at> <441BC562-AF7B-4103-81C3-82325B1B8E49@strg.at> Message-ID: <7C164087-1CB2-438A-9A7F-1FAF0995C22E@strg.at> Hi, > Am 29.07.2016 um 14:20 schrieb Guillaume Quintard : > > Silly question, but have you tried having more threads? And at the same time, thread_pool should be set to 2, unless you have a very valid reason to do otherwise. thanks for your help. I?ll try these new settings! I didn?t think about changing the startup logic and always thought our problem is coming from our quite complex VCL. Michael Dosser PS: I changed the startup to: varnishd -f /etc/varnish/fuf.vcl \ -h critbit \ -a 127.0.0.1:6081 \ -T 127.0.0.1:6082 \ -t 120 \ -S /etc/varnish/secret \ -s malloc,12G \ -p thread_pool_min=500 \ -p thread_pool_max=3000 \ -p thread_pools=2 \ -p thread_pool_add_delay=0 \ -p vcc_allow_inline_c=on \ -p feature=+esi_disable_xml_check \ -p feature=+esi_ignore_other_elements \ -p feature=+esi_ignore_https" -- strg.at gmbh michael.dosser at strg.at gumpendorferstrasse 132, top 9, 1060 wien tel +43 (1) 526 56 29 mobile +43 699 1 7777 164 From ayberk.kimsesiz at gmail.com Sun Jul 31 05:54:42 2016 From: ayberk.kimsesiz at gmail.com (Ayberk Kimsesiz) Date: Sun, 31 Jul 2016 08:54:42 +0300 Subject: Varnish CPU Usage In-Reply-To: References: Message-ID: Can you help? The problem is still continue. While Varnish is active CPU usage is always close to 100%. 2016-07-28 19:57 GMT+03:00 Ayberk Kimsesiz : > Hi, > > *CPU Monitor: * > > http://i.imgur.com/5KT1xRu.jpg > > *Apache status:* > > SrvPIDAccMCPUSSReqConnChildSlotClientProtocolVHostRequest > *0-0* - 0/0/9766 . 134.59 37 0 0.0 0.00 64.40 ::1 http/1.1 ns1.***com:8080 OPTIONS > * HTTP/1.0 > *1-0* 14612 0/16/9058 _ 17.83 13 1498 0.0 0.02 53.29 176.***.10 http/1.1 > www.***.com:8080 POST /wp-admin/admin-ajax.php HTTP/1.1 > *2-0* 10863 0/179/9795 _ 185.14 6 1424 0.0 0.58 60.32 176.***.10 http/1.1 > www.***.com:8080 POST /wp-admin/admin-ajax.php HTTP/1.1 > *3-0* 13127 0/127/9435 _ 119.80 4 1419 0.0 0.42 56.51 176.***.10 http/1.1 > www.***.com:8080 POST /wp-admin/admin-ajax.php HTTP/1.1 > *4-0* - 0/0/9187 . 0.00 50 0 0.0 0.00 56.60 ::1 http/1.1 ns1.***.com:8080 OPTIONS > * HTTP/1.0 > *5-0* 14851 0/9/8761 _ 8.95 13 1559 0.0 0.01 57.90 176.***.10 http/1.1 > www.***.com:8080 POST /wp-admin/admin-ajax.php HTTP/1.1 > *6-0* 14852 0/6/8130 _ 6.67 4 1482 0.0 0.01 51.88 176.***.10 http/1.1 > www.***.com:8080 POST /wp-admin/admin-ajax.php HTTP/1.1 > *7-0* 14192 11/57/8355 *K* 72.73 0 1363 106.6 0.44 52.79 176.***.10 > http/1.1 www.***.com:8080 POST /wp-admin/admin-ajax.php HTTP/1.1 > *8-0* 13067 0/125/7795 _ 121.19 13 1759 0.0 0.65 68.27 176.***.10 http/1.1 > www.***.com:8080 POST /wp-admin/admin-ajax.php HTTP/1.1 > > Apache error logs don't show anything about CPU. > > > 2016-07-28 19:31 GMT+03:00 Guillaume Quintard < > guillaume at varnish-software.com>: > >> You seem to have a good hit ratio, are you seeing anything on the apache >> logs that would explain the cpu usage? >> >> -- >> Guillaume Quintard >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: