From raymond.jennings.iii at gmail.com Thu Oct 1 17:53:36 2015 From: raymond.jennings.iii at gmail.com (Raymond Jennings III) Date: Thu, 1 Oct 2015 13:53:36 -0400 Subject: Trying to install Varnish 4.1 but can't find jemalloc package Message-ID: This is RH 7. I have the started yum repos but cannot find jemalloc that is required to install Varnish 4.1 Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From lkarsten at varnish-software.com Fri Oct 2 14:14:48 2015 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Fri, 2 Oct 2015 16:14:48 +0200 Subject: Trying to install Varnish 4.1 but can't find jemalloc package In-Reply-To: References: Message-ID: <20151002141447.GA17552@immer.varnish-software.com> On Thu, Oct 01, 2015 at 01:53:36PM -0400, Raymond Jennings III wrote: > This is RH 7. I have the started yum repos but cannot find jemalloc that > is required to install Varnish 4.1 Hi. You can get the jemalloc package from EPEL. -- Lasse Karstensen Varnish Software AS From tpapad+varnish at gmail.com Sun Oct 4 19:39:44 2015 From: tpapad+varnish at gmail.com (Anastasios Papadopoulos) Date: Sun, 4 Oct 2015 22:39:44 +0300 Subject: Caching authenticated responses Message-ID: Is there some easy way to configure Varnish to cache authenticated responses? The concept is quite simple: the back-end is a REST API provider and the response payload is mostly cacheable (i.e. rarely changes). The API uses basic HTTP authentication which prevents default Varnish to cache it. However, since the user base is limited, I'd like Varnish to cache (meaning: respect the back-end caching headers, not blindly cache anything) using the authorization header as part of the hash / cache key. Is this even possible? Any sample VCL or pointers would be welcomed! Thank you very much! -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Sun Oct 4 20:23:25 2015 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Sun, 4 Oct 2015 22:23:25 +0200 Subject: Caching authenticated responses In-Reply-To: References: Message-ID: Hi Anastasios, This is totally feasible, look at the vcl_hash fonction in docs, it'll allow to control what goes into the hash. On Oct 4, 2015 21:41, "Anastasios Papadopoulos" wrote: > Is there some easy way to configure Varnish to cache authenticated > responses? > The concept is quite simple: the back-end is a REST API provider and the > response payload is mostly cacheable (i.e. rarely changes). The API uses > basic HTTP authentication which prevents default Varnish to cache it. > However, since the user base is limited, I'd like Varnish to cache > (meaning: respect the back-end caching headers, not blindly cache anything) > using the authorization header as part of the hash / cache key. Is this > even possible? > Any sample VCL or pointers would be welcomed! > > Thank you very much! > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From raymond.jennings.iii at gmail.com Mon Oct 5 16:35:26 2015 From: raymond.jennings.iii at gmail.com (Raymond Jennings III) Date: Mon, 5 Oct 2015 12:35:26 -0400 Subject: Difficulty in setting varnishncsa format in RH 7 Varnish 4.1 Message-ID: It used to be easy in CentOS 6 with Varnish 4. Different service startup methods now so in the file: /usr/lib/systemd/system/varnishncsa.service I have: ExecStart=/usr/bin/varnishncsa -a -w /var/log/varnish/varnishncsa.log -D -P /run/varnishncsa.pid -F "\"%h %l %u %t %r %s %b %D\"" But the -F option is not escaping things properly. Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From raymond.jennings.iii at gmail.com Mon Oct 5 20:29:29 2015 From: raymond.jennings.iii at gmail.com (Raymond Jennings III) Date: Mon, 5 Oct 2015 16:29:29 -0400 Subject: Is there a debug vmod for 4.1 available for RH 7.1? Message-ID: Or even a zip if I can build it myself. -------------- next part -------------- An HTML attachment was scrubbed... URL: From wido at widodh.nl Tue Oct 6 11:01:05 2015 From: wido at widodh.nl (Wido den Hollander) Date: Tue, 6 Oct 2015 13:01:05 +0200 Subject: IPv6 connectivity for repo.varnish-cache.org Message-ID: <5613A9F1.7040608@widodh.nl> Hello, I'm managing more and more machines running Varnish which are IPv6-only and they currently can't use repo.varnish-cache.org to download the Varnish packages. For now I've 'fixed' this locally by setting up a HTTP proxy and pointing Apt towards that proxy, but that's not ideal. I see that both www.varnish-cache.org and repo.varnish-cache.org don't have IPv6, but varnish-software.com has. Can I request that IPv6 connectivity is added to varnish-cache.org so that IPv6-only machines can also reach it? Thanks! Wido From phk at phk.freebsd.dk Tue Oct 6 11:03:42 2015 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Tue, 06 Oct 2015 11:03:42 +0000 Subject: IPv6 connectivity for repo.varnish-cache.org In-Reply-To: <5613A9F1.7040608@widodh.nl> References: <5613A9F1.7040608@widodh.nl> Message-ID: <95747.1444129422@critter.freebsd.dk> -------- In message <5613A9F1.7040608 at widodh.nl>, Wido den Hollander writes: >Can I request that IPv6 connectivity is added to varnish-cache.org so >that IPv6-only machines can also reach it? We're just now planning a major shuffle of infrastructure, and the IPv6 issue should become fixed as a result of that. Stay tuned. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From wido at widodh.nl Tue Oct 6 18:20:09 2015 From: wido at widodh.nl (Wido den Hollander) Date: Tue, 6 Oct 2015 20:20:09 +0200 Subject: IPv6 connectivity for repo.varnish-cache.org In-Reply-To: <95747.1444129422@critter.freebsd.dk> References: <5613A9F1.7040608@widodh.nl> <95747.1444129422@critter.freebsd.dk> Message-ID: <561410D9.5090009@widodh.nl> On 10/06/2015 01:03 PM, Poul-Henning Kamp wrote: > -------- > In message <5613A9F1.7040608 at widodh.nl>, Wido den Hollander writes: > >> Can I request that IPv6 connectivity is added to varnish-cache.org so >> that IPv6-only machines can also reach it? > > We're just now planning a major shuffle of infrastructure, and the > IPv6 issue should become fixed as a result of that. > That's great to hear. From most vendors you hear it's 'somewhere' on the roadmap. > Stay tuned. > I'll do. In the meantime I'll get by with my proxy. Wido From wido at widodh.nl Tue Oct 6 18:29:46 2015 From: wido at widodh.nl (Wido den Hollander) Date: Tue, 6 Oct 2015 20:29:46 +0200 Subject: 503 backend error after backends reponds with 204 without a body Message-ID: <5614131A.6010602@widodh.nl> Hi, I've upgraded to Varnish 4.1.0 today on a system where there is Ceph's RADOS Gateway [0] running with the embedded Civetweb [1] webserver. When a client sends a DELETE command the backend responds with a HTTP 204 without a body, but this causes Varnish to throw a 503 backend error. -6- BerespProtocol HTTP/1.1 -6- BerespStatus 204 -6- BerespReason No Content -6- BerespHeader Bucket: "widodh" -6- BerespHeader Content-type: application/xml -6- BerespHeader Content-Length: 0 -6- BerespHeader Date: Tue, 06 Oct 2015 18:23:07 GMT And Varnish then says: -6- Error Body cannot be fetched -6- Timestamp Error: 1444155787.622126 0.175832 0.000177 -6- BerespProtocol HTTP/1.1 -6- BerespStatus 503 -6- BerespReason Service Unavailable -6- BerespReason Backend fetch failed -6- BerespHeader Date: Tue, 06 Oct 2015 18:23:07 GMT -6- BerespHeader Server: Varnish -6- VCL_call BACKEND_ERROR Reading the HTTP specification [2] I found this: "The 204 response MUST NOT include a message-body, and thus is always terminated by the first empty line after the header fields." I can probably catch this in vcl_backend_response, pass to vcl_synth and come up with a proper response to the client, but is that the way to go? Is this a bug in Varnish or is it keeping to the RFCs? Thanks, Wido [0]: http://docs.ceph.com/docs/master/radosgw/ [1]: https://github.com/civetweb/civetweb [2]: http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html From straightflush at gmail.com Wed Oct 7 23:10:47 2015 From: straightflush at gmail.com (AD) Date: Wed, 7 Oct 2015 16:10:47 -0700 Subject: JSON to VCL Generator Message-ID: hey all -- Just wanted to share an open source project we just released for fun. Its called VCLGenie and exposes a REST API for converting JSON to VCL (4.0). Its alpha, an early release and has a simple frontend (source and demo site available) but figured I would share. Ton of functionality still to come. Code and README outlining functionality here: https://github.com/iheartradio/vclgenie/ Demo available here: http://www.vclgenie.com Cheers A -------------- next part -------------- An HTML attachment was scrubbed... URL: From raymond.jennings.iii at gmail.com Thu Oct 8 14:08:45 2015 From: raymond.jennings.iii at gmail.com (Raymond Jennings III) Date: Thu, 8 Oct 2015 10:08:45 -0400 Subject: Yet another Varnish 4.1 install problem Message-ID: RH 6.7, RH 7.1 - it used to be easy...but here's the latest cryptic error on a most basic RH 6.7 install service varnish start Starting Varnish Cache: Assert error in vju_make_vcldir(), mgt/mgt_jail_unix.c line 246: Condition((chown(dname, vju_uid, vju_gid)) == 0) not true. errno = 1 (Operation not permitted) /bin/bash: line 1: 5198 Aborted /usr/sbin/varnishd -a :6081 -f /etc/varnish/default.vcl -T 127.0.0.1:6082 -p thread_pool_min=50 -p thread_pool_max=1000 -S /etc/varnish/secret -s malloc,256M -P /var/run/varnish.pid -------------- next part -------------- An HTML attachment was scrubbed... URL: From raymond.jennings.iii at gmail.com Thu Oct 8 14:18:37 2015 From: raymond.jennings.iii at gmail.com (Raymond Jennings III) Date: Thu, 8 Oct 2015 10:18:37 -0400 Subject: Yet another Varnish 4.1 install problem In-Reply-To: References: Message-ID: run getenforce and make sure selinux is disabled. On Thu, Oct 8, 2015 at 10:08 AM, Raymond Jennings III < raymond.jennings.iii at gmail.com> wrote: > RH 6.7, RH 7.1 - it used to be easy...but here's the latest cryptic error > on a most basic RH 6.7 install > > service varnish start > Starting Varnish Cache: Assert error in vju_make_vcldir(), > mgt/mgt_jail_unix.c line 246: > Condition((chown(dname, vju_uid, vju_gid)) == 0) not true. > errno = 1 (Operation not permitted) > /bin/bash: line 1: 5198 Aborted /usr/sbin/varnishd -a > :6081 -f /etc/varnish/default.vcl -T 127.0.0.1:6082 -p thread_pool_min=50 > -p thread_pool_max=1000 -S /etc/varnish/secret -s malloc,256M -P > /var/run/varnish.pid > -------------- next part -------------- An HTML attachment was scrubbed... URL: From viktor.villafuerte at optusnet.com.au Mon Oct 12 04:17:05 2015 From: viktor.villafuerte at optusnet.com.au (Viktor Villafuerte) Date: Mon, 12 Oct 2015 15:17:05 +1100 Subject: Threads + thread queue length Message-ID: <20151012041705.GA1990@optusnet.com.au> Hi all you carpenters and other Varnish using folk, There are couple of things in the output of varnishstat that puzzle me a little.. MAIN.sess_drop 0 0.00 Sessions dropped MAIN.sess_dropped 3809332 0.32 Sessions dropped for thread MAIN.fetch_no_thread 58746 0.01 Fetch failed (no thread) MAIN.pools 2 . Number of thread pools MAIN.threads 1255 . Total number of threads I've got 2 pools of 4000 threads set in Varnish config and man varnish-counters says: sess_drop Count of sessions silently dropped due to lack of worker thread. sess_dropped Number of times session was dropped because the queue were too long already. See also parameter queue_max. fetch_no_thread beresp fetch failed, no thread available This tells me that there's no lack of worker threads (good!), but the thread queue length does get too long and subsequently sessions get dropped (bad!). Also backend fetch failed due to no threads being available (what?) Now the puzzling bit :) 1) why would the thread queue get too long if there seems to be NO lack of threads to use? 2) why would there be no threads if there seems to be NO lack of threads 3) 'See also the parameter queue_max' - but I cannot find any mention of such parameter anywhere around? Where does this ellusive paramater live then? Could anybody shed bit of light on this for me? -- Regards Viktor Villafuerte Optus Internet Engineering t: +61 2 80825265 From hugues at betabrand.com Tue Oct 13 02:12:53 2015 From: hugues at betabrand.com (Hugues Alary) Date: Mon, 12 Oct 2015 19:12:53 -0700 Subject: Child said getnameinfo = -11 System error Message-ID: Hi there, Using Varnish 4.0.3, I've noticed my logs are spammed with these error messages. Oct 12 11:20:04 halflife varnishd[20488]: Child (20492) said getnameinfo = -11 System error Oct 12 11:20:29 halflife varnishd[20488]: Child (20492) said getnameinfo = -11 System error Oct 12 11:20:36 halflife varnishd[20488]: Child (20492) said getnameinfo = -11 System error Oct 12 11:20:37 halflife varnishd[20488]: Child (20492) said getnameinfo = -11 System error Oct 12 11:20:39 halflife varnishd[20488]: Child (20492) said getnameinfo = -11 System error Oct 12 11:20:45 halflife varnishd[20488]: Child (20492) said getnameinfo = -11 System error Oct 12 11:20:45 halflife varnishd[20488]: Child (20492) said getnameinfo = -11 System error Oct 12 11:20:51 halflife varnishd[20488]: Child (20492) said getnameinfo = -11 System error Oct 12 11:20:55 halflife varnishd[20488]: Child (20492) said getnameinfo = -11 System error Oct 12 11:20:56 halflife varnishd[20488]: Child (20492) said getnameinfo = -11 System error Oct 12 11:20:56 halflife varnishd[20488]: Child (20492) said getnameinfo = -11 System error Oct 12 11:21:04 halflife varnishd[20488]: Child (20492) said getnameinfo = -11 System error Oct 12 11:21:05 halflife varnishd[20488]: Child (20492) said getnameinfo = -11 System error Any idea what it could be? Some googling makes me think it's DNS related, but I can't confirm and my DNS seem to be working fine. (I'm using google's 8.8.8.8 servers) Thanks! -Hugues -------------- next part -------------- An HTML attachment was scrubbed... URL: From de.techno at gmail.com Wed Oct 14 11:01:41 2015 From: de.techno at gmail.com (dE) Date: Wed, 14 Oct 2015 16:31:41 +0530 Subject: Minimalistic VCL does not compile Message-ID: <561E3615.8030508@gmail.com> Hi! I got this VCL -- backend default { .host = "46.228.47.114"; .port = "https"; } sub vcl_recv { req.backend = "default"; req.http.host = "www.yahoo.com"; } But running varnishd complaints -- Message from VCC-compiler: Expected an action, 'if', '{' or '}' ('input' Line 9 Pos 9) req.backend = "default"; --------###########------------- Running VCC-compiler failed, exit 1 VCL compilation failed Thanks for any help! From viktor.gunnarson at ericsson.com Wed Oct 14 11:13:07 2015 From: viktor.gunnarson at ericsson.com (Viktor Gunnarson) Date: Wed, 14 Oct 2015 13:13:07 +0200 Subject: Minimalistic VCL does not compile In-Reply-To: <561E3615.8030508@gmail.com> References: <561E3615.8030508@gmail.com> Message-ID: <561E38C3.7040107@ericsson.com> Hi, There is no such thing as req.backend, I think what you are looking for is req.backend_hint but as you only have the default backend defined you do not need to specify it at all. You should however add "set" in front of the setting of the variables and the line "vcl 4.0;" first in the file as I hope you are running Varnish 4 or later. vcl 4.0; backend default { .host = "46.228.47.114"; .port = "https"; } sub vcl_recv { set req.http.host = "www.yahoo.com"; } Best regards, Viktor On 14/10/15 13:01, dE wrote: > Hi! > > I got this VCL -- > > backend default { > .host = "46.228.47.114"; > .port = "https"; > } > > sub vcl_recv { > req.backend = "default"; > req.http.host = "www.yahoo.com"; > } > > But running varnishd complaints -- > > Message from VCC-compiler: > Expected an action, 'if', '{' or '}' > ('input' Line 9 Pos 9) > req.backend = "default"; > --------###########------------- > > Running VCC-compiler failed, exit 1 > > VCL compilation failed > > Thanks for any help! > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc -------------- next part -------------- An HTML attachment was scrubbed... URL: From wido at widodh.nl Wed Oct 14 11:14:02 2015 From: wido at widodh.nl (Wido den Hollander) Date: Wed, 14 Oct 2015 13:14:02 +0200 Subject: Minimalistic VCL does not compile In-Reply-To: <561E3615.8030508@gmail.com> References: <561E3615.8030508@gmail.com> Message-ID: <561E38FA.1070104@widodh.nl> On 14-10-15 13:01, dE wrote: > Hi! > > I got this VCL -- > > backend default { > .host = "46.228.47.114"; > .port = "https"; > } > > sub vcl_recv { > req.backend = "default"; For Varnish 4.0 you should use req.backend_hint and it works a bit different. Check the documentation for more information. > req.http.host = "www.yahoo.com"; > } > > But running varnishd complaints -- > > Message from VCC-compiler: > Expected an action, 'if', '{' or '}' > ('input' Line 9 Pos 9) > req.backend = "default"; > --------###########------------- > > Running VCC-compiler failed, exit 1 > > VCL compilation failed > > Thanks for any help! > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc From apj at mutt.dk Wed Oct 14 13:08:26 2015 From: apj at mutt.dk (Andreas Plesner) Date: Wed, 14 Oct 2015 15:08:26 +0200 Subject: Minimalistic VCL does not compile In-Reply-To: <561E3615.8030508@gmail.com> References: <561E3615.8030508@gmail.com> Message-ID: <20151014130826.GM2656@nerd.dk> On Wed, Oct 14, 2015 at 04:31:41PM +0530, dE wrote: > req.backend = "default"; You're missing a "set". Note, that as others have pointed out, req.backend has been removed in favor of req.backend_hint since 4.0. -- Andreas From vlad.rusu at cloudtroopers.ro Wed Oct 14 19:54:31 2015 From: vlad.rusu at cloudtroopers.ro (Vlad Rusu) Date: Wed, 14 Oct 2015 22:54:31 +0300 Subject: 503s and FetchError - V3.0.7 Message-ID: <7CBC14D5-F56E-4FCB-A2DD-6F8B67FF0664@cloudtroopers.ro> Good day everyone, I am trying to get to the bottom of the 503s sporadically logged by our Varnish servers. During monitoring, it all comes down to monitoring the state of my backends ++ varnishlog. Most of the 503s come with a FetchError ?backend write error: 0 (Success)? ? and I can?t seem to find anything on the internet to describe what this might mean. ?> FetchError c backend write error: 0 (Success) ReqEnd c 221990096 1444847550.110866785 1444847550.116131306 0.000048876 0.000905752 0.004358768 From hugues at betabrand.com Wed Oct 14 20:57:43 2015 From: hugues at betabrand.com (Hugues Alary) Date: Wed, 14 Oct 2015 13:57:43 -0700 Subject: Child said getnameinfo = -11 System error In-Reply-To: References: Message-ID: Hi Jason, Those errors are unfortunately logged in my system log, not varnishlog, so I can't really link them to a specific request. However, I used not to have those errors, and I just discovered that they recently appeared along with some other errors in my varnishlogs this time: - Error out of workspace - LostHeader X-Forwarded-For: - ReqUnset X-Forwarded-For: 80.68.74.90 - ReqUnset X-Forwarded-For: 162.158.92.219 - Error out of workspace It seems that varnish is running out of workspace. Some googling tells me it could be related to ESI includes. It so happen that I did change something recently with ESI includes, and, I happen to have pages loading something like 100 ESI includes (maybe even more). I read that someone else using 4.0.3 was running out of workspace, despite setting a value up to 16mb. Maybe I should tune my workspace, but the bug report stated that raising the value to 16mb ended up eating up all the memory of the system and thus wasn't really an option. Any other ideas of things I should do? -Hugues On Wed, Oct 14, 2015 at 12:35 PM, Jason Price wrote: > Any chance of catching the full output of varnishlog for one of those > transactions? That would help figure out what's going on. > > -Jason > > On Mon, Oct 12, 2015 at 10:12 PM, Hugues Alary > wrote: > >> Hi there, >> >> Using Varnish 4.0.3, I've noticed my logs are spammed with these error >> messages. >> >> Oct 12 11:20:04 halflife varnishd[20488]: Child (20492) said getnameinfo >> = -11 System error >> Oct 12 11:20:29 halflife varnishd[20488]: Child (20492) said getnameinfo >> = -11 System error >> Oct 12 11:20:36 halflife varnishd[20488]: Child (20492) said getnameinfo >> = -11 System error >> Oct 12 11:20:37 halflife varnishd[20488]: Child (20492) said getnameinfo >> = -11 System error >> Oct 12 11:20:39 halflife varnishd[20488]: Child (20492) said getnameinfo >> = -11 System error >> Oct 12 11:20:45 halflife varnishd[20488]: Child (20492) said getnameinfo >> = -11 System error >> Oct 12 11:20:45 halflife varnishd[20488]: Child (20492) said getnameinfo >> = -11 System error >> Oct 12 11:20:51 halflife varnishd[20488]: Child (20492) said getnameinfo >> = -11 System error >> Oct 12 11:20:55 halflife varnishd[20488]: Child (20492) said getnameinfo >> = -11 System error >> Oct 12 11:20:56 halflife varnishd[20488]: Child (20492) said getnameinfo >> = -11 System error >> Oct 12 11:20:56 halflife varnishd[20488]: Child (20492) said getnameinfo >> = -11 System error >> Oct 12 11:21:04 halflife varnishd[20488]: Child (20492) said getnameinfo >> = -11 System error >> Oct 12 11:21:05 halflife varnishd[20488]: Child (20492) said getnameinfo >> = -11 System error >> >> Any idea what it could be? >> >> Some googling makes me think it's DNS related, but I can't confirm and my >> DNS seem to be working fine. (I'm using google's 8.8.8.8 servers) >> >> Thanks! >> -Hugues >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gabster at lelutin.ca Thu Oct 15 04:32:39 2015 From: gabster at lelutin.ca (Gabriel Filion) Date: Thu, 15 Oct 2015 00:32:39 -0400 Subject: Graphing number of 503s Message-ID: <561F2C67.30702@lelutin.ca> Hello there, I'd like to create a graph of how much 503 pages were served by varnish. Is there an easy way to count how much such error pages are sent to clients? The only way that I can think right now is to have a permanent varnishlog that is set to spot the 503s and then use the output from varnishlog to create the numbers. I was wondering if there might be a more clever way of doing this. -- Gabriel Filion -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: OpenPGP digital signature URL: From guillaume at varnish-software.com Thu Oct 15 06:38:39 2015 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Thu, 15 Oct 2015 08:38:39 +0200 Subject: Graphing number of 503s In-Reply-To: <561F2C67.30702@lelutin.ca> References: <561F2C67.30702@lelutin.ca> Message-ID: Hi Gabriel, Since the vsl is a circular buffer, your are forced to watch for the time period you are interested in. But it's probably easier to just use varnishncsa for what you want instead of varnishlog. On Oct 15, 2015 06:34, "Gabriel Filion" wrote: > Hello there, > > I'd like to create a graph of how much 503 pages were served by varnish. > Is there an easy way to count how much such error pages are sent to > clients? > > The only way that I can think right now is to have a permanent > varnishlog that is set to spot the 503s and then use the output from > varnishlog to create the numbers. > > I was wondering if there might be a more clever way of doing this. > > -- > Gabriel Filion > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From de.techno at gmail.com Thu Oct 15 07:58:35 2015 From: de.techno at gmail.com (dE) Date: Thu, 15 Oct 2015 13:28:35 +0530 Subject: Minimalistic VCL does not compile In-Reply-To: <20151014130826.GM2656@nerd.dk> References: <561E3615.8030508@gmail.com> <20151014130826.GM2656@nerd.dk> Message-ID: <561F5CAB.9040905@gmail.com> Thanks everyone for the solution. I'm using varnish 3 (Varnish 4 doesn't compile). One thing I don't get is why doesn't backend section variables (.host/.port etc...) does not need a 'set' while others do? On 10/14/15 18:38, Andreas Plesner wrote: > On Wed, Oct 14, 2015 at 04:31:41PM +0530, dE wrote: > >> req.backend = "default"; > You're missing a "set". > > Note, that as others have pointed out, req.backend has been removed in favor of > req.backend_hint since 4.0. > From apj at mutt.dk Thu Oct 15 08:32:00 2015 From: apj at mutt.dk (Andreas Plesner) Date: Thu, 15 Oct 2015 10:32:00 +0200 Subject: Minimalistic VCL does not compile In-Reply-To: <561F5CAB.9040905@gmail.com> References: <561E3615.8030508@gmail.com> <20151014130826.GM2656@nerd.dk> <561F5CAB.9040905@gmail.com> Message-ID: <20151015083200.GN2656@nerd.dk> On Thu, Oct 15, 2015 at 01:28:35PM +0530, dE wrote: > Thanks everyone for the solution. > > I'm using varnish 3 (Varnish 4 doesn't compile). One thing I don't > get is why doesn't backend section variables (.host/.port etc...) > does not need a 'set' while others do? Because that's a definition, not code. -- Andreas From perbu at varnish-software.com Thu Oct 15 19:00:27 2015 From: perbu at varnish-software.com (Per Buer) Date: Thu, 15 Oct 2015 21:00:27 +0200 Subject: Graphing number of 503s In-Reply-To: <561F2C67.30702@lelutin.ca> References: <561F2C67.30702@lelutin.ca> Message-ID: Hi, On Thu, Oct 15, 2015 at 6:32 AM, Gabriel Filion wrote: > Hello there, > > I'd like to create a graph of how much 503 pages were served by varnish. > Is there an easy way to count how much such error pages are sent to > clients? > Sure. Throw Google Analytics at it and you'll have tons of graphs. Just throw the GA stuff into the guru meditation page (Artur used this for all sort of stuff). Per. -- *Per Buer* CTO | Varnish Software AS Cell: +47 95839117 We Make Websites Fly! www.varnish-software.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From jdh132 at psu.edu Thu Oct 15 22:02:15 2015 From: jdh132 at psu.edu (Jason Heffner) Date: Thu, 15 Oct 2015 18:02:15 -0400 Subject: Even more varnish 4.1 issues Message-ID: <8C92F9AC-5E96-4C9F-91B0-6999AD2D52F8@psu.edu> I have SELinux in permissive mode, I uninstalled varnish 4.0.3 rpms and installed new varnish 4.1 rpms. I updated the /etc/sysconfig/varnish and varnish does start successfully. I can load any static files fine through varnish with the default config, and my own config. When I try and load a page that sets a cookie I will get the first response to the page fine, and request for a second page will return a 0 length content and white screen. I clear cookies and I can load another page till one page sets cookies and I?m back to the 0 length response. It?s really odd. Anyone have any idea what I may be missing? I disabled the only mod I was using as well. Thanks, Jason Systems Administrator Teaching and Learning with Technology, Information Technology Services The Pennsylvania State University From japrice at gmail.com Thu Oct 15 22:26:58 2015 From: japrice at gmail.com (Jason Price) Date: Thu, 15 Oct 2015 18:26:58 -0400 Subject: 503s and FetchError - V3.0.7 In-Reply-To: <7CBC14D5-F56E-4FCB-A2DD-6F8B67FF0664@cloudtroopers.ro> References: <7CBC14D5-F56E-4FCB-A2DD-6F8B67FF0664@cloudtroopers.ro> Message-ID: Party Line: varnish 3.x is EOL. Move to Varnish 4 Looks like you're logging the client side in varnishlog. Try to catch one of the Backend transactions that does this. You'll have to use varnishlog without either -c or -b or -m specified, and trawl through it manually. varnishadm debug.health might help as well. If all backends in a director are unhealthy, varnish will serve an immediate 503 if the request goes to that director. -Jason On Wed, Oct 14, 2015 at 3:54 PM, Vlad Rusu wrote: > Good day everyone, > > I am trying to get to the bottom of the 503s sporadically logged by our > Varnish servers. During monitoring, it all comes down to monitoring the > state of my backends ++ varnishlog. > > Most of the 503s come with a FetchError ?backend write error: 0 (Success)? > ? and I can?t seem to find anything on the internet to describe what this > might mean. > > ?> FetchError c backend write error: 0 (Success) > Also, none of the relevant counters in varnishstat are showing any > increase (i.e. backend_unhealthy, backend_busy, backend_fail, > fetch_failed..) > > For that same request, looking at ReqEnd, it doesn?t look to me like any > timeout was reached (like connect_timeout, first_byte_timeout, between_bytes_timeout, > ..) > > ?> ReqEnd c 221990096 1444847550.110866785 1444847550.116131306 > 0.000048876 0.000905752 0.004358768 > > *Any chance we could get a list of possible FetchError messages and what > they might mean ? at least something to put us on a right track? Would > surely help knowing where to start looking, especially when everything > seems fine.* > > Also, in this FetcError scenario, *is it expected to see retries from > Varnish?* If so, why would Varnish only retry some of the backends in a > director and not all of them (or the .retries value)? > > 44 VCL_return c hash > 44 VCL_call c pass pass > 44 Backend c 20 drpau_ssl_director drpau34ssl > 44 FetchError c backend write error: 0 (Success) > 44 Backend c 51 drpau_ssl_director drpau31ssl > 44 FetchError c backend write error: 0 (Success) > 44 VCL_call c error deliver > 44 VCL_call c deliver deliver > 44 TxProtocol c HTTP/1.1 > 44 TxStatus c 503 > 44 TxResponse c Service Unavailable > > Should these be reflected in varnishstat?s ?backend_retry?? > > Thank you guys! > Vlad Rusu > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gabster at lelutin.ca Thu Oct 15 22:37:51 2015 From: gabster at lelutin.ca (Gabriel Filion) Date: Thu, 15 Oct 2015 18:37:51 -0400 Subject: Graphing number of 503s In-Reply-To: References: <561F2C67.30702@lelutin.ca> Message-ID: <56202ABF.70908@lelutin.ca> On 15/10/15 03:00 PM, Per Buer wrote: > I'd like to create a graph of how much 503 pages were served by varnish. > Is there an easy way to count how much such error pages are sent to > clients? > > > Sure. Throw Google Analytics at it and you'll have tons of graphs. Just > throw the GA stuff into the guru meditation page (Artur used this for > all sort of stuff). ah hmm actually this is not possible in this case since the varnish servers are used in front of shared hosting for a couple of select sites and we don't really control the contents of the sites. Guillaume's suggestion of using varnishncsa is a possibility that I can explore. While brainstorming with co-workers, we've had another idea too: since we're serving error pages as static html on one special backend we could inject a header in the request made to that server to identify the backend used in the original client request and thus count errors per backend. -- Gabriel Filion -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: OpenPGP digital signature URL: From gabster at lelutin.ca Thu Oct 15 22:41:53 2015 From: gabster at lelutin.ca (Gabriel Filion) Date: Thu, 15 Oct 2015 18:41:53 -0400 Subject: 503s vs grace period [was: Re: 503s and FetchError - V3.0.7] In-Reply-To: References: <7CBC14D5-F56E-4FCB-A2DD-6F8B67FF0664@cloudtroopers.ro> Message-ID: <56202BB1.7080802@lelutin.ca> sorry to diverge a bit from the original question: On 15/10/15 06:26 PM, Jason Price wrote: > If all backends in a director are unhealthy, varnish will serve an > immediate 503 if the request goes to that director. if objects are kept with a grace time, doesn't varnish send the stale objects instead of serving a 503? -- Gabriel Filion -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: OpenPGP digital signature URL: From jdh132 at psu.edu Thu Oct 15 22:46:42 2015 From: jdh132 at psu.edu (Jason Heffner) Date: Thu, 15 Oct 2015 18:46:42 -0400 Subject: Graphing number of 503s In-Reply-To: <56202ABF.70908@lelutin.ca> References: <561F2C67.30702@lelutin.ca> <56202ABF.70908@lelutin.ca> Message-ID: <08121667-28E4-41AC-BF6B-1A28266425A7@psu.edu> I?d use varnishncsa as suggested below.. for v3 - returns any 429, 500, or 503 from varnish or front-end in log DAEMON_OPTS="-a -f -w $logfile -D -m TxStatus:\"429|500|503\" -P $pidfile? for v4 - returns any 429 only from varnish, and any above 500 to log DAEMON_OPTS="-a -F '%{X-Forwarded-For}i %l %u %t "%r" %s %b "%{Referer}i" "%{User-agent}i"' -w $logfile -D -q \"RespStatus >= 429 or BerespStatus >= 500\" -P $pidfile" Some common use examples I was using. Jason > On Oct 15, 2015, at 6:37 PM, Gabriel Filion wrote: > > On 15/10/15 03:00 PM, Per Buer wrote: >> I'd like to create a graph of how much 503 pages were served by varnish. >> Is there an easy way to count how much such error pages are sent to >> clients? >> >> >> Sure. Throw Google Analytics at it and you'll have tons of graphs. Just >> throw the GA stuff into the guru meditation page (Artur used this for >> all sort of stuff). > > ah hmm actually this is not possible in this case since the varnish > servers are used in front of shared hosting for a couple of select sites > and we don't really control the contents of the sites. > > Guillaume's suggestion of using varnishncsa is a possibility that I can > explore. > > While brainstorming with co-workers, we've had another idea too: since > we're serving error pages as static html on one special backend we could > inject a header in the request made to that server to identify the > backend used in the original client request and thus count errors per > backend. > > -- > Gabriel Filion > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc From jdh132 at psu.edu Fri Oct 16 02:04:16 2015 From: jdh132 at psu.edu (Jason Heffner) Date: Thu, 15 Oct 2015 22:04:16 -0400 Subject: Even more varnish 4.1 issues In-Reply-To: References: <8C92F9AC-5E96-4C9F-91B0-6999AD2D52F8@psu.edu> Message-ID: <00F8204B-75DE-4EB6-A741-A86ED9E69AA7@psu.edu> It took me a few tries but finally got 4.1.0 to work as expected. As it turns out I use Nginx as a reverse proxy in development to simulate our F5 load balancer on staging/production. I had to upgrade the HTTP proxy version to 1.1 in nginx and it just started working. Nginx 1.8.0 defaults to HTTP 1.0 which should have been fine according to the varnish docs. I was able to recreate through Google Chrome by setting it back to HTTP version 1.0. I was mostly interested in the HTTP range header fixes which I confirmed are fixed in 4.1 now. Yes! Wouldn?t mind seeing those back ported. I removed the custom selinux policy I created for v 4.0.3 which required some additions and confirmed the current selinux targeted default policy (v24) for varnishd is working on RHEL6.7 with 4.1.0. Jason > On Oct 15, 2015, at 7:59 PM, Jason Heffner wrote: > > It's not SELinux. Every time you disable SELinux you make Dan Walsh cry. > > Sent from my iPhone > > On Oct 15, 2015, at 7:36 PM, nick tailor > wrote: > >> Disable selinux. >> >> Permissive can still cause issues >> >> On Oct 15, 2015 3:03 PM, "Jason Heffner" > wrote: >> I have SELinux in permissive mode, I uninstalled varnish 4.0.3 rpms and installed new varnish 4.1 rpms. I updated the /etc/sysconfig/varnish and varnish does start successfully. >> >> I can load any static files fine through varnish with the default config, and my own config. When I try and load a page that sets a cookie I will get the first response to the page fine, and request for a second page will return a 0 length content and white screen. I clear cookies and I can load another page till one page sets cookies and I?m back to the 0 length response. It?s really odd. >> >> Anyone have any idea what I may be missing? I disabled the only mod I was using as well. >> >> Thanks, >> Jason >> >> Systems Administrator >> Teaching and Learning with Technology, Information Technology Services >> The Pennsylvania State University >> >> >> >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc -------------- next part -------------- An HTML attachment was scrubbed... URL: From geoff at uplex.de Fri Oct 16 06:26:18 2015 From: geoff at uplex.de (Geoff Simmons) Date: Fri, 16 Oct 2015 08:26:18 +0200 Subject: Child said getnameinfo = -11 System error In-Reply-To: References: Message-ID: <5620988A.20402@uplex.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 10/14/2015 10:57 PM, Hugues Alary wrote: > > - Error out of workspace - LostHeader > X-Forwarded-For: - ReqUnset X-Forwarded-For: 80.68.74.90 - > ReqUnset X-Forwarded-For: 162.158.92.219 - Error > out of workspace > > It seems that varnish is running out of workspace. Some googling > tells me it could be related to ESI includes. It so happen that I > did change something recently with ESI includes, and, I happen to > have pages loading something like 100 ESI includes (maybe even > more). > > I read that someone else using 4.0.3 was running out of workspace, > despite setting a value up to 16mb. Maybe I should tune my > workspace, but the bug report stated that raising the value to 16mb > ended up eating up all the memory of the system and thus wasn't > really an option. It sounds like you were looking at my thread from a while back. As it turned out, the problem was due to our own error in VCL, as I'll explain below, and unless you're making the same mistake, you're probably not having the same problem. At any rate, requests and all of their ESI subrequests use the same workspace, so if you really have up to 100 ESIs, your workspace might simply not be large enough, so you really should try increasing workspace_client. Our mistake was that, under certain error conditions, a request would restart to another URL that shows a custom error message, which also has ESI includes. The ESIs within the restart would encounter the same error, then restart to the same URL, and so on into endless recursion. max_esi_depth didn't stop the madness soon enough, because the ESI tree was expanding in breadth as well as depth. Getting VCL to notice this was happening (by checking req.esi_level) solved the problem, and we could set the workspace sizes back down to their previous values. If you don't have something crazy going on like that, then as I said, you might just have the straightforward problem that your workspaces are too small for the 100 ESIs. HTH, Geoff - -- ** * * UPLEX - Nils Goroll Systemoptimierung Scheffelstra?e 32 22301 Hamburg Tel +49 40 2880 5731 Mob +49 176 636 90917 Fax +49 40 42949753 http://uplex.de -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBCAAGBQJWIJh/AAoJEOUwvh9pJNURbOsP/0m4+E0eAWQJNkzWApMWFb0y 5T6Uat/9GmHP98DJpYVvi2e7gcIDC40PHc/EG0gM9CJ2Leatb49ujBm8MQZf47pO MqxD8RziaWTKOtfK34TPQ01PSvmhP+IDhUTAU+GPmUn8ZeS7ABVpuAXJ5+lwnZZZ bgFN6oyeTkwPEeCMb2sDcP/SnAoTZwCstEZgQ+JD8Vubz7iMzB/wDJ1jeOwwXpIw Mn7sZMe3H/3RiqyFu534IHqRvvjP/hGuwKDBiUXJVCHesT7enBOLyOMZB8eIRx/U kGuZst9Ecx2pJz0lE3u+OM5v6796nSCP4Vs9AipNF0xN8PUISWoEKJqcX85pmklG Fsygtc9GkEqTMsJsPeus7381z0gbkqnzyVhekdT93JjP1l3XigneZwbU9hPX3bxe EmCru+/56As4TYGLL+IDcNvToqIfPbf8L3U4xl6XZeCA5NhSQkZRXwbY1uPCAosK nrPQTTQZqi6Q22jYtiUfJ0kwua0xhJIYGkPJbD2HMgiS2QczrN50Q7py4J5O6xzr 7TTvQcJ+kizq3fKKBt9tCo56LYR2Cocrv9zxpFlQp7zVjzEaj/5pqYc8KQC2LHvm xdJ/g08Rf97A/va+uxx9TWsFcUBHMab0+3XRfYum4mD9ijkhxc46T5cfdD6cOwlF GC0U2fS3vYRy+94iqls5 =tRLF -----END PGP SIGNATURE----- From hugues at betabrand.com Fri Oct 16 17:24:13 2015 From: hugues at betabrand.com (Hugues Alary) Date: Fri, 16 Oct 2015 10:24:13 -0700 Subject: Child said getnameinfo = -11 System error In-Reply-To: <5620988A.20402@uplex.de> References: <5620988A.20402@uplex.de> Message-ID: Hi Geoff, It was indeed your thread. I've been tracking so many bugs recently that I forgot that you had mentioned the problem was coming from your configuration and not varnish. I'm glad you saw my email on the list and answered -I'm just going to increase my workspace_client. I'll report back if that solved my issue. Thanks for the help! -Hugues On Thu, Oct 15, 2015 at 11:26 PM, Geoff Simmons wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > On 10/14/2015 10:57 PM, Hugues Alary wrote: > > > > - Error out of workspace - LostHeader > > X-Forwarded-For: - ReqUnset X-Forwarded-For: 80.68.74.90 - > > ReqUnset X-Forwarded-For: 162.158.92.219 - Error > > out of workspace > > > > It seems that varnish is running out of workspace. Some googling > > tells me it could be related to ESI includes. It so happen that I > > did change something recently with ESI includes, and, I happen to > > have pages loading something like 100 ESI includes (maybe even > > more). > > > > I read that someone else using 4.0.3 was running out of workspace, > > despite setting a value up to 16mb. Maybe I should tune my > > workspace, but the bug report stated that raising the value to 16mb > > ended up eating up all the memory of the system and thus wasn't > > really an option. > > It sounds like you were looking at my thread from a while back. As it > turned out, the problem was due to our own error in VCL, as I'll > explain below, and unless you're making the same mistake, you're > probably not having the same problem. > > At any rate, requests and all of their ESI subrequests use the same > workspace, so if you really have up to 100 ESIs, your workspace might > simply not be large enough, so you really should try increasing > workspace_client. > > Our mistake was that, under certain error conditions, a request would > restart to another URL that shows a custom error message, which also > has ESI includes. The ESIs within the restart would encounter the same > error, then restart to the same URL, and so on into endless recursion. > max_esi_depth didn't stop the madness soon enough, because the ESI > tree was expanding in breadth as well as depth. > > Getting VCL to notice this was happening (by checking req.esi_level) > solved the problem, and we could set the workspace sizes back down to > their previous values. > > If you don't have something crazy going on like that, then as I said, > you might just have the straightforward problem that your workspaces > are too small for the 100 ESIs. > > > HTH, > Geoff > - -- > ** * * UPLEX - Nils Goroll Systemoptimierung > > Scheffelstra?e 32 > 22301 Hamburg > > Tel +49 40 2880 5731 > Mob +49 176 636 90917 > Fax +49 40 42949753 > > http://uplex.de > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1 > > iQIcBAEBCAAGBQJWIJh/AAoJEOUwvh9pJNURbOsP/0m4+E0eAWQJNkzWApMWFb0y > 5T6Uat/9GmHP98DJpYVvi2e7gcIDC40PHc/EG0gM9CJ2Leatb49ujBm8MQZf47pO > MqxD8RziaWTKOtfK34TPQ01PSvmhP+IDhUTAU+GPmUn8ZeS7ABVpuAXJ5+lwnZZZ > bgFN6oyeTkwPEeCMb2sDcP/SnAoTZwCstEZgQ+JD8Vubz7iMzB/wDJ1jeOwwXpIw > Mn7sZMe3H/3RiqyFu534IHqRvvjP/hGuwKDBiUXJVCHesT7enBOLyOMZB8eIRx/U > kGuZst9Ecx2pJz0lE3u+OM5v6796nSCP4Vs9AipNF0xN8PUISWoEKJqcX85pmklG > Fsygtc9GkEqTMsJsPeus7381z0gbkqnzyVhekdT93JjP1l3XigneZwbU9hPX3bxe > EmCru+/56As4TYGLL+IDcNvToqIfPbf8L3U4xl6XZeCA5NhSQkZRXwbY1uPCAosK > nrPQTTQZqi6Q22jYtiUfJ0kwua0xhJIYGkPJbD2HMgiS2QczrN50Q7py4J5O6xzr > 7TTvQcJ+kizq3fKKBt9tCo56LYR2Cocrv9zxpFlQp7zVjzEaj/5pqYc8KQC2LHvm > xdJ/g08Rf97A/va+uxx9TWsFcUBHMab0+3XRfYum4mD9ijkhxc46T5cfdD6cOwlF > GC0U2fS3vYRy+94iqls5 > =tRLF > -----END PGP SIGNATURE----- > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hugues at betabrand.com Fri Oct 16 17:50:06 2015 From: hugues at betabrand.com (Hugues Alary) Date: Fri, 16 Oct 2015 10:50:06 -0700 Subject: Child said getnameinfo = -11 System error In-Reply-To: References: <5620988A.20402@uplex.de> Message-ID: Just changed my workspace_client from 4k to 256k. The problem immediately went away. I might have to tune this value to find the smallest possible value I could use. Thanks! On Fri, Oct 16, 2015 at 10:24 AM, Hugues Alary wrote: > Hi Geoff, > > It was indeed your thread. > > I've been tracking so many bugs recently that I forgot that you had > mentioned the problem was coming from your configuration and not varnish. > > I'm glad you saw my email on the list and answered -I'm just going to > increase my workspace_client. > > I'll report back if that solved my issue. > > Thanks for the help! > -Hugues > > > > > > > On Thu, Oct 15, 2015 at 11:26 PM, Geoff Simmons wrote: > >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA256 >> >> On 10/14/2015 10:57 PM, Hugues Alary wrote: >> > >> > - Error out of workspace - LostHeader >> > X-Forwarded-For: - ReqUnset X-Forwarded-For: 80.68.74.90 - >> > ReqUnset X-Forwarded-For: 162.158.92.219 - Error >> > out of workspace >> > >> > It seems that varnish is running out of workspace. Some googling >> > tells me it could be related to ESI includes. It so happen that I >> > did change something recently with ESI includes, and, I happen to >> > have pages loading something like 100 ESI includes (maybe even >> > more). >> > >> > I read that someone else using 4.0.3 was running out of workspace, >> > despite setting a value up to 16mb. Maybe I should tune my >> > workspace, but the bug report stated that raising the value to 16mb >> > ended up eating up all the memory of the system and thus wasn't >> > really an option. >> >> It sounds like you were looking at my thread from a while back. As it >> turned out, the problem was due to our own error in VCL, as I'll >> explain below, and unless you're making the same mistake, you're >> probably not having the same problem. >> >> At any rate, requests and all of their ESI subrequests use the same >> workspace, so if you really have up to 100 ESIs, your workspace might >> simply not be large enough, so you really should try increasing >> workspace_client. >> >> Our mistake was that, under certain error conditions, a request would >> restart to another URL that shows a custom error message, which also >> has ESI includes. The ESIs within the restart would encounter the same >> error, then restart to the same URL, and so on into endless recursion. >> max_esi_depth didn't stop the madness soon enough, because the ESI >> tree was expanding in breadth as well as depth. >> >> Getting VCL to notice this was happening (by checking req.esi_level) >> solved the problem, and we could set the workspace sizes back down to >> their previous values. >> >> If you don't have something crazy going on like that, then as I said, >> you might just have the straightforward problem that your workspaces >> are too small for the 100 ESIs. >> >> >> HTH, >> Geoff >> - -- >> ** * * UPLEX - Nils Goroll Systemoptimierung >> >> Scheffelstra?e 32 >> 22301 Hamburg >> >> Tel +49 40 2880 5731 >> Mob +49 176 636 90917 >> Fax +49 40 42949753 >> >> http://uplex.de >> -----BEGIN PGP SIGNATURE----- >> Version: GnuPG v1 >> >> iQIcBAEBCAAGBQJWIJh/AAoJEOUwvh9pJNURbOsP/0m4+E0eAWQJNkzWApMWFb0y >> 5T6Uat/9GmHP98DJpYVvi2e7gcIDC40PHc/EG0gM9CJ2Leatb49ujBm8MQZf47pO >> MqxD8RziaWTKOtfK34TPQ01PSvmhP+IDhUTAU+GPmUn8ZeS7ABVpuAXJ5+lwnZZZ >> bgFN6oyeTkwPEeCMb2sDcP/SnAoTZwCstEZgQ+JD8Vubz7iMzB/wDJ1jeOwwXpIw >> Mn7sZMe3H/3RiqyFu534IHqRvvjP/hGuwKDBiUXJVCHesT7enBOLyOMZB8eIRx/U >> kGuZst9Ecx2pJz0lE3u+OM5v6796nSCP4Vs9AipNF0xN8PUISWoEKJqcX85pmklG >> Fsygtc9GkEqTMsJsPeus7381z0gbkqnzyVhekdT93JjP1l3XigneZwbU9hPX3bxe >> EmCru+/56As4TYGLL+IDcNvToqIfPbf8L3U4xl6XZeCA5NhSQkZRXwbY1uPCAosK >> nrPQTTQZqi6Q22jYtiUfJ0kwua0xhJIYGkPJbD2HMgiS2QczrN50Q7py4J5O6xzr >> 7TTvQcJ+kizq3fKKBt9tCo56LYR2Cocrv9zxpFlQp7zVjzEaj/5pqYc8KQC2LHvm >> xdJ/g08Rf97A/va+uxx9TWsFcUBHMab0+3XRfYum4mD9ijkhxc46T5cfdD6cOwlF >> GC0U2fS3vYRy+94iqls5 >> =tRLF >> -----END PGP SIGNATURE----- >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From de.techno at gmail.com Sat Oct 17 09:13:19 2015 From: de.techno at gmail.com (dE) Date: Sat, 17 Oct 2015 14:43:19 +0530 Subject: Minimalistic VCL does not compile In-Reply-To: <20151015083200.GN2656@nerd.dk> References: <561E3615.8030508@gmail.com> <20151014130826.GM2656@nerd.dk> <561F5CAB.9040905@gmail.com> <20151015083200.GN2656@nerd.dk> Message-ID: <5622112F.60107@gmail.com> The definition of a definition has not been provided anywhere. Will be glad if it can be put up in the book or man page. Thanks for the help and clearing the concepts up! On 10/15/15 14:02, Andreas Plesner wrote: > On Thu, Oct 15, 2015 at 01:28:35PM +0530, dE wrote: > >> Thanks everyone for the solution. >> >> I'm using varnish 3 (Varnish 4 doesn't compile). One thing I don't >> get is why doesn't backend section variables (.host/.port etc...) >> does not need a 'set' while others do? > Because that's a definition, not code. > From de.techno at gmail.com Sat Oct 17 10:46:28 2015 From: de.techno at gmail.com (dE) Date: Sat, 17 Oct 2015 16:16:28 +0530 Subject: Unexpected response from varnish (3) Message-ID: <56222704.1020403@gmail.com> I created this 'odd' varnish configuration for educational purposes -- backend default { .host = "neuro.debian.net"; .port = "http"; } sub vcl_recv { set req.backend = default; set req.http.host = "neuro.debian.net"; return(lookup); } sub vcl_miss { set bereq.http.host = "example.com"; } sub vcl_hit { error 200 "returned from cache"; } sub vcl_fetch { set beresp.status = 500; return (hit_for_pass); } Ideally it must only fetch from the backend and it must return Apache's 'It works!' every time since neuro.debian.net is configured to respond properly only if the hostname is set to neuro.debian.net, otherwise 'It works!'. There must never be a cache hit. But the expected happens only once on the first request. But then afterward, the real website of neuro.debian.net in a crippled way, and sometimes there is just a blank page. Thanks for any help! From miguel_3_gonzalez at yahoo.es Sun Oct 18 10:57:55 2015 From: miguel_3_gonzalez at yahoo.es (=?UTF-8?Q?Miguel_Gonz=c3=a1lez?=) Date: Sun, 18 Oct 2015 12:57:55 +0200 Subject: varnish 4 caching post requests Message-ID: <56237B33.9030904@yahoo.es> Dear all, I have Varnish 4 in front of an Apache (Cpanel) hosting several websites running Wordpress. Everything works fine, but there are Wordpress plugins that perform POST resquests. I?ve seen various messages that in previous versions of Varnish you couldn?t cache POST responses (which you can do with Nginx). I don?t want to overcomplicate things and put Nginx between Varnish and Apache (I don?t want to miss the security side that Apache offers) and I was wondering if that has been solved in the newers versions of Varnish. Regards, Miguel From de.techno at gmail.com Mon Oct 19 06:01:52 2015 From: de.techno at gmail.com (dE) Date: Mon, 19 Oct 2015 11:31:52 +0530 Subject: varnish 4 caching post requests In-Reply-To: <56237B33.9030904@yahoo.es> References: <56237B33.9030904@yahoo.es> Message-ID: <56248750.10304@gmail.com> No, it's just that in the default configuration, varnish will not cache POST request. In Varnish 3 -- if (req.request != "GET" && req.request != "HEAD" && req.request != "PUT" && req.request != "POST" && req.request != "TRACE" && req.request != "OPTIONS" && req.request != "DELETE") { /* Non-RFC2616 or CONNECT which is weird. */ return (pipe); } This's the default config. On 10/18/15 16:27, Miguel Gonz?lez wrote: > Dear all, > > I have Varnish 4 in front of an Apache (Cpanel) hosting several > websites running Wordpress. > > Everything works fine, but there are Wordpress plugins that perform > POST resquests. I?ve seen various messages that in previous versions of > Varnish you couldn?t cache POST responses (which you can do with Nginx). > > I don?t want to overcomplicate things and put Nginx between Varnish > and Apache (I don?t want to miss the security side that Apache offers) > and I was wondering if that has been solved in the newers versions of > Varnish. > > Regards, > > Miguel > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc -------------- next part -------------- An HTML attachment was scrubbed... URL: From wido at widodh.nl Mon Oct 19 07:40:50 2015 From: wido at widodh.nl (Wido den Hollander) Date: Mon, 19 Oct 2015 09:40:50 +0200 Subject: varnish 4 caching post requests In-Reply-To: <56237B33.9030904@yahoo.es> References: <56237B33.9030904@yahoo.es> Message-ID: <56249E82.6040100@widodh.nl> On 18-10-15 12:57, Miguel Gonz?lez wrote: > Dear all, > > I have Varnish 4 in front of an Apache (Cpanel) hosting several > websites running Wordpress. > > Everything works fine, but there are Wordpress plugins that perform > POST resquests. I?ve seen various messages that in previous versions of > Varnish you couldn?t cache POST responses (which you can do with Nginx). > > I don?t want to overcomplicate things and put Nginx between Varnish > and Apache (I don?t want to miss the security side that Apache offers) > and I was wondering if that has been solved in the newers versions of > Varnish. > I haven't used it, but it might be that vmod can do it for you? std.cache_req_body(32MB); https://www.varnish-cache.org/docs/trunk/reference/vmod_std.generated.html#func-cache-req-body The docs say: "Caching the req.body makes it possible to retry pass operations (POST, PUT)." So it seems you can only use it for restarting/retrying requests, but you can always give it a try. Wido > Regards, > > Miguel > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > From jla at fcoo.dk Wed Oct 21 22:29:44 2015 From: jla at fcoo.dk (Jesper Larsen) Date: Wed, 21 Oct 2015 22:29:44 +0000 Subject: Long URL's are truncated Message-ID: <15E71B0ACFC92F4DA016DFBAB512295770241C94@mail01.fcoo.dk> Hi Varnish people (Warning: new to Varnish:-) I am using varnishtop (varnish 4.0.3 on Ubuntu) to figure out whether my Varnish setup caches my stuff properly. The backend server is however a web service for which the URL's to the resources are very long (> 256 characters). And many of the requests are actually identical at least up to the first 256 characters. When I run: varnishtop -i BereqURL -1 it does however seem like it is only showing the first 254 characters (or so) of the URL's and it does in fact also seem like the URL's which are identical up to this number of characters are "lumped" together increasing the count of backend requests for that particular resource. This makes it hard to see if requests are in fact cached properly. The same issue seems to affect at least varnishncsa (I guess it comes from varnishlog). Is there any way to make varnishtop use the entire URL? Best regards, Jesper From raymond.jennings.iii at gmail.com Thu Oct 22 01:51:22 2015 From: raymond.jennings.iii at gmail.com (Raymond Jennings III) Date: Wed, 21 Oct 2015 21:51:22 -0400 Subject: Long URL's are truncated In-Reply-To: <15E71B0ACFC92F4DA016DFBAB512295770241C94@mail01.fcoo.dk> References: <15E71B0ACFC92F4DA016DFBAB512295770241C94@mail01.fcoo.dk> Message-ID: I believe you need to start varnishd with the following: -p shm_reclen=1024 or whatever value works for you. Depending on how you have it setup you might have it set in /etc/sysconfig/varnish with other startup parameters. On Wed, Oct 21, 2015 at 6:29 PM, Jesper Larsen wrote: > Hi Varnish people > > (Warning: new to Varnish:-) > > I am using varnishtop (varnish 4.0.3 on Ubuntu) to figure out whether my > Varnish setup caches my stuff properly. The backend server is however a web > service for which the URL's to the resources are very long (> 256 > characters). And many of the requests are actually identical at least up to > the first 256 characters. > > When I run: > > varnishtop -i BereqURL -1 > > it does however seem like it is only showing the first 254 characters (or > so) of the URL's and it does in fact also seem like the URL's which are > identical up to this number of characters are "lumped" together increasing > the count of backend requests for that particular resource. This makes it > hard to see if requests are in fact cached properly. The same issue seems > to affect at least varnishncsa (I guess it comes from varnishlog). > > Is there any way to make varnishtop use the entire URL? > > Best regards, > Jesper > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jla at fcoo.dk Thu Oct 22 07:56:48 2015 From: jla at fcoo.dk (Jesper Larsen) Date: Thu, 22 Oct 2015 07:56:48 +0000 Subject: SV: Long URL's are truncated In-Reply-To: References: <15E71B0ACFC92F4DA016DFBAB512295770241C94@mail01.fcoo.dk>, Message-ID: <15E71B0ACFC92F4DA016DFBAB512295770241D20@mail01.fcoo.dk> Hi Raymond > I believe you need to start varnishd with the following: -p shm_reclen=1024 or whatever value works for you. > > Depending on how you have it setup you might have it set in /etc/sysconfig/varnish with other startup parameters. Thanks, that solved my issues (the config file on Ubuntu is /etc/default/varnish). Regards, Jesper > When I run: > > varnishtop -i BereqURL -1 > > it does however seem like it is only showing the first 254 characters (or so) of the URL's and it does in fact also seem like the > URL's which are identical up to this number of characters are "lumped" together increasing the count of backend requests for > that particular resource. This makes it hard to see if requests are in fact cached properly. The same issue seems to affect at > least varnishncsa (I guess it comes from varnishlog). > > Is there any way to make varnishtop use the entire URL? From leiwang at rhapsody.com Fri Oct 23 18:23:03 2015 From: leiwang at rhapsody.com (Lei Wang) Date: Fri, 23 Oct 2015 11:23:03 -0700 Subject: expired objects are not removed quickly enough Message-ID: Varnish is used in one of our highest traffic API cache internally. It all worked fine for quite a long time. But recently I noticed one strange behavior that the cached objects number continue up (to about 50M objects) until Varnish ran out memory (configured to use 128GB per server). The only way to recover is restart Varnish. But few days late the same thing happened again. I noticed that maximum cache_miss+n_expired is quite stable at about 875 HPS no matter what is the request traffic. So I guess that if average cache_miss is more than half the 875 HPS, then the total objects number will keep added up since the expired objects cannot be removed quickly enough to free the space. Questions: 1. Is there any Varnish internal limit for cache_miss+n_expired. Can it be adjusted with run time value? 2. If the hard limit is there and cannot be adjusted, how do I change other configuration to avoid this? Varnish version below: -logbash-3.2$ varnishd -V varnishd (varnish-3.0.4 revision 9f83e8f) Copyright (c) 2006 Verdens Gang AS Copyright (c) 2006-2011 Varnish Software AS Thanks, Lei -------------- next part -------------- An HTML attachment was scrubbed... URL: From ross at trademe.co.nz Fri Oct 23 18:59:22 2015 From: ross at trademe.co.nz (Ross Brown) Date: Fri, 23 Oct 2015 18:59:22 +0000 Subject: expired objects are not removed quickly enough In-Reply-To: References: Message-ID: <29e0fc58aa7c4a17a471f6da2ebbc2d2@TM-WCE-EXCH1.trademe.local> Varnish 3 is considered end of life, you should really upgrade to Varnish 4. https://www.varnish-cache.org/trac/browser/doc/changes.rst From: varnish-misc-bounces+ross=trademe.co.nz at varnish-cache.org [mailto:varnish-misc-bounces+ross=trademe.co.nz at varnish-cache.org] On Behalf Of Lei Wang Sent: Saturday, 24 October 2015 7:23 a.m. To: varnish-misc at varnish-cache.org Subject: expired objects are not removed quickly enough Varnish is used in one of our highest traffic API cache internally. It all worked fine for quite a long time. But recently I noticed one strange behavior that the cached objects number continue up (to about 50M objects) until Varnish ran out memory (configured to use 128GB per server). The only way to recover is restart Varnish. But few days late the same thing happened again. I noticed that maximum cache_miss+n_expired is quite stable at about 875 HPS no matter what is the request traffic. So I guess that if average cache_miss is more than half the 875 HPS, then the total objects number will keep added up since the expired objects cannot be removed quickly enough to free the space. Questions: 1. Is there any Varnish internal limit for cache_miss+n_expired. Can it be adjusted with run time value? 2. If the hard limit is there and cannot be adjusted, how do I change other configuration to avoid this? Varnish version below: -logbash-3.2$ varnishd -V varnishd (varnish-3.0.4 revision 9f83e8f) Copyright (c) 2006 Verdens Gang AS Copyright (c) 2006-2011 Varnish Software AS Thanks, Lei From japrice at gmail.com Mon Oct 26 19:12:04 2015 From: japrice at gmail.com (Jason Price) Date: Mon, 26 Oct 2015 15:12:04 -0400 Subject: varnish eating (unset'ing) the Cookie? Message-ID: varnish version 3.0.7 (Yes, I know it's EOL. Hopefully we can get to varnish4 this quarter) OS: Centos 7.1 No VMODs other than std. Varnish is receiving a set cookie, verified by seeing "RxHeader c Cookie" in varnishlog. However, looking at "req.http.Cookie" in vcl_recv shows it as non-existent. Any ideas? Relevant VCL: sub vcl_recv { std.log("entry point cookie is " + req.http.Cookie); std.log("entry point user agent is " + req.http.User-Agent); if (req.url == "/heartbeat") { error 200 "OK"; } std.syslog(1, req.url + " checking greencookie"); call validate_greencookie; if (req.restarts == 0) { if (req.http.x-forwarded-for) { set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip; } else { set req.http.X-Forwarded-For = client.ip; } } call set_backend; return (pass); #router never looksup } sub validate_greencookie { # only the akamai cookie shall pass std.log("cookie is " + req.http.Cookie); if (req.http.Cookie ~ "^.*[REDACTED_COOKIE]" ) { } else { error 401 "Unauthorized"; } } Relevant VarnishLog output: 4 SessionOpen c 10.255.196.4 59771 :8080 4 ReqStart c 10.255.196.4 59771 1143318669 4 RxRequest c GET 4 RxURL c [NOT RELEVANT] 4 RxProtocol c HTTP/1.1 4 RxHeader c Host: [NOT RELEVANT] 4 RxHeader c User-Agent: curl/7.43.0 4 RxHeader c Accept: */* 4 RxHeader c Cookie : [REDACTED_COOKIE] 4 VCL_call c recv 4 VCL_Log c entry point cookie is 4 VCL_Log c entry point user agent is curl/7.43.0 4 VCL_Log c cookie is 4 VCL_return c error -------------- next part -------------- An HTML attachment was scrubbed... URL: From info at massivescale.net Tue Oct 27 09:39:18 2015 From: info at massivescale.net (Andrzej Godziuk) Date: Tue, 27 Oct 2015 10:39:18 +0100 Subject: varnish eating (unset'ing) the Cookie? In-Reply-To: References: Message-ID: <20151027103918.1019c9a7@gdr-desktop.gdr.name> Are you sending the Cookie header manually? You have an unnecessary space here: Cookie : It should say: Cookie: [REDACTED_COOKIE] -- Andrzej Godziuk From feld at feld.me Fri Oct 30 16:42:19 2015 From: feld at feld.me (Mark Felder) Date: Fri, 30 Oct 2015 11:42:19 -0500 Subject: varnishncsa -- please support a config file Message-ID: <1446223339.1984798.424778313.4C3BB196@webmail.messagingengine.com> Hi all, Can we get config file support for varnishncsa? (possible varnishlog, too I suppose) In FreeBSD we're fighting with a shell quote nightmare. We permit users to set the varnishncsa log file format in /etc/rc.conf via: varnishncsa_logformat="" I've had to change the way we drop privs from the rc.subr system to a custom one based on our daemon(8) utility to successfully pass the entire string to varnishncsa but we still can't get quotes passed through. Escaping them isn't working either. varnishncsa_logformat="%h %l %u %t \"%r\" %s %b \"%{Referer}i\" \"%{User-agent}i\" (%{Varnish:handling}x)" # ps ax | grep [n]csa 94729 ?? SsJ 0:00.01 /usr/local/bin/varnishncsa -t off -P /var/run/varnishncsa.pid -D -a -w /data/log/http/varnish_access.log -F %h %l %u %t %r %s %b %{Referer}i %{User-agent}i (%{Varnish:handling}x) The quotes end up missing which users (and log parsers) expect to be there. :( This is not an easy problem to solve. We could probably fix /etc/rc.subr in FreeBSD but the reality is that it will not be usable by current users. We will have to wait for the update to trickle down into newer releases and older releases to be EoL. (so around 2017) If we could read settings from a config file it would solve this problem cleanly... Thanks -- Mark Felder feld at feld.me From raymond.jennings.iii at gmail.com Fri Oct 30 16:54:15 2015 From: raymond.jennings.iii at gmail.com (Raymond Jennings III) Date: Fri, 30 Oct 2015 12:54:15 -0400 Subject: varnishncsa -- please support a config file In-Reply-To: <1446223339.1984798.424778313.4C3BB196@webmail.messagingengine.com> References: <1446223339.1984798.424778313.4C3BB196@webmail.messagingengine.com> Message-ID: I'll second that request. On Fri, Oct 30, 2015 at 12:42 PM, Mark Felder wrote: > Hi all, > > Can we get config file support for varnishncsa? (possible varnishlog, > too I suppose) > > In FreeBSD we're fighting with a shell quote nightmare. We permit users > to set the varnishncsa log file format in /etc/rc.conf via: > > varnishncsa_logformat="" > > I've had to change the way we drop privs from the rc.subr system to a > custom one based on our daemon(8) utility to successfully pass the > entire string to varnishncsa but we still can't get quotes passed > through. Escaping them isn't working either. > > varnishncsa_logformat="%h %l %u %t \"%r\" %s %b \"%{Referer}i\" > \"%{User-agent}i\" (%{Varnish:handling}x)" > > # ps ax | grep [n]csa > 94729 ?? SsJ 0:00.01 /usr/local/bin/varnishncsa -t off -P > /var/run/varnishncsa.pid -D -a -w /data/log/http/varnish_access.log -F > %h %l %u %t %r %s %b %{Referer}i %{User-agent}i (%{Varnish:handling}x) > > The quotes end up missing which users (and log parsers) expect to be > there. :( > > This is not an easy problem to solve. We could probably fix /etc/rc.subr > in FreeBSD but the reality is that it will not be usable by current > users. We will have to wait for the update to trickle down into newer > releases and older releases to be EoL. (so around 2017) > > If we could read settings from a config file it would solve this problem > cleanly... > > > Thanks > > -- > Mark Felder > feld at feld.me > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: