From lagged at gmail.com Sat Oct 1 07:08:30 2016 From: lagged at gmail.com (Andrei) Date: Sat, 1 Oct 2016 10:08:30 +0300 Subject: Round Robin Load Balancing in Varnish In-Reply-To: References: Message-ID: Have a look over the docs https://www.varnish-cache.org/docs/4.1/users-guide/vcl-backends.html On Sep 30, 2016 19:17, "Debraj Manna" wrote: > I am on varnish 4.1 > > On Fri, Sep 30, 2016 at 9:28 PM, Debraj Manna > wrote: > >> Hi >> >> Let's say I am having four backends b1, b2, b3 & b4 added in round robin. >> If one backend(say b1 ) is not healthy, will varnish take out b1 from >> backend pool & send the incoming requests only to b2, b3 or b4 or it will >> keep on doing round robin among all the four servers with the requests >> going to b1 failing? >> > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Tom.Anheyer at Berlinonline.de Tue Oct 4 11:31:44 2016 From: Tom.Anheyer at Berlinonline.de (Tom Anheyer) Date: Tue, 4 Oct 2016 13:31:44 +0200 Subject: http/2 first experiences In-Reply-To: References: Message-ID: <2999a459-37b9-90be-8a35-f8cd33b92da8@Berlinonline.de> Hello, I've setup a little test environment with varnish5 and hitch as TLS offloader. HTTP/2 works for me in FF and Chrome (after upgrading to openssl 1.0.2). Which H2 features are supported so far? How to do server push? If not supported yet - Is it planned? One crash so far. tom Oct 04 11:26:36 varnish varnishd[25116]: Child cleanup complete Oct 04 11:26:36 varnish varnishd[25116]: " Oct 04 11:26:36 varnish varnishd[25116]: }, Oct 04 11:26:36 varnish varnishd[25116]: }, Oct 04 11:26:36 varnish varnishd[25116]: flags = { Oct 04 11:26:36 varnish varnishd[25116]: }, Oct 04 11:26:36 varnish varnishd[25116]: }, Oct 04 11:26:36 varnish varnishd[25116]: hdrs { Oct 04 11:26:36 varnish varnishd[25116]: ws[] = (nil), Oct 04 11:26:36 varnish varnishd[25116]: http[req] = 0x7f551c090be0 { Oct 04 11:26:36 varnish varnishd[25116]: }, Oct 04 11:26:36 varnish varnishd[25116]: between_bytes_timeout = 0.000000, Oct 04 11:26:36 varnish varnishd[25116]: first_byte_timeout = 0.000000, Oct 04 11:26:36 varnish varnishd[25116]: body_status = none, Oct 04 11:26:36 varnish varnishd[25116]: content_length = 0, Oct 04 11:26:36 varnish varnishd[25116]: {pipeline_b, pipeline_e} = {(nil), (nil)}, Oct 04 11:26:36 varnish varnishd[25116]: {rxbuf_b, rxbuf_e} = {0x7f55410a0470, 0x7f55410a047d}, Oct 04 11:26:36 varnish varnishd[25116]: ws = 0x7f55410a0eb8, Oct 04 11:26:36 varnish varnishd[25116]: doclose = NULL, Oct 04 11:26:36 varnish varnishd[25116]: fd = 23, Oct 04 11:26:36 varnish varnishd[25116]: http_conn = 0x7f551c090a78 { Oct 04 11:26:36 varnish varnishd[25116]: }, Oct 04 11:26:36 varnish varnishd[25116]: {s, f, r, e} = {0x7f551c092948, +264, (nil), +57344}, Oct 04 11:26:36 varnish varnishd[25116]: id = \"req\", Oct 04 11:26:36 varnish varnishd[25116]: ws = 0x7f551c090b48 { Oct 04 11:26:36 varnish varnishd[25116]: }, Oct 04 11:26:36 varnish varnishd[25116]: client = 10.0.2.1 51817, Oct 04 11:26:36 varnish varnishd[25116]: } Oct 04 11:26:36 varnish varnishd[25116]: } Oct 04 11:26:36 varnish varnishd[25116]: 0x00000073 open Oct 04 11:26:36 varnish varnishd[25116]: 0x00000000 idle Oct 04 11:26:36 varnish varnishd[25116]: streams { Oct 04 11:26:36 varnish varnishd[25116]: transport = H2 { Oct 04 11:26:36 varnish varnishd[25116]: t_idle = 1475580390.904838, Oct 04 11:26:36 varnish varnishd[25116]: t_open = 1475580375.929317, Oct 04 11:26:36 varnish varnishd[25116]: fd = 23, vxid = 491543, Oct 04 11:26:36 varnish varnishd[25116]: sp = 0x7f5524001960 { Oct 04 11:26:36 varnish varnishd[25116]: restarts = 0, esi_level = 0, Oct 04 11:26:36 varnish varnishd[25116]: err_code = 1, err_reason = (null), Oct 04 11:26:36 varnish varnishd[25116]: req_body = R_BODY_INIT, Oct 04 11:26:36 varnish varnishd[25116]: step = 0x0, Oct 04 11:26:36 varnish varnishd[25116]: vxid = 491543, transport = H2 Oct 04 11:26:36 varnish varnishd[25116]: req = 0x7f551c090970 { Oct 04 11:26:36 varnish varnishd[25116]: 0x7f554fc8402d: libc.so.6(clone+0x6d) [0x7f554fc8402d] Oct 04 11:26:36 varnish varnishd[25116]: 0x7f554ff4f0a4: libpthread.so.0(+0x80a4) [0x7f554ff4f0a4] Oct 04 11:26:36 varnish varnishd[25116]: 0x44f1eb: pool_thread+0x2b Oct 04 11:26:36 varnish varnishd[25116]: 0x44ed81: WRK_Thread+0x481 Oct 04 11:26:36 varnish varnishd[25116]: 0x45e478: h2_new_session+0xe98 Oct 04 11:26:36 varnish varnishd[25116]: 0x45ce89: varnishd() [0x45ce89] Oct 04 11:26:36 varnish varnishd[25116]: 0x4373d6: pan_ic+0x156 Oct 04 11:26:36 varnish varnishd[25116]: Backtrace: Oct 04 11:26:36 varnish varnishd[25116]: ident = Linux,4.1.31-30-default,x86_64,-junix,-smalloc,-smalloc,-hcritbit,epoll Oct 04 11:26:36 varnish varnishd[25116]: version = varnish-5.0.0 revision 99d036f Oct 04 11:26:36 varnish varnishd[25116]: thread = (cache-worker) Oct 04 11:26:36 varnish varnishd[25116]: "Incomplete code in h2_rx_rst_stream(), http2/cache_http2_proto.c line 113: Oct 04 11:26:36 varnish varnishd[25116]: Error: Child (25130) Panic at: Tue, 04 Oct 2016 11:26:36 GMT Oct 04 11:26:36 varnish varnishd[25116]: Child (25130) Panic at: Tue, 04 Oct 2016 11:26:36 GMT "Incomplete code in h2_rx_rst_stream(), http2/cache_http2_proto.c line 113: thread = (cache-worker) version = varnish-5.0.0 revision 99d036f ident = Linux,4.1.31-30-default,x86_64,-junix,-smalloc,-smalloc,-hcritbit,epoll Backtrace: 0x4373d6: pan_ic+0x156 0x45ce89: varnishd() [0x45ce89] 0x45e478: h2_new_session+0xe98 0x44ed81: WRK_Thread+0x481 0x44f1eb: pool_thread+0x2b 0x7f554ff4f0a4: libpthread.so.0(+0x80a4) [0x7f554ff4f0a4] 0x7f554fc8402d: libc.so.6(clone+0x6d) [0x7f554fc8402d] req = 0x7f551c090970 { vxid = 491543, transport = H2 step = 0x0, req_body = R_BODY_INIT, err_code = 1, err_reason = (null), restarts = 0, esi_level = 0, sp = 0x7f5524001960 { fd = 23, vxid = 491543, t_open = 1475580375.929317, t_idle = 1475580390.904838, transport = H2 { streams { 0x00000000 idle 0x00000073 open } } client = 10.0.2.1 51817, }, ws = 0x7f551c090b48 { id = \"req\", {s, f, r, e} = {0x7f551c092948, +264, (nil), +57344}, }, http_conn = 0x7f551c090a78 { fd = 23, doclose = NULL, ws = 0x7f55410a0eb8, {rxbuf_b, rxbuf_e} = {0x7f55410a0470, 0x7f55410a047d}, {pipeline_b, pipeline_e} = {(nil), (nil)}, content_length = 0, body_status = none, first_byte_timeout = 0.000000, between_bytes_timeout = 0.000000, }, http[req] = 0x7f551c090be0 { ws[] = (nil), hdrs { }, }, flags = { }, }, " Oct 04 11:26:36 varnish varnishd[25116]: Child (25130) died signal=6 From dridi at varni.sh Tue Oct 4 14:06:03 2016 From: dridi at varni.sh (Dridi Boukelmoune) Date: Tue, 4 Oct 2016 16:06:03 +0200 Subject: http/2 first experiences In-Reply-To: <2999a459-37b9-90be-8a35-f8cd33b92da8@Berlinonline.de> References: <2999a459-37b9-90be-8a35-f8cd33b92da8@Berlinonline.de> Message-ID: On Tue, Oct 4, 2016 at 1:31 PM, Tom Anheyer wrote: > Hello, > > I've setup a little test environment with varnish5 and hitch as TLS > offloader. HTTP/2 works for me in FF and Chrome (after upgrading to openssl > 1.0.2). Hello Tom, This is fantastic, thanks for the feedback on h2 using hitch+varnish. > Which H2 features are supported so far? I haven't looked closely, but if everything went according to plan, the bare minimum (eg. varnish does not compress response headers etc). > How to do server push? If not supported yet - Is it planned? It is planned, I don't know when it will land, and when it lands it will probably introduce either a VMOD or new VCL syntax. > 2016 11:26:36 GMT > "Incomplete code in > h2_rx_rst_stream(), http2/cache_http2_proto.c line 113: So apparently you reached one of the things Varnish doesn't implement yet, in this case RST frames. Your browser tried to cancel a request and close the related streams and Varnish doesn't support it. Spoiler alert, your browser may crash with DATA, PUSH_PROMISE and CONTINUATION frames. The push makes no sense for a client, so you can ignore it if we assume a sane browser. You should be able to crash Varnish too if you send a request with a body (eg. POST) or with very large headers (eg. tracking cookies). Cheers, Dridi From Tom.Anheyer at Berlinonline.de Tue Oct 4 14:26:22 2016 From: Tom.Anheyer at Berlinonline.de (Tom Anheyer) Date: Tue, 4 Oct 2016 16:26:22 +0200 Subject: http/2 first experiences In-Reply-To: References: <2999a459-37b9-90be-8a35-f8cd33b92da8@Berlinonline.de> Message-ID: Ok, thanks for your answer. Im waiting for version 5.0.1 to 5.0.99 to check new features. :) tom Am 04.10.2016 um 16:06 schrieb Dridi Boukelmoune: > On Tue, Oct 4, 2016 at 1:31 PM, Tom Anheyer wrote: >> Hello, >> >> I've setup a little test environment with varnish5 and hitch as TLS >> offloader. HTTP/2 works for me in FF and Chrome (after upgrading to openssl >> 1.0.2). > Hello Tom, > > This is fantastic, thanks for the feedback on h2 using hitch+varnish. > >> Which H2 features are supported so far? > I haven't looked closely, but if everything went according to plan, > the bare minimum (eg. varnish does not compress response headers > etc). > >> How to do server push? If not supported yet - Is it planned? > It is planned, I don't know when it will land, and when it lands it > will probably introduce either a VMOD or new VCL syntax. > >> 2016 11:26:36 GMT >> "Incomplete code in >> h2_rx_rst_stream(), http2/cache_http2_proto.c line 113: > So apparently you reached one of the things Varnish doesn't implement > yet, in this case RST frames. Your browser tried to cancel a request > and close the related streams and Varnish doesn't support it. > > Spoiler alert, your browser may crash with DATA, PUSH_PROMISE and > CONTINUATION frames. > > The push makes no sense for a client, so you can ignore it if we > assume a sane browser. > > You should be able to crash Varnish too if you send a request with a > body (eg. POST) or with very large headers (eg. tracking cookies). > > Cheers, > Dridi From frederik at remote.org Sat Oct 15 09:03:51 2016 From: frederik at remote.org (Frederik Ramm) Date: Sat, 15 Oct 2016 11:03:51 +0200 Subject: Different host headers - same cache? Message-ID: <5801F0F7.8000209@remote.org> Hi, new varnish user here with a configuration question. I have two backend servers in a round-robin director. Both require different host headers to be set, but both deliver the same content and I want there to be only one cache that mixes content from both machines. Question 1: I tried to force the host header in the backend definition like so: backend A { .host = "1.2.3.4"; .host_header = "server_a" ... } but this didn't work; the .host_header did affect the "probe" requests it made automatically, but actual content requests still bore the name of the varnish server itself. I had to do a "set req.http.Host=..." in vcl_recv to actually change the Host header used, but that defeats my purpose since I want the round robin director to decide which backend to use. Is it a bug or a feature that .host_header doesn't actually override the host header for content requests? I'm on varnish 5.0.0. Question 2: Do I need to do anything to ensure that requests going to backends A and B, with host headers "server_a" and "server_b", land in the same cache? I know this is a problem I could probably solve by reconfiguring my backend servers to listen to the same host name but this would trip up another, unrelated aspect of my setup so if it can be avoided I'd like to avoid it. Bye Frederik -- Frederik Ramm ## eMail frederik at remote.org ## N49?00'09" E008?23'33" From thomas.lecomte at virtual-expo.com Sat Oct 15 22:55:31 2016 From: thomas.lecomte at virtual-expo.com (Thomas Lecomte) Date: Sun, 16 Oct 2016 00:55:31 +0200 Subject: Different host headers - same cache? In-Reply-To: <5801F0F7.8000209@remote.org> References: <5801F0F7.8000209@remote.org> Message-ID: Hello, On Sat, Oct 15, 2016 at 11:03 AM, Frederik Ramm wrote: > Question 1: You will need to rewrite bereq.http.host in the vcl_backend_fetch routine depending on the backend to which you are sending the request. > Question 2: The hash used to identify an unique object is computed in vcl_hash, and by default, includes req.http.host (the HTTP Host: header sent by the client). You don't have to modify something since it's not the Host header sent to the backend that is being hashed, but the one sent from the client to varnish. Hope this helps, -- Thomas Lecomte From guillaume at varnish-software.com Sun Oct 16 07:33:28 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Sun, 16 Oct 2016 09:33:28 +0200 Subject: Different host headers - same cache? In-Reply-To: References: <5801F0F7.8000209@remote.org> Message-ID: On Oct 16, 2016 01:15, "Thomas Lecomte" wrote: > > Hello, > > On Sat, Oct 15, 2016 at 11:03 AM, Frederik Ramm wrote: > > Question 1: > > You will need to rewrite bereq.http.host in the vcl_backend_fetch > routine depending on the backend to which you are sending the request. I think it's easier than that: just remove the host header and let varnish add the right one, as this (old) ticket seems to indicate: http://varnish-cache.org/trac/ticket/230 . I was assuming the same thing as Frederik: that host_header was a hard override. I think a docfix is in order (ie. I'll do it) -------------- next part -------------- An HTML attachment was scrubbed... URL: From frederik at remote.org Sun Oct 16 20:30:46 2016 From: frederik at remote.org (Frederik Ramm) Date: Sun, 16 Oct 2016 22:30:46 +0200 Subject: Different host headers - same cache? In-Reply-To: References: <5801F0F7.8000209@remote.org> Message-ID: <5803E376.9010300@remote.org> Thomas, thank you, that gets me a bit further. I have one more question though: On 10/16/2016 12:55 AM, Thomas Lecomte wrote: > You will need to rewrite bereq.http.host in the vcl_backend_fetch > routine depending on the backend to which you are sending the request. So which backend the request is sent to, has been determined in vcl_recv, using "set req.backend_hint=somedirector". Now I'm trying to do what you say, and in vcl_backend_fetch set the bereq.http.host according to which backend has been selected. I wanted to be clever and tried: set bereq.http.host = bereq.backend.host_header; (assuming that by the time that vcl_backend_fetch is called, the director would already have selected a specific backend). But "Symbol not found: 'bereq.backend.host_header' (expected type STRING)" Then I thought ok, since the docs say about bereq.backend: "When used in string context, returns the name of the director or backend, respectively." I should be able to do this: if (bereq.backend == "somedirector") { set bereq.http.host = "somehost"; } elsif (bereq.backend == "otherdirector") { set bereq.http.host= "otherhost"; } But (VRT_r_bereq_backend(ctx) == "somedirector") ^ vgc.c:2152:35: error: comparison with string literal results in unspecified behavior [-Werror=address] How can I force the "string context" on the backend/director, so that I can use string comparison to find out which backend has been selected? Or how can I otherwise compare bereq.backend with something? I also tried to access the req.backend_hint but that seems to be write-only. If all else files I could maybe abuse some otherwise unused variable in the "req" object that I can set in vcl_recv (where I determine which backend I want) and then read in vcl_backend_fetch? I guess it's probably all dead simple and I just didn't find it. Bye Frederik -- Frederik Ramm ## eMail frederik at remote.org ## N49?00'09" E008?23'33" From guilleret.florian at gmail.com Mon Oct 17 07:23:20 2016 From: guilleret.florian at gmail.com (Guilleret Florian) Date: Mon, 17 Oct 2016 09:23:20 +0200 Subject: Varnish Error 503 Backend fetch failed Message-ID: Hi, I'm getting weird 503 error on varnish. On page with ESI ( not on every page of my website ) i got Error 503 Backend fetch failed. On varnish log i can saw FetchError http read error: overflow : * << BeReq >> 9 - Begin bereq 8 fetch - Timestamp Start: 1476370852.736703 0.000000 0.000000 - BereqMethod GET - BereqURL /pageXXX.html - BereqProtocol HTTP/1.1 - BereqHeader Host: www.websiteXXXX.com - BereqHeader User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:47.0) Gecko/20100101 Firefox/47.0 - BereqHeader Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 - BereqHeader Accept-Language: fr-FR - BereqHeader Cookie: mage-translation-storage=%7B%7D; mage-translation-file-version=%7B%7D; mage-cache-storage=%7B%7D; mage-cache-storage-section-invalidation=%7B%7D; mage-cache-sessid=true; X-Magento-Vary=f970ba1c8edf4251be121a7535f2ce003b791953; form_key=lV5GbPY6uP - BereqHeader X-Forwarded-For: 80.74.72.196 - BereqHeader Accept-Encoding: gzip - BereqHeader X-Varnish: 9 - VCL_call BACKEND_FETCH - VCL_return fetch - BackendOpen 53 default(127.0.0.1,,81) 127.0.0.1 59720 - Backend 53 default default(127.0.0.1,,81) - Timestamp Bereq: 1476370852.736959 0.000256 0.000256 - FetchError http read error: overflow - BackendClose 53 default(127.0.0.1,,81) - Timestamp Beresp: 1476370865.707061 12.970358 12.970102 - Timestamp Error: 1476370865.707072 12.970368 0.000011 - BerespProtocol HTTP/1.1 - BerespStatus 503 - BerespReason Service Unavailable - BerespReason Backend fetch failed - BerespHeader Date: Thu, 13 Oct 2016 15:01:05 GMT - BerespHeader Server: Varnish - VCL_call BACKEND_ERROR - BerespHeader Content-Type: text/html; charset=utf-8 - BerespHeader Retry-After: 5 - VCL_return deliver - Storage malloc Transient - ObjProtocol HTTP/1.1 - ObjStatus 503 - ObjReason Backend fetch failed - ObjHeader Date: Thu, 13 Oct 2016 15:01:05 GMT - ObjHeader Server: Varnish - ObjHeader Content-Type: text/html; charset=utf-8 - ObjHeader Retry-After: 5 - Length 278 - BereqAcct 611 0 611 7767 0 7767 - End My website is a magento 2. So I suppose this is my issue : http://devdocs.magento.com/guides/v2.0/config-guide/varnish/tshoot-varnish-503.html But even with this option http_resp_hdr_len it doesn't work... Can anyone know how to debug this ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From frederik at remote.org Mon Oct 17 23:22:30 2016 From: frederik at remote.org (Frederik Ramm) Date: Tue, 18 Oct 2016 01:22:30 +0200 Subject: Different host headers - same cache? In-Reply-To: <5803E376.9010300@remote.org> References: <5801F0F7.8000209@remote.org> <5803E376.9010300@remote.org> Message-ID: <58055D36.5000302@remote.org> Hi, On 10/16/2016 10:30 PM, Frederik Ramm wrote: > I should be able to do this: > > if (bereq.backend == "somedirector") ... > vgc.c:2152:35: error: comparison with string literal results in > unspecified behavior [-Werror=address] Ok, I meanwhile found out two things: 1. bereq.backend will contain the actual backend selected by the director, not the director itself; 2. for some reason I cannot do a string equality check against bereq.backend like above, but I can do if (std.strstr(bereq.backend, "somebackend")) { ... } - which doesn't make me entirely happy since a strstr is certainly more expensive than a string equality check but at least that seems to work! Bye Frederik -- Frederik Ramm ## eMail frederik at remote.org ## N49?00'09" E008?23'33" From lagged at gmail.com Wed Oct 19 07:02:55 2016 From: lagged at gmail.com (Andrei) Date: Wed, 19 Oct 2016 02:02:55 -0500 Subject: Different host headers - same cache? In-Reply-To: <58055D36.5000302@remote.org> References: <5801F0F7.8000209@remote.org> <5803E376.9010300@remote.org> <58055D36.5000302@remote.org> Message-ID: Out of curiosity, how expensive is it to use std.strstr()? Would it even have any sort of noticeable performance impact or just a slightly elevated cpu time for somewhat elevated traffic (~25k req/s)? On Mon, Oct 17, 2016 at 6:22 PM, Frederik Ramm wrote: > Hi, > > On 10/16/2016 10:30 PM, Frederik Ramm wrote: > > I should be able to do this: > > > > if (bereq.backend == "somedirector") > > ... > > > vgc.c:2152:35: error: comparison with string literal results in > > unspecified behavior [-Werror=address] > > Ok, I meanwhile found out two things: > > 1. bereq.backend will contain the actual backend selected by the > director, not the director itself; > > 2. for some reason I cannot do a string equality check against > bereq.backend like above, but I can do > > if (std.strstr(bereq.backend, "somebackend")) > { > ... > } > > - which doesn't make me entirely happy since a strstr is certainly more > expensive than a string equality check but at least that seems to work! > > Bye > Frederik > > -- > Frederik Ramm ## eMail frederik at remote.org ## N49?00'09" E008?23'33" > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kelvin1111111 at gmail.com Wed Oct 19 10:06:15 2016 From: kelvin1111111 at gmail.com (Kelvin Loke) Date: Wed, 19 Oct 2016 10:06:15 +0000 Subject: Varnish Variable for Decoded req.http.Authorization Message-ID: Good day everyday, I have a question about req.http.Authorization. As req.http.Authorization is at the format of "Basic Uk1CVTUweWMjIyMg==", do we have any variable for the base64 decoded value from req.http.Authorization? I knew that Varnishncsa has %u for this purpose, it would be great if there is a default variable that I can use in Varnish VCL. Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Wed Oct 19 16:54:32 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Wed, 19 Oct 2016 18:54:32 +0200 Subject: Varnish Variable for Decoded req.http.Authorization In-Reply-To: References: Message-ID: Hi Kelvin, It's actually pretty simple to do: use a regex to get the second word, then use vmod-var to decode the token. On Oct 19, 2016 12:15, "Kelvin Loke" wrote: > Good day everyday, I have a question about req.http.Authorization. > As req.http.Authorization is at the format of "Basic Uk1CVTUweWMjIyMg==", > do we have any variable for the base64 decoded value from > req.http.Authorization? > > I knew that Varnishncsa has %u for this purpose, it would be great if > there is a default variable that I can use in Varnish VCL. > > Thanks! > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kelvin1111111 at gmail.com Thu Oct 20 04:05:33 2016 From: kelvin1111111 at gmail.com (Kelvin Loke) Date: Thu, 20 Oct 2016 04:05:33 +0000 Subject: Varnish Variable for Decoded req.http.Authorization In-Reply-To: References: Message-ID: > > Hi Kelvin, > > It's actually pretty simple to do: use a regex to get the second word, > then use vmod-var to decode the token. > I did check vmod-var, I couldn't find any usage/manual about the base64 decode function. Do you mean the libvmod-digest? Cheers. -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Thu Oct 20 07:17:45 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Thu, 20 Oct 2016 09:17:45 +0200 Subject: Varnish Variable for Decoded req.http.Authorization In-Reply-To: References: Message-ID: Arg, my bad, that was indeed vmod-digest (somehow, my tablet thinks vmod-var is an acceptable autocorrection for vmod-digest....) On Oct 20, 2016 06:05, "Kelvin Loke" wrote: > Hi Kelvin, >> >> It's actually pretty simple to do: use a regex to get the second word, >> then use vmod-var to decode the token. >> > I did check vmod-var, I couldn't find any usage/manual about the base64 > decode function. Do you mean the libvmod-digest? > > Cheers. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From shayne.alone at gmail.com Thu Oct 20 17:44:05 2016 From: shayne.alone at gmail.com (AliReza Taleghani) Date: Thu, 20 Oct 2016 21:14:05 +0330 Subject: No subject Message-ID: Hello, we server multiple Apache Hosted Virtual-hosts behind a Varnish. I also used https://github.com/comotion/VSF . The problem I faced is for example when i do a large select * in my phpmyadmin vhost the error rise as following: Error in Processing Request Error code: 403 Error text: Naughty, not nice! and i find this the SQL Injection module which catch the request. because we use static ip-address for our office, I need to know how can I bypass VSF for some specific source ip-address! thanks Sincerely, Ali R. Taleghani @linkedIn -------------- next part -------------- An HTML attachment was scrubbed... URL: From ayberk.kimsesiz at gmail.com Fri Oct 21 09:54:59 2016 From: ayberk.kimsesiz at gmail.com (Ayberk Kimsesiz) Date: Fri, 21 Oct 2016 12:54:59 +0300 Subject: Wordpress and error 503 backend fetch failed Message-ID: Hi, We are having serious problems with Varnish. The system was running smoothly for months but now gives the following errors Pages are either completely not loading or while they are loading it takes 5-6 seconds to load especially .jpg files. ? Varnishlog: BereqHeader Accept-Language: en-us,en - BereqHeader Accept-Charset: iso-8859-1,*,utf-8 - BereqHeader X-Actual-IP: 85.103.145.67 - BereqHeader X-Forwarded-For: 85.103.145.67, 85.103.145.67 - BereqHeader Accept-Encoding: gzip - BereqHeader X-Varnish: 1147158 - VCL_call BACKEND_FETCH - VCL_return fetch - BackendOpen 31 boot.default 176.53.126.10 8080 176.53.126.10 56062 - BackendStart 176.53.126.10 8080 - Timestamp Bereq: 1477043059.209814 0.000116 0.000116 - FetchError http first read error: EOF - BackendClose 31 boot.default - Timestamp Beresp: 1477043119.209551 59.999853 59.999737 - Timestamp Error: 1477043119.209561 59.999863 0.000010 - BerespProtocol HTTP/1.1 - BerespStatus 503 - BerespReason Service Unavailable - BerespReason Backend fetch failed - BerespHeader Date: Fri, 21 Oct 2016 09:45:19 GMT - BerespHeader Server: Varnish - VCL_call BACKEND_ERROR - BerespHeader Content-Type: text/html; charset=utf-8 - BerespHeader Retry-After: 5 - VCL_return deliver - Storage malloc Transient - ObjProtocol HTTP/1.1 - ObjStatus 503 - ObjReason Backend fetch failed - ObjHeader Date: Fri, 21 Oct 2016 09:45:19 GMT - ObjHeader Server: Varnish - ObjHeader Content-Type: text/html; charset=utf-8 - ObjHeader Retry-After: 5 - Length 284 - BereqAcct 519 0 519 0 0 0 - End * << Session >> 6717525 - Begin sess 0 HTTP/1 - SessOpen 193.86.18.178 59638 :80 176.53.126.10 80 1477043113.589802 45 - Link req 6717526 rxreq - Link req 33158 rxreq - Link req 6717528 rxreq - Link req 6717529 rxreq - Link req 6717530 rxreq - Link req 6225952 rxreq - Link req 6717532 rxreq - Link req 33165 rxreq - Link req 33167 rxreq - Link req 6717535 rxreq - Link req 3014918 rxreq - Link req 6717538 rxreq - Link req 6717539 rxreq - Link req 6225957 rxreq - Link req 6225958 rxreq - Link req 6717542 rxreq - SessClose RX_TIMEOUT 6.679 - End -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: def.jpg Type: image/jpeg Size: 23662 bytes Desc: not available URL: From lagged at gmail.com Fri Oct 21 11:08:41 2016 From: lagged at gmail.com (Andrei) Date: Fri, 21 Oct 2016 06:08:41 -0500 Subject: Wordpress and error 503 backend fetch failed In-Reply-To: References: Message-ID: Hello, Your backend server is returning a 503 with Retry-After: 5 to Varnish. I suggest reviewing your backend logs for the reason the requests aren't going through as expected. On Fri, Oct 21, 2016 at 4:54 AM, Ayberk Kimsesiz wrote: > Hi, > > We are having serious problems with Varnish. > The system was running smoothly for months but now gives the following > errors > > Pages are either completely not loading or while they are loading it takes > 5-6 seconds to load especially .jpg files. > > > ? > > Varnishlog: > > BereqHeader Accept-Language: en-us,en > - BereqHeader Accept-Charset: iso-8859-1,*,utf-8 > - BereqHeader X-Actual-IP: 85.103.145.67 > - BereqHeader X-Forwarded-For: 85.103.145.67, 85.103.145.67 > - BereqHeader Accept-Encoding: gzip > - BereqHeader X-Varnish: 1147158 > - VCL_call BACKEND_FETCH > - VCL_return fetch > - BackendOpen 31 boot.default 176.53.126.10 8080 176.53.126.10 56062 > - BackendStart 176.53.126.10 8080 > - Timestamp Bereq: 1477043059.209814 0.000116 0.000116 > - FetchError http first read error: EOF > - BackendClose 31 boot.default > - Timestamp Beresp: 1477043119.209551 59.999853 59.999737 > - Timestamp Error: 1477043119.209561 59.999863 0.000010 > - BerespProtocol HTTP/1.1 > - BerespStatus 503 > - BerespReason Service Unavailable > - BerespReason Backend fetch failed > - BerespHeader Date: Fri, 21 Oct 2016 09:45:19 GMT > - BerespHeader Server: Varnish > - VCL_call BACKEND_ERROR > - BerespHeader Content-Type: text/html; charset=utf-8 > - BerespHeader Retry-After: 5 > - VCL_return deliver > - Storage malloc Transient > - ObjProtocol HTTP/1.1 > - ObjStatus 503 > - ObjReason Backend fetch failed > - ObjHeader Date: Fri, 21 Oct 2016 09:45:19 GMT > - ObjHeader Server: Varnish > - ObjHeader Content-Type: text/html; charset=utf-8 > - ObjHeader Retry-After: 5 > - Length 284 > - BereqAcct 519 0 519 0 0 0 > - End > > * << Session >> 6717525 > - Begin sess 0 HTTP/1 > - SessOpen 193.86.18.178 59638 :80 176.53.126.10 80 > 1477043113.589802 45 > - Link req 6717526 rxreq > - Link req 33158 rxreq > - Link req 6717528 rxreq > - Link req 6717529 rxreq > - Link req 6717530 rxreq > - Link req 6225952 rxreq > - Link req 6717532 rxreq > - Link req 33165 rxreq > - Link req 33167 rxreq > - Link req 6717535 rxreq > - Link req 3014918 rxreq > - Link req 6717538 rxreq > - Link req 6717539 rxreq > - Link req 6225957 rxreq > - Link req 6225958 rxreq > - Link req 6717542 rxreq > - SessClose RX_TIMEOUT 6.679 > - End > > > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: def.jpg Type: image/jpeg Size: 23662 bytes Desc: not available URL: From guillaume at varnish-software.com Mon Oct 24 08:46:09 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Mon, 24 Oct 2016 10:46:09 +0200 Subject: Varnish Error 503 Backend fetch failed In-Reply-To: References: Message-ID: Have you changed http_resp_size too? -- Guillaume Quintard On Mon, Oct 17, 2016 at 9:23 AM, Guilleret Florian < guilleret.florian at gmail.com> wrote: > Hi, > > I'm getting weird 503 error on varnish. > > On page with ESI ( not on every page of my website ) i got Error 503 > Backend fetch failed. > > On varnish log i can saw FetchError http read error: overflow : > > > * << BeReq >> 9 > - Begin bereq 8 fetch > - Timestamp Start: 1476370852.736703 0.000000 0.000000 > - BereqMethod GET > - BereqURL /pageXXX.html > - BereqProtocol HTTP/1.1 > - BereqHeader Host: www.websiteXXXX.com > - BereqHeader User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:47.0) > Gecko/20100101 Firefox/47.0 > - BereqHeader Accept: text/html,application/xhtml+ > xml,application/xml;q=0.9,*/*;q=0.8 > - BereqHeader Accept-Language: fr-FR > - BereqHeader Cookie: mage-translation-storage=%7B%7D; > mage-translation-file-version=%7B%7D; mage-cache-storage=%7B%7D; > mage-cache-storage-section-invalidation=%7B%7D; mage-cache-sessid=true; > X-Magento-Vary=f970ba1c8edf4251be121a7535f2ce003b791953; > form_key=lV5GbPY6uP > - BereqHeader X-Forwarded-For: 80.74.72.196 > - BereqHeader Accept-Encoding: gzip > - BereqHeader X-Varnish: 9 > - VCL_call BACKEND_FETCH > - VCL_return fetch > - BackendOpen 53 default(127.0.0.1,,81) 127.0.0.1 59720 > - Backend 53 default default(127.0.0.1,,81) > - Timestamp Bereq: 1476370852.736959 0.000256 0.000256 > - FetchError http read error: overflow > - BackendClose 53 default(127.0.0.1,,81) > - Timestamp Beresp: 1476370865.707061 12.970358 12.970102 > - Timestamp Error: 1476370865.707072 12.970368 0.000011 > - BerespProtocol HTTP/1.1 > - BerespStatus 503 > - BerespReason Service Unavailable > - BerespReason Backend fetch failed > - BerespHeader Date: Thu, 13 Oct 2016 15:01:05 GMT > - BerespHeader Server: Varnish > - VCL_call BACKEND_ERROR > - BerespHeader Content-Type: text/html; charset=utf-8 > - BerespHeader Retry-After: 5 > - VCL_return deliver > - Storage malloc Transient > - ObjProtocol HTTP/1.1 > - ObjStatus 503 > - ObjReason Backend fetch failed > - ObjHeader Date: Thu, 13 Oct 2016 15:01:05 GMT > - ObjHeader Server: Varnish > - ObjHeader Content-Type: text/html; charset=utf-8 > - ObjHeader Retry-After: 5 > - Length 278 > - BereqAcct 611 0 611 7767 0 7767 > - End > > > My website is a magento 2. So I suppose this is my issue : > > http://devdocs.magento.com/guides/v2.0/config-guide/ > varnish/tshoot-varnish-503.html > > But even with this option http_resp_hdr_len it doesn't work... > > Can anyone know how to debug this ? > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Mon Oct 24 09:32:34 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Mon, 24 Oct 2016 11:32:34 +0200 Subject: Different host headers - same cache? In-Reply-To: References: <5801F0F7.8000209@remote.org> <5803E376.9010300@remote.org> <58055D36.5000302@remote.org> Message-ID: Why isn't unsetting the host header (to let varnish use .host_header) acceptable? I wouldn't worry about the performance impact of strstr, but if you really don't like it, you can do this: if ("somebackend" == bereq.backend + "") to force the type conversion. If you feel really nostalgic, you can even do: if ("xsomebackend" == "x" + bereq.backend) But only a madman would take pleasure in this. -- Guillaume Quintard -------------- next part -------------- An HTML attachment was scrubbed... URL: From frederik at remote.org Mon Oct 24 09:53:50 2016 From: frederik at remote.org (Frederik Ramm) Date: Mon, 24 Oct 2016 11:53:50 +0200 Subject: Different host headers - same cache? In-Reply-To: References: <5801F0F7.8000209@remote.org> <5803E376.9010300@remote.org> <58055D36.5000302@remote.org> Message-ID: <580DDA2E.5000601@remote.org> Guillaume, On 10/24/16 11:32, Guillaume Quintard wrote: > Why isn't unsetting the host header (to let varnish use .host_header) > acceptable? Oh that's perfectly acceptable, I just didn't know it did that! Because the Host header that came from the frontend always trumped .host_header, I didn't even think about trying that. Will do right away. > But only a madman would take pleasure in this. Mwahahaha... ;) Bye Frederik -- Frederik Ramm ## eMail frederik at remote.org ## N49?00'09" E008?23'33" From guillaume at varnish-software.com Mon Oct 24 10:02:42 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Mon, 24 Oct 2016 12:02:42 +0200 Subject: No subject In-Reply-To: References: Message-ID: I think you will have to edit vsf.vcl for this. -- Guillaume Quintard On Thu, Oct 20, 2016 at 7:44 PM, AliReza Taleghani wrote: > Hello, > > > we server multiple Apache Hosted Virtual-hosts behind a Varnish. > I also used https://github.com/comotion/VSF . > The problem I faced is for example when i do a large select * in my > phpmyadmin vhost the error rise as following: > Error in Processing Request > Error code: 403 > Error text: Naughty, not nice! > and i find this the SQL Injection module which catch the request. > > because we use static ip-address for our office, I need to know how can I > bypass VSF for some specific source ip-address! > > thanks > > > > Sincerely, > Ali R. Taleghani > @linkedIn > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From r at roze.lv Tue Oct 25 15:37:18 2016 From: r at roze.lv (Reinis Rozitis) Date: Tue, 25 Oct 2016 18:37:18 +0300 Subject: Vanishstat -f in 5.0.0 Message-ID: <6B000CD8553C4611B1C5BC901584D25B@MasterPC> Hi, has there something changed in 5.0.0 regarding varnishstat (or is there a bug) as the -f doesn't seem to work. varnishstat -V varnishstat (varnish-5.0.0 revision 99d036f) While I can get the full output (also with -1 flag): varnishstat -1 | grep MAIN.cache_hit MAIN.cache_hit 22615 17.26 Cache hits MAIN.cache_hitpass 5 0.00 Cache hits for pass varnishstat -1 -f MAIN.cache_hit doesn't return anything. 4.1.x works fine: varnishstat -V varnishstat (varnish-4.1.3 revision 5e3b6d2) varnishstat -1 -f MAIN.cache_hit MAIN.cache_hit 89691619 10.69 Cache hits rr From viktor.villafuerte at optusnet.com.au Tue Oct 25 22:02:44 2016 From: viktor.villafuerte at optusnet.com.au (Viktor Villafuerte) Date: Wed, 26 Oct 2016 09:02:44 +1100 Subject: Error out of workspace Message-ID: <20161025220244.GH12045@optusnet.com.au> Hi all, Yesterday I saw this error in Varnishlog Error out of workspace I've done some googling since and it seems that increasing workspace_client could fix the issue. Further it's suggested that this is due to excessive cookie processing. However, we don't process any cookines at all. Also I've found an old bug where it's suggested that VCL code ends up in a loop (using restart feature) and eventually runs out of space. We do use restart but it's limited to only 3 attempts.. Anybody would have any experience with 'out of workspace' error and why it could occur? Any suggestions are welcomed, let me know if more details are needed. thanks v -- Regards Viktor Villafuerte Optus Internet Engineering t: +61 2 80825265 From admin at beckspaced.com Wed Oct 26 08:03:50 2016 From: admin at beckspaced.com (Admin Beckspaced) Date: Wed, 26 Oct 2016 10:03:50 +0200 Subject: varnish 5 and multiple domains Message-ID: <6539c570-eddf-454b-702b-d7831b3651b4@beckspaced.com> dear varnish users ;) i'm new to varnish and got a question about how to best implement multiple domains into varnish 5 i got quite a few domains hosted on my linux server and would like to include some domains, exclude others and also switch between different vcl's based on the domain. i've looked around on the internet but thought why not ask the mailing list pro's for advice ;) should i use the new load and label feature with version 5 as explained here: https://varnish-cache.org/docs/trunk/users-guide/vcl-separate.html or should i use the include feature as explained here: https://www.varnish-cache.org/lists/pipermail/varnish-misc/2011-January/019815.html e.g. if (req.http.Host == "domainx.be") { include "/etc/varnish/domainx.be.vcl"; } or, instead of include, should i put all in a single vcl file? sub vcl_recv { if (req.http.Host == "www.example.nl") { do some stuff; } elsif (req.http.Host == "www.example.com") { do some stuff; } } which approach would you recommend for multiple domains? thanks for your hints & tips ;) becki From guillaume at varnish-software.com Wed Oct 26 08:10:13 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Wed, 26 Oct 2016 10:10:13 +0200 Subject: Error out of workspace In-Reply-To: <20161025220244.GH12045@optusnet.com.au> References: <20161025220244.GH12045@optusnet.com.au> Message-ID: Increasing workspace_client may not do it, you have to consider where this error occurred, if it was in the backend context, workspace_backend is your guy. Pretty much anything in your vcl will consume workspace, because everytime you need a string, Varnish allocates it in the workspaces. So cookies, regex, setting headers will eat that space. And when you restart, you keep using that workspace, and ESI sub-objects consume the top-object's workspace. Hope that helps. -- Guillaume Quintard On Wed, Oct 26, 2016 at 12:02 AM, Viktor Villafuerte < viktor.villafuerte at optusnet.com.au> wrote: > Hi all, > > Yesterday I saw this error in Varnishlog > > Error out of workspace > > I've done some googling since and it seems that increasing > workspace_client could fix the issue. Further it's suggested that this > is due to excessive cookie processing. However, we don't process any > cookines at all. > > Also I've found an old bug where it's suggested that VCL code ends up in > a loop (using restart feature) and eventually runs out of space. We do > use restart but it's limited to only 3 attempts.. > > Anybody would have any experience with 'out of workspace' error and why > it could occur? > > Any suggestions are welcomed, let me know if more details are needed. > > > thanks > > v > > > -- > Regards > > Viktor Villafuerte > Optus Internet Engineering > t: +61 2 80825265 > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Wed Oct 26 08:43:55 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Wed, 26 Oct 2016 10:43:55 +0200 Subject: varnish 5 and multiple domains In-Reply-To: <6539c570-eddf-454b-702b-d7831b3651b4@beckspaced.com> References: <6539c570-eddf-454b-702b-d7831b3651b4@beckspaced.com> Message-ID: Hi, Definitely not the first version, keep your includes at the top of your file, where we can see them, something like: in nl.vcl: sub nl_recv { do some stuff } in com.vcl: sub com_recv { do some stuff } in default.vcl: sub vcl_recv { if (req.http.Host == "www.example.nl") { call nl_recv; } else if (req.http.Host == "www.example.com") { call com_recv; } } But since you are on 5, you can do something even sexier: https://varnish-cache.org/docs/trunk/users-guide/vcl-separate.html -- Guillaume Quintard On Wed, Oct 26, 2016 at 10:03 AM, Admin Beckspaced wrote: > dear varnish users ;) > > i'm new to varnish and got a question about how to best implement multiple > domains into varnish 5 > > i got quite a few domains hosted on my linux server and would like to > include some domains, exclude others and also switch between different vcl's > based on the domain. > > i've looked around on the internet but thought why not ask the mailing list > pro's for advice ;) > > should i use the new load and label feature with version 5 as explained > here: > > https://varnish-cache.org/docs/trunk/users-guide/vcl-separate.html > > or should i use the include feature as explained here: > > https://www.varnish-cache.org/lists/pipermail/varnish-misc/2011-January/019815.html > > e.g. > > if (req.http.Host == "domainx.be") { > include "/etc/varnish/domainx.be.vcl"; > } > > or, instead of include, should i put all in a single vcl file? > > sub vcl_recv { > > if (req.http.Host == "www.example.nl") > { > do some stuff; > } > elsif (req.http.Host == "www.example.com") > { > do some stuff; > } > > } > > which approach would you recommend for multiple domains? > > thanks for your hints & tips ;) > becki > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc From guillaume at varnish-software.com Wed Oct 26 12:06:14 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Wed, 26 Oct 2016 14:06:14 +0200 Subject: Vanishstat -f in 5.0.0 In-Reply-To: <6B000CD8553C4611B1C5BC901584D25B@MasterPC> References: <6B000CD8553C4611B1C5BC901584D25B@MasterPC> Message-ID: Confirmed. Would you mind opening a ticket about it? -- Guillaume Quintard On Tue, Oct 25, 2016 at 5:37 PM, Reinis Rozitis wrote: > Hi, > has there something changed in 5.0.0 regarding varnishstat (or is there a > bug) as the -f doesn't seem to work. > > varnishstat -V > varnishstat (varnish-5.0.0 revision 99d036f) > > > While I can get the full output (also with -1 flag): > > varnishstat -1 | grep MAIN.cache_hit > MAIN.cache_hit 22615 17.26 Cache hits > MAIN.cache_hitpass 5 0.00 Cache hits for pass > > > varnishstat -1 -f MAIN.cache_hit > > doesn't return anything. > > > > 4.1.x works fine: > > varnishstat -V > varnishstat (varnish-4.1.3 revision 5e3b6d2) > > varnishstat -1 -f MAIN.cache_hit > MAIN.cache_hit 89691619 10.69 Cache hits > > > rr > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Wed Oct 26 14:14:46 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Wed, 26 Oct 2016 16:14:46 +0200 Subject: Vanishstat -f in 5.0.0 In-Reply-To: References: <6B000CD8553C4611B1C5BC901584D25B@MasterPC> Message-ID: Actually, don't bother, I did it: https://github.com/varnishcache/varnish-cache/issues/2118 And Federico has a fix on the way, thanks a lot for reporting! -- Guillaume Quintard On Wed, Oct 26, 2016 at 2:06 PM, Guillaume Quintard < guillaume at varnish-software.com> wrote: > Confirmed. Would you mind opening a ticket about it? > > -- > Guillaume Quintard > > On Tue, Oct 25, 2016 at 5:37 PM, Reinis Rozitis wrote: > >> Hi, >> has there something changed in 5.0.0 regarding varnishstat (or is there a >> bug) as the -f doesn't seem to work. >> >> varnishstat -V >> varnishstat (varnish-5.0.0 revision 99d036f) >> >> >> While I can get the full output (also with -1 flag): >> >> varnishstat -1 | grep MAIN.cache_hit >> MAIN.cache_hit 22615 17.26 Cache hits >> MAIN.cache_hitpass 5 0.00 Cache hits for pass >> >> >> varnishstat -1 -f MAIN.cache_hit >> >> doesn't return anything. >> >> >> >> 4.1.x works fine: >> >> varnishstat -V >> varnishstat (varnish-4.1.3 revision 5e3b6d2) >> >> varnishstat -1 -f MAIN.cache_hit >> MAIN.cache_hit 89691619 10.69 Cache hits >> >> >> rr >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From r at roze.lv Wed Oct 26 14:53:37 2016 From: r at roze.lv (Reinis Rozitis) Date: Wed, 26 Oct 2016 17:53:37 +0300 Subject: Vanishstat -f in 5.0.0 In-Reply-To: References: <6B000CD8553C4611B1C5BC901584D25B@MasterPC> Message-ID: <000f01d22f98$bbc8a8f0$3359fad0$@roze.lv> Thx for quick response and resolution :) rr From: Guillaume Quintard [mailto:guillaume at varnish-software.com] Sent: tre?diena, 2016. gada 26. oktobris 17:15 To: Reinis Rozitis Cc: varnish-misc Subject: Re: Vanishstat -f in 5.0.0 Actually, don't bother, I did it: https://github.com/varnishcache/varnish-cache/issues/2118 And Federico has a fix on the way, thanks a lot for reporting! -- Guillaume Quintard On Wed, Oct 26, 2016 at 2:06 PM, Guillaume Quintard wrote: Confirmed. Would you mind opening a ticket about it? -- Guillaume Quintard On Tue, Oct 25, 2016 at 5:37 PM, Reinis Rozitis wrote: Hi, has there something changed in 5.0.0 regarding varnishstat (or is there a bug) as the -f doesn't seem to work. varnishstat -V varnishstat (varnish-5.0.0 revision 99d036f) While I can get the full output (also with -1 flag): varnishstat -1 | grep MAIN.cache_hit MAIN.cache_hit 22615 17.26 Cache hits MAIN.cache_hitpass 5 0.00 Cache hits for pass varnishstat -1 -f MAIN.cache_hit doesn't return anything. 4.1.x works fine: varnishstat -V varnishstat (varnish-4.1.3 revision 5e3b6d2) varnishstat -1 -f MAIN.cache_hit MAIN.cache_hit 89691619 10.69 Cache hits rr _______________________________________________ varnish-misc mailing list varnish-misc at varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc From hugues at betabrand.com Wed Oct 26 18:22:51 2016 From: hugues at betabrand.com (Hugues Alary) Date: Wed, 26 Oct 2016 11:22:51 -0700 Subject: BAN - Memory consumption exploding under load Message-ID: Hi there, We're using Varnish 4.1.3 here and are running into an issue when using a lot of BANs. We make a heavy use of BAN-lurker friendly bans, and it seems that overtime BANs accumulate. They accumulate to the point where varnish suddenly starts using all the memory available on the system and that only a reboot of the server fixes. The CPU usage also seem to explode. The accumulation of BANs seems to be happening when we have a sudden spike in traffic triggering in turn a sudden spike in BAN requests. Not sure how useful the following information is but: I was able earlier to run `sudo varnishstat -1 | grep ban` on the server to gather some information, but unfortunately lost all the exact numbers. I do however recall seeing MAIN.bans, MAIN.ban_added and MAIN.ban_created ever increasing and MAIN.ban_deleted staying to the same constant number. MAIN.bans_persisted_bytes and MAIN.bans_persisted_fragmentation showed both around 410MBytes. We noticed a very similar issue on varnish 3.0.6 where sending too many BAN requests to Varnish would make it crawl to a halt (without necessarily eating all the memory and CPU though). We upgraded to Varnish 4.1.3 hoping it would help, but seems like it didn't. I also, thanks to varnishlog, realized that we were sending a lot of BAN that were invalid regular expressions. I fixed the issue with the regular expressions and varnishlog isn't showing errors anymore, but currently have no idea if it had anything to do with my original issue. Has anybody ever run into this issue? I found an old bug report reporting the same kind of behavior: https://www.varnish-cache.org/trac/ticket/1701 Cheers, -Hugues -------------- next part -------------- An HTML attachment was scrubbed... URL: From slink at schokola.de Thu Oct 27 16:02:52 2016 From: slink at schokola.de (Nils Goroll) Date: Thu, 27 Oct 2016 18:02:52 +0200 Subject: BAN - Memory consumption exploding under load In-Reply-To: References: Message-ID: Hi, we've added a bunch of ban and ban lurker improvements which did not get backported to 4.1.3 Please upgrade to 5.0 or master. Thanks, Nils From slink at schokola.de Thu Oct 27 16:07:43 2016 From: slink at schokola.de (Nils Goroll) Date: Thu, 27 Oct 2016 18:07:43 +0200 Subject: BAN - Memory consumption exploding under load In-Reply-To: References: Message-ID: <9534a5be-3765-ff6e-b37f-9c2cb704e153@schokola.de> BTW, should you still see the issue with 5.0/master, getting hold of varnishstat is absolutely vital for any further analysis. Also, getting regular outputs of varnishadm ban.list would be really helpful (it changes over time). Thanks, Nils From hugues at betabrand.com Thu Oct 27 17:02:36 2016 From: hugues at betabrand.com (Hugues Alary) Date: Thu, 27 Oct 2016 10:02:36 -0700 Subject: BAN - Memory consumption exploding under load In-Reply-To: <9534a5be-3765-ff6e-b37f-9c2cb704e153@schokola.de> References: <9534a5be-3765-ff6e-b37f-9c2cb704e153@schokola.de> Message-ID: Hi Nils, I'll upgrade to Varnish 5.0 and see if the problem persists. Cheers, -Hugues On Thu, Oct 27, 2016 at 9:07 AM, Nils Goroll wrote: > BTW, should you still see the issue with 5.0/master, getting hold of > varnishstat > is absolutely vital for any further analysis. Also, getting regular > outputs of > varnishadm ban.list would be really helpful (it changes over time). > > Thanks, Nils > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxcvbn4038 at gmail.com Thu Oct 27 18:41:42 2016 From: zxcvbn4038 at gmail.com (CJ Ess) Date: Thu, 27 Oct 2016 14:41:42 -0400 Subject: Cache survived varnish restart? Message-ID: I saw something today that I can't explain - I upgraded Varnish 4.1.0 to v4.1.3 by restarting varnishd on a server. I can see that inode of the executable and libraries are right for the updated files in the running varnishd via /proc//maps (on Linux), I can see that no other varnishd processes are running. I expected that restarting varnish would cause the cache to be wiped and require some period of warmup, however the hit rate held steady at 98% and I don't know why. I'm very happy to be able to do this, its going to make the rest of my upgrades non-disruptive, but I'd like to be able to explain whats happening behind the scenes. Can anyone tell me how the cache seemed to survive the restart? -------------- next part -------------- An HTML attachment was scrubbed... URL: From ayberk.kimsesiz at gmail.com Thu Oct 27 21:25:40 2016 From: ayberk.kimsesiz at gmail.com (Ayberk Kimsesiz) Date: Fri, 28 Oct 2016 00:25:40 +0300 Subject: Varnish 4.1.3 starting error Message-ID: Hi, How can I solve this? *Starting Varnish Cache: /usr/sbin/varnishd: error while loading shared libraries: libvarnish.so: cannot open shared object file: No such file or directory [FAILED]* OS: Centos 6.5 Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From ianmac51 at gmail.com Thu Oct 27 22:34:24 2016 From: ianmac51 at gmail.com (Ian Macdonald) Date: Thu, 27 Oct 2016 23:34:24 +0100 Subject: Varnish 4.1.3 starting error In-Reply-To: References: Message-ID: Hi, How did you install varnish if via separate rpms then you probably are missing the libs varnish-libs-4.1.3-1.el6.x86_64.rpm you will also need jemalloc, if you did go this way cheers Ian On 27 October 2016 at 22:25, Ayberk Kimsesiz wrote: > Hi, > > How can I solve this? > > Starting Varnish Cache: /usr/sbin/varnishd: error while loading shared > libraries: libvarnish.so: cannot open shared object file: No such file or > directory [FAILED] > > OS: Centos 6.5 > > Thanks > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc From ayberk.kimsesiz at gmail.com Thu Oct 27 22:44:33 2016 From: ayberk.kimsesiz at gmail.com (Ayberk Kimsesiz) Date: Fri, 28 Oct 2016 01:44:33 +0300 Subject: Varnish 4.1.3 starting error In-Reply-To: References: Message-ID: Hi Ian, package varnish-libs-devel-4.1.3-1.el6.x86_64 is already installed package varnish-libs-4.1.3-1.el6.x86_64 is already installed Do you have another suggestion? Thanks. 2016-10-28 1:34 GMT+03:00 Ian Macdonald : > Hi, > > How did you install varnish if via separate rpms then you probably are > missing the libs > > varnish-libs-4.1.3-1.el6.x86_64.rpm > > you will also need jemalloc, if you did go this way > > > cheers > > Ian > > On 27 October 2016 at 22:25, Ayberk Kimsesiz > wrote: > > Hi, > > > > How can I solve this? > > > > Starting Varnish Cache: /usr/sbin/varnishd: error while loading shared > > libraries: libvarnish.so: cannot open shared object file: No such file or > > directory [FAILED] > > > > OS: Centos 6.5 > > > > Thanks > > > > _______________________________________________ > > varnish-misc mailing list > > varnish-misc at varnish-cache.org > > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ayberk.kimsesiz at gmail.com Thu Oct 27 22:58:57 2016 From: ayberk.kimsesiz at gmail.com (Ayberk Kimsesiz) Date: Fri, 28 Oct 2016 01:58:57 +0300 Subject: Varnish 4.1.3 starting error In-Reply-To: <3BFAED69-92C4-442E-B6D1-0A1A50B81ADE@posteo.de> References: <3BFAED69-92C4-442E-B6D1-0A1A50B81ADE@posteo.de> Message-ID: Hi Daniel, Do you have another suggestion? Thanks 2016-10-28 1:12 GMT+03:00 Daniel Parthey : > You might retry with SELinux disabled. > Enter the following code on root console: > > setenforce 0 > > You can permanently disable selinux in /etc/sysconfig/selinux and reboot > the machine for this to become effective. > > Regards > pada -------------- next part -------------- An HTML attachment was scrubbed... URL: From admin at beckspaced.com Fri Oct 28 08:51:22 2016 From: admin at beckspaced.com (Admin Beckspaced) Date: Fri, 28 Oct 2016 10:51:22 +0200 Subject: is there a default vlc for varnish 5 Message-ID: <1adfa99c-8344-54e6-dc27-e2bf9b7a3d91@beckspaced.com> hello varnish users, I'm using varnish 5 and I'm actually quite new to varnish, so i apologize in advance, if I'm asking 'stupid' questions ;) I'm currently thinking about the following: sub vcl_hash { # If the client supports compression, keep that in a different cache if (req.http.Accept-Encoding) { hash_data(req.http.Accept-Encoding); } } do I need to do this? or is there some default vlc_hash by varnish, which automatically does those things by itself? and if so, where can i find the default varnish vlc? github? link to the documentation? thanks & greetings becki From guillaume at varnish-software.com Fri Oct 28 09:08:13 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Fri, 28 Oct 2016 11:08:13 +0200 Subject: is there a default vlc for varnish 5 In-Reply-To: <1adfa99c-8344-54e6-dc27-e2bf9b7a3d91@beckspaced.com> References: <1adfa99c-8344-54e6-dc27-e2bf9b7a3d91@beckspaced.com> Message-ID: Run "varnishadm vcl.show -v boot", this will show you what files are used to build the vcl, and you'll notice "builtin.vcl". This is the code that gets executed when you don't return from your subroutines. You can also try and "locate builtin.vcl" to get that code. Now, to answer your second question, don't care about it, but have your backend reply with a vary header if the accept-encoding header is important. -- Guillaume Quintard On Fri, Oct 28, 2016 at 10:51 AM, Admin Beckspaced wrote: > hello varnish users, > > I'm using varnish 5 and I'm actually quite new to varnish, so i apologize > in advance, if I'm asking 'stupid' questions ;) > > I'm currently thinking about the following: > > sub vcl_hash { > > # If the client supports compression, keep that in a different cache > if (req.http.Accept-Encoding) { > > hash_data(req.http.Accept-Encoding); > > } > > } > > do I need to do this? or is there some default vlc_hash by varnish, which > automatically does those things by itself? > and if so, where can i find the default varnish vlc? github? link to the > documentation? > > thanks & greetings > becki > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From admin at beckspaced.com Fri Oct 28 09:45:20 2016 From: admin at beckspaced.com (Admin Beckspaced) Date: Fri, 28 Oct 2016 11:45:20 +0200 Subject: is there a default vlc for varnish 5 In-Reply-To: References: <1adfa99c-8344-54e6-dc27-e2bf9b7a3d91@beckspaced.com> Message-ID: <6fb7aa75-ea35-231d-8897-2d461c7ba08b@beckspaced.com> thanks for your fast reply. much appreciated ;) Am 28.10.2016 um 11:08 schrieb Guillaume Quintard: > Run "varnishadm vcl.show -v boot", this will show you what files are > used to build the vcl, and you'll notice "builtin.vcl". This is the > code that gets executed when you don't return from your subroutines. > You can also try and "locate builtin.vcl" to get that code. > > Now, to answer your second question, don't care about it, but have > your backend reply with a vary header if the accept-encoding header is > important. > > -- > Guillaume Quintard From mattias at nucleus.be Fri Oct 28 09:51:38 2016 From: mattias at nucleus.be (Mattias Geniar) Date: Fri, 28 Oct 2016 09:51:38 +0000 Subject: is there a default vlc for varnish 5 In-Reply-To: <1adfa99c-8344-54e6-dc27-e2bf9b7a3d91@beckspaced.com> References: <1adfa99c-8344-54e6-dc27-e2bf9b7a3d91@beckspaced.com> Message-ID: <6955C88A-2620-4296-B5FB-B1F907A0C458@nucleus.be> > do I need to do this? or is there some default vlc_hash by varnish, > which automatically does those things by itself? > and if so, where can i find the default varnish vlc? github? link to the > documentation? While it?s nothing official from Varnish, there is a repository of Varnish 5.x configurations I maintain here: https://github.com/mattiasgeniar/varnish-5.0-configuration-templates It?s heavily based on the 4.x configurations: https://github.com/mattiasgeniar/varnish-4.0-configuration-templates Which in turn is based on the 3.x configs: https://github.com/mattiasgeniar/varnish-3.0-configuration-templates (you get the gist) It may be a bit too much if you?re just getting started, but for some boilerplate code and sane defaults, I would recommend it. Mattias From admin at beckspaced.com Fri Oct 28 18:05:39 2016 From: admin at beckspaced.com (Admin Beckspaced) Date: Fri, 28 Oct 2016 20:05:39 +0200 Subject: varnish 5 vmod for geoip country lookup Message-ID: <5e19fa4a-98cf-51f0-39c4-255fce02ffc4@beckspaced.com> hello varnish users, is there a working, up-to-date, vmod for varnish 5 which gives me a country code based on the client ip address? so i could do things like described in the docs: https://www.varnish-cache.org/docs/trunk/users-guide/vcl-hashing.html In vcl_recv: set req.http.X-Country-Code = geoip.lookup(client.ip); And then add a vcl_hash: sub vcl_hash { hash_data(req.http.X-Country-Code); } alternatively, i think i could also let the backend send a http header X-country-code and then add this to the hash instead? sub vcl_hash { hash_data(beresp.http.X-Country-Code); } am I right on this? greetings & thanks a lot for varnish! the more i play with it the more i discover the power of it ;) all the best becki From admin at beckspaced.com Sat Oct 29 07:02:52 2016 From: admin at beckspaced.com (Admin Beckspaced) Date: Sat, 29 Oct 2016 09:02:52 +0200 Subject: varnish overhead when only passing to backend without caching Message-ID: <4a0c835c-0527-a375-8440-c6a136cba01f@beckspaced.com> hello again, my current setup is varnish 5 in front of apache web server with 30 different domains / websites in my vlc i can switch between sites and decide to do varnish caching or just passing to the backend with out caching. what's the overhead / delay with varnish when i just pass everything to the backend and back to the client without varnish doing any caching? i mean there must be some overhead as varnish is in between ... ... but how big is it? time delay? does anyone here have any experience with this? thanks & greetings becki From albert.tollkuci at gmail.com Sat Oct 29 08:32:47 2016 From: albert.tollkuci at gmail.com (=?UTF-8?Q?Albert_Tollku=C3=A7i?=) Date: Sat, 29 Oct 2016 10:32:47 +0200 Subject: varnish overhead when only passing to backend without caching In-Reply-To: <4a0c835c-0527-a375-8440-c6a136cba01f@beckspaced.com> References: <4a0c835c-0527-a375-8440-c6a136cba01f@beckspaced.com> Message-ID: Varnish is very very fast. That being said, it depends on how many hits your sites are getting. If they are less than a few hundred thousands per day, you should not worry about Varnish overhead. Albert On Sat, Oct 29, 2016 at 9:02 AM, Admin Beckspaced wrote: > hello again, > > my current setup is varnish 5 in front of apache web server with 30 > different domains / websites > in my vlc i can switch between sites and decide to do varnish caching or > just passing to the backend with out caching. > > what's the overhead / delay with varnish when i just pass everything to > the backend and back to the client without varnish doing any caching? > > i mean there must be some overhead as varnish is in between ... > ... but how big is it? time delay? > > does anyone here have any experience with this? > > thanks & greetings > becki > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -- Web: http://www.tollkuci.com Follow me on: LinkedIn Google+ Facebook Twitter Career 2.0 -------------- next part -------------- An HTML attachment was scrubbed... URL: From admin at beckspaced.com Sat Oct 29 09:06:44 2016 From: admin at beckspaced.com (Admin Beckspaced) Date: Sat, 29 Oct 2016 11:06:44 +0200 Subject: varnish5 - how to view what currently is in cache? Message-ID: <1a4ea136-7e2c-a429-8f58-759c8dff4517@beckspaced.com> hello again varnish users, i have been looking in the varnish 5 documents. found useful things like: varnishadm varnishtop varnishhist varnishstat but how would i proceed if i want to know what actually is in the cache? e.g. view a cached version of an URL: http://mydomain.com/somepage or view cached versions based on custom hash_data i added via vcl http://mydomain.com/somepage - hash_data lang=en http://mydomain.com/somepage - hash_data lang=de http://mydomain.com/somepage - hash_data lang=fr is it possible to get a list of all URL's in cache? and view a cached object based on the URL? perhaps custom hash_data? e.g. varnishadm -view -url http://mydomain.com/somepage e.g. varnishadm -view -url http://mydomain.com/somepage -hash_data lang=en e.g. varnishadm -view -url http://mydomain.com/somepage -hash_data lang=de any thoughts or hints would be great ;) thanks & greetings becki From guillaume at varnish-software.com Sat Oct 29 09:14:00 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Sat, 29 Oct 2016 11:14:00 +0200 Subject: varnish overhead when only passing to backend without caching In-Reply-To: <4a0c835c-0527-a375-8440-c6a136cba01f@beckspaced.com> References: <4a0c835c-0527-a375-8440-c6a136cba01f@beckspaced.com> Message-ID: Why not measure it? The overhead will mainly be the time to go trough two network links instead of one, but since Varnish pools connections, you don't even have that extra connection cost. As to the delay introduced by Varnish, you can have a look at this: https://github.com/varnishcache/varnish-cache/commit/d7b9296012f6a5d41c48198ab2ea991a173c3b2c (this is talking about the time between opening and closing the session). I think it's telling. On Oct 29, 2016 09:12, "Admin Beckspaced" wrote: > hello again, > > my current setup is varnish 5 in front of apache web server with 30 > different domains / websites > in my vlc i can switch between sites and decide to do varnish caching or > just passing to the backend with out caching. > > what's the overhead / delay with varnish when i just pass everything to > the backend and back to the client without varnish doing any caching? > > i mean there must be some overhead as varnish is in between ... > ... but how big is it? time delay? > > does anyone here have any experience with this? > > thanks & greetings > becki > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From apj at mutt.dk Mon Oct 31 06:37:09 2016 From: apj at mutt.dk (Andreas Plesner) Date: Mon, 31 Oct 2016 07:37:09 +0100 Subject: varnish 5 vmod for geoip country lookup In-Reply-To: <5e19fa4a-98cf-51f0-39c4-255fce02ffc4@beckspaced.com> References: <5e19fa4a-98cf-51f0-39c4-255fce02ffc4@beckspaced.com> Message-ID: <20161031063709.GW23724@nerd.dk> On Fri, Oct 28, 2016 at 08:05:39PM +0200, Admin Beckspaced wrote: > > alternatively, i think i could also let the backend send a http header > X-country-code and then add this to the hash instead? No, that will be too late. You need the country code to do the lookup without talking to the backend. -- Andreas From apj at mutt.dk Mon Oct 31 06:42:20 2016 From: apj at mutt.dk (Andreas Plesner) Date: Mon, 31 Oct 2016 07:42:20 +0100 Subject: varnish5 - how to view what currently is in cache? In-Reply-To: <1a4ea136-7e2c-a429-8f58-759c8dff4517@beckspaced.com> References: <1a4ea136-7e2c-a429-8f58-759c8dff4517@beckspaced.com> Message-ID: <20161031064220.GX23724@nerd.dk> On Sat, Oct 29, 2016 at 11:06:44AM +0200, Admin Beckspaced wrote: > > but how would i proceed if i want to know what actually is in the cache? > > e.g. view a cached version of an URL: http://mydomain.com/somepage Fetch it with a HTTP client. -- Andreas From guillaume at varnish-software.com Mon Oct 31 09:21:00 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Mon, 31 Oct 2016 10:21:00 +0100 Subject: Cache survived varnish restart? In-Reply-To: References: Message-ID: I don't think that's possible :-) That would be nice though. -- Guillaume Quintard On Thu, Oct 27, 2016 at 8:41 PM, CJ Ess wrote: > I saw something today that I can't explain - I upgraded Varnish 4.1.0 to > v4.1.3 by restarting varnishd on a server. I can see that inode of the > executable and libraries are right for the updated files in the running > varnishd via /proc//maps (on Linux), I can see that no other varnishd > processes are running. I expected that restarting varnish would cause the > cache to be wiped and require some period of warmup, however the hit rate > held steady at 98% and I don't know why. > > I'm very happy to be able to do this, its going to make the rest of my > upgrades non-disruptive, but I'd like to be able to explain whats happening > behind the scenes. Can anyone tell me how the cache seemed to survive the > restart? > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From admin at beckspaced.com Mon Oct 31 09:23:50 2016 From: admin at beckspaced.com (Admin Beckspaced) Date: Mon, 31 Oct 2016 10:23:50 +0100 Subject: varnish5 - how to view what currently is in cache? In-Reply-To: <20161031064220.GX23724@nerd.dk> References: <1a4ea136-7e2c-a429-8f58-759c8dff4517@beckspaced.com> <20161031064220.GX23724@nerd.dk> Message-ID: <5a0af452-abcd-c73c-462d-e94ef14a426f@beckspaced.com> Am 31.10.2016 um 07:42 schrieb Andreas Plesner: > On Sat, Oct 29, 2016 at 11:06:44AM +0200, Admin Beckspaced wrote: >> but how would i proceed if i want to know what actually is in the cache? >> >> e.g. view a cached version of an URL: http://mydomain.com/somepage > Fetch it with a HTTP client. > thanks andreas, yes, i know i can fetch it with a http client to see the cached version ;) but i'm looking more into finding out what URL's there are in the cache to find the culprits as i never know what clients out there request. in the past i was working with other caching systems, like PEAR Cache Lite, which is based on PHP. and there I can easily see what URLs are in the cache. this way i could easily see the 'wrong' cached URLs and re-adjust my caching system to get a high positive hit rate. for example, i got URL like: http://mydomain.com/?param=foo&doe=john but anyone out there is allowed to request stuff like: http://mydomain.com/?param=foo&doe=john&culprit=this&anotherculprit=that the additional params the client sends actually do nothing but as far as I understood, varnish will cache based on URLs ... so ending up with 2 cached versions ... so ... to spot those culprits i would like to know if there is a list of URLs / cached objects in varnish that i can view ... investigate to then adjust the vlc to filter out those bad cached to get a higher positive hit rate. does this make sense? is something like that available in varnish 5? thanks & greetings becki From apj at mutt.dk Mon Oct 31 09:36:51 2016 From: apj at mutt.dk (Andreas Plesner) Date: Mon, 31 Oct 2016 10:36:51 +0100 Subject: varnish5 - how to view what currently is in cache? In-Reply-To: <5a0af452-abcd-c73c-462d-e94ef14a426f@beckspaced.com> References: <1a4ea136-7e2c-a429-8f58-759c8dff4517@beckspaced.com> <20161031064220.GX23724@nerd.dk> <5a0af452-abcd-c73c-462d-e94ef14a426f@beckspaced.com> Message-ID: <20161031093651.GY23724@nerd.dk> On Mon, Oct 31, 2016 at 10:23:50AM +0100, Admin Beckspaced wrote: > > but i'm looking more into finding out what URL's there are in the cache to > find the culprits as i never know what clients out there request. You can't. When they're in the cache, the hash is the only key. You can use varnishncsa formats to log cache misses. -- Andreas