From varnish-bugs at varnish-cache.org Fri Oct 2 12:31:07 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Fri, 02 Oct 2015 12:31:07 -0000 Subject: [Varnish] #1794: varnish*-tools do not die when terminal disconnects Message-ID: <041.5e0136e0890c547547da2629f527e416@varnish-cache.org> #1794: varnish*-tools do not die when terminal disconnects -------------------+-------------------- Reporter: kwy | Type: defect Status: new | Priority: normal Milestone: | Component: build Version: 4.0.3 | Severity: normal Keywords: | -------------------+-------------------- Open a varnishtop or varnishhist in a terminal. Then close the terminal, and the program just launched keeps running. Often the program will spin on 100% CPU and need a SIGKILL to terminate. We expect the program to terminate. Verified on RHEL6 and Arch current, so it's not a distribution-specific issue. -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Fri Oct 2 14:09:13 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Fri, 02 Oct 2015 14:09:13 -0000 Subject: [Varnish] #1794: varnish*-tools do not die when terminal disconnects In-Reply-To: <041.5e0136e0890c547547da2629f527e416@varnish-cache.org> References: <041.5e0136e0890c547547da2629f527e416@varnish-cache.org> Message-ID: <056.c925508fde16409031d5246dea9917f4@varnish-cache.org> #1794: varnish*-tools do not die when terminal disconnects --------------------+-------------------- Reporter: kwy | Owner: Type: defect | Status: new Priority: normal | Milestone: Component: build | Version: 4.0.3 Severity: normal | Resolution: Keywords: | --------------------+-------------------- Comment (by lkarsten): I can reproduce this on 82a09dc. varnishist process: {{{ root at sierra:~# gdb -p 21658 --batch --eval-command="bt full" [New LWP 21659] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". 0x00007fde11b9153d in nanosleep () at ../sysdeps/unix/syscall- template.S:81 81 ../sysdeps/unix/syscall-template.S: No such file or directory. #0 0x00007fde11b9153d in nanosleep () at ../sysdeps/unix/syscall- template.S:81 No locals. #1 0x00000000004051e0 in VTIM_sleep (t=0,01) at ../../lib/libvarnish/vtim.c:365 ts = {tv_sec = 0, tv_nsec = 10000000} #2 0x000000000040683e in VUT_Main () at ../../lib/libvarnishtools/vut.c:397 c = 0x0 i = 0 __func__ = "VUT_Main" #3 0x000000000040342f in main (argc=1, argv=0x7fffe14de6f8) at varnishhist.c:468 i = 2 colon = 0x40691d <__libc_csu_init+77> "H\203\303\001H9\353u\352H\203\304\b[]A\\A]A^A_?f.\017\037\204" profile = 0x4069d4 "responsetime" thr = 140591663175424 fnum = 3 cli_p = {name = 0x0, tag = SLT__Bogus, prefix = 0x0, field = 0, hist_low = 0, hist_high = 0} __func__ = "main" }}} -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Fri Oct 2 14:31:42 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Fri, 02 Oct 2015 14:31:42 -0000 Subject: [Varnish] #1794: varnish*-tools do not die when terminal disconnects In-Reply-To: <041.5e0136e0890c547547da2629f527e416@varnish-cache.org> References: <041.5e0136e0890c547547da2629f527e416@varnish-cache.org> Message-ID: <056.a98289f4ac5cc4ad2f74bc9b3207db2f@varnish-cache.org> #1794: varnish*-tools do not die when terminal disconnects --------------------+-------------------- Reporter: kwy | Owner: Type: defect | Status: new Priority: normal | Milestone: Component: build | Version: 4.0.3 Severity: normal | Resolution: Keywords: | --------------------+-------------------- Comment (by lkarsten): This is most likely related to there being no sighup handler defined. Following patch removes the behavior for me: {{{ diff --git a/bin/varnishhist/varnishhist.c b/bin/varnishhist/varnishhist.c index 310eccc..d40264f 100644 --- a/bin/varnishhist/varnishhist.c +++ b/bin/varnishhist/varnishhist.c @@ -133,6 +133,12 @@ profiles[] = { static struct profile *active_profile; +static int __match_proto__(VUT_cb_f) +sighup_handler(void) +{ + exit(1); +} + static void update(void) { @@ -464,6 +470,7 @@ main(int argc, char **argv) exit(1); } VUT.dispatch_f = &accumulate; + VUT.sighup_f = &sighup_handler; VUT.dispatch_priv = NULL; VUT_Main(); end_of_file = 1; }}} Downside is that we don't clean up what curses has done, so the terminal becomes garbled if it still exists. ("reset" fixes, of course) Same sitation on varnishtop. -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Fri Oct 2 16:32:28 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Fri, 02 Oct 2015 16:32:28 -0000 Subject: [Varnish] #1794: varnish*-tools do not die when terminal disconnects In-Reply-To: <041.5e0136e0890c547547da2629f527e416@varnish-cache.org> References: <041.5e0136e0890c547547da2629f527e416@varnish-cache.org> Message-ID: <056.3f7a6233eabfa04d6092cb60cc31845b@varnish-cache.org> #1794: varnish*-tools do not die when terminal disconnects --------------------+-------------------- Reporter: kwy | Owner: Type: defect | Status: new Priority: normal | Milestone: Component: build | Version: 4.0.3 Severity: normal | Resolution: Keywords: | --------------------+-------------------- Comment (by fgsch): From a quick look at the code I think the right way is to setup a sighup_f function that returns 1, so the loop in VUT_Main() is broken: {{{ if (VUT.sighup && VUT.sighup_f) { /* sighup callback */ VUT.sighup = 0; i = (VUT.sighup_f)(); if (i) break; } }}} btw, rotateout should only be used if the programs are running as daemon (VUT.d_opt equals 1). This looks like a bug to me in the current code. -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Fri Oct 2 18:56:40 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Fri, 02 Oct 2015 18:56:40 -0000 Subject: [Varnish] #1795: Man page issues in 4.1 Message-ID: <043.3248b86a133da7a2e2f3f47447587d62@varnish-cache.org> #1795: Man page issues in 4.1 ---------------------------+------------------- Reporter: Dridi | Owner: Type: documentation | Status: new Priority: normal | Milestone: Component: documentation | Version: 4.1.0 Severity: normal | Keywords: ---------------------------+------------------- I've been skimming through the docs while working on a vmod and found the following issues: varnish-cli(7) doesn't mention vcl.state. Other places documenting vcl.state always show the temperature param as optional [auto|warm|cold] instead of a mandatory notation like maybe . The copyright notice is not dated or missing in varnish-counters(7), vmod_directors(3), vmod_std(3), vsl(7), vsl-query(7). YMMV This is not comprehensive, just what I found at pseudo-random. -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Mon Oct 5 09:41:41 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Mon, 05 Oct 2015 09:41:41 -0000 Subject: [Varnish] #1796: Assert error in SES_Proto_Req Message-ID: <045.2141fa04002dfbf25286642bf4066bde@varnish-cache.org> #1796: Assert error in SES_Proto_Req --------------------------+---------------------- Reporter: llavaud | Type: defect Status: new | Priority: normal Milestone: | Component: varnishd Version: 4.1.0 | Severity: normal Keywords: Assert error | --------------------------+---------------------- {{{ Oct 5 10:48:35 webcache12 varnishd[303280]: Child (303299) Panic message: Assert error in SES_Proto_Req(), cache/cache_session.c line 320: Condition((wrk->v1l) == 0) not true. thread = (cache-worker) version = varnish-4.1.0 revision 3041728 ident = Linux,3.2.0-4-amd64,x86_64,-junix,-sfile,-smalloc,-hcritbit,epoll Backtrace: 0x4351b6: varnishd() [0x4351b6] 0x43b58c: varnishd(SES_Proto_Req+0xcc) [0x43b58c] 0x44a212: varnishd() [0x44a212] 0x44a65b: varnishd() [0x44a65b] 0x7f7f243e0b50: libpthread.so.0(+0x6b50) [0x7f7f243e0b50] 0x7f7f2412a95d: libc.so.6(clone+0x6d) [0x7f7f2412a95d] req = 0x7f660f15e020 { vxid = 0, step = R_STP_DELIVER, req_body = R_BODY_INIT, restarts = 0, esi_level = 0, sp = 0x7f66066a5120 { fd = 296, vxid = 2883737, client = 199.30.25.233 45442, step = S_STP_H1NEWREQ, }, ws = 0x7f660f15e200 { id = "req", {s,f,r,e} = {0x7f660f160000,0x7f660f160000,+32768,+253944}, }, http_conn = 0x7f660f15e128 { fd = 296, doclose = NULL, ws = 0x7f660f15e200, {rxbuf_b, rxbuf_e} = {0x7f660f160000, 0x7f660f160000}, {pipeline_b, pipeline_e} = {(nil), (nil)}, content_length = -1, body_status = none, first_byte_timeout = 0.000000, between_bytes_timeout = 0.000000, }, http[req] = 0x7f660f15e298 { ws[req] = 0x7f660f15e200, hdrs { "", "/myuri", "HTTP/1.1", "Accept: text/html, */*; q=0.01", "Referer: myreferer", "X-Requested-With: XMLHttpRequest", "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/534+ (KHTML, like Gecko) BingPreview/1.0b", "Content-Type: application/x-www-form-urlencoded; charset=UTF-8", "X-Forwarded-For: myip", "Host: myhost", "Surrogate-Capability: abc=ESI/1.0", "Cookie: user_logged -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Mon Oct 5 18:44:08 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Mon, 05 Oct 2015 18:44:08 -0000 Subject: [Varnish] #1794: varnish*-tools do not die when terminal disconnects In-Reply-To: <041.5e0136e0890c547547da2629f527e416@varnish-cache.org> References: <041.5e0136e0890c547547da2629f527e416@varnish-cache.org> Message-ID: <056.b8c45356f95a17231033d61ff43192b0@varnish-cache.org> #1794: varnish*-tools do not die when terminal disconnects --------------------+-------------------- Reporter: kwy | Owner: Type: defect | Status: new Priority: normal | Milestone: Component: build | Version: 4.0.3 Severity: normal | Resolution: Keywords: | --------------------+-------------------- Comment (by fgsch): Attached patch should fix this and refuse -D without -w. -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Tue Oct 6 09:46:16 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Tue, 06 Oct 2015 09:46:16 -0000 Subject: [Varnish] #1797: Content-Type/Content-Length should miss on 304 requests Message-ID: <048.25fbf3261b6557b2bc77d60ef0f32406@varnish-cache.org> #1797: Content-Type/Content-Length should miss on 304 requests ------------------------+---------------------- Reporter: shakalandy | Type: defect Status: new | Priority: normal Milestone: | Component: varnishd Version: 4.0.3 | Severity: normal Keywords: | ------------------------+---------------------- As far as i understand RFC 2616 correctly the 304 response must not include other entity headers such as Content-Type/Content-Length (http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.5). In our installation with merely default vcl in vcl_deliver varnish includes both Content-Type/Content-Length in a 304 response. This may confuse other upstream caches and may force an unnecessary revalidation of an object. maybe related to #529 ? -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Tue Oct 6 09:48:01 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Tue, 06 Oct 2015 09:48:01 -0000 Subject: [Varnish] #1797: Content-Type/Content-Length should miss on 304 requests In-Reply-To: <048.25fbf3261b6557b2bc77d60ef0f32406@varnish-cache.org> References: <048.25fbf3261b6557b2bc77d60ef0f32406@varnish-cache.org> Message-ID: <063.97144242c5c07edc9994ace35e0eaf6c@varnish-cache.org> #1797: Content-Type/Content-Length should miss on 304 requests ------------------------+-------------------- Reporter: shakalandy | Owner: Type: defect | Status: new Priority: normal | Milestone: Component: varnishd | Version: 4.0.3 Severity: normal | Resolution: Keywords: | ------------------------+-------------------- Comment (by shakalandy): Ticket summary should be "ontent-Type/Content-Length should miss on 304 responses" - sorry! -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Thu Oct 8 01:55:01 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Thu, 08 Oct 2015 01:55:01 -0000 Subject: [Varnish] #1798: Varnish requests painfully slow with large files Message-ID: <045.b9933fc5ccad71abef136d693b22f22f@varnish-cache.org> #1798: Varnish requests painfully slow with large files ---------------------+---------------------- Reporter: nbetham | Type: defect Status: new | Priority: normal Milestone: | Component: varnishd Version: 4.0.3 | Severity: normal Keywords: | ---------------------+---------------------- Hi, I have a Varnish instance running on a server which is being used to host some small and large-ish files. The file them selves range from 300KB to 20GB. I have Varnish setup to use malloc for storage. The server has 170GB of RAM and Varnish is configured to use 160GB for malloc. Currently we have Apache running on the same server as the backend for Varnish. The server is connected to the network with a single 10Gb ethernet link. It's currently running Varnish 4.0.3, Ubuntu 14.04, Kernel 3.13.0-65-generic. When a request comes in for a small file the request completes normally and quickly. However when a request comes in for a large file the connection starts at the local max speed, 1Gbps, and then quickly ramps down to 40Mbps after about 1GB of the file has been downloaded. Meanwhile on the server one of the worker threads is stuck at 100% CPU utilization. Subsequent requests for the same file then complete normally, and at full speed throughout the request, afterwards. For every new large file requested a single worker thread will get stuck at 100% CPU. Here's the VCL running on the server. I stripped it down to the bare minimum to see if anything I had added was causing problems. I have also disabled streaming just to see if Apache was responding slowly, but that is not the case given that most of the large files load from Apache into Varnish within a few seconds. ===== /etc/varnish/default.vcl ===== {{{ # # This is an example VCL file for Varnish. # # It does not do anything by default, delegating control to the # builtin VCL. The builtin VCL is called when there is no explicit # return statement. # # See the VCL chapters in the Users Guide at https://www.varnish- cache.org/docs/ # and http://varnish-cache.org/trac/wiki/VCLExamples for more examples. # Marker to tell the VCL compiler that this VCL has been adapted to the # new 4.0 format. vcl 4.0; # Default backend definition. Set this to point to your content server. backend default { .host = "127.0.0.1"; .port = "7080"; } sub vcl_recv { # Happens before we check if we have this in cache already. # # Typically you clean up the request here, removing cookies you don't need, # rewriting the request, etc. } sub vcl_backend_response { # Happens after we have read the response headers from the backend. # # Here you clean the response headers, removing silly Set-Cookie headers # and other mistakes your backend does. set beresp.ttl = 24 h; set beresp.do_stream = false; return(deliver); } sub vcl_deliver { # Happens when we have all the pieces we need, and are about to send the # response to the client. # # You can do accounting or modifying the final object here. } }}} ===== /etc/default/varnish ===== {{{ # Configuration file for varnish # # /etc/init.d/varnish expects the variables $DAEMON_OPTS, $NFILES and $MEMLOCK # to be set from this shell script fragment. # # Should we start varnishd at boot? Set to "no" to disable. START=yes # Maximum number of open files (for ulimit -n) NFILES=131072 # Maximum locked memory size (for ulimit -l) # Used for locking the shared memory log in memory. If you increase log size, # you need to increase this number as well MEMLOCK=82000 DAEMON_OPTS="-a :6080 \ -T localhost:6082 \ -f /etc/varnish/default.vcl \ -S /etc/varnish/secret \ -s malloc,160GB \ -p send_timeout=864000" }}} Most of what I have read about Varnish is that it should be able to handle large files. Is there something I need to set in order to prevent worker threads from hanging on these large files? It seems like there might be something broken in the system, but i'm not sure. Let me know if you need more info, I would be happy to provide it. Any help with this would be awesome! Thanks, Neil -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Thu Oct 8 13:54:39 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Thu, 08 Oct 2015 13:54:39 -0000 Subject: [Varnish] #1798: Varnish requests painfully slow with large files In-Reply-To: <045.b9933fc5ccad71abef136d693b22f22f@varnish-cache.org> References: <045.b9933fc5ccad71abef136d693b22f22f@varnish-cache.org> Message-ID: <060.130228bf7fdf96bcb1c3359d16d788f8@varnish-cache.org> #1798: Varnish requests painfully slow with large files ----------------------+-------------------- Reporter: nbetham | Owner: Type: defect | Status: new Priority: normal | Milestone: Component: varnishd | Version: 4.0.3 Severity: normal | Resolution: Keywords: | ----------------------+-------------------- Comment (by daghf): Possibly related: https://www.varnish-cache.org/trac/ticket/1788 -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Thu Oct 8 15:00:42 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Thu, 08 Oct 2015 15:00:42 -0000 Subject: [Varnish] #1799: Backend request coalescing fail Message-ID: <044.a197835a5070dbdf3a5e5ef4dd8a3ca5@varnish-cache.org> #1799: Backend request coalescing fail --------------------+--------------------- Reporter: martin | Owner: Type: defect | Status: new Priority: normal | Milestone: Component: build | Version: unknown Severity: normal | Keywords: --------------------+--------------------- See attached test case. If we have a busy (flag OC_F_BUSY set) OC on the object head during HSH_Lookup, and there is an expired candidate available, the request won't be put on waitinglist but return with the expired OC. If this OC is rejected by the VCL (return(miss) from vcl_hit), the request is turned into a pass to the backend. This is logged SLT_VCL_Error: "vcl_hit{} returns fetch without busy object. Doing pass"). If e.g. Varnish is configured to not use grace but with keep (for IMS against the backend), this happens just after the TTL of the object has expired. There is an expired object available, but not within grace, and all requests except the one that inserted the OC_F_BUSY will become passes to the backend until the OC_F_BUSY flag is removed. The attached test case covers this. Another scenario that will exhibit this is when you have a large grace period, but differentiate on how big grace you accept in vcl_hit based on the backend health status. If your object was not refreshed during the healthy backend shorter grace window, you'll get one backend request for each client request until the backend answers (slow backends suffer more). The expected outcome in my opinion is that the requests should have been placed on the waitinglist of the object head until the time the OC_F_BUSY flag is gone. I've been pondering how to fix this. But all attempts at reentering the waitinglist on the OH after HSH_Lookup has completed seems to open race conditions. The only sane way to fix this that I can see is to reintroduce the req.grace attribute, and move the graceability of the expired object back into the HSH_Lookup object head mutex critical region. Martin -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Thu Oct 8 15:02:23 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Thu, 08 Oct 2015 15:02:23 -0000 Subject: [Varnish] #1799: Backend request coalescing fail In-Reply-To: <044.a197835a5070dbdf3a5e5ef4dd8a3ca5@varnish-cache.org> References: <044.a197835a5070dbdf3a5e5ef4dd8a3ca5@varnish-cache.org> Message-ID: <059.07ff78ccac78d0ad6bd33c6555393cda@varnish-cache.org> #1799: Backend request coalescing fail ----------------------+-------------------- Reporter: martin | Owner: Type: defect | Status: new Priority: normal | Milestone: Component: varnishd | Version: trunk Severity: normal | Resolution: Keywords: | ----------------------+-------------------- Changes (by martin): * version: unknown => trunk * component: build => varnishd Comment: Note: Original test case by Dag Haavi Finstad This has been the behaviour since Varnish 4.0 -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Thu Oct 8 16:08:02 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Thu, 08 Oct 2015 16:08:02 -0000 Subject: [Varnish] #1798: Varnish requests painfully slow with large files In-Reply-To: <045.b9933fc5ccad71abef136d693b22f22f@varnish-cache.org> References: <045.b9933fc5ccad71abef136d693b22f22f@varnish-cache.org> Message-ID: <060.c52dff713dc5a8507d84d8974d0d5044@varnish-cache.org> #1798: Varnish requests painfully slow with large files ----------------------+-------------------- Reporter: nbetham | Owner: Type: defect | Status: new Priority: normal | Milestone: Component: varnishd | Version: 4.0.3 Severity: normal | Resolution: Keywords: | ----------------------+-------------------- Comment (by nbetham): Definitely seems like it given the increase in download time seem to exponential in nature. -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Sun Oct 11 18:34:05 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Sun, 11 Oct 2015 18:34:05 -0000 Subject: [Varnish] #1761: 204 responses intermittently delivered as chunk-encoded with length byte = 0 In-Reply-To: <043.4c8744d010240f820f7b2440cfafa2fc@varnish-cache.org> References: <043.4c8744d010240f820f7b2440cfafa2fc@varnish-cache.org> Message-ID: <058.d577544dd5b3a230a4d57f272c2266a5@varnish-cache.org> #1761: 204 responses intermittently delivered as chunk-encoded with length byte = 0 ----------------------+---------------------------------- Reporter: geoff | Owner: geoff Type: defect | Status: new Priority: normal | Milestone: Varnish 4.0 release Component: varnishd | Version: 4.0.3 Severity: normal | Resolution: Keywords: | ----------------------+---------------------------------- Comment (by geoff): Patch for 4.0.3 submitted to patchwork: https://www.varnish- cache.org/patchwork/patch/370/ -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Sun Oct 11 18:37:25 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Sun, 11 Oct 2015 18:37:25 -0000 Subject: [Varnish] #1761: 204 responses intermittently delivered as chunk-encoded with length byte = 0 In-Reply-To: <043.4c8744d010240f820f7b2440cfafa2fc@varnish-cache.org> References: <043.4c8744d010240f820f7b2440cfafa2fc@varnish-cache.org> Message-ID: <058.da6ff60591f387b92367e6e8ffc5ca31@varnish-cache.org> #1761: 204 responses intermittently delivered as chunk-encoded with length byte = 0 ----------------------+---------------------------------- Reporter: geoff | Owner: geoff Type: defect | Status: new Priority: normal | Milestone: Varnish 4.0 release Component: varnishd | Version: 4.0.3 Severity: normal | Resolution: Keywords: | ----------------------+---------------------------------- Comment (by geoff): Apparently I don't have the bits to close this ticket, it can be closed now. -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Sun Oct 11 19:00:20 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Sun, 11 Oct 2015 19:00:20 -0000 Subject: [Varnish] #1800: PRIV_TASK in a vmod object method leads to code that cannot be compiled/linked Message-ID: <043.aa0af4bc03cba56c27b0370c89b02a0b@varnish-cache.org> #1800: PRIV_TASK in a vmod object method leads to code that cannot be compiled/linked -------------------+-------------------- Reporter: geoff | Type: defect Status: new | Priority: normal Milestone: Later | Component: vmod Version: 4.1.0 | Severity: normal Keywords: | -------------------+-------------------- This diff in the vcc declaration of vmod_debug: {{{ diff --git a/lib/libvmod_debug/vmod.vcc b/lib/libvmod_debug/vmod.vcc index dad36ec..9dd9f5b 100644 --- a/lib/libvmod_debug/vmod.vcc +++ b/lib/libvmod_debug/vmod.vcc @@ -89,6 +89,8 @@ $Method TIME .date() You never know when you need a date. +$Method VOID .task(PRIV_TASK) + $Function VOID rot52(HTTP hdr) Encrypt the HTTP header with quad-ROT13 encryption, }}} ... causes about 25 vtc tests to fail, all of them with this error message: {{{ **** v1 0.2 CLI RX| Message from VCC-compiler:\n **** v1 0.2 CLI RX| Could not load VMOD debug\n **** v1 0.2 CLI RX| \tFile name: /home/geoff/Develop/varnish/varnish- cache/lib/libvmod_debug/.libs/libvmod_debug.so\n **** v1 0.2 CLI RX| \tdlerror:: /home/geoff/Develop/varnish/varnish- cache/lib/libvmod_debug/.libs/libvmod_debug.so: undefined symbol: vmod_obj_task\n **** v1 0.2 CLI RX| ('input' Line 5 Pos 16)\n **** v1 0.2 CLI RX| import debug from "/home/geoff/Develop/varnish/varnish- cache/lib/libvmod_debug/.libs/libvmod_debug.so";\n **** v1 0.2 CLI RX| ---------------#####-------------------------------------------------------------------------------------------\n **** v1 0.2 CLI RX| \n **** v1 0.2 CLI RX| Running VCC-compiler failed, exited with 2\n **** v1 0.2 CLI RX| VCL compilation failed ---- v1 0.2 FAIL VCL does not compile }}} In my own VMOD I'm getting a compile error for the C code generated for the VMOD: {{{ **** v1 0.3 CLI RX| Message from C-compiler:\n **** v1 0.3 CLI RX| vgc.c: In function \xe2\x80\x98VGC_function_vcl_backend_response\xe2\x80\x99:\n **** v1 0.3 CLI RX| vgc.c:1178:31: error: \xe2\x80\x98VGC_vmod_bar\xe2\x80\x99 undeclared (first use in this function)\n **** v1 0.3 CLI RX| VRT_priv_task(ctx, &VGC_vmod_bar),\n **** v1 0.3 CLI RX| ^\n **** v1 0.3 CLI RX| vgc.c:1178:31: note: each undeclared identifier is reported only once for each function it appears in\n **** v1 0.3 CLI RX| vgc.c: In function \xe2\x80\x98VGC_function_vcl_recv\xe2\x80\x99:\n **** v1 0.3 CLI RX| vgc.c:1429:31: error: \xe2\x80\x98VGC_vmod_foobar\xe2\x80\x99 undeclared (first use in this function)\n **** v1 0.3 CLI RX| VRT_priv_task(ctx, &VGC_vmod_foobar),\n **** v1 0.3 CLI RX| ^\n **** v1 0.3 CLI RX| vgc.c:1439:31: error: \xe2\x80\x98VGC_vmod_snafu\xe2\x80\x99 undeclared (first use in this function)\n **** v1 0.3 CLI RX| [2 lines truncated]\n **** v1 0.3 CLI RX| Running C-compiler failed, exited with 1\n **** v1 0.3 CLI RX| VCL compilation failed ---- v1 0.3 FAIL VCL does not compile }}} The vmodtool doesn't throw any errors, but the generated code is defective. Evidently a symbol is generated for the private object that is not declared. The use case is a VMOD object that is intended to have state across method calls within a "task" (client or backend context). In my case, it's for a regex VMOD with a backref call meant to refer back to captured expressions in a previous match call (https://code.uplex.de/uplex-varnish/libvmod-re). The VCC interface is something like this: {{{ $Object regex(STRING) $Method BOOL .match(PRIV_TASK, STRING) $Method STRING .backref(PRIV_TASK, INT) }}} With usage like this: {{{ import re; sub vcl_init { # compile a regex new myregex = re.regex("foo(bar)"); } sub vcl_recv { if (myregex.match(req.http.Foo)) { set req.http.Bar = myregex.backref(1); } }}} The immediate defect is that code is generated that either can't be compiled or linked. More generally, it seems very intuitive to me that object methods should be able to maintain a state with task scope -- the regex example is basic stuff. Up to now, I'd been maintaining state by using pthread-local keys, but I thought using PRIV_TASK would be the more natural, Varnishy solution. -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Sun Oct 11 21:00:58 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Sun, 11 Oct 2015 21:00:58 -0000 Subject: [Varnish] #1800: PRIV_TASK in a vmod object method leads to code that cannot be compiled/linked In-Reply-To: <043.aa0af4bc03cba56c27b0370c89b02a0b@varnish-cache.org> References: <043.aa0af4bc03cba56c27b0370c89b02a0b@varnish-cache.org> Message-ID: <058.def0f8e3885ab980d7d212306b65fb91@varnish-cache.org> #1800: PRIV_TASK in a vmod object method leads to code that cannot be compiled/linked --------------------+-------------------- Reporter: geoff | Owner: Type: defect | Status: new Priority: normal | Milestone: Later Component: vmod | Version: 4.1.0 Severity: normal | Resolution: Keywords: | --------------------+-------------------- Comment (by fgsch): This works fine for me. Do you have the C counterpart in vmod_debug.c? -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Sun Oct 11 21:10:53 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Sun, 11 Oct 2015 21:10:53 -0000 Subject: [Varnish] #1800: PRIV_TASK in a vmod object method leads to code that cannot be compiled/linked In-Reply-To: <043.aa0af4bc03cba56c27b0370c89b02a0b@varnish-cache.org> References: <043.aa0af4bc03cba56c27b0370c89b02a0b@varnish-cache.org> Message-ID: <058.08e9818dcc9e5e966f13436d7d8ee8ec@varnish-cache.org> #1800: PRIV_TASK in a vmod object method leads to code that cannot be compiled/linked --------------------+-------------------- Reporter: geoff | Owner: Type: defect | Status: new Priority: normal | Milestone: Later Component: vmod | Version: 4.1.0 Severity: normal | Resolution: Keywords: | --------------------+-------------------- Comment (by geoff): Holy crap, what a stupid mistake. Nope, forgot about that part, I'm sorry. OK, let me try again, to see if I can reproduce the problem with my own VMOD. -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Sun Oct 11 21:52:42 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Sun, 11 Oct 2015 21:52:42 -0000 Subject: [Varnish] #1800: PRIV_TASK in a vmod object method leads to code that cannot be compiled/linked In-Reply-To: <043.aa0af4bc03cba56c27b0370c89b02a0b@varnish-cache.org> References: <043.aa0af4bc03cba56c27b0370c89b02a0b@varnish-cache.org> Message-ID: <058.dd56bb1628ee8ef2f9b15084504c52db@varnish-cache.org> #1800: PRIV_TASK in a vmod object method leads to code that cannot be compiled/linked --------------------+-------------------- Reporter: geoff | Owner: Type: defect | Status: new Priority: normal | Milestone: Later Component: vmod | Version: 4.1.0 Severity: normal | Resolution: Keywords: | --------------------+-------------------- Comment (by geoff): OK, I've attached a patch to demonstrate the problem, which matches the problem with my own VMOD. The patch adds these two methods to the object "obj" in vmod debug: {{{ $Method VOID .task_set(PRIV_TASK, INT) $Method INT .task_get(PRIV_TASK, INT fallback=-1) }}} Set an integer in task scope, and retrieve it in the same scope. m00000.vtc tests it: {{{ sub vcl_init { new foo = debug.obj("foo"); } sub vcl_deliver { # ... foo.task_set(4711); set resp.http.task = foo.task_get(); } }}} The generated code fails to compile with this message: {{{ **** v1 0.2 CLI RX| Message from C-compiler:\n **** v1 0.2 CLI RX| vgc.c: In function \xe2\x80\x98VGC_function_vcl_deliver\xe2\x80\x99:\n **** v1 0.2 CLI RX| vgc.c:1352:29: error: \xe2\x80\x98VGC_vmod_foo\xe2\x80\x99 undeclared (first use in this function)\n **** v1 0.2 CLI RX| VRT_priv_task(ctx, &VGC_vmod_foo),\n **** v1 0.2 CLI RX| ^\n **** v1 0.2 CLI RX| vgc.c:1352:29: note: each undeclared identifier is reported only once for each function it appears in\n **** v1 0.2 CLI RX| Running C-compiler failed, exited with 1\n **** v1 0.2 CLI RX| VCL compilation failed ---- v1 0.2 FAIL VCL does not compile }}} So please try to forget what I said about linking (duh), the problem is in the generated C code. A symbol `VGC_vmod_` is used for the priv object, but is not declared. As I said above, the uncompilable code is the immediate problem, but it also seems intuitive that object methods should be able to make use of task scope -- in other words, rather than having the vmodtool generate an error, it would be better to make PRIV_TASK possible in a method. -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Mon Oct 12 10:53:33 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Mon, 12 Oct 2015 10:53:33 -0000 Subject: [Varnish] #1761: 204 responses intermittently delivered as chunk-encoded with length byte = 0 In-Reply-To: <043.4c8744d010240f820f7b2440cfafa2fc@varnish-cache.org> References: <043.4c8744d010240f820f7b2440cfafa2fc@varnish-cache.org> Message-ID: <058.aac42fbd072d5b152fd8014fa4a4cfd7@varnish-cache.org> #1761: 204 responses intermittently delivered as chunk-encoded with length byte = 0 ----------------------+---------------------------------- Reporter: geoff | Owner: geoff Type: defect | Status: closed Priority: normal | Milestone: Varnish 4.0 release Component: varnishd | Version: 4.0.3 Severity: normal | Resolution: fixed Keywords: | ----------------------+---------------------------------- Changes (by phk): * status: new => closed * resolution: => fixed -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Mon Oct 12 13:19:31 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Mon, 12 Oct 2015 13:19:31 -0000 Subject: [Varnish] #1799: Backend request coalescing fail In-Reply-To: <044.a197835a5070dbdf3a5e5ef4dd8a3ca5@varnish-cache.org> References: <044.a197835a5070dbdf3a5e5ef4dd8a3ca5@varnish-cache.org> Message-ID: <059.05eb465222003c5548d8c057960f3081@varnish-cache.org> #1799: Backend request coalescing fail ----------------------+-------------------- Reporter: martin | Owner: Type: defect | Status: new Priority: normal | Milestone: Component: varnishd | Version: trunk Severity: normal | Resolution: Keywords: | ----------------------+-------------------- Comment (by martin): This was discussed during bugwash today. The sketched solution looks like: * A return(miss) from vcl_hit without a busy OC will give the expired OC an OC_F_DONTUSE flag. The request thread runs STP_LOOKUP again (instead of falling back to pass). This flag acts like a signal to subsequent lookups that we didn't like this expired object, and subsequent attempts at using it should be prevented. * During HSH_Lookup, if the expired OC selected (the newest expired option) has the OC_F_DONTUSE flag, we will continue as if we don't have an expired option. If then busy_found is true (OC with OC_F_BUSY on OH), we'll go to waitinglist as before. * bgfetch threads will on failed fetches clear the OC_F_DONTUSE flag if the expired OC it holds a reference too has this set (on successful fetches the object is purged as normal). This will reenable the object for use as an IMS candidate when fetching again, or as an expanded grace candidate for requests after e.g. the backend is marked sick. We will ponder the issue for a day to make sure the strategy is viable. Martin -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Mon Oct 12 13:28:17 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Mon, 12 Oct 2015 13:28:17 -0000 Subject: [Varnish] #1799: Backend request coalescing fail In-Reply-To: <044.a197835a5070dbdf3a5e5ef4dd8a3ca5@varnish-cache.org> References: <044.a197835a5070dbdf3a5e5ef4dd8a3ca5@varnish-cache.org> Message-ID: <059.e4db0be06f302256d885f092f0d73c1f@varnish-cache.org> #1799: Backend request coalescing fail ----------------------+-------------------- Reporter: martin | Owner: Type: defect | Status: new Priority: normal | Milestone: Component: varnishd | Version: trunk Severity: normal | Resolution: Keywords: | ----------------------+-------------------- Comment (by martin): A couple of VSL related things to consider: * SLT_Hit probably should only after we are sure that we won't jump back to STP_LOOKUP, so we avoid multiple confusing log lines. * The waitinglist timestamp also needs to be dealt with so it's logged only when we know we won't jump back to STP_LOOKUP. Martin -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Mon Oct 12 14:55:20 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Mon, 12 Oct 2015 14:55:20 -0000 Subject: [Varnish] #1800: PRIV_TASK in a vmod object method leads to code that cannot be compiled/linked In-Reply-To: <043.aa0af4bc03cba56c27b0370c89b02a0b@varnish-cache.org> References: <043.aa0af4bc03cba56c27b0370c89b02a0b@varnish-cache.org> Message-ID: <058.c5866e7cbab080556ce7da20d085a7e5@varnish-cache.org> #1800: PRIV_TASK in a vmod object method leads to code that cannot be compiled/linked --------------------+-------------------- Reporter: geoff | Owner: Type: defect | Status: new Priority: normal | Milestone: Later Component: vmod | Version: 4.1.0 Severity: normal | Resolution: Keywords: | --------------------+-------------------- Comment (by daghf): Hi I had a look, and this comes down to overly simple parsing in vcc_expr.c:vcc_priv_arg, where we've forgot to take VMOD objects into account. The parsing code blindly assumes the VMOD name to be the initial portion of the function invocation string leading up to the first dot. This works fine for regular function calls ("vmodname.foo(..)"), but breaks down for VMOD objects, where the parsed result is the arbitrary name of the object instance ("my_object.foo(..)"). We need to expand on this to do a lookup that tells us if the parsed name is a VMOD object, and if so return the corresponding VMOD name. -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Mon Oct 12 14:56:23 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Mon, 12 Oct 2015 14:56:23 -0000 Subject: [Varnish] #1800: PRIV_TASK in a vmod object method leads to code that cannot be compiled/linked In-Reply-To: <043.aa0af4bc03cba56c27b0370c89b02a0b@varnish-cache.org> References: <043.aa0af4bc03cba56c27b0370c89b02a0b@varnish-cache.org> Message-ID: <058.14b3cd63f1be218093ff55a2d35aacb3@varnish-cache.org> #1800: PRIV_TASK in a vmod object method leads to code that cannot be compiled/linked --------------------+-------------------- Reporter: geoff | Owner: Type: defect | Status: new Priority: normal | Milestone: Later Component: vmod | Version: 4.1.0 Severity: normal | Resolution: Keywords: | --------------------+-------------------- Comment (by daghf): Forgot to mention: This also applies for PRIV_VCL and PRIV_TOP. -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Tue Oct 13 05:12:35 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Tue, 13 Oct 2015 05:12:35 -0000 Subject: [Varnish] #1801: Fail parsing IPv6 constants Message-ID: <043.f269ea63a9a5a2c445c8af43e75b18dd@varnish-cache.org> #1801: Fail parsing IPv6 constants ----------------------+------------------- Reporter: fgsch | Owner: Type: defect | Status: new Priority: normal | Milestone: Component: varnishd | Version: 4.1.0 Severity: normal | Keywords: ----------------------+------------------- Using, i.e. ::1 leads to: **** v1 0.1 CLI RX| IP constant '"::1"' could not be resolved to an IP address:\n **** v1 0.1 CLI RX| \tServname not supported for ai_socktype\n **** v1 0.1 CLI RX| (Sorry if that error message is gibberish.)\n Works fine in 4.0. This broke when Resolve_Sockaddr() was changed to use VSS_resolver(). The workaround is to use []. -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Tue Oct 13 05:12:54 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Tue, 13 Oct 2015 05:12:54 -0000 Subject: [Varnish] #1801: Fail parsing IPv6 constants In-Reply-To: <043.f269ea63a9a5a2c445c8af43e75b18dd@varnish-cache.org> References: <043.f269ea63a9a5a2c445c8af43e75b18dd@varnish-cache.org> Message-ID: <058.f105e65779ddb8f08b39d97fe6ff0854@varnish-cache.org> #1801: Fail parsing IPv6 constants ----------------------+-------------------- Reporter: fgsch | Owner: Type: defect | Status: new Priority: normal | Milestone: Component: varnishd | Version: 4.1.0 Severity: normal | Resolution: Keywords: | ----------------------+-------------------- Description changed by fgsch: Old description: > Using, i.e. ::1 leads to: > > **** v1 0.1 CLI RX| IP constant '"::1"' could not be resolved to an IP > address:\n > **** v1 0.1 CLI RX| \tServname not supported for ai_socktype\n > **** v1 0.1 CLI RX| (Sorry if that error message is gibberish.)\n > > Works fine in 4.0. This broke when Resolve_Sockaddr() was changed to use > VSS_resolver(). > > The workaround is to use []. New description: Using, i.e. ::1 leads to: {{{ **** v1 0.1 CLI RX| IP constant '"::1"' could not be resolved to an IP address:\n **** v1 0.1 CLI RX| \tServname not supported for ai_socktype\n **** v1 0.1 CLI RX| (Sorry if that error message is gibberish.)\n }}} Works fine in 4.0. This broke when Resolve_Sockaddr() was changed to use VSS_resolver(). The workaround is to use []. -- -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Tue Oct 13 05:15:04 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Tue, 13 Oct 2015 05:15:04 -0000 Subject: [Varnish] #1801: Fail parsing IPv6 constants In-Reply-To: <043.f269ea63a9a5a2c445c8af43e75b18dd@varnish-cache.org> References: <043.f269ea63a9a5a2c445c8af43e75b18dd@varnish-cache.org> Message-ID: <058.8591f99869b4b4ac9f505ee77d1f178a@varnish-cache.org> #1801: Fail parsing IPv6 constants ----------------------+-------------------- Reporter: fgsch | Owner: Type: defect | Status: new Priority: normal | Milestone: Component: varnishd | Version: 4.1.0 Severity: normal | Resolution: Keywords: | ----------------------+-------------------- Description changed by fgsch: Old description: > Using, i.e. ::1 leads to: > > {{{ > **** v1 0.1 CLI RX| IP constant '"::1"' could not be resolved to an IP > address:\n > **** v1 0.1 CLI RX| \tServname not supported for ai_socktype\n > **** v1 0.1 CLI RX| (Sorry if that error message is gibberish.)\n > }}} > > Works fine in 4.0. This broke when Resolve_Sockaddr() was changed to use > VSS_resolver(). > > The workaround is to use []. New description: Using, i.e. ::1 leads to: {{{ **** v1 0.1 CLI RX| IP constant '"::1"' could not be resolved to an IP address:\n **** v1 0.1 CLI RX| \tServname not supported for ai_socktype\n **** v1 0.1 CLI RX| (Sorry if that error message is gibberish.)\n }}} Test attached. Works fine in 4.0. This broke when Resolve_Sockaddr() was changed to use VSS_resolver(). The workaround is to use []. -- -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Wed Oct 14 18:17:28 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Wed, 14 Oct 2015 18:17:28 -0000 Subject: [Varnish] #1635: Completed bans keep accumulating In-Reply-To: <043.d9a46f331b8fae5fc5502a3059307d4c@varnish-cache.org> References: <043.d9a46f331b8fae5fc5502a3059307d4c@varnish-cache.org> Message-ID: <058.229909e821fcc16d86f5162789c95ece@varnish-cache.org> #1635: Completed bans keep accumulating ----------------------+-------------------- Reporter: Sesse | Owner: Type: defect | Status: new Priority: normal | Milestone: Component: varnishd | Version: trunk Severity: normal | Resolution: Keywords: lurker | ----------------------+-------------------- Comment (by lucascherifi): Hi ! I have exactly the same in V 4.0.3. May someone has an idea ? -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Wed Oct 14 18:22:17 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Wed, 14 Oct 2015 18:22:17 -0000 Subject: [Varnish] #1635: Completed bans keep accumulating In-Reply-To: <043.d9a46f331b8fae5fc5502a3059307d4c@varnish-cache.org> References: <043.d9a46f331b8fae5fc5502a3059307d4c@varnish-cache.org> Message-ID: <058.2b8d5054a6000ff7e44987da9e134327@varnish-cache.org> #1635: Completed bans keep accumulating ----------------------+-------------------- Reporter: Sesse | Owner: Type: defect | Status: new Priority: normal | Milestone: Component: varnishd | Version: trunk Severity: normal | Resolution: Keywords: lurker | ----------------------+-------------------- Comment (by lucascherifi): karlvr, you were right. I tried to set the cache control response header of the back end responses to 20sec and the MAIN.bans number decrease to one after 20sec. -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Wed Oct 14 22:19:39 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Wed, 14 Oct 2015 22:19:39 -0000 Subject: [Varnish] #1795: Man page issues in 4.1 In-Reply-To: <043.3248b86a133da7a2e2f3f47447587d62@varnish-cache.org> References: <043.3248b86a133da7a2e2f3f47447587d62@varnish-cache.org> Message-ID: <058.2b98e8a5c54b36eb9f2a8e26fe2654b5@varnish-cache.org> #1795: Man page issues in 4.1 ---------------------------+-------------------- Reporter: Dridi | Owner: Type: documentation | Status: new Priority: normal | Milestone: Component: documentation | Version: 4.1.0 Severity: normal | Resolution: Keywords: | ---------------------------+-------------------- Comment (by Federico G. Schwindt ): In [405e7711387bd4429fda757b2909524cebe837a6]: {{{ #!CommitTicketReference repository="" revision="405e7711387bd4429fda757b2909524cebe837a6" Document vcl.state and correct existing mention Partially addresses #1795. }}} -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Fri Oct 16 08:16:41 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Fri, 16 Oct 2015 08:16:41 -0000 Subject: [Varnish] #1800: PRIV_TASK in a vmod object method leads to code that cannot be compiled/linked In-Reply-To: <043.aa0af4bc03cba56c27b0370c89b02a0b@varnish-cache.org> References: <043.aa0af4bc03cba56c27b0370c89b02a0b@varnish-cache.org> Message-ID: <058.8a152996662d59edc64340acf738da2d@varnish-cache.org> #1800: PRIV_TASK in a vmod object method leads to code that cannot be compiled/linked --------------------+--------------------- Reporter: geoff | Owner: aondio Type: defect | Status: new Priority: normal | Milestone: Later Component: vmod | Version: 4.1.0 Severity: normal | Resolution: Keywords: | --------------------+--------------------- Changes (by aondio): * owner: => aondio -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Sat Oct 17 15:27:20 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Sat, 17 Oct 2015 15:27:20 -0000 Subject: [Varnish] #1802: Segfault after VCL change Message-ID: <046.db002b3f9fce758373ff6d1d348871ef@varnish-cache.org> #1802: Segfault after VCL change ----------------------+------------------- Reporter: lkarsten | Owner: Type: defect | Status: new Priority: normal | Milestone: Component: varnishd | Version: 4.1.0 Severity: normal | Keywords: ----------------------+------------------- Varnish 4.1.0 got into a non-working state this afternoon. After a VCL change the child segfaulted, but the master process was not able to restart successfully. Site down. {{{ root at blade:~# ps -efww | grep varnish varnish 914 1 0 Oct15 ? 00:00:02 /usr/sbin/varnishd -a :80 -a :8000,PROXY -a :6081 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -s malloc,256m root 2952 32506 0 17:24 pts/2 00:00:00 grep varnish }}} Debian stretch running 4.2.0 kernel. No other crashes/panics have been logged. {{{ Oct 17 17:04:09 blade varnishd[914]: CLI telnet ::1 50786 ::1 6082 Rd auth xx Oct 17 17:04:09 blade varnishd[914]: CLI telnet ::1 50786 ::1 6082 Wr 200 -----------------------------#012Varnish Cache CLI 1.0#012---------- -------------------#012Linux,4.2.0-1-amd64,x86_64,-junix,-smalloc,-smalloc,-hcritbit#012varnish-4.1.0 revision 3041728#012#012Type 'help' for command list.#012Type 'quit' to close CLI session. Oct 17 17:04:09 blade varnishd[914]: CLI telnet ::1 50786 ::1 6082 Rd ping Oct 17 17:04:09 blade varnishd[914]: CLI telnet ::1 50786 ::1 6082 Wr 200 PONG 1445094249 1.0 Oct 17 17:04:09 blade varnishd[914]: CLI telnet ::1 50786 ::1 6082 Rd vcl.load 9824a4ab-fb7f-4f31-9655-0df1fe76459d /etc/varnish/default.vcl Oct 17 17:04:10 blade varnishd[914]: CLI telnet ::1 50786 ::1 6082 Wr 200 VCL compiled. Oct 17 17:04:10 blade varnishd[914]: CLI telnet ::1 50788 ::1 6082 Rd auth xx Oct 17 17:04:10 blade varnishd[914]: CLI telnet ::1 50788 ::1 6082 Wr 200 -----------------------------#012Varnish Cache CLI 1.0#012---------- -------------------#012Linux,4.2.0-1-amd64,x86_64,-junix,-smalloc,-smalloc,-hcritbit#012varnish-4.1.0 revision 3041728#012#012Type 'help' for command list.#012Type 'quit' to close CLI session. Oct 17 17:04:10 blade varnishd[914]: CLI telnet ::1 50788 ::1 6082 Rd ping Oct 17 17:04:10 blade varnishd[914]: CLI telnet ::1 50788 ::1 6082 Wr 200 PONG 1445094250 1.0 Oct 17 17:04:10 blade varnishd[914]: CLI telnet ::1 50788 ::1 6082 Rd vcl.use 9824a4ab-fb7f-4f31-9655-0df1fe76459d Oct 17 17:04:10 blade varnishd[914]: CLI telnet ::1 50788 ::1 6082 Wr 200 VCL '9824a4ab-fb7f-4f31-9655-0df1fe76459d' now active Oct 17 17:17:31 blade kernel: [170387.618538] varnishd[917]: segfault at 4799ce6b ip 00007f07e2df5cb9 sp 00007ffca79c2130 error 4 in libgcc_s.so.1[7f07e2de7000+16000] Oct 17 17:17:32 blade varnishd[914]: Child (917) died signal=11 Oct 17 17:17:32 blade varnishd[914]: child (1814) Started Oct 17 17:17:33 blade varnishd[914]: Child (1814) ended Oct 17 17:17:33 blade varnishd[914]: Child (1814) said Child starts Oct 17 17:17:33 blade varnishd[914]: Child (1814) said Child dies Oct 17 17:17:34 blade varnishd[914]: CLI telnet ::1 50862 ::1 6082 Rd auth xx Oct 17 17:17:34 blade varnishd[914]: CLI telnet ::1 50862 ::1 6082 Wr 200 -----------------------------#012Varnish Cache CLI 1.0#012-----------------------------#012Linux,4.2.0-1-amd64,x86_64,-junix,-smalloc,-smalloc,-hcritbit#012varnish-4.1.0 revision 3041728#012#012Type 'help' for command list.#012Type 'quit' to close CLI session.#012Type 'start' to launch worker process. Oct 17 17:17:34 blade varnishd[914]: CLI telnet ::1 50862 ::1 6082 Rd ping Oct 17 17:17:34 blade varnishd[914]: CLI telnet ::1 50862 ::1 6082 Wr 200 PONG 1445095054 1.0 Oct 17 17:17:34 blade varnishd[914]: CLI telnet ::1 50862 ::1 6082 Rd ban req.url ~ / Oct 17 17:17:34 blade varnishd[914]: CLI telnet ::1 50862 ::1 6082 Wr 101 Unknown request in manager process (child not running).#012Type 'help' for more info. Oct 17 17:17:45 blade varnishd[914]: CLI telnet ::1 50864 ::1 6082 Rd auth xx Oct 17 17:17:45 blade varnishd[914]: CLI telnet ::1 50864 ::1 6082 Wr 200 -----------------------------#012Varnish Cache CLI 1.0#012-----------------------------#012Linux,4.2.0-1-amd64,x86_64,-junix,-smalloc,-smalloc,-hcritbit#012varnish-4.1.0 revision 3041728#012#012Type 'help' for command list.#012Type 'quit' to close CLI session.#012Type 'start' to launch worker process. Oct 17 17:17:45 blade varnishd[914]: CLI telnet ::1 50864 ::1 6082 Rd ping Oct 17 17:17:45 blade varnishd[914]: CLI telnet ::1 50864 ::1 6082 Wr 200 PONG 1445095065 1.0 }}} -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Sat Oct 17 15:30:49 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Sat, 17 Oct 2015 15:30:49 -0000 Subject: [Varnish] #1802: Segfault after VCL change In-Reply-To: <046.db002b3f9fce758373ff6d1d348871ef@varnish-cache.org> References: <046.db002b3f9fce758373ff6d1d348871ef@varnish-cache.org> Message-ID: <061.3827e477a9eb31fb8734b3a885e8e37d@varnish-cache.org> #1802: Segfault after VCL change ----------------------+-------------------- Reporter: lkarsten | Owner: Type: defect | Status: new Priority: normal | Milestone: Component: varnishd | Version: 4.1.0 Severity: normal | Resolution: Keywords: | ----------------------+-------------------- Comment (by lkarsten): {{{ root at blade:~# gdb --batch --eval-command "bt full" -p 914 /usr/sbin/varnishd [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". 0x00007f07f0e48510 in __poll_nocancel () at ../sysdeps/unix/syscall- template.S:81 81 ../sysdeps/unix/syscall-template.S: No such file or directory. #0 0x00007f07f0e48510 in __poll_nocancel () at ../sysdeps/unix/syscall- template.S:81 No locals. #1 0x00007f07f2537456 in poll (__timeout=, __nfds=, __fds=) at /usr/include/x86_64 -linux-gnu/bits/poll2.h:46 No locals. #2 vev_schedule_one (evb=0x7f07f0015040) at vev.c:480 t = e = 0x7f07f00dd1d0 e2 = e3 = i = j = tmo = pfd = __func__ = "vev_schedule_one" #3 0x00007f07f2537898 in vev_schedule (evb=0x7f07f0015040) at vev.c:367 i = __func__ = "vev_schedule" #4 0x000000000045326b in MGT_Run () at mgt/mgt_child.c:733 sac = {__sigaction_handler = {sa_handler = 0x1, sa_sigaction = 0x1}, sa_mask = {__val = {0 }}, sa_flags = 268435456, sa_restorer = 0x0} e = i = __func__ = "MGT_Run" #5 0x000000000040f8a0 in main (argc=, argv=) at mgt/mgt_main.c:773 o = C_flag = 0 F_flag = 0 b_arg = 0x0 f_arg = 0x7ffca79c3f41 "/etc/varnish/default.vcl" i_arg = h_arg = 0x472e97 "critbit" M_arg = 0x0 n_arg = P_arg = 0x0 S_arg = 0x7ffca79c3f5d "/etc/varnish/secret" s_arg = 0x4766f7 "malloc,100m" W_arg = 0x0 s_arg_given = 1 T_arg = 0x7ffca79c3f2f "localhost:6082" p = vcl = cli = {{magic = 1077466480, sb = 0x7f07f000e070, result = CLIS_OK, cmd = 0x0, auth = 0, challenge = '\000' , ident = 0x0, vlu = 0x0, cls = 0x0, limit = 0x7ffca79c326c}} dirname = 0x7f07f0010040 "/var/lib/varnish/blade/" av = clilim = 32768 jailed = __func__ = "main" root at blade:~# }}} -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Mon Oct 19 08:12:13 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Mon, 19 Oct 2015 08:12:13 -0000 Subject: [Varnish] #1635: Completed bans keep accumulating In-Reply-To: <043.d9a46f331b8fae5fc5502a3059307d4c@varnish-cache.org> References: <043.d9a46f331b8fae5fc5502a3059307d4c@varnish-cache.org> Message-ID: <058.dea448114a3a822aedaa798bf76c842f@varnish-cache.org> #1635: Completed bans keep accumulating ----------------------+-------------------- Reporter: Sesse | Owner: Type: defect | Status: new Priority: normal | Milestone: Component: varnishd | Version: trunk Severity: normal | Resolution: Keywords: lurker | ----------------------+-------------------- Comment (by phk): "so am I correct in assuming that bans can only be deleted once there are no older objects in the cache? In which case perhaps the behaviour I'm observing, and others are observing above, is expected? " Its slightly more complicated. We can only delete a ban after all *older* cached objects have been tested against it. If the ban contains any "req.*" tests, that can only happen when somebody gets a cache-hit on that object. If the ban is composed entirely of "obj.*" tests, the ban-lurker should be able to "wash" it against all the cached objects (slowly, over time) and the ban will be marked 'completed', the capital 'C' you see in the output. However, because of the way we lock the ban-list, we cannot actually remove the completed ban from the list if it is somewhere in the middle, we can only delete the last ban on the list. So if the last ban is using req.* and stuck until either the objects that hasn't been checked yet expire or get a cache-hit, you can see a lot of these 'Completed' bans accumulate in the list. This is not optimal, but we havn't found a performance-cheap way of solving this issue. -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Mon Oct 19 08:15:02 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Mon, 19 Oct 2015 08:15:02 -0000 Subject: [Varnish] #1635: Completed bans keep accumulating In-Reply-To: <043.d9a46f331b8fae5fc5502a3059307d4c@varnish-cache.org> References: <043.d9a46f331b8fae5fc5502a3059307d4c@varnish-cache.org> Message-ID: <058.cd27fb99bff89fc08543a93475be0b01@varnish-cache.org> #1635: Completed bans keep accumulating ----------------------+-------------------- Reporter: Sesse | Owner: Type: defect | Status: new Priority: normal | Milestone: Component: varnishd | Version: trunk Severity: normal | Resolution: Keywords: lurker | ----------------------+-------------------- Comment (by Sesse): I don't think this fully explains the behavior here; like I said in the very first message, I don't have any req.* bans at all. -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Mon Oct 19 08:18:22 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Mon, 19 Oct 2015 08:18:22 -0000 Subject: [Varnish] #1635: Completed bans keep accumulating In-Reply-To: <043.d9a46f331b8fae5fc5502a3059307d4c@varnish-cache.org> References: <043.d9a46f331b8fae5fc5502a3059307d4c@varnish-cache.org> Message-ID: <058.a6d6e4742e7d3ddc42a43b9ee8f37558@varnish-cache.org> #1635: Completed bans keep accumulating ----------------------+-------------------- Reporter: Sesse | Owner: Type: defect | Status: new Priority: normal | Milestone: Component: varnishd | Version: trunk Severity: normal | Resolution: Keywords: lurker | ----------------------+-------------------- Comment (by phk): ... which Is why I have left this ticket open until we find the bug :-) -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Mon Oct 19 10:22:44 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Mon, 19 Oct 2015 10:22:44 -0000 Subject: [Varnish] #1803: Varnishncsa truncated referer Message-ID: <045.b8e65587d3775974af1bb7ca0a54c6f6@varnish-cache.org> #1803: Varnishncsa truncated referer ---------------------+--------------------- Reporter: nixmind | Type: defect Status: new | Priority: highest Milestone: | Component: build Version: unknown | Severity: normal Keywords: | ---------------------+--------------------- I have a question concerning varnish and varnishncsa. In fact I've set up varnishncsa to log requests with this log format : {{{ LOG_FORMAT="%{X-Forwarded-For}i %l %u %t \"%r\" %s %b \"%{Referer}i\" \"%{User-agent}i\"" }}} But for long **refers**, I noticed that they are truncated, and when checked they are all limited to **247** characters. I've googled for a long time to figure out what can be the problem, but I didn't find a right answer. Do someone have ever have the issue, and can someone tell me how varnishncsa works in depth, and this issue can be solved please?Varnishncsa truncated referer Not all the log entry is truncated, but only the referer field. By looking the varnishncsa source code I noticed the function parse_format uses a static buffer of **256** octets. Is there where the size limit is set? Thank in advance for your help. -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Mon Oct 19 11:09:22 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Mon, 19 Oct 2015 11:09:22 -0000 Subject: [Varnish] #1803: Varnishncsa truncated referer In-Reply-To: <045.b8e65587d3775974af1bb7ca0a54c6f6@varnish-cache.org> References: <045.b8e65587d3775974af1bb7ca0a54c6f6@varnish-cache.org> Message-ID: <060.539d0e2648fbec8c521459347148c281@varnish-cache.org> #1803: Varnishncsa truncated referer ---------------------+---------------------- Reporter: nixmind | Owner: Type: defect | Status: new Priority: highest | Milestone: Component: build | Version: unknown Severity: normal | Resolution: Keywords: | ---------------------+---------------------- Comment (by lkarsten): Hi. Can you increase the vsl_reclen parameter and see if that solves your issue? -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Mon Oct 19 11:09:49 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Mon, 19 Oct 2015 11:09:49 -0000 Subject: [Varnish] #1803: Varnishncsa truncated referer In-Reply-To: <045.b8e65587d3775974af1bb7ca0a54c6f6@varnish-cache.org> References: <045.b8e65587d3775974af1bb7ca0a54c6f6@varnish-cache.org> Message-ID: <060.8b7180b6de40431291ad4decb8ca60c3@varnish-cache.org> #1803: Varnishncsa truncated referer -------------------------+----------------------- Reporter: nixmind | Owner: Type: defect | Status: needinfo Priority: normal | Milestone: Component: varnishncsa | Version: unknown Severity: normal | Resolution: Keywords: | -------------------------+----------------------- Changes (by lkarsten): * status: new => needinfo * priority: highest => normal * component: build => varnishncsa -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Mon Oct 19 11:51:55 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Mon, 19 Oct 2015 11:51:55 -0000 Subject: [Varnish] #1801: Fail parsing IPv6 constants In-Reply-To: <043.f269ea63a9a5a2c445c8af43e75b18dd@varnish-cache.org> References: <043.f269ea63a9a5a2c445c8af43e75b18dd@varnish-cache.org> Message-ID: <058.50c7ba1c80338613894302ab1a75846b@varnish-cache.org> #1801: Fail parsing IPv6 constants ----------------------+---------------------- Reporter: fgsch | Owner: Type: defect | Status: closed Priority: normal | Milestone: Component: varnishd | Version: 4.1.0 Severity: normal | Resolution: wontfix Keywords: | ----------------------+---------------------- Changes (by phk): * status: new => closed * resolution: => wontfix Comment: This is not our fault. Despite widespread use of the IP:PORT notation, some dimwits decided to format IPv6 addresses using ':' as separator. Ever since then things have been going downhill. Here is my partial list of RFC's which define how IPv6 addresses look: {{{ RFC1884 -> 2373 1080::8:800:200C:417A RFC1884 ::13.1.68.3 RFC1884 ::FFFF:129.144.52.38 RFC1924 4)+k&C#VzJ4br>0wv%Yp RFC2133 -> 2133 RFC2292 -> 3542 RFC2373 -> 3513 RFC2428 EPRT |2|1080::8:800:200C:417A|5282| RFC2553 -> RFC3493 RFC2732 -> 3986 http://[3ffe:2a00:100:7031::1]:8080/ RFC3493 "numeric format" RFC3513 -> 4291 RFC3986 BNF form RFC3986 http://[3ffe:2a00:100:7031::1]:8080/ RFC3986 http://[v%x.????]/ RFC5952 [2001:db8::1]:80 }}} Despite its publication date, I prefer RFC1924 since it would clearly encourage DNS use. Anyhow, the "[IPv6]:port" notation is the HTTP way and that's what we're using. The only other consistent policy would be to ditch IP:PORT notation entirely. We already support space as a seperator in that case, but that requires quoting stuff on the command line. We need a close-reason for tickets along the lines of "Doesn't Work For Me But The World Sucks" -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Mon Oct 19 12:02:40 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Mon, 19 Oct 2015 12:02:40 -0000 Subject: [Varnish] #1803: Varnishncsa truncated referer In-Reply-To: <045.b8e65587d3775974af1bb7ca0a54c6f6@varnish-cache.org> References: <045.b8e65587d3775974af1bb7ca0a54c6f6@varnish-cache.org> Message-ID: <060.c9a7c0e09c24780828556a3f97accb05@varnish-cache.org> #1803: Varnishncsa truncated referer -------------------------+----------------------- Reporter: nixmind | Owner: Type: defect | Status: needinfo Priority: normal | Milestone: Component: varnishncsa | Version: unknown Severity: normal | Resolution: Keywords: | -------------------------+----------------------- Comment (by nixmind): Hi, Do you mean the **shm_reclen** or **vsl_reclen** paramter? I check with the **param.show** command but I didn't see any **vsl_reclen** parameter, even if it's described in the man page. Should I set this from **varnishadm** command line or in a file? Thanks. -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Mon Oct 19 12:06:17 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Mon, 19 Oct 2015 12:06:17 -0000 Subject: [Varnish] #1803: Varnishncsa truncated referer In-Reply-To: <045.b8e65587d3775974af1bb7ca0a54c6f6@varnish-cache.org> References: <045.b8e65587d3775974af1bb7ca0a54c6f6@varnish-cache.org> Message-ID: <060.036855b29f673150e1b1c3d67a9b8f1c@varnish-cache.org> #1803: Varnishncsa truncated referer -------------------------+----------------------- Reporter: nixmind | Owner: Type: defect | Status: needinfo Priority: normal | Milestone: Component: varnishncsa | Version: unknown Severity: normal | Resolution: Keywords: | -------------------------+----------------------- Comment (by nixmind): Appologises, In fact I'm using varnish version **3.0.2** because I'm on Debian Wheezy. So the name of the parameter is still **shm_reclen** in this version of the program. Sorry -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Mon Oct 19 15:19:18 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Mon, 19 Oct 2015 15:19:18 -0000 Subject: [Varnish] #1598: Varnish asserts if the backend returns header lines without ':' and doing backend IMS In-Reply-To: <044.7e0667d88264f9f24aedc738cd60d6bd@varnish-cache.org> References: <044.7e0667d88264f9f24aedc738cd60d6bd@varnish-cache.org> Message-ID: <059.05910c4842cfa54698f142c3f5b6ab20@varnish-cache.org> #1598: Varnish asserts if the backend returns header lines without ':' and doing backend IMS ----------------------+---------------------------------------- Reporter: martin | Owner: Poul-Henning Kamp Type: defect | Status: closed Priority: normal | Milestone: Component: varnishd | Version: trunk Severity: normal | Resolution: fixed Keywords: | ----------------------+---------------------------------------- Comment (by Lasse Karstensen ): In [ef6f15e5521829d60cb44233b8d5fe5ad29f3620]: {{{ #!CommitTicketReference repository="" revision="ef6f15e5521829d60cb44233b8d5fe5ad29f3620" Allow invalid headers in 304 backend responses. Allow the backend server to send headers lacking ":"/colon in them when responding to a conditional request yielding a 304 response. In master/4.1 such responses are aborted as invalid. The backend is clearly not feeling well. Since we've accepted it nicely for 200 responses so far in Varnish 4.0, continue that trend also for 304s. Fixes: #1598 }}} -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Tue Oct 20 12:05:30 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Tue, 20 Oct 2015 12:05:30 -0000 Subject: [Varnish] #1803: Varnishncsa truncated referer In-Reply-To: <045.b8e65587d3775974af1bb7ca0a54c6f6@varnish-cache.org> References: <045.b8e65587d3775974af1bb7ca0a54c6f6@varnish-cache.org> Message-ID: <060.57d838b685f57aa1313fb6fb7ce84094@varnish-cache.org> #1803: Varnishncsa truncated referer -------------------------+----------------------- Reporter: nixmind | Owner: Type: defect | Status: needinfo Priority: normal | Milestone: Component: varnishncsa | Version: unknown Severity: normal | Resolution: Keywords: | -------------------------+----------------------- Comment (by nixmind): It seems it works, I did the modification from the command line. I will apply permanently and restart the service. I will then look for a little few tile if it works fine, and then confirm here. Thanks. -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Tue Oct 20 14:09:19 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Tue, 20 Oct 2015 14:09:19 -0000 Subject: [Varnish] #1801: Fail parsing IPv6 constants In-Reply-To: <043.f269ea63a9a5a2c445c8af43e75b18dd@varnish-cache.org> References: <043.f269ea63a9a5a2c445c8af43e75b18dd@varnish-cache.org> Message-ID: <058.0a04322aca461832d731c6274e13db08@varnish-cache.org> #1801: Fail parsing IPv6 constants ----------------------+----------------------- Reporter: fgsch | Owner: Type: defect | Status: reopened Priority: normal | Milestone: Component: varnishd | Version: 4.1.0 Severity: normal | Resolution: Keywords: | ----------------------+----------------------- Changes (by fgsch): * status: closed => reopened * resolution: wontfix => Comment: I don't agree with the resolution of this. To begin with, the rfc1884 (2001:db8::1) format used to work in previous versions. The new required format works for IP literals but doesn't work as input for std.ip(). Obviously when ports are needed using []s is the only way, for anything else this is forced and arbitrary, specially considering how short and trivial the patch to support the other notation is (attached). We don't force the IPv4 literal to be IPv4:PORT so I don't see why we need to force IPv6 to be that way. This is not consistent either. We ought to reconsider it. -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Wed Oct 21 21:56:51 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Wed, 21 Oct 2015 21:56:51 -0000 Subject: [Varnish] #1804: PROXYed client requests does not appear in varnishlog Message-ID: <044.b771cf3de742ae034ee8c620e1082173@varnish-cache.org> #1804: PROXYed client requests does not appear in varnishlog --------------------+-------------------- Reporter: trygve | Type: defect Status: new | Priority: normal Milestone: | Component: build Version: | Severity: normal Keywords: | --------------------+-------------------- Hi, I have a setup which consists of haproxy 1.6 with Varnish 4.1.0 as a backend. haproxy is configured with send-proxy-v2 and Varnish is listening with -a localhost:6086,PROXY. Requests that comes in to Varnish on a non-PROXY port gets logged like usual, but PROXYed client requests does not appear in neither varnishncsa or varnishlog (unless I add -g raw). Output from varnishlog: {{{ * << Session >> 566 - Begin sess 0 PROXY - SessOpen 127.0.0.1 34182 localhost:6086 127.0.0.1 6086 1445463298.509256 19 - Link req 567 rxreq - SessClose RESP_CLOSE 0.000 - End }}} Output from varnishlog -g raw: {{{ 566 Begin c sess 0 PROXY 566 SessOpen c 127.0.0.1 34182 localhost:6086 127.0.0.1 6086 1445463298.509256 19 566 Link c req 567 rxreq 0 Proxy - 2 2a02: 54835 2001: 443 567 Begin c req 566 rxreq 567 Timestamp c Start: 1445463298.509310 0.000000 0.000000 567 Timestamp c Req: 1445463298.509310 0.000000 0.000000 567 ReqStart c 2a02: 54835 567 ReqMethod c GET 567 ReqURL c / 567 ReqProtocol c HTTP/1.1 567 ReqHeader c User-Agent: curl/7.38.0 567 ReqHeader c Host: example.com 567 ReqHeader c Accept: */* 567 ReqHeader c X-Forwarded-Proto: https 567 ReqHeader c Connection: close 567 ReqHeader c X-Forwarded-For: 2a02: 567 VCL_call c RECV 567 VCL_acl c NO_MATCH local 567 VCL_return c hash 567 VCL_call c HASH 567 VCL_return c lookup 567 Hit c 131654 567 VCL_call c HIT 567 VCL_return c deliver 567 RespProtocol c HTTP/1.1 567 RespStatus c 200 567 RespReason c OK 567 RespHeader c Date: Wed, 21 Oct 2015 21:34:48 GMT 567 RespHeader c Server: Apache/2.4.7 (Ubuntu) 567 RespHeader c Last-Modified: Thu, 03 Apr 2014 20:04:50 GMT 567 RespHeader c ETag: "469-4f628e9e5742b-gzip" 567 RespHeader c Vary: Accept-Encoding 567 RespHeader c Content-Encoding: gzip 567 RespHeader c Content-Length: 553 567 RespHeader c Content-Type: text/html; charset=UTF-8 567 RespHeader c x-url: example.com/ 567 RespHeader c X-Varnish: 567 131654 567 RespHeader c Age: 10 567 RespHeader c Via: 1.1 varnish-v4 567 RespUnset c ETag: "469-4f628e9e5742b-gzip" 567 RespHeader c ETag: W/"469-4f628e9e5742b-gzip" 567 VCL_call c DELIVER 567 RespUnset c x-url: example.com/ 567 RespUnset c X-Varnish: 567 131654 567 RespUnset c Via: 1.1 varnish-v4 567 RespHeader c X-Cache: HIT:1 567 RespHeader c X-Age: 10 567 RespUnset c Age: 10 567 RespHeader c Age: 0 567 VCL_return c deliver 567 Timestamp c Process: 1445463298.509411 0.000100 0.000100 567 RespUnset c Content-Encoding: gzip 567 RespHeader c Accept-Ranges: bytes 567 RespUnset c Content-Length: 553 567 RespHeader c Content-Length: 1129 567 Debug c "RES_MODE 42" 567 RespHeader c Connection: close 567 Gzip c U D - 553 1129 80 80 4358 567 Timestamp c Resp: 1445463298.509507 0.000196 0.000096 567 ReqAcct c 178 0 178 328 1129 1457 567 End c 566 SessClose c RESP_CLOSE 0.000 566 End c }}} {{{ # cat /etc/apt/sources.list.d/varnish.list deb http://repo.varnish-cache.org/ubuntu/ precise varnish-4.1 }}} {{{ # dpkg -l | grep varnish ii libvarnishapi1 4.1.0-1~precise amd64 shared libraries for Varnish ii varnish 4.1.0-1~precise amd64 state of the art, high-performance web accelerator }}} -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Wed Oct 21 22:12:30 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Wed, 21 Oct 2015 22:12:30 -0000 Subject: [Varnish] #1804: PROXYed client requests does not appear in varnishlog In-Reply-To: <044.b771cf3de742ae034ee8c620e1082173@varnish-cache.org> References: <044.b771cf3de742ae034ee8c620e1082173@varnish-cache.org> Message-ID: <059.4aa0eee89c300555199bad14b68cbeea@varnish-cache.org> #1804: PROXYed client requests does not appear in varnishlog ----------------------+-------------------- Reporter: trygve | Owner: Type: defect | Status: new Priority: normal | Milestone: Component: varnishd | Version: trunk Severity: normal | Resolution: Keywords: | ----------------------+-------------------- Changes (by daghf): * version: => trunk * component: build => varnishd Comment: Confirmed in master -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Thu Oct 22 10:17:55 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Thu, 22 Oct 2015 10:17:55 -0000 Subject: [Varnish] #1805: Assert error in Tcheck() Message-ID: <043.acb5a31f440d7822afcc1e6804d74084@varnish-cache.org> #1805: Assert error in Tcheck() ---------------------------------+-------------------- Reporter: colas | Type: defect Status: new | Priority: normal Milestone: Varnish 4.0 release | Component: build Version: unknown | Severity: normal Keywords: | ---------------------------------+-------------------- Hy, We use varnish-4.0.3 revision b8c4a34. Started with all this params : {{{ NFILES=131072 MEMLOCK=164000 DAEMON_OPTS="-a :80 \ -T 0.0.0.0:6083 \ -f /etc/varnish/cfvarnish.vcl \ -s file,/opt/cache/varnishstorage1.bin,35G \ -s file,/opt/cache/varnishstorage2.bin,35G \ -s file,/opt/cache/varnishstorage3.bin,35G \ -S /etc/varnish/secret \ -p http_gzip_support=on \ -p thread_pools=6 \ -p thread_pool_min=100 \ -p thread_pool_max=800 \ -p nuke_limit=500 \ -p cli_timeout=20 \ -l 160m" }}} Sometimes I get panic like this. This error could be very frequent, each minutes for exemple. {{{ varnish> panic.show 200 Last panic at: Thu, 22 Oct 2015 09:25:49 GMT Assert error in Tcheck(), cache/cache.h line 1296: Condition((t.b) != 0) not true. thread = (cache-worker) version = varnish-4.0.3 revision b8c4a34 ident = Linux,3.2.0-4-amd64,x86_64,-sfile,-sfile,-sfile,-smalloc,-hcritbit,epoll Backtrace: 0x435424: /usr/sbin/varnishd() [0x435424] 0x40e7e8: /usr/sbin/varnishd() [0x40e7e8] 0x429e33: /usr/sbin/varnishd() [0x429e33] 0x42c856: /usr/sbin/varnishd(http_FilterResp+0x86) [0x42c856] 0x420d3e: /usr/sbin/varnishd() [0x420d3e] 0x42142f: /usr/sbin/varnishd() [0x42142f] 0x438301: /usr/sbin/varnishd(Pool_Work_Thread+0x381) [0x438301] 0x44b29d: /usr/sbin/varnishd() [0x44b29d] 0x7f5fe03dbb50: /lib/x86_64-linux-gnu/libpthread.so.0(+0x6b50) [0x7f5fe03dbb50] 0x7f5fe012595d: /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f5fe012595d] busyobj = 0x7f456d889020 { ws = 0x7f456d8890e0 { OVERFLOW id = ""o", {s,f,r,e} = {0x7f456d88b008,+57368,(nil),+57368}, }, refcnt = 2 retries = 3 failed = 0 state = 1 is_do_stream is_do_pass is_uncacheable bodystatus = 0 (none), }, http[bereq] = { ws = 0x7f456d8890e0["o] "GET", "/aaaaaaaaaaaaaaaa.htm", "HTTP/1.1", "User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:41.0) Gecko/20100101 Firefox/41.0", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", "Accept-Language: fr,fr-FR;q=0.8,en-US;q=0.5,en;q=0.3", "Accept-Encoding: gzip, deflate", "Referer: http://aaaaaaaaaaaaaaaaaaaaaaaaaaa", "Cookie: md_users_res=1280x800; p=3; md_data=cham51;, "Cache-Control: max-age=0", "X-UA-Device: pc", "Via: 1.1 varnish-v4", "grace: none", "X-Forwarded-For: xx.xxx.xx.xxx, xx.xxx.xx.xxx", "Host: aaaaaaaaaaaaaaaaaa", "X-Pass: Connected", "X-Varnish: 655363", "X-Varnish: 2326529", "X-Varnish: 2326530", }, http[beresp] = { ws = 0x7f456d8890e0["o] "HTTP/1.1", "Backend fetch failed", "Server: Varnish", }, ws = 0x7f456d889270 { id = "obj", {s,f,r,e} = {0x7f55ebdb12c8,0x7f55ebdb12c8,(nil),+56}, }, objcore (FETCH) = 0x7f4578840020 { refcnt = 2 flags = 0x106 objhead = 0x7f5fdf450500 } obj (FETCH) = 0x7f55ebdb1180 { vxid = 2149810179, http[obj] = { ws = 0x7f456d889270[obj] "HTTP/1.1", }, len = 0, store = { }, }, } }}} The error appears when I do a return(pass) in vcl_recv File attache : the configuration vcl file -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Thu Oct 22 10:50:54 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Thu, 22 Oct 2015 10:50:54 -0000 Subject: [Varnish] #1803: Varnishncsa truncated referer In-Reply-To: <045.b8e65587d3775974af1bb7ca0a54c6f6@varnish-cache.org> References: <045.b8e65587d3775974af1bb7ca0a54c6f6@varnish-cache.org> Message-ID: <060.b22a985c93a9474ca549bc3becb51553@varnish-cache.org> #1803: Varnishncsa truncated referer -------------------------+----------------------- Reporter: nixmind | Owner: Type: defect | Status: needinfo Priority: normal | Milestone: Component: varnishncsa | Version: unknown Severity: normal | Resolution: Keywords: | -------------------------+----------------------- Comment (by nixmind): Hi, Please I would like to know if there is a way to change the **shm_reclen** parameter permanently in a file. I try to put it in **/etc/default/varnish** like this : {{{ shm_reclen = 1024 }}} But it didn't change anything. As it is a '''run time parameter''', does it means it should only be set at run time? Thanks. Regards. -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Thu Oct 22 11:00:44 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Thu, 22 Oct 2015 11:00:44 -0000 Subject: [Varnish] #1803: Varnishncsa truncated referer In-Reply-To: <045.b8e65587d3775974af1bb7ca0a54c6f6@varnish-cache.org> References: <045.b8e65587d3775974af1bb7ca0a54c6f6@varnish-cache.org> Message-ID: <060.5726be35820da05825d3f1c6ab12335c@varnish-cache.org> #1803: Varnishncsa truncated referer -------------------------+----------------------- Reporter: nixmind | Owner: Type: defect | Status: needinfo Priority: normal | Milestone: Component: varnishncsa | Version: unknown Severity: normal | Resolution: Keywords: | -------------------------+----------------------- Comment (by gquintard): Hi nixmind, This parameter, as all the others in varnishadm can be set at launch time using "-p param=value" in the varnishd command. If you have /etc/defaulr/varnish, I guess you have to add it to the DAEMON_OPTS variable. -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Thu Oct 22 14:13:43 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Thu, 22 Oct 2015 14:13:43 -0000 Subject: [Varnish] #1806: one minute delay on return (pipe) and a POST-Request Message-ID: <043.2e737ea61af7354d1ac52a29a065755e@varnish-cache.org> #1806: one minute delay on return (pipe) and a POST-Request -------------------+---------------------- Reporter: butzi | Type: defect Status: new | Priority: normal Milestone: | Component: varnishd Version: 4.1.0 | Severity: major Keywords: | -------------------+---------------------- sending a POST-Request to a ressource that is return (pipe); in vcl_fetch results sometimes (but quite often >50%) in a delayed delivery of nearly one minute. The parts of my vcl: {{{ sub vcl_fetch { ... if (req.http.host ~ "callcenter") { return (pipe); } ... } sub vcl_pipe { set bereq.http.connection = "close"; return (pipe); } }}} A line from varnishlog: {{{ - Timestamp PipeSess: 1445515486.047836 61.341283 61.340986 }}} Sometimes the value of the second and third number is 0.xxx seconds, so there is not a 100% failure. Calling GET-Ressources are working all the time, there is no delay detected. Sending the request directly to the apache server (without varnish in the middle), there is no delay. Currently the workaround for this bug is, to pass the ressource to the backends, but this should not be the default way for the future. -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Thu Oct 22 14:33:56 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Thu, 22 Oct 2015 14:33:56 -0000 Subject: [Varnish] #1807: Assert error in cnt_deliver() Message-ID: <045.c846bac4413e89d548cb34e49709b1fd@varnish-cache.org> #1807: Assert error in cnt_deliver() --------------------------+---------------------- Reporter: llavaud | Type: defect Status: new | Priority: normal Milestone: | Component: varnishd Version: 4.1.0-TP1 | Severity: normal Keywords: Assert error | --------------------------+---------------------- {{{ webcache12:~# varnishadm panic.show Last panic at: Wed, 21 Oct 2015 18:17:05 GMT Assert error in cnt_deliver(), cache/cache_req_fsm.c line 119: Condition((HTTP_Decode(req->resp, ObjGetattr(req->wrk, req->objcore, OA_HEADERS, ((void *)0)))) == 0) not true. thread = (cache-worker) version = varnish-4.1.0-tp1 revision 0e4e1bc ident = Linux,3.2.0-4-amd64,x86_64,-junix,-sfile,-smalloc,-hcritbit,epoll Backtrace: 0x4343a4: pan_ic+0x134 0x43938b: CNT_Request+0x23cb 0x44eba3: HTTP1_Session+0x133 0x43a841: SES_Proto_Req+0x61 0x44947a: WRK_Thread+0x48a 0x4499eb: pool_thread+0x2b 0x7f02e6346b50: libpthread.so.0(+0x6b50) [0x7f02e6346b50] 0x7f02e609095d: libc.so.6(clone+0x6d) [0x7f02e609095d] req = 0x7ee940406020 { sp = 0x7ee905224d20, vxid = 862409941, step = R_STP_DELIVER, req_body = R_BODY_TAKEN, restarts = 0, esi_level = 0, sp = 0x7ee905224d20 { fd = 804, vxid = 862409940, client = xxx.xxx.xxx.xxx 11623, step = S_STP_H1PROC, }, worker = 0x7ee9cb336c30 { stack = {0x7ee9cb337000 -> 0x7ee9cb32b000} ws = 0x7ee9cb336e30 { id = "wrk", {s,f,r,e} = {0x7ee9cb335bd0,0x7ee9cb335bd0,(nil),+4088}, }, VCL::method = PASS, VCL::return = fetch, VCL::methods = {RECV, PASS, HASH}, }, ws = 0x7ee940406220 { id = "req", {s,f,r,e} = {0x7ee940408028,+11816,(nil),+253904}, }, http[req] = { ws = 0x7ee940406220[req] "POST", "/action/login", "HTTP/1.1", "Connection: keep-alive", "Content-Length: 260", "Accept: application/json, text/javascript, */*", "Origin: http://replay.gulli.fr", "X-Requested-With: XMLHttpRequest", "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.71 Safari/537.36", "Content-Type: application/x-www-form-urlencoded", "Referer: http://replay.gulli.fr/action/login?ReturnUrl=http://replay.gulli.fr/recherche/replay?searchText=Chica+vampiro?return_modal=http://replay.gulli.fr/layout/set/ajax/replay/favori/(prog)/chica_vampiro", "Accept-Encoding: gzip, deflate", "Accept-Language: fr-FR,fr;q=0.8,en-US;q=0.6,en;q=0.4", "X-Forwarded-For: xxx.xxx.xxx.xxx", "Host: replay.gulli.fr", "Surrogate-Capability: abc=ESI/1.0", "Cookie: my_cookies", }, http[resp] = { ws = 0x7ee940406220[req] }, vcl = { srcname = { "input", "Builtin", "/etc/varnish/backends/probe.vcl", "/etc/varnish/backends/webssi.vcl", "/etc/varnish/backends/webdyn.vcl", "/etc/varnish/backends/webservice.vcl", "/etc/varnish/backends/bograph.vcl", "/etc/varnish/acl.vcl", "/etc/varnish/rules/init.vcl", "/etc/varnish/rules/recv.vcl", "/etc/varnish/rules/hash.vcl", "/etc/varnish/rules/hit.vcl", "/etc/varnish/rules/miss.vcl", "/etc/varnish/rules/backend_fetch.vcl", "/etc/varnish/rules/backend_response.vcl", "/etc/varnish/rules/backend_error.vcl", "/etc/varnish/rules/deliver.vcl", "/etc/varnish/rules/synth.vcl", "/etc/varnish/rules/pipe.vcl", }, }, objcore (REQ) = 0x7ee9b3b2c840 { refcnt = 1 flags = 0x104 objhead = 0x7f02e54ee500 stevedore = 0x7f02e54eb240 (malloc Transient) } flags = { wantbody, } }, -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Thu Oct 22 15:24:35 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Thu, 22 Oct 2015 15:24:35 -0000 Subject: [Varnish] #1808: Error in Varnish Request Flox Message-ID: <043.713eddd679a70b3a60e6c677bb2bf1e3@varnish-cache.org> #1808: Error in Varnish Request Flox -----------------------------+-------------------- Reporter: colas | Type: defect Status: new | Priority: high Milestone: Varnish 4.0-TP2 | Component: build Version: trunk | Severity: normal Keywords: | -----------------------------+-------------------- Hello, We use varnish-4.0.3 revision b8c4a34. I think there are an error with the Varnish request Flow. There a part of my conf {{{ vcl 4.0; import directors; import std; include "/etc/varnish/cfdoc.vcl"; include "/etc/varnish/devicedetect.vcl"; sub vcl_init { new xxx = directors.round_robin(); xxx.add_backend(xxx1); } acl acl_reseaudoc { ... "127.0.0.1"; } sub vcl_recv { call devicedetect; #On identifie les devices mobiles ou non (include devicedetect.vcl) if( client.ip ~ acl_reseaudoc) { # Force lookup if request is a no-cache request from client if (req.http.Pragma ~ "no-cache" && req.http.Cache-Control ~ "no-cache") { if (req.method == "GET" ) { # On ne purge que sur un GET, jamais sur un POST ban("req.url == "+req.url); # Send the "purge" command to the purge queue in a REGEXP form return (synth(202, "Confirmation de purge de la page")); } } } set req.backend_hint = xxx.backend(); # send all traffic to the vdir director set req.http.Via = "1.1 varnish-v4"; set req.http.grace = "none"; if (req.restarts == 0) { if (req.http.X-Forwarded-For) { # set or append the client.ip to X-Forwarded-For header set req.http.X-Forwarded-For = req.http.X -Forwarded-For + ", " + client.ip; } else { set req.http.X-Forwarded-For = client.ip; } } #For debug set req.http.X-Pass = "Debug-Pass"; return(pass); .... return(hash); } # Called after vcl_recv to create a hash value for the request. # This is used as a key to look up the object in Varnish. sub vcl_hash { set req.http.X-Pass = req.http.X-Pass+" hash"; hash_data(req.url); if (req.http.host) { hash_data(req.http.host); } else { hash_data(server.ip); } # On cache sp?cifiquement par type de device client if (!req.url ~ "^[^?]*\.(bmp|bz2|js|doc|eot|flv|gif|tgz|gz|ico|jpeg|jpg|less|pdf|png|rtf|swf|txt|woff|xml|mp3|mp4)(\?.*)?$") { if (req.http.X-UA-Device ~ "mobile") { hash_data("mobile"); } } } sub vcl_hit { set req.http.X-Pass = req.http.X-Pass+" hit"; # Called when a cache lookup is successful. if (obj.ttl >= 0s) { # normal hit return (deliver); } # We have no fresh fish. Lets look at the stale ones. if (std.healthy(req.backend_hint)) { # Backend is healthy. Limit age to 10s. if (obj.ttl + 10s > 0s) { set req.http.grace = "normal(limited)"; return (deliver); } else { # No candidate for grace. Fetch a fresh object. return(fetch); } } else { # backend is sick - use full grace if (obj.ttl + obj.grace > 0s) { # Object is in grace, deliver it # Automatically triggers a background fetch set req.http.grace = "full"; return (deliver); } else { # no graced object. return (fetch); } } } sub vcl_miss { set req.http.X-Pass = req.http.X-Pass+" miss"; return(fetch); } sub vcl_pass { set req.http.X-Pass = req.http.X-Pass+" pass"; } sub vcl_backend_fetch { }}} When I request a page I get this header : X-Pass:Debug-Pass hash Pass But if I read the documentation https://www.varnish- software.com/book/4.0/chapters/VCL_Basics.html#detailed-varnish-request- flow-for-the-client-worker-thread If I make a return(pass) from vcl_recv I must bypass vcl_hash. With my debug it's not the case. I have : vcl_recv => vcl_hash => vcl_pass thank -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Fri Oct 23 10:36:34 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Fri, 23 Oct 2015 10:36:34 -0000 Subject: [Varnish] #1809: Allow a Content-Length: 0 header on a 204 response Message-ID: <044.a5dc29fe0450f6da452e4791c81f5a8c@varnish-cache.org> #1809: Allow a Content-Length: 0 header on a 204 response --------------------------------+-------------------- Reporter: widodh | Type: defect Status: new | Priority: normal Milestone: | Component: build Version: trunk | Severity: normal Keywords: 204,content-length | --------------------------------+-------------------- Currently Varnish 4 errors out when a 204 response is given by a backend and a Content-Length header is present: In cache_fetch.c: {{{#!c } else if (http_IsStatus(bo->beresp, 204)) { /* * 204 is "No Content", obviously don't expect a body. * [RFC2616 10.2.5 p60] */ wrk->stats->fetch_204++; if (http_GetHdr(bo->beresp, H_Content_Length, NULL) || http_GetHdr(bo->beresp, H_Transfer_Encoding, NULL)) bo->htc->body_status = BS_ERROR; else bo->htc->body_status = BS_NONE; }}} If a Content-Length header is present it errors out. Looking at RFC 2616 10.2.5 it says: "The 204 response MUST NOT include a message-body, and thus is always terminated by the first empty line after the header fields" However, it doesn't say that Content-Length is not an allowed header here. Content-Length with a value of 0 is a valid case which should be allowed. I think we should allow a Content-Length header if the value is 0. -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Fri Oct 23 10:40:17 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Fri, 23 Oct 2015 10:40:17 -0000 Subject: [Varnish] #1809: Allow a Content-Length: 0 header on a 204 response In-Reply-To: <044.a5dc29fe0450f6da452e4791c81f5a8c@varnish-cache.org> References: <044.a5dc29fe0450f6da452e4791c81f5a8c@varnish-cache.org> Message-ID: <059.bb4f21e95f7c4e67f960e5d2bfcb5efd@varnish-cache.org> #1809: Allow a Content-Length: 0 header on a 204 response --------------------------------+-------------------- Reporter: widodh | Owner: Type: defect | Status: new Priority: normal | Milestone: Component: build | Version: trunk Severity: normal | Resolution: Keywords: 204,content-length | --------------------------------+-------------------- Comment (by widodh): A direct link to the RFC: https://www.ietf.org/rfc/rfc2616.txt -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Fri Oct 23 12:06:05 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Fri, 23 Oct 2015 12:06:05 -0000 Subject: [Varnish] #1809: Allow a Content-Length: 0 header on a 204 response In-Reply-To: <044.a5dc29fe0450f6da452e4791c81f5a8c@varnish-cache.org> References: <044.a5dc29fe0450f6da452e4791c81f5a8c@varnish-cache.org> Message-ID: <059.cab7ed2daed50ea51812597f65a4adf1@varnish-cache.org> #1809: Allow a Content-Length: 0 header on a 204 response --------------------------------+-------------------- Reporter: widodh | Owner: Type: defect | Status: new Priority: normal | Milestone: Component: build | Version: trunk Severity: normal | Resolution: Keywords: 204,content-length | --------------------------------+-------------------- Comment (by widodh): I've looked at this with a colleague and he pointed me at RFC7230: https://tools.ietf.org/html/rfc7230#section-3.3.2 {{{ A server MUST NOT send a Content-Length header field in any response with a status code of 1xx (Informational) or 204 (No Content). }}} That clarifies it. Varnish is keeping to the standard, BUT, the comment in the code should be updated that it points to RFC7230 and not to RFC2616. -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Sun Oct 25 11:31:47 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Sun, 25 Oct 2015 11:31:47 -0000 Subject: [Varnish] #1810: HTTP/1.0 EOF from backend broken Message-ID: <041.a4930a5c1bb922879486cb9c56f83795@varnish-cache.org> #1810: HTTP/1.0 EOF from backend broken ----------------------+------------------- Reporter: phk | Owner: phk Type: defect | Status: new Priority: normal | Milestone: Component: varnishd | Version: trunk Severity: normal | Keywords: ----------------------+------------------- {{{ varnishtest "POST HTTP/1.0 response" server s1 { non-fatal rxreq txresp -nolen -hdr "Connection: close" send "Hello World\n" delay .4 } -start varnish v1 -vcl+backend { } -start client c1 { txreq -req POST -proto HTTP/1.0 -hdr "Content-Length: 0" rxresp expect resp.bodylen == 12 } -run }}} -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Sun Oct 25 11:32:16 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Sun, 25 Oct 2015 11:32:16 -0000 Subject: [Varnish] #1810: HTTP/1.0 EOF from backend broken In-Reply-To: <041.a4930a5c1bb922879486cb9c56f83795@varnish-cache.org> References: <041.a4930a5c1bb922879486cb9c56f83795@varnish-cache.org> Message-ID: <056.b539a0df6f9fb37996a11c08a4d761ef@varnish-cache.org> #1810: HTTP/1.0 EOF from backend broken ----------------------+-------------------- Reporter: phk | Owner: phk Type: defect | Status: new Priority: normal | Milestone: Component: varnishd | Version: trunk Severity: normal | Resolution: Keywords: | ----------------------+-------------------- Comment (by phk): This problem was found by the Wemm-Field on the FreeBSD-project cluster -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Sun Oct 25 11:35:14 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Sun, 25 Oct 2015 11:35:14 -0000 Subject: [Varnish] #1810: HTTP/1.0 EOF from backend broken In-Reply-To: <041.a4930a5c1bb922879486cb9c56f83795@varnish-cache.org> References: <041.a4930a5c1bb922879486cb9c56f83795@varnish-cache.org> Message-ID: <056.6175cb6806382111b820af7740c22f81@varnish-cache.org> #1810: HTTP/1.0 EOF from backend broken ----------------------+-------------------- Reporter: phk | Owner: phk Type: defect | Status: new Priority: normal | Milestone: Component: varnishd | Version: trunk Severity: normal | Resolution: Keywords: | ----------------------+-------------------- Comment (by phk): For all but pipe requests, the following workaround can be used, unless the backend is known to not support HTTP/1.1: {{{ sub vcl_backend_request { set bereq.proto = "HTTP/1.1"; } }}} -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Sun Oct 25 23:04:27 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Sun, 25 Oct 2015 23:04:27 -0000 Subject: [Varnish] #1794: varnish*-tools do not die when terminal disconnects In-Reply-To: <041.5e0136e0890c547547da2629f527e416@varnish-cache.org> References: <041.5e0136e0890c547547da2629f527e416@varnish-cache.org> Message-ID: <056.f3f5893d99d6fd9f54ab9cf182e286fd@varnish-cache.org> #1794: varnish*-tools do not die when terminal disconnects --------------------+-------------------- Reporter: kwy | Owner: Type: defect | Status: new Priority: normal | Milestone: Component: build | Version: 4.0.3 Severity: normal | Resolution: Keywords: | --------------------+-------------------- Comment (by fgsch): Tested and confirms this patch solves the issue. Updated patch with doc fixes and amended test added. -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Sun Oct 25 23:08:46 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Sun, 25 Oct 2015 23:08:46 -0000 Subject: [Varnish] #1806: one minute delay on return (pipe) and a POST-Request In-Reply-To: <043.2e737ea61af7354d1ac52a29a065755e@varnish-cache.org> References: <043.2e737ea61af7354d1ac52a29a065755e@varnish-cache.org> Message-ID: <058.aacaca9bad073c69297506bc83df6a77@varnish-cache.org> #1806: one minute delay on return (pipe) and a POST-Request ----------------------+----------------------- Reporter: butzi | Owner: Type: defect | Status: needinfo Priority: normal | Milestone: Component: varnishd | Version: 4.1.0 Severity: major | Resolution: Keywords: | ----------------------+----------------------- Changes (by fgsch): * status: new => needinfo Comment: Is this something you can reproduce? Do you have some varnishlog output you can share? The delay might come from the server sending the response or from client reading it. -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Sun Oct 25 23:21:26 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Sun, 25 Oct 2015 23:21:26 -0000 Subject: [Varnish] #1809: Allow a Content-Length: 0 header on a 204 response In-Reply-To: <044.a5dc29fe0450f6da452e4791c81f5a8c@varnish-cache.org> References: <044.a5dc29fe0450f6da452e4791c81f5a8c@varnish-cache.org> Message-ID: <059.b2ea2658312430f1574f7a89dc7bb9e4@varnish-cache.org> #1809: Allow a Content-Length: 0 header on a 204 response -----------------------------+--------------------------------------------- Reporter: widodh | Owner: Federico G. Schwindt Type: defect | Status: closed Priority: normal | Milestone: Component: build | Version: trunk Severity: normal | Resolution: fixed Keywords: 204,content- | length | -----------------------------+--------------------------------------------- Changes (by Federico G. Schwindt ): * status: new => closed * owner: => Federico G. Schwindt * resolution: => fixed Comment: In [54471f2a86ae7234a31c97c0adb5e70a1b28ee38]: {{{ #!CommitTicketReference repository="" revision="54471f2a86ae7234a31c97c0adb5e70a1b28ee38" Update RFC reference Fixes #1809. }}} -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Sun Oct 25 23:24:19 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Sun, 25 Oct 2015 23:24:19 -0000 Subject: [Varnish] #1808: Error in Varnish Request Flox In-Reply-To: <043.713eddd679a70b3a60e6c677bb2bf1e3@varnish-cache.org> References: <043.713eddd679a70b3a60e6c677bb2bf1e3@varnish-cache.org> Message-ID: <058.893c0fdee1b1dba97e7d869405201ae2@varnish-cache.org> #1808: Error in Varnish Request Flox --------------------+------------------------------ Reporter: colas | Owner: Type: defect | Status: closed Priority: high | Milestone: Varnish 4.0-TP2 Component: build | Version: trunk Severity: normal | Resolution: invalid Keywords: | --------------------+------------------------------ Changes (by fgsch): * status: new => closed * resolution: => invalid Comment: The code is correct, the documentation is not. I've moved this to https://github.com/varnish/Varnish-Book/issues/70. -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Sun Oct 25 23:26:15 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Sun, 25 Oct 2015 23:26:15 -0000 Subject: [Varnish] #1803: Varnishncsa truncated referer In-Reply-To: <045.b8e65587d3775974af1bb7ca0a54c6f6@varnish-cache.org> References: <045.b8e65587d3775974af1bb7ca0a54c6f6@varnish-cache.org> Message-ID: <060.b8308ee6737bac9c2a5471587a5d8428@varnish-cache.org> #1803: Varnishncsa truncated referer -------------------------+---------------------- Reporter: nixmind | Owner: Type: defect | Status: closed Priority: normal | Milestone: Component: varnishncsa | Version: unknown Severity: normal | Resolution: invalid Keywords: | -------------------------+---------------------- Changes (by fgsch): * status: needinfo => closed * resolution: => invalid -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Mon Oct 26 07:47:57 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Mon, 26 Oct 2015 07:47:57 -0000 Subject: [Varnish] #1808: Error in Varnish Request Flox In-Reply-To: <043.713eddd679a70b3a60e6c677bb2bf1e3@varnish-cache.org> References: <043.713eddd679a70b3a60e6c677bb2bf1e3@varnish-cache.org> Message-ID: <058.0de143121e546ae0c06f959c5abb6c89@varnish-cache.org> #1808: Error in Varnish Request Flox --------------------+------------------------------ Reporter: colas | Owner: Type: defect | Status: closed Priority: high | Milestone: Varnish 4.0-TP2 Component: build | Version: trunk Severity: normal | Resolution: invalid Keywords: | --------------------+------------------------------ Comment (by colas): Why is needed to hash the request if we want to pass it ? We don't need to lookup. I don't really understand. -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Mon Oct 26 10:02:15 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Mon, 26 Oct 2015 10:02:15 -0000 Subject: [Varnish] #1800: PRIV_TASK in a vmod object method leads to code that cannot be compiled/linked In-Reply-To: <043.aa0af4bc03cba56c27b0370c89b02a0b@varnish-cache.org> References: <043.aa0af4bc03cba56c27b0370c89b02a0b@varnish-cache.org> Message-ID: <058.99c23da134ef439e398f3463732d945f@varnish-cache.org> #1800: PRIV_TASK in a vmod object method leads to code that cannot be compiled/linked --------------------+--------------------- Reporter: geoff | Owner: aondio Type: defect | Status: new Priority: normal | Milestone: Later Component: vmod | Version: 4.1.0 Severity: normal | Resolution: Keywords: | --------------------+--------------------- Comment (by geoff): aondio, thanks, I can confirm that make check succeeds for Varnish with the patch, and also that my use of PRIV_TASK in a vmod object method succeeds. But I think that a few things are still missing, as we talked about in email. Since the patch also addresses the use of PRIV_VCL and PRIV_TOP in methods, shouldn't there be tests for those as well? (The only tests are for PRIV_TASK.) The patch doesn't provide for the use of PRIV_CALL in a method, which makes sense because you don't need that for an object (just store your private data in the object). But someone might declare PRIV_CALL for a method in their vcc spec anyway. I think the right thing in that case would be for the vmodtool to an emit error with a helpful hint. (Otherwise what would happen? The same vgc compile-time error that I encountered above?) -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Mon Oct 26 12:26:09 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Mon, 26 Oct 2015 12:26:09 -0000 Subject: [Varnish] #1805: Assert error in Tcheck() In-Reply-To: <043.acb5a31f440d7822afcc1e6804d74084@varnish-cache.org> References: <043.acb5a31f440d7822afcc1e6804d74084@varnish-cache.org> Message-ID: <058.42b48835aceb09e7b0cad948d66a0a68@varnish-cache.org> #1805: Assert error in Tcheck() --------------------+---------------------------------- Reporter: colas | Owner: Type: defect | Status: closed Priority: normal | Milestone: Varnish 4.0 release Component: build | Version: unknown Severity: normal | Resolution: worksforme Keywords: | --------------------+---------------------------------- Changes (by martin): * status: new => closed * resolution: => worksforme Comment: You are running out of backend fetch workspace. Try increasing your workspace_backend parameter. Martin -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Mon Oct 26 14:15:06 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Mon, 26 Oct 2015 14:15:06 -0000 Subject: [Varnish] #1800: PRIV_TASK in a vmod object method leads to code that cannot be compiled/linked In-Reply-To: <043.aa0af4bc03cba56c27b0370c89b02a0b@varnish-cache.org> References: <043.aa0af4bc03cba56c27b0370c89b02a0b@varnish-cache.org> Message-ID: <058.0f8db919a708d135f35ffccddd3ad93a@varnish-cache.org> #1800: PRIV_TASK in a vmod object method leads to code that cannot be compiled/linked --------------------+--------------------- Reporter: geoff | Owner: aondio Type: defect | Status: new Priority: normal | Milestone: Later Component: vmod | Version: 4.1.0 Severity: normal | Resolution: Keywords: | --------------------+--------------------- Comment (by geoff): daghf mentioned in IRC today that PRIV_CALL in an object method works fine without the patch. I gave it a try and confirm that it's all right. So all that I think is missing are tests to verify PRIV_VCL and PRIV_TOP in object methods. -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Mon Oct 26 15:52:07 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Mon, 26 Oct 2015 15:52:07 -0000 Subject: [Varnish] #1794: varnish*-tools do not die when terminal disconnects In-Reply-To: <041.5e0136e0890c547547da2629f527e416@varnish-cache.org> References: <041.5e0136e0890c547547da2629f527e416@varnish-cache.org> Message-ID: <056.5bbe275f09bcbbd607110325075bbc6e@varnish-cache.org> #1794: varnish*-tools do not die when terminal disconnects --------------------+--------------------------------------------- Reporter: kwy | Owner: Federico G. Schwindt Type: defect | Status: closed Priority: normal | Milestone: Component: build | Version: 4.0.3 Severity: normal | Resolution: fixed Keywords: | --------------------+--------------------------------------------- Changes (by Federico G. Schwindt ): * status: new => closed * owner: => Federico G. Schwindt * resolution: => fixed Comment: In [b214d4806ed9f63a8bf95db29c176f1da86be13b]: {{{ #!CommitTicketReference repository="" revision="b214d4806ed9f63a8bf95db29c176f1da86be13b" Handle terminal disconnections correctly Change SIGHUP handling depending on whether we're running in daemon mode or foreground. The former will continue rotating the logs, the latter will abort the loop and die gracefully. Fixes #1794. }}} -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Tue Oct 27 09:36:04 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Tue, 27 Oct 2015 09:36:04 -0000 Subject: [Varnish] #1805: Assert error in Tcheck() In-Reply-To: <043.acb5a31f440d7822afcc1e6804d74084@varnish-cache.org> References: <043.acb5a31f440d7822afcc1e6804d74084@varnish-cache.org> Message-ID: <058.475e78032b9f95bfeb8f8e5a3676ec17@varnish-cache.org> #1805: Assert error in Tcheck() --------------------+---------------------------------- Reporter: colas | Owner: Type: defect | Status: closed Priority: normal | Milestone: Varnish 4.0 release Component: build | Version: unknown Severity: normal | Resolution: worksforme Keywords: | --------------------+---------------------------------- Comment (by colas): Thank you. It works really better now ;) -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Tue Oct 27 14:28:42 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Tue, 27 Oct 2015 14:28:42 -0000 Subject: [Varnish] #1804: PROXYed client requests does not appear in varnishlog In-Reply-To: <044.b771cf3de742ae034ee8c620e1082173@varnish-cache.org> References: <044.b771cf3de742ae034ee8c620e1082173@varnish-cache.org> Message-ID: <059.a56f7000d340e1b2a9dc92dd687285b8@varnish-cache.org> #1804: PROXYed client requests does not appear in varnishlog ----------------------+----------------------------------------------- Reporter: trygve | Owner: Martin Blix Grydeland Type: defect | Status: closed Priority: normal | Milestone: Component: varnishd | Version: trunk Severity: normal | Resolution: fixed Keywords: | ----------------------+----------------------------------------------- Changes (by Martin Blix Grydeland ): * owner: => Martin Blix Grydeland * status: new => closed * resolution: => fixed Comment: In [86b3f9feca54af49ce194f5d6bed9e4dcfa54971]: {{{ #!CommitTicketReference repository="" revision="86b3f9feca54af49ce194f5d6bed9e4dcfa54971" Log proxy related messages on the session, not on the request. The proxy log records were attempted to be logged on the request' log buffer, which at that point in time has not yet been set up and does not posess a VXID. This caused VXID==0 log records to be inserted in the beginning of the log for the first request on the session when using proxy protocol, and subsequently the VSL to fail picking them out as valid request log transactions. Fix by logging the PROXY handling related messages on the session, in which they belong. Fixes: #1804 }}} -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Wed Oct 28 10:14:05 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Wed, 28 Oct 2015 10:14:05 -0000 Subject: [Varnish] #1811: Varnish crashes when a PRIV_TASK/PRIV_TOP function is invoked from vcl_init Message-ID: <043.a4d47d0bdfd5863ed7aebbdb313b79bc@varnish-cache.org> #1811: Varnish crashes when a PRIV_TASK/PRIV_TOP function is invoked from vcl_init ----------------------+------------------- Reporter: daghf | Owner: Type: defect | Status: new Priority: normal | Milestone: Component: varnishd | Version: trunk Severity: normal | Keywords: ----------------------+------------------- `` varnishd[17072]: Child (17074) Panic message:#012Assert error in VRT_priv_task(), cache/cache_vrt_priv.c line 119:#12 Condition((ctx->bo) != NULL) not true. `` what happens is that VRT_priv_dynamic stores its list of privs in one of ctx->req->sp->privs or ctx->bo->privs, neither of which are available from vcl_init. I'm not sure if the solution is to teach the VCL compiler to disallow the invocation of PRIV_{TASK,TOP} in vcl_init, or if we can find a better way to handle this? (Discovered by github user hirschnase, see https://github.com/varnish /libvmod-curl/issues/30) -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Thu Oct 29 13:18:53 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Thu, 29 Oct 2015 13:18:53 -0000 Subject: [Varnish] #1810: HTTP/1.0 EOF from backend broken In-Reply-To: <041.a4930a5c1bb922879486cb9c56f83795@varnish-cache.org> References: <041.a4930a5c1bb922879486cb9c56f83795@varnish-cache.org> Message-ID: <056.d8dfb376d673b3d2526ba2919cbea536@varnish-cache.org> #1810: HTTP/1.0 EOF from backend broken ----------------------+-------------------- Reporter: phk | Owner: phk Type: defect | Status: new Priority: normal | Milestone: Component: varnishd | Version: trunk Severity: normal | Resolution: Keywords: | ----------------------+-------------------- Comment (by phk): Ok, this is more complicated than I thought. If we send a HTTP/1.0 request to the backend, the backend is in its full right to send us an answer with "HTTP/1.1" as protocol, *as long* as it only uses HTTP/1.0 semantics. We fail because we trust the protocol field and semantics match. We need to look both at what we sent to the backend and what we got back, to determine semantics. -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Thu Oct 29 15:38:06 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Thu, 29 Oct 2015 15:38:06 -0000 Subject: [Varnish] #1805: Assert error in Tcheck() In-Reply-To: <043.acb5a31f440d7822afcc1e6804d74084@varnish-cache.org> References: <043.acb5a31f440d7822afcc1e6804d74084@varnish-cache.org> Message-ID: <058.2001852a55019d6f3986a6991d9d7f76@varnish-cache.org> #1805: Assert error in Tcheck() --------------------+---------------------------------- Reporter: colas | Owner: Type: defect | Status: reopened Priority: normal | Milestone: Varnish 4.0 release Component: build | Version: unknown Severity: normal | Resolution: Keywords: | --------------------+---------------------------------- Changes (by colas): * status: closed => reopened * resolution: worksforme => Comment: We changed it to ''' workspace_backend=256k''' , but now we have some backend_fetch_failed whereas backend respond with status code 200. With varnishlog we get this error : {{{ - BereqHeader X-Forwarded-For: xx.xx.xx.xx, xx.xx.xx.xx - BereqHeader Host: www.mysite.com - BereqHeader X-Pass: Connected - BereqHeader X-Pass-D: Connected - hash - Pass - BereqHeader X-Varnish: 15568164 - VCL_call BACKEND_FETCH - VCL_return fetch - BackendOpen 192 backend1(xx.xx.xx.xx,,80) xx.xx.xx.xx 53626 - Backend 192 backend backend1(xx.xx.xx.xx,,80) - Timestamp Bereq: 1446131181.479810 0.000912 0.000912 - BogoHeader Too many headers: Set-Cookie: CF_user_ - HttpGarbage "HTTP/1.1%00200%00OK%00%0aDate: Thu, 29 Oct 2015 15:06:21 GMT%00%0aServer: Apache%00%0aExpires: 0%00%0aCache-Control: private, no-store, no-cache, must-revalidate%00%0aPragma: no-cache%00 %0aSet-Cookie: CFSESSID=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; path=/; domain=.....%00" }}} Why again this error ? Something wrong ? And other parameter to change ? -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Thu Oct 29 15:56:25 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Thu, 29 Oct 2015 15:56:25 -0000 Subject: [Varnish] #1805: Assert error in Tcheck() In-Reply-To: <043.acb5a31f440d7822afcc1e6804d74084@varnish-cache.org> References: <043.acb5a31f440d7822afcc1e6804d74084@varnish-cache.org> Message-ID: <058.a32691adb491e86298b4024e16a2727b@varnish-cache.org> #1805: Assert error in Tcheck() --------------------+---------------------------------- Reporter: colas | Owner: Type: defect | Status: reopened Priority: normal | Milestone: Varnish 4.0 release Component: build | Version: unknown Severity: normal | Resolution: Keywords: | --------------------+---------------------------------- Comment (by daghf): You are hitting http_max_hdr, which controls the maximum number of header lines allowed. -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Thu Oct 29 15:57:08 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Thu, 29 Oct 2015 15:57:08 -0000 Subject: [Varnish] #1805: Assert error in Tcheck() In-Reply-To: <043.acb5a31f440d7822afcc1e6804d74084@varnish-cache.org> References: <043.acb5a31f440d7822afcc1e6804d74084@varnish-cache.org> Message-ID: <058.d50c8c3f8dd0a430e13fed2e519187f2@varnish-cache.org> #1805: Assert error in Tcheck() --------------------+---------------------------------- Reporter: colas | Owner: Type: defect | Status: closed Priority: normal | Milestone: Varnish 4.0 release Component: build | Version: unknown Severity: normal | Resolution: worksforme Keywords: | --------------------+---------------------------------- Changes (by daghf): * status: reopened => closed * resolution: => worksforme -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Thu Oct 29 16:39:24 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Thu, 29 Oct 2015 16:39:24 -0000 Subject: [Varnish] #1805: Assert error in Tcheck() In-Reply-To: <043.acb5a31f440d7822afcc1e6804d74084@varnish-cache.org> References: <043.acb5a31f440d7822afcc1e6804d74084@varnish-cache.org> Message-ID: <058.72dde211b52839621f59b879d7213acb@varnish-cache.org> #1805: Assert error in Tcheck() --------------------+---------------------------------- Reporter: colas | Owner: Type: defect | Status: reopened Priority: normal | Milestone: Varnish 4.0 release Component: build | Version: unknown Severity: normal | Resolution: Keywords: | --------------------+---------------------------------- Changes (by colas): * status: closed => reopened * resolution: worksforme => Comment: Thanks I haven't this error anymore. I get an new one : {{{ - BereqMethod POST - BereqURL /proxy.php?xdp_path=http%3A%2F%2Fmonsite.com%2Fcf_log.php - BereqProtocol HTTP/1.1 - BereqHeader Accept: application/json, text/javascript, */*; q=0.01 - BereqHeader Content-Type: application/x-www-form-urlencoded; charset=UTF-8 - BereqHeader X-Requested-With: XMLHttpRequest - BereqHeader Referer: - BereqHeader Accept-Language: fr-FR,fr;q=0.5 - BereqHeader Accept-Encoding: gzip, deflate - BereqHeader User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; LCJB; rv:11.0) like Gecko - BereqHeader Content-Length: 635 - BereqHeader DNT: 1 - BereqHeader Cache-Control: no-cache - BereqHeader Cookie: fofirdId=c652b193-cbff-4a0e-abdf-836087b20393; xtvrn=$390974$; xtan390974=-; xtant390974=1; CF_SESSION_SYNC=1446135592 - BereqHeader X-UA-Device: pc - BereqHeader X-UA-Device-Simplified: desktop - BereqHeader Via: 1.1 varnish-v4 - BereqHeader grace: none - BereqHeader X-Forwarded-For: xx.xx.xx.xx, xx.xx.xx.xx - BereqHeader Host: monsite.com - BereqHeader X-Pass: POST request - BereqHeader X-Pass-D: POST request - hash - Pass - BereqHeader X-Varnish: 2031880 - VCL_call BACKEND_FETCH - VCL_return fetch - BackendOpen 258 backend1(xx.xx.xx.xx,,80) xx.xx.xx.xx 40582 - Backend 258 backend backedn1(xx.xx.xx.xx,,80) - FetchError req.body read error: 104 (Connection reset by peer) - FetchError backend write error: 104 (Connection reset by peer) }}} is there any page where I could find all possible error and find how to solve it ? I don't get the error all time, for exemple I can't reproduce from my browser. the response of this call is empty because it's just log and the content- length = 20 -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Fri Oct 30 10:12:52 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Fri, 30 Oct 2015 10:12:52 -0000 Subject: [Varnish] #1796: Assert error in SES_Proto_Req In-Reply-To: <045.2141fa04002dfbf25286642bf4066bde@varnish-cache.org> References: <045.2141fa04002dfbf25286642bf4066bde@varnish-cache.org> Message-ID: <060.04b1ae9b5866973534e128758cca6c0d@varnish-cache.org> #1796: Assert error in SES_Proto_Req --------------------------+----------------------- Reporter: llavaud | Owner: Type: defect | Status: needinfo Priority: normal | Milestone: Component: varnishd | Version: 4.1.0 Severity: normal | Resolution: Keywords: Assert error | --------------------------+----------------------- Changes (by lkarsten): * status: new => needinfo Comment: Hi. Have you adjusted any parameters on this setup? (-p startup options) -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Fri Oct 30 10:27:21 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Fri, 30 Oct 2015 10:27:21 -0000 Subject: [Varnish] #1796: Assert error in SES_Proto_Req In-Reply-To: <045.2141fa04002dfbf25286642bf4066bde@varnish-cache.org> References: <045.2141fa04002dfbf25286642bf4066bde@varnish-cache.org> Message-ID: <060.865b4bdc53211dce5516f1560c202af1@varnish-cache.org> #1796: Assert error in SES_Proto_Req --------------------------+---------------------------------------- Reporter: llavaud | Owner: Poul-Henning Kamp Type: defect | Status: closed Priority: normal | Milestone: Component: varnishd | Version: 4.1.0 Severity: normal | Resolution: fixed Keywords: Assert error | --------------------------+---------------------------------------- Changes (by Poul-Henning Kamp ): * status: needinfo => closed * owner: => Poul-Henning Kamp * resolution: => fixed Comment: In [e06d5dc0804ce7e81ccee7a5b95459c5564a196a]: {{{ #!CommitTicketReference repository="" revision="e06d5dc0804ce7e81ccee7a5b95459c5564a196a" Don't attempt to allocate a V1L from the workspace if it is overflowed. Fixes: #1796 Also triggered by: The Wemm-Field }}} -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Fri Oct 30 14:26:20 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Fri, 30 Oct 2015 14:26:20 -0000 Subject: [Varnish] #1810: HTTP/1.0 EOF from backend broken In-Reply-To: <041.a4930a5c1bb922879486cb9c56f83795@varnish-cache.org> References: <041.a4930a5c1bb922879486cb9c56f83795@varnish-cache.org> Message-ID: <056.bebe6bc141a4414ae3b9bc9a6ed1d92e@varnish-cache.org> #1810: HTTP/1.0 EOF from backend broken ----------------------+--------------------- Reporter: phk | Owner: phk Type: defect | Status: closed Priority: normal | Milestone: Component: varnishd | Version: trunk Severity: normal | Resolution: fixed Keywords: | ----------------------+--------------------- Changes (by Poul-Henning Kamp ): * status: new => closed * resolution: => fixed Comment: In [165f191d68c2637629b6dd4303293c328745acae]: {{{ #!CommitTicketReference repository="" revision="165f191d68c2637629b6dd4303293c328745acae" So one of those strange cornercases in HTTP/1 If we send the backend a HTTP/1.0 request, and it doesn't have a Content-Length, it cannot use Chunked and must fall back to EOF. However, the protocol field in the response tells us what version backend *could* have used, not what it *does* use. So we can get a response with HTTP/1.1 and EOF, following HTTP/1.0 semantics - because we asked for it. Most sensible backends avoid this, either by buffering and creation of a C-L or, smartly, returning "HTTP/1.0", even though that is strictly speaking against the apocrphal texts. Anyway, now we cope... Fixes: #1810 }}} -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Fri Oct 30 15:44:10 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Fri, 30 Oct 2015 15:44:10 -0000 Subject: [Varnish] #1803: Varnishncsa truncated referer In-Reply-To: <045.b8e65587d3775974af1bb7ca0a54c6f6@varnish-cache.org> References: <045.b8e65587d3775974af1bb7ca0a54c6f6@varnish-cache.org> Message-ID: <060.e40ac09582551b9bfed621ac52d967dc@varnish-cache.org> #1803: Varnishncsa truncated referer -------------------------+---------------------- Reporter: nixmind | Owner: Type: defect | Status: closed Priority: normal | Milestone: Component: varnishncsa | Version: unknown Severity: normal | Resolution: invalid Keywords: | -------------------------+---------------------- Comment (by nixmind): Replying to [comment:8 fgsch]: Hi, Appologises, I forgot to close the ticket. I just want to that setting vsl_reclen solved my problem. Regards. -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Fri Oct 30 17:56:48 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Fri, 30 Oct 2015 17:56:48 -0000 Subject: [Varnish] #1812: Varnish 3.0.2 behind a SSL-Terminator Message-ID: <045.43dfeb5842011c036904461aac4b99f8@varnish-cache.org> #1812: Varnish 3.0.2 behind a SSL-Terminator -----------------------------------+-------------------- Reporter: nixmind | Type: defect Status: new | Priority: high Milestone: | Component: build Version: 3.0.2 | Severity: normal Keywords: redirect, http, https | -----------------------------------+-------------------- Hi Please do someone know if **varnish 3.0.2** support http redirection to https. In fact I have a varnish cache server behind a ssl terminator (an aws external loadbalancer on which I set a **http** and a **https** listener). I would like the varnish when it receives a http request, to redirect in https, and directly send back the response (resquest?) to the loadbalancer, and the loadbalancer will receive the response as a **https** request and forward it the varnih which will then forward it to its own backend. But it seems like my varnish cache don't redirect back to the loadbalancer but redirect the https request to its backend. However the backend behind the varnish I don't have a https backend, I get timemout when I issue a http request. When the client enter https in the browser it works. The problem is with http request. Here is my configuration : In **vcl_recv** : if (client.ip != "127.0.0.1" && server.port == 80 && req.http.host ~ "^(?i)mydomain.com") { set req.http.x-redir = "https://" + req.http.host + req.url; #return(synth(850, "Moved permanently")); error 850 "Moved permanently"; } In **vcl_error** : if (obj.status == 850) { set obj.http.Location = req.http.x-redir; set obj.status = 302; return (deliver); } Can someone help please. I can't upgrade my varnish version manually at the moment. Thanks -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Fri Oct 30 18:17:15 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Fri, 30 Oct 2015 18:17:15 -0000 Subject: [Varnish] #1812: Varnish 3.0.2 behind a SSL-Terminator In-Reply-To: <045.43dfeb5842011c036904461aac4b99f8@varnish-cache.org> References: <045.43dfeb5842011c036904461aac4b99f8@varnish-cache.org> Message-ID: <060.5624a6b46c5d7754c5655712aa02f213@varnish-cache.org> #1812: Varnish 3.0.2 behind a SSL-Terminator -----------------------------------+-------------------- Reporter: nixmind | Owner: Type: defect | Status: new Priority: high | Milestone: Component: build | Version: 3.0.2 Severity: normal | Resolution: Keywords: redirect, http, https | -----------------------------------+-------------------- Comment (by nixmind): Update, I made a mistake in the proto check the code in the vcl_recv should look like this : {{{ if (client.ip != "127.0.0.1" && req.http.host ~ "^(?i)mydomain.com" && req.http.X-Forwarded-Proto !~ "(?i)https") { set req.http.x-redir = "https://" + req.http.host + req.url; #return(synth(850, "Moved permanently")); error 850 "Moved permanently"; } }}} But it still don't work. -- Ticket URL: Varnish The Varnish HTTP Accelerator From varnish-bugs at varnish-cache.org Fri Oct 30 19:13:13 2015 From: varnish-bugs at varnish-cache.org (Varnish) Date: Fri, 30 Oct 2015 19:13:13 -0000 Subject: [Varnish] #1812: Varnish 3.0.2 behind a SSL-Terminator In-Reply-To: <045.43dfeb5842011c036904461aac4b99f8@varnish-cache.org> References: <045.43dfeb5842011c036904461aac4b99f8@varnish-cache.org> Message-ID: <060.907c8a7a0e8ebe050beb2316052117f2@varnish-cache.org> #1812: Varnish 3.0.2 behind a SSL-Terminator -----------------------------------+---------------------- Reporter: nixmind | Owner: Type: defect | Status: closed Priority: high | Milestone: Component: build | Version: 3.0.2 Severity: normal | Resolution: invalid Keywords: redirect, http, https | -----------------------------------+---------------------- Changes (by fgsch): * status: new => closed * resolution: => invalid Comment: This system is for bug reporting. For support please see https://www.varnish-cache.org/support. Thanks. -- Ticket URL: Varnish The Varnish HTTP Accelerator