From hermunn at varnish-software.com  Mon May  2 08:22:05 2016
From: hermunn at varnish-software.com (Pål Hermunn Johansen)
Date: Mon, 02 May 2016 10:22:05 +0200
Subject: [master] c9dabec Tiny doc fix for varnishncsa
Message-ID: <E1ax97F-0002k0-1K@project.varnish-software.com>


commit c9dabecc2e2a3ef6515352da4b865bec61d21c91
Author: P?l Hermunn Johansen <hermunn at varnish-software.com>
Date:   Mon May 2 10:20:49 2016 +0200

    Tiny doc fix for varnishncsa

diff --git a/doc/sphinx/reference/varnishncsa.rst b/doc/sphinx/reference/varnishncsa.rst
index 381cdd4..5d01d9a 100644
--- a/doc/sphinx/reference/varnishncsa.rst
+++ b/doc/sphinx/reference/varnishncsa.rst
@@ -177,14 +177,13 @@ Supported formatters are:
     Output value set by std.log("key:value") in VCL.
 
   VSL:tag or VSL:tag[field]
-
     The value of the varnishlog entry with the given tag.  If field is
     specified, only the selected part is shown.  Defaults to "-"
     (without the quotes) when the key is not seen, or when the field
     is out of bounds.  If a key appears several times in a single
     transaction, only the first occurrence is used.  Example: The
     formatter %{VSL:Begin[2]}x will print the second field of the
-    Begin tag, which is the vxid of the parent transaction.
+    Begin tag, which is the VXID of the parent transaction.
 
 SIGNALS
 =======



From hermunn at varnish-software.com  Mon May  2 08:29:04 2016
From: hermunn at varnish-software.com (Pål Hermunn Johansen)
Date: Mon, 02 May 2016 10:29:04 +0200
Subject: [4.1] 5b4e500 Tiny doc fix for varnishncsa
Message-ID: <E1ax9E1-0003AL-05@project.varnish-software.com>


commit 5b4e50037dc12b8c0ccfc75cb1a91bb1b9edf44a
Author: P?l Hermunn Johansen <hermunn at varnish-software.com>
Date:   Mon May 2 10:20:49 2016 +0200

    Tiny doc fix for varnishncsa

diff --git a/doc/sphinx/reference/varnishncsa.rst b/doc/sphinx/reference/varnishncsa.rst
index 381cdd4..5d01d9a 100644
--- a/doc/sphinx/reference/varnishncsa.rst
+++ b/doc/sphinx/reference/varnishncsa.rst
@@ -177,14 +177,13 @@ Supported formatters are:
     Output value set by std.log("key:value") in VCL.
 
   VSL:tag or VSL:tag[field]
-
     The value of the varnishlog entry with the given tag.  If field is
     specified, only the selected part is shown.  Defaults to "-"
     (without the quotes) when the key is not seen, or when the field
     is out of bounds.  If a key appears several times in a single
     transaction, only the first occurrence is used.  Example: The
     formatter %{VSL:Begin[2]}x will print the second field of the
-    Begin tag, which is the vxid of the parent transaction.
+    Begin tag, which is the VXID of the parent transaction.
 
 SIGNALS
 =======



From nils.goroll at uplex.de  Mon May  2 11:19:04 2016
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon, 02 May 2016 13:19:04 +0200
Subject: [master] 5ab1629 req->task members must be set in case we get onto
 the waitinglist
Message-ID: <E1axBsW-0004Zm-UR@project.varnish-software.com>


commit 5ab1629df0a9871376d95b4b37e1519c67931237
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Thu Apr 28 12:51:50 2016 +0200

    req->task members must be set in case we get onto the waitinglist
    
    Fixes #1928

diff --git a/bin/varnishd/http1/cache_http1_fsm.c b/bin/varnishd/http1/cache_http1_fsm.c
index 0174604..216dedf 100644
--- a/bin/varnishd/http1/cache_http1_fsm.c
+++ b/bin/varnishd/http1/cache_http1_fsm.c
@@ -392,13 +392,15 @@ HTTP1_Session(struct worker *wrk, struct req *req)
 			http1_setstate(sp, H1PROC);
 		} else if (st == H1PROC) {
 			req->transport = &HTTP1_transport;
+			req->task.func = http1_req;
+			req->task.priv = req;
 			if (CNT_Request(wrk, req) == REQ_FSM_DISEMBARK) {
-				req->task.func = http1_req;
-				req->task.priv = req;
 				http1_setstate(sp, H1BUSY);
 				return;
 			}
 			req->transport = NULL;
+			req->task.func = NULL;
+			req->task.priv = NULL;
 			http1_setstate(sp, H1CLEANUP);
 		} else if (st == H1CLEANUP) {
 			if (Req_Cleanup(sp, wrk, req))



From nils.goroll at uplex.de  Mon May  2 12:08:04 2016
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon, 02 May 2016 14:08:04 +0200
Subject: [master] ed7ca73 ban lurker: the olist can be empty when all bans got
 dup'ed
Message-ID: <E1axCdw-0007Jb-Ov@project.varnish-software.com>


commit ed7ca73068ecc33ce18f582cf54b939c49d17851
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Fri Apr 22 15:35:12 2016 +0200

    ban lurker: the olist can be empty when all bans got dup'ed
    
    All bans on olist can get marked completed as duplicates while we're
    working on them, in particular as we start at an offset of
    ban_lurker_age.
    
    Fixes #1919

diff --git a/bin/varnishd/cache/cache_ban_lurker.c b/bin/varnishd/cache/cache_ban_lurker.c
index 65c552e..6d038bb 100644
--- a/bin/varnishd/cache/cache_ban_lurker.c
+++ b/bin/varnishd/cache/cache_ban_lurker.c
@@ -145,9 +145,6 @@ ban_lurker_test_ban(struct worker *wrk, struct vsl_log *vsl, struct ban *bt,
 	unsigned tests;
 	int i;
 
-	/* It's an error to give an empty list to test against */
-	AZ(VTAILQ_EMPTY(obans));
-
 	/*
 	 * First see if there is anything to do, and if so, insert marker
 	 */



From hermunn at varnish-software.com  Mon May  2 14:02:05 2016
From: hermunn at varnish-software.com (Pål Hermunn Johansen)
Date: Mon, 02 May 2016 16:02:05 +0200
Subject: [4.1] ea101c7 Postpone calling VBT_wait until after V1P_Process
Message-ID: <E1axEQH-0005Gs-Vk@project.varnish-software.com>


commit ea101c7312d785670093b51aa55241de458366d9
Author: Dag Haavi Finstad <daghf at varnish-software.com>
Date:   Mon Mar 14 14:22:41 2016 +0100

    Postpone calling VBT_wait until after V1P_Process
    
    Fixes a bug where we wait for an event to fire on the backend socket
    prior to passing the request body.
    
    The VBT_Wait call is now handled in vbe_dir_finish.
    
    Test case by thomaslc
    
    Fixes: #1806

diff --git a/bin/varnishd/cache/cache_backend.c b/bin/varnishd/cache/cache_backend.c
index 48e3e97..039d3b2 100644
--- a/bin/varnishd/cache/cache_backend.c
+++ b/bin/varnishd/cache/cache_backend.c
@@ -296,8 +296,6 @@ vbe_dir_http1pipe(const struct director *d, struct req *req, struct busyobj *bo)
 	} else {
 		i = V1F_SendReq(req->wrk, bo, &v1a.bereq, 1);
 		VSLb_ts_req(req, "Pipe", W_TIM_real(req->wrk));
-		if (vbc->state == VBC_STATE_STOLEN)
-			VBT_Wait(req->wrk, vbc);
 		if (i == 0)
 			V1P_Process(req, vbc->fd, &v1a);
 		VSLb_ts_req(req, "PipeSess", W_TIM_real(req->wrk));
diff --git a/bin/varnishtest/tests/r01806.vtc b/bin/varnishtest/tests/r01806.vtc
new file mode 100644
index 0000000..5d2b376
--- /dev/null
+++ b/bin/varnishtest/tests/r01806.vtc
@@ -0,0 +1,33 @@
+varnishtest "#1806: return pipe w/ STOLEN connection"
+
+server s1 {
+	rxreq
+	expect req.url == "/pass-me"
+	txresp -body "I was PASSed\n"
+
+	rxreq
+	expect req.url == "/pipe-me"
+	txresp -body "I was PIPEd\n"
+} -start
+
+varnish v1 -vcl+backend {
+	sub vcl_recv {
+		if (req.url == "/pipe-me") {
+			return (pipe);
+		}
+
+		return (pass);
+	}
+} -start
+
+varnish v1 -cliok "param.set debug +syncvsl"
+
+client c1 {
+	txreq -url /pass-me
+	rxresp
+	expect resp.status == 200
+
+	txreq -req POST -url /pipe-me -body "asdf"
+	rxresp
+	expect resp.status == 200
+} -run



From hermunn at varnish-software.com  Tue May  3 13:03:04 2016
From: hermunn at varnish-software.com (Pål Hermunn Johansen)
Date: Tue, 03 May 2016 15:03:04 +0200
Subject: [4.1] a0c0481 ban lurker: the olist can be empty when all bans got
 dup'ed
Message-ID: <E1axZyi-00073W-Ti@project.varnish-software.com>


commit a0c0481e4f0f9e578e6a062645c62c6ebffe118e
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Fri Apr 22 15:35:12 2016 +0200

    ban lurker: the olist can be empty when all bans got dup'ed
    
    All bans on olist can get marked completed as duplicates while we're
    working on them, in particular as we start at an offset of
    ban_lurker_age.
    
    Fixes #1919

diff --git a/bin/varnishd/cache/cache_ban_lurker.c b/bin/varnishd/cache/cache_ban_lurker.c
index c3d4a19..ada0c05 100644
--- a/bin/varnishd/cache/cache_ban_lurker.c
+++ b/bin/varnishd/cache/cache_ban_lurker.c
@@ -145,9 +145,6 @@ ban_lurker_test_ban(struct worker *wrk, struct vsl_log *vsl, struct ban *bt,
 	unsigned tests;
 	int i;
 
-	/* It's an error to give an empty list to test against */
-	AZ(VTAILQ_EMPTY(obans));
-
 	/*
 	 * First see if there is anything to do, and if so, insert marker
 	 */



From nils.goroll at uplex.de  Tue May  3 14:31:06 2016
From: nils.goroll at uplex.de (Nils Goroll)
Date: Tue, 03 May 2016 16:31:06 +0200
Subject: [master] 8ae34bc ban lurker: extra ban_lurker_holdoff parameter
Message-ID: <E1axbLu-0003sX-LI@project.varnish-software.com>


commit 8ae34bc7371bcecdd75ee1d3ebf12c73eba788ef
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Mon Apr 18 21:01:24 2016 +0200

    ban lurker: extra ban_lurker_holdoff parameter

diff --git a/bin/varnishd/cache/cache_ban_lurker.c b/bin/varnishd/cache/cache_ban_lurker.c
index 6d038bb..1c3dfbe 100644
--- a/bin/varnishd/cache/cache_ban_lurker.c
+++ b/bin/varnishd/cache/cache_ban_lurker.c
@@ -131,7 +131,7 @@ ban_lurker_getfirst(struct vsl_log *vsl, struct ban *bt)
 		Lck_Unlock(&ban_mtx);
 		VSC_C_main->bans_lurker_contention++;
 		VSL_Flush(vsl, 0);
-		VTIM_sleep(cache_param->ban_lurker_sleep);
+		VTIM_sleep(cache_param->ban_lurker_holdoff);
 	}
 	return (oc);
 }
diff --git a/include/tbl/params.h b/include/tbl/params.h
index 2d6446a..3c03ce1 100644
--- a/include/tbl/params.h
+++ b/include/tbl/params.h
@@ -171,7 +171,7 @@ PARAM(
 	/* flags */	0,
 	/* s-text */
 	"How long the ban lurker sleeps after examining ${ban_lurker_batch} "
-	"objects and when it detects lock contention on object heads."
+	"objects."
 	"  Use this to pace the ban-lurker if it eats too many resources.\n"
 	"A value of zero will disable the ban lurker entirely.",
 	/* l-text */	"",
@@ -179,6 +179,21 @@ PARAM(
 )
 
 PARAM(
+	/* name */	ban_lurker_holdoff,
+	/* typ */	timeout,
+	/* min */	"0",
+	/* max */	NULL,
+	/* default */	"0.010",
+	/* units */	"seconds",
+	/* flags */	EXPERIMENTAL,
+	/* s-text */
+	"How long the ban lurker sleeps when giving way to lookup"
+	" due to lock contention.",
+	/* l-text */	"",
+	/* func */	NULL
+)
+
+PARAM(
 	/* name */	first_byte_timeout,
 	/* typ */	timeout,
 	/* min */	"0",



From nils.goroll at uplex.de  Tue May  3 14:31:06 2016
From: nils.goroll at uplex.de (Nils Goroll)
Date: Tue, 03 May 2016 16:31:06 +0200
Subject: [master] 1502d90 Handle races with lookup more efficiently
Message-ID: <E1axbLu-0003sd-N1@project.varnish-software.com>


commit 1502d90e901b36b9e49a4d0b252c9b9c1ffff2d3
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Tue Apr 26 14:01:47 2016 +0200

    Handle races with lookup more efficiently
    
    The ban lurker checks bans concurrently to lookup. Lookup checks are better
    informed then ban lurker checks, in particular because they have access
    to request information and can thus check request bans.
    
    So, do not touch ocs which were moved concurrent to the ban lurker working them.
    
    Unprotected access to oc->ban may be outdated, so re-check under the ban_mtx.
    
    Idea for great simplification by
    Martin Blix Grydeland <martin at varnish-software.com>

diff --git a/bin/varnishd/cache/cache_ban_lurker.c b/bin/varnishd/cache/cache_ban_lurker.c
index 1c3dfbe..9930b98 100644
--- a/bin/varnishd/cache/cache_ban_lurker.c
+++ b/bin/varnishd/cache/cache_ban_lurker.c
@@ -178,6 +178,14 @@ ban_lurker_test_ban(struct worker *wrk, struct vsl_log *vsl, struct ban *bt,
 			VSC_C_main->bans_lurker_tests_tested += tests;
 			if (i)
 				break;
+			/*
+			 * XXX can we do this? can we safely assert that if
+			 * lookup has raced us it will have moved the oc
+			 * above the olist?
+			 *
+			if (oc->ban != bt)
+				break;
+			*/
 		}
 		if (i) {
 			VSLb(vsl, SLT_ExpBan, "%u banned by lurker",
@@ -186,8 +194,17 @@ ban_lurker_test_ban(struct worker *wrk, struct vsl_log *vsl, struct ban *bt,
 			HSH_Kill(oc);
 			VSC_C_main->bans_lurker_obj_killed++;
 		} else {
-			if (oc->ban != bd) {
+			/*
+			 * we race lookup-time ban checks - oc may have moved up
+			 * the ban list already and we do not want to move it
+			 * down again.
+			 */
+			while (oc->ban == bt) {
 				Lck_Lock(&ban_mtx);
+				if (oc->ban != bt) {
+					Lck_Unlock(&ban_mtx);
+					break;
+				}
 				oc->ban->refcount--;
 				VTAILQ_REMOVE(&oc->ban->objcore, oc, ban_list);
 				oc->ban = bd;
@@ -195,6 +212,7 @@ ban_lurker_test_ban(struct worker *wrk, struct vsl_log *vsl, struct ban *bt,
 				VTAILQ_INSERT_TAIL(&bd->objcore, oc, ban_list);
 				Lck_Unlock(&ban_mtx);
 				ObjSendEvent(wrk, oc, OEV_BANCHG);
+				break;
 			}
 		}
 		(void)HSH_DerefObjCore(wrk, &oc);



From nils.goroll at uplex.de  Tue May  3 14:31:06 2016
From: nils.goroll at uplex.de (Nils Goroll)
Date: Tue, 03 May 2016 16:31:06 +0200
Subject: [master] 26cd05d remove an outdated and misleading comment
Message-ID: <E1axbLu-0003sm-OT@project.varnish-software.com>


commit 26cd05dd1a9df29b6138cfd411a20ff2f9697438
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Tue May 3 16:27:59 2016 +0200

    remove an outdated and misleading comment

diff --git a/bin/varnishd/cache/cache_ban_lurker.c b/bin/varnishd/cache/cache_ban_lurker.c
index 9930b98..b49f3e3 100644
--- a/bin/varnishd/cache/cache_ban_lurker.c
+++ b/bin/varnishd/cache/cache_ban_lurker.c
@@ -114,8 +114,6 @@ ban_lurker_getfirst(struct vsl_log *vsl, struct ban *bt)
 				/*
 				 * We got the lock, and the oc is not being
 				 * dismantled under our feet.
-				 * Take it off the ban and (optimistically)
-				 * put it on the * destination ban
 				 */
 				AZ(oc->flags & OC_F_BUSY);
 				oc->refcnt += 1;



From varnish-commit at varnish-cache.org  Tue May  3 14:40:55 2016
From: varnish-commit at varnish-cache.org (CamScanner)
Date: Tue, 03 May 2016 10:40:55 -0400
Subject: New Doc 122 Page 8
Message-ID: <8E7237EE07C3D7AC038D2C3E18120FC98B35571A6EDA076C9BC366239D6DD@varnish-cache.org>

Scanned by CamScanner


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-commit/attachments/20160503/2787ba20/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: New Doc 189_8.zip
Type: application/zip
Size: 5342 bytes
Desc: not available
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-commit/attachments/20160503/2787ba20/attachment.zip>

From fgsch at lodoss.net  Tue May  3 15:12:06 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Tue, 03 May 2016 17:12:06 +0200
Subject: [master] 30a98d1 Minor fixes
Message-ID: <E1axbza-0006MR-C0@project.varnish-software.com>


commit 30a98d1fd3f5ddeac985ca3e6cf6f5ce2caa1f1a
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Tue May 3 16:01:24 2016 +0100

    Minor fixes

diff --git a/bin/varnishncsa/varnishncsa.c b/bin/varnishncsa/varnishncsa.c
index f2a33a8..1a346f5 100644
--- a/bin/varnishncsa/varnishncsa.c
+++ b/bin/varnishncsa/varnishncsa.c
@@ -646,7 +646,7 @@ parse_x_format(char *buf)
 				VUT_Error(1, "Syntax error: VSL:%s]", buf);
 			if (i <= 0)
 				VUT_Error(1,
-				    "Syntax error. Field specifyer must be"
+				    "Syntax error. Field specifier must be"
 				    " positive: %s]",
 				    buf);
 			if (i > INT_MAX) {
diff --git a/doc/sphinx/reference/varnishncsa.rst b/doc/sphinx/reference/varnishncsa.rst
index 5d01d9a..94730a8 100644
--- a/doc/sphinx/reference/varnishncsa.rst
+++ b/doc/sphinx/reference/varnishncsa.rst
@@ -54,8 +54,7 @@ Client requests that results in a pipe (ie. return(pipe) in vcl), will
 not generate logging in backend mode. This is because varnish is not
 generating requests, but blindly passes on bytes in both directions.
 However, a varnishncsa instance running in normal mode can see this
-case by using the formatter %{Varnish:handling}x, which will be "pipe"
-(without the quotes).
+case by using the formatter %{Varnish:handling}x, which will be 'pipe'.
 
 In backend mode, some of the fields in the format string get different
 meanings.  Most notably, the byte counting formatters (%b, %I, %O)
@@ -67,10 +66,8 @@ backend mode, and one in client mode, logging to different files.
 FORMAT
 ======
 
-Specify the log format used. If no format is specified the default log
-format is used.
-
-The default log format is::
+Specify the log format to use. If no format is specified the default log
+format is used::
 
   %h %l %u %t "%r" %s %b "%{Referer}i" "%{User-agent}i"
 
@@ -104,10 +101,10 @@ Supported formatters are:
   The contents of request header X.
 
 %l
-   Remote logname (always '-')
+  Remote logname. Always '-'.
 
 %m
-   Request method. Defaults to '-' if not known.
+  Request method. Defaults to '-' if not known.
 
 %{X}o
   The contents of response header X.
@@ -117,7 +114,7 @@ Supported formatters are:
   bytes received from the backend.
 
 %q
-  The query string, if no query string exists, an empty string.
+  The query string. Defaults to an empty string if not present.
 
 %r
   The first line of the request. Synthesized from other fields, so it
@@ -132,7 +129,6 @@ Supported formatters are:
   date/time format.  In backend mode, time when the request was sent.
 
 %{X}t
-
   In client mode, time when the request was received, in the format
   specified by X.  In backend mode, time when the request was sent.
   The time specification format is the same as for strftime(3).
@@ -143,11 +139,11 @@ Supported formatters are:
   been received.
 
 %U
-  The request URL without any query string. Defaults to '-' if not
+  The request URL without the query string. Defaults to '-' if not
   known.
 
 %u
-  Remote user from auth
+  Remote user from auth.
 
 %{X}x
   Extended variables.  Supported variables are:
@@ -166,9 +162,9 @@ Supported formatters are:
     pass, pipe or synth.
 
   Varnish:side
-    Backend or client side. One of two values, "b" or "c" (without the
-    quotes), depending on where the request was made. In pure backend
-    or client mode, this field will be constant.
+    Backend or client side. One of two values, 'b' or 'c', depending
+    on where the request was made. In pure backend or client mode,
+    this field will be constant.
 
   Varnish:vxid
     The VXID of the varnish transaction.
@@ -177,13 +173,10 @@ Supported formatters are:
     Output value set by std.log("key:value") in VCL.
 
   VSL:tag or VSL:tag[field]
-    The value of the varnishlog entry with the given tag.  If field is
-    specified, only the selected part is shown.  Defaults to "-"
-    (without the quotes) when the key is not seen, or when the field
-    is out of bounds.  If a key appears several times in a single
-    transaction, only the first occurrence is used.  Example: The
-    formatter %{VSL:Begin[2]}x will print the second field of the
-    Begin tag, which is the VXID of the parent transaction.
+    The value of the VSL entry for the given tag.  If field is specified,
+    only the selected part is shown.  Defaults to '-' when the tag is not
+    seen, or when the field is out of bounds.  If a tag appears several
+    times in a single transaction, only the first occurrence is used.
 
 SIGNALS
 =======
@@ -194,12 +187,21 @@ SIGHUP
 SIGUSR1
   Flush any outstanding transactions
 
+EXAMPLE
+=======
+
+Log the second field of the Begin tag, corresponding to the VXID of the
+parent transaction::
+
+  varnishncsa -F "%{VSL:Begin[2]}x"
+
 SEE ALSO
 ========
 
 :ref:`varnishd(1)`
 :ref:`varnishlog(1)`
 :ref:`varnishstat(1)`
+:ref:`vsl(7)`
 
 HISTORY
 =======



From nils.goroll at uplex.de  Tue May  3 15:23:05 2016
From: nils.goroll at uplex.de (Nils Goroll)
Date: Tue, 03 May 2016 17:23:05 +0200
Subject: [master] 279ecbf fix a typo discussed with phk and martin
Message-ID: <E1axcAD-00077q-Ki@project.varnish-software.com>


commit 279ecbf78de98909fb573b64863e7de5c3d3c704
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Tue May 3 17:16:22 2016 +0200

    fix a typo discussed with phk and martin

diff --git a/bin/varnishd/cache/cache_ban.c b/bin/varnishd/cache/cache_ban.c
index 6ade8d4..2d8f2db 100644
--- a/bin/varnishd/cache/cache_ban.c
+++ b/bin/varnishd/cache/cache_ban.c
@@ -332,7 +332,7 @@ ban_info_new(const uint8_t *ban, unsigned len)
 void
 ban_info_drop(const uint8_t *ban, unsigned len)
 {
-	if (STV_BanInfoNew(ban, len))
+	if (STV_BanInfoDrop(ban, len))
 		ban_export();
 }
 



From nils.goroll at uplex.de  Tue May  3 15:23:05 2016
From: nils.goroll at uplex.de (Nils Goroll)
Date: Tue, 03 May 2016 17:23:05 +0200
Subject: [master] c4f4172 Note a todo for martin
Message-ID: <E1axcAD-00077w-M0@project.varnish-software.com>


commit c4f417261aab804a05e2490d1da378e81517f003
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Tue May 3 17:21:14 2016 +0200

    Note a todo for martin

diff --git a/bin/varnishd/cache/cache_ban.c b/bin/varnishd/cache/cache_ban.c
index 2d8f2db..6cd12f0 100644
--- a/bin/varnishd/cache/cache_ban.c
+++ b/bin/varnishd/cache/cache_ban.c
@@ -325,6 +325,8 @@ ban_export(void)
 void
 ban_info_new(const uint8_t *ban, unsigned len)
 {
+	/* XXX martin pls review if ban_mtx needs to be held */
+	Lck_AssertHeld(&ban_mtx);
 	if (STV_BanInfoNew(ban, len))
 		ban_export();
 }
@@ -332,6 +334,8 @@ ban_info_new(const uint8_t *ban, unsigned len)
 void
 ban_info_drop(const uint8_t *ban, unsigned len)
 {
+	/* XXX martin pls review if ban_mtx needs to be held */
+	Lck_AssertHeld(&ban_mtx);
 	if (STV_BanInfoDrop(ban, len))
 		ban_export();
 }



From fgsch at lodoss.net  Tue May  3 16:19:06 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Tue, 03 May 2016 18:19:06 +0200
Subject: [master] 615e63b Fix build with bmake
Message-ID: <E1axd2Q-0002Hq-2p@project.varnish-software.com>


commit 615e63b058eb7758397680c35031aeeadb1f02ef
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Tue May 3 16:43:05 2016 +0100

    Fix build with bmake

diff --git a/doc/graphviz/Makefile.am b/doc/graphviz/Makefile.am
index 61c756b..dfbf7ab 100644
--- a/doc/graphviz/Makefile.am
+++ b/doc/graphviz/Makefile.am
@@ -1,5 +1,7 @@
 # Makefile for graphviz outputs
 
+SUFFIXES: .dot .pdf .svg
+
 # for an out-of-tree build, sphinx needs the output in builddir
 # XXX is there a better way to do this?
 
@@ -54,7 +56,7 @@ else
 	@DOT@ -Tpdf -Gsize=$(SIZE) -Grotate=90 $< >$@
 endif
 
-%.pdf: %.dot
+.dot.pdf:
 if ! HAVE_DOT
 	@echo ==================================================
 	@echo You need graphviz installed to generate pdf output
@@ -64,7 +66,7 @@ else
 	@DOT@ -Tpdf -Gsize=$(SIZE) $< >$@
 endif
 
-%.svg: %.dot
+.dot.svg:
 if ! HAVE_DOT
 	@echo ==================================================
 	@echo You need graphviz installed to generate svg output



From fgsch at lodoss.net  Tue May  3 16:26:05 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Tue, 03 May 2016 18:26:05 +0200
Subject: [master] d20dc79 GC removed switch
Message-ID: <E1axd9B-0002lz-Rx@project.varnish-software.com>


commit d20dc79b34fcb1752dc90b83edce4301fe0cd52b
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Tue May 3 17:24:42 2016 +0100

    GC removed switch

diff --git a/Makefile.am b/Makefile.am
index b947ceb..04e245a 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -14,8 +14,7 @@ EXTRA_DIST = README.Packaging LICENSE autogen.sh varnishapi.pc.in varnish.m4
 DISTCHECK_CONFIGURE_FLAGS = \
     --enable-developer-warnings \
     --enable-debugging-symbols \
-    --enable-dependency-tracking \
-    --enable-tests
+    --enable-dependency-tracking
 
 install-data-local:
 	$(install_sh) -d -m 0755 $(DESTDIR)$(localstatedir)/varnish



From phk at FreeBSD.org  Tue May  3 20:15:06 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 03 May 2016 22:15:06 +0200
Subject: [master] dc39f2c Try to make it more obvious what happens and answer
 a good question with a resounding yes!
Message-ID: <E1axgio-00082i-LS@project.varnish-software.com>


commit dc39f2c25605731b889a9f9e6a7a436432898129
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue May 3 20:12:59 2016 +0000

    Try to make it more obvious what happens and answer a good question
    with a resounding yes!

diff --git a/bin/varnishd/cache/cache_ban_lurker.c b/bin/varnishd/cache/cache_ban_lurker.c
index b49f3e3..f184fbe 100644
--- a/bin/varnishd/cache/cache_ban_lurker.c
+++ b/bin/varnishd/cache/cache_ban_lurker.c
@@ -164,6 +164,13 @@ ban_lurker_test_ban(struct worker *wrk, struct vsl_log *vsl, struct ban *bt,
 			return;
 		i = 0;
 		VTAILQ_FOREACH_REVERSE_SAFE(bl, obans, banhead_s, l_list, bln) {
+			if (oc->ban != bt) {
+				/*
+				 * HSH_Lookup() grabbed this oc, killed
+				 * it or tested it to top.  We're done.
+				 */
+				break;
+			}
 			if (bl->flags & BANS_FLAG_COMPLETED) {
 				/* Ban was overtaken by new (dup) ban */
 				VTAILQ_REMOVE(obans, bl, l_list);
@@ -174,44 +181,27 @@ ban_lurker_test_ban(struct worker *wrk, struct vsl_log *vsl, struct ban *bt,
 			i = ban_evaluate(wrk, bl->spec, oc, NULL, &tests);
 			VSC_C_main->bans_lurker_tested++;
 			VSC_C_main->bans_lurker_tests_tested += tests;
-			if (i)
-				break;
-			/*
-			 * XXX can we do this? can we safely assert that if
-			 * lookup has raced us it will have moved the oc
-			 * above the olist?
-			 *
-			if (oc->ban != bt)
+			if (i) {
+				VSLb(vsl, SLT_ExpBan, "%u banned by lurker",
+				    ObjGetXID(wrk, oc));
+				HSH_Kill(oc);
+				VSC_C_main->bans_lurker_obj_killed++;
 				break;
-			*/
+			}
 		}
-		if (i) {
-			VSLb(vsl, SLT_ExpBan, "%u banned by lurker",
-			    ObjGetXID(wrk, oc));
-
-			HSH_Kill(oc);
-			VSC_C_main->bans_lurker_obj_killed++;
-		} else {
-			/*
-			 * we race lookup-time ban checks - oc may have moved up
-			 * the ban list already and we do not want to move it
-			 * down again.
-			 */
-			while (oc->ban == bt) {
-				Lck_Lock(&ban_mtx);
-				if (oc->ban != bt) {
-					Lck_Unlock(&ban_mtx);
-					break;
-				}
-				oc->ban->refcount--;
-				VTAILQ_REMOVE(&oc->ban->objcore, oc, ban_list);
+		if (i == 0 && oc->ban == bt) {
+			Lck_Lock(&ban_mtx);
+			if (oc->ban == bt) {
+				bt->refcount--;
+				VTAILQ_REMOVE(&bt->objcore, oc, ban_list);
 				oc->ban = bd;
 				bd->refcount++;
 				VTAILQ_INSERT_TAIL(&bd->objcore, oc, ban_list);
-				Lck_Unlock(&ban_mtx);
-				ObjSendEvent(wrk, oc, OEV_BANCHG);
-				break;
+				i = 1;
 			}
+			Lck_Unlock(&ban_mtx);
+			if (i)
+				ObjSendEvent(wrk, oc, OEV_BANCHG);
 		}
 		(void)HSH_DerefObjCore(wrk, &oc);
 	}



From fgsch at lodoss.net  Wed May  4 06:34:05 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Wed, 04 May 2016 08:34:05 +0200
Subject: [master] 256c566 Fix distcheck on FreeBSD and possible other BSDs
Message-ID: <E1axqNp-0000tC-T5@project.varnish-software.com>


commit 256c566a68c0a4e4db4d5a6159839948426800b0
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Wed May 4 06:59:56 2016 +0100

    Fix distcheck on FreeBSD and possible other BSDs

diff --git a/doc/sphinx/Makefile.am b/doc/sphinx/Makefile.am
index 13e2bc6..e1a5588 100644
--- a/doc/sphinx/Makefile.am
+++ b/doc/sphinx/Makefile.am
@@ -175,3 +175,5 @@ BUILT_SOURCES += reference/vmod_directors.generated.rst
 
 EXTRA_DIST += $(BUILT_SOURCES)
 MAINTAINERCLEANFILES = $(EXTRA_DIST)
+
+.NOPATH: $(BUILT_SOURCES)
diff --git a/lib/libvarnishapi/Makefile.am b/lib/libvarnishapi/Makefile.am
index b5eb2b0..1bd045b 100644
--- a/lib/libvarnishapi/Makefile.am
+++ b/lib/libvarnishapi/Makefile.am
@@ -105,3 +105,5 @@ vsl_glob_test_SOURCES = \
 vsl_glob_test_LDADD = @PCRE_LIBS@ ${RT_LIBS} ${LIBM} libvarnishapi.la
 
 vsl_glob_test_CFLAGS = -I$(top_srcdir)/include
+
+.NOPATH: $(BUILT_SOURCES)
diff --git a/man/Makefile.am b/man/Makefile.am
index 0c93e1b..536d7a2 100644
--- a/man/Makefile.am
+++ b/man/Makefile.am
@@ -82,3 +82,5 @@ vmod_std.3: $(top_builddir)/lib/libvmod_std/vmod_std.man.rst
 
 vmod_directors.3: $(top_builddir)/lib/libvmod_directors/vmod_directors.man.rst
 	${RST2MAN} $(RST2ANY_FLAGS) $? $@
+
+.NOPATH: $(dist_man_MANS)



From phk at FreeBSD.org  Wed May  4 08:07:05 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Wed, 04 May 2016 10:07:05 +0200
Subject: [master] 52c2334 Collapse two loops into one.
Message-ID: <E1axrpp-0006PH-0D@project.varnish-software.com>


commit 52c23347ae2d595b4591e7bff002647b7cf947c5
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue May 3 20:42:28 2016 +0000

    Collapse two loops into one.

diff --git a/bin/varnishd/storage/storage_simple.c b/bin/varnishd/storage/storage_simple.c
index 075d866..f6b70e1 100644
--- a/bin/varnishd/storage/storage_simple.c
+++ b/bin/varnishd/storage/storage_simple.c
@@ -252,24 +252,19 @@ sml_iterator(struct worker *wrk, struct objcore *oc,
 
 	boc = HSH_RefBoc(oc);
 
-	if (boc == NULL && final) {
-		while (!VTAILQ_EMPTY(&obj->list)) {
-			st = VTAILQ_FIRST(&obj->list);
+	if (boc == NULL) {
+		VTAILQ_FOREACH_SAFE(st, &obj->list, list, checkpoint) {
+			AN(st->len);
 			if (ret == 0 && func(priv, 1, st->ptr, st->len))
 				ret = -1;
-			VTAILQ_REMOVE(&obj->list, st, list);
-			sml_stv_free(stv, st);
+			if (final) {
+				VTAILQ_REMOVE(&obj->list, st, list);
+				sml_stv_free(stv, st);
+			} else if (ret)
+				break;
 		}
 		return (ret);
 	}
-	if (boc == NULL) {
-		VTAILQ_FOREACH(st, &obj->list, list) {
-			AN(st->len);
-			if (func(priv, 0, st->ptr, st->len))
-				return (-1);
-		}
-		return (0);
-	}
 
 	p = NULL;
 	l = 0;



From phk at FreeBSD.org  Wed May  4 08:07:05 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Wed, 04 May 2016 10:07:05 +0200
Subject: [master] 71be412 Add a bunch of protective asserts on vcl->conf,
 to help flush out a family of tickets related to it (#1933 #1920
 #1898 (?) #1888 and #1865 (?))
Message-ID: <E1axrpp-0006PN-1d@project.varnish-software.com>


commit 71be412901f68669ad5099d3a596548938551555
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed May 4 08:05:18 2016 +0000

    Add a bunch of protective asserts on vcl->conf, to help flush out
    a family of tickets related to it (#1933 #1920 #1898 (?) #1888 and #1865 (?))

diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index 1afabb9..6da6ebd 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -99,12 +99,16 @@ VCL_Panic(struct vsb *vsb, const struct vcl *vcl)
 	VSB_printf(vsb, "vcl = {\n");
 	VSB_indent(vsb, 2);
 	VSB_printf(vsb, "temp = %s\n", vcl->temp);
-	VSB_printf(vsb, "srcname = {\n");
-	VSB_indent(vsb, 2);
-	for (i = 0; i < vcl->conf->nsrc; ++i)
-		VSB_printf(vsb, "\"%s\",\n", vcl->conf->srcname[i]);
-	VSB_indent(vsb, -2);
-	VSB_printf(vsb, "},\n");
+	if (vcl->conf == NULL) {
+		VSB_printf(vsb, "conf = NULL\n");
+	} else {
+		VSB_printf(vsb, "srcname = {\n");
+		VSB_indent(vsb, 2);
+		for (i = 0; i < vcl->conf->nsrc; ++i)
+			VSB_printf(vsb, "\"%s\",\n", vcl->conf->srcname[i]);
+		VSB_indent(vsb, -2);
+		VSB_printf(vsb, "},\n");
+	}
 	VSB_indent(vsb, -2);
 	VSB_printf(vsb, "},\n");
 }
@@ -350,6 +354,7 @@ VCL_DefaultDirector(const struct vcl *vcl)
 {
 
 	CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC);
+	CHECK_OBJ_NOTNULL(vcl->conf, VCL_CONF_MAGIC);
 	return (*vcl->conf->default_director);
 }
 
@@ -366,6 +371,7 @@ VCL_DefaultProbe(const struct vcl *vcl)
 {
 
 	CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC);
+	CHECK_OBJ_NOTNULL(vcl->conf, VCL_CONF_MAGIC);
 	return (vcl->conf->default_probe);
 }
 
@@ -377,6 +383,7 @@ VRT_count(VRT_CTX, unsigned u)
 
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
 	CHECK_OBJ_NOTNULL(ctx->vcl, VCL_MAGIC);
+	CHECK_OBJ_NOTNULL(ctx->vcl->conf, VCL_CONF_MAGIC);
 	assert(u < ctx->vcl->conf->nref);
 	if (ctx->vsl != NULL)
 		VSLb(ctx->vsl, SLT_VCL_trace, "%u %u.%u", u,
@@ -460,6 +467,8 @@ vcl_setup_event(VRT_CTX, enum vcl_event_e ev)
 {
 
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
+	CHECK_OBJ_NOTNULL(ctx->vcl, VCL_MAGIC);
+	CHECK_OBJ_NOTNULL(ctx->vcl->conf, VCL_CONF_MAGIC);
 	AN(ctx->handling);
 	AN(ctx->vcl);
 	AN(ctx->msg);
@@ -473,6 +482,8 @@ vcl_failsafe_event(VRT_CTX, enum vcl_event_e ev)
 {
 
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
+	CHECK_OBJ_NOTNULL(ctx->vcl, VCL_MAGIC);
+	CHECK_OBJ_NOTNULL(ctx->vcl->conf, VCL_CONF_MAGIC);
 	AN(ctx->handling);
 	AN(ctx->vcl);
 	assert(ev == VCL_EVENT_COLD || ev == VCL_EVENT_DISCARD);
@@ -506,6 +517,8 @@ vcl_set_state(VRT_CTX, const char *state)
 
 	ASSERT_CLI();
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
+	CHECK_OBJ_NOTNULL(ctx->vcl, VCL_MAGIC);
+	CHECK_OBJ_NOTNULL(ctx->vcl->conf, VCL_CONF_MAGIC);
 	AN(ctx->handling);
 	AN(ctx->vcl);
 	AN(state);
@@ -562,6 +575,9 @@ vcl_cancel_load(VRT_CTX, struct cli *cli, const char *name, const char *step)
 {
 	struct vcl *vcl = ctx->vcl;
 
+	CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC);
+	CHECK_OBJ_NOTNULL(vcl->conf, VCL_CONF_MAGIC);
+
 	AZ(VSB_finish(ctx->msg));
 	VCLI_SetResult(cli, CLIS_CANT);
 	VCLI_Out(cli, "VCL \"%s\" Failed %s", name, step);
@@ -824,6 +840,8 @@ ccf_config_show(struct cli *cli, const char * const *av, void *priv)
 		VCLI_SetResult(cli, CLIS_PARAM);
 		return;
 	}
+	CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC);
+	CHECK_OBJ_NOTNULL(vcl->conf, VCL_CONF_MAGIC);
 	if (verbose) {
 		for (i = 0; i < vcl->conf->nsrc; i++)
 			VCLI_Out(cli, "// VCL.SHOW %d %zd %s\n%s\n",



From phk at FreeBSD.org  Wed May  4 13:04:06 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Wed, 04 May 2016 15:04:06 +0200
Subject: [master] 0df5104 whitespace & test-title
Message-ID: <E1axwTG-0006D8-52@project.varnish-software.com>


commit 0df51043b36e17a1bf67595fc53fac95384fcb67
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed May 4 13:02:45 2016 +0000

    whitespace & test-title

diff --git a/bin/varnishtest/tests/r01890.vtc b/bin/varnishtest/tests/r01890.vtc
index 6403c13..4ab6905 100644
--- a/bin/varnishtest/tests/r01890.vtc
+++ b/bin/varnishtest/tests/r01890.vtc
@@ -1,16 +1,18 @@
+varnishtest "Test return(synth) from vcl_pipe"
+
 server s1 {
-        rxreq
-        txresp
+	rxreq
+	txresp
 } -start
 
 varnish v1 -vcl+backend {
-        sub vcl_pipe {
-                return (synth(401));
-        }
+	sub vcl_pipe {
+		return (synth(401));
+	}
 } -start
 
 client c1 {
-        txreq -req PROPFIND
-        rxresp
-        expect resp.status == 401
+	txreq -req PROPFIND
+	rxresp
+	expect resp.status == 401
 } -run



From phk at FreeBSD.org  Wed May  4 13:04:06 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Wed, 04 May 2016 15:04:06 +0200
Subject: [master] cfdced3 Dump more VCL info in panic,
 and add a general macro for checking miniobj magics.
Message-ID: <E1axwTG-0006DE-6i@project.varnish-software.com>


commit cfdced3e23e513c5d7db02f64001a2a990e28ce0
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed May 4 13:02:58 2016 +0000

    Dump more VCL info in panic, and add a general macro for checking
    miniobj magics.

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index 2b14c9b..a0d47ab 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -1141,6 +1141,14 @@ DO_DEBUG(enum debug_bits x)
 			VSL(SLT_Debug, (id), __VA_ARGS__);	\
 	} while (0)
 
+#define PAN_CheckMagic(vsb, ptr, exp)					\
+	do {								\
+		if ((ptr)->magic != (exp))				\
+			VSB_printf((vsb),				\
+			    "MAGIC 0x%08x (Should:%s/0x%08x)\n",	\
+			    (ptr)->magic, #exp, exp);			\
+	} while(0)
+
 #ifdef VARNISHD_IS_NOT_A_VMOD
 #  include "cache/cache_priv.h"
 #endif
diff --git a/bin/varnishd/cache/cache_panic.c b/bin/varnishd/cache/cache_panic.c
index e0c5268..dc02731 100644
--- a/bin/varnishd/cache/cache_panic.c
+++ b/bin/varnishd/cache/cache_panic.c
@@ -142,6 +142,7 @@ pan_ws(struct vsb *vsb, const struct ws *ws)
 	if (pan_already(vsb, ws))
 		return;
 	VSB_indent(vsb, 2);
+	PAN_CheckMagic(vsb, ws, WS_MAGIC);
 	if (!(ws->id[0] & 0x20))
 		VSB_printf(vsb, "OVERFLOWED ");
 	VSB_printf(vsb, "id = \"%s\",\n",  ws->id);
@@ -173,6 +174,7 @@ pan_htc(struct vsb *vsb, const struct http_conn *htc)
 	if (pan_already(vsb, htc))
 		return;
 	VSB_indent(vsb, 2);
+	PAN_CheckMagic(vsb, htc, HTTP_CONN_MAGIC);
 	VSB_printf(vsb, "fd = %d,\n", htc->fd);
 	VSB_printf(vsb, "doclose = %s,\n", sess_close_2str(htc->doclose, 0));
 	VSB_printf(vsb, "ws = %p,\n", htc->ws);
@@ -203,6 +205,7 @@ pan_http(struct vsb *vsb, const char *id, const struct http *h)
 	if (pan_already(vsb, h))
 		return;
 	VSB_indent(vsb, 2);
+	PAN_CheckMagic(vsb, h, HTTP_MAGIC);
 	VSB_printf(vsb, "ws[%s] = %p,\n", h->ws ? h->ws->id : "", h->ws);
 	VSB_printf(vsb, "hdrs {\n");
 	VSB_indent(vsb, 2);
@@ -226,6 +229,7 @@ pan_boc(struct vsb *vsb, const struct boc *boc)
 	if (pan_already(vsb, boc))
 		return;
 	VSB_indent(vsb, 2);
+	PAN_CheckMagic(vsb, boc, BOC_MAGIC);
 	VSB_printf(vsb, "refcnt = %u,\n", boc->refcount);
 	VSB_printf(vsb, "state = %d,\n", boc->state);
 	VSB_printf(vsb, "vary = %p,\n", boc->vary);
@@ -244,6 +248,7 @@ pan_objcore(struct vsb *vsb, const char *typ, const struct objcore *oc)
 	if (pan_already(vsb, oc))
 		return;
 	VSB_indent(vsb, 2);
+	PAN_CheckMagic(vsb, oc, OBJCORE_MAGIC);
 	VSB_printf(vsb, "refcnt = %d,\n", oc->refcnt);
 	VSB_printf(vsb, "flags = 0x%x,\n", oc->flags);
 	VSB_printf(vsb, "exp_flags = 0x%x,\n", oc->exp_flags);
@@ -284,6 +289,7 @@ pan_wrk(struct vsb *vsb, const struct worker *wrk)
 	if (pan_already(vsb, wrk))
 		return;
 	VSB_indent(vsb, 2);
+	PAN_CheckMagic(vsb, wrk, WORKER_MAGIC);
 	VSB_printf(vsb, "stack = {0x%jx -> 0x%jx},\n",
 	    (uintmax_t)wrk->stack_start, (uintmax_t)wrk->stack_end);
 	pan_ws(vsb, wrk->aws);
@@ -333,6 +339,7 @@ pan_busyobj(struct vsb *vsb, const struct busyobj *bo)
 	if (pan_already(vsb, bo))
 		return;
 	VSB_indent(vsb, 2);
+	PAN_CheckMagic(vsb, bo, BUSYOBJ_MAGIC);
 	pan_ws(vsb, bo->ws);
 	VSB_printf(vsb, "retries = %d, ", bo->retries);
 	VSB_printf(vsb, "failed = %d, ", bo->vfc->failed);
@@ -387,7 +394,7 @@ pan_req(struct vsb *vsb, const struct req *req)
 	if (pan_already(vsb, req))
 		return;
 	VSB_indent(vsb, 2);
-
+	PAN_CheckMagic(vsb, req, REQ_MAGIC);
 	xp = req->transport;
 	VSB_printf(vsb, "vxid = %u, transport = %s", VXID(req->vsl->wid),
 	    xp == NULL ? "NULL" : xp->name);
@@ -467,6 +474,7 @@ pan_sess(struct vsb *vsb, const struct sess *sp)
 	if (pan_already(vsb, sp))
 		return;
 	VSB_indent(vsb, 2);
+	PAN_CheckMagic(vsb, sp, SESS_MAGIC);
 	xp = XPORT_ByNumber(sp->sattr[SA_TRANSPORT]);
 	VSB_printf(vsb, "fd = %d, vxid = %u,\n",
 	    sp->fd, VXID(sp->vxid));
diff --git a/bin/varnishd/cache/cache_priv.h b/bin/varnishd/cache/cache_priv.h
index 47330ee..415822c 100644
--- a/bin/varnishd/cache/cache_priv.h
+++ b/bin/varnishd/cache/cache_priv.h
@@ -57,7 +57,6 @@ void BAN_NewObjCore(struct objcore *oc);
 void BAN_DestroyObj(struct objcore *oc);
 int BAN_CheckObject(struct worker *, struct objcore *, struct req *);
 
-
 /* cache_busyobj.c */
 void VBO_Init(void);
 
diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index 6da6ebd..0dcaa73 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -98,10 +98,17 @@ VCL_Panic(struct vsb *vsb, const struct vcl *vcl)
 		return;
 	VSB_printf(vsb, "vcl = {\n");
 	VSB_indent(vsb, 2);
-	VSB_printf(vsb, "temp = %s\n", vcl->temp);
+	PAN_CheckMagic(vsb, vcl, VCL_MAGIC);
+	VSB_printf(vsb, "busy = %u\n", vcl->busy);
+	VSB_printf(vsb, "discard = %u,\n", vcl->discard);
+	VSB_printf(vsb, "state = %s,\n", vcl->state);
+	VSB_printf(vsb, "temp = %s,\n", vcl->temp);
+	VSB_printf(vsb, "conf = {\n");
+	VSB_indent(vsb, 2);
 	if (vcl->conf == NULL) {
 		VSB_printf(vsb, "conf = NULL\n");
 	} else {
+		PAN_CheckMagic(vsb, vcl->conf, VCL_CONF_MAGIC);
 		VSB_printf(vsb, "srcname = {\n");
 		VSB_indent(vsb, 2);
 		for (i = 0; i < vcl->conf->nsrc; ++i)
@@ -111,6 +118,8 @@ VCL_Panic(struct vsb *vsb, const struct vcl *vcl)
 	}
 	VSB_indent(vsb, -2);
 	VSB_printf(vsb, "},\n");
+	VSB_indent(vsb, -2);
+	VSB_printf(vsb, "},\n");
 }
 
 /*--------------------------------------------------------------------*/



From phk at FreeBSD.org  Wed May  4 13:17:06 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Wed, 04 May 2016 15:17:06 +0200
Subject: [master] 376bc27 Insist that tests start out with the magic string
 "varnishtest[:whitespace:]"
Message-ID: <E1axwfq-000707-7G@project.varnish-software.com>


commit 376bc2743d373233b2e703352acb3b64e95c71ef
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed May 4 13:15:34 2016 +0000

    Insist that tests start out with the magic string "varnishtest[:whitespace:]"

diff --git a/bin/varnishtest/vtc_main.c b/bin/varnishtest/vtc_main.c
index 1c20019..2b32551 100644
--- a/bin/varnishtest/vtc_main.c
+++ b/bin/varnishtest/vtc_main.c
@@ -34,6 +34,7 @@
 #include <sys/types.h>
 #include <sys/wait.h>
 
+#include <ctype.h>
 #include <errno.h>
 #include <fcntl.h>
 #include <poll.h>
@@ -568,6 +569,13 @@ main(int argc, char * const *argv)
 				continue;
 			exit(2);
 		}
+		if (strncmp(p, "varnishtest", 11) || !isspace(p[11])) {
+			fprintf(stderr, "File \"%s\" doesn't start with 'varnishtest'\n",
+			    *argv);
+			if (vtc_continue)
+				continue;
+			exit(2);
+		}
 		ALLOC_OBJ(tp, TST_MAGIC);
 		AN(tp);
 		tp->filename = *argv;



From varnish-commit at varnish-cache.org  Wed May  4 10:55:17 2016
From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org)
Date: 4 May 2016 04:55:17 -0600
Subject: Make your buddies envy
Message-ID: <002a01d1a5f5$01a59cd9$13c788af@omeqpb>

If you are unable to see the message below, click here to view.

 
  PLEASE DO NOT REPLY - This is being sent from an unattended mailbox. 
 Copyright ? 2016 Dr. MaxMan, Inc. All rights reserved.
 5 Trowbridge Drive, Bethel, CT 708842
You have received this message because you
opted in to receives MaxMan pecial offers via email.
You can unsubscribe here
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-commit/attachments/20160504/f9449e40/attachment.html>

From fgsch at lodoss.net  Wed May  4 16:57:05 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Wed, 04 May 2016 18:57:05 +0200
Subject: [master] dce6ecc Move all but vmods' rst generation to doc/sphinx
Message-ID: <E1ay06j-0003Ya-U6@project.varnish-software.com>


commit dce6eccdf3152288f8726193706e0440e15e70e9
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Wed May 4 10:59:28 2016 +0100

    Move all but vmods' rst generation to doc/sphinx
    
    Use builddir and other minor cleanups while here.

diff --git a/.gitignore b/.gitignore
index fb9cde6..fe1e27e 100644
--- a/.gitignore
+++ b/.gitignore
@@ -69,7 +69,6 @@ cscope.*out
 /lib/libvarnishapi/vxp_fixed_token.c
 /lib/libvarnishapi/vxp_tokens.h
 /lib/libvarnishapi/vsl2rst
-/lib/libvarnishapi/vsl-tags.rst
 
 
 # Misc. generated files for included vmods.
diff --git a/doc/sphinx/Makefile.am b/doc/sphinx/Makefile.am
index e1a5588..14127df 100644
--- a/doc/sphinx/Makefile.am
+++ b/doc/sphinx/Makefile.am
@@ -123,7 +123,7 @@ distclean-local:
 	rm -rf $(BUILDDIR)
 
 include/params.rst: $(top_builddir)/bin/varnishd/varnishd
-	$(top_builddir)/bin/varnishd/varnishd -x dumprstparam > include/params.rst
+	$(top_builddir)/bin/varnishd/varnishd -x dumprstparam > $@
 BUILT_SOURCES = include/params.rst
 
 include/counters.rst: $(top_builddir)/bin/varnishstat/vsc2rst
@@ -160,6 +160,10 @@ include/varnishhist_synopsis.rst: $(top_builddir)/bin/varnishhist/varnishhist_op
 BUILT_SOURCES += include/varnishhist_options.rst \
 	 include/varnishhist_synopsis.rst
 
+include/vsl-tags.rst: $(top_builddir)/lib/libvarnishapi/vsl2rst
+	$(top_builddir)/lib/libvarnishapi/vsl2rst > $@
+BUILT_SOURCES += include/vsl-tags.rst
+
 .PHONY: reference
 reference:
 	test -d $@ || mkdir $@
diff --git a/doc/sphinx/reference/vsl.rst b/doc/sphinx/reference/vsl.rst
index 1aafd28..3f39c67 100644
--- a/doc/sphinx/reference/vsl.rst
+++ b/doc/sphinx/reference/vsl.rst
@@ -22,7 +22,7 @@ logging tools supplied with Varnish.
 VSL tags
 ~~~~~~~~
 
-.. include:: ../../../lib/libvarnishapi/vsl-tags.rst
+.. include:: ../include/vsl-tags.rst
 
 TIMESTAMPS
 ==========
diff --git a/lib/libvarnishapi/Makefile.am b/lib/libvarnishapi/Makefile.am
index 1bd045b..32986c4 100644
--- a/lib/libvarnishapi/Makefile.am
+++ b/lib/libvarnishapi/Makefile.am
@@ -16,7 +16,6 @@ libvarnishapi_la_SOURCES = \
 	vsl_api.h \
 	vxp.h \
 	vxp_tokens.h \
-	\
 	../libvarnish/vas.c \
 	../libvarnish/vav.c \
 	../../include/vcs_version.h \
@@ -57,21 +56,16 @@ libvarnishapi_la_LDFLAGS += -export-symbols-regex '^V'
 endif
 
 EXTRA_DIST = \
-	generate.py \
-	vsl-tags.rst
+	generate.py
 
 BUILT_SOURCES = \
 	vxp_fixed_token.c \
-	vxp_tokens.h \
-	vsl-tags.rst
+	vxp_tokens.h
 
 CLEANFILES = \
 	$(builddir)/vxp_fixed_token.c \
 	$(builddir)/vxp_tokens.h
 
-MAINTAINERCLEANFILES = \
-	vsl-tags.rst
-
 noinst_PROGRAMS = vsl2rst
 
 vsl2rst_SOURCES = \
@@ -79,9 +73,6 @@ vsl2rst_SOURCES = \
 	$(top_srcdir)/include/tbl/vsl_tags.h \
 	$(top_srcdir)/include/tbl/vsl_tags_http.h
 
-vsl-tags.rst: vsl2rst
-	./vsl2rst > $@
-
 vxp_fixed_token.c vxp_tokens.h: \
 	$(srcdir)/generate.py
 	@PYTHON@ $(srcdir)/generate.py $(srcdir) $(top_builddir)
@@ -105,5 +96,3 @@ vsl_glob_test_SOURCES = \
 vsl_glob_test_LDADD = @PCRE_LIBS@ ${RT_LIBS} ${LIBM} libvarnishapi.la
 
 vsl_glob_test_CFLAGS = -I$(top_srcdir)/include
-
-.NOPATH: $(BUILT_SOURCES)
diff --git a/man/Makefile.am b/man/Makefile.am
index 536d7a2..05a1618 100644
--- a/man/Makefile.am
+++ b/man/Makefile.am
@@ -32,7 +32,7 @@ vcl.7: $(top_srcdir)/doc/sphinx/reference/vcl.rst \
 	${RST2MAN} $(RST2ANY_FLAGS) $(top_srcdir)/doc/sphinx/reference/vcl.rst $@
 
 vsl.7: $(top_srcdir)/doc/sphinx/reference/vsl.rst \
-	$(top_srcdir)/lib/libvarnishapi/vsl-tags.rst
+	$(top_builddir)/doc/sphinx/include/vsl-tags.rst
 	${RST2MAN} $(RST2ANY_FLAGS) $(top_srcdir)/doc/sphinx/reference/vsl.rst $@
 
 vsl-query.7: $(top_srcdir)/doc/sphinx/reference/vsl-query.rst
@@ -43,19 +43,19 @@ varnishadm.1: $(top_srcdir)/doc/sphinx/reference/varnishadm.rst
 
 varnishd.1: \
 	$(top_srcdir)/doc/sphinx/reference/varnishd.rst \
-	$(top_srcdir)/doc/sphinx/include/params.rst
+	$(top_builddir)/doc/sphinx/include/params.rst
 	${RST2MAN} $(RST2ANY_FLAGS) $(top_srcdir)/doc/sphinx/reference/varnishd.rst $@
 
 varnishncsa.1: \
 	$(top_srcdir)/doc/sphinx/reference/varnishncsa.rst \
-	$(top_srcdir)/doc/sphinx/include/varnishncsa_options.rst \
-	$(top_srcdir)/doc/sphinx/include/varnishncsa_synopsis.rst
+	$(top_builddir)/doc/sphinx/include/varnishncsa_options.rst \
+	$(top_builddir)/doc/sphinx/include/varnishncsa_synopsis.rst
 	${RST2MAN} $(RST2ANY_FLAGS) $(top_srcdir)/doc/sphinx/reference/varnishncsa.rst $@
 
 varnishlog.1: \
 	$(top_srcdir)/doc/sphinx/reference/varnishlog.rst \
-	$(top_srcdir)/doc/sphinx/include/varnishlog_options.rst \
-	$(top_srcdir)/doc/sphinx/include/varnishlog_synopsis.rst
+	$(top_builddir)/doc/sphinx/include/varnishlog_options.rst \
+	$(top_builddir)/doc/sphinx/include/varnishlog_synopsis.rst
 	${RST2MAN} $(RST2ANY_FLAGS) $(top_srcdir)/doc/sphinx/reference/varnishlog.rst $@
 
 # XXX add _options.rst and _synopsis.rst here when it's been _opt2rst'ed
@@ -67,14 +67,14 @@ varnishtest.1: $(top_srcdir)/doc/sphinx/reference/varnishtest.rst
 
 varnishtop.1: \
 	$(top_srcdir)/doc/sphinx/reference/varnishtop.rst \
-	$(top_srcdir)/doc/sphinx/include/varnishtop_options.rst \
-	$(top_srcdir)/doc/sphinx/include/varnishtop_synopsis.rst
+	$(top_builddir)/doc/sphinx/include/varnishtop_options.rst \
+	$(top_builddir)/doc/sphinx/include/varnishtop_synopsis.rst
 	${RST2MAN} $(RST2ANY_FLAGS) $(top_srcdir)/doc/sphinx/reference/varnishtop.rst $@
 
 varnishhist.1: \
 	$(top_srcdir)/doc/sphinx/reference/varnishhist.rst \
-	$(top_srcdir)/doc/sphinx/include/varnishhist_options.rst \
-	$(top_srcdir)/doc/sphinx/include/varnishhist_synopsis.rst
+	$(top_builddir)/doc/sphinx/include/varnishhist_options.rst \
+	$(top_builddir)/doc/sphinx/include/varnishhist_synopsis.rst
 	${RST2MAN} $(RST2ANY_FLAGS) $(top_srcdir)/doc/sphinx/reference/varnishhist.rst $@
 
 vmod_std.3: $(top_builddir)/lib/libvmod_std/vmod_std.man.rst



From fgsch at lodoss.net  Thu May  5 14:58:06 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Thu, 05 May 2016 16:58:06 +0200
Subject: [master] 9552aa1 Retire fetch return option from vcl_hit{}
Message-ID: <E1ayKj8-0002BH-Lk@project.varnish-software.com>


commit 9552aa16b46b38b0b3eb7ac42f3bc6295f50eecc
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Thu May 5 15:32:08 2016 +0100

    Retire fetch return option from vcl_hit{}
    
    This has been superseded by return (miss).

diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index 889cf40..72f24a7 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -439,10 +439,6 @@ cnt_lookup(struct worker *wrk, struct req *req)
 		req->is_hit = 1;
 		req->req_step = R_STP_DELIVER;
 		return (REQ_FSM_MORE);
-	case VCL_RET_FETCH:
-		VSLb(req->vsl, SLT_VCL_Error,
-		    "change return(fetch) to return(miss) in vcl_hit{}");
-		/* FALL-THROUGH */
 	case VCL_RET_MISS:
 		if (busy != NULL) {
 			req->objcore = busy;
diff --git a/bin/varnishtest/tests/r01335.vtc b/bin/varnishtest/tests/r01335.vtc
index 85ff372..1488c98 100644
--- a/bin/varnishtest/tests/r01335.vtc
+++ b/bin/varnishtest/tests/r01335.vtc
@@ -10,7 +10,7 @@ server s1 {
 varnish v1 -vcl+backend {
 	sub vcl_hit {
 		if (req.http.two == "2") {
-			return (fetch);		// also #1603
+			return (miss);		// also #1603
 		}
 	}
 } -start
diff --git a/lib/libvcc/generate.py b/lib/libvcc/generate.py
index e70068c..37aae67 100755
--- a/lib/libvcc/generate.py
+++ b/lib/libvcc/generate.py
@@ -109,7 +109,7 @@ returns =(
 	),
 	('hit',
 		"C",
-		('synth', 'restart', 'pass', 'fetch', 'miss', 'deliver',)
+		('synth', 'restart', 'pass', 'miss', 'deliver',)
 	),
 	('deliver',
 		"C",



From fgsch at lodoss.net  Thu May  5 15:24:04 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Thu, 05 May 2016 17:24:04 +0200
Subject: [master] 3e3357f Mostly whitespace OCD
Message-ID: <E1ayL8G-0003h7-Pp@project.varnish-software.com>


commit 3e3357f937c9bead546ffef4bc49f0978776d41b
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Thu May 5 16:19:36 2016 +0100

    Mostly whitespace OCD
    
    Reduces my stash list to 62 entries.

diff --git a/bin/varnishd/cache/cache_ws.c b/bin/varnishd/cache/cache_ws.c
index 5f60d73..8666b35 100644
--- a/bin/varnishd/cache/cache_ws.c
+++ b/bin/varnishd/cache/cache_ws.c
@@ -134,7 +134,7 @@ WS_Alloc(struct ws *ws, unsigned bytes)
 	assert(ws->r == NULL);
 	if (ws->f + bytes > ws->e) {
 		WS_MarkOverflow(ws);
-		return(NULL);
+		return (NULL);
 	}
 	r = ws->f;
 	ws->f += bytes;
@@ -159,7 +159,7 @@ WS_Copy(struct ws *ws, const void *str, int len)
 	bytes = PRNDUP((unsigned)len);
 	if (ws->f + bytes > ws->e) {
 		WS_MarkOverflow(ws);
-		return(NULL);
+		return (NULL);
 	}
 	r = ws->f;
 	ws->f += bytes;
diff --git a/bin/varnishncsa/varnishncsa.c b/bin/varnishncsa/varnishncsa.c
index 1a346f5..5827458 100644
--- a/bin/varnishncsa/varnishncsa.c
+++ b/bin/varnishncsa/varnishncsa.c
@@ -629,11 +629,10 @@ parse_x_format(char *buf)
 	if (!strncmp(buf, "VSL:", 4)) {
 		buf += 4;
 		e = buf;
-		while(*e != '\0')
+		while (*e != '\0')
 			e++;
-		if (e == buf) {
+		if (e == buf)
 			VUT_Error(1, "Missing tag in VSL:");
-		}
 		if (e[-1] == ']') {
 			r = e - 1;
 			while (r > buf && *r != '[')
diff --git a/doc/sphinx/dev-guide/index.rst b/doc/sphinx/dev-guide/index.rst
index 5db5217..fe4d759 100644
--- a/doc/sphinx/dev-guide/index.rst
+++ b/doc/sphinx/dev-guide/index.rst
@@ -80,5 +80,5 @@ These rules are imported from the X11 project:
 
 * Isolate complexity as much as possible.
 
-* Provide mechanism, rather than policy. 
+* Provide mechanism, rather than policy.
 
diff --git a/lib/libvmod_debug/vmod_debug.c b/lib/libvmod_debug/vmod_debug.c
index ee3a005..32f0b11 100644
--- a/lib/libvmod_debug/vmod_debug.c
+++ b/lib/libvmod_debug/vmod_debug.c
@@ -191,7 +191,7 @@ vmod_argtest(VRT_CTX, VCL_STRING one, VCL_REAL two, VCL_STRING three,
 	char buf[100];
 
 	bprintf(buf, "%s %g %s %s", one, two, three, comma);
-	return WS_Copy(ctx->ws, buf, -1);
+	return (WS_Copy(ctx->ws, buf, -1));
 }
 
 VCL_INT
@@ -352,10 +352,10 @@ event_function(VRT_CTX, struct vmod_priv *priv, enum vcl_event_e e)
 {
 
 	switch (e) {
-		case VCL_EVENT_LOAD: return event_load(ctx, priv);
-		case VCL_EVENT_WARM: return event_warm(ctx, priv);
-		case VCL_EVENT_COLD: return event_cold(ctx, priv);
-		default: return (0);
+	case VCL_EVENT_LOAD: return event_load(ctx, priv);
+	case VCL_EVENT_WARM: return event_warm(ctx, priv);
+	case VCL_EVENT_COLD: return event_cold(ctx, priv);
+	default: return (0);
 	}
 }
 
@@ -367,16 +367,18 @@ vmod_sleep(VRT_CTX, VCL_DURATION t)
 	VTIM_sleep(t);
 }
 
-static struct ws *wsfind(VRT_CTX, VCL_ENUM which) {
-	if (!strcmp(which, "client")) {
-		return ctx->ws;
-	} else if (!strcmp(which, "backend")) {
-		return ctx->bo->ws;
-	} else if (!strcmp(which, "session")) {
-		return ctx->req->sp->ws;
-	} else if (!strcmp(which, "thread")) {
-		return ctx->req->wrk->aws;
-	} else
+static struct ws *
+wsfind(VRT_CTX, VCL_ENUM which)
+{
+	if (!strcmp(which, "client"))
+		return (ctx->ws);
+	else if (!strcmp(which, "backend"))
+		return (ctx->bo->ws);
+	else if (!strcmp(which, "session"))
+		return (ctx->req->sp->ws);
+	else if (!strcmp(which, "thread"))
+		return (ctx->req->wrk->aws);
+	else
 		WRONG("No such workspace.");
 }
 
@@ -425,6 +427,7 @@ vmod_workspace_overflowed(VRT_CTX, VCL_ENUM which)
 }
 
 static char *debug_ws_snap;
+
 void
 vmod_workspace_snap(VRT_CTX, VCL_ENUM which)
 {



From varnish-commit at varnish-cache.org  Tue May 17 15:54:28 2016
From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org)
Date: Tue, 17 May 2016 17:54:28 +0200
Subject: my p*ssy is wet
Message-ID: <573B3EB4.5090802@varnish-cache.org>

r u down for right now? i'm 21/f looking for a f*ckbuddy on the side...
i'm crazy in bed ;) think you could tame my pu_$Sy?

my username is Danny65
u can see my naughty pics >>
 here  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-commit/attachments/20160517/8eeb02f1/attachment.html>

From phk at FreeBSD.org  Mon May  9 06:55:06 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Mon, 09 May 2016 08:55:06 +0200
Subject: [master] ae63cc5 We cannot assume that nobody else have killed the
 object we are trying to snipe. If they did, fail the sniping.
Message-ID: <E1azf5u-00055j-FW@project.varnish-software.com>


commit ae63cc53e77138f27af4f1a519c23161e8c29c75
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon May 9 06:53:09 2016 +0000

    We cannot assume that nobody else have killed the object we are
    trying to snipe.  If they did, fail the sniping.
    
    Fixes #1942

diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index 6df5160..cc4e96f 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -724,10 +724,8 @@ HSH_Snipe(const struct worker *wrk, struct objcore *oc)
 	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
 	CHECK_OBJ_NOTNULL(oc->objhead, OBJHEAD_MAGIC);
 
-	AZ(oc->flags & OC_F_DYING);
-
 	if (oc->refcnt == 1 && !Lck_Trylock(&oc->objhead->mtx)) {
-		if (oc->refcnt == 1) {
+		if (oc->refcnt == 1 && !(oc->flags & OC_F_DYING)) {
 			oc->flags |= OC_F_DYING;
 			oc->refcnt++;
 			retval = 1;



From nils.goroll at uplex.de  Mon May  9 08:23:05 2016
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon, 09 May 2016 10:23:05 +0200
Subject: [master] 0895d2e dont panic on pthread_cond_timedwait being
 interrupted
Message-ID: <E1azgT3-0002GB-Ry@project.varnish-software.com>


commit 0895d2e61f61c2942e6bc4db3347adec7966d6c2
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Mon May 9 10:18:32 2016 +0200

    dont panic on pthread_cond_timedwait being interrupted
    
    As discussed with phk: Callers of Lck_CondWait with timeout should
    be prepared for early wakeups.

diff --git a/bin/varnishd/cache/cache_lck.c b/bin/varnishd/cache/cache_lck.c
index b702c2f..90190dc 100644
--- a/bin/varnishd/cache/cache_lck.c
+++ b/bin/varnishd/cache/cache_lck.c
@@ -198,7 +198,9 @@ Lck_CondWait(pthread_cond_t *cond, struct lock *lck, double when)
 		ts.tv_nsec = (long)(modf(when, &t) * 1e9);
 		ts.tv_sec = (long)t;
 		retval = pthread_cond_timedwait(cond, &ilck->mtx, &ts);
-		assert(retval == 0 || retval == ETIMEDOUT);
+		assert(retval == 0 ||
+		    retval == ETIMEDOUT ||
+		    retval == EINTR);
 	}
 	AZ(ilck->held);
 	ilck->held = 1;



From phk at FreeBSD.org  Mon May  9 12:28:05 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Mon, 09 May 2016 14:28:05 +0200
Subject: [master] 867743f Split rushing of waiting list into two stages: 1 get
 the req's to be rushed under the oh->mtx. 2 rush them out of the mtx.
Message-ID: <E1azkI9-0008UP-Pl@project.varnish-software.com>


commit 867743f0257413304c5922b77513201da0b9242b
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon May 9 12:26:51 2016 +0000

    Split rushing of waiting list into two stages: 1 get the req's to be rushed
    under the oh->mtx.  2 rush them out of the mtx.

diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index cc4e96f..c808494 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -57,15 +57,21 @@
 
 #include "cache.h"
 
-
 #include "hash/hash_slinger.h"
 #include "vsha256.h"
 #include "vtim.h"
 
+struct rush {
+	unsigned		magic;
+#define RUSH_MAGIC		0xa1af5f01
+	VTAILQ_HEAD(,req)	reqs;
+};
+
 static const struct hash_slinger *hash;
 static struct objhead *private_oh;
 
-static void hsh_rush(struct worker *wrk, struct objhead *oh);
+static void hsh_rush1(struct worker *, struct objhead *, struct rush *, int);
+static void hsh_rush2(struct worker *, struct rush *);
 
 /*---------------------------------------------------------------------*/
 
@@ -258,6 +264,7 @@ HSH_Insert(struct worker *wrk, const void *digest, struct objcore *oc,
     struct ban *ban)
 {
 	struct objhead *oh;
+	struct rush rush;
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	AN(digest);
@@ -266,6 +273,7 @@ HSH_Insert(struct worker *wrk, const void *digest, struct objcore *oc,
 	AN(oc->flags & OC_F_BUSY);
 	AZ(oc->flags & OC_F_PRIVATE);
 	assert(oc->refcnt == 1);
+	INIT_OBJ(&rush, RUSH_MAGIC);
 
 	hsh_prealloc(wrk);
 
@@ -293,8 +301,9 @@ HSH_Insert(struct worker *wrk, const void *digest, struct objcore *oc,
 	VTAILQ_INSERT_HEAD(&oh->objcs, oc, hsh_list);
 	oc->flags &= ~OC_F_BUSY;
 	if (!VTAILQ_EMPTY(&oh->waitinglist))
-		hsh_rush(wrk, oh);
+		hsh_rush1(wrk, oh, &rush, 0);
 	Lck_Unlock(&oh->mtx);
+	hsh_rush2(wrk, &rush);
 }
 
 /*---------------------------------------------------------------------
@@ -495,19 +504,21 @@ HSH_Lookup(struct req *req, struct objcore **ocp, struct objcore **bocp,
 }
 
 /*---------------------------------------------------------------------
+ * Pick the req's we are going to rush from the waiting list
  */
 
 static void
-hsh_rush(struct worker *wrk, struct objhead *oh)
+hsh_rush1(struct worker *wrk, struct objhead *oh, struct rush *r, int all)
 {
 	unsigned u;
 	struct req *req;
-	struct sess *sp;
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(oh, OBJHEAD_MAGIC);
+	CHECK_OBJ_NOTNULL(r, RUSH_MAGIC);
+	VTAILQ_INIT(&r->reqs);
 	Lck_AssertHeld(&oh->mtx);
-	for (u = 0; u < cache_param->rush_exponent; u++) {
+	for (u = 0; u < cache_param->rush_exponent || all; u++) {
 		req = VTAILQ_FIRST(&oh->waitinglist);
 		if (req == NULL)
 			break;
@@ -515,32 +526,42 @@ hsh_rush(struct worker *wrk, struct objhead *oh)
 		wrk->stats->busy_wakeup++;
 		AZ(req->wrk);
 		VTAILQ_REMOVE(&oh->waitinglist, req, w_list);
+		VTAILQ_INSERT_TAIL(&r->reqs, req, w_list);
+	}
+}
+
+/*---------------------------------------------------------------------
+ * Rush req's that came from waiting list.
+ */
+
+static void
+hsh_rush2(struct worker *wrk, struct rush *r)
+{
+	struct req *req;
+	struct sess *sp;
+
+	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
+	CHECK_OBJ_NOTNULL(r, RUSH_MAGIC);
+
+	while(!VTAILQ_EMPTY(&r->reqs)) {
+		req = VTAILQ_FIRST(&r->reqs);
+		CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
+		VTAILQ_REMOVE(&r->reqs, req, w_list);
 		DSL(DBG_WAITINGLIST, req->vsl->wid, "off waiting list");
-		if (SES_Reschedule_Req(req)) {
-			/*
-			 * In case of overloads, we ditch the entire
-			 * waiting list.
-			 */
-			wrk->stats->busy_wakeup--;
-			while (1) {
-				wrk->stats->busy_killed++;
-				AN (req->vcl);
-				VCL_Rel(&req->vcl);
-				sp = req->sp;
-				CHECK_OBJ_NOTNULL(sp, SESS_MAGIC);
-				CNT_AcctLogCharge(wrk->stats, req);
-				Req_Release(req);
-				SES_Delete(sp, SC_OVERLOAD, NAN);
-				req = VTAILQ_FIRST(&oh->waitinglist);
-				if (req == NULL)
-					break;
-				CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
-				VTAILQ_REMOVE(&oh->waitinglist, req, w_list);
-				DSL(DBG_WAITINGLIST, req->vsl->wid,
-				    "kill from waiting list");
-			}
-			break;
-		}
+		if (!SES_Reschedule_Req(req))
+			continue;
+		/* Couldn't schedule, ditch */
+		wrk->stats->busy_wakeup--;
+		wrk->stats->busy_killed++;
+		AN (req->vcl);
+		VCL_Rel(&req->vcl);
+		sp = req->sp;
+		CHECK_OBJ_NOTNULL(sp, SESS_MAGIC);
+		CNT_AcctLogCharge(wrk->stats, req);
+		Req_Release(req);
+		SES_Delete(sp, SC_OVERLOAD, NAN);
+		DSL(DBG_WAITINGLIST, req->vsl->wid, "kill from waiting list");
+		(void)usleep(100000);
 	}
 }
 
@@ -657,11 +678,13 @@ void
 HSH_Unbusy(struct worker *wrk, struct objcore *oc)
 {
 	struct objhead *oh;
+	struct rush rush;
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
 	oh = oc->objhead;
 	CHECK_OBJ(oh, OBJHEAD_MAGIC);
+	INIT_OBJ(&rush, RUSH_MAGIC);
 
 	AN(oc->stobj->stevedore);
 	AN(oc->flags & OC_F_BUSY);
@@ -684,10 +707,11 @@ HSH_Unbusy(struct worker *wrk, struct objcore *oc)
 	VTAILQ_INSERT_HEAD(&oh->objcs, oc, hsh_list);
 	oc->flags &= ~OC_F_BUSY;
 	if (!VTAILQ_EMPTY(&oh->waitinglist))
-		hsh_rush(wrk, oh);
+		hsh_rush1(wrk, oh, &rush, 0);
 	Lck_Unlock(&oh->mtx);
 	if (!(oc->flags & OC_F_PRIVATE))
 		EXP_Insert(wrk, oc);
+	hsh_rush2(wrk, &rush);
 }
 
 /*====================================================================
@@ -818,6 +842,7 @@ HSH_DerefObjCore(struct worker *wrk, struct objcore **ocp)
 {
 	struct objcore *oc;
 	struct objhead *oh;
+	struct rush rush;
 	unsigned r;
 
 	AN(ocp);
@@ -827,6 +852,7 @@ HSH_DerefObjCore(struct worker *wrk, struct objcore **ocp)
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
 	assert(oc->refcnt > 0);
+	INIT_OBJ(&rush, RUSH_MAGIC);
 
 	oh = oc->objhead;
 	CHECK_OBJ_NOTNULL(oh, OBJHEAD_MAGIC);
@@ -837,8 +863,9 @@ HSH_DerefObjCore(struct worker *wrk, struct objcore **ocp)
 	if (!r)
 		VTAILQ_REMOVE(&oh->objcs, oc, hsh_list);
 	if (!VTAILQ_EMPTY(&oh->waitinglist))
-		hsh_rush(wrk, oh);
+		hsh_rush1(wrk, oh, &rush, 0);
 	Lck_Unlock(&oh->mtx);
+	hsh_rush2(wrk, &rush);
 	if (r != 0)
 		return (r);
 
@@ -861,10 +888,12 @@ int
 HSH_DerefObjHead(struct worker *wrk, struct objhead **poh)
 {
 	struct objhead *oh;
+	struct rush rush;
 	int r;
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	TAKE_OBJ_NOTNULL(oh, poh, OBJHEAD_MAGIC);
+	INIT_OBJ(&rush, RUSH_MAGIC);
 
 	if (oh == private_oh) {
 		assert(VTAILQ_EMPTY(&oh->waitinglist));
@@ -883,16 +912,14 @@ HSH_DerefObjHead(struct worker *wrk, struct objhead **poh)
 	 * just make the hold the same ref's as objcore, that would
 	 * confuse hashers.
 	 */
-	while (!VTAILQ_EMPTY(&oh->waitinglist)) {
-		Lck_Lock(&oh->mtx);
-		assert(oh->refcnt > 0);
-		r = oh->refcnt;
-		hsh_rush(wrk, oh);
+	Lck_Lock(&oh->mtx);
+	while (oh->refcnt == 1 && !VTAILQ_EMPTY(&oh->waitinglist)) {
+		hsh_rush1(wrk, oh, &rush, 1);
 		Lck_Unlock(&oh->mtx);
-		if (r > 1)
-			break;
-		usleep(100000);
+		hsh_rush2(wrk, &rush);
+		Lck_Lock(&oh->mtx);
 	}
+	Lck_Unlock(&oh->mtx);
 
 	assert(oh->refcnt > 0);
 	r = hash->deref(oh);



From phk at FreeBSD.org  Mon May  9 14:33:06 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Mon, 09 May 2016 16:33:06 +0200
Subject: [master] 2a921ec Add a new transport method for reembarking a request
 from the waiting list.
Message-ID: <E1azmF8-00071I-Q5@project.varnish-software.com>


commit 2a921ecb5e076ed1e2e423ec2861fa2cae013eac
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon May 9 14:31:49 2016 +0000

    Add a new transport method for reembarking a request from the waiting list.

diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index c808494..7e4acc3 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -56,6 +56,7 @@
 #include <stdlib.h>
 
 #include "cache.h"
+#include "cache/cache_transport.h"
 
 #include "hash/hash_slinger.h"
 #include "vsha256.h"
@@ -548,6 +549,10 @@ hsh_rush2(struct worker *wrk, struct rush *r)
 		CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
 		VTAILQ_REMOVE(&r->reqs, req, w_list);
 		DSL(DBG_WAITINGLIST, req->vsl->wid, "off waiting list");
+		if (req->transport->reembark != NULL) {
+			req->transport->reembark(wrk, req);
+			continue;
+		}
 		if (!SES_Reschedule_Req(req))
 			continue;
 		/* Couldn't schedule, ditch */
diff --git a/bin/varnishd/cache/cache_transport.h b/bin/varnishd/cache/cache_transport.h
index 2b22519..ef57325 100644
--- a/bin/varnishd/cache/cache_transport.h
+++ b/bin/varnishd/cache/cache_transport.h
@@ -41,6 +41,7 @@ typedef void vtr_req_body_f (struct req *);
 typedef void vtr_sess_panic_f (struct vsb *, const struct sess *);
 typedef void vtr_req_panic_f (struct vsb *, const struct req *);
 typedef void vtr_req_fail_f (struct req *, enum sess_close);
+typedef void vtr_reembark_f (struct worker *, struct req *);
 
 struct transport {
 	unsigned			magic;
@@ -58,6 +59,7 @@ struct transport {
 	vtr_deliver_f			*deliver;
 	vtr_sess_panic_f		*sess_panic;
 	vtr_req_panic_f			*req_panic;
+	vtr_reembark_f			*reembark;
 
 	VTAILQ_ENTRY(transport)		list;
 };



From phk at FreeBSD.org  Mon May  9 16:20:06 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Mon, 09 May 2016 18:20:06 +0200
Subject: [master] fc40ec3 Add a flag to brand req's on waiting list.
Message-ID: <E1aznug-0005OO-W4@project.varnish-software.com>


commit fc40ec31e8fcab4f4bca632294ee056764202552
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon May 9 15:48:59 2016 +0000

    Add a flag to brand req's on waiting list.

diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index 7e4acc3..3c2cdcb 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -500,6 +500,7 @@ HSH_Lookup(struct req *req, struct objcore **ocp, struct objcore **bocp,
 	 */
 	req->hash_objhead = oh;
 	req->wrk = NULL;
+	req->waitinglist = 1;
 	Lck_Unlock(&oh->mtx);
 	return (HSH_BUSY);
 }
@@ -528,6 +529,7 @@ hsh_rush1(struct worker *wrk, struct objhead *oh, struct rush *r, int all)
 		AZ(req->wrk);
 		VTAILQ_REMOVE(&oh->waitinglist, req, w_list);
 		VTAILQ_INSERT_TAIL(&r->reqs, req, w_list);
+		req->waitinglist = 0;
 	}
 }
 
diff --git a/include/tbl/req_flags.h b/include/tbl/req_flags.h
index 55bdc74..4b8ec5b 100644
--- a/include/tbl/req_flags.h
+++ b/include/tbl/req_flags.h
@@ -34,4 +34,5 @@ REQ_FLAG(disable_esi,		0, 0, "")
 REQ_FLAG(hash_ignore_busy,	1, 1, "")
 REQ_FLAG(hash_always_miss,	1, 1, "")
 REQ_FLAG(is_hit,		0, 0, "")
+REQ_FLAG(waitinglist,		0, 0, "")
 /*lint -restore */



From phk at FreeBSD.org  Mon May  9 16:20:07 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Mon, 09 May 2016 18:20:07 +0200
Subject: [master] f839d55 Use a condvar instead of a polling sleep to alert
 parent req that ESI include is off the waiting list.
Message-ID: <E1aznuh-0005OU-1k@project.varnish-software.com>


commit f839d5586842094a2cea65a2ea7337cb2836f72c
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon May 9 16:18:10 2016 +0000

    Use a condvar instead of a polling sleep to alert parent req that
    ESI include is off the waiting list.
    
    Many thanks to Nils for pushing for us to fix this.
    
    Superceedes #1902

diff --git a/bin/varnishd/cache/cache_esi_deliver.c b/bin/varnishd/cache/cache_esi_deliver.c
index 876a0b6..fa944d0 100644
--- a/bin/varnishd/cache/cache_esi_deliver.c
+++ b/bin/varnishd/cache/cache_esi_deliver.c
@@ -42,7 +42,8 @@
 #include "vend.h"
 #include "vgz.h"
 
-static vtr_deliver_f VED_Deliver;
+static vtr_deliver_f ved_deliver;
+static vtr_reembark_f ved_reembark;
 
 static const uint8_t gzip_hdr[] = {
 	0x1f, 0x8b, 0x08,
@@ -59,6 +60,7 @@ struct ecx {
 	int		state;
 	ssize_t		l;
 	int		isgzip;
+	int		woken;
 
 	struct req	*preq;
 	ssize_t		l_crc;
@@ -68,11 +70,28 @@ struct ecx {
 static const struct transport VED_transport = {
 	.magic =	TRANSPORT_MAGIC,
 	.name =		"ESI_INCLUDE",
-	.deliver =	VED_Deliver,
+	.deliver =	ved_deliver,
+	.reembark =	ved_reembark,
 };
 
 /*--------------------------------------------------------------------*/
 
+static void __match_proto__(vtr_reembark_f)
+ved_reembark(struct worker *wrk, struct req *req)
+{
+	struct ecx *ecx;
+
+	(void)wrk;
+	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
+	CAST_OBJ_NOTNULL(ecx, req->transport_priv, ECX_MAGIC);
+	Lck_Lock(&req->sp->mtx);
+	ecx->woken = 1;
+	AZ(pthread_cond_signal(&ecx->preq->wrk->cond));
+	Lck_Unlock(&req->sp->mtx);
+}
+
+/*--------------------------------------------------------------------*/
+
 static void
 ved_include(struct req *preq, const char *src, const char *host,
     struct ecx *ecx)
@@ -140,7 +159,6 @@ ved_include(struct req *preq, const char *src, const char *host,
 
 	req->vcl = preq->vcl;
 	preq->vcl = NULL;
-	req->wrk = preq->wrk;
 
 	/*
 	 * XXX: We should decide if we should cache the director
@@ -163,14 +181,20 @@ ved_include(struct req *preq, const char *src, const char *host,
 
 	while (1) {
 		req->wrk = wrk;
+		ecx->woken = 0;
 		s = CNT_Request(wrk, req);
 		if (s == REQ_FSM_DONE)
 			break;
 		DSL(DBG_WAITINGLIST, req->vsl->wid,
 		    "loop waiting for ESI (%d)", (int)s);
 		assert(s == REQ_FSM_DISEMBARK);
+		Lck_Lock(&req->sp->mtx);
+		if (!ecx->woken)
+			(void)Lck_CondWait(
+			    &ecx->preq->wrk->cond, &req->sp->mtx, 0);
+		Lck_Unlock(&req->sp->mtx);
+		ecx->woken = 0;
 		AZ(req->wrk);
-		(void)usleep(10000);
 	}
 
 	VRTPRIV_dynamic_kill(req->sp->privs, (uintptr_t)req);
@@ -748,7 +772,7 @@ ved_vdp_bytes(struct req *req, enum vdp_action act, void **priv,
 /*--------------------------------------------------------------------*/
 
 static void __match_proto__(vtr_deliver_f)
-VED_Deliver(struct req *req, struct boc *boc, int wantbody)
+ved_deliver(struct req *req, struct boc *boc, int wantbody)
 {
 	int i;
 	struct ecx *ecx;
diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index 3c2cdcb..15617a7 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -334,7 +334,7 @@ hsh_insert_busyobj(struct worker *wrk, struct objhead *oh)
 
 enum lookup_e
 HSH_Lookup(struct req *req, struct objcore **ocp, struct objcore **bocp,
-    int wait_for_busy, int always_insert)
+    int always_insert)
 {
 	struct worker *wrk;
 	struct objhead *oh;
@@ -483,14 +483,9 @@ HSH_Lookup(struct req *req, struct objcore **ocp, struct objcore **bocp,
 
 	AZ(req->hash_ignore_busy);
 
-	if (wait_for_busy) {
-		VTAILQ_INSERT_TAIL(&oh->waitinglist, req, w_list);
-		if (DO_DEBUG(DBG_WAITINGLIST))
-			VSLb(req->vsl, SLT_Debug, "on waiting list <%p>", oh);
-	} else {
-		if (DO_DEBUG(DBG_WAITINGLIST))
-			VSLb(req->vsl, SLT_Debug, "hit busy obj <%p>", oh);
-	}
+	VTAILQ_INSERT_TAIL(&oh->waitinglist, req, w_list);
+	if (DO_DEBUG(DBG_WAITINGLIST))
+		VSLb(req->vsl, SLT_Debug, "on waiting list <%p>", oh);
 
 	wrk->stats->busy_sleep++;
 	/*
diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index 72f24a7..f057e26 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -367,10 +367,7 @@ cnt_lookup(struct worker *wrk, struct req *req)
 	AZ(req->objcore);
 	if (req->hash_objhead)
 		had_objhead = 1;
-	lr = HSH_Lookup(req, &oc, &busy,
-	    req->esi_level == 0 ? 1 : 0,
-	    req->hash_always_miss ? 1 : 0
-	);
+	lr = HSH_Lookup(req, &oc, &busy, req->hash_always_miss ? 1 : 0);
 	if (lr == HSH_BUSY) {
 		/*
 		 * We lost the session to a busy object, disembark the
@@ -777,7 +774,7 @@ cnt_purge(struct worker *wrk, struct req *req)
 	VRY_Prep(req);
 
 	AZ(req->objcore);
-	lr = HSH_Lookup(req, &oc, &boc, 1, 1);
+	lr = HSH_Lookup(req, &oc, &boc, 1);
 	assert (lr == HSH_MISS);
 	AZ(oc);
 	CHECK_OBJ_NOTNULL(boc, OBJCORE_MAGIC);
diff --git a/bin/varnishd/hash/hash_slinger.h b/bin/varnishd/hash/hash_slinger.h
index db4510b..7743dec 100644
--- a/bin/varnishd/hash/hash_slinger.h
+++ b/bin/varnishd/hash/hash_slinger.h
@@ -63,7 +63,7 @@ enum lookup_e {
 struct ban;
 void HSH_Cleanup(struct worker *w);
 enum lookup_e HSH_Lookup(struct req *, struct objcore **, struct objcore **,
-    int wait_for_busy, int always_insert);
+    int always_insert);
 void HSH_Ref(struct objcore *o);
 void HSH_Init(const struct hash_slinger *slinger);
 void HSH_AddString(struct req *, void *ctx, const char *str);



From phk at FreeBSD.org  Mon May  9 21:40:07 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Mon, 09 May 2016 23:40:07 +0200
Subject: [master] cac8e85 Give/Move the previous reembark code to HTTP1,
 and close a narrow race while we're at it.
Message-ID: <E1azsuN-0006mi-Sk@project.varnish-software.com>


commit cac8e853e896d6d2448977491a4db26dbb1bf915
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon May 9 21:39:17 2016 +0000

    Give/Move the previous reembark code to HTTP1, and close a
    narrow race while we're at it.

diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index 15617a7..7f8bebb 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -536,7 +536,6 @@ static void
 hsh_rush2(struct worker *wrk, struct rush *r)
 {
 	struct req *req;
-	struct sess *sp;
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(r, RUSH_MAGIC);
@@ -546,24 +545,8 @@ hsh_rush2(struct worker *wrk, struct rush *r)
 		CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
 		VTAILQ_REMOVE(&r->reqs, req, w_list);
 		DSL(DBG_WAITINGLIST, req->vsl->wid, "off waiting list");
-		if (req->transport->reembark != NULL) {
-			req->transport->reembark(wrk, req);
-			continue;
-		}
-		if (!SES_Reschedule_Req(req))
-			continue;
-		/* Couldn't schedule, ditch */
-		wrk->stats->busy_wakeup--;
-		wrk->stats->busy_killed++;
-		AN (req->vcl);
-		VCL_Rel(&req->vcl);
-		sp = req->sp;
-		CHECK_OBJ_NOTNULL(sp, SESS_MAGIC);
-		CNT_AcctLogCharge(wrk->stats, req);
-		Req_Release(req);
-		SES_Delete(sp, SC_OVERLOAD, NAN);
-		DSL(DBG_WAITINGLIST, req->vsl->wid, "kill from waiting list");
-		(void)usleep(100000);
+		AN(req->transport->reembark);
+		req->transport->reembark(wrk, req);
 	}
 }
 
diff --git a/bin/varnishd/http1/cache_http1_fsm.c b/bin/varnishd/http1/cache_http1_fsm.c
index 216dedf..27a5597 100644
--- a/bin/varnishd/http1/cache_http1_fsm.c
+++ b/bin/varnishd/http1/cache_http1_fsm.c
@@ -175,6 +175,31 @@ http1_req_fail(struct req *req, enum sess_close reason)
 		SES_Close(req->sp, reason);
 }
 
+static void __match_proto__(vtr_reembark_f)
+http1_reembark(struct worker *wrk, struct req *req)
+{
+	struct sess *sp;
+
+	sp = req->sp;
+	CHECK_OBJ_NOTNULL(sp, SESS_MAGIC);
+
+	http1_setstate(sp, H1BUSY);
+
+	if (!SES_Reschedule_Req(req))
+		return;
+
+	/* Couldn't schedule, ditch */
+	wrk->stats->busy_wakeup--;
+	wrk->stats->busy_killed++;
+	AN (req->vcl);
+	VCL_Rel(&req->vcl);
+	CNT_AcctLogCharge(wrk->stats, req);
+	Req_Release(req);
+	SES_Delete(sp, SC_OVERLOAD, NAN);
+	DSL(DBG_WAITINGLIST, req->vsl->wid, "kill from waiting list");
+	usleep(10000);
+}
+
 struct transport HTTP1_transport = {
 	.name =			"HTTP/1",
 	.magic =		TRANSPORT_MAGIC,
@@ -185,6 +210,7 @@ struct transport HTTP1_transport = {
 	.new_session =		http1_new_session,
 	.sess_panic =		http1_sess_panic,
 	.req_panic =		http1_req_panic,
+	.reembark =		http1_reembark,
 };
 
 /*----------------------------------------------------------------------
@@ -394,10 +420,8 @@ HTTP1_Session(struct worker *wrk, struct req *req)
 			req->transport = &HTTP1_transport;
 			req->task.func = http1_req;
 			req->task.priv = req;
-			if (CNT_Request(wrk, req) == REQ_FSM_DISEMBARK) {
-				http1_setstate(sp, H1BUSY);
+			if (CNT_Request(wrk, req) == REQ_FSM_DISEMBARK)
 				return;
-			}
 			req->transport = NULL;
 			req->task.func = NULL;
 			req->task.priv = NULL;



From fgsch at lodoss.net  Mon May  9 22:17:06 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Tue, 10 May 2016 00:17:06 +0200
Subject: [master] e142a19 Correctly handle EOF responses with HTTP/1.1
Message-ID: <E1aztUA-0000Ol-0j@project.varnish-software.com>


commit e142a199c53dd9331001cb29678602e726a35690
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Thu May 5 16:44:25 2016 +0100

    Correctly handle EOF responses with HTTP/1.1
    
    Fixes #1918.

diff --git a/bin/varnishd/http1/cache_http1_proto.c b/bin/varnishd/http1/cache_http1_proto.c
index 30b318c..713a8d6 100644
--- a/bin/varnishd/http1/cache_http1_proto.c
+++ b/bin/varnishd/http1/cache_http1_proto.c
@@ -271,7 +271,7 @@ http1_splitline(struct http *hp, struct http_conn *htc, const int *hf,
 /*--------------------------------------------------------------------*/
 
 static enum body_status
-http1_body_status(const struct http *hp, struct http_conn *htc)
+http1_body_status(const struct http *hp, struct http_conn *htc, int request)
 {
 	ssize_t cl;
 	const char *b;
@@ -301,7 +301,7 @@ http1_body_status(const struct http *hp, struct http_conn *htc)
 		return (cl == 0 ? BS_NONE : BS_LENGTH);
 	}
 
-	if (hp->protover == 11)
+	if (hp->protover == 11 && (request || !http_HdrIs(hp, H_Connection, "close")))
 		return (BS_NONE);
 
 	if (http_HdrIs(hp, H_Connection, "keep-alive")) {
@@ -372,7 +372,7 @@ HTTP1_DissectRequest(struct http_conn *htc, struct http *hp)
 		}
 	}
 
-	htc->body_status = http1_body_status(hp, htc);
+	htc->body_status = http1_body_status(hp, htc, 1);
 	if (htc->body_status == BS_ERROR)
 		return (400);
 
@@ -451,7 +451,7 @@ HTTP1_DissectResponse(struct http_conn *htc, struct http *hp,
 	    !Tlen(hp->hd[HTTP_HDR_REASON]))
 		http_SetH(hp, HTTP_HDR_REASON, http_Status2Reason(hp->status));
 
-	htc->body_status = http1_body_status(hp, htc);
+	htc->body_status = http1_body_status(hp, htc, 0);
 
 	return (retval);
 }
diff --git a/bin/varnishtest/tests/r01918.vtc b/bin/varnishtest/tests/r01918.vtc
new file mode 100644
index 0000000..c0593aa
--- /dev/null
+++ b/bin/varnishtest/tests/r01918.vtc
@@ -0,0 +1,22 @@
+varnishtest "Check EOF responses work with HTTP/1.1"
+
+server s1 {
+	rxreq
+	txresp -nolen -bodylen 10
+	rxreq
+	txresp -hdr "Connection: close" -nolen -bodylen 10
+} -start
+
+varnish v1 -vcl+backend {
+} -start
+
+client c1 {
+	txreq -url /1
+	rxresp
+	expect resp.status == 200
+	expect resp.bodylen == 0
+	txreq -url /2
+	rxresp
+	expect resp.status == 200
+	expect resp.bodylen == 10
+} -run



From fgsch at lodoss.net  Mon May  9 22:29:07 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Tue, 10 May 2016 00:29:07 +0200
Subject: [master] 1c662d9 Add support for TCP Fast Open extension
Message-ID: <E1aztfn-00016j-Ja@project.varnish-software.com>


commit 1c662d992d6cc4ed84726d39f6a40cea0fdc924a
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Thu May 5 16:06:23 2016 +0100

    Add support for TCP Fast Open extension
    
    Disabled by default.  phk@ ok.

diff --git a/bin/varnishd/cache/cache_acceptor.c b/bin/varnishd/cache/cache_acceptor.c
index 9f3a341..8e62c67 100644
--- a/bin/varnishd/cache/cache_acceptor.c
+++ b/bin/varnishd/cache/cache_acceptor.c
@@ -506,6 +506,14 @@ vca_acct(void *arg)
 	VTAILQ_FOREACH(ls, &heritage.socks, list) {
 		CHECK_OBJ_NOTNULL(ls->transport, TRANSPORT_MAGIC);
 		assert (ls->sock > 0);		// We know where stdin is
+		if (cache_param->tcp_fastopen) {
+			int i;
+			i = VTCP_fastopen(ls->sock, cache_param->listen_depth);
+			if (i)
+				VSL(SLT_Error, ls->sock,
+				    "Kernel TCP Fast Open: sock=%d, ret=%d %s",
+				    ls->sock, i, strerror(errno));
+		}
 		AZ(listen(ls->sock, cache_param->listen_depth));
 		vca_tcp_opt_set(ls->sock, 1);
 		if (cache_param->accept_filter) {
diff --git a/configure.ac b/configure.ac
index edba56a..f724d7e 100644
--- a/configure.ac
+++ b/configure.ac
@@ -530,6 +530,34 @@ if test "$ac_cv_have_tcp_keep" = yes; then
 fi
 LIBS="${save_LIBS}"
 
+# Check if the OS supports TCP_FASTOPEN socket option
+save_LIBS="${LIBS}"
+LIBS="${LIBS} ${NET_LIBS}"
+AC_CACHE_CHECK([for TCP_FASTOPEN socket option],
+  [ac_cv_have_tcp_fastopen],
+  [AC_RUN_IFELSE(
+    [AC_LANG_PROGRAM([[
+#include <stdio.h>
+#include <sys/types.h>
+#include <sys/socket.h>
+#include <netinet/in.h>
+#include <netinet/tcp.h>
+    ]],[[
+int s = socket(AF_INET, SOCK_STREAM, 0);
+int i;
+i = 5;
+if (setsockopt(s, SOL_TCP, TCP_FASTOPEN, &i, sizeof i))
+  return (1);
+return (0);
+    ]])],
+    [ac_cv_have_tcp_fastopen=yes],
+    [ac_cv_have_tcp_fastopen=no])
+  ])
+if test "$ac_cv_have_tcp_fastopen" = yes; then
+   AC_DEFINE([HAVE_TCP_FASTOPEN], [1], [Define if OS supports TCP_FASTOPEN socket option])
+fi
+LIBS="${save_LIBS}"
+
 # Run-time directory
 VARNISH_STATE_DIR='${localstatedir}/varnish'
 AC_SUBST(VARNISH_STATE_DIR)
diff --git a/include/tbl/params.h b/include/tbl/params.h
index 3c03ce1..c1340a5 100644
--- a/include/tbl/params.h
+++ b/include/tbl/params.h
@@ -986,6 +986,20 @@ PARAM(
 	/* func */	NULL
 )
 
+PARAM(
+	/* name */	tcp_fastopen,
+	/* typ */	bool,
+	/* min */	NULL,
+	/* max */	NULL,
+	/* default */	"off",
+	/* units */	"bool",
+	/* flags */	MUST_RESTART,
+	/* s-text */
+	"Enable TCP Fast Open extension (if available in the kernel).",
+	/* l-text */	NULL,
+	/* func */	NULL
+)
+
 #if 0
 PARAM(
 	/* name */	tcp_keepalive_intvl,
diff --git a/include/vtcp.h b/include/vtcp.h
index a42e768..2937593 100644
--- a/include/vtcp.h
+++ b/include/vtcp.h
@@ -44,6 +44,7 @@ void VTCP_myname(int sock, char *abuf, unsigned alen,
 void VTCP_hisname(int sock, char *abuf, unsigned alen,
     char *pbuf, unsigned plen);
 int VTCP_filter_http(int sock);
+int VTCP_fastopen(int sock, int depth);
 int VTCP_blocking(int sock);
 int VTCP_nonblocking(int sock);
 int VTCP_linger(int sock, int linger);
diff --git a/lib/libvarnish/vtcp.c b/lib/libvarnish/vtcp.c
index 09bf909..83e33b4 100644
--- a/lib/libvarnish/vtcp.c
+++ b/lib/libvarnish/vtcp.c
@@ -186,6 +186,30 @@ VTCP_filter_http(int sock)
 
 #endif
 
+/*--------------------------------------------------------------------*/
+
+#ifdef HAVE_TCP_FASTOPEN
+
+int
+VTCP_fastopen(int sock, int depth)
+{
+	return (setsockopt(sock, SOL_TCP, TCP_FASTOPEN,
+	    &depth, sizeof depth));
+}
+
+#else
+
+int
+VTCP_fastopen(int sock, int depth)
+{
+	errno = EOPNOTSUPP;
+	(void)sock;
+	(void)depth;
+	return (-1);
+}
+
+#endif
+
 /*--------------------------------------------------------------------
  * Functions for controlling NONBLOCK mode.
  *



From varnish-commit at varnish-cache.org  Tue May 10 05:07:42 2016
From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org)
Date: 10 May 2016 07:07:42 +0200
Subject: want hot night?
Message-ID: <506956268843177621740022@varnish-cache.org>

r u down for right now? i'm 28/f looking for a f*ckbuddy on the side...
i'm crazy in bed ;) think you could tame my pu_$Sy?

my username is Hella32
u can see my naughty pics >>
 here  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-commit/attachments/20160510/d66f506e/attachment.html>

From phk at FreeBSD.org  Tue May 10 08:38:05 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 10 May 2016 10:38:05 +0200
Subject: [master] 7559381 Bring flopen/fltest into the V* namespace
Message-ID: <E1b03B7-0002a8-O8@project.varnish-software.com>


commit 75593811e651e423d4d14a775bbf3913de5951c9
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue May 10 08:36:59 2016 +0000

    Bring flopen/fltest into the V* namespace

diff --git a/bin/varnishd/mgt/mgt_shmem.c b/bin/varnishd/mgt/mgt_shmem.c
index 9345d38..fe6a638 100644
--- a/bin/varnishd/mgt/mgt_shmem.c
+++ b/bin/varnishd/mgt/mgt_shmem.c
@@ -44,7 +44,7 @@
 #include "mgt/mgt.h"
 #include "common/heritage.h"
 
-#include "flopen.h"
+#include "vfl.h"
 #include "vsm_priv.h"
 #include "vmb.h"
 #include "vfil.h"
@@ -112,7 +112,7 @@ vsm_n_check(void)
 		fprintf(stderr,
 		    "VSM (%s) not a regular file.\n", VSM_FILENAME);
 	} else {
-		i = fltest(fd, &pid);
+		i = VFL_Test(fd, &pid);
 		if (i < 0) {
 			fprintf(stderr,
 			    "Cannot determine locking status of VSM (%s)\n.",
@@ -150,7 +150,7 @@ vsm_zerofile(const char *fn, ssize_t size)
 	int fd;
 	int flags;
 
-	fd = flopen(fn, O_RDWR | O_CREAT | O_EXCL | O_NONBLOCK, 0640);
+	fd = VFL_Open(fn, O_RDWR | O_CREAT | O_EXCL | O_NONBLOCK, 0640);
 	if (fd < 0) {
 		fprintf(stderr, "Could not create %s: %s\n",
 		    fn, strerror(errno));
diff --git a/bin/varnishhist/Makefile.am b/bin/varnishhist/Makefile.am
index 21d4285..e4d5e97 100644
--- a/bin/varnishhist/Makefile.am
+++ b/bin/varnishhist/Makefile.am
@@ -13,7 +13,7 @@ varnishhist_SOURCES = varnishhist.c \
 	$(top_srcdir)/lib/libvarnish/version.c \
 	$(top_srcdir)/lib/libvarnish/vpf.c \
 	$(top_srcdir)/lib/libvarnish/vtim.c \
-	$(top_srcdir)/lib/libvarnish/flopen.c \
+	$(top_srcdir)/lib/libvarnish/vfl.c \
 	$(top_srcdir)/lib/libvarnishtools/vut.c
 
 varnishhist_CFLAGS = \
diff --git a/bin/varnishlog/Makefile.am b/bin/varnishlog/Makefile.am
index 9747e49..d291d62 100644
--- a/bin/varnishlog/Makefile.am
+++ b/bin/varnishlog/Makefile.am
@@ -12,8 +12,8 @@ varnishlog_SOURCES = \
 	varnishlog_options.c \
 	$(top_srcdir)/lib/libvarnishtools/vut.c \
 	$(top_srcdir)/lib/libvarnish/vas.c \
-	$(top_srcdir)/lib/libvarnish/flopen.c \
 	$(top_srcdir)/lib/libvarnish/version.c \
+	$(top_srcdir)/lib/libvarnish/vfl.c \
 	$(top_srcdir)/lib/libvarnish/vsb.c \
 	$(top_srcdir)/lib/libvarnish/vpf.c \
 	$(top_srcdir)/lib/libvarnish/vtim.c
diff --git a/bin/varnishncsa/Makefile.am b/bin/varnishncsa/Makefile.am
index b51df5f..077b33c 100644
--- a/bin/varnishncsa/Makefile.am
+++ b/bin/varnishncsa/Makefile.am
@@ -14,8 +14,8 @@ varnishncsa_SOURCES = \
 	base64.c \
 	$(top_srcdir)/lib/libvarnishtools/vut.c \
 	$(top_srcdir)/lib/libvarnish/vas.c \
-	$(top_srcdir)/lib/libvarnish/flopen.c \
 	$(top_srcdir)/lib/libvarnish/version.c \
+	$(top_srcdir)/lib/libvarnish/vfl.c \
 	$(top_srcdir)/lib/libvarnish/vpf.c \
 	$(top_srcdir)/lib/libvarnish/vtim.c \
 	$(top_srcdir)/lib/libvarnish/vsb.c
diff --git a/bin/varnishtop/Makefile.am b/bin/varnishtop/Makefile.am
index f176728..afed891 100644
--- a/bin/varnishtop/Makefile.am
+++ b/bin/varnishtop/Makefile.am
@@ -11,8 +11,8 @@ varnishtop_SOURCES = varnishtop.c \
 	varnishtop_options.c \
 	$(top_srcdir)/lib/libvarnishtools/vut.c \
 	$(top_srcdir)/lib/libvarnish/vas.c \
-	$(top_srcdir)/lib/libvarnish/flopen.c \
 	$(top_srcdir)/lib/libvarnish/version.c \
+	$(top_srcdir)/lib/libvarnish/vfl.c \
 	$(top_srcdir)/lib/libvarnish/vpf.c \
 	$(top_srcdir)/lib/libvarnish/vtim.c \
 	$(top_srcdir)/lib/libvarnish/vsb.c
diff --git a/include/Makefile.am b/include/Makefile.am
index 816e416..d1c2670 100644
--- a/include/Makefile.am
+++ b/include/Makefile.am
@@ -66,7 +66,7 @@ nobase_noinst_HEADERS = \
 	binary_heap.h \
 	compat/daemon.h \
 	compat/execinfo.h \
-	flopen.h \
+	vfl.h \
 	libvcc.h \
 	vcli_common.h \
 	vcli_priv.h \
diff --git a/include/flopen.h b/include/flopen.h
deleted file mode 100644
index 9833efa..0000000
--- a/include/flopen.h
+++ /dev/null
@@ -1,37 +0,0 @@
-/*-
- * Copyright (c) 2007 Dag-Erling Co?dan Sm?rgrav
- * All rights reserved.
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
- * 1. Redistributions of source code must retain the above copyright
- *    notice, this list of conditions and the following disclaimer
- *    in this position and unchanged.
- * 2. Redistributions in binary form must reproduce the above copyright
- *    notice, this list of conditions and the following disclaimer in the
- *    documentation and/or other materials provided with the distribution.
- *
- * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
- * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
- * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
- * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
- * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
- * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
- * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
- * SUCH DAMAGE.
- *
- * Derived from:
- * $FreeBSD: src/lib/libutil/libutil.h,v 1.44 2007/05/10 15:01:42 des Exp $
- */
-
-#ifndef FLOPEN_H_INCLUDED
-#define FLOPEN_H_INCLUDED
-
-int flopen(const char *, int, ...);
-int fltest(int fd, pid_t *pid);
-
-#endif
diff --git a/include/vfl.h b/include/vfl.h
new file mode 100644
index 0000000..a07b21f
--- /dev/null
+++ b/include/vfl.h
@@ -0,0 +1,37 @@
+/*-
+ * Copyright (c) 2007 Dag-Erling Co?dan Sm?rgrav
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer
+ *    in this position and unchanged.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ *
+ * Derived from:
+ * $FreeBSD: src/lib/libutil/libutil.h,v 1.44 2007/05/10 15:01:42 des Exp $
+ */
+
+#ifndef VFL_H_INCLUDED
+#define VFL_H_INCLUDED
+
+int VFL_Open(const char *, int, ...);
+int VFL_Test(int fd, pid_t *pid);
+
+#endif
diff --git a/lib/libvarnish/Makefile.am b/lib/libvarnish/Makefile.am
index bdda37d..6d6665a 100644
--- a/lib/libvarnish/Makefile.am
+++ b/lib/libvarnish/Makefile.am
@@ -17,7 +17,7 @@ libvarnish_la_SOURCES = \
 	cli_auth.c \
 	cli_common.c \
 	cli_serve.c \
-	flopen.c \
+	vfl.c \
 	vnum.c \
 	vtim.c \
 	vtcp.c \
diff --git a/lib/libvarnish/flopen.c b/lib/libvarnish/flopen.c
deleted file mode 100644
index 483d8d6..0000000
--- a/lib/libvarnish/flopen.c
+++ /dev/null
@@ -1,133 +0,0 @@
-/*-
- * Copyright (c) 2007 Dag-Erling Co?dan Sm?rgrav
- * All rights reserved.
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
- * 1. Redistributions of source code must retain the above copyright
- *    notice, this list of conditions and the following disclaimer
- *    in this position and unchanged.
- * 2. Redistributions in binary form must reproduce the above copyright
- *    notice, this list of conditions and the following disclaimer in the
- *    documentation and/or other materials provided with the distribution.
- *
- * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
- * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
- * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
- * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
- * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
- * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
- * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
- * SUCH DAMAGE.
- * Derived from:
- * $FreeBSD: head/lib/libutil/flopen.c 184094 2008-10-20 18:11:30Z des $
- */
-
-#include "config.h"
-
-#include <sys/stat.h>
-
-#include <errno.h>
-#include <fcntl.h>
-#include <stdarg.h>
-#include <string.h>
-#include <unistd.h>
-
-#include "flopen.h"
-
-int
-flopen(const char *path, int flags, ...)
-{
-	int fd, operation, serrno, trunc;
-	struct flock lock;
-	struct stat sb, fsb;
-	mode_t mode;
-
-#ifdef O_EXLOCK
-	flags &= ~O_EXLOCK;
-#endif
-
-	mode = 0;
-	if (flags & O_CREAT) {
-		va_list ap;
-
-		va_start(ap, flags);
-		mode = (mode_t)va_arg(ap, int); /* mode_t promoted to int */
-		va_end(ap);
-	}
-
-	memset(&lock, 0, sizeof lock);
-	lock.l_type = ((flags & O_ACCMODE) == O_RDONLY) ? F_RDLCK : F_WRLCK;
-	lock.l_whence = SEEK_SET;
-	operation = (flags & O_NONBLOCK) ? F_SETLK : F_SETLKW;
-
-	trunc = (flags & O_TRUNC);
-	flags &= ~O_TRUNC;
-
-	for (;;) {
-		if ((fd = open(path, flags, mode)) == -1)
-			/* non-existent or no access */
-			return (-1);
-		if (fcntl(fd, operation, &lock) == -1) {
-			/* unsupported or interrupted */
-			serrno = errno;
-			(void)close(fd);
-			errno = serrno;
-			return (-1);
-		}
-		if (stat(path, &sb) == -1) {
-			/* disappeared from under our feet */
-			(void)close(fd);
-			continue;
-		}
-		if (fstat(fd, &fsb) == -1) {
-			/* can't happen [tm] */
-			serrno = errno;
-			(void)close(fd);
-			errno = serrno;
-			return (-1);
-		}
-		if (sb.st_dev != fsb.st_dev ||
-		    sb.st_ino != fsb.st_ino) {
-			/* changed under our feet */
-			(void)close(fd);
-			continue;
-		}
-		if (trunc && ftruncate(fd, 0) != 0) {
-			/* can't happen [tm] */
-			serrno = errno;
-			(void)close(fd);
-			errno = serrno;
-			return (-1);
-		}
-		return (fd);
-	}
-}
-
-/* Tests if the given fd is locked through flopen
- * If pid is non-NULL, stores the pid of the process holding the lock there
- * Returns 1 if the file is locked
- * Returns 0 if the file is unlocked
- * Returns -1 on error (and errno)
- */
-int
-fltest(int fd, pid_t *pid)
-{
-	struct flock lock;
-
-	memset(&lock, 0, sizeof lock);
-	lock.l_type = F_WRLCK;
-	lock.l_whence = SEEK_SET;
-
-	if (fcntl(fd, F_GETLK, &lock) == -1)
-		return (-1);
-	if (lock.l_type == F_UNLCK)
-		return (0);
-	if (pid != NULL)
-		*pid = lock.l_pid;
-	return (1);
-}
diff --git a/lib/libvarnish/vfl.c b/lib/libvarnish/vfl.c
new file mode 100644
index 0000000..f7302ec
--- /dev/null
+++ b/lib/libvarnish/vfl.c
@@ -0,0 +1,133 @@
+/*-
+ * Copyright (c) 2007 Dag-Erling Co?dan Sm?rgrav
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer
+ *    in this position and unchanged.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ * Derived from:
+ * $FreeBSD: head/lib/libutil/flopen.c 184094 2008-10-20 18:11:30Z des $
+ */
+
+#include "config.h"
+
+#include <sys/stat.h>
+
+#include <errno.h>
+#include <fcntl.h>
+#include <stdarg.h>
+#include <string.h>
+#include <unistd.h>
+
+#include "vfl.h"
+
+int
+VFL_Open(const char *path, int flags, ...)
+{
+	int fd, operation, serrno, trunc;
+	struct flock lock;
+	struct stat sb, fsb;
+	mode_t mode;
+
+#ifdef O_EXLOCK
+	flags &= ~O_EXLOCK;
+#endif
+
+	mode = 0;
+	if (flags & O_CREAT) {
+		va_list ap;
+
+		va_start(ap, flags);
+		mode = (mode_t)va_arg(ap, int); /* mode_t promoted to int */
+		va_end(ap);
+	}
+
+	memset(&lock, 0, sizeof lock);
+	lock.l_type = ((flags & O_ACCMODE) == O_RDONLY) ? F_RDLCK : F_WRLCK;
+	lock.l_whence = SEEK_SET;
+	operation = (flags & O_NONBLOCK) ? F_SETLK : F_SETLKW;
+
+	trunc = (flags & O_TRUNC);
+	flags &= ~O_TRUNC;
+
+	for (;;) {
+		if ((fd = open(path, flags, mode)) == -1)
+			/* non-existent or no access */
+			return (-1);
+		if (fcntl(fd, operation, &lock) == -1) {
+			/* unsupported or interrupted */
+			serrno = errno;
+			(void)close(fd);
+			errno = serrno;
+			return (-1);
+		}
+		if (stat(path, &sb) == -1) {
+			/* disappeared from under our feet */
+			(void)close(fd);
+			continue;
+		}
+		if (fstat(fd, &fsb) == -1) {
+			/* can't happen [tm] */
+			serrno = errno;
+			(void)close(fd);
+			errno = serrno;
+			return (-1);
+		}
+		if (sb.st_dev != fsb.st_dev ||
+		    sb.st_ino != fsb.st_ino) {
+			/* changed under our feet */
+			(void)close(fd);
+			continue;
+		}
+		if (trunc && ftruncate(fd, 0) != 0) {
+			/* can't happen [tm] */
+			serrno = errno;
+			(void)close(fd);
+			errno = serrno;
+			return (-1);
+		}
+		return (fd);
+	}
+}
+
+/* Tests if the given fd is locked through flopen
+ * If pid is non-NULL, stores the pid of the process holding the lock there
+ * Returns 1 if the file is locked
+ * Returns 0 if the file is unlocked
+ * Returns -1 on error (and errno)
+ */
+int
+VFL_Test(int fd, pid_t *pid)
+{
+	struct flock lock;
+
+	memset(&lock, 0, sizeof lock);
+	lock.l_type = F_WRLCK;
+	lock.l_whence = SEEK_SET;
+
+	if (fcntl(fd, F_GETLK, &lock) == -1)
+		return (-1);
+	if (lock.l_type == F_UNLCK)
+		return (0);
+	if (pid != NULL)
+		*pid = lock.l_pid;
+	return (1);
+}
diff --git a/lib/libvarnish/vpf.c b/lib/libvarnish/vpf.c
index 7875c15..7670d87 100644
--- a/lib/libvarnish/vpf.c
+++ b/lib/libvarnish/vpf.c
@@ -40,7 +40,7 @@
 #include <string.h>
 #include <unistd.h>
 
-#include "flopen.h"
+#include "vfl.h"
 #include "vas.h"
 #include "vpf.h"
 
@@ -128,7 +128,7 @@ VPF_Open(const char *path, mode_t mode, pid_t *pidptr)
 	 * PID file will be truncated again in VPF_Write(), so
 	 * VPF_Write() can be called multiple times.
 	 */
-	fd = flopen(pfh->pf_path,
+	fd = VFL_Open(pfh->pf_path,
 	    O_WRONLY | O_CREAT | O_TRUNC | O_NONBLOCK, mode);
 	if (fd == -1) {
 		if (errno == EWOULDBLOCK && pidptr != NULL) {



From phk at FreeBSD.org  Tue May 10 09:11:06 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 10 May 2016 11:11:06 +0200
Subject: [master] f0e1961 Get rid of the separate opt2rst programs by
 integrating the functionaly into VUT_Init().
Message-ID: <E1b03h4-0004qy-Sd@project.varnish-software.com>


commit f0e19615e91dea011c39e2f0df83bc03d15f79cb
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue May 10 09:10:00 2016 +0000

    Get rid of the separate opt2rst programs by integrating the
    functionaly into VUT_Init().

diff --git a/doc/sphinx/Makefile.am b/doc/sphinx/Makefile.am
index 14127df..0b922e6 100644
--- a/doc/sphinx/Makefile.am
+++ b/doc/sphinx/Makefile.am
@@ -132,31 +132,31 @@ BUILT_SOURCES += include/counters.rst
 
 # XXX add varnishstat here when it's been _opt2rst'ed
 
-include/varnishncsa_options.rst: $(top_builddir)/bin/varnishncsa/varnishncsa_opt2rst
-	$(top_builddir)/bin/varnishncsa/varnishncsa_opt2rst options > $@
-include/varnishncsa_synopsis.rst: $(top_builddir)/bin/varnishncsa/varnishncsa_opt2rst
-	$(top_builddir)/bin/varnishncsa/varnishncsa_opt2rst synopsis > $@
+include/varnishncsa_options.rst: $(top_builddir)/bin/varnishncsa/varnishncsa
+	$(top_builddir)/bin/varnishncsa/varnishncsa --options > $@
+include/varnishncsa_synopsis.rst: $(top_builddir)/bin/varnishncsa/varnishncsa
+	$(top_builddir)/bin/varnishncsa/varnishncsa --synopsis > $@
 BUILT_SOURCES += include/varnishncsa_options.rst \
 	include/varnishncsa_synopsis.rst
 
-include/varnishlog_options.rst: $(top_builddir)/bin/varnishlog/varnishlog_opt2rst
-	$(top_builddir)/bin/varnishlog/varnishlog_opt2rst options > $@
-include/varnishlog_synopsis.rst: $(top_builddir)/bin/varnishlog/varnishlog_opt2rst
-	$(top_builddir)/bin/varnishlog/varnishlog_opt2rst synopsis > $@
+include/varnishlog_options.rst: $(top_builddir)/bin/varnishlog/varnishlog
+	$(top_builddir)/bin/varnishlog/varnishlog --options > $@
+include/varnishlog_synopsis.rst: $(top_builddir)/bin/varnishlog/varnishlog
+	$(top_builddir)/bin/varnishlog/varnishlog --synopsis > $@
 BUILT_SOURCES += include/varnishlog_options.rst \
 	include/varnishlog_synopsis.rst
 
-include/varnishtop_options.rst: $(top_builddir)/bin/varnishtop/varnishtop_opt2rst
-	$(top_builddir)/bin/varnishtop/varnishtop_opt2rst options > $@
-include/varnishtop_synopsis.rst: $(top_builddir)/bin/varnishtop/varnishtop_opt2rst
-	$(top_builddir)/bin/varnishtop/varnishtop_opt2rst synopsis > $@
+include/varnishtop_options.rst: $(top_builddir)/bin/varnishtop/varnishtop
+	$(top_builddir)/bin/varnishtop/varnishtop --options > $@
+include/varnishtop_synopsis.rst: $(top_builddir)/bin/varnishtop/varnishtop
+	$(top_builddir)/bin/varnishtop/varnishtop --synopsis > $@
 BUILT_SOURCES += include/varnishtop_options.rst \
 	 include/varnishtop_synopsis.rst
 
-include/varnishhist_options.rst: $(top_builddir)/bin/varnishhist/varnishhist_opt2rst
-	$(top_builddir)/bin/varnishhist/varnishhist_opt2rst options > $@
-include/varnishhist_synopsis.rst: $(top_builddir)/bin/varnishhist/varnishhist_opt2rst
-	$(top_builddir)/bin/varnishhist/varnishhist_opt2rst synopsis > $@
+include/varnishhist_options.rst: $(top_builddir)/bin/varnishhist/varnishhist
+	$(top_builddir)/bin/varnishhist/varnishhist --options > $@
+include/varnishhist_synopsis.rst: $(top_builddir)/bin/varnishhist/varnishhist
+	$(top_builddir)/bin/varnishhist/varnishhist --synopsis > $@
 BUILT_SOURCES += include/varnishhist_options.rst \
 	 include/varnishhist_synopsis.rst
 
diff --git a/doc/sphinx/Makefile.phk b/doc/sphinx/Makefile.phk
index 35bea90..57689bd 100644
--- a/doc/sphinx/Makefile.phk
+++ b/doc/sphinx/Makefile.phk
@@ -207,29 +207,29 @@ include/params.rst: ../../bin/varnishd/varnishd
 # only sphinx needs the opt2rst stuff anyway
 %_opt2rst: ../../bin/%/%_options.h %_options.c
 
-include/varnishncsa_options.rst: varnishncsa_opt2rst
-	../../bin/varnishncsa/varnishncsa_opt2rst options > $@
+include/varnishncsa_options.rst: varnishncsa
+	../../bin/varnishncsa/varnishncsa --options > $@
 
-include/varnishncsa_synopsis.rst: varnishncsa_opt2rst
-	../../bin/varnishncsa/varnishncsa_opt2rst synopsis > $@
+include/varnishncsa_synopsis.rst: varnishncsa
+	../../bin/varnishncsa/varnishncsa --synopsis > $@
 
-include/varnishlog_options.rst: varnishlog_opt2rst
-	../../bin/varnishlog/varnishlog_opt2rst options > $@
+include/varnishlog_options.rst: varnishlog
+	../../bin/varnishlog/varnishlog --options > $@
 
-include/varnishlog_synopsis.rst: varnishlog_opt2rst
-	../../bin/varnishlog/varnishlog_opt2rst synopsis > $@
+include/varnishlog_synopsis.rst: varnishlog
+	../../bin/varnishlog/varnishlog --synopsis > $@
 
-include/varnishtop_options.rst: varnishtop_opt2rst
-	../../bin/varnishtop/varnishtop_opt2rst options > $@
+include/varnishtop_options.rst: varnishtop
+	../../bin/varnishtop/varnishtop --options > $@
 
-include/varnishtop_synopsis.rst: varnishtop_opt2rst
-	../../bin/varnishtop/varnishtop_opt2rst synopsis > $@
+include/varnishtop_synopsis.rst: varnishtop
+	../../bin/varnishtop/varnishtop --synopsis > $@
 
-include/varnishhist_options.rst: varnishhist_opt2rst
-	./varnishhist_opt2rst options > $@
+include/varnishhist_options.rst: varnishhist
+	./varnishhist --options > $@
 
-include/varnishhist_synopsis.rst: varnishhist_opt2rst
-	./varnishhist_opt2rst synopsis > $@
+include/varnishhist_synopsis.rst: varnishhist
+	./varnishhist --synopsis > $@
 
 reference/vmod_std.generated.rst: reference ../../lib/libvmod_std/vmod_std.rst
 	cp ../../lib/libvmod_std/vmod_std.rst $@



From phk at FreeBSD.org  Tue May 10 10:04:08 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 10 May 2016 12:04:08 +0200
Subject: [master] 32670d7 Somehow I lost half the changed to get rid of
 opt2rst programs.
Message-ID: <E1b04WO-0007xX-8V@project.varnish-software.com>


commit 32670d7d50b21dfa842f17ef1b52adeae3434a8c
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue May 10 10:02:40 2016 +0000

    Somehow I lost half the changed to get rid of opt2rst programs.
    
    Here it is.

diff --git a/.gitignore b/.gitignore
index fe1e27e..1f5e586 100644
--- a/.gitignore
+++ b/.gitignore
@@ -83,7 +83,6 @@ cscope.*out
 /man/*.7
 /doc/sphinx/include
 /doc/sphinx/*/*generated.rst
-/bin/varnish*/varnish*_opt2rst
 /bin/varnishadm/varnishadm
 /bin/varnishd/varnishd
 /bin/varnishhist/varnishhist
diff --git a/bin/varnishhist/Makefile.am b/bin/varnishhist/Makefile.am
index e4d5e97..c8d2820 100644
--- a/bin/varnishhist/Makefile.am
+++ b/bin/varnishhist/Makefile.am
@@ -25,9 +25,3 @@ varnishhist_LDADD = \
 	-lm \
 	@SAN_LDFLAGS@ \
 	@CURSES_LIB@ ${RT_LIBS} ${PTHREAD_LIBS}
-
-noinst_PROGRAMS = varnishhist_opt2rst
-varnishhist_opt2rst_SOURCES = \
-	varnishhist_options.h \
-	varnishhist_options.c \
-	$(top_srcdir)/lib/libvarnishtools/opt2rst.c
diff --git a/bin/varnishhist/varnishhist.c b/bin/varnishhist/varnishhist.c
index f279dc3..31a40af 100644
--- a/bin/varnishhist/varnishhist.c
+++ b/bin/varnishhist/varnishhist.c
@@ -385,9 +385,7 @@ main(int argc, char **argv)
 	struct profile cli_p = {0};
 	cli_p.name = 0;
 
-	VUT_Init(progname);
-	if (0)
-		(void)usage;
+	VUT_Init(progname, argc, argv);
 
 	/* only client requests */
 	assert(VUT_Arg('c', NULL));
diff --git a/bin/varnishlog/Makefile.am b/bin/varnishlog/Makefile.am
index d291d62..80c7ed2 100644
--- a/bin/varnishlog/Makefile.am
+++ b/bin/varnishlog/Makefile.am
@@ -26,9 +26,3 @@ varnishlog_LDADD = \
 	$(top_builddir)/lib/libvarnishapi/libvarnishapi.la \
 	@SAN_LDFLAGS@ \
 	${RT_LIBS} ${LIBM} ${PTHREAD_LIBS}
-
-noinst_PROGRAMS = varnishlog_opt2rst
-varnishlog_opt2rst_SOURCES = \
-	varnishlog_options.h \
-	varnishlog_options.c \
-	$(top_srcdir)/lib/libvarnishtools/opt2rst.c
diff --git a/bin/varnishlog/varnishlog.c b/bin/varnishlog/varnishlog.c
index bc4a915..d8be34f 100644
--- a/bin/varnishlog/varnishlog.c
+++ b/bin/varnishlog/varnishlog.c
@@ -121,8 +121,8 @@ main(int argc, char * const *argv)
 {
 	int opt;
 
+	VUT_Init(progname, argc, argv);
 	memset(&LOG, 0, sizeof LOG);
-	VUT_Init(progname);
 
 	while ((opt = getopt(argc, argv, vopt_optstring)) != -1) {
 		switch (opt) {
diff --git a/bin/varnishncsa/Makefile.am b/bin/varnishncsa/Makefile.am
index 077b33c..dcdd7b2 100644
--- a/bin/varnishncsa/Makefile.am
+++ b/bin/varnishncsa/Makefile.am
@@ -28,9 +28,3 @@ varnishncsa_LDADD = \
 	$(top_builddir)/lib/libvarnishapi/libvarnishapi.la \
 	@SAN_LDFLAGS@ \
 	${RT_LIBS} ${LIBM}
-
-noinst_PROGRAMS = varnishncsa_opt2rst
-varnishncsa_opt2rst_SOURCES = \
-	varnishncsa_options.h \
-	varnishncsa_options.c \
-	$(top_srcdir)/lib/libvarnishtools/opt2rst.c
diff --git a/bin/varnishncsa/varnishncsa.c b/bin/varnishncsa/varnishncsa.c
index 5827458..13e28d9 100644
--- a/bin/varnishncsa/varnishncsa.c
+++ b/bin/varnishncsa/varnishncsa.c
@@ -1126,6 +1126,7 @@ main(int argc, char * const *argv)
 	signed char opt;
 	char *format = NULL;
 
+	VUT_Init(progname, argc, argv);
 	memset(&CTX, 0, sizeof CTX);
 	VTAILQ_INIT(&CTX.format);
 	VTAILQ_INIT(&CTX.watch_vcl_log);
@@ -1135,7 +1136,6 @@ main(int argc, char * const *argv)
 	CTX.vsb = VSB_new_auto();
 	AN(CTX.vsb);
 	VB64_init();
-	VUT_Init(progname);
 
 	while ((opt = getopt(argc, argv, vopt_optstring)) != -1) {
 		switch (opt) {
diff --git a/bin/varnishtop/Makefile.am b/bin/varnishtop/Makefile.am
index afed891..4757839 100644
--- a/bin/varnishtop/Makefile.am
+++ b/bin/varnishtop/Makefile.am
@@ -26,9 +26,3 @@ varnishtop_LDADD = \
 	$(top_builddir)/lib/libvarnishapi/libvarnishapi.la \
 	@SAN_LDFLAGS@ \
 	@CURSES_LIB@ ${RT_LIBS} ${LIBM} ${PTHREAD_LIBS}
-
-noinst_PROGRAMS = varnishtop_opt2rst
-varnishtop_opt2rst_SOURCES = \
-	varnishtop_options.h \
-	varnishtop_options.c \
-	$(top_srcdir)/lib/libvarnishtools/opt2rst.c
diff --git a/bin/varnishtop/varnishtop.c b/bin/varnishtop/varnishtop.c
index 9c555cc..e91f825 100644
--- a/bin/varnishtop/varnishtop.c
+++ b/bin/varnishtop/varnishtop.c
@@ -329,7 +329,7 @@ main(int argc, char **argv)
 	int o, once = 0;
 	pthread_t thr;
 
-	VUT_Init(progname);
+	VUT_Init(progname, argc, argv);
 
 	while ((o = getopt(argc, argv, vopt_optstring)) != -1) {
 		switch (o) {
diff --git a/include/vut.h b/include/vut.h
index 241aa94..fe99095 100644
--- a/include/vut.h
+++ b/include/vut.h
@@ -74,7 +74,7 @@ int VUT_Arg(int opt, const char *arg);
 
 void VUT_Setup(void);
 
-void VUT_Init(const char *progname);
+void VUT_Init(const char *progname, int argc, char * const *argv);
 
 void VUT_Fini(void);
 
diff --git a/lib/libvarnishtools/Makefile.am b/lib/libvarnishtools/Makefile.am
index 1a14212..5fffe27 100644
--- a/lib/libvarnishtools/Makefile.am
+++ b/lib/libvarnishtools/Makefile.am
@@ -1,5 +1,4 @@
 #
 
 EXTRA_DIST = \
-	vut.c \
-	opt2rst.c
+	vut.c
diff --git a/lib/libvarnishtools/opt2rst.c b/lib/libvarnishtools/opt2rst.c
deleted file mode 100644
index efaf10c..0000000
--- a/lib/libvarnishtools/opt2rst.c
+++ /dev/null
@@ -1,104 +0,0 @@
-/*-
- * Copyright (c) 2006 Verdens Gang AS
- * Copyright (c) 2006-2014 Varnish Software AS
- * All rights reserved.
- *
- * Author: Martin Blix Grydeland <martin at varnish-software.com>
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
- * 1. Redistributions of source code must retain the above copyright
- *    notice, this list of conditions and the following disclaimer.
- * 2. Redistributions in binary form must reproduce the above copyright
- *    notice, this list of conditions and the following disclaimer in the
- *    documentation and/or other materials provided with the distribution.
- *
- * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
- * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED.  IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
- * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
- * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
- * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
- * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
- * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
- * SUCH DAMAGE.
- */
-
-#include <stdlib.h>
-#include <stdio.h>
-#include <string.h>
-#include <ctype.h>
-
-#include "vapi/voptget.h"
-
-static void
-print_nobrackets(const char *s)
-{
-	const char *e;
-
-	/* Remove whitespace */
-	while (isspace(*s))
-		s++;
-	e = s + strlen(s);
-	while (e > s && isspace(e[-1]))
-		e--;
-
-	/* Remove outer layer brackets if present */
-	if (e > s && *s == '[' && e[-1] == ']') {
-		s++;
-		e--;
-	}
-
-	printf("%.*s", (int)(e - s), s);
-}
-
-static void
-print_tabbed(const char *string, int tabs)
-{
-	int i;
-	const char *c;
-
-	for (c = string; *c; c++) {
-		if (c == string || *(c - 1) == '\n')
-			for (i = 0; i < tabs; i++)
-				printf("\t");
-		printf("%c", *c);
-	}
-}
-
-static void
-print_opt(const struct vopt_list *opt)
-{
-	print_nobrackets(opt->synopsis);
-	printf("\n\n");
-	print_tabbed(opt->ldesc, 1);
-	printf("\n\n");
-}
-
-static void
-usage(void)
-{
-	fprintf(stderr, "Usage: opt2rst {synopsis|options}\n");
-	exit(1);
-}
-
-int
-main(int argc, char * const *argv)
-{
-	int i;
-
-	if (argc != 2)
-		usage();
-	if (!strcmp(argv[1], "synopsis"))
-		printf(".. |synopsis| replace:: %s\n", vopt_synopsis);
-	else if (!strcmp(argv[1], "options"))
-		for (i = 0; i < vopt_list_n; i++)
-			print_opt(&vopt_list[i]);
-	else
-		usage();
-
-	return (0);
-}
diff --git a/lib/libvarnishtools/vut.c b/lib/libvarnishtools/vut.c
index f534266..e62c128 100644
--- a/lib/libvarnishtools/vut.c
+++ b/lib/libvarnishtools/vut.c
@@ -31,6 +31,7 @@
 
 #include "config.h"
 
+#include <ctype.h>
 #include <stdint.h>
 #include <stdarg.h>
 #include <stdlib.h>
@@ -56,6 +57,9 @@
 
 struct VUT VUT;
 
+static int vut_synopsis(void);
+static int vut_options(void);
+
 static void
 vut_vpf_remove(void)
 {
@@ -201,8 +205,14 @@ VUT_Arg(int opt, const char *arg)
 }
 
 void
-VUT_Init(const char *progname)
+VUT_Init(const char *progname, int argc, char * const *argv)
 {
+
+	if (argc == 2 && !strcmp(argv[1], "--synopsis"))
+		exit(vut_synopsis());
+	if (argc == 2 && !strcmp(argv[1], "--options"))
+		exit(vut_options());
+
 	VUT.progname = progname;
 	REPLACE(VUT.name, "");
 	VUT.g_arg = VSL_g_vxid;
@@ -421,3 +431,69 @@ VUT_Main(void)
 
 	return (i);
 }
+
+/**********************************************************************/
+
+
+#include "vapi/voptget.h"
+
+static void
+print_nobrackets(const char *s)
+{
+	const char *e;
+
+	/* Remove whitespace */
+	while (isspace(*s))
+		s++;
+	e = s + strlen(s);
+	while (e > s && isspace(e[-1]))
+		e--;
+
+	/* Remove outer layer brackets if present */
+	if (e > s && *s == '[' && e[-1] == ']') {
+		s++;
+		e--;
+	}
+
+	printf("%.*s", (int)(e - s), s);
+}
+
+static void
+print_tabbed(const char *string, int tabs)
+{
+	int i;
+	const char *c;
+
+	for (c = string; *c; c++) {
+		if (c == string || *(c - 1) == '\n')
+			for (i = 0; i < tabs; i++)
+				printf("\t");
+		printf("%c", *c);
+	}
+}
+
+static void
+print_opt(const struct vopt_list *opt)
+{
+	print_nobrackets(opt->synopsis);
+	printf("\n\n");
+	print_tabbed(opt->ldesc, 1);
+	printf("\n\n");
+}
+
+static int
+vut_synopsis(void)
+{
+	printf(".. |synopsis| replace:: %s\n", vopt_synopsis);
+	return (0);
+}
+
+static int
+vut_options(void)
+{
+	int i;
+
+	for (i = 0; i < vopt_list_n; i++)
+		print_opt(&vopt_list[i]);
+	return (0);
+}



From phk at FreeBSD.org  Tue May 10 10:52:06 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 10 May 2016 12:52:06 +0200
Subject: [master] 42e2118 Pack VUT/VOPT's specification into a single
 structure for forward compatibility and namespace-hygiene.
Message-ID: <E1b05Go-0002EF-Re@project.varnish-software.com>


commit 42e21180f10858f9b87f30ca12413c1feb34b380
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue May 10 10:37:27 2016 +0000

    Pack VUT/VOPT's specification into a single structure for forward
    compatibility and namespace-hygiene.

diff --git a/bin/varnishhist/varnishhist.c b/bin/varnishhist/varnishhist.c
index 31a40af..d8a9905 100644
--- a/bin/varnishhist/varnishhist.c
+++ b/bin/varnishhist/varnishhist.c
@@ -369,7 +369,7 @@ usage(int status)
 
 	fprintf(stderr, "Usage: %s <options>\n\n", progname);
 	fprintf(stderr, "Options:\n");
-	for (opt = vopt_usage; *opt != NULL; opt +=2)
+	for (opt = vopt_spec.vopt_usage; *opt != NULL; opt +=2)
 		fprintf(stderr, " %-25s %s\n", *opt, *(opt + 1));
 	exit(status);
 }
@@ -385,11 +385,11 @@ main(int argc, char **argv)
 	struct profile cli_p = {0};
 	cli_p.name = 0;
 
-	VUT_Init(progname, argc, argv);
+	VUT_Init(progname, argc, argv, &vopt_spec);
 
 	/* only client requests */
 	assert(VUT_Arg('c', NULL));
-	while ((i = getopt(argc, argv, vopt_optstring)) != -1) {
+	while ((i = getopt(argc, argv, vopt_spec.vopt_optstring)) != -1) {
 		switch (i) {
 		case 'P':
 			colon = strchr(optarg, ':');
diff --git a/bin/varnishlog/varnishlog.c b/bin/varnishlog/varnishlog.c
index d8be34f..9b5e909 100644
--- a/bin/varnishlog/varnishlog.c
+++ b/bin/varnishlog/varnishlog.c
@@ -68,7 +68,7 @@ usage(int status)
 	const char **opt;
 	fprintf(stderr, "Usage: %s <options>\n\n", progname);
 	fprintf(stderr, "Options:\n");
-	for (opt = vopt_usage; *opt != NULL; opt += 2)
+	for (opt = vopt_spec.vopt_usage; *opt != NULL; opt += 2)
 		fprintf(stderr, "  %-25s %s\n", *opt, *(opt + 1));
 	exit(status);
 }
@@ -121,10 +121,10 @@ main(int argc, char * const *argv)
 {
 	int opt;
 
-	VUT_Init(progname, argc, argv);
+	VUT_Init(progname, argc, argv, &vopt_spec);
 	memset(&LOG, 0, sizeof LOG);
 
-	while ((opt = getopt(argc, argv, vopt_optstring)) != -1) {
+	while ((opt = getopt(argc, argv, vopt_spec.vopt_optstring)) != -1) {
 		switch (opt) {
 		case 'a':
 			/* Append to file */
diff --git a/bin/varnishncsa/varnishncsa.c b/bin/varnishncsa/varnishncsa.c
index 13e28d9..5b54ece 100644
--- a/bin/varnishncsa/varnishncsa.c
+++ b/bin/varnishncsa/varnishncsa.c
@@ -166,7 +166,7 @@ usage(int status)
 
 	fprintf(stderr, "Usage: %s <options>\n\n", progname);
 	fprintf(stderr, "Options:\n");
-	for (opt = vopt_usage; *opt != NULL; opt += 2)
+	for (opt = vopt_spec.vopt_usage; *opt != NULL; opt += 2)
 		fprintf(stderr, " %-25s %s\n", *opt, *(opt + 1));
 	exit(status);
 }
@@ -1126,7 +1126,7 @@ main(int argc, char * const *argv)
 	signed char opt;
 	char *format = NULL;
 
-	VUT_Init(progname, argc, argv);
+	VUT_Init(progname, argc, argv, &vopt_spec);
 	memset(&CTX, 0, sizeof CTX);
 	VTAILQ_INIT(&CTX.format);
 	VTAILQ_INIT(&CTX.watch_vcl_log);
@@ -1137,7 +1137,7 @@ main(int argc, char * const *argv)
 	AN(CTX.vsb);
 	VB64_init();
 
-	while ((opt = getopt(argc, argv, vopt_optstring)) != -1) {
+	while ((opt = getopt(argc, argv, vopt_spec.vopt_optstring)) != -1) {
 		switch (opt) {
 		case 'a':
 			/* Append to file */
diff --git a/bin/varnishtop/varnishtop.c b/bin/varnishtop/varnishtop.c
index e91f825..93d61b9 100644
--- a/bin/varnishtop/varnishtop.c
+++ b/bin/varnishtop/varnishtop.c
@@ -318,7 +318,7 @@ usage(int status)
 
 	fprintf(stderr, "Usage: %s <options>\n\n", progname);
 	fprintf(stderr, "Options:\n");
-	for (opt = vopt_usage; *opt != NULL; opt +=2)
+	for (opt = vopt_spec.vopt_usage; *opt != NULL; opt +=2)
 		fprintf(stderr, " %-25s %s\n", *opt, *(opt + 1));
 	exit(status);
 }
@@ -329,9 +329,9 @@ main(int argc, char **argv)
 	int o, once = 0;
 	pthread_t thr;
 
-	VUT_Init(progname, argc, argv);
+	VUT_Init(progname, argc, argv, &vopt_spec);
 
-	while ((o = getopt(argc, argv, vopt_optstring)) != -1) {
+	while ((o = getopt(argc, argv, vopt_spec.vopt_optstring)) != -1) {
 		switch (o) {
 		case '1':
 			AN(VUT_Arg('d', NULL));
diff --git a/include/vapi/voptget.h b/include/vapi/voptget.h
index cf39a48..f01ac94 100644
--- a/include/vapi/voptget.h
+++ b/include/vapi/voptget.h
@@ -35,17 +35,22 @@
  *   l: Long description
  */
 
-extern const char vopt_optstring[];
-extern const char vopt_synopsis[];
-extern const char *vopt_usage[];
 struct vopt_list {
 	const char *option;
 	const char *synopsis;
 	const char *desc;
 	const char *ldesc;
 };
-extern const struct vopt_list vopt_list[];
-extern unsigned vopt_list_n;
+
+struct vopt_spec {
+	const struct vopt_list	*vopt_list;
+	unsigned		vopt_list_n;
+	const char		*vopt_optstring;
+	const char		*vopt_synopsis;
+	const char		**vopt_usage;
+};
+
+extern const struct vopt_spec vopt_spec;
 
 #ifdef VOPT_DEFINITION
 
@@ -54,29 +59,36 @@ extern unsigned vopt_list_n;
 #endif
 
 #define VOPT(o,s,d,l) o
-const char vopt_optstring[] =
+static const char vopt_optstring[] =
 #include VOPT_INC
     ;
 #undef VOPT
 
 #define VOPT(o,s,d,l) " " s
-const char vopt_synopsis[] =
+static const char vopt_synopsis[] =
 #include VOPT_INC
     ;
 #undef VOPT
 
 #define VOPT(o,s,d,l) s, d,
-const char *vopt_usage[] = {
+static const char *vopt_usage[] = {
 #include VOPT_INC
 	NULL, NULL,
 };
 #undef VOPT
 
 #define VOPT(o,s,d,l) { o,s,d,l },
-const struct vopt_list vopt_list[] = {
+static const struct vopt_list vopt_list[] = {
 #include VOPT_INC
 };
 #undef VOPT
-unsigned vopt_list_n = sizeof vopt_list / sizeof vopt_list[0];
+
+const struct vopt_spec vopt_spec = {
+	vopt_list,
+	sizeof vopt_list / sizeof vopt_list[0],
+	vopt_optstring,
+	vopt_synopsis,
+	vopt_usage
+};
 
 #endif /* VOPT_DEFINITION */
diff --git a/include/vut.h b/include/vut.h
index fe99095..d1e6ab3 100644
--- a/include/vut.h
+++ b/include/vut.h
@@ -29,6 +29,8 @@
  * Common functions for the utilities
  */
 
+struct vopt_spec;
+
 typedef int VUT_cb_f(void);
 
 struct VUT {
@@ -74,7 +76,8 @@ int VUT_Arg(int opt, const char *arg);
 
 void VUT_Setup(void);
 
-void VUT_Init(const char *progname, int argc, char * const *argv);
+void VUT_Init(const char *progname, int argc, char * const *argv,
+    const struct vopt_spec *);
 
 void VUT_Fini(void);
 
diff --git a/lib/libvarnishtools/vut.c b/lib/libvarnishtools/vut.c
index e62c128..39c27fb 100644
--- a/lib/libvarnishtools/vut.c
+++ b/lib/libvarnishtools/vut.c
@@ -55,10 +55,12 @@
 
 #include "vut.h"
 
+#include "vapi/voptget.h"
+
 struct VUT VUT;
 
-static int vut_synopsis(void);
-static int vut_options(void);
+static int vut_synopsis(const struct vopt_spec *);
+static int vut_options(const struct vopt_spec *);
 
 static void
 vut_vpf_remove(void)
@@ -205,13 +207,14 @@ VUT_Arg(int opt, const char *arg)
 }
 
 void
-VUT_Init(const char *progname, int argc, char * const *argv)
+VUT_Init(const char *progname, int argc, char * const *argv,
+    const struct vopt_spec *voc)
 {
 
 	if (argc == 2 && !strcmp(argv[1], "--synopsis"))
-		exit(vut_synopsis());
+		exit(vut_synopsis(voc));
 	if (argc == 2 && !strcmp(argv[1], "--options"))
-		exit(vut_options());
+		exit(vut_options(voc));
 
 	VUT.progname = progname;
 	REPLACE(VUT.name, "");
@@ -435,8 +438,6 @@ VUT_Main(void)
 /**********************************************************************/
 
 
-#include "vapi/voptget.h"
-
 static void
 print_nobrackets(const char *s)
 {
@@ -482,18 +483,18 @@ print_opt(const struct vopt_list *opt)
 }
 
 static int
-vut_synopsis(void)
+vut_synopsis(const struct vopt_spec *voc)
 {
-	printf(".. |synopsis| replace:: %s\n", vopt_synopsis);
+	printf(".. |synopsis| replace:: %s\n", voc->vopt_synopsis);
 	return (0);
 }
 
 static int
-vut_options(void)
+vut_options(const struct vopt_spec *voc)
 {
 	int i;
 
-	for (i = 0; i < vopt_list_n; i++)
-		print_opt(&vopt_list[i]);
+	for (i = 0; i < voc->vopt_list_n; i++)
+		print_opt(&voc->vopt_list[i]);
 	return (0);
 }



From phk at FreeBSD.org  Tue May 10 10:52:06 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 10 May 2016 12:52:06 +0200
Subject: [master] 36e8ac2 Move VUT into libvarnishapi
Message-ID: <E1b05Go-0002EL-TZ@project.varnish-software.com>


commit 36e8ac2fc77e8f8b63ea0a41b14d6555ea50433c
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue May 10 10:51:20 2016 +0000

    Move VUT into libvarnishapi

diff --git a/Makefile.inc.phk b/Makefile.inc.phk
index 680bab1..ba9202f 100644
--- a/Makefile.inc.phk
+++ b/Makefile.inc.phk
@@ -28,7 +28,6 @@ INSTALL_BASE ?=       $(TOPDIR)/_install
 
 LIB_VARNISH = -L $(TOPDIR)/lib/libvarnish -lvarnish
 LIB_VARNISHAPI = $(TOPDIR)/lib/libvarnishapi/libvarnishapi.a
-LIB_VARNISHTOOLS = -L $(TOPDIR)/lib/libvarnishtools -lvarnishtools
 LIB_VCC = -L $(TOPDIR)/lib/libvcc -lvcc
 LIB_VGZ = -L $(TOPDIR)/lib/libvgz -lvgz
 LIB_PCRE = -L /usr/local/lib -lpcre
diff --git a/bin/varnishd/Makefile.phk b/bin/varnishd/Makefile.phk
index fb9df6c..54f058a 100644
--- a/bin/varnishd/Makefile.phk
+++ b/bin/varnishd/Makefile.phk
@@ -78,15 +78,17 @@ PROG_SRC += mgt/mgt_vcc.c
 PROG_SRC += mgt/mgt_vcl.c
 PROG_SRC += mgt/mgt_acceptor.c
 
-PROG_SRC += storage/stevedore.c
 PROG_SRC += storage/mgt_stevedore.c
+PROG_SRC += storage/mgt_storage_persistent.c
+PROG_SRC += storage/stevedore.c
 PROG_SRC += storage/stevedore_utils.c
 PROG_SRC += storage/storage_file.c
+PROG_SRC += storage/storage_lru.c
 PROG_SRC += storage/storage_malloc.c
 PROG_SRC += storage/storage_persistent.c
-PROG_SRC += storage/mgt_storage_persistent.c
 PROG_SRC += storage/storage_persistent_silo.c
 PROG_SRC += storage/storage_persistent_subr.c
+PROG_SRC += storage/storage_simple.c
 PROG_SRC += storage/storage_umem.c
 
 PROG_SRC += waiter/cache_waiter.c
diff --git a/bin/varnishhist/Makefile.am b/bin/varnishhist/Makefile.am
index c8d2820..f814bf2 100644
--- a/bin/varnishhist/Makefile.am
+++ b/bin/varnishhist/Makefile.am
@@ -11,10 +11,7 @@ varnishhist_SOURCES = varnishhist.c \
 	varnishhist_options.c \
 	$(top_srcdir)/lib/libvarnish/vas.c \
 	$(top_srcdir)/lib/libvarnish/version.c \
-	$(top_srcdir)/lib/libvarnish/vpf.c \
-	$(top_srcdir)/lib/libvarnish/vtim.c \
-	$(top_srcdir)/lib/libvarnish/vfl.c \
-	$(top_srcdir)/lib/libvarnishtools/vut.c
+	$(top_srcdir)/lib/libvarnish/vtim.c
 
 varnishhist_CFLAGS = \
 	@SAN_CFLAGS@
diff --git a/bin/varnishhist/Makefile.phk b/bin/varnishhist/Makefile.phk
index 77e8c31..c2df17f 100644
--- a/bin/varnishhist/Makefile.phk
+++ b/bin/varnishhist/Makefile.phk
@@ -1,7 +1,6 @@
 PROG_SRC = varnishhist.c
 PROG_SRC += varnishhist_options.c
 
-LD_ADD	+= ${LIB_VARNISHTOOLS}
 LD_ADD	+= ${LIB_VARNISHAPI}
 LD_ADD	+= ${LIB_VARNISH}
 LD_ADD += -lpthread -lncurses -lm ${LIB_PCRE}
diff --git a/bin/varnishlog/Makefile.am b/bin/varnishlog/Makefile.am
index 80c7ed2..92b93d3 100644
--- a/bin/varnishlog/Makefile.am
+++ b/bin/varnishlog/Makefile.am
@@ -10,12 +10,9 @@ varnishlog_SOURCES = \
 	varnishlog.c \
 	varnishlog_options.h \
 	varnishlog_options.c \
-	$(top_srcdir)/lib/libvarnishtools/vut.c \
 	$(top_srcdir)/lib/libvarnish/vas.c \
 	$(top_srcdir)/lib/libvarnish/version.c \
-	$(top_srcdir)/lib/libvarnish/vfl.c \
 	$(top_srcdir)/lib/libvarnish/vsb.c \
-	$(top_srcdir)/lib/libvarnish/vpf.c \
 	$(top_srcdir)/lib/libvarnish/vtim.c
 
 varnishlog_CFLAGS = \
diff --git a/bin/varnishlog/Makefile.phk b/bin/varnishlog/Makefile.phk
index 5f0de52..c024046 100644
--- a/bin/varnishlog/Makefile.phk
+++ b/bin/varnishlog/Makefile.phk
@@ -1,7 +1,6 @@
 PROG_SRC = varnishlog.c
 PROG_SRC = varnishlog_options.c
 
-LD_ADD	+= ${LIB_VARNISHTOOLS}
 LD_ADD	+= ${LIB_VARNISHAPI}
 LD_ADD	+= ${LIB_VARNISH}
 LD_ADD	+= ${LIB_PCRE}
diff --git a/bin/varnishncsa/Makefile.am b/bin/varnishncsa/Makefile.am
index dcdd7b2..bb470f7 100644
--- a/bin/varnishncsa/Makefile.am
+++ b/bin/varnishncsa/Makefile.am
@@ -12,11 +12,8 @@ varnishncsa_SOURCES = \
 	varnishncsa_options.c \
 	base64.h \
 	base64.c \
-	$(top_srcdir)/lib/libvarnishtools/vut.c \
 	$(top_srcdir)/lib/libvarnish/vas.c \
 	$(top_srcdir)/lib/libvarnish/version.c \
-	$(top_srcdir)/lib/libvarnish/vfl.c \
-	$(top_srcdir)/lib/libvarnish/vpf.c \
 	$(top_srcdir)/lib/libvarnish/vtim.c \
 	$(top_srcdir)/lib/libvarnish/vsb.c
 
diff --git a/bin/varnishncsa/Makefile.phk b/bin/varnishncsa/Makefile.phk
index 10fd638..fd73e25 100644
--- a/bin/varnishncsa/Makefile.phk
+++ b/bin/varnishncsa/Makefile.phk
@@ -2,7 +2,6 @@ PROG_SRC += base64.c
 PROG_SRC += varnishncsa.c
 PROG_SRC += varnishncsa_options.c
 
-LD_ADD	+= ${LIB_VARNISHTOOLS}
 LD_ADD	+= ${LIB_VARNISHAPI}
 LD_ADD	+= ${LIB_VARNISH}
 LD_ADD	+= ${LIB_PCRE}
diff --git a/bin/varnishtop/Makefile.am b/bin/varnishtop/Makefile.am
index 4757839..9406d62 100644
--- a/bin/varnishtop/Makefile.am
+++ b/bin/varnishtop/Makefile.am
@@ -9,11 +9,8 @@ bin_PROGRAMS = varnishtop
 varnishtop_SOURCES = varnishtop.c \
 	varnishtop_options.h \
 	varnishtop_options.c \
-	$(top_srcdir)/lib/libvarnishtools/vut.c \
 	$(top_srcdir)/lib/libvarnish/vas.c \
 	$(top_srcdir)/lib/libvarnish/version.c \
-	$(top_srcdir)/lib/libvarnish/vfl.c \
-	$(top_srcdir)/lib/libvarnish/vpf.c \
 	$(top_srcdir)/lib/libvarnish/vtim.c \
 	$(top_srcdir)/lib/libvarnish/vsb.c
 
diff --git a/bin/varnishtop/Makefile.phk b/bin/varnishtop/Makefile.phk
index 7f8d760..12d4de3 100644
--- a/bin/varnishtop/Makefile.phk
+++ b/bin/varnishtop/Makefile.phk
@@ -1,7 +1,6 @@
 PROG_SRC += varnishtop.c
 PROG_SRC += varnishtop_options.c
 
-LD_ADD	+= ${LIB_VARNISHTOOLS}
 LD_ADD	+= ${LIB_VARNISHAPI}
 LD_ADD	+= ${LIB_VARNISH}
 LD_ADD	+= ${LIB_PCRE}
diff --git a/lib/Makefile.am b/lib/Makefile.am
index c0c0e60..9359575 100644
--- a/lib/Makefile.am
+++ b/lib/Makefile.am
@@ -4,7 +4,6 @@ SUBDIRS = \
 	libvarnishcompat \
 	libvarnish \
 	libvarnishapi \
-	libvarnishtools \
 	libvcc \
 	libvgz \
 	libvmod_debug \
diff --git a/lib/Makefile.phk b/lib/Makefile.phk
index 83d0674..86ea172 100644
--- a/lib/Makefile.phk
+++ b/lib/Makefile.phk
@@ -1,4 +1,4 @@
-SUBDIRS = libvcc libvarnish libvarnishapi libvarnishtools libvcc
+SUBDIRS = libvcc libvarnish libvarnishapi libvcc
 SUBDIRS += libvgz
 SUBDIRS += libvmod_debug libvmod_directors libvmod_std
 
diff --git a/lib/libvarnish/Makefile.phk b/lib/libvarnish/Makefile.phk
index 2a0c37a..501a350 100644
--- a/lib/libvarnish/Makefile.phk
+++ b/lib/libvarnish/Makefile.phk
@@ -3,13 +3,13 @@ LIB_SRC += binary_heap.c
 LIB_SRC += cli_auth.c
 LIB_SRC += cli_common.c
 LIB_SRC += cli_serve.c
-LIB_SRC += flopen.c
 LIB_SRC += vas.c
 LIB_SRC += vav.c
 LIB_SRC += vct.c
 LIB_SRC += version.c
 LIB_SRC += vev.c
 LIB_SRC += vfil.c
+LIB_SRC += vfl.c
 LIB_SRC += vin.c
 LIB_SRC += vlu.c
 LIB_SRC += vmb.c
diff --git a/lib/libvarnishapi/Makefile.am b/lib/libvarnishapi/Makefile.am
index 32986c4..03fefe4 100644
--- a/lib/libvarnishapi/Makefile.am
+++ b/lib/libvarnishapi/Makefile.am
@@ -22,8 +22,10 @@ libvarnishapi_la_SOURCES = \
 	../libvarnish/version.c \
 	../libvarnish/cli_common.c \
 	../libvarnish/cli_auth.c \
+	../libvarnish/vfl.c \
 	../libvarnish/vin.c \
 	../libvarnish/vmb.c \
+	../libvarnish/vpf.c \
 	../libvarnish/vre.c \
 	../libvarnish/vsb.c \
 	../libvarnish/vtim.c \
@@ -36,6 +38,7 @@ libvarnishapi_la_SOURCES = \
 	vsl_query.c \
 	vsl.c \
 	vsc.c \
+	vut.c \
 	vxp.c \
 	vxp_parse.c \
 	vxp_lexer.c \
diff --git a/lib/libvarnishapi/Makefile.phk b/lib/libvarnishapi/Makefile.phk
index 2c3e907..3c4eca9 100644
--- a/lib/libvarnishapi/Makefile.phk
+++ b/lib/libvarnishapi/Makefile.phk
@@ -7,6 +7,7 @@ LIB_SRC += vsl_cursor.c
 LIB_SRC += vsl_dispatch.c
 LIB_SRC += vsl_query.c
 LIB_SRC += vsm.c
+LIB_SRC += vut.c
 LIB_SRC += vxp.c
 LIB_SRC += vxp_fixed_token.c
 LIB_SRC += vxp_lexer.c
@@ -16,11 +17,13 @@ LIB_SRC += vxp_parse.c
 # We add more stuff to the SHLIB version to make it self-contained
 SHLIB_SRC = $(LIB_SRC)
 VPATH += ../libvarnish
-SHLIB_SRC += vre.c
-SHLIB_SRC += vsb.c
 SHLIB_SRC += vas.c
 SHLIB_SRC += vav.c
+SHLIB_SRC += vfl.c
 SHLIB_SRC += vin.c
+SHLIB_SRC += vpf.c
+SHLIB_SRC += vre.c
+SHLIB_SRC += vsb.c
 SHLIB_SRC += vtim.c
 
 #LIB_SRC += vsl_glob_test.c
diff --git a/lib/libvarnishapi/libvarnishapi.map b/lib/libvarnishapi/libvarnishapi.map
index cd3940f..3401363 100644
--- a/lib/libvarnishapi/libvarnishapi.map
+++ b/lib/libvarnishapi/libvarnishapi.map
@@ -131,3 +131,15 @@ LIBVARNISHAPI_1.4 {
 	VSLQ_SetCursor;
 	VSM_IsOpen;
 } LIBVARNISHAPI_1.0;
+
+LIBVARNISHAPI_1.5 {
+  global:
+	VUT_Error;
+	VUT_g_Arg;
+	VUT_Arg;
+	VUT_Setup;
+	VUT_Init;
+	VUT_Fini;
+	VUT_Main;
+	VUT;
+} LIBVARNISHAPI_1.0;
diff --git a/lib/libvarnishapi/vut.c b/lib/libvarnishapi/vut.c
new file mode 100644
index 0000000..39c27fb
--- /dev/null
+++ b/lib/libvarnishapi/vut.c
@@ -0,0 +1,500 @@
+/*-
+ * Copyright (c) 2006 Verdens Gang AS
+ * Copyright (c) 2006-2015 Varnish Software AS
+ * All rights reserved.
+ *
+ * Author: Martin Blix Grydeland <martin at varnish-software.com>
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED.  IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ *
+ * Common functions for the utilities
+ */
+
+#include "config.h"
+
+#include <ctype.h>
+#include <stdint.h>
+#include <stdarg.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <stdio.h>
+#include <errno.h>
+#include <string.h>
+#include <signal.h>
+#include <math.h>
+
+#include "compat/daemon.h"
+#include "vdef.h"
+#include "vpf.h"
+#include "vapi/vsm.h"
+#include "vapi/vsl.h"
+#include "vtim.h"
+#include "vas.h"
+#include "miniobj.h"
+#include "vcs.h"
+#include "vnum.h"
+
+#include "vut.h"
+
+#include "vapi/voptget.h"
+
+struct VUT VUT;
+
+static int vut_synopsis(const struct vopt_spec *);
+static int vut_options(const struct vopt_spec *);
+
+static void
+vut_vpf_remove(void)
+{
+	if (VUT.pfh) {
+		VPF_Remove(VUT.pfh);
+		VUT.pfh = NULL;
+	}
+}
+
+static void
+vut_sighup(int sig)
+{
+	(void)sig;
+	VUT.sighup = 1;
+}
+
+static void
+vut_sigint(int sig)
+{
+	(void)sig;
+	VUT.sigint = 1;
+}
+
+static void
+vut_sigusr1(int sig)
+{
+	(void)sig;
+	VUT.sigusr1 = 1;
+}
+
+static int __match_proto__(VSLQ_dispatch_f)
+vut_dispatch(struct VSL_data *vsl, struct VSL_transaction * const trans[],
+    void *priv)
+{
+	int i;
+
+	(void)priv;
+	if (VUT.k_arg == 0)
+		return (-1);	/* End of file */
+	AN(VUT.dispatch_f);
+	i = VUT.dispatch_f(vsl, trans, VUT.dispatch_priv);
+	if (VUT.k_arg > 0)
+		VUT.k_arg--;
+	if (i >= 0 && VUT.k_arg == 0)
+		return (-1);	/* End of file */
+	return (i);
+}
+
+void
+VUT_Error(int status, const char *fmt, ...)
+{
+	va_list ap;
+
+	AN(fmt);
+	va_start(ap, fmt);
+	vfprintf(stderr, fmt, ap);
+	va_end(ap);
+	fprintf(stderr, "\n");
+
+	if (status)
+		exit(status);
+}
+
+int
+VUT_g_Arg(const char *arg)
+{
+
+	VUT.g_arg = VSLQ_Name2Grouping(arg, -1);
+	if (VUT.g_arg == -2)
+		VUT_Error(1, "Ambiguous grouping type: %s", arg);
+	else if (VUT.g_arg < 0)
+		VUT_Error(1, "Unknown grouping type: %s", arg);
+	return (1);
+}
+
+int
+VUT_Arg(int opt, const char *arg)
+{
+	int i;
+	char *p;
+
+	switch (opt) {
+	case 'd':
+		/* Head */
+		VUT.d_opt = 1;
+		return (1);
+	case 'D':
+		/* Daemon mode */
+		VUT.D_opt = 1;
+		return (1);
+	case 'g':
+		/* Grouping */
+		return (VUT_g_Arg(arg));
+	case 'k':
+		/* Log transaction limit */
+		VUT.k_arg = (int)strtol(arg, &p, 10);
+		if (*p != '\0' || VUT.k_arg <= 0)
+			VUT_Error(1, "-k: Invalid number '%s'", arg);
+		return (1);
+	case 'n':
+		/* Varnish instance name */
+		REPLACE(VUT.n_arg, arg);
+		return (1);
+	case 'N':
+		/* Varnish stale VSM file */
+		REPLACE(VUT.N_arg, arg);
+		return (1);
+	case 'P':
+		/* PID file */
+		REPLACE(VUT.P_arg, arg);
+		return (1);
+	case 'q':
+		/* Query to use */
+		REPLACE(VUT.q_arg, arg);
+		return (1);
+	case 'r':
+		/* Binary file input */
+		REPLACE(VUT.r_arg, arg);
+		return (1);
+	case 't':
+		/* VSM connect timeout */
+		if (!strcasecmp("off", arg))
+			VUT.t_arg = -1.;
+		else {
+			VUT.t_arg = VNUM(arg);
+			if (isnan(VUT.t_arg))
+				VUT_Error(1, "-t: Syntax error");
+			if (VUT.t_arg < 0.)
+				VUT_Error(1, "-t: Range error");
+		}
+		return (1);
+	case 'V':
+		/* Print version number and exit */
+		VCS_Message(VUT.progname);
+		exit(0);
+	default:
+		AN(VUT.vsl);
+		i = VSL_Arg(VUT.vsl, opt, arg);
+		if (i < 0)
+			VUT_Error(1, "%s", VSL_Error(VUT.vsl));
+		return (i);
+	}
+}
+
+void
+VUT_Init(const char *progname, int argc, char * const *argv,
+    const struct vopt_spec *voc)
+{
+
+	if (argc == 2 && !strcmp(argv[1], "--synopsis"))
+		exit(vut_synopsis(voc));
+	if (argc == 2 && !strcmp(argv[1], "--options"))
+		exit(vut_options(voc));
+
+	VUT.progname = progname;
+	REPLACE(VUT.name, "");
+	VUT.g_arg = VSL_g_vxid;
+	AZ(VUT.vsl);
+	VUT.vsl = VSL_New();
+	AN(VUT.vsl);
+	VUT.k_arg = -1;
+	VUT.t_arg = 5.;
+}
+
+void
+VUT_Setup(void)
+{
+	struct VSL_cursor *c;
+	double t_start;
+	int i;
+
+	AN(VUT.vsl);
+	AZ(VUT.vsm);
+	AZ(VUT.vslq);
+
+	/* Check input arguments */
+	if ((VUT.n_arg == NULL ? 0 : 1) +
+	    (VUT.N_arg == NULL ? 0 : 1) +
+	    (VUT.r_arg == NULL ? 0 : 1) > 1)
+		VUT_Error(1, "Only one of -n, -N and -r options may be used");
+
+	/* Create and validate the query expression */
+	VUT.vslq = VSLQ_New(VUT.vsl, NULL, VUT.g_arg, VUT.q_arg);
+	if (VUT.vslq == NULL)
+		VUT_Error(1, "Query expression error:\n%s", VSL_Error(VUT.vsl));
+
+	/* Setup input */
+	if (VUT.r_arg) {
+		REPLACE(VUT.name, VUT.r_arg);
+		c = VSL_CursorFile(VUT.vsl, VUT.r_arg, 0);
+		if (c == NULL)
+			VUT_Error(1, "Can't open log file (%s)",
+			    VSL_Error(VUT.vsl));
+	} else {
+		VUT.vsm = VSM_New();
+		AN(VUT.vsm);
+		if (VUT.n_arg && VSM_n_Arg(VUT.vsm, VUT.n_arg) <= 0)
+			VUT_Error(1, "%s", VSM_Error(VUT.vsm));
+		if (VUT.N_arg && VSM_N_Arg(VUT.vsm, VUT.N_arg) <= 0)
+			VUT_Error(1, "%s", VSM_Error(VUT.vsm));
+		REPLACE(VUT.name, VSM_Name(VUT.vsm));
+		t_start = NAN;
+		c = NULL;
+		while (1) {
+			i = VSM_Open(VUT.vsm);
+			if (!i)
+				c = VSL_CursorVSM(VUT.vsl, VUT.vsm,
+				    (VUT.d_opt ? VSL_COPT_TAILSTOP :
+					VSL_COPT_TAIL)
+				    | VSL_COPT_BATCH);
+			if (c)
+				break;
+
+			if (isnan(t_start) && VUT.t_arg > 0.) {
+				VUT_Error(0, "Can't open log -"
+				    " retrying for %.0f seconds", VUT.t_arg);
+				t_start = VTIM_real();
+			}
+			VSM_Close(VUT.vsm);
+			if (VUT.t_arg <= 0.)
+				break;
+			if (VTIM_real() - t_start > VUT.t_arg)
+				break;
+
+			VSM_ResetError(VUT.vsm);
+			VSL_ResetError(VUT.vsl);
+			VTIM_sleep(0.5);
+		}
+
+		if (VUT.t_arg >= 0. && (i || !c)) {
+			if (i)
+				VUT_Error(1, "Can't open VSM file (%s)",
+				    VSM_Error(VUT.vsm));
+			else
+				VUT_Error(1, "Can't open log (%s)",
+				    VSL_Error(VUT.vsl));
+		} else if (!isnan(t_start))
+			VUT_Error(0, "Log opened");
+	}
+
+	if (c)
+		VSLQ_SetCursor(VUT.vslq, &c);
+	AZ(c);
+
+	/* Signal handlers */
+	(void)signal(SIGHUP, vut_sighup);
+	(void)signal(SIGINT, vut_sigint);
+	(void)signal(SIGTERM, vut_sigint);
+	(void)signal(SIGUSR1, vut_sigusr1);
+
+	/* Open PID file */
+	if (VUT.P_arg) {
+		AZ(VUT.pfh);
+		VUT.pfh = VPF_Open(VUT.P_arg, 0644, NULL);
+		if (VUT.pfh == NULL)
+			VUT_Error(1, "%s: %s", VUT.P_arg, strerror(errno));
+	}
+
+	/* Daemon mode */
+	if (VUT.D_opt && varnish_daemon(0, 0) == -1)
+		VUT_Error(1, "Daemon mode: %s", strerror(errno));
+
+	/* Write PID and setup exit handler */
+	if (VUT.pfh != NULL) {
+		VPF_Write(VUT.pfh);
+		AZ(atexit(vut_vpf_remove));
+	}
+}
+
+void
+VUT_Fini(void)
+{
+	free(VUT.n_arg);
+	free(VUT.N_arg);
+	free(VUT.r_arg);
+	free(VUT.P_arg);
+	free(VUT.name);
+
+	vut_vpf_remove();
+	AZ(VUT.pfh);
+
+	if (VUT.vslq)
+		VSLQ_Delete(&VUT.vslq);
+	if (VUT.vsl)
+		VSL_Delete(VUT.vsl);
+	if (VUT.vsm)
+		VSM_Delete(VUT.vsm);
+
+	memset(&VUT, 0, sizeof VUT);
+}
+
+int
+VUT_Main(void)
+{
+	struct VSL_cursor *c;
+	int i = -1;
+
+	AN(VUT.vslq);
+
+	while (!VUT.sigint) {
+		if (VUT.sighup && VUT.sighup_f) {
+			/* sighup callback */
+			VUT.sighup = 0;
+			i = (VUT.sighup_f)();
+			if (i)
+				break;
+		}
+
+		if (VUT.sigusr1) {
+			/* Flush and report any incomplete records */
+			VUT.sigusr1 = 0;
+			VSLQ_Flush(VUT.vslq, vut_dispatch, NULL);
+		}
+
+		if (VUT.vsm != NULL && !VSM_IsOpen(VUT.vsm)) {
+			/* Reconnect VSM */
+			AZ(VUT.r_arg);
+			VTIM_sleep(0.1);
+			if (VSM_Open(VUT.vsm)) {
+				VSM_ResetError(VUT.vsm);
+				continue;
+			}
+			c = VSL_CursorVSM(VUT.vsl, VUT.vsm,
+			    (VUT.d_opt ? VSL_COPT_TAILSTOP : VSL_COPT_TAIL)
+			    | VSL_COPT_BATCH);
+			if (c == NULL) {
+				VSL_ResetError(VUT.vsl);
+				VSM_Close(VUT.vsm);
+				continue;
+			}
+			VSLQ_SetCursor(VUT.vslq, &c);
+			AZ(c);
+			VUT_Error(0, "Log reacquired");
+		}
+
+		i = VSLQ_Dispatch(VUT.vslq, vut_dispatch, NULL);
+		if (i == 1)
+			/* Call again */
+			continue;
+		else if (i == 0) {
+			/* Nothing to do but wait */
+			if (VUT.idle_f) {
+				i = (VUT.idle_f)();
+				if (i)
+					break;
+			}
+			VTIM_sleep(0.01);
+			continue;
+		} else if (i == -1) {
+			/* EOF */
+			break;
+		}
+
+		if (VUT.vsm == NULL)
+			break;
+
+		/* XXX: Make continuation optional */
+
+		VSLQ_Flush(VUT.vslq, vut_dispatch, NULL);
+
+		if (i == -2)
+			/* Abandoned */
+			VUT_Error(0, "Log abandoned");
+		else if (i < -2)
+			/* Overrun */
+			VUT_Error(0, "Log overrun");
+
+		VSM_Close(VUT.vsm);
+	}
+
+	return (i);
+}
+
+/**********************************************************************/
+
+
+static void
+print_nobrackets(const char *s)
+{
+	const char *e;
+
+	/* Remove whitespace */
+	while (isspace(*s))
+		s++;
+	e = s + strlen(s);
+	while (e > s && isspace(e[-1]))
+		e--;
+
+	/* Remove outer layer brackets if present */
+	if (e > s && *s == '[' && e[-1] == ']') {
+		s++;
+		e--;
+	}
+
+	printf("%.*s", (int)(e - s), s);
+}
+
+static void
+print_tabbed(const char *string, int tabs)
+{
+	int i;
+	const char *c;
+
+	for (c = string; *c; c++) {
+		if (c == string || *(c - 1) == '\n')
+			for (i = 0; i < tabs; i++)
+				printf("\t");
+		printf("%c", *c);
+	}
+}
+
+static void
+print_opt(const struct vopt_list *opt)
+{
+	print_nobrackets(opt->synopsis);
+	printf("\n\n");
+	print_tabbed(opt->ldesc, 1);
+	printf("\n\n");
+}
+
+static int
+vut_synopsis(const struct vopt_spec *voc)
+{
+	printf(".. |synopsis| replace:: %s\n", voc->vopt_synopsis);
+	return (0);
+}
+
+static int
+vut_options(const struct vopt_spec *voc)
+{
+	int i;
+
+	for (i = 0; i < voc->vopt_list_n; i++)
+		print_opt(&voc->vopt_list[i]);
+	return (0);
+}
diff --git a/lib/libvarnishtools/Makefile.am b/lib/libvarnishtools/Makefile.am
deleted file mode 100644
index 5fffe27..0000000
--- a/lib/libvarnishtools/Makefile.am
+++ /dev/null
@@ -1,4 +0,0 @@
-#
-
-EXTRA_DIST = \
-	vut.c
diff --git a/lib/libvarnishtools/Makefile.phk b/lib/libvarnishtools/Makefile.phk
deleted file mode 100644
index 0781a45..0000000
--- a/lib/libvarnishtools/Makefile.phk
+++ /dev/null
@@ -1,6 +0,0 @@
-
-LIB_SRC += opt2rst.c
-LIB_SRC += vut.c
-
-TOPDIR= $(CURDIR)/../..
-include $(TOPDIR)/Makefile.inc.phk
diff --git a/lib/libvarnishtools/vut.c b/lib/libvarnishtools/vut.c
deleted file mode 100644
index 39c27fb..0000000
--- a/lib/libvarnishtools/vut.c
+++ /dev/null
@@ -1,500 +0,0 @@
-/*-
- * Copyright (c) 2006 Verdens Gang AS
- * Copyright (c) 2006-2015 Varnish Software AS
- * All rights reserved.
- *
- * Author: Martin Blix Grydeland <martin at varnish-software.com>
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
- * 1. Redistributions of source code must retain the above copyright
- *    notice, this list of conditions and the following disclaimer.
- * 2. Redistributions in binary form must reproduce the above copyright
- *    notice, this list of conditions and the following disclaimer in the
- *    documentation and/or other materials provided with the distribution.
- *
- * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
- * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED.  IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
- * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
- * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
- * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
- * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
- * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
- * SUCH DAMAGE.
- *
- * Common functions for the utilities
- */
-
-#include "config.h"
-
-#include <ctype.h>
-#include <stdint.h>
-#include <stdarg.h>
-#include <stdlib.h>
-#include <unistd.h>
-#include <stdio.h>
-#include <errno.h>
-#include <string.h>
-#include <signal.h>
-#include <math.h>
-
-#include "compat/daemon.h"
-#include "vdef.h"
-#include "vpf.h"
-#include "vapi/vsm.h"
-#include "vapi/vsl.h"
-#include "vtim.h"
-#include "vas.h"
-#include "miniobj.h"
-#include "vcs.h"
-#include "vnum.h"
-
-#include "vut.h"
-
-#include "vapi/voptget.h"
-
-struct VUT VUT;
-
-static int vut_synopsis(const struct vopt_spec *);
-static int vut_options(const struct vopt_spec *);
-
-static void
-vut_vpf_remove(void)
-{
-	if (VUT.pfh) {
-		VPF_Remove(VUT.pfh);
-		VUT.pfh = NULL;
-	}
-}
-
-static void
-vut_sighup(int sig)
-{
-	(void)sig;
-	VUT.sighup = 1;
-}
-
-static void
-vut_sigint(int sig)
-{
-	(void)sig;
-	VUT.sigint = 1;
-}
-
-static void
-vut_sigusr1(int sig)
-{
-	(void)sig;
-	VUT.sigusr1 = 1;
-}
-
-static int __match_proto__(VSLQ_dispatch_f)
-vut_dispatch(struct VSL_data *vsl, struct VSL_transaction * const trans[],
-    void *priv)
-{
-	int i;
-
-	(void)priv;
-	if (VUT.k_arg == 0)
-		return (-1);	/* End of file */
-	AN(VUT.dispatch_f);
-	i = VUT.dispatch_f(vsl, trans, VUT.dispatch_priv);
-	if (VUT.k_arg > 0)
-		VUT.k_arg--;
-	if (i >= 0 && VUT.k_arg == 0)
-		return (-1);	/* End of file */
-	return (i);
-}
-
-void
-VUT_Error(int status, const char *fmt, ...)
-{
-	va_list ap;
-
-	AN(fmt);
-	va_start(ap, fmt);
-	vfprintf(stderr, fmt, ap);
-	va_end(ap);
-	fprintf(stderr, "\n");
-
-	if (status)
-		exit(status);
-}
-
-int
-VUT_g_Arg(const char *arg)
-{
-
-	VUT.g_arg = VSLQ_Name2Grouping(arg, -1);
-	if (VUT.g_arg == -2)
-		VUT_Error(1, "Ambiguous grouping type: %s", arg);
-	else if (VUT.g_arg < 0)
-		VUT_Error(1, "Unknown grouping type: %s", arg);
-	return (1);
-}
-
-int
-VUT_Arg(int opt, const char *arg)
-{
-	int i;
-	char *p;
-
-	switch (opt) {
-	case 'd':
-		/* Head */
-		VUT.d_opt = 1;
-		return (1);
-	case 'D':
-		/* Daemon mode */
-		VUT.D_opt = 1;
-		return (1);
-	case 'g':
-		/* Grouping */
-		return (VUT_g_Arg(arg));
-	case 'k':
-		/* Log transaction limit */
-		VUT.k_arg = (int)strtol(arg, &p, 10);
-		if (*p != '\0' || VUT.k_arg <= 0)
-			VUT_Error(1, "-k: Invalid number '%s'", arg);
-		return (1);
-	case 'n':
-		/* Varnish instance name */
-		REPLACE(VUT.n_arg, arg);
-		return (1);
-	case 'N':
-		/* Varnish stale VSM file */
-		REPLACE(VUT.N_arg, arg);
-		return (1);
-	case 'P':
-		/* PID file */
-		REPLACE(VUT.P_arg, arg);
-		return (1);
-	case 'q':
-		/* Query to use */
-		REPLACE(VUT.q_arg, arg);
-		return (1);
-	case 'r':
-		/* Binary file input */
-		REPLACE(VUT.r_arg, arg);
-		return (1);
-	case 't':
-		/* VSM connect timeout */
-		if (!strcasecmp("off", arg))
-			VUT.t_arg = -1.;
-		else {
-			VUT.t_arg = VNUM(arg);
-			if (isnan(VUT.t_arg))
-				VUT_Error(1, "-t: Syntax error");
-			if (VUT.t_arg < 0.)
-				VUT_Error(1, "-t: Range error");
-		}
-		return (1);
-	case 'V':
-		/* Print version number and exit */
-		VCS_Message(VUT.progname);
-		exit(0);
-	default:
-		AN(VUT.vsl);
-		i = VSL_Arg(VUT.vsl, opt, arg);
-		if (i < 0)
-			VUT_Error(1, "%s", VSL_Error(VUT.vsl));
-		return (i);
-	}
-}
-
-void
-VUT_Init(const char *progname, int argc, char * const *argv,
-    const struct vopt_spec *voc)
-{
-
-	if (argc == 2 && !strcmp(argv[1], "--synopsis"))
-		exit(vut_synopsis(voc));
-	if (argc == 2 && !strcmp(argv[1], "--options"))
-		exit(vut_options(voc));
-
-	VUT.progname = progname;
-	REPLACE(VUT.name, "");
-	VUT.g_arg = VSL_g_vxid;
-	AZ(VUT.vsl);
-	VUT.vsl = VSL_New();
-	AN(VUT.vsl);
-	VUT.k_arg = -1;
-	VUT.t_arg = 5.;
-}
-
-void
-VUT_Setup(void)
-{
-	struct VSL_cursor *c;
-	double t_start;
-	int i;
-
-	AN(VUT.vsl);
-	AZ(VUT.vsm);
-	AZ(VUT.vslq);
-
-	/* Check input arguments */
-	if ((VUT.n_arg == NULL ? 0 : 1) +
-	    (VUT.N_arg == NULL ? 0 : 1) +
-	    (VUT.r_arg == NULL ? 0 : 1) > 1)
-		VUT_Error(1, "Only one of -n, -N and -r options may be used");
-
-	/* Create and validate the query expression */
-	VUT.vslq = VSLQ_New(VUT.vsl, NULL, VUT.g_arg, VUT.q_arg);
-	if (VUT.vslq == NULL)
-		VUT_Error(1, "Query expression error:\n%s", VSL_Error(VUT.vsl));
-
-	/* Setup input */
-	if (VUT.r_arg) {
-		REPLACE(VUT.name, VUT.r_arg);
-		c = VSL_CursorFile(VUT.vsl, VUT.r_arg, 0);
-		if (c == NULL)
-			VUT_Error(1, "Can't open log file (%s)",
-			    VSL_Error(VUT.vsl));
-	} else {
-		VUT.vsm = VSM_New();
-		AN(VUT.vsm);
-		if (VUT.n_arg && VSM_n_Arg(VUT.vsm, VUT.n_arg) <= 0)
-			VUT_Error(1, "%s", VSM_Error(VUT.vsm));
-		if (VUT.N_arg && VSM_N_Arg(VUT.vsm, VUT.N_arg) <= 0)
-			VUT_Error(1, "%s", VSM_Error(VUT.vsm));
-		REPLACE(VUT.name, VSM_Name(VUT.vsm));
-		t_start = NAN;
-		c = NULL;
-		while (1) {
-			i = VSM_Open(VUT.vsm);
-			if (!i)
-				c = VSL_CursorVSM(VUT.vsl, VUT.vsm,
-				    (VUT.d_opt ? VSL_COPT_TAILSTOP :
-					VSL_COPT_TAIL)
-				    | VSL_COPT_BATCH);
-			if (c)
-				break;
-
-			if (isnan(t_start) && VUT.t_arg > 0.) {
-				VUT_Error(0, "Can't open log -"
-				    " retrying for %.0f seconds", VUT.t_arg);
-				t_start = VTIM_real();
-			}
-			VSM_Close(VUT.vsm);
-			if (VUT.t_arg <= 0.)
-				break;
-			if (VTIM_real() - t_start > VUT.t_arg)
-				break;
-
-			VSM_ResetError(VUT.vsm);
-			VSL_ResetError(VUT.vsl);
-			VTIM_sleep(0.5);
-		}
-
-		if (VUT.t_arg >= 0. && (i || !c)) {
-			if (i)
-				VUT_Error(1, "Can't open VSM file (%s)",
-				    VSM_Error(VUT.vsm));
-			else
-				VUT_Error(1, "Can't open log (%s)",
-				    VSL_Error(VUT.vsl));
-		} else if (!isnan(t_start))
-			VUT_Error(0, "Log opened");
-	}
-
-	if (c)
-		VSLQ_SetCursor(VUT.vslq, &c);
-	AZ(c);
-
-	/* Signal handlers */
-	(void)signal(SIGHUP, vut_sighup);
-	(void)signal(SIGINT, vut_sigint);
-	(void)signal(SIGTERM, vut_sigint);
-	(void)signal(SIGUSR1, vut_sigusr1);
-
-	/* Open PID file */
-	if (VUT.P_arg) {
-		AZ(VUT.pfh);
-		VUT.pfh = VPF_Open(VUT.P_arg, 0644, NULL);
-		if (VUT.pfh == NULL)
-			VUT_Error(1, "%s: %s", VUT.P_arg, strerror(errno));
-	}
-
-	/* Daemon mode */
-	if (VUT.D_opt && varnish_daemon(0, 0) == -1)
-		VUT_Error(1, "Daemon mode: %s", strerror(errno));
-
-	/* Write PID and setup exit handler */
-	if (VUT.pfh != NULL) {
-		VPF_Write(VUT.pfh);
-		AZ(atexit(vut_vpf_remove));
-	}
-}
-
-void
-VUT_Fini(void)
-{
-	free(VUT.n_arg);
-	free(VUT.N_arg);
-	free(VUT.r_arg);
-	free(VUT.P_arg);
-	free(VUT.name);
-
-	vut_vpf_remove();
-	AZ(VUT.pfh);
-
-	if (VUT.vslq)
-		VSLQ_Delete(&VUT.vslq);
-	if (VUT.vsl)
-		VSL_Delete(VUT.vsl);
-	if (VUT.vsm)
-		VSM_Delete(VUT.vsm);
-
-	memset(&VUT, 0, sizeof VUT);
-}
-
-int
-VUT_Main(void)
-{
-	struct VSL_cursor *c;
-	int i = -1;
-
-	AN(VUT.vslq);
-
-	while (!VUT.sigint) {
-		if (VUT.sighup && VUT.sighup_f) {
-			/* sighup callback */
-			VUT.sighup = 0;
-			i = (VUT.sighup_f)();
-			if (i)
-				break;
-		}
-
-		if (VUT.sigusr1) {
-			/* Flush and report any incomplete records */
-			VUT.sigusr1 = 0;
-			VSLQ_Flush(VUT.vslq, vut_dispatch, NULL);
-		}
-
-		if (VUT.vsm != NULL && !VSM_IsOpen(VUT.vsm)) {
-			/* Reconnect VSM */
-			AZ(VUT.r_arg);
-			VTIM_sleep(0.1);
-			if (VSM_Open(VUT.vsm)) {
-				VSM_ResetError(VUT.vsm);
-				continue;
-			}
-			c = VSL_CursorVSM(VUT.vsl, VUT.vsm,
-			    (VUT.d_opt ? VSL_COPT_TAILSTOP : VSL_COPT_TAIL)
-			    | VSL_COPT_BATCH);
-			if (c == NULL) {
-				VSL_ResetError(VUT.vsl);
-				VSM_Close(VUT.vsm);
-				continue;
-			}
-			VSLQ_SetCursor(VUT.vslq, &c);
-			AZ(c);
-			VUT_Error(0, "Log reacquired");
-		}
-
-		i = VSLQ_Dispatch(VUT.vslq, vut_dispatch, NULL);
-		if (i == 1)
-			/* Call again */
-			continue;
-		else if (i == 0) {
-			/* Nothing to do but wait */
-			if (VUT.idle_f) {
-				i = (VUT.idle_f)();
-				if (i)
-					break;
-			}
-			VTIM_sleep(0.01);
-			continue;
-		} else if (i == -1) {
-			/* EOF */
-			break;
-		}
-
-		if (VUT.vsm == NULL)
-			break;
-
-		/* XXX: Make continuation optional */
-
-		VSLQ_Flush(VUT.vslq, vut_dispatch, NULL);
-
-		if (i == -2)
-			/* Abandoned */
-			VUT_Error(0, "Log abandoned");
-		else if (i < -2)
-			/* Overrun */
-			VUT_Error(0, "Log overrun");
-
-		VSM_Close(VUT.vsm);
-	}
-
-	return (i);
-}
-
-/**********************************************************************/
-
-
-static void
-print_nobrackets(const char *s)
-{
-	const char *e;
-
-	/* Remove whitespace */
-	while (isspace(*s))
-		s++;
-	e = s + strlen(s);
-	while (e > s && isspace(e[-1]))
-		e--;
-
-	/* Remove outer layer brackets if present */
-	if (e > s && *s == '[' && e[-1] == ']') {
-		s++;
-		e--;
-	}
-
-	printf("%.*s", (int)(e - s), s);
-}
-
-static void
-print_tabbed(const char *string, int tabs)
-{
-	int i;
-	const char *c;
-
-	for (c = string; *c; c++) {
-		if (c == string || *(c - 1) == '\n')
-			for (i = 0; i < tabs; i++)
-				printf("\t");
-		printf("%c", *c);
-	}
-}
-
-static void
-print_opt(const struct vopt_list *opt)
-{
-	print_nobrackets(opt->synopsis);
-	printf("\n\n");
-	print_tabbed(opt->ldesc, 1);
-	printf("\n\n");
-}
-
-static int
-vut_synopsis(const struct vopt_spec *voc)
-{
-	printf(".. |synopsis| replace:: %s\n", voc->vopt_synopsis);
-	return (0);
-}
-
-static int
-vut_options(const struct vopt_spec *voc)
-{
-	int i;
-
-	for (i = 0; i < voc->vopt_list_n; i++)
-		print_opt(&voc->vopt_list[i]);
-	return (0);
-}



From phk at FreeBSD.org  Tue May 10 11:04:07 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 10 May 2016 13:04:07 +0200
Subject: [master] 2e32704 Expose (parts of) VTIM, VSB,
 VAS and VCS via libvarnishapi.
Message-ID: <E1b05SR-0002x6-1A@project.varnish-software.com>


commit 2e32704fee4d79a0aa44f0b833a43feb55f1801c
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue May 10 11:03:13 2016 +0000

    Expose (parts of) VTIM, VSB, VAS and VCS via libvarnishapi.

diff --git a/bin/varnishadm/Makefile.am b/bin/varnishadm/Makefile.am
index d218bc9..d4dd781 100644
--- a/bin/varnishadm/Makefile.am
+++ b/bin/varnishadm/Makefile.am
@@ -8,7 +8,6 @@ bin_PROGRAMS = varnishadm
 
 varnishadm_SOURCES = \
 	varnishadm.c \
-	$(top_srcdir)/lib/libvarnish/vas.c \
 	$(top_srcdir)/lib/libvarnish/vsa.c \
 	$(top_srcdir)/lib/libvarnish/vtcp.c \
 	$(top_srcdir)/lib/libvarnish/vss.c
diff --git a/bin/varnishhist/Makefile.am b/bin/varnishhist/Makefile.am
index f814bf2..891b425 100644
--- a/bin/varnishhist/Makefile.am
+++ b/bin/varnishhist/Makefile.am
@@ -8,10 +8,7 @@ bin_PROGRAMS = varnishhist
 
 varnishhist_SOURCES = varnishhist.c \
 	varnishhist_options.h \
-	varnishhist_options.c \
-	$(top_srcdir)/lib/libvarnish/vas.c \
-	$(top_srcdir)/lib/libvarnish/version.c \
-	$(top_srcdir)/lib/libvarnish/vtim.c
+	varnishhist_options.c
 
 varnishhist_CFLAGS = \
 	@SAN_CFLAGS@
diff --git a/bin/varnishlog/Makefile.am b/bin/varnishlog/Makefile.am
index 92b93d3..9e309e7 100644
--- a/bin/varnishlog/Makefile.am
+++ b/bin/varnishlog/Makefile.am
@@ -9,11 +9,7 @@ bin_PROGRAMS = varnishlog
 varnishlog_SOURCES = \
 	varnishlog.c \
 	varnishlog_options.h \
-	varnishlog_options.c \
-	$(top_srcdir)/lib/libvarnish/vas.c \
-	$(top_srcdir)/lib/libvarnish/version.c \
-	$(top_srcdir)/lib/libvarnish/vsb.c \
-	$(top_srcdir)/lib/libvarnish/vtim.c
+	varnishlog_options.c
 
 varnishlog_CFLAGS = \
 	@SAN_CFLAGS@
diff --git a/bin/varnishncsa/Makefile.am b/bin/varnishncsa/Makefile.am
index bb470f7..a6cbeb7 100644
--- a/bin/varnishncsa/Makefile.am
+++ b/bin/varnishncsa/Makefile.am
@@ -11,11 +11,7 @@ varnishncsa_SOURCES = \
 	varnishncsa_options.h \
 	varnishncsa_options.c \
 	base64.h \
-	base64.c \
-	$(top_srcdir)/lib/libvarnish/vas.c \
-	$(top_srcdir)/lib/libvarnish/version.c \
-	$(top_srcdir)/lib/libvarnish/vtim.c \
-	$(top_srcdir)/lib/libvarnish/vsb.c
+	base64.c
 
 varnishncsa_CFLAGS = \
 	@SAN_CFLAGS@
diff --git a/bin/varnishstat/Makefile.am b/bin/varnishstat/Makefile.am
index 52fb5d8..5c0a179 100644
--- a/bin/varnishstat/Makefile.am
+++ b/bin/varnishstat/Makefile.am
@@ -10,10 +10,7 @@ varnishstat_SOURCES = \
 	varnishstat.h \
 	\
 	varnishstat.c \
-	varnishstat_curses.c \
-	$(top_srcdir)/lib/libvarnish/vas.c \
-	$(top_srcdir)/lib/libvarnish/version.c \
-	$(top_srcdir)/lib/libvarnish/vtim.c
+	varnishstat_curses.c
 
 varnishstat_CFLAGS = \
 	@SAN_CFLAGS@
diff --git a/bin/varnishtop/Makefile.am b/bin/varnishtop/Makefile.am
index 9406d62..fdd993a 100644
--- a/bin/varnishtop/Makefile.am
+++ b/bin/varnishtop/Makefile.am
@@ -8,11 +8,7 @@ bin_PROGRAMS = varnishtop
 
 varnishtop_SOURCES = varnishtop.c \
 	varnishtop_options.h \
-	varnishtop_options.c \
-	$(top_srcdir)/lib/libvarnish/vas.c \
-	$(top_srcdir)/lib/libvarnish/version.c \
-	$(top_srcdir)/lib/libvarnish/vtim.c \
-	$(top_srcdir)/lib/libvarnish/vsb.c
+	varnishtop_options.c
 
 
 varnishtop_CFLAGS = \
diff --git a/lib/libvarnishapi/libvarnishapi.map b/lib/libvarnishapi/libvarnishapi.map
index 3401363..f9d189a 100644
--- a/lib/libvarnishapi/libvarnishapi.map
+++ b/lib/libvarnishapi/libvarnishapi.map
@@ -142,4 +142,23 @@ LIBVARNISHAPI_1.5 {
 	VUT_Fini;
 	VUT_Main;
 	VUT;
+
+	VTIM_mono;
+	VTIM_real;
+	VTIM_sleep;
+
+	VSB_new;
+	VSB_destroy;
+	VSB_error;
+	VSB_cat;
+	VSB_putc;
+	VSB_printf;
+	VSB_clear;
+	VSB_finish;
+	VSB_len;
+	VSB_data;
+
+	VAS_Fail;
+
+	VCS_Message;
 } LIBVARNISHAPI_1.0;



From phk at FreeBSD.org  Tue May 10 11:12:06 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 10 May 2016 13:12:06 +0200
Subject: [master] 7f7135d GC a straggling reference to libvarnishttools
Message-ID: <E1b05aA-0003Qh-Bt@project.varnish-software.com>


commit 7f7135d698f109fe09594f972751449084125f03
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue May 10 11:10:41 2016 +0000

    GC a straggling reference to libvarnishttools

diff --git a/lib/Makefile.am b/lib/Makefile.am
index 9359575..2fd8999 100644
--- a/lib/Makefile.am
+++ b/lib/Makefile.am
@@ -14,7 +14,6 @@ DIST_SUBDIRS = 	\
 	libvarnishcompat \
 	libvarnish \
 	libvarnishapi \
-	libvarnishtools \
 	libvcc \
 	libvgz \
 	libvmod_debug \



From phk at FreeBSD.org  Tue May 10 11:15:07 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 10 May 2016 13:15:07 +0200
Subject: [master] de91f03 GC yet another reference to libvarnishtools
Message-ID: <E1b05d5-0003fu-6R@project.varnish-software.com>


commit de91f03ad2fc2fe162f4a9bb5273c59cc1c28f8f
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue May 10 11:13:55 2016 +0000

    GC yet another reference to libvarnishtools

diff --git a/configure.ac b/configure.ac
index f724d7e..fa78ba0 100644
--- a/configure.ac
+++ b/configure.ac
@@ -721,7 +721,6 @@ AC_CONFIG_FILES([
     lib/Makefile
     lib/libvarnish/Makefile
     lib/libvarnishapi/Makefile
-    lib/libvarnishtools/Makefile
     lib/libvarnishcompat/Makefile
     lib/libvcc/Makefile
     lib/libvgz/Makefile



From nils.goroll at uplex.de  Wed May 11 12:54:06 2016
From: nils.goroll at uplex.de (Nils Goroll)
Date: Wed, 11 May 2016 14:54:06 +0200
Subject: [master] f91c6f9 check for overrun before looking at the next vsm
 record
Message-ID: <E1b0TeQ-0006h2-8v@project.varnish-software.com>


commit f91c6f9025bd2f8faf0e45542e21a4119f33c7c2
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Wed May 11 14:47:10 2016 +0200

    check for overrun before looking at the next vsm record
    
    Fixes #1873

diff --git a/lib/libvarnishapi/vsl_cursor.c b/lib/libvarnishapi/vsl_cursor.c
index e9daab0..33d08e3 100644
--- a/lib/libvarnishapi/vsl_cursor.c
+++ b/lib/libvarnishapi/vsl_cursor.c
@@ -126,6 +126,10 @@ vslc_vsm_next(const struct VSL_cursor *cursor)
 	CHECK_OBJ_NOTNULL(c->vsm, VSM_MAGIC);
 
 	while (1) {
+		i = vslc_vsm_check(&c->cursor, &c->next);
+		if (i <= 0)
+			return (-3); /* Overrun */
+
 		t = *(volatile const uint32_t *)c->next.ptr;
 		AN(t);
 
@@ -147,10 +151,6 @@ vslc_vsm_next(const struct VSL_cursor *cursor)
 			return (0);	/* No new records available */
 		}
 
-		i = vslc_vsm_check(&c->cursor, &c->next);
-		if (i <= 0)
-			return (-3); /* Overrun */
-
 		c->cursor.rec = c->next;
 		c->next.ptr = VSL_NEXT(c->next.ptr);
 



From nils.goroll at uplex.de  Wed May 11 13:57:05 2016
From: nils.goroll at uplex.de (Nils Goroll)
Date: Wed, 11 May 2016 15:57:05 +0200
Subject: [master] 114bef7 No need to rearm an object the exp thread has
 already deref'ed
Message-ID: <E1b0UdN-0001qz-SB@project.varnish-software.com>


commit 114bef78f13cdfdfbf5baa26e6a31ea36fab0bbf
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Wed May 11 15:55:36 2016 +0200

    No need to rearm an object the exp thread has already deref'ed
    
    Fixes https://github.com/nigoroll/varnish-modules/issues/1

diff --git a/bin/varnishd/cache/cache_expire.c b/bin/varnishd/cache/cache_expire.c
index e226fee..969b117 100644
--- a/bin/varnishd/cache/cache_expire.c
+++ b/bin/varnishd/cache/cache_expire.c
@@ -150,7 +150,8 @@ EXP_Rearm(struct objcore *oc, double now, double ttl, double grace, double keep)
 	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
 	assert(oc->refcnt > 0);
 
-	AN(oc->exp_flags & OC_EF_REFD);
+	if (! (oc->exp_flags & OC_EF_REFD))
+		return;
 
 	if (!isnan(ttl))
 		oc->ttl = now + ttl - oc->t_origin;



From nils.goroll at uplex.de  Wed May 11 14:15:08 2016
From: nils.goroll at uplex.de (Nils Goroll)
Date: Wed, 11 May 2016 16:15:08 +0200
Subject: [master] 104c593 document the default probe
Message-ID: <E1b0Uuq-0002u3-VO@project.varnish-software.com>


commit 104c5930a6fdfe23416d3d20d77664c3b9aee793
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Wed May 11 16:14:17 2016 +0200

    document the default probe

diff --git a/doc/sphinx/reference/vcl.rst b/doc/sphinx/reference/vcl.rst
index b8650f6..7e73504 100644
--- a/doc/sphinx/reference/vcl.rst
+++ b/doc/sphinx/reference/vcl.rst
@@ -244,6 +244,9 @@ the backend as down it they fail. A probe is defined as this::
         .attribute = "value";
     }
 
+The probe named `default` is special and will be used for all backends
+which do not explicitly reference a probe.
+
 There are no mandatory options. These are the options you can set:
 
   url



From hermunn at varnish-software.com  Wed May 11 15:08:06 2016
From: hermunn at varnish-software.com (Pål Hermunn Johansen)
Date: Wed, 11 May 2016 17:08:06 +0200
Subject: [4.1] 748725e Zero overflow VSM allocations.
Message-ID: <E1b0Vk6-0005zf-HS@project.varnish-software.com>


commit 748725ee2412fccc2e980755d2f14a764ead1a67
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Mar 9 21:02:09 2016 +0000

    Zero overflow VSM allocations.
    
    Submitted by:	P?l Hermunn Johansen

diff --git a/bin/varnishd/common/common_vsm.c b/bin/varnishd/common/common_vsm.c
index 94d583d..0dda841 100644
--- a/bin/varnishd/common/common_vsm.c
+++ b/bin/varnishd/common/common_vsm.c
@@ -233,10 +233,11 @@ VSM_common_alloc(struct vsm_sc *sc, ssize_t size,
 	if (vr == NULL) {
 		/*
 		 * No space in VSM, return malloc'd space
+		 * This space will not be visible via the VSM
 		 */
 		ALLOC_OBJ(vr, VSM_RANGE_MAGIC);
 		AN(vr);
-		vr->ptr = malloc(size);
+		vr->ptr = calloc(size, 1);
 		AN(vr->ptr);
 		vr->len = size;
 		VTAILQ_INSERT_TAIL(&sc->r_bogus, vr, list);



From hermunn at varnish-software.com  Wed May 11 15:09:06 2016
From: hermunn at varnish-software.com (Pål Hermunn Johansen)
Date: Wed, 11 May 2016 17:09:06 +0200
Subject: [4.0] 3880333 Zero overflow VSM allocations.
Message-ID: <E1b0Vl4-00063i-9N@project.varnish-software.com>


commit 3880333fcf6067530139462d08d56d6def099dad
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Mar 9 21:02:09 2016 +0000

    Zero overflow VSM allocations.
    
    Submitted by:	P?l Hermunn Johansen

diff --git a/bin/varnishd/common/common_vsm.c b/bin/varnishd/common/common_vsm.c
index 90863c3..c16261d 100644
--- a/bin/varnishd/common/common_vsm.c
+++ b/bin/varnishd/common/common_vsm.c
@@ -233,10 +233,11 @@ VSM_common_alloc(struct vsm_sc *sc, ssize_t size,
 	if (vr == NULL) {
 		/*
 		 * No space in VSM, return malloc'd space
+		 * This space will not be visible via the VSM
 		 */
 		ALLOC_OBJ(vr, VSM_RANGE_MAGIC);
 		AN(vr);
-		vr->ptr = malloc(size);
+		vr->ptr = calloc(size, 1);
 		AN(vr->ptr);
 		vr->len = size;
 		VTAILQ_INSERT_TAIL(&sc->r_bogus, vr, list);



From varnish-commit at varnish-cache.org  Wed May 11 16:11:43 2016
From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org)
Date: Wed, 11 May 2016 09:11:43 -0700
Subject: Emailing: Photo 05-11-2016, 40 44 38
Message-ID: <dcd30899f0e0$e65fd7bb840ff20a9$@varnish-cache.org>


Your message is ready to be sent with the following file or link
attachments:

Photo 05-11-2016, 40 44 38


Note: To protect against computer viruses, e-mail programs may prevent
sending or receiving certain types of file attachments.  Check your e-mail
security settings to determine how attachments are handled.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Photo 05-11-2016, 40 44 38.zip
Type: application/zip
Size: 3440 bytes
Desc: not available
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-commit/attachments/20160511/46cbf180/attachment.zip>

From phk at FreeBSD.org  Wed May 11 17:38:06 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Wed, 11 May 2016 19:38:06 +0200
Subject: [master] d23dfe3 Only ... hexdumps if they are longer than 8 digits
Message-ID: <E1b0Y5G-0006GJ-25@project.varnish-software.com>


commit d23dfe3f5733cad84538c93bc21df585d773ca2b
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed May 11 17:37:01 2016 +0000

    Only ... hexdumps if they are longer than 8 digits

diff --git a/lib/libvarnish/vsb.c b/lib/libvarnish/vsb.c
index e9e6fc9..7a15f56 100644
--- a/lib/libvarnish/vsb.c
+++ b/lib/libvarnish/vsb.c
@@ -517,7 +517,7 @@ VSB_quote(struct vsb *s, const void *v, int len, int how)
 			if (*w != 0x00)
 				break;
 		VSB_printf(s, "0x");
-		if (w == u + len) {
+		if (w == u + len && len > 4) {
 			VSB_printf(s, "0...0");
 		} else {
 			for (w = u; w < u + len; w++)



From varnish-commit at varnish-cache.org  Thu May 12 05:23:52 2016
From: varnish-commit at varnish-cache.org (lzvrya)
Date: Thu, 12 May 2016 13:23:52 +0800
Subject: =?utf-8?B?dmFybmlzaC1jb21taXQ65aaC5L2V6aKE6Ziy5Yqz?=
 =?utf-8?B?5Yqo6ICF55qE4oCc5bqU6IGY5qy66K+I4oCd77yM?=
Message-ID: <20160512132401706865@cyzkuefaa.org>

varnish-commit:   ??
1.????????????????????????????
2.??????????????????
3.????????????????????????
4.??????????????????????????
5.??????????????
6.??????????????????????????
7.??????????????????????????
8.???????????????????
9.???????????????????????
???????????????????
?????????????????????
2016/5/12 ???13:24:00
                                                                                  ????









 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-commit/attachments/20160512/553552d1/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ??????????????????????.docx
Type: application/vnd.openxmlformats-officedocument.wordprocessingml.document
Size: 27972 bytes
Desc: not available
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-commit/attachments/20160512/553552d1/attachment.docx>

From nils.goroll at uplex.de  Fri May 13 10:32:06 2016
From: nils.goroll at uplex.de (Nils Goroll)
Date: Fri, 13 May 2016 12:32:06 +0200
Subject: [master] 7dec122 allow whitespace and comments before the varnishtest
 marker
Message-ID: <E1b1AO6-0004WC-EX@project.varnish-software.com>


commit 7dec1221ebae2e66f7be81c0311491ed0f684912
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Fri May 13 08:05:31 2016 +0200

    allow whitespace and comments before the varnishtest marker

diff --git a/bin/varnishtest/tests/a00000.vtc b/bin/varnishtest/tests/a00000.vtc
index ca02903..d3fe2fd 100644
--- a/bin/varnishtest/tests/a00000.vtc
+++ b/bin/varnishtest/tests/a00000.vtc
@@ -1,3 +1,6 @@
+# the first token in a varnishtest file must be "varnishtest",
+ 	# but whitespace and comments are fine
+
 varnishtest "basic default HTTP transactions"
 
 server s1 {
diff --git a/bin/varnishtest/vtc_main.c b/bin/varnishtest/vtc_main.c
index 2b32551..be8fabf 100644
--- a/bin/varnishtest/vtc_main.c
+++ b/bin/varnishtest/vtc_main.c
@@ -55,6 +55,7 @@
 #include "vsub.h"
 #include "vtcp.h"
 #include "vtim.h"
+#include "vct.h"
 
 #define		MAX_FILESIZE		(1024 * 1024)
 
@@ -569,6 +570,19 @@ main(int argc, char * const *argv)
 				continue;
 			exit(2);
 		}
+		while (1) {
+			if (vct_islws(*p)) {
+				p++;
+				continue;
+			}
+			if (*p == '#') {
+				if ((p = strchr(p, '\n')))
+					p++;
+				continue;
+			}
+			break;
+		}
+
 		if (strncmp(p, "varnishtest", 11) || !isspace(p[11])) {
 			fprintf(stderr, "File \"%s\" doesn't start with 'varnishtest'\n",
 			    *argv);



From nils.goroll at uplex.de  Sat May 14 17:37:05 2016
From: nils.goroll at uplex.de (Nils Goroll)
Date: Sat, 14 May 2016 19:37:05 +0200
Subject: [master] d26b2b7 document what the default backend is
Message-ID: <E1b1dUv-0007KP-08@project.varnish-software.com>


commit d26b2b7d17185a77784d906f790a378617d12678
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Sat May 14 19:34:44 2016 +0200

    document what the default backend is

diff --git a/doc/sphinx/users-guide/vcl-backends.rst b/doc/sphinx/users-guide/vcl-backends.rst
index b44ba9c..5844891 100644
--- a/doc/sphinx/users-guide/vcl-backends.rst
+++ b/doc/sphinx/users-guide/vcl-backends.rst
@@ -74,6 +74,10 @@ really arbitrary data. You want to send mobile devices to a different
 backend? No problem. ``if (req.http.User-agent ~ /mobile/) ..`` should do the
 trick.
 
+Without an explicit backend selection, varnish will continue using
+the `default` backend. If there is no backend named `default`, the
+first backend found in the vcl will be used as the default backend.
+
 
 Backends and virtual hosts in Varnish
 -------------------------------------



From nils.goroll at uplex.de  Mon May 16 11:13:05 2016
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon, 16 May 2016 13:13:05 +0200
Subject: [master] 185978d ban_lurker: avoid the de-tour over
 ban_mark_completed() for the tail
Message-ID: <E1b2GSP-0002kL-KL@project.varnish-software.com>


commit 185978d79aed0cb51e3e3343f55d081434cd2e0c
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Tue May 3 17:25:54 2016 +0200

    ban_lurker: avoid the de-tour over ban_mark_completed() for the tail
    
    ban_lurker_work completes bans by moving ocs up the ban list. Unless
    there are REQ bans, this leaves a tail of completed bans which,
    previously, got marked by ban_mark_completed() only to get removed by
    ban_clean_tail() in the next step
    
    We now clean out the completed tail in the first place, avoiding the
    extra marking step.

diff --git a/bin/varnishd/cache/cache_ban.c b/bin/varnishd/cache/cache_ban.c
index 6cd12f0..a9740e1 100644
--- a/bin/varnishd/cache/cache_ban.c
+++ b/bin/varnishd/cache/cache_ban.c
@@ -321,6 +321,7 @@ ban_export(void)
  * For both of these we do a full export on info failure to remove
  * holes in the exported list.
  * XXX: we should keep track of the size of holes in the last exported list
+ * XXX: check if the ban_export should be batched in ban_cleantail
  */
 void
 ban_info_new(const uint8_t *ban, unsigned len)
diff --git a/bin/varnishd/cache/cache_ban_lurker.c b/bin/varnishd/cache/cache_ban_lurker.c
index f184fbe..6b33465 100644
--- a/bin/varnishd/cache/cache_ban_lurker.c
+++ b/bin/varnishd/cache/cache_ban_lurker.c
@@ -51,13 +51,27 @@ ban_kick_lurker(void)
 	AZ(pthread_cond_signal(&ban_lurker_cond));
 }
 
-static void
-ban_cleantail(void)
+/*
+ * ban_cleantail: clean the tail of the ban list up to the first ban which is
+ * still referenced. For already completed bans, we update statistics
+ * accordingly, but otherwise just skip the completion step and remove directly
+ *
+ * return 1 if we removed the victim, 0 otherwise
+ */
+
+static int
+ban_cleantail(const struct ban *victim)
 {
-	struct ban *b;
+	struct ban *b, *bt;
+	struct banhead_s freelist = VTAILQ_HEAD_INITIALIZER(freelist);
+	int r = 0;
+
+	/* handle the zero-length tail unprotected */
+	if (VTAILQ_LAST(&ban_head, banhead_s) == VTAILQ_FIRST(&ban_head))
+		return (r);
 
+	Lck_Lock(&ban_mtx);
 	do {
-		Lck_Lock(&ban_mtx);
 		b = VTAILQ_LAST(&ban_head, banhead_s);
 		if (b != VTAILQ_FIRST(&ban_head) && b->refcount == 0) {
 			assert(VTAILQ_EMPTY(&b->objcore));
@@ -70,16 +84,24 @@ ban_cleantail(void)
 			VSC_C_main->bans--;
 			VSC_C_main->bans_deleted++;
 			VTAILQ_REMOVE(&ban_head, b, list);
+			VTAILQ_INSERT_TAIL(&freelist, b, list);
 			VSC_C_main->bans_persisted_fragmentation +=
 			    ban_len(b->spec);
 			ban_info_drop(b->spec, ban_len(b->spec));
 		} else {
 			b = NULL;
 		}
-		Lck_Unlock(&ban_mtx);
-		if (b != NULL)
-			BAN_Free(b);
 	} while (b != NULL);
+
+	Lck_Unlock(&ban_mtx);
+
+	VTAILQ_FOREACH_SAFE(b, &freelist, list, bt) {
+		if (b == victim)
+			r = 1;
+		BAN_Free(b);
+	}
+
+	return (r);
 }
 
 /*--------------------------------------------------------------------
@@ -208,7 +230,13 @@ ban_lurker_test_ban(struct worker *wrk, struct vsl_log *vsl, struct ban *bt,
 }
 
 /*--------------------------------------------------------------------
- * Ban lurker thread
+ * Ban lurker thread:
+ *
+ * try to move ocs as far up the ban list as possible (to bd)
+ *
+ * BANS_FLAG_REQ bans act as barriers, for bans further down, ocs get moved to
+ * them. But still all bans up to the initial bd get checked and marked
+ * completed.
  */
 
 static double
@@ -219,8 +247,10 @@ ban_lurker_work(struct worker *wrk, struct vsl_log *vsl)
 	double d, dt, n;
 
 	dt = 49.62;		// Random, non-magic
-	if (cache_param->ban_lurker_sleep == 0)
+	if (cache_param->ban_lurker_sleep == 0) {
+		(void) ban_cleantail(NULL);
 		return (dt);
+	}
 
 	Lck_Lock(&ban_mtx);
 	b = ban_start;
@@ -248,9 +278,25 @@ ban_lurker_work(struct worker *wrk, struct vsl_log *vsl)
 		}
 	}
 
+	/*
+	 * conceptually, all obans are now completed. Remove the tail. If it
+	 * containted the first oban, all obans were on the tail and we're
+	 * done.
+	 */
+	if (ban_cleantail(VTAILQ_FIRST(&obans)))
+		return (dt);
+
+	/*
+	 * Mark remaining bans completed: the tail of the obans list is now
+	 * removed, but iterating over it is safe until we hit the new tail ban
+	 */
 	Lck_Lock(&ban_mtx);
-	VTAILQ_FOREACH(b, &obans, l_list)
+	bd = VTAILQ_LAST(&ban_head, banhead_s);
+	VTAILQ_FOREACH(b, &obans, l_list) {
 		ban_mark_completed(b);
+		if (b == bd)
+			break;
+	}
 	Lck_Unlock(&ban_mtx);
 	return (dt);
 }
@@ -269,7 +315,6 @@ ban_lurker(struct worker *wrk, void *priv)
 
 	while (!ban_shutdown) {
 		d = ban_lurker_work(wrk, &vsl);
-		ban_cleantail();
 		if (DO_DEBUG(DBG_LURKER))
 			VSLb(&vsl, SLT_Debug, "lurker: sleep = %lf", d);
 		d += VTIM_real();



From fgsch at lodoss.net  Mon May 16 17:46:06 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Mon, 16 May 2016 19:46:06 +0200
Subject: [master] cd13857 Remove unneeded checks
Message-ID: <E1b2Mak-0007h3-LX@project.varnish-software.com>


commit cd1385733878544d9bb49bcdd4adb4c5cfe8b633
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Mon May 16 18:37:44 2016 +0100

    Remove unneeded checks

diff --git a/configure.ac b/configure.ac
index fa78ba0..ceddc8f 100644
--- a/configure.ac
+++ b/configure.ac
@@ -239,9 +239,6 @@ AC_CHECK_FUNCS([strerror])
 AC_FUNC_STRERROR_R
 AC_CHECK_FUNCS([dladdr])
 AC_CHECK_FUNCS([socket])
-AC_CHECK_FUNCS([strptime])
-AC_CHECK_FUNCS([fmtcheck])
-AC_CHECK_FUNCS([getdtablesize])
 AC_CHECK_FUNCS([nanosleep])
 AC_CHECK_FUNCS([setppriv])
 AC_CHECK_FUNCS([fallocate])



From fgsch at lodoss.net  Mon May 16 17:46:06 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Mon, 16 May 2016 19:46:06 +0200
Subject: [master] c5b9ead Add a VSL variant that takes uses stdarg(3)
Message-ID: <E1b2Mak-0007h9-NP@project.varnish-software.com>


commit c5b9ead0a08a8b472e6a66907b17e162ff0ce8c0
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Mon May 16 14:59:48 2016 +0100

    Add a VSL variant that takes uses stdarg(3)

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index a0d47ab..d2903b9 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -984,6 +984,7 @@ void *VSM_Alloc(unsigned size, const char *class, const char *type,
     const char *ident);
 void VSM_Free(void *ptr);
 #ifdef VSL_ENDMARKER
+void VSLv(enum VSL_tag_e tag, uint32_t vxid, const char *fmt, va_list va);
 void VSL(enum VSL_tag_e tag, uint32_t vxid, const char *fmt, ...)
     __v_printflike(3, 4);
 void VSLbv(struct vsl_log *, enum VSL_tag_e tag, const char *fmt, va_list va);
diff --git a/bin/varnishd/cache/cache_shmlog.c b/bin/varnishd/cache/cache_shmlog.c
index e733686..789e902 100644
--- a/bin/varnishd/cache/cache_shmlog.c
+++ b/bin/varnishd/cache/cache_shmlog.c
@@ -207,9 +207,8 @@ vslr(enum VSL_tag_e tag, uint32_t vxid, const char *b, unsigned len)
  */
 
 void
-VSL(enum VSL_tag_e tag, uint32_t vxid, const char *fmt, ...)
+VSLv(enum VSL_tag_e tag, uint32_t vxid, const char *fmt, va_list ap)
 {
-	va_list ap;
 	unsigned n, mlen = cache_param->vsl_reclen;
 	char buf[mlen];
 
@@ -217,18 +216,26 @@ VSL(enum VSL_tag_e tag, uint32_t vxid, const char *fmt, ...)
 	if (vsl_tag_is_masked(tag))
 		return;
 
-
 	if (strchr(fmt, '%') == NULL) {
 		vslr(tag, vxid, fmt, strlen(fmt) + 1);
 	} else {
-		va_start(ap, fmt);
 		n = vsnprintf(buf, mlen, fmt, ap);
-		va_end(ap);
 		if (n > mlen - 1)
 			n = mlen - 1;
 		buf[n++] = '\0'; /* NUL-terminated */
 		vslr(tag, vxid, buf, n);
 	}
+
+}
+
+void
+VSL(enum VSL_tag_e tag, uint32_t vxid, const char *fmt, ...)
+{
+	va_list ap;
+
+	va_start(ap, fmt);
+	VSLv(tag, vxid, fmt, ap);
+	va_end(ap);
 }
 
 /*--------------------------------------------------------------------*/



From fgsch at lodoss.net  Mon May 16 17:46:06 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Mon, 16 May 2016 19:46:06 +0200
Subject: [master] dad7f7d Make std.log() work from vcl_{init,fini}
Message-ID: <E1b2Mak-0007hI-Ow@project.varnish-software.com>


commit dad7f7d810e918658049e48f91cdfc2c391ec07a
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Mon May 16 18:40:11 2016 +0100

    Make std.log() work from vcl_{init,fini}
    
    Fixes #1924.

diff --git a/bin/varnishtest/tests/r01924.vtc b/bin/varnishtest/tests/r01924.vtc
new file mode 100644
index 0000000..ed5095b
--- /dev/null
+++ b/bin/varnishtest/tests/r01924.vtc
@@ -0,0 +1,22 @@
+varnishtest "Test std.log from vcl_{init,fini}"
+
+server s1 {
+	rxreq
+	txresp
+} -start
+
+varnish v1 -vcl+backend {
+	import std;
+
+	sub vcl_init {
+		std.log("init");
+	}
+
+	sub vcl_fini {
+		std.log("fini");
+	}
+} -start
+
+varnish v1 -vcl+backend { }
+
+varnish v1 -cliok "vcl.discard vcl1"
diff --git a/lib/libvmod_std/vmod_std.c b/lib/libvmod_std/vmod_std.c
index 1a8f56b..4522147 100644
--- a/lib/libvmod_std/vmod_std.c
+++ b/lib/libvmod_std/vmod_std.c
@@ -139,17 +139,20 @@ vmod_log(VRT_CTX, const char *fmt, ...)
 	txt t;
 
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
-	u = WS_Reserve(ctx->ws, 0);
-	t.b = ctx->ws->f;
 	va_start(ap, fmt);
-	t.e = VRT_StringList(ctx->ws->f, u, fmt, ap);
+	if (ctx->vsl != NULL) {
+		u = WS_Reserve(ctx->ws, 0);
+		t.b = ctx->ws->f;
+		t.e = VRT_StringList(ctx->ws->f, u, fmt, ap);
+		if (t.e != NULL) {
+			assert(t.e > t.b);
+			t.e--;
+			VSLbt(ctx->vsl, SLT_VCL_Log, t);
+		}
+		WS_Release(ctx->ws, 0);
+	} else
+		VSLv(SLT_VCL_Log, 0, fmt, ap);
 	va_end(ap);
-	if (t.e != NULL) {
-		assert(t.e > t.b);
-		t.e--;
-		VSLbt(ctx->vsl, SLT_VCL_Log, t);
-	}
-	WS_Release(ctx->ws, 0);
 }
 
 VCL_VOID __match_proto__(td_std_syslog)



From fgsch at lodoss.net  Mon May 16 19:40:09 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Mon, 16 May 2016 21:40:09 +0200
Subject: [master] 3740956 Make std.syslog() work from vcl_{init,fini}
Message-ID: <E1b2ON7-0005j6-2N@project.varnish-software.com>


commit 37409564590a9ccac69fc595a118beeacd2676ae
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Mon May 16 20:38:09 2016 +0100

    Make std.syslog() work from vcl_{init,fini}
    
    Related to #1924.

diff --git a/bin/varnishtest/tests/r01924.vtc b/bin/varnishtest/tests/r01924.vtc
index ed5095b..d553124 100644
--- a/bin/varnishtest/tests/r01924.vtc
+++ b/bin/varnishtest/tests/r01924.vtc
@@ -1,4 +1,4 @@
-varnishtest "Test std.log from vcl_{init,fini}"
+varnishtest "Test std.{log,syslog} from vcl_{init,fini}"
 
 server s1 {
 	rxreq
@@ -10,10 +10,12 @@ varnish v1 -vcl+backend {
 
 	sub vcl_init {
 		std.log("init");
+		std.syslog(8 + 7, "init");
 	}
 
 	sub vcl_fini {
 		std.log("fini");
+		std.syslog(8 + 7, "fini");
 	}
 } -start
 
diff --git a/configure.ac b/configure.ac
index ceddc8f..ad83ab7 100644
--- a/configure.ac
+++ b/configure.ac
@@ -243,6 +243,7 @@ AC_CHECK_FUNCS([nanosleep])
 AC_CHECK_FUNCS([setppriv])
 AC_CHECK_FUNCS([fallocate])
 AC_CHECK_FUNCS([closefrom])
+AC_CHECK_FUNCS([vsyslog])
 
 save_LIBS="${LIBS}"
 LIBS="${PTHREAD_LIBS}"
diff --git a/lib/libvmod_std/vmod_std.c b/lib/libvmod_std/vmod_std.c
index 4522147..ed66405 100644
--- a/lib/libvmod_std/vmod_std.c
+++ b/lib/libvmod_std/vmod_std.c
@@ -163,14 +163,17 @@ vmod_syslog(VRT_CTX, VCL_INT fac, const char *fmt, ...)
 	txt t;
 
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
-	u = WS_Reserve(ctx->ws, 0);
-	t.b = ctx->ws->f;
 	va_start(ap, fmt);
-	t.e = VRT_StringList(ctx->ws->f, u, fmt, ap);
+	if (ctx->ws != NULL) {
+		u = WS_Reserve(ctx->ws, 0);
+		t.b = ctx->ws->f;
+		t.e = VRT_StringList(ctx->ws->f, u, fmt, ap);
+		if (t.e != NULL)
+			syslog((int)fac, "%s", t.b);
+		WS_Release(ctx->ws, 0);
+	} else
+		vsyslog((int)fac, fmt, ap);
 	va_end(ap);
-	if (t.e != NULL)
-		syslog((int)fac, "%s", t.b);
-	WS_Release(ctx->ws, 0);
 }
 
 VCL_VOID __match_proto__(td_std_collect)



From varnish-commit at varnish-cache.org  Tue May 17 02:28:07 2016
From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org)
Date: Tue, 17 May 2016 09:28:07 +0700
Subject: Your .pdf document is attached
Message-ID: <201605171398433.C2B75CED@m5354411.varnish-cache.org>

An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-commit/attachments/20160517/c1761381/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 03BC7F2.zip
Type: application/x-compressed
Size: 3001 bytes
Desc: not available
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-commit/attachments/20160517/c1761381/attachment.bin>

From nils.goroll at uplex.de  Tue May 17 12:45:09 2016
From: nils.goroll at uplex.de (Nils Goroll)
Date: Tue, 17 May 2016 14:45:09 +0200
Subject: [master] bacec6a invalidate the cli_fd of dead varnishds
Message-ID: <E1b2eN3-00048g-Uu@project.varnish-software.com>


commit bacec6ac3eddaa171374246a0494d4000f03f83c
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Tue May 17 14:38:04 2016 +0200

    invalidate the cli_fd of dead varnishds
    
    In particular, this avoids a later backend.list attempt on a file descriptor
    to a dead process and, consequently, avoids a long delay in varnishtest
    when varnishd cant be started (for instance, because it is not in PATH).

diff --git a/bin/varnishtest/vtc_varnish.c b/bin/varnishtest/vtc_varnish.c
index 794f615..d486b01 100644
--- a/bin/varnishtest/vtc_varnish.c
+++ b/bin/varnishtest/vtc_varnish.c
@@ -471,18 +471,26 @@ varnish_launch(struct varnish *v)
 	    i, fd[0].revents, fd[1].revents);
 	if (i == 0) {
 		vtc_log(v->vl, 0, "FAIL timeout waiting for CLI connection");
+		AZ(close(v->cli_fd));
+		v->cli_fd = -1;
 		return;
 	}
 	if (fd[1].revents & POLLHUP) {
 		vtc_log(v->vl, 0, "FAIL debug pipe closed");
+		AZ(close(v->cli_fd));
+		v->cli_fd = -1;
 		return;
 	}
 	if (!(fd[0].revents & POLLIN)) {
 		vtc_log(v->vl, 0, "FAIL CLI connection wait failure");
+		AZ(close(v->cli_fd));
+		v->cli_fd = -1;
 		return;
 	}
 	nfd = accept(v->cli_fd, NULL, NULL);
 	if (nfd < 0) {
+		AZ(close(v->cli_fd));
+		v->cli_fd = -1;
 		vtc_log(v->vl, 0, "FAIL no CLI connection accepted");
 		return;
 	}



From phk at FreeBSD.org  Wed May 18 07:05:08 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Wed, 18 May 2016 09:05:08 +0200
Subject: [master] 7996af0 Allow VCL set Last-Modified to be used for I-M-S
 processing.
Message-ID: <E1b2vXY-0006kZ-79@project.varnish-software.com>


commit 7996af0271325027b750c1f31318927edbafaa8d
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed May 18 07:03:30 2016 +0000

    Allow VCL set Last-Modified to be used for I-M-S processing.
    
    Submitted by:	rezan
    Fixes:		#1931

diff --git a/bin/varnishd/cache/cache_rfc2616.c b/bin/varnishd/cache/cache_rfc2616.c
index 6b80371..884aad1 100644
--- a/bin/varnishd/cache/cache_rfc2616.c
+++ b/bin/varnishd/cache/cache_rfc2616.c
@@ -256,7 +256,7 @@ rfc2616_weak_compare(const char *p, const char *e)
 int
 RFC2616_Do_Cond(const struct req *req)
 {
-	const char *p, *e;
+	const char *p, *e, *l;
 	double ims, lm;
 
 	/*
@@ -276,6 +276,12 @@ RFC2616_Do_Cond(const struct req *req)
 		ims = VTIM_parse(p);
 		if (!ims || ims > req->t_req)	/* [RFC7232 3.3 p16] */
 			return (0);
+		if (http_GetHdr(req->resp, H_Last_Modified, &l)) {
+			lm = VTIM_parse(l);
+			if (!lm || lm > ims)
+				return (0);
+			return (1);
+		}
 		AZ(ObjGetDouble(req->wrk, req->objcore, OA_LASTMODIFIED, &lm));
 		if (lm > ims)
 			return (0);
diff --git a/bin/varnishtest/tests/v00047.vtc b/bin/varnishtest/tests/v00047.vtc
new file mode 100644
index 0000000..9ca2a6c
--- /dev/null
+++ b/bin/varnishtest/tests/v00047.vtc
@@ -0,0 +1,40 @@
+varnishtest "Changing Last Modified in vcl_deliver"
+ 
+server s1 {
+	rxreq
+	txresp -hdr "Last-Modified: Wed, 27 Apr 2016 14:00:00 GMT"
+} -start
+ 
+varnish v1 -vcl+backend {
+	sub vcl_deliver {
+		if (req.http.deliver == "modify") {
+			set resp.http.Last-Modified = "Wed, 27 Apr 2016 16:00:00 GMT";
+		}
+	}
+} -start
+ 
+client c1 {
+	txreq
+	rxresp
+	expect resp.status == 200
+
+	txreq -hdr "If-Modified-Since: Wed, 27 Apr 2016 14:00:00 GMT"
+	rxresp
+	expect resp.status == 304
+
+	txreq -hdr "If-Modified-Since: Wed, 27 Apr 2016 12:00:00 GMT"
+	rxresp
+	expect resp.status == 200
+
+	txreq -hdr "If-Modified-Since: Wed, 27 Apr 2016 14:00:00 GMT" -hdr "deliver: modify"
+	rxresp
+	expect resp.status == 200
+
+	txreq -hdr "If-Modified-Since: Wed, 27 Apr 2016 16:00:00 GMT" -hdr "deliver: modify"
+	rxresp
+	expect resp.status == 304
+
+	txreq -hdr "If-Modified-Since: Wed, 27 Apr 2016 18:00:00 GMT" -hdr "deliver: modify"
+	rxresp
+	expect resp.status == 304
+} -run



From varnish-commit at varnish-cache.org  Wed May 18 16:53:36 2016
From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org)
Date: Wed, 18 May 2016 12:53:36 -0400
Subject: Emailing: DOC 05-18-2016, 10 12 71
Message-ID: <0dff652c0532$c935aa470d4dc00d7$@varnish-cache.org>


Your message is ready to be sent with the following file or link
attachments:

DOC 05-18-2016, 10 12 71


Note: To protect against computer viruses, e-mail programs may prevent
sending or receiving certain types of file attachments.  Check your e-mail
security settings to determine how attachments are handled.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: DOC 05-18-2016, 10 12 71.zip
Type: application/x-compress
Size: 2761 bytes
Desc: not available
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-commit/attachments/20160518/b2b8de53/attachment.bin>

From phk at FreeBSD.org  Wed May 18 21:08:06 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Wed, 18 May 2016 23:08:06 +0200
Subject: [master] 06874f1 Simplify VCC setup a little bit.
Message-ID: <E1b38hK-0007nV-Qp@project.varnish-software.com>


commit 06874f1d8fd9c2e89bf9625f685c35b5194112ea
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed May 18 07:23:55 2016 +0000

    Simplify VCC setup a little bit.

diff --git a/bin/varnishd/mgt/mgt.h b/bin/varnishd/mgt/mgt.h
index d174206..e230ee6 100644
--- a/bin/varnishd/mgt/mgt.h
+++ b/bin/varnishd/mgt/mgt.h
@@ -167,7 +167,6 @@ void STV_Config_Transient(void);
 /* mgt_vcc.c */
 char *mgt_VccCompile(struct cli *, const char *vclname, const char *vclsrc,
     const char *vclsrcfile, int C_flag);
-void mgt_vcc_init(void);
 
 void mgt_vcl_init(void);
 void mgt_vcc_startup(struct cli *, const char *b_arg, const char *f_arg,
diff --git a/bin/varnishd/mgt/mgt_main.c b/bin/varnishd/mgt/mgt_main.c
index 36005a3..d696839 100644
--- a/bin/varnishd/mgt/mgt_main.c
+++ b/bin/varnishd/mgt/mgt_main.c
@@ -747,7 +747,6 @@ main(int argc, char * const *argv)
 		    P_arg, strerror(errno));
 	VJ_master(JAIL_MASTER_LOW);
 
-	mgt_vcc_init();
 	mgt_vcl_init();
 
 	if (b_arg != NULL || f_arg != NULL) {
diff --git a/bin/varnishd/mgt/mgt_vcc.c b/bin/varnishd/mgt/mgt_vcc.c
index 9a094fb..8847c03 100644
--- a/bin/varnishd/mgt/mgt_vcc.c
+++ b/bin/varnishd/mgt/mgt_vcc.c
@@ -65,7 +65,6 @@ unsigned mgt_vcc_err_unref;
 unsigned mgt_vcc_allow_inline_c;
 unsigned mgt_vcc_unsafe_path;
 
-static struct vcp *vcp;
 
 #define VGC_SRC		"vgc.c"
 #define VGC_LIB		"vgc.so"
@@ -87,6 +86,7 @@ run_vcc(void *priv)
 	struct vsb *sb;
 	struct vcc_priv *vp;
 	int fd, i, l;
+	struct vcp *vcp;
 
 	VJ_subproc(JAIL_SUBPROC_VCC);
 	CAST_OBJ_NOTNULL(vp, priv, VCC_PRIV_MAGIC);
@@ -95,6 +95,9 @@ run_vcc(void *priv)
 
 	sb = VSB_new_auto();
 	XXXAN(sb);
+	vcp = VCP_New();
+	AN(vcp);
+	VCP_Builtin_VCL(vcp, builtin_vcl);
 	VCP_VCL_path(vcp, mgt_vcl_path);
 	VCP_VMOD_path(vcp, mgt_vmod_path);
 	VCP_Err_Unref(vcp, mgt_vcc_err_unref);
@@ -314,14 +317,3 @@ mgt_VccCompile(struct cli *cli, const char *vclname, const char *vclsrc,
 
 	return (vp.libfile);
 }
-
-/*--------------------------------------------------------------------*/
-
-void
-mgt_vcc_init(void)
-{
-
-	vcp = VCP_New();
-	AN(vcp);
-	VCP_Builtin_VCL(vcp, builtin_vcl);
-}



From phk at FreeBSD.org  Wed May 18 21:08:06 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Wed, 18 May 2016 23:08:06 +0200
Subject: [master] 3ffe8e1 Eliminate the distinction between SYM_FUNC and
 SYM_PROC and allow people to throw return values away if the want to,
 but not to use VOID in expressions.
Message-ID: <E1b38hK-0007nb-T2@project.varnish-software.com>


commit 3ffe8e1e3435c89110b1dce23b55c8ba56ec50da
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed May 18 21:06:29 2016 +0000

    Eliminate the distinction between SYM_FUNC and SYM_PROC and allow
    people to throw return values away if the want to, but not to
    use VOID in expressions.

diff --git a/bin/varnishtest/tests/v00020.vtc b/bin/varnishtest/tests/v00020.vtc
index ff178f6..540a560 100644
--- a/bin/varnishtest/tests/v00020.vtc
+++ b/bin/varnishtest/tests/v00020.vtc
@@ -275,6 +275,14 @@ varnish v1 -errvcl {VCL sub's named 'vcl*' are reserved names.} {
 	}
 }
 
+varnish v1 -errvcl {Function returns VOID} {
+	import debug;
+	backend b { .host = "127.0.0.1"; }
+	sub vcl_recv {
+		set req.http.foo = debug.sleep(1m);
+	}
+}
+
 varnish v1 -errvcl {Names of VCL acl's cannot contain '-'} {
 	backend b { .host = "127.0.0.1"; }
 	acl foo-bar {
diff --git a/include/tbl/symbol_kind.h b/include/tbl/symbol_kind.h
index 36f19e5..497f400 100644
--- a/include/tbl/symbol_kind.h
+++ b/include/tbl/symbol_kind.h
@@ -30,8 +30,7 @@
 /*lint -save -e525 -e539 */
 VCC_SYMB(NONE,		none)
 VCC_SYMB(VAR,		var)
-VCC_SYMB(FUNC,		func)		/* VMOD function */
-VCC_SYMB(PROC,		proc)		/* VMOD procedure */
+VCC_SYMB(FUNC,		func)		/* VMOD function/procedure */
 VCC_SYMB(VMOD,		vmod)
 VCC_SYMB(ACL,		acl)
 VCC_SYMB(SUB,		sub)		/* VCL subroutine */
diff --git a/lib/libvcc/vcc_action.c b/lib/libvcc/vcc_action.c
index 516f628..8e62d2d 100644
--- a/lib/libvcc/vcc_action.c
+++ b/lib/libvcc/vcc_action.c
@@ -240,10 +240,6 @@ parse_new(struct vcc *tl)
 		sy3->cfunc = p;
 		p += strlen(p) + 1;
 
-		/* Functions which return VOID are procedures */
-		if (!memcmp(p, "VOID\0", 5))
-			sy3->kind = SYM_PROC;
-
 		sy3->args = p;
 		sy3->extra = TlDup(tl, buf1);
 		while (p[0] != '\0' || p[1] != '\0') {
@@ -435,7 +431,7 @@ vcc_ParseAction(struct vcc *tl)
 		}
 	}
 	sym = VCC_FindSymbol(tl, tl->t, SYM_NONE);
-	if (sym != NULL && sym->kind == SYM_PROC) {
+	if (sym != NULL && sym->kind == SYM_FUNC) {
 		vcc_Expr_Call(tl, sym);
 		return (1);
 	}
diff --git a/lib/libvcc/vcc_compile.h b/lib/libvcc/vcc_compile.h
index 20bc18c..31f87e1 100644
--- a/lib/libvcc/vcc_compile.h
+++ b/lib/libvcc/vcc_compile.h
@@ -131,7 +131,7 @@ struct symbol {
 	struct proc			*proc;
 	unsigned			nref, ndef;
 
-	/* SYM_PROC, SYM_FUNC */
+	/* SYM_FUNC */
 	const char			*cfunc;
 	const char			*extra;
 	const char			*args;
@@ -289,6 +289,7 @@ void vcc_Eval_Func(struct vcc *tl, const char *cfunc, const char *extra,
 sym_expr_t vcc_Eval_Acl;
 sym_expr_t vcc_Eval_Backend;
 sym_expr_t vcc_Eval_Probe;
+enum var_type VCC_arg_type(const char **p);
 
 /* vcc_obj.c */
 extern const struct var vcc_vars[];
diff --git a/lib/libvcc/vcc_expr.c b/lib/libvcc/vcc_expr.c
index 62fbd0e..e371fb9 100644
--- a/lib/libvcc/vcc_expr.c
+++ b/lib/libvcc/vcc_expr.c
@@ -381,8 +381,8 @@ vcc_expr_fmt(struct vsb *d, int ind, const struct expr *e1)
 /*--------------------------------------------------------------------
  */
 
-static enum var_type
-vcc_arg_type(const char **p)
+enum var_type
+VCC_arg_type(const char **p)
 {
 
 #define VCC_TYPE(a) if (!strcmp(#a, *p)) { *p += strlen(#a) + 1; return (a);}
@@ -674,13 +674,13 @@ vcc_func(struct vcc *tl, struct expr **e, const char *cfunc,
 	p = args;
 	if (extra == NULL)
 		extra = "";
-	rfmt = vcc_arg_type(&p);
+	rfmt = VCC_arg_type(&p);
 	VTAILQ_INIT(&head);
 	while (*p != '\0') {
 		fa = calloc(sizeof *fa, 1);
 		AN(fa);
 		VTAILQ_INSERT_TAIL(&head, fa, list);
-		fa->type = vcc_arg_type(&p);
+		fa->type = VCC_arg_type(&p);
 		if (fa->type == VOID && !memcmp(p, "PRIV_", 5)) {
 			fa->result = vcc_priv_arg(tl, p, name);
 			fa->name = "";
@@ -796,7 +796,8 @@ void
 vcc_Eval_SymFunc(struct vcc *tl, struct expr **e, const struct symbol *sym)
 {
 
-	assert(sym->kind == SYM_FUNC || sym->kind == SYM_PROC);
+	assert(sym->kind == SYM_FUNC);
+	/* XXX */
 	AN(sym->cfunc);
 	AN(sym->name);
 	AN(sym->args);
@@ -862,6 +863,11 @@ vcc_expr4(struct vcc *tl, struct expr **e, enum var_type fmt)
 			return;
 		}
 		AN(sym);
+		if (sym->kind == SYM_FUNC && sym->fmt == VOID) {
+			VSB_printf(tl->sb, "Function returns VOID:\n");
+			vcc_ErrWhere(tl, tl->t);
+			return;
+		}
 		switch(sym->kind) {
 		case SYM_VAR:
 		case SYM_FUNC:
@@ -871,6 +877,7 @@ vcc_expr4(struct vcc *tl, struct expr **e, enum var_type fmt)
 			AN(sym->eval);
 			AZ(*e);
 			sym->eval(tl, e, sym);
+			ERRCHK(tl);
 			/* Unless asked for a HEADER, fold to string here */
 			if (*e && fmt != HEADER && (*e)->fmt == HEADER) {
 				vcc_expr_tostring(tl, e, STRING);
@@ -961,6 +968,7 @@ vcc_expr_mul(struct vcc *tl, struct expr **e, enum var_type fmt)
 	*e = NULL;
 	vcc_expr4(tl, e, fmt);
 	ERRCHK(tl);
+	AN(*e);
 	f3 = f2 = (*e)->fmt;
 
 	switch(f2) {
diff --git a/lib/libvcc/vcc_vmod.c b/lib/libvcc/vcc_vmod.c
index 38515b2..aca8890 100644
--- a/lib/libvcc/vcc_vmod.c
+++ b/lib/libvcc/vcc_vmod.c
@@ -244,10 +244,7 @@ vcc_ParseImport(struct vcc *tl)
 			sym->cfunc = p;
 			p += strlen(p) + 1;
 			sym->args = p;
-
-			/* Functions which return VOID are procedures */
-			if (!memcmp(p, "VOID\0", 5))
-				sym->kind = SYM_PROC;
+			sym->fmt = VCC_arg_type(&p);
 		}
 	}
 



From phk at FreeBSD.org  Wed May 18 21:47:06 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Wed, 18 May 2016 23:47:06 +0200
Subject: [master] 9e88eb0 Collaps a bunch of equivalent "Eval" functions.
Message-ID: <E1b39J4-0001cd-5r@project.varnish-software.com>


commit 9e88eb006a6883bf8070d9b53d0e47153636392d
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed May 18 21:45:54 2016 +0000

    Collaps a bunch of equivalent "Eval" functions.

diff --git a/lib/libvcc/vcc_acl.c b/lib/libvcc/vcc_acl.c
index 97728a9..f3e3488 100644
--- a/lib/libvcc/vcc_acl.c
+++ b/lib/libvcc/vcc_acl.c
@@ -475,7 +475,7 @@ vcc_ParseAcl(struct vcc *tl)
 {
 	struct token *an;
 	struct symbol *sym;
-	char acln[1024];
+	char *acln;
 
 	vcc_NextToken(tl);
 	VTAILQ_INIT(&tl->acl);
@@ -490,7 +490,7 @@ vcc_ParseAcl(struct vcc *tl)
 	an = tl->t;
 	vcc_NextToken(tl);
 
-	bprintf(acln, "%.*s", PF(an));
+	acln = TlDupTok(tl, an);
 
 	sym = VCC_GetSymbolTok(tl, an, SYM_ACL);
 	AN(sym);
@@ -499,9 +499,7 @@ vcc_ParseAcl(struct vcc *tl)
 		vcc_ErrWhere(tl, an);
 		return;
 	}
-	sym->fmt = ACL;
-	sym->eval = vcc_Eval_Acl;
-	sym->eval_priv = TlDup(tl, acln);
+	VCC_GenericSymbol(tl, sym, ACL, "&vrt_acl_named_%s", acln);
 	sym->ndef++;
 	ERRCHK(tl);
 
diff --git a/lib/libvcc/vcc_backend.c b/lib/libvcc/vcc_backend.c
index a76e95a..c860dc2 100644
--- a/lib/libvcc/vcc_backend.c
+++ b/lib/libvcc/vcc_backend.c
@@ -266,8 +266,7 @@ vcc_ParseProbe(struct vcc *tl)
 		vcc_ErrWhere(tl, t_probe);
 		return;
 	}
-	sym->fmt = PROBE;
-	sym->eval = vcc_Eval_Probe;
+	VCC_GenericSymbol(tl, sym, PROBE, "%s", sym->name);
 	sym->ndef++;
 	ERRCHK(tl);
 
@@ -497,9 +496,7 @@ vcc_ParseBackend(struct vcc *tl)
 		vcc_ErrWhere(tl, t_be);
 		return;
 	}
-	sym->fmt = BACKEND;
-	sym->eval = vcc_Eval_Backend;
-	sym->eval_priv = TlDup(tl, vgcname);
+	VCC_GenericSymbol(tl, sym, BACKEND, "%s", vgcname);
 	sym->ndef++;
 	ERRCHK(tl);
 
diff --git a/lib/libvcc/vcc_compile.h b/lib/libvcc/vcc_compile.h
index 31f87e1..bfb661a 100644
--- a/lib/libvcc/vcc_compile.h
+++ b/lib/libvcc/vcc_compile.h
@@ -283,12 +283,10 @@ void vcc_Expr(struct vcc *tl, enum var_type typ);
 void vcc_Expr_Call(struct vcc *tl, const struct symbol *sym);
 void vcc_Expr_Init(struct vcc *tl);
 sym_expr_t vcc_Eval_Var;
+sym_expr_t vcc_Eval_Generic;
 sym_expr_t vcc_Eval_SymFunc;
 void vcc_Eval_Func(struct vcc *tl, const char *cfunc, const char *extra,
     const char *name, const char *args);
-sym_expr_t vcc_Eval_Acl;
-sym_expr_t vcc_Eval_Backend;
-sym_expr_t vcc_Eval_Probe;
 enum var_type VCC_arg_type(const char **p);
 
 /* vcc_obj.c */
@@ -318,6 +316,8 @@ struct symbol *VCC_FindSymbol(struct vcc *tl,
 const char * VCC_SymKind(struct vcc *tl, const struct symbol *s);
 typedef void symwalk_f(struct vcc *tl, const struct symbol *s);
 void VCC_WalkSymbols(struct vcc *tl, symwalk_f *func, enum symkind kind);
+void VCC_GenericSymbol(struct vcc *, struct symbol *,
+    enum var_type, const char *str, ...);
 
 /* vcc_token.c */
 void vcc_Coord(const struct vcc *tl, struct vsb *vsb,
diff --git a/lib/libvcc/vcc_expr.c b/lib/libvcc/vcc_expr.c
index e371fb9..6b74f8e 100644
--- a/lib/libvcc/vcc_expr.c
+++ b/lib/libvcc/vcc_expr.c
@@ -494,16 +494,15 @@ vcc_Eval_BoolConst(struct vcc *tl, struct expr **e, const struct symbol *sym)
  */
 
 void
-vcc_Eval_Acl(struct vcc *tl, struct expr **e, const struct symbol *sym)
+vcc_Eval_Generic(struct vcc *tl, struct expr **e, const struct symbol *sym)
 {
 
-	assert(sym->kind == SYM_ACL);
+	assert(sym->kind == sym->kind);
 	AN(sym->eval_priv);
 
 	vcc_ExpectCid(tl);
-	vcc_AddRef(tl, tl->t, SYM_ACL);
-	*e = vcc_mk_expr(ACL, "&vrt_acl_named_%s",
-	    (const char *)sym->eval_priv);
+	vcc_AddRef(tl, tl->t, sym->kind);
+	*e = vcc_mk_expr(sym->fmt, "%s", (const char *)sym->eval_priv);
 	(*e)->constant = EXPR_VAR;	/* XXX ? */
 	vcc_NextToken(tl);
 }
@@ -512,39 +511,6 @@ vcc_Eval_Acl(struct vcc *tl, struct expr **e, const struct symbol *sym)
  */
 
 void
-vcc_Eval_Backend(struct vcc *tl, struct expr **e, const struct symbol *sym)
-{
-
-	assert(sym->kind == SYM_BACKEND);
-	AN(sym->eval_priv);
-
-	vcc_ExpectCid(tl);
-	vcc_AddRef(tl, tl->t, SYM_BACKEND);
-	*e = vcc_mk_expr(BACKEND, "%s", (const char *)sym->eval_priv);
-	(*e)->constant = EXPR_VAR;	/* XXX ? */
-	vcc_NextToken(tl);
-}
-
-/*--------------------------------------------------------------------
- */
-
-void
-vcc_Eval_Probe(struct vcc *tl, struct expr **e, const struct symbol *sym)
-{
-
-	assert(sym->kind == SYM_PROBE);
-
-	vcc_ExpectCid(tl);
-	vcc_AddRef(tl, tl->t, SYM_PROBE);
-	*e = vcc_mk_expr(PROBE, "&vgc_probe_%.*s", PF(tl->t));
-	(*e)->constant = EXPR_VAR;      /* XXX ? */
-	vcc_NextToken(tl);
-}
-
-/*--------------------------------------------------------------------
- */
-
-void
 vcc_Eval_Var(struct vcc *tl, struct expr **e, const struct symbol *sym)
 {
 	const struct var *vp;
diff --git a/lib/libvcc/vcc_symb.c b/lib/libvcc/vcc_symb.c
index e9d39f0..22b6db9 100644
--- a/lib/libvcc/vcc_symb.c
+++ b/lib/libvcc/vcc_symb.c
@@ -28,6 +28,7 @@
 
 #include "config.h"
 
+#include <stdarg.h>
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
@@ -139,3 +140,22 @@ VCC_WalkSymbols(struct vcc *tl, symwalk_f *func, enum symkind kind)
 		ERRCHK(tl);
 	}
 }
+
+void
+VCC_GenericSymbol(struct vcc *tl, struct symbol *sym,
+    enum var_type fmt, const char *str, ...)
+{
+	struct vsb *vsb;
+	va_list ap;
+
+	vsb = VSB_new_auto();
+	AN(vsb);
+	va_start(ap, str);
+	VSB_vprintf(vsb, str, ap);
+	va_end(ap);
+	AZ(VSB_finish(vsb));
+	sym->eval_priv = TlDup(tl, VSB_data(vsb));
+	sym->eval = vcc_Eval_Generic;
+	sym->fmt = fmt;
+	VSB_destroy(&vsb);
+}



From phk at FreeBSD.org  Wed May 18 21:59:06 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Wed, 18 May 2016 23:59:06 +0200
Subject: [master] 96232c8 Initialize stevedores before the VCC
Message-ID: <E1b39Ug-0002KC-8j@project.varnish-software.com>


commit 96232c8869b36d98c43af116436cff48cca372e7
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed May 18 21:57:41 2016 +0000

    Initialize stevedores before the VCC

diff --git a/bin/varnishd/mgt/mgt_main.c b/bin/varnishd/mgt/mgt_main.c
index d696839..7e598b5 100644
--- a/bin/varnishd/mgt/mgt_main.c
+++ b/bin/varnishd/mgt/mgt_main.c
@@ -747,6 +747,13 @@ main(int argc, char * const *argv)
 		    P_arg, strerror(errno));
 	VJ_master(JAIL_MASTER_LOW);
 
+	/* If no -s argument specified, process default -s argument */
+	if (!s_arg_given)
+		STV_Config(s_arg);
+
+	/* Configure Transient storage, if user did not */
+	STV_Config_Transient();
+
 	mgt_vcl_init();
 
 	if (b_arg != NULL || f_arg != NULL) {
@@ -778,13 +785,6 @@ main(int argc, char * const *argv)
 		}
 	}
 
-	/* If no -s argument specified, process default -s argument */
-	if (!s_arg_given)
-		STV_Config(s_arg);
-
-	/* Configure Transient storage, if user did not */
-	STV_Config_Transient();
-
 	HSH_config(h_arg);
 
 	Wait_config(W_arg);



From phk at FreeBSD.org  Wed May 18 22:44:05 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 19 May 2016 00:44:05 +0200
Subject: [master] 7ab13b1 Add support for stringifying generic symbols.
Message-ID: <E1b3ACD-00055X-HH@project.varnish-software.com>


commit 7ab13b1bcd5d4163f28238c03c1f5d905c49e702
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed May 18 22:41:51 2016 +0000

    Add support for stringifying generic symbols.
    
    This means that if you:
    
    	acl foobar { .... }
    
    	sub vcl_recv {
    		set req.http.acl = foobar;
    	}
    
    Then the HTTP header gets set to the name of the ACL, ie: "foobar"

diff --git a/bin/varnishtest/tests/c00005.vtc b/bin/varnishtest/tests/c00005.vtc
index afa103c..99a7e5d 100644
--- a/bin/varnishtest/tests/c00005.vtc
+++ b/bin/varnishtest/tests/c00005.vtc
@@ -19,12 +19,16 @@ varnish v1 -arg "-p vsl_mask=+VCL_trace" -vcl+backend {
 			set req.url = "/";
 		}
 	}
+	sub vcl_deliver {
+		set resp.http.acl = acl1;
+	}
 } -start
 
 client c1 {
 	txreq -url "foo"
 	rxresp
 	expect resp.status == 200
+	expect resp.http.acl == acl1
 } -run
 
 varnish v1 -vcl+backend {
@@ -39,6 +43,9 @@ varnish v1 -vcl+backend {
 			set req.url = "/";
 		}
 	}
+	sub vcl_deliver {
+		set resp.http.acl = acl1;
+	}
 }
 
 client c1 -run
diff --git a/lib/libvcc/vcc_compile.h b/lib/libvcc/vcc_compile.h
index bfb661a..d20059b 100644
--- a/lib/libvcc/vcc_compile.h
+++ b/lib/libvcc/vcc_compile.h
@@ -106,7 +106,7 @@ enum symkind {
 };
 
 typedef void sym_expr_t(struct vcc *tl, struct expr **,
-    const struct symbol *sym);
+    const struct symbol *sym, enum var_type);
 typedef struct symbol *sym_wildcard_t(struct vcc *tl, const struct token *t,
     const struct symbol *sym);
 
diff --git a/lib/libvcc/vcc_expr.c b/lib/libvcc/vcc_expr.c
index 6b74f8e..e65acd1 100644
--- a/lib/libvcc/vcc_expr.c
+++ b/lib/libvcc/vcc_expr.c
@@ -446,14 +446,16 @@ vcc_expr_tostring(struct vcc *tl, struct expr **e, enum var_type fmt)
 /*--------------------------------------------------------------------
  */
 
-static void
-vcc_Eval_Regsub(struct vcc *tl, struct expr **e, const struct symbol *sym)
+static void __match_proto__(sym_expr_t)
+vcc_Eval_Regsub(struct vcc *tl, struct expr **e, const struct symbol *sym,
+    enum var_type fmt)
 {
 	struct expr *e2;
 	int all = sym->eval_priv == NULL ? 0 : 1;
 	const char *p;
 	char buf[128];
 
+	(void)fmt;
 	vcc_delete_expr(*e);
 	SkipToken(tl, ID);
 	SkipToken(tl, '(');
@@ -481,10 +483,12 @@ vcc_Eval_Regsub(struct vcc *tl, struct expr **e, const struct symbol *sym)
 /*--------------------------------------------------------------------
  */
 
-static void
-vcc_Eval_BoolConst(struct vcc *tl, struct expr **e, const struct symbol *sym)
+static void __match_proto__(sym_expr_t)
+vcc_Eval_BoolConst(struct vcc *tl, struct expr **e, const struct symbol *sym,
+    enum var_type fmt)
 {
 
+	(void)fmt;
 	vcc_NextToken(tl);
 	*e = vcc_mk_expr(BOOL, "(0==%d)", sym->eval_priv == NULL ? 1 : 0);
 	(*e)->constant = EXPR_CONST;
@@ -493,28 +497,36 @@ vcc_Eval_BoolConst(struct vcc *tl, struct expr **e, const struct symbol *sym)
 /*--------------------------------------------------------------------
  */
 
-void
-vcc_Eval_Generic(struct vcc *tl, struct expr **e, const struct symbol *sym)
+void __match_proto__(sym_expr_t)
+vcc_Eval_Generic(struct vcc *tl, struct expr **e, const struct symbol *sym,
+    enum var_type fmt)
 {
 
 	assert(sym->kind == sym->kind);
 	AN(sym->eval_priv);
 
-	vcc_ExpectCid(tl);
-	vcc_AddRef(tl, tl->t, sym->kind);
-	*e = vcc_mk_expr(sym->fmt, "%s", (const char *)sym->eval_priv);
-	(*e)->constant = EXPR_VAR;	/* XXX ? */
+	if (sym->fmt != STRING && (fmt == STRING || fmt == STRING_LIST)) {
+		*e = vcc_mk_expr(STRING, "\"%s\"", sym->name);
+		vcc_AddRef(tl, tl->t, sym->kind);
+	} else {
+		vcc_ExpectCid(tl);
+		vcc_AddRef(tl, tl->t, sym->kind);
+		*e = vcc_mk_expr(sym->fmt, "%s", (const char *)sym->eval_priv);
+		(*e)->constant = EXPR_VAR;	/* XXX ? */
+	}
 	vcc_NextToken(tl);
 }
 
 /*--------------------------------------------------------------------
  */
 
-void
-vcc_Eval_Var(struct vcc *tl, struct expr **e, const struct symbol *sym)
+void __match_proto__(sym_expr_t)
+vcc_Eval_Var(struct vcc *tl, struct expr **e, const struct symbol *sym,
+    enum var_type fmt)
 {
 	const struct var *vp;
 
+	(void)fmt;
 	assert(sym->kind == SYM_VAR);
 	vcc_AddUses(tl, tl->t, sym->r_methods, "Not available");
 	vp = vcc_FindVar(tl, tl->t, 0, "cannot be read");
@@ -758,10 +770,12 @@ vcc_Eval_Func(struct vcc *tl, const char *cfunc,
 /*--------------------------------------------------------------------
  */
 
-void
-vcc_Eval_SymFunc(struct vcc *tl, struct expr **e, const struct symbol *sym)
+void __match_proto__(sym_expr_t)
+vcc_Eval_SymFunc(struct vcc *tl, struct expr **e, const struct symbol *sym,
+    enum var_type fmt)
 {
 
+	(void)fmt;
 	assert(sym->kind == SYM_FUNC);
 	/* XXX */
 	AN(sym->cfunc);
@@ -842,7 +856,7 @@ vcc_expr4(struct vcc *tl, struct expr **e, enum var_type fmt)
 		case SYM_PROBE:
 			AN(sym->eval);
 			AZ(*e);
-			sym->eval(tl, e, sym);
+			sym->eval(tl, e, sym, fmt);
 			ERRCHK(tl);
 			/* Unless asked for a HEADER, fold to string here */
 			if (*e && fmt != HEADER && (*e)->fmt == HEADER) {
@@ -1399,7 +1413,7 @@ vcc_Expr_Call(struct vcc *tl, const struct symbol *sym)
 
 	t1 = tl->t;
 	e = NULL;
-	vcc_Eval_SymFunc(tl, &e, sym);
+	vcc_Eval_SymFunc(tl, &e, sym, VOID);
 	if (!tl->err) {
 		vcc_expr_fmt(tl->fb, tl->indent, e);
 		VSB_cat(tl->fb, ";\n");



From phk at FreeBSD.org  Thu May 19 08:00:09 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 19 May 2016 10:00:09 +0200
Subject: [master] a31bed0 Tell VCC about our stevedores,
 and define symbols for them.
Message-ID: <E1b3IsL-0003Iy-F8@project.varnish-software.com>


commit a31bed0236f96ef8d38df1f43dcd0fee2628c808
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed May 18 22:50:59 2016 +0000

    Tell VCC about our stevedores, and define symbols for them.

diff --git a/bin/varnishd/mgt/mgt_vcc.c b/bin/varnishd/mgt/mgt_vcc.c
index 8847c03..d35ccd4 100644
--- a/bin/varnishd/mgt/mgt_vcc.c
+++ b/bin/varnishd/mgt/mgt_vcc.c
@@ -41,6 +41,7 @@
 #include <sys/stat.h>
 
 #include "mgt/mgt.h"
+#include "storage/storage.h"
 
 #include "libvcc.h"
 #include "vcli.h"
@@ -87,6 +88,7 @@ run_vcc(void *priv)
 	struct vcc_priv *vp;
 	int fd, i, l;
 	struct vcp *vcp;
+	struct stevedore *stv;
 
 	VJ_subproc(JAIL_SUBPROC_VCC);
 	CAST_OBJ_NOTNULL(vp, priv, VCC_PRIV_MAGIC);
@@ -103,6 +105,9 @@ run_vcc(void *priv)
 	VCP_Err_Unref(vcp, mgt_vcc_err_unref);
 	VCP_Allow_InlineC(vcp, mgt_vcc_allow_inline_c);
 	VCP_Unsafe_Path(vcp, mgt_vcc_unsafe_path);
+	VTAILQ_FOREACH(stv, &stv_stevedores, list)
+		VCP_Stevedore(vcp, stv->ident);
+	VCP_Stevedore(vcp, stv_transient->ident);
 	csrc = VCC_Compile(vcp, sb, vp->vclsrc, vp->vclsrcfile);
 	AZ(VSB_finish(sb));
 	if (VSB_len(sb))
diff --git a/bin/varnishd/storage/stevedore.c b/bin/varnishd/storage/stevedore.c
index 2ce76dd..da7ece4 100644
--- a/bin/varnishd/storage/stevedore.c
+++ b/bin/varnishd/storage/stevedore.c
@@ -231,6 +231,17 @@ VRT_Stv(const char *nm)
 	return (0);
 }
 
+struct stevedore *
+VRT_stevedore(const char *nm)
+{
+	struct stevedore *stv;
+
+	VTAILQ_FOREACH(stv, &stv_stevedores, list)
+		if (!strcmp(stv->ident, nm))
+			return (stv);
+	WRONG("Unknown stevedore name");
+}
+
 #define VRTSTVVAR(nm, vtype, ctype, dval)	\
 ctype						\
 VRT_Stv_##nm(const char *nm)			\
diff --git a/include/libvcc.h b/include/libvcc.h
index 0d3bb53..1ae4b8a 100644
--- a/include/libvcc.h
+++ b/include/libvcc.h
@@ -28,15 +28,16 @@
  *
  */
 
-struct vcc;
+struct vcp;
 
 struct vcp *VCP_New(void);
-void VCP_Builtin_VCL(struct vcp *, const char *str);
-void VCP_VCL_path(struct vcp *, const char *str);
-void VCP_VMOD_path(struct vcp *, const char *str);
-void VCP_Err_Unref(struct vcp *tl, unsigned u);
-void VCP_Allow_InlineC(struct vcp *tl, unsigned u);
-void VCP_Unsafe_Path(struct vcp *tl, unsigned u);
+void VCP_Allow_InlineC(struct vcp *, unsigned);
+void VCP_Builtin_VCL(struct vcp *, const char *);
+void VCP_Err_Unref(struct vcp *, unsigned);
+void VCP_Stevedore(struct vcp *, const char *);
+void VCP_Unsafe_Path(struct vcp *, unsigned);
+void VCP_VCL_path(struct vcp *, const char *);
+void VCP_VMOD_path(struct vcp *, const char *);
 
-char *VCC_Compile(const struct vcp *, struct vsb *sb,
+char *VCC_Compile(const struct vcp *, struct vsb *,
     const char *vclsrc, const char *vclsrcfile);
diff --git a/include/tbl/symbol_kind.h b/include/tbl/symbol_kind.h
index 497f400..e9add4a 100644
--- a/include/tbl/symbol_kind.h
+++ b/include/tbl/symbol_kind.h
@@ -29,14 +29,15 @@
 
 /*lint -save -e525 -e539 */
 VCC_SYMB(NONE,		none)
-VCC_SYMB(VAR,		var)
-VCC_SYMB(FUNC,		func)		/* VMOD function/procedure */
-VCC_SYMB(VMOD,		vmod)
 VCC_SYMB(ACL,		acl)
-VCC_SYMB(SUB,		sub)		/* VCL subroutine */
 VCC_SYMB(BACKEND,	backend)
+VCC_SYMB(FUNC,		func)		/* VMOD function/procedure */
+VCC_SYMB(METHOD,	method)
+VCC_SYMB(OBJECT,	object)
 VCC_SYMB(PROBE,		probe)
+VCC_SYMB(STEVEDORE,	stevedore)
+VCC_SYMB(SUB,		sub)		/* VCL subroutine */
+VCC_SYMB(VAR,		var)
+VCC_SYMB(VMOD,		vmod)
 VCC_SYMB(WILDCARD,	wildcard)
-VCC_SYMB(OBJECT,	object)
-VCC_SYMB(METHOD,	method)
 /*lint -restore */
diff --git a/include/vrt.h b/include/vrt.h
index 9963460..bcaa2cd 100644
--- a/include/vrt.h
+++ b/include/vrt.h
@@ -65,6 +65,7 @@ struct busyobj;
 struct director;
 struct http;
 struct req;
+struct stevedore;
 struct suckaddr;
 struct vcl;
 struct vmod;
@@ -91,6 +92,7 @@ typedef long					VCL_INT;
 typedef const struct suckaddr *			VCL_IP;
 typedef const struct vrt_backend_probe *	VCL_PROBE;
 typedef double					VCL_REAL;
+typedef struct stevedore *			VCL_STEVEDORE;
 typedef const char *				VCL_STRING;
 typedef double					VCL_TIME;
 typedef void					VCL_VOID;
@@ -330,6 +332,7 @@ struct vmod_priv *VRT_priv_top(VRT_CTX, void *vmod_id);
 
 /* Stevedore related functions */
 int VRT_Stv(const char *nm);
+struct stevedore *VRT_stevedore(const char *nm);
 
 /* Convert things to string */
 
diff --git a/lib/libvcc/vcc_compile.c b/lib/libvcc/vcc_compile.c
index 0ee166d..a9657e6 100644
--- a/lib/libvcc/vcc_compile.c
+++ b/lib/libvcc/vcc_compile.c
@@ -680,7 +680,7 @@ vcc_CompileSource(const struct vcp * const vcp, struct vsb *sb,
     struct source *sp)
 {
 	struct vcc *tl;
-	struct symbol *sym;
+	struct symbol *sym, *sym2;
 	const struct var *v;
 	struct vsb *vsb;
 
@@ -692,6 +692,16 @@ vcc_CompileSource(const struct vcp * const vcp, struct vsb *sb,
 
 	vcc_Expr_Init(tl);
 
+	VTAILQ_FOREACH(sym, &vcp->symbols, list) {
+		assert(sym->eval == vcc_Eval_Generic);
+		sym2 = VCC_AddSymbolStr(tl, sym->name, sym->kind);
+		sym2->fmt = sym->fmt;
+		sym2->eval = sym->eval;
+		sym2->eval_priv = sym->eval_priv;
+		sym2->ndef = 1;
+		sym2->nref = 1;
+	}
+
 	for (v = tl->vars; v->name != NULL; v++) {
 		if (v->fmt == HEADER) {
 			sym = VCC_AddSymbolStr(tl, v->name, SYM_WILDCARD);
@@ -860,6 +870,7 @@ VCP_New(void)
 
 	ALLOC_OBJ(vcp, VCP_MAGIC);
 	AN(vcp);
+	VTAILQ_INIT(&vcp->symbols);
 
 	return (vcp);
 }
@@ -927,3 +938,19 @@ VCP_Unsafe_Path(struct vcp *vcp, unsigned u)
 	CHECK_OBJ_NOTNULL(vcp, VCP_MAGIC);
 	vcp->unsafe_path = u;
 }
+
+void
+VCP_Stevedore(struct vcp *vcp, const char *stv_name)
+{
+	struct symbol *sym;
+
+	CHECK_OBJ_NOTNULL(vcp, VCP_MAGIC);
+	ALLOC_OBJ(sym, SYMBOL_MAGIC);
+	AN(sym);
+	REPLACE(sym->name, stv_name);		/* XXX storage.* ? */
+	sym->kind = SYM_STEVEDORE;
+	VCC_GenericSymbol(NULL, sym, STEVEDORE,
+	    "VRT_stevedore(\"%s\")", stv_name);
+	VTAILQ_INSERT_TAIL(&vcp->symbols, sym, list);
+}
+
diff --git a/lib/libvcc/vcc_compile.h b/lib/libvcc/vcc_compile.h
index d20059b..6d6b7da 100644
--- a/lib/libvcc/vcc_compile.h
+++ b/lib/libvcc/vcc_compile.h
@@ -165,6 +165,8 @@ struct vcp {
 	unsigned		err_unref;
 	unsigned		allow_inline_c;
 	unsigned		unsafe_path;
+
+	VTAILQ_HEAD(,symbol)	symbols;
 };
 
 struct vcc {
diff --git a/lib/libvcc/vcc_expr.c b/lib/libvcc/vcc_expr.c
index e65acd1..d043d21 100644
--- a/lib/libvcc/vcc_expr.c
+++ b/lib/libvcc/vcc_expr.c
@@ -824,6 +824,7 @@ vcc_expr4(struct vcc *tl, struct expr **e, enum var_type fmt)
 		case ACL:	kind = SYM_ACL; break;
 		case BACKEND:	kind = SYM_BACKEND; break;
 		case PROBE:	kind = SYM_PROBE; break;
+		case STEVEDORE:	kind = SYM_STEVEDORE; break;
 		default:	kind = SYM_NONE; break;
 		}
 		if (kind != SYM_NONE)
@@ -853,6 +854,7 @@ vcc_expr4(struct vcc *tl, struct expr **e, enum var_type fmt)
 		case SYM_FUNC:
 		case SYM_ACL:
 		case SYM_BACKEND:
+		case SYM_STEVEDORE:
 		case SYM_PROBE:
 			AN(sym->eval);
 			AZ(*e);
diff --git a/lib/libvcc/vcc_symb.c b/lib/libvcc/vcc_symb.c
index 22b6db9..02e9fe0 100644
--- a/lib/libvcc/vcc_symb.c
+++ b/lib/libvcc/vcc_symb.c
@@ -154,7 +154,11 @@ VCC_GenericSymbol(struct vcc *tl, struct symbol *sym,
 	VSB_vprintf(vsb, str, ap);
 	va_end(ap);
 	AZ(VSB_finish(vsb));
-	sym->eval_priv = TlDup(tl, VSB_data(vsb));
+	if (tl != NULL)
+		sym->eval_priv = TlDup(tl, VSB_data(vsb));
+	else
+		sym->eval_priv = strdup(VSB_data(vsb));
+	AN(sym->eval_priv);
 	sym->eval = vcc_Eval_Generic;
 	sym->fmt = fmt;
 	VSB_destroy(&vsb);



From phk at FreeBSD.org  Thu May 19 08:00:09 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 19 May 2016 10:00:09 +0200
Subject: [master] 5a4d3be Collapse the variables fully into symbols
Message-ID: <E1b3IsL-0003J4-Gu@project.varnish-software.com>


commit 5a4d3be3a6fcc52b629aba77597db6517e898662
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu May 19 00:12:29 2016 +0000

    Collapse the variables fully into symbols

diff --git a/lib/libvcc/vcc_action.c b/lib/libvcc/vcc_action.c
index 8e62d2d..425b993 100644
--- a/lib/libvcc/vcc_action.c
+++ b/lib/libvcc/vcc_action.c
@@ -82,26 +82,26 @@ static const struct arith {
 static void
 parse_set(struct vcc *tl)
 {
-	const struct var *vp;
+	const struct symbol *sym;
 	const struct arith *ap;
 	enum var_type fmt;
 
 	vcc_NextToken(tl);
 	ExpectErr(tl, ID);
-	vp = vcc_FindVar(tl, tl->t, 1, "cannot be set");
+	sym = vcc_FindVar(tl, tl->t, 1, "cannot be set");
 	ERRCHK(tl);
-	assert(vp != NULL);
-	Fb(tl, 1, "%s\n", vp->lname);
+	assert(sym != NULL);
+	Fb(tl, 1, "%s\n", sym->lname);
 	tl->indent += INDENT;
 	vcc_NextToken(tl);
-	fmt = vp->fmt;
+	fmt = sym->fmt;
 	for (ap = arith; ap->type != VOID; ap++) {
 		if (ap->type != fmt)
 			continue;
 		if (ap->oper != tl->t->tok)
 			continue;
 		if (ap->oper != '=')
-			Fb(tl, 1, "%s %c ", vp->rname, *tl->t->b);
+			Fb(tl, 1, "%s %c ", sym->rname, *tl->t->b);
 		vcc_NextToken(tl);
 		fmt = ap->want;
 		break;
@@ -124,22 +124,22 @@ parse_set(struct vcc *tl)
 static void
 parse_unset(struct vcc *tl)
 {
-	const struct var *vp;
+	const struct symbol *sym;
 
 	/* XXX: Wrong, should use VCC_Expr(HEADER) */
 	vcc_NextToken(tl);
 	ExpectErr(tl, ID);
-	vp = vcc_FindVar(tl, tl->t, 1, "cannot be unset");
+	sym = vcc_FindVar(tl, tl->t, 1, "cannot be unset");
 	ERRCHK(tl);
-	assert(vp != NULL);
-	if (vp->fmt != HEADER) {
+	assert(sym != NULL);
+	if (sym->fmt != HEADER) {
 		VSB_printf(tl->sb,
 		    "Only HTTP header variables can be unset.\n");
 		vcc_ErrWhere(tl, tl->t);
 		return;
 	}
 	ERRCHK(tl);
-	Fb(tl, 1, "%svrt_magic_string_unset);\n", vp->lname);
+	Fb(tl, 1, "%svrt_magic_string_unset);\n", sym->lname);
 	vcc_NextToken(tl);
 }
 
diff --git a/lib/libvcc/vcc_compile.c b/lib/libvcc/vcc_compile.c
index a9657e6..fc5ccc1 100644
--- a/lib/libvcc/vcc_compile.c
+++ b/lib/libvcc/vcc_compile.c
@@ -609,7 +609,6 @@ vcc_NewVcc(const struct vcp *vcp)
 	ALLOC_OBJ(tl, VCC_MAGIC);
 	AN(tl);
 	tl->param = vcp;
-	tl->vars = vcc_vars;
 	VTAILQ_INIT(&tl->symbols);
 	VTAILQ_INIT(&tl->inifin);
 	VTAILQ_INIT(&tl->membits);
@@ -702,17 +701,20 @@ vcc_CompileSource(const struct vcp * const vcp, struct vsb *sb,
 		sym2->nref = 1;
 	}
 
-	for (v = tl->vars; v->name != NULL; v++) {
+	for (v = vcc_vars; v->name != NULL; v++) {
 		if (v->fmt == HEADER) {
 			sym = VCC_AddSymbolStr(tl, v->name, SYM_WILDCARD);
 			sym->wildcard = vcc_Var_Wildcard;
+			sym->wildcard_priv = v;
 		} else {
 			sym = VCC_AddSymbolStr(tl, v->name, SYM_VAR);
 		}
-		sym->var = v;
 		sym->fmt = v->fmt;
 		sym->eval = vcc_Eval_Var;
 		sym->r_methods = v->r_methods;
+		sym->rname = v->rname;
+		sym->w_methods = v->w_methods;
+		sym->lname = v->lname;
 	}
 
 	sym = VCC_AddSymbolStr(tl, "storage.", SYM_WILDCARD);
diff --git a/lib/libvcc/vcc_compile.h b/lib/libvcc/vcc_compile.h
index 6d6b7da..d02029d 100644
--- a/lib/libvcc/vcc_compile.h
+++ b/lib/libvcc/vcc_compile.h
@@ -118,6 +118,7 @@ struct symbol {
 	char				*name;
 	unsigned			nlen;
 	sym_wildcard_t			*wildcard;
+	const void			*wildcard_priv;
 	enum symkind			kind;
 
 	const struct token		*def_b, *def_e;
@@ -137,8 +138,10 @@ struct symbol {
 	const char			*args;
 
 	/* SYM_VAR */
-	const struct var		*var;
+	const char			*rname;
 	unsigned			r_methods;
+	const char			*lname;
+	unsigned			w_methods;
 };
 
 VTAILQ_HEAD(tokenhead, token);
@@ -176,7 +179,6 @@ struct vcc {
 	/* Parameter/Template section */
 	const struct vcp	*param;
 
-	const struct var	*vars;
 	VTAILQ_HEAD(, symbol)	symbols;
 
 	struct inifinhead	inifin;
@@ -340,7 +342,7 @@ void vcc_AddToken(struct vcc *tl, unsigned tok, const char *b,
 
 /* vcc_var.c */
 sym_wildcard_t vcc_Var_Wildcard;
-const struct var *vcc_FindVar(struct vcc *tl, const struct token *t,
+const struct symbol *vcc_FindVar(struct vcc *tl, const struct token *t,
     int wr_access, const char *use);
 
 /* vcc_vmod.c */
diff --git a/lib/libvcc/vcc_expr.c b/lib/libvcc/vcc_expr.c
index d043d21..506f196 100644
--- a/lib/libvcc/vcc_expr.c
+++ b/lib/libvcc/vcc_expr.c
@@ -524,15 +524,12 @@ void __match_proto__(sym_expr_t)
 vcc_Eval_Var(struct vcc *tl, struct expr **e, const struct symbol *sym,
     enum var_type fmt)
 {
-	const struct var *vp;
 
 	(void)fmt;
 	assert(sym->kind == SYM_VAR);
 	vcc_AddUses(tl, tl->t, sym->r_methods, "Not available");
-	vp = vcc_FindVar(tl, tl->t, 0, "cannot be read");
 	ERRCHK(tl);
-	assert(vp != NULL);
-	*e = vcc_mk_expr(vp->fmt, "%s", vp->rname);
+	*e = vcc_mk_expr(sym->fmt, "%s", sym->rname);
 	vcc_NextToken(tl);
 }
 
diff --git a/lib/libvcc/vcc_storage.c b/lib/libvcc/vcc_storage.c
index 2c3f81c..efc4f11 100644
--- a/lib/libvcc/vcc_storage.c
+++ b/lib/libvcc/vcc_storage.c
@@ -139,10 +139,12 @@ vcc_Stv_Wildcard(struct vcc *tl, const struct token *t,
 
 	sym = VCC_AddSymbolTok(tl, t, SYM_VAR);
 	AN(sym);
-	sym->var = v;
 	sym->fmt = v->fmt;
 	sym->eval = vcc_Eval_Var;
 	sym->r_methods = v->r_methods;
+	sym->rname = v->rname;
+	sym->w_methods = v->w_methods;
+	sym->lname = v->lname;
 
 	return (sym);
 }
diff --git a/lib/libvcc/vcc_var.c b/lib/libvcc/vcc_var.c
index 4b03269..6bd5dec 100644
--- a/lib/libvcc/vcc_var.c
+++ b/lib/libvcc/vcc_var.c
@@ -47,7 +47,7 @@ vcc_Var_Wildcard(struct vcc *tl, const struct token *t, const struct symbol *wc)
 	const char *p, *leaf;
 	struct vsb *vsb;
 
-	vh = wc->var;
+	vh = wc->wildcard_priv;
 	assert(vh->fmt == HEADER);
 
 	v = TlAlloc(tl, sizeof *v);
@@ -83,29 +83,26 @@ vcc_Var_Wildcard(struct vcc *tl, const struct token *t, const struct symbol *wc)
 
 	sym = VCC_AddSymbolTok(tl, t, SYM_VAR);
 	AN(sym);
-	sym->var = v;
 	sym->fmt = v->fmt;
 	sym->eval = vcc_Eval_Var;
 	sym->r_methods = v->r_methods;
+	sym->rname = v->rname;
+	sym->w_methods = v->w_methods;
+	sym->lname = v->lname;
 	return (sym);
 }
 
 /*--------------------------------------------------------------------*/
 
-const struct var *
+const struct symbol *
 vcc_FindVar(struct vcc *tl, const struct token *t, int wr_access,
     const char *use)
 {
-	const struct var *v;
 	const struct symbol *sym;
 
-	AN(tl->vars);
 	sym = VCC_FindSymbol(tl, t, SYM_VAR);
 	if (sym != NULL) {
-		v = sym->var;
-		AN(v);
-
-		if (wr_access && v->w_methods == 0) {
+		if (wr_access && sym->w_methods == 0) {
 			VSB_printf(tl->sb, "Variable ");
 			vcc_ErrToken(tl, t);
 			VSB_printf(tl->sb, " is read only.");
@@ -113,8 +110,8 @@ vcc_FindVar(struct vcc *tl, const struct token *t, int wr_access,
 			vcc_ErrWhere(tl, t);
 			return (NULL);
 		} else if (wr_access) {
-			vcc_AddUses(tl, t, v->w_methods, use);
-		} else if (v->r_methods == 0) {
+			vcc_AddUses(tl, t, sym->w_methods, use);
+		} else if (sym->r_methods == 0) {
 			VSB_printf(tl->sb, "Variable ");
 			vcc_ErrToken(tl, t);
 			VSB_printf(tl->sb, " is write only.");
@@ -122,9 +119,9 @@ vcc_FindVar(struct vcc *tl, const struct token *t, int wr_access,
 			vcc_ErrWhere(tl, t);
 			return (NULL);
 		} else {
-			vcc_AddUses(tl, t, v->r_methods, use);
+			vcc_AddUses(tl, t, sym->r_methods, use);
 		}
-		return (v);
+		return (sym);
 	}
 	VSB_printf(tl->sb, "Unknown variable ");
 	vcc_ErrToken(tl, t);



From phk at FreeBSD.org  Thu May 19 08:00:09 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 19 May 2016 10:00:09 +0200
Subject: [master] ec88f33 Test a particular error handling
Message-ID: <E1b3IsL-0003JD-IZ@project.varnish-software.com>


commit ec88f3380d9857a3639b7b6a57fd42f03534c87e
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu May 19 00:12:52 2016 +0000

    Test a particular error handling

diff --git a/bin/varnishtest/tests/v00020.vtc b/bin/varnishtest/tests/v00020.vtc
index 540a560..a05b7fb 100644
--- a/bin/varnishtest/tests/v00020.vtc
+++ b/bin/varnishtest/tests/v00020.vtc
@@ -283,6 +283,14 @@ varnish v1 -errvcl {Function returns VOID} {
 	}
 }
 
+varnish v1 -errvcl {'beresp.status': Not available in method 'vcl_recv'.} {
+	backend b { .host = "127.0.0.1"; }
+
+	sub vcl_recv {
+		set req.http.foo = 100 + beresp.status;
+	}
+}
+
 varnish v1 -errvcl {Names of VCL acl's cannot contain '-'} {
 	backend b { .host = "127.0.0.1"; }
 	acl foo-bar {



From phk at FreeBSD.org  Thu May 19 08:00:09 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 19 May 2016 10:00:09 +0200
Subject: [master] ba63e4e Introduce "Handle" symbols and "Global" symbols as a
 special case.
Message-ID: <E1b3IsL-0003JM-Kn@project.varnish-software.com>


commit ba63e4eb2051355c6639ad50c19240bf6da449da
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu May 19 07:51:11 2016 +0000

    Introduce "Handle" symbols and "Global" symbols as a special case.
    
    Handle symbols are ACL, BACKEND, PROBE and STEVEDORE, they are not
    variables, but "things which do stuff".
    
    The only real difference from variables (like resp.status) is that
    when they are stringified (resp.http.foo = BLA) they return the
    name of the thing (ie: backend, acl ... name) rather than a value,
    which they don't have.
    
    Handles are by definition available everywhere.
    
    Global symbols are handles available in any VCL (ie: STEVEDOREs).

diff --git a/bin/varnishtest/tests/r00916.vtc b/bin/varnishtest/tests/r00916.vtc
index 8057f1a..ac268ca 100644
--- a/bin/varnishtest/tests/r00916.vtc
+++ b/bin/varnishtest/tests/r00916.vtc
@@ -5,7 +5,7 @@ server s1 {
 	txresp -body "FOO"
 } -start
 
-varnish v1 -errvcl {Backend not found: 's-1'} {
+varnish v1 -errvcl {Symbol not found: 's-1' (expected type BACKEND):} {
 	backend b { .host = "127.0.0.1"; }
 	sub s1 {
 	}
diff --git a/bin/varnishtest/tests/v00016.vtc b/bin/varnishtest/tests/v00016.vtc
index 990f39e..009f17d 100644
--- a/bin/varnishtest/tests/v00016.vtc
+++ b/bin/varnishtest/tests/v00016.vtc
@@ -119,7 +119,7 @@ varnish v1 -errvcl {'bereq.between_bytes_timeout': cannot be set} {
 	}
 }
 
-varnish v1 -errvcl {Backend not found: 'c'} {
+varnish v1 -errvcl {Symbol not found: 'c' (expected type BACKEND):} {
 	backend b { .host = "127.0.0.1"; }
 	sub vcl_backend_response {
 		if (beresp.backend == c) {
diff --git a/lib/libvcc/vcc_acl.c b/lib/libvcc/vcc_acl.c
index f3e3488..c6f57fb 100644
--- a/lib/libvcc/vcc_acl.c
+++ b/lib/libvcc/vcc_acl.c
@@ -474,7 +474,6 @@ void
 vcc_ParseAcl(struct vcc *tl)
 {
 	struct token *an;
-	struct symbol *sym;
 	char *acln;
 
 	vcc_NextToken(tl);
@@ -492,15 +491,7 @@ vcc_ParseAcl(struct vcc *tl)
 
 	acln = TlDupTok(tl, an);
 
-	sym = VCC_GetSymbolTok(tl, an, SYM_ACL);
-	AN(sym);
-	if (sym->ndef > 0) {
-		VSB_printf(tl->sb, "ACL %.*s redefined\n", PF(an));
-		vcc_ErrWhere(tl, an);
-		return;
-	}
-	VCC_GenericSymbol(tl, sym, ACL, "&vrt_acl_named_%s", acln);
-	sym->ndef++;
+	(void)VCC_HandleSymbol(tl, an, ACL, "&vrt_acl_named_%s", acln);
 	ERRCHK(tl);
 
 	SkipToken(tl, '{');
diff --git a/lib/libvcc/vcc_backend.c b/lib/libvcc/vcc_backend.c
index c860dc2..6e9c498 100644
--- a/lib/libvcc/vcc_backend.c
+++ b/lib/libvcc/vcc_backend.c
@@ -249,7 +249,6 @@ void
 vcc_ParseProbe(struct vcc *tl)
 {
 	struct token *t_probe;
-	struct symbol *sym;
 	char *p;
 
 	vcc_NextToken(tl);		/* ID: probe */
@@ -259,15 +258,7 @@ vcc_ParseProbe(struct vcc *tl)
 	t_probe = tl->t;
 	vcc_NextToken(tl);
 
-	sym = VCC_GetSymbolTok(tl, t_probe, SYM_PROBE);
-	AN(sym);
-	if (sym->ndef > 0) {
-		VSB_printf(tl->sb, "Probe %.*s redefined\n", PF(t_probe));
-		vcc_ErrWhere(tl, t_probe);
-		return;
-	}
-	VCC_GenericSymbol(tl, sym, PROBE, "%s", sym->name);
-	sym->ndef++;
+	(void)VCC_HandleSymbol(tl, t_probe, PROBE, "%.s", PF(t_probe));
 	ERRCHK(tl);
 
 	vcc_ParseProbeSpec(tl, t_probe, &p);
@@ -489,15 +480,7 @@ vcc_ParseBackend(struct vcc *tl)
 	sprintf(vgcname, "vgc_backend_%.*s", PF(t_be));
 	Fh(tl, 0, "\nstatic struct director *%s;\n", vgcname);
 
-	sym = VCC_GetSymbolTok(tl, t_be, SYM_BACKEND);
-	AN(sym);
-	if (sym->ndef > 0) {
-		VSB_printf(tl->sb, "Backend %.*s redefined\n", PF(t_be));
-		vcc_ErrWhere(tl, t_be);
-		return;
-	}
-	VCC_GenericSymbol(tl, sym, BACKEND, "%s", vgcname);
-	sym->ndef++;
+	sym = VCC_HandleSymbol(tl, t_be, BACKEND, "%s", vgcname);
 	ERRCHK(tl);
 
 	vcc_ParseHostDef(tl, t_be, vgcname);
@@ -511,7 +494,7 @@ vcc_ParseBackend(struct vcc *tl)
 	}
 
 	if (tl->default_director == NULL || vcc_IdIs(t_be, "default")) {
-		tl->default_director = sym->eval_priv;
+		tl->default_director = sym->rname;
 		tl->t_default_director = t_be;
 	}
 }
diff --git a/lib/libvcc/vcc_compile.c b/lib/libvcc/vcc_compile.c
index fc5ccc1..bc22cad 100644
--- a/lib/libvcc/vcc_compile.c
+++ b/lib/libvcc/vcc_compile.c
@@ -692,11 +692,12 @@ vcc_CompileSource(const struct vcp * const vcp, struct vsb *sb,
 	vcc_Expr_Init(tl);
 
 	VTAILQ_FOREACH(sym, &vcp->symbols, list) {
-		assert(sym->eval == vcc_Eval_Generic);
 		sym2 = VCC_AddSymbolStr(tl, sym->name, sym->kind);
 		sym2->fmt = sym->fmt;
 		sym2->eval = sym->eval;
 		sym2->eval_priv = sym->eval_priv;
+		sym2->rname = sym->rname;
+		sym2->r_methods = sym->r_methods;
 		sym2->ndef = 1;
 		sym2->nref = 1;
 	}
@@ -951,8 +952,7 @@ VCP_Stevedore(struct vcp *vcp, const char *stv_name)
 	AN(sym);
 	REPLACE(sym->name, stv_name);		/* XXX storage.* ? */
 	sym->kind = SYM_STEVEDORE;
-	VCC_GenericSymbol(NULL, sym, STEVEDORE,
-	    "VRT_stevedore(\"%s\")", stv_name);
+	VCC_GlobalSymbol(sym, STEVEDORE, "VRT_stevedore(\"%s\")", stv_name);
 	VTAILQ_INSERT_TAIL(&vcp->symbols, sym, list);
 }
 
diff --git a/lib/libvcc/vcc_compile.h b/lib/libvcc/vcc_compile.h
index d02029d..647c760 100644
--- a/lib/libvcc/vcc_compile.h
+++ b/lib/libvcc/vcc_compile.h
@@ -287,11 +287,14 @@ void vcc_Expr(struct vcc *tl, enum var_type typ);
 void vcc_Expr_Call(struct vcc *tl, const struct symbol *sym);
 void vcc_Expr_Init(struct vcc *tl);
 sym_expr_t vcc_Eval_Var;
-sym_expr_t vcc_Eval_Generic;
+sym_expr_t vcc_Eval_Handle;
 sym_expr_t vcc_Eval_SymFunc;
 void vcc_Eval_Func(struct vcc *tl, const char *cfunc, const char *extra,
     const char *name, const char *args);
 enum var_type VCC_arg_type(const char **p);
+enum symkind VCC_HandleKind(enum var_type fmt);
+struct symbol *VCC_HandleSymbol(struct vcc *, const struct token *,
+    enum var_type fmt, const char *str, ...);
 
 /* vcc_obj.c */
 extern const struct var vcc_vars[];
@@ -320,8 +323,7 @@ struct symbol *VCC_FindSymbol(struct vcc *tl,
 const char * VCC_SymKind(struct vcc *tl, const struct symbol *s);
 typedef void symwalk_f(struct vcc *tl, const struct symbol *s);
 void VCC_WalkSymbols(struct vcc *tl, symwalk_f *func, enum symkind kind);
-void VCC_GenericSymbol(struct vcc *, struct symbol *,
-    enum var_type, const char *str, ...);
+void VCC_GlobalSymbol(struct symbol *, enum var_type, const char *str, ...);
 
 /* vcc_token.c */
 void vcc_Coord(const struct vcc *tl, struct vsb *vsb,
diff --git a/lib/libvcc/vcc_expr.c b/lib/libvcc/vcc_expr.c
index 506f196..1d0108c 100644
--- a/lib/libvcc/vcc_expr.c
+++ b/lib/libvcc/vcc_expr.c
@@ -498,12 +498,12 @@ vcc_Eval_BoolConst(struct vcc *tl, struct expr **e, const struct symbol *sym,
  */
 
 void __match_proto__(sym_expr_t)
-vcc_Eval_Generic(struct vcc *tl, struct expr **e, const struct symbol *sym,
+vcc_Eval_Handle(struct vcc *tl, struct expr **e, const struct symbol *sym,
     enum var_type fmt)
 {
 
 	assert(sym->kind == sym->kind);
-	AN(sym->eval_priv);
+	AN(sym->rname);
 
 	if (sym->fmt != STRING && (fmt == STRING || fmt == STRING_LIST)) {
 		*e = vcc_mk_expr(STRING, "\"%s\"", sym->name);
@@ -511,7 +511,7 @@ vcc_Eval_Generic(struct vcc *tl, struct expr **e, const struct symbol *sym,
 	} else {
 		vcc_ExpectCid(tl);
 		vcc_AddRef(tl, tl->t, sym->kind);
-		*e = vcc_mk_expr(sym->fmt, "%s", (const char *)sym->eval_priv);
+		*e = vcc_mk_expr(sym->fmt, "%s", sym->rname);
 		(*e)->constant = EXPR_VAR;	/* XXX ? */
 	}
 	vcc_NextToken(tl);
@@ -817,13 +817,7 @@ vcc_expr4(struct vcc *tl, struct expr **e, enum var_type fmt)
 		 * XXX: look for SYM_VAR first for consistency ?
 		 */
 		sym = NULL;
-		switch (fmt) {
-		case ACL:	kind = SYM_ACL; break;
-		case BACKEND:	kind = SYM_BACKEND; break;
-		case PROBE:	kind = SYM_PROBE; break;
-		case STEVEDORE:	kind = SYM_STEVEDORE; break;
-		default:	kind = SYM_NONE; break;
-		}
+		kind = VCC_HandleKind(fmt);
 		if (kind != SYM_NONE)
 			sym = VCC_FindSymbol(tl, tl->t, kind);
 		if (sym == NULL)
@@ -1157,7 +1151,7 @@ vcc_expr_cmp(struct vcc *tl, struct expr **e, enum var_type fmt)
 	const char *re;
 	const char *not;
 	struct token *tk;
-	struct symbol *sym;
+	enum symkind kind;
 
 	*e = NULL;
 
@@ -1215,24 +1209,14 @@ vcc_expr_cmp(struct vcc *tl, struct expr **e, enum var_type fmt)
 		*e = vcc_expr_edit(BOOL, buf, *e, NULL);
 		return;
 	}
-	if ((*e)->fmt == BACKEND &&
-	    (tl->t->tok == T_EQ || tl->t->tok == T_NEQ)) {
-		// XXX: just ask for a BACKEND expression instead ?
-		vcc_NextToken(tl);
-		ExpectErr(tl, ID);
-		sym = VCC_FindSymbol(tl, tl->t, SYM_BACKEND);
-		if (sym == NULL) {
-			VSB_printf(tl->sb, "Backend not found: ");
-			vcc_ErrToken(tl, tl->t);
-			VSB_printf(tl->sb, "\n");
-			vcc_ErrWhere(tl, tl->t);
-			return;
-		}
-		vcc_AddRef(tl, tl->t, SYM_BACKEND);
-		bprintf(buf, "(\v1 %.*s %s)", PF(tk),
-		    (const char *)sym->eval_priv);
+	kind = VCC_HandleKind((*e)->fmt);
+	if (kind != SYM_NONE && (tl->t->tok == T_EQ || tl->t->tok == T_NEQ)) {
+		bprintf(buf, "(\v1 %.*s \v2)", PF(tk));
 		vcc_NextToken(tl);
-		*e = vcc_expr_edit(BOOL, buf, *e, NULL);
+		e2 = NULL;
+		vcc_expr0(tl, &e2, (*e)->fmt);
+		ERRCHK(tl);
+		*e = vcc_expr_edit(BOOL, buf, *e, e2);
 		return;
 	}
 	switch (tl->t->tok) {
diff --git a/lib/libvcc/vcc_symb.c b/lib/libvcc/vcc_symb.c
index 02e9fe0..762e27b 100644
--- a/lib/libvcc/vcc_symb.c
+++ b/lib/libvcc/vcc_symb.c
@@ -28,6 +28,7 @@
 
 #include "config.h"
 
+#include <ctype.h>
 #include <stdarg.h>
 #include <stdio.h>
 #include <stdlib.h>
@@ -37,6 +38,18 @@
 
 /*--------------------------------------------------------------------*/
 
+enum symkind
+VCC_HandleKind(enum var_type fmt)
+{
+	switch(fmt) {
+	case ACL:       return(SYM_ACL);
+	case BACKEND:   return(SYM_BACKEND);
+	case PROBE:     return(SYM_PROBE);
+	case STEVEDORE: return(SYM_STEVEDORE);
+	default:        return(SYM_NONE);
+	}
+}
+
 const char *
 VCC_SymKind(struct vcc *tl, const struct symbol *s)
 {
@@ -141,25 +154,68 @@ VCC_WalkSymbols(struct vcc *tl, symwalk_f *func, enum symkind kind)
 	}
 }
 
-void
-VCC_GenericSymbol(struct vcc *tl, struct symbol *sym,
-    enum var_type fmt, const char *str, ...)
+static void
+vcc_global(struct vcc *tl, struct symbol *sym,
+    enum var_type fmt, const char *str, va_list ap)
 {
 	struct vsb *vsb;
-	va_list ap;
 
 	vsb = VSB_new_auto();
 	AN(vsb);
-	va_start(ap, str);
 	VSB_vprintf(vsb, str, ap);
-	va_end(ap);
 	AZ(VSB_finish(vsb));
 	if (tl != NULL)
-		sym->eval_priv = TlDup(tl, VSB_data(vsb));
+		sym->rname = TlDup(tl, VSB_data(vsb));
 	else
-		sym->eval_priv = strdup(VSB_data(vsb));
-	AN(sym->eval_priv);
-	sym->eval = vcc_Eval_Generic;
-	sym->fmt = fmt;
+		sym->rname = strdup(VSB_data(vsb));
+	AN(sym->rname);
 	VSB_destroy(&vsb);
+	sym->fmt = fmt;
+	sym->kind = VCC_HandleKind(sym->fmt);
+	if (sym->kind != SYM_NONE)
+		sym->eval = vcc_Eval_Handle;
+	else
+		WRONG("Wrong kind of global symbol");
+
+#define VCL_MET_MAC(l,u,t,b)   sym->r_methods |= VCL_MET_##u;
+#include "tbl/vcl_returns.h"
+#undef VCL_MET_MAC
+}
+
+void
+VCC_GlobalSymbol(struct symbol *sym, enum var_type fmt, const char *str, ...)
+{
+	va_list ap;
+
+	va_start(ap, str);
+	vcc_global(NULL, sym, fmt, str, ap);
+	va_end(ap);
+}
+
+struct symbol *
+VCC_HandleSymbol(struct vcc *tl, const struct token *tk, enum var_type fmt,
+    const char *str, ...)
+{
+	struct symbol *sym;
+	enum symkind kind;
+	va_list ap;
+	const char *p;
+
+	kind = VCC_HandleKind(fmt);
+	assert(kind != SYM_NONE);
+
+	sym = VCC_GetSymbolTok(tl, tk, kind);
+	AN(sym);
+	if (sym->ndef > 0) {
+		p = VCC_SymKind(tl, sym);
+		VSB_printf(tl->sb, "%c%s %.*s redefined\n",
+		    toupper(*p), p+1, PF(tk));
+		vcc_ErrWhere(tl, tk);
+		return (sym);
+	}
+	va_start(ap, str);
+	vcc_global(tl, sym, fmt, str, ap);
+	va_end(ap);
+	sym->ndef = 1;
+	return (sym);
 }



From hermunn at varnish-software.com  Thu May 19 08:43:06 2016
From: hermunn at varnish-software.com (Pål Hermunn Johansen)
Date: Thu, 19 May 2016 10:43:06 +0200
Subject: [4.1] 61a66b7 Correctly handle EOF responses with HTTP/1.1
Message-ID: <E1b3JXu-00068K-BI@project.varnish-software.com>


commit 61a66b7b04e8ed32bbc89ead91389968a2811d47
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Thu May 5 16:44:25 2016 +0100

    Correctly handle EOF responses with HTTP/1.1
    
    Fixes #1918.

diff --git a/bin/varnishd/http1/cache_http1_proto.c b/bin/varnishd/http1/cache_http1_proto.c
index 49478da..67d7b2b 100644
--- a/bin/varnishd/http1/cache_http1_proto.c
+++ b/bin/varnishd/http1/cache_http1_proto.c
@@ -277,7 +277,7 @@ http1_splitline(struct http *hp, struct http_conn *htc, const int *hf)
 /*--------------------------------------------------------------------*/
 
 static enum body_status
-http1_body_status(const struct http *hp, struct http_conn *htc)
+http1_body_status(const struct http *hp, struct http_conn *htc, int request)
 {
 	ssize_t cl;
 	const char *b;
@@ -307,7 +307,7 @@ http1_body_status(const struct http *hp, struct http_conn *htc)
 		return (cl == 0 ? BS_NONE : BS_LENGTH);
 	}
 
-	if (hp->protover == 11)
+	if (hp->protover == 11 && (request || !http_HdrIs(hp, H_Connection, "close")))
 		return (BS_NONE);
 
 	if (http_HdrIs(hp, H_Connection, "keep-alive")) {
@@ -378,7 +378,7 @@ HTTP1_DissectRequest(struct http_conn *htc, struct http *hp)
 		}
 	}
 
-	htc->body_status = http1_body_status(hp, htc);
+	htc->body_status = http1_body_status(hp, htc, 1);
 	if (htc->body_status == BS_ERROR)
 		return (400);
 
@@ -456,7 +456,7 @@ HTTP1_DissectResponse(struct http_conn *htc, struct http *hp,
 	    !Tlen(hp->hd[HTTP_HDR_REASON]))
 		http_SetH(hp, HTTP_HDR_REASON, http_Status2Reason(hp->status));
 
-	htc->body_status = http1_body_status(hp, htc);
+	htc->body_status = http1_body_status(hp, htc, 0);
 
 	return (retval);
 }
diff --git a/bin/varnishtest/tests/r01918.vtc b/bin/varnishtest/tests/r01918.vtc
new file mode 100644
index 0000000..c0593aa
--- /dev/null
+++ b/bin/varnishtest/tests/r01918.vtc
@@ -0,0 +1,22 @@
+varnishtest "Check EOF responses work with HTTP/1.1"
+
+server s1 {
+	rxreq
+	txresp -nolen -bodylen 10
+	rxreq
+	txresp -hdr "Connection: close" -nolen -bodylen 10
+} -start
+
+varnish v1 -vcl+backend {
+} -start
+
+client c1 {
+	txreq -url /1
+	rxresp
+	expect resp.status == 200
+	expect resp.bodylen == 0
+	txreq -url /2
+	rxresp
+	expect resp.status == 200
+	expect resp.bodylen == 10
+} -run



From phk at FreeBSD.org  Thu May 19 10:14:06 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 19 May 2016 12:14:06 +0200
Subject: [master] 252f371 Save the VCL syntax version,
 we will need it later on.
Message-ID: <E1b3Kxy-0003Oh-FJ@project.varnish-software.com>


commit 252f37106a5efca30da75d08fc21f05c8220dc10
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu May 19 09:59:45 2016 +0000

    Save the VCL syntax version, we will need it later on.

diff --git a/lib/libvcc/vcc_compile.h b/lib/libvcc/vcc_compile.h
index 647c760..c9c1396 100644
--- a/lib/libvcc/vcc_compile.h
+++ b/lib/libvcc/vcc_compile.h
@@ -175,6 +175,7 @@ struct vcp {
 struct vcc {
 	unsigned		magic;
 #define VCC_MAGIC		0x24ad719d
+	float			syntax;
 
 	/* Parameter/Template section */
 	const struct vcp	*param;
diff --git a/lib/libvcc/vcc_parse.c b/lib/libvcc/vcc_parse.c
index 8a94749..811c90f 100644
--- a/lib/libvcc/vcc_parse.c
+++ b/lib/libvcc/vcc_parse.c
@@ -291,17 +291,17 @@ vcc_ParseDirector(struct vcc *tl)
 static void
 vcc_ParseVcl(struct vcc *tl)
 {
-	double ver;
 	struct token *tok;
 
 	assert(vcc_IdIs(tl->t, "vcl"));
 	vcc_NextToken(tl);
 	tok = tl->t;
-	ver = vcc_DoubleVal(tl);
+	tl->syntax = vcc_DoubleVal(tl);
 	ERRCHK(tl);
-	if (ver != 4.0) {
+	if (tl->syntax != 4.0) {
 		// see TODO above
-		VSB_printf(tl->sb, "VCL version %.1f not supported.\n", ver);
+		VSB_printf(tl->sb, "VCL version %.1f not supported.\n",
+		    tl->syntax);
 		vcc_ErrWhere(tl, tok);
 		ERRCHK(tl);
 	}



From phk at FreeBSD.org  Thu May 19 10:14:06 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 19 May 2016 12:14:06 +0200
Subject: [master] 4c36d7c Replace EncString() with VSB_quote()
Message-ID: <E1b3Kxy-0003On-HJ@project.varnish-software.com>


commit 4c36d7ca34444ffe304c2fb9f6cf5b27505213d1
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu May 19 10:13:08 2016 +0000

    Replace EncString() with VSB_quote()

diff --git a/include/vsb.h b/include/vsb.h
index 4c2c9b7..35e752b 100644
--- a/include/vsb.h
+++ b/include/vsb.h
@@ -79,6 +79,7 @@ void		 VSB_destroy(struct vsb **);
 #define VSB_QUOTE_NONL	1
 #define VSB_QUOTE_JSON	2
 #define VSB_QUOTE_HEX	4
+#define VSB_QUOTE_CSTR	8
 void		 VSB_quote(struct vsb *, const void *, int len, int how);
 void		 VSB_indent(struct vsb *, int);
 #ifdef __cplusplus
diff --git a/lib/libvarnish/vsb.c b/lib/libvarnish/vsb.c
index 7a15f56..6a6f169 100644
--- a/lib/libvarnish/vsb.c
+++ b/lib/libvarnish/vsb.c
@@ -533,13 +533,18 @@ VSB_quote(struct vsb *s, const void *v, int len, int how)
 			break;
 		}
 	}
-	if (!quote && !(how & VSB_QUOTE_JSON)) {
+	if (!quote && !(how & (VSB_QUOTE_JSON|VSB_QUOTE_CSTR))) {
 		(void)VSB_bcat(s, p, len);
 		return;
 	}
 	(void)VSB_putc(s, '"');
 	for (q = p; q < p + len; q++) {
 		switch (*q) {
+		case '?':
+			if (how & VSB_QUOTE_CSTR)
+				(void)VSB_putc(s, '\\');
+			(void)VSB_putc(s, *q);
+			break;
 		case ' ':
 			(void)VSB_putc(s, *q);
 			break;
@@ -549,7 +554,9 @@ VSB_quote(struct vsb *s, const void *v, int len, int how)
 			(void)VSB_putc(s, *q);
 			break;
 		case '\n':
-			if (how & VSB_QUOTE_NONL)
+			if (how & VSB_QUOTE_CSTR)
+				(void)VSB_cat(s, "\"\n\t\"");
+			else if (how & VSB_QUOTE_NONL)
 				(void)VSB_cat(s, "\n");
 			else
 				(void)VSB_cat(s, "\\n");
diff --git a/lib/libvcc/vcc_compile.c b/lib/libvcc/vcc_compile.c
index bc22cad..4a182cf 100644
--- a/lib/libvcc/vcc_compile.c
+++ b/lib/libvcc/vcc_compile.c
@@ -203,45 +203,11 @@ Fc(const struct vcc *tl, int indent, const char *fmt, ...)
 /*--------------------------------------------------------------------*/
 
 void
-EncString(struct vsb *sb, const char *b, const char *e, int mode)
-{
-
-	if (e == NULL)
-		e = strchr(b, '\0');
-
-	VSB_cat(sb, "\"");
-	for (; b < e; b++) {
-		switch (*b) {
-		case '?':	// Trigraphs
-		case '\\':
-		case '"':
-			VSB_printf(sb, "\\%c", *b);
-			break;
-		case '\n':
-			VSB_printf(sb, "\\n");
-			if (mode)
-				VSB_printf(sb, "\"\n\t\"");
-			break;
-		case '\t': VSB_printf(sb, "\\t"); break;
-		case '\r': VSB_printf(sb, "\\r"); break;
-		case ' ': VSB_printf(sb, " "); break;
-		default:
-			if (isgraph(*b))
-				VSB_printf(sb, "%c", *b);
-			else
-				VSB_printf(sb, "\\%03o", (uint8_t)*b);
-			break;
-		}
-	}
-	VSB_cat(sb, "\"");
-}
-
-void
 EncToken(struct vsb *sb, const struct token *t)
 {
 
 	assert(t->tok == CSTR);
-	EncString(sb, t->dec, NULL, 1);
+	VSB_quote(sb, t->dec, -1, VSB_QUOTE_CSTR);
 }
 
 /*--------------------------------------------------------------------
@@ -264,7 +230,7 @@ EmitCoordinates(const struct vcc *tl, struct vsb *vsb)
 	VSB_printf(vsb, "\nstatic const char *srcname[VGC_NSRCS] = {\n");
 	VTAILQ_FOREACH(sp, &tl->sources, list) {
 		VSB_printf(vsb, "\t");
-		EncString(vsb, sp->name, NULL, 0);
+		VSB_quote(vsb, sp->name, -1, VSB_QUOTE_CSTR);
 		VSB_printf(vsb, ",\n");
 	}
 	VSB_printf(vsb, "};\n");
@@ -272,10 +238,10 @@ EmitCoordinates(const struct vcc *tl, struct vsb *vsb)
 	VSB_printf(vsb, "\nstatic const char *srcbody[%u] = {\n", tl->nsources);
 	VTAILQ_FOREACH(sp, &tl->sources, list) {
 		VSB_printf(vsb, "    /* ");
-		EncString(vsb, sp->name, NULL, 0);
+		VSB_quote(vsb, sp->name, -1, VSB_QUOTE_CSTR);
 		VSB_printf(vsb, "*/\n");
 		VSB_printf(vsb, "\t");
-		EncString(vsb, sp->b, sp->e, 1);
+		VSB_quote(vsb, sp->b, sp->e - sp->b, VSB_QUOTE_CSTR);
 		VSB_printf(vsb, ",\n");
 	}
 	VSB_printf(vsb, "};\n\n");
diff --git a/lib/libvcc/vcc_compile.h b/lib/libvcc/vcc_compile.h
index c9c1396..da00d26 100644
--- a/lib/libvcc/vcc_compile.h
+++ b/lib/libvcc/vcc_compile.h
@@ -278,8 +278,6 @@ void *TlAlloc(struct vcc *tl, unsigned len);
 char *TlDup(struct vcc *tl, const char *s);
 char *TlDupTok(struct vcc *tl, const struct token *tok);
 
-void EncString(struct vsb *sb, const char *b, const char *e, int mode);
-
 /* vcc_expr.c */
 double vcc_DoubleVal(struct vcc *tl);
 void vcc_Duration(struct vcc *tl, double *);
diff --git a/lib/libvcc/vcc_vmod.c b/lib/libvcc/vcc_vmod.c
index aca8890..5d36bd3 100644
--- a/lib/libvcc/vcc_vmod.c
+++ b/lib/libvcc/vcc_vmod.c
@@ -200,7 +200,7 @@ vcc_ParseImport(struct vcc *tl)
 	VSB_printf(ifp->ini, "\t    sizeof(Vmod_%.*s_Func),\n", PF(mod));
 	VSB_printf(ifp->ini, "\t    \"%.*s\",\n", PF(mod));
 	VSB_printf(ifp->ini, "\t    ");
-	EncString(ifp->ini, fnp, NULL, 0);
+	VSB_quote(ifp->ini, fnp, -1, VSB_QUOTE_CSTR);
 	VSB_printf(ifp->ini, ",\n");
 	AN(vmd);
 	AN(vmd->file_id);



From nils.goroll at uplex.de  Thu May 19 15:55:07 2016
From: nils.goroll at uplex.de (Nils Goroll)
Date: Thu, 19 May 2016 17:55:07 +0200
Subject: [master] 4d8aa0c ban lurker: When loosing the lock race to lookup,
 continue with other objects if possible
Message-ID: <E1b3QHz-0008L1-SJ@project.varnish-software.com>


commit 4d8aa0c306664ca93194d80de82ba108b712f36e
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Tue May 3 16:13:14 2016 +0200

    ban lurker: When loosing the lock race to lookup, continue with other objects if possible

diff --git a/bin/varnishd/cache/cache_ban_lurker.c b/bin/varnishd/cache/cache_ban_lurker.c
index 6b33465..2712ef3 100644
--- a/bin/varnishd/cache/cache_ban_lurker.c
+++ b/bin/varnishd/cache/cache_ban_lurker.c
@@ -36,7 +36,8 @@
 #include "hash/hash_slinger.h"
 #include "vtim.h"
 
-static struct objcore oc_marker = { .magic = OBJCORE_MAGIC, };
+static struct objcore oc_mark_cnt = { .magic = OBJCORE_MAGIC, };
+static struct objcore oc_mark_end = { .magic = OBJCORE_MAGIC, };
 static unsigned ban_batch;
 static unsigned ban_generation;
 
@@ -110,23 +111,56 @@ ban_cleantail(const struct ban *victim)
  * makes most sense in HSH_Lookup(), but we come the other way.
  * We optimistically try to get them the other way, and get out of
  * the way if that fails, and retry again later.
+ *
+ * To avoid hammering on contested ocs, we first move those behind a marker
+ * once. When we only have contested ocs left, we stop moving them around and
+ * re-try them in order.
  */
 
 static struct objcore *
 ban_lurker_getfirst(struct vsl_log *vsl, struct ban *bt)
 {
 	struct objhead *oh;
-	struct objcore *oc;
+	struct objcore *oc, *noc;
+	int move_oc = 1;
+
+	Lck_Lock(&ban_mtx);
 
+	oc = VTAILQ_FIRST(&bt->objcore);
 	while (1) {
-		Lck_Lock(&ban_mtx);
-		oc = VTAILQ_FIRST(&bt->objcore);
 		CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
-		if (oc == &oc_marker) {
-			VTAILQ_REMOVE(&bt->objcore, oc, ban_list);
+
+		if (oc == &oc_mark_cnt) {
+			if (VTAILQ_NEXT(oc, ban_list) == &oc_mark_end) {
+				/* done with this ban's oc list */
+				VTAILQ_REMOVE(&bt->objcore, &oc_mark_cnt,
+				    ban_list);
+				VTAILQ_REMOVE(&bt->objcore, &oc_mark_end,
+				    ban_list);
+				oc = NULL;
+				break;
+			}
+			oc = VTAILQ_NEXT(oc, ban_list);
+			CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
+			move_oc = 0;
+		} else if (oc == &oc_mark_end) {
+			assert(move_oc == 0);
+
+			/* hold off to give lookup a chance and reiterate */
 			Lck_Unlock(&ban_mtx);
-			return (NULL);
+			VSC_C_main->bans_lurker_contention++;
+			VSL_Flush(vsl, 0);
+			VTIM_sleep(cache_param->ban_lurker_holdoff);
+			Lck_Lock(&ban_mtx);
+
+			oc = VTAILQ_FIRST(&bt->objcore);
+			assert(oc == &oc_mark_cnt);
+			continue;
 		}
+
+		assert(oc != &oc_mark_cnt);
+		assert(oc != &oc_mark_end);
+
 		oh = oc->objhead;
 		CHECK_OBJ_NOTNULL(oh, OBJHEAD_MAGIC);
 		if (!Lck_Trylock(&oh->mtx)) {
@@ -135,24 +169,28 @@ ban_lurker_getfirst(struct vsl_log *vsl, struct ban *bt)
 			} else {
 				/*
 				 * We got the lock, and the oc is not being
-				 * dismantled under our feet.
+				 * dismantled under our feet - grab a ref
 				 */
 				AZ(oc->flags & OC_F_BUSY);
 				oc->refcnt += 1;
 				VTAILQ_REMOVE(&bt->objcore, oc, ban_list);
 				VTAILQ_INSERT_TAIL(&bt->objcore, oc, ban_list);
 				Lck_Unlock(&oh->mtx);
-				Lck_Unlock(&ban_mtx);
 				break;
 			}
 		}
 
-		/* Try again, later */
-		Lck_Unlock(&ban_mtx);
-		VSC_C_main->bans_lurker_contention++;
-		VSL_Flush(vsl, 0);
-		VTIM_sleep(cache_param->ban_lurker_holdoff);
+		noc = VTAILQ_NEXT(oc, ban_list);
+
+		if (move_oc) {
+			/* contested ocs go between the two markers */
+			VTAILQ_REMOVE(&bt->objcore, oc, ban_list);
+			VTAILQ_INSERT_BEFORE(&oc_mark_end, oc, ban_list);
+		}
+
+		oc = noc;
 	}
+	Lck_Unlock(&ban_mtx);
 	return (oc);
 }
 
@@ -166,12 +204,14 @@ ban_lurker_test_ban(struct worker *wrk, struct vsl_log *vsl, struct ban *bt,
 	int i;
 
 	/*
-	 * First see if there is anything to do, and if so, insert marker
+	 * First see if there is anything to do, and if so, insert markers
 	 */
 	Lck_Lock(&ban_mtx);
 	oc = VTAILQ_FIRST(&bt->objcore);
-	if (oc != NULL)
-		VTAILQ_INSERT_TAIL(&bt->objcore, &oc_marker, ban_list);
+	if (oc != NULL) {
+		VTAILQ_INSERT_TAIL(&bt->objcore, &oc_mark_cnt, ban_list);
+		VTAILQ_INSERT_TAIL(&bt->objcore, &oc_mark_end, ban_list);
+	}
 	Lck_Unlock(&ban_mtx);
 	if (oc == NULL)
 		return;



From phk at FreeBSD.org  Fri May 20 09:01:07 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Fri, 20 May 2016 11:01:07 +0200
Subject: [master] e41fd1c Refactor a VCC error message
Message-ID: <E1b3gIt-0007eT-Bq@project.varnish-software.com>


commit e41fd1cbce06129062900dc8e362ed2212c9a295
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri May 20 09:00:07 2016 +0000

    Refactor a VCC error message

diff --git a/bin/varnishtest/tests/a00000.vtc b/bin/varnishtest/tests/a00000.vtc
index d3fe2fd..eb42f48 100644
--- a/bin/varnishtest/tests/a00000.vtc
+++ b/bin/varnishtest/tests/a00000.vtc
@@ -1,5 +1,6 @@
 # the first token in a varnishtest file must be "varnishtest",
- 	# but whitespace and comments are fine
+    # but whitespace (SP)
+	# and (TAB) and comments are fine
 
 varnishtest "basic default HTTP transactions"
 
diff --git a/bin/varnishtest/tests/d00006.vtc b/bin/varnishtest/tests/d00006.vtc
index e98d66a..195e8ba 100644
--- a/bin/varnishtest/tests/d00006.vtc
+++ b/bin/varnishtest/tests/d00006.vtc
@@ -22,7 +22,7 @@ server s4 {
 	txresp -body "4444"
 } -start
 
-varnish v1 -errvcl {Names of VCL objects cannot contain '-'} {
+varnish v1 -errvcl {Name of VCL object, 'rr1-xx', contains illegal character '-'} {
 	import directors;
 	backend b1 { .host = "127.0.0.1"; .port = "8080";}
 	sub vcl_init {
diff --git a/bin/varnishtest/tests/v00020.vtc b/bin/varnishtest/tests/v00020.vtc
index a05b7fb..d8e4622 100644
--- a/bin/varnishtest/tests/v00020.vtc
+++ b/bin/varnishtest/tests/v00020.vtc
@@ -257,7 +257,7 @@ varnish v1 -vcl {
 	}
 }
 
-varnish v1 -errvcl {Names of VCL sub's cannot contain '-'} {
+varnish v1 -errvcl {Name of function, 'foo-bar', contains illegal character '-'} {
 	backend b { .host = "127.0.0.1"; }
 	sub foo-bar {
 	}
@@ -291,7 +291,7 @@ varnish v1 -errvcl {'beresp.status': Not available in method 'vcl_recv'.} {
 	}
 }
 
-varnish v1 -errvcl {Names of VCL acl's cannot contain '-'} {
+varnish v1 -errvcl {Name of ACL, 'foo-bar', contains illegal character '-'} {
 	backend b { .host = "127.0.0.1"; }
 	acl foo-bar {
 	}
diff --git a/bin/varnishtest/tests/v00047.vtc b/bin/varnishtest/tests/v00047.vtc
index 9ca2a6c..4f03588 100644
--- a/bin/varnishtest/tests/v00047.vtc
+++ b/bin/varnishtest/tests/v00047.vtc
@@ -1,10 +1,10 @@
 varnishtest "Changing Last Modified in vcl_deliver"
- 
+
 server s1 {
 	rxreq
 	txresp -hdr "Last-Modified: Wed, 27 Apr 2016 14:00:00 GMT"
 } -start
- 
+
 varnish v1 -vcl+backend {
 	sub vcl_deliver {
 		if (req.http.deliver == "modify") {
@@ -12,7 +12,7 @@ varnish v1 -vcl+backend {
 		}
 	}
 } -start
- 
+
 client c1 {
 	txreq
 	rxresp
diff --git a/lib/libvcc/vcc_acl.c b/lib/libvcc/vcc_acl.c
index c6f57fb..18d967d 100644
--- a/lib/libvcc/vcc_acl.c
+++ b/lib/libvcc/vcc_acl.c
@@ -479,13 +479,8 @@ vcc_ParseAcl(struct vcc *tl)
 	vcc_NextToken(tl);
 	VTAILQ_INIT(&tl->acl);
 
-	ExpectErr(tl, ID);
-	if (!vcc_isCid(tl->t)) {
-		VSB_printf(tl->sb,
-		    "Names of VCL acl's cannot contain '-'\n");
-		vcc_ErrWhere(tl, tl->t);
-		return;
-	}
+	vcc_ExpectCid(tl, "ACL");
+	ERRCHK(tl);
 	an = tl->t;
 	vcc_NextToken(tl);
 
diff --git a/lib/libvcc/vcc_action.c b/lib/libvcc/vcc_action.c
index 425b993..2e1a2fb 100644
--- a/lib/libvcc/vcc_action.c
+++ b/lib/libvcc/vcc_action.c
@@ -156,12 +156,8 @@ parse_new(struct vcc *tl)
 
 	vcc_NextToken(tl);
 	ExpectErr(tl, ID);
-	if (!vcc_isCid(tl->t)) {
-		VSB_printf(tl->sb,
-		    "Names of VCL objects cannot contain '-'\n");
-		vcc_ErrWhere(tl, tl->t);
-		return;
-	}
+	vcc_ExpectCid(tl, "VCL object");
+	ERRCHK(tl);
 	sy1 = VCC_FindSymbol(tl, tl->t, SYM_NONE);
 	if (sy1 != NULL) {
 		VSB_printf(tl->sb, "Object name '%.*s' already used.\n",
diff --git a/lib/libvcc/vcc_backend.c b/lib/libvcc/vcc_backend.c
index 6e9c498..abf454d 100644
--- a/lib/libvcc/vcc_backend.c
+++ b/lib/libvcc/vcc_backend.c
@@ -251,9 +251,9 @@ vcc_ParseProbe(struct vcc *tl)
 	struct token *t_probe;
 	char *p;
 
-	vcc_NextToken(tl);		/* ID: probe */
+	vcc_NextToken(tl);			/* ID: probe */
 
-	vcc_ExpectCid(tl);		/* ID: name */
+	vcc_ExpectCid(tl, "backend probe");	/* ID: name */
 	ERRCHK(tl);
 	t_probe = tl->t;
 	vcc_NextToken(tl);
@@ -462,7 +462,7 @@ vcc_ParseBackend(struct vcc *tl)
 	t_first = tl->t;
 	vcc_NextToken(tl);		/* ID: backend */
 
-	vcc_ExpectCid(tl);		/* ID: name */
+	vcc_ExpectCid(tl, "backend");	/* ID: name */
 	ERRCHK(tl);
 
 	if (tl->t->e - tl->t->b > MAX_BACKEND_NAME) {
diff --git a/lib/libvcc/vcc_compile.h b/lib/libvcc/vcc_compile.h
index da00d26..e9e72cc 100644
--- a/lib/libvcc/vcc_compile.h
+++ b/lib/libvcc/vcc_compile.h
@@ -54,7 +54,6 @@ struct sockaddr_storage;
 #define isident1(c) (isalpha(c))
 #define isident(c) (isalpha(c) || isdigit(c) || (c) == '_' || (c) == '-')
 #define isvar(c) (isident(c) || (c) == '.')
-int vcc_isCid(const struct token *t);
 unsigned vcl_fixed_token(const char *p, const char **q);
 extern const char * const vcl_tnames[256];
 void vcl_output_lang_h(struct vsb *sb);
@@ -333,7 +332,7 @@ void vcc_ErrWhere2(struct vcc *, const struct token *, const struct token *);
 
 void vcc__Expect(struct vcc *tl, unsigned tok, unsigned line);
 int vcc_IdIs(const struct token *t, const char *p);
-void vcc_ExpectCid(struct vcc *tl);
+void vcc_ExpectCid(struct vcc *tl, const char *what);
 void vcc_Lexer(struct vcc *tl, struct source *sp);
 void vcc_NextToken(struct vcc *tl);
 void vcc__ErrInternal(struct vcc *tl, const char *func,
diff --git a/lib/libvcc/vcc_expr.c b/lib/libvcc/vcc_expr.c
index 1d0108c..7ad85ee 100644
--- a/lib/libvcc/vcc_expr.c
+++ b/lib/libvcc/vcc_expr.c
@@ -509,7 +509,7 @@ vcc_Eval_Handle(struct vcc *tl, struct expr **e, const struct symbol *sym,
 		*e = vcc_mk_expr(STRING, "\"%s\"", sym->name);
 		vcc_AddRef(tl, tl->t, sym->kind);
 	} else {
-		vcc_ExpectCid(tl);
+		vcc_ExpectCid(tl, "handle");
 		vcc_AddRef(tl, tl->t, sym->kind);
 		*e = vcc_mk_expr(sym->fmt, "%s", sym->rname);
 		(*e)->constant = EXPR_VAR;	/* XXX ? */
@@ -1196,7 +1196,7 @@ vcc_expr_cmp(struct vcc *tl, struct expr **e, enum var_type fmt)
 	    (tl->t->tok == '~' || tl->t->tok == T_NOMATCH)) {
 		not = tl->t->tok == '~' ? "" : "!";
 		vcc_NextToken(tl);
-		vcc_ExpectCid(tl);
+		vcc_ExpectCid(tl, "ACL");
 		vcc_AddRef(tl, tl->t, SYM_ACL);
 		bprintf(buf, "%smatch_acl_named_%.*s(ctx, \v1)",
 		    not, PF(tl->t));
diff --git a/lib/libvcc/vcc_parse.c b/lib/libvcc/vcc_parse.c
index 811c90f..e6df4c8 100644
--- a/lib/libvcc/vcc_parse.c
+++ b/lib/libvcc/vcc_parse.c
@@ -211,13 +211,8 @@ vcc_ParseFunction(struct vcc *tl)
 	int m, i;
 
 	vcc_NextToken(tl);
-	ExpectErr(tl, ID);
-	if (!vcc_isCid(tl->t)) {
-		VSB_printf(tl->sb,
-		    "Names of VCL sub's cannot contain '-'\n");
-		vcc_ErrWhere(tl, tl->t);
-		return;
-	}
+	vcc_ExpectCid(tl, "function");
+	ERRCHK(tl);
 
 	m = IsMethod(tl->t);
 	if (m == -2) {
diff --git a/lib/libvcc/vcc_token.c b/lib/libvcc/vcc_token.c
index a8513f8..c4771bd 100644
--- a/lib/libvcc/vcc_token.c
+++ b/lib/libvcc/vcc_token.c
@@ -293,32 +293,23 @@ vcc_IdIs(const struct token *t, const char *p)
  * Check that we have a C-identifier
  */
 
-int
-vcc_isCid(const struct token *t)
-{
-	const char *q;
-
-	assert(t->tok == ID);
-	for (q = t->b; q < t->e; q++) {
-		if (!isalnum(*q) && *q != '_')
-			return (0);
-	}
-	return (1);
-}
-
 void
-vcc_ExpectCid(struct vcc *tl)
+vcc_ExpectCid(struct vcc *tl, const char *what)
 {
+	const char *q;
 
 	ExpectErr(tl, ID);
 	ERRCHK(tl);
-	if (vcc_isCid(tl->t))
-		return;
-	VSB_printf(tl->sb, "Identifier ");
-	vcc_ErrToken(tl, tl->t);
-	VSB_printf(tl->sb,
-	    " contains illegal characters, use [0-9a-zA-Z_] only.\n");
-	vcc_ErrWhere(tl, tl->t);
+	for (q = tl->t->b; q < tl->t->e; q++) {
+		if (!isalnum(*q) && *q != '_') {
+			VSB_printf(tl->sb, "Name of %s, ", what);
+			vcc_ErrToken(tl, tl->t);
+			VSB_printf(tl->sb,
+			    ", contains illegal character '%c'\n", *q);
+			vcc_ErrWhere(tl, tl->t);
+			return;
+		}
+	}
 }
 
 /*--------------------------------------------------------------------



From nils.goroll at uplex.de  Fri May 20 09:04:07 2016
From: nils.goroll at uplex.de (Nils Goroll)
Date: Fri, 20 May 2016 11:04:07 +0200
Subject: [master] bff2e6d Clarify the help text for d_opt/-d
Message-ID: <E1b3gLn-0007nx-I3@project.varnish-software.com>


commit bff2e6de3d03faefef5fb93005b644020b465682
Author: Nicole Izumi <nicole at kivikakk.ee>
Date:   Fri May 20 11:59:35 2016 +1000

    Clarify the help text for d_opt/-d

diff --git a/include/vut_options.h b/include/vut_options.h
index a5b11ef..16258b8 100644
--- a/include/vut_options.h
+++ b/include/vut_options.h
@@ -30,9 +30,8 @@
 /* VUT options */
 
 #define VUT_OPT_d \
-	VOPT("d", "[-d]", "Process old log entries on startup",		\
-	    "Start processing log records at the head of the log"	\
-	    " instead of the tail."					\
+	VOPT("d", "[-d]", "Process old log entries and exit",		\
+	    "Process log records at the head of the log and exit."	\
 	)
 
 #define VUT_OPT_D							\



From phk at FreeBSD.org  Sat May 21 13:36:05 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Sat, 21 May 2016 15:36:05 +0200
Subject: [master] 467c5bf Renovate the very old tabelization of CLI commands.
Message-ID: <E1b474X-0004ZX-W9@project.varnish-software.com>


commit 467c5bfb2df9efb7a9a2f99b00e07c47a65859c6
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Sat May 21 13:24:33 2016 +0000

    Renovate the very old tabelization of CLI commands.
    
    This makes it possible to add a "varnishd -x dumprstcli" command.

diff --git a/bin/varnishd/cache/cache_acceptor.c b/bin/varnishd/cache/cache_acceptor.c
index 8e62c67..5719720 100644
--- a/bin/varnishd/cache/cache_acceptor.c
+++ b/bin/varnishd/cache/cache_acceptor.c
@@ -45,7 +45,6 @@
 #include "cache_pool.h"
 #include "common/heritage.h"
 
-#include "vcli.h"
 #include "vcli_priv.h"
 #include "vsa.h"
 #include "vtcp.h"
@@ -589,10 +588,8 @@ ccf_listen_address(struct cli *cli, const char * const *av, void *priv)
 /*--------------------------------------------------------------------*/
 
 static struct cli_proto vca_cmds[] = {
-	{ CLI_SERVER_START,	"i", ccf_start },
-	{ "debug.listen_address", "debug.listen_address",
-	    "\tReport the actual listen address.", 0, 0,
-	    "d", ccf_listen_address },
+	{ CLICMD_SERVER_START,		"i", ccf_start },
+	{ CLICMD_DEBUG_LISTEN_ADDRESS,	"d", ccf_listen_address },
 	{ NULL }
 };
 
diff --git a/bin/varnishd/cache/cache_backend_cfg.c b/bin/varnishd/cache/cache_backend_cfg.c
index a1ae800..ad94900 100644
--- a/bin/varnishd/cache/cache_backend_cfg.c
+++ b/bin/varnishd/cache/cache_backend_cfg.c
@@ -423,13 +423,8 @@ cli_backend_set_health(struct cli *cli, const char * const *av, void *priv)
 /*---------------------------------------------------------------------*/
 
 static struct cli_proto backend_cmds[] = {
-	{ "backend.list", "backend.list [-p] [<backend_expression>]",
-	    "\tList backends.",
-	    0, 2, "", cli_backend_list },
-	{ "backend.set_health",
-	    "backend.set_health <backend_expression> <state>",
-	    "\tSet health status on the backends.",
-	    2, 2, "", cli_backend_set_health },
+	{ CLICMD_BACKEND_LIST,		"", cli_backend_list },
+	{ CLICMD_BACKEND_SET_HEALTH,	"", cli_backend_set_health },
 	{ NULL }
 };
 
diff --git a/bin/varnishd/cache/cache_ban.c b/bin/varnishd/cache/cache_ban.c
index a9740e1..302d1e9 100644
--- a/bin/varnishd/cache/cache_ban.c
+++ b/bin/varnishd/cache/cache_ban.c
@@ -748,8 +748,8 @@ ccf_ban_list(struct cli *cli, const char * const *av, void *priv)
 }
 
 static struct cli_proto ban_cmds[] = {
-	{ CLI_BAN,				"", ccf_ban },
-	{ CLI_BAN_LIST,				"", ccf_ban_list },
+	{ CLICMD_BAN,				"", ccf_ban },
+	{ CLICMD_BAN_LIST,			"", ccf_ban_list },
 	{ NULL }
 };
 
diff --git a/bin/varnishd/cache/cache_cli.c b/bin/varnishd/cache/cache_cli.c
index da8ee40..337c2dc 100644
--- a/bin/varnishd/cache/cache_cli.c
+++ b/bin/varnishd/cache/cache_cli.c
@@ -110,8 +110,8 @@ CLI_Run(void)
 /*--------------------------------------------------------------------*/
 
 static struct cli_proto cli_cmds[] = {
-	{ CLI_PING,		"i", VCLS_func_ping },
-	{ CLI_HELP,             "i", VCLS_func_help, VCLS_func_help_json, cli_cmds },
+	{ CLICMD_PING,	"i", VCLS_func_ping },
+	{ CLICMD_HELP,	"i", VCLS_func_help, VCLS_func_help_json, cli_cmds },
 	{ NULL }
 };
 
diff --git a/bin/varnishd/cache/cache_fetch_proc.c b/bin/varnishd/cache/cache_fetch_proc.c
index 9840b3d..49862db 100644
--- a/bin/varnishd/cache/cache_fetch_proc.c
+++ b/bin/varnishd/cache/cache_fetch_proc.c
@@ -233,8 +233,7 @@ debug_fragfetch(struct cli *cli, const char * const *av, void *priv)
 }
 
 static struct cli_proto debug_cmds[] = {
-	{ "debug.fragfetch", "debug.fragfetch",
-		"\tEnable fetch fragmentation.", 1, 1, "d", debug_fragfetch },
+	{ CLICMD_DEBUG_FRAGFETCH,		"d", debug_fragfetch },
 	{ NULL }
 };
 
diff --git a/bin/varnishd/cache/cache_main.c b/bin/varnishd/cache/cache_main.c
index 04b1ee4..413affb 100644
--- a/bin/varnishd/cache/cache_main.c
+++ b/bin/varnishd/cache/cache_main.c
@@ -182,15 +182,11 @@ cli_debug_srandom(struct cli *cli, const char * const *av, void *priv)
 }
 
 static struct cli_proto debug_cmds[] = {
-	{ "debug.xid", "debug.xid",
-		"\tExamine or set XID.", 0, 1, "d", cli_debug_xid },
-	{ "debug.srandom", "debug.srandom",
-		"\tSeed the random(3) function.", 0, 1, "d",
-		cli_debug_srandom },
+	{ CLICMD_DEBUG_XID,			"d", cli_debug_xid },
+	{ CLICMD_DEBUG_SRANDOM,			"d", cli_debug_srandom },
 	{ NULL }
 };
 
-
 /*--------------------------------------------------------------------
  * XXX: Think more about which order we start things
  */
diff --git a/bin/varnishd/cache/cache_panic.c b/bin/varnishd/cache/cache_panic.c
index dc02731..51aeb46 100644
--- a/bin/varnishd/cache/cache_panic.c
+++ b/bin/varnishd/cache/cache_panic.c
@@ -637,9 +637,7 @@ ccf_panic(struct cli *cli, const char * const *av, void *priv)
 /*--------------------------------------------------------------------*/
 
 static struct cli_proto debug_cmds[] = {
-	{ "debug.panic.worker", "debug.panic.worker",
-		"\tPanic the worker process.",
-		0, 0, "d", ccf_panic },
+	{ CLICMD_DEBUG_PANIC_WORKER,		"d",	ccf_panic },
 	{ NULL }
 };
 
diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index 0dcaa73..8b0ac91 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -947,12 +947,12 @@ VCL_##func##_method(struct vcl *vcl, struct worker *wrk,		\
 /*--------------------------------------------------------------------*/
 
 static struct cli_proto vcl_cmds[] = {
-	{ CLI_VCL_LOAD,		"i", ccf_config_load },
-	{ CLI_VCL_LIST,		"i", ccf_config_list },
-	{ CLI_VCL_STATE,	"i", ccf_config_state },
-	{ CLI_VCL_DISCARD,	"i", ccf_config_discard },
-	{ CLI_VCL_USE,		"i", ccf_config_use },
-	{ CLI_VCL_SHOW,		"", ccf_config_show },
+	{ CLICMD_VCL_LOAD,		"i", ccf_config_load },
+	{ CLICMD_VCL_LIST,		"i", ccf_config_list },
+	{ CLICMD_VCL_STATE,		"i", ccf_config_state },
+	{ CLICMD_VCL_DISCARD,		"i", ccf_config_discard },
+	{ CLICMD_VCL_USE,		"i", ccf_config_use },
+	{ CLICMD_VCL_SHOW,		"", ccf_config_show },
 	{ NULL }
 };
 
diff --git a/bin/varnishd/cache/cache_vrt_vmod.c b/bin/varnishd/cache/cache_vrt_vmod.c
index b18ae1d..3904f28 100644
--- a/bin/varnishd/cache/cache_vrt_vmod.c
+++ b/bin/varnishd/cache/cache_vrt_vmod.c
@@ -183,8 +183,7 @@ ccf_debug_vmod(struct cli *cli, const char * const *av, void *priv)
 }
 
 static struct cli_proto vcl_cmds[] = {
-	{ "debug.vmod", "debug.vmod", "\tShow loaded vmods.", 0, 0,
-		"d", ccf_debug_vmod },
+	{ CLICMD_DEBUG_VMOD,			"d", ccf_debug_vmod },
 	{ NULL }
 };
 
diff --git a/bin/varnishd/hash/hash_critbit.c b/bin/varnishd/hash/hash_critbit.c
index 4b9cc6b..9ac2518 100644
--- a/bin/varnishd/hash/hash_critbit.c
+++ b/bin/varnishd/hash/hash_critbit.c
@@ -305,50 +305,6 @@ hcb_delete(struct hcb_root *r, struct objhead *oh)
 
 /*--------------------------------------------------------------------*/
 
-static void
-dumptree(struct cli *cli, uintptr_t p, int indent)
-{
-	int i;
-	const struct objhead *oh;
-	const struct hcb_y *y;
-
-	if (p == 0)
-		return;
-	if (hcb_is_node(p)) {
-		oh = hcb_l_node(p);
-		VCLI_Out(cli, "%*.*sN %d r%u <%02x%02x%02x...>\n",
-		    indent, indent, "", indent / 2, oh->refcnt,
-		    oh->digest[0], oh->digest[1], oh->digest[2]);
-		return;
-	}
-	assert(hcb_is_y(p));
-	y = hcb_l_y(p);
-	VCLI_Out(cli, "%*.*sY c %u p %u b %02x i %d\n",
-	    indent, indent, "",
-	    y->critbit, y->ptr, y->bitmask, indent / 2);
-	indent += 2;
-	for (i = 0; i < 2; i++)
-		dumptree(cli, y->leaf[i], indent);
-}
-
-static void
-hcb_dump(struct cli *cli, const char * const *av, void *priv)
-{
-
-	(void)priv;
-	(void)av;
-	VCLI_Out(cli, "HCB dump:\n");
-	dumptree(cli, hcb_root.origo, 0);
-	VCLI_Out(cli, "Coollist:\n");
-}
-
-static struct cli_proto hcb_cmds[] = {
-	{ "hcb.dump", "hcb.dump", "\tDump HCB tree.", 0, 0, "d", hcb_dump },
-	{ NULL }
-};
-
-/*--------------------------------------------------------------------*/
-
 static void * __match_proto__(bgthread_t)
 hcb_cleaner(struct worker *wrk, void *priv)
 {
@@ -384,7 +340,6 @@ hcb_start(void)
 	pthread_t tp;
 
 	(void)oh;
-	CLI_AddFuncs(hcb_cmds);
 	Lck_New(&hcb_mtx, lck_hcb);
 	WRK_BgThread(&tp, "hcb-cleaner", hcb_cleaner, NULL);
 	memset(&hcb_root, 0, sizeof hcb_root);
diff --git a/bin/varnishd/mgt/mgt.h b/bin/varnishd/mgt/mgt.h
index e230ee6..3d03cc3 100644
--- a/bin/varnishd/mgt/mgt.h
+++ b/bin/varnishd/mgt/mgt.h
@@ -67,6 +67,7 @@ void mgt_cli_telnet(const char *T_arg);
 void mgt_cli_master(const char *M_arg);
 void mgt_cli_secret(const char *S_arg);
 void mgt_cli_close_all(void);
+void mgt_DumpRstCli(void);
 
 /* mgt_jail.c */
 
diff --git a/bin/varnishd/mgt/mgt_cli.c b/bin/varnishd/mgt/mgt_cli.c
index 06e1989..293a5e2 100644
--- a/bin/varnishd/mgt/mgt_cli.c
+++ b/bin/varnishd/mgt/mgt_cli.c
@@ -57,6 +57,19 @@
 
 #include "mgt_cli.h"
 
+#define CLI_CMD(U,l,s,h,d,m,M) \
+const struct cli_cmd_desc CLICMD_##U[1] = {{ l, s, h, d, m, M }};
+#include "tbl/cli_cmds.h"
+#undef CLI_CMD
+
+static const struct cli_cmd_desc *cmds[] = {
+#define CLI_CMD(U,l,s,h,d,m,M) CLICMD_##U,
+#include "tbl/cli_cmds.h"
+#undef CLI_CMD
+};
+
+static const int ncmds = sizeof cmds / sizeof cmds[0];
+
 static int		cli_i = -1, cli_o = -1;
 static struct VCLS	*cls;
 static const char	*secret_file;
@@ -89,20 +102,20 @@ mcf_banner(struct cli *cli, const char *const *av, void *priv)
 
 /* XXX: what order should this list be in ? */
 static struct cli_proto cli_proto[] = {
-	{ CLI_BANNER,		"", mcf_banner },
-	{ CLI_SERVER_STATUS,	"", mcf_server_status },
-	{ CLI_SERVER_START,	"", mcf_server_start },
-	{ CLI_SERVER_STOP,	"", mcf_server_stop },
-	{ CLI_VCL_LOAD,		"", mcf_vcl_load },
-	{ CLI_VCL_INLINE,	"", mcf_vcl_inline },
-	{ CLI_VCL_USE,		"", mcf_vcl_use },
-	{ CLI_VCL_STATE,	"", mcf_vcl_state },
-	{ CLI_VCL_DISCARD,	"", mcf_vcl_discard },
-	{ CLI_VCL_LIST,		"", mcf_vcl_list },
-	{ CLI_PARAM_SHOW,	"", mcf_param_show },
-	{ CLI_PARAM_SET,	"", mcf_param_set },
-	{ CLI_PANIC_SHOW,	"", mcf_panic_show },
-	{ CLI_PANIC_CLEAR,	"", mcf_panic_clear },
+	{ CLICMD_BANNER,		"", mcf_banner },
+	{ CLICMD_SERVER_STATUS,		"", mcf_server_status },
+	{ CLICMD_SERVER_START,		"", mcf_server_start },
+	{ CLICMD_SERVER_STOP,		"", mcf_server_stop },
+	{ CLICMD_VCL_LOAD,		"", mcf_vcl_load },
+	{ CLICMD_VCL_INLINE,		"", mcf_vcl_inline },
+	{ CLICMD_VCL_USE,		"", mcf_vcl_use },
+	{ CLICMD_VCL_STATE,		"", mcf_vcl_state },
+	{ CLICMD_VCL_DISCARD,		"", mcf_vcl_discard },
+	{ CLICMD_VCL_LIST,		"", mcf_vcl_list },
+	{ CLICMD_PARAM_SHOW,		"", mcf_param_show },
+	{ CLICMD_PARAM_SET,		"", mcf_param_set },
+	{ CLICMD_PANIC_SHOW,		"", mcf_panic_show },
+	{ CLICMD_PANIC_CLEAR,		"", mcf_panic_clear },
 	{ NULL }
 };
 
@@ -119,10 +132,7 @@ mcf_panic(struct cli *cli, const char * const *av, void *priv)
 }
 
 static struct cli_proto cli_debug[] = {
-	{ "debug.panic.master", "debug.panic.master",
-		"\tPanic the master process.",
-		0, 0, "d", mcf_panic},
-	{ NULL }
+	{ CLICMD_DEBUG_PANIC_MASTER,		"d", mcf_panic },
 };
 
 /*--------------------------------------------------------------------*/
@@ -177,9 +187,11 @@ mcf_askchild(struct cli *cli, const char * const *av, void *priv)
 	free(q);
 }
 
+static const struct cli_cmd_desc CLICMD_WILDCARD[1] =
+    {{ "*", "<wild-card-entry>", "<fall through to cacher>", "", 0, -1 }};
+
 static struct cli_proto cli_askchild[] = {
-	{ "*", "<wild-card-entry>", "\t<fall through to cacher>\n",
-		0, 9999, "h*", mcf_askchild},
+	{ CLICMD_WILDCARD, "h*", mcf_askchild },
 	{ NULL }
 };
 
@@ -310,10 +322,10 @@ mcf_auth(struct cli *cli, const char *const *av, void *priv)
 }
 
 static struct cli_proto cli_auth[] = {
-	{ CLI_HELP,		"", VCLS_func_help, VCLS_func_help_json },
-	{ CLI_PING,		"", VCLS_func_ping },
-	{ CLI_AUTH,		"", mcf_auth },
-	{ CLI_QUIT,		"", VCLS_func_close },
+	{ CLICMD_HELP,		"", VCLS_func_help, VCLS_func_help_json },
+	{ CLICMD_PING,		"", VCLS_func_ping },
+	{ CLICMD_AUTH,		"", mcf_auth },
+	{ CLICMD_QUIT,		"", VCLS_func_close },
 	{ NULL }
 };
 
@@ -691,3 +703,36 @@ mgt_cli_master(const char *M_arg)
 	M_poker->name = "-M poker";
 	AZ(vev_add(mgt_evb, M_poker));
 }
+
+static int
+cli_cmp(const void *a, const void *b)
+{
+	struct cli_cmd_desc * const * const aa = a;
+	struct cli_cmd_desc * const * const bb = b;
+
+	return (strcmp((*aa)->request, (*bb)->request));
+}
+
+void
+mgt_DumpRstCli(void)
+{
+	const struct cli_cmd_desc *cp;
+	int i, j;
+
+	qsort(cmds, ncmds, sizeof cmds[0], cli_cmp);
+	for (i = 0; i < ncmds; i++, cp++) {
+		cp = cmds[i];
+		if (!strncmp(cp->request, "debug.", 6))
+			continue;
+		printf(".. _ref_cli_%s:\n\n", cp->syntax);
+		printf("%s\n", cp->syntax);
+		for (j = 0; j < strlen(cp->syntax); j++)
+			printf("~");
+		printf("\n");
+		printf("  %s\n", cp->help);
+		if (*cp->doc != '\0')
+			printf("\n%s\n", cp->doc);
+
+		printf("\n");
+	}
+}
diff --git a/bin/varnishd/mgt/mgt_main.c b/bin/varnishd/mgt/mgt_main.c
index 7e598b5..47ae1ea 100644
--- a/bin/varnishd/mgt/mgt_main.c
+++ b/bin/varnishd/mgt/mgt_main.c
@@ -680,6 +680,10 @@ main(int argc, char * const *argv)
 				mgt_DumpRstVsl();
 				exit(0);
 			}
+			if (!strcmp(optarg, "dumprstcli")) {
+				mgt_DumpRstCli();
+				exit(0);
+			}
 			usage();
 			break;
 		default:
diff --git a/bin/varnishd/storage/mgt_stevedore.c b/bin/varnishd/storage/mgt_stevedore.c
index b2acaa8..644b899 100644
--- a/bin/varnishd/storage/mgt_stevedore.c
+++ b/bin/varnishd/storage/mgt_stevedore.c
@@ -70,8 +70,7 @@ stv_cli_list(struct cli *cli, const char * const *av, void *priv)
 /*--------------------------------------------------------------------*/
 
 struct cli_proto cli_stv[] = {
-	{ "storage.list", "storage.list", "\tList storage devices.",
-	    0, 0, "", stv_cli_list },
+	{ CLICMD_STORAGE_LIST,		"", stv_cli_list },
 	{ NULL}
 };
 
diff --git a/bin/varnishd/storage/storage_persistent.c b/bin/varnishd/storage/storage_persistent.c
index d52c2cc..d439517 100644
--- a/bin/varnishd/storage/storage_persistent.c
+++ b/bin/varnishd/storage/storage_persistent.c
@@ -669,14 +669,7 @@ debug_persistent(struct cli *cli, const char * const * av, void *priv)
 }
 
 static struct cli_proto debug_cmds[] = {
-	{ "debug.persistent", "debug.persistent",
-		"Persistent debugging magic:\n"
-		"\tdebug.persistent [<stevedore> [<cmd>]]\n"
-		"With no cmd arg, a summary of the silo is returned.\n"
-		"Possible commands:\n"
-		"\tsync\tClose current segment, open a new one\n"
-		"\tdump\tinclude objcores in silo summary",
-		0, 2, "d", debug_persistent },
+	{ CLICMD_DEBUG_PERSISTENT,		"d", debug_persistent },
 	{ NULL }
 };
 
diff --git a/bin/varnishtest/tests/c00023.vtc b/bin/varnishtest/tests/c00023.vtc
index f8b6a84..74b3cc0 100644
--- a/bin/varnishtest/tests/c00023.vtc
+++ b/bin/varnishtest/tests/c00023.vtc
@@ -146,8 +146,6 @@ client c1 {
 	expect resp.http.X-Varnish == "1028 1018"
 } -run
 
-varnish v1 -cliok "hcb.dump"
-
 varnish v1 -expect sess_conn == 2
 varnish v1 -expect cache_hit == 9
 varnish v1 -expect cache_miss == 9
diff --git a/include/Makefile.am b/include/Makefile.am
index d1c2670..caae1c7 100644
--- a/include/Makefile.am
+++ b/include/Makefile.am
@@ -8,6 +8,7 @@ nobase_pkginclude_HEADERS = \
 	tbl/ban_vars.h \
 	tbl/bo_flags.h \
 	tbl/body_status.h \
+	tbl/cli_cmds.h \
 	tbl/debug_bits.h \
 	tbl/feature_bits.h \
 	tbl/http_headers.h \
diff --git a/include/tbl/cli_cmds.h b/include/tbl/cli_cmds.h
new file mode 100644
index 0000000..4e9d1ba
--- /dev/null
+++ b/include/tbl/cli_cmds.h
@@ -0,0 +1,316 @@
+/*-
+ * Copyright (c) 2006 Verdens Gang AS
+ * Copyright (c) 2006-2011 Varnish Software AS
+ * All rights reserved.
+ *
+ * Author: Poul-Henning Kamp <phk at phk.freebsd.dk>
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED.  IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ *
+ */
+
+/*
+ * These macros define the common data for requests in the CLI protocol.
+ * The fields are:
+ *	const char *	upper-case C-ident request_name
+ *	const char *	request_name
+ *	const char *	request_syntax (for short help)
+ *	const char *	request_help (for long help)
+ *	const char *	documentation (for sphinx)
+ *	int		minimum_arguments
+ *	int		maximum_arguments
+ */
+
+CLI_CMD(BAN,
+	"ban",
+	"ban <field> <operator> <arg> [&& <field> <oper> <arg> ...]",
+	"Mark obsolete all objects where all the conditions match.",
+	"",
+	3, -1
+)
+
+CLI_CMD(BAN_LIST,
+	"ban.list",
+	"ban.list",
+	"List the active bans.",
+
+	"  The output format is:\n\n"
+	"  * Time the ban was issued.\n\n"
+	"  * Objects referencing this ban.\n\n"
+	"  * ``C`` if ban is completed = no further testing against it.\n\n"
+	"  * if ``lurker`` debugging is enabled:\n\n"
+	"    * ``R`` for req.* tests\n\n"
+	"    * ``O`` for obj.* tests\n\n"
+	"    * Pointer to ban object\n\n"
+	"  * Ban specification",
+
+	0, 0
+)
+
+CLI_CMD(VCL_LOAD,
+	"vcl.load",
+	"vcl.load <configname> <filename> [auto|cold|warm]",
+	"Compile and load the VCL file under the name provided.",
+	"",
+	2, 3
+)
+
+CLI_CMD(VCL_INLINE,
+	"vcl.inline",
+	"vcl.inline <configname> <quoted_VCLstring> [auto|cold|warm]",
+	"Compile and load the VCL data under the name provided.",
+
+	"  Multi-line VCL can be input using the here document"
+	" :ref:`ref_syntax`.",
+
+	2, 3
+)
+
+CLI_CMD(VCL_STATE,
+	"vcl.state",
+	"vcl.state <configname> [auto|cold|warm]",
+	"Force the state of the named configuration.",
+	"",
+	2, 2
+)
+
+CLI_CMD(VCL_DISCARD,
+	"vcl.discard",
+	"vcl.discard <configname>",
+	"Unload the named configuration (when possible).",
+	"",
+	1, 1
+)
+
+CLI_CMD(VCL_LIST,
+	"vcl.list",
+	"vcl.list",
+	"List all loaded configuration.",
+	"",
+	0, 0
+)
+
+CLI_CMD(VCL_SHOW,
+	"vcl.show",
+	"vcl.show [-v] <configname>",
+	"Display the source code for the specified configuration.",
+	"",
+	1, 2
+)
+
+CLI_CMD(VCL_USE,
+	"vcl.use",
+	"vcl.use <configname>",
+	"Switch to the named configuration immediately.",
+	"",
+	1, 1
+)
+
+CLI_CMD(PARAM_SHOW,
+	"param.show",
+	"param.show [-l] [<param>]",
+	"Show parameters and their values.",
+	"",
+	0, 2
+)
+
+CLI_CMD(PARAM_SET,
+	"param.set",
+	"param.set <param> <value>",
+	"Set parameter value.",
+	"",
+	2,2
+)
+
+CLI_CMD(SERVER_STOP,
+	"stop",
+	"stop",
+	"Stop the Varnish cache process.",
+	"",
+	0, 0
+)
+
+CLI_CMD(SERVER_START,
+	"start",
+	"start",
+	"Start the Varnish cache process.",
+	"",
+	0, 0
+)
+
+CLI_CMD(PING,
+	"ping",
+	"ping [<timestamp>]",
+	"Keep connection alive.",
+	"",
+	0, 1
+)
+
+CLI_CMD(HELP,
+	"help",
+	"help [<command>]",
+	"Show command/protocol help.",
+	"",
+	0, 1
+)
+
+CLI_CMD(QUIT,
+	"quit",
+	"quit",
+	"Close connection.",
+	"",
+	0, 0
+)
+
+CLI_CMD(SERVER_STATUS,
+	"status",
+	"status",
+	"Check status of Varnish cache process.",
+	"",
+	0, 0
+)
+
+CLI_CMD(BANNER,
+	"banner",
+	"banner",
+	"Print welcome banner.",
+	"",
+	0, 0
+)
+
+CLI_CMD(AUTH,
+	"auth",
+	"auth <response>",
+	"Authenticate.",
+	"",
+	1, 1
+)
+
+CLI_CMD(PANIC_SHOW,
+	"panic.show",
+	"panic.show",
+	"Return the last panic, if any.",
+	"",
+	0, 0
+)
+
+CLI_CMD(PANIC_CLEAR,
+	"panic.clear",
+	"panic.clear [-z]",
+	"Clear the last panic, if any,"
+	" -z will clear related varnishstat counter(s)",
+	"",
+	0, 1
+)
+
+CLI_CMD(DEBUG_LISTEN_ADDRESS,
+	"debug.listen_address",
+	"debug.listen_address",
+	"Report the actual listen address.",
+	"",
+	0, 0
+)
+
+CLI_CMD(BACKEND_LIST,
+	"backend.list",
+	"baclend.list [-p] [<backend_pattern>]",
+	"List backends.  -p also shows probe status.",
+	"",
+	0, 2
+)
+
+CLI_CMD(BACKEND_SET_HEALTH,
+	"backend.set_health",
+	"backend.set_health <backend_pattern> [auto|healthy|sick]",
+	"Set health status on the backends.",
+	"",
+	2, 2
+)
+
+CLI_CMD(DEBUG_FRAGFETCH,
+	"debug.fragfetch",
+	"debug.fragfetch",
+	"Enable fetch fragmentation.",
+	"",
+	1, 1
+)
+
+CLI_CMD(DEBUG_XID,
+	"debug.xid",
+	"debug.xid",
+	"Examine or set XID.",
+	"",
+	0, 1
+)
+
+CLI_CMD(DEBUG_SRANDOM,
+	"debug.srandom",
+	"debug.srandom",
+	"Seed the random(3) function.",
+	"",
+	0, 1
+)
+
+CLI_CMD(DEBUG_PANIC_WORKER,
+	"debug.panic.worker",
+	"debug.panic.worker",
+	"Panic the worker process.",
+	"",
+	0, 0
+)
+
+CLI_CMD(DEBUG_PANIC_MASTER,
+	"debug.panic.master",
+	"debug.panic.master",
+	"Panic the master process.",
+	"",
+	0, 0
+)
+
+CLI_CMD(DEBUG_VMOD,
+	"debug.vmod",
+	"debug.vmod",
+	"Show loaded vmods.",
+	"",
+	0, 0
+)
+
+CLI_CMD(DEBUG_PERSISTENT,
+	"debug.persistent",
+	"debug.persistent [<stevedore>] [<cmd>]",
+	"Persistent debugging magic:\n"
+	"With no cmd arg, a summary of the silo is returned.\n"
+	"Possible commands:\n"
+	"\tsync\tClose current segment, open a new one\n"
+	"\tdump\tinclude objcores in silo summary",
+	"",
+	0, 2
+)
+
+CLI_CMD(STORAGE_LIST,
+	"storage.list",
+	"storage.list",
+	"List storage devices.",
+	"",
+	0, 0
+)
+
diff --git a/include/vcli.h b/include/vcli.h
index de6e324..f147ef0 100644
--- a/include/vcli.h
+++ b/include/vcli.h
@@ -36,142 +36,6 @@
  */
 
 /*
- * These macros define the common data for requests in the CLI protocol.
- * The fields are:
- *	const char *	request_name
- *	const char *	request_syntax (for short help)
- *	const char *	request_help (for long help)
- *	unsigned	minimum_arguments
- *	unsigned	maximum_arguments
- */
-
-#define CLI_BAN								\
-	"ban",								\
-	"ban <field> <operator> <arg> [&& <field> <oper> <arg> ...]",	\
-	"\tMark obsolete all objects where all the conditions match.",	\
-	3, UINT_MAX
-
-#define CLI_BAN_LIST							\
-	"ban.list",							\
-	"ban.list",							\
-	"\tList the active bans.",					\
-	0, 0
-
-#define CLI_VCL_LOAD							\
-	"vcl.load",							\
-	"vcl.load <configname> <filename> [auto|cold|warm]",		\
-	"\tCompile and load the VCL file under the name provided.",	\
-	2, 3
-
-#define CLI_VCL_INLINE							\
-	"vcl.inline",							\
-	"vcl.inline <configname> <quoted_VCLstring> [auto|cold|warm]",	\
-	"\tCompile and load the VCL data under the name provided.",	\
-	2, 3
-
-#define CLI_VCL_STATE							\
-	"vcl.state",							\
-	"vcl.state <configname> [auto|cold|warm]",			\
-	"\tForce the state of the named configuration.",		\
-	2, 2
-
-#define CLI_VCL_DISCARD							\
-	"vcl.discard",							\
-	"vcl.discard <configname>",					\
-	"\tUnload the named configuration (when possible).",		\
-	1, 1
-
-#define CLI_VCL_LIST							\
-	"vcl.list",							\
-	"vcl.list",							\
-	"\tList all loaded configuration.",				\
-	0, 0
-
-#define CLI_VCL_SHOW							\
-	"vcl.show",							\
-	"vcl.show [-v] <configname>",					\
-	"\tDisplay the source code for the specified configuration.",	\
-	1, 2
-
-#define CLI_VCL_USE							\
-	"vcl.use",							\
-	"vcl.use <configname>",						\
-	"\tSwitch to the named configuration immediately.",		\
-	1, 1
-
-#define CLI_PARAM_SHOW							\
-	"param.show",							\
-	"param.show [-l] [<param>]",					\
-	"\tShow parameters and their values.",				\
-	0, 2
-
-#define CLI_PARAM_SET							\
-	"param.set",							\
-	"param.set <param> <value>",					\
-	"\tSet parameter value.",					\
-	2,2
-
-#define CLI_SERVER_STOP							\
-	"stop",								\
-	"stop",								\
-	"\tStop the Varnish cache process.",				\
-	0, 0
-
-#define CLI_SERVER_START						\
-	"start",							\
-	"start",							\
-	"\tStart the Varnish cache process.",				\
-	0, 0
-
-#define CLI_PING							\
-	"ping",								\
-	"ping [<timestamp>]",						\
-	"\tKeep connection alive.",					\
-	0, 1
-
-#define CLI_HELP							\
-	"help",								\
-	"help [<command>]",						\
-	"\tShow command/protocol help.",				\
-	0, 1
-
-#define CLI_QUIT							\
-	"quit",								\
-	"quit",								\
-	"\tClose connection.",						\
-	0, 0
-
-#define CLI_SERVER_STATUS						\
-	"status",							\
-	"status",							\
-	"\tCheck status of Varnish cache process.",			\
-	0, 0
-
-#define CLI_BANNER							\
-	"banner",							\
-	"banner",							\
-	"\tPrint welcome banner.",					\
-	0, 0
-
-#define CLI_AUTH							\
-	"auth",								\
-	"auth <response>",						\
-	"\tAuthenticate.",						\
-	1, 1
-
-#define CLI_PANIC_SHOW							\
-	"panic.show",							\
-	"panic.show",							\
-	"\tReturn the last panic, if any.",				\
-	0, 0
-
-#define CLI_PANIC_CLEAR							\
-	"panic.clear",							\
-	"panic.clear [-z]",						\
-	"\tClear the last panic, if any. -z will clear related varnishstat counter(s)",	\
-	0, 1
-
-/*
  * Status/return codes in the CLI protocol
  */
 
diff --git a/include/vcli_priv.h b/include/vcli_priv.h
index 61f5720..c1b44f0 100644
--- a/include/vcli_priv.h
+++ b/include/vcli_priv.h
@@ -37,19 +37,28 @@ struct cli;	/* NB: struct cli is opaque at this level.  */
 
 typedef void cli_func_t(struct cli*, const char * const *av, void *priv);
 
+struct cli_cmd_desc {
+	const char			*request;
+	const char			*syntax;
+	const char			*help;
+	const char			*doc;
+	int				minarg;
+	int				maxarg;
+};
+
+#define CLI_CMD(U,l,s,h,d,m,M) extern const struct cli_cmd_desc CLICMD_##U[1];
+#include "tbl/cli_cmds.h"
+#undef CLI_CMD
+
 struct cli_proto {
 	/* These must match the CLI_* macros in cli.h */
-	const char		*request;
-	const char		*syntax;
-	const char		*help;
-	unsigned		minarg;
-	unsigned		maxarg;
-	char			flags[4];
+	const struct cli_cmd_desc	*desc;
+	char				flags[4];
 
 	/* Dispatch information */
-	cli_func_t		*func;
-	cli_func_t		*jsonfunc;
-	void			*priv;
+	cli_func_t			*func;
+	cli_func_t			*jsonfunc;
+	void				*priv;
 };
 
 /* The implementation must provide these functions */
diff --git a/lib/libvarnish/cli_serve.c b/lib/libvarnish/cli_serve.c
index 6e3e41f..5c3e2f5 100644
--- a/lib/libvarnish/cli_serve.c
+++ b/lib/libvarnish/cli_serve.c
@@ -138,10 +138,10 @@ VCLS_func_help(struct cli *cli, const char * const *av, void *priv)
 		VTAILQ_FOREACH(cfn, &cs->funcs, list) {
 			if (cfn->auth > cli->auth)
 				continue;
-			for (cp = cfn->clp; cp->request != NULL; cp++) {
-				if (!strcmp(cp->request, av[2])) {
+			for (cp = cfn->clp; cp->desc != NULL; cp++) {
+				if (!strcmp(cp->desc->request, av[2])) {
 					VCLI_Out(cli, "%s\n%s\n",
-					    cp->syntax, cp->help);
+					    cp->desc->syntax, cp->desc->help);
 					return;
 				}
 				for (u = 0; u < sizeof cp->flags; u++) {
@@ -159,7 +159,7 @@ VCLS_func_help(struct cli *cli, const char * const *av, void *priv)
 	VTAILQ_FOREACH(cfn, &cs->funcs, list) {
 		if (cfn->auth > cli->auth)
 			continue;
-		for (cp = cfn->clp; cp->request != NULL; cp++) {
+		for (cp = cfn->clp; cp->desc != NULL; cp++) {
 			d = 0;
 			h = 0;
 			i = 0;
@@ -186,8 +186,8 @@ VCLS_func_help(struct cli *cli, const char * const *av, void *priv)
 				continue;
 			if (h && !all)
 				continue;
-			if (cp->syntax != NULL)
-				VCLI_Out(cli, "%s\n", cp->syntax);
+			if (cp->desc->syntax != NULL)
+				VCLI_Out(cli, "%s\n", cp->desc->syntax);
 		}
 	}
 }
@@ -209,7 +209,7 @@ VCLS_func_help_json(struct cli *cli, const char * const *av, void *priv)
 	VTAILQ_FOREACH(cfn, &cs->funcs, list) {
 		if (cfn->auth > cli->auth)
 			continue;
-		for (cp = cfn->clp; cp->request != NULL; cp++) {
+		for (cp = cfn->clp; cp->desc != NULL; cp++) {
 			f_wc = f_i = 0;
 			for (u = 0; u < sizeof cp->flags; u++) {
 				if (cp->flags[u] == '*')
@@ -225,13 +225,13 @@ VCLS_func_help_json(struct cli *cli, const char * const *av, void *priv)
 				continue;
 			VCLI_Out(cli, ",\n  {");
 			VCLI_Out(cli, "\n  \"request\": ");
-			VCLI_JSON_str(cli, cp->request);
+			VCLI_JSON_str(cli, cp->desc->request);
 			VCLI_Out(cli, ",\n  \"syntax\": ");
-			VCLI_JSON_str(cli, cp->syntax);
+			VCLI_JSON_str(cli, cp->desc->syntax);
 			VCLI_Out(cli, ",\n  \"help\": ");
-			VCLI_JSON_str(cli, cp->help);
-			VCLI_Out(cli, ",\n  \"minarg\": %d", cp->minarg);
-			VCLI_Out(cli, ", \"maxarg\": %d", cp->maxarg);
+			VCLI_JSON_str(cli, cp->desc->help);
+			VCLI_Out(cli, ",\n  \"minarg\": %d", cp->desc->minarg);
+			VCLI_Out(cli, ", \"maxarg\": %d", cp->desc->maxarg);
 			VCLI_Out(cli, ", \"flags\": ");
 			VCLI_JSON_str(cli, cp->flags);
 			VCLI_Out(cli, ", \"json\": %s",
@@ -255,13 +255,13 @@ cls_dispatch(struct cli *cli, struct cli_proto *clp, char * const * av,
 	int json = 0;
 
 	AN(av);
-	for (cp = clp; cp->request != NULL; cp++) {
-		if (!strcmp(av[1], cp->request))
+	for (cp = clp; cp->desc != NULL; cp++) {
+		if (!strcmp(av[1], cp->desc->request))
 			break;
-		if (!strcmp("*", cp->request))
+		if (!strcmp("*", cp->desc->request))
 			break;
 	}
-	if (cp->request == NULL)
+	if (cp->desc == NULL)
 		return (0);
 
 	if (ac > 1 && !strcmp(av[2], "-j"))
@@ -278,13 +278,13 @@ cls_dispatch(struct cli *cli, struct cli_proto *clp, char * const * av,
 		return(1);
 	}
 
-	if (ac - 1 < cp->minarg + json) {
+	if (ac - 1 < cp->desc->minarg + json) {
 		VCLI_Out(cli, "Too few parameters\n");
 		VCLI_SetResult(cli, CLIS_TOOFEW);
 		return(1);
 	}
 
-	if (ac - 1> cp->maxarg + json) {
+	if (ac - 1> cp->desc->maxarg + json) {
 		VCLI_Out(cli, "Too many parameters\n");
 		VCLI_SetResult(cli, CLIS_TOOMANY);
 		return(1);
@@ -524,7 +524,6 @@ cls_close_fd(struct VCLS *cs, struct VCLS_fd *cfd)
 	FREE_OBJ(cfd);
 }
 
-
 int
 VCLS_AddFunc(struct VCLS *cs, unsigned auth, struct cli_proto *clp)
 {



From phk at FreeBSD.org  Sat May 21 13:36:06 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Sat, 21 May 2016 15:36:06 +0200
Subject: [master] 9cbd371 Let varnishd produce the CLI reference section.
Message-ID: <E1b474Y-0004Zd-20@project.varnish-software.com>


commit 9cbd37102dabc3799d2a17a8a7bfdd3d081d6a15
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Sat May 21 13:33:37 2016 +0000

    Let varnishd produce the CLI reference section.

diff --git a/doc/sphinx/Makefile.am b/doc/sphinx/Makefile.am
index 0b922e6..6baf438 100644
--- a/doc/sphinx/Makefile.am
+++ b/doc/sphinx/Makefile.am
@@ -122,9 +122,13 @@ dist-hook:
 distclean-local:
 	rm -rf $(BUILDDIR)
 
+include/cli.rst: $(top_builddir)/bin/varnishd/varnishd
+	$(top_builddir)/bin/varnishd/varnishd -x dumprstcli > $@
+BUILT_SOURCES = include/cli.rst
+
 include/params.rst: $(top_builddir)/bin/varnishd/varnishd
 	$(top_builddir)/bin/varnishd/varnishd -x dumprstparam > $@
-BUILT_SOURCES = include/params.rst
+BUILT_SOURCES += include/params.rst
 
 include/counters.rst: $(top_builddir)/bin/varnishstat/vsc2rst
 	$(top_builddir)/bin/varnishstat/vsc2rst > $@
diff --git a/doc/sphinx/reference/varnish-cli.rst b/doc/sphinx/reference/varnish-cli.rst
index 2ae20e5..314e9a0 100644
--- a/doc/sphinx/reference/varnish-cli.rst
+++ b/doc/sphinx/reference/varnish-cli.rst
@@ -78,101 +78,12 @@ be entered with the \\xnn syntax.
 Commands
 --------
 
-help [<command>]
-  Show command/protocol help.
+.. include:: ../include/cli.rst
 
-ping [<timestamp>]
-  Keep connection alive.
-
-auth <response>
-  Authenticate.
-
-quit
-  Close connection.
-
-banner
-  Print welcome banner.
-
-status
-  Check status of Varnish cache process.
-
-start
-  Start the Varnish cache process.
-
-stop
-  Stop the Varnish cache process.
-
-vcl.load <configname> <filename> [auto|cold|warm]
-  Compile and load the VCL file under the name provided.
-
-vcl.inline <configname> <quoted_VCLstring> [auto|cold|warm]
-  Compile and load the VCL data under the name provided.
-  Multi-line VCL can be input using the here document :ref:`ref_syntax`.
-
-vcl.use <configname>
-  Switch to the named configuration immediately.
-
-vcl.discard <configname>
-  Unload the named configuration (when possible).
-
-vcl.list
-  List all loaded configuration.
-
-vcl.show [-v] <configname>
-  Display the source code for the specified configuration.
-
-vcl.state <configname> <state>
-  Force the state of the specified configuration.
-  State is any of auto, warm or cold values.
-
-param.show [-l] [<param>]
-  Show parameters and their values.
-
-param.set <param> <value>
-  Set parameter value.
-
-panic.show
-  Return the last panic, if any.
-
-panic.clear [-z]
-  Clear the last panic, if any. -z will clear related varnishstat counter(s).
-
-storage.list
-  List storage devices.
-
-backend.list [-p] [<backend_expression>]
-  List backends. If -p is specified include probe details.
-
-backend.set_health <backend_expression> <state>
-  Set health status on the backends.
-  State is any of auto, healthy or sick values.
-
-ban <field> <operator> <arg> [&& <field> <oper> <arg> ...]
-  Mark obsolete all objects where all the conditions match.
-
-ban.list
-  List the active bans. The output format is:
-
-  * time the ban was issued
-
-  * reference count
-
-  * ``C`` for completed bans (replaced by a newer ban) or ``-``
-
-  * if ``lurker`` debugging is enabled
-
-    * ``R`` for bans on request properties or ``-``
-
-    * ``O`` for bans on object properties or ``-``
-
-    * pointer to the ban object
-
-  * ban specification
-
-Backend Expression
-------------------
+Backend Pattern
+---------------
 
-A backend expression can be a backend name or a combination of a VCL name
+A backend pattern can be a backend name or a combination of a VCL name
 and backend name in "VCL.backend" format.  If the VCL name is omitted,
 the active VCL is assumed.  Partial matching on the backend name, VCL
 name or both is possible using shell-style wilcards, i.e. asterisk (*),



From phk at FreeBSD.org  Sat May 21 13:36:06 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Sat, 21 May 2016 15:36:06 +0200
Subject: [master] 9e97862 Also exercise the dumprstcli code.
Message-ID: <E1b474Y-0004Zm-3p@project.varnish-software.com>


commit 9e97862c4438b313fd7f13606a4ab7a4178811c1
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Sat May 21 13:35:33 2016 +0000

    Also exercise the dumprstcli code.

diff --git a/bin/varnishtest/tests/a00009.vtc b/bin/varnishtest/tests/a00009.vtc
index b62483d..b751383 100644
--- a/bin/varnishtest/tests/a00009.vtc
+++ b/bin/varnishtest/tests/a00009.vtc
@@ -3,6 +3,7 @@ varnishtest "Code coverage of VCL compiler and RSTdump etc"
 shell "varnishd -b 127.0.0.1:80 -C -n ${tmpdir} > /dev/null 2>&1"
 shell "varnishd -x dumprstparam > /dev/null 2>&1"
 shell "varnishd -x dumprstvsl > /dev/null 2>&1"
+shell "varnishd -x dumprstcli > /dev/null 2>&1"
 err_shell {VCL version declaration missing} "echo 'bad vcl' >${tmpdir}/t.vcl ; varnishd -f ${tmpdir}/t.vcl -n ${tmpdir} 2>&1"
 err_shell {VCL version declaration missing} "echo 'bad vcl' >${tmpdir}/t.vcl ; varnishd -C -f ${tmpdir}/t.vcl -n ${tmpdir} 2>&1"
 err_shell {-spersistent has been deprecated} "varnishd -spersistent 2>&1"



From phk at FreeBSD.org  Sat May 21 17:04:04 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Sat, 21 May 2016 19:04:04 +0200
Subject: [master] 84fc883 Eliminate a NULL pointer issue with empty (apart
 from comments and whitespace) testfiles.
Message-ID: <E1b4AJo-0007m2-LS@project.varnish-software.com>


commit 84fc883b91a25c62a8a855e4d36533de718ef6dc
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Sat May 21 17:01:35 2016 +0000

    Eliminate a NULL pointer issue with empty (apart from comments and
    whitespace) testfiles.
    
    Spotted by:	Coverity

diff --git a/bin/varnishtest/vtc_main.c b/bin/varnishtest/vtc_main.c
index be8fabf..f30acf0 100644
--- a/bin/varnishtest/vtc_main.c
+++ b/bin/varnishtest/vtc_main.c
@@ -472,13 +472,55 @@ ip_magic(void)
  * Main
  */
 
+static int
+read_file(const char *fn, int ntest)
+{
+	struct vtc_tst *tp;
+	char *p;
+
+	p = VFIL_readfile(NULL, fn, NULL);
+	if (p == NULL) {
+		fprintf(stderr, "Cannot stat file \"%s\": %s\n",
+		    fn, strerror(errno));
+		return (2);
+	}
+	for (;p != NULL && *p != '\0'; p++) {
+		if (vct_islws(*p))
+			continue;
+		if (*p != '#')
+			break;
+		p = strchr(p, '\n');
+	}
+
+	if (p == NULL || *p == '\0') {
+		fprintf(stderr, "File \"%s\" has no content.\n", fn);
+		return (2);
+	}
+
+	if (strncmp(p, "varnishtest", 11) || !isspace(p[11])) {
+		fprintf(stderr,
+		    "File \"%s\" doesn't start with 'varnishtest'\n",
+		    fn);
+		return(2);
+	}
+	ALLOC_OBJ(tp, TST_MAGIC);
+	AN(tp);
+	tp->filename = fn;
+	tp->script = p;
+	tp->ntodo = ntest;
+	VTAILQ_INSERT_TAIL(&tst_head, tp, list);
+	return(0);
+}
+
+/**********************************************************************
+ * Main
+ */
+
 int
 main(int argc, char * const *argv)
 {
 	int ch, i;
 	int ntest = 1;			/* Run tests this many times */
-	struct vtc_tst *tp;
-	char *p;
 	uintmax_t bufsiz;
 
 	if (getenv("TMPDIR") != NULL)
@@ -562,40 +604,10 @@ main(int argc, char * const *argv)
 	argv += optind;
 
 	for (;argc > 0; argc--, argv++) {
-		p = VFIL_readfile(NULL, *argv, NULL);
-		if (p == NULL) {
-			fprintf(stderr, "Cannot stat file \"%s\": %s\n",
-			    *argv, strerror(errno));
-			if (vtc_continue)
-				continue;
-			exit(2);
-		}
-		while (1) {
-			if (vct_islws(*p)) {
-				p++;
-				continue;
-			}
-			if (*p == '#') {
-				if ((p = strchr(p, '\n')))
-					p++;
-				continue;
-			}
-			break;
-		}
-
-		if (strncmp(p, "varnishtest", 11) || !isspace(p[11])) {
-			fprintf(stderr, "File \"%s\" doesn't start with 'varnishtest'\n",
-			    *argv);
-			if (vtc_continue)
-				continue;
+		if (!read_file(*argv, ntest))
+			continue;
+		if (!vtc_continue)
 			exit(2);
-		}
-		ALLOC_OBJ(tp, TST_MAGIC);
-		AN(tp);
-		tp->filename = *argv;
-		tp->script = p;
-		tp->ntodo = ntest;
-		VTAILQ_INSERT_TAIL(&tst_head, tp, list);
 	}
 
 	AZ(VSB_finish(params_vsb));



From phk at FreeBSD.org  Sat May 21 20:42:05 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Sat, 21 May 2016 22:42:05 +0200
Subject: [master] f467671 Add a missing NULL terminator
Message-ID: <E1b4Din-0003Cm-IF@project.varnish-software.com>


commit f46767104b335c7dffa9e2e226f044ea2847962e
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Sat May 21 18:30:36 2016 +0000

    Add a missing NULL terminator

diff --git a/bin/varnishd/mgt/mgt_cli.c b/bin/varnishd/mgt/mgt_cli.c
index 293a5e2..40d6c2f 100644
--- a/bin/varnishd/mgt/mgt_cli.c
+++ b/bin/varnishd/mgt/mgt_cli.c
@@ -133,6 +133,7 @@ mcf_panic(struct cli *cli, const char * const *av, void *priv)
 
 static struct cli_proto cli_debug[] = {
 	{ CLICMD_DEBUG_PANIC_MASTER,		"d", mcf_panic },
+	{ NULL }
 };
 
 /*--------------------------------------------------------------------*/



From phk at FreeBSD.org  Sat May 21 20:42:05 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Sat, 21 May 2016 22:42:05 +0200
Subject: [master] a928731 typo
Message-ID: <E1b4Din-0003Cs-Jj@project.varnish-software.com>


commit a92873175a88826638cb6e5e5cb7b7091b0c80d5
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Sat May 21 18:42:13 2016 +0000

    typo

diff --git a/include/tbl/cli_cmds.h b/include/tbl/cli_cmds.h
index 4e9d1ba..7869313 100644
--- a/include/tbl/cli_cmds.h
+++ b/include/tbl/cli_cmds.h
@@ -232,7 +232,7 @@ CLI_CMD(DEBUG_LISTEN_ADDRESS,
 
 CLI_CMD(BACKEND_LIST,
 	"backend.list",
-	"baclend.list [-p] [<backend_pattern>]",
+	"backend.list [-p] [<backend_pattern>]",
 	"List backends.  -p also shows probe status.",
 	"",
 	0, 2



From phk at FreeBSD.org  Sat May 21 20:42:05 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Sat, 21 May 2016 22:42:05 +0200
Subject: [master] 91dcf6a Simplify the cli servicing by reducing a level of
 data structure indirection.
Message-ID: <E1b4Din-0003D1-L6@project.varnish-software.com>


commit 91dcf6a90d30cde718df1f6f1752ac02a7bbdde2
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Sat May 21 18:57:01 2016 +0000

    Simplify the cli servicing by reducing a level of data structure indirection.

diff --git a/bin/varnishd/cache/cache_cli.c b/bin/varnishd/cache/cache_cli.c
index 337c2dc..b6d1e82 100644
--- a/bin/varnishd/cache/cache_cli.c
+++ b/bin/varnishd/cache/cache_cli.c
@@ -67,7 +67,7 @@ CLI_AddFuncs(struct cli_proto *p)
 
 	AZ(add_check);
 	Lck_Lock(&cli_mtx);
-	AZ(VCLS_AddFunc(cls, 0, p));
+	VCLS_AddFunc(cls, 0, p);
 	Lck_Unlock(&cli_mtx);
 }
 
diff --git a/bin/varnishd/mgt/mgt_cli.c b/bin/varnishd/mgt/mgt_cli.c
index 40d6c2f..806b4e9 100644
--- a/bin/varnishd/mgt/mgt_cli.c
+++ b/bin/varnishd/mgt/mgt_cli.c
@@ -356,11 +356,11 @@ mgt_cli_init_cls(void)
 	cls = VCLS_New(mgt_cli_cb_before, mgt_cli_cb_after,
 	    &mgt_param.cli_buffer, &mgt_param.cli_limit);
 	AN(cls);
-	AZ(VCLS_AddFunc(cls, MCF_NOAUTH, cli_auth));
-	AZ(VCLS_AddFunc(cls, MCF_AUTH, cli_proto));
-	AZ(VCLS_AddFunc(cls, MCF_AUTH, cli_debug));
-	AZ(VCLS_AddFunc(cls, MCF_AUTH, cli_stv));
-	AZ(VCLS_AddFunc(cls, MCF_AUTH, cli_askchild));
+	VCLS_AddFunc(cls, MCF_NOAUTH, cli_auth);
+	VCLS_AddFunc(cls, MCF_AUTH, cli_proto);
+	VCLS_AddFunc(cls, MCF_AUTH, cli_debug);
+	VCLS_AddFunc(cls, MCF_AUTH, cli_stv);
+	VCLS_AddFunc(cls, MCF_AUTH, cli_askchild);
 }
 
 /*--------------------------------------------------------------------
diff --git a/include/vcli_priv.h b/include/vcli_priv.h
index c1b44f0..cc7d111 100644
--- a/include/vcli_priv.h
+++ b/include/vcli_priv.h
@@ -38,6 +38,7 @@ struct cli;	/* NB: struct cli is opaque at this level.  */
 typedef void cli_func_t(struct cli*, const char * const *av, void *priv);
 
 struct cli_cmd_desc {
+	/* Must match CLI_CMD macro in include/tbl/cli_cmds.h */
 	const char			*request;
 	const char			*syntax;
 	const char			*help;
@@ -51,7 +52,6 @@ struct cli_cmd_desc {
 #undef CLI_CMD
 
 struct cli_proto {
-	/* These must match the CLI_* macros in cli.h */
 	const struct cli_cmd_desc	*desc;
 	char				flags[4];
 
@@ -59,6 +59,9 @@ struct cli_proto {
 	cli_func_t			*func;
 	cli_func_t			*jsonfunc;
 	void				*priv;
+
+	unsigned			auth;
+	VTAILQ_ENTRY(cli_proto)		list;
 };
 
 /* The implementation must provide these functions */
diff --git a/include/vcli_serve.h b/include/vcli_serve.h
index 0ff65d0..226fad6 100644
--- a/include/vcli_serve.h
+++ b/include/vcli_serve.h
@@ -34,7 +34,7 @@ struct VCLS *VCLS_New(cls_cbc_f *before, cls_cbc_f *after,
     volatile unsigned *maxlen, volatile unsigned *limit);
 struct cli *VCLS_AddFd(struct VCLS *cs, int fdi, int fdo, cls_cb_f *closefunc,
     void *priv);
-int VCLS_AddFunc(struct VCLS *cs, unsigned auth, struct cli_proto *clp);
+void VCLS_AddFunc(struct VCLS *cs, unsigned auth, struct cli_proto *clp);
 int VCLS_Poll(struct VCLS *cs, int timeout);
 int VCLS_PollFd(struct VCLS *cs, int fd, int timeout);
 void VCLS_Destroy(struct VCLS **);
diff --git a/lib/libvarnish/cli_common.c b/lib/libvarnish/cli_common.c
index b735d1f..a2dd4f6 100644
--- a/lib/libvarnish/cli_common.c
+++ b/lib/libvarnish/cli_common.c
@@ -46,6 +46,7 @@
 #include "vdef.h"
 #include "vas.h"
 #include "miniobj.h"
+#include "vqueue.h"
 
 #include "vcli.h"
 #include "vcli_common.h"
diff --git a/lib/libvarnish/cli_serve.c b/lib/libvarnish/cli_serve.c
index 5c3e2f5..2945539 100644
--- a/lib/libvarnish/cli_serve.c
+++ b/lib/libvarnish/cli_serve.c
@@ -43,6 +43,7 @@
 
 #include "vdef.h"
 #include "vas.h"
+#include "vqueue.h"
 #include "miniobj.h"
 
 #include "vav.h"
@@ -51,17 +52,8 @@
 #include "vcli_priv.h"
 #include "vcli_serve.h"
 #include "vlu.h"
-#include "vqueue.h"
 #include "vsb.h"
 
-struct VCLS_func {
-	unsigned			magic;
-#define VCLS_FUNC_MAGIC			0x7d280c9b
-	VTAILQ_ENTRY(VCLS_func)		list;
-	unsigned			auth;
-	struct cli_proto		*clp;
-};
-
 struct VCLS_fd {
 	unsigned			magic;
 #define VCLS_FD_MAGIC			0x010dbd1e
@@ -81,10 +73,11 @@ struct VCLS {
 #define VCLS_MAGIC			0x60f044a3
 	VTAILQ_HEAD(,VCLS_fd)		fds;
 	unsigned			nfd;
-	VTAILQ_HEAD(,VCLS_func)		funcs;
+	VTAILQ_HEAD(,cli_proto)		funcs;
 	cls_cbc_f			*before, *after;
 	volatile unsigned		*maxlen;
 	volatile unsigned		*limit;
+	struct cli_proto		*wildcard;
 };
 
 /*--------------------------------------------------------------------*/
@@ -117,9 +110,8 @@ VCLS_func_ping(struct cli *cli, const char * const *av, void *priv)
 void
 VCLS_func_help(struct cli *cli, const char * const *av, void *priv)
 {
-	struct cli_proto *cp;
-	struct VCLS_func *cfn;
-	unsigned all, debug, u, d, h, i, wc;
+	struct cli_proto *clp;
+	unsigned all, debug, u, d, h, i;
 	struct VCLS *cs;
 
 	(void)priv;
@@ -135,70 +127,59 @@ VCLS_func_help(struct cli *cli, const char * const *av, void *priv)
 		all = 0;
 		debug = 1;
 	} else {
-		VTAILQ_FOREACH(cfn, &cs->funcs, list) {
-			if (cfn->auth > cli->auth)
+		VTAILQ_FOREACH(clp, &cs->funcs, list) {
+			if (clp->auth > cli->auth)
 				continue;
-			for (cp = cfn->clp; cp->desc != NULL; cp++) {
-				if (!strcmp(cp->desc->request, av[2])) {
-					VCLI_Out(cli, "%s\n%s\n",
-					    cp->desc->syntax, cp->desc->help);
-					return;
-				}
-				for (u = 0; u < sizeof cp->flags; u++) {
-					if (cp->flags[u] == '*') {
-						cp->func(cli, av, NULL);
-						return;
-					}
-				}
+			if (!strcmp(clp->desc->request, av[2])) {
+				VCLI_Out(cli, "%s\n%s\n",
+				    clp->desc->syntax, clp->desc->help);
+				return;
 			}
 		}
-		VCLI_Out(cli, "Unknown request.\nType 'help' for more info.\n");
-		VCLI_SetResult(cli, CLIS_UNKNOWN);
+		if (cs->wildcard != NULL) {
+			cs->wildcard->func(cli, av, NULL);
+		} else {
+			VCLI_Out(cli,
+			    "Unknown request.\nType 'help' for more info.\n");
+			VCLI_SetResult(cli, CLIS_UNKNOWN);
+		}
 		return;
 	}
-	VTAILQ_FOREACH(cfn, &cs->funcs, list) {
-		if (cfn->auth > cli->auth)
+	VTAILQ_FOREACH(clp, &cs->funcs, list) {
+		if (clp->auth > cli->auth)
 			continue;
-		for (cp = cfn->clp; cp->desc != NULL; cp++) {
-			d = 0;
-			h = 0;
-			i = 0;
-			wc = 0;
-			for (u = 0; u < sizeof cp->flags; u++) {
-				if (cp->flags[u] == '\0')
-					continue;
-				if (cp->flags[u] == 'd')
-					d = 1;
-				if (cp->flags[u] == 'h')
-					h = 1;
-				if (cp->flags[u] == 'i')
-					i = 1;
-				if (cp->flags[u] == '*')
-					wc = 1;
-			}
-			if (i)
-				continue;
-			if (wc) {
-				cp->func(cli, av, NULL);
-				continue;
-			}
-			if (debug != d)
-				continue;
-			if (h && !all)
+		d = 0;
+		h = 0;
+		i = 0;
+		for (u = 0; u < sizeof clp->flags; u++) {
+			if (clp->flags[u] == '\0')
 				continue;
-			if (cp->desc->syntax != NULL)
-				VCLI_Out(cli, "%s\n", cp->desc->syntax);
+			if (clp->flags[u] == 'd')
+				d = 1;
+			if (clp->flags[u] == 'h')
+				h = 1;
+			if (clp->flags[u] == 'i')
+				i = 1;
 		}
+		if (i)
+			continue;
+		if (debug != d)
+			continue;
+		if (h && !all)
+			continue;
+		if (clp->desc->syntax != NULL)
+			VCLI_Out(cli, "%s\n", clp->desc->syntax);
 	}
+	if (cs->wildcard != NULL)
+		cs->wildcard->func(cli, av, NULL);
 }
 
 void
 VCLS_func_help_json(struct cli *cli, const char * const *av, void *priv)
 {
-	struct cli_proto *cp;
-	struct VCLS_func *cfn;
+	struct cli_proto *clp;
 	struct VCLS *cs;
-	unsigned u, f_wc, f_i;
+	unsigned u, f_i;
 
 	(void)priv;
 	cs = cli->cls;
@@ -206,39 +187,33 @@ VCLS_func_help_json(struct cli *cli, const char * const *av, void *priv)
 
 	if (priv == NULL)
 		VCLI_JSON_ver(cli, 1, av);
-	VTAILQ_FOREACH(cfn, &cs->funcs, list) {
-		if (cfn->auth > cli->auth)
+	VTAILQ_FOREACH(clp, &cs->funcs, list) {
+		if (clp->auth > cli->auth)
 			continue;
-		for (cp = cfn->clp; cp->desc != NULL; cp++) {
-			f_wc = f_i = 0;
-			for (u = 0; u < sizeof cp->flags; u++) {
-				if (cp->flags[u] == '*')
-					f_wc = 1;
-				if (cp->flags[u] == 'i')
-					f_i = 1;
-			}
-			if (f_wc) {
-				cp->func(cli, av, priv);
-				continue;
-			}
-			if (f_i)
-				continue;
-			VCLI_Out(cli, ",\n  {");
-			VCLI_Out(cli, "\n  \"request\": ");
-			VCLI_JSON_str(cli, cp->desc->request);
-			VCLI_Out(cli, ",\n  \"syntax\": ");
-			VCLI_JSON_str(cli, cp->desc->syntax);
-			VCLI_Out(cli, ",\n  \"help\": ");
-			VCLI_JSON_str(cli, cp->desc->help);
-			VCLI_Out(cli, ",\n  \"minarg\": %d", cp->desc->minarg);
-			VCLI_Out(cli, ", \"maxarg\": %d", cp->desc->maxarg);
-			VCLI_Out(cli, ", \"flags\": ");
-			VCLI_JSON_str(cli, cp->flags);
-			VCLI_Out(cli, ", \"json\": %s",
-			    cp->jsonfunc == NULL ? "false" : "true");
-			VCLI_Out(cli, "\n  }");
-		}
+		f_i = 0;
+		for (u = 0; u < sizeof clp->flags; u++)
+			if (clp->flags[u] == 'i')
+				f_i = 1;
+		if (f_i)
+			continue;
+		VCLI_Out(cli, ",\n  {");
+		VCLI_Out(cli, "\n  \"request\": ");
+		VCLI_JSON_str(cli, clp->desc->request);
+		VCLI_Out(cli, ",\n  \"syntax\": ");
+		VCLI_JSON_str(cli, clp->desc->syntax);
+		VCLI_Out(cli, ",\n  \"help\": ");
+		VCLI_JSON_str(cli, clp->desc->help);
+		VCLI_Out(cli, ",\n  \"minarg\": %d", clp->desc->minarg);
+		VCLI_Out(cli, ", \"maxarg\": %d", clp->desc->maxarg);
+		VCLI_Out(cli, ", \"flags\": ");
+		VCLI_JSON_str(cli, clp->flags);
+		VCLI_Out(cli, ", \"json\": %s",
+		    clp->jsonfunc == NULL ? "false" : "true");
+		VCLI_Out(cli, "\n  }");
 	}
+	if (cs->wildcard != NULL)
+		cs->wildcard->func(cli, av, priv);
+
 	if (priv == NULL)
 		VCLI_Out(cli, "\n]\n");
 }
@@ -247,22 +222,13 @@ VCLS_func_help_json(struct cli *cli, const char * const *av, void *priv)
  * Look for a CLI command to execute
  */
 
-static int
-cls_dispatch(struct cli *cli, struct cli_proto *clp, char * const * av,
-    unsigned ac)
+static void
+cls_dispatch(struct cli *cli, const struct cli_proto *cp,
+    char * const * av, unsigned ac)
 {
-	struct cli_proto *cp;
 	int json = 0;
 
 	AN(av);
-	for (cp = clp; cp->desc != NULL; cp++) {
-		if (!strcmp(av[1], cp->desc->request))
-			break;
-		if (!strcmp("*", cp->desc->request))
-			break;
-	}
-	if (cp->desc == NULL)
-		return (0);
 
 	if (ac > 1 && !strcmp(av[2], "-j"))
 		json = 1;
@@ -270,24 +236,24 @@ cls_dispatch(struct cli *cli, struct cli_proto *clp, char * const * av,
 	if (cp->func == NULL && !json) {
 		VCLI_Out(cli, "Unimplemented\n");
 		VCLI_SetResult(cli, CLIS_UNIMPL);
-		return(1);
+		return;
 	}
 	if (cp->jsonfunc == NULL && json) {
 		VCLI_Out(cli, "JSON unimplemented\n");
 		VCLI_SetResult(cli, CLIS_UNIMPL);
-		return(1);
+		return;
 	}
 
 	if (ac - 1 < cp->desc->minarg + json) {
 		VCLI_Out(cli, "Too few parameters\n");
 		VCLI_SetResult(cli, CLIS_TOOFEW);
-		return(1);
+		return;
 	}
 
 	if (ac - 1> cp->desc->maxarg + json) {
 		VCLI_Out(cli, "Too many parameters\n");
 		VCLI_SetResult(cli, CLIS_TOOMANY);
-		return(1);
+		return;
 	}
 
 	cli->result = CLIS_OK;
@@ -296,7 +262,6 @@ cls_dispatch(struct cli *cli, struct cli_proto *clp, char * const * av,
 		cp->jsonfunc(cli, (const char * const *)av, cp->priv);
 	else
 		cp->func(cli, (const char * const *)av, cp->priv);
-	return (1);
 }
 
 /*--------------------------------------------------------------------
@@ -308,7 +273,7 @@ cls_vlu2(void *priv, char * const *av)
 {
 	struct VCLS_fd *cfd;
 	struct VCLS *cs;
-	struct VCLS_func *cfn;
+	struct cli_proto *clp;
 	struct cli *cli;
 	unsigned na;
 	ssize_t len;
@@ -352,12 +317,18 @@ cls_vlu2(void *priv, char * const *av)
 		for (na = 0; av[na + 1] != NULL; na++)
 			continue;
 
-		VTAILQ_FOREACH(cfn, &cs->funcs, list) {
-			if (cfn->auth > cli->auth)
+		VTAILQ_FOREACH(clp, &cs->funcs, list) {
+			if (clp->auth > cli->auth)
 				continue;
-			if (cls_dispatch(cli, cfn->clp, av, na))
+			if (!strcmp(clp->desc->request, av[1])) {
+				cls_dispatch(cli, clp, av, na);
 				break;
+			}
 		}
+		if (clp == NULL &&
+		    cs->wildcard && cs->wildcard->auth <= cli->auth)
+			cls_dispatch(cli, cs->wildcard, av, na);
+
 	} while (0);
 
 	AZ(VSB_finish(cli->sb));
@@ -524,18 +495,33 @@ cls_close_fd(struct VCLS *cs, struct VCLS_fd *cfd)
 	FREE_OBJ(cfd);
 }
 
-int
+void
 VCLS_AddFunc(struct VCLS *cs, unsigned auth, struct cli_proto *clp)
 {
-	struct VCLS_func *cfn;
+	struct cli_proto *clp2;
+	int i;
 
 	CHECK_OBJ_NOTNULL(cs, VCLS_MAGIC);
-	ALLOC_OBJ(cfn, VCLS_FUNC_MAGIC);
-	AN(cfn);
-	cfn->clp = clp;
-	cfn->auth = auth;
-	VTAILQ_INSERT_TAIL(&cs->funcs, cfn, list);
-	return (0);
+	AN(clp);
+
+	for (;clp->desc != NULL; clp++) {
+		clp->auth = auth;
+		if (!strcmp(clp->desc->request, "*")) {
+			cs->wildcard = clp;
+		} else {
+			VTAILQ_FOREACH(clp2, &cs->funcs, list) {
+				i = strcmp(clp->desc->request,
+				    clp2->desc->request);
+				assert(i != 0);
+				if (i < 0)
+					break;
+			}
+			if (clp2 != NULL)
+				VTAILQ_INSERT_BEFORE(clp2, clp, list);
+			else
+				VTAILQ_INSERT_TAIL(&cs->funcs, clp, list);
+		}
+	}
 }
 
 int
@@ -625,7 +611,7 @@ VCLS_Destroy(struct VCLS **csp)
 {
 	struct VCLS *cs;
 	struct VCLS_fd *cfd, *cfd2;
-	struct VCLS_func *cfn;
+	struct cli_proto *clp;
 
 	cs = *csp;
 	*csp = NULL;
@@ -634,9 +620,8 @@ VCLS_Destroy(struct VCLS **csp)
 		cls_close_fd(cs, cfd);
 
 	while (!VTAILQ_EMPTY(&cs->funcs)) {
-		cfn = VTAILQ_FIRST(&cs->funcs);
-		VTAILQ_REMOVE(&cs->funcs, cfn, list);
-		FREE_OBJ(cfn);
+		clp = VTAILQ_FIRST(&cs->funcs);
+		VTAILQ_REMOVE(&cs->funcs, clp, list);
 	}
 	FREE_OBJ(cs);
 }



From phk at FreeBSD.org  Sat May 21 20:42:05 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Sat, 21 May 2016 22:42:05 +0200
Subject: [master] 552292e Hack the help with/without worker process thing up a
 different way, resulting in alphabetically sorted command listing.
Message-ID: <E1b4Din-0003DA-Mj@project.varnish-software.com>


commit 552292e6943e6edcd289b51fca1dce03cec4cf09
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Sat May 21 20:38:46 2016 +0000

    Hack the help with/without worker process thing up a different way,
    resulting in alphabetically sorted command listing.
    
    Not what I set out to do right now, but somebody asked for this
    recently and I happened to spot a way to do it now...

diff --git a/bin/varnishd/cache/cache_acceptor.c b/bin/varnishd/cache/cache_acceptor.c
index 5719720..b45c4f4 100644
--- a/bin/varnishd/cache/cache_acceptor.c
+++ b/bin/varnishd/cache/cache_acceptor.c
@@ -588,7 +588,7 @@ ccf_listen_address(struct cli *cli, const char * const *av, void *priv)
 /*--------------------------------------------------------------------*/
 
 static struct cli_proto vca_cmds[] = {
-	{ CLICMD_SERVER_START,		"i", ccf_start },
+	{ CLICMD_SERVER_START,		"", ccf_start },
 	{ CLICMD_DEBUG_LISTEN_ADDRESS,	"d", ccf_listen_address },
 	{ NULL }
 };
diff --git a/bin/varnishd/cache/cache_cli.c b/bin/varnishd/cache/cache_cli.c
index b6d1e82..db72904 100644
--- a/bin/varnishd/cache/cache_cli.c
+++ b/bin/varnishd/cache/cache_cli.c
@@ -111,7 +111,7 @@ CLI_Run(void)
 
 static struct cli_proto cli_cmds[] = {
 	{ CLICMD_PING,	"i", VCLS_func_ping },
-	{ CLICMD_HELP,	"i", VCLS_func_help, VCLS_func_help_json, cli_cmds },
+	{ CLICMD_HELP,	"i", VCLS_func_help, VCLS_func_help_json },
 	{ NULL }
 };
 
@@ -129,6 +129,7 @@ CLI_Init(void)
 	cls = VCLS_New(cli_cb_before, cli_cb_after,
 	    &cache_param->cli_buffer, &cache_param->cli_limit);
 	AN(cls);
+	VCLS_Clone(cls, mgt_cls);
 
 	CLI_AddFuncs(cli_cmds);
 }
diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index 8b0ac91..a8702e7 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -947,11 +947,11 @@ VCL_##func##_method(struct vcl *vcl, struct worker *wrk,		\
 /*--------------------------------------------------------------------*/
 
 static struct cli_proto vcl_cmds[] = {
-	{ CLICMD_VCL_LOAD,		"i", ccf_config_load },
-	{ CLICMD_VCL_LIST,		"i", ccf_config_list },
-	{ CLICMD_VCL_STATE,		"i", ccf_config_state },
-	{ CLICMD_VCL_DISCARD,		"i", ccf_config_discard },
-	{ CLICMD_VCL_USE,		"i", ccf_config_use },
+	{ CLICMD_VCL_LOAD,		"", ccf_config_load },
+	{ CLICMD_VCL_LIST,		"", ccf_config_list },
+	{ CLICMD_VCL_STATE,		"", ccf_config_state },
+	{ CLICMD_VCL_DISCARD,		"", ccf_config_discard },
+	{ CLICMD_VCL_USE,		"", ccf_config_use },
 	{ CLICMD_VCL_SHOW,		"", ccf_config_show },
 	{ NULL }
 };
diff --git a/bin/varnishd/common/common.h b/bin/varnishd/common/common.h
index 7b6a6b8..afdeafd 100644
--- a/bin/varnishd/common/common.h
+++ b/bin/varnishd/common/common.h
@@ -85,3 +85,6 @@ void VSM_common_delete(struct vsm_sc **sc);
 void VSM_common_copy(struct vsm_sc *to, const struct vsm_sc *from);
 void VSM_common_cleaner(struct vsm_sc *sc, struct VSC_C_main *stats);
 void VSM_common_ageupdate(const struct vsm_sc *sc);
+
+/* mgt_cli.c */
+extern struct VCLS	*mgt_cls;
diff --git a/bin/varnishd/mgt/mgt_cli.c b/bin/varnishd/mgt/mgt_cli.c
index 806b4e9..dcebb2a 100644
--- a/bin/varnishd/mgt/mgt_cli.c
+++ b/bin/varnishd/mgt/mgt_cli.c
@@ -71,7 +71,7 @@ static const struct cli_cmd_desc *cmds[] = {
 static const int ncmds = sizeof cmds / sizeof cmds[0];
 
 static int		cli_i = -1, cli_o = -1;
-static struct VCLS	*cls;
+struct VCLS		*mgt_cls;
 static const char	*secret_file;
 
 #define	MCF_NOAUTH	0	/* NB: zero disables here-documents */
@@ -152,12 +152,6 @@ mcf_askchild(struct cli *cli, const char * const *av, void *priv)
 	 * running.
 	 */
 	if (cli_o <= 0) {
-		if (!strcmp(av[1], "help")) {
-			if (av[2] == NULL || strcmp(av[2], "-j"))
-				VCLI_Out(cli,
-				   "No help from child, (not running).\n");
-			return;
-		}
 		VCLI_SetResult(cli, CLIS_UNKNOWN);
 		VCLI_Out(cli,
 		    "Unknown request in manager process "
@@ -322,8 +316,28 @@ mcf_auth(struct cli *cli, const char *const *av, void *priv)
 	mcf_banner(cli, av, priv);
 }
 
+/*--------------------------------------------------------------------*/
+
+static void __match_proto__(cli_func_t)
+mcf_help(struct cli *cli, const char * const *av, void *priv)
+{
+	if (cli_o <= 0)
+		VCLS_func_help(cli, av, priv);
+	else
+		mcf_askchild(cli, av, priv);
+}
+
+static void __match_proto__(cli_func_t)
+mcf_help_json(struct cli *cli, const char * const *av, void *priv)
+{
+	if (cli_o <= 0)
+		VCLS_func_help_json(cli, av, priv);
+	else
+		mcf_askchild(cli, av, priv);
+}
+
 static struct cli_proto cli_auth[] = {
-	{ CLICMD_HELP,		"", VCLS_func_help, VCLS_func_help_json },
+	{ CLICMD_HELP,		"", mcf_help, mcf_help_json },
 	{ CLICMD_PING,		"", VCLS_func_ping },
 	{ CLICMD_AUTH,		"", mcf_auth },
 	{ CLICMD_QUIT,		"", VCLS_func_close },
@@ -353,14 +367,14 @@ static void
 mgt_cli_init_cls(void)
 {
 
-	cls = VCLS_New(mgt_cli_cb_before, mgt_cli_cb_after,
+	mgt_cls = VCLS_New(mgt_cli_cb_before, mgt_cli_cb_after,
 	    &mgt_param.cli_buffer, &mgt_param.cli_limit);
-	AN(cls);
-	VCLS_AddFunc(cls, MCF_NOAUTH, cli_auth);
-	VCLS_AddFunc(cls, MCF_AUTH, cli_proto);
-	VCLS_AddFunc(cls, MCF_AUTH, cli_debug);
-	VCLS_AddFunc(cls, MCF_AUTH, cli_stv);
-	VCLS_AddFunc(cls, MCF_AUTH, cli_askchild);
+	AN(mgt_cls);
+	VCLS_AddFunc(mgt_cls, MCF_NOAUTH, cli_auth);
+	VCLS_AddFunc(mgt_cls, MCF_AUTH, cli_proto);
+	VCLS_AddFunc(mgt_cls, MCF_AUTH, cli_debug);
+	VCLS_AddFunc(mgt_cls, MCF_AUTH, cli_stv);
+	VCLS_AddFunc(mgt_cls, MCF_AUTH, cli_askchild);
 }
 
 /*--------------------------------------------------------------------
@@ -371,7 +385,7 @@ void
 mgt_cli_close_all(void)
 {
 
-	VCLS_Destroy(&cls);
+	VCLS_Destroy(&mgt_cls);
 }
 
 /*--------------------------------------------------------------------
@@ -385,7 +399,7 @@ mgt_cli_callback2(const struct vev *e, int what)
 
 	(void)e;
 	(void)what;
-	i = VCLS_PollFd(cls, e->fd, 0);
+	i = VCLS_PollFd(mgt_cls, e->fd, 0);
 	return (i);
 }
 
@@ -400,10 +414,10 @@ mgt_cli_setup(int fdi, int fdo, int verbose, const char *ident,
 
 	(void)ident;
 	(void)verbose;
-	if (cls == NULL)
+	if (mgt_cls == NULL)
 		mgt_cli_init_cls();
 
-	cli = VCLS_AddFd(cls, fdi, fdo, closefunc, priv);
+	cli = VCLS_AddFd(mgt_cls, fdi, fdo, closefunc, priv);
 
 	cli->ident = strdup(ident);
 
diff --git a/bin/varnishtest/tests/b00008.vtc b/bin/varnishtest/tests/b00008.vtc
index ac18701..19bc2d9 100644
--- a/bin/varnishtest/tests/b00008.vtc
+++ b/bin/varnishtest/tests/b00008.vtc
@@ -10,7 +10,7 @@ varnish v1 -cliok "help -d"
 
 varnish v1 -cliok "help vcl.load"
 
-varnish v1 -cliok "help ban"
+varnish v1 -clierr 101 "help ban"
 
 varnish v1 -clierr 101 "FOO?"
 
diff --git a/include/vcli_priv.h b/include/vcli_priv.h
index cc7d111..09bd042 100644
--- a/include/vcli_priv.h
+++ b/include/vcli_priv.h
@@ -53,7 +53,7 @@ struct cli_cmd_desc {
 
 struct cli_proto {
 	const struct cli_cmd_desc	*desc;
-	char				flags[4];
+	const char			*flags;
 
 	/* Dispatch information */
 	cli_func_t			*func;
diff --git a/include/vcli_serve.h b/include/vcli_serve.h
index 226fad6..1c5ac88 100644
--- a/include/vcli_serve.h
+++ b/include/vcli_serve.h
@@ -38,6 +38,7 @@ void VCLS_AddFunc(struct VCLS *cs, unsigned auth, struct cli_proto *clp);
 int VCLS_Poll(struct VCLS *cs, int timeout);
 int VCLS_PollFd(struct VCLS *cs, int fd, int timeout);
 void VCLS_Destroy(struct VCLS **);
+void VCLS_Clone(struct VCLS *cs, struct VCLS *cso);
 
 /* From libvarnish/cli.c */
 cli_func_t	VCLS_func_close;
diff --git a/lib/libvarnish/cli_serve.c b/lib/libvarnish/cli_serve.c
index 2945539..4e88bdc 100644
--- a/lib/libvarnish/cli_serve.c
+++ b/lib/libvarnish/cli_serve.c
@@ -82,7 +82,7 @@ struct VCLS {
 
 /*--------------------------------------------------------------------*/
 
-void
+void __match_proto__(cli_func_t)
 VCLS_func_close(struct cli *cli, const char *const *av, void *priv)
 {
 
@@ -94,24 +94,24 @@ VCLS_func_close(struct cli *cli, const char *const *av, void *priv)
 
 /*--------------------------------------------------------------------*/
 
-void
+void __match_proto__(cli_func_t)
 VCLS_func_ping(struct cli *cli, const char * const *av, void *priv)
 {
 	time_t t;
 
-	(void)priv;
 	(void)av;
+	(void)priv;
 	t = time(NULL);
 	VCLI_Out(cli, "PONG %jd 1.0", (intmax_t)t);
 }
 
 /*--------------------------------------------------------------------*/
 
-void
+void __match_proto__(cli_func_t)
 VCLS_func_help(struct cli *cli, const char * const *av, void *priv)
 {
 	struct cli_proto *clp;
-	unsigned all, debug, u, d, h, i;
+	unsigned all, debug, d;
 	struct VCLS *cs;
 
 	(void)priv;
@@ -128,74 +128,44 @@ VCLS_func_help(struct cli *cli, const char * const *av, void *priv)
 		debug = 1;
 	} else {
 		VTAILQ_FOREACH(clp, &cs->funcs, list) {
-			if (clp->auth > cli->auth)
-				continue;
-			if (!strcmp(clp->desc->request, av[2])) {
+			if (clp->auth <= cli->auth &&
+			    !strcmp(clp->desc->request, av[2])) {
 				VCLI_Out(cli, "%s\n%s\n",
 				    clp->desc->syntax, clp->desc->help);
 				return;
 			}
 		}
-		if (cs->wildcard != NULL) {
-			cs->wildcard->func(cli, av, NULL);
-		} else {
-			VCLI_Out(cli,
-			    "Unknown request.\nType 'help' for more info.\n");
-			VCLI_SetResult(cli, CLIS_UNKNOWN);
-		}
+		VCLI_Out(cli, "Unknown request.\nType 'help' for more info.\n");
+		VCLI_SetResult(cli, CLIS_UNKNOWN);
 		return;
 	}
 	VTAILQ_FOREACH(clp, &cs->funcs, list) {
 		if (clp->auth > cli->auth)
 			continue;
-		d = 0;
-		h = 0;
-		i = 0;
-		for (u = 0; u < sizeof clp->flags; u++) {
-			if (clp->flags[u] == '\0')
-				continue;
-			if (clp->flags[u] == 'd')
-				d = 1;
-			if (clp->flags[u] == 'h')
-				h = 1;
-			if (clp->flags[u] == 'i')
-				i = 1;
-		}
-		if (i)
-			continue;
-		if (debug != d)
+		d =  strchr(clp->flags, 'd') != NULL ? 1 : 0;
+		if (d && (!all && !debug))
 			continue;
-		if (h && !all)
+		if (debug && !d)
 			continue;
 		if (clp->desc->syntax != NULL)
 			VCLI_Out(cli, "%s\n", clp->desc->syntax);
 	}
-	if (cs->wildcard != NULL)
-		cs->wildcard->func(cli, av, NULL);
 }
 
-void
+void __match_proto__(cli_func_t)
 VCLS_func_help_json(struct cli *cli, const char * const *av, void *priv)
 {
 	struct cli_proto *clp;
 	struct VCLS *cs;
-	unsigned u, f_i;
 
 	(void)priv;
 	cs = cli->cls;
 	CHECK_OBJ_NOTNULL(cs, VCLS_MAGIC);
 
-	if (priv == NULL)
-		VCLI_JSON_ver(cli, 1, av);
+	VCLI_JSON_ver(cli, 1, av);
 	VTAILQ_FOREACH(clp, &cs->funcs, list) {
 		if (clp->auth > cli->auth)
 			continue;
-		f_i = 0;
-		for (u = 0; u < sizeof clp->flags; u++)
-			if (clp->flags[u] == 'i')
-				f_i = 1;
-		if (f_i)
-			continue;
 		VCLI_Out(cli, ",\n  {");
 		VCLI_Out(cli, "\n  \"request\": ");
 		VCLI_JSON_str(cli, clp->desc->request);
@@ -211,11 +181,7 @@ VCLS_func_help_json(struct cli *cli, const char * const *av, void *priv)
 		    clp->jsonfunc == NULL ? "false" : "true");
 		VCLI_Out(cli, "\n  }");
 	}
-	if (cs->wildcard != NULL)
-		cs->wildcard->func(cli, av, priv);
-
-	if (priv == NULL)
-		VCLI_Out(cli, "\n]\n");
+	VCLI_Out(cli, "\n]\n");
 }
 
 /*--------------------------------------------------------------------
@@ -509,14 +475,17 @@ VCLS_AddFunc(struct VCLS *cs, unsigned auth, struct cli_proto *clp)
 		if (!strcmp(clp->desc->request, "*")) {
 			cs->wildcard = clp;
 		} else {
+			i = 0;
 			VTAILQ_FOREACH(clp2, &cs->funcs, list) {
 				i = strcmp(clp->desc->request,
 				    clp2->desc->request);
-				assert(i != 0);
-				if (i < 0)
+				if (i <= 0)
 					break;
 			}
-			if (clp2 != NULL)
+			if (clp2 != NULL && i == 0) {
+				VTAILQ_INSERT_BEFORE(clp2, clp, list);
+				VTAILQ_REMOVE(&cs->funcs, clp2, list);
+			} else if (clp2 != NULL)
 				VTAILQ_INSERT_BEFORE(clp2, clp, list);
 			else
 				VTAILQ_INSERT_TAIL(&cs->funcs, clp, list);
@@ -524,6 +493,26 @@ VCLS_AddFunc(struct VCLS *cs, unsigned auth, struct cli_proto *clp)
 	}
 }
 
+/*
+ * This function has *very* special semantics, related to the mgt/worker
+ * process Copy-On-Write memory relationship.
+ */
+
+void
+VCLS_Clone(struct VCLS *cs, struct VCLS *cso)
+{
+	struct cli_proto *clp, *clp2;
+
+	CHECK_OBJ_NOTNULL(cs, VCLS_MAGIC);
+	CHECK_OBJ_NOTNULL(cso, VCLS_MAGIC);
+	VTAILQ_FOREACH_SAFE(clp, &cso->funcs, list, clp2) {
+		VTAILQ_REMOVE(&cso->funcs, clp, list);
+		VTAILQ_INSERT_TAIL(&cs->funcs, clp, list);
+		clp->auth = 0;
+		clp->func = NULL;
+	}
+}
+
 int
 VCLS_PollFd(struct VCLS *cs, int fd, int timeout)
 {



From phk at FreeBSD.org  Sat May 21 23:06:05 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Sun, 22 May 2016 01:06:05 +0200
Subject: [master] 1819377 Let mgt_vcl register its own cli commands
Message-ID: <E1b4Fy9-00031s-3X@project.varnish-software.com>


commit 1819377293a4dfa106fa45e3717577e9219c228f
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Sat May 21 22:22:45 2016 +0000

    Let mgt_vcl register its own cli commands

diff --git a/bin/varnishd/mgt/mgt.h b/bin/varnishd/mgt/mgt.h
index 3d03cc3..7152530 100644
--- a/bin/varnishd/mgt/mgt.h
+++ b/bin/varnishd/mgt/mgt.h
@@ -68,6 +68,7 @@ void mgt_cli_master(const char *M_arg);
 void mgt_cli_secret(const char *S_arg);
 void mgt_cli_close_all(void);
 void mgt_DumpRstCli(void);
+void mgt_cli_init_cls(void);
 
 /* mgt_jail.c */
 
diff --git a/bin/varnishd/mgt/mgt_cli.c b/bin/varnishd/mgt/mgt_cli.c
index dcebb2a..26a7a50 100644
--- a/bin/varnishd/mgt/mgt_cli.c
+++ b/bin/varnishd/mgt/mgt_cli.c
@@ -100,18 +100,11 @@ mcf_banner(struct cli *cli, const char *const *av, void *priv)
 
 /*--------------------------------------------------------------------*/
 
-/* XXX: what order should this list be in ? */
 static struct cli_proto cli_proto[] = {
 	{ CLICMD_BANNER,		"", mcf_banner },
 	{ CLICMD_SERVER_STATUS,		"", mcf_server_status },
 	{ CLICMD_SERVER_START,		"", mcf_server_start },
 	{ CLICMD_SERVER_STOP,		"", mcf_server_stop },
-	{ CLICMD_VCL_LOAD,		"", mcf_vcl_load },
-	{ CLICMD_VCL_INLINE,		"", mcf_vcl_inline },
-	{ CLICMD_VCL_USE,		"", mcf_vcl_use },
-	{ CLICMD_VCL_STATE,		"", mcf_vcl_state },
-	{ CLICMD_VCL_DISCARD,		"", mcf_vcl_discard },
-	{ CLICMD_VCL_LIST,		"", mcf_vcl_list },
 	{ CLICMD_PARAM_SHOW,		"", mcf_param_show },
 	{ CLICMD_PARAM_SET,		"", mcf_param_set },
 	{ CLICMD_PANIC_SHOW,		"", mcf_panic_show },
@@ -363,7 +356,7 @@ mgt_cli_cb_after(const struct cli *cli)
 
 /*--------------------------------------------------------------------*/
 
-static void
+void
 mgt_cli_init_cls(void)
 {
 
@@ -414,8 +407,6 @@ mgt_cli_setup(int fdi, int fdo, int verbose, const char *ident,
 
 	(void)ident;
 	(void)verbose;
-	if (mgt_cls == NULL)
-		mgt_cli_init_cls();
 
 	cli = VCLS_AddFd(mgt_cls, fdi, fdo, closefunc, priv);
 
diff --git a/bin/varnishd/mgt/mgt_cli.h b/bin/varnishd/mgt/mgt_cli.h
index 49d849a..8757c2c 100644
--- a/bin/varnishd/mgt/mgt_cli.h
+++ b/bin/varnishd/mgt/mgt_cli.h
@@ -39,13 +39,5 @@ cli_func_t mcf_panic_clear;
 cli_func_t mcf_param_show;
 cli_func_t mcf_param_set;
 
-/* mgt_vcc.c */
-cli_func_t mcf_vcl_load;
-cli_func_t mcf_vcl_inline;
-cli_func_t mcf_vcl_use;
-cli_func_t mcf_vcl_state;
-cli_func_t mcf_vcl_discard;
-cli_func_t mcf_vcl_list;
-
 /* stevedore.c */
 extern struct cli_proto cli_stv[];
diff --git a/bin/varnishd/mgt/mgt_main.c b/bin/varnishd/mgt/mgt_main.c
index 47ae1ea..42a0977 100644
--- a/bin/varnishd/mgt/mgt_main.c
+++ b/bin/varnishd/mgt/mgt_main.c
@@ -691,6 +691,8 @@ main(int argc, char * const *argv)
 		}
 	}
 
+	mgt_cli_init_cls();		// CLI commands can be registered
+
 	if (!jailed)
 		VJ_Init(NULL);
 
diff --git a/bin/varnishd/mgt/mgt_vcl.c b/bin/varnishd/mgt/mgt_vcl.c
index 858c5ec..770c551 100644
--- a/bin/varnishd/mgt/mgt_vcl.c
+++ b/bin/varnishd/mgt/mgt_vcl.c
@@ -42,6 +42,7 @@
 
 #include "vcli.h"
 #include "vcli_priv.h"
+#include "vcli_serve.h"
 #include "vev.h"
 #include "vtim.h"
 
@@ -271,7 +272,7 @@ mgt_push_vcls_and_start(struct cli *cli, unsigned *status, char **p)
 
 /*--------------------------------------------------------------------*/
 
-void
+static void __match_proto__(cli_func_t)
 mcf_vcl_inline(struct cli *cli, const char * const *av, void *priv)
 {
 	struct vclprog *vp;
@@ -288,7 +289,7 @@ mcf_vcl_inline(struct cli *cli, const char * const *av, void *priv)
 	mgt_new_vcl(cli, av[2], av[3], "<vcl.inline>", av[4], 0);
 }
 
-void
+static void __match_proto__(cli_func_t)
 mcf_vcl_load(struct cli *cli, const char * const *av, void *priv)
 {
 	struct vclprog *vp;
@@ -317,7 +318,7 @@ mcf_find_vcl(struct cli *cli, const char *name)
 	return (NULL);
 }
 
-void
+static void __match_proto__(cli_func_t)
 mcf_vcl_state(struct cli *cli, const char * const *av, void *priv)
 {
 	struct vclprog *vp;
@@ -353,7 +354,7 @@ mcf_vcl_state(struct cli *cli, const char * const *av, void *priv)
 	}
 }
 
-void
+static void __match_proto__(cli_func_t)
 mcf_vcl_use(struct cli *cli, const char * const *av, void *priv)
 {
 	unsigned status;
@@ -386,7 +387,7 @@ mcf_vcl_use(struct cli *cli, const char * const *av, void *priv)
 	free(p);
 }
 
-void
+static void __match_proto__(cli_func_t)
 mcf_vcl_discard(struct cli *cli, const char * const *av, void *priv)
 {
 	unsigned status;
@@ -411,7 +412,7 @@ mcf_vcl_discard(struct cli *cli, const char * const *av, void *priv)
 	mgt_vcl_del(vp);
 }
 
-void
+static void __match_proto__(cli_func_t)
 mcf_vcl_list(struct cli *cli, const char * const *av, void *priv)
 {
 	unsigned status;
@@ -455,6 +456,18 @@ mgt_vcl_poker(const struct vev *e, int what)
 
 /*--------------------------------------------------------------------*/
 
+static struct cli_proto cli_vcl[] = {
+	{ CLICMD_VCL_LOAD,		"", mcf_vcl_load },
+	{ CLICMD_VCL_INLINE,		"", mcf_vcl_inline },
+	{ CLICMD_VCL_USE,		"", mcf_vcl_use },
+	{ CLICMD_VCL_STATE,		"", mcf_vcl_state },
+	{ CLICMD_VCL_DISCARD,		"", mcf_vcl_discard },
+	{ CLICMD_VCL_LIST,		"", mcf_vcl_list },
+	{ NULL }
+};
+
+/*--------------------------------------------------------------------*/
+
 static void
 mgt_vcl_atexit(void)
 {
@@ -482,4 +495,6 @@ mgt_vcl_init(void)
 	AZ(vev_add(mgt_evb, e_poker));
 
 	AZ(atexit(mgt_vcl_atexit));
+
+	VCLS_AddFunc(mgt_cls, 0, cli_vcl);
 }



From phk at FreeBSD.org  Sat May 21 23:06:05 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Sun, 22 May 2016 01:06:05 +0200
Subject: [master] d835dfc Make mgt_child register its own CLI commands
Message-ID: <E1b4Fy9-00031y-5S@project.varnish-software.com>


commit d835dfc51b9924a68c6ae984e618ee9c39eb8ffa
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Sat May 21 22:26:36 2016 +0000

    Make mgt_child register its own CLI commands

diff --git a/bin/varnishd/mgt/mgt_child.c b/bin/varnishd/mgt/mgt_child.c
index 3aee527..0c6482f 100644
--- a/bin/varnishd/mgt/mgt_child.c
+++ b/bin/varnishd/mgt/mgt_child.c
@@ -49,6 +49,7 @@
 #include "vbm.h"
 #include "vcli.h"
 #include "vcli_priv.h"
+#include "vcli_serve.h"
 #include "vev.h"
 #include "vlu.h"
 #include "vtim.h"
@@ -130,7 +131,7 @@ mgt_panic_clear(void)
 	VSB_destroy(&child_panic);
 }
 
-void __match_proto__(cli_func_t)
+static void __match_proto__(cli_func_t)
 mcf_panic_show(struct cli *cli, const char * const *av, void *priv)
 {
 	(void)av;
@@ -146,7 +147,7 @@ mcf_panic_show(struct cli *cli, const char * const *av, void *priv)
 	VCLI_Out(cli, "%s\n", VSB_data(child_panic));
 }
 
-void __match_proto__(cli_func_t)
+static void __match_proto__(cli_func_t)
 mcf_panic_clear(struct cli *cli, const char * const *av, void *priv)
 {
 	(void)priv;
@@ -616,7 +617,7 @@ mgt_stop_child(void)
  * CLI commands to start/stop child
  */
 
-void __match_proto__(cli_func_t)
+static void __match_proto__(cli_func_t)
 mcf_server_start(struct cli *cli, const char * const *av, void *priv)
 {
 
@@ -635,7 +636,7 @@ mcf_server_start(struct cli *cli, const char * const *av, void *priv)
 	}
 }
 
-void __match_proto__(cli_func_t)
+static void __match_proto__(cli_func_t)
 mcf_server_stop(struct cli *cli, const char * const *av, void *priv)
 {
 
@@ -651,7 +652,7 @@ mcf_server_stop(struct cli *cli, const char * const *av, void *priv)
 
 /*--------------------------------------------------------------------*/
 
-void
+static void
 mcf_server_status(struct cli *cli, const char * const *av, void *priv)
 {
 	(void)av;
@@ -690,6 +691,17 @@ mgt_uptime(const struct vev *e, int what)
 	return (0);
 }
 
+/*--------------------------------------------------------------------*/
+
+static struct cli_proto cli_child[] = {
+	{ CLICMD_SERVER_STATUS,		"", mcf_server_status },
+	{ CLICMD_SERVER_START,		"", mcf_server_start },
+	{ CLICMD_SERVER_STOP,		"", mcf_server_stop },
+	{ CLICMD_PANIC_SHOW,		"", mcf_panic_show },
+	{ CLICMD_PANIC_CLEAR,		"", mcf_panic_clear },
+	{ NULL }
+};
+
 /*=====================================================================
  * This thread is the master thread in the management process.
  * The relatively simple task is to start and stop the child process
@@ -703,6 +715,8 @@ MGT_Run(void)
 	struct vev *e;
 	int i;
 
+	VCLS_AddFunc(mgt_cls, 0, cli_child);
+
 	mgt_uptime_t0 = VTIM_real();
 	e = vev_new();
 	XXXAN(e);
diff --git a/bin/varnishd/mgt/mgt_cli.c b/bin/varnishd/mgt/mgt_cli.c
index 26a7a50..7e7a0af 100644
--- a/bin/varnishd/mgt/mgt_cli.c
+++ b/bin/varnishd/mgt/mgt_cli.c
@@ -102,13 +102,8 @@ mcf_banner(struct cli *cli, const char *const *av, void *priv)
 
 static struct cli_proto cli_proto[] = {
 	{ CLICMD_BANNER,		"", mcf_banner },
-	{ CLICMD_SERVER_STATUS,		"", mcf_server_status },
-	{ CLICMD_SERVER_START,		"", mcf_server_start },
-	{ CLICMD_SERVER_STOP,		"", mcf_server_stop },
 	{ CLICMD_PARAM_SHOW,		"", mcf_param_show },
 	{ CLICMD_PARAM_SET,		"", mcf_param_set },
-	{ CLICMD_PANIC_SHOW,		"", mcf_panic_show },
-	{ CLICMD_PANIC_CLEAR,		"", mcf_panic_clear },
 	{ NULL }
 };
 
diff --git a/bin/varnishd/mgt/mgt_cli.h b/bin/varnishd/mgt/mgt_cli.h
index 8757c2c..5d57817 100644
--- a/bin/varnishd/mgt/mgt_cli.h
+++ b/bin/varnishd/mgt/mgt_cli.h
@@ -28,13 +28,6 @@
  *
  */
 
-/* mgt_child.c */
-cli_func_t mcf_server_start;
-cli_func_t mcf_server_stop;
-cli_func_t mcf_server_status;
-cli_func_t mcf_panic_show;
-cli_func_t mcf_panic_clear;
-
 /* mgt_param.c */
 cli_func_t mcf_param_show;
 cli_func_t mcf_param_set;



From phk at FreeBSD.org  Sat May 21 23:06:05 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Sun, 22 May 2016 01:06:05 +0200
Subject: [master] 5eba657 Have the stevedore register its own cli commands and
 retire mgt_cli.h
Message-ID: <E1b4Fy9-000327-75@project.varnish-software.com>


commit 5eba65726f1e03d4f536b707200c52b4cf4f1504
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Sat May 21 22:41:38 2016 +0000

    Have the stevedore register its own cli commands and retire mgt_cli.h

diff --git a/bin/varnishd/Makefile.am b/bin/varnishd/Makefile.am
index 792e7fa..1a328fb 100644
--- a/bin/varnishd/Makefile.am
+++ b/bin/varnishd/Makefile.am
@@ -113,7 +113,6 @@ noinst_HEADERS = \
 	hash/hash_slinger.h \
 	http1/cache_http1.h \
 	mgt/mgt.h \
-	mgt/mgt_cli.h \
 	mgt/mgt_param.h \
 	storage/storage.h \
 	storage/storage_simple.h \
diff --git a/bin/varnishd/mgt/mgt.h b/bin/varnishd/mgt/mgt.h
index 7152530..20fd20d 100644
--- a/bin/varnishd/mgt/mgt.h
+++ b/bin/varnishd/mgt/mgt.h
@@ -69,6 +69,8 @@ void mgt_cli_secret(const char *S_arg);
 void mgt_cli_close_all(void);
 void mgt_DumpRstCli(void);
 void mgt_cli_init_cls(void);
+#define MCF_NOAUTH	0	/* NB: zero disables here-documents */
+#define MCF_AUTH	16
 
 /* mgt_jail.c */
 
diff --git a/bin/varnishd/mgt/mgt_child.c b/bin/varnishd/mgt/mgt_child.c
index 0c6482f..d451e97 100644
--- a/bin/varnishd/mgt/mgt_child.c
+++ b/bin/varnishd/mgt/mgt_child.c
@@ -54,8 +54,6 @@
 #include "vlu.h"
 #include "vtim.h"
 
-#include "mgt_cli.h"
-
 pid_t			child_pid = -1;
 
 static struct vbitmap	*fd_map;
@@ -715,7 +713,7 @@ MGT_Run(void)
 	struct vev *e;
 	int i;
 
-	VCLS_AddFunc(mgt_cls, 0, cli_child);
+	VCLS_AddFunc(mgt_cls, MCF_AUTH, cli_child);
 
 	mgt_uptime_t0 = VTIM_real();
 	e = vev_new();
diff --git a/bin/varnishd/mgt/mgt_cli.c b/bin/varnishd/mgt/mgt_cli.c
index 7e7a0af..e6ed6a5 100644
--- a/bin/varnishd/mgt/mgt_cli.c
+++ b/bin/varnishd/mgt/mgt_cli.c
@@ -55,8 +55,6 @@
 #include "vss.h"
 #include "vtcp.h"
 
-#include "mgt_cli.h"
-
 #define CLI_CMD(U,l,s,h,d,m,M) \
 const struct cli_cmd_desc CLICMD_##U[1] = {{ l, s, h, d, m, M }};
 #include "tbl/cli_cmds.h"
@@ -74,9 +72,6 @@ static int		cli_i = -1, cli_o = -1;
 struct VCLS		*mgt_cls;
 static const char	*secret_file;
 
-#define	MCF_NOAUTH	0	/* NB: zero disables here-documents */
-#define MCF_AUTH	16
-
 /*--------------------------------------------------------------------*/
 
 static void
@@ -102,8 +97,6 @@ mcf_banner(struct cli *cli, const char *const *av, void *priv)
 
 static struct cli_proto cli_proto[] = {
 	{ CLICMD_BANNER,		"", mcf_banner },
-	{ CLICMD_PARAM_SHOW,		"", mcf_param_show },
-	{ CLICMD_PARAM_SET,		"", mcf_param_set },
 	{ NULL }
 };
 
@@ -361,7 +354,6 @@ mgt_cli_init_cls(void)
 	VCLS_AddFunc(mgt_cls, MCF_NOAUTH, cli_auth);
 	VCLS_AddFunc(mgt_cls, MCF_AUTH, cli_proto);
 	VCLS_AddFunc(mgt_cls, MCF_AUTH, cli_debug);
-	VCLS_AddFunc(mgt_cls, MCF_AUTH, cli_stv);
 	VCLS_AddFunc(mgt_cls, MCF_AUTH, cli_askchild);
 }
 
diff --git a/bin/varnishd/mgt/mgt_cli.h b/bin/varnishd/mgt/mgt_cli.h
deleted file mode 100644
index 5d57817..0000000
--- a/bin/varnishd/mgt/mgt_cli.h
+++ /dev/null
@@ -1,36 +0,0 @@
-/*-
- * Copyright (c) 2006 Verdens Gang AS
- * Copyright (c) 2006-2011 Varnish Software AS
- * All rights reserved.
- *
- * Author: Poul-Henning Kamp <phk at phk.freebsd.dk>
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
- * 1. Redistributions of source code must retain the above copyright
- *    notice, this list of conditions and the following disclaimer.
- * 2. Redistributions in binary form must reproduce the above copyright
- *    notice, this list of conditions and the following disclaimer in the
- *    documentation and/or other materials provided with the distribution.
- *
- * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
- * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED.  IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
- * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
- * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
- * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
- * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
- * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
- * SUCH DAMAGE.
- *
- */
-
-/* mgt_param.c */
-cli_func_t mcf_param_show;
-cli_func_t mcf_param_set;
-
-/* stevedore.c */
-extern struct cli_proto cli_stv[];
diff --git a/bin/varnishd/mgt/mgt_main.c b/bin/varnishd/mgt/mgt_main.c
index 42a0977..f5b41c9 100644
--- a/bin/varnishd/mgt/mgt_main.c
+++ b/bin/varnishd/mgt/mgt_main.c
@@ -560,6 +560,8 @@ main(int argc, char * const *argv)
 	clilim = 32768;
 	cli[0].limit = &clilim;
 
+	mgt_cli_init_cls();		// CLI commands can be registered
+
 	/* Various initializations */
 	VTAILQ_INIT(&heritage.socks);
 	mgt_evb = vev_new_base();
@@ -691,8 +693,6 @@ main(int argc, char * const *argv)
 		}
 	}
 
-	mgt_cli_init_cls();		// CLI commands can be registered
-
 	if (!jailed)
 		VJ_Init(NULL);
 
diff --git a/bin/varnishd/mgt/mgt_param.c b/bin/varnishd/mgt/mgt_param.c
index 44c64d7..4211920 100644
--- a/bin/varnishd/mgt/mgt_param.c
+++ b/bin/varnishd/mgt/mgt_param.c
@@ -43,8 +43,7 @@
 #include "vcli.h"
 #include "vcli_common.h"
 #include "vcli_priv.h"
-
-#include "mgt_cli.h"
+#include "vcli_serve.h"
 
 struct plist {
 	unsigned			magic;
@@ -230,7 +229,7 @@ mcf_wrap(struct cli *cli, const char *text)
 
 /*--------------------------------------------------------------------*/
 
-void
+static void __match_proto__(cli_func_t)
 mcf_param_show(struct cli *cli, const char * const *av, void *priv)
 {
 	int n;
@@ -381,7 +380,7 @@ MCF_ParamSet(struct cli *cli, const char *param, const char *val)
 
 /*--------------------------------------------------------------------*/
 
-void
+static void __match_proto__(cli_func_t)
 mcf_param_set(struct cli *cli, const char * const *av, void *priv)
 {
 
@@ -444,6 +443,14 @@ mcf_wash_param(struct cli *cli, const struct parspec *pp, const char **val,
 	}
 }
 
+/*--------------------------------------------------------------------*/
+
+static struct cli_proto cli_params[] = {
+	{ CLICMD_PARAM_SHOW,		"", mcf_param_show },
+	{ CLICMD_PARAM_SET,		"", mcf_param_set },
+	{ NULL }
+};
+
 /*--------------------------------------------------------------------
  * Wash the min/max/default values, and leave the default set.
  */
@@ -455,6 +462,8 @@ MCF_InitParams(struct cli *cli)
 	struct parspec *pp;
 	struct vsb *vsb;
 
+	VCLS_AddFunc(mgt_cls, MCF_AUTH, cli_params);
+
 	vsb = VSB_new_auto();
 	AN(vsb);
 	VTAILQ_FOREACH(pl, &phead, list) {
diff --git a/bin/varnishd/mgt/mgt_vcl.c b/bin/varnishd/mgt/mgt_vcl.c
index 770c551..67d8c40 100644
--- a/bin/varnishd/mgt/mgt_vcl.c
+++ b/bin/varnishd/mgt/mgt_vcl.c
@@ -46,8 +46,6 @@
 #include "vev.h"
 #include "vtim.h"
 
-#include "mgt_cli.h"
-
 static const char * const VCL_STATE_COLD = "cold";
 static const char * const VCL_STATE_WARM = "warm";
 static const char * const VCL_STATE_AUTO = "auto";
@@ -496,5 +494,5 @@ mgt_vcl_init(void)
 
 	AZ(atexit(mgt_vcl_atexit));
 
-	VCLS_AddFunc(mgt_cls, 0, cli_vcl);
+	VCLS_AddFunc(mgt_cls, MCF_AUTH, cli_vcl);
 }
diff --git a/bin/varnishd/storage/mgt_stevedore.c b/bin/varnishd/storage/mgt_stevedore.c
index 644b899..a540136 100644
--- a/bin/varnishd/storage/mgt_stevedore.c
+++ b/bin/varnishd/storage/mgt_stevedore.c
@@ -40,7 +40,7 @@
 
 #include "mgt/mgt.h"
 #include "vcli_priv.h"
-#include "mgt/mgt_cli.h"
+#include "vcli_serve.h"
 
 #include "storage/storage.h"
 #include "vav.h"
@@ -52,7 +52,7 @@ struct stevedore *stv_transient;
 
 /*--------------------------------------------------------------------*/
 
-static void
+static void __match_proto__(cli_func_t)
 stv_cli_list(struct cli *cli, const char * const *av, void *priv)
 {
 	struct stevedore *stv;
@@ -69,7 +69,7 @@ stv_cli_list(struct cli *cli, const char * const *av, void *priv)
 
 /*--------------------------------------------------------------------*/
 
-struct cli_proto cli_stv[] = {
+static struct cli_proto cli_stv[] = {
 	{ CLICMD_STORAGE_LIST,		"", stv_cli_list },
 	{ NULL}
 };
@@ -199,8 +199,7 @@ STV_Config_Transient(void)
 
 	ASSERT_MGT();
 
+	VCLS_AddFunc(mgt_cls, MCF_AUTH, cli_stv);
 	if (stv_transient == NULL)
 		STV_Config(TRANSIENT_STORAGE "=malloc");
 }
-
-/*--------------------------------------------------------------------*/



From phk at FreeBSD.org  Sat May 21 23:06:05 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Sun, 22 May 2016 01:06:05 +0200
Subject: [master] 597b3fb Collapse the VCLI .c and .h files,
 leaving only two: The bare protocol and the VCLI core serve code.
Message-ID: <E1b4Fy9-00032G-9K@project.varnish-software.com>


commit 597b3fbc6755bbfc17f2d6dbc0ba8ef93a56ba77
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Sat May 21 23:03:57 2016 +0000

    Collapse the VCLI .c and .h files, leaving only two: The bare protocol
    and the VCLI core serve code.

diff --git a/include/vcli_common.h b/include/vcli_common.h
deleted file mode 100644
index 8d0a623..0000000
--- a/include/vcli_common.h
+++ /dev/null
@@ -1,46 +0,0 @@
-/*-
- * Copyright (c) 2006 Verdens Gang AS
- * Copyright (c) 2006-2011 Varnish Software AS
- * All rights reserved.
- *
- * Author: Poul-Henning Kamp <phk at phk.freebsd.dk>
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
- * 1. Redistributions of source code must retain the above copyright
- *    notice, this list of conditions and the following disclaimer.
- * 2. Redistributions in binary form must reproduce the above copyright
- *    notice, this list of conditions and the following disclaimer in the
- *    documentation and/or other materials provided with the distribution.
- *
- * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
- * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED.  IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
- * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
- * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
- * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
- * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
- * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
- * SUCH DAMAGE.
- *
- */
-
-struct vlu;
-struct VCLS;
-
-struct cli {
-	unsigned		magic;
-#define CLI_MAGIC		0x4038d570
-	struct vsb		*sb;
-	enum VCLI_status_e	result;
-	char			*cmd;
-	unsigned		auth;
-	char			challenge[34];
-	char			*ident;
-	struct vlu		*vlu;
-	struct VCLS		*cls;
-	volatile unsigned	*limit;
-};
diff --git a/include/vcli_serve.h b/include/vcli_serve.h
deleted file mode 100644
index 1c5ac88..0000000
--- a/include/vcli_serve.h
+++ /dev/null
@@ -1,47 +0,0 @@
-/*-
- * Copyright (c) 2010-2011 Varnish Software AS
- * All rights reserved.
- *
- * Author: Poul-Henning Kamp <phk at phk.freebsd.dk>
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
- * 1. Redistributions of source code must retain the above copyright
- *    notice, this list of conditions and the following disclaimer.
- * 2. Redistributions in binary form must reproduce the above copyright
- *    notice, this list of conditions and the following disclaimer in the
- *    documentation and/or other materials provided with the distribution.
- *
- * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
- * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED.  IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
- * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
- * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
- * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
- * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
- * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
- * SUCH DAMAGE.
- *
- */
-
-struct VCLS;
-typedef void cls_cb_f(void *priv);
-typedef void cls_cbc_f(const struct cli*);
-struct VCLS *VCLS_New(cls_cbc_f *before, cls_cbc_f *after,
-    volatile unsigned *maxlen, volatile unsigned *limit);
-struct cli *VCLS_AddFd(struct VCLS *cs, int fdi, int fdo, cls_cb_f *closefunc,
-    void *priv);
-void VCLS_AddFunc(struct VCLS *cs, unsigned auth, struct cli_proto *clp);
-int VCLS_Poll(struct VCLS *cs, int timeout);
-int VCLS_PollFd(struct VCLS *cs, int fd, int timeout);
-void VCLS_Destroy(struct VCLS **);
-void VCLS_Clone(struct VCLS *cs, struct VCLS *cso);
-
-/* From libvarnish/cli.c */
-cli_func_t	VCLS_func_close;
-cli_func_t	VCLS_func_help;
-cli_func_t	VCLS_func_help_json;
-cli_func_t	VCLS_func_ping;
diff --git a/lib/libvarnish/cli_auth.c b/lib/libvarnish/cli_auth.c
deleted file mode 100644
index d7b7bf2..0000000
--- a/lib/libvarnish/cli_auth.c
+++ /dev/null
@@ -1,64 +0,0 @@
-/*-
- * Copyright (c) 2010-2011 Varnish Software AS
- * All rights reserved.
- *
- * Author: Poul-Henning Kamp <phk at phk.freebsd.dk>
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
- * 1. Redistributions of source code must retain the above copyright
- *    notice, this list of conditions and the following disclaimer.
- * 2. Redistributions in binary form must reproduce the above copyright
- *    notice, this list of conditions and the following disclaimer in the
- *    documentation and/or other materials provided with the distribution.
- *
- * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
- * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED.  IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
- * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
- * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
- * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
- * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
- * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
- * SUCH DAMAGE.
- */
-
-#include "config.h"
-
-#include <sys/types.h>
-
-#include <stdint.h>
-#include <stdio.h>
-#include <unistd.h>
-
-#include "vas.h"
-#include "vcli.h"
-#include "vsha256.h"
-
-void
-VCLI_AuthResponse(int S_fd, const char *challenge,
-    char response[CLI_AUTH_RESPONSE_LEN + 1])
-{
-	SHA256_CTX ctx;
-	uint8_t buf[SHA256_LEN];
-	int i;
-
-	assert(CLI_AUTH_RESPONSE_LEN == (SHA256_LEN * 2));
-
-	SHA256_Init(&ctx);
-	SHA256_Update(&ctx, challenge, 32);
-	SHA256_Update(&ctx, "\n", 1);
-	do {
-		i = read(S_fd, buf, 1);
-		if (i == 1)
-			SHA256_Update(&ctx, buf, i);
-	} while (i > 0);
-	SHA256_Update(&ctx, challenge, 32);
-	SHA256_Update(&ctx, "\n", 1);
-	SHA256_Final(buf, &ctx);
-	for(i = 0; i < SHA256_LEN; i++)
-		sprintf(response + 2 * i, "%02x", buf[i]);
-}
diff --git a/lib/libvarnish/cli_common.c b/lib/libvarnish/cli_common.c
deleted file mode 100644
index a2dd4f6..0000000
--- a/lib/libvarnish/cli_common.c
+++ /dev/null
@@ -1,258 +0,0 @@
-/*-
- * Copyright (c) 2006 Verdens Gang AS
- * Copyright (c) 2006-2011 Varnish Software AS
- * All rights reserved.
- *
- * Author: Poul-Henning Kamp <phk at phk.freebsd.dk>
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
- * 1. Redistributions of source code must retain the above copyright
- *    notice, this list of conditions and the following disclaimer.
- * 2. Redistributions in binary form must reproduce the above copyright
- *    notice, this list of conditions and the following disclaimer in the
- *    documentation and/or other materials provided with the distribution.
- *
- * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
- * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED.  IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
- * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
- * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
- * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
- * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
- * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
- * SUCH DAMAGE.
- */
-
-#include "config.h"
-
-#include <sys/types.h>
-#include <sys/uio.h>
-
-#include <errno.h>
-#include <poll.h>
-#include <signal.h>
-#include <stdarg.h>
-#include <stdint.h>
-#include <stdio.h>
-#include <stdlib.h>
-#include <string.h>
-#include <time.h>
-#include <unistd.h>
-
-#include "vdef.h"
-#include "vas.h"
-#include "miniobj.h"
-#include "vqueue.h"
-
-#include "vcli.h"
-#include "vcli_common.h"
-#include "vcli_priv.h"
-#include "vsb.h"
-
-/*lint -e{818} cli could be const */
-void
-VCLI_Out(struct cli *cli, const char *fmt, ...)
-{
-	va_list ap;
-
-	va_start(ap, fmt);
-	if (cli != NULL) {
-		CHECK_OBJ_NOTNULL(cli, CLI_MAGIC);
-		if (VSB_len(cli->sb) < *cli->limit)
-			(void)VSB_vprintf(cli->sb, fmt, ap);
-		else if (cli->result == CLIS_OK)
-			cli->result = CLIS_TRUNCATED;
-	} else {
-		(void)vfprintf(stdout, fmt, ap);
-	}
-	va_end(ap);
-}
-
-/*lint -e{818} cli could be const */
-int
-VCLI_Overflow(struct cli *cli)
-{
-	CHECK_OBJ_NOTNULL(cli, CLI_MAGIC);
-	if (cli->result == CLIS_TRUNCATED ||
-	    VSB_len(cli->sb) >= *cli->limit)
-		return (1);
-	return (0);
-}
-
-/*lint -e{818} cli could be const */
-void
-VCLI_JSON_str(struct cli *cli, const char *s)
-{
-
-	CHECK_OBJ_NOTNULL(cli, CLI_MAGIC);
-	VSB_quote(cli->sb, s, -1, VSB_QUOTE_JSON);
-}
-
-/*lint -e{818} cli could be const */
-void
-VCLI_JSON_ver(struct cli *cli, unsigned ver, const char * const * av)
-{
-	int i;
-
-	CHECK_OBJ_NOTNULL(cli, CLI_MAGIC);
-	VCLI_Out(cli, "[ %u, [", ver);
-	for (i = 1; av[i] != NULL; i++) {
-		VCLI_JSON_str(cli, av[i]);
-		if (av[i + 1] != NULL)
-			VCLI_Out(cli, ", ");
-	}
-	VCLI_Out(cli, "]");
-}
-
-/*lint -e{818} cli could be const */
-void
-VCLI_Quote(struct cli *cli, const char *s)
-{
-
-	CHECK_OBJ_NOTNULL(cli, CLI_MAGIC);
-	VSB_quote(cli->sb, s, -1, 0);
-}
-
-void
-VCLI_SetResult(struct cli *cli, unsigned res)
-{
-
-	if (cli != NULL) {
-		CHECK_OBJ_NOTNULL(cli, CLI_MAGIC);
-		if (cli->result != CLIS_TRUNCATED || res != CLIS_OK)
-			cli->result = res;	/*lint !e64 type mismatch */
-	} else {
-		printf("CLI result = %u\n", res);
-	}
-}
-
-int
-VCLI_WriteResult(int fd, unsigned status, const char *result)
-{
-	int i, l;
-	struct iovec iov[3];
-	char nl[2] = "\n";
-	size_t len;
-	char res[CLI_LINE0_LEN + 2];	/*
-					 * NUL + one more so we can catch
-					 * any misformats by snprintf
-					 */
-
-	assert(status >= 100);
-	assert(status <= 999);		/*lint !e650 const out of range */
-
-	len = strlen(result);
-
-	i = snprintf(res, sizeof res,
-	    "%-3d %-8zd\n", status, len);
-	assert(i == CLI_LINE0_LEN);
-	assert(strtoul(res + 3, NULL, 10) == len);
-
-	iov[0].iov_base = res;
-	iov[0].iov_len = CLI_LINE0_LEN;
-
-	iov[1].iov_base = (void*)(uintptr_t)result;	/* TRUST ME */
-	iov[1].iov_len = len;
-
-	iov[2].iov_base = nl;
-	iov[2].iov_len = 1;
-
-	for (l = i = 0; i < 3; i++)
-		l += iov[i].iov_len;
-	i = writev(fd, iov, 3);
-	return (i != l);
-}
-
-static int
-read_tmo(int fd, char *ptr, unsigned len, double tmo)
-{
-	int i, j, to;
-	struct pollfd pfd;
-
-	if (tmo > 0)
-		to = (int)(tmo * 1e3);
-	else
-		to = -1;
-	pfd.fd = fd;
-	pfd.events = POLLIN;
-	for (j = 0; len > 0; ) {
-		i = poll(&pfd, 1, to);
-		if (i == 0) {
-			errno = ETIMEDOUT;
-			return (-1);
-		}
-		i = read(fd, ptr, len);
-		if (i < 0)
-			return (i);
-		if (i == 0)
-			break;
-		len -= i;
-		ptr += i;
-		j += i;
-	}
-	return (j);
-}
-
-int
-VCLI_ReadResult(int fd, unsigned *status, char **ptr, double tmo)
-{
-	char res[CLI_LINE0_LEN];	/* For NUL */
-	int i, j;
-	unsigned u, v, s;
-	char *p = NULL;
-	const char *err = "CLI communication error (hdr)";
-
-	if (status == NULL)
-		status = &s;
-	if (ptr != NULL)
-		*ptr = NULL;
-	do {
-		i = read_tmo(fd, res, CLI_LINE0_LEN, tmo);
-		if (i != CLI_LINE0_LEN)
-			break;
-
-		if (res[3] != ' ')
-			break;
-
-		if (res[CLI_LINE0_LEN - 1] != '\n')
-			break;
-
-		res[CLI_LINE0_LEN - 1] = '\0';
-		j = sscanf(res, "%u %u\n", &u, &v);
-		if (j != 2)
-			break;
-
-		err = "CLI communication error (body)";
-
-		*status = u;
-		p = malloc(v + 1L);
-		if (p == NULL)
-			break;
-
-		i = read_tmo(fd, p, v + 1, tmo);
-		if (i < 0)
-			break;
-		if (i != v + 1)
-			break;
-		if (p[v] != '\n')
-			break;
-
-		p[v] = '\0';
-		if (ptr == NULL)
-			free(p);
-		else
-			*ptr = p;
-		return (0);
-	} while(0);
-
-	if (p != NULL)
-		free(p);
-	*status = CLIS_COMMS;
-	if (ptr != NULL)
-		*ptr = strdup(err);
-	return (*status);
-}
diff --git a/lib/libvarnish/cli_serve.c b/lib/libvarnish/cli_serve.c
deleted file mode 100644
index 4e88bdc..0000000
--- a/lib/libvarnish/cli_serve.c
+++ /dev/null
@@ -1,616 +0,0 @@
-/*-
- * Copyright (c) 2006 Verdens Gang AS
- * Copyright (c) 2006-2011 Varnish Software AS
- * All rights reserved.
- *
- * Author: Poul-Henning Kamp <phk at phk.freebsd.dk>
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
- * 1. Redistributions of source code must retain the above copyright
- *    notice, this list of conditions and the following disclaimer.
- * 2. Redistributions in binary form must reproduce the above copyright
- *    notice, this list of conditions and the following disclaimer in the
- *    documentation and/or other materials provided with the distribution.
- *
- * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
- * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED.  IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
- * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
- * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
- * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
- * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
- * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
- * SUCH DAMAGE.
- *
- * Stuff for handling the CLI protocol
- */
-
-#include "config.h"
-
-#include <ctype.h>
-#include <errno.h>
-#include <poll.h>
-#include <stdint.h>
-#include <stdio.h>
-#include <stdlib.h>
-#include <string.h>
-#include <time.h>
-#include <unistd.h>
-
-#include "vdef.h"
-#include "vas.h"
-#include "vqueue.h"
-#include "miniobj.h"
-
-#include "vav.h"
-#include "vcli.h"
-#include "vcli_common.h"
-#include "vcli_priv.h"
-#include "vcli_serve.h"
-#include "vlu.h"
-#include "vsb.h"
-
-struct VCLS_fd {
-	unsigned			magic;
-#define VCLS_FD_MAGIC			0x010dbd1e
-	VTAILQ_ENTRY(VCLS_fd)		list;
-	int				fdi, fdo;
-	struct VCLS			*cls;
-	struct cli			*cli, clis;
-	cls_cb_f			*closefunc;
-	void				*priv;
-	struct vsb			*last_arg;
-	int				last_idx;
-	char				**argv;
-};
-
-struct VCLS {
-	unsigned			magic;
-#define VCLS_MAGIC			0x60f044a3
-	VTAILQ_HEAD(,VCLS_fd)		fds;
-	unsigned			nfd;
-	VTAILQ_HEAD(,cli_proto)		funcs;
-	cls_cbc_f			*before, *after;
-	volatile unsigned		*maxlen;
-	volatile unsigned		*limit;
-	struct cli_proto		*wildcard;
-};
-
-/*--------------------------------------------------------------------*/
-
-void __match_proto__(cli_func_t)
-VCLS_func_close(struct cli *cli, const char *const *av, void *priv)
-{
-
-	(void)av;
-	(void)priv;
-	VCLI_Out(cli, "Closing CLI connection");
-	VCLI_SetResult(cli, CLIS_CLOSE);
-}
-
-/*--------------------------------------------------------------------*/
-
-void __match_proto__(cli_func_t)
-VCLS_func_ping(struct cli *cli, const char * const *av, void *priv)
-{
-	time_t t;
-
-	(void)av;
-	(void)priv;
-	t = time(NULL);
-	VCLI_Out(cli, "PONG %jd 1.0", (intmax_t)t);
-}
-
-/*--------------------------------------------------------------------*/
-
-void __match_proto__(cli_func_t)
-VCLS_func_help(struct cli *cli, const char * const *av, void *priv)
-{
-	struct cli_proto *clp;
-	unsigned all, debug, d;
-	struct VCLS *cs;
-
-	(void)priv;
-	cs = cli->cls;
-	CHECK_OBJ_NOTNULL(cs, VCLS_MAGIC);
-
-	if (av[2] == NULL) {
-		all = debug = 0;
-	} else if (!strcmp(av[2], "-a")) {
-		all = 1;
-		debug = 0;
-	} else if (!strcmp(av[2], "-d")) {
-		all = 0;
-		debug = 1;
-	} else {
-		VTAILQ_FOREACH(clp, &cs->funcs, list) {
-			if (clp->auth <= cli->auth &&
-			    !strcmp(clp->desc->request, av[2])) {
-				VCLI_Out(cli, "%s\n%s\n",
-				    clp->desc->syntax, clp->desc->help);
-				return;
-			}
-		}
-		VCLI_Out(cli, "Unknown request.\nType 'help' for more info.\n");
-		VCLI_SetResult(cli, CLIS_UNKNOWN);
-		return;
-	}
-	VTAILQ_FOREACH(clp, &cs->funcs, list) {
-		if (clp->auth > cli->auth)
-			continue;
-		d =  strchr(clp->flags, 'd') != NULL ? 1 : 0;
-		if (d && (!all && !debug))
-			continue;
-		if (debug && !d)
-			continue;
-		if (clp->desc->syntax != NULL)
-			VCLI_Out(cli, "%s\n", clp->desc->syntax);
-	}
-}
-
-void __match_proto__(cli_func_t)
-VCLS_func_help_json(struct cli *cli, const char * const *av, void *priv)
-{
-	struct cli_proto *clp;
-	struct VCLS *cs;
-
-	(void)priv;
-	cs = cli->cls;
-	CHECK_OBJ_NOTNULL(cs, VCLS_MAGIC);
-
-	VCLI_JSON_ver(cli, 1, av);
-	VTAILQ_FOREACH(clp, &cs->funcs, list) {
-		if (clp->auth > cli->auth)
-			continue;
-		VCLI_Out(cli, ",\n  {");
-		VCLI_Out(cli, "\n  \"request\": ");
-		VCLI_JSON_str(cli, clp->desc->request);
-		VCLI_Out(cli, ",\n  \"syntax\": ");
-		VCLI_JSON_str(cli, clp->desc->syntax);
-		VCLI_Out(cli, ",\n  \"help\": ");
-		VCLI_JSON_str(cli, clp->desc->help);
-		VCLI_Out(cli, ",\n  \"minarg\": %d", clp->desc->minarg);
-		VCLI_Out(cli, ", \"maxarg\": %d", clp->desc->maxarg);
-		VCLI_Out(cli, ", \"flags\": ");
-		VCLI_JSON_str(cli, clp->flags);
-		VCLI_Out(cli, ", \"json\": %s",
-		    clp->jsonfunc == NULL ? "false" : "true");
-		VCLI_Out(cli, "\n  }");
-	}
-	VCLI_Out(cli, "\n]\n");
-}
-
-/*--------------------------------------------------------------------
- * Look for a CLI command to execute
- */
-
-static void
-cls_dispatch(struct cli *cli, const struct cli_proto *cp,
-    char * const * av, unsigned ac)
-{
-	int json = 0;
-
-	AN(av);
-
-	if (ac > 1 && !strcmp(av[2], "-j"))
-		json = 1;
-
-	if (cp->func == NULL && !json) {
-		VCLI_Out(cli, "Unimplemented\n");
-		VCLI_SetResult(cli, CLIS_UNIMPL);
-		return;
-	}
-	if (cp->jsonfunc == NULL && json) {
-		VCLI_Out(cli, "JSON unimplemented\n");
-		VCLI_SetResult(cli, CLIS_UNIMPL);
-		return;
-	}
-
-	if (ac - 1 < cp->desc->minarg + json) {
-		VCLI_Out(cli, "Too few parameters\n");
-		VCLI_SetResult(cli, CLIS_TOOFEW);
-		return;
-	}
-
-	if (ac - 1> cp->desc->maxarg + json) {
-		VCLI_Out(cli, "Too many parameters\n");
-		VCLI_SetResult(cli, CLIS_TOOMANY);
-		return;
-	}
-
-	cli->result = CLIS_OK;
-	VSB_clear(cli->sb);
-	if (json)
-		cp->jsonfunc(cli, (const char * const *)av, cp->priv);
-	else
-		cp->func(cli, (const char * const *)av, cp->priv);
-}
-
-/*--------------------------------------------------------------------
- * We have collected a full cli line, parse it and execute, if possible.
- */
-
-static int
-cls_vlu2(void *priv, char * const *av)
-{
-	struct VCLS_fd *cfd;
-	struct VCLS *cs;
-	struct cli_proto *clp;
-	struct cli *cli;
-	unsigned na;
-	ssize_t len;
-	char *s;
-	unsigned lim;
-	const char *trunc = "!\n[response was truncated]\n";
-
-	CAST_OBJ_NOTNULL(cfd, priv, VCLS_FD_MAGIC);
-	cs = cfd->cls;
-	CHECK_OBJ_NOTNULL(cs, VCLS_MAGIC);
-
-	cli = cfd->cli;
-	CHECK_OBJ_NOTNULL(cli, CLI_MAGIC);
-	AN(cli->cmd);
-
-	cli->cls = cs;
-
-	cli->result = CLIS_UNKNOWN;
-	VSB_clear(cli->sb);
-	VCLI_Out(cli, "Unknown request.\nType 'help' for more info.\n");
-
-	if (cs->before != NULL)
-		cs->before(cli);
-
-	do {
-		if (av[0] != NULL) {
-			VCLI_Out(cli, "Syntax Error: %s\n", av[0]);
-			VCLI_SetResult(cli, CLIS_SYNTAX);
-			break;
-		}
-
-		if (isupper(av[1][0])) {
-			VCLI_Out(cli, "all commands are in lower-case.\n");
-			VCLI_SetResult(cli, CLIS_UNKNOWN);
-			break;
-		}
-
-		if (!islower(av[1][0]))
-			break;
-
-		for (na = 0; av[na + 1] != NULL; na++)
-			continue;
-
-		VTAILQ_FOREACH(clp, &cs->funcs, list) {
-			if (clp->auth > cli->auth)
-				continue;
-			if (!strcmp(clp->desc->request, av[1])) {
-				cls_dispatch(cli, clp, av, na);
-				break;
-			}
-		}
-		if (clp == NULL &&
-		    cs->wildcard && cs->wildcard->auth <= cli->auth)
-			cls_dispatch(cli, cs->wildcard, av, na);
-
-	} while (0);
-
-	AZ(VSB_finish(cli->sb));
-
-	if (cs->after != NULL)
-		cs->after(cli);
-
-	cli->cls = NULL;
-
-	s = VSB_data(cli->sb);
-	len = VSB_len(cli->sb);
-	lim = *cs->limit;
-	if (len > lim) {
-		if (cli->result == CLIS_OK)
-			cli->result = CLIS_TRUNCATED;
-		strcpy(s + (lim - strlen(trunc)), trunc);
-		assert(strlen(s) <= lim);
-	}
-	if (VCLI_WriteResult(cfd->fdo, cli->result, s) ||
-	    cli->result == CLIS_CLOSE)
-		return (1);
-
-	return (0);
-}
-
-static int
-cls_vlu(void *priv, const char *p)
-{
-	struct VCLS_fd *cfd;
-	struct cli *cli;
-	int i;
-	char **av;
-
-	CAST_OBJ_NOTNULL(cfd, priv, VCLS_FD_MAGIC);
-	AN(p);
-
-	cli = cfd->cli;
-	CHECK_OBJ_NOTNULL(cli, CLI_MAGIC);
-
-	if (cfd->argv == NULL) {
-		/*
-		 * Lines with only whitespace are simply ignored, in order
-		 * to not complicate CLI-client side scripts and TELNET users
-		 */
-		for (; isspace(*p); p++)
-			continue;
-		if (*p == '\0')
-			return (0);
-		REPLACE(cli->cmd, p);
-
-		av = VAV_Parse(p, NULL, 0);
-		AN(av);
-		if (av[0] != NULL) {
-			i = cls_vlu2(priv, av);
-			VAV_Free(av);
-			free(cli->cmd);
-			cli->cmd = NULL;
-			return (i);
-		}
-		for (i = 1; av[i] != NULL; i++)
-			continue;
-		if (i < 3 || cli->auth == 0 || strcmp(av[i - 2], "<<")) {
-			i = cls_vlu2(priv, av);
-			VAV_Free(av);
-			free(cli->cmd);
-			cli->cmd = NULL;
-			return (i);
-		}
-		cfd->argv = av;
-		cfd->last_idx = i - 2;
-		cfd->last_arg = VSB_new_auto();
-		AN(cfd->last_arg);
-		return (0);
-	} else {
-		AN(cfd->argv[cfd->last_idx]);
-		AZ(strcmp(cfd->argv[cfd->last_idx], "<<"));
-		AN(cfd->argv[cfd->last_idx + 1]);
-		if (strcmp(p, cfd->argv[cfd->last_idx + 1])) {
-			VSB_cat(cfd->last_arg, p);
-			VSB_cat(cfd->last_arg, "\n");
-			return (0);
-		}
-		AZ(VSB_finish(cfd->last_arg));
-		free(cfd->argv[cfd->last_idx]);
-		cfd->argv[cfd->last_idx] = NULL;
-		free(cfd->argv[cfd->last_idx + 1]);
-		cfd->argv[cfd->last_idx + 1] = NULL;
-		cfd->argv[cfd->last_idx] = VSB_data(cfd->last_arg);
-		i = cls_vlu2(priv, cfd->argv);
-		cfd->argv[cfd->last_idx] = NULL;
-		VAV_Free(cfd->argv);
-		cfd->argv = NULL;
-		free(cli->cmd);
-		cli->cmd = NULL;
-		VSB_destroy(&cfd->last_arg);
-		cfd->last_idx = 0;
-		return (i);
-	}
-}
-
-struct VCLS *
-VCLS_New(cls_cbc_f *before, cls_cbc_f *after, volatile unsigned *maxlen,
-    volatile unsigned *limit)
-{
-	struct VCLS *cs;
-
-	ALLOC_OBJ(cs, VCLS_MAGIC);
-	AN(cs);
-	VTAILQ_INIT(&cs->fds);
-	VTAILQ_INIT(&cs->funcs);
-	cs->before = before;
-	cs->after = after;
-	cs->maxlen = maxlen;
-	cs->limit = limit;
-	return (cs);
-}
-
-struct cli *
-VCLS_AddFd(struct VCLS *cs, int fdi, int fdo, cls_cb_f *closefunc, void *priv)
-{
-	struct VCLS_fd *cfd;
-
-	CHECK_OBJ_NOTNULL(cs, VCLS_MAGIC);
-	assert(fdi >= 0);
-	assert(fdo >= 0);
-	ALLOC_OBJ(cfd, VCLS_FD_MAGIC);
-	AN(cfd);
-	cfd->cls = cs;
-	cfd->fdi = fdi;
-	cfd->fdo = fdo;
-	cfd->cli = &cfd->clis;
-	cfd->cli->magic = CLI_MAGIC;
-	cfd->cli->vlu = VLU_New(cfd, cls_vlu, *cs->maxlen);
-	cfd->cli->sb = VSB_new_auto();
-	cfd->cli->limit = cs->limit;
-	cfd->closefunc = closefunc;
-	cfd->priv = priv;
-	AN(cfd->cli->sb);
-	VTAILQ_INSERT_TAIL(&cs->fds, cfd, list);
-	cs->nfd++;
-	return (cfd->cli);
-}
-
-static void
-cls_close_fd(struct VCLS *cs, struct VCLS_fd *cfd)
-{
-
-	CHECK_OBJ_NOTNULL(cs, VCLS_MAGIC);
-	CHECK_OBJ_NOTNULL(cfd, VCLS_FD_MAGIC);
-
-	VTAILQ_REMOVE(&cs->fds, cfd, list);
-	cs->nfd--;
-	VLU_Destroy(cfd->cli->vlu);
-	VSB_destroy(&cfd->cli->sb);
-	if (cfd->closefunc == NULL) {
-		(void)close(cfd->fdi);
-		if (cfd->fdo != cfd->fdi)
-			(void)close(cfd->fdo);
-	} else {
-		cfd->closefunc(cfd->priv);
-	}
-	if (cfd->cli->ident != NULL)
-		free(cfd->cli->ident);
-	FREE_OBJ(cfd);
-}
-
-void
-VCLS_AddFunc(struct VCLS *cs, unsigned auth, struct cli_proto *clp)
-{
-	struct cli_proto *clp2;
-	int i;
-
-	CHECK_OBJ_NOTNULL(cs, VCLS_MAGIC);
-	AN(clp);
-
-	for (;clp->desc != NULL; clp++) {
-		clp->auth = auth;
-		if (!strcmp(clp->desc->request, "*")) {
-			cs->wildcard = clp;
-		} else {
-			i = 0;
-			VTAILQ_FOREACH(clp2, &cs->funcs, list) {
-				i = strcmp(clp->desc->request,
-				    clp2->desc->request);
-				if (i <= 0)
-					break;
-			}
-			if (clp2 != NULL && i == 0) {
-				VTAILQ_INSERT_BEFORE(clp2, clp, list);
-				VTAILQ_REMOVE(&cs->funcs, clp2, list);
-			} else if (clp2 != NULL)
-				VTAILQ_INSERT_BEFORE(clp2, clp, list);
-			else
-				VTAILQ_INSERT_TAIL(&cs->funcs, clp, list);
-		}
-	}
-}
-
-/*
- * This function has *very* special semantics, related to the mgt/worker
- * process Copy-On-Write memory relationship.
- */
-
-void
-VCLS_Clone(struct VCLS *cs, struct VCLS *cso)
-{
-	struct cli_proto *clp, *clp2;
-
-	CHECK_OBJ_NOTNULL(cs, VCLS_MAGIC);
-	CHECK_OBJ_NOTNULL(cso, VCLS_MAGIC);
-	VTAILQ_FOREACH_SAFE(clp, &cso->funcs, list, clp2) {
-		VTAILQ_REMOVE(&cso->funcs, clp, list);
-		VTAILQ_INSERT_TAIL(&cs->funcs, clp, list);
-		clp->auth = 0;
-		clp->func = NULL;
-	}
-}
-
-int
-VCLS_PollFd(struct VCLS *cs, int fd, int timeout)
-{
-	struct VCLS_fd *cfd;
-	struct pollfd pfd[1];
-	int i, j, k;
-
-	CHECK_OBJ_NOTNULL(cs, VCLS_MAGIC);
-	if (cs->nfd == 0) {
-		errno = 0;
-		return (-1);
-	}
-	assert(cs->nfd > 0);
-
-	i = 0;
-	VTAILQ_FOREACH(cfd, &cs->fds, list) {
-		if (cfd->fdi != fd)
-			continue;
-		pfd[i].fd = cfd->fdi;
-		pfd[i].events = POLLIN;
-		pfd[i].revents = 0;
-		i++;
-		break;
-	}
-	assert(i == 1);
-	CHECK_OBJ_NOTNULL(cfd, VCLS_FD_MAGIC);
-
-	j = poll(pfd, 1, timeout);
-	if (j <= 0)
-		return (j);
-	if (pfd[0].revents & POLLHUP)
-		k = 1;
-	else
-		k = VLU_Fd(cfd->fdi, cfd->cli->vlu);
-	if (k)
-		cls_close_fd(cs, cfd);
-	return (k);
-}
-
-int
-VCLS_Poll(struct VCLS *cs, int timeout)
-{
-	struct VCLS_fd *cfd, *cfd2;
-	int i, j, k;
-
-	CHECK_OBJ_NOTNULL(cs, VCLS_MAGIC);
-	if (cs->nfd == 0) {
-		errno = 0;
-		return (-1);
-	}
-	assert(cs->nfd > 0);
-	{
-		struct pollfd pfd[cs->nfd];
-
-		i = 0;
-		VTAILQ_FOREACH(cfd, &cs->fds, list) {
-			pfd[i].fd = cfd->fdi;
-			pfd[i].events = POLLIN;
-			pfd[i].revents = 0;
-			i++;
-		}
-		assert(i == cs->nfd);
-
-		j = poll(pfd, cs->nfd, timeout);
-		if (j <= 0)
-			return (j);
-		i = 0;
-		VTAILQ_FOREACH_SAFE(cfd, &cs->fds, list, cfd2) {
-			assert(pfd[i].fd == cfd->fdi);
-			if (pfd[i].revents & POLLHUP)
-				k = 1;
-			else
-				k = VLU_Fd(cfd->fdi, cfd->cli->vlu);
-			if (k)
-				cls_close_fd(cs, cfd);
-			i++;
-		}
-		assert(i == j);
-	}
-	return (j);
-}
-
-void
-VCLS_Destroy(struct VCLS **csp)
-{
-	struct VCLS *cs;
-	struct VCLS_fd *cfd, *cfd2;
-	struct cli_proto *clp;
-
-	cs = *csp;
-	*csp = NULL;
-	CHECK_OBJ_NOTNULL(cs, VCLS_MAGIC);
-	VTAILQ_FOREACH_SAFE(cfd, &cs->fds, list, cfd2)
-		cls_close_fd(cs, cfd);
-
-	while (!VTAILQ_EMPTY(&cs->funcs)) {
-		clp = VTAILQ_FIRST(&cs->funcs);
-		VTAILQ_REMOVE(&cs->funcs, clp, list);
-	}
-	FREE_OBJ(cs);
-}
diff --git a/lib/libvarnish/vcli_proto.c b/lib/libvarnish/vcli_proto.c
new file mode 100644
index 0000000..d7b7bf2
--- /dev/null
+++ b/lib/libvarnish/vcli_proto.c
@@ -0,0 +1,64 @@
+/*-
+ * Copyright (c) 2010-2011 Varnish Software AS
+ * All rights reserved.
+ *
+ * Author: Poul-Henning Kamp <phk at phk.freebsd.dk>
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED.  IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+#include "config.h"
+
+#include <sys/types.h>
+
+#include <stdint.h>
+#include <stdio.h>
+#include <unistd.h>
+
+#include "vas.h"
+#include "vcli.h"
+#include "vsha256.h"
+
+void
+VCLI_AuthResponse(int S_fd, const char *challenge,
+    char response[CLI_AUTH_RESPONSE_LEN + 1])
+{
+	SHA256_CTX ctx;
+	uint8_t buf[SHA256_LEN];
+	int i;
+
+	assert(CLI_AUTH_RESPONSE_LEN == (SHA256_LEN * 2));
+
+	SHA256_Init(&ctx);
+	SHA256_Update(&ctx, challenge, 32);
+	SHA256_Update(&ctx, "\n", 1);
+	do {
+		i = read(S_fd, buf, 1);
+		if (i == 1)
+			SHA256_Update(&ctx, buf, i);
+	} while (i > 0);
+	SHA256_Update(&ctx, challenge, 32);
+	SHA256_Update(&ctx, "\n", 1);
+	SHA256_Final(buf, &ctx);
+	for(i = 0; i < SHA256_LEN; i++)
+		sprintf(response + 2 * i, "%02x", buf[i]);
+}
diff --git a/lib/libvarnish/vcli_serve.c b/lib/libvarnish/vcli_serve.c
new file mode 100644
index 0000000..4e88bdc
--- /dev/null
+++ b/lib/libvarnish/vcli_serve.c
@@ -0,0 +1,616 @@
+/*-
+ * Copyright (c) 2006 Verdens Gang AS
+ * Copyright (c) 2006-2011 Varnish Software AS
+ * All rights reserved.
+ *
+ * Author: Poul-Henning Kamp <phk at phk.freebsd.dk>
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED.  IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ *
+ * Stuff for handling the CLI protocol
+ */
+
+#include "config.h"
+
+#include <ctype.h>
+#include <errno.h>
+#include <poll.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <time.h>
+#include <unistd.h>
+
+#include "vdef.h"
+#include "vas.h"
+#include "vqueue.h"
+#include "miniobj.h"
+
+#include "vav.h"
+#include "vcli.h"
+#include "vcli_common.h"
+#include "vcli_priv.h"
+#include "vcli_serve.h"
+#include "vlu.h"
+#include "vsb.h"
+
+struct VCLS_fd {
+	unsigned			magic;
+#define VCLS_FD_MAGIC			0x010dbd1e
+	VTAILQ_ENTRY(VCLS_fd)		list;
+	int				fdi, fdo;
+	struct VCLS			*cls;
+	struct cli			*cli, clis;
+	cls_cb_f			*closefunc;
+	void				*priv;
+	struct vsb			*last_arg;
+	int				last_idx;
+	char				**argv;
+};
+
+struct VCLS {
+	unsigned			magic;
+#define VCLS_MAGIC			0x60f044a3
+	VTAILQ_HEAD(,VCLS_fd)		fds;
+	unsigned			nfd;
+	VTAILQ_HEAD(,cli_proto)		funcs;
+	cls_cbc_f			*before, *after;
+	volatile unsigned		*maxlen;
+	volatile unsigned		*limit;
+	struct cli_proto		*wildcard;
+};
+
+/*--------------------------------------------------------------------*/
+
+void __match_proto__(cli_func_t)
+VCLS_func_close(struct cli *cli, const char *const *av, void *priv)
+{
+
+	(void)av;
+	(void)priv;
+	VCLI_Out(cli, "Closing CLI connection");
+	VCLI_SetResult(cli, CLIS_CLOSE);
+}
+
+/*--------------------------------------------------------------------*/
+
+void __match_proto__(cli_func_t)
+VCLS_func_ping(struct cli *cli, const char * const *av, void *priv)
+{
+	time_t t;
+
+	(void)av;
+	(void)priv;
+	t = time(NULL);
+	VCLI_Out(cli, "PONG %jd 1.0", (intmax_t)t);
+}
+
+/*--------------------------------------------------------------------*/
+
+void __match_proto__(cli_func_t)
+VCLS_func_help(struct cli *cli, const char * const *av, void *priv)
+{
+	struct cli_proto *clp;
+	unsigned all, debug, d;
+	struct VCLS *cs;
+
+	(void)priv;
+	cs = cli->cls;
+	CHECK_OBJ_NOTNULL(cs, VCLS_MAGIC);
+
+	if (av[2] == NULL) {
+		all = debug = 0;
+	} else if (!strcmp(av[2], "-a")) {
+		all = 1;
+		debug = 0;
+	} else if (!strcmp(av[2], "-d")) {
+		all = 0;
+		debug = 1;
+	} else {
+		VTAILQ_FOREACH(clp, &cs->funcs, list) {
+			if (clp->auth <= cli->auth &&
+			    !strcmp(clp->desc->request, av[2])) {
+				VCLI_Out(cli, "%s\n%s\n",
+				    clp->desc->syntax, clp->desc->help);
+				return;
+			}
+		}
+		VCLI_Out(cli, "Unknown request.\nType 'help' for more info.\n");
+		VCLI_SetResult(cli, CLIS_UNKNOWN);
+		return;
+	}
+	VTAILQ_FOREACH(clp, &cs->funcs, list) {
+		if (clp->auth > cli->auth)
+			continue;
+		d =  strchr(clp->flags, 'd') != NULL ? 1 : 0;
+		if (d && (!all && !debug))
+			continue;
+		if (debug && !d)
+			continue;
+		if (clp->desc->syntax != NULL)
+			VCLI_Out(cli, "%s\n", clp->desc->syntax);
+	}
+}
+
+void __match_proto__(cli_func_t)
+VCLS_func_help_json(struct cli *cli, const char * const *av, void *priv)
+{
+	struct cli_proto *clp;
+	struct VCLS *cs;
+
+	(void)priv;
+	cs = cli->cls;
+	CHECK_OBJ_NOTNULL(cs, VCLS_MAGIC);
+
+	VCLI_JSON_ver(cli, 1, av);
+	VTAILQ_FOREACH(clp, &cs->funcs, list) {
+		if (clp->auth > cli->auth)
+			continue;
+		VCLI_Out(cli, ",\n  {");
+		VCLI_Out(cli, "\n  \"request\": ");
+		VCLI_JSON_str(cli, clp->desc->request);
+		VCLI_Out(cli, ",\n  \"syntax\": ");
+		VCLI_JSON_str(cli, clp->desc->syntax);
+		VCLI_Out(cli, ",\n  \"help\": ");
+		VCLI_JSON_str(cli, clp->desc->help);
+		VCLI_Out(cli, ",\n  \"minarg\": %d", clp->desc->minarg);
+		VCLI_Out(cli, ", \"maxarg\": %d", clp->desc->maxarg);
+		VCLI_Out(cli, ", \"flags\": ");
+		VCLI_JSON_str(cli, clp->flags);
+		VCLI_Out(cli, ", \"json\": %s",
+		    clp->jsonfunc == NULL ? "false" : "true");
+		VCLI_Out(cli, "\n  }");
+	}
+	VCLI_Out(cli, "\n]\n");
+}
+
+/*--------------------------------------------------------------------
+ * Look for a CLI command to execute
+ */
+
+static void
+cls_dispatch(struct cli *cli, const struct cli_proto *cp,
+    char * const * av, unsigned ac)
+{
+	int json = 0;
+
+	AN(av);
+
+	if (ac > 1 && !strcmp(av[2], "-j"))
+		json = 1;
+
+	if (cp->func == NULL && !json) {
+		VCLI_Out(cli, "Unimplemented\n");
+		VCLI_SetResult(cli, CLIS_UNIMPL);
+		return;
+	}
+	if (cp->jsonfunc == NULL && json) {
+		VCLI_Out(cli, "JSON unimplemented\n");
+		VCLI_SetResult(cli, CLIS_UNIMPL);
+		return;
+	}
+
+	if (ac - 1 < cp->desc->minarg + json) {
+		VCLI_Out(cli, "Too few parameters\n");
+		VCLI_SetResult(cli, CLIS_TOOFEW);
+		return;
+	}
+
+	if (ac - 1> cp->desc->maxarg + json) {
+		VCLI_Out(cli, "Too many parameters\n");
+		VCLI_SetResult(cli, CLIS_TOOMANY);
+		return;
+	}
+
+	cli->result = CLIS_OK;
+	VSB_clear(cli->sb);
+	if (json)
+		cp->jsonfunc(cli, (const char * const *)av, cp->priv);
+	else
+		cp->func(cli, (const char * const *)av, cp->priv);
+}
+
+/*--------------------------------------------------------------------
+ * We have collected a full cli line, parse it and execute, if possible.
+ */
+
+static int
+cls_vlu2(void *priv, char * const *av)
+{
+	struct VCLS_fd *cfd;
+	struct VCLS *cs;
+	struct cli_proto *clp;
+	struct cli *cli;
+	unsigned na;
+	ssize_t len;
+	char *s;
+	unsigned lim;
+	const char *trunc = "!\n[response was truncated]\n";
+
+	CAST_OBJ_NOTNULL(cfd, priv, VCLS_FD_MAGIC);
+	cs = cfd->cls;
+	CHECK_OBJ_NOTNULL(cs, VCLS_MAGIC);
+
+	cli = cfd->cli;
+	CHECK_OBJ_NOTNULL(cli, CLI_MAGIC);
+	AN(cli->cmd);
+
+	cli->cls = cs;
+
+	cli->result = CLIS_UNKNOWN;
+	VSB_clear(cli->sb);
+	VCLI_Out(cli, "Unknown request.\nType 'help' for more info.\n");
+
+	if (cs->before != NULL)
+		cs->before(cli);
+
+	do {
+		if (av[0] != NULL) {
+			VCLI_Out(cli, "Syntax Error: %s\n", av[0]);
+			VCLI_SetResult(cli, CLIS_SYNTAX);
+			break;
+		}
+
+		if (isupper(av[1][0])) {
+			VCLI_Out(cli, "all commands are in lower-case.\n");
+			VCLI_SetResult(cli, CLIS_UNKNOWN);
+			break;
+		}
+
+		if (!islower(av[1][0]))
+			break;
+
+		for (na = 0; av[na + 1] != NULL; na++)
+			continue;
+
+		VTAILQ_FOREACH(clp, &cs->funcs, list) {
+			if (clp->auth > cli->auth)
+				continue;
+			if (!strcmp(clp->desc->request, av[1])) {
+				cls_dispatch(cli, clp, av, na);
+				break;
+			}
+		}
+		if (clp == NULL &&
+		    cs->wildcard && cs->wildcard->auth <= cli->auth)
+			cls_dispatch(cli, cs->wildcard, av, na);
+
+	} while (0);
+
+	AZ(VSB_finish(cli->sb));
+
+	if (cs->after != NULL)
+		cs->after(cli);
+
+	cli->cls = NULL;
+
+	s = VSB_data(cli->sb);
+	len = VSB_len(cli->sb);
+	lim = *cs->limit;
+	if (len > lim) {
+		if (cli->result == CLIS_OK)
+			cli->result = CLIS_TRUNCATED;
+		strcpy(s + (lim - strlen(trunc)), trunc);
+		assert(strlen(s) <= lim);
+	}
+	if (VCLI_WriteResult(cfd->fdo, cli->result, s) ||
+	    cli->result == CLIS_CLOSE)
+		return (1);
+
+	return (0);
+}
+
+static int
+cls_vlu(void *priv, const char *p)
+{
+	struct VCLS_fd *cfd;
+	struct cli *cli;
+	int i;
+	char **av;
+
+	CAST_OBJ_NOTNULL(cfd, priv, VCLS_FD_MAGIC);
+	AN(p);
+
+	cli = cfd->cli;
+	CHECK_OBJ_NOTNULL(cli, CLI_MAGIC);
+
+	if (cfd->argv == NULL) {
+		/*
+		 * Lines with only whitespace are simply ignored, in order
+		 * to not complicate CLI-client side scripts and TELNET users
+		 */
+		for (; isspace(*p); p++)
+			continue;
+		if (*p == '\0')
+			return (0);
+		REPLACE(cli->cmd, p);
+
+		av = VAV_Parse(p, NULL, 0);
+		AN(av);
+		if (av[0] != NULL) {
+			i = cls_vlu2(priv, av);
+			VAV_Free(av);
+			free(cli->cmd);
+			cli->cmd = NULL;
+			return (i);
+		}
+		for (i = 1; av[i] != NULL; i++)
+			continue;
+		if (i < 3 || cli->auth == 0 || strcmp(av[i - 2], "<<")) {
+			i = cls_vlu2(priv, av);
+			VAV_Free(av);
+			free(cli->cmd);
+			cli->cmd = NULL;
+			return (i);
+		}
+		cfd->argv = av;
+		cfd->last_idx = i - 2;
+		cfd->last_arg = VSB_new_auto();
+		AN(cfd->last_arg);
+		return (0);
+	} else {
+		AN(cfd->argv[cfd->last_idx]);
+		AZ(strcmp(cfd->argv[cfd->last_idx], "<<"));
+		AN(cfd->argv[cfd->last_idx + 1]);
+		if (strcmp(p, cfd->argv[cfd->last_idx + 1])) {
+			VSB_cat(cfd->last_arg, p);
+			VSB_cat(cfd->last_arg, "\n");
+			return (0);
+		}
+		AZ(VSB_finish(cfd->last_arg));
+		free(cfd->argv[cfd->last_idx]);
+		cfd->argv[cfd->last_idx] = NULL;
+		free(cfd->argv[cfd->last_idx + 1]);
+		cfd->argv[cfd->last_idx + 1] = NULL;
+		cfd->argv[cfd->last_idx] = VSB_data(cfd->last_arg);
+		i = cls_vlu2(priv, cfd->argv);
+		cfd->argv[cfd->last_idx] = NULL;
+		VAV_Free(cfd->argv);
+		cfd->argv = NULL;
+		free(cli->cmd);
+		cli->cmd = NULL;
+		VSB_destroy(&cfd->last_arg);
+		cfd->last_idx = 0;
+		return (i);
+	}
+}
+
+struct VCLS *
+VCLS_New(cls_cbc_f *before, cls_cbc_f *after, volatile unsigned *maxlen,
+    volatile unsigned *limit)
+{
+	struct VCLS *cs;
+
+	ALLOC_OBJ(cs, VCLS_MAGIC);
+	AN(cs);
+	VTAILQ_INIT(&cs->fds);
+	VTAILQ_INIT(&cs->funcs);
+	cs->before = before;
+	cs->after = after;
+	cs->maxlen = maxlen;
+	cs->limit = limit;
+	return (cs);
+}
+
+struct cli *
+VCLS_AddFd(struct VCLS *cs, int fdi, int fdo, cls_cb_f *closefunc, void *priv)
+{
+	struct VCLS_fd *cfd;
+
+	CHECK_OBJ_NOTNULL(cs, VCLS_MAGIC);
+	assert(fdi >= 0);
+	assert(fdo >= 0);
+	ALLOC_OBJ(cfd, VCLS_FD_MAGIC);
+	AN(cfd);
+	cfd->cls = cs;
+	cfd->fdi = fdi;
+	cfd->fdo = fdo;
+	cfd->cli = &cfd->clis;
+	cfd->cli->magic = CLI_MAGIC;
+	cfd->cli->vlu = VLU_New(cfd, cls_vlu, *cs->maxlen);
+	cfd->cli->sb = VSB_new_auto();
+	cfd->cli->limit = cs->limit;
+	cfd->closefunc = closefunc;
+	cfd->priv = priv;
+	AN(cfd->cli->sb);
+	VTAILQ_INSERT_TAIL(&cs->fds, cfd, list);
+	cs->nfd++;
+	return (cfd->cli);
+}
+
+static void
+cls_close_fd(struct VCLS *cs, struct VCLS_fd *cfd)
+{
+
+	CHECK_OBJ_NOTNULL(cs, VCLS_MAGIC);
+	CHECK_OBJ_NOTNULL(cfd, VCLS_FD_MAGIC);
+
+	VTAILQ_REMOVE(&cs->fds, cfd, list);
+	cs->nfd--;
+	VLU_Destroy(cfd->cli->vlu);
+	VSB_destroy(&cfd->cli->sb);
+	if (cfd->closefunc == NULL) {
+		(void)close(cfd->fdi);
+		if (cfd->fdo != cfd->fdi)
+			(void)close(cfd->fdo);
+	} else {
+		cfd->closefunc(cfd->priv);
+	}
+	if (cfd->cli->ident != NULL)
+		free(cfd->cli->ident);
+	FREE_OBJ(cfd);
+}
+
+void
+VCLS_AddFunc(struct VCLS *cs, unsigned auth, struct cli_proto *clp)
+{
+	struct cli_proto *clp2;
+	int i;
+
+	CHECK_OBJ_NOTNULL(cs, VCLS_MAGIC);
+	AN(clp);
+
+	for (;clp->desc != NULL; clp++) {
+		clp->auth = auth;
+		if (!strcmp(clp->desc->request, "*")) {
+			cs->wildcard = clp;
+		} else {
+			i = 0;
+			VTAILQ_FOREACH(clp2, &cs->funcs, list) {
+				i = strcmp(clp->desc->request,
+				    clp2->desc->request);
+				if (i <= 0)
+					break;
+			}
+			if (clp2 != NULL && i == 0) {
+				VTAILQ_INSERT_BEFORE(clp2, clp, list);
+				VTAILQ_REMOVE(&cs->funcs, clp2, list);
+			} else if (clp2 != NULL)
+				VTAILQ_INSERT_BEFORE(clp2, clp, list);
+			else
+				VTAILQ_INSERT_TAIL(&cs->funcs, clp, list);
+		}
+	}
+}
+
+/*
+ * This function has *very* special semantics, related to the mgt/worker
+ * process Copy-On-Write memory relationship.
+ */
+
+void
+VCLS_Clone(struct VCLS *cs, struct VCLS *cso)
+{
+	struct cli_proto *clp, *clp2;
+
+	CHECK_OBJ_NOTNULL(cs, VCLS_MAGIC);
+	CHECK_OBJ_NOTNULL(cso, VCLS_MAGIC);
+	VTAILQ_FOREACH_SAFE(clp, &cso->funcs, list, clp2) {
+		VTAILQ_REMOVE(&cso->funcs, clp, list);
+		VTAILQ_INSERT_TAIL(&cs->funcs, clp, list);
+		clp->auth = 0;
+		clp->func = NULL;
+	}
+}
+
+int
+VCLS_PollFd(struct VCLS *cs, int fd, int timeout)
+{
+	struct VCLS_fd *cfd;
+	struct pollfd pfd[1];
+	int i, j, k;
+
+	CHECK_OBJ_NOTNULL(cs, VCLS_MAGIC);
+	if (cs->nfd == 0) {
+		errno = 0;
+		return (-1);
+	}
+	assert(cs->nfd > 0);
+
+	i = 0;
+	VTAILQ_FOREACH(cfd, &cs->fds, list) {
+		if (cfd->fdi != fd)
+			continue;
+		pfd[i].fd = cfd->fdi;
+		pfd[i].events = POLLIN;
+		pfd[i].revents = 0;
+		i++;
+		break;
+	}
+	assert(i == 1);
+	CHECK_OBJ_NOTNULL(cfd, VCLS_FD_MAGIC);
+
+	j = poll(pfd, 1, timeout);
+	if (j <= 0)
+		return (j);
+	if (pfd[0].revents & POLLHUP)
+		k = 1;
+	else
+		k = VLU_Fd(cfd->fdi, cfd->cli->vlu);
+	if (k)
+		cls_close_fd(cs, cfd);
+	return (k);
+}
+
+int
+VCLS_Poll(struct VCLS *cs, int timeout)
+{
+	struct VCLS_fd *cfd, *cfd2;
+	int i, j, k;
+
+	CHECK_OBJ_NOTNULL(cs, VCLS_MAGIC);
+	if (cs->nfd == 0) {
+		errno = 0;
+		return (-1);
+	}
+	assert(cs->nfd > 0);
+	{
+		struct pollfd pfd[cs->nfd];
+
+		i = 0;
+		VTAILQ_FOREACH(cfd, &cs->fds, list) {
+			pfd[i].fd = cfd->fdi;
+			pfd[i].events = POLLIN;
+			pfd[i].revents = 0;
+			i++;
+		}
+		assert(i == cs->nfd);
+
+		j = poll(pfd, cs->nfd, timeout);
+		if (j <= 0)
+			return (j);
+		i = 0;
+		VTAILQ_FOREACH_SAFE(cfd, &cs->fds, list, cfd2) {
+			assert(pfd[i].fd == cfd->fdi);
+			if (pfd[i].revents & POLLHUP)
+				k = 1;
+			else
+				k = VLU_Fd(cfd->fdi, cfd->cli->vlu);
+			if (k)
+				cls_close_fd(cs, cfd);
+			i++;
+		}
+		assert(i == j);
+	}
+	return (j);
+}
+
+void
+VCLS_Destroy(struct VCLS **csp)
+{
+	struct VCLS *cs;
+	struct VCLS_fd *cfd, *cfd2;
+	struct cli_proto *clp;
+
+	cs = *csp;
+	*csp = NULL;
+	CHECK_OBJ_NOTNULL(cs, VCLS_MAGIC);
+	VTAILQ_FOREACH_SAFE(cfd, &cs->fds, list, cfd2)
+		cls_close_fd(cs, cfd);
+
+	while (!VTAILQ_EMPTY(&cs->funcs)) {
+		clp = VTAILQ_FIRST(&cs->funcs);
+		VTAILQ_REMOVE(&cs->funcs, clp, list);
+	}
+	FREE_OBJ(cs);
+}



From phk at FreeBSD.org  Sat May 21 23:19:05 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Sun, 22 May 2016 01:19:05 +0200
Subject: [master] 13b4890 Rename vcli_priv.h to vcli_serve.h for consistency,
 and eliminate a couple of unnecessary includes of it.
Message-ID: <E1b4GAj-0003o8-9s@project.varnish-software.com>


commit 13b4890b3375150453fb8f1610465bf41c31e4fe
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Sat May 21 23:18:18 2016 +0000

    Rename vcli_priv.h to vcli_serve.h for consistency, and eliminate
    a couple of unnecessary includes of it.

diff --git a/bin/varnishd/cache/cache_acceptor.c b/bin/varnishd/cache/cache_acceptor.c
index b45c4f4..58481d5 100644
--- a/bin/varnishd/cache/cache_acceptor.c
+++ b/bin/varnishd/cache/cache_acceptor.c
@@ -45,7 +45,7 @@
 #include "cache_pool.h"
 #include "common/heritage.h"
 
-#include "vcli_priv.h"
+#include "vcli_serve.h"
 #include "vsa.h"
 #include "vtcp.h"
 #include "vtim.h"
diff --git a/bin/varnishd/cache/cache_backend_cfg.c b/bin/varnishd/cache/cache_backend_cfg.c
index ad94900..6c7d8e4 100644
--- a/bin/varnishd/cache/cache_backend_cfg.c
+++ b/bin/varnishd/cache/cache_backend_cfg.c
@@ -39,8 +39,7 @@
 #include "cache.h"
 
 #include "vcl.h"
-#include "vcli.h"
-#include "vcli_priv.h"
+#include "vcli_serve.h"
 #include "vrt.h"
 #include "vtim.h"
 
diff --git a/bin/varnishd/cache/cache_backend_probe.c b/bin/varnishd/cache/cache_backend_probe.c
index ea63abe..0adf25d 100644
--- a/bin/varnishd/cache/cache_backend_probe.c
+++ b/bin/varnishd/cache/cache_backend_probe.c
@@ -44,7 +44,7 @@
 #include "cache.h"
 
 #include "binary_heap.h"
-#include "vcli_priv.h"
+#include "vcli_serve.h"
 #include "vrt.h"
 #include "vsa.h"
 #include "vtcp.h"
diff --git a/bin/varnishd/cache/cache_ban.c b/bin/varnishd/cache/cache_ban.c
index 302d1e9..e4b25e1 100644
--- a/bin/varnishd/cache/cache_ban.c
+++ b/bin/varnishd/cache/cache_ban.c
@@ -36,8 +36,7 @@
 #include "cache_ban.h"
 
 #include "hash/hash_slinger.h"
-#include "vcli.h"
-#include "vcli_priv.h"
+#include "vcli_serve.h"
 #include "vend.h"
 #include "vmb.h"
 
diff --git a/bin/varnishd/cache/cache_cli.c b/bin/varnishd/cache/cache_cli.c
index db72904..fa8aba8 100644
--- a/bin/varnishd/cache/cache_cli.c
+++ b/bin/varnishd/cache/cache_cli.c
@@ -39,9 +39,6 @@
 #include "cache.h"
 #include "common/heritage.h"
 
-#include "vcli.h"
-#include "vcli_common.h"
-#include "vcli_priv.h"
 #include "vcli_serve.h"
 
 pthread_t		cli_thread;
diff --git a/bin/varnishd/cache/cache_fetch_proc.c b/bin/varnishd/cache/cache_fetch_proc.c
index 49862db..cb094fd 100644
--- a/bin/varnishd/cache/cache_fetch_proc.c
+++ b/bin/varnishd/cache/cache_fetch_proc.c
@@ -33,8 +33,7 @@
 
 #include "cache.h"
 #include "cache_filter.h"
-
-#include "vcli_priv.h"
+#include "vcli_serve.h"
 
 static unsigned fetchfrag;
 
diff --git a/bin/varnishd/cache/cache_main.c b/bin/varnishd/cache/cache_main.c
index 413affb..19b519d 100644
--- a/bin/varnishd/cache/cache_main.c
+++ b/bin/varnishd/cache/cache_main.c
@@ -35,7 +35,7 @@
 #include "cache.h"
 #include "common/heritage.h"
 
-#include "vcli_priv.h"
+#include "vcli_serve.h"
 #include "vrnd.h"
 
 #include "hash/hash_slinger.h"
diff --git a/bin/varnishd/cache/cache_panic.c b/bin/varnishd/cache/cache_panic.c
index 51aeb46..7fb6bd8 100644
--- a/bin/varnishd/cache/cache_panic.c
+++ b/bin/varnishd/cache/cache_panic.c
@@ -48,7 +48,7 @@
 #include "vrt.h"
 #include "cache_director.h"
 #include "storage/storage.h"
-#include "vcli_priv.h"
+#include "vcli_serve.h"
 
 /*
  * The panic string is constructed in memory, then copied to the
diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index a8702e7..f36f2b4 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -44,8 +44,7 @@
 
 #include "cache_director.h"
 #include "cache_backend.h"
-#include "vcli.h"
-#include "vcli_priv.h"
+#include "vcli_serve.h"
 
 static const char * const VCL_TEMP_INIT = "init";
 static const char * const VCL_TEMP_COLD = "cold";
diff --git a/bin/varnishd/cache/cache_vrt_vmod.c b/bin/varnishd/cache/cache_vrt_vmod.c
index 3904f28..0b7e15f 100644
--- a/bin/varnishd/cache/cache_vrt_vmod.c
+++ b/bin/varnishd/cache/cache_vrt_vmod.c
@@ -37,7 +37,7 @@
 
 #include "cache.h"
 
-#include "vcli_priv.h"
+#include "vcli_serve.h"
 #include "vrt.h"
 
 /*--------------------------------------------------------------------
diff --git a/bin/varnishd/hash/hash_critbit.c b/bin/varnishd/hash/hash_critbit.c
index 9ac2518..3b09fc5 100644
--- a/bin/varnishd/hash/hash_critbit.c
+++ b/bin/varnishd/hash/hash_critbit.c
@@ -37,7 +37,6 @@
 #include "cache/cache.h"
 
 #include "hash/hash_slinger.h"
-#include "vcli_priv.h"
 #include "vmb.h"
 #include "vtim.h"
 
diff --git a/bin/varnishd/http1/cache_http1_fetch.c b/bin/varnishd/http1/cache_http1_fetch.c
index 1cf127c..153f1d3 100644
--- a/bin/varnishd/http1/cache_http1_fetch.c
+++ b/bin/varnishd/http1/cache_http1_fetch.c
@@ -35,7 +35,6 @@
 
 #include "cache/cache.h"
 
-#include "vcli_priv.h"
 #include "vrt.h"
 #include "vtcp.h"
 #include "vtim.h"
diff --git a/bin/varnishd/mgt/mgt_acceptor.c b/bin/varnishd/mgt/mgt_acceptor.c
index aa3ce5a..70a333a 100644
--- a/bin/varnishd/mgt/mgt_acceptor.c
+++ b/bin/varnishd/mgt/mgt_acceptor.c
@@ -44,7 +44,7 @@
 #include "common/heritage.h"
 
 #include "vav.h"
-#include "vcli_priv.h"
+#include "vcli_serve.h"
 #include "vsa.h"
 #include "vss.h"
 #include "vtcp.h"
diff --git a/bin/varnishd/mgt/mgt_child.c b/bin/varnishd/mgt/mgt_child.c
index d451e97..836a049 100644
--- a/bin/varnishd/mgt/mgt_child.c
+++ b/bin/varnishd/mgt/mgt_child.c
@@ -47,8 +47,6 @@
 #include "common/heritage.h"
 
 #include "vbm.h"
-#include "vcli.h"
-#include "vcli_priv.h"
 #include "vcli_serve.h"
 #include "vev.h"
 #include "vlu.h"
diff --git a/bin/varnishd/mgt/mgt_cli.c b/bin/varnishd/mgt/mgt_cli.c
index e6ed6a5..2fbcdb3 100644
--- a/bin/varnishd/mgt/mgt_cli.c
+++ b/bin/varnishd/mgt/mgt_cli.c
@@ -45,9 +45,6 @@
 
 #include "mgt/mgt.h"
 
-#include "vcli.h"
-#include "vcli_common.h"
-#include "vcli_priv.h"
 #include "vcli_serve.h"
 #include "vev.h"
 #include "vrnd.h"
diff --git a/bin/varnishd/mgt/mgt_main.c b/bin/varnishd/mgt/mgt_main.c
index f5b41c9..56394be 100644
--- a/bin/varnishd/mgt/mgt_main.c
+++ b/bin/varnishd/mgt/mgt_main.c
@@ -49,8 +49,7 @@
 
 #include "hash/hash_slinger.h"
 #include "vav.h"
-#include "vcli.h"
-#include "vcli_common.h"
+#include "vcli_serve.h"
 #include "vev.h"
 #include "vfil.h"
 #include "vin.h"
diff --git a/bin/varnishd/mgt/mgt_param.c b/bin/varnishd/mgt/mgt_param.c
index 4211920..ef63fcd 100644
--- a/bin/varnishd/mgt/mgt_param.c
+++ b/bin/varnishd/mgt/mgt_param.c
@@ -40,9 +40,6 @@
 
 #include "mgt/mgt_param.h"
 #include "vav.h"
-#include "vcli.h"
-#include "vcli_common.h"
-#include "vcli_priv.h"
 #include "vcli_serve.h"
 
 struct plist {
diff --git a/bin/varnishd/mgt/mgt_vcc.c b/bin/varnishd/mgt/mgt_vcc.c
index d35ccd4..62b12e8 100644
--- a/bin/varnishd/mgt/mgt_vcc.c
+++ b/bin/varnishd/mgt/mgt_vcc.c
@@ -44,8 +44,7 @@
 #include "storage/storage.h"
 
 #include "libvcc.h"
-#include "vcli.h"
-#include "vcli_priv.h"
+#include "vcli_serve.h"
 #include "vfil.h"
 #include "vsub.h"
 
diff --git a/bin/varnishd/mgt/mgt_vcl.c b/bin/varnishd/mgt/mgt_vcl.c
index 67d8c40..8817440 100644
--- a/bin/varnishd/mgt/mgt_vcl.c
+++ b/bin/varnishd/mgt/mgt_vcl.c
@@ -40,8 +40,6 @@
 
 #include "mgt/mgt.h"
 
-#include "vcli.h"
-#include "vcli_priv.h"
 #include "vcli_serve.h"
 #include "vev.h"
 #include "vtim.h"
diff --git a/bin/varnishd/storage/mgt_stevedore.c b/bin/varnishd/storage/mgt_stevedore.c
index a540136..dff69fd 100644
--- a/bin/varnishd/storage/mgt_stevedore.c
+++ b/bin/varnishd/storage/mgt_stevedore.c
@@ -39,7 +39,6 @@
 #include <unistd.h>
 
 #include "mgt/mgt.h"
-#include "vcli_priv.h"
 #include "vcli_serve.h"
 
 #include "storage/storage.h"
diff --git a/bin/varnishd/storage/storage_persistent.c b/bin/varnishd/storage/storage_persistent.c
index d439517..7991d9b 100644
--- a/bin/varnishd/storage/storage_persistent.c
+++ b/bin/varnishd/storage/storage_persistent.c
@@ -48,8 +48,7 @@
 #include "storage/storage_simple.h"
 
 #include "hash/hash_slinger.h"
-#include "vcli.h"
-#include "vcli_priv.h"
+#include "vcli_serve.h"
 #include "vsha256.h"
 #include "vtim.h"
 
diff --git a/doc/sphinx/reference/varnish-cli.rst b/doc/sphinx/reference/varnish-cli.rst
index 314e9a0..67c8dc8 100644
--- a/doc/sphinx/reference/varnish-cli.rst
+++ b/doc/sphinx/reference/varnish-cli.rst
@@ -149,8 +149,8 @@ One particular magic number to know, is that the line with the status
 code and length field always is exactly 13 characters long, including
 the NL character.
 
-For your reference the sourcefile lib/libvarnish/cli_common.h contains
-the functions Varnish code uses to read and write CLI response.
+The varnishapi library contains functions to implement the basics of
+the CLI protocol, see the `vcli.h` include file.
 
 .. _ref_psk_auth:
 
diff --git a/include/Makefile.am b/include/Makefile.am
index caae1c7..ca51410 100644
--- a/include/Makefile.am
+++ b/include/Makefile.am
@@ -69,8 +69,6 @@ nobase_noinst_HEADERS = \
 	compat/execinfo.h \
 	vfl.h \
 	libvcc.h \
-	vcli_common.h \
-	vcli_priv.h \
 	vcli_serve.h \
 	vcs_version.h \
 	vct.h \
diff --git a/include/vcli_priv.h b/include/vcli_priv.h
deleted file mode 100644
index 09bd042..0000000
--- a/include/vcli_priv.h
+++ /dev/null
@@ -1,73 +0,0 @@
-/*-
- * Copyright (c) 2006 Verdens Gang AS
- * Copyright (c) 2006-2011 Varnish Software AS
- * All rights reserved.
- *
- * Author: Poul-Henning Kamp <phk at phk.freebsd.dk>
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
- * 1. Redistributions of source code must retain the above copyright
- *    notice, this list of conditions and the following disclaimer.
- * 2. Redistributions in binary form must reproduce the above copyright
- *    notice, this list of conditions and the following disclaimer in the
- *    documentation and/or other materials provided with the distribution.
- *
- * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
- * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED.  IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
- * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
- * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
- * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
- * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
- * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
- * SUCH DAMAGE.
- *
- * Varnish process internal CLI stuff.
- *
- * XXX: at a latter date we may want to move some to cli.h/libvarnishapi
- */
-
-#define CLI_PRIV_H
-
-struct cli;	/* NB: struct cli is opaque at this level.  */
-
-typedef void cli_func_t(struct cli*, const char * const *av, void *priv);
-
-struct cli_cmd_desc {
-	/* Must match CLI_CMD macro in include/tbl/cli_cmds.h */
-	const char			*request;
-	const char			*syntax;
-	const char			*help;
-	const char			*doc;
-	int				minarg;
-	int				maxarg;
-};
-
-#define CLI_CMD(U,l,s,h,d,m,M) extern const struct cli_cmd_desc CLICMD_##U[1];
-#include "tbl/cli_cmds.h"
-#undef CLI_CMD
-
-struct cli_proto {
-	const struct cli_cmd_desc	*desc;
-	const char			*flags;
-
-	/* Dispatch information */
-	cli_func_t			*func;
-	cli_func_t			*jsonfunc;
-	void				*priv;
-
-	unsigned			auth;
-	VTAILQ_ENTRY(cli_proto)		list;
-};
-
-/* The implementation must provide these functions */
-int VCLI_Overflow(struct cli *cli);
-void VCLI_Out(struct cli *cli, const char *fmt, ...) __v_printflike(2, 3);
-void VCLI_Quote(struct cli *cli, const char *str);
-void VCLI_JSON_str(struct cli *cli, const char *str);
-void VCLI_JSON_ver(struct cli *cli, unsigned ver, const char * const * av);
-void VCLI_SetResult(struct cli *cli, unsigned r);
diff --git a/include/vcli_serve.h b/include/vcli_serve.h
new file mode 100644
index 0000000..cfe4265
--- /dev/null
+++ b/include/vcli_serve.h
@@ -0,0 +1,109 @@
+/*-
+ * Copyright (c) 2006 Verdens Gang AS
+ * Copyright (c) 2006-2011 Varnish Software AS
+ * All rights reserved.
+ *
+ * Author: Poul-Henning Kamp <phk at phk.freebsd.dk>
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED.  IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ *
+ * Varnish process internal CLI stuff.
+ *
+ * XXX: at a latter date we may want to move some to cli.h/libvarnishapi
+ */
+
+#include "vcli.h"
+
+struct cli;	/* NB: struct cli is opaque at this level.  */
+struct vlu;
+struct VCLS;
+
+typedef void cli_func_t(struct cli*, const char * const *av, void *priv);
+
+struct cli_cmd_desc {
+	/* Must match CLI_CMD macro in include/tbl/cli_cmds.h */
+	const char			*request;
+	const char			*syntax;
+	const char			*help;
+	const char			*doc;
+	int				minarg;
+	int				maxarg;
+};
+
+#define CLI_CMD(U,l,s,h,d,m,M) extern const struct cli_cmd_desc CLICMD_##U[1];
+#include "tbl/cli_cmds.h"
+#undef CLI_CMD
+
+/* A CLI command */
+struct cli_proto {
+	const struct cli_cmd_desc	*desc;
+	const char			*flags;
+
+	/* Dispatch information */
+	cli_func_t			*func;
+	cli_func_t			*jsonfunc;
+	void				*priv;
+
+	unsigned			auth;
+	VTAILQ_ENTRY(cli_proto)		list;
+};
+
+/* a CLI session */
+struct cli {
+	unsigned                magic;
+#define CLI_MAGIC              0x4038d570
+	struct vsb              *sb;
+	enum VCLI_status_e      result;
+	char                    *cmd;
+	unsigned                auth;
+	char                    challenge[34];
+	char                    *ident;
+	struct vlu              *vlu;
+	struct VCLS             *cls;
+	volatile unsigned       *limit;
+};
+
+/* The implementation must provide these functions */
+int VCLI_Overflow(struct cli *cli);
+void VCLI_Out(struct cli *cli, const char *fmt, ...) __v_printflike(2, 3);
+void VCLI_Quote(struct cli *cli, const char *str);
+void VCLI_JSON_str(struct cli *cli, const char *str);
+void VCLI_JSON_ver(struct cli *cli, unsigned ver, const char * const * av);
+void VCLI_SetResult(struct cli *cli, unsigned r);
+
+typedef void cls_cb_f(void *priv);
+typedef void cls_cbc_f(const struct cli*);
+struct VCLS *VCLS_New(cls_cbc_f *before, cls_cbc_f *after,
+    volatile unsigned *maxlen, volatile unsigned *limit);
+struct cli *VCLS_AddFd(struct VCLS *cs, int fdi, int fdo, cls_cb_f *closefunc,
+    void *priv);
+void VCLS_AddFunc(struct VCLS *cs, unsigned auth, struct cli_proto *clp);
+int VCLS_Poll(struct VCLS *cs, int timeout);
+int VCLS_PollFd(struct VCLS *cs, int fd, int timeout);
+void VCLS_Destroy(struct VCLS **);
+void VCLS_Clone(struct VCLS *cs, struct VCLS *cso);
+
+/* From libvarnish/cli.c */
+cli_func_t	VCLS_func_close;
+cli_func_t	VCLS_func_help;
+cli_func_t	VCLS_func_help_json;
+cli_func_t	VCLS_func_ping;
diff --git a/lib/libvarnish/Makefile.am b/lib/libvarnish/Makefile.am
index 6d6665a..5dd728e 100644
--- a/lib/libvarnish/Makefile.am
+++ b/lib/libvarnish/Makefile.am
@@ -14,9 +14,8 @@ libvarnish_la_SOURCES = \
 	vas.c \
 	binary_heap.c \
 	vsub.c \
-	cli_auth.c \
-	cli_common.c \
-	cli_serve.c \
+	vcli_proto.c \
+	vcli_serve.c \
 	vfl.c \
 	vnum.c \
 	vtim.c \
diff --git a/lib/libvarnish/vcli_proto.c b/lib/libvarnish/vcli_proto.c
index d7b7bf2..27e2104 100644
--- a/lib/libvarnish/vcli_proto.c
+++ b/lib/libvarnish/vcli_proto.c
@@ -29,9 +29,14 @@
 #include "config.h"
 
 #include <sys/types.h>
+#include <sys/uio.h>
 
+#include <errno.h>
+#include <poll.h>
 #include <stdint.h>
 #include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
 #include <unistd.h>
 
 #include "vas.h"
@@ -62,3 +67,130 @@ VCLI_AuthResponse(int S_fd, const char *challenge,
 	for(i = 0; i < SHA256_LEN; i++)
 		sprintf(response + 2 * i, "%02x", buf[i]);
 }
+
+int
+VCLI_WriteResult(int fd, unsigned status, const char *result)
+{
+	int i, l;
+	struct iovec iov[3];
+	char nl[2] = "\n";
+	size_t len;
+	char res[CLI_LINE0_LEN + 2];	/*
+					 * NUL + one more so we can catch
+					 * any misformats by snprintf
+					 */
+
+	assert(status >= 100);
+	assert(status <= 999);		/*lint !e650 const out of range */
+
+	len = strlen(result);
+
+	i = snprintf(res, sizeof res,
+	    "%-3d %-8zd\n", status, len);
+	assert(i == CLI_LINE0_LEN);
+	assert(strtoul(res + 3, NULL, 10) == len);
+
+	iov[0].iov_base = res;
+	iov[0].iov_len = CLI_LINE0_LEN;
+
+	iov[1].iov_base = (void*)(uintptr_t)result;	/* TRUST ME */
+	iov[1].iov_len = len;
+
+	iov[2].iov_base = nl;
+	iov[2].iov_len = 1;
+
+	for (l = i = 0; i < 3; i++)
+		l += iov[i].iov_len;
+	i = writev(fd, iov, 3);
+	return (i != l);
+}
+
+static int
+read_tmo(int fd, char *ptr, unsigned len, double tmo)
+{
+	int i, j, to;
+	struct pollfd pfd;
+
+	if (tmo > 0)
+		to = (int)(tmo * 1e3);
+	else
+		to = -1;
+	pfd.fd = fd;
+	pfd.events = POLLIN;
+	for (j = 0; len > 0; ) {
+		i = poll(&pfd, 1, to);
+		if (i == 0) {
+			errno = ETIMEDOUT;
+			return (-1);
+		}
+		i = read(fd, ptr, len);
+		if (i < 0)
+			return (i);
+		if (i == 0)
+			break;
+		len -= i;
+		ptr += i;
+		j += i;
+	}
+	return (j);
+}
+
+int
+VCLI_ReadResult(int fd, unsigned *status, char **ptr, double tmo)
+{
+	char res[CLI_LINE0_LEN];	/* For NUL */
+	int i, j;
+	unsigned u, v, s;
+	char *p = NULL;
+	const char *err = "CLI communication error (hdr)";
+
+	if (status == NULL)
+		status = &s;
+	if (ptr != NULL)
+		*ptr = NULL;
+	do {
+		i = read_tmo(fd, res, CLI_LINE0_LEN, tmo);
+		if (i != CLI_LINE0_LEN)
+			break;
+
+		if (res[3] != ' ')
+			break;
+
+		if (res[CLI_LINE0_LEN - 1] != '\n')
+			break;
+
+		res[CLI_LINE0_LEN - 1] = '\0';
+		j = sscanf(res, "%u %u\n", &u, &v);
+		if (j != 2)
+			break;
+
+		err = "CLI communication error (body)";
+
+		*status = u;
+		p = malloc(v + 1L);
+		if (p == NULL)
+			break;
+
+		i = read_tmo(fd, p, v + 1, tmo);
+		if (i < 0)
+			break;
+		if (i != v + 1)
+			break;
+		if (p[v] != '\n')
+			break;
+
+		p[v] = '\0';
+		if (ptr == NULL)
+			free(p);
+		else
+			*ptr = p;
+		return (0);
+	} while(0);
+
+	if (p != NULL)
+		free(p);
+	*status = CLIS_COMMS;
+	if (ptr != NULL)
+		*ptr = strdup(err);
+	return (*status);
+}
diff --git a/lib/libvarnish/vcli_serve.c b/lib/libvarnish/vcli_serve.c
index 4e88bdc..80d09de 100644
--- a/lib/libvarnish/vcli_serve.c
+++ b/lib/libvarnish/vcli_serve.c
@@ -34,6 +34,7 @@
 #include <ctype.h>
 #include <errno.h>
 #include <poll.h>
+#include <stdarg.h>
 #include <stdint.h>
 #include <stdio.h>
 #include <stdlib.h>
@@ -47,9 +48,6 @@
 #include "miniobj.h"
 
 #include "vav.h"
-#include "vcli.h"
-#include "vcli_common.h"
-#include "vcli_priv.h"
 #include "vcli_serve.h"
 #include "vlu.h"
 #include "vsb.h"
@@ -614,3 +612,84 @@ VCLS_Destroy(struct VCLS **csp)
 	}
 	FREE_OBJ(cs);
 }
+
+/**********************************************************************
+ * Utility functions for implementing CLI commands
+ */
+
+/*lint -e{818} cli could be const */
+void
+VCLI_Out(struct cli *cli, const char *fmt, ...)
+{
+	va_list ap;
+
+	va_start(ap, fmt);
+	if (cli != NULL) {
+		CHECK_OBJ_NOTNULL(cli, CLI_MAGIC);
+		if (VSB_len(cli->sb) < *cli->limit)
+			(void)VSB_vprintf(cli->sb, fmt, ap);
+		else if (cli->result == CLIS_OK)
+			cli->result = CLIS_TRUNCATED;
+	} else {
+		(void)vfprintf(stdout, fmt, ap);
+	}
+	va_end(ap);
+}
+
+/*lint -e{818} cli could be const */
+int
+VCLI_Overflow(struct cli *cli)
+{
+	CHECK_OBJ_NOTNULL(cli, CLI_MAGIC);
+	if (cli->result == CLIS_TRUNCATED ||
+	    VSB_len(cli->sb) >= *cli->limit)
+		return (1);
+	return (0);
+}
+
+/*lint -e{818} cli could be const */
+void
+VCLI_JSON_str(struct cli *cli, const char *s)
+{
+
+	CHECK_OBJ_NOTNULL(cli, CLI_MAGIC);
+	VSB_quote(cli->sb, s, -1, VSB_QUOTE_JSON);
+}
+
+/*lint -e{818} cli could be const */
+void
+VCLI_JSON_ver(struct cli *cli, unsigned ver, const char * const * av)
+{
+	int i;
+
+	CHECK_OBJ_NOTNULL(cli, CLI_MAGIC);
+	VCLI_Out(cli, "[ %u, [", ver);
+	for (i = 1; av[i] != NULL; i++) {
+		VCLI_JSON_str(cli, av[i]);
+		if (av[i + 1] != NULL)
+			VCLI_Out(cli, ", ");
+	}
+	VCLI_Out(cli, "]");
+}
+
+/*lint -e{818} cli could be const */
+void
+VCLI_Quote(struct cli *cli, const char *s)
+{
+
+	CHECK_OBJ_NOTNULL(cli, CLI_MAGIC);
+	VSB_quote(cli->sb, s, -1, 0);
+}
+
+void
+VCLI_SetResult(struct cli *cli, unsigned res)
+{
+
+	if (cli != NULL) {
+		CHECK_OBJ_NOTNULL(cli, CLI_MAGIC);
+		if (cli->result != CLIS_TRUNCATED || res != CLIS_OK)
+			cli->result = res;	/*lint !e64 type mismatch */
+	} else {
+		printf("CLI result = %u\n", res);
+	}
+}
diff --git a/lib/libvarnishapi/Makefile.am b/lib/libvarnishapi/Makefile.am
index 03fefe4..455cf3a 100644
--- a/lib/libvarnishapi/Makefile.am
+++ b/lib/libvarnishapi/Makefile.am
@@ -20,8 +20,7 @@ libvarnishapi_la_SOURCES = \
 	../libvarnish/vav.c \
 	../../include/vcs_version.h \
 	../libvarnish/version.c \
-	../libvarnish/cli_common.c \
-	../libvarnish/cli_auth.c \
+	../libvarnish/vcli_proto.c \
 	../libvarnish/vfl.c \
 	../libvarnish/vin.c \
 	../libvarnish/vmb.c \



From phk at FreeBSD.org  Sun May 22 10:07:04 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Sun, 22 May 2016 12:07:04 +0200
Subject: [master] 34350d5 Introduce "VCL labels".
Message-ID: <E1b4QHo-000732-UU@project.varnish-software.com>


commit 34350d5e183ef4e04285729d1f63b784d1bc6454
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Sun May 22 09:37:22 2016 +0000

    Introduce "VCL labels".
    
    A VCL label is a symbolic name pointing to a "real" VCL program.
    
    VCL labels can be the active VCL and can be repointed to a different
    VCL at any time.
    
    Labels are always warm, and can be discarded with vcl.discard if not
    in use.
    
    Labeled VCLs are _also_ always warm.
    
    One possible use could be for site develpment to label a "production",
    and a "emergency" VCL so operations personel only have to know
    these two labels, not worrying about versioning of the VCLs.
    
    Of course I have other interesting evil plans for this too...

diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index f36f2b4..dfc7235 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -51,6 +51,7 @@ static const char * const VCL_TEMP_COLD = "cold";
 static const char * const VCL_TEMP_WARM = "warm";
 static const char * const VCL_TEMP_BUSY = "busy";
 static const char * const VCL_TEMP_COOLING = "cooling";
+static const char * const VCL_TEMP_LABEL = "label";
 
 struct vcl {
 	unsigned		magic;
@@ -65,6 +66,7 @@ struct vcl {
 	const char		*temp;
 	VTAILQ_HEAD(,backend)	backend_list;
 	VTAILQ_HEAD(,vclref)	ref_list;
+	struct vcl		*label;
 };
 
 struct vclref {
@@ -161,7 +163,10 @@ VCL_Get(struct vcl **vcc)
 	assert(vcl_active->temp == VCL_TEMP_WARM);
 	Lck_Lock(&vcl_mtx);
 	AN(vcl_active);
-	*vcc = vcl_active;
+	if (vcl_active->label == NULL)
+		*vcc = vcl_active;
+	else
+		*vcc = vcl_active->label;
 	AN(*vcc);
 	AZ((*vcc)->discard);
 	(*vcc)->busy++;
@@ -609,11 +614,7 @@ VCL_Load(struct cli *cli, const char *name, const char *fn, const char *state)
 	ASSERT_CLI();
 
 	vcl = vcl_find(name);
-	if (vcl != NULL) {
-		VCLI_SetResult(cli, CLIS_PARAM);
-		VCLI_Out(cli, "Config '%s' already loaded", name);
-		return;
-	}
+	AZ(vcl);
 
 	vsb = VSB_new_auto();
 	AN(vsb);
@@ -736,8 +737,14 @@ ccf_config_list(struct cli *cli, const char * const *av, void *priv)
 			flg = "discarded";
 		} else
 			flg = "available";
-		VCLI_Out(cli, "%-10s %4s/%-8s %6u %s\n",
+		VCLI_Out(cli, "%-10s %5s/%-8s %6u %s",
 		    flg, vcl->state, vcl->temp, vcl->busy, vcl->loaded_name);
+		if (vcl->label != NULL) {
+			VCLI_Out(cli, " %s %s",
+			    strcmp(vcl->state, VCL_TEMP_LABEL) ?
+			    "<-" : "->", vcl->label->loaded_name);
+		}
+		VCLI_Out(cli, "\n");
 	}
 }
 
@@ -797,13 +804,49 @@ ccf_config_discard(struct cli *cli, const char * const *av, void *priv)
 	VSC_C_main->n_vcl_discard++;
 	VSC_C_main->n_vcl_avail--;
 	vcl->discard = 1;
+	if (vcl->label != NULL) {
+		AZ(strcmp(vcl->state, VCL_TEMP_LABEL));
+		vcl->label->label = NULL;
+		vcl->label= NULL;
+	}
 	Lck_Unlock(&vcl_mtx);
 
-	if (vcl->temp == VCL_TEMP_COLD)
+	if (!strcmp(vcl->state, VCL_TEMP_LABEL)) {
+		VTAILQ_REMOVE(&vcl_head, vcl, list);
+		free(vcl->loaded_name);
+	} else if (vcl->temp == VCL_TEMP_COLD)
 		VCL_Nuke(vcl);
 }
 
 static void __match_proto__(cli_func_t)
+ccf_config_label(struct cli *cli, const char * const *av, void *priv)
+{
+	struct vcl *lbl;
+	struct vcl *vcl;
+
+	ASSERT_CLI();
+	(void)cli;
+	(void)priv;
+	vcl = vcl_find(av[3]);
+	AN(vcl);
+	lbl = vcl_find(av[2]);
+	if (lbl == NULL) {
+		ALLOC_OBJ(lbl, VCL_MAGIC);
+		AN(lbl);
+		strcpy(lbl->state, VCL_TEMP_LABEL);
+		lbl->temp = VCL_TEMP_WARM;
+		lbl->loaded_name = strdup(av[2]);
+		AN(lbl->loaded_name);
+		VTAILQ_INSERT_TAIL(&vcl_head, lbl, list);
+	}
+	if (lbl->label != NULL)
+		lbl->label->label = NULL;
+	lbl->label = vcl;
+	vcl->label = lbl;
+	return;
+}
+
+static void __match_proto__(cli_func_t)
 ccf_config_use(struct cli *cli, const char * const *av, void *priv)
 {
 	struct vcl *vcl;
@@ -952,6 +995,7 @@ static struct cli_proto vcl_cmds[] = {
 	{ CLICMD_VCL_DISCARD,		"", ccf_config_discard },
 	{ CLICMD_VCL_USE,		"", ccf_config_use },
 	{ CLICMD_VCL_SHOW,		"", ccf_config_show },
+	{ CLICMD_VCL_LABEL,		"", ccf_config_label },
 	{ NULL }
 };
 
diff --git a/bin/varnishd/mgt/mgt_vcl.c b/bin/varnishd/mgt/mgt_vcl.c
index 8817440..1b3950c 100644
--- a/bin/varnishd/mgt/mgt_vcl.c
+++ b/bin/varnishd/mgt/mgt_vcl.c
@@ -47,6 +47,7 @@
 static const char * const VCL_STATE_COLD = "cold";
 static const char * const VCL_STATE_WARM = "warm";
 static const char * const VCL_STATE_AUTO = "auto";
+static const char * const VCL_STATE_LABEL = "label";
 
 struct vclprog {
 	VTAILQ_ENTRY(vclprog)	list;
@@ -55,6 +56,7 @@ struct vclprog {
 	unsigned		warm;
 	char			state[8];
 	double			go_cold;
+	struct vclprog		*label;
 };
 
 static VTAILQ_HEAD(, vclprog) vclhead = VTAILQ_HEAD_INITIALIZER(vclhead);
@@ -91,7 +93,8 @@ mgt_vcl_del(struct vclprog *vp)
 	char dn[256];
 
 	VTAILQ_REMOVE(&vclhead, vp, list);
-	XXXAZ(unlink(vp->fname));
+	if (strcmp(vp->state, VCL_STATE_LABEL))
+		XXXAZ(unlink(vp->fname));
 	bprintf(dn, "vcl_%s", vp->name);
 	VJ_master(JAIL_MASTER_FILE);
 	(void)rmdir(dn);		// compiler droppings, eg gcov
@@ -127,6 +130,10 @@ mgt_vcl_setstate(struct cli *cli, struct vclprog *vp, const char *vs)
 	char *p;
 	int i;
 
+	if (!strcmp(vp->state, VCL_STATE_LABEL)) {
+		AN(vp->warm);
+		return (0);
+	}
 	if (vs == VCL_STATE_AUTO) {
 		assert(vp != active_vcl);
 		now = VTIM_mono();
@@ -202,16 +209,13 @@ mgt_new_vcl(struct cli *cli, const char *vclname, const char *vclsrc,
 	if (child_pid < 0)
 		return;
 
-	if (!mgt_cli_askchild(&status, &p, "vcl.load %s %s %d%s\n",
+	if (mgt_cli_askchild(&status, &p, "vcl.load %s %s %d%s\n",
 	    vp->name, vp->fname, vp->warm, vp->state)) {
-		free(p);
-		return;
+		mgt_vcl_del(vp);
+		VCLI_Out(cli, "%s", p);
+		VCLI_SetResult(cli, CLIS_PARAM);
 	}
-
-	mgt_vcl_del(vp);
-	VCLI_Out(cli, "%s", p);
 	free(p);
-	VCLI_SetResult(cli, CLIS_PARAM);
 }
 
 /*--------------------------------------------------------------------*/
@@ -251,14 +255,27 @@ mgt_push_vcls_and_start(struct cli *cli, unsigned *status, char **p)
 	AZ(mgt_vcl_setstate(cli, active_vcl, VCL_STATE_WARM));
 
 	VTAILQ_FOREACH(vp, &vclhead, list) {
+		if (!strcmp(vp->state, VCL_STATE_LABEL))
+			continue;
 		if (mgt_cli_askchild(status, p, "vcl.load \"%s\" %s %d%s\n",
 		    vp->name, vp->fname, vp->warm, vp->state))
 			return (1);
 		free(*p);
+		*p = NULL;
+	}
+	VTAILQ_FOREACH(vp, &vclhead, list) {
+		if (strcmp(vp->state, VCL_STATE_LABEL))
+			continue;
+		if (mgt_cli_askchild(status, p, "vcl.label %s %s\n",
+		    vp->name, vp->label->name))
+			return (1);
+		free(*p);
+		*p = NULL;
 	}
 	if (mgt_cli_askchild(status, p, "vcl.use \"%s\"\n", active_vcl->name))
 		return (1);
 	free(*p);
+	*p = NULL;
 	if (mgt_cli_askchild(status, p, "start\n"))
 		return (1);
 	free(*p);
@@ -307,11 +324,11 @@ mcf_find_vcl(struct cli *cli, const char *name)
 	struct vclprog *vp;
 
 	vp = mgt_vcl_byname(name);
-	if (vp != NULL)
-		return (vp);
-	VCLI_SetResult(cli, CLIS_PARAM);
-	VCLI_Out(cli, "No configuration named %s known.", name);
-	return (NULL);
+	if (vp == NULL) {
+		VCLI_SetResult(cli, CLIS_PARAM);
+		VCLI_Out(cli, "No configuration named %s known.", name);
+	}
+	return (vp);
 }
 
 static void __match_proto__(cli_func_t)
@@ -323,6 +340,11 @@ mcf_vcl_state(struct cli *cli, const char * const *av, void *priv)
 	vp = mcf_find_vcl(cli, av[2]);
 	if (vp == NULL)
 		return;
+	if (!strcmp(vp->state, VCL_STATE_LABEL)) {
+		VCLI_Out(cli, "Labels are always warm");
+		VCLI_SetResult(cli, CLIS_PARAM);
+		return;
+	}
 
 	if (!strcmp(vp->state, av[3]))
 		return;
@@ -399,6 +421,16 @@ mcf_vcl_discard(struct cli *cli, const char * const *av, void *priv)
 		VCLI_Out(cli, "Cannot discard active VCL program\n");
 		return;
 	}
+	if (!strcmp(vp->state, VCL_STATE_LABEL)) {
+		vp->label->label = NULL;
+		vp->label = NULL;
+	}
+
+	if (vp->label != NULL) {
+		VCLI_SetResult(cli, CLIS_PARAM);
+		VCLI_Out(cli, "Must remove label to discard VCL\n");
+		return;
+	}
 	(void)mgt_vcl_setstate(cli, vp, VCL_STATE_COLD);
 	if (child_pid >= 0) {
 		/* XXX If this fails the child is crashing, figure that later */
@@ -425,14 +457,64 @@ mcf_vcl_list(struct cli *cli, const char * const *av, void *priv)
 		free(p);
 	} else {
 		VTAILQ_FOREACH(vp, &vclhead, list) {
-			VCLI_Out(cli, "%-10s %4s/%-8s %6s %s\n",
+			VCLI_Out(cli, "%-10s %5s",
 			    vp == active_vcl ? "active" : "available",
-			    vp->state, vp->warm ? "warm" : "cold", "",
-			    vp->name);
+			    vp->state);
+			VCLI_Out(cli, "/%-8s", vp->warm ? "warm" : "cold");
+			VCLI_Out(cli, " %6s %s", "", vp->name);
+			if (vp->label != NULL)
+				VCLI_Out(cli, " %s %s",
+				    strcmp(vp->state, VCL_STATE_LABEL) ?
+				    "<-" : "->", vp->label->name);
+			VCLI_Out(cli, "\n");
 		}
 	}
 }
 
+static void __match_proto__(cli_func_t)
+mcf_vcl_label(struct cli *cli, const char * const *av, void *priv)
+{
+	struct vclprog *vpl;
+	struct vclprog *vpt;
+	unsigned status;
+	char *p;
+	int i;
+
+	(void)av;
+	(void)priv;
+	vpt = mcf_find_vcl(cli, av[3]);
+	if (vpt == NULL)
+		return;
+	if (!strcmp(vpt->state, VCL_STATE_LABEL)) {
+		VCLI_SetResult(cli, CLIS_PARAM);
+		VCLI_Out(cli, "VCL labels cannot point to labels");
+		return;
+	}
+	vpl = mgt_vcl_byname(av[2]);
+	if (vpl == NULL)
+		vpl = mgt_vcl_add(av[2], NULL, VCL_STATE_LABEL);
+	AN(vpl);
+	if (strcmp(vpl->state, VCL_STATE_LABEL)) {
+		VCLI_SetResult(cli, CLIS_PARAM);
+		VCLI_Out(cli, "%s is not a label", vpl->name);
+		return;
+	}
+	vpl->warm = 1;
+	if (vpl->label != NULL)
+		vpl->label->label = NULL;
+	vpl->label = vpt;
+	vpt->label = vpl;
+	if (child_pid < 0)
+		return;
+
+	i = mgt_cli_askchild(&status, &p, "vcl.label %s %s\n", av[2], av[3]);
+	if (i) {
+		VCLI_SetResult(cli, status);
+		VCLI_Out(cli, "%s", p);
+	}
+	free(p);
+}
+
 /*--------------------------------------------------------------------*/
 
 static int __match_proto__(vev_cb_f)
@@ -459,6 +541,7 @@ static struct cli_proto cli_vcl[] = {
 	{ CLICMD_VCL_STATE,		"", mcf_vcl_state },
 	{ CLICMD_VCL_DISCARD,		"", mcf_vcl_discard },
 	{ CLICMD_VCL_LIST,		"", mcf_vcl_list },
+	{ CLICMD_VCL_LABEL,		"", mcf_vcl_label },
 	{ NULL }
 };
 
diff --git a/bin/varnishtest/tests/v00048.vtc b/bin/varnishtest/tests/v00048.vtc
new file mode 100644
index 0000000..6accba4
--- /dev/null
+++ b/bin/varnishtest/tests/v00048.vtc
@@ -0,0 +1,84 @@
+varnishtest "Test VCL labels"
+
+server s1 {
+	rxreq
+	txresp
+} -start
+
+varnish v1 -vcl+backend {}
+
+varnish v1 -vcl+backend {
+	sub vcl_recv {
+		return (synth(400));
+	}
+}
+
+varnish v1 -start
+
+client c1 {
+	txreq
+	rxresp
+	expect resp.status == 400
+} -run
+
+varnish v1 -cliok "vcl.use vcl1"
+
+client c1 {
+	txreq
+	rxresp
+	expect resp.status == 200
+} -run
+
+varnish v1 -cliok "vcl.list"
+varnish v1 -clierr 106 "vcl.label foo vcl0"
+varnish v1 -cliok "vcl.label foo vcl1"
+varnish v1 -clierr 106 "vcl.state foo cold"
+varnish v1 -clierr 106 "vcl.label bar foo"
+varnish v1 -clierr 106 "vcl.discard vcl1"
+varnish v1 -cliok "vcl.list"
+varnish v1 -cliok "vcl.use foo"
+varnish v1 -clierr 106 "vcl.discard foo"
+varnish v1 -cliok "vcl.list"
+
+client c1 -run
+
+varnish v1 -cliok "vcl.label foo vcl2"
+
+client c1 {
+	txreq
+	rxresp
+	expect resp.status == 400
+} -run
+
+varnish v1 -cliok "vcl.use vcl1"
+varnish v1 -cliok "vcl.list"
+
+client c1 {
+	txreq
+	rxresp
+	expect resp.status == 200
+} -run
+
+varnish v1 -cliok "vcl.discard foo"
+varnish v1 -clierr 106 "vcl.discard foo"
+
+varnish v1 -stop
+varnish v1 -cliok "vcl.list"
+varnish v1 -clierr 106 "vcl.label foo vcl0"
+varnish v1 -cliok "vcl.label foo vcl1"
+varnish v1 -clierr 106 "vcl.label bar foo"
+varnish v1 -clierr 106 "vcl.discard vcl1"
+varnish v1 -cliok "vcl.list"
+varnish v1 -cliok "vcl.use foo"
+varnish v1 -clierr 106 "vcl.discard foo"
+varnish v1 -cliok "vcl.list"
+
+server s1 -start
+
+varnish v1 -start
+client c1 -run
+varnish v1 -stop
+varnish v1 -cliok "vcl.use vcl1"
+varnish v1 -cliok "vcl.discard foo"
+varnish v1 -clierr 106 "vcl.discard foo"
+
diff --git a/include/tbl/cli_cmds.h b/include/tbl/cli_cmds.h
index 7869313..a0ccb4b 100644
--- a/include/tbl/cli_cmds.h
+++ b/include/tbl/cli_cmds.h
@@ -95,7 +95,7 @@ CLI_CMD(VCL_STATE,
 
 CLI_CMD(VCL_DISCARD,
 	"vcl.discard",
-	"vcl.discard <configname>",
+	"vcl.discard <configname|label>",
 	"Unload the named configuration (when possible).",
 	"",
 	1, 1
@@ -119,12 +119,20 @@ CLI_CMD(VCL_SHOW,
 
 CLI_CMD(VCL_USE,
 	"vcl.use",
-	"vcl.use <configname>",
+	"vcl.use <configname|label>",
 	"Switch to the named configuration immediately.",
 	"",
 	1, 1
 )
 
+CLI_CMD(VCL_LABEL,
+	"vcl.label",
+	"vcl.label <label> <configname>",
+	"Apply label to configuration.",
+	"",
+	2, 2
+)
+
 CLI_CMD(PARAM_SHOW,
 	"param.show",
 	"param.show [-l] [<param>]",



From phk at FreeBSD.org  Sun May 22 10:07:04 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Sun, 22 May 2016 12:07:04 +0200
Subject: [master] dcbf38c Introduce "VCL labels".
Message-ID: <E1b4QHo-000738-WA@project.varnish-software.com>


commit dcbf38c59c99399ef0a5a381e55a6f0b5ae2bd00
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Sun May 22 10:02:15 2016 +0000

    Introduce "VCL labels".
    
    A VCL label is a symbolic name pointing to a "real" VCL program.
    
    Labels are created with "vcl.label <labelname> <vclname>"
    
    Labels are removed with vcl.discard.
    
    VCL labels can be the active VCL and can be repointed to a different
    VCL while being so.
    
    Labels and the VCLs they point to are always warm.
    
    One possible use could be for site develpment to label a "production",
    and a "emergency" VCL so operations personel only have to know
    these two labels, not worrying about versioning of the VCLs.
    
    Of course I have evil plans which are more interesting...

diff --git a/bin/varnishd/mgt/mgt_vcl.c b/bin/varnishd/mgt/mgt_vcl.c
index 1b3950c..f9cc6bd 100644
--- a/bin/varnishd/mgt/mgt_vcl.c
+++ b/bin/varnishd/mgt/mgt_vcl.c
@@ -130,12 +130,11 @@ mgt_vcl_setstate(struct cli *cli, struct vclprog *vp, const char *vs)
 	char *p;
 	int i;
 
-	if (!strcmp(vp->state, VCL_STATE_LABEL)) {
+	if (vp == active_vcl || vp->label != NULL) {
 		AN(vp->warm);
 		return (0);
 	}
 	if (vs == VCL_STATE_AUTO) {
-		assert(vp != active_vcl);
 		now = VTIM_mono();
 		vs = vp->warm ? VCL_STATE_WARM : VCL_STATE_COLD;
 		if (vp->go_cold > 0 && !strcmp(vp->state, "auto") &&
@@ -340,11 +339,20 @@ mcf_vcl_state(struct cli *cli, const char * const *av, void *priv)
 	vp = mcf_find_vcl(cli, av[2]);
 	if (vp == NULL)
 		return;
+
 	if (!strcmp(vp->state, VCL_STATE_LABEL)) {
 		VCLI_Out(cli, "Labels are always warm");
 		VCLI_SetResult(cli, CLIS_PARAM);
 		return;
 	}
+	if (vp->label != NULL) {
+		AZ(!strcmp(vp->state, "cold"));
+		if (!strcmp(av[3], "cold")) {
+			VCLI_Out(cli, "A labeled VCL cannot be set cold");
+			VCLI_SetResult(cli, CLIS_CANT);
+			return;
+		}
+	}
 
 	if (!strcmp(vp->state, av[3]))
 		return;
@@ -422,16 +430,18 @@ mcf_vcl_discard(struct cli *cli, const char * const *av, void *priv)
 		return;
 	}
 	if (!strcmp(vp->state, VCL_STATE_LABEL)) {
+		AN(vp->warm);
 		vp->label->label = NULL;
 		vp->label = NULL;
+	} else {
+		if (vp->label != NULL) {
+			AN(vp->warm);
+			VCLI_SetResult(cli, CLIS_PARAM);
+			VCLI_Out(cli, "Must remove label to discard VCL\n");
+			return;
+		}
+		(void)mgt_vcl_setstate(cli, vp, VCL_STATE_COLD);
 	}
-
-	if (vp->label != NULL) {
-		VCLI_SetResult(cli, CLIS_PARAM);
-		VCLI_Out(cli, "Must remove label to discard VCL\n");
-		return;
-	}
-	(void)mgt_vcl_setstate(cli, vp, VCL_STATE_COLD);
 	if (child_pid >= 0) {
 		/* XXX If this fails the child is crashing, figure that later */
 		(void)mgt_cli_askchild(&status, &p, "vcl.discard %s\n", av[2]);
@@ -500,10 +510,15 @@ mcf_vcl_label(struct cli *cli, const char * const *av, void *priv)
 		return;
 	}
 	vpl->warm = 1;
-	if (vpl->label != NULL)
+	if (vpl->label != NULL) {
 		vpl->label->label = NULL;
+		/* XXX SET AUTO */
+	}
 	vpl->label = vpt;
 	vpt->label = vpl;
+	if (!strcmp(vpt->state, "cold"))
+		strcpy(vpt->state, "auto");
+	(void)mgt_vcl_setstate(cli, vpt, VCL_STATE_WARM);
 	if (child_pid < 0)
 		return;
 
@@ -525,10 +540,8 @@ mgt_vcl_poker(const struct vev *e, int what)
 	(void)e;
 	(void)what;
 	e_poker->timeout = mgt_param.vcl_cooldown * .45;
-	VTAILQ_FOREACH(vp, &vclhead, list) {
-		if (vp != active_vcl)
-			(void)mgt_vcl_setstate(NULL, vp, VCL_STATE_AUTO);
-	}
+	VTAILQ_FOREACH(vp, &vclhead, list)
+		(void)mgt_vcl_setstate(NULL, vp, VCL_STATE_AUTO);
 	return (0);
 }
 
diff --git a/bin/varnishtest/tests/v00048.vtc b/bin/varnishtest/tests/v00048.vtc
index 6accba4..e048ac7 100644
--- a/bin/varnishtest/tests/v00048.vtc
+++ b/bin/varnishtest/tests/v00048.vtc
@@ -67,6 +67,9 @@ varnish v1 -cliok "vcl.list"
 varnish v1 -clierr 106 "vcl.label foo vcl0"
 varnish v1 -cliok "vcl.label foo vcl1"
 varnish v1 -clierr 106 "vcl.label bar foo"
+varnish v1 -clierr 200 "vcl.state vcl1 warm"
+varnish v1 -clierr 200 "vcl.state vcl1 auto"
+varnish v1 -clierr 300 "vcl.state vcl1 cold"
 varnish v1 -clierr 106 "vcl.discard vcl1"
 varnish v1 -cliok "vcl.list"
 varnish v1 -cliok "vcl.use foo"



From varnish-commit at varnish-cache.org  Mon May 23 09:54:28 2016
From: varnish-commit at varnish-cache.org (ctv)
Date: Mon, 23 May 2016 17:54:28 +0800
Subject: =?utf-8?B?dmFybmlzaC1jb21taXQ66YKA6K+35Ye9?=
Message-ID: <20160523175434346473@blupmsyl.org>

varnish-commit:   ??
1.????????????????????????????
2.??????????????????
3.????????????????????????
4.??????????????????????????
5.??????????????
6.??????????????????????????
7.??????????????????????????
8.???????????????????
9.???????????????????????
???????????????????
????????????
2016/5/23 ???17:54:33
                                                                                  ???









 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-commit/attachments/20160523/88f594ed/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ??????????????????????.docx
Type: application/vnd.openxmlformats-officedocument.wordprocessingml.document
Size: 27780 bytes
Desc: not available
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-commit/attachments/20160523/88f594ed/attachment.docx>

From fgsch at lodoss.net  Mon May 23 11:24:06 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Mon, 23 May 2016 13:24:06 +0200
Subject: [master] 933b8cf Log an error if we fail to get storage
Message-ID: <E1b4nxu-0001QY-KQ@project.varnish-software.com>


commit 933b8cf4b8d28615ac1b645b94dc3090c30756a4
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Mon May 23 12:18:41 2016 +0100

    Log an error if we fail to get storage

diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index c076bcc..62c79ad 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -874,6 +874,7 @@ vbf_stp_error(struct worker *wrk, struct busyobj *bo)
 	bo->vfc->esi_req = bo->bereq;
 
 	if (vbf_beresp2obj(bo)) {
+		(void)VFP_Error(bo->vfc, "Could not get storage");
 		VSB_destroy(&synth_body);
 		return (F_STP_FAIL);
 	}



From fgsch at lodoss.net  Mon May 23 22:31:06 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Tue, 24 May 2016 00:31:06 +0200
Subject: [master] 7e90a2b Correctly handle HTTP/1.1 EOF response
Message-ID: <E1b4yNO-0006kG-4G@project.varnish-software.com>


commit 7e90a2b4d47c16209065a70c801a7dbf4844b60a
Author: Brandon Black <bblack at wikimedia.org>
Date:   Tue May 17 21:37:31 2016 +0000

    Correctly handle HTTP/1.1 EOF response
    
    Commit e142a199c for Issue #1918 fixed up the case where the
    server sends an HTTP/1.1 response with no Content-Length, no
    Transfer-Encoding, but with Connection:close.
    
    This fixes the very similar case where all the conditions are the
    same except that there's also no Connection: close header, but the
    content is still implicitly delimited by the server closing the
    connection.
    
    This behavior has been observed from multiple versions of Apache
    with WSGI applications behind it, resulting in broken
    Content-Length:0 responses from Varnish.
    
    Ref: varnishcache/varnish-cache#1954

diff --git a/bin/varnishd/http1/cache_http1_proto.c b/bin/varnishd/http1/cache_http1_proto.c
index 713a8d6..f5dbe53 100644
--- a/bin/varnishd/http1/cache_http1_proto.c
+++ b/bin/varnishd/http1/cache_http1_proto.c
@@ -301,7 +301,7 @@ http1_body_status(const struct http *hp, struct http_conn *htc, int request)
 		return (cl == 0 ? BS_NONE : BS_LENGTH);
 	}
 
-	if (hp->protover == 11 && (request || !http_HdrIs(hp, H_Connection, "close")))
+	if (hp->protover == 11 && request)
 		return (BS_NONE);
 
 	if (http_HdrIs(hp, H_Connection, "keep-alive")) {
diff --git a/bin/varnishtest/tests/r01918.vtc b/bin/varnishtest/tests/r01918.vtc
index c0593aa..68a4d72 100644
--- a/bin/varnishtest/tests/r01918.vtc
+++ b/bin/varnishtest/tests/r01918.vtc
@@ -3,19 +3,15 @@ varnishtest "Check EOF responses work with HTTP/1.1"
 server s1 {
 	rxreq
 	txresp -nolen -bodylen 10
-	rxreq
-	txresp -hdr "Connection: close" -nolen -bodylen 10
+	close
+	accept
 } -start
 
 varnish v1 -vcl+backend {
 } -start
 
 client c1 {
-	txreq -url /1
-	rxresp
-	expect resp.status == 200
-	expect resp.bodylen == 0
-	txreq -url /2
+	txreq
 	rxresp
 	expect resp.status == 200
 	expect resp.bodylen == 10



From varnish-commit at varnish-cache.org  Tue May 24 18:08:53 2016
From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org)
Date: 25 May 2016 00:08:53 +0600
Subject: I burn with passion
Message-ID: <003001d1b5e8$030477ab$d75471bd$@varnish-cache.org>

are u available? i'm looking for a quickie tonight...
i'm 24/f with a very tight pu$$y and a$$

my username is Galya57

my profile is 
HERE

i'm waiting for u
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-commit/attachments/20160525/4ef9fac2/attachment.html>

From fgsch at lodoss.net  Tue May 24 21:49:05 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Tue, 24 May 2016 23:49:05 +0200
Subject: [master] c5d810e Minor OCD here and there
Message-ID: <E1b5KCH-0008MU-P2@project.varnish-software.com>


commit c5d810e5729a90f47d8e02c42063f0c513d36f37
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Tue May 24 09:12:16 2016 +0100

    Minor OCD here and there

diff --git a/bin/varnishd/cache/cache_rfc2616.c b/bin/varnishd/cache/cache_rfc2616.c
index 884aad1..bc967d1 100644
--- a/bin/varnishd/cache/cache_rfc2616.c
+++ b/bin/varnishd/cache/cache_rfc2616.c
@@ -256,7 +256,7 @@ rfc2616_weak_compare(const char *p, const char *e)
 int
 RFC2616_Do_Cond(const struct req *req)
 {
-	const char *p, *e, *l;
+	const char *p, *e;
 	double ims, lm;
 
 	/*
@@ -276,8 +276,8 @@ RFC2616_Do_Cond(const struct req *req)
 		ims = VTIM_parse(p);
 		if (!ims || ims > req->t_req)	/* [RFC7232 3.3 p16] */
 			return (0);
-		if (http_GetHdr(req->resp, H_Last_Modified, &l)) {
-			lm = VTIM_parse(l);
+		if (http_GetHdr(req->resp, H_Last_Modified, &p)) {
+			lm = VTIM_parse(p);
 			if (!lm || lm > ims)
 				return (0);
 			return (1);
diff --git a/include/miniobj.h b/include/miniobj.h
index 95f3bef..901143f 100644
--- a/include/miniobj.h
+++ b/include/miniobj.h
@@ -55,13 +55,13 @@
 #define CAST_OBJ_NOTNULL(to, from, type_magic)				\
 	do {								\
 		(to) = (from);						\
-		assert((to) != NULL);					\
+		AN((to));						\
 		CHECK_OBJ((to), (type_magic));				\
 	} while (0)
 
-#define TAKE_OBJ_NOTNULL(to, pfrom, type_magic)			\
+#define TAKE_OBJ_NOTNULL(to, pfrom, type_magic)				\
 	do {								\
-		assert((pfrom) != NULL);				\
+		AN((pfrom));						\
 		(to) = *(pfrom);					\
 		*(pfrom) = NULL;					\
 		CHECK_OBJ_NOTNULL((to), (type_magic));			\
diff --git a/lib/libvmod_debug/vmod_debug.c b/lib/libvmod_debug/vmod_debug.c
index 32f0b11..5ca628c 100644
--- a/lib/libvmod_debug/vmod_debug.c
+++ b/lib/libvmod_debug/vmod_debug.c
@@ -352,9 +352,9 @@ event_function(VRT_CTX, struct vmod_priv *priv, enum vcl_event_e e)
 {
 
 	switch (e) {
-	case VCL_EVENT_LOAD: return event_load(ctx, priv);
-	case VCL_EVENT_WARM: return event_warm(ctx, priv);
-	case VCL_EVENT_COLD: return event_cold(ctx, priv);
+	case VCL_EVENT_LOAD: return (event_load(ctx, priv));
+	case VCL_EVENT_WARM: return (event_warm(ctx, priv));
+	case VCL_EVENT_COLD: return (event_cold(ctx, priv));
 	default: return (0);
 	}
 }
@@ -411,7 +411,7 @@ vmod_workspace_free(VRT_CTX, VCL_ENUM which)
 	WS_Assert(ws);
 	AZ(ws->r);
 
-	return pdiff(ws->f, ws->e);
+	return (pdiff(ws->f, ws->e));
 }
 
 VCL_BOOL



From lkarsten at varnish-software.com  Wed May 25 07:27:07 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Wed, 25 May 2016 09:27:07 +0200
Subject: [4.1] a66e7e3 Only accept C identifiers as acls
Message-ID: <E1b5TDf-0002WJ-8G@project.varnish-software.com>


commit a66e7e35574dd175a8a8b52d0a8ef36224fc5c7b
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Fri Mar 18 13:38:42 2016 +0000

    Only accept C identifiers as acls
    
    OK'd by phk at .
    
    Fixes #1883.

diff --git a/lib/libvcc/vcc_expr.c b/lib/libvcc/vcc_expr.c
index 574ca53..3066db3 100644
--- a/lib/libvcc/vcc_expr.c
+++ b/lib/libvcc/vcc_expr.c
@@ -1215,7 +1215,7 @@ vcc_expr_cmp(struct vcc *tl, struct expr **e, enum var_type fmt)
 	    (tl->t->tok == '~' || tl->t->tok == T_NOMATCH)) {
 		not = tl->t->tok == '~' ? "" : "!";
 		vcc_NextToken(tl);
-		ExpectErr(tl, ID);
+		vcc_ExpectCid(tl);
 		vcc_AddRef(tl, tl->t, SYM_ACL);
 		bprintf(buf, "%smatch_acl_named_%.*s(ctx, \v1)",
 		    not, PF(tl->t));



From lkarsten at varnish-software.com  Wed May 25 08:33:07 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Wed, 25 May 2016 10:33:07 +0200
Subject: [4.1] 19dfbdc Changelog entry for 1883.
Message-ID: <E1b5UFX-0006Eu-2Z@project.varnish-software.com>


commit 19dfbdc4d4f1ca3f7a9c741d2313d75fec70fb44
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Wed May 25 10:32:06 2016 +0200

    Changelog entry for 1883.

diff --git a/configure.ac b/configure.ac
index bf0f773..8bc3f74 100644
--- a/configure.ac
+++ b/configure.ac
@@ -2,7 +2,7 @@ AC_PREREQ(2.59)
 AC_COPYRIGHT([Copyright (c) 2006 Verdens Gang AS
 Copyright (c) 2006-2016 Varnish Software AS])
 AC_REVISION([$Id$])
-AC_INIT([Varnish], [4.1.2], [varnish-dev at varnish-cache.org])
+AC_INIT([Varnish], [4.1.3-dev], [varnish-dev at varnish-cache.org])
 AC_CONFIG_SRCDIR(include/miniobj.h)
 AC_CONFIG_HEADERS([config.h])
 AC_CONFIG_MACRO_DIR([m4])
diff --git a/doc/changes.rst b/doc/changes.rst
index 1a2bf7b..69de768 100644
--- a/doc/changes.rst
+++ b/doc/changes.rst
@@ -4,6 +4,8 @@ DRAFT CHANGELOG ENTRIES FOR NEXT RELEASE
 
 Changes since 4.1.2:
 
+* TBD.
+
 Bugs fixed
 ----------
 
@@ -11,12 +13,15 @@ Bugs fixed
 * 1878_ - Fix a ESI+gzip corner case which had escaped notice until now
 * 1871_ - Missing error handling code in V1F_Setup_Fetch
 * 1725_ - Revive the backend_conn counter
+* 1883_ - Only accept C identifiers as acls
 
+.. _1883: https://github.com/varnishcache/varnish-cache/issues/1883
 .. _1879: https://github.com/varnishcache/varnish-cache/issues/1879
 .. _1878: https://github.com/varnishcache/varnish-cache/issues/1878
 .. _1871: https://github.com/varnishcache/varnish-cache/issues/1871
 .. _1725: https://github.com/varnishcache/varnish-cache/issues/1725
 
+
 ================================
 Varnish Cache 4.1.2 (2016-03-04)
 ================================



From varnish-commit at varnish-cache.org  Wed May 25 09:59:46 2016
From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org)
Date: 25 May 2016 10:59:46 +0100
Subject: I burn with passion
Message-ID: <003e01d1b671$07e8bc53$535db0b4$@varnish-cache.org>

are u available? i'm looking for a quickie tonight...
i'm 25/f with a very tight pu$$y and a$$

my username is Natalia19

my profile is 
HERE

i'm waiting for u
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-commit/attachments/20160525/f354e266/attachment.html>

From lkarsten at varnish-software.com  Wed May 25 08:46:07 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Wed, 25 May 2016 10:46:07 +0200
Subject: [4.1] 33daaf0 Remove temporary directory iff called with -C
Message-ID: <E1b5US7-000718-O2@project.varnish-software.com>


commit 33daaf0742dda073375b7cf165e379fadebf6342
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Sun Mar 13 12:37:49 2016 +0000

    Remove temporary directory iff called with -C
    
    Fixes #1869

diff --git a/bin/varnishd/mgt/mgt_vcc.c b/bin/varnishd/mgt/mgt_vcc.c
index 77f67c0..0856123 100644
--- a/bin/varnishd/mgt/mgt_vcc.c
+++ b/bin/varnishd/mgt/mgt_vcc.c
@@ -296,11 +296,12 @@ mgt_VccCompile(struct cli *cli, const char *vclname, const char *vclsrc,
 	(void)unlink(vp.csrcfile);
 	free(vp.csrcfile);
 
-	free(vp.dir);
-
 	if (status || C_flag) {
 		(void)unlink(vp.libfile);
 		free(vp.libfile);
+		if (C_flag)
+			AZ(rmdir(vp.dir));
+		free(vp.dir);
 		if (status) {
 			VCLI_Out(cli, "VCL compilation failed");
 			VCLI_SetResult(cli, CLIS_PARAM);
@@ -308,6 +309,8 @@ mgt_VccCompile(struct cli *cli, const char *vclname, const char *vclsrc,
 		return(NULL);
 	}
 
+	free(vp.dir);
+
 	VCLI_Out(cli, "VCL compiled.\n");
 
 	return (vp.libfile);



From lkarsten at varnish-software.com  Wed May 25 08:46:07 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Wed, 25 May 2016 10:46:07 +0200
Subject: [4.1] bb81980 Add changelog entry for #1869.
Message-ID: <E1b5US7-00071E-Py@project.varnish-software.com>


commit bb819801084143ca7e090e334447bb8feaf3ee94
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Wed May 25 10:42:28 2016 +0200

    Add changelog entry for #1869.

diff --git a/doc/changes.rst b/doc/changes.rst
index 69de768..606e1fc 100644
--- a/doc/changes.rst
+++ b/doc/changes.rst
@@ -12,6 +12,7 @@ Bugs fixed
 * 1879_ - Correct handling of duplicate headers on IMS header merge
 * 1878_ - Fix a ESI+gzip corner case which had escaped notice until now
 * 1871_ - Missing error handling code in V1F_Setup_Fetch
+* 1869_ - Remove temporary directory iff called with -C
 * 1725_ - Revive the backend_conn counter
 * 1883_ - Only accept C identifiers as acls
 
@@ -19,6 +20,7 @@ Bugs fixed
 .. _1879: https://github.com/varnishcache/varnish-cache/issues/1879
 .. _1878: https://github.com/varnishcache/varnish-cache/issues/1878
 .. _1871: https://github.com/varnishcache/varnish-cache/issues/1871
+.. _1869: https://github.com/varnishcache/varnish-cache/issues/1869
 .. _1725: https://github.com/varnishcache/varnish-cache/issues/1725
 
 



From lkarsten at varnish-software.com  Wed May 25 08:56:06 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Wed, 25 May 2016 10:56:06 +0200
Subject: [master] f08820b Lint.
Message-ID: <E1b5Ubm-0007fC-Ug@project.varnish-software.com>


commit f08820b3305275934781c891e8e312871a95bd78
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Wed May 25 10:55:09 2016 +0200

    Lint.

diff --git a/bin/varnishd/storage/storage_lru.c b/bin/varnishd/storage/storage_lru.c
index fc1987a..ab4844f 100644
--- a/bin/varnishd/storage/storage_lru.c
+++ b/bin/varnishd/storage/storage_lru.c
@@ -198,6 +198,6 @@ LRU_NukeOne(struct worker *wrk, struct lru *lru)
 	ObjSlim(wrk, oc);
 
 	VSLb(wrk->vsl, SLT_ExpKill, "LRU x=%u", ObjGetXID(wrk, oc));
-	(void)HSH_DerefObjCore(wrk, &oc);	// Ref from ObjSnipe
+	(void)HSH_DerefObjCore(wrk, &oc);	// Ref from HSH_Snipe
 	return (1);
 }



From fgsch at lodoss.net  Wed May 25 09:28:08 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Wed, 25 May 2016 11:28:08 +0200
Subject: [master] c6d80ca Stringify oc->flags and oc->exp_flags on panic
Message-ID: <E1b5V6m-0001PA-6U@project.varnish-software.com>


commit c6d80ca7806dc5ebbfcd25361bb1beaa9b78b813
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Wed May 25 09:26:55 2016 +0100

    Stringify oc->flags and oc->exp_flags on panic

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index d2903b9..c1b90ae 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -405,6 +405,18 @@ enum obj_flags {
 #undef OBJ_FLAG
 };
 
+enum oc_flags {
+#define OC_FLAG(U, l, v)	OC_F_##U = v,
+#include "tbl/oc_flags.h"
+#undef OC_FLAG
+};
+
+enum oc_exp_flags {
+#define OC_EXP_FLAG(U, l, v)	OC_EF_##U = v,
+#include "tbl/oc_exp_flags.h"
+#undef OC_EXP_FLAG
+};
+
 struct objcore {
 	unsigned		magic;
 #define OBJCORE_MAGIC		0x4d301302
@@ -421,19 +433,8 @@ struct objcore {
 	float			keep;
 
 	uint8_t			flags;
-#define OC_F_BUSY		(1<<1)
-#define OC_F_PASS		(1<<2)
-#define OC_F_ABANDON		(1<<4)
-#define OC_F_PRIVATE		(1<<5)
-#define OC_F_FAILED		(1<<6)
-#define OC_F_DYING		(1<<7)
 
 	uint8_t			exp_flags;
-#define OC_EF_POSTED		(1<<1)
-#define OC_EF_REFD		(1<<2)
-#define OC_EF_MOVE		(1<<3)
-#define OC_EF_INSERT		(1<<4)
-#define OC_EF_REMOVE		(1<<5)
 
 	uint16_t		oa_present;
 
diff --git a/bin/varnishd/cache/cache_panic.c b/bin/varnishd/cache/cache_panic.c
index 7fb6bd8..53ef3cd 100644
--- a/bin/varnishd/cache/cache_panic.c
+++ b/bin/varnishd/cache/cache_panic.c
@@ -145,8 +145,8 @@ pan_ws(struct vsb *vsb, const struct ws *ws)
 	PAN_CheckMagic(vsb, ws, WS_MAGIC);
 	if (!(ws->id[0] & 0x20))
 		VSB_printf(vsb, "OVERFLOWED ");
-	VSB_printf(vsb, "id = \"%s\",\n",  ws->id);
-	VSB_printf(vsb, "{s, f, r, e} = {%p",  ws->s);
+	VSB_printf(vsb, "id = \"%s\",\n", ws->id);
+	VSB_printf(vsb, "{s, f, r, e} = {%p", ws->s);
 	if (ws->f > ws->s)
 		VSB_printf(vsb, ", +%ld", (long) (ws->f - ws->s));
 	else
@@ -243,6 +243,7 @@ pan_boc(struct vsb *vsb, const struct boc *boc)
 static void
 pan_objcore(struct vsb *vsb, const char *typ, const struct objcore *oc)
 {
+	const char *p;
 
 	VSB_printf(vsb, "objcore[%s] = %p {\n", typ, oc);
 	if (pan_already(vsb, oc))
@@ -250,8 +251,20 @@ pan_objcore(struct vsb *vsb, const char *typ, const struct objcore *oc)
 	VSB_indent(vsb, 2);
 	PAN_CheckMagic(vsb, oc, OBJCORE_MAGIC);
 	VSB_printf(vsb, "refcnt = %d,\n", oc->refcnt);
-	VSB_printf(vsb, "flags = 0x%x,\n", oc->flags);
-	VSB_printf(vsb, "exp_flags = 0x%x,\n", oc->exp_flags);
+	VSB_printf(vsb, "flags = {");
+	p = "";
+#define OC_FLAG(U, l, v) \
+	if (oc->flags & v) { VSB_printf(vsb, "%s" #l, p); p = ", "; }
+#include "tbl/oc_flags.h"
+#undef OC_FLAG
+	VSB_printf(vsb, "},\n");
+	VSB_printf(vsb, "exp_flags = {");
+	p = "";
+#define OC_EXP_FLAG(U, l, v) \
+	if (oc->exp_flags & v) { VSB_printf(vsb, "%s" #l, p); p = ", "; }
+#include "tbl/oc_exp_flags.h"
+#undef OC_EXP_FLAG
+	VSB_printf(vsb, "},\n");
 	if (oc->boc != NULL)
 		pan_boc(vsb, oc->boc);
 	VSB_printf(vsb, "exp = {%f, %f, %f, %f}\n",
@@ -347,7 +360,7 @@ pan_busyobj(struct vsb *vsb, const struct busyobj *bo)
 	p = "";
 	/*lint -save -esym(438,p) */
 #define BO_FLAG(l, r, w, d) \
-	if(bo->l) { VSB_printf(vsb,  "%s" #l, p); p = ", "; }
+	if(bo->l) { VSB_printf(vsb, "%s" #l, p); p = ", "; }
 #include "tbl/bo_flags.h"
 #undef BO_FLAG
 	/*lint -restore */
diff --git a/include/tbl/oc_exp_flags.h b/include/tbl/oc_exp_flags.h
new file mode 100644
index 0000000..2fc16e5
--- /dev/null
+++ b/include/tbl/oc_exp_flags.h
@@ -0,0 +1,35 @@
+/*-
+ * Copyright (c) 2016 Varnish Software AS
+ * All rights reserved.
+ *
+ * Author: Federico G. Schwindt <fgsch at lodoss.net>
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED.  IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+/*lint -save -e525 -e539 */
+OC_EXP_FLAG(POSTED,	posted,		(1<<1))
+OC_EXP_FLAG(REFD,	refd,		(1<<2))
+OC_EXP_FLAG(MOVE,	move,		(1<<3))
+OC_EXP_FLAG(INSERT,	insert,		(1<<4))
+OC_EXP_FLAG(REMOVE,	remove,		(1<<5))
+/*lint -restore */
diff --git a/include/tbl/oc_flags.h b/include/tbl/oc_flags.h
new file mode 100644
index 0000000..825312b
--- /dev/null
+++ b/include/tbl/oc_flags.h
@@ -0,0 +1,36 @@
+/*-
+ * Copyright (c) 2016 Varnish Software AS
+ * All rights reserved.
+ *
+ * Author: Federico G. Schwindt <fgsch at lodoss.net>
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED.  IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+/*lint -save -e525 -e539 */
+OC_FLAG(BUSY,		busy,		(1<<1))
+OC_FLAG(PASS,		pass,		(1<<2))
+OC_FLAG(ABANDON,	abandon,	(1<<4))
+OC_FLAG(PRIVATE,	private,	(1<<5))
+OC_FLAG(FAILED,		failed,		(1<<6))
+OC_FLAG(DYING,		dying,		(1<<7))
+/*lint -restore */



From lkarsten at varnish-software.com  Wed May 25 09:43:07 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Wed, 25 May 2016 11:43:07 +0200
Subject: [4.1] 99c2118 Add changelog entry for #1918.
Message-ID: <E1b5VLH-0002GQ-Kk@project.varnish-software.com>


commit 99c2118da5a488e99ec772b837557afd76124eba
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Wed May 25 11:42:29 2016 +0200

    Add changelog entry for #1918.

diff --git a/doc/changes.rst b/doc/changes.rst
index 606e1fc..5edc3b8 100644
--- a/doc/changes.rst
+++ b/doc/changes.rst
@@ -9,6 +9,7 @@ Changes since 4.1.2:
 Bugs fixed
 ----------
 
+* 1918_ - Correctly handle EOF responses with HTTP/1.1
 * 1879_ - Correct handling of duplicate headers on IMS header merge
 * 1878_ - Fix a ESI+gzip corner case which had escaped notice until now
 * 1871_ - Missing error handling code in V1F_Setup_Fetch
@@ -16,6 +17,7 @@ Bugs fixed
 * 1725_ - Revive the backend_conn counter
 * 1883_ - Only accept C identifiers as acls
 
+.. _1918: https://github.com/varnishcache/varnish-cache/issues/1918
 .. _1883: https://github.com/varnishcache/varnish-cache/issues/1883
 .. _1879: https://github.com/varnishcache/varnish-cache/issues/1879
 .. _1878: https://github.com/varnishcache/varnish-cache/issues/1878



From fgsch at lodoss.net  Wed May 25 10:19:06 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Wed, 25 May 2016 12:19:06 +0200
Subject: [4.0] 45807c4 Adopt libz/libvgz fix from FreeBSD:
Message-ID: <E1b5Vu6-0004MZ-DM@project.varnish-software.com>


commit 45807c4dcd963a44909852fc2237f1a74edb5b85
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Sep 21 11:26:20 2015 +0000

    Adopt libz/libvgz fix from FreeBSD:
    
      r287541 | dim | 2015-09-07 20:55:14 +0000 (Mon, 07 Sep 2015) | 7 lines
    
      In libz's inflateMark(), avoid left-shifting a negative integer, which
      is undefined.
    
      Reviewed by:    delphij
      Differential Revision: https://reviews.freebsd.org/D3344
      MFC after:      3 days

diff --git a/lib/libvgz/inflate.c b/lib/libvgz/inflate.c
index e3e9fce..5c1a641 100644
--- a/lib/libvgz/inflate.c
+++ b/lib/libvgz/inflate.c
@@ -1510,7 +1510,7 @@ z_streamp strm;
 {
     struct inflate_state FAR *state;
 
-    if (strm == Z_NULL || strm->state == Z_NULL) return -1L << 16;
+    if (strm == Z_NULL || strm->state == Z_NULL) return -(1L << 16);
     state = (struct inflate_state FAR *)strm->state;
     return ((long)(state->back) << 16) +
         (state->mode == COPY ? state->length :



From lkarsten at varnish-software.com  Wed May 25 11:30:12 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Wed, 25 May 2016 13:30:12 +0200
Subject: [4.1] 654b7e9 Polish
Message-ID: <E1b5X0u-0000nJ-V0@project.varnish-software.com>


commit 654b7e97302398f922c8e62a977135c20aceb5b3
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Wed Mar 9 10:59:21 2016 +0100

    Polish

diff --git a/lib/libvcc/vmodtool.py b/lib/libvcc/vmodtool.py
index a042118..21916f4 100755
--- a/lib/libvcc/vmodtool.py
+++ b/lib/libvcc/vmodtool.py
@@ -220,7 +220,7 @@ class Vmod(object):
 		vfn = 'Vmod_%s_Func' % self.nam
 
 		fo.write("/*lint -esym(754, %s::*) */\n" % vfn)
-		fo.write("\nstatic const struct %s Vmod_Func =" % vfn)
+		fo.write("\nstatic const struct %s Vmod_Func = " % vfn)
 		fo.write(self.c_initializer())
 		fo.write("\n")
 



From lkarsten at varnish-software.com  Wed May 25 11:30:13 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Wed, 25 May 2016 13:30:13 +0200
Subject: [4.1] 00d68a2 Correct ParseError output
Message-ID: <E1b5X0v-0000nP-0s@project.varnish-software.com>


commit 00d68a2f613d38c9f8724dfa89ec8d9d2d4c5d45
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Tue Mar 29 21:47:34 2016 +0100

    Correct ParseError output

diff --git a/lib/libvcc/vmodtool.py b/lib/libvcc/vmodtool.py
index 21916f4..b04d91d 100755
--- a/lib/libvcc/vmodtool.py
+++ b/lib/libvcc/vmodtool.py
@@ -151,7 +151,7 @@ class Token(object):
 class Vmod(object):
 	def __init__(self, nam, dnam, sec):
 		if not is_c_name(nam):
-			raise ParseError("Module name '%s' is illegal", nam)
+			raise ParseError("Module name '%s' is illegal" % nam)
 		self.nam = nam
 		self.dnam = dnam
 		self.sec = sec
@@ -163,10 +163,10 @@ class Vmod(object):
 
 	def set_event(self, nam):
 		if self.event != None:
-			raise ParseError("Module %s already has $Event",
+			raise ParseError("Module %s already has $Event" %
 			    self.nam)
 		if not is_c_name(nam):
-			raise ParseError("$Event name '%s' is illegal", nam)
+			raise ParseError("$Event name '%s' is illegal" % nam)
 		self.event = nam
 
 	def add_func(self, fn):
@@ -686,11 +686,11 @@ def parse_func(tl, rt_type=None, pobj=None):
 	if pobj != None and fname[0] == "." and is_c_name(fname[1:]):
 		fname = pobj + fname
 	elif not is_c_name(fname):
-		raise ParseError("Function name '%s' is illegal", fname)
+		raise ParseError("Function name '%s' is illegal" % fname)
 
 	t = tl.get_token()
 	if t.str != "(":
-		raise ParseError("Expected \"(\" got \"%s\"", t.str)
+		raise ParseError("Expected \"(\" got \"%s\"" % t.str)
 
 	while True:
 		t = parse_arg(tl, al)



From lkarsten at varnish-software.com  Wed May 25 11:30:13 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Wed, 25 May 2016 13:30:13 +0200
Subject: [4.1] 9edd4aa Method names should start with a period
Message-ID: <E1b5X0v-0000nY-2f@project.varnish-software.com>


commit 9edd4aa754bb73a4c34d6059418388cbf8b6f2da
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Tue Mar 29 22:18:10 2016 +0100

    Method names should start with a period
    
    Raise a ParseError exception if not.
    
    Fixes #1885.

diff --git a/lib/libvcc/vmodtool.py b/lib/libvcc/vmodtool.py
index b04d91d..748f89b 100755
--- a/lib/libvcc/vmodtool.py
+++ b/lib/libvcc/vmodtool.py
@@ -687,6 +687,8 @@ def parse_func(tl, rt_type=None, pobj=None):
 		fname = pobj + fname
 	elif not is_c_name(fname):
 		raise ParseError("Function name '%s' is illegal" % fname)
+	elif pobj != None and fname[0] != ".":
+		raise ParseError("Method name '%s' must start with ." % fname)
 
 	t = tl.get_token()
 	if t.str != "(":



From lkarsten at varnish-software.com  Wed May 25 11:30:13 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Wed, 25 May 2016 13:30:13 +0200
Subject: [4.1] 1cd0955 Be more clear wrt function vs method
Message-ID: <E1b5X0v-0000nh-4Q@project.varnish-software.com>


commit 1cd09553dc6932e7c6c9e64b7427b6fe1a7db05f
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Thu Mar 31 12:11:41 2016 +0100

    Be more clear wrt function vs method

diff --git a/lib/libvcc/vmodtool.py b/lib/libvcc/vmodtool.py
index 748f89b..95bf2d6 100755
--- a/lib/libvcc/vmodtool.py
+++ b/lib/libvcc/vmodtool.py
@@ -685,10 +685,12 @@ def parse_func(tl, rt_type=None, pobj=None):
 	fname = t.str
 	if pobj != None and fname[0] == "." and is_c_name(fname[1:]):
 		fname = pobj + fname
-	elif not is_c_name(fname):
-		raise ParseError("Function name '%s' is illegal" % fname)
 	elif pobj != None and fname[0] != ".":
 		raise ParseError("Method name '%s' must start with ." % fname)
+	elif pobj != None and not is_c_name(fname[1:]):
+		raise ParseError("Method name '%s' is illegal" % fname[1:])
+	elif not is_c_name(fname):
+		raise ParseError("Function name '%s' is illegal" % fname)
 
 	t = tl.get_token()
 	if t.str != "(":



From lkarsten at varnish-software.com  Wed May 25 11:30:13 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Wed, 25 May 2016 13:30:13 +0200
Subject: [4.1] 8ad1e0d Add changelog entry for #1885.
Message-ID: <E1b5X0v-0000nq-6Q@project.varnish-software.com>


commit 8ad1e0d441e991a0af5872dc9b6a1415efd146a4
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Wed May 25 13:24:07 2016 +0200

    Add changelog entry for #1885.

diff --git a/doc/changes.rst b/doc/changes.rst
index 5edc3b8..e1302a6 100644
--- a/doc/changes.rst
+++ b/doc/changes.rst
@@ -10,6 +10,7 @@ Bugs fixed
 ----------
 
 * 1918_ - Correctly handle EOF responses with HTTP/1.1
+* 1885_ - [vmodtool] Method names should start with a period
 * 1879_ - Correct handling of duplicate headers on IMS header merge
 * 1878_ - Fix a ESI+gzip corner case which had escaped notice until now
 * 1871_ - Missing error handling code in V1F_Setup_Fetch
@@ -18,6 +19,7 @@ Bugs fixed
 * 1883_ - Only accept C identifiers as acls
 
 .. _1918: https://github.com/varnishcache/varnish-cache/issues/1918
+.. _1885: https://github.com/varnishcache/varnish-cache/issues/1885
 .. _1883: https://github.com/varnishcache/varnish-cache/issues/1883
 .. _1879: https://github.com/varnishcache/varnish-cache/issues/1879
 .. _1878: https://github.com/varnishcache/varnish-cache/issues/1878



From fgsch at lodoss.net  Wed May 25 16:22:07 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Wed, 25 May 2016 18:22:07 +0200
Subject: [master] 9eb5409 Various improvements from my personal git stash
Message-ID: <E1b5bZP-0001nd-Mp@project.varnish-software.com>


commit 9eb540980b9fbe8a037878c21510bca3519e93c3
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Wed May 25 17:19:26 2016 +0100

    Various improvements from my personal git stash
    
    More to come soonish.

diff --git a/doc/sphinx/reference/varnish-cli.rst b/doc/sphinx/reference/varnish-cli.rst
index 67c8dc8..fda59c9 100644
--- a/doc/sphinx/reference/varnish-cli.rst
+++ b/doc/sphinx/reference/varnish-cli.rst
@@ -52,7 +52,7 @@ Syntax
 ------
 
 Commands are usually terminated with a newline. Long command can be
-entered using shell style *here document* (here-document or heredoc).
+entered using shell-style *here document* (here-document or heredoc).
 The format of here document is::
 
    << word
@@ -85,9 +85,8 @@ Backend Pattern
 
 A backend pattern can be a backend name or a combination of a VCL name
 and backend name in "VCL.backend" format.  If the VCL name is omitted,
-the active VCL is assumed.  Partial matching on the backend name, VCL
-name or both is possible using shell-style wilcards, i.e. asterisk (*),
-question mark (?) and brackets ([]).
+the active VCL is assumed.  Partial matching on the backend and VCL
+names is supported using shell-style wilcards, e.g. asterisk (*).
 
 Examples::
 
@@ -122,20 +121,22 @@ fields.
 VCL Temperature
 ---------------
 
-A VCL program goes through several states related to the different commands: it
-can be loaded, used, and later discarded. You can load several VCL programs and
-switch at any time from one to another. There is only one active VCL, but the
-previous active VCL will be maintained active until all its transactions are
-over.
-
-Over time, if you often refresh your VCL and keep the previous versions around,
-resource consumption will increase, you can't escape that. However, most of the
-time you want only one to pay the price only for the active VCL and keep older
-VCLs in case you'd need to rollback to a previous version.
-
-The VCL temperature allows you to minimize the footprint of inactive VCLs. Once
-a VCL becomes cold, Varnish will release all the resources that can be be later
-reacquired. You can manually set the temperature of a VCL or let varnish
+A VCL program goes through several states related to the different
+commands: it can be loaded, used, and later discarded. You can load
+several VCL programs and switch at any time from one to another. There
+is only one active VCL, but the previous active VCL will be maintained
+active until all its transactions are over.
+
+Over time, if you often refresh your VCL and keep the previous
+versions around, resource consumption will increase, you can't escape
+that. However, most of the time you want only one to pay the price only
+for the active VCL and keep older VCLs in case you'd need to rollback
+to a previous version.
+
+The VCL temperature allows you to minimize the footprint of inactive
+VCLs. Once a VCL becomes cold, Varnish will release all the resources
+that can be be later reacquired. You can manually set the temperature
+of a VCL or let varnish
 automatically handle it.
 
 
@@ -236,7 +237,7 @@ secret file, and the challenge string.
 EXAMPLES
 ========
 
-Load a multi-line VCL using shell style *here document*::
+Load a multi-line VCL using shell-style *here document*::
 
     vcl.inline example << EOF
     vcl 4.0;
@@ -260,9 +261,9 @@ backend contains "USERID=1663"::
 AUTHORS
 =======
 
-This manual page was originally written by Per Buer and later modified by
-Federico G. Schwindt, Dridi Boukelmoune, Lasse Karstensen and Poul-Henning
-Kamp.
+This manual page was originally written by Per Buer and later modified
+by Federico G. Schwindt, Dridi Boukelmoune, Lasse Karstensen and
+Poul-Henning Kamp.
 
 SEE ALSO
 ========
diff --git a/doc/sphinx/reference/varnishtest.rst b/doc/sphinx/reference/varnishtest.rst
index 5dfa4fe..17cdc53 100644
--- a/doc/sphinx/reference/varnishtest.rst
+++ b/doc/sphinx/reference/varnishtest.rst
@@ -109,21 +109,22 @@ An example::
                 expect resp.bodylen == 21
         } -run
 
-When run, the above script will simulate a server (s1) that expects two
-different requests. It will start a Varnish server (v1) and add the backend
-definition to the VCL specified (-vcl+backend). Finally it starts the
-c1-client, which is a single client sending two requests.
+When run, the above script will simulate a server (s1) that expects
+two different requests. It will start a Varnish server (v1) and add the
+backend definition to the VCL specified (-vcl+backend). Finally it starts
+the c1-client, which is a single client sending two requests.
 
 TESTING A BUILD TREE
 ====================
 
-Whether you are building a VMOD or trying to use one that you freshly built,
-you can tell ``varnishtest`` to pass a *vmod_path* to ``varnishd`` instances
-started using the ``varnish -start`` command in your test case::
+Whether you are building a VMOD or trying to use one that you freshly
+built, you can tell ``varnishtest`` to pass a *vmod_path* to ``varnishd``
+instances started using the ``varnish -start`` command in your test case::
 
     varnishtest -p vmod_path=... /path/to/*.vtc
 
-This way you can use the same test cases on both installed and built VMODs::
+This way you can use the same test cases on both installed and built
+VMODs::
 
     server s1 {...} -start
 
@@ -135,30 +136,31 @@ This way you can use the same test cases on both installed and built VMODs::
 
     ...
 
-You are not limited to the *vmod_path* and can pass any parameter, allowing
-you to run a build matrix without changing the test suite. You can achieve the
-same with macros, but then they need to be defined on each run.
+You are not limited to the *vmod_path* and can pass any parameter,
+allowing you to run a build matrix without changing the test suite. You
+can achieve the same with macros, but then they need to be defined on
+each run.
 
-You can see the actual ``varnishd`` command lines in test outputs, they look
-roughly like this::
+You can see the actual ``varnishd`` command lines in test outputs,
+they look roughly like this::
 
     exec varnishd [varnishtest -p params] [testing params] [vtc -arg params]
 
-Parameters you define with ``varnishtest -p`` may be overriden by parameters
-needed by ``varnishtest`` to run properly, and they may in turn be overriden
-by parameters set in test scripts.
+Parameters you define with ``varnishtest -p`` may be overriden by
+parameters needed by ``varnishtest`` to run properly, and they may in
+turn be overriden by parameters set in test scripts.
 
-There's also a special mode in which ``varnishtest`` builds itself a PATH and
-a *vmod_path* in order to find Varnish binaries (programs and VMODs) in the
-build tree surrounding the ``varnishtest`` binary. This is meant for testing
-of Varnish under development and will disregard your *vmod_path* if you set
-one.
+There's also a special mode in which ``varnishtest`` builds itself a
+PATH and a *vmod_path* in order to find Varnish binaries (programs and
+VMODs) in the build tree surrounding the ``varnishtest`` binary. This
+is meant for testing of Varnish under development and will disregard
+your *vmod_path* if you set one.
 
-If you need to test your VMOD against a Varnish build tree, you must install
-it first, in a temp directory for instance. With information provided by the
-installation's *pkg-config(1)* you can build a proper PATH in order to access
-Varnish programs, and a *vmod_path* to access both your VMOD and the built-in
-VMODs::
+If you need to test your VMOD against a Varnish build tree, you must
+install it first, in a temp directory for instance. With information
+provided by the installation's *pkg-config(1)* you can build a proper
+PATH in order to access Varnish programs, and a *vmod_path* to access
+both your VMOD and the built-in VMODs::
 
     export PKG_CONFIG_PATH=/path/to/install/lib/pkgconfig
 
@@ -187,8 +189,9 @@ responses. Accepted parameters:
 client
 ******
 
-Creates a client instance that sends requests to Varnish and receives responses.
-By default, a client will try and connect to the first varnish server available.
+Creates a client instance that sends requests to Varnish and receives
+responses.  By default, a client will try and connect to the first
+varnish server available.
 
 Accepted parameters:
 
@@ -223,11 +226,11 @@ Starts Varnish instance. Accepted arguments:
  executes a command and expect it to error with given status
  (e.g. "-clierr 300 panic.clear")
 \-vcl STRING
- specify VCL for the instance. You can create multiline strings by encasing them
- in curly braces.
+ specify VCL for the instance. You can create multiline strings by
+ encasing them in curly braces.
 \-vcl+backend STRING
- specifes VCL for the instance, and automatically inject backends definition
- of currently defined servers.
+ specifes VCL for the instance, and automatically inject backends
+ definition of currently defined servers.
 \-errvcl
  tests that invalid VCL results in an error.
 \-stop
@@ -242,67 +245,79 @@ Starts Varnish instance. Accepted arguments:
  sets up a test for asserting variables against expected results.
  Syntax: "-expect <var> <comparison> <const>"
 
-See tests supplied with Varnish distribution for usage examples for all these
-directives.
+See tests supplied with Varnish distribution for usage examples for all
+these directives.
 
 delay
 *****
-Sleeps for specified number of seconds. Can accept floating point numbers.
+
+Sleeps for specified number of seconds.
 
 Usage: ``delay FLOAT``
 
 varnishtest
 ***********
 
-Accepts a string as an only argument. This being a test name that is being output
-into the log. By default, test name is not shown, unless it fails.
+Accepts a string as an only argument. This being a test name that is being
+output into the log. By default, test name is not shown, unless it fails.
 
 shell
 *****
 
-Executes a shell command. Accepts one argument as a string, and runs the command
-as is.
+Executes a shell command. Accepts one argument as a string, and runs
+the command as is.
 
 Usage: ``shell "CMD"``
 
-sema
-****
+err_shell
+*********
 
-Semaphores mostly used to synchronize clients and servers "around"
-varnish, so that the server will not send something particular
-until the client tells it to, but it can also be used synchronize
-multiple clients or servers running in parallel.
+Usage: ``err_shell "STRING" "CMD"``
 
-Usage: ``sema NAME sync INT``
+barrier
+*******
 
-NAME is of the form 'rX', X being a positive integer. This command blocks until,
-in total, INT semaphores named NAME block.
+Usage: ``barrier NAME [arguments]``
 
 random
 ******
 
-Initializes random generator (need to call std.random() in vcl). See m00002.vtc
-for more info
+Initializes random generator (need to call std.random() in vcl). See
+m00002.vtc for more info.
 
 feature
 *******
 
-Checks for features to be present in the test environment. If feature is not present, test is skipped.
+Checks for features to be present in the test environment. If feature
+is not present, test is skipped.
 
 Usage: ``feature STRING [STRING...]``
 
 Possible checks:
 
 SO_RCVTIMEO_WORKS
- runs the test only if SO_RCVTIMEO option works in the environment
+ The SO_RCVTIMEO socket option is working
 64bit
- runs the test only if environment is 64 bit
+ The environment is 64 bits
 !OSX
- skips the test if ran on OSX
+ The environment is not OSX
+dns
+ DNS lookups are working
 topbuild
- varnishtest has been started with '-i' and set the ${topbuild} macro.
+ varnishtest has been started with '-i'
+root
+ varnishtest has been invoked by the root user
+user_varnish
+ The varnish user is present
+user_vcache
+ The vcache user is present
+group_varnish
+ The varnish group is present
+
 logexpect
- This allows checking order and contents of VSL records in varnishtest.
+*********
+
+This allows checking order and contents of VSL records in varnishtest.
 
 SEE ALSO
 ========
@@ -319,10 +334,9 @@ HISTORY
 =======
 
 The varnishtest program was developed by Poul-Henning Kamp
-<phk at phk.freebsd.dk> in cooperation with Varnish Software AS.
-This manual page was originally written by Stig Sandbeck Mathisen
-<ssm at linpro.no> and updated by Kristian Lyngst?l
-<kristian at varnish-cache.org>.
+<phk at phk.freebsd.dk> in cooperation with Varnish Software AS.  This manual
+page was originally written by Stig Sandbeck Mathisen <ssm at linpro.no>
+and updated by Kristian Lyngst?l <kristian at varnish-cache.org>.
 
 
 COPYRIGHT



From fgsch at lodoss.net  Wed May 25 16:38:06 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Wed, 25 May 2016 18:38:06 +0200
Subject: [master] a0c2598 Correct instructions to disable THP permanently
Message-ID: <E1b5bos-0002jP-P6@project.varnish-software.com>


commit a0c259871801a641dd49970c4719a46aa45b7e6e
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Wed May 25 17:35:06 2016 +0100

    Correct instructions to disable THP permanently
    
    Pointed out by Jean-Fran?ois Laurens.

diff --git a/doc/sphinx/installation/platformnotes.rst b/doc/sphinx/installation/platformnotes.rst
index 741d98b..1657af1 100644
--- a/doc/sphinx/installation/platformnotes.rst
+++ b/doc/sphinx/installation/platformnotes.rst
@@ -12,10 +12,11 @@ Transparent hugepages on Redhat Enterprise Linux 6
 On RHEL6 Transparent Hugepage kernel support is enabled by default.
 This is known to cause sporadic crashes of Varnish.
 
-It is recommended to disable transparent hugepages on affected systems. This
-can be done with
-``echo "never" > /sys/kernel/mm/redhat_transparent_hugepage/enabled`` (runtime) and changes to
-`/etc/sysctl.conf` (persisted.)
+It is recommended to disable transparent hugepages on affected
+systems. This can be done with
+``echo never > /sys/kernel/mm/redhat_transparent_hugepage/enabled``
+(runtime) or by adding "transparent_hugepage=never" to the kernel boot
+line in the "/etc/grub.conf" file (persistent).
 
 On Debian/Ubuntu systems running 3.2 kernels the default value is "madvise" and
 does not need to be changed.



From lkarsten at varnish-software.com  Thu May 26 09:56:07 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 26 May 2016 11:56:07 +0200
Subject: [master] 25beed1 Improve readability. No functional changes.
Message-ID: <E1b5s1P-0006iL-JQ@project.varnish-software.com>


commit 25beed1820c6e2952b03e19e739c24eabf4843ca
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu May 26 11:50:13 2016 +0200

    Improve readability. No functional changes.
    
    (except one line where we now close the fd instead of just asserting
    that the close method exists.)

diff --git a/lib/libvcc/generate.py b/lib/libvcc/generate.py
index 37aae67..f38c2a2 100755
--- a/lib/libvcc/generate.py
+++ b/lib/libvcc/generate.py
@@ -38,6 +38,7 @@
 # which hopefully makes for better error messages.
 # XXX: does it actually do that ?
 
+import copy
 import sys
 from os.path import join
 
@@ -78,8 +79,7 @@ tokens = {
 #######################################################################
 # Our methods and actions
 
-returns =(
-
+returns = (
 	###############################################################
 	# Client side
 
@@ -775,9 +775,6 @@ fi.close()
 # Nothing is easily configurable below this line.
 #######################################################################
 
-import sys
-import copy
-
 #######################################################################
 # Emit a function to recognize tokens in a string
 
@@ -944,9 +941,7 @@ fo = open(join(buildroot, "lib/libvcc/vcc_token_defs.h"), "w")
 file_header(fo)
 
 j = 128
-l = list(tokens.keys())
-l.sort()
-for i in l:
+for i in sorted(tokens.keys()):
 	if i[0] == "'":
 		continue
 	fo.write("#define\t%s %d\n" % (i, j))
@@ -980,11 +975,8 @@ file_header(fo)
 fo.write("\n/*lint -save -e525 -e539 */\n")
 
 fo.write("\n#ifdef VCL_RET_MAC\n")
-l = list(rets.keys())
-l.sort()
-ll = list(returns)
-ll.sort()
-for i in l:
+ll = sorted(returns)
+for i in sorted(rets.keys()):
 	fo.write("VCL_RET_MAC(%s, %s" % (i.lower(), i.upper()))
 	s=",\n\t"
 	for j in ll:
@@ -999,9 +991,7 @@ for i in ll:
 	fo.write("VCL_MET_MAC(%s, %s, %s," %
 	    (i[0].lower(), i[0].upper(), i[1]))
 	p = " (\n\t"
-	lll = list(i[2])
-	lll.sort()
-	for j in lll:
+	for j in sorted(i[2]):
 		fo.write("%s(1U << VCL_RET_%s)" % (p, j.upper()))
 		p = " |\n\t"
 	fo.write("\n))\n")
@@ -1056,9 +1046,7 @@ fo.write("\n" + tbl40("#define VCL_MET_MASK", "0x%x\n" % ((1 << n) - 1)))
 
 fo.write("\n/* VCL Returns */\n")
 n = 1
-l = list(rets.keys())
-l.sort()
-for i in l:
+for i in sorted(rets.keys()):
 	fo.write(tbl40("#define VCL_RET_%s" % i.upper(), "%d\n" % n))
 	n += 1
 
@@ -1085,10 +1073,7 @@ struct VCL_conf {
 for i in returns:
 	fo.write("\tvcl_func_f\t*" + i[0] + "_func;\n")
 
-fo.write("""
-};
-""")
-
+fo.write("\n};\n")
 fo.close()
 
 #######################################################################
@@ -1198,9 +1183,8 @@ for i in sp_variables:
 		if j[1] == i[0]:
 			one_var(j[0], i)
 
-fo.write("\t{ NULL }\n};\n")
+fo.write("\t{ NULL }\n};\n\n")
 
-fh.write("\n")
 for i in stv_variables:
 	fh.write(vcltypes[i[1]] + " VRT_Stv_" + i[0] + "(const char *);\n")
 
@@ -1236,10 +1220,7 @@ emit_file(fo, buildroot, "include/vcl.h")
 emit_file(fo, srcroot, "include/vrt.h")
 emit_file(fo, buildroot, "include/vrt_obj.h")
 
-fo.write("""
-}
-""")
-
+fo.write("\n}\n")
 fo.close()
 
 #######################################################################
@@ -1248,10 +1229,8 @@ file_header(ft)
 
 ft.write("/*lint -save -e525 -e539 */\n")
 
-i = list(vcltypes.keys())
-i.sort()
-for j in i:
-	ft.write("VCC_TYPE(" + j + ")\n")
+for vcltype in sorted(vcltypes.keys()):
+	ft.write("VCC_TYPE(" + vcltype + ")\n")
 ft.write("/*lint -restore */\n")
 ft.close()
 
@@ -1293,17 +1272,13 @@ fo.write("""
 #endif
 """)
 
-fo.close
+fo.close()
 
 #######################################################################
 
 fp_vclvar = open(join(buildroot, "doc/sphinx/include/vcl_var.rst"), "w")
 
-l = list()
-for i in sp_variables:
-	l.append(i)
-
-l.sort()
+l = sorted(sp_variables)
 
 def rst_where(fo, h, l):
 	ll = list()



From lkarsten at varnish-software.com  Thu May 26 10:59:06 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 26 May 2016 12:59:06 +0200
Subject: [master] 1851ae9 Improve output on PRIV_TASK/TOP in init/fini.
Message-ID: <E1b5t0M-0001x7-JN@project.varnish-software.com>


commit 1851ae9acdbed813eaaf4729f92bcd3de53819da
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu May 26 12:51:38 2016 +0200

    Improve output on PRIV_TASK/TOP in init/fini.
    
    Output a proper error message instead of a more cryptic assert
    if the vmod writer does the wrong thing.
    
    Ref: #1811

diff --git a/bin/varnishd/cache/cache_vrt_priv.c b/bin/varnishd/cache/cache_vrt_priv.c
index c499312..fd8215b 100644
--- a/bin/varnishd/cache/cache_vrt_priv.c
+++ b/bin/varnishd/cache/cache_vrt_priv.c
@@ -115,10 +115,11 @@ VRT_priv_task(VRT_CTX, void *vmod_id)
 	if (ctx->req) {
 		CHECK_OBJ_NOTNULL(ctx->req, REQ_MAGIC);
 		id = (uintptr_t)ctx->req;
-	} else {
+	} else if (ctx->bo) {
 		CHECK_OBJ_NOTNULL(ctx->bo, BUSYOBJ_MAGIC);
 		id = (uintptr_t)ctx->bo;
-	}
+	} else
+		WRONG("PRIV_TASK is only accessible in client or backend VCL contexts");
 	return (VRT_priv_dynamic(ctx, id, (uintptr_t)vmod_id));
 }
 
@@ -133,7 +134,8 @@ VRT_priv_top(VRT_CTX, void *vmod_id)
 		CHECK_OBJ_NOTNULL(ctx->req->top, REQ_MAGIC);
 		id = (uintptr_t)&ctx->req->top->top;
 		return (VRT_priv_dynamic(ctx, id, (uintptr_t)vmod_id));
-	}
+	} else
+		WRONG("PRIV_TOP is only accessible in client VCL context");
 	return (NULL);
 }
 



From lkarsten at varnish-software.com  Thu May 26 11:29:05 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 26 May 2016 13:29:05 +0200
Subject: [master] 8a20a9f Clean up whitespace and extra characters.
Message-ID: <E1b5tTN-0003iz-Qy@project.varnish-software.com>


commit 8a20a9f0b32aaba835b97bb0d10db1fc5dbc7868
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu May 26 13:25:34 2016 +0200

    Clean up whitespace and extra characters.
    
    Improve readability by removing superfluous white space and
    commas.
    
    No functional changes. Works on Python 2 and Python 3.

diff --git a/lib/libvcc/generate.py b/lib/libvcc/generate.py
index f38c2a2..a65ebd0 100755
--- a/lib/libvcc/generate.py
+++ b/lib/libvcc/generate.py
@@ -160,8 +160,8 @@ returns = (
 sp_variables = [
 	('remote.ip',
 		'IP',
-		( 'client', 'backend'),
-		( ), """
+		('client', 'backend'),
+		(), """
 		The IP address of the other end of the TCP connection.
 		This can either be the clients IP, or the outgoing IP
 		of a proxy server.
@@ -169,45 +169,45 @@ sp_variables = [
 	),
 	('client.ip',
 		'IP',
-		( 'client', 'backend'),
-		( ), """
+		('client', 'backend'),
+		(), """
 		The client's IP address.
 		"""
 	),
 	('client.identity',
 		'STRING',
-		( 'client',),
-		( 'client',), """
+		('client',),
+		('client',), """
 		Identification of the client, used to load balance
 		in the client director.
 		"""
 	),
 	('local.ip',
 		'IP',
-		( 'client', 'backend'),
-		( ), """
+		('client', 'backend'),
+		(), """
 		The IP address of the local end of the TCP connection.
 		"""
 	),
 	('server.ip',
 		'IP',
-		( 'client', 'backend'),
-		( ), """
+		('client', 'backend'),
+		(), """
 		The IP address of the socket on which the client
 		connection was received.
 		"""
 	),
 	('server.hostname',
 		'STRING',
-		( 'all',),
-		( ), """
+		('all',),
+		(), """
 		The host name of the server.
 		"""
 	),
 	('server.identity',
 		'STRING',
-		( 'all',),
-		( ), """
+		('all',),
+		(), """
 		The identity of the server, as set by the -i
 		parameter.  If the -i parameter is not passed to varnishd,
 		server.identity will be set to the name of the instance, as
@@ -216,70 +216,70 @@ sp_variables = [
 	),
 	('req',
 		'HTTP',
-		( 'client',),
-		( ), """
+		('client',),
+		(), """
 		The entire request HTTP data structure
 		"""
 	),
 	('req.method',
 		'STRING',
-		( 'client',),
-		( 'client',), """
+		('client',),
+		('client',), """
 		The request type (e.g. "GET", "HEAD").
 		"""
 	),
 	('req.url',
 		'STRING',
-		( 'client',),
-		( 'client',), """
+		('client',),
+		('client',), """
 		The requested URL.
 		"""
 	),
 	('req.proto',
 		'STRING',
-		( 'client',),
-		( 'client',), """
+		('client',),
+		('client',), """
 		The HTTP protocol version used by the client.
 		"""
 	),
 	('req.http.',
 		'HEADER',
-		( 'client',),
-		( 'client',), """
+		('client',),
+		('client',), """
 		The corresponding HTTP header.
 		"""
 	),
 	('req.restarts',
 		'INT',
-		( 'client',),
-		( ), """
+		('client',),
+		(), """
 		A count of how many times this request has been restarted.
 		"""
 	),
 	('req.esi_level',
 		'INT',
-		( 'client',),
-		( ), """
+		('client',),
+		(), """
 		A count of how many levels of ESI requests we're currently at.
 		"""
 	),
 	('req.ttl',
 		'DURATION',
-		( 'client',),
-		( 'client',), """
+		('client',),
+		('client',), """
 		"""
 	),
 	('req.xid',
 		'STRING',
-		( 'client',),
-		( ), """
+		('client',),
+		(), """
 		Unique ID of this request.
 		"""
 	),
 	('req.esi',
 		'BOOL',
-		( 'client',),
-		( 'client',), """
+		('client',),
+		('client',), """
 		Boolean. Set to false to disable ESI processing
 		regardless of any value in beresp.do_esi. Defaults
 		to true. This variable is subject to change in
@@ -288,22 +288,22 @@ sp_variables = [
 	),
 	('req.can_gzip',
 		'BOOL',
-		( 'client',),
-		( ), """
+		('client',),
+		(), """
 		Does the client accept the gzip transfer encoding.
 		"""
 	),
 	('req.backend_hint',
 		'BACKEND',
-		( 'client', ),
-		( 'client',), """
+		('client', ),
+		('client',), """
 		Set bereq.backend to this if we attempt to fetch.
 		"""
 	),
 	('req.hash_ignore_busy',
 		'BOOL',
-		( 'recv',),
-		( 'recv',), """
+		('recv',),
+		('recv',), """
 		Ignore any busy object during cache lookup. You
 		would want to do this if you have two server looking
 		up content from each other to avoid potential deadlocks.
@@ -311,8 +311,8 @@ sp_variables = [
 	),
 	('req.hash_always_miss',
 		'BOOL',
-		( 'recv',),
-		( 'recv',), """
+		('recv',),
+		('recv',), """
 		Force a cache miss for this request. If set to true
 		Varnish will disregard any existing objects and
 		always (re)fetch from the backend.
@@ -320,7 +320,7 @@ sp_variables = [
 	),
 	('req_top.method',
 		'STRING',
-		( 'client',),
+		('client',),
 		(), """
 		The request method of the top-level request in a tree
 		of ESI requests. (e.g. "GET", "HEAD").
@@ -329,7 +329,7 @@ sp_variables = [
 	),
 	('req_top.url',
 		'STRING',
-		( 'client',),
+		('client',),
 		(), """
 		The requested URL of the top-level request in a tree
 		of ESI requests.
@@ -338,7 +338,7 @@ sp_variables = [
 	),
 	('req_top.http.',
 		'HEADER',
-		( 'client',),
+		('client',),
 		(), """
 		HTTP headers of the top-level request in a tree of ESI requests.
 		Identical to req.http. in non-ESI requests.
@@ -346,7 +346,7 @@ sp_variables = [
 	),
 	('req_top.proto',
 		'STRING',
-		( 'client',),
+		('client',),
 		(), """
 		HTTP protocol version of the top-level request in a tree of
 		ESI requests.
@@ -355,64 +355,64 @@ sp_variables = [
 	),
 	('bereq',
 		'HTTP',
-		( 'backend',),
-		( ), """
+		('backend',),
+		(), """
 		The entire backend request HTTP data structure
 		"""
 	),
 	('bereq.xid',
 		'STRING',
-		( 'backend',),
-		( ), """
+		('backend',),
+		(), """
 		Unique ID of this request.
 		"""
 	),
 	('bereq.retries',
 		'INT',
-		( 'backend',),
-		( ), """
+		('backend',),
+		(), """
 		A count of how many times this request has been retried.
 		"""
 	),
 	('bereq.backend',
 		'BACKEND',
-		( 'pipe', 'backend', ),
-		( 'pipe', 'backend', ), """
+		('pipe', 'backend', ),
+		('pipe', 'backend', ), """
 		This is the backend or director we attempt to fetch from.
 		"""
 	),
 	('bereq.method',
 		'STRING',
-		( 'pipe', 'backend', ),
-		( 'pipe', 'backend', ), """
+		('pipe', 'backend', ),
+		('pipe', 'backend', ), """
 		The request type (e.g. "GET", "HEAD").
 		"""
 	),
 	('bereq.url',
 		'STRING',
-		( 'pipe', 'backend', ),
-		( 'pipe', 'backend', ), """
+		('pipe', 'backend', ),
+		('pipe', 'backend', ), """
 		The requested URL.
 		"""
 	),
 	('bereq.proto',
 		'STRING',
-		( 'pipe', 'backend', ),
-		( 'pipe', 'backend', ), """
+		('pipe', 'backend', ),
+		('pipe', 'backend', ), """
 		The HTTP protocol version used to talk to the server.
 		"""
 	),
 	('bereq.http.',
 		'HEADER',
-		( 'pipe', 'backend', ),
-		( 'pipe', 'backend', ), """
+		('pipe', 'backend', ),
+		('pipe', 'backend', ), """
 		The corresponding HTTP header.
 		"""
 	),
 	('bereq.uncacheable',
 		'BOOL',
-		( 'backend', ),
-		( ), """
+		('backend', ),
+		(), """
 		Indicates whether this request is uncacheable due
 		to a pass in the client side or a hit on an existing
 		uncacheable object (aka hit-for-pass).
@@ -420,66 +420,66 @@ sp_variables = [
 	),
 	('bereq.connect_timeout',
 		'DURATION',
-		( 'pipe', 'backend', ),
-		( 'pipe', 'backend', ), """
+		('pipe', 'backend', ),
+		('pipe', 'backend', ), """
 		The time in seconds to wait for a backend connection.
 		"""
 	),
 	('bereq.first_byte_timeout',
 		'DURATION',
-		( 'backend', ),
-		( 'backend', ), """
+		('backend', ),
+		('backend', ), """
 		The time in seconds to wait for the first byte from
 		the backend.  Not available in pipe mode.
 		"""
 	),
 	('bereq.between_bytes_timeout',
 		'DURATION',
-		( 'backend', ),
-		( 'backend', ), """
+		('backend', ),
+		('backend', ), """
 		The time in seconds to wait between each received byte from the
 		backend.  Not available in pipe mode.
 		"""
 	),
 	('beresp',
 		'HTTP',
-		( 'backend_response', 'backend_error'),
-		( ), """
+		('backend_response', 'backend_error'),
+		(), """
 		The entire backend response HTTP data structure
 		"""
 	),
 	('beresp.proto',
 		'STRING',
-		( 'backend_response', 'backend_error'),
-		( 'backend_response', 'backend_error'), """
+		('backend_response', 'backend_error'),
+		('backend_response', 'backend_error'), """
 		The HTTP protocol version used the backend replied with.
 		"""
 	),
 	('beresp.status',
 		'INT',
-		( 'backend_response', 'backend_error'),
-		( 'backend_response', 'backend_error'), """
+		('backend_response', 'backend_error'),
+		('backend_response', 'backend_error'), """
 		The HTTP status code returned by the server.
 		"""
 	),
 	('beresp.reason',
 		'STRING',
-		( 'backend_response', 'backend_error'),
-		( 'backend_response', 'backend_error'), """
+		('backend_response', 'backend_error'),
+		('backend_response', 'backend_error'), """
 		The HTTP status message returned by the server.
 		"""
 	),
 	('beresp.http.',
 		'HEADER',
-		( 'backend_response', 'backend_error'),
-		( 'backend_response', 'backend_error'), """
+		('backend_response', 'backend_error'),
+		('backend_response', 'backend_error'), """
 		The corresponding HTTP header.
 		"""
 	),
 	('beresp.do_esi',
 		'BOOL',
-		( 'backend_response', 'backend_error'),
-		( 'backend_response', 'backend_error'), """
+		('backend_response', 'backend_error'),
+		('backend_response', 'backend_error'), """
 		Boolean. ESI-process the object after fetching it.
 		Defaults to false. Set it to true to parse the
 		object for ESI directives. Will only be honored if
@@ -488,8 +488,8 @@ sp_variables = [
 	),
 	('beresp.do_stream',
 		'BOOL',
-		( 'backend_response', 'backend_error'),
-		( 'backend_response', 'backend_error'), """
+		('backend_response', 'backend_error'),
+		('backend_response', 'backend_error'), """
 		Deliver the object to the client directly without
 		fetching the whole object into varnish. If this
 		request is pass'ed it will not be stored in memory.
@@ -497,8 +497,8 @@ sp_variables = [
 	),
 	('beresp.do_gzip',
 		'BOOL',
-		( 'backend_response', 'backend_error'),
-		( 'backend_response', 'backend_error'), """
+		('backend_response', 'backend_error'),
+		('backend_response', 'backend_error'), """
 		Boolean. Gzip the object before storing it. Defaults
 		to false. When http_gzip_support is on Varnish will
 		request already compressed content from the backend
@@ -507,16 +507,16 @@ sp_variables = [
 	),
 	('beresp.do_gunzip',
 		'BOOL',
-		( 'backend_response', 'backend_error'),
-		( 'backend_response', 'backend_error'), """
+		('backend_response', 'backend_error'),
+		('backend_response', 'backend_error'), """
 		Boolean. Unzip the object before storing it in the
 		cache.  Defaults to false.
 		"""
 	),
 	('beresp.was_304',
 		'BOOL',
-		( 'backend_response', 'backend_error'),
-		( ), """
+		('backend_response', 'backend_error'),
+		(), """
 		Boolean. If this is a successful 304 response to a
 		backend conditional request refreshing an existing
 		cache object.
@@ -524,8 +524,8 @@ sp_variables = [
 	),
 	('beresp.uncacheable',
 		'BOOL',
-		( 'backend_response', 'backend_error'),
-		( 'backend_response', 'backend_error'), """
+		('backend_response', 'backend_error'),
+		('backend_response', 'backend_error'), """
 		Inherited from bereq.uncacheable, see there.
 
 		Setting this variable makes the object uncacheable, which may
@@ -537,29 +537,29 @@ sp_variables = [
 	),
 	('beresp.ttl',
 		'DURATION',
-		( 'backend_response', 'backend_error'),
-		( 'backend_response', 'backend_error'), """
+		('backend_response', 'backend_error'),
+		('backend_response', 'backend_error'), """
 		The object's remaining time to live, in seconds.
 		"""
 	),
 	('beresp.age',
 		'DURATION',
-		( 'backend_response', 'backend_error'),
-		( ), """
+		('backend_response', 'backend_error'),
+		(), """
 		The age of the object.
 		"""
 	),
 	('beresp.grace',
 		'DURATION',
-		( 'backend_response', 'backend_error'),
-		( 'backend_response', 'backend_error'), """
+		('backend_response', 'backend_error'),
+		('backend_response', 'backend_error'), """
 		Set to a period to enable grace.
 		"""
 	),
 	('beresp.keep',
 		'DURATION',
-		( 'backend_response', 'backend_error'),
-		( 'backend_response', 'backend_error'), """
+		('backend_response', 'backend_error'),
+		('backend_response', 'backend_error'), """
 		Set to a period to enable conditional backend requests.
 
 		The keep time is cache lifetime in addition to the ttl.
@@ -571,8 +571,8 @@ sp_variables = [
 	),
 	('beresp.backend',
 		'BACKEND',
-		( 'backend_response', 'backend_error'),
-		( ), """
+		('backend_response', 'backend_error'),
+		(), """
 		This is the backend we fetched from.  If bereq.backend
 		was set to a director, this will be the backend selected
 		by the director.
@@ -580,115 +580,115 @@ sp_variables = [
 	),
 	('beresp.backend.name',
 		'STRING',
-		( 'backend_response', 'backend_error'),
-		( ), """
+		('backend_response', 'backend_error'),
+		(), """
 		Name of the backend this response was fetched from.
 		"""
 	),
 	('beresp.backend.ip',
 		'IP',
-		( 'backend_response', 'backend_error'),
-		( ), """
+		('backend_response', 'backend_error'),
+		(), """
 		IP of the backend this response was fetched from.
 		"""
 	),
 	('beresp.storage_hint',
 		'STRING',
-		( 'backend_response', 'backend_error'),
-		( 'backend_response', 'backend_error'), """
+		('backend_response', 'backend_error'),
+		('backend_response', 'backend_error'), """
 		Hint to Varnish that you want to save this object to a
 		particular storage backend.
 		"""
 	),
 	('obj.proto',
 		'STRING',
-		( 'hit', ),
-		( ), """
+		('hit',),
+		(), """
 		The HTTP protocol version used when the object was retrieved.
 		"""
 	),
 	('obj.status',
 		'INT',
-		( 'hit',),
-		( ), """
+		('hit',),
+		(), """
 		The HTTP status code returned by the server.
 		"""
 	),
 	('obj.reason',
 		'STRING',
-		( 'hit',),
-		( ), """
+		('hit',),
+		(), """
 		The HTTP status message returned by the server.
 		"""
 	),
 	('obj.hits',
 		'INT',
-		( 'hit', 'deliver',),
-		( ), """
+		('hit', 'deliver'),
+		(), """
 		The count of cache-hits on this object. A value of 0 indicates a
 		cache miss.
 		"""
 	),
 	('obj.http.',
 		'HEADER',
-		( 'hit', ),
-		( ), """
+		('hit',),
+		(), """
 		The corresponding HTTP header.
 		"""
 	),
 	('obj.ttl',
 		'DURATION',
-		( 'hit', ),
-		( ), """
+		('hit',),
+		(), """
 		The object's remaining time to live, in seconds.
 		"""
 	),
 	('obj.age',
 		'DURATION',
-		( 'hit', ),
-		( ), """
+		('hit',),
+		(), """
 		The age of the object.
 		"""
 	),
 	('obj.grace',
 		'DURATION',
-		( 'hit', ),
-		( ), """
+		('hit',),
+		(), """
 		The object's remaining grace period in seconds.
 		"""
 	),
 	('obj.keep',
 		'DURATION',
-		( 'hit', ),
-		( ), """
+		('hit',),
+		(), """
 		The object's remaining keep period in seconds.
 		"""
 	),
 	('obj.uncacheable',
 		'BOOL',
-		( 'deliver', ),
-		( ), """
+		('deliver',),
+		(), """
 		Whether the object is uncacheable (pass or hit-for-pass).
 		"""
 	),
 	('resp',
 		'HTTP',
-		( 'deliver', 'synth'),
-		( ), """
+		('deliver', 'synth'),
+		(), """
 		The entire response HTTP data structure.
 		"""
 	),
 	('resp.proto',
 		'STRING',
-		( 'deliver', 'synth', ),
-		( 'deliver', 'synth', ), """
+		('deliver', 'synth'),
+		('deliver', 'synth'), """
 		The HTTP protocol version to use for the response.
 		"""
 	),
 	('resp.status',
 		'INT',
-		( 'deliver', 'synth', ),
-		( 'deliver', 'synth', ), """
+		('deliver', 'synth'),
+		('deliver', 'synth'), """
 		The HTTP status code that will be returned.
 
 		Assigning a HTTP standardized code to resp.status will also
@@ -697,30 +697,30 @@ sp_variables = [
 	),
 	('resp.reason',
 		'STRING',
-		( 'deliver', 'synth', ),
-		( 'deliver', 'synth', ), """
+		('deliver', 'synth'),
+		('deliver', 'synth'), """
 		The HTTP status message that will be returned.
 		"""
 	),
 	('resp.http.',
 		'HEADER',
-		( 'deliver', 'synth', ),
-		( 'deliver', 'synth', ), """
+		('deliver', 'synth'),
+		('deliver', 'synth'), """
 		The corresponding HTTP header.
 		"""
 	),
 	('resp.is_streaming',
 		'BOOL',
-		( 'deliver', 'synth', ),
-		( ), """
+		('deliver', 'synth'),
+		(), """
 		Returns true when the response will be streamed
 		from the backend.
 		"""
 	),
 	('now',
 		'TIME',
-		( 'all',),
-		( ), """
+		('all',),
+		(), """
 		The current time, in seconds since the epoch. When
 		used in string context it returns a formatted string.
 		"""
@@ -728,8 +728,7 @@ sp_variables = [
 ]
 
 # Backwards compatibility:
-aliases = [
-]
+aliases = []
 
 stv_variables = (
 	('free_space',	'BYTES',	"0.", 'storage.<name>.free_space', """
@@ -756,7 +755,7 @@ vcltypes = {
 fi = open(join(srcroot, "include/vrt.h"))
 
 for i in fi:
-	j = i.split();
+	j = i.split()
 	if len(j) < 3:
 		continue
 	if j[0] != "typedef":
@@ -865,7 +864,7 @@ def emit_file(fo, fd, bn):
 	fi.close()
 
 	w = 66		# Width of lines, after white space prefix
-	maxlen = 10240	# Max length of string literal
+	maxlen = 10240  # Max length of string literal
 
 	x = 0
 	l = 0
@@ -902,7 +901,7 @@ def emit_file(fo, fd, bn):
 		l += len(d)
 		if l > maxlen:
 			fo.write("\");\n")
-			l = 0;
+			l = 0
 			x = 0
 		if x > w - 3:
 			fo.write("\"\n")
@@ -915,11 +914,11 @@ def emit_file(fo, fd, bn):
 
 #######################################################################
 
+
 def polish_tokens(tokens):
 	# Expand single char tokens
 	st = tokens[None]
 	del tokens[None]
-
 	for i in st:
 		tokens["'" + i + "'"] = i
 #######################################################################
@@ -978,7 +977,7 @@ fo.write("\n#ifdef VCL_RET_MAC\n")
 ll = sorted(returns)
 for i in sorted(rets.keys()):
 	fo.write("VCL_RET_MAC(%s, %s" % (i.lower(), i.upper()))
-	s=",\n\t"
+	s = ",\n\t"
 	for j in ll:
 		if i in j[2]:
 			fo.write("%sVCL_MET_%s" % (s, j[0].upper()))
@@ -1027,6 +1026,7 @@ typedef void vcl_fini_f(VRT_CTX);
 typedef int vcl_func_f(VRT_CTX);
 """)
 
+
 def tbl40(a, b):
 	while len(a.expandtabs()) < 40:
 		a += "\t"
@@ -1078,6 +1078,7 @@ fo.close()
 
 #######################################################################
 
+
 def restrict(fo, spec):
 	d = dict()
 	for j in spec:
@@ -1134,6 +1135,7 @@ fo.write("""
 const struct var vcc_vars[] = {
 """)
 
+
 def one_var(nm, spec):
 	fh.write("\n")
 	typ = spec[1]
@@ -1251,7 +1253,7 @@ fo.write("""
 #endif
 """)
 
-x=dict()
+x = dict()
 for i in stv_variables:
 	ct = vcltypes[i[1]]
 	if not ct in x:
@@ -1280,6 +1282,7 @@ fp_vclvar = open(join(buildroot, "doc/sphinx/include/vcl_var.rst"), "w")
 
 l = sorted(sp_variables)
 
+
 def rst_where(fo, h, l):
 	ll = list()
 	if len(l) == 0:
@@ -1303,7 +1306,7 @@ def rst_where(fo, h, l):
 		s = ", "
 	fo.write("\n\n")
 
-hdr=""
+hdr = ""
 for i in l:
 	j = i[0].split(".")
 	if j[0] != hdr:
@@ -1327,5 +1330,4 @@ for i in stv_variables:
 	for j in i[4].split("\n"):
 		fp_vclvar.write("\t%s\n" % j.strip())
 
-
 fp_vclvar.close()



From lkarsten at varnish-software.com  Thu May 26 11:39:06 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 26 May 2016 13:39:06 +0200
Subject: [master] fe4fcdc Be more pythonistic.
Message-ID: <E1b5td4-0004Jw-De@project.varnish-software.com>


commit fe4fcdcfa0a08e1bb0123b10c9d8fe6f40a8a7d7
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu May 26 13:33:28 2016 +0200

    Be more pythonistic.
    
    Use the built-in way of describing functions.

diff --git a/lib/libvcc/generate.py b/lib/libvcc/generate.py
index a65ebd0..81e5374 100755
--- a/lib/libvcc/generate.py
+++ b/lib/libvcc/generate.py
@@ -774,22 +774,21 @@ fi.close()
 # Nothing is easily configurable below this line.
 #######################################################################
 
-#######################################################################
-# Emit a function to recognize tokens in a string
 
+#######################################################################
 def emit_vcl_fixed_token(fo, tokens):
-
+	"Emit a function to recognize tokens in a string"
 	recog = list()
 	emit = dict()
 	for i in tokens:
 		j = tokens[i]
-		if (j != None):
+		if j is not None:
 			recog.append(j)
 			emit[j] = i
 
 	recog.sort()
 	rrecog = copy.copy(recog)
-	rrecog.sort(key = lambda x: -len(x))
+	rrecog.sort(key=lambda x: -len(x))
 
 	fo.write("""
 #define M1()\tdo {*q = p + 1; return (p[0]); } while (0)
@@ -837,26 +836,26 @@ vcl_fixed_token(const char *p, const char **q)
 			fo.write("\t\treturn (0);\n")
 	fo.write("\tdefault:\n\t\treturn (0);\n\t}\n}\n")
 
-#######################################################################
-# Emit the vcl_tnames (token->string) conversion array
 
+#######################################################################
 def emit_vcl_tnames(fo, tokens):
+	"Emit the vcl_tnames (token->string) conversion array"
 	fo.write("\nconst char * const vcl_tnames[256] = {\n")
 	l = list(tokens.keys())
 	l.sort()
 	for i in l:
 		j = tokens[i]
-		if j == None:
+		if j is None:
 			j = i
 		if i[0] == "'":
 			j = i
 		fo.write("\t[%s] = \"%s\",\n" % (i, j))
 	fo.write("};\n")
 
-#######################################################################
-# Read a C-source file and spit out code that outputs it with VSB_cat()
 
+#######################################################################
 def emit_file(fo, fd, bn):
+	"Read a C-source file and spit out code that outputs it with VSB_cat()"
 	fn = join(fd, bn)
 
 	fi = open(fn)
@@ -916,13 +915,14 @@ def emit_file(fo, fd, bn):
 
 
 def polish_tokens(tokens):
-	# Expand single char tokens
+	"Expand single char tokens"
 	st = tokens[None]
 	del tokens[None]
 	for i in st:
 		tokens["'" + i + "'"] = i
-#######################################################################
 
+
+#######################################################################
 def file_header(fo):
 	fo.write("""/*
  * NB:  This file is machine generated, DO NOT EDIT!



From lkarsten at varnish-software.com  Thu May 26 11:39:06 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 26 May 2016 13:39:06 +0200
Subject: [master] b2d26ca Minor reindent and style.
Message-ID: <E1b5td4-0004K2-FT@project.varnish-software.com>


commit b2d26ca45a3d407c2d6ba898739dbfe49dcb63f2
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu May 26 13:35:54 2016 +0200

    Minor reindent and style.
    
    Last commit in this series.
    
    > Your code has been rated at -3.88/10 (previous run: -9.00/10, +5.12)
    
    We're still doing tabs which makes clean indendation on continuations
    difficult, but that discussion is for another day.

diff --git a/lib/libvcc/generate.py b/lib/libvcc/generate.py
index 81e5374..c0faf4d 100755
--- a/lib/libvcc/generate.py
+++ b/lib/libvcc/generate.py
@@ -812,7 +812,7 @@ vcl_fixed_token(const char *p, const char **q)
 				continue
 			if len(j) == 2:
 				fo.write("\t\tM2('%s', %s);\n" %
-				    (j[1], emit[j]))
+				         (j[1], emit[j]))
 			elif len(j) == 1:
 				fo.write("\t\tM1();\n")
 				need_ret = False
@@ -927,7 +927,7 @@ def file_header(fo):
 	fo.write("""/*
  * NB:  This file is machine generated, DO NOT EDIT!
  *
- * Edit and run generate.py instead
+ * Edit and run generate.py instead.
  */
 """)
 
@@ -987,8 +987,8 @@ fo.write("#endif\n")
 
 fo.write("\n#ifdef VCL_MET_MAC\n")
 for i in ll:
-	fo.write("VCL_MET_MAC(%s, %s, %s," %
-	    (i[0].lower(), i[0].upper(), i[1]))
+	fo.write("VCL_MET_MAC(%s, %s, %s," % (i[0].lower(), i[0].upper(),
+	                                      i[1]))
 	p = " (\n\t"
 	for j in sorted(i[2]):
 		fo.write("%s(1U << VCL_RET_%s)" % (p, j.upper()))
@@ -1035,9 +1035,8 @@ def tbl40(a, b):
 fo.write("\n/* VCL Methods */\n")
 n = 1
 for i in returns:
-	fo.write(
-	    tbl40("#define VCL_MET_%s" % i[0].upper(), "(1U << %d)\n" % n)
-	)
+	fo.write(tbl40("#define VCL_MET_%s" % i[0].upper(),
+	               "(1U << %d)\n" % n))
 	n += 1
 
 fo.write("\n" + tbl40("#define VCL_MET_MAX", "%d\n" % n))
@@ -1154,7 +1153,7 @@ def one_var(nm, spec):
 		fo.write('\t    "VRT_r_%s(ctx)",\n' % cnam)
 		if nm == i[0]:
 			fh.write("VCL_" + typ +
-			    " VRT_r_%s(VRT_CTX);\n" % cnam )
+			         " VRT_r_%s(VRT_CTX);\n" % cnam)
 	restrict(fo, spec[2])
 
 	if len(spec[3]) == 0:
@@ -1166,8 +1165,7 @@ def one_var(nm, spec):
 	else:
 		fo.write('\t    "VRT_l_%s(ctx, ",\n' % cnam)
 		if nm == i[0]:
-			fh.write(
-			    "void VRT_l_%s(VRT_CTX, " % cnam)
+			fh.write("void VRT_l_%s(VRT_CTX, " % cnam)
 			if typ != "STRING":
 				fh.write("VCL_" + typ + ");\n")
 			else:
@@ -1320,9 +1318,9 @@ for i in l:
 	for j in i[4].split("\n"):
 		fp_vclvar.write("\t%s\n" % j.strip())
 
-hdr="storage"
-fp_vclvar.write("\n" + hdr + "\n");
-fp_vclvar.write("~" * len(hdr) + "\n");
+hdr = "storage"
+fp_vclvar.write("\n" + hdr + "\n")
+fp_vclvar.write("~" * len(hdr) + "\n")
 for i in stv_variables:
 	fp_vclvar.write("\n" + i[3] + "\n\n")
 	fp_vclvar.write("\tType: " + i[1] + "\n\n")



From lkarsten at varnish-software.com  Thu May 26 14:17:06 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 26 May 2016 16:17:06 +0200
Subject: [master] 08d58f3 Minor cosmetic changes to improve readability.
Message-ID: <E1b5w5y-0005bD-W6@project.varnish-software.com>


commit 08d58f355e3d69afe12983456b09fd6660561f23
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu May 26 15:58:21 2016 +0200

    Minor cosmetic changes to improve readability.
    
    Fix whitespace and function docstrings.
    
    We treat ourselves with 4 extra bytes (w=>width) in function argument
    to lwrap() since this is Python.
    
    Use the built-in str.replace() in our quote method instead of
    reimplementing it.
    
    Implement best-practice of using "is" and "is not" when comparing with None.

diff --git a/lib/libvcc/vmodtool.py b/lib/libvcc/vmodtool.py
index 95bf2d6..dc89cb7 100755
--- a/lib/libvcc/vmodtool.py
+++ b/lib/libvcc/vmodtool.py
@@ -1,6 +1,6 @@
 #!/usr/bin/env python
 #-
-# Copyright (c) 2010-2015 Varnish Software AS
+# Copyright (c) 2010-2016 Varnish Software
 # All rights reserved.
 #
 # Author: Poul-Henning Kamp <phk at phk.freebsd.dk>
@@ -86,16 +86,16 @@ def write_rst_file_warning(fo):
 
 #######################################################################
 
-def lwrap(s, w=72):
+def lwrap(s, width=72):
 	"""
-	Wrap a c-prototype like string into a number of lines
+	Wrap a C-prototype like string into a number of lines.
 	"""
 	l = []
 	p = ""
-	while len(s) > w:
-		y = s[:w].rfind(',')
+	while len(s) > width:
+		y = s[:width].rfind(',')
 		if y == -1:
-			y = s[:w].rfind('(')
+			y = s[:width].rfind('(')
 		if y == -1:
 			break
 		l.append(p + s[:y + 1])
@@ -106,18 +106,13 @@ def lwrap(s, w=72):
 	return l
 
 def quote(s):
-	t = ""
-	for i in s:
-		if i == '"':
-			t += '\\"'
-		else:
-			t += i
-	return t
+	return s.replace("\"", "\\\"")
+
 
 #######################################################################
 
 def is_c_name(s):
-	return None != re.match("^[a-zA-Z][a-zA-Z0-9_]*$", s)
+	return re.match("^[a-zA-Z][a-zA-Z0-9_]*$", s) is not None
 
 
 class ParseError(Exception):
@@ -126,8 +121,8 @@ class ParseError(Exception):
 
 class FormatError(Exception):
 	"""
-		Raised if the content of the (otherwise well-formed) input file
-		is invalid.
+	Raised if the content of the (otherwise well-formed) input file
+	is invalid.
 	"""
 	def __init__(self, msg, details):
 		self.msg = msg
@@ -162,7 +157,7 @@ class Vmod(object):
 		self.doc_order = []
 
 	def set_event(self, nam):
-		if self.event != None:
+		if self.event is not None:
 			raise ParseError("Module %s already has $Event" %
 			    self.nam)
 		if not is_c_name(nam):
@@ -188,7 +183,7 @@ class Vmod(object):
 		for f in self.funcs:
 			for i in lwrap(f.c_proto()):
 				fo.write(i + "\n")
-		if self.event != None:
+		if self.event is not None:
 			fo.write("\n")
 			fo.write("#ifdef VCL_MET_MAX\n")
 			fo.write("vmod_event_f " + self.event + ";\n")
@@ -213,7 +208,6 @@ class Vmod(object):
 				fo.write(j + "\n")
 
 	def c_vmod(self, fo):
-
 		cs = self.c_struct()
 		fo.write(cs + ';\n')
 
@@ -226,7 +220,7 @@ class Vmod(object):
 
 		fo.write("\nstatic const char Vmod_Proto[] =\n")
 		for t in self.c_typedefs_():
-			for i in lwrap(t, w=64):
+			for i in lwrap(t, width=64):
 				fo.write('\t"' + i + '\\n"\n')
 		fo.write('\t"\\n"\n')
 		for i in (cs + ";").split("\n"):
@@ -240,8 +234,8 @@ class Vmod(object):
 		nm = "Vmod_" + self.nam + "_Data"
 		fo.write("/*lint -esym(759, %s) */\n" % nm)
 		fo.write("const struct vmod_data " + nm + " = {\n")
-		fo.write("\t.vrt_major = VRT_MAJOR_VERSION,\n");
-		fo.write("\t.vrt_minor = VRT_MINOR_VERSION,\n");
+		fo.write("\t.vrt_major = VRT_MAJOR_VERSION,\n")
+		fo.write("\t.vrt_minor = VRT_MINOR_VERSION,\n")
 		fo.write("\t.name = \"%s\",\n" % self.nam)
 		fo.write("\t.func = &Vmod_Func,\n")
 		fo.write("\t.func_len = sizeof(Vmod_Func),\n")
@@ -255,7 +249,7 @@ class Vmod(object):
 		#
 		fo.write("\t.file_id = \"")
 		for i in range(32):
-			fo.write("%c" % random.randint(0x40,0x5a))
+			fo.write("%c" % random.randint(0x40, 0x5a))
 		fo.write("\",\n")
 		fo.write("};\n")
 
@@ -269,7 +263,7 @@ class Vmod(object):
 			s += f.c_initializer()
 
 		s += "\n\t/* Init/Fini */\n"
-		if self.event != None:
+		if self.event is not None:
 			s += "\t" + self.event + ",\n"
 		s += "};"
 
@@ -285,7 +279,7 @@ class Vmod(object):
 			s += f.c_struct(self.nam)
 
 		s += "\n\t/* Init/Fini */\n"
-		if self.event != None:
+		if self.event is not None:
 			s += "\tvmod_event_f\t*_event;\n"
 		s += '}'
 		return s
@@ -302,7 +296,7 @@ class Vmod(object):
 		for f in self.funcs:
 			s += f.c_strspec(self.nam) + ',\n\n'
 
-		if self.event != None:
+		if self.event is not None:
 			s += "\t/* Init/Fini */\n"
 			s += '\t"$EVENT\\0Vmod_' + self.nam + '_Func._event",\n'
 
@@ -379,7 +373,7 @@ class Func(object):
 		if not fini:
 			s += "VRT_CTX"
 			p = ", "
-		if self.pfx != None:
+		if self.pfx is not None:
 			s += p + self.pfx
 			p = ", "
 		for a in self.al:
@@ -396,7 +390,7 @@ class Func(object):
 		if not fini:
 			s += "VRT_CTX"
 			p = ", "
-		if self.pfx != None:
+		if self.pfx is not None:
 			s += p + self.pfx
 			p = ", "
 		for a in self.al:
@@ -457,7 +451,7 @@ class Func(object):
 		d = ""
 		for i in self.al:
 			s += d + i.typ
-			if i.nam != None:
+			if i.nam is not None:
 				s += " " + i.nam
 			d = ", "
 		fo.write(s + ")\n")
@@ -477,7 +471,7 @@ class Obj(object):
 		self.st = None
 
 	def fixup(self, modnam):
-		assert self.nam != None
+		assert self.nam is not None
 		self.st = "struct vmod_" + modnam + "_" + self.nam
 		self.init.set_pfx(self.st + " **, const char *")
 		self.fini.set_pfx(self.st + " **")
@@ -579,13 +573,13 @@ class Arg(object):
 		return "<ARG %s %s %s>" % (self.nam, self.typ, str(self.det))
 
 	def c_strspec(self):
-		if self.det == None:
+		if self.det is None:
 			s = self.typ + "\\0"
 		else:
 			s = self.det
-		if self.nam != None:
+		if self.nam is not None:
 			s += '"\n\t\t    "\\1' + self.nam + '\\0'
-		if self.val != None:
+		if self.val is not None:
 			# The space before the value is important to
 			# terminate the \2 escape sequence
 			s += '"\n\t\t\t"\\2 ' + quote(self.val) + "\\0"
@@ -594,6 +588,7 @@ class Arg(object):
 #######################################################################
 #
 #
+
 def parse_enum2(tl):
 	t = tl.get_token()
 	if t.str != "{":
@@ -601,7 +596,7 @@ def parse_enum2(tl):
 	s = "ENUM\\0"
 	t = None
 	while True:
-		if t == None:
+		if t is None:
 			t = tl.get_token()
 		if t.str == "}":
 			break
@@ -617,9 +612,10 @@ def parse_enum2(tl):
 	s += "\\0"
 	return Arg("ENUM", det=s)
 
+
 def parse_arg(tl, al):
 	t = tl.get_token()
-	assert t != None
+	assert t is not None
 
 	if t.str == ")":
 		return t
@@ -668,13 +664,10 @@ def parse_module(tl):
 	dnm = s[1:]
 	return Vmod(nm, dnm, sec)
 
-#######################################################################
-#
-#
 
 def parse_func(tl, rt_type=None, pobj=None):
 	al = list()
-	if rt_type == None:
+	if rt_type is None:
 		t = tl.get_token()
 		rt_type = t.str
 	if rt_type not in ctypes:
@@ -683,11 +676,11 @@ def parse_func(tl, rt_type=None, pobj=None):
 
 	t = tl.get_token()
 	fname = t.str
-	if pobj != None and fname[0] == "." and is_c_name(fname[1:]):
+	if pobj is not None and fname[0] == "." and is_c_name(fname[1:]):
 		fname = pobj + fname
-	elif pobj != None and fname[0] != ".":
+	elif pobj is not None and fname[0] != ".":
 		raise ParseError("Method name '%s' must start with ." % fname)
-	elif pobj != None and not is_c_name(fname[1:]):
+	elif pobj is not None and not is_c_name(fname[1:]):
 		raise ParseError("Method name '%s' is illegal" % fname[1:])
 	elif not is_c_name(fname):
 		raise ParseError("Function name '%s' is illegal" % fname)
@@ -719,9 +712,11 @@ def parse_obj(tl):
 
 
 #######################################################################
-# A section of the inputvcc, starting at a keyword
 
 class FileSection(object):
+	"""
+	A section of the inputvcc, starting at a keyword.
+	"""
 	def __init__(self):
 		self.l = []
 		self.tl = []
@@ -769,7 +764,7 @@ class FileSection(object):
 
 	def parse(self, vx):
 		t = self.get_token()
-		if t == None:
+		if t is None:
 			return
 		t0 = t.str
 		if t.str == "$Module":
@@ -820,9 +815,11 @@ class FileSection(object):
 				o.doc(i)
 
 #######################################################################
-# Polish the copyright message
-#
+
 def polish(l):
+	"""
+	Polish the copyright message.
+	"""
 	if len(l[0]) == 0:
 		l.pop(0)
 		return True
@@ -833,7 +830,7 @@ def polish(l):
 		if i[0] != c:
 			c = None
 			break
-	if c != None:
+	if c is not None:
 		for i in range(len(l)):
 			l[i] = l[i][1:]
 		return True



From lkarsten at varnish-software.com  Thu May 26 14:56:07 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 26 May 2016 16:56:07 +0200
Subject: [master] 6693cfc Implement atomic creation of output files.
Message-ID: <E1b5whj-0008MW-UO@project.varnish-software.com>


commit 6693cfc538883dd9f5e99c186aaf78c9829e6fa0
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu May 26 16:49:07 2016 +0200

    Implement atomic creation of output files.
    
    By writing output byte by byte, the calling parallel make would start
    jobs depending on the completed files too soon.
    
    Avoid this by writing to temporary files and move the completed results
    into place.
    
    Fixes: #1921

diff --git a/lib/libvcc/vmodtool.py b/lib/libvcc/vmodtool.py
index dc89cb7..4c5aeac 100755
--- a/lib/libvcc/vmodtool.py
+++ b/lib/libvcc/vmodtool.py
@@ -41,9 +41,10 @@ import re
 import optparse
 import unittest
 import random
-from os import unlink
+from os import fdopen, rename, unlink
 from os.path import dirname, exists, join, realpath
 from pprint import pprint, pformat
+from tempfile import mkstemp
 
 ctypes = {
 	'ACL':		"VCL_ACL",
@@ -913,34 +914,37 @@ def runmain(inputvcc, rstdir, outputprefix):
 	#######################################################################
 	# Parsing done, now process
 	#
+	fd_cfile, fn_cfile = mkstemp()
+	cfile = fdopen(fd_cfile, "w")
 
-	fc = open("%s.c" % outputprefix, "w")
-	fh = open("%s.h" % outputprefix, "w")
+	fd_headerfile, fn_headerfile = mkstemp()
+	headerfile = fdopen(fd_headerfile, "w")
 
-	write_c_file_warning(fc)
-	write_c_file_warning(fh)
+	write_c_file_warning(cfile)
+	write_c_file_warning(headerfile)
 
-	fh.write('struct vmod_priv;\n\n')
+	headerfile.write('struct vmod_priv;\n\n')
 
-	fh.write('extern const struct vmod_data Vmod_%s_Data;\n\n' % vx[0].nam)
+	headerfile.write('extern const struct vmod_data Vmod_%s_Data;\n\n' % vx[0].nam)
 
-	vx[0].c_proto(fh)
+	vx[0].c_proto(headerfile)
 
-	fc.write('#include "config.h"\n')
-	fc.write('#include "vcl.h"\n')
-	fc.write('#include "vrt.h"\n')
-	fc.write('#include "%s.h"\n' % outputprefix)
-	fc.write('#include "vmod_abi.h"\n')
-	fc.write('\n')
+	cfile.write('#include "config.h"\n')
+	cfile.write('#include "vcl.h"\n')
+	cfile.write('#include "vrt.h"\n')
+	cfile.write('#include "%s.h"\n' % outputprefix)
+	cfile.write('#include "vmod_abi.h"\n')
+	cfile.write('\n')
 
-	vx[0].c_typedefs(fc)
-	vx[0].c_vmod(fc)
+	vx[0].c_typedefs(cfile)
+	vx[0].c_vmod(cfile)
 
-	fc.close()
-	fh.close()
+	cfile.close()
+	headerfile.close()
 
 	for suf in ("", ".man"):
-		fp = open(join(rstdir, "vmod_%s%s.rst" % (vx[0].nam, suf)), "w")
+		fd, fn_fp = mkstemp()
+		fp = fdopen(fd, "w")
 		write_rst_file_warning(fp)
 
 		vx[0].doc_dump(fp, suf)
@@ -953,6 +957,12 @@ def runmain(inputvcc, rstdir, outputprefix):
 			for i in copy_right:
 				fp.write("  %s\n" % i)
 			fp.write("\n")
+		fp.close()
+		rename(fn_fp, join(rstdir, "vmod_%s%s.rst" % (vx[0].nam, suf)))
+
+	# Our makefile targets the .c file so make sure to put that in place last.
+	rename(fn_headerfile, outputprefix + ".h")
+	rename(fn_cfile, outputprefix + ".h")
 
 
 if __name__ == "__main__":



From lkarsten at varnish-software.com  Thu May 26 15:14:10 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 26 May 2016 17:14:10 +0200
Subject: [master] 8a622fa Output to the correct file name.
Message-ID: <E1b5wzC-00016S-5X@project.varnish-software.com>


commit 8a622fa05aee88f25852abbf8cc6f911d6a22cbf
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu May 26 17:13:16 2016 +0200

    Output to the correct file name.
    
    Last minute changes...

diff --git a/lib/libvcc/vmodtool.py b/lib/libvcc/vmodtool.py
index 4c5aeac..bd4f99c 100755
--- a/lib/libvcc/vmodtool.py
+++ b/lib/libvcc/vmodtool.py
@@ -962,7 +962,7 @@ def runmain(inputvcc, rstdir, outputprefix):
 
 	# Our makefile targets the .c file so make sure to put that in place last.
 	rename(fn_headerfile, outputprefix + ".h")
-	rename(fn_cfile, outputprefix + ".h")
+	rename(fn_cfile, outputprefix + ".c")
 
 
 if __name__ == "__main__":



From nils.goroll at uplex.de  Thu May 26 15:19:07 2016
From: nils.goroll at uplex.de (Nils Goroll)
Date: Thu, 26 May 2016 17:19:07 +0200
Subject: [master] dde9cd6 vcl_hash always gets called after vcl_recv
Message-ID: <E1b5x3z-0001Qs-HC@project.varnish-software.com>


commit dde9cd60d11dabf42845fb4d5082754e83e56c44
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Thu May 26 17:00:46 2016 +0200

    vcl_hash always gets called after vcl_recv
    
    but the state after vcl_hash is determined by the return from vcl_recv

diff --git a/doc/graphviz/cache_req_fsm.dot b/doc/graphviz/cache_req_fsm.dot
index 4f934da..c8e6a5c 100644
--- a/doc/graphviz/cache_req_fsm.dot
+++ b/doc/graphviz/cache_req_fsm.dot
@@ -195,10 +195,13 @@ digraph cache_req_fsm {
 		      label="{cnt_recv:|{vcl_hash\{\}|req.*}|{<lookup>lookup}}"
 		      ]
 	}
-	recv:pipe -> pipe [style=bold,color=orange]
-	recv:pass -> pass [style=bold,color=red]
+	recv:pipe -> hash [style=bold,color=orange]
+	recv:pass -> hash [style=bold,color=red]
 	hash:lookup:w -> lookup [style=bold,color=green]
 	hash:lookup:s -> purge:top:n [style=bold,color=purple]
+	hash:lookup:s -> pass [style=bold,color=red]
+	hash:lookup:e -> pipe [style=bold,color=orange]
+
 	recv:purge:s -> hash [style=bold,color=purple]
 
 	/* cnt_purge */
diff --git a/doc/graphviz/cache_req_fsm.svg b/doc/graphviz/cache_req_fsm.svg
index d9976c3..21828c8 100644
--- a/doc/graphviz/cache_req_fsm.svg
+++ b/doc/graphviz/cache_req_fsm.svg
@@ -14,123 +14,91 @@
 </g>
 <!-- acceptor -->
 <g id="node1" class="node"><title>acceptor</title>
-<polygon fill="none" stroke="black" points="502.608,-1208.5 461.304,-1226.5 378.696,-1226.5 337.392,-1208.5 378.696,-1190.5 461.304,-1190.5 502.608,-1208.5"/>
-<text text-anchor="middle" x="420" y="-1204.8" font-family="Times,serif" font-size="14.00">Request received</text>
+<polygon fill="none" stroke="black" points="604.608,-1208.5 563.304,-1226.5 480.696,-1226.5 439.392,-1208.5 480.696,-1190.5 563.304,-1190.5 604.608,-1208.5"/>
+<text text-anchor="middle" x="522" y="-1204.8" font-family="Times,serif" font-size="14.00">Request received</text>
 </g>
 <!-- recv -->
 <g id="node4" class="node"><title>recv</title>
-<polygon fill="none" stroke="black" points="466,-1078.5 466,-1147.5 680,-1147.5 680,-1078.5 466,-1078.5"/>
-<text text-anchor="middle" x="573" y="-1132.3" font-family="Times,serif" font-size="14.00">cnt_recv:</text>
-<polyline fill="none" stroke="black" points="466,-1124.5 680,-1124.5 "/>
-<text text-anchor="middle" x="527.5" y="-1109.3" font-family="Times,serif" font-size="14.00">vcl_recv{}</text>
-<polyline fill="none" stroke="black" points="589,-1101.5 589,-1124.5 "/>
-<text text-anchor="middle" x="634.5" y="-1109.3" font-family="Times,serif" font-size="14.00">req.*</text>
-<polyline fill="none" stroke="black" points="466,-1101.5 680,-1101.5 "/>
-<text text-anchor="middle" x="486.5" y="-1086.3" font-family="Times,serif" font-size="14.00">hash</text>
-<polyline fill="none" stroke="black" points="507,-1078.5 507,-1101.5 "/>
-<text text-anchor="middle" x="530.5" y="-1086.3" font-family="Times,serif" font-size="14.00">purge</text>
-<polyline fill="none" stroke="black" points="554,-1078.5 554,-1101.5 "/>
-<text text-anchor="middle" x="574" y="-1086.3" font-family="Times,serif" font-size="14.00">pass</text>
-<polyline fill="none" stroke="black" points="594,-1078.5 594,-1101.5 "/>
-<text text-anchor="middle" x="614" y="-1086.3" font-family="Times,serif" font-size="14.00">pipe</text>
-<polyline fill="none" stroke="black" points="634,-1078.5 634,-1101.5 "/>
-<text text-anchor="middle" x="657" y="-1086.3" font-family="Times,serif" font-size="14.00">synth</text>
+<polygon fill="none" stroke="black" points="415,-1078.5 415,-1147.5 629,-1147.5 629,-1078.5 415,-1078.5"/>
+<text text-anchor="middle" x="522" y="-1132.3" font-family="Times,serif" font-size="14.00">cnt_recv:</text>
+<polyline fill="none" stroke="black" points="415,-1124.5 629,-1124.5 "/>
+<text text-anchor="middle" x="476.5" y="-1109.3" font-family="Times,serif" font-size="14.00">vcl_recv{}</text>
+<polyline fill="none" stroke="black" points="538,-1101.5 538,-1124.5 "/>
+<text text-anchor="middle" x="583.5" y="-1109.3" font-family="Times,serif" font-size="14.00">req.*</text>
+<polyline fill="none" stroke="black" points="415,-1101.5 629,-1101.5 "/>
+<text text-anchor="middle" x="435.5" y="-1086.3" font-family="Times,serif" font-size="14.00">hash</text>
+<polyline fill="none" stroke="black" points="456,-1078.5 456,-1101.5 "/>
+<text text-anchor="middle" x="479.5" y="-1086.3" font-family="Times,serif" font-size="14.00">purge</text>
+<polyline fill="none" stroke="black" points="503,-1078.5 503,-1101.5 "/>
+<text text-anchor="middle" x="523" y="-1086.3" font-family="Times,serif" font-size="14.00">pass</text>
+<polyline fill="none" stroke="black" points="543,-1078.5 543,-1101.5 "/>
+<text text-anchor="middle" x="563" y="-1086.3" font-family="Times,serif" font-size="14.00">pipe</text>
+<polyline fill="none" stroke="black" points="583,-1078.5 583,-1101.5 "/>
+<text text-anchor="middle" x="606" y="-1086.3" font-family="Times,serif" font-size="14.00">synth</text>
 </g>
 <!-- acceptor&#45;&gt;recv -->
 <g id="edge2" class="edge"><title>acceptor&#45;&gt;recv</title>
-<path fill="none" stroke="black" stroke-width="2" d="M448.063,-1190.35C465.274,-1179.83 488.043,-1165.92 509.425,-1152.85"/>
-<polygon fill="black" stroke="black" stroke-width="2" points="511.458,-1155.71 518.165,-1147.51 507.807,-1149.74 511.458,-1155.71"/>
+<path fill="none" stroke="black" stroke-width="2" d="M522,-1190.35C522,-1181.14 522,-1169.31 522,-1157.74"/>
+<polygon fill="black" stroke="black" stroke-width="2" points="525.5,-1157.7 522,-1147.7 518.5,-1157.7 525.5,-1157.7"/>
 </g>
 <!-- ESI_REQ -->
 <g id="node2" class="node"><title>ESI_REQ</title>
-<polygon fill="none" stroke="black" points="627.07,-1208.5 600.535,-1226.5 547.465,-1226.5 520.93,-1208.5 547.465,-1190.5 600.535,-1190.5 627.07,-1208.5"/>
-<text text-anchor="middle" x="574" y="-1204.8" font-family="Times,serif" font-size="14.00">ESI_REQ</text>
+<polygon fill="none" stroke="black" points="421.07,-1208.5 394.535,-1226.5 341.465,-1226.5 314.93,-1208.5 341.465,-1190.5 394.535,-1190.5 421.07,-1208.5"/>
+<text text-anchor="middle" x="368" y="-1204.8" font-family="Times,serif" font-size="14.00">ESI_REQ</text>
 </g>
 <!-- ESI_REQ&#45;&gt;recv -->
 <g id="edge1" class="edge"><title>ESI_REQ&#45;&gt;recv</title>
-<path fill="none" stroke="black" d="M573.817,-1190.35C573.718,-1181.14 573.592,-1169.31 573.468,-1157.74"/>
-<polygon fill="black" stroke="black" points="576.967,-1157.66 573.36,-1147.7 569.968,-1157.74 576.967,-1157.66"/>
+<path fill="none" stroke="black" d="M395.532,-1190.78C412.946,-1180.21 436.214,-1166.08 458.041,-1152.83"/>
+<polygon fill="black" stroke="black" points="459.909,-1155.79 466.64,-1147.61 456.276,-1149.81 459.909,-1155.79"/>
 </g>
 <!-- RESTART -->
 <g id="node3" class="node"><title>RESTART</title>
-<text text-anchor="middle" x="710" y="-1283.3" font-family="Times,serif" font-size="14.00">RESTART</text>
+<text text-anchor="middle" x="687" y="-1283.3" font-family="Times,serif" font-size="14.00">RESTART</text>
 </g>
 <!-- restart -->
 <g id="node22" class="node"><title>restart</title>
-<polygon fill="none" stroke="grey" points="646,-1185.5 646,-1231.5 774,-1231.5 774,-1185.5 646,-1185.5"/>
-<text text-anchor="middle" x="710" y="-1216.3" font-family="Times,serif" font-size="14.00" fill="grey">cnt_restart:</text>
-<polyline fill="none" stroke="grey" points="646,-1208.5 774,-1208.5 "/>
-<text text-anchor="middle" x="664" y="-1193.3" font-family="Times,serif" font-size="14.00" fill="grey">ok?</text>
-<polyline fill="none" stroke="grey" points="682,-1185.5 682,-1208.5 "/>
-<text text-anchor="middle" x="728" y="-1193.3" font-family="Times,serif" font-size="14.00" fill="grey">max_restarts?</text>
+<polygon fill="none" stroke="grey" points="623,-1185.5 623,-1231.5 751,-1231.5 751,-1185.5 623,-1185.5"/>
+<text text-anchor="middle" x="687" y="-1216.3" font-family="Times,serif" font-size="14.00" fill="grey">cnt_restart:</text>
+<polyline fill="none" stroke="grey" points="623,-1208.5 751,-1208.5 "/>
+<text text-anchor="middle" x="641" y="-1193.3" font-family="Times,serif" font-size="14.00" fill="grey">ok?</text>
+<polyline fill="none" stroke="grey" points="659,-1185.5 659,-1208.5 "/>
+<text text-anchor="middle" x="705" y="-1193.3" font-family="Times,serif" font-size="14.00" fill="grey">max_restarts?</text>
 </g>
 <!-- RESTART&#45;&gt;restart -->
 <g id="edge28" class="edge"><title>RESTART&#45;&gt;restart</title>
-<path fill="none" stroke="purple" d="M710,-1268.64C710,-1260.7 710,-1251.02 710,-1241.81"/>
-<polygon fill="purple" stroke="purple" points="713.5,-1241.74 710,-1231.74 706.5,-1241.74 713.5,-1241.74"/>
-</g>
-<!-- pass -->
-<g id="node19" class="node"><title>pass</title>
-<polygon fill="none" stroke="black" points="223,-445.5 223,-514.5 363,-514.5 363,-445.5 223,-445.5"/>
-<text text-anchor="middle" x="293" y="-499.3" font-family="Times,serif" font-size="14.00">cnt_pass:</text>
-<polyline fill="none" stroke="black" points="223,-491.5 363,-491.5 "/>
-<text text-anchor="middle" x="266" y="-476.3" font-family="Times,serif" font-size="14.00">vcl_pass{}</text>
-<polyline fill="none" stroke="black" points="309,-468.5 309,-491.5 "/>
-<text text-anchor="middle" x="336" y="-476.3" font-family="Times,serif" font-size="14.00">req.*</text>
-<polyline fill="none" stroke="black" points="223,-468.5 363,-468.5 "/>
-<text text-anchor="middle" x="245" y="-453.3" font-family="Times,serif" font-size="14.00">fetch</text>
-<polyline fill="none" stroke="black" points="267,-445.5 267,-468.5 "/>
-<text text-anchor="middle" x="290" y="-453.3" font-family="Times,serif" font-size="14.00">synth</text>
-<polyline fill="none" stroke="black" points="313,-445.5 313,-468.5 "/>
-<text text-anchor="middle" x="338" y="-453.3" font-family="Times,serif" font-size="14.00">restart</text>
-</g>
-<!-- recv&#45;&gt;pass -->
-<g id="edge33" class="edge"><title>recv:pass&#45;&gt;pass</title>
-<path fill="none" stroke="red" stroke-width="2" d="M574,-1078C574,-1036.22 575,-1025.78 575,-984 575,-984 575,-984 575,-615 575,-525.019 456.424,-495.389 373.053,-485.677"/>
-<polygon fill="red" stroke="red" stroke-width="2" points="373.391,-482.193 363.07,-484.588 372.632,-489.152 373.391,-482.193"/>
-</g>
-<!-- pipe -->
-<g id="node20" class="node"><title>pipe</title>
-<polygon fill="none" stroke="black" points="619,-925.5 619,-1040.5 753,-1040.5 753,-925.5 619,-925.5"/>
-<text text-anchor="middle" x="686" y="-1025.3" font-family="Times,serif" font-size="14.00">cnt_pipe:</text>
-<polyline fill="none" stroke="black" points="619,-1017.5 753,-1017.5 "/>
-<text text-anchor="middle" x="686" y="-1002.3" font-family="Times,serif" font-size="14.00">filter req.*&#45;&gt;bereq.*</text>
-<polyline fill="none" stroke="black" points="619,-994.5 753,-994.5 "/>
-<text text-anchor="middle" x="657.5" y="-967.8" font-family="Times,serif" font-size="14.00">vcl_pipe{}</text>
-<polyline fill="none" stroke="black" points="696,-948.5 696,-994.5 "/>
-<text text-anchor="middle" x="724.5" y="-979.3" font-family="Times,serif" font-size="14.00">req.*</text>
-<polyline fill="none" stroke="black" points="696,-971.5 753,-971.5 "/>
-<text text-anchor="middle" x="724.5" y="-956.3" font-family="Times,serif" font-size="14.00">bereq.*</text>
-<polyline fill="none" stroke="black" points="619,-948.5 753,-948.5 "/>
-<text text-anchor="middle" x="651" y="-933.3" font-family="Times,serif" font-size="14.00">pipe</text>
-<polyline fill="none" stroke="black" points="683,-925.5 683,-948.5 "/>
-<text text-anchor="middle" x="718" y="-933.3" font-family="Times,serif" font-size="14.00">synth</text>
-</g>
-<!-- recv&#45;&gt;pipe -->
-<g id="edge32" class="edge"><title>recv:pipe&#45;&gt;pipe</title>
-<path fill="none" stroke="orange" stroke-width="2" d="M614,-1078C614,-1068.08 616.557,-1058.57 620.707,-1049.65"/>
-<polygon fill="orange" stroke="orange" stroke-width="2" points="623.927,-1051.05 625.568,-1040.58 617.758,-1047.74 623.927,-1051.05"/>
+<path fill="none" stroke="purple" d="M687,-1268.64C687,-1260.7 687,-1251.02 687,-1241.81"/>
+<polygon fill="purple" stroke="purple" points="690.5,-1241.74 687,-1231.74 683.5,-1241.74 690.5,-1241.74"/>
 </g>
 <!-- hash -->
 <g id="node24" class="node"><title>hash</title>
-<polygon fill="none" stroke="black" points="423,-948.5 423,-1017.5 545,-1017.5 545,-948.5 423,-948.5"/>
-<text text-anchor="middle" x="484" y="-1002.3" font-family="Times,serif" font-size="14.00">cnt_recv:</text>
-<polyline fill="none" stroke="black" points="423,-994.5 545,-994.5 "/>
-<text text-anchor="middle" x="462" y="-979.3" font-family="Times,serif" font-size="14.00">vcl_hash{}</text>
-<polyline fill="none" stroke="black" points="501,-971.5 501,-994.5 "/>
-<text text-anchor="middle" x="523" y="-979.3" font-family="Times,serif" font-size="14.00">req.*</text>
-<polyline fill="none" stroke="black" points="423,-971.5 545,-971.5 "/>
-<text text-anchor="middle" x="484" y="-956.3" font-family="Times,serif" font-size="14.00">lookup</text>
+<polygon fill="none" stroke="black" points="440,-971.5 440,-1040.5 562,-1040.5 562,-971.5 440,-971.5"/>
+<text text-anchor="middle" x="501" y="-1025.3" font-family="Times,serif" font-size="14.00">cnt_recv:</text>
+<polyline fill="none" stroke="black" points="440,-1017.5 562,-1017.5 "/>
+<text text-anchor="middle" x="479" y="-1002.3" font-family="Times,serif" font-size="14.00">vcl_hash{}</text>
+<polyline fill="none" stroke="black" points="518,-994.5 518,-1017.5 "/>
+<text text-anchor="middle" x="540" y="-1002.3" font-family="Times,serif" font-size="14.00">req.*</text>
+<polyline fill="none" stroke="black" points="440,-994.5 562,-994.5 "/>
+<text text-anchor="middle" x="501" y="-979.3" font-family="Times,serif" font-size="14.00">lookup</text>
 </g>
 <!-- recv&#45;&gt;hash -->
 <g id="edge31" class="edge"><title>recv:hash&#45;&gt;hash</title>
-<path fill="none" stroke="green" stroke-width="2" d="M486,-1078C486,-1061.66 485.694,-1043.73 485.322,-1027.95"/>
-<polygon fill="green" stroke="green" stroke-width="2" points="488.82,-1027.84 485.07,-1017.93 481.823,-1028.02 488.82,-1027.84"/>
+<path fill="none" stroke="green" stroke-width="2" d="M435,-1078C435,-1067.05 439.265,-1057.08 445.627,-1048.29"/>
+<polygon fill="green" stroke="green" stroke-width="2" points="448.335,-1050.51 451.997,-1040.57 442.934,-1046.06 448.335,-1050.51"/>
 </g>
 <!-- recv&#45;&gt;hash -->
-<g id="edge36" class="edge"><title>recv:purge:s&#45;&gt;hash</title>
-<path fill="none" stroke="purple" stroke-width="2" d="M530,-1078C530,-1060.08 523.17,-1042.01 514.774,-1026.57"/>
-<polygon fill="purple" stroke="purple" stroke-width="2" points="517.735,-1024.7 509.69,-1017.81 511.68,-1028.21 517.735,-1024.7"/>
+<g id="edge32" class="edge"><title>recv:pipe&#45;&gt;hash</title>
+<path fill="none" stroke="orange" stroke-width="2" d="M563,-1078C563,-1067.33 558.982,-1057.5 552.991,-1048.76"/>
+<polygon fill="orange" stroke="orange" stroke-width="2" points="555.627,-1046.45 546.71,-1040.73 550.113,-1050.76 555.627,-1046.45"/>
+</g>
+<!-- recv&#45;&gt;hash -->
+<g id="edge33" class="edge"><title>recv:pass&#45;&gt;hash</title>
+<path fill="none" stroke="red" stroke-width="2" d="M523,-1078C523,-1068.91 521.333,-1059.44 518.916,-1050.49"/>
+<polygon fill="red" stroke="red" stroke-width="2" points="522.182,-1049.2 515.918,-1040.66 515.487,-1051.24 522.182,-1049.2"/>
+</g>
+<!-- recv&#45;&gt;hash -->
+<g id="edge38" class="edge"><title>recv:purge:s&#45;&gt;hash</title>
+<path fill="none" stroke="purple" stroke-width="2" d="M479,-1078C479,-1068.91 480.667,-1059.44 483.084,-1050.49"/>
+<polygon fill="purple" stroke="purple" stroke-width="2" points="486.513,-1051.24 486.082,-1040.66 479.818,-1049.2 486.513,-1051.24"/>
 </g>
 <!-- SYNTH -->
 <g id="node5" class="node"><title>SYNTH</title>
@@ -325,6 +293,21 @@
 <path fill="none" stroke="blue" stroke-width="2" d="M75,-688C75,-676.143 79.7319,-666.085 87.137,-657.604"/>
 <polygon fill="blue" stroke="blue" stroke-width="2" points="89.6233,-660.068 94.281,-650.552 84.7056,-655.086 89.6233,-660.068"/>
 </g>
+<!-- pass -->
+<g id="node19" class="node"><title>pass</title>
+<polygon fill="none" stroke="black" points="223,-445.5 223,-514.5 363,-514.5 363,-445.5 223,-445.5"/>
+<text text-anchor="middle" x="293" y="-499.3" font-family="Times,serif" font-size="14.00">cnt_pass:</text>
+<polyline fill="none" stroke="black" points="223,-491.5 363,-491.5 "/>
+<text text-anchor="middle" x="266" y="-476.3" font-family="Times,serif" font-size="14.00">vcl_pass{}</text>
+<polyline fill="none" stroke="black" points="309,-468.5 309,-491.5 "/>
+<text text-anchor="middle" x="336" y="-476.3" font-family="Times,serif" font-size="14.00">req.*</text>
+<polyline fill="none" stroke="black" points="223,-468.5 363,-468.5 "/>
+<text text-anchor="middle" x="245" y="-453.3" font-family="Times,serif" font-size="14.00">fetch</text>
+<polyline fill="none" stroke="black" points="267,-445.5 267,-468.5 "/>
+<text text-anchor="middle" x="290" y="-453.3" font-family="Times,serif" font-size="14.00">synth</text>
+<polyline fill="none" stroke="black" points="313,-445.5 313,-468.5 "/>
+<text text-anchor="middle" x="338" y="-453.3" font-family="Times,serif" font-size="14.00">restart</text>
+</g>
 <!-- lookup2&#45;&gt;pass -->
 <g id="edge22" class="edge"><title>lookup2:pass:s&#45;&gt;pass</title>
 <path fill="none" stroke="red" stroke-width="2" d="M212,-688C212,-655.911 256.629,-677.981 274,-651 298.143,-613.501 300.499,-561.496 298.321,-524.909"/>
@@ -332,38 +315,38 @@
 </g>
 <!-- lookup -->
 <g id="node17" class="node"><title>lookup</title>
-<polygon fill="none" stroke="grey" points="98,-818.5 98,-887.5 316,-887.5 316,-818.5 98,-818.5"/>
-<text text-anchor="middle" x="207" y="-872.3" font-family="Times,serif" font-size="14.00" fill="grey">cnt_lookup:</text>
+<polygon fill="none" stroke="grey" points="98,-841.5 98,-910.5 316,-910.5 316,-841.5 98,-841.5"/>
+<text text-anchor="middle" x="207" y="-895.3" font-family="Times,serif" font-size="14.00" fill="grey">cnt_lookup:</text>
+<polyline fill="none" stroke="grey" points="98,-887.5 316,-887.5 "/>
+<text text-anchor="middle" x="207" y="-872.3" font-family="Times,serif" font-size="14.00" fill="grey">hash lookup</text>
 <polyline fill="none" stroke="grey" points="98,-864.5 316,-864.5 "/>
-<text text-anchor="middle" x="207" y="-849.3" font-family="Times,serif" font-size="14.00" fill="grey">hash lookup</text>
-<polyline fill="none" stroke="grey" points="98,-841.5 316,-841.5 "/>
-<text text-anchor="middle" x="116.5" y="-826.3" font-family="Times,serif" font-size="14.00" fill="grey">hit?</text>
-<polyline fill="none" stroke="grey" points="135,-818.5 135,-841.5 "/>
-<text text-anchor="middle" x="159" y="-826.3" font-family="Times,serif" font-size="14.00" fill="grey">miss?</text>
-<polyline fill="none" stroke="grey" points="183,-818.5 183,-841.5 "/>
-<text text-anchor="middle" x="225.5" y="-826.3" font-family="Times,serif" font-size="14.00" fill="grey">hit&#45;for&#45;pass?</text>
-<polyline fill="none" stroke="grey" points="268,-818.5 268,-841.5 "/>
-<text text-anchor="middle" x="292" y="-826.3" font-family="Times,serif" font-size="14.00" fill="grey">busy?</text>
+<text text-anchor="middle" x="116.5" y="-849.3" font-family="Times,serif" font-size="14.00" fill="grey">hit?</text>
+<polyline fill="none" stroke="grey" points="135,-841.5 135,-864.5 "/>
+<text text-anchor="middle" x="159" y="-849.3" font-family="Times,serif" font-size="14.00" fill="grey">miss?</text>
+<polyline fill="none" stroke="grey" points="183,-841.5 183,-864.5 "/>
+<text text-anchor="middle" x="225.5" y="-849.3" font-family="Times,serif" font-size="14.00" fill="grey">hit&#45;for&#45;pass?</text>
+<polyline fill="none" stroke="grey" points="268,-841.5 268,-864.5 "/>
+<text text-anchor="middle" x="292" y="-849.3" font-family="Times,serif" font-size="14.00" fill="grey">busy?</text>
 </g>
 <!-- lookup&#45;&gt;lookup2 -->
 <g id="edge19" class="edge"><title>lookup:h:s&#45;&gt;lookup2</title>
-<path fill="none" stroke="green" stroke-width="2" d="M116,-818C116,-809.259 116,-799.996 116,-790.958"/>
-<polygon fill="green" stroke="green" stroke-width="2" points="119.5,-790.888 116,-780.888 112.5,-790.889 119.5,-790.888"/>
+<path fill="none" stroke="green" stroke-width="2" d="M116,-840C116,-824.001 116,-806.625 116,-790.719"/>
+<polygon fill="green" stroke="green" stroke-width="2" points="119.5,-790.507 116,-780.507 112.5,-790.507 119.5,-790.507"/>
 </g>
 <!-- lookup&#45;&gt;lookup -->
 <g id="edge16" class="edge"><title>lookup:busy:e&#45;&gt;lookup:top:e</title>
-<path fill="none" stroke="grey" d="M316,-830C322,-817 334,-817 334,-853 334,-877.75 328.328,-885.484 322.833,-883.677"/>
-<polygon fill="grey" stroke="grey" points="325.263,-881.142 316,-876 320.034,-885.797 325.263,-881.142"/>
-<text text-anchor="middle" x="367.5" y="-849.3" font-family="Times,serif" font-size="14.00" fill="grey">(waitinglist)</text>
+<path fill="none" stroke="grey" d="M316,-853C322,-834 334,-834 334,-876 334,-908.812 326.676,-915.99 320.61,-908.5"/>
+<polygon fill="grey" stroke="grey" points="323.515,-906.469 316,-899 317.217,-909.525 323.515,-906.469"/>
+<text text-anchor="middle" x="367.5" y="-872.3" font-family="Times,serif" font-size="14.00" fill="grey">(waitinglist)</text>
 </g>
 <!-- lookup&#45;&gt;miss -->
 <g id="edge17" class="edge"><title>lookup:miss:s&#45;&gt;miss</title>
-<path fill="none" stroke="blue" stroke-width="2" d="M159,-818C159,-778.017 219.308,-814.587 241,-781 263.425,-746.279 256.118,-726.469 241,-688 236.81,-677.339 230.183,-667.328 222.711,-658.375"/>
+<path fill="none" stroke="blue" stroke-width="2" d="M159,-840C159,-795.102 220.525,-820.957 241,-781 259.85,-744.215 256.118,-726.469 241,-688 236.81,-677.339 230.183,-667.328 222.711,-658.375"/>
 <polygon fill="blue" stroke="blue" stroke-width="2" points="225.227,-655.938 215.966,-650.79 219.996,-660.589 225.227,-655.938"/>
 </g>
 <!-- lookup&#45;&gt;pass -->
 <g id="edge18" class="edge"><title>lookup:hfp:s&#45;&gt;pass</title>
-<path fill="none" stroke="red" stroke-width="2" d="M226,-818C226,-794.744 251.216,-801.05 263,-781 313.367,-695.3 305.717,-662.137 313,-563 313.977,-549.703 315.434,-546.109 313,-533 312.47,-530.147 311.79,-527.249 311.002,-524.355"/>
+<path fill="none" stroke="red" stroke-width="2" d="M226,-840C226,-809.282 249.583,-809.097 262,-781 302.222,-689.987 305.71,-662.238 313,-563 313.977,-549.703 315.434,-546.109 313,-533 312.47,-530.147 311.79,-527.249 311.002,-524.355"/>
 <polygon fill="red" stroke="red" stroke-width="2" points="314.281,-523.113 307.968,-514.604 307.597,-525.193 314.281,-523.113"/>
 </g>
 <!-- miss&#45;&gt;FETCH -->
@@ -381,58 +364,85 @@
 <path fill="none" stroke="red" stroke-width="2" d="M245,-445C245,-435.486 228.709,-422.805 211.625,-412.122"/>
 <polygon fill="red" stroke="red" stroke-width="2" points="213.073,-408.908 202.701,-406.745 209.46,-414.904 213.073,-408.908"/>
 </g>
+<!-- pipe -->
+<g id="node20" class="node"><title>pipe</title>
+<polygon fill="none" stroke="black" points="619,-818.5 619,-933.5 753,-933.5 753,-818.5 619,-818.5"/>
+<text text-anchor="middle" x="686" y="-918.3" font-family="Times,serif" font-size="14.00">cnt_pipe:</text>
+<polyline fill="none" stroke="black" points="619,-910.5 753,-910.5 "/>
+<text text-anchor="middle" x="686" y="-895.3" font-family="Times,serif" font-size="14.00">filter req.*&#45;&gt;bereq.*</text>
+<polyline fill="none" stroke="black" points="619,-887.5 753,-887.5 "/>
+<text text-anchor="middle" x="657.5" y="-860.8" font-family="Times,serif" font-size="14.00">vcl_pipe{}</text>
+<polyline fill="none" stroke="black" points="696,-841.5 696,-887.5 "/>
+<text text-anchor="middle" x="724.5" y="-872.3" font-family="Times,serif" font-size="14.00">req.*</text>
+<polyline fill="none" stroke="black" points="696,-864.5 753,-864.5 "/>
+<text text-anchor="middle" x="724.5" y="-849.3" font-family="Times,serif" font-size="14.00">bereq.*</text>
+<polyline fill="none" stroke="black" points="619,-841.5 753,-841.5 "/>
+<text text-anchor="middle" x="651" y="-826.3" font-family="Times,serif" font-size="14.00">pipe</text>
+<polyline fill="none" stroke="black" points="683,-818.5 683,-841.5 "/>
+<text text-anchor="middle" x="718" y="-826.3" font-family="Times,serif" font-size="14.00">synth</text>
+</g>
 <!-- pipe_do -->
 <g id="node21" class="node"><title>pipe_do</title>
-<ellipse fill="none" stroke="black" cx="698" cy="-853" rx="95.4188" ry="26.7407"/>
-<text text-anchor="middle" x="698" y="-856.8" font-family="Times,serif" font-size="14.00">send bereq,</text>
-<text text-anchor="middle" x="698" y="-841.8" font-family="Times,serif" font-size="14.00">copy bytes until close</text>
+<ellipse fill="none" stroke="black" cx="698" cy="-734.5" rx="95.4188" ry="26.7407"/>
+<text text-anchor="middle" x="698" y="-738.3" font-family="Times,serif" font-size="14.00">send bereq,</text>
+<text text-anchor="middle" x="698" y="-723.3" font-family="Times,serif" font-size="14.00">copy bytes until close</text>
 </g>
 <!-- pipe&#45;&gt;pipe_do -->
 <g id="edge26" class="edge"><title>pipe:pipe&#45;&gt;pipe_do</title>
-<path fill="none" stroke="orange" stroke-width="2" d="M651,-925C651,-911.335 657.128,-898.282 664.958,-887.18"/>
-<polygon fill="orange" stroke="orange" stroke-width="2" points="667.862,-889.146 671.205,-879.092 662.322,-884.867 667.862,-889.146"/>
+<path fill="none" stroke="orange" stroke-width="2" d="M651,-818C651,-800.39 659.184,-783.111 668.723,-768.953"/>
+<polygon fill="orange" stroke="orange" stroke-width="2" points="671.761,-770.724 674.775,-760.566 666.085,-766.627 671.761,-770.724"/>
 </g>
 <!-- pipe_do&#45;&gt;DONE -->
 <g id="edge27" class="edge"><title>pipe_do&#45;&gt;DONE</title>
-<path fill="none" stroke="orange" stroke-width="2" d="M682.476,-826.166C670.159,-803.197 655,-768.27 655,-735.5 655,-735.5 655,-735.5 655,-90 655,-58.2915 621.5,-39.6576 592.711,-29.4983"/>
+<path fill="none" stroke="orange" stroke-width="2" d="M682.476,-707.666C670.159,-684.697 655,-649.77 655,-617 655,-617 655,-617 655,-90 655,-58.2915 621.5,-39.6576 592.711,-29.4983"/>
 <polygon fill="orange" stroke="orange" stroke-width="2" points="593.776,-26.1642 583.183,-26.3712 591.593,-32.8151 593.776,-26.1642"/>
 </g>
 <!-- restart&#45;&gt;recv -->
 <g id="edge29" class="edge"><title>restart:ok:s&#45;&gt;recv</title>
-<path fill="none" stroke="black" d="M664,-1185C664,-1173.49 659.422,-1163.58 652.319,-1155.12"/>
-<polygon fill="black" stroke="black" points="654.629,-1152.47 645.145,-1147.75 649.615,-1157.36 654.629,-1152.47"/>
+<path fill="none" stroke="black" d="M641,-1185C641,-1173.12 636.572,-1163.18 629.424,-1154.88"/>
+<polygon fill="black" stroke="black" points="631.723,-1152.23 622.144,-1147.7 626.809,-1157.21 631.723,-1152.23"/>
 </g>
 <!-- err_restart -->
 <g id="node23" class="node"><title>err_restart</title>
-<text text-anchor="middle" x="729" y="-1109.3" font-family="Times,serif" font-size="14.00">SYNTH</text>
+<text text-anchor="middle" x="705" y="-1109.3" font-family="Times,serif" font-size="14.00">SYNTH</text>
 </g>
 <!-- restart&#45;&gt;err_restart -->
 <g id="edge30" class="edge"><title>restart:max:s&#45;&gt;err_restart</title>
-<path fill="none" stroke="purple" d="M728,-1185C728,-1170.65 728.207,-1154.67 728.432,-1141.51"/>
-<polygon fill="purple" stroke="purple" points="731.937,-1141.28 728.622,-1131.22 724.938,-1141.15 731.937,-1141.28"/>
+<path fill="none" stroke="purple" d="M705,-1185C705,-1170.65 705,-1154.67 705,-1141.51"/>
+<polygon fill="purple" stroke="purple" points="708.5,-1141.22 705,-1131.22 701.5,-1141.22 708.5,-1141.22"/>
 </g>
 <!-- hash&#45;&gt;lookup -->
 <g id="edge34" class="edge"><title>hash:lookup:w&#45;&gt;lookup</title>
-<path fill="none" stroke="green" stroke-width="2" d="M422,-960C359.706,-960 296.753,-923.759 255.21,-893.659"/>
-<polygon fill="green" stroke="green" stroke-width="2" points="257.057,-890.672 246.936,-887.539 252.895,-896.3 257.057,-890.672"/>
+<path fill="none" stroke="green" stroke-width="2" d="M439,-983C372.437,-983 304.057,-946.456 258.942,-916.282"/>
+<polygon fill="green" stroke="green" stroke-width="2" points="260.75,-913.279 250.515,-910.539 256.807,-919.063 260.75,-913.279"/>
+</g>
+<!-- hash&#45;&gt;pass -->
+<g id="edge36" class="edge"><title>hash:lookup:s&#45;&gt;pass</title>
+<path fill="none" stroke="red" stroke-width="2" d="M501,-971C501,-941.539 538.166,-957.45 556,-934 572.165,-912.745 575,-903.704 575,-877 575,-877 575,-877 575,-615 575,-525.019 456.424,-495.389 373.053,-485.677"/>
+<polygon fill="red" stroke="red" stroke-width="2" points="373.391,-482.193 363.07,-484.588 372.632,-489.152 373.391,-482.193"/>
+</g>
+<!-- hash&#45;&gt;pipe -->
+<g id="edge37" class="edge"><title>hash:lookup:e&#45;&gt;pipe</title>
+<path fill="none" stroke="orange" stroke-width="2" d="M563,-983C593.899,-983 620.887,-963.568 641.648,-941.344"/>
+<polygon fill="orange" stroke="orange" stroke-width="2" points="644.401,-943.516 648.447,-933.724 639.178,-938.855 644.401,-943.516"/>
 </g>
 <!-- purge -->
 <g id="node25" class="node"><title>purge</title>
-<polygon fill="none" stroke="black" points="419,-818.5 419,-887.5 547,-887.5 547,-818.5 419,-818.5"/>
-<text text-anchor="middle" x="483" y="-872.3" font-family="Times,serif" font-size="14.00">cnt_purge:</text>
+<polygon fill="none" stroke="black" points="419,-841.5 419,-910.5 547,-910.5 547,-841.5 419,-841.5"/>
+<text text-anchor="middle" x="483" y="-895.3" font-family="Times,serif" font-size="14.00">cnt_purge:</text>
+<polyline fill="none" stroke="black" points="419,-887.5 547,-887.5 "/>
+<text text-anchor="middle" x="461" y="-872.3" font-family="Times,serif" font-size="14.00">vcl_purge{}</text>
+<polyline fill="none" stroke="black" points="503,-864.5 503,-887.5 "/>
+<text text-anchor="middle" x="525" y="-872.3" font-family="Times,serif" font-size="14.00">req.*</text>
 <polyline fill="none" stroke="black" points="419,-864.5 547,-864.5 "/>
-<text text-anchor="middle" x="461" y="-849.3" font-family="Times,serif" font-size="14.00">vcl_purge{}</text>
-<polyline fill="none" stroke="black" points="503,-841.5 503,-864.5 "/>
-<text text-anchor="middle" x="525" y="-849.3" font-family="Times,serif" font-size="14.00">req.*</text>
-<polyline fill="none" stroke="black" points="419,-841.5 547,-841.5 "/>
-<text text-anchor="middle" x="450" y="-826.3" font-family="Times,serif" font-size="14.00">synth</text>
-<polyline fill="none" stroke="black" points="481,-818.5 481,-841.5 "/>
-<text text-anchor="middle" x="514" y="-826.3" font-family="Times,serif" font-size="14.00">restart</text>
+<text text-anchor="middle" x="450" y="-849.3" font-family="Times,serif" font-size="14.00">synth</text>
+<polyline fill="none" stroke="black" points="481,-841.5 481,-864.5 "/>
+<text text-anchor="middle" x="514" y="-849.3" font-family="Times,serif" font-size="14.00">restart</text>
 </g>
 <!-- hash&#45;&gt;purge -->
 <g id="edge35" class="edge"><title>hash:lookup:s&#45;&gt;purge:top:n</title>
-<path fill="none" stroke="purple" stroke-width="2" d="M484,-947C484,-924.462 483.261,-916.457 483.054,-898.006"/>
-<polygon fill="purple" stroke="purple" stroke-width="2" points="486.554,-897.981 483,-888 479.554,-898.019 486.554,-897.981"/>
+<path fill="none" stroke="purple" stroke-width="2" d="M501,-971C501,-947.333 487.585,-940.99 483.917,-922.06"/>
+<polygon fill="purple" stroke="purple" stroke-width="2" points="487.394,-921.641 483,-912 480.423,-922.277 487.394,-921.641"/>
 </g>
 </g>
 </svg>



From varnish-commit at varnish-cache.org  Thu May 26 20:15:49 2016
From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org)
Date: Thu, 26 May 2016 20:15:49 -0000
Subject: Emailing: photo 05-26-2016, 02 23 82
Message-ID: <3c5968b67684$dee01c0416994a8ee$@varnish-cache.org>


Your message is ready to be sent with the following file or link
attachments:

photo 05-26-2016, 02 23 82


Note: To protect against computer viruses, e-mail programs may prevent
sending or receiving certain types of file attachments.  Check your e-mail
security settings to determine how attachments are handled.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: photo 05-26-2016, 02 23 82.zip
Type: application/zip
Size: 3427 bytes
Desc: not available
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-commit/attachments/20160526/56d2ba18/attachment.zip>

From fgsch at lodoss.net  Fri May 27 08:00:09 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Fri, 27 May 2016 10:00:09 +0200
Subject: [master] 81138bf Log the keep value rather than -1
Message-ID: <E1b6Cgj-0004YJ-Ra@project.varnish-software.com>


commit 81138bf598fce112f84be5b2e74d29da24e5a832
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Fri May 27 08:37:46 2016 +0100

    Log the keep value rather than -1
    
    Useful if default_keep is not the default (0).

diff --git a/bin/varnishd/cache/cache_rfc2616.c b/bin/varnishd/cache/cache_rfc2616.c
index bc967d1..39902c5 100644
--- a/bin/varnishd/cache/cache_rfc2616.c
+++ b/bin/varnishd/cache/cache_rfc2616.c
@@ -199,7 +199,7 @@ RFC2616_Ttl(struct busyobj *bo, double now, double *t_origin,
 
 	VSLb(bo->vsl, SLT_TTL,
 	    "RFC %.0f %.0f %.0f %.0f %.0f %.0f %.0f %u",
-	    *ttl, *grace, -1., now,
+	    *ttl, *grace, *keep, now,
 	    *t_origin, h_date, h_expires, max_age);
 }
 



From fgsch at lodoss.net  Fri May 27 08:00:09 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Fri, 27 May 2016 10:00:09 +0200
Subject: [master] f962730 Forgot to add oc_flags.h and oc_exp_flags.h
Message-ID: <E1b6Cgj-0004YP-TP@project.varnish-software.com>


commit f962730f3dfe4d95aa51f32c7208cafa5cf3c718
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Fri May 27 08:44:00 2016 +0100

    Forgot to add oc_flags.h and oc_exp_flags.h

diff --git a/include/Makefile.am b/include/Makefile.am
index ca51410..c833d6a 100644
--- a/include/Makefile.am
+++ b/include/Makefile.am
@@ -15,6 +15,8 @@ nobase_pkginclude_HEADERS = \
 	tbl/http_response.h \
 	tbl/locks.h \
 	tbl/obj_attr.h \
+	tbl/oc_exp_flags.h \
+	tbl/oc_flags.h \
 	tbl/params.h \
 	tbl/req_body.h \
 	tbl/req_flags.h \



From fgsch at lodoss.net  Fri May 27 08:00:09 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Fri, 27 May 2016 10:00:09 +0200
Subject: [master] 3522896 Create the temporary files in the same dir
Message-ID: <E1b6Cgj-0004YY-Uz@project.varnish-software.com>


commit 3522896b8ad857430dc6a9d18662edad84040d67
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Fri May 27 08:46:23 2016 +0100

    Create the temporary files in the same dir
    
    Rename will fail across different mount points.

diff --git a/lib/libvcc/vmodtool.py b/lib/libvcc/vmodtool.py
index bd4f99c..4ee6341 100755
--- a/lib/libvcc/vmodtool.py
+++ b/lib/libvcc/vmodtool.py
@@ -914,10 +914,10 @@ def runmain(inputvcc, rstdir, outputprefix):
 	#######################################################################
 	# Parsing done, now process
 	#
-	fd_cfile, fn_cfile = mkstemp()
+	fd_cfile, fn_cfile = mkstemp(dir='.')
 	cfile = fdopen(fd_cfile, "w")
 
-	fd_headerfile, fn_headerfile = mkstemp()
+	fd_headerfile, fn_headerfile = mkstemp(dir='.')
 	headerfile = fdopen(fd_headerfile, "w")
 
 	write_c_file_warning(cfile)
@@ -943,7 +943,7 @@ def runmain(inputvcc, rstdir, outputprefix):
 	headerfile.close()
 
 	for suf in ("", ".man"):
-		fd, fn_fp = mkstemp()
+		fd, fn_fp = mkstemp(dir='.')
 		fp = fdopen(fd, "w")
 		write_rst_file_warning(fp)
 



From phk at FreeBSD.org  Sun May 29 20:28:05 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Sun, 29 May 2016 22:28:05 +0200
Subject: [master] 243d815 At least one platform has a backtrace(3) which
 returns -1 in a size_t :-(
Message-ID: <E1b77Jd-000301-7l@project.varnish-software.com>


commit 243d81567caebaf22f726bf1dea3eadadc07de84
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Sun May 29 20:27:09 2016 +0000

    At least one platform has a backtrace(3) which returns -1 in a size_t :-(

diff --git a/bin/varnishd/cache/cache_panic.c b/bin/varnishd/cache/cache_panic.c
index 53ef3cd..5657535 100644
--- a/bin/varnishd/cache/cache_panic.c
+++ b/bin/varnishd/cache/cache_panic.c
@@ -516,14 +516,16 @@ static void
 pan_backtrace(struct vsb *vsb)
 {
 	void *array[10];
-	size_t size;
-	size_t i;
+	ssize_t size;
+	ssize_t i;
 	char **strings;
 	char *p;
 
 	size = backtrace (array, 10);
-	if (size == 0)
+	if (size <= 0) {
+		VSB_printf(vsb, "Backtrace not available (%zd)\n", size);
 		return;
+	}
 	VSB_printf(vsb, "Backtrace:\n");
 	VSB_indent(vsb, 2);
 	for (i = 0; i < size; i++) {



From phk at FreeBSD.org  Mon May 30 07:43:05 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Mon, 30 May 2016 09:43:05 +0200
Subject: [master] acf04df With FreeBSD 10.3 on a BeagleBone PCRE JIT explodes
 in the pcre_study() function.
Message-ID: <E1b7Hqr-0006tX-AJ@project.varnish-software.com>


commit acf04dff5d763cb57a702645ace55edeb46dc318
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon May 30 07:41:39 2016 +0000

    With FreeBSD 10.3 on a BeagleBone PCRE JIT explodes in the pcre_study()
    function.
    
    Maybe we should have a configure-time test to see if it actually works,
    but for now, just make autogen.des handle it.

diff --git a/autogen.des b/autogen.des
index 3cfc1fa..717c5cf 100755
--- a/autogen.des
+++ b/autogen.des
@@ -6,6 +6,8 @@ set -ex
 
 make -k distclean > /dev/null 2>&1 || true
 
+OPTS=""
+
 # Prefer CLANG if we have it, and have not given preferences
 if [ -f /usr/bin/clang -a "x${CC}" = "x" ] ; then
 	CC=clang
@@ -18,6 +20,10 @@ else
 	DST="--prefix=/opt/varnish --mandir=/opt/varnish/man"
 fi
 
+if [ "x`uname -po`" = "xFreeBSD armv6" ] ; then
+	OPTS="${OPTS} --disable-pcre-jit"
+fi
+
 rm -f configure
 . ./autogen.sh 2>&1 | egrep -v "(subdir-objects|is in a subdirectory)"
 
@@ -33,4 +39,5 @@ export CONFIG_SHELL=/bin/sh
     --enable-developer-warnings \
     --enable-debugging-symbols \
     --enable-dependency-tracking \
+    $OPTS \
     "$@"



From phk at FreeBSD.org  Mon May 30 08:43:05 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Mon, 30 May 2016 10:43:05 +0200
Subject: [master] 367b5e1 Make the santity check of backtrace(3) returns even
 more robust.
Message-ID: <E1b7Imv-0001sy-RR@project.varnish-software.com>


commit 367b5e1c668bc57436af367f913c6220591c785a
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon May 30 07:50:34 2016 +0000

    Make the santity check of backtrace(3) returns even more robust.

diff --git a/bin/varnishd/cache/cache_panic.c b/bin/varnishd/cache/cache_panic.c
index 5657535..b3171f9 100644
--- a/bin/varnishd/cache/cache_panic.c
+++ b/bin/varnishd/cache/cache_panic.c
@@ -512,18 +512,20 @@ pan_sess(struct vsb *vsb, const struct sess *sp)
 
 /*--------------------------------------------------------------------*/
 
+#define BACKTRACE_LEVELS	10
+
 static void
 pan_backtrace(struct vsb *vsb)
 {
-	void *array[10];
-	ssize_t size;
-	ssize_t i;
+	void *array[BACKTRACE_LEVELS];
+	size_t size;
+	size_t i;
 	char **strings;
 	char *p;
 
-	size = backtrace (array, 10);
-	if (size <= 0) {
-		VSB_printf(vsb, "Backtrace not available (%zd)\n", size);
+	size = backtrace (array, BACKTRACE_LEVELS);
+	if (size > BACKTRACE_LEVELS) {
+		VSB_printf(vsb, "Backtrace not available (ret=%zu)\n", size);
 		return;
 	}
 	VSB_printf(vsb, "Backtrace:\n");



From phk at FreeBSD.org  Mon May 30 08:43:05 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Mon, 30 May 2016 10:43:05 +0200
Subject: [master] 99407d3 This bug shows why having both BE and LE test-boxes
 is important.
Message-ID: <E1b7Imv-0001t4-T0@project.varnish-software.com>


commit 99407d37a7c95fdcbccfc4b4e20220f7567ece9e
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon May 30 08:41:08 2016 +0000

    This bug shows why having both BE and LE test-boxes is important.

diff --git a/bin/varnishd/common/params.h b/bin/varnishd/common/params.h
index d5ac4f5..678ed26 100644
--- a/bin/varnishd/common/params.h
+++ b/bin/varnishd/common/params.h
@@ -96,7 +96,7 @@ struct params {
 	double			wthread_add_delay;
 	double			wthread_fail_delay;
 	double			wthread_destroy_delay;
-	double			wthread_stats_rate;
+	unsigned		wthread_stats_rate;
 	ssize_t			wthread_stacksize;
 	unsigned		wthread_queue_limit;
 



From martin at varnish-software.com  Mon May 30 09:35:07 2016
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Mon, 30 May 2016 11:35:07 +0200
Subject: [master] 12b4809 Do not use an objcore flagged OC_F_PASS as a stale
 object
Message-ID: <E1b7JbH-00054P-8N@project.varnish-software.com>


commit 12b480972e4da66ef734d8d0237c75141ec35d7d
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Fri May 27 14:30:18 2016 +0200

    Do not use an objcore flagged OC_F_PASS as a stale object
    
    If an object marked OF_IMSCAND is used as the template during backend
    IMS revalidation, and the new object also is marked OC_F_PASS
    (hit-for-pass) in vcl_backend_response, the resulting object would
    (though ObjCopyAttr) have both OF_IMSCAND and OC_F_PASS. The object
    could then be used again for backend IMS revalidation (since it has
    the OF_IMSCAND flag), which would cause troubles as the body would by
    this time be deleted.
    
    Fix by not considering objects marked OC_F_PASS as a stale object
    candidate during lookup.
    
    Fixes: #1956

diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index 62c79ad..23d8d58 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -189,6 +189,7 @@ vbf_stp_mkbereq(struct worker *wrk, struct busyobj *bo)
 	if (bo->stale_oc != NULL &&
 	    ObjCheckFlag(bo->wrk, bo->stale_oc, OF_IMSCAND) &&
 	    (bo->stale_oc->boc != NULL || ObjGetLen(wrk, bo->stale_oc) != 0)) {
+		AZ(bo->stale_oc->flags & OC_F_PASS);
 		q = HTTP_GetHdrPack(bo->wrk, bo->stale_oc, H_Last_Modified);
 		if (q != NULL)
 			http_PrintfHeader(bo->bereq0,
diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index f057e26..ef73ccd 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -404,8 +404,9 @@ cnt_lookup(struct worker *wrk, struct req *req)
 
 	if ((oc->flags & OC_F_PASS) && busy != NULL) {
 		/* Treat a graced Hit-For-Pass as a miss */
+		(void)HSH_DerefObjCore(wrk, &req->objcore);
+		AZ(req->objcore);
 		req->objcore = busy;
-		req->stale_oc = oc;
 		req->req_step = R_STP_MISS;
 		return (REQ_FSM_MORE);
 	} else if (oc->flags & OC_F_PASS) {
diff --git a/bin/varnishtest/tests/r01956.vtc b/bin/varnishtest/tests/r01956.vtc
new file mode 100644
index 0000000..9d2d505
--- /dev/null
+++ b/bin/varnishtest/tests/r01956.vtc
@@ -0,0 +1,53 @@
+varnishtest "#1956: graced hit-for-pass, backend ims and obj flags inheritance"
+
+server s1 {
+	rxreq
+	txresp -hdr {ETag: "foo"} -body "asdf"
+
+	rxreq
+	expect req.http.if-none-match == {"foo"}
+	txresp -status 304 -hdr {ETag: "bar"}
+
+	rxreq
+	expect req.http.if-none-match == "<undef>"
+	txresp -status 200 -hdr {ETag: "baz"} -body "asdf"
+} -start
+
+varnish v1 -vcl+backend {
+	sub vcl_backend_response {
+		set beresp.ttl = 0.1s;
+		set beresp.keep = 60s;
+		if (beresp.http.etag == {""bar""}) {
+			set beresp.grace = 60s;
+			set beresp.uncacheable = true;
+		} else {
+			set beresp.grace = 0s;
+		}
+		return (deliver);
+	}
+} -start
+
+client c1 {
+	timeout 5
+
+	txreq -hdr "cnt: 1"
+	rxresp
+	expect resp.http.etag == {"foo"}
+	expect resp.body == "asdf"
+
+	delay 0.2
+
+	txreq -hdr "cnt: 2"
+	rxresp
+	expect resp.status == 200
+	expect resp.http.etag == {"bar"}
+	expect resp.body == "asdf"
+
+	delay 0.2
+
+	txreq -hdr "cnt: 3"
+	rxresp
+	expect resp.status == 200
+	expect resp.http.etag == {"baz"}
+	expect resp.body == "asdf"
+} -run



From lkarsten at varnish-software.com  Mon May 30 11:38:05 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Mon, 30 May 2016 13:38:05 +0200
Subject: [master] 1e0bb69 Remove implicit line feed when using formatfile.
Message-ID: <E1b7LWH-00044f-N4@project.varnish-software.com>


commit 1e0bb6946361937edd0fc5fa382e4827758b6dcf
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Mon May 30 13:35:53 2016 +0200

    Remove implicit line feed when using formatfile.
    
    Fixes: #1967

diff --git a/bin/varnishncsa/varnishncsa.c b/bin/varnishncsa/varnishncsa.c
index 5b54ece..086a779 100644
--- a/bin/varnishncsa/varnishncsa.c
+++ b/bin/varnishncsa/varnishncsa.c
@@ -1103,12 +1103,14 @@ read_format(const char *formatfile)
 {
 	FILE *fmtfile;
 	size_t len = 0;
+	int fmtlen;
 	char *fmt = NULL;
 
 	fmtfile = fopen(formatfile, "r");
 	if (fmtfile == NULL)
 		VUT_Error(1, "Can't open format file (%s)", strerror(errno));
-	if (getline(&fmt, &len, fmtfile) == -1) {
+	fmtlen = getline(&fmt, &len, fmtfile);
+	if (fmtlen == -1) {
 		free(fmt);
 		if (feof(fmtfile))
 			VUT_Error(1, "Empty format file");
@@ -1117,6 +1119,8 @@ read_format(const char *formatfile)
 			    strerror(errno));
 	}
 	fclose(fmtfile);
+	if (fmt[fmtlen - 1] == '\n')
+		fmt[fmtlen - 1] = '\0';
 	return (fmt);
 }
 



From lkarsten at varnish-software.com  Mon May 30 12:06:06 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Mon, 30 May 2016 14:06:06 +0200
Subject: [4.1] ccf8775 Add varnishncsa backend mode to changelog.
Message-ID: <E1b7LxO-00066C-6M@project.varnish-software.com>


commit ccf8775b8748308579daf1fdb253ea044e892e69
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Mon May 30 14:05:26 2016 +0200

    Add varnishncsa backend mode to changelog.
    
    Ref: #1905

diff --git a/doc/changes.rst b/doc/changes.rst
index e1302a6..cef6b0c 100644
--- a/doc/changes.rst
+++ b/doc/changes.rst
@@ -4,7 +4,9 @@ DRAFT CHANGELOG ENTRIES FOR NEXT RELEASE
 
 Changes since 4.1.2:
 
-* TBD.
+* varnishncsa can now access and log backend requests. (PR #1905)
+
+.. _1905: https://github.com/varnishcache/varnish-cache/pull/1905
 
 Bugs fixed
 ----------



From lkarsten at varnish-software.com  Mon May 30 12:28:06 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Mon, 30 May 2016 14:28:06 +0200
Subject: [4.1] 4ac5079 Add #1912 to changelog.
Message-ID: <E1b7MIg-0007hX-7y@project.varnish-software.com>


commit 4ac5079e2d3e542446573537ba2032d9b5b60ff7
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Mon May 30 14:27:06 2016 +0200

    Add #1912 to changelog.

diff --git a/doc/changes.rst b/doc/changes.rst
index cef6b0c..0116a43 100644
--- a/doc/changes.rst
+++ b/doc/changes.rst
@@ -12,6 +12,7 @@ Bugs fixed
 ----------
 
 * 1918_ - Correctly handle EOF responses with HTTP/1.1
+* 1912_ - Fix (insignificant) memory leak with mal-formed ESI directives.
 * 1885_ - [vmodtool] Method names should start with a period
 * 1879_ - Correct handling of duplicate headers on IMS header merge
 * 1878_ - Fix a ESI+gzip corner case which had escaped notice until now
@@ -21,6 +22,7 @@ Bugs fixed
 * 1883_ - Only accept C identifiers as acls
 
 .. _1918: https://github.com/varnishcache/varnish-cache/issues/1918
+.. _1912: https://github.com/varnishcache/varnish-cache/issues/1912
 .. _1885: https://github.com/varnishcache/varnish-cache/issues/1885
 .. _1883: https://github.com/varnishcache/varnish-cache/issues/1883
 .. _1879: https://github.com/varnishcache/varnish-cache/issues/1879



From lkarsten at varnish-software.com  Mon May 30 12:46:06 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Mon, 30 May 2016 14:46:06 +0200
Subject: [4.1] 8782c7a Backfill with forgotten changelog entries.
Message-ID: <E1b7Ma6-0000Vp-AY@project.varnish-software.com>


commit 8782c7a21ea910cdfc8d664b2c591ecbc6dc316a
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Mon May 30 14:31:11 2016 +0200

    Backfill with forgotten changelog entries.
    
    Ref: #1813
    Ref: #1810
    Ref: #1807
    Ref: #1804
    Ref: #1801
    Ref: #1796
    Ref: #1794

diff --git a/doc/changes.rst b/doc/changes.rst
index 0116a43..8a7857f 100644
--- a/doc/changes.rst
+++ b/doc/changes.rst
@@ -159,6 +159,8 @@ Changes since 4.1.0:
 Bugs fixed
 ----------
 
+- 1796_ - Don't attempt to allocate a V1L from the workspace if it is overflowed.
+- 1794_ - Fail if multiple -a arguments return the same suckaddr.
 - 1763_ - Restart epoll_wait on EINTR error
 - 1788_ - ObjIter has terrible performance profile when busyobj != NULL
 - 1798_ - Varnish requests painfully slow with large files
@@ -167,7 +169,14 @@ Bugs fixed
 - 1821_ - Always slim private & pass objects after delivery.
 - 1823_ - Rush the objheader if there is a waiting list when it is deref'ed.
 - 1826_ - Ignore 0 Content-Lengths in 204 responses
-
+- 1813_ - Fail if multiple -a arguments return the same suckaddr.
+- 1810_ - Improve handling of HTTP/1.0 clients
+- 1807_ - Return 500 if we cannot decode the stored object into the resp.*
+- 1804_ - Log proxy related messages on the session, not on the request.
+- 1801_ - Relax IP constant parsing
+
+.. _1796: https://www.varnish-cache.org/trac/ticket/1796
+.. _1794: https://www.varnish-cache.org/trac/ticket/1794
 .. _1763: https://www.varnish-cache.org/trac/ticket/1763
 .. _1788: https://www.varnish-cache.org/trac/ticket/1788
 .. _1798: https://www.varnish-cache.org/trac/ticket/1798
@@ -176,6 +185,11 @@ Bugs fixed
 .. _1821: https://www.varnish-cache.org/trac/ticket/1821
 .. _1823: https://www.varnish-cache.org/trac/ticket/1823
 .. _1826: https://www.varnish-cache.org/trac/ticket/1826
+.. _1813: https://www.varnish-cache.org/trac/ticket/1813
+.. _1810: https://www.varnish-cache.org/trac/ticket/1810
+.. _1807: https://www.varnish-cache.org/trac/ticket/1807
+.. _1804: https://www.varnish-cache.org/trac/ticket/1804
+.. _1801: https://www.varnish-cache.org/trac/ticket/1801
 
 
 ================================



From fgsch at lodoss.net  Mon May 30 17:19:06 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Mon, 30 May 2016 19:19:06 +0200
Subject: [master] 5d27296 Don't emit multiple Age or Accept-Ranges headers
Message-ID: <E1b7QqI-0000EB-5L@project.varnish-software.com>


commit 5d272966fec974c8b8d511734816eaf9b3951b2b
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Sun May 29 14:57:00 2016 +0100

    Don't emit multiple Age or Accept-Ranges headers
    
    This might happen during pass'd requests or in a multi-tier setup.
    Fixes #1955.

diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index 23d8d58..90235c1 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -119,7 +119,7 @@ vbf_beresp2obj(struct busyobj *bo)
 	}
 
 	l2 = http_EstimateWS(bo->beresp,
-	    bo->uncacheable ? HTTPH_R_PASS : HTTPH_A_INS);
+	    bo->uncacheable ? HTTPH_A_PASS : HTTPH_A_INS);
 	l += l2;
 
 	if (bo->uncacheable)
@@ -143,7 +143,7 @@ vbf_beresp2obj(struct busyobj *bo)
 	bp = ObjSetAttr(bo->wrk, bo->fetch_objcore, OA_HEADERS, l2, NULL);
 	AN(bp);
 	HTTP_Encode(bo->beresp, bp, l2,
-	    bo->uncacheable ? HTTPH_R_PASS : HTTPH_A_INS);
+	    bo->uncacheable ? HTTPH_A_PASS : HTTPH_A_INS);
 
 	if (http_GetHdr(bo->beresp, H_Last_Modified, &b))
 		AZ(ObjSetDouble(bo->wrk, bo->fetch_objcore, OA_LASTMODIFIED,
diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index ef73ccd..4db4fe7 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -278,7 +278,7 @@ cnt_transmit(struct worker *wrk, struct req *req)
 
 		if (cache_param->http_range_support &&
 		    http_IsStatus(req->resp, 200)) {
-			http_SetHeader(req->resp, "Accept-Ranges: bytes");
+			http_ForceHeader(req->resp, H_Accept_Ranges, "bytes");
 			if (sendbody && http_GetHdr(req->http, H_Range, &r))
 				VRG_dorange(req, r);
 		}
diff --git a/bin/varnishtest/tests/c00071.vtc b/bin/varnishtest/tests/c00071.vtc
index e589386..50574df 100644
--- a/bin/varnishtest/tests/c00071.vtc
+++ b/bin/varnishtest/tests/c00071.vtc
@@ -14,7 +14,7 @@ server s1 {
 varnish v1 -vcl+backend {
 	import debug;
 	sub vcl_deliver {
-		debug.workspace_allocate(client, debug.workspace_free(client) - 150);
+		debug.workspace_allocate(client, debug.workspace_free(client) - 160);
 
 		if (req.url ~ "/bar") {
 			set resp.http.x-foo = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz";
diff --git a/bin/varnishtest/tests/r01955.vtc b/bin/varnishtest/tests/r01955.vtc
new file mode 100644
index 0000000..756ebc0
--- /dev/null
+++ b/bin/varnishtest/tests/r01955.vtc
@@ -0,0 +1,22 @@
+varnishtest "Correct handling multiple Age headers in pass mode"
+
+# This affects Accept-Range as well but it's not possible to test
+# for that.
+
+server s1 {
+	 rxreq
+	 txresp -hdr "Cache-Control: max-age=1" -hdr "Age: 2"
+} -start
+
+varnish v1 -vcl+backend {
+	import std;
+	sub vcl_deliver {
+		std.collect(resp.http.age);
+	}
+} -start
+
+client c1 {
+	 txreq
+	 rxresp
+	 expect resp.http.age == "2"
+} -run
diff --git a/include/tbl/http_headers.h b/include/tbl/http_headers.h
index 0d973c7..5ada35a 100644
--- a/include/tbl/http_headers.h
+++ b/include/tbl/http_headers.h
@@ -43,75 +43,78 @@
 #define HTTPH_R_PASS	(1 << 0)	/* Request (c->b) in pass mode */
 #define HTTPH_R_FETCH	(1 << 1)	/* Request (c->b) for fetch */
 #define HTTPH_A_INS	(1 << 2)	/* Response (b->o) for insert */
+#define HTTPH_A_PASS	(1 << 3)	/* Response (b->o) for pass */
 #endif
 
 /* Shorthand for this file only, to keep table narrow */
 
-#if defined(P) || defined(F) || defined(I) || defined(H)
+#if defined(P) || defined(F) || defined(I) || defined(H) || defined(S)
 #error "Macro overloading"  // Trust but verify
 #endif
 
 #define P HTTPH_R_PASS
 #define F HTTPH_R_FETCH
 #define I HTTPH_A_INS
+#define S HTTPH_A_PASS
 #define H(s,e,f) HTTPH(s, e, f)
 
-H("Keep-Alive",		H_Keep_Alive,		P|F  )	// 2068
-H("Accept",		H_Accept,		0    )	// 2616 14.1
-H("Accept-Charset",	H_Accept_Charset,	0    )	// 2616 14.2
-H("Accept-Encoding",	H_Accept_Encoding,	0    )	// 2616 14.3
-H("Accept-Language",	H_Accept_Language,	0    )	// 2616 14.4
-H("Accept-Ranges",	H_Accept_Ranges,	  F|I)	// 2616 14.5
-H("Age",		H_Age,			    I)	// 2616 14.6
-H("Allow",		H_Allow,		0    )	// 2616 14.7
-H("Authorization",	H_Authorization,	0    )	// 2616 14.8
-H("Cache-Control",	H_Cache_Control,	  F  )	// 2616 14.9
-H("Connection",		H_Connection,		P|F|I)	// 2616 14.10
-H("Content-Encoding",	H_Content_Encoding,	0    )	// 2616 14.11
-H("Content-Language",	H_Content_Language,	0    )	// 2616 14.12
-H("Content-Length",	H_Content_Length,	  F  )	// 2616 14.13
-H("Content-Location",	H_Content_Location,	0    )	// 2616 14.14
-H("Content-MD5",	H_Content_MD5,		0    )	// 2616 14.15
-H("Content-Range",	H_Content_Range,	  F|I)	// 2616 14.16
-H("Content-Type",	H_Content_Type,		0    )	// 2616 14.17
-H("Cookie",		H_Cookie,		0    )	// 6265 4.2
-H("Date",		H_Date,			0    )	// 2616 14.18
-H("ETag",		H_ETag,			0    )	// 2616 14.19
-H("Expect",		H_Expect,		0    )	// 2616 14.20
-H("Expires",		H_Expires,		0    )	// 2616 14.21
-H("From",		H_From,			0    )	// 2616 14.22
-H("Host",		H_Host,			0    )	// 2616 14.23
-H("HTTP2-Settings",	H_HTTP2_Settings,	P|F|I)	// httpbis-http2-16.txt
-H("If-Match",		H_If_Match,		  F  )	// 2616 14.24
-H("If-Modified-Since",	H_If_Modified_Since,	  F  )	// 2616 14.25
-H("If-None-Match",	H_If_None_Match,	  F  )	// 2616 14.26
-H("If-Range",		H_If_Range,		  F  )	// 2616 14.27
-H("If-Unmodified-Since",H_If_Unmodified_Since,	  F  )	// 2616 14.28
-H("Last-Modified",	H_Last_Modified,	0    )	// 2616 14.29
-H("Location",		H_Location,		0    )	// 2616 14.30
-H("Max-Forwards",	H_Max_Forwards,		0    )	// 2616 14.31
-H("Pragma",		H_Pragma,		0    )	// 2616 14.32
-H("Proxy-Authenticate",	H_Proxy_Authenticate,	  F|I)	// 2616 14.33
-H("Proxy-Authorization",H_Proxy_Authorization,	  F|I)	// 2616 14.34
-H("Range",		H_Range,		  F|I)	// 2616 14.35
-H("Referer",		H_Referer,		0    )	// 2616 14.36
-H("Retry-After",	H_Retry_After,		0    )	// 2616 14.37
-H("Server",		H_Server,		0    )	// 2616 14.38
-H("Set-Cookie",		H_Set_Cookie,		0    )	// 6265 4.1
-H("TE",			H_TE,			P|F|I)	// 2616 14.39
-H("Trailer",		H_Trailer,		P|F|I)	// 2616 14.40
-H("Transfer-Encoding",	H_Transfer_Encoding,	P|F|I)	// 2616 14.41
-H("Upgrade",		H_Upgrade,		P|F|I)	// 2616 14.42
-H("User-Agent",		H_User_Agent,		0    )	// 2616 14.43
-H("Vary",		H_Vary,			0    )	// 2616 14.44
-H("Via",		H_Via,			0    )	// 2616 14.45
-H("Warning",		H_Warning,		0    )	// 2616 14.46
-H("WWW-Authenticate",	H_WWW_Authenticate,	0    )	// 2616 14.47
-H("X-Forwarded-For",	H_X_Forwarded_For,	0    )	// No RFC
+H("Keep-Alive",		H_Keep_Alive,		P|F  |S)	// 2068
+H("Accept",		H_Accept,		0      )	// 2616 14.1
+H("Accept-Charset",	H_Accept_Charset,	0      )	// 2616 14.2
+H("Accept-Encoding",	H_Accept_Encoding,	0      )	// 2616 14.3
+H("Accept-Language",	H_Accept_Language,	0      )	// 2616 14.4
+H("Accept-Ranges",	H_Accept_Ranges,	P|F|I  )	// 2616 14.5
+H("Age",		H_Age,			    I|S)	// 2616 14.6
+H("Allow",		H_Allow,		0      )	// 2616 14.7
+H("Authorization",	H_Authorization,	0      )	// 2616 14.8
+H("Cache-Control",	H_Cache_Control,	  F    )	// 2616 14.9
+H("Connection",		H_Connection,		P|F|I|S)	// 2616 14.10
+H("Content-Encoding",	H_Content_Encoding,	0      )	// 2616 14.11
+H("Content-Language",	H_Content_Language,	0      )	// 2616 14.12
+H("Content-Length",	H_Content_Length,	  F    )	// 2616 14.13
+H("Content-Location",	H_Content_Location,	0      )	// 2616 14.14
+H("Content-MD5",	H_Content_MD5,		0      )	// 2616 14.15
+H("Content-Range",	H_Content_Range,	  F|I  )	// 2616 14.16
+H("Content-Type",	H_Content_Type,		0      )	// 2616 14.17
+H("Cookie",		H_Cookie,		0      )	// 6265 4.2
+H("Date",		H_Date,			0      )	// 2616 14.18
+H("ETag",		H_ETag,			0      )	// 2616 14.19
+H("Expect",		H_Expect,		0      )	// 2616 14.20
+H("Expires",		H_Expires,		0      )	// 2616 14.21
+H("From",		H_From,			0      )	// 2616 14.22
+H("Host",		H_Host,			0      )	// 2616 14.23
+H("HTTP2-Settings",	H_HTTP2_Settings,	P|F|I|S)	// httpbis-http2-16.txt
+H("If-Match",		H_If_Match,		  F    )	// 2616 14.24
+H("If-Modified-Since",	H_If_Modified_Since,	  F    )	// 2616 14.25
+H("If-None-Match",	H_If_None_Match,	  F    )	// 2616 14.26
+H("If-Range",		H_If_Range,		  F    )	// 2616 14.27
+H("If-Unmodified-Since",H_If_Unmodified_Since,	  F    )	// 2616 14.28
+H("Last-Modified",	H_Last_Modified,	0      )	// 2616 14.29
+H("Location",		H_Location,		0      )	// 2616 14.30
+H("Max-Forwards",	H_Max_Forwards,		0      )	// 2616 14.31
+H("Pragma",		H_Pragma,		0      )	// 2616 14.32
+H("Proxy-Authenticate",	H_Proxy_Authenticate,	  F|I  )	// 2616 14.33
+H("Proxy-Authorization",H_Proxy_Authorization,	  F|I  )	// 2616 14.34
+H("Range",		H_Range,		  F|I  )	// 2616 14.35
+H("Referer",		H_Referer,		0      )	// 2616 14.36
+H("Retry-After",	H_Retry_After,		0      )	// 2616 14.37
+H("Server",		H_Server,		0      )	// 2616 14.38
+H("Set-Cookie",		H_Set_Cookie,		0      )	// 6265 4.1
+H("TE",			H_TE,			P|F|I|S)	// 2616 14.39
+H("Trailer",		H_Trailer,		P|F|I|S)	// 2616 14.40
+H("Transfer-Encoding",	H_Transfer_Encoding,	P|F|I|S)	// 2616 14.41
+H("Upgrade",		H_Upgrade,		P|F|I|S)	// 2616 14.42
+H("User-Agent",		H_User_Agent,		0      )	// 2616 14.43
+H("Vary",		H_Vary,			0      )	// 2616 14.44
+H("Via",		H_Via,			0      )	// 2616 14.45
+H("Warning",		H_Warning,		0      )	// 2616 14.46
+H("WWW-Authenticate",	H_WWW_Authenticate,	0      )	// 2616 14.47
+H("X-Forwarded-For",	H_X_Forwarded_For,	0      )	// No RFC
 
 #undef P
 #undef F
 #undef I
+#undef S
 #undef H
 
 /*lint -restore */



From fgsch at lodoss.net  Mon May 30 17:19:06 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Mon, 30 May 2016 19:19:06 +0200
Subject: [master] 2d4565b Check pcre_study() is working in configure
Message-ID: <E1b7QqI-0000E5-2t@project.varnish-software.com>


commit 2d4565b6bf10b4a2f68865fe0902b2299f2d08e5
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Mon May 30 18:03:27 2016 +0100

    Check pcre_study() is working in configure

diff --git a/autogen.des b/autogen.des
index 717c5cf..42b1508 100755
--- a/autogen.des
+++ b/autogen.des
@@ -6,8 +6,6 @@ set -ex
 
 make -k distclean > /dev/null 2>&1 || true
 
-OPTS=""
-
 # Prefer CLANG if we have it, and have not given preferences
 if [ -f /usr/bin/clang -a "x${CC}" = "x" ] ; then
 	CC=clang
@@ -20,14 +18,9 @@ else
 	DST="--prefix=/opt/varnish --mandir=/opt/varnish/man"
 fi
 
-if [ "x`uname -po`" = "xFreeBSD armv6" ] ; then
-	OPTS="${OPTS} --disable-pcre-jit"
-fi
-
 rm -f configure
 . ./autogen.sh 2>&1 | egrep -v "(subdir-objects|is in a subdirectory)"
 
-
 # autoconf prior to 2.62 has issues with zsh 4.2 and newer
 export CONFIG_SHELL=/bin/sh
 
@@ -39,5 +32,4 @@ export CONFIG_SHELL=/bin/sh
     --enable-developer-warnings \
     --enable-debugging-symbols \
     --enable-dependency-tracking \
-    $OPTS \
     "$@"
diff --git a/configure.ac b/configure.ac
index ad83ab7..2ae3f99 100644
--- a/configure.ac
+++ b/configure.ac
@@ -161,17 +161,29 @@ if test "$enable_pcre_jit" = yes; then
   AC_MSG_CHECKING(for PCRE JIT usability)
   save_CFLAGS="${CFLAGS}"
   CFLAGS="${PCRE_CFLAGS}"
-  AC_COMPILE_IFELSE(
-    [AC_LANG_PROGRAM([[#include "pcre.h"
-                       #if PCRE_MAJOR != 8 || PCRE_MINOR < 32
-                       #error no jit
-                       #endif]])],
+  save_LIBS="${CFLAGS}"
+  LIBS="${PCRE_LIBS}"
+  AC_RUN_IFELSE(
+    [AC_LANG_PROGRAM([[
+#include <pcre.h>
+#if PCRE_MAJOR != 8 || PCRE_MINOR < 32
+#error no jit
+#endif
+    ]],[[
+const char *error;
+pcre *re;
+int erroroffset;
+re = pcre_compile(".", 0, &error, &erroroffset, NULL);
+if (!pcre_study(re, PCRE_STUDY_JIT_COMPILE, &error))
+  return (1);
+    ]])],
     [AC_MSG_RESULT(yes)
      AC_DEFINE([USE_PCRE_JIT], [1], [Use the PCRE JIT compiler])
     ],
     [AC_MSG_RESULT(no)]
   )
   CFLAGS="${save_CFLAGS}"
+  LIBS="${save_LIBS}"
 fi
 
 



From fgsch at lodoss.net  Mon May 30 17:19:06 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Mon, 30 May 2016 19:19:06 +0200
Subject: [master] 416a919 Make now constant within a VCL subroutine
Message-ID: <E1b7QqI-0000EK-7F@project.varnish-software.com>


commit 416a919023c455f261f0be9a6a74d178f0dbf403
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Mon May 30 18:05:13 2016 +0100

    Make now constant within a VCL subroutine
    
    Idea from Devon H. O'Dell (dho at fastly dot com).

diff --git a/bin/varnishd/cache/cache_vrt.c b/bin/varnishd/cache/cache_vrt.c
index a82b1e4..efb2949 100644
--- a/bin/varnishd/cache/cache_vrt.c
+++ b/bin/varnishd/cache/cache_vrt.c
@@ -292,7 +292,7 @@ VRT_r_now(VRT_CTX)
 {
 
 	(void)ctx;
-	return (VTIM_real());
+	return (ctx->now);
 }
 
 /*--------------------------------------------------------------------*/



From phk at FreeBSD.org  Mon May 30 17:26:06 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Mon, 30 May 2016 19:26:06 +0200
Subject: [master] 9f27212 Make hit-for-pass cause HSH_MISS instead of
 HSH_HIT+OC_F_PASS.
Message-ID: <E1b7Qx4-0000nE-Bv@project.varnish-software.com>


commit 9f272127c6fba76e6758d7ab7ba6527d9aad98b0
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon May 30 17:21:33 2016 +0000

    Make hit-for-pass cause HSH_MISS instead of HSH_HIT+OC_F_PASS.
    
    it is more consistent to hit vcl_miss{} every time for these objects,
    as opposed to vcl_pass{} most of the time but occationally vcl_miss{}.
    
    This also almost entirely takes the TTL of HFP objects out of the
    picture, in the sense that any cacheable object fetched as a result
    of HFP will be put into the cache and be usable, rather than the
    HFP preventing cacheable objects for whatever TTL the HFP might
    have.
    
    See c00076 for an example of this.

diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index 7f8bebb..b808670 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -435,10 +435,18 @@ HSH_Lookup(struct req *req, struct objcore **ocp, struct objcore **bocp,
 			/* If still valid, use it */
 			assert(oh->refcnt > 1);
 			assert(oc->objhead == oh);
-			oc->refcnt++;
-			if (oc->hits < LONG_MAX)
-				oc->hits++;
+			if (oc->flags & OC_F_PASS) {
+				wrk->stats->cache_hitpass++;
+				oc = NULL;
+				*bocp = hsh_insert_busyobj(wrk, oh);
+			} else {
+				oc->refcnt++;
+				if (oc->hits < LONG_MAX)
+					oc->hits++;
+			}
 			Lck_Unlock(&oh->mtx);
+			if (oc == NULL) 
+				return (HSH_MISS);
 			assert(HSH_DerefObjHead(wrk, &oh));
 			*ocp = oc;
 			return (HSH_HIT);
@@ -450,6 +458,12 @@ HSH_Lookup(struct req *req, struct objcore **ocp, struct objcore **bocp,
 		}
 	}
 
+	if (exp_oc != NULL && exp_oc->flags & OC_F_PASS) {
+		wrk->stats->cache_hitpass++;
+		exp_oc = NULL;
+		busy_found = 0;
+	}
+
 	if (exp_oc != NULL) {
 		assert(oh->refcnt > 1);
 		assert(exp_oc->objhead == oh);
diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index 4db4fe7..a2d830c 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -401,24 +401,7 @@ cnt_lookup(struct worker *wrk, struct req *req)
 	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
 	AZ(oc->flags & OC_F_BUSY);
 	req->objcore = oc;
-
-	if ((oc->flags & OC_F_PASS) && busy != NULL) {
-		/* Treat a graced Hit-For-Pass as a miss */
-		(void)HSH_DerefObjCore(wrk, &req->objcore);
-		AZ(req->objcore);
-		req->objcore = busy;
-		req->req_step = R_STP_MISS;
-		return (REQ_FSM_MORE);
-	} else if (oc->flags & OC_F_PASS) {
-		/* Found a hit-for-pass */
-		VSLb(req->vsl, SLT_Debug, "XXXX HIT-FOR-PASS");
-		VSLb(req->vsl, SLT_HitPass, "%u",
-		    ObjGetXID(wrk, req->objcore));
-		(void)HSH_DerefObjCore(wrk, &req->objcore);
-		wrk->stats->cache_hitpass++;
-		req->req_step = R_STP_PASS;
-		return (REQ_FSM_MORE);
-	}
+	AZ(oc->flags & OC_F_PASS);
 
 	VSLb(req->vsl, SLT_Hit, "%u", ObjGetXID(wrk, req->objcore));
 
diff --git a/bin/varnishd/flint.lnt b/bin/varnishd/flint.lnt
index 89bfae9..190e4e1 100644
--- a/bin/varnishd/flint.lnt
+++ b/bin/varnishd/flint.lnt
@@ -157,3 +157,4 @@
 -e663	// Suspicious array to pointer conversion
 -e778   // Constant expression evaluates to 0 in operation '___'
 -e736	// Loss of precision (___) (___ bits to ___ bits)
+-e655	// bitwise compatible enums
diff --git a/bin/varnishtest/tests/c00076.vtc b/bin/varnishtest/tests/c00076.vtc
new file mode 100644
index 0000000..fe193be
--- /dev/null
+++ b/bin/varnishtest/tests/c00076.vtc
@@ -0,0 +1,113 @@
+varnishtest "Complex son-of-hit-for-pass test"
+
+server s1 {
+	rxreq
+	txresp -hdr "Connection: close" -hdr "Set-Cookie: c1" -bodylen 1
+} -start
+
+varnish v1 -vcl+backend {} -start
+
+client c1 {
+	txreq
+	rxresp
+	expect resp.bodylen == 1
+	delay .1
+} -run
+
+server s1 {
+	rxreq
+	expect req.http.foo == xyz
+	txresp -hdr "Set-Cookie: c2" -hdr "Vary: foo" -bodylen 2
+	rxreq
+	expect req.http.foo == abc
+	txresp -hdr "Connection: close" -hdr "Set-Cookie: c2" -hdr "Vary: foo" -bodylen 3
+} -start
+
+client c1 {
+	txreq -hdr "foo: xyz"
+	rxresp
+	expect resp.bodylen == 2
+	delay .1
+} -run
+
+client c1 {
+	txreq -hdr "foo: abc"
+	rxresp
+	expect resp.bodylen == 3
+	delay .1
+} -run
+
+server s1 {
+	rxreq
+	expect req.http.foo == 123
+	txresp -hdr "Connection: close" -hdr "Vary: foo" -bodylen 4
+} -start
+
+client c1 {
+	txreq -hdr "foo: 123"
+	rxresp
+	expect resp.bodylen == 4
+	delay .1
+} -run
+
+client c1 {
+	# Cache hit
+	txreq -hdr "foo: 123"
+	rxresp
+	expect resp.bodylen == 4
+	delay .1
+} -run
+
+server s1 {
+	rxreq
+	expect req.http.foo == 987
+	txresp -hdr "Connection: close" -hdr "Vary: foo" -bodylen 5
+} -start
+
+client c1 {
+	txreq -hdr "foo: 987"
+	rxresp
+	expect resp.bodylen == 5
+	delay .1
+} -run
+
+client c1 {
+	# Cache hit
+	txreq -hdr "foo: 123"
+	rxresp
+	expect resp.bodylen == 4
+	delay .1
+} -run
+
+server s1 {
+	rxreq
+	expect req.http.foo == 000
+	txresp -hdr "Connection: close" -bodylen 6
+} -start
+
+client c1 {
+	txreq -hdr "foo: 000"
+	rxresp
+	expect resp.bodylen == 6
+	delay .1
+
+	txreq -hdr "foo: 123"
+	rxresp
+	expect resp.bodylen == 6
+	delay .1
+
+	txreq -hdr "foo: abc"
+	rxresp
+	expect resp.bodylen == 6
+	delay .1
+
+	txreq -hdr "foo: 987"
+	rxresp
+	expect resp.bodylen == 6
+	delay .1
+
+	txreq -hdr "foo: xyz"
+	rxresp
+	expect resp.bodylen == 6
+	delay .1
+} -run
diff --git a/bin/varnishtest/tests/r01818.vtc b/bin/varnishtest/tests/r01818.vtc
index 2fe8ae3..c7dffcd 100644
--- a/bin/varnishtest/tests/r01818.vtc
+++ b/bin/varnishtest/tests/r01818.vtc
@@ -74,5 +74,5 @@ client c2 {
 	barrier b2 sync
 	txreq -hdr "c: 1"
 	rxresp
-	expect resp.http.pass == "1"
+	expect resp.http.miss == "1"
 } -run



From fgsch at lodoss.net  Mon May 30 18:03:05 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Mon, 30 May 2016 20:03:05 +0200
Subject: [master] f83a75d Polish
Message-ID: <E1b7RWr-0002rC-Oh@project.varnish-software.com>


commit f83a75dd859e5dc18fed5fa8ecc29d69aa5d3cc6
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Mon May 30 18:38:37 2016 +0100

    Polish

diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index b808670..c552416 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -445,7 +445,7 @@ HSH_Lookup(struct req *req, struct objcore **ocp, struct objcore **bocp,
 					oc->hits++;
 			}
 			Lck_Unlock(&oh->mtx);
-			if (oc == NULL) 
+			if (oc == NULL)
 				return (HSH_MISS);
 			assert(HSH_DerefObjHead(wrk, &oh));
 			*ocp = oc;
diff --git a/bin/varnishd/cache/cache_vrt.c b/bin/varnishd/cache/cache_vrt.c
index efb2949..b6c5879 100644
--- a/bin/varnishd/cache/cache_vrt.c
+++ b/bin/varnishd/cache/cache_vrt.c
@@ -290,8 +290,7 @@ VRT_hashdata(VRT_CTX, const char *str, ...)
 double
 VRT_r_now(VRT_CTX)
 {
-
-	(void)ctx;
+	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
 	return (ctx->now);
 }
 
@@ -500,10 +499,7 @@ VRT_purge(VRT_CTX, double ttl, double grace, double keep)
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
 	CHECK_OBJ_NOTNULL(ctx->req, REQ_MAGIC);
 	CHECK_OBJ_NOTNULL(ctx->req->wrk, WORKER_MAGIC);
-	if (ctx->method == VCL_MET_HIT)
-		HSH_Purge(ctx->req->wrk, ctx->req->objcore->objhead,
-		    ttl, grace, keep);
-	else if (ctx->method == VCL_MET_MISS)
+	if (ctx->method == VCL_MET_HIT || ctx->method == VCL_MET_MISS)
 		HSH_Purge(ctx->req->wrk, ctx->req->objcore->objhead,
 		    ttl, grace, keep);
 }



From phk at FreeBSD.org  Mon May 30 21:17:05 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Mon, 30 May 2016 23:17:05 +0200
Subject: [master] bebe5e4 Coverity polishing.
Message-ID: <E1b7UYb-0005gU-N0@project.varnish-software.com>


commit bebe5e407f5343a1d7897d68b1d3ea44addf3c83
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon May 30 21:14:52 2016 +0000

    Coverity polishing.

diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index dfc7235..055c841 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -833,7 +833,7 @@ ccf_config_label(struct cli *cli, const char * const *av, void *priv)
 	if (lbl == NULL) {
 		ALLOC_OBJ(lbl, VCL_MAGIC);
 		AN(lbl);
-		strcpy(lbl->state, VCL_TEMP_LABEL);
+		bprintf(lbl->state, "%s", VCL_TEMP_LABEL);
 		lbl->temp = VCL_TEMP_WARM;
 		lbl->loaded_name = strdup(av[2]);
 		AN(lbl->loaded_name);
diff --git a/bin/varnishtest/vtc_main.c b/bin/varnishtest/vtc_main.c
index f30acf0..48f2a71 100644
--- a/bin/varnishtest/vtc_main.c
+++ b/bin/varnishtest/vtc_main.c
@@ -476,7 +476,7 @@ static int
 read_file(const char *fn, int ntest)
 {
 	struct vtc_tst *tp;
-	char *p;
+	char *p, *q;
 
 	p = VFIL_readfile(NULL, fn, NULL);
 	if (p == NULL) {
@@ -484,23 +484,24 @@ read_file(const char *fn, int ntest)
 		    fn, strerror(errno));
 		return (2);
 	}
-	for (;p != NULL && *p != '\0'; p++) {
-		if (vct_islws(*p))
+	for (q = p ;q != NULL && *q != '\0'; q++) {
+		if (vct_islws(*q))
 			continue;
-		if (*p != '#')
+		if (*q != '#')
 			break;
-		p = strchr(p, '\n');
+		q = strchr(q, '\n');
 	}
 
-	if (p == NULL || *p == '\0') {
+	if (q == NULL || *q == '\0') {
 		fprintf(stderr, "File \"%s\" has no content.\n", fn);
+		free(p);
 		return (2);
 	}
 
-	if (strncmp(p, "varnishtest", 11) || !isspace(p[11])) {
+	if (strncmp(q, "varnishtest", 11) || !isspace(q[11])) {
 		fprintf(stderr,
-		    "File \"%s\" doesn't start with 'varnishtest'\n",
-		    fn);
+		    "File \"%s\" doesn't start with 'varnishtest'\n", fn);
+		free(p);
 		return(2);
 	}
 	ALLOC_OBJ(tp, TST_MAGIC);



From phk at FreeBSD.org  Tue May 31 08:36:07 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 31 May 2016 10:36:07 +0200
Subject: [master] bd535f5 Sort file list
Message-ID: <E1b7f9j-0003AC-BH@project.varnish-software.com>


commit bd535f51c8ca56c92d3a7d68c9d6ae0206b99ab5
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue May 31 06:16:10 2016 +0000

    Sort file list

diff --git a/lib/libvarnish/Makefile.am b/lib/libvarnish/Makefile.am
index 5dd728e..d98cfca 100644
--- a/lib/libvarnish/Makefile.am
+++ b/lib/libvarnish/Makefile.am
@@ -10,30 +10,30 @@ pkglib_LTLIBRARIES = libvarnish.la
 libvarnish_la_LDFLAGS = $(AM_LDFLAGS) -avoid-version
 
 libvarnish_la_SOURCES = \
-	vav.c \
-	vas.c \
 	binary_heap.c \
-	vsub.c \
+	vas.c \
+	vav.c \
 	vcli_proto.c \
 	vcli_serve.c \
-	vfl.c \
-	vnum.c \
-	vtim.c \
-	vtcp.c \
 	vct.c \
 	version.c \
 	vev.c \
 	vfil.c \
+	vfl.c \
 	vin.c \
 	vlu.c \
 	vmb.c \
+	vnum.c \
 	vpf.c \
 	vre.c \
 	vrnd.c \
 	vsa.c \
 	vsb.c \
 	vsha256.c \
-	vss.c
+	vss.c \
+	vsub.c \
+	vtcp.c \
+	vtim.c
 
 libvarnish_la_CFLAGS = \
 	-DVARNISH_STATE_DIR='"${VARNISH_STATE_DIR}"' \



From phk at FreeBSD.org  Tue May 31 08:36:07 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 31 May 2016 10:36:07 +0200
Subject: [master] 2e806f0 Change the VCL_BYTES to be "long long" rather than
 double.
Message-ID: <E1b7f9j-0003AI-DO@project.varnish-software.com>


commit 2e806f088bf1be1152f853f1eaf9939a1604ea7f
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue May 31 08:35:17 2016 +0000

    Change the VCL_BYTES to be "long long" rather than double.

diff --git a/bin/varnishd/cache/cache_vrt.c b/bin/varnishd/cache/cache_vrt.c
index b6c5879..d3e832d 100644
--- a/bin/varnishd/cache/cache_vrt.c
+++ b/bin/varnishd/cache/cache_vrt.c
@@ -474,8 +474,8 @@ VRT_ban_string(VRT_CTX, const char *str)
 	VAV_Free(av);
 }
 
-ssize_t
-VRT_CacheReqBody(VRT_CTX, size_t maxsize)
+VCL_BYTES
+VRT_CacheReqBody(VRT_CTX, VCL_BYTES maxsize)
 {
 
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
diff --git a/bin/varnishd/storage/storage.h b/bin/varnishd/storage/storage.h
index 68d7106..113b4da 100644
--- a/bin/varnishd/storage/storage.h
+++ b/bin/varnishd/storage/storage.h
@@ -79,9 +79,10 @@ typedef struct storage *sml_alloc_f(const struct stevedore *, size_t size);
 typedef void sml_free_f(struct storage *);
 
 /* Prototypes for VCL variable responders */
-#define VRTSTVTYPE(ct) typedef ct storage_var_##ct(const struct stevedore *);
+#define VRTSTVVAR(nm,vt,ct,def) \
+    typedef ct stv_var_##nm(const struct stevedore *);
 #include "tbl/vrt_stv_var.h"
-#undef VRTSTVTYPE
+#undef VRTSTVVAR
 
 /*--------------------------------------------------------------------*/
 
@@ -112,7 +113,7 @@ struct stevedore {
 	/* Only if LRU is used */
 	struct lru		*lru;
 
-#define VRTSTVVAR(nm, vtype, ctype, dval) storage_var_##ctype *var_##nm;
+#define VRTSTVVAR(nm, vtype, ctype, dval) stv_var_##nm *var_##nm;
 #include "tbl/vrt_stv_var.h"
 #undef VRTSTVVAR
 
diff --git a/bin/varnishd/storage/storage_malloc.c b/bin/varnishd/storage/storage_malloc.c
index a936b2d..e6022e8 100644
--- a/bin/varnishd/storage/storage_malloc.c
+++ b/bin/varnishd/storage/storage_malloc.c
@@ -38,6 +38,7 @@
 #include "storage/storage.h"
 #include "storage/storage_simple.h"
 
+#include "vrt.h"
 #include "vnum.h"
 
 struct sma_sc {
@@ -147,7 +148,7 @@ sma_free(struct storage *s)
 	free(sma);
 }
 
-static double
+static VCL_BYTES __match_proto__(stv_var_used_space)
 sma_used_space(const struct stevedore *st)
 {
 	struct sma_sc *sma_sc;
@@ -156,7 +157,7 @@ sma_used_space(const struct stevedore *st)
 	return (sma_sc->sma_alloc);
 }
 
-static double
+static VCL_BYTES __match_proto__(stv_var_free_space)
 sma_free_space(const struct stevedore *st)
 {
 	struct sma_sc *sma_sc;
diff --git a/include/vrt.h b/include/vrt.h
index bcaa2cd..ef679bb 100644
--- a/include/vrt.h
+++ b/include/vrt.h
@@ -31,8 +31,6 @@
  * NB: When this file is changed, lib/libvcc/generate.py *MUST* be rerun.
  */
 
-#include <sys/types.h>
-
 /***********************************************************************
  * Major and minor VRT API versions.
  *
@@ -40,6 +38,9 @@
  * Whenever something is deleted or changed in a way which is not
  * binary/load-time compatible, increment MAJOR version
  *
+ *
+ * 4.0:
+ *	VCL_BYTES changed to long long
  * 3.2:
  *	vrt_backend grew .proxy_header field
  *	vrt_ctx grew .sp field.
@@ -52,9 +53,9 @@
  *
  */
 
-#define VRT_MAJOR_VERSION	3U
+#define VRT_MAJOR_VERSION	4U
 
-#define VRT_MINOR_VERSION	2U
+#define VRT_MINOR_VERSION	0U
 
 
 /***********************************************************************/
@@ -83,7 +84,7 @@ typedef const struct vrt_acl *			VCL_ACL;
 typedef const struct director *			VCL_BACKEND;
 typedef const struct vmod_priv *		VCL_BLOB;
 typedef unsigned				VCL_BOOL;
-typedef double					VCL_BYTES;
+typedef long long				VCL_BYTES;
 typedef double					VCL_DURATION;
 typedef const char *				VCL_ENUM;
 typedef const struct gethdr_s *			VCL_HEADER;
@@ -268,7 +269,7 @@ int VRT_acl_match(VRT_CTX, VCL_ACL, VCL_IP);
 
 /* req related */
 
-ssize_t VRT_CacheReqBody(VRT_CTX, size_t maxsize);
+VCL_BYTES VRT_CacheReqBody(VRT_CTX, VCL_BYTES maxsize);
 
 /* Regexp related */
 void VRT_re_init(void **, const char *);
diff --git a/lib/libvcc/generate.py b/lib/libvcc/generate.py
index c0faf4d..5da2c95 100755
--- a/lib/libvcc/generate.py
+++ b/lib/libvcc/generate.py
@@ -812,7 +812,7 @@ vcl_fixed_token(const char *p, const char **q)
 				continue
 			if len(j) == 2:
 				fo.write("\t\tM2('%s', %s);\n" %
-				         (j[1], emit[j]))
+				    (j[1], emit[j]))
 			elif len(j) == 1:
 				fo.write("\t\tM1();\n")
 				need_ret = False
@@ -987,8 +987,8 @@ fo.write("#endif\n")
 
 fo.write("\n#ifdef VCL_MET_MAC\n")
 for i in ll:
-	fo.write("VCL_MET_MAC(%s, %s, %s," % (i[0].lower(), i[0].upper(),
-	                                      i[1]))
+	fo.write("VCL_MET_MAC(%s, %s, %s," %
+	    (i[0].lower(), i[0].upper(), i[1]))
 	p = " (\n\t"
 	for j in sorted(i[2]):
 		fo.write("%s(1U << VCL_RET_%s)" % (p, j.upper()))
@@ -1035,8 +1035,7 @@ def tbl40(a, b):
 fo.write("\n/* VCL Methods */\n")
 n = 1
 for i in returns:
-	fo.write(tbl40("#define VCL_MET_%s" % i[0].upper(),
-	               "(1U << %d)\n" % n))
+	fo.write(tbl40("#define VCL_MET_%s" % i[0].upper(), "(1U << %d)\n" % n))
 	n += 1
 
 fo.write("\n" + tbl40("#define VCL_MET_MAX", "%d\n" % n))
@@ -1152,8 +1151,7 @@ def one_var(nm, spec):
 	else:
 		fo.write('\t    "VRT_r_%s(ctx)",\n' % cnam)
 		if nm == i[0]:
-			fh.write("VCL_" + typ +
-			         " VRT_r_%s(VRT_CTX);\n" % cnam)
+			fh.write("VCL_" + typ + " VRT_r_%s(VRT_CTX);\n" % cnam)
 	restrict(fo, spec[2])
 
 	if len(spec[3]) == 0:
@@ -1240,38 +1238,12 @@ fo = open(join(buildroot, "include/tbl/vrt_stv_var.h"), "w")
 
 file_header(fo)
 
-fo.write("""
-#ifndef VRTSTVTYPE
-#define VRTSTVTYPE(ct)
-#define VRTSTVTYPEX
-#endif
-#ifndef VRTSTVVAR
-#define VRTSTVVAR(nm, vtype, ctype, dval)
-#define VRTSTVVARX
-#endif
-""")
-
-x = dict()
 for i in stv_variables:
 	ct = vcltypes[i[1]]
-	if not ct in x:
-		fo.write("VRTSTVTYPE(" + ct + ")\n")
-		x[ct] = 1
 	fo.write("VRTSTVVAR(" + i[0] + ",\t" + i[1] + ",\t")
 	fo.write(ct + ",\t" + i[2] + ")")
 	fo.write("\n")
 
-fo.write("""
-#ifdef VRTSTVTYPEX
-#undef VRTSTVTYPEX
-#undef VRTSTVTYPE
-#endif
-#ifdef VRTSTVVARX
-#undef VRTSTVVARX
-#undef VRTSTVVAR
-#endif
-""")
-
 fo.close()
 
 #######################################################################



From hermunn at varnish-software.com  Tue May 31 11:11:07 2016
From: hermunn at varnish-software.com (Pål Hermunn Johansen)
Date: Tue, 31 May 2016 13:11:07 +0200
Subject: [master] 11b91d1 Don't allow partial allocation of stevedore space
 for ESI
Message-ID: <E1b7hZj-0003PV-G1@project.varnish-software.com>


commit 11b91d198f828ef7f567464a990bb78f7e5e6893
Author: P?l Hermunn Johansen <hermunn at varnish-software.com>
Date:   Tue May 10 11:37:05 2016 +0200

    Don't allow partial allocation of stevedore space for ESI
    
    When allocating ESI memory through objallocwithnuke, tell the sevedore
    in storage_simple.c that getting less than requested, is not
    interesting. At the same time, eliminate the possibility of getting
    less memory than requested in sml_trimstore.
    
    Fixes #1941

diff --git a/bin/varnishd/storage/storage_simple.c b/bin/varnishd/storage/storage_simple.c
index f6b70e1..ef427cc 100644
--- a/bin/varnishd/storage/storage_simple.c
+++ b/bin/varnishd/storage/storage_simple.c
@@ -43,15 +43,25 @@
 
 #include "vtim.h"
 
+/* Flags for allocating memory in sml_stv_alloc */
+#define LESS_MEM_ALLOCED_IS_OK	1
+
 /*-------------------------------------------------------------------*/
 
 static struct storage *
-sml_stv_alloc(const struct stevedore *stv, size_t size)
+sml_stv_alloc(const struct stevedore *stv, size_t size, int flags)
 {
 	struct storage *st;
 
 	CHECK_OBJ_NOTNULL(stv, STEVEDORE_MAGIC);
 
+	if (!(flags & LESS_MEM_ALLOCED_IS_OK)) {
+		if (size > cache_param->fetch_maxchunksize)
+			return (NULL);
+		else
+			return (stv->sml_alloc(stv, size));
+	}
+
 	if (size > cache_param->fetch_maxchunksize)
 		size = cache_param->fetch_maxchunksize;
 
@@ -334,7 +344,8 @@ sml_iterator(struct worker *wrk, struct objcore *oc,
  */
 
 static struct storage *
-objallocwithnuke(struct worker *wrk, const struct stevedore *stv, size_t size)
+objallocwithnuke(struct worker *wrk, const struct stevedore *stv, size_t size,
+    int flags)
 {
 	struct storage *st = NULL;
 	unsigned fail;
@@ -342,14 +353,17 @@ objallocwithnuke(struct worker *wrk, const struct stevedore *stv, size_t size)
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(stv, STEVEDORE_MAGIC);
 
-	if (size > cache_param->fetch_maxchunksize)
+	if (size > cache_param->fetch_maxchunksize) {
+		if (!(flags & LESS_MEM_ALLOCED_IS_OK))
+			return (NULL);
 		size = cache_param->fetch_maxchunksize;
+	}
 
 	assert(size <= UINT_MAX);	/* field limit in struct storage */
 
 	for (fail = 0; fail <= cache_param->nuke_limit; fail++) {
 		/* try to allocate from it */
-		st = sml_stv_alloc(stv, size);
+		st = sml_stv_alloc(stv, size, flags);
 		if (st != NULL)
 			break;
 
@@ -388,7 +402,8 @@ sml_getspace(struct worker *wrk, struct objcore *oc, ssize_t *sz,
 		return (1);
 	}
 
-	st = objallocwithnuke(wrk, oc->stobj->stevedore, *sz);
+	st = objallocwithnuke(wrk, oc->stobj->stevedore, *sz,
+	    LESS_MEM_ALLOCED_IS_OK);
 	if (st == NULL)
 		return (0);
 
@@ -455,14 +470,10 @@ sml_trimstore(struct worker *wrk, struct objcore *oc)
 	if (st->space - st->len < 512)
 		return;
 
-	st1 = sml_stv_alloc(stv, st->len);
+	st1 = sml_stv_alloc(stv, st->len, 0);
 	if (st1 == NULL)
 		return;
-
-	if (st1->space < st->len) {
-		sml_stv_free(stv, st1);
-		return;
-	}
+	assert(st1->space >= st->len);
 
 	memcpy(st1->ptr, st->ptr, st->len);
 	st1->len = st->len;
@@ -605,7 +616,8 @@ sml_setattr(struct worker *wrk, struct objcore *oc, enum obj_attr attr,
 		}							\
 		if (len == 0)						\
 			break;						\
-		o->aa_##l = objallocwithnuke(wrk, oc->stobj->stevedore,	len); \
+		o->aa_##l = objallocwithnuke(wrk, oc->stobj->stevedore,	\
+		    len, 0);						\
 		if (o->aa_##l == NULL)					\
 			break;						\
 		CHECK_OBJ_NOTNULL(o->aa_##l, STORAGE_MAGIC);		\
diff --git a/bin/varnishtest/tests/r01941.vtc b/bin/varnishtest/tests/r01941.vtc
new file mode 100644
index 0000000..2531ded
--- /dev/null
+++ b/bin/varnishtest/tests/r01941.vtc
@@ -0,0 +1,126 @@
+
+varnishtest "ESI memory should only be fully allocated"
+
+# In this test case, the cache is almost filled. Then we force the
+# allocation of ESI memory that does not fit in the cache, is bigger
+# than fetch_chunksize, but where half the amount can be allocated by
+# the stevedore.
+
+server s1 {
+	rxreq
+	expect req.url == /big
+	txresp -bodylen 1037480
+	rxreq
+	txresp -body {<html>
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+</html>}
+	rxreq
+	expect req.url == /long1234567890123456789012345678901234567890.txt
+	txresp -body {fo}
+} -start
+
+varnish v1 -arg "-pfetch_chunksize=4k" \
+	-arg "-smalloc,1m" -vcl+backend {
+	sub vcl_backend_response {
+		set beresp.do_esi = true;
+	}
+} -start
+
+client c1 {
+	txreq -url /big
+	rxresp
+	expect resp.bodylen == 1037480
+	txreq -url /
+	rxresp
+} -run



From hermunn at varnish-software.com  Tue May 31 11:19:05 2016
From: hermunn at varnish-software.com (Pål Hermunn Johansen)
Date: Tue, 31 May 2016 13:19:05 +0200
Subject: [4.1] ae16ab8 Don't allow partial allocation of stevedore space for
 ESI
Message-ID: <E1b7hhR-0003uO-PK@project.varnish-software.com>


commit ae16ab8e3071e94e96bacc3af6f7964d12aca05c
Author: P?l Hermunn Johansen <hermunn at varnish-software.com>
Date:   Tue May 31 11:32:37 2016 +0200

    Don't allow partial allocation of stevedore space for ESI
    
    When allocating ESI memory through objallocwithnuke, tell the sevedore
    in storage_simple.c that getting less than requested, is not
    interesting.
    
    Fixes #1941
    
    This is a back port of #1961

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index f908af1..39bdef7 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -1051,7 +1051,8 @@ void RFC2616_Vary_AE(struct http *hp);
 /* stevedore.c */
 int STV_NewObject(struct objcore *, struct worker *,
     const char *hint, unsigned len);
-struct storage *STV_alloc(const struct stevedore *, size_t size);
+struct storage *STV_alloc(const struct stevedore *, size_t size, int flags);
+#define LESS_MEM_ALLOCED_IS_OK	1
 void STV_trim(const struct stevedore *, struct storage *, size_t size,
     int move_ok);
 void STV_free(const struct stevedore *, struct storage *st);
diff --git a/bin/varnishd/cache/cache_obj.c b/bin/varnishd/cache/cache_obj.c
index 53fe080..edfd797 100644
--- a/bin/varnishd/cache/cache_obj.c
+++ b/bin/varnishd/cache/cache_obj.c
@@ -235,7 +235,8 @@ ObjIterEnd(struct objcore *oc, void **oix)
  */
 
 static struct storage *
-objallocwithnuke(struct worker *wrk, const struct stevedore *stv, size_t size)
+objallocwithnuke(struct worker *wrk, const struct stevedore *stv,
+    size_t size, int flags)
 {
 	struct storage *st = NULL;
 	unsigned fail;
@@ -243,15 +244,18 @@ objallocwithnuke(struct worker *wrk, const struct stevedore *stv, size_t size)
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(stv, STEVEDORE_MAGIC);
 
-	if (size > cache_param->fetch_maxchunksize)
+	if (size > cache_param->fetch_maxchunksize) {
+		if (!(flags & LESS_MEM_ALLOCED_IS_OK))
+			return (NULL);
 		size = cache_param->fetch_maxchunksize;
+	}
 
 	assert(size <= UINT_MAX);	/* field limit in struct storage */
 
 	for (fail = 0; fail <= cache_param->nuke_limit; fail++) {
 		/* try to allocate from it */
 		AN(stv->alloc);
-		st = STV_alloc(stv, size);
+		st = STV_alloc(stv, size, flags);
 		if (st != NULL)
 			break;
 
@@ -299,7 +303,8 @@ ObjGetSpace(struct worker *wrk, struct objcore *oc, ssize_t *sz, uint8_t **ptr)
 		return (1);
 	}
 
-	st = objallocwithnuke(wrk, oc->stobj->stevedore, *sz);
+	st = objallocwithnuke(wrk, oc->stobj->stevedore, *sz,
+			      LESS_MEM_ALLOCED_IS_OK);
 	if (st == NULL)
 		return (0);
 
@@ -564,9 +569,11 @@ ObjSetattr(struct worker *wrk, struct objcore *oc, enum obj_attr attr,
 	st = o->objstore;
 	switch (attr) {
 	case OA_ESIDATA:
-		o->esidata = objallocwithnuke(wrk, oc->stobj->stevedore, len);
+		o->esidata = objallocwithnuke(wrk, oc->stobj->stevedore,
+		    len, 0);
 		if (o->esidata == NULL)
 			return (NULL);
+		assert(o->esidata->space >= len);
 		o->esidata->len = len;
 		retval  = o->esidata->ptr;
 		break;
diff --git a/bin/varnishd/storage/stevedore.c b/bin/varnishd/storage/stevedore.c
index 77e54a3..4845dba 100644
--- a/bin/varnishd/storage/stevedore.c
+++ b/bin/varnishd/storage/stevedore.c
@@ -156,12 +156,18 @@ stv_pick_stevedore(struct vsl_log *vsl, const char **hint)
 /*-------------------------------------------------------------------*/
 
 struct storage *
-STV_alloc(const struct stevedore *stv, size_t size)
+STV_alloc(const struct stevedore *stv, size_t size, int flags)
 {
 	struct storage *st;
 
 	CHECK_OBJ_NOTNULL(stv, STEVEDORE_MAGIC);
 
+	if (!(flags & LESS_MEM_ALLOCED_IS_OK)) {
+		if (size > cache_param->fetch_maxchunksize)
+			return (NULL);
+		else
+			return (stv->alloc(stv, size));
+	}
 	if (size > cache_param->fetch_maxchunksize)
 		size = cache_param->fetch_maxchunksize;
 
diff --git a/bin/varnishtest/tests/r01941.vtc b/bin/varnishtest/tests/r01941.vtc
new file mode 100644
index 0000000..2531ded
--- /dev/null
+++ b/bin/varnishtest/tests/r01941.vtc
@@ -0,0 +1,126 @@
+
+varnishtest "ESI memory should only be fully allocated"
+
+# In this test case, the cache is almost filled. Then we force the
+# allocation of ESI memory that does not fit in the cache, is bigger
+# than fetch_chunksize, but where half the amount can be allocated by
+# the stevedore.
+
+server s1 {
+	rxreq
+	expect req.url == /big
+	txresp -bodylen 1037480
+	rxreq
+	txresp -body {<html>
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+<esi:include src="/long1234567890123456789012345678901234567890.txt" />
+</html>}
+	rxreq
+	expect req.url == /long1234567890123456789012345678901234567890.txt
+	txresp -body {fo}
+} -start
+
+varnish v1 -arg "-pfetch_chunksize=4k" \
+	-arg "-smalloc,1m" -vcl+backend {
+	sub vcl_backend_response {
+		set beresp.do_esi = true;
+	}
+} -start
+
+client c1 {
+	txreq -url /big
+	rxresp
+	expect resp.bodylen == 1037480
+	txreq -url /
+	rxresp
+} -run



From varnish-commit at varnish-cache.org  Tue May 31 13:36:27 2016
From: varnish-commit at varnish-cache.org (biieid)
Date: Tue, 31 May 2016 21:36:27 +0800
Subject: =?utf-8?B?dmFybmlzaC1jb21taXTvvJrkvIHkuJrlupTmgKXnrqE=?=
 =?utf-8?B?55CG57O757uf5p6E5bu6?=
Message-ID: <20160531213632640545@bk.com>

      ???????????????
?????
          ?????????????????????????????????????291?????????????????????????????????
???????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
?????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
???????????????????????????????????????????????????????????
?????
?????EHS?????EHS?????????
??????????????????????????????
?????2??06?23-24????
?????3800?/????????????????????
?????021-31006787  18917870808    ???
QQ????320588808
????
???????????????????????????????????????????????????????????????????
????
???????????????????????????????????????
?????????????????????????????????????????????????????????????????????
?????
????????????
1.????????????????????????????????
2.????????????????????
3.?????2.18?????????
4.?????4.16???????????
5.???????????????????
6.??????
7.????????
8.?????????????
????????
1.??????????????????????
2.???????
?1????????????
?2????????????????
?3?????
?4????????
?5????????????
3.????????
4.????
?1?????????
?2?????????????
?3????????????????
?4??????????????????
?5????????????????
?6????????????????
?7??????????????????
?8??????????????????
?????????ERT??
?1.??????????????
?2.???????ERT????????
?3.?????????
?4.ERT??????????
?5.????????
?6.????????
???????????
?1.??????????
?2.????????????????
?3.??????????
????
????
1.???????????2?????????????KTV?????
2.?????????1???????????
3.????????????2?????????????????????
4.?????????1??
5.?????????2??
6.?????????3??
???? 
1.??????????
2.??????????
3.????????
????
????---????????????????????????
?????
?????
??????|????-EHS??????????????-???????????????
????????
?????????????????
????????????????????????
????????????????
?????
?????????????APP?????????????????????????????????11??????????6???????????????6???????????????????????????????????????????????????????????????????????????????????????????AON???????????????????????????????????????????????

???????
?????????????????????????????????????????????????????????6????????????????????????2000????????5???????????????????????????????ISO14001&OHSAS18001???????6S????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
????
????????????????????????????????????????EHS??????????????????????????????????6S????????????????
?????????
??/?????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
?     ??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????SEW?????????????????????????????????????????????????????NGK???????????????????????????????????????????????????????
?     ??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-commit/attachments/20160531/08362e38/attachment.html>

From lkarsten at varnish-software.com  Tue May 31 14:16:09 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 31 May 2016 16:16:09 +0200
Subject: [4.1] 9690462 req->task members must be set in case we get onto the
 waitinglist
Message-ID: <E1b7kSn-0006qc-HN@project.varnish-software.com>


commit 96904623e6f91052a8676df148eedde719f8aaf4
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Thu Apr 28 12:51:50 2016 +0200

    req->task members must be set in case we get onto the waitinglist
    
    Fixes #1928
    
    Conflicts:
    	bin/varnishd/http1/cache_http1_fsm.c

diff --git a/bin/varnishd/http1/cache_http1_fsm.c b/bin/varnishd/http1/cache_http1_fsm.c
index 210be90..ad9b069 100644
--- a/bin/varnishd/http1/cache_http1_fsm.c
+++ b/bin/varnishd/http1/cache_http1_fsm.c
@@ -264,11 +264,15 @@ HTTP1_Session(struct worker *wrk, struct req *req)
 			break;
 		case S_STP_H1PROC:
 			req->transport = &http1_transport;
+			req->task.func = SES_Proto_Req;
+			req->task.priv = req;
 			if (CNT_Request(wrk, req) == REQ_FSM_DISEMBARK) {
 				sp->sess_step = S_STP_H1BUSY;
 				return;
 			}
 			req->transport = NULL;
+			req->task.func = NULL;
+			req->task.priv = NULL;
 			sp->sess_step = S_STP_H1CLEANUP;
 			break;
 		case S_STP_H1CLEANUP:



From lkarsten at varnish-software.com  Tue May 31 14:16:09 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 31 May 2016 16:16:09 +0200
Subject: [4.1] 4ff7cb5 Add #1928 to changelog.
Message-ID: <E1b7kSn-0006qi-JG@project.varnish-software.com>


commit 4ff7cb5571e07cb65e426417a11049e1ff272680
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Tue May 31 16:10:57 2016 +0200

    Add #1928 to changelog.
    
    Ref: #1928

diff --git a/doc/changes.rst b/doc/changes.rst
index 8a7857f..7bc837f 100644
--- a/doc/changes.rst
+++ b/doc/changes.rst
@@ -11,6 +11,7 @@ Changes since 4.1.2:
 Bugs fixed
 ----------
 
+* 1928_ - req->task members must be set in case we get onto the waitinglist
 * 1918_ - Correctly handle EOF responses with HTTP/1.1
 * 1912_ - Fix (insignificant) memory leak with mal-formed ESI directives.
 * 1885_ - [vmodtool] Method names should start with a period
@@ -21,6 +22,7 @@ Bugs fixed
 * 1725_ - Revive the backend_conn counter
 * 1883_ - Only accept C identifiers as acls
 
+.. _1928: https://github.com/varnishcache/varnish-cache/issues/1928
 .. _1918: https://github.com/varnishcache/varnish-cache/issues/1918
 .. _1912: https://github.com/varnishcache/varnish-cache/issues/1912
 .. _1885: https://github.com/varnishcache/varnish-cache/issues/1885



From lkarsten at varnish-software.com  Tue May 31 14:21:08 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 31 May 2016 16:21:08 +0200
Subject: [4.1] a284fef Truncate output if it's wider than 12 chars
Message-ID: <E1b7kXc-0007Bg-3Q@project.varnish-software.com>


commit a284fefea9164de71ca49a0e66475fd14562e647
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Fri Mar 4 09:00:47 2016 +0000

    Truncate output if it's wider than 12 chars
    
    Early version of the patch OK'd by martin at .
    
    Fixes #1855.

diff --git a/bin/varnishstat/varnishstat_curses.c b/bin/varnishstat/varnishstat_curses.c
index 484d369..01c180c 100644
--- a/bin/varnishstat/varnishstat_curses.c
+++ b/bin/varnishstat/varnishstat_curses.c
@@ -65,6 +65,8 @@
 #define COLW			14
 #define COLW_NAME_MIN		24
 
+#define VALUE_MAX		999999999999
+
 struct ma {
 	unsigned n, nmax;
 	double acc;
@@ -715,6 +717,17 @@ print_bytes(WINDOW *w, double val)
 }
 
 static void
+print_trunc(WINDOW *w, uintmax_t val)
+{
+	if (val > VALUE_MAX) {
+		while (val > VALUE_MAX)
+			val /= 1000;
+		wprintw(w, " %9ju...", val);
+	} else
+		wprintw(w, " %12ju", val);
+}
+
+static void
 draw_line_bytes(WINDOW *w, int y, int x, int X, struct pt *pt)
 {
 	enum {
@@ -740,7 +753,7 @@ draw_line_bytes(WINDOW *w, int y, int x, int X, struct pt *pt)
 			if (scale && pt->cur > 1024)
 				print_bytes(w, (double)pt->cur);
 			else
-				wprintw(w, " %12ju", (uintmax_t)pt->cur);
+				print_trunc(w, (uintmax_t)pt->cur);
 			break;
 		case COL_CHG:
 			if (pt->t_last)



From lkarsten at varnish-software.com  Tue May 31 14:21:08 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 31 May 2016 16:21:08 +0200
Subject: [4.1] 01b97db Add #1855 to the changelog.
Message-ID: <E1b7kXc-0007Bp-5x@project.varnish-software.com>


commit 01b97dbddf7ecac3d8b2ff1d422e1b95c1d3b551
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Tue May 31 16:18:58 2016 +0200

    Add #1855 to the changelog.
    
    Ref: #1855

diff --git a/doc/changes.rst b/doc/changes.rst
index 7bc837f..73aaa7d 100644
--- a/doc/changes.rst
+++ b/doc/changes.rst
@@ -1,6 +1,6 @@
-========================================
-DRAFT CHANGELOG ENTRIES FOR NEXT RELEASE
-========================================
+======================================
+Varnish Cache 4.1.3-beta1 (unreleased)
+======================================
 
 Changes since 4.1.2:
 
@@ -19,8 +19,9 @@ Bugs fixed
 * 1878_ - Fix a ESI+gzip corner case which had escaped notice until now
 * 1871_ - Missing error handling code in V1F_Setup_Fetch
 * 1869_ - Remove temporary directory iff called with -C
-* 1725_ - Revive the backend_conn counter
 * 1883_ - Only accept C identifiers as acls
+* 1855_ - Truncate output if it's wider than 12 chars
+* 1725_ - Revive the backend_conn counter
 
 .. _1928: https://github.com/varnishcache/varnish-cache/issues/1928
 .. _1918: https://github.com/varnishcache/varnish-cache/issues/1918
@@ -31,6 +32,7 @@ Bugs fixed
 .. _1878: https://github.com/varnishcache/varnish-cache/issues/1878
 .. _1871: https://github.com/varnishcache/varnish-cache/issues/1871
 .. _1869: https://github.com/varnishcache/varnish-cache/issues/1869
+.. _1855: https://github.com/varnishcache/varnish-cache/issues/1855
 .. _1725: https://github.com/varnishcache/varnish-cache/issues/1725
 
 



From nils.goroll at uplex.de  Tue May 31 14:29:08 2016
From: nils.goroll at uplex.de (Nils Goroll)
Date: Tue, 31 May 2016 16:29:08 +0200
Subject: [master] 45ca71b Current SmartOS also has epoll(),
 so we need to prefer ports ...
Message-ID: <E1b7kfM-0007h4-57@project.varnish-software.com>


commit 45ca71b559adb9b0fbbd6cd3a56cda52af97b8ba
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Tue May 31 16:05:29 2016 +0200

    Current SmartOS also has epoll(), so we need to prefer ports ...
    
    ... in order to actually use it by default

diff --git a/bin/varnishd/mgt/mgt_main.c b/bin/varnishd/mgt/mgt_main.c
index 56394be..e189e61 100644
--- a/bin/varnishd/mgt/mgt_main.c
+++ b/bin/varnishd/mgt/mgt_main.c
@@ -247,12 +247,12 @@ usage(void)
 #if defined(HAVE_KQUEUE)
 	fprintf(stderr, FMT, "", "  -W kqueue");
 #endif
-#if defined(HAVE_EPOLL_CTL)
-	fprintf(stderr, FMT, "", "  -W epoll");
-#endif
 #if defined(HAVE_PORT_CREATE)
 	fprintf(stderr, FMT, "", "  -W ports");
 #endif
+#if defined(HAVE_EPOLL_CTL)
+	fprintf(stderr, FMT, "", "  -W epoll");
+#endif
 	fprintf(stderr, FMT, "", "  -W poll");
 
 #undef FMT
diff --git a/bin/varnishd/waiter/mgt_waiter.c b/bin/varnishd/waiter/mgt_waiter.c
index c0e450d..58d4307 100644
--- a/bin/varnishd/waiter/mgt_waiter.c
+++ b/bin/varnishd/waiter/mgt_waiter.c
@@ -40,12 +40,12 @@ static const struct choice waiter_choice[] = {
     #if defined(HAVE_KQUEUE)
 	{ "kqueue",	&waiter_kqueue },
     #endif
-    #if defined(HAVE_EPOLL_CTL)
-	{ "epoll",	&waiter_epoll },
-    #endif
     #if defined(HAVE_PORT_CREATE)
 	{ "ports",	&waiter_ports },
     #endif
+    #if defined(HAVE_EPOLL_CTL)
+	{ "epoll",	&waiter_epoll },
+    #endif
 	{ "poll",	&waiter_poll },
 	{ NULL,		NULL}
 };



From hermunn at varnish-software.com  Tue May 31 14:40:11 2016
From: hermunn at varnish-software.com (Pål Hermunn Johansen)
Date: Tue, 31 May 2016 16:40:11 +0200
Subject: [4.1] 925b5f7 Don't emit multiple Age or Accept-Ranges headers
Message-ID: <E1b7kq3-0008UY-S8@project.varnish-software.com>


commit 925b5f77c5142340a3c21a0c7a1f4283ab0a3d9c
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Sun May 29 14:57:00 2016 +0100

    Don't emit multiple Age or Accept-Ranges headers
    
    This might happen during pass'd requests or in a multi-tier setup.
    Fixes #1955.

diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index 03e833a..e3773e8 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -119,7 +119,7 @@ vbf_beresp2obj(struct busyobj *bo)
 	}
 
 	l2 = http_EstimateWS(bo->beresp,
-	    bo->uncacheable ? HTTPH_R_PASS : HTTPH_A_INS);
+	    bo->uncacheable ? HTTPH_A_PASS : HTTPH_A_INS);
 	l += l2;
 
 	if (bo->uncacheable)
@@ -143,7 +143,7 @@ vbf_beresp2obj(struct busyobj *bo)
 	bp = ObjSetattr(bo->wrk, bo->fetch_objcore, OA_HEADERS, l2, NULL);
 	AN(bp);
 	HTTP_Encode(bo->beresp, bp, l2,
-	    bo->uncacheable ? HTTPH_R_PASS : HTTPH_A_INS);
+	    bo->uncacheable ? HTTPH_A_PASS : HTTPH_A_INS);
 
 	if (http_GetHdr(bo->beresp, H_Last_Modified, &b))
 		AZ(ObjSetDouble(bo->wrk, bo->fetch_objcore, OA_LASTMODIFIED,
diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index 1a59ae2..2540a74 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -88,7 +88,7 @@ cnt_vdp(struct req *req, struct busyobj *bo)
 		VDP_push(req, VDP_gunzip, NULL, 1);
 
 	if (cache_param->http_range_support && http_IsStatus(req->resp, 200)) {
-		http_SetHeader(req->resp, "Accept-Ranges: bytes");
+		http_ForceHeader(req->resp, H_Accept_Ranges, "bytes");
 		if (sendbody && http_GetHdr(req->http, H_Range, &r))
 			VRG_dorange(req, r);
 	}
diff --git a/bin/varnishtest/tests/c00071.vtc b/bin/varnishtest/tests/c00071.vtc
index 7572613..e281c90 100644
--- a/bin/varnishtest/tests/c00071.vtc
+++ b/bin/varnishtest/tests/c00071.vtc
@@ -14,7 +14,7 @@ server s1 {
 varnish v1 -vcl+backend {
 	import ${vmod_debug};
 	sub vcl_deliver {
-		debug.workspace_allocate(client, debug.workspace_free(client) - 150);
+		debug.workspace_allocate(client, debug.workspace_free(client) - 160);
 
 		if (req.url ~ "/bar") {
 			set resp.http.x-foo = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz";
diff --git a/bin/varnishtest/tests/r01955.vtc b/bin/varnishtest/tests/r01955.vtc
new file mode 100644
index 0000000..756ebc0
--- /dev/null
+++ b/bin/varnishtest/tests/r01955.vtc
@@ -0,0 +1,22 @@
+varnishtest "Correct handling multiple Age headers in pass mode"
+
+# This affects Accept-Range as well but it's not possible to test
+# for that.
+
+server s1 {
+	 rxreq
+	 txresp -hdr "Cache-Control: max-age=1" -hdr "Age: 2"
+} -start
+
+varnish v1 -vcl+backend {
+	import std;
+	sub vcl_deliver {
+		std.collect(resp.http.age);
+	}
+} -start
+
+client c1 {
+	 txreq
+	 rxresp
+	 expect resp.http.age == "2"
+} -run
diff --git a/include/tbl/http_headers.h b/include/tbl/http_headers.h
index 0d973c7..5ada35a 100644
--- a/include/tbl/http_headers.h
+++ b/include/tbl/http_headers.h
@@ -43,75 +43,78 @@
 #define HTTPH_R_PASS	(1 << 0)	/* Request (c->b) in pass mode */
 #define HTTPH_R_FETCH	(1 << 1)	/* Request (c->b) for fetch */
 #define HTTPH_A_INS	(1 << 2)	/* Response (b->o) for insert */
+#define HTTPH_A_PASS	(1 << 3)	/* Response (b->o) for pass */
 #endif
 
 /* Shorthand for this file only, to keep table narrow */
 
-#if defined(P) || defined(F) || defined(I) || defined(H)
+#if defined(P) || defined(F) || defined(I) || defined(H) || defined(S)
 #error "Macro overloading"  // Trust but verify
 #endif
 
 #define P HTTPH_R_PASS
 #define F HTTPH_R_FETCH
 #define I HTTPH_A_INS
+#define S HTTPH_A_PASS
 #define H(s,e,f) HTTPH(s, e, f)
 
-H("Keep-Alive",		H_Keep_Alive,		P|F  )	// 2068
-H("Accept",		H_Accept,		0    )	// 2616 14.1
-H("Accept-Charset",	H_Accept_Charset,	0    )	// 2616 14.2
-H("Accept-Encoding",	H_Accept_Encoding,	0    )	// 2616 14.3
-H("Accept-Language",	H_Accept_Language,	0    )	// 2616 14.4
-H("Accept-Ranges",	H_Accept_Ranges,	  F|I)	// 2616 14.5
-H("Age",		H_Age,			    I)	// 2616 14.6
-H("Allow",		H_Allow,		0    )	// 2616 14.7
-H("Authorization",	H_Authorization,	0    )	// 2616 14.8
-H("Cache-Control",	H_Cache_Control,	  F  )	// 2616 14.9
-H("Connection",		H_Connection,		P|F|I)	// 2616 14.10
-H("Content-Encoding",	H_Content_Encoding,	0    )	// 2616 14.11
-H("Content-Language",	H_Content_Language,	0    )	// 2616 14.12
-H("Content-Length",	H_Content_Length,	  F  )	// 2616 14.13
-H("Content-Location",	H_Content_Location,	0    )	// 2616 14.14
-H("Content-MD5",	H_Content_MD5,		0    )	// 2616 14.15
-H("Content-Range",	H_Content_Range,	  F|I)	// 2616 14.16
-H("Content-Type",	H_Content_Type,		0    )	// 2616 14.17
-H("Cookie",		H_Cookie,		0    )	// 6265 4.2
-H("Date",		H_Date,			0    )	// 2616 14.18
-H("ETag",		H_ETag,			0    )	// 2616 14.19
-H("Expect",		H_Expect,		0    )	// 2616 14.20
-H("Expires",		H_Expires,		0    )	// 2616 14.21
-H("From",		H_From,			0    )	// 2616 14.22
-H("Host",		H_Host,			0    )	// 2616 14.23
-H("HTTP2-Settings",	H_HTTP2_Settings,	P|F|I)	// httpbis-http2-16.txt
-H("If-Match",		H_If_Match,		  F  )	// 2616 14.24
-H("If-Modified-Since",	H_If_Modified_Since,	  F  )	// 2616 14.25
-H("If-None-Match",	H_If_None_Match,	  F  )	// 2616 14.26
-H("If-Range",		H_If_Range,		  F  )	// 2616 14.27
-H("If-Unmodified-Since",H_If_Unmodified_Since,	  F  )	// 2616 14.28
-H("Last-Modified",	H_Last_Modified,	0    )	// 2616 14.29
-H("Location",		H_Location,		0    )	// 2616 14.30
-H("Max-Forwards",	H_Max_Forwards,		0    )	// 2616 14.31
-H("Pragma",		H_Pragma,		0    )	// 2616 14.32
-H("Proxy-Authenticate",	H_Proxy_Authenticate,	  F|I)	// 2616 14.33
-H("Proxy-Authorization",H_Proxy_Authorization,	  F|I)	// 2616 14.34
-H("Range",		H_Range,		  F|I)	// 2616 14.35
-H("Referer",		H_Referer,		0    )	// 2616 14.36
-H("Retry-After",	H_Retry_After,		0    )	// 2616 14.37
-H("Server",		H_Server,		0    )	// 2616 14.38
-H("Set-Cookie",		H_Set_Cookie,		0    )	// 6265 4.1
-H("TE",			H_TE,			P|F|I)	// 2616 14.39
-H("Trailer",		H_Trailer,		P|F|I)	// 2616 14.40
-H("Transfer-Encoding",	H_Transfer_Encoding,	P|F|I)	// 2616 14.41
-H("Upgrade",		H_Upgrade,		P|F|I)	// 2616 14.42
-H("User-Agent",		H_User_Agent,		0    )	// 2616 14.43
-H("Vary",		H_Vary,			0    )	// 2616 14.44
-H("Via",		H_Via,			0    )	// 2616 14.45
-H("Warning",		H_Warning,		0    )	// 2616 14.46
-H("WWW-Authenticate",	H_WWW_Authenticate,	0    )	// 2616 14.47
-H("X-Forwarded-For",	H_X_Forwarded_For,	0    )	// No RFC
+H("Keep-Alive",		H_Keep_Alive,		P|F  |S)	// 2068
+H("Accept",		H_Accept,		0      )	// 2616 14.1
+H("Accept-Charset",	H_Accept_Charset,	0      )	// 2616 14.2
+H("Accept-Encoding",	H_Accept_Encoding,	0      )	// 2616 14.3
+H("Accept-Language",	H_Accept_Language,	0      )	// 2616 14.4
+H("Accept-Ranges",	H_Accept_Ranges,	P|F|I  )	// 2616 14.5
+H("Age",		H_Age,			    I|S)	// 2616 14.6
+H("Allow",		H_Allow,		0      )	// 2616 14.7
+H("Authorization",	H_Authorization,	0      )	// 2616 14.8
+H("Cache-Control",	H_Cache_Control,	  F    )	// 2616 14.9
+H("Connection",		H_Connection,		P|F|I|S)	// 2616 14.10
+H("Content-Encoding",	H_Content_Encoding,	0      )	// 2616 14.11
+H("Content-Language",	H_Content_Language,	0      )	// 2616 14.12
+H("Content-Length",	H_Content_Length,	  F    )	// 2616 14.13
+H("Content-Location",	H_Content_Location,	0      )	// 2616 14.14
+H("Content-MD5",	H_Content_MD5,		0      )	// 2616 14.15
+H("Content-Range",	H_Content_Range,	  F|I  )	// 2616 14.16
+H("Content-Type",	H_Content_Type,		0      )	// 2616 14.17
+H("Cookie",		H_Cookie,		0      )	// 6265 4.2
+H("Date",		H_Date,			0      )	// 2616 14.18
+H("ETag",		H_ETag,			0      )	// 2616 14.19
+H("Expect",		H_Expect,		0      )	// 2616 14.20
+H("Expires",		H_Expires,		0      )	// 2616 14.21
+H("From",		H_From,			0      )	// 2616 14.22
+H("Host",		H_Host,			0      )	// 2616 14.23
+H("HTTP2-Settings",	H_HTTP2_Settings,	P|F|I|S)	// httpbis-http2-16.txt
+H("If-Match",		H_If_Match,		  F    )	// 2616 14.24
+H("If-Modified-Since",	H_If_Modified_Since,	  F    )	// 2616 14.25
+H("If-None-Match",	H_If_None_Match,	  F    )	// 2616 14.26
+H("If-Range",		H_If_Range,		  F    )	// 2616 14.27
+H("If-Unmodified-Since",H_If_Unmodified_Since,	  F    )	// 2616 14.28
+H("Last-Modified",	H_Last_Modified,	0      )	// 2616 14.29
+H("Location",		H_Location,		0      )	// 2616 14.30
+H("Max-Forwards",	H_Max_Forwards,		0      )	// 2616 14.31
+H("Pragma",		H_Pragma,		0      )	// 2616 14.32
+H("Proxy-Authenticate",	H_Proxy_Authenticate,	  F|I  )	// 2616 14.33
+H("Proxy-Authorization",H_Proxy_Authorization,	  F|I  )	// 2616 14.34
+H("Range",		H_Range,		  F|I  )	// 2616 14.35
+H("Referer",		H_Referer,		0      )	// 2616 14.36
+H("Retry-After",	H_Retry_After,		0      )	// 2616 14.37
+H("Server",		H_Server,		0      )	// 2616 14.38
+H("Set-Cookie",		H_Set_Cookie,		0      )	// 6265 4.1
+H("TE",			H_TE,			P|F|I|S)	// 2616 14.39
+H("Trailer",		H_Trailer,		P|F|I|S)	// 2616 14.40
+H("Transfer-Encoding",	H_Transfer_Encoding,	P|F|I|S)	// 2616 14.41
+H("Upgrade",		H_Upgrade,		P|F|I|S)	// 2616 14.42
+H("User-Agent",		H_User_Agent,		0      )	// 2616 14.43
+H("Vary",		H_Vary,			0      )	// 2616 14.44
+H("Via",		H_Via,			0      )	// 2616 14.45
+H("Warning",		H_Warning,		0      )	// 2616 14.46
+H("WWW-Authenticate",	H_WWW_Authenticate,	0      )	// 2616 14.47
+H("X-Forwarded-For",	H_X_Forwarded_For,	0      )	// No RFC
 
 #undef P
 #undef F
 #undef I
+#undef S
 #undef H
 
 /*lint -restore */



From hermunn at varnish-software.com  Tue May 31 14:40:11 2016
From: hermunn at varnish-software.com (Pål Hermunn Johansen)
Date: Tue, 31 May 2016 16:40:11 +0200
Subject: [4.1] b4c569a Add #1955 to the changelog.
Message-ID: <E1b7kq3-0008Ue-Tk@project.varnish-software.com>


commit b4c569a4c682a7668b6b358fdc291e4ec4274008
Author: P?l Hermunn Johansen <hermunn at varnish-software.com>
Date:   Tue May 31 16:34:52 2016 +0200

    Add #1955 to the changelog.
    
    Ref: #1955

diff --git a/doc/changes.rst b/doc/changes.rst
index 73aaa7d..7cbe430 100644
--- a/doc/changes.rst
+++ b/doc/changes.rst
@@ -11,6 +11,7 @@ Changes since 4.1.2:
 Bugs fixed
 ----------
 
+* 1955_ - 4.1.x sometimes duplicates Age and Accept-Ranges headers
 * 1928_ - req->task members must be set in case we get onto the waitinglist
 * 1918_ - Correctly handle EOF responses with HTTP/1.1
 * 1912_ - Fix (insignificant) memory leak with mal-formed ESI directives.
@@ -23,6 +24,7 @@ Bugs fixed
 * 1855_ - Truncate output if it's wider than 12 chars
 * 1725_ - Revive the backend_conn counter
 
+.. _1955: https://github.com/varnishcache/varnish-cache/issues/1955
 .. _1928: https://github.com/varnishcache/varnish-cache/issues/1928
 .. _1918: https://github.com/varnishcache/varnish-cache/issues/1918
 .. _1912: https://github.com/varnishcache/varnish-cache/issues/1912



From nils.goroll at uplex.de  Tue May 31 15:38:07 2016
From: nils.goroll at uplex.de (Nils Goroll)
Date: Tue, 31 May 2016 17:38:07 +0200
Subject: [master] cf056c4 missing Wait_HeapDelete
Message-ID: <E1b7lk7-0003IC-Gl@project.varnish-software.com>


commit cf056c4869fc92cfc074fa8e7c5c2b6016a5300d
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Tue May 31 17:36:56 2016 +0200

    missing Wait_HeapDelete
    
    fixes #1971

diff --git a/bin/varnishd/waiter/cache_waiter_ports.c b/bin/varnishd/waiter/cache_waiter_ports.c
index 49c6790..87e4fc8 100644
--- a/bin/varnishd/waiter/cache_waiter_ports.c
+++ b/bin/varnishd/waiter/cache_waiter_ports.c
@@ -3,7 +3,7 @@
  * Copyright (c) 2006 Varnish Software AS
  * Copyright (c) 2007 OmniTI Computer Consulting, Inc.
  * Copyright (c) 2007 Theo Schlossnagle
- * Copyright (c) 2010-2015 UPLEX, Nils Goroll
+ * Copyright (c) 2010-2016 UPLEX, Nils Goroll
  * All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
@@ -165,6 +165,7 @@ vws_thread(void *priv)
 			}
 			CHECK_OBJ_NOTNULL(wp, WAITED_MAGIC);
 			vws_del(vws, wp->fd);
+			Wait_HeapDelete(w, wp);
 			Wait_Call(w, wp, WAITER_TIMEOUT, now);
 		}
 		then = vws->next - now;