From lkarsten at varnish-software.com  Mon Jan  4 14:26:02 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Mon, 04 Jan 2016 15:26:02 +0100
Subject: [master] dbcca92 Devicedetect git repository has moved.
Message-ID: <E1aG65C-0002lK-Mw@project.varnish-software.com>


commit dbcca92e7b6312c4976e07c643360facb37f94ce
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Mon Jan 4 15:25:33 2016 +0100

    Devicedetect git repository has moved.

diff --git a/doc/sphinx/users-guide/devicedetection.rst b/doc/sphinx/users-guide/devicedetection.rst
index 8027909..91a608e 100644
--- a/doc/sphinx/users-guide/devicedetection.rst
+++ b/doc/sphinx/users-guide/devicedetection.rst
@@ -29,7 +29,7 @@ Setting this header can be as simple as::
 There are different commercial and free offerings in doing grouping and
 identifying clients in further detail. For a basic and community
 based regular expression set, see
-https://github.com/varnish/varnish-devicedetect/.
+https://github.com/varnishcache/varnish-devicedetect/.
 
 
 Serve the different content on the same URL
diff --git a/etc/devicedetect.vcl b/etc/devicedetect.vcl
index 7b5c721..d1a85d9 100644
--- a/etc/devicedetect.vcl
+++ b/etc/devicedetect.vcl
@@ -1,4 +1,4 @@
-# Copyright (c) 2012-2015 Varnish Software AS
+# Copyright (c) 2012-2014 Varnish Software AS
 #
 # Redistribution and use in source and binary forms, with or without
 # modification, are permitted provided that the following conditions
@@ -22,7 +22,7 @@
 # SUCH DAMAGE.
 #
 # detectdevice.vcl - regex based device detection for Varnish
-# https://github.com/varnish/varnish-devicedetect/
+# https://github.com/varnishcache/varnish-devicedetect/
 #
 # Author: Lasse Karstensen <lkarsten at varnish-software.com>
 



From phk at FreeBSD.org  Mon Jan  4 20:45:02 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Mon, 04 Jan 2016 21:45:02 +0100
Subject: [master] a9b6d0e Make the "debug +vclrel" trick work for all worker
 threads.
Message-ID: <E1aGBzy-0007WH-Hy@project.varnish-software.com>


commit a9b6d0e8fa1f7edcd9aa8c0fc7b054ce35ecd6fc
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Jan 4 20:44:19 2016 +0000

    Make the "debug +vclrel" trick work for all worker threads.

diff --git a/bin/varnishd/cache/cache_session.c b/bin/varnishd/cache/cache_session.c
index f1bb4c0..34c4374 100644
--- a/bin/varnishd/cache/cache_session.c
+++ b/bin/varnishd/cache/cache_session.c
@@ -321,8 +321,6 @@ SES_Proto_Req(struct worker *wrk, void *arg)
 		WRONG("Wrong session step");
 	}
 	WS_Assert(wrk->aws);
-	if (DO_DEBUG(DBG_VCLREL) && wrk->vcl != NULL)
-		VCL_Rel(&wrk->vcl);
 	THR_SetRequest(NULL);
 }
 
diff --git a/bin/varnishd/cache/cache_wrk.c b/bin/varnishd/cache/cache_wrk.c
index 7b774b1..7176f09 100644
--- a/bin/varnishd/cache/cache_wrk.c
+++ b/bin/varnishd/cache/cache_wrk.c
@@ -339,6 +339,8 @@ Pool_Work_Thread(struct pool *pp, struct worker *wrk)
 			memset(&wrk->task, 0, sizeof wrk->task);
 			assert(wrk->pool == pp);
 			tp->func(wrk, tp->priv);
+			if (DO_DEBUG(DBG_VCLREL) && wrk->vcl != NULL)
+				VCL_Rel(&wrk->vcl);
 			tpx = wrk->task;
 			tp = &tpx;
 		} while (tp->func != NULL);



From phk at FreeBSD.org  Tue Jan  5 11:53:38 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 05 Jan 2016 12:53:38 +0100
Subject: [master] 81c756e Dump oc->expire info
Message-ID: <E1aGQBG-0002No-GS@project.varnish-software.com>


commit 81c756e5df5f9592c6da906fc47e9f21d36846ca
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Jan 5 10:19:31 2016 +0000

    Dump oc->expire info

diff --git a/bin/varnishd/cache/cache_panic.c b/bin/varnishd/cache/cache_panic.c
index b60ddbd..0613604 100644
--- a/bin/varnishd/cache/cache_panic.c
+++ b/bin/varnishd/cache/cache_panic.c
@@ -197,6 +197,9 @@ pan_objcore(struct vsb *vsb, const char *typ, const struct objcore *oc)
 	VSB_indent(vsb, 2);
 	VSB_printf(vsb, "refcnt = %d,\n", oc->refcnt);
 	VSB_printf(vsb, "flags = 0x%x,\n", oc->flags);
+	VSB_printf(vsb, "exp_flags = 0x%x,\n", oc->exp_flags);
+	VSB_printf(vsb, "exp = { %f, %f, %f, %f }\n",
+	    oc->exp.t_origin, oc->exp.ttl, oc->exp.grace, oc->exp.keep);
 	VSB_printf(vsb, "objhead = %p,\n", oc->objhead);
 	VSB_printf(vsb, "stevedore = %p", oc->stobj->stevedore);
 	if (oc->stobj->stevedore != NULL) {



From phk at FreeBSD.org  Tue Jan  5 11:53:38 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 05 Jan 2016 12:53:38 +0100
Subject: [master] 86d73a6 Cleanup
Message-ID: <E1aGQBG-0002Nu-I0@project.varnish-software.com>


commit 86d73a65b52359e82f5eb42204c7249c7d236754
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Jan 5 11:53:29 2016 +0000

    Cleanup

diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index cc93d97..709a03b 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -779,18 +779,14 @@ vbf_stp_error(struct worker *wrk, struct busyobj *bo)
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
 	CHECK_OBJ_NOTNULL(bo->fetch_objcore, OBJCORE_MAGIC);
+	AN(bo->fetch_objcore->flags & OC_F_BUSY);
 	assert(bo->director_state == DIR_S_NULL);
 
-	if(bo->fetch_objcore->stobj->stevedore != NULL)
-		ObjFreeObj(bo->wrk, bo->fetch_objcore);
-
 	now = W_TIM_real(wrk);
 	VSLb_ts_busyobj(bo, "Error", now);
 
-	AN(bo->fetch_objcore->flags & OC_F_BUSY);
-
-	synth_body = VSB_new_auto();
-	AN(synth_body);
+	if(bo->fetch_objcore->stobj->stevedore != NULL)
+		ObjFreeObj(bo->wrk, bo->fetch_objcore);
 
 	// XXX: reset all beresp flags ?
 
@@ -802,26 +798,23 @@ vbf_stp_error(struct worker *wrk, struct busyobj *bo)
 	EXP_Clr(&bo->fetch_objcore->exp);
 	bo->fetch_objcore->exp.t_origin = bo->t_prev;
 
+	synth_body = VSB_new_auto();
+	AN(synth_body);
+
 	VCL_backend_error_method(bo->vcl, wrk, NULL, bo, synth_body);
 
 	AZ(VSB_finish(synth_body));
 
-	if (wrk->handling == VCL_RET_RETRY ||
-	    wrk->handling == VCL_RET_ABANDON) {
+	if (wrk->handling == VCL_RET_ABANDON) {
 		VSB_delete(synth_body);
+		return (F_STP_FAIL);
+	}
 
-		if (bo->director_state != DIR_S_NULL) {
-			bo->htc->doclose = SC_RESP_CLOSE;
-			VDI_Finish(bo->wrk, bo);
-		}
-
-		if (wrk->handling == VCL_RET_RETRY) {
-			if (bo->retries++ < cache_param->max_retries)
-				return (F_STP_RETRY);
-			VSLb(bo->vsl, SLT_VCL_Error,
-			    "Too many retries, delivering 503");
-		}
-
+	if (wrk->handling == VCL_RET_RETRY) {
+		VSB_delete(synth_body);
+		if (bo->retries++ < cache_param->max_retries)
+			return (F_STP_RETRY);
+		VSLb(bo->vsl, SLT_VCL_Error, "Too many retries, failing");
 		return (F_STP_FAIL);
 	}
 
diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index 85da4f0..dd03596 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -814,6 +814,7 @@ HSH_DerefObjHead(struct worker *wrk, struct objhead **poh)
 	CHECK_OBJ_NOTNULL(oh, OBJHEAD_MAGIC);
 
 	if (oh == private_oh) {
+		AZ(oh->waitinglist);
 		Lck_Lock(&oh->mtx);
 		assert(oh->refcnt > 1);
 		oh->refcnt--;



From fgsch at lodoss.net  Tue Jan  5 16:06:57 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Tue, 05 Jan 2016 17:06:57 +0100
Subject: [master] 446695e Spring some AN() love
Message-ID: <E1aGU8P-0002X1-7U@project.varnish-software.com>


commit 446695e99af181d214ce8355d866010ae5c09587
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Tue Jan 5 10:42:02 2016 +0000

    Spring some AN() love

diff --git a/bin/varnishncsa/varnishncsa.c b/bin/varnishncsa/varnishncsa.c
index dfb4885..249efee 100644
--- a/bin/varnishncsa/varnishncsa.c
+++ b/bin/varnishncsa/varnishncsa.c
@@ -531,6 +531,7 @@ addf_auth(const char *str)
 	struct format *f;
 
 	ALLOC_OBJ(f, FORMAT_MAGIC);
+	AN(f);
 	f->func = &format_auth;
 	if (str != NULL) {
 		f->string = strdup(str);
diff --git a/lib/libvarnishapi/vsc.c b/lib/libvarnishapi/vsc.c
index c86852d..f051cea 100644
--- a/lib/libvarnishapi/vsc.c
+++ b/lib/libvarnishapi/vsc.c
@@ -105,6 +105,7 @@ vsc_setup(struct VSM_data *vd)
 	CHECK_OBJ_NOTNULL(vd, VSM_MAGIC);
 	if (vd->vsc == NULL) {
 		ALLOC_OBJ(vd->vsc, VSC_MAGIC);
+		AN(vd->vsc);
 		VTAILQ_INIT(&vd->vsc->vf_list);
 		VTAILQ_INIT(&vd->vsc->pt_list);
 		VTAILQ_INIT(&vd->vsc->sf_list);



From fgsch at lodoss.net  Tue Jan  5 16:06:57 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Tue, 05 Jan 2016 17:06:57 +0100
Subject: [master] 5921f91 Use realloc if possible
Message-ID: <E1aGU8P-0002X7-9R@project.varnish-software.com>


commit 5921f91a7bc115a0d7dced7bcd9838a03255008e
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Mon Dec 28 16:41:31 2015 +0000

    Use realloc if possible
    
    Avoids memcpy/free if the buffer was dynamic.

diff --git a/lib/libvarnish/vsb.c b/lib/libvarnish/vsb.c
index 7b9380a..4d80b8a 100644
--- a/lib/libvarnish/vsb.c
+++ b/lib/libvarnish/vsb.c
@@ -142,14 +142,16 @@ VSB_extend(struct vsb *s, int addlen)
 	if (!VSB_CANEXTEND(s))
 		return (-1);
 	newsize = VSB_extendsize(s->s_size + addlen);
-	newbuf = SBMALLOC(newsize);
-	if (newbuf == NULL)
-		return (-1);
-	memcpy(newbuf, s->s_buf, s->s_size);
 	if (VSB_ISDYNAMIC(s))
-		SBFREE(s->s_buf);
+		newbuf = realloc(s->s_buf, newsize);
 	else
+		newbuf = SBMALLOC(newsize);
+	if (newbuf == NULL)
+		return (-1);
+	if (!VSB_ISDYNAMIC(s)) {
+		memcpy(newbuf, s->s_buf, s->s_size);
 		VSB_SETFLAG(s, VSB_DYNAMIC);
+	}
 	s->s_buf = newbuf;
 	s->s_size = newsize;
 	return (0);



From fgsch at lodoss.net  Tue Jan  5 23:23:18 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Wed, 06 Jan 2016 00:23:18 +0100
Subject: [master] f2de8e9 Update parameters in preparation for further work
Message-ID: <E1aGawg-0008Rv-1H@project.varnish-software.com>


commit f2de8e9dac224dbe6699a7cfe431abc7e6d6fb57
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Tue Jan 5 22:43:58 2016 +0000

    Update parameters in preparation for further work

diff --git a/bin/varnishd/common/params.h b/bin/varnishd/common/params.h
index 44f2f2a..957b8a9 100644
--- a/bin/varnishd/common/params.h
+++ b/bin/varnishd/common/params.h
@@ -63,16 +63,26 @@ struct poolparam {
 struct params {
 
 #define	ptyp_bool	unsigned
+#define	ptyp_bytes	ssize_t
+#define	ptyp_bytes_u	unsigned
 #define	ptyp_double	double
+#define	ptyp_poolparam	struct poolparam
 #define	ptyp_timeout	double
 #define	ptyp_uint	unsigned
+#define	ptyp_vsl_buffer	unsigned
+#define	ptyp_vsl_reclen	unsigned
 #define PARAM(nm, ty, mi, ma, de, un, fl, st, lt, fn) ptyp_##ty nm;
 #include <tbl/params.h>
 #undef PARAM
 #undef ptyp_bool
+#undef ptyp_bytes
+#undef ptyp_bytes_u
 #undef ptyp_double
+#undef ptyp_poolparam
 #undef ptyp_timeout
 #undef ptyp_uint
+#undef ptyp_vsl_buffer
+#undef ptyp_vsl_reclen
 
 	/* Unprivileged user / group */
 	char			*user;
diff --git a/include/tbl/params.h b/include/tbl/params.h
index 41f5b86..f372351 100644
--- a/include/tbl/params.h
+++ b/include/tbl/params.h
@@ -128,7 +128,7 @@ PARAM(
 
 PARAM(
 	/* name */	ban_lurker_age,
-	/* tweak */	timeout,
+	/* typ */	timeout,
 	/* min */	"0",
 	/* max */	NULL,
 	/* default */	"60",
@@ -145,7 +145,7 @@ PARAM(
 
 PARAM(
 	/* name */	ban_lurker_batch,
-	/* tweak */	uint,
+	/* typ */	uint,
 	/* min */	"1",
 	/* max */	NULL,
 	/* default */	"1000",
@@ -161,7 +161,7 @@ PARAM(
 
 PARAM(
 	/* name */	ban_lurker_sleep,
-	/* tweak */	timeout,
+	/* typ */	timeout,
 	/* min */	"0",
 	/* max */	NULL,
 	/* default */	"0.010",
@@ -178,7 +178,7 @@ PARAM(
 
 PARAM(
 	/* name */	first_byte_timeout,
-	/* tweak */	timeout,
+	/* typ */	timeout,
 	/* min */	"0",
 	/* max */	NULL,
 	/* default */	"60",
@@ -196,7 +196,7 @@ PARAM(
 
 PARAM(
 	/* name */	between_bytes_timeout,
-	/* tweak */	timeout,
+	/* typ */	timeout,
 	/* min */	"0",
 	/* max */	NULL,
 	/* default */	"60",
@@ -214,7 +214,7 @@ PARAM(
 
 PARAM(
 	/* name */	backend_idle_timeout,
-	/* tweak */	timeout,
+	/* typ */	timeout,
 	/* min */	"1",
 	/* max */	NULL,
 	/* default */	"60",
@@ -231,108 +231,101 @@ PARAM(
 
 PARAM(
 	/* name */	cli_buffer,
-	/* tweak */	tweak_bytes_u,
-	/* var */	cli_buffer,
-	/* min */	4k,
-	/* max */	none,
-	/* default */	8k,
-	/* units */	bytes,
-	/* flags */	00,
+	/* typ */	bytes_u,
+	/* min */	"4k",
+	/* max */	NULL,
+	/* default */	"8k",
+	/* units */	"bytes",
+	/* flags */	0,
 	/* s-text */
 	"Size of buffer for CLI command input.\n"
 	"You may need to increase this if you have big VCL files and use "
 	"the vcl.inline CLI command.\n"
-	"NB: Must be specified with -p to have effect.\n",
+	"NB: Must be specified with -p to have effect.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	cli_limit,
-	/* tweak */	tweak_bytes_u,
-	/* var */	cli_limit,
-	/* min */	128b,
-	/* max */	99999999b,
-	/* default */	48k,
-	/* units */	bytes,
-	/* flags */	00,
+	/* typ */	bytes_u,
+	/* min */	"128b",
+	/* max */	"99999999b",
+	/* default */	"48k",
+	/* units */	"bytes",
+	/* flags */	0,
 	/* s-text */
 	"Maximum size of CLI response.  If the response exceeds this "
 	"limit, the response code will be 201 instead of 200 and the last "
-	"line will indicate the truncation.\n",
+	"line will indicate the truncation.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	cli_timeout,
-	/* tweak */	tweak_timeout,
-	/* var */	cli_timeout,
-	/* min */	0.000,
-	/* max */	none,
-	/* default */	60.000,
-	/* units */	seconds,
-	/* flags */	00,
+	/* typ */	timeout,
+	/* min */	"0.000",
+	/* max */	NULL,
+	/* default */	"60.000",
+	/* units */	"seconds",
+	/* flags */	0,
 	/* s-text */
 	"Timeout for the childs replies to CLI requests from the "
-	"mgt_param.\n",
+	"mgt_param.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	clock_skew,
-	/* tweak */	tweak_uint,
-	/* var */	clock_skew,
-	/* min */	0,
-	/* max */	none,
-	/* default */	10,
-	/* units */	seconds,
-	/* flags */	00,
+	/* typ */	uint,
+	/* min */	"0",
+	/* max */	NULL,
+	/* default */	"10",
+	/* units */	"seconds",
+	/* flags */	0,
 	/* s-text */
 	"How much clockskew we are willing to accept between the backend "
-	"and our own clock.\n",
+	"and our own clock.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	connect_timeout,
-	/* tweak */	tweak_timeout,
-	/* var */	connect_timeout,
-	/* min */	0.000,
-	/* max */	none,
-	/* default */	3.500,
-	/* units */	seconds,
-	/* flags */	00,
+	/* typ */	timeout,
+	/* min */	"0.000",
+	/* max */	NULL,
+	/* default */	"3.500",
+	/* units */	"seconds",
+	/* flags */	0,
 	/* s-text */
 	"Default connection timeout for backend connections. We only try "
 	"to connect to the backend for this many seconds before giving up. "
 	"VCL can override this default value for each backend and backend "
-	"request.\n",
+	"request.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	critbit_cooloff,
-	/* tweak */	tweak_timeout,
-	/* var */	critbit_cooloff,
-	/* min */	60.000,
-	/* max */	254.000,
-	/* default */	180.000,
-	/* units */	seconds,
-	/* flags */	0| WIZARD,
+	/* typ */	timeout,
+	/* min */	"60.000",
+	/* max */	"254.000",
+	/* default */	"180.000",
+	/* units */	"seconds",
+	/* flags */	WIZARD,
 	/* s-text */
 	"How long the critbit hasher keeps deleted objheads on the cooloff "
-	"list.\n",
+	"list.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #if 0
 PARAM(
 	/* name */	debug,
-	/* tweak */	tweak_mask,
-	/* var */	debug,
-	/* min */	none,
-	/* max */	none,
-	/* default */	none,
-	/* units */	,
+	/* typ */	debug,
+	/* min */	NULL,
+	/* max */	NULL,
+	/* default */	NULL,
+	/* units */	NULL,
 	/* flags */	0,
 	/* s-text */
 	"Enable/Disable various kinds of debugging.\n"
@@ -349,69 +342,65 @@ PARAM(
 	"	lurker	VSL Ban lurker\n"
 	"	esi_chop	Chop ESI fetch to bits\n"
 	"	flush_head	Flush after http1 head\n"
-	"	vtc_mode	Varnishtest Mode\n",
+	"	vtc_mode	Varnishtest Mode",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #endif
 PARAM(
 	/* name */	default_grace,
-	/* tweak */	tweak_timeout,
-	/* var */	default_grace,
-	/* min */	0.000,
-	/* max */	none,
-	/* default */	10.000,
-	/* units */	seconds,
-	/* flags */	0,
+	/* typ */	timeout,
+	/* min */	"0.000",
+	/* max */	NULL,
+	/* default */	"10.000",
+	/* units */	"seconds",
+	/* flags */	OBJ_STICKY,
 	/* s-text */
 	"Default grace period.  We will deliver an object this long after "
 	"it has expired, provided another thread is attempting to get a "
-	"new copy.\n",
+	"new copy.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	default_keep,
-	/* tweak */	tweak_timeout,
-	/* var */	default_keep,
-	/* min */	0.000,
-	/* max */	none,
-	/* default */	0.000,
-	/* units */	seconds,
-	/* flags */	0,
+	/* typ */	timeout,
+	/* min */	"0.000",
+	/* max */	NULL,
+	/* default */	"0.000",
+	/* units */	"seconds",
+	/* flags */	OBJ_STICKY,
 	/* s-text */
 	"Default keep period.  We will keep a useless object around this "
 	"long, making it available for conditional backend fetches.  That "
 	"means that the object will be removed from the cache at the end "
-	"of ttl+grace+keep.\n",
+	"of ttl+grace+keep.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	default_ttl,
-	/* tweak */	tweak_timeout,
-	/* var */	default_ttl,
-	/* min */	0.000,
-	/* max */	none,
-	/* default */	120.000,
-	/* units */	seconds,
-	/* flags */	0,
+	/* typ */	timeout,
+	/* min */	"0.000",
+	/* max */	NULL,
+	/* default */	"120.000",
+	/* units */	"seconds",
+	/* flags */	OBJ_STICKY,
 	/* s-text */
 	"The TTL assigned to objects if neither the backend nor the VCL "
-	"code assigns one.\n",
+	"code assigns one.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #if 0
 PARAM(
 	/* name */	feature,
-	/* tweak */	tweak_mask
-	/* var */	feature,
-	/* min */	none,
-	/* max */	none,
-	/* default */	none,
-	/* units */	,
-	/* flags */	00,
+	/* typ */	feature,
+	/* min */	NULL,
+	/* max */	NULL,
+	/* default */	NULL,
+	/* units */	NULL,
+	/* flags */	0,
 	/* s-text */
 	"Enable/Disable various minor features.\n"
 	"	none	Disable all features.\n"
@@ -423,99 +412,93 @@ PARAM(
 	"	esi_ignore_https	Treat HTTPS as HTTP in ESI:includes\n"
 	"	esi_disable_xml_check	Don't check of body looks like XML\n"
 	"	esi_ignore_other_elements	Ignore non-esi XML-elements\n"
-	"	esi_remove_bom	Remove UTF-8 BOM\n",
+	"	esi_remove_bom	Remove UTF-8 BOM",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #endif
 PARAM(
 	/* name */	fetch_chunksize,
-	/* tweak */	tweak_bytes,
-	/* var */	fetch_chunksize,
-	/* min */	4k,
-	/* max */	none,
-	/* default */	16k,
-	/* units */	bytes,
-	/* flags */	0| EXPERIMENTAL,
+	/* typ */	bytes,
+	/* min */	"4k",
+	/* max */	NULL,
+	/* default */	"16k",
+	/* units */	"bytes",
+	/* flags */	EXPERIMENTAL,
 	/* s-text */
 	"The default chunksize used by fetcher. This should be bigger than "
 	"the majority of objects with short TTLs.\n"
 	"Internal limits in the storage_file module makes increases above "
-	"128kb a dubious idea.\n",
+	"128kb a dubious idea.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	fetch_maxchunksize,
-	/* tweak */	tweak_bytes,
-	/* var */	fetch_maxchunksize,
-	/* min */	64k,
-	/* max */	none,
-	/* default */	0.25G,
-	/* units */	bytes,
-	/* flags */	0| EXPERIMENTAL,
+	/* typ */	bytes,
+	/* min */	"64k",
+	/* max */	NULL,
+	/* default */	"0.25G",
+	/* units */	"bytes",
+	/* flags */	EXPERIMENTAL,
 	/* s-text */
 	"The maximum chunksize we attempt to allocate from storage. Making "
-	"this too large may cause delays and storage fragmentation.\n",
+	"this too large may cause delays and storage fragmentation.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	gzip_buffer,
-	/* tweak */	tweak_bytes_u,
-	/* var */	gzip_buffer,
-	/* min */	2k,
-	/* max */	none,
-	/* default */	32k,
-	/* units */	bytes,
-	/* flags */	0| EXPERIMENTAL,
+	/* typ */	bytes_u,
+	/* min */	"2k",
+	/* max */	NULL,
+	/* default */	"32k",
+	/* units */	"bytes",
+	/* flags */	EXPERIMENTAL,
 	/* s-text */
 	"Size of malloc buffer used for gzip processing.\n"
 	"These buffers are used for in-transit data, for instance "
 	"gunzip'ed data being sent to a client.Making this space to small "
 	"results in more overhead, writes to sockets etc, making it too "
-	"big is probably just a waste of memory.\n",
+	"big is probably just a waste of memory.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	gzip_level,
-	/* tweak */	tweak_uint,
-	/* var */	gzip_level,
-	/* min */	0,
-	/* max */	9,
-	/* default */	6,
-	/* units */	,
-	/* flags */	00,
+	/* typ */	uint,
+	/* min */	"0",
+	/* max */	"9",
+	/* default */	"6",
+	/* units */	NULL,
+	/* flags */	0,
 	/* s-text */
-	"Gzip compression level: 0=debug, 1=fast, 9=best\n",
+	"Gzip compression level: 0=debug, 1=fast, 9=best",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	gzip_memlevel,
-	/* tweak */	tweak_uint,
-	/* var */	gzip_memlevel,
-	/* min */	1,
-	/* max */	9,
-	/* default */	8,
-	/* units */	,
-	/* flags */	00,
+	/* typ */	uint,
+	/* min */	"1",
+	/* max */	"9",
+	/* default */	"8",
+	/* units */	NULL,
+	/* flags */	0,
 	/* s-text */
 	"Gzip memory level 1=slow/least, 9=fast/most compression.\n"
-	"Memory impact is 1=1k, 2=2k, ... 9=256k.\n",
+	"Memory impact is 1=1k, 2=2k, ... 9=256k.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	http_gzip_support,
-	/* tweak */	tweak_bool,
-	/* var */	http_gzip_support,
-	/* min */	none,
-	/* max */	none,
-	/* default */	on,
-	/* units */	bool,
-	/* flags */	00,
+	/* typ */	bool,
+	/* min */	NULL,
+	/* max */	NULL,
+	/* default */	"on",
+	/* units */	"bool",
+	/* flags */	0,
 	/* s-text */
 	"Enable gzip support. When enabled Varnish request compressed "
 	"objects from the backend and store them compressed. If a client "
@@ -526,519 +509,478 @@ PARAM(
 	"\n"
 	"Clients that do not support gzip will have their Accept-Encoding "
 	"header removed. For more information on how gzip is implemented "
-	"please see the chapter on gzip in the Varnish reference.\n",
+	"please see the chapter on gzip in the Varnish reference.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	http_max_hdr,
-	/* tweak */	tweak_uint,
-	/* var */	http_max_hdr,
-	/* min */	32,
-	/* max */	65535,
-	/* default */	64,
-	/* units */	header lines,
-	/* flags */	00,
+	/* typ */	uint,
+	/* min */	"32",
+	/* max */	"65535",
+	/* default */	"64",
+	/* units */	"header lines",
+	/* flags */	0,
 	/* s-text */
 	"Maximum number of HTTP header lines we allow in "
 	"{req|resp|bereq|beresp}.http (obj.http is autosized to the exact "
 	"number of headers).\n"
 	"Cheap, ~20 bytes, in terms of workspace memory.\n"
-	"Note that the first line occupies five header lines.\n",
+	"Note that the first line occupies five header lines.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	http_range_support,
-	/* tweak */	tweak_bool,
-	/* var */	http_range_support,
-	/* min */	none,
-	/* max */	none,
-	/* default */	on,
-	/* units */	bool,
-	/* flags */	00,
+	/* typ */	bool,
+	/* min */	NULL,
+	/* max */	NULL,
+	/* default */	"on",
+	/* units */	"bool",
+	/* flags */	0,
 	/* s-text */
-	"Enable support for HTTP Range headers.\n",
+	"Enable support for HTTP Range headers.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	http_req_hdr_len,
-	/* tweak */	tweak_bytes_u,
-	/* var */	http_req_hdr_len,
-	/* min */	40b,
-	/* max */	none,
-	/* default */	8k,
-	/* units */	bytes,
-	/* flags */	00,
+	/* typ */	bytes_u,
+	/* min */	"40b",
+	/* max */	NULL,
+	/* default */	"8k",
+	/* units */	"bytes",
+	/* flags */	0,
 	/* s-text */
 	"Maximum length of any HTTP client request header we will allow.  "
-	"The limit is inclusive its continuation lines.\n",
+	"The limit is inclusive its continuation lines.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	http_req_size,
-	/* tweak */	tweak_bytes_u,
-	/* var */	http_req_size,
-	/* min */	0.25k,
-	/* max */	none,
-	/* default */	32k,
-	/* units */	bytes,
-	/* flags */	00,
+	/* typ */	bytes_u,
+	/* min */	"0.25k",
+	/* max */	NULL,
+	/* default */	"32k",
+	/* units */	"bytes",
+	/* flags */	0,
 	/* s-text */
 	"Maximum number of bytes of HTTP client request we will deal with. "
 	" This is a limit on all bytes up to the double blank line which "
 	"ends the HTTP request.\n"
 	"The memory for the request is allocated from the client workspace "
 	"(param: workspace_client) and this parameter limits how much of "
-	"that the request is allowed to take up.\n",
+	"that the request is allowed to take up.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	http_resp_hdr_len,
-	/* tweak */	tweak_bytes_u,
-	/* var */	http_resp_hdr_len,
-	/* min */	40b,
-	/* max */	none,
-	/* default */	8k,
-	/* units */	bytes,
-	/* flags */	00,
+	/* typ */	bytes_u,
+	/* min */	"40b",
+	/* max */	NULL,
+	/* default */	"8k",
+	/* units */	"bytes",
+	/* flags */	0,
 	/* s-text */
 	"Maximum length of any HTTP backend response header we will allow. "
-	" The limit is inclusive its continuation lines.\n",
+	" The limit is inclusive its continuation lines.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	http_resp_size,
-	/* tweak */	tweak_bytes_u,
-	/* var */	http_resp_size,
-	/* min */	0.25k,
-	/* max */	none,
-	/* default */	32k,
-	/* units */	bytes,
-	/* flags */	00,
+	/* typ */	bytes_u,
+	/* min */	"0.25k",
+	/* max */	NULL,
+	/* default */	"32k",
+	/* units */	"bytes",
+	/* flags */	0,
 	/* s-text */
 	"Maximum number of bytes of HTTP backend response we will deal "
 	"with.  This is a limit on all bytes up to the double blank line "
 	"which ends the HTTP request.\n"
 	"The memory for the request is allocated from the worker workspace "
 	"(param: thread_pool_workspace) and this parameter limits how much "
-	"of that the request is allowed to take up.\n",
+	"of that the request is allowed to take up.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	idle_send_timeout,
-	/* tweak */	tweak_timeout,
-	/* var */	idle_send_timeout,
-	/* min */	0.000,
-	/* max */	none,
-	/* default */	60.000,
-	/* units */	seconds,
-	/* flags */	0| DELAYED_EFFECT,
+	/* typ */	timeout,
+	/* min */	"0.000",
+	/* max */	NULL,
+	/* default */	"60.000",
+	/* units */	"seconds",
+	/* flags */	DELAYED_EFFECT,
 	/* s-text */
 	"Time to wait with no data sent. If no data has been transmitted "
 	"in this many\n"
 	"seconds the session is closed.\n"
-	"See setsockopt(2) under SO_SNDTIMEO for more information.\n",
+	"See setsockopt(2) under SO_SNDTIMEO for more information.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	listen_depth,
-	/* tweak */	tweak_uint,
-	/* var */	listen_depth,
-	/* min */	0,
-	/* max */	none,
-	/* default */	1024,
-	/* units */	connections,
-	/* flags */	0| MUST_RESTART,
+	/* typ */	uint,
+	/* min */	"0",
+	/* max */	NULL,
+	/* default */	"1024",
+	/* units */	"connections",
+	/* flags */	MUST_RESTART,
 	/* s-text */
-	"Listen queue depth.\n",
+	"Listen queue depth.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	lru_interval,
-	/* tweak */	tweak_timeout,
-	/* var */	lru_interval,
-	/* min */	0.000,
-	/* max */	none,
-	/* default */	2.000,
-	/* units */	seconds,
-	/* flags */	0| EXPERIMENTAL,
+	/* typ */	timeout,
+	/* min */	"0.000",
+	/* max */	NULL,
+	/* default */	"2.000",
+	/* units */	"seconds",
+	/* flags */	EXPERIMENTAL,
 	/* s-text */
 	"Grace period before object moves on LRU list.\n"
 	"Objects are only moved to the front of the LRU list if they have "
 	"not been moved there already inside this timeout period.  This "
 	"reduces the amount of lock operations necessary for LRU list "
-	"access.\n",
+	"access.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	max_esi_depth,
-	/* tweak */	tweak_uint,
-	/* var */	max_esi_depth,
-	/* min */	0,
-	/* max */	none,
-	/* default */	5,
-	/* units */	levels,
-	/* flags */	00,
+	/* typ */	uint,
+	/* min */	"0",
+	/* max */	NULL,
+	/* default */	"5",
+	/* units */	"levels",
+	/* flags */	0,
 	/* s-text */
-	"Maximum depth of esi:include processing.\n",
+	"Maximum depth of esi:include processing.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	max_restarts,
-	/* tweak */	tweak_uint,
-	/* var */	max_restarts,
-	/* min */	0,
-	/* max */	none,
-	/* default */	4,
-	/* units */	restarts,
-	/* flags */	00,
+	/* typ */	uint,
+	/* min */	"0",
+	/* max */	NULL,
+	/* default */	"4",
+	/* units */	"restarts",
+	/* flags */	0,
 	/* s-text */
 	"Upper limit on how many times a request can restart.\n"
 	"Be aware that restarts are likely to cause a hit against the "
-	"backend, so don't increase thoughtlessly.\n",
+	"backend, so don't increase thoughtlessly.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	max_retries,
-	/* tweak */	tweak_uint,
-	/* var */	max_retries,
-	/* min */	0,
-	/* max */	none,
-	/* default */	4,
-	/* units */	retries,
-	/* flags */	00,
+	/* typ */	uint,
+	/* min */	"0",
+	/* max */	NULL,
+	/* default */	"4",
+	/* units */	"retries",
+	/* flags */	0,
 	/* s-text */
-	"Upper limit on how many times a backend fetch can retry.\n",
+	"Upper limit on how many times a backend fetch can retry.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	nuke_limit,
-	/* tweak */	tweak_uint,
-	/* var */	nuke_limit,
-	/* min */	0,
-	/* max */	none,
-	/* default */	50,
-	/* units */	allocations,
-	/* flags */	0| EXPERIMENTAL,
+	/* typ */	uint,
+	/* min */	"0",
+	/* max */	NULL,
+	/* default */	"50",
+	/* units */	"allocations",
+	/* flags */	EXPERIMENTAL,
 	/* s-text */
 	"Maximum number of objects we attempt to nuke in order to make "
-	"space for a object body.\n",
+	"space for a object body.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #if 0
 PARAM(
 	/* name */	pcre_match_limit,
-	/* tweak */	tweak_uint,
-	/* var */	pcre_match_limit,
-	/* min */	1,
-	/* max */	none,
-	/* default */	10000,
-	/* units */	,
-	/* flags */	00,
+	/* typ */	uint,
+	/* min */	"1",
+	/* max */	NULL,
+	/* default */	"1.000",
+	/* units */	NULL,
+	/* flags */	0,
 	/* s-text */
 	"The limit for the  number of internal matching function calls in "
-	"a pcre_exec() execution.\n",
+	"a pcre_exec() execution.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	pcre_match_limit_recursion,
-	/* tweak */	tweak_uint,
-	/* var */	pcre_match_limit_recursion,
-	/* min */	1,
-	/* max */	none,
-	/* default */	10000,
-	/* units */	,
-	/* flags */	00,
+	/* typ */	uint,
+	/* min */	"1",
+	/* max */	NULL,
+	/* default */	"1.000",
+	/* units */	NULL,
+	/* flags */	0,
 	/* s-text */
 	"The limit for the  number of internal matching function "
-	"recursions in a pcre_exec() execution.\n",
+	"recursions in a pcre_exec() execution.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #endif
 PARAM(
 	/* name */	ping_interval,
-	/* tweak */	tweak_uint,
-	/* var */	ping_interval,
-	/* min */	0,
-	/* max */	none,
-	/* default */	3,
-	/* units */	seconds,
-	/* flags */	0| MUST_RESTART,
+	/* typ */	uint,
+	/* min */	"0",
+	/* max */	NULL,
+	/* default */	"3",
+	/* units */	"seconds",
+	/* flags */	MUST_RESTART,
 	/* s-text */
 	"Interval between pings from parent to child.\n"
 	"Zero will disable pinging entirely, which makes it possible to "
-	"attach a debugger to the child.\n",
+	"attach a debugger to the child.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	pipe_timeout,
-	/* tweak */	tweak_timeout,
-	/* var */	pipe_timeout,
-	/* min */	0.000,
-	/* max */	none,
-	/* default */	60.000,
-	/* units */	seconds,
-	/* flags */	00,
+	/* typ */	timeout,
+	/* min */	"0.000",
+	/* max */	NULL,
+	/* default */	"60.000",
+	/* units */	"seconds",
+	/* flags */	0,
 	/* s-text */
 	"Idle timeout for PIPE sessions. If nothing have been received in "
-	"either direction for this many seconds, the session is closed.\n",
+	"either direction for this many seconds, the session is closed.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #if 0
 PARAM(
 	/* name */	pool_req,
-	/* tweak */	tweak_poolparam,
-	/* var */	pool_req,
-	/* min */	none,
-	/* max */	none,
-	/* default */	10\,100\,10,
-	/* units */	none,
-	/* flags */	00,
+	/* typ */	poolparam,
+	/* min */	NULL,
+	/* max */	NULL,
+	/* default */	"10,100,10",
+	/* units */	NULL,
+	/* flags */	0,
 	/* s-text */
 	"Parameters for per worker pool request memory pool.\n"
-	"The three numbers are:\n"
-	"	min_pool	minimum size of free pool.\n"
-	"	max_pool	maximum size of free pool.\n"
-	"	max_age		max age of free element.\n",
+	MEMPOOL_TEXT,
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	pool_sess,
-	/* tweak */	tweak_poolparam,
-	/* var */	pool_sess,
-	/* min */	none,
-	/* max */	none,
-	/* default */	10,100,10,
-	/* units */	,
-	/* flags */	00,
+	/* typ */	poolparam,
+	/* min */	NULL,
+	/* max */	NULL,
+	/* default */	"10,100,10",
+	/* units */	NULL,
+	/* flags */	0,
 	/* s-text */
 	"Parameters for per worker pool session memory pool.\n"
-	"The three numbers are:\n"
-	"	min_pool	minimum size of free pool.\n"
-	"	max_pool	maximum size of free pool.\n"
-	"	max_age	max age of free element.\n",
+	MEMPOOL_TEXT,
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	pool_vbo,
-	/* tweak */	tweak_poolparam,
-	/* var */	pool_vbo,
-	/* min */	none,
-	/* max */	none,
-	/* default */	10,100,10,
-	/* units */	,
-	/* flags */	00,
+	/* typ */	poolparam,
+	/* min */	NULL,
+	/* max */	NULL,
+	/* default */	"10,100,10",
+	/* units */	NULL,
+	/* flags */	0,
 	/* s-text */
 	"Parameters for backend object fetch memory pool.\n"
-	"The three numbers are:\n"
-	"	min_pool	minimum size of free pool.\n"
-	"	max_pool	maximum size of free pool.\n"
-	"	max_age	max age of free element.\n",
+	MEMPOOL_TEXT,
 	/* l-text */	"",
 	/* func */	NULL
 )
 #endif
 PARAM(
 	/* name */	prefer_ipv6,
-	/* tweak */	tweak_bool,
-	/* var */	prefer_ipv6,
-	/* min */	none,
-	/* max */	none,
-	/* default */	off,
-	/* units */	bool,
-	/* flags */	00,
+	/* typ */	bool,
+	/* min */	NULL,
+	/* max */	NULL,
+	/* default */	"off",
+	/* units */	"bool",
+	/* flags */	0,
 	/* s-text */
 	"Prefer IPv6 address when connecting to backends which have both "
-	"IPv4 and IPv6 addresses.\n",
+	"IPv4 and IPv6 addresses.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	rush_exponent,
-	/* tweak */	tweak_uint,
-	/* var */	rush_exponent,
-	/* min */	2,
-	/* max */	none,
-	/* default */	3,
-	/* units */	requests per request,
-	/* flags */	0| EXPERIMENTAL,
+	/* typ */	uint,
+	/* min */	"2",
+	/* max */	NULL,
+	/* default */	"3",
+	/* units */	"requests per request",
+	/* flags */	EXPERIMENTAL,
 	/* s-text */
 	"How many parked request we start for each completed request on "
 	"the object.\n"
 	"NB: Even with the implict delay of delivery, this parameter "
-	"controls an exponential increase in number of worker threads.\n",
+	"controls an exponential increase in number of worker threads.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	send_timeout,
-	/* tweak */	tweak_timeout,
-	/* var */	send_timeout,
-	/* min */	0.000,
-	/* max */	none,
-	/* default */	600.000,
-	/* units */	seconds,
-	/* flags */	0| DELAYED_EFFECT,
+	/* typ */	timeout,
+	/* min */	"0.000",
+	/* max */	NULL,
+	/* default */	"600.000",
+	/* units */	"seconds",
+	/* flags */	DELAYED_EFFECT,
 	/* s-text */
 	"Send timeout for client connections. If the HTTP response hasn't "
 	"been transmitted in this many\n"
 	"seconds the session is closed.\n"
-	"See setsockopt(2) under SO_SNDTIMEO for more information.\n",
+	"See setsockopt(2) under SO_SNDTIMEO for more information.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #if 0
 PARAM(
 	/* name */	session_max,
-	/* tweak */	tweak_uint,
-	/* var */	session_max,
-	/* min */	1000,
-	/* max */	none,
-	/* default */	100000,
-	/* units */	sessions,
-	/* flags */	00,
+	/* typ */	uint,
+	/* min */	"1000",
+	/* max */	NULL,
+	/* default */	"10.000",
+	/* units */	"sessions",
+	/* flags */	0,
 	/* s-text */
 	"Maximum number of sessions we will allocate from one pool before "
 	"just dropping connections.\n"
 	"This is mostly an anti-DoS measure, and setting it plenty high "
-	"should not hurt, as long as you have the memory for it.\n",
+	"should not hurt, as long as you have the memory for it.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 
 PARAM(
 	/* name */	shm_reclen,
-	/* tweak */	tweak_vsl_reclen,
-	/* var */	shm_reclen,
-	/* min */	16b,
-	/* max */	4084,
-	/* default */	255b,
-	/* units */	bytes,
-	/* flags */	00,
+	/* typ */	vsl_reclen,
+	/* min */	"16b",
+	/* max */	"4084",
+	/* default */	"255b",
+	/* units */	"bytes",
+	/* flags */	0,
 	/* s-text */
-	"Old name for vsl_reclen, use that instead.\n",
+	"Old name for vsl_reclen, use that instead.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #endif
 PARAM(
 	/* name */	shortlived,
-	/* tweak */	tweak_timeout,
-	/* var */	shortlived,
-	/* min */	0.000,
-	/* max */	none,
-	/* default */	10.000,
-	/* units */	seconds,
-	/* flags */	00,
+	/* typ */	timeout,
+	/* min */	"0.000",
+	/* max */	NULL,
+	/* default */	"10.000",
+	/* units */	"seconds",
+	/* flags */	0,
 	/* s-text */
 	"Objects created with (ttl+grace+keep) shorter than this are "
-	"always put in transient storage.\n",
+	"always put in transient storage.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	sigsegv_handler,
-	/* tweak */	tweak_bool,
-	/* var */	sigsegv_handler,
-	/* min */	none,
-	/* max */	none,
-	/* default */	on,
-	/* units */	bool,
-	/* flags */	0| MUST_RESTART,
+	/* typ */	bool,
+	/* min */	NULL,
+	/* max */	NULL,
+	/* default */	"on",
+	/* units */	"bool",
+	/* flags */	MUST_RESTART,
 	/* s-text */
 	"Install a signal handler which tries to dump debug information on "
-	"segmentation faults, bus errors and abort signals.\n",
+	"segmentation faults, bus errors and abort signals.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	syslog_cli_traffic,
-	/* tweak */	tweak_bool,
-	/* var */	syslog_cli_traffic,
-	/* min */	none,
-	/* max */	none,
-	/* default */	on,
-	/* units */	bool,
-	/* flags */	00,
+	/* typ */	bool,
+	/* min */	NULL,
+	/* max */	NULL,
+	/* default */	"on",
+	/* units */	"bool",
+	/* flags */	0,
 	/* s-text */
-	"Log all CLI traffic to syslog(LOG_INFO).\n",
+	"Log all CLI traffic to syslog(LOG_INFO).",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	tcp_keepalive_intvl,
-	/* tweak */	tweak_timeout,
-	/* var */	tcp_keepalive_intvl,
-	/* min */	1.000,
-	/* max */	100.000,
-	/* default */	5.000,
-	/* units */	seconds,
-	/* flags */	0| EXPERIMENTAL,
+	/* typ */	timeout,
+	/* min */	"1.000",
+	/* max */	"100.000",
+	/* default */	"5.000",
+	/* units */	"seconds",
+	/* flags */	EXPERIMENTAL,
 	/* s-text */
-	"The number of seconds between TCP keep-alive probes.\n",
+	"The number of seconds between TCP keep-alive probes.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	tcp_keepalive_probes,
-	/* tweak */	tweak_uint,
-	/* var */	tcp_keepalive_probes,
-	/* min */	1,
-	/* max */	100,
-	/* default */	5,
-	/* units */	probes,
-	/* flags */	0| EXPERIMENTAL,
+	/* typ */	uint,
+	/* min */	"1",
+	/* max */	"100",
+	/* default */	"5",
+	/* units */	"probes",
+	/* flags */	EXPERIMENTAL,
 	/* s-text */
 	"The maximum number of TCP keep-alive probes to send before giving "
 	"up and killing the connection if no response is obtained from the "
-	"other end.\n",
+	"other end.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	tcp_keepalive_time,
-	/* tweak */	tweak_timeout,
-	/* var */	tcp_keepalive_time,
-	/* min */	1.000,
-	/* max */	7200.000,
-	/* default */	600.000,
-	/* units */	seconds,
-	/* flags */	0| EXPERIMENTAL,
+	/* typ */	timeout,
+	/* min */	"1.000",
+	/* max */	"7200.000",
+	/* default */	"600.000",
+	/* units */	"seconds",
+	/* flags */	EXPERIMENTAL,
 	/* s-text */
 	"The number of seconds a connection needs to be idle before TCP "
-	"begins sending out keep-alive probes.\n",
+	"begins sending out keep-alive probes.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #if 0
 PARAM(
 	/* name */	thread_pool_add_delay,
-	/* tweak */	tweak_timeout,
-	/* var */	thread_pool_add_delay,
-	/* min */	0.000,
-	/* max */	none,
-	/* default */	0.000,
-	/* units */	seconds,
-	/* flags */	0| EXPERIMENTAL,
+	/* typ */	timeout,
+	/* min */	"0.000",
+	/* max */	NULL,
+	/* default */	"0.000",
+	/* units */	"seconds",
+	/* flags */	EXPERIMENTAL,
 	/* s-text */
 	"Wait at least this long after creating a thread.\n"
 	"\n"
@@ -1046,34 +988,32 @@ PARAM(
 	"creating threads.\n"
 	"Set this to a few milliseconds if you see the 'threads_failed' "
 	"counter grow too much.\n"
-	"Setting this too high results in insuffient worker threads.\n",
+	"Setting this too high results in insuffient worker threads.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	thread_pool_destroy_delay,
-	/* tweak */	tweak_timeout,
-	/* var */	thread_pool_destroy_delay,
-	/* min */	0.010,
-	/* max */	none,
-	/* default */	1.000,
-	/* units */	seconds,
-	/* flags */	0| DELAYED_EFFECT| EXPERIMENTAL,
+	/* typ */	timeout,
+	/* min */	"0.010",
+	/* max */	NULL,
+	/* default */	"1.000",
+	/* units */	"seconds",
+	/* flags */	DELAYED_EFFECT| EXPERIMENTAL,
 	/* s-text */
 	"Wait this long after destroying a thread.\n"
-	"This controls the decay of thread pools when idle(-ish).\n",
+	"This controls the decay of thread pools when idle(-ish).",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	thread_pool_fail_delay,
-	/* tweak */	tweak_timeout,
-	/* var */	thread_pool_fail_delay,
-	/* min */	0.010,
-	/* max */	none,
-	/* default */	0.200,
-	/* units */	seconds,
-	/* flags */	0| EXPERIMENTAL,
+	/* typ */	timeout,
+	/* min */	"0.010",
+	/* max */	NULL,
+	/* default */	"0.200",
+	/* units */	"seconds",
+	/* flags */	EXPERIMENTAL,
 	/* s-text */
 	"Wait at least this long after a failed thread creation before "
 	"trying to create another thread.\n"
@@ -1087,88 +1027,83 @@ PARAM(
 	"\n"
 	"It may also help to increase thread_pool_timeout and "
 	"thread_pool_min, to reduce the rate at which treads are destroyed "
-	"and later recreated.\n",
+	"and later recreated.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	thread_pool_max,
-	/* tweak */	tweak_***,
-	/* var */	thread_pool_max,
-	/* min */	100,
-	/* max */	none,
-	/* default */	5000,
-	/* units */	threads,
-	/* flags */	0| DELAYED_EFFECT,
+	/* typ */	thread_pool_max,
+	/* min */	"100",
+	/* max */	NULL,
+	/* default */	"5000",
+	/* units */	"threads",
+	/* flags */	DELAYED_EFFECT,
 	/* s-text */
 	"The maximum number of worker threads in each pool.\n"
 	"\n"
 	"Do not set this higher than you have to, since excess worker "
 	"threads soak up RAM and CPU and generally just get in the way of "
-	"getting work done.\n",
+	"getting work done.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	thread_pool_min,
-	/* tweak */	tweak_***,
-	/* var */	thread_pool_min,
-	/* min */	none,
-	/* max */	5000,
-	/* default */	100,
-	/* units */	threads,
-	/* flags */	0| DELAYED_EFFECT,
+	/* typ */	thread_pool_min,
+	/* min */	NULL,
+	/* max */	"5000",
+	/* default */	"100",
+	/* units */	"threads",
+	/* flags */	DELAYED_EFFECT,
 	/* s-text */
 	"The minimum number of worker threads in each pool.\n"
 	"\n"
 	"Increasing this may help ramp up faster from low load situations "
 	"or when threads have expired."
-	"Minimum is 10 threads.\n",
+	"Minimum is 10 threads.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	thread_pool_stack,
-	/* tweak */	tweak_bytes,
-	/* var */	thread_pool_stack,
-	/* min */	2k,
-	/* max */	none,
-	/* default */	48k,
-	/* units */	bytes,
-	/* flags */	0| EXPERIMENTAL,
+	/* typ */	bytes,
+	/* min */	"2k",
+	/* max */	NULL,
+	/* default */	"48k",
+	/* units */	"bytes",
+	/* flags */	EXPERIMENTAL,
 	/* s-text */
 	"Worker thread stack size.\n"
 	"This will likely be rounded up to a multiple of 4k (or whatever "
-	"the page_size might be) by the kernel.\n",
+	"the page_size might be) by the kernel.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	thread_pool_timeout,
-	/* tweak */	tweak_timeout,
-	/* var */	thread_pool_timeout,
-	/* min */	10.000,
-	/* max */	none,
-	/* default */	300.000,
-	/* units */	seconds,
-	/* flags */	0| DELAYED_EFFECT| EXPERIMENTAL,
+	/* typ */	timeout,
+	/* min */	"10.000",
+	/* max */	NULL,
+	/* default */	"300.000",
+	/* units */	"seconds",
+	/* flags */	DELAYED_EFFECT| EXPERIMENTAL,
 	/* s-text */
 	"Thread idle threshold.\n"
 	"\n"
 	"Threads in excess of thread_pool_min, which have been idle for at "
-	"least this long, will be destroyed.\n",
+	"least this long, will be destroyed.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	thread_pools,
-	/* tweak */	tweak_uint,
-	/* var */	thread_pools,
-	/* min */	1,
-	/* max */	none,
-	/* default */	2,
-	/* units */	pools,
-	/* flags */	0| DELAYED_EFFECT| EXPERIMENTAL,
+	/* typ */	uint,
+	/* min */	"1",
+	/* max */	NULL,
+	/* default */	"2",
+	/* units */	"pools",
+	/* flags */	DELAYED_EFFECT| EXPERIMENTAL,
 	/* s-text */
 	"Number of worker thread pools.\n"
 	"\n"
@@ -1178,73 +1113,69 @@ PARAM(
 	"pool for each CPU is probably detrimal to performance.\n"
 	"\n"
 	"Can be increased on the fly, but decreases require a restart to "
-	"take effect.\n",
+	"take effect.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	thread_queue_limit,
-	/* tweak */	tweak_uint,
-	/* var */	thread_queue_limit,
-	/* min */	0,
-	/* max */	none,
-	/* default */	20,
-	/* units */	,
-	/* flags */	0| EXPERIMENTAL,
+	/* typ */	uint,
+	/* min */	"0",
+	/* max */	NULL,
+	/* default */	"20",
+	/* units */	NULL,
+	/* flags */	EXPERIMENTAL,
 	/* s-text */
 	"Permitted queue length per thread-pool.\n"
 	"\n"
 	"This sets the number of requests we will queue, waiting for an "
 	"available thread.  Above this limit sessions will be dropped "
-	"instead of queued.\n",
+	"instead of queued.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	thread_stats_rate,
-	/* tweak */	tweak_uint,
-	/* var */	thread_stats_rate,
-	/* min */	0,
-	/* max */	none,
-	/* default */	10,
-	/* units */	requests,
-	/* flags */	0| EXPERIMENTAL,
+	/* typ */	uint,
+	/* min */	"0",
+	/* max */	NULL,
+	/* default */	"10",
+	/* units */	"requests",
+	/* flags */	EXPERIMENTAL,
 	/* s-text */
 	"Worker threads accumulate statistics, and dump these into the "
 	"global stats counters if the lock is free when they finish a job "
 	"(request/fetch etc.)\n"
 	"This parameters defines the maximum number of jobs a worker "
 	"thread may handle, before it is forced to dump its accumulated "
-	"stats into the global counters.\n",
+	"stats into the global counters.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #endif
 PARAM(
 	/* name */	timeout_idle,
-	/* tweak */	tweak_timeout,
-	/* var */	timeout_idle,
-	/* min */	0.000,
-	/* max */	none,
-	/* default */	5.000,
-	/* units */	seconds,
-	/* flags */	00,
+	/* typ */	timeout,
+	/* min */	"0.000",
+	/* max */	NULL,
+	/* default */	"5.000",
+	/* units */	"seconds",
+	/* flags */	0,
 	/* s-text */
 	"Idle timeout for client connections.\n"
 	"A connection is considered idle, until we have received the full "
-	"request headers.\n",
+	"request headers.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	timeout_linger,
-	/* tweak */	tweak_timeout,
-	/* var */	timeout_linger,
-	/* min */	0.000,
-	/* max */	none,
-	/* default */	0.050,
-	/* units */	seconds,
-	/* flags */	0| EXPERIMENTAL,
+	/* typ */	timeout,
+	/* min */	"0.000",
+	/* max */	NULL,
+	/* default */	"0.050",
+	/* units */	"seconds",
+	/* flags */	EXPERIMENTAL,
 	/* s-text */
 	"How long the worker thread lingers on an idle session before "
 	"handing it over to the waiter.\n"
@@ -1252,22 +1183,21 @@ PARAM(
 	"within the first 100 msec of the previous request completing.\n"
 	"Setting this too high results in worker threads not doing "
 	"anything for their keep, setting it too low just means that more "
-	"sessions take a detour around the waiter.\n",
+	"sessions take a detour around the waiter.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #if 0
 PARAM(
 	/* name */	vcc_allow_inline_c,
-	/* tweak */	tweak_bool,
-	/* var */	vcc_allow_inline_c,
-	/* min */	none,
-	/* max */	none,
-	/* default */	off,
-	/* units */	bool,
-	/* flags */	00,
+	/* typ */	bool,
+	/* min */	NULL,
+	/* max */	NULL,
+	/* default */	"off",
+	/* units */	"bool",
+	/* flags */	0,
 	/* s-text */
-	"Allow inline C code in VCL.\n",
+	"Allow inline C code in VCL.",
 	/* l-text */	"",
 	/* func */	NULL
 )
@@ -1275,15 +1205,14 @@ PARAM(
 #if 0
 PARAM(
 	/* name */	vcc_err_unref,
-	/* tweak */	tweak_bool,
-	/* var */	vcc_err_unref,
-	/* min */	none,
-	/* max */	none,
-	/* default */	on,
-	/* units */	bool,
-	/* flags */	00,
+	/* typ */	bool,
+	/* min */	NULL,
+	/* max */	NULL,
+	/* default */	"on",
+	/* units */	"bool",
+	/* flags */	0,
 	/* s-text */
-	"Unreferenced VCL objects result in error.\n",
+	"Unreferenced VCL objects result in error.",
 	/* l-text */	"",
 	/* func */	NULL
 )
@@ -1291,231 +1220,217 @@ PARAM(
 #if 0
 PARAM(
 	/* name */	vcc_unsafe_path,
-	/* tweak */	tweak_bool,
-	/* var */	vcc_unsafe_path,
-	/* min */	none,
-	/* max */	none,
-	/* default */	on,
-	/* units */	bool,
-	/* flags */	00,
+	/* typ */	bool,
+	/* min */	NULL,
+	/* max */	NULL,
+	/* default */	"on",
+	/* units */	"bool",
+	/* flags */	0,
 	/* s-text */
 	"Allow '/' in vmod & include paths.\n"
-	"Allow 'import ... from ...'.\n",
+	"Allow 'import ... from ...'.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #endif
 PARAM(
 	/* name */	vcl_cooldown,
-	/* tweak */	tweak_timeout,
-	/* var */	vcl_cooldown,
-	/* min */	0.000,
-	/* max */	none,
-	/* default */	600.000,
-	/* units */	seconds,
-	/* flags */	00,
+	/* typ */	timeout,
+	/* min */	"0.000",
+	/* max */	NULL,
+	/* default */	"600.000",
+	/* units */	"seconds",
+	/* flags */	0,
 	/* s-text */
 	"How long time a VCL is kept warm after being replaced as the "
-	"active VCL.  (Granularity approximately 30 seconds.)\n",
+	"active VCL.  (Granularity approximately 30 seconds.)",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #if 0
 PARAM(
 	/* name */	vcl_dir,
-	/* tweak */	tweak_string,
-	/* var */	vcl_dir,
-	/* min */	none,
-	/* max */	none,
+	/* typ */	string,
+	/* min */	NULL,
+	/* max */	NULL,
 	/* default */	/opt/varnish/etc/varnish,
-	/* units */	(null),
-	/* flags */	00,
+	/* units */	NULL,
+	/* flags */	0,
 	/* s-text */
 	"Directory from which relative VCL filenames (vcl.load and "
-	"include) are opened.\n",
+	"include) are opened.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	vmod_dir,
-	/* tweak */	tweak_string,
-	/* var */	vmod_dir,
-	/* min */	none,
-	/* max */	none,
+	/* typ */	string,
+	/* min */	NULL,
+	/* max */	NULL,
 	/* default */	/opt/varnish/lib/varnish/vmods,
-	/* units */	(null),
-	/* flags */	00,
+	/* units */	NULL,
+	/* flags */	0,
 	/* s-text */
-	"Directory where VCL modules are to be found.\n",
+	"Directory where VCL modules are to be found.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #endif
 PARAM(
 	/* name */	vsl_buffer,
-	/* tweak */	tweak_vsl_buffer,
-	/* var */	vsl_buffer,
-	/* min */	267,
-	/* max */	none,
-	/* default */	4k,
-	/* units */	bytes,
-	/* flags */	00,
+	/* typ */	vsl_buffer,
+	/* min */	"267",
+	/* max */	NULL,
+	/* default */	"4k",
+	/* units */	"bytes",
+	/* flags */	0,
 	/* s-text */
 	"Bytes of (req-/backend-)workspace dedicated to buffering VSL "
 	"records.\n"
 	"Setting this too high costs memory, setting it too low will cause "
 	"more VSL flushes and likely increase lock-contention on the VSL "
 	"mutex.\n"
-	"The minimum tracks the vsl_reclen parameter + 12 bytes.\n",
+	"The minimum tracks the vsl_reclen parameter + 12 bytes.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #if 0
 PARAM(
 	/* name */	vsl_mask,
-	/* tweak */	tweak_mask
-	/* var */	vsl_mask,
-	/* min */	none,
-	/* max */	none,
-	/* default */	-VCL_trace,-WorkThread,-Hash,-VfpAcct,
-	/* units */	,
-	/* flags */	00,
+	/* typ */	vsl_mask,
+	/* min */	NULL,
+	/* max */	NULL,
+	/* default */	"default",
+	/* units */	NULL,
+	/* flags */	0,
 	/* s-text */
 	"Mask individual VSL messages from being logged.\n"
 	"	default	Set default value\n"
 	"\n"
 	"Use +/- prefixe in front of VSL tag name, to mask/unmask "
-	"individual VSL messages.\n",
+	"individual VSL messages.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #endif
 PARAM(
 	/* name */	vsl_reclen,
-	/* tweak */	tweak_vsl_reclen,
-	/* var */	vsl_reclen,
-	/* min */	16b,
-	/* max */	4084b,
-	/* default */	255b,
-	/* units */	bytes,
-	/* flags */	00,
+	/* typ */	vsl_reclen,
+	/* min */	"16b",
+	/* max */	"4084b",
+	/* default */	"255b",
+	/* units */	"bytes",
+	/* flags */	0,
 	/* s-text */
 	"Maximum number of bytes in SHM log record.\n"
-	"The maximum tracks the vsl_buffer parameter - 12 bytes.\n",
+	"The maximum tracks the vsl_buffer parameter - 12 bytes.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	vsl_space,
-	/* tweak */	tweak_bytes,
-	/* var */	vsl_space,
-	/* min */	1M,
-	/* max */	none,
-	/* default */	80M,
-	/* units */	bytes,
-	/* flags */	0| MUST_RESTART,
+	/* typ */	bytes,
+	/* min */	"1M",
+	/* max */	NULL,
+	/* default */	"80M",
+	/* units */	"bytes",
+	/* flags */	MUST_RESTART,
 	/* s-text */
 	"The amount of space to allocate for the VSL fifo buffer in the "
 	"VSM memory segment.  If you make this too small, "
 	"varnish{ncsa|log} etc will not be able to keep up.  Making it too "
-	"large just costs memory resources.\n",
+	"large just costs memory resources.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	vsm_space,
-	/* tweak */	tweak_bytes,
-	/* var */	vsm_space,
-	/* min */	1M,
-	/* max */	none,
-	/* default */	1M,
-	/* units */	bytes,
-	/* flags */	0| MUST_RESTART,
+	/* typ */	bytes,
+	/* min */	"1M",
+	/* max */	NULL,
+	/* default */	"1M",
+	/* units */	"bytes",
+	/* flags */	MUST_RESTART,
 	/* s-text */
 	"The amount of space to allocate for stats counters in the VSM "
 	"memory segment.  If you make this too small, some counters will "
-	"be invisible.  Making it too large just costs memory resources.\n",
+	"be invisible.  Making it too large just costs memory resources.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #if 0
 PARAM(
 	/* name */	waiter,
-	/* tweak */	tweak_waiter,
-	/* var */	waiter,
-	/* min */	none,
-	/* max */	none,
+	/* typ */	waiter,
+	/* min */	NULL,
+	/* max */	NULL,
 	/* default */	kqueue (possible values: kqueue, poll),
-	/* units */	(null),
-	/* flags */	0| MUST_RESTART| WIZARD,
+	/* units */	NULL,
+	/* flags */	MUST_RESTART| WIZARD,
 	/* s-text */
-	"Select the waiter kernel interface.\n",
+	"Select the waiter kernel interface.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #endif
 PARAM(
 	/* name */	workspace_backend,
-	/* tweak */	tweak_bytes_u,
-	/* var */	workspace_backend,
-	/* min */	1k,
-	/* max */	none,
-	/* default */	64k,
-	/* units */	bytes,
-	/* flags */	0| DELAYED_EFFECT,
+	/* typ */	bytes_u,
+	/* min */	"1k",
+	/* max */	NULL,
+	/* default */	"64k",
+	/* units */	"bytes",
+	/* flags */	DELAYED_EFFECT,
 	/* s-text */
 	"Bytes of HTTP protocol workspace for backend HTTP req/resp.  If "
-	"larger than 4k, use a multiple of 4k for VM efficiency.\n",
+	"larger than 4k, use a multiple of 4k for VM efficiency.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	workspace_client,
-	/* tweak */	tweak_bytes_u,
-	/* var */	workspace_client,
-	/* min */	9k,
-	/* max */	none,
-	/* default */	64k,
-	/* units */	bytes,
-	/* flags */	0| DELAYED_EFFECT,
+	/* typ */	bytes_u,
+	/* min */	"9k",
+	/* max */	NULL,
+	/* default */	"64k",
+	/* units */	"bytes",
+	/* flags */	DELAYED_EFFECT,
 	/* s-text */
 	"Bytes of HTTP protocol workspace for clients HTTP req/resp.  If "
-	"larger than 4k, use a multiple of 4k for VM efficiency.\n",
+	"larger than 4k, use a multiple of 4k for VM efficiency.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	workspace_session,
-	/* tweak */	tweak_bytes_u,
-	/* var */	workspace_session,
-	/* min */	0.25k,
-	/* max */	none,
-	/* default */	0.50k,
-	/* units */	bytes,
-	/* flags */	0| DELAYED_EFFECT,
+	/* typ */	bytes_u,
+	/* min */	"0.25k",
+	/* max */	NULL,
+	/* default */	"0.50k",
+	/* units */	"bytes",
+	/* flags */	DELAYED_EFFECT,
 	/* s-text */
 	"Allocation size for session structure and workspace.    The "
 	"workspace is primarily used for TCP connection addresses.  If "
-	"larger than 4k, use a multiple of 4k for VM efficiency.\n",
+	"larger than 4k, use a multiple of 4k for VM efficiency.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	workspace_thread,
-	/* tweak */	tweak_bytes_u,
-	/* var */	workspace_thread,
-	/* min */	0.25k,
-	/* max */	8k,
-	/* default */	2k,
-	/* units */	bytes,
-	/* flags */	0| DELAYED_EFFECT,
+	/* typ */	bytes_u,
+	/* min */	"0.25k",
+	/* max */	"8k",
+	/* default */	"2k",
+	/* units */	"bytes",
+	/* flags */	DELAYED_EFFECT,
 	/* s-text */
 	"Bytes of auxiliary workspace per thread.\n"
 	"This workspace is used for certain temporary data structures "
 	"during the operation of a worker thread.\n"
 	"One use is for the io-vectors for writing requests and responses "
 	"to sockets, having too little space will result in more writev(2) "
-	"system calls, having too much just wastes the space.\n",
+	"system calls, having too much just wastes the space.",
 	/* l-text */	"",
 	/* func */	NULL
 )



From fgsch at lodoss.net  Tue Jan  5 23:47:17 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Wed, 06 Jan 2016 00:47:17 +0100
Subject: [master] 1d5e5f0 Handle OBJ_STICKY in flags
Message-ID: <E1aGbJt-0002Ku-Hj@project.varnish-software.com>


commit 1d5e5f050fb84cf754bf3782a081ef490d452a2c
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Tue Jan 5 23:25:07 2016 +0000

    Handle OBJ_STICKY in flags

diff --git a/bin/varnishd/mgt/mgt_param.c b/bin/varnishd/mgt/mgt_param.c
index 2a52709..908643f 100644
--- a/bin/varnishd/mgt/mgt_param.c
+++ b/bin/varnishd/mgt/mgt_param.c
@@ -575,6 +575,10 @@ MCF_DumpRstParam(void)
 				printf("%sonly_root", q);
 				q = ", ";
 			}
+			if (pp->flags & OBJ_STICKY) {
+				printf("%sobj_sticky", q);
+				q = ", ";
+			}
 			printf("\n");
 		}
 		printf("\n");



From fgsch at lodoss.net  Tue Jan  5 23:47:17 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Wed, 06 Jan 2016 00:47:17 +0100
Subject: [master] 882afac Spelling
Message-ID: <E1aGbJt-0002L0-JI@project.varnish-software.com>


commit 882aface1e660e5038b30eef72ccc1fee68d3bd4
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Tue Jan 5 23:26:23 2016 +0000

    Spelling

diff --git a/bin/varnishd/mgt/mgt_param.c b/bin/varnishd/mgt/mgt_param.c
index 908643f..0ee9397 100644
--- a/bin/varnishd/mgt/mgt_param.c
+++ b/bin/varnishd/mgt/mgt_param.c
@@ -138,7 +138,7 @@ mcf_addpar(struct parspec *ps)
 
 /*--------------------------------------------------------------------
  * Wrap the text nicely.
- * Lines are allowed to contain to TABS and we render that as a table
+ * Lines are allowed to contain two TABS and we render that as a table
  * taking the width of the first column into account.
  */
 
diff --git a/bin/varnishd/mgt/mgt_param_bits.c b/bin/varnishd/mgt/mgt_param_bits.c
index 4713ac6..100d961 100644
--- a/bin/varnishd/mgt/mgt_param_bits.c
+++ b/bin/varnishd/mgt/mgt_param_bits.c
@@ -238,7 +238,7 @@ struct parspec VSL_parspec[] = {
 	{ "vsl_mask", tweak_vsl_mask, NULL, NULL, NULL,
 		"Mask individual VSL messages from being logged.\n"
 		"\tdefault\tSet default value\n"
-		"\nUse +/- prefixe in front of VSL tag name, to mask/unmask "
+		"\nUse +/- prefix in front of VSL tag name, to mask/unmask "
 		"individual VSL messages.",
 		0, "default", "" },
 	{ "debug", tweak_debug, NULL, NULL, NULL,
diff --git a/bin/varnishd/mgt/mgt_param_tbl.c b/bin/varnishd/mgt/mgt_param_tbl.c
index 129300f..3a6239b 100644
--- a/bin/varnishd/mgt/mgt_param_tbl.c
+++ b/bin/varnishd/mgt/mgt_param_tbl.c
@@ -207,7 +207,7 @@ struct parspec mgt_parspec[] = {
 	{ "nuke_limit",
 		tweak_uint, &mgt_param.nuke_limit,
 		"0", NULL,
-		"Maximum number of objects we attempt to nuke in order"
+		"Maximum number of objects we attempt to nuke in order "
 		"to make space for a object body.",
 		EXPERIMENTAL,
 		"50", "allocations" },



From fgsch at lodoss.net  Tue Jan  5 23:47:17 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Wed, 06 Jan 2016 00:47:17 +0100
Subject: [master] 5e6d168 Kill long forgotten and unused parameters
Message-ID: <E1aGbJt-0002L9-Kn@project.varnish-software.com>


commit 5e6d168683708ec88468a68b2626954785cfd99a
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Tue Jan 5 23:27:06 2016 +0000

    Kill long forgotten and unused parameters

diff --git a/bin/varnishd/common/params.h b/bin/varnishd/common/params.h
index 957b8a9..13ab904 100644
--- a/bin/varnishd/common/params.h
+++ b/bin/varnishd/common/params.h
@@ -85,15 +85,9 @@ struct params {
 #undef ptyp_vsl_reclen
 
 	/* Unprivileged user / group */
-	char			*user;
 	uid_t			uid;
-	char			*group;
 	gid_t			gid;
 
-	/* Extra group for compiler access */
-	char			*group_cc;
-	gid_t			gid_cc;
-
 	/* TTL used for lack of anything better */
 	double			default_ttl;
 
@@ -111,7 +105,6 @@ struct params {
 	unsigned		wthread_max;
 	double			wthread_timeout;
 	unsigned		wthread_pools;
-	unsigned		wthread_add_threshold;
 	double			wthread_add_delay;
 	double			wthread_fail_delay;
 	double			wthread_destroy_delay;
@@ -127,7 +120,6 @@ struct params {
 
 	unsigned		vsl_buffer;
 
-	unsigned		shm_workspace;
 	unsigned		http_req_size;
 	unsigned		http_req_hdr_len;
 	unsigned		http_resp_size;
@@ -196,8 +188,6 @@ struct params {
 	unsigned		gzip_level;
 	unsigned		gzip_memlevel;
 
-	unsigned		obj_readonly;
-
 	double			critbit_cooloff;
 
 	double			vcl_cooldown;



From fgsch at lodoss.net  Tue Jan  5 23:47:17 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Wed, 06 Jan 2016 00:47:17 +0100
Subject: [master] 38c4a7d This has been replaced by jails
Message-ID: <E1aGbJt-0002LI-MF@project.varnish-software.com>


commit 38c4a7d66ef6d6d4bfeca7cc75f82d8df6f5f7f0
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Tue Jan 5 23:27:50 2016 +0000

    This has been replaced by jails

diff --git a/bin/varnishd/mgt/mgt_sandbox.c b/bin/varnishd/mgt/mgt_sandbox.c
deleted file mode 100644
index 2b89a38..0000000
--- a/bin/varnishd/mgt/mgt_sandbox.c
+++ /dev/null
@@ -1,317 +0,0 @@
-/*-
- * Copyright (c) 2006-2011 Varnish Software AS
- * All rights reserved.
- *
- * Author: Poul-Henning Kamp <phk at phk.freebsd.dk>
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
- * 1. Redistributions of source code must retain the above copyright
- *    notice, this list of conditions and the following disclaimer.
- * 2. Redistributions in binary form must reproduce the above copyright
- *    notice, this list of conditions and the following disclaimer in the
- *    documentation and/or other materials provided with the distribution.
- *
- * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
- * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED.  IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
- * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
- * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
- * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
- * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
- * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
- * SUCH DAMAGE.
- *
- * Sandboxing child processes
- *
- * The worker/manager process border is one of the major security barriers
- * in Varnish, and therefore subject to whatever restrictions we have access
- * to under the given operating system.
- *
- * Unfortunately there is no consensus on APIs for this purpose, so each
- * operating system will require its own methods.
- *
- * This sourcefile tries to encapsulate the resulting mess on place.
- *
- * TODO:
- *	Unix:	chroot
- *	FreeBSD: jail
- *	FreeBSD: capsicum
- */
-
-#if 0
-
-#include "config.h"
-
-#ifdef __linux__
-#include <sys/prctl.h>
-#endif
-
-#include <pwd.h>
-#include <grp.h>
-#include <stdio.h>
-#include <stdlib.h>
-#include <string.h>
-#include <unistd.h>
-
-#include "mgt/mgt.h"
-#include "mgt/mgt_param.h"
-
-#include <vsub.h>
-
-mgt_sandbox_f *mgt_sandbox;
-
-/*--------------------------------------------------------------------
- * XXX: slightly magic.  We want to initialize to "nobody" (XXX: shouldn't
- * XXX: that be something autocrap found for us ?) but we don't want to
- * XXX: fail initialization if that user doesn't exists, even though we
- * XXX: do want to fail it, in subsequent sets.
- * XXX: The magic init string is a hack for this.
- */
-
-static int
-tweak_user(struct vsb *vsb, const struct parspec *par, const char *arg)
-{
-	struct passwd *pw;
-
-	(void)par;
-	if (arg != NULL) {
-		pw = getpwnam(arg);
-		if (pw == NULL) {
-			VSB_printf(vsb, "Unknown user '%s'", arg);
-			return(-1);
-		}
-		REPLACE(mgt_param.user, pw->pw_name);
-		mgt_param.uid = pw->pw_uid;
-		endpwent();
-	} else if (mgt_param.user) {
-		VSB_printf(vsb, "%s (%d)", mgt_param.user, (int)mgt_param.uid);
-	} else {
-		VSB_printf(vsb, "UID %d", (int)mgt_param.uid);
-	}
-	return (0);
-}
-
-/*--------------------------------------------------------------------
- * XXX: see comment for tweak_user, same thing here.
- */
-
-static int
-tweak_group(struct vsb *vsb, const struct parspec *par, const char *arg)
-{
-	struct group *gr;
-
-	(void)par;
-	if (arg != NULL) {
-		gr = getgrnam(arg);
-		if (gr == NULL) {
-			VSB_printf(vsb, "Unknown group '%s'", arg);
-			return(-1);
-		}
-		REPLACE(mgt_param.group, gr->gr_name);
-		mgt_param.gid = gr->gr_gid;
-		endgrent();
-	} else if (mgt_param.group) {
-		VSB_printf(vsb, "%s (%d)", mgt_param.group, (int)mgt_param.gid);
-	} else {
-		VSB_printf(vsb, "GID %d", (int)mgt_param.gid);
-	}
-	return (0);
-}
-
-/*--------------------------------------------------------------------
- * XXX: see comment for tweak_user, same thing here.
- */
-
-static int
-tweak_group_cc(struct vsb *vsb, const struct parspec *par, const char *arg)
-{
-	struct group *gr;
-
-	(void)par;
-	if (arg != NULL) {
-		if (*arg != '\0') {
-			gr = getgrnam(arg);
-			if (gr == NULL) {
-				VSB_printf(vsb, "Unknown group");
-				return(-1);
-			}
-			REPLACE(mgt_param.group_cc, gr->gr_name);
-			mgt_param.gid_cc = gr->gr_gid;
-		} else {
-			REPLACE(mgt_param.group_cc, "");
-			mgt_param.gid_cc = 0;
-		}
-	} else if (strlen(mgt_param.group_cc) > 0) {
-		VSB_printf(vsb, "%s (%d)",
-		    mgt_param.group_cc, (int)mgt_param.gid_cc);
-	} else {
-		VSB_printf(vsb, "<not set>");
-	}
-	return (0);
-}
-
-/*--------------------------------------------------------------------
- */
-
-static struct parspec mgt_parspec_sandbox[] = {
-	{ "user", tweak_user, NULL, NULL, NULL,
-		"The unprivileged user to run as.",
-		MUST_RESTART | ONLY_ROOT,
-		"" },
-	{ "group", tweak_group, NULL, NULL, NULL,
-		"The unprivileged group to run as.",
-		MUST_RESTART | ONLY_ROOT,
-		"" },
-	{ "group_cc", tweak_group_cc, NULL, NULL, NULL,
-		"On some systems the C-compiler is restricted so not"
-		" everybody can run it.  This parameter makes it possible"
-		" to add an extra group to the sandbox process which runs the"
-		" cc_command, in order to gain access to such a restricted"
-		" C-compiler.",
-		ONLY_ROOT,
-		"" },
-	{ NULL, NULL, NULL }
-};
-
-/*--------------------------------------------------------------------*/
-
-static void __match_proto__(mgt_sandbox_f)
-mgt_sandbox_null(enum sandbox_e who)
-{
-	(void)who;
-}
-
-/*--------------------------------------------------------------------*/
-
-#ifndef HAVE_SETPPRIV
-static void __match_proto__(mgt_sandbox_f)
-mgt_sandbox_unix(enum sandbox_e who)
-{
-#define NGID 2000
-	int i;
-	gid_t gid, gid_list[NGID];
-	uid_t uid;
-
-	if (who == SANDBOX_TESTING) {
-		/*
-		 * Test if sandboxing is going to work.
-		 * Do not assert on failure here, but simply exit non-zero.
-		 */
-		gid = getgid();
-		gid += 1;
-		if (setgid(gid))
-			exit(1);
-		uid = getuid();
-		uid += 1;
-		if (setuid(uid))
-			exit(2);
-		exit(0);
-	}
-
-	/*
-	 * Do the real thing, assert if we fail
-	 */
-
-	AZ(setgid(mgt_param.gid));
-	AZ(initgroups(mgt_param.user, mgt_param.gid));
-
-	if (who == SANDBOX_CC && strlen(mgt_param.group_cc) > 0) {
-		/* Add the optional extra group for the C-compiler access */
-		i = getgroups(NGID, gid_list);
-		assert(i >= 0);
-		gid_list[i++] = mgt_param.gid_cc;
-		AZ(setgroups(i, gid_list));
-	}
-
-	AZ(setuid(mgt_param.uid));
-
-#ifdef __linux__
-	/*
-	 * On linux mucking about with uid/gid disables core-dumps,
-	 * reenable them again.
-	 */
-	if (prctl(PR_SET_DUMPABLE, 1) != 0) {
-		MGT_complain(C_INFO,
-		    "Could not set dumpable bit.  Core dumps turned off\n");
-	}
-#endif
-}
-#endif
-
-/*--------------------------------------------------------------------*/
-
-static void __match_proto__(vsub_func_f)
-run_sandbox_test(void *priv)
-{
-
-	(void)priv;
-	mgt_sandbox(SANDBOX_TESTING);
-}
-
-/*--------------------------------------------------------------------*/
-
-void
-mgt_sandbox_init(void)
-{
-	struct passwd *pwd;
-	struct group *grp;
-	struct vsb *sb;
-	unsigned subs;
-
-	/* Pick a sandbox */
-
-#ifdef HAVE_SETPPRIV
-	mgt_sandbox = mgt_sandbox_solaris;
-#else
-	mgt_sandbox = mgt_sandbox_unix;
-#endif
-
-	/* Test it */
-
-	sb = VSB_new_auto();
-	subs = VSUB_run(sb, run_sandbox_test, NULL, "SANDBOX-test", 10);
-	VSB_delete(sb);
-	if (subs) {
-		MGT_complain(C_SECURITY,
-		    "Platform-specific sandbox failed - sandboxing disabled");
-		MGT_complain(C_SECURITY,
-		    "Varnish runs with elevated privileges");
-		mgt_sandbox = mgt_sandbox_null;
-	}
-
-	MCF_AddParams(mgt_parspec_sandbox);
-
-	/*
-	 * If we have nobody/nogroup, use them as defaults for sandboxes,
-	 * else fall back to whoever we run as.
-	 */
-	if (getpwnam("nobody") != NULL) {
-		MCF_SetDefault("user", "nobody");
-	} else {
-		pwd = getpwuid(getuid());
-		if (pwd == NULL)
-			ARGV_ERR("Neither user 'nobody' or my uid (%jd)"
-			    " found in password database.\n",
-			    (intmax_t)getuid());
-		MCF_SetDefault("user", pwd->pw_name);
-	}
-	endpwent();
-
-	if (getgrnam("nogroup") != NULL) {
-		MCF_SetDefault("group", "nogroup");
-	} else {
-		grp = getgrgid(getgid());
-		if (grp == NULL)
-			ARGV_ERR("Neither group 'nogroup' or my gid (%jd)"
-			    " found in password database.\n",
-			    (intmax_t)getgid());
-		MCF_SetDefault("group", grp->gr_name);
-	}
-	endgrent();
-}
-
-#endif



From phk at FreeBSD.org  Wed Jan  6 09:21:38 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Wed, 06 Jan 2016 10:21:38 +0100
Subject: [master] 2074927 Add a hack so we always get the VSL logged on
 varnish -wait
Message-ID: <E1aGkHi-0005iB-Tg@project.varnish-software.com>


commit 207492768439398e2adef0f63d13a783cb74d74c
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Jan 6 09:21:18 2016 +0000

    Add a hack so we always get the VSL logged on varnish -wait

diff --git a/bin/varnishtest/vtc_varnish.c b/bin/varnishtest/vtc_varnish.c
index 292be63..16d5d6c 100644
--- a/bin/varnishtest/vtc_varnish.c
+++ b/bin/varnishtest/vtc_varnish.c
@@ -77,6 +77,8 @@ struct varnish {
 	struct VSM_data		*vd;		/* vsc use */
 
 	unsigned		vsl_tag_count[256];
+	
+	volatile int		vsl_idle;
 };
 
 #define NONSENSE	"%XJEIFLH|)Xspa8P"
@@ -223,8 +225,9 @@ varnishlog_thread(void *priv)
 
 		i = VSL_Next(c);
 		if (i == 0) {
+			v->vsl_idle++;
 			/* Nothing to do but wait */
-			VTIM_sleep(0.01);
+			VTIM_sleep(0.1);
 			continue;
 		} else if (i == -2) {
 			/* Abandoned - try reconnect */
@@ -235,6 +238,8 @@ varnishlog_thread(void *priv)
 		} else if (i != 1)
 			break;
 
+		v->vsl_idle = 0;
+
 		tag = VSL_TAG(c->rec.ptr);
 		vxid = VSL_ID(c->rec.ptr);
 		if (tag == SLT__Batch)
@@ -595,9 +600,11 @@ varnish_wait(struct varnish *v)
 
 	if (v->cli_fd < 0)
 		return;
+	varnish_ask_cli(v, "backend.list", &resp);
+	while (v->vsl_idle < 10)
+		(void)usleep(200000);
 	if (vtc_error)
 		(void)sleep(1);	/* give panic messages a chance */
-	varnish_ask_cli(v, "backend.list", &resp);
 	varnish_stop(v);
 	vtc_log(v->vl, 2, "Wait");
 	AZ(close(v->cli_fd));



From phk at FreeBSD.org  Wed Jan  6 11:43:22 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Wed, 06 Jan 2016 12:43:22 +0100
Subject: [master] 68768d9 Ignore SLT_CLI when waiting for vsl
Message-ID: <E1aGmUs-00067k-9U@project.varnish-software.com>


commit 68768d9744b18337d82c46918d6bad22e40ce5d3
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Jan 6 11:43:03 2016 +0000

    Ignore SLT_CLI when waiting for vsl

diff --git a/bin/varnishtest/vtc_varnish.c b/bin/varnishtest/vtc_varnish.c
index 16d5d6c..500f0ae 100644
--- a/bin/varnishtest/vtc_varnish.c
+++ b/bin/varnishtest/vtc_varnish.c
@@ -242,6 +242,8 @@ varnishlog_thread(void *priv)
 
 		tag = VSL_TAG(c->rec.ptr);
 		vxid = VSL_ID(c->rec.ptr);
+		if (tag != SLT_CLI)
+			v->vsl_idle = 0;
 		if (tag == SLT__Batch)
 			continue;
 		tagname = VSL_tags[tag];



From phk at FreeBSD.org  Wed Jan  6 13:51:54 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Wed, 06 Jan 2016 14:51:54 +0100
Subject: [master] b695053 Save a tons of sysconf() syscalls.
Message-ID: <E1aGoVG-0004uL-9f@project.varnish-software.com>


commit b6950530c17bdacfce5372c2a37ab14c492710c0
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Jan 6 13:27:06 2016 +0000

    Save a tons of sysconf() syscalls.

diff --git a/lib/libvarnish/vsub.c b/lib/libvarnish/vsub.c
index 4e3067a..f7b6a07 100644
--- a/lib/libvarnish/vsub.c
+++ b/lib/libvarnish/vsub.c
@@ -70,7 +70,7 @@ unsigned
 VSUB_run(struct vsb *sb, vsub_func_f *func, void *priv, const char *name,
     int maxlines)
 {
-	int rv, p[2], sfd, status;
+	int rv, p[2], sfd, hfd, status;
 	pid_t pid;
 	struct vlu *vlu;
 	struct vsub_priv sp;
@@ -100,7 +100,8 @@ VSUB_run(struct vsb *sb, vsub_func_f *func, void *priv, const char *name,
 		assert(dup2(p[1], STDOUT_FILENO) == STDOUT_FILENO);
 		assert(dup2(p[1], STDERR_FILENO) == STDERR_FILENO);
 		/* Close all other fds */
-		for (sfd = STDERR_FILENO+1; sfd < sysconf(_SC_OPEN_MAX); sfd++)
+		hfd = sysconf(_SC_OPEN_MAX);
+		for (sfd = STDERR_FILENO+1; sfd < hfd; sfd++)
 			(void)close(sfd);
 		func(priv);
 		/*



From phk at FreeBSD.org  Wed Jan  6 13:51:54 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Wed, 06 Jan 2016 14:51:54 +0100
Subject: [master] 4d5dd87 Don't use getdtablesize() for closing existing fd's
 it only represents the current limit.
Message-ID: <E1aGoVG-0004uR-Ba@project.varnish-software.com>


commit 4d5dd87b0d2e17d647b7ebe81e96164b828244b1
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Jan 6 13:32:47 2016 +0000

    Don't use getdtablesize() for closing existing fd's it only
    represents the current limit.
    
    Also don't call it every iteration of the loop.

diff --git a/bin/varnishd/mgt/mgt_main.c b/bin/varnishd/mgt/mgt_main.c
index 1073b8e..25c7ce4 100644
--- a/bin/varnishd/mgt/mgt_main.c
+++ b/bin/varnishd/mgt/mgt_main.c
@@ -530,7 +530,7 @@ main(int argc, char * const *argv)
 	 * Start out by closing all unwanted file descriptors we might
 	 * have inherited from sloppy process control daemons.
 	 */
-	for (o = getdtablesize(); o > STDERR_FILENO; o--)
+	for (o = sysconf(_SC_OPEN_MAX); o > STDERR_FILENO; o--)
 		(void)close(o);
 
 	VRND_Seed();
diff --git a/bin/varnishtest/vtc_process.c b/bin/varnishtest/vtc_process.c
index c0a15c2..8770b5f 100644
--- a/bin/varnishtest/vtc_process.c
+++ b/bin/varnishtest/vtc_process.c
@@ -203,7 +203,7 @@ process_start(struct process *p)
 		assert(dup2(p->fds[0], 0) == 0);
 		assert(dup2(out_fd, 1) == 1);
 		assert(dup2(err_fd, 2) == 2);
-		for (i = 3; i <getdtablesize(); i++)
+		for (i = sysconf(_SC_OPEN_MAX); i > STDERR_FILENO; i--)
 			(void)close(i);
 		AZ(execl("/bin/sh", "/bin/sh", "-c", VSB_data(cl), (char*)0));
 		exit(1);



From phk at FreeBSD.org  Wed Jan  6 13:51:54 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Wed, 06 Jan 2016 14:51:54 +0100
Subject: [master] 9b80c2e Add VSUB_closefrom() to close all fd's higher than
 some number.
Message-ID: <E1aGoVG-0004ua-DD@project.varnish-software.com>


commit 9b80c2e10d4f05d5051de6b7314e725809e84e96
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Jan 6 13:48:04 2016 +0000

    Add VSUB_closefrom() to close all fd's higher than some number.
    
    Check if system has closefrom(2) and use that where available

diff --git a/bin/varnishd/mgt/mgt_main.c b/bin/varnishd/mgt/mgt_main.c
index 25c7ce4..c54ac60 100644
--- a/bin/varnishd/mgt/mgt_main.c
+++ b/bin/varnishd/mgt/mgt_main.c
@@ -57,6 +57,7 @@
 #include "vpf.h"
 #include "vrnd.h"
 #include "vsha256.h"
+#include "vsub.h"
 #include "vtim.h"
 #include "waiter/mgt_waiter.h"
 
@@ -530,8 +531,7 @@ main(int argc, char * const *argv)
 	 * Start out by closing all unwanted file descriptors we might
 	 * have inherited from sloppy process control daemons.
 	 */
-	for (o = sysconf(_SC_OPEN_MAX); o > STDERR_FILENO; o--)
-		(void)close(o);
+	VSUB_closefrom(STDERR_FILENO + 1);
 
 	VRND_Seed();
 
diff --git a/bin/varnishd/mgt/mgt_vcl.c b/bin/varnishd/mgt/mgt_vcl.c
index 37359f2..dec0ec0 100644
--- a/bin/varnishd/mgt/mgt_vcl.c
+++ b/bin/varnishd/mgt/mgt_vcl.c
@@ -93,6 +93,7 @@ mgt_vcl_del(struct vclprog *vp)
 	VTAILQ_REMOVE(&vclhead, vp, list);
 	XXXAZ(unlink(vp->fname));
 	bprintf(dn, "vcl_%s", vp->name);
+	WRONG("TEST");
 	VJ_master(JAIL_MASTER_FILE);
 	(void)rmdir(dn);		// compiler droppings, eg gcov
 	VJ_master(JAIL_MASTER_LOW);
diff --git a/configure.ac b/configure.ac
index f89977b..da9f7ca 100644
--- a/configure.ac
+++ b/configure.ac
@@ -245,6 +245,7 @@ AC_CHECK_FUNCS([getdtablesize])
 AC_CHECK_FUNCS([nanosleep])
 AC_CHECK_FUNCS([setppriv])
 AC_CHECK_FUNCS([fallocate])
+AC_CHECK_FUNCS([closefrom])
 
 save_LIBS="${LIBS}"
 LIBS="${PTHREAD_LIBS}"
diff --git a/include/vsub.h b/include/vsub.h
index bc337a1..de62908 100644
--- a/include/vsub.h
+++ b/include/vsub.h
@@ -33,3 +33,5 @@ typedef void vsub_func_f(void*);
 
 unsigned VSUB_run(struct vsb *, vsub_func_f *, void *priv, const char *name,
     int maxlines);
+
+void VSUB_closefrom(int fd);
diff --git a/lib/libvarnish/vsub.c b/lib/libvarnish/vsub.c
index f7b6a07..c87d297 100644
--- a/lib/libvarnish/vsub.c
+++ b/lib/libvarnish/vsub.c
@@ -52,6 +52,18 @@ struct vsub_priv {
 	int		maxlines;
 };
 
+void
+VSUB_closefrom(int fd)
+{
+#ifdef HAVE_CLOSEFROM
+	closefrom(fd);
+#else
+	int i;= sysconf(_SC_OPEN_MAX);
+	for (i = sysconf(_SC_OPEN_MAX); i > STDERR_FILENO; i--)
+		(void)close(i);
+#endif
+}
+
 static int
 vsub_vlu(void *priv, const char *str)
 {
@@ -70,7 +82,7 @@ unsigned
 VSUB_run(struct vsb *sb, vsub_func_f *func, void *priv, const char *name,
     int maxlines)
 {
-	int rv, p[2], sfd, hfd, status;
+	int rv, p[2], status;
 	pid_t pid;
 	struct vlu *vlu;
 	struct vsub_priv sp;
@@ -100,9 +112,7 @@ VSUB_run(struct vsb *sb, vsub_func_f *func, void *priv, const char *name,
 		assert(dup2(p[1], STDOUT_FILENO) == STDOUT_FILENO);
 		assert(dup2(p[1], STDERR_FILENO) == STDERR_FILENO);
 		/* Close all other fds */
-		hfd = sysconf(_SC_OPEN_MAX);
-		for (sfd = STDERR_FILENO+1; sfd < hfd; sfd++)
-			(void)close(sfd);
+		VSUB_closefrom(STDERR_FILENO + 1);
 		func(priv);
 		/*
 		 * func should either exec or exit, so getting here should be



From phk at FreeBSD.org  Wed Jan  6 13:51:54 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Wed, 06 Jan 2016 14:51:54 +0100
Subject: [master] 97a02d2 Use VSUB_closefrom()
Message-ID: <E1aGoVG-0004uj-Et@project.varnish-software.com>


commit 97a02d2e60f4238ed177ff9cb416804a3b920bab
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Jan 6 13:48:58 2016 +0000

    Use VSUB_closefrom()

diff --git a/bin/varnishtest/vtc_process.c b/bin/varnishtest/vtc_process.c
index 8770b5f..f63c2e5 100644
--- a/bin/varnishtest/vtc_process.c
+++ b/bin/varnishtest/vtc_process.c
@@ -42,6 +42,7 @@
 #include <unistd.h>
 
 #include "vtc.h"
+#include "vsub.h"
 
 struct process {
 	unsigned		magic;
@@ -183,7 +184,7 @@ static void
 process_start(struct process *p)
 {
 	struct vsb *cl;
-	int i, out_fd, err_fd;
+	int out_fd, err_fd;
 
 	CHECK_OBJ_NOTNULL(p, PROCESS_MAGIC);
 
@@ -203,8 +204,7 @@ process_start(struct process *p)
 		assert(dup2(p->fds[0], 0) == 0);
 		assert(dup2(out_fd, 1) == 1);
 		assert(dup2(err_fd, 2) == 2);
-		for (i = sysconf(_SC_OPEN_MAX); i > STDERR_FILENO; i--)
-			(void)close(i);
+		VSUB_closefrom(STDERR_FILENO + 1);
 		AZ(execl("/bin/sh", "/bin/sh", "-c", VSB_data(cl), (char*)0));
 		exit(1);
 	}



From phk at FreeBSD.org  Wed Jan  6 13:51:54 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Wed, 06 Jan 2016 14:51:54 +0100
Subject: [master] f43e32d Remove debugging hack which slipped into previous
 commit.
Message-ID: <E1aGoVG-0004us-GQ@project.varnish-software.com>


commit f43e32d8279605668419810589d8f91dfaf7f149
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Jan 6 13:50:24 2016 +0000

    Remove debugging hack which slipped into previous commit.

diff --git a/bin/varnishd/mgt/mgt_vcl.c b/bin/varnishd/mgt/mgt_vcl.c
index dec0ec0..37359f2 100644
--- a/bin/varnishd/mgt/mgt_vcl.c
+++ b/bin/varnishd/mgt/mgt_vcl.c
@@ -93,7 +93,6 @@ mgt_vcl_del(struct vclprog *vp)
 	VTAILQ_REMOVE(&vclhead, vp, list);
 	XXXAZ(unlink(vp->fname));
 	bprintf(dn, "vcl_%s", vp->name);
-	WRONG("TEST");
 	VJ_master(JAIL_MASTER_FILE);
 	(void)rmdir(dn);		// compiler droppings, eg gcov
 	VJ_master(JAIL_MASTER_LOW);



From phk at FreeBSD.org  Wed Jan  6 13:51:54 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Wed, 06 Jan 2016 14:51:54 +0100
Subject: [master] 7340726 Also use VSUB_closefrom() here.
Message-ID: <E1aGoVG-0004v1-Hr@project.varnish-software.com>


commit 7340726504ef92a6fcc415c8d8830708f0152acb
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Jan 6 13:51:39 2016 +0000

    Also use VSUB_closefrom() here.

diff --git a/bin/varnishtest/vtc_varnish.c b/bin/varnishtest/vtc_varnish.c
index 500f0ae..77d8715 100644
--- a/bin/varnishtest/vtc_varnish.c
+++ b/bin/varnishtest/vtc_varnish.c
@@ -49,6 +49,7 @@
 #include "vapi/vsm.h"
 #include "vcli.h"
 #include "vss.h"
+#include "vsub.h"
 #include "vtcp.h"
 #include "vtim.h"
 
@@ -434,8 +435,7 @@ varnish_launch(struct varnish *v)
 		AZ(close(v->fds[1]));
 		AZ(close(v->fds[2]));
 		AZ(close(v->fds[3]));
-		for (i = 3; i <getdtablesize(); i++)
-			(void)close(i);
+		VSUB_closefrom(STDERR_FILENO + 1);
 		AZ(execl("/bin/sh", "/bin/sh", "-c", VSB_data(vsb), (char*)0));
 		exit(1);
 	} else {



From fgsch at lodoss.net  Wed Jan  6 14:14:56 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Wed, 06 Jan 2016 15:14:56 +0100
Subject: [master] fe7f054 Fix build if closefrom is not available
Message-ID: <E1aGorY-00008w-FM@project.varnish-software.com>


commit fe7f0547db771ba736f702c8a96376ad02d0946d
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Wed Jan 6 14:14:33 2016 +0000

    Fix build if closefrom is not available

diff --git a/lib/libvarnish/vsub.c b/lib/libvarnish/vsub.c
index c87d297..b669fff 100644
--- a/lib/libvarnish/vsub.c
+++ b/lib/libvarnish/vsub.c
@@ -58,8 +58,8 @@ VSUB_closefrom(int fd)
 #ifdef HAVE_CLOSEFROM
 	closefrom(fd);
 #else
-	int i;= sysconf(_SC_OPEN_MAX);
-	for (i = sysconf(_SC_OPEN_MAX); i > STDERR_FILENO; i--)
+	int i = sysconf(_SC_OPEN_MAX);
+	for (i = sysconf(_SC_OPEN_MAX); i > fd; i--)
 		(void)close(i);
 #endif
 }



From martin at varnish-software.com  Wed Jan  6 16:16:11 2016
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Wed, 06 Jan 2016 17:16:11 +0100
Subject: [master] eb4106e Fix r01684.vtc logexpect handling
Message-ID: <E1aGqkt-0006t5-Pi@project.varnish-software.com>


commit eb4106e385ae0116cea9bd8367b5c1b64e5c80ca
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Wed Jan 6 17:14:38 2016 +0100

    Fix r01684.vtc logexpect handling
    
    The script lacked a logexpect -wait at the end telling it to actually
    wait for the required log content.
    
    Added a new rule to match first on an actual retry backend transaction
    and match the header set/unset/set tests on that vxid.

diff --git a/bin/varnishtest/tests/r01684.vtc b/bin/varnishtest/tests/r01684.vtc
index f4eb71e..9c2f01f 100644
--- a/bin/varnishtest/tests/r01684.vtc
+++ b/bin/varnishtest/tests/r01684.vtc
@@ -21,10 +21,11 @@ varnish v1 -vcl+backend {
 } -start
 
 # check log for the aborted POST
-logexpect l1 -v v1 {
-	expect * =	BereqHeader "^X-Varnish:"
+logexpect l1 -v v1 -g request {
+	expect * *	Begin		"^bereq .* retry"
+	expect * =	BereqHeader	"^X-Varnish:"
 	expect * =	BereqUnset	"^X-Varnish:"
-	expect * =	BereqHeader	"^X-Varnish"
+	expect * =	BereqHeader	"^X-Varnish:"
 } -start
 
 varnish v1 -cliok "param.set debug +syncvsl"
@@ -35,3 +36,5 @@ client c1 {
 	rxresp
 	expect resp.http.foo == 3
 } -run
+
+logexpect l1 -wait



From phk at FreeBSD.org  Wed Jan  6 16:39:05 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Wed, 06 Jan 2016 17:39:05 +0100
Subject: [master] d69de94 Another use for VSUB_closefrom()
Message-ID: <E1aGr73-0001xo-JO@project.varnish-software.com>


commit d69de94aeac90c4c28dd22e3d97efaf61ac44e4a
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Jan 6 14:01:19 2016 +0000

    Another use for VSUB_closefrom()

diff --git a/bin/varnishtest/vtc_main.c b/bin/varnishtest/vtc_main.c
index 9c9f255..601f245 100644
--- a/bin/varnishtest/vtc_main.c
+++ b/bin/varnishtest/vtc_main.c
@@ -51,6 +51,7 @@
 #include "vrnd.h"
 #include "vsa.h"
 #include "vss.h"
+#include "vsub.h"
 #include "vtcp.h"
 #include "vtim.h"
 
@@ -232,7 +233,7 @@ static void
 start_test(void)
 {
 	struct vtc_tst *tp;
-	int p[2], sfd, retval;
+	int p[2], retval;
 	struct vtc_job *jp;
 	char tmpdir[PATH_MAX];
 
@@ -274,8 +275,7 @@ start_test(void)
 		assert(open("/dev/null", O_RDONLY) == STDIN_FILENO);
 		assert(dup2(p[1], STDOUT_FILENO) == STDOUT_FILENO);
 		assert(dup2(p[1], STDERR_FILENO) == STDERR_FILENO);
-		for (sfd = STDERR_FILENO + 1; sfd < 100; sfd++)
-			(void)close(sfd);
+		VSUB_closefrom(STDERR_FILENO + 1);
 		retval = exec_file(jp->tst->filename, jp->tst->script,
 		    jp->tmpdir, jp->buf, jp->bufsiz);
 		exit(retval);



From phk at FreeBSD.org  Wed Jan  6 16:39:05 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Wed, 06 Jan 2016 17:39:05 +0100
Subject: [master] 62fad03 Fix typo spotted by fgs
Message-ID: <E1aGr73-0001xu-Kr@project.varnish-software.com>


commit 62fad03f1b3611dcd14a058857f25e935f9fb2c1
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Jan 6 14:34:30 2016 +0000

    Fix typo spotted by fgs

diff --git a/lib/libvarnish/vsub.c b/lib/libvarnish/vsub.c
index b669fff..d774e53 100644
--- a/lib/libvarnish/vsub.c
+++ b/lib/libvarnish/vsub.c
@@ -58,7 +58,7 @@ VSUB_closefrom(int fd)
 #ifdef HAVE_CLOSEFROM
 	closefrom(fd);
 #else
-	int i = sysconf(_SC_OPEN_MAX);
+	int i;
 	for (i = sysconf(_SC_OPEN_MAX); i > fd; i--)
 		(void)close(i);
 #endif



From phk at FreeBSD.org  Wed Jan  6 16:39:05 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Wed, 06 Jan 2016 17:39:05 +0100
Subject: [master] 67399b6 Don't close stdout/stderr when we stop due to EOF on
 stdin in -d mode.
Message-ID: <E1aGr73-0001y6-Mc@project.varnish-software.com>


commit 67399b63c6bf14a8fb3a9791d0fc2a3cd02707da
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Jan 6 16:22:44 2016 +0000

    Don't close stdout/stderr when we stop due to EOF on stdin in -d mode.
    
    That obscured problems in cleanup

diff --git a/bin/varnishd/mgt/mgt_main.c b/bin/varnishd/mgt/mgt_main.c
index c54ac60..a7533e9 100644
--- a/bin/varnishd/mgt/mgt_main.c
+++ b/bin/varnishd/mgt/mgt_main.c
@@ -355,12 +355,6 @@ cli_stdin_close(void *priv)
 {
 
 	(void)priv;
-	(void)close(0);
-	(void)close(1);
-	(void)close(2);
-	AZ(open("/dev/null", O_RDONLY));
-	assert(open("/dev/null", O_WRONLY) == 1);
-	assert(open("/dev/null", O_WRONLY) == 2);
 
 	if (d_flag) {
 		mgt_stop_child();
@@ -368,6 +362,13 @@ cli_stdin_close(void *priv)
 		if (pfh != NULL)
 			(void)VPF_Remove(pfh);
 		exit(0);
+	} else {
+		(void)close(0);
+		(void)close(1);
+		(void)close(2);
+		AZ(open("/dev/null", O_RDONLY));
+		assert(open("/dev/null", O_WRONLY) == 1);
+		assert(open("/dev/null", O_WRONLY) == 2);
 	}
 }
 



From phk at FreeBSD.org  Wed Jan  6 16:39:05 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Wed, 06 Jan 2016 17:39:05 +0100
Subject: [master] b4545b1 This seems to get test-termination to stabilize,
 including the new improved probability of getting the VSL out.
Message-ID: <E1aGr73-0001yC-OB@project.varnish-software.com>


commit b4545b106a906aa1261ebd4975d957f31e64c806
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Jan 6 16:25:13 2016 +0000

    This seems to get test-termination to stabilize, including the
    new improved probability of getting the VSL out.

diff --git a/bin/varnishtest/tests/b00044.vtc b/bin/varnishtest/tests/b00044.vtc
index 5d02319..463e383 100644
--- a/bin/varnishtest/tests/b00044.vtc
+++ b/bin/varnishtest/tests/b00044.vtc
@@ -15,3 +15,5 @@ client c1 {
 server s1 -wait
 
 shell "kill -15 ${v1_pid}"
+
+varnish v1 -cleanup
diff --git a/bin/varnishtest/vtc.c b/bin/varnishtest/vtc.c
index 3c6c884..33b56e1 100644
--- a/bin/varnishtest/vtc.c
+++ b/bin/varnishtest/vtc.c
@@ -684,7 +684,7 @@ exec_file(const char *fn, const char *script, const char *tmpdir,
 	vtc_stop = 1;
 	vtc_log(vltop, 1, "RESETTING after %s", fn);
 	reset_cmds(cmds);
-	vtc_error = old_err;
+	vtc_error |= old_err;
 
 	if (vtc_error)
 		vtc_log(vltop, 1, "TEST %s FAILED", fn);
diff --git a/bin/varnishtest/vtc_log.c b/bin/varnishtest/vtc_log.c
index 4a73ebd..30ae6b8 100644
--- a/bin/varnishtest/vtc_log.c
+++ b/bin/varnishtest/vtc_log.c
@@ -112,8 +112,6 @@ vtc_log_emit(const struct vtclog *vl, int lvl)
 
 	if (lvl < 0)
 		lvl = 0;
-	if (vtc_stop && lvl == 0)
-		return;
 	l = VSB_len(vl->vsb);
 	AZ(pthread_mutex_lock(&vtclog_mtx));
 	assert(vtclog_left > l);
diff --git a/bin/varnishtest/vtc_varnish.c b/bin/varnishtest/vtc_varnish.c
index 77d8715..91c4072 100644
--- a/bin/varnishtest/vtc_varnish.c
+++ b/bin/varnishtest/vtc_varnish.c
@@ -78,7 +78,7 @@ struct varnish {
 	struct VSM_data		*vd;		/* vsc use */
 
 	unsigned		vsl_tag_count[256];
-	
+
 	volatile int		vsl_idle;
 };
 
@@ -209,6 +209,7 @@ varnishlog_thread(void *priv)
 	opt = 0;
 	while (v->pid) {
 		if (c == NULL) {
+			v->vsl_idle++;
 			VTIM_sleep(0.1);
 			if (VSM_Open(vsm)) {
 				VSM_ResetError(vsm);
@@ -257,6 +258,8 @@ varnishlog_thread(void *priv)
 		    type, (int)len, data);
 	}
 
+	v->vsl_idle = 100;
+
 	if (c)
 		VSL_DeleteCursor(c);
 	VSL_Delete(vsl);
@@ -343,28 +346,29 @@ varnish_thread(void *priv)
 {
 	struct varnish *v;
 	char buf[65536];
-	struct pollfd *fds, fd;
+	struct pollfd fds[1];
 	int i;
 
 	CAST_OBJ_NOTNULL(v, priv, VARNISH_MAGIC);
 	(void)VTCP_nonblocking(v->fds[0]);
 	while (1) {
-		fds = &fd;
-		memset(fds, 0, sizeof *fds);
+		memset(fds, 0, sizeof fds);
 		fds->fd = v->fds[0];
 		fds->events = POLLIN;
 		i = poll(fds, 1, 1000);
 		if (i == 0)
 			continue;
+		if (fds->revents & POLLIN) {
+			i = read(v->fds[0], buf, sizeof buf - 1);
+			if (i > 0) {
+				buf[i] = '\0';
+				vtc_dump(v->vl, 3, "debug", buf, -2);
+			}
+		}
 		if (fds->revents & (POLLERR|POLLHUP)) {
 			vtc_log(v->vl, 4, "STDOUT poll 0x%x", fds->revents);
 			break;
 		}
-		i = read(v->fds[0], buf, sizeof buf - 1);
-		if (i <= 0)
-			break;
-		buf[i] = '\0';
-		vtc_dump(v->vl, 3, "debug", buf, -2);
 	}
 	return (NULL);
 }
@@ -589,41 +593,42 @@ varnish_stop(struct varnish *v)
 }
 
 /**********************************************************************
- * Wait for a Varnish
+ * Cleanup
  */
 
 static void
-varnish_wait(struct varnish *v)
+varnish_cleanup(struct varnish *v)
 {
 	void *p;
 	int status, r;
 	struct rusage ru;
-	char *resp;
 
-	if (v->cli_fd < 0)
-		return;
-	varnish_ask_cli(v, "backend.list", &resp);
+	/* Give the VSL log time to finish */
 	while (v->vsl_idle < 10)
 		(void)usleep(200000);
-	if (vtc_error)
-		(void)sleep(1);	/* give panic messages a chance */
-	varnish_stop(v);
-	vtc_log(v->vl, 2, "Wait");
+
+	/* Close the CLI connection */
 	AZ(close(v->cli_fd));
 	v->cli_fd = -1;
 
-	(void)close(v->fds[1]);		/* May already have been closed */
+	/* Close the STDIN connection. */
+	AZ(close(v->fds[1]));
 
+	/* Wait until STDOUT+STDERR closes */
 	AZ(pthread_join(v->tp, &p));
 	AZ(close(v->fds[0]));
+
 	r = wait4(v->pid, &status, 0, &ru);
 	v->pid = 0;
 	vtc_log(v->vl, 2, "R %d Status: %04x (u %.6f s %.6f)", r, status,
 	    ru.ru_utime.tv_sec + 1e-6 * ru.ru_utime.tv_usec,
 	    ru.ru_stime.tv_sec + 1e-6 * ru.ru_stime.tv_usec
 	);
+
+	/* Pick up the VSL thread */
 	AZ(pthread_join(v->tp_vsl, &p));
-	if (WIFEXITED(status) && WEXITSTATUS(status) == 0)
+
+	if (WIFEXITED(status) && (WEXITSTATUS(status) == 0))
 		return;
 #ifdef WCOREDUMP
 	vtc_log(v->vl, 0, "Bad exit code: %04x sig %x exit %x core %x",
@@ -636,6 +641,30 @@ varnish_wait(struct varnish *v)
 }
 
 /**********************************************************************
+ * Wait for a Varnish
+ */
+
+static void
+varnish_wait(struct varnish *v)
+{
+	char *resp;
+
+	if (v->cli_fd < 0)
+		return;
+
+	vtc_log(v->vl, 2, "Wait");
+
+	/* Do a backend.list to log if child is still running */
+	varnish_ask_cli(v, "backend.list", &resp);
+
+	/* Then stop it */
+	varnish_stop(v);
+
+	varnish_cleanup(v);
+}
+
+
+/**********************************************************************
  * Ask a CLI question
  */
 
@@ -930,6 +959,11 @@ cmd_varnish(CMD_ARGS)
 			av += 2;
 			continue;
 		}
+		if (!strcmp(*av, "-cleanup")) {
+			AZ(av[1]);
+			varnish_cleanup(v);
+			continue;
+		}
 		if (!strcmp(*av, "-cliok")) {
 			AN(av[1]);
 			varnish_cli(v, av[1], (unsigned)CLIS_OK);



From fgsch at lodoss.net  Wed Jan  6 16:51:11 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Wed, 06 Jan 2016 17:51:11 +0100
Subject: [master] e632147 SmartOS needs stdlib.h for closefrom()
Message-ID: <E1aGrIl-0003Uq-9M@project.varnish-software.com>


commit e63214798cc485d1bebdc2a11a4aafba15a4924a
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Wed Jan 6 16:49:11 2016 +0000

    SmartOS needs stdlib.h for closefrom()

diff --git a/lib/libvarnish/vsub.c b/lib/libvarnish/vsub.c
index d774e53..a7c658c 100644
--- a/lib/libvarnish/vsub.c
+++ b/lib/libvarnish/vsub.c
@@ -36,6 +36,7 @@
 #include <errno.h>
 #include <fcntl.h>
 #include <stdio.h>
+#include <stdlib.h>
 #include <string.h>
 #include <unistd.h>
 



From fgsch at lodoss.net  Wed Jan  6 18:39:36 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Wed, 06 Jan 2016 19:39:36 +0100
Subject: [master] 70de08d First pass at using PARAM
Message-ID: <E1aGszg-0000YD-M5@project.varnish-software.com>


commit 70de08d2fc84a135e3971063ddd7db9572b1b289
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Wed Jan 6 18:35:17 2016 +0000

    First pass at using PARAM

diff --git a/bin/varnishd/common/params.h b/bin/varnishd/common/params.h
index 13ab904..d5ac4f5 100644
--- a/bin/varnishd/common/params.h
+++ b/bin/varnishd/common/params.h
@@ -88,18 +88,6 @@ struct params {
 	uid_t			uid;
 	gid_t			gid;
 
-	/* TTL used for lack of anything better */
-	double			default_ttl;
-
-	/* Default grace period */
-	double			default_grace;
-
-	/* Default keep period */
-	double			default_keep;
-
-	/* Maximum concurrent sessions */
-	unsigned		max_sess;
-
 	/* Worker threads and pool */
 	unsigned		wthread_min;
 	unsigned		wthread_max;
@@ -112,97 +100,14 @@ struct params {
 	ssize_t			wthread_stacksize;
 	unsigned		wthread_queue_limit;
 
-	/* Memory allocation hints */
-	unsigned		workspace_backend;
-	unsigned		workspace_client;
-	unsigned		workspace_session;
-	unsigned		workspace_thread;
-
-	unsigned		vsl_buffer;
-
-	unsigned		http_req_size;
-	unsigned		http_req_hdr_len;
-	unsigned		http_resp_size;
-	unsigned		http_resp_hdr_len;
-	unsigned		http_max_hdr;
-
-	unsigned		vsl_reclen;
-
-	double			timeout_linger;
-	double			timeout_idle;
-	double			pipe_timeout;
-	double			send_timeout;
-	double			idle_send_timeout;
 #ifdef HAVE_TCP_KEEP
 	double			tcp_keepalive_time;
 	unsigned		tcp_keepalive_probes;
 	double			tcp_keepalive_intvl;
 #endif
 
-	/* Fetcher hints */
-	ssize_t			fetch_chunksize;
-	ssize_t			fetch_maxchunksize;
-	unsigned		nuke_limit;
-
-	/* Listen depth */
-	unsigned		listen_depth;
-
-	/* CLI related */
-	double			cli_timeout;
-	unsigned		cli_limit;
-	unsigned		ping_interval;
-
-	/* LRU list ordering interval */
-	double			lru_interval;
-
-	/* Maximum restarts allowed */
-	unsigned		max_restarts;
-
-	/* Maximum backend retriesallowed */
-	unsigned		max_retries;
-
-	/* Maximum esi:include depth allowed */
-	unsigned		max_esi_depth;
-
-	/* Rush exponent */
-	unsigned		rush_exponent;
-
-	/* Default connection_timeout */
-	double			connect_timeout;
-
-	/* CLI buffer size */
-	unsigned		cli_buffer;
-
-	/* Prefer IPv6 connections to backend*/
-	unsigned		prefer_ipv6;
-
-	/* Acceptable clockskew with backends */
-	unsigned		clock_skew;
-
-	unsigned		syslog_cli_traffic;
-
-	unsigned		http_range_support;
-
-	unsigned		http_gzip_support;
-	unsigned		gzip_buffer;
-	unsigned		gzip_level;
-	unsigned		gzip_memlevel;
-
-	double			critbit_cooloff;
-
-	double			vcl_cooldown;
-
-	double			shortlived;
-
 	struct vre_limits	vre_limits;
 
-	/* Install a SIGSEGV handler */
-	unsigned		sigsegv_handler;
-
-	/* VSM dimensions */
-	ssize_t			vsm_space;
-	ssize_t			vsl_space;
-
 	struct poolparam	req_pool;
 	struct poolparam	sess_pool;
 	struct poolparam	vbo_pool;
diff --git a/bin/varnishd/mgt/mgt_param_tbl.c b/bin/varnishd/mgt/mgt_param_tbl.c
index 3a6239b..16886b2 100644
--- a/bin/varnishd/mgt/mgt_param_tbl.c
+++ b/bin/varnishd/mgt/mgt_param_tbl.c
@@ -48,231 +48,6 @@ struct parspec mgt_parspec[] = {
 #include "tbl/params.h"
 #undef PARAM
 
-	{ "default_ttl", tweak_timeout, &mgt_param.default_ttl,
-		"0", NULL,
-		"The TTL assigned to objects if neither the backend nor "
-		"the VCL code assigns one.",
-		OBJ_STICKY,
-		"120", "seconds" },
-	{ "default_grace", tweak_timeout, &mgt_param.default_grace,
-		"0", NULL,
-		"Default grace period.  We will deliver an object "
-		"this long after it has expired, provided another thread "
-		"is attempting to get a new copy.",
-		OBJ_STICKY,
-		"10", "seconds" },
-	{ "default_keep", tweak_timeout, &mgt_param.default_keep,
-		"0", NULL,
-		"Default keep period.  We will keep a useless object "
-		"around this long, making it available for conditional "
-		"backend fetches.  "
-		"That means that the object will be removed from the "
-		"cache at the end of ttl+grace+keep.",
-		OBJ_STICKY,
-		"0", "seconds" },
-	{ "workspace_session",
-		tweak_bytes_u, &mgt_param.workspace_session,
-		"256", NULL,
-		"Allocation size for session structure and workspace.  "
-		"  The workspace is primarily used for TCP connection "
-		"addresses."
-		"  If larger than 4k, use a multiple of 4k for VM efficiency.",
-		DELAYED_EFFECT,
-		"512", "bytes" },
-	{ "workspace_client",
-		tweak_bytes_u, &mgt_param.workspace_client,
-		"9k", NULL,
-		"Bytes of HTTP protocol workspace for clients HTTP req/resp."
-		"  If larger than 4k, use a multiple of 4k for VM efficiency.",
-		DELAYED_EFFECT,
-		"64k", "bytes" },
-	{ "workspace_backend",
-		tweak_bytes_u, &mgt_param.workspace_backend,
-		"1024", NULL,
-		"Bytes of HTTP protocol workspace for backend HTTP req/resp."
-		"  If larger than 4k, use a multiple of 4k for VM efficiency.",
-		DELAYED_EFFECT,
-		"64k", "bytes" },
-	{ "workspace_thread",
-		tweak_bytes_u, &mgt_param.workspace_thread,
-		"256", "8192",
-		"Bytes of auxiliary workspace per thread.\n"
-		"This workspace is used for certain temporary data structures"
-		" during the operation of a worker thread.\n"
-		"One use is for the io-vectors for writing requests and"
-		" responses to sockets, having too little space will"
-		" result in more writev(2) system calls, having too much"
-		" just wastes the space.",
-		DELAYED_EFFECT,
-		"2048", "bytes" },
-	{ "http_req_hdr_len",
-		tweak_bytes_u, &mgt_param.http_req_hdr_len,
-		"40", NULL,
-		"Maximum length of any HTTP client request header we will "
-		"allow.  The limit is inclusive its continuation lines.",
-		0,
-		"8k", "bytes" },
-	{ "http_req_size",
-		tweak_bytes_u, &mgt_param.http_req_size,
-		"256", NULL,
-		"Maximum number of bytes of HTTP client request we will deal "
-		"with.  This is a limit on all bytes up to the double blank "
-		"line which ends the HTTP request.\n"
-		"The memory for the request is allocated from the client "
-		"workspace (param: workspace_client) and this parameter limits "
-		"how much of that the request is allowed to take up.",
-		0,
-		"32k", "bytes" },
-	{ "http_resp_hdr_len",
-		tweak_bytes_u, &mgt_param.http_resp_hdr_len,
-		"40", NULL,
-		"Maximum length of any HTTP backend response header we will "
-		"allow.  The limit is inclusive its continuation lines.",
-		0,
-		"8k", "bytes" },
-	{ "http_resp_size",
-		tweak_bytes_u, &mgt_param.http_resp_size,
-		"256", NULL,
-		"Maximum number of bytes of HTTP backend response we will deal "
-		"with.  This is a limit on all bytes up to the double blank "
-		"line which ends the HTTP request.\n"
-		"The memory for the request is allocated from the worker "
-		"workspace (param: thread_pool_workspace) and this parameter "
-		"limits how much of that the request is allowed to take up.",
-		0,
-		"32k", "bytes" },
-	{ "http_max_hdr", tweak_uint, &mgt_param.http_max_hdr,
-		"32", "65535",
-		"Maximum number of HTTP header lines we allow in "
-		"{req|resp|bereq|beresp}.http "
-		"(obj.http is autosized to the exact number of headers).\n"
-		"Cheap, ~20 bytes, in terms of workspace memory.\n"
-		"Note that the first line occupies five header lines.",
-		0,
-		"64", "header lines" },
-	{ "vsl_buffer",
-		tweak_vsl_buffer, &mgt_param.vsl_buffer,
-		"1024", NULL,
-		"Bytes of (req-/backend-)workspace dedicated to buffering"
-		" VSL records.\n"
-		"Setting this too high costs memory, setting it too low"
-		" will cause more VSL flushes and likely increase"
-		" lock-contention on the VSL mutex.\n\n"
-		"The minimum tracks the vsl_reclen parameter + 12 bytes.",
-		0,
-		"4k", "bytes" },
-	{ "vsl_reclen",
-		tweak_vsl_reclen, &mgt_param.vsl_reclen,
-		"16", "65535",
-		"Maximum number of bytes in SHM log record.\n\n"
-		"The maximum tracks the vsl_buffer parameter - 12 bytes.",
-		0,
-		"255", "bytes" },
-	{ "shm_reclen",
-		tweak_vsl_reclen, &mgt_param.vsl_reclen,
-		"16", "65535",
-		"Old name for vsl_reclen, use that instead.",
-		0,
-		"255", "bytes" },
-	{ "timeout_idle", tweak_timeout, &mgt_param.timeout_idle,
-		"0", NULL,
-		"Idle timeout for client connections.\n"
-		"A connection is considered idle, until we have "
-		"received the full request headers.",
-		0,
-		"5", "seconds" },
-	{ "pipe_timeout", tweak_timeout, &mgt_param.pipe_timeout,
-		"0", NULL,
-		"Idle timeout for PIPE sessions. "
-		"If nothing have been received in either direction for "
-		"this many seconds, the session is closed.",
-		0,
-		"60", "seconds" },
-	{ "send_timeout", tweak_timeout, &mgt_param.send_timeout,
-		"0", NULL,
-		"Send timeout for client connections. "
-		"If the HTTP response hasn't been transmitted in this many\n"
-		"seconds the session is closed.\n"
-		"See setsockopt(2) under SO_SNDTIMEO for more information.",
-		DELAYED_EFFECT,
-		"600", "seconds" },
-	{ "idle_send_timeout", tweak_timeout, &mgt_param.idle_send_timeout,
-		"0", NULL,
-		"Time to wait with no data sent. "
-		"If no data has been transmitted in this many\n"
-		"seconds the session is closed.\n"
-		"See setsockopt(2) under SO_SNDTIMEO for more information.",
-		DELAYED_EFFECT,
-		"60", "seconds" },
-	{ "nuke_limit",
-		tweak_uint, &mgt_param.nuke_limit,
-		"0", NULL,
-		"Maximum number of objects we attempt to nuke in order "
-		"to make space for a object body.",
-		EXPERIMENTAL,
-		"50", "allocations" },
-	{ "fetch_chunksize",
-		tweak_bytes, &mgt_param.fetch_chunksize,
-		"4096", NULL,
-		"The default chunksize used by fetcher. "
-		"This should be bigger than the majority of objects with "
-		"short TTLs.\n"
-		"Internal limits in the storage_file module makes increases "
-		"above 128kb a dubious idea.",
-		EXPERIMENTAL,
-		"16k", "bytes" },
-	{ "fetch_maxchunksize",
-		tweak_bytes, &mgt_param.fetch_maxchunksize,
-		"65536", NULL,
-		"The maximum chunksize we attempt to allocate from storage. "
-		"Making this too large may cause delays and storage "
-		"fragmentation.",
-		EXPERIMENTAL,
-		"256m", "bytes" },
-	{ "listen_depth", tweak_uint, &mgt_param.listen_depth,
-		"0", NULL,
-		"Listen queue depth.",
-		MUST_RESTART,
-		"1024", "connections" },
-	{ "cli_buffer",
-		tweak_bytes_u, &mgt_param.cli_buffer,
-		"4096", NULL,
-		"Size of buffer for CLI command input."
-		"\nYou may need to increase this if you have big VCL files "
-		"and use the vcl.inline CLI command.\n"
-		"NB: Must be specified with -p to have effect.",
-		0,
-		"8k", "bytes" },
-	{ "cli_limit",
-		tweak_bytes_u, &mgt_param.cli_limit,
-		"128", "99999999",
-		"Maximum size of CLI response.  If the response exceeds"
-		" this limit, the response code will be 201 instead of"
-		" 200 and the last line will indicate the truncation.",
-		0,
-		"48k", "bytes" },
-	{ "cli_timeout", tweak_timeout, &mgt_param.cli_timeout,
-		"0", NULL,
-		"Timeout for the childs replies to CLI requests from "
-		"the mgt_param.",
-		0,
-		"60", "seconds" },
-	{ "ping_interval", tweak_uint, &mgt_param.ping_interval,
-		"0", NULL,
-		"Interval between pings from parent to child.\n"
-		"Zero will disable pinging entirely, which makes "
-		"it possible to attach a debugger to the child.",
-		MUST_RESTART,
-		"3", "seconds" },
-	{ "lru_interval", tweak_timeout, &mgt_param.lru_interval,
-		"0", NULL,
-		"Grace period before object moves on LRU list.\n"
-		"Objects are only moved to the front of the LRU "
-		"list if they have not been moved there already inside "
-		"this timeout period.  This reduces the amount of lock "
-		"operations necessary for LRU list access.",
-		EXPERIMENTAL,
-		"2", "seconds" },
 	{ "cc_command", tweak_string, &mgt_cc_cmd,
 		NULL, NULL,
 		"Command used for compiling the C source code to a "
@@ -281,134 +56,6 @@ struct parspec mgt_parspec[] = {
 		"and %o will be replaced with the output file name.",
 		MUST_RELOAD,
 		VCC_CC , NULL },
-	{ "max_restarts", tweak_uint, &mgt_param.max_restarts,
-		"0", NULL,
-		"Upper limit on how many times a request can restart."
-		"\nBe aware that restarts are likely to cause a hit against "
-		"the backend, so don't increase thoughtlessly.",
-		0,
-		"4", "restarts" },
-	{ "max_retries", tweak_uint, &mgt_param.max_retries,
-		"0", NULL,
-		"Upper limit on how many times a backend fetch can retry.",
-		0,
-		"4", "retries" },
-	{ "max_esi_depth", tweak_uint, &mgt_param.max_esi_depth,
-		"0", NULL,
-		"Maximum depth of esi:include processing.",
-		0,
-		"5", "levels" },
-	{ "connect_timeout", tweak_timeout, &mgt_param.connect_timeout,
-		"0", NULL,
-		"Default connection timeout for backend connections. "
-		"We only try to connect to the backend for this many "
-		"seconds before giving up. "
-		"VCL can override this default value for each backend and "
-		"backend request.",
-		0,
-		"3.5", "seconds" },
-	{ "clock_skew", tweak_uint, &mgt_param.clock_skew,
-		"0", NULL,
-		"How much clockskew we are willing to accept between the "
-		"backend and our own clock.",
-		0,
-		"10", "seconds" },
-	{ "prefer_ipv6", tweak_bool, &mgt_param.prefer_ipv6,
-		NULL, NULL,
-		"Prefer IPv6 address when connecting to backends which "
-		"have both IPv4 and IPv6 addresses.",
-		0,
-		"off", "bool" },
-	{ "session_max", tweak_uint,
-		&mgt_param.max_sess,
-		"1000", NULL,
-		"Maximum number of sessions we will allocate from one pool "
-		"before just dropping connections.\n"
-		"This is mostly an anti-DoS measure, and setting it plenty "
-		"high should not hurt, as long as you have the memory for "
-		"it.",
-		0,
-		"100000", "sessions" },
-	{ "timeout_linger", tweak_timeout, &mgt_param.timeout_linger,
-		"0", NULL,
-		"How long the worker thread lingers on an idle session "
-		"before handing it over to the waiter.\n"
-		"When sessions are reused, as much as half of all reuses "
-		"happen within the first 100 msec of the previous request "
-		"completing.\n"
-		"Setting this too high results in worker threads not doing "
-		"anything for their keep, setting it too low just means that "
-		"more sessions take a detour around the waiter.",
-		EXPERIMENTAL,
-		"0.050", "seconds" },
-	{ "syslog_cli_traffic", tweak_bool, &mgt_param.syslog_cli_traffic,
-		NULL, NULL,
-		"Log all CLI traffic to syslog(LOG_INFO).",
-		0,
-		"on", "bool" },
-	{ "http_range_support", tweak_bool, &mgt_param.http_range_support,
-		NULL, NULL,
-		"Enable support for HTTP Range headers.",
-		0,
-		"on", "bool" },
-	{ "http_gzip_support", tweak_bool, &mgt_param.http_gzip_support,
-		NULL, NULL,
-		"Enable gzip support. When enabled Varnish request compressed "
-		"objects from the backend and store them compressed. "
-		"If a client does not support gzip encoding Varnish will "
-		"uncompress compressed objects on demand. Varnish will also "
-		"rewrite the Accept-Encoding header of clients indicating "
-		"support for gzip to:\n"
-		"  Accept-Encoding: gzip\n\n"
-		"Clients that do not support gzip will have their "
-		"Accept-Encoding header removed. For more information on how "
-		"gzip is implemented please see the chapter on gzip in the "
-		"Varnish reference.",
-		0,
-		"on", "bool" },
-	{ "gzip_level", tweak_uint, &mgt_param.gzip_level,
-		"0", "9",
-		"Gzip compression level: 0=debug, 1=fast, 9=best",
-		0,
-		"6", ""},
-	{ "gzip_memlevel", tweak_uint, &mgt_param.gzip_memlevel,
-		"1", "9",
-		"Gzip memory level 1=slow/least, 9=fast/most compression.\n"
-		"Memory impact is 1=1k, 2=2k, ... 9=256k.",
-		0,
-		"8", ""},
-	{ "gzip_buffer",
-		tweak_bytes_u, &mgt_param.gzip_buffer,
-		"2048", NULL,
-		"Size of malloc buffer used for gzip processing.\n"
-		"These buffers are used for in-transit data,"
-		" for instance gunzip'ed data being sent to a client."
-		"Making this space to small results in more overhead,"
-		" writes to sockets etc, making it too big is probably"
-		" just a waste of memory.",
-		EXPERIMENTAL,
-		"32k", "bytes" },
-	{ "shortlived", tweak_timeout,
-		&mgt_param.shortlived,
-		"0", NULL,
-		"Objects created with (ttl+grace+keep) shorter than this"
-		" are always put in transient storage.",
-		0,
-		"10", "seconds" },
-	{ "critbit_cooloff", tweak_timeout,
-		&mgt_param.critbit_cooloff,
-		"60", "254",
-		"How long the critbit hasher keeps deleted objheads "
-		"on the cooloff list.",
-		WIZARD,
-		"180", "seconds" },
-	{ "sigsegv_handler", tweak_bool, &mgt_param.sigsegv_handler,
-		NULL, NULL,
-		"Install a signal handler which tries to dump debug"
-		" information on segmentation faults, bus errors and abort"
-		" signals.",
-		MUST_RESTART,
-		"on", "bool" },
 	{ "vcl_dir", tweak_string, &mgt_vcl_dir,
 		NULL, NULL,
 		"Directory from which relative VCL filenames (vcl.load and "
@@ -422,31 +69,22 @@ struct parspec mgt_parspec[] = {
 		0,
 		VARNISH_VMOD_DIR,
 		NULL },
-	{ "vcl_cooldown", tweak_timeout, &mgt_param.vcl_cooldown,
-		"0", NULL,
-		"How long a VCL is kept warm after being replaced as the"
-		" active VCL (granularity approximately 30 seconds).",
-		0,
-		"600", "seconds" },
 	{ "vcc_err_unref", tweak_bool, &mgt_vcc_err_unref,
 		NULL, NULL,
 		"Unreferenced VCL objects result in error.",
 		0,
 		"on", "bool" },
-
 	{ "vcc_allow_inline_c", tweak_bool, &mgt_vcc_allow_inline_c,
 		NULL, NULL,
 		"Allow inline C code in VCL.",
 		0,
 		"off", "bool" },
-
 	{ "vcc_unsafe_path", tweak_bool, &mgt_vcc_unsafe_path,
 		NULL, NULL,
 		"Allow '/' in vmod & include paths.\n"
 		"Allow 'import ... from ...'.",
 		0,
 		"on", "bool" },
-
 	{ "pcre_match_limit", tweak_uint,
 		&mgt_param.vre_limits.match,
 		"1", NULL,
@@ -473,28 +111,6 @@ struct parspec mgt_parspec[] = {
 		"Testcase r01576 can be useful when tuning this parameter.",
 		0,
 		"20", ""},
-	{ "vsl_space", tweak_bytes,
-		&mgt_param.vsl_space,
-		"1M", NULL,
-		"The amount of space to allocate for the VSL fifo buffer"
-		" in the VSM memory segment."
-		"  If you make this too small, varnish{ncsa|log} etc will"
-		" not be able to keep up."
-		"  Making it too large just costs memory resources.",
-		MUST_RESTART,
-		"80M", "bytes"},
-
-	{ "vsm_space", tweak_bytes,
-		&mgt_param.vsm_space,
-		"1M", NULL,
-		"The amount of space to allocate for stats counters"
-		" in the VSM memory segment."
-		"  If you make this too small, some counters will be"
-		" invisible."
-		"  Making it too large just costs memory resources.",
-		MUST_RESTART,
-		"1M", "bytes"},
-
 	{ "pool_req", tweak_poolparam, &mgt_param.req_pool,
 		NULL, NULL,
 		"Parameters for per worker pool request memory pool.\n"
diff --git a/bin/varnishd/mgt/mgt_pool.c b/bin/varnishd/mgt/mgt_pool.c
index c08a2af..d14a32f 100644
--- a/bin/varnishd/mgt/mgt_pool.c
+++ b/bin/varnishd/mgt/mgt_pool.c
@@ -191,15 +191,6 @@ struct parspec WRK_parspec[] = {
 		"be dropped instead of queued.",
 		EXPERIMENTAL,
 		"20", "" },
-	{ "rush_exponent", tweak_uint, &mgt_param.rush_exponent,
-		"2", NULL,
-		"How many parked request we start for each completed "
-		"request on the object.\n"
-		"NB: Even with the implict delay of delivery, "
-		"this parameter controls an exponential increase in "
-		"number of worker threads.",
-		EXPERIMENTAL,
-		"3", "requests per request" },
 	{ "thread_pool_stack",
 		tweak_bytes, &mgt_param.wthread_stacksize,
 		NULL, NULL,
diff --git a/include/tbl/params.h b/include/tbl/params.h
index f372351..388c1ee 100644
--- a/include/tbl/params.h
+++ b/include/tbl/params.h
@@ -226,9 +226,6 @@ PARAM(
 	/* func */	NULL
 )
 
-/**********************************************************************/
-#if 0 /* NOT YET */
-
 PARAM(
 	/* name */	cli_buffer,
 	/* typ */	bytes_u,
@@ -245,6 +242,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	cli_limit,
 	/* typ */	bytes_u,
@@ -260,6 +258,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	cli_timeout,
 	/* typ */	timeout,
@@ -274,6 +273,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	clock_skew,
 	/* typ */	uint,
@@ -288,6 +288,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	connect_timeout,
 	/* typ */	timeout,
@@ -304,6 +305,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	critbit_cooloff,
 	/* typ */	timeout,
@@ -318,6 +320,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 #if 0
 PARAM(
 	/* name */	debug,
@@ -347,6 +350,7 @@ PARAM(
 	/* func */	NULL
 )
 #endif
+
 PARAM(
 	/* name */	default_grace,
 	/* typ */	timeout,
@@ -362,6 +366,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	default_keep,
 	/* typ */	timeout,
@@ -378,6 +383,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	default_ttl,
 	/* typ */	timeout,
@@ -392,6 +398,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 #if 0
 PARAM(
 	/* name */	feature,
@@ -417,6 +424,7 @@ PARAM(
 	/* func */	NULL
 )
 #endif
+
 PARAM(
 	/* name */	fetch_chunksize,
 	/* typ */	bytes,
@@ -433,6 +441,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	fetch_maxchunksize,
 	/* typ */	bytes,
@@ -447,6 +456,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	gzip_buffer,
 	/* typ */	bytes_u,
@@ -464,6 +474,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	gzip_level,
 	/* typ */	uint,
@@ -477,6 +488,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	gzip_memlevel,
 	/* typ */	uint,
@@ -491,6 +503,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	http_gzip_support,
 	/* typ */	bool,
@@ -513,6 +526,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	http_max_hdr,
 	/* typ */	uint,
@@ -530,6 +544,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	http_range_support,
 	/* typ */	bool,
@@ -543,6 +558,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	http_req_hdr_len,
 	/* typ */	bytes_u,
@@ -557,6 +573,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	http_req_size,
 	/* typ */	bytes_u,
@@ -575,6 +592,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	http_resp_hdr_len,
 	/* typ */	bytes_u,
@@ -589,6 +607,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	http_resp_size,
 	/* typ */	bytes_u,
@@ -607,6 +626,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	idle_send_timeout,
 	/* typ */	timeout,
@@ -623,6 +643,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	listen_depth,
 	/* typ */	uint,
@@ -636,6 +657,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	lru_interval,
 	/* typ */	timeout,
@@ -653,6 +675,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	max_esi_depth,
 	/* typ */	uint,
@@ -666,6 +689,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	max_restarts,
 	/* typ */	uint,
@@ -681,6 +705,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	max_retries,
 	/* typ */	uint,
@@ -694,6 +719,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	nuke_limit,
 	/* typ */	uint,
@@ -708,6 +734,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 #if 0
 PARAM(
 	/* name */	pcre_match_limit,
@@ -723,6 +750,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	pcre_match_limit_recursion,
 	/* typ */	uint,
@@ -738,6 +766,7 @@ PARAM(
 	/* func */	NULL
 )
 #endif
+
 PARAM(
 	/* name */	ping_interval,
 	/* typ */	uint,
@@ -753,6 +782,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	pipe_timeout,
 	/* typ */	timeout,
@@ -767,6 +797,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 #if 0
 PARAM(
 	/* name */	pool_req,
@@ -782,6 +813,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	pool_sess,
 	/* typ */	poolparam,
@@ -796,6 +828,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	pool_vbo,
 	/* typ */	poolparam,
@@ -811,6 +844,7 @@ PARAM(
 	/* func */	NULL
 )
 #endif
+
 PARAM(
 	/* name */	prefer_ipv6,
 	/* typ */	bool,
@@ -825,6 +859,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	rush_exponent,
 	/* typ */	uint,
@@ -841,6 +876,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	send_timeout,
 	/* typ */	timeout,
@@ -857,13 +893,13 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
-#if 0
+
 PARAM(
 	/* name */	session_max,
 	/* typ */	uint,
 	/* min */	"1000",
 	/* max */	NULL,
-	/* default */	"10.000",
+	/* default */	"100000",
 	/* units */	"sessions",
 	/* flags */	0,
 	/* s-text */
@@ -888,7 +924,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
-#endif
+
 PARAM(
 	/* name */	shortlived,
 	/* typ */	timeout,
@@ -903,6 +939,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	sigsegv_handler,
 	/* typ */	bool,
@@ -917,6 +954,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	syslog_cli_traffic,
 	/* typ */	bool,
@@ -930,6 +968,8 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
+#if 0
 PARAM(
 	/* name */	tcp_keepalive_intvl,
 	/* typ */	timeout,
@@ -943,6 +983,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	tcp_keepalive_probes,
 	/* typ */	uint,
@@ -958,6 +999,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	tcp_keepalive_time,
 	/* typ */	timeout,
@@ -972,7 +1014,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
-#if 0
+
 PARAM(
 	/* name */	thread_pool_add_delay,
 	/* typ */	timeout,
@@ -992,6 +1034,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	thread_pool_destroy_delay,
 	/* typ */	timeout,
@@ -1006,6 +1049,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	thread_pool_fail_delay,
 	/* typ */	timeout,
@@ -1031,6 +1075,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	thread_pool_max,
 	/* typ */	thread_pool_max,
@@ -1048,6 +1093,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	thread_pool_min,
 	/* typ */	thread_pool_min,
@@ -1065,6 +1111,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	thread_pool_stack,
 	/* typ */	bytes,
@@ -1080,6 +1127,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	thread_pool_timeout,
 	/* typ */	timeout,
@@ -1096,6 +1144,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	thread_pools,
 	/* typ */	uint,
@@ -1117,6 +1166,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	thread_queue_limit,
 	/* typ */	uint,
@@ -1134,6 +1184,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	thread_stats_rate,
 	/* typ */	uint,
@@ -1145,7 +1196,7 @@ PARAM(
 	/* s-text */
 	"Worker threads accumulate statistics, and dump these into the "
 	"global stats counters if the lock is free when they finish a job "
-	"(request/fetch etc.)\n"
+	"(request/fetch etc).\n"
 	"This parameters defines the maximum number of jobs a worker "
 	"thread may handle, before it is forced to dump its accumulated "
 	"stats into the global counters.",
@@ -1153,6 +1204,7 @@ PARAM(
 	/* func */	NULL
 )
 #endif
+
 PARAM(
 	/* name */	timeout_idle,
 	/* typ */	timeout,
@@ -1168,6 +1220,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	timeout_linger,
 	/* typ */	timeout,
@@ -1187,6 +1240,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 #if 0
 PARAM(
 	/* name */	vcc_allow_inline_c,
@@ -1201,8 +1255,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
-#endif
-#if 0
+
 PARAM(
 	/* name */	vcc_err_unref,
 	/* typ */	bool,
@@ -1216,8 +1269,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
-#endif
-#if 0
+
 PARAM(
 	/* name */	vcc_unsafe_path,
 	/* typ */	bool,
@@ -1233,6 +1285,7 @@ PARAM(
 	/* func */	NULL
 )
 #endif
+
 PARAM(
 	/* name */	vcl_cooldown,
 	/* typ */	timeout,
@@ -1242,11 +1295,12 @@ PARAM(
 	/* units */	"seconds",
 	/* flags */	0,
 	/* s-text */
-	"How long time a VCL is kept warm after being replaced as the "
-	"active VCL.  (Granularity approximately 30 seconds.)",
+	"How long a VCL is kept warm after being replaced as the "
+	"active VCL (granularity approximately 30 seconds).",
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 #if 0
 PARAM(
 	/* name */	vcl_dir,
@@ -1262,6 +1316,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	vmod_dir,
 	/* typ */	string,
@@ -1276,6 +1331,7 @@ PARAM(
 	/* func */	NULL
 )
 #endif
+
 PARAM(
 	/* name */	vsl_buffer,
 	/* typ */	vsl_buffer,
@@ -1289,11 +1345,12 @@ PARAM(
 	"records.\n"
 	"Setting this too high costs memory, setting it too low will cause "
 	"more VSL flushes and likely increase lock-contention on the VSL "
-	"mutex.\n"
+	"mutex.\n\n"
 	"The minimum tracks the vsl_reclen parameter + 12 bytes.",
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 #if 0
 PARAM(
 	/* name */	vsl_mask,
@@ -1307,12 +1364,13 @@ PARAM(
 	"Mask individual VSL messages from being logged.\n"
 	"	default	Set default value\n"
 	"\n"
-	"Use +/- prefixe in front of VSL tag name, to mask/unmask "
+	"Use +/- prefix in front of VSL tag name, to mask/unmask "
 	"individual VSL messages.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #endif
+
 PARAM(
 	/* name */	vsl_reclen,
 	/* typ */	vsl_reclen,
@@ -1322,11 +1380,12 @@ PARAM(
 	/* units */	"bytes",
 	/* flags */	0,
 	/* s-text */
-	"Maximum number of bytes in SHM log record.\n"
+	"Maximum number of bytes in SHM log record.\n\n"
 	"The maximum tracks the vsl_buffer parameter - 12 bytes.",
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	vsl_space,
 	/* typ */	bytes,
@@ -1343,6 +1402,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	vsm_space,
 	/* typ */	bytes,
@@ -1358,6 +1418,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 #if 0
 PARAM(
 	/* name */	waiter,
@@ -1373,6 +1434,7 @@ PARAM(
 	/* func */	NULL
 )
 #endif
+
 PARAM(
 	/* name */	workspace_backend,
 	/* typ */	bytes_u,
@@ -1387,6 +1449,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	workspace_client,
 	/* typ */	bytes_u,
@@ -1401,6 +1464,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	workspace_session,
 	/* typ */	bytes_u,
@@ -1416,6 +1480,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	workspace_thread,
 	/* typ */	bytes_u,
@@ -1434,5 +1499,5 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
-#endif
+
 /*lint -restore */



From phk at FreeBSD.org  Wed Jan  6 19:06:50 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Wed, 06 Jan 2016 20:06:50 +0100
Subject: [master] 21eb520 On fetch failure,
 make the vcl_backend_error{} object cacheable for 5 seconds if there
 is a waiting list.
Message-ID: <E1aGtQ2-0003qE-0C@project.varnish-software.com>


commit 21eb520406f688000e1a5d9349f483d87b7949af
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Jan 6 18:57:31 2016 +0000

    On fetch failure, make the vcl_backend_error{} object cacheable
    for 5 seconds if there is a waiting list.
    
    This allows the waiting list to drain (returning 503) quickly,
    rather than one by one trying to pester the backend and only
    giving up after the connect_timeout.
    
    Related to: #1823

diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index 709a03b..efdffff 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -795,8 +795,22 @@ vbf_stp_error(struct worker *wrk, struct busyobj *bo)
 	http_TimeHeader(bo->beresp, "Date: ", now);
 	http_SetHeader(bo->beresp, "Server: Varnish");
 
-	EXP_Clr(&bo->fetch_objcore->exp);
-	bo->fetch_objcore->exp.t_origin = bo->t_prev;
+	if (bo->fetch_objcore->objhead->waitinglist != NULL) {
+		/*
+		 * If there is a waitinglist, it means that there is no
+		 * grace-able object, so cache the error return for a
+		 * short time, so the waiting list can drain, rather than
+		 * each objcore on the waiting list sequentially attempt
+		 * to fetch from the backend.
+		 */
+		bo->fetch_objcore->exp.t_origin = now;
+		bo->fetch_objcore->exp.ttl = 1;
+		bo->fetch_objcore->exp.grace = 5;
+		bo->fetch_objcore->exp.keep = 5;
+	} else {
+		EXP_Clr(&bo->fetch_objcore->exp);
+		bo->fetch_objcore->exp.t_origin = now;
+	}
 
 	synth_body = VSB_new_auto();
 	AN(synth_body);



From phk at FreeBSD.org  Wed Jan  6 19:14:29 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Wed, 06 Jan 2016 20:14:29 +0100
Subject: [master] 0be9404 Rush the objheader if there is a waiting list when
 it is deref'ed.
Message-ID: <E1aGtXR-0004gc-Gq@project.varnish-software.com>


commit 0be9404f18efeccd430698837c2806662b32523a
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Jan 6 19:13:47 2016 +0000

    Rush the objheader if there is a waiting list when it is deref'ed.
    
    Fixes:	#1823

diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index dd03596..0bb827d 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -820,6 +820,10 @@ HSH_DerefObjHead(struct worker *wrk, struct objhead **poh)
 		oh->refcnt--;
 		Lck_Unlock(&oh->mtx);
 		return(1);
+	} else if (oh->waitinglist != NULL) {
+		Lck_Lock(&oh->mtx);
+		hsh_rush(wrk, oh);
+		Lck_Unlock(&oh->mtx);
 	}
 
 	assert(oh->refcnt > 0);



From phk at FreeBSD.org  Wed Jan  6 20:20:54 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Wed, 06 Jan 2016 21:20:54 +0100
Subject: [master] 5c82680 Integrate the waiting list into the objhead (again).
 The net saving of one pointer does not amortize the extra CPU overhead and
 cache misses.
Message-ID: <E1aGuZi-0000cB-In@project.varnish-software.com>


commit 5c8268062bf06a2700dd27331c29c48d650c9197
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Jan 6 20:19:25 2016 +0000

    Integrate the waiting list into the objhead (again).  The net saving
    of one pointer does not amortize the extra CPU overhead and cache misses.

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index 4d60624..1e82ea2 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -121,7 +121,6 @@ struct sess;
 struct suckaddr;
 struct vrt_priv;
 struct vsb;
-struct waitinglist;
 struct worker;
 struct v1l;
 
@@ -329,7 +328,6 @@ struct worker {
 	struct pool		*pool;
 	struct objhead		*nobjhead;
 	struct objcore		*nobjcore;
-	struct waitinglist	*nwaitinglist;
 	void			*nhashpriv;
 	struct dstat		stats[1];
 	struct vsl_log		*vsl;		// borrowed from req/bo
diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index efdffff..d7411b0 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -795,7 +795,7 @@ vbf_stp_error(struct worker *wrk, struct busyobj *bo)
 	http_TimeHeader(bo->beresp, "Date: ", now);
 	http_SetHeader(bo->beresp, "Server: Varnish");
 
-	if (bo->fetch_objcore->objhead->waitinglist != NULL) {
+	if (!VTAILQ_EMPTY(&bo->fetch_objcore->objhead->waitinglist)) {
 		/*
 		 * If there is a waitinglist, it means that there is no
 		 * grace-able object, so cache the error return for a
diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index 0bb827d..e22289d 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -90,6 +90,7 @@ hsh_newobjhead(void)
 	XXXAN(oh);
 	oh->refcnt = 1;
 	VTAILQ_INIT(&oh->objcs);
+	VTAILQ_INIT(&oh->waitinglist);
 	Lck_New(&oh->mtx, lck_objhdr);
 	return (oh);
 }
@@ -99,7 +100,6 @@ hsh_newobjhead(void)
 static void
 hsh_prealloc(struct worker *wrk)
 {
-	struct waitinglist *wl;
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 
@@ -113,15 +113,6 @@ hsh_prealloc(struct worker *wrk)
 	}
 	CHECK_OBJ_NOTNULL(wrk->nobjhead, OBJHEAD_MAGIC);
 
-	if (wrk->nwaitinglist == NULL) {
-		ALLOC_OBJ(wl, WAITINGLIST_MAGIC);
-		XXXAN(wl);
-		VTAILQ_INIT(&wl->list);
-		wrk->nwaitinglist = wl;
-		wrk->stats->n_waitinglist++;
-	}
-	CHECK_OBJ_NOTNULL(wrk->nwaitinglist, WAITINGLIST_MAGIC);
-
 	if (hash->prep != NULL)
 		hash->prep(wrk);
 }
@@ -163,11 +154,6 @@ HSH_Cleanup(struct worker *wrk)
 		wrk->nobjhead = NULL;
 		wrk->stats->n_objecthead--;
 	}
-	if (wrk->nwaitinglist != NULL) {
-		FREE_OBJ(wrk->nwaitinglist);
-		wrk->nwaitinglist = NULL;
-		wrk->stats->n_waitinglist--;
-	}
 	if (wrk->nhashpriv != NULL) {
 		/* XXX: If needed, add slinger method for this */
 		free(wrk->nhashpriv);
@@ -181,6 +167,7 @@ HSH_DeleteObjHead(struct worker *wrk, struct objhead *oh)
 
 	AZ(oh->refcnt);
 	assert(VTAILQ_EMPTY(&oh->objcs));
+	assert(VTAILQ_EMPTY(&oh->waitinglist));
 	Lck_Delete(&oh->mtx);
 	wrk->stats->n_objecthead--;
 	FREE_OBJ(oh);
@@ -482,13 +469,7 @@ HSH_Lookup(struct req *req, struct objcore **ocp, struct objcore **bocp,
 	AZ(req->hash_ignore_busy);
 
 	if (wait_for_busy) {
-		CHECK_OBJ_NOTNULL(wrk->nwaitinglist, WAITINGLIST_MAGIC);
-		if (oh->waitinglist == NULL) {
-			oh->waitinglist = wrk->nwaitinglist;
-			wrk->nwaitinglist = NULL;
-		}
-		VTAILQ_INSERT_TAIL(&oh->waitinglist->list,
-		    req, w_list);
+		VTAILQ_INSERT_TAIL(&oh->waitinglist, req, w_list);
 		if (DO_DEBUG(DBG_WAITINGLIST))
 			VSLb(req->vsl, SLT_Debug, "on waiting list <%p>", oh);
 	} else {
@@ -517,21 +498,18 @@ hsh_rush(struct worker *wrk, struct objhead *oh)
 	unsigned u;
 	struct req *req;
 	struct sess *sp;
-	struct waitinglist *wl;
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(oh, OBJHEAD_MAGIC);
 	Lck_AssertHeld(&oh->mtx);
-	wl = oh->waitinglist;
-	CHECK_OBJ_NOTNULL(wl, WAITINGLIST_MAGIC);
 	for (u = 0; u < cache_param->rush_exponent; u++) {
-		req = VTAILQ_FIRST(&wl->list);
+		req = VTAILQ_FIRST(&oh->waitinglist);
 		if (req == NULL)
 			break;
 		CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
 		wrk->stats->busy_wakeup++;
 		AZ(req->wrk);
-		VTAILQ_REMOVE(&wl->list, req, w_list);
+		VTAILQ_REMOVE(&oh->waitinglist, req, w_list);
 		DSL(DBG_WAITINGLIST, req->vsl->wid, "off waiting list");
 		if (SES_Reschedule_Req(req)) {
 			/*
@@ -548,22 +526,17 @@ hsh_rush(struct worker *wrk, struct objhead *oh)
 				CNT_AcctLogCharge(wrk->stats, req);
 				Req_Release(req);
 				SES_Delete(sp, SC_OVERLOAD, NAN);
-				req = VTAILQ_FIRST(&wl->list);
+				req = VTAILQ_FIRST(&oh->waitinglist);
 				if (req == NULL)
 					break;
 				CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
-				VTAILQ_REMOVE(&wl->list, req, w_list);
+				VTAILQ_REMOVE(&oh->waitinglist, req, w_list);
 				DSL(DBG_WAITINGLIST, req->vsl->wid,
 				    "kill from waiting list");
 			}
 			break;
 		}
 	}
-	if (VTAILQ_EMPTY(&wl->list)) {
-		oh->waitinglist = NULL;
-		FREE_OBJ(wl);
-		wrk->stats->n_waitinglist--;
-	}
 }
 
 /*---------------------------------------------------------------------
@@ -706,7 +679,7 @@ HSH_Unbusy(struct worker *wrk, struct objcore *oc)
 	VTAILQ_REMOVE(&oh->objcs, oc, list);
 	VTAILQ_INSERT_HEAD(&oh->objcs, oc, list);
 	oc->flags &= ~OC_F_BUSY;
-	if (oh->waitinglist != NULL)
+	if (!VTAILQ_EMPTY(&oh->waitinglist))
 		hsh_rush(wrk, oh);
 	Lck_Unlock(&oh->mtx);
 }
@@ -781,7 +754,7 @@ HSH_DerefObjCore(struct worker *wrk, struct objcore **ocp)
 	r = --oc->refcnt;
 	if (!r)
 		VTAILQ_REMOVE(&oh->objcs, oc, list);
-	if (oh->waitinglist != NULL)
+	if (!VTAILQ_EMPTY(&oh->waitinglist))
 		hsh_rush(wrk, oh);
 	Lck_Unlock(&oh->mtx);
 	if (r != 0)
@@ -814,13 +787,13 @@ HSH_DerefObjHead(struct worker *wrk, struct objhead **poh)
 	CHECK_OBJ_NOTNULL(oh, OBJHEAD_MAGIC);
 
 	if (oh == private_oh) {
-		AZ(oh->waitinglist);
+		assert(VTAILQ_EMPTY(&oh->waitinglist));
 		Lck_Lock(&oh->mtx);
 		assert(oh->refcnt > 1);
 		oh->refcnt--;
 		Lck_Unlock(&oh->mtx);
 		return(1);
-	} else if (oh->waitinglist != NULL) {
+	} else if (!VTAILQ_EMPTY(&oh->waitinglist)) {
 		Lck_Lock(&oh->mtx);
 		hsh_rush(wrk, oh);
 		Lck_Unlock(&oh->mtx);
diff --git a/bin/varnishd/hash/hash_critbit.c b/bin/varnishd/hash/hash_critbit.c
index c4239b6..da671f5 100644
--- a/bin/varnishd/hash/hash_critbit.c
+++ b/bin/varnishd/hash/hash_critbit.c
@@ -406,8 +406,6 @@ hcb_deref(struct objhead *oh)
 		hcb_delete(&hcb_root, oh);
 		VTAILQ_INSERT_TAIL(&cool_h, oh, hoh_list);
 		Lck_Unlock(&hcb_mtx);
-		assert(VTAILQ_EMPTY(&oh->objcs));
-		AZ(oh->waitinglist);
 	}
 	Lck_Unlock(&oh->mtx);
 #ifdef PHK
diff --git a/bin/varnishd/hash/hash_slinger.h b/bin/varnishd/hash/hash_slinger.h
index 736f1b5..1fa219e 100644
--- a/bin/varnishd/hash/hash_slinger.h
+++ b/bin/varnishd/hash/hash_slinger.h
@@ -77,12 +77,6 @@ struct objcore *HSH_Private(struct worker *wrk);
 
 #ifdef VARNISH_CACHE_CHILD
 
-struct waitinglist {
-	unsigned		magic;
-#define WAITINGLIST_MAGIC	0x063a477a
-	VTAILQ_HEAD(, req)	list;
-};
-
 struct objhead {
 	unsigned		magic;
 #define OBJHEAD_MAGIC		0x1b96615d
@@ -91,7 +85,7 @@ struct objhead {
 	struct lock		mtx;
 	VTAILQ_HEAD(,objcore)	objcs;
 	uint8_t			digest[DIGEST_LEN];
-	struct waitinglist	*waitinglist;
+	VTAILQ_HEAD(, req)	waitinglist;
 
 	/*----------------------------------------------------
 	 * The fields below are for the sole private use of



From phk at FreeBSD.org  Wed Jan  6 23:30:43 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 07 Jan 2016 00:30:43 +0100
Subject: [master] 9b0a053 Remove the waiting list counter
Message-ID: <E1aGxXP-0007X0-Ah@project.varnish-software.com>


commit 9b0a053a198023d0b7ca3d828085e4bfce993c8e
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Jan 6 23:30:31 2016 +0000

    Remove the waiting list counter

diff --git a/include/tbl/vsc_f_main.h b/include/tbl/vsc_f_main.h
index 94341b5..1a01341 100644
--- a/include/tbl/vsc_f_main.h
+++ b/include/tbl/vsc_f_main.h
@@ -290,10 +290,6 @@ VSC_F(n_objecthead,		uint64_t, 1, 'g', 'i', info,
     "objecthead structs made",
 	"Number of objecthead structs made"
 )
-VSC_F(n_waitinglist,		uint64_t, 1, 'g', 'i', debug,
-    "waitinglist structs made",
-	"Number of waitinglist structs made"
-)
 
 VSC_F(n_backend,		uint64_t, 0, 'g', 'i', info,
     "Number of backends",



From fgsch at lodoss.net  Wed Jan  6 23:42:04 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Thu, 07 Jan 2016 00:42:04 +0100
Subject: [master] 10e6d90 Do not send a zero Content-Length for 204 and 304
Message-ID: <E1aGxiO-0000PD-3s@project.varnish-software.com>


commit 10e6d90bdb71a924fdb4b5a95c40b9969ec1749d
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Sun Jan 3 19:10:29 2016 +0000

    Do not send a zero Content-Length for 204 and 304
    
    In the 304 case we were already skipping it for non-zero C-L.
    For 204 responses this was introduced as a side effect of 271e1c52.

diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index 8279bdd..107ec45 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -73,7 +73,7 @@ cnt_vdp(struct req *req, struct busyobj *bo)
 		else
 			sendbody = 0;
 	} else if (status < 200 || status == 204 || status == 304) {
-		req->resp_len = 0;
+		req->resp_len = -1;
 		sendbody = 0;
 	} else
 		sendbody = 1;
diff --git a/bin/varnishtest/tests/c00008.vtc b/bin/varnishtest/tests/c00008.vtc
index 690b362..eb3c472 100644
--- a/bin/varnishtest/tests/c00008.vtc
+++ b/bin/varnishtest/tests/c00008.vtc
@@ -6,6 +6,10 @@ server s1 {
 	txresp -hdr "Last-Modified: Thu, 26 Jun 2008 12:00:01 GMT" \
 	    -hdr {ETag: "foo"} \
 	    -body "11111\n"
+	rxreq
+	expect req.url == "/bar"
+	txresp -hdr "Last-Modified: Thu, 26 Jun 2008 12:00:01 GMT" \
+	    -hdr {ETag: "bar"}
 } -start
 
 varnish v1 -vcl+backend { } -start
@@ -41,6 +45,21 @@ client c1 {
 	expect resp.http.etag == {"foo"}
 	expect resp.http.content-length == "<undef>"
 	expect resp.bodylen == "<undef>"
+
+	txreq -url "/bar"
+	rxresp
+	expect resp.status == 200
+	expect resp.http.etag == {"bar"}
+	expect resp.http.content-length == "0"
+	expect resp.bodylen == 0
+
+	txreq -url "/bar" \
+	    -hdr "If-Modified-Since: Thu, 26 Jun 2008 12:00:01 GMT"
+	rxresp -no_obj
+	expect resp.status == 304
+	expect resp.http.etag == {"bar"}
+	expect resp.http.content-length == <undef>
+	expect resp.bodylen == <undef>
 }
 
 client c1 -run
diff --git a/bin/varnishtest/tests/r01826.vtc b/bin/varnishtest/tests/r01826.vtc
index 6a553d5..d830d7b 100644
--- a/bin/varnishtest/tests/r01826.vtc
+++ b/bin/varnishtest/tests/r01826.vtc
@@ -2,6 +2,11 @@ varnishtest "Check we ignore a zero C-L with a 204"
 
 server s1 {
 	rxreq
+	txresp -status 204 -bodylen 5
+	expect_close
+
+	accept
+	rxreq
 	txresp -status 204
 } -start
 
@@ -11,5 +16,10 @@ varnish v1 -vcl+backend {
 client c1 {
 	txreq
 	rxresp
+	expect resp.status == 503
+
+	txreq
+	rxresp
 	expect resp.status == 204
+	expect resp.http.content-length == <undef>
 } -run



From phk at FreeBSD.org  Thu Jan  7 00:11:05 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 07 Jan 2016 01:11:05 +0100
Subject: [master] 9c4f3e8 Add a final backstop,
 so we absolutely 100% certainly do not try to delete a objhead while
 it still has a waiting list, by forcing the last ref holder to rush the WL.
Message-ID: <E1aGyAT-0001lW-La@project.varnish-software.com>


commit 9c4f3e8097bacaceee647a0e5f01d2323c02421c
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu Jan 7 00:03:35 2016 +0000

    Add a final backstop, so we absolutely 100% certainly do not
    try to delete a objhead while it still has a waiting list,
    by forcing the last ref holder to rush the WL.
    
    Since the hasher owns the refcounts on objhead, we cannot just
    mingle req and objcore refcounts.
    
    Fortunately we don't need to add another refcounter, because all
    we really care about is the wl being empty when we drop the last
    ref.
    
    The wl/hsh_rush() mechanism will work differently with different
    thread-scheduling schenarios, and I cannot definitively rule out
    that we can drop the last ref on an oh, while there are still req's
    on the waiting list.
    
    Given that, and the existence proof in ticket #1823's race, this
    might have been the indicated memory-trampler.

diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index e22289d..12fd0d6 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -793,10 +793,25 @@ HSH_DerefObjHead(struct worker *wrk, struct objhead **poh)
 		oh->refcnt--;
 		Lck_Unlock(&oh->mtx);
 		return(1);
-	} else if (!VTAILQ_EMPTY(&oh->waitinglist)) {
+	}
+
+	/*
+	 * Make absolutely certain that we do not let the final ref
+	 * disappear until the waitinglist is empty.
+	 * This is necessary because the req's on the waiting list do
+	 * not hold any ref on the objhead of their own, and we cannot
+	 * just make the hold the same ref's as objcore, that would
+	 * confuse hashers.
+	 */
+	while (!VTAILQ_EMPTY(&oh->waitinglist)) {
 		Lck_Lock(&oh->mtx);
+		assert(oh->refcnt > 0);
+		r = oh->refcnt;
 		hsh_rush(wrk, oh);
 		Lck_Unlock(&oh->mtx);
+		if (r > 1)
+			break;
+		usleep(100000);
 	}
 
 	assert(oh->refcnt > 0);
diff --git a/bin/varnishd/hash/hash_critbit.c b/bin/varnishd/hash/hash_critbit.c
index da671f5..4b9cc6b 100644
--- a/bin/varnishd/hash/hash_critbit.c
+++ b/bin/varnishd/hash/hash_critbit.c
@@ -394,9 +394,7 @@ hcb_start(void)
 static int __match_proto__(hash_deref_f)
 hcb_deref(struct objhead *oh)
 {
-	int r;
 
-	r = 1;
 	CHECK_OBJ_NOTNULL(oh, OBJHEAD_MAGIC);
 	Lck_Lock(&oh->mtx);
 	assert(oh->refcnt > 0);
@@ -411,7 +409,7 @@ hcb_deref(struct objhead *oh)
 #ifdef PHK
 	fprintf(stderr, "hcb_defef %d %d <%s>\n", __LINE__, r, oh->hash);
 #endif
-	return (r);
+	return (1);
 }
 
 static struct objhead * __match_proto__(hash_lookup_f)



From phk at FreeBSD.org  Fri Jan  8 09:54:17 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Fri, 08 Jan 2016 10:54:17 +0100
Subject: [master] 852d564 Add a couple of asserts for Coverity
Message-ID: <E1aHTkP-0002dd-8w@project.varnish-software.com>


commit 852d564f32d8e570afdb810d229910f28110c896
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Jan 8 09:37:15 2016 +0000

    Add a couple of asserts for Coverity

diff --git a/lib/libvarnish/vsub.c b/lib/libvarnish/vsub.c
index a7c658c..dd0cb9d 100644
--- a/lib/libvarnish/vsub.c
+++ b/lib/libvarnish/vsub.c
@@ -56,10 +56,14 @@ struct vsub_priv {
 void
 VSUB_closefrom(int fd)
 {
+
+	assert(fd >= 0);
+
 #ifdef HAVE_CLOSEFROM
 	closefrom(fd);
 #else
-	int i;
+	int i = sysconf(_SC_OPEN_MAX);
+	assert(i > 0);
 	for (i = sysconf(_SC_OPEN_MAX); i > fd; i--)
 		(void)close(i);
 #endif



From phk at FreeBSD.org  Fri Jan  8 09:54:17 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Fri, 08 Jan 2016 10:54:17 +0100
Subject: [master] 1719067 Reset storage_malloc counter on allocation failure.
Message-ID: <E1aHTkP-0002dj-AY@project.varnish-software.com>


commit 1719067353442a17ebbc042841834593ad1adf85
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Jan 8 09:53:40 2016 +0000

    Reset storage_malloc counter on allocation failure.
    
    Submitted by:	github::onovy

diff --git a/bin/varnishd/storage/storage_malloc.c b/bin/varnishd/storage/storage_malloc.c
index 8f085f7..7060f55 100644
--- a/bin/varnishd/storage/storage_malloc.c
+++ b/bin/varnishd/storage/storage_malloc.c
@@ -107,6 +107,7 @@ sma_alloc(const struct stevedore *st, size_t size)
 		 * XXX: Not want to pick up the lock twice just for stats.
 		 */
 		sma_sc->stats->c_fail++;
+		sma_sc->sma_alloc -= size;
 		sma_sc->stats->c_bytes -= size;
 		sma_sc->stats->g_alloc--;
 		sma_sc->stats->g_bytes -= size;



From phk at FreeBSD.org  Fri Jan  8 10:33:58 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Fri, 08 Jan 2016 11:33:58 +0100
Subject: [master] ac6cf0f Split the parameters for the VCC compiler into a
 separate structure for more clarity, before we start hacking VCC further.
Message-ID: <E1aHUMo-0006Ar-6W@project.varnish-software.com>


commit ac6cf0fd9cd9ad03e5279eb7efa4f31788df564f
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Jan 8 10:32:18 2016 +0000

    Split the parameters for the VCC compiler into a separate structure
    for more clarity, before we start hacking VCC further.

diff --git a/bin/varnishd/mgt/mgt_vcc.c b/bin/varnishd/mgt/mgt_vcc.c
index d13432e..9d28804 100644
--- a/bin/varnishd/mgt/mgt_vcc.c
+++ b/bin/varnishd/mgt/mgt_vcc.c
@@ -64,7 +64,7 @@ unsigned mgt_vcc_err_unref;
 unsigned mgt_vcc_allow_inline_c;
 unsigned mgt_vcc_unsafe_path;
 
-static struct vcc *vcc;
+static struct vcp *vcp;
 
 #define VGC_SRC		"vgc.c"
 #define VGC_LIB		"vgc.so"
@@ -94,12 +94,12 @@ run_vcc(void *priv)
 
 	sb = VSB_new_auto();
 	XXXAN(sb);
-	VCC_VCL_dir(vcc, mgt_vcl_dir);
-	VCC_VMOD_dir(vcc, mgt_vmod_dir);
-	VCC_Err_Unref(vcc, mgt_vcc_err_unref);
-	VCC_Allow_InlineC(vcc, mgt_vcc_allow_inline_c);
-	VCC_Unsafe_Path(vcc, mgt_vcc_unsafe_path);
-	csrc = VCC_Compile(vcc, sb, vp->src);
+	VCP_VCL_dir(vcp, mgt_vcl_dir);
+	VCP_VMOD_dir(vcp, mgt_vmod_dir);
+	VCP_Err_Unref(vcp, mgt_vcc_err_unref);
+	VCP_Allow_InlineC(vcp, mgt_vcc_allow_inline_c);
+	VCP_Unsafe_Path(vcp, mgt_vcc_unsafe_path);
+	csrc = VCC_Compile(vcp, sb, vp->src);
 	AZ(VSB_finish(sb));
 	if (VSB_len(sb))
 		printf("%s", VSB_data(sb));
@@ -318,7 +318,7 @@ void
 mgt_vcc_init(void)
 {
 
-	vcc = VCC_New();
-	AN(vcc);
-	VCC_Builtin_VCL(vcc, builtin_vcl);
+	vcp = VCP_New();
+	AN(vcp);
+	VCP_Builtin_VCL(vcp, builtin_vcl);
 }
diff --git a/include/libvcc.h b/include/libvcc.h
index 6b16252..696123e 100644
--- a/include/libvcc.h
+++ b/include/libvcc.h
@@ -30,12 +30,12 @@
 
 struct vcc;
 
-struct vcc *VCC_New(void);
-void VCC_Builtin_VCL(struct vcc *, const char *str);
-void VCC_VCL_dir(struct vcc *, const char *str);
-void VCC_VMOD_dir(struct vcc *, const char *str);
-void VCC_Err_Unref(struct vcc *tl, unsigned u);
-void VCC_Allow_InlineC(struct vcc *tl, unsigned u);
-void VCC_Unsafe_Path(struct vcc *tl, unsigned u);
+struct vcp *VCP_New(void);
+void VCP_Builtin_VCL(struct vcp *, const char *str);
+void VCP_VCL_dir(struct vcp *, const char *str);
+void VCP_VMOD_dir(struct vcp *, const char *str);
+void VCP_Err_Unref(struct vcp *tl, unsigned u);
+void VCP_Allow_InlineC(struct vcp *tl, unsigned u);
+void VCP_Unsafe_Path(struct vcp *tl, unsigned u);
 
-char *VCC_Compile(const struct vcc *, struct vsb *sb, const char *b);
+char *VCC_Compile(const struct vcp *, struct vsb *sb, const char *b);
diff --git a/lib/libvcc/vcc_acl.c b/lib/libvcc/vcc_acl.c
index b5ee224..78d9ec5 100644
--- a/lib/libvcc/vcc_acl.c
+++ b/lib/libvcc/vcc_acl.c
@@ -364,7 +364,7 @@ vcc_acl_emit(struct vcc *tl, const char *acln, int anon)
 	Fh(tl, 0, "\t\tVRT_acl_log(ctx, \"NO_FAM %s\");\n", acln);
 	Fh(tl, 0, "\t\treturn(0);\n");
 	Fh(tl, 0, "\t}\n\n");
-	if (!tl->err_unref && !anon ) {
+	if (!tl->param->err_unref && !anon ) {
 		ifp = New_IniFin(tl);
 		VSB_printf(ifp->ini,
 			"\tif (0) match_acl_named_%s(0, 0);\n", acln);
diff --git a/lib/libvcc/vcc_compile.c b/lib/libvcc/vcc_compile.c
index e79597f..aaba4bf 100644
--- a/lib/libvcc/vcc_compile.c
+++ b/lib/libvcc/vcc_compile.c
@@ -450,16 +450,16 @@ vcc_destroy_source(struct source *sp)
 /*--------------------------------------------------------------------*/
 
 static struct source *
-vcc_file_source(const struct vcc *tl, struct vsb *sb, const char *fn)
+vcc_file_source(const struct vcp * const vcp, struct vsb *sb, const char *fn)
 {
 	char *f;
 	struct source *sp;
 
-	if (!tl->unsafe_path && strchr(fn, '/') != NULL) {
+	if (!vcp->unsafe_path && strchr(fn, '/') != NULL) {
 		VSB_printf(sb, "Include path is unsafe '%s'\n", fn);
 		return (NULL);
 	}
-	f = VFIL_readfile(tl->vcl_dir, fn, NULL);
+	f = VFIL_readfile(vcp->vcl_dir, fn, NULL);
 	if (f == NULL) {
 		VSB_printf(sb, "Cannot read file '%s': %s\n",
 		    fn, strerror(errno));
@@ -500,7 +500,7 @@ vcc_resolve_includes(struct vcc *tl)
 			return;
 		}
 
-		sp = vcc_file_source(tl, tl->sb, t1->dec);
+		sp = vcc_file_source(tl->param, tl->sb, t1->dec);
 		if (sp == NULL) {
 			vcc_ErrWhere(tl, t1);
 			return;
@@ -522,24 +522,16 @@ vcc_resolve_includes(struct vcc *tl)
 /*--------------------------------------------------------------------*/
 
 static struct vcc *
-vcc_NewVcc(const struct vcc *tl0)
+vcc_NewVcc(const struct vcp *vcp)
 {
 	struct vcc *tl;
 	int i;
 
+	CHECK_OBJ_NOTNULL(vcp, VCP_MAGIC);
 	ALLOC_OBJ(tl, VCC_MAGIC);
 	AN(tl);
-	if (tl0 != NULL) {
-		REPLACE(tl->builtin_vcl, tl0->builtin_vcl);
-		REPLACE(tl->vcl_dir, tl0->vcl_dir);
-		REPLACE(tl->vmod_dir, tl0->vmod_dir);
-		tl->vars = tl0->vars;
-		tl->err_unref = tl0->err_unref;
-		tl->allow_inline_c = tl0->allow_inline_c;
-		tl->unsafe_path = tl0->unsafe_path;
-	} else {
-		tl->err_unref = 1;
-	}
+	tl->param = vcp;
+	tl->vars = vcc_vars;
 	VTAILQ_INIT(&tl->symbols);
 	VTAILQ_INIT(&tl->inifin);
 	VTAILQ_INIT(&tl->membits);
@@ -606,7 +598,8 @@ vcc_DestroyTokenList(struct vcc *tl, char *ret)
  */
 
 static char *
-vcc_CompileSource(const struct vcc *tl0, struct vsb *sb, struct source *sp)
+vcc_CompileSource(const struct vcp * const vcp, struct vsb *sb,
+    struct source *sp)
 {
 	struct vcc *tl;
 	struct symbol *sym;
@@ -616,7 +609,7 @@ vcc_CompileSource(const struct vcc *tl0, struct vsb *sb, struct source *sp)
 	char *of;
 	int i;
 
-	tl = vcc_NewVcc(tl0);
+	tl = vcc_NewVcc(vcp);
 	tl->sb = sb;
 
 	vcc_Expr_Init(tl);
@@ -649,7 +642,7 @@ vcc_CompileSource(const struct vcc *tl0, struct vsb *sb, struct source *sp)
 		return (vcc_DestroyTokenList(tl, NULL));
 
 	/* Register and lex the builtin VCL */
-	sp = vcc_new_source(tl->builtin_vcl, NULL, "Builtin");
+	sp = vcc_new_source(tl->param->builtin_vcl, NULL, "Builtin");
 	assert(sp != NULL);
 	VTAILQ_INSERT_TAIL(&tl->sources, sp, list);
 	sp->idx = tl->nsources++;
@@ -759,7 +752,7 @@ vcc_CompileSource(const struct vcc *tl0, struct vsb *sb, struct source *sp)
  */
 
 char *
-VCC_Compile(const struct vcc *tl, struct vsb *sb, const char *b)
+VCC_Compile(const struct vcp *vcp, struct vsb *sb, const char *b)
 {
 	struct source *sp;
 	char *r;
@@ -767,7 +760,7 @@ VCC_Compile(const struct vcc *tl, struct vsb *sb, const char *b)
 	sp = vcc_new_source(b, NULL, "input");
 	if (sp == NULL)
 		return (NULL);
-	r = vcc_CompileSource(tl, sb, sp);
+	r = vcc_CompileSource(vcp, sb, sp);
 	return (r);
 }
 
@@ -775,16 +768,15 @@ VCC_Compile(const struct vcc *tl, struct vsb *sb, const char *b)
  * Allocate a compiler instance
  */
 
-struct vcc *
-VCC_New(void)
+struct vcp *
+VCP_New(void)
 {
-	struct vcc *tl;
+	struct vcp *vcp;
 
-	tl = vcc_NewVcc(NULL);
+	ALLOC_OBJ(vcp, VCP_MAGIC);
+	AN(vcp);
 
-	tl->vars = vcc_vars;
-
-	return (tl);
+	return (vcp);
 }
 
 /*--------------------------------------------------------------------
@@ -792,11 +784,11 @@ VCC_New(void)
  */
 
 void
-VCC_Builtin_VCL(struct vcc *tl, const char *str)
+VCP_Builtin_VCL(struct vcp *vcp, const char *str)
 {
 
-	CHECK_OBJ_NOTNULL(tl, VCC_MAGIC);
-	REPLACE(tl->builtin_vcl, str);
+	CHECK_OBJ_NOTNULL(vcp, VCP_MAGIC);
+	REPLACE(vcp->builtin_vcl, str);
 }
 
 /*--------------------------------------------------------------------
@@ -804,11 +796,11 @@ VCC_Builtin_VCL(struct vcc *tl, const char *str)
  */
 
 void
-VCC_VCL_dir(struct vcc *tl, const char *str)
+VCP_VCL_dir(struct vcp *vcp, const char *str)
 {
 
-	CHECK_OBJ_NOTNULL(tl, VCC_MAGIC);
-	REPLACE(tl->vcl_dir, str);
+	CHECK_OBJ_NOTNULL(vcp, VCP_MAGIC);
+	REPLACE(vcp->vcl_dir, str);
 }
 
 /*--------------------------------------------------------------------
@@ -816,11 +808,11 @@ VCC_VCL_dir(struct vcc *tl, const char *str)
  */
 
 void
-VCC_VMOD_dir(struct vcc *tl, const char *str)
+VCP_VMOD_dir(struct vcp *vcp, const char *str)
 {
 
-	CHECK_OBJ_NOTNULL(tl, VCC_MAGIC);
-	REPLACE(tl->vmod_dir, str);
+	CHECK_OBJ_NOTNULL(vcp, VCP_MAGIC);
+	REPLACE(vcp->vmod_dir, str);
 }
 
 /*--------------------------------------------------------------------
@@ -828,25 +820,25 @@ VCC_VMOD_dir(struct vcc *tl, const char *str)
  */
 
 void
-VCC_Err_Unref(struct vcc *tl, unsigned u)
+VCP_Err_Unref(struct vcp *vcp, unsigned u)
 {
 
-	CHECK_OBJ_NOTNULL(tl, VCC_MAGIC);
-	tl->err_unref = u;
+	CHECK_OBJ_NOTNULL(vcp, VCP_MAGIC);
+	vcp->err_unref = u;
 }
 
 void
-VCC_Allow_InlineC(struct vcc *tl, unsigned u)
+VCP_Allow_InlineC(struct vcp *vcp, unsigned u)
 {
 
-	CHECK_OBJ_NOTNULL(tl, VCC_MAGIC);
-	tl->allow_inline_c = u;
+	CHECK_OBJ_NOTNULL(vcp, VCP_MAGIC);
+	vcp->allow_inline_c = u;
 }
 
 void
-VCC_Unsafe_Path(struct vcc *tl, unsigned u)
+VCP_Unsafe_Path(struct vcp *vcp, unsigned u)
 {
 
-	CHECK_OBJ_NOTNULL(tl, VCC_MAGIC);
-	tl->unsafe_path = u;
+	CHECK_OBJ_NOTNULL(vcp, VCP_MAGIC);
+	vcp->unsafe_path = u;
 }
diff --git a/lib/libvcc/vcc_compile.h b/lib/libvcc/vcc_compile.h
index 4c155a1..72ff7b7 100644
--- a/lib/libvcc/vcc_compile.h
+++ b/lib/libvcc/vcc_compile.h
@@ -155,14 +155,24 @@ struct inifin {
 
 VTAILQ_HEAD(inifinhead, inifin);
 
-struct vcc {
+struct vcp {
 	unsigned		magic;
-#define VCC_MAGIC		0x24ad719d
+#define VCP_MAGIC		0xd90acfbc
 
-	/* Parameter/Template section */
 	char			*builtin_vcl;
 	char			*vcl_dir;
 	char			*vmod_dir;
+	unsigned		err_unref;
+	unsigned		allow_inline_c;
+	unsigned		unsafe_path;
+};
+
+struct vcc {
+	unsigned		magic;
+#define VCC_MAGIC		0x24ad719d
+
+	/* Parameter/Template section */
+	const struct vcp	*param;
 
 	const struct var	*vars;
 	VTAILQ_HEAD(, symbol)	symbols;
@@ -202,9 +212,6 @@ struct vcc {
 
 	unsigned		unique;
 
-	unsigned		err_unref;
-	unsigned		allow_inline_c;
-	unsigned		unsafe_path;
 };
 
 struct var {
diff --git a/lib/libvcc/vcc_parse.c b/lib/libvcc/vcc_parse.c
index accabd6..8a94749 100644
--- a/lib/libvcc/vcc_parse.c
+++ b/lib/libvcc/vcc_parse.c
@@ -160,7 +160,7 @@ vcc_Compound(struct vcc *tl)
 			Fb(tl, 1, "}\n");
 			return;
 		case CSRC:
-			if (tl->allow_inline_c) {
+			if (tl->param->allow_inline_c) {
 				Fb(tl, 1, "%.*s\n",
 				    (int) (tl->t->e - (tl->t->b + 2)),
 				    tl->t->b + 1);
@@ -357,7 +357,7 @@ vcc_Parse(struct vcc *tl)
 		ERRCHK(tl);
 		switch (tl->t->tok) {
 		case CSRC:
-			if (tl->allow_inline_c) {
+			if (tl->param->allow_inline_c) {
 				Fc(tl, 0, "%.*s\n",
 				    (int) (tl->t->e - (tl->t->b + 4)),
 				    tl->t->b + 2);
diff --git a/lib/libvcc/vcc_vmod.c b/lib/libvcc/vcc_vmod.c
index 60d2ed2..d82ad3e 100644
--- a/lib/libvcc/vcc_vmod.c
+++ b/lib/libvcc/vcc_vmod.c
@@ -85,7 +85,7 @@ vcc_ParseImport(struct vcc *tl)
 	sym->def_e = tl->t;
 
 	if (tl->t->tok == ID) {
-		if (!tl->unsafe_path) {
+		if (!tl->param->unsafe_path) {
 			VSB_printf(tl->sb,
 			    "'import ... from path ...' not allowed.\nAt:");
 			vcc_ErrToken(tl, tl->t);
@@ -102,7 +102,7 @@ vcc_ParseImport(struct vcc *tl)
 		bprintf(fn, "%s", tl->t->dec);
 		vcc_NextToken(tl);
 	} else {
-		bprintf(fn, "%s/libvmod_%.*s.so", tl->vmod_dir, PF(mod));
+		bprintf(fn, "%s/libvmod_%.*s.so", tl->param->vmod_dir, PF(mod));
 	}
 
 	SkipToken(tl, ';');
diff --git a/lib/libvcc/vcc_xref.c b/lib/libvcc/vcc_xref.c
index c3ef375..5440829 100644
--- a/lib/libvcc/vcc_xref.c
+++ b/lib/libvcc/vcc_xref.c
@@ -108,7 +108,7 @@ vcc_checkref(struct vcc *tl, const struct symbol *sym)
 		VSB_printf(tl->sb, "Unused %s %.*s, defined:\n",
 		    VCC_SymKind(tl, sym), PF(sym->def_b));
 		vcc_ErrWhere(tl, sym->def_b);
-		if (!tl->err_unref) {
+		if (!tl->param->err_unref) {
 			VSB_printf(tl->sb, "(That was just a warning)\n");
 			tl->err = 0;
 		}
@@ -289,7 +289,7 @@ vcc_checkaction2(struct vcc *tl, const struct symbol *sym)
 		return;
 	VSB_printf(tl->sb, "Function unused\n");
 	vcc_ErrWhere(tl, p->name);
-	if (!tl->err_unref) {
+	if (!tl->param->err_unref) {
 		VSB_printf(tl->sb, "(That was just a warning)\n");
 		tl->err = 0;
 	}



From dridi.boukelmoune at gmail.com  Fri Jan  8 10:53:07 2016
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Fri, 08 Jan 2016 11:53:07 +0100
Subject: [master] 3220a17 Display the WRONG() message in VAS_Fail_default
Message-ID: <E1aHUfL-0007fE-79@project.varnish-software.com>


commit 3220a178d1379e19de8f15b358328fa2bdbe979e
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Tue Jan 5 15:00:03 2016 +0100

    Display the WRONG() message in VAS_Fail_default

diff --git a/lib/libvarnish/vas.c b/lib/libvarnish/vas.c
index 2d810d2..0c857a0 100644
--- a/lib/libvarnish/vas.c
+++ b/lib/libvarnish/vas.c
@@ -1,6 +1,6 @@
 /*-
  * Copyright (c) 2006 Verdens Gang AS
- * Copyright (c) 2006-2011 Varnish Software AS
+ * Copyright (c) 2006-2016 Varnish Software AS
  * All rights reserved.
  *
  * Author: Poul-Henning Kamp <phk at phk.freebsd.dk>
@@ -46,7 +46,7 @@ VAS_Fail_default(const char *func, const char *file, int line,
 
 	if (kind == VAS_MISSING) {
 		fprintf(stderr,
-		    "Missing errorhandling code in %s(), %s line %d:\n"
+		    "Missing error handling code in %s(), %s line %d:\n"
 		    "  Condition(%s) not true.\n",
 		    func, file, line, cond);
 	} else if (kind == VAS_INCOMPLETE) {
@@ -55,8 +55,8 @@ VAS_Fail_default(const char *func, const char *file, int line,
 		    func, file, line);
 	} else if (kind == VAS_WRONG) {
 		fprintf(stderr,
-		    "Wrong turn in %s(), %s line %d:\n",
-		    func, file, line);
+		    "Wrong turn in %s(), %s line %d: %s\n",
+		    func, file, line, cond);
 	} else {
 		fprintf(stderr,
 		    "Assert error in %s(), %s line %d:\n"



From phk at FreeBSD.org  Fri Jan  8 14:52:14 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Fri, 08 Jan 2016 15:52:14 +0100
Subject: [master] 6545ac1 White-space fix and minor flexelint-comforting
Message-ID: <E1aHYOk-000603-CQ@project.varnish-software.com>


commit 6545ac19effe8caffbb07a35e1a4a07e81464421
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Jan 8 14:51:56 2016 +0000

    White-space fix and minor flexelint-comforting

diff --git a/lib/libvmod_directors/random.c b/lib/libvmod_directors/random.c
index d283f04..dee16ff 100644
--- a/lib/libvmod_directors/random.c
+++ b/lib/libvmod_directors/random.c
@@ -114,7 +114,7 @@ vmod_random_add_backend(VRT_CTX,
 }
 
 VCL_VOID vmod_random_remove_backend(VRT_CTX,
-    struct vmod_directors_random *rr, VCL_BACKEND be)
+    struct vmod_directors_random *rr, VCL_BACKEND be) __match_proto__()
 {
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
 	CHECK_OBJ_NOTNULL(rr, VMOD_DIRECTORS_RANDOM_MAGIC);
diff --git a/lib/libvmod_directors/vdir.c b/lib/libvmod_directors/vdir.c
index ddbfbcc..22dfee3 100644
--- a/lib/libvmod_directors/vdir.c
+++ b/lib/libvmod_directors/vdir.c
@@ -140,27 +140,27 @@ vdir_add_backend(struct vdir *vd, VCL_BACKEND be, double weight)
 unsigned
 vdir_remove_backend(struct vdir *vd, VCL_BACKEND be)
 {
-        unsigned u, n;
-
-        CHECK_OBJ_NOTNULL(vd, VDIR_MAGIC);
-        if (be == NULL)
-                return vd->n_backend;
-        CHECK_OBJ(be, DIRECTOR_MAGIC);
-        vdir_wrlock(vd);
-        for (u = 0; u < vd->n_backend; u++)
-                if (vd->backend[u] == be)
-                        break;
-        if (u == vd->n_backend) {
-                vdir_unlock(vd);
-                return vd->n_backend;
-        }
-        vd->total_weight -= vd->weight[u];
-        n = vd->n_backend - u - 1;
-        memmove(&vd->backend[u], &vd->backend[u+1], n * sizeof(vd->backend[0]));
-        memmove(&vd->weight[u], &vd->weight[u+1], n * sizeof(vd->weight[0]));
-        vd->n_backend--;
-        vdir_unlock(vd);
-        return vd->n_backend;
+	unsigned u, n;
+
+	CHECK_OBJ_NOTNULL(vd, VDIR_MAGIC);
+	if (be == NULL)
+		return (vd->n_backend);
+	CHECK_OBJ(be, DIRECTOR_MAGIC);
+	vdir_wrlock(vd);
+	for (u = 0; u < vd->n_backend; u++)
+		if (vd->backend[u] == be)
+			break;
+	if (u == vd->n_backend) {
+		vdir_unlock(vd);
+		return (vd->n_backend);
+	}
+	vd->total_weight -= vd->weight[u];
+	n = (vd->n_backend - u) - 1;
+	memmove(&vd->backend[u], &vd->backend[u+1], n * sizeof(vd->backend[0]));
+	memmove(&vd->weight[u], &vd->weight[u+1], n * sizeof(vd->weight[0]));
+	vd->n_backend--;
+	vdir_unlock(vd);
+	return (vd->n_backend);
 }
 
 unsigned



From dridi.boukelmoune at gmail.com  Fri Jan  8 15:48:02 2016
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Fri, 08 Jan 2016 16:48:02 +0100
Subject: [master] ad8d91f Use UPPER-case names for pseudo-enums constants
Message-ID: <E1aHZGk-00019U-8T@project.varnish-software.com>


commit ad8d91fb7a2e43c35feb2bb1466b3fc9f7987c5a
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Fri Jan 8 16:42:36 2016 +0100

    Use UPPER-case names for pseudo-enums constants

diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index 54effd1..11e4f54 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -47,11 +47,11 @@
 #include "vcli.h"
 #include "vcli_priv.h"
 
-static const char * const vcl_temp_init = "init";
-static const char * const vcl_temp_cold = "cold";
-static const char * const vcl_temp_warm = "warm";
-static const char * const vcl_temp_busy = "busy";
-static const char * const vcl_temp_cooling = "cooling";
+static const char * const VCL_TEMP_INIT = "init";
+static const char * const VCL_TEMP_COLD = "cold";
+static const char * const VCL_TEMP_WARM = "warm";
+static const char * const VCL_TEMP_BUSY = "busy";
+static const char * const VCL_TEMP_COOLING = "cooling";
 
 struct vcl {
 	unsigned		magic;
@@ -138,7 +138,7 @@ VCL_Get(struct vcl **vcc)
 		(void)usleep(100000);
 
 	CHECK_OBJ_NOTNULL(vcl_active, VCL_MAGIC);
-	assert(vcl_active->temp == vcl_temp_warm);
+	assert(vcl_active->temp == VCL_TEMP_WARM);
 	Lck_Lock(&vcl_mtx);
 	AN(vcl_active);
 	*vcc = vcl_active;
@@ -152,7 +152,7 @@ void
 VCL_Refresh(struct vcl **vcc)
 {
 	CHECK_OBJ_NOTNULL(vcl_active, VCL_MAGIC);
-	assert(vcl_active->temp == vcl_temp_warm);
+	assert(vcl_active->temp == VCL_TEMP_WARM);
 	if (*vcc == vcl_active)
 		return;
 	if (*vcc != NULL)
@@ -165,7 +165,7 @@ VCL_Ref(struct vcl *vcl)
 {
 
 	CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC);
-	assert(vcl->temp != vcl_temp_init && vcl->temp != vcl_temp_cold);
+	assert(vcl->temp != VCL_TEMP_INIT && vcl->temp != VCL_TEMP_COLD);
 	Lck_Lock(&vcl_mtx);
 	assert(vcl->busy > 0);
 	vcl->busy++;
@@ -201,17 +201,17 @@ VCL_AddBackend(struct vcl *vcl, struct backend *be)
 	CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC);
 	CHECK_OBJ_NOTNULL(be, BACKEND_MAGIC);
 
-	if (vcl->temp == vcl_temp_cooling)
+	if (vcl->temp == VCL_TEMP_COOLING)
 		return (1);
 
 	Lck_Lock(&vcl_mtx);
 	VTAILQ_INSERT_TAIL(&vcl->backend_list, be, vcl_list);
 	Lck_Unlock(&vcl_mtx);
 
-	if (vcl->temp == vcl_temp_warm || vcl->temp == vcl_temp_busy)
+	if (vcl->temp == VCL_TEMP_WARM || vcl->temp == VCL_TEMP_BUSY)
 		/* Only when adding backend to already warm VCL */
 		VBE_Event(be, VCL_EVENT_WARM);
-	else if (vcl->temp != vcl_temp_init)
+	else if (vcl->temp != VCL_TEMP_INIT)
 		WRONG("Dynamic Backends can only be added to warm VCLs");
 
 	return (0);
@@ -228,7 +228,7 @@ VCL_DelBackend(struct backend *be)
 	Lck_Lock(&vcl_mtx);
 	VTAILQ_REMOVE(&vcl->backend_list, be, vcl_list);
 	Lck_Unlock(&vcl_mtx);
-	if (vcl->temp == vcl_temp_warm)
+	if (vcl->temp == VCL_TEMP_WARM)
 		VBE_Event(be, VCL_EVENT_COLD);
 }
 
@@ -385,7 +385,7 @@ VRT_ref_vcl(VRT_CTX)
 
 	vcl = ctx->vcl;
 	CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC);
-	xxxassert(vcl->temp == vcl_temp_warm);
+	xxxassert(vcl->temp == VCL_TEMP_WARM);
 
 	Lck_Lock(&vcl_mtx);
 	vcl->refcount++;
@@ -401,8 +401,8 @@ VRT_rel_vcl(VRT_CTX)
 
 	vcl = ctx->vcl;
 	CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC);
-	assert(vcl->temp == vcl_temp_warm || vcl->temp == vcl_temp_busy ||
-	    vcl->temp == vcl_temp_cooling);
+	assert(vcl->temp == VCL_TEMP_WARM || vcl->temp == VCL_TEMP_BUSY ||
+	    vcl->temp == VCL_TEMP_COOLING);
 
 	Lck_Lock(&vcl_mtx);
 	assert(vcl->refcount > 0);
@@ -443,29 +443,29 @@ vcl_set_state(struct vcl *vcl, const char *state)
 
 	switch(state[0]) {
 	case '0':
-		assert(vcl->temp != vcl_temp_cold);
-		if (vcl->busy == 0 && (vcl->temp == vcl_temp_warm ||
-		    vcl->temp == vcl_temp_busy)) {
+		assert(vcl->temp != VCL_TEMP_COLD);
+		if (vcl->busy == 0 && (vcl->temp == VCL_TEMP_WARM ||
+		    vcl->temp == VCL_TEMP_BUSY)) {
 
-			vcl->temp = vcl->refcount ? vcl_temp_cooling :
-			    vcl_temp_cold;
+			vcl->temp = vcl->refcount ? VCL_TEMP_COOLING :
+			    VCL_TEMP_COLD;
 			AZ(vcl->conf->event_vcl(&ctx, VCL_EVENT_COLD));
 			vcl_BackendEvent(vcl, VCL_EVENT_COLD);
 		}
 		else if (vcl->busy)
-			vcl->temp = vcl_temp_busy;
+			vcl->temp = VCL_TEMP_BUSY;
 		else
-			vcl->temp = vcl->refcount ? vcl_temp_cooling :
-			    vcl_temp_cold;
+			vcl->temp = vcl->refcount ? VCL_TEMP_COOLING :
+			    VCL_TEMP_COLD;
 		break;
 	case '1':
-		assert(vcl->temp != vcl_temp_warm);
+		assert(vcl->temp != VCL_TEMP_WARM);
 		/* The warm VCL hasn't seen a cold event yet */
-		if (vcl->temp == vcl_temp_busy)
-			vcl->temp = vcl_temp_warm;
+		if (vcl->temp == VCL_TEMP_BUSY)
+			vcl->temp = VCL_TEMP_WARM;
 		/* The VCL must first reach a stable cold state */
-		else if (vcl->temp != vcl_temp_cooling) {
-			vcl->temp = vcl_temp_warm;
+		else if (vcl->temp != VCL_TEMP_COOLING) {
+			vcl->temp = VCL_TEMP_WARM;
 			(void)vcl->conf->event_vcl(&ctx, VCL_EVENT_WARM);
 			vcl_BackendEvent(vcl, VCL_EVENT_WARM);
 		}
@@ -507,7 +507,7 @@ VCL_Load(struct cli *cli, const char *name, const char *fn, const char *state)
 	XXXAN(vcl->loaded_name);
 	VTAILQ_INIT(&vcl->backend_list);
 
-	vcl->temp = vcl_temp_init;
+	vcl->temp = VCL_TEMP_INIT;
 
 	INIT_OBJ(&ctx, VRT_CTX_MAGIC);
 	ctx.method = VCL_MET_INIT;
@@ -581,10 +581,10 @@ VCL_Poll(void)
 
 	ASSERT_CLI();
 	VTAILQ_FOREACH_SAFE(vcl, &vcl_head, list, vcl2) {
-		if (vcl->temp == vcl_temp_busy ||
-		    vcl->temp == vcl_temp_cooling)
+		if (vcl->temp == VCL_TEMP_BUSY ||
+		    vcl->temp == VCL_TEMP_COOLING)
 			vcl_set_state(vcl, "0");
-		if (vcl->discard && vcl->temp == vcl_temp_cold)
+		if (vcl->discard && vcl->temp == VCL_TEMP_COLD)
 			VCL_Nuke(vcl);
 	}
 }
@@ -672,7 +672,7 @@ ccf_config_use(struct cli *cli, const char * const *av, void *priv)
 	AZ(priv);
 	vcl = vcl_find(av[2]);
 	AN(vcl);				// MGT ensures this
-	assert(vcl->temp == vcl_temp_warm);	// MGT ensures this
+	assert(vcl->temp == VCL_TEMP_WARM);	// MGT ensures this
 	INIT_OBJ(&ctx, VRT_CTX_MAGIC);
 	ctx.handling = &hand;
 	vsb = VSB_new_auto();



From phk at FreeBSD.org  Fri Jan  8 18:52:21 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Fri, 08 Jan 2016 19:52:21 +0100
Subject: [master] 731f514 Add a VFIL facility for searching paths of files.
Message-ID: <E1aHc97-0004Hj-UU@project.varnish-software.com>


commit 731f51487b6f17f9d375f3be004fd08a54a7ecce
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Jan 8 18:48:01 2016 +0000

    Add a VFIL facility for searching paths of files.

diff --git a/include/vfil.h b/include/vfil.h
index 97484ad..4269fe0 100644
--- a/include/vfil.h
+++ b/include/vfil.h
@@ -28,10 +28,16 @@
  *
  */
 
+struct vfil_path;
+
 /* from libvarnish/vfil.c */
 int seed_random(void);
 char *VFIL_readfile(const char *pfx, const char *fn, ssize_t *sz);
-char *VFIL_readfd(int fd, ssize_t *sz);
 int VFIL_nonblocking(int fd);
 int VFIL_fsinfo(int fd, unsigned *pbs, uintmax_t *size, uintmax_t *space);
 int VFIL_allocate(int fd, off_t size, int insist);
+void VFIL_setpath(struct vfil_path**, const char *path);
+typedef int vfil_path_func_f(void *priv, const char *fn);
+int VFIL_searchpath(const struct vfil_path *, vfil_path_func_f *func,
+    void *priv, char **fn);
+
diff --git a/lib/libvarnish/vfil.c b/lib/libvarnish/vfil.c
index 4b57d01..b6bdc9e 100644
--- a/lib/libvarnish/vfil.c
+++ b/lib/libvarnish/vfil.c
@@ -55,12 +55,16 @@
 #  include <linux/magic.h>
 #endif
 
-#include "vas.h"
 #include "vdef.h"
+
+#include "miniobj.h"
+#include "vas.h"
+#include "vsb.h"
 #include "vfil.h"
+#include "vqueue.h"
 
-char *
-VFIL_readfd(int fd, ssize_t *sz)
+static char *
+vfil_readfd(int fd, ssize_t *sz)
 {
 	struct stat st;
 	char *f;
@@ -99,7 +103,7 @@ VFIL_readfile(const char *pfx, const char *fn, ssize_t *sz)
 		fd = open(fn, O_RDONLY);
 	if (fd < 0)
 		return (NULL);
-	r = VFIL_readfd(fd, sz);
+	r = vfil_readfd(fd, sz);
 	err = errno;
 	AZ(close(fd));
 	errno = err;
@@ -158,7 +162,8 @@ VFIL_fsinfo(int fd, unsigned *pbs, uintmax_t *psize, uintmax_t *pspace)
 	return (0);
 }
 
-/* Make sure that the file system can accommodate the file of the given
+/*
+ * Make sure that the file system can accommodate the file of the given
  * size. Will use fallocate if available. If fallocate is not available
  * and insist is true, it will write size zero bytes.
  *
@@ -222,3 +227,117 @@ VFIL_allocate(int fd, off_t size, int insist)
 	assert(lseek(fd, 0, SEEK_SET) == 0);
 	return (0);
 }
+
+struct vfil_dir {
+	unsigned		magic;
+#define VFIL_DIR_MAGIC		0x3e214967
+	char			*dir;
+	VTAILQ_ENTRY(vfil_dir)	list;
+};
+
+struct vfil_path {
+	unsigned		magic;
+#define VFIL_PATH_MAGIC		0x92dbcc31
+	char			*str;
+	VTAILQ_HEAD(,vfil_dir)	paths;
+};
+
+/*
+ * Path searching functions
+ */
+
+void
+VFIL_setpath(struct vfil_path **pp, const char *path)
+{
+	struct vfil_path *vp;
+	struct vfil_dir *vd;
+	char *p, *q;
+
+	AN(pp);
+	AN(path);
+
+	vp = *pp;
+	if (vp == NULL) {
+		ALLOC_OBJ(vp, VFIL_PATH_MAGIC);
+		AN(vp);
+		VTAILQ_INIT(&vp->paths);
+		*pp = vp;
+	}
+	REPLACE(vp->str, path);
+	while (!VTAILQ_EMPTY(&vp->paths)) {
+		vd = VTAILQ_FIRST(&vp->paths);
+		VTAILQ_REMOVE(&vp->paths, vd, list);
+		FREE_OBJ(vd);
+	}
+	for (p = vp->str; p != NULL; p = q) {
+		q = strchr(p, ':');
+		if (q != NULL)
+			*q++ = '\0';
+		ALLOC_OBJ(vd, VFIL_DIR_MAGIC);
+		AN(vd);
+		vd->dir = p;
+		VTAILQ_INSERT_TAIL(&vp->paths, vd, list);
+	}
+}
+
+static int
+vfil_path_openfile(void *priv, const char *fn)
+{
+	char *p, **pp;
+
+	AN(priv);
+	AN(fn);
+	p = VFIL_readfile(NULL, fn, NULL);
+	if (p == NULL)
+		return (1);
+
+	pp = priv;
+	*pp = p;
+	return (0);
+}
+
+int
+VFIL_searchpath(const struct vfil_path *vp, vfil_path_func_f *func, void *priv,
+    char **fnp)
+{
+	struct vsb *vsb;
+	struct vfil_dir *vd;
+	const char *fn;
+	int i, e;
+
+	CHECK_OBJ_NOTNULL(vp, VFIL_PATH_MAGIC);
+	AN(fnp);
+	AN(*fnp);
+	fn = *fnp;
+	*fnp = NULL;
+
+	if (func == NULL) {
+		func = vfil_path_openfile;
+		AN(priv);
+	}
+
+	if (*fn == '/') {
+		i = func(priv, fn);
+		if (i <= 0)
+			REPLACE(*fnp, fn);
+		return (i);
+	}
+	vsb = VSB_new_auto();
+	AN(vsb);
+	VTAILQ_FOREACH(vd, &vp->paths, list) {
+		VSB_clear(vsb);
+		VSB_printf(vsb, "%s/%s", vd->dir, fn);
+		AZ(VSB_finish(vsb));
+		i = func(priv, VSB_data(vsb));
+		if (i <= 0) {
+			e = errno;
+			*fnp = strdup(VSB_data(vsb));
+			AN(*fnp);
+			VSB_delete(vsb);
+			errno = e;
+			return (i);
+		}
+	}
+	VSB_delete(vsb);
+	return (-1);
+}



From phk at FreeBSD.org  Fri Jan  8 18:52:21 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Fri, 08 Jan 2016 19:52:21 +0100
Subject: [master] 3fea946 Let vcl_dir and vmod_dir also take colon separated
 paths.
Message-ID: <E1aHc97-0004Hq-WD@project.varnish-software.com>


commit 3fea94618a7ac2b0f13548ab13794586d4bba4b8
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Jan 8 18:51:44 2016 +0000

    Let vcl_dir and vmod_dir also take colon separated paths.

diff --git a/bin/varnishd/mgt/mgt_param_tbl.c b/bin/varnishd/mgt/mgt_param_tbl.c
index 16886b2..59fcfed 100644
--- a/bin/varnishd/mgt/mgt_param_tbl.c
+++ b/bin/varnishd/mgt/mgt_param_tbl.c
@@ -58,14 +58,16 @@ struct parspec mgt_parspec[] = {
 		VCC_CC , NULL },
 	{ "vcl_dir", tweak_string, &mgt_vcl_dir,
 		NULL, NULL,
-		"Directory from which relative VCL filenames (vcl.load and "
-		"include) are opened.",
+		"Directory (or colon separated list of directories) "
+		"from which relative VCL filenames (vcl.load and "
+		"include) are to be found.",
 		0,
 		VARNISH_VCL_DIR,
 		NULL },
 	{ "vmod_dir", tweak_string, &mgt_vmod_dir,
 		NULL, NULL,
-		"Directory where VCL modules are to be found.",
+		"Directory (or colon separated list of directories) "
+		"where VMODs are to be found.",
 		0,
 		VARNISH_VMOD_DIR,
 		NULL },
diff --git a/bin/varnishtest/tests/m00008.vtc b/bin/varnishtest/tests/m00008.vtc
index 232d3c0..833a282 100644
--- a/bin/varnishtest/tests/m00008.vtc
+++ b/bin/varnishtest/tests/m00008.vtc
@@ -13,12 +13,12 @@ varnish v1 -vcl+backend {
 
 varnish v1 -cliok "param.set vcc_unsafe_path off"
 
-varnish v1 -errvcl {'import ... from path ...' not allowed.} {
+varnish v1 -errvcl {'import ... from path ...' is unsafe.} {
 	backend default { .host = "${s1_sock}"; }
 	import ${vmod_std};
 }
 
-varnish v1 -cliok "param.set vmod_dir ${topbuild}/lib/libvmod_std/.libs/"
+varnish v1 -cliok "param.set vmod_dir /nowhere:${topbuild}/lib/libvmod_std/.libs/:/else"
 
 varnish v1 -vcl+backend {
 	import std;
diff --git a/lib/libvcc/vcc_compile.c b/lib/libvcc/vcc_compile.c
index aaba4bf..c52dd8d 100644
--- a/lib/libvcc/vcc_compile.c
+++ b/lib/libvcc/vcc_compile.c
@@ -450,22 +450,24 @@ vcc_destroy_source(struct source *sp)
 /*--------------------------------------------------------------------*/
 
 static struct source *
-vcc_file_source(const struct vcp * const vcp, struct vsb *sb, const char *fn)
+vcc_file_source(const struct vcp * const vcp, struct vsb *sb, char *fn)
 {
-	char *f;
+	char *f, *fnp;
 	struct source *sp;
 
 	if (!vcp->unsafe_path && strchr(fn, '/') != NULL) {
 		VSB_printf(sb, "Include path is unsafe '%s'\n", fn);
 		return (NULL);
 	}
-	f = VFIL_readfile(vcp->vcl_dir, fn, NULL);
-	if (f == NULL) {
-		VSB_printf(sb, "Cannot read file '%s': %s\n",
-		    fn, strerror(errno));
+	f = NULL;
+	fnp = fn;
+	if (VFIL_searchpath(vcp->vcl_path, NULL, &f, &fnp) || f == NULL) {
+		VSB_printf(sb, "Cannot read file '%s' (%s)\n",
+		    fnp != NULL ? fnp : fn, strerror(errno));
 		return (NULL);
 	}
-	sp = vcc_new_source(f, NULL, fn);
+	sp = vcc_new_source(f, NULL, fnp);
+	free(fnp);
 	sp->freeit = f;
 	return (sp);
 }
@@ -801,6 +803,7 @@ VCP_VCL_dir(struct vcp *vcp, const char *str)
 
 	CHECK_OBJ_NOTNULL(vcp, VCP_MAGIC);
 	REPLACE(vcp->vcl_dir, str);
+	VFIL_setpath(&vcp->vcl_path, str);
 }
 
 /*--------------------------------------------------------------------
@@ -813,6 +816,7 @@ VCP_VMOD_dir(struct vcp *vcp, const char *str)
 
 	CHECK_OBJ_NOTNULL(vcp, VCP_MAGIC);
 	REPLACE(vcp->vmod_dir, str);
+	VFIL_setpath(&vcp->vmod_path, str);
 }
 
 /*--------------------------------------------------------------------
diff --git a/lib/libvcc/vcc_compile.h b/lib/libvcc/vcc_compile.h
index 72ff7b7..3feecd3 100644
--- a/lib/libvcc/vcc_compile.h
+++ b/lib/libvcc/vcc_compile.h
@@ -161,7 +161,9 @@ struct vcp {
 
 	char			*builtin_vcl;
 	char			*vcl_dir;
+	struct vfil_path	*vcl_path;
 	char			*vmod_dir;
+	struct vfil_path	*vmod_path;
 	unsigned		err_unref;
 	unsigned		allow_inline_c;
 	unsigned		unsafe_path;
diff --git a/lib/libvcc/vcc_vmod.c b/lib/libvcc/vcc_vmod.c
index d82ad3e..0a165c7 100644
--- a/lib/libvcc/vcc_vmod.c
+++ b/lib/libvcc/vcc_vmod.c
@@ -30,20 +30,39 @@
 
 #include <dlfcn.h>
 #include <stdio.h>
+#include <stdlib.h>
 #include <string.h>
 
 #include "vcc_compile.h"
 
 #include "vcs_version.h"
 
+#include "vfil.h"
 #include "vmod_abi.h"
 #include "vrt.h"
 
+static int
+vcc_path_dlopen(void *priv, const char *fn)
+{
+	void *hdl, **pp;
+
+	AN(priv);
+	AN(fn);
+
+fprintf(stderr, "TRY <%s>\n", fn);
+	hdl = dlopen(fn, RTLD_NOW | RTLD_LOCAL);
+	if (hdl == NULL)
+		return (1);
+	pp = priv;
+	*pp = hdl;
+	return (0);
+}
+
 void
 vcc_ParseImport(struct vcc *tl)
 {
 	void *hdl;
-	char fn[1024];
+	char fn[1024], *fnp;
 	char buf[256];
 	struct token *mod, *t1;
 	struct inifin *ifp;
@@ -85,32 +104,33 @@ vcc_ParseImport(struct vcc *tl)
 	sym->def_e = tl->t;
 
 	if (tl->t->tok == ID) {
-		if (!tl->param->unsafe_path) {
-			VSB_printf(tl->sb,
-			    "'import ... from path ...' not allowed.\nAt:");
-			vcc_ErrToken(tl, tl->t);
-			vcc_ErrWhere(tl, tl->t);
-			return;
-		}
 		if (!vcc_IdIs(tl->t, "from")) {
 			VSB_printf(tl->sb, "Expected 'from path ...'\n");
 			vcc_ErrWhere(tl, tl->t);
 			return;
 		}
 		vcc_NextToken(tl);
+		if (!tl->param->unsafe_path && strchr(tl->t->dec, '/')) {
+			VSB_printf(tl->sb,
+			    "'import ... from path ...' is unsafe.\nAt:");
+			vcc_ErrToken(tl, tl->t);
+			vcc_ErrWhere(tl, tl->t);
+			return;
+		}
 		ExpectErr(tl, CSTR);
 		bprintf(fn, "%s", tl->t->dec);
 		vcc_NextToken(tl);
 	} else {
-		bprintf(fn, "%s/libvmod_%.*s.so", tl->param->vmod_dir, PF(mod));
+		bprintf(fn, "libvmod_%.*s.so", PF(mod));
 	}
 
 	SkipToken(tl, ';');
 
-	hdl = dlopen(fn, RTLD_NOW | RTLD_LOCAL);
-	if (hdl == NULL) {
+	fnp = fn;
+	if (VFIL_searchpath(tl->param->vmod_path,
+	    vcc_path_dlopen, &hdl, &fnp)) {
 		VSB_printf(tl->sb, "Could not load VMOD %.*s\n", PF(mod));
-		VSB_printf(tl->sb, "\tFile name: %s\n", fn);
+		VSB_printf(tl->sb, "\tFile name: %s\n", fnp != NULL ? fnp : fn);
 		VSB_printf(tl->sb, "\tdlerror: %s\n", dlerror());
 		vcc_ErrWhere(tl, mod);
 		return;
@@ -120,7 +140,7 @@ vcc_ParseImport(struct vcc *tl)
 	vmd = dlsym(hdl, buf);
 	if (vmd == NULL) {
 		VSB_printf(tl->sb, "Malformed VMOD %.*s\n", PF(mod));
-		VSB_printf(tl->sb, "\tFile name: %s\n", fn);
+		VSB_printf(tl->sb, "\tFile name: %s\n", fnp);
 		VSB_printf(tl->sb, "\t(no Vmod_Data symbol)\n");
 		vcc_ErrWhere(tl, mod);
 		return;
@@ -128,7 +148,7 @@ vcc_ParseImport(struct vcc *tl)
 	if (strcmp(VCS_Branch, "master") == 0 &&
 	    strcmp(vmd->abi, VMOD_ABI_Version) != 0) {
 		VSB_printf(tl->sb, "Incompatible VMOD %.*s\n", PF(mod));
-		VSB_printf(tl->sb, "\tFile name: %s\n", fn);
+		VSB_printf(tl->sb, "\tFile name: %s\n", fnp);
 		VSB_printf(tl->sb, "\tABI mismatch, expected <%s>, got <%s>\n",
 			   VMOD_ABI_Version, vmd->abi);
 		vcc_ErrWhere(tl, mod);
@@ -137,7 +157,7 @@ vcc_ParseImport(struct vcc *tl)
 	if (vmd->vrt_major != VRT_MAJOR_VERSION ||
 	    vmd->vrt_minor > VRT_MINOR_VERSION) {
 		VSB_printf(tl->sb, "Incompatible VMOD %.*s\n", PF(mod));
-		VSB_printf(tl->sb, "\tFile name: %s\n", fn);
+		VSB_printf(tl->sb, "\tFile name: %s\n", fnp);
 		VSB_printf(tl->sb, "\tVMOD version %u.%u\n",
 		    vmd->vrt_major, vmd->vrt_minor);
 		VSB_printf(tl->sb, "\tvarnishd version %u.%u\n",
@@ -151,7 +171,7 @@ vcc_ParseImport(struct vcc *tl)
 	    vmd->proto == NULL ||
 	    vmd->abi == NULL) {
 		VSB_printf(tl->sb, "Mangled VMOD %.*s\n", PF(mod));
-		VSB_printf(tl->sb, "\tFile name: %s\n", fn);
+		VSB_printf(tl->sb, "\tFile name: %s\n", fnp);
 		VSB_printf(tl->sb, "\tInconsistent metadata\n");
 		vcc_ErrWhere(tl, mod);
 		return;
@@ -159,7 +179,7 @@ vcc_ParseImport(struct vcc *tl)
 
 	if (!vcc_IdIs(mod, vmd->name)) {
 		VSB_printf(tl->sb, "Wrong VMOD file %.*s\n", PF(mod));
-		VSB_printf(tl->sb, "\tFile name: %s\n", fn);
+		VSB_printf(tl->sb, "\tFile name: %s\n", fnp);
 		VSB_printf(tl->sb, "\tContains vmod \"%s\"\n", vmd->name);
 		vcc_ErrWhere(tl, mod);
 		return;
@@ -172,7 +192,8 @@ vcc_ParseImport(struct vcc *tl)
 	VSB_printf(ifp->ini, "\t    sizeof(Vmod_%.*s_Func),\n", PF(mod));
 	VSB_printf(ifp->ini, "\t    \"%.*s\",\n", PF(mod));
 	VSB_printf(ifp->ini, "\t    ");
-	EncString(ifp->ini, fn, NULL, 0);
+	EncString(ifp->ini, fnp, NULL, 0);
+	free(fnp);
 	VSB_printf(ifp->ini, ",\n");
 	AN(vmd);
 	AN(vmd->file_id);



From phk at FreeBSD.org  Fri Jan  8 19:02:14 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Fri, 08 Jan 2016 20:02:14 +0100
Subject: [master] 5bfbb6c Also make vcl_dir act as path for the vcl.load
 command
Message-ID: <E1aHcIg-0005Lt-IN@project.varnish-software.com>


commit 5bfbb6c91a5737518cbd453805ec75f9cb6c1c01
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Jan 8 19:01:48 2016 +0000

    Also make vcl_dir act as path for the vcl.load command

diff --git a/bin/varnishd/mgt/mgt_vcl.c b/bin/varnishd/mgt/mgt_vcl.c
index 37359f2..7d51690 100644
--- a/bin/varnishd/mgt/mgt_vcl.c
+++ b/bin/varnishd/mgt/mgt_vcl.c
@@ -60,6 +60,7 @@ struct vclprog {
 static VTAILQ_HEAD(, vclprog) vclhead = VTAILQ_HEAD_INITIALIZER(vclhead);
 static struct vclprog		*active_vcl;
 static struct vev *e_poker;
+static struct vfil_path *vcl_path;
 
 /*--------------------------------------------------------------------*/
 
@@ -269,7 +270,7 @@ mcf_vcl_inline(struct cli *cli, const char * const *av, void *priv)
 void
 mcf_vcl_load(struct cli *cli, const char * const *av, void *priv)
 {
-	char *vcl;
+	char *vcl, *fn;
 	struct vclprog *vp;
 
 	(void)priv;
@@ -280,9 +281,10 @@ mcf_vcl_load(struct cli *cli, const char * const *av, void *priv)
 		return;
 	}
 
-	vcl = VFIL_readfile(mgt_vcl_dir, av[3], NULL);
-	if (vcl == NULL) {
-		VCLI_Out(cli, "Cannot open '%s'", av[3]);
+	VFIL_setpath(&vcl_path, mgt_vcl_dir);
+	fn = TRUST_ME(av[3]);
+	if (VFIL_searchpath(vcl_path, NULL, &vcl, &fn)) {
+		VCLI_Out(cli, "Cannot open '%s'", fn != NULL ? fn : av[3]);
 		VCLI_SetResult(cli, CLIS_PARAM);
 		return;
 	}



From phk at FreeBSD.org  Fri Jan  8 22:08:14 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Fri, 08 Jan 2016 23:08:14 +0100
Subject: [master] 0f128f4 rename srcfile to csrcfile,
 it confused me for too long
Message-ID: <E1aHfCg-0000j9-7E@project.varnish-software.com>


commit 0f128f4efb0338bc2849ed9af48a3cc25c2e6825
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Jan 8 20:58:33 2016 +0000

    rename srcfile to csrcfile, it confused me for too long

diff --git a/bin/varnishd/mgt/mgt_vcc.c b/bin/varnishd/mgt/mgt_vcc.c
index 9d28804..c7a641b 100644
--- a/bin/varnishd/mgt/mgt_vcc.c
+++ b/bin/varnishd/mgt/mgt_vcc.c
@@ -53,7 +53,7 @@ struct vcc_priv {
 #define VCC_PRIV_MAGIC	0x70080cb8
 	char		*dir;
 	const char	*src;
-	char		*srcfile;
+	char		*csrcfile;
 	char		*libfile;
 };
 
@@ -109,13 +109,13 @@ run_vcc(void *priv)
 
 	fd = open(VGC_SRC, O_WRONLY|O_TRUNC|O_CREAT, 0600);
 	if (fd < 0) {
-		fprintf(stderr, "VCC cannot open %s", vp->srcfile);
+		fprintf(stderr, "VCC cannot open %s", vp->csrcfile);
 		exit(2);
 	}
 	l = strlen(csrc);
 	i = write(fd, csrc, l);
 	if (i != l) {
-		fprintf(stderr, "VCC cannot write %s", vp->srcfile);
+		fprintf(stderr, "VCC cannot write %s", vp->csrcfile);
 		exit(2);
 	}
 	AZ(close(fd));
@@ -223,7 +223,7 @@ mgt_vcc_compile(struct vcc_priv *vp, struct vsb *sb, int C_flag)
 	char *csrc;
 	unsigned subs;
 
-	if (mgt_vcc_touchfile(vp->srcfile, sb))
+	if (mgt_vcc_touchfile(vp->csrcfile, sb))
 		return (2);
 	if (mgt_vcc_touchfile(vp->libfile, sb))
 		return (2);
@@ -233,7 +233,7 @@ mgt_vcc_compile(struct vcc_priv *vp, struct vsb *sb, int C_flag)
 		return (subs);
 
 	if (C_flag) {
-		csrc = VFIL_readfile(NULL, vp->srcfile, NULL);
+		csrc = VFIL_readfile(NULL, vp->csrcfile, NULL);
 		AN(csrc);
 		VSB_cat(sb, csrc);
 		free(csrc);
@@ -275,14 +275,14 @@ mgt_VccCompile(struct cli *cli, const char *vclname, const char *vclsrc,
 	VSB_clear(sb);
 	VSB_printf(sb, "%s/%s", vp.dir, VGC_SRC);
 	AZ(VSB_finish(sb));
-	vp.srcfile = strdup(VSB_data(sb));
-	AN(vp.srcfile);
+	vp.csrcfile = strdup(VSB_data(sb));
+	AN(vp.csrcfile);
 	VSB_clear(sb);
 
 	VSB_printf(sb, "%s/%s", vp.dir, VGC_LIB);
 	AZ(VSB_finish(sb));
 	vp.libfile = strdup(VSB_data(sb));
-	AN(vp.srcfile);
+	AN(vp.csrcfile);
 	VSB_clear(sb);
 
 	status = mgt_vcc_compile(&vp, sb, C_flag);
@@ -292,8 +292,8 @@ mgt_VccCompile(struct cli *cli, const char *vclname, const char *vclsrc,
 		VCLI_Out(cli, "%s", VSB_data(sb));
 	VSB_delete(sb);
 
-	(void)unlink(vp.srcfile);
-	free(vp.srcfile);
+	(void)unlink(vp.csrcfile);
+	free(vp.csrcfile);
 
 	free(vp.dir);
 



From phk at FreeBSD.org  Fri Jan  8 22:08:14 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Fri, 08 Jan 2016 23:08:14 +0100
Subject: [master] f1d9f82 Sanitize the VFIL_searchpath() caling convention to
 not need throwing away const.
Message-ID: <E1aHfCg-0000jF-92@project.varnish-software.com>


commit f1d9f8270ec7a94f547faa0a1bcdf97b720a6773
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Jan 8 21:14:11 2016 +0000

    Sanitize the VFIL_searchpath() caling convention to not need throwing
    away const.

diff --git a/bin/varnishd/mgt/mgt_vcl.c b/bin/varnishd/mgt/mgt_vcl.c
index 7d51690..ae2f898 100644
--- a/bin/varnishd/mgt/mgt_vcl.c
+++ b/bin/varnishd/mgt/mgt_vcl.c
@@ -282,12 +282,13 @@ mcf_vcl_load(struct cli *cli, const char * const *av, void *priv)
 	}
 
 	VFIL_setpath(&vcl_path, mgt_vcl_dir);
-	fn = TRUST_ME(av[3]);
-	if (VFIL_searchpath(vcl_path, NULL, &vcl, &fn)) {
+	if (VFIL_searchpath(vcl_path, NULL, &vcl, av[3], &fn)) {
 		VCLI_Out(cli, "Cannot open '%s'", fn != NULL ? fn : av[3]);
+		REPLACE(fn, NULL);
 		VCLI_SetResult(cli, CLIS_PARAM);
 		return;
 	}
+	REPLACE(fn, NULL);
 
 	mgt_new_vcl(cli, av[2], vcl, av[4], 0);
 	free(vcl);
diff --git a/include/vfil.h b/include/vfil.h
index 4269fe0..a542e4d 100644
--- a/include/vfil.h
+++ b/include/vfil.h
@@ -39,5 +39,5 @@ int VFIL_allocate(int fd, off_t size, int insist);
 void VFIL_setpath(struct vfil_path**, const char *path);
 typedef int vfil_path_func_f(void *priv, const char *fn);
 int VFIL_searchpath(const struct vfil_path *, vfil_path_func_f *func,
-    void *priv, char **fn);
+    void *priv, const char *fni, char **fno);
 
diff --git a/lib/libvarnish/vfil.c b/lib/libvarnish/vfil.c
index b6bdc9e..319c3e1 100644
--- a/lib/libvarnish/vfil.c
+++ b/lib/libvarnish/vfil.c
@@ -298,41 +298,38 @@ vfil_path_openfile(void *priv, const char *fn)
 
 int
 VFIL_searchpath(const struct vfil_path *vp, vfil_path_func_f *func, void *priv,
-    char **fnp)
+    const char *fni, char **fno)
 {
 	struct vsb *vsb;
 	struct vfil_dir *vd;
-	const char *fn;
 	int i, e;
 
 	CHECK_OBJ_NOTNULL(vp, VFIL_PATH_MAGIC);
-	AN(fnp);
-	AN(*fnp);
-	fn = *fnp;
-	*fnp = NULL;
+	AN(fno);
+	*fno = NULL;
 
 	if (func == NULL) {
 		func = vfil_path_openfile;
 		AN(priv);
 	}
 
-	if (*fn == '/') {
-		i = func(priv, fn);
+	if (*fni == '/') {
+		i = func(priv, fni);
 		if (i <= 0)
-			REPLACE(*fnp, fn);
+			REPLACE(*fno, fni);
 		return (i);
 	}
 	vsb = VSB_new_auto();
 	AN(vsb);
 	VTAILQ_FOREACH(vd, &vp->paths, list) {
 		VSB_clear(vsb);
-		VSB_printf(vsb, "%s/%s", vd->dir, fn);
+		VSB_printf(vsb, "%s/%s", vd->dir, fni);
 		AZ(VSB_finish(vsb));
 		i = func(priv, VSB_data(vsb));
 		if (i <= 0) {
 			e = errno;
-			*fnp = strdup(VSB_data(vsb));
-			AN(*fnp);
+			*fno = strdup(VSB_data(vsb));
+			AN(*fno);
 			VSB_delete(vsb);
 			errno = e;
 			return (i);
diff --git a/lib/libvcc/vcc_compile.c b/lib/libvcc/vcc_compile.c
index c52dd8d..f35e07b 100644
--- a/lib/libvcc/vcc_compile.c
+++ b/lib/libvcc/vcc_compile.c
@@ -450,7 +450,7 @@ vcc_destroy_source(struct source *sp)
 /*--------------------------------------------------------------------*/
 
 static struct source *
-vcc_file_source(const struct vcp * const vcp, struct vsb *sb, char *fn)
+vcc_file_source(const struct vcp * const vcp, struct vsb *sb, const char *fn)
 {
 	char *f, *fnp;
 	struct source *sp;
@@ -460,8 +460,7 @@ vcc_file_source(const struct vcp * const vcp, struct vsb *sb, char *fn)
 		return (NULL);
 	}
 	f = NULL;
-	fnp = fn;
-	if (VFIL_searchpath(vcp->vcl_path, NULL, &f, &fnp) || f == NULL) {
+	if (VFIL_searchpath(vcp->vcl_path, NULL, &f, fn, &fnp) || f == NULL) {
 		VSB_printf(sb, "Cannot read file '%s' (%s)\n",
 		    fnp != NULL ? fnp : fn, strerror(errno));
 		return (NULL);
diff --git a/lib/libvcc/vcc_vmod.c b/lib/libvcc/vcc_vmod.c
index 0a165c7..0f90e4f 100644
--- a/lib/libvcc/vcc_vmod.c
+++ b/lib/libvcc/vcc_vmod.c
@@ -49,7 +49,6 @@ vcc_path_dlopen(void *priv, const char *fn)
 	AN(priv);
 	AN(fn);
 
-fprintf(stderr, "TRY <%s>\n", fn);
 	hdl = dlopen(fn, RTLD_NOW | RTLD_LOCAL);
 	if (hdl == NULL)
 		return (1);
@@ -126,9 +125,8 @@ vcc_ParseImport(struct vcc *tl)
 
 	SkipToken(tl, ';');
 
-	fnp = fn;
 	if (VFIL_searchpath(tl->param->vmod_path,
-	    vcc_path_dlopen, &hdl, &fnp)) {
+	    vcc_path_dlopen, &hdl, fn, &fnp)) {
 		VSB_printf(tl->sb, "Could not load VMOD %.*s\n", PF(mod));
 		VSB_printf(tl->sb, "\tFile name: %s\n", fnp != NULL ? fnp : fn);
 		VSB_printf(tl->sb, "\tdlerror: %s\n", dlerror());



From phk at FreeBSD.org  Fri Jan  8 22:08:14 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Fri, 08 Jan 2016 23:08:14 +0100
Subject: [master] 35f82e2 Pass either vclsrc or vclfilename to the VCC
 compiler,
 and eliminate the VCL fileopening logic in mgt so we have it only one place.
Message-ID: <E1aHfCg-0000jP-Aw@project.varnish-software.com>


commit 35f82e2b25fca4f979faa3b1bfab829096d94953
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Jan 8 21:35:13 2016 +0000

    Pass either vclsrc or vclfilename to the VCC compiler, and eliminate
    the VCL fileopening logic in mgt so we have it only one place.

diff --git a/bin/varnishd/mgt/mgt.h b/bin/varnishd/mgt/mgt.h
index 5fb12d4..a31bc6a 100644
--- a/bin/varnishd/mgt/mgt.h
+++ b/bin/varnishd/mgt/mgt.h
@@ -161,7 +161,7 @@ void STV_Config_Transient(void);
 
 /* mgt_vcc.c */
 char *mgt_VccCompile(struct cli *, const char *vclname, const char *vclsrc,
-    int C_flag);
+    const char *vclsrcfile, int C_flag);
 void mgt_vcc_init(void);
 
 void mgt_vcl_init(void);
diff --git a/bin/varnishd/mgt/mgt_vcc.c b/bin/varnishd/mgt/mgt_vcc.c
index c7a641b..77f67c0 100644
--- a/bin/varnishd/mgt/mgt_vcc.c
+++ b/bin/varnishd/mgt/mgt_vcc.c
@@ -52,7 +52,8 @@ struct vcc_priv {
 	unsigned	magic;
 #define VCC_PRIV_MAGIC	0x70080cb8
 	char		*dir;
-	const char	*src;
+	const char	*vclsrc;
+	const char	*vclsrcfile;
 	char		*csrcfile;
 	char		*libfile;
 };
@@ -99,7 +100,7 @@ run_vcc(void *priv)
 	VCP_Err_Unref(vcp, mgt_vcc_err_unref);
 	VCP_Allow_InlineC(vcp, mgt_vcc_allow_inline_c);
 	VCP_Unsafe_Path(vcp, mgt_vcc_unsafe_path);
-	csrc = VCC_Compile(vcp, sb, vp->src);
+	csrc = VCC_Compile(vcp, sb, vp->vclsrc, vp->vclsrcfile);
 	AZ(VSB_finish(sb));
 	if (VSB_len(sb))
 		printf("%s", VSB_data(sb));
@@ -251,7 +252,7 @@ mgt_vcc_compile(struct vcc_priv *vp, struct vsb *sb, int C_flag)
 
 char *
 mgt_VccCompile(struct cli *cli, const char *vclname, const char *vclsrc,
-    int C_flag)
+    const char *vclsrcfile, int C_flag)
 {
 	struct vcc_priv vp;
 	struct vsb *sb;
@@ -263,7 +264,8 @@ mgt_VccCompile(struct cli *cli, const char *vclname, const char *vclsrc,
 	XXXAN(sb);
 
 	INIT_OBJ(&vp, VCC_PRIV_MAGIC);
-	vp.src = vclsrc;
+	vp.vclsrc = vclsrc;
+	vp.vclsrcfile = vclsrcfile;
 
 	VSB_printf(sb, "vcl_%s", vclname);
 	AZ(VSB_finish(sb));
@@ -271,7 +273,6 @@ mgt_VccCompile(struct cli *cli, const char *vclname, const char *vclsrc,
 	AN(vp.dir);
 	VJ_make_vcldir(vp.dir);
 
-
 	VSB_clear(sb);
 	VSB_printf(sb, "%s/%s", vp.dir, VGC_SRC);
 	AZ(VSB_finish(sb));
diff --git a/bin/varnishd/mgt/mgt_vcl.c b/bin/varnishd/mgt/mgt_vcl.c
index ae2f898..0b24c65 100644
--- a/bin/varnishd/mgt/mgt_vcl.c
+++ b/bin/varnishd/mgt/mgt_vcl.c
@@ -43,7 +43,6 @@
 #include "vcli.h"
 #include "vcli_priv.h"
 #include "vev.h"
-#include "vfil.h"
 #include "vtim.h"
 
 #include "mgt_cli.h"
@@ -60,7 +59,6 @@ struct vclprog {
 static VTAILQ_HEAD(, vclprog) vclhead = VTAILQ_HEAD_INITIALIZER(vclhead);
 static struct vclprog		*active_vcl;
 static struct vev *e_poker;
-static struct vfil_path *vcl_path;
 
 /*--------------------------------------------------------------------*/
 
@@ -160,7 +158,7 @@ mgt_vcl_setstate(struct vclprog *vp, int warm)
 
 static void
 mgt_new_vcl(struct cli *cli, const char *vclname, const char *vclsrc,
-    const char *state, int C_flag)
+    const char *vclsrcfile, const char *state, int C_flag)
 {
 	unsigned status;
 	char *lib, *p;
@@ -178,7 +176,7 @@ mgt_new_vcl(struct cli *cli, const char *vclname, const char *vclsrc,
 		return;
 	}
 
-	lib = mgt_VccCompile(cli, vclname, vclsrc, C_flag);
+	lib = mgt_VccCompile(cli, vclname, vclsrc, vclsrcfile, C_flag);
 	if (lib == NULL)
 		return;
 
@@ -210,7 +208,7 @@ mgt_vcc_default(struct cli *cli, const char *b_arg, const char *vclsrc,
 
 	if (b_arg == NULL) {
 		AN(vclsrc);
-		mgt_new_vcl(cli, "boot", vclsrc, NULL, C_flag);
+		mgt_new_vcl(cli, "boot", vclsrc, NULL, NULL, C_flag);
 		return;
 	}
 
@@ -220,7 +218,7 @@ mgt_vcc_default(struct cli *cli, const char *b_arg, const char *vclsrc,
 	    "backend default {\n"
 	    "    .host = \"%s\";\n"
 	    "}\n", b_arg);
-	mgt_new_vcl(cli, "boot", buf, NULL, C_flag);
+	mgt_new_vcl(cli, "boot", buf, NULL, NULL, C_flag);
 }
 
 /*--------------------------------------------------------------------*/
@@ -264,13 +262,12 @@ mcf_vcl_inline(struct cli *cli, const char * const *av, void *priv)
 		return;
 	}
 
-	mgt_new_vcl(cli, av[2], av[3], av[4], 0);
+	mgt_new_vcl(cli, av[2], av[3], NULL, av[4], 0);
 }
 
 void
 mcf_vcl_load(struct cli *cli, const char * const *av, void *priv)
 {
-	char *vcl, *fn;
 	struct vclprog *vp;
 
 	(void)priv;
@@ -281,17 +278,7 @@ mcf_vcl_load(struct cli *cli, const char * const *av, void *priv)
 		return;
 	}
 
-	VFIL_setpath(&vcl_path, mgt_vcl_dir);
-	if (VFIL_searchpath(vcl_path, NULL, &vcl, av[3], &fn)) {
-		VCLI_Out(cli, "Cannot open '%s'", fn != NULL ? fn : av[3]);
-		REPLACE(fn, NULL);
-		VCLI_SetResult(cli, CLIS_PARAM);
-		return;
-	}
-	REPLACE(fn, NULL);
-
-	mgt_new_vcl(cli, av[2], vcl, av[4], 0);
-	free(vcl);
+	mgt_new_vcl(cli, av[2], NULL, av[3], av[4], 0);
 }
 
 static struct vclprog *
diff --git a/bin/varnishtest/tests/c00053.vtc b/bin/varnishtest/tests/c00053.vtc
index 9b87fa1..6a34045 100644
--- a/bin/varnishtest/tests/c00053.vtc
+++ b/bin/varnishtest/tests/c00053.vtc
@@ -13,7 +13,7 @@ varnish v1 -vcl+backend {
 
 varnish v1 -cliok "param.set vcc_unsafe_path off"
 
-varnish v1 -errvcl {Include path is unsafe} {
+varnish v1 -errvcl {' is unsafe} {
 	backend default {
 		.host = "${s1_sock}";
 	}
diff --git a/include/libvcc.h b/include/libvcc.h
index 696123e..fbd7bd1 100644
--- a/include/libvcc.h
+++ b/include/libvcc.h
@@ -38,4 +38,5 @@ void VCP_Err_Unref(struct vcp *tl, unsigned u);
 void VCP_Allow_InlineC(struct vcp *tl, unsigned u);
 void VCP_Unsafe_Path(struct vcp *tl, unsigned u);
 
-char *VCC_Compile(const struct vcp *, struct vsb *sb, const char *b);
+char *VCC_Compile(const struct vcp *, struct vsb *sb,
+    const char *vclsrc, const char *vclsrcfile);
diff --git a/lib/libvcc/vcc_compile.c b/lib/libvcc/vcc_compile.c
index f35e07b..95eab85 100644
--- a/lib/libvcc/vcc_compile.c
+++ b/lib/libvcc/vcc_compile.c
@@ -456,7 +456,7 @@ vcc_file_source(const struct vcp * const vcp, struct vsb *sb, const char *fn)
 	struct source *sp;
 
 	if (!vcp->unsafe_path && strchr(fn, '/') != NULL) {
-		VSB_printf(sb, "Include path is unsafe '%s'\n", fn);
+		VSB_printf(sb, "VCL filename '%s' is unsafe.\n", fn);
 		return (NULL);
 	}
 	f = NULL;
@@ -753,12 +753,19 @@ vcc_CompileSource(const struct vcp * const vcp, struct vsb *sb,
  */
 
 char *
-VCC_Compile(const struct vcp *vcp, struct vsb *sb, const char *b)
+VCC_Compile(const struct vcp *vcp, struct vsb *sb,
+    const char *vclsrc, const char *vclsrcfile)
 {
 	struct source *sp;
 	char *r;
 
-	sp = vcc_new_source(b, NULL, "input");
+	if (vclsrc != NULL) {
+		AZ(vclsrcfile);
+		sp = vcc_new_source(vclsrc, NULL, "input");
+	} else {
+		AN(vclsrcfile);
+		sp = vcc_file_source(vcp, sb, vclsrcfile);
+	}
 	if (sp == NULL)
 		return (NULL);
 	r = vcc_CompileSource(vcp, sb, sp);



From phk at FreeBSD.org  Fri Jan  8 22:08:14 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Fri, 08 Jan 2016 23:08:14 +0100
Subject: [master] 1607872 Make nested includes starting with "./" relative to
 the including VCL file, and illegal everywhere else.
Message-ID: <E1aHfCg-0000jZ-CV@project.varnish-software.com>


commit 160787269e7efb11a6bc124db30da35043341402
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Jan 8 22:06:26 2016 +0000

    Make nested includes starting with "./" relative to the including
    VCL file, and illegal everywhere else.
    
    This makes it easier to distribute "VCL-packages"
    
    Test-case written by:	Kacper

diff --git a/bin/varnishtest/tests/v00046.vtc b/bin/varnishtest/tests/v00046.vtc
new file mode 100644
index 0000000..567d3af
--- /dev/null
+++ b/bin/varnishtest/tests/v00046.vtc
@@ -0,0 +1,73 @@
+varnishtest "Test relative to vcl_dir, dot-include and absolute includes"
+
+# relative plain
+shell "true > ${tmpdir}/_start.vcl"
+varnish v1 -arg "-p vcl_dir=${tmpdir}" -vcl {
+	backend b { .host = "127.0.0.1"; }
+	include "_start.vcl" ;
+}
+
+# absolute include
+varnish v1 -vcl {
+	backend b { .host = "127.0.0.1"; }
+	include "${tmpdir}/_start.vcl" ;
+}
+
+# absolute -> relative include
+shell "mkdir -p ${tmpdir}/1/2/3"
+shell "true  > ${tmpdir}/1/2/b.vcl"
+shell "echo 'include \"./2/b.vcl\";' > ${tmpdir}/1/a.vcl"
+varnish v1 -vcl {
+	backend b { .host = "127.0.0.1"; }
+	include "${tmpdir}/1/a.vcl" ;
+}
+
+# same but relative to vcl_dir
+shell "echo 'include \"1/2/b.vcl\";' > ${tmpdir}/1/ab.vcl"
+varnish v1 -vcl {
+	backend b { .host = "127.0.0.1"; }
+	include "1/ab.vcl" ;
+}
+
+# dot-relative -> relative
+varnish v1 -vcl {
+	backend b { .host = "127.0.0.1"; }
+	include "1/a.vcl" ;
+}
+
+# relative -> relative -> relative
+shell "echo 'include \"./3/c.vcl\";' > ${tmpdir}/1/2/b.vcl"
+shell "true  > ${tmpdir}/1/2/3/c.vcl"
+varnish v1 -vcl {
+	backend b { .host = "127.0.0.1"; }
+	include "1/a.vcl" ;
+}
+
+# relative -> absolute
+shell "echo 'include \"${tmpdir}/1/2/3/c.vcl\";' > ${tmpdir}/1/aa.vcl"
+varnish v1 -vcl {
+	backend b { .host = "127.0.0.1"; }
+	include "1/aa.vcl" ;
+}
+
+# relative -> absolute -> relative
+shell "echo 'include \"${tmpdir}/1/2/b.vcl\";' > ${tmpdir}/1/aaa.vcl"
+varnish v1 -vcl {
+	backend b { .host = "127.0.0.1"; }
+	include "1/aaa.vcl" ;
+}
+
+# includes and parses out
+shell "echo 'zool' > ${tmpdir}/1/2/3/c.vcl"
+varnish v1 -errvcl {Found: 'zool' at} {
+	backend b { .host = "127.0.0.1"; }
+	include "1/a.vcl";
+}
+
+shell "rm -f ${tmpdir}/a"
+shell "rm -f ${tmpdir}/_start.vcl"
+
+varnish v1 -errvcl {only works in nested VCL include files} {
+	include "./foobar";
+}
+
diff --git a/lib/libvcc/vcc_compile.c b/lib/libvcc/vcc_compile.c
index 95eab85..8bc60ed 100644
--- a/lib/libvcc/vcc_compile.c
+++ b/lib/libvcc/vcc_compile.c
@@ -478,6 +478,8 @@ vcc_resolve_includes(struct vcc *tl)
 {
 	struct token *t, *t1, *t2;
 	struct source *sp;
+	struct vsb *vsb;
+	const char *p;
 
 	VTAILQ_FOREACH(t, &tl->tokens, list) {
 		if (t->tok != ID || !vcc_IdIs(t, "include"))
@@ -501,7 +503,31 @@ vcc_resolve_includes(struct vcc *tl)
 			return;
 		}
 
-		sp = vcc_file_source(tl->param, tl->sb, t1->dec);
+		if (t1->dec[0] == '.' && t1->dec[1] == '/') {
+			/*
+			 * Nested include filenames, starting with "./" are
+			 * resolved relative to the VCL file which contains
+			 * the include directive.
+			 */
+			if (t1->src->name[0] != '/') {
+				VSB_printf(tl->sb,
+				    "include \"./xxxxx\"; only works in "
+				    "nested VCL include files\n");
+				vcc_ErrWhere(tl, t1);
+				return;
+			}
+			vsb = VSB_new_auto();
+			AN(vsb);
+			p = strrchr(t1->src->name, '/');
+			AN(p);
+			VSB_bcat(vsb, t1->src->name, p - t1->src->name);
+			VSB_cat(vsb, t1->dec + 1);
+			AZ(VSB_finish(vsb));
+			sp = vcc_file_source(tl->param, tl->sb, VSB_data(vsb));
+			VSB_delete(vsb);
+		} else {
+			sp = vcc_file_source(tl->param, tl->sb, t1->dec);
+		}
 		if (sp == NULL) {
 			vcc_ErrWhere(tl, t1);
 			return;
@@ -761,7 +787,7 @@ VCC_Compile(const struct vcp *vcp, struct vsb *sb,
 
 	if (vclsrc != NULL) {
 		AZ(vclsrcfile);
-		sp = vcc_new_source(vclsrc, NULL, "input");
+		sp = vcc_new_source(vclsrc, NULL, "<input>");
 	} else {
 		AN(vclsrcfile);
 		sp = vcc_file_source(vcp, sb, vclsrcfile);



From phk at FreeBSD.org  Fri Jan  8 22:33:21 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Fri, 08 Jan 2016 23:33:21 +0100
Subject: [master] 6564a0c Fix up a couple of straggling issues with include
 "./xxx";
 If we know the filename of the -f argument, it should be possible to include
 "./" relative to it.
Message-ID: <E1aHfaz-0002nw-U7@project.varnish-software.com>


commit 6564a0cf0eb882ba4bada46b555352cac789b462
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Jan 8 22:31:49 2016 +0000

    Fix up a couple of straggling issues with include "./xxx";  If we
    know the filename of the -f argument, it should be possible to
    include "./" relative to it.
    
    The footnote is that the include is resolved at lower privlevel
    than the -f file is read, so in really obscure cases you could
    have really obscure error messages.

diff --git a/bin/varnishd/mgt/mgt.h b/bin/varnishd/mgt/mgt.h
index a31bc6a..1aeba55 100644
--- a/bin/varnishd/mgt/mgt.h
+++ b/bin/varnishd/mgt/mgt.h
@@ -165,8 +165,8 @@ char *mgt_VccCompile(struct cli *, const char *vclname, const char *vclsrc,
 void mgt_vcc_init(void);
 
 void mgt_vcl_init(void);
-void mgt_vcc_default(struct cli *, const char *b_arg, const char *vclsrc,
-    int Cflag);
+void mgt_vcc_startup(struct cli *, const char *b_arg, const char *f_arg,
+    const char *vclsrc, int Cflag);
 int mgt_push_vcls_and_start(unsigned *status, char **p);
 int mgt_has_vcl(void);
 extern char *mgt_cc_cmd;
diff --git a/bin/varnishd/mgt/mgt_main.c b/bin/varnishd/mgt/mgt_main.c
index a7533e9..dab70fb 100644
--- a/bin/varnishd/mgt/mgt_main.c
+++ b/bin/varnishd/mgt/mgt_main.c
@@ -752,7 +752,7 @@ main(int argc, char * const *argv)
 	mgt_vcl_init();
 
 	if (b_arg != NULL || f_arg != NULL) {
-		mgt_vcc_default(cli, b_arg, vcl, C_flag);
+		mgt_vcc_startup(cli, b_arg, f_arg, vcl, C_flag);
 		if (C_flag && cli->result == CLIS_OK) {
 			AZ(VSB_finish(cli->sb));
 			fprintf(stderr, "%s\n", VSB_data(cli->sb));
diff --git a/bin/varnishd/mgt/mgt_vcl.c b/bin/varnishd/mgt/mgt_vcl.c
index 0b24c65..498c0b8 100644
--- a/bin/varnishd/mgt/mgt_vcl.c
+++ b/bin/varnishd/mgt/mgt_vcl.c
@@ -201,14 +201,15 @@ mgt_new_vcl(struct cli *cli, const char *vclname, const char *vclsrc,
 /*--------------------------------------------------------------------*/
 
 void
-mgt_vcc_default(struct cli *cli, const char *b_arg, const char *vclsrc,
-    int C_flag)
+mgt_vcc_startup(struct cli *cli, const char *b_arg, const char *f_arg,
+    const char *vclsrc, int C_flag)
 {
 	char buf[BUFSIZ];
 
 	if (b_arg == NULL) {
 		AN(vclsrc);
-		mgt_new_vcl(cli, "boot", vclsrc, NULL, NULL, C_flag);
+		AN(f_arg);
+		mgt_new_vcl(cli, "boot", vclsrc, f_arg, NULL, C_flag);
 		return;
 	}
 
@@ -218,7 +219,7 @@ mgt_vcc_default(struct cli *cli, const char *b_arg, const char *vclsrc,
 	    "backend default {\n"
 	    "    .host = \"%s\";\n"
 	    "}\n", b_arg);
-	mgt_new_vcl(cli, "boot", buf, NULL, NULL, C_flag);
+	mgt_new_vcl(cli, "boot", buf, "<-b argument>", NULL, C_flag);
 }
 
 /*--------------------------------------------------------------------*/
@@ -262,7 +263,7 @@ mcf_vcl_inline(struct cli *cli, const char * const *av, void *priv)
 		return;
 	}
 
-	mgt_new_vcl(cli, av[2], av[3], NULL, av[4], 0);
+	mgt_new_vcl(cli, av[2], av[3], "<vcl.inline>", av[4], 0);
 }
 
 void



From phk at FreeBSD.org  Fri Jan  8 23:08:51 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Sat, 09 Jan 2016 00:08:51 +0100
Subject: [master] d9a6488 Make it possible to do
Message-ID: <E1aHg9L-0005IP-Vn@project.varnish-software.com>


commit d9a6488dbee9ac8170e22c4444f532e2024bad1c
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Jan 8 23:08:20 2016 +0000

    Make it possible to do
    
    	import vmod from "/some/dir/"
    
    and have the default vmod shlib filename appended.

diff --git a/bin/varnishtest/tests/m00008.vtc b/bin/varnishtest/tests/m00008.vtc
index 833a282..10cb23d 100644
--- a/bin/varnishtest/tests/m00008.vtc
+++ b/bin/varnishtest/tests/m00008.vtc
@@ -23,3 +23,11 @@ varnish v1 -cliok "param.set vmod_dir /nowhere:${topbuild}/lib/libvmod_std/.libs
 varnish v1 -vcl+backend {
 	import std;
 }
+
+varnish v1 -cliok "param.set vcc_unsafe_path on"
+
+varnish v1 -cliok "param.set vmod_dir /nowhere:/else"
+
+varnish v1 -vcl+backend {
+	import std from "${topbuild}/lib/libvmod_std/.libs/";
+}
diff --git a/bin/varnishtest/tests/v00046.vtc b/bin/varnishtest/tests/v00046.vtc
index 567d3af..f87cd21 100644
--- a/bin/varnishtest/tests/v00046.vtc
+++ b/bin/varnishtest/tests/v00046.vtc
@@ -67,7 +67,7 @@ varnish v1 -errvcl {Found: 'zool' at} {
 shell "rm -f ${tmpdir}/a"
 shell "rm -f ${tmpdir}/_start.vcl"
 
-varnish v1 -errvcl {only works in nested VCL include files} {
+varnish v1 -errvcl {needs absolute filename of including file.} {
 	include "./foobar";
 }
 
diff --git a/lib/libvcc/vcc_compile.c b/lib/libvcc/vcc_compile.c
index 8bc60ed..62aaeae 100644
--- a/lib/libvcc/vcc_compile.c
+++ b/lib/libvcc/vcc_compile.c
@@ -511,8 +511,8 @@ vcc_resolve_includes(struct vcc *tl)
 			 */
 			if (t1->src->name[0] != '/') {
 				VSB_printf(tl->sb,
-				    "include \"./xxxxx\"; only works in "
-				    "nested VCL include files\n");
+				    "include \"./xxxxx\"; needs absolute "
+				    "filename of including file.\n");
 				vcc_ErrWhere(tl, t1);
 				return;
 			}
@@ -784,14 +784,14 @@ VCC_Compile(const struct vcp *vcp, struct vsb *sb,
 {
 	struct source *sp;
 	char *r;
+	CHECK_OBJ_NOTNULL(vcp, VCP_MAGIC);
+	AN(sb);
+	AN(vclsrcfile);
 
-	if (vclsrc != NULL) {
-		AZ(vclsrcfile);
-		sp = vcc_new_source(vclsrc, NULL, "<input>");
-	} else {
-		AN(vclsrcfile);
+	if (vclsrc != NULL)
+		sp = vcc_new_source(vclsrc, NULL, vclsrcfile);
+	else
 		sp = vcc_file_source(vcp, sb, vclsrcfile);
-	}
 	if (sp == NULL)
 		return (NULL);
 	r = vcc_CompileSource(vcp, sb, sp);
diff --git a/lib/libvcc/vcc_vmod.c b/lib/libvcc/vcc_vmod.c
index 0f90e4f..d1b8f26 100644
--- a/lib/libvcc/vcc_vmod.c
+++ b/lib/libvcc/vcc_vmod.c
@@ -117,7 +117,11 @@ vcc_ParseImport(struct vcc *tl)
 			return;
 		}
 		ExpectErr(tl, CSTR);
-		bprintf(fn, "%s", tl->t->dec);
+		p = strrchr(tl->t->dec, '/');
+		if (p != NULL && p[1] == '\0')
+			bprintf(fn, "%slibvmod_%.*s.so", tl->t->dec, PF(mod));
+		else
+			bprintf(fn, "%s", tl->t->dec);
 		vcc_NextToken(tl);
 	} else {
 		bprintf(fn, "libvmod_%.*s.so", PF(mod));



From dridi.boukelmoune at gmail.com  Mon Jan 11 09:44:53 2016
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Mon, 11 Jan 2016 10:44:53 +0100
Subject: [master] 524dbe0 Turn VCL state magic numbers into an enum
Message-ID: <E1aIZ1x-0003lS-2P@project.varnish-software.com>


commit 524dbe008ecde9c5397b51b1c56a2ee5cd57ca89
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Fri Dec 4 09:48:11 2015 +0100

    Turn VCL state magic numbers into an enum
    
    Make a clear distinction between (struct vclprog).warm that is used as a
    boolean, unlike the second parameter of mgt_vcl_setstate. We don't use
    an actual C enum but const char[] to make debugging easier.

diff --git a/bin/varnishd/mgt/mgt_vcl.c b/bin/varnishd/mgt/mgt_vcl.c
index 498c0b8..3766eb8 100644
--- a/bin/varnishd/mgt/mgt_vcl.c
+++ b/bin/varnishd/mgt/mgt_vcl.c
@@ -47,11 +47,15 @@
 
 #include "mgt_cli.h"
 
+static const char * const VCL_STATE_COLD = "cold";
+static const char * const VCL_STATE_WARM = "warm";
+static const char * const VCL_STATE_AUTO = "auto";
+
 struct vclprog {
 	VTAILQ_ENTRY(vclprog)	list;
 	char			*name;
 	char			*fname;
-	int			warm;
+	unsigned		warm;
 	char			state[8];
 	double			go_cold;
 };
@@ -119,21 +123,24 @@ mgt_has_vcl(void)
 }
 
 static void
-mgt_vcl_setstate(struct vclprog *vp, int warm)
+mgt_vcl_setstate(struct vclprog *vp, const char *vs)
 {
-	unsigned status;
+	unsigned status, warm;
 	double now;
 	char *p;
 
-	if (warm == -1) {
+	if (vs == VCL_STATE_AUTO) {
 		assert(vp != active_vcl);
 		now = VTIM_mono();
-		warm = vp->warm;
+		vs = vp->warm ? VCL_STATE_WARM : VCL_STATE_COLD;
 		if (vp->go_cold > 0 && !strcmp(vp->state, "auto") &&
 		    vp->go_cold + mgt_param.vcl_cooldown < now)
-			warm = 0;
+			vs = VCL_STATE_COLD;
 	}
 
+	assert(vs != VCL_STATE_AUTO);
+	warm = vs == VCL_STATE_WARM ? 1 : 0;
+
 	if (vp->warm == warm)
 		return;
 
@@ -230,7 +237,7 @@ mgt_push_vcls_and_start(unsigned *status, char **p)
 	struct vclprog *vp;
 
 	AN(active_vcl);
-	mgt_vcl_setstate(active_vcl, 1);
+	mgt_vcl_setstate(active_vcl, VCL_STATE_WARM);
 	VTAILQ_FOREACH(vp, &vclhead, list) {
 		if (mgt_cli_askchild(status, p, "vcl.load \"%s\" %s %d%s\n",
 		    vp->name, vp->fname, vp->warm, vp->state))
@@ -312,7 +319,7 @@ mcf_vcl_state(struct cli *cli, const char * const *av, void *priv)
 		bprintf(vp->state, "%s", "auto");
 		if (vp != active_vcl) {
 			vp->go_cold = VTIM_mono();
-			mgt_vcl_setstate(vp, -1);
+			mgt_vcl_setstate(vp, VCL_STATE_AUTO);
 		}
 	} else if (!strcmp(av[3], "cold")) {
 		if (vp == active_vcl) {
@@ -321,10 +328,10 @@ mcf_vcl_state(struct cli *cli, const char * const *av, void *priv)
 			return;
 		}
 		bprintf(vp->state, "%s", "auto");
-		mgt_vcl_setstate(vp, 0);
+		mgt_vcl_setstate(vp, VCL_STATE_COLD);
 	} else if (!strcmp(av[3], "warm")) {
 		bprintf(vp->state, "%s", av[3]);
-		mgt_vcl_setstate(vp, 1);
+		mgt_vcl_setstate(vp, VCL_STATE_WARM);
 	} else {
 		VCLI_Out(cli, "State must be one of auto, cold or warm.");
 		VCLI_SetResult(cli, CLIS_PARAM);
@@ -344,20 +351,20 @@ mcf_vcl_use(struct cli *cli, const char * const *av, void *priv)
 		return;
 	if (vp == active_vcl)
 		return;
-	mgt_vcl_setstate(vp, 1);
+	mgt_vcl_setstate(vp, VCL_STATE_WARM);
 	if (child_pid >= 0 &&
 	    mgt_cli_askchild(&status, &p, "vcl.use %s\n", av[2])) {
 		VCLI_SetResult(cli, status);
 		VCLI_Out(cli, "%s", p);
 		vp->go_cold = VTIM_mono();
-		mgt_vcl_setstate(vp, -1);
+		mgt_vcl_setstate(vp, VCL_STATE_AUTO);
 	} else {
 		VCLI_Out(cli, "VCL '%s' now active", av[2]);
 		vp2 = active_vcl;
 		active_vcl = vp;
 		if (vp2 != NULL) {
 			vp2->go_cold = VTIM_mono();
-			mgt_vcl_setstate(vp2, -1);
+			mgt_vcl_setstate(vp2, VCL_STATE_AUTO);
 		}
 	}
 	free(p);
@@ -379,7 +386,7 @@ mcf_vcl_discard(struct cli *cli, const char * const *av, void *priv)
 		VCLI_Out(cli, "Cannot discard active VCL program\n");
 		return;
 	}
-	mgt_vcl_setstate(vp, 0);
+	mgt_vcl_setstate(vp, VCL_STATE_COLD);
 	if (child_pid >= 0) {
 		/* If this fails the child is crashing, figure that later */
 		(void)mgt_cli_askchild(&status, &p, "vcl.discard %s\n", av[2]);
@@ -425,7 +432,7 @@ mgt_vcl_poker(const struct vev *e, int what)
 	e_poker->timeout = mgt_param.vcl_cooldown * .45;
 	VTAILQ_FOREACH(vp, &vclhead, list) {
 		if (vp != active_vcl)
-			mgt_vcl_setstate(vp, -1);
+			mgt_vcl_setstate(vp, VCL_STATE_AUTO);
 	}
 	return (0);
 }



From dridi.boukelmoune at gmail.com  Mon Jan 11 09:44:53 2016
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Mon, 11 Jan 2016 10:44:53 +0100
Subject: [master] f8b7621 Make vcl_set_state accept a ctx instead of a vcl
Message-ID: <E1aIZ1x-0003lZ-3q@project.varnish-software.com>


commit f8b762100a829af39560f0e59d42a9ae9b690e44
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Fri Dec 4 11:14:29 2015 +0100

    Make vcl_set_state accept a ctx instead of a vcl
    
    This paves the way towards failing VCL_EVENT_WARM events in VMODs. If
    setting the temperature fails, we may need to carry a message back to
    the CLI and other bits of context.

diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index 11e4f54..e19c9ce 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -429,17 +429,16 @@ vcl_find(const char *name)
 }
 
 static void
-vcl_set_state(struct vcl *vcl, const char *state)
+vcl_set_state(VRT_CTX, const char *state)
 {
-	struct vrt_ctx ctx;
-	unsigned hand = 0;
+	struct vcl *vcl;
 
 	ASSERT_CLI();
-	AN(vcl->temp);
+	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
+	AN(ctx->handling);
 
-	INIT_OBJ(&ctx, VRT_CTX_MAGIC);
-	ctx.handling = &hand;
-	ctx.vcl = vcl;
+	vcl = ctx->vcl;
+	AN(vcl->temp);
 
 	switch(state[0]) {
 	case '0':
@@ -449,7 +448,7 @@ vcl_set_state(struct vcl *vcl, const char *state)
 
 			vcl->temp = vcl->refcount ? VCL_TEMP_COOLING :
 			    VCL_TEMP_COLD;
-			AZ(vcl->conf->event_vcl(&ctx, VCL_EVENT_COLD));
+			AZ(vcl->conf->event_vcl(ctx, VCL_EVENT_COLD));
 			vcl_BackendEvent(vcl, VCL_EVENT_COLD);
 		}
 		else if (vcl->busy)
@@ -466,7 +465,7 @@ vcl_set_state(struct vcl *vcl, const char *state)
 		/* The VCL must first reach a stable cold state */
 		else if (vcl->temp != VCL_TEMP_COOLING) {
 			vcl->temp = VCL_TEMP_WARM;
-			(void)vcl->conf->event_vcl(&ctx, VCL_EVENT_WARM);
+			(void)vcl->conf->event_vcl(ctx, VCL_EVENT_WARM);
 			vcl_BackendEvent(vcl, VCL_EVENT_WARM);
 		}
 		break;
@@ -529,7 +528,7 @@ VCL_Load(struct cli *cli, const char *name, const char *fn, const char *state)
 		return (1);
 	}
 	VSB_delete(vsb);
-	vcl_set_state(vcl, state);
+	vcl_set_state(&ctx, state);
 	bprintf(vcl->state, "%s", state + 1);
 	assert(hand == VCL_RET_OK);
 	VCLI_Out(cli, "Loaded \"%s\" as \"%s\"", fn , name);
@@ -577,13 +576,19 @@ VCL_Nuke(struct vcl *vcl)
 void
 VCL_Poll(void)
 {
+	struct vrt_ctx ctx;
 	struct vcl *vcl, *vcl2;
+	unsigned hand;
 
 	ASSERT_CLI();
 	VTAILQ_FOREACH_SAFE(vcl, &vcl_head, list, vcl2) {
 		if (vcl->temp == VCL_TEMP_BUSY ||
-		    vcl->temp == VCL_TEMP_COOLING)
-			vcl_set_state(vcl, "0");
+		    vcl->temp == VCL_TEMP_COOLING) {
+			INIT_OBJ(&ctx, VRT_CTX_MAGIC);
+			ctx.vcl = vcl;
+			ctx.handling = &hand;
+			vcl_set_state(&ctx, "0");
+		}
 		if (vcl->discard && vcl->temp == VCL_TEMP_COLD)
 			VCL_Nuke(vcl);
 	}
@@ -625,17 +630,21 @@ ccf_config_load(struct cli *cli, const char * const *av, void *priv)
 static void __match_proto__(cli_func_t)
 ccf_config_state(struct cli *cli, const char * const *av, void *priv)
 {
-	struct vcl *vcl;
+	struct vrt_ctx ctx;
+	unsigned hand;
+
+	INIT_OBJ(&ctx, VRT_CTX_MAGIC);
+	ctx.handling = &hand;
 
 	(void)cli;
 	AZ(priv);
 	ASSERT_CLI();
 	AN(av[2]);
 	AN(av[3]);
-	vcl = vcl_find(av[2]);
-	AN(vcl);			// MGT ensures this
-	vcl_set_state(vcl, av[3]);
-	bprintf(vcl->state, "%s", av[3] + 1);
+	ctx.vcl = vcl_find(av[2]);
+	AN(ctx.vcl);			// MGT ensures this
+	vcl_set_state(&ctx, av[3]);
+	bprintf(ctx.vcl->state, "%s", av[3] + 1);
 }
 
 static void __match_proto__(cli_func_t)



From dridi.boukelmoune at gmail.com  Mon Jan 11 10:52:34 2016
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Mon, 11 Jan 2016 11:52:34 +0100
Subject: [master] c67cb24 Wrap VCL event calls in dedicated functions
Message-ID: <E1aIa5S-0007nV-QI@project.varnish-software.com>


commit c67cb243ae3e29cbecf48e1f42f0e12206e417d3
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Thu Dec 10 11:44:05 2015 +0100

    Wrap VCL event calls in dedicated functions
    
    The two functions are separated in order to have self-documenting
    signatures because one class of events is (or will be) allowed to
    fail while the other MUST be failsafe.

diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index e19c9ce..b1b7756 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -428,6 +428,35 @@ vcl_find(const char *name)
 	return (NULL);
 }
 
+static int
+vcl_setup_event(VRT_CTX, enum vcl_event_e ev)
+{
+
+	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
+	AN(ctx->handling);
+	AN(ctx->vcl);
+	assert(ev == VCL_EVENT_LOAD || ev == VCL_EVENT_WARM ||
+	    ev == VCL_EVENT_USE);
+
+	if (ev == VCL_EVENT_LOAD)
+		AN(ctx->msg);
+
+	return (ctx->vcl->conf->event_vcl(ctx, ev));
+}
+
+static void
+vcl_failsafe_event(VRT_CTX, enum vcl_event_e ev)
+{
+
+	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
+	AN(ctx->handling);
+	AN(ctx->vcl);
+	assert(ev == VCL_EVENT_COLD || ev == VCL_EVENT_DISCARD);
+
+	if (ctx->vcl->conf->event_vcl(ctx, ev) != 0)
+		WRONG("A VMOD cannot fail COLD or DISCARD events");
+}
+
 static void
 vcl_set_state(VRT_CTX, const char *state)
 {
@@ -448,7 +477,7 @@ vcl_set_state(VRT_CTX, const char *state)
 
 			vcl->temp = vcl->refcount ? VCL_TEMP_COOLING :
 			    VCL_TEMP_COLD;
-			AZ(vcl->conf->event_vcl(ctx, VCL_EVENT_COLD));
+			vcl_failsafe_event(ctx, VCL_EVENT_COLD);
 			vcl_BackendEvent(vcl, VCL_EVENT_COLD);
 		}
 		else if (vcl->busy)
@@ -465,7 +494,7 @@ vcl_set_state(VRT_CTX, const char *state)
 		/* The VCL must first reach a stable cold state */
 		else if (vcl->temp != VCL_TEMP_COOLING) {
 			vcl->temp = VCL_TEMP_WARM;
-			(void)vcl->conf->event_vcl(ctx, VCL_EVENT_WARM);
+			(void)vcl_setup_event(ctx, VCL_EVENT_WARM);
 			vcl_BackendEvent(vcl, VCL_EVENT_WARM);
 		}
 		break;
@@ -515,13 +544,13 @@ VCL_Load(struct cli *cli, const char *name, const char *fn, const char *state)
 
 	VSB_clear(vsb);
 	ctx.msg = vsb;
-	i = vcl->conf->event_vcl(&ctx, VCL_EVENT_LOAD);
+	i = vcl_setup_event(&ctx, VCL_EVENT_LOAD);
 	AZ(VSB_finish(vsb));
 	if (i) {
 		VCLI_Out(cli, "VCL \"%s\" Failed initialization", name);
 		if (VSB_len(vsb))
 			VCLI_Out(cli, "\nMessage:\n\t%s", VSB_data(vsb));
-		AZ(vcl->conf->event_vcl(&ctx, VCL_EVENT_DISCARD));
+		vcl_failsafe_event(&ctx, VCL_EVENT_DISCARD);
 		vcl_KillBackends(vcl);
 		VCL_Close(&vcl);
 		VSB_delete(vsb);
@@ -563,7 +592,7 @@ VCL_Nuke(struct vcl *vcl)
 	ctx.method = VCL_MET_FINI;
 	ctx.handling = &hand;
 	ctx.vcl = vcl;
-	AZ(vcl->conf->event_vcl(&ctx, VCL_EVENT_DISCARD));
+	vcl_failsafe_event(&ctx, VCL_EVENT_DISCARD);
 	vcl_KillBackends(vcl);
 	free(vcl->loaded_name);
 	VCL_Close(&vcl);
@@ -688,7 +717,7 @@ ccf_config_use(struct cli *cli, const char * const *av, void *priv)
 	AN(vsb);
 	ctx.msg = vsb;
 	ctx.vcl = vcl;
-	i = vcl->conf->event_vcl(&ctx, VCL_EVENT_USE);
+	i = vcl_setup_event(&ctx, VCL_EVENT_USE);
 	AZ(VSB_finish(vsb));
 	if (i) {
 		VCLI_Out(cli, "VCL \"%s\" Failed to activate", av[2]);



From phk at FreeBSD.org  Mon Jan 11 10:57:33 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Mon, 11 Jan 2016 11:57:33 +0100
Subject: [master] 3882b45 Move the trim-ability check into the stevedore
 implementation, rather than have the fetch code make a blanket ban.
Message-ID: <E1aIaAH-0008JV-Gp@project.varnish-software.com>


commit 3882b45ace3932a7514d63f748f6ae2fadb67b8b
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Jan 11 09:50:58 2016 +0000

    Move the trim-ability check into the stevedore implementation,
    rather than have the fetch code make a blanket ban.

diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index d7411b0..503dc3c 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -527,8 +527,7 @@ vbf_fetch_body_helper(struct busyobj *bo)
 		bo->htc->doclose = SC_RX_BODY;
 	}
 
-	if (!bo->do_stream)
-		ObjTrimStore(bo->wrk, vfc->oc);
+	ObjTrimStore(bo->wrk, vfc->oc);
 }
 
 /*--------------------------------------------------------------------
diff --git a/bin/varnishd/storage/storage_simple.c b/bin/varnishd/storage/storage_simple.c
index 6550c0f..82c2925 100644
--- a/bin/varnishd/storage/storage_simple.c
+++ b/bin/varnishd/storage/storage_simple.c
@@ -165,7 +165,7 @@ SML_allocobj(struct worker *wrk, const struct stevedore *stv,
  */
 
 static struct object *
-getobj(struct worker *wrk, struct objcore *oc)
+sml_getobj(struct worker *wrk, struct objcore *oc)
 {
 	const struct obj_methods *m;
 	struct object *o;
@@ -192,7 +192,7 @@ sml_slim(struct worker *wrk, struct objcore *oc)
 
 	stv = oc->stobj->stevedore;
 	CHECK_OBJ_NOTNULL(stv, STEVEDORE_MAGIC);
-	o = getobj(wrk, oc);
+	o = sml_getobj(wrk, oc);
 	CHECK_OBJ_NOTNULL(o, OBJECT_MAGIC);
 
 	if (o->esidata != NULL) {
@@ -252,7 +252,7 @@ sml_iterator(struct worker *wrk, struct objcore *oc,
 	void *p;
 	ssize_t l;
 
-	obj = getobj(wrk, oc);
+	obj = sml_getobj(wrk, oc);
 	CHECK_OBJ_NOTNULL(obj, OBJECT_MAGIC);
 
 	bo = HSH_RefBusy(oc);
@@ -370,7 +370,7 @@ sml_getspace(struct worker *wrk, struct objcore *oc, ssize_t *sz,
 	AN(ptr);
 	assert(*sz > 0);
 
-	o = getobj(wrk, oc);
+	o = sml_getobj(wrk, oc);
 	CHECK_OBJ_NOTNULL(o, OBJECT_MAGIC);
 
 	st = VTAILQ_LAST(&o->list, storagehead);
@@ -409,7 +409,7 @@ sml_extend(struct worker *wrk, struct objcore *oc, ssize_t l)
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	assert(l > 0);
 
-	o = getobj(wrk, oc);
+	o = sml_getobj(wrk, oc);
 	CHECK_OBJ_NOTNULL(o, OBJECT_MAGIC);
 	st = VTAILQ_LAST(&o->list, storagehead);
 	CHECK_OBJ_NOTNULL(st, STORAGE_MAGIC);
@@ -425,7 +425,7 @@ sml_getlen(struct worker *wrk, struct objcore *oc)
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 
-	o = getobj(wrk, oc);
+	o = sml_getobj(wrk, oc);
 	CHECK_OBJ_NOTNULL(o, OBJECT_MAGIC);
 	return (o->len);
 }
@@ -436,12 +436,21 @@ sml_trimstore(struct worker *wrk, struct objcore *oc)
 	const struct stevedore *stv;
 	struct storage *st;
 	struct object *o;
+	struct busyobj *bo;
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
+	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
+
+	bo = oc->busyobj;
+	if (bo != NULL) {
+		CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
+		if (bo->do_stream)
+			return;
+	}
 
 	stv = oc->stobj->stevedore;
 	CHECK_OBJ_NOTNULL(stv, STEVEDORE_MAGIC);
-	o = getobj(wrk, oc);
+	o = sml_getobj(wrk, oc);
 	CHECK_OBJ_NOTNULL(o, OBJECT_MAGIC);
 	st = VTAILQ_LAST(&o->list, storagehead);
 	if (st == NULL)
@@ -465,7 +474,7 @@ sml_getattr(struct worker *wrk, struct objcore *oc, enum obj_attr attr,
 
 	if (len == NULL)
 		len = &dummy;
-	o = getobj(wrk, oc);
+	o = sml_getobj(wrk, oc);
 	CHECK_OBJ_NOTNULL(o, OBJECT_MAGIC);
 	switch (attr) {
 	case OA_ESIDATA:
@@ -507,7 +516,7 @@ sml_setattr(struct worker *wrk, struct objcore *oc, enum obj_attr attr,
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 
-	o = getobj(wrk, oc);
+	o = sml_getobj(wrk, oc);
 	CHECK_OBJ_NOTNULL(o, OBJECT_MAGIC);
 	st = o->objstore;
 	switch (attr) {



From phk at FreeBSD.org  Mon Jan 11 10:57:33 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Mon, 11 Jan 2016 11:57:33 +0100
Subject: [master] c4eef1e Make the simple stevedores even simpler by making
 trim a alloc/free sequence.
Message-ID: <E1aIaAH-0008Jb-Ic@project.varnish-software.com>


commit c4eef1e216330630a18aac85a7b5d22b22aef692
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Jan 11 10:24:12 2016 +0000

    Make the simple stevedores even simpler by making trim a alloc/free
    sequence.

diff --git a/bin/varnishd/storage/storage.h b/bin/varnishd/storage/storage.h
index 0793b4e..ee10ae2 100644
--- a/bin/varnishd/storage/storage.h
+++ b/bin/varnishd/storage/storage.h
@@ -87,7 +87,6 @@ struct stevedore {
 	storage_init_f		*init;		/* called by mgt process */
 	storage_open_f		*open;		/* called by cache process */
 	storage_alloc_f		*alloc;		/* --//-- only if SML */
-	storage_trim_f		*trim;		/* --//-- only if SML */
 	storage_free_f		*free;		/* --//-- only if SML */
 	storage_close_f		*close;		/* --//-- */
 	storage_allocobj_f	*allocobj;	/* --//-- */
diff --git a/bin/varnishd/storage/storage_file.c b/bin/varnishd/storage/storage_file.c
index 9df6d9f..89258e6 100644
--- a/bin/varnishd/storage/storage_file.c
+++ b/bin/varnishd/storage/storage_file.c
@@ -307,36 +307,6 @@ free_smf(struct smf *sp)
 }
 
 /*--------------------------------------------------------------------
- * Trim the tail of a range.
- */
-
-static void
-trim_smf(struct smf *sp, size_t bytes)
-{
-	struct smf *sp2;
-	struct smf_sc *sc = sp->sc;
-
-	AN(sp->alloc);
-	assert(bytes > 0);
-	assert(bytes < sp->size);
-	AZ(bytes % sc->pagesize);
-	AZ(sp->size % sc->pagesize);
-	CHECK_OBJ_NOTNULL(sp, SMF_MAGIC);
-	sp2 = malloc(sizeof *sp2);
-	XXXAN(sp2);
-	sc->stats->g_smf++;
-	*sp2 = *sp;
-
-	sp2->size -= bytes;
-	sp->size = bytes;
-	sp2->ptr += bytes;
-	sp2->offset += bytes;
-	VTAILQ_INSERT_AFTER(&sc->order, sp, sp2, order);
-	VTAILQ_INSERT_TAIL(&sc->used, sp2, status);
-	free_smf(sp2);
-}
-
-/*--------------------------------------------------------------------
  * Insert a newly created range as busy, then free it to do any collapses
  */
 
@@ -480,39 +450,6 @@ smf_alloc(const struct stevedore *st, size_t size)
 
 /*--------------------------------------------------------------------*/
 
-static void
-smf_trim(struct storage *s, size_t size, int move_ok)
-{
-	struct smf *smf;
-	struct smf_sc *sc;
-
-	CHECK_OBJ_NOTNULL(s, STORAGE_MAGIC);
-	assert(size > 0);
-	assert(size <= s->space);
-	xxxassert(size > 0);	/* XXX: seen */
-	CAST_OBJ_NOTNULL(smf, s->priv, SMF_MAGIC);
-	assert(size <= smf->size);
-
-	if (!move_ok)
-		return;		/* XXX: trim_smf needs fixed */
-
-	sc = smf->sc;
-	size += (sc->pagesize - 1);
-	size &= ~(sc->pagesize - 1);
-	if (smf->size > size) {
-		Lck_Lock(&sc->mtx);
-		sc->stats->c_freed += (smf->size - size);
-		sc->stats->g_bytes -= (smf->size - size);
-		sc->stats->g_space += (smf->size - size);
-		trim_smf(smf, size);
-		assert(smf->size == size);
-		Lck_Unlock(&sc->mtx);
-		s->space = size;
-	}
-}
-
-/*--------------------------------------------------------------------*/
-
 static void __match_proto__(storage_free_f)
 smf_free(struct storage *s)
 {
@@ -539,7 +476,6 @@ const struct stevedore smf_stevedore = {
 	.init		=	smf_init,
 	.open		=	smf_open,
 	.alloc		=	smf_alloc,
-	.trim		=	smf_trim,
 	.free		=	smf_free,
 	.allocobj	=	SML_allocobj,
 	.methods	=	&SML_methods,
@@ -597,9 +533,6 @@ main(int argc, char **argv)
 		if (s[i] == NULL) {
 			s[i] = smf_alloc(&smf_stevedore, j);
 			printf("A %10p %12d\n", s[i], j);
-		} else if (j < s[i]->space) {
-			smf_trim(s[i], j);
-			printf("T %10p %12d\n", s[i], j);
 		} else {
 			smf_free(s[i]);
 			printf("D %10p\n", s[i]);
diff --git a/bin/varnishd/storage/storage_malloc.c b/bin/varnishd/storage/storage_malloc.c
index 7060f55..2aacd41 100644
--- a/bin/varnishd/storage/storage_malloc.c
+++ b/bin/varnishd/storage/storage_malloc.c
@@ -147,41 +147,6 @@ sma_free(struct storage *s)
 	free(sma);
 }
 
-static void
-sma_trim(struct storage *s, size_t size, int move_ok)
-{
-	struct sma_sc *sma_sc;
-	struct sma *sma;
-	void *p;
-	size_t delta;
-
-	CHECK_OBJ_NOTNULL(s, STORAGE_MAGIC);
-	CAST_OBJ_NOTNULL(sma, s->priv, SMA_MAGIC);
-	sma_sc = sma->sc;
-
-	assert(sma->sz == sma->s.space);
-	assert(size < sma->sz);
-
-	if (!move_ok)
-		return;
-
-	delta = sma->sz - size;
-	if (delta < 256)
-		return;
-	if ((p = realloc(sma->s.ptr, size)) != NULL) {
-		Lck_Lock(&sma_sc->sma_mtx);
-		sma_sc->sma_alloc -= delta;
-		sma_sc->stats->g_bytes -= delta;
-		sma_sc->stats->c_freed += delta;
-		if (sma_sc->sma_max != SIZE_MAX)
-			sma_sc->stats->g_space += delta;
-		sma->sz = size;
-		Lck_Unlock(&sma_sc->sma_mtx);
-		sma->s.ptr = p;
-		s->space = size;
-	}
-}
-
 static double
 sma_used_space(const struct stevedore *st)
 {
@@ -257,7 +222,6 @@ const struct stevedore sma_stevedore = {
 	.open		=	sma_open,
 	.alloc		=	sma_alloc,
 	.free		=	sma_free,
-	.trim		=	sma_trim,
 	.allocobj	=	SML_allocobj,
 	.methods	=	&SML_methods,
 	.var_free_space =	sma_free_space,
diff --git a/bin/varnishd/storage/storage_simple.c b/bin/varnishd/storage/storage_simple.c
index 82c2925..462a8ac 100644
--- a/bin/varnishd/storage/storage_simple.c
+++ b/bin/varnishd/storage/storage_simple.c
@@ -79,17 +79,6 @@ sml_stv_free(const struct stevedore *stv, struct storage *st)
 	stv->free(st);
 }
 
-static void
-sml_stv_trim(const struct stevedore *stv, struct storage *st, size_t size,
-    int move_ok)
-{
-
-	CHECK_OBJ_NOTNULL(stv, STEVEDORE_MAGIC);
-	CHECK_OBJ_NOTNULL(st, STORAGE_MAGIC);
-	if (stv->trim)
-		stv->trim(st, size, move_ok);
-}
-
 /*--------------------------------------------------------------------
  * This function is called by stevedores ->allocobj() method, which
  * very often will be SML_allocobj() below, to convert a slab
@@ -104,6 +93,7 @@ SML_MkObject(const struct stevedore *stv, struct objcore *oc, void *ptr)
 	struct object *o;
 
 	CHECK_OBJ_NOTNULL(stv, STEVEDORE_MAGIC);
+	AN(stv->methods);
 	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
 
 	assert(PAOK(ptr));
@@ -113,9 +103,8 @@ SML_MkObject(const struct stevedore *stv, struct objcore *oc, void *ptr)
 
 	VTAILQ_INIT(&o->list);
 
-	oc->stobj->magic = STOREOBJ_MAGIC;
+	INIT_OBJ(oc->stobj, STOREOBJ_MAGIC);
 	oc->stobj->stevedore = stv;
-	AN(stv->methods);
 	oc->stobj->priv = o;
 	return (o);
 }
@@ -434,7 +423,7 @@ static void __match_proto__(objtrimstore_f)
 sml_trimstore(struct worker *wrk, struct objcore *oc)
 {
 	const struct stevedore *stv;
-	struct storage *st;
+	struct storage *st, *st1;
 	struct object *o;
 	struct busyobj *bo;
 
@@ -453,14 +442,33 @@ sml_trimstore(struct worker *wrk, struct objcore *oc)
 	o = sml_getobj(wrk, oc);
 	CHECK_OBJ_NOTNULL(o, OBJECT_MAGIC);
 	st = VTAILQ_LAST(&o->list, storagehead);
+
 	if (st == NULL)
 		return;
+
 	if (st->len == 0) {
 		VTAILQ_REMOVE(&o->list, st, list);
 		sml_stv_free(stv, st);
-	} else if (st->len < st->space) {
-		sml_stv_trim(stv, st, st->len, 1);
+		return;
+	}
+
+	if (st->space - st->len < 512)
+		return;
+
+	st1 = sml_stv_alloc(stv, st->len);
+	if (st1 == NULL)
+		return;
+
+	if (st1->space < st->len) {
+		sml_stv_free(stv, st1);
+		return;
 	}
+
+	memcpy(st1->ptr, st->ptr, st->len);
+	st1->len = st->len;
+	VTAILQ_REMOVE(&o->list, st, list);
+	VTAILQ_INSERT_TAIL(&o->list, st1, list);
+	sml_stv_free(stv, st);
 }
 
 static void * __match_proto__(objgetattr_f)



From phk at FreeBSD.org  Mon Jan 11 10:57:33 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Mon, 11 Jan 2016 11:57:33 +0100
Subject: [master] 817c7be Introduce "ObjStable" callback which informs
 obj/stevedore that the fetching busyobj is no longer.
Message-ID: <E1aIaAH-0008Jl-KE@project.varnish-software.com>


commit 817c7be50dc36661ea5002570da3fa001f9efa27
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Jan 11 10:52:36 2016 +0000

    Introduce "ObjStable" callback which informs obj/stevedore that
    the fetching busyobj is no longer.

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index 1e82ea2..e04b544 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -859,6 +859,7 @@ void *ObjSetattr(struct worker *, struct objcore *, enum obj_attr attr,
     ssize_t len, const void *);
 int ObjCopyAttr(struct worker *, struct objcore *, struct objcore *,
     enum obj_attr attr);
+void ObjStable(struct worker *, struct objcore *, struct busyobj *);
 
 int ObjSetDouble(struct worker *, struct objcore *, enum obj_attr, double);
 int ObjSetU32(struct worker *, struct objcore *, enum obj_attr, uint32_t);
diff --git a/bin/varnishd/cache/cache_busyobj.c b/bin/varnishd/cache/cache_busyobj.c
index 78e2512..f32831f 100644
--- a/bin/varnishd/cache/cache_busyobj.c
+++ b/bin/varnishd/cache/cache_busyobj.c
@@ -204,6 +204,8 @@ VBO_DerefBusyObj(struct worker *wrk, struct busyobj **pbo)
 
 	if (bo->fetch_objcore != NULL) {
 		AN(wrk);
+		if (bo->fetch_objcore->stobj->stevedore != NULL)
+			ObjStable(bo->wrk, bo->fetch_objcore, bo);
 		(void)HSH_DerefObjCore(wrk, &bo->fetch_objcore);
 	}
 
diff --git a/bin/varnishd/cache/cache_obj.c b/bin/varnishd/cache/cache_obj.c
index ebbb76f..67faea0 100644
--- a/bin/varnishd/cache/cache_obj.c
+++ b/bin/varnishd/cache/cache_obj.c
@@ -186,6 +186,22 @@ ObjUpdateMeta(struct worker *wrk, struct objcore *oc)
 }
 
 /*====================================================================
+ * Called when the busyobj used to populate the objcore is going away.
+ * Useful for releasing any leftovers from Trim.
+ */
+
+void
+ObjStable(struct worker *wrk, struct objcore *oc, struct busyobj *bo)
+{
+	const struct obj_methods *m = obj_getmethods(oc);
+
+	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
+	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
+	if (m->objstable != NULL)
+		m->objstable(wrk, oc, bo);
+}
+
+/*====================================================================
  */
 void
 ObjFreeObj(struct worker *wrk, struct objcore *oc)
diff --git a/bin/varnishd/cache/cache_obj.h b/bin/varnishd/cache/cache_obj.h
index c08d75c..0058434 100644
--- a/bin/varnishd/cache/cache_obj.h
+++ b/bin/varnishd/cache/cache_obj.h
@@ -43,6 +43,7 @@ typedef int objgetspace_f(struct worker *, struct objcore *,
      ssize_t *sz, uint8_t **ptr);
 typedef void objextend_f(struct worker *, struct objcore *, ssize_t l);
 typedef void objtrimstore_f(struct worker *, struct objcore *);
+typedef void objstable_f(struct worker *, struct objcore *, struct busyobj *);
 typedef void objslim_f(struct worker *, struct objcore *);
 typedef void *objgetattr_f(struct worker *, struct objcore *,
     enum obj_attr attr, ssize_t *len);
@@ -63,6 +64,7 @@ struct obj_methods {
 	objextend_f	*objextend;
 	objgetlen_f	*objgetlen;
 	objtrimstore_f	*objtrimstore;
+	objstable_f	*objstable;
 	objslim_f	*objslim;
 	objgetattr_f	*objgetattr;
 	objsetattr_f	*objsetattr;



From dridi.boukelmoune at gmail.com  Mon Jan 11 11:51:45 2016
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Mon, 11 Jan 2016 12:51:45 +0100
Subject: [4.0] 8795c23 Prevent a segmentation fault in VBE_AddBackend
Message-ID: <E1aIb0j-0003O8-F1@project.varnish-software.com>


commit 8795c23ecf20f1c789cbae4f13ca6ffa4aef23ca
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Mon Jan 11 12:46:06 2016 +0100

    Prevent a segmentation fault in VBE_AddBackend

diff --git a/bin/varnishd/cache/cache_backend_cfg.c b/bin/varnishd/cache/cache_backend_cfg.c
index a5d913f..5b122d0 100644
--- a/bin/varnishd/cache/cache_backend_cfg.c
+++ b/bin/varnishd/cache/cache_backend_cfg.c
@@ -192,6 +192,10 @@ VBE_AddBackend(struct cli *cli, const struct vrt_backend *vb)
 		CHECK_OBJ_NOTNULL(b, BACKEND_MAGIC);
 		if (strcmp(b->vcl_name, vb->vcl_name))
 			continue;
+		if ((vb->ipv4_suckaddr != NULL) ^ (b->ipv4 != NULL))
+			continue;
+		if ((vb->ipv6_suckaddr != NULL) ^ (b->ipv6 != NULL))
+			continue;
 		if (vb->ipv4_suckaddr != NULL &&
 		    VSA_Compare(b->ipv4, vb->ipv4_suckaddr))
 			continue;



From phk at FreeBSD.org  Mon Jan 11 12:36:14 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Mon, 11 Jan 2016 13:36:14 +0100
Subject: [master] bef42fa Implement trimming for streamed objects in simple
 stevedores.
Message-ID: <E1aIbhm-0006pm-I1@project.varnish-software.com>


commit bef42fa7a544836a2ce4a481c1865ea956563baa
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Jan 11 12:35:41 2016 +0000

    Implement trimming for streamed objects in simple stevedores.

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index e04b544..b5f67f8 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -365,12 +365,10 @@ struct lru {
 };
 
 /* Stored object -----------------------------------------------------
- * Pointer to a stored object, and the methods it supports
+ * This is just to encapsulate the fields owned by the stevedore
  */
 
 struct storeobj {
-	unsigned		magic;
-#define STOREOBJ_MAGIC		0x6faed850
 	const struct stevedore	*stevedore;
 	void			*priv;
 	uintptr_t		priv2;
@@ -502,6 +500,8 @@ struct busyobj {
 
 	struct vsl_log		vsl[1];
 
+	void			*stevedore_priv;
+
 	uint8_t			digest[DIGEST_LEN];
 	struct vrt_privs	privs[1];
 };
diff --git a/bin/varnishd/storage/storage_persistent.c b/bin/varnishd/storage/storage_persistent.c
index df00621..c068a44 100644
--- a/bin/varnishd/storage/storage_persistent.c
+++ b/bin/varnishd/storage/storage_persistent.c
@@ -585,22 +585,6 @@ smp_alloc(const struct stevedore *st, size_t size)
 	    size > 4096 ? 4096 : size, size, NULL, NULL, NULL));
 }
 
-/*--------------------------------------------------------------------
- * We don't track frees of storage, we track the objects which own the
- * storage and when there are no more objects in in the first segment,
- * it can be reclaimed.
- * XXX: We could free the last allocation, but does that happen ?
- */
-
-static void __match_proto__(storage_free_f)
-smp_free(struct storage *st)
-{
-
-	/* XXX */
-	(void)st;
-}
-
-
 /*--------------------------------------------------------------------*/
 
 const struct stevedore smp_stevedore = {
@@ -611,7 +595,7 @@ const struct stevedore smp_stevedore = {
 	.close	=	smp_close,
 	.alloc	=	smp_alloc,
 	.allocobj =	smp_allocobj,
-	.free	=	smp_free,
+	.free	=	NULL,
 	.signal_close = smp_signal_close,
 	.baninfo =	smp_baninfo,
 	.banexport =	smp_banexport,
diff --git a/bin/varnishd/storage/storage_persistent_silo.c b/bin/varnishd/storage/storage_persistent_silo.c
index 90b9fb6..fe0d889 100644
--- a/bin/varnishd/storage/storage_persistent_silo.c
+++ b/bin/varnishd/storage/storage_persistent_silo.c
@@ -528,7 +528,7 @@ smp_oc_objgetlru(const struct objcore *oc)
 
 const struct obj_methods smp_oc_methods = {
 	.sml_getobj =		smp_oc_sml_getobj,
-	.objupdatemeta =		smp_oc_objupdatemeta,
+	.objupdatemeta =	smp_oc_objupdatemeta,
 	.objfree =		smp_oc_objfree,
 	.objgetlru =		smp_oc_objgetlru,
 };
diff --git a/bin/varnishd/storage/storage_simple.c b/bin/varnishd/storage/storage_simple.c
index 462a8ac..e039707 100644
--- a/bin/varnishd/storage/storage_simple.c
+++ b/bin/varnishd/storage/storage_simple.c
@@ -75,8 +75,8 @@ sml_stv_free(const struct stevedore *stv, struct storage *st)
 
 	CHECK_OBJ_NOTNULL(stv, STEVEDORE_MAGIC);
 	CHECK_OBJ_NOTNULL(st, STORAGE_MAGIC);
-	AN(stv->free);
-	stv->free(st);
+	if (stv->free != NULL)
+		stv->free(st);
 }
 
 /*--------------------------------------------------------------------
@@ -103,9 +103,9 @@ SML_MkObject(const struct stevedore *stv, struct objcore *oc, void *ptr)
 
 	VTAILQ_INIT(&o->list);
 
-	INIT_OBJ(oc->stobj, STOREOBJ_MAGIC);
 	oc->stobj->stevedore = stv;
 	oc->stobj->priv = o;
+	oc->stobj->priv2 = 0;
 	return (o);
 }
 
@@ -202,7 +202,6 @@ sml_objfree(struct worker *wrk, struct objcore *oc)
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
-	CHECK_OBJ_NOTNULL(oc->stobj, STOREOBJ_MAGIC);
 	sml_slim(wrk, oc);
 	CAST_OBJ_NOTNULL(o, oc->stobj->priv, OBJECT_MAGIC);
 	o->magic = 0;
@@ -425,20 +424,16 @@ sml_trimstore(struct worker *wrk, struct objcore *oc)
 	const struct stevedore *stv;
 	struct storage *st, *st1;
 	struct object *o;
-	struct busyobj *bo;
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
 
-	bo = oc->busyobj;
-	if (bo != NULL) {
-		CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
-		if (bo->do_stream)
-			return;
-	}
-
 	stv = oc->stobj->stevedore;
 	CHECK_OBJ_NOTNULL(stv, STEVEDORE_MAGIC);
+
+	if (stv->free == NULL)
+		return;
+
 	o = sml_getobj(wrk, oc);
 	CHECK_OBJ_NOTNULL(o, OBJECT_MAGIC);
 	st = VTAILQ_LAST(&o->list, storagehead);
@@ -468,6 +463,32 @@ sml_trimstore(struct worker *wrk, struct objcore *oc)
 	st1->len = st->len;
 	VTAILQ_REMOVE(&o->list, st, list);
 	VTAILQ_INSERT_TAIL(&o->list, st1, list);
+	if (oc->busyobj == NULL) {
+		sml_stv_free(stv, st);
+	} else {
+		/* sml_stable frees this */
+		AZ(oc->busyobj->stevedore_priv);
+		oc->busyobj->stevedore_priv = st;
+	}
+}
+
+static void __match_proto__(objstable_f)
+sml_stable(struct worker *wrk, struct objcore *oc, struct busyobj *bo)
+{
+	const struct stevedore *stv;
+	struct storage *st;
+
+	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
+	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
+	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
+
+	if (bo->stevedore_priv == NULL)
+		return;
+	CAST_OBJ_NOTNULL(st, bo->stevedore_priv, STORAGE_MAGIC);
+	bo->stevedore_priv = 0;
+	CHECK_OBJ_NOTNULL(st, STORAGE_MAGIC);
+	stv = oc->stobj->stevedore;
+	CHECK_OBJ_NOTNULL(stv, STEVEDORE_MAGIC);
 	sml_stv_free(stv, st);
 }
 
@@ -621,6 +642,7 @@ const struct obj_methods SML_methods = {
 	.objextend	= sml_extend,
 	.objgetlen	= sml_getlen,
 	.objtrimstore	= sml_trimstore,
+	.objstable	= sml_stable,
 	.objslim	= sml_slim,
 	.objgetattr	= sml_getattr,
 	.objsetattr	= sml_setattr,
diff --git a/bin/varnishtest/tests/c00073.vtc b/bin/varnishtest/tests/c00073.vtc
new file mode 100644
index 0000000..f638e20
--- /dev/null
+++ b/bin/varnishtest/tests/c00073.vtc
@@ -0,0 +1,27 @@
+varnishtest "Test object trimming"
+
+server s1 {
+	rxreq
+	txresp -nolen -hdr "Transfer-encoding: chunked"
+	delay .2
+	chunkedlen 4096
+	sema r1 sync 2
+	sema r2 sync 2
+	chunkedlen 0
+} -start
+
+varnish v1 \
+	-arg "-s malloc,1m" -vcl+backend { } -start
+
+client c1 {
+	txreq
+	rxresp
+} -start
+
+sema r1 sync 2
+varnish v1 -expect SMA.s0.g_bytes > 10000
+
+sema r2 sync 2
+
+client c1 -wait
+varnish v1 -expect SMA.s0.g_bytes < 6000



From phk at FreeBSD.org  Mon Jan 11 14:54:18 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Mon, 11 Jan 2016 15:54:18 +0100
Subject: [master] f4895da Split VCL event sender functions out individually.
Message-ID: <E1aIdrO-00078X-Uk@project.varnish-software.com>


commit f4895da5b08b83c2190ff96e2a6364e4699e962f
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Jan 11 14:53:48 2016 +0000

    Split VCL event sender functions out individually.
    
    Dridi:  Not XXX comments

diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index b1b7756..5856f3c 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -411,50 +411,75 @@ VRT_rel_vcl(VRT_CTX)
 	Lck_Unlock(&vcl_mtx);
 }
 
-/*--------------------------------------------------------------------*/
+/*--------------------------------------------------------------------
+ * Wrapper functions to send events to VCL and guarantee semantics.
+ */
 
-static struct vcl *
-vcl_find(const char *name)
+static int
+vcl_event_load(VRT_CTX)
 {
-	struct vcl *vcl;
-
-	ASSERT_CLI();
-	VTAILQ_FOREACH(vcl, &vcl_head, list) {
-		if (vcl->discard)
-			continue;
-		if (!strcmp(vcl->loaded_name, name))
-			return (vcl);
-	}
-	return (NULL);
+	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
+	AN(ctx->handling);
+	AN(ctx->vcl);
+	AN(ctx->msg);
+	return (ctx->vcl->conf->event_vcl(ctx, VCL_EVENT_LOAD));
 }
 
 static int
-vcl_setup_event(VRT_CTX, enum vcl_event_e ev)
+vcl_event_warm(VRT_CTX)
 {
-
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
 	AN(ctx->handling);
 	AN(ctx->vcl);
-	assert(ev == VCL_EVENT_LOAD || ev == VCL_EVENT_WARM ||
-	    ev == VCL_EVENT_USE);
-
-	if (ev == VCL_EVENT_LOAD)
-		AN(ctx->msg);
+	// AN/AZ(ctx->msg);		// XXX: Dridi: which is it ?
+	return (ctx->vcl->conf->event_vcl(ctx, VCL_EVENT_WARM));
+}
 
-	return (ctx->vcl->conf->event_vcl(ctx, ev));
+static int
+vcl_event_use(VRT_CTX)
+{
+	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
+	AN(ctx->handling);
+	AN(ctx->vcl);
+	AN(ctx->msg);
+	return (ctx->vcl->conf->event_vcl(ctx, VCL_EVENT_USE));
 }
 
 static void
-vcl_failsafe_event(VRT_CTX, enum vcl_event_e ev)
+vcl_event_cold(VRT_CTX)
 {
+	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
+	AN(ctx->handling);
+	AN(ctx->vcl);
+	AZ(ctx->msg);
+	AZ(ctx->vcl->conf->event_vcl(ctx, VCL_EVENT_COLD));
+}
 
+static void
+vcl_event_discard(VRT_CTX)
+{
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
 	AN(ctx->handling);
 	AN(ctx->vcl);
-	assert(ev == VCL_EVENT_COLD || ev == VCL_EVENT_DISCARD);
+	// AN/AZ(ctx->msg);		// XXX:  Dridi: which is it ?
+	AZ(ctx->vcl->conf->event_vcl(ctx, VCL_EVENT_DISCARD));
+}
+
+/*--------------------------------------------------------------------*/
+
+static struct vcl *
+vcl_find(const char *name)
+{
+	struct vcl *vcl;
 
-	if (ctx->vcl->conf->event_vcl(ctx, ev) != 0)
-		WRONG("A VMOD cannot fail COLD or DISCARD events");
+	ASSERT_CLI();
+	VTAILQ_FOREACH(vcl, &vcl_head, list) {
+		if (vcl->discard)
+			continue;
+		if (!strcmp(vcl->loaded_name, name))
+			return (vcl);
+	}
+	return (NULL);
 }
 
 static void
@@ -477,7 +502,7 @@ vcl_set_state(VRT_CTX, const char *state)
 
 			vcl->temp = vcl->refcount ? VCL_TEMP_COOLING :
 			    VCL_TEMP_COLD;
-			vcl_failsafe_event(ctx, VCL_EVENT_COLD);
+			vcl_event_cold(ctx);
 			vcl_BackendEvent(vcl, VCL_EVENT_COLD);
 		}
 		else if (vcl->busy)
@@ -494,7 +519,7 @@ vcl_set_state(VRT_CTX, const char *state)
 		/* The VCL must first reach a stable cold state */
 		else if (vcl->temp != VCL_TEMP_COOLING) {
 			vcl->temp = VCL_TEMP_WARM;
-			(void)vcl_setup_event(ctx, VCL_EVENT_WARM);
+			vcl_event_warm(ctx);	// XXX: Dridi: what if it fails?
 			vcl_BackendEvent(vcl, VCL_EVENT_WARM);
 		}
 		break;
@@ -544,13 +569,13 @@ VCL_Load(struct cli *cli, const char *name, const char *fn, const char *state)
 
 	VSB_clear(vsb);
 	ctx.msg = vsb;
-	i = vcl_setup_event(&ctx, VCL_EVENT_LOAD);
+	i = vcl_event_load(&ctx);
 	AZ(VSB_finish(vsb));
 	if (i) {
 		VCLI_Out(cli, "VCL \"%s\" Failed initialization", name);
 		if (VSB_len(vsb))
 			VCLI_Out(cli, "\nMessage:\n\t%s", VSB_data(vsb));
-		vcl_failsafe_event(&ctx, VCL_EVENT_DISCARD);
+		vcl_event_discard(&ctx);
 		vcl_KillBackends(vcl);
 		VCL_Close(&vcl);
 		VSB_delete(vsb);
@@ -592,7 +617,7 @@ VCL_Nuke(struct vcl *vcl)
 	ctx.method = VCL_MET_FINI;
 	ctx.handling = &hand;
 	ctx.vcl = vcl;
-	vcl_failsafe_event(&ctx, VCL_EVENT_DISCARD);
+	vcl_event_discard(&ctx);
 	vcl_KillBackends(vcl);
 	free(vcl->loaded_name);
 	VCL_Close(&vcl);
@@ -717,7 +742,7 @@ ccf_config_use(struct cli *cli, const char * const *av, void *priv)
 	AN(vsb);
 	ctx.msg = vsb;
 	ctx.vcl = vcl;
-	i = vcl_setup_event(&ctx, VCL_EVENT_USE);
+	i = vcl_event_use(&ctx);
 	AZ(VSB_finish(vsb));
 	if (i) {
 		VCLI_Out(cli, "VCL \"%s\" Failed to activate", av[2]);



From dridi.boukelmoune at gmail.com  Mon Jan 11 17:41:32 2016
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Mon, 11 Jan 2016 18:41:32 +0100
Subject: [master] 7f6f2c5 Revert "Split VCL event sender functions out
 individually."
Message-ID: <E1aIgTE-0003Mw-J7@project.varnish-software.com>


commit 7f6f2c54cd6a9dee02b6e75e6dbd8c61b5d75f3d
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Mon Jan 11 18:40:09 2016 +0100

    Revert "Split VCL event sender functions out individually."
    
    This reverts commit f4895da5b08b83c2190ff96e2a6364e4699e962f, there are
    pending changes in semantics in VCL temperature that conflict with this
    commit.

diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index 5856f3c..b1b7756 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -411,75 +411,50 @@ VRT_rel_vcl(VRT_CTX)
 	Lck_Unlock(&vcl_mtx);
 }
 
-/*--------------------------------------------------------------------
- * Wrapper functions to send events to VCL and guarantee semantics.
- */
+/*--------------------------------------------------------------------*/
 
-static int
-vcl_event_load(VRT_CTX)
+static struct vcl *
+vcl_find(const char *name)
 {
-	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
-	AN(ctx->handling);
-	AN(ctx->vcl);
-	AN(ctx->msg);
-	return (ctx->vcl->conf->event_vcl(ctx, VCL_EVENT_LOAD));
-}
+	struct vcl *vcl;
 
-static int
-vcl_event_warm(VRT_CTX)
-{
-	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
-	AN(ctx->handling);
-	AN(ctx->vcl);
-	// AN/AZ(ctx->msg);		// XXX: Dridi: which is it ?
-	return (ctx->vcl->conf->event_vcl(ctx, VCL_EVENT_WARM));
+	ASSERT_CLI();
+	VTAILQ_FOREACH(vcl, &vcl_head, list) {
+		if (vcl->discard)
+			continue;
+		if (!strcmp(vcl->loaded_name, name))
+			return (vcl);
+	}
+	return (NULL);
 }
 
 static int
-vcl_event_use(VRT_CTX)
+vcl_setup_event(VRT_CTX, enum vcl_event_e ev)
 {
-	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
-	AN(ctx->handling);
-	AN(ctx->vcl);
-	AN(ctx->msg);
-	return (ctx->vcl->conf->event_vcl(ctx, VCL_EVENT_USE));
-}
 
-static void
-vcl_event_cold(VRT_CTX)
-{
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
 	AN(ctx->handling);
 	AN(ctx->vcl);
-	AZ(ctx->msg);
-	AZ(ctx->vcl->conf->event_vcl(ctx, VCL_EVENT_COLD));
+	assert(ev == VCL_EVENT_LOAD || ev == VCL_EVENT_WARM ||
+	    ev == VCL_EVENT_USE);
+
+	if (ev == VCL_EVENT_LOAD)
+		AN(ctx->msg);
+
+	return (ctx->vcl->conf->event_vcl(ctx, ev));
 }
 
 static void
-vcl_event_discard(VRT_CTX)
+vcl_failsafe_event(VRT_CTX, enum vcl_event_e ev)
 {
+
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
 	AN(ctx->handling);
 	AN(ctx->vcl);
-	// AN/AZ(ctx->msg);		// XXX:  Dridi: which is it ?
-	AZ(ctx->vcl->conf->event_vcl(ctx, VCL_EVENT_DISCARD));
-}
-
-/*--------------------------------------------------------------------*/
-
-static struct vcl *
-vcl_find(const char *name)
-{
-	struct vcl *vcl;
+	assert(ev == VCL_EVENT_COLD || ev == VCL_EVENT_DISCARD);
 
-	ASSERT_CLI();
-	VTAILQ_FOREACH(vcl, &vcl_head, list) {
-		if (vcl->discard)
-			continue;
-		if (!strcmp(vcl->loaded_name, name))
-			return (vcl);
-	}
-	return (NULL);
+	if (ctx->vcl->conf->event_vcl(ctx, ev) != 0)
+		WRONG("A VMOD cannot fail COLD or DISCARD events");
 }
 
 static void
@@ -502,7 +477,7 @@ vcl_set_state(VRT_CTX, const char *state)
 
 			vcl->temp = vcl->refcount ? VCL_TEMP_COOLING :
 			    VCL_TEMP_COLD;
-			vcl_event_cold(ctx);
+			vcl_failsafe_event(ctx, VCL_EVENT_COLD);
 			vcl_BackendEvent(vcl, VCL_EVENT_COLD);
 		}
 		else if (vcl->busy)
@@ -519,7 +494,7 @@ vcl_set_state(VRT_CTX, const char *state)
 		/* The VCL must first reach a stable cold state */
 		else if (vcl->temp != VCL_TEMP_COOLING) {
 			vcl->temp = VCL_TEMP_WARM;
-			vcl_event_warm(ctx);	// XXX: Dridi: what if it fails?
+			(void)vcl_setup_event(ctx, VCL_EVENT_WARM);
 			vcl_BackendEvent(vcl, VCL_EVENT_WARM);
 		}
 		break;
@@ -569,13 +544,13 @@ VCL_Load(struct cli *cli, const char *name, const char *fn, const char *state)
 
 	VSB_clear(vsb);
 	ctx.msg = vsb;
-	i = vcl_event_load(&ctx);
+	i = vcl_setup_event(&ctx, VCL_EVENT_LOAD);
 	AZ(VSB_finish(vsb));
 	if (i) {
 		VCLI_Out(cli, "VCL \"%s\" Failed initialization", name);
 		if (VSB_len(vsb))
 			VCLI_Out(cli, "\nMessage:\n\t%s", VSB_data(vsb));
-		vcl_event_discard(&ctx);
+		vcl_failsafe_event(&ctx, VCL_EVENT_DISCARD);
 		vcl_KillBackends(vcl);
 		VCL_Close(&vcl);
 		VSB_delete(vsb);
@@ -617,7 +592,7 @@ VCL_Nuke(struct vcl *vcl)
 	ctx.method = VCL_MET_FINI;
 	ctx.handling = &hand;
 	ctx.vcl = vcl;
-	vcl_event_discard(&ctx);
+	vcl_failsafe_event(&ctx, VCL_EVENT_DISCARD);
 	vcl_KillBackends(vcl);
 	free(vcl->loaded_name);
 	VCL_Close(&vcl);
@@ -742,7 +717,7 @@ ccf_config_use(struct cli *cli, const char * const *av, void *priv)
 	AN(vsb);
 	ctx.msg = vsb;
 	ctx.vcl = vcl;
-	i = vcl_event_use(&ctx);
+	i = vcl_setup_event(&ctx, VCL_EVENT_USE);
 	AZ(VSB_finish(vsb));
 	if (i) {
 		VCLI_Out(cli, "VCL \"%s\" Failed to activate", av[2]);



From phk at FreeBSD.org  Tue Jan 12 14:35:28 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 12 Jan 2016 15:35:28 +0100
Subject: [master] 44ce398 Flexlinting
Message-ID: <E1aJ02i-0000AB-9A@project.varnish-software.com>


commit 44ce39826190665fbf155556d2409086997a1856
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Jan 12 14:27:58 2016 +0000

    Flexlinting

diff --git a/bin/varnishd/storage/storage_file.c b/bin/varnishd/storage/storage_file.c
index 89258e6..5985f91 100644
--- a/bin/varnishd/storage/storage_file.c
+++ b/bin/varnishd/storage/storage_file.c
@@ -422,8 +422,8 @@ smf_alloc(const struct stevedore *st, size_t size)
 
 	CAST_OBJ_NOTNULL(sc, st->priv, SMF_SC_MAGIC);
 	assert(size > 0);
-	size += (sc->pagesize - 1);
-	size &= ~(sc->pagesize - 1);
+	size += (sc->pagesize - 1L);
+	size &= ~(sc->pagesize - 1L);
 	Lck_Lock(&sc->mtx);
 	sc->stats->c_req++;
 	smf = alloc_smf(sc, size);



From phk at FreeBSD.org  Wed Jan 13 09:38:54 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Wed, 13 Jan 2016 10:38:54 +0100
Subject: [master] 5d04b20 This is sort of embarrasing: I forgot to lock the
 busyobj when changing the storage list.
Message-ID: <E1aJHtG-0006Vz-2T@project.varnish-software.com>


commit 5d04b20099d30407ee3f8f885b6bcbfd7145544d
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Jan 13 09:38:14 2016 +0000

    This is sort of embarrasing:  I forgot to lock the busyobj when
    changing the storage list.
    
    Fixes: #1839

diff --git a/bin/varnishd/storage/storage_simple.c b/bin/varnishd/storage/storage_simple.c
index e039707..afbf6cc 100644
--- a/bin/varnishd/storage/storage_simple.c
+++ b/bin/varnishd/storage/storage_simple.c
@@ -442,7 +442,13 @@ sml_trimstore(struct worker *wrk, struct objcore *oc)
 		return;
 
 	if (st->len == 0) {
-		VTAILQ_REMOVE(&o->list, st, list);
+		if (oc->busyobj != NULL) {
+			Lck_Lock(&oc->busyobj->mtx);
+			VTAILQ_REMOVE(&o->list, st, list);
+			Lck_Unlock(&oc->busyobj->mtx);
+		} else {
+			VTAILQ_REMOVE(&o->list, st, list);
+		}
 		sml_stv_free(stv, st);
 		return;
 	}
@@ -461,8 +467,15 @@ sml_trimstore(struct worker *wrk, struct objcore *oc)
 
 	memcpy(st1->ptr, st->ptr, st->len);
 	st1->len = st->len;
-	VTAILQ_REMOVE(&o->list, st, list);
-	VTAILQ_INSERT_TAIL(&o->list, st1, list);
+	if (oc->busyobj != NULL) {
+		Lck_Lock(&oc->busyobj->mtx);
+		VTAILQ_REMOVE(&o->list, st, list);
+		VTAILQ_INSERT_TAIL(&o->list, st1, list);
+		Lck_Unlock(&oc->busyobj->mtx);
+	} else {
+		VTAILQ_REMOVE(&o->list, st, list);
+		VTAILQ_INSERT_TAIL(&o->list, st1, list);
+	}
 	if (oc->busyobj == NULL) {
 		sml_stv_free(stv, st);
 	} else {



From fgsch at lodoss.net  Wed Jan 13 19:27:11 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Wed, 13 Jan 2016 20:27:11 +0100
Subject: [master] a3d47c2 Handle WS_Reserve() overflows gracefully
Message-ID: <E1aJR4Z-0001VD-HF@project.varnish-software.com>


commit a3d47c258fb7938f67a053f6d041257edb69afe9
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Wed Jan 13 19:18:36 2016 +0000

    Handle WS_Reserve() overflows gracefully
    
    If we don't have enough space for the requested size, don't xxxassert,
    just mark the overflow and return 0.
    
    Discussed with daghf@ and dridi at .

diff --git a/bin/varnishd/cache/cache_ws.c b/bin/varnishd/cache/cache_ws.c
index 7070f1a..a825fdc 100644
--- a/bin/varnishd/cache/cache_ws.c
+++ b/bin/varnishd/cache/cache_ws.c
@@ -217,7 +217,10 @@ WS_Reserve(struct ws *ws, unsigned bytes)
 	if (bytes != 0 && bytes < b2)
 		b2 = PRNDUP(bytes);
 
-	xxxassert(ws->f + b2 <= ws->e);
+	if (ws->f + b2 > ws->e) {
+		WS_MarkOverflow(ws);
+		return (0);
+	}
 	ws->r = ws->f + b2;
 	DSL(DBG_WORKSPACE, 0, "WS_Reserve(%p, %u/%u) = %u",
 	    ws, b2, bytes, pdiff(ws->f, ws->r));



From fgsch at lodoss.net  Wed Jan 13 19:27:11 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Wed, 13 Jan 2016 20:27:11 +0100
Subject: [master] 3e7a1a5 Remove superflous calls to WS_Assert()
Message-ID: <E1aJR4Z-0001VJ-Ij@project.varnish-software.com>


commit 3e7a1a50d0d5e2672d9e3999bb3cc60e1194c914
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Wed Jan 13 19:21:28 2016 +0000

    Remove superflous calls to WS_Assert()
    
    This was already called and has not been any changes since then.

diff --git a/bin/varnishd/cache/cache_ws.c b/bin/varnishd/cache/cache_ws.c
index a825fdc..a1b81eb 100644
--- a/bin/varnishd/cache/cache_ws.c
+++ b/bin/varnishd/cache/cache_ws.c
@@ -134,7 +134,6 @@ WS_Alloc(struct ws *ws, unsigned bytes)
 	assert(ws->r == NULL);
 	if (ws->f + bytes > ws->e) {
 		WS_MarkOverflow(ws);
-		WS_Assert(ws);
 		return(NULL);
 	}
 	r = ws->f;
@@ -160,7 +159,6 @@ WS_Copy(struct ws *ws, const void *str, int len)
 	bytes = PRNDUP((unsigned)len);
 	if (ws->f + bytes > ws->e) {
 		WS_MarkOverflow(ws);
-		WS_Assert(ws);
 		return(NULL);
 	}
 	r = ws->f;



From lkarsten at varnish-software.com  Thu Jan 14 14:15:00 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:00 +0100
Subject: [4.1] 657e0ea Display the WRONG() message in VAS_Fail_default
Message-ID: <E1aJig0-0004vH-2u@project.varnish-software.com>


commit 657e0ea2b5d29f6d81e9f8d2c0caa4733c85d316
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Tue Jan 5 15:00:03 2016 +0100

    Display the WRONG() message in VAS_Fail_default

diff --git a/lib/libvarnish/vas.c b/lib/libvarnish/vas.c
index 2d810d2..0c857a0 100644
--- a/lib/libvarnish/vas.c
+++ b/lib/libvarnish/vas.c
@@ -1,6 +1,6 @@
 /*-
  * Copyright (c) 2006 Verdens Gang AS
- * Copyright (c) 2006-2011 Varnish Software AS
+ * Copyright (c) 2006-2016 Varnish Software AS
  * All rights reserved.
  *
  * Author: Poul-Henning Kamp <phk at phk.freebsd.dk>
@@ -46,7 +46,7 @@ VAS_Fail_default(const char *func, const char *file, int line,
 
 	if (kind == VAS_MISSING) {
 		fprintf(stderr,
-		    "Missing errorhandling code in %s(), %s line %d:\n"
+		    "Missing error handling code in %s(), %s line %d:\n"
 		    "  Condition(%s) not true.\n",
 		    func, file, line, cond);
 	} else if (kind == VAS_INCOMPLETE) {
@@ -55,8 +55,8 @@ VAS_Fail_default(const char *func, const char *file, int line,
 		    func, file, line);
 	} else if (kind == VAS_WRONG) {
 		fprintf(stderr,
-		    "Wrong turn in %s(), %s line %d:\n",
-		    func, file, line);
+		    "Wrong turn in %s(), %s line %d: %s\n",
+		    func, file, line, cond);
 	} else {
 		fprintf(stderr,
 		    "Assert error in %s(), %s line %d:\n"



From lkarsten at varnish-software.com  Thu Jan 14 14:15:00 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:00 +0100
Subject: [4.1] a8e36e0 "appearantly" > "apparently"
Message-ID: <E1aJig0-0004vN-4K@project.varnish-software.com>


commit a8e36e00e4fd5ff97bc6691619fbe5bf97fe2d22
Author: Andrew Langhorn <andrew.langhorn at digital.cabinet-office.gov.uk>
Date:   Wed Sep 30 22:50:07 2015 +0100

    "appearantly" > "apparently"
    
    A very minor spelling fix for something I noticed whilst reading the source
    for CIDR notation inside ACLs.

diff --git a/lib/libvcc/vcc_acl.c b/lib/libvcc/vcc_acl.c
index 72dd304..b9722ce 100644
--- a/lib/libvcc/vcc_acl.c
+++ b/lib/libvcc/vcc_acl.c
@@ -241,7 +241,7 @@ vcc_acl_try_getaddrinfo(struct vcc *tl, struct acl_e *ae)
 /*--------------------------------------------------------------------
  * Ancient stupidity on the part of X/Open and other standards orgs
  * dictate that "192.168" be translated to 192.0.0.168.  Ever since
- * CIDR happened, "192.168/16" notation has been used, but appearantly
+ * CIDR happened, "192.168/16" notation has been used, but apparently
  * no API supports parsing this, so roll our own.
  */
 



From lkarsten at varnish-software.com  Thu Jan 14 14:15:00 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:00 +0100
Subject: [4.1] 231e729 Spelling
Message-ID: <E1aJig0-0004vW-5j@project.varnish-software.com>


commit 231e729e612d3f50701d01412285a6ab6ca6774a
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Thu Oct 1 11:03:58 2015 +0100

    Spelling

diff --git a/doc/sphinx/reference/states.rst b/doc/sphinx/reference/states.rst
index 2f80f29..9effce1 100644
--- a/doc/sphinx/reference/states.rst
+++ b/doc/sphinx/reference/states.rst
@@ -24,7 +24,7 @@ The following graphs attempt to provide an overview over the
 processing states, their transitions and the most relevant functions
 in core code. They represent a compromise between usefulness for
 core/VMOD developers and administrators and are intended to serve as
-the reference basis for deriavtive work, such as more VCL-centric
+the reference basis for derivative work, such as more VCL-centric
 views.
 
 -----------



From lkarsten at varnish-software.com  Thu Jan 14 14:15:00 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:00 +0100
Subject: [4.1] fd51c52 Correct argument
Message-ID: <E1aJig0-0004vf-77@project.varnish-software.com>


commit fd51c52fa1f2d52e2fa829586c60c33ee5754568
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Thu Oct 1 17:05:26 2015 +0100

    Correct argument

diff --git a/lib/libvmod_std/vmod.vcc b/lib/libvmod_std/vmod.vcc
index 0002426..2e20825 100644
--- a/lib/libvmod_std/vmod.vcc
+++ b/lib/libvmod_std/vmod.vcc
@@ -65,7 +65,7 @@ Description
 	utilize it.
 Example
 	| if (req.url ~ ^/slow/) {
-	|	std.set_ip_tos(0x0);
+	|	std.set_ip_tos(0);
 	| }
 
 $Function REAL random(REAL lo, REAL hi)



From lkarsten at varnish-software.com  Thu Jan 14 14:15:00 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:00 +0100
Subject: [4.1] fe1ec6d Fail the VSB if vsnprintf(3) fails.
Message-ID: <E1aJig0-0004vo-8q@project.varnish-software.com>


commit fe1ec6daef18f3e9787a2f0386e32f5c8720b168
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Oct 2 09:23:33 2015 +0000

    Fail the VSB if vsnprintf(3) fails.

diff --git a/lib/libvarnish/vsb.c b/lib/libvarnish/vsb.c
index eba639e..63c196d 100644
--- a/lib/libvarnish/vsb.c
+++ b/lib/libvarnish/vsb.c
@@ -350,6 +350,10 @@ VSB_vprintf(struct vsb *s, const char *fmt, va_list ap)
 		va_copy(ap_copy, ap);
 		len = vsnprintf(&s->s_buf[s->s_len], VSB_FREESPACE(s) + 1,
 		    fmt, ap_copy);
+		if (len < 0) {
+			s->s_error = errno;
+			return (-1);
+		}
 		va_end(ap_copy);
 	} while (len > VSB_FREESPACE(s) &&
 	    VSB_extend(s, len - VSB_FREESPACE(s)) == 0);



From lkarsten at varnish-software.com  Thu Jan 14 14:15:00 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:00 +0100
Subject: [4.1] 32af514 Add explict expect for 206 status
Message-ID: <E1aJig0-0004vx-AW@project.varnish-software.com>


commit 32af514e066cdf98bfdf7b7756a620314557df1e
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Oct 5 12:25:37 2015 +0000

    Add explict expect for 206 status

diff --git a/bin/varnishtest/tests/g00005.vtc b/bin/varnishtest/tests/g00005.vtc
index bc47a5d..7cb2664 100644
--- a/bin/varnishtest/tests/g00005.vtc
+++ b/bin/varnishtest/tests/g00005.vtc
@@ -27,6 +27,7 @@ client c1 {
 
 	txreq -hdr "Range: bytes=3-5"
 	rxresp
+	expect resp.status == 206
 	expect resp.http.content-encoding == "<undef>"
 	expect resp.bodylen == "3"
 	expect resp.body == "BAR"



From lkarsten at varnish-software.com  Thu Jan 14 14:15:00 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:00 +0100
Subject: [4.1] bd9fb1a Lint.
Message-ID: <E1aJig0-0004w7-CV@project.varnish-software.com>


commit bd9fb1ad709218ade2c748e5d6f066e101a3c8c5
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Sun Oct 11 10:02:16 2015 +0200

    Lint.

diff --git a/lib/libvcc/generate.py b/lib/libvcc/generate.py
index 6d6915a..81e773c 100755
--- a/lib/libvcc/generate.py
+++ b/lib/libvcc/generate.py
@@ -674,7 +674,7 @@ sp_variables = [
 		'HTTP',
 		( 'deliver', 'synth'),
 		( ), """
-		The entire response HTTP data structure
+		The entire response HTTP data structure.
 		"""
 	),
 	('resp.proto',



From lkarsten at varnish-software.com  Thu Jan 14 14:15:00 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:00 +0100
Subject: [4.1] 0d8c80b Document that resp.status overwrites resp.reason.
Message-ID: <E1aJig0-0004wG-Du@project.varnish-software.com>


commit 0d8c80ba2eccb6a06913b42a44ea5a87cf1a9a13
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Sun Oct 11 10:02:21 2015 +0200

    Document that resp.status overwrites resp.reason.
    
    For those cases where you want to send a different reason
    message, but use the normal status code.
    
    Noticed by Kacper when doing 301/302 in vcl_synth().

diff --git a/lib/libvcc/generate.py b/lib/libvcc/generate.py
index 81e773c..062d180 100755
--- a/lib/libvcc/generate.py
+++ b/lib/libvcc/generate.py
@@ -689,6 +689,9 @@ sp_variables = [
 		( 'deliver', 'synth', ),
 		( 'deliver', 'synth', ), """
 		The HTTP status code that will be returned.
+
+                Assigning a HTTP standardized code to resp.status will also
+                set resp.reason to the corresponding status message.
 		"""
 	),
 	('resp.reason',



From lkarsten at varnish-software.com  Thu Jan 14 14:15:00 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:00 +0100
Subject: [4.1] b764913 Add vsl arguments to the comment
Message-ID: <E1aJig0-0004wR-FQ@project.varnish-software.com>


commit b764913efb1ef9a0b3a30b542ffca037f4b549ed
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Tue Oct 13 04:40:35 2015 +0100

    Add vsl arguments to the comment

diff --git a/bin/varnishtest/vtc_logexp.c b/bin/varnishtest/vtc_logexp.c
index 520feb7..767f7e6 100644
--- a/bin/varnishtest/vtc_logexp.c
+++ b/bin/varnishtest/vtc_logexp.c
@@ -43,7 +43,7 @@
  *   -T <seconds>         Transaction end timeout
  *
  *
- * logexpect lN -v <id> [-g <grouping>] [-d 0|1] [-q query] {
+ * logexpect lN -v <id> [-g <grouping>] [-d 0|1] [-q query] [vsl arguments] {
  *    expect <skip> <vxid> <tag> <regex>
  * }
  *



From lkarsten at varnish-software.com  Thu Jan 14 14:15:00 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:00 +0100
Subject: [4.1] d5679ae Doc fixes
Message-ID: <E1aJig0-0004we-HQ@project.varnish-software.com>


commit d5679aefc1109922059b6acfb1d560885e19d3e6
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Tue Oct 13 05:07:37 2015 +0100

    Doc fixes
    
    From minusf via github with minor tweaks.

diff --git a/doc/sphinx/users-guide/run_security.rst b/doc/sphinx/users-guide/run_security.rst
index 41c68a8..2beb4aa 100644
--- a/doc/sphinx/users-guide/run_security.rst
+++ b/doc/sphinx/users-guide/run_security.rst
@@ -57,11 +57,9 @@ much anything the kernel will accept::
 	-T '[fe80::1]:8082'
 
 The default is ``-T localhost:0`` which will pick a random
-port number, which `varnishadm(8)` can learn in the shared
+port number, which `varnishadm(8)` can learn from the shared
 memory.
 
-.. XXX:Me no understand sentence above, (8)? and learn in the shared memory? Stored and retrieved by varnishadm from th e shared memory? benc 
-
 By using a "localhost" address, you restrict CLI access
 to the local machine.
 
@@ -77,9 +75,9 @@ Alternatively you can bind the CLI port to a 'localhost' address,
 and give remote users access via a secure connection to the local
 machine, using ssh/VPN or similar.
 
-If you use `ssh` you can restrict which commands each user can execute to
-just `varnishadm`, or even to wrapper scripts around `varnishadm`, which
-only allow specific CLI commands.
+If you use `ssh` you can restrict which commands each user can execute
+to just `varnishadm`, or even use a wrapper scripts around `varnishadm`
+to allow specific CLI commands.
 
 It is also possible to configure `varnishd` for "reverse mode", using
 the '-M' argument.  In that case `varnishd` will attempt to open a
@@ -99,11 +97,9 @@ would be hard to prevent you getting CLI access, wouldn't it ?
 CLI interface authentication
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-By default the CLI interface is protected with a simple, yet
-strong "Pre Shared Key" authentication method, which do not provide
-secrecy (ie: The CLI commands and responses are not encrypted).
-
-.. XXX:Encryption instead of secrecy? benc
+By default the CLI interface is protected with a simple, yet powerful
+"Pre Shared Key" authentication method, which do not provide secrecy
+(ie: The CLI commands and responses are not encrypted).
 
 The way -S/PSK works is really simple:  During startup a file is
 created with a random content and the file is only accessible to
@@ -111,18 +107,14 @@ the user who started `varnishd` (or the superuser).
 
 To authenticate and use a CLI connection, you need to know the
 contents of that file, in order to answer the cryptographic
-challenge `varnishd` issues. 
-
-
-(XXX: xref to algo in refman)
-.. XXX:Dunno what this is? benc
+challenge `varnishd` issues, see :ref:`ref_psk_auth`.
 
 `varnishadm` uses all of this to restrict access, it will only function,
 provided it can read the secret file.
 
-If you want to allow other users, local or remote, to be able to access CLI connections, you must create your
-own secret file and make it possible for (only!) these users to
-read it.
+If you want to allow other users, local or remote, to be able to access
+CLI connections, you must create your own secret file and make it possible
+for (only!) these users to read it.
 
 A good way to create the secret file is::
 
@@ -168,7 +160,7 @@ Furthermore you may want to look at and lock down:
 :ref:`ref_param_vcc_unsafe_path`
 	Restrict VCL/VMODS to :ref:`ref_param_vcl_dir` and :ref:`ref_param_vmod_dir`
 
-:ref:`ref_param_vmod_dir` 
+:ref:`ref_param_vmod_dir`
         The directory where Varnish will will look
         for modules. This could potentially be used to load rouge
         modules into Varnish.
@@ -188,13 +180,9 @@ We do not currently have a way to restrict specific CLI commands
 to specific CLI connections. One way to get such an effect is to
 "wrap" all CLI access in pre-approved scripts which use `varnishadm(1)`
 
-.. XXX:what does the 1 stand for? benc
-
 to submit the sanitized CLI commands, and restrict a remote user
 to only those scripts, for instance using sshd(8)'s configuration.
 
-.. XXX:what does the 8 stand for? benc
-
 VCL programs
 ------------
 
@@ -214,9 +202,9 @@ lower the privilege of a child process...
 Inline-C is disabled by default starting with Varnish version 4, so unless
 you enable it, you don't have to worry about it.
 
-The parameters mentioned above can restrict the loading of VMODs to only 
-be loaded from a designated directory, restricting VCL wranglers
-to a pre-approved subset of VMODs.
+The parameters mentioned above can restrict the loading of VMODs to only
+be loaded from a designated directory, restricting VCL wranglers to a
+pre-approved subset of VMODs.
 
 If you do that, we are confident that your local system cannot be compromised
 from VCL code.



From lkarsten at varnish-software.com  Thu Jan 14 14:15:00 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:00 +0100
Subject: [4.1] 4fe739e This requires feature user_varnish as well
Message-ID: <E1aJig0-0004wq-JA@project.varnish-software.com>


commit 4fe739ee4f1258529ea11d71992f1997dac9d40a
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Wed Oct 14 09:58:57 2015 +0100

    This requires feature user_varnish as well

diff --git a/bin/varnishtest/tests/j00002.vtc b/bin/varnishtest/tests/j00002.vtc
index cb58437..1ab0ade 100644
--- a/bin/varnishtest/tests/j00002.vtc
+++ b/bin/varnishtest/tests/j00002.vtc
@@ -6,4 +6,7 @@ err_shell "unknown sub-argument" "${varnishd} -junix,bla=foo 2>&1"
 err_shell "user not found" "${varnishd} -junix,user=/// 2>&1"
 err_shell "user not found" "${varnishd} -junix,workuser=/// 2>&1"
 err_shell "group not found" "${varnishd} -junix,ccgroup=/// 2>&1"
+
+feature user_varnish
+
 err_shell "have different login groups" "${varnishd} -junix,workuser=root 2>&1"



From lkarsten at varnish-software.com  Thu Jan 14 14:15:00 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:00 +0100
Subject: [4.1] a999fee Expose accept_filter parameter to Linux
Message-ID: <E1aJig0-0004wz-LQ@project.varnish-software.com>


commit a999fee90d2b4a13fe9d559ab5907067590579e2
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Tue Oct 13 17:11:13 2015 +0100

    Expose accept_filter parameter to Linux
    
    Disabled by default. Prompted by Rafael Zalamena
    (rafaelfz at taghos dot com dot br) on varnish-dev.

diff --git a/include/tbl/params.h b/include/tbl/params.h
index d3a2982..c7f1479 100644
--- a/include/tbl/params.h
+++ b/include/tbl/params.h
@@ -30,21 +30,23 @@
 
 /*lint -save -e525 -e539 */
 
-#ifdef HAVE_ACCEPT_FILTERS
 PARAM(
 	/* name */	accept_filter,
 	/* typ */	bool,
 	/* min */	NULL,
 	/* max */	NULL,
+#if defined(HAVE_ACCEPT_FILTERS) || defined(__linux)
 	/* default */	"on",
+#else
+	/* default */	"off",
+#endif /* HAVE_ACCEPT_FILTERS || __linux */
 	/* units */	"bool",
 	/* flags */	MUST_RESTART,
 	/* s-text */
-	"Enable kernel accept-filters, (if available in the kernel).",
+	"Enable kernel accept-filters (if available in the kernel).",
 	/* l-text */	NULL,
 	/* func */	NULL
 )
-#endif /* HAVE_ACCEPT_FILTERS */
 
 PARAM(
 	/* name */	acceptor_sleep_decay,



From lkarsten at varnish-software.com  Thu Jan 14 14:15:00 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:00 +0100
Subject: [4.1] 04c0218 Really expose accept_filter parameter to Linux
Message-ID: <E1aJig0-0004x8-N8@project.varnish-software.com>


commit 04c021894ed315fc54e4289e7eba748990e9fca3
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Wed Oct 14 22:33:26 2015 +0100

    Really expose accept_filter parameter to Linux
    
    This was missing on my previous commit.

diff --git a/bin/varnishd/cache/cache_acceptor.c b/bin/varnishd/cache/cache_acceptor.c
index 9736ca9..a17351d 100644
--- a/bin/varnishd/cache/cache_acceptor.c
+++ b/bin/varnishd/cache/cache_acceptor.c
@@ -489,7 +489,6 @@ vca_acct(void *arg)
 		assert (ls->sock > 0);		// We know where stdin is
 		AZ(listen(ls->sock, cache_param->listen_depth));
 		vca_tcp_opt_set(ls->sock, 1);
-#ifdef HAVE_ACCEPT_FILTERS
 		if (cache_param->accept_filter) {
 			int i;
 			i = VTCP_filter_http(ls->sock);
@@ -498,7 +497,6 @@ vca_acct(void *arg)
 				    "Kernel filtering: sock=%d, ret=%d %s",
 				    ls->sock, i, strerror(errno));
 		}
-#endif /* HAVE_ACCEPT_FILTERS */
 	}
 
 	need_test = 1;



From lkarsten at varnish-software.com  Thu Jan 14 14:15:00 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:00 +0100
Subject: [4.1] 346ea97 Add some tests using std.ip() with IPv6
Message-ID: <E1aJig0-0004xH-P7@project.varnish-software.com>


commit 346ea9760c54640fcd6bdec50d9a0057f771552e
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Wed Oct 14 22:41:50 2015 +0100

    Add some tests using std.ip() with IPv6

diff --git a/bin/varnishtest/tests/m00011.vtc b/bin/varnishtest/tests/m00011.vtc
index 29c4261..bf96dd5 100644
--- a/bin/varnishtest/tests/m00011.vtc
+++ b/bin/varnishtest/tests/m00011.vtc
@@ -16,6 +16,8 @@ varnish v1 -arg "-Wpoll" -vcl+backend {
 		set resp.http.foo1 = std.ip("9.9.9.*", server.ip);
 		set resp.http.foo2 = std.ip("1.2.3.*", "127.0.0.2");
 		set resp.http.foo3 = std.ip("1.2.3.5", "127.0.0.3");
+		set resp.http.foo4 = std.ip("2001:db8::", "[::1]");
+		set resp.http.foo5 = std.ip("2001::db8::", "[::1]");
 	}
 } -start
 
@@ -27,6 +29,8 @@ client c1 {
 	expect resp.http.foo1 == "127.0.0.1"
 	expect resp.http.foo2 == "127.0.0.2"
 	expect resp.http.foo3 == "1.2.3.5"
+	expect resp.http.foo4 == "2001:db8::"
+	expect resp.http.foo5 == "::1"
 	txreq -url "/foo2"
 	rxresp
 	expect resp.bodylen == 2



From lkarsten at varnish-software.com  Thu Jan 14 14:15:00 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:00 +0100
Subject: [4.1] ffd3735 Document vcl.state and correct existing mention
Message-ID: <E1aJig0-0004xT-Qt@project.varnish-software.com>


commit ffd37357952845336a7e5c3c4997d89480e2754c
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Wed Oct 14 23:13:42 2015 +0100

    Document vcl.state and correct existing mention
    
    Partially addresses #1795.

diff --git a/doc/sphinx/reference/varnish-cli.rst b/doc/sphinx/reference/varnish-cli.rst
index 9a2d0de..934ff1e 100644
--- a/doc/sphinx/reference/varnish-cli.rst
+++ b/doc/sphinx/reference/varnish-cli.rst
@@ -119,6 +119,10 @@ vcl.list
 vcl.show [-v] <configname>
   Display the source code for the specified configuration.
 
+vcl.state <configname> <state>
+  Force the state of the specified configuration.
+  State is any of auto, warm or cold values.
+
 param.show [-l] [<param>]
   Show parameters and their values.
 
diff --git a/doc/sphinx/whats-new/changes.rst b/doc/sphinx/whats-new/changes.rst
index 5077c0a..647b102 100644
--- a/doc/sphinx/whats-new/changes.rst
+++ b/doc/sphinx/whats-new/changes.rst
@@ -44,11 +44,12 @@ Output from `vcl.list`::
     available  auto/warm       0 62f5275f-a937-4df9-9fbb-c12336bdfdb8
 
 
-A single VCL's state can be chanced with the `vcl.state` call in
+A single VCL's state can be changed with the `vcl.state` call in
 ``varnishadm``::
 
-    vcl.state <configname> [auto|cold|warm]
-        Force the state of the named configuration.
+    vcl.state <configname> <state>
+        Force the state of the specified configuration.
+        State is any of auto, warm or cold values.
 
 Example::
 



From lkarsten at varnish-software.com  Thu Jan 14 14:15:00 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:00 +0100
Subject: [4.1] 4fcc6b0 Use `all' for vars marked as all
Message-ID: <E1aJig0-0004xc-Sz@project.varnish-software.com>


commit 4fcc6b026007f0d5688a5283319f3e351efa56e8
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Thu Oct 15 15:24:22 2015 +0100

    Use `all' for vars marked as all
    
    Previously it said `vcl_all' which doesn't exist.  Minor cosmetics while
    I'm here.

diff --git a/lib/libvcc/generate.py b/lib/libvcc/generate.py
index 062d180..f8aeb79 100755
--- a/lib/libvcc/generate.py
+++ b/lib/libvcc/generate.py
@@ -690,8 +690,8 @@ sp_variables = [
 		( 'deliver', 'synth', ), """
 		The HTTP status code that will be returned.
 
-                Assigning a HTTP standardized code to resp.status will also
-                set resp.reason to the corresponding status message.
+		Assigning a HTTP standardized code to resp.status will also
+		set resp.reason to the corresponding status message.
 		"""
 	),
 	('resp.reason',
@@ -1316,6 +1316,8 @@ def rst_where(fo, h, l):
 			ll.append(j)
 		elif j == "backend":
 			ll.append(j)
+		elif j == "all":
+			ll.append(j)
 		else:
 			ll.append("vcl_" + j)
 	for j in ll:



From lkarsten at varnish-software.com  Thu Jan 14 14:15:00 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:00 +0100
Subject: [4.1] b160be4 It is not kosher to have #ifdefs inside macros.
Message-ID: <E1aJig0-0004xm-V4@project.varnish-software.com>


commit b160be42c1624dbcbf5b4e20163f6e94308456f3
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Oct 19 09:43:29 2015 +0000

    It is not kosher to have #ifdefs inside macros.

diff --git a/bin/varnishd/mgt/mgt_main.c b/bin/varnishd/mgt/mgt_main.c
index 899b9ba..5ceb80d 100644
--- a/bin/varnishd/mgt/mgt_main.c
+++ b/bin/varnishd/mgt/mgt_main.c
@@ -395,6 +395,10 @@ init_params(struct cli *cli)
 		MCF_SetDefault("gzip_buffer", "4k");
 	}
 
+#if !defined(HAVE_ACCEPT_FILTERS) || defined(__linux)
+	MCF_SetDefault("accept_filter", "off");
+#endif
+
 	low = sysconf(_SC_THREAD_STACK_MIN);
 	bprintf(stackmin, "%jd", (intmax_t)low);
 	MCF_SetMinimum("thread_pool_stack", stackmin);
diff --git a/include/tbl/params.h b/include/tbl/params.h
index c7f1479..d2650f4 100644
--- a/include/tbl/params.h
+++ b/include/tbl/params.h
@@ -35,11 +35,7 @@ PARAM(
 	/* typ */	bool,
 	/* min */	NULL,
 	/* max */	NULL,
-#if defined(HAVE_ACCEPT_FILTERS) || defined(__linux)
 	/* default */	"on",
-#else
-	/* default */	"off",
-#endif /* HAVE_ACCEPT_FILTERS || __linux */
 	/* units */	"bool",
 	/* flags */	MUST_RESTART,
 	/* s-text */



From lkarsten at varnish-software.com  Thu Jan 14 14:15:01 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:01 +0100
Subject: [4.1] 2cf3fd2 backend threads fail tovcl_backend_error
Message-ID: <E1aJig1-0004xv-0e@project.varnish-software.com>


commit 2cf3fd21d3f379871d93e8ea29ce26b842b557ac
Author: Per Buer <perbu at varnish-software.com>
Date:   Tue Oct 20 13:17:56 2015 +0200

    backend threads fail tovcl_backend_error

diff --git a/doc/sphinx/users-guide/vcl-built-in-subs.rst b/doc/sphinx/users-guide/vcl-built-in-subs.rst
index a1eba28..12d3377 100644
--- a/doc/sphinx/users-guide/vcl-built-in-subs.rst
+++ b/doc/sphinx/users-guide/vcl-built-in-subs.rst
@@ -268,8 +268,8 @@ The `vcl_backend_fetch` subroutine may terminate with calling
 
   ``abandon``
     Abandon the backend request. Unless the backend request was a
-    background fetch, control is passed to :ref:`vcl_synth` on the
-    client side with ``resp.status`` preset to 503.
+    background fetch, control is passed to :ref:`vcl_backend_error`
+    with ``resp.status`` preset to 503.
 
 .. _vcl_backend_response:
 



From lkarsten at varnish-software.com  Thu Jan 14 14:15:01 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:01 +0100
Subject: [4.1] c2c7df3 Filter subdir-include messages.
Message-ID: <E1aJig1-0004y4-3O@project.varnish-software.com>


commit c2c7df3b71d08b88318c3b12a14f4675c9165c7c
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Mon Dec 21 14:51:05 2015 +0100

    Filter subdir-include messages.

diff --git a/autogen.des b/autogen.des
index 0cbbbc0..45698f4 100755
--- a/autogen.des
+++ b/autogen.des
@@ -13,7 +13,8 @@ if [ -f /usr/bin/clang -a "x${CC}" = "x" ] ; then
 fi
 
 rm -f configure
-. ./autogen.sh
+. ./autogen.sh 2>&1 | egrep -v "(subdir-objects|is in a subdirectory)"
+
 
 # autoconf prior to 2.62 has issues with zsh 4.2 and newer
 export CONFIG_SHELL=/bin/sh



From lkarsten at varnish-software.com  Thu Jan 14 14:15:01 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:01 +0100
Subject: [4.1] 23cf15c Polish the heritage structure a little bit,
 and don't pass the panic_string to syslog until we have washed it.
Message-ID: <E1aJig1-0004yD-5C@project.varnish-software.com>


commit 23cf15ca63ff2a76926e1ca0e5a761c59850563d
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Wed Jan 13 13:38:56 2016 +0100

    Polish the heritage structure a little bit, and don't pass the
    panic_string to syslog until we have washed it.
    
    4.1.1 merge note: Initial panic.show output string changed 4.1.0 format,
    to keep scripts looking for panics working across upgrade.

diff --git a/bin/varnishd/cache/cache_vrt_var.c b/bin/varnishd/cache/cache_vrt_var.c
index c51007b..5dd890e 100644
--- a/bin/varnishd/cache/cache_vrt_var.c
+++ b/bin/varnishd/cache/cache_vrt_var.c
@@ -623,7 +623,7 @@ VRT_r_server_identity(VRT_CTX)
 {
 
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
-	if (heritage.identity[0] != '\0')
+	if (heritage.identity != NULL)
 		return (heritage.identity);
 	else
 		return (heritage.name);
diff --git a/bin/varnishd/common/heritage.h b/bin/varnishd/common/heritage.h
index 27c3a33..42e9ec7 100644
--- a/bin/varnishd/common/heritage.h
+++ b/bin/varnishd/common/heritage.h
@@ -56,7 +56,6 @@ struct heritage {
 
 	/* Sockets from which to accept connections */
 	struct listen_sock_head		socks;
-	unsigned			nsocks;
 
 	/* Hash method */
 	const struct hash_slinger	*hash;
@@ -66,7 +65,7 @@ struct heritage {
 	struct params			*param;
 
 	char				*name;
-	char                            identity[1024];
+	const char			*identity;
 
 	char				*panic_str;
 	ssize_t				panic_str_len;
diff --git a/bin/varnishd/mgt/mgt_child.c b/bin/varnishd/mgt/mgt_child.c
index 1d2a7a3..639c5b3 100644
--- a/bin/varnishd/mgt/mgt_child.c
+++ b/bin/varnishd/mgt/mgt_child.c
@@ -125,18 +125,17 @@ mgt_panic_record(pid_t r)
 {
 	char time_str[30];
 
-	AN(heritage.panic_str[0]);
-	REPORT(LOG_ERR, "Child (%jd) Panic message:\n%s",
-	    (intmax_t)r, heritage.panic_str);
-
 	if (child_panic != NULL)
 		VSB_delete(child_panic);
 	child_panic = VSB_new_auto();
 	AN(child_panic);
 	VTIM_format(VTIM_real(), time_str);
 	VSB_printf(child_panic, "Last panic at: %s\n", time_str);
-	VSB_cat(child_panic, heritage.panic_str);
+	VSB_quote(child_panic, heritage.panic_str,
+	    strnlen(heritage.panic_str, heritage.panic_str_len), 0);
 	AZ(VSB_finish(child_panic));
+	REPORT(LOG_ERR, "Child (%jd) %s",
+	    (intmax_t)r, VSB_data(child_panic));
 }
 
 static void
diff --git a/bin/varnishd/mgt/mgt_main.c b/bin/varnishd/mgt/mgt_main.c
index 5ceb80d..5dea3d4 100644
--- a/bin/varnishd/mgt/mgt_main.c
+++ b/bin/varnishd/mgt/mgt_main.c
@@ -424,9 +424,10 @@ identify(const char *i_arg)
 	strcpy(id, "varnishd");
 
 	if (i_arg != NULL) {
-		if (strlen(i_arg) + 1 > sizeof heritage.identity)
-			ARGV_ERR("Identity (-i) name too long.\n");
-		strcpy(heritage.identity, i_arg);
+		if (strlen(i_arg) + 1 > 1024)
+			ARGV_ERR("Identity (-i) name too long (max 1023).\n");
+		heritage.identity = strdup(i_arg);
+		AN(heritage.identity);
 		i = strlen(id);
 		id[i++] = '/';
 		for (; i < (sizeof(id) - 1L); i++) {



From lkarsten at varnish-software.com  Thu Jan 14 14:15:01 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:01 +0100
Subject: [4.1] fa5727e Centralize mgt's include of common/params.h
Message-ID: <E1aJig1-0004yM-8N@project.varnish-software.com>


commit fa5727e50989d2117c24700c45b6179fb1a0d584
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Oct 21 08:42:32 2015 +0000

    Centralize mgt's include of common/params.h

diff --git a/bin/varnishd/mgt/mgt.h b/bin/varnishd/mgt/mgt.h
index 18356dc..3b44172 100644
--- a/bin/varnishd/mgt/mgt.h
+++ b/bin/varnishd/mgt/mgt.h
@@ -31,6 +31,7 @@
 #include <stdint.h>
 
 #include "common/common.h"
+#include "common/params.h"
 
 struct cli;
 struct parspec;
diff --git a/bin/varnishd/mgt/mgt_child.c b/bin/varnishd/mgt/mgt_child.c
index 639c5b3..bde2a5e 100644
--- a/bin/varnishd/mgt/mgt_child.c
+++ b/bin/varnishd/mgt/mgt_child.c
@@ -45,7 +45,6 @@
 
 #include "mgt/mgt.h"
 #include "common/heritage.h"
-#include "common/params.h"
 
 #include "vbm.h"
 #include "vcli.h"
diff --git a/bin/varnishd/mgt/mgt_cli.c b/bin/varnishd/mgt/mgt_cli.c
index da3cf69..125a91d 100644
--- a/bin/varnishd/mgt/mgt_cli.c
+++ b/bin/varnishd/mgt/mgt_cli.c
@@ -45,7 +45,6 @@
 #include <unistd.h>
 
 #include "mgt/mgt.h"
-#include "common/params.h"
 
 #include "vcli.h"
 #include "vcli_common.h"
diff --git a/bin/varnishd/mgt/mgt_jail_solaris.c b/bin/varnishd/mgt/mgt_jail_solaris.c
index 16f3ccc..142a43c 100644
--- a/bin/varnishd/mgt/mgt_jail_solaris.c
+++ b/bin/varnishd/mgt/mgt_jail_solaris.c
@@ -216,7 +216,6 @@
 #include "mgt/mgt.h"
 
 #include "common/heritage.h"
-#include "common/params.h"
 
 #ifdef HAVE_PRIV_H
 #include <priv.h>
diff --git a/bin/varnishd/mgt/mgt_param.c b/bin/varnishd/mgt/mgt_param.c
index 1fd3f33..2a52709 100644
--- a/bin/varnishd/mgt/mgt_param.c
+++ b/bin/varnishd/mgt/mgt_param.c
@@ -36,7 +36,6 @@
 
 #include "mgt/mgt.h"
 #include "common/heritage.h"
-#include "common/params.h"
 
 #include "mgt/mgt_param.h"
 #include "vav.h"
diff --git a/bin/varnishd/mgt/mgt_param_bits.c b/bin/varnishd/mgt/mgt_param_bits.c
index 627c7a9..4713ac6 100644
--- a/bin/varnishd/mgt/mgt_param_bits.c
+++ b/bin/varnishd/mgt/mgt_param_bits.c
@@ -33,7 +33,6 @@
 #include <string.h>
 #include <stdio.h>
 
-#include "common/params.h"
 #include "mgt/mgt.h"
 #include "mgt/mgt_param.h"
 
diff --git a/bin/varnishd/mgt/mgt_param_tbl.c b/bin/varnishd/mgt/mgt_param_tbl.c
index 876b6d3..129300f 100644
--- a/bin/varnishd/mgt/mgt_param_tbl.c
+++ b/bin/varnishd/mgt/mgt_param_tbl.c
@@ -32,7 +32,6 @@
 #include <stdio.h>
 
 #include "mgt/mgt.h"
-#include "common/params.h"
 
 #include "mgt/mgt_param.h"
 
diff --git a/bin/varnishd/mgt/mgt_param_tcp.c b/bin/varnishd/mgt/mgt_param_tcp.c
index 12baf83..ab8f6d4 100644
--- a/bin/varnishd/mgt/mgt_param_tcp.c
+++ b/bin/varnishd/mgt/mgt_param_tcp.c
@@ -44,7 +44,6 @@
 #include <unistd.h>
 
 #include "mgt/mgt.h"
-#include "common/params.h"
 
 #include "vtcp.h"
 
diff --git a/bin/varnishd/mgt/mgt_param_tweak.c b/bin/varnishd/mgt/mgt_param_tweak.c
index b5f653c..00ad35c 100644
--- a/bin/varnishd/mgt/mgt_param_tweak.c
+++ b/bin/varnishd/mgt/mgt_param_tweak.c
@@ -40,7 +40,6 @@
 #include <unistd.h>
 
 #include "mgt/mgt.h"
-#include "common/params.h"
 
 #include "mgt/mgt_param.h"
 #include "vav.h"
diff --git a/bin/varnishd/mgt/mgt_pool.c b/bin/varnishd/mgt/mgt_pool.c
index 7affacd..c08a2af 100644
--- a/bin/varnishd/mgt/mgt_pool.c
+++ b/bin/varnishd/mgt/mgt_pool.c
@@ -45,7 +45,6 @@
 #include <stdio.h>
 
 #include "mgt/mgt.h"
-#include "common/params.h"
 
 #include "mgt/mgt_param.h"
 
diff --git a/bin/varnishd/mgt/mgt_sandbox.c b/bin/varnishd/mgt/mgt_sandbox.c
index 1f0962d..785014c 100644
--- a/bin/varnishd/mgt/mgt_sandbox.c
+++ b/bin/varnishd/mgt/mgt_sandbox.c
@@ -59,7 +59,6 @@
 #include <unistd.h>
 
 #include "mgt/mgt.h"
-#include "common/params.h"
 #include "mgt/mgt_param.h"
 
 #include <vsub.h>
diff --git a/bin/varnishd/mgt/mgt_shmem.c b/bin/varnishd/mgt/mgt_shmem.c
index 1afcfbf..9345d38 100644
--- a/bin/varnishd/mgt/mgt_shmem.c
+++ b/bin/varnishd/mgt/mgt_shmem.c
@@ -43,7 +43,6 @@
 
 #include "mgt/mgt.h"
 #include "common/heritage.h"
-#include "common/params.h"
 
 #include "flopen.h"
 #include "vsm_priv.h"
diff --git a/bin/varnishd/mgt/mgt_vcc.c b/bin/varnishd/mgt/mgt_vcc.c
index 8a72f20..d13432e 100644
--- a/bin/varnishd/mgt/mgt_vcc.c
+++ b/bin/varnishd/mgt/mgt_vcc.c
@@ -40,7 +40,6 @@
 #include <unistd.h>
 #include <sys/stat.h>
 
-#include "common/params.h"
 #include "mgt/mgt.h"
 
 #include "libvcc.h"
diff --git a/bin/varnishd/mgt/mgt_vcl.c b/bin/varnishd/mgt/mgt_vcl.c
index 7971346..bb298bb 100644
--- a/bin/varnishd/mgt/mgt_vcl.c
+++ b/bin/varnishd/mgt/mgt_vcl.c
@@ -39,7 +39,6 @@
 #include <sys/stat.h>
 
 #include "mgt/mgt.h"
-#include "common/params.h"
 
 #include "vcli.h"
 #include "vcli_priv.h"



From lkarsten at varnish-software.com  Thu Jan 14 14:15:01 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:01 +0100
Subject: [4.1] 58d10b5 Go over mgt process's complaining on stderr and syslog
 and try to make somewhat more predictable.
Message-ID: <E1aJig1-0004yb-Cd@project.varnish-software.com>


commit 58d10b59705c958eaeb1f4ad1f06f5af2f2a61ef
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Oct 21 08:48:24 2015 +0000

    Go over mgt process's complaining on stderr and syslog and try to
    make somewhat more predictable.
    
    Disable syslogging when we're running under varnishtest.

diff --git a/bin/varnishd/common/params.h b/bin/varnishd/common/params.h
index 3dbfe6a..44f2f2a 100644
--- a/bin/varnishd/common/params.h
+++ b/bin/varnishd/common/params.h
@@ -29,6 +29,11 @@
  * This file contains the heritage passed when mgt forks cache
  */
 
+#ifdef COMMON_PARAMS_H
+#error "Multiple includes of common/params.h"
+#endif
+#define COMMON_PARAMS_H
+
 #include <stdint.h>
 
 #include "vre.h"
diff --git a/bin/varnishd/mgt/mgt.h b/bin/varnishd/mgt/mgt.h
index 3b44172..5fb12d4 100644
--- a/bin/varnishd/mgt/mgt.h
+++ b/bin/varnishd/mgt/mgt.h
@@ -123,6 +123,13 @@ struct choice {
 };
 const void *pick(const struct choice *cp, const char *which, const char *kind);
 
+extern const char C_ERR[];	// Things are not as they should be
+extern const char C_INFO[];	// Normal stuff, keep a record for later
+extern const char C_DEBUG[];	// More detail than you'd normally want
+extern const char C_SECURITY[];	// Security issues
+extern const char C_CLI[];	// CLI traffic between master and child
+void MGT_complain(const char *loud, const char *, ...) __v_printflike(2, 3);
+
 /* mgt_param.c */
 void MCF_InitParams(struct cli *);
 void MCF_CollectParams(void);
@@ -169,18 +176,20 @@ extern unsigned mgt_vcc_err_unref;
 extern unsigned mgt_vcc_allow_inline_c;
 extern unsigned mgt_vcc_unsafe_path;
 
-#define REPORT0(pri, fmt)				\
-	do {						\
-		fprintf(stderr, fmt "\n");		\
-		syslog(pri, fmt);			\
-	} while (0)
-
-#define REPORT(pri, fmt, ...)				\
-	do {						\
-		fprintf(stderr, fmt "\n", __VA_ARGS__);	\
-		syslog(pri, fmt, __VA_ARGS__);		\
-	} while (0)
-
 #if defined(PTHREAD_CANCELED) || defined(PTHREAD_MUTEX_DEFAULT)
 #error "Keep pthreads out of in manager process"
 #endif
+
+static inline int
+MGT_FEATURE(enum feature_bits x)
+{
+	return (mgt_param.feature_bits[(unsigned)x>>3] &
+	    (0x80U >> ((unsigned)x & 7)));
+}
+
+static inline int
+MGT_DO_DEBUG(enum debug_bits x)
+{
+	return (mgt_param.debug_bits[(unsigned)x>>3] &
+	    (0x80U >> ((unsigned)x & 7)));
+}
diff --git a/bin/varnishd/mgt/mgt_child.c b/bin/varnishd/mgt/mgt_child.c
index bde2a5e..368d359 100644
--- a/bin/varnishd/mgt/mgt_child.c
+++ b/bin/varnishd/mgt/mgt_child.c
@@ -86,20 +86,6 @@ static struct vlu	*child_std_vlu;
 static struct vsb *child_panic = NULL;
 static double mgt_uptime_t0 = 0.;
 
-/* XXX: Doesn't really belong here, but only place we use it */
-static inline int
-MGT_FEATURE(enum feature_bits x)
-{
-	return (mgt_param.feature_bits[(unsigned)x>>3] &
-	    (0x80U >> ((unsigned)x & 7)));
-}
-static inline int
-MGT_DO_DEBUG(enum debug_bits x)
-{
-	return (mgt_param.debug_bits[(unsigned)x>>3] &
-	    (0x80U >> ((unsigned)x & 7)));
-}
-
 static void mgt_reap_child(void);
 
 /*---------------------------------------------------------------------
@@ -133,7 +119,7 @@ mgt_panic_record(pid_t r)
 	VSB_quote(child_panic, heritage.panic_str,
 	    strnlen(heritage.panic_str, heritage.panic_str_len), 0);
 	AZ(VSB_finish(child_panic));
-	REPORT(LOG_ERR, "Child (%jd) %s",
+	MGT_complain(C_ERR, "Child (%jd) %s",
 	    (intmax_t)r, VSB_data(child_panic));
 }
 
@@ -237,7 +223,7 @@ child_line(void *priv, const char *p)
 {
 	(void)priv;
 
-	REPORT(LOG_NOTICE, "Child (%jd) said %s", (intmax_t)child_pid, p);
+	MGT_complain(C_INFO, "Child (%jd) said %s", (intmax_t)child_pid, p);
 	return (0);
 }
 
@@ -320,7 +306,7 @@ mgt_launch_child(struct cli *cli)
 			VCLI_SetResult(cli, CLIS_CANT);
 			return;
 		}
-		REPORT0(LOG_ERR,
+		MGT_complain(C_ERR,
 		    "Child start failed: could not open sockets");
 		return;
 	}
@@ -394,7 +380,7 @@ mgt_launch_child(struct cli *cli)
 		exit(0);
 	}
 	assert(pid > 1);
-	REPORT(LOG_NOTICE, "child (%jd) Started", (intmax_t)pid);
+	MGT_complain(C_DEBUG, "Child (%jd) Started", (intmax_t)pid);
 	VSC_C_mgt->child_start = ++static_VSC_C_mgt.child_start;
 
 	/* Close stuff the child got */
@@ -432,7 +418,8 @@ mgt_launch_child(struct cli *cli)
 	mgt_cli_start_child(child_cli_in, child_cli_out);
 	child_pid = pid;
 	if (mgt_push_vcls_and_start(&u, &p)) {
-		REPORT(LOG_ERR, "Pushing vcls failed:\n%s", p);
+		MGT_complain(C_ERR, "Child (%jd) Pushing vcls failed:\n%s",
+		    (intmax_t)child_pid, p);
 		free(p);
 		child_state = CH_RUNNING;
 		mgt_stop_child();
@@ -502,7 +489,8 @@ mgt_reap_child(void)
 	/* Compose obituary */
 	vsb = VSB_new_auto();
 	XXXAN(vsb);
-	VSB_printf(vsb, "Child (%ld) %s", (long)r, status ? "died" : "ended");
+	VSB_printf(vsb, "Child (%jd) %s", (intmax_t)r,
+	    status ? "died" : "ended");
 	if (WIFEXITED(status) && WEXITSTATUS(status)) {
 		VSB_printf(vsb, " status=%d", WEXITSTATUS(status));
 		exit_status |= 0x20;
@@ -524,7 +512,7 @@ mgt_reap_child(void)
 	}
 #endif
 	AZ(VSB_finish(vsb));
-	REPORT(LOG_INFO, "%s", VSB_data(vsb));
+	MGT_complain(status ? C_ERR : C_INFO, "%s", VSB_data(vsb));
 	VSB_delete(vsb);
 
 	/* Dispose of shared memory but evacuate panic messages first */
@@ -548,7 +536,7 @@ mgt_reap_child(void)
 
 	child_pid = -1;
 
-	REPORT0(LOG_DEBUG, "Child cleanup complete");
+	MGT_complain(C_DEBUG, "Child cleanup complete");
 
 	if (child_state == CH_DIED && mgt_param.auto_restart)
 		mgt_launch_child(NULL);
@@ -576,7 +564,7 @@ MGT_Child_Cli_Fail(void)
 		return;
 	if (child_pid < 0)
 		return;
-	REPORT(LOG_ERR, "Child (%jd) not responding to CLI, killing it.",
+	MGT_complain(C_ERR, "Child (%jd) not responding to CLI, killing it.",
 	    (intmax_t)child_pid);
 	if (MGT_FEATURE(FEATURE_NO_COREDUMP))
 		(void)kill(child_pid, SIGKILL);
@@ -599,7 +587,7 @@ mgt_stop_child(void)
 
 	child_state = CH_STOPPING;
 
-	REPORT0(LOG_DEBUG, "Stopping Child");
+	MGT_complain(C_DEBUG, "Stopping Child");
 
 	mgt_reap_child();
 }
@@ -646,7 +634,7 @@ mgt_sigint(const struct vev *e, int what)
 
 	(void)e;
 	(void)what;
-	REPORT0(LOG_ERR, "Manager got SIGINT");
+	MGT_complain(C_ERR, "Manager got SIGINT");
 	(void)fflush(stdout);
 	if (child_pid >= 0)
 		mgt_stop_child();
@@ -716,7 +704,7 @@ MGT_Run(void)
 	AZ(sigaction(SIGHUP, &sac, NULL));
 
 	if (!d_flag && !mgt_has_vcl())
-		REPORT0(LOG_ERR, "No VCL loaded yet");
+		MGT_complain(C_ERR, "No VCL loaded yet");
 	else if (!d_flag) {
 		mgt_launch_child(NULL);
 		if (child_state != CH_RUNNING) {
@@ -730,7 +718,7 @@ MGT_Run(void)
 
 	i = vev_schedule(mgt_evb);
 	if (i != 0)
-		REPORT(LOG_ERR, "vev_schedule() = %d", i);
+		MGT_complain(C_ERR, "vev_schedule() = %d", i);
 
-	REPORT0(LOG_ERR, "manager dies");
+	MGT_complain(C_INFO, "manager dies");
 }
diff --git a/bin/varnishd/mgt/mgt_cli.c b/bin/varnishd/mgt/mgt_cli.c
index 125a91d..fa021d6 100644
--- a/bin/varnishd/mgt/mgt_cli.c
+++ b/bin/varnishd/mgt/mgt_cli.c
@@ -41,7 +41,6 @@
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
-#include <syslog.h>
 #include <unistd.h>
 
 #include "mgt/mgt.h"
@@ -58,10 +57,6 @@
 
 #include "mgt_cli.h"
 
-#ifndef LOG_AUTHPRIV
-#  define LOG_AUTHPRIV	0
-#endif
-
 static int		cli_i = -1, cli_o = -1;
 static struct VCLS	*cls;
 static const char	*secret_file;
@@ -301,7 +296,7 @@ mcf_auth(struct cli *cli, const char *const *av, void *priv)
 	VCLI_AuthResponse(fd, cli->challenge, buf);
 	AZ(close(fd));
 	if (strcasecmp(buf, av[2])) {
-		syslog(LOG_WARNING|LOG_AUTHPRIV,
+		MGT_complain(C_SECURITY,
 		    "CLI Authentication failure from %s", cli->ident);
 		VCLI_SetResult(cli, CLIS_CLOSE);
 		return;
@@ -321,21 +316,20 @@ static struct cli_proto cli_auth[] = {
 };
 
 /*--------------------------------------------------------------------*/
+
 static void
 mgt_cli_cb_before(const struct cli *cli)
 {
 
-	if (mgt_param.syslog_cli_traffic)
-		syslog(LOG_NOTICE, "CLI %s Rd %s", cli->ident, cli->cmd);
+	MGT_complain(C_CLI, "CLI %s Rd %s", cli->ident, cli->cmd);
 }
 
 static void
 mgt_cli_cb_after(const struct cli *cli)
 {
 
-	if (mgt_param.syslog_cli_traffic)
-		syslog(LOG_NOTICE, "CLI %s Wr %03u %s",
-		    cli->ident, cli->result, VSB_data(cli->sb));
+	MGT_complain(C_CLI, "CLI %s Wr %03u %s",
+	    cli->ident, cli->result, VSB_data(cli->sb));
 }
 
 /*--------------------------------------------------------------------*/
@@ -610,7 +604,7 @@ Marg_connect(const struct vev *e, int what)
 
 	M_fd = VTCP_connected(M_fd);
 	if (M_fd < 0) {
-		syslog(LOG_INFO, "Could not connect to CLI-master: %m");
+		MGT_complain(C_INFO, "Could not connect to CLI-master: %m");
 		ma = VTAILQ_FIRST(&m_addr_list);
 		AN(ma);
 		VTAILQ_REMOVE(&m_addr_list, ma, list);
diff --git a/bin/varnishd/mgt/mgt_jail_solaris.c b/bin/varnishd/mgt/mgt_jail_solaris.c
index 142a43c..bb90598 100644
--- a/bin/varnishd/mgt/mgt_jail_solaris.c
+++ b/bin/varnishd/mgt/mgt_jail_solaris.c
@@ -210,7 +210,6 @@
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
-#include <syslog.h>
 #include <unistd.h>
 
 #include "mgt/mgt.h"
@@ -390,7 +389,7 @@ vjs_setup(enum jail_gen_e jge)
 	priv_set_t *priv_all;
 
 	if (! (priv_all = priv_allocset())) {
-		REPORT(LOG_ERR,
+		MGT_complain(C_SECURITY,
 		    "Solaris Jail warning: "
 		    " vjs_setup - priv_allocset failed: errno=%d (%s)",
 		    errno, strerror(errno));
@@ -423,7 +422,7 @@ vjs_privsep(enum jail_gen_e jge)
 		if (getuid() != mgt_param.uid)
 			XXXAZ(setuid(mgt_param.uid));
 	} else {
-		REPORT(LOG_INFO,
+		MGT_complain(C_SECURITY,
 		    "Privilege %s missing, will not change uid/gid",
 		    PRIV_PROC_SETID);
 	}
@@ -454,7 +453,7 @@ vjs_waive(enum jail_gen_e jge)
 	    !(inheritable = priv_allocset()) ||
 	    !(permitted = priv_allocset()) ||
 	    !(limited = priv_allocset())) {
-		REPORT(LOG_ERR,
+		MGT_complain(C_SECURITY,
 		    "Solaris Jail warning: "
 		    " vjs_waive - priv_allocset failed: errno=%d (%s)",
 		    errno, strerror(errno));
diff --git a/bin/varnishd/mgt/mgt_jail_unix.c b/bin/varnishd/mgt/mgt_jail_unix.c
index 3b3ce12..edb4653 100644
--- a/bin/varnishd/mgt/mgt_jail_unix.c
+++ b/bin/varnishd/mgt/mgt_jail_unix.c
@@ -43,7 +43,6 @@
 #include "mgt/mgt.h"
 
 #ifdef __linux__
-#include <syslog.h>
 #include <sys/prctl.h>
 #endif
 
@@ -231,7 +230,7 @@ vju_subproc(enum jail_subproc_e jse)
 	 * reenable them again.
 	 */
 	if (prctl(PR_SET_DUMPABLE, 1) != 0) {
-		REPORT0(LOG_INFO,
+		MGT_complain(C_INFO,
 		    "Could not set dumpable bit.  Core dumps turned off\n");
 	}
 #endif
diff --git a/bin/varnishd/mgt/mgt_main.c b/bin/varnishd/mgt/mgt_main.c
index 5dea3d4..1073b8e 100644
--- a/bin/varnishd/mgt/mgt_main.c
+++ b/bin/varnishd/mgt/mgt_main.c
@@ -37,6 +37,7 @@
 #include <errno.h>
 #include <fcntl.h>
 #include <stdio.h>
+#include <stdarg.h>
 #include <stdlib.h>
 #include <string.h>
 #include <syslog.h>
@@ -120,6 +121,57 @@ build_vident(void)
 	}
 }
 
+/*--------------------------------------------------------------------
+ * 'Ello, I wish to register a complaint...
+ */
+
+#ifndef LOG_AUTHPRIV
+#  define LOG_AUTHPRIV 0
+#endif
+
+const char C_ERR[] = "Error:";
+const char C_INFO[] = "Info:";
+const char C_DEBUG[] = "Debug:";
+const char C_SECURITY[] = "Security:";
+const char C_CLI[] = "Cli:";
+
+void
+MGT_complain(const char *loud, const char *fmt, ...)
+{
+	va_list ap;
+	struct vsb *vsb;
+	int sf;
+
+	if (loud == C_CLI && !mgt_param.syslog_cli_traffic)
+		return;
+	vsb = VSB_new_auto();
+	AN(vsb);
+	va_start(ap, fmt);
+	VSB_vprintf(vsb, fmt, ap);
+	va_end(ap);
+	AZ(VSB_finish(vsb));
+
+	if (loud == C_ERR)
+		sf = LOG_ERR;
+	else if (loud == C_INFO)
+		sf = LOG_INFO;
+	else if (loud == C_DEBUG)
+		sf = LOG_DEBUG;
+	else if (loud == C_SECURITY)
+		sf = LOG_WARNING | LOG_AUTHPRIV;
+	else if (loud == C_CLI)
+		sf = LOG_INFO;
+	else
+		WRONG("Wrong complaint loudness");
+
+	if (loud != C_CLI)
+		fprintf(stderr, "%s %s\n", loud, VSB_data(vsb));
+
+	if (!MGT_DO_DEBUG(DBG_VTC_MODE))
+		syslog(sf, "%s", VSB_data(vsb));
+	VSB_delete(vsb);
+}
+
 /*--------------------------------------------------------------------*/
 
 const void *
@@ -755,9 +807,7 @@ main(int argc, char * const *argv)
 
 	assert(pfh == NULL || !VPF_Write(pfh));
 
-	if (d_flag)
-		fprintf(stderr, "Platform: %s\n", VSB_data(vident) + 1);
-	syslog(LOG_NOTICE, "Platform: %s\n", VSB_data(vident) + 1);
+	MGT_complain(C_DEBUG, "Platform: %s\n", VSB_data(vident) + 1);
 
 	mgt_pid = getpid();	/* daemon() changed this */
 
diff --git a/bin/varnishd/mgt/mgt_sandbox.c b/bin/varnishd/mgt/mgt_sandbox.c
index 785014c..2b89a38 100644
--- a/bin/varnishd/mgt/mgt_sandbox.c
+++ b/bin/varnishd/mgt/mgt_sandbox.c
@@ -54,7 +54,6 @@
 #include <grp.h>
 #include <stdio.h>
 #include <stdlib.h>
-#include <syslog.h>
 #include <string.h>
 #include <unistd.h>
 
@@ -236,7 +235,7 @@ mgt_sandbox_unix(enum sandbox_e who)
 	 * reenable them again.
 	 */
 	if (prctl(PR_SET_DUMPABLE, 1) != 0) {
-		REPORT0(LOG_INFO,
+		MGT_complain(C_INFO,
 		    "Could not set dumpable bit.  Core dumps turned off\n");
 	}
 #endif
@@ -277,10 +276,10 @@ mgt_sandbox_init(void)
 	subs = VSUB_run(sb, run_sandbox_test, NULL, "SANDBOX-test", 10);
 	VSB_delete(sb);
 	if (subs) {
-		REPORT0(LOG_INFO, "Warning: init of platform-specific sandbox "
-		    "failed - sandboxing disabled");
-		REPORT0(LOG_INFO, "Warning: Varnish might run with elevated "
-		    "privileges");
+		MGT_complain(C_SECURITY,
+		    "Platform-specific sandbox failed - sandboxing disabled");
+		MGT_complain(C_SECURITY,
+		    "Varnish runs with elevated privileges");
 		mgt_sandbox = mgt_sandbox_null;
 	}
 



From lkarsten at varnish-software.com  Thu Jan 14 14:15:01 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:01 +0100
Subject: [4.1] da3d71e Revert "backend threads fail tovcl_backend_error"
Message-ID: <E1aJig1-0004yi-Eb@project.varnish-software.com>


commit da3d71e8bd3ec76ea7018b33dfd4efb0821dfaff
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Thu Oct 22 08:35:58 2015 +0100

    Revert "backend threads fail tovcl_backend_error"
    
    This reverts commit ed6dca6b9ced699591470ddae65d3fab862d1b73.
    abandon in vcl_backend_* will take us to vcl_synth{}.

diff --git a/doc/sphinx/users-guide/vcl-built-in-subs.rst b/doc/sphinx/users-guide/vcl-built-in-subs.rst
index 12d3377..a1eba28 100644
--- a/doc/sphinx/users-guide/vcl-built-in-subs.rst
+++ b/doc/sphinx/users-guide/vcl-built-in-subs.rst
@@ -268,8 +268,8 @@ The `vcl_backend_fetch` subroutine may terminate with calling
 
   ``abandon``
     Abandon the backend request. Unless the backend request was a
-    background fetch, control is passed to :ref:`vcl_backend_error`
-    with ``resp.status`` preset to 503.
+    background fetch, control is passed to :ref:`vcl_synth` on the
+    client side with ``resp.status`` preset to 503.
 
 .. _vcl_backend_response:
 



From lkarsten at varnish-software.com  Thu Jan 14 14:15:01 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:01 +0100
Subject: [4.1] e044ee1 Split ban_lurker into separate source file.
Message-ID: <E1aJig1-0004ys-GL@project.varnish-software.com>


commit e044ee151f3b269180742389068323aab9e3e5ee
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu Oct 22 05:44:51 2015 +0000

    Split ban_lurker into separate source file.
    
    No functional changes.

diff --git a/bin/varnishd/Makefile.am b/bin/varnishd/Makefile.am
index e4d2ab5..86906a2 100644
--- a/bin/varnishd/Makefile.am
+++ b/bin/varnishd/Makefile.am
@@ -15,6 +15,7 @@ varnishd_SOURCES = \
 	cache/cache_backend_probe.c \
 	cache/cache_backend_tcp.c \
 	cache/cache_ban.c \
+	cache/cache_ban_lurker.c \
 	cache/cache_busyobj.c \
 	cache/cache_cli.c \
 	cache/cache_deliver_proc.c \
@@ -99,6 +100,7 @@ varnishd_SOURCES = \
 
 noinst_HEADERS = \
 	builtin_vcl.h \
+	cache/cache_ban.h \
 	cache/cache_esi.h \
 	cache/cache_pool.h \
 	cache/cache_priv.h \
diff --git a/bin/varnishd/cache/cache_ban.c b/bin/varnishd/cache/cache_ban.c
index f63c8b4..016c77c 100644
--- a/bin/varnishd/cache/cache_ban.c
+++ b/bin/varnishd/cache/cache_ban.c
@@ -26,38 +26,6 @@
  * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
  * SUCH DAMAGE.
  *
- * Ban processing
- *
- * A ban consists of a number of conditions (or tests), all of which must be
- * satisfied.  Here are some potential bans we could support:
- *
- *	req.url == "/foo"
- *	req.url ~ ".iso" && obj.size > 10MB
- *	req.http.host ~ "web1.com" && obj.http.set-cookie ~ "USER=29293"
- *
- * We make the "&&" mandatory from the start, leaving the syntax space
- * for latter handling of "||" as well.
- *
- * Bans are compiled into bytestrings as follows:
- *	8 bytes	- double: timestamp		XXX: Byteorder ?
- *	4 bytes - be32: length
- *	1 byte - flags: 0x01: BAN_F_{REQ|OBJ|COMPLETED}
- *	N tests
- * A test have this form:
- *	1 byte - arg (see ban_vars.h col 3 "BANS_ARG_XXX")
- *	(n bytes) - http header name, canonical encoding
- *	lump - comparison arg
- *	1 byte - operation (BANS_OPER_)
- *	(lump) - compiled regexp
- * A lump is:
- *	4 bytes - be32: length
- *	n bytes - content
- *
- * In a perfect world, we should vector through VRE to get to PCRE,
- * but since we rely on PCRE's ability to encode the regexp into a
- * byte string, that would be a little bit artificial, so this is
- * the exception that confirms the rule.
- *
  */
 
 #include "config.h"
@@ -65,6 +33,7 @@
 #include <pcre.h>
 
 #include "cache.h"
+#include "cache_ban.h"
 
 #include "hash/hash_slinger.h"
 #include "vcli.h"
@@ -73,66 +42,13 @@
 #include "vmb.h"
 #include "vtim.h"
 
-/*--------------------------------------------------------------------
- * BAN string defines & magic markers
- */
-
-#define BANS_TIMESTAMP		0
-#define BANS_LENGTH		8
-#define BANS_FLAGS		12
-#define BANS_HEAD_LEN		16
-
-#define BANS_FLAG_REQ		(1<<0)
-#define BANS_FLAG_OBJ		(1<<1)
-#define BANS_FLAG_COMPLETED	(1<<2)
-#define BANS_FLAG_HTTP		(1<<3)
-#define BANS_FLAG_ERROR		(1<<4)
-
-#define BANS_OPER_EQ		0x10
-#define BANS_OPER_NEQ		0x11
-#define BANS_OPER_MATCH		0x12
-#define BANS_OPER_NMATCH	0x13
-
-#define BANS_ARG_URL		0x18
-#define BANS_ARG_REQHTTP	0x19
-#define BANS_ARG_OBJHTTP	0x1a
-#define BANS_ARG_OBJSTATUS	0x1b
-
-/*--------------------------------------------------------------------*/
-
-struct ban {
-	unsigned		magic;
-#define BAN_MAGIC		0x700b08ea
-	VTAILQ_ENTRY(ban)	list;
-	VTAILQ_ENTRY(ban)	l_list;
-	int			refcount;
-	unsigned		flags;		/* BANS_FLAG_* */
-
-	VTAILQ_HEAD(,objcore)	objcore;
-	struct vsb		*vsb;
-	uint8_t			*spec;
-};
-
-VTAILQ_HEAD(banhead_s,ban);
+struct lock ban_mtx;
+int ban_shutdown = 0;
+struct banhead_s ban_head = VTAILQ_HEAD_INITIALIZER(ban_head);
+struct ban * volatile ban_start;
 
-struct ban_test {
-	uint8_t			arg1;
-	const char		*arg1_spec;
-	uint8_t			oper;
-	const char		*arg2;
-	const void		*arg2_spec;
-};
-
-static void ban_info(enum baninfo event, const uint8_t *ban, unsigned len);
-static struct banhead_s ban_head = VTAILQ_HEAD_INITIALIZER(ban_head);
-static struct lock ban_mtx;
 static struct ban *ban_magic;
 static pthread_t ban_thread;
-static struct ban * volatile ban_start;
-static struct objcore oc_marker = { .magic = OBJCORE_MAGIC, };
-static bgthread_t ban_lurker;
-static unsigned ban_batch;
-static int ban_shutdown = 0;
 
 /*--------------------------------------------------------------------
  * Variables we can purge on
@@ -231,7 +147,7 @@ BAN_TailDeref(struct ban **bb)
  * Extract time and length from ban-spec
  */
 
-static double
+double
 ban_time(const uint8_t *banspec)
 {
 	double t;
@@ -241,7 +157,7 @@ ban_time(const uint8_t *banspec)
 	return (t);
 }
 
-static unsigned
+unsigned
 ban_len(const uint8_t *banspec)
 {
 	unsigned u;
@@ -264,7 +180,7 @@ ban_equal(const uint8_t *bs1, const uint8_t *bs2)
 	return (!memcmp(bs1 + BANS_LENGTH, bs2 + BANS_LENGTH, u - BANS_LENGTH));
 }
 
-static void
+void
 ban_mark_completed(struct ban *b)
 {
 	unsigned ln;
@@ -677,7 +593,7 @@ ban_export(void)
 	VSC_C_main->bans_persisted_fragmentation = 0;
 }
 
-static void
+void
 ban_info(enum baninfo event, const uint8_t *ban, unsigned len)
 {
 	if (STV_BanInfo(event, ban, len)) {
@@ -799,36 +715,10 @@ BAN_Time(const struct ban *b)
 }
 
 /*--------------------------------------------------------------------
- * All silos have read their bans, ready for action
- */
-
-void
-BAN_Compile(void)
-{
-
-	ASSERT_CLI();
-	AZ(ban_shutdown);
-
-	Lck_Lock(&ban_mtx);
-
-	/* Do late reporting of ban_magic */
-	AZ(STV_BanInfo(BI_NEW, ban_magic->spec, ban_len(ban_magic->spec)));
-
-	/* All bans have been read from all persistent stevedores. Export
-	   the compiled list */
-	ban_export();
-
-	Lck_Unlock(&ban_mtx);
-
-	ban_start = VTAILQ_FIRST(&ban_head);
-	WRK_BgThread(&ban_thread, "ban-lurker", ban_lurker, NULL);
-}
-
-/*--------------------------------------------------------------------
  * Evaluate ban-spec
  */
 
-static int
+int
 ban_evaluate(struct worker *wrk, const uint8_t *bs, struct objcore *oc,
     const struct http *reqhttp, unsigned *tests)
 {
@@ -971,227 +861,6 @@ BAN_CheckObject(struct worker *wrk, struct objcore *oc, struct req *req)
 	}
 }
 
-static void
-ban_cleantail(void)
-{
-	struct ban *b;
-
-	do {
-		Lck_Lock(&ban_mtx);
-		b = VTAILQ_LAST(&ban_head, banhead_s);
-		if (b != VTAILQ_FIRST(&ban_head) && b->refcount == 0) {
-			if (b->flags & BANS_FLAG_COMPLETED)
-				VSC_C_main->bans_completed--;
-			if (b->flags & BANS_FLAG_OBJ)
-				VSC_C_main->bans_obj--;
-			if (b->flags & BANS_FLAG_REQ)
-				VSC_C_main->bans_req--;
-			VSC_C_main->bans--;
-			VSC_C_main->bans_deleted++;
-			VTAILQ_REMOVE(&ban_head, b, list);
-			VSC_C_main->bans_persisted_fragmentation +=
-			    ban_len(b->spec);
-			ban_info(BI_DROP, b->spec, ban_len(b->spec));
-		} else {
-			b = NULL;
-		}
-		Lck_Unlock(&ban_mtx);
-		if (b != NULL)
-			BAN_Free(b);
-	} while (b != NULL);
-}
-
-/*--------------------------------------------------------------------
- * Our task here is somewhat tricky:  The canonical locking order is
- * objhead->mtx first, then ban_mtx, because that is the order which
- * makes most sense in HSH_Lookup(), but we come the other way.
- * We optimistically try to get them the other way, and get out of
- * the way if that fails, and retry again later.
- */
-
-static struct objcore *
-ban_lurker_getfirst(struct vsl_log *vsl, struct ban *bt)
-{
-	struct objhead *oh;
-	struct objcore *oc;
-
-	while (1) {
-		Lck_Lock(&ban_mtx);
-		oc = VTAILQ_FIRST(&bt->objcore);
-		CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
-		if (oc == &oc_marker) {
-			VTAILQ_REMOVE(&bt->objcore, oc, ban_list);
-			Lck_Unlock(&ban_mtx);
-			return (NULL);
-		}
-		oh = oc->objhead;
-		CHECK_OBJ_NOTNULL(oh, OBJHEAD_MAGIC);
-		if (!Lck_Trylock(&oh->mtx)) {
-			if (oc->refcnt == 0) {
-				Lck_Unlock(&oh->mtx);
-			} else {
-				/*
-				 * We got the lock, and the oc is not being
-				 * dismantled under our feet, run with it...
-				 */
-				AZ(oc->flags & OC_F_BUSY);
-				oc->refcnt += 1;
-				VTAILQ_REMOVE(&bt->objcore, oc, ban_list);
-				VTAILQ_INSERT_TAIL(&bt->objcore, oc, ban_list);
-				Lck_Unlock(&oh->mtx);
-				Lck_Unlock(&ban_mtx);
-				break;
-			}
-		}
-
-		/* Try again, later */
-		Lck_Unlock(&ban_mtx);
-		VSC_C_main->bans_lurker_contention++;
-		VSL_Flush(vsl, 0);
-		VTIM_sleep(cache_param->ban_lurker_sleep);
-	}
-	return (oc);
-}
-
-static void
-ban_lurker_test_ban(struct worker *wrk, struct vsl_log *vsl, struct ban *bt,
-    struct banhead_s *obans)
-{
-	struct ban *bl, *bln;
-	struct objcore *oc;
-	unsigned tests;
-	int i;
-
-	/*
-	 * First see if there is anything to do, and if so, insert marker
-	 */
-	Lck_Lock(&ban_mtx);
-	oc = VTAILQ_FIRST(&bt->objcore);
-	if (oc != NULL)
-		VTAILQ_INSERT_TAIL(&bt->objcore, &oc_marker, ban_list);
-	Lck_Unlock(&ban_mtx);
-	if (oc == NULL)
-		return;
-
-	while (1) {
-		if (++ban_batch > cache_param->ban_lurker_batch) {
-			VTIM_sleep(cache_param->ban_lurker_sleep);
-			ban_batch = 0;
-		}
-		oc = ban_lurker_getfirst(vsl, bt);
-		if (oc == NULL)
-			return;
-		i = 0;
-		VTAILQ_FOREACH_REVERSE_SAFE(bl, obans, banhead_s, l_list, bln) {
-			if (bl->flags & BANS_FLAG_COMPLETED) {
-				/* Ban was overtaken by new (dup) ban */
-				VTAILQ_REMOVE(obans, bl, l_list);
-				continue;
-			}
-			tests = 0;
-			i = ban_evaluate(wrk, bl->spec, oc, NULL, &tests);
-			VSC_C_main->bans_lurker_tested++;
-			VSC_C_main->bans_lurker_tests_tested += tests;
-			if (i)
-				break;
-		}
-		if (i) {
-			VSLb(vsl, SLT_ExpBan, "%u banned by lurker",
-			    ObjGetXID(wrk, oc));
-
-			EXP_Rearm(oc, oc->exp.t_origin, 0, 0, 0);
-					// XXX ^ fake now
-			VSC_C_main->bans_lurker_obj_killed++;
-		}
-		(void)HSH_DerefObjCore(wrk, &oc);
-	}
-}
-
-/*--------------------------------------------------------------------
- * Ban lurker thread
- */
-
-static int
-ban_lurker_work(struct worker *wrk, struct vsl_log *vsl)
-{
-	struct ban *b, *bt;
-	struct banhead_s obans;
-	double d;
-	int i;
-
-	/* Make a list of the bans we can do something about */
-	VTAILQ_INIT(&obans);
-	Lck_Lock(&ban_mtx);
-	b = ban_start;
-	Lck_Unlock(&ban_mtx);
-	i = 0;
-	d = VTIM_real() - cache_param->ban_lurker_age;
-	while (b != NULL) {
-		if (b->flags & BANS_FLAG_COMPLETED) {
-			;
-		} else if (b->flags & BANS_FLAG_REQ) {
-			;
-		} else if (b == VTAILQ_LAST(&ban_head, banhead_s)) {
-			;
-		} else if (ban_time(b->spec) > d) {
-			;
-		} else {
-			VTAILQ_INSERT_TAIL(&obans, b, l_list);
-			i++;
-		}
-		b = VTAILQ_NEXT(b, list);
-	}
-	if (DO_DEBUG(DBG_LURKER))
-		VSLb(vsl, SLT_Debug, "lurker: %d actionable bans", i);
-	if (i == 0)
-		return (0);
-
-	/* Go though all the bans to test the objects */
-	VTAILQ_FOREACH_REVERSE(bt, &ban_head, banhead_s, list) {
-		if (bt == VTAILQ_LAST(&obans, banhead_s)) {
-			if (DO_DEBUG(DBG_LURKER))
-				VSLb(vsl, SLT_Debug,
-				    "Lurk bt completed %p", bt);
-			Lck_Lock(&ban_mtx);
-			/* We can be raced by a new ban */
-			if (!(bt->flags & BANS_FLAG_COMPLETED))
-				ban_mark_completed(bt);
-			Lck_Unlock(&ban_mtx);
-			VTAILQ_REMOVE(&obans, bt, l_list);
-			if (VTAILQ_EMPTY(&obans))
-				break;
-		}
-		if (DO_DEBUG(DBG_LURKER))
-			VSLb(vsl, SLT_Debug, "Lurk bt %p", bt);
-		ban_lurker_test_ban(wrk, vsl, bt, &obans);
-		if (VTAILQ_EMPTY(&obans))
-			break;
-	}
-	return (1);
-}
-
-static void * __match_proto__(bgthread_t)
-ban_lurker(struct worker *wrk, void *priv)
-{
-	struct vsl_log vsl;
-	volatile double d;
-
-	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
-	AZ(priv);
-
-	VSL_Setup(&vsl, NULL, 0);
-
-	while (!ban_shutdown) {
-		d = cache_param->ban_lurker_sleep;
-		if (d <= 0.0 || !ban_lurker_work(wrk, &vsl))
-			d = 0.609;	// Random, non-magic
-		ban_cleantail();
-		VTIM_sleep(d);
-	}
-	pthread_exit(0);
-	NEEDLESS_RETURN(NULL);
-}
-
 /*--------------------------------------------------------------------
  * CLI functions to add bans
  */
@@ -1326,6 +995,34 @@ static struct cli_proto ban_cmds[] = {
 	{ NULL }
 };
 
+/*--------------------------------------------------------------------
+ */
+
+void
+BAN_Compile(void)
+{
+
+	/* All bans have been read from all persistent stevedores. Export
+	 * the compiled list
+	 */
+
+	ASSERT_CLI();
+	AZ(ban_shutdown);
+
+	Lck_Lock(&ban_mtx);
+
+	/* Do late reporting of ban_magic */
+	AZ(STV_BanInfo(BI_NEW, ban_magic->spec, ban_len(ban_magic->spec)));
+
+	ban_export();
+
+	Lck_Unlock(&ban_mtx);
+
+	ban_start = VTAILQ_FIRST(&ban_head);
+	WRK_BgThread(&ban_thread, "ban-lurker", ban_lurker, NULL);
+}
+
+
 void
 BAN_Init(void)
 {
diff --git a/bin/varnishd/cache/cache_ban.h b/bin/varnishd/cache/cache_ban.h
new file mode 100644
index 0000000..fae2b80
--- /dev/null
+++ b/bin/varnishd/cache/cache_ban.h
@@ -0,0 +1,124 @@
+/*-
+ * Copyright (c) 2006 Verdens Gang AS
+ * Copyright (c) 2006-2015 Varnish Software AS
+ * All rights reserved.
+ *
+ * Author: Poul-Henning Kamp <phk at phk.freebsd.dk>
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED.  IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ *
+ * Ban processing
+ *
+ * A ban consists of a number of conditions (or tests), all of which must be
+ * satisfied.  Here are some potential bans we could support:
+ *
+ *	req.url == "/foo"
+ *	req.url ~ ".iso" && obj.size > 10MB
+ *	req.http.host ~ "web1.com" && obj.http.set-cookie ~ "USER=29293"
+ *
+ * We make the "&&" mandatory from the start, leaving the syntax space
+ * for latter handling of "||" as well.
+ *
+ * Bans are compiled into bytestrings as follows:
+ *	8 bytes	- double: timestamp		XXX: Byteorder ?
+ *	4 bytes - be32: length
+ *	1 byte - flags: 0x01: BAN_F_{REQ|OBJ|COMPLETED}
+ *	N tests
+ * A test have this form:
+ *	1 byte - arg (see ban_vars.h col 3 "BANS_ARG_XXX")
+ *	(n bytes) - http header name, canonical encoding
+ *	lump - comparison arg
+ *	1 byte - operation (BANS_OPER_)
+ *	(lump) - compiled regexp
+ * A lump is:
+ *	4 bytes - be32: length
+ *	n bytes - content
+ *
+ * In a perfect world, we should vector through VRE to get to PCRE,
+ * but since we rely on PCRE's ability to encode the regexp into a
+ * byte string, that would be a little bit artificial, so this is
+ * the exception that confirms the rule.
+ *
+ */
+
+/*--------------------------------------------------------------------
+ * BAN string defines & magic markers
+ */
+
+#define BANS_TIMESTAMP		0
+#define BANS_LENGTH		8
+#define BANS_FLAGS		12
+#define BANS_HEAD_LEN		16
+
+#define BANS_FLAG_REQ		(1<<0)
+#define BANS_FLAG_OBJ		(1<<1)
+#define BANS_FLAG_COMPLETED	(1<<2)
+#define BANS_FLAG_HTTP		(1<<3)
+#define BANS_FLAG_ERROR		(1<<4)
+
+#define BANS_OPER_EQ		0x10
+#define BANS_OPER_NEQ		0x11
+#define BANS_OPER_MATCH		0x12
+#define BANS_OPER_NMATCH	0x13
+
+#define BANS_ARG_URL		0x18
+#define BANS_ARG_REQHTTP	0x19
+#define BANS_ARG_OBJHTTP	0x1a
+#define BANS_ARG_OBJSTATUS	0x1b
+
+/*--------------------------------------------------------------------*/
+
+struct ban {
+	unsigned		magic;
+#define BAN_MAGIC		0x700b08ea
+	VTAILQ_ENTRY(ban)	list;
+	VTAILQ_ENTRY(ban)	l_list;
+	int			refcount;
+	unsigned		flags;		/* BANS_FLAG_* */
+
+	VTAILQ_HEAD(,objcore)	objcore;
+	struct vsb		*vsb;
+	uint8_t			*spec;
+};
+
+VTAILQ_HEAD(banhead_s,ban);
+
+struct ban_test {
+	uint8_t			arg1;
+	const char		*arg1_spec;
+	uint8_t			oper;
+	const char		*arg2;
+	const void		*arg2_spec;
+};
+
+bgthread_t ban_lurker;
+extern struct lock ban_mtx;
+extern int ban_shutdown;
+extern struct banhead_s ban_head;
+extern struct ban * volatile ban_start;
+
+void ban_mark_completed(struct ban *b);
+unsigned ban_len(const uint8_t *banspec);
+void ban_info(enum baninfo event, const uint8_t *ban, unsigned len);
+int ban_evaluate(struct worker *wrk, const uint8_t *bs, struct objcore *oc,
+    const struct http *reqhttp, unsigned *tests);
+double ban_time(const uint8_t *banspec);
diff --git a/bin/varnishd/cache/cache_ban_lurker.c b/bin/varnishd/cache/cache_ban_lurker.c
new file mode 100644
index 0000000..cfcdd26
--- /dev/null
+++ b/bin/varnishd/cache/cache_ban_lurker.c
@@ -0,0 +1,261 @@
+/*-
+ * Copyright (c) 2006 Verdens Gang AS
+ * Copyright (c) 2006-2015 Varnish Software AS
+ * All rights reserved.
+ *
+ * Author: Poul-Henning Kamp <phk at phk.freebsd.dk>
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED.  IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ *
+ */
+
+#include "config.h"
+
+#include "cache.h"
+#include "cache_ban.h"
+
+#include "hash/hash_slinger.h"
+#include "vtim.h"
+
+static struct objcore oc_marker = { .magic = OBJCORE_MAGIC, };
+static unsigned ban_batch;
+
+static void
+ban_cleantail(void)
+{
+	struct ban *b;
+
+	do {
+		Lck_Lock(&ban_mtx);
+		b = VTAILQ_LAST(&ban_head, banhead_s);
+		if (b != VTAILQ_FIRST(&ban_head) && b->refcount == 0) {
+			if (b->flags & BANS_FLAG_COMPLETED)
+				VSC_C_main->bans_completed--;
+			if (b->flags & BANS_FLAG_OBJ)
+				VSC_C_main->bans_obj--;
+			if (b->flags & BANS_FLAG_REQ)
+				VSC_C_main->bans_req--;
+			VSC_C_main->bans--;
+			VSC_C_main->bans_deleted++;
+			VTAILQ_REMOVE(&ban_head, b, list);
+			VSC_C_main->bans_persisted_fragmentation +=
+			    ban_len(b->spec);
+			ban_info(BI_DROP, b->spec, ban_len(b->spec));
+		} else {
+			b = NULL;
+		}
+		Lck_Unlock(&ban_mtx);
+		if (b != NULL)
+			BAN_Free(b);
+	} while (b != NULL);
+}
+
+/*--------------------------------------------------------------------
+ * Our task here is somewhat tricky:  The canonical locking order is
+ * objhead->mtx first, then ban_mtx, because that is the order which
+ * makes most sense in HSH_Lookup(), but we come the other way.
+ * We optimistically try to get them the other way, and get out of
+ * the way if that fails, and retry again later.
+ */
+
+static struct objcore *
+ban_lurker_getfirst(struct vsl_log *vsl, struct ban *bt)
+{
+	struct objhead *oh;
+	struct objcore *oc;
+
+	while (1) {
+		Lck_Lock(&ban_mtx);
+		oc = VTAILQ_FIRST(&bt->objcore);
+		CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
+		if (oc == &oc_marker) {
+			VTAILQ_REMOVE(&bt->objcore, oc, ban_list);
+			Lck_Unlock(&ban_mtx);
+			return (NULL);
+		}
+		oh = oc->objhead;
+		CHECK_OBJ_NOTNULL(oh, OBJHEAD_MAGIC);
+		if (!Lck_Trylock(&oh->mtx)) {
+			if (oc->refcnt == 0) {
+				Lck_Unlock(&oh->mtx);
+			} else {
+				/*
+				 * We got the lock, and the oc is not being
+				 * dismantled under our feet, run with it...
+				 */
+				AZ(oc->flags & OC_F_BUSY);
+				oc->refcnt += 1;
+				VTAILQ_REMOVE(&bt->objcore, oc, ban_list);
+				VTAILQ_INSERT_TAIL(&bt->objcore, oc, ban_list);
+				Lck_Unlock(&oh->mtx);
+				Lck_Unlock(&ban_mtx);
+				break;
+			}
+		}
+
+		/* Try again, later */
+		Lck_Unlock(&ban_mtx);
+		VSC_C_main->bans_lurker_contention++;
+		VSL_Flush(vsl, 0);
+		VTIM_sleep(cache_param->ban_lurker_sleep);
+	}
+	return (oc);
+}
+
+static void
+ban_lurker_test_ban(struct worker *wrk, struct vsl_log *vsl, struct ban *bt,
+    struct banhead_s *obans)
+{
+	struct ban *bl, *bln;
+	struct objcore *oc;
+	unsigned tests;
+	int i;
+
+	/*
+	 * First see if there is anything to do, and if so, insert marker
+	 */
+	Lck_Lock(&ban_mtx);
+	oc = VTAILQ_FIRST(&bt->objcore);
+	if (oc != NULL)
+		VTAILQ_INSERT_TAIL(&bt->objcore, &oc_marker, ban_list);
+	Lck_Unlock(&ban_mtx);
+	if (oc == NULL)
+		return;
+
+	while (1) {
+		if (++ban_batch > cache_param->ban_lurker_batch) {
+			VTIM_sleep(cache_param->ban_lurker_sleep);
+			ban_batch = 0;
+		}
+		oc = ban_lurker_getfirst(vsl, bt);
+		if (oc == NULL)
+			return;
+		i = 0;
+		VTAILQ_FOREACH_REVERSE_SAFE(bl, obans, banhead_s, l_list, bln) {
+			if (bl->flags & BANS_FLAG_COMPLETED) {
+				/* Ban was overtaken by new (dup) ban */
+				VTAILQ_REMOVE(obans, bl, l_list);
+				continue;
+			}
+			tests = 0;
+			i = ban_evaluate(wrk, bl->spec, oc, NULL, &tests);
+			VSC_C_main->bans_lurker_tested++;
+			VSC_C_main->bans_lurker_tests_tested += tests;
+			if (i)
+				break;
+		}
+		if (i) {
+			VSLb(vsl, SLT_ExpBan, "%u banned by lurker",
+			    ObjGetXID(wrk, oc));
+
+			EXP_Rearm(oc, oc->exp.t_origin, 0, 0, 0);
+					// XXX ^ fake now
+			VSC_C_main->bans_lurker_obj_killed++;
+		}
+		(void)HSH_DerefObjCore(wrk, &oc);
+	}
+}
+
+/*--------------------------------------------------------------------
+ * Ban lurker thread
+ */
+
+static int
+ban_lurker_work(struct worker *wrk, struct vsl_log *vsl)
+{
+	struct ban *b, *bt;
+	struct banhead_s obans;
+	double d;
+	int i;
+
+	/* Make a list of the bans we can do something about */
+	VTAILQ_INIT(&obans);
+	Lck_Lock(&ban_mtx);
+	b = ban_start;
+	Lck_Unlock(&ban_mtx);
+	i = 0;
+	d = VTIM_real() - cache_param->ban_lurker_age;
+	while (b != NULL) {
+		if (b->flags & BANS_FLAG_COMPLETED) {
+			;
+		} else if (b->flags & BANS_FLAG_REQ) {
+			;
+		} else if (b == VTAILQ_LAST(&ban_head, banhead_s)) {
+			;
+		} else if (ban_time(b->spec) > d) {
+			;
+		} else {
+			VTAILQ_INSERT_TAIL(&obans, b, l_list);
+			i++;
+		}
+		b = VTAILQ_NEXT(b, list);
+	}
+	if (DO_DEBUG(DBG_LURKER))
+		VSLb(vsl, SLT_Debug, "lurker: %d actionable bans", i);
+	if (i == 0)
+		return (0);
+
+	/* Go though all the bans to test the objects */
+	VTAILQ_FOREACH_REVERSE(bt, &ban_head, banhead_s, list) {
+		if (bt == VTAILQ_LAST(&obans, banhead_s)) {
+			if (DO_DEBUG(DBG_LURKER))
+				VSLb(vsl, SLT_Debug,
+				    "Lurk bt completed %p", bt);
+			Lck_Lock(&ban_mtx);
+			/* We can be raced by a new ban */
+			if (!(bt->flags & BANS_FLAG_COMPLETED))
+				ban_mark_completed(bt);
+			Lck_Unlock(&ban_mtx);
+			VTAILQ_REMOVE(&obans, bt, l_list);
+			if (VTAILQ_EMPTY(&obans))
+				break;
+		}
+		if (DO_DEBUG(DBG_LURKER))
+			VSLb(vsl, SLT_Debug, "Lurk bt %p", bt);
+		ban_lurker_test_ban(wrk, vsl, bt, &obans);
+		if (VTAILQ_EMPTY(&obans))
+			break;
+	}
+	return (1);
+}
+
+void * __match_proto__(bgthread_t)
+ban_lurker(struct worker *wrk, void *priv)
+{
+	struct vsl_log vsl;
+	volatile double d;
+
+	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
+	AZ(priv);
+
+	VSL_Setup(&vsl, NULL, 0);
+
+	while (!ban_shutdown) {
+		d = cache_param->ban_lurker_sleep;
+		if (d <= 0.0 || !ban_lurker_work(wrk, &vsl))
+			d = 0.609;	// Random, non-magic
+		ban_cleantail();
+		VTIM_sleep(d);
+	}
+	pthread_exit(0);
+	NEEDLESS_RETURN(NULL);
+}



From lkarsten at varnish-software.com  Thu Jan 14 14:15:01 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:01 +0100
Subject: [4.1] 86b3fbf struct ban_test is only used in cache_ban.c
Message-ID: <E1aJig1-0004zN-Kp@project.varnish-software.com>


commit 86b3fbfd86ac822bf230dcebba2fcc8daae327a4
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu Oct 22 05:47:28 2015 +0000

    struct ban_test is only used in cache_ban.c

diff --git a/bin/varnishd/cache/cache_ban.c b/bin/varnishd/cache/cache_ban.c
index 016c77c..8d6337c 100644
--- a/bin/varnishd/cache/cache_ban.c
+++ b/bin/varnishd/cache/cache_ban.c
@@ -50,6 +50,14 @@ struct ban * volatile ban_start;
 static struct ban *ban_magic;
 static pthread_t ban_thread;
 
+struct ban_test {
+	uint8_t			arg1;
+	const char		*arg1_spec;
+	uint8_t			oper;
+	const char		*arg2;
+	const void		*arg2_spec;
+};
+
 /*--------------------------------------------------------------------
  * Variables we can purge on
  */
diff --git a/bin/varnishd/cache/cache_ban.h b/bin/varnishd/cache/cache_ban.h
index fae2b80..ce0ae9e 100644
--- a/bin/varnishd/cache/cache_ban.h
+++ b/bin/varnishd/cache/cache_ban.h
@@ -102,14 +102,6 @@ struct ban {
 
 VTAILQ_HEAD(banhead_s,ban);
 
-struct ban_test {
-	uint8_t			arg1;
-	const char		*arg1_spec;
-	uint8_t			oper;
-	const char		*arg2;
-	const void		*arg2_spec;
-};
-
 bgthread_t ban_lurker;
 extern struct lock ban_mtx;
 extern int ban_shutdown;



From lkarsten at varnish-software.com  Thu Jan 14 14:15:01 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:01 +0100
Subject: [4.1] 4e2bda5 BAN_Compile() is private stuff.
Message-ID: <E1aJig1-0004zt-Nq@project.varnish-software.com>


commit 4e2bda503a31964f6275cc6af305c62819e54eb6
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu Oct 22 07:40:28 2015 +0000

    BAN_Compile() is private stuff.

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index 742f89e..65ed081 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -665,7 +665,6 @@ void BAN_DestroyObj(struct objcore *oc);
 int BAN_CheckObject(struct worker *, struct objcore *, struct req *);
 void BAN_Reload(const uint8_t *ban, unsigned len);
 struct ban *BAN_TailRef(void);
-void BAN_Compile(void);
 struct ban *BAN_RefBan(struct objcore *oc, double t0, const struct ban *tail);
 void BAN_TailDeref(struct ban **ban);
 double BAN_Time(const struct ban *ban);
diff --git a/bin/varnishd/cache/cache_priv.h b/bin/varnishd/cache/cache_priv.h
index 7d69cfa..2423afd 100644
--- a/bin/varnishd/cache/cache_priv.h
+++ b/bin/varnishd/cache/cache_priv.h
@@ -59,6 +59,7 @@ void VBP_Init(void);
 
 /* cache_ban.c */
 void BAN_Init(void);
+void BAN_Compile(void);
 void BAN_Shutdown(void);
 
 /* cache_busyobj.c */



From lkarsten at varnish-software.com  Thu Jan 14 14:15:01 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:01 +0100
Subject: [4.1] f20aeae Convert the (persistent) stevedores tail reference into
 a proper "hold" on the ban lurker not starting.
Message-ID: <E1aJig1-000508-Th@project.varnish-software.com>


commit f20aeae84c4638c20e8cb34338735759f46e522a
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu Oct 22 08:07:14 2015 +0000

    Convert the (persistent) stevedores tail reference into a proper "hold"
    on the ban lurker not starting.
    
    Move cache_hash's access to bans into cache_priv.h

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index 65ed081..11c8be1 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -655,18 +655,19 @@ struct sess {
 typedef enum htc_status_e htc_complete_f(struct http_conn *);
 
 /* cache_ban.c */
+
+/* for constructing bans */
 struct ban *BAN_New(void);
 int BAN_AddTest(struct ban *, const char *, const char *, const char *);
 void BAN_Free(struct ban *b);
 char *BAN_Insert(struct ban *b);
 void BAN_Free_Errormsg(char *);
-void BAN_NewObjCore(struct objcore *oc);
-void BAN_DestroyObj(struct objcore *oc);
-int BAN_CheckObject(struct worker *, struct objcore *, struct req *);
+
+/* for stevedoes resurrecting bans */
+void BAN_Hold(void);
+void BAN_Release(void);
 void BAN_Reload(const uint8_t *ban, unsigned len);
-struct ban *BAN_TailRef(void);
-struct ban *BAN_RefBan(struct objcore *oc, double t0, const struct ban *tail);
-void BAN_TailDeref(struct ban **ban);
+struct ban *BAN_RefBan(struct objcore *oc, double t0);
 double BAN_Time(const struct ban *ban);
 
 /* cache_busyobj.c */
diff --git a/bin/varnishd/cache/cache_ban.c b/bin/varnishd/cache/cache_ban.c
index 8d6337c..e5e03e6 100644
--- a/bin/varnishd/cache/cache_ban.c
+++ b/bin/varnishd/cache/cache_ban.c
@@ -43,12 +43,13 @@
 #include "vtim.h"
 
 struct lock ban_mtx;
-int ban_shutdown = 0;
+int ban_shutdown;
 struct banhead_s ban_head = VTAILQ_HEAD_INITIALIZER(ban_head);
 struct ban * volatile ban_start;
 
 static struct ban *ban_magic;
 static pthread_t ban_thread;
+static int ban_holds;
 
 struct ban_test {
 	uint8_t			arg1;
@@ -121,34 +122,31 @@ BAN_Free(struct ban *b)
 }
 
 /*--------------------------------------------------------------------
- * Get & Release a tail reference, used to hold the list stable for
- * traversals etc.
+ * Get/release holds which prevent the ban_lurker from starting.
+ * Holds are held while stevedores load zombie objects.
  */
 
-struct ban *
-BAN_TailRef(void)
+void
+BAN_Hold(void)
 {
-	struct ban *b;
 
-	ASSERT_CLI();
 	Lck_Lock(&ban_mtx);
-	b = VTAILQ_LAST(&ban_head, banhead_s);
-	AN(b);
-	b->refcount++;
+	/* Once holds are released, we allow no more */
+	assert(ban_holds > 0);
+	ban_holds++;
 	Lck_Unlock(&ban_mtx);
-	return (b);
 }
 
 void
-BAN_TailDeref(struct ban **bb)
+BAN_Release(void)
 {
-	struct ban *b;
 
-	b = *bb;
-	*bb = NULL;
 	Lck_Lock(&ban_mtx);
-	b->refcount--;
+	assert(ban_holds > 0);
+	ban_holds--;
 	Lck_Unlock(&ban_mtx);
+	if (ban_holds == 0)
+		WRK_BgThread(&ban_thread, "ban-lurker", ban_lurker, NULL);
 }
 
 /*--------------------------------------------------------------------
@@ -549,11 +547,11 @@ BAN_DestroyObj(struct objcore *oc)
 
 /*--------------------------------------------------------------------
  * Find and/or Grab a reference to an objects ban based on timestamp
- * Assume we hold a TailRef, so list traversal is safe.
+ * Assume we have a BAN_Hold, so list traversal is safe.
  */
 
 struct ban *
-BAN_RefBan(struct objcore *oc, double t0, const struct ban *tail)
+BAN_RefBan(struct objcore *oc, double t0)
 {
 	struct ban *b;
 	double t1 = 0;
@@ -562,12 +560,11 @@ BAN_RefBan(struct objcore *oc, double t0, const struct ban *tail)
 		t1 = ban_time(b->spec);
 		if (t1 <= t0)
 			break;
-		if (b == tail)
-			break;
 	}
 	AN(b);
 	assert(t1 == t0);
 	Lck_Lock(&ban_mtx);
+	assert(ban_holds > 0);
 	b->refcount++;
 	VTAILQ_INSERT_TAIL(&b->objcore, oc, ban_list);
 	Lck_Unlock(&ban_mtx);
@@ -966,7 +963,10 @@ ccf_ban_list(struct cli *cli, const char * const *av, void *priv)
 	(void)priv;
 
 	/* Get a reference so we are safe to traverse the list */
-	bl = BAN_TailRef();
+	Lck_Lock(&ban_mtx);
+	bl = VTAILQ_LAST(&ban_head, banhead_s);
+	bl->refcount++;
+	Lck_Unlock(&ban_mtx);
 
 	VCLI_Out(cli, "Present bans:\n");
 	VTAILQ_FOREACH(b, &ban_head, list) {
@@ -994,7 +994,9 @@ ccf_ban_list(struct cli *cli, const char * const *av, void *priv)
 		}
 	}
 
-	BAN_TailDeref(&bl);
+	Lck_Lock(&ban_mtx);
+	bl->refcount--;
+	Lck_Unlock(&ban_mtx);
 }
 
 static struct cli_proto ban_cmds[] = {
@@ -1010,7 +1012,8 @@ void
 BAN_Compile(void)
 {
 
-	/* All bans have been read from all persistent stevedores. Export
+	/*
+	 * All bans have been read from all persistent stevedores. Export
 	 * the compiled list
 	 */
 
@@ -1027,10 +1030,9 @@ BAN_Compile(void)
 	Lck_Unlock(&ban_mtx);
 
 	ban_start = VTAILQ_FIRST(&ban_head);
-	WRK_BgThread(&ban_thread, "ban-lurker", ban_lurker, NULL);
+	BAN_Release();
 }
 
-
 void
 BAN_Init(void)
 {
@@ -1043,6 +1045,7 @@ BAN_Init(void)
 	AZ(BAN_Insert(ban_magic));
 	Lck_Lock(&ban_mtx);
 	ban_mark_completed(ban_magic);
+	ban_holds = 1;
 	Lck_Unlock(&ban_mtx);
 }
 
diff --git a/bin/varnishd/cache/cache_priv.h b/bin/varnishd/cache/cache_priv.h
index 2423afd..483d041 100644
--- a/bin/varnishd/cache/cache_priv.h
+++ b/bin/varnishd/cache/cache_priv.h
@@ -57,11 +57,19 @@ void VBE_Poll(void);
 /* cache_backend_poll.c */
 void VBP_Init(void);
 
-/* cache_ban.c */
+/* == cache_ban.c == */
+
+/* From cache_main.c */
 void BAN_Init(void);
 void BAN_Compile(void);
 void BAN_Shutdown(void);
 
+/* From cache_hash.c */
+void BAN_NewObjCore(struct objcore *oc);
+void BAN_DestroyObj(struct objcore *oc);
+int BAN_CheckObject(struct worker *, struct objcore *, struct req *);
+
+
 /* cache_busyobj.c */
 void VBO_Init(void);
 
diff --git a/bin/varnishd/storage/storage_persistent.c b/bin/varnishd/storage/storage_persistent.c
index 2679675..b87fece 100644
--- a/bin/varnishd/storage/storage_persistent.c
+++ b/bin/varnishd/storage/storage_persistent.c
@@ -287,8 +287,7 @@ smp_thread(struct worker *wrk, void *priv)
 			smp_load_seg(wrk, sc, sg);
 
 	sc->flags |= SMP_SC_LOADED;
-	BAN_TailDeref(&sc->tailban);
-	AZ(sc->tailban);
+	BAN_Release();
 	printf("Silo completely loaded\n");
 
 	/* Housekeeping loop */
@@ -358,8 +357,7 @@ smp_open(const struct stevedore *st)
 	 * has loaded all objects, so we can be sure that all of our
 	 * proto-bans survive until then.
 	 */
-	sc->tailban = BAN_TailRef();
-	AN(sc->tailban);
+	BAN_Hold();
 
 	/* XXX: save segments to ensure consistency between seg1 & seg2 ? */
 
diff --git a/bin/varnishd/storage/storage_persistent.h b/bin/varnishd/storage/storage_persistent.h
index 8dac5e2..3a1d4f2 100644
--- a/bin/varnishd/storage/storage_persistent.h
+++ b/bin/varnishd/storage/storage_persistent.h
@@ -253,8 +253,6 @@ struct smp_sc {
 	struct smp_signspace	seg1;
 	struct smp_signspace	seg2;
 
-	struct ban		*tailban;
-
 	struct lock		mtx;
 
 	/* Cleaner metrics */
diff --git a/bin/varnishd/storage/storage_persistent_silo.c b/bin/varnishd/storage/storage_persistent_silo.c
index 45c8a66..ab604d8 100644
--- a/bin/varnishd/storage/storage_persistent_silo.c
+++ b/bin/varnishd/storage/storage_persistent_silo.c
@@ -163,7 +163,7 @@ smp_load_seg(struct worker *wrk, const struct smp_sc *sc,
 		oc->stobj->stevedore = sc->parent;
 		smp_init_oc(oc, sg, no);
 		oc->stobj->priv2 |= NEED_FIXUP;
-		oc->ban = BAN_RefBan(oc, so->ban, sc->tailban);
+		oc->ban = BAN_RefBan(oc, so->ban);
 		HSH_Insert(wrk, so->hash, oc);
 		oc->exp = so->exp;
 		sg->nobj++;



From lkarsten at varnish-software.com  Thu Jan 14 14:15:02 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:02 +0100
Subject: [4.1] ee8a2dc Split BAN construction code into a separate source file
Message-ID: <E1aJig2-00050I-1Q@project.varnish-software.com>


commit ee8a2dc6d4ad0fc39cbf3d348487e3038344feb4
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu Oct 22 08:53:53 2015 +0000

    Split BAN construction code into a separate source file

diff --git a/bin/varnishd/Makefile.am b/bin/varnishd/Makefile.am
index 86906a2..d252c1b 100644
--- a/bin/varnishd/Makefile.am
+++ b/bin/varnishd/Makefile.am
@@ -15,6 +15,7 @@ varnishd_SOURCES = \
 	cache/cache_backend_probe.c \
 	cache/cache_backend_tcp.c \
 	cache/cache_ban.c \
+	cache/cache_ban_build.c \
 	cache/cache_ban_lurker.c \
 	cache/cache_busyobj.c \
 	cache/cache_cli.c \
diff --git a/bin/varnishd/cache/cache_ban.c b/bin/varnishd/cache/cache_ban.c
index e5e03e6..0247d44 100644
--- a/bin/varnishd/cache/cache_ban.c
+++ b/bin/varnishd/cache/cache_ban.c
@@ -40,14 +40,13 @@
 #include "vcli_priv.h"
 #include "vend.h"
 #include "vmb.h"
-#include "vtim.h"
 
 struct lock ban_mtx;
 int ban_shutdown;
 struct banhead_s ban_head = VTAILQ_HEAD_INITIALIZER(ban_head);
 struct ban * volatile ban_start;
+struct ban *ban_magic;
 
-static struct ban *ban_magic;
 static pthread_t ban_thread;
 static int ban_holds;
 
@@ -60,21 +59,6 @@ struct ban_test {
 };
 
 /*--------------------------------------------------------------------
- * Variables we can purge on
- */
-
-static const struct pvar {
-	const char		*name;
-	unsigned		flag;
-	uint8_t			tag;
-} pvars[] = {
-#define PVAR(a, b, c)	{ (a), (b), (c) },
-#include "tbl/ban_vars.h"
-#undef PVAR
-	{ 0, 0, 0}
-};
-
-/*--------------------------------------------------------------------
  * Storage handling of bans
  */
 
@@ -172,7 +156,7 @@ ban_len(const uint8_t *banspec)
 	return (u);
 }
 
-static int
+int
 ban_equal(const uint8_t *bs1, const uint8_t *bs2)
 {
 	unsigned u;
@@ -209,19 +193,6 @@ ban_mark_completed(struct ban *b)
  * Access a lump of bytes in a ban test spec
  */
 
-static void
-ban_add_lump(const struct ban *b, const void *p, uint32_t len)
-{
-	uint8_t buf[sizeof len];
-
-	buf[0] = 0xff;
-	while (VSB_len(b->vsb) & PALGN)
-		VSB_bcat(b->vsb, buf, 1);
-	vbe32enc(buf, len);
-	VSB_bcat(b->vsb, buf, sizeof buf);
-	VSB_bcat(b->vsb, p, len);
-}
-
 static const void *
 ban_get_lump(const uint8_t **bs)
 {
@@ -258,256 +229,6 @@ ban_iter(const uint8_t **bs, struct ban_test *bt)
 }
 
 /*--------------------------------------------------------------------
- */
-
-static int
-ban_error(struct ban *b, const char *fmt, ...)
-{
-	va_list ap;
-
-	CHECK_OBJ_NOTNULL(b, BAN_MAGIC);
-	AN(b->vsb);
-
-	/* First error is sticky */
-	if (!(b->flags & BANS_FLAG_ERROR)) {
-		b->flags |= BANS_FLAG_ERROR;
-
-		/* Record the error message in the vsb */
-		VSB_clear(b->vsb);
-		va_start(ap, fmt);
-		(void)VSB_vprintf(b->vsb, fmt, ap);
-		va_end(ap);
-	}
-	return (-1);
-}
-
-/*--------------------------------------------------------------------
- * Parse and add a http argument specification
- * Output something which HTTP_GetHdr understands
- */
-
-static void
-ban_parse_http(const struct ban *b, const char *a1)
-{
-	int l;
-
-	l = strlen(a1) + 1;
-	assert(l <= 127);
-	VSB_putc(b->vsb, (char)l);
-	VSB_cat(b->vsb, a1);
-	VSB_putc(b->vsb, ':');
-	VSB_putc(b->vsb, '\0');
-}
-
-/*--------------------------------------------------------------------
- * Parse and add a ban test specification
- */
-
-static int
-ban_parse_regexp(struct ban *b, const char *a3)
-{
-	const char *error;
-	int erroroffset, rc;
-	size_t sz;
-	pcre *re;
-
-	re = pcre_compile(a3, 0, &error, &erroroffset, NULL);
-	if (re == NULL)
-		return (ban_error(b, "Regex compile error: %s", error));
-	rc = pcre_fullinfo(re, NULL, PCRE_INFO_SIZE, &sz);
-	AZ(rc);
-	ban_add_lump(b, re, sz);
-	pcre_free(re);
-	return (0);
-}
-
-/*--------------------------------------------------------------------
- * Add a (and'ed) test-condition to a ban
- */
-
-int
-BAN_AddTest(struct ban *b, const char *a1, const char *a2, const char *a3)
-{
-	const struct pvar *pv;
-	int i;
-
-	CHECK_OBJ_NOTNULL(b, BAN_MAGIC);
-	AN(b->vsb);
-	AN(a1);
-	AN(a2);
-	AN(a3);
-
-	if (b->flags & BANS_FLAG_ERROR)
-		return (-1);
-
-	for (pv = pvars; pv->name != NULL; pv++)
-		if (!strncmp(a1, pv->name, strlen(pv->name)))
-			break;
-
-	if (pv->name == NULL)
-		return (ban_error(b,
-		    "Unknown or unsupported field \"%s\"", a1));
-
-	b->flags |= pv->flag;
-
-	VSB_putc(b->vsb, pv->tag);
-	if (pv->flag & BANS_FLAG_HTTP)
-		ban_parse_http(b, a1 + strlen(pv->name));
-
-	ban_add_lump(b, a3, strlen(a3) + 1);
-	if (!strcmp(a2, "~")) {
-		VSB_putc(b->vsb, BANS_OPER_MATCH);
-		i = ban_parse_regexp(b, a3);
-		if (i)
-			return (i);
-	} else if (!strcmp(a2, "!~")) {
-		VSB_putc(b->vsb, BANS_OPER_NMATCH);
-		i = ban_parse_regexp(b, a3);
-		if (i)
-			return (i);
-	} else if (!strcmp(a2, "==")) {
-		VSB_putc(b->vsb, BANS_OPER_EQ);
-	} else if (!strcmp(a2, "!=")) {
-		VSB_putc(b->vsb, BANS_OPER_NEQ);
-	} else {
-		return (ban_error(b,
-		    "expected conditional (~, !~, == or !=) got \"%s\"", a2));
-	}
-	return (0);
-}
-
-/*--------------------------------------------------------------------
- * We maintain ban_start as a pointer to the first element of the list
- * as a separate variable from the VTAILQ, to avoid depending on the
- * internals of the VTAILQ macros.  We tacitly assume that a pointer
- * write is always atomic in doing so.
- *
- * Returns:
- *   0: Ban successfully inserted
- *  -1: Ban not inserted due to shutdown in progress. The ban has been
- *      deleted.
- */
-
-static char ban_error_nomem[] = "Could not get memory";
-
-static char *
-ban_ins_error(const char *p)
-{
-	char *r = NULL;
-
-	if (p != NULL)
-		r = strdup(p);
-	if (r == NULL)
-		r = ban_error_nomem;
-	return (r);
-}
-
-void
-BAN_Free_Errormsg(char *p)
-{
-	if (p != ban_error_nomem)
-		free(p);
-}
-
-char *
-BAN_Insert(struct ban *b)
-{
-	struct ban  *bi, *be;
-	ssize_t ln;
-	double t0;
-	char *p;
-
-	CHECK_OBJ_NOTNULL(b, BAN_MAGIC);
-	AN(b->vsb);
-
-	if (ban_shutdown) {
-		BAN_Free(b);
-		return (ban_ins_error("Shutting down"));
-	}
-
-	AZ(VSB_finish(b->vsb));
-	ln = VSB_len(b->vsb);
-	assert(ln >= 0);
-
-	if (b->flags & BANS_FLAG_ERROR) {
-		p = ban_ins_error(VSB_data(b->vsb));
-		BAN_Free(b);
-		return (p);
-	}
-
-	b->spec = malloc(ln + BANS_HEAD_LEN);
-	if (b->spec == NULL) {
-		BAN_Free(b);
-		return (ban_ins_error(NULL));
-	}
-
-	memset(b->spec, 0, BANS_HEAD_LEN);
-	t0 = VTIM_real();
-	memcpy(b->spec + BANS_TIMESTAMP, &t0, sizeof t0);
-	b->spec[BANS_FLAGS] = b->flags & 0xff;
-	memcpy(b->spec + BANS_HEAD_LEN, VSB_data(b->vsb), ln);
-	ln += BANS_HEAD_LEN;
-	vbe32enc(b->spec + BANS_LENGTH, ln);
-
-	VSB_delete(b->vsb);
-	b->vsb = NULL;
-
-	Lck_Lock(&ban_mtx);
-	if (ban_shutdown) {
-		/* Check again, we might have raced */
-		Lck_Unlock(&ban_mtx);
-		BAN_Free(b);
-		return (ban_ins_error("Shutting down"));
-	}
-	VTAILQ_INSERT_HEAD(&ban_head, b, list);
-	ban_start = b;
-	VSC_C_main->bans++;
-	VSC_C_main->bans_added++;
-	if (b->flags & BANS_FLAG_OBJ)
-		VSC_C_main->bans_obj++;
-	if (b->flags & BANS_FLAG_REQ)
-		VSC_C_main->bans_req++;
-
-	be = VTAILQ_LAST(&ban_head, banhead_s);
-	if (cache_param->ban_dups && be != b)
-		be->refcount++;
-	else
-		be = NULL;
-
-	/* ban_magic is magic, and needs to be inserted early to give
-	 * a handle to grab a ref on. We don't report it here as the
-	 * stevedores will not be opened and ready to accept it
-	 * yet. Instead it is reported on BAN_Compile, which is after
-	 * the stevedores has been opened, but before any new objects
-	 * can have entered the cache (thus no objects in the mean
-	 * time depending on ban_magic in the list) */
-	VSC_C_main->bans_persisted_bytes += ln;
-	if (b != ban_magic)
-		ban_info(BI_NEW, b->spec, ln); /* Notify stevedores */
-	Lck_Unlock(&ban_mtx);
-
-	if (be == NULL)
-		return (NULL);
-
-	/* Hunt down duplicates, and mark them as completed */
-	bi = b;
-	Lck_Lock(&ban_mtx);
-	while (!ban_shutdown && bi != be) {
-		bi = VTAILQ_NEXT(bi, list);
-		if (bi->flags & BANS_FLAG_COMPLETED)
-			continue;
-		if (!ban_equal(b->spec, bi->spec))
-			continue;
-		ban_mark_completed(bi);
-		VSC_C_main->bans_dups++;
-	}
-	be->refcount--;
-	Lck_Unlock(&ban_mtx);
-
-	return (NULL);
-}
-
-/*--------------------------------------------------------------------
  * A new object is created, grab a reference to the newest ban
  */
 
diff --git a/bin/varnishd/cache/cache_ban.h b/bin/varnishd/cache/cache_ban.h
index ce0ae9e..755b2f4 100644
--- a/bin/varnishd/cache/cache_ban.h
+++ b/bin/varnishd/cache/cache_ban.h
@@ -107,6 +107,7 @@ extern struct lock ban_mtx;
 extern int ban_shutdown;
 extern struct banhead_s ban_head;
 extern struct ban * volatile ban_start;
+extern struct ban *ban_magic;
 
 void ban_mark_completed(struct ban *b);
 unsigned ban_len(const uint8_t *banspec);
@@ -114,3 +115,4 @@ void ban_info(enum baninfo event, const uint8_t *ban, unsigned len);
 int ban_evaluate(struct worker *wrk, const uint8_t *bs, struct objcore *oc,
     const struct http *reqhttp, unsigned *tests);
 double ban_time(const uint8_t *banspec);
+int ban_equal(const uint8_t *bs1, const uint8_t *bs2);
diff --git a/bin/varnishd/cache/cache_ban_build.c b/bin/varnishd/cache/cache_ban_build.c
new file mode 100644
index 0000000..ee2f0e7
--- /dev/null
+++ b/bin/varnishd/cache/cache_ban_build.c
@@ -0,0 +1,317 @@
+/*-
+ * Copyright (c) 2006 Verdens Gang AS
+ * Copyright (c) 2006-2015 Varnish Software AS
+ * All rights reserved.
+ *
+ * Author: Poul-Henning Kamp <phk at phk.freebsd.dk>
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED.  IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ *
+ */
+
+#include "config.h"
+
+#include <pcre.h>
+
+#include "cache.h"
+#include "cache_ban.h"
+
+#include "vend.h"
+#include "vtim.h"
+
+/*--------------------------------------------------------------------
+ * Variables we can purge on
+ */
+
+static const struct pvar {
+	const char		*name;
+	unsigned		flag;
+	uint8_t			tag;
+} pvars[] = {
+#define PVAR(a, b, c)	{ (a), (b), (c) },
+#include "tbl/ban_vars.h"
+#undef PVAR
+	{ 0, 0, 0}
+};
+
+static void
+ban_add_lump(const struct ban *b, const void *p, uint32_t len)
+{
+	uint8_t buf[sizeof len];
+
+	buf[0] = 0xff;
+	while (VSB_len(b->vsb) & PALGN)
+		VSB_bcat(b->vsb, buf, 1);
+	vbe32enc(buf, len);
+	VSB_bcat(b->vsb, buf, sizeof buf);
+	VSB_bcat(b->vsb, p, len);
+}
+
+/*--------------------------------------------------------------------
+ */
+
+static int
+ban_error(struct ban *b, const char *fmt, ...)
+{
+	va_list ap;
+
+	CHECK_OBJ_NOTNULL(b, BAN_MAGIC);
+	AN(b->vsb);
+
+	/* First error is sticky */
+	if (!(b->flags & BANS_FLAG_ERROR)) {
+		b->flags |= BANS_FLAG_ERROR;
+
+		/* Record the error message in the vsb */
+		VSB_clear(b->vsb);
+		va_start(ap, fmt);
+		(void)VSB_vprintf(b->vsb, fmt, ap);
+		va_end(ap);
+	}
+	return (-1);
+}
+
+/*--------------------------------------------------------------------
+ * Parse and add a http argument specification
+ * Output something which HTTP_GetHdr understands
+ */
+
+static void
+ban_parse_http(const struct ban *b, const char *a1)
+{
+	int l;
+
+	l = strlen(a1) + 1;
+	assert(l <= 127);
+	VSB_putc(b->vsb, (char)l);
+	VSB_cat(b->vsb, a1);
+	VSB_putc(b->vsb, ':');
+	VSB_putc(b->vsb, '\0');
+}
+
+/*--------------------------------------------------------------------
+ * Parse and add a ban test specification
+ */
+
+static int
+ban_parse_regexp(struct ban *b, const char *a3)
+{
+	const char *error;
+	int erroroffset, rc;
+	size_t sz;
+	pcre *re;
+
+	re = pcre_compile(a3, 0, &error, &erroroffset, NULL);
+	if (re == NULL)
+		return (ban_error(b, "Regex compile error: %s", error));
+	rc = pcre_fullinfo(re, NULL, PCRE_INFO_SIZE, &sz);
+	AZ(rc);
+	ban_add_lump(b, re, sz);
+	pcre_free(re);
+	return (0);
+}
+
+/*--------------------------------------------------------------------
+ * Add a (and'ed) test-condition to a ban
+ */
+
+int
+BAN_AddTest(struct ban *b, const char *a1, const char *a2, const char *a3)
+{
+	const struct pvar *pv;
+	int i;
+
+	CHECK_OBJ_NOTNULL(b, BAN_MAGIC);
+	AN(b->vsb);
+	AN(a1);
+	AN(a2);
+	AN(a3);
+
+	if (b->flags & BANS_FLAG_ERROR)
+		return (-1);
+
+	for (pv = pvars; pv->name != NULL; pv++)
+		if (!strncmp(a1, pv->name, strlen(pv->name)))
+			break;
+
+	if (pv->name == NULL)
+		return (ban_error(b,
+		    "Unknown or unsupported field \"%s\"", a1));
+
+	b->flags |= pv->flag;
+
+	VSB_putc(b->vsb, pv->tag);
+	if (pv->flag & BANS_FLAG_HTTP)
+		ban_parse_http(b, a1 + strlen(pv->name));
+
+	ban_add_lump(b, a3, strlen(a3) + 1);
+	if (!strcmp(a2, "~")) {
+		VSB_putc(b->vsb, BANS_OPER_MATCH);
+		i = ban_parse_regexp(b, a3);
+		if (i)
+			return (i);
+	} else if (!strcmp(a2, "!~")) {
+		VSB_putc(b->vsb, BANS_OPER_NMATCH);
+		i = ban_parse_regexp(b, a3);
+		if (i)
+			return (i);
+	} else if (!strcmp(a2, "==")) {
+		VSB_putc(b->vsb, BANS_OPER_EQ);
+	} else if (!strcmp(a2, "!=")) {
+		VSB_putc(b->vsb, BANS_OPER_NEQ);
+	} else {
+		return (ban_error(b,
+		    "expected conditional (~, !~, == or !=) got \"%s\"", a2));
+	}
+	return (0);
+}
+
+/*--------------------------------------------------------------------
+ * We maintain ban_start as a pointer to the first element of the list
+ * as a separate variable from the VTAILQ, to avoid depending on the
+ * internals of the VTAILQ macros.  We tacitly assume that a pointer
+ * write is always atomic in doing so.
+ *
+ * Returns:
+ *   0: Ban successfully inserted
+ *  -1: Ban not inserted due to shutdown in progress. The ban has been
+ *      deleted.
+ */
+
+static char ban_error_nomem[] = "Could not get memory";
+
+static char *
+ban_ins_error(const char *p)
+{
+	char *r = NULL;
+
+	if (p != NULL)
+		r = strdup(p);
+	if (r == NULL)
+		r = ban_error_nomem;
+	return (r);
+}
+
+void
+BAN_Free_Errormsg(char *p)
+{
+	if (p != ban_error_nomem)
+		free(p);
+}
+
+char *
+BAN_Insert(struct ban *b)
+{
+	struct ban  *bi, *be;
+	ssize_t ln;
+	double t0;
+	char *p;
+
+	CHECK_OBJ_NOTNULL(b, BAN_MAGIC);
+	AN(b->vsb);
+
+	if (ban_shutdown) {
+		BAN_Free(b);
+		return (ban_ins_error("Shutting down"));
+	}
+
+	AZ(VSB_finish(b->vsb));
+	ln = VSB_len(b->vsb);
+	assert(ln >= 0);
+
+	if (b->flags & BANS_FLAG_ERROR) {
+		p = ban_ins_error(VSB_data(b->vsb));
+		BAN_Free(b);
+		return (p);
+	}
+
+	b->spec = malloc(ln + BANS_HEAD_LEN);
+	if (b->spec == NULL) {
+		BAN_Free(b);
+		return (ban_ins_error(NULL));
+	}
+
+	memset(b->spec, 0, BANS_HEAD_LEN);
+	t0 = VTIM_real();
+	memcpy(b->spec + BANS_TIMESTAMP, &t0, sizeof t0);
+	b->spec[BANS_FLAGS] = b->flags & 0xff;
+	memcpy(b->spec + BANS_HEAD_LEN, VSB_data(b->vsb), ln);
+	ln += BANS_HEAD_LEN;
+	vbe32enc(b->spec + BANS_LENGTH, ln);
+
+	VSB_delete(b->vsb);
+	b->vsb = NULL;
+
+	Lck_Lock(&ban_mtx);
+	if (ban_shutdown) {
+		/* Check again, we might have raced */
+		Lck_Unlock(&ban_mtx);
+		BAN_Free(b);
+		return (ban_ins_error("Shutting down"));
+	}
+	VTAILQ_INSERT_HEAD(&ban_head, b, list);
+	ban_start = b;
+	VSC_C_main->bans++;
+	VSC_C_main->bans_added++;
+	if (b->flags & BANS_FLAG_OBJ)
+		VSC_C_main->bans_obj++;
+	if (b->flags & BANS_FLAG_REQ)
+		VSC_C_main->bans_req++;
+
+	be = VTAILQ_LAST(&ban_head, banhead_s);
+	if (cache_param->ban_dups && be != b)
+		be->refcount++;
+	else
+		be = NULL;
+
+	/* ban_magic is magic, and needs to be inserted early to give
+	 * a handle to grab a ref on. We don't report it here as the
+	 * stevedores will not be opened and ready to accept it
+	 * yet. Instead it is reported on BAN_Compile, which is after
+	 * the stevedores has been opened, but before any new objects
+	 * can have entered the cache (thus no objects in the mean
+	 * time depending on ban_magic in the list) */
+	VSC_C_main->bans_persisted_bytes += ln;
+	if (b != ban_magic)
+		ban_info(BI_NEW, b->spec, ln); /* Notify stevedores */
+	Lck_Unlock(&ban_mtx);
+
+	if (be == NULL)
+		return (NULL);
+
+	/* Hunt down duplicates, and mark them as completed */
+	bi = b;
+	Lck_Lock(&ban_mtx);
+	while (!ban_shutdown && bi != be) {
+		bi = VTAILQ_NEXT(bi, list);
+		if (bi->flags & BANS_FLAG_COMPLETED)
+			continue;
+		if (!ban_equal(b->spec, bi->spec))
+			continue;
+		ban_mark_completed(bi);
+		VSC_C_main->bans_dups++;
+	}
+	be->refcount--;
+	Lck_Unlock(&ban_mtx);
+
+	return (NULL);
+}



From lkarsten at varnish-software.com  Thu Jan 14 14:15:02 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:02 +0100
Subject: [4.1] 23dbd02 Split the datastructures we use to construct bans
 (ban_proto) from the data structure we use to store and operate on the ban
 (ban).
Message-ID: <E1aJig2-00050U-5O@project.varnish-software.com>


commit 23dbd02cd64f3002c8af0e2ae77cd334047bc9f2
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu Oct 22 15:39:51 2015 +0000

    Split the datastructures we use to construct bans (ban_proto)
    from the data structure we use to store and operate on the ban (ban).

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index 11c8be1..93e5de4 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -102,6 +102,7 @@ enum {
 
 struct VSC_C_lck;
 struct ban;
+struct ban_proto;
 struct backend;
 struct busyobj;
 struct cli;
@@ -657,11 +658,11 @@ typedef enum htc_status_e htc_complete_f(struct http_conn *);
 /* cache_ban.c */
 
 /* for constructing bans */
-struct ban *BAN_New(void);
-int BAN_AddTest(struct ban *, const char *, const char *, const char *);
-void BAN_Free(struct ban *b);
-char *BAN_Insert(struct ban *b);
-void BAN_Free_Errormsg(char *);
+struct ban_proto *BAN_Build(void);
+const char *BAN_AddTest(struct ban_proto *,
+    const char *, const char *, const char *);
+const char *BAN_Commit(struct ban_proto *b);
+void BAN_Abandon(struct ban_proto *b);
 
 /* for stevedoes resurrecting bans */
 void BAN_Hold(void);
diff --git a/bin/varnishd/cache/cache_ban.c b/bin/varnishd/cache/cache_ban.c
index 0247d44..95ca144 100644
--- a/bin/varnishd/cache/cache_ban.c
+++ b/bin/varnishd/cache/cache_ban.c
@@ -45,7 +45,6 @@ struct lock ban_mtx;
 int ban_shutdown;
 struct banhead_s ban_head = VTAILQ_HEAD_INITIALIZER(ban_head);
 struct ban * volatile ban_start;
-struct ban *ban_magic;
 
 static pthread_t ban_thread;
 static int ban_holds;
@@ -73,23 +72,6 @@ ban_alloc(void)
 	return (b);
 }
 
-struct ban *
-BAN_New(void)
-{
-	struct ban *b;
-
-	b = ban_alloc();
-	if (b != NULL) {
-		b->vsb = VSB_new_auto();
-		if (b->vsb == NULL) {
-			FREE_OBJ(b);
-			return (NULL);
-		}
-		VTAILQ_INIT(&b->objcore);
-	}
-	return (b);
-}
-
 void
 BAN_Free(struct ban *b)
 {
@@ -98,8 +80,6 @@ BAN_Free(struct ban *b)
 	AZ(b->refcount);
 	assert(VTAILQ_EMPTY(&b->objcore));
 
-	if (b->vsb != NULL)
-		VSB_delete(b->vsb);
 	if (b->spec != NULL)
 		free(b->spec);
 	FREE_OBJ(b);
@@ -595,8 +575,8 @@ static void
 ccf_ban(struct cli *cli, const char * const *av, void *priv)
 {
 	int narg, i;
-	struct ban *b;
-	char *p;
+	struct ban_proto *bp;
+	const char *err = NULL;
 
 	(void)priv;
 
@@ -616,19 +596,24 @@ ccf_ban(struct cli *cli, const char * const *av, void *priv)
 		}
 	}
 
-	b = BAN_New();
-	if (b == NULL) {
+	bp = BAN_Build();
+	if (bp == NULL) {
 		VCLI_Out(cli, "Out of Memory");
 		VCLI_SetResult(cli, CLIS_CANT);
 		return;
 	}
-	for (i = 0; i < narg; i += 4)
-		if (BAN_AddTest(b, av[i + 2], av[i + 3], av[i + 4]))
+	for (i = 0; i < narg; i += 4) {
+		err = BAN_AddTest(bp, av[i + 2], av[i + 3], av[i + 4]);
+		if (err)
 			break;
-	p = BAN_Insert(b);
-	if (p != NULL) {
-		VCLI_Out(cli, "%s", p);
-		BAN_Free_Errormsg(p);
+	}
+
+	if (err == NULL)
+		err = BAN_Commit(bp);
+
+	if (err != NULL) {
+		VCLI_Out(cli, "%s", err);
+		BAN_Abandon(bp);
 		VCLI_SetResult(cli, CLIS_PARAM);
 	}
 }
@@ -679,6 +664,7 @@ static void
 ccf_ban_list(struct cli *cli, const char * const *av, void *priv)
 {
 	struct ban *b, *bl;
+	int64_t o;
 
 	(void)av;
 	(void)priv;
@@ -691,14 +677,14 @@ ccf_ban_list(struct cli *cli, const char * const *av, void *priv)
 
 	VCLI_Out(cli, "Present bans:\n");
 	VTAILQ_FOREACH(b, &ban_head, list) {
-		VCLI_Out(cli, "%10.6f %5u %s", ban_time(b->spec),
-		    bl == b ? b->refcount - 1 : b->refcount,
+		o = bl == b ? 1 : 0;
+		VCLI_Out(cli, "%10.6f %5ju %s", ban_time(b->spec),
+		    (intmax_t)b->refcount - o,
 		    b->flags & BANS_FLAG_COMPLETED ? "C" : " ");
 		if (DO_DEBUG(DBG_LURKER)) {
-			VCLI_Out(cli, "%s%s%s %p ",
+			VCLI_Out(cli, "%s%s %p ",
 			    b->flags & BANS_FLAG_REQ ? "R" : "-",
 			    b->flags & BANS_FLAG_OBJ ? "O" : "-",
-			    b->flags & BANS_FLAG_ERROR ? "E" : "-",
 			    b);
 		}
 		VCLI_Out(cli, "  ");
@@ -732,6 +718,7 @@ static struct cli_proto ban_cmds[] = {
 void
 BAN_Compile(void)
 {
+	struct ban *b;
 
 	/*
 	 * All bans have been read from all persistent stevedores. Export
@@ -743,8 +730,9 @@ BAN_Compile(void)
 
 	Lck_Lock(&ban_mtx);
 
-	/* Do late reporting of ban_magic */
-	AZ(STV_BanInfo(BI_NEW, ban_magic->spec, ban_len(ban_magic->spec)));
+	/* Report the place-holder ban */
+	b = VTAILQ_FIRST(&ban_head);
+	AZ(STV_BanInfo(BI_NEW, b->spec, ban_len(b->spec)));
 
 	ban_export();
 
@@ -757,16 +745,19 @@ BAN_Compile(void)
 void
 BAN_Init(void)
 {
+	struct ban_proto *bp;
 
 	Lck_New(&ban_mtx, lck_ban);
 	CLI_AddFuncs(ban_cmds);
 
-	ban_magic = BAN_New();
-	AN(ban_magic);
-	AZ(BAN_Insert(ban_magic));
-	Lck_Lock(&ban_mtx);
-	ban_mark_completed(ban_magic);
 	ban_holds = 1;
+
+	/* Add a placeholder ban */
+	bp = BAN_Build();
+	AN(bp);
+	AZ(BAN_Commit(bp));
+	Lck_Lock(&ban_mtx);
+	ban_mark_completed(VTAILQ_FIRST(&ban_head));
 	Lck_Unlock(&ban_mtx);
 }
 
diff --git a/bin/varnishd/cache/cache_ban.h b/bin/varnishd/cache/cache_ban.h
index 755b2f4..047d0f9 100644
--- a/bin/varnishd/cache/cache_ban.h
+++ b/bin/varnishd/cache/cache_ban.h
@@ -73,7 +73,6 @@
 #define BANS_FLAG_OBJ		(1<<1)
 #define BANS_FLAG_COMPLETED	(1<<2)
 #define BANS_FLAG_HTTP		(1<<3)
-#define BANS_FLAG_ERROR		(1<<4)
 
 #define BANS_OPER_EQ		0x10
 #define BANS_OPER_NEQ		0x11
@@ -90,13 +89,12 @@
 struct ban {
 	unsigned		magic;
 #define BAN_MAGIC		0x700b08ea
+	unsigned		flags;		/* BANS_FLAG_* */
 	VTAILQ_ENTRY(ban)	list;
 	VTAILQ_ENTRY(ban)	l_list;
-	int			refcount;
-	unsigned		flags;		/* BANS_FLAG_* */
+	int64_t			refcount;
 
 	VTAILQ_HEAD(,objcore)	objcore;
-	struct vsb		*vsb;
 	uint8_t			*spec;
 };
 
@@ -107,7 +105,6 @@ extern struct lock ban_mtx;
 extern int ban_shutdown;
 extern struct banhead_s ban_head;
 extern struct ban * volatile ban_start;
-extern struct ban *ban_magic;
 
 void ban_mark_completed(struct ban *b);
 unsigned ban_len(const uint8_t *banspec);
@@ -116,3 +113,4 @@ int ban_evaluate(struct worker *wrk, const uint8_t *bs, struct objcore *oc,
     const struct http *reqhttp, unsigned *tests);
 double ban_time(const uint8_t *banspec);
 int ban_equal(const uint8_t *bs1, const uint8_t *bs2);
+void BAN_Free(struct ban *b);
diff --git a/bin/varnishd/cache/cache_ban_build.c b/bin/varnishd/cache/cache_ban_build.c
index ee2f0e7..eb10c30 100644
--- a/bin/varnishd/cache/cache_ban_build.c
+++ b/bin/varnishd/cache/cache_ban_build.c
@@ -38,6 +38,15 @@
 #include "vend.h"
 #include "vtim.h"
 
+struct ban_proto {
+	unsigned		magic;
+#define BAN_PROTO_MAGIC		0xd8adc494
+	unsigned		flags;		/* BANS_FLAG_* */
+
+	struct vsb		*vsb;
+	char			*err;
+};
+
 /*--------------------------------------------------------------------
  * Variables we can purge on
  */
@@ -53,41 +62,83 @@ static const struct pvar {
 	{ 0, 0, 0}
 };
 
+/*--------------------------------------------------------------------
+ */
+
+static char ban_build_err_no_mem[] = "No Memory";
+
+/*--------------------------------------------------------------------
+ */
+
+struct ban_proto *
+BAN_Build(void)
+{
+	struct ban_proto *bp;
+
+	ALLOC_OBJ(bp, BAN_PROTO_MAGIC);
+	if (bp == NULL)
+		return (bp);
+	bp->vsb = VSB_new_auto();
+	if (bp->vsb == NULL) {
+		FREE_OBJ(bp);
+		return (NULL);
+	}
+	return (bp);
+}
+
+void
+BAN_Abandon(struct ban_proto *bp)
+{
+
+	CHECK_OBJ_NOTNULL(bp, BAN_PROTO_MAGIC);
+	VSB_delete(bp->vsb);
+	if (bp->err != NULL && bp->err != ban_build_err_no_mem)
+		REPLACE(bp->err, NULL);
+	FREE_OBJ(bp);
+}
+
+/*--------------------------------------------------------------------
+ */
+
 static void
-ban_add_lump(const struct ban *b, const void *p, uint32_t len)
+ban_add_lump(const struct ban_proto *bp, const void *p, uint32_t len)
 {
 	uint8_t buf[sizeof len];
 
 	buf[0] = 0xff;
-	while (VSB_len(b->vsb) & PALGN)
-		VSB_bcat(b->vsb, buf, 1);
+	while (VSB_len(bp->vsb) & PALGN)
+		VSB_bcat(bp->vsb, buf, 1);
 	vbe32enc(buf, len);
-	VSB_bcat(b->vsb, buf, sizeof buf);
-	VSB_bcat(b->vsb, p, len);
+	VSB_bcat(bp->vsb, buf, sizeof buf);
+	VSB_bcat(bp->vsb, p, len);
 }
 
 /*--------------------------------------------------------------------
  */
 
-static int
-ban_error(struct ban *b, const char *fmt, ...)
+static const char *
+ban_error(struct ban_proto *bp, const char *fmt, ...)
 {
 	va_list ap;
 
-	CHECK_OBJ_NOTNULL(b, BAN_MAGIC);
-	AN(b->vsb);
+	CHECK_OBJ_NOTNULL(bp, BAN_PROTO_MAGIC);
+	AN(bp->vsb);
 
 	/* First error is sticky */
-	if (!(b->flags & BANS_FLAG_ERROR)) {
-		b->flags |= BANS_FLAG_ERROR;
-
-		/* Record the error message in the vsb */
-		VSB_clear(b->vsb);
-		va_start(ap, fmt);
-		(void)VSB_vprintf(b->vsb, fmt, ap);
-		va_end(ap);
+	if (bp->err == NULL) {
+		if (fmt == ban_build_err_no_mem) {
+			bp->err = ban_build_err_no_mem;
+		} else {
+			/* Record the error message in the vsb */
+			VSB_clear(bp->vsb);
+			va_start(ap, fmt);
+			(void)VSB_vprintf(bp->vsb, fmt, ap);
+			va_end(ap);
+			AZ(VSB_finish(bp->vsb));
+			bp->err = VSB_data(bp->vsb);
+		}
 	}
-	return (-1);
+	return (bp->err);
 }
 
 /*--------------------------------------------------------------------
@@ -96,24 +147,24 @@ ban_error(struct ban *b, const char *fmt, ...)
  */
 
 static void
-ban_parse_http(const struct ban *b, const char *a1)
+ban_parse_http(const struct ban_proto *bp, const char *a1)
 {
 	int l;
 
 	l = strlen(a1) + 1;
 	assert(l <= 127);
-	VSB_putc(b->vsb, (char)l);
-	VSB_cat(b->vsb, a1);
-	VSB_putc(b->vsb, ':');
-	VSB_putc(b->vsb, '\0');
+	VSB_putc(bp->vsb, (char)l);
+	VSB_cat(bp->vsb, a1);
+	VSB_putc(bp->vsb, ':');
+	VSB_putc(bp->vsb, '\0');
 }
 
 /*--------------------------------------------------------------------
  * Parse and add a ban test specification
  */
 
-static int
-ban_parse_regexp(struct ban *b, const char *a3)
+static const char *
+ban_parse_regexp(struct ban_proto *bp, const char *a3)
 {
 	const char *error;
 	int erroroffset, rc;
@@ -122,10 +173,10 @@ ban_parse_regexp(struct ban *b, const char *a3)
 
 	re = pcre_compile(a3, 0, &error, &erroroffset, NULL);
 	if (re == NULL)
-		return (ban_error(b, "Regex compile error: %s", error));
+		return (ban_error(bp, "Regex compile error: %s", error));
 	rc = pcre_fullinfo(re, NULL, PCRE_INFO_SIZE, &sz);
 	AZ(rc);
-	ban_add_lump(b, re, sz);
+	ban_add_lump(bp, re, sz);
 	pcre_free(re);
 	return (0);
 }
@@ -134,55 +185,56 @@ ban_parse_regexp(struct ban *b, const char *a3)
  * Add a (and'ed) test-condition to a ban
  */
 
-int
-BAN_AddTest(struct ban *b, const char *a1, const char *a2, const char *a3)
+const char *
+BAN_AddTest(struct ban_proto *bp,
+    const char *a1, const char *a2, const char *a3)
 {
 	const struct pvar *pv;
-	int i;
+	const char *err;
 
-	CHECK_OBJ_NOTNULL(b, BAN_MAGIC);
-	AN(b->vsb);
+	CHECK_OBJ_NOTNULL(bp, BAN_PROTO_MAGIC);
+	AN(bp->vsb);
 	AN(a1);
 	AN(a2);
 	AN(a3);
 
-	if (b->flags & BANS_FLAG_ERROR)
-		return (-1);
+	if (bp->err != NULL)
+		return (bp->err);
 
 	for (pv = pvars; pv->name != NULL; pv++)
 		if (!strncmp(a1, pv->name, strlen(pv->name)))
 			break;
 
 	if (pv->name == NULL)
-		return (ban_error(b,
+		return (ban_error(bp,
 		    "Unknown or unsupported field \"%s\"", a1));
 
-	b->flags |= pv->flag;
+	bp->flags |= pv->flag;
 
-	VSB_putc(b->vsb, pv->tag);
+	VSB_putc(bp->vsb, pv->tag);
 	if (pv->flag & BANS_FLAG_HTTP)
-		ban_parse_http(b, a1 + strlen(pv->name));
+		ban_parse_http(bp, a1 + strlen(pv->name));
 
-	ban_add_lump(b, a3, strlen(a3) + 1);
+	ban_add_lump(bp, a3, strlen(a3) + 1);
 	if (!strcmp(a2, "~")) {
-		VSB_putc(b->vsb, BANS_OPER_MATCH);
-		i = ban_parse_regexp(b, a3);
-		if (i)
-			return (i);
+		VSB_putc(bp->vsb, BANS_OPER_MATCH);
+		err = ban_parse_regexp(bp, a3);
+		if (err)
+			return (err);
 	} else if (!strcmp(a2, "!~")) {
-		VSB_putc(b->vsb, BANS_OPER_NMATCH);
-		i = ban_parse_regexp(b, a3);
-		if (i)
-			return (i);
+		VSB_putc(bp->vsb, BANS_OPER_NMATCH);
+		err = ban_parse_regexp(bp, a3);
+		if (err)
+			return (err);
 	} else if (!strcmp(a2, "==")) {
-		VSB_putc(b->vsb, BANS_OPER_EQ);
+		VSB_putc(bp->vsb, BANS_OPER_EQ);
 	} else if (!strcmp(a2, "!=")) {
-		VSB_putc(b->vsb, BANS_OPER_NEQ);
+		VSB_putc(bp->vsb, BANS_OPER_NEQ);
 	} else {
-		return (ban_error(b,
+		return (ban_error(bp,
 		    "expected conditional (~, !~, == or !=) got \"%s\"", a2));
 	}
-	return (0);
+	return (NULL);
 }
 
 /*--------------------------------------------------------------------
@@ -197,121 +249,80 @@ BAN_AddTest(struct ban *b, const char *a1, const char *a2, const char *a3)
  *      deleted.
  */
 
-static char ban_error_nomem[] = "Could not get memory";
-
-static char *
-ban_ins_error(const char *p)
+const char *
+BAN_Commit(struct ban_proto *bp)
 {
-	char *r = NULL;
-
-	if (p != NULL)
-		r = strdup(p);
-	if (r == NULL)
-		r = ban_error_nomem;
-	return (r);
-}
-
-void
-BAN_Free_Errormsg(char *p)
-{
-	if (p != ban_error_nomem)
-		free(p);
-}
-
-char *
-BAN_Insert(struct ban *b)
-{
-	struct ban  *bi, *be;
+	struct ban  *b, *bi;
 	ssize_t ln;
 	double t0;
-	char *p;
 
-	CHECK_OBJ_NOTNULL(b, BAN_MAGIC);
-	AN(b->vsb);
+	CHECK_OBJ_NOTNULL(bp, BAN_PROTO_MAGIC);
+	AN(bp->vsb);
 
-	if (ban_shutdown) {
-		BAN_Free(b);
-		return (ban_ins_error("Shutting down"));
-	}
+	if (ban_shutdown)
+		return (ban_error(bp, "Shutting down"));
 
-	AZ(VSB_finish(b->vsb));
-	ln = VSB_len(b->vsb);
+	AZ(VSB_finish(bp->vsb));
+	ln = VSB_len(bp->vsb);
 	assert(ln >= 0);
 
-	if (b->flags & BANS_FLAG_ERROR) {
-		p = ban_ins_error(VSB_data(b->vsb));
-		BAN_Free(b);
-		return (p);
-	}
+	ALLOC_OBJ(b, BAN_MAGIC);
+	if (b == NULL)
+		return (ban_error(bp, ban_build_err_no_mem));
+	VTAILQ_INIT(&b->objcore);
 
 	b->spec = malloc(ln + BANS_HEAD_LEN);
 	if (b->spec == NULL) {
-		BAN_Free(b);
-		return (ban_ins_error(NULL));
+		free(b);
+		return (ban_error(bp, ban_build_err_no_mem));
 	}
 
+	b->flags = bp->flags;
+
 	memset(b->spec, 0, BANS_HEAD_LEN);
 	t0 = VTIM_real();
 	memcpy(b->spec + BANS_TIMESTAMP, &t0, sizeof t0);
 	b->spec[BANS_FLAGS] = b->flags & 0xff;
-	memcpy(b->spec + BANS_HEAD_LEN, VSB_data(b->vsb), ln);
+	memcpy(b->spec + BANS_HEAD_LEN, VSB_data(bp->vsb), ln);
 	ln += BANS_HEAD_LEN;
 	vbe32enc(b->spec + BANS_LENGTH, ln);
 
-	VSB_delete(b->vsb);
-	b->vsb = NULL;
-
 	Lck_Lock(&ban_mtx);
 	if (ban_shutdown) {
-		/* Check again, we might have raced */
+		/* We could have raced a shutdown */
 		Lck_Unlock(&ban_mtx);
 		BAN_Free(b);
-		return (ban_ins_error("Shutting down"));
+		return (ban_error(bp, "Shutting down"));
 	}
+	bi = VTAILQ_FIRST(&ban_head);
 	VTAILQ_INSERT_HEAD(&ban_head, b, list);
 	ban_start = b;
+
 	VSC_C_main->bans++;
 	VSC_C_main->bans_added++;
+	VSC_C_main->bans_persisted_bytes += ln;
+
 	if (b->flags & BANS_FLAG_OBJ)
 		VSC_C_main->bans_obj++;
 	if (b->flags & BANS_FLAG_REQ)
 		VSC_C_main->bans_req++;
 
-	be = VTAILQ_LAST(&ban_head, banhead_s);
-	if (cache_param->ban_dups && be != b)
-		be->refcount++;
-	else
-		be = NULL;
-
-	/* ban_magic is magic, and needs to be inserted early to give
-	 * a handle to grab a ref on. We don't report it here as the
-	 * stevedores will not be opened and ready to accept it
-	 * yet. Instead it is reported on BAN_Compile, which is after
-	 * the stevedores has been opened, but before any new objects
-	 * can have entered the cache (thus no objects in the mean
-	 * time depending on ban_magic in the list) */
-	VSC_C_main->bans_persisted_bytes += ln;
-	if (b != ban_magic)
-		ban_info(BI_NEW, b->spec, ln); /* Notify stevedores */
-	Lck_Unlock(&ban_mtx);
-
-	if (be == NULL)
-		return (NULL);
-
-	/* Hunt down duplicates, and mark them as completed */
-	bi = b;
-	Lck_Lock(&ban_mtx);
-	while (!ban_shutdown && bi != be) {
-		bi = VTAILQ_NEXT(bi, list);
-		if (bi->flags & BANS_FLAG_COMPLETED)
-			continue;
-		if (!ban_equal(b->spec, bi->spec))
-			continue;
-		ban_mark_completed(bi);
-		VSC_C_main->bans_dups++;
+	if (bi != NULL)
+		ban_info(BI_NEW, b->spec, ln);	/* Notify stevedores */
+
+	if (cache_param->ban_dups) {
+		/* Hunt down duplicates, and mark them as completed */
+		for (bi = VTAILQ_NEXT(b, list); bi != NULL;
+		    bi = VTAILQ_NEXT(bi, list)) {
+			if (!(bi->flags & BANS_FLAG_COMPLETED) &&
+			    ban_equal(b->spec, bi->spec)) {
+				ban_mark_completed(bi);
+				VSC_C_main->bans_dups++;
+			}
+		}
 	}
-	be->refcount--;
 	Lck_Unlock(&ban_mtx);
 
+	BAN_Abandon(bp);
 	return (NULL);
 }
diff --git a/bin/varnishd/cache/cache_vrt.c b/bin/varnishd/cache/cache_vrt.c
index 642e754..09441e0 100644
--- a/bin/varnishd/cache/cache_vrt.c
+++ b/bin/varnishd/cache/cache_vrt.c
@@ -395,15 +395,16 @@ VRT_ban_string(VRT_CTX, const char *str)
 {
 	char *a1, *a2, *a3;
 	char **av;
-	struct ban *b;
+	struct ban_proto *bp;
+	const char *err;
 	int i;
 
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
 	AN(ctx->vsl);
 	AN(str);
 
-	b = BAN_New();
-	if (b == NULL) {
+	bp = BAN_Build();
+	if (bp == NULL) {
 		VSLb(ctx->vsl, SLT_VCL_Error, "ban(): Out of Memory");
 		return;
 	}
@@ -412,7 +413,7 @@ VRT_ban_string(VRT_CTX, const char *str)
 	if (av[0] != NULL) {
 		VSLb(ctx->vsl, SLT_VCL_Error, "ban(): %s", av[0]);
 		VAV_Free(av);
-		BAN_Free(b);
+		BAN_Abandon(bp);
 		return;
 	}
 	for (i = 0; ;) {
@@ -434,13 +435,17 @@ VRT_ban_string(VRT_CTX, const char *str)
 			    "ban(): Expected second operand.");
 			break;
 		}
-		if (BAN_AddTest(b, a1, a2, a3) || av[++i] == NULL) {
-			a1 = BAN_Insert(b);
-			if (a1 != NULL) {
-				VSLb(ctx->vsl, SLT_VCL_Error,
-				    "ban(): %s", a1);
-				BAN_Free_Errormsg(a1);
-			}
+		err = BAN_AddTest(bp, a1, a2, a3);
+		if (err) {
+			VSLb(ctx->vsl, SLT_VCL_Error, "ban(): %s", err);
+			break;
+		}
+		if (av[++i] == NULL) {
+			err = BAN_Commit(bp);
+			if (err == NULL)
+				bp = NULL;
+			else
+				VSLb(ctx->vsl, SLT_VCL_Error, "ban(): %s", err);
 			break;
 		}
 		if (strcmp(av[i], "&&")) {
@@ -450,6 +455,8 @@ VRT_ban_string(VRT_CTX, const char *str)
 			break;
 		}
 	}
+	if (bp != NULL)
+		BAN_Abandon(bp);
 	VAV_Free(av);
 }
 
diff --git a/bin/varnishd/flint.lnt b/bin/varnishd/flint.lnt
index 5904b1e..38453d1 100644
--- a/bin/varnishd/flint.lnt
+++ b/bin/varnishd/flint.lnt
@@ -106,6 +106,7 @@
 
 -emacro(527, NEEDLESS_RETURN)	// unreachable code
 
+-sem(BAN_Free, custodial(1))
 -sem(EXP_Inject, custodial(1))
 -sem(HSH_Insert, custodial(3))
 -sem(WS_Init, custodial(2))



From lkarsten at varnish-software.com  Thu Jan 14 14:15:02 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:02 +0100
Subject: [4.1] 978b5b0 Sigh... almost all fun in C gets outlawed these days...
Message-ID: <E1aJig2-00050j-9C@project.varnish-software.com>


commit 978b5b07811a9cff2eff18c232ab2a1f43b633ee
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu Oct 22 15:57:14 2015 +0000

    Sigh... almost all fun in C gets outlawed these days...

diff --git a/bin/varnishd/cache/cache_ban_build.c b/bin/varnishd/cache/cache_ban_build.c
index eb10c30..bec9062 100644
--- a/bin/varnishd/cache/cache_ban_build.c
+++ b/bin/varnishd/cache/cache_ban_build.c
@@ -93,7 +93,7 @@ BAN_Abandon(struct ban_proto *bp)
 	CHECK_OBJ_NOTNULL(bp, BAN_PROTO_MAGIC);
 	VSB_delete(bp->vsb);
 	if (bp->err != NULL && bp->err != ban_build_err_no_mem)
-		REPLACE(bp->err, NULL);
+		free(bp->err);
 	FREE_OBJ(bp);
 }
 



From lkarsten at varnish-software.com  Thu Jan 14 14:15:02 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:02 +0100
Subject: [4.1] c0dfa2a Don't double-free.
Message-ID: <E1aJig2-00050s-D0@project.varnish-software.com>


commit c0dfa2ae67cc73262e3c7461e4a6300df24b041a
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu Oct 22 18:00:41 2015 +0000

    Don't double-free.
    
    Amazingly JEMALLOC didn't spot this one.

diff --git a/bin/varnishd/cache/cache_ban_build.c b/bin/varnishd/cache/cache_ban_build.c
index bec9062..b807c9a 100644
--- a/bin/varnishd/cache/cache_ban_build.c
+++ b/bin/varnishd/cache/cache_ban_build.c
@@ -92,8 +92,6 @@ BAN_Abandon(struct ban_proto *bp)
 
 	CHECK_OBJ_NOTNULL(bp, BAN_PROTO_MAGIC);
 	VSB_delete(bp->vsb);
-	if (bp->err != NULL && bp->err != ban_build_err_no_mem)
-		free(bp->err);
 	FREE_OBJ(bp);
 }
 



From lkarsten at varnish-software.com  Thu Jan 14 14:15:02 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:02 +0100
Subject: [4.1] c6289aa Make the ban lurker sleep on a condvar so other code
 can wake it up.
Message-ID: <E1aJig2-00051D-JY@project.varnish-software.com>


commit c6289aaed5ccedfa6659f585626654a265c14143
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu Oct 22 20:34:18 2015 +0000

    Make the ban lurker sleep on a condvar so other code can wake it up.

diff --git a/bin/varnishd/cache/cache_ban.c b/bin/varnishd/cache/cache_ban.c
index 95ca144..6d66999 100644
--- a/bin/varnishd/cache/cache_ban.c
+++ b/bin/varnishd/cache/cache_ban.c
@@ -756,6 +756,7 @@ BAN_Init(void)
 	bp = BAN_Build();
 	AN(bp);
 	AZ(BAN_Commit(bp));
+	AZ(pthread_cond_init(&ban_lurker_cond, NULL));
 	Lck_Lock(&ban_mtx);
 	ban_mark_completed(VTAILQ_FIRST(&ban_head));
 	Lck_Unlock(&ban_mtx);
@@ -774,9 +775,8 @@ BAN_Shutdown(void)
 {
 	void *status;
 
-	Lck_Lock(&ban_mtx);
 	ban_shutdown = 1;
-	Lck_Unlock(&ban_mtx);
+	ban_kick_lurker();
 
 	AZ(pthread_join(ban_thread, &status));
 	AZ(status);
diff --git a/bin/varnishd/cache/cache_ban.h b/bin/varnishd/cache/cache_ban.h
index 047d0f9..100acee 100644
--- a/bin/varnishd/cache/cache_ban.h
+++ b/bin/varnishd/cache/cache_ban.h
@@ -105,6 +105,7 @@ extern struct lock ban_mtx;
 extern int ban_shutdown;
 extern struct banhead_s ban_head;
 extern struct ban * volatile ban_start;
+extern pthread_cond_t	ban_lurker_cond;
 
 void ban_mark_completed(struct ban *b);
 unsigned ban_len(const uint8_t *banspec);
@@ -114,3 +115,4 @@ int ban_evaluate(struct worker *wrk, const uint8_t *bs, struct objcore *oc,
 double ban_time(const uint8_t *banspec);
 int ban_equal(const uint8_t *bs1, const uint8_t *bs2);
 void BAN_Free(struct ban *b);
+void ban_kick_lurker(void);
diff --git a/bin/varnishd/cache/cache_ban_lurker.c b/bin/varnishd/cache/cache_ban_lurker.c
index cfcdd26..821d8c8 100644
--- a/bin/varnishd/cache/cache_ban_lurker.c
+++ b/bin/varnishd/cache/cache_ban_lurker.c
@@ -38,6 +38,18 @@
 
 static struct objcore oc_marker = { .magic = OBJCORE_MAGIC, };
 static unsigned ban_batch;
+static unsigned ban_generation;
+
+pthread_cond_t	ban_lurker_cond;
+
+void
+ban_kick_lurker(void)
+{
+	Lck_Lock(&ban_mtx);
+	ban_generation++;
+	AZ(pthread_cond_signal(&ban_lurker_cond));
+	Lck_Unlock(&ban_mtx);
+}
 
 static void
 ban_cleantail(void)
@@ -254,7 +266,10 @@ ban_lurker(struct worker *wrk, void *priv)
 		if (d <= 0.0 || !ban_lurker_work(wrk, &vsl))
 			d = 0.609;	// Random, non-magic
 		ban_cleantail();
-		VTIM_sleep(d);
+		d += VTIM_real();
+		Lck_Lock(&ban_mtx);
+		(void)Lck_CondWait(&ban_lurker_cond, &ban_mtx, d);
+		Lck_Unlock(&ban_mtx);
 	}
 	pthread_exit(0);
 	NEEDLESS_RETURN(NULL);



From lkarsten at varnish-software.com  Thu Jan 14 14:15:02 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:02 +0100
Subject: [4.1] cfeb9fb Make sure complaints from (je)malloc aren't lost,
 at least on FreeBSD.
Message-ID: <E1aJig2-000514-FJ@project.varnish-software.com>


commit cfeb9fb38d21df7267ccb41dcac01a9893d7c668
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu Oct 22 19:31:28 2015 +0000

    Make sure complaints from (je)malloc aren't lost, at least on FreeBSD.

diff --git a/bin/varnishd/cache/cache_main.c b/bin/varnishd/cache/cache_main.c
index e47f685..cc5471c 100644
--- a/bin/varnishd/cache/cache_main.c
+++ b/bin/varnishd/cache/cache_main.c
@@ -188,6 +188,16 @@ static struct cli_proto debug_cmds[] = {
  * XXX: Think more about which order we start things
  */
 
+#ifdef __FreeBSD__
+static void
+child_malloc_fail(void *p, const char *s)
+{
+	VSL(SLT_Error, 0, "MALLOC ERROR: %s (%p)", s, p);
+	fprintf(stderr, "MALLOC ERROR: %s (%p)\n", s, p);
+	WRONG("Malloc Error");
+}
+#endif
+
 void
 child_main(void)
 {
@@ -195,6 +205,9 @@ child_main(void)
 	setbuf(stdout, NULL);
 	setbuf(stderr, NULL);
 	printf("Child starts\n");
+#ifdef __FreeBSD__
+	malloc_message = child_malloc_fail;
+#endif
 
 	cache_param = heritage.param;
 



From lkarsten at varnish-software.com  Thu Jan 14 14:15:02 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:02 +0100
Subject: [4.1] 56b1329 Kick the ban_lurker into action whenever we do
 something that makes work for it. Have it sleep as long as possible,
 rather than hot-polling for work to do.
Message-ID: <E1aJig2-00051O-Np@project.varnish-software.com>


commit 56b13297c89550dadbedbdb3611936aa0dce868e
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu Oct 22 22:17:34 2015 +0000

    Kick the ban_lurker into action whenever we do something that
    makes work for it.  Have it sleep as long as possible, rather
    than hot-polling for work to do.

diff --git a/bin/varnishd/cache/cache_ban.c b/bin/varnishd/cache/cache_ban.c
index 6d66999..3bf9073 100644
--- a/bin/varnishd/cache/cache_ban.c
+++ b/bin/varnishd/cache/cache_ban.c
@@ -552,6 +552,9 @@ BAN_CheckObject(struct worker *wrk, struct objcore *oc, struct req *req)
 		b0->refcount++;
 	}
 
+	if (oc->ban->refcount == 0 && VTAILQ_NEXT(oc->ban, list) == NULL)
+		ban_kick_lurker();
+
 	Lck_Unlock(&ban_mtx);
 
 	if (b == oc->ban) {	/* not banned */
@@ -703,6 +706,7 @@ ccf_ban_list(struct cli *cli, const char * const *av, void *priv)
 
 	Lck_Lock(&ban_mtx);
 	bl->refcount--;
+	ban_kick_lurker();	// XXX: Mostly for testcase b00009.vtc
 	Lck_Unlock(&ban_mtx);
 }
 
@@ -775,8 +779,10 @@ BAN_Shutdown(void)
 {
 	void *status;
 
+	Lck_Lock(&ban_mtx);
 	ban_shutdown = 1;
 	ban_kick_lurker();
+	Lck_Unlock(&ban_mtx);
 
 	AZ(pthread_join(ban_thread, &status));
 	AZ(status);
diff --git a/bin/varnishd/cache/cache_ban_build.c b/bin/varnishd/cache/cache_ban_build.c
index b807c9a..801774f 100644
--- a/bin/varnishd/cache/cache_ban_build.c
+++ b/bin/varnishd/cache/cache_ban_build.c
@@ -319,6 +319,8 @@ BAN_Commit(struct ban_proto *bp)
 			}
 		}
 	}
+	if (!(b->flags & BANS_FLAG_REQ))
+		ban_kick_lurker();
 	Lck_Unlock(&ban_mtx);
 
 	BAN_Abandon(bp);
diff --git a/bin/varnishd/cache/cache_ban_lurker.c b/bin/varnishd/cache/cache_ban_lurker.c
index 821d8c8..352b219 100644
--- a/bin/varnishd/cache/cache_ban_lurker.c
+++ b/bin/varnishd/cache/cache_ban_lurker.c
@@ -45,10 +45,10 @@ pthread_cond_t	ban_lurker_cond;
 void
 ban_kick_lurker(void)
 {
-	Lck_Lock(&ban_mtx);
+
+	Lck_AssertHeld(&ban_mtx);
 	ban_generation++;
 	AZ(pthread_cond_signal(&ban_lurker_cond));
-	Lck_Unlock(&ban_mtx);
 }
 
 static void
@@ -191,14 +191,17 @@ ban_lurker_test_ban(struct worker *wrk, struct vsl_log *vsl, struct ban *bt,
  * Ban lurker thread
  */
 
-static int
+static double
 ban_lurker_work(struct worker *wrk, struct vsl_log *vsl)
 {
 	struct ban *b, *bt;
 	struct banhead_s obans;
-	double d;
+	double d, dt, n;
 	int i;
 
+	dt = 49.62;		// Random, non-magic
+	if (cache_param->ban_lurker_sleep == 0)
+		return (dt);
 	/* Make a list of the bans we can do something about */
 	VTAILQ_INIT(&obans);
 	Lck_Lock(&ban_mtx);
@@ -206,26 +209,27 @@ ban_lurker_work(struct worker *wrk, struct vsl_log *vsl)
 	Lck_Unlock(&ban_mtx);
 	i = 0;
 	d = VTIM_real() - cache_param->ban_lurker_age;
-	while (b != NULL) {
-		if (b->flags & BANS_FLAG_COMPLETED) {
-			;
-		} else if (b->flags & BANS_FLAG_REQ) {
-			;
-		} else if (b == VTAILQ_LAST(&ban_head, banhead_s)) {
-			;
-		} else if (ban_time(b->spec) > d) {
-			;
-		} else {
+	for (; b != NULL; b = VTAILQ_NEXT(b, list)) {
+		if (b->flags & BANS_FLAG_COMPLETED)
+			continue;
+		if (b->flags & BANS_FLAG_REQ)
+			continue;
+		if (b == VTAILQ_LAST(&ban_head, banhead_s))
+			continue;	// XXX: why ?
+		n = ban_time(b->spec) - d;
+		if (n < 0) {
 			VTAILQ_INSERT_TAIL(&obans, b, l_list);
 			i++;
+		} else if (n < dt) {
+			dt = n;
 		}
-		b = VTAILQ_NEXT(b, list);
 	}
 	if (DO_DEBUG(DBG_LURKER))
-		VSLb(vsl, SLT_Debug, "lurker: %d actionable bans", i);
+		VSLb(vsl, SLT_Debug, "lurker: %d actionable bans, dt = %lf", i, dt);
 	if (i == 0)
-		return (0);
+		return (dt);
 
+	dt = cache_param->ban_lurker_sleep;
 	/* Go though all the bans to test the objects */
 	VTAILQ_FOREACH_REVERSE(bt, &ban_head, banhead_s, list) {
 		if (bt == VTAILQ_LAST(&obans, banhead_s)) {
@@ -247,7 +251,7 @@ ban_lurker_work(struct worker *wrk, struct vsl_log *vsl)
 		if (VTAILQ_EMPTY(&obans))
 			break;
 	}
-	return (1);
+	return (dt);
 }
 
 void * __match_proto__(bgthread_t)
@@ -255,6 +259,7 @@ ban_lurker(struct worker *wrk, void *priv)
 {
 	struct vsl_log vsl;
 	volatile double d;
+	unsigned gen = ban_generation + 1;
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	AZ(priv);
@@ -262,13 +267,17 @@ ban_lurker(struct worker *wrk, void *priv)
 	VSL_Setup(&vsl, NULL, 0);
 
 	while (!ban_shutdown) {
-		d = cache_param->ban_lurker_sleep;
-		if (d <= 0.0 || !ban_lurker_work(wrk, &vsl))
-			d = 0.609;	// Random, non-magic
+		d = ban_lurker_work(wrk, &vsl);
+		if (DO_DEBUG(DBG_LURKER))
+			VSLb(&vsl, SLT_Debug, "lurker: sleep = %lf", d);
 		ban_cleantail();
 		d += VTIM_real();
 		Lck_Lock(&ban_mtx);
-		(void)Lck_CondWait(&ban_lurker_cond, &ban_mtx, d);
+		if (gen == ban_generation) {
+			(void)Lck_CondWait(&ban_lurker_cond, &ban_mtx, d);
+			ban_batch = 0;
+		}
+		gen = ban_generation;
 		Lck_Unlock(&ban_mtx);
 	}
 	pthread_exit(0);
diff --git a/bin/varnishtest/tests/c00049.vtc b/bin/varnishtest/tests/c00049.vtc
index 40f7b94..8d10e68 100644
--- a/bin/varnishtest/tests/c00049.vtc
+++ b/bin/varnishtest/tests/c00049.vtc
@@ -40,6 +40,7 @@ varnish v1 -vcl+backend {} -start
 varnish v1 -cliok "param.set ban_lurker_age 0"
 varnish v1 -cliok "param.set ban_lurker_sleep 0"
 varnish v1 -cliok "param.set debug +lurker"
+varnish v1 -cliok "param.set debug +syncvsl"
 
 
 client c1 {
@@ -114,6 +115,8 @@ varnish v1 -expect bans_dups == 0
 
 varnish v1 -cliok "param.set ban_lurker_sleep .01"
 
+varnish v1 -cliok "ban.list"
+
 delay 2
 
 varnish v1 -cliok "ban.list"
diff --git a/bin/varnishtest/tests/p00009.vtc b/bin/varnishtest/tests/p00009.vtc
index 6ec15d4..21d65a6 100644
--- a/bin/varnishtest/tests/p00009.vtc
+++ b/bin/varnishtest/tests/p00009.vtc
@@ -53,5 +53,4 @@ client c1 {
 
 # Expect our duplicate
 varnish v1 -expect bans_dups == 1
-# One more than before restart due to the new ban_magic
-varnish v1 -expect bans_completed == 3
+varnish v1 -expect bans_completed == 1
diff --git a/bin/varnishtest/tests/r01030.vtc b/bin/varnishtest/tests/r01030.vtc
index 302b918..3f7343a 100644
--- a/bin/varnishtest/tests/r01030.vtc
+++ b/bin/varnishtest/tests/r01030.vtc
@@ -40,9 +40,9 @@ client c1 {
 } -run
 
 #delay 0.1
-varnish v1 -expect bans_lurker_tests_tested == 0
+#varnish v1 -expect bans_lurker_tests_tested == 0
 
-delay 1.5
+#delay 1.5
 varnish v1 -expect bans_lurker_tests_tested >= 1
 
 varnish v1 -cliok "param.set ban_lurker_sleep 5.01"
@@ -58,7 +58,7 @@ client c2 {
 } -run
 
 #delay 0.1
-varnish v1 -expect bans_lurker_tests_tested == 1
+#varnish v1 -expect bans_lurker_tests_tested == 1
 
-delay 1.1
+#delay 1.1
 varnish v1 -expect bans_lurker_tests_tested == 2
diff --git a/include/tbl/params.h b/include/tbl/params.h
index d2650f4..1968df4 100644
--- a/include/tbl/params.h
+++ b/include/tbl/params.h
@@ -135,10 +135,10 @@ PARAM(
 	/* units */	"seconds",
 	/* flags */	0,
 	/* s-text */
-	"The ban lurker only process bans when they are this old.  "
-	"When a ban is added, the most frequently hit objects will "
-	"get tested against it as part of object lookup.  This parameter "
-	"prevents the ban-lurker from kicking in, until the rush is over.",
+	"The ban lurker will ignore bans until they are this old.  "
+	"When a ban is added, the active traffic will be tested against it "
+	"as part of object lookup.  This parameter "
+	"holds the ban-lurker off, until the rush is over.",
 	/* l-text */	"",
 	/* func */	NULL
 )
@@ -152,9 +152,9 @@ PARAM(
 	/* units */	NULL,
 	/* flags */	0,
 	/* s-text */
-	"The ban lurker slees ${ban_lurker_sleep} after examining this "
-	"many objects.  Use this to pace the ban-lurker if it eats too "
-	"many resources.",
+	"The ban lurker sleeps ${ban_lurker_sleep} after examining this "
+	"many objects."
+	"  Use this to pace the ban-lurker if it eats too many resources.",
 	/* l-text */	"",
 	/* func */	NULL
 )
@@ -169,7 +169,8 @@ PARAM(
 	/* flags */	0,
 	/* s-text */
 	"How long the ban lurker sleeps after examining ${ban_lurker_batch} "
-	"objects.\n"
+	"objects."
+	"  Use this to pace the ban-lurker if it eats too many resources.\n"
 	"A value of zero will disable the ban lurker entirely.",
 	/* l-text */	"",
 	/* func */	NULL



From lkarsten at varnish-software.com  Thu Jan 14 14:15:02 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:02 +0100
Subject: [4.1] 12e38ae Do not force python3
Message-ID: <E1aJig2-00051b-QL@project.varnish-software.com>


commit 12e38ae1cccd23c79569c5b178ac152733f97b7f
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Thu Oct 22 18:51:29 2015 +0100

    Do not force python3
    
    Use `python' as with the other Python scripts.

diff --git a/lib/libvcc/generate.py b/lib/libvcc/generate.py
index f8aeb79..7a7413f 100755
--- a/lib/libvcc/generate.py
+++ b/lib/libvcc/generate.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
 #-
 # Copyright (c) 2006 Verdens Gang AS
 # Copyright (c) 2006-2015 Varnish Software AS



From lkarsten at varnish-software.com  Thu Jan 14 14:15:02 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:02 +0100
Subject: [4.1] f4843a8 First stab at fixing some corner case
Message-ID: <E1aJig2-00051m-TA@project.varnish-software.com>


commit f4843a84491e6b64fdaef1f8426b85cb6c6cbd69
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Fri Oct 23 05:10:20 2015 +0100

    First stab at fixing some corner case
    
    Found by ingvar while packaging 4.1.0.

diff --git a/bin/varnishtest/tests/v00017.vtc b/bin/varnishtest/tests/v00017.vtc
index b0d6c20..e00ad4f 100644
--- a/bin/varnishtest/tests/v00017.vtc
+++ b/bin/varnishtest/tests/v00017.vtc
@@ -36,9 +36,9 @@ varnish v1 -errvcl {DNS lookup(...com): } {
 	sub vcl_recv { if (client.ip ~ a) { return(pass); } }
 }
 
-varnish v1 -errvcl {DNS lookup(10.1.2.): } {
+varnish v1 -errvcl {DNS lookup(10.1..2): } {
 	backend b { .host = "127.0.0.1"; }
-	acl a { "10.1.2."; }
+	acl a { "10.1..2"; }
 	sub vcl_recv { if (client.ip ~ a) { return(pass); } }
 }
 



From lkarsten at varnish-software.com  Thu Jan 14 14:15:02 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:02 +0100
Subject: [4.1] f418da9 Add a note to clarify the event function's priv object.
Message-ID: <E1aJig2-00051x-W0@project.varnish-software.com>


commit f418da9896cf0196eb1154bca9e76e2b60c0e49e
Author: Dag Haavi Finstad <daghf at varnish-software.com>
Date:   Sun Oct 25 20:22:05 2015 +0100

    Add a note to clarify the event function's priv object.

diff --git a/doc/sphinx/reference/vmod.rst b/doc/sphinx/reference/vmod.rst
index 697fc6a..91953b0 100644
--- a/doc/sphinx/reference/vmod.rst
+++ b/doc/sphinx/reference/vmod.rst
@@ -302,7 +302,8 @@ The VCL compiler supports the following private pointers:
 * ``PRIV_VCL`` "per vcl" private pointers are useful for such global
   state that applies to all calls in this VCL, for instance flags that
   determine if regular expressions are case-sensitive in this vmod or
-  similar.
+  similar. The ``PRIV_VCL`` object is the same object that is passed
+  to the VMOD's event function.
 
 The way it works in the vmod code, is that a ``struct vmod_priv *`` is
 passed to the functions where one of the ``PRIV_*`` argument types is



From lkarsten at varnish-software.com  Thu Jan 14 14:15:03 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:03 +0100
Subject: [4.1] a75a278 Update RFC reference
Message-ID: <E1aJig3-000527-1v@project.varnish-software.com>


commit a75a278e7ebedac0e34232e216f945881bd1d523
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Sun Oct 25 23:16:28 2015 +0000

    Update RFC reference
    
    Fixes #1809.

diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index 8ee198b..8e0a70b 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -342,7 +342,7 @@ vbf_stp_startfetch(struct worker *wrk, struct busyobj *bo)
 	} else if (http_IsStatus(bo->beresp, 204)) {
 		/*
 		 * 204 is "No Content", obviously don't expect a body.
-		 * [RFC2616 10.2.5 p60]
+		 * [RFC7230 3.3.1 p28 and 3.3.2 p30]
 		 */
 		wrk->stats->fetch_204++;
 		if (http_GetHdr(bo->beresp, H_Content_Length, NULL) ||



From lkarsten at varnish-software.com  Thu Jan 14 14:15:03 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:03 +0100
Subject: [4.1] b5d305e Remove unnecessary check
Message-ID: <E1aJig3-00052K-4G@project.varnish-software.com>


commit b5d305ee904400fd9b412dfafc05b84fa6a46f56
Author: Guillaume Quintard <guillaume at varnish-software.com>
Date:   Mon Oct 26 10:59:05 2015 +0100

    Remove unnecessary check

diff --git a/include/miniobj.h b/include/miniobj.h
index 3de4091..8b4f043 100644
--- a/include/miniobj.h
+++ b/include/miniobj.h
@@ -61,8 +61,7 @@
 
 #define REPLACE(ptr, val)						\
 	do {								\
-		if ((ptr) != NULL)					\
-			free(ptr);					\
+		free(ptr);						\
 		if ((val) != NULL) {					\
 			ptr = strdup(val);				\
 			AN((ptr));					\



From lkarsten at varnish-software.com  Thu Jan 14 14:15:03 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:03 +0100
Subject: [4.1] 65e3466 Require -w when running in daemon mode
Message-ID: <E1aJig3-00052U-6Z@project.varnish-software.com>


commit 65e34661479855c190f97b24a7660de1e1d11467
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Mon Oct 26 13:50:15 2015 +0000

    Require -w when running in daemon mode

diff --git a/bin/varnishlog/varnishlog.c b/bin/varnishlog/varnishlog.c
index 3b5061f..8e600b5 100644
--- a/bin/varnishlog/varnishlog.c
+++ b/bin/varnishlog/varnishlog.c
@@ -145,6 +145,9 @@ main(int argc, char * const *argv)
 	if (optind != argc)
 		usage(1);
 
+	if (VUT.D_opt && !LOG.w_arg)
+		VUT_Error(1, "Missing -w option");
+
 	/* Setup output */
 	if (LOG.A_opt || !LOG.w_arg)
 		VUT.dispatch_f = VSL_PrintTransactions;
diff --git a/bin/varnishlog/varnishlog_options.h b/bin/varnishlog/varnishlog_options.h
index 187ac57..ed62abd 100644
--- a/bin/varnishlog/varnishlog_options.h
+++ b/bin/varnishlog/varnishlog_options.h
@@ -49,7 +49,8 @@
 	    " receives a SIGHUP the file will be reopened allowing"	\
 	    " the old one to be rotated away. The file can then be"	\
 	    " read by varnishlog and other tools with the -r option,"   \
-	    " unless the -A option was specified."			\
+	    " unless the -A option was specified. This option is"	\
+	    " required when running in daemon mode."			\
 	)
 
 LOG_OPT_a
diff --git a/bin/varnishncsa/varnishncsa.c b/bin/varnishncsa/varnishncsa.c
index 77e6354..2ee2137 100644
--- a/bin/varnishncsa/varnishncsa.c
+++ b/bin/varnishncsa/varnishncsa.c
@@ -972,6 +972,9 @@ main(int argc, char * const *argv)
 	if (optind != argc)
 		usage(1);
 
+	if (VUT.D_opt && !CTX.w_arg)
+		VUT_Error(1, "Missing -w option");
+
 	/* Check for valid grouping mode */
 	assert(VUT.g_arg < VSL_g__MAX);
 	if (VUT.g_arg != VSL_g_vxid && VUT.g_arg != VSL_g_request)
diff --git a/bin/varnishncsa/varnishncsa_options.h b/bin/varnishncsa/varnishncsa_options.h
index edb5a39..87075cc 100644
--- a/bin/varnishncsa/varnishncsa_options.h
+++ b/bin/varnishncsa/varnishncsa_options.h
@@ -51,7 +51,8 @@
 	    "Redirect output to file. The file will be overwritten"	\
 	    " unless the -a option was specified. If the application"	\
 	    " receives a SIGHUP the file will be reopened allowing"	\
-	    " the old one to be rotated away."				\
+	    " the old one to be rotated away. This option is required"	\
+	    " when running in daemon mode."				\
 	)
 
 NCSA_OPT_a



From lkarsten at varnish-software.com  Thu Jan 14 14:15:03 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:03 +0100
Subject: [4.1] 01a6d89 Handle terminal disconnections correctly
Message-ID: <E1aJig3-00052h-8c@project.varnish-software.com>


commit 01a6d89a58500111d9b92749a267b3c22ae4b847
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Mon Oct 26 13:54:29 2015 +0000

    Handle terminal disconnections correctly
    
    Change SIGHUP handling depending on whether we're running in daemon mode
    or foreground. The former will continue rotating the logs, the latter
    will abort the loop and die gracefully.
    
    Fixes #1794.

diff --git a/bin/varnishhist/varnishhist.c b/bin/varnishhist/varnishhist.c
index aec72e9..9eab9bf 100644
--- a/bin/varnishhist/varnishhist.c
+++ b/bin/varnishhist/varnishhist.c
@@ -291,6 +291,12 @@ accumulate(struct VSL_data *vsl, struct VSL_transaction * const pt[],
 	return (0);
 }
 
+static int __match_proto__(VUT_cb_f)
+sighup(void)
+{
+	return (1);
+}
+
 static void *
 do_curses(void *arg)
 {
@@ -465,6 +471,7 @@ main(int argc, char **argv)
 	}
 	VUT.dispatch_f = &accumulate;
 	VUT.dispatch_priv = NULL;
+	VUT.sighup_f = sighup;
 	VUT_Main();
 	end_of_file = 1;
 	AZ(pthread_join(thr, NULL));
diff --git a/bin/varnishlog/varnishlog.c b/bin/varnishlog/varnishlog.c
index 8e600b5..bc4a915 100644
--- a/bin/varnishlog/varnishlog.c
+++ b/bin/varnishlog/varnishlog.c
@@ -110,6 +110,12 @@ flushout(void)
 	return (0);
 }
 
+static int __match_proto__(VUT_cb_f)
+sighup(void)
+{
+	return (1);
+}
+
 int
 main(int argc, char * const *argv)
 {
@@ -153,10 +159,12 @@ main(int argc, char * const *argv)
 		VUT.dispatch_f = VSL_PrintTransactions;
 	else
 		VUT.dispatch_f = VSL_WriteTransactions;
+	VUT.sighup_f = sighup;
 	if (LOG.w_arg) {
 		openout(LOG.a_opt);
 		AN(LOG.fo);
-		VUT.sighup_f = rotateout;
+		if (VUT.D_opt)
+			VUT.sighup_f = rotateout;
 	} else
 		LOG.fo = stdout;
 	VUT.idle_f = flushout;
diff --git a/bin/varnishlog/varnishlog_options.h b/bin/varnishlog/varnishlog_options.h
index ed62abd..7b23a6c 100644
--- a/bin/varnishlog/varnishlog_options.h
+++ b/bin/varnishlog/varnishlog_options.h
@@ -46,11 +46,11 @@
 	VOPT("w:", "[-w filename]", "Output filename",			\
 	    "Redirect output to file. The file will be overwritten"	\
 	    " unless the -a option was specified. If the application"	\
-	    " receives a SIGHUP the file will be reopened allowing"	\
-	    " the old one to be rotated away. The file can then be"	\
-	    " read by varnishlog and other tools with the -r option,"   \
-	    " unless the -A option was specified. This option is"	\
-	    " required when running in daemon mode."			\
+	    " receives a SIGHUP in daemon mode the file will be "	\
+	    " reopened allowing the old one to be rotated away. The"	\
+	    " file can then be read by varnishlog and other tools with"	\
+	    " the -r option, unless the -A option was specified. This"	\
+	    " option is required when running in daemon mode."		\
 	)
 
 LOG_OPT_a
diff --git a/bin/varnishncsa/varnishncsa.c b/bin/varnishncsa/varnishncsa.c
index 2ee2137..b19b679 100644
--- a/bin/varnishncsa/varnishncsa.c
+++ b/bin/varnishncsa/varnishncsa.c
@@ -927,6 +927,12 @@ dispatch_f(struct VSL_data *vsl, struct VSL_transaction * const pt[],
 	return (0);
 }
 
+static int __match_proto__(VUT_cb_f)
+sighup(void)
+{
+	return (1);
+}
+
 int
 main(int argc, char * const *argv)
 {
@@ -991,10 +997,12 @@ main(int argc, char * const *argv)
 	/* Setup output */
 	VUT.dispatch_f = &dispatch_f;
 	VUT.dispatch_priv = NULL;
+	VUT.sighup_f = sighup;
 	if (CTX.w_arg) {
 		openout(CTX.a_opt);
 		AN(CTX.fo);
-		VUT.sighup_f = &rotateout;
+		if (VUT.D_opt)
+			VUT.sighup_f = &rotateout;
 	} else
 		CTX.fo = stdout;
 	VUT.idle_f = &flushout;
diff --git a/bin/varnishncsa/varnishncsa_options.h b/bin/varnishncsa/varnishncsa_options.h
index 87075cc..291a1c7 100644
--- a/bin/varnishncsa/varnishncsa_options.h
+++ b/bin/varnishncsa/varnishncsa_options.h
@@ -41,7 +41,7 @@
 	)
 
 #define NCSA_OPT_g							\
-	VOPT("g:", "[-g <request|vxid>]", "Grouping mode (default: vxid)",		\
+	VOPT("g:", "[-g <request|vxid>]", "Grouping mode (default: vxid)", \
 	    "The grouping of the log records. The default is to group"	\
 	    " by vxid."							\
 	)
@@ -50,9 +50,9 @@
 	VOPT("w:", "[-w filename]", "Output filename",			\
 	    "Redirect output to file. The file will be overwritten"	\
 	    " unless the -a option was specified. If the application"	\
-	    " receives a SIGHUP the file will be reopened allowing"	\
-	    " the old one to be rotated away. This option is required"	\
-	    " when running in daemon mode."				\
+	    " receives a SIGHUP in daemon mode the file will be"	\
+	    " reopened allowing the old one to be rotated away. This"	\
+	    " option is required when running in daemon mode."		\
 	)
 
 NCSA_OPT_a
diff --git a/bin/varnishtest/tests/u00001.vtc b/bin/varnishtest/tests/u00001.vtc
index 8f35294..1a1c039 100644
--- a/bin/varnishtest/tests/u00001.vtc
+++ b/bin/varnishtest/tests/u00001.vtc
@@ -12,7 +12,7 @@ server s1 {
 
 varnish v1 -vcl+backend "" -start
 
-process p1 "exec ${varnishncsa} -n ${v1_name} -w ${tmpdir}/ncsa.log -F %s" -start
+shell "${varnishncsa} -D -P ${tmpdir}/ncsa.pid -n ${v1_name} -w ${tmpdir}/ncsa.log -F %s"
 
 # give varnishncsa enough time to open the VSM
 delay 2
@@ -27,7 +27,7 @@ delay 2
 
 # rotate logs
 shell "mv ${tmpdir}/ncsa.log ${tmpdir}/ncsa.old.log >/dev/null 2>&1"
-process p1 -kill "HUP"
+shell "kill -HUP `cat ${tmpdir}/ncsa.pid`"
 
 client c1 {
 	txreq -url "/bar"
@@ -37,7 +37,7 @@ client c1 {
 # give varnishncsa enough time to write
 delay 2
 
-process p1 -stop
+shell "kill `cat ${tmpdir}/ncsa.pid`"
 
 shell "grep 200 ${tmpdir}/ncsa.old.log >/dev/null"
 shell "grep 404 ${tmpdir}/ncsa.log >/dev/null"
diff --git a/bin/varnishtop/varnishtop.c b/bin/varnishtop/varnishtop.c
index 3ab9009..f56b9a7 100644
--- a/bin/varnishtop/varnishtop.c
+++ b/bin/varnishtop/varnishtop.c
@@ -180,6 +180,12 @@ accumulate(struct VSL_data *vsl, struct VSL_transaction * const pt[],
 	return (0);
 }
 
+static int __match_proto__(VUT_cb_f)
+sighup(void)
+{
+	return (1);
+}
+
 static void
 update(int p)
 {
@@ -362,6 +368,7 @@ main(int argc, char **argv)
 	}
 	VUT.dispatch_f = &accumulate;
 	VUT.dispatch_priv = NULL;
+	VUT.sighup_f = sighup;
 	VUT_Main();
 	end_of_file = 1;
 	if (once)



From lkarsten at varnish-software.com  Thu Jan 14 14:15:03 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:03 +0100
Subject: [4.1] 979fbc6 Fix building on FreeBSD < 10.0
Message-ID: <E1aJig3-00052u-AY@project.varnish-software.com>


commit 979fbc6026635112fff9c50a270eac586447c457
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Mon Oct 26 16:46:18 2015 +0000

    Fix building on FreeBSD < 10.0

diff --git a/bin/varnishd/cache/cache_main.c b/bin/varnishd/cache/cache_main.c
index cc5471c..83de0cd 100644
--- a/bin/varnishd/cache/cache_main.c
+++ b/bin/varnishd/cache/cache_main.c
@@ -188,7 +188,7 @@ static struct cli_proto debug_cmds[] = {
  * XXX: Think more about which order we start things
  */
 
-#ifdef __FreeBSD__
+#if defined(__FreeBSD__) && __FreeBSD__version >= 1000000
 static void
 child_malloc_fail(void *p, const char *s)
 {
@@ -205,7 +205,7 @@ child_main(void)
 	setbuf(stdout, NULL);
 	setbuf(stderr, NULL);
 	printf("Child starts\n");
-#ifdef __FreeBSD__
+#if defined(__FreeBSD__) && __FreeBSD__version >= 1000000
 	malloc_message = child_malloc_fail;
 #endif
 



From lkarsten at varnish-software.com  Thu Jan 14 14:15:03 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:03 +0100
Subject: [4.1] 6b2cc56 Log proxy related messages on the session,
 not on the request.
Message-ID: <E1aJig3-000533-Dd@project.varnish-software.com>


commit 6b2cc56eb26da79c08c16dea5e897be3dd34fa05
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Fri Oct 23 15:51:58 2015 +0200

    Log proxy related messages on the session, not on the request.
    
    The proxy log records were attempted to be logged on the request' log
    buffer, which at that point in time has not yet been set up and does
    not posess a VXID. This caused VXID==0 log records to be inserted in
    the beginning of the log for the first request on the session when
    using proxy protocol, and subsequently the VSL to fail picking them
    out as valid request log transactions.
    
    Fix by logging the PROXY handling related messages on the session, in
    which they belong.
    
    Fixes: #1804

diff --git a/bin/varnishd/proxy/cache_proxy_proto.c b/bin/varnishd/proxy/cache_proxy_proto.c
index 433581e..954beef 100644
--- a/bin/varnishd/proxy/cache_proxy_proto.c
+++ b/bin/varnishd/proxy/cache_proxy_proto.c
@@ -62,8 +62,9 @@ vpx_proto1(const struct worker *wrk, struct req *req)
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
+	CHECK_OBJ_NOTNULL(req->sp, SESS_MAGIC);
 
-	VSL(SLT_Debug, req->sp->fd, "PROXY1");
+	VSL(SLT_Debug, req->sp->vxid, "PROXY1");
 
 	q = strchr(req->htc->rxbuf_b, '\r');
 	if (q == NULL)
@@ -80,7 +81,7 @@ vpx_proto1(const struct worker *wrk, struct req *req)
 	for (i = 0; i < 5; i++) {
 		p = strchr(p, ' ');
 		if (p == NULL) {
-			VSLb(req->vsl, SLT_ProxyGarbage,
+			VSL(SLT_ProxyGarbage, req->sp->vxid,
 			    "PROXY1: Too few fields");
 			return (-1);
 		}
@@ -89,7 +90,7 @@ vpx_proto1(const struct worker *wrk, struct req *req)
 	}
 
 	if (strchr(p, ' ')) {
-		VSLb(req->vsl, SLT_ProxyGarbage,
+		VSL(SLT_ProxyGarbage, req->sp->vxid,
 		    "PROXY1: Too many fields");
 		return (-1);
 	}
@@ -102,7 +103,7 @@ vpx_proto1(const struct worker *wrk, struct req *req)
 	else if (!strcmp(fld[0], "TCP6"))
 		pfam = AF_INET6;
 	else {
-		VSLb(req->vsl, SLT_ProxyGarbage,
+		VSL(SLT_ProxyGarbage, req->sp->vxid,
 		    "PROXY1: Wrong TCP[46] field");
 		return (-1);
 	}
@@ -113,14 +114,14 @@ vpx_proto1(const struct worker *wrk, struct req *req)
 
 	i = getaddrinfo(fld[1], fld[3], &hints, &res);
 	if (i != 0) {
-		VSLb(req->vsl, SLT_ProxyGarbage,
+		VSL(SLT_ProxyGarbage, req->sp->vxid,
 		    "PROXY1: Cannot resolve source address (%s)",
 		    gai_strerror(i));
 		return (-1);
 	}
 	AZ(res->ai_next);
 	if (res->ai_family != pfam) {
-		VSLb(req->vsl, SLT_ProxyGarbage,
+		VSL(SLT_ProxyGarbage, req->sp->vxid,
 		    "PROXY1: %s got wrong protocol (%d)",
 		    fld[0], res->ai_family);
 		freeaddrinfo(res);
@@ -134,14 +135,14 @@ vpx_proto1(const struct worker *wrk, struct req *req)
 
 	i = getaddrinfo(fld[2], fld[4], &hints, &res);
 	if (i != 0) {
-		VSLb(req->vsl, SLT_ProxyGarbage,
+		VSL(SLT_ProxyGarbage, req->sp->vxid,
 		    "PROXY1: Cannot resolve destination address (%s)",
 		    gai_strerror(i));
 		return (-1);
 	}
 	AZ(res->ai_next);
 	if (res->ai_family != pfam) {
-		VSLb(req->vsl, SLT_ProxyGarbage,
+		VSL(SLT_ProxyGarbage, req->sp->vxid,
 		    "PROXY1: %s got wrong protocol (%d)",
 		    fld[0], res->ai_family);
 		freeaddrinfo(res);
@@ -151,7 +152,7 @@ vpx_proto1(const struct worker *wrk, struct req *req)
 	AN(VSA_Build(sa, res->ai_addr, res->ai_addrlen));
 	freeaddrinfo(res);
 
-	VSLb(req->vsl, SLT_Proxy, "1 %s %s %s %s",
+	VSL(SLT_Proxy, req->sp->vxid, "1 %s %s %s %s",
 	    fld[1], fld[3], fld[2], fld[4]);
 	req->htc->pipeline_b = q;
 	return (0);
@@ -182,6 +183,7 @@ vpx_proto2(const struct worker *wrk, struct req *req)
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
+	CHECK_OBJ_NOTNULL(req->sp, SESS_MAGIC);
 
 	assert(req->htc->rxbuf_e - req->htc->rxbuf_b >= 16L);
 	l = vbe16dec(req->htc->rxbuf_b + 14);
@@ -191,7 +193,7 @@ vpx_proto2(const struct worker *wrk, struct req *req)
 
 	/* Version @12 top half */
 	if ((p[12] >> 4) != 2) {
-		VSLb(req->vsl, SLT_ProxyGarbage,
+		VSL(SLT_ProxyGarbage, req->sp->vxid,
 		    "PROXY2: bad version (%d)", p[12] >> 4);
 		return (-1);
 	}
@@ -205,7 +207,7 @@ vpx_proto2(const struct worker *wrk, struct req *req)
 		/* Proxied connection */
 		break;
 	default:
-		VSLb(req->vsl, SLT_ProxyGarbage,
+		VSL(SLT_ProxyGarbage, req->sp->vxid,
 		    "PROXY2: bad command (%d)", p[12] & 0x0f);
 		return (-1);
 	}
@@ -214,14 +216,14 @@ vpx_proto2(const struct worker *wrk, struct req *req)
 	switch(p[13]) {
 	case 0x00:
 		/* UNSPEC|UNSPEC, ignore proxy header */
-		VSLb(req->vsl, SLT_ProxyGarbage,
+		VSL(SLT_ProxyGarbage, req->sp->vxid,
 		    "PROXY2: Ignoring UNSPEC|UNSPEC addresses");
 		return (0);
 	case 0x11:
 		/* IPv4|TCP */
 		pfam = AF_INET;
 		if (l < 12) {
-			VSLb(req->vsl, SLT_ProxyGarbage,
+			VSL(SLT_ProxyGarbage, req->sp->vxid,
 			    "PROXY2: Ignoring short IPv4 addresses (%d)", l);
 			return (0);
 		}
@@ -230,14 +232,14 @@ vpx_proto2(const struct worker *wrk, struct req *req)
 		/* IPv6|TCP */
 		pfam = AF_INET6;
 		if (l < 36) {
-			VSLb(req->vsl, SLT_ProxyGarbage,
+			VSL(SLT_ProxyGarbage, req->sp->vxid,
 			    "PROXY2: Ignoring short IPv6 addresses (%d)", l);
 			return (0);
 		}
 		break;
 	default:
 		/* Ignore proxy header */
-		VSLb(req->vsl, SLT_ProxyGarbage,
+		VSL(SLT_ProxyGarbage, req->sp->vxid,
 		    "PROXY2: Ignoring unsupported protocol (0x%02x)", p[13]);
 		return (0);
 	}
@@ -286,7 +288,7 @@ vpx_proto2(const struct worker *wrk, struct req *req)
 	SES_Set_String_Attr(req->sp, SA_CLIENT_IP, hb);
 	SES_Set_String_Attr(req->sp, SA_CLIENT_PORT, pb);
 
-	VSLb(req->vsl, SLT_Proxy, "2 %s %s %s %s", hb, pb, ha, pa);
+	VSL(SLT_Proxy, req->sp->vxid, "2 %s %s %s %s", hb, pb, ha, pa);
 	return (0);
 }
 
diff --git a/bin/varnishtest/tests/r01804.vtc b/bin/varnishtest/tests/r01804.vtc
new file mode 100644
index 0000000..c6705eb
--- /dev/null
+++ b/bin/varnishtest/tests/r01804.vtc
@@ -0,0 +1,39 @@
+varnishtest "#1804: varnishapi transaction grouping fails for PROXY"
+
+server s1 {
+	rxreq
+	txresp
+} -start
+
+
+varnish v1 -proto "PROXY" -vcl+backend {
+} -start
+
+logexpect l1 -v v1 -d 0 -g session {
+	expect * *	Begin	{^sess .* PROXY$}
+	expect * =	Proxy	{^1 }
+	expect * *	Begin	{^req}
+	expect * *	Begin	{^sess .* PROXY$}
+	expect * =	Proxy	{^2 }
+	expect * *	Begin	{^req}
+} -start
+
+client c1 {
+	send "PROXY TCP4 1.2.3.4 5.6.7.8 1234 5678\r\n"
+	txreq
+	rxresp
+} -run
+
+client c2 {
+	# good IPv4
+	sendhex "0d 0a 0d 0a 00 0d 0a 51 55 49 54 0a"
+	sendhex "21 11 00 0c"
+	sendhex "01 02 03 04"
+	sendhex "05 06 07 08"
+	sendhex "09 0a"
+	sendhex "0b 0c"
+	txreq
+	rxresp
+} -run
+
+logexpect l1 -wait



From lkarsten at varnish-software.com  Thu Jan 14 14:15:03 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:03 +0100
Subject: [4.1] be937f9 Remove some debug log records in the proxy v1 code
Message-ID: <E1aJig3-00053D-FM@project.varnish-software.com>


commit be937f9031cd4cab226f9fadd5175d60c71c3e33
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Tue Oct 27 15:37:03 2015 +0100

    Remove some debug log records in the proxy v1 code

diff --git a/bin/varnishd/proxy/cache_proxy_proto.c b/bin/varnishd/proxy/cache_proxy_proto.c
index 954beef..604d796 100644
--- a/bin/varnishd/proxy/cache_proxy_proto.c
+++ b/bin/varnishd/proxy/cache_proxy_proto.c
@@ -64,8 +64,6 @@ vpx_proto1(const struct worker *wrk, struct req *req)
 	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
 	CHECK_OBJ_NOTNULL(req->sp, SESS_MAGIC);
 
-	VSL(SLT_Debug, req->sp->vxid, "PROXY1");
-
 	q = strchr(req->htc->rxbuf_b, '\r');
 	if (q == NULL)
 		return (-1);
@@ -95,9 +93,6 @@ vpx_proto1(const struct worker *wrk, struct req *req)
 		return (-1);
 	}
 
-	VSL(SLT_Debug, req->sp->fd, "PROXY1 <%s> <%s> <%s> <%s> <%s>",
-	    fld[0], fld[1], fld[2], fld[3], fld[4]);
-
 	if (!strcmp(fld[0], "TCP4"))
 		pfam = AF_INET;
 	else if (!strcmp(fld[0], "TCP6"))



From lkarsten at varnish-software.com  Thu Jan 14 14:15:03 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:03 +0100
Subject: [4.1] c902892 Don't attempt to allocate a V1L from the workspace if
 it is overflowed.
Message-ID: <E1aJig3-00053P-Gq@project.varnish-software.com>


commit c9028924e230e5eb13372aedd84e81144e701d37
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Oct 30 10:26:29 2015 +0000

    Don't attempt to allocate a V1L from the workspace if it is overflowed.
    
    Fixes:	#1796
    Also triggered by:	The Wemm-Field

diff --git a/bin/varnishd/http1/cache_http1_deliver.c b/bin/varnishd/http1/cache_http1_deliver.c
index 997d28b..0135ed8 100644
--- a/bin/varnishd/http1/cache_http1_deliver.c
+++ b/bin/varnishd/http1/cache_http1_deliver.c
@@ -112,10 +112,12 @@ V1D_Deliver(struct req *req, struct busyobj *bo, int sendbody)
 	if (sendbody && req->resp_len != 0)
 		VDP_push(req, v1d_bytes, NULL, 1);
 
+	AZ(req->wrk->v1l);
 	V1L_Reserve(req->wrk, req->ws, &req->sp->fd, req->vsl, req->t_prev);
 
 	if (WS_Overflowed(req->ws)) {
 		v1d_error(req, "workspace_client overflow");
+		AZ(req->wrk->v1l);
 		return;
 	}
 
@@ -134,5 +136,6 @@ V1D_Deliver(struct req *req, struct busyobj *bo, int sendbody)
 
 	if ((V1L_FlushRelease(req->wrk) || ois != OIS_DONE) && req->sp->fd >= 0)
 		SES_Close(req->sp, SC_REM_CLOSE);
+	AZ(req->wrk->v1l);
 	VDP_close(req);
 }
diff --git a/bin/varnishd/http1/cache_http1_line.c b/bin/varnishd/http1/cache_http1_line.c
index 9c49fca..9f13543 100644
--- a/bin/varnishd/http1/cache_http1_line.c
+++ b/bin/varnishd/http1/cache_http1_line.c
@@ -78,6 +78,8 @@ V1L_Reserve(struct worker *wrk, struct ws *ws, int *fd, struct vsl_log *vsl,
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	AZ(wrk->v1l);
 
+	if (WS_Overflowed(ws))
+		return;
 	res = WS_Snapshot(ws);
 	v1l = WS_Alloc(ws, sizeof *v1l);
 	if (v1l == NULL)



From lkarsten at varnish-software.com  Thu Jan 14 14:15:03 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:03 +0100
Subject: [4.1] d327423 So one of those strange cornercases in HTTP/1
Message-ID: <E1aJig3-00053Z-Iu@project.varnish-software.com>


commit d327423897e1f49323496640224227935046ebf1
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Oct 30 14:22:11 2015 +0000

    So one of those strange cornercases in HTTP/1
    
    If we send the backend a HTTP/1.0 request, and it doesn't have a
    Content-Length, it cannot use Chunked and must fall back to EOF.
    
    However, the protocol field in the response tells us what version
    backend *could* have used, not what it *does* use.
    
    So we can get a response with HTTP/1.1 and EOF, following HTTP/1.0
    semantics - because we asked for it.
    
    Most sensible backends avoid this, either by buffering and creation
    of a C-L or, smartly, returning "HTTP/1.0", even though that
    is strictly speaking against the apocrphal texts.
    
    Anyway, now we cope...
    
    Fixes: 	#1810

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index 93e5de4..e7700d0 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -805,7 +805,8 @@ enum sess_close http_DoConnection(struct http *hp);
 
 htc_complete_f HTTP1_Complete;
 uint16_t HTTP1_DissectRequest(struct http_conn *, struct http *);
-uint16_t HTTP1_DissectResponse(struct http_conn *, struct http *);
+uint16_t HTTP1_DissectResponse(struct http_conn *, struct http *resp,
+    const struct http *req);
 unsigned HTTP1_Write(const struct worker *w, const struct http *hp, const int*);
 
 #define HTTPH(a, b, c) extern char b[];
diff --git a/bin/varnishd/http1/cache_http1_fetch.c b/bin/varnishd/http1/cache_http1_fetch.c
index 1b6ceb4..a69fb92 100644
--- a/bin/varnishd/http1/cache_http1_fetch.c
+++ b/bin/varnishd/http1/cache_http1_fetch.c
@@ -193,7 +193,7 @@ V1F_FetchRespHdr(struct busyobj *bo)
 
 	hp = bo->beresp;
 
-	i = HTTP1_DissectResponse(htc, hp);
+	i = HTTP1_DissectResponse(htc, hp, bo->bereq);
 	bo->acct.beresp_hdrbytes += htc->rxbuf_e - htc->rxbuf_b;
 	if (i) {
 		VSLb(bo->vsl, SLT_FetchError, "http format error");
diff --git a/bin/varnishd/http1/cache_http1_proto.c b/bin/varnishd/http1/cache_http1_proto.c
index 8d08278..deb191d 100644
--- a/bin/varnishd/http1/cache_http1_proto.c
+++ b/bin/varnishd/http1/cache_http1_proto.c
@@ -321,15 +321,15 @@ http1_body_status(const struct http *hp, struct http_conn *htc)
 
 /*--------------------------------------------------------------------*/
 
-static void
-http1_proto_ver(struct http *hp)
+static int8_t
+http1_proto_ver(const struct http *hp)
 {
 	if (!strcasecmp(hp->hd[HTTP_HDR_PROTO].b, "HTTP/1.0"))
-		hp->protover = 10;
+		return (10);
 	else if (!strcasecmp(hp->hd[HTTP_HDR_PROTO].b, "HTTP/1.1"))
-		hp->protover = 11;
+		return (11);
 	else
-		hp->protover = 0;
+		return (0);
 }
 
 /*--------------------------------------------------------------------*/
@@ -347,7 +347,7 @@ HTTP1_DissectRequest(struct http_conn *htc, struct http *hp)
 	retval = http1_splitline(hp, htc, HTTP1_Req);
 	if (retval != 0)
 		return (retval);
-	http1_proto_ver(hp);
+	hp->protover = http1_proto_ver(hp);
 	if (hp->protover == 0)
 		return (400);
 
@@ -391,22 +391,28 @@ HTTP1_DissectRequest(struct http_conn *htc, struct http *hp)
 /*--------------------------------------------------------------------*/
 
 uint16_t
-HTTP1_DissectResponse(struct http_conn *htc, struct http *hp)
+HTTP1_DissectResponse(struct http_conn *htc, struct http *hp,
+    const struct http *req)
 {
 	uint16_t retval = 0;
 	const char *p;
+	int8_t rv;
 
 
 	CHECK_OBJ_NOTNULL(htc, HTTP_CONN_MAGIC);
 	CHECK_OBJ_NOTNULL(hp, HTTP_MAGIC);
+	CHECK_OBJ_NOTNULL(req, HTTP_MAGIC);
 
 	if (http1_splitline(hp, htc, HTTP1_Resp))
 		retval = 503;
 
 	if (retval == 0) {
-		http1_proto_ver(hp);
+		hp->protover = http1_proto_ver(hp);
 		if (hp->protover == 0)
 			retval = 503;
+		rv = http1_proto_ver(req);
+		if (hp->protover > rv)
+			hp->protover = rv;
 	}
 
 	if (retval == 0 && Tlen(hp->hd[HTTP_HDR_STATUS]) != 3)
diff --git a/bin/varnishtest/tests/r01810.vtc b/bin/varnishtest/tests/r01810.vtc
new file mode 100644
index 0000000..77e84db
--- /dev/null
+++ b/bin/varnishtest/tests/r01810.vtc
@@ -0,0 +1,21 @@
+varnishtest "POST HTTP/1.0 response"
+
+server s1 {
+	non-fatal
+	rxreq
+	txresp -proto HTTP/1.1 -nolen -hdr "Connection: close"
+	send "Hello World\n"
+	delay .4
+} -start
+
+varnish v1 -vcl+backend { 
+	sub vcl_backend_fetch {
+		set bereq.proto = "HTTP/1.0";
+	}
+} -start
+
+client c1 {
+	txreq -req POST -hdr "Content-Length: 0"
+	rxresp
+	expect resp.bodylen == 12
+} -run



From lkarsten at varnish-software.com  Thu Jan 14 14:15:03 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:03 +0100
Subject: [4.1] abc86a8 Convert arguments to strings as well
Message-ID: <E1aJig3-00053j-KK@project.varnish-software.com>


commit abc86a8a9f90c5cfc4e849413efe22ffc1c31ec3
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Thu Oct 1 11:06:07 2015 +0100

    Convert arguments to strings as well
    
    This is consistent with what we do in other cases.

diff --git a/bin/varnishtest/tests/d00003.vtc b/bin/varnishtest/tests/d00003.vtc
index 0c95d61..56b3035 100644
--- a/bin/varnishtest/tests/d00003.vtc
+++ b/bin/varnishtest/tests/d00003.vtc
@@ -5,6 +5,8 @@ server s1 {
 	txresp -hdr "Foo: 1" -body "1"
 	rxreq
 	txresp -hdr "Foo: 3" -body "3"
+	rxreq
+	txresp -hdr "Foo: 9" -body "9"
 } -start
 
 server s2 {
@@ -28,19 +30,18 @@ varnish v1 -vcl+backend {
 	}
 
 	sub vcl_recv {
-		return(pass);
-	}
-
-	sub vcl_backend_fetch {
-		if (bereq.url == "/nohdr") {
-			set bereq.backend = h1.backend(bereq.http.Void);
-		} else if (bereq.url == "/emptystring") {
-			set bereq.backend = h1.backend("");
-		} else if (bereq.url == "/13") {
-			set bereq.backend = h1.backend(bereq.http.Void + "" + bereq.url);
+		if (req.url == "/nohdr") {
+			set req.backend_hint = h1.backend(req.http.Void);
+		} else if (req.url == "/emptystring") {
+			set req.backend_hint = h1.backend("");
+		} else if (req.url == "/13") {
+			set req.backend_hint = h1.backend(req.http.Void + "" + req.url);
+		} else if (req.url == "/ip") {
+			set req.backend_hint = h1.backend(client.ip);
 		} else {
-			set bereq.backend = h1.backend(bereq.url);
+			set req.backend_hint = h1.backend(req.url);
 		}
+		return (pass);
 	}
 } -start
 
@@ -68,4 +69,8 @@ client c1 {
 	txreq -url /nohdr
 	rxresp
 	expect resp.http.foo == "8"
+
+	txreq -url /ip
+	rxresp
+	expect resp.http.foo == "9"
 } -run
diff --git a/lib/libvcc/vcc_expr.c b/lib/libvcc/vcc_expr.c
index 294d12a..b498a53 100644
--- a/lib/libvcc/vcc_expr.c
+++ b/lib/libvcc/vcc_expr.c
@@ -626,6 +626,9 @@ vcc_do_arg(struct vcc *tl, struct func_arg *fa)
 	} else {
 		vcc_expr0(tl, &e2, fa->type);
 		ERRCHK(tl);
+		if (e2->fmt != fa->type &&
+		    (fa->type == STRING || fa->type == STRING_LIST))
+			vcc_expr_tostring(tl, &e2, fa->type);
 		if (e2->fmt != fa->type) {
 			VSB_printf(tl->sb, "Wrong argument type.");
 			VSB_printf(tl->sb, "  Expected %s.",



From lkarsten at varnish-software.com  Thu Jan 14 14:15:03 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:03 +0100
Subject: [4.1] fc9b45f Fail if multiple -a arguments return the same suckaddr.
Message-ID: <E1aJig3-00053t-MP@project.varnish-software.com>


commit fc9b45f167b2b5c1b206e8bdaf4a00e06617c654
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Nov 2 11:47:44 2015 +0000

    Fail if multiple -a arguments return the same suckaddr.
    
    Fixes	#1813

diff --git a/bin/varnishd/mgt/mgt_acceptor.c b/bin/varnishd/mgt/mgt_acceptor.c
index 658aca4..49cc0c8 100644
--- a/bin/varnishd/mgt/mgt_acceptor.c
+++ b/bin/varnishd/mgt/mgt_acceptor.c
@@ -128,6 +128,13 @@ mac_callback(void *priv, const struct suckaddr *sa)
 
 	CAST_OBJ_NOTNULL(mh, priv, MAC_HELP_MAGIC);
 
+	VTAILQ_FOREACH(ls, &heritage.socks, list) {
+		if (!VSA_Compare(sa, ls->addr)) {
+			ARGV_ERR("-a arguments %s and %s have same address\n",
+			    ls->name, mh->name);
+			return (-1);
+		}
+	}
 	ALLOC_OBJ(ls, LISTEN_SOCK_MAGIC);
 	AN(ls);
 	ls->sock = -1;
diff --git a/bin/varnishtest/tests/r01813.vtc b/bin/varnishtest/tests/r01813.vtc
new file mode 100644
index 0000000..f37aa54
--- /dev/null
+++ b/bin/varnishtest/tests/r01813.vtc
@@ -0,0 +1,3 @@
+varnishtest "Duplicate -a arguments"
+
+err_shell {have same address} "${varnishd} -d -a 127.0.0.1:38484 -a 127.0.0.1:38484 -b localhost:80 2>&1"



From lkarsten at varnish-software.com  Thu Jan 14 14:15:03 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:03 +0100
Subject: [4.1] b6e5463 Fix this test-case relative to the fix for #1813
Message-ID: <E1aJig3-000543-OM@project.varnish-software.com>


commit b6e54632c729a27b838dc84a65937f8b4d8e6175
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Nov 2 15:24:38 2015 +0000

    Fix this test-case relative to the fix for #1813

diff --git a/bin/varnishtest/tests/c00007.vtc b/bin/varnishtest/tests/c00007.vtc
index 4edff4f..5238226 100644
--- a/bin/varnishtest/tests/c00007.vtc
+++ b/bin/varnishtest/tests/c00007.vtc
@@ -1,5 +1,5 @@
 varnishtest "Test banning a hash"
 
-varnish v1 \
-	-arg "-b 127.0.0.1:80 -a 127.0.0.1:0" \
-	-start -clierr 101 "ban.hash foo"
+varnish v1 -arg "-b 127.0.0.1:80" -start 
+
+varnish v1 -clierr 101 "ban.hash foo"



From lkarsten at varnish-software.com  Thu Jan 14 14:15:03 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:03 +0100
Subject: [4.1] 4b52631 Use client.ip instead of client.identity
Message-ID: <E1aJig3-00054C-Qi@project.varnish-software.com>


commit 4b52631910d4d1bfc645610c9497340f770902cf
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Mon Nov 2 12:41:04 2015 +0000

    Use client.ip instead of client.identity
    
    Indentation while I'm here.

diff --git a/lib/libvmod_directors/vmod.vcc b/lib/libvmod_directors/vmod.vcc
index bb81751..02b5a3f 100644
--- a/lib/libvmod_directors/vmod.vcc
+++ b/lib/libvmod_directors/vmod.vcc
@@ -67,17 +67,17 @@ Example
 $Method VOID .add_backend(BACKEND)
 
 Description
-       Add a backend to the round-robin director.
+	Add a backend to the round-robin director.
 Example
-       vdir.add_backend(backend1);
-       vdir.add_backend(backend2);
+	vdir.add_backend(backend1);
+	vdir.add_backend(backend2);
 
 $Method BACKEND .backend()
 
 Description
-       Pick a backend from the director.
+	Pick a backend from the director.
 Example
-       set req.backend_hint = vdir.backend();
+	set req.backend_hint = vdir.backend();
 
 
 $Object fallback()
@@ -106,9 +106,9 @@ Example
 $Method BACKEND .backend()
 
 Description
-       Pick a backend from the director.
+	Pick a backend from the director.
 Example
-       set req.backend_hint = vdir.backend();
+	set req.backend_hint = vdir.backend();
 
 
 $Object random()
@@ -149,10 +149,10 @@ $Object hash()
 Description
 	Create a hashing backend director.
 
-	The director chooses the backend server by computing a hash/digest of
-	the string given to .backend().
+	The director chooses the backend server by computing a hash/digest
+	of the string given to .backend().
 
-	Commonly used with ``client.identity`` or a session cookie to get
+	Commonly used with ``client.ip`` or a session cookie to get
 	sticky sessions.
 
 Example



From lkarsten at varnish-software.com  Thu Jan 14 14:15:03 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:03 +0100
Subject: [4.1] 815a19c Squash trailing whitespace.
Message-ID: <E1aJig3-00054N-SZ@project.varnish-software.com>


commit 815a19cc97d8cb9d48366c205bd68d27850d8f89
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Wed Nov 4 10:06:53 2015 +0100

    Squash trailing whitespace.

diff --git a/configure.ac b/configure.ac
index ec02717..faee354 100644
--- a/configure.ac
+++ b/configure.ac
@@ -229,29 +229,29 @@ AC_CHECK_FUNCS([pthread_mutex_isowned_np])
 AC_CHECK_FUNCS([pthread_timedjoin_np])
 LIBS="${save_LIBS}"
 
-# Support for visibility attribute 
-save_CFLAGS="${CFLAGS}" 
-CFLAGS="${CFLAGS} -Werror" 
-
-AC_CACHE_CHECK([whether we have support for visibility attributes], 
-	[ac_cv_have_viz], 
-	[AC_RUN_IFELSE( 
-		[AC_LANG_PROGRAM([[ 
-			#if ((__GNUC__-0) * 10 + __GNUC_MINOR__-0 >= 33) 
-			#  define ZLIB_INTERNAL __attribute__((visibility ("hidden"))) 
-			#else 
-			#  define ZLIB_INTERNAL 
-			#endif 
-			int ZLIB_INTERNAL foo; 
-		]],[])], 
-	[ac_cv_have_viz=yes], 
-	[ac_cv_have_viz=no]) 
+# Support for visibility attribute
+save_CFLAGS="${CFLAGS}"
+CFLAGS="${CFLAGS} -Werror"
+
+AC_CACHE_CHECK([whether we have support for visibility attributes],
+	[ac_cv_have_viz],
+	[AC_RUN_IFELSE(
+		[AC_LANG_PROGRAM([[
+			#if ((__GNUC__-0) * 10 + __GNUC_MINOR__-0 >= 33)
+			#  define ZLIB_INTERNAL __attribute__((visibility ("hidden")))
+			#else
+			#  define ZLIB_INTERNAL
+			#endif
+			int ZLIB_INTERNAL foo;
+		]],[])],
+	[ac_cv_have_viz=yes],
+	[ac_cv_have_viz=no])
 ])
-if test "$ac_cv_have_viz" = no; then 
-	libvgz_extra_cflags="-DNO_VIZ" 
-	AC_SUBST(libvgz_extra_cflags) 
+if test "$ac_cv_have_viz" = no; then
+	libvgz_extra_cflags="-DNO_VIZ"
+	AC_SUBST(libvgz_extra_cflags)
 fi 
-CFLAGS="${save_CFLAGS}" 
+CFLAGS="${save_CFLAGS}"
 
 # Use jemalloc on Linux
 JEMALLOC_LDADD=



From lkarsten at varnish-software.com  Thu Jan 14 14:15:03 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:03 +0100
Subject: [4.1] ed1eb98 Relax IP constant parsing
Message-ID: <E1aJig3-00054W-V9@project.varnish-software.com>


commit ed1eb987ee9c2eb6224ff37b719c0df79d93b5a2
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Wed Nov 4 12:00:09 2015 +0000

    Relax IP constant parsing
    
    Fixes #1801.

diff --git a/bin/varnishtest/tests/r01801.vtc b/bin/varnishtest/tests/r01801.vtc
new file mode 100644
index 0000000..7052ead
--- /dev/null
+++ b/bin/varnishtest/tests/r01801.vtc
@@ -0,0 +1,58 @@
+varnishtest "Test parsing IP constants"
+
+server s1 {
+	rxreq
+	txresp
+} -start
+
+varnish v1 -vcl+backend {
+	import ${vmod_std};
+
+	sub vcl_deliver {
+		set resp.http.foo1 = std.ip("..", "1.2.3.4");
+		set resp.http.foo2 = std.ip("..", "1.2.3.4:8000");
+		set resp.http.foo3 = std.ip("..", "1.2.3.4 8000");
+		set resp.http.foo4 = std.ip("..", "::1");
+		set resp.http.foo5 = std.ip("..", "[::1]");
+		set resp.http.foo6 = std.ip("..", "[::1]:8000");
+		set resp.http.bar1 = std.port("1.2.3.4");
+		set resp.http.bar2 = std.port("1.2.3.4:8000");
+		set resp.http.bar3 = std.port("1.2.3.4 8000");
+		set resp.http.bar4 = std.port("::1");
+		set resp.http.bar5 = std.port("[::1]");
+		set resp.http.bar6 = std.port("[::1]:8000");
+	}
+} -start
+
+client c1 {
+	txreq
+	rxresp
+	expect resp.http.foo1 == "1.2.3.4"
+	expect resp.http.foo2 == "1.2.3.4"
+	expect resp.http.foo3 == "1.2.3.4"
+	expect resp.http.foo4 == "::1"
+	expect resp.http.foo5 == "::1"
+	expect resp.http.foo6 == "::1"
+	expect resp.http.bar1 == "80"
+	expect resp.http.bar2 == "8000"
+	expect resp.http.bar3 == "8000"
+	expect resp.http.bar4 == "80"
+	expect resp.http.bar5 == "80"
+	expect resp.http.bar6 == "8000"
+} -run
+
+varnish v1 -errvcl "could not be resolved to an IP address" {
+	import ${vmod_std};
+
+	sub vcl_deliver {
+		set resp.http.foo = std.ip("..", "::1::2");
+	}
+}
+
+varnish v1 -errvcl "could not be resolved to an IP address" {
+	import ${vmod_std};
+
+	sub vcl_deliver {
+		set resp.http.foo = std.ip("..", "1.2.3.4::80");
+	}
+}
diff --git a/lib/libvarnish/vss.c b/lib/libvarnish/vss.c
index 6d1b809..3d40473 100644
--- a/lib/libvarnish/vss.c
+++ b/lib/libvarnish/vss.c
@@ -56,6 +56,7 @@
  * "0.0.0.0" - "0.0.0.0:80"
  * "[::1]" - "[::1]:80"
  * "[::]" - "[::]:80"
+ * "::1"
  *
  * See also RFC5952
  */
@@ -86,6 +87,8 @@ vss_parse(char *str, char **addr, char **port)
 			p = strchr(str, ':');
 		if (p == NULL)
 			return (NULL);
+		if (p[0] == ':' && strchr(&p[1], ':'))
+			return (NULL);
 		if (p == str)
 			*addr = NULL;
 	}



From lkarsten at varnish-software.com  Thu Jan 14 14:15:04 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:04 +0100
Subject: [4.1] a5821f9 Make vcc_expr0() do the stringification (if necessary)
 rather than have all (but one) caller do it.
Message-ID: <E1aJig4-00054f-15@project.varnish-software.com>


commit a5821f9da893032045fc051adca43ee1fc15ef34
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu Nov 5 09:46:59 2015 +0000

    Make vcc_expr0() do the stringification (if necessary) rather than
    have all (but one) caller do it.

diff --git a/lib/libvcc/vcc_expr.c b/lib/libvcc/vcc_expr.c
index b498a53..5989f07 100644
--- a/lib/libvcc/vcc_expr.c
+++ b/lib/libvcc/vcc_expr.c
@@ -461,10 +461,6 @@ vcc_Eval_Regsub(struct vcc *tl, struct expr **e, const struct symbol *sym)
 	vcc_expr0(tl, &e2, STRING);
 	if (e2 == NULL)
 		return;
-	if (e2->fmt != STRING) {
-		vcc_expr_tostring(tl, &e2, STRING);
-		ERRCHK(tl);
-	}
 
 	SkipToken(tl, ',');
 	ExpectErr(tl, CSTR);
@@ -478,10 +474,6 @@ vcc_Eval_Regsub(struct vcc *tl, struct expr **e, const struct symbol *sym)
 	vcc_expr0(tl, &e2, STRING);
 	if (e2 == NULL)
 		return;
-	if (e2->fmt != STRING) {
-		vcc_expr_tostring(tl, &e2, STRING);
-		ERRCHK(tl);
-	}
 	*e = vcc_expr_edit(STRING, "\v1,\n\v2)\v-", *e, e2);
 	SkipToken(tl, ')');
 }
@@ -626,9 +618,6 @@ vcc_do_arg(struct vcc *tl, struct func_arg *fa)
 	} else {
 		vcc_expr0(tl, &e2, fa->type);
 		ERRCHK(tl);
-		if (e2->fmt != fa->type &&
-		    (fa->type == STRING || fa->type == STRING_LIST))
-			vcc_expr_tostring(tl, &e2, fa->type);
 		if (e2->fmt != fa->type) {
 			VSB_printf(tl->sb, "Wrong argument type.");
 			VSB_printf(tl->sb, "  Expected %s.",
@@ -1334,24 +1323,28 @@ vcc_expr0(struct vcc *tl, struct expr **e, enum var_type fmt)
 	*e = NULL;
 	vcc_expr_cand(tl, e, fmt);
 	ERRCHK(tl);
-	if ((*e)->fmt != BOOL || tl->t->tok != T_COR)
-		return;
-	*e = vcc_expr_edit(BOOL, "(\v+\n\v1", *e, NULL);
-	while (tl->t->tok == T_COR) {
-		vcc_NextToken(tl);
-		tk = tl->t;
-		vcc_expr_cand(tl, &e2, fmt);
-		ERRCHK(tl);
-		if (e2->fmt != BOOL) {
-			VSB_printf(tl->sb,
-			    "'||' must be followed by BOOL, found ");
-			VSB_printf(tl->sb, "%s.\n", vcc_Type(e2->fmt));
-			vcc_ErrWhere2(tl, tk, tl->t);
-			return;
+	if ((*e)->fmt == BOOL && tl->t->tok == T_COR) {
+		*e = vcc_expr_edit(BOOL, "(\v+\n\v1", *e, NULL);
+		while (tl->t->tok == T_COR) {
+			vcc_NextToken(tl);
+			tk = tl->t;
+			vcc_expr_cand(tl, &e2, fmt);
+			ERRCHK(tl);
+			if (e2->fmt != BOOL) {
+				VSB_printf(tl->sb,
+				    "'||' must be followed by BOOL, found ");
+				VSB_printf(tl->sb, "%s.\n", vcc_Type(e2->fmt));
+				vcc_ErrWhere2(tl, tk, tl->t);
+				return;
+			}
+			*e = vcc_expr_edit(BOOL, "\v1\v-\n||\v+\n\v2", *e, e2);
 		}
-		*e = vcc_expr_edit(BOOL, "\v1\v-\n||\v+\n\v2", *e, e2);
+		*e = vcc_expr_edit(BOOL, "\v1\v-\n)", *e, NULL);
+	}
+	if (fmt == STRING || fmt == STRING_LIST) {
+		vcc_expr_tostring(tl, e, fmt);
+		ERRCHK(tl);
 	}
-	*e = vcc_expr_edit(BOOL, "\v1\v-\n)", *e, NULL);
 }
 
 /*--------------------------------------------------------------------
@@ -1373,10 +1366,6 @@ vcc_Expr(struct vcc *tl, enum var_type fmt)
 	vcc_expr0(tl, &e, fmt);
 	ERRCHK(tl);
 	e->t1 = t1;
-	if (fmt == STRING || fmt == STRING_LIST) {
-		vcc_expr_tostring(tl, &e, fmt);
-		ERRCHK(tl);
-	}
 	if (!tl->err && fmt != e->fmt)  {
 		VSB_printf(tl->sb, "Expression has type %s, expected %s\n",
 		    vcc_Type(e->fmt), vcc_Type(fmt));



From lkarsten at varnish-software.com  Thu Jan 14 14:15:04 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:04 +0100
Subject: [4.1] 62165ec Fix proxy protocol v1 client port
Message-ID: <E1aJig4-00054q-2l@project.varnish-software.com>


commit 62165ec9328ccec83f329d3315ff9403fa16172e
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Sat Nov 7 18:03:55 2015 +0000

    Fix proxy protocol v1 client port
    
    Most likely missed when #1727 was addressed.
    
    Submitted by: Rafael Zalamena on varnish-dev

diff --git a/bin/varnishd/proxy/cache_proxy_proto.c b/bin/varnishd/proxy/cache_proxy_proto.c
index 604d796..6f64f3a 100644
--- a/bin/varnishd/proxy/cache_proxy_proto.c
+++ b/bin/varnishd/proxy/cache_proxy_proto.c
@@ -125,7 +125,7 @@ vpx_proto1(const struct worker *wrk, struct req *req)
 	SES_Reserve_client_addr(req->sp, &sa);
 	AN(VSA_Build(sa, res->ai_addr, res->ai_addrlen));
 	SES_Set_String_Attr(req->sp, SA_CLIENT_IP, fld[1]);
-	SES_Set_String_Attr(req->sp, SA_CLIENT_PORT, fld[2]);
+	SES_Set_String_Attr(req->sp, SA_CLIENT_PORT, fld[3]);
 	freeaddrinfo(res);
 
 	i = getaddrinfo(fld[2], fld[4], &hints, &res);
diff --git a/bin/varnishtest/tests/o00000.vtc b/bin/varnishtest/tests/o00000.vtc
index 0cb518a..7cebb63 100644
--- a/bin/varnishtest/tests/o00000.vtc
+++ b/bin/varnishtest/tests/o00000.vtc
@@ -134,6 +134,11 @@ client c1 {
 } -run
 delay .1
 
+logexpect l1 -v v1 {
+	expect * * 	ReqStart	"1.2.3.4 1234"
+	expect * * 	ReqStart	"1:f::2 1234"
+} -start
+
 # Finally try something which works...
 client c1 {
 	send "PROXY TCP4 1.2.3.4 5.6.7.8 1234 5678\r\n"
@@ -167,6 +172,8 @@ client c1 {
 } -run
 delay .1
 
+logexpect l1 -wait
+
 # Try with appended request (See also: #1728)
 client c1 {
 	send "PROXY TCP6 1:f::3 5:a::8 1234 5678\r\nGET /3 HTTP/1.1\r\nHdr1: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\r\n\r\n"



From lkarsten at varnish-software.com  Thu Jan 14 14:15:04 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:04 +0100
Subject: [4.1] 5a67684 White space ocd
Message-ID: <E1aJig4-00055u-60@project.varnish-software.com>


commit 5a67684db46405221c06f40d59e2c193be6df7e8
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Nov 9 08:54:16 2015 +0000

    White space ocd

diff --git a/bin/varnishtest/tests/c00007.vtc b/bin/varnishtest/tests/c00007.vtc
index 5238226..016ae78 100644
--- a/bin/varnishtest/tests/c00007.vtc
+++ b/bin/varnishtest/tests/c00007.vtc
@@ -1,5 +1,5 @@
 varnishtest "Test banning a hash"
 
-varnish v1 -arg "-b 127.0.0.1:80" -start 
+varnish v1 -arg "-b 127.0.0.1:80" -start
 
 varnish v1 -clierr 101 "ban.hash foo"
diff --git a/bin/varnishtest/tests/o00000.vtc b/bin/varnishtest/tests/o00000.vtc
index 7cebb63..ccfaa65 100644
--- a/bin/varnishtest/tests/o00000.vtc
+++ b/bin/varnishtest/tests/o00000.vtc
@@ -135,8 +135,8 @@ client c1 {
 delay .1
 
 logexpect l1 -v v1 {
-	expect * * 	ReqStart	"1.2.3.4 1234"
-	expect * * 	ReqStart	"1:f::2 1234"
+	expect * *	ReqStart	"1.2.3.4 1234"
+	expect * *	ReqStart	"1:f::2 1234"
 } -start
 
 # Finally try something which works...
diff --git a/bin/varnishtest/tests/r01810.vtc b/bin/varnishtest/tests/r01810.vtc
index 77e84db..bf6d0f1 100644
--- a/bin/varnishtest/tests/r01810.vtc
+++ b/bin/varnishtest/tests/r01810.vtc
@@ -8,7 +8,7 @@ server s1 {
 	delay .4
 } -start
 
-varnish v1 -vcl+backend { 
+varnish v1 -vcl+backend {
 	sub vcl_backend_fetch {
 		set bereq.proto = "HTTP/1.0";
 	}



From lkarsten at varnish-software.com  Thu Jan 14 14:15:04 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:04 +0100
Subject: [4.1] 2dc7d09 Return 500 if we cannot decode the stored object into
 the resp.* This can happen in a number of obscure corner-cases,
 which do not warrant a panic.
Message-ID: <E1aJig4-000565-9N@project.varnish-software.com>


commit 2dc7d093e3533008c333db9717bc0eac8050a005
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Nov 9 09:45:13 2015 +0000

    Return 500 if we cannot decode the stored object into the resp.*
    This can happen in a number of obscure corner-cases, which do not
    warrant a panic.
    
    Fixes:	#1807

diff --git a/bin/varnishd/cache/cache_http.c b/bin/varnishd/cache/cache_http.c
index 9080641..8e3fe0f 100644
--- a/bin/varnishd/cache/cache_http.c
+++ b/bin/varnishd/cache/cache_http.c
@@ -863,26 +863,30 @@ HTTP_Decode(struct http *to, const uint8_t *fm)
 {
 
 	CHECK_OBJ_NOTNULL(to, HTTP_MAGIC);
+	AN(to->vsl);
 	AN(fm);
-	if (vbe16dec(fm) > to->shd)
-		return(-1);
-	to->status = vbe16dec(fm + 2);
-	fm += 4;
-	for (to->nhd = 0; to->nhd < to->shd; to->nhd++) {
-		if (to->nhd == HTTP_HDR_METHOD || to->nhd == HTTP_HDR_URL) {
-			to->hd[to->nhd].b = NULL;
-			to->hd[to->nhd].e = NULL;
-			continue;
-		}
-		if (*fm == '\0')
-			return (0);
-		to->hd[to->nhd].b = (const void*)fm;
-		fm = (const void*)strchr((const void*)fm, '\0');
-		to->hd[to->nhd].e = (const void*)fm;
-		fm++;
-		if (to->vsl != NULL)
+	if (vbe16dec(fm) <= to->shd) {
+		to->status = vbe16dec(fm + 2);
+		fm += 4;
+		for (to->nhd = 0; to->nhd < to->shd; to->nhd++) {
+			if (to->nhd == HTTP_HDR_METHOD ||
+			    to->nhd == HTTP_HDR_URL) {
+				to->hd[to->nhd].b = NULL;
+				to->hd[to->nhd].e = NULL;
+				continue;
+			}
+			if (*fm == '\0')
+				return (0);
+			to->hd[to->nhd].b = (const void*)fm;
+			fm = (const void*)strchr((const void*)fm, '\0');
+			to->hd[to->nhd].e = (const void*)fm;
+			fm++;
 			http_VSLH(to, to->nhd);
+		}
 	}
+	VSLb(to->vsl, SLT_Error,
+	    "Too many headers to Decode object (%u vs. %u)",
+	    vbe16dec(fm), to->shd);
 	return (-1);
 }
 
diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index b7afe84..da59078 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -129,8 +129,12 @@ cnt_deliver(struct worker *wrk, struct req *req)
 		EXP_Touch(req->objcore, req->t_prev);
 
 	HTTP_Setup(req->resp, req->ws, req->vsl, SLT_RespMethod);
-	AZ(HTTP_Decode(req->resp,
-	    ObjGetattr(req->wrk, req->objcore, OA_HEADERS, NULL)));
+	if (HTTP_Decode(req->resp,
+	    ObjGetattr(req->wrk, req->objcore, OA_HEADERS, NULL))) {
+		req->err_code = 500;
+		req->req_step = R_STP_SYNTH;
+		return (REQ_FSM_MORE);
+	}
 	http_ForceField(req->resp, HTTP_HDR_PROTO, "HTTP/1.1");
 
 	if (req->is_hit)
@@ -493,9 +497,9 @@ cnt_miss(struct worker *wrk, struct req *req)
 	case VCL_RET_FETCH:
 		wrk->stats->cache_miss++;
 		VBF_Fetch(wrk, req, req->objcore, req->stale_oc, VBF_NORMAL);
-		req->req_step = R_STP_FETCH;
 		if (req->stale_oc != NULL)
 			(void)HSH_DerefObjCore(wrk, &req->stale_oc);
+		req->req_step = R_STP_FETCH;
 		return (REQ_FSM_MORE);
 	case VCL_RET_SYNTH:
 		req->req_step = R_STP_SYNTH;
diff --git a/bin/varnishtest/tests/r01807.vtc b/bin/varnishtest/tests/r01807.vtc
new file mode 100644
index 0000000..4696cd1
--- /dev/null
+++ b/bin/varnishtest/tests/r01807.vtc
@@ -0,0 +1,50 @@
+varnishtest "Decreasing http_max_hdr"
+
+server s1 {
+	rxreq
+	txresp \
+		-hdr "h00: 00" \
+		-hdr "h01: 01" \
+		-hdr "h02: 02" \
+		-hdr "h03: 03" \
+		-hdr "h04: 04" \
+		-hdr "h05: 05" \
+		-hdr "h06: 06" \
+		-hdr "h07: 07" \
+		-hdr "h08: 08" \
+		-hdr "h09: 09" \
+		-hdr "h10: 10" \
+		-hdr "h11: 11" \
+		-hdr "h12: 12" \
+		-hdr "h13: 13" \
+		-hdr "h14: 14" \
+		-hdr "h15: 15" \
+		-hdr "h16: 16" \
+		-hdr "h17: 17" \
+		-hdr "h18: 18" \
+		-hdr "h19: 19" \
+		-hdr "h20: 20" \
+		-hdr "h21: 21" \
+		-hdr "h22: 22" \
+		-hdr "h23: 23" \
+		-hdr "h24: 24"
+} -start
+
+varnish v1 -vcl+backend {
+} -start
+
+client c1 {
+	txreq
+	rxresp
+	expect resp.status == 200
+	expect resp.http.h24 == 24
+} -run
+
+varnish v1 -cliok {param.set http_max_hdr 32}
+
+client c1 {
+	txreq
+	rxresp
+	expect resp.status == 500
+} -run
+



From lkarsten at varnish-software.com  Thu Jan 14 14:15:04 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:04 +0100
Subject: [4.1] 47a724b Add copyright notice and date where missing
Message-ID: <E1aJig4-00056M-Cs@project.varnish-software.com>


commit 47a724bb1748a265ad6a86be4d81b2c94f34bad9
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Mon Nov 9 10:31:28 2015 +0000

    Add copyright notice and date where missing
    
    Also sort see also correctly.
    
    Fix #1795

diff --git a/doc/sphinx/reference/varnish-cli.rst b/doc/sphinx/reference/varnish-cli.rst
index 934ff1e..c55ecf8 100644
--- a/doc/sphinx/reference/varnish-cli.rst
+++ b/doc/sphinx/reference/varnish-cli.rst
@@ -308,8 +308,8 @@ the backend contains "USERID=1663"::
 SEE ALSO
 ========
 
-* :ref:`varnishd(1)`
 * :ref:`varnishadm(1)`
+* :ref:`varnishd(1)`
 * :ref:`vcl(7)`
 
 HISTORY
diff --git a/doc/sphinx/reference/varnishd.rst b/doc/sphinx/reference/varnishd.rst
index f76f031..3372939 100644
--- a/doc/sphinx/reference/varnishd.rst
+++ b/doc/sphinx/reference/varnishd.rst
@@ -350,12 +350,12 @@ The `varnishd` master process may also OR its exit code
 SEE ALSO
 ========
 
-* :ref:`varnish-cli(7)`
 * :ref:`varnishlog(1)`
 * :ref:`varnishhist(1)`
 * :ref:`varnishncsa(1)`
 * :ref:`varnishstat(1)`
 * :ref:`varnishtop(1)`
+* :ref:`varnish-cli(7)`
 * :ref:`vcl(7)`
 
 HISTORY
diff --git a/doc/sphinx/reference/vsl-query.rst b/doc/sphinx/reference/vsl-query.rst
index 2c15453..e757285 100644
--- a/doc/sphinx/reference/vsl-query.rst
+++ b/doc/sphinx/reference/vsl-query.rst
@@ -278,3 +278,12 @@ HISTORY
 
 This document was written by Martin Blix Grydeland.
 
+
+COPYRIGHT
+=========
+
+This document is licensed under the same licence as Varnish
+itself. See LICENCE for details.
+
+* Copyright (c) 2006 Verdens Gang AS
+* Copyright (c) 2006-2015 Varnish Software AS
diff --git a/doc/sphinx/reference/vsl.rst b/doc/sphinx/reference/vsl.rst
index 0e753b0..1aafd28 100644
--- a/doc/sphinx/reference/vsl.rst
+++ b/doc/sphinx/reference/vsl.rst
@@ -112,7 +112,16 @@ Martin Blix Grydeland.
 SEE ALSO
 ========
 
-* :ref:`varnishlog(1)`
 * :ref:`varnishhist(1)`
+* :ref:`varnishlog(1)`
 * :ref:`varnishncsa(1)`
 * :ref:`varnishtop(1)`
+
+COPYRIGHT
+=========
+
+This document is licensed under the same licence as Varnish
+itself. See LICENCE for details.
+
+* Copyright (c) 2006 Verdens Gang AS
+* Copyright (c) 2006-2015 Varnish Software AS
diff --git a/lib/libvmod_directors/vmod.vcc b/lib/libvmod_directors/vmod.vcc
index 02b5a3f..8522920 100644
--- a/lib/libvmod_directors/vmod.vcc
+++ b/lib/libvmod_directors/vmod.vcc
@@ -132,9 +132,9 @@ Description
 	to this director.
 
 Example
+	# 2/3 to backend1, 1/3 to backend2.
 	vdir.add_backend(backend1, 10.0);
 	vdir.add_backend(backend2, 5.0);
-	# 2/3 to backend1, 1/3 to backend2.
 
 
 $Method BACKEND .backend()
@@ -180,3 +180,11 @@ Description
 Example
 	# pick a backend based on the cookie header from the client
 	set req.backend_hint = vdir.backend(req.http.cookie);
+
+COPYRIGHT
+=========
+
+This document is licensed under the same licence as Varnish
+itself. See LICENCE for details.
+
+* Copyright (c) 2013-2015 Varnish Software AS
diff --git a/lib/libvmod_std/vmod.vcc b/lib/libvmod_std/vmod.vcc
index 2e20825..b0dba07 100644
--- a/lib/libvmod_std/vmod.vcc
+++ b/lib/libvmod_std/vmod.vcc
@@ -272,8 +272,8 @@ Example
 SEE ALSO
 ========
 
-* :ref:`vsl(7)`
 * :ref:`varnishd(1)`
+* :ref:`vsl(7)`
 
 HISTORY
 =======
@@ -287,3 +287,5 @@ COPYRIGHT
 
 This document is licensed under the same licence as Varnish
 itself. See LICENCE for details.
+
+* Copyright (c) 2010-2015 Varnish Software AS
diff --git a/man/vsc2rst.c b/man/vsc2rst.c
index d9ad471..93950f0 100644
--- a/man/vsc2rst.c
+++ b/man/vsc2rst.c
@@ -118,5 +118,13 @@ int main(int argc, char **argv)
 #include "tbl/vsc_fields.h"
 #undef VSC_DO_LCK
 
+	P("");
+	P("COPYRIGHT");
+	P("=========");
+	P("This document is licensed under the same licence as Varnish");
+	P("itself. See LICENCE for details.");
+	P("");
+	P("* Copyright (c) 2006 Verdens Gang AS");
+	P("* Copyright (c) 2006-2015 Varnish Software AS");
 	return (0);
 }



From lkarsten at varnish-software.com  Thu Jan 14 14:15:04 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:04 +0100
Subject: [4.1] 9b0c876 Allow varnishncsa logformat to be read from a file.
 Adds supports to varnishncsa for specifying the logging format by reading the
 log format from a file using the option "-f". Since -f means prefer
 X-Forwarded-For over client.ip in Varnish 3,
 maybe we should use another option letter.
Message-ID: <E1aJig4-00056X-FU@project.varnish-software.com>


commit 9b0c876579a5b67cad85d27a79bab20b2baf2dc2
Author: Kristian Gr?nfeldt S?rensen <ksorensen at nordija.com>
Date:   Mon Dec 15 12:59:17 2014 +0100

    Allow varnishncsa logformat to be read from a file. Adds supports to
    varnishncsa for specifying the logging format by reading the log
    format from a file using the option "-f". Since -f means prefer
    X-Forwarded-For over client.ip in Varnish 3, maybe we should use
    another option letter.

diff --git a/bin/varnishncsa/varnishncsa.c b/bin/varnishncsa/varnishncsa.c
index b19b679..f9de015 100644
--- a/bin/varnishncsa/varnishncsa.c
+++ b/bin/varnishncsa/varnishncsa.c
@@ -933,6 +933,28 @@ sighup(void)
 	return (1);
 }
 
+static char *
+read_format(const char *formatfile)
+{
+        FILE *fmtfile;
+        size_t len = 0;
+        char *fmt = NULL;
+
+        fmtfile = fopen(formatfile, "r");
+        if (fmtfile == NULL)
+                VUT_Error(1, "Can't open format file (%s)", strerror(errno));
+        if (getline(&fmt, &len, fmtfile) == -1) {
+                free(fmt);
+                if (feof(fmtfile))
+                        VUT_Error(1, "Empty format file");
+                else
+                        VUT_Error(1, "Can't read format from file (%s)",
+			    strerror(errno));
+        }
+        fclose(fmtfile);
+        return (fmt);
+}
+
 int
 main(int argc, char * const *argv)
 {
@@ -962,6 +984,13 @@ main(int argc, char * const *argv)
 			format = strdup(optarg);
 			AN(format);
 			break;
+		case 'f':
+			/* Format string from file */
+			if (format != NULL)
+				free(format);
+			format = read_format(optarg);
+			AN(format);
+			break;
 		case 'h':
 			/* Usage help */
 			usage(0);
diff --git a/bin/varnishncsa/varnishncsa_options.h b/bin/varnishncsa/varnishncsa_options.h
index 291a1c7..24567c2 100644
--- a/bin/varnishncsa/varnishncsa_options.h
+++ b/bin/varnishncsa/varnishncsa_options.h
@@ -40,6 +40,14 @@
 	    "Set the output log format string."				\
 	)
 
+#define NCSA_OPT_f                                                      \
+        VOPT("f:", "[-f formatfile]", "Read output format from file",   \
+            "Read output format from a file. Will read a single line"   \
+            " from the specified file, and use that line as the"	\
+	    " format."							\
+	)
+
+
 #define NCSA_OPT_g							\
 	VOPT("g:", "[-g <request|vxid>]", "Grouping mode (default: vxid)", \
 	    "The grouping of the log records. The default is to group"	\
@@ -60,6 +68,7 @@ VSL_OPT_C
 VUT_OPT_d
 VUT_OPT_D
 NCSA_OPT_F
+NCSA_OPT_f
 NCSA_OPT_g
 VUT_OPT_h
 VUT_OPT_n



From lkarsten at varnish-software.com  Thu Jan 14 14:15:04 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:04 +0100
Subject: [4.1] 2e7b89e Modify the ban_lurker so we lift oc's as far up the
 ban-list as we can when we check a batch of lurkable bans.
Message-ID: <E1aJig4-00056j-HW@project.varnish-software.com>


commit 2e7b89e0ea1273f881019200cf4e1748ba3a093a
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Nov 10 10:52:01 2015 +0000

    Modify the ban_lurker so we lift oc's as far up the ban-list as we
    can when we check a batch of lurkable bans.
    
    For obj.* only bans, this should concentrate the entire list at the
    very top.
    
    If there are req.* bans, the oc's end up on the bans right below
    the req.* bans.
    
    Fixes	#1635
    
    (As much as we can fix it)

diff --git a/bin/varnishd/cache/cache_ban.c b/bin/varnishd/cache/cache_ban.c
index 3bf9073..c8971bc 100644
--- a/bin/varnishd/cache/cache_ban.c
+++ b/bin/varnishd/cache/cache_ban.c
@@ -159,14 +159,16 @@ ban_mark_completed(struct ban *b)
 	Lck_AssertHeld(&ban_mtx);
 
 	AN(b->spec);
-	AZ(b->flags & BANS_FLAG_COMPLETED);
-	ln = ban_len(b->spec);
-	b->flags |= BANS_FLAG_COMPLETED;
-	b->spec[BANS_FLAGS] |= BANS_FLAG_COMPLETED;
-	VWMB();
-	vbe32enc(b->spec + BANS_LENGTH, BANS_HEAD_LEN);
-	VSC_C_main->bans_completed++;
-	VSC_C_main->bans_persisted_fragmentation += ln - ban_len(b->spec);
+	if (!(b->flags & BANS_FLAG_COMPLETED)) {
+		ln = ban_len(b->spec);
+		b->flags |= BANS_FLAG_COMPLETED;
+		b->spec[BANS_FLAGS] |= BANS_FLAG_COMPLETED;
+		VWMB();
+		vbe32enc(b->spec + BANS_LENGTH, BANS_HEAD_LEN);
+		VSC_C_main->bans_completed++;
+		VSC_C_main->bans_persisted_fragmentation +=
+		    ln - ban_len(b->spec);
+	}
 }
 
 /*--------------------------------------------------------------------
@@ -683,7 +685,7 @@ ccf_ban_list(struct cli *cli, const char * const *av, void *priv)
 		o = bl == b ? 1 : 0;
 		VCLI_Out(cli, "%10.6f %5ju %s", ban_time(b->spec),
 		    (intmax_t)b->refcount - o,
-		    b->flags & BANS_FLAG_COMPLETED ? "C" : " ");
+		    b->flags & BANS_FLAG_COMPLETED ? "C" : "-");
 		if (DO_DEBUG(DBG_LURKER)) {
 			VCLI_Out(cli, "%s%s %p ",
 			    b->flags & BANS_FLAG_REQ ? "R" : "-",
diff --git a/bin/varnishd/cache/cache_ban_lurker.c b/bin/varnishd/cache/cache_ban_lurker.c
index 352b219..63371f8 100644
--- a/bin/varnishd/cache/cache_ban_lurker.c
+++ b/bin/varnishd/cache/cache_ban_lurker.c
@@ -112,7 +112,9 @@ ban_lurker_getfirst(struct vsl_log *vsl, struct ban *bt)
 			} else {
 				/*
 				 * We got the lock, and the oc is not being
-				 * dismantled under our feet, run with it...
+				 * dismantled under our feet.
+				 * Take it off the ban and (optimistically)
+				 * put it on the * destination ban
 				 */
 				AZ(oc->flags & OC_F_BUSY);
 				oc->refcnt += 1;
@@ -135,7 +137,7 @@ ban_lurker_getfirst(struct vsl_log *vsl, struct ban *bt)
 
 static void
 ban_lurker_test_ban(struct worker *wrk, struct vsl_log *vsl, struct ban *bt,
-    struct banhead_s *obans)
+    struct banhead_s *obans, struct ban *bd)
 {
 	struct ban *bl, *bln;
 	struct objcore *oc;
@@ -182,6 +184,17 @@ ban_lurker_test_ban(struct worker *wrk, struct vsl_log *vsl, struct ban *bt,
 			EXP_Rearm(oc, oc->exp.t_origin, 0, 0, 0);
 					// XXX ^ fake now
 			VSC_C_main->bans_lurker_obj_killed++;
+		} else {
+			if (oc->ban != bd) {
+				Lck_Lock(&ban_mtx);
+				oc->ban->refcount--;
+				VTAILQ_REMOVE(&oc->ban->objcore, oc, ban_list);
+				oc->ban = bd;
+				bd->refcount++;
+				VTAILQ_INSERT_TAIL(&bd->objcore, oc, ban_list);
+				Lck_Unlock(&ban_mtx);
+				ObjUpdateMeta(wrk, oc);
+			}
 		}
 		(void)HSH_DerefObjCore(wrk, &oc);
 	}
@@ -194,63 +207,43 @@ ban_lurker_test_ban(struct worker *wrk, struct vsl_log *vsl, struct ban *bt,
 static double
 ban_lurker_work(struct worker *wrk, struct vsl_log *vsl)
 {
-	struct ban *b, *bt;
+	struct ban *b, *bd;
 	struct banhead_s obans;
 	double d, dt, n;
-	int i;
 
 	dt = 49.62;		// Random, non-magic
 	if (cache_param->ban_lurker_sleep == 0)
 		return (dt);
-	/* Make a list of the bans we can do something about */
-	VTAILQ_INIT(&obans);
+
 	Lck_Lock(&ban_mtx);
 	b = ban_start;
 	Lck_Unlock(&ban_mtx);
-	i = 0;
 	d = VTIM_real() - cache_param->ban_lurker_age;
+	bd = NULL;
+	VTAILQ_INIT(&obans);
 	for (; b != NULL; b = VTAILQ_NEXT(b, list)) {
+		if (bd != NULL)
+			ban_lurker_test_ban(wrk, vsl, b, &obans, bd);
 		if (b->flags & BANS_FLAG_COMPLETED)
 			continue;
-		if (b->flags & BANS_FLAG_REQ)
+		if (b->flags & BANS_FLAG_REQ) {
+			bd = VTAILQ_NEXT(b, list);
 			continue;
-		if (b == VTAILQ_LAST(&ban_head, banhead_s))
-			continue;	// XXX: why ?
+		}
 		n = ban_time(b->spec) - d;
 		if (n < 0) {
 			VTAILQ_INSERT_TAIL(&obans, b, l_list);
-			i++;
+			if (bd == NULL)
+				bd = b;
 		} else if (n < dt) {
 			dt = n;
 		}
 	}
-	if (DO_DEBUG(DBG_LURKER))
-		VSLb(vsl, SLT_Debug, "lurker: %d actionable bans, dt = %lf", i, dt);
-	if (i == 0)
-		return (dt);
 
-	dt = cache_param->ban_lurker_sleep;
-	/* Go though all the bans to test the objects */
-	VTAILQ_FOREACH_REVERSE(bt, &ban_head, banhead_s, list) {
-		if (bt == VTAILQ_LAST(&obans, banhead_s)) {
-			if (DO_DEBUG(DBG_LURKER))
-				VSLb(vsl, SLT_Debug,
-				    "Lurk bt completed %p", bt);
-			Lck_Lock(&ban_mtx);
-			/* We can be raced by a new ban */
-			if (!(bt->flags & BANS_FLAG_COMPLETED))
-				ban_mark_completed(bt);
-			Lck_Unlock(&ban_mtx);
-			VTAILQ_REMOVE(&obans, bt, l_list);
-			if (VTAILQ_EMPTY(&obans))
-				break;
-		}
-		if (DO_DEBUG(DBG_LURKER))
-			VSLb(vsl, SLT_Debug, "Lurk bt %p", bt);
-		ban_lurker_test_ban(wrk, vsl, bt, &obans);
-		if (VTAILQ_EMPTY(&obans))
-			break;
-	}
+	Lck_Lock(&ban_mtx);
+	VTAILQ_FOREACH(b, &obans, l_list)
+		ban_mark_completed(b);
+	Lck_Unlock(&ban_mtx);
 	return (dt);
 }
 
@@ -268,9 +261,9 @@ ban_lurker(struct worker *wrk, void *priv)
 
 	while (!ban_shutdown) {
 		d = ban_lurker_work(wrk, &vsl);
+		ban_cleantail();
 		if (DO_DEBUG(DBG_LURKER))
 			VSLb(&vsl, SLT_Debug, "lurker: sleep = %lf", d);
-		ban_cleantail();
 		d += VTIM_real();
 		Lck_Lock(&ban_mtx);
 		if (gen == ban_generation) {
diff --git a/bin/varnishtest/tests/c00049.vtc b/bin/varnishtest/tests/c00049.vtc
index 8d10e68..e86c513 100644
--- a/bin/varnishtest/tests/c00049.vtc
+++ b/bin/varnishtest/tests/c00049.vtc
@@ -94,7 +94,7 @@ client c1 {
 } -run
 
 # Get the VSL out of the way
-delay .1
+delay 1
 
 varnish v1 -cliok "ban.list"
 
@@ -115,18 +115,20 @@ varnish v1 -expect bans_dups == 0
 
 varnish v1 -cliok "param.set ban_lurker_sleep .01"
 
+delay 1
+
 varnish v1 -cliok "ban.list"
 
-delay 2
+delay 1
 
 varnish v1 -cliok "ban.list"
 
-varnish v1 -expect bans == 5
-varnish v1 -expect bans_completed == 4
+varnish v1 -expect bans == 4
+varnish v1 -expect bans_completed == 3
 varnish v1 -expect bans_req == 1
-varnish v1 -expect bans_obj == 4
+varnish v1 -expect bans_obj == 3
 varnish v1 -expect bans_added == 6
-varnish v1 -expect bans_deleted == 1
+varnish v1 -expect bans_deleted == 2
 varnish v1 -expect bans_tested == 0
 varnish v1 -expect bans_tests_tested == 0
 varnish v1 -expect bans_obj_killed == 0



From lkarsten at varnish-software.com  Thu Jan 14 14:15:04 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:04 +0100
Subject: [4.1] ecc1dec White space OCD
Message-ID: <E1aJig4-000570-Jr@project.varnish-software.com>


commit ecc1dec08bcf5e6e425391b24d507f9056fb58be
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Nov 11 10:04:23 2015 +0000

    White space OCD

diff --git a/bin/varnishncsa/varnishncsa.c b/bin/varnishncsa/varnishncsa.c
index f9de015..d46bee8 100644
--- a/bin/varnishncsa/varnishncsa.c
+++ b/bin/varnishncsa/varnishncsa.c
@@ -936,23 +936,23 @@ sighup(void)
 static char *
 read_format(const char *formatfile)
 {
-        FILE *fmtfile;
-        size_t len = 0;
-        char *fmt = NULL;
-
-        fmtfile = fopen(formatfile, "r");
-        if (fmtfile == NULL)
-                VUT_Error(1, "Can't open format file (%s)", strerror(errno));
-        if (getline(&fmt, &len, fmtfile) == -1) {
-                free(fmt);
-                if (feof(fmtfile))
-                        VUT_Error(1, "Empty format file");
-                else
-                        VUT_Error(1, "Can't read format from file (%s)",
+	FILE *fmtfile;
+	size_t len = 0;
+	char *fmt = NULL;
+
+	fmtfile = fopen(formatfile, "r");
+	if (fmtfile == NULL)
+		VUT_Error(1, "Can't open format file (%s)", strerror(errno));
+	if (getline(&fmt, &len, fmtfile) == -1) {
+		free(fmt);
+		if (feof(fmtfile))
+			VUT_Error(1, "Empty format file");
+		else
+			VUT_Error(1, "Can't read format from file (%s)",
 			    strerror(errno));
-        }
-        fclose(fmtfile);
-        return (fmt);
+	}
+	fclose(fmtfile);
+	return (fmt);
 }
 
 int
diff --git a/bin/varnishncsa/varnishncsa_options.h b/bin/varnishncsa/varnishncsa_options.h
index 24567c2..f141ad0 100644
--- a/bin/varnishncsa/varnishncsa_options.h
+++ b/bin/varnishncsa/varnishncsa_options.h
@@ -40,10 +40,10 @@
 	    "Set the output log format string."				\
 	)
 
-#define NCSA_OPT_f                                                      \
-        VOPT("f:", "[-f formatfile]", "Read output format from file",   \
-            "Read output format from a file. Will read a single line"   \
-            " from the specified file, and use that line as the"	\
+#define NCSA_OPT_f							\
+	VOPT("f:", "[-f formatfile]", "Read output format from file",   \
+	    "Read output format from a file. Will read a single line"   \
+	    " from the specified file, and use that line as the"	\
 	    " format."							\
 	)
 



From lkarsten at varnish-software.com  Thu Jan 14 14:15:04 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:04 +0100
Subject: [4.1] 420794c Make test r01576 depend on PCRE JIT being enabled.
Message-ID: <E1aJig4-00057B-Lr@project.varnish-software.com>


commit 420794cbf1a117d118da3cf178f00a6e839d6c6f
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Nov 11 10:04:32 2015 +0000

    Make test r01576 depend on PCRE JIT being enabled.

diff --git a/bin/varnishtest/tests/r01576.vtc b/bin/varnishtest/tests/r01576.vtc
index 99398e3..b0fdb9f 100644
--- a/bin/varnishtest/tests/r01576.vtc
+++ b/bin/varnishtest/tests/r01576.vtc
@@ -1,5 +1,7 @@
 varnishtest "Test recursive regexp's fail before consuming all the stack"
 
+feature pcre_jit
+
 server s1 {
 	rxreq
 	expect req.http.found == "1"
diff --git a/bin/varnishtest/vtc.c b/bin/varnishtest/vtc.c
index f77f617..3c6c884 100644
--- a/bin/varnishtest/vtc.c
+++ b/bin/varnishtest/vtc.c
@@ -47,6 +47,7 @@
 
 #include "vav.h"
 #include "vnum.h"
+#include "vre.h"
 #include "vtim.h"
 
 #define		MAX_TOKENS		200
@@ -578,6 +579,9 @@ cmd_feature(CMD_ARGS)
 		if (sizeof(void*) == 8 && !strcmp(av[i], "64bit"))
 			continue;
 
+		if (!strcmp(av[i], "pcre_jit") && VRE_has_jit)
+			continue;
+
 		if (!strcmp(av[i], "!OSX")) {
 #if !defined(__APPLE__) || !defined(__MACH__)
 			continue;
diff --git a/include/vre.h b/include/vre.h
index a59e8d7..f19ff2c 100644
--- a/include/vre.h
+++ b/include/vre.h
@@ -48,6 +48,7 @@ typedef struct vre vre_t;
 #define VRE_ERROR_NOMATCH         (-1)
 
 /* And those to PCRE options */
+extern const unsigned VRE_has_jit;
 extern const unsigned VRE_CASELESS;
 extern const unsigned VRE_NOTEMPTY;
 
diff --git a/lib/libvarnish/vre.c b/lib/libvarnish/vre.c
index 94400fa..0bf89f9 100644
--- a/lib/libvarnish/vre.c
+++ b/lib/libvarnish/vre.c
@@ -42,6 +42,8 @@
 #define VRE_STUDY_JIT_COMPILE 0
 #endif
 
+const unsigned VRE_has_jit = VRE_STUDY_JIT_COMPILE;
+
 #if PCRE_MAJOR < 8 || (PCRE_MAJOR == 8 && PCRE_MINOR < 20)
 #  define pcre_free_study pcre_free
 #endif



From lkarsten at varnish-software.com  Thu Jan 14 14:15:04 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:04 +0100
Subject: [4.1] d78f2c0 printf format fix
Message-ID: <E1aJig4-00057O-P9@project.varnish-software.com>


commit d78f2c0028c0dd551df60598ffa581fced205ed7
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Nov 11 10:08:48 2015 +0000

    printf format fix

diff --git a/bin/varnishd/cache/cache_ban.c b/bin/varnishd/cache/cache_ban.c
index c8971bc..3042653 100644
--- a/bin/varnishd/cache/cache_ban.c
+++ b/bin/varnishd/cache/cache_ban.c
@@ -684,7 +684,7 @@ ccf_ban_list(struct cli *cli, const char * const *av, void *priv)
 	VTAILQ_FOREACH(b, &ban_head, list) {
 		o = bl == b ? 1 : 0;
 		VCLI_Out(cli, "%10.6f %5ju %s", ban_time(b->spec),
-		    (intmax_t)b->refcount - o,
+		    (intmax_t)(b->refcount - o),
 		    b->flags & BANS_FLAG_COMPLETED ? "C" : "-");
 		if (DO_DEBUG(DBG_LURKER)) {
 			VCLI_Out(cli, "%s%s %p ",



From lkarsten at varnish-software.com  Thu Jan 14 14:15:04 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:04 +0100
Subject: [4.1] 8f7313e init the condvar before we signal it
Message-ID: <E1aJig4-00057c-Sf@project.varnish-software.com>


commit 8f7313e5a433e87c1bef8893581a65deef15a6f2
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Nov 11 10:36:06 2015 +0000

    init the condvar before we signal it

diff --git a/bin/varnishd/cache/cache_ban.c b/bin/varnishd/cache/cache_ban.c
index 3042653..565b5d1 100644
--- a/bin/varnishd/cache/cache_ban.c
+++ b/bin/varnishd/cache/cache_ban.c
@@ -761,8 +761,8 @@ BAN_Init(void)
 	/* Add a placeholder ban */
 	bp = BAN_Build();
 	AN(bp);
-	AZ(BAN_Commit(bp));
 	AZ(pthread_cond_init(&ban_lurker_cond, NULL));
+	AZ(BAN_Commit(bp));
 	Lck_Lock(&ban_mtx);
 	ban_mark_completed(VTAILQ_FIRST(&ban_head));
 	Lck_Unlock(&ban_mtx);



From lkarsten at varnish-software.com  Thu Jan 14 14:15:04 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:04 +0100
Subject: [4.1] a5fb550 Disable test r01576,
 it takes too much time and effort to keep it working across platforms.
Message-ID: <E1aJig4-00057m-VQ@project.varnish-software.com>


commit a5fb550836eb68b33ec6dd25a4fdca05dabb704e
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Nov 11 10:50:50 2015 +0000

    Disable test r01576, it takes too much time and effort to keep
    it working across platforms.

diff --git a/bin/varnishtest/tests.disabled/r01576.vtc b/bin/varnishtest/tests.disabled/r01576.vtc
new file mode 100644
index 0000000..b0fdb9f
--- /dev/null
+++ b/bin/varnishtest/tests.disabled/r01576.vtc
@@ -0,0 +1,51 @@
+varnishtest "Test recursive regexp's fail before consuming all the stack"
+
+feature pcre_jit
+
+server s1 {
+	rxreq
+	expect req.http.found == "1"
+	txresp
+	rxreq
+	expect req.http.found == <undef>
+	txresp
+} -start
+
+# If you want to play around, uncomment the next lines and adjust
+# the length of the aaaaaaaaaaa strings below to suit your needs.
+# Better yet:  Rewrite your regexps to avoid this madness.
+
+# varnish v1 -arg "-p thread_pool_stack=48k"
+# varnish v1 -arg "-p pcre_match_limit=1000"
+# varnish v1 -arg "-p pcre_match_limit_recursion=89"
+
+# Approximate formua for FreeBSD/amd64:
+#	pcre_match_limit_recursion = thread_pool_stack * 2 - 9
+
+varnish v1 -vcl+backend {
+	sub vcl_recv {
+		if (req.url ~ "^/a((?!/.).)*$") {
+			set req.http.found = "1";
+		}
+	}
+} -start
+
+# This should succeed with default params and JIT/no-JIT
+client c1 {
+	txreq -url /aaaaaaaa
+	rxresp
+} -run
+
+# PCRE_ERROR_RECURSIONLIMIT  (-21)
+# PCRE_ERROR_JIT_STACKLIMIT  (-27)
+logexpect l1 -v v1 {
+	expect * * VCL_Error "Regexp matching returned -2[71]"
+} -start
+
+# This should fail with default params and JIT/no-JIT
+client c1 {
+	txreq -url /aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
+	rxresp
+} -run
+
+logexpect l1 -wait
diff --git a/bin/varnishtest/tests/r01576.vtc b/bin/varnishtest/tests/r01576.vtc
deleted file mode 100644
index b0fdb9f..0000000
--- a/bin/varnishtest/tests/r01576.vtc
+++ /dev/null
@@ -1,51 +0,0 @@
-varnishtest "Test recursive regexp's fail before consuming all the stack"
-
-feature pcre_jit
-
-server s1 {
-	rxreq
-	expect req.http.found == "1"
-	txresp
-	rxreq
-	expect req.http.found == <undef>
-	txresp
-} -start
-
-# If you want to play around, uncomment the next lines and adjust
-# the length of the aaaaaaaaaaa strings below to suit your needs.
-# Better yet:  Rewrite your regexps to avoid this madness.
-
-# varnish v1 -arg "-p thread_pool_stack=48k"
-# varnish v1 -arg "-p pcre_match_limit=1000"
-# varnish v1 -arg "-p pcre_match_limit_recursion=89"
-
-# Approximate formua for FreeBSD/amd64:
-#	pcre_match_limit_recursion = thread_pool_stack * 2 - 9
-
-varnish v1 -vcl+backend {
-	sub vcl_recv {
-		if (req.url ~ "^/a((?!/.).)*$") {
-			set req.http.found = "1";
-		}
-	}
-} -start
-
-# This should succeed with default params and JIT/no-JIT
-client c1 {
-	txreq -url /aaaaaaaa
-	rxresp
-} -run
-
-# PCRE_ERROR_RECURSIONLIMIT  (-21)
-# PCRE_ERROR_JIT_STACKLIMIT  (-27)
-logexpect l1 -v v1 {
-	expect * * VCL_Error "Regexp matching returned -2[71]"
-} -start
-
-# This should fail with default params and JIT/no-JIT
-client c1 {
-	txreq -url /aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
-	rxresp
-} -run
-
-logexpect l1 -wait



From lkarsten at varnish-software.com  Thu Jan 14 14:15:05 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:05 +0100
Subject: [4.1] 4bdb3de Move the pcre jit check to configure
Message-ID: <E1aJig5-00057x-1Q@project.varnish-software.com>


commit 4bdb3de3d9fd8548ca7a5e16b73d4dd07cdf1cb4
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Wed Nov 11 13:33:07 2015 +0000

    Move the pcre jit check to configure
    
    Output whether pcre jit is usable at configure time.

diff --git a/configure.ac b/configure.ac
index faee354..4357c3c 100644
--- a/configure.ac
+++ b/configure.ac
@@ -151,6 +151,27 @@ fi
 AC_SUBST(PCRE_CFLAGS)
 AC_SUBST(PCRE_LIBS)
 
+# --enable-pcre-jit
+AC_ARG_ENABLE(pcre-jit,
+  AS_HELP_STRING([--enable-pcre-jit],
+    [use the PCRE JIT compiler (default is YES)]),
+    [],
+    [enable_pcre_jit=yes])
+if test "$enable_pcre_jit" = yes; then
+  AC_MSG_CHECKING(for PCRE JIT usability)
+  AC_PREPROC_IFELSE(
+    [AC_LANG_PROGRAM([[#include "pcre.h"
+                       #if PCRE_MAJOR != 8 || PCRE_MINOR < 32
+                       #error no jit
+                       #endif]])],
+    [AC_MSG_RESULT(yes)
+     AC_DEFINE([USE_PCRE_JIT], [1], [Use the PCRE JIT compiler])
+    ],
+    [AC_MSG_RESULT(no)]
+  )
+fi
+
+
 AC_CHECK_HEADERS([edit/readline/readline.h],
 	[AC_DEFINE([HAVE_LIBEDIT], [1], [Define if we have libedit])
 	LIBEDIT_LIBS="-ledit"],
@@ -611,17 +632,6 @@ fi
 
 AC_DEFINE_UNQUOTED([VCC_CC],"$VCC_CC",[C compiler command line for VCL code])
 
-# --enable-pcre-jit
-AC_ARG_ENABLE(pcre-jit,
-    AS_HELP_STRING([--enable-pcre-jit],
-	[use the PCRE JIT compiler (default is YES)]),
-    ,
-    [enable_pcre_jit=yes])
-
-if test "$enable_pcre_jit" = yes; then
-   AC_DEFINE([USE_PCRE_JIT],[1],[use the PCRE JIT compiler])
-fi
-
 # Stupid automake needs this
 VTC_TESTS="$(cd $srcdir/bin/varnishtest && echo tests/*.vtc)"
 AC_SUBST(VTC_TESTS)
diff --git a/lib/libvarnish/vre.c b/lib/libvarnish/vre.c
index 0bf89f9..56abccf 100644
--- a/lib/libvarnish/vre.c
+++ b/lib/libvarnish/vre.c
@@ -36,7 +36,7 @@
 
 #include "vre.h"
 
-#if defined(USE_PCRE_JIT) && PCRE_MAJOR == 8 && PCRE_MINOR >= 32
+#if defined(USE_PCRE_JIT)
 #define VRE_STUDY_JIT_COMPILE PCRE_STUDY_JIT_COMPILE
 #else
 #define VRE_STUDY_JIT_COMPILE 0



From lkarsten at varnish-software.com  Thu Jan 14 14:15:05 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:05 +0100
Subject: [4.1] 04c70e6 Retire varnishreplay for good
Message-ID: <E1aJig5-00058D-4w@project.varnish-software.com>


commit 04c70e66f4013e3ef7898daae590623fbcedea10
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Wed Nov 11 12:50:05 2015 +0000

    Retire varnishreplay for good

diff --git a/bin/Makefile.am b/bin/Makefile.am
index 8aa2ef9..157fbc8 100644
--- a/bin/Makefile.am
+++ b/bin/Makefile.am
@@ -2,7 +2,6 @@
 
 # Disabling building of the tools while api is in flux
 
-#SUBDIRS = varnishreplay
 SUBDIRS = \
 	varnishadm \
 	varnishd \
diff --git a/bin/Makefile.phk b/bin/Makefile.phk
index 07ed226..ce538a1 100644
--- a/bin/Makefile.phk
+++ b/bin/Makefile.phk
@@ -2,7 +2,6 @@ SUBDIRS += varnishadm
 SUBDIRS += varnishd 
 SUBDIRS += varnishlog
 SUBDIRS += varnishncsa
-# varnishreplay
 SUBDIRS += varnishtest 
 SUBDIRS += varnishstat
 SUBDIRS += varnishhist 
diff --git a/bin/varnishreplay/Makefile.am b/bin/varnishreplay/Makefile.am
deleted file mode 100644
index 38d1c30..0000000
--- a/bin/varnishreplay/Makefile.am
+++ /dev/null
@@ -1,18 +0,0 @@
-#
-
-AM_CPPFLAGS = \
-	-I$(top_srcdir)/include \
-	-I$(top_builddir)/include
-
-bin_PROGRAMS = varnishreplay
-
-varnishreplay_SOURCES = \
-	varnishreplay.c \
-	$(top_srcdir)/lib/libvarnish/vas.c \
-	$(top_srcdir)/lib/libvarnish/vtcp.c \
-	$(top_srcdir)/lib/libvarnish/vss.c
-
-varnishreplay_LDADD = \
-	$(top_builddir)/lib/libvarnishcompat/libvarnishcompat.la \
-	$(top_builddir)/lib/libvarnishapi/libvarnishapi.la \
-	${RT_LIBS} ${PTHREAD_LIBS} ${NET_LIBS} ${LIBM}
diff --git a/bin/varnishreplay/Makefile.phk b/bin/varnishreplay/Makefile.phk
deleted file mode 100644
index d69f03e..0000000
--- a/bin/varnishreplay/Makefile.phk
+++ /dev/null
@@ -1,7 +0,0 @@
-PROG_SRC = varnishreplay.c
-
-LD_ADD	+= ${LIB_VARNISH}
-LD_ADD	+= ${LIB_VARNISHAPI}
-
-TOPDIR  = $(CURDIR)/../..
-include $(TOPDIR)/Makefile.inc.phk
diff --git a/bin/varnishreplay/varnishreplay.c b/bin/varnishreplay/varnishreplay.c
deleted file mode 100644
index 12cdc0d..0000000
--- a/bin/varnishreplay/varnishreplay.c
+++ /dev/null
@@ -1,757 +0,0 @@
-/*-
- * Copyright (c) 2010 Varnish Software AS
- * All rights reserved.
- *
- * Author: Cecilie Fritzvold <cecilihf at linpro.no>
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
- * 1. Redistributions of source code must retain the above copyright
- *    notice, this list of conditions and the following disclaimer.
- * 2. Redistributions in binary form must reproduce the above copyright
- *    notice, this list of conditions and the following disclaimer in the
- *    documentation and/or other materials provided with the distribution.
- *
- * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
- * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED.  IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
- * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
- * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
- * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
- * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
- * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
- * SUCH DAMAGE.
- */
-
-#include "config.h"
-
-#include <sys/types.h>
-#include <sys/signal.h>
-#include <sys/uio.h>
-
-#include <ctype.h>
-#include <fcntl.h>
-#include <pthread.h>
-#include <signal.h>
-#include <stdarg.h>
-#include <stdint.h>
-#include <stdio.h>
-#include <stdlib.h>
-#include <string.h>
-#include <unistd.h>
-
-#include "vdef.h"
-#include "vapi/vsl.h"
-#include "vapi/vsm.h"
-#include "vas.h"
-#include "vcs.h"
-#include "vqueue.h"
-#include "vss.h"
-
-#define freez(x) do { if (x) free(x); x = NULL; } while (0);
-
-static struct vss_addr *addr_info;
-static int debug;
-
-static int
-isprefix(const char *str, const char *prefix, const char **next)
-{
-
-	while (*str && *prefix &&
-	    tolower((int)*str) == tolower((int)*prefix))
-		++str, ++prefix;
-	if (*str && *str != ' ')
-		return (0);
-	if (next) {
-		while (*str && *str == ' ')
-			++str;
-		*next = str;
-	}
-	return (1);
-}
-
-#if 0
-static int
-isequal(const char *str, const char *reference, const char *end)
-{
-
-	while (str < end && *str && *reference &&
-	    tolower((int)*str) == tolower((int)*reference))
-		++str, ++reference;
-	if (str != end || *reference)
-		return (0);
-	return (1);
-}
-#endif
-
-/*
- * mailbox toolkit
- */
-
-struct message {
-	enum VSL_tag_e tag;
-	size_t len;
-	char *ptr;
-	VSTAILQ_ENTRY(message) list;
-};
-
-#define MAX_MAILBOX_SIZE 30
-
-struct mailbox {
-	pthread_mutex_t lock;
-	pthread_cond_t has_mail;
-	int open;
-	VSTAILQ_HEAD(msgq_head, message) messages;
-};
-
-static void
-mailbox_create(struct mailbox *mbox)
-{
-
-	VSTAILQ_INIT(&mbox->messages);
-	pthread_mutex_init(&mbox->lock, NULL);
-	pthread_cond_init(&mbox->has_mail, NULL);
-	mbox->open = 1;
-}
-
-static void
-mailbox_destroy(struct mailbox *mbox)
-{
-	struct message *msg;
-
-	while ((msg = VSTAILQ_FIRST(&mbox->messages))) {
-		VSTAILQ_REMOVE_HEAD(&mbox->messages, list);
-		free(msg);
-	}
-	pthread_cond_destroy(&mbox->has_mail);
-	pthread_mutex_destroy(&mbox->lock);
-}
-
-static void
-mailbox_put(struct mailbox *mbox, struct message *msg)
-{
-
-	pthread_mutex_lock(&mbox->lock);
-	VSTAILQ_INSERT_TAIL(&mbox->messages, msg, list);
-	pthread_cond_signal(&mbox->has_mail);
-	pthread_mutex_unlock(&mbox->lock);
-}
-
-static struct message *
-mailbox_get(struct mailbox *mbox)
-{
-	struct message *msg;
-
-	pthread_mutex_lock(&mbox->lock);
-	while ((msg = VSTAILQ_FIRST(&mbox->messages)) == NULL && mbox->open)
-		pthread_cond_wait(&mbox->has_mail, &mbox->lock);
-	if (msg != NULL)
-		VSTAILQ_REMOVE_HEAD(&mbox->messages, list);
-	pthread_mutex_unlock(&mbox->lock);
-	return (msg);
-}
-
-static void
-mailbox_close(struct mailbox *mbox)
-{
-	pthread_mutex_lock(&mbox->lock);
-	mbox->open = 0;
-	pthread_cond_signal(&mbox->has_mail);
-	pthread_mutex_unlock(&mbox->lock);
-}
-
-/*
- * thread toolkit
- */
-
-static pthread_mutex_t log_mutex = PTHREAD_MUTEX_INITIALIZER;
-
-static void
-thread_log(int lvl, int errcode, const char *fmt, ...)
-    __v_printflike(3, 4);
-
-static void
-thread_log(int lvl, int errcode, const char *fmt, ...)
-{
-	va_list ap;
-
-	if (lvl > debug)
-		return;
-	pthread_mutex_lock(&log_mutex);
-	fprintf(stderr, "%p ", (void *)(uintptr_t)pthread_self());
-	va_start(ap, fmt);
-	vfprintf(stderr, fmt, ap);
-	va_end(ap);
-	if (errcode)
-		fprintf(stderr, ": %s", strerror(errcode));
-	fprintf(stderr, "\n");
-	pthread_mutex_unlock(&log_mutex);
-}
-
-struct replay_thread {
-	pthread_t thread_id;
-	struct mailbox mbox;
-
-	int sock;
-
-	int fd;			/* original fd from logs */
-
-	char *method;		/* Request method*/
-	char *proto;		/* Protocol version */
-	char *url;		/* URL and query string */
-	const char *conn;	/* Connection info (keep-alive, close) */
-	char *hdr[64];		/* Headers */
-	int nhdr;		/* Number of headers */
-	int bogus;		/* bogus request */
-
-	char arena[4096];
-	int top;
-	char line[2048];
-	char temp[2048];
-};
-
-static struct replay_thread **threads;
-static size_t nthreads;
-
-/*
- * Clear thread state
- */
-static void
-thread_clear(struct replay_thread *thr)
-{
-
-	thr->method = thr->proto = thr->url = NULL;
-	thr->conn = NULL;
-	memset(&thr->hdr, 0, sizeof thr->hdr);
-	thr->nhdr = 0;
-	thr->bogus = 0;
-	memset(&thr->arena, 0, sizeof thr->arena);
-	thr->top = 0;
-	memset(&thr->line, 0, sizeof thr->line);
-	memset(&thr->temp, 0, sizeof thr->temp);
-	if (thr->sock != -1)
-		close(thr->sock);
-	thr->sock = -1;
-}
-
-#define THREAD_FAIL ((struct replay_thread *)-1)
-
-static pthread_attr_t thread_attr;
-
-static struct replay_thread *
-thread_get(int fd, void *(*thread_main)(void *))
-{
-
-	AN(fd);
-	if (fd >= nthreads) {
-		struct replay_thread **newthreads = threads;
-		size_t newnthreads = nthreads;
-
-		while (fd >= newnthreads)
-			newnthreads += newnthreads + 1;
-		newthreads = realloc(newthreads,
-		    newnthreads * sizeof *newthreads);
-		XXXAN(newthreads != NULL);
-		memset(newthreads + nthreads, 0,
-		    (newnthreads - nthreads) * sizeof *newthreads);
-		threads = newthreads;
-		nthreads = newnthreads;
-	}
-	if (threads[fd] == NULL) {
-		threads[fd] = malloc(sizeof *threads[fd]);
-		assert(threads[fd] != NULL);
-		threads[fd]->sock = -1;
-		thread_clear(threads[fd]);
-		mailbox_create(&threads[fd]->mbox);
-		if (pthread_create(&threads[fd]->thread_id, &thread_attr,
-		    thread_main, threads[fd]) != 0) {
-			thread_log(0, errno, "pthread_create()");
-			mailbox_destroy(&threads[fd]->mbox);
-			freez(threads[fd]);
-			threads[fd] = THREAD_FAIL;
-		} else {
-			threads[fd]->fd = fd;
-			thread_log(0, 0, "thread %p:%d started",
-			    (void *)(uintptr_t)threads[fd]->thread_id, fd);
-		}
-	}
-	if (threads[fd] == THREAD_FAIL)
-		return (NULL);
-	return (threads[fd]);
-}
-
-static void
-thread_close(int fd)
-{
-
-	if (fd == -1) {
-		for (fd = 0; fd < nthreads; ++fd)
-			thread_close(fd);
-		return;
-	}
-
-	assert(fd < nthreads);
-
-	if (threads[fd] == NULL)
-		return;
-	mailbox_close(&threads[fd]->mbox);
-	pthread_join(threads[fd]->thread_id, NULL);
-	thread_log(0, 0, "thread %p stopped",
-	    (void *)(uintptr_t)threads[fd]->thread_id);
-	thread_clear(threads[fd]);
-	mailbox_destroy(&threads[fd]->mbox);
-	freez(threads[fd]);
-}
-
-/*
- * Allocate from thread arena
- */
-static void *
-thread_alloc(struct replay_thread *thr, size_t len)
-{
-	void *ptr;
-
-	if (sizeof thr->arena - thr->top < len)
-		return (NULL);
-	ptr = thr->arena + thr->top;
-	thr->top += len;
-	return (ptr);
-}
-
-/*
- * Returns a copy of the entire string with leading and trailing spaces
- * trimmed.
- */
-static char *
-trimline(struct replay_thread *thr, const char *str)
-{
-	size_t len;
-	char *p;
-
-	/* skip leading space */
-	while (*str && *str == ' ')
-		++str;
-
-	/* seek to end of string */
-	for (len = 0; str[len]; ++len)
-		 /* nothing */ ;
-
-	/* trim trailing space */
-	while (len && str[len - 1] == ' ')
-		--len;
-
-	/* copy and return */
-	if ((p = thread_alloc(thr, len + 1)) == NULL)
-		return (NULL);
-	memcpy(p, str, len);
-	p[len] = '\0';
-	return (p);
-}
-
-/* Read a line from the socket and return the number of bytes read.
- * After returning, line will point to the read bytes in memory.
- * A line is terminated by \r\n
- */
-static int
-read_line(struct replay_thread *thr)
-{
-	int i, len;
-
-	len = 0;
-	while (1) {
-		if (len + 2 > sizeof thr->line) {
-			thread_log(0, 0, "overflow");
-			return (-1);
-		}
-		i = read(thr->sock, thr->line + len, 1);
-		if (i < 0) {
-			thread_log(0, errno, "read(%d, %p, %d)",
-			    thr->sock, thr->line + len, 1);
-			return (-1);
-		}
-		if (i == 0)
-			break;
-		len += i;
-		if (len >= 2 && thr->line[len - 2] == '\r' &&
-		    thr->line[len - 1] == '\n') {
-			len -= 2;
-			break;
-		}
-	}
-	thr->line[len] = '\0';
-	return (len);
-}
-
-/* Read a block of data from the socket, and do nothing with it.
- * length says how many bytes to read, and the function returns
- * the number of bytes read.
- */
-static int
-read_block(struct replay_thread *thr, int len)
-{
-	int n, r, tot;
-
-	for (tot = 0; tot < len; tot += r) {
-		n = len - tot;
-		if (n > sizeof thr->temp)
-			n = sizeof thr->temp;
-		r = read(thr->sock, thr->temp, n);
-		if (r < 0) {
-			thread_log(0, errno, "read(%d, %p, %d)",
-			    thr->sock, thr->temp, n);
-			return (-1);
-		}
-		if (r == 0)
-			break;
-	}
-	return (tot);
-}
-
-/* Receive the response after sending a request.
- */
-static int
-receive_response(struct replay_thread *thr)
-{
-	const char *next;
-	int line_len;
-	long chunk_length, content_length;
-	int chunked, connclose, failed;
-	int n, status;
-
-	content_length = 0;
-	chunked = connclose = failed = 0;
-
-	/* Read header */
-	for (;;) {
-		line_len = read_line(thr);
-		if (line_len < 0)
-			return (-1);
-		thread_log(2, 0, "< %.*s", line_len, thr->line);
-		if (line_len == 0)
-			break;
-		if (strncmp(thr->line, "HTTP", 4) == 0) {
-			sscanf(thr->line, "%*s %d %*s\r\n", &status);
-			failed = (status != 200);
-		} else if (isprefix(thr->line, "content-length:", &next)) {
-			content_length = strtol(next, NULL, 10);
-		} else if (isprefix(thr->line, "transfer-encoding:", &next)) {
-			chunked = (strcasecmp(next, "chunked") == 0);
-		} else if (isprefix(thr->line, "connection:", &next)) {
-			connclose = (strcasecmp(next, "close") == 0);
-		}
-	}
-
-	thread_log(1, 0, "status: %d", status);
-
-	/* Read body */
-	if (chunked) {
-		/* Chunked encoding, read size and bytes until no more */
-		thread_log(1, 0, "chunked encoding");
-		for (;;) {
-			line_len = read_line(thr);
-			if (line_len < 0)
-				return (-1);
-			/* read_line() guarantees null-termination */
-			chunk_length = strtol(thr->line, NULL, 16);
-			if (chunk_length == 0)
-				break;
-			if ((n = read_block(thr, chunk_length)) < 0)
-				return (-1);
-			if (n < chunk_length)
-				thread_log(0, 0, "short read: %d/%ld",
-				    n, chunk_length);
-			thread_log(1, 0, "chunk length: %ld", chunk_length);
-			thread_log(1, 0, "bytes read: %d", n);
-			/* trailing CR LF */
-			if ((n = read_line(thr)) < 0)
-				return (-1);
-		}
-		/* trailing CR LF */
-		n = read_line(thr);
-		if (n < 0)
-			return (-1);
-	} else if (content_length > 0) {
-		/* Fixed body size, read content_length bytes */
-		thread_log(1, 0, "fixed length");
-		thread_log(1, 0, "content length: %ld", content_length);
-		if ((n = read_block(thr, content_length)) < 0)
-			return (1);
-		if (n < content_length)
-			thread_log(0, 0, "short read: %d/%ld",
-			    n, content_length);
-		thread_log(1, 0, "bytes read: %d", n);
-	} else {
-		/* No body --> stop reading. */
-		thread_log(1, 0, "no body");
-		return (-1);
-	}
-
-	return (connclose);
-}
-
-static void *
-replay_thread(void *arg)
-{
-	struct iovec iov[6];
-	char space[1] = " ", crlf[2] = "\r\n";
-	struct replay_thread *thr = arg;
-	struct message *msg;
-	enum VSL_tag_e tag;
-	char *ptr;
-	const char *next;
-
-	int i;
-
-	int reopen = 1;
-
-	while ((msg = mailbox_get(&thr->mbox)) != NULL) {
-		tag = msg->tag;
-		ptr = msg->ptr;
-
-		thread_log(2, 0, "%s(%s)", VSL_tags[tag], msg->ptr);
-
-		switch (tag) {
-		case SLT_ReqMethod:
-			if (thr->method != NULL)
-				thr->bogus = 1;
-			else
-				thr->method = trimline(thr, ptr);
-			break;
-
-		case SLT_ReqURL:
-			if (thr->url != NULL)
-				thr->bogus = 1;
-			else
-				thr->url = trimline(thr, ptr);
-			break;
-
-		case SLT_ReqProtocol:
-			if (thr->proto != NULL)
-				thr->bogus = 1;
-			else
-				thr->proto = trimline(thr, ptr);
-			break;
-
-		case SLT_ReqHeader:
-			if (thr->nhdr >= sizeof thr->hdr / sizeof *thr->hdr) {
-				thr->bogus = 1;
-			} else {
-				thr->hdr[thr->nhdr++] = trimline(thr, ptr);
-				if (isprefix(ptr, "connection:", &next))
-					thr->conn = trimline(thr, next);
-			}
-			break;
-
-		default:
-			break;
-		}
-
-		freez(msg->ptr);
-		freez(msg);
-
-		if (tag != SLT_ReqEnd)
-			continue;
-
-		if (!thr->method || !thr->url || !thr->proto) {
-			thr->bogus = 1;
-		} else if (strcmp(thr->method, "GET") != 0 &&
-		    strcmp(thr->method, "HEAD") != 0) {
-			thr->bogus = 1;
-		} else if (strcmp(thr->proto, "HTTP/1.0") == 0) {
-			reopen = !(thr->conn &&
-			    strcasecmp(thr->conn, "keep-alive") == 0);
-		} else if (strcmp(thr->proto, "HTTP/1.1") == 0) {
-			reopen = (thr->conn &&
-			    strcasecmp(thr->conn, "close") == 0);
-		} else {
-			thr->bogus = 1;
-		}
-
-		if (thr->bogus) {
-			thread_log(1, 0, "bogus");
-			goto clear;
-		}
-
-		if (thr->sock == -1) {
-			for (;;) {
-				thread_log(1, 0, "sleeping before connect...");
-				usleep(1000 * (thr->fd % 3001));
-				thr->sock = VSS_connect(addr_info, 0);
-				if (thr->sock >= 0)
-					break;
-				thread_log(0, errno, "connect failed");
-			}
-		}
-
-		thread_log(1, 0, "%s %s %s", thr->method, thr->url, thr->proto);
-
-		iov[0].iov_base = thr->method;
-		iov[0].iov_len = strlen(thr->method);
-		iov[2].iov_base = thr->url;
-		iov[2].iov_len = strlen(thr->url);
-		iov[4].iov_base = thr->proto;
-		iov[4].iov_len = strlen(thr->proto);
-		iov[1].iov_base = iov[3].iov_base = space;
-		iov[1].iov_len = iov[3].iov_len = 1;
-		iov[5].iov_base = crlf;
-		iov[5].iov_len = 2;
-		if (writev(thr->sock, iov, 6) == -1) {
-			thread_log(0, errno, "writev()");
-			goto close;
-		}
-
-		for (i = 0; i < thr->nhdr; ++i) {
-			thread_log(2, 0, "%d %s", i, thr->hdr[i]);
-			iov[0].iov_base = thr->hdr[i];
-			iov[0].iov_len = strlen(thr->hdr[i]);
-			iov[1].iov_base = crlf;
-			iov[1].iov_len = 2;
-			if (writev(thr->sock, iov, 2) == -1) {
-				thread_log(0, errno, "writev()");
-				goto close;
-			}
-		}
-		if (write(thr->sock, crlf, 2) == -1) {
-			thread_log(0, errno, "writev()");
-			goto close;
-		}
-		if (receive_response(thr) || reopen) {
-close:
-			thread_log(1, 0, "close");
-			assert(thr->sock != -1);
-			close(thr->sock);
-			thr->sock = -1;
-		}
-
-		sleep(1);
-clear:
-		/* clean up */
-		thread_clear(thr);
-	}
-
-	/* leftovers */
-	thread_clear(thr);
-
-	return (0);
-}
-
-static int
-gen_traffic(void *priv, enum VSL_tag_e tag, unsigned fd,
-    unsigned len, unsigned spec, const char *ptr, uint64_t bitmap)
-{
-	struct replay_thread *thr;
-	struct message *msg;
-
-	(void)priv;
-	(void)bitmap;
-
-	if (fd == 0 || !(spec & VSL_S_CLIENT))
-		return (0);
-
-	thread_log(3, 0, "%d %s", fd, VSL_tags[tag]);
-	thr = thread_get(fd, replay_thread);
-	if (thr == NULL)
-		return (0);
-	msg = malloc(sizeof (struct message));
-	msg->tag = tag;
-	msg->len = len;
-	msg->ptr = malloc(len);
-	AN(msg->ptr);
-	memcpy(msg->ptr, ptr, len);
-	mailbox_put(&thr->mbox, msg);
-
-	return (0);
-}
-
-/* Initiate a connection to <address> by resolving the
- * hostname and returning a struct with necessary
- * connection info.
- */
-static struct vss_addr *
-init_connection(const char *address)
-{
-	struct vss_addr **ta;
-	struct vss_addr *tap;
-	int i, n;
-
-	n = VSS_resolve(address, NULL, &ta);
-	if (n == 0) {
-		thread_log(0, 0, "Could not connect to server");
-		exit(2);
-	}
-	for (i = 1; i < n; ++i) {
-		free(ta[i]);
-		ta[i] = NULL;
-	}
-	tap = ta[0];
-	free(ta);
-
-	return (tap);
-}
-
-/*--------------------------------------------------------------------*/
-
-static void
-usage(void)
-{
-
-	fprintf(stderr,
-	    "usage: varnishreplay [-D] -a address:port -r logfile\n");
-	exit(1);
-}
-
-int
-main(int argc, char *argv[])
-{
-	int c;
-	struct VSM_data *vd;
-	const char *address = NULL;
-
-	vd = VSM_New();
-	debug = 0;
-
-	VSL_Arg(vd, 'c', NULL);
-	while ((c = getopt(argc, argv, "a:Dr:n:")) != -1) {
-		switch (c) {
-		case 'a':
-			address = optarg;
-			break;
-		case 'D':
-			++debug;
-			break;
-		default:
-			if (VSL_Arg(vd, c, optarg) > 0)
-				break;
-			usage();
-		}
-	}
-
-	if (address == NULL) {
-		usage();
-	}
-
-	if (VSM_Open(vd)) {
-		fprintf(stderr, "%s\n", VSM_Error(vd));
-		exit(1);
-	}
-
-	addr_info = init_connection(address);
-
-	signal(SIGPIPE, SIG_IGN);
-
-	pthread_attr_init(&thread_attr);
-
-	/*
-	 * XXX: seting the stack size manually reduces the memory usage
-	 * XXX: (allowing more threads) and increases speed (?)
-	 */
-	pthread_attr_setstacksize(&thread_attr, 32768);
-
-	while (VSL_Dispatch(vd, gen_traffic, NULL) == 0)
-		/* nothing */ ;
-	thread_close(-1);
-	exit(0);
-}
diff --git a/man/Makefile.am b/man/Makefile.am
index d791d1f..82f5471 100644
--- a/man/Makefile.am
+++ b/man/Makefile.am
@@ -64,9 +64,6 @@ varnishlog.1: \
 	$(top_srcdir)/doc/sphinx/include/varnishlog_synopsis.rst
 	${RST2MAN} $(RST2ANY_FLAGS) $(top_srcdir)/doc/sphinx/reference/varnishlog.rst $@
 
-varnishreplay.1: $(top_srcdir)/doc/sphinx/reference/varnishreplay.rst
-	${RST2MAN} $(RST2ANY_FLAGS) $(top_srcdir)/doc/sphinx/reference/varnishreplay.rst $@
-
 # XXX add _options.rst and _synopsis.rst here when it's been _opt2rst'ed
 varnishstat.1: $(top_srcdir)/doc/sphinx/reference/varnishstat.rst
 	${RST2MAN} $(RST2ANY_FLAGS) $(top_srcdir)/doc/sphinx/reference/varnishstat.rst $@



From lkarsten at varnish-software.com  Thu Jan 14 14:15:05 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:05 +0100
Subject: [4.1] 2a7fbd5 Typos and grammar
Message-ID: <E1aJig5-00058c-85@project.varnish-software.com>


commit 2a7fbd5014340b3b4682f94d4d6bb0ebdc5de884
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Wed Nov 11 13:36:07 2015 +0000

    Typos and grammar
    
    Fixes #1814 and more.

diff --git a/bin/varnishadm/varnishadm.c b/bin/varnishadm/varnishadm.c
index 1f02da9..12650b2 100644
--- a/bin/varnishadm/varnishadm.c
+++ b/bin/varnishadm/varnishadm.c
@@ -382,7 +382,7 @@ usage(void)
 	fprintf(stderr,
 	    "usage: varnishadm [-n ident] [-t timeout] [-S secretfile] "
 	    "-T [address]:port command [...]\n");
-	fprintf(stderr, "\t-n is mutually exlusive with -S and -T\n");
+	fprintf(stderr, "\t-n is mutually exclusive with -S and -T\n");
 	exit(1);
 }
 
diff --git a/bin/varnishd/cache/cache_esi_parse.c b/bin/varnishd/cache/cache_esi_parse.c
index aa31aa4..76591d0 100644
--- a/bin/varnishd/cache/cache_esi_parse.c
+++ b/bin/varnishd/cache/cache_esi_parse.c
@@ -1087,7 +1087,7 @@ VEP_Finish(struct vep_state *vep)
 		lcb = vep->cb(vep->vc, vep->cb_priv, 0, VGZ_ALIGN);
 		vep_emit_common(vep, lcb - vep->o_last, vep->last_mark);
 	}
-	// NB: We don't acount for PAD+SUM+LEN in gzip'ed objects
+	// NB: We don't account for PAD+SUM+LEN in gzip'ed objects
 	(void)vep->cb(vep->vc, vep->cb_priv, 0, VGZ_FINISH);
 
 	AZ(VSB_finish(vep->vsb));
diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index 8e0a70b..662048d 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -578,7 +578,7 @@ vbf_stp_fetch(struct worker *wrk, struct busyobj *bo)
 	    (bo->do_gunzip && !bo->is_gzip))
 		bo->do_gunzip = 0;
 
-	/* We wont gzip unless it is non-empty and ungziped */
+	/* We wont gzip unless it is non-empty and ungzip'ed */
 	if (bo->htc->body_status == BS_NONE ||
 	    bo->htc->content_length == 0 ||
 	    (bo->do_gzip && !bo->is_gunzip))
diff --git a/bin/varnishd/cache/cache_wrk.c b/bin/varnishd/cache/cache_wrk.c
index 86e0b92..7b774b1 100644
--- a/bin/varnishd/cache/cache_wrk.c
+++ b/bin/varnishd/cache/cache_wrk.c
@@ -44,7 +44,7 @@
 static void Pool_Work_Thread(struct pool *pp, struct worker *wrk);
 
 /*--------------------------------------------------------------------
- * Create and starte a back-ground thread which as its own worker and
+ * Create and start a back-ground thread which as its own worker and
  * session data structures;
  */
 
diff --git a/bin/varnishd/common/common.h b/bin/varnishd/common/common.h
index 7b2c371..da489fc 100644
--- a/bin/varnishd/common/common.h
+++ b/bin/varnishd/common/common.h
@@ -127,7 +127,7 @@ void VSM_common_ageupdate(const struct vsm_sc *sc);
 #define RUP2(x, y)  (((x)+((y)-1))&(~((uintptr_t)(y)-1UL))) /* PWR2(y) true */
 
 /*--------------------------------------------------------------------
- * Pointer aligment magic
+ * Pointer alignment magic
  */
 
 #if defined(__sparc__)
diff --git a/bin/varnishd/mgt/mgt_jail_solaris.c b/bin/varnishd/mgt/mgt_jail_solaris.c
index bb90598..2dd9322 100644
--- a/bin/varnishd/mgt/mgt_jail_solaris.c
+++ b/bin/varnishd/mgt/mgt_jail_solaris.c
@@ -27,8 +27,8 @@
  * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
  * SUCH DAMAGE.
  *
- * "Jailing" *1) child processes on Solaris and Solaris-derivates *2)
- * ==================================================================
+ * "Jailing" *1) child processes on Solaris and Solaris-derivatives *2)
+ * ====================================================================
  *
  * *1) The name is motivated by the availability of the -j command line
  *     option. Jailing Varnish is not to be confused with BSD Jails or
diff --git a/bin/varnishd/proxy/cache_proxy_proto.c b/bin/varnishd/proxy/cache_proxy_proto.c
index 6f64f3a..4799e4b 100644
--- a/bin/varnishd/proxy/cache_proxy_proto.c
+++ b/bin/varnishd/proxy/cache_proxy_proto.c
@@ -344,7 +344,7 @@ VPX_Proto_Sess(struct worker *wrk, void *priv)
 	CAST_OBJ_NOTNULL(req, priv, REQ_MAGIC);
 	sp = req->sp;
 
-	/* Per specifiction */
+	/* Per specification */
 	assert(sizeof vpx1_sig == 5);
 	assert(sizeof vpx2_sig == 12);
 
diff --git a/bin/varnishd/storage/storage_malloc.c b/bin/varnishd/storage/storage_malloc.c
index 445afab..ce6b0c2 100644
--- a/bin/varnishd/storage/storage_malloc.c
+++ b/bin/varnishd/storage/storage_malloc.c
@@ -86,7 +86,7 @@ sma_alloc(const struct stevedore *st, size_t size)
 	 * Do not collaps the sma allocation with sma->s.ptr: it is not
 	 * a good idea.  Not only would it make ->trim impossible,
 	 * performance-wise it would be a catastropy with chunksized
-	 * allocations growing another full page, just to accomodate the sma.
+	 * allocations growing another full page, just to accommodate the sma.
 	 */
 
 	p = malloc(size);
diff --git a/bin/varnishtest/tests.disabled/r01576.vtc b/bin/varnishtest/tests.disabled/r01576.vtc
index b0fdb9f..9ed5f7a 100644
--- a/bin/varnishtest/tests.disabled/r01576.vtc
+++ b/bin/varnishtest/tests.disabled/r01576.vtc
@@ -19,7 +19,7 @@ server s1 {
 # varnish v1 -arg "-p pcre_match_limit=1000"
 # varnish v1 -arg "-p pcre_match_limit_recursion=89"
 
-# Approximate formua for FreeBSD/amd64:
+# Approximate formula for FreeBSD/amd64:
 #	pcre_match_limit_recursion = thread_pool_stack * 2 - 9
 
 varnish v1 -vcl+backend {
diff --git a/config.phk b/config.phk
index 760ea22..906cbff 100644
--- a/config.phk
+++ b/config.phk
@@ -3,7 +3,7 @@
 set -e
 
 #######################################################################
-# Adminstrative settings
+# Administrative settings
 
 ADM_PROJECT=varnish
 ADM_VERSION=trunk
diff --git a/doc/changes.rst b/doc/changes.rst
index 6016ddc..04aed34 100644
--- a/doc/changes.rst
+++ b/doc/changes.rst
@@ -1557,7 +1557,7 @@ Changes from 2.1.1 to 2.1.2
 varnishd
 --------
 
--  When adding Range support for 2.1.1, we accidentially introduced a
+-  When adding Range support for 2.1.1, we accidentally introduced a
    bug which would append garbage to objects larger than the chunk size,
    by default 128k. Browsers would do the right thing due to
    Content-Length, but some load balancers would get very confused.
@@ -1964,7 +1964,7 @@ varnishd
    send out a zero-sized chunk which signifies end-of-transmission. We
    now ignore zero-sized chunks.
 
--  We accidentially slept for far too long when we reached the maximum
+-  We accidentally slept for far too long when we reached the maximum
    number of open file descriptors. This has been corrected and
    accept\_fd\_holdoff now works correctly.
 
@@ -2196,7 +2196,7 @@ varnishd
 -  Solaris is now supported.
 
 -  There is now a regsuball function, which works like regsub except it
-   replaces all occurences of the regex, not just the first.
+   replaces all occurrences of the regex, not just the first.
 
 -  Backend and director declarations can have a .connect\_timeout
    parameter, which tells us how long to wait for a successful
diff --git a/doc/sphinx/phk/autocrap.rst b/doc/sphinx/phk/autocrap.rst
index 308dde3..e97ac4c 100644
--- a/doc/sphinx/phk/autocrap.rst
+++ b/doc/sphinx/phk/autocrap.rst
@@ -55,7 +55,7 @@ different dialects to just a handful: Linux, \*BSD, Solaris and AIX
 and the autocrap tools have become part of the portability problem,
 rather than part of the solution.
 
-Amongst the silly activites of the autocrap generated configure script
+Amongst the silly activities of the autocrap generated configure script
 in Varnish are:
 
 * Looks for ANSI-C header files (show me a system later
diff --git a/doc/sphinx/phk/brinch-hansens-arrows.rst b/doc/sphinx/phk/brinch-hansens-arrows.rst
index cba1f44..f4299b5 100644
--- a/doc/sphinx/phk/brinch-hansens-arrows.rst
+++ b/doc/sphinx/phk/brinch-hansens-arrows.rst
@@ -16,7 +16,7 @@ lock B trying to get lock A.
 
 Brinch-Hansen did a lot of both theoretical and practical work in the
 area of multiprogramming and being both good at it and one of the
-pioneers, he was awardede the ACM Turing Prize for it.
+pioneers, he was awarded the ACM Turing Prize for it.
 
 You can read more about him here:
 `Brinch-Hansen Archive <http://brinch-hansen.net/>`_
@@ -26,7 +26,7 @@ that a given multiprogramming system was free of deadlocks:  Draw
 the locking order and make sure all the arrows point to the right.
 
 When we started working with multi-core systems in FreeBSD, we were
-sure to have deadlocks in our future, and we adobted and expanded
+sure to have deadlocks in our future, and we adopted and expanded
 a facility called "WITNESS" originally written for BSDI, which
 keeps an eye on Brinch-Hansens arrows in real time.
 
diff --git a/doc/sphinx/phk/dough.rst b/doc/sphinx/phk/dough.rst
index b629e8b..aef598b 100644
--- a/doc/sphinx/phk/dough.rst
+++ b/doc/sphinx/phk/dough.rst
@@ -184,7 +184,7 @@ the message to other communities, that Free and Open Source Software
 does not materialize out of empty space, it is written by people.
 
 People who love what we do, which is why I'm sitting here,
-way past midnight on a friday evening, writing this phamplet.
+way past midnight on a friday evening, writing this pamphlet.
 
 But software *is* written by people, real people with kids, cars,
 mortgages, leaky roofs, sick pets, infirm parents and all other
diff --git a/doc/sphinx/phk/http20.rst b/doc/sphinx/phk/http20.rst
index 9860cab..c2b4d17 100644
--- a/doc/sphinx/phk/http20.rst
+++ b/doc/sphinx/phk/http20.rst
@@ -59,7 +59,7 @@ What if they made a new protocol, and nobody used it ?
 We have learned, painfully, that an IPv6 which is only marginally
 better than IPv4 and which offers no tangible benefit for the people
 who have the cost/trouble of the upgrade, does not penetrate the
-network on its own, and barely even on goverments mandate.
+network on its own, and barely even on governments mandate.
 
 We have also learned that a protocol which delivers the goods can
 replace all competition in virtually no time.
@@ -86,16 +86,16 @@ Most notably HTTP/1.1 lacks a working session/endpoint-identity
 facility, a shortcoming which people have pasted over with the
 ill-conceived Cookie hack.
 
-Cookies are, as the EU commision correctly noted, fundamentally
+Cookies are, as the EU commission correctly noted, fundamentally
 flawed, because they store potentially sensitive information on
 whatever computer the user happens to use, and as a result of various
 abuses and incompetences, EU felt compelled to legislate a "notice
 and announce" policy for HTTP-cookies.
 
 But it doesn't stop there:  The information stored in cookies have
-potentialiiy very high value for the HTTP server, and because the
+potentially very high value for the HTTP server, and because the
 server has no control over the integrity of the storage, we are now
-seing cookies being crypto-signed, to prevent forgeries.
+seeing cookies being crypto-signed, to prevent forgeries.
 
 The term "bass ackwards" comes to mind.
 
@@ -124,7 +124,7 @@ new layer of complexity without removing any of the old complexity
 from the protocol.
 
 My conclusion is that HTTP/2.0 is really just a grandiose name for
-HTTP/1.2:  An attempt to smoothe out some sharp corners, to save a
+HTTP/1.2:  An attempt to smooth out some sharp corners, to save a
 bit of bandwidth, but not get anywhere near all the architectural
 problems of HTTP/1.1 and to preserve faithfully its heritage of
 badly thought out sedimentary hacks.
@@ -151,7 +151,7 @@ mobs and special event traffic spikes.
 
 In the time frame where HTTP/2.0 will become standardized, HTTP
 routers will routinely deal with 40Gbit/s traffic and people will
-start to arcitect for 1Tbit/s traffic.
+start to architect for 1Tbit/s traffic.
 
 HTTP routers are usually only interested in a small part of the
 HTTP request and barely in the response at all, usually only the
@@ -217,7 +217,7 @@ expend resources, and foresee a lot of complexity in implementing
 the server side to mitigate and deflect malicious traffic.
 
 Server Push breaks the HTTP transaction model, and opens a pile of
-cans of security and privacy issues, which whould not be sneaked
+cans of security and privacy issues, which would not be sneaked
 in during the design of a transport-encoding for HTTP/1+ traffic,
 but rather be standardized as an independent and well analysed
 extension to HTTP in general.
@@ -228,7 +228,7 @@ HTTP Speed+Mobility
 
 Is really just SPDY with WebSockets underneath.
 
-I'm really not sure I see any benefit to that, execept that the
+I'm really not sure I see any benefit to that, except that the
 encoding chosen is marginally more efficient to implement in
 hardware than SPDY.
 
@@ -268,7 +268,7 @@ Overall, I don't see any of the three proposals offer anything that
 will make the majority of web-sites go "Ohh we've been waiting for
 that!"
 
-Bigger sites will be entised by small bandwidth savings, but the
+Bigger sites will be enticed by small bandwidth savings, but the
 majority of the HTTP users will see scant or no net positive benefit
 if one or more of these three proposals were to become HTTP/2.0
 
diff --git a/doc/sphinx/phk/ssl_again.rst b/doc/sphinx/phk/ssl_again.rst
index ba4ecf1..31b6efe 100644
--- a/doc/sphinx/phk/ssl_again.rst
+++ b/doc/sphinx/phk/ssl_again.rst
@@ -54,7 +54,7 @@ The next morning `CloudFlare announced the very same thing`_:
 
 .. _CloudFlare announced the very same thing: https://blog.cloudflare.com/keyless-ssl-the-nitty-gritty-technical-details/
 
-This could conceiveably be a way to terminate TLS/SSL in the Varnish-worker
+This could conceivably be a way to terminate TLS/SSL in the Varnish-worker
 process, while keeping the most valuable crypto-bits away from it.
 
 But it's still a bad idea
@@ -117,7 +117,7 @@ I'm not too thrilled about the "SSL Everywhere" idea, for a large
 number of reasons.
 
 The most obvious example is that you don't want to bog down your
-countrys civil defence agency with SSL/TLS protocol negotiations,
+country's civil defence agency with SSL/TLS protocol negotiations,
 if their website is being deluged by people trying to survive a
 natural disaster.
 
diff --git a/doc/sphinx/phk/thetoolsweworkwith.rst b/doc/sphinx/phk/thetoolsweworkwith.rst
index 2631c73..4ee56db 100644
--- a/doc/sphinx/phk/thetoolsweworkwith.rst
+++ b/doc/sphinx/phk/thetoolsweworkwith.rst
@@ -153,7 +153,7 @@ duration for the call, because then it doesn't matter what time
 it is, only how long time has transpired.
 
 Ohh, and setting the stack-size for a new thread ?
-That is appearantly "too dangerous" so there is no argument in the
+That is apparently "too dangerous" so there is no argument in the
 C1X API for doing so, a clear step backwards from pthreads.
 
 But guess what:  Thread stacks are like T-shirts:  There is no "one
@@ -177,7 +177,7 @@ compiler can understand and use to issue warnings.
 Heck, even a simple basic object facility would be good addition,
 now that C++ have become this huge bloated monster language.
 
-But none of that is appearantly as important as <stdnoreturn.h>
+But none of that is apparently as important as <stdnoreturn.h>
 and a new, crippled and therefore useless thread API.
 
 The neat thing about the C language, and the one feature that made
diff --git a/doc/sphinx/reference/vcl.rst b/doc/sphinx/reference/vcl.rst
index cd942e6..c982cb9 100644
--- a/doc/sphinx/reference/vcl.rst
+++ b/doc/sphinx/reference/vcl.rst
@@ -59,9 +59,9 @@ Conditionals
 ------------
 
 VCL has *if* and *else* statements. Nested logic can be implemented
-with the *elseif* statement. (*elsif*/*elif*/*else if* is equivalent.)
+with the *elseif* statement (*elsif*/*elif*/*else if* are equivalent).
 
-Note that are no loops or iterators of any kind in VCL.
+Note that there are no loops or iterators of any kind in VCL.
 
 
 Strings, booleans, time, duration and integers
diff --git a/doc/sphinx/users-guide/vcl.rst b/doc/sphinx/users-guide/vcl.rst
index 3528e7b..0458704 100644
--- a/doc/sphinx/users-guide/vcl.rst
+++ b/doc/sphinx/users-guide/vcl.rst
@@ -13,7 +13,7 @@ switches. We have instead chosen to use a domain specific language called VCL fo
 
 Every inbound request flows through Varnish and you can influence how
 the request is being handled by altering the VCL code. You can direct
-certain requests to certains backends, you can alter the requests and
+certain requests to particular backends, you can alter the requests and
 the responses or have Varnish take various actions depending on
 arbitrary properties of the request or the response. This makes
 Varnish an extremely powerful HTTP processor, not just for caching.
diff --git a/include/tbl/http_headers.h b/include/tbl/http_headers.h
index 91ea213..0d973c7 100644
--- a/include/tbl/http_headers.h
+++ b/include/tbl/http_headers.h
@@ -86,7 +86,7 @@ H("If-Match",		H_If_Match,		  F  )	// 2616 14.24
 H("If-Modified-Since",	H_If_Modified_Since,	  F  )	// 2616 14.25
 H("If-None-Match",	H_If_None_Match,	  F  )	// 2616 14.26
 H("If-Range",		H_If_Range,		  F  )	// 2616 14.27
-H("If-Unmodified-Since",H_If_Unmodifed_Since,	  F  )	// 2616 14.28
+H("If-Unmodified-Since",H_If_Unmodified_Since,	  F  )	// 2616 14.28
 H("Last-Modified",	H_Last_Modified,	0    )	// 2616 14.29
 H("Location",		H_Location,		0    )	// 2616 14.30
 H("Max-Forwards",	H_Max_Forwards,		0    )	// 2616 14.31
diff --git a/include/vapi/vsl.h b/include/vapi/vsl.h
index 599c654..c6afb2f 100644
--- a/include/vapi/vsl.h
+++ b/include/vapi/vsl.h
@@ -215,7 +215,7 @@ int VSL_List2Tags(const char *list, int l, VSL_tagfind_f *func, void *priv);
 	 *   func: The function to call (can be NULL)
 	 *   priv: An argument that will be passed to func.
 	 *
-	 * Return valus:
+	 * Return values:
 	 *     >0: Number of times func was called for matching tags.
 	 *     -1: No tag matches for list element
 	 *     -2: Multiple tags match non-glob list element
diff --git a/lib/libvarnish/vfil.c b/lib/libvarnish/vfil.c
index 19928ff..4b57d01 100644
--- a/lib/libvarnish/vfil.c
+++ b/lib/libvarnish/vfil.c
@@ -158,7 +158,7 @@ VFIL_fsinfo(int fd, unsigned *pbs, uintmax_t *psize, uintmax_t *pspace)
 	return (0);
 }
 
-/* Make sure that the file system can accomodate the file of the given
+/* Make sure that the file system can accommodate the file of the given
  * size. Will use fallocate if available. If fallocate is not available
  * and insist is true, it will write size zero bytes.
  *
@@ -191,7 +191,7 @@ VFIL_allocate(int fd, off_t size, int insist)
 		   the already allocated blocks of the file into
 		   account. This will cause fallocate to report ENOSPC
 		   when called on an existing fully allocated file unless
-		   the filesystem has enough free space to accomodate the
+		   the filesystem has enough free space to accommodate the
 		   complete new file size. Because of this we enable
 		   fallocate only on filesystems that are known to work as
 		   we expect. */
diff --git a/lib/libvarnish/vsub.c b/lib/libvarnish/vsub.c
index cf71c26..4e3067a 100644
--- a/lib/libvarnish/vsub.c
+++ b/lib/libvarnish/vsub.c
@@ -106,7 +106,7 @@ VSUB_run(struct vsb *sb, vsub_func_f *func, void *priv, const char *name,
 		/*
 		 * func should either exec or exit, so getting here should be
 		 * treated like an assertion failure - except that we don't know
-		 * if it's safe to trigger an acutal assertion
+		 * if it's safe to trigger an actual assertion
 		 */
 		_exit(4);
 	}
diff --git a/lib/libvarnish/vtcp.c b/lib/libvarnish/vtcp.c
index 3992555..cb4d3c4 100644
--- a/lib/libvarnish/vtcp.c
+++ b/lib/libvarnish/vtcp.c
@@ -545,7 +545,7 @@ VTCP_Check(int a)
 		return (1);
 #if (defined (__SVR4) && defined (__sun)) || defined (__NetBSD__)
 	/*
-	 * Solaris returns EINVAL if the other end unexepectedly reset the
+	 * Solaris returns EINVAL if the other end unexpectedly reset the
 	 * connection.
 	 * This is a bug in Solaris and documented behaviour on NetBSD.
 	 */
diff --git a/lib/libvcc/vcc_compile.c b/lib/libvcc/vcc_compile.c
index 4bd7821..e79597f 100644
--- a/lib/libvcc/vcc_compile.c
+++ b/lib/libvcc/vcc_compile.c
@@ -30,7 +30,7 @@
 /*
  * XXX:
  *	Better error messages, throughout.
- *	>It also accured to me that we could link the errors to the error
+ *	>It also occurred to me that we could link the errors to the error
  *	>documentation.
  *	>
  *	>Unreferenced  function 'request_policy', first mention is



From lkarsten at varnish-software.com  Thu Jan 14 14:15:05 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:05 +0100
Subject: [4.1] 772777a Adding a newline to fix comment
Message-ID: <E1aJig5-00059J-Cv@project.varnish-software.com>


commit 772777ac3d103fa8a2413ca207cbe29d217d67cc
Author: franciscovg <francisco at varnish-software.com>
Date:   Thu Aug 20 11:57:00 2015 +0200

    Adding a newline to fix comment

diff --git a/doc/sphinx/users-guide/vcl-actions.rst b/doc/sphinx/users-guide/vcl-actions.rst
index d4ede58..ed2b8ba 100644
--- a/doc/sphinx/users-guide/vcl-actions.rst
+++ b/doc/sphinx/users-guide/vcl-actions.rst
@@ -18,6 +18,7 @@ The most common actions to return are these:
   should be passed. 
 
 *pipe*
+
 .. XXX:What is pipe? benc
 
   Pipe can be returned from `vcl_recv` as well. Pipe short circuits the



From lkarsten at varnish-software.com  Thu Jan 14 14:15:05 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:05 +0100
Subject: [4.1] 52d5d99 Use AC_COMPILE_IFELSE
Message-ID: <E1aJig5-00058p-B0@project.varnish-software.com>


commit 52d5d99394b09ec07873d2f09c16e7708f2a56f4
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Thu Nov 12 01:22:18 2015 +0000

    Use AC_COMPILE_IFELSE
    
    AC_PREPROC_IFELSE is frowned upon.  Take into account installations
    outside the compiler's default search path.

diff --git a/configure.ac b/configure.ac
index 4357c3c..d4f1923 100644
--- a/configure.ac
+++ b/configure.ac
@@ -159,7 +159,9 @@ AC_ARG_ENABLE(pcre-jit,
     [enable_pcre_jit=yes])
 if test "$enable_pcre_jit" = yes; then
   AC_MSG_CHECKING(for PCRE JIT usability)
-  AC_PREPROC_IFELSE(
+  save_CFLAGS="${CFLAGS}"
+  CFLAGS="${PCRE_CFLAGS}"
+  AC_COMPILE_IFELSE(
     [AC_LANG_PROGRAM([[#include "pcre.h"
                        #if PCRE_MAJOR != 8 || PCRE_MINOR < 32
                        #error no jit
@@ -169,6 +171,7 @@ if test "$enable_pcre_jit" = yes; then
     ],
     [AC_MSG_RESULT(no)]
   )
+  CFLAGS="${save_CFLAGS}"
 fi
 
 



From lkarsten at varnish-software.com  Thu Jan 14 14:15:05 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:05 +0100
Subject: [4.1] 23fc002 Adding detials to installation from source code
Message-ID: <E1aJig5-00059W-FG@project.varnish-software.com>


commit 23fc0028b45ce0eac086d358636b8abb1d7ba109
Author: Francisco Vel?zquez <francisco at varnish-software.com>
Date:   Tue Aug 25 18:29:15 2015 +0200

    Adding detials to installation from source code

diff --git a/doc/sphinx/installation/install.rst b/doc/sphinx/installation/install.rst
index 398e9af..e67757f 100644
--- a/doc/sphinx/installation/install.rst
+++ b/doc/sphinx/installation/install.rst
@@ -141,9 +141,13 @@ Installing
 
 And finally, the true test of a brave heart: ``sudo make install``
 
-Varnish will now be installed in `/usr/local`. The `varnishd` binary is in
-`/usr/local/sbin/varnishd` and its default configuration will be
-`/usr/local/etc/varnish/default.vcl`.
+Varnish will now be installed in ``/usr/local``. The ``varnishd`` binary is in
+`/usr/local/sbin/varnishd`. To make sure that the necessary links and caches
+to the most recent shared libraries are found, run ``sudo ldconfig``.
 
 After successful installation you are ready to proceed to the :ref:`tutorial-index`.
+This tutorial, however, is written for installations from binary packages.
+In practice, it means that you cannot call ``service varnish start`` out of the box.
+Instead, you start the varnish daemon manually, for example::
 
+        varnishd -a :6081 -T localhost:6082 -b localhost:8080 -f /path/my_file.vcl



From lkarsten at varnish-software.com  Thu Jan 14 14:15:05 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:05 +0100
Subject: [4.1] dfa5c5e Separating next steps after installation in its own
 section
Message-ID: <E1aJig5-00059r-HA@project.varnish-software.com>


commit dfa5c5e54502944cf502f0ea300d3a295a4effae
Author: Francisco Vel?zquez <francisco at varnish-software.com>
Date:   Tue Aug 25 18:47:20 2015 +0200

    Separating next steps after installation in its own section

diff --git a/doc/sphinx/installation/install.rst b/doc/sphinx/installation/install.rst
index e67757f..8f19f91 100644
--- a/doc/sphinx/installation/install.rst
+++ b/doc/sphinx/installation/install.rst
@@ -143,11 +143,15 @@ And finally, the true test of a brave heart: ``sudo make install``
 
 Varnish will now be installed in ``/usr/local``. The ``varnishd`` binary is in
 `/usr/local/sbin/varnishd`. To make sure that the necessary links and caches
-to the most recent shared libraries are found, run ``sudo ldconfig``.
+of the most recent shared libraries are found, run ``sudo ldconfig``.
+
+Next steps
+==========
 
 After successful installation you are ready to proceed to the :ref:`tutorial-index`.
-This tutorial, however, is written for installations from binary packages.
-In practice, it means that you cannot call ``service varnish start`` out of the box.
-Instead, you start the varnish daemon manually, for example::
+
+This tutorial is written for installations from binary packages.
+In practice, it means that some configurations are not in place for installations from source code.
+For example, instead of calling ``service varnish start``, you start the varnish daemon manually by typing::
 
         varnishd -a :6081 -T localhost:6082 -b localhost:8080 -f /path/my_file.vcl



From lkarsten at varnish-software.com  Thu Jan 14 14:15:05 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:05 +0100
Subject: [4.1] d637844 varnishd cannot start with have both options -b and -f
Message-ID: <E1aJig5-0005A2-IZ@project.varnish-software.com>


commit d6378448d04c1c00a3df48c1862408be670d8c87
Author: Francisco Vel?zquez <francisco at varnish-software.com>
Date:   Mon Sep 14 16:56:47 2015 +0200

    varnishd cannot start with have both options -b and -f

diff --git a/doc/sphinx/installation/install.rst b/doc/sphinx/installation/install.rst
index 8f19f91..e455016 100644
--- a/doc/sphinx/installation/install.rst
+++ b/doc/sphinx/installation/install.rst
@@ -154,4 +154,4 @@ This tutorial is written for installations from binary packages.
 In practice, it means that some configurations are not in place for installations from source code.
 For example, instead of calling ``service varnish start``, you start the varnish daemon manually by typing::
 
-        varnishd -a :6081 -T localhost:6082 -b localhost:8080 -f /path/my_file.vcl
+        varnishd -a :6081 -T localhost:6082 -b localhost:8080



From lkarsten at varnish-software.com  Thu Jan 14 14:15:05 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:05 +0100
Subject: [4.1] c447a87 Remove dead code
Message-ID: <E1aJig5-0005AE-Km@project.varnish-software.com>


commit c447a87c0af588db4ea46e493a7c25b131e26fd0
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Wed Nov 18 00:48:30 2015 +0900

    Remove dead code

diff --git a/lib/libvmod_directors/hash.c b/lib/libvmod_directors/hash.c
index 3029c7b..a858a09 100644
--- a/lib/libvmod_directors/hash.c
+++ b/lib/libvmod_directors/hash.c
@@ -35,7 +35,6 @@
 #include "cache/cache_director.h"
 
 #include "vrt.h"
-#include "vbm.h"
 #include "vend.h"
 #include "vsha256.h"
 
@@ -47,7 +46,6 @@ struct vmod_directors_hash {
 	unsigned				magic;
 #define VMOD_DIRECTORS_HASH_MAGIC		0xc08dd611
 	struct vdir				*vd;
-	struct vbitmap				*vbm;
 };
 
 VCL_VOID __match_proto__()
@@ -61,8 +59,6 @@ vmod_hash__init(VRT_CTX, struct vmod_directors_hash **rrp,
 	AZ(*rrp);
 	ALLOC_OBJ(rr, VMOD_DIRECTORS_HASH_MAGIC);
 	AN(rr);
-	rr->vbm = vbit_init(8);
-	AN(rr->vbm);
 	*rrp = rr;
 	vdir_new(&rr->vd, "hash", vcl_name, NULL, NULL, rr);
 }
@@ -76,7 +72,6 @@ vmod_hash__fini(struct vmod_directors_hash **rrp)
 	*rrp = NULL;
 	CHECK_OBJ_NOTNULL(rr, VMOD_DIRECTORS_HASH_MAGIC);
 	vdir_delete(&rr->vd);
-	vbit_destroy(rr->vbm);
 	FREE_OBJ(rr);
 }
 



From lkarsten at varnish-software.com  Thu Jan 14 14:15:05 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:05 +0100
Subject: [4.1] 4590fd9 Fix test in case someone has enough time to run it
Message-ID: <E1aJig5-0005AO-MH@project.varnish-software.com>


commit 4590fd99eb8bddc82a2a2e575a365d8cd6f2f040
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Wed Nov 18 01:00:40 2015 +0900

    Fix test in case someone has enough time to run it
    
    P.S. It requires -t 80.

diff --git a/bin/varnishtest/tests.disabled/r01252.vtc b/bin/varnishtest/tests.disabled/r01252.vtc
index 174ee97..fd820e9 100644
--- a/bin/varnishtest/tests.disabled/r01252.vtc
+++ b/bin/varnishtest/tests.disabled/r01252.vtc
@@ -22,19 +22,19 @@ server s2 {
 varnish v1 -arg "-p debug=+waitinglist" -vcl+backend {
 	sub vcl_recv {
 		if (req.http.x-client == "2") {
-			set req.backend = s2;
+			set req.backend_hint = s2;
 		}
 	}
 } -start
 
 varnish v1 -cliok "param.show tcp_keepalive_time"
-varnish v1 -cliok "param.set tcp_keepalive_time 1s"
+varnish v1 -cliok "param.set tcp_keepalive_time 1"
 
 varnish v1 -cliok "param.show tcp_keepalive_probes"
 varnish v1 -cliok "param.set tcp_keepalive_probes 1"
 
 varnish v1 -cliok "param.show tcp_keepalive_intvl"
-varnish v1 -cliok "param.set tcp_keepalive_intvl 1s"
+varnish v1 -cliok "param.set tcp_keepalive_intvl 1"
 
 varnish v1 -cliok "param.show first_byte_timeout"
 varnish v1 -cliok "param.set first_byte_timeout 70"



From lkarsten at varnish-software.com  Thu Jan 14 14:15:05 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:05 +0100
Subject: [4.1] 3c8b8e1 Remove a couple of commented out asserts which will not
 come back.
Message-ID: <E1aJig5-0005AZ-OP@project.varnish-software.com>


commit 3c8b8e1de54e2bb960308ebf657d991941d99990
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Nov 18 12:11:57 2015 +0000

    Remove a couple of commented out asserts which will not come back.

diff --git a/bin/varnishd/cache/cache_expire.c b/bin/varnishd/cache/cache_expire.c
index e74b328..65236b7 100644
--- a/bin/varnishd/cache/cache_expire.c
+++ b/bin/varnishd/cache/cache_expire.c
@@ -435,8 +435,6 @@ exp_inbox(struct exp_priv *ep, struct objcore *oc, double now)
 	VSLb(&ep->vsl, SLT_ExpKill, "EXP_Inbox p=%p e=%.9f f=0x%x", oc,
 	    oc->timer_when, oc->flags);
 
-	// AZ(oc->flags & OC_F_BUSY);
-
 	lru = ObjGetLRU(oc);
 	CHECK_OBJ_NOTNULL(lru, LRU_MAGIC);
 
@@ -519,7 +517,7 @@ exp_expire(struct exp_priv *ep, double now)
 	lru = ObjGetLRU(oc);
 	CHECK_OBJ_NOTNULL(lru, LRU_MAGIC);
 	Lck_Lock(&lru->mtx);
-	// AZ(oc->flags & OC_F_BUSY);
+
 	oc->exp_flags |= OC_EF_DYING;
 	if (oc->exp_flags & OC_EF_OFFLRU)
 		oc = NULL;



From lkarsten at varnish-software.com  Thu Jan 14 14:15:05 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:05 +0100
Subject: [4.1] acba53a Be more rigid about locking in cache_ban.
Message-ID: <E1aJig5-0005Al-Qy@project.varnish-software.com>


commit acba53a8049b348b7c3c98911c224c73e01948c1
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu Nov 19 08:54:17 2015 +0000

    Be more rigid about locking in cache_ban.

diff --git a/bin/varnishd/cache/cache_ban.c b/bin/varnishd/cache/cache_ban.c
index 565b5d1..ed9fcc2 100644
--- a/bin/varnishd/cache/cache_ban.c
+++ b/bin/varnishd/cache/cache_ban.c
@@ -237,14 +237,14 @@ BAN_DestroyObj(struct objcore *oc)
 {
 
 	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
-	if (oc->ban == NULL)
-		return;
-	CHECK_OBJ_NOTNULL(oc->ban, BAN_MAGIC);
 	Lck_Lock(&ban_mtx);
-	assert(oc->ban->refcount > 0);
-	oc->ban->refcount--;
-	VTAILQ_REMOVE(&oc->ban->objcore, oc, ban_list);
-	oc->ban = NULL;
+	CHECK_OBJ_ORNULL(oc->ban, BAN_MAGIC);
+	if (oc->ban != NULL) {
+		assert(oc->ban->refcount > 0);
+		oc->ban->refcount--;
+		VTAILQ_REMOVE(&oc->ban->objcore, oc, ban_list);
+		oc->ban = NULL;
+	}
 	Lck_Unlock(&ban_mtx);
 }
 
@@ -549,22 +549,26 @@ BAN_CheckObject(struct worker *wrk, struct objcore *oc, struct req *req)
 
 	oc->ban->refcount--;
 	VTAILQ_REMOVE(&oc->ban->objcore, oc, ban_list);
-	if (b == oc->ban) {	/* not banned */
+	if (b == oc->ban) {
+		/* not banned */
 		VTAILQ_INSERT_TAIL(&b0->objcore, oc, ban_list);
 		b0->refcount++;
+		oc->ban = b0;
+		b = NULL;
+	} else {
+		oc->ban = NULL;
 	}
 
-	if (oc->ban->refcount == 0 && VTAILQ_NEXT(oc->ban, list) == NULL)
+	if (VTAILQ_LAST(&ban_head, banhead_s)->refcount == 0)
 		ban_kick_lurker();
 
 	Lck_Unlock(&ban_mtx);
 
-	if (b == oc->ban) {	/* not banned */
-		oc->ban = b0;
+	if (b == NULL) {
+		/* not banned */
 		ObjUpdateMeta(wrk, oc);
 		return (0);
 	} else {
-		oc->ban = NULL;
 		VSLb(vsl, SLT_ExpBan, "%u banned lookup", ObjGetXID(wrk, oc));
 		VSC_C_main->bans_obj_killed++;
 		EXP_Rearm(oc, oc->exp.t_origin, 0, 0, 0);	// XXX fake now



From lkarsten at varnish-software.com  Thu Jan 14 14:15:05 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:05 +0100
Subject: [4.1] 3dd8866 Fix output
Message-ID: <E1aJig5-0005Aw-Sp@project.varnish-software.com>


commit 3dd8866490e3d91355cc06aa94f0055e4b5dedc1
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Sun Nov 22 10:52:09 2015 +0900

    Fix output

diff --git a/bin/varnishd/mgt/mgt_acceptor.c b/bin/varnishd/mgt/mgt_acceptor.c
index 49cc0c8..8edfd1c 100644
--- a/bin/varnishd/mgt/mgt_acceptor.c
+++ b/bin/varnishd/mgt/mgt_acceptor.c
@@ -202,19 +202,19 @@ MAC_Arg(const char *arg)
 		mh->first_step = S_STP_H1NEWSESS;
 		mh->proto_name = "HTTP/1";
 		if (av[2] != NULL && av[3] != NULL)
-			ARGV_ERR("Too many sub-arguments to -a(HTTP/1))\n");
+			ARGV_ERR("Too many sub-arguments to -a(HTTP/1)\n");
 	} else if (!strcmp(av[2], "PROXY")) {
 		mh->first_step = S_STP_PROXYNEWSESS;
 		mh->proto_name = "PROXY";
 		if (av[3] != NULL)
-			ARGV_ERR("Too many sub-arguments to -a(PROXY))\n");
+			ARGV_ERR("Too many sub-arguments to -a(PROXY)\n");
 	} else {
 		ARGV_ERR("Unknown protocol '%s'\n", av[2]);
 	}
 
 	error = VSS_resolver(av[1], "80", mac_callback, mh, &err);
 	if (mh->good == 0 || error)
-		ARGV_ERR("socket %s didn't resolve \n", av[1]);
+		ARGV_ERR("socket %s didn't resolve\n", av[1]);
 	VAV_Free(av);
 	FREE_OBJ(mh);
 }



From lkarsten at varnish-software.com  Thu Jan 14 14:15:05 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:05 +0100
Subject: [4.1] 45c1605 Add a defensive assert
Message-ID: <E1aJig5-0005BD-Uf@project.varnish-software.com>


commit 45c16052f7df4ed22b988054a5400c2d3388ea29
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu Nov 19 10:11:19 2015 +0000

    Add a defensive assert

diff --git a/bin/varnishd/storage/stevedore.c b/bin/varnishd/storage/stevedore.c
index 0929ba8..77e54a3 100644
--- a/bin/varnishd/storage/stevedore.c
+++ b/bin/varnishd/storage/stevedore.c
@@ -68,6 +68,7 @@ default_oc_freeobj(struct worker *wrk, struct objcore *oc)
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
+	CHECK_OBJ_NOTNULL(oc->stobj, STOREOBJ_MAGIC);
 	ObjSlim(wrk, oc);
 	CAST_OBJ_NOTNULL(o, oc->stobj->priv, OBJECT_MAGIC);
 	o->magic = 0;



From lkarsten at varnish-software.com  Thu Jan 14 14:15:05 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:05 +0100
Subject: [4.1] 4cb9782 vslc_file can fit on 64 bytes
Message-ID: <E1aJig5-0005BN-Vy@project.varnish-software.com>


commit 4cb9782f37626bd1edda7cdbc7b7a67a5bcff8c9
Author: Guillaume Quintard <guillaume.quintard at gmail.com>
Date:   Mon Mar 16 11:51:49 2015 +0100

    vslc_file can fit on 64 bytes

diff --git a/lib/libvarnishapi/vsl_cursor.c b/lib/libvarnishapi/vsl_cursor.c
index e908bd7..e9daab0 100644
--- a/lib/libvarnishapi/vsl_cursor.c
+++ b/lib/libvarnishapi/vsl_cursor.c
@@ -288,13 +288,14 @@ struct vslc_file {
 	unsigned			magic;
 #define VSLC_FILE_MAGIC			0x1D65FFEF
 
-	struct VSL_cursor		cursor;
-
 	int				error;
 	int				fd;
 	int				close_fd;
 	ssize_t				buflen;
 	uint32_t			*buf;
+
+	struct VSL_cursor		cursor;
+
 };
 
 static void



From lkarsten at varnish-software.com  Thu Jan 14 14:15:06 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:06 +0100
Subject: [4.1] b8afbfb format can fit on 64 bytes
Message-ID: <E1aJig6-0005Ba-1a@project.varnish-software.com>


commit b8afbfbec789cf46d26bc53e2bb84191ef040904
Author: Guillaume Quintard <guillaume.quintard at gmail.com>
Date:   Mon Mar 16 12:01:50 2015 +0100

    format can fit on 64 bytes

diff --git a/bin/varnishncsa/varnishncsa.c b/bin/varnishncsa/varnishncsa.c
index d46bee8..95c4454 100644
--- a/bin/varnishncsa/varnishncsa.c
+++ b/bin/varnishncsa/varnishncsa.c
@@ -100,12 +100,12 @@ struct format {
 	unsigned		magic;
 #define FORMAT_MAGIC		0xC3119CDA
 
+	char			time_type;
 	VTAILQ_ENTRY(format)	list;
 	format_f		*func;
 	struct fragment		*frag;
 	char			*string;
 	const char *const	*strptr;
-	char			time_type;
 	char			*time_fmt;
 };
 



From lkarsten at varnish-software.com  Thu Jan 14 14:15:06 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:06 +0100
Subject: [4.1] c0cae3f vef_priv can fit on 64 bytes
Message-ID: <E1aJig6-0005Bk-3D@project.varnish-software.com>


commit c0cae3fc646b6be24b8f23de8fe9b8be8c360538
Author: Guillaume Quintard <guillaume.quintard at gmail.com>
Date:   Mon Mar 16 12:04:16 2015 +0100

    vef_priv can fit on 64 bytes

diff --git a/bin/varnishd/cache/cache_esi_fetch.c b/bin/varnishd/cache/cache_esi_fetch.c
index 558ee15..0d83b9f 100644
--- a/bin/varnishd/cache/cache_esi_fetch.c
+++ b/bin/varnishd/cache/cache_esi_fetch.c
@@ -44,13 +44,13 @@
 struct vef_priv {
 	unsigned		magic;
 #define VEF_MAGIC		0xf104b51f
+	int			error;
+	ssize_t			tot;
+
 	struct vgz		*vgz;
 
 	struct vep_state	*vep;
 
-	ssize_t			tot;
-	int			error;
-
 	char			*ibuf;
 	char			*ibuf_i;
 	char			*ibuf_o;



From lkarsten at varnish-software.com  Thu Jan 14 14:15:06 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:06 +0100
Subject: [4.1] f9662e8 ban_test can fit on 32 bytes
Message-ID: <E1aJig6-0005Bu-4m@project.varnish-software.com>


commit f9662e805b81a864dede130c41404db950274b9b
Author: Guillaume Quintard <guillaume.quintard at gmail.com>
Date:   Mon Mar 16 13:52:01 2015 +0100

    ban_test can fit on 32 bytes

diff --git a/bin/varnishd/cache/cache_ban.c b/bin/varnishd/cache/cache_ban.c
index ed9fcc2..2b71d94 100644
--- a/bin/varnishd/cache/cache_ban.c
+++ b/bin/varnishd/cache/cache_ban.c
@@ -50,9 +50,9 @@ static pthread_t ban_thread;
 static int ban_holds;
 
 struct ban_test {
+	uint8_t			oper;
 	uint8_t			arg1;
 	const char		*arg1_spec;
-	uint8_t			oper;
 	const char		*arg2;
 	const void		*arg2_spec;
 };



From lkarsten at varnish-software.com  Thu Jan 14 14:15:06 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:06 +0100
Subject: [4.1] b36a810 client can fit on 5 cache lines instead of 4
Message-ID: <E1aJig6-0005C8-6C@project.varnish-software.com>


commit b36a810f6e8f1703e1a18555820ba93aaa720976
Author: Guillaume Quintard <guillaume.quintard at gmail.com>
Date:   Mon Mar 16 14:14:13 2015 +0100

    client can fit on 5 cache lines instead of 4

diff --git a/bin/varnishtest/vtc_client.c b/bin/varnishtest/vtc_client.c
index 70e6cae..804ef6d 100644
--- a/bin/varnishtest/vtc_client.c
+++ b/bin/varnishtest/vtc_client.c
@@ -53,8 +53,8 @@ struct client {
 
 	unsigned		repeat;
 
-	pthread_t		tp;
 	unsigned		running;
+	pthread_t		tp;
 };
 
 static VTAILQ_HEAD(, client)	clients =



From lkarsten at varnish-software.com  Thu Jan 14 14:15:06 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:06 +0100
Subject: [4.1] 463feeb Fix typo
Message-ID: <E1aJig6-0005CV-7v@project.varnish-software.com>


commit 463feebc628a2ae67fbe62f226c03853bf01d28b
Author: Andrew Martin <sublimino at gmail.com>
Date:   Wed Nov 18 12:33:19 2015 +0000

    Fix typo
    
    `orderto` -> `order to` for docs at https://www.varnish-cache.org/docs/trunk/reference/varnishd.html

diff --git a/include/tbl/params.h b/include/tbl/params.h
index 1968df4..41f5b86 100644
--- a/include/tbl/params.h
+++ b/include/tbl/params.h
@@ -733,7 +733,7 @@ PARAM(
 	/* units */	allocations,
 	/* flags */	0| EXPERIMENTAL,
 	/* s-text */
-	"Maximum number of objects we attempt to nuke in orderto make "
+	"Maximum number of objects we attempt to nuke in order to make "
 	"space for a object body.\n",
 	/* l-text */	"",
 	/* func */	NULL



From lkarsten at varnish-software.com  Thu Jan 14 14:15:06 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:06 +0100
Subject: [4.1] bc3927b Minor test name correction
Message-ID: <E1aJig6-0005Cg-9d@project.varnish-software.com>


commit bc3927b74e9ef48c620373829d0eb7c66ff857fe
Author: Francisco Vel?zquez <francisco at varnish-software.com>
Date:   Thu Nov 19 16:46:55 2015 +0100

    Minor test name correction

diff --git a/bin/varnishtest/tests/b00020.vtc b/bin/varnishtest/tests/b00020.vtc
index 1174cd5..8427742 100644
--- a/bin/varnishtest/tests/b00020.vtc
+++ b/bin/varnishtest/tests/b00020.vtc
@@ -1,4 +1,4 @@
-varnishtest "Check the between_bytes_timeout behaves from parameters"
+varnishtest "Check that between_bytes_timeout behaves from parameters"
 
 feature SO_RCVTIMEO_WORKS
 



From lkarsten at varnish-software.com  Thu Jan 14 14:15:06 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:06 +0100
Subject: [4.1] 8b7cb51 Allow grace-hits on hit-for-pass objects,
 and treat the first one (which gets the busyobj) as a miss.
Message-ID: <E1aJig6-0005D2-BV@project.varnish-software.com>


commit 8b7cb51b76ad616143040ee955f1f6d9306251b9
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Nov 23 22:17:53 2015 +0000

    Allow grace-hits on hit-for-pass objects, and treat the first one
    (which gets the busyobj) as a miss.
    
    Testcase by:	daghf
    Fixes:	1818

diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index 3580204..f841c53 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -441,8 +441,7 @@ HSH_Lookup(struct req *req, struct objcore **ocp, struct objcore **bocp,
 			*ocp = oc;
 			return (HSH_HIT);
 		}
-		if (oc->exp.t_origin > exp_t_origin &&
-		    !(oc->flags & OC_F_PASS)) {
+		if (oc->exp.t_origin > exp_t_origin) {
 			/* record the newest object */
 			exp_oc = oc;
 			exp_t_origin = oc->exp.t_origin;
diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index da59078..f30ab59 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -403,16 +403,22 @@ cnt_lookup(struct worker *wrk, struct req *req)
 	AZ(oc->flags & OC_F_BUSY);
 	req->objcore = oc;
 
-	if (oc->flags & OC_F_PASS) {
+	if ((oc->flags & OC_F_PASS) && boc != NULL) {
+		/* Treat a graced Hit-For-Pass as a miss */
+		req->objcore = boc;
+		req->stale_oc = oc;
+		req->req_step = R_STP_MISS;
+		return (REQ_FSM_MORE);
+	} else if (oc->flags & OC_F_PASS) {
 		/* Found a hit-for-pass */
 		VSLb(req->vsl, SLT_Debug, "XXXX HIT-FOR-PASS");
 		VSLb(req->vsl, SLT_HitPass, "%u",
 		    ObjGetXID(wrk, req->objcore));
-		AZ(boc);
 		(void)HSH_DerefObjCore(wrk, &req->objcore);
 		wrk->stats->cache_hitpass++;
 		req->req_step = R_STP_PASS;
 		return (REQ_FSM_MORE);
+	} else if (oc->flags & OC_F_PASS) {
 	}
 
 	VSLb(req->vsl, SLT_Hit, "%u", ObjGetXID(wrk, req->objcore));
diff --git a/bin/varnishtest/tests/r01818.vtc b/bin/varnishtest/tests/r01818.vtc
new file mode 100644
index 0000000..8b38c9b
--- /dev/null
+++ b/bin/varnishtest/tests/r01818.vtc
@@ -0,0 +1,75 @@
+varnishtest "#1818: verify that grace works for hit_for_pass objects"
+
+server s1 {
+	rxreq
+	expect req.http.a == "1"
+	txresp
+
+	rxreq
+	expect req.http.b == "1"
+	sema r2 sync 2
+	sema r1 sync 2
+ 	txresp
+} -start
+
+server s2 {
+	rxreq
+	expect req.http.c == "1"
+	sema r1 sync 2
+	txresp
+} -start
+
+varnish v1 -vcl+backend {
+	sub vcl_recv {
+		if (req.http.c) {
+			set req.backend_hint = s2;
+		}
+	}
+
+	sub vcl_miss {
+		set req.http.miss = "1";
+	}
+
+	sub vcl_pass {
+		set req.http.pass = "1";
+	}
+
+	sub vcl_deliver {
+		if (req.http.miss) {
+			set resp.http.miss = req.http.miss;
+		}
+		if (req.http.pass) {
+			set resp.http.pass = req.http.pass;
+		}
+	}
+
+	sub vcl_backend_response {
+		set beresp.ttl = 0.1s;
+		set beresp.grace = 1m;
+		set beresp.uncacheable = true;
+	}
+} -start 
+
+varnish v1 -cliok "param.set debug +syncvsl"
+varnish v1 -cliok "param.set debug +waitinglist"
+varnish v1 -cliok "param.show debug"
+
+client c1 {
+	# This is a plain miss
+	txreq -hdr "a: 1"
+	rxresp
+	expect resp.http.miss == "1"
+	delay .2
+
+	# This should also miss, because the HFP is expired
+	txreq -hdr "b: 1"
+	rxresp
+	expect resp.http.miss == "1"
+} -start
+
+client c2 {
+	sema r2 sync 2
+	txreq -hdr "c: 1"
+	rxresp
+	expect resp.http.pass == "1"
+} -run



From lkarsten at varnish-software.com  Thu Jan 14 14:15:06 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:06 +0100
Subject: [4.1] ebb4ce3 Emit ULL constants for unsigned long long type.
Message-ID: <E1aJig6-0005DD-D9@project.varnish-software.com>


commit ebb4ce30f3096e021ad3cee377bd542cb22f5cf6
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Nov 24 10:50:05 2015 +0000

    Emit ULL constants for unsigned long long type.

diff --git a/lib/libvcc/vcc_utils.c b/lib/libvcc/vcc_utils.c
index 6b56959..40f0f86 100644
--- a/lib/libvcc/vcc_utils.c
+++ b/lib/libvcc/vcc_utils.c
@@ -118,7 +118,7 @@ vcc_suckaddr(struct vcc *tl, const char *host, const struct suckaddr *vsa,
 	Fh(tl, 0, " suckaddr_%u[%d] = {\n", tl->unique, n);
 	memcpy(b, vsa, vsa_suckaddr_len);
 	for (len = 0; len < n; len++)
-		Fh(tl, 0, "%s    0x%0*llxLL",
+		Fh(tl, 0, "%s    0x%0*llxULL",
 		    len ? ",\n" : "", sz * 2, b[len]);
 	Fh(tl, 0, "\n};\n");
 



From lkarsten at varnish-software.com  Thu Jan 14 14:15:06 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:06 +0100
Subject: [4.1] 87a37c5 White-space OCD
Message-ID: <E1aJig6-0005EC-F7@project.varnish-software.com>


commit 87a37c5aba25c1b82f990ed7961629f443fc7552
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Nov 24 10:50:42 2015 +0000

    White-space OCD

diff --git a/bin/varnishtest/tests/r01818.vtc b/bin/varnishtest/tests/r01818.vtc
index 8b38c9b..ee887a8 100644
--- a/bin/varnishtest/tests/r01818.vtc
+++ b/bin/varnishtest/tests/r01818.vtc
@@ -9,7 +9,7 @@ server s1 {
 	expect req.http.b == "1"
 	sema r2 sync 2
 	sema r1 sync 2
- 	txresp
+	txresp
 } -start
 
 server s2 {
@@ -48,7 +48,7 @@ varnish v1 -vcl+backend {
 		set beresp.grace = 1m;
 		set beresp.uncacheable = true;
 	}
-} -start 
+} -start
 
 varnish v1 -cliok "param.set debug +syncvsl"
 varnish v1 -cliok "param.set debug +waitinglist"



From lkarsten at varnish-software.com  Thu Jan 14 14:15:06 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:06 +0100
Subject: [4.1] b55bff1 Allow inspection of remote TCP address in HTTP (both
 client and server) context.
Message-ID: <E1aJig6-0005EQ-Gy@project.varnish-software.com>


commit b55bff1cbfa82102d5586a551e1e0dceb128e3cb
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Nov 24 10:55:44 2015 +0000

    Allow inspection of remote TCP address in HTTP (both client and server)
    context.
    
    Cherry-picked and modified from patch (pull#71) by:	lochiiconnectivity

diff --git a/bin/varnishtest/vtc_http.c b/bin/varnishtest/vtc_http.c
index 64ecef5..0440542 100644
--- a/bin/varnishtest/vtc_http.c
+++ b/bin/varnishtest/vtc_http.c
@@ -61,6 +61,8 @@ struct http {
 
 	int			nrxbuf;
 	char			*rxbuf;
+	char			*rem_ip;
+	char			*rem_port;
 	int			prxbuf;
 	char			*body;
 	unsigned		bodyl;
@@ -197,7 +199,10 @@ static const char *
 cmd_var_resolve(struct http *hp, char *spec)
 {
 	char **hh, *hdr;
-
+	if (!strcmp(spec, "remote.ip"))
+		return(hp->rem_ip);
+	if (!strcmp(spec, "remote.port"))
+		return(hp->rem_port);
 	if (!strcmp(spec, "req.method"))
 		return(hp->req[0]);
 	if (!strcmp(spec, "req.url"))
@@ -1369,20 +1374,33 @@ http_process(struct vtclog *vl, const char *spec, int sock, int *sfd)
 	AN(hp);
 	hp->fd = sock;
 	hp->timeout = vtc_maxdur * 1000 / 2;
+
 	hp->nrxbuf = 2048*1024;
-	hp->vsb = VSB_new_auto();
 	hp->rxbuf = malloc(hp->nrxbuf);		/* XXX */
+	AN(hp->rxbuf);
+
+	hp->vsb = VSB_new_auto();
+	AN(hp->vsb);
+
 	hp->sfd = sfd;
+
+	hp->rem_ip = malloc(VTCP_ADDRBUFSIZE);
+	AN(hp->rem_ip);
+
+	hp->rem_port = malloc(VTCP_PORTBUFSIZE);
+	AN(hp->rem_port);
+
 	hp->vl = vl;
 	hp->gziplevel = 0;
 	hp->gzipresidual = -1;
-	AN(hp->rxbuf);
-	AN(hp->vsb);
 
+	VTCP_hisname(sock, hp->rem_ip, VTCP_ADDRBUFSIZE, hp->rem_port, VTCP_PORTBUFSIZE);
 	parse_string(spec, http_cmds, hp, vl);
 	retval = hp->fd;
 	VSB_delete(hp->vsb);
 	free(hp->rxbuf);
+	free(hp->rem_ip);
+	free(hp->rem_port);
 	free(hp);
 	return (retval);
 }



From lkarsten at varnish-software.com  Thu Jan 14 14:15:06 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:06 +0100
Subject: [4.1] e76461b Don't quote NL in panic messages
Message-ID: <E1aJig6-0005El-Iq@project.varnish-software.com>


commit e76461b7055cf2bcdc46ddcd3ee3d1a35195b12e
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Nov 24 14:12:46 2015 +0000

    Don't quote NL in panic messages

diff --git a/bin/varnishd/mgt/mgt_child.c b/bin/varnishd/mgt/mgt_child.c
index 368d359..11578ce 100644
--- a/bin/varnishd/mgt/mgt_child.c
+++ b/bin/varnishd/mgt/mgt_child.c
@@ -117,7 +117,8 @@ mgt_panic_record(pid_t r)
 	VTIM_format(VTIM_real(), time_str);
 	VSB_printf(child_panic, "Last panic at: %s\n", time_str);
 	VSB_quote(child_panic, heritage.panic_str,
-	    strnlen(heritage.panic_str, heritage.panic_str_len), 0);
+	    strnlen(heritage.panic_str, heritage.panic_str_len),
+	    VSB_QUOTE_NONL);
 	AZ(VSB_finish(child_panic));
 	MGT_complain(C_ERR, "Child (%jd) %s",
 	    (intmax_t)r, VSB_data(child_panic));
diff --git a/include/vsb.h b/include/vsb.h
index eefaa12..61b64c0 100644
--- a/include/vsb.h
+++ b/include/vsb.h
@@ -75,6 +75,7 @@ int		 VSB_finish(struct vsb *);
 char		*VSB_data(const struct vsb *);
 ssize_t		 VSB_len(const struct vsb *);
 void		 VSB_delete(struct vsb *);
+#define VSB_QUOTE_NONL	1
 void		 VSB_quote(struct vsb *s, const char *p, int len, int how);
 void		 VSB_indent(struct vsb *, int);
 #ifdef __cplusplus
diff --git a/lib/libvarnish/vsb.c b/lib/libvarnish/vsb.c
index 63c196d..92abdf2 100644
--- a/lib/libvarnish/vsb.c
+++ b/lib/libvarnish/vsb.c
@@ -519,7 +519,10 @@ VSB_quote(struct vsb *s, const char *p, int len, int how)
 			(void)VSB_putc(s, *q);
 			break;
 		case '\n':
-			(void)VSB_cat(s, "\\n");
+			if (how & VSB_QUOTE_NONL)
+				(void)VSB_cat(s, "\n");
+			else
+				(void)VSB_cat(s, "\\n");
 			break;
 		case '\r':
 			(void)VSB_cat(s, "\\r");



From lkarsten at varnish-software.com  Thu Jan 14 14:15:06 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:06 +0100
Subject: [4.1] 86c5739 Always slim private & pass objects after delivery.
Message-ID: <E1aJig6-0005Ey-LP@project.varnish-software.com>


commit 86c57392b325ec0aa616d2c5c2b07d2375c518fb
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Nov 24 14:28:59 2015 +0000

    Always slim private & pass objects after delivery.
    
    Fixes:	#1821

diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index f30ab59..c493e02 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -209,19 +209,20 @@ cnt_deliver(struct worker *wrk, struct req *req)
 
 	cnt_vdp(req, bo);
 
-	if (bo != NULL)
-		VBO_DerefBusyObj(wrk, &bo);
-
 	VSLb_ts_req(req, "Resp", W_TIM_real(wrk));
 
 	if (http_HdrIs(req->resp, H_Connection, "close"))
 		req->doclose = SC_RESP_CLOSE;
 
-	if ((req->objcore->flags & OC_F_PASS) && bo != NULL) {
-		VBO_waitstate(bo, BOS_FINISHED);
+	if (req->objcore->flags & (OC_F_PRIVATE | OC_F_PASS)) {
+		if (bo != NULL)
+			VBO_waitstate(bo, BOS_FINISHED);
 		ObjSlim(wrk, req->objcore);
 	}
 
+	if (bo != NULL)
+		VBO_DerefBusyObj(wrk, &bo);
+
 	(void)HSH_DerefObjCore(wrk, &req->objcore);
 	http_Teardown(req->resp);
 	return (REQ_FSM_DONE);
diff --git a/bin/varnishtest/tests/r01821.vtc b/bin/varnishtest/tests/r01821.vtc
new file mode 100644
index 0000000..1c8fa87
--- /dev/null
+++ b/bin/varnishtest/tests/r01821.vtc
@@ -0,0 +1,28 @@
+varnishtest	"Slim down hit-for-pass objects"
+
+server s1 -repeat 2 {
+	rxreq
+	txresp -hdr "Cache-Control: max-age=5" -bodylen 65535
+} -start
+
+varnish v1 -vcl+backend {
+	sub vcl_backend_response {
+		set beresp.uncacheable = true;
+	}
+} -start
+
+logexpect l1 -v v1 {
+	expect * * Storage Transient
+} -start
+
+client c1 {
+	txreq -url /foo
+	rxresp
+	txreq -url /bar
+	rxresp
+} -run
+
+logexpect l1 -wait
+
+varnish v1 -expect SMA.Transient.c_bytes != 0
+varnish v1 -expect SMA.Transient.g_bytes < 65536



From lkarsten at varnish-software.com  Thu Jan 14 14:15:06 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:06 +0100
Subject: [4.1] 8068d72 Keep Makefile.phk working
Message-ID: <E1aJig6-0005FK-N5@project.varnish-software.com>


commit 8068d724155741144999e5a5bc45eccae39dc1af
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Nov 25 10:59:20 2015 +0000

    Keep Makefile.phk working

diff --git a/bin/varnishd/Makefile.phk b/bin/varnishd/Makefile.phk
index e849a28..fb9df6c 100644
--- a/bin/varnishd/Makefile.phk
+++ b/bin/varnishd/Makefile.phk
@@ -6,6 +6,8 @@ PROG_SRC += cache/cache_backend_cfg.c
 PROG_SRC += cache/cache_backend_tcp.c
 PROG_SRC += cache/cache_backend_probe.c
 PROG_SRC += cache/cache_ban.c
+PROG_SRC += cache/cache_ban_build.c
+PROG_SRC += cache/cache_ban_lurker.c
 PROG_SRC += cache/cache_busyobj.c
 PROG_SRC += cache/cache_cli.c
 PROG_SRC += cache/cache_director.c
@@ -110,11 +112,21 @@ MADE_FILES += builtin_vcl.h
 
 TODO_INSTALL += install_etc
 
+PARST=$(TOPDIR)/doc/sphinx/include/params.rst
+
+TODO_ALL += $(PARST)
+CLEAN_FILES += $(PARST)
+
 CFLAGS	+= -DVARNISHD_IS_NOT_A_VMOD
 
 TOPDIR  = $(CURDIR)/../..
 include $(TOPDIR)/Makefile.inc.phk
 
+$(PARST): $(PROGNAME)
+	-ls -l $(PARST) $(PROGNAME)
+	mkdir -p $(TOPDIR)/doc/sphinx/include
+	./varnishd -x dumprstparam > $(PARST)
+
 #
 # Turn the builtin.vcl file into a C-string we can include in the program.
 #
diff --git a/bin/varnishncsa/varnishncsa.c b/bin/varnishncsa/varnishncsa.c
index 95c4454..dfb4885 100644
--- a/bin/varnishncsa/varnishncsa.c
+++ b/bin/varnishncsa/varnishncsa.c
@@ -41,6 +41,8 @@
 
 #include "config.h"
 
+#define _WITH_GETLINE
+
 #include <stdlib.h>
 #include <stdio.h>
 #include <unistd.h>
diff --git a/config.phk b/config.phk
index 906cbff..dea0eed 100644
--- a/config.phk
+++ b/config.phk
@@ -93,7 +93,6 @@ if true ; then
     #define VARNISH_VCL_DIR         "/etc/varnish"
 		#define VARNISH_STATE_DIR       "/tmp/phk/"
 		#define VARNISH_VMOD_DIR        "/usr/lib/varnish/vmods"
-		#define VCS_Branch "master"
 	' >> ${CONFIG_H}_
 	if [ ! -f ${CONFIG_H} ] ; then
 		mv ${CONFIG_H}_ ${CONFIG_H}



From lkarsten at varnish-software.com  Thu Jan 14 14:15:06 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:06 +0100
Subject: [4.1] 0fca187 Use private objcore's for req.body to reduce number of
 special cases.
Message-ID: <E1aJig6-0005FT-Oz@project.varnish-software.com>


commit 0fca187e8300b1bddc9baf2e0dd855156a8eadd5
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Nov 25 23:57:36 2015 +0000

    Use private objcore's for req.body to reduce number of special cases.

diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index 662048d..b084680 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -964,6 +964,7 @@ VBF_Fetch(struct worker *wrk, struct req *req, struct objcore *oc,
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
 	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
+	AN(oc->flags & OC_F_BUSY);
 	CHECK_OBJ_ORNULL(oldoc, OBJCORE_MAGIC);
 
 
diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index f841c53..d93320d 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -67,8 +67,8 @@ static struct objhead *private_oh;
 
 /*---------------------------------------------------------------------*/
 
-struct objcore *
-HSH_NewObjCore(struct worker *wrk)
+static struct objcore *
+hsh_NewObjCore(struct worker *wrk)
 {
 	struct objcore *oc;
 
@@ -104,7 +104,7 @@ hsh_prealloc(struct worker *wrk)
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 
 	if (wrk->nobjcore == NULL)
-		wrk->nobjcore = HSH_NewObjCore(wrk);
+		wrk->nobjcore = hsh_NewObjCore(wrk);
 	CHECK_OBJ_NOTNULL(wrk->nobjcore, OBJCORE_MAGIC);
 
 	if (wrk->nobjhead == NULL) {
@@ -135,11 +135,11 @@ HSH_Private(struct worker *wrk)
 
 	CHECK_OBJ_NOTNULL(private_oh, OBJHEAD_MAGIC);
 
-	oc = HSH_NewObjCore(wrk);
+	oc = hsh_NewObjCore(wrk);
 	AN(oc);
 	oc->refcnt = 1;
 	oc->objhead = private_oh;
-	oc->flags |= OC_F_PRIVATE;
+	oc->flags |= OC_F_PRIVATE | OC_F_BUSY;
 	Lck_Lock(&private_oh->mtx);
 	VTAILQ_INSERT_TAIL(&private_oh->objcs, oc, list);
 	private_oh->refcnt++;
diff --git a/bin/varnishd/cache/cache_obj.c b/bin/varnishd/cache/cache_obj.c
index 7413329..06785d3 100644
--- a/bin/varnishd/cache/cache_obj.c
+++ b/bin/varnishd/cache/cache_obj.c
@@ -288,6 +288,7 @@ ObjGetSpace(struct worker *wrk, struct objcore *oc, ssize_t *sz, uint8_t **ptr)
 		VTAILQ_INSERT_TAIL(&o->list, st, list);
 		Lck_Unlock(&oc->busyobj->mtx);
 	} else {
+		AN(oc->flags & (OC_F_PRIVATE));
 		VTAILQ_INSERT_TAIL(&o->list, st, list);
 	}
 	*sz = st->space - st->len;
diff --git a/bin/varnishd/cache/cache_req_body.c b/bin/varnishd/cache/cache_req_body.c
index 585a63e..ccd7a6a 100644
--- a/bin/varnishd/cache/cache_req_body.c
+++ b/bin/varnishd/cache/cache_req_body.c
@@ -178,11 +178,8 @@ VRB_Free(struct req *req)
 
 	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
 
-	if (req->body_oc != NULL) {
-		ObjFreeObj(req->wrk, req->body_oc);
-		FREE_OBJ(req->body_oc);
-		req->body_oc = NULL;
-	}
+	if (req->body_oc != NULL)
+		AZ(HSH_DerefObjCore(req->wrk, &req->body_oc));
 }
 
 /*----------------------------------------------------------------------
@@ -228,7 +225,7 @@ VRB_Cache(struct req *req, ssize_t maxsize)
 		return (-1);
 	}
 
-	req->body_oc = HSH_NewObjCore(req->wrk);
+	req->body_oc = HSH_Private(req->wrk);
 	AN(req->body_oc);
 	XXXAN(STV_NewObject(req->body_oc, req->wrk, TRANSIENT_STORAGE, 8));
 
diff --git a/bin/varnishd/hash/hash_slinger.h b/bin/varnishd/hash/hash_slinger.h
index f8d7a08..736f1b5 100644
--- a/bin/varnishd/hash/hash_slinger.h
+++ b/bin/varnishd/hash/hash_slinger.h
@@ -74,7 +74,6 @@ void HSH_Purge(struct worker *, struct objhead *, double ttl, double grace,
 void HSH_config(const char *h_arg);
 struct busyobj *HSH_RefBusy(const struct objcore *oc);
 struct objcore *HSH_Private(struct worker *wrk);
-struct objcore *HSH_NewObjCore(struct worker *wrk);
 
 #ifdef VARNISH_CACHE_CHILD
 



From lkarsten at varnish-software.com  Thu Jan 14 14:15:06 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:06 +0100
Subject: [4.1] 232b04b Try to be a bit systematic about the asserts
Message-ID: <E1aJig6-0005Ff-QO@project.varnish-software.com>


commit 232b04b54e7c0b0737ef68224ae1213e5aa1b02d
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu Nov 26 00:27:03 2015 +0000

    Try to be a bit systematic about the asserts

diff --git a/bin/varnishd/cache/cache_obj.c b/bin/varnishd/cache/cache_obj.c
index 06785d3..02d5fdc 100644
--- a/bin/varnishd/cache/cache_obj.c
+++ b/bin/varnishd/cache/cache_obj.c
@@ -102,6 +102,8 @@ ObjIterBegin(struct worker *wrk, struct objcore *oc)
 	struct object *obj;
 	const struct storeobj_methods *om = obj_getmethods(oc);
 
+	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
+
 	if (om->objiterbegin != NULL)
 		return (om->objiterbegin(wrk, oc));
 
@@ -127,13 +129,15 @@ ObjIter(struct objcore *oc, void *oix, void **p, ssize_t *l)
 	ssize_t nl;
 	const struct storeobj_methods *om = obj_getmethods(oc);
 
+	AN(oix);
+	AN(p);
+	AN(l);
+
 	if (om->objiter != NULL)
 		return (om->objiter(oc, oix, p, l));
 
 	CAST_OBJ_NOTNULL(oi, oix, OBJITER_MAGIC);
 	CHECK_OBJ_NOTNULL(oi->obj, OBJECT_MAGIC);
-	AN(p);
-	AN(l);
 	*p = NULL;
 	*l = 0;
 
@@ -193,13 +197,13 @@ ObjIterEnd(struct objcore *oc, void **oix)
 	struct objiter *oi;
 	const struct storeobj_methods *om = obj_getmethods(oc);
 
+	AN(oix);
+
 	if (om->objiterend != NULL) {
 		om->objiterend(oc, oix);
 		return;
 	}
 
-	AN(oc);
-	AN(oix);
 	CAST_OBJ_NOTNULL(oi, (*oix), OBJITER_MAGIC);
 	*oix = NULL;
 	CHECK_OBJ_NOTNULL(oi->obj, OBJECT_MAGIC);
@@ -215,11 +219,12 @@ ObjIterEnd(struct objcore *oc, void **oix)
  */
 
 static struct storage *
-objallocwithnuke(const struct stevedore *stv, struct worker *wrk, size_t size)
+objallocwithnuke(struct worker *wrk, const struct stevedore *stv, size_t size)
 {
 	struct storage *st = NULL;
 	unsigned fail;
 
+	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(stv, STEVEDORE_MAGIC);
 
 	if (size > cache_param->fetch_maxchunksize)
@@ -259,14 +264,14 @@ ObjGetSpace(struct worker *wrk, struct objcore *oc, ssize_t *sz, uint8_t **ptr)
 	struct storage *st;
 	const struct storeobj_methods *om = obj_getmethods(oc);
 
-	if (om->objgetspace != NULL)
-		return (om->objgetspace(wrk, oc, sz, ptr));
-
-	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	AN(sz);
 	AN(ptr);
 	assert(*sz > 0);
+
+	if (om->objgetspace != NULL)
+		return (om->objgetspace(wrk, oc, sz, ptr));
+
 	o = obj_getobj(wrk, oc);
 	CHECK_OBJ_NOTNULL(o, OBJECT_MAGIC);
 
@@ -278,7 +283,7 @@ ObjGetSpace(struct worker *wrk, struct objcore *oc, ssize_t *sz, uint8_t **ptr)
 		return (1);
 	}
 
-	st = objallocwithnuke(oc->stobj->stevedore, wrk, *sz);
+	st = objallocwithnuke(wrk, oc->stobj->stevedore, *sz);
 	if (st == NULL)
 		return (0);
 
@@ -311,13 +316,14 @@ ObjExtend(struct worker *wrk, struct objcore *oc, ssize_t l)
 	struct storage *st;
 	const struct storeobj_methods *om = obj_getmethods(oc);
 
+	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
+	assert(l > 0);
+
 	if (om->objextend != NULL) {
 		om->objextend(wrk, oc, l);
 		return;
 	}
 
-	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
-	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
 	o = obj_getobj(wrk, oc);
 	CHECK_OBJ_NOTNULL(o, OBJECT_MAGIC);
 	st = VTAILQ_LAST(&o->list, storagehead);
@@ -339,6 +345,8 @@ ObjGetLen(struct worker *wrk, struct objcore *oc)
 	struct object *o;
 	const struct storeobj_methods *om = obj_getmethods(oc);
 
+	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
+
 	if (om->objgetlen != NULL)
 		return (om->objgetlen(wrk, oc));
 
@@ -363,13 +371,13 @@ ObjTrimStore(struct worker *wrk, struct objcore *oc)
 	struct object *o;
 	const struct storeobj_methods *om = obj_getmethods(oc);
 
+	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
+
 	if (om->objtrimstore != NULL) {
 		om->objtrimstore(wrk, oc);
 		return;
 	}
 
-	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
-	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	stv = oc->stobj->stevedore;
 	CHECK_OBJ_NOTNULL(stv, STEVEDORE_MAGIC);
 	o = obj_getobj(wrk, oc);
@@ -400,13 +408,13 @@ ObjSlim(struct worker *wrk, struct objcore *oc)
 	struct storage *st, *stn;
 	const struct storeobj_methods *om = obj_getmethods(oc);
 
+	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
+
 	if (om->objslim != NULL) {
 		om->objslim(wrk, oc);
 		return;
 	}
 
-	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
-	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
 	stv = oc->stobj->stevedore;
 	CHECK_OBJ_NOTNULL(stv, STEVEDORE_MAGIC);
 	o = obj_getobj(wrk, oc);
@@ -430,6 +438,8 @@ ObjUpdateMeta(struct worker *wrk, struct objcore *oc)
 {
 	const struct storeobj_methods *m = obj_getmethods(oc);
 
+	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
+
 	if (m->updatemeta != NULL)
 		m->updatemeta(wrk, oc);
 }
@@ -442,7 +452,7 @@ ObjFreeObj(struct worker *wrk, struct objcore *oc)
 	const struct storeobj_methods *m = obj_getmethods(oc);
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
-	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
+
 	AN(m->freeobj);
 	m->freeobj(wrk, oc);
 	AZ(oc->stobj->stevedore);
@@ -455,7 +465,6 @@ ObjGetLRU(const struct objcore *oc)
 {
 	const struct storeobj_methods *m = obj_getmethods(oc);
 
-	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
 	AN(m->getlru);
 	return (m->getlru(oc));
 }
@@ -474,11 +483,11 @@ ObjGetattr(struct worker *wrk, struct objcore *oc, enum obj_attr attr,
 	ssize_t dummy;
 	const struct storeobj_methods *om = obj_getmethods(oc);
 
+	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
+
 	if (om->objgetattr != NULL)
 		return (om->objgetattr(wrk, oc, attr, len));
 
-	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
-	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
 	if (len == NULL)
 		len = &dummy;
 	o = obj_getobj(wrk, oc);
@@ -529,17 +538,17 @@ ObjSetattr(struct worker *wrk, struct objcore *oc, enum obj_attr attr,
 	struct storage *st;
 	const struct storeobj_methods *om = obj_getmethods(oc);
 
+	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
+
 	if (om->objsetattr != NULL)
 		return (om->objsetattr(wrk, oc, attr, len, ptr));
 
-	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
-	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
 	o = obj_getobj(wrk, oc);
 	CHECK_OBJ_NOTNULL(o, OBJECT_MAGIC);
 	st = o->objstore;
 	switch (attr) {
 	case OA_ESIDATA:
-		o->esidata = objallocwithnuke(oc->stobj->stevedore, wrk, len);
+		o->esidata = objallocwithnuke(wrk, oc->stobj->stevedore, len);
 		if (o->esidata == NULL)
 			return (NULL);
 		o->esidata->len = len;
@@ -597,6 +606,7 @@ ObjCopyAttr(struct worker *wrk, struct objcore *oc, struct objcore *ocs,
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
+	CHECK_OBJ_NOTNULL(ocs, OBJCORE_MAGIC);
 
 	vps = ObjGetattr(wrk, ocs, attr, &l);
 	// XXX: later we want to have zero-length OA's too



From lkarsten at varnish-software.com  Thu Jan 14 14:15:06 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:06 +0100
Subject: [4.1] 9f6dd2b Various nitpicking
Message-ID: <E1aJig6-0005Fp-S8@project.varnish-software.com>


commit 9f6dd2b15353bb8eee8c9c58029475e3cd5a1fce
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu Nov 26 08:13:41 2015 +0000

    Various nitpicking

diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index b084680..5d1d1ab 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -997,6 +997,7 @@ VBF_Fetch(struct worker *wrk, struct req *req, struct objcore *oc,
 
 	if (mode != VBF_BACKGROUND)
 		HSH_Ref(oc);
+	AZ(bo->fetch_objcore);
 	bo->fetch_objcore = oc;
 
 	AZ(bo->stale_oc);
diff --git a/bin/varnishd/cache/cache_obj.c b/bin/varnishd/cache/cache_obj.c
index 02d5fdc..54dd26f 100644
--- a/bin/varnishd/cache/cache_obj.c
+++ b/bin/varnishd/cache/cache_obj.c
@@ -116,8 +116,7 @@ ObjIterBegin(struct worker *wrk, struct objcore *oc)
 	oi->oc = oc;
 	oi->obj = obj;
 	oi->wrk = wrk;
-	if (oc->objhead != NULL)
-		oi->bo = HSH_RefBusy(oc);
+	oi->bo = HSH_RefBusy(oc);
 	return (oi);
 }
 
diff --git a/bin/varnishd/cache/cache_req_body.c b/bin/varnishd/cache/cache_req_body.c
index ccd7a6a..f50621e 100644
--- a/bin/varnishd/cache/cache_req_body.c
+++ b/bin/varnishd/cache/cache_req_body.c
@@ -268,6 +268,12 @@ VRB_Cache(struct req *req, ssize_t maxsize)
 
 	} while (vfps == VFP_OK);
 	VFP_Close(vfc);
+	ObjTrimStore(req->wrk, req->body_oc);
+
+	/* XXX: check missing:
+	    if (req->htc->content_length >= 0)
+		MUSTBE (req->req_bodybytes == req->htc->content_length);
+	*/
 
 	if (vfps == VFP_END) {
 		assert(req->req_bodybytes >= 0);
diff --git a/bin/varnishd/mgt/mgt_acceptor.c b/bin/varnishd/mgt/mgt_acceptor.c
index 8edfd1c..3cf1bbc 100644
--- a/bin/varnishd/mgt/mgt_acceptor.c
+++ b/bin/varnishd/mgt/mgt_acceptor.c
@@ -129,11 +129,9 @@ mac_callback(void *priv, const struct suckaddr *sa)
 	CAST_OBJ_NOTNULL(mh, priv, MAC_HELP_MAGIC);
 
 	VTAILQ_FOREACH(ls, &heritage.socks, list) {
-		if (!VSA_Compare(sa, ls->addr)) {
+		if (!VSA_Compare(sa, ls->addr))
 			ARGV_ERR("-a arguments %s and %s have same address\n",
 			    ls->name, mh->name);
-			return (-1);
-		}
 	}
 	ALLOC_OBJ(ls, LISTEN_SOCK_MAGIC);
 	AN(ls);



From lkarsten at varnish-software.com  Thu Jan 14 14:15:06 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:06 +0100
Subject: [4.1] a6bf9db Cache a checkpoint when we iterate over busy objects,
 so we don't have to walk the full list all the time.
Message-ID: <E1aJig6-0005Fy-UW@project.varnish-software.com>


commit a6bf9db091e8c039ecdc191a6916c72ccc5315ed
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu Nov 26 08:39:30 2015 +0000

    Cache a checkpoint when we iterate over busy objects, so we don't
    have to walk the full list all the time.
    
    This is a minimal fix for backporting to 4.1.x
    
    Fixes:	#1798
    Fixes:	#1788

diff --git a/bin/varnishd/cache/cache_obj.c b/bin/varnishd/cache/cache_obj.c
index 54dd26f..53fe080 100644
--- a/bin/varnishd/cache/cache_obj.c
+++ b/bin/varnishd/cache/cache_obj.c
@@ -93,6 +93,8 @@ struct objiter {
 	struct storage			*st;
 	struct worker			*wrk;
 	ssize_t				len;
+	struct storage			*checkpoint;
+	ssize_t				checkpoint_len;
 };
 
 void *
@@ -126,6 +128,7 @@ ObjIter(struct objcore *oc, void *oix, void **p, ssize_t *l)
 	struct objiter *oi;
 	ssize_t ol;
 	ssize_t nl;
+	ssize_t sl;
 	const struct storeobj_methods *om = obj_getmethods(oc);
 
 	AN(oix);
@@ -167,7 +170,15 @@ ObjIter(struct objcore *oc, void *oix, void **p, ssize_t *l)
 		}
 		Lck_Lock(&oi->bo->mtx);
 		AZ(VTAILQ_EMPTY(&oi->obj->list));
-		VTAILQ_FOREACH(oi->st, &oi->obj->list, list) {
+		if (oi->checkpoint == NULL) {
+			oi->st = VTAILQ_FIRST(&oi->obj->list);
+			sl = 0;
+		} else {
+			oi->st = oi->checkpoint;
+			sl = oi->checkpoint_len;
+			ol -= oi->checkpoint_len;
+		}
+		while (oi->st != NULL) {
 			if (oi->st->len > ol) {
 				*p = oi->st->ptr + ol;
 				*l = oi->st->len - ol;
@@ -178,6 +189,12 @@ ObjIter(struct objcore *oc, void *oix, void **p, ssize_t *l)
 			assert(ol >= 0);
 			nl -= oi->st->len;
 			assert(nl > 0);
+			sl += oi->st->len;
+			oi->st = VTAILQ_NEXT(oi->st, list);
+			if (VTAILQ_NEXT(oi->st, list) != NULL) {
+				oi->checkpoint = oi->st;
+				oi->checkpoint_len = sl;
+			}
 		}
 		CHECK_OBJ_NOTNULL(oi->obj, OBJECT_MAGIC);
 		CHECK_OBJ_NOTNULL(oi->st, STORAGE_MAGIC);



From lkarsten at varnish-software.com  Thu Jan 14 14:15:06 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:06 +0100
Subject: [4.1] cb1e0b5 Simplify
Message-ID: <E1aJig6-0005G7-W2@project.varnish-software.com>


commit cb1e0b5a1891a3a8209e2b19d58b6cf572de44e4
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Tue Nov 24 14:56:45 2015 +0000

    Simplify

diff --git a/bin/varnishtest/tests/r01821.vtc b/bin/varnishtest/tests/r01821.vtc
index 1c8fa87..aac4020 100644
--- a/bin/varnishtest/tests/r01821.vtc
+++ b/bin/varnishtest/tests/r01821.vtc
@@ -1,8 +1,8 @@
-varnishtest	"Slim down hit-for-pass objects"
+varnishtest "Slim down hit-for-pass objects"
 
 server s1 -repeat 2 {
 	rxreq
-	txresp -hdr "Cache-Control: max-age=5" -bodylen 65535
+	txresp -bodylen 65535
 } -start
 
 varnish v1 -vcl+backend {
@@ -12,13 +12,11 @@ varnish v1 -vcl+backend {
 } -start
 
 logexpect l1 -v v1 {
-	expect * * Storage Transient
+	expect * *	Storage		"Transient"
 } -start
 
 client c1 {
-	txreq -url /foo
-	rxresp
-	txreq -url /bar
+	txreq
 	rxresp
 } -run
 



From lkarsten at varnish-software.com  Thu Jan 14 14:15:07 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:07 +0100
Subject: [4.1] bd10b8b Use req as intended
Message-ID: <E1aJig7-0005GG-1k@project.varnish-software.com>


commit bd10b8bc25c97e50859f861ef4d069d3f220a38f
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Fri Nov 27 16:39:36 2015 +0000

    Use req as intended

diff --git a/doc/sphinx/phk/gzip.rst b/doc/sphinx/phk/gzip.rst
index ae244d9..c5e4aad 100644
--- a/doc/sphinx/phk/gzip.rst
+++ b/doc/sphinx/phk/gzip.rst
@@ -151,7 +151,7 @@ compression efficiency, you should::
 
 	sub vcl_miss {
 		if (object needs ESI processing) {
-			unset bereq.http.accept-encoding;
+			unset req.http.accept-encoding;
 		}
 	}
 
diff --git a/doc/sphinx/reference/vmod.rst b/doc/sphinx/reference/vmod.rst
index 91953b0..0d30162 100644
--- a/doc/sphinx/reference/vmod.rst
+++ b/doc/sphinx/reference/vmod.rst
@@ -231,7 +231,7 @@ STRING_LIST
 
 	Consider this contrived example::
 
-		set bereq.http.foo = std.toupper(req.http.foo + req.http.bar);
+		set req.http.foo = std.toupper(req.http.foo + req.http.bar);
 
 	The usual way to do this, would be be to allocate memory for
 	the concatenated string, then pass that to ``toupper()`` which in
diff --git a/doc/sphinx/users-guide/devicedetection.rst b/doc/sphinx/users-guide/devicedetection.rst
index 9138816..67f711a 100644
--- a/doc/sphinx/users-guide/devicedetection.rst
+++ b/doc/sphinx/users-guide/devicedetection.rst
@@ -127,8 +127,8 @@ VCL::
     }
 
     # override the header before it is sent to the backend
-    sub vcl_miss { if (req.http.X-UA-Device) { set bereq.http.User-Agent = req.http.X-UA-Device; } }
-    sub vcl_pass { if (req.http.X-UA-Device) { set bereq.http.User-Agent = req.http.X-UA-Device; } }
+    sub vcl_miss { if (req.http.X-UA-Device) { set req.http.User-Agent = req.http.X-UA-Device; } }
+    sub vcl_pass { if (req.http.X-UA-Device) { set req.http.User-Agent = req.http.X-UA-Device; } }
 
     # standard Vary handling code from previous examples.
     sub vcl_backend_response {



From lkarsten at varnish-software.com  Thu Jan 14 14:15:07 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:07 +0100
Subject: [4.1] 812ac70 Remove pasto
Message-ID: <E1aJig7-0005GP-3W@project.varnish-software.com>


commit 812ac70811b2c9b80c8b1267c1db98cea1370da8
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Fri Nov 27 17:42:26 2015 +0000

    Remove pasto

diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index c493e02..a539817 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -419,7 +419,6 @@ cnt_lookup(struct worker *wrk, struct req *req)
 		wrk->stats->cache_hitpass++;
 		req->req_step = R_STP_PASS;
 		return (REQ_FSM_MORE);
-	} else if (oc->flags & OC_F_PASS) {
 	}
 
 	VSLb(req->vsl, SLT_Hit, "%u", ObjGetXID(wrk, req->objcore));



From lkarsten at varnish-software.com  Thu Jan 14 14:15:07 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:07 +0100
Subject: [4.1] dd050f0 Reword and update code
Message-ID: <E1aJig7-0005GY-5E@project.varnish-software.com>


commit dd050f03e07fb1e7a1dab6e33b28285fc148f630
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Fri Nov 27 19:59:05 2015 +0000

    Reword and update code

diff --git a/doc/sphinx/users-guide/vcl-example-websockets.rst b/doc/sphinx/users-guide/vcl-example-websockets.rst
index 609fe61..24ea561 100644
--- a/doc/sphinx/users-guide/vcl-example-websockets.rst
+++ b/doc/sphinx/users-guide/vcl-example-websockets.rst
@@ -1,21 +1,14 @@
 
-Implementing websocket support
-------------------------------
+Adding WebSockets support
+-------------------------
 
-Websockets is a technology for creating a bidirectional stream-based channel over HTTP.
+WebSockets is a technology for creating a bidirectional stream-based
+channel over HTTP.
 
-To run websockets through Varnish you need to pipe it, and copy the Upgrade header. Use the following
-VCL config to do so::
+To run WebSockets through Varnish you need to pipe it as follows::
 
-    sub vcl_pipe {
-         if (req.http.upgrade) {
-             set bereq.http.upgrade = req.http.upgrade;
-         }
-    }
     sub vcl_recv {
-         if (req.http.Upgrade ~ "(?i)websocket") {
-             return (pipe);
-         }
+        if (req.http.upgrade ~ "(?i)websocket") {
+            return (pipe);
+        }
     }
-
-.. XXX: Pipe it? maybe a bit more explanation here? benc



From lkarsten at varnish-software.com  Thu Jan 14 14:15:07 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:07 +0100
Subject: [4.1] 2e0fb17 Update references
Message-ID: <E1aJig7-0005Gh-6s@project.varnish-software.com>


commit 2e0fb17940bca84ebaff297e8f296a6afcfa46c0
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Mon Nov 30 18:04:48 2015 +0000

    Update references

diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index 5d1d1ab..17aba0e 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -327,14 +327,14 @@ vbf_stp_startfetch(struct worker *wrk, struct busyobj *bo)
 		/*
 		 * A HEAD request can never have a body in the reply,
 		 * no matter what the headers might say.
-		 * [RFC2516 4.3 p33]
+		 * [RFC7231 4.3.2 p25]
 		 */
 		wrk->stats->fetch_head++;
 		bo->htc->body_status = BS_NONE;
 	} else if (http_GetStatus(bo->beresp) <= 199) {
 		/*
 		 * 1xx responses never have a body.
-		 * [RFC2616 4.3 p33]
+		 * [RFC7230 3.3.2 p31]
 		 * ... but we should never see them.
 		 */
 		wrk->stats->fetch_1xx++;
@@ -342,7 +342,7 @@ vbf_stp_startfetch(struct worker *wrk, struct busyobj *bo)
 	} else if (http_IsStatus(bo->beresp, 204)) {
 		/*
 		 * 204 is "No Content", obviously don't expect a body.
-		 * [RFC7230 3.3.1 p28 and 3.3.2 p30]
+		 * [RFC7230 3.3.1 p29 and 3.3.2 p31]
 		 */
 		wrk->stats->fetch_204++;
 		if (http_GetHdr(bo->beresp, H_Content_Length, NULL) ||
@@ -353,7 +353,7 @@ vbf_stp_startfetch(struct worker *wrk, struct busyobj *bo)
 	} else if (http_IsStatus(bo->beresp, 304)) {
 		/*
 		 * 304 is "Not Modified" it has no body.
-		 * [RFC2616 10.3.5 p63]
+		 * [RFC7230 3.3 p28]
 		 */
 		wrk->stats->fetch_304++;
 		bo->htc->body_status = BS_NONE;



From lkarsten at varnish-software.com  Thu Jan 14 14:15:07 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:07 +0100
Subject: [4.1] 5e21eb3 VMODs handling of VCL_EVENT_COLD must be failsafe
Message-ID: <E1aJig7-0005Gq-8s@project.varnish-software.com>


commit 5e21eb3b915080c1a770a5f57fdce94ae188578a
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Tue Dec 1 15:48:38 2015 +0100

    VMODs handling of VCL_EVENT_COLD must be failsafe
    
    Also for 4.1

diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index c747ead..de39d60 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -404,7 +404,7 @@ vcl_set_state(struct vcl *vcl, const char *state)
 			break;
 		if (vcl->busy == 0) {
 			vcl->temp = vcl_temp_cold;
-			(void)vcl->conf->event_vcl(&ctx, VCL_EVENT_COLD);
+			AZ(vcl->conf->event_vcl(&ctx, VCL_EVENT_COLD));
 			vcl_BackendEvent(vcl, VCL_EVENT_COLD);
 		} else {
 			vcl->temp = vcl_temp_cooling;



From lkarsten at varnish-software.com  Thu Jan 14 14:15:07 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:07 +0100
Subject: [4.1] dffd38a Reserve space for the "cooling" state in vcl.list
Message-ID: <E1aJig7-0005Gz-AF@project.varnish-software.com>


commit dffd38afb926320e7f212060d0f62122757a2024
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Tue Dec 1 12:30:16 2015 +0100

    Reserve space for the "cooling" state in vcl.list
    
    A VCL may show up as auto/cooling in the CLI, breaking the columns align
    in the output.
    
    Also for 4.1

diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index de39d60..cdd4124 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -554,7 +554,7 @@ ccf_config_list(struct cli *cli, const char * const *av, void *priv)
 			flg = "discarded";
 		} else
 			flg = "available";
-		VCLI_Out(cli, "%-10s %4s/%s  %6u %s\n",
+		VCLI_Out(cli, "%-10s %4s/%-8s %6u %s\n",
 		    flg, vcl->state, vcl->temp, vcl->busy, vcl->loaded_name);
 	}
 }
diff --git a/bin/varnishd/mgt/mgt_vcl.c b/bin/varnishd/mgt/mgt_vcl.c
index bb298bb..b12bdb6 100644
--- a/bin/varnishd/mgt/mgt_vcl.c
+++ b/bin/varnishd/mgt/mgt_vcl.c
@@ -414,7 +414,7 @@ mcf_vcl_list(struct cli *cli, const char * const *av, void *priv)
 		free(p);
 	} else {
 		VTAILQ_FOREACH(vp, &vclhead, list) {
-			VCLI_Out(cli, "%-10s %4s/%s  %6s %s\n",
+			VCLI_Out(cli, "%-10s %4s/%-8s %6s %s\n",
 			    vp == active_vcl ? "active" : "available",
 			    vp->state,
 			    vp->warm ? "warm" : "cold", "", vp->name);



From lkarsten at varnish-software.com  Thu Jan 14 14:15:07 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:07 +0100
Subject: [4.1] 0519806 Avoid connection attempts to ${bad_ip}:80
Message-ID: <E1aJig7-0005H8-C1@project.varnish-software.com>


commit 05198064870546dfbdd5984a18d05404489c6474
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Tue Dec 1 16:37:01 2015 +0100

    Avoid connection attempts to ${bad_ip}:80
    
    There is a Squid server listening in today's office...

diff --git a/bin/varnishtest/tests/b00015.vtc b/bin/varnishtest/tests/b00015.vtc
index e26b16d..ec1990f 100644
--- a/bin/varnishtest/tests/b00015.vtc
+++ b/bin/varnishtest/tests/b00015.vtc
@@ -4,7 +4,8 @@ varnishtest "Check synthetic error page caching"
 
 varnish v1 -vcl {
 	backend foo {
-		.host = "${bad_ip}"; .port = "9080";
+		.host = "${bad_ip}";
+		.port = "9080";
 	}
 } -start
 
diff --git a/bin/varnishtest/tests/b00017.vtc b/bin/varnishtest/tests/b00017.vtc
index 81776bb..d0088aa 100644
--- a/bin/varnishtest/tests/b00017.vtc
+++ b/bin/varnishtest/tests/b00017.vtc
@@ -2,7 +2,8 @@ varnishtest "Check that we do not close one error"
 
 varnish v1 -vcl {
 	backend foo {
-		.host = "${bad_ip}"; .port = "9080";
+		.host = "${bad_ip}";
+		.port = "9080";
 	}
 	sub vcl_recv {
 		return (synth(888));
diff --git a/bin/varnishtest/tests/c00028.vtc b/bin/varnishtest/tests/c00028.vtc
index 10caf43..7ecfb7c 100644
--- a/bin/varnishtest/tests/c00028.vtc
+++ b/bin/varnishtest/tests/c00028.vtc
@@ -2,7 +2,8 @@ varnishtest "Test that we can't recurse restarts forever"
 
 varnish v1 -vcl {
 	backend bad {
-		.host = "${bad_ip}"; .port = "9090";
+		.host = "${bad_ip}";
+		.port = "9090";
 	}
 
 	sub vcl_backend_fetch {
diff --git a/bin/varnishtest/tests/c00061.vtc b/bin/varnishtest/tests/c00061.vtc
index ef2ae76..b22f259 100644
--- a/bin/varnishtest/tests/c00061.vtc
+++ b/bin/varnishtest/tests/c00061.vtc
@@ -2,7 +2,7 @@ varnishtest "retry in vcl_backend_error"
 
 varnish v1 -vcl {
 
-	backend b1 { .host = "${bad_ip}"; }
+	backend b1 { .host = "${bad_ip}"; .port = "9080"; }
 
 	sub vcl_backend_error {
 		return (retry);
diff --git a/bin/varnishtest/tests/c00063.vtc b/bin/varnishtest/tests/c00063.vtc
index 0da903e..9da60a0 100644
--- a/bin/varnishtest/tests/c00063.vtc
+++ b/bin/varnishtest/tests/c00063.vtc
@@ -2,7 +2,7 @@ varnishtest "cache backend synth object"
 
 varnish v1 -vcl {
 
-	backend b { .host = "${bad_ip}"; }
+	backend b { .host = "${bad_ip}"; .port = "9080"; }
 
 	sub vcl_backend_error {
 		set beresp.ttl = 1s;
diff --git a/bin/varnishtest/tests/c00066.vtc b/bin/varnishtest/tests/c00066.vtc
index ec88da6..14d4126 100644
--- a/bin/varnishtest/tests/c00066.vtc
+++ b/bin/varnishtest/tests/c00066.vtc
@@ -6,7 +6,7 @@ server s1 {
 } -start
 
 varnish v1 -vcl+backend {
-	backend bad { .host = "${bad_ip}"; }
+	backend bad { .host = "${bad_ip}"; .port = "9080"; }
 	sub vcl_recv {
 		if (req.url == "/synth") {
 			return (synth(200, "Synth test"));
diff --git a/bin/varnishtest/tests/d00007.vtc b/bin/varnishtest/tests/d00007.vtc
index 298fbf7..26112e6 100644
--- a/bin/varnishtest/tests/d00007.vtc
+++ b/bin/varnishtest/tests/d00007.vtc
@@ -8,7 +8,7 @@ server s1 {
 varnish v1 -vcl {
 	import ${vmod_debug};
 
-	backend dummy { .host = "${bad_ip}"; }
+	backend dummy { .host = "${bad_ip}"; .port = "9080"; }
 
 	sub vcl_init {
 		new s1 = debug.dyn("${s1_addr}", "${s1_port}");
diff --git a/bin/varnishtest/tests/d00008.vtc b/bin/varnishtest/tests/d00008.vtc
index a30edd8..79096b9 100644
--- a/bin/varnishtest/tests/d00008.vtc
+++ b/bin/varnishtest/tests/d00008.vtc
@@ -15,7 +15,7 @@ server s2 {
 varnish v1 -vcl {
 	import ${vmod_debug};
 
-	backend dummy { .host = "${bad_ip}"; }
+	backend dummy { .host = "${bad_ip}"; .port = "9080"; }
 
 	sub vcl_init {
 		new s1 = debug.dyn("${s1_addr}", "${s1_port}");
diff --git a/bin/varnishtest/tests/d00009.vtc b/bin/varnishtest/tests/d00009.vtc
index c888d20..46d05c4 100644
--- a/bin/varnishtest/tests/d00009.vtc
+++ b/bin/varnishtest/tests/d00009.vtc
@@ -18,7 +18,7 @@ server s2 {
 varnish v1 -vcl {
 	import ${vmod_debug};
 
-	backend dummy { .host = "${bad_ip}"; }
+	backend dummy { .host = "${bad_ip}"; .port = "9080"; }
 
 	sub vcl_init {
 		new s1 = debug.dyn("${s1_addr}", "${s1_port}");
diff --git a/bin/varnishtest/tests/d00010.vtc b/bin/varnishtest/tests/d00010.vtc
index 1db0ea7..145112a 100644
--- a/bin/varnishtest/tests/d00010.vtc
+++ b/bin/varnishtest/tests/d00010.vtc
@@ -18,7 +18,7 @@ server s2 {
 varnish v1 -vcl {
 	import ${vmod_debug};
 
-	backend dummy { .host = "${bad_ip}"; }
+	backend dummy { .host = "${bad_ip}"; .port = "9080"; }
 
 	sub vcl_init {
 		new s1 = debug.dyn("${s1_addr}", "${s1_port}");
diff --git a/bin/varnishtest/tests/d00011.vtc b/bin/varnishtest/tests/d00011.vtc
index 01d45a1..639161e 100644
--- a/bin/varnishtest/tests/d00011.vtc
+++ b/bin/varnishtest/tests/d00011.vtc
@@ -12,7 +12,7 @@ varnish v1 -vcl {
 	import ${vmod_std};
 	import ${vmod_debug};
 
-	backend dummy { .host = "${bad_ip}"; }
+	backend dummy { .host = "${bad_ip}"; .port = "9080"; }
 
 	sub vcl_init {
 		new s1 = debug.dyn("${s1_addr}", "${s1_port}");
diff --git a/bin/varnishtest/tests/d00012.vtc b/bin/varnishtest/tests/d00012.vtc
index 74aaa8d..41d23e4 100644
--- a/bin/varnishtest/tests/d00012.vtc
+++ b/bin/varnishtest/tests/d00012.vtc
@@ -12,7 +12,7 @@ server s1 {
 varnish v1 -arg "-p thread_pools=1" -vcl {
 	import ${vmod_debug};
 
-	backend dummy { .host = "${bad_ip}"; }
+	backend dummy { .host = "${bad_ip}"; .port = "9080"; }
 
 	sub vcl_init {
 		new s1 = debug.dyn("${s1_addr}", "${s1_port}");
@@ -41,7 +41,7 @@ server s2 {
 varnish v1 -vcl {
 	import ${vmod_debug};
 
-	backend dummy { .host = "${bad_ip}"; }
+	backend dummy { .host = "${bad_ip}"; .port = "9080"; }
 
 	sub vcl_init {
 		new s2 = debug.dyn("${s2_addr}", "${s2_port}");
diff --git a/bin/varnishtest/tests/d00013.vtc b/bin/varnishtest/tests/d00013.vtc
index 25ab33f..6762f41 100644
--- a/bin/varnishtest/tests/d00013.vtc
+++ b/bin/varnishtest/tests/d00013.vtc
@@ -12,7 +12,7 @@ varnish v1 -vcl {
 	import ${vmod_std};
 	import ${vmod_debug};
 
-	backend dummy { .host = "${bad_ip}"; }
+	backend dummy { .host = "${bad_ip}"; .port = "9080"; }
 
 	sub vcl_init {
 		new s1 = debug.dyn("${s1_addr}", "${s1_port}");
diff --git a/bin/varnishtest/tests/r01002.vtc b/bin/varnishtest/tests/r01002.vtc
index 2697d77..6e9e3fd 100644
--- a/bin/varnishtest/tests/r01002.vtc
+++ b/bin/varnishtest/tests/r01002.vtc
@@ -3,7 +3,7 @@ varnishtest "Real relational comparisons"
 varnish v1 -vcl {
 	import ${vmod_std};
 
-	backend foo { .host = "${bad_ip}"; }
+	backend foo { .host = "${bad_ip}"; .port = "9080"; }
 	sub vcl_recv {
 		if (std.random(0,5) < 1.0) {
 			return (pipe);
diff --git a/bin/varnishtest/tests/r01395.vtc b/bin/varnishtest/tests/r01395.vtc
index 591bb2e..06962db 100644
--- a/bin/varnishtest/tests/r01395.vtc
+++ b/bin/varnishtest/tests/r01395.vtc
@@ -2,7 +2,8 @@ varnishtest "Test vcl_synth is called even if vcl_backend_fetch failed"
 
 varnish v1 -vcl {
 	backend default {
-		.host = "${bad_ip}"; .port = "9090";
+		.host = "${bad_ip}";
+		.port = "9090";
 	}
 
 	sub vcl_backend_error {
diff --git a/bin/varnishtest/tests/r01398.vtc b/bin/varnishtest/tests/r01398.vtc
index fe89e0d..ec0003e 100644
--- a/bin/varnishtest/tests/r01398.vtc
+++ b/bin/varnishtest/tests/r01398.vtc
@@ -2,7 +2,8 @@ varnishtest "ticket 1398"
 
 varnish v1 -vcl {
 	backend foo {
-		.host = "${bad_ip}"; .port = "9080";
+		.host = "${bad_ip}";
+		.port = "9080";
 	}
 	sub vcl_backend_response {
 		set beresp.http.X-BE-Name = beresp.backend.name;
diff --git a/bin/varnishtest/tests/r01504.vtc b/bin/varnishtest/tests/r01504.vtc
index 0277c65..8a466f7 100644
--- a/bin/varnishtest/tests/r01504.vtc
+++ b/bin/varnishtest/tests/r01504.vtc
@@ -3,6 +3,7 @@ varnishtest "unreferenced acls"
 varnish v1 -arg "-p vcc_err_unref=off" -vcl {
 	backend s1 {
 		.host = "${bad_ip}";
+		.port = "9090";
 	}
 	acl foo {
 		"127.0.0.1";
diff --git a/bin/varnishtest/tests/r01510.vtc b/bin/varnishtest/tests/r01510.vtc
index 47b87cb..6b74f54 100644
--- a/bin/varnishtest/tests/r01510.vtc
+++ b/bin/varnishtest/tests/r01510.vtc
@@ -11,7 +11,7 @@ varnish v1 -errvcl {Object name 'first' already used.} {
 varnish v1 -errvcl {Object name 'first' already used.} {
 	import ${vmod_debug};
 
-	backend first { .host = "${bad_ip}"; }
+	backend first { .host = "${bad_ip}"; .port = "9080"; }
 
 	sub vcl_init {
 		new first = debug.obj("FOO");
diff --git a/bin/varnishtest/tests/r01512.vtc b/bin/varnishtest/tests/r01512.vtc
index 71496a2..87556f1 100644
--- a/bin/varnishtest/tests/r01512.vtc
+++ b/bin/varnishtest/tests/r01512.vtc
@@ -33,7 +33,7 @@ client c1 {
 # Then across v_b_e and v_b_f
 
 varnish v1 -vcl {
-	backend bad { .host = "${bad_ip}"; }
+	backend bad { .host = "${bad_ip}"; .port = "9080"; }
 	sub vcl_backend_fetch {
 		set bereq.backend = bad;
 		if (bereq.http.x-abandon == "2") {
diff --git a/bin/varnishtest/tests/r01730.vtc b/bin/varnishtest/tests/r01730.vtc
index 2e15ead..e54a410 100644
--- a/bin/varnishtest/tests/r01730.vtc
+++ b/bin/varnishtest/tests/r01730.vtc
@@ -1,7 +1,7 @@
 varnishtest "Test connection error on pipe"
 
 varnish v1 -vcl {
-       backend default { .host = "${bad_ip}"; }
+       backend default { .host = "${bad_ip}"; .port = "9080"; }
        sub vcl_recv {
 		return (pipe);
        }



From lkarsten at varnish-software.com  Thu Jan 14 14:15:07 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:07 +0100
Subject: [4.1] a16aa3b Document that VCL_EVENT_USE is deprecated
Message-ID: <E1aJig7-0005HJ-ET@project.varnish-software.com>


commit a16aa3bc871b092b67f4b9132452271141a999c3
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Tue Dec 1 12:46:43 2015 +0100

    Document that VCL_EVENT_USE is deprecated
    
    And stop using it in the test suite.
    
    Also for 4.1

diff --git a/bin/varnishtest/tests/v00044.vtc b/bin/varnishtest/tests/v00044.vtc
index c3fd107..aa79e6a 100644
--- a/bin/varnishtest/tests/v00044.vtc
+++ b/bin/varnishtest/tests/v00044.vtc
@@ -7,7 +7,7 @@ server s1 -repeat 20 {
 	close
 } -start
 
-# The debug vmod logs some vcl events
+# The debug vmod logs temperature vcl events
 varnish v1 -arg "-p vcl_cooldown=1" -vcl {
 	import ${vmod_debug};
 	backend default {
@@ -69,7 +69,6 @@ varnish v1 -clierr 106 "vcl.state vcl2 cold"
 logexpect l1 -v v1 -g raw {
 	expect * 0 Debug "vcl1: VCL_EVENT_COLD"
 	expect * 0 Debug "vcl1: VCL_EVENT_WARM"
-	expect * 0 Debug "vcl1: VCL_EVENT_USE"
 } -start
 
 # ...when you use a cold VCL
diff --git a/doc/sphinx/reference/vmod.rst b/doc/sphinx/reference/vmod.rst
index 0d30162..fac9e70 100644
--- a/doc/sphinx/reference/vmod.rst
+++ b/doc/sphinx/reference/vmod.rst
@@ -358,10 +358,10 @@ Event functions
 
 VMODs can have an "event" function which is called when a VCL which imports
 the VMOD is loaded, made active, or discarded.  This corresponds to the
-``VCL_EVENT_LOAD``, ``VCL_EVENT_USE``, and ``VCL_EVENT_DISCARD`` events,
-respectively.  In addition, this function will be called when the VCL state is
-changed to cold or warm, corresponding to the ``VCL_EVENT_COLD`` and
-``VCL_EVENT_WARM`` events.
+``VCL_EVENT_LOAD``, and ``VCL_EVENT_DISCARD`` events, respectively.  In
+addition, this function will be called when the VCL temperature is changed to
+cold or warm, corresponding to the ``VCL_EVENT_COLD`` and ``VCL_EVENT_WARM``
+events.
 
 The first argument to the event function is a VRT context.
 
@@ -379,11 +379,13 @@ discarded and free this global state when the count reaches zero.
 VMOD writers are *strongly* encouraged to release all per-VCL resources for a
 given VCL when it emits a ``VCL_EVENT_COLD`` event. You will get a chance to
 reacquire the resources before the VCL becomes active again and be notified
-first with a ``VCL_EVENT_WARM`` event, and then a ``VCL_EVENT_USE`` event.
-Unless a user decides that a given VCL should always be warm, an inactive VMOD
-will eventually become cold and should manage resources accordingly.
+first with a ``VCL_EVENT_WARM`` event. Unless a user decides that a given VCL
+should always be warm, an inactive VMOD will eventually become cold and should
+manage resources accordingly.
 
-.. TODO vmod objects
+There is also a ``VCL_EVENT_USE`` event. Please note that this event is now
+deprecated and may be removed in a future release. A warm VCL should be ready
+to use so no additional task should be postponed at use time.
 
 When to lock, and when not to lock
 ==================================
diff --git a/lib/libvmod_debug/vmod_debug.c b/lib/libvmod_debug/vmod_debug.c
index 732d89a..b065b7f 100644
--- a/lib/libvmod_debug/vmod_debug.c
+++ b/lib/libvmod_debug/vmod_debug.c
@@ -258,7 +258,6 @@ event_function(VRT_CTX, struct vmod_priv *priv, enum vcl_event_e e)
 	switch (e) {
 		case VCL_EVENT_COLD: ev = "VCL_EVENT_COLD"; break;
 		case VCL_EVENT_WARM: ev = "VCL_EVENT_WARM"; break;
-		case VCL_EVENT_USE:  ev = "VCL_EVENT_USE";  break;
 		default: ev = NULL;
 	}
 



From lkarsten at varnish-software.com  Thu Jan 14 14:15:07 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:07 +0100
Subject: [4.1] 656113e Minor cleanup of varnishadm man page.
Message-ID: <E1aJig7-0005HS-GB@project.varnish-software.com>


commit 656113e61618bec5e46bb6e73e013d01683bfc44
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Fri Dec 4 11:06:03 2015 +0100

    Minor cleanup of varnishadm man page.
    
    * Adhere to current style guide.
    * Sort argument list.
    * Add missing reference to related man pages.

diff --git a/doc/sphinx/reference/varnishadm.rst b/doc/sphinx/reference/varnishadm.rst
index 41e879a..4f24c2c 100644
--- a/doc/sphinx/reference/varnishadm.rst
+++ b/doc/sphinx/reference/varnishadm.rst
@@ -14,15 +14,16 @@ Control a running Varnish instance
 SYNOPSIS
 ========
 
-varnishadm [-t timeout] [-S secret_file] [-T address:port] [-n name] [command [...]]
+varnishadm [-n ident] [-t timeout] [-S secretfile] -T [address]:port [command [...]]
+
 
 DESCRIPTION
 ===========
 
-The varnishadm utility establishes a CLI connection to varnishd either
+The `varnishadm` utility establishes a CLI connection to varnishd either
 using -n *name* or using the -T and -S arguments. If -n *name* is
 given the location of the secret file and the address:port is looked
-up in shared memory. If neither is given varnishadm will look for an
+up in shared memory. If neither is given `varnishadm` will look for an
 instance without a given name.
 
 If a command is given, the command and arguments are sent over the CLI
@@ -34,16 +35,16 @@ replies between the CLI socket and stdin/stdout.
 OPTIONS
 =======
 
--t timeout
-    Wait no longer than this many seconds for an operation to finish.
+-n ident
+    Connect to the instance of `varnishd` with this name.
 
--S secret_file
+-S secretfile
     Specify the authentication secret file. This should be the same -S
-    argument as was given to varnishd. Only processes which can read
+    argument as was given to `varnishd`. Only processes which can read
     the contents of this file, will be able to authenticate the CLI connection.
 
--n name
-    Connect to the instance of varnishd with this name.
+-t timeout
+    Wait no longer than this many seconds for an operation to finish.
 
 -T <address:port>
     Connect to the management interface at the specified address and port.
@@ -60,7 +61,7 @@ the *param.show* command.
 EXIT STATUS
 ===========
 
-If a command is given, the exit status of the varnishadm utility is
+If a command is given, the exit status of the `varnishadm` utility is
 zero if the command succeeded, and non-zero otherwise.
 
 EXAMPLES
@@ -76,6 +77,7 @@ SEE ALSO
 ========
 
 * :ref:`varnishd(1)`
+* :ref:`varnish-cli(7)`
 
 HISTORY
 =======



From lkarsten at varnish-software.com  Thu Jan 14 14:15:07 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:07 +0100
Subject: [4.1] bc81b85 Document what panic.clear -z does.
Message-ID: <E1aJig7-0005Hb-Ha@project.varnish-software.com>


commit bc81b85ae9020924aa202388783559422ed3fceb
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Fri Dec 4 11:29:58 2015 +0100

    Document what panic.clear -z does.

diff --git a/doc/sphinx/reference/varnish-cli.rst b/doc/sphinx/reference/varnish-cli.rst
index c55ecf8..390bbe9 100644
--- a/doc/sphinx/reference/varnish-cli.rst
+++ b/doc/sphinx/reference/varnish-cli.rst
@@ -133,7 +133,7 @@ panic.show
   Return the last panic, if any.
 
 panic.clear [-z]
-  Clear the last panic, if any.
+  Clear the last panic, if any. -z will clear related varnishstat counter(s).
 
 storage.list
   List storage devices.



From lkarsten at varnish-software.com  Thu Jan 14 14:15:07 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:07 +0100
Subject: [4.1] eab2be2 Fold history and copyright into an authors section.
Message-ID: <E1aJig7-0005Hk-Jl@project.varnish-software.com>


commit eab2be292133945a6ff6556b61bcb85ce9dd3e04
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Fri Dec 4 11:30:23 2015 +0100

    Fold history and copyright into an authors section.

diff --git a/doc/sphinx/reference/varnish-cli.rst b/doc/sphinx/reference/varnish-cli.rst
index 390bbe9..cc716f6 100644
--- a/doc/sphinx/reference/varnish-cli.rst
+++ b/doc/sphinx/reference/varnish-cli.rst
@@ -305,24 +305,16 @@ the backend contains "USERID=1663"::
 
     req.http.host ~ "^(?i)(www\.)example.com$" && obj.http.set-cookie ~ "USERID=1663"
 
+AUTHORS
+=======
+
+This manual page was originally written by Per Buer and later modified by
+Federico G. Schwindt, Dridi Boukelmoune, Lasse Karstensen and Poul-Henning
+Kamp.
+
 SEE ALSO
 ========
 
 * :ref:`varnishadm(1)`
 * :ref:`varnishd(1)`
 * :ref:`vcl(7)`
-
-HISTORY
-=======
-
-The Varnish manual page was written by Per Buer in 2011. Some of the
-text was taken from the Varnish Cache wiki, the :ref:`varnishd(1)` man
-page or the Varnish source code.
-
-COPYRIGHT
-=========
-
-This document is licensed under the same licence as Varnish
-itself. See LICENCE for details.
-
-* Copyright (c) 2011-2015 Varnish Software AS



From lkarsten at varnish-software.com  Thu Jan 14 14:15:07 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:07 +0100
Subject: [4.1] 530ba31 Add extra sentence on -z in inline help as well.
Message-ID: <E1aJig7-0005Ht-Lb@project.varnish-software.com>


commit 530ba31881a80dc836c316388a5a364ba1171704
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Fri Dec 4 11:46:06 2015 +0100

    Add extra sentence on -z in inline help as well.
    
    Forgotten in commit 8139164.

diff --git a/include/vcli.h b/include/vcli.h
index da61d30..de6e324 100644
--- a/include/vcli.h
+++ b/include/vcli.h
@@ -168,7 +168,7 @@
 #define CLI_PANIC_CLEAR							\
 	"panic.clear",							\
 	"panic.clear [-z]",						\
-	"\tClear the last panic, if any.",				\
+	"\tClear the last panic, if any. -z will clear related varnishstat counter(s)",	\
 	0, 1
 
 /*



From lkarsten at varnish-software.com  Thu Jan 14 14:15:07 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:07 +0100
Subject: [4.1] 1f32fae Update usage text to reflect reality.
Message-ID: <E1aJig7-0005I2-Nk@project.varnish-software.com>


commit 1f32fae8323fa77b5209bb55b52359d7795eb79e
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Fri Dec 4 11:54:18 2015 +0100

    Update usage text to reflect reality.
    
    Update built-in usage text to reflect that -T is optional.

diff --git a/bin/varnishadm/varnishadm.c b/bin/varnishadm/varnishadm.c
index 12650b2..621d696 100644
--- a/bin/varnishadm/varnishadm.c
+++ b/bin/varnishadm/varnishadm.c
@@ -380,8 +380,8 @@ static void
 usage(void)
 {
 	fprintf(stderr,
-	    "usage: varnishadm [-n ident] [-t timeout] [-S secretfile] "
-	    "-T [address]:port command [...]\n");
+	    "Usage: varnishadm [-n ident] [-t timeout] [-S secretfile] "
+	    "[-T [address]:port] [command [...]]\n");
 	fprintf(stderr, "\t-n is mutually exclusive with -S and -T\n");
 	exit(1);
 }
diff --git a/doc/sphinx/reference/varnishadm.rst b/doc/sphinx/reference/varnishadm.rst
index 4f24c2c..29a34ec 100644
--- a/doc/sphinx/reference/varnishadm.rst
+++ b/doc/sphinx/reference/varnishadm.rst
@@ -14,7 +14,7 @@ Control a running Varnish instance
 SYNOPSIS
 ========
 
-varnishadm [-n ident] [-t timeout] [-S secretfile] -T [address]:port [command [...]]
+varnishadm [-n ident] [-t timeout] [-S secretfile] [-T [address]:port] [command [...]]
 
 
 DESCRIPTION



From lkarsten at varnish-software.com  Thu Jan 14 14:15:07 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:07 +0100
Subject: [4.1] b3c24c0 Fold history into authors.
Message-ID: <E1aJig7-0005IC-P5@project.varnish-software.com>


commit b3c24c0258cc84888830791d4fdc568377fba64d
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Fri Dec 4 11:55:37 2015 +0100

    Fold history into authors.

diff --git a/doc/sphinx/reference/varnishadm.rst b/doc/sphinx/reference/varnishadm.rst
index 29a34ec..395a72a 100644
--- a/doc/sphinx/reference/varnishadm.rst
+++ b/doc/sphinx/reference/varnishadm.rst
@@ -29,7 +29,7 @@ instance without a given name.
 If a command is given, the command and arguments are sent over the CLI
 connection and the result returned on stdout.
 
-If no command argument is given varnishadm will pass commands and
+If no command argument is given `varnishadm` will pass commands and
 replies between the CLI socket and stdin/stdout.
 
 OPTIONS
@@ -79,17 +79,9 @@ SEE ALSO
 * :ref:`varnishd(1)`
 * :ref:`varnish-cli(7)`
 
-HISTORY
+AUTHORS
 =======
 
-The varnishadm utility and this manual page were written by Cecilie
-Fritzvold. Converted to reStructuredText and updated in 2010 by Per
-Buer.
-
-COPYRIGHT
-=========
-
-This document is licensed under the same licence as Varnish
-itself. See LICENCE for details.
-
-* Copyright (c) 2007-2015 Varnish Software AS
+The `varnishadm` utility and this manual page were written by Cecilie
+Fritzvold. This man page has later been modified by Per Buer, Federico G.
+Schwindt and Lasse Karstensen.



From lkarsten at varnish-software.com  Thu Jan 14 14:15:07 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:07 +0100
Subject: [4.1] 0cf1262 Add some examples of backend expressions.
Message-ID: <E1aJig7-0005IM-Qq@project.varnish-software.com>


commit 0cf12629cf3d6ff488de33e0cdb7d82d37fb607f
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Fri Dec 4 12:00:36 2015 +0100

    Add some examples of backend expressions.

diff --git a/doc/sphinx/reference/varnish-cli.rst b/doc/sphinx/reference/varnish-cli.rst
index cc716f6..3149b83 100644
--- a/doc/sphinx/reference/varnish-cli.rst
+++ b/doc/sphinx/reference/varnish-cli.rst
@@ -159,6 +159,14 @@ name, IP address and port in "name(IP address:port)" format. All fields
 are optional. If no exact matching backend is found, partial matching
 will be attempted based on the provided name, IP address and port fields.
 
+Examples::
+
+   backend.list def*
+   backend.set_health default sick
+   backend.set_health def* healthy
+   backend.set_health * auto
+
+
 Ban Expressions
 ---------------
 



From lkarsten at varnish-software.com  Thu Jan 14 14:15:07 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:07 +0100
Subject: [4.1] df4ef3b Restart epoll_wait on EINTR error
Message-ID: <E1aJig7-0005IW-Sj@project.varnish-software.com>


commit df4ef3b99acba107fc69e1d0e07ab262bb2d1ba6
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Fri Dec 4 13:12:05 2015 +0100

    Restart epoll_wait on EINTR error
    
    This works around a Linux kernel bug where the epoll_wait will return
    EINTR when the process is subjected to a ptrace or the OS wakes from
    suspend.
    
    Fixes: #1763

diff --git a/bin/varnishd/waiter/cache_waiter_epoll.c b/bin/varnishd/waiter/cache_waiter_epoll.c
index cb6dbdc..f50ae46 100644
--- a/bin/varnishd/waiter/cache_waiter_epoll.c
+++ b/bin/varnishd/waiter/cache_waiter_epoll.c
@@ -38,6 +38,7 @@
 #include <sys/epoll.h>
 
 #include <stdlib.h>
+#include <errno.h>
 
 #include "cache/cache.h"
 
@@ -110,7 +111,12 @@ vwe_thread(void *priv)
 		i = (int)ceil(1e3 * then);
 		assert(i > 0);
 		Lck_Unlock(&vwe->mtx);
-		n = epoll_wait(vwe->epfd, ev, NEEV, i);
+		do {
+			/* Due to a linux kernel bug, epoll_wait can
+			   return EINTR when the process is subjected to
+			   ptrace or waking from OS suspend. */
+			n = epoll_wait(vwe->epfd, ev, NEEV, i);
+		} while (n < 0 && errno == EINTR);
 		assert(n >= 0);
 		assert(n <= NEEV);
 		now = VTIM_real();



From lkarsten at varnish-software.com  Thu Jan 14 14:15:07 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:07 +0100
Subject: [4.1] 6f3fdd2 Allow VMODs to hold a reference on a warm VCL
Message-ID: <E1aJig7-0005If-UN@project.varnish-software.com>


commit 6f3fdd21ced30d4c29cecd32a4757e751a0d1eb3
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Fri Dec 4 15:24:19 2015 +0100

    Allow VMODs to hold a reference on a warm VCL

diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index cdd4124..8f41773 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -61,6 +61,7 @@ struct vcl {
 	char			state[8];
 	char			*loaded_name;
 	unsigned		busy;
+	unsigned		refcount;
 	unsigned		discard;
 	const char		*temp;
 	VTAILQ_HEAD(,backend)	backend_list;
@@ -244,6 +245,7 @@ vcl_KillBackends(struct vcl *vcl)
 
 	CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC);
 	AZ(vcl->busy);
+	AZ(vcl->refcount);
 	while (1) {
 		be = VTAILQ_FIRST(&vcl->backend_list);
 		if (be == NULL)
@@ -366,6 +368,41 @@ VRT_count(VRT_CTX, unsigned u)
 		    ctx->vcl->conf->ref[u].line, ctx->vcl->conf->ref[u].pos);
 }
 
+void
+VRT_ref_vcl(VRT_CTX)
+{
+	struct vcl *vcl;
+
+	ASSERT_CLI();
+	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
+
+	vcl = ctx->vcl;
+	CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC);
+	xxxassert(vcl->temp == vcl_temp_warm);
+
+	Lck_Lock(&vcl_mtx);
+	vcl->refcount++;
+	Lck_Unlock(&vcl_mtx);
+}
+
+void
+VRT_rel_vcl(VRT_CTX)
+{
+	struct vcl *vcl;
+
+	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
+
+	vcl = ctx->vcl;
+	CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC);
+	assert(vcl->temp == vcl_temp_warm || vcl->temp == vcl_temp_cooling);
+
+	Lck_Lock(&vcl_mtx);
+	assert(vcl->refcount > 0);
+	vcl->refcount--;
+	/* No garbage collection here, for the same reasons as in VCL_Rel. */
+	Lck_Unlock(&vcl_mtx);
+}
+
 /*--------------------------------------------------------------------*/
 
 static struct vcl *
@@ -402,7 +439,7 @@ vcl_set_state(struct vcl *vcl, const char *state)
 			vcl->temp = vcl_temp_cold;
 		if (vcl->temp == vcl_temp_cold)
 			break;
-		if (vcl->busy == 0) {
+		if (vcl->busy == 0 && vcl->refcount == 0) {
 			vcl->temp = vcl_temp_cold;
 			AZ(vcl->conf->event_vcl(&ctx, VCL_EVENT_COLD));
 			vcl_BackendEvent(vcl, VCL_EVENT_COLD);
@@ -508,6 +545,7 @@ VCL_Nuke(struct vcl *vcl)
 	assert(vcl != vcl_active);
 	assert(vcl->discard);
 	AZ(vcl->busy);
+	AZ(vcl->refcount);
 	VTAILQ_REMOVE(&vcl_head, vcl, list);
 	ctx.method = VCL_MET_FINI;
 	ctx.handling = &hand;
@@ -531,7 +569,7 @@ VCL_Poll(void)
 	VTAILQ_FOREACH_SAFE(vcl, &vcl_head, list, vcl2) {
 		if (vcl->temp == vcl_temp_cooling)
 			vcl_set_state(vcl, "0");
-		if (vcl->discard && vcl->busy == 0)
+		if (vcl->discard && vcl->busy == 0 && vcl->refcount == 0)
 			VCL_Nuke(vcl);
 	}
 }
@@ -602,7 +640,7 @@ ccf_config_discard(struct cli *cli, const char * const *av, void *priv)
 	vcl->discard = 1;
 	Lck_Unlock(&vcl_mtx);
 
-	if (vcl->busy == 0)
+	if (vcl->busy == 0 && vcl->refcount == 0)
 		VCL_Nuke(vcl);
 }
 
diff --git a/doc/sphinx/reference/vmod.rst b/doc/sphinx/reference/vmod.rst
index fac9e70..ad6e03f 100644
--- a/doc/sphinx/reference/vmod.rst
+++ b/doc/sphinx/reference/vmod.rst
@@ -383,6 +383,13 @@ first with a ``VCL_EVENT_WARM`` event. Unless a user decides that a given VCL
 should always be warm, an inactive VMOD will eventually become cold and should
 manage resources accordingly.
 
+If your VMOD is running an asynchronous background job you can hold a reference
+to the VCL to prevent it from going cold too soon and get the same guarantees
+as backends with ongoing requests for instance. For that, you must acquire the
+reference by calling ``VRT_ref_vcl`` when you receive a ``VCL_EVENT_WARM`` and
+later calling ``VRT_rel_vcl`` once the background job is over. Receiving a
+``VCL_EVENT_COLD`` is your cue to terminate any background job bound to a VCL.
+
 There is also a ``VCL_EVENT_USE`` event. Please note that this event is now
 deprecated and may be removed in a future release. A warm VCL should be ready
 to use so no additional task should be postponed at use time.
@@ -396,7 +403,7 @@ their own locking to protect shared resources.
 When a VCL is loaded or unloaded, the event and priv->free are
 run sequentially all in a single thread, and there is guaranteed
 to be no other activity related to this particular VCL, nor are
-there  init/fini activity in any other VCL or VMOD at this time.
+there init/fini activity in any other VCL or VMOD at this time.
 
 That means that the VMOD init, and any object init/fini functions
 are already serialized in sensible order, and won't need any locking,
diff --git a/include/vrt.h b/include/vrt.h
index 2d0a012..2203bd8 100644
--- a/include/vrt.h
+++ b/include/vrt.h
@@ -295,6 +295,9 @@ struct vmod_priv {
 typedef int vmod_event_f(VRT_CTX, struct vmod_priv *, enum vcl_event_e);
 #endif
 
+void VRT_ref_vcl(VRT_CTX);
+void VRT_rel_vcl(VRT_CTX);
+
 void VRT_priv_fini(const struct vmod_priv *p);
 struct vmod_priv *VRT_priv_task(VRT_CTX, void *vmod_id);
 struct vmod_priv *VRT_priv_top(VRT_CTX, void *vmod_id);



From lkarsten at varnish-software.com  Thu Jan 14 14:15:08 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:08 +0100
Subject: [4.1] ed41172 Don't create backends on cooling VCLs, return NULL
Message-ID: <E1aJig8-0005Ip-0d@project.varnish-software.com>


commit ed411722de15844803bf448731541fcffacc05b4
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Fri Dec 4 15:44:44 2015 +0100

    Don't create backends on cooling VCLs, return NULL

diff --git a/bin/varnishd/cache/cache_backend.h b/bin/varnishd/cache/cache_backend.h
index 17b90f0..0bc73ed 100644
--- a/bin/varnishd/cache/cache_backend.h
+++ b/bin/varnishd/cache/cache_backend.h
@@ -130,5 +130,5 @@ struct vbc *VBT_Get(struct tcp_pool *, double tmo, const struct backend *,
 void VBT_Wait(struct worker *, struct vbc *);
 
 /* cache_vcl.c */
-void VCL_AddBackend(struct vcl *, struct backend *);
+int VCL_AddBackend(struct vcl *, struct backend *);
 void VCL_DelBackend(struct backend *);
diff --git a/bin/varnishd/cache/cache_backend_cfg.c b/bin/varnishd/cache/cache_backend_cfg.c
index 0a58de1..c734d80 100644
--- a/bin/varnishd/cache/cache_backend_cfg.c
+++ b/bin/varnishd/cache/cache_backend_cfg.c
@@ -65,10 +65,12 @@ struct director *
 VRT_new_backend(VRT_CTX, const struct vrt_backend *vrt)
 {
 	struct backend *b;
+	struct director *d;
 	struct vsb *vsb;
 	struct vcl *vcl;
 	struct tcp_pool *tp = NULL;
 	const struct vrt_backend_probe *vbp;
+	int retval;
 
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
 	CHECK_OBJ_NOTNULL(vrt, VRT_BACKEND_MAGIC);
@@ -123,9 +125,15 @@ VRT_new_backend(VRT_CTX, const struct vrt_backend *vrt)
 	if (vbp != NULL)
 		VBP_Insert(b, vbp, tp);
 
-	VCL_AddBackend(ctx->vcl, b);
+	retval = VCL_AddBackend(ctx->vcl, b);
+
+	if (retval == 0)
+		return (b->director);
 
-	return (b->director);
+	d = b->director;
+	VRT_delete_backend(ctx, &d);
+	AZ(d);
+	return (NULL);
 }
 
 /*--------------------------------------------------------------------
diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index 8f41773..a6d0c26 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -193,12 +193,16 @@ VCL_Rel(struct vcl **vcc)
 
 /*--------------------------------------------------------------------*/
 
-void
+int
 VCL_AddBackend(struct vcl *vcl, struct backend *be)
 {
 
 	CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC);
 	CHECK_OBJ_NOTNULL(be, BACKEND_MAGIC);
+
+	if (vcl->temp == vcl_temp_cooling)
+		return (1);
+
 	Lck_Lock(&vcl_mtx);
 	VTAILQ_INSERT_TAIL(&vcl->backend_list, be, vcl_list);
 	Lck_Unlock(&vcl_mtx);
@@ -208,6 +212,8 @@ VCL_AddBackend(struct vcl *vcl, struct backend *be)
 		VBE_Event(be, VCL_EVENT_WARM);
 	} else if (vcl->temp != vcl_temp_init)
 		WRONG("Dynamic Backends can only be added to warm VCLs");
+
+	return (0);
 }
 
 void
diff --git a/doc/sphinx/reference/directors.rst b/doc/sphinx/reference/directors.rst
index 8ad919b..10e41d6 100644
--- a/doc/sphinx/reference/directors.rst
+++ b/doc/sphinx/reference/directors.rst
@@ -110,7 +110,9 @@ Finally, Varnish will take care of event propagation for *all* native backends,
 but dynamic backends can only be created when the VCL is warm. If your backends
 are created by an independent thread (basically outside of VCL scope) you must
 subscribe to VCL events and watch for VCL state (see
-:ref:`ref-vmod-event-functions`). You are also encouraged to comply with the
+:ref:`ref-vmod-event-functions`). Varnish will panic if you try to create a
+backend on a cold VCL, and ``VRT_new_backend`` will return ``NULL`` if the VCL
+is cooling. You are also encouraged to comply with the
 :ref:`ref_vcl_temperature` in general.
 
 



From lkarsten at varnish-software.com  Thu Jan 14 14:15:08 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:08 +0100
Subject: [4.1] d243b4a Clarify VRT_delete_backend a bit
Message-ID: <E1aJig8-0005Iy-1n@project.varnish-software.com>


commit d243b4a1ebd117eef1dd1ade667be033ed88882c
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Fri Dec 4 15:09:02 2015 +0100

    Clarify VRT_delete_backend a bit

diff --git a/bin/varnishd/cache/cache_backend_cfg.c b/bin/varnishd/cache/cache_backend_cfg.c
index c734d80..ad96330 100644
--- a/bin/varnishd/cache/cache_backend_cfg.c
+++ b/bin/varnishd/cache/cache_backend_cfg.c
@@ -160,6 +160,9 @@ VRT_delete_backend(VRT_CTX, struct director **dp)
 	VTAILQ_REMOVE(&backends, be, list);
 	VTAILQ_INSERT_TAIL(&cool_backends, be, list);
 	Lck_Unlock(&be->mtx);
+
+	// NB. The backend is still usable for the ongoing transactions,
+	// this is why we don't bust the director's magic number.
 }
 
 /*---------------------------------------------------------------------



From lkarsten at varnish-software.com  Thu Jan 14 14:15:08 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:08 +0100
Subject: [4.1] 8c95a2f Start documenting that VCL cool down should go fast
Message-ID: <E1aJig8-0005J7-3r@project.varnish-software.com>


commit 8c95a2f2b5a3d9342c43ca9c088f916f1f899aa7
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Fri Dec 4 16:01:08 2015 +0100

    Start documenting that VCL cool down should go fast

diff --git a/doc/sphinx/reference/vmod.rst b/doc/sphinx/reference/vmod.rst
index ad6e03f..57fa09c 100644
--- a/doc/sphinx/reference/vmod.rst
+++ b/doc/sphinx/reference/vmod.rst
@@ -390,6 +390,10 @@ reference by calling ``VRT_ref_vcl`` when you receive a ``VCL_EVENT_WARM`` and
 later calling ``VRT_rel_vcl`` once the background job is over. Receiving a
 ``VCL_EVENT_COLD`` is your cue to terminate any background job bound to a VCL.
 
+In the case where properly releasing resources may take some time, you can
+opt for an asynchronous worker, either by spawning a thread and tracking it, or
+by using Varnish's worker pools.
+
 There is also a ``VCL_EVENT_USE`` event. Please note that this event is now
 deprecated and may be removed in a future release. A warm VCL should be ready
 to use so no additional task should be postponed at use time.



From lkarsten at varnish-software.com  Thu Jan 14 14:15:08 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:08 +0100
Subject: [4.1] 82238c1 Spelling
Message-ID: <E1aJig8-0005JG-5c@project.varnish-software.com>


commit 82238c16062bf3dc2334dc72741acd10decf8d9a
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Fri Dec 4 12:29:52 2015 +0000

    Spelling

diff --git a/bin/varnishd/cache/cache_esi_deliver.c b/bin/varnishd/cache/cache_esi_deliver.c
index d79cde2..abc705b 100644
--- a/bin/varnishd/cache/cache_esi_deliver.c
+++ b/bin/varnishd/cache/cache_esi_deliver.c
@@ -113,7 +113,7 @@ ved_include(struct req *preq, const char *src, const char *host,
 	http_ForceField(req->http0, HTTP_HDR_METHOD, "GET");
 	http_ForceField(req->http0, HTTP_HDR_PROTO, "HTTP/1.1");
 
-	/* Don't allow conditionalss, we can't use a 304 */
+	/* Don't allow conditionals, we can't use a 304 */
 	http_Unset(req->http0, H_If_Modified_Since);
 	http_Unset(req->http0, H_If_None_Match);
 



From lkarsten at varnish-software.com  Thu Jan 14 14:15:08 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:08 +0100
Subject: [4.1] 368280d Ignore 0 Content-Lengths in 204 responses
Message-ID: <E1aJig8-0005JP-7M@project.varnish-software.com>


commit 368280d4c30f28175d38dbdd9e587d0f80ed7631
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Fri Dec 4 16:08:06 2015 +0000

    Ignore 0 Content-Lengths in 204 responses
    
    Broken implementations do it contrary to what the RFC mandates.
    
    Fixes #1826.

diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index 17aba0e..fe26ac4 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -345,7 +345,8 @@ vbf_stp_startfetch(struct worker *wrk, struct busyobj *bo)
 		 * [RFC7230 3.3.1 p29 and 3.3.2 p31]
 		 */
 		wrk->stats->fetch_204++;
-		if (http_GetHdr(bo->beresp, H_Content_Length, NULL) ||
+		if ((http_GetHdr(bo->beresp, H_Content_Length, NULL) &&
+		    bo->htc->content_length != 0) ||
 		    http_GetHdr(bo->beresp, H_Transfer_Encoding, NULL))
 			bo->htc->body_status = BS_ERROR;
 		else
diff --git a/bin/varnishtest/tests/r01826.vtc b/bin/varnishtest/tests/r01826.vtc
new file mode 100644
index 0000000..eeb4503
--- /dev/null
+++ b/bin/varnishtest/tests/r01826.vtc
@@ -0,0 +1,15 @@
+varnishtest "Check we ignore a zero C-L with a 204"
+
+server s1 {
+	rxreq
+	txresp -status 204 -nolen -hdr "Content-Length: 0"
+} -start
+
+varnish v1 -vcl+backend {
+} -start
+
+client c1 {
+	txreq
+	rxresp
+	expect resp.status == 204
+} -run



From lkarsten at varnish-software.com  Thu Jan 14 14:15:08 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:08 +0100
Subject: [4.1] bce6d9a Drop extra colon (:)
Message-ID: <E1aJig8-0005JY-9P@project.varnish-software.com>


commit bce6d9a33f64ead2115298ce9b730d013ed0bf83
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Sat Dec 5 10:43:20 2015 +0000

    Drop extra colon (:)

diff --git a/lib/libvcc/vcc_vmod.c b/lib/libvcc/vcc_vmod.c
index e8bab4f..60d2ed2 100644
--- a/lib/libvcc/vcc_vmod.c
+++ b/lib/libvcc/vcc_vmod.c
@@ -111,7 +111,7 @@ vcc_ParseImport(struct vcc *tl)
 	if (hdl == NULL) {
 		VSB_printf(tl->sb, "Could not load VMOD %.*s\n", PF(mod));
 		VSB_printf(tl->sb, "\tFile name: %s\n", fn);
-		VSB_printf(tl->sb, "\tdlerror:: %s\n", dlerror());
+		VSB_printf(tl->sb, "\tdlerror: %s\n", dlerror());
 		vcc_ErrWhere(tl, mod);
 		return;
 	}



From lkarsten at varnish-software.com  Thu Jan 14 14:15:08 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:08 +0100
Subject: [4.1] e275141 Introduce a new "busy" VCL temperature
Message-ID: <E1aJig8-0005Jh-BF@project.varnish-software.com>


commit e275141bd5dd360912a4407a7b94994ebe3ce453
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Fri Dec 4 22:07:14 2015 +0100

    Introduce a new "busy" VCL temperature
    
    Now that VCLs can be referenced, there are two different situations
    stalling the cooldown. The "cooling" one is kept for VCLs that have
    received a COLD event, and the "busy" one is introduced to wait for
    ongoing transactions.
    
    Basically, if there are transactions, the VCL is still active even
    though it's not *the* active VCL and should therefore be considered
    warm. This guarantees that WARM and COLD events are balanced, and
    also that a "busy" VCL doesn't need to warm up if it is set to warm
    again.
    
    The new VCL temperature engine looks like this:
    
      .----------W>---------------.
      |              .--W>---.    v
    init ---> cold --|       |-- warm --.
              ^  ^   '---<C--'    |     |
              |  |                |     |
              |  '--- cooling <C--'     |
              |         ^               |
              |         C               |
              |         |    .---<---.  |
              '--<C-- busy --|       |--'
                             '--->---'
    
    The transitions marked with a 'W' or a 'C' are the one dispatching
    WARM and COLD events respectively.
    
    Since the "busy" state is considered warm, VMODs should operate as
    such and backend creation remains possible.

diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index a6d0c26..664c2c1 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -50,6 +50,7 @@
 static const char * const vcl_temp_init = "init";
 static const char * const vcl_temp_cold = "cold";
 static const char * const vcl_temp_warm = "warm";
+static const char * const vcl_temp_busy = "busy";
 static const char * const vcl_temp_cooling = "cooling";
 
 struct vcl {
@@ -164,7 +165,7 @@ VCL_Ref(struct vcl *vcl)
 {
 
 	CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC);
-	assert(vcl->temp == vcl_temp_warm || vcl->temp == vcl_temp_cooling);
+	assert(vcl->temp != vcl_temp_init && vcl->temp != vcl_temp_cold);
 	Lck_Lock(&vcl_mtx);
 	assert(vcl->busy > 0);
 	vcl->busy++;
@@ -207,10 +208,10 @@ VCL_AddBackend(struct vcl *vcl, struct backend *be)
 	VTAILQ_INSERT_TAIL(&vcl->backend_list, be, vcl_list);
 	Lck_Unlock(&vcl_mtx);
 
-	if (vcl->temp == vcl_temp_warm) {
+	if (vcl->temp == vcl_temp_warm || vcl->temp == vcl_temp_busy)
 		/* Only when adding backend to already warm VCL */
 		VBE_Event(be, VCL_EVENT_WARM);
-	} else if (vcl->temp != vcl_temp_init)
+	else if (vcl->temp != vcl_temp_init)
 		WRONG("Dynamic Backends can only be added to warm VCLs");
 
 	return (0);
@@ -400,7 +401,8 @@ VRT_rel_vcl(VRT_CTX)
 
 	vcl = ctx->vcl;
 	CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC);
-	assert(vcl->temp == vcl_temp_warm || vcl->temp == vcl_temp_cooling);
+	assert(vcl->temp == vcl_temp_warm || vcl->temp == vcl_temp_busy ||
+	    vcl->temp == vcl_temp_cooling);
 
 	Lck_Lock(&vcl_mtx);
 	assert(vcl->refcount > 0);
@@ -430,6 +432,7 @@ static void
 vcl_set_state(struct vcl *vcl, const char *state)
 {
 	struct vrt_ctx ctx;
+	const char *target;
 	unsigned hand = 0;
 
 	ASSERT_CLI();
@@ -441,22 +444,29 @@ vcl_set_state(struct vcl *vcl, const char *state)
 
 	switch(state[0]) {
 	case '0':
-		if (vcl->temp == vcl_temp_init)
-			vcl->temp = vcl_temp_cold;
-		if (vcl->temp == vcl_temp_cold)
-			break;
-		if (vcl->busy == 0 && vcl->refcount == 0) {
-			vcl->temp = vcl_temp_cold;
+		assert(vcl->temp != vcl_temp_cold);
+		target = vcl->busy ? vcl_temp_busy : vcl_temp_cold;
+		if (target == vcl_temp_cold && (vcl->temp == vcl_temp_warm ||
+		    vcl->temp == vcl_temp_busy)) {
+
+			vcl->temp = vcl->refcount ? vcl_temp_cooling :
+			    vcl_temp_cold;
 			AZ(vcl->conf->event_vcl(&ctx, VCL_EVENT_COLD));
 			vcl_BackendEvent(vcl, VCL_EVENT_COLD);
-		} else {
-			vcl->temp = vcl_temp_cooling;
 		}
+		else if (vcl->busy)
+			vcl->temp = vcl_temp_busy;
+		else
+			vcl->temp = vcl->refcount ? vcl_temp_cooling :
+			    vcl_temp_cold;
 		break;
 	case '1':
-		if (vcl->temp == vcl_temp_cooling)
+		assert(vcl->temp != vcl_temp_warm);
+		/* The warm VCL hasn't seen a cold event yet */
+		if (vcl->temp == vcl_temp_busy)
 			vcl->temp = vcl_temp_warm;
-		else {
+		/* The VCL must first reach a stable cold state */
+		else if (vcl->temp != vcl_temp_cooling) {
 			vcl->temp = vcl_temp_warm;
 			(void)vcl->conf->event_vcl(&ctx, VCL_EVENT_WARM);
 			vcl_BackendEvent(vcl, VCL_EVENT_WARM);
@@ -573,9 +583,10 @@ VCL_Poll(void)
 
 	ASSERT_CLI();
 	VTAILQ_FOREACH_SAFE(vcl, &vcl_head, list, vcl2) {
-		if (vcl->temp == vcl_temp_cooling)
+		if (vcl->temp == vcl_temp_busy ||
+		    vcl->temp == vcl_temp_cooling)
 			vcl_set_state(vcl, "0");
-		if (vcl->discard && vcl->busy == 0 && vcl->refcount == 0)
+		if (vcl->discard && vcl->temp == vcl_temp_cold)
 			VCL_Nuke(vcl);
 	}
 }



From lkarsten at varnish-software.com  Thu Jan 14 14:15:08 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:08 +0100
Subject: [4.1] 241a64d Make event handling more readable in vmod-debug
Message-ID: <E1aJig8-0005Jr-D0@project.varnish-software.com>


commit 241a64dcdf009096acd6ac47f84eda146621dbe6
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Sat Dec 5 15:48:30 2015 +0100

    Make event handling more readable in vmod-debug

diff --git a/lib/libvmod_debug/vmod_debug.c b/lib/libvmod_debug/vmod_debug.c
index b065b7f..9b3449c 100644
--- a/lib/libvmod_debug/vmod_debug.c
+++ b/lib/libvmod_debug/vmod_debug.c
@@ -249,23 +249,10 @@ priv_vcl_free(void *priv)
 	AZ(priv_vcl);
 }
 
-int __match_proto__(vmod_event_f)
-event_function(VRT_CTX, struct vmod_priv *priv, enum vcl_event_e e)
+static int
+event_load(VRT_CTX, struct vmod_priv *priv)
 {
 	struct priv_vcl *priv_vcl;
-	const char *ev;
-
-	switch (e) {
-		case VCL_EVENT_COLD: ev = "VCL_EVENT_COLD"; break;
-		case VCL_EVENT_WARM: ev = "VCL_EVENT_WARM"; break;
-		default: ev = NULL;
-	}
-
-	if (ev != NULL)
-		VSL(SLT_Debug, 0, "%s: %s", VCL_Name(ctx->vcl), ev);
-
-	if (e != VCL_EVENT_LOAD)
-		return (0);
 
 	AN(ctx->msg);
 	if (cache_param->nuke_limit == 42) {
@@ -282,6 +269,40 @@ event_function(VRT_CTX, struct vmod_priv *priv, enum vcl_event_e e)
 	return (0);
 }
 
+static int
+event_warm(VRT_CTX)
+{
+
+	VSL(SLT_Debug, 0, "%s: VCL_EVENT_WARM", VCL_Name(ctx->vcl));
+
+	if (cache_param->max_esi_depth == 42) {
+		VSB_printf(ctx->msg, "max_esi_depth is not the answer.");
+		return (-1);
+	}
+
+	return (0);
+}
+
+static int
+event_cold(VRT_CTX)
+{
+
+	VSL(SLT_Debug, 0, "%s: VCL_EVENT_COLD", VCL_Name(ctx->vcl));
+	return (0);
+}
+
+int __match_proto__(vmod_event_f)
+event_function(VRT_CTX, struct vmod_priv *priv, enum vcl_event_e e)
+{
+
+	switch (e) {
+		case VCL_EVENT_LOAD: return event_load(ctx, priv);
+		case VCL_EVENT_COLD: return event_cold(ctx);
+		case VCL_EVENT_WARM: return event_warm(ctx);
+		default: return (0);
+	}
+}
+
 VCL_VOID __match_proto__(td_debug_sleep)
 vmod_sleep(VRT_CTX, VCL_DURATION t)
 {



From lkarsten at varnish-software.com  Thu Jan 14 14:15:08 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:08 +0100
Subject: [4.1] cdd5fda Hold a VCL reference in the debug vmod
Message-ID: <E1aJig8-0005K3-Ev@project.varnish-software.com>


commit cdd5fdabe88c34162f959cc85dc0adcaa26ef596
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Sat Dec 5 16:09:14 2015 +0100

    Hold a VCL reference in the debug vmod
    
    By default the reference is released as soon as the VCL goes cold, which
    doesn't harm the existing tests yet allows to cover the new VRT_ref_vcl
    and VRT_rel_vcl functions.
    
    It is possible to ask the vmod to release the reference after a delay.

diff --git a/lib/libvmod_debug/vmod.vcc b/lib/libvmod_debug/vmod.vcc
index dad36ec..42159fb 100644
--- a/lib/libvmod_debug/vmod.vcc
+++ b/lib/libvmod_debug/vmod.vcc
@@ -138,3 +138,7 @@ Mark a workspace as overflowed.
 $Function INT workspace_free(ENUM { client, backend, session, thread })
 
 Find how much unallocated space there is left in a workspace.
+
+$Function VOID vcl_release_delay(DURATION)
+
+Hold a reference to the VCL when it goes cold for the given delay.
diff --git a/lib/libvmod_debug/vmod_debug.c b/lib/libvmod_debug/vmod_debug.c
index 9b3449c..91b49c7 100644
--- a/lib/libvmod_debug/vmod_debug.c
+++ b/lib/libvmod_debug/vmod_debug.c
@@ -28,6 +28,7 @@
 
 #include "config.h"
 
+#include <pthread.h>
 #include <stdlib.h>
 #include <stdio.h>
 
@@ -46,6 +47,8 @@ struct priv_vcl {
 	uintptr_t		exp_cb;
 };
 
+VCL_DURATION vcl_release_delay = 0.0;
+
 VCL_VOID __match_proto__(td_debug_panic)
 vmod_panic(VRT_CTX, const char *str, ...)
 {
@@ -280,14 +283,38 @@ event_warm(VRT_CTX)
 		return (-1);
 	}
 
+	VRT_ref_vcl(ctx);
 	return (0);
 }
 
+static void*
+cooldown_thread(void *priv)
+{
+	struct vrt_ctx ctx;
+
+	AN(priv);
+	INIT_OBJ(&ctx, VRT_CTX_MAGIC);
+	ctx.vcl = (struct vcl*)priv;
+
+	VTIM_sleep(vcl_release_delay);
+	VRT_rel_vcl(&ctx);
+	return (NULL);
+}
+
 static int
 event_cold(VRT_CTX)
 {
+	pthread_t thread;
 
 	VSL(SLT_Debug, 0, "%s: VCL_EVENT_COLD", VCL_Name(ctx->vcl));
+
+	if (vcl_release_delay == 0.0) {
+		VRT_rel_vcl(ctx);
+		return (0);
+	}
+
+	AZ(pthread_create(&thread, NULL, cooldown_thread, ctx->vcl));
+	AZ(pthread_detach(thread));
 	return (0);
 }
 
@@ -379,3 +406,12 @@ vmod_workspace_overflow(VRT_CTX, VCL_ENUM which)
 
 	WS_MarkOverflow(ws);
 }
+
+void
+vmod_vcl_release_delay(VRT_CTX, VCL_DURATION delay)
+{
+
+	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
+	assert(delay > 0.0);
+	vcl_release_delay = delay;
+}



From lkarsten at varnish-software.com  Thu Jan 14 14:15:08 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:08 +0100
Subject: [4.1] e0b97cf Coverage for the cooling state
Message-ID: <E1aJig8-0005K9-GP@project.varnish-software.com>


commit e0b97cf0c2cbda05c828aa76807135d12c108f39
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Sat Dec 5 16:17:02 2015 +0100

    Coverage for the cooling state

diff --git a/bin/varnishtest/tests/v00045.vtc b/bin/varnishtest/tests/v00045.vtc
new file mode 100644
index 0000000..b1754ee
--- /dev/null
+++ b/bin/varnishtest/tests/v00045.vtc
@@ -0,0 +1,29 @@
+varnishtest "Hold a reference to a VCL after a COLD event"
+
+server s1 -start
+
+# Load and use a new VCL that will hold a reference
+varnish v1 -vcl+backend {
+	import ${vmod_debug};
+	sub vcl_init {
+		debug.vcl_release_delay(2s);
+	}
+} -start
+
+# Load and use a new VCL, freeze the first
+varnish v1 -vcl+backend {}
+varnish v1 -cliok "vcl.state vcl1 cold"
+
+# We should now see it as cooling
+delay 1
+shell {
+	${varnishadm} -n ${v1_name} vcl.list |
+	grep "auto/cooling.*vcl1" >/dev/null
+}
+
+# It will eventually cool down
+delay 2
+shell {
+	${varnishadm} -n ${v1_name} vcl.list |
+	grep "auto/cold.*vcl1" >/dev/null
+}



From lkarsten at varnish-software.com  Thu Jan 14 14:15:08 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:08 +0100
Subject: [4.1] 2e678b1 Wording
Message-ID: <E1aJig8-0005KI-II@project.varnish-software.com>


commit 2e678b1647759672e13113fece558fff16b88015
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Sat Dec 5 17:14:19 2015 +0100

    Wording

diff --git a/bin/varnishtest/tests/v00045.vtc b/bin/varnishtest/tests/v00045.vtc
index b1754ee..1cc1259 100644
--- a/bin/varnishtest/tests/v00045.vtc
+++ b/bin/varnishtest/tests/v00045.vtc
@@ -2,7 +2,7 @@ varnishtest "Hold a reference to a VCL after a COLD event"
 
 server s1 -start
 
-# Load and use a new VCL that will hold a reference
+# Load and use a VCL that will hold a reference
 varnish v1 -vcl+backend {
 	import ${vmod_debug};
 	sub vcl_init {



From lkarsten at varnish-software.com  Thu Jan 14 14:15:08 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:08 +0100
Subject: [4.1] fb6088b Polish
Message-ID: <E1aJig8-0005KR-Je@project.varnish-software.com>


commit fb6088bc2ba00b361435deda67254fae56fb6b70
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Sat Dec 5 17:20:37 2015 +0100

    Polish

diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index 664c2c1..54effd1 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -432,7 +432,6 @@ static void
 vcl_set_state(struct vcl *vcl, const char *state)
 {
 	struct vrt_ctx ctx;
-	const char *target;
 	unsigned hand = 0;
 
 	ASSERT_CLI();
@@ -445,8 +444,7 @@ vcl_set_state(struct vcl *vcl, const char *state)
 	switch(state[0]) {
 	case '0':
 		assert(vcl->temp != vcl_temp_cold);
-		target = vcl->busy ? vcl_temp_busy : vcl_temp_cold;
-		if (target == vcl_temp_cold && (vcl->temp == vcl_temp_warm ||
+		if (vcl->busy == 0 && (vcl->temp == vcl_temp_warm ||
 		    vcl->temp == vcl_temp_busy)) {
 
 			vcl->temp = vcl->refcount ? vcl_temp_cooling :



From lkarsten at varnish-software.com  Thu Jan 14 14:15:08 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:08 +0100
Subject: [4.1] e64fc15 Polish
Message-ID: <E1aJig8-0005Kd-Lb@project.varnish-software.com>


commit e64fc15e47c0c948d413613d8e4c77ae849a5668
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Mon Dec 7 12:14:12 2015 +0100

    Polish

diff --git a/bin/varnishd/mgt/mgt_vcl.c b/bin/varnishd/mgt/mgt_vcl.c
index b12bdb6..37359f2 100644
--- a/bin/varnishd/mgt/mgt_vcl.c
+++ b/bin/varnishd/mgt/mgt_vcl.c
@@ -416,8 +416,8 @@ mcf_vcl_list(struct cli *cli, const char * const *av, void *priv)
 		VTAILQ_FOREACH(vp, &vclhead, list) {
 			VCLI_Out(cli, "%-10s %4s/%-8s %6s %s\n",
 			    vp == active_vcl ? "active" : "available",
-			    vp->state,
-			    vp->warm ? "warm" : "cold", "", vp->name);
+			    vp->state, vp->warm ? "warm" : "cold", "",
+			    vp->name);
 		}
 	}
 }



From lkarsten at varnish-software.com  Thu Jan 14 14:15:08 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:08 +0100
Subject: [4.1] b2bd6aa typo
Message-ID: <E1aJig8-0005Kj-N3@project.varnish-software.com>


commit b2bd6aa3be1e3e170a14e522a2435a3fc74707f9
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Dec 8 09:32:02 2015 +0000

    typo

diff --git a/lib/libvmod_debug/vmod.vcc b/lib/libvmod_debug/vmod.vcc
index 42159fb..74c8c12 100644
--- a/lib/libvmod_debug/vmod.vcc
+++ b/lib/libvmod_debug/vmod.vcc
@@ -78,7 +78,7 @@ Test object
 .. NOTE: .enum before .foo as part of test r01332.vtc
 $Method VOID .enum(ENUM { phk, des, kristian, mithrandir, martin })
 
-Testing that emums work as part of object and that the parser isn't
+Testing that enums work as part of object and that the parser isn't
 (too) buggy.
 
 $Method STRING .foo(STRING why)



From lkarsten at varnish-software.com  Thu Jan 14 14:15:08 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:08 +0100
Subject: [4.1] 7a67bf1 Pass the busyobj pointer also for vdir_pick_be.
Message-ID: <E1aJig8-0005Kw-Oe@project.varnish-software.com>


commit 7a67bf1d8f17d728a75a5b447a5993a1314dbbcb
Author: Dag Haavi Finstad <daghf at varnish-software.com>
Date:   Mon Dec 7 16:19:26 2015 +0100

    Pass the busyobj pointer also for vdir_pick_be.
    
    It's needed for the vdi_healthy_f invocation.

diff --git a/lib/libvmod_directors/hash.c b/lib/libvmod_directors/hash.c
index a858a09..9aa62f0 100644
--- a/lib/libvmod_directors/hash.c
+++ b/lib/libvmod_directors/hash.c
@@ -97,6 +97,7 @@ vmod_hash_backend(VRT_CTX, struct vmod_directors_hash *rr,
 	double r;
 
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
+	CHECK_OBJ_ORNULL(ctx->bo, BUSYOBJ_MAGIC);
 
 	CHECK_OBJ_NOTNULL(rr, VMOD_DIRECTORS_HASH_MAGIC);
 	SHA256_Init(&sha_ctx);
@@ -113,6 +114,6 @@ vmod_hash_backend(VRT_CTX, struct vmod_directors_hash *rr,
 	r = vbe32dec(sha256);
 	r = scalbn(r, -32);
 	assert(r >= 0 && r <= 1.0);
-	be = vdir_pick_be(rr->vd, r);
+	be = vdir_pick_be(rr->vd, r, ctx->bo);
 	return (be);
 }
diff --git a/lib/libvmod_directors/random.c b/lib/libvmod_directors/random.c
index 2f29887..980ef94 100644
--- a/lib/libvmod_directors/random.c
+++ b/lib/libvmod_directors/random.c
@@ -71,7 +71,7 @@ vmod_random_resolve(const struct director *dir, struct worker *wrk,
 	CAST_OBJ_NOTNULL(rr, dir->priv, VMOD_DIRECTORS_RANDOM_MAGIC);
 	r = scalbn(random(), -31);
 	assert(r >= 0 && r < 1.0);
-	be = vdir_pick_be(rr->vd, r);
+	be = vdir_pick_be(rr->vd, r, bo);
 	return (be);
 }
 
diff --git a/lib/libvmod_directors/vdir.c b/lib/libvmod_directors/vdir.c
index 411df34..1112bad 100644
--- a/lib/libvmod_directors/vdir.c
+++ b/lib/libvmod_directors/vdir.c
@@ -177,7 +177,7 @@ vdir_pick_by_weight(const struct vdir *vd, double w,
 }
 
 VCL_BACKEND
-vdir_pick_be(struct vdir *vd, double w)
+vdir_pick_be(struct vdir *vd, double w, const struct busyobj *bo)
 {
 	unsigned u;
 	double tw = 0.0;
@@ -185,7 +185,7 @@ vdir_pick_be(struct vdir *vd, double w)
 
 	vdir_lock(vd);
 	for (u = 0; u < vd->n_backend; u++) {
-		if (vd->backend[u]->healthy(vd->backend[u], NULL, NULL)) {
+		if (vd->backend[u]->healthy(vd->backend[u], bo, NULL)) {
 			vbit_clr(vd->vbm, u);
 			tw += vd->weight[u];
 		} else
diff --git a/lib/libvmod_directors/vdir.h b/lib/libvmod_directors/vdir.h
index 2e1cfe1..b420f68 100644
--- a/lib/libvmod_directors/vdir.h
+++ b/lib/libvmod_directors/vdir.h
@@ -49,4 +49,4 @@ void vdir_unlock(struct vdir *vd);
 unsigned vdir_add_backend(struct vdir *, VCL_BACKEND be, double weight);
 unsigned vdir_any_healthy(struct vdir *, const struct busyobj *,
     double *changed);
-VCL_BACKEND vdir_pick_be(struct vdir *, double w);
+VCL_BACKEND vdir_pick_be(struct vdir *, double w, const struct busyobj *);



From lkarsten at varnish-software.com  Thu Jan 14 14:15:08 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:08 +0100
Subject: [4.1] 3dc012b Add missing double quotes
Message-ID: <E1aJig8-0005L2-Qw@project.varnish-software.com>


commit 3dc012b7a4d25ad7a96a48c9fac763ce62c1f776
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Thu Dec 10 01:44:35 2015 +0000

    Add missing double quotes

diff --git a/lib/libvmod_std/vmod.vcc b/lib/libvmod_std/vmod.vcc
index b0dba07..61c334f 100644
--- a/lib/libvmod_std/vmod.vcc
+++ b/lib/libvmod_std/vmod.vcc
@@ -64,7 +64,7 @@ Description
 	request so probably want to set it on every request should you
 	utilize it.
 Example
-	| if (req.url ~ ^/slow/) {
+	| if (req.url ~ "^/slow/") {
 	|	std.set_ip_tos(0);
 	| }
 



From lkarsten at varnish-software.com  Thu Jan 14 14:15:08 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:08 +0100
Subject: [4.1] 324f762 Reorder some includes to be alphabetical
Message-ID: <E1aJig8-0005LB-T7@project.varnish-software.com>


commit 324f76270c9227f9777d36d709d7a5a143bb8ec7
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu Dec 10 09:21:33 2015 +0000

    Reorder some includes to be alphabetical
    
    Conflicts:
    	bin/varnishd/cache/cache.h
    
    4.1 merge note: req_body_iter_f interface kept.

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index e7700d0..76357f8 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -679,17 +679,6 @@ ssize_t VBO_waitlen(struct worker *, struct busyobj *, ssize_t l);
 void VBO_setstate(struct busyobj *bo, enum busyobj_state_e next);
 void VBO_waitstate(struct busyobj *bo, enum busyobj_state_e want);
 
-/* cache_req_body.c */
-int VRB_Ignore(struct req *req);
-ssize_t VRB_Cache(struct req *req, ssize_t maxsize);
-typedef int (req_body_iter_f)(struct req *, void *priv, void *ptr, size_t);
-ssize_t VRB_Iterate(struct req *req, req_body_iter_f *func, void *priv);
-void VRB_Free(struct req *req);
-
-/* cache_req_fsm.c [CNT] */
-enum req_fsm_nxt CNT_Request(struct worker *, struct req *);
-void CNT_AcctLogCharge(struct dstat *, struct req *);
-
 /* cache_cli.c [CLI] */
 extern pthread_t cli_thread;
 #define ASSERT_CLI() do {assert(pthread_self() == cli_thread);} while (0)
@@ -912,6 +901,17 @@ void Req_Release(struct req *);
 int Req_Cleanup(struct sess *sp, struct worker *wrk, struct req *req);
 void Req_Fail(struct req *req, enum sess_close reason);
 
+/* cache_req_body.c */
+int VRB_Ignore(struct req *req);
+ssize_t VRB_Cache(struct req *req, ssize_t maxsize);
+typedef int (req_body_iter_f)(struct req *, void *priv, void *ptr, size_t);
+ssize_t VRB_Iterate(struct req *req, req_body_iter_f *func, void *priv);
+void VRB_Free(struct req *req);
+
+/* cache_req_fsm.c [CNT] */
+enum req_fsm_nxt CNT_Request(struct worker *, struct req *);
+void CNT_AcctLogCharge(struct dstat *, struct req *);
+
 /* cache_session.c [SES] */
 struct sess *SES_New(struct pool *);
 void SES_Close(struct sess *sp, enum sess_close reason);



From lkarsten at varnish-software.com  Thu Jan 14 14:15:08 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:08 +0100
Subject: [4.1] 069420b Whitespace
Message-ID: <E1aJig8-0005LK-Vw@project.varnish-software.com>


commit 069420bcfcc7d5fd26eacafbb776c59cb63c822e
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Thu Dec 10 11:10:13 2015 +0000

    Whitespace

diff --git a/Makefile.inc.phk b/Makefile.inc.phk
index c864efa..680bab1 100644
--- a/Makefile.inc.phk
+++ b/Makefile.inc.phk
@@ -1,6 +1,6 @@
 
 .PHONY: default
-default:	all 
+default:	all
 
 WARNS	?=	1
 
@@ -39,7 +39,7 @@ LIB_READLINE = ${CFG_LINEDISC}
 #######################################################################
 ifdef USE_THREADS
 CFLAGS	+= -pthread -D_THREAD_SAFE -g -O2
-PROG_LDFLAGS += -pthread -D_THREAD_SAFE -g -O2 
+PROG_LDFLAGS += -pthread -D_THREAD_SAFE -g -O2
 endif
 
 ifdef USE_DLOPEN
@@ -164,7 +164,7 @@ ifeq ($(TGT), all)
 include _.deplib
 endif
 
-$(LIBNAME).a:	$(LIB_OBJ) 
+$(LIBNAME).a:	$(LIB_OBJ)
 	$(AR) -rc $(LIBNAME).a $^
 	ranlib $(LIBNAME).a
 
diff --git a/bin/Makefile.phk b/bin/Makefile.phk
index ce538a1..66816ff 100644
--- a/bin/Makefile.phk
+++ b/bin/Makefile.phk
@@ -1,10 +1,10 @@
-SUBDIRS += varnishadm 
-SUBDIRS += varnishd 
+SUBDIRS += varnishadm
+SUBDIRS += varnishd
 SUBDIRS += varnishlog
 SUBDIRS += varnishncsa
-SUBDIRS += varnishtest 
+SUBDIRS += varnishtest
 SUBDIRS += varnishstat
-SUBDIRS += varnishhist 
+SUBDIRS += varnishhist
 SUBDIRS += varnishtop
 
 TOPDIR = $(CURDIR)/..
diff --git a/bin/flint.lnt b/bin/flint.lnt
index 0bc6639..0c9412c 100644
--- a/bin/flint.lnt
+++ b/bin/flint.lnt
@@ -35,7 +35,7 @@
 -elib(659)		// no tokens after struct def.
 -elib(123)		// macro def. with arg at, (just warn)
 
--emacro(702, WEXITSTATUS)	// signed shift right 
+-emacro(702, WEXITSTATUS)	// signed shift right
 
 -e786	// String concatenation within initializer
 -e726	// Extraneous comma ignored
diff --git a/bin/varnishadm/Makefile.phk b/bin/varnishadm/Makefile.phk
index 6d6cb2a..368b80a 100644
--- a/bin/varnishadm/Makefile.phk
+++ b/bin/varnishadm/Makefile.phk
@@ -4,7 +4,7 @@ LD_ADD	+= ${LIB_VARNISHAPI}
 LD_ADD	+= ${LIB_VARNISH}
 LD_ADD	+= ${LIB_PCRE}
 LD_ADD	+= ${LIB_READLINE}
-LD_ADD	+= -lm 
+LD_ADD	+= -lm
 
 RST_MAN +=  $(TOPDIR)/doc/sphinx/reference/varnishadm.rst
 
diff --git a/bin/varnishd/Makefile.am b/bin/varnishd/Makefile.am
index d252c1b..0648822 100644
--- a/bin/varnishd/Makefile.am
+++ b/bin/varnishd/Makefile.am
@@ -97,7 +97,7 @@ varnishd_SOURCES = \
 	waiter/cache_waiter_epoll.c \
 	waiter/cache_waiter_kqueue.c \
 	waiter/cache_waiter_poll.c \
-	waiter/cache_waiter_ports.c 
+	waiter/cache_waiter_ports.c
 
 noinst_HEADERS = \
 	builtin_vcl.h \
diff --git a/bin/varnishd/flint.lnt b/bin/varnishd/flint.lnt
index 38453d1..200e37b 100644
--- a/bin/varnishd/flint.lnt
+++ b/bin/varnishd/flint.lnt
@@ -118,9 +118,9 @@
 -e455	// thread lock
 -e458	// unprotected read
 -e728	// Symbol ... not explicitly initialized
--e716	// while(1) ... 
--e785	// Too few initializers for aggregate 
--e850	// for loop index variable '___' whose type category is '___' 
+-e716	// while(1) ...
+-e785	// Too few initializers for aggregate
+-e850	// for loop index variable '___' whose type category is '___'
 	// is modified in body of the for loop that began at '___'
 
 
diff --git a/bin/varnishd/vclflint.sh b/bin/varnishd/vclflint.sh
index cd3e7d9..24d47bf 100755
--- a/bin/varnishd/vclflint.sh
+++ b/bin/varnishd/vclflint.sh
@@ -10,4 +10,4 @@ else
 	echo "usage!" 1>&2
 fi
 
-flexelint vclflint.lnt /tmp/_.c 
+flexelint vclflint.lnt /tmp/_.c
diff --git a/bin/varnishlog/flint.lnt b/bin/varnishlog/flint.lnt
index c90f53b..f2b336d 100644
--- a/bin/varnishlog/flint.lnt
+++ b/bin/varnishlog/flint.lnt
@@ -3,11 +3,11 @@
 -e747	// 16      Info    747     Significant prototype coercion (___) ___ to ___
 
 -e763	// Redundant declaration for symbol '...' previously declared
--e716	// while(1) ... 
+-e716	// while(1) ...
 
--e457 // unprotected write access 
--e459 // unprotected access 
--e458 // unprotected access 
+-e457 // unprotected write access
+-e459 // unprotected access
+-e458 // unprotected access
 
 -e732   // Loss of sign (arg. no. 2) (int to unsigned
 -e737	// [45]  Loss of sign in promotion from int to unsigned
diff --git a/bin/varnishncsa/flint.lnt b/bin/varnishncsa/flint.lnt
index ce8b3f4..fd2c864 100644
--- a/bin/varnishncsa/flint.lnt
+++ b/bin/varnishncsa/flint.lnt
@@ -4,8 +4,8 @@
 -e763	// Redundant declaration for symbol '...' previously declared
 -e726	// Extraneous comma ignored
 -e728	// Symbol ... not explicitly initialized
--e716	// while(1) ... 
--e785	// Too few initializers for aggregate 
+-e716	// while(1) ...
+-e785	// Too few initializers for aggregate
 -e786	// String concatenation within initializer
 
 // cache_center.c
diff --git a/bin/varnishstat/flint.lnt b/bin/varnishstat/flint.lnt
index 005b4aa..509e13a 100644
--- a/bin/varnishstat/flint.lnt
+++ b/bin/varnishstat/flint.lnt
@@ -7,11 +7,11 @@
 -e747	// 16      Info    747     Significant prototype coercion (___) ___ to ___
 
 -e763	// Redundant declaration for symbol '...' previously declared
--e716	// while(1) ... 
+-e716	// while(1) ...
 
--e457 // unprotected write access 
--e459 // unprotected access 
--e458 // unprotected access 
+-e457 // unprotected write access
+-e459 // unprotected access
+-e458 // unprotected access
 
 -e732   // Loss of sign (arg. no. 2) (int to unsigned
 -e737	// [45]  Loss of sign in promotion from int to unsigned
diff --git a/bin/varnishtest/flint.lnt b/bin/varnishtest/flint.lnt
index 8ee39d2..b7306f1 100644
--- a/bin/varnishtest/flint.lnt
+++ b/bin/varnishtest/flint.lnt
@@ -16,12 +16,12 @@
 -e679	// Suspicious Truncation in arithmetic expression combining with pointer
 
 -e763	// Redundant declaration for symbol '...' previously declared
--e716	// while(1) ... 
+-e716	// while(1) ...
 
--e457 // unprotected write access 
--e459 // unprotected access 
--e458 // unprotected access 
-// 
+-e457 // unprotected write access
+-e459 // unprotected access
+-e458 // unprotected access
+//
 -e732   // Loss of sign (arg. no. 2) (int to unsigned
 -e737	// [45]  Loss of sign in promotion from int to unsigned
 -e713	// Loss of precision (assignment) (unsigned long long to long long)
diff --git a/bin/varnishtop/flint.lnt b/bin/varnishtop/flint.lnt
index 5f41983..4920e47 100644
--- a/bin/varnishtop/flint.lnt
+++ b/bin/varnishtop/flint.lnt
@@ -3,11 +3,11 @@
 -e747	// 16      Info    747     Significant prototype coercion (___) ___ to ___
 
 -e763	// Redundant declaration for symbol '...' previously declared
--e716	// while(1) ... 
+-e716	// while(1) ...
 
--e457 // unprotected write access 
--e459 // unprotected access 
--e458 // unprotected access 
+-e457 // unprotected write access
+-e459 // unprotected access
+-e458 // unprotected access
 
 -e732   // Loss of sign (arg. no. 2) (int to unsigned
 -e737	// [45]  Loss of sign in promotion from int to unsigned
diff --git a/config.phk b/config.phk
index dea0eed..470faec 100644
--- a/config.phk
+++ b/config.phk
@@ -72,7 +72,7 @@ if true ; then
 		sys/mount.h \
 		sys/statvfs.h \
 		sys/vfs.h \
-		umem.h 
+		umem.h
 	do
 		if [ -f /usr/include/$i ] ; then
 			n=`echo $i | tr '[a-z/.]' '[A-Z__]'`
@@ -109,7 +109,7 @@ fi
 VCSF=include/vcs_version.h
 VMAV=include/vmod_abi.h
 
-if [ -d ./.git ] ; then 
+if [ -d ./.git ] ; then
 	V=`git show -s --pretty=format:%h`
 	B=`git rev-parse --abbrev-ref HEAD`
 else
@@ -147,7 +147,7 @@ fi
 ask_compiler() {
 	a=`echo "$2" |
 	    ${CC-cc} -E - -DWARNS=${WARNS-1} |
-	    sed -e '/^#/d' -e '/^$/d' -e 's/"//g' -e 's/~/"/g' ` 
+	    sed -e '/^#/d' -e '/^$/d' -e 's/"//g' -e 's/~/"/g' `
 	echo "$1='" $a "'" >> _.cache
 }
 
@@ -301,5 +301,5 @@ exit 0
 # HAVE_LIBUMEM -
 # USE_PCRE_JIT -
 # _FILE_OFFSET_BITS -
-# 
+#
 # #echo 'CF_CFLAGS="-Wall"'
diff --git a/configure.ac b/configure.ac
index d4f1923..8c753a3 100644
--- a/configure.ac
+++ b/configure.ac
@@ -179,7 +179,7 @@ AC_CHECK_HEADERS([edit/readline/readline.h],
 	[AC_DEFINE([HAVE_LIBEDIT], [1], [Define if we have libedit])
 	LIBEDIT_LIBS="-ledit"],
 
-	[PKG_CHECK_MODULES([LIBEDIT], [libedit], 
+	[PKG_CHECK_MODULES([LIBEDIT], [libedit],
 		# having the module does not imply having the header
 		[AC_CHECK_HEADERS([editline/readline.h],
 				  [AC_DEFINE([HAVE_LIBEDIT], [1], [Define if we have libedit])],
@@ -274,7 +274,7 @@ AC_CACHE_CHECK([whether we have support for visibility attributes],
 if test "$ac_cv_have_viz" = no; then
 	libvgz_extra_cflags="-DNO_VIZ"
 	AC_SUBST(libvgz_extra_cflags)
-fi 
+fi
 CFLAGS="${save_CFLAGS}"
 
 # Use jemalloc on Linux
@@ -333,7 +333,7 @@ AC_SYS_LARGEFILE
 save_LIBS="${LIBS}"
 LIBS="${LIBS} ${RT_LIBS}"
 AC_CHECK_FUNCS([clock_gettime])
-AC_CHECK_FUNCS([gethrtime]) 
+AC_CHECK_FUNCS([gethrtime])
 LIBS="${save_LIBS}"
 
 # --enable-kqueue
diff --git a/doc/sphinx/Makefile.phk b/doc/sphinx/Makefile.phk
index 8ed14ae..35bea90 100644
--- a/doc/sphinx/Makefile.phk
+++ b/doc/sphinx/Makefile.phk
@@ -189,7 +189,7 @@ EXTRA_DIST = \
 	include/varnishhist_options.rst \
 	include/varnishhist_synopsis.rst \
 	reference/vmod_std.generated.rst \
-	reference/vmod_directors.generated.rst 
+	reference/vmod_directors.generated.rst
 
 
 dist-hook:
diff --git a/doc/sphinx/glossary/index.rst b/doc/sphinx/glossary/index.rst
index a3d9761..7877f60 100644
--- a/doc/sphinx/glossary/index.rst
+++ b/doc/sphinx/glossary/index.rst
@@ -4,7 +4,7 @@
 Varnish Glossary
 ================
 
-.. glossary:: 
+.. glossary::
    :sorted:
 
    ..
diff --git a/doc/sphinx/installation/bugs.rst b/doc/sphinx/installation/bugs.rst
index b5fd756..091b7c7 100644
--- a/doc/sphinx/installation/bugs.rst
+++ b/doc/sphinx/installation/bugs.rst
@@ -19,7 +19,7 @@ do not have a way to simulate high levels of real-life web-traffic,
 so telling us to "have 10.000 clients hit at once" does not really
 allow us to reproduce.
 
-To report a bug please follow the suggested procedure described in the "Trouble Tickets" 
+To report a bug please follow the suggested procedure described in the "Trouble Tickets"
 section of the documentation (above).
 
 Roughly we categorize bugs in to three kinds of bugs (described below) with Varnish. The information
@@ -119,7 +119,7 @@ furiously on some futile condition.
 Commands like ``top -H`` or ``ps -Haxlw`` or ``ps -efH`` should be
 able to figure that out.
 
-.. XXX:Maybe a short description of what valuable information the various commands above generates? /benc 
+.. XXX:Maybe a short description of what valuable information the various commands above generates? /benc
 
 
 If one or more threads are spinning, use ``strace`` or ``ktrace`` or ``truss``
diff --git a/doc/sphinx/installation/help.rst b/doc/sphinx/installation/help.rst
index 22cd106..ba6dbd8 100644
--- a/doc/sphinx/installation/help.rst
+++ b/doc/sphinx/installation/help.rst
@@ -8,7 +8,7 @@ this process as much as possible.
 
 But before you grab hold of us, spend a moment composing your thoughts and
 formulate your question. From our perspective there is nothing as pointless as simply telling
-us "Varnish does not work for me" with no further information. This does not give us any relevant information to use when trying to figure out whats wrong. 
+us "Varnish does not work for me" with no further information. This does not give us any relevant information to use when trying to figure out whats wrong.
 
 And before you even do that, do a couple of searches to see if your
 question is already answered, if it has been, you will get your answer
@@ -78,7 +78,7 @@ bug in Varnish.  Ask on IRC first if you are in doubt.
 
 The reason for this policy, is to avoid bugs being drowned in a
 pile of other `issues`, feature suggestions for future releases, and double postings of calls for help
-from people who forgot to check back on already opened Tickets. 
+from people who forgot to check back on already opened Tickets.
 
 .. XXX: Not sure what you want with the last sentence above. benc
 
diff --git a/doc/sphinx/installation/platformnotes.rst b/doc/sphinx/installation/platformnotes.rst
index 0eca96f..741d98b 100644
--- a/doc/sphinx/installation/platformnotes.rst
+++ b/doc/sphinx/installation/platformnotes.rst
@@ -13,7 +13,7 @@ On RHEL6 Transparent Hugepage kernel support is enabled by default.
 This is known to cause sporadic crashes of Varnish.
 
 It is recommended to disable transparent hugepages on affected systems. This
-can be done with 
+can be done with
 ``echo "never" > /sys/kernel/mm/redhat_transparent_hugepage/enabled`` (runtime) and changes to
 `/etc/sysctl.conf` (persisted.)
 
diff --git a/doc/sphinx/phk/autocrap.rst b/doc/sphinx/phk/autocrap.rst
index e97ac4c..86a2dd1 100644
--- a/doc/sphinx/phk/autocrap.rst
+++ b/doc/sphinx/phk/autocrap.rst
@@ -47,7 +47,7 @@ right before you can even think about compiling the source code.
 
 Second, it doesn't actually work, you still have to do the hard work
 and figure out the right way to explain to the autocrap tools what
-you are trying to do and how to do it, only you have to do so in 
+you are trying to do and how to do it, only you have to do so in
 a language which is used to produce M4 macro invocations etc. etc.
 
 In the meantime, the UNIX diversity has shrunk from 50+ significantly
diff --git a/doc/sphinx/phk/backends.rst b/doc/sphinx/phk/backends.rst
index 952a791..2a2a348 100644
--- a/doc/sphinx/phk/backends.rst
+++ b/doc/sphinx/phk/backends.rst
@@ -52,7 +52,7 @@ VCLs where backend "b1" points at two different physical machines,
 for instance when we migrate or upgrade the backend.
 
 The identity of the state than can be shared is therefore the triplet:
-	{VCL-name, IPv4+port, IPv6+port} 
+	{VCL-name, IPv4+port, IPv6+port}
 
 No Information without Representation
 =====================================
diff --git a/doc/sphinx/phk/barriers.rst b/doc/sphinx/phk/barriers.rst
index 4eac78a..e85e81a 100644
--- a/doc/sphinx/phk/barriers.rst
+++ b/doc/sphinx/phk/barriers.rst
@@ -48,7 +48,7 @@ barriers between the various components::
 The really Important Barrier
 ============================
 
-The central actor in Varnish is the Manager process, "MGR", which is the 
+The central actor in Varnish is the Manager process, "MGR", which is the
 process the administrator "(ADMIN)" starts to get web-cache service.
 
 Having been there myself, I do not subscribe to the "I feel cool and important
diff --git a/doc/sphinx/phk/gzip.rst b/doc/sphinx/phk/gzip.rst
index c5e4aad..68ff0f0 100644
--- a/doc/sphinx/phk/gzip.rst
+++ b/doc/sphinx/phk/gzip.rst
@@ -47,7 +47,7 @@ always set to:
 
 	Accept-Encoding: gzip
 
-Even if this particular client does not support 
+Even if this particular client does not support
 
 To always entice the backend into sending us gzip'ed content.
 
diff --git a/doc/sphinx/phk/http20.rst b/doc/sphinx/phk/http20.rst
index c2b4d17..525f0f3 100644
--- a/doc/sphinx/phk/http20.rst
+++ b/doc/sphinx/phk/http20.rst
@@ -257,7 +257,7 @@ and repeat markers, rather than throwing everything after deflate
 and hope for a miracle.
 
 I think this protocol is the best basis to start from, but like
-the other two, it has a long way to go, before it can truly 
+the other two, it has a long way to go, before it can truly
 earn the name HTTP/2.0.
 
 
diff --git a/doc/sphinx/phk/platforms.rst b/doc/sphinx/phk/platforms.rst
index 18ee017..e85f925 100644
--- a/doc/sphinx/phk/platforms.rst
+++ b/doc/sphinx/phk/platforms.rst
@@ -39,7 +39,7 @@ We care about these platforms because our users use them and
 because they deliver a lot of bang for the buck with Varnish.
 
 These platforms are in our "tinderbox" tests, we use them ourselves
-and they pass all regression tests all the time. 
+and they pass all regression tests all the time.
 Platform specific bug reports gets acted on.
 
 *FreeBSD*
diff --git a/doc/sphinx/phk/spdy.rst b/doc/sphinx/phk/spdy.rst
index 7cc1f4f..7f70236 100644
--- a/doc/sphinx/phk/spdy.rst
+++ b/doc/sphinx/phk/spdy.rst
@@ -61,7 +61,7 @@ Treaty and the convention about chemical and biological weapons.)
 
 So my insight about Varnish, that it has to become a socket-wrench-like
 toolchest for doing things with HTTP traffic, will probably elicit a lot
-of "duh!" reactions, until people, including me, understand the 
+of "duh!" reactions, until people, including me, understand the
 ramifications more fully.
 
 Things you cannot do with Varnish today
diff --git a/doc/sphinx/phk/three-zero.rst b/doc/sphinx/phk/three-zero.rst
index 41499b0..c1fd3b8 100644
--- a/doc/sphinx/phk/three-zero.rst
+++ b/doc/sphinx/phk/three-zero.rst
@@ -53,7 +53,7 @@ shows a nice trend on Google and #varnish confuses the heck out of
 teenage girls and wood workers on Twitter, so clearly I am doing
 something right.
 
-I still worry about the `The Fraud Police 
+I still worry about the `The Fraud Police
 <http://www.theshadowbox.net/forum/index.php?topic=18041.0>`_ though,
 "I have no idea what I'm doing, and I totally make shit up as I go
 along." is a disturbingly precise summary of how I feel about my
diff --git a/doc/sphinx/phk/wanton_destruction.rst b/doc/sphinx/phk/wanton_destruction.rst
index 4289cf1..8ad757d 100644
--- a/doc/sphinx/phk/wanton_destruction.rst
+++ b/doc/sphinx/phk/wanton_destruction.rst
@@ -24,7 +24,7 @@ rebuild it, to become the house we want now.
 
 For one thing, doing major surgery on a house while you live in it
 is double-minus-fun and it always slows the rebuild project down
-when you have to keep at least one toilet working and sanitary and 
+when you have to keep at least one toilet working and sanitary and
 be able to cook and sleep on the building site.
 
 So we'll be building a new house on a plot of land on the other
@@ -40,7 +40,7 @@ Using a major piece of software, is like moving into a house: You
 need to adapt your life and the house or the software to each other,
 since nothing is ever quite perfect, there will be limitations.
 
-And those limitations affect how you think:  If you live in a 
+And those limitations affect how you think:  If you live in a
 2 bedroom apartment, you won't even be considering inviting 80
 guests to a party.
 
@@ -55,7 +55,7 @@ the next couple of years, it'll be called Varnish V4 and that means
 that you will have to decide who gets which room and where to store
 the towels and grandmothers old china, all over again.
 
-I'm sure you'll hate me for it, "Why do we have to move ?", "It 
+I'm sure you'll hate me for it, "Why do we have to move ?", "It
 really wasn't that bad after all" and so on and so forth.
 But if I do my job right, that will soon turn into "Ohh, that's
 pretty neat, I always wanted one of those..." and "Hey... Watch me
diff --git a/doc/sphinx/tutorial/backend_servers.rst b/doc/sphinx/tutorial/backend_servers.rst
index ce39bc1..7d7b70a 100644
--- a/doc/sphinx/tutorial/backend_servers.rst
+++ b/doc/sphinx/tutorial/backend_servers.rst
@@ -16,7 +16,7 @@ If you've been following the tutorial there is probably a section of
 the configuration that looks like this::
 
   vcl 4.0;
-  
+
   backend default {
       .host = "www.varnish-cache.org";
       .port = "80";
@@ -43,7 +43,7 @@ localhost, port 8080.::
 
 Varnish can have several backends defined and can even join several backends
 together into clusters of backends for load balancing purposes, having Varnish
-pick one backend based on different algorithms. 
+pick one backend based on different algorithms.
 
 Next, let's have a look at some of what makes Varnish unique and what you can do with it.
 
diff --git a/doc/sphinx/tutorial/starting_varnish.rst b/doc/sphinx/tutorial/starting_varnish.rst
index 8d9b64c..b90e6b6 100644
--- a/doc/sphinx/tutorial/starting_varnish.rst
+++ b/doc/sphinx/tutorial/starting_varnish.rst
@@ -39,7 +39,7 @@ of it is commented out but there is some text that is not. It will
 probably look like this::
 
   vcl 4.0;
-  
+
   backend default {
       .host = "127.0.0.1";
       .port = "8080";
@@ -49,7 +49,7 @@ We'll change it and make it point to something that works. Hopefully
 http://www.varnish-cache.org/ is up. Let's use that. Replace the text with::
 
   vcl 4.0;
-  
+
   backend default {
       .host = "www.varnish-cache.org";
       .port = "80";
diff --git a/doc/sphinx/users-guide/command-line.rst b/doc/sphinx/users-guide/command-line.rst
index 854fb32..ad04c02 100644
--- a/doc/sphinx/users-guide/command-line.rst
+++ b/doc/sphinx/users-guide/command-line.rst
@@ -3,8 +3,8 @@
 Important command line arguments
 --------------------------------
 
-There a two command line arguments you have to set when starting Varnish, these are: 
-* what TCP port to serve HTTP from, and 
+There a two command line arguments you have to set when starting Varnish, these are:
+* what TCP port to serve HTTP from, and
 * where the backend server can be contacted.
 
 If you have installed Varnish through using a provided operating system bound package,
diff --git a/doc/sphinx/users-guide/compression.rst b/doc/sphinx/users-guide/compression.rst
index edd3c6f..d88e351 100644
--- a/doc/sphinx/users-guide/compression.rst
+++ b/doc/sphinx/users-guide/compression.rst
@@ -4,7 +4,7 @@ Compression
 -----------
 
 In Varnish 3.0 we introduced native support for compression, using gzip
-encoding. *Before* 3.0, Varnish would never compress objects. 
+encoding. *Before* 3.0, Varnish would never compress objects.
 
 In Varnish 4.0 compression defaults to "on", meaning that it tries to
 be smart and do the sensible thing.
diff --git a/doc/sphinx/users-guide/devicedetection.rst b/doc/sphinx/users-guide/devicedetection.rst
index 67f711a..8027909 100644
--- a/doc/sphinx/users-guide/devicedetection.rst
+++ b/doc/sphinx/users-guide/devicedetection.rst
@@ -7,7 +7,7 @@ Device detection is figuring out what kind of content to serve to a
 client based on the User-Agent string supplied in a request.
 
 Use cases for this are for example to send size reduced files to mobile
-clients with small screens and on high latency networks, or to 
+clients with small screens and on high latency networks, or to
 provide a streaming video codec that the client understands.
 
 There are a couple of typical strategies to use for this type of scenario:
@@ -16,7 +16,7 @@ There are a couple of typical strategies to use for this type of scenario:
 3) Change the backend request so that the backend sends tailored content.
 
 To perhaps make the strategies easier to understand, we, in this context, assume
-that the `req.http.X-UA-Device` header is present and unique per client class. 
+that the `req.http.X-UA-Device` header is present and unique per client class.
 
 Setting this header can be as simple as::
 
@@ -35,7 +35,7 @@ https://github.com/varnish/varnish-devicedetect/.
 Serve the different content on the same URL
 -------------------------------------------
 
-The tricks involved are: 
+The tricks involved are:
 1. Detect the client (pretty simple, just include `devicedetect.vcl` and call
 it).
 2. Figure out how to signal the backend the client class. This
@@ -55,7 +55,7 @@ Example 1: Send HTTP header to backend
 
 The basic case is that Varnish adds the 'X-UA-Device' HTTP header on the backend
 requests, and the backend mentions in the response 'Vary' header that the content
-is dependant on this header. 
+is dependant on this header.
 
 Everything works out of the box from Varnish' perspective.
 
@@ -63,7 +63,7 @@ Everything works out of the box from Varnish' perspective.
 
 VCL::
 
-    sub vcl_recv { 
+    sub vcl_recv {
         # call some detection engine that set req.http.X-UA-Device
     }
     # req.http.X-UA-Device is copied by Varnish into bereq.http.X-UA-Device
@@ -73,15 +73,15 @@ VCL::
     # use the same cached object for all U-As that map to the same X-UA-Device.
     #
     # If the backend does not mention in Vary that it has crafted special
-    # content based on the User-Agent (==X-UA-Device), add it. 
+    # content based on the User-Agent (==X-UA-Device), add it.
     # If your backend does set Vary: User-Agent, you may have to remove that here.
     sub vcl_backend_response {
         if (bereq.http.X-UA-Device) {
             if (!beresp.http.Vary) { # no Vary at all
-                set beresp.http.Vary = "X-UA-Device"; 
+                set beresp.http.Vary = "X-UA-Device";
             } elseif (beresp.http.Vary !~ "X-UA-Device") { # add to existing Vary
-                set beresp.http.Vary = beresp.http.Vary + ", X-UA-Device"; 
-            } 
+                set beresp.http.Vary = beresp.http.Vary + ", X-UA-Device";
+            }
         }
         # comment this out if you don't want the client to know your
         # classification
@@ -122,7 +122,7 @@ headers are (by default) available for the script.
 
 VCL::
 
-    sub vcl_recv { 
+    sub vcl_recv {
         # call some detection engine that set req.http.X-UA-Device
     }
 
@@ -163,7 +163,7 @@ is changed.
 
 VCL::
 
-    sub vcl_recv { 
+    sub vcl_recv {
         # call some detection engine that set req.http.X-UA-Device
     }
 
@@ -172,7 +172,7 @@ VCL::
             # if there are existing GET arguments;
             if (req.url ~ "\?") {
                 set req.http.X-get-devicetype = "&devicetype=" + req.http.X-UA-Device;
-            } else { 
+            } else {
                 set req.http.X-get-devicetype = "?devicetype=" + req.http.X-UA-Device;
             }
             set req.url = req.url + req.http.X-get-devicetype;
@@ -253,7 +253,7 @@ VCL::
             return(synth(750, "Moved Temporarily"));
         }
     }
-     
+
     sub vcl_synth {
         if (obj.status == 750) {
             set obj.http.Location = "http://m.example.com" + req.url;
diff --git a/doc/sphinx/users-guide/esi.rst b/doc/sphinx/users-guide/esi.rst
index b81d2ea..30379a1 100644
--- a/doc/sphinx/users-guide/esi.rst
+++ b/doc/sphinx/users-guide/esi.rst
@@ -12,12 +12,12 @@ in all the other pages.
 .. XXX:What other pages? benc
 
 Used properly this strategy can dramatically increase
-your hit rate and reduce the load on your servers. 
+your hit rate and reduce the load on your servers.
 
 In Varnish we've only so far implemented a small subset of ESI. As of version 2.1 we
 have three ESI statements::
 
- esi:include 
+ esi:include
  esi:remove
  <!--esi ...-->
 
@@ -34,7 +34,7 @@ Lets see an example how this could be used. This simple cgi script
 outputs the date::
 
      #!/bin/sh
-     
+
      echo 'Content-type: text/html'
      echo ''
      date "+%Y-%m-%d %H:%M"
@@ -73,10 +73,10 @@ in them, allowing you to only render the content when the page is not
 being ESI-processed.
 For example::
 
-  <esi:remove> 
+  <esi:remove>
     <a href="http://www.example.com/LICENSE">The license</a>
   </esi:remove>
-  <!--esi  
+  <!--esi
   <p>The full text of the license:</p>
   <esi:include src="http://example.com/LICENSE" />
   -->
diff --git a/doc/sphinx/users-guide/index.rst b/doc/sphinx/users-guide/index.rst
index e1c3e83..4aae504 100644
--- a/doc/sphinx/users-guide/index.rst
+++ b/doc/sphinx/users-guide/index.rst
@@ -8,7 +8,7 @@ The Varnish documentation consists of three main documents:
 * :ref:`tutorial-index` explains the basics and gets you started with Varnish.
 
 * :ref:`users-guide-index` (this document), explains how Varnish works
-  and how you can use it to improve your website. 
+  and how you can use it to improve your website.
 
 * :ref:`reference-index` contains hard facts and is useful for
   looking up specific questions.
diff --git a/doc/sphinx/users-guide/intro.rst b/doc/sphinx/users-guide/intro.rst
index 927292a..f430d75 100644
--- a/doc/sphinx/users-guide/intro.rst
+++ b/doc/sphinx/users-guide/intro.rst
@@ -5,7 +5,7 @@ The Big Varnish Picture
 
 In this section we will cover answers to the questions:
 - What is in this package called "Varnish"?
-- what are all the different bits and pieces named? 
+- what are all the different bits and pieces named?
 - Will you need a hex-wrench for assembly?
 
 The two main parts of Varnish are the two processes in the `varnishd`
diff --git a/doc/sphinx/users-guide/operation-logging.rst b/doc/sphinx/users-guide/operation-logging.rst
index fdbcf58..bb4c7cc 100644
--- a/doc/sphinx/users-guide/operation-logging.rst
+++ b/doc/sphinx/users-guide/operation-logging.rst
@@ -6,7 +6,7 @@ Logging in Varnish
 One of the really nice features in Varnish is the way logging
 works. Instead of logging to a normal log file Varnish logs to a shared
 memory segment, called the VSL - the Varnish Shared Log. When the end
-of the segment is reached we start over, overwriting old data. 
+of the segment is reached we start over, overwriting old data.
 
 This is much, much faster than logging to a file and it doesn't
 require disk space. Besides it gives you much, much more information
@@ -32,7 +32,7 @@ These is the Varnish master process checking up on the caching process
 to see that everything is OK.
 
 Now go to the browser and reload the page displaying your web
-app. 
+app.
 .. XXX:Doesn't this require a setup of a running varnishd and a web application being cached? benc
 
 You'll see lines like these.::
diff --git a/doc/sphinx/users-guide/params.rst b/doc/sphinx/users-guide/params.rst
index df4bfdf..5422c98 100644
--- a/doc/sphinx/users-guide/params.rst
+++ b/doc/sphinx/users-guide/params.rst
@@ -20,7 +20,7 @@ a somewhat longer description on what it does and what the default is
 type ``param.show`` and the name of the parameter, like this::
 
   varnish> param.show shortlived
-  200        
+  200
   shortlived                  10.000000 [s]
                               Default is 10.0
                               Objects created with TTL shorter than this are
diff --git a/doc/sphinx/users-guide/vcl-actions.rst b/doc/sphinx/users-guide/vcl-actions.rst
index ed2b8ba..e66250f 100644
--- a/doc/sphinx/users-guide/vcl-actions.rst
+++ b/doc/sphinx/users-guide/vcl-actions.rst
@@ -13,9 +13,9 @@ The most common actions to return are these:
   `vcl_recv`.
 
 *hash*
-  When you return hash from `vcl_recv` you tell Varnish to deliver content 
-  from cache even if the request othervise indicates that the request 
-  should be passed. 
+  When you return hash from `vcl_recv` you tell Varnish to deliver content
+  from cache even if the request othervise indicates that the request
+  should be passed.
 
 *pipe*
 
@@ -23,11 +23,11 @@ The most common actions to return are these:
 
   Pipe can be returned from `vcl_recv` as well. Pipe short circuits the
   client and the backend connections and Varnish will just sit there
-  and shuffle bytes back and forth. Varnish will not look at the data being 
-  send back and forth - so your logs will be incomplete. 
+  and shuffle bytes back and forth. Varnish will not look at the data being
+  send back and forth - so your logs will be incomplete.
 
 *deliver*
-  Deliver the object to the client. Usually returned from `vcl_backend_response`. 
+  Deliver the object to the client. Usually returned from `vcl_backend_response`.
 
 *restart*
   Restart processing of the request. You can restart the processing of
@@ -35,5 +35,5 @@ The most common actions to return are these:
 
 *retry*
   Retry the request against the backend. This can be returned from
-  `vcl_backend_response` or `vcl_backend_error` if you don't like the response 
+  `vcl_backend_response` or `vcl_backend_error` if you don't like the response
   that the backend delivered.
diff --git a/doc/sphinx/users-guide/vcl-example-acls.rst b/doc/sphinx/users-guide/vcl-example-acls.rst
index 45afaa2..c76842f 100644
--- a/doc/sphinx/users-guide/vcl-example-acls.rst
+++ b/doc/sphinx/users-guide/vcl-example-acls.rst
@@ -11,7 +11,7 @@ the IP address of the client against an ACL with the match operator.::
       "192.168.1.0"/24; /* and everyone on the local network */
       ! "192.168.1.23"; /* except for the dialin router */
   }
-  
+
   sub vcl_recv {
     if (req.method == "PURGE") {
       if (client.ip ~ local) {
@@ -19,6 +19,6 @@ the IP address of the client against an ACL with the match operator.::
       } else {
          return(synth(403, "Access denied."));
       }
-    } 
+    }
   }
- 
+
diff --git a/doc/sphinx/users-guide/vcl-example-manipulating-headers.rst b/doc/sphinx/users-guide/vcl-example-manipulating-headers.rst
index 7e65663..1ea81b1 100644
--- a/doc/sphinx/users-guide/vcl-example-manipulating-headers.rst
+++ b/doc/sphinx/users-guide/vcl-example-manipulating-headers.rst
@@ -17,5 +17,5 @@ Now, when the request is handled to the backend server there will be
 no cookie header. The interesting line is the one with the
 if-statement. It matches the URL, taken from the request object, and
 matches it against the regular expression. Note the match operator. If
-it matches the Cookie: header of the request is unset (deleted). 
+it matches the Cookie: header of the request is unset (deleted).
 
diff --git a/doc/sphinx/users-guide/vcl-inline-c.rst b/doc/sphinx/users-guide/vcl-inline-c.rst
index e0d0998..d9748a3 100644
--- a/doc/sphinx/users-guide/vcl-inline-c.rst
+++ b/doc/sphinx/users-guide/vcl-inline-c.rst
@@ -16,7 +16,7 @@ One of the first uses of inline C was logging to `syslog`.::
         C{
                 #include <syslog.h>
         }C
-        
+
         sub vcl_something {
                 C{
                         syslog(LOG_INFO, "Something happened at VCL line XX.");
diff --git a/doc/sphinx/users-guide/vcl-variables.rst b/doc/sphinx/users-guide/vcl-variables.rst
index 70862e5..4531537 100644
--- a/doc/sphinx/users-guide/vcl-variables.rst
+++ b/doc/sphinx/users-guide/vcl-variables.rst
@@ -28,7 +28,7 @@ objects can be accessed and manipulated using VCL.
  The HTTP response right before it is delivered to the client. It is
  typically modified in `vcl_deliver`.
 
-*obj* 
+*obj*
  The object as it is stored in cache. Read only.
 
 .. XXX:What object? the current request? benc
diff --git a/include/Makefile.am b/include/Makefile.am
index 7088579..ced33c8 100644
--- a/include/Makefile.am
+++ b/include/Makefile.am
@@ -98,7 +98,7 @@ vcl.h: $(top_srcdir)/lib/libvcc/generate.py $(top_srcdir)/include/vrt.h
 	@PYTHON@ $(top_srcdir)/lib/libvcc/generate.py $(top_srcdir) $(top_builddir)
 
 BUILT_SOURCES = vcs_version.h vmod_abi.h
-MAINTAINERCLEANFILES = vcs_version.h 
+MAINTAINERCLEANFILES = vcs_version.h
 vcs_version.h: FORCE
 	@if [ -d "$(top_srcdir)/.git" ]; then \
 		V="$$(git show -s --pretty=format:%h)" \
diff --git a/lib/Makefile.phk b/lib/Makefile.phk
index 6eba2c8..83d0674 100644
--- a/lib/Makefile.phk
+++ b/lib/Makefile.phk
@@ -1,5 +1,5 @@
 SUBDIRS = libvcc libvarnish libvarnishapi libvarnishtools libvcc
-SUBDIRS += libvgz 
+SUBDIRS += libvgz
 SUBDIRS += libvmod_debug libvmod_directors libvmod_std
 
 TOPDIR	= $(CURDIR)/..
diff --git a/lib/libvarnish/flint.lnt b/lib/libvarnish/flint.lnt
index f86bb8d..c98d791 100644
--- a/lib/libvarnish/flint.lnt
+++ b/lib/libvarnish/flint.lnt
@@ -26,8 +26,8 @@
 -e763	// Redundant declaration for symbol '...' previously declared
 -e726	// Extraneous comma ignored
 -e728	// Symbol ... not explicitly initialized
--e716	// while(1) ... 
--e785	// Too few initializers for aggregate 
+-e716	// while(1) ...
+-e785	// Too few initializers for aggregate
 -e786	// String concatenation within initializer
 
 -esym(534, sprintf)	// Ignoring return value of function
diff --git a/lib/libvarnishapi/libvarnishapi.map b/lib/libvarnishapi/libvarnishapi.map
index 653342f..e5aa91d 100644
--- a/lib/libvarnishapi/libvarnishapi.map
+++ b/lib/libvarnishapi/libvarnishapi.map
@@ -2,7 +2,7 @@
  * Copyright (c) 2011-2015 Varnish Software AS
  * All rights reserved.
  *
- * Author: Tollef Fog Heen <tfheen at varnish-software.com> 
+ * Author: Tollef Fog Heen <tfheen at varnish-software.com>
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
diff --git a/lib/libvcc/flint.lnt b/lib/libvcc/flint.lnt
index 60b3850..e264dab 100644
--- a/lib/libvcc/flint.lnt
+++ b/lib/libvcc/flint.lnt
@@ -26,7 +26,7 @@
 -esym(534, vsb_vprintf)	// Ignoring return value of function
 -esym(534, memset)	// Ignoring return value of function
 -e788			// enum constant 'HND_Unclass' not used within defaulted switch
--e716			// while(1) ... 
+-e716			// while(1) ...
 -e786			// String concatenation within initializer
 -e732			// Loss of sign (arg. no. 2) (int to unsigned int)
 
diff --git a/lib/libvmod_std/vmod.vcc b/lib/libvmod_std/vmod.vcc
index 61c334f..cc2506e 100644
--- a/lib/libvmod_std/vmod.vcc
+++ b/lib/libvmod_std/vmod.vcc
@@ -243,7 +243,7 @@ Description
 	Note that the comparison is case sensitive.
 Example
 	| if (std.strstr(req.url, req.http.restrict)) {
-	|     ...
+	|	...
 	| }
 
 	This will check if the content of req.http.restrict occurs
@@ -265,7 +265,7 @@ Description
 	| "784111777"
 Example
 	| if (std.time(resp.http.last-modified, now) < now - 1w) {
-	|     ...
+	|	...
 	| }
 
 
diff --git a/varnish.m4 b/varnish.m4
index fcf6b2d..8a9f647 100644
--- a/varnish.m4
+++ b/varnish.m4
@@ -31,7 +31,7 @@
 # For compatibility with autoconf < 2.63b
 m4_ifndef([AS_VAR_COPY],
   [m4_define([AS_VAR_COPY],
-     [AS_LITERAL_IF([$1[]$2], [$1=$$2], [eval $1=\$$2])])]) 
+     [AS_LITERAL_IF([$1[]$2], [$1=$$2], [eval $1=\$$2])])])
 
 # VARNISH_VMOD_INCLUDE_DIR([])
 # ----------------------------



From lkarsten at varnish-software.com  Thu Jan 14 14:15:09 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:09 +0100
Subject: [4.1] da8b3c9 Random pass at sanitising and cleaning some tests
Message-ID: <E1aJig9-0005LU-1A@project.varnish-software.com>


commit da8b3c9334c70c64ea2439c50b95d3ec80ed840b
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Fri Dec 11 15:02:28 2015 +0000

    Random pass at sanitising and cleaning some tests

diff --git a/bin/varnishtest/tests/c00025.vtc b/bin/varnishtest/tests/c00025.vtc
index ea0866a..bf519f2 100644
--- a/bin/varnishtest/tests/c00025.vtc
+++ b/bin/varnishtest/tests/c00025.vtc
@@ -2,30 +2,24 @@ varnishtest "Test If-None-Match"
 
 server s1 {
 	rxreq
-	expect req.url == "/foo"
-	txresp -hdr {ETag: "123456789"} \
-	    -body "11111\n"
+	expect req.url == /
+	txresp -hdr {ETag: "123456789"} -bodylen 10
 } -start
 
 varnish v1 -vcl+backend { } -start
 
 client c1 {
-	txreq -url "/foo"
+	txreq
 	rxresp
 	expect resp.status == 200
-	expect resp.bodylen == 6
+	expect resp.bodylen == 10
+	expect resp.http.etag == {"123456789"}
 
-	txreq -url "/foo" \
-	    -hdr {If-None-Match: "12345678"}
+	txreq -hdr {If-None-Match: "12345678"}
 	rxresp
 	expect resp.status == 200
 
-	txreq -url "/foo" \
-	    -hdr {If-None-Match: "123456789"}
+	txreq -hdr {If-None-Match: "123456789"}
 	rxresp -no_obj
 	expect resp.status == 304
-}
-
-client c1 -run
-
-client c1 -run
+} -run
diff --git a/bin/varnishtest/tests/c00026.vtc b/bin/varnishtest/tests/c00026.vtc
index d528c0b..49e4984 100644
--- a/bin/varnishtest/tests/c00026.vtc
+++ b/bin/varnishtest/tests/c00026.vtc
@@ -1,51 +1,34 @@
-varnishtest "Client IMS/INM: Test Combination of If-None-Match and If-Modified-Since"
+varnishtest "Test client If-None-Match and If-Modified-Since together"
 
 server s1 {
 	rxreq
-	expect req.url == "/foo"
+	expect req.url == /
 	txresp -hdr {ETag: "123456789"} \
-	       -hdr "Last-Modified: Thu, 26 Jun 2008 12:00:01 GMT" \
-	    -body "11111\n"
+	    -hdr "Last-Modified: Thu, 26 Jun 2008 12:00:01 GMT" \
+	    -bodylen 10
 } -start
 
 varnish v1 -vcl+backend { } -start
 
 client c1 {
-	txreq -url "/foo"
+	txreq
 	rxresp
 	expect resp.status == 200
-	expect resp.bodylen == 6
+	expect resp.bodylen == 10
+	expect resp.http.etag == {"123456789"}
 
-	txreq -url "/foo" \
-	    -hdr {If-None-Match: "123456789"}
-	rxresp -no_obj
-	expect resp.status == 304
-
-	txreq -url "/foo" \
-	    -hdr "If-Modified-Since: Thu, 26 Jun 2008 12:00:01 GMT"
-	rxresp -no_obj
-	expect resp.status == 304
-
-	txreq -url "/foo" \
-	    -hdr "If-Modified-Since: Thu, 26 Jun 2008 12:00:00 GMT" \
+	txreq -hdr "If-Modified-Since: Thu, 26 Jun 2008 12:00:00 GMT" \
 	    -hdr {If-None-Match: "123456789"}
 	rxresp
 	expect resp.status == 200
 
-	txreq -url "/foo" \
-	    -hdr "If-Modified-Since: Thu, 26 Jun 2008 12:00:01 GMT" \
+	txreq -hdr "If-Modified-Since: Thu, 26 Jun 2008 12:00:01 GMT" \
 	    -hdr {If-None-Match: "12345678"}
 	rxresp
 	expect resp.status == 200
 
-	txreq -url "/foo" \
-	    -hdr "If-Modified-Since: Thu, 26 Jun 2008 12:00:01 GMT" \
+	txreq -hdr "If-Modified-Since: Thu, 26 Jun 2008 12:00:01 GMT" \
 	    -hdr {If-None-Match: "123456789"}
 	rxresp -no_obj
 	expect resp.status == 304
-
-}
-
-client c1 -run
-
-client c1 -run
+} -run
diff --git a/bin/varnishtest/tests/m00001.vtc b/bin/varnishtest/tests/m00001.vtc
index 1e0b45f..968345a 100644
--- a/bin/varnishtest/tests/m00001.vtc
+++ b/bin/varnishtest/tests/m00001.vtc
@@ -5,7 +5,7 @@ server s1 {
 	txresp -hdr "foo: bAr" -hdr "bar: fOo" -bodylen 4
 } -start
 
-varnish v1 -arg "-pthread_pools=1" -vcl+backend {
+varnish v1 -vcl+backend {
 	import ${vmod_std};
 
 	sub vcl_deliver {
diff --git a/bin/varnishtest/tests/m00002.vtc b/bin/varnishtest/tests/m00002.vtc
index b0c3ebe..bb5626d 100644
--- a/bin/varnishtest/tests/m00002.vtc
+++ b/bin/varnishtest/tests/m00002.vtc
@@ -1,21 +1,21 @@
-varnishtest "Test std.random"
+varnishtest "Test std.random()"
 
 # needs random generator
 random
 
 server s1 {
 	rxreq
-	txresp -proto HTTP/1.0 -nolen -bodylen 9
+	txresp
 } -start
 
 varnish v1 -vcl+backend {
 	import ${vmod_std};
 
-	sub vcl_backend_response {
-		set beresp.http.rnd1 = std.random(0,1);
-		set beresp.http.rnd2 = std.random(0,10);
-		set beresp.http.rnd3 = std.random(8,10);
-		set beresp.http.rnd4 = std.random(99,100);
+	sub vcl_deliver {
+		set resp.http.rnd1 = std.random(0, 1);
+		set resp.http.rnd2 = std.random(0, 10);
+		set resp.http.rnd3 = std.random(8, 10);
+		set resp.http.rnd4 = std.random(99, 100);
 	}
 } -start
 
@@ -30,5 +30,4 @@ client c1 {
 	expect resp.http.rnd2 == 7.478
 	expect resp.http.rnd3 == 9.034
 	expect resp.http.rnd4 == 99.502
-	expect resp.bodylen == 9
 } -run
diff --git a/bin/varnishtest/tests/m00003.vtc b/bin/varnishtest/tests/m00003.vtc
index c0a5aeb..073632d 100644
--- a/bin/varnishtest/tests/m00003.vtc
+++ b/bin/varnishtest/tests/m00003.vtc
@@ -4,29 +4,19 @@ feature topbuild
 
 server s1 {
 	rxreq
-	txresp -hdr "foo: bAr" -hdr "bar: fOo" -bodylen 4
+	txresp
 } -start
 
-varnish v1 -arg "-pthread_pools=1" \
-	-arg "-pvmod_dir=${topbuild}/lib/libvmod_std/.libs/" \
-	-vcl+backend {
+varnish v1 -arg "-pvmod_dir=${topbuild}/lib/libvmod_std/.libs/" \
+    -vcl+backend {
 	import std;
-
-	sub vcl_deliver {
-		set resp.http.foo = std.toupper(resp.http.foo);
-		set resp.http.bar = std.tolower(resp.http.bar);
-		std.set_ip_tos(32);
-	}
 } -start
 
-client c1 {
-	txreq -url "/bar"
-	rxresp
-	expect resp.status == 200
-	expect resp.bodylen == "4"
-	expect resp.http.foo == "BAR"
-	expect resp.http.bar == "foo"
-} -run
+varnish v1 -cliok "param.set vmod_dir /nonexistent"
+
+varnish v1 -errvcl {No such file or directory} {
+	import std;
+}
 
 varnish v1 -errvcl {Expected 'from path ...'} {
 	import std to;
diff --git a/bin/varnishtest/tests/m00004.vtc b/bin/varnishtest/tests/m00004.vtc
index 80300ff..9e1d047 100644
--- a/bin/varnishtest/tests/m00004.vtc
+++ b/bin/varnishtest/tests/m00004.vtc
@@ -13,9 +13,7 @@ server s1 {
 	}
 } -start
 
-varnish v1 \
-	-arg "-pthread_pools=1" \
-	-vcl+backend {
+varnish v1 -vcl+backend {
 	import ${vmod_std};
 
 	sub vcl_deliver {
diff --git a/bin/varnishtest/tests/m00005.vtc b/bin/varnishtest/tests/m00005.vtc
index 5ef9dde..d89396c 100644
--- a/bin/varnishtest/tests/m00005.vtc
+++ b/bin/varnishtest/tests/m00005.vtc
@@ -1,106 +1,72 @@
-varnishtest "test vmod_std.duration conversion"
+varnishtest "Test std.duration()"
 
 server s1 {
 	rxreq
-	expect req.url == "/1"
-	txresp -bodylen 1
-
+	txresp
 } -start
 
 varnish v1 -vcl+backend {
 	import ${vmod_std};
 
-	sub vcl_recv {
-		set req.ttl = 1000000s + std.duration(req.http.ttl, 1s);
-	}
 	sub vcl_deliver {
-		set resp.http.ttl = (req.ttl + 1s);
+		set resp.http.ttl = std.duration(req.http.ttl, 1s) + 1000000s;
 	}
 } -start
 
 client c1 {
-	txreq -url "/1"  -hdr "ttl: 10ms "
+	txreq -hdr "ttl: 10ms"
 	rxresp
-	expect resp.status == 200
-	expect resp.http.ttl == 1000001.010
-	expect resp.bodylen == 1
+	expect resp.http.ttl == 1000000.010
 
-	txreq -url "/1"  -hdr "ttl: 10s "
+	txreq -hdr "ttl: 10s"
 	rxresp
-	expect resp.status == 200
-	expect resp.http.ttl == 1000011.000
-	expect resp.bodylen == 1
+	expect resp.http.ttl == 1000010.000
 
-	txreq -url "/1"  -hdr "ttl: 10m "
+	txreq -hdr "ttl: 10m"
 	rxresp
-	expect resp.status == 200
-	expect resp.http.ttl == 1000601.000
-	expect resp.bodylen == 1
+	expect resp.http.ttl == 1000600.000
 
-	txreq -url "/1"  -hdr "ttl: 10h "
+	txreq -hdr "ttl: 10h"
 	rxresp
-	expect resp.status == 200
-	expect resp.http.ttl == 1036001.000
-	expect resp.bodylen == 1
+	expect resp.http.ttl == 1036000.000
 
-	txreq -url "/1"  -hdr "ttl: 10d "
+	txreq -hdr "ttl: 10d"
 	rxresp
-	expect resp.status == 200
-	expect resp.http.ttl == 1864001.000
-	expect resp.bodylen == 1
+	expect resp.http.ttl == 1864000.000
 
-	txreq -url "/1"  -hdr "ttl: 10w "
+	txreq -hdr "ttl: 10w"
 	rxresp
-	expect resp.status == 200
-	expect resp.http.ttl == 7048001.000
-	expect resp.bodylen == 1
+	expect resp.http.ttl == 7048000.000
 
-	txreq -url "/1"  -hdr "ttl: -100s "
+	txreq -hdr "ttl: 1y"
 	rxresp
-	expect resp.status == 200
-	expect resp.http.ttl ==  999901.000
-	expect resp.bodylen == 1
+	expect resp.http.ttl == 32536000.000
 
-	txreq -url "/1"  -hdr "ttl: s "
+	txreq -hdr "ttl: -100s"
 	rxresp
-	expect resp.status == 200
-	expect resp.http.ttl ==  1000002.000
-	expect resp.bodylen == 1
+	expect resp.http.ttl ==  999900.000
 
-	txreq -url "/1"  -hdr "ttl: 3wx "
+	txreq -hdr "ttl: s"
 	rxresp
-	expect resp.status == 200
-	expect resp.http.ttl ==  1000002.000
-	expect resp.bodylen == 1
+	expect resp.http.ttl ==  1000001.000
 
-	txreq -url "/1"  -hdr "ttl: -inf "
+	txreq -hdr "ttl: 3wx"
 	rxresp
-	expect resp.status == 200
-	expect resp.http.ttl ==  1000002.000
-	expect resp.bodylen == 1
+	expect resp.http.ttl ==  1000001.000
 
-	txreq -url "/1"  -hdr "ttl: 2x "
+	txreq -hdr "ttl: -inf"
 	rxresp
-	expect resp.status == 200
-	expect resp.http.ttl ==  1000002.000
-	expect resp.bodylen == 1
+	expect resp.http.ttl ==  1000001.000
 
-	txreq -url "/1"  -hdr "ttl: 2h x "
+	txreq -hdr "ttl: 2x"
 	rxresp
-	expect resp.status == 200
-	expect resp.http.ttl ==  1000002.000
-	expect resp.bodylen == 1
+	expect resp.http.ttl ==  1000001.000
 
-	txreq -url "/1"  -hdr "ttl: 100"
+	txreq -hdr "ttl: 2h x"
 	rxresp
-	expect resp.status == 200
-	expect resp.http.ttl == 1000002.000
-	expect resp.bodylen == 1
+	expect resp.http.ttl ==  1000001.000
 
-	txreq -url "/1"  -hdr "ttl: 1y"
+	txreq -hdr "ttl: 100"
 	rxresp
-	expect resp.status == 200
-	expect resp.http.ttl == 32536001.000
-	expect resp.bodylen == 1
-
+	expect resp.http.ttl == 1000001.000
 } -run
diff --git a/bin/varnishtest/tests/m00006.vtc b/bin/varnishtest/tests/m00006.vtc
index 1dd4dec..d002112 100644
--- a/bin/varnishtest/tests/m00006.vtc
+++ b/bin/varnishtest/tests/m00006.vtc
@@ -2,14 +2,9 @@ varnishtest "test vmod_std.collect()"
 
 server s1 {
 	rxreq
-	expect req.url == "/1"
 	expect req.http.foo == "1, 2"
-	txresp -hdr "bar: a" -hdr "bar: b" -bodylen 1
-
-	rxreq
-	expect req.url == "/2"
-	expect req.http.baz == "1, 2"
-	txresp -hdr "qux: a" -hdr "qux: b" -bodylen 1
+	expect req.http.baz == "3, 4"
+	txresp -hdr "bar: a" -hdr "bar: b" -hdr "qux: c" -hdr "qux: d"
 } -start
 
 varnish v1 -vcl+backend {
@@ -30,15 +25,8 @@ varnish v1 -vcl+backend {
 } -start
 
 client c1 {
-	txreq  -url "/1" -hdr "Foo: 1" -hdr "Foo: 2"
+	txreq -hdr "Foo: 1" -hdr "Foo: 2" -hdr "Baz: 3" -hdr "Baz: 4"
 	rxresp
 	expect resp.http.bar == "a, b"
-	expect resp.status == 200
-	expect resp.bodylen == 1
-
-	txreq -url "/2" -hdr "Baz: 1" -hdr "Baz: 2"
-	rxresp
-	expect resp.http.qux == "a, b"
-	expect resp.status == 200
-	expect resp.bodylen == 1
+	expect resp.http.qux == "c, d"
 } -run
diff --git a/bin/varnishtest/tests/m00007.vtc b/bin/varnishtest/tests/m00007.vtc
index a99ac10..9a1fe39 100644
--- a/bin/varnishtest/tests/m00007.vtc
+++ b/bin/varnishtest/tests/m00007.vtc
@@ -1,53 +1,47 @@
-varnishtest "test vmod_std.integer conversion"
+varnishtest "Test std.integer()"
 
 server s1 {
 	rxreq
-	expect req.url == "/1"
-	txresp -bodylen 1
-
+	txresp
 } -start
 
 varnish v1 -vcl+backend {
 	import ${vmod_std};
 
 	sub vcl_deliver {
-		set resp.http.biggerthanzero = (std.integer(req.http.foo,0) > 0);
-		set resp.http.smallerthanzero = (std.integer(req.http.foo,0) < 0);
-		set resp.http.iszero = (std.integer(req.http.foo,0) == 0);
-		set resp.http.converted = std.integer(req.http.foo,0);
+		set resp.http.gtzero = (std.integer(req.http.foo, 0) > 0);
+		set resp.http.ltzero = (std.integer(req.http.foo, 0) < 0);
+		set resp.http.iszero = (std.integer(req.http.foo, 0) == 0);
+		set resp.http.converted = std.integer(req.http.foo, 0);
 	}
 } -start
 
 client c1 {
-	txreq -url "/1"  -hdr "foo: 1"
+	txreq -hdr "foo: 1"
 	rxresp
-	expect resp.status == 200
-	expect resp.http.biggerthanzero == true
-	expect resp.http.smallerthanzero == false
+	expect resp.http.gtzero == true
+	expect resp.http.ltzero == false
 	expect resp.http.iszero == false
 	expect resp.http.converted == 1
 
-	txreq -url "/1"  -hdr "foo: -1"
+	txreq -hdr "foo: -1"
 	rxresp
-	expect resp.status == 200
-	expect resp.http.biggerthanzero == false
-	expect resp.http.smallerthanzero == true
+	expect resp.http.gtzero == false
+	expect resp.http.ltzero == true
 	expect resp.http.iszero == false
 	expect resp.http.converted == -1
 
-	txreq -url "/1"
+	txreq -hdr "foo: 0"
 	rxresp
-	expect resp.status == 200
-	expect resp.http.biggerthanzero == false
-	expect resp.http.smallerthanzero == false
+	expect resp.http.gtzero == false
+	expect resp.http.ltzero == false
 	expect resp.http.iszero == true
 	expect resp.http.converted == 0
 
-	txreq -url "/1"  -hdr "foo: bar"
+	txreq -hdr "foo: bar"
 	rxresp
-	expect resp.status == 200
-	expect resp.http.biggerthanzero == false
-	expect resp.http.smallerthanzero == false
+	expect resp.http.gtzero == false
+	expect resp.http.ltzero == false
 	expect resp.http.iszero == true
 	expect resp.http.converted == 0
 } -run
diff --git a/bin/varnishtest/tests/m00011.vtc b/bin/varnishtest/tests/m00011.vtc
index bf96dd5..930ed8e 100644
--- a/bin/varnishtest/tests/m00011.vtc
+++ b/bin/varnishtest/tests/m00011.vtc
@@ -1,16 +1,13 @@
-varnishtest "Test std.ip"
+varnishtest "Test std.ip()"
 
 server s1 {
 	rxreq
-	txresp -body "1"
-	rxreq
-	txresp -body "22"
+	txresp
 } -start
 
-
-varnish v1 -arg "-Wpoll" -vcl+backend {
-
+varnish v1 -vcl+backend {
 	import ${vmod_std};
+
 	sub vcl_deliver {
 		set resp.http.foo0 = std.ip("8.8.8.*", client.ip);
 		set resp.http.foo1 = std.ip("9.9.9.*", server.ip);
@@ -22,16 +19,12 @@ varnish v1 -arg "-Wpoll" -vcl+backend {
 } -start
 
 client c1 {
-	txreq -url "/foo1"
+	txreq
 	rxresp
-	expect resp.bodylen == 1
 	expect resp.http.foo0 == "127.0.0.1"
 	expect resp.http.foo1 == "127.0.0.1"
 	expect resp.http.foo2 == "127.0.0.2"
 	expect resp.http.foo3 == "1.2.3.5"
 	expect resp.http.foo4 == "2001:db8::"
 	expect resp.http.foo5 == "::1"
-	txreq -url "/foo2"
-	rxresp
-	expect resp.bodylen == 2
 } -run
diff --git a/bin/varnishtest/tests/m00013.vtc b/bin/varnishtest/tests/m00013.vtc
index c3a6295..6f27c57 100644
--- a/bin/varnishtest/tests/m00013.vtc
+++ b/bin/varnishtest/tests/m00013.vtc
@@ -1,4 +1,4 @@
-varnishtest "Timestamps from std vmod"
+varnishtest "Test std.timestamp()"
 
 server s1 {
 	rxreq
@@ -38,7 +38,6 @@ logexpect l1 -v v1 -g request {
 client c1 {
 	txreq
 	rxresp
-	expect resp.status == 200
 } -run
 
 logexpect l1 -wait
diff --git a/bin/varnishtest/tests/r01826.vtc b/bin/varnishtest/tests/r01826.vtc
index eeb4503..6a553d5 100644
--- a/bin/varnishtest/tests/r01826.vtc
+++ b/bin/varnishtest/tests/r01826.vtc
@@ -2,7 +2,7 @@ varnishtest "Check we ignore a zero C-L with a 204"
 
 server s1 {
 	rxreq
-	txresp -status 204 -nolen -hdr "Content-Length: 0"
+	txresp -status 204
 } -start
 
 varnish v1 -vcl+backend {



From lkarsten at varnish-software.com  Thu Jan 14 14:15:09 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:09 +0100
Subject: [4.1] 7782833 Use a weak comparison function for If-None-Match
Message-ID: <E1aJig9-0005Le-2a@project.varnish-software.com>


commit 778283389bae7809dd8fc40565ae64ba85fbfa4e
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Fri Dec 11 15:26:03 2015 +0000

    Use a weak comparison function for If-None-Match
    
    As per RFC 7232.
    
    Fixes #1816.

diff --git a/bin/varnishd/cache/cache_rfc2616.c b/bin/varnishd/cache/cache_rfc2616.c
index 7608755..d3fa8a2 100644
--- a/bin/varnishd/cache/cache_rfc2616.c
+++ b/bin/varnishd/cache/cache_rfc2616.c
@@ -232,6 +232,16 @@ RFC2616_Req_Gzip(const struct http *hp)
 
 /*--------------------------------------------------------------------*/
 
+static inline int
+rfc2616_weak_compare(const char *p, const char *e)
+{
+	if (p[0] == 'W' && p[1] == '/')
+		p += 2;
+	if (e[0] == 'W' && e[1] == '/')
+		e += 2;
+	return (strcmp(p, e) != 0);
+}
+
 int
 RFC2616_Do_Cond(const struct req *req)
 {
@@ -239,12 +249,20 @@ RFC2616_Do_Cond(const struct req *req)
 	double ims, lm;
 	int do_cond = 0;
 
-	/* RFC 2616 13.3.4 states we need to match both ETag
-	   and If-Modified-Since if present*/
+	/*
+	 * RFC 2616 13.3.4 states we need to match both ETag and
+	 * If-Modified-Since if present.
+	 */
+	if (http_GetHdr(req->http, H_If_None_Match, &p) &&
+	    http_GetHdr(req->resp, H_ETag, &e)) {
+		if (rfc2616_weak_compare(p, e))
+			return (0);
+		do_cond = 1;
+	}
 
-	if (http_GetHdr(req->http, H_If_Modified_Since, &p) ) {
+	if (http_GetHdr(req->http, H_If_Modified_Since, &p)) {
 		ims = VTIM_parse(p);
-		if (ims > req->t_req)	/* [RFC2616 14.25] */
+		if (!ims || ims > req->t_req)	/* [RFC2616 14.25] */
 			return (0);
 		AZ(ObjGetDouble(req->wrk, req->objcore, OA_LASTMODIFIED, &lm));
 		if (lm > ims)
@@ -252,13 +270,6 @@ RFC2616_Do_Cond(const struct req *req)
 		do_cond = 1;
 	}
 
-	if (http_GetHdr(req->http, H_If_None_Match, &p) &&
-	    http_GetHdr(req->resp, H_ETag, &e)) {
-		if (strcmp(p,e) != 0)
-			return (0);
-		do_cond = 1;
-	}
-
 	return (do_cond);
 }
 
diff --git a/bin/varnishtest/tests/c00025.vtc b/bin/varnishtest/tests/c00025.vtc
index bf519f2..9b7fc95 100644
--- a/bin/varnishtest/tests/c00025.vtc
+++ b/bin/varnishtest/tests/c00025.vtc
@@ -4,6 +4,10 @@ server s1 {
 	rxreq
 	expect req.url == /
 	txresp -hdr {ETag: "123456789"} -bodylen 10
+
+	rxreq
+	expect req.url == /other
+	txresp -hdr {ETag: W/"123456789"} -bodylen 10
 } -start
 
 varnish v1 -vcl+backend { } -start
@@ -22,4 +26,36 @@ client c1 {
 	txreq -hdr {If-None-Match: "123456789"}
 	rxresp -no_obj
 	expect resp.status == 304
+
+	txreq -hdr {If-None-Match: W/"12345678"}
+	rxresp
+	expect resp.status == 200
+
+	txreq -hdr {If-None-Match: W/"123456789"}
+	rxresp -no_obj
+	expect resp.status == 304
+} -run
+
+client c2 {
+	txreq -url /other
+	rxresp
+	expect resp.status == 200
+	expect resp.bodylen == 10
+	expect resp.http.etag == {W/"123456789"}
+
+	txreq -url /other -hdr {If-None-Match: "12345678"}
+	rxresp
+	expect resp.status == 200
+
+	txreq -url /other -hdr {If-None-Match: "123456789"}
+	rxresp -no_obj
+	expect resp.status == 304
+
+	txreq -url /other -hdr {If-None-Match: W/"12345678"}
+	rxresp
+	expect resp.status == 200
+
+	txreq -url /other -hdr {If-None-Match: W/"123456789"}
+	rxresp -no_obj
+	expect resp.status == 304
 } -run



From lkarsten at varnish-software.com  Thu Jan 14 14:15:09 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:09 +0100
Subject: [4.1] 926c9ee For range requests we need a strong validator
Message-ID: <E1aJig9-0005Lo-41@project.varnish-software.com>


commit 926c9ee46b7a3d30e9d7ca75dc2ee7660921cbe2
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Sat Dec 12 12:14:50 2015 +0000

    For range requests we need a strong validator
    
    That is, if both If-None-Match and Range are present we need to use a
    strong comparison as the representation needs to be byte-for-byte
    identical.

diff --git a/bin/varnishd/cache/cache_rfc2616.c b/bin/varnishd/cache/cache_rfc2616.c
index d3fa8a2..0c0e127 100644
--- a/bin/varnishd/cache/cache_rfc2616.c
+++ b/bin/varnishd/cache/cache_rfc2616.c
@@ -233,13 +233,22 @@ RFC2616_Req_Gzip(const struct http *hp)
 /*--------------------------------------------------------------------*/
 
 static inline int
+rfc2616_strong_compare(const char *p, const char *e)
+{
+	if ((p[0] == 'W' && p[1] == '/') ||
+	    (e[0] == 'W' && e[1] == '/'))
+		return (0);
+	return (strcmp(p, e) == 0);
+}
+
+static inline int
 rfc2616_weak_compare(const char *p, const char *e)
 {
 	if (p[0] == 'W' && p[1] == '/')
 		p += 2;
 	if (e[0] == 'W' && e[1] == '/')
 		e += 2;
-	return (strcmp(p, e) != 0);
+	return (strcmp(p, e) == 0);
 }
 
 int
@@ -255,9 +264,12 @@ RFC2616_Do_Cond(const struct req *req)
 	 */
 	if (http_GetHdr(req->http, H_If_None_Match, &p) &&
 	    http_GetHdr(req->resp, H_ETag, &e)) {
-		if (rfc2616_weak_compare(p, e))
+		if (http_GetHdr(req->http, H_Range, NULL))
+			do_cond = rfc2616_strong_compare(p, e);
+		else
+			do_cond = rfc2616_weak_compare(p, e);
+		if (!do_cond)
 			return (0);
-		do_cond = 1;
 	}
 
 	if (http_GetHdr(req->http, H_If_Modified_Since, &p)) {
diff --git a/bin/varnishtest/tests/c00025.vtc b/bin/varnishtest/tests/c00025.vtc
index 9b7fc95..f5b8988 100644
--- a/bin/varnishtest/tests/c00025.vtc
+++ b/bin/varnishtest/tests/c00025.vtc
@@ -27,6 +27,10 @@ client c1 {
 	rxresp -no_obj
 	expect resp.status == 304
 
+	txreq -hdr "Range: bytes=1-2" -hdr {If-None-Match: "123456789"}
+	rxresp -no_obj
+	expect resp.status == 304
+
 	txreq -hdr {If-None-Match: W/"12345678"}
 	rxresp
 	expect resp.status == 200
@@ -34,9 +38,11 @@ client c1 {
 	txreq -hdr {If-None-Match: W/"123456789"}
 	rxresp -no_obj
 	expect resp.status == 304
-} -run
 
-client c2 {
+	txreq -hdr "Range: bytes=1-2" -hdr {If-None-Match: W/"123456789"}
+	rxresp
+	expect resp.status == 206
+
 	txreq -url /other
 	rxresp
 	expect resp.status == 200
@@ -51,6 +57,11 @@ client c2 {
 	rxresp -no_obj
 	expect resp.status == 304
 
+	txreq -url /other -hdr "Range: bytes=1-2" \
+	    -hdr {If-None-Match: "123456789"}
+	rxresp
+	expect resp.status == 206
+
 	txreq -url /other -hdr {If-None-Match: W/"12345678"}
 	rxresp
 	expect resp.status == 200
@@ -58,4 +69,9 @@ client c2 {
 	txreq -url /other -hdr {If-None-Match: W/"123456789"}
 	rxresp -no_obj
 	expect resp.status == 304
+
+	txreq -url /other -hdr "Range: bytes=1-2" \
+	    -hdr {If-None-Match: W/"123456789"}
+	rxresp
+	expect resp.status == 206
 } -run



From lkarsten at varnish-software.com  Thu Jan 14 14:15:09 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:09 +0100
Subject: [4.1] 8cd3632 Staticize
Message-ID: <E1aJig9-0005Lx-5C@project.varnish-software.com>


commit 8cd3632f60eb94e1e6fabdb69289ec108df994cf
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Dec 14 13:51:37 2015 +0000

    Staticize

diff --git a/lib/libvmod_debug/vmod_debug.c b/lib/libvmod_debug/vmod_debug.c
index 91b49c7..a9e2cde 100644
--- a/lib/libvmod_debug/vmod_debug.c
+++ b/lib/libvmod_debug/vmod_debug.c
@@ -47,7 +47,7 @@ struct priv_vcl {
 	uintptr_t		exp_cb;
 };
 
-VCL_DURATION vcl_release_delay = 0.0;
+static VCL_DURATION vcl_release_delay = 0.0;
 
 VCL_VOID __match_proto__(td_debug_panic)
 vmod_panic(VRT_CTX, const char *str, ...)



From lkarsten at varnish-software.com  Thu Jan 14 14:15:09 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:09 +0100
Subject: [4.1] 7586f27 FreeBSD's dlopen doesn't return a strerror(3) output,
 hunt the VCC's own complaint instead.
Message-ID: <E1aJig9-0005M6-6v@project.varnish-software.com>


commit 7586f27007d45213ca9f9e6140c9245d949b0e9d
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Dec 14 13:51:43 2015 +0000

    FreeBSD's dlopen doesn't return a strerror(3) output, hunt the
    VCC's own complaint instead.

diff --git a/bin/varnishtest/tests/m00003.vtc b/bin/varnishtest/tests/m00003.vtc
index 073632d..85970dd 100644
--- a/bin/varnishtest/tests/m00003.vtc
+++ b/bin/varnishtest/tests/m00003.vtc
@@ -14,7 +14,7 @@ varnish v1 -arg "-pvmod_dir=${topbuild}/lib/libvmod_std/.libs/" \
 
 varnish v1 -cliok "param.set vmod_dir /nonexistent"
 
-varnish v1 -errvcl {No such file or directory} {
+varnish v1 -errvcl {Could not load VMOD std} {
 	import std;
 }
 



From lkarsten at varnish-software.com  Thu Jan 14 14:15:09 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:09 +0100
Subject: [4.1] b6bc9cd Disable the last test, it fails consistently on FreeBSD.
Message-ID: <E1aJig9-0005MF-8I@project.varnish-software.com>


commit b6bc9cd63918821c48d4b665ebcb703c3b8d4d33
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Dec 14 14:27:15 2015 +0000

    Disable the last test, it fails consistently on FreeBSD.

diff --git a/bin/varnishtest/tests/d00012.vtc b/bin/varnishtest/tests/d00012.vtc
index 41d23e4..edc028b 100644
--- a/bin/varnishtest/tests/d00012.vtc
+++ b/bin/varnishtest/tests/d00012.vtc
@@ -74,4 +74,6 @@ varnish v1 -cli "vcl.list"
 #varnish v1 -expect MAIN.n_backend == 2
 # expected: vcl2.dummy, vcl2.s2
 varnish v1 -expect n_vcl_avail == 1
-varnish v1 -expect n_vcl_discard == 0
+
+# XXX This test fails on FreeBSD
+# XXX varnish v1 -expect n_vcl_discard == 0



From lkarsten at varnish-software.com  Thu Jan 14 14:15:09 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:09 +0100
Subject: [4.1] d25080d Use VSB_putc for single char ops
Message-ID: <E1aJig9-0005MO-9u@project.varnish-software.com>


commit d25080d165bee2ead747019e68d1c2a63b8ca872
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Mon Dec 14 11:06:39 2015 +0000

    Use VSB_putc for single char ops

diff --git a/bin/varnishd/cache/cache_ban_build.c b/bin/varnishd/cache/cache_ban_build.c
index 801774f..b3bda03 100644
--- a/bin/varnishd/cache/cache_ban_build.c
+++ b/bin/varnishd/cache/cache_ban_build.c
@@ -105,7 +105,7 @@ ban_add_lump(const struct ban_proto *bp, const void *p, uint32_t len)
 
 	buf[0] = 0xff;
 	while (VSB_len(bp->vsb) & PALGN)
-		VSB_bcat(bp->vsb, buf, 1);
+		VSB_putc(bp->vsb, buf[0]);
 	vbe32enc(buf, len);
 	VSB_bcat(bp->vsb, buf, sizeof buf);
 	VSB_bcat(bp->vsb, p, len);
diff --git a/bin/varnishd/cache/cache_esi_parse.c b/bin/varnishd/cache/cache_esi_parse.c
index 76591d0..383731f 100644
--- a/bin/varnishd/cache/cache_esi_parse.c
+++ b/bin/varnishd/cache/cache_esi_parse.c
@@ -906,7 +906,7 @@ VEP_Parse(struct vep_state *vep, const char *p, size_t l)
 			while (p < e && *p != '>' && *p != vep->attr_delim &&
 			   (vep->attr_delim != ' ' || !vct_issp(*p))) {
 				if (vep->attr_vsb != NULL)
-					VSB_bcat(vep->attr_vsb, p, 1);
+					VSB_putc(vep->attr_vsb, *p);
 				p++;
 			}
 			if (p < e && *p == '>') {
diff --git a/bin/varnishd/cache/cache_panic.c b/bin/varnishd/cache/cache_panic.c
index 614317a..b60ddbd 100644
--- a/bin/varnishd/cache/cache_panic.c
+++ b/bin/varnishd/cache/cache_panic.c
@@ -526,7 +526,7 @@ pan_ic(const char *func, const char *file, int line, const char *cond,
 		}
 	}
 	VSB_printf(pan_vsb, "\n");
-	VSB_bcat(pan_vsb, "", 1);	/* NUL termination */
+	VSB_putc(pan_vsb, '\0');	/* NUL termination */
 
 	if (FEATURE(FEATURE_NO_COREDUMP))
 		exit(4);
diff --git a/lib/libvarnishapi/vxp.c b/lib/libvarnishapi/vxp.c
index 5dad106..db91b30 100644
--- a/lib/libvarnishapi/vxp.c
+++ b/lib/libvarnishapi/vxp.c
@@ -82,9 +82,9 @@ vxp_quote(const struct vxp *vxp, const char *b, const char *e, int tokoff)
 	assert(e <= vxp->e);
 	for (p = vxp->b; p < vxp->e; p++) {
 		if (isspace(*p))
-			VSB_bcat(vxp->sb, " ", 1);
+			VSB_putc(vxp->sb, ' ');
 		else
-			VSB_bcat(vxp->sb, p, 1);
+			VSB_putc(vxp->sb, *p);
 	}
 	VSB_putc(vxp->sb, '\n');
 	for (p = vxp->b; p < vxp->e; p++) {
diff --git a/lib/libvcc/vcc_token.c b/lib/libvcc/vcc_token.c
index 6201fea..a8513f8 100644
--- a/lib/libvcc/vcc_token.c
+++ b/lib/libvcc/vcc_token.c
@@ -135,13 +135,13 @@ vcc_quoteline(const struct vcc *tl, const char *l, const char *le)
 			y &= ~7;
 			y += 8;
 			while (x < y) {
-				VSB_bcat(tl->sb, " ", 1);
+				VSB_putc(tl->sb, ' ');
 				x++;
 			}
 		} else {
 			x++;
 			y++;
-			VSB_bcat(tl->sb, p, 1);
+			VSB_putc(tl->sb, *p);
 		}
 	}
 	VSB_putc(tl->sb, '\n');



From lkarsten at varnish-software.com  Thu Jan 14 14:15:09 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:09 +0100
Subject: [4.1] 0af979e Fix format string warnings
Message-ID: <E1aJig9-0005MX-BM@project.varnish-software.com>


commit 0af979e3403d02dc640ea2b2c60aebeb9ae359a8
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Mon Dec 14 18:49:17 2015 +0000

    Fix format string warnings

diff --git a/bin/varnishstat/varnishstat_curses.c b/bin/varnishstat/varnishstat_curses.c
index 3dd4230..1c90652 100644
--- a/bin/varnishstat/varnishstat_curses.c
+++ b/bin/varnishstat/varnishstat_curses.c
@@ -540,7 +540,7 @@ static void
 print_duration(WINDOW *w, time_t t)
 {
 
-	wprintw(w, "%4d+%02d:%02d:%02d",
+	wprintw(w, "%4lu+%02lu:%02lu:%02lu",
 	    t / 86400, (t % 86400) / 3600, (t % 3600) / 60, t % 60);
 }
 
@@ -790,8 +790,8 @@ draw_line_bitmap(WINDOW *w, int y, int x, int X, struct pt *pt)
 		case COL_VAL:
 			if (X - x < COLW)
 				return;
-			mvwprintw(w, y, x, "   %010.10jx",
-			    (pt->cur >> 24) & 0xffffffffffLL);
+			mvwprintw(w, y, x, "   %10.10jx",
+			    (uintmax_t)((pt->cur >> 24) & 0xffffffffffLL));
 			x += COLW;
 			break;
 		case COL_MAP:



From lkarsten at varnish-software.com  Thu Jan 14 14:15:09 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:09 +0100
Subject: [4.1] ff52c2a Check format string in some curses implementations
Message-ID: <E1aJig9-0005Mg-CX@project.varnish-software.com>


commit ff52c2a9cb2326f8e87c8efece920773eab5c170
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Mon Dec 14 18:51:24 2015 +0000

    Check format string in some curses implementations
    
    This is enough to get the warnings fixed in the previous commit on Linux.

diff --git a/include/vcurses.h b/include/vcurses.h
index d24b556..911cf96 100644
--- a/include/vcurses.h
+++ b/include/vcurses.h
@@ -33,6 +33,8 @@
 #elif defined HAVE_NCURSES_CURSES_H
 #  include <ncurses/curses.h>
 #elif defined HAVE_NCURSES_H
+#  define GCC_PRINTF
+#  define GCC_SCANF
 #  include <ncurses.h>
 #elif defined HAVE_CURSES_H
 #  include <curses.h>



From lkarsten at varnish-software.com  Thu Jan 14 14:15:09 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:09 +0100
Subject: [4.1] a9b355b Add missing header on NetBSD
Message-ID: <E1aJig9-0005Mp-Di@project.varnish-software.com>


commit a9b355be8c9cb64e0160ab3566f2891bdb00aca4
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Mon Dec 14 18:52:59 2015 +0000

    Add missing header on NetBSD

diff --git a/lib/libvarnish/vtcp.c b/lib/libvarnish/vtcp.c
index cb4d3c4..0c6a312 100644
--- a/lib/libvarnish/vtcp.c
+++ b/lib/libvarnish/vtcp.c
@@ -32,6 +32,7 @@
 #include <sys/types.h>
 #include <sys/socket.h>
 #include <sys/ioctl.h>
+#include <sys/time.h>
 #ifdef HAVE_SYS_FILIO_H
 #  include <sys/filio.h>
 #endif



From lkarsten at varnish-software.com  Thu Jan 14 14:15:09 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:09 +0100
Subject: [4.1] 09631cc Fix another warning spotted on FreeBSD
Message-ID: <E1aJig9-0005My-FK@project.varnish-software.com>


commit 09631cc7b02d177a68c6299c63d797276aed4b42
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Mon Dec 14 19:28:28 2015 +0000

    Fix another warning spotted on FreeBSD

diff --git a/bin/varnishstat/varnishstat_curses.c b/bin/varnishstat/varnishstat_curses.c
index 1c90652..a27d438 100644
--- a/bin/varnishstat/varnishstat_curses.c
+++ b/bin/varnishstat/varnishstat_curses.c
@@ -934,7 +934,7 @@ draw_bar_b(void)
 	    page_start + l_points < n_ptarray ?
 		page_start + l_points : n_ptarray,
 	    n_ptarray);
-	mvwprintw(w_bar_b, 0, X - strlen(buf), buf);
+	mvwprintw(w_bar_b, 0, X - strlen(buf), "%s", buf);
 	X -= strlen(buf) + 2;
 
 	level = VSC_LevelDesc(verbosity);



From lkarsten at varnish-software.com  Thu Jan 14 14:15:09 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:09 +0100
Subject: [4.1] 37d1892 Add an explicit OC_F_INCOMPLETE flag to limit the
 amount of code that needs to know about oc->busyobj semantics
Message-ID: <E1aJig9-0005N7-Gk@project.varnish-software.com>


commit 37d189211cc05443dc97718966386e08f4030769
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Dec 14 20:33:50 2015 +0000

    Add an explicit OC_F_INCOMPLETE flag to limit the amount of code
    that needs to know about oc->busyobj semantics

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index 76357f8..84eef1d 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -404,6 +404,7 @@ struct objcore {
 	uint16_t		flags;
 #define OC_F_BUSY		(1<<1)
 #define OC_F_PASS		(1<<2)
+#define OC_F_INCOMPLETE		(1<<3)
 #define OC_F_PRIVATE		(1<<8)
 #define OC_F_FAILED		(1<<9)
 
diff --git a/bin/varnishd/cache/cache_expire.c b/bin/varnishd/cache/cache_expire.c
index 65236b7..5c27d07 100644
--- a/bin/varnishd/cache/cache_expire.c
+++ b/bin/varnishd/cache/cache_expire.c
@@ -231,7 +231,7 @@ EXP_Touch(struct objcore *oc, double now)
 
 	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
 
-	if (oc->busyobj != NULL)
+	if (oc->flags & OC_F_INCOMPLETE)
 		return;
 
 	if (now - oc->last_lru < cache_param->lru_interval)
diff --git a/bin/varnishd/cache/cache_gzip.c b/bin/varnishd/cache/cache_gzip.c
index aa516bf..917109a 100644
--- a/bin/varnishd/cache/cache_gzip.c
+++ b/bin/varnishd/cache/cache_gzip.c
@@ -307,8 +307,8 @@ VDP_gunzip(struct req *req, enum vdp_action act, void **priv,
 		http_Unset(req->resp, H_Content_Encoding);
 
 		req->resp_len = -1;
-		if (req->objcore->busyobj != NULL)
-			return (0); /* Incomplete, no idea about length */
+		if (req->objcore->flags & OC_F_INCOMPLETE)
+			return (0);	/* No idea about length */
 
 		p = ObjGetattr(req->wrk, req->objcore, OA_GZIPBITS, &dl);
 		if (p == NULL || dl != 32)
diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index d93320d..85da4f0 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -75,7 +75,7 @@ hsh_NewObjCore(struct worker *wrk)
 	ALLOC_OBJ(oc, OBJCORE_MAGIC);
 	XXXAN(oc);
 	wrk->stats->n_objectcore++;
-	oc->flags |= OC_F_BUSY;
+	oc->flags |= OC_F_BUSY | OC_F_INCOMPLETE;
 	return (oc);
 }
 
@@ -650,6 +650,7 @@ HSH_Fail(struct objcore *oc)
 
 	Lck_Lock(&oh->mtx);
 	oc->flags |= OC_F_FAILED;
+	oc->flags &= ~OC_F_INCOMPLETE;
 	oc->busyobj = NULL;
 	Lck_Unlock(&oh->mtx);
 }
@@ -669,6 +670,7 @@ HSH_Complete(struct objcore *oc)
 
 	Lck_Lock(&oh->mtx);
 	oc->busyobj = NULL;
+	oc->flags &= ~OC_F_INCOMPLETE;
 	Lck_Unlock(&oh->mtx);
 }
 
diff --git a/bin/varnishd/cache/cache_vrt_var.c b/bin/varnishd/cache/cache_vrt_var.c
index 5dd890e..b76c069 100644
--- a/bin/varnishd/cache/cache_vrt_var.c
+++ b/bin/varnishd/cache/cache_vrt_var.c
@@ -670,7 +670,7 @@ VRT_r_resp_is_streaming(VRT_CTX)
 	if (ctx->req->objcore == NULL)
 		return (0);	/* When called from vcl_synth */
 	CHECK_OBJ_NOTNULL(ctx->req->objcore, OBJCORE_MAGIC);
-	return (ctx->req->objcore->busyobj != NULL ? 1 : 0);
+	return (ctx->req->objcore->flags & OC_F_INCOMPLETE ? 1 : 0);
 }
 
 /*--------------------------------------------------------------------*/



From lkarsten at varnish-software.com  Thu Jan 14 14:15:09 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:09 +0100
Subject: [4.1] d699e14 Make VSB_bcat() take a ssize_t to be able to assert
 non-negative lengths.
Message-ID: <E1aJig9-0005NP-JQ@project.varnish-software.com>


commit d699e1406a5713f54f07368536bd8b41ff941167
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Dec 15 22:50:21 2015 +0000

    Make VSB_bcat() take a ssize_t to be able to assert non-negative lengths.
    
    Spotted by:	FlexeLint

diff --git a/include/vsb.h b/include/vsb.h
index 61b64c0..4b34314 100644
--- a/include/vsb.h
+++ b/include/vsb.h
@@ -61,7 +61,7 @@ struct vsb	*VSB_new(struct vsb *, char *, int, int);
 #define		 VSB_new_auto()				\
 	VSB_new(NULL, NULL, 0, VSB_AUTOEXTEND)
 void		 VSB_clear(struct vsb *);
-int		 VSB_bcat(struct vsb *, const void *, size_t);
+int		 VSB_bcat(struct vsb *, const void *, ssize_t);
 int		 VSB_cat(struct vsb *, const char *);
 int		 VSB_printf(struct vsb *, const char *, ...)
 	__v_printflike(2, 3);
diff --git a/lib/libvarnish/vsb.c b/lib/libvarnish/vsb.c
index ad98c8a..7b9380a 100644
--- a/lib/libvarnish/vsb.c
+++ b/lib/libvarnish/vsb.c
@@ -276,13 +276,16 @@ VSB_put_byte(struct vsb *s, int c)
  * Append a byte string to an vsb.
  */
 int
-VSB_bcat(struct vsb *s, const void *buf, size_t len)
+VSB_bcat(struct vsb *s, const void *buf, ssize_t len)
 {
 	assert_VSB_integrity(s);
 	assert_VSB_state(s, 0);
 
+	assert(len >= 0);
 	if (s->s_error != 0)
 		return (-1);
+	if (len == 0)
+		return (0);
 	_vsb_indent(s);
 	if (len > VSB_FREESPACE(s)) {
 		if (VSB_extend(s, len - VSB_FREESPACE(s)) < 0)



From lkarsten at varnish-software.com  Thu Jan 14 14:15:09 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:09 +0100
Subject: [4.1] 3a1093b Speed up VSB_bcat()
Message-ID: <E1aJig9-0005NG-IE@project.varnish-software.com>


commit 3a1093bfc368f7e7973f4c94a6843caf30142fc6
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Tue Dec 15 15:24:26 2015 +0000

    Speed up VSB_bcat()
    
    Rather than copying one char at the time extend the buffer as needed and
    use memcpy.

diff --git a/lib/libvarnish/vsb.c b/lib/libvarnish/vsb.c
index 92abdf2..ad98c8a 100644
--- a/lib/libvarnish/vsb.c
+++ b/lib/libvarnish/vsb.c
@@ -278,20 +278,20 @@ VSB_put_byte(struct vsb *s, int c)
 int
 VSB_bcat(struct vsb *s, const void *buf, size_t len)
 {
-	const char *str = buf;
-	const char *end = str + len;
-
 	assert_VSB_integrity(s);
 	assert_VSB_state(s, 0);
 
 	if (s->s_error != 0)
 		return (-1);
 	_vsb_indent(s);
-	for (; str < end; str++) {
-		VSB_put_byte(s, *str);
+	if (len > VSB_FREESPACE(s)) {
+		if (VSB_extend(s, len - VSB_FREESPACE(s)) < 0)
+			s->s_error = ENOMEM;
 		if (s->s_error != 0)
 			return (-1);
 	}
+	memcpy(s->s_buf + s->s_len, buf, len);
+	s->s_len += len;
 	return (0);
 }
 



From lkarsten at varnish-software.com  Thu Jan 14 14:15:09 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:09 +0100
Subject: [4.1] b4f8e2c Recognize NetBSD
Message-ID: <E1aJig9-0005NY-Kg@project.varnish-software.com>


commit b4f8e2c7947df795dff61bc757fcc7749b6e9043
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Wed Dec 16 16:31:17 2015 +0000

    Recognize NetBSD

diff --git a/autogen.sh b/autogen.sh
index 8a13e62..f299ab6 100755
--- a/autogen.sh
+++ b/autogen.sh
@@ -12,6 +12,9 @@ Darwin)
 FreeBSD)
     LIBTOOLIZE=libtoolize
     ;;
+NetBSD)
+    LIBTOOLIZE=libtoolize
+    ;;
 OpenBSD)
     LIBTOOLIZE=libtoolize
     ;;



From lkarsten at varnish-software.com  Thu Jan 14 14:15:09 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:09 +0100
Subject: [4.1] c9debc7 Remove the list of all mutexes,
 we have never used it for anything.
Message-ID: <E1aJig9-0005Nh-MX@project.varnish-software.com>


commit c9debc783016d35d16006fc039ce9fb43fe157e1
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Dec 16 22:36:51 2015 +0000

    Remove the list of all mutexes, we have never used it for anything.

diff --git a/bin/varnishd/cache/cache_lck.c b/bin/varnishd/cache/cache_lck.c
index c623ab7..f1cea24 100644
--- a/bin/varnishd/cache/cache_lck.c
+++ b/bin/varnishd/cache/cache_lck.c
@@ -41,25 +41,18 @@
 
 #include "cache.h"
 
-/*The constability of lck depends on platform pthreads implementation */
-
 struct ilck {
 	unsigned		magic;
 #define ILCK_MAGIC		0x7b86c8a5
-	pthread_mutex_t		mtx;
 	int			held;
+	pthread_mutex_t		mtx;
 	pthread_t		owner;
-	VTAILQ_ENTRY(ilck)	list;
 	const char		*w;
 	struct VSC_C_lck	*stat;
 };
 
 static pthread_mutexattr_t attr;
 
-static VTAILQ_HEAD(, ilck)	ilck_head = VTAILQ_HEAD_INITIALIZER(ilck_head);
-
-static pthread_mutex_t		lck_mtx;
-
 /*--------------------------------------------------------------------*/
 
 static void
@@ -226,9 +219,6 @@ Lck__New(struct lock *lck, struct VSC_C_lck *st, const char *w)
 	ilck->stat = st;
 	ilck->stat->creat++;
 	AZ(pthread_mutex_init(&ilck->mtx, &attr));
-	AZ(pthread_mutex_lock(&lck_mtx));
-	VTAILQ_INSERT_TAIL(&ilck_head, ilck, list);
-	AZ(pthread_mutex_unlock(&lck_mtx));
 	lck->priv = ilck;
 }
 
@@ -240,9 +230,6 @@ Lck_Delete(struct lock *lck)
 	CAST_OBJ_NOTNULL(ilck, lck->priv, ILCK_MAGIC);
 	ilck->stat->destroy++;
 	lck->priv = NULL;
-	AZ(pthread_mutex_lock(&lck_mtx));
-	VTAILQ_REMOVE(&ilck_head, ilck, list);
-	AZ(pthread_mutex_unlock(&lck_mtx));
 	AZ(pthread_mutex_destroy(&ilck->mtx));
 	FREE_OBJ(ilck);
 }
@@ -259,7 +246,6 @@ LCK_Init(void)
 #if !defined(__APPLE__) && !defined(__MACH__)
 	AZ(pthread_mutexattr_settype(&attr, PTHREAD_MUTEX_ERRORCHECK));
 #endif
-	AZ(pthread_mutex_init(&lck_mtx, &attr));
 #define LOCK(nam)						\
 	lck_##nam = VSM_Alloc(sizeof(struct VSC_C_lck),		\
 	   VSC_CLASS, VSC_type_lck, #nam);



From lkarsten at varnish-software.com  Thu Jan 14 14:15:09 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:09 +0100
Subject: [4.1] c153c93 Make it possible to creat LCK classes at runtime.
Message-ID: <E1aJig9-0005Nq-OH@project.varnish-software.com>


commit c153c9320ac66d523dc9cf8e03a2e44c0ca1e278
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Dec 16 23:04:42 2015 +0000

    Make it possible to creat LCK classes at runtime.
    
    Use the hash_classic class as proof of concept

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index 84eef1d..6c6a6ef 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -831,6 +831,8 @@ int Lck_CondWait(pthread_cond_t *cond, struct lock *lck, double);
 #define Lck_Trylock(a) Lck__Trylock(a, __func__, __LINE__)
 #define Lck_AssertHeld(a) Lck__Assert(a, 1)
 
+struct VSC_C_lck *Lck_CreateClass(const char *name);
+
 #define LOCK(nam) extern struct VSC_C_lck *lck_##nam;
 #include "tbl/locks.h"
 #undef LOCK
diff --git a/bin/varnishd/cache/cache_lck.c b/bin/varnishd/cache/cache_lck.c
index f1cea24..b702c2f 100644
--- a/bin/varnishd/cache/cache_lck.c
+++ b/bin/varnishd/cache/cache_lck.c
@@ -212,6 +212,7 @@ Lck__New(struct lock *lck, struct VSC_C_lck *st, const char *w)
 	struct ilck *ilck;
 
 	AN(st);
+	AN(w);
 	AZ(lck->priv);
 	ALLOC_OBJ(ilck, ILCK_MAGIC);
 	AN(ilck);
@@ -234,6 +235,13 @@ Lck_Delete(struct lock *lck)
 	FREE_OBJ(ilck);
 }
 
+struct VSC_C_lck *
+Lck_CreateClass(const char *name)
+{
+	return(VSM_Alloc(sizeof(struct VSC_C_lck),
+	   VSC_CLASS, VSC_type_lck, name));
+}
+
 #define LOCK(nam) struct VSC_C_lck *lck_##nam;
 #include "tbl/locks.h"
 #undef LOCK
@@ -246,9 +254,7 @@ LCK_Init(void)
 #if !defined(__APPLE__) && !defined(__MACH__)
 	AZ(pthread_mutexattr_settype(&attr, PTHREAD_MUTEX_ERRORCHECK));
 #endif
-#define LOCK(nam)						\
-	lck_##nam = VSM_Alloc(sizeof(struct VSC_C_lck),		\
-	   VSC_CLASS, VSC_type_lck, #nam);
+#define LOCK(nam)	lck_##nam = Lck_CreateClass(#nam);
 #include "tbl/locks.h"
 #undef LOCK
 }
diff --git a/bin/varnishd/hash/hash_classic.c b/bin/varnishd/hash/hash_classic.c
index ef817ea..29fffd9 100644
--- a/bin/varnishd/hash/hash_classic.c
+++ b/bin/varnishd/hash/hash_classic.c
@@ -38,6 +38,8 @@
 
 #include "hash/hash_slinger.h"
 
+static struct VSC_C_lck *lck_hcl;
+
 /*--------------------------------------------------------------------*/
 
 struct hcl_hd {
@@ -90,6 +92,7 @@ hcl_start(void)
 {
 	unsigned u;
 
+	lck_hcl = Lck_CreateClass("hcl");
 	hcl_head = calloc(sizeof *hcl_head, hcl_nhash);
 	XXXAN(hcl_head);
 
diff --git a/include/tbl/locks.h b/include/tbl/locks.h
index 8fe77fe..16af3ac 100644
--- a/include/tbl/locks.h
+++ b/include/tbl/locks.h
@@ -35,7 +35,6 @@ LOCK(busyobj)
 LOCK(cli)
 LOCK(exp)
 LOCK(hcb)
-LOCK(hcl)
 LOCK(hsl)
 LOCK(lru)
 LOCK(mempool)



From lkarsten at varnish-software.com  Thu Jan 14 14:15:09 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:09 +0100
Subject: [4.1] 33c2ac2 Also make this debugging hash class' locks dynamic
Message-ID: <E1aJig9-0005Nz-PT@project.varnish-software.com>


commit 33c2ac251e4446e8699556c9f0aa73fade318652
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Dec 16 23:26:32 2015 +0000

    Also make this debugging hash class' locks dynamic

diff --git a/bin/varnishd/hash/hash_simple_list.c b/bin/varnishd/hash/hash_simple_list.c
index d99bb89..a946d16 100644
--- a/bin/varnishd/hash/hash_simple_list.c
+++ b/bin/varnishd/hash/hash_simple_list.c
@@ -35,6 +35,8 @@
 
 #include "hash/hash_slinger.h"
 
+static struct VSC_C_lck *lck_hsl;
+
 /*--------------------------------------------------------------------*/
 
 static VTAILQ_HEAD(, objhead)	hsl_head = VTAILQ_HEAD_INITIALIZER(hsl_head);
@@ -49,6 +51,7 @@ static void __match_proto__(hash_start_f)
 hsl_start(void)
 {
 
+	lck_hsl = Lck_CreateClass("hsl");
 	Lck_New(&hsl_mtx, lck_hsl);
 }
 
diff --git a/include/tbl/locks.h b/include/tbl/locks.h
index 16af3ac..4a1539e 100644
--- a/include/tbl/locks.h
+++ b/include/tbl/locks.h
@@ -35,7 +35,6 @@ LOCK(busyobj)
 LOCK(cli)
 LOCK(exp)
 LOCK(hcb)
-LOCK(hsl)
 LOCK(lru)
 LOCK(mempool)
 LOCK(objhdr)



From lkarsten at varnish-software.com  Thu Jan 14 14:15:09 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:09 +0100
Subject: [4.1] f4f0fa3 Make the sma and smf locks dynamic.
Message-ID: <E1aJig9-0005O8-Qj@project.varnish-software.com>


commit f4f0fa33d9adbd7124b3082f991907db1df7c332
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu Dec 17 10:48:24 2015 +0000

    Make the sma and smf locks dynamic.
    
    Conflicts:
    	include/tbl/locks.h
    
    Merge note: Keeping persistence locks are they were, due to non-merged SML reorganisation.

diff --git a/bin/varnishd/storage/storage_file.c b/bin/varnishd/storage/storage_file.c
index 151f75e..152d3b2 100644
--- a/bin/varnishd/storage/storage_file.c
+++ b/bin/varnishd/storage/storage_file.c
@@ -62,6 +62,8 @@
  */
 #define NBUCKET			(128 / 4 + 1)
 
+static struct VSC_C_lck *lck_smf;
+
 /*--------------------------------------------------------------------*/
 
 VTAILQ_HEAD(smfhead, smf);
@@ -419,6 +421,9 @@ smf_open(const struct stevedore *st)
 	off_t fail = 1 << 30;	/* XXX: where is OFF_T_MAX ? */
 	off_t sum = 0;
 
+	ASSERT_CLI();
+	if (lck_smf == NULL)
+		lck_smf = Lck_CreateClass("smf");
 	CAST_OBJ_NOTNULL(sc, st->priv, SMF_SC_MAGIC);
 	sc->stats = VSM_Alloc(sizeof *sc->stats,
 	    VSC_CLASS, VSC_type_smf, st->ident);
diff --git a/bin/varnishd/storage/storage_malloc.c b/bin/varnishd/storage/storage_malloc.c
index ce6b0c2..87b634f 100644
--- a/bin/varnishd/storage/storage_malloc.c
+++ b/bin/varnishd/storage/storage_malloc.c
@@ -56,6 +56,8 @@ struct sma {
 	struct sma_sc		*sc;
 };
 
+static struct VSC_C_lck *lck_sma;
+
 static struct storage *
 sma_alloc(const struct stevedore *st, size_t size)
 {
@@ -234,6 +236,9 @@ sma_open(const struct stevedore *st)
 {
 	struct sma_sc *sma_sc;
 
+	ASSERT_CLI();
+	if (lck_sma == NULL)
+		lck_sma = Lck_CreateClass("sma");
 	CAST_OBJ_NOTNULL(sma_sc, st->priv, SMA_SC_MAGIC);
 	Lck_New(&sma_sc->sma_mtx, lck_sma);
 	sma_sc->stats = VSM_Alloc(sizeof *sma_sc->stats,
diff --git a/include/tbl/locks.h b/include/tbl/locks.h
index 4a1539e..7b4b171 100644
--- a/include/tbl/locks.h
+++ b/include/tbl/locks.h
@@ -40,8 +40,6 @@ LOCK(mempool)
 LOCK(objhdr)
 LOCK(pipestat)
 LOCK(sess)
-LOCK(sma)
-LOCK(smf)
 LOCK(smp)
 LOCK(vbe)
 LOCK(vcapace)



From lkarsten at varnish-software.com  Thu Jan 14 14:15:09 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:09 +0100
Subject: [4.1] ab04527 Polish
Message-ID: <E1aJig9-0005OH-S3@project.varnish-software.com>


commit ab045277aa47ab1d046a64338ad315fee924de56
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Fri Dec 18 13:55:49 2015 +0000

    Polish

diff --git a/lib/libvarnish/vtcp.c b/lib/libvarnish/vtcp.c
index 0c6a312..09bf909 100644
--- a/lib/libvarnish/vtcp.c
+++ b/lib/libvarnish/vtcp.c
@@ -324,10 +324,10 @@ VTCP_close(int *s)
 void
 VTCP_set_read_timeout(int s, double seconds)
 {
+#ifdef SO_RCVTIMEO_WORKS
 	struct timeval timeout;
 	timeout.tv_sec = (int)floor(seconds);
 	timeout.tv_usec = (int)(1e6 * (seconds - timeout.tv_sec));
-#ifdef SO_RCVTIMEO_WORKS
 	/*
 	 * Solaris bug (present at least in snv_151 and older): If this fails
 	 * with EINVAL, the socket is half-closed (SS_CANTSENDMORE) and the
@@ -338,6 +338,7 @@ VTCP_set_read_timeout(int s, double seconds)
 	    &timeout, sizeof timeout));
 #else
 	(void)s;
+	(void)seconds;
 #endif
 }
 



From lkarsten at varnish-software.com  Thu Jan 14 14:15:09 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:09 +0100
Subject: [4.1] 0a42d76 Add reference to varnish-counters(7).
Message-ID: <E1aJig9-0005OQ-TW@project.varnish-software.com>


commit 0a42d7676a98ca2506f4e28d0cf597b1660322de
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Mon Dec 21 13:20:51 2015 +0100

    Add reference to varnish-counters(7).

diff --git a/doc/sphinx/reference/varnishstat.rst b/doc/sphinx/reference/varnishstat.rst
index 5eef680..4b8d98c 100644
--- a/doc/sphinx/reference/varnishstat.rst
+++ b/doc/sphinx/reference/varnishstat.rst
@@ -201,6 +201,7 @@ between each block of output.
 SEE ALSO
 ========
 
+* :ref:`varnish-counters(7)`
 * :ref:`varnishd(1)`
 * :ref:`varnishhist(1)`
 * :ref:`varnishlog(1)`



From lkarsten at varnish-software.com  Thu Jan 14 14:15:09 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:09 +0100
Subject: [4.1] 9f66720 Change man page history into authors section.
Message-ID: <E1aJig9-0005OZ-VD@project.varnish-software.com>


commit 9f667202c68992c4ad3d168ac577386afa552cf9
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Mon Dec 21 13:20:54 2015 +0100

    Change man page history into authors section.
    
    We don't maintain any history, and the man page is covered by the
    distribution copyright.

diff --git a/doc/sphinx/reference/varnishstat.rst b/doc/sphinx/reference/varnishstat.rst
index 4b8d98c..80ba96b 100644
--- a/doc/sphinx/reference/varnishstat.rst
+++ b/doc/sphinx/reference/varnishstat.rst
@@ -209,20 +209,9 @@ SEE ALSO
 * :ref:`varnishtop(1)`
 * curses(3)
 
-HISTORY
+
+AUTHORS
 =======
 
-The varnishstat utility was originally developed by Poul-Henning Kamp
-<phk at phk.freebsd.dk> in cooperation with Verdens Gang AS and Varnish
-Software AS. Manual page written by Dag-Erling Sm?rgrav, Per Buer,
+This manual page was written by Dag-Erling Sm?rgrav, Per Buer,
 Lasse Karstensen and Martin Blix Grydeland.
-
-
-COPYRIGHT
-=========
-
-This document is licensed under the same licence as Varnish
-itself. See LICENCE for details.
-
-* Copyright (c) 2006 Verdens Gang AS
-* Copyright (c) 2006-2015 Varnish Software AS



From lkarsten at varnish-software.com  Thu Jan 14 14:15:09 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:09 +0100
Subject: [4.1] 01f0bd4 Add ref for sphinx.
Message-ID: <E1aJigA-0005Oi-01@project.varnish-software.com>


commit 01f0bd4dd3ddd18d3aaa47c9b44bf127a0fddffe
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Mon Dec 21 13:26:49 2015 +0100

    Add ref for sphinx.

diff --git a/man/vsc2rst.c b/man/vsc2rst.c
index 93950f0..11c3d2a 100644
--- a/man/vsc2rst.c
+++ b/man/vsc2rst.c
@@ -47,6 +47,8 @@ int main(int argc, char **argv)
 {
 	(void)argc;
 	(void)argv;
+	P(".. _varnish-counters(7): ");
+	P("");
 	P("================");
 	P("varnish-counters");
 	P("================");



From lkarsten at varnish-software.com  Thu Jan 14 14:15:10 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:10 +0100
Subject: [4.1] 5727c89 Move varnish-counters man page.
Message-ID: <E1aJigA-0005Or-1p@project.varnish-software.com>


commit 5727c89a86f9d589eb96ed32c9d3225d7104b46e
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Mon Dec 21 14:24:14 2015 +0100

    Move varnish-counters man page.
    
    To get this into the normal Sphinx build, move it to bin/varnishstat/
    (where it feels like it belongs the most), and use our normal build flow for man
    pages.

diff --git a/.gitignore b/.gitignore
index 5795c63..0cfaa04 100644
--- a/.gitignore
+++ b/.gitignore
@@ -76,7 +76,6 @@ cscope.*out
 
 
 # Man-files and binaries
-/man/vsc2rst
 /man/*.1
 /man/*.3
 /man/*.7
@@ -90,6 +89,7 @@ cscope.*out
 /bin/varnishncsa/varnishncsa
 /bin/varnishreplay/varnishreplay
 /bin/varnishstat/varnishstat
+/bin/varnishstat/vsc2rst
 /bin/varnishtest/varnishtest
 /bin/varnishtop/varnishtop
 
diff --git a/bin/varnishstat/Makefile.am b/bin/varnishstat/Makefile.am
index 664131d..a276444 100644
--- a/bin/varnishstat/Makefile.am
+++ b/bin/varnishstat/Makefile.am
@@ -19,3 +19,6 @@ varnishstat_LDADD = \
 	$(top_builddir)/lib/libvarnishcompat/libvarnishcompat.la \
 	$(top_builddir)/lib/libvarnishapi/libvarnishapi.la \
 	@CURSES_LIB@ ${RT_LIBS} ${LIBM} ${PTHREAD_LIBS}
+
+noinst_PROGRAMS = vsc2rst
+vsc2rst_SOURCES = vsc2rst.c $(top_srcdir)/include/tbl/vsc_fields.h
diff --git a/bin/varnishstat/vsc2rst.c b/bin/varnishstat/vsc2rst.c
new file mode 100644
index 0000000..3358db4
--- /dev/null
+++ b/bin/varnishstat/vsc2rst.c
@@ -0,0 +1,109 @@
+/*-
+ * Copyright (c) 2011-2015 Varnish Software AS
+ * All rights reserved.
+ *
+ * Author: Tollef Fog Heen <tfheen at varnish-software.com>
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED.  IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ *
+ */
+
+#include "config.h"
+
+#include <stdio.h>
+
+#define VSC_LEVEL_F(v,l,e,d)		\
+	static const char VSC_level_##v[] = l;
+#include "tbl/vsc_levels.h"
+#undef VSC_LEVEL_F
+
+#define P(x, ...)			\
+	printf(x "\n", ##__VA_ARGS__)
+#define VSC_LEVEL_F(v,l,e,d)		\
+	printf("%s ? %s\n\t%s\n\n", l, e, d);
+#define VSC_F(n, t, l, s, f, v, d, e)	\
+	printf("%s ? %s (%s)\n\t%s\n\n", #n, e, VSC_level_##v, d);
+
+int main(int argc, char **argv)
+{
+	(void)argc;
+	(void)argv;
+	P("COUNTER LEVELS");
+	P("==============");
+	P("");
+#include "tbl/vsc_levels.h"
+
+	P("");
+	P("MAIN COUNTERS (MAIN.*)");
+	P("======================");
+	P("");
+#include "tbl/vsc_f_main.h"
+
+	P("MANAGEMENT PROCESS COUNTERS (MGT.*)");
+	P("===================================");
+	P("");
+#define VSC_DO_MGT
+#include "tbl/vsc_fields.h"
+#undef VSC_DO_MGT
+
+	P("");
+	P("PER MEMORY POOL COUNTERS (MEMPOOL.*)");
+	P("====================================");
+	P("");
+#define VSC_DO_MEMPOOL
+#include "tbl/vsc_fields.h"
+#undef VSC_DO_MEMPOOL
+
+	P("");
+	P("PER MALLOC STORAGE COUNTERS (SMA.*)");
+	P("===================================");
+	P("");
+#define VSC_DO_SMA
+#include "tbl/vsc_fields.h"
+#undef  VSC_DO_SMA
+
+	P("");
+	P("PER FILE STORAGE COUNTERS (SMF.*)");
+	P("=================================");
+	P("");
+#define VSC_DO_SMF
+#include "tbl/vsc_fields.h"
+#undef VSC_DO_SMF
+
+	P("");
+	P("PER BACKEND COUNTERS (VBE.*)");
+	P("============================");
+	P("");
+#define VSC_DO_VBE
+#include "tbl/vsc_fields.h"
+#undef VSC_DO_VBE
+
+	P("");
+	P("LOCK COUNTERS (LCK.*)");
+	P("=====================");
+	P("");
+#define VSC_DO_LCK
+#include "tbl/vsc_fields.h"
+#undef VSC_DO_LCK
+
+	return (0);
+}
diff --git a/doc/sphinx/Makefile.am b/doc/sphinx/Makefile.am
index fbfe06b..b17ec99 100644
--- a/doc/sphinx/Makefile.am
+++ b/doc/sphinx/Makefile.am
@@ -128,6 +128,10 @@ include/params.rst: $(top_builddir)/bin/varnishd/varnishd
 	$(top_builddir)/bin/varnishd/varnishd -x dumprstparam > include/params.rst
 BUILT_SOURCES = include/params.rst
 
+include/counters.rst: $(top_builddir)/bin/varnishstat/vsc2rst
+	$(top_builddir)/bin/varnishstat/vsc2rst > $@
+BUILT_SOURCES += include/counters.rst
+
 # XXX add varnishstat here when it's been _opt2rst'ed
 
 include/varnishncsa_options.rst: $(top_builddir)/bin/varnishncsa/varnishncsa_opt2rst
diff --git a/doc/sphinx/reference/index.rst b/doc/sphinx/reference/index.rst
index e787b4e..603d1a0 100644
--- a/doc/sphinx/reference/index.rst
+++ b/doc/sphinx/reference/index.rst
@@ -23,6 +23,7 @@ The Varnish Reference Manual
 	vmod_std.generated.rst
 	vmod_directors.generated.rst
 	directors.rst
+	varnish-counters.rst
 	vsl.rst
 	vsl-query.rst
 
diff --git a/doc/sphinx/reference/varnish-counters.rst b/doc/sphinx/reference/varnish-counters.rst
new file mode 100644
index 0000000..781eeaa
--- /dev/null
+++ b/doc/sphinx/reference/varnish-counters.rst
@@ -0,0 +1,21 @@
+.. _varnish-counters(7):
+
+================
+varnish-counters
+================
+
+---------------------------------
+Varnish counter field definitions
+---------------------------------
+
+:Manual section: 7
+
+
+.. include:: ../include/counters.rst
+
+
+AUTHORS
+=======
+
+This man page was written by Lasse Karstensen, using content from vsc2rst
+written by Tollef Fog Heen.
diff --git a/man/Makefile.am b/man/Makefile.am
index 82f5471..35a8c52 100644
--- a/man/Makefile.am
+++ b/man/Makefile.am
@@ -1,11 +1,5 @@
 #
 
-noinst_PROGRAMS = vsc2rst
-vsc2rst_SOURCES = vsc2rst.c \
-	$(top_srcdir)/include/tbl/vsc_fields.h
-
-AM_CPPFLAGS = -I$(top_srcdir)/include
-
 dist_man_MANS = \
 	varnish-cli.7 \
 	varnish-counters.7 \
@@ -30,8 +24,8 @@ RST2ANY_FLAGS = --halt=2
 varnish-cli.7: $(top_srcdir)/doc/sphinx/reference/varnish-cli.rst
 	${RST2MAN} $(RST2ANY_FLAGS) $(top_srcdir)/doc/sphinx/reference/varnish-cli.rst $@
 
-varnish-counters.7: vsc2rst
-	./vsc2rst | ${RST2MAN} $(RST2ANY_FLAGS) - $@
+varnish-counters.7: $(top_srcdir)/doc/sphinx/reference/varnish-counters.rst
+	${RST2MAN} $(RST2ANY_FLAGS) $(top_srcdir)/doc/sphinx/reference/varnish-counters.rst $@
 
 vcl.7: $(top_srcdir)/doc/sphinx/reference/vcl.rst \
 	$(top_srcdir)/bin/varnishd/builtin.vcl
diff --git a/man/vsc2rst.c b/man/vsc2rst.c
deleted file mode 100644
index 11c3d2a..0000000
--- a/man/vsc2rst.c
+++ /dev/null
@@ -1,132 +0,0 @@
-/*-
- * Copyright (c) 2011-2015 Varnish Software AS
- * All rights reserved.
- *
- * Author: Tollef Fog Heen <tfheen at varnish-software.com>
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
- * 1. Redistributions of source code must retain the above copyright
- *    notice, this list of conditions and the following disclaimer.
- * 2. Redistributions in binary form must reproduce the above copyright
- *    notice, this list of conditions and the following disclaimer in the
- *    documentation and/or other materials provided with the distribution.
- *
- * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
- * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED.  IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
- * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
- * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
- * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
- * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
- * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
- * SUCH DAMAGE.
- *
- */
-
-#include "config.h"
-
-#include <stdio.h>
-
-#define VSC_LEVEL_F(v,l,e,d)		\
-	static const char VSC_level_##v[] = l;
-#include "tbl/vsc_levels.h"
-#undef VSC_LEVEL_F
-
-#define P(x, ...)			\
-	printf(x "\n", ##__VA_ARGS__)
-#define VSC_LEVEL_F(v,l,e,d)		\
-	printf("%s ? %s\n\t%s\n\n", l, e, d);
-#define VSC_F(n, t, l, s, f, v, d, e)	\
-	printf("%s ? %s (%s)\n\t%s\n\n", #n, e, VSC_level_##v, d);
-
-int main(int argc, char **argv)
-{
-	(void)argc;
-	(void)argv;
-	P(".. _varnish-counters(7): ");
-	P("");
-	P("================");
-	P("varnish-counters");
-	P("================");
-	P("");
-
-	P("---------------------------------");
-	P("Varnish counter field definitions");
-	P("---------------------------------");
-	P("");
-
-	P(":Manual section: 7");
-	P("");
-
-	P("COUNTER LEVELS");
-	P("==============");
-	P("");
-#include "tbl/vsc_levels.h"
-
-	P("");
-	P("MAIN COUNTERS (MAIN.*)");
-	P("======================");
-	P("");
-#include "tbl/vsc_f_main.h"
-
-	P("MANAGEMENT PROCESS COUNTERS (MGT.*)");
-	P("===================================");
-	P("");
-#define VSC_DO_MGT
-#include "tbl/vsc_fields.h"
-#undef VSC_DO_MGT
-
-	P("");
-	P("PER MEMORY POOL COUNTERS (MEMPOOL.*)");
-	P("====================================");
-	P("");
-#define VSC_DO_MEMPOOL
-#include "tbl/vsc_fields.h"
-#undef VSC_DO_MEMPOOL
-
-	P("");
-	P("PER MALLOC STORAGE COUNTERS (SMA.*)");
-	P("===================================");
-	P("");
-#define VSC_DO_SMA
-#include "tbl/vsc_fields.h"
-#undef  VSC_DO_SMA
-
-	P("");
-	P("PER FILE STORAGE COUNTERS (SMF.*)");
-	P("=================================");
-	P("");
-#define VSC_DO_SMF
-#include "tbl/vsc_fields.h"
-#undef VSC_DO_SMF
-
-	P("");
-	P("PER BACKEND COUNTERS (VBE.*)");
-	P("============================");
-	P("");
-#define VSC_DO_VBE
-#include "tbl/vsc_fields.h"
-#undef VSC_DO_VBE
-
-	P("");
-	P("LOCK COUNTERS (LCK.*)");
-	P("=====================");
-	P("");
-#define VSC_DO_LCK
-#include "tbl/vsc_fields.h"
-#undef VSC_DO_LCK
-
-	P("");
-	P("COPYRIGHT");
-	P("=========");
-	P("This document is licensed under the same licence as Varnish");
-	P("itself. See LICENCE for details.");
-	P("");
-	P("* Copyright (c) 2006 Verdens Gang AS");
-	P("* Copyright (c) 2006-2015 Varnish Software AS");
-	return (0);
-}



From lkarsten at varnish-software.com  Thu Jan 14 14:15:10 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:10 +0100
Subject: [4.1] 3605725 Use one level less of indirection.
Message-ID: <E1aJigA-0005P1-2s@project.varnish-software.com>


commit 36057259dfd4459349054143938c79da65dba4ff
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Mon Dec 21 14:29:06 2015 +0100

    Use one level less of indirection.
    
    Trips up my syntax highlighting.

diff --git a/doc/sphinx/Makefile.am b/doc/sphinx/Makefile.am
index b17ec99..e47a60d 100644
--- a/doc/sphinx/Makefile.am
+++ b/doc/sphinx/Makefile.am
@@ -7,10 +7,7 @@ SPHINXBUILD   = sphinx-build -W -q -N
 PAPER         = a4
 BUILDDIR      = build
 
-# Internal variables.
-PAPEROPT_a4     = -D latex_paper_size=a4
-PAPEROPT_letter = -D latex_paper_size=letter
-ALLSPHINXOPTS   = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) $(builddir)
+ALLSPHINXOPTS   = -d $(BUILDDIR)/doctrees -D latex_paper_size=$(PAPER) $(SPHINXOPTS) $(builddir)
 
 .PHONY: help clean html dirhtml pickle json htmlhelp qthelp latex changes linkcheck doctest
 



From lkarsten at varnish-software.com  Thu Jan 14 14:15:10 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:10 +0100
Subject: [4.1] 91eb520 Sort list of references.
Message-ID: <E1aJigA-0005PA-4d@project.varnish-software.com>


commit 91eb5208ff72b2345be496ce93152c15b135b695
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Mon Dec 21 14:47:31 2015 +0100

    Sort list of references.

diff --git a/doc/sphinx/reference/varnishstat.rst b/doc/sphinx/reference/varnishstat.rst
index 80ba96b..a0619ae 100644
--- a/doc/sphinx/reference/varnishstat.rst
+++ b/doc/sphinx/reference/varnishstat.rst
@@ -201,13 +201,13 @@ between each block of output.
 SEE ALSO
 ========
 
-* :ref:`varnish-counters(7)`
 * :ref:`varnishd(1)`
 * :ref:`varnishhist(1)`
 * :ref:`varnishlog(1)`
 * :ref:`varnishncsa(1)`
 * :ref:`varnishtop(1)`
 * curses(3)
+* :ref:`varnish-counters(7)`
 
 
 AUTHORS



From lkarsten at varnish-software.com  Thu Jan 14 14:15:10 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:10 +0100
Subject: [4.1] eeb2368 Avoid repeated copying of generated RST files.
Message-ID: <E1aJigA-0005PJ-63@project.varnish-software.com>


commit eeb2368fb37583e3e75eaf46349f1d2855c4839e
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Mon Dec 21 15:09:00 2015 +0100

    Avoid repeated copying of generated RST files.

diff --git a/doc/sphinx/Makefile.am b/doc/sphinx/Makefile.am
index e47a60d..52fa8de 100644
--- a/doc/sphinx/Makefile.am
+++ b/doc/sphinx/Makefile.am
@@ -159,17 +159,12 @@ include/varnishhist_synopsis.rst: $(top_builddir)/bin/varnishhist/varnishhist_op
 BUILT_SOURCES += include/varnishhist_options.rst \
 	 include/varnishhist_synopsis.rst
 
-.PHONY: reference
-reference:
-	test -d $@ || mkdir $@
-BUILT_SOURCES += reference
-
-reference/vmod_std.generated.rst: reference $(top_builddir)/lib/libvmod_std/vmod_std.rst
-	cp $(top_builddir)/lib/libvmod_std/vmod_std.rst $@
+reference/vmod_std.generated.rst: $(top_builddir)/lib/libvmod_std/vmod_std.rst
+	mkdir -p reference && cp $(top_builddir)/lib/libvmod_std/vmod_std.rst $@
 BUILT_SOURCES += reference/vmod_std.generated.rst
 
-reference/vmod_directors.generated.rst: reference $(top_builddir)/lib/libvmod_directors/vmod_directors.rst
-	cp $(top_builddir)/lib/libvmod_directors/vmod_directors.rst $@
+reference/vmod_directors.generated.rst: $(top_builddir)/lib/libvmod_directors/vmod_directors.rst
+	mkdir -p reference && cp $(top_builddir)/lib/libvmod_directors/vmod_directors.rst $@
 BUILT_SOURCES += reference/vmod_directors.generated.rst
 
 EXTRA_DIST += $(BUILT_SOURCES)



From lkarsten at varnish-software.com  Thu Jan 14 14:15:10 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:10 +0100
Subject: [4.1] 8149583 Fix distcheck
Message-ID: <E1aJigA-0005PS-7T@project.varnish-software.com>


commit 81495835c7bb0c551547ab59e3c226674a01585e
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Mon Dec 21 20:19:49 2015 +0000

    Fix distcheck

diff --git a/doc/sphinx/Makefile.am b/doc/sphinx/Makefile.am
index 52fa8de..0a4973c 100644
--- a/doc/sphinx/Makefile.am
+++ b/doc/sphinx/Makefile.am
@@ -159,12 +159,17 @@ include/varnishhist_synopsis.rst: $(top_builddir)/bin/varnishhist/varnishhist_op
 BUILT_SOURCES += include/varnishhist_options.rst \
 	 include/varnishhist_synopsis.rst
 
+.PHONY: reference
+reference:
+	test -d $@ || mkdir $@
+BUILT_SOURCES += reference
+
 reference/vmod_std.generated.rst: $(top_builddir)/lib/libvmod_std/vmod_std.rst
-	mkdir -p reference && cp $(top_builddir)/lib/libvmod_std/vmod_std.rst $@
+	cp $(top_builddir)/lib/libvmod_std/vmod_std.rst $@
 BUILT_SOURCES += reference/vmod_std.generated.rst
 
 reference/vmod_directors.generated.rst: $(top_builddir)/lib/libvmod_directors/vmod_directors.rst
-	mkdir -p reference && cp $(top_builddir)/lib/libvmod_directors/vmod_directors.rst $@
+	cp $(top_builddir)/lib/libvmod_directors/vmod_directors.rst $@
 BUILT_SOURCES += reference/vmod_directors.generated.rst
 
 EXTRA_DIST += $(BUILT_SOURCES)



From lkarsten at varnish-software.com  Thu Jan 14 14:15:10 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:10 +0100
Subject: [4.1] 6ba8a7b Revert d2fd2b9e and e28ba055
Message-ID: <E1aJigA-0005Pd-9P@project.varnish-software.com>


commit 6ba8a7b36368db8ff7dcb1b412f230eeb31bee29
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Wed Dec 23 08:58:23 2015 +0000

    Revert d2fd2b9e and e28ba055
    
    Unbreaks distcheck.

diff --git a/doc/sphinx/Makefile.am b/doc/sphinx/Makefile.am
index 0a4973c..e47a60d 100644
--- a/doc/sphinx/Makefile.am
+++ b/doc/sphinx/Makefile.am
@@ -164,11 +164,11 @@ reference:
 	test -d $@ || mkdir $@
 BUILT_SOURCES += reference
 
-reference/vmod_std.generated.rst: $(top_builddir)/lib/libvmod_std/vmod_std.rst
+reference/vmod_std.generated.rst: reference $(top_builddir)/lib/libvmod_std/vmod_std.rst
 	cp $(top_builddir)/lib/libvmod_std/vmod_std.rst $@
 BUILT_SOURCES += reference/vmod_std.generated.rst
 
-reference/vmod_directors.generated.rst: $(top_builddir)/lib/libvmod_directors/vmod_directors.rst
+reference/vmod_directors.generated.rst: reference $(top_builddir)/lib/libvmod_directors/vmod_directors.rst
 	cp $(top_builddir)/lib/libvmod_directors/vmod_directors.rst $@
 BUILT_SOURCES += reference/vmod_directors.generated.rst
 



From lkarsten at varnish-software.com  Thu Jan 14 14:15:10 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:10 +0100
Subject: [4.1] e124a58 Do not leak comments into the generated html
Message-ID: <E1aJigA-0005Pm-B1@project.varnish-software.com>


commit e124a585bf2b39e1d288f50a2192781f3f7953e1
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Wed Dec 23 10:40:37 2015 +0000

    Do not leak comments into the generated html

diff --git a/doc/sphinx/users-guide/increasing-your-hitrate.rst b/doc/sphinx/users-guide/increasing-your-hitrate.rst
index fe9219c..08eb09d 100644
--- a/doc/sphinx/users-guide/increasing-your-hitrate.rst
+++ b/doc/sphinx/users-guide/increasing-your-hitrate.rst
@@ -193,7 +193,9 @@ A somewhat simpler example that can accomplish almost the same functionality can
 found below. Instead of filtering out "other" cookies it instead picks out
 "the one" cookie that is needed, copies it to another header and then
 copies it back to the request, deleting the original cookie header.
+
 .. XXX:Verify correctness of request above! benc
+
 ::
 
     sub vcl_recv {
@@ -298,11 +300,9 @@ Normalizing your namespace
 Some sites are accessed via lots of hostnames.
 http://www.varnish-software.com/, http://varnish-software.com/ and
 http://varnishsoftware.com/ all point at the same site. Since Varnish
-doesn't know they are the same,
-.. XXX: heavy meaning change above. benc
-Varnish will cache different versions of every page for every
-hostname. You can mitigate this in your web server configuration
-by setting up redirects or by using the following VCL::
+doesn't know they are the same, Varnish will cache different versions of
+every page for every hostname. You can mitigate this in your web server
+configuration by setting up redirects or by using the following VCL::
 
     if (req.http.host ~ "(?i)^(www.)?varnish-?software.com") {
         set req.http.host = "varnish-software.com";
diff --git a/doc/sphinx/users-guide/operation-logging.rst b/doc/sphinx/users-guide/operation-logging.rst
index bb4c7cc..e6b58f2 100644
--- a/doc/sphinx/users-guide/operation-logging.rst
+++ b/doc/sphinx/users-guide/operation-logging.rst
@@ -31,8 +31,8 @@ You'll see lines like these scrolling slowly by.::
 These is the Varnish master process checking up on the caching process
 to see that everything is OK.
 
-Now go to the browser and reload the page displaying your web
-app.
+Now go to the browser and reload the page displaying your web app.
+
 .. XXX:Doesn't this require a setup of a running varnishd and a web application being cached? benc
 
 You'll see lines like these.::
diff --git a/doc/sphinx/users-guide/operation-statistics.rst b/doc/sphinx/users-guide/operation-statistics.rst
index 7a4827d..af062bd 100644
--- a/doc/sphinx/users-guide/operation-statistics.rst
+++ b/doc/sphinx/users-guide/operation-statistics.rst
@@ -29,11 +29,9 @@ varnishhist
 The :ref:`varnishhist(1)` utility reads :ref:`varnishd(1)` shared
 memory logs and presents a continuously updated histogram showing the
 distribution of the last N requests by their processing.
-.. XXX:1? benc
-The value of N and the
-vertical scale are displayed in the top left corner.  The horizontal
-scale is logarithmic.  Hits are marked with a pipe character ("|"),
-and misses are marked with a hash character ("#").
+The value of N and the vertical scale are displayed in the top left
+corner.  The horizontal scale is logarithmic.  Hits are marked with a
+pipe character ("|"), and misses are marked with a hash character ("#").
 
 varnishstat
 ~~~~~~~~~~~



From lkarsten at varnish-software.com  Thu Jan 14 14:15:10 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:10 +0100
Subject: [4.1] 47f0412 Fixing sentence
Message-ID: <E1aJigA-0005Pv-CP@project.varnish-software.com>


commit 47f04129d3ee8ea38046cc0e5afddd7e06319c7a
Author: Francisco Vel?zquez <francisco at varnish-software.com>
Date:   Wed Dec 23 15:22:52 2015 +0000

    Fixing sentence

diff --git a/doc/sphinx/users-guide/esi.rst b/doc/sphinx/users-guide/esi.rst
index 30379a1..0f16405 100644
--- a/doc/sphinx/users-guide/esi.rst
+++ b/doc/sphinx/users-guide/esi.rst
@@ -3,7 +3,7 @@
 Content composition with Edge Side Includes
 -------------------------------------------
 
-Varnish can cache create web pages by assembling different pages, called `fragments`,
+Varnish can create web pages by assembling different pages, called `fragments`,
 together into one page. These `fragments` can have individual cache policies. If you
 have a web site with a list showing the five most popular articles on
 your site, this list can probably be cached as a `fragment` and included



From lkarsten at varnish-software.com  Thu Jan 14 14:15:10 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:10 +0100
Subject: [4.1] 4994034 Kill some dead assignments
Message-ID: <E1aJigA-0005Q6-EL@project.varnish-software.com>


commit 499403472c9107d2ab47d097d964102a5119afa0
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Thu Dec 24 13:32:10 2015 +0000

    Kill some dead assignments

diff --git a/bin/varnishd/cache/cache_session.c b/bin/varnishd/cache/cache_session.c
index 3d23c7c..f1bb4c0 100644
--- a/bin/varnishd/cache/cache_session.c
+++ b/bin/varnishd/cache/cache_session.c
@@ -238,7 +238,6 @@ SES_RxStuff(struct http_conn *htc, htc_complete_f *func,
 			/* Working on it */
 			if (t1 != NULL && isnan(*t1))
 				*t1 = now;
-			tmo = tn - now;
 		} else if (hs != HTC_S_EMPTY)
 			WRONG("htc_status_e");
 
diff --git a/bin/varnishstat/varnishstat_curses.c b/bin/varnishstat/varnishstat_curses.c
index a27d438..174df57 100644
--- a/bin/varnishstat/varnishstat_curses.c
+++ b/bin/varnishstat/varnishstat_curses.c
@@ -943,10 +943,8 @@ draw_bar_b(void)
 		    level->label);
 		X -= strlen(level->label) + 2;
 	}
-	if (!hide_unseen) {
+	if (!hide_unseen)
 		mvwprintw(w_bar_b, 0, X - 6, "%s", "UNSEEN");
-		X -= 8;
-	}
 
 	wnoutrefresh(w_bar_b);
 }
diff --git a/bin/varnishtest/vtc_logexp.c b/bin/varnishtest/vtc_logexp.c
index 767f7e6..1c0ea70 100644
--- a/bin/varnishtest/vtc_logexp.c
+++ b/bin/varnishtest/vtc_logexp.c
@@ -283,7 +283,6 @@ logexp_thread(void *priv)
 	AN(le->vsm);
 	AN(le->vslq);
 
-	i = 0;
 	AZ(le->test);
 	vtc_log(le->vl, 4, "beg|");
 	if (le->query != NULL)
diff --git a/bin/varnishtest/vtc_varnish.c b/bin/varnishtest/vtc_varnish.c
index 1c00ed0..292be63 100644
--- a/bin/varnishtest/vtc_varnish.c
+++ b/bin/varnishtest/vtc_varnish.c
@@ -237,14 +237,10 @@ varnishlog_thread(void *priv)
 
 		tag = VSL_TAG(c->rec.ptr);
 		vxid = VSL_ID(c->rec.ptr);
-		if (tag == SLT__Batch) {
-			tagname = "Batch";
-			len = 0;
+		if (tag == SLT__Batch)
 			continue;
-		} else {
-			tagname = VSL_tags[tag];
-			len = VSL_LEN(c->rec.ptr);
-		}
+		tagname = VSL_tags[tag];
+		len = VSL_LEN(c->rec.ptr);
 		type = VSL_CLIENT(c->rec.ptr) ? 'c' : VSL_BACKEND(c->rec.ptr) ?
 		    'b' : '-';
 		data = VSL_CDATA(c->rec.ptr);
diff --git a/lib/libvcc/vcc_acl.c b/lib/libvcc/vcc_acl.c
index b9722ce..b5ee224 100644
--- a/lib/libvcc/vcc_acl.c
+++ b/lib/libvcc/vcc_acl.c
@@ -348,7 +348,6 @@ vcc_acl_emit(struct vcc *tl, const char *acln, int anon)
 	struct acl_e *ae;
 	int depth, l, m, i;
 	unsigned at[VRT_ACL_MAXADDR + 1];
-	const char *oc;
 	struct token *t;
 	struct inifin *ifp;
 
@@ -371,7 +370,6 @@ vcc_acl_emit(struct vcc *tl, const char *acln, int anon)
 			"\tif (0) match_acl_named_%s(0, 0);\n", acln);
 	}
 	depth = -1;
-	oc = 0;
 	at[0] = 256;
 	VTAILQ_FOREACH(ae, &tl->acl, list) {
 
@@ -383,7 +381,6 @@ vcc_acl_emit(struct vcc *tl, const char *acln, int anon)
 		}
 
 		/* Back down, if necessary */
-		oc = "";
 		while (l <= depth) {
 			Fh(tl, 0, "\t%*s}\n", -depth, "");
 			depth--;
@@ -397,23 +394,21 @@ vcc_acl_emit(struct vcc *tl, const char *acln, int anon)
 		for (i = l; m >= 8; m -= 8, i++) {
 			if (i == 0)
 				Fh(tl, 0, "\t%*s%sif (fam == %d) {\n",
-				    -i, "", oc, ae->data[i]);
+				    -i, "", "", ae->data[i]);
 			else
 				Fh(tl, 0, "\t%*s%sif (a[%d] == %d) {\n",
-				    -i, "", oc, i - 1, ae->data[i]);
+				    -i, "", "", i - 1, ae->data[i]);
 			at[i] = ae->data[i];
 			depth = i;
-			oc = "";
 		}
 
 		if (m > 0) {
 			/* Do fractional byte compares */
 			Fh(tl, 0, "\t%*s%sif ((a[%d] & 0x%x) == %d) {\n",
-			    -i, "", oc, i - 1, (0xff00 >> m) & 0xff,
+			    -i, "", "", i - 1, (0xff00 >> m) & 0xff,
 			    ae->data[i] & ((0xff00 >> m) & 0xff));
 			at[i] = 256;
 			depth = i;
-			oc = "";
 		}
 
 		i = (ae->mask + 7) / 8;



From lkarsten at varnish-software.com  Thu Jan 14 14:15:10 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:10 +0100
Subject: [4.1] 538f8bb vdef.h is not generated
Message-ID: <E1aJigA-0005QF-Fv@project.varnish-software.com>


commit 538f8bb8be96c341935242486f45a2a8ec51fb00
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Tue Dec 29 01:35:21 2015 +0000

    vdef.h is not generated

diff --git a/lib/libvcc/generate.py b/lib/libvcc/generate.py
index 7a7413f..c799bed 100755
--- a/lib/libvcc/generate.py
+++ b/lib/libvcc/generate.py
@@ -1228,7 +1228,7 @@ vcl_output_lang_h(struct vsb *sb)
 {
 """)
 
-emit_file(fo, buildroot, "include/vdef.h")
+emit_file(fo, srcroot, "include/vdef.h")
 emit_file(fo, buildroot, "include/vcl.h")
 emit_file(fo, srcroot, "include/vrt.h")
 emit_file(fo, buildroot, "include/vrt_obj.h")



From lkarsten at varnish-software.com  Thu Jan 14 14:15:10 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:10 +0100
Subject: [4.1] 03f3394 Devicedetect git repository has moved.
Message-ID: <E1aJigA-0005QO-IJ@project.varnish-software.com>


commit 03f3394dcbc6bd7eb0f35c28fd1b61e008ea2ea4
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Mon Jan 4 15:25:33 2016 +0100

    Devicedetect git repository has moved.

diff --git a/doc/sphinx/users-guide/devicedetection.rst b/doc/sphinx/users-guide/devicedetection.rst
index 8027909..91a608e 100644
--- a/doc/sphinx/users-guide/devicedetection.rst
+++ b/doc/sphinx/users-guide/devicedetection.rst
@@ -29,7 +29,7 @@ Setting this header can be as simple as::
 There are different commercial and free offerings in doing grouping and
 identifying clients in further detail. For a basic and community
 based regular expression set, see
-https://github.com/varnish/varnish-devicedetect/.
+https://github.com/varnishcache/varnish-devicedetect/.
 
 
 Serve the different content on the same URL
diff --git a/etc/devicedetect.vcl b/etc/devicedetect.vcl
index 7b5c721..d1a85d9 100644
--- a/etc/devicedetect.vcl
+++ b/etc/devicedetect.vcl
@@ -1,4 +1,4 @@
-# Copyright (c) 2012-2015 Varnish Software AS
+# Copyright (c) 2012-2014 Varnish Software AS
 #
 # Redistribution and use in source and binary forms, with or without
 # modification, are permitted provided that the following conditions
@@ -22,7 +22,7 @@
 # SUCH DAMAGE.
 #
 # detectdevice.vcl - regex based device detection for Varnish
-# https://github.com/varnish/varnish-devicedetect/
+# https://github.com/varnishcache/varnish-devicedetect/
 #
 # Author: Lasse Karstensen <lkarsten at varnish-software.com>
 



From lkarsten at varnish-software.com  Thu Jan 14 14:15:10 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:10 +0100
Subject: [4.1] 0450885 Make the "debug +vclrel" trick work for all worker
 threads.
Message-ID: <E1aJigA-0005QX-Jo@project.varnish-software.com>


commit 04508856b38fbce1cefc67e8d0e501b2257874c1
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Jan 4 20:44:19 2016 +0000

    Make the "debug +vclrel" trick work for all worker threads.

diff --git a/bin/varnishd/cache/cache_session.c b/bin/varnishd/cache/cache_session.c
index f1bb4c0..34c4374 100644
--- a/bin/varnishd/cache/cache_session.c
+++ b/bin/varnishd/cache/cache_session.c
@@ -321,8 +321,6 @@ SES_Proto_Req(struct worker *wrk, void *arg)
 		WRONG("Wrong session step");
 	}
 	WS_Assert(wrk->aws);
-	if (DO_DEBUG(DBG_VCLREL) && wrk->vcl != NULL)
-		VCL_Rel(&wrk->vcl);
 	THR_SetRequest(NULL);
 }
 
diff --git a/bin/varnishd/cache/cache_wrk.c b/bin/varnishd/cache/cache_wrk.c
index 7b774b1..7176f09 100644
--- a/bin/varnishd/cache/cache_wrk.c
+++ b/bin/varnishd/cache/cache_wrk.c
@@ -339,6 +339,8 @@ Pool_Work_Thread(struct pool *pp, struct worker *wrk)
 			memset(&wrk->task, 0, sizeof wrk->task);
 			assert(wrk->pool == pp);
 			tp->func(wrk, tp->priv);
+			if (DO_DEBUG(DBG_VCLREL) && wrk->vcl != NULL)
+				VCL_Rel(&wrk->vcl);
 			tpx = wrk->task;
 			tp = &tpx;
 		} while (tp->func != NULL);



From lkarsten at varnish-software.com  Thu Jan 14 14:15:10 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:10 +0100
Subject: [4.1] 634da2a Dump oc->expire info
Message-ID: <E1aJigA-0005Qh-LD@project.varnish-software.com>


commit 634da2ac0b5475bdc947a1c41631a92d5f040619
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Jan 5 10:19:31 2016 +0000

    Dump oc->expire info

diff --git a/bin/varnishd/cache/cache_panic.c b/bin/varnishd/cache/cache_panic.c
index b60ddbd..0613604 100644
--- a/bin/varnishd/cache/cache_panic.c
+++ b/bin/varnishd/cache/cache_panic.c
@@ -197,6 +197,9 @@ pan_objcore(struct vsb *vsb, const char *typ, const struct objcore *oc)
 	VSB_indent(vsb, 2);
 	VSB_printf(vsb, "refcnt = %d,\n", oc->refcnt);
 	VSB_printf(vsb, "flags = 0x%x,\n", oc->flags);
+	VSB_printf(vsb, "exp_flags = 0x%x,\n", oc->exp_flags);
+	VSB_printf(vsb, "exp = { %f, %f, %f, %f }\n",
+	    oc->exp.t_origin, oc->exp.ttl, oc->exp.grace, oc->exp.keep);
 	VSB_printf(vsb, "objhead = %p,\n", oc->objhead);
 	VSB_printf(vsb, "stevedore = %p", oc->stobj->stevedore);
 	if (oc->stobj->stevedore != NULL) {



From lkarsten at varnish-software.com  Thu Jan 14 14:15:10 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:10 +0100
Subject: [4.1] 9838972 Cleanup
Message-ID: <E1aJigA-0005Qq-MV@project.varnish-software.com>


commit 98389728c42d9c28b27613e88ff8ac7cd564d20c
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Jan 5 11:53:29 2016 +0000

    Cleanup

diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index fe26ac4..cd64257 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -777,18 +777,14 @@ vbf_stp_error(struct worker *wrk, struct busyobj *bo)
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
 	CHECK_OBJ_NOTNULL(bo->fetch_objcore, OBJCORE_MAGIC);
+	AN(bo->fetch_objcore->flags & OC_F_BUSY);
 	assert(bo->director_state == DIR_S_NULL);
 
-	if(bo->fetch_objcore->stobj->stevedore != NULL)
-		ObjFreeObj(bo->wrk, bo->fetch_objcore);
-
 	now = W_TIM_real(wrk);
 	VSLb_ts_busyobj(bo, "Error", now);
 
-	AN(bo->fetch_objcore->flags & OC_F_BUSY);
-
-	synth_body = VSB_new_auto();
-	AN(synth_body);
+	if(bo->fetch_objcore->stobj->stevedore != NULL)
+		ObjFreeObj(bo->wrk, bo->fetch_objcore);
 
 	// XXX: reset all beresp flags ?
 
@@ -800,26 +796,23 @@ vbf_stp_error(struct worker *wrk, struct busyobj *bo)
 	EXP_Clr(&bo->fetch_objcore->exp);
 	bo->fetch_objcore->exp.t_origin = bo->t_prev;
 
+	synth_body = VSB_new_auto();
+	AN(synth_body);
+
 	VCL_backend_error_method(bo->vcl, wrk, NULL, bo, synth_body);
 
 	AZ(VSB_finish(synth_body));
 
-	if (wrk->handling == VCL_RET_RETRY ||
-	    wrk->handling == VCL_RET_ABANDON) {
+	if (wrk->handling == VCL_RET_ABANDON) {
 		VSB_delete(synth_body);
+		return (F_STP_FAIL);
+	}
 
-		if (bo->director_state != DIR_S_NULL) {
-			bo->htc->doclose = SC_RESP_CLOSE;
-			VDI_Finish(bo->wrk, bo);
-		}
-
-		if (wrk->handling == VCL_RET_RETRY) {
-			if (bo->retries++ < cache_param->max_retries)
-				return (F_STP_RETRY);
-			VSLb(bo->vsl, SLT_VCL_Error,
-			    "Too many retries, delivering 503");
-		}
-
+	if (wrk->handling == VCL_RET_RETRY) {
+		VSB_delete(synth_body);
+		if (bo->retries++ < cache_param->max_retries)
+			return (F_STP_RETRY);
+		VSLb(bo->vsl, SLT_VCL_Error, "Too many retries, failing");
 		return (F_STP_FAIL);
 	}
 
diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index 85da4f0..dd03596 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -814,6 +814,7 @@ HSH_DerefObjHead(struct worker *wrk, struct objhead **poh)
 	CHECK_OBJ_NOTNULL(oh, OBJHEAD_MAGIC);
 
 	if (oh == private_oh) {
+		AZ(oh->waitinglist);
 		Lck_Lock(&oh->mtx);
 		assert(oh->refcnt > 1);
 		oh->refcnt--;



From lkarsten at varnish-software.com  Thu Jan 14 14:15:10 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:10 +0100
Subject: [4.1] 66273bb Spring some AN() love
Message-ID: <E1aJigA-0005Qz-OF@project.varnish-software.com>


commit 66273bb38e68eb11b4f4bd99c450f1f1c6958095
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Tue Jan 5 10:42:02 2016 +0000

    Spring some AN() love

diff --git a/bin/varnishncsa/varnishncsa.c b/bin/varnishncsa/varnishncsa.c
index dfb4885..249efee 100644
--- a/bin/varnishncsa/varnishncsa.c
+++ b/bin/varnishncsa/varnishncsa.c
@@ -531,6 +531,7 @@ addf_auth(const char *str)
 	struct format *f;
 
 	ALLOC_OBJ(f, FORMAT_MAGIC);
+	AN(f);
 	f->func = &format_auth;
 	if (str != NULL) {
 		f->string = strdup(str);
diff --git a/lib/libvarnishapi/vsc.c b/lib/libvarnishapi/vsc.c
index c86852d..f051cea 100644
--- a/lib/libvarnishapi/vsc.c
+++ b/lib/libvarnishapi/vsc.c
@@ -105,6 +105,7 @@ vsc_setup(struct VSM_data *vd)
 	CHECK_OBJ_NOTNULL(vd, VSM_MAGIC);
 	if (vd->vsc == NULL) {
 		ALLOC_OBJ(vd->vsc, VSC_MAGIC);
+		AN(vd->vsc);
 		VTAILQ_INIT(&vd->vsc->vf_list);
 		VTAILQ_INIT(&vd->vsc->pt_list);
 		VTAILQ_INIT(&vd->vsc->sf_list);



From lkarsten at varnish-software.com  Thu Jan 14 14:15:10 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:10 +0100
Subject: [4.1] b42feb8 Use realloc if possible
Message-ID: <E1aJigA-0005R8-Pu@project.varnish-software.com>


commit b42feb8029bb93f6125d53acd04c71eda45fa385
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Mon Dec 28 16:41:31 2015 +0000

    Use realloc if possible
    
    Avoids memcpy/free if the buffer was dynamic.

diff --git a/lib/libvarnish/vsb.c b/lib/libvarnish/vsb.c
index 7b9380a..4d80b8a 100644
--- a/lib/libvarnish/vsb.c
+++ b/lib/libvarnish/vsb.c
@@ -142,14 +142,16 @@ VSB_extend(struct vsb *s, int addlen)
 	if (!VSB_CANEXTEND(s))
 		return (-1);
 	newsize = VSB_extendsize(s->s_size + addlen);
-	newbuf = SBMALLOC(newsize);
-	if (newbuf == NULL)
-		return (-1);
-	memcpy(newbuf, s->s_buf, s->s_size);
 	if (VSB_ISDYNAMIC(s))
-		SBFREE(s->s_buf);
+		newbuf = realloc(s->s_buf, newsize);
 	else
+		newbuf = SBMALLOC(newsize);
+	if (newbuf == NULL)
+		return (-1);
+	if (!VSB_ISDYNAMIC(s)) {
+		memcpy(newbuf, s->s_buf, s->s_size);
 		VSB_SETFLAG(s, VSB_DYNAMIC);
+	}
 	s->s_buf = newbuf;
 	s->s_size = newsize;
 	return (0);



From lkarsten at varnish-software.com  Thu Jan 14 14:15:10 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:10 +0100
Subject: [4.1] 28aef48 Update parameters in preparation for further work
Message-ID: <E1aJigA-0005RH-RZ@project.varnish-software.com>


commit 28aef48a41060b774840db6436fda84ef79fb888
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Tue Jan 5 22:43:58 2016 +0000

    Update parameters in preparation for further work

diff --git a/bin/varnishd/common/params.h b/bin/varnishd/common/params.h
index 44f2f2a..957b8a9 100644
--- a/bin/varnishd/common/params.h
+++ b/bin/varnishd/common/params.h
@@ -63,16 +63,26 @@ struct poolparam {
 struct params {
 
 #define	ptyp_bool	unsigned
+#define	ptyp_bytes	ssize_t
+#define	ptyp_bytes_u	unsigned
 #define	ptyp_double	double
+#define	ptyp_poolparam	struct poolparam
 #define	ptyp_timeout	double
 #define	ptyp_uint	unsigned
+#define	ptyp_vsl_buffer	unsigned
+#define	ptyp_vsl_reclen	unsigned
 #define PARAM(nm, ty, mi, ma, de, un, fl, st, lt, fn) ptyp_##ty nm;
 #include <tbl/params.h>
 #undef PARAM
 #undef ptyp_bool
+#undef ptyp_bytes
+#undef ptyp_bytes_u
 #undef ptyp_double
+#undef ptyp_poolparam
 #undef ptyp_timeout
 #undef ptyp_uint
+#undef ptyp_vsl_buffer
+#undef ptyp_vsl_reclen
 
 	/* Unprivileged user / group */
 	char			*user;
diff --git a/include/tbl/params.h b/include/tbl/params.h
index 41f5b86..f372351 100644
--- a/include/tbl/params.h
+++ b/include/tbl/params.h
@@ -128,7 +128,7 @@ PARAM(
 
 PARAM(
 	/* name */	ban_lurker_age,
-	/* tweak */	timeout,
+	/* typ */	timeout,
 	/* min */	"0",
 	/* max */	NULL,
 	/* default */	"60",
@@ -145,7 +145,7 @@ PARAM(
 
 PARAM(
 	/* name */	ban_lurker_batch,
-	/* tweak */	uint,
+	/* typ */	uint,
 	/* min */	"1",
 	/* max */	NULL,
 	/* default */	"1000",
@@ -161,7 +161,7 @@ PARAM(
 
 PARAM(
 	/* name */	ban_lurker_sleep,
-	/* tweak */	timeout,
+	/* typ */	timeout,
 	/* min */	"0",
 	/* max */	NULL,
 	/* default */	"0.010",
@@ -178,7 +178,7 @@ PARAM(
 
 PARAM(
 	/* name */	first_byte_timeout,
-	/* tweak */	timeout,
+	/* typ */	timeout,
 	/* min */	"0",
 	/* max */	NULL,
 	/* default */	"60",
@@ -196,7 +196,7 @@ PARAM(
 
 PARAM(
 	/* name */	between_bytes_timeout,
-	/* tweak */	timeout,
+	/* typ */	timeout,
 	/* min */	"0",
 	/* max */	NULL,
 	/* default */	"60",
@@ -214,7 +214,7 @@ PARAM(
 
 PARAM(
 	/* name */	backend_idle_timeout,
-	/* tweak */	timeout,
+	/* typ */	timeout,
 	/* min */	"1",
 	/* max */	NULL,
 	/* default */	"60",
@@ -231,108 +231,101 @@ PARAM(
 
 PARAM(
 	/* name */	cli_buffer,
-	/* tweak */	tweak_bytes_u,
-	/* var */	cli_buffer,
-	/* min */	4k,
-	/* max */	none,
-	/* default */	8k,
-	/* units */	bytes,
-	/* flags */	00,
+	/* typ */	bytes_u,
+	/* min */	"4k",
+	/* max */	NULL,
+	/* default */	"8k",
+	/* units */	"bytes",
+	/* flags */	0,
 	/* s-text */
 	"Size of buffer for CLI command input.\n"
 	"You may need to increase this if you have big VCL files and use "
 	"the vcl.inline CLI command.\n"
-	"NB: Must be specified with -p to have effect.\n",
+	"NB: Must be specified with -p to have effect.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	cli_limit,
-	/* tweak */	tweak_bytes_u,
-	/* var */	cli_limit,
-	/* min */	128b,
-	/* max */	99999999b,
-	/* default */	48k,
-	/* units */	bytes,
-	/* flags */	00,
+	/* typ */	bytes_u,
+	/* min */	"128b",
+	/* max */	"99999999b",
+	/* default */	"48k",
+	/* units */	"bytes",
+	/* flags */	0,
 	/* s-text */
 	"Maximum size of CLI response.  If the response exceeds this "
 	"limit, the response code will be 201 instead of 200 and the last "
-	"line will indicate the truncation.\n",
+	"line will indicate the truncation.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	cli_timeout,
-	/* tweak */	tweak_timeout,
-	/* var */	cli_timeout,
-	/* min */	0.000,
-	/* max */	none,
-	/* default */	60.000,
-	/* units */	seconds,
-	/* flags */	00,
+	/* typ */	timeout,
+	/* min */	"0.000",
+	/* max */	NULL,
+	/* default */	"60.000",
+	/* units */	"seconds",
+	/* flags */	0,
 	/* s-text */
 	"Timeout for the childs replies to CLI requests from the "
-	"mgt_param.\n",
+	"mgt_param.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	clock_skew,
-	/* tweak */	tweak_uint,
-	/* var */	clock_skew,
-	/* min */	0,
-	/* max */	none,
-	/* default */	10,
-	/* units */	seconds,
-	/* flags */	00,
+	/* typ */	uint,
+	/* min */	"0",
+	/* max */	NULL,
+	/* default */	"10",
+	/* units */	"seconds",
+	/* flags */	0,
 	/* s-text */
 	"How much clockskew we are willing to accept between the backend "
-	"and our own clock.\n",
+	"and our own clock.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	connect_timeout,
-	/* tweak */	tweak_timeout,
-	/* var */	connect_timeout,
-	/* min */	0.000,
-	/* max */	none,
-	/* default */	3.500,
-	/* units */	seconds,
-	/* flags */	00,
+	/* typ */	timeout,
+	/* min */	"0.000",
+	/* max */	NULL,
+	/* default */	"3.500",
+	/* units */	"seconds",
+	/* flags */	0,
 	/* s-text */
 	"Default connection timeout for backend connections. We only try "
 	"to connect to the backend for this many seconds before giving up. "
 	"VCL can override this default value for each backend and backend "
-	"request.\n",
+	"request.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	critbit_cooloff,
-	/* tweak */	tweak_timeout,
-	/* var */	critbit_cooloff,
-	/* min */	60.000,
-	/* max */	254.000,
-	/* default */	180.000,
-	/* units */	seconds,
-	/* flags */	0| WIZARD,
+	/* typ */	timeout,
+	/* min */	"60.000",
+	/* max */	"254.000",
+	/* default */	"180.000",
+	/* units */	"seconds",
+	/* flags */	WIZARD,
 	/* s-text */
 	"How long the critbit hasher keeps deleted objheads on the cooloff "
-	"list.\n",
+	"list.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #if 0
 PARAM(
 	/* name */	debug,
-	/* tweak */	tweak_mask,
-	/* var */	debug,
-	/* min */	none,
-	/* max */	none,
-	/* default */	none,
-	/* units */	,
+	/* typ */	debug,
+	/* min */	NULL,
+	/* max */	NULL,
+	/* default */	NULL,
+	/* units */	NULL,
 	/* flags */	0,
 	/* s-text */
 	"Enable/Disable various kinds of debugging.\n"
@@ -349,69 +342,65 @@ PARAM(
 	"	lurker	VSL Ban lurker\n"
 	"	esi_chop	Chop ESI fetch to bits\n"
 	"	flush_head	Flush after http1 head\n"
-	"	vtc_mode	Varnishtest Mode\n",
+	"	vtc_mode	Varnishtest Mode",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #endif
 PARAM(
 	/* name */	default_grace,
-	/* tweak */	tweak_timeout,
-	/* var */	default_grace,
-	/* min */	0.000,
-	/* max */	none,
-	/* default */	10.000,
-	/* units */	seconds,
-	/* flags */	0,
+	/* typ */	timeout,
+	/* min */	"0.000",
+	/* max */	NULL,
+	/* default */	"10.000",
+	/* units */	"seconds",
+	/* flags */	OBJ_STICKY,
 	/* s-text */
 	"Default grace period.  We will deliver an object this long after "
 	"it has expired, provided another thread is attempting to get a "
-	"new copy.\n",
+	"new copy.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	default_keep,
-	/* tweak */	tweak_timeout,
-	/* var */	default_keep,
-	/* min */	0.000,
-	/* max */	none,
-	/* default */	0.000,
-	/* units */	seconds,
-	/* flags */	0,
+	/* typ */	timeout,
+	/* min */	"0.000",
+	/* max */	NULL,
+	/* default */	"0.000",
+	/* units */	"seconds",
+	/* flags */	OBJ_STICKY,
 	/* s-text */
 	"Default keep period.  We will keep a useless object around this "
 	"long, making it available for conditional backend fetches.  That "
 	"means that the object will be removed from the cache at the end "
-	"of ttl+grace+keep.\n",
+	"of ttl+grace+keep.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	default_ttl,
-	/* tweak */	tweak_timeout,
-	/* var */	default_ttl,
-	/* min */	0.000,
-	/* max */	none,
-	/* default */	120.000,
-	/* units */	seconds,
-	/* flags */	0,
+	/* typ */	timeout,
+	/* min */	"0.000",
+	/* max */	NULL,
+	/* default */	"120.000",
+	/* units */	"seconds",
+	/* flags */	OBJ_STICKY,
 	/* s-text */
 	"The TTL assigned to objects if neither the backend nor the VCL "
-	"code assigns one.\n",
+	"code assigns one.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #if 0
 PARAM(
 	/* name */	feature,
-	/* tweak */	tweak_mask
-	/* var */	feature,
-	/* min */	none,
-	/* max */	none,
-	/* default */	none,
-	/* units */	,
-	/* flags */	00,
+	/* typ */	feature,
+	/* min */	NULL,
+	/* max */	NULL,
+	/* default */	NULL,
+	/* units */	NULL,
+	/* flags */	0,
 	/* s-text */
 	"Enable/Disable various minor features.\n"
 	"	none	Disable all features.\n"
@@ -423,99 +412,93 @@ PARAM(
 	"	esi_ignore_https	Treat HTTPS as HTTP in ESI:includes\n"
 	"	esi_disable_xml_check	Don't check of body looks like XML\n"
 	"	esi_ignore_other_elements	Ignore non-esi XML-elements\n"
-	"	esi_remove_bom	Remove UTF-8 BOM\n",
+	"	esi_remove_bom	Remove UTF-8 BOM",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #endif
 PARAM(
 	/* name */	fetch_chunksize,
-	/* tweak */	tweak_bytes,
-	/* var */	fetch_chunksize,
-	/* min */	4k,
-	/* max */	none,
-	/* default */	16k,
-	/* units */	bytes,
-	/* flags */	0| EXPERIMENTAL,
+	/* typ */	bytes,
+	/* min */	"4k",
+	/* max */	NULL,
+	/* default */	"16k",
+	/* units */	"bytes",
+	/* flags */	EXPERIMENTAL,
 	/* s-text */
 	"The default chunksize used by fetcher. This should be bigger than "
 	"the majority of objects with short TTLs.\n"
 	"Internal limits in the storage_file module makes increases above "
-	"128kb a dubious idea.\n",
+	"128kb a dubious idea.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	fetch_maxchunksize,
-	/* tweak */	tweak_bytes,
-	/* var */	fetch_maxchunksize,
-	/* min */	64k,
-	/* max */	none,
-	/* default */	0.25G,
-	/* units */	bytes,
-	/* flags */	0| EXPERIMENTAL,
+	/* typ */	bytes,
+	/* min */	"64k",
+	/* max */	NULL,
+	/* default */	"0.25G",
+	/* units */	"bytes",
+	/* flags */	EXPERIMENTAL,
 	/* s-text */
 	"The maximum chunksize we attempt to allocate from storage. Making "
-	"this too large may cause delays and storage fragmentation.\n",
+	"this too large may cause delays and storage fragmentation.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	gzip_buffer,
-	/* tweak */	tweak_bytes_u,
-	/* var */	gzip_buffer,
-	/* min */	2k,
-	/* max */	none,
-	/* default */	32k,
-	/* units */	bytes,
-	/* flags */	0| EXPERIMENTAL,
+	/* typ */	bytes_u,
+	/* min */	"2k",
+	/* max */	NULL,
+	/* default */	"32k",
+	/* units */	"bytes",
+	/* flags */	EXPERIMENTAL,
 	/* s-text */
 	"Size of malloc buffer used for gzip processing.\n"
 	"These buffers are used for in-transit data, for instance "
 	"gunzip'ed data being sent to a client.Making this space to small "
 	"results in more overhead, writes to sockets etc, making it too "
-	"big is probably just a waste of memory.\n",
+	"big is probably just a waste of memory.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	gzip_level,
-	/* tweak */	tweak_uint,
-	/* var */	gzip_level,
-	/* min */	0,
-	/* max */	9,
-	/* default */	6,
-	/* units */	,
-	/* flags */	00,
+	/* typ */	uint,
+	/* min */	"0",
+	/* max */	"9",
+	/* default */	"6",
+	/* units */	NULL,
+	/* flags */	0,
 	/* s-text */
-	"Gzip compression level: 0=debug, 1=fast, 9=best\n",
+	"Gzip compression level: 0=debug, 1=fast, 9=best",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	gzip_memlevel,
-	/* tweak */	tweak_uint,
-	/* var */	gzip_memlevel,
-	/* min */	1,
-	/* max */	9,
-	/* default */	8,
-	/* units */	,
-	/* flags */	00,
+	/* typ */	uint,
+	/* min */	"1",
+	/* max */	"9",
+	/* default */	"8",
+	/* units */	NULL,
+	/* flags */	0,
 	/* s-text */
 	"Gzip memory level 1=slow/least, 9=fast/most compression.\n"
-	"Memory impact is 1=1k, 2=2k, ... 9=256k.\n",
+	"Memory impact is 1=1k, 2=2k, ... 9=256k.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	http_gzip_support,
-	/* tweak */	tweak_bool,
-	/* var */	http_gzip_support,
-	/* min */	none,
-	/* max */	none,
-	/* default */	on,
-	/* units */	bool,
-	/* flags */	00,
+	/* typ */	bool,
+	/* min */	NULL,
+	/* max */	NULL,
+	/* default */	"on",
+	/* units */	"bool",
+	/* flags */	0,
 	/* s-text */
 	"Enable gzip support. When enabled Varnish request compressed "
 	"objects from the backend and store them compressed. If a client "
@@ -526,519 +509,478 @@ PARAM(
 	"\n"
 	"Clients that do not support gzip will have their Accept-Encoding "
 	"header removed. For more information on how gzip is implemented "
-	"please see the chapter on gzip in the Varnish reference.\n",
+	"please see the chapter on gzip in the Varnish reference.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	http_max_hdr,
-	/* tweak */	tweak_uint,
-	/* var */	http_max_hdr,
-	/* min */	32,
-	/* max */	65535,
-	/* default */	64,
-	/* units */	header lines,
-	/* flags */	00,
+	/* typ */	uint,
+	/* min */	"32",
+	/* max */	"65535",
+	/* default */	"64",
+	/* units */	"header lines",
+	/* flags */	0,
 	/* s-text */
 	"Maximum number of HTTP header lines we allow in "
 	"{req|resp|bereq|beresp}.http (obj.http is autosized to the exact "
 	"number of headers).\n"
 	"Cheap, ~20 bytes, in terms of workspace memory.\n"
-	"Note that the first line occupies five header lines.\n",
+	"Note that the first line occupies five header lines.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	http_range_support,
-	/* tweak */	tweak_bool,
-	/* var */	http_range_support,
-	/* min */	none,
-	/* max */	none,
-	/* default */	on,
-	/* units */	bool,
-	/* flags */	00,
+	/* typ */	bool,
+	/* min */	NULL,
+	/* max */	NULL,
+	/* default */	"on",
+	/* units */	"bool",
+	/* flags */	0,
 	/* s-text */
-	"Enable support for HTTP Range headers.\n",
+	"Enable support for HTTP Range headers.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	http_req_hdr_len,
-	/* tweak */	tweak_bytes_u,
-	/* var */	http_req_hdr_len,
-	/* min */	40b,
-	/* max */	none,
-	/* default */	8k,
-	/* units */	bytes,
-	/* flags */	00,
+	/* typ */	bytes_u,
+	/* min */	"40b",
+	/* max */	NULL,
+	/* default */	"8k",
+	/* units */	"bytes",
+	/* flags */	0,
 	/* s-text */
 	"Maximum length of any HTTP client request header we will allow.  "
-	"The limit is inclusive its continuation lines.\n",
+	"The limit is inclusive its continuation lines.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	http_req_size,
-	/* tweak */	tweak_bytes_u,
-	/* var */	http_req_size,
-	/* min */	0.25k,
-	/* max */	none,
-	/* default */	32k,
-	/* units */	bytes,
-	/* flags */	00,
+	/* typ */	bytes_u,
+	/* min */	"0.25k",
+	/* max */	NULL,
+	/* default */	"32k",
+	/* units */	"bytes",
+	/* flags */	0,
 	/* s-text */
 	"Maximum number of bytes of HTTP client request we will deal with. "
 	" This is a limit on all bytes up to the double blank line which "
 	"ends the HTTP request.\n"
 	"The memory for the request is allocated from the client workspace "
 	"(param: workspace_client) and this parameter limits how much of "
-	"that the request is allowed to take up.\n",
+	"that the request is allowed to take up.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	http_resp_hdr_len,
-	/* tweak */	tweak_bytes_u,
-	/* var */	http_resp_hdr_len,
-	/* min */	40b,
-	/* max */	none,
-	/* default */	8k,
-	/* units */	bytes,
-	/* flags */	00,
+	/* typ */	bytes_u,
+	/* min */	"40b",
+	/* max */	NULL,
+	/* default */	"8k",
+	/* units */	"bytes",
+	/* flags */	0,
 	/* s-text */
 	"Maximum length of any HTTP backend response header we will allow. "
-	" The limit is inclusive its continuation lines.\n",
+	" The limit is inclusive its continuation lines.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	http_resp_size,
-	/* tweak */	tweak_bytes_u,
-	/* var */	http_resp_size,
-	/* min */	0.25k,
-	/* max */	none,
-	/* default */	32k,
-	/* units */	bytes,
-	/* flags */	00,
+	/* typ */	bytes_u,
+	/* min */	"0.25k",
+	/* max */	NULL,
+	/* default */	"32k",
+	/* units */	"bytes",
+	/* flags */	0,
 	/* s-text */
 	"Maximum number of bytes of HTTP backend response we will deal "
 	"with.  This is a limit on all bytes up to the double blank line "
 	"which ends the HTTP request.\n"
 	"The memory for the request is allocated from the worker workspace "
 	"(param: thread_pool_workspace) and this parameter limits how much "
-	"of that the request is allowed to take up.\n",
+	"of that the request is allowed to take up.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	idle_send_timeout,
-	/* tweak */	tweak_timeout,
-	/* var */	idle_send_timeout,
-	/* min */	0.000,
-	/* max */	none,
-	/* default */	60.000,
-	/* units */	seconds,
-	/* flags */	0| DELAYED_EFFECT,
+	/* typ */	timeout,
+	/* min */	"0.000",
+	/* max */	NULL,
+	/* default */	"60.000",
+	/* units */	"seconds",
+	/* flags */	DELAYED_EFFECT,
 	/* s-text */
 	"Time to wait with no data sent. If no data has been transmitted "
 	"in this many\n"
 	"seconds the session is closed.\n"
-	"See setsockopt(2) under SO_SNDTIMEO for more information.\n",
+	"See setsockopt(2) under SO_SNDTIMEO for more information.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	listen_depth,
-	/* tweak */	tweak_uint,
-	/* var */	listen_depth,
-	/* min */	0,
-	/* max */	none,
-	/* default */	1024,
-	/* units */	connections,
-	/* flags */	0| MUST_RESTART,
+	/* typ */	uint,
+	/* min */	"0",
+	/* max */	NULL,
+	/* default */	"1024",
+	/* units */	"connections",
+	/* flags */	MUST_RESTART,
 	/* s-text */
-	"Listen queue depth.\n",
+	"Listen queue depth.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	lru_interval,
-	/* tweak */	tweak_timeout,
-	/* var */	lru_interval,
-	/* min */	0.000,
-	/* max */	none,
-	/* default */	2.000,
-	/* units */	seconds,
-	/* flags */	0| EXPERIMENTAL,
+	/* typ */	timeout,
+	/* min */	"0.000",
+	/* max */	NULL,
+	/* default */	"2.000",
+	/* units */	"seconds",
+	/* flags */	EXPERIMENTAL,
 	/* s-text */
 	"Grace period before object moves on LRU list.\n"
 	"Objects are only moved to the front of the LRU list if they have "
 	"not been moved there already inside this timeout period.  This "
 	"reduces the amount of lock operations necessary for LRU list "
-	"access.\n",
+	"access.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	max_esi_depth,
-	/* tweak */	tweak_uint,
-	/* var */	max_esi_depth,
-	/* min */	0,
-	/* max */	none,
-	/* default */	5,
-	/* units */	levels,
-	/* flags */	00,
+	/* typ */	uint,
+	/* min */	"0",
+	/* max */	NULL,
+	/* default */	"5",
+	/* units */	"levels",
+	/* flags */	0,
 	/* s-text */
-	"Maximum depth of esi:include processing.\n",
+	"Maximum depth of esi:include processing.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	max_restarts,
-	/* tweak */	tweak_uint,
-	/* var */	max_restarts,
-	/* min */	0,
-	/* max */	none,
-	/* default */	4,
-	/* units */	restarts,
-	/* flags */	00,
+	/* typ */	uint,
+	/* min */	"0",
+	/* max */	NULL,
+	/* default */	"4",
+	/* units */	"restarts",
+	/* flags */	0,
 	/* s-text */
 	"Upper limit on how many times a request can restart.\n"
 	"Be aware that restarts are likely to cause a hit against the "
-	"backend, so don't increase thoughtlessly.\n",
+	"backend, so don't increase thoughtlessly.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	max_retries,
-	/* tweak */	tweak_uint,
-	/* var */	max_retries,
-	/* min */	0,
-	/* max */	none,
-	/* default */	4,
-	/* units */	retries,
-	/* flags */	00,
+	/* typ */	uint,
+	/* min */	"0",
+	/* max */	NULL,
+	/* default */	"4",
+	/* units */	"retries",
+	/* flags */	0,
 	/* s-text */
-	"Upper limit on how many times a backend fetch can retry.\n",
+	"Upper limit on how many times a backend fetch can retry.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	nuke_limit,
-	/* tweak */	tweak_uint,
-	/* var */	nuke_limit,
-	/* min */	0,
-	/* max */	none,
-	/* default */	50,
-	/* units */	allocations,
-	/* flags */	0| EXPERIMENTAL,
+	/* typ */	uint,
+	/* min */	"0",
+	/* max */	NULL,
+	/* default */	"50",
+	/* units */	"allocations",
+	/* flags */	EXPERIMENTAL,
 	/* s-text */
 	"Maximum number of objects we attempt to nuke in order to make "
-	"space for a object body.\n",
+	"space for a object body.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #if 0
 PARAM(
 	/* name */	pcre_match_limit,
-	/* tweak */	tweak_uint,
-	/* var */	pcre_match_limit,
-	/* min */	1,
-	/* max */	none,
-	/* default */	10000,
-	/* units */	,
-	/* flags */	00,
+	/* typ */	uint,
+	/* min */	"1",
+	/* max */	NULL,
+	/* default */	"1.000",
+	/* units */	NULL,
+	/* flags */	0,
 	/* s-text */
 	"The limit for the  number of internal matching function calls in "
-	"a pcre_exec() execution.\n",
+	"a pcre_exec() execution.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	pcre_match_limit_recursion,
-	/* tweak */	tweak_uint,
-	/* var */	pcre_match_limit_recursion,
-	/* min */	1,
-	/* max */	none,
-	/* default */	10000,
-	/* units */	,
-	/* flags */	00,
+	/* typ */	uint,
+	/* min */	"1",
+	/* max */	NULL,
+	/* default */	"1.000",
+	/* units */	NULL,
+	/* flags */	0,
 	/* s-text */
 	"The limit for the  number of internal matching function "
-	"recursions in a pcre_exec() execution.\n",
+	"recursions in a pcre_exec() execution.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #endif
 PARAM(
 	/* name */	ping_interval,
-	/* tweak */	tweak_uint,
-	/* var */	ping_interval,
-	/* min */	0,
-	/* max */	none,
-	/* default */	3,
-	/* units */	seconds,
-	/* flags */	0| MUST_RESTART,
+	/* typ */	uint,
+	/* min */	"0",
+	/* max */	NULL,
+	/* default */	"3",
+	/* units */	"seconds",
+	/* flags */	MUST_RESTART,
 	/* s-text */
 	"Interval between pings from parent to child.\n"
 	"Zero will disable pinging entirely, which makes it possible to "
-	"attach a debugger to the child.\n",
+	"attach a debugger to the child.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	pipe_timeout,
-	/* tweak */	tweak_timeout,
-	/* var */	pipe_timeout,
-	/* min */	0.000,
-	/* max */	none,
-	/* default */	60.000,
-	/* units */	seconds,
-	/* flags */	00,
+	/* typ */	timeout,
+	/* min */	"0.000",
+	/* max */	NULL,
+	/* default */	"60.000",
+	/* units */	"seconds",
+	/* flags */	0,
 	/* s-text */
 	"Idle timeout for PIPE sessions. If nothing have been received in "
-	"either direction for this many seconds, the session is closed.\n",
+	"either direction for this many seconds, the session is closed.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #if 0
 PARAM(
 	/* name */	pool_req,
-	/* tweak */	tweak_poolparam,
-	/* var */	pool_req,
-	/* min */	none,
-	/* max */	none,
-	/* default */	10\,100\,10,
-	/* units */	none,
-	/* flags */	00,
+	/* typ */	poolparam,
+	/* min */	NULL,
+	/* max */	NULL,
+	/* default */	"10,100,10",
+	/* units */	NULL,
+	/* flags */	0,
 	/* s-text */
 	"Parameters for per worker pool request memory pool.\n"
-	"The three numbers are:\n"
-	"	min_pool	minimum size of free pool.\n"
-	"	max_pool	maximum size of free pool.\n"
-	"	max_age		max age of free element.\n",
+	MEMPOOL_TEXT,
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	pool_sess,
-	/* tweak */	tweak_poolparam,
-	/* var */	pool_sess,
-	/* min */	none,
-	/* max */	none,
-	/* default */	10,100,10,
-	/* units */	,
-	/* flags */	00,
+	/* typ */	poolparam,
+	/* min */	NULL,
+	/* max */	NULL,
+	/* default */	"10,100,10",
+	/* units */	NULL,
+	/* flags */	0,
 	/* s-text */
 	"Parameters for per worker pool session memory pool.\n"
-	"The three numbers are:\n"
-	"	min_pool	minimum size of free pool.\n"
-	"	max_pool	maximum size of free pool.\n"
-	"	max_age	max age of free element.\n",
+	MEMPOOL_TEXT,
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	pool_vbo,
-	/* tweak */	tweak_poolparam,
-	/* var */	pool_vbo,
-	/* min */	none,
-	/* max */	none,
-	/* default */	10,100,10,
-	/* units */	,
-	/* flags */	00,
+	/* typ */	poolparam,
+	/* min */	NULL,
+	/* max */	NULL,
+	/* default */	"10,100,10",
+	/* units */	NULL,
+	/* flags */	0,
 	/* s-text */
 	"Parameters for backend object fetch memory pool.\n"
-	"The three numbers are:\n"
-	"	min_pool	minimum size of free pool.\n"
-	"	max_pool	maximum size of free pool.\n"
-	"	max_age	max age of free element.\n",
+	MEMPOOL_TEXT,
 	/* l-text */	"",
 	/* func */	NULL
 )
 #endif
 PARAM(
 	/* name */	prefer_ipv6,
-	/* tweak */	tweak_bool,
-	/* var */	prefer_ipv6,
-	/* min */	none,
-	/* max */	none,
-	/* default */	off,
-	/* units */	bool,
-	/* flags */	00,
+	/* typ */	bool,
+	/* min */	NULL,
+	/* max */	NULL,
+	/* default */	"off",
+	/* units */	"bool",
+	/* flags */	0,
 	/* s-text */
 	"Prefer IPv6 address when connecting to backends which have both "
-	"IPv4 and IPv6 addresses.\n",
+	"IPv4 and IPv6 addresses.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	rush_exponent,
-	/* tweak */	tweak_uint,
-	/* var */	rush_exponent,
-	/* min */	2,
-	/* max */	none,
-	/* default */	3,
-	/* units */	requests per request,
-	/* flags */	0| EXPERIMENTAL,
+	/* typ */	uint,
+	/* min */	"2",
+	/* max */	NULL,
+	/* default */	"3",
+	/* units */	"requests per request",
+	/* flags */	EXPERIMENTAL,
 	/* s-text */
 	"How many parked request we start for each completed request on "
 	"the object.\n"
 	"NB: Even with the implict delay of delivery, this parameter "
-	"controls an exponential increase in number of worker threads.\n",
+	"controls an exponential increase in number of worker threads.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	send_timeout,
-	/* tweak */	tweak_timeout,
-	/* var */	send_timeout,
-	/* min */	0.000,
-	/* max */	none,
-	/* default */	600.000,
-	/* units */	seconds,
-	/* flags */	0| DELAYED_EFFECT,
+	/* typ */	timeout,
+	/* min */	"0.000",
+	/* max */	NULL,
+	/* default */	"600.000",
+	/* units */	"seconds",
+	/* flags */	DELAYED_EFFECT,
 	/* s-text */
 	"Send timeout for client connections. If the HTTP response hasn't "
 	"been transmitted in this many\n"
 	"seconds the session is closed.\n"
-	"See setsockopt(2) under SO_SNDTIMEO for more information.\n",
+	"See setsockopt(2) under SO_SNDTIMEO for more information.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #if 0
 PARAM(
 	/* name */	session_max,
-	/* tweak */	tweak_uint,
-	/* var */	session_max,
-	/* min */	1000,
-	/* max */	none,
-	/* default */	100000,
-	/* units */	sessions,
-	/* flags */	00,
+	/* typ */	uint,
+	/* min */	"1000",
+	/* max */	NULL,
+	/* default */	"10.000",
+	/* units */	"sessions",
+	/* flags */	0,
 	/* s-text */
 	"Maximum number of sessions we will allocate from one pool before "
 	"just dropping connections.\n"
 	"This is mostly an anti-DoS measure, and setting it plenty high "
-	"should not hurt, as long as you have the memory for it.\n",
+	"should not hurt, as long as you have the memory for it.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 
 PARAM(
 	/* name */	shm_reclen,
-	/* tweak */	tweak_vsl_reclen,
-	/* var */	shm_reclen,
-	/* min */	16b,
-	/* max */	4084,
-	/* default */	255b,
-	/* units */	bytes,
-	/* flags */	00,
+	/* typ */	vsl_reclen,
+	/* min */	"16b",
+	/* max */	"4084",
+	/* default */	"255b",
+	/* units */	"bytes",
+	/* flags */	0,
 	/* s-text */
-	"Old name for vsl_reclen, use that instead.\n",
+	"Old name for vsl_reclen, use that instead.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #endif
 PARAM(
 	/* name */	shortlived,
-	/* tweak */	tweak_timeout,
-	/* var */	shortlived,
-	/* min */	0.000,
-	/* max */	none,
-	/* default */	10.000,
-	/* units */	seconds,
-	/* flags */	00,
+	/* typ */	timeout,
+	/* min */	"0.000",
+	/* max */	NULL,
+	/* default */	"10.000",
+	/* units */	"seconds",
+	/* flags */	0,
 	/* s-text */
 	"Objects created with (ttl+grace+keep) shorter than this are "
-	"always put in transient storage.\n",
+	"always put in transient storage.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	sigsegv_handler,
-	/* tweak */	tweak_bool,
-	/* var */	sigsegv_handler,
-	/* min */	none,
-	/* max */	none,
-	/* default */	on,
-	/* units */	bool,
-	/* flags */	0| MUST_RESTART,
+	/* typ */	bool,
+	/* min */	NULL,
+	/* max */	NULL,
+	/* default */	"on",
+	/* units */	"bool",
+	/* flags */	MUST_RESTART,
 	/* s-text */
 	"Install a signal handler which tries to dump debug information on "
-	"segmentation faults, bus errors and abort signals.\n",
+	"segmentation faults, bus errors and abort signals.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	syslog_cli_traffic,
-	/* tweak */	tweak_bool,
-	/* var */	syslog_cli_traffic,
-	/* min */	none,
-	/* max */	none,
-	/* default */	on,
-	/* units */	bool,
-	/* flags */	00,
+	/* typ */	bool,
+	/* min */	NULL,
+	/* max */	NULL,
+	/* default */	"on",
+	/* units */	"bool",
+	/* flags */	0,
 	/* s-text */
-	"Log all CLI traffic to syslog(LOG_INFO).\n",
+	"Log all CLI traffic to syslog(LOG_INFO).",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	tcp_keepalive_intvl,
-	/* tweak */	tweak_timeout,
-	/* var */	tcp_keepalive_intvl,
-	/* min */	1.000,
-	/* max */	100.000,
-	/* default */	5.000,
-	/* units */	seconds,
-	/* flags */	0| EXPERIMENTAL,
+	/* typ */	timeout,
+	/* min */	"1.000",
+	/* max */	"100.000",
+	/* default */	"5.000",
+	/* units */	"seconds",
+	/* flags */	EXPERIMENTAL,
 	/* s-text */
-	"The number of seconds between TCP keep-alive probes.\n",
+	"The number of seconds between TCP keep-alive probes.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	tcp_keepalive_probes,
-	/* tweak */	tweak_uint,
-	/* var */	tcp_keepalive_probes,
-	/* min */	1,
-	/* max */	100,
-	/* default */	5,
-	/* units */	probes,
-	/* flags */	0| EXPERIMENTAL,
+	/* typ */	uint,
+	/* min */	"1",
+	/* max */	"100",
+	/* default */	"5",
+	/* units */	"probes",
+	/* flags */	EXPERIMENTAL,
 	/* s-text */
 	"The maximum number of TCP keep-alive probes to send before giving "
 	"up and killing the connection if no response is obtained from the "
-	"other end.\n",
+	"other end.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	tcp_keepalive_time,
-	/* tweak */	tweak_timeout,
-	/* var */	tcp_keepalive_time,
-	/* min */	1.000,
-	/* max */	7200.000,
-	/* default */	600.000,
-	/* units */	seconds,
-	/* flags */	0| EXPERIMENTAL,
+	/* typ */	timeout,
+	/* min */	"1.000",
+	/* max */	"7200.000",
+	/* default */	"600.000",
+	/* units */	"seconds",
+	/* flags */	EXPERIMENTAL,
 	/* s-text */
 	"The number of seconds a connection needs to be idle before TCP "
-	"begins sending out keep-alive probes.\n",
+	"begins sending out keep-alive probes.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #if 0
 PARAM(
 	/* name */	thread_pool_add_delay,
-	/* tweak */	tweak_timeout,
-	/* var */	thread_pool_add_delay,
-	/* min */	0.000,
-	/* max */	none,
-	/* default */	0.000,
-	/* units */	seconds,
-	/* flags */	0| EXPERIMENTAL,
+	/* typ */	timeout,
+	/* min */	"0.000",
+	/* max */	NULL,
+	/* default */	"0.000",
+	/* units */	"seconds",
+	/* flags */	EXPERIMENTAL,
 	/* s-text */
 	"Wait at least this long after creating a thread.\n"
 	"\n"
@@ -1046,34 +988,32 @@ PARAM(
 	"creating threads.\n"
 	"Set this to a few milliseconds if you see the 'threads_failed' "
 	"counter grow too much.\n"
-	"Setting this too high results in insuffient worker threads.\n",
+	"Setting this too high results in insuffient worker threads.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	thread_pool_destroy_delay,
-	/* tweak */	tweak_timeout,
-	/* var */	thread_pool_destroy_delay,
-	/* min */	0.010,
-	/* max */	none,
-	/* default */	1.000,
-	/* units */	seconds,
-	/* flags */	0| DELAYED_EFFECT| EXPERIMENTAL,
+	/* typ */	timeout,
+	/* min */	"0.010",
+	/* max */	NULL,
+	/* default */	"1.000",
+	/* units */	"seconds",
+	/* flags */	DELAYED_EFFECT| EXPERIMENTAL,
 	/* s-text */
 	"Wait this long after destroying a thread.\n"
-	"This controls the decay of thread pools when idle(-ish).\n",
+	"This controls the decay of thread pools when idle(-ish).",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	thread_pool_fail_delay,
-	/* tweak */	tweak_timeout,
-	/* var */	thread_pool_fail_delay,
-	/* min */	0.010,
-	/* max */	none,
-	/* default */	0.200,
-	/* units */	seconds,
-	/* flags */	0| EXPERIMENTAL,
+	/* typ */	timeout,
+	/* min */	"0.010",
+	/* max */	NULL,
+	/* default */	"0.200",
+	/* units */	"seconds",
+	/* flags */	EXPERIMENTAL,
 	/* s-text */
 	"Wait at least this long after a failed thread creation before "
 	"trying to create another thread.\n"
@@ -1087,88 +1027,83 @@ PARAM(
 	"\n"
 	"It may also help to increase thread_pool_timeout and "
 	"thread_pool_min, to reduce the rate at which treads are destroyed "
-	"and later recreated.\n",
+	"and later recreated.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	thread_pool_max,
-	/* tweak */	tweak_***,
-	/* var */	thread_pool_max,
-	/* min */	100,
-	/* max */	none,
-	/* default */	5000,
-	/* units */	threads,
-	/* flags */	0| DELAYED_EFFECT,
+	/* typ */	thread_pool_max,
+	/* min */	"100",
+	/* max */	NULL,
+	/* default */	"5000",
+	/* units */	"threads",
+	/* flags */	DELAYED_EFFECT,
 	/* s-text */
 	"The maximum number of worker threads in each pool.\n"
 	"\n"
 	"Do not set this higher than you have to, since excess worker "
 	"threads soak up RAM and CPU and generally just get in the way of "
-	"getting work done.\n",
+	"getting work done.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	thread_pool_min,
-	/* tweak */	tweak_***,
-	/* var */	thread_pool_min,
-	/* min */	none,
-	/* max */	5000,
-	/* default */	100,
-	/* units */	threads,
-	/* flags */	0| DELAYED_EFFECT,
+	/* typ */	thread_pool_min,
+	/* min */	NULL,
+	/* max */	"5000",
+	/* default */	"100",
+	/* units */	"threads",
+	/* flags */	DELAYED_EFFECT,
 	/* s-text */
 	"The minimum number of worker threads in each pool.\n"
 	"\n"
 	"Increasing this may help ramp up faster from low load situations "
 	"or when threads have expired."
-	"Minimum is 10 threads.\n",
+	"Minimum is 10 threads.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	thread_pool_stack,
-	/* tweak */	tweak_bytes,
-	/* var */	thread_pool_stack,
-	/* min */	2k,
-	/* max */	none,
-	/* default */	48k,
-	/* units */	bytes,
-	/* flags */	0| EXPERIMENTAL,
+	/* typ */	bytes,
+	/* min */	"2k",
+	/* max */	NULL,
+	/* default */	"48k",
+	/* units */	"bytes",
+	/* flags */	EXPERIMENTAL,
 	/* s-text */
 	"Worker thread stack size.\n"
 	"This will likely be rounded up to a multiple of 4k (or whatever "
-	"the page_size might be) by the kernel.\n",
+	"the page_size might be) by the kernel.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	thread_pool_timeout,
-	/* tweak */	tweak_timeout,
-	/* var */	thread_pool_timeout,
-	/* min */	10.000,
-	/* max */	none,
-	/* default */	300.000,
-	/* units */	seconds,
-	/* flags */	0| DELAYED_EFFECT| EXPERIMENTAL,
+	/* typ */	timeout,
+	/* min */	"10.000",
+	/* max */	NULL,
+	/* default */	"300.000",
+	/* units */	"seconds",
+	/* flags */	DELAYED_EFFECT| EXPERIMENTAL,
 	/* s-text */
 	"Thread idle threshold.\n"
 	"\n"
 	"Threads in excess of thread_pool_min, which have been idle for at "
-	"least this long, will be destroyed.\n",
+	"least this long, will be destroyed.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	thread_pools,
-	/* tweak */	tweak_uint,
-	/* var */	thread_pools,
-	/* min */	1,
-	/* max */	none,
-	/* default */	2,
-	/* units */	pools,
-	/* flags */	0| DELAYED_EFFECT| EXPERIMENTAL,
+	/* typ */	uint,
+	/* min */	"1",
+	/* max */	NULL,
+	/* default */	"2",
+	/* units */	"pools",
+	/* flags */	DELAYED_EFFECT| EXPERIMENTAL,
 	/* s-text */
 	"Number of worker thread pools.\n"
 	"\n"
@@ -1178,73 +1113,69 @@ PARAM(
 	"pool for each CPU is probably detrimal to performance.\n"
 	"\n"
 	"Can be increased on the fly, but decreases require a restart to "
-	"take effect.\n",
+	"take effect.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	thread_queue_limit,
-	/* tweak */	tweak_uint,
-	/* var */	thread_queue_limit,
-	/* min */	0,
-	/* max */	none,
-	/* default */	20,
-	/* units */	,
-	/* flags */	0| EXPERIMENTAL,
+	/* typ */	uint,
+	/* min */	"0",
+	/* max */	NULL,
+	/* default */	"20",
+	/* units */	NULL,
+	/* flags */	EXPERIMENTAL,
 	/* s-text */
 	"Permitted queue length per thread-pool.\n"
 	"\n"
 	"This sets the number of requests we will queue, waiting for an "
 	"available thread.  Above this limit sessions will be dropped "
-	"instead of queued.\n",
+	"instead of queued.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	thread_stats_rate,
-	/* tweak */	tweak_uint,
-	/* var */	thread_stats_rate,
-	/* min */	0,
-	/* max */	none,
-	/* default */	10,
-	/* units */	requests,
-	/* flags */	0| EXPERIMENTAL,
+	/* typ */	uint,
+	/* min */	"0",
+	/* max */	NULL,
+	/* default */	"10",
+	/* units */	"requests",
+	/* flags */	EXPERIMENTAL,
 	/* s-text */
 	"Worker threads accumulate statistics, and dump these into the "
 	"global stats counters if the lock is free when they finish a job "
 	"(request/fetch etc.)\n"
 	"This parameters defines the maximum number of jobs a worker "
 	"thread may handle, before it is forced to dump its accumulated "
-	"stats into the global counters.\n",
+	"stats into the global counters.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #endif
 PARAM(
 	/* name */	timeout_idle,
-	/* tweak */	tweak_timeout,
-	/* var */	timeout_idle,
-	/* min */	0.000,
-	/* max */	none,
-	/* default */	5.000,
-	/* units */	seconds,
-	/* flags */	00,
+	/* typ */	timeout,
+	/* min */	"0.000",
+	/* max */	NULL,
+	/* default */	"5.000",
+	/* units */	"seconds",
+	/* flags */	0,
 	/* s-text */
 	"Idle timeout for client connections.\n"
 	"A connection is considered idle, until we have received the full "
-	"request headers.\n",
+	"request headers.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	timeout_linger,
-	/* tweak */	tweak_timeout,
-	/* var */	timeout_linger,
-	/* min */	0.000,
-	/* max */	none,
-	/* default */	0.050,
-	/* units */	seconds,
-	/* flags */	0| EXPERIMENTAL,
+	/* typ */	timeout,
+	/* min */	"0.000",
+	/* max */	NULL,
+	/* default */	"0.050",
+	/* units */	"seconds",
+	/* flags */	EXPERIMENTAL,
 	/* s-text */
 	"How long the worker thread lingers on an idle session before "
 	"handing it over to the waiter.\n"
@@ -1252,22 +1183,21 @@ PARAM(
 	"within the first 100 msec of the previous request completing.\n"
 	"Setting this too high results in worker threads not doing "
 	"anything for their keep, setting it too low just means that more "
-	"sessions take a detour around the waiter.\n",
+	"sessions take a detour around the waiter.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #if 0
 PARAM(
 	/* name */	vcc_allow_inline_c,
-	/* tweak */	tweak_bool,
-	/* var */	vcc_allow_inline_c,
-	/* min */	none,
-	/* max */	none,
-	/* default */	off,
-	/* units */	bool,
-	/* flags */	00,
+	/* typ */	bool,
+	/* min */	NULL,
+	/* max */	NULL,
+	/* default */	"off",
+	/* units */	"bool",
+	/* flags */	0,
 	/* s-text */
-	"Allow inline C code in VCL.\n",
+	"Allow inline C code in VCL.",
 	/* l-text */	"",
 	/* func */	NULL
 )
@@ -1275,15 +1205,14 @@ PARAM(
 #if 0
 PARAM(
 	/* name */	vcc_err_unref,
-	/* tweak */	tweak_bool,
-	/* var */	vcc_err_unref,
-	/* min */	none,
-	/* max */	none,
-	/* default */	on,
-	/* units */	bool,
-	/* flags */	00,
+	/* typ */	bool,
+	/* min */	NULL,
+	/* max */	NULL,
+	/* default */	"on",
+	/* units */	"bool",
+	/* flags */	0,
 	/* s-text */
-	"Unreferenced VCL objects result in error.\n",
+	"Unreferenced VCL objects result in error.",
 	/* l-text */	"",
 	/* func */	NULL
 )
@@ -1291,231 +1220,217 @@ PARAM(
 #if 0
 PARAM(
 	/* name */	vcc_unsafe_path,
-	/* tweak */	tweak_bool,
-	/* var */	vcc_unsafe_path,
-	/* min */	none,
-	/* max */	none,
-	/* default */	on,
-	/* units */	bool,
-	/* flags */	00,
+	/* typ */	bool,
+	/* min */	NULL,
+	/* max */	NULL,
+	/* default */	"on",
+	/* units */	"bool",
+	/* flags */	0,
 	/* s-text */
 	"Allow '/' in vmod & include paths.\n"
-	"Allow 'import ... from ...'.\n",
+	"Allow 'import ... from ...'.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #endif
 PARAM(
 	/* name */	vcl_cooldown,
-	/* tweak */	tweak_timeout,
-	/* var */	vcl_cooldown,
-	/* min */	0.000,
-	/* max */	none,
-	/* default */	600.000,
-	/* units */	seconds,
-	/* flags */	00,
+	/* typ */	timeout,
+	/* min */	"0.000",
+	/* max */	NULL,
+	/* default */	"600.000",
+	/* units */	"seconds",
+	/* flags */	0,
 	/* s-text */
 	"How long time a VCL is kept warm after being replaced as the "
-	"active VCL.  (Granularity approximately 30 seconds.)\n",
+	"active VCL.  (Granularity approximately 30 seconds.)",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #if 0
 PARAM(
 	/* name */	vcl_dir,
-	/* tweak */	tweak_string,
-	/* var */	vcl_dir,
-	/* min */	none,
-	/* max */	none,
+	/* typ */	string,
+	/* min */	NULL,
+	/* max */	NULL,
 	/* default */	/opt/varnish/etc/varnish,
-	/* units */	(null),
-	/* flags */	00,
+	/* units */	NULL,
+	/* flags */	0,
 	/* s-text */
 	"Directory from which relative VCL filenames (vcl.load and "
-	"include) are opened.\n",
+	"include) are opened.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	vmod_dir,
-	/* tweak */	tweak_string,
-	/* var */	vmod_dir,
-	/* min */	none,
-	/* max */	none,
+	/* typ */	string,
+	/* min */	NULL,
+	/* max */	NULL,
 	/* default */	/opt/varnish/lib/varnish/vmods,
-	/* units */	(null),
-	/* flags */	00,
+	/* units */	NULL,
+	/* flags */	0,
 	/* s-text */
-	"Directory where VCL modules are to be found.\n",
+	"Directory where VCL modules are to be found.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #endif
 PARAM(
 	/* name */	vsl_buffer,
-	/* tweak */	tweak_vsl_buffer,
-	/* var */	vsl_buffer,
-	/* min */	267,
-	/* max */	none,
-	/* default */	4k,
-	/* units */	bytes,
-	/* flags */	00,
+	/* typ */	vsl_buffer,
+	/* min */	"267",
+	/* max */	NULL,
+	/* default */	"4k",
+	/* units */	"bytes",
+	/* flags */	0,
 	/* s-text */
 	"Bytes of (req-/backend-)workspace dedicated to buffering VSL "
 	"records.\n"
 	"Setting this too high costs memory, setting it too low will cause "
 	"more VSL flushes and likely increase lock-contention on the VSL "
 	"mutex.\n"
-	"The minimum tracks the vsl_reclen parameter + 12 bytes.\n",
+	"The minimum tracks the vsl_reclen parameter + 12 bytes.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #if 0
 PARAM(
 	/* name */	vsl_mask,
-	/* tweak */	tweak_mask
-	/* var */	vsl_mask,
-	/* min */	none,
-	/* max */	none,
-	/* default */	-VCL_trace,-WorkThread,-Hash,-VfpAcct,
-	/* units */	,
-	/* flags */	00,
+	/* typ */	vsl_mask,
+	/* min */	NULL,
+	/* max */	NULL,
+	/* default */	"default",
+	/* units */	NULL,
+	/* flags */	0,
 	/* s-text */
 	"Mask individual VSL messages from being logged.\n"
 	"	default	Set default value\n"
 	"\n"
 	"Use +/- prefixe in front of VSL tag name, to mask/unmask "
-	"individual VSL messages.\n",
+	"individual VSL messages.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #endif
 PARAM(
 	/* name */	vsl_reclen,
-	/* tweak */	tweak_vsl_reclen,
-	/* var */	vsl_reclen,
-	/* min */	16b,
-	/* max */	4084b,
-	/* default */	255b,
-	/* units */	bytes,
-	/* flags */	00,
+	/* typ */	vsl_reclen,
+	/* min */	"16b",
+	/* max */	"4084b",
+	/* default */	"255b",
+	/* units */	"bytes",
+	/* flags */	0,
 	/* s-text */
 	"Maximum number of bytes in SHM log record.\n"
-	"The maximum tracks the vsl_buffer parameter - 12 bytes.\n",
+	"The maximum tracks the vsl_buffer parameter - 12 bytes.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	vsl_space,
-	/* tweak */	tweak_bytes,
-	/* var */	vsl_space,
-	/* min */	1M,
-	/* max */	none,
-	/* default */	80M,
-	/* units */	bytes,
-	/* flags */	0| MUST_RESTART,
+	/* typ */	bytes,
+	/* min */	"1M",
+	/* max */	NULL,
+	/* default */	"80M",
+	/* units */	"bytes",
+	/* flags */	MUST_RESTART,
 	/* s-text */
 	"The amount of space to allocate for the VSL fifo buffer in the "
 	"VSM memory segment.  If you make this too small, "
 	"varnish{ncsa|log} etc will not be able to keep up.  Making it too "
-	"large just costs memory resources.\n",
+	"large just costs memory resources.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	vsm_space,
-	/* tweak */	tweak_bytes,
-	/* var */	vsm_space,
-	/* min */	1M,
-	/* max */	none,
-	/* default */	1M,
-	/* units */	bytes,
-	/* flags */	0| MUST_RESTART,
+	/* typ */	bytes,
+	/* min */	"1M",
+	/* max */	NULL,
+	/* default */	"1M",
+	/* units */	"bytes",
+	/* flags */	MUST_RESTART,
 	/* s-text */
 	"The amount of space to allocate for stats counters in the VSM "
 	"memory segment.  If you make this too small, some counters will "
-	"be invisible.  Making it too large just costs memory resources.\n",
+	"be invisible.  Making it too large just costs memory resources.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #if 0
 PARAM(
 	/* name */	waiter,
-	/* tweak */	tweak_waiter,
-	/* var */	waiter,
-	/* min */	none,
-	/* max */	none,
+	/* typ */	waiter,
+	/* min */	NULL,
+	/* max */	NULL,
 	/* default */	kqueue (possible values: kqueue, poll),
-	/* units */	(null),
-	/* flags */	0| MUST_RESTART| WIZARD,
+	/* units */	NULL,
+	/* flags */	MUST_RESTART| WIZARD,
 	/* s-text */
-	"Select the waiter kernel interface.\n",
+	"Select the waiter kernel interface.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #endif
 PARAM(
 	/* name */	workspace_backend,
-	/* tweak */	tweak_bytes_u,
-	/* var */	workspace_backend,
-	/* min */	1k,
-	/* max */	none,
-	/* default */	64k,
-	/* units */	bytes,
-	/* flags */	0| DELAYED_EFFECT,
+	/* typ */	bytes_u,
+	/* min */	"1k",
+	/* max */	NULL,
+	/* default */	"64k",
+	/* units */	"bytes",
+	/* flags */	DELAYED_EFFECT,
 	/* s-text */
 	"Bytes of HTTP protocol workspace for backend HTTP req/resp.  If "
-	"larger than 4k, use a multiple of 4k for VM efficiency.\n",
+	"larger than 4k, use a multiple of 4k for VM efficiency.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	workspace_client,
-	/* tweak */	tweak_bytes_u,
-	/* var */	workspace_client,
-	/* min */	9k,
-	/* max */	none,
-	/* default */	64k,
-	/* units */	bytes,
-	/* flags */	0| DELAYED_EFFECT,
+	/* typ */	bytes_u,
+	/* min */	"9k",
+	/* max */	NULL,
+	/* default */	"64k",
+	/* units */	"bytes",
+	/* flags */	DELAYED_EFFECT,
 	/* s-text */
 	"Bytes of HTTP protocol workspace for clients HTTP req/resp.  If "
-	"larger than 4k, use a multiple of 4k for VM efficiency.\n",
+	"larger than 4k, use a multiple of 4k for VM efficiency.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	workspace_session,
-	/* tweak */	tweak_bytes_u,
-	/* var */	workspace_session,
-	/* min */	0.25k,
-	/* max */	none,
-	/* default */	0.50k,
-	/* units */	bytes,
-	/* flags */	0| DELAYED_EFFECT,
+	/* typ */	bytes_u,
+	/* min */	"0.25k",
+	/* max */	NULL,
+	/* default */	"0.50k",
+	/* units */	"bytes",
+	/* flags */	DELAYED_EFFECT,
 	/* s-text */
 	"Allocation size for session structure and workspace.    The "
 	"workspace is primarily used for TCP connection addresses.  If "
-	"larger than 4k, use a multiple of 4k for VM efficiency.\n",
+	"larger than 4k, use a multiple of 4k for VM efficiency.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 PARAM(
 	/* name */	workspace_thread,
-	/* tweak */	tweak_bytes_u,
-	/* var */	workspace_thread,
-	/* min */	0.25k,
-	/* max */	8k,
-	/* default */	2k,
-	/* units */	bytes,
-	/* flags */	0| DELAYED_EFFECT,
+	/* typ */	bytes_u,
+	/* min */	"0.25k",
+	/* max */	"8k",
+	/* default */	"2k",
+	/* units */	"bytes",
+	/* flags */	DELAYED_EFFECT,
 	/* s-text */
 	"Bytes of auxiliary workspace per thread.\n"
 	"This workspace is used for certain temporary data structures "
 	"during the operation of a worker thread.\n"
 	"One use is for the io-vectors for writing requests and responses "
 	"to sockets, having too little space will result in more writev(2) "
-	"system calls, having too much just wastes the space.\n",
+	"system calls, having too much just wastes the space.",
 	/* l-text */	"",
 	/* func */	NULL
 )



From lkarsten at varnish-software.com  Thu Jan 14 14:15:11 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:11 +0100
Subject: [4.1] 53efa72 Kill long forgotten and unused parameters
Message-ID: <E1aJigB-0005Rk-1v@project.varnish-software.com>


commit 53efa72ac54d317fe558adb2fe09a6a4df80dfb5
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Tue Jan 5 23:27:06 2016 +0000

    Kill long forgotten and unused parameters

diff --git a/bin/varnishd/common/params.h b/bin/varnishd/common/params.h
index 957b8a9..13ab904 100644
--- a/bin/varnishd/common/params.h
+++ b/bin/varnishd/common/params.h
@@ -85,15 +85,9 @@ struct params {
 #undef ptyp_vsl_reclen
 
 	/* Unprivileged user / group */
-	char			*user;
 	uid_t			uid;
-	char			*group;
 	gid_t			gid;
 
-	/* Extra group for compiler access */
-	char			*group_cc;
-	gid_t			gid_cc;
-
 	/* TTL used for lack of anything better */
 	double			default_ttl;
 
@@ -111,7 +105,6 @@ struct params {
 	unsigned		wthread_max;
 	double			wthread_timeout;
 	unsigned		wthread_pools;
-	unsigned		wthread_add_threshold;
 	double			wthread_add_delay;
 	double			wthread_fail_delay;
 	double			wthread_destroy_delay;
@@ -127,7 +120,6 @@ struct params {
 
 	unsigned		vsl_buffer;
 
-	unsigned		shm_workspace;
 	unsigned		http_req_size;
 	unsigned		http_req_hdr_len;
 	unsigned		http_resp_size;
@@ -196,8 +188,6 @@ struct params {
 	unsigned		gzip_level;
 	unsigned		gzip_memlevel;
 
-	unsigned		obj_readonly;
-
 	double			critbit_cooloff;
 
 	double			vcl_cooldown;



From lkarsten at varnish-software.com  Thu Jan 14 14:15:11 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:11 +0100
Subject: [4.1] 542c7f4 This has been replaced by jails
Message-ID: <E1aJigB-0005Rt-3L@project.varnish-software.com>


commit 542c7f4ed873eef81d90cacf82a46d47f7ac6e4e
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Tue Jan 5 23:27:50 2016 +0000

    This has been replaced by jails

diff --git a/bin/varnishd/mgt/mgt_sandbox.c b/bin/varnishd/mgt/mgt_sandbox.c
deleted file mode 100644
index 2b89a38..0000000
--- a/bin/varnishd/mgt/mgt_sandbox.c
+++ /dev/null
@@ -1,317 +0,0 @@
-/*-
- * Copyright (c) 2006-2011 Varnish Software AS
- * All rights reserved.
- *
- * Author: Poul-Henning Kamp <phk at phk.freebsd.dk>
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
- * 1. Redistributions of source code must retain the above copyright
- *    notice, this list of conditions and the following disclaimer.
- * 2. Redistributions in binary form must reproduce the above copyright
- *    notice, this list of conditions and the following disclaimer in the
- *    documentation and/or other materials provided with the distribution.
- *
- * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
- * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED.  IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
- * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
- * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
- * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
- * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
- * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
- * SUCH DAMAGE.
- *
- * Sandboxing child processes
- *
- * The worker/manager process border is one of the major security barriers
- * in Varnish, and therefore subject to whatever restrictions we have access
- * to under the given operating system.
- *
- * Unfortunately there is no consensus on APIs for this purpose, so each
- * operating system will require its own methods.
- *
- * This sourcefile tries to encapsulate the resulting mess on place.
- *
- * TODO:
- *	Unix:	chroot
- *	FreeBSD: jail
- *	FreeBSD: capsicum
- */
-
-#if 0
-
-#include "config.h"
-
-#ifdef __linux__
-#include <sys/prctl.h>
-#endif
-
-#include <pwd.h>
-#include <grp.h>
-#include <stdio.h>
-#include <stdlib.h>
-#include <string.h>
-#include <unistd.h>
-
-#include "mgt/mgt.h"
-#include "mgt/mgt_param.h"
-
-#include <vsub.h>
-
-mgt_sandbox_f *mgt_sandbox;
-
-/*--------------------------------------------------------------------
- * XXX: slightly magic.  We want to initialize to "nobody" (XXX: shouldn't
- * XXX: that be something autocrap found for us ?) but we don't want to
- * XXX: fail initialization if that user doesn't exists, even though we
- * XXX: do want to fail it, in subsequent sets.
- * XXX: The magic init string is a hack for this.
- */
-
-static int
-tweak_user(struct vsb *vsb, const struct parspec *par, const char *arg)
-{
-	struct passwd *pw;
-
-	(void)par;
-	if (arg != NULL) {
-		pw = getpwnam(arg);
-		if (pw == NULL) {
-			VSB_printf(vsb, "Unknown user '%s'", arg);
-			return(-1);
-		}
-		REPLACE(mgt_param.user, pw->pw_name);
-		mgt_param.uid = pw->pw_uid;
-		endpwent();
-	} else if (mgt_param.user) {
-		VSB_printf(vsb, "%s (%d)", mgt_param.user, (int)mgt_param.uid);
-	} else {
-		VSB_printf(vsb, "UID %d", (int)mgt_param.uid);
-	}
-	return (0);
-}
-
-/*--------------------------------------------------------------------
- * XXX: see comment for tweak_user, same thing here.
- */
-
-static int
-tweak_group(struct vsb *vsb, const struct parspec *par, const char *arg)
-{
-	struct group *gr;
-
-	(void)par;
-	if (arg != NULL) {
-		gr = getgrnam(arg);
-		if (gr == NULL) {
-			VSB_printf(vsb, "Unknown group '%s'", arg);
-			return(-1);
-		}
-		REPLACE(mgt_param.group, gr->gr_name);
-		mgt_param.gid = gr->gr_gid;
-		endgrent();
-	} else if (mgt_param.group) {
-		VSB_printf(vsb, "%s (%d)", mgt_param.group, (int)mgt_param.gid);
-	} else {
-		VSB_printf(vsb, "GID %d", (int)mgt_param.gid);
-	}
-	return (0);
-}
-
-/*--------------------------------------------------------------------
- * XXX: see comment for tweak_user, same thing here.
- */
-
-static int
-tweak_group_cc(struct vsb *vsb, const struct parspec *par, const char *arg)
-{
-	struct group *gr;
-
-	(void)par;
-	if (arg != NULL) {
-		if (*arg != '\0') {
-			gr = getgrnam(arg);
-			if (gr == NULL) {
-				VSB_printf(vsb, "Unknown group");
-				return(-1);
-			}
-			REPLACE(mgt_param.group_cc, gr->gr_name);
-			mgt_param.gid_cc = gr->gr_gid;
-		} else {
-			REPLACE(mgt_param.group_cc, "");
-			mgt_param.gid_cc = 0;
-		}
-	} else if (strlen(mgt_param.group_cc) > 0) {
-		VSB_printf(vsb, "%s (%d)",
-		    mgt_param.group_cc, (int)mgt_param.gid_cc);
-	} else {
-		VSB_printf(vsb, "<not set>");
-	}
-	return (0);
-}
-
-/*--------------------------------------------------------------------
- */
-
-static struct parspec mgt_parspec_sandbox[] = {
-	{ "user", tweak_user, NULL, NULL, NULL,
-		"The unprivileged user to run as.",
-		MUST_RESTART | ONLY_ROOT,
-		"" },
-	{ "group", tweak_group, NULL, NULL, NULL,
-		"The unprivileged group to run as.",
-		MUST_RESTART | ONLY_ROOT,
-		"" },
-	{ "group_cc", tweak_group_cc, NULL, NULL, NULL,
-		"On some systems the C-compiler is restricted so not"
-		" everybody can run it.  This parameter makes it possible"
-		" to add an extra group to the sandbox process which runs the"
-		" cc_command, in order to gain access to such a restricted"
-		" C-compiler.",
-		ONLY_ROOT,
-		"" },
-	{ NULL, NULL, NULL }
-};
-
-/*--------------------------------------------------------------------*/
-
-static void __match_proto__(mgt_sandbox_f)
-mgt_sandbox_null(enum sandbox_e who)
-{
-	(void)who;
-}
-
-/*--------------------------------------------------------------------*/
-
-#ifndef HAVE_SETPPRIV
-static void __match_proto__(mgt_sandbox_f)
-mgt_sandbox_unix(enum sandbox_e who)
-{
-#define NGID 2000
-	int i;
-	gid_t gid, gid_list[NGID];
-	uid_t uid;
-
-	if (who == SANDBOX_TESTING) {
-		/*
-		 * Test if sandboxing is going to work.
-		 * Do not assert on failure here, but simply exit non-zero.
-		 */
-		gid = getgid();
-		gid += 1;
-		if (setgid(gid))
-			exit(1);
-		uid = getuid();
-		uid += 1;
-		if (setuid(uid))
-			exit(2);
-		exit(0);
-	}
-
-	/*
-	 * Do the real thing, assert if we fail
-	 */
-
-	AZ(setgid(mgt_param.gid));
-	AZ(initgroups(mgt_param.user, mgt_param.gid));
-
-	if (who == SANDBOX_CC && strlen(mgt_param.group_cc) > 0) {
-		/* Add the optional extra group for the C-compiler access */
-		i = getgroups(NGID, gid_list);
-		assert(i >= 0);
-		gid_list[i++] = mgt_param.gid_cc;
-		AZ(setgroups(i, gid_list));
-	}
-
-	AZ(setuid(mgt_param.uid));
-
-#ifdef __linux__
-	/*
-	 * On linux mucking about with uid/gid disables core-dumps,
-	 * reenable them again.
-	 */
-	if (prctl(PR_SET_DUMPABLE, 1) != 0) {
-		MGT_complain(C_INFO,
-		    "Could not set dumpable bit.  Core dumps turned off\n");
-	}
-#endif
-}
-#endif
-
-/*--------------------------------------------------------------------*/
-
-static void __match_proto__(vsub_func_f)
-run_sandbox_test(void *priv)
-{
-
-	(void)priv;
-	mgt_sandbox(SANDBOX_TESTING);
-}
-
-/*--------------------------------------------------------------------*/
-
-void
-mgt_sandbox_init(void)
-{
-	struct passwd *pwd;
-	struct group *grp;
-	struct vsb *sb;
-	unsigned subs;
-
-	/* Pick a sandbox */
-
-#ifdef HAVE_SETPPRIV
-	mgt_sandbox = mgt_sandbox_solaris;
-#else
-	mgt_sandbox = mgt_sandbox_unix;
-#endif
-
-	/* Test it */
-
-	sb = VSB_new_auto();
-	subs = VSUB_run(sb, run_sandbox_test, NULL, "SANDBOX-test", 10);
-	VSB_delete(sb);
-	if (subs) {
-		MGT_complain(C_SECURITY,
-		    "Platform-specific sandbox failed - sandboxing disabled");
-		MGT_complain(C_SECURITY,
-		    "Varnish runs with elevated privileges");
-		mgt_sandbox = mgt_sandbox_null;
-	}
-
-	MCF_AddParams(mgt_parspec_sandbox);
-
-	/*
-	 * If we have nobody/nogroup, use them as defaults for sandboxes,
-	 * else fall back to whoever we run as.
-	 */
-	if (getpwnam("nobody") != NULL) {
-		MCF_SetDefault("user", "nobody");
-	} else {
-		pwd = getpwuid(getuid());
-		if (pwd == NULL)
-			ARGV_ERR("Neither user 'nobody' or my uid (%jd)"
-			    " found in password database.\n",
-			    (intmax_t)getuid());
-		MCF_SetDefault("user", pwd->pw_name);
-	}
-	endpwent();
-
-	if (getgrnam("nogroup") != NULL) {
-		MCF_SetDefault("group", "nogroup");
-	} else {
-		grp = getgrgid(getgid());
-		if (grp == NULL)
-			ARGV_ERR("Neither group 'nogroup' or my gid (%jd)"
-			    " found in password database.\n",
-			    (intmax_t)getgid());
-		MCF_SetDefault("group", grp->gr_name);
-	}
-	endgrent();
-}
-
-#endif



From lkarsten at varnish-software.com  Thu Jan 14 14:15:11 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:11 +0100
Subject: [4.1] 936a42a Add a hack so we always get the VSL logged on varnish
 -wait
Message-ID: <E1aJigB-0005S3-6S@project.varnish-software.com>


commit 936a42a6c8c52af1827aa3b59b67a8b6eb42c05c
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Jan 6 09:21:18 2016 +0000

    Add a hack so we always get the VSL logged on varnish -wait

diff --git a/bin/varnishtest/vtc_varnish.c b/bin/varnishtest/vtc_varnish.c
index 292be63..16d5d6c 100644
--- a/bin/varnishtest/vtc_varnish.c
+++ b/bin/varnishtest/vtc_varnish.c
@@ -77,6 +77,8 @@ struct varnish {
 	struct VSM_data		*vd;		/* vsc use */
 
 	unsigned		vsl_tag_count[256];
+	
+	volatile int		vsl_idle;
 };
 
 #define NONSENSE	"%XJEIFLH|)Xspa8P"
@@ -223,8 +225,9 @@ varnishlog_thread(void *priv)
 
 		i = VSL_Next(c);
 		if (i == 0) {
+			v->vsl_idle++;
 			/* Nothing to do but wait */
-			VTIM_sleep(0.01);
+			VTIM_sleep(0.1);
 			continue;
 		} else if (i == -2) {
 			/* Abandoned - try reconnect */
@@ -235,6 +238,8 @@ varnishlog_thread(void *priv)
 		} else if (i != 1)
 			break;
 
+		v->vsl_idle = 0;
+
 		tag = VSL_TAG(c->rec.ptr);
 		vxid = VSL_ID(c->rec.ptr);
 		if (tag == SLT__Batch)
@@ -595,9 +600,11 @@ varnish_wait(struct varnish *v)
 
 	if (v->cli_fd < 0)
 		return;
+	varnish_ask_cli(v, "backend.list", &resp);
+	while (v->vsl_idle < 10)
+		(void)usleep(200000);
 	if (vtc_error)
 		(void)sleep(1);	/* give panic messages a chance */
-	varnish_ask_cli(v, "backend.list", &resp);
 	varnish_stop(v);
 	vtc_log(v->vl, 2, "Wait");
 	AZ(close(v->cli_fd));



From lkarsten at varnish-software.com  Thu Jan 14 14:15:11 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:11 +0100
Subject: [4.1] 43f9af4 Ignore SLT_CLI when waiting for vsl
Message-ID: <E1aJigB-0005SD-8f@project.varnish-software.com>


commit 43f9af4647237ba041152aaf055c7bbef8df3a40
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Jan 6 11:43:03 2016 +0000

    Ignore SLT_CLI when waiting for vsl

diff --git a/bin/varnishtest/vtc_varnish.c b/bin/varnishtest/vtc_varnish.c
index 16d5d6c..500f0ae 100644
--- a/bin/varnishtest/vtc_varnish.c
+++ b/bin/varnishtest/vtc_varnish.c
@@ -242,6 +242,8 @@ varnishlog_thread(void *priv)
 
 		tag = VSL_TAG(c->rec.ptr);
 		vxid = VSL_ID(c->rec.ptr);
+		if (tag != SLT_CLI)
+			v->vsl_idle = 0;
 		if (tag == SLT__Batch)
 			continue;
 		tagname = VSL_tags[tag];



From lkarsten at varnish-software.com  Thu Jan 14 14:15:11 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:11 +0100
Subject: [4.1] 8522bad Save a tons of sysconf() syscalls.
Message-ID: <E1aJigB-0005SM-AG@project.varnish-software.com>


commit 8522bad93f3283bb484968cbc0c048c7fab6aff5
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Jan 6 13:27:06 2016 +0000

    Save a tons of sysconf() syscalls.

diff --git a/lib/libvarnish/vsub.c b/lib/libvarnish/vsub.c
index 4e3067a..f7b6a07 100644
--- a/lib/libvarnish/vsub.c
+++ b/lib/libvarnish/vsub.c
@@ -70,7 +70,7 @@ unsigned
 VSUB_run(struct vsb *sb, vsub_func_f *func, void *priv, const char *name,
     int maxlines)
 {
-	int rv, p[2], sfd, status;
+	int rv, p[2], sfd, hfd, status;
 	pid_t pid;
 	struct vlu *vlu;
 	struct vsub_priv sp;
@@ -100,7 +100,8 @@ VSUB_run(struct vsb *sb, vsub_func_f *func, void *priv, const char *name,
 		assert(dup2(p[1], STDOUT_FILENO) == STDOUT_FILENO);
 		assert(dup2(p[1], STDERR_FILENO) == STDERR_FILENO);
 		/* Close all other fds */
-		for (sfd = STDERR_FILENO+1; sfd < sysconf(_SC_OPEN_MAX); sfd++)
+		hfd = sysconf(_SC_OPEN_MAX);
+		for (sfd = STDERR_FILENO+1; sfd < hfd; sfd++)
 			(void)close(sfd);
 		func(priv);
 		/*



From lkarsten at varnish-software.com  Thu Jan 14 14:15:11 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:11 +0100
Subject: [4.1] 891619e Don't use getdtablesize() for closing existing fd's it
 only represents the current limit.
Message-ID: <E1aJigB-0005SV-Bt@project.varnish-software.com>


commit 891619e235338c1a77d27527c7237a07f16e910a
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Jan 6 13:32:47 2016 +0000

    Don't use getdtablesize() for closing existing fd's it only
    represents the current limit.
    
    Also don't call it every iteration of the loop.

diff --git a/bin/varnishd/mgt/mgt_main.c b/bin/varnishd/mgt/mgt_main.c
index 1073b8e..25c7ce4 100644
--- a/bin/varnishd/mgt/mgt_main.c
+++ b/bin/varnishd/mgt/mgt_main.c
@@ -530,7 +530,7 @@ main(int argc, char * const *argv)
 	 * Start out by closing all unwanted file descriptors we might
 	 * have inherited from sloppy process control daemons.
 	 */
-	for (o = getdtablesize(); o > STDERR_FILENO; o--)
+	for (o = sysconf(_SC_OPEN_MAX); o > STDERR_FILENO; o--)
 		(void)close(o);
 
 	VRND_Seed();
diff --git a/bin/varnishtest/vtc_process.c b/bin/varnishtest/vtc_process.c
index c0a15c2..8770b5f 100644
--- a/bin/varnishtest/vtc_process.c
+++ b/bin/varnishtest/vtc_process.c
@@ -203,7 +203,7 @@ process_start(struct process *p)
 		assert(dup2(p->fds[0], 0) == 0);
 		assert(dup2(out_fd, 1) == 1);
 		assert(dup2(err_fd, 2) == 2);
-		for (i = 3; i <getdtablesize(); i++)
+		for (i = sysconf(_SC_OPEN_MAX); i > STDERR_FILENO; i--)
 			(void)close(i);
 		AZ(execl("/bin/sh", "/bin/sh", "-c", VSB_data(cl), (char*)0));
 		exit(1);



From lkarsten at varnish-software.com  Thu Jan 14 14:15:11 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:11 +0100
Subject: [4.1] e0f9b7a Add VSUB_closefrom() to close all fd's higher than some
 number.
Message-ID: <E1aJigB-0005Se-Dn@project.varnish-software.com>


commit e0f9b7a2b4dfb9962cd0fb4d848516daee4cfb6d
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Jan 6 13:48:04 2016 +0000

    Add VSUB_closefrom() to close all fd's higher than some number.
    
    Check if system has closefrom(2) and use that where available

diff --git a/bin/varnishd/mgt/mgt_main.c b/bin/varnishd/mgt/mgt_main.c
index 25c7ce4..c54ac60 100644
--- a/bin/varnishd/mgt/mgt_main.c
+++ b/bin/varnishd/mgt/mgt_main.c
@@ -57,6 +57,7 @@
 #include "vpf.h"
 #include "vrnd.h"
 #include "vsha256.h"
+#include "vsub.h"
 #include "vtim.h"
 #include "waiter/mgt_waiter.h"
 
@@ -530,8 +531,7 @@ main(int argc, char * const *argv)
 	 * Start out by closing all unwanted file descriptors we might
 	 * have inherited from sloppy process control daemons.
 	 */
-	for (o = sysconf(_SC_OPEN_MAX); o > STDERR_FILENO; o--)
-		(void)close(o);
+	VSUB_closefrom(STDERR_FILENO + 1);
 
 	VRND_Seed();
 
diff --git a/bin/varnishd/mgt/mgt_vcl.c b/bin/varnishd/mgt/mgt_vcl.c
index 37359f2..dec0ec0 100644
--- a/bin/varnishd/mgt/mgt_vcl.c
+++ b/bin/varnishd/mgt/mgt_vcl.c
@@ -93,6 +93,7 @@ mgt_vcl_del(struct vclprog *vp)
 	VTAILQ_REMOVE(&vclhead, vp, list);
 	XXXAZ(unlink(vp->fname));
 	bprintf(dn, "vcl_%s", vp->name);
+	WRONG("TEST");
 	VJ_master(JAIL_MASTER_FILE);
 	(void)rmdir(dn);		// compiler droppings, eg gcov
 	VJ_master(JAIL_MASTER_LOW);
diff --git a/configure.ac b/configure.ac
index 8c753a3..227a9bf 100644
--- a/configure.ac
+++ b/configure.ac
@@ -245,6 +245,7 @@ AC_CHECK_FUNCS([getdtablesize])
 AC_CHECK_FUNCS([nanosleep])
 AC_CHECK_FUNCS([setppriv])
 AC_CHECK_FUNCS([fallocate])
+AC_CHECK_FUNCS([closefrom])
 
 save_LIBS="${LIBS}"
 LIBS="${PTHREAD_LIBS}"
diff --git a/include/vsub.h b/include/vsub.h
index bc337a1..de62908 100644
--- a/include/vsub.h
+++ b/include/vsub.h
@@ -33,3 +33,5 @@ typedef void vsub_func_f(void*);
 
 unsigned VSUB_run(struct vsb *, vsub_func_f *, void *priv, const char *name,
     int maxlines);
+
+void VSUB_closefrom(int fd);
diff --git a/lib/libvarnish/vsub.c b/lib/libvarnish/vsub.c
index f7b6a07..c87d297 100644
--- a/lib/libvarnish/vsub.c
+++ b/lib/libvarnish/vsub.c
@@ -52,6 +52,18 @@ struct vsub_priv {
 	int		maxlines;
 };
 
+void
+VSUB_closefrom(int fd)
+{
+#ifdef HAVE_CLOSEFROM
+	closefrom(fd);
+#else
+	int i;= sysconf(_SC_OPEN_MAX);
+	for (i = sysconf(_SC_OPEN_MAX); i > STDERR_FILENO; i--)
+		(void)close(i);
+#endif
+}
+
 static int
 vsub_vlu(void *priv, const char *str)
 {
@@ -70,7 +82,7 @@ unsigned
 VSUB_run(struct vsb *sb, vsub_func_f *func, void *priv, const char *name,
     int maxlines)
 {
-	int rv, p[2], sfd, hfd, status;
+	int rv, p[2], status;
 	pid_t pid;
 	struct vlu *vlu;
 	struct vsub_priv sp;
@@ -100,9 +112,7 @@ VSUB_run(struct vsb *sb, vsub_func_f *func, void *priv, const char *name,
 		assert(dup2(p[1], STDOUT_FILENO) == STDOUT_FILENO);
 		assert(dup2(p[1], STDERR_FILENO) == STDERR_FILENO);
 		/* Close all other fds */
-		hfd = sysconf(_SC_OPEN_MAX);
-		for (sfd = STDERR_FILENO+1; sfd < hfd; sfd++)
-			(void)close(sfd);
+		VSUB_closefrom(STDERR_FILENO + 1);
 		func(priv);
 		/*
 		 * func should either exec or exit, so getting here should be



From lkarsten at varnish-software.com  Thu Jan 14 14:15:11 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:11 +0100
Subject: [4.1] 5d935b1 Use VSUB_closefrom()
Message-ID: <E1aJigB-0005Sn-FP@project.varnish-software.com>


commit 5d935b1ab207b9944ff475972a91ca9a4f7d25e9
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Jan 6 13:48:58 2016 +0000

    Use VSUB_closefrom()

diff --git a/bin/varnishtest/vtc_process.c b/bin/varnishtest/vtc_process.c
index 8770b5f..f63c2e5 100644
--- a/bin/varnishtest/vtc_process.c
+++ b/bin/varnishtest/vtc_process.c
@@ -42,6 +42,7 @@
 #include <unistd.h>
 
 #include "vtc.h"
+#include "vsub.h"
 
 struct process {
 	unsigned		magic;
@@ -183,7 +184,7 @@ static void
 process_start(struct process *p)
 {
 	struct vsb *cl;
-	int i, out_fd, err_fd;
+	int out_fd, err_fd;
 
 	CHECK_OBJ_NOTNULL(p, PROCESS_MAGIC);
 
@@ -203,8 +204,7 @@ process_start(struct process *p)
 		assert(dup2(p->fds[0], 0) == 0);
 		assert(dup2(out_fd, 1) == 1);
 		assert(dup2(err_fd, 2) == 2);
-		for (i = sysconf(_SC_OPEN_MAX); i > STDERR_FILENO; i--)
-			(void)close(i);
+		VSUB_closefrom(STDERR_FILENO + 1);
 		AZ(execl("/bin/sh", "/bin/sh", "-c", VSB_data(cl), (char*)0));
 		exit(1);
 	}



From lkarsten at varnish-software.com  Thu Jan 14 14:15:11 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:11 +0100
Subject: [4.1] 63c7766 Remove debugging hack which slipped into previous
 commit.
Message-ID: <E1aJigB-0005Sw-HO@project.varnish-software.com>


commit 63c77662407f78144a03a0059deded374714b0c4
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Jan 6 13:50:24 2016 +0000

    Remove debugging hack which slipped into previous commit.

diff --git a/bin/varnishd/mgt/mgt_vcl.c b/bin/varnishd/mgt/mgt_vcl.c
index dec0ec0..37359f2 100644
--- a/bin/varnishd/mgt/mgt_vcl.c
+++ b/bin/varnishd/mgt/mgt_vcl.c
@@ -93,7 +93,6 @@ mgt_vcl_del(struct vclprog *vp)
 	VTAILQ_REMOVE(&vclhead, vp, list);
 	XXXAZ(unlink(vp->fname));
 	bprintf(dn, "vcl_%s", vp->name);
-	WRONG("TEST");
 	VJ_master(JAIL_MASTER_FILE);
 	(void)rmdir(dn);		// compiler droppings, eg gcov
 	VJ_master(JAIL_MASTER_LOW);



From lkarsten at varnish-software.com  Thu Jan 14 14:15:11 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:11 +0100
Subject: [4.1] e821059 Also use VSUB_closefrom() here.
Message-ID: <E1aJigB-0005T5-J9@project.varnish-software.com>


commit e8210595c9b030a8577a5f0e1385b9fa1f76ec92
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Jan 6 13:51:39 2016 +0000

    Also use VSUB_closefrom() here.

diff --git a/bin/varnishtest/vtc_varnish.c b/bin/varnishtest/vtc_varnish.c
index 500f0ae..77d8715 100644
--- a/bin/varnishtest/vtc_varnish.c
+++ b/bin/varnishtest/vtc_varnish.c
@@ -49,6 +49,7 @@
 #include "vapi/vsm.h"
 #include "vcli.h"
 #include "vss.h"
+#include "vsub.h"
 #include "vtcp.h"
 #include "vtim.h"
 
@@ -434,8 +435,7 @@ varnish_launch(struct varnish *v)
 		AZ(close(v->fds[1]));
 		AZ(close(v->fds[2]));
 		AZ(close(v->fds[3]));
-		for (i = 3; i <getdtablesize(); i++)
-			(void)close(i);
+		VSUB_closefrom(STDERR_FILENO + 1);
 		AZ(execl("/bin/sh", "/bin/sh", "-c", VSB_data(vsb), (char*)0));
 		exit(1);
 	} else {



From lkarsten at varnish-software.com  Thu Jan 14 14:15:11 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:11 +0100
Subject: [4.1] c1784aa Fix build if closefrom is not available
Message-ID: <E1aJigB-0005TE-Ks@project.varnish-software.com>


commit c1784aa33ca05954aea777f29d9a97c5b0256fa9
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Wed Jan 6 14:14:33 2016 +0000

    Fix build if closefrom is not available

diff --git a/lib/libvarnish/vsub.c b/lib/libvarnish/vsub.c
index c87d297..b669fff 100644
--- a/lib/libvarnish/vsub.c
+++ b/lib/libvarnish/vsub.c
@@ -58,8 +58,8 @@ VSUB_closefrom(int fd)
 #ifdef HAVE_CLOSEFROM
 	closefrom(fd);
 #else
-	int i;= sysconf(_SC_OPEN_MAX);
-	for (i = sysconf(_SC_OPEN_MAX); i > STDERR_FILENO; i--)
+	int i = sysconf(_SC_OPEN_MAX);
+	for (i = sysconf(_SC_OPEN_MAX); i > fd; i--)
 		(void)close(i);
 #endif
 }



From lkarsten at varnish-software.com  Thu Jan 14 14:15:11 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:11 +0100
Subject: [4.1] 55dc8fd Fix r01684.vtc logexpect handling
Message-ID: <E1aJigB-0005TN-MQ@project.varnish-software.com>


commit 55dc8fd45be498c3f22d3516e3648577e743f6a7
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Wed Jan 6 17:14:38 2016 +0100

    Fix r01684.vtc logexpect handling
    
    The script lacked a logexpect -wait at the end telling it to actually
    wait for the required log content.
    
    Added a new rule to match first on an actual retry backend transaction
    and match the header set/unset/set tests on that vxid.

diff --git a/bin/varnishtest/tests/r01684.vtc b/bin/varnishtest/tests/r01684.vtc
index f4eb71e..9c2f01f 100644
--- a/bin/varnishtest/tests/r01684.vtc
+++ b/bin/varnishtest/tests/r01684.vtc
@@ -21,10 +21,11 @@ varnish v1 -vcl+backend {
 } -start
 
 # check log for the aborted POST
-logexpect l1 -v v1 {
-	expect * =	BereqHeader "^X-Varnish:"
+logexpect l1 -v v1 -g request {
+	expect * *	Begin		"^bereq .* retry"
+	expect * =	BereqHeader	"^X-Varnish:"
 	expect * =	BereqUnset	"^X-Varnish:"
-	expect * =	BereqHeader	"^X-Varnish"
+	expect * =	BereqHeader	"^X-Varnish:"
 } -start
 
 varnish v1 -cliok "param.set debug +syncvsl"
@@ -35,3 +36,5 @@ client c1 {
 	rxresp
 	expect resp.http.foo == 3
 } -run
+
+logexpect l1 -wait



From lkarsten at varnish-software.com  Thu Jan 14 14:15:11 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:11 +0100
Subject: [4.1] c1f8013 Another use for VSUB_closefrom()
Message-ID: <E1aJigB-0005TW-Nr@project.varnish-software.com>


commit c1f801315268ee2984d4dffca4ddd4b1cc200499
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Jan 6 14:01:19 2016 +0000

    Another use for VSUB_closefrom()

diff --git a/bin/varnishtest/vtc_main.c b/bin/varnishtest/vtc_main.c
index 9c9f255..601f245 100644
--- a/bin/varnishtest/vtc_main.c
+++ b/bin/varnishtest/vtc_main.c
@@ -51,6 +51,7 @@
 #include "vrnd.h"
 #include "vsa.h"
 #include "vss.h"
+#include "vsub.h"
 #include "vtcp.h"
 #include "vtim.h"
 
@@ -232,7 +233,7 @@ static void
 start_test(void)
 {
 	struct vtc_tst *tp;
-	int p[2], sfd, retval;
+	int p[2], retval;
 	struct vtc_job *jp;
 	char tmpdir[PATH_MAX];
 
@@ -274,8 +275,7 @@ start_test(void)
 		assert(open("/dev/null", O_RDONLY) == STDIN_FILENO);
 		assert(dup2(p[1], STDOUT_FILENO) == STDOUT_FILENO);
 		assert(dup2(p[1], STDERR_FILENO) == STDERR_FILENO);
-		for (sfd = STDERR_FILENO + 1; sfd < 100; sfd++)
-			(void)close(sfd);
+		VSUB_closefrom(STDERR_FILENO + 1);
 		retval = exec_file(jp->tst->filename, jp->tst->script,
 		    jp->tmpdir, jp->buf, jp->bufsiz);
 		exit(retval);



From lkarsten at varnish-software.com  Thu Jan 14 14:15:11 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:11 +0100
Subject: [4.1] 5f9cf70 Fix typo spotted by fgs
Message-ID: <E1aJigB-0005Tf-PG@project.varnish-software.com>


commit 5f9cf70f1a0d00619e110c7ad2200b346992a719
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Jan 6 14:34:30 2016 +0000

    Fix typo spotted by fgs

diff --git a/lib/libvarnish/vsub.c b/lib/libvarnish/vsub.c
index b669fff..d774e53 100644
--- a/lib/libvarnish/vsub.c
+++ b/lib/libvarnish/vsub.c
@@ -58,7 +58,7 @@ VSUB_closefrom(int fd)
 #ifdef HAVE_CLOSEFROM
 	closefrom(fd);
 #else
-	int i = sysconf(_SC_OPEN_MAX);
+	int i;
 	for (i = sysconf(_SC_OPEN_MAX); i > fd; i--)
 		(void)close(i);
 #endif



From lkarsten at varnish-software.com  Thu Jan 14 14:15:11 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:11 +0100
Subject: [4.1] c957d3e Don't close stdout/stderr when we stop due to EOF on
 stdin in -d mode.
Message-ID: <E1aJigB-0005To-Qs@project.varnish-software.com>


commit c957d3efd3a26ef46a884b6d0b9ec8b0604e0a07
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Jan 6 16:22:44 2016 +0000

    Don't close stdout/stderr when we stop due to EOF on stdin in -d mode.
    
    That obscured problems in cleanup

diff --git a/bin/varnishd/mgt/mgt_main.c b/bin/varnishd/mgt/mgt_main.c
index c54ac60..a7533e9 100644
--- a/bin/varnishd/mgt/mgt_main.c
+++ b/bin/varnishd/mgt/mgt_main.c
@@ -355,12 +355,6 @@ cli_stdin_close(void *priv)
 {
 
 	(void)priv;
-	(void)close(0);
-	(void)close(1);
-	(void)close(2);
-	AZ(open("/dev/null", O_RDONLY));
-	assert(open("/dev/null", O_WRONLY) == 1);
-	assert(open("/dev/null", O_WRONLY) == 2);
 
 	if (d_flag) {
 		mgt_stop_child();
@@ -368,6 +362,13 @@ cli_stdin_close(void *priv)
 		if (pfh != NULL)
 			(void)VPF_Remove(pfh);
 		exit(0);
+	} else {
+		(void)close(0);
+		(void)close(1);
+		(void)close(2);
+		AZ(open("/dev/null", O_RDONLY));
+		assert(open("/dev/null", O_WRONLY) == 1);
+		assert(open("/dev/null", O_WRONLY) == 2);
 	}
 }
 



From lkarsten at varnish-software.com  Thu Jan 14 14:15:11 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:11 +0100
Subject: [4.1] 1a3b0ff This seems to get test-termination to stabilize,
 including the new improved probability of getting the VSL out.
Message-ID: <E1aJigB-0005Tx-SW@project.varnish-software.com>


commit 1a3b0ff51af1a258e7cbb77cd34341f9ccbedb1e
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Jan 6 16:25:13 2016 +0000

    This seems to get test-termination to stabilize, including the
    new improved probability of getting the VSL out.

diff --git a/bin/varnishtest/tests/b00044.vtc b/bin/varnishtest/tests/b00044.vtc
index 5d02319..463e383 100644
--- a/bin/varnishtest/tests/b00044.vtc
+++ b/bin/varnishtest/tests/b00044.vtc
@@ -15,3 +15,5 @@ client c1 {
 server s1 -wait
 
 shell "kill -15 ${v1_pid}"
+
+varnish v1 -cleanup
diff --git a/bin/varnishtest/vtc.c b/bin/varnishtest/vtc.c
index 3c6c884..33b56e1 100644
--- a/bin/varnishtest/vtc.c
+++ b/bin/varnishtest/vtc.c
@@ -684,7 +684,7 @@ exec_file(const char *fn, const char *script, const char *tmpdir,
 	vtc_stop = 1;
 	vtc_log(vltop, 1, "RESETTING after %s", fn);
 	reset_cmds(cmds);
-	vtc_error = old_err;
+	vtc_error |= old_err;
 
 	if (vtc_error)
 		vtc_log(vltop, 1, "TEST %s FAILED", fn);
diff --git a/bin/varnishtest/vtc_log.c b/bin/varnishtest/vtc_log.c
index 4a73ebd..30ae6b8 100644
--- a/bin/varnishtest/vtc_log.c
+++ b/bin/varnishtest/vtc_log.c
@@ -112,8 +112,6 @@ vtc_log_emit(const struct vtclog *vl, int lvl)
 
 	if (lvl < 0)
 		lvl = 0;
-	if (vtc_stop && lvl == 0)
-		return;
 	l = VSB_len(vl->vsb);
 	AZ(pthread_mutex_lock(&vtclog_mtx));
 	assert(vtclog_left > l);
diff --git a/bin/varnishtest/vtc_varnish.c b/bin/varnishtest/vtc_varnish.c
index 77d8715..91c4072 100644
--- a/bin/varnishtest/vtc_varnish.c
+++ b/bin/varnishtest/vtc_varnish.c
@@ -78,7 +78,7 @@ struct varnish {
 	struct VSM_data		*vd;		/* vsc use */
 
 	unsigned		vsl_tag_count[256];
-	
+
 	volatile int		vsl_idle;
 };
 
@@ -209,6 +209,7 @@ varnishlog_thread(void *priv)
 	opt = 0;
 	while (v->pid) {
 		if (c == NULL) {
+			v->vsl_idle++;
 			VTIM_sleep(0.1);
 			if (VSM_Open(vsm)) {
 				VSM_ResetError(vsm);
@@ -257,6 +258,8 @@ varnishlog_thread(void *priv)
 		    type, (int)len, data);
 	}
 
+	v->vsl_idle = 100;
+
 	if (c)
 		VSL_DeleteCursor(c);
 	VSL_Delete(vsl);
@@ -343,28 +346,29 @@ varnish_thread(void *priv)
 {
 	struct varnish *v;
 	char buf[65536];
-	struct pollfd *fds, fd;
+	struct pollfd fds[1];
 	int i;
 
 	CAST_OBJ_NOTNULL(v, priv, VARNISH_MAGIC);
 	(void)VTCP_nonblocking(v->fds[0]);
 	while (1) {
-		fds = &fd;
-		memset(fds, 0, sizeof *fds);
+		memset(fds, 0, sizeof fds);
 		fds->fd = v->fds[0];
 		fds->events = POLLIN;
 		i = poll(fds, 1, 1000);
 		if (i == 0)
 			continue;
+		if (fds->revents & POLLIN) {
+			i = read(v->fds[0], buf, sizeof buf - 1);
+			if (i > 0) {
+				buf[i] = '\0';
+				vtc_dump(v->vl, 3, "debug", buf, -2);
+			}
+		}
 		if (fds->revents & (POLLERR|POLLHUP)) {
 			vtc_log(v->vl, 4, "STDOUT poll 0x%x", fds->revents);
 			break;
 		}
-		i = read(v->fds[0], buf, sizeof buf - 1);
-		if (i <= 0)
-			break;
-		buf[i] = '\0';
-		vtc_dump(v->vl, 3, "debug", buf, -2);
 	}
 	return (NULL);
 }
@@ -589,41 +593,42 @@ varnish_stop(struct varnish *v)
 }
 
 /**********************************************************************
- * Wait for a Varnish
+ * Cleanup
  */
 
 static void
-varnish_wait(struct varnish *v)
+varnish_cleanup(struct varnish *v)
 {
 	void *p;
 	int status, r;
 	struct rusage ru;
-	char *resp;
 
-	if (v->cli_fd < 0)
-		return;
-	varnish_ask_cli(v, "backend.list", &resp);
+	/* Give the VSL log time to finish */
 	while (v->vsl_idle < 10)
 		(void)usleep(200000);
-	if (vtc_error)
-		(void)sleep(1);	/* give panic messages a chance */
-	varnish_stop(v);
-	vtc_log(v->vl, 2, "Wait");
+
+	/* Close the CLI connection */
 	AZ(close(v->cli_fd));
 	v->cli_fd = -1;
 
-	(void)close(v->fds[1]);		/* May already have been closed */
+	/* Close the STDIN connection. */
+	AZ(close(v->fds[1]));
 
+	/* Wait until STDOUT+STDERR closes */
 	AZ(pthread_join(v->tp, &p));
 	AZ(close(v->fds[0]));
+
 	r = wait4(v->pid, &status, 0, &ru);
 	v->pid = 0;
 	vtc_log(v->vl, 2, "R %d Status: %04x (u %.6f s %.6f)", r, status,
 	    ru.ru_utime.tv_sec + 1e-6 * ru.ru_utime.tv_usec,
 	    ru.ru_stime.tv_sec + 1e-6 * ru.ru_stime.tv_usec
 	);
+
+	/* Pick up the VSL thread */
 	AZ(pthread_join(v->tp_vsl, &p));
-	if (WIFEXITED(status) && WEXITSTATUS(status) == 0)
+
+	if (WIFEXITED(status) && (WEXITSTATUS(status) == 0))
 		return;
 #ifdef WCOREDUMP
 	vtc_log(v->vl, 0, "Bad exit code: %04x sig %x exit %x core %x",
@@ -636,6 +641,30 @@ varnish_wait(struct varnish *v)
 }
 
 /**********************************************************************
+ * Wait for a Varnish
+ */
+
+static void
+varnish_wait(struct varnish *v)
+{
+	char *resp;
+
+	if (v->cli_fd < 0)
+		return;
+
+	vtc_log(v->vl, 2, "Wait");
+
+	/* Do a backend.list to log if child is still running */
+	varnish_ask_cli(v, "backend.list", &resp);
+
+	/* Then stop it */
+	varnish_stop(v);
+
+	varnish_cleanup(v);
+}
+
+
+/**********************************************************************
  * Ask a CLI question
  */
 
@@ -930,6 +959,11 @@ cmd_varnish(CMD_ARGS)
 			av += 2;
 			continue;
 		}
+		if (!strcmp(*av, "-cleanup")) {
+			AZ(av[1]);
+			varnish_cleanup(v);
+			continue;
+		}
 		if (!strcmp(*av, "-cliok")) {
 			AN(av[1]);
 			varnish_cli(v, av[1], (unsigned)CLIS_OK);



From lkarsten at varnish-software.com  Thu Jan 14 14:15:11 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:11 +0100
Subject: [4.1] fef00c4 SmartOS needs stdlib.h for closefrom()
Message-ID: <E1aJigB-0005U7-Ua@project.varnish-software.com>


commit fef00c4c1f98088bac2abaf3042bcae655ceb31c
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Wed Jan 6 16:49:11 2016 +0000

    SmartOS needs stdlib.h for closefrom()

diff --git a/lib/libvarnish/vsub.c b/lib/libvarnish/vsub.c
index d774e53..a7c658c 100644
--- a/lib/libvarnish/vsub.c
+++ b/lib/libvarnish/vsub.c
@@ -36,6 +36,7 @@
 #include <errno.h>
 #include <fcntl.h>
 #include <stdio.h>
+#include <stdlib.h>
 #include <string.h>
 #include <unistd.h>
 



From lkarsten at varnish-software.com  Thu Jan 14 14:15:11 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:11 +0100
Subject: [4.1] 8ba2781 On fetch failure,
 make the vcl_backend_error{} object cacheable for 5 seconds if there
 is a waiting list.
Message-ID: <E1aJigB-0005UG-W8@project.varnish-software.com>


commit 8ba2781e72ceb4ce6046e74a239094931f1b1b88
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Jan 6 18:57:31 2016 +0000

    On fetch failure, make the vcl_backend_error{} object cacheable
    for 5 seconds if there is a waiting list.
    
    This allows the waiting list to drain (returning 503) quickly,
    rather than one by one trying to pester the backend and only
    giving up after the connect_timeout.
    
    Related to: #1823

diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index cd64257..3d0517c 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -793,8 +793,22 @@ vbf_stp_error(struct worker *wrk, struct busyobj *bo)
 	http_TimeHeader(bo->beresp, "Date: ", now);
 	http_SetHeader(bo->beresp, "Server: Varnish");
 
-	EXP_Clr(&bo->fetch_objcore->exp);
-	bo->fetch_objcore->exp.t_origin = bo->t_prev;
+	if (bo->fetch_objcore->objhead->waitinglist != NULL) {
+		/*
+		 * If there is a waitinglist, it means that there is no
+		 * grace-able object, so cache the error return for a
+		 * short time, so the waiting list can drain, rather than
+		 * each objcore on the waiting list sequentially attempt
+		 * to fetch from the backend.
+		 */
+		bo->fetch_objcore->exp.t_origin = now;
+		bo->fetch_objcore->exp.ttl = 1;
+		bo->fetch_objcore->exp.grace = 5;
+		bo->fetch_objcore->exp.keep = 5;
+	} else {
+		EXP_Clr(&bo->fetch_objcore->exp);
+		bo->fetch_objcore->exp.t_origin = now;
+	}
 
 	synth_body = VSB_new_auto();
 	AN(synth_body);



From lkarsten at varnish-software.com  Thu Jan 14 14:15:12 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:12 +0100
Subject: [4.1] b2ee567 Rush the objheader if there is a waiting list when it
 is deref'ed.
Message-ID: <E1aJigC-0005UP-1N@project.varnish-software.com>


commit b2ee5677764ce90e59083e665af2188fa255b100
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Jan 6 19:13:47 2016 +0000

    Rush the objheader if there is a waiting list when it is deref'ed.
    
    Fixes:	#1823

diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index dd03596..0bb827d 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -820,6 +820,10 @@ HSH_DerefObjHead(struct worker *wrk, struct objhead **poh)
 		oh->refcnt--;
 		Lck_Unlock(&oh->mtx);
 		return(1);
+	} else if (oh->waitinglist != NULL) {
+		Lck_Lock(&oh->mtx);
+		hsh_rush(wrk, oh);
+		Lck_Unlock(&oh->mtx);
 	}
 
 	assert(oh->refcnt > 0);



From lkarsten at varnish-software.com  Thu Jan 14 14:15:12 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:12 +0100
Subject: [4.1] 59dfdbd Do not send a zero Content-Length for 204 and 304
Message-ID: <E1aJigC-0005UY-2y@project.varnish-software.com>


commit 59dfdbd740c38a3a7316dd7af24490c872bd95bc
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Sun Jan 3 19:10:29 2016 +0000

    Do not send a zero Content-Length for 204 and 304
    
    In the 304 case we were already skipping it for non-zero C-L.
    For 204 responses this was introduced as a side effect of 271e1c52.

diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index a539817..7627726 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -73,7 +73,7 @@ cnt_vdp(struct req *req, struct busyobj *bo)
 		else
 			sendbody = 0;
 	} else if (status < 200 || status == 204 || status == 304) {
-		req->resp_len = 0;
+		req->resp_len = -1;
 		sendbody = 0;
 	} else
 		sendbody = 1;
diff --git a/bin/varnishtest/tests/c00008.vtc b/bin/varnishtest/tests/c00008.vtc
index 690b362..eb3c472 100644
--- a/bin/varnishtest/tests/c00008.vtc
+++ b/bin/varnishtest/tests/c00008.vtc
@@ -6,6 +6,10 @@ server s1 {
 	txresp -hdr "Last-Modified: Thu, 26 Jun 2008 12:00:01 GMT" \
 	    -hdr {ETag: "foo"} \
 	    -body "11111\n"
+	rxreq
+	expect req.url == "/bar"
+	txresp -hdr "Last-Modified: Thu, 26 Jun 2008 12:00:01 GMT" \
+	    -hdr {ETag: "bar"}
 } -start
 
 varnish v1 -vcl+backend { } -start
@@ -41,6 +45,21 @@ client c1 {
 	expect resp.http.etag == {"foo"}
 	expect resp.http.content-length == "<undef>"
 	expect resp.bodylen == "<undef>"
+
+	txreq -url "/bar"
+	rxresp
+	expect resp.status == 200
+	expect resp.http.etag == {"bar"}
+	expect resp.http.content-length == "0"
+	expect resp.bodylen == 0
+
+	txreq -url "/bar" \
+	    -hdr "If-Modified-Since: Thu, 26 Jun 2008 12:00:01 GMT"
+	rxresp -no_obj
+	expect resp.status == 304
+	expect resp.http.etag == {"bar"}
+	expect resp.http.content-length == <undef>
+	expect resp.bodylen == <undef>
 }
 
 client c1 -run
diff --git a/bin/varnishtest/tests/r01826.vtc b/bin/varnishtest/tests/r01826.vtc
index 6a553d5..d830d7b 100644
--- a/bin/varnishtest/tests/r01826.vtc
+++ b/bin/varnishtest/tests/r01826.vtc
@@ -2,6 +2,11 @@ varnishtest "Check we ignore a zero C-L with a 204"
 
 server s1 {
 	rxreq
+	txresp -status 204 -bodylen 5
+	expect_close
+
+	accept
+	rxreq
 	txresp -status 204
 } -start
 
@@ -11,5 +16,10 @@ varnish v1 -vcl+backend {
 client c1 {
 	txreq
 	rxresp
+	expect resp.status == 503
+
+	txreq
+	rxresp
 	expect resp.status == 204
+	expect resp.http.content-length == <undef>
 } -run



From lkarsten at varnish-software.com  Thu Jan 14 14:15:12 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:12 +0100
Subject: [4.1] ae4449e Add a couple of asserts for Coverity
Message-ID: <E1aJigC-0005Uh-4M@project.varnish-software.com>


commit ae4449ee60cca489d4641b40c083952e98c4a5ab
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Jan 8 09:37:15 2016 +0000

    Add a couple of asserts for Coverity

diff --git a/lib/libvarnish/vsub.c b/lib/libvarnish/vsub.c
index a7c658c..dd0cb9d 100644
--- a/lib/libvarnish/vsub.c
+++ b/lib/libvarnish/vsub.c
@@ -56,10 +56,14 @@ struct vsub_priv {
 void
 VSUB_closefrom(int fd)
 {
+
+	assert(fd >= 0);
+
 #ifdef HAVE_CLOSEFROM
 	closefrom(fd);
 #else
-	int i;
+	int i = sysconf(_SC_OPEN_MAX);
+	assert(i > 0);
 	for (i = sysconf(_SC_OPEN_MAX); i > fd; i--)
 		(void)close(i);
 #endif



From lkarsten at varnish-software.com  Thu Jan 14 14:15:12 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:12 +0100
Subject: [4.1] da727cb Reset storage_malloc counter on allocation failure.
Message-ID: <E1aJigC-0005Us-69@project.varnish-software.com>


commit da727cb9c4889a5575e7cce37d31deaadc99ed3b
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Jan 8 09:53:40 2016 +0000

    Reset storage_malloc counter on allocation failure.
    
    Submitted by:	github::onovy

diff --git a/bin/varnishd/storage/storage_malloc.c b/bin/varnishd/storage/storage_malloc.c
index 87b634f..23d7012 100644
--- a/bin/varnishd/storage/storage_malloc.c
+++ b/bin/varnishd/storage/storage_malloc.c
@@ -106,6 +106,7 @@ sma_alloc(const struct stevedore *st, size_t size)
 		 * XXX: Not want to pick up the lock twice just for stats.
 		 */
 		sma_sc->stats->c_fail++;
+		sma_sc->sma_alloc -= size;
 		sma_sc->stats->c_bytes -= size;
 		sma_sc->stats->g_alloc--;
 		sma_sc->stats->g_bytes -= size;



From lkarsten at varnish-software.com  Thu Jan 14 14:15:12 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:12 +0100
Subject: [4.1] 9a71465 director VMODs use a rwlock for mutexing,
 add_backend() uses a wrlock, all other methods use rdlock
Message-ID: <E1aJigC-0005V5-7r@project.varnish-software.com>


commit 9a7146509ed33cc1b54e031daddfe170ebd5f9ec
Author: Geoff Simmons <geoff at uplex.de>
Date:   Fri Jan 8 15:26:03 2016 +0100

    director VMODs use a rwlock for mutexing, add_backend() uses a wrlock,
    all other methods use rdlock

diff --git a/lib/libvmod_directors/fall_back.c b/lib/libvmod_directors/fall_back.c
index a47aa08..241a675 100644
--- a/lib/libvmod_directors/fall_back.c
+++ b/lib/libvmod_directors/fall_back.c
@@ -66,7 +66,7 @@ vmod_fallback_resolve(const struct director *dir, struct worker *wrk,
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
 	CAST_OBJ_NOTNULL(rr, dir->priv, VMOD_DIRECTORS_FALLBACK_MAGIC);
-	vdir_lock(rr->vd);
+	vdir_rdlock(rr->vd);
 	for (u = 0; u < rr->vd->n_backend; u++) {
 		be = rr->vd->backend[u];
 		CHECK_OBJ_NOTNULL(be, DIRECTOR_MAGIC);
diff --git a/lib/libvmod_directors/round_robin.c b/lib/libvmod_directors/round_robin.c
index 3690893..75ca9e4 100644
--- a/lib/libvmod_directors/round_robin.c
+++ b/lib/libvmod_directors/round_robin.c
@@ -67,7 +67,7 @@ vmod_rr_resolve(const struct director *dir, struct worker *wrk,
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
 	CAST_OBJ_NOTNULL(rr, dir->priv, VMOD_DIRECTORS_ROUND_ROBIN_MAGIC);
-	vdir_lock(rr->vd);
+	vdir_rdlock(rr->vd);
 	for (u = 0; u < rr->vd->n_backend; u++) {
 		rr->nxt %= rr->vd->n_backend;
 		be = rr->vd->backend[rr->nxt];
diff --git a/lib/libvmod_directors/vdir.c b/lib/libvmod_directors/vdir.c
index 1112bad..5f2968a 100644
--- a/lib/libvmod_directors/vdir.c
+++ b/lib/libvmod_directors/vdir.c
@@ -63,7 +63,7 @@ vdir_new(struct vdir **vdp, const char *name, const char *vcl_name,
 	ALLOC_OBJ(vd, VDIR_MAGIC);
 	AN(vd);
 	*vdp = vd;
-	AZ(pthread_mutex_init(&vd->mtx, NULL));
+	AZ(pthread_rwlock_init(&vd->mtx, NULL));
 
 	ALLOC_OBJ(vd->dir, DIRECTOR_MAGIC);
 	AN(vd->dir);
@@ -89,7 +89,7 @@ vdir_delete(struct vdir **vdp)
 
 	free(vd->backend);
 	free(vd->weight);
-	AZ(pthread_mutex_destroy(&vd->mtx));
+	AZ(pthread_rwlock_destroy(&vd->mtx));
 	free(vd->dir->vcl_name);
 	FREE_OBJ(vd->dir);
 	vbit_destroy(vd->vbm);
@@ -97,17 +97,24 @@ vdir_delete(struct vdir **vdp)
 }
 
 void
-vdir_lock(struct vdir *vd)
+vdir_rdlock(struct vdir *vd)
 {
 	CHECK_OBJ_NOTNULL(vd, VDIR_MAGIC);
-	AZ(pthread_mutex_lock(&vd->mtx));
+	AZ(pthread_rwlock_rdlock(&vd->mtx));
+}
+
+void
+vdir_wrlock(struct vdir *vd)
+{
+	CHECK_OBJ_NOTNULL(vd, VDIR_MAGIC);
+	AZ(pthread_rwlock_wrlock(&vd->mtx));
 }
 
 void
 vdir_unlock(struct vdir *vd)
 {
 	CHECK_OBJ_NOTNULL(vd, VDIR_MAGIC);
-	AZ(pthread_mutex_unlock(&vd->mtx));
+	AZ(pthread_rwlock_unlock(&vd->mtx));
 }
 
 
@@ -118,7 +125,7 @@ vdir_add_backend(struct vdir *vd, VCL_BACKEND be, double weight)
 
 	CHECK_OBJ_NOTNULL(vd, VDIR_MAGIC);
 	AN(be);
-	vdir_lock(vd);
+	vdir_wrlock(vd);
 	if (vd->n_backend >= vd->l_backend)
 		vdir_expand(vd, vd->l_backend + 16);
 	assert(vd->n_backend < vd->l_backend);
@@ -140,7 +147,7 @@ vdir_any_healthy(struct vdir *vd, const struct busyobj *bo, double *changed)
 
 	CHECK_OBJ_NOTNULL(vd, VDIR_MAGIC);
 	CHECK_OBJ_ORNULL(bo, BUSYOBJ_MAGIC);
-	vdir_lock(vd);
+	vdir_rdlock(vd);
 	if (changed != NULL)
 		*changed = 0;
 	for (u = 0; u < vd->n_backend; u++) {
@@ -183,7 +190,7 @@ vdir_pick_be(struct vdir *vd, double w, const struct busyobj *bo)
 	double tw = 0.0;
 	VCL_BACKEND be = NULL;
 
-	vdir_lock(vd);
+	vdir_rdlock(vd);
 	for (u = 0; u < vd->n_backend; u++) {
 		if (vd->backend[u]->healthy(vd->backend[u], bo, NULL)) {
 			vbit_clr(vd->vbm, u);
diff --git a/lib/libvmod_directors/vdir.h b/lib/libvmod_directors/vdir.h
index b420f68..3a43cc2 100644
--- a/lib/libvmod_directors/vdir.h
+++ b/lib/libvmod_directors/vdir.h
@@ -31,7 +31,7 @@ struct vbitmap;
 struct vdir {
 	unsigned				magic;
 #define VDIR_MAGIC				0x99f4b726
-	pthread_mutex_t				mtx;
+	pthread_rwlock_t			mtx;
 	unsigned				n_backend;
 	unsigned				l_backend;
 	VCL_BACKEND				*backend;
@@ -44,7 +44,8 @@ struct vdir {
 void vdir_new(struct vdir **vdp, const char *name, const char *vcl_name,
     vdi_healthy_f *healthy, vdi_resolve_f *resolve, void *priv);
 void vdir_delete(struct vdir **vdp);
-void vdir_lock(struct vdir *vd);
+void vdir_rdlock(struct vdir *vd);
+void vdir_wrlock(struct vdir *vd);
 void vdir_unlock(struct vdir *vd);
 unsigned vdir_add_backend(struct vdir *, VCL_BACKEND be, double weight);
 unsigned vdir_any_healthy(struct vdir *, const struct busyobj *,



From lkarsten at varnish-software.com  Thu Jan 14 14:15:12 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:12 +0100
Subject: [4.1] 0f5a64f add a remove_backend() method to each of the directors
 in the VMOD
Message-ID: <E1aJigC-0005VF-9j@project.varnish-software.com>


commit 0f5a64fa058c5664327c2ca3ded092a099c0005f
Author: Geoff Simmons <geoff at uplex.de>
Date:   Fri Jan 8 15:38:46 2016 +0100

    add a remove_backend() method to each of the directors in the VMOD

diff --git a/bin/varnishtest/tests/d00000.vtc b/bin/varnishtest/tests/d00000.vtc
index a88634f..2ec92ff 100644
--- a/bin/varnishtest/tests/d00000.vtc
+++ b/bin/varnishtest/tests/d00000.vtc
@@ -31,6 +31,15 @@ varnish v1 -vcl+backend {
 		rr.add_backend(s4);
 	}
 
+	sub vcl_recv {
+		if (req.method == "DELETE") {
+			rr.remove_backend(s1);
+			rr.remove_backend(s2);
+			rr.remove_backend(s3);
+			return(synth(204));
+		}
+	}
+
 	sub vcl_backend_fetch {
 		set bereq.backend = rr.backend();
 	}
@@ -64,3 +73,14 @@ client c2 {
 	rxresp
 	expect resp.bodylen == 2
 } -run
+
+server s4 -start
+
+client c3 {
+	txreq -req "DELETE"
+	rxresp
+	expect resp.status == 204
+	txreq -url "/foo31"
+	rxresp
+	expect resp.bodylen == 4
+} -run
diff --git a/bin/varnishtest/tests/d00001.vtc b/bin/varnishtest/tests/d00001.vtc
index c373b4d..2311b0a 100644
--- a/bin/varnishtest/tests/d00001.vtc
+++ b/bin/varnishtest/tests/d00001.vtc
@@ -26,6 +26,10 @@ varnish v1 -vcl+backend {
 	}
 
 	sub vcl_recv {
+		if (req.method == "DELETE") {
+			fb1.remove_backend(s2);
+			return(synth(204));
+		}
 		return (pass);
 	}
 
@@ -59,3 +63,15 @@ client c1 {
 	rxresp
 	expect resp.http.foo == "1"
 } -run
+
+varnish v1 -cliok "backend.set_health s1 sick"
+server s3 -start
+
+client c1 {
+	txreq -req "DELETE"
+	rxresp
+	expect resp.status == 204
+	txreq
+	rxresp
+	expect resp.http.foo == "3"
+} -run
diff --git a/bin/varnishtest/tests/d00002.vtc b/bin/varnishtest/tests/d00002.vtc
index b477176..0a0ba98 100644
--- a/bin/varnishtest/tests/d00002.vtc
+++ b/bin/varnishtest/tests/d00002.vtc
@@ -33,3 +33,46 @@ client c1 {
 	rxresp
 	expect resp.http.where == "foo-->s1"
 } -run
+
+server s1 -start
+server s2 {
+	loop 20 {
+		rxreq
+		txresp
+	}
+} -start
+
+varnish v1 -vcl+backend {
+	import ${vmod_directors};
+
+	sub vcl_init {
+		new foo = directors.random();
+		foo.add_backend(s1, 1);
+		foo.add_backend(s2, 1);
+	}
+
+	sub vcl_recv {
+		if (req.method == "DELETE") {
+			foo.remove_backend(s1);
+			return(synth(204));
+		}
+		set req.backend_hint = foo.backend();
+		return (pass);
+	}
+
+	sub vcl_backend_response {
+		set beresp.http.where = bereq.backend + "-->" + beresp.backend;
+	}
+}
+
+client c1 {
+	txreq -req "DELETE"
+	rxresp
+	expect resp.status == 204
+	loop 20 {
+		txreq
+		rxresp
+		expect resp.status == 200
+		expect resp.http.where == "foo-->s2"
+	}
+} -run
diff --git a/bin/varnishtest/tests/d00003.vtc b/bin/varnishtest/tests/d00003.vtc
index 56b3035..e6182d7 100644
--- a/bin/varnishtest/tests/d00003.vtc
+++ b/bin/varnishtest/tests/d00003.vtc
@@ -30,6 +30,10 @@ varnish v1 -vcl+backend {
 	}
 
 	sub vcl_recv {
+		if (req.method == "DELETE") {
+			h1.remove_backend(s1);
+			return(synth(204));
+		}
 		if (req.url == "/nohdr") {
 			set req.backend_hint = h1.backend(req.http.Void);
 		} else if (req.url == "/emptystring") {
@@ -74,3 +78,27 @@ client c1 {
 	rxresp
 	expect resp.http.foo == "9"
 } -run
+
+server s2 -start
+
+client c1 {
+	txreq -req "DELETE"
+	rxresp
+	expect resp.status == 204
+
+	txreq
+	rxresp
+	expect resp.http.foo == "2"
+
+	txreq
+	rxresp
+	expect resp.http.foo == "4"
+
+	txreq
+	rxresp
+	expect resp.http.foo == "6"
+
+	txreq
+	rxresp
+	expect resp.http.foo == "8"
+} -run
diff --git a/lib/libvmod_directors/fall_back.c b/lib/libvmod_directors/fall_back.c
index 241a675..a54807b 100644
--- a/lib/libvmod_directors/fall_back.c
+++ b/lib/libvmod_directors/fall_back.c
@@ -117,6 +117,15 @@ vmod_fallback_add_backend(VRT_CTX,
 	(void)vdir_add_backend(rr->vd, be, 0.0);
 }
 
+VCL_VOID __match_proto__()
+vmod_fallback_remove_backend(VRT_CTX,
+    struct vmod_directors_fallback *fb, VCL_BACKEND be)
+{
+	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
+	CHECK_OBJ_NOTNULL(fb, VMOD_DIRECTORS_FALLBACK_MAGIC);
+	(void)vdir_remove_backend(fb->vd, be);
+}
+
 VCL_BACKEND __match_proto__()
 vmod_fallback_backend(VRT_CTX,
     struct vmod_directors_fallback *rr)
diff --git a/lib/libvmod_directors/hash.c b/lib/libvmod_directors/hash.c
index 9aa62f0..db9626f 100644
--- a/lib/libvmod_directors/hash.c
+++ b/lib/libvmod_directors/hash.c
@@ -85,6 +85,16 @@ vmod_hash_add_backend(VRT_CTX,
 	(void)vdir_add_backend(rr->vd, be, w);
 }
 
+VCL_VOID __match_proto__()
+vmod_hash_remove_backend(VRT_CTX,
+    struct vmod_directors_hash *rr, VCL_BACKEND be)
+{
+
+	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
+	CHECK_OBJ_NOTNULL(rr, VMOD_DIRECTORS_HASH_MAGIC);
+	(void)vdir_remove_backend(rr->vd, be);
+}
+
 VCL_BACKEND __match_proto__()
 vmod_hash_backend(VRT_CTX, struct vmod_directors_hash *rr,
     const char *arg, ...)
diff --git a/lib/libvmod_directors/random.c b/lib/libvmod_directors/random.c
index 980ef94..d283f04 100644
--- a/lib/libvmod_directors/random.c
+++ b/lib/libvmod_directors/random.c
@@ -113,6 +113,14 @@ vmod_random_add_backend(VRT_CTX,
 	(void)vdir_add_backend(rr->vd, be, w);
 }
 
+VCL_VOID vmod_random_remove_backend(VRT_CTX,
+    struct vmod_directors_random *rr, VCL_BACKEND be)
+{
+	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
+	CHECK_OBJ_NOTNULL(rr, VMOD_DIRECTORS_RANDOM_MAGIC);
+	(void)vdir_remove_backend(rr->vd, be);
+}
+
 VCL_BACKEND __match_proto__()
 vmod_random_backend(VRT_CTX, struct vmod_directors_random *rr)
 {
diff --git a/lib/libvmod_directors/round_robin.c b/lib/libvmod_directors/round_robin.c
index 75ca9e4..3aee3ad 100644
--- a/lib/libvmod_directors/round_robin.c
+++ b/lib/libvmod_directors/round_robin.c
@@ -120,6 +120,15 @@ vmod_round_robin_add_backend(VRT_CTX,
 	(void)vdir_add_backend(rr->vd, be, 0.0);
 }
 
+VCL_VOID __match_proto__()
+vmod_round_robin_remove_backend(VRT_CTX,
+    struct vmod_directors_round_robin *rr, VCL_BACKEND be)
+{
+	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
+	CHECK_OBJ_NOTNULL(rr, VMOD_DIRECTORS_ROUND_ROBIN_MAGIC);
+	(void)vdir_remove_backend(rr->vd, be);
+}
+
 VCL_BACKEND __match_proto__()
 vmod_round_robin_backend(VRT_CTX,
     struct vmod_directors_round_robin *rr)
diff --git a/lib/libvmod_directors/vdir.c b/lib/libvmod_directors/vdir.c
index 5f2968a..ddbfbcc 100644
--- a/lib/libvmod_directors/vdir.c
+++ b/lib/libvmod_directors/vdir.c
@@ -138,6 +138,32 @@ vdir_add_backend(struct vdir *vd, VCL_BACKEND be, double weight)
 }
 
 unsigned
+vdir_remove_backend(struct vdir *vd, VCL_BACKEND be)
+{
+        unsigned u, n;
+
+        CHECK_OBJ_NOTNULL(vd, VDIR_MAGIC);
+        if (be == NULL)
+                return vd->n_backend;
+        CHECK_OBJ(be, DIRECTOR_MAGIC);
+        vdir_wrlock(vd);
+        for (u = 0; u < vd->n_backend; u++)
+                if (vd->backend[u] == be)
+                        break;
+        if (u == vd->n_backend) {
+                vdir_unlock(vd);
+                return vd->n_backend;
+        }
+        vd->total_weight -= vd->weight[u];
+        n = vd->n_backend - u - 1;
+        memmove(&vd->backend[u], &vd->backend[u+1], n * sizeof(vd->backend[0]));
+        memmove(&vd->weight[u], &vd->weight[u+1], n * sizeof(vd->weight[0]));
+        vd->n_backend--;
+        vdir_unlock(vd);
+        return vd->n_backend;
+}
+
+unsigned
 vdir_any_healthy(struct vdir *vd, const struct busyobj *bo, double *changed)
 {
 	unsigned retval = 0;
diff --git a/lib/libvmod_directors/vdir.h b/lib/libvmod_directors/vdir.h
index 3a43cc2..aaac9e5 100644
--- a/lib/libvmod_directors/vdir.h
+++ b/lib/libvmod_directors/vdir.h
@@ -48,6 +48,7 @@ void vdir_rdlock(struct vdir *vd);
 void vdir_wrlock(struct vdir *vd);
 void vdir_unlock(struct vdir *vd);
 unsigned vdir_add_backend(struct vdir *, VCL_BACKEND be, double weight);
+unsigned vdir_remove_backend(struct vdir *, VCL_BACKEND be);
 unsigned vdir_any_healthy(struct vdir *, const struct busyobj *,
     double *changed);
 VCL_BACKEND vdir_pick_be(struct vdir *, double w, const struct busyobj *);
diff --git a/lib/libvmod_directors/vmod.vcc b/lib/libvmod_directors/vmod.vcc
index 8522920..529db05 100644
--- a/lib/libvmod_directors/vmod.vcc
+++ b/lib/libvmod_directors/vmod.vcc
@@ -72,6 +72,14 @@ Example
 	vdir.add_backend(backend1);
 	vdir.add_backend(backend2);
 
+$Method VOID .remove_backend(BACKEND)
+
+Description
+	Remove a backend from the round-robin director.
+Example
+	vdir.remove_backend(backend1);
+	vdir.remove_backend(backend2);
+
 $Method BACKEND .backend()
 
 Description
@@ -103,6 +111,14 @@ Example
 	vdir.add_backend(backend1);
 	vdir.add_backend(backend2);
 
+$Method VOID .remove_backend(BACKEND)
+
+Description
+	Remove a backend from the director.
+Example
+	vdir.remove_backend(backend1);
+	vdir.remove_backend(backend2);
+
 $Method BACKEND .backend()
 
 Description
@@ -136,6 +152,13 @@ Example
 	vdir.add_backend(backend1, 10.0);
 	vdir.add_backend(backend2, 5.0);
 
+$Method VOID .remove_backend(BACKEND)
+
+Description
+	Remove a backend from the director.
+Example
+	vdir.remove_backend(backend1);
+	vdir.remove_backend(backend2);
 
 $Method BACKEND .backend()
 
@@ -170,6 +193,13 @@ Example
 	vdir.add_backend(backend1, 1.0);
 	vdir.add_backend(backend2, 1.0);
 
+$Method VOID .remove_backend(BACKEND)
+
+Description
+	Remove a backend from the director.
+Example
+	vdir.remove_backend(backend1);
+	vdir.remove_backend(backend2);
 
 $Method BACKEND .backend(STRING_LIST)
 



From lkarsten at varnish-software.com  Thu Jan 14 14:15:12 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:12 +0100
Subject: [4.1] e0577a0 White-space fix and minor flexelint-comforting
Message-ID: <E1aJigC-0005VP-BJ@project.varnish-software.com>


commit e0577a097ab5d901da0d2b7f33d7fd619dc5a7cf
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Jan 8 14:51:56 2016 +0000

    White-space fix and minor flexelint-comforting

diff --git a/lib/libvmod_directors/random.c b/lib/libvmod_directors/random.c
index d283f04..dee16ff 100644
--- a/lib/libvmod_directors/random.c
+++ b/lib/libvmod_directors/random.c
@@ -114,7 +114,7 @@ vmod_random_add_backend(VRT_CTX,
 }
 
 VCL_VOID vmod_random_remove_backend(VRT_CTX,
-    struct vmod_directors_random *rr, VCL_BACKEND be)
+    struct vmod_directors_random *rr, VCL_BACKEND be) __match_proto__()
 {
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
 	CHECK_OBJ_NOTNULL(rr, VMOD_DIRECTORS_RANDOM_MAGIC);
diff --git a/lib/libvmod_directors/vdir.c b/lib/libvmod_directors/vdir.c
index ddbfbcc..22dfee3 100644
--- a/lib/libvmod_directors/vdir.c
+++ b/lib/libvmod_directors/vdir.c
@@ -140,27 +140,27 @@ vdir_add_backend(struct vdir *vd, VCL_BACKEND be, double weight)
 unsigned
 vdir_remove_backend(struct vdir *vd, VCL_BACKEND be)
 {
-        unsigned u, n;
-
-        CHECK_OBJ_NOTNULL(vd, VDIR_MAGIC);
-        if (be == NULL)
-                return vd->n_backend;
-        CHECK_OBJ(be, DIRECTOR_MAGIC);
-        vdir_wrlock(vd);
-        for (u = 0; u < vd->n_backend; u++)
-                if (vd->backend[u] == be)
-                        break;
-        if (u == vd->n_backend) {
-                vdir_unlock(vd);
-                return vd->n_backend;
-        }
-        vd->total_weight -= vd->weight[u];
-        n = vd->n_backend - u - 1;
-        memmove(&vd->backend[u], &vd->backend[u+1], n * sizeof(vd->backend[0]));
-        memmove(&vd->weight[u], &vd->weight[u+1], n * sizeof(vd->weight[0]));
-        vd->n_backend--;
-        vdir_unlock(vd);
-        return vd->n_backend;
+	unsigned u, n;
+
+	CHECK_OBJ_NOTNULL(vd, VDIR_MAGIC);
+	if (be == NULL)
+		return (vd->n_backend);
+	CHECK_OBJ(be, DIRECTOR_MAGIC);
+	vdir_wrlock(vd);
+	for (u = 0; u < vd->n_backend; u++)
+		if (vd->backend[u] == be)
+			break;
+	if (u == vd->n_backend) {
+		vdir_unlock(vd);
+		return (vd->n_backend);
+	}
+	vd->total_weight -= vd->weight[u];
+	n = (vd->n_backend - u) - 1;
+	memmove(&vd->backend[u], &vd->backend[u+1], n * sizeof(vd->backend[0]));
+	memmove(&vd->weight[u], &vd->weight[u+1], n * sizeof(vd->weight[0]));
+	vd->n_backend--;
+	vdir_unlock(vd);
+	return (vd->n_backend);
 }
 
 unsigned



From lkarsten at varnish-software.com  Thu Jan 14 14:15:12 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:12 +0100
Subject: [4.1] e59560e Flexlinting
Message-ID: <E1aJigC-0005VY-Ca@project.varnish-software.com>


commit e59560ee9046afa3793e193413989e9793fe3568
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Jan 12 14:27:58 2016 +0000

    Flexlinting

diff --git a/bin/varnishd/storage/storage_file.c b/bin/varnishd/storage/storage_file.c
index 152d3b2..395e608 100644
--- a/bin/varnishd/storage/storage_file.c
+++ b/bin/varnishd/storage/storage_file.c
@@ -451,8 +451,8 @@ smf_alloc(const struct stevedore *st, size_t size)
 
 	CAST_OBJ_NOTNULL(sc, st->priv, SMF_SC_MAGIC);
 	assert(size > 0);
-	size += (sc->pagesize - 1);
-	size &= ~(sc->pagesize - 1);
+	size += (sc->pagesize - 1L);
+	size &= ~(sc->pagesize - 1L);
 	Lck_Lock(&sc->mtx);
 	sc->stats->c_req++;
 	smf = alloc_smf(sc, size);



From lkarsten at varnish-software.com  Thu Jan 14 14:15:12 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:12 +0100
Subject: [4.1] a3fcf45 First pass at using PARAM
Message-ID: <E1aJigC-0005Vi-Ep@project.varnish-software.com>


commit a3fcf45ece99ff3b27e984e12d4492ff7c5b5904
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Wed Jan 6 18:35:17 2016 +0000

    First pass at using PARAM

diff --git a/bin/varnishd/common/params.h b/bin/varnishd/common/params.h
index 13ab904..d5ac4f5 100644
--- a/bin/varnishd/common/params.h
+++ b/bin/varnishd/common/params.h
@@ -88,18 +88,6 @@ struct params {
 	uid_t			uid;
 	gid_t			gid;
 
-	/* TTL used for lack of anything better */
-	double			default_ttl;
-
-	/* Default grace period */
-	double			default_grace;
-
-	/* Default keep period */
-	double			default_keep;
-
-	/* Maximum concurrent sessions */
-	unsigned		max_sess;
-
 	/* Worker threads and pool */
 	unsigned		wthread_min;
 	unsigned		wthread_max;
@@ -112,97 +100,14 @@ struct params {
 	ssize_t			wthread_stacksize;
 	unsigned		wthread_queue_limit;
 
-	/* Memory allocation hints */
-	unsigned		workspace_backend;
-	unsigned		workspace_client;
-	unsigned		workspace_session;
-	unsigned		workspace_thread;
-
-	unsigned		vsl_buffer;
-
-	unsigned		http_req_size;
-	unsigned		http_req_hdr_len;
-	unsigned		http_resp_size;
-	unsigned		http_resp_hdr_len;
-	unsigned		http_max_hdr;
-
-	unsigned		vsl_reclen;
-
-	double			timeout_linger;
-	double			timeout_idle;
-	double			pipe_timeout;
-	double			send_timeout;
-	double			idle_send_timeout;
 #ifdef HAVE_TCP_KEEP
 	double			tcp_keepalive_time;
 	unsigned		tcp_keepalive_probes;
 	double			tcp_keepalive_intvl;
 #endif
 
-	/* Fetcher hints */
-	ssize_t			fetch_chunksize;
-	ssize_t			fetch_maxchunksize;
-	unsigned		nuke_limit;
-
-	/* Listen depth */
-	unsigned		listen_depth;
-
-	/* CLI related */
-	double			cli_timeout;
-	unsigned		cli_limit;
-	unsigned		ping_interval;
-
-	/* LRU list ordering interval */
-	double			lru_interval;
-
-	/* Maximum restarts allowed */
-	unsigned		max_restarts;
-
-	/* Maximum backend retriesallowed */
-	unsigned		max_retries;
-
-	/* Maximum esi:include depth allowed */
-	unsigned		max_esi_depth;
-
-	/* Rush exponent */
-	unsigned		rush_exponent;
-
-	/* Default connection_timeout */
-	double			connect_timeout;
-
-	/* CLI buffer size */
-	unsigned		cli_buffer;
-
-	/* Prefer IPv6 connections to backend*/
-	unsigned		prefer_ipv6;
-
-	/* Acceptable clockskew with backends */
-	unsigned		clock_skew;
-
-	unsigned		syslog_cli_traffic;
-
-	unsigned		http_range_support;
-
-	unsigned		http_gzip_support;
-	unsigned		gzip_buffer;
-	unsigned		gzip_level;
-	unsigned		gzip_memlevel;
-
-	double			critbit_cooloff;
-
-	double			vcl_cooldown;
-
-	double			shortlived;
-
 	struct vre_limits	vre_limits;
 
-	/* Install a SIGSEGV handler */
-	unsigned		sigsegv_handler;
-
-	/* VSM dimensions */
-	ssize_t			vsm_space;
-	ssize_t			vsl_space;
-
 	struct poolparam	req_pool;
 	struct poolparam	sess_pool;
 	struct poolparam	vbo_pool;
diff --git a/bin/varnishd/mgt/mgt_param_tbl.c b/bin/varnishd/mgt/mgt_param_tbl.c
index 3a6239b..16886b2 100644
--- a/bin/varnishd/mgt/mgt_param_tbl.c
+++ b/bin/varnishd/mgt/mgt_param_tbl.c
@@ -48,231 +48,6 @@ struct parspec mgt_parspec[] = {
 #include "tbl/params.h"
 #undef PARAM
 
-	{ "default_ttl", tweak_timeout, &mgt_param.default_ttl,
-		"0", NULL,
-		"The TTL assigned to objects if neither the backend nor "
-		"the VCL code assigns one.",
-		OBJ_STICKY,
-		"120", "seconds" },
-	{ "default_grace", tweak_timeout, &mgt_param.default_grace,
-		"0", NULL,
-		"Default grace period.  We will deliver an object "
-		"this long after it has expired, provided another thread "
-		"is attempting to get a new copy.",
-		OBJ_STICKY,
-		"10", "seconds" },
-	{ "default_keep", tweak_timeout, &mgt_param.default_keep,
-		"0", NULL,
-		"Default keep period.  We will keep a useless object "
-		"around this long, making it available for conditional "
-		"backend fetches.  "
-		"That means that the object will be removed from the "
-		"cache at the end of ttl+grace+keep.",
-		OBJ_STICKY,
-		"0", "seconds" },
-	{ "workspace_session",
-		tweak_bytes_u, &mgt_param.workspace_session,
-		"256", NULL,
-		"Allocation size for session structure and workspace.  "
-		"  The workspace is primarily used for TCP connection "
-		"addresses."
-		"  If larger than 4k, use a multiple of 4k for VM efficiency.",
-		DELAYED_EFFECT,
-		"512", "bytes" },
-	{ "workspace_client",
-		tweak_bytes_u, &mgt_param.workspace_client,
-		"9k", NULL,
-		"Bytes of HTTP protocol workspace for clients HTTP req/resp."
-		"  If larger than 4k, use a multiple of 4k for VM efficiency.",
-		DELAYED_EFFECT,
-		"64k", "bytes" },
-	{ "workspace_backend",
-		tweak_bytes_u, &mgt_param.workspace_backend,
-		"1024", NULL,
-		"Bytes of HTTP protocol workspace for backend HTTP req/resp."
-		"  If larger than 4k, use a multiple of 4k for VM efficiency.",
-		DELAYED_EFFECT,
-		"64k", "bytes" },
-	{ "workspace_thread",
-		tweak_bytes_u, &mgt_param.workspace_thread,
-		"256", "8192",
-		"Bytes of auxiliary workspace per thread.\n"
-		"This workspace is used for certain temporary data structures"
-		" during the operation of a worker thread.\n"
-		"One use is for the io-vectors for writing requests and"
-		" responses to sockets, having too little space will"
-		" result in more writev(2) system calls, having too much"
-		" just wastes the space.",
-		DELAYED_EFFECT,
-		"2048", "bytes" },
-	{ "http_req_hdr_len",
-		tweak_bytes_u, &mgt_param.http_req_hdr_len,
-		"40", NULL,
-		"Maximum length of any HTTP client request header we will "
-		"allow.  The limit is inclusive its continuation lines.",
-		0,
-		"8k", "bytes" },
-	{ "http_req_size",
-		tweak_bytes_u, &mgt_param.http_req_size,
-		"256", NULL,
-		"Maximum number of bytes of HTTP client request we will deal "
-		"with.  This is a limit on all bytes up to the double blank "
-		"line which ends the HTTP request.\n"
-		"The memory for the request is allocated from the client "
-		"workspace (param: workspace_client) and this parameter limits "
-		"how much of that the request is allowed to take up.",
-		0,
-		"32k", "bytes" },
-	{ "http_resp_hdr_len",
-		tweak_bytes_u, &mgt_param.http_resp_hdr_len,
-		"40", NULL,
-		"Maximum length of any HTTP backend response header we will "
-		"allow.  The limit is inclusive its continuation lines.",
-		0,
-		"8k", "bytes" },
-	{ "http_resp_size",
-		tweak_bytes_u, &mgt_param.http_resp_size,
-		"256", NULL,
-		"Maximum number of bytes of HTTP backend response we will deal "
-		"with.  This is a limit on all bytes up to the double blank "
-		"line which ends the HTTP request.\n"
-		"The memory for the request is allocated from the worker "
-		"workspace (param: thread_pool_workspace) and this parameter "
-		"limits how much of that the request is allowed to take up.",
-		0,
-		"32k", "bytes" },
-	{ "http_max_hdr", tweak_uint, &mgt_param.http_max_hdr,
-		"32", "65535",
-		"Maximum number of HTTP header lines we allow in "
-		"{req|resp|bereq|beresp}.http "
-		"(obj.http is autosized to the exact number of headers).\n"
-		"Cheap, ~20 bytes, in terms of workspace memory.\n"
-		"Note that the first line occupies five header lines.",
-		0,
-		"64", "header lines" },
-	{ "vsl_buffer",
-		tweak_vsl_buffer, &mgt_param.vsl_buffer,
-		"1024", NULL,
-		"Bytes of (req-/backend-)workspace dedicated to buffering"
-		" VSL records.\n"
-		"Setting this too high costs memory, setting it too low"
-		" will cause more VSL flushes and likely increase"
-		" lock-contention on the VSL mutex.\n\n"
-		"The minimum tracks the vsl_reclen parameter + 12 bytes.",
-		0,
-		"4k", "bytes" },
-	{ "vsl_reclen",
-		tweak_vsl_reclen, &mgt_param.vsl_reclen,
-		"16", "65535",
-		"Maximum number of bytes in SHM log record.\n\n"
-		"The maximum tracks the vsl_buffer parameter - 12 bytes.",
-		0,
-		"255", "bytes" },
-	{ "shm_reclen",
-		tweak_vsl_reclen, &mgt_param.vsl_reclen,
-		"16", "65535",
-		"Old name for vsl_reclen, use that instead.",
-		0,
-		"255", "bytes" },
-	{ "timeout_idle", tweak_timeout, &mgt_param.timeout_idle,
-		"0", NULL,
-		"Idle timeout for client connections.\n"
-		"A connection is considered idle, until we have "
-		"received the full request headers.",
-		0,
-		"5", "seconds" },
-	{ "pipe_timeout", tweak_timeout, &mgt_param.pipe_timeout,
-		"0", NULL,
-		"Idle timeout for PIPE sessions. "
-		"If nothing have been received in either direction for "
-		"this many seconds, the session is closed.",
-		0,
-		"60", "seconds" },
-	{ "send_timeout", tweak_timeout, &mgt_param.send_timeout,
-		"0", NULL,
-		"Send timeout for client connections. "
-		"If the HTTP response hasn't been transmitted in this many\n"
-		"seconds the session is closed.\n"
-		"See setsockopt(2) under SO_SNDTIMEO for more information.",
-		DELAYED_EFFECT,
-		"600", "seconds" },
-	{ "idle_send_timeout", tweak_timeout, &mgt_param.idle_send_timeout,
-		"0", NULL,
-		"Time to wait with no data sent. "
-		"If no data has been transmitted in this many\n"
-		"seconds the session is closed.\n"
-		"See setsockopt(2) under SO_SNDTIMEO for more information.",
-		DELAYED_EFFECT,
-		"60", "seconds" },
-	{ "nuke_limit",
-		tweak_uint, &mgt_param.nuke_limit,
-		"0", NULL,
-		"Maximum number of objects we attempt to nuke in order "
-		"to make space for a object body.",
-		EXPERIMENTAL,
-		"50", "allocations" },
-	{ "fetch_chunksize",
-		tweak_bytes, &mgt_param.fetch_chunksize,
-		"4096", NULL,
-		"The default chunksize used by fetcher. "
-		"This should be bigger than the majority of objects with "
-		"short TTLs.\n"
-		"Internal limits in the storage_file module makes increases "
-		"above 128kb a dubious idea.",
-		EXPERIMENTAL,
-		"16k", "bytes" },
-	{ "fetch_maxchunksize",
-		tweak_bytes, &mgt_param.fetch_maxchunksize,
-		"65536", NULL,
-		"The maximum chunksize we attempt to allocate from storage. "
-		"Making this too large may cause delays and storage "
-		"fragmentation.",
-		EXPERIMENTAL,
-		"256m", "bytes" },
-	{ "listen_depth", tweak_uint, &mgt_param.listen_depth,
-		"0", NULL,
-		"Listen queue depth.",
-		MUST_RESTART,
-		"1024", "connections" },
-	{ "cli_buffer",
-		tweak_bytes_u, &mgt_param.cli_buffer,
-		"4096", NULL,
-		"Size of buffer for CLI command input."
-		"\nYou may need to increase this if you have big VCL files "
-		"and use the vcl.inline CLI command.\n"
-		"NB: Must be specified with -p to have effect.",
-		0,
-		"8k", "bytes" },
-	{ "cli_limit",
-		tweak_bytes_u, &mgt_param.cli_limit,
-		"128", "99999999",
-		"Maximum size of CLI response.  If the response exceeds"
-		" this limit, the response code will be 201 instead of"
-		" 200 and the last line will indicate the truncation.",
-		0,
-		"48k", "bytes" },
-	{ "cli_timeout", tweak_timeout, &mgt_param.cli_timeout,
-		"0", NULL,
-		"Timeout for the childs replies to CLI requests from "
-		"the mgt_param.",
-		0,
-		"60", "seconds" },
-	{ "ping_interval", tweak_uint, &mgt_param.ping_interval,
-		"0", NULL,
-		"Interval between pings from parent to child.\n"
-		"Zero will disable pinging entirely, which makes "
-		"it possible to attach a debugger to the child.",
-		MUST_RESTART,
-		"3", "seconds" },
-	{ "lru_interval", tweak_timeout, &mgt_param.lru_interval,
-		"0", NULL,
-		"Grace period before object moves on LRU list.\n"
-		"Objects are only moved to the front of the LRU "
-		"list if they have not been moved there already inside "
-		"this timeout period.  This reduces the amount of lock "
-		"operations necessary for LRU list access.",
-		EXPERIMENTAL,
-		"2", "seconds" },
 	{ "cc_command", tweak_string, &mgt_cc_cmd,
 		NULL, NULL,
 		"Command used for compiling the C source code to a "
@@ -281,134 +56,6 @@ struct parspec mgt_parspec[] = {
 		"and %o will be replaced with the output file name.",
 		MUST_RELOAD,
 		VCC_CC , NULL },
-	{ "max_restarts", tweak_uint, &mgt_param.max_restarts,
-		"0", NULL,
-		"Upper limit on how many times a request can restart."
-		"\nBe aware that restarts are likely to cause a hit against "
-		"the backend, so don't increase thoughtlessly.",
-		0,
-		"4", "restarts" },
-	{ "max_retries", tweak_uint, &mgt_param.max_retries,
-		"0", NULL,
-		"Upper limit on how many times a backend fetch can retry.",
-		0,
-		"4", "retries" },
-	{ "max_esi_depth", tweak_uint, &mgt_param.max_esi_depth,
-		"0", NULL,
-		"Maximum depth of esi:include processing.",
-		0,
-		"5", "levels" },
-	{ "connect_timeout", tweak_timeout, &mgt_param.connect_timeout,
-		"0", NULL,
-		"Default connection timeout for backend connections. "
-		"We only try to connect to the backend for this many "
-		"seconds before giving up. "
-		"VCL can override this default value for each backend and "
-		"backend request.",
-		0,
-		"3.5", "seconds" },
-	{ "clock_skew", tweak_uint, &mgt_param.clock_skew,
-		"0", NULL,
-		"How much clockskew we are willing to accept between the "
-		"backend and our own clock.",
-		0,
-		"10", "seconds" },
-	{ "prefer_ipv6", tweak_bool, &mgt_param.prefer_ipv6,
-		NULL, NULL,
-		"Prefer IPv6 address when connecting to backends which "
-		"have both IPv4 and IPv6 addresses.",
-		0,
-		"off", "bool" },
-	{ "session_max", tweak_uint,
-		&mgt_param.max_sess,
-		"1000", NULL,
-		"Maximum number of sessions we will allocate from one pool "
-		"before just dropping connections.\n"
-		"This is mostly an anti-DoS measure, and setting it plenty "
-		"high should not hurt, as long as you have the memory for "
-		"it.",
-		0,
-		"100000", "sessions" },
-	{ "timeout_linger", tweak_timeout, &mgt_param.timeout_linger,
-		"0", NULL,
-		"How long the worker thread lingers on an idle session "
-		"before handing it over to the waiter.\n"
-		"When sessions are reused, as much as half of all reuses "
-		"happen within the first 100 msec of the previous request "
-		"completing.\n"
-		"Setting this too high results in worker threads not doing "
-		"anything for their keep, setting it too low just means that "
-		"more sessions take a detour around the waiter.",
-		EXPERIMENTAL,
-		"0.050", "seconds" },
-	{ "syslog_cli_traffic", tweak_bool, &mgt_param.syslog_cli_traffic,
-		NULL, NULL,
-		"Log all CLI traffic to syslog(LOG_INFO).",
-		0,
-		"on", "bool" },
-	{ "http_range_support", tweak_bool, &mgt_param.http_range_support,
-		NULL, NULL,
-		"Enable support for HTTP Range headers.",
-		0,
-		"on", "bool" },
-	{ "http_gzip_support", tweak_bool, &mgt_param.http_gzip_support,
-		NULL, NULL,
-		"Enable gzip support. When enabled Varnish request compressed "
-		"objects from the backend and store them compressed. "
-		"If a client does not support gzip encoding Varnish will "
-		"uncompress compressed objects on demand. Varnish will also "
-		"rewrite the Accept-Encoding header of clients indicating "
-		"support for gzip to:\n"
-		"  Accept-Encoding: gzip\n\n"
-		"Clients that do not support gzip will have their "
-		"Accept-Encoding header removed. For more information on how "
-		"gzip is implemented please see the chapter on gzip in the "
-		"Varnish reference.",
-		0,
-		"on", "bool" },
-	{ "gzip_level", tweak_uint, &mgt_param.gzip_level,
-		"0", "9",
-		"Gzip compression level: 0=debug, 1=fast, 9=best",
-		0,
-		"6", ""},
-	{ "gzip_memlevel", tweak_uint, &mgt_param.gzip_memlevel,
-		"1", "9",
-		"Gzip memory level 1=slow/least, 9=fast/most compression.\n"
-		"Memory impact is 1=1k, 2=2k, ... 9=256k.",
-		0,
-		"8", ""},
-	{ "gzip_buffer",
-		tweak_bytes_u, &mgt_param.gzip_buffer,
-		"2048", NULL,
-		"Size of malloc buffer used for gzip processing.\n"
-		"These buffers are used for in-transit data,"
-		" for instance gunzip'ed data being sent to a client."
-		"Making this space to small results in more overhead,"
-		" writes to sockets etc, making it too big is probably"
-		" just a waste of memory.",
-		EXPERIMENTAL,
-		"32k", "bytes" },
-	{ "shortlived", tweak_timeout,
-		&mgt_param.shortlived,
-		"0", NULL,
-		"Objects created with (ttl+grace+keep) shorter than this"
-		" are always put in transient storage.",
-		0,
-		"10", "seconds" },
-	{ "critbit_cooloff", tweak_timeout,
-		&mgt_param.critbit_cooloff,
-		"60", "254",
-		"How long the critbit hasher keeps deleted objheads "
-		"on the cooloff list.",
-		WIZARD,
-		"180", "seconds" },
-	{ "sigsegv_handler", tweak_bool, &mgt_param.sigsegv_handler,
-		NULL, NULL,
-		"Install a signal handler which tries to dump debug"
-		" information on segmentation faults, bus errors and abort"
-		" signals.",
-		MUST_RESTART,
-		"on", "bool" },
 	{ "vcl_dir", tweak_string, &mgt_vcl_dir,
 		NULL, NULL,
 		"Directory from which relative VCL filenames (vcl.load and "
@@ -422,31 +69,22 @@ struct parspec mgt_parspec[] = {
 		0,
 		VARNISH_VMOD_DIR,
 		NULL },
-	{ "vcl_cooldown", tweak_timeout, &mgt_param.vcl_cooldown,
-		"0", NULL,
-		"How long a VCL is kept warm after being replaced as the"
-		" active VCL (granularity approximately 30 seconds).",
-		0,
-		"600", "seconds" },
 	{ "vcc_err_unref", tweak_bool, &mgt_vcc_err_unref,
 		NULL, NULL,
 		"Unreferenced VCL objects result in error.",
 		0,
 		"on", "bool" },
-
 	{ "vcc_allow_inline_c", tweak_bool, &mgt_vcc_allow_inline_c,
 		NULL, NULL,
 		"Allow inline C code in VCL.",
 		0,
 		"off", "bool" },
-
 	{ "vcc_unsafe_path", tweak_bool, &mgt_vcc_unsafe_path,
 		NULL, NULL,
 		"Allow '/' in vmod & include paths.\n"
 		"Allow 'import ... from ...'.",
 		0,
 		"on", "bool" },
-
 	{ "pcre_match_limit", tweak_uint,
 		&mgt_param.vre_limits.match,
 		"1", NULL,
@@ -473,28 +111,6 @@ struct parspec mgt_parspec[] = {
 		"Testcase r01576 can be useful when tuning this parameter.",
 		0,
 		"20", ""},
-	{ "vsl_space", tweak_bytes,
-		&mgt_param.vsl_space,
-		"1M", NULL,
-		"The amount of space to allocate for the VSL fifo buffer"
-		" in the VSM memory segment."
-		"  If you make this too small, varnish{ncsa|log} etc will"
-		" not be able to keep up."
-		"  Making it too large just costs memory resources.",
-		MUST_RESTART,
-		"80M", "bytes"},
-
-	{ "vsm_space", tweak_bytes,
-		&mgt_param.vsm_space,
-		"1M", NULL,
-		"The amount of space to allocate for stats counters"
-		" in the VSM memory segment."
-		"  If you make this too small, some counters will be"
-		" invisible."
-		"  Making it too large just costs memory resources.",
-		MUST_RESTART,
-		"1M", "bytes"},
-
 	{ "pool_req", tweak_poolparam, &mgt_param.req_pool,
 		NULL, NULL,
 		"Parameters for per worker pool request memory pool.\n"
diff --git a/bin/varnishd/mgt/mgt_pool.c b/bin/varnishd/mgt/mgt_pool.c
index c08a2af..d14a32f 100644
--- a/bin/varnishd/mgt/mgt_pool.c
+++ b/bin/varnishd/mgt/mgt_pool.c
@@ -191,15 +191,6 @@ struct parspec WRK_parspec[] = {
 		"be dropped instead of queued.",
 		EXPERIMENTAL,
 		"20", "" },
-	{ "rush_exponent", tweak_uint, &mgt_param.rush_exponent,
-		"2", NULL,
-		"How many parked request we start for each completed "
-		"request on the object.\n"
-		"NB: Even with the implict delay of delivery, "
-		"this parameter controls an exponential increase in "
-		"number of worker threads.",
-		EXPERIMENTAL,
-		"3", "requests per request" },
 	{ "thread_pool_stack",
 		tweak_bytes, &mgt_param.wthread_stacksize,
 		NULL, NULL,
diff --git a/include/tbl/params.h b/include/tbl/params.h
index f372351..388c1ee 100644
--- a/include/tbl/params.h
+++ b/include/tbl/params.h
@@ -226,9 +226,6 @@ PARAM(
 	/* func */	NULL
 )
 
-/**********************************************************************/
-#if 0 /* NOT YET */
-
 PARAM(
 	/* name */	cli_buffer,
 	/* typ */	bytes_u,
@@ -245,6 +242,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	cli_limit,
 	/* typ */	bytes_u,
@@ -260,6 +258,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	cli_timeout,
 	/* typ */	timeout,
@@ -274,6 +273,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	clock_skew,
 	/* typ */	uint,
@@ -288,6 +288,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	connect_timeout,
 	/* typ */	timeout,
@@ -304,6 +305,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	critbit_cooloff,
 	/* typ */	timeout,
@@ -318,6 +320,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 #if 0
 PARAM(
 	/* name */	debug,
@@ -347,6 +350,7 @@ PARAM(
 	/* func */	NULL
 )
 #endif
+
 PARAM(
 	/* name */	default_grace,
 	/* typ */	timeout,
@@ -362,6 +366,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	default_keep,
 	/* typ */	timeout,
@@ -378,6 +383,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	default_ttl,
 	/* typ */	timeout,
@@ -392,6 +398,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 #if 0
 PARAM(
 	/* name */	feature,
@@ -417,6 +424,7 @@ PARAM(
 	/* func */	NULL
 )
 #endif
+
 PARAM(
 	/* name */	fetch_chunksize,
 	/* typ */	bytes,
@@ -433,6 +441,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	fetch_maxchunksize,
 	/* typ */	bytes,
@@ -447,6 +456,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	gzip_buffer,
 	/* typ */	bytes_u,
@@ -464,6 +474,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	gzip_level,
 	/* typ */	uint,
@@ -477,6 +488,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	gzip_memlevel,
 	/* typ */	uint,
@@ -491,6 +503,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	http_gzip_support,
 	/* typ */	bool,
@@ -513,6 +526,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	http_max_hdr,
 	/* typ */	uint,
@@ -530,6 +544,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	http_range_support,
 	/* typ */	bool,
@@ -543,6 +558,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	http_req_hdr_len,
 	/* typ */	bytes_u,
@@ -557,6 +573,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	http_req_size,
 	/* typ */	bytes_u,
@@ -575,6 +592,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	http_resp_hdr_len,
 	/* typ */	bytes_u,
@@ -589,6 +607,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	http_resp_size,
 	/* typ */	bytes_u,
@@ -607,6 +626,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	idle_send_timeout,
 	/* typ */	timeout,
@@ -623,6 +643,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	listen_depth,
 	/* typ */	uint,
@@ -636,6 +657,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	lru_interval,
 	/* typ */	timeout,
@@ -653,6 +675,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	max_esi_depth,
 	/* typ */	uint,
@@ -666,6 +689,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	max_restarts,
 	/* typ */	uint,
@@ -681,6 +705,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	max_retries,
 	/* typ */	uint,
@@ -694,6 +719,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	nuke_limit,
 	/* typ */	uint,
@@ -708,6 +734,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 #if 0
 PARAM(
 	/* name */	pcre_match_limit,
@@ -723,6 +750,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	pcre_match_limit_recursion,
 	/* typ */	uint,
@@ -738,6 +766,7 @@ PARAM(
 	/* func */	NULL
 )
 #endif
+
 PARAM(
 	/* name */	ping_interval,
 	/* typ */	uint,
@@ -753,6 +782,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	pipe_timeout,
 	/* typ */	timeout,
@@ -767,6 +797,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 #if 0
 PARAM(
 	/* name */	pool_req,
@@ -782,6 +813,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	pool_sess,
 	/* typ */	poolparam,
@@ -796,6 +828,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	pool_vbo,
 	/* typ */	poolparam,
@@ -811,6 +844,7 @@ PARAM(
 	/* func */	NULL
 )
 #endif
+
 PARAM(
 	/* name */	prefer_ipv6,
 	/* typ */	bool,
@@ -825,6 +859,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	rush_exponent,
 	/* typ */	uint,
@@ -841,6 +876,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	send_timeout,
 	/* typ */	timeout,
@@ -857,13 +893,13 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
-#if 0
+
 PARAM(
 	/* name */	session_max,
 	/* typ */	uint,
 	/* min */	"1000",
 	/* max */	NULL,
-	/* default */	"10.000",
+	/* default */	"100000",
 	/* units */	"sessions",
 	/* flags */	0,
 	/* s-text */
@@ -888,7 +924,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
-#endif
+
 PARAM(
 	/* name */	shortlived,
 	/* typ */	timeout,
@@ -903,6 +939,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	sigsegv_handler,
 	/* typ */	bool,
@@ -917,6 +954,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	syslog_cli_traffic,
 	/* typ */	bool,
@@ -930,6 +968,8 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
+#if 0
 PARAM(
 	/* name */	tcp_keepalive_intvl,
 	/* typ */	timeout,
@@ -943,6 +983,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	tcp_keepalive_probes,
 	/* typ */	uint,
@@ -958,6 +999,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	tcp_keepalive_time,
 	/* typ */	timeout,
@@ -972,7 +1014,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
-#if 0
+
 PARAM(
 	/* name */	thread_pool_add_delay,
 	/* typ */	timeout,
@@ -992,6 +1034,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	thread_pool_destroy_delay,
 	/* typ */	timeout,
@@ -1006,6 +1049,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	thread_pool_fail_delay,
 	/* typ */	timeout,
@@ -1031,6 +1075,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	thread_pool_max,
 	/* typ */	thread_pool_max,
@@ -1048,6 +1093,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	thread_pool_min,
 	/* typ */	thread_pool_min,
@@ -1065,6 +1111,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	thread_pool_stack,
 	/* typ */	bytes,
@@ -1080,6 +1127,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	thread_pool_timeout,
 	/* typ */	timeout,
@@ -1096,6 +1144,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	thread_pools,
 	/* typ */	uint,
@@ -1117,6 +1166,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	thread_queue_limit,
 	/* typ */	uint,
@@ -1134,6 +1184,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	thread_stats_rate,
 	/* typ */	uint,
@@ -1145,7 +1196,7 @@ PARAM(
 	/* s-text */
 	"Worker threads accumulate statistics, and dump these into the "
 	"global stats counters if the lock is free when they finish a job "
-	"(request/fetch etc.)\n"
+	"(request/fetch etc).\n"
 	"This parameters defines the maximum number of jobs a worker "
 	"thread may handle, before it is forced to dump its accumulated "
 	"stats into the global counters.",
@@ -1153,6 +1204,7 @@ PARAM(
 	/* func */	NULL
 )
 #endif
+
 PARAM(
 	/* name */	timeout_idle,
 	/* typ */	timeout,
@@ -1168,6 +1220,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	timeout_linger,
 	/* typ */	timeout,
@@ -1187,6 +1240,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 #if 0
 PARAM(
 	/* name */	vcc_allow_inline_c,
@@ -1201,8 +1255,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
-#endif
-#if 0
+
 PARAM(
 	/* name */	vcc_err_unref,
 	/* typ */	bool,
@@ -1216,8 +1269,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
-#endif
-#if 0
+
 PARAM(
 	/* name */	vcc_unsafe_path,
 	/* typ */	bool,
@@ -1233,6 +1285,7 @@ PARAM(
 	/* func */	NULL
 )
 #endif
+
 PARAM(
 	/* name */	vcl_cooldown,
 	/* typ */	timeout,
@@ -1242,11 +1295,12 @@ PARAM(
 	/* units */	"seconds",
 	/* flags */	0,
 	/* s-text */
-	"How long time a VCL is kept warm after being replaced as the "
-	"active VCL.  (Granularity approximately 30 seconds.)",
+	"How long a VCL is kept warm after being replaced as the "
+	"active VCL (granularity approximately 30 seconds).",
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 #if 0
 PARAM(
 	/* name */	vcl_dir,
@@ -1262,6 +1316,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	vmod_dir,
 	/* typ */	string,
@@ -1276,6 +1331,7 @@ PARAM(
 	/* func */	NULL
 )
 #endif
+
 PARAM(
 	/* name */	vsl_buffer,
 	/* typ */	vsl_buffer,
@@ -1289,11 +1345,12 @@ PARAM(
 	"records.\n"
 	"Setting this too high costs memory, setting it too low will cause "
 	"more VSL flushes and likely increase lock-contention on the VSL "
-	"mutex.\n"
+	"mutex.\n\n"
 	"The minimum tracks the vsl_reclen parameter + 12 bytes.",
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 #if 0
 PARAM(
 	/* name */	vsl_mask,
@@ -1307,12 +1364,13 @@ PARAM(
 	"Mask individual VSL messages from being logged.\n"
 	"	default	Set default value\n"
 	"\n"
-	"Use +/- prefixe in front of VSL tag name, to mask/unmask "
+	"Use +/- prefix in front of VSL tag name, to mask/unmask "
 	"individual VSL messages.",
 	/* l-text */	"",
 	/* func */	NULL
 )
 #endif
+
 PARAM(
 	/* name */	vsl_reclen,
 	/* typ */	vsl_reclen,
@@ -1322,11 +1380,12 @@ PARAM(
 	/* units */	"bytes",
 	/* flags */	0,
 	/* s-text */
-	"Maximum number of bytes in SHM log record.\n"
+	"Maximum number of bytes in SHM log record.\n\n"
 	"The maximum tracks the vsl_buffer parameter - 12 bytes.",
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	vsl_space,
 	/* typ */	bytes,
@@ -1343,6 +1402,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	vsm_space,
 	/* typ */	bytes,
@@ -1358,6 +1418,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 #if 0
 PARAM(
 	/* name */	waiter,
@@ -1373,6 +1434,7 @@ PARAM(
 	/* func */	NULL
 )
 #endif
+
 PARAM(
 	/* name */	workspace_backend,
 	/* typ */	bytes_u,
@@ -1387,6 +1449,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	workspace_client,
 	/* typ */	bytes_u,
@@ -1401,6 +1464,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	workspace_session,
 	/* typ */	bytes_u,
@@ -1416,6 +1480,7 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
+
 PARAM(
 	/* name */	workspace_thread,
 	/* typ */	bytes_u,
@@ -1434,5 +1499,5 @@ PARAM(
 	/* l-text */	"",
 	/* func */	NULL
 )
-#endif
+
 /*lint -restore */



From lkarsten at varnish-software.com  Thu Jan 14 14:15:12 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:12 +0100
Subject: [4.1] 7c73f98 Split the parameters for the VCC compiler into a
 separate structure for more clarity, before we start hacking VCC further.
Message-ID: <E1aJigC-0005Vs-Gb@project.varnish-software.com>


commit 7c73f98ad1062b85dc11ea6483aae248f52fbc4f
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Jan 8 10:32:18 2016 +0000

    Split the parameters for the VCC compiler into a separate structure
    for more clarity, before we start hacking VCC further.

diff --git a/bin/varnishd/mgt/mgt_vcc.c b/bin/varnishd/mgt/mgt_vcc.c
index d13432e..9d28804 100644
--- a/bin/varnishd/mgt/mgt_vcc.c
+++ b/bin/varnishd/mgt/mgt_vcc.c
@@ -64,7 +64,7 @@ unsigned mgt_vcc_err_unref;
 unsigned mgt_vcc_allow_inline_c;
 unsigned mgt_vcc_unsafe_path;
 
-static struct vcc *vcc;
+static struct vcp *vcp;
 
 #define VGC_SRC		"vgc.c"
 #define VGC_LIB		"vgc.so"
@@ -94,12 +94,12 @@ run_vcc(void *priv)
 
 	sb = VSB_new_auto();
 	XXXAN(sb);
-	VCC_VCL_dir(vcc, mgt_vcl_dir);
-	VCC_VMOD_dir(vcc, mgt_vmod_dir);
-	VCC_Err_Unref(vcc, mgt_vcc_err_unref);
-	VCC_Allow_InlineC(vcc, mgt_vcc_allow_inline_c);
-	VCC_Unsafe_Path(vcc, mgt_vcc_unsafe_path);
-	csrc = VCC_Compile(vcc, sb, vp->src);
+	VCP_VCL_dir(vcp, mgt_vcl_dir);
+	VCP_VMOD_dir(vcp, mgt_vmod_dir);
+	VCP_Err_Unref(vcp, mgt_vcc_err_unref);
+	VCP_Allow_InlineC(vcp, mgt_vcc_allow_inline_c);
+	VCP_Unsafe_Path(vcp, mgt_vcc_unsafe_path);
+	csrc = VCC_Compile(vcp, sb, vp->src);
 	AZ(VSB_finish(sb));
 	if (VSB_len(sb))
 		printf("%s", VSB_data(sb));
@@ -318,7 +318,7 @@ void
 mgt_vcc_init(void)
 {
 
-	vcc = VCC_New();
-	AN(vcc);
-	VCC_Builtin_VCL(vcc, builtin_vcl);
+	vcp = VCP_New();
+	AN(vcp);
+	VCP_Builtin_VCL(vcp, builtin_vcl);
 }
diff --git a/include/libvcc.h b/include/libvcc.h
index 6b16252..696123e 100644
--- a/include/libvcc.h
+++ b/include/libvcc.h
@@ -30,12 +30,12 @@
 
 struct vcc;
 
-struct vcc *VCC_New(void);
-void VCC_Builtin_VCL(struct vcc *, const char *str);
-void VCC_VCL_dir(struct vcc *, const char *str);
-void VCC_VMOD_dir(struct vcc *, const char *str);
-void VCC_Err_Unref(struct vcc *tl, unsigned u);
-void VCC_Allow_InlineC(struct vcc *tl, unsigned u);
-void VCC_Unsafe_Path(struct vcc *tl, unsigned u);
+struct vcp *VCP_New(void);
+void VCP_Builtin_VCL(struct vcp *, const char *str);
+void VCP_VCL_dir(struct vcp *, const char *str);
+void VCP_VMOD_dir(struct vcp *, const char *str);
+void VCP_Err_Unref(struct vcp *tl, unsigned u);
+void VCP_Allow_InlineC(struct vcp *tl, unsigned u);
+void VCP_Unsafe_Path(struct vcp *tl, unsigned u);
 
-char *VCC_Compile(const struct vcc *, struct vsb *sb, const char *b);
+char *VCC_Compile(const struct vcp *, struct vsb *sb, const char *b);
diff --git a/lib/libvcc/vcc_acl.c b/lib/libvcc/vcc_acl.c
index b5ee224..78d9ec5 100644
--- a/lib/libvcc/vcc_acl.c
+++ b/lib/libvcc/vcc_acl.c
@@ -364,7 +364,7 @@ vcc_acl_emit(struct vcc *tl, const char *acln, int anon)
 	Fh(tl, 0, "\t\tVRT_acl_log(ctx, \"NO_FAM %s\");\n", acln);
 	Fh(tl, 0, "\t\treturn(0);\n");
 	Fh(tl, 0, "\t}\n\n");
-	if (!tl->err_unref && !anon ) {
+	if (!tl->param->err_unref && !anon ) {
 		ifp = New_IniFin(tl);
 		VSB_printf(ifp->ini,
 			"\tif (0) match_acl_named_%s(0, 0);\n", acln);
diff --git a/lib/libvcc/vcc_compile.c b/lib/libvcc/vcc_compile.c
index e79597f..aaba4bf 100644
--- a/lib/libvcc/vcc_compile.c
+++ b/lib/libvcc/vcc_compile.c
@@ -450,16 +450,16 @@ vcc_destroy_source(struct source *sp)
 /*--------------------------------------------------------------------*/
 
 static struct source *
-vcc_file_source(const struct vcc *tl, struct vsb *sb, const char *fn)
+vcc_file_source(const struct vcp * const vcp, struct vsb *sb, const char *fn)
 {
 	char *f;
 	struct source *sp;
 
-	if (!tl->unsafe_path && strchr(fn, '/') != NULL) {
+	if (!vcp->unsafe_path && strchr(fn, '/') != NULL) {
 		VSB_printf(sb, "Include path is unsafe '%s'\n", fn);
 		return (NULL);
 	}
-	f = VFIL_readfile(tl->vcl_dir, fn, NULL);
+	f = VFIL_readfile(vcp->vcl_dir, fn, NULL);
 	if (f == NULL) {
 		VSB_printf(sb, "Cannot read file '%s': %s\n",
 		    fn, strerror(errno));
@@ -500,7 +500,7 @@ vcc_resolve_includes(struct vcc *tl)
 			return;
 		}
 
-		sp = vcc_file_source(tl, tl->sb, t1->dec);
+		sp = vcc_file_source(tl->param, tl->sb, t1->dec);
 		if (sp == NULL) {
 			vcc_ErrWhere(tl, t1);
 			return;
@@ -522,24 +522,16 @@ vcc_resolve_includes(struct vcc *tl)
 /*--------------------------------------------------------------------*/
 
 static struct vcc *
-vcc_NewVcc(const struct vcc *tl0)
+vcc_NewVcc(const struct vcp *vcp)
 {
 	struct vcc *tl;
 	int i;
 
+	CHECK_OBJ_NOTNULL(vcp, VCP_MAGIC);
 	ALLOC_OBJ(tl, VCC_MAGIC);
 	AN(tl);
-	if (tl0 != NULL) {
-		REPLACE(tl->builtin_vcl, tl0->builtin_vcl);
-		REPLACE(tl->vcl_dir, tl0->vcl_dir);
-		REPLACE(tl->vmod_dir, tl0->vmod_dir);
-		tl->vars = tl0->vars;
-		tl->err_unref = tl0->err_unref;
-		tl->allow_inline_c = tl0->allow_inline_c;
-		tl->unsafe_path = tl0->unsafe_path;
-	} else {
-		tl->err_unref = 1;
-	}
+	tl->param = vcp;
+	tl->vars = vcc_vars;
 	VTAILQ_INIT(&tl->symbols);
 	VTAILQ_INIT(&tl->inifin);
 	VTAILQ_INIT(&tl->membits);
@@ -606,7 +598,8 @@ vcc_DestroyTokenList(struct vcc *tl, char *ret)
  */
 
 static char *
-vcc_CompileSource(const struct vcc *tl0, struct vsb *sb, struct source *sp)
+vcc_CompileSource(const struct vcp * const vcp, struct vsb *sb,
+    struct source *sp)
 {
 	struct vcc *tl;
 	struct symbol *sym;
@@ -616,7 +609,7 @@ vcc_CompileSource(const struct vcc *tl0, struct vsb *sb, struct source *sp)
 	char *of;
 	int i;
 
-	tl = vcc_NewVcc(tl0);
+	tl = vcc_NewVcc(vcp);
 	tl->sb = sb;
 
 	vcc_Expr_Init(tl);
@@ -649,7 +642,7 @@ vcc_CompileSource(const struct vcc *tl0, struct vsb *sb, struct source *sp)
 		return (vcc_DestroyTokenList(tl, NULL));
 
 	/* Register and lex the builtin VCL */
-	sp = vcc_new_source(tl->builtin_vcl, NULL, "Builtin");
+	sp = vcc_new_source(tl->param->builtin_vcl, NULL, "Builtin");
 	assert(sp != NULL);
 	VTAILQ_INSERT_TAIL(&tl->sources, sp, list);
 	sp->idx = tl->nsources++;
@@ -759,7 +752,7 @@ vcc_CompileSource(const struct vcc *tl0, struct vsb *sb, struct source *sp)
  */
 
 char *
-VCC_Compile(const struct vcc *tl, struct vsb *sb, const char *b)
+VCC_Compile(const struct vcp *vcp, struct vsb *sb, const char *b)
 {
 	struct source *sp;
 	char *r;
@@ -767,7 +760,7 @@ VCC_Compile(const struct vcc *tl, struct vsb *sb, const char *b)
 	sp = vcc_new_source(b, NULL, "input");
 	if (sp == NULL)
 		return (NULL);
-	r = vcc_CompileSource(tl, sb, sp);
+	r = vcc_CompileSource(vcp, sb, sp);
 	return (r);
 }
 
@@ -775,16 +768,15 @@ VCC_Compile(const struct vcc *tl, struct vsb *sb, const char *b)
  * Allocate a compiler instance
  */
 
-struct vcc *
-VCC_New(void)
+struct vcp *
+VCP_New(void)
 {
-	struct vcc *tl;
+	struct vcp *vcp;
 
-	tl = vcc_NewVcc(NULL);
+	ALLOC_OBJ(vcp, VCP_MAGIC);
+	AN(vcp);
 
-	tl->vars = vcc_vars;
-
-	return (tl);
+	return (vcp);
 }
 
 /*--------------------------------------------------------------------
@@ -792,11 +784,11 @@ VCC_New(void)
  */
 
 void
-VCC_Builtin_VCL(struct vcc *tl, const char *str)
+VCP_Builtin_VCL(struct vcp *vcp, const char *str)
 {
 
-	CHECK_OBJ_NOTNULL(tl, VCC_MAGIC);
-	REPLACE(tl->builtin_vcl, str);
+	CHECK_OBJ_NOTNULL(vcp, VCP_MAGIC);
+	REPLACE(vcp->builtin_vcl, str);
 }
 
 /*--------------------------------------------------------------------
@@ -804,11 +796,11 @@ VCC_Builtin_VCL(struct vcc *tl, const char *str)
  */
 
 void
-VCC_VCL_dir(struct vcc *tl, const char *str)
+VCP_VCL_dir(struct vcp *vcp, const char *str)
 {
 
-	CHECK_OBJ_NOTNULL(tl, VCC_MAGIC);
-	REPLACE(tl->vcl_dir, str);
+	CHECK_OBJ_NOTNULL(vcp, VCP_MAGIC);
+	REPLACE(vcp->vcl_dir, str);
 }
 
 /*--------------------------------------------------------------------
@@ -816,11 +808,11 @@ VCC_VCL_dir(struct vcc *tl, const char *str)
  */
 
 void
-VCC_VMOD_dir(struct vcc *tl, const char *str)
+VCP_VMOD_dir(struct vcp *vcp, const char *str)
 {
 
-	CHECK_OBJ_NOTNULL(tl, VCC_MAGIC);
-	REPLACE(tl->vmod_dir, str);
+	CHECK_OBJ_NOTNULL(vcp, VCP_MAGIC);
+	REPLACE(vcp->vmod_dir, str);
 }
 
 /*--------------------------------------------------------------------
@@ -828,25 +820,25 @@ VCC_VMOD_dir(struct vcc *tl, const char *str)
  */
 
 void
-VCC_Err_Unref(struct vcc *tl, unsigned u)
+VCP_Err_Unref(struct vcp *vcp, unsigned u)
 {
 
-	CHECK_OBJ_NOTNULL(tl, VCC_MAGIC);
-	tl->err_unref = u;
+	CHECK_OBJ_NOTNULL(vcp, VCP_MAGIC);
+	vcp->err_unref = u;
 }
 
 void
-VCC_Allow_InlineC(struct vcc *tl, unsigned u)
+VCP_Allow_InlineC(struct vcp *vcp, unsigned u)
 {
 
-	CHECK_OBJ_NOTNULL(tl, VCC_MAGIC);
-	tl->allow_inline_c = u;
+	CHECK_OBJ_NOTNULL(vcp, VCP_MAGIC);
+	vcp->allow_inline_c = u;
 }
 
 void
-VCC_Unsafe_Path(struct vcc *tl, unsigned u)
+VCP_Unsafe_Path(struct vcp *vcp, unsigned u)
 {
 
-	CHECK_OBJ_NOTNULL(tl, VCC_MAGIC);
-	tl->unsafe_path = u;
+	CHECK_OBJ_NOTNULL(vcp, VCP_MAGIC);
+	vcp->unsafe_path = u;
 }
diff --git a/lib/libvcc/vcc_compile.h b/lib/libvcc/vcc_compile.h
index 4c155a1..72ff7b7 100644
--- a/lib/libvcc/vcc_compile.h
+++ b/lib/libvcc/vcc_compile.h
@@ -155,14 +155,24 @@ struct inifin {
 
 VTAILQ_HEAD(inifinhead, inifin);
 
-struct vcc {
+struct vcp {
 	unsigned		magic;
-#define VCC_MAGIC		0x24ad719d
+#define VCP_MAGIC		0xd90acfbc
 
-	/* Parameter/Template section */
 	char			*builtin_vcl;
 	char			*vcl_dir;
 	char			*vmod_dir;
+	unsigned		err_unref;
+	unsigned		allow_inline_c;
+	unsigned		unsafe_path;
+};
+
+struct vcc {
+	unsigned		magic;
+#define VCC_MAGIC		0x24ad719d
+
+	/* Parameter/Template section */
+	const struct vcp	*param;
 
 	const struct var	*vars;
 	VTAILQ_HEAD(, symbol)	symbols;
@@ -202,9 +212,6 @@ struct vcc {
 
 	unsigned		unique;
 
-	unsigned		err_unref;
-	unsigned		allow_inline_c;
-	unsigned		unsafe_path;
 };
 
 struct var {
diff --git a/lib/libvcc/vcc_parse.c b/lib/libvcc/vcc_parse.c
index accabd6..8a94749 100644
--- a/lib/libvcc/vcc_parse.c
+++ b/lib/libvcc/vcc_parse.c
@@ -160,7 +160,7 @@ vcc_Compound(struct vcc *tl)
 			Fb(tl, 1, "}\n");
 			return;
 		case CSRC:
-			if (tl->allow_inline_c) {
+			if (tl->param->allow_inline_c) {
 				Fb(tl, 1, "%.*s\n",
 				    (int) (tl->t->e - (tl->t->b + 2)),
 				    tl->t->b + 1);
@@ -357,7 +357,7 @@ vcc_Parse(struct vcc *tl)
 		ERRCHK(tl);
 		switch (tl->t->tok) {
 		case CSRC:
-			if (tl->allow_inline_c) {
+			if (tl->param->allow_inline_c) {
 				Fc(tl, 0, "%.*s\n",
 				    (int) (tl->t->e - (tl->t->b + 4)),
 				    tl->t->b + 2);
diff --git a/lib/libvcc/vcc_vmod.c b/lib/libvcc/vcc_vmod.c
index 60d2ed2..d82ad3e 100644
--- a/lib/libvcc/vcc_vmod.c
+++ b/lib/libvcc/vcc_vmod.c
@@ -85,7 +85,7 @@ vcc_ParseImport(struct vcc *tl)
 	sym->def_e = tl->t;
 
 	if (tl->t->tok == ID) {
-		if (!tl->unsafe_path) {
+		if (!tl->param->unsafe_path) {
 			VSB_printf(tl->sb,
 			    "'import ... from path ...' not allowed.\nAt:");
 			vcc_ErrToken(tl, tl->t);
@@ -102,7 +102,7 @@ vcc_ParseImport(struct vcc *tl)
 		bprintf(fn, "%s", tl->t->dec);
 		vcc_NextToken(tl);
 	} else {
-		bprintf(fn, "%s/libvmod_%.*s.so", tl->vmod_dir, PF(mod));
+		bprintf(fn, "%s/libvmod_%.*s.so", tl->param->vmod_dir, PF(mod));
 	}
 
 	SkipToken(tl, ';');
diff --git a/lib/libvcc/vcc_xref.c b/lib/libvcc/vcc_xref.c
index c3ef375..5440829 100644
--- a/lib/libvcc/vcc_xref.c
+++ b/lib/libvcc/vcc_xref.c
@@ -108,7 +108,7 @@ vcc_checkref(struct vcc *tl, const struct symbol *sym)
 		VSB_printf(tl->sb, "Unused %s %.*s, defined:\n",
 		    VCC_SymKind(tl, sym), PF(sym->def_b));
 		vcc_ErrWhere(tl, sym->def_b);
-		if (!tl->err_unref) {
+		if (!tl->param->err_unref) {
 			VSB_printf(tl->sb, "(That was just a warning)\n");
 			tl->err = 0;
 		}
@@ -289,7 +289,7 @@ vcc_checkaction2(struct vcc *tl, const struct symbol *sym)
 		return;
 	VSB_printf(tl->sb, "Function unused\n");
 	vcc_ErrWhere(tl, p->name);
-	if (!tl->err_unref) {
+	if (!tl->param->err_unref) {
 		VSB_printf(tl->sb, "(That was just a warning)\n");
 		tl->err = 0;
 	}



From lkarsten at varnish-software.com  Thu Jan 14 14:15:12 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:12 +0100
Subject: [4.1] 772d258 Add a VFIL facility for searching paths of files.
Message-ID: <E1aJigC-0005W2-I8@project.varnish-software.com>


commit 772d258ede613e200a48d4b0a6c662dfee7681ca
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Jan 8 18:48:01 2016 +0000

    Add a VFIL facility for searching paths of files.

diff --git a/include/vfil.h b/include/vfil.h
index 97484ad..4269fe0 100644
--- a/include/vfil.h
+++ b/include/vfil.h
@@ -28,10 +28,16 @@
  *
  */
 
+struct vfil_path;
+
 /* from libvarnish/vfil.c */
 int seed_random(void);
 char *VFIL_readfile(const char *pfx, const char *fn, ssize_t *sz);
-char *VFIL_readfd(int fd, ssize_t *sz);
 int VFIL_nonblocking(int fd);
 int VFIL_fsinfo(int fd, unsigned *pbs, uintmax_t *size, uintmax_t *space);
 int VFIL_allocate(int fd, off_t size, int insist);
+void VFIL_setpath(struct vfil_path**, const char *path);
+typedef int vfil_path_func_f(void *priv, const char *fn);
+int VFIL_searchpath(const struct vfil_path *, vfil_path_func_f *func,
+    void *priv, char **fn);
+
diff --git a/lib/libvarnish/vfil.c b/lib/libvarnish/vfil.c
index 4b57d01..b6bdc9e 100644
--- a/lib/libvarnish/vfil.c
+++ b/lib/libvarnish/vfil.c
@@ -55,12 +55,16 @@
 #  include <linux/magic.h>
 #endif
 
-#include "vas.h"
 #include "vdef.h"
+
+#include "miniobj.h"
+#include "vas.h"
+#include "vsb.h"
 #include "vfil.h"
+#include "vqueue.h"
 
-char *
-VFIL_readfd(int fd, ssize_t *sz)
+static char *
+vfil_readfd(int fd, ssize_t *sz)
 {
 	struct stat st;
 	char *f;
@@ -99,7 +103,7 @@ VFIL_readfile(const char *pfx, const char *fn, ssize_t *sz)
 		fd = open(fn, O_RDONLY);
 	if (fd < 0)
 		return (NULL);
-	r = VFIL_readfd(fd, sz);
+	r = vfil_readfd(fd, sz);
 	err = errno;
 	AZ(close(fd));
 	errno = err;
@@ -158,7 +162,8 @@ VFIL_fsinfo(int fd, unsigned *pbs, uintmax_t *psize, uintmax_t *pspace)
 	return (0);
 }
 
-/* Make sure that the file system can accommodate the file of the given
+/*
+ * Make sure that the file system can accommodate the file of the given
  * size. Will use fallocate if available. If fallocate is not available
  * and insist is true, it will write size zero bytes.
  *
@@ -222,3 +227,117 @@ VFIL_allocate(int fd, off_t size, int insist)
 	assert(lseek(fd, 0, SEEK_SET) == 0);
 	return (0);
 }
+
+struct vfil_dir {
+	unsigned		magic;
+#define VFIL_DIR_MAGIC		0x3e214967
+	char			*dir;
+	VTAILQ_ENTRY(vfil_dir)	list;
+};
+
+struct vfil_path {
+	unsigned		magic;
+#define VFIL_PATH_MAGIC		0x92dbcc31
+	char			*str;
+	VTAILQ_HEAD(,vfil_dir)	paths;
+};
+
+/*
+ * Path searching functions
+ */
+
+void
+VFIL_setpath(struct vfil_path **pp, const char *path)
+{
+	struct vfil_path *vp;
+	struct vfil_dir *vd;
+	char *p, *q;
+
+	AN(pp);
+	AN(path);
+
+	vp = *pp;
+	if (vp == NULL) {
+		ALLOC_OBJ(vp, VFIL_PATH_MAGIC);
+		AN(vp);
+		VTAILQ_INIT(&vp->paths);
+		*pp = vp;
+	}
+	REPLACE(vp->str, path);
+	while (!VTAILQ_EMPTY(&vp->paths)) {
+		vd = VTAILQ_FIRST(&vp->paths);
+		VTAILQ_REMOVE(&vp->paths, vd, list);
+		FREE_OBJ(vd);
+	}
+	for (p = vp->str; p != NULL; p = q) {
+		q = strchr(p, ':');
+		if (q != NULL)
+			*q++ = '\0';
+		ALLOC_OBJ(vd, VFIL_DIR_MAGIC);
+		AN(vd);
+		vd->dir = p;
+		VTAILQ_INSERT_TAIL(&vp->paths, vd, list);
+	}
+}
+
+static int
+vfil_path_openfile(void *priv, const char *fn)
+{
+	char *p, **pp;
+
+	AN(priv);
+	AN(fn);
+	p = VFIL_readfile(NULL, fn, NULL);
+	if (p == NULL)
+		return (1);
+
+	pp = priv;
+	*pp = p;
+	return (0);
+}
+
+int
+VFIL_searchpath(const struct vfil_path *vp, vfil_path_func_f *func, void *priv,
+    char **fnp)
+{
+	struct vsb *vsb;
+	struct vfil_dir *vd;
+	const char *fn;
+	int i, e;
+
+	CHECK_OBJ_NOTNULL(vp, VFIL_PATH_MAGIC);
+	AN(fnp);
+	AN(*fnp);
+	fn = *fnp;
+	*fnp = NULL;
+
+	if (func == NULL) {
+		func = vfil_path_openfile;
+		AN(priv);
+	}
+
+	if (*fn == '/') {
+		i = func(priv, fn);
+		if (i <= 0)
+			REPLACE(*fnp, fn);
+		return (i);
+	}
+	vsb = VSB_new_auto();
+	AN(vsb);
+	VTAILQ_FOREACH(vd, &vp->paths, list) {
+		VSB_clear(vsb);
+		VSB_printf(vsb, "%s/%s", vd->dir, fn);
+		AZ(VSB_finish(vsb));
+		i = func(priv, VSB_data(vsb));
+		if (i <= 0) {
+			e = errno;
+			*fnp = strdup(VSB_data(vsb));
+			AN(*fnp);
+			VSB_delete(vsb);
+			errno = e;
+			return (i);
+		}
+	}
+	VSB_delete(vsb);
+	return (-1);
+}



From lkarsten at varnish-software.com  Thu Jan 14 14:15:12 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:12 +0100
Subject: [4.1] 2765c02 Let vcl_dir and vmod_dir also take colon separated
 paths.
Message-ID: <E1aJigC-0005WC-Jr@project.varnish-software.com>


commit 2765c02d6eec7599c2b16ffd76ea28b1a9748c31
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Jan 8 18:51:44 2016 +0000

    Let vcl_dir and vmod_dir also take colon separated paths.

diff --git a/bin/varnishd/mgt/mgt_param_tbl.c b/bin/varnishd/mgt/mgt_param_tbl.c
index 16886b2..59fcfed 100644
--- a/bin/varnishd/mgt/mgt_param_tbl.c
+++ b/bin/varnishd/mgt/mgt_param_tbl.c
@@ -58,14 +58,16 @@ struct parspec mgt_parspec[] = {
 		VCC_CC , NULL },
 	{ "vcl_dir", tweak_string, &mgt_vcl_dir,
 		NULL, NULL,
-		"Directory from which relative VCL filenames (vcl.load and "
-		"include) are opened.",
+		"Directory (or colon separated list of directories) "
+		"from which relative VCL filenames (vcl.load and "
+		"include) are to be found.",
 		0,
 		VARNISH_VCL_DIR,
 		NULL },
 	{ "vmod_dir", tweak_string, &mgt_vmod_dir,
 		NULL, NULL,
-		"Directory where VCL modules are to be found.",
+		"Directory (or colon separated list of directories) "
+		"where VMODs are to be found.",
 		0,
 		VARNISH_VMOD_DIR,
 		NULL },
diff --git a/bin/varnishtest/tests/m00008.vtc b/bin/varnishtest/tests/m00008.vtc
index 232d3c0..833a282 100644
--- a/bin/varnishtest/tests/m00008.vtc
+++ b/bin/varnishtest/tests/m00008.vtc
@@ -13,12 +13,12 @@ varnish v1 -vcl+backend {
 
 varnish v1 -cliok "param.set vcc_unsafe_path off"
 
-varnish v1 -errvcl {'import ... from path ...' not allowed.} {
+varnish v1 -errvcl {'import ... from path ...' is unsafe.} {
 	backend default { .host = "${s1_sock}"; }
 	import ${vmod_std};
 }
 
-varnish v1 -cliok "param.set vmod_dir ${topbuild}/lib/libvmod_std/.libs/"
+varnish v1 -cliok "param.set vmod_dir /nowhere:${topbuild}/lib/libvmod_std/.libs/:/else"
 
 varnish v1 -vcl+backend {
 	import std;
diff --git a/lib/libvcc/vcc_compile.c b/lib/libvcc/vcc_compile.c
index aaba4bf..c52dd8d 100644
--- a/lib/libvcc/vcc_compile.c
+++ b/lib/libvcc/vcc_compile.c
@@ -450,22 +450,24 @@ vcc_destroy_source(struct source *sp)
 /*--------------------------------------------------------------------*/
 
 static struct source *
-vcc_file_source(const struct vcp * const vcp, struct vsb *sb, const char *fn)
+vcc_file_source(const struct vcp * const vcp, struct vsb *sb, char *fn)
 {
-	char *f;
+	char *f, *fnp;
 	struct source *sp;
 
 	if (!vcp->unsafe_path && strchr(fn, '/') != NULL) {
 		VSB_printf(sb, "Include path is unsafe '%s'\n", fn);
 		return (NULL);
 	}
-	f = VFIL_readfile(vcp->vcl_dir, fn, NULL);
-	if (f == NULL) {
-		VSB_printf(sb, "Cannot read file '%s': %s\n",
-		    fn, strerror(errno));
+	f = NULL;
+	fnp = fn;
+	if (VFIL_searchpath(vcp->vcl_path, NULL, &f, &fnp) || f == NULL) {
+		VSB_printf(sb, "Cannot read file '%s' (%s)\n",
+		    fnp != NULL ? fnp : fn, strerror(errno));
 		return (NULL);
 	}
-	sp = vcc_new_source(f, NULL, fn);
+	sp = vcc_new_source(f, NULL, fnp);
+	free(fnp);
 	sp->freeit = f;
 	return (sp);
 }
@@ -801,6 +803,7 @@ VCP_VCL_dir(struct vcp *vcp, const char *str)
 
 	CHECK_OBJ_NOTNULL(vcp, VCP_MAGIC);
 	REPLACE(vcp->vcl_dir, str);
+	VFIL_setpath(&vcp->vcl_path, str);
 }
 
 /*--------------------------------------------------------------------
@@ -813,6 +816,7 @@ VCP_VMOD_dir(struct vcp *vcp, const char *str)
 
 	CHECK_OBJ_NOTNULL(vcp, VCP_MAGIC);
 	REPLACE(vcp->vmod_dir, str);
+	VFIL_setpath(&vcp->vmod_path, str);
 }
 
 /*--------------------------------------------------------------------
diff --git a/lib/libvcc/vcc_compile.h b/lib/libvcc/vcc_compile.h
index 72ff7b7..3feecd3 100644
--- a/lib/libvcc/vcc_compile.h
+++ b/lib/libvcc/vcc_compile.h
@@ -161,7 +161,9 @@ struct vcp {
 
 	char			*builtin_vcl;
 	char			*vcl_dir;
+	struct vfil_path	*vcl_path;
 	char			*vmod_dir;
+	struct vfil_path	*vmod_path;
 	unsigned		err_unref;
 	unsigned		allow_inline_c;
 	unsigned		unsafe_path;
diff --git a/lib/libvcc/vcc_vmod.c b/lib/libvcc/vcc_vmod.c
index d82ad3e..0a165c7 100644
--- a/lib/libvcc/vcc_vmod.c
+++ b/lib/libvcc/vcc_vmod.c
@@ -30,20 +30,39 @@
 
 #include <dlfcn.h>
 #include <stdio.h>
+#include <stdlib.h>
 #include <string.h>
 
 #include "vcc_compile.h"
 
 #include "vcs_version.h"
 
+#include "vfil.h"
 #include "vmod_abi.h"
 #include "vrt.h"
 
+static int
+vcc_path_dlopen(void *priv, const char *fn)
+{
+	void *hdl, **pp;
+
+	AN(priv);
+	AN(fn);
+
+fprintf(stderr, "TRY <%s>\n", fn);
+	hdl = dlopen(fn, RTLD_NOW | RTLD_LOCAL);
+	if (hdl == NULL)
+		return (1);
+	pp = priv;
+	*pp = hdl;
+	return (0);
+}
+
 void
 vcc_ParseImport(struct vcc *tl)
 {
 	void *hdl;
-	char fn[1024];
+	char fn[1024], *fnp;
 	char buf[256];
 	struct token *mod, *t1;
 	struct inifin *ifp;
@@ -85,32 +104,33 @@ vcc_ParseImport(struct vcc *tl)
 	sym->def_e = tl->t;
 
 	if (tl->t->tok == ID) {
-		if (!tl->param->unsafe_path) {
-			VSB_printf(tl->sb,
-			    "'import ... from path ...' not allowed.\nAt:");
-			vcc_ErrToken(tl, tl->t);
-			vcc_ErrWhere(tl, tl->t);
-			return;
-		}
 		if (!vcc_IdIs(tl->t, "from")) {
 			VSB_printf(tl->sb, "Expected 'from path ...'\n");
 			vcc_ErrWhere(tl, tl->t);
 			return;
 		}
 		vcc_NextToken(tl);
+		if (!tl->param->unsafe_path && strchr(tl->t->dec, '/')) {
+			VSB_printf(tl->sb,
+			    "'import ... from path ...' is unsafe.\nAt:");
+			vcc_ErrToken(tl, tl->t);
+			vcc_ErrWhere(tl, tl->t);
+			return;
+		}
 		ExpectErr(tl, CSTR);
 		bprintf(fn, "%s", tl->t->dec);
 		vcc_NextToken(tl);
 	} else {
-		bprintf(fn, "%s/libvmod_%.*s.so", tl->param->vmod_dir, PF(mod));
+		bprintf(fn, "libvmod_%.*s.so", PF(mod));
 	}
 
 	SkipToken(tl, ';');
 
-	hdl = dlopen(fn, RTLD_NOW | RTLD_LOCAL);
-	if (hdl == NULL) {
+	fnp = fn;
+	if (VFIL_searchpath(tl->param->vmod_path,
+	    vcc_path_dlopen, &hdl, &fnp)) {
 		VSB_printf(tl->sb, "Could not load VMOD %.*s\n", PF(mod));
-		VSB_printf(tl->sb, "\tFile name: %s\n", fn);
+		VSB_printf(tl->sb, "\tFile name: %s\n", fnp != NULL ? fnp : fn);
 		VSB_printf(tl->sb, "\tdlerror: %s\n", dlerror());
 		vcc_ErrWhere(tl, mod);
 		return;
@@ -120,7 +140,7 @@ vcc_ParseImport(struct vcc *tl)
 	vmd = dlsym(hdl, buf);
 	if (vmd == NULL) {
 		VSB_printf(tl->sb, "Malformed VMOD %.*s\n", PF(mod));
-		VSB_printf(tl->sb, "\tFile name: %s\n", fn);
+		VSB_printf(tl->sb, "\tFile name: %s\n", fnp);
 		VSB_printf(tl->sb, "\t(no Vmod_Data symbol)\n");
 		vcc_ErrWhere(tl, mod);
 		return;
@@ -128,7 +148,7 @@ vcc_ParseImport(struct vcc *tl)
 	if (strcmp(VCS_Branch, "master") == 0 &&
 	    strcmp(vmd->abi, VMOD_ABI_Version) != 0) {
 		VSB_printf(tl->sb, "Incompatible VMOD %.*s\n", PF(mod));
-		VSB_printf(tl->sb, "\tFile name: %s\n", fn);
+		VSB_printf(tl->sb, "\tFile name: %s\n", fnp);
 		VSB_printf(tl->sb, "\tABI mismatch, expected <%s>, got <%s>\n",
 			   VMOD_ABI_Version, vmd->abi);
 		vcc_ErrWhere(tl, mod);
@@ -137,7 +157,7 @@ vcc_ParseImport(struct vcc *tl)
 	if (vmd->vrt_major != VRT_MAJOR_VERSION ||
 	    vmd->vrt_minor > VRT_MINOR_VERSION) {
 		VSB_printf(tl->sb, "Incompatible VMOD %.*s\n", PF(mod));
-		VSB_printf(tl->sb, "\tFile name: %s\n", fn);
+		VSB_printf(tl->sb, "\tFile name: %s\n", fnp);
 		VSB_printf(tl->sb, "\tVMOD version %u.%u\n",
 		    vmd->vrt_major, vmd->vrt_minor);
 		VSB_printf(tl->sb, "\tvarnishd version %u.%u\n",
@@ -151,7 +171,7 @@ vcc_ParseImport(struct vcc *tl)
 	    vmd->proto == NULL ||
 	    vmd->abi == NULL) {
 		VSB_printf(tl->sb, "Mangled VMOD %.*s\n", PF(mod));
-		VSB_printf(tl->sb, "\tFile name: %s\n", fn);
+		VSB_printf(tl->sb, "\tFile name: %s\n", fnp);
 		VSB_printf(tl->sb, "\tInconsistent metadata\n");
 		vcc_ErrWhere(tl, mod);
 		return;
@@ -159,7 +179,7 @@ vcc_ParseImport(struct vcc *tl)
 
 	if (!vcc_IdIs(mod, vmd->name)) {
 		VSB_printf(tl->sb, "Wrong VMOD file %.*s\n", PF(mod));
-		VSB_printf(tl->sb, "\tFile name: %s\n", fn);
+		VSB_printf(tl->sb, "\tFile name: %s\n", fnp);
 		VSB_printf(tl->sb, "\tContains vmod \"%s\"\n", vmd->name);
 		vcc_ErrWhere(tl, mod);
 		return;
@@ -172,7 +192,8 @@ vcc_ParseImport(struct vcc *tl)
 	VSB_printf(ifp->ini, "\t    sizeof(Vmod_%.*s_Func),\n", PF(mod));
 	VSB_printf(ifp->ini, "\t    \"%.*s\",\n", PF(mod));
 	VSB_printf(ifp->ini, "\t    ");
-	EncString(ifp->ini, fn, NULL, 0);
+	EncString(ifp->ini, fnp, NULL, 0);
+	free(fnp);
 	VSB_printf(ifp->ini, ",\n");
 	AN(vmd);
 	AN(vmd->file_id);



From lkarsten at varnish-software.com  Thu Jan 14 14:15:12 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:12 +0100
Subject: [4.1] 35a5f63 Also make vcl_dir act as path for the vcl.load command
Message-ID: <E1aJigC-0005WM-Ln@project.varnish-software.com>


commit 35a5f6360f6063727d064a9e9597ef37925dc33f
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Jan 8 19:01:48 2016 +0000

    Also make vcl_dir act as path for the vcl.load command

diff --git a/bin/varnishd/mgt/mgt_vcl.c b/bin/varnishd/mgt/mgt_vcl.c
index 37359f2..7d51690 100644
--- a/bin/varnishd/mgt/mgt_vcl.c
+++ b/bin/varnishd/mgt/mgt_vcl.c
@@ -60,6 +60,7 @@ struct vclprog {
 static VTAILQ_HEAD(, vclprog) vclhead = VTAILQ_HEAD_INITIALIZER(vclhead);
 static struct vclprog		*active_vcl;
 static struct vev *e_poker;
+static struct vfil_path *vcl_path;
 
 /*--------------------------------------------------------------------*/
 
@@ -269,7 +270,7 @@ mcf_vcl_inline(struct cli *cli, const char * const *av, void *priv)
 void
 mcf_vcl_load(struct cli *cli, const char * const *av, void *priv)
 {
-	char *vcl;
+	char *vcl, *fn;
 	struct vclprog *vp;
 
 	(void)priv;
@@ -280,9 +281,10 @@ mcf_vcl_load(struct cli *cli, const char * const *av, void *priv)
 		return;
 	}
 
-	vcl = VFIL_readfile(mgt_vcl_dir, av[3], NULL);
-	if (vcl == NULL) {
-		VCLI_Out(cli, "Cannot open '%s'", av[3]);
+	VFIL_setpath(&vcl_path, mgt_vcl_dir);
+	fn = TRUST_ME(av[3]);
+	if (VFIL_searchpath(vcl_path, NULL, &vcl, &fn)) {
+		VCLI_Out(cli, "Cannot open '%s'", fn != NULL ? fn : av[3]);
 		VCLI_SetResult(cli, CLIS_PARAM);
 		return;
 	}



From lkarsten at varnish-software.com  Thu Jan 14 14:15:12 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:12 +0100
Subject: [4.1] d1d86be rename srcfile to csrcfile, it confused me for too long
Message-ID: <E1aJigC-0005WY-Nr@project.varnish-software.com>


commit d1d86be7a73b321336e781d950ea129eef98b671
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Jan 8 20:58:33 2016 +0000

    rename srcfile to csrcfile, it confused me for too long

diff --git a/bin/varnishd/mgt/mgt_vcc.c b/bin/varnishd/mgt/mgt_vcc.c
index 9d28804..c7a641b 100644
--- a/bin/varnishd/mgt/mgt_vcc.c
+++ b/bin/varnishd/mgt/mgt_vcc.c
@@ -53,7 +53,7 @@ struct vcc_priv {
 #define VCC_PRIV_MAGIC	0x70080cb8
 	char		*dir;
 	const char	*src;
-	char		*srcfile;
+	char		*csrcfile;
 	char		*libfile;
 };
 
@@ -109,13 +109,13 @@ run_vcc(void *priv)
 
 	fd = open(VGC_SRC, O_WRONLY|O_TRUNC|O_CREAT, 0600);
 	if (fd < 0) {
-		fprintf(stderr, "VCC cannot open %s", vp->srcfile);
+		fprintf(stderr, "VCC cannot open %s", vp->csrcfile);
 		exit(2);
 	}
 	l = strlen(csrc);
 	i = write(fd, csrc, l);
 	if (i != l) {
-		fprintf(stderr, "VCC cannot write %s", vp->srcfile);
+		fprintf(stderr, "VCC cannot write %s", vp->csrcfile);
 		exit(2);
 	}
 	AZ(close(fd));
@@ -223,7 +223,7 @@ mgt_vcc_compile(struct vcc_priv *vp, struct vsb *sb, int C_flag)
 	char *csrc;
 	unsigned subs;
 
-	if (mgt_vcc_touchfile(vp->srcfile, sb))
+	if (mgt_vcc_touchfile(vp->csrcfile, sb))
 		return (2);
 	if (mgt_vcc_touchfile(vp->libfile, sb))
 		return (2);
@@ -233,7 +233,7 @@ mgt_vcc_compile(struct vcc_priv *vp, struct vsb *sb, int C_flag)
 		return (subs);
 
 	if (C_flag) {
-		csrc = VFIL_readfile(NULL, vp->srcfile, NULL);
+		csrc = VFIL_readfile(NULL, vp->csrcfile, NULL);
 		AN(csrc);
 		VSB_cat(sb, csrc);
 		free(csrc);
@@ -275,14 +275,14 @@ mgt_VccCompile(struct cli *cli, const char *vclname, const char *vclsrc,
 	VSB_clear(sb);
 	VSB_printf(sb, "%s/%s", vp.dir, VGC_SRC);
 	AZ(VSB_finish(sb));
-	vp.srcfile = strdup(VSB_data(sb));
-	AN(vp.srcfile);
+	vp.csrcfile = strdup(VSB_data(sb));
+	AN(vp.csrcfile);
 	VSB_clear(sb);
 
 	VSB_printf(sb, "%s/%s", vp.dir, VGC_LIB);
 	AZ(VSB_finish(sb));
 	vp.libfile = strdup(VSB_data(sb));
-	AN(vp.srcfile);
+	AN(vp.csrcfile);
 	VSB_clear(sb);
 
 	status = mgt_vcc_compile(&vp, sb, C_flag);
@@ -292,8 +292,8 @@ mgt_VccCompile(struct cli *cli, const char *vclname, const char *vclsrc,
 		VCLI_Out(cli, "%s", VSB_data(sb));
 	VSB_delete(sb);
 
-	(void)unlink(vp.srcfile);
-	free(vp.srcfile);
+	(void)unlink(vp.csrcfile);
+	free(vp.csrcfile);
 
 	free(vp.dir);
 



From lkarsten at varnish-software.com  Thu Jan 14 14:15:12 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:12 +0100
Subject: [4.1] 83d642b Sanitize the VFIL_searchpath() caling convention to not
 need throwing away const.
Message-ID: <E1aJigC-0005Wg-Q6@project.varnish-software.com>


commit 83d642bb4c615c2675fbbe1f34484af5e4c1fc4b
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Jan 8 21:14:11 2016 +0000

    Sanitize the VFIL_searchpath() caling convention to not need throwing
    away const.

diff --git a/bin/varnishd/mgt/mgt_vcl.c b/bin/varnishd/mgt/mgt_vcl.c
index 7d51690..ae2f898 100644
--- a/bin/varnishd/mgt/mgt_vcl.c
+++ b/bin/varnishd/mgt/mgt_vcl.c
@@ -282,12 +282,13 @@ mcf_vcl_load(struct cli *cli, const char * const *av, void *priv)
 	}
 
 	VFIL_setpath(&vcl_path, mgt_vcl_dir);
-	fn = TRUST_ME(av[3]);
-	if (VFIL_searchpath(vcl_path, NULL, &vcl, &fn)) {
+	if (VFIL_searchpath(vcl_path, NULL, &vcl, av[3], &fn)) {
 		VCLI_Out(cli, "Cannot open '%s'", fn != NULL ? fn : av[3]);
+		REPLACE(fn, NULL);
 		VCLI_SetResult(cli, CLIS_PARAM);
 		return;
 	}
+	REPLACE(fn, NULL);
 
 	mgt_new_vcl(cli, av[2], vcl, av[4], 0);
 	free(vcl);
diff --git a/include/vfil.h b/include/vfil.h
index 4269fe0..a542e4d 100644
--- a/include/vfil.h
+++ b/include/vfil.h
@@ -39,5 +39,5 @@ int VFIL_allocate(int fd, off_t size, int insist);
 void VFIL_setpath(struct vfil_path**, const char *path);
 typedef int vfil_path_func_f(void *priv, const char *fn);
 int VFIL_searchpath(const struct vfil_path *, vfil_path_func_f *func,
-    void *priv, char **fn);
+    void *priv, const char *fni, char **fno);
 
diff --git a/lib/libvarnish/vfil.c b/lib/libvarnish/vfil.c
index b6bdc9e..319c3e1 100644
--- a/lib/libvarnish/vfil.c
+++ b/lib/libvarnish/vfil.c
@@ -298,41 +298,38 @@ vfil_path_openfile(void *priv, const char *fn)
 
 int
 VFIL_searchpath(const struct vfil_path *vp, vfil_path_func_f *func, void *priv,
-    char **fnp)
+    const char *fni, char **fno)
 {
 	struct vsb *vsb;
 	struct vfil_dir *vd;
-	const char *fn;
 	int i, e;
 
 	CHECK_OBJ_NOTNULL(vp, VFIL_PATH_MAGIC);
-	AN(fnp);
-	AN(*fnp);
-	fn = *fnp;
-	*fnp = NULL;
+	AN(fno);
+	*fno = NULL;
 
 	if (func == NULL) {
 		func = vfil_path_openfile;
 		AN(priv);
 	}
 
-	if (*fn == '/') {
-		i = func(priv, fn);
+	if (*fni == '/') {
+		i = func(priv, fni);
 		if (i <= 0)
-			REPLACE(*fnp, fn);
+			REPLACE(*fno, fni);
 		return (i);
 	}
 	vsb = VSB_new_auto();
 	AN(vsb);
 	VTAILQ_FOREACH(vd, &vp->paths, list) {
 		VSB_clear(vsb);
-		VSB_printf(vsb, "%s/%s", vd->dir, fn);
+		VSB_printf(vsb, "%s/%s", vd->dir, fni);
 		AZ(VSB_finish(vsb));
 		i = func(priv, VSB_data(vsb));
 		if (i <= 0) {
 			e = errno;
-			*fnp = strdup(VSB_data(vsb));
-			AN(*fnp);
+			*fno = strdup(VSB_data(vsb));
+			AN(*fno);
 			VSB_delete(vsb);
 			errno = e;
 			return (i);
diff --git a/lib/libvcc/vcc_compile.c b/lib/libvcc/vcc_compile.c
index c52dd8d..f35e07b 100644
--- a/lib/libvcc/vcc_compile.c
+++ b/lib/libvcc/vcc_compile.c
@@ -450,7 +450,7 @@ vcc_destroy_source(struct source *sp)
 /*--------------------------------------------------------------------*/
 
 static struct source *
-vcc_file_source(const struct vcp * const vcp, struct vsb *sb, char *fn)
+vcc_file_source(const struct vcp * const vcp, struct vsb *sb, const char *fn)
 {
 	char *f, *fnp;
 	struct source *sp;
@@ -460,8 +460,7 @@ vcc_file_source(const struct vcp * const vcp, struct vsb *sb, char *fn)
 		return (NULL);
 	}
 	f = NULL;
-	fnp = fn;
-	if (VFIL_searchpath(vcp->vcl_path, NULL, &f, &fnp) || f == NULL) {
+	if (VFIL_searchpath(vcp->vcl_path, NULL, &f, fn, &fnp) || f == NULL) {
 		VSB_printf(sb, "Cannot read file '%s' (%s)\n",
 		    fnp != NULL ? fnp : fn, strerror(errno));
 		return (NULL);
diff --git a/lib/libvcc/vcc_vmod.c b/lib/libvcc/vcc_vmod.c
index 0a165c7..0f90e4f 100644
--- a/lib/libvcc/vcc_vmod.c
+++ b/lib/libvcc/vcc_vmod.c
@@ -49,7 +49,6 @@ vcc_path_dlopen(void *priv, const char *fn)
 	AN(priv);
 	AN(fn);
 
-fprintf(stderr, "TRY <%s>\n", fn);
 	hdl = dlopen(fn, RTLD_NOW | RTLD_LOCAL);
 	if (hdl == NULL)
 		return (1);
@@ -126,9 +125,8 @@ vcc_ParseImport(struct vcc *tl)
 
 	SkipToken(tl, ';');
 
-	fnp = fn;
 	if (VFIL_searchpath(tl->param->vmod_path,
-	    vcc_path_dlopen, &hdl, &fnp)) {
+	    vcc_path_dlopen, &hdl, fn, &fnp)) {
 		VSB_printf(tl->sb, "Could not load VMOD %.*s\n", PF(mod));
 		VSB_printf(tl->sb, "\tFile name: %s\n", fnp != NULL ? fnp : fn);
 		VSB_printf(tl->sb, "\tdlerror: %s\n", dlerror());



From lkarsten at varnish-software.com  Thu Jan 14 14:15:10 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:10 +0100
Subject: [4.1] 0a89402 Handle OBJ_STICKY in flags
Message-ID: <E1aJigA-0005RS-U8@project.varnish-software.com>


commit 0a894029b0756958dcc02c054e3ed971fc076ddc
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Tue Jan 5 23:25:07 2016 +0000

    Handle OBJ_STICKY in flags

diff --git a/bin/varnishd/mgt/mgt_param.c b/bin/varnishd/mgt/mgt_param.c
index 2a52709..908643f 100644
--- a/bin/varnishd/mgt/mgt_param.c
+++ b/bin/varnishd/mgt/mgt_param.c
@@ -575,6 +575,10 @@ MCF_DumpRstParam(void)
 				printf("%sonly_root", q);
 				q = ", ";
 			}
+			if (pp->flags & OBJ_STICKY) {
+				printf("%sobj_sticky", q);
+				q = ", ";
+			}
 			printf("\n");
 		}
 		printf("\n");



From lkarsten at varnish-software.com  Thu Jan 14 14:15:11 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:11 +0100
Subject: [4.1] cde5903 Spelling
Message-ID: <E1aJigB-0005Rb-0F@project.varnish-software.com>


commit cde5903e72b6a511090884ccfb954ee1b5c87baf
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Tue Jan 5 23:26:23 2016 +0000

    Spelling

diff --git a/bin/varnishd/mgt/mgt_param.c b/bin/varnishd/mgt/mgt_param.c
index 908643f..0ee9397 100644
--- a/bin/varnishd/mgt/mgt_param.c
+++ b/bin/varnishd/mgt/mgt_param.c
@@ -138,7 +138,7 @@ mcf_addpar(struct parspec *ps)
 
 /*--------------------------------------------------------------------
  * Wrap the text nicely.
- * Lines are allowed to contain to TABS and we render that as a table
+ * Lines are allowed to contain two TABS and we render that as a table
  * taking the width of the first column into account.
  */
 
diff --git a/bin/varnishd/mgt/mgt_param_bits.c b/bin/varnishd/mgt/mgt_param_bits.c
index 4713ac6..100d961 100644
--- a/bin/varnishd/mgt/mgt_param_bits.c
+++ b/bin/varnishd/mgt/mgt_param_bits.c
@@ -238,7 +238,7 @@ struct parspec VSL_parspec[] = {
 	{ "vsl_mask", tweak_vsl_mask, NULL, NULL, NULL,
 		"Mask individual VSL messages from being logged.\n"
 		"\tdefault\tSet default value\n"
-		"\nUse +/- prefixe in front of VSL tag name, to mask/unmask "
+		"\nUse +/- prefix in front of VSL tag name, to mask/unmask "
 		"individual VSL messages.",
 		0, "default", "" },
 	{ "debug", tweak_debug, NULL, NULL, NULL,
diff --git a/bin/varnishd/mgt/mgt_param_tbl.c b/bin/varnishd/mgt/mgt_param_tbl.c
index 129300f..3a6239b 100644
--- a/bin/varnishd/mgt/mgt_param_tbl.c
+++ b/bin/varnishd/mgt/mgt_param_tbl.c
@@ -207,7 +207,7 @@ struct parspec mgt_parspec[] = {
 	{ "nuke_limit",
 		tweak_uint, &mgt_param.nuke_limit,
 		"0", NULL,
-		"Maximum number of objects we attempt to nuke in order"
+		"Maximum number of objects we attempt to nuke in order "
 		"to make space for a object body.",
 		EXPERIMENTAL,
 		"50", "allocations" },



From lkarsten at varnish-software.com  Thu Jan 14 14:15:12 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:12 +0100
Subject: [4.1] 87e0e7b Pass either vclsrc or vclfilename to the VCC compiler,
 and eliminate the VCL fileopening logic in mgt so we have it only one
 place.
Message-ID: <E1aJigC-0005Wq-T6@project.varnish-software.com>


commit 87e0e7b45d304daccec1da317cb64f1b3f8117ab
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Jan 8 21:35:13 2016 +0000

    Pass either vclsrc or vclfilename to the VCC compiler, and eliminate
    the VCL fileopening logic in mgt so we have it only one place.

diff --git a/bin/varnishd/mgt/mgt.h b/bin/varnishd/mgt/mgt.h
index 5fb12d4..a31bc6a 100644
--- a/bin/varnishd/mgt/mgt.h
+++ b/bin/varnishd/mgt/mgt.h
@@ -161,7 +161,7 @@ void STV_Config_Transient(void);
 
 /* mgt_vcc.c */
 char *mgt_VccCompile(struct cli *, const char *vclname, const char *vclsrc,
-    int C_flag);
+    const char *vclsrcfile, int C_flag);
 void mgt_vcc_init(void);
 
 void mgt_vcl_init(void);
diff --git a/bin/varnishd/mgt/mgt_vcc.c b/bin/varnishd/mgt/mgt_vcc.c
index c7a641b..77f67c0 100644
--- a/bin/varnishd/mgt/mgt_vcc.c
+++ b/bin/varnishd/mgt/mgt_vcc.c
@@ -52,7 +52,8 @@ struct vcc_priv {
 	unsigned	magic;
 #define VCC_PRIV_MAGIC	0x70080cb8
 	char		*dir;
-	const char	*src;
+	const char	*vclsrc;
+	const char	*vclsrcfile;
 	char		*csrcfile;
 	char		*libfile;
 };
@@ -99,7 +100,7 @@ run_vcc(void *priv)
 	VCP_Err_Unref(vcp, mgt_vcc_err_unref);
 	VCP_Allow_InlineC(vcp, mgt_vcc_allow_inline_c);
 	VCP_Unsafe_Path(vcp, mgt_vcc_unsafe_path);
-	csrc = VCC_Compile(vcp, sb, vp->src);
+	csrc = VCC_Compile(vcp, sb, vp->vclsrc, vp->vclsrcfile);
 	AZ(VSB_finish(sb));
 	if (VSB_len(sb))
 		printf("%s", VSB_data(sb));
@@ -251,7 +252,7 @@ mgt_vcc_compile(struct vcc_priv *vp, struct vsb *sb, int C_flag)
 
 char *
 mgt_VccCompile(struct cli *cli, const char *vclname, const char *vclsrc,
-    int C_flag)
+    const char *vclsrcfile, int C_flag)
 {
 	struct vcc_priv vp;
 	struct vsb *sb;
@@ -263,7 +264,8 @@ mgt_VccCompile(struct cli *cli, const char *vclname, const char *vclsrc,
 	XXXAN(sb);
 
 	INIT_OBJ(&vp, VCC_PRIV_MAGIC);
-	vp.src = vclsrc;
+	vp.vclsrc = vclsrc;
+	vp.vclsrcfile = vclsrcfile;
 
 	VSB_printf(sb, "vcl_%s", vclname);
 	AZ(VSB_finish(sb));
@@ -271,7 +273,6 @@ mgt_VccCompile(struct cli *cli, const char *vclname, const char *vclsrc,
 	AN(vp.dir);
 	VJ_make_vcldir(vp.dir);
 
-
 	VSB_clear(sb);
 	VSB_printf(sb, "%s/%s", vp.dir, VGC_SRC);
 	AZ(VSB_finish(sb));
diff --git a/bin/varnishd/mgt/mgt_vcl.c b/bin/varnishd/mgt/mgt_vcl.c
index ae2f898..0b24c65 100644
--- a/bin/varnishd/mgt/mgt_vcl.c
+++ b/bin/varnishd/mgt/mgt_vcl.c
@@ -43,7 +43,6 @@
 #include "vcli.h"
 #include "vcli_priv.h"
 #include "vev.h"
-#include "vfil.h"
 #include "vtim.h"
 
 #include "mgt_cli.h"
@@ -60,7 +59,6 @@ struct vclprog {
 static VTAILQ_HEAD(, vclprog) vclhead = VTAILQ_HEAD_INITIALIZER(vclhead);
 static struct vclprog		*active_vcl;
 static struct vev *e_poker;
-static struct vfil_path *vcl_path;
 
 /*--------------------------------------------------------------------*/
 
@@ -160,7 +158,7 @@ mgt_vcl_setstate(struct vclprog *vp, int warm)
 
 static void
 mgt_new_vcl(struct cli *cli, const char *vclname, const char *vclsrc,
-    const char *state, int C_flag)
+    const char *vclsrcfile, const char *state, int C_flag)
 {
 	unsigned status;
 	char *lib, *p;
@@ -178,7 +176,7 @@ mgt_new_vcl(struct cli *cli, const char *vclname, const char *vclsrc,
 		return;
 	}
 
-	lib = mgt_VccCompile(cli, vclname, vclsrc, C_flag);
+	lib = mgt_VccCompile(cli, vclname, vclsrc, vclsrcfile, C_flag);
 	if (lib == NULL)
 		return;
 
@@ -210,7 +208,7 @@ mgt_vcc_default(struct cli *cli, const char *b_arg, const char *vclsrc,
 
 	if (b_arg == NULL) {
 		AN(vclsrc);
-		mgt_new_vcl(cli, "boot", vclsrc, NULL, C_flag);
+		mgt_new_vcl(cli, "boot", vclsrc, NULL, NULL, C_flag);
 		return;
 	}
 
@@ -220,7 +218,7 @@ mgt_vcc_default(struct cli *cli, const char *b_arg, const char *vclsrc,
 	    "backend default {\n"
 	    "    .host = \"%s\";\n"
 	    "}\n", b_arg);
-	mgt_new_vcl(cli, "boot", buf, NULL, C_flag);
+	mgt_new_vcl(cli, "boot", buf, NULL, NULL, C_flag);
 }
 
 /*--------------------------------------------------------------------*/
@@ -264,13 +262,12 @@ mcf_vcl_inline(struct cli *cli, const char * const *av, void *priv)
 		return;
 	}
 
-	mgt_new_vcl(cli, av[2], av[3], av[4], 0);
+	mgt_new_vcl(cli, av[2], av[3], NULL, av[4], 0);
 }
 
 void
 mcf_vcl_load(struct cli *cli, const char * const *av, void *priv)
 {
-	char *vcl, *fn;
 	struct vclprog *vp;
 
 	(void)priv;
@@ -281,17 +278,7 @@ mcf_vcl_load(struct cli *cli, const char * const *av, void *priv)
 		return;
 	}
 
-	VFIL_setpath(&vcl_path, mgt_vcl_dir);
-	if (VFIL_searchpath(vcl_path, NULL, &vcl, av[3], &fn)) {
-		VCLI_Out(cli, "Cannot open '%s'", fn != NULL ? fn : av[3]);
-		REPLACE(fn, NULL);
-		VCLI_SetResult(cli, CLIS_PARAM);
-		return;
-	}
-	REPLACE(fn, NULL);
-
-	mgt_new_vcl(cli, av[2], vcl, av[4], 0);
-	free(vcl);
+	mgt_new_vcl(cli, av[2], NULL, av[3], av[4], 0);
 }
 
 static struct vclprog *
diff --git a/bin/varnishtest/tests/c00053.vtc b/bin/varnishtest/tests/c00053.vtc
index 9b87fa1..6a34045 100644
--- a/bin/varnishtest/tests/c00053.vtc
+++ b/bin/varnishtest/tests/c00053.vtc
@@ -13,7 +13,7 @@ varnish v1 -vcl+backend {
 
 varnish v1 -cliok "param.set vcc_unsafe_path off"
 
-varnish v1 -errvcl {Include path is unsafe} {
+varnish v1 -errvcl {' is unsafe} {
 	backend default {
 		.host = "${s1_sock}";
 	}
diff --git a/include/libvcc.h b/include/libvcc.h
index 696123e..fbd7bd1 100644
--- a/include/libvcc.h
+++ b/include/libvcc.h
@@ -38,4 +38,5 @@ void VCP_Err_Unref(struct vcp *tl, unsigned u);
 void VCP_Allow_InlineC(struct vcp *tl, unsigned u);
 void VCP_Unsafe_Path(struct vcp *tl, unsigned u);
 
-char *VCC_Compile(const struct vcp *, struct vsb *sb, const char *b);
+char *VCC_Compile(const struct vcp *, struct vsb *sb,
+    const char *vclsrc, const char *vclsrcfile);
diff --git a/lib/libvcc/vcc_compile.c b/lib/libvcc/vcc_compile.c
index f35e07b..95eab85 100644
--- a/lib/libvcc/vcc_compile.c
+++ b/lib/libvcc/vcc_compile.c
@@ -456,7 +456,7 @@ vcc_file_source(const struct vcp * const vcp, struct vsb *sb, const char *fn)
 	struct source *sp;
 
 	if (!vcp->unsafe_path && strchr(fn, '/') != NULL) {
-		VSB_printf(sb, "Include path is unsafe '%s'\n", fn);
+		VSB_printf(sb, "VCL filename '%s' is unsafe.\n", fn);
 		return (NULL);
 	}
 	f = NULL;
@@ -753,12 +753,19 @@ vcc_CompileSource(const struct vcp * const vcp, struct vsb *sb,
  */
 
 char *
-VCC_Compile(const struct vcp *vcp, struct vsb *sb, const char *b)
+VCC_Compile(const struct vcp *vcp, struct vsb *sb,
+    const char *vclsrc, const char *vclsrcfile)
 {
 	struct source *sp;
 	char *r;
 
-	sp = vcc_new_source(b, NULL, "input");
+	if (vclsrc != NULL) {
+		AZ(vclsrcfile);
+		sp = vcc_new_source(vclsrc, NULL, "input");
+	} else {
+		AN(vclsrcfile);
+		sp = vcc_file_source(vcp, sb, vclsrcfile);
+	}
 	if (sp == NULL)
 		return (NULL);
 	r = vcc_CompileSource(vcp, sb, sp);



From lkarsten at varnish-software.com  Thu Jan 14 14:15:12 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:12 +0100
Subject: [4.1] 298dc44 Make nested includes starting with "./" relative to the
 including VCL file, and illegal everywhere else.
Message-ID: <E1aJigC-0005X0-Uv@project.varnish-software.com>


commit 298dc4444ee73189833629214e06dad38b31b208
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Jan 8 22:06:26 2016 +0000

    Make nested includes starting with "./" relative to the including
    VCL file, and illegal everywhere else.
    
    This makes it easier to distribute "VCL-packages"
    
    Test-case written by:	Kacper

diff --git a/bin/varnishtest/tests/v00046.vtc b/bin/varnishtest/tests/v00046.vtc
new file mode 100644
index 0000000..567d3af
--- /dev/null
+++ b/bin/varnishtest/tests/v00046.vtc
@@ -0,0 +1,73 @@
+varnishtest "Test relative to vcl_dir, dot-include and absolute includes"
+
+# relative plain
+shell "true > ${tmpdir}/_start.vcl"
+varnish v1 -arg "-p vcl_dir=${tmpdir}" -vcl {
+	backend b { .host = "127.0.0.1"; }
+	include "_start.vcl" ;
+}
+
+# absolute include
+varnish v1 -vcl {
+	backend b { .host = "127.0.0.1"; }
+	include "${tmpdir}/_start.vcl" ;
+}
+
+# absolute -> relative include
+shell "mkdir -p ${tmpdir}/1/2/3"
+shell "true  > ${tmpdir}/1/2/b.vcl"
+shell "echo 'include \"./2/b.vcl\";' > ${tmpdir}/1/a.vcl"
+varnish v1 -vcl {
+	backend b { .host = "127.0.0.1"; }
+	include "${tmpdir}/1/a.vcl" ;
+}
+
+# same but relative to vcl_dir
+shell "echo 'include \"1/2/b.vcl\";' > ${tmpdir}/1/ab.vcl"
+varnish v1 -vcl {
+	backend b { .host = "127.0.0.1"; }
+	include "1/ab.vcl" ;
+}
+
+# dot-relative -> relative
+varnish v1 -vcl {
+	backend b { .host = "127.0.0.1"; }
+	include "1/a.vcl" ;
+}
+
+# relative -> relative -> relative
+shell "echo 'include \"./3/c.vcl\";' > ${tmpdir}/1/2/b.vcl"
+shell "true  > ${tmpdir}/1/2/3/c.vcl"
+varnish v1 -vcl {
+	backend b { .host = "127.0.0.1"; }
+	include "1/a.vcl" ;
+}
+
+# relative -> absolute
+shell "echo 'include \"${tmpdir}/1/2/3/c.vcl\";' > ${tmpdir}/1/aa.vcl"
+varnish v1 -vcl {
+	backend b { .host = "127.0.0.1"; }
+	include "1/aa.vcl" ;
+}
+
+# relative -> absolute -> relative
+shell "echo 'include \"${tmpdir}/1/2/b.vcl\";' > ${tmpdir}/1/aaa.vcl"
+varnish v1 -vcl {
+	backend b { .host = "127.0.0.1"; }
+	include "1/aaa.vcl" ;
+}
+
+# includes and parses out
+shell "echo 'zool' > ${tmpdir}/1/2/3/c.vcl"
+varnish v1 -errvcl {Found: 'zool' at} {
+	backend b { .host = "127.0.0.1"; }
+	include "1/a.vcl";
+}
+
+shell "rm -f ${tmpdir}/a"
+shell "rm -f ${tmpdir}/_start.vcl"
+
+varnish v1 -errvcl {only works in nested VCL include files} {
+	include "./foobar";
+}
+
diff --git a/lib/libvcc/vcc_compile.c b/lib/libvcc/vcc_compile.c
index 95eab85..8bc60ed 100644
--- a/lib/libvcc/vcc_compile.c
+++ b/lib/libvcc/vcc_compile.c
@@ -478,6 +478,8 @@ vcc_resolve_includes(struct vcc *tl)
 {
 	struct token *t, *t1, *t2;
 	struct source *sp;
+	struct vsb *vsb;
+	const char *p;
 
 	VTAILQ_FOREACH(t, &tl->tokens, list) {
 		if (t->tok != ID || !vcc_IdIs(t, "include"))
@@ -501,7 +503,31 @@ vcc_resolve_includes(struct vcc *tl)
 			return;
 		}
 
-		sp = vcc_file_source(tl->param, tl->sb, t1->dec);
+		if (t1->dec[0] == '.' && t1->dec[1] == '/') {
+			/*
+			 * Nested include filenames, starting with "./" are
+			 * resolved relative to the VCL file which contains
+			 * the include directive.
+			 */
+			if (t1->src->name[0] != '/') {
+				VSB_printf(tl->sb,
+				    "include \"./xxxxx\"; only works in "
+				    "nested VCL include files\n");
+				vcc_ErrWhere(tl, t1);
+				return;
+			}
+			vsb = VSB_new_auto();
+			AN(vsb);
+			p = strrchr(t1->src->name, '/');
+			AN(p);
+			VSB_bcat(vsb, t1->src->name, p - t1->src->name);
+			VSB_cat(vsb, t1->dec + 1);
+			AZ(VSB_finish(vsb));
+			sp = vcc_file_source(tl->param, tl->sb, VSB_data(vsb));
+			VSB_delete(vsb);
+		} else {
+			sp = vcc_file_source(tl->param, tl->sb, t1->dec);
+		}
 		if (sp == NULL) {
 			vcc_ErrWhere(tl, t1);
 			return;
@@ -761,7 +787,7 @@ VCC_Compile(const struct vcp *vcp, struct vsb *sb,
 
 	if (vclsrc != NULL) {
 		AZ(vclsrcfile);
-		sp = vcc_new_source(vclsrc, NULL, "input");
+		sp = vcc_new_source(vclsrc, NULL, "<input>");
 	} else {
 		AN(vclsrcfile);
 		sp = vcc_file_source(vcp, sb, vclsrcfile);



From lkarsten at varnish-software.com  Thu Jan 14 14:15:13 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:13 +0100
Subject: [4.1] d7e898c Fix up a couple of straggling issues with include
 "./xxx";
 If we know the filename of the -f argument, it should be possible to include
 "./" relative to it.
Message-ID: <E1aJigD-0005XA-0c@project.varnish-software.com>


commit d7e898c0479a8ad739ef645b6f0d1c3380797e48
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Jan 8 22:31:49 2016 +0000

    Fix up a couple of straggling issues with include "./xxx";  If we
    know the filename of the -f argument, it should be possible to
    include "./" relative to it.
    
    The footnote is that the include is resolved at lower privlevel
    than the -f file is read, so in really obscure cases you could
    have really obscure error messages.

diff --git a/bin/varnishd/mgt/mgt.h b/bin/varnishd/mgt/mgt.h
index a31bc6a..1aeba55 100644
--- a/bin/varnishd/mgt/mgt.h
+++ b/bin/varnishd/mgt/mgt.h
@@ -165,8 +165,8 @@ char *mgt_VccCompile(struct cli *, const char *vclname, const char *vclsrc,
 void mgt_vcc_init(void);
 
 void mgt_vcl_init(void);
-void mgt_vcc_default(struct cli *, const char *b_arg, const char *vclsrc,
-    int Cflag);
+void mgt_vcc_startup(struct cli *, const char *b_arg, const char *f_arg,
+    const char *vclsrc, int Cflag);
 int mgt_push_vcls_and_start(unsigned *status, char **p);
 int mgt_has_vcl(void);
 extern char *mgt_cc_cmd;
diff --git a/bin/varnishd/mgt/mgt_main.c b/bin/varnishd/mgt/mgt_main.c
index a7533e9..dab70fb 100644
--- a/bin/varnishd/mgt/mgt_main.c
+++ b/bin/varnishd/mgt/mgt_main.c
@@ -752,7 +752,7 @@ main(int argc, char * const *argv)
 	mgt_vcl_init();
 
 	if (b_arg != NULL || f_arg != NULL) {
-		mgt_vcc_default(cli, b_arg, vcl, C_flag);
+		mgt_vcc_startup(cli, b_arg, f_arg, vcl, C_flag);
 		if (C_flag && cli->result == CLIS_OK) {
 			AZ(VSB_finish(cli->sb));
 			fprintf(stderr, "%s\n", VSB_data(cli->sb));
diff --git a/bin/varnishd/mgt/mgt_vcl.c b/bin/varnishd/mgt/mgt_vcl.c
index 0b24c65..498c0b8 100644
--- a/bin/varnishd/mgt/mgt_vcl.c
+++ b/bin/varnishd/mgt/mgt_vcl.c
@@ -201,14 +201,15 @@ mgt_new_vcl(struct cli *cli, const char *vclname, const char *vclsrc,
 /*--------------------------------------------------------------------*/
 
 void
-mgt_vcc_default(struct cli *cli, const char *b_arg, const char *vclsrc,
-    int C_flag)
+mgt_vcc_startup(struct cli *cli, const char *b_arg, const char *f_arg,
+    const char *vclsrc, int C_flag)
 {
 	char buf[BUFSIZ];
 
 	if (b_arg == NULL) {
 		AN(vclsrc);
-		mgt_new_vcl(cli, "boot", vclsrc, NULL, NULL, C_flag);
+		AN(f_arg);
+		mgt_new_vcl(cli, "boot", vclsrc, f_arg, NULL, C_flag);
 		return;
 	}
 
@@ -218,7 +219,7 @@ mgt_vcc_default(struct cli *cli, const char *b_arg, const char *vclsrc,
 	    "backend default {\n"
 	    "    .host = \"%s\";\n"
 	    "}\n", b_arg);
-	mgt_new_vcl(cli, "boot", buf, NULL, NULL, C_flag);
+	mgt_new_vcl(cli, "boot", buf, "<-b argument>", NULL, C_flag);
 }
 
 /*--------------------------------------------------------------------*/
@@ -262,7 +263,7 @@ mcf_vcl_inline(struct cli *cli, const char * const *av, void *priv)
 		return;
 	}
 
-	mgt_new_vcl(cli, av[2], av[3], NULL, av[4], 0);
+	mgt_new_vcl(cli, av[2], av[3], "<vcl.inline>", av[4], 0);
 }
 
 void



From lkarsten at varnish-software.com  Thu Jan 14 14:15:13 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:13 +0100
Subject: [4.1] 44ae23e Make it possible to do
Message-ID: <E1aJigD-0005XK-3C@project.varnish-software.com>


commit 44ae23e39e8d7f6e099a4c79af3deaece8acc2b0
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Jan 8 23:08:20 2016 +0000

    Make it possible to do
    
    	import vmod from "/some/dir/"
    
    and have the default vmod shlib filename appended.

diff --git a/bin/varnishtest/tests/m00008.vtc b/bin/varnishtest/tests/m00008.vtc
index 833a282..10cb23d 100644
--- a/bin/varnishtest/tests/m00008.vtc
+++ b/bin/varnishtest/tests/m00008.vtc
@@ -23,3 +23,11 @@ varnish v1 -cliok "param.set vmod_dir /nowhere:${topbuild}/lib/libvmod_std/.libs
 varnish v1 -vcl+backend {
 	import std;
 }
+
+varnish v1 -cliok "param.set vcc_unsafe_path on"
+
+varnish v1 -cliok "param.set vmod_dir /nowhere:/else"
+
+varnish v1 -vcl+backend {
+	import std from "${topbuild}/lib/libvmod_std/.libs/";
+}
diff --git a/bin/varnishtest/tests/v00046.vtc b/bin/varnishtest/tests/v00046.vtc
index 567d3af..f87cd21 100644
--- a/bin/varnishtest/tests/v00046.vtc
+++ b/bin/varnishtest/tests/v00046.vtc
@@ -67,7 +67,7 @@ varnish v1 -errvcl {Found: 'zool' at} {
 shell "rm -f ${tmpdir}/a"
 shell "rm -f ${tmpdir}/_start.vcl"
 
-varnish v1 -errvcl {only works in nested VCL include files} {
+varnish v1 -errvcl {needs absolute filename of including file.} {
 	include "./foobar";
 }
 
diff --git a/lib/libvcc/vcc_compile.c b/lib/libvcc/vcc_compile.c
index 8bc60ed..62aaeae 100644
--- a/lib/libvcc/vcc_compile.c
+++ b/lib/libvcc/vcc_compile.c
@@ -511,8 +511,8 @@ vcc_resolve_includes(struct vcc *tl)
 			 */
 			if (t1->src->name[0] != '/') {
 				VSB_printf(tl->sb,
-				    "include \"./xxxxx\"; only works in "
-				    "nested VCL include files\n");
+				    "include \"./xxxxx\"; needs absolute "
+				    "filename of including file.\n");
 				vcc_ErrWhere(tl, t1);
 				return;
 			}
@@ -784,14 +784,14 @@ VCC_Compile(const struct vcp *vcp, struct vsb *sb,
 {
 	struct source *sp;
 	char *r;
+	CHECK_OBJ_NOTNULL(vcp, VCP_MAGIC);
+	AN(sb);
+	AN(vclsrcfile);
 
-	if (vclsrc != NULL) {
-		AZ(vclsrcfile);
-		sp = vcc_new_source(vclsrc, NULL, "<input>");
-	} else {
-		AN(vclsrcfile);
+	if (vclsrc != NULL)
+		sp = vcc_new_source(vclsrc, NULL, vclsrcfile);
+	else
 		sp = vcc_file_source(vcp, sb, vclsrcfile);
-	}
 	if (sp == NULL)
 		return (NULL);
 	r = vcc_CompileSource(vcp, sb, sp);
diff --git a/lib/libvcc/vcc_vmod.c b/lib/libvcc/vcc_vmod.c
index 0f90e4f..d1b8f26 100644
--- a/lib/libvcc/vcc_vmod.c
+++ b/lib/libvcc/vcc_vmod.c
@@ -117,7 +117,11 @@ vcc_ParseImport(struct vcc *tl)
 			return;
 		}
 		ExpectErr(tl, CSTR);
-		bprintf(fn, "%s", tl->t->dec);
+		p = strrchr(tl->t->dec, '/');
+		if (p != NULL && p[1] == '\0')
+			bprintf(fn, "%slibvmod_%.*s.so", tl->t->dec, PF(mod));
+		else
+			bprintf(fn, "%s", tl->t->dec);
 		vcc_NextToken(tl);
 	} else {
 		bprintf(fn, "libvmod_%.*s.so", PF(mod));



From lkarsten at varnish-software.com  Thu Jan 14 14:15:13 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:13 +0100
Subject: [4.1] 1667ef3 Use UPPER-case names for pseudo-enums constants
Message-ID: <E1aJigD-0005XV-5i@project.varnish-software.com>


commit 1667ef3032c1033118f5418adbbd73e46ddfd174
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Fri Jan 8 16:42:36 2016 +0100

    Use UPPER-case names for pseudo-enums constants

diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index 54effd1..11e4f54 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -47,11 +47,11 @@
 #include "vcli.h"
 #include "vcli_priv.h"
 
-static const char * const vcl_temp_init = "init";
-static const char * const vcl_temp_cold = "cold";
-static const char * const vcl_temp_warm = "warm";
-static const char * const vcl_temp_busy = "busy";
-static const char * const vcl_temp_cooling = "cooling";
+static const char * const VCL_TEMP_INIT = "init";
+static const char * const VCL_TEMP_COLD = "cold";
+static const char * const VCL_TEMP_WARM = "warm";
+static const char * const VCL_TEMP_BUSY = "busy";
+static const char * const VCL_TEMP_COOLING = "cooling";
 
 struct vcl {
 	unsigned		magic;
@@ -138,7 +138,7 @@ VCL_Get(struct vcl **vcc)
 		(void)usleep(100000);
 
 	CHECK_OBJ_NOTNULL(vcl_active, VCL_MAGIC);
-	assert(vcl_active->temp == vcl_temp_warm);
+	assert(vcl_active->temp == VCL_TEMP_WARM);
 	Lck_Lock(&vcl_mtx);
 	AN(vcl_active);
 	*vcc = vcl_active;
@@ -152,7 +152,7 @@ void
 VCL_Refresh(struct vcl **vcc)
 {
 	CHECK_OBJ_NOTNULL(vcl_active, VCL_MAGIC);
-	assert(vcl_active->temp == vcl_temp_warm);
+	assert(vcl_active->temp == VCL_TEMP_WARM);
 	if (*vcc == vcl_active)
 		return;
 	if (*vcc != NULL)
@@ -165,7 +165,7 @@ VCL_Ref(struct vcl *vcl)
 {
 
 	CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC);
-	assert(vcl->temp != vcl_temp_init && vcl->temp != vcl_temp_cold);
+	assert(vcl->temp != VCL_TEMP_INIT && vcl->temp != VCL_TEMP_COLD);
 	Lck_Lock(&vcl_mtx);
 	assert(vcl->busy > 0);
 	vcl->busy++;
@@ -201,17 +201,17 @@ VCL_AddBackend(struct vcl *vcl, struct backend *be)
 	CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC);
 	CHECK_OBJ_NOTNULL(be, BACKEND_MAGIC);
 
-	if (vcl->temp == vcl_temp_cooling)
+	if (vcl->temp == VCL_TEMP_COOLING)
 		return (1);
 
 	Lck_Lock(&vcl_mtx);
 	VTAILQ_INSERT_TAIL(&vcl->backend_list, be, vcl_list);
 	Lck_Unlock(&vcl_mtx);
 
-	if (vcl->temp == vcl_temp_warm || vcl->temp == vcl_temp_busy)
+	if (vcl->temp == VCL_TEMP_WARM || vcl->temp == VCL_TEMP_BUSY)
 		/* Only when adding backend to already warm VCL */
 		VBE_Event(be, VCL_EVENT_WARM);
-	else if (vcl->temp != vcl_temp_init)
+	else if (vcl->temp != VCL_TEMP_INIT)
 		WRONG("Dynamic Backends can only be added to warm VCLs");
 
 	return (0);
@@ -228,7 +228,7 @@ VCL_DelBackend(struct backend *be)
 	Lck_Lock(&vcl_mtx);
 	VTAILQ_REMOVE(&vcl->backend_list, be, vcl_list);
 	Lck_Unlock(&vcl_mtx);
-	if (vcl->temp == vcl_temp_warm)
+	if (vcl->temp == VCL_TEMP_WARM)
 		VBE_Event(be, VCL_EVENT_COLD);
 }
 
@@ -385,7 +385,7 @@ VRT_ref_vcl(VRT_CTX)
 
 	vcl = ctx->vcl;
 	CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC);
-	xxxassert(vcl->temp == vcl_temp_warm);
+	xxxassert(vcl->temp == VCL_TEMP_WARM);
 
 	Lck_Lock(&vcl_mtx);
 	vcl->refcount++;
@@ -401,8 +401,8 @@ VRT_rel_vcl(VRT_CTX)
 
 	vcl = ctx->vcl;
 	CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC);
-	assert(vcl->temp == vcl_temp_warm || vcl->temp == vcl_temp_busy ||
-	    vcl->temp == vcl_temp_cooling);
+	assert(vcl->temp == VCL_TEMP_WARM || vcl->temp == VCL_TEMP_BUSY ||
+	    vcl->temp == VCL_TEMP_COOLING);
 
 	Lck_Lock(&vcl_mtx);
 	assert(vcl->refcount > 0);
@@ -443,29 +443,29 @@ vcl_set_state(struct vcl *vcl, const char *state)
 
 	switch(state[0]) {
 	case '0':
-		assert(vcl->temp != vcl_temp_cold);
-		if (vcl->busy == 0 && (vcl->temp == vcl_temp_warm ||
-		    vcl->temp == vcl_temp_busy)) {
+		assert(vcl->temp != VCL_TEMP_COLD);
+		if (vcl->busy == 0 && (vcl->temp == VCL_TEMP_WARM ||
+		    vcl->temp == VCL_TEMP_BUSY)) {
 
-			vcl->temp = vcl->refcount ? vcl_temp_cooling :
-			    vcl_temp_cold;
+			vcl->temp = vcl->refcount ? VCL_TEMP_COOLING :
+			    VCL_TEMP_COLD;
 			AZ(vcl->conf->event_vcl(&ctx, VCL_EVENT_COLD));
 			vcl_BackendEvent(vcl, VCL_EVENT_COLD);
 		}
 		else if (vcl->busy)
-			vcl->temp = vcl_temp_busy;
+			vcl->temp = VCL_TEMP_BUSY;
 		else
-			vcl->temp = vcl->refcount ? vcl_temp_cooling :
-			    vcl_temp_cold;
+			vcl->temp = vcl->refcount ? VCL_TEMP_COOLING :
+			    VCL_TEMP_COLD;
 		break;
 	case '1':
-		assert(vcl->temp != vcl_temp_warm);
+		assert(vcl->temp != VCL_TEMP_WARM);
 		/* The warm VCL hasn't seen a cold event yet */
-		if (vcl->temp == vcl_temp_busy)
-			vcl->temp = vcl_temp_warm;
+		if (vcl->temp == VCL_TEMP_BUSY)
+			vcl->temp = VCL_TEMP_WARM;
 		/* The VCL must first reach a stable cold state */
-		else if (vcl->temp != vcl_temp_cooling) {
-			vcl->temp = vcl_temp_warm;
+		else if (vcl->temp != VCL_TEMP_COOLING) {
+			vcl->temp = VCL_TEMP_WARM;
 			(void)vcl->conf->event_vcl(&ctx, VCL_EVENT_WARM);
 			vcl_BackendEvent(vcl, VCL_EVENT_WARM);
 		}
@@ -507,7 +507,7 @@ VCL_Load(struct cli *cli, const char *name, const char *fn, const char *state)
 	XXXAN(vcl->loaded_name);
 	VTAILQ_INIT(&vcl->backend_list);
 
-	vcl->temp = vcl_temp_init;
+	vcl->temp = VCL_TEMP_INIT;
 
 	INIT_OBJ(&ctx, VRT_CTX_MAGIC);
 	ctx.method = VCL_MET_INIT;
@@ -581,10 +581,10 @@ VCL_Poll(void)
 
 	ASSERT_CLI();
 	VTAILQ_FOREACH_SAFE(vcl, &vcl_head, list, vcl2) {
-		if (vcl->temp == vcl_temp_busy ||
-		    vcl->temp == vcl_temp_cooling)
+		if (vcl->temp == VCL_TEMP_BUSY ||
+		    vcl->temp == VCL_TEMP_COOLING)
 			vcl_set_state(vcl, "0");
-		if (vcl->discard && vcl->temp == vcl_temp_cold)
+		if (vcl->discard && vcl->temp == VCL_TEMP_COLD)
 			VCL_Nuke(vcl);
 	}
 }
@@ -672,7 +672,7 @@ ccf_config_use(struct cli *cli, const char * const *av, void *priv)
 	AZ(priv);
 	vcl = vcl_find(av[2]);
 	AN(vcl);				// MGT ensures this
-	assert(vcl->temp == vcl_temp_warm);	// MGT ensures this
+	assert(vcl->temp == VCL_TEMP_WARM);	// MGT ensures this
 	INIT_OBJ(&ctx, VRT_CTX_MAGIC);
 	ctx.handling = &hand;
 	vsb = VSB_new_auto();



From lkarsten at varnish-software.com  Thu Jan 14 14:15:13 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:13 +0100
Subject: [4.1] 5016d8e Make vcl_set_state accept a ctx instead of a vcl
Message-ID: <E1aJigD-0005Xf-7z@project.varnish-software.com>


commit 5016d8eb8d7723c85a4bcbddb8daa7803ddbd5e6
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Fri Dec 4 11:14:29 2015 +0100

    Make vcl_set_state accept a ctx instead of a vcl
    
    This paves the way towards failing VCL_EVENT_WARM events in VMODs. If
    setting the temperature fails, we may need to carry a message back to
    the CLI and other bits of context.

diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index 11e4f54..e19c9ce 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -429,17 +429,16 @@ vcl_find(const char *name)
 }
 
 static void
-vcl_set_state(struct vcl *vcl, const char *state)
+vcl_set_state(VRT_CTX, const char *state)
 {
-	struct vrt_ctx ctx;
-	unsigned hand = 0;
+	struct vcl *vcl;
 
 	ASSERT_CLI();
-	AN(vcl->temp);
+	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
+	AN(ctx->handling);
 
-	INIT_OBJ(&ctx, VRT_CTX_MAGIC);
-	ctx.handling = &hand;
-	ctx.vcl = vcl;
+	vcl = ctx->vcl;
+	AN(vcl->temp);
 
 	switch(state[0]) {
 	case '0':
@@ -449,7 +448,7 @@ vcl_set_state(struct vcl *vcl, const char *state)
 
 			vcl->temp = vcl->refcount ? VCL_TEMP_COOLING :
 			    VCL_TEMP_COLD;
-			AZ(vcl->conf->event_vcl(&ctx, VCL_EVENT_COLD));
+			AZ(vcl->conf->event_vcl(ctx, VCL_EVENT_COLD));
 			vcl_BackendEvent(vcl, VCL_EVENT_COLD);
 		}
 		else if (vcl->busy)
@@ -466,7 +465,7 @@ vcl_set_state(struct vcl *vcl, const char *state)
 		/* The VCL must first reach a stable cold state */
 		else if (vcl->temp != VCL_TEMP_COOLING) {
 			vcl->temp = VCL_TEMP_WARM;
-			(void)vcl->conf->event_vcl(&ctx, VCL_EVENT_WARM);
+			(void)vcl->conf->event_vcl(ctx, VCL_EVENT_WARM);
 			vcl_BackendEvent(vcl, VCL_EVENT_WARM);
 		}
 		break;
@@ -529,7 +528,7 @@ VCL_Load(struct cli *cli, const char *name, const char *fn, const char *state)
 		return (1);
 	}
 	VSB_delete(vsb);
-	vcl_set_state(vcl, state);
+	vcl_set_state(&ctx, state);
 	bprintf(vcl->state, "%s", state + 1);
 	assert(hand == VCL_RET_OK);
 	VCLI_Out(cli, "Loaded \"%s\" as \"%s\"", fn , name);
@@ -577,13 +576,19 @@ VCL_Nuke(struct vcl *vcl)
 void
 VCL_Poll(void)
 {
+	struct vrt_ctx ctx;
 	struct vcl *vcl, *vcl2;
+	unsigned hand;
 
 	ASSERT_CLI();
 	VTAILQ_FOREACH_SAFE(vcl, &vcl_head, list, vcl2) {
 		if (vcl->temp == VCL_TEMP_BUSY ||
-		    vcl->temp == VCL_TEMP_COOLING)
-			vcl_set_state(vcl, "0");
+		    vcl->temp == VCL_TEMP_COOLING) {
+			INIT_OBJ(&ctx, VRT_CTX_MAGIC);
+			ctx.vcl = vcl;
+			ctx.handling = &hand;
+			vcl_set_state(&ctx, "0");
+		}
 		if (vcl->discard && vcl->temp == VCL_TEMP_COLD)
 			VCL_Nuke(vcl);
 	}
@@ -625,17 +630,21 @@ ccf_config_load(struct cli *cli, const char * const *av, void *priv)
 static void __match_proto__(cli_func_t)
 ccf_config_state(struct cli *cli, const char * const *av, void *priv)
 {
-	struct vcl *vcl;
+	struct vrt_ctx ctx;
+	unsigned hand;
+
+	INIT_OBJ(&ctx, VRT_CTX_MAGIC);
+	ctx.handling = &hand;
 
 	(void)cli;
 	AZ(priv);
 	ASSERT_CLI();
 	AN(av[2]);
 	AN(av[3]);
-	vcl = vcl_find(av[2]);
-	AN(vcl);			// MGT ensures this
-	vcl_set_state(vcl, av[3]);
-	bprintf(vcl->state, "%s", av[3] + 1);
+	ctx.vcl = vcl_find(av[2]);
+	AN(ctx.vcl);			// MGT ensures this
+	vcl_set_state(&ctx, av[3]);
+	bprintf(ctx.vcl->state, "%s", av[3] + 1);
 }
 
 static void __match_proto__(cli_func_t)



From lkarsten at varnish-software.com  Thu Jan 14 14:15:13 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:15:13 +0100
Subject: [4.1] e80a754 Start preparing for 4.1.1-beta1.
Message-ID: <E1aJigD-0005Xp-9z@project.varnish-software.com>


commit e80a7549db38dad16fdd7ec6c46f65c03cf8d647
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu Jan 14 15:10:19 2016 +0100

    Start preparing for 4.1.1-beta1.

diff --git a/configure.ac b/configure.ac
index 227a9bf..abfdafc 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1,8 +1,8 @@
 AC_PREREQ(2.59)
 AC_COPYRIGHT([Copyright (c) 2006 Verdens Gang AS
-Copyright (c) 2006-2015 Varnish Software AS])
+Copyright (c) 2006-2016 Varnish Software AS])
 AC_REVISION([$Id$])
-AC_INIT([Varnish], [4.1.0], [varnish-dev at varnish-cache.org])
+AC_INIT([Varnish], [4.1.1-beta1], [varnish-dev at varnish-cache.org])
 AC_CONFIG_SRCDIR(include/miniobj.h)
 AC_CONFIG_HEADERS([config.h])
 AC_CONFIG_MACRO_DIR([m4])



From lkarsten at varnish-software.com  Thu Jan 14 14:33:51 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:33:51 +0100
Subject: [master] 0b8f408 Increase VRT_MINOR_VERSION.
Message-ID: <E1aJiyF-000298-S0@project.varnish-software.com>


commit 0b8f4087c6f8e16c4b023ba592f36a2345739d9f
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu Jan 14 15:33:40 2016 +0100

    Increase VRT_MINOR_VERSION.

diff --git a/include/vrt.h b/include/vrt.h
index 96a3c64..d3113a8 100644
--- a/include/vrt.h
+++ b/include/vrt.h
@@ -41,13 +41,11 @@
  * changes to consider with next VRT_MAJOR_VERSION bump:
  * - cache_vrt.c: -> ssize_t VRT_CacheReqBody(VRT_CTX, size_t)
  *
- * changes to consider with next VRT_MINOR_VERSION bump for 4.1:
- * - cache_vcl.c: -> VRT_ref_vcl, VRT_rel_vcl
  */
 
 #define VRT_MAJOR_VERSION	3U
 
-#define VRT_MINOR_VERSION	0U
+#define VRT_MINOR_VERSION	1U
 
 
 /***********************************************************************/



From lkarsten at varnish-software.com  Thu Jan 14 14:34:47 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:34:47 +0100
Subject: [4.1] 602156c Increase VRT_MINOR_VERSION.
Message-ID: <E1aJiz9-0002KR-Se@project.varnish-software.com>


commit 602156c02b1c5c6a4e4c935e2fbbd2ff253704c2
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu Jan 14 15:33:40 2016 +0100

    Increase VRT_MINOR_VERSION.
    
    Conflicts:
    	include/vrt.h

diff --git a/include/vrt.h b/include/vrt.h
index 2203bd8..4cfd71d 100644
--- a/include/vrt.h
+++ b/include/vrt.h
@@ -44,7 +44,7 @@
 
 #define VRT_MAJOR_VERSION	3U
 
-#define VRT_MINOR_VERSION	0U
+#define VRT_MINOR_VERSION	1U
 
 
 /***********************************************************************/



From lkarsten at varnish-software.com  Thu Jan 14 14:48:01 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 15:48:01 +0100
Subject: [4.1] 166516e Ignore If-Modified-Since if we have If-None-Match
Message-ID: <E1aJjBx-0004aV-GE@project.varnish-software.com>


commit 166516e7f50ecf4fe5e996dba44252a2e2ad2c56
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Sat Dec 12 13:32:28 2015 +0000

    Ignore If-Modified-Since if we have If-None-Match
    
    The latter takes precedence as per RFC 7232.  Also if we have an I-N-M
    but no entity don't check I-M-S, just do a non-conditional request.

diff --git a/bin/varnishd/cache/cache_rfc2616.c b/bin/varnishd/cache/cache_rfc2616.c
index 0c0e127..cb089cf 100644
--- a/bin/varnishd/cache/cache_rfc2616.c
+++ b/bin/varnishd/cache/cache_rfc2616.c
@@ -256,33 +256,31 @@ RFC2616_Do_Cond(const struct req *req)
 {
 	const char *p, *e;
 	double ims, lm;
-	int do_cond = 0;
 
 	/*
-	 * RFC 2616 13.3.4 states we need to match both ETag and
-	 * If-Modified-Since if present.
+	 * We MUST ignore If-Modified-Since if we have an If-None-Match
+	 * header [RFC7232 3.3 p16].
 	 */
-	if (http_GetHdr(req->http, H_If_None_Match, &p) &&
-	    http_GetHdr(req->resp, H_ETag, &e)) {
+	if (http_GetHdr(req->http, H_If_None_Match, &p)) {
+		if (!http_GetHdr(req->resp, H_ETag, &e))
+			return (0);
 		if (http_GetHdr(req->http, H_Range, NULL))
-			do_cond = rfc2616_strong_compare(p, e);
+			return (rfc2616_strong_compare(p, e));
 		else
-			do_cond = rfc2616_weak_compare(p, e);
-		if (!do_cond)
-			return (0);
+			return (rfc2616_weak_compare(p, e));
 	}
 
 	if (http_GetHdr(req->http, H_If_Modified_Since, &p)) {
 		ims = VTIM_parse(p);
-		if (!ims || ims > req->t_req)	/* [RFC2616 14.25] */
+		if (!ims || ims > req->t_req)	/* [RFC7232 3.3 p16] */
 			return (0);
 		AZ(ObjGetDouble(req->wrk, req->objcore, OA_LASTMODIFIED, &lm));
 		if (lm > ims)
 			return (0);
-		do_cond = 1;
+		return (1);
 	}
 
-	return (do_cond);
+	return (0);
 }
 
 /*--------------------------------------------------------------------*/
diff --git a/bin/varnishtest/tests/c00026.vtc b/bin/varnishtest/tests/c00026.vtc
index 49e4984..82019bd 100644
--- a/bin/varnishtest/tests/c00026.vtc
+++ b/bin/varnishtest/tests/c00026.vtc
@@ -6,6 +6,11 @@ server s1 {
 	txresp -hdr {ETag: "123456789"} \
 	    -hdr "Last-Modified: Thu, 26 Jun 2008 12:00:01 GMT" \
 	    -bodylen 10
+
+	rxreq
+	expect req.url == /other
+	txresp -hdr "Last-Modified: Thu, 26 Jun 2008 12:00:01 GMT" \
+	    -bodylen 10
 } -start
 
 varnish v1 -vcl+backend { } -start
@@ -19,8 +24,8 @@ client c1 {
 
 	txreq -hdr "If-Modified-Since: Thu, 26 Jun 2008 12:00:00 GMT" \
 	    -hdr {If-None-Match: "123456789"}
-	rxresp
-	expect resp.status == 200
+	rxresp -no_obj
+	expect resp.status == 304
 
 	txreq -hdr "If-Modified-Since: Thu, 26 Jun 2008 12:00:01 GMT" \
 	    -hdr {If-None-Match: "12345678"}
@@ -31,4 +36,10 @@ client c1 {
 	    -hdr {If-None-Match: "123456789"}
 	rxresp -no_obj
 	expect resp.status == 304
+
+	txreq -url /other \
+	    -hdr "If-Modified-Since: Thu, 26 Jun 2008 12:00:01 GMT" \
+	    -hdr {If-None-Match: "123456789"}
+	rxresp
+	expect resp.status == 200
 } -run
diff --git a/bin/varnishtest/tests/r00907.vtc b/bin/varnishtest/tests/r00907.vtc
deleted file mode 100644
index 4bd767f..0000000
--- a/bin/varnishtest/tests/r00907.vtc
+++ /dev/null
@@ -1,31 +0,0 @@
-varnishtest "Ticket #907 200/304 handling with Etags + Last-Modified"
-
-server s1 {
-	rxreq
-	txresp \
-		-hdr {ETag: "saengei1Ohshicich4iteesu"} \
-		-hdr "Last-Modified: Tue, 20 Sep 2011 18:55:00 GMT"
-} -start
-
-varnish v1 -vcl+backend {
-	sub vcl_deliver {
-		set resp.http.x-timestamp = now;
-	}
-} -start
-
-client c1 {
-	txreq -hdr {If-None-Match: "saengei1Ohshicich4iteesu"}
-	rxresp -no_obj
-	expect resp.status == 304
-
-	txreq -hdr {If-None-Match: "saengei1Ohshicich4iteesu"} \
-	      -hdr "If-Modified-Since: Tue, 20 Sep 2011 18:54:59 GMT"
-	rxresp -no_obj
-	expect resp.status == 200
-
-	txreq -hdr {If-None-Match: "saengei1Ohshicich4iteesu"} \
-	      -hdr "If-Modified-Since: Tue, 20 Sep 2011 18:55:00 GMT"
-	rxresp -no_obj
-	expect resp.status == 304
-
-} -run



From lkarsten at varnish-software.com  Thu Jan 14 15:27:18 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 14 Jan 2016 16:27:18 +0100
Subject: [master] db68407 Add initial list of 4.1.1-beta1 changes.
Message-ID: <E1aJjny-0001Lv-I6@project.varnish-software.com>


commit db68407e7d9ab7539a1c8ad1cf49a773a6cd1648
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu Jan 14 16:26:52 2016 +0100

    Add initial list of 4.1.1-beta1 changes.

diff --git a/doc/changes.rst b/doc/changes.rst
index 04aed34..c182ec8 100644
--- a/doc/changes.rst
+++ b/doc/changes.rst
@@ -1,3 +1,43 @@
+======================================
+Varnish Cache 4.1.1-beta1 (unreleased)
+======================================
+
+Changes since 4.1.0:
+
+- Format of "ban.list" has changed slightly.
+- [varnishncsa] -w is now required when running deamonized.
+- [varnishncsa] Log format can now be read from file.
+- Port fields extracted from PROXY1 header now work as expected.
+- New VCL state "busy" introduced (mostly for vmod writers.)
+- New configure option: --enable-pcre-jit. Enabled by default.
+- Last traces of varnishreplay removed.
+- If-Modified-Since is now ignored if we have If-None-Match.
+- Zero Content-Length is no longer sent on empty 204 and 304 responses.
+
+
+Bugs fixed
+----------
+
+- 1763_ - Restart epoll_wait on EINTR error
+- 1788_ - ObjIter has terrible performance profile when busyobj != NULL
+- 1798_ - Varnish requests painfully slow with large files
+- 1816_ - Use a weak comparison function for If-None-Match
+- 1818_ - Allow grace-hits on hit-for-pass objects, [..]
+- 1821_ - Always slim private & pass objects after delivery.
+- 1823_ - Rush the objheader if there is a waiting list when it is deref'ed.
+- 1826_ - Ignore 0 Content-Lengths in 204 responses
+
+.. _1763: https://www.varnish-cache.org/trac/ticket/1763
+.. _1788: https://www.varnish-cache.org/trac/ticket/1788
+.. _1798: https://www.varnish-cache.org/trac/ticket/1798
+.. _1816: https://www.varnish-cache.org/trac/ticket/1816
+.. _1818: https://www.varnish-cache.org/trac/ticket/1818
+.. _1821: https://www.varnish-cache.org/trac/ticket/1821
+.. _1823: https://www.varnish-cache.org/trac/ticket/1823
+.. _1826: https://www.varnish-cache.org/trac/ticket/1826
+
+
+
 ================================
 Varnish Cache 4.1.0 (2015-09-30)
 ================================



From lkarsten at varnish-software.com  Fri Jan 15 11:39:04 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Fri, 15 Jan 2016 12:39:04 +0100
Subject: [4.1] 16c34d6 Polish
Message-ID: <E1aK2ie-0000vf-9W@project.varnish-software.com>


commit 16c34d62a627b5e7060112d06fb5f921b44c9e9c
Author: Ond?ej Nov? <ondrej.novy at firma.seznam.cz>
Date:   Fri Jan 8 16:07:10 2016 +0000

    Polish

diff --git a/lib/libvarnish/vsub.c b/lib/libvarnish/vsub.c
index dd0cb9d..90b1c7c 100644
--- a/lib/libvarnish/vsub.c
+++ b/lib/libvarnish/vsub.c
@@ -64,7 +64,7 @@ VSUB_closefrom(int fd)
 #else
 	int i = sysconf(_SC_OPEN_MAX);
 	assert(i > 0);
-	for (i = sysconf(_SC_OPEN_MAX); i > fd; i--)
+	for (; i > fd; i--)
 		(void)close(i);
 #endif
 }



From lkarsten at varnish-software.com  Fri Jan 15 11:39:04 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Fri, 15 Jan 2016 12:39:04 +0100
Subject: [4.1] c17c701 Add beresp.was_304 to changes list.
Message-ID: <E1aK2ie-0000vm-B6@project.varnish-software.com>


commit c17c701b6ecf89a331d1c495991f6e2997d31a12
Author: Dag Haavi Finstad <daghf at varnish-software.com>
Date:   Wed Sep 30 15:28:33 2015 +0200

    Add beresp.was_304 to changes list.

diff --git a/doc/sphinx/whats-new/changes.rst b/doc/sphinx/whats-new/changes.rst
index 647b102..22e61a0 100644
--- a/doc/sphinx/whats-new/changes.rst
+++ b/doc/sphinx/whats-new/changes.rst
@@ -158,4 +158,8 @@ Other noteworthy small changes
 * Varnish will now use the ``stale-while-revalidate`` defined in RFC5861
   to set object grace time.
 * -smalloc storage is now recommended over -sfile on Linux systems.
+* New VCL variable ``beresp.was_304`` has been introduced in
+  ``vcl_backend_response``. Will be set to ``true`` if the response
+  from the backend was a positive result of a conditional fetch (``304
+  Not Modified``).
 



From lkarsten at varnish-software.com  Fri Jan 15 11:39:04 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Fri, 15 Jan 2016 12:39:04 +0100
Subject: [4.1] 32af38d Add a final backstop,
 so we absolutely 100% certainly do not try to delete a objhead while
 it still has a waiting list, by forcing the last ref holder to rush the WL.
Message-ID: <E1aK2ie-0000vv-Ch@project.varnish-software.com>


commit 32af38d43c9cefa10ef24775ef999fbd61fd49ed
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu Jan 7 00:03:35 2016 +0000

    Add a final backstop, so we absolutely 100% certainly do not
    try to delete a objhead while it still has a waiting list,
    by forcing the last ref holder to rush the WL.
    
    Since the hasher owns the refcounts on objhead, we cannot just
    mingle req and objcore refcounts.
    
    Fortunately we don't need to add another refcounter, because all
    we really care about is the wl being empty when we drop the last
    ref.
    
    The wl/hsh_rush() mechanism will work differently with different
    thread-scheduling schenarios, and I cannot definitively rule out
    that we can drop the last ref on an oh, while there are still req's
    on the waiting list.
    
    Given that, and the existence proof in ticket #1823's race, this
    might have been the indicated memory-trampler.
    
    Conflicts:
    	bin/varnishd/cache/cache_hash.c

diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index 0bb827d..18b9628 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -820,10 +820,25 @@ HSH_DerefObjHead(struct worker *wrk, struct objhead **poh)
 		oh->refcnt--;
 		Lck_Unlock(&oh->mtx);
 		return(1);
-	} else if (oh->waitinglist != NULL) {
+	}
+
+	/*
+	 * Make absolutely certain that we do not let the final ref
+	 * disappear until the waitinglist is empty.
+	 * This is necessary because the req's on the waiting list do
+	 * not hold any ref on the objhead of their own, and we cannot
+	 * just make the hold the same ref's as objcore, that would
+	 * confuse hashers.
+	 */
+	while (oh->waitinglist != NULL) {
 		Lck_Lock(&oh->mtx);
+		assert(oh->refcnt > 0);
+		r = oh->refcnt;
 		hsh_rush(wrk, oh);
 		Lck_Unlock(&oh->mtx);
+		if (r > 1)
+			break;
+		usleep(100000);
 	}
 
 	assert(oh->refcnt > 0);
diff --git a/bin/varnishd/hash/hash_critbit.c b/bin/varnishd/hash/hash_critbit.c
index c4239b6..944985d 100644
--- a/bin/varnishd/hash/hash_critbit.c
+++ b/bin/varnishd/hash/hash_critbit.c
@@ -394,9 +394,7 @@ hcb_start(void)
 static int __match_proto__(hash_deref_f)
 hcb_deref(struct objhead *oh)
 {
-	int r;
 
-	r = 1;
 	CHECK_OBJ_NOTNULL(oh, OBJHEAD_MAGIC);
 	Lck_Lock(&oh->mtx);
 	assert(oh->refcnt > 0);
@@ -413,7 +411,7 @@ hcb_deref(struct objhead *oh)
 #ifdef PHK
 	fprintf(stderr, "hcb_defef %d %d <%s>\n", __LINE__, r, oh->hash);
 #endif
-	return (r);
+	return (1);
 }
 
 static struct objhead * __match_proto__(hash_lookup_f)



From lkarsten at varnish-software.com  Fri Jan 15 13:08:13 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Fri, 15 Jan 2016 14:08:13 +0100
Subject: [master] c7c0351 Correct URLs used in packaging links.
Message-ID: <E1aK46v-0001NB-Bc@project.varnish-software.com>


commit c7c03513a67c6f7bbe7a6c1d236ca16f30dc2783
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Fri Jan 15 14:07:41 2016 +0100

    Correct URLs used in packaging links.

diff --git a/README.Packaging b/README.Packaging
index dbe4680..6663f88 100644
--- a/README.Packaging
+++ b/README.Packaging
@@ -13,19 +13,11 @@ Official packages
 -----------------
 
 The official Debian and Redhat packages are built by the Varnish Cache team
-and made available on http://repo.varnish-cache.org/ .
+and made available on https://repo.varnish-cache.org/ .
 
-Packaging files for Debian:
+Packaging files and scripts for Debian and Redhat:
 
-    git://git.varnish-cache.org/varnish-cache-debian.git
-
-Packaging files for Redhat:
-
-    https://github.com/varnish/varnish-cache-redhat
-
-Scripts that run the builds are available on:
-
-    https://github.com/varnish/pkg-varnish-cache
+    https://github.com/varnishcache/pkg-varnish-cache
 
 
 Third-party packages



From fgsch at lodoss.net  Fri Jan 15 13:10:12 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Fri, 15 Jan 2016 14:10:12 +0100
Subject: [master] 97ae3f8 Minor corrections
Message-ID: <E1aK48q-0001iW-LE@project.varnish-software.com>


commit 97ae3f8772039bf9d0450bb793399df5ec6bab98
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Fri Jan 15 13:09:55 2016 +0000

    Minor corrections

diff --git a/doc/changes.rst b/doc/changes.rst
index c182ec8..529e71b 100644
--- a/doc/changes.rst
+++ b/doc/changes.rst
@@ -8,11 +8,10 @@ Changes since 4.1.0:
 - [varnishncsa] -w is now required when running deamonized.
 - [varnishncsa] Log format can now be read from file.
 - Port fields extracted from PROXY1 header now work as expected.
-- New VCL state "busy" introduced (mostly for vmod writers.)
-- New configure option: --enable-pcre-jit. Enabled by default.
+- New VCL state "busy" introduced (mostly for vmod writers).
 - Last traces of varnishreplay removed.
 - If-Modified-Since is now ignored if we have If-None-Match.
-- Zero Content-Length is no longer sent on empty 204 and 304 responses.
+- Zero Content-Length is no longer sent on 304 responses.
 
 
 Bugs fixed
@@ -37,7 +36,6 @@ Bugs fixed
 .. _1826: https://www.varnish-cache.org/trac/ticket/1826
 
 
-
 ================================
 Varnish Cache 4.1.0 (2015-09-30)
 ================================
@@ -2641,7 +2639,7 @@ varnishreplay
 varnishstat
 -----------
 
--  Don't print lifetime averages when it doesn't make any sense?for
+-  Don't print lifetime averages when it doesn't make any sense, for
    instance, there is no point in dividing the amount in bytes of free
    cache space by the lifetime in seconds of the varnishd process.
 
@@ -2774,7 +2772,7 @@ varnishncsa
 Documentation
 -------------
 
--  The documentation?especially the VCL documentation?has been greatly
+-  The documentation, especially the VCL documentation, has been greatly
    extended and improved.
 
 Build system



From fgsch at lodoss.net  Fri Jan 15 14:02:41 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Fri, 15 Jan 2016 15:02:41 +0100
Subject: [master] 68a58ef More changes
Message-ID: <E1aK4xd-0000QR-LE@project.varnish-software.com>


commit 68a58ef6ff212c59140b71f4fbcac0e66f91639d
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Fri Jan 15 14:01:47 2016 +0000

    More changes

diff --git a/doc/changes.rst b/doc/changes.rst
index 529e71b..afa024f 100644
--- a/doc/changes.rst
+++ b/doc/changes.rst
@@ -12,6 +12,9 @@ Changes since 4.1.0:
 - Last traces of varnishreplay removed.
 - If-Modified-Since is now ignored if we have If-None-Match.
 - Zero Content-Length is no longer sent on 304 responses.
+- vcl_dir and vmod_dir now accept a colon separated list of directories.
+- Nested includes starting with "./" are relative to the including
+  VCL file now.
 
 
 Bugs fixed



From lkarsten at varnish-software.com  Fri Jan 15 15:32:18 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Fri, 15 Jan 2016 16:32:18 +0100
Subject: [4.1] 1fd0d39 Add initial list of 4.1.1-beta1 changes.
Message-ID: <E1aK6MM-0000IN-5z@project.varnish-software.com>


commit 1fd0d39445f67f32cb15c70a89fa5c74a5119c4d
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu Jan 14 16:26:52 2016 +0100

    Add initial list of 4.1.1-beta1 changes.

diff --git a/doc/changes.rst b/doc/changes.rst
index 04aed34..c182ec8 100644
--- a/doc/changes.rst
+++ b/doc/changes.rst
@@ -1,3 +1,43 @@
+======================================
+Varnish Cache 4.1.1-beta1 (unreleased)
+======================================
+
+Changes since 4.1.0:
+
+- Format of "ban.list" has changed slightly.
+- [varnishncsa] -w is now required when running deamonized.
+- [varnishncsa] Log format can now be read from file.
+- Port fields extracted from PROXY1 header now work as expected.
+- New VCL state "busy" introduced (mostly for vmod writers.)
+- New configure option: --enable-pcre-jit. Enabled by default.
+- Last traces of varnishreplay removed.
+- If-Modified-Since is now ignored if we have If-None-Match.
+- Zero Content-Length is no longer sent on empty 204 and 304 responses.
+
+
+Bugs fixed
+----------
+
+- 1763_ - Restart epoll_wait on EINTR error
+- 1788_ - ObjIter has terrible performance profile when busyobj != NULL
+- 1798_ - Varnish requests painfully slow with large files
+- 1816_ - Use a weak comparison function for If-None-Match
+- 1818_ - Allow grace-hits on hit-for-pass objects, [..]
+- 1821_ - Always slim private & pass objects after delivery.
+- 1823_ - Rush the objheader if there is a waiting list when it is deref'ed.
+- 1826_ - Ignore 0 Content-Lengths in 204 responses
+
+.. _1763: https://www.varnish-cache.org/trac/ticket/1763
+.. _1788: https://www.varnish-cache.org/trac/ticket/1788
+.. _1798: https://www.varnish-cache.org/trac/ticket/1798
+.. _1816: https://www.varnish-cache.org/trac/ticket/1816
+.. _1818: https://www.varnish-cache.org/trac/ticket/1818
+.. _1821: https://www.varnish-cache.org/trac/ticket/1821
+.. _1823: https://www.varnish-cache.org/trac/ticket/1823
+.. _1826: https://www.varnish-cache.org/trac/ticket/1826
+
+
+
 ================================
 Varnish Cache 4.1.0 (2015-09-30)
 ================================



From lkarsten at varnish-software.com  Mon Jan 18 13:59:47 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Mon, 18 Jan 2016 14:59:47 +0100
Subject: [4.1] d8ac05a Handle missing waiting list gracefully.
Message-ID: <E1aLALT-0001yv-6F@project.varnish-software.com>


commit d8ac05a3c4b7b5838eee2d94150e6853cfc64d43
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Mon Jan 18 14:57:37 2016 +0100

    Handle missing waiting list gracefully.
    
    In 5c8268062bf06a2700dd27331c29c48d650c9197 the checks are relaxed,
    but this commit (or part of) was not brought into 4.1.
    
    Fixes: #1842

diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index 18b9628..4355cc6 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -523,7 +523,9 @@ hsh_rush(struct worker *wrk, struct objhead *oh)
 	CHECK_OBJ_NOTNULL(oh, OBJHEAD_MAGIC);
 	Lck_AssertHeld(&oh->mtx);
 	wl = oh->waitinglist;
-	CHECK_OBJ_NOTNULL(wl, WAITINGLIST_MAGIC);
+	CHECK_OBJ_ORNULL(wl, WAITINGLIST_MAGIC);
+	if (wl == NULL)
+		return;
 	for (u = 0; u < cache_param->rush_exponent; u++) {
 		req = VTAILQ_FIRST(&wl->list);
 		if (req == NULL)



From phk at FreeBSD.org  Mon Jan 18 14:07:26 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Mon, 18 Jan 2016 15:07:26 +0100
Subject: [master] 00736b7 Use a unique (and now reserved) VCL name of -C
 testing.
Message-ID: <E1aLASs-0002zT-Om@project.varnish-software.com>


commit 00736b717316cac500c9508d6d46ac5fb6e0eca4
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Jan 18 14:06:45 2016 +0000

    Use a unique (and now reserved) VCL name of -C testing.
    
    Fixes:	#1802

diff --git a/bin/varnishd/mgt/mgt_vcl.c b/bin/varnishd/mgt/mgt_vcl.c
index 3766eb8..46359df 100644
--- a/bin/varnishd/mgt/mgt_vcl.c
+++ b/bin/varnishd/mgt/mgt_vcl.c
@@ -173,6 +173,9 @@ mgt_new_vcl(struct cli *cli, const char *vclname, const char *vclsrc,
 
 	AN(cli);
 
+	if (C_flag)
+		vclname = ".Cflagtest";
+
 	if (state == NULL)
 		state = "auto";
 



From phk at FreeBSD.org  Mon Jan 18 14:38:05 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Mon, 18 Jan 2016 15:38:05 +0100
Subject: [master] 7d97972 Qualify the -C test-vcl name with the PID of the
 process, so two concurrent -C tests don't collided.
Message-ID: <E1aLAwX-0006LR-T2@project.varnish-software.com>


commit 7d979720f14d7c22a5a3b3498fa4dd173af94958
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Jan 18 14:37:24 2016 +0000

    Qualify the -C test-vcl name with the PID of the process, so two
    concurrent -C tests don't collided.
    
    Problem spotted by:	fgs

diff --git a/bin/varnishd/mgt/mgt_vcl.c b/bin/varnishd/mgt/mgt_vcl.c
index 46359df..b651f83 100644
--- a/bin/varnishd/mgt/mgt_vcl.c
+++ b/bin/varnishd/mgt/mgt_vcl.c
@@ -170,11 +170,14 @@ mgt_new_vcl(struct cli *cli, const char *vclname, const char *vclsrc,
 	unsigned status;
 	char *lib, *p;
 	struct vclprog *vp;
+	char buf[32];
 
 	AN(cli);
 
-	if (C_flag)
-		vclname = ".Cflagtest";
+	if (C_flag) {
+		bprintf(buf, ".CflagTest.%d", (int)getpid());
+		vclname = buf;
+	}
 
 	if (state == NULL)
 		state = "auto";
@@ -190,6 +193,7 @@ mgt_new_vcl(struct cli *cli, const char *vclname, const char *vclsrc,
 	if (lib == NULL)
 		return;
 
+	AZ(C_flag);
 	vp = mgt_vcl_add(vclname, lib, state);
 	free(lib);
 



From lkarsten at varnish-software.com  Mon Jan 18 16:38:16 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Mon, 18 Jan 2016 17:38:16 +0100
Subject: [4.1] 8de2086 Use a unique (and now reserved) VCL name of -C testing.
Message-ID: <E1aLCoq-0000m5-Ax@project.varnish-software.com>


commit 8de208636c4425cbc3257e1ce509794835dfdd73
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Jan 18 14:06:45 2016 +0000

    Use a unique (and now reserved) VCL name of -C testing.
    
    Fixes:	#1802

diff --git a/bin/varnishd/mgt/mgt_vcl.c b/bin/varnishd/mgt/mgt_vcl.c
index 498c0b8..e533c69 100644
--- a/bin/varnishd/mgt/mgt_vcl.c
+++ b/bin/varnishd/mgt/mgt_vcl.c
@@ -166,6 +166,9 @@ mgt_new_vcl(struct cli *cli, const char *vclname, const char *vclsrc,
 
 	AN(cli);
 
+	if (C_flag)
+		vclname = ".Cflagtest";
+
 	if (state == NULL)
 		state = "auto";
 



From lkarsten at varnish-software.com  Mon Jan 18 16:38:16 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Mon, 18 Jan 2016 17:38:16 +0100
Subject: [4.1] 11d46e7 Qualify the -C test-vcl name with the PID of the
 process, so two concurrent -C tests don't collided.
Message-ID: <E1aLCoq-0000mB-CC@project.varnish-software.com>


commit 11d46e757219603f2898f466e100d0ac623a1e59
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Jan 18 14:37:24 2016 +0000

    Qualify the -C test-vcl name with the PID of the process, so two
    concurrent -C tests don't collided.
    
    Problem spotted by:	fgs

diff --git a/bin/varnishd/mgt/mgt_vcl.c b/bin/varnishd/mgt/mgt_vcl.c
index e533c69..65b50bf 100644
--- a/bin/varnishd/mgt/mgt_vcl.c
+++ b/bin/varnishd/mgt/mgt_vcl.c
@@ -163,11 +163,14 @@ mgt_new_vcl(struct cli *cli, const char *vclname, const char *vclsrc,
 	unsigned status;
 	char *lib, *p;
 	struct vclprog *vp;
+	char buf[32];
 
 	AN(cli);
 
-	if (C_flag)
-		vclname = ".Cflagtest";
+	if (C_flag) {
+		bprintf(buf, ".CflagTest.%d", (int)getpid());
+		vclname = buf;
+	}
 
 	if (state == NULL)
 		state = "auto";
@@ -183,6 +186,7 @@ mgt_new_vcl(struct cli *cli, const char *vclname, const char *vclsrc,
 	if (lib == NULL)
 		return;
 
+	AZ(C_flag);
 	vp = mgt_vcl_add(vclname, lib, state);
 	free(lib);
 



From lkarsten at varnish-software.com  Mon Jan 18 16:38:16 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Mon, 18 Jan 2016 17:38:16 +0100
Subject: [4.1] 9b159cb Minor corrections
Message-ID: <E1aLCoq-0000mK-DG@project.varnish-software.com>


commit 9b159cb557ccf9f3205288521ac56b137bc06dec
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Fri Jan 15 13:09:55 2016 +0000

    Minor corrections

diff --git a/doc/changes.rst b/doc/changes.rst
index c182ec8..529e71b 100644
--- a/doc/changes.rst
+++ b/doc/changes.rst
@@ -8,11 +8,10 @@ Changes since 4.1.0:
 - [varnishncsa] -w is now required when running deamonized.
 - [varnishncsa] Log format can now be read from file.
 - Port fields extracted from PROXY1 header now work as expected.
-- New VCL state "busy" introduced (mostly for vmod writers.)
-- New configure option: --enable-pcre-jit. Enabled by default.
+- New VCL state "busy" introduced (mostly for vmod writers).
 - Last traces of varnishreplay removed.
 - If-Modified-Since is now ignored if we have If-None-Match.
-- Zero Content-Length is no longer sent on empty 204 and 304 responses.
+- Zero Content-Length is no longer sent on 304 responses.
 
 
 Bugs fixed
@@ -37,7 +36,6 @@ Bugs fixed
 .. _1826: https://www.varnish-cache.org/trac/ticket/1826
 
 
-
 ================================
 Varnish Cache 4.1.0 (2015-09-30)
 ================================
@@ -2641,7 +2639,7 @@ varnishreplay
 varnishstat
 -----------
 
--  Don't print lifetime averages when it doesn't make any sense?for
+-  Don't print lifetime averages when it doesn't make any sense, for
    instance, there is no point in dividing the amount in bytes of free
    cache space by the lifetime in seconds of the varnishd process.
 
@@ -2774,7 +2772,7 @@ varnishncsa
 Documentation
 -------------
 
--  The documentation?especially the VCL documentation?has been greatly
+-  The documentation, especially the VCL documentation, has been greatly
    extended and improved.
 
 Build system



From lkarsten at varnish-software.com  Mon Jan 18 16:38:16 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Mon, 18 Jan 2016 17:38:16 +0100
Subject: [4.1] 40461b8 More changes
Message-ID: <E1aLCoq-0000mT-EW@project.varnish-software.com>


commit 40461b8a3762385598fec7b6a880d70384f43395
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Fri Jan 15 14:01:47 2016 +0000

    More changes

diff --git a/doc/changes.rst b/doc/changes.rst
index 529e71b..afa024f 100644
--- a/doc/changes.rst
+++ b/doc/changes.rst
@@ -12,6 +12,9 @@ Changes since 4.1.0:
 - Last traces of varnishreplay removed.
 - If-Modified-Since is now ignored if we have If-None-Match.
 - Zero Content-Length is no longer sent on 304 responses.
+- vcl_dir and vmod_dir now accept a colon separated list of directories.
+- Nested includes starting with "./" are relative to the including
+  VCL file now.
 
 
 Bugs fixed



From lkarsten at varnish-software.com  Mon Jan 18 16:38:16 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Mon, 18 Jan 2016 17:38:16 +0100
Subject: [4.1] d74b134 Nudge closer to 4.1.1-beta2.
Message-ID: <E1aLCoq-0000mc-Fg@project.varnish-software.com>


commit d74b134e179213f631df8a1a01d6be4e4eaed707
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Mon Jan 18 17:32:18 2016 +0100

    Nudge closer to 4.1.1-beta2.

diff --git a/configure.ac b/configure.ac
index abfdafc..0ccc0d7 100644
--- a/configure.ac
+++ b/configure.ac
@@ -2,7 +2,7 @@ AC_PREREQ(2.59)
 AC_COPYRIGHT([Copyright (c) 2006 Verdens Gang AS
 Copyright (c) 2006-2016 Varnish Software AS])
 AC_REVISION([$Id$])
-AC_INIT([Varnish], [4.1.1-beta1], [varnish-dev at varnish-cache.org])
+AC_INIT([Varnish], [4.1.1-beta2], [varnish-dev at varnish-cache.org])
 AC_CONFIG_SRCDIR(include/miniobj.h)
 AC_CONFIG_HEADERS([config.h])
 AC_CONFIG_MACRO_DIR([m4])
diff --git a/doc/changes.rst b/doc/changes.rst
index afa024f..6f0f5d8 100644
--- a/doc/changes.rst
+++ b/doc/changes.rst
@@ -1,5 +1,17 @@
 ======================================
-Varnish Cache 4.1.1-beta1 (unreleased)
+Varnish Cache 4.1.1-beta2 (unreleased)
+======================================
+
+Bugs fixed
+----------
+
+* 1802_ - Segfault after VCL change
+* 1825_ - Cannot Start Varnish After Just Restarting The Service
+* 1842_ - Handle missing waiting list gracefully.
+
+
+======================================
+Varnish Cache 4.1.1-beta1 (2016-01-15)
 ======================================
 
 Changes since 4.1.0:



From dridi.boukelmoune at gmail.com  Mon Jan 18 21:55:38 2016
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Mon, 18 Jan 2016 22:55:38 +0100
Subject: [master] 64c3853 Allow VMODs to fail a warm-up
Message-ID: <E1aLHly-0003b9-I6@project.varnish-software.com>


commit 64c3853f42e3b63044f56ba6dfa3cb45bad7b5bc
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Fri Dec 4 11:45:16 2015 +0100

    Allow VMODs to fail a warm-up
    
    It is also possible to convey a message to the CLI. In case of a failure
    the VCL is cooled down. VCL_Load may now fail with either CLIS_PARAM or
    CLIS_CANT.

diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index b1b7756..45d3e8b 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -438,7 +438,7 @@ vcl_setup_event(VRT_CTX, enum vcl_event_e ev)
 	assert(ev == VCL_EVENT_LOAD || ev == VCL_EVENT_WARM ||
 	    ev == VCL_EVENT_USE);
 
-	if (ev == VCL_EVENT_LOAD)
+	if (ev != VCL_EVENT_USE)
 		AN(ctx->msg);
 
 	return (ctx->vcl->conf->event_vcl(ctx, ev));
@@ -457,14 +457,18 @@ vcl_failsafe_event(VRT_CTX, enum vcl_event_e ev)
 		WRONG("A VMOD cannot fail COLD or DISCARD events");
 }
 
-static void
+static int
 vcl_set_state(VRT_CTX, const char *state)
 {
 	struct vcl *vcl;
+	int i = 0;
 
 	ASSERT_CLI();
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
 	AN(ctx->handling);
+	AN(ctx->vcl);
+	AN(state);
+	assert(ctx->msg != NULL || *state == '0');
 
 	vcl = ctx->vcl;
 	AN(vcl->temp);
@@ -494,16 +498,36 @@ vcl_set_state(VRT_CTX, const char *state)
 		/* The VCL must first reach a stable cold state */
 		else if (vcl->temp != VCL_TEMP_COOLING) {
 			vcl->temp = VCL_TEMP_WARM;
-			(void)vcl_setup_event(ctx, VCL_EVENT_WARM);
-			vcl_BackendEvent(vcl, VCL_EVENT_WARM);
+			i = vcl_setup_event(ctx, VCL_EVENT_WARM);
+			if (i == 0)
+				vcl_BackendEvent(vcl, VCL_EVENT_WARM);
+			else
+				AZ(vcl->conf->event_vcl(ctx, VCL_EVENT_COLD));
 		}
 		break;
 	default:
 		WRONG("Wrong enum state");
 	}
+	return (i);
 }
 
-static int
+static void
+vcl_cancel_load(VRT_CTX, struct cli *cli, const char *name, const char *step)
+{
+	struct vcl *vcl = ctx->vcl;
+
+	AZ(VSB_finish(ctx->msg));
+	VCLI_SetResult(cli, CLIS_CANT);
+	VCLI_Out(cli, "VCL \"%s\" Failed %s", name, step);
+	if (VSB_len(ctx->msg))
+		VCLI_Out(cli, "\nMessage:\n\t%s", VSB_data(ctx->msg));
+	AZ(vcl->conf->event_vcl(ctx, VCL_EVENT_DISCARD));
+	vcl_KillBackends(vcl);
+	VCL_Close(&vcl);
+	VSB_delete(ctx->msg);
+}
+
+static void
 VCL_Load(struct cli *cli, const char *name, const char *fn, const char *state)
 {
 	struct vcl *vcl;
@@ -516,8 +540,9 @@ VCL_Load(struct cli *cli, const char *name, const char *fn, const char *state)
 
 	vcl = vcl_find(name);
 	if (vcl != NULL) {
+		VCLI_SetResult(cli, CLIS_PARAM);
 		VCLI_Out(cli, "Config '%s' already loaded", name);
-		return (1);
+		return;
 	}
 
 	vsb = VSB_new_auto();
@@ -526,9 +551,10 @@ VCL_Load(struct cli *cli, const char *name, const char *fn, const char *state)
 	vcl = VCL_Open(fn, vsb);
 	if (vcl == NULL) {
 		AZ(VSB_finish(vsb));
+		VCLI_SetResult(cli, CLIS_PARAM);
 		VCLI_Out(cli, "%s", VSB_data(vsb));
 		VSB_delete(vsb);
-		return (1);
+		return;
 	}
 
 	vcl->loaded_name = strdup(name);
@@ -545,19 +571,18 @@ VCL_Load(struct cli *cli, const char *name, const char *fn, const char *state)
 	VSB_clear(vsb);
 	ctx.msg = vsb;
 	i = vcl_setup_event(&ctx, VCL_EVENT_LOAD);
-	AZ(VSB_finish(vsb));
 	if (i) {
-		VCLI_Out(cli, "VCL \"%s\" Failed initialization", name);
-		if (VSB_len(vsb))
-			VCLI_Out(cli, "\nMessage:\n\t%s", VSB_data(vsb));
-		vcl_failsafe_event(&ctx, VCL_EVENT_DISCARD);
-		vcl_KillBackends(vcl);
-		VCL_Close(&vcl);
-		VSB_delete(vsb);
-		return (1);
+		vcl_cancel_load(&ctx, cli, name, "initialization");
+		return;
+	}
+	VSB_clear(vsb);
+	i = vcl_set_state(&ctx, state);
+	if (i) {
+		assert(*state == '1');
+		vcl_cancel_load(&ctx, cli, name, "warmup");
+		return;
 	}
 	VSB_delete(vsb);
-	vcl_set_state(&ctx, state);
 	bprintf(vcl->state, "%s", state + 1);
 	assert(hand == VCL_RET_OK);
 	VCLI_Out(cli, "Loaded \"%s\" as \"%s\"", fn , name);
@@ -568,7 +593,6 @@ VCL_Load(struct cli *cli, const char *name, const char *fn, const char *state)
 	Lck_Unlock(&vcl_mtx);
 	VSC_C_main->n_vcl++;
 	VSC_C_main->n_vcl_avail++;
-	return (0);
 }
 
 /*--------------------------------------------------------------------
@@ -616,7 +640,7 @@ VCL_Poll(void)
 			INIT_OBJ(&ctx, VRT_CTX_MAGIC);
 			ctx.vcl = vcl;
 			ctx.handling = &hand;
-			vcl_set_state(&ctx, "0");
+			(void)vcl_set_state(&ctx, "0");
 		}
 		if (vcl->discard && vcl->temp == VCL_TEMP_COLD)
 			VCL_Nuke(vcl);
@@ -652,8 +676,7 @@ ccf_config_load(struct cli *cli, const char * const *av, void *priv)
 
 	AZ(priv);
 	ASSERT_CLI();
-	if (VCL_Load(cli, av[2], av[3], av[4]))
-		VCLI_SetResult(cli, CLIS_PARAM);
+	VCL_Load(cli, av[2], av[3], av[4]);
 }
 
 static void __match_proto__(cli_func_t)
@@ -663,6 +686,8 @@ ccf_config_state(struct cli *cli, const char * const *av, void *priv)
 	unsigned hand;
 
 	INIT_OBJ(&ctx, VRT_CTX_MAGIC);
+	ctx.msg = VSB_new_auto();
+	AN(ctx.msg);
 	ctx.handling = &hand;
 
 	(void)cli;
@@ -672,8 +697,18 @@ ccf_config_state(struct cli *cli, const char * const *av, void *priv)
 	AN(av[3]);
 	ctx.vcl = vcl_find(av[2]);
 	AN(ctx.vcl);			// MGT ensures this
-	vcl_set_state(&ctx, av[3]);
-	bprintf(ctx.vcl->state, "%s", av[3] + 1);
+	if (vcl_set_state(&ctx, av[3]) == 0) {
+		bprintf(ctx.vcl->state, "%s", av[3] + 1);
+		VSB_delete(ctx.msg);
+		return;
+	}
+	AZ(VSB_finish(ctx.msg));
+	VCLI_SetResult(cli, CLIS_CANT);
+	VCLI_Out(cli, "Failed <vcl.state %s %s>", ctx.vcl->loaded_name,
+	    av[3] + 1);
+	if (VSB_len(ctx.msg))
+		VCLI_Out(cli, "\nMessage:\n\t%s", VSB_data(ctx.msg));
+	VSB_delete(ctx.msg);
 }
 
 static void __match_proto__(cli_func_t)



From dridi.boukelmoune at gmail.com  Mon Jan 18 21:55:38 2016
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Mon, 18 Jan 2016 22:55:38 +0100
Subject: [master] 36293d0 Make VMODs actually fail warm-ups
Message-ID: <E1aLHly-0003bG-Jv@project.varnish-software.com>


commit 36293d0d563f62cd07acacd65ce74011aab31106
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Mon Dec 7 11:18:33 2015 +0100

    Make VMODs actually fail warm-ups
    
    The implementation is similar to the load/discard dance when a load
    fails. New VGC functions are introduced iff the VCL has at least one
    VMOD handling events.
    
    The generated code looks like this:
    
    	static unsigned vgc_inistep;
    	static unsigned vgc_warmupstep;
    
    	...
    
    	static int
    	VGC_Load(VRT_CTX)
    	{
    		...
    	}
    
    	static int
    	VGC_Discard(VRT_CTX)
    	{
    		...
    	}
    
    	static int
    	VGC_Warmup(VRT_CTX, enum vcl_event_e ev)
    	{
    
    		vgc_warmupstep = 0;
    
    		/* 4 */
    		if (Vmod_debug_Func._event(ctx, &vmod_priv_debug, ev))
    			return (1);
    		vgc_warmupstep = 4;
    
    		return (0);
    	}
    
    	static int
    	VGC_Use(VRT_CTX, enum vcl_event_e ev)
    	{
    
    		/* 4 */
    		if (Vmod_debug_Func._event(ctx, &vmod_priv_debug, ev))
    			return (1);
    
    		return (0);
    	}
    
    	static int
    	VGC_Cooldown(VRT_CTX, enum vcl_event_e ev)
    	{
    		int retval = 0;
    
    		/* 4 */
    		if (vgc_warmupstep >= 4 &&
    		    Vmod_debug_Func._event(ctx, &vmod_priv_debug, ev) != 0)
    			retval = 1;
    
    		return (retval);
    	}
    
    	static int
    	VGC_Event(VRT_CTX, enum vcl_event_e ev)
    	{
    		if (ev == VCL_EVENT_LOAD)
    			return(VGC_Load(ctx));
    		if (ev == VCL_EVENT_WARM)
    			return(VGC_Warmup(ctx, ev));
    		if (ev == VCL_EVENT_USE)
    			return(VGC_Use(ctx, ev));
    		if (ev == VCL_EVENT_COLD)
    			return(VGC_Cooldown(ctx, ev));
    		if (ev == VCL_EVENT_DISCARD)
    			return(VGC_Discard(ctx));
    
    		return (1);
    	}
    
    However, if there are no VMODs handling events, no new functions shall
    be generated, leading to code looking like this:
    
    	static unsigned vgc_inistep;
    	static unsigned vgc_warmupstep;
    
    	...
    
    	static int
    	VGC_Load(VRT_CTX)
    	{
    		...
    	}
    
    	static int
    	VGC_Discard(VRT_CTX)
    	{
    		...
    	}
    
    	static int
    	VGC_Event(VRT_CTX, enum vcl_event_e ev)
    	{
    		if (ev == VCL_EVENT_LOAD)
    			return(VGC_Load(ctx));
    		if (ev == VCL_EVENT_DISCARD)
    			return(VGC_Discard(ctx));
    
    		(void)vgc_warmupstep;
    		return (0);
    	}

diff --git a/doc/sphinx/reference/vmod.rst b/doc/sphinx/reference/vmod.rst
index 57fa09c..c6eb274 100644
--- a/doc/sphinx/reference/vmod.rst
+++ b/doc/sphinx/reference/vmod.rst
@@ -383,6 +383,13 @@ first with a ``VCL_EVENT_WARM`` event. Unless a user decides that a given VCL
 should always be warm, an inactive VMOD will eventually become cold and should
 manage resources accordingly.
 
+An event function must return zero upon success. It is only possible to fail
+an initialization with the ``VCL_EVENT_LOAD`` or ``VCL_EVENT_WARM`` events.
+Should such a failure happen, a ``VCL_EVENT_DISCARD`` or ``VCL_EVENT_COLD``
+event will be sent to the VMODs that succeeded to put them back in a cold
+state. The VMOD that failed will not receive this event, and therefore must
+not be left half-initialized should a failure occur.
+
 If your VMOD is running an asynchronous background job you can hold a reference
 to the VCL to prevent it from going cold too soon and get the same guarantees
 as backends with ongoing requests for instance. For that, you must acquire the
diff --git a/lib/libvcc/vcc_compile.c b/lib/libvcc/vcc_compile.c
index 62aaeae..da6d8b4 100644
--- a/lib/libvcc/vcc_compile.c
+++ b/lib/libvcc/vcc_compile.c
@@ -325,20 +325,22 @@ EmitCoordinates(const struct vcc *tl, struct vsb *vsb)
 /*--------------------------------------------------------------------
  * Init/Fini/Event
  *
- * We call Fini-s in the opposite order of init-s.
- * Other events are called in same order as init-s, no matter which
- * event it might be.
+ * We call DISCARD and COLD events in the opposite order of LOAD and
+ * WARM.
  */
 
 static void
 EmitInitFini(const struct vcc *tl)
 {
 	struct inifin *p;
+	unsigned has_event = 0;
 
-	Fh(tl, 0, "\nstatic unsigned vgc_inistep;\n");
+	Fh(tl, 0, "\n");
+	Fh(tl, 0, "static unsigned vgc_inistep;\n");
+	Fh(tl, 0, "static unsigned vgc_warmupstep;\n");
 
 	/*
-	 * INIT
+	 * LOAD
 	 */
 	Fc(tl, 0, "\nstatic int\nVGC_Load(VRT_CTX)\n{\n\n");
 	Fc(tl, 0, "\tvgc_inistep = 0;\n\n");
@@ -349,6 +351,10 @@ EmitInitFini(const struct vcc *tl)
 			Fc(tl, 0, "\t/* %u */\n%s\n", p->n, VSB_data(p->ini));
 		Fc(tl, 0, "\tvgc_inistep = %u;\n\n", p->n);
 		VSB_delete(p->ini);
+
+		AZ(VSB_finish(p->event));
+		if (VSB_len(p->event))
+			has_event = 1;
 	}
 
 	Fc(tl, 0, "\t(void)VGC_function_vcl_init(ctx);\n");
@@ -356,7 +362,7 @@ EmitInitFini(const struct vcc *tl)
 	Fc(tl, 0, "}\n");
 
 	/*
-	 * FINI
+	 * DISCARD
 	 */
 	Fc(tl, 0, "\nstatic int\nVGC_Discard(VRT_CTX)\n{\n\n");
 
@@ -375,6 +381,66 @@ EmitInitFini(const struct vcc *tl)
 	Fc(tl, 0, "\treturn(0);\n");
 	Fc(tl, 0, "}\n");
 
+	if (has_event) {
+		/*
+		 * WARM
+		 */
+		Fc(tl, 0, "\nstatic int\n");
+		Fc(tl, 0, "VGC_Warmup(VRT_CTX, enum vcl_event_e ev)\n{\n\n");
+
+		Fc(tl, 0, "\tvgc_warmupstep = 0;\n\n");
+		VTAILQ_FOREACH(p, &tl->inifin, list) {
+			assert(p->n > 0);
+			if (VSB_len(p->event)) {
+				Fc(tl, 0, "\t/* %u */\n", p->n);
+				Fc(tl, 0, "\tif (%s)\n", VSB_data(p->event));
+				Fc(tl, 0, "\t\treturn (1);\n");
+				Fc(tl, 0, "\tvgc_warmupstep = %u;\n\n", p->n);
+			}
+		}
+
+		Fc(tl, 0, "\treturn (0);\n");
+		Fc(tl, 0, "}\n");
+
+		/*
+		 * USE (deprecated)
+		 */
+		Fc(tl, 0, "\nstatic int\n");
+		Fc(tl, 0, "VGC_Use(VRT_CTX, enum vcl_event_e ev)\n{\n\n");
+
+		VTAILQ_FOREACH(p, &tl->inifin, list) {
+			assert(p->n > 0);
+			if (VSB_len(p->event)) {
+				Fc(tl, 0, "\t/* %u */\n", p->n);
+				Fc(tl, 0, "\tif (%s)\n", VSB_data(p->event));
+				Fc(tl, 0, "\t\treturn (1);\n\n");
+			}
+		}
+
+		Fc(tl, 0, "\treturn (0);\n");
+		Fc(tl, 0, "}\n");
+
+		/*
+		 * COLD
+		 */
+		Fc(tl, 0, "\nstatic int\n");
+		Fc(tl, 0, "VGC_Cooldown(VRT_CTX, enum vcl_event_e ev)\n{\n");
+		Fc(tl, 0, "\tint retval = 0;\n\n");
+
+		VTAILQ_FOREACH_REVERSE(p, &tl->inifin, inifinhead, list) {
+			if (VSB_len(p->event)) {
+				Fc(tl, 0, "\t/* %u */\n", p->n);
+				Fc(tl, 0, "\tif (vgc_warmupstep >= %u &&\n", p->n);
+				Fc(tl, 0, "\t    %s != 0)\n", VSB_data(p->event));
+				Fc(tl, 0, "\t\tretval = 1;\n\n");
+			}
+			VSB_delete(p->event);
+		}
+
+		Fc(tl, 0, "\treturn (retval);\n");
+		Fc(tl, 0, "}\n");
+	}
+
 	/*
 	 * EVENTS
 	 */
@@ -383,16 +449,20 @@ EmitInitFini(const struct vcc *tl)
 	Fc(tl, 0, "{\n");
 	Fc(tl, 0, "\tif (ev == VCL_EVENT_LOAD)\n");
 	Fc(tl, 0, "\t\treturn(VGC_Load(ctx));\n");
+	if (has_event) {
+		Fc(tl, 0, "\tif (ev == VCL_EVENT_WARM)\n");
+		Fc(tl, 0, "\t\treturn(VGC_Warmup(ctx, ev));\n");
+		Fc(tl, 0, "\tif (ev == VCL_EVENT_USE)\n");
+		Fc(tl, 0, "\t\treturn(VGC_Use(ctx, ev));\n");
+		Fc(tl, 0, "\tif (ev == VCL_EVENT_COLD)\n");
+		Fc(tl, 0, "\t\treturn(VGC_Cooldown(ctx, ev));\n");
+	}
 	Fc(tl, 0, "\tif (ev == VCL_EVENT_DISCARD)\n");
 	Fc(tl, 0, "\t\treturn(VGC_Discard(ctx));\n");
 	Fc(tl, 0, "\n");
-	VTAILQ_FOREACH(p, &tl->inifin, list) {
-		AZ(VSB_finish(p->event));
-		if (VSB_len(p->event))
-			Fc(tl, 0, "\t/* %u */\n%s\n", p->n, VSB_data(p->event));
-		VSB_delete(p->event);
-	}
-	Fc(tl, 0, "\treturn (0);\n");
+	if (!has_event)
+		Fc(tl, 0, "\t(void)vgc_warmupstep;\n");
+	Fc(tl, 0, "\treturn (%d);\n", has_event ? 1 : 0);
 	Fc(tl, 0, "}\n");
 }
 
diff --git a/lib/libvcc/vcc_vmod.c b/lib/libvcc/vcc_vmod.c
index d1b8f26..3a522a5 100644
--- a/lib/libvcc/vcc_vmod.c
+++ b/lib/libvcc/vcc_vmod.c
@@ -228,8 +228,7 @@ vcc_ParseImport(struct vcc *tl)
 			VSB_printf(ifp->fin,
 			    "\t\t(void)%s(ctx, &vmod_priv_%.*s,\n"
 			    "\t\t    VCL_EVENT_DISCARD);\n", p, PF(mod));
-			VSB_printf(ifp->event,
-			    "\t(void)%s(ctx, &vmod_priv_%.*s, ev);\n",
+			VSB_printf(ifp->event, "\t%s(ctx, &vmod_priv_%.*s, ev)",
 			    p, PF(mod));
 		} else {
 			sym = VCC_AddSymbolStr(tl, p, SYM_FUNC);



From dridi.boukelmoune at gmail.com  Mon Jan 18 21:55:38 2016
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Mon, 18 Jan 2016 22:55:38 +0100
Subject: [master] 1826539 Catch a vcl.state failure on the manager side
Message-ID: <E1aLHly-0003bQ-LW@project.varnish-software.com>


commit 182653966173b8cd4c3c74dfb2d9cc31cb1a6fe0
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Mon Dec 7 12:07:27 2015 +0100

    Catch a vcl.state failure on the manager side
    
    Don't update the state of the VCL to warm if it failed, and don't start
    the child if the active VCL failed to warm up.

diff --git a/bin/varnishd/mgt/mgt.h b/bin/varnishd/mgt/mgt.h
index 1aeba55..fe96360 100644
--- a/bin/varnishd/mgt/mgt.h
+++ b/bin/varnishd/mgt/mgt.h
@@ -167,7 +167,7 @@ void mgt_vcc_init(void);
 void mgt_vcl_init(void);
 void mgt_vcc_startup(struct cli *, const char *b_arg, const char *f_arg,
     const char *vclsrc, int Cflag);
-int mgt_push_vcls_and_start(unsigned *status, char **p);
+int mgt_push_vcls_and_start(struct cli *, unsigned *status, char **p);
 int mgt_has_vcl(void);
 extern char *mgt_cc_cmd;
 extern const char *mgt_vcl_dir;
diff --git a/bin/varnishd/mgt/mgt_child.c b/bin/varnishd/mgt/mgt_child.c
index 8a26eb4..ae01a91 100644
--- a/bin/varnishd/mgt/mgt_child.c
+++ b/bin/varnishd/mgt/mgt_child.c
@@ -418,7 +418,8 @@ mgt_launch_child(struct cli *cli)
 
 	mgt_cli_start_child(child_cli_in, child_cli_out);
 	child_pid = pid;
-	if (mgt_push_vcls_and_start(&u, &p)) {
+	if (mgt_push_vcls_and_start(cli, &u, &p)) {
+		VCLI_SetResult(cli, u);
 		MGT_complain(C_ERR, "Child (%jd) Pushing vcls failed:\n%s",
 		    (intmax_t)child_pid, p);
 		free(p);
diff --git a/bin/varnishd/mgt/mgt_vcl.c b/bin/varnishd/mgt/mgt_vcl.c
index b651f83..858c5ec 100644
--- a/bin/varnishd/mgt/mgt_vcl.c
+++ b/bin/varnishd/mgt/mgt_vcl.c
@@ -122,12 +122,13 @@ mgt_has_vcl(void)
 	return (!VTAILQ_EMPTY(&vclhead));
 }
 
-static void
-mgt_vcl_setstate(struct vclprog *vp, const char *vs)
+static int
+mgt_vcl_setstate(struct cli *cli, struct vclprog *vp, const char *vs)
 {
 	unsigned status, warm;
 	double now;
 	char *p;
+	int i;
 
 	if (vs == VCL_STATE_AUTO) {
 		assert(vp != active_vcl);
@@ -142,7 +143,7 @@ mgt_vcl_setstate(struct vclprog *vp, const char *vs)
 	warm = vs == VCL_STATE_WARM ? 1 : 0;
 
 	if (vp->warm == warm)
-		return;
+		return (0);
 
 	vp->warm = warm;
 
@@ -150,15 +151,19 @@ mgt_vcl_setstate(struct vclprog *vp, const char *vs)
 		vp->go_cold = 0;
 
 	if (child_pid < 0)
-		return;
+		return (0);
 
-	/*
-	 * We ignore the result here so we don't croak if the child did.
-	 */
-	(void)mgt_cli_askchild(&status, &p, "vcl.state %s %d%s\n",
+	i = mgt_cli_askchild(&status, &p, "vcl.state %s %d%s\n",
 	    vp->name, vp->warm, vp->state);
+	if (i) {
+		AN(cli);
+		AN(vp->warm);
+		VCLI_SetResult(cli, status);
+		VCLI_Out(cli, "%s", p);
+	}
 
 	free(p);
+	return (i);
 }
 
 /*--------------------------------------------------------------------*/
@@ -239,12 +244,15 @@ mgt_vcc_startup(struct cli *cli, const char *b_arg, const char *f_arg,
 /*--------------------------------------------------------------------*/
 
 int
-mgt_push_vcls_and_start(unsigned *status, char **p)
+mgt_push_vcls_and_start(struct cli *cli, unsigned *status, char **p)
 {
 	struct vclprog *vp;
 
 	AN(active_vcl);
-	mgt_vcl_setstate(active_vcl, VCL_STATE_WARM);
+
+	/* The VCL has not been loaded yet, it cannot fail */
+	AZ(mgt_vcl_setstate(cli, active_vcl, VCL_STATE_WARM));
+
 	VTAILQ_FOREACH(vp, &vclhead, list) {
 		if (mgt_cli_askchild(status, p, "vcl.load \"%s\" %s %d%s\n",
 		    vp->name, vp->fname, vp->warm, vp->state))
@@ -326,7 +334,7 @@ mcf_vcl_state(struct cli *cli, const char * const *av, void *priv)
 		bprintf(vp->state, "%s", "auto");
 		if (vp != active_vcl) {
 			vp->go_cold = VTIM_mono();
-			mgt_vcl_setstate(vp, VCL_STATE_AUTO);
+			(void)mgt_vcl_setstate(cli, vp, VCL_STATE_AUTO);
 		}
 	} else if (!strcmp(av[3], "cold")) {
 		if (vp == active_vcl) {
@@ -335,10 +343,10 @@ mcf_vcl_state(struct cli *cli, const char * const *av, void *priv)
 			return;
 		}
 		bprintf(vp->state, "%s", "auto");
-		mgt_vcl_setstate(vp, VCL_STATE_COLD);
+		(void)mgt_vcl_setstate(cli, vp, VCL_STATE_COLD);
 	} else if (!strcmp(av[3], "warm")) {
-		bprintf(vp->state, "%s", av[3]);
-		mgt_vcl_setstate(vp, VCL_STATE_WARM);
+		if (mgt_vcl_setstate(cli, vp, VCL_STATE_WARM) == 0)
+			bprintf(vp->state, "%s", av[3]);
 	} else {
 		VCLI_Out(cli, "State must be one of auto, cold or warm.");
 		VCLI_SetResult(cli, CLIS_PARAM);
@@ -358,20 +366,21 @@ mcf_vcl_use(struct cli *cli, const char * const *av, void *priv)
 		return;
 	if (vp == active_vcl)
 		return;
-	mgt_vcl_setstate(vp, VCL_STATE_WARM);
+	if (mgt_vcl_setstate(cli, vp, VCL_STATE_WARM))
+		return;
 	if (child_pid >= 0 &&
 	    mgt_cli_askchild(&status, &p, "vcl.use %s\n", av[2])) {
 		VCLI_SetResult(cli, status);
 		VCLI_Out(cli, "%s", p);
 		vp->go_cold = VTIM_mono();
-		mgt_vcl_setstate(vp, VCL_STATE_AUTO);
+		(void)mgt_vcl_setstate(cli, vp, VCL_STATE_AUTO);
 	} else {
 		VCLI_Out(cli, "VCL '%s' now active", av[2]);
 		vp2 = active_vcl;
 		active_vcl = vp;
 		if (vp2 != NULL) {
 			vp2->go_cold = VTIM_mono();
-			mgt_vcl_setstate(vp2, VCL_STATE_AUTO);
+			(void)mgt_vcl_setstate(cli, vp2, VCL_STATE_AUTO);
 		}
 	}
 	free(p);
@@ -393,9 +402,9 @@ mcf_vcl_discard(struct cli *cli, const char * const *av, void *priv)
 		VCLI_Out(cli, "Cannot discard active VCL program\n");
 		return;
 	}
-	mgt_vcl_setstate(vp, VCL_STATE_COLD);
+	(void)mgt_vcl_setstate(cli, vp, VCL_STATE_COLD);
 	if (child_pid >= 0) {
-		/* If this fails the child is crashing, figure that later */
+		/* XXX If this fails the child is crashing, figure that later */
 		(void)mgt_cli_askchild(&status, &p, "vcl.discard %s\n", av[2]);
 		free(p);
 	}
@@ -439,7 +448,7 @@ mgt_vcl_poker(const struct vev *e, int what)
 	e_poker->timeout = mgt_param.vcl_cooldown * .45;
 	VTAILQ_FOREACH(vp, &vclhead, list) {
 		if (vp != active_vcl)
-			mgt_vcl_setstate(vp, VCL_STATE_AUTO);
+			(void)mgt_vcl_setstate(NULL, vp, VCL_STATE_AUTO);
 	}
 	return (0);
 }
diff --git a/bin/varnishtest/tests/v00044.vtc b/bin/varnishtest/tests/v00044.vtc
index aa79e6a..c056d6c 100644
--- a/bin/varnishtest/tests/v00044.vtc
+++ b/bin/varnishtest/tests/v00044.vtc
@@ -83,3 +83,11 @@ delay .4
 varnish v1 -expect VBE.vcl1.default.happy >= 0
 delay 4
 varnish v1 -expect !VBE.vcl1.default.happy
+
+# A VMOD's warm-up can fail
+varnish v1 -cliok "param.set max_esi_depth 42"
+varnish v1 -clierr 300 "vcl.state vcl1 warm"
+
+# A warm-up failure can also fail a child start
+varnish v1 -cliok stop
+varnish v1 -clierr 300 start



From dridi.boukelmoune at gmail.com  Mon Jan 18 21:55:38 2016
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Mon, 18 Jan 2016 22:55:38 +0100
Subject: [master] 81655e5 Simplify `vcl.use` by making it failsafe
Message-ID: <E1aLHly-0003ba-NC@project.varnish-software.com>


commit 81655e5345397d2ad223d7ab3cb1107eb363257b
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Mon Dec 7 15:02:13 2015 +0100

    Simplify `vcl.use` by making it failsafe
    
    By the time we decide to switch to a VCL, it must be warm and usable.
    The deprecated VCL_EVENT_USE should not get in the way.

diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index 45d3e8b..e10f0c8 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -435,11 +435,8 @@ vcl_setup_event(VRT_CTX, enum vcl_event_e ev)
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
 	AN(ctx->handling);
 	AN(ctx->vcl);
-	assert(ev == VCL_EVENT_LOAD || ev == VCL_EVENT_WARM ||
-	    ev == VCL_EVENT_USE);
-
-	if (ev != VCL_EVENT_USE)
-		AN(ctx->msg);
+	AN(ctx->msg);
+	assert(ev == VCL_EVENT_LOAD || ev == VCL_EVENT_WARM);
 
 	return (ctx->vcl->conf->event_vcl(ctx, ev));
 }
@@ -451,10 +448,11 @@ vcl_failsafe_event(VRT_CTX, enum vcl_event_e ev)
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
 	AN(ctx->handling);
 	AN(ctx->vcl);
-	assert(ev == VCL_EVENT_COLD || ev == VCL_EVENT_DISCARD);
+	assert(ev == VCL_EVENT_COLD || ev == VCL_EVENT_DISCARD ||
+	    ev == VCL_EVENT_USE);
 
 	if (ctx->vcl->conf->event_vcl(ctx, ev) != 0)
-		WRONG("A VMOD cannot fail COLD or DISCARD events");
+		WRONG("A VMOD cannot fail USE, COLD or DISCARD events");
 }
 
 static int
@@ -738,33 +736,20 @@ ccf_config_use(struct cli *cli, const char * const *av, void *priv)
 	struct vcl *vcl;
 	struct vrt_ctx ctx;
 	unsigned hand = 0;
-	struct vsb *vsb;
-	int i;
 
 	ASSERT_CLI();
+	AN(cli);
 	AZ(priv);
 	vcl = vcl_find(av[2]);
 	AN(vcl);				// MGT ensures this
 	assert(vcl->temp == VCL_TEMP_WARM);	// MGT ensures this
 	INIT_OBJ(&ctx, VRT_CTX_MAGIC);
 	ctx.handling = &hand;
-	vsb = VSB_new_auto();
-	AN(vsb);
-	ctx.msg = vsb;
 	ctx.vcl = vcl;
-	i = vcl_setup_event(&ctx, VCL_EVENT_USE);
-	AZ(VSB_finish(vsb));
-	if (i) {
-		VCLI_Out(cli, "VCL \"%s\" Failed to activate", av[2]);
-		if (VSB_len(vsb) > 0)
-			VCLI_Out(cli, "\nMessage:\n\t%s", VSB_data(vsb));
-		VCLI_SetResult(cli, CLIS_CANT);
-	} else {
-		Lck_Lock(&vcl_mtx);
-		vcl_active = vcl;
-		Lck_Unlock(&vcl_mtx);
-	}
-	VSB_delete(vsb);
+	vcl_failsafe_event(&ctx, VCL_EVENT_USE);
+	Lck_Lock(&vcl_mtx);
+	vcl_active = vcl;
+	Lck_Unlock(&vcl_mtx);
 }
 
 static void __match_proto__(cli_func_t)
diff --git a/lib/libvcc/vcc_compile.c b/lib/libvcc/vcc_compile.c
index da6d8b4..cc22496 100644
--- a/lib/libvcc/vcc_compile.c
+++ b/lib/libvcc/vcc_compile.c
@@ -326,7 +326,8 @@ EmitCoordinates(const struct vcc *tl, struct vsb *vsb)
  * Init/Fini/Event
  *
  * We call DISCARD and COLD events in the opposite order of LOAD and
- * WARM.
+ * WARM. The child will panic if a USE event fails, since a WARM event
+ * leads to a usable state.
  */
 
 static void
diff --git a/lib/libvmod_debug/vmod_debug.c b/lib/libvmod_debug/vmod_debug.c
index a9e2cde..9485268 100644
--- a/lib/libvmod_debug/vmod_debug.c
+++ b/lib/libvmod_debug/vmod_debug.c
@@ -278,6 +278,7 @@ event_warm(VRT_CTX)
 
 	VSL(SLT_Debug, 0, "%s: VCL_EVENT_WARM", VCL_Name(ctx->vcl));
 
+	AN(ctx->msg);
 	if (cache_param->max_esi_depth == 42) {
 		VSB_printf(ctx->msg, "max_esi_depth is not the answer.");
 		return (-1);



From dridi.boukelmoune at gmail.com  Mon Jan 18 21:55:38 2016
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Mon, 18 Jan 2016 22:55:38 +0100
Subject: [master] 8756bac Replace the VCL refcount by a self-desribing list
Message-ID: <E1aLHly-0003bj-Ox@project.varnish-software.com>


commit 8756bacca6550588797eb1c8a9189306b7be86d8
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Mon Dec 7 17:06:36 2015 +0100

    Replace the VCL refcount by a self-desribing list
    
    Instead of counting the references, the VCL keeps track of them with up
    to 31 characters in the description. It is the VMOD's responsibility to
    keep track of the opaque struct vclref * and provide a meaningful user-
    friendly description.

diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index e10f0c8..694b486 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -62,10 +62,18 @@ struct vcl {
 	char			state[8];
 	char			*loaded_name;
 	unsigned		busy;
-	unsigned		refcount;
 	unsigned		discard;
 	const char		*temp;
 	VTAILQ_HEAD(,backend)	backend_list;
+	VTAILQ_HEAD(,vclref)	ref_list;
+};
+
+struct vclref {
+	unsigned		magic;
+#define VCLREF_MAGIC		0x47fb6848
+	const struct vcl	*vcl;
+	VTAILQ_ENTRY(vclref)	list;
+	char			desc[32];
 };
 
 /*
@@ -252,7 +260,7 @@ vcl_KillBackends(struct vcl *vcl)
 
 	CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC);
 	AZ(vcl->busy);
-	AZ(vcl->refcount);
+	assert(VTAILQ_EMPTY(&vcl->ref_list));
 	while (1) {
 		be = VTAILQ_FIRST(&vcl->backend_list);
 		if (be == NULL)
@@ -375,40 +383,59 @@ VRT_count(VRT_CTX, unsigned u)
 		    ctx->vcl->conf->ref[u].line, ctx->vcl->conf->ref[u].pos);
 }
 
-void
-VRT_ref_vcl(VRT_CTX)
+struct vclref *
+VRT_ref_vcl(VRT_CTX, const char *desc)
 {
 	struct vcl *vcl;
+	struct vclref* ref;
 
 	ASSERT_CLI();
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
+	AN(desc);
+	AN(*desc);
 
 	vcl = ctx->vcl;
 	CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC);
 	xxxassert(vcl->temp == VCL_TEMP_WARM);
 
+	ALLOC_OBJ(ref, VCLREF_MAGIC);
+	AN(ref);
+	ref->vcl = vcl;
+	snprintf(ref->desc, sizeof ref->desc, "%s", desc);
+
 	Lck_Lock(&vcl_mtx);
-	vcl->refcount++;
+	VTAILQ_INSERT_TAIL(&vcl->ref_list, ref, list);
 	Lck_Unlock(&vcl_mtx);
+
+	return (ref);
 }
 
 void
-VRT_rel_vcl(VRT_CTX)
+VRT_rel_vcl(VRT_CTX, struct vclref **refp)
 {
 	struct vcl *vcl;
+	struct vclref *ref;
+
+	AN(refp);
+	ref = *refp;
+	*refp = NULL;
 
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
+	CHECK_OBJ_NOTNULL(ref, VCLREF_MAGIC);
 
 	vcl = ctx->vcl;
 	CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC);
+	assert(vcl == ref->vcl);
 	assert(vcl->temp == VCL_TEMP_WARM || vcl->temp == VCL_TEMP_BUSY ||
 	    vcl->temp == VCL_TEMP_COOLING);
 
 	Lck_Lock(&vcl_mtx);
-	assert(vcl->refcount > 0);
-	vcl->refcount--;
+	assert(!VTAILQ_EMPTY(&vcl->ref_list));
+	VTAILQ_REMOVE(&vcl->ref_list, ref, list);
 	/* No garbage collection here, for the same reasons as in VCL_Rel. */
 	Lck_Unlock(&vcl_mtx);
+
+	FREE_OBJ(ref);
 }
 
 /*--------------------------------------------------------------------*/
@@ -455,6 +482,23 @@ vcl_failsafe_event(VRT_CTX, enum vcl_event_e ev)
 		WRONG("A VMOD cannot fail USE, COLD or DISCARD events");
 }
 
+static void
+vcl_print_refs(VRT_CTX)
+{
+	struct vcl *vcl;
+	struct vclref *ref;
+
+	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
+	CHECK_OBJ_NOTNULL(ctx->vcl, VCL_MAGIC);
+	AN(ctx->msg);
+	vcl = ctx->vcl;
+	VSB_printf(ctx->msg, "VCL %s is waiting for:", vcl->loaded_name);
+	Lck_Lock(&vcl_mtx);
+	VTAILQ_FOREACH(ref, &ctx->vcl->ref_list, list)
+		VSB_printf(ctx->msg, "\n\t- %s", ref->desc);
+	Lck_Unlock(&vcl_mtx);
+}
+
 static int
 vcl_set_state(VRT_CTX, const char *state)
 {
@@ -477,16 +521,17 @@ vcl_set_state(VRT_CTX, const char *state)
 		if (vcl->busy == 0 && (vcl->temp == VCL_TEMP_WARM ||
 		    vcl->temp == VCL_TEMP_BUSY)) {
 
-			vcl->temp = vcl->refcount ? VCL_TEMP_COOLING :
-			    VCL_TEMP_COLD;
+			vcl->temp = VTAILQ_EMPTY(&vcl->ref_list) ?
+			    VCL_TEMP_COLD : VCL_TEMP_COOLING;
 			vcl_failsafe_event(ctx, VCL_EVENT_COLD);
 			vcl_BackendEvent(vcl, VCL_EVENT_COLD);
 		}
 		else if (vcl->busy)
 			vcl->temp = VCL_TEMP_BUSY;
+		else if (VTAILQ_EMPTY(&vcl->ref_list))
+			vcl->temp = VCL_TEMP_COLD;
 		else
-			vcl->temp = vcl->refcount ? VCL_TEMP_COOLING :
-			    VCL_TEMP_COLD;
+			vcl->temp = VCL_TEMP_COOLING;
 		break;
 	case '1':
 		assert(vcl->temp != VCL_TEMP_WARM);
@@ -494,7 +539,11 @@ vcl_set_state(VRT_CTX, const char *state)
 		if (vcl->temp == VCL_TEMP_BUSY)
 			vcl->temp = VCL_TEMP_WARM;
 		/* The VCL must first reach a stable cold state */
-		else if (vcl->temp != VCL_TEMP_COOLING) {
+		else if (vcl->temp == VCL_TEMP_COOLING) {
+			vcl_print_refs(ctx);
+			i = -1;
+		}
+		else {
 			vcl->temp = VCL_TEMP_WARM;
 			i = vcl_setup_event(ctx, VCL_EVENT_WARM);
 			if (i == 0)
@@ -558,6 +607,7 @@ VCL_Load(struct cli *cli, const char *name, const char *fn, const char *state)
 	vcl->loaded_name = strdup(name);
 	XXXAN(vcl->loaded_name);
 	VTAILQ_INIT(&vcl->backend_list);
+	VTAILQ_INIT(&vcl->ref_list);
 
 	vcl->temp = VCL_TEMP_INIT;
 
@@ -609,7 +659,7 @@ VCL_Nuke(struct vcl *vcl)
 	assert(vcl != vcl_active);
 	assert(vcl->discard);
 	AZ(vcl->busy);
-	AZ(vcl->refcount);
+	assert(VTAILQ_EMPTY(&vcl->ref_list));
 	VTAILQ_REMOVE(&vcl_head, vcl, list);
 	ctx.method = VCL_MET_FINI;
 	ctx.handling = &hand;
@@ -726,7 +776,7 @@ ccf_config_discard(struct cli *cli, const char * const *av, void *priv)
 	vcl->discard = 1;
 	Lck_Unlock(&vcl_mtx);
 
-	if (vcl->busy == 0 && vcl->refcount == 0)
+	if (vcl->temp == VCL_TEMP_COLD)
 		VCL_Nuke(vcl);
 }
 
diff --git a/bin/varnishtest/tests/v00045.vtc b/bin/varnishtest/tests/v00045.vtc
index 1cc1259..936dec9 100644
--- a/bin/varnishtest/tests/v00045.vtc
+++ b/bin/varnishtest/tests/v00045.vtc
@@ -6,7 +6,7 @@ server s1 -start
 varnish v1 -vcl+backend {
 	import ${vmod_debug};
 	sub vcl_init {
-		debug.vcl_release_delay(2s);
+		debug.vcl_release_delay(3s);
 	}
 } -start
 
@@ -21,9 +21,19 @@ shell {
 	grep "auto/cooling.*vcl1" >/dev/null
 }
 
+# It can't be warmed up yet
+delay 1
+shell {
+	${varnishadm} -n ${v1_name} vcl.state vcl1 warm 2>/dev/null |
+	grep "vmod-debug ref on vcl1" >/dev/null
+}
+
 # It will eventually cool down
 delay 2
 shell {
 	${varnishadm} -n ${v1_name} vcl.list |
 	grep "auto/cold.*vcl1" >/dev/null
 }
+
+# At this point it becomes possible to warm up again
+varnish v1 -cliok "vcl.state vcl1 warm"
diff --git a/doc/sphinx/reference/vmod.rst b/doc/sphinx/reference/vmod.rst
index c6eb274..a5cd8ce 100644
--- a/doc/sphinx/reference/vmod.rst
+++ b/doc/sphinx/reference/vmod.rst
@@ -397,6 +397,28 @@ reference by calling ``VRT_ref_vcl`` when you receive a ``VCL_EVENT_WARM`` and
 later calling ``VRT_rel_vcl`` once the background job is over. Receiving a
 ``VCL_EVENT_COLD`` is your cue to terminate any background job bound to a VCL.
 
+You can find an example of VCL references in vmod-debug::
+
+	priv_vcl->vclref = VRT_ref_vcl(ctx, "vmod-debug");
+	...
+	VRT_rel_vcl(&ctx, &priv_vcl->vclref);
+
+In this simplified version, you can see that you need at least a VCL-bound data
+structure like a ``PRIV_VCL`` or a VMOD object to keep track of the reference
+and later release it. You also have to provide a description, it will be printed
+to the user if they try to warm up a cooling VCL::
+
+	$ varnishadm vcl.list
+	available  auto/cooling       0 vcl1
+	active     auto/warm          0 vcl2
+
+	$ varnishadm vcl.state vcl1 warm
+	Command failed with error code 300
+	Failed <vcl.state vcl1 auto>
+	Message:
+		VCL vcl1 is waiting for:
+		- vmod-debug
+
 In the case where properly releasing resources may take some time, you can
 opt for an asynchronous worker, either by spawning a thread and tracking it, or
 by using Varnish's worker pools.
diff --git a/include/vrt.h b/include/vrt.h
index d3113a8..1f19cd4 100644
--- a/include/vrt.h
+++ b/include/vrt.h
@@ -296,8 +296,9 @@ struct vmod_priv {
 typedef int vmod_event_f(VRT_CTX, struct vmod_priv *, enum vcl_event_e);
 #endif
 
-void VRT_ref_vcl(VRT_CTX);
-void VRT_rel_vcl(VRT_CTX);
+struct vclref;
+struct vclref * VRT_ref_vcl(VRT_CTX, const char *);
+void VRT_rel_vcl(VRT_CTX, struct vclref **);
 
 void VRT_priv_fini(const struct vmod_priv *p);
 struct vmod_priv *VRT_priv_task(VRT_CTX, void *vmod_id);
diff --git a/lib/libvmod_debug/vmod_debug.c b/lib/libvmod_debug/vmod_debug.c
index 9485268..76287df 100644
--- a/lib/libvmod_debug/vmod_debug.c
+++ b/lib/libvmod_debug/vmod_debug.c
@@ -45,6 +45,8 @@ struct priv_vcl {
 #define PRIV_VCL_MAGIC		0x8E62FA9D
 	char			*foo;
 	uintptr_t		exp_cb;
+	struct vcl		*vcl;
+	struct vclref		*vclref;
 };
 
 static VCL_DURATION vcl_release_delay = 0.0;
@@ -248,6 +250,8 @@ priv_vcl_free(void *priv)
 		EXP_Deregister_Callback(&priv_vcl->exp_cb);
 		VSL(SLT_Debug, 0, "exp_cb: deregistered");
 	}
+	AZ(priv_vcl->vcl);
+	AZ(priv_vcl->vclref);
 	FREE_OBJ(priv_vcl);
 	AZ(priv_vcl);
 }
@@ -273,8 +277,10 @@ event_load(VRT_CTX, struct vmod_priv *priv)
 }
 
 static int
-event_warm(VRT_CTX)
+event_warm(VRT_CTX, struct vmod_priv *priv)
 {
+	struct priv_vcl *priv_vcl;
+	char buf[32];
 
 	VSL(SLT_Debug, 0, "%s: VCL_EVENT_WARM", VCL_Name(ctx->vcl));
 
@@ -284,7 +290,13 @@ event_warm(VRT_CTX)
 		return (-1);
 	}
 
-	VRT_ref_vcl(ctx);
+	CAST_OBJ_NOTNULL(priv_vcl, priv->priv, PRIV_VCL_MAGIC);
+	AZ(priv_vcl->vcl);
+	AZ(priv_vcl->vclref);
+
+	snprintf(buf, sizeof buf, "vmod-debug ref on %s", VCL_Name(ctx->vcl));
+	priv_vcl->vcl = ctx->vcl;
+	priv_vcl->vclref = VRT_ref_vcl(ctx, buf);
 	return (0);
 }
 
@@ -292,29 +304,40 @@ static void*
 cooldown_thread(void *priv)
 {
 	struct vrt_ctx ctx;
+	struct priv_vcl *priv_vcl;
+
+	CAST_OBJ_NOTNULL(priv_vcl, priv, PRIV_VCL_MAGIC);
+	AN(priv_vcl->vcl);
+	AN(priv_vcl->vclref);
 
-	AN(priv);
 	INIT_OBJ(&ctx, VRT_CTX_MAGIC);
-	ctx.vcl = (struct vcl*)priv;
+	ctx.vcl = priv_vcl->vcl;
 
 	VTIM_sleep(vcl_release_delay);
-	VRT_rel_vcl(&ctx);
+	VRT_rel_vcl(&ctx, &priv_vcl->vclref);
+	priv_vcl->vcl = NULL;
 	return (NULL);
 }
 
 static int
-event_cold(VRT_CTX)
+event_cold(VRT_CTX, struct vmod_priv *priv)
 {
 	pthread_t thread;
+	struct priv_vcl *priv_vcl;
+
+	CAST_OBJ_NOTNULL(priv_vcl, priv->priv, PRIV_VCL_MAGIC);
+	AN(priv_vcl->vcl);
+	AN(priv_vcl->vclref);
 
 	VSL(SLT_Debug, 0, "%s: VCL_EVENT_COLD", VCL_Name(ctx->vcl));
 
 	if (vcl_release_delay == 0.0) {
-		VRT_rel_vcl(ctx);
+		VRT_rel_vcl(ctx, &priv_vcl->vclref);
+		priv_vcl->vcl = NULL;
 		return (0);
 	}
 
-	AZ(pthread_create(&thread, NULL, cooldown_thread, ctx->vcl));
+	AZ(pthread_create(&thread, NULL, cooldown_thread, priv_vcl));
 	AZ(pthread_detach(thread));
 	return (0);
 }
@@ -325,8 +348,8 @@ event_function(VRT_CTX, struct vmod_priv *priv, enum vcl_event_e e)
 
 	switch (e) {
 		case VCL_EVENT_LOAD: return event_load(ctx, priv);
-		case VCL_EVENT_COLD: return event_cold(ctx);
-		case VCL_EVENT_WARM: return event_warm(ctx);
+		case VCL_EVENT_WARM: return event_warm(ctx, priv);
+		case VCL_EVENT_COLD: return event_cold(ctx, priv);
 		default: return (0);
 	}
 }



From phk at FreeBSD.org  Mon Jan 18 22:46:13 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Mon, 18 Jan 2016 23:46:13 +0100
Subject: [master] 9957868 Pass the type of VCL method
 (B(ackend)/C(lient)/H(ousekeeping) into the macro-table.
Message-ID: <E1aLIYv-0001Ll-KO@project.varnish-software.com>


commit 9957868abaae3bdb0954d6b6fe5700ad96756b60
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Jan 18 22:45:32 2016 +0000

    Pass the type of VCL method (B(ackend)/C(lient)/H(ousekeeping)
    into the macro-table.

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index b5f67f8..cdcfc3a 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -994,7 +994,7 @@ void VCL_Refresh(struct vcl **);
 void VCL_Rel(struct vcl **);
 const char *VCL_Return_Name(unsigned);
 
-#define VCL_MET_MAC(l,u,b) \
+#define VCL_MET_MAC(l,u,t,b) \
     void VCL_##l##_method(struct vcl *, struct worker *, struct req *, \
 	struct busyobj *bo, void *specific);
 #include "tbl/vcl_returns.h"
diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index 694b486..824a8c5 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -129,7 +129,7 @@ VCL_Method_Name(unsigned m)
 {
 
 	switch (m) {
-#define VCL_MET_MAC(func, upper, bitmap) case VCL_MET_##upper: return (#upper);
+#define VCL_MET_MAC(func, upper, typ, bitmap) case VCL_MET_##upper: return (#upper);
 #include "tbl/vcl_returns.h"
 #undef VCL_MET_MAC
 	default:
@@ -905,7 +905,7 @@ vcl_call_method(struct worker *wrk, struct req *req, struct busyobj *bo,
 	assert(aws == WS_Snapshot(wrk->aws));
 }
 
-#define VCL_MET_MAC(func, upper, bitmap)				\
+#define VCL_MET_MAC(func, upper, typ, bitmap)				\
 void									\
 VCL_##func##_method(struct vcl *vcl, struct worker *wrk,		\
      struct req *req, struct busyobj *bo, void *specific)		\
diff --git a/lib/libvcc/generate.py b/lib/libvcc/generate.py
index c799bed..a54eaa2 100755
--- a/lib/libvcc/generate.py
+++ b/lib/libvcc/generate.py
@@ -139,11 +139,11 @@ returns =(
 	# Housekeeping
 
 	('init',
-		"",
+		"H",
 		('ok', 'fail')
 	),
 	('fini',
-		"",
+		"H",
 		('ok',)
 	),
 )
@@ -993,7 +993,8 @@ fo.write("#endif\n")
 
 fo.write("\n#ifdef VCL_MET_MAC\n")
 for i in ll:
-	fo.write("VCL_MET_MAC(%s, %s," % (i[0].lower(), i[0].upper()))
+	fo.write("VCL_MET_MAC(%s, %s, %s," %
+	    (i[0].lower(), i[0].upper(), i[1]))
 	p = " (\n\t"
 	lll = list(i[2])
 	lll.sort()
diff --git a/lib/libvcc/vcc_compile.c b/lib/libvcc/vcc_compile.c
index cc22496..87331ff 100644
--- a/lib/libvcc/vcc_compile.c
+++ b/lib/libvcc/vcc_compile.c
@@ -66,7 +66,7 @@
 
 struct method method_tab[] = {
 	{ "none", 0U, 0},
-#define VCL_MET_MAC(l,U,m)	{ "vcl_"#l, m, VCL_MET_##U },
+#define VCL_MET_MAC(l,U,t,b)	{ "vcl_"#l, b, VCL_MET_##U },
 #include "tbl/vcl_returns.h"
 #undef VCL_MET_MAC
 	{ NULL, 0U, 0}
@@ -483,7 +483,7 @@ EmitStruct(const struct vcc *tl)
 	Fc(tl, 0, "\t.nsrc = VGC_NSRCS,\n");
 	Fc(tl, 0, "\t.srcname = srcname,\n");
 	Fc(tl, 0, "\t.srcbody = srcbody,\n");
-#define VCL_MET_MAC(l,u,b) \
+#define VCL_MET_MAC(l,u,t,b) \
 	Fc(tl, 0, "\t." #l "_func = VGC_function_vcl_" #l ",\n");
 #include "tbl/vcl_returns.h"
 #undef VCL_MET_MAC
diff --git a/lib/libvcc/vcc_storage.c b/lib/libvcc/vcc_storage.c
index 3882e5a..2c3f81c 100644
--- a/lib/libvcc/vcc_storage.c
+++ b/lib/libvcc/vcc_storage.c
@@ -77,7 +77,7 @@ vcc_Stv_mkvar(struct vcc *tl, const struct token *t, enum var_type fmt)
 
 	v->name = TlDupTok(tl, t);
 	v->r_methods = 0;
-#define VCL_MET_MAC(l,u,b)	v->r_methods |= VCL_MET_##u;
+#define VCL_MET_MAC(l,u,t,b)	v->r_methods |= VCL_MET_##u;
 #include "tbl/vcl_returns.h"
 #undef VCL_MET_MAC
 	v->fmt = fmt;



From phk at FreeBSD.org  Mon Jan 18 23:26:36 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 19 Jan 2016 00:26:36 +0100
Subject: [master] 085d3b2 Minor flexelinting
Message-ID: <E1aLJC0-00055q-Du@project.varnish-software.com>


commit 085d3b265f071b1e790a0e73bd52f6bb053bf478
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Jan 18 23:26:19 2016 +0000

    Minor flexelinting

diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index 824a8c5..e167db3 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -401,7 +401,7 @@ VRT_ref_vcl(VRT_CTX, const char *desc)
 	ALLOC_OBJ(ref, VCLREF_MAGIC);
 	AN(ref);
 	ref->vcl = vcl;
-	snprintf(ref->desc, sizeof ref->desc, "%s", desc);
+	bprintf(ref->desc, "%s", desc);
 
 	Lck_Lock(&vcl_mtx);
 	VTAILQ_INSERT_TAIL(&vcl->ref_list, ref, list);
diff --git a/lib/libvmod_debug/vmod_debug.c b/lib/libvmod_debug/vmod_debug.c
index 76287df..b87e673 100644
--- a/lib/libvmod_debug/vmod_debug.c
+++ b/lib/libvmod_debug/vmod_debug.c
@@ -277,7 +277,7 @@ event_load(VRT_CTX, struct vmod_priv *priv)
 }
 
 static int
-event_warm(VRT_CTX, struct vmod_priv *priv)
+event_warm(VRT_CTX, const struct vmod_priv *priv)
 {
 	struct priv_vcl *priv_vcl;
 	char buf[32];
@@ -294,7 +294,7 @@ event_warm(VRT_CTX, struct vmod_priv *priv)
 	AZ(priv_vcl->vcl);
 	AZ(priv_vcl->vclref);
 
-	snprintf(buf, sizeof buf, "vmod-debug ref on %s", VCL_Name(ctx->vcl));
+	bprintf(buf, "vmod-debug ref on %s", VCL_Name(ctx->vcl));
 	priv_vcl->vcl = ctx->vcl;
 	priv_vcl->vclref = VRT_ref_vcl(ctx, buf);
 	return (0);
@@ -320,7 +320,7 @@ cooldown_thread(void *priv)
 }
 
 static int
-event_cold(VRT_CTX, struct vmod_priv *priv)
+event_cold(VRT_CTX, const struct vmod_priv *priv)
 {
 	pthread_t thread;
 	struct priv_vcl *priv_vcl;



From lkarsten at varnish-software.com  Tue Jan 19 14:13:37 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 19 Jan 2016 15:13:37 +0100
Subject: [4.1] ad697c3 Add forgotten targets.
Message-ID: <E1aLX2P-0002hn-HA@project.varnish-software.com>


commit ad697c3c71b790d94b164dd97072facc1c183110
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Tue Jan 19 15:13:35 2016 +0100

    Add forgotten targets.

diff --git a/doc/changes.rst b/doc/changes.rst
index 6f0f5d8..cbb5af4 100644
--- a/doc/changes.rst
+++ b/doc/changes.rst
@@ -9,6 +9,10 @@ Bugs fixed
 * 1825_ - Cannot Start Varnish After Just Restarting The Service
 * 1842_ - Handle missing waiting list gracefully.
 
+.. _1802: https://www.varnish-cache.org/trac/ticket/1802
+.. _1825: https://www.varnish-cache.org/trac/ticket/1825
+.. _1842: https://www.varnish-cache.org/trac/ticket/1842
+
 
 ======================================
 Varnish Cache 4.1.1-beta1 (2016-01-15)



From fgsch at lodoss.net  Tue Jan 19 16:11:57 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Tue, 19 Jan 2016 17:11:57 +0100
Subject: [master] 13886d9 Correct description
Message-ID: <E1aLYsv-00072m-LL@project.varnish-software.com>


commit 13886d94af1483ae8cb8a2c300705c3cf8f15b22
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Tue Jan 19 15:59:32 2016 +0000

    Correct description

diff --git a/include/tbl/params.h b/include/tbl/params.h
index 388c1ee..611640c 100644
--- a/include/tbl/params.h
+++ b/include/tbl/params.h
@@ -620,8 +620,8 @@ PARAM(
 	"Maximum number of bytes of HTTP backend response we will deal "
 	"with.  This is a limit on all bytes up to the double blank line "
 	"which ends the HTTP request.\n"
-	"The memory for the request is allocated from the worker workspace "
-	"(param: thread_pool_workspace) and this parameter limits how much "
+	"The memory for the request is allocated from the backend workspace "
+	"(param: workspace_backend) and this parameter limits how much "
 	"of that the request is allowed to take up.",
 	/* l-text */	"",
 	/* func */	NULL



From lkarsten at varnish-software.com  Wed Jan 20 14:31:29 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Wed, 20 Jan 2016 15:31:29 +0100
Subject: [master] dc80c51 Use the correct word.
Message-ID: <E1aLtnF-0002Bu-Ti@project.varnish-software.com>


commit dc80c5198ea9f5cbe739280c5e2b8cbc0f3b6b2b
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Wed Jan 20 15:30:53 2016 +0100

    Use the correct word.

diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index 107ec45..fda7912 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -454,7 +454,7 @@ cnt_lookup(struct worker *wrk, struct req *req)
 			 * like a pass
 			 */
 			VSLb(req->vsl, SLT_VCL_Error,
-			    "vcl_hit{} returns fetch without busy object."
+			    "vcl_hit{} returns miss without busy object."
 			    "  Doing pass.");
 			req->req_step = R_STP_PASS;
 		}



From lkarsten at varnish-software.com  Wed Jan 20 14:31:55 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Wed, 20 Jan 2016 15:31:55 +0100
Subject: [4.1] 94edcc3 Use the correct word.
Message-ID: <E1aLtnf-0002JS-0G@project.varnish-software.com>


commit 94edcc3d877af7ddb9c72247c01cc2b1a2b5c3a1
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Wed Jan 20 15:30:53 2016 +0100

    Use the correct word.

diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index 7627726..1a59ae2 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -454,7 +454,7 @@ cnt_lookup(struct worker *wrk, struct req *req)
 			 * like a pass
 			 */
 			VSLb(req->vsl, SLT_VCL_Error,
-			    "vcl_hit{} returns fetch without busy object."
+			    "vcl_hit{} returns miss without busy object."
 			    "  Doing pass.");
 			req->req_step = R_STP_PASS;
 		}



From fgsch at lodoss.net  Wed Jan 20 17:24:37 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Wed, 20 Jan 2016 18:24:37 +0100
Subject: [master] 610cbd9 Add a few more variants
Message-ID: <E1aLwUn-0006Ul-VG@project.varnish-software.com>


commit 610cbd9323f8e06eaf76627c56862076fc251096
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Wed Jan 20 17:06:52 2016 +0000

    Add a few more variants
    
    Prompted by #1845.  I'm not entirely sure whether some literals like
    "1.00 " should be accepted but that's the current behaviour.

diff --git a/bin/varnishtest/tests/m00015.vtc b/bin/varnishtest/tests/m00015.vtc
index 077bbaf..a5ec8ba 100644
--- a/bin/varnishtest/tests/m00015.vtc
+++ b/bin/varnishtest/tests/m00015.vtc
@@ -9,15 +9,27 @@ varnish v1 -vcl+backend {
 	import ${vmod_std};
 
 	sub vcl_deliver {
-		set resp.http.x-real = std.real(req.http.foo, 0.0);
-		set resp.http.x-fallback = std.real(req.http.bar, 0.0);
+		set resp.http.t0 = std.real(req.http.foo, 0.0);
+		set resp.http.t1 = std.real(req.http.bar, 0.0);
+		set resp.http.t2 = std.real(req.http.nonexistent, 0.0);
+		set resp.http.t3 = std.real("1.", 0.0);
+		set resp.http.t4 = std.real("1.0", 0.0);
+		set resp.http.t5 = std.real("1.0 ", 0.0);
+		set resp.http.t6 = std.real(" 1.0", 0.0);
+		set resp.http.t7 = std.real(" 1.0 ", 0.0);
 	}
 } -start
 
 client c1 {
-	txreq -hdr "foo: 1.00"
+	txreq -hdr "foo: 1.00" -hdr "bar: 1.00a"
 	rxresp
 	expect resp.status == 200
-	expect resp.http.x-real == 1.000
-	expect resp.http.x-fallback == 0.000
+	expect resp.http.t0 == 1.000
+	expect resp.http.t1 == 0.000
+	expect resp.http.t2 == 0.000
+	expect resp.http.t3 == 1.000
+	expect resp.http.t4 == 1.000
+	expect resp.http.t5 == 1.000
+	expect resp.http.t6 == 1.000
+	expect resp.http.t7 == 1.000
 } -run



From fgsch at lodoss.net  Thu Jan 21 12:50:11 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Thu, 21 Jan 2016 13:50:11 +0100
Subject: [master] 3a78ae0 Display the correct error if varnish.pc is missing
Message-ID: <E1aMEgl-0004rL-AM@project.varnish-software.com>


commit 3a78ae0774a451bb29eba5adbcc7725c1d752a48
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Thu Jan 21 12:39:36 2016 +0000

    Display the correct error if varnish.pc is missing
    
    Discussed with mithrandir at .

diff --git a/varnish.m4 b/varnish.m4
index 8a9f647..105a224 100644
--- a/varnish.m4
+++ b/varnish.m4
@@ -42,7 +42,7 @@ m4_pattern_forbid([^_?VARNISH[A-Z_]+$])
 m4_pattern_allow([^VARNISH_VMOD(_INCLUDE_DIR|TOOL)$])
 # Check for pkg-config
 PKG_CHECK_EXISTS([varnishapi],[],[
-	if test -n "$PKG_CONFIG"; then
+	if test -z "$PKG_CONFIG"; then
 		AC_MSG_FAILURE(
 [The pkg-config script could not be found or is too old.  Make sure it
 is in your PATH or set the PKG_CONFIG environment variable to the full



From lkarsten at varnish-software.com  Fri Jan 22 11:59:06 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Fri, 22 Jan 2016 12:59:06 +0100
Subject: [4.1] 83c1c54 Display the correct error if varnish.pc is missing
Message-ID: <E1aMaMs-0000Qo-GY@project.varnish-software.com>


commit 83c1c54e9385df740d9a14158c055e4a343acca9
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Thu Jan 21 12:39:36 2016 +0000

    Display the correct error if varnish.pc is missing
    
    Discussed with mithrandir at .

diff --git a/varnish.m4 b/varnish.m4
index 8a9f647..105a224 100644
--- a/varnish.m4
+++ b/varnish.m4
@@ -42,7 +42,7 @@ m4_pattern_forbid([^_?VARNISH[A-Z_]+$])
 m4_pattern_allow([^VARNISH_VMOD(_INCLUDE_DIR|TOOL)$])
 # Check for pkg-config
 PKG_CHECK_EXISTS([varnishapi],[],[
-	if test -n "$PKG_CONFIG"; then
+	if test -z "$PKG_CONFIG"; then
 		AC_MSG_FAILURE(
 [The pkg-config script could not be found or is too old.  Make sure it
 is in your PATH or set the PKG_CONFIG environment variable to the full



From lkarsten at varnish-software.com  Fri Jan 22 11:59:06 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Fri, 22 Jan 2016 12:59:06 +0100
Subject: [4.1] 366ce6e Correct description
Message-ID: <E1aMaMs-0000Qu-I9@project.varnish-software.com>


commit 366ce6e0cb3448c49823c631a683fe8d6fe2f6d7
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Tue Jan 19 15:59:32 2016 +0000

    Correct description

diff --git a/include/tbl/params.h b/include/tbl/params.h
index 388c1ee..611640c 100644
--- a/include/tbl/params.h
+++ b/include/tbl/params.h
@@ -620,8 +620,8 @@ PARAM(
 	"Maximum number of bytes of HTTP backend response we will deal "
 	"with.  This is a limit on all bytes up to the double blank line "
 	"which ends the HTTP request.\n"
-	"The memory for the request is allocated from the worker workspace "
-	"(param: thread_pool_workspace) and this parameter limits how much "
+	"The memory for the request is allocated from the backend workspace "
+	"(param: workspace_backend) and this parameter limits how much "
 	"of that the request is allowed to take up.",
 	/* l-text */	"",
 	/* func */	NULL



From fgsch at lodoss.net  Fri Jan 22 12:15:27 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Fri, 22 Jan 2016 13:15:27 +0100
Subject: [master] 0e0b374 Handle whitespace after floats in test fields
Message-ID: <E1aMach-0001sT-V2@project.varnish-software.com>


commit 0e0b37412e1766bccb8e0db447784ce2f23fb307
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Fri Jan 22 12:00:53 2016 +0000

    Handle whitespace after floats in test fields
    
    Broken in 9bb8f96223d248beb5962961b493b16dcc8ecc2a.
    Committed solution proposed by phk@, discussed with phk@ and martin at .
    
    Fixes #1845.

diff --git a/bin/varnishtest/tests/l00001.vtc b/bin/varnishtest/tests/l00001.vtc
index b1ec63c..bcd4391 100644
--- a/bin/varnishtest/tests/l00001.vtc
+++ b/bin/varnishtest/tests/l00001.vtc
@@ -151,6 +151,12 @@ logexpect l1 -d 1 -g vxid -q "RespHeader[2] == 123" {
 	  expect * =	End
 } -run
 
+# Test field on floats
+logexpect l1 -d 1 -g vxid -q "RespHeader[2] == 123." {
+	  expect 0 *	Begin	req
+	  expect * =	End
+} -run
+
 # Test taglists
 logexpect l1 -d 1 -g vxid -q "Debug,Resp* == 200" {
 	  expect 0 *	Begin	req
diff --git a/lib/libvarnishapi/vsl_query.c b/lib/libvarnishapi/vsl_query.c
index 1ad6811..60cc54d 100644
--- a/lib/libvarnishapi/vsl_query.c
+++ b/lib/libvarnishapi/vsl_query.c
@@ -78,7 +78,7 @@ vslq_test_rec(const struct vex *vex, const struct VSLC_ptr *rec)
 	const struct vex_rhs *rhs;
 	long long lhs_int = 0;
 	double lhs_float = 0.;
-	const char *b, *e;
+	const char *b, *e, *q;
 	char *p;
 	int i;
 
@@ -145,9 +145,11 @@ vslq_test_rec(const struct vex *vex, const struct VSLC_ptr *rec)
 			/* Can't parse - no match */
 			return (0);
 		case VEX_FLOAT:
-			lhs_float = VNUM(b);
+			lhs_float = VNUMpfx(b, &q);
 			if (isnan(lhs_float))
 				return (0);
+			if (q != NULL && q > p && !isspace(q[-1]))
+				return (0);
 			break;
 		default:
 			WRONG("Wrong RHS type");



From fgsch at lodoss.net  Fri Jan 22 12:37:03 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Fri, 22 Jan 2016 13:37:03 +0100
Subject: [master] 9a5bb77 Use the right variable
Message-ID: <E1aMaxb-0003TX-S9@project.varnish-software.com>


commit 9a5bb779a2db0a4cffd96a31483348572f97e96d
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Fri Jan 22 12:35:16 2016 +0000

    Use the right variable
    
    Fixes previous commit. Sigh.

diff --git a/lib/libvarnishapi/vsl_query.c b/lib/libvarnishapi/vsl_query.c
index 60cc54d..1741c13 100644
--- a/lib/libvarnishapi/vsl_query.c
+++ b/lib/libvarnishapi/vsl_query.c
@@ -148,7 +148,7 @@ vslq_test_rec(const struct vex *vex, const struct VSLC_ptr *rec)
 			lhs_float = VNUMpfx(b, &q);
 			if (isnan(lhs_float))
 				return (0);
-			if (q != NULL && q > p && !isspace(q[-1]))
+			if (q != NULL && q > b && !isspace(q[-1]))
 				return (0);
 			break;
 		default:



From lkarsten at varnish-software.com  Fri Jan 22 15:46:58 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Fri, 22 Jan 2016 16:46:58 +0100
Subject: [4.1] 83e5fc5 Turn VCL state magic numbers into an enum
Message-ID: <E1aMdvO-0001iv-Js@project.varnish-software.com>


commit 83e5fc599b8a3680049384cb412ff61c78c97a6c
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Fri Dec 4 09:48:11 2015 +0100

    Turn VCL state magic numbers into an enum
    
    Make a clear distinction between (struct vclprog).warm that is used as a
    boolean, unlike the second parameter of mgt_vcl_setstate. We don't use
    an actual C enum but const char[] to make debugging easier.

diff --git a/bin/varnishd/mgt/mgt_vcl.c b/bin/varnishd/mgt/mgt_vcl.c
index 65b50bf..b651f83 100644
--- a/bin/varnishd/mgt/mgt_vcl.c
+++ b/bin/varnishd/mgt/mgt_vcl.c
@@ -47,11 +47,15 @@
 
 #include "mgt_cli.h"
 
+static const char * const VCL_STATE_COLD = "cold";
+static const char * const VCL_STATE_WARM = "warm";
+static const char * const VCL_STATE_AUTO = "auto";
+
 struct vclprog {
 	VTAILQ_ENTRY(vclprog)	list;
 	char			*name;
 	char			*fname;
-	int			warm;
+	unsigned		warm;
 	char			state[8];
 	double			go_cold;
 };
@@ -119,21 +123,24 @@ mgt_has_vcl(void)
 }
 
 static void
-mgt_vcl_setstate(struct vclprog *vp, int warm)
+mgt_vcl_setstate(struct vclprog *vp, const char *vs)
 {
-	unsigned status;
+	unsigned status, warm;
 	double now;
 	char *p;
 
-	if (warm == -1) {
+	if (vs == VCL_STATE_AUTO) {
 		assert(vp != active_vcl);
 		now = VTIM_mono();
-		warm = vp->warm;
+		vs = vp->warm ? VCL_STATE_WARM : VCL_STATE_COLD;
 		if (vp->go_cold > 0 && !strcmp(vp->state, "auto") &&
 		    vp->go_cold + mgt_param.vcl_cooldown < now)
-			warm = 0;
+			vs = VCL_STATE_COLD;
 	}
 
+	assert(vs != VCL_STATE_AUTO);
+	warm = vs == VCL_STATE_WARM ? 1 : 0;
+
 	if (vp->warm == warm)
 		return;
 
@@ -237,7 +244,7 @@ mgt_push_vcls_and_start(unsigned *status, char **p)
 	struct vclprog *vp;
 
 	AN(active_vcl);
-	mgt_vcl_setstate(active_vcl, 1);
+	mgt_vcl_setstate(active_vcl, VCL_STATE_WARM);
 	VTAILQ_FOREACH(vp, &vclhead, list) {
 		if (mgt_cli_askchild(status, p, "vcl.load \"%s\" %s %d%s\n",
 		    vp->name, vp->fname, vp->warm, vp->state))
@@ -319,7 +326,7 @@ mcf_vcl_state(struct cli *cli, const char * const *av, void *priv)
 		bprintf(vp->state, "%s", "auto");
 		if (vp != active_vcl) {
 			vp->go_cold = VTIM_mono();
-			mgt_vcl_setstate(vp, -1);
+			mgt_vcl_setstate(vp, VCL_STATE_AUTO);
 		}
 	} else if (!strcmp(av[3], "cold")) {
 		if (vp == active_vcl) {
@@ -328,10 +335,10 @@ mcf_vcl_state(struct cli *cli, const char * const *av, void *priv)
 			return;
 		}
 		bprintf(vp->state, "%s", "auto");
-		mgt_vcl_setstate(vp, 0);
+		mgt_vcl_setstate(vp, VCL_STATE_COLD);
 	} else if (!strcmp(av[3], "warm")) {
 		bprintf(vp->state, "%s", av[3]);
-		mgt_vcl_setstate(vp, 1);
+		mgt_vcl_setstate(vp, VCL_STATE_WARM);
 	} else {
 		VCLI_Out(cli, "State must be one of auto, cold or warm.");
 		VCLI_SetResult(cli, CLIS_PARAM);
@@ -351,20 +358,20 @@ mcf_vcl_use(struct cli *cli, const char * const *av, void *priv)
 		return;
 	if (vp == active_vcl)
 		return;
-	mgt_vcl_setstate(vp, 1);
+	mgt_vcl_setstate(vp, VCL_STATE_WARM);
 	if (child_pid >= 0 &&
 	    mgt_cli_askchild(&status, &p, "vcl.use %s\n", av[2])) {
 		VCLI_SetResult(cli, status);
 		VCLI_Out(cli, "%s", p);
 		vp->go_cold = VTIM_mono();
-		mgt_vcl_setstate(vp, -1);
+		mgt_vcl_setstate(vp, VCL_STATE_AUTO);
 	} else {
 		VCLI_Out(cli, "VCL '%s' now active", av[2]);
 		vp2 = active_vcl;
 		active_vcl = vp;
 		if (vp2 != NULL) {
 			vp2->go_cold = VTIM_mono();
-			mgt_vcl_setstate(vp2, -1);
+			mgt_vcl_setstate(vp2, VCL_STATE_AUTO);
 		}
 	}
 	free(p);
@@ -386,7 +393,7 @@ mcf_vcl_discard(struct cli *cli, const char * const *av, void *priv)
 		VCLI_Out(cli, "Cannot discard active VCL program\n");
 		return;
 	}
-	mgt_vcl_setstate(vp, 0);
+	mgt_vcl_setstate(vp, VCL_STATE_COLD);
 	if (child_pid >= 0) {
 		/* If this fails the child is crashing, figure that later */
 		(void)mgt_cli_askchild(&status, &p, "vcl.discard %s\n", av[2]);
@@ -432,7 +439,7 @@ mgt_vcl_poker(const struct vev *e, int what)
 	e_poker->timeout = mgt_param.vcl_cooldown * .45;
 	VTAILQ_FOREACH(vp, &vclhead, list) {
 		if (vp != active_vcl)
-			mgt_vcl_setstate(vp, -1);
+			mgt_vcl_setstate(vp, VCL_STATE_AUTO);
 	}
 	return (0);
 }



From lkarsten at varnish-software.com  Fri Jan 22 15:46:58 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Fri, 22 Jan 2016 16:46:58 +0100
Subject: [4.1] 37abc30 Wrap VCL event calls in dedicated functions
Message-ID: <E1aMdvO-0001j2-LY@project.varnish-software.com>


commit 37abc3010603a7b264d0964281fadebf502400c6
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Thu Dec 10 11:44:05 2015 +0100

    Wrap VCL event calls in dedicated functions
    
    The two functions are separated in order to have self-documenting
    signatures because one class of events is (or will be) allowed to
    fail while the other MUST be failsafe.

diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index e19c9ce..b1b7756 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -428,6 +428,35 @@ vcl_find(const char *name)
 	return (NULL);
 }
 
+static int
+vcl_setup_event(VRT_CTX, enum vcl_event_e ev)
+{
+
+	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
+	AN(ctx->handling);
+	AN(ctx->vcl);
+	assert(ev == VCL_EVENT_LOAD || ev == VCL_EVENT_WARM ||
+	    ev == VCL_EVENT_USE);
+
+	if (ev == VCL_EVENT_LOAD)
+		AN(ctx->msg);
+
+	return (ctx->vcl->conf->event_vcl(ctx, ev));
+}
+
+static void
+vcl_failsafe_event(VRT_CTX, enum vcl_event_e ev)
+{
+
+	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
+	AN(ctx->handling);
+	AN(ctx->vcl);
+	assert(ev == VCL_EVENT_COLD || ev == VCL_EVENT_DISCARD);
+
+	if (ctx->vcl->conf->event_vcl(ctx, ev) != 0)
+		WRONG("A VMOD cannot fail COLD or DISCARD events");
+}
+
 static void
 vcl_set_state(VRT_CTX, const char *state)
 {
@@ -448,7 +477,7 @@ vcl_set_state(VRT_CTX, const char *state)
 
 			vcl->temp = vcl->refcount ? VCL_TEMP_COOLING :
 			    VCL_TEMP_COLD;
-			AZ(vcl->conf->event_vcl(ctx, VCL_EVENT_COLD));
+			vcl_failsafe_event(ctx, VCL_EVENT_COLD);
 			vcl_BackendEvent(vcl, VCL_EVENT_COLD);
 		}
 		else if (vcl->busy)
@@ -465,7 +494,7 @@ vcl_set_state(VRT_CTX, const char *state)
 		/* The VCL must first reach a stable cold state */
 		else if (vcl->temp != VCL_TEMP_COOLING) {
 			vcl->temp = VCL_TEMP_WARM;
-			(void)vcl->conf->event_vcl(ctx, VCL_EVENT_WARM);
+			(void)vcl_setup_event(ctx, VCL_EVENT_WARM);
 			vcl_BackendEvent(vcl, VCL_EVENT_WARM);
 		}
 		break;
@@ -515,13 +544,13 @@ VCL_Load(struct cli *cli, const char *name, const char *fn, const char *state)
 
 	VSB_clear(vsb);
 	ctx.msg = vsb;
-	i = vcl->conf->event_vcl(&ctx, VCL_EVENT_LOAD);
+	i = vcl_setup_event(&ctx, VCL_EVENT_LOAD);
 	AZ(VSB_finish(vsb));
 	if (i) {
 		VCLI_Out(cli, "VCL \"%s\" Failed initialization", name);
 		if (VSB_len(vsb))
 			VCLI_Out(cli, "\nMessage:\n\t%s", VSB_data(vsb));
-		AZ(vcl->conf->event_vcl(&ctx, VCL_EVENT_DISCARD));
+		vcl_failsafe_event(&ctx, VCL_EVENT_DISCARD);
 		vcl_KillBackends(vcl);
 		VCL_Close(&vcl);
 		VSB_delete(vsb);
@@ -563,7 +592,7 @@ VCL_Nuke(struct vcl *vcl)
 	ctx.method = VCL_MET_FINI;
 	ctx.handling = &hand;
 	ctx.vcl = vcl;
-	AZ(vcl->conf->event_vcl(&ctx, VCL_EVENT_DISCARD));
+	vcl_failsafe_event(&ctx, VCL_EVENT_DISCARD);
 	vcl_KillBackends(vcl);
 	free(vcl->loaded_name);
 	VCL_Close(&vcl);
@@ -688,7 +717,7 @@ ccf_config_use(struct cli *cli, const char * const *av, void *priv)
 	AN(vsb);
 	ctx.msg = vsb;
 	ctx.vcl = vcl;
-	i = vcl->conf->event_vcl(&ctx, VCL_EVENT_USE);
+	i = vcl_setup_event(&ctx, VCL_EVENT_USE);
 	AZ(VSB_finish(vsb));
 	if (i) {
 		VCLI_Out(cli, "VCL \"%s\" Failed to activate", av[2]);



From lkarsten at varnish-software.com  Fri Jan 22 15:46:58 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Fri, 22 Jan 2016 16:46:58 +0100
Subject: [4.1] 3c2d14c Allow VMODs to fail a warm-up
Message-ID: <E1aMdvO-0001jB-Mw@project.varnish-software.com>


commit 3c2d14c60ec0abda66becc06371f48c9cb45affb
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Fri Dec 4 11:45:16 2015 +0100

    Allow VMODs to fail a warm-up
    
    It is also possible to convey a message to the CLI. In case of a failure
    the VCL is cooled down. VCL_Load may now fail with either CLIS_PARAM or
    CLIS_CANT.

diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index b1b7756..45d3e8b 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -438,7 +438,7 @@ vcl_setup_event(VRT_CTX, enum vcl_event_e ev)
 	assert(ev == VCL_EVENT_LOAD || ev == VCL_EVENT_WARM ||
 	    ev == VCL_EVENT_USE);
 
-	if (ev == VCL_EVENT_LOAD)
+	if (ev != VCL_EVENT_USE)
 		AN(ctx->msg);
 
 	return (ctx->vcl->conf->event_vcl(ctx, ev));
@@ -457,14 +457,18 @@ vcl_failsafe_event(VRT_CTX, enum vcl_event_e ev)
 		WRONG("A VMOD cannot fail COLD or DISCARD events");
 }
 
-static void
+static int
 vcl_set_state(VRT_CTX, const char *state)
 {
 	struct vcl *vcl;
+	int i = 0;
 
 	ASSERT_CLI();
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
 	AN(ctx->handling);
+	AN(ctx->vcl);
+	AN(state);
+	assert(ctx->msg != NULL || *state == '0');
 
 	vcl = ctx->vcl;
 	AN(vcl->temp);
@@ -494,16 +498,36 @@ vcl_set_state(VRT_CTX, const char *state)
 		/* The VCL must first reach a stable cold state */
 		else if (vcl->temp != VCL_TEMP_COOLING) {
 			vcl->temp = VCL_TEMP_WARM;
-			(void)vcl_setup_event(ctx, VCL_EVENT_WARM);
-			vcl_BackendEvent(vcl, VCL_EVENT_WARM);
+			i = vcl_setup_event(ctx, VCL_EVENT_WARM);
+			if (i == 0)
+				vcl_BackendEvent(vcl, VCL_EVENT_WARM);
+			else
+				AZ(vcl->conf->event_vcl(ctx, VCL_EVENT_COLD));
 		}
 		break;
 	default:
 		WRONG("Wrong enum state");
 	}
+	return (i);
 }
 
-static int
+static void
+vcl_cancel_load(VRT_CTX, struct cli *cli, const char *name, const char *step)
+{
+	struct vcl *vcl = ctx->vcl;
+
+	AZ(VSB_finish(ctx->msg));
+	VCLI_SetResult(cli, CLIS_CANT);
+	VCLI_Out(cli, "VCL \"%s\" Failed %s", name, step);
+	if (VSB_len(ctx->msg))
+		VCLI_Out(cli, "\nMessage:\n\t%s", VSB_data(ctx->msg));
+	AZ(vcl->conf->event_vcl(ctx, VCL_EVENT_DISCARD));
+	vcl_KillBackends(vcl);
+	VCL_Close(&vcl);
+	VSB_delete(ctx->msg);
+}
+
+static void
 VCL_Load(struct cli *cli, const char *name, const char *fn, const char *state)
 {
 	struct vcl *vcl;
@@ -516,8 +540,9 @@ VCL_Load(struct cli *cli, const char *name, const char *fn, const char *state)
 
 	vcl = vcl_find(name);
 	if (vcl != NULL) {
+		VCLI_SetResult(cli, CLIS_PARAM);
 		VCLI_Out(cli, "Config '%s' already loaded", name);
-		return (1);
+		return;
 	}
 
 	vsb = VSB_new_auto();
@@ -526,9 +551,10 @@ VCL_Load(struct cli *cli, const char *name, const char *fn, const char *state)
 	vcl = VCL_Open(fn, vsb);
 	if (vcl == NULL) {
 		AZ(VSB_finish(vsb));
+		VCLI_SetResult(cli, CLIS_PARAM);
 		VCLI_Out(cli, "%s", VSB_data(vsb));
 		VSB_delete(vsb);
-		return (1);
+		return;
 	}
 
 	vcl->loaded_name = strdup(name);
@@ -545,19 +571,18 @@ VCL_Load(struct cli *cli, const char *name, const char *fn, const char *state)
 	VSB_clear(vsb);
 	ctx.msg = vsb;
 	i = vcl_setup_event(&ctx, VCL_EVENT_LOAD);
-	AZ(VSB_finish(vsb));
 	if (i) {
-		VCLI_Out(cli, "VCL \"%s\" Failed initialization", name);
-		if (VSB_len(vsb))
-			VCLI_Out(cli, "\nMessage:\n\t%s", VSB_data(vsb));
-		vcl_failsafe_event(&ctx, VCL_EVENT_DISCARD);
-		vcl_KillBackends(vcl);
-		VCL_Close(&vcl);
-		VSB_delete(vsb);
-		return (1);
+		vcl_cancel_load(&ctx, cli, name, "initialization");
+		return;
+	}
+	VSB_clear(vsb);
+	i = vcl_set_state(&ctx, state);
+	if (i) {
+		assert(*state == '1');
+		vcl_cancel_load(&ctx, cli, name, "warmup");
+		return;
 	}
 	VSB_delete(vsb);
-	vcl_set_state(&ctx, state);
 	bprintf(vcl->state, "%s", state + 1);
 	assert(hand == VCL_RET_OK);
 	VCLI_Out(cli, "Loaded \"%s\" as \"%s\"", fn , name);
@@ -568,7 +593,6 @@ VCL_Load(struct cli *cli, const char *name, const char *fn, const char *state)
 	Lck_Unlock(&vcl_mtx);
 	VSC_C_main->n_vcl++;
 	VSC_C_main->n_vcl_avail++;
-	return (0);
 }
 
 /*--------------------------------------------------------------------
@@ -616,7 +640,7 @@ VCL_Poll(void)
 			INIT_OBJ(&ctx, VRT_CTX_MAGIC);
 			ctx.vcl = vcl;
 			ctx.handling = &hand;
-			vcl_set_state(&ctx, "0");
+			(void)vcl_set_state(&ctx, "0");
 		}
 		if (vcl->discard && vcl->temp == VCL_TEMP_COLD)
 			VCL_Nuke(vcl);
@@ -652,8 +676,7 @@ ccf_config_load(struct cli *cli, const char * const *av, void *priv)
 
 	AZ(priv);
 	ASSERT_CLI();
-	if (VCL_Load(cli, av[2], av[3], av[4]))
-		VCLI_SetResult(cli, CLIS_PARAM);
+	VCL_Load(cli, av[2], av[3], av[4]);
 }
 
 static void __match_proto__(cli_func_t)
@@ -663,6 +686,8 @@ ccf_config_state(struct cli *cli, const char * const *av, void *priv)
 	unsigned hand;
 
 	INIT_OBJ(&ctx, VRT_CTX_MAGIC);
+	ctx.msg = VSB_new_auto();
+	AN(ctx.msg);
 	ctx.handling = &hand;
 
 	(void)cli;
@@ -672,8 +697,18 @@ ccf_config_state(struct cli *cli, const char * const *av, void *priv)
 	AN(av[3]);
 	ctx.vcl = vcl_find(av[2]);
 	AN(ctx.vcl);			// MGT ensures this
-	vcl_set_state(&ctx, av[3]);
-	bprintf(ctx.vcl->state, "%s", av[3] + 1);
+	if (vcl_set_state(&ctx, av[3]) == 0) {
+		bprintf(ctx.vcl->state, "%s", av[3] + 1);
+		VSB_delete(ctx.msg);
+		return;
+	}
+	AZ(VSB_finish(ctx.msg));
+	VCLI_SetResult(cli, CLIS_CANT);
+	VCLI_Out(cli, "Failed <vcl.state %s %s>", ctx.vcl->loaded_name,
+	    av[3] + 1);
+	if (VSB_len(ctx.msg))
+		VCLI_Out(cli, "\nMessage:\n\t%s", VSB_data(ctx.msg));
+	VSB_delete(ctx.msg);
 }
 
 static void __match_proto__(cli_func_t)



From lkarsten at varnish-software.com  Fri Jan 22 15:46:58 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Fri, 22 Jan 2016 16:46:58 +0100
Subject: [4.1] fcd4a30 Make VMODs actually fail warm-ups
Message-ID: <E1aMdvO-0001jL-OT@project.varnish-software.com>


commit fcd4a3083b111299c46dc3bbac02f49611c7d998
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Mon Dec 7 11:18:33 2015 +0100

    Make VMODs actually fail warm-ups
    
    The implementation is similar to the load/discard dance when a load
    fails. New VGC functions are introduced iff the VCL has at least one
    VMOD handling events.
    
    The generated code looks like this:
    
    	static unsigned vgc_inistep;
    	static unsigned vgc_warmupstep;
    
    	...
    
    	static int
    	VGC_Load(VRT_CTX)
    	{
    		...
    	}
    
    	static int
    	VGC_Discard(VRT_CTX)
    	{
    		...
    	}
    
    	static int
    	VGC_Warmup(VRT_CTX, enum vcl_event_e ev)
    	{
    
    		vgc_warmupstep = 0;
    
    		/* 4 */
    		if (Vmod_debug_Func._event(ctx, &vmod_priv_debug, ev))
    			return (1);
    		vgc_warmupstep = 4;
    
    		return (0);
    	}
    
    	static int
    	VGC_Use(VRT_CTX, enum vcl_event_e ev)
    	{
    
    		/* 4 */
    		if (Vmod_debug_Func._event(ctx, &vmod_priv_debug, ev))
    			return (1);
    
    		return (0);
    	}
    
    	static int
    	VGC_Cooldown(VRT_CTX, enum vcl_event_e ev)
    	{
    		int retval = 0;
    
    		/* 4 */
    		if (vgc_warmupstep >= 4 &&
    		    Vmod_debug_Func._event(ctx, &vmod_priv_debug, ev) != 0)
    			retval = 1;
    
    		return (retval);
    	}
    
    	static int
    	VGC_Event(VRT_CTX, enum vcl_event_e ev)
    	{
    		if (ev == VCL_EVENT_LOAD)
    			return(VGC_Load(ctx));
    		if (ev == VCL_EVENT_WARM)
    			return(VGC_Warmup(ctx, ev));
    		if (ev == VCL_EVENT_USE)
    			return(VGC_Use(ctx, ev));
    		if (ev == VCL_EVENT_COLD)
    			return(VGC_Cooldown(ctx, ev));
    		if (ev == VCL_EVENT_DISCARD)
    			return(VGC_Discard(ctx));
    
    		return (1);
    	}
    
    However, if there are no VMODs handling events, no new functions shall
    be generated, leading to code looking like this:
    
    	static unsigned vgc_inistep;
    	static unsigned vgc_warmupstep;
    
    	...
    
    	static int
    	VGC_Load(VRT_CTX)
    	{
    		...
    	}
    
    	static int
    	VGC_Discard(VRT_CTX)
    	{
    		...
    	}
    
    	static int
    	VGC_Event(VRT_CTX, enum vcl_event_e ev)
    	{
    		if (ev == VCL_EVENT_LOAD)
    			return(VGC_Load(ctx));
    		if (ev == VCL_EVENT_DISCARD)
    			return(VGC_Discard(ctx));
    
    		(void)vgc_warmupstep;
    		return (0);
    	}

diff --git a/doc/sphinx/reference/vmod.rst b/doc/sphinx/reference/vmod.rst
index 57fa09c..c6eb274 100644
--- a/doc/sphinx/reference/vmod.rst
+++ b/doc/sphinx/reference/vmod.rst
@@ -383,6 +383,13 @@ first with a ``VCL_EVENT_WARM`` event. Unless a user decides that a given VCL
 should always be warm, an inactive VMOD will eventually become cold and should
 manage resources accordingly.
 
+An event function must return zero upon success. It is only possible to fail
+an initialization with the ``VCL_EVENT_LOAD`` or ``VCL_EVENT_WARM`` events.
+Should such a failure happen, a ``VCL_EVENT_DISCARD`` or ``VCL_EVENT_COLD``
+event will be sent to the VMODs that succeeded to put them back in a cold
+state. The VMOD that failed will not receive this event, and therefore must
+not be left half-initialized should a failure occur.
+
 If your VMOD is running an asynchronous background job you can hold a reference
 to the VCL to prevent it from going cold too soon and get the same guarantees
 as backends with ongoing requests for instance. For that, you must acquire the
diff --git a/lib/libvcc/vcc_compile.c b/lib/libvcc/vcc_compile.c
index 62aaeae..da6d8b4 100644
--- a/lib/libvcc/vcc_compile.c
+++ b/lib/libvcc/vcc_compile.c
@@ -325,20 +325,22 @@ EmitCoordinates(const struct vcc *tl, struct vsb *vsb)
 /*--------------------------------------------------------------------
  * Init/Fini/Event
  *
- * We call Fini-s in the opposite order of init-s.
- * Other events are called in same order as init-s, no matter which
- * event it might be.
+ * We call DISCARD and COLD events in the opposite order of LOAD and
+ * WARM.
  */
 
 static void
 EmitInitFini(const struct vcc *tl)
 {
 	struct inifin *p;
+	unsigned has_event = 0;
 
-	Fh(tl, 0, "\nstatic unsigned vgc_inistep;\n");
+	Fh(tl, 0, "\n");
+	Fh(tl, 0, "static unsigned vgc_inistep;\n");
+	Fh(tl, 0, "static unsigned vgc_warmupstep;\n");
 
 	/*
-	 * INIT
+	 * LOAD
 	 */
 	Fc(tl, 0, "\nstatic int\nVGC_Load(VRT_CTX)\n{\n\n");
 	Fc(tl, 0, "\tvgc_inistep = 0;\n\n");
@@ -349,6 +351,10 @@ EmitInitFini(const struct vcc *tl)
 			Fc(tl, 0, "\t/* %u */\n%s\n", p->n, VSB_data(p->ini));
 		Fc(tl, 0, "\tvgc_inistep = %u;\n\n", p->n);
 		VSB_delete(p->ini);
+
+		AZ(VSB_finish(p->event));
+		if (VSB_len(p->event))
+			has_event = 1;
 	}
 
 	Fc(tl, 0, "\t(void)VGC_function_vcl_init(ctx);\n");
@@ -356,7 +362,7 @@ EmitInitFini(const struct vcc *tl)
 	Fc(tl, 0, "}\n");
 
 	/*
-	 * FINI
+	 * DISCARD
 	 */
 	Fc(tl, 0, "\nstatic int\nVGC_Discard(VRT_CTX)\n{\n\n");
 
@@ -375,6 +381,66 @@ EmitInitFini(const struct vcc *tl)
 	Fc(tl, 0, "\treturn(0);\n");
 	Fc(tl, 0, "}\n");
 
+	if (has_event) {
+		/*
+		 * WARM
+		 */
+		Fc(tl, 0, "\nstatic int\n");
+		Fc(tl, 0, "VGC_Warmup(VRT_CTX, enum vcl_event_e ev)\n{\n\n");
+
+		Fc(tl, 0, "\tvgc_warmupstep = 0;\n\n");
+		VTAILQ_FOREACH(p, &tl->inifin, list) {
+			assert(p->n > 0);
+			if (VSB_len(p->event)) {
+				Fc(tl, 0, "\t/* %u */\n", p->n);
+				Fc(tl, 0, "\tif (%s)\n", VSB_data(p->event));
+				Fc(tl, 0, "\t\treturn (1);\n");
+				Fc(tl, 0, "\tvgc_warmupstep = %u;\n\n", p->n);
+			}
+		}
+
+		Fc(tl, 0, "\treturn (0);\n");
+		Fc(tl, 0, "}\n");
+
+		/*
+		 * USE (deprecated)
+		 */
+		Fc(tl, 0, "\nstatic int\n");
+		Fc(tl, 0, "VGC_Use(VRT_CTX, enum vcl_event_e ev)\n{\n\n");
+
+		VTAILQ_FOREACH(p, &tl->inifin, list) {
+			assert(p->n > 0);
+			if (VSB_len(p->event)) {
+				Fc(tl, 0, "\t/* %u */\n", p->n);
+				Fc(tl, 0, "\tif (%s)\n", VSB_data(p->event));
+				Fc(tl, 0, "\t\treturn (1);\n\n");
+			}
+		}
+
+		Fc(tl, 0, "\treturn (0);\n");
+		Fc(tl, 0, "}\n");
+
+		/*
+		 * COLD
+		 */
+		Fc(tl, 0, "\nstatic int\n");
+		Fc(tl, 0, "VGC_Cooldown(VRT_CTX, enum vcl_event_e ev)\n{\n");
+		Fc(tl, 0, "\tint retval = 0;\n\n");
+
+		VTAILQ_FOREACH_REVERSE(p, &tl->inifin, inifinhead, list) {
+			if (VSB_len(p->event)) {
+				Fc(tl, 0, "\t/* %u */\n", p->n);
+				Fc(tl, 0, "\tif (vgc_warmupstep >= %u &&\n", p->n);
+				Fc(tl, 0, "\t    %s != 0)\n", VSB_data(p->event));
+				Fc(tl, 0, "\t\tretval = 1;\n\n");
+			}
+			VSB_delete(p->event);
+		}
+
+		Fc(tl, 0, "\treturn (retval);\n");
+		Fc(tl, 0, "}\n");
+	}
+
 	/*
 	 * EVENTS
 	 */
@@ -383,16 +449,20 @@ EmitInitFini(const struct vcc *tl)
 	Fc(tl, 0, "{\n");
 	Fc(tl, 0, "\tif (ev == VCL_EVENT_LOAD)\n");
 	Fc(tl, 0, "\t\treturn(VGC_Load(ctx));\n");
+	if (has_event) {
+		Fc(tl, 0, "\tif (ev == VCL_EVENT_WARM)\n");
+		Fc(tl, 0, "\t\treturn(VGC_Warmup(ctx, ev));\n");
+		Fc(tl, 0, "\tif (ev == VCL_EVENT_USE)\n");
+		Fc(tl, 0, "\t\treturn(VGC_Use(ctx, ev));\n");
+		Fc(tl, 0, "\tif (ev == VCL_EVENT_COLD)\n");
+		Fc(tl, 0, "\t\treturn(VGC_Cooldown(ctx, ev));\n");
+	}
 	Fc(tl, 0, "\tif (ev == VCL_EVENT_DISCARD)\n");
 	Fc(tl, 0, "\t\treturn(VGC_Discard(ctx));\n");
 	Fc(tl, 0, "\n");
-	VTAILQ_FOREACH(p, &tl->inifin, list) {
-		AZ(VSB_finish(p->event));
-		if (VSB_len(p->event))
-			Fc(tl, 0, "\t/* %u */\n%s\n", p->n, VSB_data(p->event));
-		VSB_delete(p->event);
-	}
-	Fc(tl, 0, "\treturn (0);\n");
+	if (!has_event)
+		Fc(tl, 0, "\t(void)vgc_warmupstep;\n");
+	Fc(tl, 0, "\treturn (%d);\n", has_event ? 1 : 0);
 	Fc(tl, 0, "}\n");
 }
 
diff --git a/lib/libvcc/vcc_vmod.c b/lib/libvcc/vcc_vmod.c
index d1b8f26..3a522a5 100644
--- a/lib/libvcc/vcc_vmod.c
+++ b/lib/libvcc/vcc_vmod.c
@@ -228,8 +228,7 @@ vcc_ParseImport(struct vcc *tl)
 			VSB_printf(ifp->fin,
 			    "\t\t(void)%s(ctx, &vmod_priv_%.*s,\n"
 			    "\t\t    VCL_EVENT_DISCARD);\n", p, PF(mod));
-			VSB_printf(ifp->event,
-			    "\t(void)%s(ctx, &vmod_priv_%.*s, ev);\n",
+			VSB_printf(ifp->event, "\t%s(ctx, &vmod_priv_%.*s, ev)",
 			    p, PF(mod));
 		} else {
 			sym = VCC_AddSymbolStr(tl, p, SYM_FUNC);



From lkarsten at varnish-software.com  Fri Jan 22 15:46:58 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Fri, 22 Jan 2016 16:46:58 +0100
Subject: [4.1] 8dd463a Catch a vcl.state failure on the manager side
Message-ID: <E1aMdvO-0001jV-QP@project.varnish-software.com>


commit 8dd463a32e3cb9c17762c44148dcdafc2ec39441
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Mon Dec 7 12:07:27 2015 +0100

    Catch a vcl.state failure on the manager side
    
    Don't update the state of the VCL to warm if it failed, and don't start
    the child if the active VCL failed to warm up.

diff --git a/bin/varnishd/mgt/mgt.h b/bin/varnishd/mgt/mgt.h
index 1aeba55..fe96360 100644
--- a/bin/varnishd/mgt/mgt.h
+++ b/bin/varnishd/mgt/mgt.h
@@ -167,7 +167,7 @@ void mgt_vcc_init(void);
 void mgt_vcl_init(void);
 void mgt_vcc_startup(struct cli *, const char *b_arg, const char *f_arg,
     const char *vclsrc, int Cflag);
-int mgt_push_vcls_and_start(unsigned *status, char **p);
+int mgt_push_vcls_and_start(struct cli *, unsigned *status, char **p);
 int mgt_has_vcl(void);
 extern char *mgt_cc_cmd;
 extern const char *mgt_vcl_dir;
diff --git a/bin/varnishd/mgt/mgt_child.c b/bin/varnishd/mgt/mgt_child.c
index 11578ce..4c5ffc3 100644
--- a/bin/varnishd/mgt/mgt_child.c
+++ b/bin/varnishd/mgt/mgt_child.c
@@ -418,7 +418,8 @@ mgt_launch_child(struct cli *cli)
 
 	mgt_cli_start_child(child_cli_in, child_cli_out);
 	child_pid = pid;
-	if (mgt_push_vcls_and_start(&u, &p)) {
+	if (mgt_push_vcls_and_start(cli, &u, &p)) {
+		VCLI_SetResult(cli, u);
 		MGT_complain(C_ERR, "Child (%jd) Pushing vcls failed:\n%s",
 		    (intmax_t)child_pid, p);
 		free(p);
diff --git a/bin/varnishd/mgt/mgt_vcl.c b/bin/varnishd/mgt/mgt_vcl.c
index b651f83..858c5ec 100644
--- a/bin/varnishd/mgt/mgt_vcl.c
+++ b/bin/varnishd/mgt/mgt_vcl.c
@@ -122,12 +122,13 @@ mgt_has_vcl(void)
 	return (!VTAILQ_EMPTY(&vclhead));
 }
 
-static void
-mgt_vcl_setstate(struct vclprog *vp, const char *vs)
+static int
+mgt_vcl_setstate(struct cli *cli, struct vclprog *vp, const char *vs)
 {
 	unsigned status, warm;
 	double now;
 	char *p;
+	int i;
 
 	if (vs == VCL_STATE_AUTO) {
 		assert(vp != active_vcl);
@@ -142,7 +143,7 @@ mgt_vcl_setstate(struct vclprog *vp, const char *vs)
 	warm = vs == VCL_STATE_WARM ? 1 : 0;
 
 	if (vp->warm == warm)
-		return;
+		return (0);
 
 	vp->warm = warm;
 
@@ -150,15 +151,19 @@ mgt_vcl_setstate(struct vclprog *vp, const char *vs)
 		vp->go_cold = 0;
 
 	if (child_pid < 0)
-		return;
+		return (0);
 
-	/*
-	 * We ignore the result here so we don't croak if the child did.
-	 */
-	(void)mgt_cli_askchild(&status, &p, "vcl.state %s %d%s\n",
+	i = mgt_cli_askchild(&status, &p, "vcl.state %s %d%s\n",
 	    vp->name, vp->warm, vp->state);
+	if (i) {
+		AN(cli);
+		AN(vp->warm);
+		VCLI_SetResult(cli, status);
+		VCLI_Out(cli, "%s", p);
+	}
 
 	free(p);
+	return (i);
 }
 
 /*--------------------------------------------------------------------*/
@@ -239,12 +244,15 @@ mgt_vcc_startup(struct cli *cli, const char *b_arg, const char *f_arg,
 /*--------------------------------------------------------------------*/
 
 int
-mgt_push_vcls_and_start(unsigned *status, char **p)
+mgt_push_vcls_and_start(struct cli *cli, unsigned *status, char **p)
 {
 	struct vclprog *vp;
 
 	AN(active_vcl);
-	mgt_vcl_setstate(active_vcl, VCL_STATE_WARM);
+
+	/* The VCL has not been loaded yet, it cannot fail */
+	AZ(mgt_vcl_setstate(cli, active_vcl, VCL_STATE_WARM));
+
 	VTAILQ_FOREACH(vp, &vclhead, list) {
 		if (mgt_cli_askchild(status, p, "vcl.load \"%s\" %s %d%s\n",
 		    vp->name, vp->fname, vp->warm, vp->state))
@@ -326,7 +334,7 @@ mcf_vcl_state(struct cli *cli, const char * const *av, void *priv)
 		bprintf(vp->state, "%s", "auto");
 		if (vp != active_vcl) {
 			vp->go_cold = VTIM_mono();
-			mgt_vcl_setstate(vp, VCL_STATE_AUTO);
+			(void)mgt_vcl_setstate(cli, vp, VCL_STATE_AUTO);
 		}
 	} else if (!strcmp(av[3], "cold")) {
 		if (vp == active_vcl) {
@@ -335,10 +343,10 @@ mcf_vcl_state(struct cli *cli, const char * const *av, void *priv)
 			return;
 		}
 		bprintf(vp->state, "%s", "auto");
-		mgt_vcl_setstate(vp, VCL_STATE_COLD);
+		(void)mgt_vcl_setstate(cli, vp, VCL_STATE_COLD);
 	} else if (!strcmp(av[3], "warm")) {
-		bprintf(vp->state, "%s", av[3]);
-		mgt_vcl_setstate(vp, VCL_STATE_WARM);
+		if (mgt_vcl_setstate(cli, vp, VCL_STATE_WARM) == 0)
+			bprintf(vp->state, "%s", av[3]);
 	} else {
 		VCLI_Out(cli, "State must be one of auto, cold or warm.");
 		VCLI_SetResult(cli, CLIS_PARAM);
@@ -358,20 +366,21 @@ mcf_vcl_use(struct cli *cli, const char * const *av, void *priv)
 		return;
 	if (vp == active_vcl)
 		return;
-	mgt_vcl_setstate(vp, VCL_STATE_WARM);
+	if (mgt_vcl_setstate(cli, vp, VCL_STATE_WARM))
+		return;
 	if (child_pid >= 0 &&
 	    mgt_cli_askchild(&status, &p, "vcl.use %s\n", av[2])) {
 		VCLI_SetResult(cli, status);
 		VCLI_Out(cli, "%s", p);
 		vp->go_cold = VTIM_mono();
-		mgt_vcl_setstate(vp, VCL_STATE_AUTO);
+		(void)mgt_vcl_setstate(cli, vp, VCL_STATE_AUTO);
 	} else {
 		VCLI_Out(cli, "VCL '%s' now active", av[2]);
 		vp2 = active_vcl;
 		active_vcl = vp;
 		if (vp2 != NULL) {
 			vp2->go_cold = VTIM_mono();
-			mgt_vcl_setstate(vp2, VCL_STATE_AUTO);
+			(void)mgt_vcl_setstate(cli, vp2, VCL_STATE_AUTO);
 		}
 	}
 	free(p);
@@ -393,9 +402,9 @@ mcf_vcl_discard(struct cli *cli, const char * const *av, void *priv)
 		VCLI_Out(cli, "Cannot discard active VCL program\n");
 		return;
 	}
-	mgt_vcl_setstate(vp, VCL_STATE_COLD);
+	(void)mgt_vcl_setstate(cli, vp, VCL_STATE_COLD);
 	if (child_pid >= 0) {
-		/* If this fails the child is crashing, figure that later */
+		/* XXX If this fails the child is crashing, figure that later */
 		(void)mgt_cli_askchild(&status, &p, "vcl.discard %s\n", av[2]);
 		free(p);
 	}
@@ -439,7 +448,7 @@ mgt_vcl_poker(const struct vev *e, int what)
 	e_poker->timeout = mgt_param.vcl_cooldown * .45;
 	VTAILQ_FOREACH(vp, &vclhead, list) {
 		if (vp != active_vcl)
-			mgt_vcl_setstate(vp, VCL_STATE_AUTO);
+			(void)mgt_vcl_setstate(NULL, vp, VCL_STATE_AUTO);
 	}
 	return (0);
 }
diff --git a/bin/varnishtest/tests/v00044.vtc b/bin/varnishtest/tests/v00044.vtc
index aa79e6a..c056d6c 100644
--- a/bin/varnishtest/tests/v00044.vtc
+++ b/bin/varnishtest/tests/v00044.vtc
@@ -83,3 +83,11 @@ delay .4
 varnish v1 -expect VBE.vcl1.default.happy >= 0
 delay 4
 varnish v1 -expect !VBE.vcl1.default.happy
+
+# A VMOD's warm-up can fail
+varnish v1 -cliok "param.set max_esi_depth 42"
+varnish v1 -clierr 300 "vcl.state vcl1 warm"
+
+# A warm-up failure can also fail a child start
+varnish v1 -cliok stop
+varnish v1 -clierr 300 start



From lkarsten at varnish-software.com  Fri Jan 22 15:46:58 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Fri, 22 Jan 2016 16:46:58 +0100
Subject: [4.1] 88e4ad9 Simplify `vcl.use` by making it failsafe
Message-ID: <E1aMdvO-0001jf-Sc@project.varnish-software.com>


commit 88e4ad90a3a3312aa260f22cdd5f17e6ba376db0
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Mon Dec 7 15:02:13 2015 +0100

    Simplify `vcl.use` by making it failsafe
    
    By the time we decide to switch to a VCL, it must be warm and usable.
    The deprecated VCL_EVENT_USE should not get in the way.

diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index 45d3e8b..e10f0c8 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -435,11 +435,8 @@ vcl_setup_event(VRT_CTX, enum vcl_event_e ev)
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
 	AN(ctx->handling);
 	AN(ctx->vcl);
-	assert(ev == VCL_EVENT_LOAD || ev == VCL_EVENT_WARM ||
-	    ev == VCL_EVENT_USE);
-
-	if (ev != VCL_EVENT_USE)
-		AN(ctx->msg);
+	AN(ctx->msg);
+	assert(ev == VCL_EVENT_LOAD || ev == VCL_EVENT_WARM);
 
 	return (ctx->vcl->conf->event_vcl(ctx, ev));
 }
@@ -451,10 +448,11 @@ vcl_failsafe_event(VRT_CTX, enum vcl_event_e ev)
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
 	AN(ctx->handling);
 	AN(ctx->vcl);
-	assert(ev == VCL_EVENT_COLD || ev == VCL_EVENT_DISCARD);
+	assert(ev == VCL_EVENT_COLD || ev == VCL_EVENT_DISCARD ||
+	    ev == VCL_EVENT_USE);
 
 	if (ctx->vcl->conf->event_vcl(ctx, ev) != 0)
-		WRONG("A VMOD cannot fail COLD or DISCARD events");
+		WRONG("A VMOD cannot fail USE, COLD or DISCARD events");
 }
 
 static int
@@ -738,33 +736,20 @@ ccf_config_use(struct cli *cli, const char * const *av, void *priv)
 	struct vcl *vcl;
 	struct vrt_ctx ctx;
 	unsigned hand = 0;
-	struct vsb *vsb;
-	int i;
 
 	ASSERT_CLI();
+	AN(cli);
 	AZ(priv);
 	vcl = vcl_find(av[2]);
 	AN(vcl);				// MGT ensures this
 	assert(vcl->temp == VCL_TEMP_WARM);	// MGT ensures this
 	INIT_OBJ(&ctx, VRT_CTX_MAGIC);
 	ctx.handling = &hand;
-	vsb = VSB_new_auto();
-	AN(vsb);
-	ctx.msg = vsb;
 	ctx.vcl = vcl;
-	i = vcl_setup_event(&ctx, VCL_EVENT_USE);
-	AZ(VSB_finish(vsb));
-	if (i) {
-		VCLI_Out(cli, "VCL \"%s\" Failed to activate", av[2]);
-		if (VSB_len(vsb) > 0)
-			VCLI_Out(cli, "\nMessage:\n\t%s", VSB_data(vsb));
-		VCLI_SetResult(cli, CLIS_CANT);
-	} else {
-		Lck_Lock(&vcl_mtx);
-		vcl_active = vcl;
-		Lck_Unlock(&vcl_mtx);
-	}
-	VSB_delete(vsb);
+	vcl_failsafe_event(&ctx, VCL_EVENT_USE);
+	Lck_Lock(&vcl_mtx);
+	vcl_active = vcl;
+	Lck_Unlock(&vcl_mtx);
 }
 
 static void __match_proto__(cli_func_t)
diff --git a/lib/libvcc/vcc_compile.c b/lib/libvcc/vcc_compile.c
index da6d8b4..cc22496 100644
--- a/lib/libvcc/vcc_compile.c
+++ b/lib/libvcc/vcc_compile.c
@@ -326,7 +326,8 @@ EmitCoordinates(const struct vcc *tl, struct vsb *vsb)
  * Init/Fini/Event
  *
  * We call DISCARD and COLD events in the opposite order of LOAD and
- * WARM.
+ * WARM. The child will panic if a USE event fails, since a WARM event
+ * leads to a usable state.
  */
 
 static void
diff --git a/lib/libvmod_debug/vmod_debug.c b/lib/libvmod_debug/vmod_debug.c
index a9e2cde..9485268 100644
--- a/lib/libvmod_debug/vmod_debug.c
+++ b/lib/libvmod_debug/vmod_debug.c
@@ -278,6 +278,7 @@ event_warm(VRT_CTX)
 
 	VSL(SLT_Debug, 0, "%s: VCL_EVENT_WARM", VCL_Name(ctx->vcl));
 
+	AN(ctx->msg);
 	if (cache_param->max_esi_depth == 42) {
 		VSB_printf(ctx->msg, "max_esi_depth is not the answer.");
 		return (-1);



From lkarsten at varnish-software.com  Fri Jan 22 15:46:58 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Fri, 22 Jan 2016 16:46:58 +0100
Subject: [4.1] 1589700 Replace the VCL refcount by a self-desribing list
Message-ID: <E1aMdvO-0001jr-UU@project.varnish-software.com>


commit 158970071cd2e1ed3ea97367b6106f2d6ed82597
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Mon Dec 7 17:06:36 2015 +0100

    Replace the VCL refcount by a self-desribing list
    
    Instead of counting the references, the VCL keeps track of them with up
    to 31 characters in the description. It is the VMOD's responsibility to
    keep track of the opaque struct vclref * and provide a meaningful user-
    friendly description.

diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index e10f0c8..694b486 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -62,10 +62,18 @@ struct vcl {
 	char			state[8];
 	char			*loaded_name;
 	unsigned		busy;
-	unsigned		refcount;
 	unsigned		discard;
 	const char		*temp;
 	VTAILQ_HEAD(,backend)	backend_list;
+	VTAILQ_HEAD(,vclref)	ref_list;
+};
+
+struct vclref {
+	unsigned		magic;
+#define VCLREF_MAGIC		0x47fb6848
+	const struct vcl	*vcl;
+	VTAILQ_ENTRY(vclref)	list;
+	char			desc[32];
 };
 
 /*
@@ -252,7 +260,7 @@ vcl_KillBackends(struct vcl *vcl)
 
 	CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC);
 	AZ(vcl->busy);
-	AZ(vcl->refcount);
+	assert(VTAILQ_EMPTY(&vcl->ref_list));
 	while (1) {
 		be = VTAILQ_FIRST(&vcl->backend_list);
 		if (be == NULL)
@@ -375,40 +383,59 @@ VRT_count(VRT_CTX, unsigned u)
 		    ctx->vcl->conf->ref[u].line, ctx->vcl->conf->ref[u].pos);
 }
 
-void
-VRT_ref_vcl(VRT_CTX)
+struct vclref *
+VRT_ref_vcl(VRT_CTX, const char *desc)
 {
 	struct vcl *vcl;
+	struct vclref* ref;
 
 	ASSERT_CLI();
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
+	AN(desc);
+	AN(*desc);
 
 	vcl = ctx->vcl;
 	CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC);
 	xxxassert(vcl->temp == VCL_TEMP_WARM);
 
+	ALLOC_OBJ(ref, VCLREF_MAGIC);
+	AN(ref);
+	ref->vcl = vcl;
+	snprintf(ref->desc, sizeof ref->desc, "%s", desc);
+
 	Lck_Lock(&vcl_mtx);
-	vcl->refcount++;
+	VTAILQ_INSERT_TAIL(&vcl->ref_list, ref, list);
 	Lck_Unlock(&vcl_mtx);
+
+	return (ref);
 }
 
 void
-VRT_rel_vcl(VRT_CTX)
+VRT_rel_vcl(VRT_CTX, struct vclref **refp)
 {
 	struct vcl *vcl;
+	struct vclref *ref;
+
+	AN(refp);
+	ref = *refp;
+	*refp = NULL;
 
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
+	CHECK_OBJ_NOTNULL(ref, VCLREF_MAGIC);
 
 	vcl = ctx->vcl;
 	CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC);
+	assert(vcl == ref->vcl);
 	assert(vcl->temp == VCL_TEMP_WARM || vcl->temp == VCL_TEMP_BUSY ||
 	    vcl->temp == VCL_TEMP_COOLING);
 
 	Lck_Lock(&vcl_mtx);
-	assert(vcl->refcount > 0);
-	vcl->refcount--;
+	assert(!VTAILQ_EMPTY(&vcl->ref_list));
+	VTAILQ_REMOVE(&vcl->ref_list, ref, list);
 	/* No garbage collection here, for the same reasons as in VCL_Rel. */
 	Lck_Unlock(&vcl_mtx);
+
+	FREE_OBJ(ref);
 }
 
 /*--------------------------------------------------------------------*/
@@ -455,6 +482,23 @@ vcl_failsafe_event(VRT_CTX, enum vcl_event_e ev)
 		WRONG("A VMOD cannot fail USE, COLD or DISCARD events");
 }
 
+static void
+vcl_print_refs(VRT_CTX)
+{
+	struct vcl *vcl;
+	struct vclref *ref;
+
+	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
+	CHECK_OBJ_NOTNULL(ctx->vcl, VCL_MAGIC);
+	AN(ctx->msg);
+	vcl = ctx->vcl;
+	VSB_printf(ctx->msg, "VCL %s is waiting for:", vcl->loaded_name);
+	Lck_Lock(&vcl_mtx);
+	VTAILQ_FOREACH(ref, &ctx->vcl->ref_list, list)
+		VSB_printf(ctx->msg, "\n\t- %s", ref->desc);
+	Lck_Unlock(&vcl_mtx);
+}
+
 static int
 vcl_set_state(VRT_CTX, const char *state)
 {
@@ -477,16 +521,17 @@ vcl_set_state(VRT_CTX, const char *state)
 		if (vcl->busy == 0 && (vcl->temp == VCL_TEMP_WARM ||
 		    vcl->temp == VCL_TEMP_BUSY)) {
 
-			vcl->temp = vcl->refcount ? VCL_TEMP_COOLING :
-			    VCL_TEMP_COLD;
+			vcl->temp = VTAILQ_EMPTY(&vcl->ref_list) ?
+			    VCL_TEMP_COLD : VCL_TEMP_COOLING;
 			vcl_failsafe_event(ctx, VCL_EVENT_COLD);
 			vcl_BackendEvent(vcl, VCL_EVENT_COLD);
 		}
 		else if (vcl->busy)
 			vcl->temp = VCL_TEMP_BUSY;
+		else if (VTAILQ_EMPTY(&vcl->ref_list))
+			vcl->temp = VCL_TEMP_COLD;
 		else
-			vcl->temp = vcl->refcount ? VCL_TEMP_COOLING :
-			    VCL_TEMP_COLD;
+			vcl->temp = VCL_TEMP_COOLING;
 		break;
 	case '1':
 		assert(vcl->temp != VCL_TEMP_WARM);
@@ -494,7 +539,11 @@ vcl_set_state(VRT_CTX, const char *state)
 		if (vcl->temp == VCL_TEMP_BUSY)
 			vcl->temp = VCL_TEMP_WARM;
 		/* The VCL must first reach a stable cold state */
-		else if (vcl->temp != VCL_TEMP_COOLING) {
+		else if (vcl->temp == VCL_TEMP_COOLING) {
+			vcl_print_refs(ctx);
+			i = -1;
+		}
+		else {
 			vcl->temp = VCL_TEMP_WARM;
 			i = vcl_setup_event(ctx, VCL_EVENT_WARM);
 			if (i == 0)
@@ -558,6 +607,7 @@ VCL_Load(struct cli *cli, const char *name, const char *fn, const char *state)
 	vcl->loaded_name = strdup(name);
 	XXXAN(vcl->loaded_name);
 	VTAILQ_INIT(&vcl->backend_list);
+	VTAILQ_INIT(&vcl->ref_list);
 
 	vcl->temp = VCL_TEMP_INIT;
 
@@ -609,7 +659,7 @@ VCL_Nuke(struct vcl *vcl)
 	assert(vcl != vcl_active);
 	assert(vcl->discard);
 	AZ(vcl->busy);
-	AZ(vcl->refcount);
+	assert(VTAILQ_EMPTY(&vcl->ref_list));
 	VTAILQ_REMOVE(&vcl_head, vcl, list);
 	ctx.method = VCL_MET_FINI;
 	ctx.handling = &hand;
@@ -726,7 +776,7 @@ ccf_config_discard(struct cli *cli, const char * const *av, void *priv)
 	vcl->discard = 1;
 	Lck_Unlock(&vcl_mtx);
 
-	if (vcl->busy == 0 && vcl->refcount == 0)
+	if (vcl->temp == VCL_TEMP_COLD)
 		VCL_Nuke(vcl);
 }
 
diff --git a/bin/varnishtest/tests/v00045.vtc b/bin/varnishtest/tests/v00045.vtc
index 1cc1259..936dec9 100644
--- a/bin/varnishtest/tests/v00045.vtc
+++ b/bin/varnishtest/tests/v00045.vtc
@@ -6,7 +6,7 @@ server s1 -start
 varnish v1 -vcl+backend {
 	import ${vmod_debug};
 	sub vcl_init {
-		debug.vcl_release_delay(2s);
+		debug.vcl_release_delay(3s);
 	}
 } -start
 
@@ -21,9 +21,19 @@ shell {
 	grep "auto/cooling.*vcl1" >/dev/null
 }
 
+# It can't be warmed up yet
+delay 1
+shell {
+	${varnishadm} -n ${v1_name} vcl.state vcl1 warm 2>/dev/null |
+	grep "vmod-debug ref on vcl1" >/dev/null
+}
+
 # It will eventually cool down
 delay 2
 shell {
 	${varnishadm} -n ${v1_name} vcl.list |
 	grep "auto/cold.*vcl1" >/dev/null
 }
+
+# At this point it becomes possible to warm up again
+varnish v1 -cliok "vcl.state vcl1 warm"
diff --git a/doc/sphinx/reference/vmod.rst b/doc/sphinx/reference/vmod.rst
index c6eb274..a5cd8ce 100644
--- a/doc/sphinx/reference/vmod.rst
+++ b/doc/sphinx/reference/vmod.rst
@@ -397,6 +397,28 @@ reference by calling ``VRT_ref_vcl`` when you receive a ``VCL_EVENT_WARM`` and
 later calling ``VRT_rel_vcl`` once the background job is over. Receiving a
 ``VCL_EVENT_COLD`` is your cue to terminate any background job bound to a VCL.
 
+You can find an example of VCL references in vmod-debug::
+
+	priv_vcl->vclref = VRT_ref_vcl(ctx, "vmod-debug");
+	...
+	VRT_rel_vcl(&ctx, &priv_vcl->vclref);
+
+In this simplified version, you can see that you need at least a VCL-bound data
+structure like a ``PRIV_VCL`` or a VMOD object to keep track of the reference
+and later release it. You also have to provide a description, it will be printed
+to the user if they try to warm up a cooling VCL::
+
+	$ varnishadm vcl.list
+	available  auto/cooling       0 vcl1
+	active     auto/warm          0 vcl2
+
+	$ varnishadm vcl.state vcl1 warm
+	Command failed with error code 300
+	Failed <vcl.state vcl1 auto>
+	Message:
+		VCL vcl1 is waiting for:
+		- vmod-debug
+
 In the case where properly releasing resources may take some time, you can
 opt for an asynchronous worker, either by spawning a thread and tracking it, or
 by using Varnish's worker pools.
diff --git a/include/vrt.h b/include/vrt.h
index 4cfd71d..ad43ef6 100644
--- a/include/vrt.h
+++ b/include/vrt.h
@@ -295,8 +295,9 @@ struct vmod_priv {
 typedef int vmod_event_f(VRT_CTX, struct vmod_priv *, enum vcl_event_e);
 #endif
 
-void VRT_ref_vcl(VRT_CTX);
-void VRT_rel_vcl(VRT_CTX);
+struct vclref;
+struct vclref * VRT_ref_vcl(VRT_CTX, const char *);
+void VRT_rel_vcl(VRT_CTX, struct vclref **);
 
 void VRT_priv_fini(const struct vmod_priv *p);
 struct vmod_priv *VRT_priv_task(VRT_CTX, void *vmod_id);
diff --git a/lib/libvmod_debug/vmod_debug.c b/lib/libvmod_debug/vmod_debug.c
index 9485268..76287df 100644
--- a/lib/libvmod_debug/vmod_debug.c
+++ b/lib/libvmod_debug/vmod_debug.c
@@ -45,6 +45,8 @@ struct priv_vcl {
 #define PRIV_VCL_MAGIC		0x8E62FA9D
 	char			*foo;
 	uintptr_t		exp_cb;
+	struct vcl		*vcl;
+	struct vclref		*vclref;
 };
 
 static VCL_DURATION vcl_release_delay = 0.0;
@@ -248,6 +250,8 @@ priv_vcl_free(void *priv)
 		EXP_Deregister_Callback(&priv_vcl->exp_cb);
 		VSL(SLT_Debug, 0, "exp_cb: deregistered");
 	}
+	AZ(priv_vcl->vcl);
+	AZ(priv_vcl->vclref);
 	FREE_OBJ(priv_vcl);
 	AZ(priv_vcl);
 }
@@ -273,8 +277,10 @@ event_load(VRT_CTX, struct vmod_priv *priv)
 }
 
 static int
-event_warm(VRT_CTX)
+event_warm(VRT_CTX, struct vmod_priv *priv)
 {
+	struct priv_vcl *priv_vcl;
+	char buf[32];
 
 	VSL(SLT_Debug, 0, "%s: VCL_EVENT_WARM", VCL_Name(ctx->vcl));
 
@@ -284,7 +290,13 @@ event_warm(VRT_CTX)
 		return (-1);
 	}
 
-	VRT_ref_vcl(ctx);
+	CAST_OBJ_NOTNULL(priv_vcl, priv->priv, PRIV_VCL_MAGIC);
+	AZ(priv_vcl->vcl);
+	AZ(priv_vcl->vclref);
+
+	snprintf(buf, sizeof buf, "vmod-debug ref on %s", VCL_Name(ctx->vcl));
+	priv_vcl->vcl = ctx->vcl;
+	priv_vcl->vclref = VRT_ref_vcl(ctx, buf);
 	return (0);
 }
 
@@ -292,29 +304,40 @@ static void*
 cooldown_thread(void *priv)
 {
 	struct vrt_ctx ctx;
+	struct priv_vcl *priv_vcl;
+
+	CAST_OBJ_NOTNULL(priv_vcl, priv, PRIV_VCL_MAGIC);
+	AN(priv_vcl->vcl);
+	AN(priv_vcl->vclref);
 
-	AN(priv);
 	INIT_OBJ(&ctx, VRT_CTX_MAGIC);
-	ctx.vcl = (struct vcl*)priv;
+	ctx.vcl = priv_vcl->vcl;
 
 	VTIM_sleep(vcl_release_delay);
-	VRT_rel_vcl(&ctx);
+	VRT_rel_vcl(&ctx, &priv_vcl->vclref);
+	priv_vcl->vcl = NULL;
 	return (NULL);
 }
 
 static int
-event_cold(VRT_CTX)
+event_cold(VRT_CTX, struct vmod_priv *priv)
 {
 	pthread_t thread;
+	struct priv_vcl *priv_vcl;
+
+	CAST_OBJ_NOTNULL(priv_vcl, priv->priv, PRIV_VCL_MAGIC);
+	AN(priv_vcl->vcl);
+	AN(priv_vcl->vclref);
 
 	VSL(SLT_Debug, 0, "%s: VCL_EVENT_COLD", VCL_Name(ctx->vcl));
 
 	if (vcl_release_delay == 0.0) {
-		VRT_rel_vcl(ctx);
+		VRT_rel_vcl(ctx, &priv_vcl->vclref);
+		priv_vcl->vcl = NULL;
 		return (0);
 	}
 
-	AZ(pthread_create(&thread, NULL, cooldown_thread, ctx->vcl));
+	AZ(pthread_create(&thread, NULL, cooldown_thread, priv_vcl));
 	AZ(pthread_detach(thread));
 	return (0);
 }
@@ -325,8 +348,8 @@ event_function(VRT_CTX, struct vmod_priv *priv, enum vcl_event_e e)
 
 	switch (e) {
 		case VCL_EVENT_LOAD: return event_load(ctx, priv);
-		case VCL_EVENT_COLD: return event_cold(ctx);
-		case VCL_EVENT_WARM: return event_warm(ctx);
+		case VCL_EVENT_WARM: return event_warm(ctx, priv);
+		case VCL_EVENT_COLD: return event_cold(ctx, priv);
 		default: return (0);
 	}
 }



From lkarsten at varnish-software.com  Fri Jan 22 15:46:59 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Fri, 22 Jan 2016 16:46:59 +0100
Subject: [4.1] 681da73 Pass the type of VCL method
 (B(ackend)/C(lient)/H(ousekeeping) into the macro-table.
Message-ID: <E1aMdvP-0001k1-0O@project.varnish-software.com>


commit 681da739746dbc336c79c8f7b357e45ebacae1f2
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Jan 18 22:45:32 2016 +0000

    Pass the type of VCL method (B(ackend)/C(lient)/H(ousekeeping)
    into the macro-table.

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index 6c6a6ef..f908af1 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -1005,7 +1005,7 @@ void VCL_Refresh(struct vcl **);
 void VCL_Rel(struct vcl **);
 const char *VCL_Return_Name(unsigned);
 
-#define VCL_MET_MAC(l,u,b) \
+#define VCL_MET_MAC(l,u,t,b) \
     void VCL_##l##_method(struct vcl *, struct worker *, struct req *, \
 	struct busyobj *bo, void *specific);
 #include "tbl/vcl_returns.h"
diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index 694b486..824a8c5 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -129,7 +129,7 @@ VCL_Method_Name(unsigned m)
 {
 
 	switch (m) {
-#define VCL_MET_MAC(func, upper, bitmap) case VCL_MET_##upper: return (#upper);
+#define VCL_MET_MAC(func, upper, typ, bitmap) case VCL_MET_##upper: return (#upper);
 #include "tbl/vcl_returns.h"
 #undef VCL_MET_MAC
 	default:
@@ -905,7 +905,7 @@ vcl_call_method(struct worker *wrk, struct req *req, struct busyobj *bo,
 	assert(aws == WS_Snapshot(wrk->aws));
 }
 
-#define VCL_MET_MAC(func, upper, bitmap)				\
+#define VCL_MET_MAC(func, upper, typ, bitmap)				\
 void									\
 VCL_##func##_method(struct vcl *vcl, struct worker *wrk,		\
      struct req *req, struct busyobj *bo, void *specific)		\
diff --git a/lib/libvcc/generate.py b/lib/libvcc/generate.py
index c799bed..a54eaa2 100755
--- a/lib/libvcc/generate.py
+++ b/lib/libvcc/generate.py
@@ -139,11 +139,11 @@ returns =(
 	# Housekeeping
 
 	('init',
-		"",
+		"H",
 		('ok', 'fail')
 	),
 	('fini',
-		"",
+		"H",
 		('ok',)
 	),
 )
@@ -993,7 +993,8 @@ fo.write("#endif\n")
 
 fo.write("\n#ifdef VCL_MET_MAC\n")
 for i in ll:
-	fo.write("VCL_MET_MAC(%s, %s," % (i[0].lower(), i[0].upper()))
+	fo.write("VCL_MET_MAC(%s, %s, %s," %
+	    (i[0].lower(), i[0].upper(), i[1]))
 	p = " (\n\t"
 	lll = list(i[2])
 	lll.sort()
diff --git a/lib/libvcc/vcc_compile.c b/lib/libvcc/vcc_compile.c
index cc22496..87331ff 100644
--- a/lib/libvcc/vcc_compile.c
+++ b/lib/libvcc/vcc_compile.c
@@ -66,7 +66,7 @@
 
 struct method method_tab[] = {
 	{ "none", 0U, 0},
-#define VCL_MET_MAC(l,U,m)	{ "vcl_"#l, m, VCL_MET_##U },
+#define VCL_MET_MAC(l,U,t,b)	{ "vcl_"#l, b, VCL_MET_##U },
 #include "tbl/vcl_returns.h"
 #undef VCL_MET_MAC
 	{ NULL, 0U, 0}
@@ -483,7 +483,7 @@ EmitStruct(const struct vcc *tl)
 	Fc(tl, 0, "\t.nsrc = VGC_NSRCS,\n");
 	Fc(tl, 0, "\t.srcname = srcname,\n");
 	Fc(tl, 0, "\t.srcbody = srcbody,\n");
-#define VCL_MET_MAC(l,u,b) \
+#define VCL_MET_MAC(l,u,t,b) \
 	Fc(tl, 0, "\t." #l "_func = VGC_function_vcl_" #l ",\n");
 #include "tbl/vcl_returns.h"
 #undef VCL_MET_MAC
diff --git a/lib/libvcc/vcc_storage.c b/lib/libvcc/vcc_storage.c
index 3882e5a..2c3f81c 100644
--- a/lib/libvcc/vcc_storage.c
+++ b/lib/libvcc/vcc_storage.c
@@ -77,7 +77,7 @@ vcc_Stv_mkvar(struct vcc *tl, const struct token *t, enum var_type fmt)
 
 	v->name = TlDupTok(tl, t);
 	v->r_methods = 0;
-#define VCL_MET_MAC(l,u,b)	v->r_methods |= VCL_MET_##u;
+#define VCL_MET_MAC(l,u,t,b)	v->r_methods |= VCL_MET_##u;
 #include "tbl/vcl_returns.h"
 #undef VCL_MET_MAC
 	v->fmt = fmt;



From lkarsten at varnish-software.com  Fri Jan 22 15:46:59 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Fri, 22 Jan 2016 16:46:59 +0100
Subject: [4.1] c371fe8 Minor flexelinting
Message-ID: <E1aMdvP-0001kA-23@project.varnish-software.com>


commit c371fe88eb4c999daf81963132d38078cb0b09e9
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Jan 18 23:26:19 2016 +0000

    Minor flexelinting

diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index 824a8c5..e167db3 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -401,7 +401,7 @@ VRT_ref_vcl(VRT_CTX, const char *desc)
 	ALLOC_OBJ(ref, VCLREF_MAGIC);
 	AN(ref);
 	ref->vcl = vcl;
-	snprintf(ref->desc, sizeof ref->desc, "%s", desc);
+	bprintf(ref->desc, "%s", desc);
 
 	Lck_Lock(&vcl_mtx);
 	VTAILQ_INSERT_TAIL(&vcl->ref_list, ref, list);
diff --git a/lib/libvmod_debug/vmod_debug.c b/lib/libvmod_debug/vmod_debug.c
index 76287df..b87e673 100644
--- a/lib/libvmod_debug/vmod_debug.c
+++ b/lib/libvmod_debug/vmod_debug.c
@@ -277,7 +277,7 @@ event_load(VRT_CTX, struct vmod_priv *priv)
 }
 
 static int
-event_warm(VRT_CTX, struct vmod_priv *priv)
+event_warm(VRT_CTX, const struct vmod_priv *priv)
 {
 	struct priv_vcl *priv_vcl;
 	char buf[32];
@@ -294,7 +294,7 @@ event_warm(VRT_CTX, struct vmod_priv *priv)
 	AZ(priv_vcl->vcl);
 	AZ(priv_vcl->vclref);
 
-	snprintf(buf, sizeof buf, "vmod-debug ref on %s", VCL_Name(ctx->vcl));
+	bprintf(buf, "vmod-debug ref on %s", VCL_Name(ctx->vcl));
 	priv_vcl->vcl = ctx->vcl;
 	priv_vcl->vclref = VRT_ref_vcl(ctx, buf);
 	return (0);
@@ -320,7 +320,7 @@ cooldown_thread(void *priv)
 }
 
 static int
-event_cold(VRT_CTX, struct vmod_priv *priv)
+event_cold(VRT_CTX, const struct vmod_priv *priv)
 {
 	pthread_t thread;
 	struct priv_vcl *priv_vcl;



From lkarsten at varnish-software.com  Fri Jan 22 15:46:59 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Fri, 22 Jan 2016 16:46:59 +0100
Subject: [4.1] 152dd84 Handle whitespace after floats in test fields
Message-ID: <E1aMdvP-0001kJ-3e@project.varnish-software.com>


commit 152dd84ad66717e2b248d57840cf6c26269d3648
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Fri Jan 22 12:00:53 2016 +0000

    Handle whitespace after floats in test fields
    
    Broken in 9bb8f96223d248beb5962961b493b16dcc8ecc2a.
    Committed solution proposed by phk@, discussed with phk@ and martin at .
    
    Fixes #1845.

diff --git a/bin/varnishtest/tests/l00001.vtc b/bin/varnishtest/tests/l00001.vtc
index b1ec63c..bcd4391 100644
--- a/bin/varnishtest/tests/l00001.vtc
+++ b/bin/varnishtest/tests/l00001.vtc
@@ -151,6 +151,12 @@ logexpect l1 -d 1 -g vxid -q "RespHeader[2] == 123" {
 	  expect * =	End
 } -run
 
+# Test field on floats
+logexpect l1 -d 1 -g vxid -q "RespHeader[2] == 123." {
+	  expect 0 *	Begin	req
+	  expect * =	End
+} -run
+
 # Test taglists
 logexpect l1 -d 1 -g vxid -q "Debug,Resp* == 200" {
 	  expect 0 *	Begin	req
diff --git a/lib/libvarnishapi/vsl_query.c b/lib/libvarnishapi/vsl_query.c
index 1ad6811..60cc54d 100644
--- a/lib/libvarnishapi/vsl_query.c
+++ b/lib/libvarnishapi/vsl_query.c
@@ -78,7 +78,7 @@ vslq_test_rec(const struct vex *vex, const struct VSLC_ptr *rec)
 	const struct vex_rhs *rhs;
 	long long lhs_int = 0;
 	double lhs_float = 0.;
-	const char *b, *e;
+	const char *b, *e, *q;
 	char *p;
 	int i;
 
@@ -145,9 +145,11 @@ vslq_test_rec(const struct vex *vex, const struct VSLC_ptr *rec)
 			/* Can't parse - no match */
 			return (0);
 		case VEX_FLOAT:
-			lhs_float = VNUM(b);
+			lhs_float = VNUMpfx(b, &q);
 			if (isnan(lhs_float))
 				return (0);
+			if (q != NULL && q > p && !isspace(q[-1]))
+				return (0);
 			break;
 		default:
 			WRONG("Wrong RHS type");



From lkarsten at varnish-software.com  Fri Jan 22 15:46:59 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Fri, 22 Jan 2016 16:46:59 +0100
Subject: [4.1] 2806edf Use the right variable
Message-ID: <E1aMdvP-0001kS-52@project.varnish-software.com>


commit 2806edf391cb4533134c7acb49f4b747e13b9625
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Fri Jan 22 12:35:16 2016 +0000

    Use the right variable
    
    Fixes previous commit. Sigh.

diff --git a/lib/libvarnishapi/vsl_query.c b/lib/libvarnishapi/vsl_query.c
index 60cc54d..1741c13 100644
--- a/lib/libvarnishapi/vsl_query.c
+++ b/lib/libvarnishapi/vsl_query.c
@@ -148,7 +148,7 @@ vslq_test_rec(const struct vex *vex, const struct VSLC_ptr *rec)
 			lhs_float = VNUMpfx(b, &q);
 			if (isnan(lhs_float))
 				return (0);
-			if (q != NULL && q > p && !isspace(q[-1]))
+			if (q != NULL && q > b && !isspace(q[-1]))
 				return (0);
 			break;
 		default:



From lkarsten at varnish-software.com  Fri Jan 22 15:46:59 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Fri, 22 Jan 2016 16:46:59 +0100
Subject: [4.1] 876f541 Add the last set of changes.
Message-ID: <E1aMdvP-0001kb-6v@project.varnish-software.com>


commit 876f54190c89329e209d6e8acb56d455b1fe0a42
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Fri Jan 22 16:46:16 2016 +0100

    Add the last set of changes.

diff --git a/doc/changes.rst b/doc/changes.rst
index cbb5af4..154c84d 100644
--- a/doc/changes.rst
+++ b/doc/changes.rst
@@ -1,17 +1,22 @@
 ======================================
-Varnish Cache 4.1.1-beta2 (unreleased)
+Varnish Cache 4.1.1-beta2 (2016-01-22)
 ======================================
 
+* Improvements to VCL temperature handling added. This opens for reliably
+  deny warming a cooling VCL from a VMOD.
+
 Bugs fixed
 ----------
 
 * 1802_ - Segfault after VCL change
 * 1825_ - Cannot Start Varnish After Just Restarting The Service
 * 1842_ - Handle missing waiting list gracefully.
+* 1845_ - Handle whitespace after floats in test fields
 
 .. _1802: https://www.varnish-cache.org/trac/ticket/1802
 .. _1825: https://www.varnish-cache.org/trac/ticket/1825
 .. _1842: https://www.varnish-cache.org/trac/ticket/1842
+.. _1845: https://www.varnish-cache.org/trac/ticket/1845
 
 
 ======================================



From lkarsten at varnish-software.com  Fri Jan 22 16:12:51 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Fri, 22 Jan 2016 17:12:51 +0100
Subject: [master] 3633640 Import changes from 4.1 branch.
Message-ID: <E1aMeKR-0006It-90@project.varnish-software.com>


commit 3633640dfab24edc8f3e32def538e46cc0dfdb58
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Fri Jan 22 17:12:47 2016 +0100

    Import changes from 4.1 branch.

diff --git a/doc/changes.rst b/doc/changes.rst
index afa024f..154c84d 100644
--- a/doc/changes.rst
+++ b/doc/changes.rst
@@ -1,5 +1,26 @@
 ======================================
-Varnish Cache 4.1.1-beta1 (unreleased)
+Varnish Cache 4.1.1-beta2 (2016-01-22)
+======================================
+
+* Improvements to VCL temperature handling added. This opens for reliably
+  deny warming a cooling VCL from a VMOD.
+
+Bugs fixed
+----------
+
+* 1802_ - Segfault after VCL change
+* 1825_ - Cannot Start Varnish After Just Restarting The Service
+* 1842_ - Handle missing waiting list gracefully.
+* 1845_ - Handle whitespace after floats in test fields
+
+.. _1802: https://www.varnish-cache.org/trac/ticket/1802
+.. _1825: https://www.varnish-cache.org/trac/ticket/1825
+.. _1842: https://www.varnish-cache.org/trac/ticket/1842
+.. _1845: https://www.varnish-cache.org/trac/ticket/1845
+
+
+======================================
+Varnish Cache 4.1.1-beta1 (2016-01-15)
 ======================================
 
 Changes since 4.1.0:



From fgsch at lodoss.net  Sat Jan 23 09:17:51 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Sat, 23 Jan 2016 10:17:51 +0100
Subject: [master] 84a2903 Correct ENUM handling in object constructors
Message-ID: <E1aMuKN-0000nZ-93@project.varnish-software.com>


commit 84a2903805580973c59635bacb99df5da901c02c
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Sat Jan 23 09:09:29 2016 +0000

    Correct ENUM handling in object constructors
    
    Discussed with phk at .  Fix confirmed by geoff at .
    
    Fixes #1844.

diff --git a/lib/libvcc/vcc_action.c b/lib/libvcc/vcc_action.c
index 41eac6b..790efea 100644
--- a/lib/libvcc/vcc_action.c
+++ b/lib/libvcc/vcc_action.c
@@ -203,9 +203,14 @@ parse_new(struct vcc *tl)
 	s_obj = p;
 	p += strlen(p) + 1;
 	s_init = p;
-	while (p[0] != '\0' || p[1] != '\0')
+	/*
+	 * Check for the end marked (\0\0) followed by s(truct) to avoid
+	 * matching an ENUM half-way through and generating illegal C code.
+	 */
+	while (p[0] != '\0' || p[1] != '\0' || p[2] != 's')
 		p++;
 	p += 2;
+	AZ(strncmp(p, "struct vmod_", 12));
 	s_struct = p;
 	p += strlen(p) + 1;
 	s_fini = p + strlen(p) + 1;
diff --git a/lib/libvmod_debug/vmod.vcc b/lib/libvmod_debug/vmod.vcc
index 74c8c12..9218b5e 100644
--- a/lib/libvmod_debug/vmod.vcc
+++ b/lib/libvmod_debug/vmod.vcc
@@ -71,7 +71,7 @@ $Function BACKEND no_backend()
 
 Fails at backend selection
 
-$Object obj(STRING)
+$Object obj(STRING, ENUM { one, two, three } number="one")
 
 Test object
 
diff --git a/lib/libvmod_debug/vmod_debug_obj.c b/lib/libvmod_debug/vmod_debug_obj.c
index 0b9c248..2511f9a 100644
--- a/lib/libvmod_debug/vmod_debug_obj.c
+++ b/lib/libvmod_debug/vmod_debug_obj.c
@@ -43,13 +43,14 @@ struct vmod_debug_obj {
 
 VCL_VOID
 vmod_obj__init(VRT_CTX, struct vmod_debug_obj **op,
-    const char *vcl_name, VCL_STRING s)
+    const char *vcl_name, VCL_STRING s, VCL_ENUM e)
 {
 	struct vmod_debug_obj *o;
 
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
 	(void)vcl_name;
 	(void)s;
+	(void)e;
 	AN(op);
 	AZ(*op);
 	ALLOC_OBJ(o, VMOD_DEBUG_OBJ_MAGIC);



From daghf at varnish-software.com  Mon Jan 25 12:42:50 2016
From: daghf at varnish-software.com (Dag Haavi Finstad)
Date: Mon, 25 Jan 2016 13:42:50 +0100
Subject: [master] e33cb2f Test for range request "bytes=0-"
Message-ID: <E1aNgTq-0003u4-Ir@project.varnish-software.com>


commit e33cb2faceb788e6a9a13e12867fce649423a054
Author: Ond?ej Nov? <ondrej.novy at firma.seznam.cz>
Date:   Tue Jan 12 18:37:36 2016 +0100

    Test for range request "bytes=0-"
    
    This (useless) range request is send by Chrome with HTML5 Video.
    This was not working correctly in 4.0 and was fixed somewhere
    in master.

diff --git a/bin/varnishtest/tests/c00034.vtc b/bin/varnishtest/tests/c00034.vtc
index 2b440c9..1344fe2 100644
--- a/bin/varnishtest/tests/c00034.vtc
+++ b/bin/varnishtest/tests/c00034.vtc
@@ -112,9 +112,15 @@ client c1 {
 	expect resp.status == 206
 	expect resp.bodylen == 100
 	expect resp.http.content-range == "bytes 0-99/100"
+
+	txreq -hdr "Range: bytes=0-"
+	rxresp
+	expect resp.status == 206
+	expect resp.bodylen == 100
+	expect resp.http.content-range == "bytes 0-99/100"
 } -run
 
-varnish v1 -expect s_resp_bodybytes == 401
+varnish v1 -expect s_resp_bodybytes == 501
 
 # Test Range streaming with streaming objects with C-L
 



From lkarsten at varnish-software.com  Mon Jan 25 12:44:27 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Mon, 25 Jan 2016 13:44:27 +0100
Subject: [master] 86e67d8 Update section on pull requests.
Message-ID: <E1aNgVQ-00046X-07@project.varnish-software.com>


commit 86e67d8e161814e425c2bf1c9efa351eef156502
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Mon Jan 25 13:44:18 2016 +0100

    Update section on pull requests.

diff --git a/CONTRIBUTING b/CONTRIBUTING
index 64d63d9..c7ff877 100644
--- a/CONTRIBUTING
+++ b/CONTRIBUTING
@@ -15,6 +15,8 @@ Patches can be sent to varnish-dev at varnish-cache.org.
 Github pull requests
 --------------------
 
-Documentation updates and other non-trivial items for the documentation
-team may be sent as Github pull requests.
+Pull requests are handled like other tickets.
+
+Trivial pull requests (fix typos, etc) are welcomed, but they will be commited
+by a core team member and the author credited in the commit message.
 



From martin at varnish-software.com  Mon Jan 25 14:59:37 2016
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Mon, 25 Jan 2016 15:59:37 +0100
Subject: [4.0] d418ffa Rush the objheader if there is a waiting list when it
 is deref'ed.
Message-ID: <E1aNicD-0004n9-JN@project.varnish-software.com>


commit d418ffab7c86071f330acfd3e45bbb59616036b4
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Jan 6 19:13:47 2016 +0000

    Rush the objheader if there is a waiting list when it is deref'ed.
    
    Fixes:	#1823

diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index b666d07..a948372 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -864,6 +864,11 @@ HSH_DerefObjHead(struct dstat *ds, struct objhead **poh)
 		oh->refcnt--;
 		Lck_Unlock(&oh->mtx);
 		return(1);
+	} else if (oh->waitinglist != NULL) {
+		Lck_Lock(&oh->mtx);
+		if (oh->waitinglist != NULL)
+			hsh_rush(ds, oh);
+		Lck_Unlock(&oh->mtx);
 	}
 
 	assert(oh->refcnt > 0);



From martin at varnish-software.com  Mon Jan 25 14:59:37 2016
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Mon, 25 Jan 2016 15:59:37 +0100
Subject: [4.0] 9c490b6 Add a final backstop,
 so we absolutely 100% certainly do not try to delete a objhead while
 it still has a waiting list, by forcing the last ref holder to rush the WL.
Message-ID: <E1aNicD-0004nF-L1@project.varnish-software.com>


commit 9c490b6cfa3a5c3912987c8fdc6fea1aaf845ab7
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu Jan 7 00:03:35 2016 +0000

    Add a final backstop, so we absolutely 100% certainly do not
    try to delete a objhead while it still has a waiting list,
    by forcing the last ref holder to rush the WL.
    
    Since the hasher owns the refcounts on objhead, we cannot just
    mingle req and objcore refcounts.
    
    Fortunately we don't need to add another refcounter, because all
    we really care about is the wl being empty when we drop the last
    ref.
    
    The wl/hsh_rush() mechanism will work differently with different
    thread-scheduling schenarios, and I cannot definitively rule out
    that we can drop the last ref on an oh, while there are still req's
    on the waiting list.
    
    Given that, and the existence proof in ticket #1823's race, this
    might have been the indicated memory-trampler.
    
    Conflicts:
    	bin/varnishd/cache/cache_hash.c

diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index a948372..7cf085d 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -864,11 +864,26 @@ HSH_DerefObjHead(struct dstat *ds, struct objhead **poh)
 		oh->refcnt--;
 		Lck_Unlock(&oh->mtx);
 		return(1);
-	} else if (oh->waitinglist != NULL) {
+	}
+
+	/*
+	 * Make absolutely certain that we do not let the final ref
+	 * disappear until the waitinglist is empty.
+	 * This is necessary because the req's on the waiting list do
+	 * not hold any ref on the objhead of their own, and we cannot
+	 * just make the hold the same ref's as objcore, that would
+	 * confuse hashers.
+	 */
+	while (oh->waitinglist != NULL) {
 		Lck_Lock(&oh->mtx);
+		assert(oh->refcnt > 0);
+		r = oh->refcnt;
 		if (oh->waitinglist != NULL)
 			hsh_rush(ds, oh);
 		Lck_Unlock(&oh->mtx);
+		if (r > 1)
+			break;
+		usleep(100000);
 	}
 
 	assert(oh->refcnt > 0);
diff --git a/bin/varnishd/hash/hash_critbit.c b/bin/varnishd/hash/hash_critbit.c
index ff148ba..1496797 100644
--- a/bin/varnishd/hash/hash_critbit.c
+++ b/bin/varnishd/hash/hash_critbit.c
@@ -394,9 +394,7 @@ hcb_start(void)
 static int __match_proto__(hash_deref_f)
 hcb_deref(struct objhead *oh)
 {
-	int r;
 
-	r = 1;
 	CHECK_OBJ_NOTNULL(oh, OBJHEAD_MAGIC);
 	Lck_Lock(&oh->mtx);
 	assert(oh->refcnt > 0);
@@ -413,7 +411,7 @@ hcb_deref(struct objhead *oh)
 #ifdef PHK
 	fprintf(stderr, "hcb_defef %d %d <%s>\n", __LINE__, r, oh->hash);
 #endif
-	return (r);
+	return (1);
 }
 
 static struct objhead * __match_proto__(hash_lookup_f)



From tfheen at err.no  Tue Jan 26 08:57:55 2016
From: tfheen at err.no (Tollef Fog Heen)
Date: Tue, 26 Jan 2016 09:57:55 +0100
Subject: [master] 8656fec WS: Fix off-by-one in WS_Reset
Message-ID: <E1aNzRj-0007zf-Ia@project.varnish-software.com>


commit 8656fec9848ce71e995f0abadb822c91d58a1c07
Author: Devon H. O'Dell <dho at fastly.com>
Date:   Thu Jan 14 15:20:22 2016 -0800

    WS: Fix off-by-one in WS_Reset
    
    WS_Assert allows the workspace to be full in its check that `ws->f <= ws->e`.
    WS_Reset does not honor this; if a snapshot was taken of a full workspace, and
    that snapshot is reverted with WS_Reset, we panic.
    
    This issue was identified by Jozef Hatala <jozef at fastly.com>

diff --git a/bin/varnishd/cache/cache_ws.c b/bin/varnishd/cache/cache_ws.c
index a1b81eb..9293d9e 100644
--- a/bin/varnishd/cache/cache_ws.c
+++ b/bin/varnishd/cache/cache_ws.c
@@ -116,7 +116,7 @@ WS_Reset(struct ws *ws, char *p)
 		ws->f = ws->s;
 	else {
 		assert(p >= ws->s);
-		assert(p < ws->e);
+		assert(p <= ws->e);
 		ws->f = p;
 	}
 	ws_ClearOverflow(ws);
diff --git a/bin/varnishtest/tests/c00074.vtc b/bin/varnishtest/tests/c00074.vtc
new file mode 100644
index 0000000..f20f71c
--- /dev/null
+++ b/bin/varnishtest/tests/c00074.vtc
@@ -0,0 +1,25 @@
+varnishtest "Test WS_Reset off-by-one when workspace is full"
+
+server s1 {
+	rxreq
+	txresp
+} -start
+
+varnish v1 -vcl+backend {
+	import ${vmod_debug};
+	import ${vmod_std};
+
+	sub vcl_recv {
+		set req.http.ws-free = debug.workspace_free(session);
+		debug.workspace_allocate(session, std.integer(req.http.ws-free, 0));
+		debug.workspace_snap(session);
+		debug.workspace_reset(session);
+	}
+} -start
+
+client c1 {
+	txreq -url /foo
+	rxresp
+	expect resp.status == 200
+} -run
+
diff --git a/lib/libvmod_debug/vmod.vcc b/lib/libvmod_debug/vmod.vcc
index 9218b5e..8bc0dd2 100644
--- a/lib/libvmod_debug/vmod.vcc
+++ b/lib/libvmod_debug/vmod.vcc
@@ -139,6 +139,14 @@ $Function INT workspace_free(ENUM { client, backend, session, thread })
 
 Find how much unallocated space there is left in a workspace.
 
+$Function VOID workspace_snap(ENUM { client, backend, session, thread})
+
+Snapshot the named workspace. Only one snapshot may be active at a time.
+
+$Function VOID workspace_reset(ENUM { client, backend, session, thread })
+
+Reset to the previous snapshot of a workspace, taken from debug.workspace_snap.
+
 $Function VOID vcl_release_delay(DURATION)
 
 Hold a reference to the VCL when it goes cold for the given delay.
diff --git a/lib/libvmod_debug/vmod_debug.c b/lib/libvmod_debug/vmod_debug.c
index b87e673..839d0ad 100644
--- a/lib/libvmod_debug/vmod_debug.c
+++ b/lib/libvmod_debug/vmod_debug.c
@@ -419,6 +419,31 @@ vmod_workspace_overflowed(VRT_CTX, VCL_ENUM which)
 	return (WS_Overflowed(ws));
 }
 
+static char *debug_ws_snap;
+void
+vmod_workspace_snap(VRT_CTX, VCL_ENUM which)
+{
+	struct ws *ws;
+	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
+
+	ws = wsfind(ctx, which);
+	WS_Assert(ws);
+
+	debug_ws_snap = WS_Snapshot(ws);
+}
+
+void
+vmod_workspace_reset(VRT_CTX, VCL_ENUM which)
+{
+	struct ws *ws;
+	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
+
+	ws = wsfind(ctx, which);
+	WS_Assert(ws);
+
+	WS_Reset(ws, debug_ws_snap);
+}
+
 void
 vmod_workspace_overflow(VRT_CTX, VCL_ENUM which)
 {



From lkarsten at varnish-software.com  Thu Jan 28 10:28:23 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 28 Jan 2016 11:28:23 +0100
Subject: [4.1] 66bd789 Correct URLs used in packaging links.
Message-ID: <E1aOjoN-0004Qf-0C@project.varnish-software.com>


commit 66bd78951b24605e6305796852570216a82d8be9
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Fri Jan 15 14:07:41 2016 +0100

    Correct URLs used in packaging links.

diff --git a/README.Packaging b/README.Packaging
index dbe4680..6663f88 100644
--- a/README.Packaging
+++ b/README.Packaging
@@ -13,19 +13,11 @@ Official packages
 -----------------
 
 The official Debian and Redhat packages are built by the Varnish Cache team
-and made available on http://repo.varnish-cache.org/ .
+and made available on https://repo.varnish-cache.org/ .
 
-Packaging files for Debian:
+Packaging files and scripts for Debian and Redhat:
 
-    git://git.varnish-cache.org/varnish-cache-debian.git
-
-Packaging files for Redhat:
-
-    https://github.com/varnish/varnish-cache-redhat
-
-Scripts that run the builds are available on:
-
-    https://github.com/varnish/pkg-varnish-cache
+    https://github.com/varnishcache/pkg-varnish-cache
 
 
 Third-party packages



From lkarsten at varnish-software.com  Thu Jan 28 10:28:23 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 28 Jan 2016 11:28:23 +0100
Subject: [4.1] bf07b9b Update section on pull requests.
Message-ID: <E1aOjoN-0004Ql-1w@project.varnish-software.com>


commit bf07b9b115836e17f58105c3be2c9ab0d9f98a8e
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Mon Jan 25 13:44:18 2016 +0100

    Update section on pull requests.

diff --git a/CONTRIBUTING b/CONTRIBUTING
index d7c0277..5111b60 100644
--- a/CONTRIBUTING
+++ b/CONTRIBUTING
@@ -17,7 +17,9 @@ Patches can be sent to varnish-dev at varnish-cache.org.
 Github pull requests
 --------------------
 
-Documentation updates and other non-trivial items for the documentation
-team may be sent as Github pull requests.
+Pull requests are handled like other tickets.
+
+Trivial pull requests (fix typos, etc) are welcomed, but they will be commited
+by a core team member and the author credited in the commit message.
 
 Pull requests outside of this scope will be closed without review.



From lkarsten at varnish-software.com  Thu Jan 28 10:28:23 2016
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 28 Jan 2016 11:28:23 +0100
Subject: [4.1] 66bb824 Prepare for 4.1.1 (final).
Message-ID: <E1aOjoN-0004Qu-3X@project.varnish-software.com>


commit 66bb824548c32e588affe203ea8033430fea2da1
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu Jan 28 11:27:47 2016 +0100

    Prepare for 4.1.1 (final).

diff --git a/configure.ac b/configure.ac
index 0ccc0d7..7b71eaa 100644
--- a/configure.ac
+++ b/configure.ac
@@ -2,7 +2,7 @@ AC_PREREQ(2.59)
 AC_COPYRIGHT([Copyright (c) 2006 Verdens Gang AS
 Copyright (c) 2006-2016 Varnish Software AS])
 AC_REVISION([$Id$])
-AC_INIT([Varnish], [4.1.1-beta2], [varnish-dev at varnish-cache.org])
+AC_INIT([Varnish], [4.1.1], [varnish-dev at varnish-cache.org])
 AC_CONFIG_SRCDIR(include/miniobj.h)
 AC_CONFIG_HEADERS([config.h])
 AC_CONFIG_MACRO_DIR([m4])
diff --git a/doc/changes.rst b/doc/changes.rst
index 154c84d..f06c776 100644
--- a/doc/changes.rst
+++ b/doc/changes.rst
@@ -1,3 +1,10 @@
+================================
+Varnish Cache 4.1.1 (2016-01-28)
+================================
+
+* No code changes since 4.1.1-beta2.
+
+
 ======================================
 Varnish Cache 4.1.1-beta2 (2016-01-22)
 ======================================



From phk at FreeBSD.org  Thu Jan 28 11:49:18 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 28 Jan 2016 12:49:18 +0100
Subject: [master] b98d1b0 Pass the objcore to VBO_waitlen() instead of
 deriving it from the busyobj
Message-ID: <E1aOl4g-0006Wb-O6@project.varnish-software.com>


commit b98d1b05747efb64954cf0ae79708cb8326d43ae
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Jan 19 09:49:11 2016 +0000

    Pass the objcore to VBO_waitlen() instead of deriving it from the busyobj

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index cdcfc3a..9fdb6f1 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -671,7 +671,8 @@ double BAN_Time(const struct ban *ban);
 struct busyobj *VBO_GetBusyObj(struct worker *, const struct req *);
 void VBO_DerefBusyObj(struct worker *wrk, struct busyobj **busyobj);
 void VBO_extend(struct busyobj *, ssize_t);
-ssize_t VBO_waitlen(struct worker *, struct busyobj *, ssize_t l);
+ssize_t VBO_waitlen(struct worker *, struct objcore *, struct busyobj *,
+    ssize_t l);
 void VBO_setstate(struct busyobj *bo, enum busyobj_state_e next);
 void VBO_waitstate(struct busyobj *bo, enum busyobj_state_e want);
 
diff --git a/bin/varnishd/cache/cache_busyobj.c b/bin/varnishd/cache/cache_busyobj.c
index f32831f..f9c239a 100644
--- a/bin/varnishd/cache/cache_busyobj.c
+++ b/bin/varnishd/cache/cache_busyobj.c
@@ -236,20 +236,22 @@ VBO_extend(struct busyobj *bo, ssize_t l)
 }
 
 ssize_t
-VBO_waitlen(struct worker *wrk, struct busyobj *bo, ssize_t l)
+VBO_waitlen(struct worker *wrk, struct objcore *oc, struct busyobj *bo,
+    ssize_t l)
 {
 	ssize_t rv;
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
+	xxxassert(bo->fetch_objcore == oc);
 	Lck_Lock(&bo->mtx);
-	rv = ObjGetLen(wrk, bo->fetch_objcore);
+	rv = ObjGetLen(wrk, oc);
 	while (1) {
 		assert(l <= rv || bo->state == BOS_FAILED);
 		if (rv > l || bo->state >= BOS_FINISHED)
 			break;
 		(void)Lck_CondWait(&bo->cond, &bo->mtx, 0);
-		rv = ObjGetLen(wrk, bo->fetch_objcore);
+		rv = ObjGetLen(wrk, oc);
 	}
 	Lck_Unlock(&bo->mtx);
 	return (rv);
diff --git a/bin/varnishd/storage/storage_simple.c b/bin/varnishd/storage/storage_simple.c
index afbf6cc..3c253a5 100644
--- a/bin/varnishd/storage/storage_simple.c
+++ b/bin/varnishd/storage/storage_simple.c
@@ -257,7 +257,7 @@ sml_iterator(struct worker *wrk, struct objcore *oc,
 
 	while (1) {
 		ol = len;
-		nl = VBO_waitlen(wrk, bo, ol);
+		nl = VBO_waitlen(wrk, oc, bo, ol);
 		if (bo->state == BOS_FAILED) {
 			ret = -1;
 			break;



From phk at FreeBSD.org  Thu Jan 28 11:49:18 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 28 Jan 2016 12:49:18 +0100
Subject: [master] 393762b Add VFP_Extend() to pair with VFP_GetStorage() for
 general sanity
Message-ID: <E1aOl4g-0006Wk-Pg@project.varnish-software.com>


commit 393762beeb13ab2f66c61193b2d2025fc89cf701
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Jan 19 10:05:19 2016 +0000

    Add VFP_Extend() to pair with VFP_GetStorage() for general sanity

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index 9fdb6f1..ec9ced0 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -670,7 +670,7 @@ double BAN_Time(const struct ban *ban);
 /* cache_busyobj.c */
 struct busyobj *VBO_GetBusyObj(struct worker *, const struct req *);
 void VBO_DerefBusyObj(struct worker *wrk, struct busyobj **busyobj);
-void VBO_extend(struct busyobj *, ssize_t);
+void VBO_extend(struct objcore *, struct busyobj *, ssize_t);
 ssize_t VBO_waitlen(struct worker *, struct objcore *, struct busyobj *,
     ssize_t l);
 void VBO_setstate(struct busyobj *bo, enum busyobj_state_e next);
diff --git a/bin/varnishd/cache/cache_busyobj.c b/bin/varnishd/cache/cache_busyobj.c
index f9c239a..fdfa0e7 100644
--- a/bin/varnishd/cache/cache_busyobj.c
+++ b/bin/varnishd/cache/cache_busyobj.c
@@ -221,16 +221,16 @@ VBO_DerefBusyObj(struct worker *wrk, struct busyobj **pbo)
 }
 
 void
-VBO_extend(struct busyobj *bo, ssize_t l)
+VBO_extend(struct objcore *oc, struct busyobj *bo, ssize_t l)
 {
 
+	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
 	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
-	CHECK_OBJ_NOTNULL(bo->vfc, VFP_CTX_MAGIC);
 	if (l == 0)
 		return;
 	assert(l > 0);
 	Lck_Lock(&bo->mtx);
-	ObjExtend(bo->wrk, bo->vfc->oc, l);
+	ObjExtend(bo->wrk, oc, l);
 	AZ(pthread_cond_broadcast(&bo->cond));
 	Lck_Unlock(&bo->mtx);
 }
diff --git a/bin/varnishd/cache/cache_esi_fetch.c b/bin/varnishd/cache/cache_esi_fetch.c
index 0d83b9f..bce3e45 100644
--- a/bin/varnishd/cache/cache_esi_fetch.c
+++ b/bin/varnishd/cache/cache_esi_fetch.c
@@ -96,7 +96,7 @@ vfp_vep_callback(struct vfp_ctx *vc, void *priv, ssize_t l, enum vgz_flag flg)
 		i = VGZ_Gzip(vef->vgz, &dp, &dl, flg);
 		VGZ_UpdateObj(vc, vef->vgz, VUA_UPDATE);
 		vef->tot += dl;
-		VBO_extend(vc->bo, dl);
+		VFP_Extend(vc, dl);
 	} while (i != VGZ_ERROR &&
 	    (!VGZ_IbufEmpty(vef->vgz) || VGZ_ObufFull(vef->vgz)));
 	assert(i == VGZ_ERROR || VGZ_IbufEmpty(vef->vgz));
diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index 503dc3c..f781111 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -511,7 +511,7 @@ vbf_fetch_body_helper(struct busyobj *bo)
 		vfps = VFP_Suck(vfc, ptr, &l);
 		if (l > 0 && vfps != VFP_ERROR) {
 			bo->acct.beresp_bodybytes += l;
-			VBO_extend(bo, l);
+			VFP_Extend(vfc, l);
 			if (est >= l)
 				est -= l;
 			else
@@ -711,7 +711,7 @@ vbf_objiterator(void *priv, int flush, const void *ptr, ssize_t len)
 		if (len < l)
 			l = len;
 		memcpy(pd, ps, l);
-		VBO_extend(bo, l);
+		VFP_Extend(bo->vfc, l);
 		ps += l;
 		len -= l;
 	}
@@ -851,7 +851,7 @@ vbf_stp_error(struct worker *wrk, struct busyobj *bo)
 		if (VFP_GetStorage(bo->vfc, &l, &ptr) != VFP_OK)
 			break;
 		memcpy(ptr, VSB_data(synth_body) + o, l);
-		VBO_extend(bo, l);
+		VFP_Extend(bo->vfc, l);
 		ll -= l;
 		o += l;
 	}
diff --git a/bin/varnishd/cache/cache_fetch_proc.c b/bin/varnishd/cache/cache_fetch_proc.c
index 5333c62..d297104 100644
--- a/bin/varnishd/cache/cache_fetch_proc.c
+++ b/bin/varnishd/cache/cache_fetch_proc.c
@@ -72,7 +72,6 @@ VFP_GetStorage(struct vfp_ctx *vc, ssize_t *sz, uint8_t **ptr)
 {
 	ssize_t l;
 
-
 	CHECK_OBJ_NOTNULL(vc, VFP_CTX_MAGIC);
 	AN(sz);
 	assert(*sz >= 0);
@@ -94,6 +93,14 @@ VFP_GetStorage(struct vfp_ctx *vc, ssize_t *sz, uint8_t **ptr)
 	return (VFP_OK);
 }
 
+void
+VFP_Extend(const struct vfp_ctx *vc, ssize_t sz)
+{
+	CHECK_OBJ_NOTNULL(vc, VFP_CTX_MAGIC);
+
+	VBO_extend(vc->oc, vc->bo, sz);
+}
+
 /**********************************************************************
  */
 
diff --git a/bin/varnishd/cache/cache_filter.h b/bin/varnishd/cache/cache_filter.h
index aba9d65..3ffed7e 100644
--- a/bin/varnishd/cache/cache_filter.h
+++ b/bin/varnishd/cache/cache_filter.h
@@ -84,6 +84,7 @@ enum vfp_status VFP_Error(struct vfp_ctx *, const char *fmt, ...)
 
 /* cache_fetch_proc.c */
 enum vfp_status VFP_GetStorage(struct vfp_ctx *, ssize_t *sz, uint8_t **ptr);
+void VFP_Extend(const struct vfp_ctx *, ssize_t sz);
 
 /* Deliver processors ------------------------------------------------*/
 



From phk at FreeBSD.org  Thu Jan 28 11:49:18 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 28 Jan 2016 12:49:18 +0100
Subject: [master] 559e1cf Pass worker explicitly to VBO_extend()
Message-ID: <E1aOl4g-0006Ww-SO@project.varnish-software.com>


commit 559e1cf57403d879abd009a0f0c07ada32a35599
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Jan 19 10:14:09 2016 +0000

    Pass worker explicitly to VBO_extend()

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index ec9ced0..4bdea99 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -670,7 +670,7 @@ double BAN_Time(const struct ban *ban);
 /* cache_busyobj.c */
 struct busyobj *VBO_GetBusyObj(struct worker *, const struct req *);
 void VBO_DerefBusyObj(struct worker *wrk, struct busyobj **busyobj);
-void VBO_extend(struct objcore *, struct busyobj *, ssize_t);
+void VBO_extend(struct worker *, struct objcore *, struct busyobj *, ssize_t);
 ssize_t VBO_waitlen(struct worker *, struct objcore *, struct busyobj *,
     ssize_t l);
 void VBO_setstate(struct busyobj *bo, enum busyobj_state_e next);
diff --git a/bin/varnishd/cache/cache_busyobj.c b/bin/varnishd/cache/cache_busyobj.c
index fdfa0e7..a0a5abb 100644
--- a/bin/varnishd/cache/cache_busyobj.c
+++ b/bin/varnishd/cache/cache_busyobj.c
@@ -221,16 +221,18 @@ VBO_DerefBusyObj(struct worker *wrk, struct busyobj **pbo)
 }
 
 void
-VBO_extend(struct objcore *oc, struct busyobj *bo, ssize_t l)
+VBO_extend(struct worker *wrk, struct objcore *oc, struct busyobj *bo,
+    ssize_t l)
 {
 
+	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
 	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
 	if (l == 0)
 		return;
 	assert(l > 0);
 	Lck_Lock(&bo->mtx);
-	ObjExtend(bo->wrk, oc, l);
+	ObjExtend(wrk, oc, l);
 	AZ(pthread_cond_broadcast(&bo->cond));
 	Lck_Unlock(&bo->mtx);
 }
diff --git a/bin/varnishd/cache/cache_fetch_proc.c b/bin/varnishd/cache/cache_fetch_proc.c
index d297104..87f943e 100644
--- a/bin/varnishd/cache/cache_fetch_proc.c
+++ b/bin/varnishd/cache/cache_fetch_proc.c
@@ -98,7 +98,7 @@ VFP_Extend(const struct vfp_ctx *vc, ssize_t sz)
 {
 	CHECK_OBJ_NOTNULL(vc, VFP_CTX_MAGIC);
 
-	VBO_extend(vc->oc, vc->bo, sz);
+	VBO_extend(vc->wrk, vc->oc, vc->bo, sz);
 }
 
 /**********************************************************************



From phk at FreeBSD.org  Thu Jan 28 11:49:18 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 28 Jan 2016 12:49:18 +0100
Subject: [master] 9f8dba3 Start the divorce that separates struct busyobj into
 a "busy objcore" (struct boc) and "the backend fetch context" (struct bereq).
Message-ID: <E1aOl4g-0006X7-VE@project.varnish-software.com>


commit 9f8dba345dc67c054172edb9f4c0a25942060987
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Jan 19 10:26:15 2016 +0000

    Start the divorce that separates struct busyobj into a "busy objcore"
    (struct boc) and "the backend fetch context" (struct bereq).

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index 4bdea99..4a3ce05 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -374,6 +374,33 @@ struct storeobj {
 	uintptr_t		priv2;
 };
 
+/* Busy Objcore structure --------------------------------------------
+ *
+ */
+
+/*
+ * The macro-states we expose outside the fetch code
+ */
+enum busyobj_state_e {
+	BOS_INVALID = 0,	/* don't touch (yet) */
+	BOS_REQ_DONE,		/* beresp.* can be examined */
+	BOS_STREAM,		/* beresp.* can be examined */
+	BOS_FINISHED,		/* object is complete */
+	BOS_FAILED,		/* something went wrong */
+};
+
+struct boc {
+	unsigned		magic;
+#define BOC_MAGIC		0x70c98476
+	unsigned		refcount;
+	struct lock		mtx;
+	pthread_cond_t		cond;
+	void			*stevedore_priv;
+	enum busyobj_state_e	state;
+	uint8_t			*vary;
+
+};
+
 /* Object core structure ---------------------------------------------
  * Objects have sideways references in the binary heap and the LRU list
  * and we want to avoid paging in a lot of objects just to move them up
@@ -425,17 +452,6 @@ struct objcore {
  * streaming delivery will make use of.
  */
 
-/*
- * The macro-states we expose outside the fetch code
- */
-enum busyobj_state_e {
-	BOS_INVALID = 0,	/* don't touch (yet) */
-	BOS_REQ_DONE,		/* beresp.* can be examined */
-	BOS_STREAM,		/* beresp.* can be examined */
-	BOS_FINISHED,		/* object is complete */
-	BOS_FAILED,		/* something went wrong */
-};
-
 enum director_state_e {
 	DIR_S_NULL = 0,
 	DIR_S_HDRS = 1,
@@ -445,25 +461,21 @@ enum director_state_e {
 struct busyobj {
 	unsigned		magic;
 #define BUSYOBJ_MAGIC		0x23b95567
-	struct lock		mtx;
-	pthread_cond_t		cond;
+
+	struct boc		boc[1];
+
 	char			*end;
 
 	/*
 	 * All fields from refcount and down are zeroed when the busyobj
 	 * is recycled.
 	 */
-	unsigned		refcount;
 	int			retries;
 	struct req		*req;
 	struct worker		*wrk;
 
-	uint8_t			*vary;
-
 	struct vfp_ctx		vfc[1];
 
-	enum busyobj_state_e	state;
-
 	struct ws		ws[1];
 	char			*ws_bo;
 	struct http		*bereq0;
@@ -500,8 +512,6 @@ struct busyobj {
 
 	struct vsl_log		vsl[1];
 
-	void			*stevedore_priv;
-
 	uint8_t			digest[DIGEST_LEN];
 	struct vrt_privs	privs[1];
 };
diff --git a/bin/varnishd/cache/cache_busyobj.c b/bin/varnishd/cache/cache_busyobj.c
index a0a5abb..3f63eb5 100644
--- a/bin/varnishd/cache/cache_busyobj.c
+++ b/bin/varnishd/cache/cache_busyobj.c
@@ -68,8 +68,9 @@ vbo_New(void)
 	XXXAN(bo);
 	bo->magic = BUSYOBJ_MAGIC;
 	bo->end = (char *)bo + sz;
-	Lck_New(&bo->mtx, lck_busyobj);
-	AZ(pthread_cond_init(&bo->cond, NULL));
+	INIT_OBJ(bo->boc, BOC_MAGIC);
+	Lck_New(&bo->boc->mtx, lck_busyobj);
+	AZ(pthread_cond_init(&bo->boc->cond, NULL));
 	return (bo);
 }
 
@@ -83,9 +84,9 @@ vbo_Free(struct busyobj **bop)
 	*bop = NULL;
 	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
 	AZ(bo->htc);
-	AZ(bo->refcount);
-	AZ(pthread_cond_destroy(&bo->cond));
-	Lck_Delete(&bo->mtx);
+	AZ(bo->boc->refcount);
+	AZ(pthread_cond_destroy(&bo->boc->cond));
+	Lck_Delete(&bo->boc->mtx);
 	MPL_Free(vbopool, bo);
 }
 
@@ -101,9 +102,9 @@ VBO_GetBusyObj(struct worker *wrk, const struct req *req)
 
 	bo = vbo_New();
 	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
-	AZ(bo->refcount);
+	AZ(bo->boc->refcount);
 
-	bo->refcount = 1;
+	bo->boc->refcount = 1;
 
 	p = (void*)(bo + 1);
 	p = (void*)PRNDUP(p);
@@ -171,14 +172,14 @@ VBO_DerefBusyObj(struct worker *wrk, struct busyobj **pbo)
 		CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
 		CHECK_OBJ_NOTNULL(oc->objhead, OBJHEAD_MAGIC);
 		Lck_Lock(&oc->objhead->mtx);
-		assert(bo->refcount > 0);
-		r = --bo->refcount;
+		assert(bo->boc->refcount > 0);
+		r = --bo->boc->refcount;
 		Lck_Unlock(&oc->objhead->mtx);
 	} else {
-		Lck_Lock(&bo->mtx);
-		assert(bo->refcount > 0);
-		r = --bo->refcount;
-		Lck_Unlock(&bo->mtx);
+		Lck_Lock(&bo->boc->mtx);
+		assert(bo->boc->refcount > 0);
+		r = --bo->boc->refcount;
+		Lck_Unlock(&bo->boc->mtx);
 	}
 
 	if (r)
@@ -211,11 +212,16 @@ VBO_DerefBusyObj(struct worker *wrk, struct busyobj **pbo)
 
 	VCL_Rel(&bo->vcl);
 
-	if (bo->vary != NULL)
-		free(bo->vary);
+	AZ(bo->boc->stevedore_priv);
+	AZ(bo->boc->refcount);
+	bo->boc->state = BOS_INVALID;
+	if (bo->boc->vary != NULL) {
+		free(bo->boc->vary);
+		bo->boc->vary = NULL;
+	}
 
-	memset(&bo->refcount, 0,
-	    sizeof *bo - offsetof(struct busyobj, refcount));
+	memset(&bo->retries, 0,
+	    sizeof *bo - offsetof(struct busyobj, retries));
 
 	vbo_Free(&bo);
 }
@@ -231,10 +237,10 @@ VBO_extend(struct worker *wrk, struct objcore *oc, struct busyobj *bo,
 	if (l == 0)
 		return;
 	assert(l > 0);
-	Lck_Lock(&bo->mtx);
+	Lck_Lock(&bo->boc->mtx);
 	ObjExtend(wrk, oc, l);
-	AZ(pthread_cond_broadcast(&bo->cond));
-	Lck_Unlock(&bo->mtx);
+	AZ(pthread_cond_broadcast(&bo->boc->cond));
+	Lck_Unlock(&bo->boc->mtx);
 }
 
 ssize_t
@@ -246,16 +252,16 @@ VBO_waitlen(struct worker *wrk, struct objcore *oc, struct busyobj *bo,
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
 	xxxassert(bo->fetch_objcore == oc);
-	Lck_Lock(&bo->mtx);
+	Lck_Lock(&bo->boc->mtx);
 	rv = ObjGetLen(wrk, oc);
 	while (1) {
-		assert(l <= rv || bo->state == BOS_FAILED);
-		if (rv > l || bo->state >= BOS_FINISHED)
+		assert(l <= rv || bo->boc->state == BOS_FAILED);
+		if (rv > l || bo->boc->state >= BOS_FINISHED)
 			break;
-		(void)Lck_CondWait(&bo->cond, &bo->mtx, 0);
+		(void)Lck_CondWait(&bo->boc->cond, &bo->boc->mtx, 0);
 		rv = ObjGetLen(wrk, oc);
 	}
-	Lck_Unlock(&bo->mtx);
+	Lck_Unlock(&bo->boc->mtx);
 	return (rv);
 }
 
@@ -264,21 +270,21 @@ VBO_setstate(struct busyobj *bo, enum busyobj_state_e next)
 {
 	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
 	assert(bo->do_stream || next != BOS_STREAM);
-	assert(next > bo->state);
-	Lck_Lock(&bo->mtx);
-	bo->state = next;
-	AZ(pthread_cond_broadcast(&bo->cond));
-	Lck_Unlock(&bo->mtx);
+	assert(next > bo->boc->state);
+	Lck_Lock(&bo->boc->mtx);
+	bo->boc->state = next;
+	AZ(pthread_cond_broadcast(&bo->boc->cond));
+	Lck_Unlock(&bo->boc->mtx);
 }
 
 void
 VBO_waitstate(struct busyobj *bo, enum busyobj_state_e want)
 {
-	Lck_Lock(&bo->mtx);
+	Lck_Lock(&bo->boc->mtx);
 	while (1) {
-		if (bo->state >= want)
+		if (bo->boc->state >= want)
 			break;
-		(void)Lck_CondWait(&bo->cond, &bo->mtx, 0);
+		(void)Lck_CondWait(&bo->boc->cond, &bo->boc->mtx, 0);
 	}
-	Lck_Unlock(&bo->mtx);
+	Lck_Unlock(&bo->boc->mtx);
 }
diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index f781111..32b58ac 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -168,7 +168,7 @@ vbf_stp_mkbereq(const struct worker *wrk, struct busyobj *bo)
 	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
 	CHECK_OBJ_NOTNULL(bo->req, REQ_MAGIC);
 
-	assert(bo->state == BOS_INVALID);
+	assert(bo->boc->state == BOS_INVALID);
 	AZ(bo->storage_hint);
 
 	HTTP_Setup(bo->bereq0, bo->ws, bo->vsl, SLT_BereqMethod);
@@ -221,7 +221,7 @@ vbf_stp_retry(struct worker *wrk, struct busyobj *bo)
 	vfc = bo->vfc;
 	CHECK_OBJ_NOTNULL(vfc, VFP_CTX_MAGIC);
 
-	assert(bo->state == BOS_REQ_DONE);
+	assert(bo->boc->state == BOS_REQ_DONE);
 
 	VSLb_ts_busyobj(bo, "Retry", W_TIM_real(wrk));
 
@@ -281,7 +281,7 @@ vbf_stp_startfetch(struct worker *wrk, struct busyobj *bo)
 
 	HTTP_Setup(bo->beresp, bo->ws, bo->vsl, SLT_BerespMethod);
 
-	assert(bo->state <= BOS_REQ_DONE);
+	assert(bo->boc->state <= BOS_REQ_DONE);
 
 	AZ(bo->htc);
 	i = VDI_GetHdr(wrk, bo);
@@ -451,7 +451,7 @@ vbf_stp_startfetch(struct worker *wrk, struct busyobj *bo)
 		return (F_STP_ERROR);
 	}
 
-	assert(bo->state == BOS_REQ_DONE);
+	assert(bo->boc->state == BOS_REQ_DONE);
 
 	if (bo->do_esi)
 		bo->do_stream = 0;
@@ -642,9 +642,9 @@ vbf_stp_fetch(struct worker *wrk, struct busyobj *bo)
 	if (bo->htc->body_status != BS_NONE)
 		AZ(VDI_GetBody(bo->wrk, bo));
 
-	assert(bo->refcount >= 1);
+	assert(bo->boc->refcount >= 1);
 
-	assert(bo->state == BOS_REQ_DONE);
+	assert(bo->boc->state == BOS_REQ_DONE);
 
 	if (bo->do_stream) {
 		HSH_Unbusy(wrk, bo->fetch_objcore);
@@ -663,7 +663,7 @@ vbf_stp_fetch(struct worker *wrk, struct busyobj *bo)
 	if (bo->vfc->failed) {
 		VDI_Finish(bo->wrk, bo);
 		if (!bo->do_stream) {
-			assert(bo->state < BOS_STREAM);
+			assert(bo->boc->state < BOS_STREAM);
 			// XXX: doclose = ?
 			return (F_STP_ERROR);
 		} else {
@@ -672,9 +672,9 @@ vbf_stp_fetch(struct worker *wrk, struct busyobj *bo)
 	}
 
 	if (bo->do_stream)
-		assert(bo->state == BOS_STREAM);
+		assert(bo->boc->state == BOS_STREAM);
 	else {
-		assert(bo->state == BOS_REQ_DONE);
+		assert(bo->boc->state == BOS_REQ_DONE);
 		HSH_Unbusy(wrk, bo->fetch_objcore);
 	}
 
@@ -872,7 +872,7 @@ vbf_stp_fail(struct worker *wrk, struct busyobj *bo)
 	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
 	CHECK_OBJ_NOTNULL(bo->fetch_objcore, OBJCORE_MAGIC);
 
-	assert(bo->state < BOS_FINISHED);
+	assert(bo->boc->state < BOS_FINISHED);
 	HSH_Fail(bo->fetch_objcore);
 	if (bo->fetch_objcore->exp_flags & OC_EF_EXP) {
 		/* Already unbusied - expire it */
@@ -926,7 +926,7 @@ vbf_fetch_thread(struct worker *wrk, void *priv)
 
 	while (stp != F_STP_DONE) {
 		CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
-		assert(bo->refcount >= 1);
+		assert(bo->boc->refcount >= 1);
 		switch(stp) {
 #define FETCH_STEP(l, U, arg)						\
 		case F_STP_##U:						\
@@ -944,7 +944,7 @@ vbf_fetch_thread(struct worker *wrk, void *priv)
 	http_Teardown(bo->bereq);
 	http_Teardown(bo->beresp);
 
-	if (bo->state == BOS_FINISHED) {
+	if (bo->boc->state == BOS_FINISHED) {
 		AZ(bo->fetch_objcore->flags & OC_F_FAILED);
 		HSH_Complete(bo->fetch_objcore);
 		VSLb(bo->vsl, SLT_Length, "%ju",
@@ -992,7 +992,7 @@ VBF_Fetch(struct worker *wrk, struct req *req, struct objcore *oc,
 	THR_SetBusyobj(bo);
 
 	bo_fetch = bo;
-	bo->refcount = 2;
+	bo->boc->refcount = 2;
 
 	oc->busyobj = bo;
 
@@ -1001,7 +1001,7 @@ VBF_Fetch(struct worker *wrk, struct req *req, struct objcore *oc,
 	if (mode == VBF_PASS)
 		bo->do_pass = 1;
 
-	bo->vary = req->vary_b;
+	bo->boc->vary = req->vary_b;
 	req->vary_b = NULL;
 
 	if (mode != VBF_BACKGROUND)
@@ -1034,7 +1034,7 @@ VBF_Fetch(struct worker *wrk, struct req *req, struct objcore *oc,
 			VBO_waitstate(bo, BOS_REQ_DONE);
 		} else {
 			VBO_waitstate(bo, BOS_STREAM);
-			if (bo->state == BOS_FAILED) {
+			if (bo->boc->state == BOS_FAILED) {
 				AN((oc->flags & OC_F_FAILED));
 			} else {
 				AZ(bo->fetch_objcore->flags & OC_F_BUSY);
diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index 12fd0d6..a670143 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -397,8 +397,8 @@ HSH_Lookup(struct req *req, struct objcore **ocp, struct objcore **bocp,
 				continue;
 
 			if (oc->busyobj != NULL &&
-			    oc->busyobj->vary != NULL &&
-			    !VRY_Match(req, oc->busyobj->vary))
+			    oc->busyobj->boc->vary != NULL &&
+			    !VRY_Match(req, oc->busyobj->boc->vary))
 				continue;
 
 			busy_found = 1;
@@ -720,7 +720,7 @@ HSH_RefBusy(const struct objcore *oc)
 	bo = oc->busyobj;
 	CHECK_OBJ_ORNULL(bo, BUSYOBJ_MAGIC);
 	if (bo != NULL)
-		bo->refcount++;
+		bo->boc->refcount++;
 	Lck_Unlock(&oh->mtx);
 	return (bo);
 }
diff --git a/bin/varnishd/cache/cache_panic.c b/bin/varnishd/cache/cache_panic.c
index 0613604..6dbc3c0 100644
--- a/bin/varnishd/cache/cache_panic.c
+++ b/bin/varnishd/cache/cache_panic.c
@@ -272,10 +272,10 @@ pan_busyobj(struct vsb *vsb, const struct busyobj *bo)
 	VSB_printf(vsb, "busyobj = %p {\n", bo);
 	VSB_indent(vsb, 2);
 	pan_ws(vsb, bo->ws);
-	VSB_printf(vsb, "refcnt = %u,\n", bo->refcount);
+	VSB_printf(vsb, "refcnt = %u,\n", bo->boc->refcount);
 	VSB_printf(vsb, "retries = %d, ", bo->retries);
 	VSB_printf(vsb, "failed = %d, ", bo->vfc->failed);
-	VSB_printf(vsb, "state = %d,\n", (int)bo->state);
+	VSB_printf(vsb, "state = %d,\n", (int)bo->boc->state);
 	VSB_printf(vsb, "flags = {");
 	p = "";
 	/*lint -save -esym(438,p) */
diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index fda7912..0c793ef 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -199,7 +199,7 @@ cnt_deliver(struct worker *wrk, struct req *req)
 	/* Grab a ref to the bo if there is one, and hand it down */
 	bo = HSH_RefBusy(req->objcore);
 	if (bo != NULL) {
-		if (req->esi_level == 0 && bo->state == BOS_FINISHED) {
+		if (req->esi_level == 0 && bo->boc->state == BOS_FINISHED) {
 			VBO_DerefBusyObj(wrk, &bo);
 		} else if (!bo->do_stream) {
 			VBO_waitstate(bo, BOS_FINISHED);
diff --git a/bin/varnishd/storage/storage_simple.c b/bin/varnishd/storage/storage_simple.c
index 3c253a5..d2286ba 100644
--- a/bin/varnishd/storage/storage_simple.c
+++ b/bin/varnishd/storage/storage_simple.c
@@ -258,16 +258,16 @@ sml_iterator(struct worker *wrk, struct objcore *oc,
 	while (1) {
 		ol = len;
 		nl = VBO_waitlen(wrk, oc, bo, ol);
-		if (bo->state == BOS_FAILED) {
+		if (bo->boc->state == BOS_FAILED) {
 			ret = -1;
 			break;
 		}
 		if (nl == ol) {
-			if (bo->state == BOS_FINISHED)
+			if (bo->boc->state == BOS_FINISHED)
 				break;
 			continue;
 		}
-		Lck_Lock(&bo->mtx);
+		Lck_Lock(&bo->boc->mtx);
 		AZ(VTAILQ_EMPTY(&obj->list));
 		if (checkpoint == NULL) {
 			st = VTAILQ_FIRST(&obj->list);
@@ -300,8 +300,8 @@ sml_iterator(struct worker *wrk, struct objcore *oc,
 		st = VTAILQ_NEXT(st, list);
 		if (st != NULL && st->len == 0)
 			st = NULL;
-		Lck_Unlock(&bo->mtx);
-		assert(l > 0 || bo->state == BOS_FINISHED);
+		Lck_Unlock(&bo->boc->mtx);
+		assert(l > 0 || bo->boc->state == BOS_FINISHED);
 		if (func(priv, st != NULL ? 0 : 1, p, l)) {
 			ret = -1;
 			break;
@@ -375,9 +375,9 @@ sml_getspace(struct worker *wrk, struct objcore *oc, ssize_t *sz,
 
 	if (oc->busyobj != NULL) {
 		CHECK_OBJ_NOTNULL(oc->busyobj, BUSYOBJ_MAGIC);
-		Lck_Lock(&oc->busyobj->mtx);
+		Lck_Lock(&oc->busyobj->boc->mtx);
 		VTAILQ_INSERT_TAIL(&o->list, st, list);
-		Lck_Unlock(&oc->busyobj->mtx);
+		Lck_Unlock(&oc->busyobj->boc->mtx);
 	} else {
 		AN(oc->flags & (OC_F_PRIVATE));
 		VTAILQ_INSERT_TAIL(&o->list, st, list);
@@ -443,9 +443,9 @@ sml_trimstore(struct worker *wrk, struct objcore *oc)
 
 	if (st->len == 0) {
 		if (oc->busyobj != NULL) {
-			Lck_Lock(&oc->busyobj->mtx);
+			Lck_Lock(&oc->busyobj->boc->mtx);
 			VTAILQ_REMOVE(&o->list, st, list);
-			Lck_Unlock(&oc->busyobj->mtx);
+			Lck_Unlock(&oc->busyobj->boc->mtx);
 		} else {
 			VTAILQ_REMOVE(&o->list, st, list);
 		}
@@ -468,10 +468,10 @@ sml_trimstore(struct worker *wrk, struct objcore *oc)
 	memcpy(st1->ptr, st->ptr, st->len);
 	st1->len = st->len;
 	if (oc->busyobj != NULL) {
-		Lck_Lock(&oc->busyobj->mtx);
+		Lck_Lock(&oc->busyobj->boc->mtx);
 		VTAILQ_REMOVE(&o->list, st, list);
 		VTAILQ_INSERT_TAIL(&o->list, st1, list);
-		Lck_Unlock(&oc->busyobj->mtx);
+		Lck_Unlock(&oc->busyobj->boc->mtx);
 	} else {
 		VTAILQ_REMOVE(&o->list, st, list);
 		VTAILQ_INSERT_TAIL(&o->list, st1, list);
@@ -480,8 +480,8 @@ sml_trimstore(struct worker *wrk, struct objcore *oc)
 		sml_stv_free(stv, st);
 	} else {
 		/* sml_stable frees this */
-		AZ(oc->busyobj->stevedore_priv);
-		oc->busyobj->stevedore_priv = st;
+		AZ(oc->busyobj->boc->stevedore_priv);
+		oc->busyobj->boc->stevedore_priv = st;
 	}
 }
 
@@ -495,10 +495,10 @@ sml_stable(struct worker *wrk, struct objcore *oc, struct busyobj *bo)
 	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
 	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
 
-	if (bo->stevedore_priv == NULL)
+	if (bo->boc->stevedore_priv == NULL)
 		return;
-	CAST_OBJ_NOTNULL(st, bo->stevedore_priv, STORAGE_MAGIC);
-	bo->stevedore_priv = 0;
+	CAST_OBJ_NOTNULL(st, bo->boc->stevedore_priv, STORAGE_MAGIC);
+	bo->boc->stevedore_priv = 0;
 	CHECK_OBJ_NOTNULL(st, STORAGE_MAGIC);
 	stv = oc->stobj->stevedore;
 	CHECK_OBJ_NOTNULL(stv, STEVEDORE_MAGIC);



From phk at FreeBSD.org  Thu Jan 28 11:49:19 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 28 Jan 2016 12:49:19 +0100
Subject: [master] de6e9b6 Reduce strength arguments from busyobj to boc
Message-ID: <E1aOl4h-0006XH-0h@project.varnish-software.com>


commit de6e9b672f722c8a9bedf6641e48f5a32e0d4e41
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Jan 19 10:34:27 2016 +0000

    Reduce strength arguments from busyobj to boc

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index 4a3ce05..951e438 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -680,11 +680,10 @@ double BAN_Time(const struct ban *ban);
 /* cache_busyobj.c */
 struct busyobj *VBO_GetBusyObj(struct worker *, const struct req *);
 void VBO_DerefBusyObj(struct worker *wrk, struct busyobj **busyobj);
-void VBO_extend(struct worker *, struct objcore *, struct busyobj *, ssize_t);
-ssize_t VBO_waitlen(struct worker *, struct objcore *, struct busyobj *,
-    ssize_t l);
-void VBO_setstate(struct busyobj *bo, enum busyobj_state_e next);
-void VBO_waitstate(struct busyobj *bo, enum busyobj_state_e want);
+void VBO_extend(struct worker *, struct objcore *, struct boc *, ssize_t);
+ssize_t VBO_waitlen(struct worker *, struct objcore *, struct boc *, ssize_t l);
+void VBO_setstate(struct boc *, enum busyobj_state_e next);
+void VBO_waitstate(struct boc *, enum busyobj_state_e want);
 
 /* cache_cli.c [CLI] */
 extern pthread_t cli_thread;
diff --git a/bin/varnishd/cache/cache_busyobj.c b/bin/varnishd/cache/cache_busyobj.c
index 3f63eb5..f970442 100644
--- a/bin/varnishd/cache/cache_busyobj.c
+++ b/bin/varnishd/cache/cache_busyobj.c
@@ -227,64 +227,69 @@ VBO_DerefBusyObj(struct worker *wrk, struct busyobj **pbo)
 }
 
 void
-VBO_extend(struct worker *wrk, struct objcore *oc, struct busyobj *bo,
+VBO_extend(struct worker *wrk, struct objcore *oc, struct boc *boc,
     ssize_t l)
 {
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
-	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
+	CHECK_OBJ_NOTNULL(boc, BOC_MAGIC);
 	if (l == 0)
 		return;
 	assert(l > 0);
-	Lck_Lock(&bo->boc->mtx);
+	Lck_Lock(&boc->mtx);
 	ObjExtend(wrk, oc, l);
-	AZ(pthread_cond_broadcast(&bo->boc->cond));
-	Lck_Unlock(&bo->boc->mtx);
+	AZ(pthread_cond_broadcast(&boc->cond));
+	Lck_Unlock(&boc->mtx);
 }
 
 ssize_t
-VBO_waitlen(struct worker *wrk, struct objcore *oc, struct busyobj *bo,
+VBO_waitlen(struct worker *wrk, struct objcore *oc, struct boc *boc,
     ssize_t l)
 {
 	ssize_t rv;
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
-	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
-	xxxassert(bo->fetch_objcore == oc);
-	Lck_Lock(&bo->boc->mtx);
+	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
+	CHECK_OBJ_NOTNULL(boc, BOC_MAGIC);
+	Lck_Lock(&boc->mtx);
 	rv = ObjGetLen(wrk, oc);
 	while (1) {
-		assert(l <= rv || bo->boc->state == BOS_FAILED);
-		if (rv > l || bo->boc->state >= BOS_FINISHED)
+		assert(l <= rv || boc->state == BOS_FAILED);
+		if (rv > l || boc->state >= BOS_FINISHED)
 			break;
-		(void)Lck_CondWait(&bo->boc->cond, &bo->boc->mtx, 0);
+		(void)Lck_CondWait(&boc->cond, &boc->mtx, 0);
 		rv = ObjGetLen(wrk, oc);
 	}
-	Lck_Unlock(&bo->boc->mtx);
+	Lck_Unlock(&boc->mtx);
 	return (rv);
 }
 
 void
-VBO_setstate(struct busyobj *bo, enum busyobj_state_e next)
+VBO_setstate(struct boc *boc, enum busyobj_state_e next)
 {
-	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
-	assert(bo->do_stream || next != BOS_STREAM);
-	assert(next > bo->boc->state);
-	Lck_Lock(&bo->boc->mtx);
-	bo->boc->state = next;
-	AZ(pthread_cond_broadcast(&bo->boc->cond));
-	Lck_Unlock(&bo->boc->mtx);
+
+	CHECK_OBJ_NOTNULL(boc, BOC_MAGIC);
+
+	// assert(bo->do_stream || next != BOS_STREAM);
+	assert(next > boc->state);
+	Lck_Lock(&boc->mtx);
+	boc->state = next;
+	AZ(pthread_cond_broadcast(&boc->cond));
+	Lck_Unlock(&boc->mtx);
 }
 
 void
-VBO_waitstate(struct busyobj *bo, enum busyobj_state_e want)
+VBO_waitstate(struct boc *boc, enum busyobj_state_e want)
 {
-	Lck_Lock(&bo->boc->mtx);
+
+	CHECK_OBJ_NOTNULL(boc, BOC_MAGIC);
+
+	Lck_Lock(&boc->mtx);
 	while (1) {
-		if (bo->boc->state >= want)
+		if (boc->state >= want)
 			break;
-		(void)Lck_CondWait(&bo->boc->cond, &bo->boc->mtx, 0);
+		(void)Lck_CondWait(&boc->cond, &boc->mtx, 0);
 	}
-	Lck_Unlock(&bo->boc->mtx);
+	Lck_Unlock(&boc->mtx);
 }
diff --git a/bin/varnishd/cache/cache_esi_deliver.c b/bin/varnishd/cache/cache_esi_deliver.c
index a4f8d72..90d7e78 100644
--- a/bin/varnishd/cache/cache_esi_deliver.c
+++ b/bin/varnishd/cache/cache_esi_deliver.c
@@ -689,7 +689,7 @@ ved_stripgzip(struct req *req, struct busyobj *bo)
 
 	/* XXX: Is this really required ? */
 	if (bo != NULL)
-		VBO_waitstate(bo, BOS_FINISHED);
+		VBO_waitstate(bo->boc, BOS_FINISHED);
 
 	AN(ObjCheckFlag(req->wrk, req->objcore, OF_GZIPED));
 
diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index 32b58ac..55e10ca 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -202,7 +202,7 @@ vbf_stp_mkbereq(const struct worker *wrk, struct busyobj *bo)
 	bo->ws_bo = WS_Snapshot(bo->ws);
 	HTTP_Copy(bo->bereq, bo->bereq0);
 
-	VBO_setstate(bo, BOS_REQ_DONE);
+	VBO_setstate(bo->boc, BOS_REQ_DONE);
 	return (F_STP_STARTFETCH);
 }
 
@@ -648,7 +648,7 @@ vbf_stp_fetch(struct worker *wrk, struct busyobj *bo)
 
 	if (bo->do_stream) {
 		HSH_Unbusy(wrk, bo->fetch_objcore);
-		VBO_setstate(bo, BOS_STREAM);
+		VBO_setstate(bo->boc, BOS_STREAM);
 	}
 
 	VSLb(bo->vsl, SLT_Fetch_Body, "%u %s %s",
@@ -682,7 +682,7 @@ vbf_stp_fetch(struct worker *wrk, struct busyobj *bo)
 	   give predictable backend reuse behavior for varnishtest */
 	VDI_Finish(bo->wrk, bo);
 
-	VBO_setstate(bo, BOS_FINISHED);
+	VBO_setstate(bo->boc, BOS_FINISHED);
 	VSLb_ts_busyobj(bo, "BerespBody", W_TIM_real(wrk));
 	if (bo->stale_oc != NULL)
 		EXP_Rearm(bo->stale_oc, bo->stale_oc->exp.t_origin, 0, 0, 0);
@@ -736,7 +736,7 @@ vbf_stp_condfetch(struct worker *wrk, struct busyobj *bo)
 
 	if (bo->do_stream) {
 		HSH_Unbusy(wrk, bo->fetch_objcore);
-		VBO_setstate(bo, BOS_STREAM);
+		VBO_setstate(bo->boc, BOS_STREAM);
 	}
 
 	if (ObjIterate(wrk, bo->stale_oc, bo, vbf_objiterator))
@@ -758,7 +758,7 @@ vbf_stp_condfetch(struct worker *wrk, struct busyobj *bo)
 	   give predictable backend reuse behavior for varnishtest */
 	VDI_Finish(bo->wrk, bo);
 
-	VBO_setstate(bo, BOS_FINISHED);
+	VBO_setstate(bo->boc, BOS_FINISHED);
 	VSLb_ts_busyobj(bo, "BerespBody", W_TIM_real(wrk));
 	return (F_STP_DONE);
 }
@@ -858,7 +858,7 @@ vbf_stp_error(struct worker *wrk, struct busyobj *bo)
 	VSB_delete(synth_body);
 
 	HSH_Unbusy(wrk, bo->fetch_objcore);
-	VBO_setstate(bo, BOS_FINISHED);
+	VBO_setstate(bo->boc, BOS_FINISHED);
 	return (F_STP_DONE);
 }
 
@@ -881,7 +881,7 @@ vbf_stp_fail(struct worker *wrk, struct busyobj *bo)
 		    bo->fetch_objcore->exp.t_origin, 0, 0, 0);
 	}
 	wrk->stats->fetch_failed++;
-	VBO_setstate(bo, BOS_FAILED);
+	VBO_setstate(bo->boc, BOS_FAILED);
 	return (F_STP_DONE);
 }
 
@@ -1031,9 +1031,9 @@ VBF_Fetch(struct worker *wrk, struct req *req, struct objcore *oc,
 	} else {
 		bo_fetch = NULL; /* ref transferred to fetch thread */
 		if (mode == VBF_BACKGROUND) {
-			VBO_waitstate(bo, BOS_REQ_DONE);
+			VBO_waitstate(bo->boc, BOS_REQ_DONE);
 		} else {
-			VBO_waitstate(bo, BOS_STREAM);
+			VBO_waitstate(bo->boc, BOS_STREAM);
 			if (bo->boc->state == BOS_FAILED) {
 				AN((oc->flags & OC_F_FAILED));
 			} else {
diff --git a/bin/varnishd/cache/cache_fetch_proc.c b/bin/varnishd/cache/cache_fetch_proc.c
index 87f943e..f404a35 100644
--- a/bin/varnishd/cache/cache_fetch_proc.c
+++ b/bin/varnishd/cache/cache_fetch_proc.c
@@ -98,7 +98,7 @@ VFP_Extend(const struct vfp_ctx *vc, ssize_t sz)
 {
 	CHECK_OBJ_NOTNULL(vc, VFP_CTX_MAGIC);
 
-	VBO_extend(vc->wrk, vc->oc, vc->bo, sz);
+	VBO_extend(vc->wrk, vc->oc, vc->bo->boc, sz);
 }
 
 /**********************************************************************
diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index 0c793ef..95ef52f 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -202,7 +202,7 @@ cnt_deliver(struct worker *wrk, struct req *req)
 		if (req->esi_level == 0 && bo->boc->state == BOS_FINISHED) {
 			VBO_DerefBusyObj(wrk, &bo);
 		} else if (!bo->do_stream) {
-			VBO_waitstate(bo, BOS_FINISHED);
+			VBO_waitstate(bo->boc, BOS_FINISHED);
 			VBO_DerefBusyObj(wrk, &bo);
 		}
 	}
@@ -216,7 +216,7 @@ cnt_deliver(struct worker *wrk, struct req *req)
 
 	if (req->objcore->flags & (OC_F_PRIVATE | OC_F_PASS)) {
 		if (bo != NULL)
-			VBO_waitstate(bo, BOS_FINISHED);
+			VBO_waitstate(bo->boc, BOS_FINISHED);
 		ObjSlim(wrk, req->objcore);
 	}
 
diff --git a/bin/varnishd/storage/storage_simple.c b/bin/varnishd/storage/storage_simple.c
index d2286ba..a7d3236 100644
--- a/bin/varnishd/storage/storage_simple.c
+++ b/bin/varnishd/storage/storage_simple.c
@@ -257,7 +257,7 @@ sml_iterator(struct worker *wrk, struct objcore *oc,
 
 	while (1) {
 		ol = len;
-		nl = VBO_waitlen(wrk, oc, bo, ol);
+		nl = VBO_waitlen(wrk, oc, bo->boc, ol);
 		if (bo->boc->state == BOS_FAILED) {
 			ret = -1;
 			break;



From phk at FreeBSD.org  Thu Jan 28 11:49:19 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 28 Jan 2016 12:49:19 +0100
Subject: [master] 3479b0d Move VBO_*() for state management of objcores being
 filled to OBJ*().
Message-ID: <E1aOl4h-0006XR-3W@project.varnish-software.com>


commit 3479b0d8305a86bc452cc2848bf7e63a055fa6c8
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Jan 19 10:54:20 2016 +0000

    Move VBO_*() for state management of objcores being filled to OBJ*().

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index 951e438..d314942 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -680,10 +680,6 @@ double BAN_Time(const struct ban *ban);
 /* cache_busyobj.c */
 struct busyobj *VBO_GetBusyObj(struct worker *, const struct req *);
 void VBO_DerefBusyObj(struct worker *wrk, struct busyobj **busyobj);
-void VBO_extend(struct worker *, struct objcore *, struct boc *, ssize_t);
-ssize_t VBO_waitlen(struct worker *, struct objcore *, struct boc *, ssize_t l);
-void VBO_setstate(struct boc *, enum busyobj_state_e next);
-void VBO_waitstate(struct boc *, enum busyobj_state_e want);
 
 /* cache_cli.c [CLI] */
 extern pthread_t cli_thread;
@@ -854,7 +850,10 @@ typedef int objiterate_f(void *priv, int flush, const void *ptr, ssize_t len);
 int ObjIterate(struct worker *, struct objcore *,
     void *priv, objiterate_f *func);
 int ObjGetSpace(struct worker *, struct objcore *, ssize_t *sz, uint8_t **ptr);
-void ObjExtend(struct worker *, struct objcore *, ssize_t l);
+void ObjExtend(struct worker *, struct objcore *, struct boc *, ssize_t l);
+ssize_t ObjWaitExtend(struct worker *, struct objcore *, struct boc *, ssize_t l);
+void ObjSetState(struct boc *, enum busyobj_state_e next);
+void ObjWaitState(struct boc *, enum busyobj_state_e want);
 void ObjTrimStore(struct worker *, struct objcore *);
 void ObjTouch(struct worker *wrk, struct objcore *oc, double now);
 unsigned ObjGetXID(struct worker *, struct objcore *);
diff --git a/bin/varnishd/cache/cache_busyobj.c b/bin/varnishd/cache/cache_busyobj.c
index f970442..c7c562b 100644
--- a/bin/varnishd/cache/cache_busyobj.c
+++ b/bin/varnishd/cache/cache_busyobj.c
@@ -225,71 +225,3 @@ VBO_DerefBusyObj(struct worker *wrk, struct busyobj **pbo)
 
 	vbo_Free(&bo);
 }
-
-void
-VBO_extend(struct worker *wrk, struct objcore *oc, struct boc *boc,
-    ssize_t l)
-{
-
-	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
-	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
-	CHECK_OBJ_NOTNULL(boc, BOC_MAGIC);
-	if (l == 0)
-		return;
-	assert(l > 0);
-	Lck_Lock(&boc->mtx);
-	ObjExtend(wrk, oc, l);
-	AZ(pthread_cond_broadcast(&boc->cond));
-	Lck_Unlock(&boc->mtx);
-}
-
-ssize_t
-VBO_waitlen(struct worker *wrk, struct objcore *oc, struct boc *boc,
-    ssize_t l)
-{
-	ssize_t rv;
-
-	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
-	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
-	CHECK_OBJ_NOTNULL(boc, BOC_MAGIC);
-	Lck_Lock(&boc->mtx);
-	rv = ObjGetLen(wrk, oc);
-	while (1) {
-		assert(l <= rv || boc->state == BOS_FAILED);
-		if (rv > l || boc->state >= BOS_FINISHED)
-			break;
-		(void)Lck_CondWait(&boc->cond, &boc->mtx, 0);
-		rv = ObjGetLen(wrk, oc);
-	}
-	Lck_Unlock(&boc->mtx);
-	return (rv);
-}
-
-void
-VBO_setstate(struct boc *boc, enum busyobj_state_e next)
-{
-
-	CHECK_OBJ_NOTNULL(boc, BOC_MAGIC);
-
-	// assert(bo->do_stream || next != BOS_STREAM);
-	assert(next > boc->state);
-	Lck_Lock(&boc->mtx);
-	boc->state = next;
-	AZ(pthread_cond_broadcast(&boc->cond));
-	Lck_Unlock(&boc->mtx);
-}
-
-void
-VBO_waitstate(struct boc *boc, enum busyobj_state_e want)
-{
-
-	CHECK_OBJ_NOTNULL(boc, BOC_MAGIC);
-
-	Lck_Lock(&boc->mtx);
-	while (1) {
-		if (boc->state >= want)
-			break;
-		(void)Lck_CondWait(&boc->cond, &boc->mtx, 0);
-	}
-	Lck_Unlock(&boc->mtx);
-}
diff --git a/bin/varnishd/cache/cache_esi_deliver.c b/bin/varnishd/cache/cache_esi_deliver.c
index 90d7e78..e7bc423 100644
--- a/bin/varnishd/cache/cache_esi_deliver.c
+++ b/bin/varnishd/cache/cache_esi_deliver.c
@@ -689,7 +689,7 @@ ved_stripgzip(struct req *req, struct busyobj *bo)
 
 	/* XXX: Is this really required ? */
 	if (bo != NULL)
-		VBO_waitstate(bo->boc, BOS_FINISHED);
+		ObjWaitState(bo->boc, BOS_FINISHED);
 
 	AN(ObjCheckFlag(req->wrk, req->objcore, OF_GZIPED));
 
diff --git a/bin/varnishd/cache/cache_esi_fetch.c b/bin/varnishd/cache/cache_esi_fetch.c
index bce3e45..e3a50db 100644
--- a/bin/varnishd/cache/cache_esi_fetch.c
+++ b/bin/varnishd/cache/cache_esi_fetch.c
@@ -95,8 +95,10 @@ vfp_vep_callback(struct vfp_ctx *vc, void *priv, ssize_t l, enum vgz_flag flg)
 		VGZ_Obuf(vef->vgz, ptr, dl);
 		i = VGZ_Gzip(vef->vgz, &dp, &dl, flg);
 		VGZ_UpdateObj(vc, vef->vgz, VUA_UPDATE);
-		vef->tot += dl;
-		VFP_Extend(vc, dl);
+		if (dl > 0) {
+			vef->tot += dl;
+			VFP_Extend(vc, dl);
+		}
 	} while (i != VGZ_ERROR &&
 	    (!VGZ_IbufEmpty(vef->vgz) || VGZ_ObufFull(vef->vgz)));
 	assert(i == VGZ_ERROR || VGZ_IbufEmpty(vef->vgz));
diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index 55e10ca..f2e5e91 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -202,7 +202,7 @@ vbf_stp_mkbereq(const struct worker *wrk, struct busyobj *bo)
 	bo->ws_bo = WS_Snapshot(bo->ws);
 	HTTP_Copy(bo->bereq, bo->bereq0);
 
-	VBO_setstate(bo->boc, BOS_REQ_DONE);
+	ObjSetState(bo->boc, BOS_REQ_DONE);
 	return (F_STP_STARTFETCH);
 }
 
@@ -648,7 +648,7 @@ vbf_stp_fetch(struct worker *wrk, struct busyobj *bo)
 
 	if (bo->do_stream) {
 		HSH_Unbusy(wrk, bo->fetch_objcore);
-		VBO_setstate(bo->boc, BOS_STREAM);
+		ObjSetState(bo->boc, BOS_STREAM);
 	}
 
 	VSLb(bo->vsl, SLT_Fetch_Body, "%u %s %s",
@@ -682,7 +682,7 @@ vbf_stp_fetch(struct worker *wrk, struct busyobj *bo)
 	   give predictable backend reuse behavior for varnishtest */
 	VDI_Finish(bo->wrk, bo);
 
-	VBO_setstate(bo->boc, BOS_FINISHED);
+	ObjSetState(bo->boc, BOS_FINISHED);
 	VSLb_ts_busyobj(bo, "BerespBody", W_TIM_real(wrk));
 	if (bo->stale_oc != NULL)
 		EXP_Rearm(bo->stale_oc, bo->stale_oc->exp.t_origin, 0, 0, 0);
@@ -736,7 +736,7 @@ vbf_stp_condfetch(struct worker *wrk, struct busyobj *bo)
 
 	if (bo->do_stream) {
 		HSH_Unbusy(wrk, bo->fetch_objcore);
-		VBO_setstate(bo->boc, BOS_STREAM);
+		ObjSetState(bo->boc, BOS_STREAM);
 	}
 
 	if (ObjIterate(wrk, bo->stale_oc, bo, vbf_objiterator))
@@ -758,7 +758,7 @@ vbf_stp_condfetch(struct worker *wrk, struct busyobj *bo)
 	   give predictable backend reuse behavior for varnishtest */
 	VDI_Finish(bo->wrk, bo);
 
-	VBO_setstate(bo->boc, BOS_FINISHED);
+	ObjSetState(bo->boc, BOS_FINISHED);
 	VSLb_ts_busyobj(bo, "BerespBody", W_TIM_real(wrk));
 	return (F_STP_DONE);
 }
@@ -858,7 +858,7 @@ vbf_stp_error(struct worker *wrk, struct busyobj *bo)
 	VSB_delete(synth_body);
 
 	HSH_Unbusy(wrk, bo->fetch_objcore);
-	VBO_setstate(bo->boc, BOS_FINISHED);
+	ObjSetState(bo->boc, BOS_FINISHED);
 	return (F_STP_DONE);
 }
 
@@ -881,7 +881,7 @@ vbf_stp_fail(struct worker *wrk, struct busyobj *bo)
 		    bo->fetch_objcore->exp.t_origin, 0, 0, 0);
 	}
 	wrk->stats->fetch_failed++;
-	VBO_setstate(bo->boc, BOS_FAILED);
+	ObjSetState(bo->boc, BOS_FAILED);
 	return (F_STP_DONE);
 }
 
@@ -1031,9 +1031,9 @@ VBF_Fetch(struct worker *wrk, struct req *req, struct objcore *oc,
 	} else {
 		bo_fetch = NULL; /* ref transferred to fetch thread */
 		if (mode == VBF_BACKGROUND) {
-			VBO_waitstate(bo->boc, BOS_REQ_DONE);
+			ObjWaitState(bo->boc, BOS_REQ_DONE);
 		} else {
-			VBO_waitstate(bo->boc, BOS_STREAM);
+			ObjWaitState(bo->boc, BOS_STREAM);
 			if (bo->boc->state == BOS_FAILED) {
 				AN((oc->flags & OC_F_FAILED));
 			} else {
diff --git a/bin/varnishd/cache/cache_fetch_proc.c b/bin/varnishd/cache/cache_fetch_proc.c
index f404a35..f017f5d 100644
--- a/bin/varnishd/cache/cache_fetch_proc.c
+++ b/bin/varnishd/cache/cache_fetch_proc.c
@@ -98,7 +98,7 @@ VFP_Extend(const struct vfp_ctx *vc, ssize_t sz)
 {
 	CHECK_OBJ_NOTNULL(vc, VFP_CTX_MAGIC);
 
-	VBO_extend(vc->wrk, vc->oc, vc->bo->boc, sz);
+	ObjExtend(vc->wrk, vc->oc, vc->bo->boc, sz);
 }
 
 /**********************************************************************
diff --git a/bin/varnishd/cache/cache_obj.c b/bin/varnishd/cache/cache_obj.c
index 67faea0..d168044 100644
--- a/bin/varnishd/cache/cache_obj.c
+++ b/bin/varnishd/cache/cache_obj.c
@@ -107,15 +107,83 @@ ObjGetSpace(struct worker *wrk, struct objcore *oc, ssize_t *sz, uint8_t **ptr)
  */
 
 void
-ObjExtend(struct worker *wrk, struct objcore *oc, ssize_t l)
+ObjExtend(struct worker *wrk, struct objcore *oc, struct boc *boc, ssize_t l)
 {
 	const struct obj_methods *om = obj_getmethods(oc);
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
+	AN(om->objextend);
 	assert(l > 0);
 
-	AN(om->objextend);
-	om->objextend(wrk, oc, l);
+	if (boc != NULL) {
+		CHECK_OBJ_NOTNULL(boc, BOC_MAGIC);
+		Lck_Lock(&boc->mtx);
+		om->objextend(wrk, oc, l);
+		AZ(pthread_cond_broadcast(&boc->cond));
+		Lck_Unlock(&boc->mtx);
+	} else {
+		om->objextend(wrk, oc, l);
+	}
+}
+
+/*====================================================================
+ */
+
+ssize_t
+ObjWaitExtend(struct worker *wrk, struct objcore *oc, struct boc *boc,
+    ssize_t l)
+{
+	ssize_t rv;
+
+	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
+	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
+	CHECK_OBJ_NOTNULL(boc, BOC_MAGIC);
+	Lck_Lock(&boc->mtx);
+	rv = ObjGetLen(wrk, oc);
+	while (1) {
+		assert(l <= rv || boc->state == BOS_FAILED);
+		if (rv > l || boc->state >= BOS_FINISHED)
+			break;
+		(void)Lck_CondWait(&boc->cond, &boc->mtx, 0);
+		rv = ObjGetLen(wrk, oc);
+	}
+	Lck_Unlock(&boc->mtx);
+	return (rv);
+}
+
+/*====================================================================
+ */
+
+void
+ObjSetState(struct boc *boc, enum busyobj_state_e next)
+{
+
+	CHECK_OBJ_NOTNULL(boc, BOC_MAGIC);
+
+	// assert(bo->do_stream || next != BOS_STREAM);
+	assert(next > boc->state);
+	Lck_Lock(&boc->mtx);
+	boc->state = next;
+	AZ(pthread_cond_broadcast(&boc->cond));
+	Lck_Unlock(&boc->mtx);
+}
+
+/*====================================================================
+ */
+
+void
+ObjWaitState(struct boc *boc, enum busyobj_state_e want)
+{
+
+	CHECK_OBJ_NOTNULL(boc, BOC_MAGIC);
+
+	Lck_Lock(&boc->mtx);
+	while (1) {
+		if (boc->state >= want)
+			break;
+		(void)Lck_CondWait(&boc->cond, &boc->mtx, 0);
+	}
+	Lck_Unlock(&boc->mtx);
 }
 
 /*====================================================================
diff --git a/bin/varnishd/cache/cache_req_body.c b/bin/varnishd/cache/cache_req_body.c
index 30089d1..1c05484 100644
--- a/bin/varnishd/cache/cache_req_body.c
+++ b/bin/varnishd/cache/cache_req_body.c
@@ -254,7 +254,7 @@ VRB_Cache(struct req *req, ssize_t maxsize)
 			req->acct.req_bodybytes += l;
 			if (yet >= l)
 				yet -= l;
-			ObjExtend(req->wrk, req->body_oc, l);
+			ObjExtend(req->wrk, req->body_oc, NULL, l);
 		}
 
 	} while (vfps == VFP_OK);
diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index 95ef52f..cccb046 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -202,7 +202,7 @@ cnt_deliver(struct worker *wrk, struct req *req)
 		if (req->esi_level == 0 && bo->boc->state == BOS_FINISHED) {
 			VBO_DerefBusyObj(wrk, &bo);
 		} else if (!bo->do_stream) {
-			VBO_waitstate(bo->boc, BOS_FINISHED);
+			ObjWaitState(bo->boc, BOS_FINISHED);
 			VBO_DerefBusyObj(wrk, &bo);
 		}
 	}
@@ -216,7 +216,7 @@ cnt_deliver(struct worker *wrk, struct req *req)
 
 	if (req->objcore->flags & (OC_F_PRIVATE | OC_F_PASS)) {
 		if (bo != NULL)
-			VBO_waitstate(bo->boc, BOS_FINISHED);
+			ObjWaitState(bo->boc, BOS_FINISHED);
 		ObjSlim(wrk, req->objcore);
 	}
 
@@ -296,7 +296,7 @@ cnt_synth(struct worker *wrk, struct req *req)
 		sz = szl;
 		if (ObjGetSpace(wrk, req->objcore, &sz, &ptr) && sz >= szl) {
 			memcpy(ptr, VSB_data(synth_body), szl);
-			ObjExtend(wrk, req->objcore, szl);
+			ObjExtend(wrk, req->objcore, NULL, szl);
 		} else
 			szl = -1;
 	}
diff --git a/bin/varnishd/storage/storage_simple.c b/bin/varnishd/storage/storage_simple.c
index a7d3236..f071930 100644
--- a/bin/varnishd/storage/storage_simple.c
+++ b/bin/varnishd/storage/storage_simple.c
@@ -257,7 +257,7 @@ sml_iterator(struct worker *wrk, struct objcore *oc,
 
 	while (1) {
 		ol = len;
-		nl = VBO_waitlen(wrk, oc, bo->boc, ol);
+		nl = ObjWaitExtend(wrk, oc, bo->boc, ol);
 		if (bo->boc->state == BOS_FAILED) {
 			ret = -1;
 			break;



From phk at FreeBSD.org  Thu Jan 28 11:49:19 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 28 Jan 2016 12:49:19 +0100
Subject: [master] 0290623 Reduce strength of arguments from busyobj to boc
Message-ID: <E1aOl4h-0006Xb-6H@project.varnish-software.com>


commit 0290623e66ba6524334b08c5a4a0cadf6cdef20c
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Jan 19 11:29:33 2016 +0000

    Reduce strength of arguments from busyobj to boc

diff --git a/bin/varnishd/cache/cache_esi_deliver.c b/bin/varnishd/cache/cache_esi_deliver.c
index e7bc423..a9a8348 100644
--- a/bin/varnishd/cache/cache_esi_deliver.c
+++ b/bin/varnishd/cache/cache_esi_deliver.c
@@ -668,7 +668,7 @@ ved_objiterate(void *priv, int flush, const void *ptr, ssize_t len)
 }
 
 static void
-ved_stripgzip(struct req *req, struct busyobj *bo)
+ved_stripgzip(struct req *req, struct boc *boc)
 {
 	ssize_t l;
 	char *p;
@@ -687,9 +687,9 @@ ved_stripgzip(struct req *req, struct busyobj *bo)
 	foo.preq = ecx->preq;
 	memset(foo.tailbuf, 0xdd, sizeof foo.tailbuf);
 
-	/* XXX: Is this really required ? */
-	if (bo != NULL)
-		ObjWaitState(bo->boc, BOS_FINISHED);
+	/* OA_GZIPBITS is not valid until BOS_FINISHED */
+	if (boc != NULL)
+		ObjWaitState(boc, BOS_FINISHED);
 
 	AN(ObjCheckFlag(req->wrk, req->objcore, OF_GZIPED));
 
@@ -751,13 +751,13 @@ ved_vdp_bytes(struct req *req, enum vdp_action act, void **priv,
 /*--------------------------------------------------------------------*/
 
 static void __match_proto__(vtr_deliver_f)
-VED_Deliver(struct req *req, struct busyobj *bo, int wantbody)
+VED_Deliver(struct req *req, struct boc *boc, int wantbody)
 {
 	int i;
 	struct ecx *ecx;
 
 	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
-	CHECK_OBJ_ORNULL(bo, BUSYOBJ_MAGIC);
+	CHECK_OBJ_ORNULL(boc, BOC_MAGIC);
 	CHECK_OBJ_NOTNULL(req->objcore, OBJCORE_MAGIC);
 
 	CAST_OBJ_NOTNULL(ecx, req->transport_priv, ECX_MAGIC);
@@ -768,7 +768,7 @@ VED_Deliver(struct req *req, struct busyobj *bo, int wantbody)
 	req->res_mode |= RES_ESI_CHILD;
 	i = ObjCheckFlag(req->wrk, req->objcore, OF_GZIPED);
 	if (ecx->isgzip && i && !(req->res_mode & RES_ESI)) {
-		ved_stripgzip(req, bo);
+		ved_stripgzip(req, boc);
 	} else {
 		if (ecx->isgzip && !i)
 			VDP_push(req, ved_pretend_gzip, ecx, 1);
diff --git a/bin/varnishd/cache/cache_priv.h b/bin/varnishd/cache/cache_priv.h
index 483d041..f3a3aaa 100644
--- a/bin/varnishd/cache/cache_priv.h
+++ b/bin/varnishd/cache/cache_priv.h
@@ -36,7 +36,7 @@
  * or may not, be talking a "real" HTTP protocol itself.
  */
 
-typedef void vtr_deliver_f (struct req *, struct busyobj *, int sendbody);
+typedef void vtr_deliver_f (struct req *, struct boc *, int sendbody);
 
 struct transport {
 	unsigned		magic;
diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index cccb046..315b334 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -48,7 +48,7 @@
 #include "vtim.h"
 
 static void
-cnt_vdp(struct req *req, struct busyobj *bo)
+cnt_vdp(struct req *req, struct boc *boc)
 {
 	const char *r;
 	uint16_t status;
@@ -58,7 +58,7 @@ cnt_vdp(struct req *req, struct busyobj *bo)
 	CHECK_OBJ_NOTNULL(req->transport, TRANSPORT_MAGIC);
 
 	resp_len = http_GetContentLength(req->resp);
-	if (bo != NULL)
+	if (boc != NULL)
 		req->resp_len = resp_len;
 	else
 		req->resp_len = ObjGetLen(req->wrk, req->objcore);
@@ -105,7 +105,7 @@ cnt_vdp(struct req *req, struct busyobj *bo)
 			    "Content-Length: %jd", req->resp_len);
 	}
 
-	req->transport->deliver(req, bo, sendbody);
+	req->transport->deliver(req, boc, sendbody);
 }
 
 /*--------------------------------------------------------------------
@@ -207,7 +207,7 @@ cnt_deliver(struct worker *wrk, struct req *req)
 		}
 	}
 
-	cnt_vdp(req, bo);
+	cnt_vdp(req, bo == NULL ? NULL : bo->boc);
 
 	VSLb_ts_req(req, "Resp", W_TIM_real(wrk));
 
diff --git a/bin/varnishd/http1/cache_http1_deliver.c b/bin/varnishd/http1/cache_http1_deliver.c
index 99ac91f..7885ad2 100644
--- a/bin/varnishd/http1/cache_http1_deliver.c
+++ b/bin/varnishd/http1/cache_http1_deliver.c
@@ -79,12 +79,12 @@ v1d_error(struct req *req, const char *msg)
  */
 
 void __match_proto__(vtr_deliver_f)
-V1D_Deliver(struct req *req, struct busyobj *bo, int sendbody)
+V1D_Deliver(struct req *req, struct boc *boc, int sendbody)
 {
 	int err = 0;
 
 	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
-	CHECK_OBJ_ORNULL(bo, BUSYOBJ_MAGIC);
+	CHECK_OBJ_ORNULL(boc, BOC_MAGIC);
 	CHECK_OBJ_NOTNULL(req->objcore, OBJCORE_MAGIC);
 
 	if (sendbody) {



From phk at FreeBSD.org  Thu Jan 28 11:49:19 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 28 Jan 2016 12:49:19 +0100
Subject: [master] db2c2eb Move the pass-abandon up where it belongs.
Message-ID: <E1aOl4h-0006Xl-8a@project.varnish-software.com>


commit db2c2ebb80568c42110bcb9a73fd7e5af1313e88
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Jan 19 11:52:11 2016 +0000

    Move the pass-abandon up where it belongs.

diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index 315b334..ccb76e4 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -209,6 +209,10 @@ cnt_deliver(struct worker *wrk, struct req *req)
 
 	cnt_vdp(req, bo == NULL ? NULL : bo->boc);
 
+	/* pass+streaming, abandon fetch in case delivery terminated early */
+	if (bo != NULL && req->objcore->flags & OC_F_PASS)
+		bo->abandon = 1;
+
 	VSLb_ts_req(req, "Resp", W_TIM_real(wrk));
 
 	if (http_HdrIs(req->resp, H_Connection, "close"))
diff --git a/bin/varnishd/storage/storage_simple.c b/bin/varnishd/storage/storage_simple.c
index f071930..4b61aeb 100644
--- a/bin/varnishd/storage/storage_simple.c
+++ b/bin/varnishd/storage/storage_simple.c
@@ -307,8 +307,6 @@ sml_iterator(struct worker *wrk, struct objcore *oc,
 			break;
 		}
 	}
-	if (oc->flags & OC_F_PASS)
-		bo->abandon = 1;
 	VBO_DerefBusyObj(wrk, &bo);
 	return (ret);
 }



From phk at FreeBSD.org  Thu Jan 28 11:49:19 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 28 Jan 2016 12:49:19 +0100
Subject: [master] 3096620 fix comment
Message-ID: <E1aOl4h-0006Xx-Al@project.varnish-software.com>


commit 30966200182b0f0836fe05e2f5231856c094dbfc
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Jan 19 12:25:27 2016 +0000

    fix comment

diff --git a/bin/varnishd/cache/cache_req_body.c b/bin/varnishd/cache/cache_req_body.c
index 1c05484..bc7d270 100644
--- a/bin/varnishd/cache/cache_req_body.c
+++ b/bin/varnishd/cache/cache_req_body.c
@@ -132,7 +132,7 @@ VRB_Iterate(struct req *req, objiterate_f *func, void *priv)
 }
 
 /*----------------------------------------------------------------------
- * DiscardReqBody() is a dedicated function, because we might
+ * VRB_Ignore() is a dedicated function, because we might
  * be able to disuade or terminate its transmission in some protocols.
  * For HTTP1 we have no such luck, and we just iterate it into oblivion.
  */



From phk at FreeBSD.org  Thu Jan 28 11:49:19 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 28 Jan 2016 12:49:19 +0100
Subject: [master] e25593a Fix esi:include syntax
Message-ID: <E1aOl4h-0006Y6-Ci@project.varnish-software.com>


commit e25593a2d54dcc6da8f50d910004a72e64a21856
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Jan 19 12:50:45 2016 +0000

    Fix esi:include syntax

diff --git a/bin/varnishtest/tests/r00427.vtc b/bin/varnishtest/tests/r00427.vtc
index a75247b..ebeb2ba 100644
--- a/bin/varnishtest/tests/r00427.vtc
+++ b/bin/varnishtest/tests/r00427.vtc
@@ -3,9 +3,9 @@ varnishtest "client close in ESI delivery"
 server s1 {
 	rxreq
 	txresp -body {
-		<esi:include src="/foo">
-		<esi:include src="/bar">
-		<esi:include src="/barf">
+		<esi:include src="/foo"/>
+		<esi:include src="/bar"/>
+		<esi:include src="/barf"/>
 	}
 
 	rxreq



From phk at FreeBSD.org  Thu Jan 28 11:49:19 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 28 Jan 2016 12:49:19 +0100
Subject: [master] 4ddf701 Rename the 'boc' variable to 'busy' to avoid
 misunderstandings now that we have a struct boc.
Message-ID: <E1aOl4h-0006YF-E7@project.varnish-software.com>


commit 4ddf70120e1c89fc0f1d2a122ceae3b013867846
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Jan 20 20:25:28 2016 +0000

    Rename the 'boc' variable to 'busy' to avoid misunderstandings
    now that we have a struct boc.

diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index ccb76e4..52c05b4 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -355,7 +355,7 @@ cnt_fetch(struct worker *wrk, struct req *req)
 static enum req_fsm_nxt
 cnt_lookup(struct worker *wrk, struct req *req)
 {
-	struct objcore *oc, *boc;
+	struct objcore *oc, *busy;
 	enum lookup_e lr;
 	int had_objhead = 0;
 
@@ -370,7 +370,7 @@ cnt_lookup(struct worker *wrk, struct req *req)
 	AZ(req->objcore);
 	if (req->hash_objhead)
 		had_objhead = 1;
-	lr = HSH_Lookup(req, &oc, &boc,
+	lr = HSH_Lookup(req, &oc, &busy,
 	    req->esi_level == 0 ? 1 : 0,
 	    req->hash_always_miss ? 1 : 0
 	);
@@ -386,10 +386,10 @@ cnt_lookup(struct worker *wrk, struct req *req)
 	if (had_objhead)
 		VSLb_ts_req(req, "Waitinglist", W_TIM_real(wrk));
 
-	if (boc == NULL) {
+	if (busy == NULL) {
 		VRY_Finish(req, DISCARD);
 	} else {
-		AN(boc->flags & OC_F_BUSY);
+		AN(busy->flags & OC_F_BUSY);
 		VRY_Finish(req, KEEP);
 	}
 
@@ -397,9 +397,9 @@ cnt_lookup(struct worker *wrk, struct req *req)
 	if (lr == HSH_MISS) {
 		/* Found nothing */
 		AZ(oc);
-		AN(boc);
-		AN(boc->flags & OC_F_BUSY);
-		req->objcore = boc;
+		AN(busy);
+		AN(busy->flags & OC_F_BUSY);
+		req->objcore = busy;
 		req->req_step = R_STP_MISS;
 		return (REQ_FSM_MORE);
 	}
@@ -408,9 +408,9 @@ cnt_lookup(struct worker *wrk, struct req *req)
 	AZ(oc->flags & OC_F_BUSY);
 	req->objcore = oc;
 
-	if ((oc->flags & OC_F_PASS) && boc != NULL) {
+	if ((oc->flags & OC_F_PASS) && busy != NULL) {
 		/* Treat a graced Hit-For-Pass as a miss */
-		req->objcore = boc;
+		req->objcore = busy;
 		req->stale_oc = oc;
 		req->req_step = R_STP_MISS;
 		return (REQ_FSM_MORE);
@@ -431,10 +431,10 @@ cnt_lookup(struct worker *wrk, struct req *req)
 
 	switch (wrk->handling) {
 	case VCL_RET_DELIVER:
-		if (boc != NULL) {
+		if (busy != NULL) {
 			AZ(oc->flags & OC_F_PASS);
-			AZ(boc->busyobj);
-			VBF_Fetch(wrk, req, boc, oc, VBF_BACKGROUND);
+			AZ(busy->busyobj);
+			VBF_Fetch(wrk, req, busy, oc, VBF_BACKGROUND);
 		} else {
 			(void)VRB_Ignore(req);// XXX: handle err
 		}
@@ -447,8 +447,8 @@ cnt_lookup(struct worker *wrk, struct req *req)
 		    "change return(fetch) to return(miss) in vcl_hit{}");
 		/* FALL-THROUGH */
 	case VCL_RET_MISS:
-		if (boc != NULL) {
-			req->objcore = boc;
+		if (busy != NULL) {
+			req->objcore = busy;
 			req->stale_oc = oc;
 			req->req_step = R_STP_MISS;
 		} else {
@@ -481,8 +481,8 @@ cnt_lookup(struct worker *wrk, struct req *req)
 	/* Drop our object, we won't need it */
 	(void)HSH_DerefObjCore(wrk, &req->objcore);
 
-	if (boc != NULL) {
-		(void)HSH_DerefObjCore(wrk, &boc);
+	if (busy != NULL) {
+		(void)HSH_DerefObjCore(wrk, &busy);
 		VRY_Clear(req);
 	}
 



From phk at FreeBSD.org  Thu Jan 28 11:49:19 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 28 Jan 2016 12:49:19 +0100
Subject: [master] 5efe0cd Add a backpointer from boc to busyobj for now.
Message-ID: <E1aOl4h-0006YO-Gh@project.varnish-software.com>


commit 5efe0cda4edb9f652856b3d907a7483264e84fa8
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Jan 20 20:28:16 2016 +0000

    Add a backpointer from boc to busyobj for now.
    
    I don't think this will be necessary at the end state, but it makes
    the transformation easier.

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index d314942..596e572 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -399,6 +399,7 @@ struct boc {
 	enum busyobj_state_e	state;
 	uint8_t			*vary;
 
+	struct busyobj		*busyobj;
 };
 
 /* Object core structure ---------------------------------------------
diff --git a/bin/varnishd/cache/cache_busyobj.c b/bin/varnishd/cache/cache_busyobj.c
index c7c562b..4c199a3 100644
--- a/bin/varnishd/cache/cache_busyobj.c
+++ b/bin/varnishd/cache/cache_busyobj.c
@@ -71,6 +71,7 @@ vbo_New(void)
 	INIT_OBJ(bo->boc, BOC_MAGIC);
 	Lck_New(&bo->boc->mtx, lck_busyobj);
 	AZ(pthread_cond_init(&bo->boc->cond, NULL));
+	bo->boc->busyobj = bo;
 	return (bo);
 }
 



From phk at FreeBSD.org  Thu Jan 28 11:49:19 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 28 Jan 2016 12:49:19 +0100
Subject: [master] 98791be When we ref/deref busy oc's,
 act on struct boc instead of busyobj
Message-ID: <E1aOl4h-0006Ya-Ix@project.varnish-software.com>


commit 98791be34788a626da906f5058a8335490b5e344
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Jan 20 21:00:49 2016 +0000

    When we ref/deref busy oc's, act on struct boc instead of busyobj

diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index a670143..f24871f 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -706,11 +706,12 @@ HSH_Ref(struct objcore *oc)
  * Gain a reference on the busyobj, if the objcore has one
  */
 
-struct busyobj *
+struct boc *
 HSH_RefBusy(const struct objcore *oc)
 {
 	struct objhead *oh;
 	struct busyobj *bo;
+	struct boc *boc = NULL;
 
 	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
 	oh = oc->objhead;
@@ -719,10 +720,30 @@ HSH_RefBusy(const struct objcore *oc)
 	assert(oc->refcnt > 0);
 	bo = oc->busyobj;
 	CHECK_OBJ_ORNULL(bo, BUSYOBJ_MAGIC);
-	if (bo != NULL)
-		bo->boc->refcount++;
+	if (bo != NULL) {
+		CHECK_OBJ_NOTNULL(bo->boc, BOC_MAGIC);
+		boc = bo->boc;
+		assert(boc->busyobj == bo);
+		boc->refcount++;
+	}
 	Lck_Unlock(&oh->mtx);
-	return (bo);
+	return (boc);
+}
+
+void
+HSH_DerefBusy(struct worker *wrk, struct boc **pp)
+{
+	struct boc *boc;
+	struct busyobj *bo;
+
+	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
+	AN(pp);
+	boc = *pp;
+	*pp = NULL;
+	CHECK_OBJ_NOTNULL(boc, BOC_MAGIC);
+	CHECK_OBJ_NOTNULL(boc->busyobj, BUSYOBJ_MAGIC);
+	bo = boc->busyobj;
+	VBO_DerefBusyObj(wrk, &bo);
 }
 
 /*--------------------------------------------------------------------
diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index 52c05b4..7fc3f1e 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -115,7 +115,7 @@ cnt_vdp(struct req *req, struct boc *boc)
 static enum req_fsm_nxt
 cnt_deliver(struct worker *wrk, struct req *req)
 {
-	struct busyobj *bo;
+	struct boc *boc;
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
@@ -197,21 +197,23 @@ cnt_deliver(struct worker *wrk, struct req *req)
 		http_PutResponse(req->resp, "HTTP/1.1", 304, NULL);
 
 	/* Grab a ref to the bo if there is one, and hand it down */
-	bo = HSH_RefBusy(req->objcore);
-	if (bo != NULL) {
-		if (req->esi_level == 0 && bo->boc->state == BOS_FINISHED) {
-			VBO_DerefBusyObj(wrk, &bo);
-		} else if (!bo->do_stream) {
-			ObjWaitState(bo->boc, BOS_FINISHED);
-			VBO_DerefBusyObj(wrk, &bo);
+	boc = HSH_RefBusy(req->objcore);
+	if (boc != NULL) {
+		if (req->esi_level == 0 && boc->state == BOS_FINISHED) {
+			HSH_DerefBusy(wrk, &boc);
+		} else if (!boc->busyobj->do_stream) {
+			ObjWaitState(boc, BOS_FINISHED);
+			HSH_DerefBusy(wrk, &boc);
 		}
 	}
 
-	cnt_vdp(req, bo == NULL ? NULL : bo->boc);
+	cnt_vdp(req, boc);
 
 	/* pass+streaming, abandon fetch in case delivery terminated early */
-	if (bo != NULL && req->objcore->flags & OC_F_PASS)
-		bo->abandon = 1;
+	if (boc != NULL && req->objcore->flags & OC_F_PASS) {
+		CHECK_OBJ_NOTNULL(boc->busyobj, BUSYOBJ_MAGIC);
+		boc->busyobj->abandon = 1;
+	}
 
 	VSLb_ts_req(req, "Resp", W_TIM_real(wrk));
 
@@ -219,13 +221,13 @@ cnt_deliver(struct worker *wrk, struct req *req)
 		req->doclose = SC_RESP_CLOSE;
 
 	if (req->objcore->flags & (OC_F_PRIVATE | OC_F_PASS)) {
-		if (bo != NULL)
-			ObjWaitState(bo->boc, BOS_FINISHED);
+		if (boc != NULL)
+			ObjWaitState(boc, BOS_FINISHED);
 		ObjSlim(wrk, req->objcore);
 	}
 
-	if (bo != NULL)
-		VBO_DerefBusyObj(wrk, &bo);
+	if (boc != NULL)
+		HSH_DerefBusy(wrk, &boc);
 
 	(void)HSH_DerefObjCore(wrk, &req->objcore);
 	http_Teardown(req->resp);
diff --git a/bin/varnishd/hash/hash_slinger.h b/bin/varnishd/hash/hash_slinger.h
index 1fa219e..f307ba5 100644
--- a/bin/varnishd/hash/hash_slinger.h
+++ b/bin/varnishd/hash/hash_slinger.h
@@ -72,7 +72,8 @@ void HSH_Insert(struct worker *, const void *hash, struct objcore *);
 void HSH_Purge(struct worker *, struct objhead *, double ttl, double grace,
     double keep);
 void HSH_config(const char *h_arg);
-struct busyobj *HSH_RefBusy(const struct objcore *oc);
+struct boc *HSH_RefBusy(const struct objcore *);
+void HSH_DerefBusy(struct worker *wrk, struct boc **);
 struct objcore *HSH_Private(struct worker *wrk);
 
 #ifdef VARNISH_CACHE_CHILD
diff --git a/bin/varnishd/storage/storage_simple.c b/bin/varnishd/storage/storage_simple.c
index 4b61aeb..d0a003b 100644
--- a/bin/varnishd/storage/storage_simple.c
+++ b/bin/varnishd/storage/storage_simple.c
@@ -227,7 +227,7 @@ static int __match_proto__(objiterate_f)
 sml_iterator(struct worker *wrk, struct objcore *oc,
     void *priv, objiterate_f *func)
 {
-	struct busyobj *bo;
+	struct boc *boc;
 	struct object *obj;
 	struct storage *st;
 	struct storage *checkpoint = NULL;
@@ -243,9 +243,9 @@ sml_iterator(struct worker *wrk, struct objcore *oc,
 	obj = sml_getobj(wrk, oc);
 	CHECK_OBJ_NOTNULL(obj, OBJECT_MAGIC);
 
-	bo = HSH_RefBusy(oc);
+	boc = HSH_RefBusy(oc);
 
-	if (bo == NULL) {
+	if (boc == NULL) {
 		VTAILQ_FOREACH(st, &obj->list, list)
 			if (func(priv, 0, st->ptr, st->len))
 				return (-1);
@@ -257,17 +257,17 @@ sml_iterator(struct worker *wrk, struct objcore *oc,
 
 	while (1) {
 		ol = len;
-		nl = ObjWaitExtend(wrk, oc, bo->boc, ol);
-		if (bo->boc->state == BOS_FAILED) {
+		nl = ObjWaitExtend(wrk, oc, boc, ol);
+		if (boc->state == BOS_FAILED) {
 			ret = -1;
 			break;
 		}
 		if (nl == ol) {
-			if (bo->boc->state == BOS_FINISHED)
+			if (boc->state == BOS_FINISHED)
 				break;
 			continue;
 		}
-		Lck_Lock(&bo->boc->mtx);
+		Lck_Lock(&boc->mtx);
 		AZ(VTAILQ_EMPTY(&obj->list));
 		if (checkpoint == NULL) {
 			st = VTAILQ_FIRST(&obj->list);
@@ -300,14 +300,14 @@ sml_iterator(struct worker *wrk, struct objcore *oc,
 		st = VTAILQ_NEXT(st, list);
 		if (st != NULL && st->len == 0)
 			st = NULL;
-		Lck_Unlock(&bo->boc->mtx);
-		assert(l > 0 || bo->boc->state == BOS_FINISHED);
+		Lck_Unlock(&boc->mtx);
+		assert(l > 0 || boc->state == BOS_FINISHED);
 		if (func(priv, st != NULL ? 0 : 1, p, l)) {
 			ret = -1;
 			break;
 		}
 	}
-	VBO_DerefBusyObj(wrk, &bo);
+	HSH_DerefBusy(wrk, &boc);
 	return (ret);
 }
 



From phk at FreeBSD.org  Thu Jan 28 11:49:19 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 28 Jan 2016 12:49:19 +0100
Subject: [master] 9d2255a Have objcore point to a boc rather than busyobj
Message-ID: <E1aOl4h-0006Yk-Kz@project.varnish-software.com>


commit 9d2255a539c0ad9634cbf43caccc0b8aa544ce73
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Jan 20 21:16:27 2016 +0000

    Have objcore point to a boc rather than busyobj

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index 596e572..23f9091 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -416,7 +416,7 @@ struct objcore {
 	int			refcnt;
 	struct storeobj		stobj[1];
 	struct objhead		*objhead;
-	struct busyobj		*busyobj;
+	struct boc		*boc;
 	double			timer_when;
 	long			hits;
 
diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index f2e5e91..9e12f18 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -950,7 +950,7 @@ vbf_fetch_thread(struct worker *wrk, void *priv)
 		VSLb(bo->vsl, SLT_Length, "%ju",
 		    (uintmax_t)ObjGetLen(bo->wrk, bo->fetch_objcore));
 	}
-	AZ(bo->fetch_objcore->busyobj);
+	AZ(bo->fetch_objcore->boc);
 
 	if (bo->stale_oc != NULL)
 		(void)HSH_DerefObjCore(wrk, &bo->stale_oc);
@@ -994,7 +994,7 @@ VBF_Fetch(struct worker *wrk, struct req *req, struct objcore *oc,
 	bo_fetch = bo;
 	bo->boc->refcount = 2;
 
-	oc->busyobj = bo;
+	oc->boc = bo->boc;
 
 	AN(bo->vcl);
 
diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index f24871f..0e6f269 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -392,13 +392,12 @@ HSH_Lookup(struct req *req, struct objcore **ocp, struct objcore **bocp,
 			continue;
 
 		if (oc->flags & OC_F_BUSY) {
-			CHECK_OBJ_ORNULL(oc->busyobj, BUSYOBJ_MAGIC);
+			CHECK_OBJ_ORNULL(oc->boc, BOC_MAGIC);
 			if (req->hash_ignore_busy)
 				continue;
 
-			if (oc->busyobj != NULL &&
-			    oc->busyobj->boc->vary != NULL &&
-			    !VRY_Match(req, oc->busyobj->boc->vary))
+			if (oc->boc != NULL && oc->boc->vary != NULL &&
+			    !VRY_Match(req, oc->boc->vary))
 				continue;
 
 			busy_found = 1;
@@ -624,7 +623,7 @@ HSH_Fail(struct objcore *oc)
 	Lck_Lock(&oh->mtx);
 	oc->flags |= OC_F_FAILED;
 	oc->flags &= ~OC_F_INCOMPLETE;
-	oc->busyobj = NULL;
+	oc->boc = NULL;
 	Lck_Unlock(&oh->mtx);
 }
 
@@ -642,7 +641,7 @@ HSH_Complete(struct objcore *oc)
 	CHECK_OBJ(oh, OBJHEAD_MAGIC);
 
 	Lck_Lock(&oh->mtx);
-	oc->busyobj = NULL;
+	oc->boc = NULL;
 	oc->flags &= ~OC_F_INCOMPLETE;
 	Lck_Unlock(&oh->mtx);
 }
@@ -710,20 +709,17 @@ struct boc *
 HSH_RefBusy(const struct objcore *oc)
 {
 	struct objhead *oh;
-	struct busyobj *bo;
-	struct boc *boc = NULL;
+	struct boc *boc;
 
 	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
 	oh = oc->objhead;
 	CHECK_OBJ_NOTNULL(oh, OBJHEAD_MAGIC);
 	Lck_Lock(&oh->mtx);
 	assert(oc->refcnt > 0);
-	bo = oc->busyobj;
-	CHECK_OBJ_ORNULL(bo, BUSYOBJ_MAGIC);
-	if (bo != NULL) {
-		CHECK_OBJ_NOTNULL(bo->boc, BOC_MAGIC);
-		boc = bo->boc;
-		assert(boc->busyobj == bo);
+	boc = oc->boc;
+	CHECK_OBJ_ORNULL(boc, BOC_MAGIC);
+	if (boc != NULL) {
+		CHECK_OBJ_NOTNULL(boc->busyobj, BUSYOBJ_MAGIC);
 		boc->refcount++;
 	}
 	Lck_Unlock(&oh->mtx);
diff --git a/bin/varnishd/cache/cache_panic.c b/bin/varnishd/cache/cache_panic.c
index 6dbc3c0..7d8f47b 100644
--- a/bin/varnishd/cache/cache_panic.c
+++ b/bin/varnishd/cache/cache_panic.c
@@ -315,6 +315,12 @@ pan_busyobj(struct vsb *vsb, const struct busyobj *bo)
 	VSB_printf(vsb, "},\n");
 }
 
+static void
+pan_boc(struct vsb *vsb, const struct boc *boc)
+{
+	pan_busyobj(vsb, boc->busyobj);
+}
+
 /*--------------------------------------------------------------------*/
 
 static void
@@ -366,8 +372,8 @@ pan_req(struct vsb *vsb, const struct req *req)
 
 	if (req->objcore != NULL) {
 		pan_objcore(vsb, "REQ", req->objcore);
-		if (req->objcore->busyobj != NULL)
-			pan_busyobj(vsb, req->objcore->busyobj);
+		if (req->objcore->boc != NULL)
+			pan_boc(vsb, req->objcore->boc);
 	}
 
 	VSB_printf(vsb, "flags = {\n");
diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index 7fc3f1e..dde710c 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -435,7 +435,7 @@ cnt_lookup(struct worker *wrk, struct req *req)
 	case VCL_RET_DELIVER:
 		if (busy != NULL) {
 			AZ(oc->flags & OC_F_PASS);
-			AZ(busy->busyobj);
+			AZ(busy->boc);
 			VBF_Fetch(wrk, req, busy, oc, VBF_BACKGROUND);
 		} else {
 			(void)VRB_Ignore(req);// XXX: handle err
diff --git a/bin/varnishd/storage/storage_simple.c b/bin/varnishd/storage/storage_simple.c
index d0a003b..5f0ea1b 100644
--- a/bin/varnishd/storage/storage_simple.c
+++ b/bin/varnishd/storage/storage_simple.c
@@ -371,11 +371,11 @@ sml_getspace(struct worker *wrk, struct objcore *oc, ssize_t *sz,
 	if (st == NULL)
 		return (0);
 
-	if (oc->busyobj != NULL) {
-		CHECK_OBJ_NOTNULL(oc->busyobj, BUSYOBJ_MAGIC);
-		Lck_Lock(&oc->busyobj->boc->mtx);
+	if (oc->boc != NULL) {
+		CHECK_OBJ_NOTNULL(oc->boc, BOC_MAGIC);
+		Lck_Lock(&oc->boc->mtx);
 		VTAILQ_INSERT_TAIL(&o->list, st, list);
-		Lck_Unlock(&oc->busyobj->boc->mtx);
+		Lck_Unlock(&oc->boc->mtx);
 	} else {
 		AN(oc->flags & (OC_F_PRIVATE));
 		VTAILQ_INSERT_TAIL(&o->list, st, list);
@@ -440,10 +440,10 @@ sml_trimstore(struct worker *wrk, struct objcore *oc)
 		return;
 
 	if (st->len == 0) {
-		if (oc->busyobj != NULL) {
-			Lck_Lock(&oc->busyobj->boc->mtx);
+		if (oc->boc != NULL) {
+			Lck_Lock(&oc->boc->mtx);
 			VTAILQ_REMOVE(&o->list, st, list);
-			Lck_Unlock(&oc->busyobj->boc->mtx);
+			Lck_Unlock(&oc->boc->mtx);
 		} else {
 			VTAILQ_REMOVE(&o->list, st, list);
 		}
@@ -465,21 +465,21 @@ sml_trimstore(struct worker *wrk, struct objcore *oc)
 
 	memcpy(st1->ptr, st->ptr, st->len);
 	st1->len = st->len;
-	if (oc->busyobj != NULL) {
-		Lck_Lock(&oc->busyobj->boc->mtx);
+	if (oc->boc != NULL) {
+		Lck_Lock(&oc->boc->mtx);
 		VTAILQ_REMOVE(&o->list, st, list);
 		VTAILQ_INSERT_TAIL(&o->list, st1, list);
-		Lck_Unlock(&oc->busyobj->boc->mtx);
+		Lck_Unlock(&oc->boc->mtx);
 	} else {
 		VTAILQ_REMOVE(&o->list, st, list);
 		VTAILQ_INSERT_TAIL(&o->list, st1, list);
 	}
-	if (oc->busyobj == NULL) {
+	if (oc->boc == NULL) {
 		sml_stv_free(stv, st);
 	} else {
 		/* sml_stable frees this */
-		AZ(oc->busyobj->boc->stevedore_priv);
-		oc->busyobj->boc->stevedore_priv = st;
+		AZ(oc->boc->stevedore_priv);
+		oc->boc->stevedore_priv = st;
 	}
 }
 



From phk at FreeBSD.org  Thu Jan 28 11:49:19 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 28 Jan 2016 12:49:19 +0100
Subject: [master] 6460bff rename busyobj_state_e to boc_state_e
Message-ID: <E1aOl4h-0006Yv-N9@project.varnish-software.com>


commit 6460bff24e0ed6aecfddaf35697c5cdf3737e400
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Jan 20 21:58:18 2016 +0000

    rename busyobj_state_e to boc_state_e

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index 23f9091..2de7093 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -381,7 +381,7 @@ struct storeobj {
 /*
  * The macro-states we expose outside the fetch code
  */
-enum busyobj_state_e {
+enum boc_state_e {
 	BOS_INVALID = 0,	/* don't touch (yet) */
 	BOS_REQ_DONE,		/* beresp.* can be examined */
 	BOS_STREAM,		/* beresp.* can be examined */
@@ -396,7 +396,7 @@ struct boc {
 	struct lock		mtx;
 	pthread_cond_t		cond;
 	void			*stevedore_priv;
-	enum busyobj_state_e	state;
+	enum boc_state_e	state;
 	uint8_t			*vary;
 
 	struct busyobj		*busyobj;
@@ -853,8 +853,8 @@ int ObjIterate(struct worker *, struct objcore *,
 int ObjGetSpace(struct worker *, struct objcore *, ssize_t *sz, uint8_t **ptr);
 void ObjExtend(struct worker *, struct objcore *, struct boc *, ssize_t l);
 ssize_t ObjWaitExtend(struct worker *, struct objcore *, struct boc *, ssize_t l);
-void ObjSetState(struct boc *, enum busyobj_state_e next);
-void ObjWaitState(struct boc *, enum busyobj_state_e want);
+void ObjSetState(struct boc *, enum boc_state_e next);
+void ObjWaitState(struct boc *, enum boc_state_e want);
 void ObjTrimStore(struct worker *, struct objcore *);
 void ObjTouch(struct worker *wrk, struct objcore *oc, double now);
 unsigned ObjGetXID(struct worker *, struct objcore *);
diff --git a/bin/varnishd/cache/cache_obj.c b/bin/varnishd/cache/cache_obj.c
index d168044..85cba50 100644
--- a/bin/varnishd/cache/cache_obj.c
+++ b/bin/varnishd/cache/cache_obj.c
@@ -155,7 +155,7 @@ ObjWaitExtend(struct worker *wrk, struct objcore *oc, struct boc *boc,
  */
 
 void
-ObjSetState(struct boc *boc, enum busyobj_state_e next)
+ObjSetState(struct boc *boc, enum boc_state_e next)
 {
 
 	CHECK_OBJ_NOTNULL(boc, BOC_MAGIC);
@@ -172,7 +172,7 @@ ObjSetState(struct boc *boc, enum busyobj_state_e next)
  */
 
 void
-ObjWaitState(struct boc *boc, enum busyobj_state_e want)
+ObjWaitState(struct boc *boc, enum boc_state_e want)
 {
 
 	CHECK_OBJ_NOTNULL(boc, BOC_MAGIC);



From phk at FreeBSD.org  Thu Jan 28 11:49:19 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 28 Jan 2016 12:49:19 +0100
Subject: [master] ef7ad9c Clean up boc->refcount initialization
Message-ID: <E1aOl4h-0006Z4-Oh@project.varnish-software.com>


commit ef7ad9c043a9e9d4abbf8a183867c5549f1d9604
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Jan 20 23:18:41 2016 +0000

    Clean up boc->refcount initialization

diff --git a/bin/varnishd/cache/cache_busyobj.c b/bin/varnishd/cache/cache_busyobj.c
index 4c199a3..45a6968 100644
--- a/bin/varnishd/cache/cache_busyobj.c
+++ b/bin/varnishd/cache/cache_busyobj.c
@@ -72,6 +72,7 @@ vbo_New(void)
 	Lck_New(&bo->boc->mtx, lck_busyobj);
 	AZ(pthread_cond_init(&bo->boc->cond, NULL));
 	bo->boc->busyobj = bo;
+	bo->boc->refcount = 1;
 	return (bo);
 }
 
@@ -103,9 +104,6 @@ VBO_GetBusyObj(struct worker *wrk, const struct req *req)
 
 	bo = vbo_New();
 	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
-	AZ(bo->boc->refcount);
-
-	bo->boc->refcount = 1;
 
 	p = (void*)(bo + 1);
 	p = (void*)PRNDUP(p);
diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index 9e12f18..3087972 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -985,16 +985,17 @@ VBF_Fetch(struct worker *wrk, struct req *req, struct objcore *oc,
 	}
 
 	bo = VBO_GetBusyObj(wrk, req);
+	oc->boc = bo->boc;
+
 	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
 	VSLb(bo->vsl, SLT_Begin, "bereq %u %s", VXID(req->vsl->wid), how);
 	VSLb(req->vsl, SLT_Link, "bereq %u %s", VXID(bo->vsl->wid), how);
 
 	THR_SetBusyobj(bo);
 
+	assert(HSH_RefBusy(oc) == bo->boc);
 	bo_fetch = bo;
-	bo->boc->refcount = 2;
 
-	oc->boc = bo->boc;
 
 	AN(bo->vcl);
 



From phk at FreeBSD.org  Thu Jan 28 11:49:19 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 28 Jan 2016 12:49:19 +0100
Subject: [master] 5d43251 Only drop objcore->boc when the refcount goes to
 zero.
Message-ID: <E1aOl4h-0006ZD-Rp@project.varnish-software.com>


commit 5d4325176d8541134f6190fd630177a497694b21
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Jan 20 23:24:45 2016 +0000

    Only drop objcore->boc when the refcount goes to zero.

diff --git a/bin/varnishd/cache/cache_busyobj.c b/bin/varnishd/cache/cache_busyobj.c
index 45a6968..809138e 100644
--- a/bin/varnishd/cache/cache_busyobj.c
+++ b/bin/varnishd/cache/cache_busyobj.c
@@ -157,8 +157,6 @@ void
 VBO_DerefBusyObj(struct worker *wrk, struct busyobj **pbo)
 {
 	struct busyobj *bo;
-	struct objcore *oc = NULL;
-	unsigned r;
 
 	CHECK_OBJ_ORNULL(wrk, WORKER_MAGIC);
 	AN(pbo);
@@ -166,23 +164,6 @@ VBO_DerefBusyObj(struct worker *wrk, struct busyobj **pbo)
 	*pbo = NULL;
 	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
 	CHECK_OBJ_ORNULL(bo->fetch_objcore, OBJCORE_MAGIC);
-	if (bo->fetch_objcore != NULL) {
-		oc = bo->fetch_objcore;
-		CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
-		CHECK_OBJ_NOTNULL(oc->objhead, OBJHEAD_MAGIC);
-		Lck_Lock(&oc->objhead->mtx);
-		assert(bo->boc->refcount > 0);
-		r = --bo->boc->refcount;
-		Lck_Unlock(&oc->objhead->mtx);
-	} else {
-		Lck_Lock(&bo->boc->mtx);
-		assert(bo->boc->refcount > 0);
-		r = --bo->boc->refcount;
-		Lck_Unlock(&bo->boc->mtx);
-	}
-
-	if (r)
-		return;
 
 	AZ(bo->htc);
 	AZ(bo->stale_oc);
diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index 3087972..e7dc558 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -950,13 +950,13 @@ vbf_fetch_thread(struct worker *wrk, void *priv)
 		VSLb(bo->vsl, SLT_Length, "%ju",
 		    (uintmax_t)ObjGetLen(bo->wrk, bo->fetch_objcore));
 	}
-	AZ(bo->fetch_objcore->boc);
+	// AZ(bo->fetch_objcore->boc);	// XXX
 
 	if (bo->stale_oc != NULL)
 		(void)HSH_DerefObjCore(wrk, &bo->stale_oc);
 
 	wrk->vsl = NULL;
-	VBO_DerefBusyObj(wrk, &bo);
+	HSH_DerefBusy(wrk, bo->fetch_objcore);
 	THR_SetBusyobj(NULL);
 }
 
@@ -1028,7 +1028,7 @@ VBF_Fetch(struct worker *wrk, struct req *req, struct objcore *oc,
 		(void)vbf_stp_fail(req->wrk, bo);
 		if (bo->stale_oc != NULL)
 			(void)HSH_DerefObjCore(wrk, &bo->stale_oc);
-		VBO_DerefBusyObj(wrk, &bo_fetch);
+		HSH_DerefBusy(wrk, bo->fetch_objcore);
 	} else {
 		bo_fetch = NULL; /* ref transferred to fetch thread */
 		if (mode == VBF_BACKGROUND) {
@@ -1044,6 +1044,6 @@ VBF_Fetch(struct worker *wrk, struct req *req, struct objcore *oc,
 	}
 	AZ(bo_fetch);
 	VSLb_ts_req(req, "Fetch", W_TIM_real(wrk));
-	VBO_DerefBusyObj(wrk, &bo);
+	HSH_DerefBusy(wrk, bo->fetch_objcore);
 	THR_SetBusyobj(NULL);
 }
diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index 0e6f269..ddd8279 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -623,7 +623,6 @@ HSH_Fail(struct objcore *oc)
 	Lck_Lock(&oh->mtx);
 	oc->flags |= OC_F_FAILED;
 	oc->flags &= ~OC_F_INCOMPLETE;
-	oc->boc = NULL;
 	Lck_Unlock(&oh->mtx);
 }
 
@@ -641,7 +640,6 @@ HSH_Complete(struct objcore *oc)
 	CHECK_OBJ(oh, OBJHEAD_MAGIC);
 
 	Lck_Lock(&oh->mtx);
-	oc->boc = NULL;
 	oc->flags &= ~OC_F_INCOMPLETE;
 	Lck_Unlock(&oh->mtx);
 }
@@ -727,19 +725,24 @@ HSH_RefBusy(const struct objcore *oc)
 }
 
 void
-HSH_DerefBusy(struct worker *wrk, struct boc **pp)
+HSH_DerefBusy(struct worker *wrk, struct objcore *oc)
 {
 	struct boc *boc;
-	struct busyobj *bo;
+	unsigned r;
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
-	AN(pp);
-	boc = *pp;
-	*pp = NULL;
+	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
+	boc = oc->boc;
 	CHECK_OBJ_NOTNULL(boc, BOC_MAGIC);
 	CHECK_OBJ_NOTNULL(boc->busyobj, BUSYOBJ_MAGIC);
-	bo = boc->busyobj;
-	VBO_DerefBusyObj(wrk, &bo);
+	Lck_Lock(&oc->objhead->mtx);
+	assert(boc->refcount > 0);
+	r = --boc->refcount;
+	if (r == 0)
+		oc->boc = NULL;
+	Lck_Unlock(&oc->objhead->mtx);
+	if (r == 0)
+		VBO_DerefBusyObj(wrk, &boc->busyobj);
 }
 
 /*--------------------------------------------------------------------
diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index dde710c..4bb06d8 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -200,10 +200,12 @@ cnt_deliver(struct worker *wrk, struct req *req)
 	boc = HSH_RefBusy(req->objcore);
 	if (boc != NULL) {
 		if (req->esi_level == 0 && boc->state == BOS_FINISHED) {
-			HSH_DerefBusy(wrk, &boc);
+			HSH_DerefBusy(wrk, req->objcore);
+			boc = NULL;
 		} else if (!boc->busyobj->do_stream) {
 			ObjWaitState(boc, BOS_FINISHED);
-			HSH_DerefBusy(wrk, &boc);
+			HSH_DerefBusy(wrk, req->objcore);
+			boc = NULL;
 		}
 	}
 
@@ -227,7 +229,7 @@ cnt_deliver(struct worker *wrk, struct req *req)
 	}
 
 	if (boc != NULL)
-		HSH_DerefBusy(wrk, &boc);
+		HSH_DerefBusy(wrk, req->objcore);
 
 	(void)HSH_DerefObjCore(wrk, &req->objcore);
 	http_Teardown(req->resp);
@@ -600,6 +602,7 @@ cnt_pipe(struct worker *wrk, struct req *req)
 	if (VDI_Http1Pipe(req, bo) < 0)
 		VSLb(bo->vsl, SLT_VCL_Error, "Backend does not support pipe");
 	http_Teardown(bo->bereq);
+bo->boc->refcount = 0;
 	VBO_DerefBusyObj(wrk, &bo);
 	THR_SetBusyobj(NULL);
 	return (REQ_FSM_DONE);
diff --git a/bin/varnishd/hash/hash_slinger.h b/bin/varnishd/hash/hash_slinger.h
index f307ba5..384b5af 100644
--- a/bin/varnishd/hash/hash_slinger.h
+++ b/bin/varnishd/hash/hash_slinger.h
@@ -73,7 +73,7 @@ void HSH_Purge(struct worker *, struct objhead *, double ttl, double grace,
     double keep);
 void HSH_config(const char *h_arg);
 struct boc *HSH_RefBusy(const struct objcore *);
-void HSH_DerefBusy(struct worker *wrk, struct boc **);
+void HSH_DerefBusy(struct worker *wrk, struct objcore *);
 struct objcore *HSH_Private(struct worker *wrk);
 
 #ifdef VARNISH_CACHE_CHILD
diff --git a/bin/varnishd/storage/storage_simple.c b/bin/varnishd/storage/storage_simple.c
index 5f0ea1b..8d313fb 100644
--- a/bin/varnishd/storage/storage_simple.c
+++ b/bin/varnishd/storage/storage_simple.c
@@ -307,7 +307,7 @@ sml_iterator(struct worker *wrk, struct objcore *oc,
 			break;
 		}
 	}
-	HSH_DerefBusy(wrk, &boc);
+	HSH_DerefBusy(wrk, oc);
 	return (ret);
 }
 



From phk at FreeBSD.org  Thu Jan 28 11:49:19 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 28 Jan 2016 12:49:19 +0100
Subject: [master] e6aaefc Allocate the boc with the objcore. Dispose when
 refcount goes to zero. Private objcores for pipe neither want nor need a boc,
 so make it optional.
Message-ID: <E1aOl4h-0006ZN-VK@project.varnish-software.com>


commit e6aaefc5928d2c2ab96fda5a074e9b5b2bd6326d
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu Jan 21 00:21:19 2016 +0000

    Allocate the boc with the objcore.
    Dispose when refcount goes to zero.
    Private objcores for pipe neither want nor need a boc, so make it optional.

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index 2de7093..b080036 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -463,7 +463,7 @@ struct busyobj {
 	unsigned		magic;
 #define BUSYOBJ_MAGIC		0x23b95567
 
-	struct boc		boc[1];
+	struct boc		*boc;
 
 	char			*end;
 
diff --git a/bin/varnishd/cache/cache_busyobj.c b/bin/varnishd/cache/cache_busyobj.c
index 809138e..110c44f 100644
--- a/bin/varnishd/cache/cache_busyobj.c
+++ b/bin/varnishd/cache/cache_busyobj.c
@@ -68,11 +68,6 @@ vbo_New(void)
 	XXXAN(bo);
 	bo->magic = BUSYOBJ_MAGIC;
 	bo->end = (char *)bo + sz;
-	INIT_OBJ(bo->boc, BOC_MAGIC);
-	Lck_New(&bo->boc->mtx, lck_busyobj);
-	AZ(pthread_cond_init(&bo->boc->cond, NULL));
-	bo->boc->busyobj = bo;
-	bo->boc->refcount = 1;
 	return (bo);
 }
 
@@ -86,9 +81,6 @@ vbo_Free(struct busyobj **bop)
 	*bop = NULL;
 	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
 	AZ(bo->htc);
-	AZ(bo->boc->refcount);
-	AZ(pthread_cond_destroy(&bo->boc->cond));
-	Lck_Delete(&bo->boc->mtx);
 	MPL_Free(vbopool, bo);
 }
 
@@ -192,14 +184,6 @@ VBO_DerefBusyObj(struct worker *wrk, struct busyobj **pbo)
 
 	VCL_Rel(&bo->vcl);
 
-	AZ(bo->boc->stevedore_priv);
-	AZ(bo->boc->refcount);
-	bo->boc->state = BOS_INVALID;
-	if (bo->boc->vary != NULL) {
-		free(bo->boc->vary);
-		bo->boc->vary = NULL;
-	}
-
 	memset(&bo->retries, 0,
 	    sizeof *bo - offsetof(struct busyobj, retries));
 
diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index e7dc558..6c353b6 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -985,7 +985,8 @@ VBF_Fetch(struct worker *wrk, struct req *req, struct objcore *oc,
 	}
 
 	bo = VBO_GetBusyObj(wrk, req);
-	oc->boc = bo->boc;
+	bo->boc = oc->boc;
+	bo->boc->busyobj = bo;
 
 	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
 	VSLb(bo->vsl, SLT_Begin, "bereq %u %s", VXID(req->vsl->wid), how);
diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index ddd8279..96f1f7a 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -68,14 +68,21 @@ static struct objhead *private_oh;
 /*---------------------------------------------------------------------*/
 
 static struct objcore *
-hsh_NewObjCore(struct worker *wrk)
+hsh_NewObjCore(struct worker *wrk, int boc)
 {
 	struct objcore *oc;
 
 	ALLOC_OBJ(oc, OBJCORE_MAGIC);
-	XXXAN(oc);
+	AN(oc);
 	wrk->stats->n_objectcore++;
 	oc->flags |= OC_F_BUSY | OC_F_INCOMPLETE;
+	if (boc) {
+		ALLOC_OBJ(oc->boc, BOC_MAGIC);
+		AN(oc->boc);
+		Lck_New(&oc->boc->mtx, lck_busyobj);
+		AZ(pthread_cond_init(&oc->boc->cond, NULL));
+		oc->boc->refcount = 1;
+	}
 	return (oc);
 }
 
@@ -104,7 +111,7 @@ hsh_prealloc(struct worker *wrk)
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 
 	if (wrk->nobjcore == NULL)
-		wrk->nobjcore = hsh_NewObjCore(wrk);
+		wrk->nobjcore = hsh_NewObjCore(wrk, 1);
 	CHECK_OBJ_NOTNULL(wrk->nobjcore, OBJCORE_MAGIC);
 
 	if (wrk->nobjhead == NULL) {
@@ -120,13 +127,13 @@ hsh_prealloc(struct worker *wrk)
 /*---------------------------------------------------------------------*/
 
 struct objcore *
-HSH_Private(struct worker *wrk)
+HSH_Private(struct worker *wrk, int wantboc)
 {
 	struct objcore *oc;
 
 	CHECK_OBJ_NOTNULL(private_oh, OBJHEAD_MAGIC);
 
-	oc = hsh_NewObjCore(wrk);
+	oc = hsh_NewObjCore(wrk, wantboc);
 	AN(oc);
 	oc->refcnt = 1;
 	oc->objhead = private_oh;
@@ -717,7 +724,7 @@ HSH_RefBusy(const struct objcore *oc)
 	boc = oc->boc;
 	CHECK_OBJ_ORNULL(boc, BOC_MAGIC);
 	if (boc != NULL) {
-		CHECK_OBJ_NOTNULL(boc->busyobj, BUSYOBJ_MAGIC);
+		//CHECK_OBJ_NOTNULL(boc->busyobj, BUSYOBJ_MAGIC);
 		boc->refcount++;
 	}
 	Lck_Unlock(&oh->mtx);
@@ -741,8 +748,13 @@ HSH_DerefBusy(struct worker *wrk, struct objcore *oc)
 	if (r == 0)
 		oc->boc = NULL;
 	Lck_Unlock(&oc->objhead->mtx);
-	if (r == 0)
+	if (r == 0) {
 		VBO_DerefBusyObj(wrk, &boc->busyobj);
+		AZ(pthread_cond_destroy(&boc->cond));
+		Lck_Delete(&boc->mtx);
+		free(boc->vary);
+		FREE_OBJ(boc);
+	}
 }
 
 /*--------------------------------------------------------------------
diff --git a/bin/varnishd/cache/cache_req_body.c b/bin/varnishd/cache/cache_req_body.c
index bc7d270..dbdf15d 100644
--- a/bin/varnishd/cache/cache_req_body.c
+++ b/bin/varnishd/cache/cache_req_body.c
@@ -216,7 +216,7 @@ VRB_Cache(struct req *req, ssize_t maxsize)
 		return (-1);
 	}
 
-	req->body_oc = HSH_Private(req->wrk);
+	req->body_oc = HSH_Private(req->wrk, 0);
 	AN(req->body_oc);
 	XXXAN(STV_NewObject(req->wrk, req->body_oc, TRANSIENT_STORAGE, 8));
 
diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index 4bb06d8..365c482 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -293,7 +293,8 @@ cnt_synth(struct worker *wrk, struct req *req)
 	if (http_HdrIs(req->resp, H_Connection, "close"))
 		req->doclose = SC_RESP_CLOSE;
 
-	req->objcore = HSH_Private(wrk);
+	req->objcore = HSH_Private(wrk, 0);
+	AZ(req->objcore->boc);
 	CHECK_OBJ_NOTNULL(req->objcore, OBJCORE_MAGIC);
 	if (STV_NewObject(wrk, req->objcore, TRANSIENT_STORAGE, 1024)) {
 		szl = VSB_len(synth_body);
@@ -311,8 +312,9 @@ cnt_synth(struct worker *wrk, struct req *req)
 	if (szl < 0) {
 		VSLb(req->vsl, SLT_Error, "Could not get storage");
 		req->doclose = SC_OVERLOAD;
-	} else
+	} else {
 		cnt_vdp(req, NULL);
+	}
 
 	(void)HSH_DerefObjCore(wrk, &req->objcore);
 	VSB_delete(synth_body);
@@ -437,7 +439,7 @@ cnt_lookup(struct worker *wrk, struct req *req)
 	case VCL_RET_DELIVER:
 		if (busy != NULL) {
 			AZ(oc->flags & OC_F_PASS);
-			AZ(busy->boc);
+			CHECK_OBJ_NOTNULL(busy->boc, BOC_MAGIC);
 			VBF_Fetch(wrk, req, busy, oc, VBF_BACKGROUND);
 		} else {
 			(void)VRB_Ignore(req);// XXX: handle err
@@ -557,7 +559,7 @@ cnt_pass(struct worker *wrk, struct req *req)
 		break;
 	case VCL_RET_FETCH:
 		wrk->stats->s_pass++;
-		req->objcore = HSH_Private(wrk);
+		req->objcore = HSH_Private(wrk, 1);
 		CHECK_OBJ_NOTNULL(req->objcore, OBJCORE_MAGIC);
 		VBF_Fetch(wrk, req, req->objcore, NULL, VBF_PASS);
 		req->req_step = R_STP_FETCH;
@@ -602,7 +604,6 @@ cnt_pipe(struct worker *wrk, struct req *req)
 	if (VDI_Http1Pipe(req, bo) < 0)
 		VSLb(bo->vsl, SLT_VCL_Error, "Backend does not support pipe");
 	http_Teardown(bo->bereq);
-bo->boc->refcount = 0;
 	VBO_DerefBusyObj(wrk, &bo);
 	THR_SetBusyobj(NULL);
 	return (REQ_FSM_DONE);
diff --git a/bin/varnishd/hash/hash_slinger.h b/bin/varnishd/hash/hash_slinger.h
index 384b5af..7001204 100644
--- a/bin/varnishd/hash/hash_slinger.h
+++ b/bin/varnishd/hash/hash_slinger.h
@@ -74,7 +74,7 @@ void HSH_Purge(struct worker *, struct objhead *, double ttl, double grace,
 void HSH_config(const char *h_arg);
 struct boc *HSH_RefBusy(const struct objcore *);
 void HSH_DerefBusy(struct worker *wrk, struct objcore *);
-struct objcore *HSH_Private(struct worker *wrk);
+struct objcore *HSH_Private(struct worker *wrk, int wantboc);
 
 #ifdef VARNISH_CACHE_CHILD
 



From phk at FreeBSD.org  Thu Jan 28 11:49:20 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 28 Jan 2016 12:49:20 +0100
Subject: [master] 0375791 Start using our new "busy" criteria for objcores
Message-ID: <E1aOl4i-0006ZX-0y@project.varnish-software.com>


commit 0375791cac1f2333ab54932ba1d2025261082fab
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu Jan 21 10:22:00 2016 +0000

    Start using our new "busy" criteria for objcores

diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index 96f1f7a..d75178a 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -398,8 +398,9 @@ HSH_Lookup(struct req *req, struct objcore **ocp, struct objcore **bocp,
 		if (oc->flags & OC_F_FAILED)
 			continue;
 
-		if (oc->flags & OC_F_BUSY) {
+		if (oc->boc != NULL && oc->boc->state < BOS_STREAM) {
 			CHECK_OBJ_ORNULL(oc->boc, BOC_MAGIC);
+
 			if (req->hash_ignore_busy)
 				continue;
 
diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index 365c482..19f3797 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -203,7 +203,9 @@ cnt_deliver(struct worker *wrk, struct req *req)
 			HSH_DerefBusy(wrk, req->objcore);
 			boc = NULL;
 		} else if (!boc->busyobj->do_stream) {
-			ObjWaitState(boc, BOS_FINISHED);
+			/* XXX These should go away */
+			xxxassert(boc->state >= BOS_STREAM);
+			xxxassert(boc->state >= BOS_FINISHED);
 			HSH_DerefBusy(wrk, req->objcore);
 			boc = NULL;
 		}
diff --git a/bin/varnishtest/tests/c00013.vtc b/bin/varnishtest/tests/c00013.vtc
index 689e118..70eedad 100644
--- a/bin/varnishtest/tests/c00013.vtc
+++ b/bin/varnishtest/tests/c00013.vtc
@@ -43,5 +43,5 @@ client c2 {
 
 client c1 -wait
 
-varnish v1 -expect busy_sleep == 1
-varnish v1 -expect busy_wakeup == 1
+varnish v1 -expect busy_sleep >= 1
+varnish v1 -expect busy_wakeup >= 1



From phk at FreeBSD.org  Thu Jan 28 11:49:20 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 28 Jan 2016 12:49:20 +0100
Subject: [master] 99a40f5 Only ref the busy boc if it is not yet BOS_FINISHED
Message-ID: <E1aOl4i-0006Zg-2X@project.varnish-software.com>


commit 99a40f5ae307d757115c15b64a413c42298bc949
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu Jan 21 10:55:57 2016 +0000

    Only ref the busy boc if it is not yet BOS_FINISHED

diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index 6c353b6..84caa85 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -866,7 +866,7 @@ vbf_stp_error(struct worker *wrk, struct busyobj *bo)
  */
 
 static enum fetch_step
-vbf_stp_fail(struct worker *wrk, struct busyobj *bo)
+vbf_stp_fail(struct worker *wrk, const struct busyobj *bo)
 {
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index d75178a..3f2d9d8 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -724,10 +724,10 @@ HSH_RefBusy(const struct objcore *oc)
 	assert(oc->refcnt > 0);
 	boc = oc->boc;
 	CHECK_OBJ_ORNULL(boc, BOC_MAGIC);
-	if (boc != NULL) {
-		//CHECK_OBJ_NOTNULL(boc->busyobj, BUSYOBJ_MAGIC);
+	if (boc != NULL && boc->state < BOS_FINISHED) 
 		boc->refcount++;
-	}
+	else
+		boc = NULL;
 	Lck_Unlock(&oh->mtx);
 	return (boc);
 }
diff --git a/bin/varnishd/cache/cache_obj.c b/bin/varnishd/cache/cache_obj.c
index 85cba50..9bb6071 100644
--- a/bin/varnishd/cache/cache_obj.c
+++ b/bin/varnishd/cache/cache_obj.c
@@ -217,6 +217,8 @@ ObjTrimStore(struct worker *wrk, struct objcore *oc)
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 
+	assert(oc->boc == NULL || oc->boc->state < BOS_FINISHED);
+
 	if (om->objtrimstore != NULL)
 		om->objtrimstore(wrk, oc);
 	return;
diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index 19f3797..115473a 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -198,35 +198,20 @@ cnt_deliver(struct worker *wrk, struct req *req)
 
 	/* Grab a ref to the bo if there is one, and hand it down */
 	boc = HSH_RefBusy(req->objcore);
-	if (boc != NULL) {
-		if (req->esi_level == 0 && boc->state == BOS_FINISHED) {
-			HSH_DerefBusy(wrk, req->objcore);
-			boc = NULL;
-		} else if (!boc->busyobj->do_stream) {
-			/* XXX These should go away */
-			xxxassert(boc->state >= BOS_STREAM);
-			xxxassert(boc->state >= BOS_FINISHED);
-			HSH_DerefBusy(wrk, req->objcore);
-			boc = NULL;
-		}
-	}
 
 	cnt_vdp(req, boc);
 
-	/* pass+streaming, abandon fetch in case delivery terminated early */
-	if (boc != NULL && req->objcore->flags & OC_F_PASS) {
-		CHECK_OBJ_NOTNULL(boc->busyobj, BUSYOBJ_MAGIC);
-		boc->busyobj->abandon = 1;
-	}
-
 	VSLb_ts_req(req, "Resp", W_TIM_real(wrk));
 
 	if (http_HdrIs(req->resp, H_Connection, "close"))
 		req->doclose = SC_RESP_CLOSE;
 
 	if (req->objcore->flags & (OC_F_PRIVATE | OC_F_PASS)) {
-		if (boc != NULL)
+		if (boc != NULL) {
+			CHECK_OBJ_NOTNULL(boc->busyobj, BUSYOBJ_MAGIC);
+			boc->busyobj->abandon = 1;
 			ObjWaitState(boc, BOS_FINISHED);
+		}
 		ObjSlim(wrk, req->objcore);
 	}
 



From phk at FreeBSD.org  Thu Jan 28 11:49:20 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 28 Jan 2016 12:49:20 +0100
Subject: [master] ba104c9 Move the abandon flag from busyobj to objcore
Message-ID: <E1aOl4i-0006Zp-4J@project.varnish-software.com>


commit ba104c9249a51cad3e8917590a932a7c51dc9626
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu Jan 21 12:28:02 2016 +0000

    Move the abandon flag from busyobj to objcore

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index b080036..564023e 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -426,6 +426,7 @@ struct objcore {
 #define OC_F_BUSY		(1<<1)
 #define OC_F_PASS		(1<<2)
 #define OC_F_INCOMPLETE		(1<<3)
+#define OC_F_ABANDON		(1<<4)
 #define OC_F_PRIVATE		(1<<8)
 #define OC_F_FAILED		(1<<9)
 
diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index 84caa85..e4f8ea3 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -486,7 +486,7 @@ vbf_fetch_body_helper(struct busyobj *bo)
 		est = 0;
 
 	do {
-		if (bo->abandon) {
+		if (vfc->oc->flags & OC_F_ABANDON) {
 			/*
 			 * A pass object and delivery was terminated
 			 * We don't fail the fetch, in order for hit-for-pass
diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index 3f2d9d8..039c694 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -653,6 +653,24 @@ HSH_Complete(struct objcore *oc)
 }
 
 /*---------------------------------------------------------------------
+ * Abandon a fetch we will not need
+ */
+
+void
+HSH_Abandon(struct objcore *oc)
+{
+	struct objhead *oh;
+
+	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
+	oh = oc->objhead;
+	CHECK_OBJ(oh, OBJHEAD_MAGIC);
+
+	Lck_Lock(&oh->mtx);
+	oc->flags |= OC_F_ABANDON;
+	Lck_Unlock(&oh->mtx);
+}
+
+/*---------------------------------------------------------------------
  * Unbusy an objcore when the object is completely fetched.
  */
 
@@ -724,7 +742,7 @@ HSH_RefBusy(const struct objcore *oc)
 	assert(oc->refcnt > 0);
 	boc = oc->boc;
 	CHECK_OBJ_ORNULL(boc, BOC_MAGIC);
-	if (boc != NULL && boc->state < BOS_FINISHED) 
+	if (boc != NULL && boc->state < BOS_FINISHED)
 		boc->refcount++;
 	else
 		boc = NULL;
diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index 115473a..b2ee4bb 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -208,8 +208,7 @@ cnt_deliver(struct worker *wrk, struct req *req)
 
 	if (req->objcore->flags & (OC_F_PRIVATE | OC_F_PASS)) {
 		if (boc != NULL) {
-			CHECK_OBJ_NOTNULL(boc->busyobj, BUSYOBJ_MAGIC);
-			boc->busyobj->abandon = 1;
+			HSH_Abandon(req->objcore);
 			ObjWaitState(boc, BOS_FINISHED);
 		}
 		ObjSlim(wrk, req->objcore);
diff --git a/bin/varnishd/hash/hash_slinger.h b/bin/varnishd/hash/hash_slinger.h
index 7001204..0451006 100644
--- a/bin/varnishd/hash/hash_slinger.h
+++ b/bin/varnishd/hash/hash_slinger.h
@@ -75,6 +75,7 @@ void HSH_config(const char *h_arg);
 struct boc *HSH_RefBusy(const struct objcore *);
 void HSH_DerefBusy(struct worker *wrk, struct objcore *);
 struct objcore *HSH_Private(struct worker *wrk, int wantboc);
+void HSH_Abandon(struct objcore *oc);
 
 #ifdef VARNISH_CACHE_CHILD
 
diff --git a/bin/varnishtest/tests/r01391.vtc b/bin/varnishtest/tests/r01391.vtc
index 3858558..9669e16 100644
--- a/bin/varnishtest/tests/r01391.vtc
+++ b/bin/varnishtest/tests/r01391.vtc
@@ -15,9 +15,7 @@ server s1 {
 	chunkedlen 0
 } -start
 
-varnish v1 -vcl+backend {
-} -start
-
+varnish v1 -vcl+backend { } -start
 
 client c1 {
 	txreq
diff --git a/include/tbl/bo_flags.h b/include/tbl/bo_flags.h
index fedde3f..06608e3 100644
--- a/include/tbl/bo_flags.h
+++ b/include/tbl/bo_flags.h
@@ -36,7 +36,6 @@ BO_FLAG(do_gunzip,	1, 1, "")
 BO_FLAG(do_stream,	1, 1, "")
 BO_FLAG(do_pass,	0, 0, "")
 BO_FLAG(uncacheable,	0, 0, "")
-BO_FLAG(abandon,	0, 0, "")
 BO_FLAG(is_gzip,	0, 0, "")
 BO_FLAG(is_gunzip,	0, 0, "")
 BO_FLAG(was_304,	1, 0, "")



From phk at FreeBSD.org  Thu Jan 28 11:49:20 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 28 Jan 2016 12:49:20 +0100
Subject: [master] c95addc Simplify the handover to the fetch-thread: the
 req-side doesn't need the busyobj at all now.
Message-ID: <E1aOl4i-0006Zy-5g@project.varnish-software.com>


commit c95addc6dcd6bec3a477715f0aec09cc9784a56f
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu Jan 21 15:36:04 2016 +0000

    Simplify the handover to the fetch-thread:  the req-side doesn't
    need the busyobj at all now.

diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index e4f8ea3..8c49ee4 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -967,7 +967,8 @@ void
 VBF_Fetch(struct worker *wrk, struct req *req, struct objcore *oc,
     struct objcore *oldoc, enum vbf_fetch_mode_e mode)
 {
-	struct busyobj *bo, *bo_fetch;
+	struct boc *boc;
+	struct busyobj *bo;
 	const char *how;
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
@@ -994,9 +995,9 @@ VBF_Fetch(struct worker *wrk, struct req *req, struct objcore *oc,
 
 	THR_SetBusyobj(bo);
 
-	assert(HSH_RefBusy(oc) == bo->boc);
-	bo_fetch = bo;
-
+	boc = HSH_RefBusy(oc);
+	AN(boc);
+	assert(boc == bo->boc);
 
 	AN(bo->vcl);
 
@@ -1021,7 +1022,7 @@ VBF_Fetch(struct worker *wrk, struct req *req, struct objcore *oc,
 	AZ(bo->req);
 	bo->req = req;
 
-	bo->fetch_task.priv = bo_fetch;
+	bo->fetch_task.priv = bo;
 	bo->fetch_task.func = vbf_fetch_thread;
 
 	if (Pool_Task(wrk->pool, &bo->fetch_task, TASK_QUEUE_BO)) {
@@ -1029,22 +1030,23 @@ VBF_Fetch(struct worker *wrk, struct req *req, struct objcore *oc,
 		(void)vbf_stp_fail(req->wrk, bo);
 		if (bo->stale_oc != NULL)
 			(void)HSH_DerefObjCore(wrk, &bo->stale_oc);
-		HSH_DerefBusy(wrk, bo->fetch_objcore);
+		HSH_DerefBusy(wrk, oc);
 	} else {
-		bo_fetch = NULL; /* ref transferred to fetch thread */
+		bo = NULL; /* ref transferred to fetch thread */
 		if (mode == VBF_BACKGROUND) {
-			ObjWaitState(bo->boc, BOS_REQ_DONE);
+			ObjWaitState(boc, BOS_REQ_DONE);
 		} else {
-			ObjWaitState(bo->boc, BOS_STREAM);
-			if (bo->boc->state == BOS_FAILED) {
+			ObjWaitState(boc, BOS_STREAM);
+			if (boc->state == BOS_FAILED) {
 				AN((oc->flags & OC_F_FAILED));
 			} else {
-				AZ(bo->fetch_objcore->flags & OC_F_BUSY);
+				AZ(oc->flags & OC_F_BUSY);
 			}
 		}
 	}
-	AZ(bo_fetch);
+	AZ(bo);
 	VSLb_ts_req(req, "Fetch", W_TIM_real(wrk));
-	HSH_DerefBusy(wrk, bo->fetch_objcore);
+	assert(oc->boc == boc);
+	HSH_DerefBusy(wrk, oc);
 	THR_SetBusyobj(NULL);
 }



From phk at FreeBSD.org  Thu Jan 28 11:49:20 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 28 Jan 2016 12:49:20 +0100
Subject: [master] c8e7c20 Call ObjStable() when the boc goes away.
Message-ID: <E1aOl4i-0006a7-73@project.varnish-software.com>


commit c8e7c20e136073c7acedc2a61b8a13f1388ec74a
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu Jan 21 20:24:34 2016 +0000

    Call ObjStable() when the boc goes away.

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index 564023e..442fe48 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -870,7 +870,7 @@ void *ObjSetattr(struct worker *, struct objcore *, enum obj_attr attr,
     ssize_t len, const void *);
 int ObjCopyAttr(struct worker *, struct objcore *, struct objcore *,
     enum obj_attr attr);
-void ObjStable(struct worker *, struct objcore *, struct busyobj *);
+void ObjStable(struct worker *, struct objcore *, struct boc *);
 
 int ObjSetDouble(struct worker *, struct objcore *, enum obj_attr, double);
 int ObjSetU32(struct worker *, struct objcore *, enum obj_attr, uint32_t);
diff --git a/bin/varnishd/cache/cache_busyobj.c b/bin/varnishd/cache/cache_busyobj.c
index 110c44f..fbe6046 100644
--- a/bin/varnishd/cache/cache_busyobj.c
+++ b/bin/varnishd/cache/cache_busyobj.c
@@ -177,8 +177,6 @@ VBO_DerefBusyObj(struct worker *wrk, struct busyobj **pbo)
 
 	if (bo->fetch_objcore != NULL) {
 		AN(wrk);
-		if (bo->fetch_objcore->stobj->stevedore != NULL)
-			ObjStable(bo->wrk, bo->fetch_objcore, bo);
 		(void)HSH_DerefObjCore(wrk, &bo->fetch_objcore);
 	}
 
diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index 039c694..5057f59 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -768,6 +768,8 @@ HSH_DerefBusy(struct worker *wrk, struct objcore *oc)
 		oc->boc = NULL;
 	Lck_Unlock(&oc->objhead->mtx);
 	if (r == 0) {
+		if (oc->stobj->stevedore != NULL)
+			ObjStable(wrk, oc, boc);
 		VBO_DerefBusyObj(wrk, &boc->busyobj);
 		AZ(pthread_cond_destroy(&boc->cond));
 		Lck_Delete(&boc->mtx);
diff --git a/bin/varnishd/cache/cache_obj.c b/bin/varnishd/cache/cache_obj.c
index 9bb6071..018e280 100644
--- a/bin/varnishd/cache/cache_obj.c
+++ b/bin/varnishd/cache/cache_obj.c
@@ -256,19 +256,19 @@ ObjUpdateMeta(struct worker *wrk, struct objcore *oc)
 }
 
 /*====================================================================
- * Called when the busyobj used to populate the objcore is going away.
+ * Called when the boc used to populate the objcore is going away.
  * Useful for releasing any leftovers from Trim.
  */
 
 void
-ObjStable(struct worker *wrk, struct objcore *oc, struct busyobj *bo)
+ObjStable(struct worker *wrk, struct objcore *oc, struct boc *boc)
 {
 	const struct obj_methods *m = obj_getmethods(oc);
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
-	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
+	CHECK_OBJ_NOTNULL(boc, BOC_MAGIC);
 	if (m->objstable != NULL)
-		m->objstable(wrk, oc, bo);
+		m->objstable(wrk, oc, boc);
 }
 
 /*====================================================================
diff --git a/bin/varnishd/cache/cache_obj.h b/bin/varnishd/cache/cache_obj.h
index 0058434..3fba1ef 100644
--- a/bin/varnishd/cache/cache_obj.h
+++ b/bin/varnishd/cache/cache_obj.h
@@ -43,7 +43,7 @@ typedef int objgetspace_f(struct worker *, struct objcore *,
      ssize_t *sz, uint8_t **ptr);
 typedef void objextend_f(struct worker *, struct objcore *, ssize_t l);
 typedef void objtrimstore_f(struct worker *, struct objcore *);
-typedef void objstable_f(struct worker *, struct objcore *, struct busyobj *);
+typedef void objstable_f(struct worker *, struct objcore *, struct boc *);
 typedef void objslim_f(struct worker *, struct objcore *);
 typedef void *objgetattr_f(struct worker *, struct objcore *,
     enum obj_attr attr, ssize_t *len);
diff --git a/bin/varnishd/storage/storage_simple.c b/bin/varnishd/storage/storage_simple.c
index 8d313fb..79440d2 100644
--- a/bin/varnishd/storage/storage_simple.c
+++ b/bin/varnishd/storage/storage_simple.c
@@ -484,19 +484,19 @@ sml_trimstore(struct worker *wrk, struct objcore *oc)
 }
 
 static void __match_proto__(objstable_f)
-sml_stable(struct worker *wrk, struct objcore *oc, struct busyobj *bo)
+sml_stable(struct worker *wrk, struct objcore *oc, struct boc *boc)
 {
 	const struct stevedore *stv;
 	struct storage *st;
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
-	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
+	CHECK_OBJ_NOTNULL(boc, BOC_MAGIC);
 
-	if (bo->boc->stevedore_priv == NULL)
+	if (boc->stevedore_priv == NULL)
 		return;
-	CAST_OBJ_NOTNULL(st, bo->boc->stevedore_priv, STORAGE_MAGIC);
-	bo->boc->stevedore_priv = 0;
+	CAST_OBJ_NOTNULL(st, boc->stevedore_priv, STORAGE_MAGIC);
+	boc->stevedore_priv = 0;
 	CHECK_OBJ_NOTNULL(st, STORAGE_MAGIC);
 	stv = oc->stobj->stevedore;
 	CHECK_OBJ_NOTNULL(stv, STEVEDORE_MAGIC);



From phk at FreeBSD.org  Thu Jan 28 11:49:20 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 28 Jan 2016 12:49:20 +0100
Subject: [master] 38a635c Use objcore argument for setting and waiting for
 (b)oc state
Message-ID: <E1aOl4i-0006aH-8f@project.varnish-software.com>


commit 38a635c4b2056262385cb6edb09f611186672b86
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu Jan 21 20:43:14 2016 +0000

    Use objcore argument for setting and waiting for (b)oc state

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index 442fe48..6f4f890 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -854,8 +854,8 @@ int ObjIterate(struct worker *, struct objcore *,
 int ObjGetSpace(struct worker *, struct objcore *, ssize_t *sz, uint8_t **ptr);
 void ObjExtend(struct worker *, struct objcore *, struct boc *, ssize_t l);
 ssize_t ObjWaitExtend(struct worker *, struct objcore *, struct boc *, ssize_t l);
-void ObjSetState(struct boc *, enum boc_state_e next);
-void ObjWaitState(struct boc *, enum boc_state_e want);
+void ObjSetState(const struct objcore *, enum boc_state_e next);
+void ObjWaitState(const struct objcore *, enum boc_state_e want);
 void ObjTrimStore(struct worker *, struct objcore *);
 void ObjTouch(struct worker *wrk, struct objcore *oc, double now);
 unsigned ObjGetXID(struct worker *, struct objcore *);
diff --git a/bin/varnishd/cache/cache_esi_deliver.c b/bin/varnishd/cache/cache_esi_deliver.c
index a9a8348..703109b 100644
--- a/bin/varnishd/cache/cache_esi_deliver.c
+++ b/bin/varnishd/cache/cache_esi_deliver.c
@@ -668,7 +668,7 @@ ved_objiterate(void *priv, int flush, const void *ptr, ssize_t len)
 }
 
 static void
-ved_stripgzip(struct req *req, struct boc *boc)
+ved_stripgzip(struct req *req, const struct boc *boc)
 {
 	ssize_t l;
 	char *p;
@@ -689,7 +689,7 @@ ved_stripgzip(struct req *req, struct boc *boc)
 
 	/* OA_GZIPBITS is not valid until BOS_FINISHED */
 	if (boc != NULL)
-		ObjWaitState(boc, BOS_FINISHED);
+		ObjWaitState(req->objcore, BOS_FINISHED);
 
 	AN(ObjCheckFlag(req->wrk, req->objcore, OF_GZIPED));
 
diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index 8c49ee4..df86a5a 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -202,7 +202,7 @@ vbf_stp_mkbereq(const struct worker *wrk, struct busyobj *bo)
 	bo->ws_bo = WS_Snapshot(bo->ws);
 	HTTP_Copy(bo->bereq, bo->bereq0);
 
-	ObjSetState(bo->boc, BOS_REQ_DONE);
+	ObjSetState(bo->fetch_objcore, BOS_REQ_DONE);
 	return (F_STP_STARTFETCH);
 }
 
@@ -648,7 +648,7 @@ vbf_stp_fetch(struct worker *wrk, struct busyobj *bo)
 
 	if (bo->do_stream) {
 		HSH_Unbusy(wrk, bo->fetch_objcore);
-		ObjSetState(bo->boc, BOS_STREAM);
+		ObjSetState(bo->fetch_objcore, BOS_STREAM);
 	}
 
 	VSLb(bo->vsl, SLT_Fetch_Body, "%u %s %s",
@@ -682,7 +682,7 @@ vbf_stp_fetch(struct worker *wrk, struct busyobj *bo)
 	   give predictable backend reuse behavior for varnishtest */
 	VDI_Finish(bo->wrk, bo);
 
-	ObjSetState(bo->boc, BOS_FINISHED);
+	ObjSetState(bo->fetch_objcore, BOS_FINISHED);
 	VSLb_ts_busyobj(bo, "BerespBody", W_TIM_real(wrk));
 	if (bo->stale_oc != NULL)
 		EXP_Rearm(bo->stale_oc, bo->stale_oc->exp.t_origin, 0, 0, 0);
@@ -736,7 +736,7 @@ vbf_stp_condfetch(struct worker *wrk, struct busyobj *bo)
 
 	if (bo->do_stream) {
 		HSH_Unbusy(wrk, bo->fetch_objcore);
-		ObjSetState(bo->boc, BOS_STREAM);
+		ObjSetState(bo->fetch_objcore, BOS_STREAM);
 	}
 
 	if (ObjIterate(wrk, bo->stale_oc, bo, vbf_objiterator))
@@ -758,7 +758,7 @@ vbf_stp_condfetch(struct worker *wrk, struct busyobj *bo)
 	   give predictable backend reuse behavior for varnishtest */
 	VDI_Finish(bo->wrk, bo);
 
-	ObjSetState(bo->boc, BOS_FINISHED);
+	ObjSetState(bo->fetch_objcore, BOS_FINISHED);
 	VSLb_ts_busyobj(bo, "BerespBody", W_TIM_real(wrk));
 	return (F_STP_DONE);
 }
@@ -858,7 +858,7 @@ vbf_stp_error(struct worker *wrk, struct busyobj *bo)
 	VSB_delete(synth_body);
 
 	HSH_Unbusy(wrk, bo->fetch_objcore);
-	ObjSetState(bo->boc, BOS_FINISHED);
+	ObjSetState(bo->fetch_objcore, BOS_FINISHED);
 	return (F_STP_DONE);
 }
 
@@ -881,7 +881,7 @@ vbf_stp_fail(struct worker *wrk, const struct busyobj *bo)
 		    bo->fetch_objcore->exp.t_origin, 0, 0, 0);
 	}
 	wrk->stats->fetch_failed++;
-	ObjSetState(bo->boc, BOS_FAILED);
+	ObjSetState(bo->fetch_objcore, BOS_FAILED);
 	return (F_STP_DONE);
 }
 
@@ -1034,10 +1034,10 @@ VBF_Fetch(struct worker *wrk, struct req *req, struct objcore *oc,
 	} else {
 		bo = NULL; /* ref transferred to fetch thread */
 		if (mode == VBF_BACKGROUND) {
-			ObjWaitState(boc, BOS_REQ_DONE);
+			ObjWaitState(oc, BOS_REQ_DONE);
 		} else {
-			ObjWaitState(boc, BOS_STREAM);
-			if (boc->state == BOS_FAILED) {
+			ObjWaitState(oc, BOS_STREAM);
+			if (oc->boc->state == BOS_FAILED) {
 				AN((oc->flags & OC_F_FAILED));
 			} else {
 				AZ(oc->flags & OC_F_BUSY);
diff --git a/bin/varnishd/cache/cache_obj.c b/bin/varnishd/cache/cache_obj.c
index 018e280..986254f 100644
--- a/bin/varnishd/cache/cache_obj.c
+++ b/bin/varnishd/cache/cache_obj.c
@@ -155,35 +155,36 @@ ObjWaitExtend(struct worker *wrk, struct objcore *oc, struct boc *boc,
  */
 
 void
-ObjSetState(struct boc *boc, enum boc_state_e next)
+ObjSetState(const struct objcore *oc, enum boc_state_e next)
 {
 
-	CHECK_OBJ_NOTNULL(boc, BOC_MAGIC);
+	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
+	CHECK_OBJ_NOTNULL(oc->boc, BOC_MAGIC);
 
-	// assert(bo->do_stream || next != BOS_STREAM);
-	assert(next > boc->state);
-	Lck_Lock(&boc->mtx);
-	boc->state = next;
-	AZ(pthread_cond_broadcast(&boc->cond));
-	Lck_Unlock(&boc->mtx);
+	assert(next > oc->boc->state);
+	Lck_Lock(&oc->boc->mtx);
+	oc->boc->state = next;
+	AZ(pthread_cond_broadcast(&oc->boc->cond));
+	Lck_Unlock(&oc->boc->mtx);
 }
 
 /*====================================================================
  */
 
 void
-ObjWaitState(struct boc *boc, enum boc_state_e want)
+ObjWaitState(const struct objcore *oc, enum boc_state_e want)
 {
 
-	CHECK_OBJ_NOTNULL(boc, BOC_MAGIC);
+	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
+	CHECK_OBJ_NOTNULL(oc->boc, BOC_MAGIC);
 
-	Lck_Lock(&boc->mtx);
+	Lck_Lock(&oc->boc->mtx);
 	while (1) {
-		if (boc->state >= want)
+		if (oc->boc->state >= want)
 			break;
-		(void)Lck_CondWait(&boc->cond, &boc->mtx, 0);
+		(void)Lck_CondWait(&oc->boc->cond, &oc->boc->mtx, 0);
 	}
-	Lck_Unlock(&boc->mtx);
+	Lck_Unlock(&oc->boc->mtx);
 }
 
 /*====================================================================
diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index b2ee4bb..8686634 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -209,7 +209,7 @@ cnt_deliver(struct worker *wrk, struct req *req)
 	if (req->objcore->flags & (OC_F_PRIVATE | OC_F_PASS)) {
 		if (boc != NULL) {
 			HSH_Abandon(req->objcore);
-			ObjWaitState(boc, BOS_FINISHED);
+			ObjWaitState(req->objcore, BOS_FINISHED);
 		}
 		ObjSlim(wrk, req->objcore);
 	}



From phk at FreeBSD.org  Thu Jan 28 11:49:20 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 28 Jan 2016 12:49:20 +0100
Subject: [master] 9011096 Eliminate unneeded boc argument
Message-ID: <E1aOl4i-0006aR-Av@project.varnish-software.com>


commit 9011096fdb7ade3d4103909e7f5c42585931b81b
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu Jan 21 20:56:40 2016 +0000

    Eliminate unneeded boc argument

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index 6f4f890..7cc4c5c 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -852,7 +852,7 @@ typedef int objiterate_f(void *priv, int flush, const void *ptr, ssize_t len);
 int ObjIterate(struct worker *, struct objcore *,
     void *priv, objiterate_f *func);
 int ObjGetSpace(struct worker *, struct objcore *, ssize_t *sz, uint8_t **ptr);
-void ObjExtend(struct worker *, struct objcore *, struct boc *, ssize_t l);
+void ObjExtend(struct worker *, struct objcore *, ssize_t l);
 ssize_t ObjWaitExtend(struct worker *, struct objcore *, struct boc *, ssize_t l);
 void ObjSetState(const struct objcore *, enum boc_state_e next);
 void ObjWaitState(const struct objcore *, enum boc_state_e want);
diff --git a/bin/varnishd/cache/cache_fetch_proc.c b/bin/varnishd/cache/cache_fetch_proc.c
index f017f5d..cb0e1d1 100644
--- a/bin/varnishd/cache/cache_fetch_proc.c
+++ b/bin/varnishd/cache/cache_fetch_proc.c
@@ -98,7 +98,7 @@ VFP_Extend(const struct vfp_ctx *vc, ssize_t sz)
 {
 	CHECK_OBJ_NOTNULL(vc, VFP_CTX_MAGIC);
 
-	ObjExtend(vc->wrk, vc->oc, vc->bo->boc, sz);
+	ObjExtend(vc->wrk, vc->oc, sz);
 }
 
 /**********************************************************************
diff --git a/bin/varnishd/cache/cache_obj.c b/bin/varnishd/cache/cache_obj.c
index 986254f..00fc45d 100644
--- a/bin/varnishd/cache/cache_obj.c
+++ b/bin/varnishd/cache/cache_obj.c
@@ -107,20 +107,20 @@ ObjGetSpace(struct worker *wrk, struct objcore *oc, ssize_t *sz, uint8_t **ptr)
  */
 
 void
-ObjExtend(struct worker *wrk, struct objcore *oc, struct boc *boc, ssize_t l)
+ObjExtend(struct worker *wrk, struct objcore *oc, ssize_t l)
 {
 	const struct obj_methods *om = obj_getmethods(oc);
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
+	CHECK_OBJ_ORNULL(oc->boc, BOC_MAGIC);
 	AN(om->objextend);
 	assert(l > 0);
 
-	if (boc != NULL) {
-		CHECK_OBJ_NOTNULL(boc, BOC_MAGIC);
-		Lck_Lock(&boc->mtx);
+	if (oc->boc != NULL) {
+		Lck_Lock(&oc->boc->mtx);
 		om->objextend(wrk, oc, l);
-		AZ(pthread_cond_broadcast(&boc->cond));
-		Lck_Unlock(&boc->mtx);
+		AZ(pthread_cond_broadcast(&oc->boc->cond));
+		Lck_Unlock(&oc->boc->mtx);
 	} else {
 		om->objextend(wrk, oc, l);
 	}
diff --git a/bin/varnishd/cache/cache_req_body.c b/bin/varnishd/cache/cache_req_body.c
index dbdf15d..c3fd142 100644
--- a/bin/varnishd/cache/cache_req_body.c
+++ b/bin/varnishd/cache/cache_req_body.c
@@ -254,7 +254,7 @@ VRB_Cache(struct req *req, ssize_t maxsize)
 			req->acct.req_bodybytes += l;
 			if (yet >= l)
 				yet -= l;
-			ObjExtend(req->wrk, req->body_oc, NULL, l);
+			ObjExtend(req->wrk, req->body_oc, l);
 		}
 
 	} while (vfps == VFP_OK);
diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index 8686634..1ba9642 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -291,7 +291,7 @@ cnt_synth(struct worker *wrk, struct req *req)
 		sz = szl;
 		if (ObjGetSpace(wrk, req->objcore, &sz, &ptr) && sz >= szl) {
 			memcpy(ptr, VSB_data(synth_body), szl);
-			ObjExtend(wrk, req->objcore, NULL, szl);
+			ObjExtend(wrk, req->objcore, szl);
 		} else
 			szl = -1;
 	}



From phk at FreeBSD.org  Thu Jan 28 11:49:20 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 28 Jan 2016 12:49:20 +0100
Subject: [master] 8c5bf3f Get to the boc through the fetch_objcore
Message-ID: <E1aOl4i-0006ac-Df@project.varnish-software.com>


commit 8c5bf3ff7ee0729691b274d612d6791fb0e14c7f
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu Jan 21 21:05:47 2016 +0000

    Get to the boc through the fetch_objcore

diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index df86a5a..75a4d38 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -168,7 +168,7 @@ vbf_stp_mkbereq(const struct worker *wrk, struct busyobj *bo)
 	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
 	CHECK_OBJ_NOTNULL(bo->req, REQ_MAGIC);
 
-	assert(bo->boc->state == BOS_INVALID);
+	assert(bo->fetch_objcore->boc->state == BOS_INVALID);
 	AZ(bo->storage_hint);
 
 	HTTP_Setup(bo->bereq0, bo->ws, bo->vsl, SLT_BereqMethod);
@@ -221,7 +221,7 @@ vbf_stp_retry(struct worker *wrk, struct busyobj *bo)
 	vfc = bo->vfc;
 	CHECK_OBJ_NOTNULL(vfc, VFP_CTX_MAGIC);
 
-	assert(bo->boc->state == BOS_REQ_DONE);
+	assert(bo->fetch_objcore->boc->state == BOS_REQ_DONE);
 
 	VSLb_ts_busyobj(bo, "Retry", W_TIM_real(wrk));
 
@@ -281,7 +281,7 @@ vbf_stp_startfetch(struct worker *wrk, struct busyobj *bo)
 
 	HTTP_Setup(bo->beresp, bo->ws, bo->vsl, SLT_BerespMethod);
 
-	assert(bo->boc->state <= BOS_REQ_DONE);
+	assert(bo->fetch_objcore->boc->state <= BOS_REQ_DONE);
 
 	AZ(bo->htc);
 	i = VDI_GetHdr(wrk, bo);
@@ -451,7 +451,7 @@ vbf_stp_startfetch(struct worker *wrk, struct busyobj *bo)
 		return (F_STP_ERROR);
 	}
 
-	assert(bo->boc->state == BOS_REQ_DONE);
+	assert(bo->fetch_objcore->boc->state == BOS_REQ_DONE);
 
 	if (bo->do_esi)
 		bo->do_stream = 0;
@@ -642,9 +642,9 @@ vbf_stp_fetch(struct worker *wrk, struct busyobj *bo)
 	if (bo->htc->body_status != BS_NONE)
 		AZ(VDI_GetBody(bo->wrk, bo));
 
-	assert(bo->boc->refcount >= 1);
+	assert(bo->fetch_objcore->boc->refcount >= 1);
 
-	assert(bo->boc->state == BOS_REQ_DONE);
+	assert(bo->fetch_objcore->boc->state == BOS_REQ_DONE);
 
 	if (bo->do_stream) {
 		HSH_Unbusy(wrk, bo->fetch_objcore);
@@ -663,7 +663,7 @@ vbf_stp_fetch(struct worker *wrk, struct busyobj *bo)
 	if (bo->vfc->failed) {
 		VDI_Finish(bo->wrk, bo);
 		if (!bo->do_stream) {
-			assert(bo->boc->state < BOS_STREAM);
+			assert(bo->fetch_objcore->boc->state < BOS_STREAM);
 			// XXX: doclose = ?
 			return (F_STP_ERROR);
 		} else {
@@ -672,9 +672,9 @@ vbf_stp_fetch(struct worker *wrk, struct busyobj *bo)
 	}
 
 	if (bo->do_stream)
-		assert(bo->boc->state == BOS_STREAM);
+		assert(bo->fetch_objcore->boc->state == BOS_STREAM);
 	else {
-		assert(bo->boc->state == BOS_REQ_DONE);
+		assert(bo->fetch_objcore->boc->state == BOS_REQ_DONE);
 		HSH_Unbusy(wrk, bo->fetch_objcore);
 	}
 
@@ -872,7 +872,7 @@ vbf_stp_fail(struct worker *wrk, const struct busyobj *bo)
 	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
 	CHECK_OBJ_NOTNULL(bo->fetch_objcore, OBJCORE_MAGIC);
 
-	assert(bo->boc->state < BOS_FINISHED);
+	assert(bo->fetch_objcore->boc->state < BOS_FINISHED);
 	HSH_Fail(bo->fetch_objcore);
 	if (bo->fetch_objcore->exp_flags & OC_EF_EXP) {
 		/* Already unbusied - expire it */
@@ -926,7 +926,7 @@ vbf_fetch_thread(struct worker *wrk, void *priv)
 
 	while (stp != F_STP_DONE) {
 		CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
-		assert(bo->boc->refcount >= 1);
+		assert(bo->fetch_objcore->boc->refcount >= 1);
 		switch(stp) {
 #define FETCH_STEP(l, U, arg)						\
 		case F_STP_##U:						\
@@ -944,7 +944,7 @@ vbf_fetch_thread(struct worker *wrk, void *priv)
 	http_Teardown(bo->bereq);
 	http_Teardown(bo->beresp);
 
-	if (bo->boc->state == BOS_FINISHED) {
+	if (bo->fetch_objcore->boc->state == BOS_FINISHED) {
 		AZ(bo->fetch_objcore->flags & OC_F_FAILED);
 		HSH_Complete(bo->fetch_objcore);
 		VSLb(bo->vsl, SLT_Length, "%ju",
@@ -1004,7 +1004,7 @@ VBF_Fetch(struct worker *wrk, struct req *req, struct objcore *oc,
 	if (mode == VBF_PASS)
 		bo->do_pass = 1;
 
-	bo->boc->vary = req->vary_b;
+	oc->boc->vary = req->vary_b;
 	req->vary_b = NULL;
 
 	if (mode != VBF_BACKGROUND)



From phk at FreeBSD.org  Thu Jan 28 11:49:20 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 28 Jan 2016 12:49:20 +0100
Subject: [master] 413a772 Remove the direct pointer from boc to busyobj
Message-ID: <E1aOl4i-0006al-Ft@project.varnish-software.com>


commit 413a7720a6ef80dde4f038d861d24329e325a6e2
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu Jan 21 21:46:29 2016 +0000

    Remove the direct pointer from boc to busyobj
    
    Dump boc's in panic output

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index 7cc4c5c..4d58308 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -398,8 +398,6 @@ struct boc {
 	void			*stevedore_priv;
 	enum boc_state_e	state;
 	uint8_t			*vary;
-
-	struct busyobj		*busyobj;
 };
 
 /* Object core structure ---------------------------------------------
diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index 75a4d38..05b5482 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -957,6 +957,7 @@ vbf_fetch_thread(struct worker *wrk, void *priv)
 
 	wrk->vsl = NULL;
 	HSH_DerefBusy(wrk, bo->fetch_objcore);
+	VBO_DerefBusyObj(wrk, &bo);
 	THR_SetBusyobj(NULL);
 }
 
@@ -987,7 +988,6 @@ VBF_Fetch(struct worker *wrk, struct req *req, struct objcore *oc,
 
 	bo = VBO_GetBusyObj(wrk, req);
 	bo->boc = oc->boc;
-	bo->boc->busyobj = bo;
 
 	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
 	VSLb(bo->vsl, SLT_Begin, "bereq %u %s", VXID(req->vsl->wid), how);
diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index 5057f59..7d22c25 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -760,7 +760,6 @@ HSH_DerefBusy(struct worker *wrk, struct objcore *oc)
 	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
 	boc = oc->boc;
 	CHECK_OBJ_NOTNULL(boc, BOC_MAGIC);
-	CHECK_OBJ_NOTNULL(boc->busyobj, BUSYOBJ_MAGIC);
 	Lck_Lock(&oc->objhead->mtx);
 	assert(boc->refcount > 0);
 	r = --boc->refcount;
@@ -770,7 +769,6 @@ HSH_DerefBusy(struct worker *wrk, struct objcore *oc)
 	if (r == 0) {
 		if (oc->stobj->stevedore != NULL)
 			ObjStable(wrk, oc, boc);
-		VBO_DerefBusyObj(wrk, &boc->busyobj);
 		AZ(pthread_cond_destroy(&boc->cond));
 		Lck_Delete(&boc->mtx);
 		free(boc->vary);
diff --git a/bin/varnishd/cache/cache_panic.c b/bin/varnishd/cache/cache_panic.c
index 7d8f47b..54789ac 100644
--- a/bin/varnishd/cache/cache_panic.c
+++ b/bin/varnishd/cache/cache_panic.c
@@ -188,6 +188,20 @@ pan_http(struct vsb *vsb, const char *id, const struct http *h)
 }
 
 /*--------------------------------------------------------------------*/
+static void
+pan_boc(struct vsb *vsb, const struct boc *boc)
+{
+	VSB_printf(vsb, "boc = %p {\n", boc);
+	VSB_indent(vsb, 2);
+	VSB_printf(vsb, "refcnt = %u,\n", boc->refcount);
+	VSB_printf(vsb, "state = %d,\n", boc->state);
+	VSB_printf(vsb, "vary = %p,\n", boc->vary);
+	VSB_printf(vsb, "stevedore_priv = %p,\n", boc->stevedore_priv);
+	VSB_indent(vsb, -2);
+	VSB_printf(vsb, "},\n");
+}
+
+/*--------------------------------------------------------------------*/
 
 static void
 pan_objcore(struct vsb *vsb, const char *typ, const struct objcore *oc)
@@ -198,6 +212,8 @@ pan_objcore(struct vsb *vsb, const char *typ, const struct objcore *oc)
 	VSB_printf(vsb, "refcnt = %d,\n", oc->refcnt);
 	VSB_printf(vsb, "flags = 0x%x,\n", oc->flags);
 	VSB_printf(vsb, "exp_flags = 0x%x,\n", oc->exp_flags);
+	if (oc->boc != NULL)
+		pan_boc(vsb, oc->boc);
 	VSB_printf(vsb, "exp = { %f, %f, %f, %f }\n",
 	    oc->exp.t_origin, oc->exp.ttl, oc->exp.grace, oc->exp.keep);
 	VSB_printf(vsb, "objhead = %p,\n", oc->objhead);
@@ -315,12 +331,6 @@ pan_busyobj(struct vsb *vsb, const struct busyobj *bo)
 	VSB_printf(vsb, "},\n");
 }
 
-static void
-pan_boc(struct vsb *vsb, const struct boc *boc)
-{
-	pan_busyobj(vsb, boc->busyobj);
-}
-
 /*--------------------------------------------------------------------*/
 
 static void
@@ -370,11 +380,8 @@ pan_req(struct vsb *vsb, const struct req *req)
 
 	VCL_Panic(vsb, req->vcl);
 
-	if (req->objcore != NULL) {
+	if (req->objcore != NULL)
 		pan_objcore(vsb, "REQ", req->objcore);
-		if (req->objcore->boc != NULL)
-			pan_boc(vsb, req->objcore->boc);
-	}
 
 	VSB_printf(vsb, "flags = {\n");
 	VSB_indent(vsb, 2);



From phk at FreeBSD.org  Thu Jan 28 11:49:20 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 28 Jan 2016 12:49:20 +0100
Subject: [master] 4071b64 Add a mechanism to avoid dumping the same structure
 more than once.
Message-ID: <E1aOl4i-0006av-HI@project.varnish-software.com>


commit 4071b64acad3965ad9d2b26d3f48f73309e8401f
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu Jan 21 22:13:50 2016 +0000

    Add a mechanism to avoid dumping the same structure more than once.

diff --git a/bin/varnishd/cache/cache_panic.c b/bin/varnishd/cache/cache_panic.c
index 54789ac..47421fe 100644
--- a/bin/varnishd/cache/cache_panic.c
+++ b/bin/varnishd/cache/cache_panic.c
@@ -109,11 +109,37 @@ sess_close_2str(enum sess_close sc, int want_desc)
 
 /*--------------------------------------------------------------------*/
 
+#define N_ALREADY 64
+static const void *already_list[N_ALREADY];
+static int already_idx;
+
+static int
+pan_already(struct vsb *vsb, const void *ptr)
+{
+	int i;
+
+	for (i = 0; i < already_idx; i++) {
+		if (already_list[i] == ptr) {
+			VSB_printf(vsb, "  [Already dumped, see above]\n");
+			VSB_printf(vsb, "},\n");
+			return (1);
+		}
+	}
+	if (already_idx < N_ALREADY)
+		already_list[already_idx++] = ptr;
+	return (0);
+}
+
+
+/*--------------------------------------------------------------------*/
+
 static void
 pan_ws(struct vsb *vsb, const struct ws *ws)
 {
 
 	VSB_printf(vsb, "ws = %p {\n", ws);
+	if (pan_already(vsb, ws))
+		return;
 	VSB_indent(vsb, 2);
 	if (WS_Overflowed(ws))
 		VSB_printf(vsb, "OVERFLOW ");
@@ -143,6 +169,8 @@ pan_htc(struct vsb *vsb, const struct http_conn *htc)
 {
 
 	VSB_printf(vsb, "http_conn = %p {\n", htc);
+	if (pan_already(vsb, htc))
+		return;
 	VSB_indent(vsb, 2);
 	VSB_printf(vsb, "fd = %d,\n", htc->fd);
 	VSB_printf(vsb, "doclose = %s,\n", sess_close_2str(htc->doclose, 0));
@@ -171,6 +199,8 @@ pan_http(struct vsb *vsb, const char *id, const struct http *h)
 	int i;
 
 	VSB_printf(vsb, "http[%s] = %p {\n", id, h);
+	if (pan_already(vsb, h))
+		return;
 	VSB_indent(vsb, 2);
 	VSB_printf(vsb, "ws[%s] = %p,\n", h->ws ? h->ws->id : "", h->ws);
 	VSB_printf(vsb, "hdrs {\n");
@@ -192,6 +222,8 @@ static void
 pan_boc(struct vsb *vsb, const struct boc *boc)
 {
 	VSB_printf(vsb, "boc = %p {\n", boc);
+	if (pan_already(vsb, boc))
+		return;
 	VSB_indent(vsb, 2);
 	VSB_printf(vsb, "refcnt = %u,\n", boc->refcount);
 	VSB_printf(vsb, "state = %d,\n", boc->state);
@@ -208,6 +240,8 @@ pan_objcore(struct vsb *vsb, const char *typ, const struct objcore *oc)
 {
 
 	VSB_printf(vsb, "objcore[%s] = %p {\n", typ, oc);
+	if (pan_already(vsb, oc))
+		return;
 	VSB_indent(vsb, 2);
 	VSB_printf(vsb, "refcnt = %d,\n", oc->refcnt);
 	VSB_printf(vsb, "flags = 0x%x,\n", oc->flags);
@@ -239,6 +273,8 @@ pan_wrk(struct vsb *vsb, const struct worker *wrk)
 	const char *p;
 
 	VSB_printf(vsb, "worker = %p {\n", wrk);
+	if (pan_already(vsb, wrk))
+		return;
 	VSB_indent(vsb, 2);
 	VSB_printf(vsb, "stack = {0x%jx -> 0x%jx},\n",
 	    (uintmax_t)wrk->stack_start, (uintmax_t)wrk->stack_end);
@@ -286,12 +322,12 @@ pan_busyobj(struct vsb *vsb, const struct busyobj *bo)
 	const char *p;
 
 	VSB_printf(vsb, "busyobj = %p {\n", bo);
+	if (pan_already(vsb, bo))
+		return;
 	VSB_indent(vsb, 2);
 	pan_ws(vsb, bo->ws);
-	VSB_printf(vsb, "refcnt = %u,\n", bo->boc->refcount);
 	VSB_printf(vsb, "retries = %d, ", bo->retries);
 	VSB_printf(vsb, "failed = %d, ", bo->vfc->failed);
-	VSB_printf(vsb, "state = %d,\n", (int)bo->boc->state);
 	VSB_printf(vsb, "flags = {");
 	p = "";
 	/*lint -save -esym(438,p) */
@@ -339,6 +375,8 @@ pan_req(struct vsb *vsb, const struct req *req)
 	const char *stp;
 
 	VSB_printf(vsb, "req = %p {\n", req);
+	if (pan_already(vsb, req))
+		return;
 	VSB_indent(vsb, 2);
 
 	VSB_printf(vsb, "vxid = %u, ", VXID(req->vsl->wid));
@@ -405,6 +443,8 @@ pan_sess(struct vsb *vsb, const struct sess *sp)
 	char *cp;
 
 	VSB_printf(vsb, "sp = %p {\n", sp);
+	if (pan_already(vsb, sp))
+		return;
 	VSB_indent(vsb, 2);
 	VSB_printf(vsb, "fd = %d, vxid = %u,\n", sp->fd, VXID(sp->vxid));
 	AZ(SES_Get_client_ip(sp, &ci));



From phk at FreeBSD.org  Thu Jan 28 11:49:20 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 28 Jan 2016 12:49:20 +0100
Subject: [master] bba570d Remove bo->boc pointer.  No longer needed.
Message-ID: <E1aOl4i-0006b4-K4@project.varnish-software.com>


commit bba570dcdec7c861757afb2b99d28b68d1aa1abe
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu Jan 21 22:14:14 2016 +0000

    Remove bo->boc pointer.  No longer needed.

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index 4d58308..ba98386 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -462,8 +462,6 @@ struct busyobj {
 	unsigned		magic;
 #define BUSYOBJ_MAGIC		0x23b95567
 
-	struct boc		*boc;
-
 	char			*end;
 
 	/*
diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index 05b5482..0555d98 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -987,17 +987,16 @@ VBF_Fetch(struct worker *wrk, struct req *req, struct objcore *oc,
 	}
 
 	bo = VBO_GetBusyObj(wrk, req);
-	bo->boc = oc->boc;
-
 	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
+
+	boc = HSH_RefBusy(oc);
+	CHECK_OBJ_NOTNULL(boc, BOC_MAGIC);
+
 	VSLb(bo->vsl, SLT_Begin, "bereq %u %s", VXID(req->vsl->wid), how);
 	VSLb(req->vsl, SLT_Link, "bereq %u %s", VXID(bo->vsl->wid), how);
 
 	THR_SetBusyobj(bo);
 
-	boc = HSH_RefBusy(oc);
-	AN(boc);
-	assert(boc == bo->boc);
 
 	AN(bo->vcl);
 



From phk at FreeBSD.org  Thu Jan 28 11:49:20 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 28 Jan 2016 12:49:20 +0100
Subject: [master] 53d181b Polish
Message-ID: <E1aOl4i-0006bE-Ly@project.varnish-software.com>


commit 53d181b01c2e452276c2ee9e60be0a6d1d04bf23
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Jan 22 08:54:58 2016 +0000

    Polish

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index ba98386..310d254 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -677,7 +677,7 @@ double BAN_Time(const struct ban *ban);
 
 /* cache_busyobj.c */
 struct busyobj *VBO_GetBusyObj(struct worker *, const struct req *);
-void VBO_DerefBusyObj(struct worker *wrk, struct busyobj **busyobj);
+void VBO_ReleaseBusyObj(struct worker *wrk, struct busyobj **busyobj);
 
 /* cache_cli.c [CLI] */
 extern pthread_t cli_thread;
@@ -849,7 +849,7 @@ int ObjIterate(struct worker *, struct objcore *,
     void *priv, objiterate_f *func);
 int ObjGetSpace(struct worker *, struct objcore *, ssize_t *sz, uint8_t **ptr);
 void ObjExtend(struct worker *, struct objcore *, ssize_t l);
-ssize_t ObjWaitExtend(struct worker *, struct objcore *, struct boc *, ssize_t l);
+ssize_t ObjWaitExtend(struct worker *, struct objcore *, ssize_t l);
 void ObjSetState(const struct objcore *, enum boc_state_e next);
 void ObjWaitState(const struct objcore *, enum boc_state_e want);
 void ObjTrimStore(struct worker *, struct objcore *);
diff --git a/bin/varnishd/cache/cache_busyobj.c b/bin/varnishd/cache/cache_busyobj.c
index fbe6046..c3fa436 100644
--- a/bin/varnishd/cache/cache_busyobj.c
+++ b/bin/varnishd/cache/cache_busyobj.c
@@ -146,7 +146,7 @@ VBO_GetBusyObj(struct worker *wrk, const struct req *req)
 }
 
 void
-VBO_DerefBusyObj(struct worker *wrk, struct busyobj **pbo)
+VBO_ReleaseBusyObj(struct worker *wrk, struct busyobj **pbo)
 {
 	struct busyobj *bo;
 
diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index 0555d98..83d72be 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -957,7 +957,7 @@ vbf_fetch_thread(struct worker *wrk, void *priv)
 
 	wrk->vsl = NULL;
 	HSH_DerefBusy(wrk, bo->fetch_objcore);
-	VBO_DerefBusyObj(wrk, &bo);
+	VBO_ReleaseBusyObj(wrk, &bo);
 	THR_SetBusyobj(NULL);
 }
 
diff --git a/bin/varnishd/cache/cache_obj.c b/bin/varnishd/cache/cache_obj.c
index 00fc45d..b6a60c2 100644
--- a/bin/varnishd/cache/cache_obj.c
+++ b/bin/varnishd/cache/cache_obj.c
@@ -130,24 +130,23 @@ ObjExtend(struct worker *wrk, struct objcore *oc, ssize_t l)
  */
 
 ssize_t
-ObjWaitExtend(struct worker *wrk, struct objcore *oc, struct boc *boc,
-    ssize_t l)
+ObjWaitExtend(struct worker *wrk, struct objcore *oc, ssize_t l)
 {
 	ssize_t rv;
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
-	CHECK_OBJ_NOTNULL(boc, BOC_MAGIC);
-	Lck_Lock(&boc->mtx);
+	CHECK_OBJ_NOTNULL(oc->boc, BOC_MAGIC);
+	Lck_Lock(&oc->boc->mtx);
 	rv = ObjGetLen(wrk, oc);
 	while (1) {
-		assert(l <= rv || boc->state == BOS_FAILED);
-		if (rv > l || boc->state >= BOS_FINISHED)
+		assert(l <= rv || oc->boc->state == BOS_FAILED);
+		if (rv > l || oc->boc->state >= BOS_FINISHED)
 			break;
-		(void)Lck_CondWait(&boc->cond, &boc->mtx, 0);
+		(void)Lck_CondWait(&oc->boc->cond, &oc->boc->mtx, 0);
 		rv = ObjGetLen(wrk, oc);
 	}
-	Lck_Unlock(&boc->mtx);
+	Lck_Unlock(&oc->boc->mtx);
 	return (rv);
 }
 
diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index 1ba9642..70adcdb 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -590,7 +590,7 @@ cnt_pipe(struct worker *wrk, struct req *req)
 	if (VDI_Http1Pipe(req, bo) < 0)
 		VSLb(bo->vsl, SLT_VCL_Error, "Backend does not support pipe");
 	http_Teardown(bo->bereq);
-	VBO_DerefBusyObj(wrk, &bo);
+	VBO_ReleaseBusyObj(wrk, &bo);
 	THR_SetBusyobj(NULL);
 	return (REQ_FSM_DONE);
 }
diff --git a/bin/varnishd/hash/hash_slinger.h b/bin/varnishd/hash/hash_slinger.h
index 0451006..e9752be 100644
--- a/bin/varnishd/hash/hash_slinger.h
+++ b/bin/varnishd/hash/hash_slinger.h
@@ -31,7 +31,6 @@
 struct sess;
 struct req;
 struct objcore;
-struct busyobj;
 struct worker;
 struct object;
 
diff --git a/bin/varnishd/storage/storage.h b/bin/varnishd/storage/storage.h
index ee10ae2..b7dd9a1 100644
--- a/bin/varnishd/storage/storage.h
+++ b/bin/varnishd/storage/storage.h
@@ -33,7 +33,6 @@
 
 struct stevedore;
 struct sess;
-struct busyobj;
 struct objcore;
 struct worker;
 struct lru;
diff --git a/bin/varnishd/storage/storage_simple.c b/bin/varnishd/storage/storage_simple.c
index 79440d2..5939a57 100644
--- a/bin/varnishd/storage/storage_simple.c
+++ b/bin/varnishd/storage/storage_simple.c
@@ -257,7 +257,7 @@ sml_iterator(struct worker *wrk, struct objcore *oc,
 
 	while (1) {
 		ol = len;
-		nl = ObjWaitExtend(wrk, oc, boc, ol);
+		nl = ObjWaitExtend(wrk, oc, ol);
 		if (boc->state == BOS_FAILED) {
 			ret = -1;
 			break;



From phk at FreeBSD.org  Thu Jan 28 11:49:20 2016
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Thu, 28 Jan 2016 12:49:20 +0100
Subject: [master] e3aecb2 Use the correct test for streaming.
Message-ID: <E1aOl4i-0006bO-Oj@project.varnish-software.com>


commit e3aecb2b5928897206dceb1a00e66ff22f2d8ae5
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Jan 22 10:27:55 2016 +0000

    Use the correct test for streaming.

diff --git a/bin/varnishd/cache/cache_vrt_var.c b/bin/varnishd/cache/cache_vrt_var.c
index b76c069..cf25f67 100644
--- a/bin/varnishd/cache/cache_vrt_var.c
+++ b/bin/varnishd/cache/cache_vrt_var.c
@@ -670,7 +670,7 @@ VRT_r_resp_is_streaming(VRT_CTX)
 	if (ctx->req->objcore == NULL)
 		return (0);	/* When called from vcl_synth */
 	CHECK_OBJ_NOTNULL(ctx->req->objcore, OBJCORE_MAGIC);
-	return (ctx->req->objcore->flags & OC_F_INCOMPLETE ? 1 : 0);
+	return (ctx->req->objcore->boc == NULL ? 0 : 1);
 }
 
 /*--------------------------------------------------------------------*/



From fgsch at lodoss.net  Fri Jan 29 14:58:20 2016
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Fri, 29 Jan 2016 15:58:20 +0100
Subject: [master] 2b8b174 Reword
Message-ID: <E1aPAVA-0004ue-NP@project.varnish-software.com>


commit 2b8b174d95c85a5c1d7d7c249c6c16c5068ec006
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Fri Jan 29 14:57:19 2016 +0000

    Reword

diff --git a/doc/sphinx/reference/vmod.rst b/doc/sphinx/reference/vmod.rst
index a5cd8ce..40b7bf7 100644
--- a/doc/sphinx/reference/vmod.rst
+++ b/doc/sphinx/reference/vmod.rst
@@ -356,12 +356,12 @@ The per-call vmod_privs are freed before the per-vcl vmod_priv.
 Event functions
 ===============
 
-VMODs can have an "event" function which is called when a VCL which imports
-the VMOD is loaded, made active, or discarded.  This corresponds to the
-``VCL_EVENT_LOAD``, and ``VCL_EVENT_DISCARD`` events, respectively.  In
-addition, this function will be called when the VCL temperature is changed to
-cold or warm, corresponding to the ``VCL_EVENT_COLD`` and ``VCL_EVENT_WARM``
-events.
+VMODs can have an "event" function which is called when a VCL which
+imports the VMOD is loaded or discarded.  This corresponds to the
+``VCL_EVENT_LOAD`` and ``VCL_EVENT_DISCARD`` events, respectively.
+In addition, this function will be called when the VCL temperature is
+changed to cold or warm, corresponding to the ``VCL_EVENT_COLD`` and
+``VCL_EVENT_WARM`` events.
 
 The first argument to the event function is a VRT context.