From lkarsten at varnish-software.com  Tue Apr  1 09:30:05 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 11:30:05 +0200
Subject: [master] d1a6814 sess_workspace -> workspace_client
Message-ID: <E1WUv1B-0000Q7-P2@project.varnish-software.com>


commit d1a6814c7b262875cb5a504f3c480f4491479ed8
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Tue Apr 1 11:28:54 2014 +0200

    sess_workspace -> workspace_client

diff --git a/doc/sphinx/whats-new/upgrading.rst b/doc/sphinx/whats-new/upgrading.rst
index d64772d..e1fb55f 100644
--- a/doc/sphinx/whats-new/upgrading.rst
+++ b/doc/sphinx/whats-new/upgrading.rst
@@ -140,3 +140,16 @@ linger
 
 sess_timeout
 ~~~~~~~~~~~~
+
+sess_workspace
+~~~~~~~~~~~~~~
+
+In 3.0 it was often necessary to increase `sess_workspace` if a
+lot of VMODs, complex header operations or ESI were in use.
+
+This memory segment has been split into two in 4.0;
+`workspace_backend` and `workspace_client`.
+
+In most cases where you increased `sess_workspace` before, you
+want to increase `workspace_client` now.
+



From lkarsten at varnish-software.com  Tue Apr  1 09:30:05 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 11:30:05 +0200
Subject: [master] a592241 Short note about vcc_allow_inline_c
Message-ID: <E1WUv1B-0000QQ-QY@project.varnish-software.com>


commit a5922414aac9c1195a40b37831282bbd68a8e39b
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Tue Apr 1 11:30:02 2014 +0200

    Short note about vcc_allow_inline_c

diff --git a/doc/sphinx/whats-new/upgrading.rst b/doc/sphinx/whats-new/upgrading.rst
index e1fb55f..7f3d951 100644
--- a/doc/sphinx/whats-new/upgrading.rst
+++ b/doc/sphinx/whats-new/upgrading.rst
@@ -153,3 +153,8 @@ This memory segment has been split into two in 4.0;
 In most cases where you increased `sess_workspace` before, you
 want to increase `workspace_client` now.
 
+vcc_allow_inline_c
+~~~~~~~~~~~~~~~~~~
+
+This parameter is new since 3.0, and prohibits the use of inline
+C code in VCL by default.



From fgsch at lodoss.net  Tue Apr  1 12:06:13 2014
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Tue, 01 Apr 2014 14:06:13 +0200
Subject: [master] fc2adf0 Correct vmod name
Message-ID: <E1WUxSH-0006Sh-5M@project.varnish-software.com>


commit fc2adf04e60da468745f989e2de7efe691fe25e5
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Tue Apr 1 13:04:21 2014 +0100

    Correct vmod name
    
    Spotted by Martin Boer on #varnish

diff --git a/lib/libvmod_directors/vmod.vcc b/lib/libvmod_directors/vmod.vcc
index 230bf7f..4815c43 100644
--- a/lib/libvmod_directors/vmod.vcc
+++ b/lib/libvmod_directors/vmod.vcc
@@ -112,7 +112,7 @@ Description
        more or less 1% of the traffic of a backend in the same
        director with a weight of 100.
 Example
-       new rand_dir = director.random();
+       new rand_dir = directors.random();
 
 $Method VOID .add_backend(BACKEND, REAL)
 
@@ -136,7 +136,7 @@ Description
        session cookie, you'll have the client connecting to the same
        backend every time.
 Example
-       new hdir = director.hash();
+       new hdir = directors.hash();
 
 $Method VOID .add_backend(BACKEND, REAL)
 



From fgsch at lodoss.net  Tue Apr  1 12:17:58 2014
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Tue, 01 Apr 2014 14:17:58 +0200
Subject: [master] 6cc6c64 Missing set on req.backend_hint = ..
Message-ID: <E1WUxde-0007GS-0p@project.varnish-software.com>


commit 6cc6c642a51420979bffb2bd8e82f339023b6751
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Tue Apr 1 13:17:23 2014 +0100

    Missing set on req.backend_hint = ..
    
    From Martin Boer on #varnish

diff --git a/lib/libvmod_directors/vmod.vcc b/lib/libvmod_directors/vmod.vcc
index 4815c43..9b5732a 100644
--- a/lib/libvmod_directors/vmod.vcc
+++ b/lib/libvmod_directors/vmod.vcc
@@ -76,7 +76,7 @@ $Method BACKEND .backend()
 Description
        Picks a backend from the director.
 Example
-       req.backend_hint = rrdir.backend();
+       set req.backend_hint = rrdir.backend();
 
 $Object fallback()
 
@@ -100,7 +100,7 @@ $Method BACKEND .backend()
 Description
        Picks a backend from the director.
 Example
-       req.backend_hint = rrdir.backend();
+       set req.backend_hint = rrdir.backend();
 
 $Object random()
 
@@ -126,7 +126,7 @@ $Method BACKEND .backend()
 Description
        Picks a backend from the director.
 Example
-       req.backend_hint = rrdir.backend();
+       set req.backend_hint = rrdir.backend();
 
 $Object hash()
 
@@ -156,5 +156,5 @@ Description
        strings provided to pick the backend.
 Example
        # pick a backend based on the cookie header from the client
-       req.backend_hint = hdir.backend(req.http.cookie);
+       set req.backend_hint = hdir.backend(req.http.cookie);
 



From lkarsten at varnish-software.com  Tue Apr  1 12:40:33 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 14:40:33 +0200
Subject: [master] 3be7659 Require same version of libvarnishapi for now
Message-ID: <E1WUxzV-0000OJ-Gw@project.varnish-software.com>


commit 3be765973de3c5e0b68ade2535776de2051da72e
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Tue Apr 1 14:38:50 2014 +0200

    Require same version of libvarnishapi for now

diff --git a/control b/control
index 810fd85..dd20a99 100644
--- a/control
+++ b/control
@@ -27,9 +27,8 @@ Standards-Version: 3.9.3
 
 Package: varnish
 Architecture: any
-Depends: ${shlibs:Depends}, ${misc:Depends}, gcc ( >= 3.3), libc6-dev | libc6.1-dev | libc-dev, adduser
+Depends: ${shlibs:Depends}, ${misc:Depends}, gcc ( >= 3.3), libc6-dev | libc6.1-dev | libc-dev, adduser, libvarnishapi1 (= ${binary:Version})
 Suggests: varnish-doc
-Replaces: libvarnishapi1 (<< 3.0.0-5)
 Description: state of the art, high-performance web accelerator
  Varnish Cache is a state of the art web accelerator written with
  performance and flexibility in mind.



From lkarsten at varnish-software.com  Tue Apr  1 12:40:33 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 14:40:33 +0200
Subject: [master] e6fe0ef Reflect Makefile changes
Message-ID: <E1WUxzV-0000OP-Hx@project.varnish-software.com>


commit e6fe0efa2c5d73c3063788dc1ab78e2c88f7992b
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Tue Apr 1 14:39:00 2014 +0200

    Reflect Makefile changes

diff --git a/rules b/rules
index 41bc703..5ff61d6 100755
--- a/rules
+++ b/rules
@@ -34,9 +34,10 @@ override_dh_auto_configure:
 override_dh_auto_install:
 	dh_auto_install -a
 	@ # Install the example as default.
-	mkdir -p $(CURDIR)/debian/varnish/etc/
+	mkdir -p $(CURDIR)/debian/tmp/etc/varnish/
 	install -o root -g root -m 644 $(CURDIR)/etc/example.vcl \
-		$(CURDIR)/debian/varnish/etc/default.vcl
+		$(CURDIR)/debian/tmp/etc/varnish/default.vcl
+	rm $(CURDIR)/debian/tmp/usr/share/doc/varnish/*vcl
 	@ # Remove .la files
 	@ # (See http://wiki.debian.org/ReleaseGoals/LAFileRemoval)
 	find $(CURDIR)/debian/ -name \*.la -delete
diff --git a/varnish-doc.docs b/varnish-doc.docs
index 11a5caa..f77d301 100644
--- a/varnish-doc.docs
+++ b/varnish-doc.docs
@@ -1,3 +1,3 @@
-doc/sphinx/=build/html
+doc/sphinx/build/html
 doc/*.html
 doc/*.css
diff --git a/varnish.examples b/varnish.examples
index 5dd46eb..19a5d10 100644
--- a/varnish.examples
+++ b/varnish.examples
@@ -1 +1 @@
-bin/varnishd/default.vcl
+bin/varnishd/builtin.vcl



From phk at FreeBSD.org  Tue Apr  1 13:09:48 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 01 Apr 2014 15:09:48 +0200
Subject: [4.0] 4f66a21 rename "rxhdrs" to "rxreqhdrs" and add "rxresphdrs"
Message-ID: <E1WUyRo-0002yw-Gz@project.varnish-software.com>


commit 4f66a21d84a687eaf71a23f8e86a0c867a1a5aa5
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu Mar 13 14:46:06 2014 +0000

    rename "rxhdrs" to "rxreqhdrs" and add "rxresphdrs"

diff --git a/bin/varnishtest/tests/r01356.vtc b/bin/varnishtest/tests/r01356.vtc
index 1aaa337..341f2c8 100644
--- a/bin/varnishtest/tests/r01356.vtc
+++ b/bin/varnishtest/tests/r01356.vtc
@@ -1,7 +1,7 @@
 varnishtest "#1356, req.body failure"
 
 server s1 {
-	rxhdrs
+	rxreqhdrs
 	expect_close
 } -start
 
diff --git a/bin/varnishtest/vtc_http.c b/bin/varnishtest/vtc_http.c
index 271540b..4388f73 100644
--- a/bin/varnishtest/vtc_http.c
+++ b/bin/varnishtest/vtc_http.c
@@ -233,8 +233,12 @@ cmd_http_expect(CMD_ARGS)
 	AN(av[2]);
 	AZ(av[3]);
 	lhs = cmd_var_resolve(hp, av[0]);
+	if (lhs == NULL)
+		lhs = "<missing>";
 	cmp = av[1];
 	rhs = cmd_var_resolve(hp, av[2]);
+	if (rhs == NULL)
+		rhs = "<missing>";
 	if (!strcmp(cmp, "==")) {
 		retval = strcmp(lhs, rhs) == 0;
 	} else if (!strcmp(cmp, "<")) {
@@ -541,6 +545,25 @@ cmd_http_rxresp(CMD_ARGS)
 	vtc_log(hp->vl, 4, "bodylen = %s", hp->bodylen);
 }
 
+static void
+cmd_http_rxresphdrs(CMD_ARGS)
+{
+	struct http *hp;
+
+	(void)cmd;
+	(void)vl;
+	CAST_OBJ_NOTNULL(hp, priv, HTTP_MAGIC);
+	ONLY_CLIENT(hp, av);
+	assert(!strcmp(av[0], "rxresphdrs"));
+	av++;
+
+	for(; *av != NULL; av++)
+		vtc_log(hp->vl, 0, "Unknown http rxreq spec: %s\n", *av);
+	http_rxhdr(hp);
+	http_splitheader(hp, 0);
+}
+
+
 /**********************************************************************
  * Ungzip rx'ed body
  */
@@ -802,14 +825,14 @@ cmd_http_rxreq(CMD_ARGS)
 }
 
 static void
-cmd_http_rxhdrs(CMD_ARGS)
+cmd_http_rxreqhdrs(CMD_ARGS)
 {
 	struct http *hp;
 
 	(void)cmd;
 	(void)vl;
 	CAST_OBJ_NOTNULL(hp, priv, HTTP_MAGIC);
-	assert(!strcmp(av[0], "rxhdrs"));
+	assert(!strcmp(av[0], "rxreqhdrs"));
 	av++;
 
 	for(; *av != NULL; av++)
@@ -1169,12 +1192,13 @@ static const struct cmds http_cmds[] = {
 	{ "txreq",		cmd_http_txreq },
 
 	{ "rxreq",		cmd_http_rxreq },
-	{ "rxhdrs",		cmd_http_rxhdrs },
+	{ "rxreqhdrs",		cmd_http_rxreqhdrs },
 	{ "rxchunk",		cmd_http_rxchunk },
 	{ "rxbody",		cmd_http_rxbody },
 
 	{ "txresp",		cmd_http_txresp },
 	{ "rxresp",		cmd_http_rxresp },
+	{ "rxresphdrs",		cmd_http_rxresphdrs },
 	{ "gunzip",		cmd_http_gunzip_body },
 	{ "expect",		cmd_http_expect },
 	{ "send",		cmd_http_send },



From fgsch at lodoss.net  Tue Apr  1 13:09:48 2014
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Tue, 01 Apr 2014 15:09:48 +0200
Subject: [4.0] bf996bc There is no `y' (year) suffix
Message-ID: <E1WUyRo-0002z3-IU@project.varnish-software.com>


commit bf996bc2551d924d22c818c0c4f4f2401f0c1b34
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Thu Mar 13 14:01:39 2014 +0000

    There is no `y' (year) suffix

diff --git a/doc/sphinx/reference/vcl.rst b/doc/sphinx/reference/vcl.rst
index 5a7fe6b..ce0872c 100644
--- a/doc/sphinx/reference/vcl.rst
+++ b/doc/sphinx/reference/vcl.rst
@@ -123,9 +123,6 @@ Durations are defined by a number and a designation. The number can be a real so
   w
     weeks
 
-  y
-    years.
-
 
 Ints
 ----



From fgsch at lodoss.net  Tue Apr  1 13:09:48 2014
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Tue, 01 Apr 2014 15:09:48 +0200
Subject: [4.0] e757cc1 More spelling fixes
Message-ID: <E1WUyRo-0002zC-Jd@project.varnish-software.com>


commit e757cc1a23e0bff04b937e8318f6a5749bfe86ed
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Thu Mar 13 14:14:08 2014 +0000

    More spelling fixes

diff --git a/doc/sphinx/reference/vcl.rst b/doc/sphinx/reference/vcl.rst
index ce0872c..ab22ba4 100644
--- a/doc/sphinx/reference/vcl.rst
+++ b/doc/sphinx/reference/vcl.rst
@@ -29,7 +29,7 @@ VCL consists of the following elements:
  * Strings, booleans, time, duration, ints
  * Regular expressions
 
-In addition VCL has the following contructs:
+In addition VCL has the following constructs:
  * Include
  * Backend definitions
  * Probes
@@ -67,7 +67,7 @@ The following operators are available in VCL:
 Conditionals
 ------------
 
-VCL has *if* statments.
+VCL has *if* statements.
 
 
 Strings, booleans, time, duration and ints
@@ -319,8 +319,8 @@ Variables
 ---------
 
 In VCL you have access to certain variable objects. These contain
-requests and responses currently beeing worked on. What variables are
-availble depends on context.
+requests and responses currently being worked on. What variables are
+available depends on context.
 
 .. include:: vcl_var.rst
 



From fgsch at lodoss.net  Tue Apr  1 13:09:48 2014
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Tue, 01 Apr 2014 15:09:48 +0200
Subject: [4.0] 899b87e A few more spelling fixes
Message-ID: <E1WUyRo-0002zL-N2@project.varnish-software.com>


commit 899b87ee73edee4bb036d41c42163c80fab6383d
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Thu Mar 13 14:27:07 2014 +0000

    A few more spelling fixes

diff --git a/doc/sphinx/reference/vcl.rst b/doc/sphinx/reference/vcl.rst
index ab22ba4..a4b6f7b 100644
--- a/doc/sphinx/reference/vcl.rst
+++ b/doc/sphinx/reference/vcl.rst
@@ -19,7 +19,7 @@ When a new configuration is loaded, the varnishd management process
 translates the VCL code to C and compiles it to a shared object which
 is then dynamically linked into the server process.
 
-This document focuses on the syntax of the VCL language. Full a full
+This document focuses on the syntax of the VCL language. For a full
 description of syntax and semantics, with ample examples, please see
 the users guide at https://www.varnish-cache.org/doc/
 
@@ -175,7 +175,7 @@ key/value fashion.::
 
 The only mandatory attribute is host. The attributes will inherit
 their defaults from the global parameters. The following attributes
-are availble:
+are available:
 
   host (mandatory)
     The host to be used. IP address or a hostname that resolves to a
@@ -237,7 +237,7 @@ There are no mandatory options. These are the options you can set:
     How many of the polls in .window are considered good when Varnish
     starts. Defaults to the value of threshold - 1. In this case, the
     backend starts as sick and requires one single poll to be
-    conqsidered healthy.
+    considered healthy.
             
   window
     How many of the latest polls we examine to determine backend health. Defaults to 8.
@@ -300,7 +300,7 @@ Return statements
 
 The subroutine executions ends when a return(*action*) statement is
 made. The *action* specifies how execution should proceed. The context
-defines which actions are availble. See the user guide for information
+defines which actions are available. See the user guide for information
 on what actions are available where.
 
 Multiple subroutines



From phk at FreeBSD.org  Tue Apr  1 13:09:48 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 01 Apr 2014 15:09:48 +0200
Subject: [4.0] 717f0fb Spotted while looking for 1449:
Message-ID: <E1WUyRo-0002zU-OW@project.varnish-software.com>


commit 717f0fb9995b25ebead1549f32ebd06e65fe7da7
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu Mar 13 15:02:43 2014 +0000

    Spotted while looking for 1449:
    
    Make sure we do not send a regular "end-chunk" to the
    client if the streaming fetch fails.

diff --git a/bin/varnishd/cache/cache_http1_deliver.c b/bin/varnishd/cache/cache_http1_deliver.c
index 5a3da11..231df94 100644
--- a/bin/varnishd/cache/cache_http1_deliver.c
+++ b/bin/varnishd/cache/cache_http1_deliver.c
@@ -157,7 +157,7 @@ v1d_dorange(struct req *req, const char *r)
 
 /*--------------------------------------------------------------------*/
 
-static void
+static enum objiter_status
 v1d_WriteDirObj(struct req *req)
 {
 	enum objiter_status ois;
@@ -190,12 +190,14 @@ v1d_WriteDirObj(struct req *req)
 	} while (ois == OIS_DATA || ois == OIS_STREAM);
 	(void)VDP_bytes(req, VDP_FINISH,  NULL, 0);
 	ObjIterEnd(&oi);
+	return (ois);
 }
 
 void
 V1D_Deliver(struct req *req)
 {
 	char *r;
+	enum objiter_status ois;
 
 	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
 	CHECK_OBJ_NOTNULL(req->obj, OBJECT_MAGIC);
@@ -291,6 +293,7 @@ V1D_Deliver(struct req *req)
 	if (req->res_mode & RES_CHUNKED)
 		WRW_Chunked(req->wrk);
 
+	ois = OIS_DONE;
 	if (!req->wantbody) {
 		/* This was a HEAD or conditional request */
 	} else if (req->res_mode & RES_ESI) {
@@ -305,17 +308,19 @@ V1D_Deliver(struct req *req)
 		VDP_push(req, VDP_gunzip);
 		req->vgz = VGZ_NewUngzip(req->vsl, "U D -");
 		AZ(VGZ_WrwInit(req->vgz));
-		v1d_WriteDirObj(req);
+		ois = v1d_WriteDirObj(req);
 		(void)VGZ_Destroy(&req->vgz);
 		VDP_pop(req, VDP_gunzip);
 	} else {
-		v1d_WriteDirObj(req);
+		ois = v1d_WriteDirObj(req);
 	}
 
-	if (req->res_mode & RES_CHUNKED && !(req->res_mode & RES_ESI_CHILD))
+	if (ois == OIS_DONE &&
+	    (req->res_mode & RES_CHUNKED) &&
+	    !(req->res_mode & RES_ESI_CHILD))
 		WRW_EndChunk(req->wrk);
 
-	if (WRW_FlushRelease(req->wrk) && req->sp->fd >= 0)
+	if ((WRW_FlushRelease(req->wrk) || ois != OIS_DONE) && req->sp->fd >= 0)
 		SES_Close(req->sp, SC_REM_CLOSE);
 }
 
diff --git a/bin/varnishtest/tests/c00062.vtc b/bin/varnishtest/tests/c00062.vtc
new file mode 100644
index 0000000..643a2f6
--- /dev/null
+++ b/bin/varnishtest/tests/c00062.vtc
@@ -0,0 +1,29 @@
+varnishtest "Check that aborted backend body aborts client in streaming mode"
+
+server s1 {
+	rxreq
+	txresp -nolen -hdr "Transfer-encoding: chunked"
+	chunked {<HTML>}
+	sema r1 sync 2
+	chunked {<HTML>}
+	sema r1 sync 2
+} -start
+
+varnish v1 -cliok "param.set debug +syncvsl" -vcl+backend {
+
+} -start
+
+
+client c1 {
+	txreq
+	rxresphdrs
+	expect resp.status == 200
+	rxchunk
+	sema r1 sync 2
+	rxchunk
+	sema r1 sync 2
+	expect_close
+} -run
+
+
+
diff --git a/bin/varnishtest/tests/r01086.vtc b/bin/varnishtest/tests/r01086.vtc
index 667cc4d..d861a0d 100644
--- a/bin/varnishtest/tests/r01086.vtc
+++ b/bin/varnishtest/tests/r01086.vtc
@@ -65,9 +65,9 @@ varnish v1 -vcl+backend {
 
 client c1 {
 	txreq -hdr "Cookie: FOO"
-	rxresp
+	rxresphdrs
 	expect resp.status == 200
-	expect resp.bodylen == 0
+	expect_close
 } -run
 
 delay .1



From phk at FreeBSD.org  Tue Apr  1 13:09:48 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 01 Apr 2014 15:09:48 +0200
Subject: [4.0] e177b46 Add a test-case to check that vcl_backend_error{}
 objects can be cached.
Message-ID: <E1WUyRo-0002zd-Pd@project.varnish-software.com>


commit e177b4620e3a8e734c751c431541c5b7c27b29d7
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Thu Mar 13 15:53:12 2014 +0000

    Add a test-case to check that vcl_backend_error{} objects can be cached.

diff --git a/bin/varnishtest/tests/c00063.vtc b/bin/varnishtest/tests/c00063.vtc
new file mode 100644
index 0000000..0da903e
--- /dev/null
+++ b/bin/varnishtest/tests/c00063.vtc
@@ -0,0 +1,32 @@
+varnishtest "cache backend synth object"
+
+varnish v1 -vcl {
+
+	backend b { .host = "${bad_ip}"; }
+
+	sub vcl_backend_error {
+		set beresp.ttl = 1s;
+		set beresp.grace = 3s;
+		set beresp.http.foobar = "BLA" + bereq.xid;
+		synthetic(beresp.http.foobar);
+		return (deliver);
+	}
+} -start
+
+varnish v1 -cliok "param.set connect_timeout 1.0"
+
+client c1 {
+	txreq
+	rxresp
+	expect resp.http.foobar == "BLA1002"
+
+	delay 2
+	txreq
+	rxresp
+	expect resp.http.foobar == "BLA1002"
+
+	delay 3
+	txreq
+	rxresp
+	expect resp.http.foobar == "BLA1004"
+} -run



From phk at FreeBSD.org  Tue Apr  1 13:09:48 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 01 Apr 2014 15:09:48 +0200
Subject: [4.0] b135934 Fix a race when we Unbusy and abandonned oc which has
 no object.
Message-ID: <E1WUyRo-0002zm-Ub@project.varnish-software.com>


commit b135934f9eace9bc281b3e727f37dfd302170fa8
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Mar 14 09:24:41 2014 +0000

    Fix a race when we Unbusy and abandonned oc which has no object.
    
    Fixes	#1449

diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index e0c9350..b5cb463 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -682,10 +682,9 @@ vbf_stp_fail(struct worker *wrk, struct busyobj *bo)
 	CHECK_OBJ_NOTNULL(bo->fetch_objcore, OBJCORE_MAGIC);
 
 	assert(bo->state < BOS_FINISHED);
-	bo->fetch_objcore->flags |= OC_F_PRIVATE;
+	HSH_Fail(bo->fetch_objcore);
 	if (bo->fetch_objcore->flags & OC_F_BUSY)
 		HSH_Unbusy(&wrk->stats, bo->fetch_objcore);
-	HSH_Fail(bo->fetch_objcore);
 	wrk->stats.fetch_failed++;
 	VBO_setstate(bo, BOS_FAILED);
 	return (F_STP_DONE);
diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index 882c84b..dbcfaee 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -629,6 +629,13 @@ HSH_Fail(struct objcore *oc)
 	oh = oc->objhead;
 	CHECK_OBJ(oh, OBJHEAD_MAGIC);
 
+	/*
+	 * We have to have either a busy bit, so that HSH_Lookup
+	 * will not consider this oc, or an object hung of the oc
+	 * so that it can consider it.
+	 */
+	assert((oc->flags & OC_F_BUSY) || (oc->methods != NULL));
+
 	Lck_Lock(&oh->mtx);
 	oc->flags |= OC_F_FAILED;
 	Lck_Unlock(&oh->mtx);
@@ -678,7 +685,7 @@ HSH_Unbusy(struct dstat *ds, struct objcore *oc)
 	if (oh->waitinglist != NULL)
 		hsh_rush(ds, oh);
 	Lck_Unlock(&oh->mtx);
-	if (!(oc->flags & OC_F_PRIVATE)) {
+	if (!(oc->flags & OC_F_PRIVATE) && oc->methods != NULL) {
 		BAN_NewObjCore(oc);
 		EXP_Insert(oc);
 		AN(oc->flags & OC_F_EXP);



From perbu at varnish-software.com  Tue Apr  1 13:09:49 2014
From: perbu at varnish-software.com (Per Buer)
Date: Tue, 01 Apr 2014 15:09:49 +0200
Subject: [4.0] 02843b8 Fix URL
Message-ID: <E1WUyRp-0002zv-5a@project.varnish-software.com>


commit 02843b8a0d70e5fe73d522f0ac8cde3e7a5032af
Author: Per Buer <perbu at varnish-software.com>
Date:   Fri Mar 14 11:57:31 2014 +0100

    Fix URL

diff --git a/doc/sphinx/reference/vcl.rst b/doc/sphinx/reference/vcl.rst
index a4b6f7b..6f91b91 100644
--- a/doc/sphinx/reference/vcl.rst
+++ b/doc/sphinx/reference/vcl.rst
@@ -21,7 +21,7 @@ is then dynamically linked into the server process.
 
 This document focuses on the syntax of the VCL language. For a full
 description of syntax and semantics, with ample examples, please see
-the users guide at https://www.varnish-cache.org/doc/
+the users guide at https://www.varnish-cache.org/docs/
 
 VCL consists of the following elements:
  * Operators



From tfheen at err.no  Tue Apr  1 13:09:49 2014
From: tfheen at err.no (Tollef Fog Heen)
Date: Tue, 01 Apr 2014 15:09:49 +0200
Subject: [4.0] 79c241e Typos and itsos
Message-ID: <E1WUyRp-000304-85@project.varnish-software.com>


commit 79c241e54e08a742e591b20bde253ec16f9f346d
Author: Tollef Fog Heen <tfheen at fastly.com>
Date:   Fri Mar 14 19:25:47 2014 +0100

    Typos and itsos

diff --git a/doc/sphinx/tutorial/introduction.rst b/doc/sphinx/tutorial/introduction.rst
index 3f65968..50a7296 100644
--- a/doc/sphinx/tutorial/introduction.rst
+++ b/doc/sphinx/tutorial/introduction.rst
@@ -3,7 +3,7 @@
 The fundamentals of web proxy caching with Varnish
 --------------------------------------------------
 
-Varnish is a caching HTTP reverse proxy. It recieves requests from
+Varnish is a caching HTTP reverse proxy. It receives requests from
 clients and tries to answer them from its cache. If it cannot answer
 the request from its cache it will forward the request to the backend,
 fetch the response, store it and deliver it to the client.
@@ -14,11 +14,11 @@ typical application server, so you want to make sure to have it answer
 as many of the requests as possible.
 
 Varnish decides whether it can store the content or not based on the
-response it's gets back from the backend. The backend can instruct
+response it gets back from the backend. The backend can instruct
 Varnish to cache the content with the HTTP response header
 Cache-Control. There are a few other conditions where Varnish will not
 cache, the most common one being cookies. Since cookies is a good
-indication that a web object is personlised, Varnish will with it's
+indication that a web object is personalised, Varnish will with its
 default configuration, not cache it.
 
 This behaviour and most other behaviour can be changed using policies
@@ -29,7 +29,7 @@ Performance
 ~~~~~~~~~~~
 
 Varnish has a modern architecture and is written with performance in
-mind.  It is usually bound by the speed of the network, effectivly
+mind.  It is usually bound by the speed of the network, effectively
 turning performance into a non-issue. You get to focus on how your web
 application work and you can allow yourself, to some degree, to care
 less about performance and scalability.
@@ -37,8 +37,8 @@ less about performance and scalability.
 Flexibility
 ~~~~~~~~~~~
 
-One of the key features of Varnish Cache, in addition to it's
-performance, is the flexibility of it's configuration language,
+One of the key features of Varnish Cache, in addition to its
+performance, is the flexibility of its configuration language,
 VCL. VCL enables you to write policies on how incoming requests should
 be handled. 
 
@@ -68,7 +68,7 @@ You can get in touch with us trough many channels. For real time chat
 you can reach us on IRC trough the server irc.linpro.net on the
 #varnish and #varnish-hacking channels.
 The are two mailing lists available. One for user questions and one
-for development discussions. See varnish-cache.org/mailinglist for
+for development discussions. See https://www.varnish-cache.org/mailinglist for
 information and signup.  There is also a web forum on the same site.
 
 Now that you have a vague idea on what Varnish Cache is, let see if we



From phk at FreeBSD.org  Tue Apr  1 13:09:49 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 01 Apr 2014 15:09:49 +0200
Subject: [4.0] 44d4609 Don't unbusy failed objects,
 it is pointless and introduces a race.
Message-ID: <E1WUyRp-00030D-9l@project.varnish-software.com>


commit 44d46093dd34b4db102964a49bc90d7039f65db2
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Mar 17 09:46:22 2014 +0000

    Don't unbusy failed objects, it is pointless and introduces a race.

diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index b5cb463..2f5bb9c 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -683,8 +683,6 @@ vbf_stp_fail(struct worker *wrk, struct busyobj *bo)
 
 	assert(bo->state < BOS_FINISHED);
 	HSH_Fail(bo->fetch_objcore);
-	if (bo->fetch_objcore->flags & OC_F_BUSY)
-		HSH_Unbusy(&wrk->stats, bo->fetch_objcore);
 	wrk->stats.fetch_failed++;
 	VBO_setstate(bo, BOS_FAILED);
 	return (F_STP_DONE);
@@ -738,9 +736,6 @@ vbf_fetch_thread(struct worker *wrk, void *priv)
 	}
 	assert(WRW_IsReleased(wrk));
 
-	AZ(bo->fetch_objcore->flags & OC_F_BUSY);
-	HSH_Complete(bo->fetch_objcore);
-
 	bo->stats = NULL;
 
 	if (bo->vbc != NULL) {
@@ -756,6 +751,7 @@ vbf_fetch_thread(struct worker *wrk, void *priv)
 
 	if (bo->state == BOS_FINISHED) {
 		assert(!(bo->fetch_objcore->flags & OC_F_FAILED));
+		HSH_Complete(bo->fetch_objcore);
 		VSLb(bo->vsl, SLT_Length, "%zd", bo->fetch_obj->len);
 		{
 		/* Sanity check fetch methods accounting */



From lkarsten at varnish-software.com  Tue Apr  1 13:09:49 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:49 +0200
Subject: [4.0] e7466ac Reserve h key for a future help screen
Message-ID: <E1WUyRp-00030M-Bl@project.varnish-software.com>


commit e7466ac4183f6271560ffd0ce48617fc29607419
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Mon Mar 17 12:53:02 2014 +0100

    Reserve h key for a future help screen
    
    Move showing of debug counters from 'h' key to 'd' inside a running
    varnishstat.

diff --git a/bin/varnishstat/varnishstat_curses.c b/bin/varnishstat/varnishstat_curses.c
index e722408..6e8bae8 100644
--- a/bin/varnishstat/varnishstat_curses.c
+++ b/bin/varnishstat/varnishstat_curses.c
@@ -821,7 +821,7 @@ handle_keypress(int ch)
 		current = n_ptarray - 1;
 		page_start = current - l_points + 1;
 		break;
-	case 'h':
+	case 'd':
 		hide_unseen = 1 - hide_unseen;
 		rebuild = 1;
 		break;



From lkarsten at varnish-software.com  Tue Apr  1 13:09:49 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:49 +0200
Subject: [4.0] 793d67a Helpful pointers for future contributors
Message-ID: <E1WUyRp-00030V-Gz@project.varnish-software.com>


commit 793d67ad00916ab55d089e932673a8000dffb62a
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Mon Mar 17 13:06:36 2014 +0100

    Helpful pointers for future contributors
    
    Mostly an attempt at pointing people that send pull requests on Github
    in the right direction.

diff --git a/CONTRIBUTING b/CONTRIBUTING
new file mode 100644
index 0000000..cfccc09
--- /dev/null
+++ b/CONTRIBUTING
@@ -0,0 +1,15 @@
+Contributing to Varnish Cache
+=============================
+
+Varnish Cache is developed with Git hosted on our own infrastructure.
+
+Official development tree is here:
+
+    git://git.varnish-cache.org/varnish-cache
+
+Patches can be sent to varnish-dev at varnish-cache.org.
+
+There is a READ-ONLY git-mirror on Github.  Please do not send contributions
+there, we need them by email to -dev@ for now.
+
+



From lkarsten at varnish-software.com  Tue Apr  1 13:09:49 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:49 +0200
Subject: [4.0] a4d1d5d Small example on how to do synthetic responses
Message-ID: <E1WUyRp-00030e-Ip@project.varnish-software.com>


commit a4d1d5d2ad8dd3d9a1c585e1d5e009bb81352944
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Mon Mar 17 14:19:46 2014 +0100

    Small example on how to do synthetic responses

diff --git a/doc/sphinx/whats-new/upgrading.rst b/doc/sphinx/whats-new/upgrading.rst
index 68a41d0..a0da998 100644
--- a/doc/sphinx/whats-new/upgrading.rst
+++ b/doc/sphinx/whats-new/upgrading.rst
@@ -41,6 +41,24 @@ You must now explicitly return an error::
 
 	return(error(999, "Response));
 
+
+Synthetic responses in vcl_error
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Setting headers on synthetic response bodies made in vcl_error are now done on
+resp.http instead of obj.http.
+
+The synthetic keyword is now a function::
+
+    if (resp.status == 799) {
+        set resp.status = 200;
+        set resp.http.Content-Type = "text/plain; charset=utf-8";
+        synthetic({"You are "} + client.ip);
+        return (deliver);
+    }
+
+
+
 hit_for_pass objects are created using beresp.uncacheable
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Example::



From phk at FreeBSD.org  Tue Apr  1 13:09:49 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 01 Apr 2014 15:09:49 +0200
Subject: [4.0] 4e3169c Dump some info on fetch_objcore during panics
Message-ID: <E1WUyRp-00030n-KM@project.varnish-software.com>


commit 4e3169c626d0591189c4de67f9b08e6a7f9aa44e
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Mar 17 16:05:42 2014 +0000

    Dump some info on fetch_objcore during panics

diff --git a/bin/varnishd/cache/cache_panic.c b/bin/varnishd/cache/cache_panic.c
index 8df8bfa..a01e771 100644
--- a/bin/varnishd/cache/cache_panic.c
+++ b/bin/varnishd/cache/cache_panic.c
@@ -228,6 +228,19 @@ pan_object(const char *typ, const struct object *o)
 /*--------------------------------------------------------------------*/
 
 static void
+pan_objcore(const char *typ, const struct objcore *oc)
+{
+
+	VSB_printf(pan_vsp, "  objcore (%s) = %p {\n", typ, oc);
+	VSB_printf(pan_vsp, "    refcnt = %d\n", oc->refcnt);
+	VSB_printf(pan_vsp, "    flags = 0x%x\n", oc->flags);
+	VSB_printf(pan_vsp, "    objhead = %p\n", oc->objhead);
+	VSB_printf(pan_vsp, "  }\n");
+}
+
+/*--------------------------------------------------------------------*/
+
+static void
 pan_vcl(const struct VCL_conf *vcl)
 {
 	int i;
@@ -284,6 +297,8 @@ pan_busyobj(const struct busyobj *bo)
 	if (bo->beresp->ws != NULL)
 		pan_http("beresp", bo->beresp, 4);
 	pan_ws(bo->ws_o, 4);
+	if (bo->fetch_objcore)
+		pan_objcore("FETCH", bo->fetch_objcore);
 	if (bo->fetch_obj)
 		pan_object("FETCH", bo->fetch_obj);
 	if (bo->ims_obj)



From phk at FreeBSD.org  Tue Apr  1 13:09:49 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 01 Apr 2014 15:09:49 +0200
Subject: [4.0] 99851a1 Try to close the #1449 race definitively.
Message-ID: <E1WUyRp-00030w-NP@project.varnish-software.com>


commit 99851a10c7314dd6a4eb6a0de60161732b1b7b55
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Mar 17 16:06:06 2014 +0000

    Try to close the #1449 race definitively.

diff --git a/bin/varnishd/cache/cache_expire.c b/bin/varnishd/cache/cache_expire.c
index 1471a64..a1fcbc3 100644
--- a/bin/varnishd/cache/cache_expire.c
+++ b/bin/varnishd/cache/cache_expire.c
@@ -137,7 +137,7 @@ EXP_Inject(struct objcore *oc, struct lru *lru, double when)
 
 	AZ(oc->flags & (OC_F_OFFLRU | OC_F_INSERT | OC_F_MOVE));
 	AZ(oc->flags & OC_F_DYING);
-	AZ(oc->flags & OC_F_BUSY);
+	// AN(oc->flags & OC_F_BUSY);
 
 	if (lru == NULL)
 		lru = oc_getlru(oc);
diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index 2f5bb9c..42471ed 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -565,9 +565,10 @@ vbf_stp_condfetch(struct worker *wrk, struct busyobj *bo)
 	obj->gzip_stop = bo->ims_obj->gzip_stop;
 
 	AZ(WS_Overflowed(bo->ws_o));
-	if (bo->do_stream)
+	if (bo->do_stream) {
+		HSH_Unbusy(&wrk->stats, obj->objcore);
 		VBO_setstate(bo, BOS_STREAM);
-	HSH_Unbusy(&wrk->stats, obj->objcore);
+	}
 
 	st = NULL;
 	al = 0;
@@ -596,6 +597,9 @@ vbf_stp_condfetch(struct worker *wrk, struct busyobj *bo)
 	if (bo->failed)
 		return (F_STP_FAIL);
 
+	if (!bo->do_stream)
+		HSH_Unbusy(&wrk->stats, obj->objcore);
+
 	assert(al == bo->ims_obj->len);
 	assert(obj->len == al);
 	EXP_Rearm(bo->ims_obj, bo->ims_obj->exp.t_origin, 0, 0, 0);
@@ -802,6 +806,7 @@ VBF_Fetch(struct worker *wrk, struct req *req, struct objcore *oc,
 
 	bo = VBO_GetBusyObj(wrk, req);
 	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
+	THR_SetBusyobj(bo);
 
 	switch(mode) {
 	case VBF_PASS:		how = "pass"; break;
@@ -853,7 +858,12 @@ VBF_Fetch(struct worker *wrk, struct req *req, struct objcore *oc,
 		VBO_waitstate(bo, BOS_REQ_DONE);
 	} else {
 		VBO_waitstate(bo, BOS_STREAM);
-		assert(bo->state != BOS_FAILED || (oc->flags & OC_F_FAILED));
+		if (bo->state == BOS_FAILED) {
+			AN((oc->flags & OC_F_FAILED));
+		} else {
+			AZ(bo->fetch_objcore->flags & OC_F_BUSY);
+		}
 	}
+	THR_SetBusyobj(NULL);
 	VBO_DerefBusyObj(wrk, &bo);
 }
diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index dbcfaee..07c5931 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -672,9 +672,17 @@ HSH_Unbusy(struct dstat *ds, struct objcore *oc)
 	oh = oc->objhead;
 	CHECK_OBJ(oh, OBJHEAD_MAGIC);
 
+	AN(oc->methods);
 	AN(oc->flags & OC_F_BUSY);
 	assert(oh->refcnt > 0);
 
+	if (!(oc->flags & OC_F_PRIVATE)) {
+		BAN_NewObjCore(oc);
+		EXP_Insert(oc);
+		AN(oc->flags & OC_F_EXP);
+		AN(oc->ban);
+	}
+
 	/* XXX: pretouch neighbors on oh->objcs to prevent page-on under mtx */
 	Lck_Lock(&oh->mtx);
 	assert(oh->refcnt > 0);
@@ -685,12 +693,6 @@ HSH_Unbusy(struct dstat *ds, struct objcore *oc)
 	if (oh->waitinglist != NULL)
 		hsh_rush(ds, oh);
 	Lck_Unlock(&oh->mtx);
-	if (!(oc->flags & OC_F_PRIVATE) && oc->methods != NULL) {
-		BAN_NewObjCore(oc);
-		EXP_Insert(oc);
-		AN(oc->flags & OC_F_EXP);
-		AN(oc->ban);
-	}
 }
 
 /*---------------------------------------------------------------------



From phk at FreeBSD.org  Tue Apr  1 13:09:49 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 01 Apr 2014 15:09:49 +0200
Subject: [4.0] 8aec118 Comment out an assert to get jenkins happier
Message-ID: <E1WUyRp-000315-Q2@project.varnish-software.com>


commit 8aec1184ecb4d72ccaf70e9d8bf40139cf72a024
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Mar 18 09:25:43 2014 +0000

    Comment out an assert to get jenkins happier

diff --git a/bin/varnishd/cache/cache_expire.c b/bin/varnishd/cache/cache_expire.c
index a1fcbc3..dfec9ee 100644
--- a/bin/varnishd/cache/cache_expire.c
+++ b/bin/varnishd/cache/cache_expire.c
@@ -427,7 +427,7 @@ exp_inbox(struct exp_priv *ep, struct objcore *oc, double now)
 	VSLb(&ep->vsl, SLT_ExpKill, "EXP_Inbox p=%p e=%.9f f=0x%x", oc,
 	    oc->timer_when, oc->flags);
 
-	AZ(oc->flags & OC_F_BUSY);
+	// AZ(oc->flags & OC_F_BUSY);
 
 	lru = oc_getlru(oc);
 	CHECK_OBJ_NOTNULL(lru, LRU_MAGIC);



From phk at FreeBSD.org  Tue Apr  1 13:09:49 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 01 Apr 2014 15:09:49 +0200
Subject: [4.0] 07ff86d Add varnishhist
Message-ID: <E1WUyRp-00031E-Rx@project.varnish-software.com>


commit 07ff86df5aabda77abf3f013dd2e16fd4a32c5a8
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Mar 18 09:34:54 2014 +0000

    Add varnishhist

diff --git a/bin/Makefile.phk b/bin/Makefile.phk
index 5380231..07ed226 100644
--- a/bin/Makefile.phk
+++ b/bin/Makefile.phk
@@ -5,7 +5,7 @@ SUBDIRS += varnishncsa
 # varnishreplay
 SUBDIRS += varnishtest 
 SUBDIRS += varnishstat
-# SUBDIRS += varnishhist 
+SUBDIRS += varnishhist 
 SUBDIRS += varnishtop
 
 TOPDIR = $(CURDIR)/..



From tfheen at err.no  Tue Apr  1 13:09:49 2014
From: tfheen at err.no (Tollef Fog Heen)
Date: Tue, 01 Apr 2014 15:09:49 +0200
Subject: [4.0] 166c557 Typo in makefile.phk
Message-ID: <E1WUyRp-00031N-Sy@project.varnish-software.com>


commit 166c557144414588c5ab0557428f5a280669c53c
Author: Tollef Fog Heen <tfheen at fastly.com>
Date:   Tue Mar 18 11:00:39 2014 +0100

    Typo in makefile.phk

diff --git a/bin/varnishd/Makefile.phk b/bin/varnishd/Makefile.phk
index f8bf48a..9922d9f 100644
--- a/bin/varnishd/Makefile.phk
+++ b/bin/varnishd/Makefile.phk
@@ -104,7 +104,7 @@ include $(TOPDIR)/Makefile.inc.phk
 #
 # Turn the builtin.vcl file into a C-string we can include in the program.
 #
-builtint_vcl.h:	builtin.vcl
+builtin_vcl.h:	builtin.vcl
 	echo '/*' > $@
 	echo ' * NB:  This file is machine generated, DO NOT EDIT!' >> $@
 	echo ' *' >> $@



From phk at FreeBSD.org  Tue Apr  1 13:09:49 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 01 Apr 2014 15:09:49 +0200
Subject: [4.0] 70fda4f Comment out another busy check
Message-ID: <E1WUyRp-00031W-U3@project.varnish-software.com>


commit 70fda4f4d85465c9672c208bab5a5343e35d92b6
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Mar 18 10:34:39 2014 +0000

    Comment out another busy check

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index 94fdf32..eac7546 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -456,7 +456,7 @@ oc_getobj(struct dstat *ds, struct objcore *oc)
 {
 
 	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
-	AZ(oc->flags & OC_F_BUSY);
+	// AZ(oc->flags & OC_F_BUSY);
 	AN(oc->methods);
 	AN(oc->methods->getobj);
 	return (oc->methods->getobj(ds, oc));



From phk at FreeBSD.org  Tue Apr  1 13:09:49 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 01 Apr 2014 15:09:49 +0200
Subject: [4.0] 44b8341 White space fix
Message-ID: <E1WUyRp-00031f-VW@project.varnish-software.com>


commit 44b8341390a05677d79a5c04cfce61d17a3bbc02
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Mar 18 10:36:35 2014 +0000

    White space fix

diff --git a/lib/libvcc/generate.py b/lib/libvcc/generate.py
index 48b0175..523f05f 100755
--- a/lib/libvcc/generate.py
+++ b/lib/libvcc/generate.py
@@ -161,8 +161,8 @@ sp_variables = [
 		'IP',
 		( 'both',),
 		( ), """
-The client's IP address.
-"""
+		The client's IP address.
+		"""
 	),
 	('client.identity',
 		'STRING',



From phk at FreeBSD.org  Tue Apr  1 13:09:50 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 01 Apr 2014 15:09:50 +0200
Subject: [4.0] 2247fc8 More commenting out busy checks
Message-ID: <E1WUyRq-00031o-10@project.varnish-software.com>


commit 2247fc840d9fe8e44b31cddd81be0d8f646b361c
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Mar 18 11:12:52 2014 +0000

    More commenting out busy checks

diff --git a/bin/varnishd/cache/cache_expire.c b/bin/varnishd/cache/cache_expire.c
index dfec9ee..4f21152 100644
--- a/bin/varnishd/cache/cache_expire.c
+++ b/bin/varnishd/cache/cache_expire.c
@@ -513,7 +513,7 @@ exp_expire(struct exp_priv *ep, double now)
 	lru = oc_getlru(oc);
 	CHECK_OBJ_NOTNULL(lru, LRU_MAGIC);
 	Lck_Lock(&lru->mtx);
-	AZ(oc->flags & OC_F_BUSY);
+	// AZ(oc->flags & OC_F_BUSY);
 	oc->flags |= OC_F_DYING;
 	if (oc->flags & OC_F_OFFLRU)
 		oc = NULL;
diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index 42471ed..5b5be58 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -203,11 +203,10 @@ vbf_stp_mkbereq(const struct worker *wrk, struct busyobj *bo)
 		if (http_GetHdr(bo->ims_obj->http, H_Last_Modified, &p)) {
 			http_PrintfHeader(bo->bereq0,
 			    "If-Modified-Since: %s", p);
-		} else if (http_GetHdr(bo->ims_obj->http, H_ETag, &p)) {
+		}
+		if (http_GetHdr(bo->ims_obj->http, H_ETag, &p)) {
 			http_PrintfHeader(bo->bereq0,
 			    "If-None-Match: %s", p);
-		} else {
-			WRONG("Shouldn't have bo->ims_obj");
 		}
 	}
 



From tfheen at err.no  Tue Apr  1 13:09:50 2014
From: tfheen at err.no (Tollef Fog Heen)
Date: Tue, 01 Apr 2014 15:09:50 +0200
Subject: [4.0] ae54868 Move X-Forwarded-For handled into C code
Message-ID: <E1WUyRq-00031y-25@project.varnish-software.com>


commit ae548683b8f91d0a92799f6c746b80773a4c9f05
Author: Tollef Fog Heen <tfheen at fastly.com>
Date:   Tue Mar 18 12:18:33 2014 +0100

    Move X-Forwarded-For handled into C code
    
    Remove adding of X-Forwarded-For from builtin.vcl and put it into the
    C code that runs before vcl_recv.  This makes it more likely that it's
    properly set when the user VCL runs.
    
    Fixes: #1454

diff --git a/bin/varnishd/builtin.vcl b/bin/varnishd/builtin.vcl
index da30346..ac795fd 100644
--- a/bin/varnishd/builtin.vcl
+++ b/bin/varnishd/builtin.vcl
@@ -45,14 +45,6 @@ vcl 4.0;
 # Client side
 
 sub vcl_recv {
-    if (req.restarts == 0) {
-        if (req.http.x-forwarded-for) {
-            set req.http.X-Forwarded-For =
-                req.http.X-Forwarded-For + ", " + client.ip;
-        } else {
-            set req.http.X-Forwarded-For = client.ip;
-        }
-    }
     if (req.method != "GET" &&
       req.method != "HEAD" &&
       req.method != "PUT" &&
diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index 467ea8d..5df16e6 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -660,6 +660,7 @@ cnt_recv(struct worker *wrk, struct req *req)
 {
 	unsigned recv_handling;
 	struct SHA256Context sha256ctx;
+	char *xff;
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
@@ -688,6 +689,16 @@ cnt_recv(struct worker *wrk, struct req *req)
 	req->hash_always_miss = 0;
 	req->hash_ignore_busy = 0;
 	req->client_identity = NULL;
+	if (req->restarts == 0) {
+		if (http_GetHdr(req->http, H_X_Forwarded_For, &xff)) {
+			http_Unset(req->http, H_X_Forwarded_For);
+			http_PrintfHeader(req->http, "X-Forwarded-For: %s, %s", xff,
+					  req->sp->client_addr_str);
+		} else {
+			http_PrintfHeader(req->http, "X-Forwarded-For: %s",
+					  req->sp->client_addr_str);
+		}
+	}
 
 	http_CollectHdr(req->http, H_Cache_Control);
 
diff --git a/include/tbl/http_headers.h b/include/tbl/http_headers.h
index 758c7e6..1db7784 100644
--- a/include/tbl/http_headers.h
+++ b/include/tbl/http_headers.h
@@ -94,5 +94,6 @@ HTTPH("Vary",			H_Vary,			0					  )	/* RFC2616 14.44 */
 HTTPH("Via",			H_Via,			0					  )	/* RFC2616 14.45 */
 HTTPH("Warning",		H_Warning,		0					  )	/* RFC2616 14.46 */
 HTTPH("WWW-Authenticate",	H_WWW_Authenticate,	0					  )	/* RFC2616 14.47 */
+HTTPH("X-Forwarded-For",	H_X_Forwarded_For,	0					  )	/* Not RFC */
 
 /*lint -restore */



From tfheen at err.no  Tue Apr  1 13:09:50 2014
From: tfheen at err.no (Tollef Fog Heen)
Date: Tue, 01 Apr 2014 15:09:50 +0200
Subject: [4.0] 880c3c2 Set Connection: close before calling vcl_pipe
Message-ID: <E1WUyRq-000327-3E@project.varnish-software.com>


commit 880c3c2a564aeeaad508dab136611973ca71d6aa
Author: Tollef Fog Heen <tfheen at fastly.com>
Date:   Tue Mar 18 12:43:54 2014 +0100

    Set Connection: close before calling vcl_pipe
    
    It makes sense to default to setting Connection: close for vcl_pipe.
    Do this before calling vcl_pipe so it's overrideable.
    
    Fixes #1455

diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index 5df16e6..f43416f 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -579,6 +579,7 @@ cnt_pipe(struct worker *wrk, struct req *req)
 	http_FilterReq(bo->bereq, req->http, 0);	// XXX: 0 ?
 	http_PrintfHeader(bo->bereq,
 	    "X-Varnish: %u", req->vsl->wid & VSL_IDENTMASK);
+	http_SetHeader(bo->bereq, "Connection: close");
 
 	VCL_pipe_method(req->vcl, wrk, req, bo, req->http->ws);
 



From phk at FreeBSD.org  Tue Apr  1 13:09:50 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 01 Apr 2014 15:09:50 +0200
Subject: [4.0] fa955a8 Make this test more robust
Message-ID: <E1WUyRq-00032G-4L@project.varnish-software.com>


commit fa955a8134f64510db446967f8ea45736870824a
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Mar 18 13:25:06 2014 +0000

    Make this test more robust

diff --git a/bin/varnishtest/tests/r01030.vtc b/bin/varnishtest/tests/r01030.vtc
index fcc9594..543f93f 100644
--- a/bin/varnishtest/tests/r01030.vtc
+++ b/bin/varnishtest/tests/r01030.vtc
@@ -43,7 +43,7 @@ client c1 {
 varnish v1 -expect bans_lurker_tests_tested == 0
 
 delay 1.0
-varnish v1 -expect bans_lurker_tests_tested == 1
+varnish v1 -expect bans_lurker_tests_tested >= 1
 
 varnish v1 -cliok "param.set ban_lurker_sleep 5.01"
 



From nils.goroll at uplex.de  Tue Apr  1 13:09:50 2014
From: nils.goroll at uplex.de (Nils Goroll)
Date: Tue, 01 Apr 2014 15:09:50 +0200
Subject: [4.0] 40112d5 reference to wiki page
Message-ID: <E1WUyRq-00032P-5b@project.varnish-software.com>


commit 40112d580e27501df3e297ff2f5cd586a02b4a2c
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Tue Mar 18 16:51:44 2014 +0100

    reference to wiki page

diff --git a/CONTRIBUTING b/CONTRIBUTING
index cfccc09..2c2db8c 100644
--- a/CONTRIBUTING
+++ b/CONTRIBUTING
@@ -1,6 +1,9 @@
 Contributing to Varnish Cache
 =============================
 
+To contribute to Varnish, read https://www.varnish-cache.org/trac/wiki/Contributing
+as a staring point and follow relevant parts linked there.
+
 Varnish Cache is developed with Git hosted on our own infrastructure.
 
 Official development tree is here:



From nils.goroll at uplex.de  Tue Apr  1 13:09:50 2014
From: nils.goroll at uplex.de (Nils Goroll)
Date: Tue, 01 Apr 2014 15:09:50 +0200
Subject: [4.0] 2e12bb7 another reference to the Contributing wiki page
Message-ID: <E1WUyRq-00032Y-6f@project.varnish-software.com>


commit 2e12bb7fde08aa294cb1e57947f0fc90d515bf71
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Tue Mar 18 17:05:02 2014 +0100

    another reference to the Contributing wiki page

diff --git a/doc/sphinx/tutorial/introduction.rst b/doc/sphinx/tutorial/introduction.rst
index 50a7296..ad8784a 100644
--- a/doc/sphinx/tutorial/introduction.rst
+++ b/doc/sphinx/tutorial/introduction.rst
@@ -61,6 +61,9 @@ the Varnish Governing Board which currently consist of Poul-Henning
 Kamp (Architect), Rogier Mulhuijzen (Fastly) and Kristian Lyngst?l
 (Varnish Software).
 
+Please see https://www.varnish-cache.org/trac/wiki/Contributing as
+a starting point if you would like to contribute to Varnish.
+
 Getting in touch
 ----------------
 



From nils.goroll at uplex.de  Tue Apr  1 13:09:50 2014
From: nils.goroll at uplex.de (Nils Goroll)
Date: Tue, 01 Apr 2014 15:09:50 +0200
Subject: [4.0] dbc9a28 solaris platforms
Message-ID: <E1WUyRq-00032h-7r@project.varnish-software.com>


commit dbc9a2822e965aabbfe29f43b6f7273bf1576492
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Tue Mar 18 17:07:21 2014 +0100

    solaris platforms

diff --git a/doc/sphinx/tutorial/introduction.rst b/doc/sphinx/tutorial/introduction.rst
index ad8784a..9e6c541 100644
--- a/doc/sphinx/tutorial/introduction.rst
+++ b/doc/sphinx/tutorial/introduction.rst
@@ -51,7 +51,8 @@ Supported platforms
 
 Varnish is written to run on modern versions of Linux and FreeBSD and
 the best experience is had on those platforms. Thanks to our
-contributors it also runs on NetBSD, OpenBSD and OS X.
+contributors it also runs on NetBSD, OpenBSD, OS X and various
+Solaris-descendants like Oracle Solaris, OmniOS and SmartOS.
 
 About the Varnish development process
 -------------------------------------



From nils.goroll at uplex.de  Tue Apr  1 13:09:50 2014
From: nils.goroll at uplex.de (Nils Goroll)
Date: Tue, 01 Apr 2014 15:09:50 +0200
Subject: [4.0] 7ebc542 platform notes about solaris
Message-ID: <E1WUyRq-00032r-9B@project.varnish-software.com>


commit 7ebc54253fc14604a6ca7c28e13d12371f6e72f9
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Tue Mar 18 17:10:15 2014 +0100

    platform notes about solaris

diff --git a/doc/sphinx/phk/platforms.rst b/doc/sphinx/phk/platforms.rst
index 2b60443..6f448e6 100644
--- a/doc/sphinx/phk/platforms.rst
+++ b/doc/sphinx/phk/platforms.rst
@@ -64,12 +64,12 @@ Platform specific bug reports without patches will likely live a quiet life.
 
 *Mac OS/X*
 
-*Solaris*.
+*Solaris-decendants* (Oracle Solaris, OmniOS, Joyent SmartOS)
 
-Yes, we'd like to bump Solaris to tier-A but I have to say that the
-uncertainty about the future for OpenSolaris, and lack of time to
-care and feed the somewhat altmodishe socket-API on Solaris, does
-keep the enthusiasm bounded.
+Mac OS/X is regarded as a developer platform, not as a production
+platform.
+
+Solaris-decendants are regarded as a production platform.
 
 NetBSD, AIX and HP-UX are conceivably candidates for this level, but
 so far I have not heard much, if any, user interest.
@@ -108,3 +108,4 @@ are right about Oracles intentions, Solaris may get demoted.
 Until next time,
 
 Poul-Henning, 2010-08-03
+Edited Nils, 2014-03-18 with Poul-Hennings concent



From phk at FreeBSD.org  Tue Apr  1 13:09:50 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 01 Apr 2014 15:09:50 +0200
Subject: [4.0] 05a462d Rename vcl_error{} to vcl_synth{}
Message-ID: <E1WUyRq-000331-Ae@project.varnish-software.com>


commit 05a462d98c3ee38c862c9006924f8555d133f95d
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Mar 18 16:31:34 2014 +0000

    Rename vcl_error{} to vcl_synth{}

diff --git a/bin/varnishd/builtin.vcl b/bin/varnishd/builtin.vcl
index ac795fd..cdac8d9 100644
--- a/bin/varnishd/builtin.vcl
+++ b/bin/varnishd/builtin.vcl
@@ -126,7 +126,7 @@ sub vcl_deliver {
 /*
  * We can come here "invisibly" with the following errors: 413, 417 & 503
  */
-sub vcl_error {
+sub vcl_synth {
     set resp.http.Content-Type = "text/html; charset=utf-8";
     set resp.http.Retry-After = "5";
     synthetic( {"
diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index f43416f..792b1ec 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -193,7 +193,7 @@ cnt_error(struct worker *wrk, struct req *req)
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
 
-	req->acct_req.error++;
+	req->acct_req.synth++;
 
 	HTTP_Setup(req->resp, req->ws, req->vsl, SLT_RespMethod);
 	h = req->resp;
@@ -219,7 +219,7 @@ cnt_error(struct worker *wrk, struct req *req)
 	req->synth_body = VSB_new_auto();
 	AN(req->synth_body);
 
-	VCL_error_method(req->vcl, wrk, req, NULL, req->http->ws);
+	VCL_synth_method(req->vcl, wrk, req, NULL, req->http->ws);
 
 	http_Unset(h, H_Content_Length);
 
@@ -421,7 +421,7 @@ cnt_lookup(struct worker *wrk, struct req *req)
 	case VCL_RET_RESTART:
 		req->req_step = R_STP_RESTART;
 		break;
-	case VCL_RET_ERROR:
+	case VCL_RET_SYNTH:
 		req->req_step = R_STP_ERROR;
 		break;
 	case VCL_RET_PASS:
@@ -479,7 +479,7 @@ cnt_miss(struct worker *wrk, struct req *req)
 		VBF_Fetch(wrk, req, req->objcore, o, VBF_NORMAL);
 		req->req_step = R_STP_FETCH;
 		return (REQ_FSM_MORE);
-	case VCL_RET_ERROR:
+	case VCL_RET_SYNTH:
 		req->req_step = R_STP_ERROR;
 		break;
 	case VCL_RET_RESTART:
@@ -527,7 +527,7 @@ cnt_pass(struct worker *wrk, struct req *req)
 
 	VCL_pass_method(req->vcl, wrk, req, NULL, req->http->ws);
 	switch (wrk->handling) {
-	case VCL_RET_ERROR:
+	case VCL_RET_SYNTH:
 		req->req_step = R_STP_ERROR;
 		break;
 	case VCL_RET_RESTART:
@@ -583,7 +583,7 @@ cnt_pipe(struct worker *wrk, struct req *req)
 
 	VCL_pipe_method(req->vcl, wrk, req, bo, req->http->ws);
 
-	if (wrk->handling == VCL_RET_ERROR)
+	if (wrk->handling == VCL_RET_SYNTH)
 		INCOMPL();
 	assert(wrk->handling == VCL_RET_PIPE);
 
@@ -754,7 +754,7 @@ cnt_recv(struct worker *wrk, struct req *req)
 	case VCL_RET_PASS:
 		req->req_step = R_STP_PASS;
 		return (REQ_FSM_MORE);
-	case VCL_RET_ERROR:
+	case VCL_RET_SYNTH:
 		req->req_step = R_STP_ERROR;
 		return (REQ_FSM_MORE);
 	default:
diff --git a/bin/varnishtest/tests/b00017.vtc b/bin/varnishtest/tests/b00017.vtc
index dbe673b..81776bb 100644
--- a/bin/varnishtest/tests/b00017.vtc
+++ b/bin/varnishtest/tests/b00017.vtc
@@ -5,11 +5,11 @@ varnish v1 -vcl {
 		.host = "${bad_ip}"; .port = "9080";
 	}
 	sub vcl_recv {
-		return (error(888));
+		return (synth(888));
 	}
 
-	sub vcl_error {
-		synthetic("Custom vcl_error's synth output");
+	sub vcl_synth {
+		synthetic("Custom vcl_synth's synth output");
 		return (deliver);
 	}
 } -start
@@ -22,4 +22,4 @@ client c1 {
 	expect resp.http.connection != close
 } -run
 
-varnish v1 -expect s_error == 1
+varnish v1 -expect s_synth == 1
diff --git a/bin/varnishtest/tests/b00019.vtc b/bin/varnishtest/tests/b00019.vtc
index 59ceb61..33d3dbc 100644
--- a/bin/varnishtest/tests/b00019.vtc
+++ b/bin/varnishtest/tests/b00019.vtc
@@ -25,7 +25,7 @@ varnish v1 -vcl+backend {
 	       return (restart);
 	}
 
-	sub vcl_error {
+	sub vcl_synth {
 		if (req.restarts == 2) {
 			set resp.status = 200;
 			set resp.reason = "restart=2";
diff --git a/bin/varnishtest/tests/b00030.vtc b/bin/varnishtest/tests/b00030.vtc
index a7b6c57..1115613 100644
--- a/bin/varnishtest/tests/b00030.vtc
+++ b/bin/varnishtest/tests/b00030.vtc
@@ -10,9 +10,9 @@ server s1 {
 
 varnish v1 -vcl+backend {
 	sub vcl_recv {
-		return (error(200,req.ttl));
+		return (synth(200,req.ttl));
 	}
-	sub vcl_error {
+	sub vcl_synth {
 		set resp.http.x-timestamp = now;
 	}
 } -start
diff --git a/bin/varnishtest/tests/c00022.vtc b/bin/varnishtest/tests/c00022.vtc
index 841d2cf..39ce5dc 100644
--- a/bin/varnishtest/tests/c00022.vtc
+++ b/bin/varnishtest/tests/c00022.vtc
@@ -22,11 +22,11 @@ varnish v1 -vcl+backend {
 	sub vcl_recv {
 		if (req.method == "PURGE") {
 			ban ("req.url == " + req.url);
-			return (error(410));
+			return (synth(410));
 		}
 		if (req.method == "PURGESTR") {
 			ban ("" + req.http.ban);
-			return (error(410));
+			return (synth(410));
 		}
 	}
 } -start
diff --git a/bin/varnishtest/tests/c00024.vtc b/bin/varnishtest/tests/c00024.vtc
index 6167914..80b3c59 100644
--- a/bin/varnishtest/tests/c00024.vtc
+++ b/bin/varnishtest/tests/c00024.vtc
@@ -1,4 +1,4 @@
-varnishtest "Test restart in vcl_error"
+varnishtest "Test restart in vcl_synth"
 
 server s1 {
         rxreq
@@ -8,10 +8,10 @@ server s1 {
 varnish v1 -vcl+backend {
 	sub vcl_recv {
 		if (req.restarts == 0) {
-			return (error(701, "FOO"));
+			return (synth(701, "FOO"));
 		}
 	}
-	sub vcl_error {
+	sub vcl_synth {
 		if (req.restarts < 1) {
 			return (restart);
 		} else {
diff --git a/bin/varnishtest/tests/c00028.vtc b/bin/varnishtest/tests/c00028.vtc
index 91abe45..0ad1466 100644
--- a/bin/varnishtest/tests/c00028.vtc
+++ b/bin/varnishtest/tests/c00028.vtc
@@ -8,7 +8,7 @@ varnish v1 -vcl {
 	sub vcl_backend_fetch {
 	    set bereq.backend = bad;
 	}
-	sub vcl_error {
+	sub vcl_synth {
 	    return (restart);
 	}
  } -start
diff --git a/bin/varnishtest/tests/c00061.vtc b/bin/varnishtest/tests/c00061.vtc
index c99f324..ef2ae76 100644
--- a/bin/varnishtest/tests/c00061.vtc
+++ b/bin/varnishtest/tests/c00061.vtc
@@ -8,7 +8,7 @@ varnish v1 -vcl {
 		return (retry);
 	}
 
-	sub vcl_error {
+	sub vcl_synth {
 		set resp.status = 504;
 	}
 } -start
diff --git a/bin/varnishtest/tests/r00558.vtc b/bin/varnishtest/tests/r00558.vtc
index 3a8ba97..c889e79 100644
--- a/bin/varnishtest/tests/r00558.vtc
+++ b/bin/varnishtest/tests/r00558.vtc
@@ -7,7 +7,7 @@ server s1 {
 
 varnish v1 -vcl+backend {
 	sub vcl_recv {
-		return (error(501));
+		return (synth(501));
 	}
 } -start
 
diff --git a/bin/varnishtest/tests/r00561.vtc b/bin/varnishtest/tests/r00561.vtc
index 1efc0a7..416a97d 100644
--- a/bin/varnishtest/tests/r00561.vtc
+++ b/bin/varnishtest/tests/r00561.vtc
@@ -1,4 +1,4 @@
-varnishtest "Junk request should not go to vcl_error"
+varnishtest "Junk request should not go to vcl_synth"
 
 server s1 {
 	rxreq
@@ -6,7 +6,7 @@ server s1 {
 } -start
 
 varnish v1 -vcl+backend {
-	sub vcl_error {
+	sub vcl_synth {
 		return (restart);
 	}
 } -start
diff --git a/bin/varnishtest/tests/r00769.vtc b/bin/varnishtest/tests/r00769.vtc
index 451029c..6778648 100644
--- a/bin/varnishtest/tests/r00769.vtc
+++ b/bin/varnishtest/tests/r00769.vtc
@@ -8,7 +8,7 @@ server s1 {
 varnish v1 -vcl+backend {
 	sub vcl_recv {
 		if (req.url ~ "^/test1") {
-			return (error(700));
+			return (synth(700));
 		}
 	}
 	sub vcl_backend_response {
@@ -16,7 +16,7 @@ varnish v1 -vcl+backend {
 		set beresp.http.X-status = beresp.status;
 		return (deliver);
 	}
-	sub vcl_error {
+	sub vcl_synth {
 		if (resp.status == 700) {
 			set resp.status=404;
 			set resp.http.X-status = resp.status;
diff --git a/bin/varnishtest/tests/r00896.vtc b/bin/varnishtest/tests/r00896.vtc
index c713a59..0e0c086 100644
--- a/bin/varnishtest/tests/r00896.vtc
+++ b/bin/varnishtest/tests/r00896.vtc
@@ -8,7 +8,7 @@ server s1 {
 varnish v1 -vcl+backend {
 	sub vcl_recv {
 	    if (req.http.host ~ "^(abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij)") {
-		return (error(500,"not ok"));
+		return (synth(500,"not ok"));
 		}
 	}
 } -start
diff --git a/bin/varnishtest/tests/r00936.vtc b/bin/varnishtest/tests/r00936.vtc
index 7e6ad64..ddb2f79 100644
--- a/bin/varnishtest/tests/r00936.vtc
+++ b/bin/varnishtest/tests/r00936.vtc
@@ -10,7 +10,7 @@ varnish v1 -errvcl {'synthetic': not a valid action in method 'vcl_recv'} {
 
 	sub vcl_recv {
 		synthetic("HELLOO");
-		return (error(503));
+		return (synth(503));
 	}
 }
 
diff --git a/bin/varnishtest/tests/r00965.vtc b/bin/varnishtest/tests/r00965.vtc
index d9cbdba..e094e3b 100644
--- a/bin/varnishtest/tests/r00965.vtc
+++ b/bin/varnishtest/tests/r00965.vtc
@@ -27,19 +27,19 @@ varnish v1 -vcl+backend {
 		}
 	}
 
-	sub vcl_error {
+	sub vcl_synth {
 		if (resp.status == 988) { return (restart); }
 	}
 
 	sub vcl_miss {
 		if (req.http.X-Banned == "check") {
-			return (error(988,"restarting"));
+			return (synth(988,"restarting"));
 		}
 	}
 
 	sub vcl_pass {
 		if (req.http.X-Banned == "check") {
-			return (error(988,"restarting"));
+			return (synth(988,"restarting"));
 		}
 	}
 } -start
diff --git a/bin/varnishtest/tests/r01027.vtc b/bin/varnishtest/tests/r01027.vtc
index d6d0991..716e6f0 100644
--- a/bin/varnishtest/tests/r01027.vtc
+++ b/bin/varnishtest/tests/r01027.vtc
@@ -1,9 +1,9 @@
 varnishtest "Test if you can error in vcl_deliver"
 
-varnish v1 -errvcl {Invalid return "error"} {
+varnish v1 -errvcl {Invalid return "synth"} {
 	backend b { .host = "127.0.0.1"; }
 	sub vcl_deliver {
-		return (error(201,"ok"));
+		return (synth(201,"ok"));
 	}
 }
 
diff --git a/bin/varnishtest/tests/r01030.vtc b/bin/varnishtest/tests/r01030.vtc
index 543f93f..a84e3a5 100644
--- a/bin/varnishtest/tests/r01030.vtc
+++ b/bin/varnishtest/tests/r01030.vtc
@@ -16,7 +16,7 @@ varnish v1 -vcl+backend {
 	sub vcl_recv {
 		if (req.method == "BAN") {
 			ban("obj.http.url ~ /");
-			return (error(201,"banned"));
+			return (synth(201,"banned"));
 		}
 		return (hash);
 	}
diff --git a/bin/varnishtest/tests/r01287.vtc b/bin/varnishtest/tests/r01287.vtc
index 7e329ac..bfb220a 100644
--- a/bin/varnishtest/tests/r01287.vtc
+++ b/bin/varnishtest/tests/r01287.vtc
@@ -5,9 +5,9 @@ server s1 {
 
 varnish v1 -vcl+backend {
 	sub vcl_recv {
-		return (error(200, "OK"));
+		return (synth(200, "OK"));
 	}
-	sub vcl_error {
+	sub vcl_synth {
 		synthetic(resp.http.blank);
 		return (deliver);
 	}
diff --git a/bin/varnishtest/tests/r01367.vtc b/bin/varnishtest/tests/r01367.vtc
index 3a57258..18c9ca5 100644
--- a/bin/varnishtest/tests/r01367.vtc
+++ b/bin/varnishtest/tests/r01367.vtc
@@ -6,7 +6,7 @@ server s1 {
 } -start
 
 varnish v1 -vcl+backend {
-	sub vcl_error {
+	sub vcl_synth {
 		return (restart);
 	}
 } -start
diff --git a/bin/varnishtest/tests/r01395.vtc b/bin/varnishtest/tests/r01395.vtc
index acad0fe..591bb2e 100644
--- a/bin/varnishtest/tests/r01395.vtc
+++ b/bin/varnishtest/tests/r01395.vtc
@@ -1,4 +1,4 @@
-varnishtest "Test vcl_error is called even if vcl_backend_fetch failed"
+varnishtest "Test vcl_synth is called even if vcl_backend_fetch failed"
 
 varnish v1 -vcl {
 	backend default {
diff --git a/bin/varnishtest/tests/v00011.vtc b/bin/varnishtest/tests/v00011.vtc
index 3a3d842..0f470ec 100644
--- a/bin/varnishtest/tests/v00011.vtc
+++ b/bin/varnishtest/tests/v00011.vtc
@@ -18,7 +18,7 @@ varnish v1 -vcl+backend {
 			ban("req.url == bar //");
 			ban("foo == bar //");
 			ban("req.url ~ ^/$");
-			return (error(209,"foo"));
+			return (synth(209,"foo"));
 		}
 	}
 
diff --git a/bin/varnishtest/tests/v00014.vtc b/bin/varnishtest/tests/v00014.vtc
index 3a48188..8f776b7 100644
--- a/bin/varnishtest/tests/v00014.vtc
+++ b/bin/varnishtest/tests/v00014.vtc
@@ -28,9 +28,9 @@ varnish v1 -vcl {
 
 	sub vcl_recv {
 		if (std.healthy(default)) {
-			return(error(200,"Backend healthy"));
+			return(synth(200,"Backend healthy"));
 		} else {
-			return(error(500,"Backend sick"));
+			return(synth(500,"Backend sick"));
 		}
 	}
 } -start
diff --git a/bin/varnishtest/tests/v00018.vtc b/bin/varnishtest/tests/v00018.vtc
index 9e97762..004ec15 100644
--- a/bin/varnishtest/tests/v00018.vtc
+++ b/bin/varnishtest/tests/v00018.vtc
@@ -8,8 +8,8 @@ varnish v1 -vcl {
 
 varnish v1 -vcl {
 	backend b { .host = "127.0.0.1"; }
-	sub vcl_miss { return(error(100,req.url)); }
-	sub vcl_hit { return(error(100,"the butter please")); }
+	sub vcl_miss { return(synth(100,req.url)); }
+	sub vcl_hit { return(synth(100,"the butter please")); }
 }
 
 varnish v1 -errvcl {Variable 'now' is read only.} {
@@ -39,7 +39,7 @@ varnish v1 -errvcl {Expected '=' got '+='} {
 
 varnish v1 -errvcl {Expected ';' got 'if'} {
 	backend b { .host = "127.0.0.1"; }
-	/* XXX: This should not really be an error */
+	/* XXX: This should not really be an synth */
 	sub vcl_recv { set req.url = "foo" if "bar"; }
 }
 
@@ -100,15 +100,10 @@ varnish v1 -errvcl {Expected an action, 'if', '{' or '}'} {
 
 varnish v1 -errvcl {Unknown token '<<' when looking for STRING_LIST} {
 	backend b { .host = "127.0.0.1"; }
-	sub vcl_error { synthetic( << "foo"; }
+	sub vcl_synth { synthetic( << "foo"; }
 }
 
 varnish v1 -errvcl {Syntax has changed, use:} {
 	backend b { .host = "127.0.0.1"; }
-	sub vcl_recv { error 200; }
-}
-
-varnish v1 -errvcl {Syntax has changed, use:} {
-	backend b { .host = "127.0.0.1"; }
-	sub vcl_recv { return(error); }
+	sub vcl_recv { return(synth); }
 }
diff --git a/bin/varnishtest/tests/v00024.vtc b/bin/varnishtest/tests/v00024.vtc
index f112559..e502eee 100644
--- a/bin/varnishtest/tests/v00024.vtc
+++ b/bin/varnishtest/tests/v00024.vtc
@@ -9,7 +9,7 @@ server s1 {
 varnish v1 -vcl+backend {
 	sub vcl_recv {
 		if (req.http.etag == req.http.if-none-match) {
-			return(error(400,"FOO"));
+			return(synth(400,"FOO"));
 		}
 	}
 } -start
diff --git a/include/tbl/acct_fields.h b/include/tbl/acct_fields.h
index 1184240..ac205a7 100644
--- a/include/tbl/acct_fields.h
+++ b/include/tbl/acct_fields.h
@@ -35,6 +35,6 @@ ACCT(req)
 ACCT(pipe)
 ACCT(pass)
 ACCT(fetch)
-ACCT(error)
+ACCT(synth)
 ACCT(hdrbytes)
 ACCT(bodybytes)
diff --git a/include/tbl/vsc_f_main.h b/include/tbl/vsc_f_main.h
index d72b325..3874ec1 100644
--- a/include/tbl/vsc_f_main.h
+++ b/include/tbl/vsc_f_main.h
@@ -380,8 +380,8 @@ VSC_F(s_fetch,			uint64_t, 1, 'a', info,
     "Total fetch",
 	""
 )
-VSC_F(s_error,			uint64_t, 1, 'a', info,
-    "Total error",
+VSC_F(s_synth,			uint64_t, 1, 'a', info,
+    "Total synth",
 	""
 )
 VSC_F(s_hdrbytes,		uint64_t, 1, 'a', info,
diff --git a/lib/libvcc/generate.py b/lib/libvcc/generate.py
index 523f05f..7b9ab4e 100755
--- a/lib/libvcc/generate.py
+++ b/lib/libvcc/generate.py
@@ -84,15 +84,15 @@ returns =(
 
 	('recv',
 		"C",
-		('error', 'pass', 'pipe', 'hash', 'purge',)
+		('synth', 'pass', 'pipe', 'hash', 'purge',)
 	),
 	('pipe',
 		"C",
-		('error', 'pipe',)
+		('synth', 'pipe',)
 	),
 	('pass',
 		"C",
-		('error', 'restart', 'fetch',)
+		('synth', 'restart', 'fetch',)
 	),
 	('hash',
 		"C",
@@ -100,21 +100,21 @@ returns =(
 	),
 	('purge',
 		"C",
-		('error', 'fetch',)
+		('synth', 'fetch',)
 	),
 	('miss',
 		"C",
-		('error', 'restart', 'pass', 'fetch',)
+		('synth', 'restart', 'pass', 'fetch',)
 	),
 	('hit',
 		"C",
-		('error', 'restart', 'pass', 'fetch', 'deliver',)
+		('synth', 'restart', 'pass', 'fetch', 'deliver',)
 	),
 	('deliver',
 		"C",
 		('restart', 'deliver',)
 	),
-	('error',
+	('synth',
 		"C",
 		('restart', 'deliver',)
 	),
@@ -254,8 +254,8 @@ sp_variables = [
 	),
 	('req.esi',
 		'BOOL',
-		( 'recv', 'backend_response', 'deliver', 'error',),
-		( 'recv', 'backend_response', 'deliver', 'error',), """
+		( 'recv', 'backend_response', 'deliver', 'synth',),
+		( 'recv', 'backend_response', 'deliver', 'synth',), """
 		Boolean. Set to false to disable ESI processing
 		regardless of any value in beresp.do_esi. Defaults
 		to true. This variable is subject to change in
@@ -497,15 +497,15 @@ sp_variables = [
 	),
 	('obj.status',
 		'INT',
-		( 'error',),
-		( 'error',), """
+		( 'synth',),
+		( 'synth',), """
 		The HTTP status code returned by the server.
 		"""
 	),
 	('obj.reason',
 		'STRING',
-		( 'error',),
-		( 'error',), """
+		( 'synth',),
+		( 'synth',), """
 		The HTTP status message returned by the server.
 		"""
 	),
@@ -563,29 +563,29 @@ sp_variables = [
 	),
 	('resp.proto',
 		'STRING',
-		( 'deliver', 'error', ),
-		( 'deliver', 'error', ), """
+		( 'deliver', 'synth', ),
+		( 'deliver', 'synth', ), """
 		The HTTP protocol version to use for the response.
 		"""
 	),
 	('resp.status',
 		'INT',
-		( 'deliver', 'error', ),
-		( 'deliver', 'error', ), """
+		( 'deliver', 'synth', ),
+		( 'deliver', 'synth', ), """
 		The HTTP status code that will be returned.
 		"""
 	),
 	('resp.reason',
 		'STRING',
-		( 'deliver', 'error', ),
-		( 'deliver', 'error', ), """
+		( 'deliver', 'synth', ),
+		( 'deliver', 'synth', ), """
 		The HTTP status message that will be returned.
 		"""
 	),
 	('resp.http.',
 		'HEADER',
-		( 'deliver', 'error', ),
-		( 'deliver', 'error', ), """
+		( 'deliver', 'synth', ),
+		( 'deliver', 'synth', ), """
 		The corresponding HTTP header.
 		"""
 	),
@@ -1011,6 +1011,7 @@ def one_var(nm, spec):
 		if nm == i[0]:
 			fh.write("VCL_" + typ +
 			    " VRT_r_%s(const struct vrt_ctx *);\n" % cnam )
+	print(spec)
 	restrict(fo, spec[2])
 
 	if len(spec[3]) == 0:
diff --git a/lib/libvcc/vcc_action.c b/lib/libvcc/vcc_action.c
index 5ed9146..602beff 100644
--- a/lib/libvcc/vcc_action.c
+++ b/lib/libvcc/vcc_action.c
@@ -54,21 +54,6 @@ parse_call(struct vcc *tl)
 
 /*--------------------------------------------------------------------*/
 
-static void
-parse_error(struct vcc *tl)
-{
-
-	VSB_printf(tl->sb,
-	    "Syntax has changed, use:\n"
-	    "\treturn(error(999));\n"
-	    "or\n"
-	    "\treturn(error(999, \"Response text\"));\n");
-	vcc_ErrWhere(tl, tl->t);
-	return;
-}
-
-/*--------------------------------------------------------------------*/
-
 static const struct arith {
 	enum var_type		type;
 	unsigned		oper;
@@ -311,14 +296,14 @@ parse_return(struct vcc *tl)
 	ExpectErr(tl, ID);
 
 	/* 'error' gets special handling, to allow optional status/response */
-	if (vcc_IdIs(tl->t, "error")) {
+	if (vcc_IdIs(tl->t, "synth")) {
 		vcc_NextToken(tl);
 		if (tl->t->tok == ')') {
 			VSB_printf(tl->sb,
 			    "Syntax has changed, use:\n"
-			    "\treturn(error(999));\n"
+			    "\treturn(synth(999));\n"
 			    "or\n"
-			    "\treturn(error(999, \"Response text\"));\n");
+			    "\treturn(synth(999, \"Response text\"));\n");
 			vcc_ErrWhere(tl, tl->t);
 			return;
 		}
@@ -340,9 +325,9 @@ parse_return(struct vcc *tl)
 		ExpectErr(tl, ')');
 		vcc_NextToken(tl);
 		Fb(tl, 1, ");\n");
-		Fb(tl, 1, "VRT_handling(ctx, VCL_RET_ERROR);\n");
+		Fb(tl, 1, "VRT_handling(ctx, VCL_RET_SYNTH);\n");
 		Fb(tl, 1, "return (1);\n");
-		vcc_ProcAction(tl->curproc, VCL_RET_ERROR, tl->t);
+		vcc_ProcAction(tl->curproc, VCL_RET_SYNTH, tl->t);
 		ExpectErr(tl, ')');
 		vcc_NextToken(tl);
 		return;
@@ -422,7 +407,6 @@ static struct action_table {
 	/* Keep list sorted from here */
 	{ "ban",		parse_ban },
 	{ "call",		parse_call },
-	{ "error",		parse_error },
 	{ "hash_data",		parse_hash_data, VCL_MET_HASH },
 	{ "new",		parse_new, VCL_MET_INIT},
 	{ "purge",		parse_purge, VCL_MET_MISS | VCL_MET_HIT },
@@ -430,7 +414,7 @@ static struct action_table {
 	{ "rollback",		parse_rollback },
 	{ "set",		parse_set },
 	{ "synthetic",		parse_synthetic,
-		VCL_MET_ERROR | VCL_MET_BACKEND_ERROR },
+		VCL_MET_SYNTH | VCL_MET_BACKEND_ERROR },
 	{ "unset",		parse_unset },
 	{ NULL,			NULL }
 };



From phk at FreeBSD.org  Tue Apr  1 13:09:50 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 01 Apr 2014 15:09:50 +0200
Subject: [4.0] 3c33b93 Forgot to change error -> synth here.
Message-ID: <E1WUyRq-00033B-Bd@project.varnish-software.com>


commit 3c33b93461793ad69c5a3aa644b44f3c64e07855
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Mar 18 16:42:36 2014 +0000

    Forgot to change error -> synth here.

diff --git a/bin/varnishd/builtin.vcl b/bin/varnishd/builtin.vcl
index cdac8d9..5ac9664 100644
--- a/bin/varnishd/builtin.vcl
+++ b/bin/varnishd/builtin.vcl
@@ -91,7 +91,7 @@ sub vcl_hash {
 }
 
 sub vcl_purge {
-    return (error(200, "Purged"));
+    return (synth(200, "Purged"));
 }
 
 sub vcl_hit {



From lkarsten at varnish-software.com  Tue Apr  1 13:09:50 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:50 +0200
Subject: [4.0] f89015a Add major version in Via header
Message-ID: <E1WUyRq-00033K-Cb@project.varnish-software.com>


commit f89015acfc28b6fe301b22a56174e9b85785d29a
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Tue Mar 18 16:12:22 2014 +0100

    Add major version in Via header
    
    To help track our adoption rate of the new Varnish versions, add the major
    version in the default Via header added by us.

diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index 792b1ec..6650c31 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -123,7 +123,7 @@ cnt_deliver(struct worker *wrk, struct req *req)
 	http_PrintfHeader(req->resp, "Age: %.0f",
 	    req->t_resp - req->obj->exp.t_origin);
 
-	http_SetHeader(req->resp, "Via: 1.1 varnish");
+	http_SetHeader(req->resp, "Via: 1.1 varnish (v4)");
 
 	if (cache_param->http_gzip_support && req->obj->gziped &&
 	    !RFC2616_Req_Gzip(req->http))



From fgsch at lodoss.net  Tue Apr  1 13:09:50 2014
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Tue, 01 Apr 2014 15:09:50 +0200
Subject: [4.0] 101a5f7 Fix spelling and reformat a bit
Message-ID: <E1WUyRq-00033T-Dc@project.varnish-software.com>


commit 101a5f7e8452bfae879b1e012ebc8e0ccf049b0a
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Wed Mar 19 00:07:01 2014 +0000

    Fix spelling and reformat a bit

diff --git a/CONTRIBUTING b/CONTRIBUTING
index 2c2db8c..8199602 100644
--- a/CONTRIBUTING
+++ b/CONTRIBUTING
@@ -1,8 +1,9 @@
 Contributing to Varnish Cache
 =============================
 
-To contribute to Varnish, read https://www.varnish-cache.org/trac/wiki/Contributing
-as a staring point and follow relevant parts linked there.
+To contribute to Varnish, read
+https://www.varnish-cache.org/trac/wiki/Contributing as a starting
+point and follow relevant parts linked there.
 
 Varnish Cache is developed with Git hosted on our own infrastructure.
 
@@ -12,7 +13,7 @@ Official development tree is here:
 
 Patches can be sent to varnish-dev at varnish-cache.org.
 
-There is a READ-ONLY git-mirror on Github.  Please do not send contributions
-there, we need them by email to -dev@ for now.
+There is a READ-ONLY git-mirror on Github.  Please do not send
+contributions there, we need them by email to varnish-dev@ for now.
 
 



From fgsch at lodoss.net  Tue Apr  1 13:09:50 2014
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Tue, 01 Apr 2014 15:09:50 +0200
Subject: [4.0] 243e690 Better example
Message-ID: <E1WUyRq-00033c-En@project.varnish-software.com>


commit 243e6909fd4c742158b46eb1e2d9b59b5dd25ccd
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Wed Mar 19 00:07:36 2014 +0000

    Better example
    
    Don't use {} in one-liners

diff --git a/doc/sphinx/whats-new/upgrading.rst b/doc/sphinx/whats-new/upgrading.rst
index a0da998..5f52318 100644
--- a/doc/sphinx/whats-new/upgrading.rst
+++ b/doc/sphinx/whats-new/upgrading.rst
@@ -53,7 +53,7 @@ The synthetic keyword is now a function::
     if (resp.status == 799) {
         set resp.status = 200;
         set resp.http.Content-Type = "text/plain; charset=utf-8";
-        synthetic({"You are "} + client.ip);
+        synthetic("You are " + client.ip);
         return (deliver);
     }
 



From tfheen at err.no  Tue Apr  1 13:09:50 2014
From: tfheen at err.no (Tollef Fog Heen)
Date: Tue, 01 Apr 2014 15:09:50 +0200
Subject: [4.0] 83319e4 Remove obj.last_use
Message-ID: <E1WUyRq-00033l-Gd@project.varnish-software.com>


commit 83319e42e9e5a274679a6aa578635b1335407658
Author: Tollef Fog Heen <tfheen at fastly.com>
Date:   Tue Mar 18 17:25:10 2014 +0100

    Remove obj.last_use

diff --git a/bin/varnishd/builtin.vcl b/bin/varnishd/builtin.vcl
index 5ac9664..92f4641 100644
--- a/bin/varnishd/builtin.vcl
+++ b/bin/varnishd/builtin.vcl
@@ -118,7 +118,6 @@ sub vcl_deliver {
      * Enable them only if you need them.
      *
      * set obj.hits = obj.hits + 1;
-     * set obj.last_use = now;
      */
     return (deliver);
 }
diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index eac7546..ce90e58 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -610,7 +610,6 @@ struct object {
 
 	/* VCL only variables */
 	long			hits;
-	double			last_use;
 
 	double			last_modified;
 
diff --git a/bin/varnishd/cache/cache_cli.c b/bin/varnishd/cache/cache_cli.c
index 55145cd..8aa022b 100644
--- a/bin/varnishd/cache/cache_cli.c
+++ b/bin/varnishd/cache/cache_cli.c
@@ -200,7 +200,6 @@ cli_debug_sizeof(struct cli *cli, const char * const *av, void *priv)
 	OFOF(struct object, http);
 	OFOF(struct object, store);
 	OFOF(struct object, esidata);
-	OFOF(struct object, last_use);
 #endif
 #undef OFOF
 #endif
diff --git a/bin/varnishd/cache/cache_vrt_var.c b/bin/varnishd/cache/cache_vrt_var.c
index 8a999c9..0bcd5d5 100644
--- a/bin/varnishd/cache/cache_vrt_var.c
+++ b/bin/varnishd/cache/cache_vrt_var.c
@@ -592,8 +592,6 @@ VRT_r_obj_##field(const struct vrt_ctx *ctx)				\
 
 VOBJ_L(long, hits)
 VOBJ_R(long, hits)
-VOBJ_L(double, last_use)
-VOBJ_R(double, last_use)
 
 unsigned
 VRT_r_obj_uncacheable(const struct vrt_ctx *ctx)
diff --git a/bin/varnishd/storage/stevedore.c b/bin/varnishd/storage/stevedore.c
index c2788a9..30ae453 100644
--- a/bin/varnishd/storage/stevedore.c
+++ b/bin/varnishd/storage/stevedore.c
@@ -286,7 +286,6 @@ STV_MkObject(struct stevedore *stv, struct busyobj *bo,
 	HTTP_Setup(o->http, bo->ws_o, bo->vsl, SLT_ObjMethod);
 	o->http->magic = HTTP_MAGIC;
 	o->exp = bo->exp;
-	o->last_use = bo->t_fetch;
 	VTAILQ_INIT(&o->store);
 	bo->stats->n_object++;
 
diff --git a/bin/varnishtest/tests/v00013.vtc b/bin/varnishtest/tests/v00013.vtc
index fa3afda..0bb2858 100644
--- a/bin/varnishtest/tests/v00013.vtc
+++ b/bin/varnishtest/tests/v00013.vtc
@@ -1,4 +1,4 @@
-varnishtest "Check obj.hits and obj.last_use"
+varnishtest "Check obj.hits"
 
 server s1 {
 	rxreq
@@ -13,9 +13,7 @@ varnish v1 -vcl+backend {
 
 	sub vcl_deliver {
 		set resp.http.foo = obj.hits;
-		set resp.http.bar = now - obj.last_use;
 		set obj.hits = obj.hits + 1;
-		set obj.last_use = now;
 	}
 } -start
 
diff --git a/lib/libvcc/generate.py b/lib/libvcc/generate.py
index 7b9ab4e..9d8239b 100755
--- a/lib/libvcc/generate.py
+++ b/lib/libvcc/generate.py
@@ -546,15 +546,6 @@ sp_variables = [
 		( 'hit', ), """
 		"""
 	),
-	('obj.last_use',
-		'TIME',
-		( 'hit', 'deliver',),
-		( 'hit', 'deliver',), """
-		The approximate time elapsed since the object was
-		last requests, in seconds. This variable is also
-		available in vcl_deliver.
-		"""
-	),
 	('obj.uncacheable',
 		'BOOL',
 		( 'hit', ),



From tfheen at err.no  Tue Apr  1 13:09:50 2014
From: tfheen at err.no (Tollef Fog Heen)
Date: Tue, 01 Apr 2014 15:09:50 +0200
Subject: [4.0] fb4c1f8 Rework a bit of the build system for vmod docs
Message-ID: <E1WUyRq-00033u-IM@project.varnish-software.com>


commit fb4c1f855f0d21bf56799f57bf4232b736b90a05
Author: Tollef Fog Heen <tfheen at fastly.com>
Date:   Wed Mar 19 10:08:09 2014 +0100

    Rework a bit of the build system for vmod docs
    
    Move the vmod man pages to man/ and build and ship the vmod_director
    page.  Also make sure we use the vmod_std docs from the .vcc file, not
    the old copy in docs/

diff --git a/doc/sphinx/Makefile.am b/doc/sphinx/Makefile.am
index 64e6020..716e8da 100644
--- a/doc/sphinx/Makefile.am
+++ b/doc/sphinx/Makefile.am
@@ -127,7 +127,6 @@ EXTRA_DIST = \
 	reference/varnishtop.rst \
 	reference/vcl.rst \
 	reference/vmod.rst \
-	reference/vmod_std.rst \
 	reference/vsl-query.rst \
 	reference/vsl.rst \
 	reference/vsm.rst \
@@ -212,5 +211,13 @@ include/varnishhist_synopsis.rst: $(top_builddir)/bin/varnishhist/varnishhist_op
 BUILT_SOURCES += include/varnishhist_options.rst \
 	 include/varnishhist_synopsis.rst
 
+include/vmod_std.rst: $(top_builddir)/lib/libvmod_std/vmod_std.rst
+	cp $? $@
+BUILT_SOURCES += include/vmod_std.rst
+
+include/vmod_directors.rst: $(top_builddir)/lib/libvmod_directors/vmod_directors.rst
+	cp $? $@
+BUILT_SOURCES += include/vmod_directors.rst
+
 EXTRA_DIST += $(BUILT_SOURCES)
 MAINTAINERCLEANFILES = $(EXTRA_DIST)
diff --git a/doc/sphinx/reference/index.rst b/doc/sphinx/reference/index.rst
index 1537fb4..20ac975 100644
--- a/doc/sphinx/reference/index.rst
+++ b/doc/sphinx/reference/index.rst
@@ -21,7 +21,8 @@ The Varnish Reference Manual
 	varnishtop.rst
 	vsm.rst
 	vmod.rst
-	vmod_std.rst
+	../include/vmod_std.rst
+	../include/vmod_directors.rst
 	vsl.rst
 	vsl-query.rst
 
diff --git a/doc/sphinx/reference/vmod_std.rst b/doc/sphinx/reference/vmod_std.rst
deleted file mode 100644
index b8f2aa3..0000000
--- a/doc/sphinx/reference/vmod_std.rst
+++ /dev/null
@@ -1,188 +0,0 @@
-========
-vmod_std
-========
-
------------------------
-Varnish Standard Module
------------------------
-
-SYNOPSIS
-========
-
-import std [from "path"] ;
-
-DESCRIPTION
-===========
-
-Vmod_std contains basic functions which are part and parcel of Varnish,
-but which for reasons of architecture fit better in a VMOD.
-
-One particular class of functions in vmod_std is the conversions functions
-which all have the form::
-
-	TYPE type(STRING, TYPE)
-
-These functions attempt to convert STRING to the TYPE, and if that fails,
-they return the second argument, which must have the given TYPE.
-
-FUNCTIONS
-=========
-
-toupper
--------
-
-Prototype
-	STRING toupper(STRING s)
-Description
-	Converts the string *s* to upper case.
-Example
-	set beresp.http.x-scream = std.toupper("yes!");
-
-tolower
--------
-Prototype
-	STRING tolower(STRING s)
-Description
-	Converts the string *s* to lower case.
-Example
-	set beresp.http.x-nice = std.tolower("VerY");
-
-set_ip_tos
-----------
-Prototype
-	VOID set_ip_tos(INT i)
-Description
-	Sets the Type-of-Service flag for the current session. Please
-	note that the TOS flag is not removed by the end of the
-	request so probably want to set it on every request should you
-	utilize it.
-Example
-	| if (req.url ~ ^/slow/) {
-	|    std.set_ip_tos(0x0);
-	| }
-
-random
-------
-Prototype
-	REAL random(REAL a, REAL b)
-Description
-	Returns a random REAL number between *a* and *b*.
-Example
-	set beresp.http.x-random-number = std.random(1, 100);
-
-log
----
-Prototype
-	VOID log(STRING string)
-Description
-	Logs *string* to the shared memory log, using VSL tag *SLT_VCL_Log*.
-Example
-	std.log("Something fishy is going on with the vhost " + req.host);
-
-syslog
-------
-Prototype
-	VOID syslog(INT priority, STRING string)
-Description
-	Logs *string* to syslog marked with *priority*.  See your
-	system's syslog.h file for the legal values of *priority*.
-Example
-	std.syslog(8 + 1, "Something is wrong");
-
-fileread
---------
-Prototype
-	STRING fileread(STRING filename)
-Description
-	Reads a file and returns a string with the content. Please
-	note that it is not recommended to send variables to this
-	function the caching in the function doesn't take this into
-	account. Also, files are not re-read.
-Example
-	set beresp.http.x-served-by = std.fileread("/etc/hostname");
-
-collect
--------
-Prototype
-	VOID collect(HEADER header)
-Description
-	Collapses the header, joining the headers into one.
-Example
-	std.collect(req.http.cookie);
-	This will collapse several Cookie: headers into one, long
-	cookie header.
-
-
-CONVERSION FUNCTIONS
-====================
-
-duration
---------
-Prototype
-	DURATION duration(STRING s, DURATION fallback)
-Description
-	Converts the string *s* to seconds. *s* must be quantified
-	with ms (milliseconds), s (seconds), m (minutes), h (hours),
-	d (days) or w (weeks) units. If *s* fails to parse,
-	*fallback* will be returned.
-Example
-	set beresp.ttl = std.duration("1w", 3600s);
-
-integer
---------
-Prototype
-	INT integer(STRING s, INT fallback)
-Description
-	Converts the string *s* to an integer.  If *s* fails to parse,
-	*fallback* will be returned.
-Example
-	if (std.integer(beresp.http.x-foo, 0) > 5) { ... }
-
-ip
---
-Prototype
-	IP ip(STRING s, IP fallback)
-Description
-	Converts string *s* to the first IP number returned by
-	the system library function getaddrinfo(3).  If conversion
-	fails, *fallback* will be returned.
-Example
-	if (std.ip(req.http.X-forwarded-for, "0.0.0.0") ~ my_acl) { ... }
-
-healthy
--------
-Prototype
-	BOOL healthy(BACKEND backend)
- 
-Description
-	Returns true if the backend is healthy.
-
-port
-----
-Prototype
-	INT port(IP ip)
-
-Description
-	Returns the port number of an IP address.
-
-
-SEE ALSO
-========
-
-* vcl(7)
-* varnishd(1)
-
-HISTORY
-=======
-
-The Varnish standard module was released along with Varnish Cache 3.0.
-This manual page was written by Per Buer with help from Martin Blix
-Grydeland.
-
-COPYRIGHT
-=========
-
-This document is licensed under the same licence as Varnish
-itself. See LICENCE for details.
-
-* Copyright (c) 2011-2014 Varnish Software
diff --git a/lib/libvmod_directors/Makefile.am b/lib/libvmod_directors/Makefile.am
index 6a85ad5..05912e0 100644
--- a/lib/libvmod_directors/Makefile.am
+++ b/lib/libvmod_directors/Makefile.am
@@ -28,7 +28,7 @@ nodist_libvmod_directors_la_SOURCES = \
 # BUILT_SOURCES is only a hack and dependency tracking does not help for the first build
 vdir.lo fall_back.lo hash.lo random.lo round_robin.lo: vcc_if.h
 
-vcc_if.c vcc_if.h: $(vmodtool) $(vmod_srcdir)/vmod.vcc
+vcc_if.c vcc_if.h vmod_directors.man.rst: $(vmodtool) $(vmod_srcdir)/vmod.vcc
 	@PYTHON@ $(vmodtool) $(vmod_srcdir)/vmod.vcc
 
 EXTRA_DIST = vmod.vcc
diff --git a/lib/libvmod_std/Makefile.am b/lib/libvmod_std/Makefile.am
index 8967bf5..07dda95 100644
--- a/lib/libvmod_std/Makefile.am
+++ b/lib/libvmod_std/Makefile.am
@@ -7,8 +7,6 @@ AM_CPPFLAGS = \
 	-I$(top_srcdir)/bin/varnishd \
 	-I$(top_builddir)/include
 
-dist_man_MANS = vmod_std.3
-
 vmoddir = $(pkglibdir)/vmods
 vmod_srcdir = $(top_srcdir)/lib/libvmod_std
 vmodtool = $(top_srcdir)/lib/libvcc/vmodtool.py
@@ -36,6 +34,3 @@ EXTRA_DIST = vmod.vcc
 CLEANFILES = $(builddir)/vcc_if.c $(builddir)/vcc_if.h \
 	$(builddir)/vmod_std.rst \
 	$(builddir)/vmod_std.man.rst
-
-vmod_std.3: $(top_srcdir)/doc/sphinx/reference/vmod_std.rst
-	${RST2MAN} $? $@
diff --git a/man/Makefile.am b/man/Makefile.am
index 581a9de..c25804e 100644
--- a/man/Makefile.am
+++ b/man/Makefile.am
@@ -19,7 +19,9 @@ dist_man_MANS = \
 	varnishncsa.1 \
 	varnishstat.1 \
 	varnishtest.1 \
-	varnishtop.1
+	varnishtop.1 \
+	vmod_directors.3 \
+	vmod_std.3
 
 MAINTAINERCLEANFILES = $(dist_man_MANS)
 
@@ -80,3 +82,9 @@ varnishhist.1: \
 	$(top_srcdir)/doc/sphinx/include/varnishhist_options.rst \
 	$(top_srcdir)/doc/sphinx/include/varnishhist_synopsis.rst
 	${RST2MAN} $(top_srcdir)/doc/sphinx/reference/varnishhist.rst $@
+
+vmod_std.3: $(top_srcdir)/lib/libvmod_std/vmod_std.man.rst
+	${RST2MAN} $? $@
+
+vmod_directors.3: $(top_srcdir)/lib/libvmod_directors/vmod_directors.man.rst
+	${RST2MAN} $? $@



From martin at varnish-software.com  Tue Apr  1 13:09:50 2014
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Tue, 01 Apr 2014 15:09:50 +0200
Subject: [4.0] 0f90ee2 Reset busyobj state to BOS_INVALID in VBO_GetBusyObj
Message-ID: <E1WUyRq-000344-JY@project.varnish-software.com>


commit 0f90ee223daf97d4e6d83a78ae919a95e590bcca
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Wed Mar 19 10:23:11 2014 +0100

    Reset busyobj state to BOS_INVALID in VBO_GetBusyObj
    
    Fixes: #1458

diff --git a/bin/varnishd/cache/cache_busyobj.c b/bin/varnishd/cache/cache_busyobj.c
index db1c7a1..d8be0f6 100644
--- a/bin/varnishd/cache/cache_busyobj.c
+++ b/bin/varnishd/cache/cache_busyobj.c
@@ -109,6 +109,7 @@ VBO_GetBusyObj(struct worker *wrk, const struct req *req)
 	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
 	AZ(bo->refcount);
 
+	bo->state = BOS_INVALID;
 	bo->refcount = 1;
 
 	p = (void*)(bo + 1);



From perbu at varnish-software.com  Tue Apr  1 13:09:50 2014
From: perbu at varnish-software.com (Per Buer)
Date: Tue, 01 Apr 2014 15:09:50 +0200
Subject: [4.0] 493b979 Document the version marker
Message-ID: <E1WUyRq-00034D-Kv@project.varnish-software.com>


commit 493b979fc5b01ce2203ad91f85a107e22281bcda
Author: Per Buer <perbu at varnish-software.com>
Date:   Wed Mar 19 11:28:09 2014 +0100

    Document the version marker

diff --git a/doc/sphinx/reference/vcl.rst b/doc/sphinx/reference/vcl.rst
index 6f91b91..43295bb 100644
--- a/doc/sphinx/reference/vcl.rst
+++ b/doc/sphinx/reference/vcl.rst
@@ -40,6 +40,10 @@ In addition VCL has the following constructs:
 
 Note that are no loops or iterators of any kind in VCL.
 
+Each VCL file must start by declaring its version. For Varnish 4.0 you
+must include a "vcl 4.0;" at the top of the file.
+
+
 Operators
 ---------
 



From nils.goroll at uplex.de  Tue Apr  1 13:09:50 2014
From: nils.goroll at uplex.de (Nils Goroll)
Date: Tue, 01 Apr 2014 15:09:50 +0200
Subject: [4.0] 1b34638 split DEVELOPER_CFLAGS to multiple lines for legibility
Message-ID: <E1WUyRq-00034M-ON@project.varnish-software.com>


commit 1b34638e87c3694138fa9ac160eeec3e289ad8f1
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Wed Mar 19 18:44:10 2014 +0100

    split DEVELOPER_CFLAGS to multiple lines for legibility

diff --git a/configure.ac b/configure.ac
index 6f43695..55c62ab 100644
--- a/configure.ac
+++ b/configure.ac
@@ -465,7 +465,29 @@ AX_CHECK_COMPILE_FLAG([-Werror=unused-result],
       OCFLAGS="${OCFLAGS} -Wno-unused-result"])])
 
 # This corresponds to FreeBSD's WARNS level 6
-DEVELOPER_CFLAGS="-Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Wnested-externs -Wno-pointer-sign -Wextra -Wno-missing-field-initializers -Wno-sign-compare"
+DEVELOPER_CFLAGS=" \
+	-Werror \
+	-Wall \
+	-Wno-format-y2k \
+	-W \
+	-Wno-unused-parameter \
+	-Wstrict-prototypes \
+	-Wmissing-prototypes \
+	-Wpointer-arith \
+	-Wreturn-type \
+	-Wcast-qual \
+	-Wwrite-strings \
+	-Wswitch \
+	-Wshadow \
+	-Wunused-parameter \
+	-Wcast-align \
+	-Wchar-subscripts \
+	-Wnested-externs \
+	-Wno-pointer-sign \
+	-Wextra \
+	-Wno-missing-field-initializers \
+	-Wno-sign-compare \
+"
 
 # These are not compliable yet
 DEVELOPER_GCC_CFLAGS="-Wold-style-definition -Wredundant-decls "



From nils.goroll at uplex.de  Tue Apr  1 13:09:50 2014
From: nils.goroll at uplex.de (Nils Goroll)
Date: Tue, 01 Apr 2014 15:09:50 +0200
Subject: [4.0] 1cebffd dont try to build documentation for tools we cant build
Message-ID: <E1WUyRq-00034V-Pq@project.varnish-software.com>


commit 1cebffd4196dcb50f04ec519b0d0b97d8f497f0c
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Wed Mar 19 19:13:00 2014 +0100

    dont try to build documentation for tools we cant build

diff --git a/doc/sphinx/Makefile.am b/doc/sphinx/Makefile.am
index 716e8da..bb44131 100644
--- a/doc/sphinx/Makefile.am
+++ b/doc/sphinx/Makefile.am
@@ -197,6 +197,9 @@ include/varnishlog_synopsis.rst: $(top_builddir)/bin/varnishlog/varnishlog_opt2r
 BUILT_SOURCES += include/varnishlog_options.rst \
 	include/varnishlog_synopsis.rst
 
+# tools which only get built when curses is available
+# XXX add varnishstat here when _opt2rst is ready
+if VARNISH_CURSES
 include/varnishtop_options.rst: $(top_builddir)/bin/varnishtop/varnishtop_opt2rst
 	$(top_builddir)/bin/varnishtop/varnishtop_opt2rst options > $@
 include/varnishtop_synopsis.rst: $(top_builddir)/bin/varnishtop/varnishtop_opt2rst
@@ -210,6 +213,8 @@ include/varnishhist_synopsis.rst: $(top_builddir)/bin/varnishhist/varnishhist_op
 	$(top_builddir)/bin/varnishhist/varnishhist_opt2rst synopsis > $@
 BUILT_SOURCES += include/varnishhist_options.rst \
 	 include/varnishhist_synopsis.rst
+endif
+
 
 include/vmod_std.rst: $(top_builddir)/lib/libvmod_std/vmod_std.rst
 	cp $? $@
diff --git a/man/Makefile.am b/man/Makefile.am
index c25804e..ae3e40d 100644
--- a/man/Makefile.am
+++ b/man/Makefile.am
@@ -14,12 +14,10 @@ dist_man_MANS = \
 	vsl-query.7 \
 	varnishadm.1 \
 	varnishd.1 \
-	varnishhist.1 \
 	varnishlog.1 \
 	varnishncsa.1 \
 	varnishstat.1 \
 	varnishtest.1 \
-	varnishtop.1 \
 	vmod_directors.3 \
 	vmod_std.3
 
@@ -71,6 +69,12 @@ varnishstat.1: $(top_srcdir)/doc/sphinx/reference/varnishstat.rst
 varnishtest.1: $(top_srcdir)/doc/sphinx/reference/varnishtest.rst
 	${RST2MAN} $(top_srcdir)/doc/sphinx/reference/varnishtest.rst $@
 
+# tools which only get built when curses is available
+# XXX add varnishstat here when _opt2rst is ready
+if VARNISH_CURSES
+dist_man_MANS += varnishtop.1 \
+	varnishhist.1
+
 varnishtop.1: \
 	$(top_srcdir)/doc/sphinx/reference/varnishtop.rst \
 	$(top_srcdir)/doc/sphinx/include/varnishtop_options.rst \
@@ -82,6 +86,8 @@ varnishhist.1: \
 	$(top_srcdir)/doc/sphinx/include/varnishhist_options.rst \
 	$(top_srcdir)/doc/sphinx/include/varnishhist_synopsis.rst
 	${RST2MAN} $(top_srcdir)/doc/sphinx/reference/varnishhist.rst $@
+endif
+
 
 vmod_std.3: $(top_srcdir)/lib/libvmod_std/vmod_std.man.rst
 	${RST2MAN} $? $@



From nils.goroll at uplex.de  Tue Apr  1 13:09:50 2014
From: nils.goroll at uplex.de (Nils Goroll)
Date: Tue, 01 Apr 2014 15:09:50 +0200
Subject: [4.0] 2001b7e DEVELOPER_CFLAGS: - use -fstack-protector only if both compiler and linker support it - use -Wno-missing-field-initializers vs -Wno-extra depending on compiler support - -Wno-pointer-sign only if compiler supports it - add -Wno-address to avoid "the address of ‘buf’ will never be NULL" warning in   cache_backend_cfg.c (and in other places where REPLACE() is used with local   buf variable)
Message-ID: <E1WUyRq-00034e-Qs@project.varnish-software.com>


commit 2001b7efbb114a9c4de5184c87e3c31fc4bee14e
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Wed Mar 19 19:57:34 2014 +0100

    DEVELOPER_CFLAGS:
    - use -fstack-protector only if both compiler and linker support it
    - use -Wno-missing-field-initializers vs -Wno-extra depending on compiler support
    - -Wno-pointer-sign only if compiler supports it
    - add -Wno-address to avoid "the address of ?buf? will never be NULL" warning in
      cache_backend_cfg.c (and in other places where REPLACE() is used with local
      buf variable)

diff --git a/configure.ac b/configure.ac
index 55c62ab..e2a0b98 100644
--- a/configure.ac
+++ b/configure.ac
@@ -470,7 +470,6 @@ DEVELOPER_CFLAGS=" \
 	-Wall \
 	-Wno-format-y2k \
 	-W \
-	-Wno-unused-parameter \
 	-Wstrict-prototypes \
 	-Wmissing-prototypes \
 	-Wpointer-arith \
@@ -483,10 +482,9 @@ DEVELOPER_CFLAGS=" \
 	-Wcast-align \
 	-Wchar-subscripts \
 	-Wnested-externs \
-	-Wno-pointer-sign \
 	-Wextra \
-	-Wno-missing-field-initializers \
 	-Wno-sign-compare \
+	-Wno-address \
 "
 
 # These are not compliable yet
@@ -505,7 +503,10 @@ AC_ARG_ENABLE(stack-protector,
        [enable_stack_protector=yes])
 
 if test "x$enable_stack_protector" != "xno"; then
-       DEVELOPER_CFLAGS="${DEVELOPER_CFLAGS} -fstack-protector"
+	AX_CHECK_COMPILE_FLAG([-fstack-protector],
+		AX_CHECK_LINK_FLAG([-fstack-protector],
+			[DEVELOPER_CFLAGS="${DEVELOPER_CFLAGS} -fstack-protector"], [], []),
+		[], [])
 fi
 
 # --enable-developer-warnings
@@ -515,6 +516,16 @@ AC_ARG_ENABLE(developer-warnings,
 	[enable_developer_warnings=no])
 
 if test "x$enable_developer_warnings" != "xno"; then
+	# compiler flags not available on gcc3
+	AX_CHECK_COMPILE_FLAG([-Wno-pointer-sign],
+		[DEVELOPER_CFLAGS="${DEVELOPER_CFLAGS} -Wno-pointer-sign"], [], [])
+	# no known way to specifically disabling missing-field-initializers warnings
+	# keeping the rest of Wextra
+	AX_CHECK_COMPILE_FLAG([-Wno-missing-field-initializers],
+		[DEVELOPER_CFLAGS="${DEVELOPER_CFLAGS} -Wno-missing-field-initializers"],
+		[DEVELOPER_CFLAGS="${DEVELOPER_CFLAGS} -Wno-extra"],
+		[])
+
 	CFLAGS="${CFLAGS} ${DEVELOPER_CFLAGS}"
 	OCFLAGS="${OCFLAGS} ${DEVELOPER_CFLAGS}"
 fi
diff --git a/m4/ax_check_link_flag.m4 b/m4/ax_check_link_flag.m4
new file mode 100644
index 0000000..db899dd
--- /dev/null
+++ b/m4/ax_check_link_flag.m4
@@ -0,0 +1,73 @@
+# ===========================================================================
+#    http://www.gnu.org/software/autoconf-archive/ax_check_link_flag.html
+# ===========================================================================
+#
+# SYNOPSIS
+#
+#   AX_CHECK_LINK_FLAG(FLAG, [ACTION-SUCCESS], [ACTION-FAILURE], [EXTRA-FLAGS], [INPUT])
+#
+# DESCRIPTION
+#
+#   Check whether the given FLAG works with the linker or gives an error.
+#   (Warnings, however, are ignored)
+#
+#   ACTION-SUCCESS/ACTION-FAILURE are shell commands to execute on
+#   success/failure.
+#
+#   If EXTRA-FLAGS is defined, it is added to the linker's default flags
+#   when the check is done.  The check is thus made with the flags: "LDFLAGS
+#   EXTRA-FLAGS FLAG".  This can for example be used to force the linker to
+#   issue an error when a bad flag is given.
+#
+#   INPUT gives an alternative input source to AC_LINK_IFELSE.
+#
+#   NOTE: Implementation based on AX_CFLAGS_GCC_OPTION. Please keep this
+#   macro in sync with AX_CHECK_{PREPROC,COMPILE}_FLAG.
+#
+# LICENSE
+#
+#   Copyright (c) 2008 Guido U. Draheim <guidod at gmx.de>
+#   Copyright (c) 2011 Maarten Bosmans <mkbosmans at gmail.com>
+#
+#   This program is free software: you can redistribute it and/or modify it
+#   under the terms of the GNU General Public License as published by the
+#   Free Software Foundation, either version 3 of the License, or (at your
+#   option) any later version.
+#
+#   This program is distributed in the hope that it will be useful, but
+#   WITHOUT ANY WARRANTY; without even the implied warranty of
+#   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General
+#   Public License for more details.
+#
+#   You should have received a copy of the GNU General Public License along
+#   with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+#   As a special exception, the respective Autoconf Macro's copyright owner
+#   gives unlimited permission to copy, distribute and modify the configure
+#   scripts that are the output of Autoconf when processing the Macro. You
+#   need not follow the terms of the GNU General Public License when using
+#   or distributing such scripts, even though portions of the text of the
+#   Macro appear in them. The GNU General Public License (GPL) does govern
+#   all other use of the material that constitutes the Autoconf Macro.
+#
+#   This special exception to the GPL applies to versions of the Autoconf
+#   Macro released by the Autoconf Archive. When you make and distribute a
+#   modified version of the Autoconf Macro, you may extend this special
+#   exception to the GPL to apply to your modified version as well.
+
+#serial 3
+
+AC_DEFUN([AX_CHECK_LINK_FLAG],
+[AS_VAR_PUSHDEF([CACHEVAR],[ax_cv_check_ldflags_$4_$1])dnl
+AC_CACHE_CHECK([whether the linker accepts $1], CACHEVAR, [
+  ax_check_save_flags=$LDFLAGS
+  LDFLAGS="$LDFLAGS $4 $1"
+  AC_LINK_IFELSE([m4_default([$5],[AC_LANG_PROGRAM()])],
+    [AS_VAR_SET(CACHEVAR,[yes])],
+    [AS_VAR_SET(CACHEVAR,[no])])
+  LDFLAGS=$ax_check_save_flags])
+AS_IF([test x"AS_VAR_GET(CACHEVAR)" = xyes],
+  [m4_default([$2], :)],
+  [m4_default([$3], :)])
+AS_VAR_POPDEF([CACHEVAR])dnl
+])dnl AX_CHECK_LINK_FLAGS



From phk at FreeBSD.org  Tue Apr  1 13:09:50 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 01 Apr 2014 15:09:50 +0200
Subject: [4.0] bba23ef Move obj.hits to objhead, make it read-only
Message-ID: <E1WUyRq-00034o-S7@project.varnish-software.com>


commit bba23ef151986671cef6d1ad8a126d30d093daa0
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Mar 19 21:33:44 2014 +0000

    Move obj.hits to objhead, make it read-only

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index ce90e58..ac615a4 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -609,8 +609,6 @@ struct object {
 	struct exp		exp;
 
 	/* VCL only variables */
-	long			hits;
-
 	double			last_modified;
 
 	struct http		*http;
diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index 07c5931..d7263e1 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -433,6 +433,8 @@ HSH_Lookup(struct req *req, struct objcore **ocp, struct objcore **bocp,
 			assert(oh->refcnt > 1);
 			assert(oc->objhead == oh);
 			oc->refcnt++;
+			if (oh->hits < LONG_MAX)
+				oh->hits++;
 			Lck_Unlock(&oh->mtx);
 			assert(HSH_DerefObjHead(&wrk->stats, &oh));
 			*ocp = oc;
@@ -460,12 +462,11 @@ HSH_Lookup(struct req *req, struct objcore **ocp, struct objcore **bocp,
 		} else {
 			retval = HSH_EXP;
 		}
-
+		if (oh->hits < LONG_MAX)
+			oh->hits++;
 		Lck_Unlock(&oh->mtx);
 		if (retval == HSH_EXP)
 			assert(HSH_DerefObjHead(&wrk->stats, &oh));
-		if (!cache_param->obj_readonly && exp_o->hits < INT_MAX)
-			exp_o->hits++;
 		*ocp = exp_oc;
 		return (retval);
 	}
diff --git a/bin/varnishd/cache/cache_vrt_var.c b/bin/varnishd/cache/cache_vrt_var.c
index 0bcd5d5..b51d2de 100644
--- a/bin/varnishd/cache/cache_vrt_var.c
+++ b/bin/varnishd/cache/cache_vrt_var.c
@@ -36,6 +36,7 @@
 
 #include "cache.h"
 #include "common/heritage.h"
+#include "hash/hash_slinger.h"
 
 #include "cache_backend.h"
 #include "vrt.h"
@@ -570,29 +571,17 @@ VRT_r_server_hostname(const struct vrt_ctx *ctx)
 
 /*--------------------------------------------------------------------*/
 
-#define VOBJ_L(type, field)						\
-void									\
-VRT_l_obj_##field(const struct vrt_ctx *ctx, type a)			\
-{									\
-	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);				\
-	CHECK_OBJ_NOTNULL(ctx->req, REQ_MAGIC);				\
-	CHECK_OBJ_NOTNULL(ctx->req->obj, OBJECT_MAGIC);			\
-	ctx->req->obj->field = a;					\
-}
-
-#define VOBJ_R(type, field)						\
-type									\
-VRT_r_obj_##field(const struct vrt_ctx *ctx)				\
-{									\
-	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);				\
-	CHECK_OBJ_NOTNULL(ctx->req, REQ_MAGIC);				\
-	CHECK_OBJ_NOTNULL(ctx->req->obj, OBJECT_MAGIC);			\
-	return (ctx->req->obj->field);					\
+long
+VRT_r_obj_hits(const struct vrt_ctx *ctx)
+{
+	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
+	CHECK_OBJ_NOTNULL(ctx->req, REQ_MAGIC);
+	CHECK_OBJ_NOTNULL(ctx->req->obj, OBJECT_MAGIC);
+	CHECK_OBJ_NOTNULL(ctx->req->obj->objcore, OBJCORE_MAGIC);
+	CHECK_OBJ_NOTNULL(ctx->req->obj->objcore->objhead, OBJHEAD_MAGIC);
+	return (ctx->req->obj->objcore->objhead->hits);
 }
 
-VOBJ_L(long, hits)
-VOBJ_R(long, hits)
-
 unsigned
 VRT_r_obj_uncacheable(const struct vrt_ctx *ctx)
 {
diff --git a/bin/varnishd/hash/hash_slinger.h b/bin/varnishd/hash/hash_slinger.h
index f38c84d..40d3aef 100644
--- a/bin/varnishd/hash/hash_slinger.h
+++ b/bin/varnishd/hash/hash_slinger.h
@@ -94,6 +94,8 @@ struct objhead {
 	uint8_t			digest[DIGEST_LEN];
 	struct waitinglist	*waitinglist;
 
+	long			hits;
+
 	/*----------------------------------------------------
 	 * The fields below are for the sole private use of
 	 * the hash implementation(s).
diff --git a/bin/varnishtest/tests/v00013.vtc b/bin/varnishtest/tests/v00013.vtc
index 0bb2858..c428add 100644
--- a/bin/varnishtest/tests/v00013.vtc
+++ b/bin/varnishtest/tests/v00013.vtc
@@ -13,7 +13,6 @@ varnish v1 -vcl+backend {
 
 	sub vcl_deliver {
 		set resp.http.foo = obj.hits;
-		set obj.hits = obj.hits + 1;
 	}
 } -start
 
diff --git a/lib/libvcc/generate.py b/lib/libvcc/generate.py
index 9d8239b..c63274e 100755
--- a/lib/libvcc/generate.py
+++ b/lib/libvcc/generate.py
@@ -512,7 +512,7 @@ sp_variables = [
 	('obj.hits',
 		'INT',
 		( 'hit', 'deliver',),
-		( 'hit', 'deliver',), """
+		( ), """
 		The approximate number of times the object has been
 		delivered. A value of 0 indicates a cache miss.
 		This variable is also available in vcl_deliver.



From phk at FreeBSD.org  Tue Apr  1 13:09:50 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 01 Apr 2014 15:09:50 +0200
Subject: [4.0] 4a81bc6 Remove obsolete comment about obj.hits
Message-ID: <E1WUyRq-00034y-TF@project.varnish-software.com>


commit 4a81bc626e01e7e329b69184c0a6e95be8afbaa8
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Mar 19 21:34:29 2014 +0000

    Remove obsolete comment about obj.hits

diff --git a/bin/varnishd/builtin.vcl b/bin/varnishd/builtin.vcl
index 92f4641..8e5ca36 100644
--- a/bin/varnishd/builtin.vcl
+++ b/bin/varnishd/builtin.vcl
@@ -113,12 +113,6 @@ sub vcl_miss {
 }
 
 sub vcl_deliver {
-    /*
-     * These two write to the stored object causing extra page faults
-     * Enable them only if you need them.
-     *
-     * set obj.hits = obj.hits + 1;
-     */
     return (deliver);
 }
 



From phk at FreeBSD.org  Tue Apr  1 13:09:50 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 01 Apr 2014 15:09:50 +0200
Subject: [4.0] d260e98 Fix the three obj.{proto, reason,
 status} variables to match reality.
Message-ID: <E1WUyRq-000357-Uv@project.varnish-software.com>


commit d260e9874bc89d550f89462f7ea237a672c0f265
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Mar 19 22:18:44 2014 +0000

    Fix the three obj.{proto,reason,status} variables to match reality.

diff --git a/lib/libvcc/generate.py b/lib/libvcc/generate.py
index c63274e..7c7e9db 100755
--- a/lib/libvcc/generate.py
+++ b/lib/libvcc/generate.py
@@ -491,21 +491,21 @@ sp_variables = [
 	('obj.proto',
 		'STRING',
 		( 'hit', ),
-		( 'hit', ), """
+		( ), """
 		The HTTP protocol version used when the object was retrieved.
 		"""
 	),
 	('obj.status',
 		'INT',
-		( 'synth',),
-		( 'synth',), """
+		( 'hit',),
+		( ), """
 		The HTTP status code returned by the server.
 		"""
 	),
 	('obj.reason',
 		'STRING',
-		( 'synth',),
-		( 'synth',), """
+		( 'hit',),
+		( ), """
 		The HTTP status message returned by the server.
 		"""
 	),



From fgsch at lodoss.net  Tue Apr  1 13:09:50 2014
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Tue, 01 Apr 2014 15:09:50 +0200
Subject: [4.0] af27926 Update.. somewhat
Message-ID: <E1WUyRq-00035G-Vu@project.varnish-software.com>


commit af27926a9111247472faff6a5924b84293349f48
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Wed Mar 19 22:54:59 2014 +0000

    Update.. somewhat

diff --git a/doc/sphinx/reference/vmod.rst b/doc/sphinx/reference/vmod.rst
index 95987e1..580caff 100644
--- a/doc/sphinx/reference/vmod.rst
+++ b/doc/sphinx/reference/vmod.rst
@@ -22,7 +22,7 @@ For instance::
 The "std" vmod is one you get with Varnish, it will always be there
 and we will put "boutique" functions in it, such as the "toupper"
 function shown above.  The full contents of the "std" module is
-documented in vmod_std(7).
+documented in vmod_std(3).
 
 This part of the manual is about how you go about writing your own
 VMOD, how the language interface between C and VCC works, where you
@@ -43,15 +43,15 @@ The vmod.vcc file
 
 The interface between your VMOD and the VCL compiler ("VCC") and the
 VCL runtime ("VRT") is defined in the vmod.vcc file which a python
-script called "vmod.py" turns into thaumaturgically challenged C
+script called "vmodtool.py" turns into thaumaturgically challenged C
 data structures that does all the hard work.
 
 The std VMODs vmod.vcc file looks somewhat like this::
 
-	$Module std
+	$Module std 3
 	$Init init_function
-	$Function STRING toupper(PRIV_CALL, STRING_LIST)
-	$Function STRING tolower(PRIV_VCL, STRING_LIST)
+	$Function STRING toupper(STRING_LIST)
+	$Function STRING tolower(STRING_LIST)
 	$Function VOID set_ip_tos(INT)
 
 The first line gives the name of the module, nothing special there.
@@ -75,7 +75,7 @@ primary action, something functions which return a value can not::
 		std.set_ip_tos(32);
 	}
 
-Running vmod.py on the vmod.vcc file, produces an "vcc_if.c" and
+Running vmodtool.py on the vmod.vcc file, produces an "vcc_if.c" and
 "vcc_if.h" files, which you must use to build your shared library
 file.
 
@@ -88,14 +88,18 @@ the functions you want to export to VCL.
 
 For the std VMOD, the compiled vcc_if.h file looks like this::
 
-	struct sess;
+	struct vrt_ctx;
 	struct VCL_conf;
-	const char * vmod_toupper(struct sess *, struct vmod_priv *, const char *, ...);
-	const char * vmod_tolower(struct sess *, struct vmod_priv *, const char *, ...);
-	int meta_function(void **, const struct VCL_conf *);
+	struct vmod_priv;
+
+	VCL_STRING vmod_toupper(const struct vrt_ctx *, const char *, ...);
+	VCL_STRING vmod_tolower(const struct vrt_ctx *, const char *, ...);
+	VCL_VOID vmod_set_ip_tos(const struct vrt_ctx *, VCL_INT);
+
+	int init_function(struct vmod_priv *, const struct VCL_conf *);
 
 Those are your C prototypes.  Notice the ``vmod_`` prefix on the function
-names and the C-types as return types and arguments.
+names and the C-types as arguments.
 
 VCL and C data types
 ====================
@@ -319,5 +323,5 @@ That means that the VMOD init, and any object init/fini functions
 are already serialized in sensible order, and won't need any locking,
 unless they access VMOD specific global state, shared with other VCLs.
 
-Trafic in other VCLs which also import this VMOD, will be happening
+Traffic in other VCLs which also import this VMOD, will be happening
 while housekeeping is going on.



From fgsch at lodoss.net  Tue Apr  1 13:09:51 2014
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Tue, 01 Apr 2014 15:09:51 +0200
Subject: [4.0] 0f38ff3 Add missing dependency
Message-ID: <E1WUyRr-00035P-15@project.varnish-software.com>


commit 0f38ff3b4bd06203cf7d209b21411bc08856c228
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Wed Mar 19 22:59:34 2014 +0000

    Add missing dependency

diff --git a/lib/libvmod_std/Makefile.am b/lib/libvmod_std/Makefile.am
index 07dda95..2e9832a 100644
--- a/lib/libvmod_std/Makefile.am
+++ b/lib/libvmod_std/Makefile.am
@@ -26,7 +26,7 @@ nodist_libvmod_std_la_SOURCES = \
 # BUILT_SOURCES is only a hack and dependency tracking does not help for the first build
 vmod_std.lo vmod_std_fileread.lo vmod_std_conversions.lo: vcc_if.h
 
-vcc_if.c vcc_if.h: $(vmodtool) $(vmod_srcdir)/vmod.vcc
+vcc_if.c vcc_if.h vmod_std.man.rst: $(vmodtool) $(vmod_srcdir)/vmod.vcc
 	@PYTHON@ $(vmodtool) $(vmod_srcdir)/vmod.vcc
 
 EXTRA_DIST = vmod.vcc



From fgsch at lodoss.net  Tue Apr  1 13:09:51 2014
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Tue, 01 Apr 2014 15:09:51 +0200
Subject: [4.0] 4362464 Typo
Message-ID: <E1WUyRr-00035Y-3W@project.varnish-software.com>


commit 4362464111e676d127c3ee10972aa5aec177b3a2
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Wed Mar 19 23:01:27 2014 +0000

    Typo

diff --git a/include/tbl/sess_close.h b/include/tbl/sess_close.h
index b132b39..522eb02 100644
--- a/include/tbl/sess_close.h
+++ b/include/tbl/sess_close.h
@@ -31,7 +31,7 @@
 
 SESS_CLOSE(REM_CLOSE,	"Client Closed")
 SESS_CLOSE(REQ_CLOSE,	"Client requested close")
-SESS_CLOSE(REQ_HTTP10,	"proto < HTTP.1.1")
+SESS_CLOSE(REQ_HTTP10,	"Proto < HTTP/1.1")
 SESS_CLOSE(RX_BODY,	"Failure receiving req.body")
 SESS_CLOSE(RX_JUNK,	"Received junk data")
 SESS_CLOSE(RX_OVERFLOW,	"Received buffer overflow")



From fgsch at lodoss.net  Tue Apr  1 13:09:51 2014
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Tue, 01 Apr 2014 15:09:51 +0200
Subject: [4.0] f9fd935 Update after moving vmod docs to man/
Message-ID: <E1WUyRr-00035h-4Z@project.varnish-software.com>


commit f9fd935793e534bebe26afded486038774fbca30
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Wed Mar 19 23:05:34 2014 +0000

    Update after moving vmod docs to man/

diff --git a/.gitignore b/.gitignore
index dd684da..a37080a 100644
--- a/.gitignore
+++ b/.gitignore
@@ -71,14 +71,14 @@ cscope.*out
 # Misc. generated files for included vmods.
 /lib/libvmod_*/vcc_if.c
 /lib/libvmod_*/vcc_if.h
-/lib/libvmod_*/vmod_*.3
 /lib/libvmod_*/vmod_*.rst
 
 
 # Man-files and binaries
 /man/vsc2rst
-/man/*.7
 /man/*.1
+/man/*.3
+/man/*.7
 /doc/sphinx/include
 /bin/varnish*/varnish*_opt2rst
 /bin/varnishadm/varnishadm



From martin at varnish-software.com  Tue Apr  1 13:09:51 2014
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Tue, 01 Apr 2014 15:09:51 +0200
Subject: [4.0] 5261bdb Revert "Reset busyobj state to BOS_INVALID in
 VBO_GetBusyObj"
Message-ID: <E1WUyRr-00035q-5j@project.varnish-software.com>


commit 5261bdb4e83ee71789f8db64af91f7010c15ce6f
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Thu Mar 20 10:30:39 2014 +0100

    Revert "Reset busyobj state to BOS_INVALID in VBO_GetBusyObj"
    
    This reverts commit 0f90ee223daf97d4e6d83a78ae919a95e590bcca.
    
    After looking at this again, I see that the busyobj is cleared as it
    should as part of VBO_DerefBusyObj(). Reverting the patch.

diff --git a/bin/varnishd/cache/cache_busyobj.c b/bin/varnishd/cache/cache_busyobj.c
index d8be0f6..db1c7a1 100644
--- a/bin/varnishd/cache/cache_busyobj.c
+++ b/bin/varnishd/cache/cache_busyobj.c
@@ -109,7 +109,6 @@ VBO_GetBusyObj(struct worker *wrk, const struct req *req)
 	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
 	AZ(bo->refcount);
 
-	bo->state = BOS_INVALID;
 	bo->refcount = 1;
 
 	p = (void*)(bo + 1);



From lkarsten at varnish-software.com  Tue Apr  1 13:09:51 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:51 +0200
Subject: [4.0] cd2e297 Reflect new C: close default in vcl_pipe
Message-ID: <E1WUyRr-00035z-6o@project.varnish-software.com>


commit cd2e297fabde5356d906abb9d44f803263bcd730
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu Mar 20 11:20:50 2014 +0100

    Reflect new C: close default in vcl_pipe
    
    We now set Connection: Close on all piped backend requests, to avoid some of
    the debugging pain seen over the last 2-3 years.
    
    Basic problem is that request backend routing done in vcl_recv would not be
    honored for subsequent requests after one was piped.
    
    This was, at least for new users, a rather nasty surprise.
    
    Original change in commit 880c3c2.

diff --git a/bin/varnishd/builtin.vcl b/bin/varnishd/builtin.vcl
index 8e5ca36..3949689 100644
--- a/bin/varnishd/builtin.vcl
+++ b/bin/varnishd/builtin.vcl
@@ -67,12 +67,11 @@ sub vcl_recv {
 }
 
 sub vcl_pipe {
-    # Note that only the first request to the backend will have
-    # X-Forwarded-For set.  If you use X-Forwarded-For and want to
-    # have it set for all requests, make sure to have:
-    # set bereq.http.connection = "close";
-    # here.  It is not set by default as it might break some broken web
-    # applications, like IIS with NTLM authentication.
+    # By default Connection: close is set on all piped requests, to stop
+    # connection reuse from sending future requests directly to the
+    # (potentially) wrong backend. If you do want this to happen, you can undo
+    # it here.
+    # unset bereq.http.connection;
     return (pipe);
 }
 



From perbu at varnish-software.com  Tue Apr  1 13:09:51 2014
From: perbu at varnish-software.com (Per Buer)
Date: Tue, 01 Apr 2014 15:09:51 +0200
Subject: [4.0] 47565d7 Emphasise the values.
Message-ID: <E1WUyRr-000368-84@project.varnish-software.com>


commit 47565d7e237745c9619707a4ce6d0aa3e1f96e47
Author: Per Buer <perbu at varnish-software.com>
Date:   Thu Mar 20 11:27:16 2014 +0100

    Emphasise the values.

diff --git a/doc/sphinx/reference/vcl.rst b/doc/sphinx/reference/vcl.rst
index 43295bb..321c251 100644
--- a/doc/sphinx/reference/vcl.rst
+++ b/doc/sphinx/reference/vcl.rst
@@ -95,7 +95,7 @@ the NUL (0x00) character
 Booleans
 ~~~~~~~~
 
-Booleans can be either true or false.
+Booleans can be either *true* or *false*.
 
 Time
 ----



From perbu at varnish-software.com  Tue Apr  1 13:09:51 2014
From: perbu at varnish-software.com (Per Buer)
Date: Tue, 01 Apr 2014 15:09:51 +0200
Subject: [4.0] afbdcbb Note that directors stack.
Message-ID: <E1WUyRr-00036H-90@project.varnish-software.com>


commit afbdcbb88b37873e5ffac4e1c9a30713cc1e928c
Author: Per Buer <perbu at varnish-software.com>
Date:   Thu Mar 20 11:27:29 2014 +0100

    Note that directors stack.

diff --git a/doc/sphinx/reference/vcl.rst b/doc/sphinx/reference/vcl.rst
index 321c251..942d1ac 100644
--- a/doc/sphinx/reference/vcl.rst
+++ b/doc/sphinx/reference/vcl.rst
@@ -208,6 +208,9 @@ are available:
     Varnish reaches the maximum Varnish it will start failing
     connections.
 
+Backends can be used with *directors*. Please see the
+vmod_directors(3) man page for more information.
+
 Probes
 ------
 



From perbu at varnish-software.com  Tue Apr  1 13:09:51 2014
From: perbu at varnish-software.com (Per Buer)
Date: Tue, 01 Apr 2014 15:09:51 +0200
Subject: [4.0] 6284634 Update purging docs for V4.
Message-ID: <E1WUyRr-00036Q-9u@project.varnish-software.com>


commit 6284634299582a1caed72b9fc38396d303be6c2f
Author: Per Buer <perbu at varnish-software.com>
Date:   Thu Mar 20 11:27:55 2014 +0100

    Update purging docs for V4.

diff --git a/doc/sphinx/users-guide/purging.rst b/doc/sphinx/users-guide/purging.rst
index 38de2d3..5d7deda 100644
--- a/doc/sphinx/users-guide/purging.rst
+++ b/doc/sphinx/users-guide/purging.rst
@@ -40,31 +40,16 @@ following VCL in place::
 			  error 405 "Not allowed.";
 		  }
                   # jump to hit/miss
-		  return (lookup);
+		  return (purge);
 	  }
   }
 
-  sub vcl_hit {
-	  if (req.method == "PURGE") {
-	          purge;
-		  error 200 "Purged.";
-	  }
-  }
-
-  sub vcl_miss {
-	  if (req.method == "PURGE") {
-	          purge;
-		  error 200 "Purged.";
-	  }
-  }
-
-As you can see we have used two new VCL subroutines, vcl_hit and
-vcl_miss. When we call lookup Varnish will try to lookup the object in
-its cache. It will either hit an object or miss it and so the
-corresponding subroutine is called. In vcl_hit the object that is
-stored in cache is available and we can set the TTL. The purge in
-vcl_miss is necessary to purge all variants in the cases where you hit an
-object, but miss a particular variant.
+As you can see we have used a new action - return(purge). This ends
+execution of vcl_recv and jumps to vcl_hash. This is just like we
+handle a regular request. When vcl_hash calls return(lookup) varnish
+will purge the object and then call vcl_purge. Here you have the
+option of adding any particular actions you want Varnish to take once
+it has purge the object.
 
 So for example.com to invalidate their front page they would call out
 to Varnish like this::
@@ -121,7 +106,7 @@ You can also add bans to Varnish via HTTP. Doing so requires a bit of VCL::
 
 		  # Throw a synthetic page so the
                   # request won't go to the backend.
-		  error 200 "Ban added";
+		  return(synth(200m"Ban added"));
 	  }
   }
 



From perbu at varnish-software.com  Tue Apr  1 13:09:51 2014
From: perbu at varnish-software.com (Per Buer)
Date: Tue, 01 Apr 2014 15:09:51 +0200
Subject: [4.0] 06762cf How to call purge from recv.
Message-ID: <E1WUyRr-00036Z-BE@project.varnish-software.com>


commit 06762cf62ac4284b0c33a503b9d57f8b22306368
Author: Per Buer <perbu at varnish-software.com>
Date:   Thu Mar 20 11:28:28 2014 +0100

    How to call purge from recv.

diff --git a/doc/sphinx/users-guide/vcl-built-in-subs.rst b/doc/sphinx/users-guide/vcl-built-in-subs.rst
index 7a31aef..f71dfc1 100644
--- a/doc/sphinx/users-guide/vcl-built-in-subs.rst
+++ b/doc/sphinx/users-guide/vcl-built-in-subs.rst
@@ -36,8 +36,8 @@ of the following keywords:
     caching. Passes the control over to vcl_hash.
 
   purge
-    Calls vcl_purge where the object will be purged.
-
+    Purge the object and it's variants. Control passes through 
+    vcl_hash to vcl_purge.
 
 vcl_pipe
 ~~~~~~~~



From perbu at varnish-software.com  Tue Apr  1 13:09:51 2014
From: perbu at varnish-software.com (Per Buer)
Date: Tue, 01 Apr 2014 15:09:51 +0200
Subject: [4.0] 526ca86 Directors can stack.
Message-ID: <E1WUyRr-00036i-CR@project.varnish-software.com>


commit 526ca86006db9b10ed678ceb22da7fc34e047b6b
Author: Per Buer <perbu at varnish-software.com>
Date:   Thu Mar 20 11:28:42 2014 +0100

    Directors can stack.

diff --git a/lib/libvmod_directors/vmod.vcc b/lib/libvmod_directors/vmod.vcc
index f7ec8f4..24e8839 100644
--- a/lib/libvmod_directors/vmod.vcc
+++ b/lib/libvmod_directors/vmod.vcc
@@ -54,6 +54,8 @@ As you can see there is nothing keeping you from manipulating the
 directors elsewhere in VCL. So, you could have VCL code that would
 add more backends to a director when a certain URL is called.
 
+Note that directors can use other directors as backends.
+
 $Object round_robin()
 
 Description



From perbu at varnish-software.com  Tue Apr  1 13:09:51 2014
From: perbu at varnish-software.com (Per Buer)
Date: Tue, 01 Apr 2014 15:09:51 +0200
Subject: [4.0] cda2ecf Error is gone.
Message-ID: <E1WUyRr-00036r-E6@project.varnish-software.com>


commit cda2ecfae0eb12328d613be2d9bfcc9f6b953ee0
Author: Per Buer <perbu at varnish-software.com>
Date:   Thu Mar 20 11:29:46 2014 +0100

    Error is gone.

diff --git a/doc/sphinx/users-guide/vcl-built-in-subs.rst b/doc/sphinx/users-guide/vcl-built-in-subs.rst
index f71dfc1..4772374 100644
--- a/doc/sphinx/users-guide/vcl-built-in-subs.rst
+++ b/doc/sphinx/users-guide/vcl-built-in-subs.rst
@@ -93,7 +93,7 @@ Called is a cache lookup is successful.
   deliver
     Deliver the object. Control passes to vcl_deliver.
 
-  error [reason]
+  synth(error code, reason)
     Return the specified error code to the client and abandon the request.
 
 
@@ -107,7 +107,7 @@ retrieve the document from the backend, and which backend to use.
 The vcl_miss subroutine may terminate with calling return() with one
 of the following keywords:
 
-  error [reason]
+  synth(error code, reason)
     Return the specified error code to the client and abandon the request.
 
   pass
@@ -124,21 +124,19 @@ Called after vcl_recv to create a hash value for the request. This is
 used as a key to look up the object in Varnish.
 
   lookup
-    Look up the object in cache.
+    Look up the object in cache. Control passes to vcl_miss, vcl_hit
+    or vcl_purge.
+
+
 
 
 vcl_purge
 ~~~~~~~~~
 
-Purge the object and all it's variants. Variants created when the
-backend issues a Vary response.
-
-  fetch
-    Execute the purge.
+Called after the purge has been executed and all it's variant have been evited. 
 
-  error
-    Fail the purge request. Typically you would call error here it
-    request doesn't pass the ACLs.
+  synth
+    Produce a response.
 
 
 



From perbu at varnish-software.com  Tue Apr  1 13:09:51 2014
From: perbu at varnish-software.com (Per Buer)
Date: Tue, 01 Apr 2014 15:09:51 +0200
Subject: [4.0] c890439 Fix error syntax.
Message-ID: <E1WUyRr-000370-FK@project.varnish-software.com>


commit c890439f01e916c3385f34d6c91992563b21c638
Author: Per Buer <perbu at varnish-software.com>
Date:   Thu Mar 20 11:29:57 2014 +0100

    Fix error syntax.

diff --git a/doc/sphinx/users-guide/vcl-built-in-subs.rst b/doc/sphinx/users-guide/vcl-built-in-subs.rst
index 4772374..73934c7 100644
--- a/doc/sphinx/users-guide/vcl-built-in-subs.rst
+++ b/doc/sphinx/users-guide/vcl-built-in-subs.rst
@@ -194,7 +194,7 @@ of the following keywords:
     Control will eventually pass to vcl_deliver. Caching is dependant
     on beresp.cacheable.
 
-  error [reason]
+  error(error code, reason)
     Return the specified error code to the client and abandon the request.
 
   retry



From perbu at varnish-software.com  Tue Apr  1 13:09:51 2014
From: perbu at varnish-software.com (Per Buer)
Date: Tue, 01 Apr 2014 15:09:51 +0200
Subject: [4.0] 7f5dbd3 Update for V4 syntax.
Message-ID: <E1WUyRr-000379-GK@project.varnish-software.com>


commit 7f5dbd315095d5d1d8519b88deaad43c237f242a
Author: Per Buer <perbu at varnish-software.com>
Date:   Thu Mar 20 11:42:37 2014 +0100

    Update for V4 syntax.

diff --git a/doc/sphinx/users-guide/vcl-example-acls.rst b/doc/sphinx/users-guide/vcl-example-acls.rst
index 3d0226a..b460bbe 100644
--- a/doc/sphinx/users-guide/vcl-example-acls.rst
+++ b/doc/sphinx/users-guide/vcl-example-acls.rst
@@ -15,20 +15,10 @@ the IP address of the client against an ACL with the match operator.::
   sub vcl_recv {
     if (req.method == "PURGE") {
       if (client.ip ~ local) {
-         return(lookup);
+         return(purge);
+      } else {
+         return(synth(403, "Access denied."));
       }
     } 
   }
   
-  sub vcl_hit {
-     if (req.method == "PURGE") {
-       set obj.ttl = 0s;
-       error 200 "Purged.";
-      }
-  }
-
-  sub vcl_miss {
-    if (req.method == "PURGE") {
-      error 404 "Not in cache.";
-    }
-  }



From perbu at varnish-software.com  Tue Apr  1 13:09:51 2014
From: perbu at varnish-software.com (Per Buer)
Date: Tue, 01 Apr 2014 15:09:51 +0200
Subject: [4.0] 1a25542 Another error/synth.
Message-ID: <E1WUyRr-00037I-HY@project.varnish-software.com>


commit 1a25542ffd040759aecf49c91eab4b2489d7fa74
Author: Per Buer <perbu at varnish-software.com>
Date:   Thu Mar 20 11:42:50 2014 +0100

    Another error/synth.

diff --git a/doc/sphinx/users-guide/purging.rst b/doc/sphinx/users-guide/purging.rst
index 5d7deda..4aad344 100644
--- a/doc/sphinx/users-guide/purging.rst
+++ b/doc/sphinx/users-guide/purging.rst
@@ -37,7 +37,7 @@ following VCL in place::
 
 	  if (req.method == "PURGE") {
 		  if (!client.ip ~ purge) {
-			  error 405 "Not allowed.";
+			  return(synth(405,"Not allowed.");
 		  }
                   # jump to hit/miss
 		  return (purge);



From perbu at varnish-software.com  Tue Apr  1 13:09:51 2014
From: perbu at varnish-software.com (Per Buer)
Date: Tue, 01 Apr 2014 15:09:51 +0200
Subject: [4.0] 7b5f540 Remove saint mode.
Message-ID: <E1WUyRr-00037R-JB@project.varnish-software.com>


commit 7b5f54055666b21c03ddd79053645d7e735a8e54
Author: Per Buer <perbu at varnish-software.com>
Date:   Thu Mar 20 11:44:33 2014 +0100

    Remove saint mode.

diff --git a/doc/sphinx/users-guide/vcl-saint-and-grace.rst b/doc/sphinx/users-guide/vcl-saint-and-grace.rst
index bb5fd35..f318aa3 100644
--- a/doc/sphinx/users-guide/vcl-saint-and-grace.rst
+++ b/doc/sphinx/users-guide/vcl-saint-and-grace.rst
@@ -56,48 +56,3 @@ So, to sum up, grace mode solves two problems:
  * it serves stale content to avoid request pile-up.
  * it serves stale content if the backend is not healthy.
 
-Saint mode
-~~~~~~~~~~
-
-Sometimes servers get flaky. They start throwing out random
-errors. You can instruct Varnish to try to handle this in a
-more-than-graceful way - enter *Saint mode*. Saint mode enables you to
-discard a certain page from one backend server and either try another
-server or serve stale content from cache. Lets have a look at how this
-can be enabled in VCL::
-
-  sub vcl_fetch {
-    if (beresp.status == 500) { 
-      set beresp.saintmode = 10s;
-      return(restart);
-    }
-    set beresp.grace = 5m;
-  } 
-
-When we set beresp.saintmode to 10 seconds Varnish will not ask *that*
-server for URL for 10 seconds. A blacklist, more or less. Also a
-restart is performed so if you have other backends capable of serving
-that content Varnish will try those. When you are out of backends
-Varnish will serve the content from its stale cache.
-
-This can really be a life saver.
-
-Known limitations on grace- and saint mode
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-If your request fails while it is being fetched you're thrown into
-vcl_error. vcl_error has access to a rather limited set of data so you
-can't enable saint mode or grace mode here. This will be addressed in a
-future release but a work-around available.
-
-* Declare a backend that is always sick.
-* Set a magic marker in vcl_error
-* Restart the transaction
-* Note the magic marker in vcl_recv and set the backend to the one mentioned
-* Varnish will now serve stale data is any is available
-
-
-God mode
-~~~~~~~~
-Not implemented yet. :-)
-



From perbu at varnish-software.com  Tue Apr  1 13:09:51 2014
From: perbu at varnish-software.com (Per Buer)
Date: Tue, 01 Apr 2014 15:09:51 +0200
Subject: [4.0] dd54b29 Remove saint mode references.
Message-ID: <E1WUyRr-00037d-L7@project.varnish-software.com>


commit dd54b29a468e4b70af00bd004f7be4b3d8d2c986
Author: Per Buer <perbu at varnish-software.com>
Date:   Thu Mar 20 11:46:49 2014 +0100

    Remove saint mode references.

diff --git a/doc/sphinx/Makefile.am b/doc/sphinx/Makefile.am
index bb44131..f98a044 100644
--- a/doc/sphinx/Makefile.am
+++ b/doc/sphinx/Makefile.am
@@ -167,7 +167,7 @@ EXTRA_DIST = \
 	users-guide/vcl-hashing.rst \
 	users-guide/vcl-inline-c.rst \
 	users-guide/vcl-intro.rst \
-	users-guide/vcl-saint-and-grace.rst \
+	users-guide/vcl-grace.rst \
 	users-guide/vcl-syntax.rst \
 	users-guide/vcl-variables.rst \
 	users-guide/vcl.rst
diff --git a/doc/sphinx/users-guide/vcl-grace.rst b/doc/sphinx/users-guide/vcl-grace.rst
new file mode 100644
index 0000000..f318aa3
--- /dev/null
+++ b/doc/sphinx/users-guide/vcl-grace.rst
@@ -0,0 +1,58 @@
+.. _users-guide-handling_misbehaving_servers:
+
+Misbehaving servers
+-------------------
+
+A key feature of Varnish is its ability to shield you from misbehaving
+web- and application servers.
+
+
+
+Grace mode
+~~~~~~~~~~
+
+When several clients are requesting the same page Varnish will send
+one request to the backend and place the others on hold while fetching
+one copy from the back end. In some products this is called request
+coalescing and Varnish does this automatically.
+
+If you are serving thousands of hits per second the queue of waiting
+requests can get huge. There are two potential problems - one is a
+thundering herd problem - suddenly releasing a thousand threads to
+serve content might send the load sky high. Secondly - nobody likes to
+wait. To deal with this we can instruct Varnish to keep
+the objects in cache beyond their TTL and to serve the waiting
+requests somewhat stale content.
+
+So, in order to serve stale content we must first have some content to
+serve. So to make Varnish keep all objects for 30 minutes beyond their
+TTL use the following VCL::
+
+  sub vcl_fetch {
+    set beresp.grace = 30m;
+  }
+
+Varnish still won't serve the stale objects. In order to enable
+Varnish to actually serve the stale object we must enable this on the
+request. Lets us say that we accept serving 15s old object.::
+
+  sub vcl_recv {
+    set req.grace = 15s;
+  }
+
+You might wonder why we should keep the objects in the cache for 30
+minutes if we are unable to serve them? Well, if you have enabled
+:ref:`users-guide-advanced_backend_servers-health` you can check if the
+backend is sick and if it is we can serve the stale content for a bit
+longer.::
+
+   if (! req.backend.healthy) {
+      set req.grace = 5m;
+   } else {
+      set req.grace = 15s;
+   }
+
+So, to sum up, grace mode solves two problems:
+ * it serves stale content to avoid request pile-up.
+ * it serves stale content if the backend is not healthy.
+
diff --git a/doc/sphinx/users-guide/vcl-saint-and-grace.rst b/doc/sphinx/users-guide/vcl-saint-and-grace.rst
deleted file mode 100644
index f318aa3..0000000
--- a/doc/sphinx/users-guide/vcl-saint-and-grace.rst
+++ /dev/null
@@ -1,58 +0,0 @@
-.. _users-guide-handling_misbehaving_servers:
-
-Misbehaving servers
--------------------
-
-A key feature of Varnish is its ability to shield you from misbehaving
-web- and application servers.
-
-
-
-Grace mode
-~~~~~~~~~~
-
-When several clients are requesting the same page Varnish will send
-one request to the backend and place the others on hold while fetching
-one copy from the back end. In some products this is called request
-coalescing and Varnish does this automatically.
-
-If you are serving thousands of hits per second the queue of waiting
-requests can get huge. There are two potential problems - one is a
-thundering herd problem - suddenly releasing a thousand threads to
-serve content might send the load sky high. Secondly - nobody likes to
-wait. To deal with this we can instruct Varnish to keep
-the objects in cache beyond their TTL and to serve the waiting
-requests somewhat stale content.
-
-So, in order to serve stale content we must first have some content to
-serve. So to make Varnish keep all objects for 30 minutes beyond their
-TTL use the following VCL::
-
-  sub vcl_fetch {
-    set beresp.grace = 30m;
-  }
-
-Varnish still won't serve the stale objects. In order to enable
-Varnish to actually serve the stale object we must enable this on the
-request. Lets us say that we accept serving 15s old object.::
-
-  sub vcl_recv {
-    set req.grace = 15s;
-  }
-
-You might wonder why we should keep the objects in the cache for 30
-minutes if we are unable to serve them? Well, if you have enabled
-:ref:`users-guide-advanced_backend_servers-health` you can check if the
-backend is sick and if it is we can serve the stale content for a bit
-longer.::
-
-   if (! req.backend.healthy) {
-      set req.grace = 5m;
-   } else {
-      set req.grace = 15s;
-   }
-
-So, to sum up, grace mode solves two problems:
- * it serves stale content to avoid request pile-up.
- * it serves stale content if the backend is not healthy.
-
diff --git a/doc/sphinx/users-guide/vcl.rst b/doc/sphinx/users-guide/vcl.rst
index defe566..ad65403 100644
--- a/doc/sphinx/users-guide/vcl.rst
+++ b/doc/sphinx/users-guide/vcl.rst
@@ -39,7 +39,7 @@ code commented out in builtin.vcl that ships with Varnish Cache.
    vcl-actions
    vcl-backends
    vcl-hashing
-   vcl-saint-and-grace
+   vcl-grace
    vcl-inline-c
    vcl-examples
    websockets



From perbu at varnish-software.com  Tue Apr  1 13:09:51 2014
From: perbu at varnish-software.com (Per Buer)
Date: Tue, 01 Apr 2014 15:09:51 +0200
Subject: [4.0] 836d855 Remove references to vcl_error.
Message-ID: <E1WUyRr-00037k-MQ@project.varnish-software.com>


commit 836d85599d38836df5d74c8245c240e4ad362e84
Author: Per Buer <perbu at varnish-software.com>
Date:   Thu Mar 20 12:25:55 2014 +0100

    Remove references to vcl_error.

diff --git a/doc/sphinx/users-guide/devicedetection.rst b/doc/sphinx/users-guide/devicedetection.rst
index e60b438..5750918 100644
--- a/doc/sphinx/users-guide/devicedetection.rst
+++ b/doc/sphinx/users-guide/devicedetection.rst
@@ -251,11 +251,11 @@ VCL::
         # call some detection engine
 
         if (req.http.X-UA-Device ~ "^mobile" || req.http.X-UA-device ~ "^tablet") {
-            error 750 "Moved Temporarily";
+            return(synth(750, "Moved Temporarily"));
         }
     }
      
-    sub vcl_error {
+    sub vcl_synth {
         if (obj.status == 750) {
             set obj.http.Location = "http://m.example.com" + req.url;
             set obj.status = 302;
diff --git a/doc/sphinx/users-guide/purging.rst b/doc/sphinx/users-guide/purging.rst
index 4aad344..058bd7d 100644
--- a/doc/sphinx/users-guide/purging.rst
+++ b/doc/sphinx/users-guide/purging.rst
@@ -99,14 +99,14 @@ You can also add bans to Varnish via HTTP. Doing so requires a bit of VCL::
 	  if (req.method == "BAN") {
                   # Same ACL check as above:
 		  if (!client.ip ~ purge) {
-			  error 405 "Not allowed.";
+			  return(synth(403, "Not allowed."));
 		  }
 		  ban("req.http.host == " + req.http.host +
 		        "&& req.url == " + req.url);
 
 		  # Throw a synthetic page so the
                   # request won't go to the backend.
-		  return(synth(200m"Ban added"));
+		  return(synth(200, "Ban added"));
 	  }
   }
 
@@ -130,7 +130,7 @@ You can use the following template to write ban lurker friendly bans::
   sub vcl_recv {
     if (req.method == "PURGE") {
       if (client.ip !~ purge) {
-        error 401 "Not allowed";
+        return(synth(403, "Not allowed"));
       }
       ban("obj.http.x-url ~ " + req.url); # Assumes req.url is a regex. This might be a bit too simple
     }
diff --git a/doc/sphinx/users-guide/vcl-built-in-subs.rst b/doc/sphinx/users-guide/vcl-built-in-subs.rst
index 73934c7..b30985c 100644
--- a/doc/sphinx/users-guide/vcl-built-in-subs.rst
+++ b/doc/sphinx/users-guide/vcl-built-in-subs.rst
@@ -22,8 +22,9 @@ yourself doing frequently.
 The vcl_recv subroutine may terminate with calling ``return()`` on one
 of the following keywords:
 
-  error 
-    Return the specified error code to the client and abandon the request.
+  synth 
+    Return a synthetic object with the specified error code to the
+    client and abandon the request.
 
   pass
     Switch to pass mode.  Control will eventually pass to vcl_pass.
@@ -51,7 +52,7 @@ shuffling bytes back and forth.
 The vcl_pipe subroutine may terminate with calling return() with one
 of the following keywords:
 
-  error code [reason]
+  synth(error code, reason)
     Return the specified error code to the client and abandon the request.
 
   pipe
@@ -68,7 +69,7 @@ submitted over the same client connection are handled normally.
 The vcl_pass subroutine may terminate with calling return() with one
 of the following keywords:
 
-  error [reason]
+  synth(error code, reason)
     Return the specified error code to the client and abandon the request.
 
   pass
@@ -157,13 +158,6 @@ keywords:
     error.
 
 
-.. XXX
-.. vcl_error
-.. ~~~~~~~~~
-
-.. Not sure if we're going to keep this around.
-
-
 vcl_backend_fetch
 ~~~~~~~~~~~~~~~~~
 



From lkarsten at varnish-software.com  Tue Apr  1 13:09:51 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:51 +0200
Subject: [4.0] 22dcc15 Updated to running.rst in users guide
Message-ID: <E1WUyRr-00037t-NV@project.varnish-software.com>


commit 22dcc15faf1aa41c90eeee64c06d25cb25a55cd6
Author: benc <benc at redpill-linpro.com>
Date:   Fri Mar 14 13:51:03 2014 +0100

    Updated to running.rst in users guide

diff --git a/doc/sphinx/index.rst b/doc/sphinx/index.rst
index dde63a6..830149a 100644
--- a/doc/sphinx/index.rst
+++ b/doc/sphinx/index.rst
@@ -18,19 +18,18 @@ Welcome to the Varnish documentation!
 Introduction
 ------------
 
-Varnish is a state of the art web accelerator. It has its mission in front of a
-web server and cache content.
+Varnish Cache is a web application accelerator also known as a caching HTTP reverse proxy. You install it in front of any server that speaks HTTP and configure it to cache the contents. Varnish Cache is really, really fast. It typically speeds up delivery with a factor of 300 - 1000x, depending on your architecture. 
+It has its mission in front of a
+web server and cache content and it makes your web site go faster.
 
-It makes your web site go faster.
-
-We suggest you start by reading the installation guide
-:ref:`install-index`. Once you have Varnish up and running go through
+To get started with Varnish-Cache we recommend that you read the installation guide
+:ref:`install-index`. Once you have Varnish up and running we recommend that you go through
 our tutorial - :ref:`tutorial-index`, and finally the :ref:`users_guide_index`.
 
-If you need to look up how a specific Varnish tool works, the
-:ref:`reference-index` can help you out. Changes from previous versions are in
-the :ref:`whats-new-index` chapter. In closing we have :ref:`phk`, a collection
-of blog posts related to Varnish and HTTP.
+If you need to find out how to use a specific Varnish tool, the
+:ref:`reference-index` contains detailed documentation over the tools. Changes from previous versions are located in
+the :ref:`whats-new-index` chapter. In closing, we have :ref:`phk`, a collection
+of blog posts from Poul-Henning Kamp related to Varnish and HTTP.
 
 
 
diff --git a/doc/sphinx/installation/bugs.rst b/doc/sphinx/installation/bugs.rst
index 98c0e19..dd71cc8 100644
--- a/doc/sphinx/installation/bugs.rst
+++ b/doc/sphinx/installation/bugs.rst
@@ -13,14 +13,17 @@ So if you run into a bug, it is important that you spend a little bit
 of time collecting the right information, to help us fix the bug.
 
 The most valuable information you can give us, is **always** how
-to trigger and reproduce the problem.  If you can tell us that, we
-rarely need anything else to solve it.  The caveat being, that we
+to trigger and reproduce the problem. If you can tell us that, we
+rarely need anything else to solve it.The caveat being, that we
 do not have a way to simulate high levels of real-life web-traffic,
 so telling us to "have 10.000 clients hit at once" does not really
 allow us to reproduce.
 
-Roughly we have three clases of bugs with Varnish, and the information
-we need to debug them depends on the kind of bug.
+To report a bug please follow the suggested procedure described in the "Trouble Tickets" 
+section of the documentation (above).
+
+Roughly we categorize bugs in to three kinds of bugs (described below) with Varnish. The information
+we need to debug them depends on what kind of bug we are facing.
 
 Varnish crashes
 ===============
@@ -32,11 +35,11 @@ does all the work, and the manager hangs around to resurrect it if it
 crashes.
 
 Therefore, the first thing to do if you see a Varnish crash, is to examine
-your syslogs to see if it has happened before.  (One site is rumoured
+your syslogs to see if it has happened before. (One site is rumoured
 to have had Varnish restarting every 10 minutes and *still* provide better
 service than their CMS system.)
 
-When it crashes, if at all possible, Varnish will spew out a crash dump
+When it crashes, which is highly unlikely to begin with, Varnish will spew out a crash dump
 that looks something like::
 
 	Child (32619) died signal=6 (core dumped)
@@ -62,7 +65,7 @@ If you can get that information to us, we are usually able to
 see exactly where things went haywire, and that speeds up bugfixing
 a lot.
 
-There will be a lot more information than this, and before sending
+There will be a lot more information in the crash dump besides this, and before sending
 it all to us, you should obscure any sensitive/secret
 data/cookies/passwords/ip# etc.  Please make sure to keep context
 when you do so, ie: do not change all the IP# to "X.X.X.X", but
@@ -73,7 +76,7 @@ The most important line is the "Panic Message", which comes in two
 general forms:
 
 "Missing errorhandling code in ..."
-	This is a place where we can conceive ending up, and have not
+	This is a situation where we can conceive Varnish ending up, which we have not
 	(yet) written the padded-box error handling code for.
 
 	The most likely cause here, is that you need a larger workspace
@@ -84,11 +87,11 @@ general forms:
 
 "Assert error in ..."
 	This is something bad that should never happen, and a bug
-	report is almost certainly in order.  As always, if in doubt
+	report is almost certainly in order. As always, if in doubt
 	ask us on IRC before opening the ticket.
 
 In your syslog it may all be joined into one single line, but if you
-can reproduce the crash, do so while running varnishd manually:
+can reproduce the crash, do so while running `varnishd` manually:
 
 	``varnishd -d <your other arguments> |& tee /tmp/_catch_bug``
 
@@ -105,7 +108,7 @@ kill the process and send us an email saying "Varnish hung, I
 restarted it" which gives us only about 1.01 bit of usable debug
 information to work with.
 
-What we need here is all the information can you squeeze out of
+What we need here is all the information you can squeeze out of
 your operating system **before** you kill the Varnish process.
 
 One of the most valuable bits of information, is if all Varnish'
@@ -115,18 +118,21 @@ furiously on some futile condition.
 Commands like ``top -H`` or ``ps -Haxlw`` or ``ps -efH`` should be
 able to figure that out.
 
+.. XXX:Maybe a short description of what valuable information the various commands above generates? /benc 
+
+
 If one or more threads are spinning, use ``strace`` or ``ktrace`` or ``truss``
 (or whatever else your OS provides) to get a trace of which system calls
-the Varnish process issues.  Be aware that this may generate a lot
-of very repetitive data, usually one second worth is more than enough.
+the Varnish process issues. Be aware that this may generate a lot
+of very repetitive data, usually one second worth of data is more than enough.
 
 Also, run ``varnishlog`` for a second, and collect the output
 for us, and if ``varnishstat`` shows any activity, capture that also.
 
 When you have done this, kill the Varnish *child* process, and let
 the *master* process restart it.  Remember to tell us if that does
-or does not work.  If it does not, kill all Varnish processes, and
-start from scratch.  If that does not work either, tell us, that
+or does not work. If it does not, kill all Varnish processes, and
+start from scratch. If that does not work either, tell us, that
 means that we have wedged your kernel.
 
 
@@ -139,10 +145,12 @@ what is wrong about what Varnish does.
 
 Be aware, that often Varnish does exactly what you asked it to, rather
 than what you intended it to do. If it sounds like a bug that would
-have tripped up everybody else, take a moment to read though your
-VCL and see if it really does what you think.
+have tripped up everybody else, take a moment to read through your
+VCL and see if it really does what you think it does.
 
 You can also try setting the ``vcl_trace`` parameter, that will generate log
 records with like and character number for each statement executed in your VCL
 program.
 
+.. XXX:Example of the command perhaps? benc
+
diff --git a/doc/sphinx/installation/help.rst b/doc/sphinx/installation/help.rst
index aaea9ec..946e921 100644
--- a/doc/sphinx/installation/help.rst
+++ b/doc/sphinx/installation/help.rst
@@ -1,15 +1,14 @@
-%%%%%%%%%%%%%%%%%%
-Getting hold of us
-%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%
+Getting help
+%%%%%%%%%%%%
 
 Getting hold of the gang behind Varnish is pretty straight forward,
-we try to help as much as time permits and have tried to streamline
+we try to help out as much as time permits and have tried to streamline
 this process as much as possible.
 
 But before you grab hold of us, spend a moment composing your thoughts and
-formulate your question. There is nothing as pointless as simply telling
-us "Varnish does not work for me" with no further information to give
-any clue to why.
+formulate your question. From our perspective there is nothing as pointless as simply telling
+us "Varnish does not work for me" with no further information. This does not give us any relevant information to use when trying to figure out whats wrong. 
 
 And before you even do that, do a couple of searches to see if your
 question is already answered, if it has been, you will get your answer
@@ -20,7 +19,7 @@ IRC Channel
 
 The most immediate way to get hold of us is to join our IRC channel:
 
-	``#varnish on server irc.linpro.no``
+	`#varnish on server irc.linpro.no`
 
 The main timezone of the channel is Europe work hours.
 
@@ -32,27 +31,27 @@ If the channel is all quiet, try again some time later, we do have lives,
 families and jobs to deal with also.
 
 You are more than welcome to just hang out, and while we don't mind
-the occational intrusion of the real world into the flow, keep
-it mostly on topic, and don't paste random links unless they are
+the occasional intrusion from the real world into our flow, we try and keep
+it mostly on topic, and please don't paste random links unless they are
 *really* spectacular and intelligent.
 
 Mailing Lists
 =============
 
-Getting on or off our mailing lists happens through Mailman_.
+Subscribing or unsubscribing to our mailing lists is handled through our Mailman_.
 
-If you are going to use Varnish, subscribing to our ``varnish-announce``
-mailing list is probably a very good idea. The typical pattern is that
+If you are going to use Varnish, subscribing to our `varnish-announce`
+mailing list is a very good idea. The typical pattern is that
 people spend some time getting Varnish running, and then more or less
 forget about it. Therefore the announce list is a good way to be
-reminded about new releases, bad bugs or security holes.
+reminded about new releases, bugs or potential (security) vulnerabilities.
 
 
-The ``varnish-misc`` mailing list is for general banter, questions,
+The `varnish-misc` mailing list is for general banter, questions,
 suggestions, ideas and so on.  If you are new to Varnish it may pay
 off to subscribe to it, simply to have an ear to the telegraph-pole
-and learn some smart tricks.  This is a good place to ask for help
-with more complex issues, that require quoting of files and long
+and potentially learn some smart tricks.  This is also a good place to ask for help
+with more complex issues, that may require file-chunks, references to files and/or long
 explanations.
 
 Make sure to pick a good subject line, and if the subject of the
@@ -61,26 +60,29 @@ with hundreds of emails per day, after spam-filters, and we need all
 the help we can get to pick the interesting ones.
 
 
-The ``varnish-dev`` mailing list is used by the developers and is
-usually quite focused on source-code and such.  Everybody on
-the -dev list is also on -misc, so cross-posting only serves to annoy
+The `varnish-dev` mailing list is used by the developers and is
+usually quite focused on source-code and such. Everybody on
+the `-dev` list is also on `-misc`, so cross-posting only serves to annoy
 those people.
 
+We also maintain a community wiki_ for Varnish, there you will find information on planned events, meetings, current backlog, troube tickets ,  and links to resources and documentation. 
+
 .. XXX: we should introduce the wiki (if we care about it) before
 .. we start referring to it (below). Make a wiki chapter?
 
-
 Trouble Tickets
 ===============
 
 Please do not open a trouble ticket, unless you have spotted an actual
 bug in Varnish.  Ask on IRC first if you are in doubt.
 
-The reason for this policy, is to avoid the bugs being drowned in a
-pile of sensible suggestions for future enhancements and call for help
-from people who forget to check back if they get it and so on.
+The reason for this policy, is to avoid bugs being drowned in a
+pile of other `issues`, feature suggestions for future releases, and double postings of calls for help
+from people who forgot to check back on already opened Tickets. 
 
-We track suggestions and ideas in our `"Shopping-List" wiki page`_, and user
+.. XXX: Not sure what you want with the last sentence above. benc
+
+We instead track suggestions and feature ideas in our `"Shopping-List" wiki page`_, and through user
 support via email and IRC.
 
 Commercial Support
@@ -90,6 +92,8 @@ The following companies offer commercial Varnish support, and are listed
 here for your convenience.  If you want your company listed here, drop
 an email to phk at FreeBSD.org.
 
+.. XXX: Should we perhaps enhance this to explain Varnish Plus? benc
+
 	Varnish Software
 	sales at varnish-software.com
 
@@ -97,3 +101,4 @@ an email to phk at FreeBSD.org.
 .. _mailman: http://lists.varnish-cache.org/mailman/listinfo
 .. _pastebin: http://gist.github.com/
 .. _"Shopping-List" wiki page: http://varnish-cache.org/wiki/PostTwoShoppingList
+.. _wiki: https://www.varnish-cache.org/trac   
diff --git a/doc/sphinx/installation/index.rst b/doc/sphinx/installation/index.rst
index 7952958..03e60b8 100644
--- a/doc/sphinx/installation/index.rst
+++ b/doc/sphinx/installation/index.rst
@@ -4,11 +4,8 @@
 Varnish Installation
 %%%%%%%%%%%%%%%%%%%%
 
-This document explains how to get Varnish onto your system, where
-to get help, how report bugs and so on.
-
-In other words, it is a manual about pretty much everything else than
-actually using Varnish to move traffic.
+This section covers installation prerequisites, a step-by-step installation procedure, how and where
+to get help, and how to report bugs. It also contains a set of platform specific notes to aid you when installing Varnish on certain platforms.
 
 .. XXX: rewrite the last paragraph.
 
diff --git a/doc/sphinx/installation/install.rst b/doc/sphinx/installation/install.rst
index a32eb68..348b6ef 100644
--- a/doc/sphinx/installation/install.rst
+++ b/doc/sphinx/installation/install.rst
@@ -6,15 +6,15 @@ Installing Varnish
 With open source software, you can choose to install binary packages
 or compile it yourself from source code. To install a package or compile
 from source is a matter of personal taste. If you don't know which
-method to choose, read the whole document and choose the method you
-are most comfortable with.
+method to choose, we recommend that you read this whole section and then choose the method you
+feel most comfortable with.
 
 
 Source or packages?
 -------------------
 
 Installing Varnish on most relevant operating systems can usually
-be done with with the systems package manager, typical examples
+be done with with the specific systems package manager, typical examples
 being:
 
 FreeBSD
@@ -38,13 +38,13 @@ Varnish is included in the `EPEL
 incompatible syntax changes in newer versions of Varnish, only older
 versions are available.
 
-We recommend that you install the latest version from our repository.
+We therefore recommend that you install the latest version directly from our repository, as described above.
 
 Debian/Ubuntu
 -------------
 
 Varnish is distributed with both Debian and Ubuntu. In order to get
-Varnish up and running type `sudo apt-get install varnish`. Please
+Varnish up and running type ``sudo apt-get install varnish``. Please
 note that this might not be the latest version of Varnish.  If you
 need a later version of Varnish, please follow the online installation
 instructions for `Debian
@@ -59,16 +59,16 @@ If there are no binary packages available for your system, or if you
 want to compile Varnish from source for other reasons, follow these
 steps:
 
-We recommend downloading a release tarball, which you can find on
+Download the appropriate release tarball, which you can find on
 `repo.varnish-cache.org/source <http://repo.varnish-cache.org/source/>`_.
 
 Alternatively, if you want to hack on Varnish, you should clone our
 git repository by doing.
 
-      git clone git://git.varnish-cache.org/varnish-cache
+      ``git clone git://git.varnish-cache.org/varnish-cache``
 
 Please note that a git checkout will need some more build-dependencies
-than listed below, in particular the Python Docutils and Sphinx.
+than listed below, in particular the `Python Docutis` and `Sphinx`.
 
 Build dependencies on Debian / Ubuntu
 --------------------------------------
@@ -76,21 +76,21 @@ Build dependencies on Debian / Ubuntu
 In order to build Varnish from source you need a number of packages
 installed. On a Debian or Ubuntu system these are:
 
-* autotools-dev
-* automake1.11
-* libtool
-* autoconf
-* libncurses-dev
-* groff-base
-* libpcre3-dev
-* pkg-config
-* make
-* libedit-dev
+* `autotools-dev`
+* `automake1.1`
+* `libtool`
+* `autoconf`
+* `libncurses-dev`
+* `groff-base`
+* `libpcre3-dev`
+* `pkg-config`
+* `make`
+* `libedit-dev`
 
 If you're building from git, you also need the following:
 
-* python-docutils
-* python-sphinx (optional, if you want to build the documentation)
+* `python-docutils`
+* `python-sphinx` (optional, if you want to build the documentation)
 
 Build dependencies on RedHat / CentOS
 --------------------------------------
@@ -98,19 +98,19 @@ Build dependencies on RedHat / CentOS
 To build Varnish on a RedHat or CentOS system you need the following
 packages installed:
 
-* automake
-* autoconf
-* libtool
-* ncurses-devel
-* groff
-* pcre-devel
-* pkgconfig
-* libedit-devel
+* `automake`
+* `autoconf`
+* `libtool`
+* `ncurses-devel`
+* `groff`
+* `pcre-devel`
+* `pkgconfig`
+* `libedit-devel`
 
 If you're building from git, you also need the following:
 
-* docutils
-* python-sphinx (optional, if you want to build the documentation)
+* `docutils`
+* `python-sphinx` (optional, if you want to build the documentation)
 
 Compiling Varnish
 -----------------
@@ -123,18 +123,17 @@ taken care of::
 	sh configure
 	make
 
-The ``configure`` script takes some arguments, but more likely than
-not you can forget about that for now, almost everything in Varnish
-are run time parameters.
+The `configure` script takes some arguments, but more likely than
+not you can forget about that for now, almost everything in Varnish can be tweaked with run time parameters.
 
 Before you install, you may want to run the test suite, make a cup of
-tea while it runs, it takes some minutes::
+tea while it runs, it usually takes a couple of minutes::
 
 	make check
 
-Don't worry if a single or two tests fail, some of the tests are a
-bit too timing sensitive (Please tell us which so we can fix it) but
-if a lot of them fails, and in particular if the ``b00000.vtc`` test
+Don't worry if one or two tests fail, some of the tests are a
+bit too timing sensitive (Please tell us which so we can fix them.) but
+if a lot of them fails, and in particular if the `b00000.vtc` test
 fails, something is horribly wrong, and you will get nowhere without
 figuring out what.
 
@@ -145,9 +144,9 @@ And finally, the true test of a brave heart::
 
 	sudo make install
 
-Varnish will now be installed in /usr/local. The varnishd binary is in
-/usr/local/sbin/varnishd and its default configuration will be
-/usr/local/etc/varnish/default.vcl.
+Varnish will now be installed in `/usr/loca``l. The ``varnishd` binary is in
+`/usr/local/sbin/varnishd` and its default configuration will be
+`/usr/local/etc/varnish/default.vcl`.
 
-You can now proceed to the :ref:`tutorial-index`.
+After succesful installation you are ready to proceed to the :ref:`tutorial-index`.
 
diff --git a/doc/sphinx/installation/platformnotes.rst b/doc/sphinx/installation/platformnotes.rst
index a7d3fb4..7f11eaa 100644
--- a/doc/sphinx/installation/platformnotes.rst
+++ b/doc/sphinx/installation/platformnotes.rst
@@ -31,7 +31,7 @@ performance.
 If you are running on 64bit OpenVZ (or Parallels VPS), you must reduce
 the maximum stack size before starting Varnish.
 
-The default allocates to much memory per thread, which will make Varnish fail
+The default allocates too much memory per thread, which will make Varnish fail
 as soon as the number of threads (traffic) increases.
 
 Reduce the maximum stack size by running::
@@ -44,13 +44,13 @@ TCP keep-alive configuration
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 On some Solaris, FreeBSD and OS X systems, Varnish is not able to set the TCP
-keep-alive values per socket, and therefor the tcp_keepalive_* Varnish runtime
-parameters are not available. On these platforms it can be benefitial to tune
+keep-alive values per socket, and therefore the *tcp_keepalive_* Varnish runtime
+parameters are not available. On these platforms it can be beneficial to tune
 the system wide values for these in order to more reliably detect remote close
 for sessions spending long time on waitinglists. This will help free up
 resources faster.
 
-Systems to not support TCP keep-alive values per socket include:
+Systems that does not support TCP keep-alive values per socket include:
 
 - Solaris releases prior to version 11
 - FreeBSD releases prior to version 9.1
@@ -59,9 +59,11 @@ Systems to not support TCP keep-alive values per socket include:
 On platforms with the necessary socket options the defaults are set
 to:
 
-- tcp_keepalive_time = 600 seconds
-- tcp_keepalive_probes = 5
-- tcp_keepalive_intvl = 5 seconds
+- `tcp_keepalive_time` = 600 seconds
+- `tcp_keepalive_probes` = 5
+- `tcp_keepalive_intvl` = 5 seconds
 
 Note that Varnish will only apply these run-time parameters so long as
 they are less than the system default value.
+
+.. XXX:Maybe a sample-command of using/setting/changing these values? benc
diff --git a/doc/sphinx/installation/prerequisites.rst b/doc/sphinx/installation/prerequisites.rst
index 736bcd8..7ef0bd7 100644
--- a/doc/sphinx/installation/prerequisites.rst
+++ b/doc/sphinx/installation/prerequisites.rst
@@ -7,18 +7,17 @@ In order for you to install Varnish you must have the following:
   * A recent, preferably server grade, computer.
   * A fairly modern and 64 bit version of either
     - Linux
-    - FreeBSD
-    - Solaris (x86 only)
-  * root access to said system.
+    - FreeBSD, or
+    - Solaris (x86 only).
+  * Root access.
 
 
-Varnish can be installed on other UNIX systems as well, but it is not
-tested particularly well on these platforms. Varnish is, from time to
+Varnish can be installed on other UNIX systems as well, but it is not extensively or systematically tested by us on other systems than the above. Varnish is, from time to
 time, said to work on:
 
-  * 32 bit versions of the before-mentioned systems.
-  * OS X
-  * NetBSD
-  * OpenBSD
-  * Windows with Cygwin
+  * 32 bit versions of the before-mentioned systems,
+  * OS X,
+  * NetBSD,
+  * OpenBSD, and
+  * Windows with Cygwin.
 
diff --git a/doc/sphinx/tutorial/backend_servers.rst b/doc/sphinx/tutorial/backend_servers.rst
index 476d325..1c9ab65 100644
--- a/doc/sphinx/tutorial/backend_servers.rst
+++ b/doc/sphinx/tutorial/backend_servers.rst
@@ -3,14 +3,14 @@
 Backend servers
 ---------------
 
-Varnish has a concept of "backend" or "origin" servers. A backend
-server is the server providing the content Varnish will accelerate.
+Varnish has a concept of `backend` or `origin` servers. A backend
+server is the server providing the content Varnish will accelerate via the cache.
 
 Our first task is to tell Varnish where it can find its content. Start
 your favorite text editor and open the Varnish default configuration
 file. If you installed from source this is
-/usr/local/etc/varnish/default.vcl, if you installed from a package it
-is probably /etc/varnish/default.vcl.
+`/usr/local/etc/varnish/default.vcl`, if you installed from a package it
+is probably `/etc/varnish/default.vcl`.
 
 If you've been following the tutorial there is probably a section of
 the configuration that looks like this:::
@@ -41,11 +41,16 @@ localhost, port 8080.::
   }
 
 
-Varnish can have several backends defined and can you can even join
+Varnish can have several backends defined and can even join
 several backends together into clusters of backends for load balancing
 purposes, having Varnish pick one backend based on different
 algorithms. 
 
 A lot of the power of Varnish Cache comes from it's design, which
-might not be what you are expecting. Next, let's have a look at some of
-them.
+might not be what you are expecting.
+
+.. XXX:What am I expecting? benc
+
+Next, let's have a look at some of what makes Varnish unique and what you can do with it.
+
+
diff --git a/doc/sphinx/tutorial/index.rst b/doc/sphinx/tutorial/index.rst
index c540f8e..44008e7 100644
--- a/doc/sphinx/tutorial/index.rst
+++ b/doc/sphinx/tutorial/index.rst
@@ -4,12 +4,12 @@
 The Varnish Tutorial
 %%%%%%%%%%%%%%%%%%%%
 
-This documents the basic concept. What is Varnish and how does it
-work? How can I get it up and running. Most likely you would want to
-continue with the users guide (:ref:`users-guide-index`) afterwards.
+This section covers the Varnish basics in a tutorial form. It will cover what Varnish is and how it
+works. It also covers how to get Varnish up and running. After this section you probably would want to
+continue with the users guide (:ref:`users-guide-index`).
 
 If you're reading this on the web note the "Next topic" and "Previous
-topic" on the right side.
+topic" links on the right side of each page.
 
 .. toctree:: :maxdepth: 1
 
diff --git a/doc/sphinx/tutorial/introduction.rst b/doc/sphinx/tutorial/introduction.rst
index 9e6c541..98cfddc 100644
--- a/doc/sphinx/tutorial/introduction.rst
+++ b/doc/sphinx/tutorial/introduction.rst
@@ -3,25 +3,22 @@
 The fundamentals of web proxy caching with Varnish
 --------------------------------------------------
 
-Varnish is a caching HTTP reverse proxy. It receives requests from
-clients and tries to answer them from its cache. If it cannot answer
-the request from its cache it will forward the request to the backend,
-fetch the response, store it and deliver it to the client.
+Varnish is a caching HTTP reverse proxy. It recieves requests from
+clients and tries to answer them from the cache. If Varnish cannot answer
+the request from the cache it will forward the request to the backend,
+fetch the response, store it in the cache and deliver it to the client.
 
 When Varnish has a cached response ready it is typically delivered in
 a matter of microseconds, several orders of magnitude faster than your
-typical application server, so you want to make sure to have it answer
-as many of the requests as possible.
+typical backend server, so you want to make sure to have Varnish answer
+as many of the requests as possible directly from the cache.
 
 Varnish decides whether it can store the content or not based on the
 response it gets back from the backend. The backend can instruct
 Varnish to cache the content with the HTTP response header
-Cache-Control. There are a few other conditions where Varnish will not
-cache, the most common one being cookies. Since cookies is a good
-indication that a web object is personalised, Varnish will with its
-default configuration, not cache it.
-
-This behaviour and most other behaviour can be changed using policies
+`Cache-Control`. There are a few conditions where Varnish will not
+cache, the most common one being the use of cookies. Since cookies indicates a client-specific web object, Varnish will by default not cache it. 
+This behaviour as most of Varnish functionality can be changed using policies
 written in the Varnish Configuration Language. See the Users Guide
 for more information on how to do that.
 
@@ -34,6 +31,8 @@ turning performance into a non-issue. You get to focus on how your web
 application work and you can allow yourself, to some degree, to care
 less about performance and scalability.
 
+.. XXX:Not totally sure what the last sentence above means. benc
+
 Flexibility
 ~~~~~~~~~~~
 
@@ -77,3 +76,5 @@ information and signup.  There is also a web forum on the same site.
 
 Now that you have a vague idea on what Varnish Cache is, let see if we
 can get it up and running.
+
+.. XXX:The above three paragraphs are repetetive this is already handled in previos chapters. The only new information is Governing Board which could be moved to the introduction and the paragraphs scrapped. benc
diff --git a/doc/sphinx/tutorial/peculiarities.rst b/doc/sphinx/tutorial/peculiarities.rst
index af12eff..45f2f25 100644
--- a/doc/sphinx/tutorial/peculiarities.rst
+++ b/doc/sphinx/tutorial/peculiarities.rst
@@ -3,16 +3,19 @@ Peculiarities
 -------------
 
 There are a couple of things that are different with Varnish Cache, as
-opposed to other programs. One thing you've already seen - VCL. I'll
-just give you a very quick tour of the other pecularities.
+opposed to other programs. One thing you've already seen - VCL. In this section we provide a very quick tour of other pecularities you need to know about to get the most out of Varnish.
 
 Configuration
 ~~~~~~~~~~~~~
 
 The Varnish Configuration is written in VCL. When Varnish is ran this
 configuration is transformed into C code and then fed into a C
-compiler, loaded and run. So, as opposed to declaring various
-settings, you write polices on how the incoming traffic should be
+compiler, loaded and executed.
+
+.. XXX:Ran sounds strange above, maybe "is running" "is started" "executes"? benc
+
+So, as opposed to switching various
+settings on or off, you write polices on how the incoming traffic should be
 handled.
 
 
@@ -20,8 +23,8 @@ varnishadm
 ~~~~~~~~~~
 
 Varnish Cache has an admin console. You can connect it it through the
-"varnishadm" command. In order to connect the user needs to be able to
-read /etc/varnish/secret in order to authenticate.
+``varnishadm`` command. In order to connect the user needs to be able to
+read `/etc/varnish/secret` in order to authenticate.
 
 Once you've started the console you can do quite a few operations on
 Varnish, like stopping and starting the cache process, load VCL,
@@ -30,11 +33,13 @@ adjust the built in load balancer and invalidate cached content.
 It has a built in command "help" which will give you some hints on
 what it does.
 
+.. XXX:sample of the command here. benc
+
 varnishlog
 ~~~~~~~~~~
 
-Varnish does not log to disk. Instead it logs to a bit of memory. It
-is like a stream of logs. At any time you'll be able to connect to the
+Varnish does not log to disk. Instead it logs to a chunk of memory. It
+is actually streaming the logs. At any time you'll be able to connect to the
 stream and see what is going on. Varnish logs quite a bit of
 information. You can have a look at the logstream with the command
 ``varnishlog``.
diff --git a/doc/sphinx/tutorial/putting_varnish_on_port_80.rst b/doc/sphinx/tutorial/putting_varnish_on_port_80.rst
index acef0c6..0fc2ddc 100644
--- a/doc/sphinx/tutorial/putting_varnish_on_port_80.rst
+++ b/doc/sphinx/tutorial/putting_varnish_on_port_80.rst
@@ -9,13 +9,15 @@ First we stop varnish::
 
      # service varnish stop
 
+.. XXX:This renders to a different font than other commands. it should be the double backtick format for the command. benc
+
 Now we need to edit the configuration file that starts Varnish. 
 
 
 Debian/Ubuntu
 ~~~~~~~~~~~~~
 
-On Debian/Ubuntu this is /etc/default/varnish. In the file you'll find
+On Debian/Ubuntu this is `/etc/default/varnish`. In the file you'll find
 some text that looks like this::
 
   DAEMON_OPTS="-a :6081 \
@@ -36,7 +38,7 @@ Red Hat EL / Centos
 ~~~~~~~~~~~~~~~~~~~
 
 On Red Hat EL / Centos
-On Red Hat/Centos it is /etc/sysconfig/varnish
+On Red Hat/Centos it is `/etc/sysconfig/varnish`
 
 
 Restarting Varnish
diff --git a/doc/sphinx/tutorial/starting_varnish.rst b/doc/sphinx/tutorial/starting_varnish.rst
index 58bf574..2896394 100644
--- a/doc/sphinx/tutorial/starting_varnish.rst
+++ b/doc/sphinx/tutorial/starting_varnish.rst
@@ -4,36 +4,35 @@
 Starting Varnish
 ----------------
 
-This tutorial will assume that you are either running Ubuntu, Debian,
+This tutorial will assume that you are running Varnish on Ubuntu, Debian,
 Red Hat Enterprise Linux or Centos. Those of you running on other
 platforms might have to do some mental translation exercises in order
-to follow me. Since you're on a "weird" platform you're probably used
+to follow this. Since you're on a "weird" platform you're probably used
 to it. :-)
 
-I assume you have Varnish Cache installed.
+Make sure you have Varnish succesfully installed (following one of the procedures described in "Installing Varnish" above.
 
-You start Varnish with ``service varnish start``.  This will start
+When properly installed you start Varnish with ``service varnish start``.  This will start
 Varnish if it isn't already running.
 
+.. XXX:What does it do if it is already running? benc
+
 Now you have Varnish running. Let us make sure that it works
 properly. Use your browser to go to http://127.0.0.1:6081/
-(obviously, you should replace the IP address with one on your own
-system). 
-
+(obviously, you should replace the IP address with the IP for the machine that currently runs Varnish. 
 The default configuration will try to forward requests to a web
-service running on the same computer as Varnish was installed at,
-port 8080.
+application running on the same machine as Varnish was installed on. Varnish expects the web application to be exposed over http on port 8080.
 
-If there is no web application being served up there Varnish will
+If there is no web application being served up on that location Varnish will
 issue an error. Varnish Cache is very conservative about telling the
 world what is wrong so whenever something is amiss it will issue the
 same generic "Error 503 Service Unavilable".
 
 You might have a web application running on some other port or some
-other computer. Let's edit the configuration and make it point to
+other machine. Let's edit the configuration and make it point to
 something that actually works.
 
-Fire up your favorite editor and edit /etc/varnish/default.vcl. Most
+Fire up your favorite editor and edit `/etc/varnish/default.vcl`. Most
 of it is commented out but there is some text that is not. It will
 probably look like this::
 
diff --git a/doc/sphinx/users-guide/index.rst b/doc/sphinx/users-guide/index.rst
index 7cd0c25..3d4d831 100644
--- a/doc/sphinx/users-guide/index.rst
+++ b/doc/sphinx/users-guide/index.rst
@@ -15,7 +15,7 @@ The Varnish documentation consists of three main documents:
   looking up specific questions.
 
 After :ref:`users_intro`, this Users Guide is organized in sections
-along the major interfaces to Varnish as a service:
+following the major interfaces to Varnish as a service:
 
 :ref:`users_running` is about getting Varnish configured, with
 respect to storage, sockets, security and how you can control and
@@ -25,12 +25,12 @@ communicate with Varnish once it is running.
 HTTP requests the way you want, what to cache, how to cache it,
 modifying HTTP headers etc. etc.
 
-:ref:`users_report` explains how you can see and monitor what Varnish does,
-from transaction level to aggregate statistics.
+:ref:`users_report` explains how you can monitor what Varnish does,
+from a transactional level to aggregating statistics.
 
 :ref:`users_performance` is about tuning your website with Varnish.
 
-:ref:`users_trouble` is for locating and fixing trouble with Varnish.
+:ref:`users_trouble` is for locating and fixing common issues with Varnish.
 
 .. toctree::
    :maxdepth: 2
diff --git a/doc/sphinx/users-guide/intro.rst b/doc/sphinx/users-guide/intro.rst
index 6a67fc5..8b31b32 100644
--- a/doc/sphinx/users-guide/intro.rst
+++ b/doc/sphinx/users-guide/intro.rst
@@ -3,10 +3,12 @@
 The Big Varnish Picture
 =======================
 
-What is in this package called "Varnish", what are all the different
-bits and pieces named and will you need a hex-wrench for assembly?
+In this section we will cover the questions:
+- What is in this package called "Varnish"?
+- what are all the different bits and pieces named? 
+- Will you need a hex-wrench for assembly?
 
-The two main parts of Varnish are the two processes in the varnishd
+The two main parts of Varnish are the two processes in the `varnishd`
 program. The first process is called "the manager", and its job is to
 talk to you, the administrator, and make the things you ask for
 happen.
@@ -15,12 +17,12 @@ The second process is called "the worker" or just "the child" and
 this is the process which does all the actual work with your HTTP
 traffic.
 
-When you start varnishd, you start the manager process, and once it is
+When you start `varnishd`, you start the manager process, and once it is
 done handling all the command line flags, it will start the child
 process for you. Should the child process die, the manager will start
 it again for you, automatically and right away.
 
-The main reason for this division of labor is security:  The manager
+The main reason for this division of labor is security: The manager
 process will typically run with "root" permissions, in order to
 open TCP socket port 80, but it starts the child process with minimal
 permissions, as a defensive measure.
@@ -39,11 +41,15 @@ or tie it to your CMS.
 
 All this is covered in :ref:`users_running`.
 
-How the child process should deal with the HTTP requests, what to
-cache, which headers to remove etc, is al specified in a small
+Things like, how the child process should deal with the HTTP requests, what to
+cache, which headers to remove etc, is all specified using a small
 programming language called VCL -- Varnish Configuration Language.
 The manager process will compile the VCL program and check it for
-errors, but it is the child process which runs the VCL program, for
+errors,
+
+.. XXX:What does manager do after compile and error-check? Maybe a short description of further handling when no errors as well as when errors? benc
+
+but it is the child process which runs the VCL program, for
 each and every HTTP request which comes in.
 
 Because the VCL is compiled to C code, and the C code is compiled
@@ -69,9 +75,9 @@ can do for your HTTP traffic, there really is no limit.
 
 :ref:`users_vcl` describes VCL and what it can do in great detail.
 
-Varnish uses a piece of shared memory to report its activity and
+Varnish uses a segment of shared memory to report and log its activities and
 status. For each HTTP request, a number of very detailed records will
-be appended to the log segment in this shared memory.  Other processes
+be appended to the log memory segment. Other processes
 can subscribe to log-records, filter them, and format them, for
 instance as Apache/NCSA style log records.
 
@@ -81,24 +87,25 @@ of cache hit-rate, resource usage and specific performance indicating
 metrics.
 
 Varnish comes with a number of tools which reports from shared
-memory, varnishlog, varnishstats, varnishncsa etc, and with a API
+memory, `varnishlog`, `varnishstats`, `varnishncsa` etc, and with an API
 library so you can write your own tools, should you need that.
-writing
 
 :ref:`users_report` explains how all that work.
 
-Presumably the reason why you are interested in Varnish, is that you
+Presumably the reason for your interest in Varnish, is that you
 want your website to work better. There are many aspects of
 performance tuning a website, from relatively simple policy decisions
 about what to cache, to designing a geographically diverse multilevel
 CDNs using ESI and automatic failover.
 
+.. XXX:CDNs or CDN? benc
+
 :ref:`users_performance` will take you through the possibilities
 and facilities Varnish offers.
 
-Finally, Murphys Law must be contended with: Things will go wrong, and
+Finally, Murphys Law must be referenced here: Things will go wrong, and
 more likely than not, they will do so at zero-zero-dark O'clock. Most
-likely during a hurricane, when your phones battery is flat and your
+likely during a hurricane, when your phone battery is flat and your
 wife had prepared a intimate evening to celebrate your anniversary.
 
 Yes, we've all been there, haven't we?



From lkarsten at varnish-software.com  Tue Apr  1 13:09:51 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:51 +0200
Subject: [4.0] 45637bb Done. Important changes are consequently commented and
 all comments are marked with the string 'benc' for easy identification. There
 are some heavy refactorings done, that fundamentally changes content meaing.
Message-ID: <E1WUyRr-000383-Os@project.varnish-software.com>


commit 45637bb9988ef7f20e1fab51ca8744c0a62a6651
Author: benc <benc at redpill-linpro.com>
Date:   Sun Mar 16 12:50:38 2014 +0100

    Done. Important changes are consequently commented and all comments are marked with the string 'benc' for easy identification. There are some heavy refactorings done, that fundamentally changes content meaing.

diff --git a/doc/sphinx/users-guide/command-line.rst b/doc/sphinx/users-guide/command-line.rst
index dc25087..9016d51 100644
--- a/doc/sphinx/users-guide/command-line.rst
+++ b/doc/sphinx/users-guide/command-line.rst
@@ -3,22 +3,22 @@
 Important command line arguments
 --------------------------------
 
-There a two command line arguments you will simply have choose
-values for, what TCP port serve HTTP from and where the backend
-server can be contacted.
+There a two command line arguments you have to set when starting Varnish, these are: 
+* what TCP port to serve HTTP from, and 
+* where the backend server can be contacted.
 
-If you run Varnish from a package for your operating system,
+If you have installed Varnish through using a provided operating system bound package,
 you will find the startup options here:
 
-* Debian, Ubuntu: /etc/default/varnish
-* Red Hat, Centos: /etc/sysconfig/varnish
-* FreeBSD: /etc/rc.conf (See also: /usr/local/etc/rc.d/varnishd)
+* Debian, Ubuntu: `/etc/default/varnish`
+* Red Hat, Centos: `/etc/sysconfig/varnish`
+* FreeBSD: `/etc/rc.conf` (See also: /usr/local/etc/rc.d/varnishd)
 
 
--a *listen_address*
+'-a' *listen_address*
 ^^^^^^^^^^^^^^^^^^^
 
-What address should Varnish listen to, and service HTTP requests from.
+The '-a' argument defines what address Varnish should listen to, and service HTTP requests from.
 
 You will most likely want to set this to ":80" which is the Well
 Known Port for HTTP.
@@ -26,7 +26,7 @@ Known Port for HTTP.
 You can specify multiple addresses separated by a comma, and you
 can use numeric or host/service names if you like, Varnish will try
 to open and service as many of them as possible, but if none of them
-can be opened, varnishd will not start.
+can be opened, `varnishd` will not start.
 
 Here are some examples::
 
@@ -36,16 +36,19 @@ Here are some examples::
 	-a '[fe80::1]:80'
 	-a '0.0.0.0:8080,[::]:8081'
 
-If your webserver runs on the same computer, you will have to move
+.. XXX:brief explanation of some of the more comples examples perhaps? benc
+
+If your webserver runs on the same machine, you will have to move
 it to another port number first.
 
--f *VCL-file* or -b *backend*
+'-f' *VCL-file* or '-b' *backend*
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
+
 Varnish needs to know where to find the HTTP server it is caching for.
-You can either specify it with -b, or you can put it in your own VCL file.
+You can either specify it with the '-b' argument, or you can put it in your own VCL file, specified with the '-f' argument.
 
-Using -b is a quick way to get started::
+Using '-b' is a quick way to get started::
 
 	-b localhost:81
 	-b thatotherserver.example.com:80
@@ -54,31 +57,34 @@ Using -b is a quick way to get started::
 Notice that if you specify a name, it can at most resolve to one IPv4
 *and* one IPv6 address.
 
-If you go with -f, you can start with a VCL file containing just::
+If you go with '-f', you can start with a VCL file containing just::
 
 	backend default {
 		.host = "localhost:81";
 	}
 
-which is exactly what -b does.
+which is exactly what '-b' does.
+
+.. XXX:What happens if I start with -b and then have the backend defined in my VCL? benc
 
 In both cases the built-in VCL code is appended.
 
 Other options
 ^^^^^^^^^^^^^
 
-Varnish has more command line arguments you can and maybe want
-to tweak, but to get started, the above will be sufficient.
+Varnish comes with an abundance of useful command line arguments. We recommend that you study them but not necessary use them all, but to get started, the above will be sufficient.
 
 By default Varnish will use 100 megabytes of malloc(3) storage
 for caching objects, if you want to cache more than that, you
 should look at the '-s' argument.
 
+.. XXX: 3? benc
+
 If you run a really big site, you may want to tune the number of
 worker threads and other parameters with the '-p' argument,
 but we generally advice not to do that unless you need to.
 
-Before you go into production, you may also want to re-visit the
+Before you go into production, you may also want to revisit the
 chapter
 :ref:`run_security` to see if you need to partition administrative
 privileges.
diff --git a/doc/sphinx/users-guide/compression.rst b/doc/sphinx/users-guide/compression.rst
index d611807..fc84f9f 100644
--- a/doc/sphinx/users-guide/compression.rst
+++ b/doc/sphinx/users-guide/compression.rst
@@ -3,31 +3,33 @@
 Compression
 -----------
 
-New in Varnish 3.0 was native support for compression, using gzip
+In Varnish 3.0 we introduced native support for compression, using gzip
 encoding. *Before* 3.0, Varnish would never compress objects. 
 
-In Varnish 3.0 compression defaults to "on", meaning that it tries to
+In Varnish 4.0 compression defaults to "on", meaning that it tries to
 be smart and do the sensible thing.
 
+.. XXX:Heavy refactoring to VArnish 4 above. benc
+
 If you don't want Varnish tampering with the encoding you can disable
-compression all together by setting the parameter http_gzip_support to
-*false*. Please see man :ref:`ref-varnishd` for details.
+compression all together by setting the parameter 'http_gzip_support' to
+false. Please see man :ref:`ref-varnishd` for details.
 
 
 Default behaviour
 ~~~~~~~~~~~~~~~~~
 
-The default for Varnish is to check if the client supports our
+The default behaviour for Varnish is to check if the client supports our
 compression scheme (gzip) and if it does it will override the
-Accept-Encoding header and set it to "gzip".
+'Accept-Encoding' header and set it to "gzip".
 
-When Varnish then issues a backend request the Accept-Encoding will
+When Varnish then issues a backend request the 'Accept-Encoding' will
 then only consist of "gzip". If the server responds with gzip'ed
 content it will be stored in memory in its compressed form. If the
-backend sends content in clear text it will be stored like that.
+backend sends content in clear text it will be stored in clear text.
 
 You can make Varnish compress content before storing it in cache in
-vcl_fetch by setting do_gzip to true, like this::
+`vcl_fetch` by setting 'do_gzip' to true, like this::
 
    sub vcl_backend_response {
         if (beresp.http.content-type ~ "text") {
@@ -38,9 +40,7 @@ vcl_fetch by setting do_gzip to true, like this::
 Please make sure that you don't try to compress content that is
 uncompressable, like jpgs, gifs and mp3. You'll only waste CPU
 cycles. You can also uncompress objects before storing it in memory by
-setting do_gunzip to *true* but I have no idea why anybody would want
-to do that.
-
+setting 'do_gunzip' to true but that will ususally not be the most sensible thing to do.
 Generally, Varnish doesn't use much CPU so it might make more sense to
 have Varnish spend CPU cycles compressing content than doing it in
 your web- or application servers, which are more likely to be
@@ -49,7 +49,7 @@ CPU-bound.
 GZIP and ESI
 ~~~~~~~~~~~~
 
-If you are using Edge Side Includes you'll be happy to note that ESI
+If you are using Edge Side Includes (ESIs) you'll be happy to note that ESI
 and GZIP work together really well. Varnish will magically decompress
 the content to do the ESI-processing, then recompress it for efficient
 storage and delivery. 
@@ -58,10 +58,10 @@ storage and delivery.
 Clients that don't support gzip
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-If the client does not support gzip the Accept-Encoding header is left
-alone and we'll end up serving whatever we get from the backend
-server. Remember that the Backend might tell Varnish to *Vary* on the
-Accept-Encoding.
+If the client does not support gzip the 'Accept-Encoding' header is left
+alone then we'll end up serving whatever we get from the backend
+server. Remember that the backend might tell Varnish to *Vary* on the
+'Accept-Encoding'.
 
 If the client does not support gzip but we've already got a compressed
 version of the page in memory Varnish will automatically decompress
@@ -71,5 +71,5 @@ the page while delivering it.
 A random outburst
 ~~~~~~~~~~~~~~~~~
 
-Poul has written :ref:`phk_gzip` which talks abit more about how the
+Poul-Henning Kamp has written :ref:`phk_gzip` which talks abit more about how the
 implementation works. 
diff --git a/doc/sphinx/users-guide/devicedetection.rst b/doc/sphinx/users-guide/devicedetection.rst
index 5750918..5067c4c 100644
--- a/doc/sphinx/users-guide/devicedetection.rst
+++ b/doc/sphinx/users-guide/devicedetection.rst
@@ -10,14 +10,13 @@ Use cases for this are for example to send size reduced files to mobile
 clients with small screens and on high latency networks, or to 
 provide a streaming video codec that the client understands.
 
-There are a couple of strategies on what to do with such clients:
-1) Redirect them to another URL.
-2) Use a different backend for the special clients.
-3) Change the backend requests so the usual backend sends tailored content.
+There are a couple of typical strategies to use for this type of scenario:
+1) Redirect to another URL.
+2) Use a different backend for the special client.
+3) Change the backend request so that the backend sends tailored content.
 
-To make the examples easier to understand, it is assumed in this text 
-that all the req.http.X-UA-Device header is present and unique per client class
-that content is to be served to. 
+To perhaps make the strategies easier to understand, we, in this context, assume
+that the `req.http.X-UA-Device` header is present and unique per client class. 
 
 Setting this header can be as simple as::
 
@@ -28,34 +27,34 @@ Setting this header can be as simple as::
    }
 
 There are different commercial and free offerings in doing grouping and
-identifying clients in further detail than this. For a basic and community
+identifying clients in further detail. For a basic and community
 based regular expression set, see
-https://github.com/varnish/varnish-devicedetect/ .
+https://github.com/varnish/varnish-devicedetect/.
 
 
 Serve the different content on the same URL
 -------------------------------------------
 
 The tricks involved are: 
-1. Detect the client (pretty simple, just include devicedetect.vcl and call
-it)
-2. Figure out how to signal the backend what client class this is. This
+1. Detect the client (pretty simple, just include `devicedetect.vcl` and call
+it).
+2. Figure out how to signal the backend the client class. This
 includes for example setting a header, changing a header or even changing the
 backend request URL.
-3. Modify any response from the backend to add missing Vary headers, so
+3. Modify any response from the backend to add missing 'Vary' headers, so
 Varnish' internal handling of this kicks in.
 4. Modify output sent to the client so any caches outside our control don't
 serve the wrong content.
 
-All this while still making sure that we only get 1 cached object per URL per
+All this needs to be done while still making sure that we only get one cached object per URL per
 device class.
 
 
 Example 1: Send HTTP header to backend
 ''''''''''''''''''''''''''''''''''''''
 
-The basic case is that Varnish adds the X-UA-Device HTTP header on the backend
-requests, and the backend mentions in the response Vary header that the content
+The basic case is that Varnish adds the 'X-UA-Device' HTTP header on the backend
+requests, and the backend mentions in the response 'Vary' header that the content
 is dependant on this header. 
 
 Everything works out of the box from Varnish' perspective.
@@ -103,13 +102,13 @@ Example 2: Normalize the User-Agent string
 ''''''''''''''''''''''''''''''''''''''''''
 
 Another way of signaling the device type is to override or normalize the
-User-Agent header sent to the backend.
+'User-Agent' header sent to the backend.
 
-For example
+For example::
 
     User-Agent: Mozilla/5.0 (Linux; U; Android 2.2; nb-no; HTC Desire Build/FRF91) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1
 
-becomes:
+becomes::
 
     User-Agent: mobile-android
 
@@ -218,7 +217,7 @@ Different backend for mobile clients
 ------------------------------------
 
 If you have a different backend that serves pages for mobile clients, or any
-special needs in VCL, you can use the X-UA-Device header like this::
+special needs in VCL, you can use the 'X-UA-Device' header like this::
 
     backend mobile {
         .host = "10.0.0.1";
diff --git a/doc/sphinx/users-guide/esi.rst b/doc/sphinx/users-guide/esi.rst
index 9035224..45f8814 100644
--- a/doc/sphinx/users-guide/esi.rst
+++ b/doc/sphinx/users-guide/esi.rst
@@ -3,19 +3,23 @@
 Content composition with Edge Side Includes
 -------------------------------------------
 
-Varnish can cache create web pages by putting different pages
-together. These *fragments* can have individual cache policies. If you
-have a web site with a list showing the 5 most popular articles on
-your site, this list can probably be cached as a fragment and included
-in all the other pages. Used properly it can dramatically increase
+Varnish can cache create web pages by assembling different pages, called `fragments`,
+together into one page. These `fragments` can have individual cache policies. If you
+have a web site with a list showing the five most popular articles on
+your site, this list can probably be cached as a `fragment` and included
+in all the other pages.
+
+.. XXX:What other pages? benc
+
+Used properly this strategy can dramatically increase
 your hit rate and reduce the load on your servers. 
 
-In Varnish we've only implemented a small subset of ESI. As of 2.1 we
-have three ESI statements:
+In Varnish we've only so far implemented a small subset of ESI. As of version 2.1 we
+have three ESI statements::
 
- * esi:include 
- * esi:remove
- * <!--esi ...-->
+ esi:include 
+ esi:remove
+ <!--esi ...-->
 
 Content substitution based on variables and cookies is not implemented
 but is on the roadmap. At least if you look at the roadmap from a
@@ -58,13 +62,13 @@ For ESI to work you need to activate ESI processing in VCL, like this::
 
 Example: esi:remove and <!--esi ... -->
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-The <esi:remove> and <!--esi ... --> constructs can be used to present
+The `<esi:remove>` and `<!--esi ... -->` constructs can be used to present
 appropriate content whether or not ESI is available, for example you can
 include content when ESI is available or link to it when it is not.
-ESI processors will remove the start ("<!--esi") and end ("-->") when
+ESI processors will remove the start ("<!--esi") and the end ("-->") when
 the page is processed, while still processing the contents. If the page
-is not processed, it will remain, becoming an HTML/XML comment tag.
-ESI processors will remove <esi:remove> tags and all content contained
+is not processed, it will remain intact, becoming a HTML/XML comment tag.
+ESI processors will remove `<esi:remove>` tags and all content contained
 in them, allowing you to only render the content when the page is not
 being ESI-processed.
 For example::
@@ -77,10 +81,10 @@ For example::
   <esi:include src="http://example.com/LICENSE" />
   -->
 
-Doing ESI on JSON and other non-XMLish content
+Doing ESI on JSON and other non-XML'ish content
 ----------------------------------------------
 
 Please note that Varnish will peek at the included content. If it
 doesn't start with a "<" Varnish assumes you didn't really mean to
 include it and disregard it. You can alter this behaviour by setting
-the esi_syntax parameter (see ref:`ref-varnishd`).
+the 'esi_syntax' parameter (see ref:`ref-varnishd`).
diff --git a/doc/sphinx/users-guide/increasing-your-hitrate.rst b/doc/sphinx/users-guide/increasing-your-hitrate.rst
index 3e1adc5..2f22e3c 100644
--- a/doc/sphinx/users-guide/increasing-your-hitrate.rst
+++ b/doc/sphinx/users-guide/increasing-your-hitrate.rst
@@ -7,37 +7,37 @@ Now that Varnish is up and running, and you can access your web
 application through Varnish. Unless your application is specifically
 written to work behind a web accelerator you'll probably need to do
 some changes to either the configuration or the application in order
-to get a high hit rate in Varnish.
+to get a high hitrate in Varnish.
 
 Varnish will not cache your data unless it's absolutely sure it is
 safe to do so. So, for you to understand how Varnish decides if and
-how to cache a page, I'll guide you through a couple of tools that you
-will find useful.
+how to cache a page, We'll guide you through a couple of tools that you
+should find useful to understand what is happening in your Varnish setup.
 
-Note that you need a tool to see what HTTP headers fly between you and
-the web server. On the Varnish server, the easiest is to use
-varnishlog and varnishtop but sometimes a client-side tool makes
-sense. Here are the ones I use.
+Note that you need a tool to see the HTTP headers that fly between Varnish and
+the backend. On the Varnish server, the easiest way to do this is to use
+`varnishlog` and `varnishtop` but sometimes a client-side tool makes
+sense. Here are the ones we commonly use.
 
 Tool: varnishtop
 ~~~~~~~~~~~~~~~~
 
 You can use varnishtop to identify what URLs are hitting the backend
 the most. ``varnishtop -i txurl`` is an essential command, showing you
-the top txurl requests Varnish is sending towards the backend. You can
-see some other examples of varnishtop usage in
+the top `txurl` requests Varnish is sending to the backend. You can
+see some other examples of `varnishtop` usage in
 :ref:`users-guide-statistics`.
 
 
 Tool: varnishlog
 ~~~~~~~~~~~~~~~~
 
-When you have identified the an URL which is frequently sent to the
-backend you can use varnishlog to have a look at the request.
+When you have identified an URL which is frequently sent to the
+backend you can use `varnishlog` to have a look at the request.
 ``varnishlog -c -m 'RxURL:^/foo/bar`` will show you the requests
-coming from the client (-c) matching /foo/bar.
+coming from the client ('-c') matching `/foo/bar`.
 
-For more information on how varnishlog works please see
+For more information on how `varnishlog` works please see
 :ref:`users-guide-logging` or man :ref:`ref-varnishlog`.
 
 For extended diagnostics headers, see
@@ -47,9 +47,9 @@ http://www.varnish-cache.org/trac/wiki/VCLExampleHitMissHeader
 Tool: lwp-request
 ~~~~~~~~~~~~~~~~~
 
-lwp-request is part of The World-Wide Web library for Perl. It's a
-couple of really basic programs that can execute an HTTP request and
-give you the result. I mostly use two programs, GET and HEAD.
+`lwp-request` is tool that is a part of The World-Wide Web library for Perl. It's a
+couple of really basic programs that can execute a HTTP request and
+show you the result. We mostly use the two programs, ``GET`` and ``HEAD``.
 
 vg.no was the first site to use Varnish and the people running Varnish
 there are quite clueful. So it's interesting to look at their HTTP
@@ -72,26 +72,28 @@ Headers. Let's send a GET request for their home page::
   X-VG-WebCache: joanie
   X-VG-WebServer: leon
 
-OK. Let me explain what it does. GET usually sends off HTTP 0.9
-requests, which lack the Host header. So I add a Host header with the
--H option. -U print request headers, -s prints response status, -e
-prints response headers and -d discards the actual content. We don't
+OK. Lets look at what ``GET`` does. ``GET`` usually sends off HTTP 0.9
+requests, which lack the 'Host' header. So we add a 'Host' header with the
+'-H' option. '-U' print request headers, '-s' prints response status, '-e'
+prints response headers and '-d' discards the actual content. We don't
 really care about the content, only the headers.
 
 As you can see, VG adds quite a bit of information in their
-headers. Some of the headers, like the X-Rick-Would-Never are specific
+headers. Some of the headers, like the 'X-Rick-Would-Never' are specific
 to vg.no and their somewhat odd sense of humour. Others, like the
-X-VG-Webcache are for debugging purposes. 
+'X-VG-Webcache' are for debugging purposes. 
 
 So, to check whether a site sets cookies for a specific URL, just do::
 
   GET -Used http://example.com/ |grep ^Set-Cookie
 
+.. XXX:Missing explanation and sample for HEAD here. benc
+
 Tool: Live HTTP Headers
 ~~~~~~~~~~~~~~~~~~~~~~~
 
-There is also a plugin for Firefox. *Live HTTP Headers* can show you
-what headers are being sent and recieved. Live HTTP Headers can be
+There is also a plugin for Firefox called `Live HTTP Headers`. This plugin can show you
+what headers are being sent and recieved. `Live HTTP Headers` can be
 found at https://addons.mozilla.org/en-US/firefox/addon/3829/ or by
 googling "Live HTTP Headers".
 
@@ -102,14 +104,14 @@ The role of HTTP Headers
 Along with each HTTP request and response comes a bunch of headers
 carrying metadata. Varnish will look at these headers to determine if
 it is appropriate to cache the contents and how long Varnish can keep
-the content.
+the content cached.
 
-Please note that when considering these headers Varnish actually
+Please note that when Varnish considers these headers Varnish actually
 considers itself *part of* the actual webserver. The rationale being
 that both are under your control. 
 
 The term *surrogate origin cache* is not really well defined by the
-IETF so RFC 2616 so the various ways Varnish works might differ from
+IETF or RFC 2616 so the various ways Varnish works might differ from
 your expectations.
 
 Let's take a look at the important headers you should be aware of:
@@ -119,8 +121,8 @@ Let's take a look at the important headers you should be aware of:
 Cookies
 -------
 
-Varnish will, in the default configuration, not cache a object coming
-from the backend with a Set-Cookie header present. Also, if the client
+Varnish will, in the default configuration, not cache an object coming
+from the backend with a 'Set-Cookie' header present. Also, if the client
 sends a Cookie header, Varnish will bypass the cache and go directly to
 the backend.
 
@@ -132,10 +134,10 @@ interest to the server.
 Cookies from the client
 ~~~~~~~~~~~~~~~~~~~~~~~
 
-For a lot of web application it makes sense to completely disregard the
+For a lot of web applications it makes sense to completely disregard the
 cookies unless you are accessing a special part of the web site. This
-VCL snippet in vcl_recv will disregard cookies unless you are
-accessing /admin/::
+VCL snippet in `vcl_recv` will disregard cookies unless you are
+accessing `/admin/`::
 
   if ( !( req.url ~ ^/admin/) ) {
     unset req.http.Cookie;
@@ -146,15 +148,15 @@ like removing one out of several cookies, things get
 difficult. Unfortunately Varnish doesn't have good tools for
 manipulating the Cookies. We have to use regular expressions to do the
 work. If you are familiar with regular expressions you'll understand
-whats going on. If you don't I suggest you either pick up a book on
-the subject, read through the *pcrepattern* man page or read through
+whats going on. If you aren't we recommend that you either pick up a book on
+the subject, read through the *pcrepattern* man page, or read through
 one of many online guides.
 
-Let me show you what Varnish Software uses. We use some cookies for
+Lets use the Varnish Software (VS) web as an example here. Very simplified the setup VS uses can be described as a Drupal-based backend with a Varnish cache infront. VS uses some cookies for
 Google Analytics tracking and similar tools. The cookies are all set
 and used by Javascript. Varnish and Drupal doesn't need to see those
 cookies and since Varnish will cease caching of pages when the client
-sends cookies we will discard these unnecessary cookies in VCL. 
+sends cookies Varnish will discard these unnecessary cookies in VCL. 
 
 In the following VCL we discard all cookies that start with a
 underscore::
@@ -164,8 +166,8 @@ underscore::
   // Remove a ";" prefix, if present.
   set req.http.Cookie = regsub(req.http.Cookie, "^;\s*", "");
 
-Let me show you an example where we remove everything except the
-cookies named COOKIE1 and COOKIE2 and you can marvel at it::
+Lets look at an example where we remove everything except the
+cookies named "COOKIE1" and "COOKIE2" and you can marvel at the "beauty" of it::
 
   sub vcl_recv {
     if (req.http.Cookie) {
@@ -181,10 +183,12 @@ cookies named COOKIE1 and COOKIE2 and you can marvel at it::
     }
   }
 
-A somewhat simpler example that can accomplish almost the same can be
-found below. Instead of filtering out the other cookies it picks out
-the one cookie that is needed, copies it to another header and then
-copies it back, deleting the original cookie header.::
+A somewhat simpler example that can accomplish almost the same functionality can be
+found below. Instead of filtering out "other" cookies it instead picks out
+"the one" cookie that is needed, copies it to another header and then
+copies it back to the request, deleting the original cookie header.
+.. XXX:Verify correctness of request above! benc
+::
 
   sub vcl_recv {
          # save the original cookie header so we can mangle it
@@ -200,30 +204,31 @@ copies it back, deleting the original cookie header.::
 There are other scary examples of what can be done in VCL in the
 Varnish Cache Wiki.
 
+.. XXX:Missing link here.
+
 
 Cookies coming from the backend
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-If your backend server sets a cookie using the Set-Cookie header
-Varnish will not cache the page in the default configuration.  A
-hit-for-pass object (see :ref:`user-guide-vcl_actions`) is created.
+If your backend server sets a cookie using the 'Set-Cookie' header
+Varnish will not cache the page when using the default configuration. A
+`hit-for-pass` object (see :ref:`user-guide-vcl_actions`) is created.
 So, if the backend server acts silly and sets unwanted cookies just unset
-the Set-Cookie header and all should be fine. 
-
+the 'Set-Cookie' header and all should be fine. 
 
 
 Cache-Control
 ~~~~~~~~~~~~~
 
-The Cache-Control instructs caches how to handle the content. Varnish
+The 'Cache-Control' header instructs caches how to handle the content. Varnish
 cares about the *max-age* parameter and uses it to calculate the TTL
 for an object. 
 
-"Cache-Control: nocache" is ignored but if you need this you can
+``Cache-Control: nocache`` is ignored but if you need this you can
 easily add support for it.
 
-So make sure you issue a Cache-Control header with a max-age
-header. You can have a look at what Varnish Software's drupal server
+So make sure you issue a 'Cache-Control' header with a max-age
+header. You can have a look at what Varnish Software's Drupal server
 issues::
 
   $ GET -Used http://www.varnish-software.com/|grep ^Cache-Control
@@ -232,18 +237,18 @@ issues::
 Age
 ~~~
 
-Varnish adds an Age header to indicate how long the object has been
-kept inside Varnish. You can grep out Age from varnishlog like this::
+Varnish adds an 'Age' header to indicate how long the object has been
+kept inside Varnish. You can grep out 'Age' from `varnishlog` like this::
 
   varnishlog -i TxHeader -I ^Age
 
 Pragma
 ~~~~~~
 
-An HTTP 1.0 server might send "Pragma: nocache". Varnish ignores this
+An HTTP 1.0 server might send the header ``Pragma: nocache``. Varnish ignores this
 header. You could easily add support for this header in VCL.
 
-In vcl_backend_response::
+In `vcl_backend_response`::
 
   if (beresp.http.Pragma ~ "nocache") {
         set beresp.uncacheable = true;
@@ -253,7 +258,7 @@ In vcl_backend_response::
 Authorization
 ~~~~~~~~~~~~~
 
-If Varnish sees an Authorization header it will pass the request. If
+If Varnish sees an 'Authorization' header it will pass the request. If
 this is not what you want you can unset the header.
 
 Overriding the time-to-live (ttl)
@@ -264,7 +269,7 @@ setup, be easier to override the ttl in Varnish than to fix your
 somewhat cumbersome backend. 
 
 You need VCL to identify the objects you want and then you set the
-beresp.ttl to whatever you want::
+'beresp.ttl' to whatever you want::
 
   sub vcl_backend_response {
       if (req.url ~ "^/legacy_broken_cms/") {
@@ -272,13 +277,13 @@ beresp.ttl to whatever you want::
       }
   }
 
-The example will set the TTL to 5 days for the old legacy stuff on
+This example will set the ttl to 5 days for the old legacy stuff on
 your site.
 
 Forcing caching for certain requests and certain responses
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-Since you still have this cumbersome backend that isn't very friendly
+Since you still might have this cumbersome backend that isn't very friendly
 to work with you might want to override more stuff in Varnish. We
 recommend that you rely as much as you can on the default caching
 rules. It is perfectly easy to force Varnish to lookup an object in
@@ -291,7 +296,8 @@ Normalizing your namespace
 Some sites are accessed via lots of
 hostnames. http://www.varnish-software.com/,
 http://varnish-software.com/ and http://varnishsoftware.com/ all point
-at the same site. Since Varnish doesn't know they are different,
+at the same site. Since Varnish doesn't know they are the same,
+.. XXX: heavy meaning change above. benc
 Varnish will cache different versions of every page for every
 hostname. You can mitigate this in your web server configuration by
 setting up redirects or by using the following VCL::
@@ -310,33 +316,33 @@ HTTP Vary
 misunderstood HTTP header.*
 
 A lot of the response headers tell the client something about the HTTP
-object being delivered. Clients can request different variants a an
+object being delivered. Clients can request different variants of a
 HTTP object, based on their preference. Their preferences might cover
 stuff like encoding or language. When a client prefers UK English this
-is indicated through "Accept-Language: en-uk". Caches need to keep
+is indicated through ``Accept-Language: en-uk``. Caches need to keep
 these different variants apart and this is done through the HTTP
-response header "Vary".
+response header 'Vary'.
 
-When a backend server issues a "Vary: Accept-Language" it tells
+When a backend server issues a ``Vary: Accept-Language`` it tells
 Varnish that its needs to cache a separate version for every different
 Accept-Language that is coming from the clients.
 
 If two clients say they accept the languages "en-us, en-uk" and "da,
 de" respectively, Varnish will cache and serve two different versions
 of the page if the backend indicated that Varnish needs to vary on the
-Accept-Language header.
+'Accept-Language' header.
 
-Please note that the headers that Vary refer to need to match
+Please note that the headers that 'Vary' refer to need to match
 *exactly* for there to be a match. So Varnish will keep two copies of
 a page if one of them was created for "en-us, en-uk" and the other for
-"en-us,en-uk". Just the lack of space will force Varnish to cache
+"en-us,en-uk". Just the lack of a whitespace will force Varnish to cache
 another version.
 
-To achieve a high hitrate whilst using Vary is there therefor crucial
+To achieve a high hitrate whilst using Vary is there therefore crucial
 to normalize the headers the backends varies on. Remember, just a
-difference in case can force different cache entries.
+difference in casing can force different cache entries.
 
-The following VCL code will normalize the Accept-Language headers, to
+The following VCL code will normalize the 'Accept-Language' headers, to
 one of either "en","de" or "fr"::
 
     if (req.http.Accept-Language) {
@@ -353,28 +359,28 @@ one of either "en","de" or "fr"::
         }
     }
 
-The code sets the Accept-Encoding header from the client to either
+The code sets the 'Accept-Encoding' header from the client to either
 gzip, deflate with a preference for gzip.
 
 Vary parse errors
 ~~~~~~~~~~~~~~~~~
 
-Varnish will return a 503 internal server error page when it fails to
-parse the Vary server header, or if any of the client headers listed
-in the Vary header exceeds the limit of 65k characters. An SLT_Error
+Varnish will return a "503 internal server error" page when it fails to
+parse the 'Vary' header, or if any of the client headers listed
+in the Vary header exceeds the limit of 65k characters. An 'SLT_Error'
 log entry is added in these cases.
 
 Pitfall - Vary: User-Agent
 ~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-Some applications or application servers send *Vary: User-Agent* along
+Some applications or application servers send ``Vary: User-Agent`` along
 with their content. This instructs Varnish to cache a separate copy
-for every variation of User-Agent there is. There are plenty. Even a
+for every variation of 'User-Agent' there is and there are plenty. Even a
 single patchlevel of the same browser will generate at least 10
-different User-Agent headers based just on what operating system they
+different 'User-Agent' headers based just on what operating system they
 are running. 
 
-So if you *really* need to Vary based on User-Agent be sure to
+So if you *really* need to vary based on 'User-Agent' be sure to
 normalize the header or your hit rate will suffer badly. Use the above
 code as a template.
 
diff --git a/doc/sphinx/users-guide/intro.rst b/doc/sphinx/users-guide/intro.rst
index 8b31b32..f9c5587 100644
--- a/doc/sphinx/users-guide/intro.rst
+++ b/doc/sphinx/users-guide/intro.rst
@@ -3,7 +3,7 @@
 The Big Varnish Picture
 =======================
 
-In this section we will cover the questions:
+In this section we will cover answers to the questions:
 - What is in this package called "Varnish"?
 - what are all the different bits and pieces named? 
 - Will you need a hex-wrench for assembly?
diff --git a/doc/sphinx/users-guide/operation-logging.rst b/doc/sphinx/users-guide/operation-logging.rst
index 547a987..4bfdca0 100644
--- a/doc/sphinx/users-guide/operation-logging.rst
+++ b/doc/sphinx/users-guide/operation-logging.rst
@@ -3,8 +3,8 @@
 Logging in Varnish
 ------------------
 
-One of the really nice features in Varnish is how logging
-works. Instead of logging to normal log file Varnish logs to a shared
+One of the really nice features in Varnish is the way logging
+works. Instead of logging to a normal log file Varnish logs to a shared
 memory segment, called the VSL - the Varnish Shared Log. When the end
 of the segment is reached we start over, overwriting old data. 
 
@@ -13,14 +13,14 @@ require disk space. Besides it gives you much, much more information
 when you need it.
 
 The flip side is that if you forget to have a program actually write the
-logs to disk they will disappear.
+logs to disk they will be overwritten.
 
-varnishlog is one of the programs you can use to look at what Varnish
-is logging. Varnishlog gives you the raw logs, everything that is
-written to the logs. There are other clients as well, we'll show you
+`varnishlog` is one of the programs you can use to look at what Varnish
+is logging. `varnishlog` gives you the raw logs, everything that is
+written to the logs. There are other clients that can access the logs as well, we'll show you
 these later.
 
-In the terminal window you started Varnish now type *varnishlog* and
+In the terminal window you started Varnish now type ``varnishlog`` and
 press enter.
 
 You'll see lines like these scrolling slowly by.::
@@ -32,7 +32,10 @@ These is the Varnish master process checking up on the caching process
 to see that everything is OK.
 
 Now go to the browser and reload the page displaying your web
-app. You'll see lines like these.::
+app. 
+.. XXX:Doesn't this require a setup of a running varnishd and a web application being cached? benc
+
+You'll see lines like these.::
 
    11 SessionOpen  c 127.0.0.1 58912 0.0.0.0:8080
    11 ReqStart     c 127.0.0.1 58912 595005213
@@ -42,29 +45,32 @@ app. You'll see lines like these.::
    11 RxHeader     c Host: localhost:8080
    11 RxHeader     c Connection: keep-alive
 
+
 The first column is an arbitrary number, it identifies the
 session. Lines with the same number are coming from the same session
 and are being handled by the same thread. The second column is the
 *tag* of the log message. All log entries are tagged with a tag
 indicating what sort of activity is being logged. Tags starting with
-Rx indicate Varnish is recieving data and Tx indicates sending data.
+'Rx' indicate Varnish is recieving data and 'Tx' indicates sending data.
 
 The third column tell us whether this is is data coming or going to
-the client (c) or to/from the backend (b). The forth column is the
+the client ('c') or to/from the backend ('b'). The forth column is the
 data being logged.
 
-Now, you can filter quite a bit with varnishlog. The basic option you
+Now, you can filter quite a bit with `varnishlog`. The basic options we think you
 want to know are:
 
--b
+'-b'
  Only show log lines from traffic going between Varnish and the backend
  servers. This will be useful when we want to optimize cache hit rates.
 
--c
- Same as -b but for client side traffic.
+'-c'
+ Same as '-b' but for client side traffic.
 
--m tag:regex
+'-m tag:regex'
  Only list transactions where the tag matches a regular expression. If
  it matches you will get the whole transaction.
 
+.. XXX:Maybe a couple of sample commands here? benc
+
 For more information on this topic please see :ref:`ref-varnishlog`.
diff --git a/doc/sphinx/users-guide/operation-statistics.rst b/doc/sphinx/users-guide/operation-statistics.rst
index c891342..788e600 100644
--- a/doc/sphinx/users-guide/operation-statistics.rst
+++ b/doc/sphinx/users-guide/operation-statistics.rst
@@ -4,13 +4,14 @@
 Statistics
 ----------
 
-Now that your Varnish is up and running let's have a look at how it is
-doing. There are several tools that can help.
+Varnish comes with a couple of nifty and very useful statistics generating tools that generates statistics in real time by constantly updating and presenting a specific dataset by aggregating and analyzing logdata from the shared memory logs.
+
+.. XXX:Heavy rewrite above. benc
 
 varnishtop
 ~~~~~~~~~~
 
-The varnishtop utility reads the shared memory logs and presents a
+The `varnishtop` utility reads the shared memory logs and presents a
 continuously updated list of the most commonly occurring log entries.
 
 With suitable filtering using the -I, -i, -X and -x options, it can be
@@ -28,9 +29,11 @@ For more information please see :ref:`ref-varnishtop`.
 varnishhist
 ~~~~~~~~~~~
 
-The varnishhist utility reads varnishd(1) shared memory logs and
+The `varnishhist` utility reads `varnishd(1)` shared memory logs and
 presents a continuously updated histogram showing the distribution of
-the last N requests by their processing.  The value of N and the
+the last N requests by their processing.  
+.. XXX:1? benc
+The value of N and the
 vertical scale are displayed in the top left corner.  The horizontal
 scale is logarithmic.  Hits are marked with a pipe character ("|"),
 and misses are marked with a hash character ("#").
@@ -43,10 +46,10 @@ varnishstat
 
 Varnish has lots of counters. We count misses, hits, information about
 the storage, threads created, deleted objects. Just about
-everything. varnishstat will dump these counters. This is useful when
+everything. `varnishstat` will dump these counters. This is useful when
 tuning Varnish.
 
-There are programs that can poll varnishstat regularly and make nice
+There are programs that can poll `varnishstat` regularly and make nice
 graphs of these counters. One such program is Munin. Munin can be
 found at http://munin-monitoring.org/ . There is a plugin for munin in
 the Varnish source code.
diff --git a/doc/sphinx/users-guide/params.rst b/doc/sphinx/users-guide/params.rst
index a283d51..df4bfdf 100644
--- a/doc/sphinx/users-guide/params.rst
+++ b/doc/sphinx/users-guide/params.rst
@@ -3,21 +3,21 @@
 Parameters
 ----------
 
-Varnish Cache has a set of parameters that affect its behaviour and
+Varnish Cache comes with a set of parameters that affects behaviour and
 performance. Most of these parameters can be set on the Varnish
-command line (through varnishadm) using the param.set keyword.
+command line (through `varnishadm`) using the ``param.set`` keyword.
 
-Some parameters can, for security purposes be read only using the "-r"
-command line switch to varnishd.
+Some parameters can, for security purposes be read only using the '-r'
+command line switch to `varnishd`.
 
-I don't recommend tweaking the parameters unless you're sure of what
+We don't recommend that you tweak parameters unless you're sure of what
 you're doing. We've worked hard to make the defaults sane and Varnish
 should be able to handle most workloads with the default settings.
 
 For a complete listing of all the parameters and a short descriptions
 type ``param.show`` in the CLI. To inspect a certain parameter and get
 a somewhat longer description on what it does and what the default is
-type param.show and the name of the parameter, like this::
+type ``param.show`` and the name of the parameter, like this::
 
   varnish> param.show shortlived
   200        
diff --git a/doc/sphinx/users-guide/performance.rst b/doc/sphinx/users-guide/performance.rst
index baf9c75..cd00ff5 100644
--- a/doc/sphinx/users-guide/performance.rst
+++ b/doc/sphinx/users-guide/performance.rst
@@ -3,19 +3,18 @@
 Varnish and Website Performance
 ===============================
 
-This section is about tuning the performance of your Varnish server,
-and about tuning the performance of your website using Varnish.
+This section focuses on how to tune the performance of your Varnish server,
+and how to tune the performance of your website using Varnish.
 
-The section is split in three sections. One deals with the various tools and
-functions of Varnish that you should be aware of and the other focuses
+The section is split in three subsections. The first subsection deals with the various tools and
+functions of Varnish that you should be aware of. The next subsection focuses
 on the how to purge content out of your cache. Purging of content is
 essential in a performance context because it allows you to extend the
 *time-to-live* (TTL) of your cached objects. Having a long TTL allows
-Varnish to keep the content in cache longer, meaning Varnish will make
-send fewer requests to your relativly slow backend.
+Varnish to keep the content in cache longer, meaning Varnish will make fewer requests to your relativly slower backend.
 
-The final section deals with compression of web content. Varnish can
-gzip content when fetching it from the backend and then deliver
+The final subsection deals with compression of web content. Varnish can
+gzip content when fetching it from the backend and then deliver it
 compressed. This will reduce the time it takes to download the content
 thereby increasing the performance of your website.
 
diff --git a/doc/sphinx/users-guide/purging.rst b/doc/sphinx/users-guide/purging.rst
index 058bd7d..2be32e3 100644
--- a/doc/sphinx/users-guide/purging.rst
+++ b/doc/sphinx/users-guide/purging.rst
@@ -6,24 +6,24 @@ Purging and banning
 
 One of the most effective ways of increasing your hit ratio is to
 increase the time-to-live (ttl) of your objects. But, as you're aware
-of, in this twitterific day of age serving content that is outdated is
+of, in this twitterific day of age, serving content that is outdated is
 bad for business.
 
 The solution is to notify Varnish when there is fresh content
 available. This can be done through three mechanisms. HTTP purging,
-banning and forced cache misses. First, let me explain the HTTP purges.
+banning and forced cache misses. First, lets look at HTTP purging.
 
 
-HTTP Purges
-~~~~~~~~~~~
+HTTP Purging
+~~~~~~~~~~~~
 
 A *purge* is what happens when you pick out an object from the cache
 and discard it along with its variants. Usually a purge is invoked
-through HTTP with the method PURGE.
+through HTTP with the method `PURGE`.
 
-An HTTP purge is similar to an HTTP GET request, except that the
-*method* is PURGE. Actually you can call the method whatever you'd
-like, but most people refer to this as purging. Squid supports the
+An HTTP purge is similar to a HTTP GET request, except that the
+*method* is `PURGE`. Actually you can call the method whatever you'd
+like, but most people refer to this as purging. Squid, for example, supports the
 same mechanism. In order to support purging in Varnish you need the
 following VCL in place::
 
@@ -64,7 +64,7 @@ Bans
 ~~~~
 
 There is another way to invalidate content: Bans. You can think of
-bans as a sort of a filter on objects already in the cache. You *ban*
+bans as a sort of a filter on objects already in the cache. You ``ban``
 certain content from being served from your cache. You can ban
 content based on any metadata we have.
 A ban will only work on objects already in the cache, it does not
@@ -81,16 +81,18 @@ Quite powerful, really.
 Bans are checked when we hit an object in the cache, but before we
 deliver it. *An object is only checked against newer bans*.
 
-Bans that only match against obj.* are also processed by a background
-worker threads called the *ban lurker*. The ban lurker will walk the
+Bans that only match against `obj.*` are also processed by a background
+worker threads called the `ban lurker`. The `ban lurker` will walk the
 heap and try to match objects and will evict the matching objects. How
-aggressive the ban lurker is can be controlled by the parameter
-ban_lurker_sleep. The ban lurker can be disabled by setting
-ban_lurker_sleep to 0.
+aggressive the `ban lurker` is can be controlled by the parameter
+'ban_lurker_sleep'. The `ban lurker` can be disabled by setting
+'ban_lurker_sleep' to 0.
+
+.. XXX: sample here? benc
 
 Bans that are older than the oldest objects in the cache are discarded
-without evaluation.  If you have a lot of objects with long TTL, that
-are seldom accessed you might accumulate a lot of bans. This might
+without evaluation. If you have a lot of objects with long TTL, that
+are seldom accessed, you might accumulate a lot of bans. This might
 impact CPU usage and thereby performance.
 
 You can also add bans to Varnish via HTTP. Doing so requires a bit of VCL::
@@ -110,14 +112,14 @@ You can also add bans to Varnish via HTTP. Doing so requires a bit of VCL::
 	  }
   }
 
-This VCL sniplet enables Varnish to handle an HTTP BAN method, adding a
+This VCL stanza enables Varnish to handle a `HTTP BAN` method, adding a
 ban on the URL, including the host part.
 
-The ban lurker can help you keep the ban list at a manageable size, so
-we recommend that you avoid using req.* in your bans, as the request
-object is not available in the ban lurker thread.
+The `ban lurker` can help you keep the ban list at a manageable size, so
+we recommend that you avoid using `req.*` in your bans, as the request
+object is not available in the `ban lurker` thread.
 
-You can use the following template to write ban lurker friendly bans::
+You can use the following template to write `ban lurker` friendly bans::
 
   sub vcl_backend_response {
     set beresp.http.x-url = req.url;
@@ -136,7 +138,7 @@ You can use the following template to write ban lurker friendly bans::
     }
   }
 
-To inspect the current ban list, issue the ban.list command in CLI. This
+To inspect the current ban list, issue the ``ban.list`` command in the CLI. This
 will produce a status of all current bans::
 
   0xb75096d0 1318329475.377475    10      obj.http.x-url ~ test
@@ -146,15 +148,15 @@ The ban list contains the ID of the ban, the timestamp when the ban
 entered the ban list. A count of the objects that has reached this point
 in the ban list, optionally postfixed with a 'G' for "Gone", if the ban
 is no longer valid.  Finally, the ban expression is listed. The ban can
-be marked as Gone if it is a duplicate ban, but is still kept in the list
+be marked as "Gone" if it is a duplicate ban, but is still kept in the list
 for optimization purposes.
 
 Forcing a cache miss
 ~~~~~~~~~~~~~~~~~~~~
 
 The final way to invalidate an object is a method that allows you to
-refresh an object by forcing a hash miss for a single request. If you set
-req.hash_always_miss to true, Varnish will miss the current object in the
+refresh an object by forcing a `hash miss` for a single request. If you set
+'req.hash_always_miss' to true, Varnish will miss the current object in the
 cache, thus forcing a fetch from the backend. This can in turn add the
 freshly fetched object to the cache, thus overriding the current one. The
 old object will stay in the cache until ttl expires or it is evicted by
diff --git a/doc/sphinx/users-guide/report.rst b/doc/sphinx/users-guide/report.rst
index 5a444ab..13daf3e 100644
--- a/doc/sphinx/users-guide/report.rst
+++ b/doc/sphinx/users-guide/report.rst
@@ -3,7 +3,7 @@
 Reporting and statistics
 ========================
 
-This section is about how to find out what Varnish is doing, from
+This section covers how to find out what Varnish is doing, from
 the detailed per HTTP request blow-by-blow logrecords to the global
 summary statistics counters.
 
diff --git a/doc/sphinx/users-guide/run_cli.rst b/doc/sphinx/users-guide/run_cli.rst
index b534971..3e75ac5 100644
--- a/doc/sphinx/users-guide/run_cli.rst
+++ b/doc/sphinx/users-guide/run_cli.rst
@@ -3,46 +3,46 @@
 CLI - bossing Varnish around
 ============================
 
-Once varnishd is started, you can control it using the command line
+Once `varnishd` is started, you can control it using the command line
 interface.
 
-The easiest way to do this, is using the varnishadm program on the
-same machine as varnishd is running::
+The easiest way to do this, is using `varnishadm` on the
+same machine as `varnishd` is running::
 
 	varnishadm help
 
-If you want to run varnishadm from a remote system, you can do it
+If you want to run `varnishadm` from a remote system, you can do it
 two ways.
 
-You can SSH into the varnishd computer and run varnishadm::
+You can SSH into the `varnishd` computer and run `varnishadm`::
 
 	ssh $http_front_end varnishadm help
 
-But you can also configure varnishd to accept remote CLI connections
-(using the -T and -S arguments)::
+But you can also configure `varnishd` to accept remote CLI connections
+(using the '-T' and '-S' arguments)::
 
 	varnishd -T :6082 -S /etc/varnish_secret
 
-And then on the remote system run varnishadm::
+And then on the remote system run `varnishadm`::
 
 	varnishadm -T $http_front_end -S /etc/copy_of_varnish_secret help
 
 but as you can see, SSH is much more convenient.
 
-If you run varnishadm without arguments, it will read CLI commands from
-stdin, if you give it arguments, it will treat those as the single
+If you run `varnishadm` without arguments, it will read CLI commands from
+`stdin`, if you give it arguments, it will treat those as the single
 CLI command to execute.
 
 The CLI always returns a status code to tell how it went:  '200'
 means OK, anything else means there were some kind of trouble.
 
-varnishadm will exit with status 1 and print the status code on
+`varnishadm` will exit with status 1 and print the status code on
 standard error if it is not 200.
 
 What can you do with the CLI
 ----------------------------
 
-The CLI gives you almost total control over varnishd:
+The CLI gives you almost total control over `varnishd` some of the more important tasks you can perform are:
 
 * load/use/discard VCL programs
 * ban (invalidate) cache content
@@ -64,7 +64,7 @@ To load new VCL program::
 
 	varnish> vcl.load some_name some_filename
 
-Loading will read the VCL program from the file, and compile it.  If
+Loading will read the VCL program from the file, and compile it. If
 the compilation fails, you will get an error messages::
 
 	.../mask is not numeric.
@@ -93,6 +93,8 @@ It is good idea to design an emergency-VCL before you need it,
 and always have it loaded, so you can switch to it with a single
 vcl.use command.
 
+.. XXX:Should above have a clearer admonition like a NOTE:? benc
+
 Ban cache content
 ^^^^^^^^^^^^^^^^^
 
@@ -103,7 +105,7 @@ But sometimes it is useful to be able to throw things out of cache
 without having an exact list of what to throw out.
 
 Imagine for instance that the company logo changed and now you need
-to get all versions of it out of the cache::
+Varnish to stop serving the old logo out of the cache::
 
 	varnish> ban req.url ~ "logo.*[.]png"
 
@@ -119,7 +121,7 @@ a HTTP request asks for it.
 Banning stuff is much cheaper than restarting Varnish to get rid
 of wronly cached content.
 
-.. In addition to handling such special occations, banning can be used
+.. In addition to handling such special occasions, banning can be used
 .. in many creative ways to keep the cache up to date, more about
 .. that in: (TODO: xref)
 
@@ -130,7 +132,7 @@ Change parameters
 Parameters can be set on the command line with the '-p' argument,
 but they can also be examined and changed on the fly from the CLI::
 
-	varnish> param.show perfer_ipv6
+	varnish> param.show prefer_ipv6
 	200
 	prefer_ipv6         off [bool]
                             Default is off
@@ -144,8 +146,12 @@ In general it is not a good idea to modify parameters unless you
 have a good reason, such as performance tuning or security configuration.
 
 Most parameters will take effect instantly, or with a natural delay
-of some duration, but a few of them requires you to restart the
-child process before they take effect.  This is always noted in the
+of some duration,
+
+.. XXX: Natural delay of some duration sounds vague. benc
+
+but a few of them requires you to restart the
+child process before they take effect. This is always noted in the
 description of the parameter.
 
 Starting and stopping the worker process
@@ -160,7 +166,7 @@ and::
 
 	varnish> start
 
-If you start varnishd with the '-d' (debugging) argument, you will
+If you start `varnishd` with the '-d' (debugging) argument, you will
 always need to start the child process explicitly.
 
 Should the child process die, the master process will automatically
diff --git a/doc/sphinx/users-guide/run_security.rst b/doc/sphinx/users-guide/run_security.rst
index dfcfc6d..d7340a0 100644
--- a/doc/sphinx/users-guide/run_security.rst
+++ b/doc/sphinx/users-guide/run_security.rst
@@ -5,8 +5,8 @@ Security first
 
 If you are the only person involved in running Varnish, or if all
 the people involved are trusted to the same degree, you can skip
-this chapter:  We have protected Varnish as well as we can from
-anything which can come in through HTTP socket.
+this chapter. We have protected Varnish as well as we can from
+anything which can come in through a HTTP socket.
 
 If parts of your web infrastructure are outsourced or otherwise
 partitioned along administrative lines, you need to think about
@@ -15,19 +15,18 @@ security.
 Varnish provides four levels of authority, roughly related to
 how and where the command comes into Varnish:
 
-  * The command line arguments
+  * the command line arguments,
 
-  * The CLI interface
+  * the CLI interface,
 
-  * VCL programs
+  * VCL programs, and
 
-  * HTTP requests
+  * HTTP requests.
 
 Command line arguments
 ----------------------
 
-The top level security decisions is taken on and from the command
-line, in order to make them invulnerable to subsequent manipulation.
+The top level security decisions is decided and defined when starting Varnish in the form of command line arguments, we use this strategy in order to make them invulnerable to subsequent manipulation.
 
 The important decisions to make are:
 
@@ -44,8 +43,8 @@ CLI interface access
 
 The command line interface can be accessed three ways.
 
-Varnishd can be told til listen and offer CLI connections
-on a TCP socket.  You can bind the socket to pretty
+`Varnishd` can be told til listen and offer CLI connections
+on a TCP socket. You can bind the socket to pretty
 much anything the kernel will accept::
 
 	-T 127.0.0.1:631
@@ -53,15 +52,17 @@ much anything the kernel will accept::
 	-T 192.168.1.1:34
 	-T '[fe80::1]:8082'
 
-The default is '-T localhost:0' which will pick a random
-port number, which varnishadm(8) can learn in the shared
+The default is ``-T localhost:0`` which will pick a random
+port number, which `varnishadm(8)` can learn in the shared
 memory.
 
+.. XXX:Me no understand sentence above, (8)? and learn in the shared memory? Stored and retrieved by varnishadm from th e shared memory? benc 
+
 By using a "localhost" address, you restrict CLI access
 to the local machine.
 
 You can also bind the CLI port to an IP number reachable across
-the net, and let other computers connect directly.
+the net, and let other machines connect directly.
 
 This gives you no secrecy, ie, the CLI commands will
 go across the network as ASCII text with no encryption, but
@@ -72,42 +73,52 @@ Alternatively you can bind the CLI port to a 'localhost' address,
 and give remote users access via a secure connection to the local
 machine, using ssh/VPN or similar.
 
-If you use ssh you can restrict which commands each user can execute to
-just varnishadm, or even to wrapper scripts around varnishadm, which
+If you use `ssh` you can restrict which commands each user can execute to
+just `varnishadm`, or even to wrapper scripts around `varnishadm`, which
 only allow specific CLI commands.
 
-It is also possible to configure varnishd for "reverse mode", using
-the '-M' argument.  In that case varnishd will attempt to open a
+It is also possible to configure `varnishd` for "reverse mode", using
+the '-M' argument.  In that case `varnishd` will attempt to open a
 TCP connection to the specified address, and initiate a CLI connection
 to your central Varnish management facility.
 
+.. XXX:Maybe a sample command here with a brief explanation? benc
+
 The connection is also in this case without secrecy, but
 the remote end must still satisfy -S/PSK authentication.
 
+.. XXX:Without encryption instead of secrecy? benc
+
 Finally, if you run varnishd with the '-d' option, you get a CLI
 command on stdin/stdout, but since you started the process, it
 would be hard to prevent you getting CLI access, wouldn't it ?
 
+
 CLI interface authentication
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-By default the CLI interface is protected with a simple,  yet
+By default the CLI interface is protected with a simple, yet
 strong "Pre Shared Key" authentication method, which do not provide
 secrecy (ie: The CLI commands and responses are not encrypted).
 
+.. XXX:Encryption instead of secrecy? benc
+
 The way -S/PSK works is really simple:  During startup a file is
 created with a random content and the file is only accessible to
-the user who started varnishd (or the superuser).
+the user who started `varnishd` (or the superuser).
 
 To authenticate and use a CLI connection, you need to know the
 contents of that file, in order to answer the cryptographic
-challenge varnishd issues. (XXX: xref to algo in refman)
+challenge `varnishd` issues. 
+
 
-The varnishadm program knows all about this, it will just work,
+(XXX: xref to algo in refman)
+.. XXX:Dunno what this is? benc
+
+`varnishadm` uses all of this to restrict access, it will only function,
 provided it can read the secret file.
 
-If you want to allow other users on the local system or remote
-users, to be able to access CLI connections, you must create your
+If you want to allow other users, local or remote, to be able to access CLI connections, you must create your
 own secret file and make it possible for (only!) these users to
 read it.
 
@@ -115,18 +126,18 @@ A good way to create the secret file is::
 
 	dd if=/dev/random of=/etc/varnish_secret count=1
 
-When you start varnishd, you specify the filename with -S, and
-it goes without saying that the varnishd master process needs
+When you start `varnishd`, you specify the filename with '-S', and
+it goes without saying that the `varnishd` master process needs
 to be able to read the file too.
 
-You can change the contents of the secret file while varnishd
+You can change the contents of the secret file while `varnishd`
 runs, it is read every time a CLI connection is authenticated.
 
-On the local system, varnishadm can find the filename from
-shared memory, but on remote systems, you need to give it
+On the local system, `varnishadm` can retrieve the filename from
+shared memory, but on remote systems, you need to give `varnishadm`
 a copy of the secret file, with the -S argument.
 
-If you want to disable -S/PSK authentication, specify -S with
+If you want to disable -S/PSK authentication, specify '-S' with
 an empty argument to varnishd::
 
 	varnishd [...] -S "" [...]
@@ -135,17 +146,17 @@ Parameters
 ^^^^^^^^^^
 
 Parameters can be set from the command line, and made "read-only"
-(using -r) so they cannot subsequently be modified from the CLI
+(using '-r') so they cannot subsequently be modified from the CLI
 interface.
 
 Pretty much any parameter can be used to totally mess up your
-HTTP service, but a few can do more damage than that:
+HTTP service, but a few can do more damage than others:
 
 :ref:`ref_param_user` and :ref:`ref_param_group`
 	Access to local system via VCL
 
 :ref:`ref_param_listen_address`
-	Trojan other TCP sockets, like ssh
+	Trojan other TCP sockets, like `ssh`
 
 :ref:`ref_param_cc_command`
 	Execute arbitrary programs
@@ -156,10 +167,12 @@ HTTP service, but a few can do more damage than that:
 Furthermore you may want to look at and lock down:
 
 :ref:`ref_param_syslog_cli_traffic`
-	Log all CLI commands to syslog(8), so you know what goes on.
+	Log all CLI commands to `syslog(8)`, so you know what goes on.
+.. XXX: syslog(8)? benc
+
 
 :ref:`ref_param_vcc_unsafe_path`
-	Retrict VCL/VMODS to :ref:`ref_param_vcl_dir` and :ref:`ref_param_vmod_dir`
+	Restrict VCL/VMODS to :ref:`ref_param_vcl_dir` and :ref:`ref_param_vmod_dir`
 
 :ref:`ref_param_vmod_dir` 
         The directory where Varnish will will look
@@ -178,11 +191,16 @@ certain parameters, but that will only protect the local filesystem,
 and operating system, it will not protect your HTTP service.
 
 We do not currently have a way to restrict specific CLI commands
-to specific CLI connections.   One way to get such an effect is to
-"wrap" all CLI access in pre-approved scripts which use varnishadm(1)
+to specific CLI connections. One way to get such an effect is to
+"wrap" all CLI access in pre-approved scripts which use `varnishadm(1)`
+
+.. XXX:what does the 1 stand for? benc
+
 to submit the sanitized CLI commands, and restrict a remote user
 to only those scripts, for instance using sshd(8)'s configuration.
 
+.. XXX:what does the 8 stand for? benc
+
 VCL programs
 ------------
 
@@ -190,35 +208,35 @@ There are two "dangerous" mechanisms available in VCL code:  VMODs
 and inline-C.
 
 Both of these mechanisms allow execution of arbitrary code and will
-therefore allow a person to get access on the computer, with the
+thus allow a person to get access to the machine, with the
 privileges of the child process.
 
-If varnishd is started as root/superuser, we sandbox the child
+If `varnishd` is started as root/superuser, we sandbox the child
 process, using whatever facilities are available on the operating
-system, but if varnishd is not started as root/superuser, this is
-not possible.  No, don't ask me why you have to be superuser to
+system, but if `varnishd` is not started as root/superuser, this is
+not possible. No, don't ask me why you have to be superuser to
 lower the privilege of a child process...
 
-Inline-C is disabled by default starting with Varnish 4, so unless
+Inline-C is disabled by default starting with Varnish version 4, so unless
 you enable it, you don't have to worry about it.
 
-The parameters mentioned above can restrict VMOD, so they can only
-be imported from a designated directory, restricting VCL wranglers
+The parameters mentioned above can restrict the loading of VMODs to only 
+be loaded from a designated directory, restricting VCL wranglers
 to a pre-approved subset of VMODs.
 
-If you do that, we believe that your local system cannot be compromised
+If you do that, we are confident that your local system cannot be compromised
 from VCL code.
 
 HTTP requests
 -------------
 
 We have gone to great lengths to make Varnish resistant to anything
-coming in throught he socket where HTTP requests are received, and
+coming in throught the socket where HTTP requests are received, and
 you should, generally speaking, not need to protect it any further.
 
 The caveat is that since VCL is a programming language which lets you
-decide exactly what to do about HTTP requests, you can also decide
-to do exactly stupid things to them, including opening youself up
+decide exactly what to do with HTTP requests, you can also decide
+to do stupid and potentially dangerous things with them, including opening youself up
 to various kinds of attacks and subversive activities.
 
 If you have "administrative" HTTP requests, for instance PURGE
diff --git a/doc/sphinx/users-guide/running.rst b/doc/sphinx/users-guide/running.rst
index 2b49147..5cfd539 100644
--- a/doc/sphinx/users-guide/running.rst
+++ b/doc/sphinx/users-guide/running.rst
@@ -3,8 +3,8 @@
 Starting and running Varnish
 ============================
 
-This section is about starting, running, and stopping Varnish, about
-command line flags and options, communicating with the running
+This section covers starting, running, and stopping Varnish,
+command line flags and options, and communicating with the running
 Varnish processes, configuring storage and sockets and, and about
 securing and protecting Varnish against attacks.
 
diff --git a/doc/sphinx/users-guide/sizing-your-cache.rst b/doc/sphinx/users-guide/sizing-your-cache.rst
index 8f2dfba..497ce0e 100644
--- a/doc/sphinx/users-guide/sizing-your-cache.rst
+++ b/doc/sphinx/users-guide/sizing-your-cache.rst
@@ -2,7 +2,7 @@
 Sizing your cache
 -----------------
 
-Picking how much memory you should give Varnish can be a tricky
+Deciding on cache size can be a tricky
 task. A few things to consider:
 
  * How big is your *hot* data set. For a portal or news site that
@@ -12,14 +12,16 @@ task. A few things to consider:
    to only cache images a little while or not to cache them at all if
    they are cheap to serve from the backend and you have a limited
    amount of memory.
- * Watch the n_lru_nuked counter with :ref:`reference-varnishstat` or
+ * Watch the `n_lru_nuked` counter with :ref:`reference-varnishstat` or
    some other tool. If you have a lot of LRU activity then your cache
    is evicting objects due to space constraints and you should
    consider increasing the size of the cache.
 
 Be aware that every object that is stored also carries overhead that
-is kept outside the actually storage area. So, even if you specify -s
-malloc,16G Varnish might actually use **double** that. Varnish has a
+is kept outside the actually storage area. So, even if you specify '-s
+malloc,16G' Varnish might actually use **double** that. Varnish has a
 overhead of about 1k per object. So, if you have lots of small objects
 in your cache the overhead might be significant.
 
+.. XXX:This seems to contradict the last paragraph in "storage-backends". benc
+
diff --git a/doc/sphinx/users-guide/storage-backends.rst b/doc/sphinx/users-guide/storage-backends.rst
index b12dba0..6664d8d 100644
--- a/doc/sphinx/users-guide/storage-backends.rst
+++ b/doc/sphinx/users-guide/storage-backends.rst
@@ -8,9 +8,9 @@ Intro
 ~~~~~
 
 Varnish has pluggable storage backends. It can store data in various
-backends which have different performance characteristics. The default
+backends which can have different performance characteristics. The default
 configuration is to use the malloc backend with a limited size. For a
-serious Varnish deployment you probably need to adjust the storage
+serious Varnish deployment you probably would want to adjust the storage
 settings.
 
 malloc
@@ -21,11 +21,13 @@ syntax: malloc[,size]
 Malloc is a memory based backend. Each object will be allocated from
 memory. If your system runs low on memory swap will be used.
 
-Be aware that the size limitation only limits the actual storage and that
-approximately 1k of memory per object will be used for various internal
-structures.
+Be aware that the size limitation only limits the actual storage and that the
+approximately 1k of memory per object, used for various internal
+structures, is included in the actual storage as well.
 
-The size parameter specifies the maximum amount of memory varnishd
+.. XXX:This seems to contradict the last paragraph in "sizing-your-cache". benc
+
+The size parameter specifies the maximum amount of memory `varnishd`
 will allocate.  The size is assumed to be in bytes, unless followed by
 one of the following suffixes:
 
@@ -39,9 +41,9 @@ one of the following suffixes:
 
 The default size is unlimited.
 
-malloc's performance is bound by memory speed so it is very fast. If
-the dataset is bigger than what can fit in memory performance will
-depend on the operating system and how well it does paging.
+malloc's performance is bound to memory speed so it is very fast. If
+the dataset is bigger than available memory performance will
+depend on the operating systems ability to page effectively.
 
 file
 ~~~~
@@ -49,13 +51,13 @@ file
 syntax: file[,path[,size[,granularity]]]
 
 The file backend stores objects in memory backed by an unlinked file on disk
-with mmap.
+with `mmap`.
 
-The path parameter specifies either the path to the backing file or
-the path to a directory in which varnishd will create the backing
-file.  The default is /tmp.
+The 'path' parameter specifies either the path to the backing file or
+the path to a directory in which `varnishd` will create the backing
+file. The default is `/tmp`.
 
-The size parameter specifies the size of the backing file.  The size
+The size parameter specifies the size of the backing file. The size
 is assumed to be in bytes, unless followed by one of the following
 suffixes:
 
@@ -75,20 +77,22 @@ The default size is to use 50% of the space available on the device.
 If the backing file already exists, it will be truncated or expanded
 to the specified size.
 
-Note that if varnishd has to create or expand the file, it will not
+Note that if `varnishd` has to create or expand the file, it will not
 pre-allocate the added space, leading to fragmentation, which may
 adversely impact performance on rotating hard drives.  Pre-creating
-the storage file using dd(1) will reduce fragmentation to a minimum.
+the storage file using `dd(1)` will reduce fragmentation to a minimum.
+
+.. XXX:1? benc
 
-The granularity parameter specifies the granularity of
-allocation.  All allocations are rounded up to this size.  The
-is assumed to be in bytes, unless followed by one of the
+The 'granularity' parameter specifies the granularity of
+allocation. All allocations are rounded up to this size. The granularity is
+is assumed to be expressed in bytes, unless followed by one of the
 suffixes described for size except for %.
 
-The default granularity is the VM page size.  The size should be reduced if you
+The default granularity is the VM page size. The size should be reduced if you
 have many small objects.
 
-File performance is typically limited by the write speed of the
+File performance is typically limited to the write speed of the
 device, and depending on use, the seek time.
 
 persistent (experimental)
@@ -100,11 +104,11 @@ Persistent storage. Varnish will store objects in a file in a manner
 that will secure the survival of *most* of the objects in the event of
 a planned or unplanned shutdown of Varnish.
 
-The path parameter specifies the path to the backing file. If
+The 'path' parameter specifies the path to the backing file. If
 the file doesn't exist Varnish will create it.
 
-The size parameter specifies the size of the backing file.  The
-size is assumed to be in bytes, unless followed by one of the
+The 'size' parameter specifies the size of the backing file. The
+size is expressed in bytes, unless followed by one of the
 following suffixes:
 
       K, k    The size is expressed in kibibytes.
@@ -122,9 +126,9 @@ starts after a shutdown it will discard the content of any silo that
 isn't sealed.
 
 Note that taking persistent silos offline and at the same time using
-bans can cause problems. This because bans added while the silo was
-offline will not be applied to the silo when it reenters the cache,
-and can make previously banned objects reappear.
+bans can cause problems. This is due to the fact that bans added while the silo was
+offline will not be applied to the silo when it reenters the cache. Consequently enabling
+previously banned objects to reappear.
 
 Transient Storage
 -----------------
@@ -133,6 +137,10 @@ If you name any of your storage backend "Transient" it will be
 used for transient (short lived) objects. By default Varnish
 would use an unlimited malloc backend for this.
 
+.. XXX: Is this another paramater? In that case handled in the same manner as above? benc
+
 Varnish will consider an object short lived if the TTL is below the
-parameter "shortlived".
+parameter 'shortlived'.
+
 
+.. XXX: I am generally missing samples of setting all of these parameters, maybe one sample per section or a couple of examples here with a brief explanation to also work as a summary? benc
diff --git a/doc/sphinx/users-guide/troubleshooting.rst b/doc/sphinx/users-guide/troubleshooting.rst
index fa01890..9d57744 100644
--- a/doc/sphinx/users-guide/troubleshooting.rst
+++ b/doc/sphinx/users-guide/troubleshooting.rst
@@ -3,9 +3,9 @@
 Troubleshooting Varnish
 =======================
 
-Sometimes Varnish misbehaves. In order for you to understand whats
-going on there are a couple of places you can check. varnishlog,
-/var/log/syslog, /var/log/messages are all places where Varnish might
+Sometimes Varnish misbehaves or rather behaves the way you told it to behave but not necessarily the way you want it to behave. In order for you to understand whats
+going on there are a couple of places you can check. `varnishlog`,
+`/var/log/syslog`, `/var/log/messages` are all good places where Varnish might
 leave clues of whats going on. This section will guide you through
 basic troubleshooting in Varnish.
 
@@ -13,9 +13,9 @@ basic troubleshooting in Varnish.
 When Varnish won't start
 ------------------------
 
-Sometimes Varnish wont start. There is a plethora of reasons why
+Sometimes Varnish wont start. There is a plethora of possible reasons why
 Varnish wont start on your machine. We've seen everything from wrong
-permissions on /dev/null to other processes blocking the ports.
+permissions on `/dev/null` to other processes blocking the ports.
 
 Starting Varnish in debug mode to see what is going on.
 
@@ -23,8 +23,8 @@ Try to start Varnish by::
 
     # varnishd -f /usr/local/etc/varnish/default.vcl -s malloc,1G -T 127.0.0.1: 2000  -a 0.0.0.0:8080 -d
 
-Notice the -d option. It will give you some more information on what
-is going on. Let us see how Varnish will react to something else
+Notice the '-d' parameter. It will give you some more information on what
+is going on. Let us see how Varnish will react when something else is
 listening on its port.::
 
     # varnishd -n foo -f /usr/local/etc/varnish/default.vcl -s malloc,1G -T 127.0.0.1:2000  -a 0.0.0.0:8080 -d
@@ -39,7 +39,7 @@ listening on its port.::
     Type 'quit' to close CLI session.
     Type 'start' to launch worker process.
 
-Now Varnish is running. Only the master process is running, in debug
+Now Varnish is running but only the master process is running, in debug
 mode the cache does not start. Now you're on the console. You can
 instruct the master process to start the cache by issuing "start".::
 
@@ -49,7 +49,7 @@ instruct the master process to start the cache by issuing "start".::
 	 Could not open sockets
 
 And here we have our problem. Something else is bound to the HTTP port
-of Varnish. If this doesn't help try strace or truss or come find us
+of Varnish. If this doesn't help try ``strace`` or ``truss`` or come find us
 on IRC.
 
 
@@ -57,34 +57,33 @@ Varnish is crashing - panics
 ----------------------------
 
 When Varnish goes bust the child processes crashes. Most of the
-crashes are caught by one of the many consistency checks spread around
-the Varnish source code. When Varnish hits one of these the caching
-process it will crash itself in a controlled manner, leaving a nice
+crashes are caught by one of the many consistency checks we have included in the Varnish source code. When Varnish hits one of these the caching
+process will crash itself in a controlled manner, leaving a nice
 stack trace with the mother process.
 
-You can inspect any panic messages by typing panic.show in the CLI.
-
-| panic.show
-| Last panic at: Tue, 15 Mar 2011 13:09:05 GMT
-| Assert error in ESI_Deliver(), cache_esi_deliver.c line 354:
-|   Condition(i == Z_OK || i == Z_STREAM_END) not true.
-| thread = (cache-worker)
-| ident = Linux,2.6.32-28-generic,x86_64,-sfile,-smalloc,-hcritbit,epoll
-| Backtrace:
-|   0x42cbe8: pan_ic+b8
-|   0x41f778: ESI_Deliver+438
-|   0x42f838: RES_WriteObj+248
-|   0x416a70: cnt_deliver+230
-|   0x4178fd: CNT_Session+31d
-|   (..)
+You can inspect any panic messages by typing ``panic.show`` in the CLI.::
+
+ panic.show
+ Last panic at: Tue, 15 Mar 2011 13:09:05 GMT
+ Assert error in ESI_Deliver(), cache_esi_deliver.c line 354:
+   Condition(i == Z_OK || i == Z_STREAM_END) not true.
+ thread = (cache-worker)
+ ident = Linux,2.6.32-28-generic,x86_64,-sfile,-smalloc,-hcritbit,epoll
+ Backtrace:
+   0x42cbe8: pan_ic+b8
+   0x41f778: ESI_Deliver+438
+   0x42f838: RES_WriteObj+248
+   0x416a70: cnt_deliver+230
+   0x4178fd: CNT_Session+31d
+   (..)
 
 The crash might be due to misconfiguration or a bug. If you suspect it
-is a bug you can use the output in a bug report.
+is a bug you can use the output in a bug report, see the "Trouble Tickets" section in the Introduction chapter above.
 
 Varnish is crashing - segfaults
 -------------------------------
 
-Sometimes the bug escapes the consistency checks and Varnish get hit
+Sometimes a bug escapes the consistency checks and Varnish gets hit
 with a segmentation error. When this happens with the child process it
 is logged, the core is dumped and the child process starts up again.
 
@@ -93,29 +92,31 @@ debug a segfault the developers need you to provide a fair bit of
 data.
 
  * Make sure you have Varnish installed with symbols
+.. XXX:Symbols? benc
  * Make sure core dumps are enabled (ulimit)
+.. XXX:ulimit? benc
 
-Once you have the core you open it with gdb and issue the command "bt"
+Once you have the core you open it with `gdb` and issue the command ``bt``
 to get a stack trace of the thread that caused the segfault.
 
 
 Varnish gives me Guru meditation
 --------------------------------
 
-First find the relevant log entries in varnishlog. That will probably
-give you a clue. Since varnishlog logs so much data it might be hard
-to track the entries down. You can set varnishlog to log all your 503
+First find the relevant log entries in `varnishlog`. That will probably
+give you a clue. Since `varnishlog` logs a lot of data it might be hard
+to track the entries down. You can set `varnishlog` to log all your 503
 errors by issuing the following command::
 
    $ varnishlog -c -m TxStatus:503
 
 If the error happened just a short time ago the transaction might still
-be in the shared memory log segment. To get varnishlog to process the
-whole shared memory log just add the -d option::
+be in the shared memory log segment. To get `varnishlog` to process the
+whole shared memory log just add the '-d' parameter::
 
    $ varnishlog -d -c -m TxStatus:503
 
-Please see the varnishlog man page for elaborations on further
+Please see the `varnishlog` man page for elaborations on further
 filtering capabilities and explanation of the various options.
 
 
diff --git a/doc/sphinx/users-guide/vcl-actions.rst b/doc/sphinx/users-guide/vcl-actions.rst
index 691baa4..0775afe 100644
--- a/doc/sphinx/users-guide/vcl-actions.rst
+++ b/doc/sphinx/users-guide/vcl-actions.rst
@@ -5,18 +5,22 @@ actions
 
 The most common actions to return are these:
 
+.. XXX:Maybe a bit more explanation here what is an action and how it is returned? benc
+
 *pass*
  When you return pass the request and subsequent response will be passed to
- and from the backend server. It won't be cached. pass can be returned from
- vcl_recv
+ and from the backend server. It won't be cached. `pass` can be returned from
+ `vcl_recv`
 
 *lookup*
-  When you return lookup from vcl_recv you tell Varnish to deliver content 
+  When you return lookup from `vcl_recv` you tell Varnish to deliver content 
   from cache even if the request othervise indicates that the request 
   should be passed. 
 
 *pipe*
-  Pipe can be returned from vcl_recv as well. Pipe short circuits the
+.. XXX:What is pipe? benc
+
+  Pipe can be returned from `vcl_recv` as well. Pipe short circuits the
   client and the backend connections and Varnish will just sit there
   and shuffle bytes back and forth. Varnish will not look at the data being 
   send back and forth - so your logs will be incomplete. 
@@ -25,13 +29,13 @@ The most common actions to return are these:
   header before actually returning pipe. 
 
 *deliver*
- Deliver the object to the client.  Usually returned from vcl_backend_response. 
+ Deliver the object to the client. Usually returned from `vcl_backend_response`. 
 
 *restart*
  Restart processing of the request. You can restart the processing of
- the whole transaction. Changes to the req object are retained.
+ the whole transaction. Changes to the `req` object are retained.
 
 *retry*
  Retry the request against the backend. This can be called from
- vcl_backend_response or vcl_backend_error if you don't like the response 
+ `vcl_backend_response` or `vcl_backend_error` if you don't like the response 
  that the backend delivered.
diff --git a/doc/sphinx/users-guide/vcl-backends.rst b/doc/sphinx/users-guide/vcl-backends.rst
index 099e2a6..c54d951 100644
--- a/doc/sphinx/users-guide/vcl-backends.rst
+++ b/doc/sphinx/users-guide/vcl-backends.rst
@@ -6,7 +6,7 @@ Backend servers
 Varnish has a concept of "backend" or "origin" servers. A backend
 server is the server providing the content Varnish will accelerate.
 
-Our first task is to tell Varnish where it can find its content. Start
+Our first task is to tell Varnish where it can find its backends. Start
 your favorite text editor and open the relevant VCL file.
 
 Somewhere in the top there will be a section that looks a bit like this.::
@@ -16,7 +16,7 @@ Somewhere in the top there will be a section that looks a bit like this.::
     #     .port = "8080";
     # }
 
-We comment in this bit of text making the text look like.::
+We remove the comment markings in this text stanza making the it look like.::
 
     backend default {
         .host = "127.0.0.1";
@@ -27,7 +27,7 @@ Now, this piece of configuration defines a backend in Varnish called
 *default*. When Varnish needs to get content from this backend it will
 connect to port 8080 on localhost (127.0.0.1).
 
-Varnish can have several backends defined and can you can even join
+Varnish can have several backends defined you can even join
 several backends together into clusters of backends for load balancing
 purposes.
 
@@ -41,10 +41,10 @@ host or not. There are lot of options.
 
 Lets say we need to introduce a Java application into out PHP web
 site. Lets say our Java application should handle URL beginning with
-/java/.
+`/java/`.
 
 We manage to get the thing up and running on port 8000. Now, lets have
-a look at the default.vcl.::
+a look at the `default.vcl`.::
 
     backend default {
         .host = "127.0.0.1";
@@ -58,7 +58,7 @@ We add a new backend.::
         .port = "8000";
     }
 
-Now we need tell where to send the difference URL. Lets look at vcl_recv.::
+Now we need tell Varnish where to send the difference URL. Lets look at `vcl_recv`.::
 
     sub vcl_recv {
         if (req.url ~ "^/java/") {
@@ -71,18 +71,18 @@ Now we need tell where to send the difference URL. Lets look at vcl_recv.::
 It's quite simple, really. Lets stop and think about this for a
 moment. As you can see you can define how you choose backends based on
 really arbitrary data. You want to send mobile devices to a different
-backend? No problem. if (req.User-agent ~ /mobile/) .. should do the
+backend? No problem. ``if (req.User-agent ~ /mobile/) ..`` should do the
 trick.
 
 
 Backends and virtual hosts in Varnish
 -------------------------------------
 
-Varnish fully supports virtual hosts. They might work in a somewhat
+Varnish fully supports virtual hosts. They might however work in a somewhat
 counter intuitive fashion since they are never declared
 explicitly. You set up the routing of incoming HTTP requests in
-vcl_recv. If you want this routing to be done on the basis of virtual
-hosts you just need to inspect req.http.host.
+`vcl_recv`. If you want this routing to be done on the basis of virtual
+hosts you just need to inspect `req.http.host`.
 
 You can have something like this:::
 
@@ -94,10 +94,10 @@ You can have something like this:::
         }
     }
 
-Note that the first regular expressions will match foo.com,
-www.foo.com, zoop.foo.com and any other host ending in foo.com. In
+Note that the first regular expressions will match "foo.com",
+"www.foo.com", "zoop.foo.com" and any other host ending in "foo.com". In
 this example this is intentional but you might want it to be a bit
-more tight, maybe relying on the == operator in stead, like this:::
+more tight, maybe relying on the ``==`` operator in stead, like this:::
 
     sub vcl_recv {
         if (req.http.host == "foo.com" or req.http.host == "www.foo.com") {
@@ -118,7 +118,7 @@ and resilience.
 
 You can define several backends and group them together in a
 director. This requires you to load a VMOD, a Varnish module, and then to
-call certain actions in vcl_init.::
+call certain actions in `vcl_init`.::
 
 
     import directors;    # load the directors
@@ -180,11 +180,11 @@ define the backends.::
         }
     }
 
-Whats new here is the probe. Varnish will check the health of each
+Whats new here is the ``probe``. Varnish will check the health of each
 backend with a probe. The options are:
 
 url
-    What URL should Varnish request.
+    The URL Varnish will use to send a probe request.
 
 interval
     How often should we poll
@@ -197,13 +197,17 @@ window
     window has five checks.
 
 threshold
-    How many of the .window last polls must be good for the backend to be declared healthy.
+    How many of the '.window' last polls must be good for the backend to be declared healthy.
+
+.. XXX: .window probably means something but not to me :) benc
 
 initial
-    How many of the of the probes a good when Varnish starts - defaults
+    How many of the probes that needs to be succesful when Varnish starts - defaults
     to the same amount as the threshold.
 
-Now we define the director.::
+Now we define the 'director'.::
+
+.. XXX: Where and why? benc
 
     import directors;
 
diff --git a/doc/sphinx/users-guide/vcl-built-in-subs.rst b/doc/sphinx/users-guide/vcl-built-in-subs.rst
index b30985c..708c05e 100644
--- a/doc/sphinx/users-guide/vcl-built-in-subs.rst
+++ b/doc/sphinx/users-guide/vcl-built-in-subs.rst
@@ -1,7 +1,7 @@
 
 .. _vcl-built-in-subs:
 
-.. XXX This document needs substational review.
+.. XXX:This document needs substantional review.
 
 
 Built in subroutines
@@ -12,14 +12,14 @@ vcl_recv
 ~~~~~~~~
 
 Called at the beginning of a request, after the complete request has
-been received and parsed.  Its purpose is to decide whether or not to
+been received and parsed. Its purpose is to decide whether or not to
 serve the request, how to do it, and, if applicable, which backend to
 use.
 
 It is also used to modify the request, something you'll probably find
 yourself doing frequently. 
 
-The vcl_recv subroutine may terminate with calling ``return()`` on one
+The `vcl_recv` subroutine may terminate with calling ``return()`` on one
 of the following keywords:
 
   synth 
@@ -27,10 +27,10 @@ of the following keywords:
     client and abandon the request.
 
   pass
-    Switch to pass mode.  Control will eventually pass to vcl_pass.
+    Switch to pass mode. Control will eventually pass to vcl_pass.
 
   pipe
-    Switch to pipe mode.  Control will eventually pass to vcl_pipe.
+    Switch to pipe mode. Control will eventually pass to vcl_pipe.
 
   hash
     Continue processing the object as a potential candidate for
@@ -43,13 +43,13 @@ of the following keywords:
 vcl_pipe
 ~~~~~~~~
 
-Called upon entering pipe mode.  In this mode, the request is passed
+Called upon entering pipe mode. In this mode, the request is passed
 on to the backend, and any further data from either client or backend
 is passed on unaltered until either end closes the
 connection. Basically, Varnish will degrade into a simple TCP proxy,
 shuffling bytes back and forth.
 
-The vcl_pipe subroutine may terminate with calling return() with one
+The `vcl_pipe` subroutine may terminate with calling ``return()`` with one
 of the following keywords:
 
   synth(error code, reason)
@@ -61,12 +61,12 @@ of the following keywords:
 vcl_pass
 ~~~~~~~~
 
-Called upon entering pass mode.  In this mode, the request is passed
+Called upon entering pass mode. In this mode, the request is passed
 on to the backend, and the backend's response is passed on to the
-client, but is not entered into the cache.  Subsequent requests
+client, but is not entered into the cache. Subsequent requests
 submitted over the same client connection are handled normally.
 
-The vcl_pass subroutine may terminate with calling return() with one
+The `vcl_pass` subroutine may terminate with calling ``return()`` with one
 of the following keywords:
 
   synth(error code, reason)
@@ -84,7 +84,10 @@ of the following keywords:
 vcl_hit
 ~~~~~~~
 
-Called is a cache lookup is successful. 
+Called when a cache lookup is successful. 
+
+.. XXX: missing the "The `vcl_hit` subroutine may terminate with calling ``return()`` with one of the following keywords:" thing. benc
+
 
   restart
     Restart the transaction. Increases the restart counter. If the number
@@ -92,7 +95,7 @@ Called is a cache lookup is successful.
     error.
 
   deliver
-    Deliver the object. Control passes to vcl_deliver.
+    Deliver the object. Control passes to `vcl_deliver`.
 
   synth(error code, reason)
     Return the specified error code to the client and abandon the request.
@@ -102,26 +105,26 @@ vcl_miss
 ~~~~~~~~
 
 Called after a cache lookup if the requested document was not found in
-the cache.  Its purpose is to decide whether or not to attempt to
+the cache. Its purpose is to decide whether or not to attempt to
 retrieve the document from the backend, and which backend to use.
 
-The vcl_miss subroutine may terminate with calling return() with one
+The `vcl_miss` subroutine may terminate with calling ``return()`` with one
 of the following keywords:
 
   synth(error code, reason)
     Return the specified error code to the client and abandon the request.
 
   pass
-    Switch to pass mode.  Control will eventually pass to vcl_pass.
+    Switch to pass mode. Control will eventually pass to `vcl_pass`.
 
   fetch
-    Retrieve the requested object from the backend.  Control will
-    eventually pass to vcl_fetch.
+    Retrieve the requested object from the backend. Control will
+    eventually pass to `vcl_fetch`.
 
 vcl_hash
 ~~~~~~~~
 
-Called after vcl_recv to create a hash value for the request. This is
+Called after `vcl_recv` to create a hash value for the request. This is
 used as a key to look up the object in Varnish.
 
   lookup
@@ -134,20 +137,21 @@ used as a key to look up the object in Varnish.
 vcl_purge
 ~~~~~~~~~
 
-Called after the purge has been executed and all it's variant have been evited. 
+Called after the purge has been executed and all its variants have been evited.
 
   synth
     Produce a response.
 
 
-
 vcl_deliver
 ~~~~~~~~~~~
 
 Called before a cached object is delivered to the client.
 
-The vcl_deliver subroutine may terminate with one of the following
-keywords:
+The ``vcl_deliver`` subroutine may terminate calling ``return()`` with one
+of the following keywords:
+
+.. XXX: Should perhaps be return as above? benc
 
   deliver
     Deliver the object to the client.
@@ -164,35 +168,46 @@ vcl_backend_fetch
 Called before sending the backend request. In this subroutine you
 typically alter the request before it gets to the backend.
 
+.. XXX: Missing terminate..keywords sentence? benc
+
   fetch
     Fetch the object from the backend.
 
   abandon
     Abandon the backend request and generates an error.
-  
+
 
 vcl_backend_response
 ~~~~~~~~~~~~~~~~~~~~
 
-Called after an response has been successfully retrieved from the
-backend. The response is availble as beresp. Note that Varnish might
+Called after a response has been successfully retrieved from the
+backend. The response is available as `beresp`. 
+
+.. XXX: beresp comes out of the blue here. maybe a short description? benc
+
+Note that Varnish might
 not be talking to an actual client, so operations that require a
-client to be present are not allowed. Specifically there is no req
-object and restarts are not allowed.
+client to be present are not allowed. Specifically there is no `req
+object` and restarts are not allowed.
+
+.. XXX: I do not follow sentence above. benc
 
-The vcl_backend_response subroutine may terminate with calling return() with one
+The `vcl_backend_response` subroutine may terminate with calling ``return()`` with one
 of the following keywords:
 
   deliver
     Possibly insert the object into the cache, then deliver it to the
-    Control will eventually pass to vcl_deliver. Caching is dependant
-    on beresp.cacheable.
+    Control will eventually pass to `vcl_deliver`. Caching is dependant
+    on 'beresp.cacheable'.
+
+.. XXX:A parameter? that is set how? benc
+    
 
   error(error code, reason)
     Return the specified error code to the client and abandon the request.
 
   retry
-    Retry the backend transaction. Increases the retries counter. If the number
+    Retry the backend transaction. Increases the `retries` counter. If the number
     of retries is higher than *max_retries* Varnish emits a guru meditation
     error.
 
@@ -201,11 +216,13 @@ vcl_backend_error
 
 This subroutine is called if we fail the backend fetch. 
 
+.. XXX:Missing the terminate return structure? benc
+
   deliver
     Deliver the error.
 
   retry
-    Retry the backend transaction. Increases the retries counter. If the number
+    Retry the backend transaction. Increases the `retries` counter. If the number
     of retries is higher than *max_retries* Varnish emits a guru meditation
     error.
 
@@ -213,10 +230,12 @@ This subroutine is called if we fail the backend fetch.
 vcl_backend_error
 ~~~~~~~~~~~~~~~~~
 
+.. XXX: Same name as section above? benc
+
 Called when we hit an error, either explicitly or implicitly due to
 backend or internal errors.
 
-The vcl_backend_error subroutine may terminate by calling return with one of
+The `vcl_backend_error` subroutine may terminate by calling ``return()`` with one of
 the following keywords:
 
   deliver
@@ -234,7 +253,9 @@ vcl_init
 Called when VCL is loaded, before any requests pass through it.
 Typically used to initialize VMODs.
 
-  return() values:
+.. XXX: Missing the terminate return structure? benc
+
+  ``return()`` values:
 
   ok
     Normal return, VCL continues loading.
@@ -246,7 +267,13 @@ vcl_fini
 Called when VCL is discarded only after all requests have exited the VCL.
 Typically used to clean up VMODs.
 
-  return() values:
+
+.. XXX: Missing the terminate return structure? benc
+
+  ``return()`` values:
 
   ok
     Normal return, VCL will be discarded.
+
+
+.. XXX: Maybe end here with the detailed flowchart from the book together with a reference to the book? benc
diff --git a/doc/sphinx/users-guide/vcl-example-acls.rst b/doc/sphinx/users-guide/vcl-example-acls.rst
index b460bbe..45afaa2 100644
--- a/doc/sphinx/users-guide/vcl-example-acls.rst
+++ b/doc/sphinx/users-guide/vcl-example-acls.rst
@@ -2,7 +2,7 @@
 ACLs
 ~~~~
 
-You create a named access control list with the *acl* keyword. You can match
+You create a named access control list with the ``acl`` keyword. You can match
 the IP address of the client against an ACL with the match operator.::
 
   # Who is allowed to purge....
@@ -21,4 +21,4 @@ the IP address of the client against an ACL with the match operator.::
       }
     } 
   }
-  
+ 
diff --git a/doc/sphinx/users-guide/vcl-example-manipulating-headers.rst b/doc/sphinx/users-guide/vcl-example-manipulating-headers.rst
index 35dcb1e..7e65663 100644
--- a/doc/sphinx/users-guide/vcl-example-manipulating-headers.rst
+++ b/doc/sphinx/users-guide/vcl-example-manipulating-headers.rst
@@ -4,7 +4,7 @@
 Manipulating request headers in VCL
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-Lets say we want to remove the cookie for all objects in the /images
+Lets say we want to remove the cookie for all objects in the `/images`
 directory of our web server::
 
   sub vcl_recv {
diff --git a/doc/sphinx/users-guide/vcl-example-manipulating-responses.rst b/doc/sphinx/users-guide/vcl-example-manipulating-responses.rst
index b5724b4..6719524 100644
--- a/doc/sphinx/users-guide/vcl-example-manipulating-responses.rst
+++ b/doc/sphinx/users-guide/vcl-example-manipulating-responses.rst
@@ -15,5 +15,5 @@ matches certain criteria::
 
 
 
-We also remove any Set-Cookie headers in order to avoid a hit-for-pass
+We also remove any Set-Cookie headers in order to avoid a `hit-for-pass`
 object to be created. See :ref:`user-guide-vcl_actions`.
diff --git a/doc/sphinx/users-guide/vcl-example-websockets.rst b/doc/sphinx/users-guide/vcl-example-websockets.rst
index 7217b88..609fe61 100644
--- a/doc/sphinx/users-guide/vcl-example-websockets.rst
+++ b/doc/sphinx/users-guide/vcl-example-websockets.rst
@@ -18,3 +18,4 @@ VCL config to do so::
          }
     }
 
+.. XXX: Pipe it? maybe a bit more explanation here? benc
diff --git a/doc/sphinx/users-guide/vcl-grace.rst b/doc/sphinx/users-guide/vcl-grace.rst
index f318aa3..afc92bd 100644
--- a/doc/sphinx/users-guide/vcl-grace.rst
+++ b/doc/sphinx/users-guide/vcl-grace.rst
@@ -7,13 +7,12 @@ A key feature of Varnish is its ability to shield you from misbehaving
 web- and application servers.
 
 
-
 Grace mode
 ~~~~~~~~~~
 
 When several clients are requesting the same page Varnish will send
 one request to the backend and place the others on hold while fetching
-one copy from the back end. In some products this is called request
+one copy from the backend. In some products this is called request
 coalescing and Varnish does this automatically.
 
 If you are serving thousands of hits per second the queue of waiting
diff --git a/doc/sphinx/users-guide/vcl-hashing.rst b/doc/sphinx/users-guide/vcl-hashing.rst
index fdbe37f..83758ad 100644
--- a/doc/sphinx/users-guide/vcl-hashing.rst
+++ b/doc/sphinx/users-guide/vcl-hashing.rst
@@ -1,12 +1,12 @@
 Hashing
 -------
 
-Internally, when Varnish stores content in it's store it uses a hash
+Internally, when Varnish stores content in the cache it stores the object together with a hash
 key to find the object again. In the default setup this key is
 calculated based on the content of the *Host* header or the IP address
 of the server and the URL.
 
-Behold the default vcl::
+Behold the `default vcl`::
 
  sub vcl_hash {
      hash_data(req.url);
@@ -18,10 +18,10 @@ Behold the default vcl::
      return (hash);
  }
 
-As you can see it first chucks in req.url then req.http.host if it
+As you can see it first checks in `req.url` then `req.http.host` if it
 exists. It is worth pointing out that Varnish doesn't lowercase the
-hostname or the URL before hashing it so in theory having Varnish.org/
-and varnish.org/ would result in different cache entries. Browers
+hostname or the URL before hashing it so in theory having "Varnish.org/"
+and "varnish.org/" would result in different cache entries. Browsers
 however, tend to lowercase hostnames.
 
 You can change what goes into the hash. This way you can make Varnish
@@ -33,11 +33,11 @@ based on where their IP address is located. You would need some Vmod
 to get a country code and then put it into the hash. It might look
 like this.
 
-In vcl_recv::
+In `vcl_recv`::
 
   set req.http.X-Country-Code = geoip.lookup(client.ip);
 
-And then add a vcl_hash::
+And then add a `vcl_hash`::
 
  sub vcl_hash {
    hash_data(req.http.X-Country-Code);
@@ -45,6 +45,6 @@ And then add a vcl_hash::
 
 As the default VCL will take care of adding the host and URL to the
 hash we don't have to do anything else. Be careful calling
-return(hash) as this will abort the execution of the default VCL and
-thereby you can end up with a Varnish that will return data based on
+``return(hash)`` as this will abort the execution of the default VCL and
+Varnish can end up returning data based on
 more or less random inputs.
diff --git a/doc/sphinx/users-guide/vcl-inline-c.rst b/doc/sphinx/users-guide/vcl-inline-c.rst
index 7c88cf9..5cc0ead 100644
--- a/doc/sphinx/users-guide/vcl-inline-c.rst
+++ b/doc/sphinx/users-guide/vcl-inline-c.rst
@@ -10,7 +10,7 @@ You can use *in-line C* to extend Varnish. Please note that you can
 seriously mess up Varnish this way. The C code runs within the Varnish
 Cache process so if your code generates a segfault the cache will crash.
 
-One of the first uses I saw of In-line C was logging to syslog.::
+One of the first uses of In-line C was logging to `syslog`.::
 
         # The include statements must be outside the subroutines.
         C{
diff --git a/doc/sphinx/users-guide/vcl-syntax.rst b/doc/sphinx/users-guide/vcl-syntax.rst
index 889ea97..4615291 100644
--- a/doc/sphinx/users-guide/vcl-syntax.rst
+++ b/doc/sphinx/users-guide/vcl-syntax.rst
@@ -9,7 +9,7 @@ preferences.
 
 Note that VCL doesn't contain any loops or jump statements.
 
-This document gives an outline of the most important parts of the
+This section provides an outline of the more important parts of the
 syntax. For a full documentation of VCL syntax please see
 :ref:`reference-vcl` in the reference.
 
@@ -18,7 +18,7 @@ Strings
 
 Basic strings are enclosed in " ... ", and may not contain newlines.
 
-Backslash is not special, so for instance in regsub() you do not need
+Backslash is not special, so for instance in `regsub()` you do not need
 to do the "count-the-backslashes" polka:::
 
   regsub("barf", "(b)(a)(r)(f)", "\4\3\2p") -> "frap"
@@ -43,9 +43,9 @@ which can later be used to match client addresses::
        }
 
 If an ACL entry specifies a host name which Varnish is unable to
-resolve, it will match any address it is compared to.  Consequently,
+resolve, it will match any address it is compared to. Consequently,
 if it is preceded by a negation mark, it will reject any address it is
-compared to, which may not be what you intended.  If the entry is
+compared to, which may not be what you intended. If the entry is
 enclosed in parentheses, however, it will simply be ignored.
 
 To match an IP address against an ACL, simply use the match operator::
@@ -93,10 +93,13 @@ A subroutine is used to group code for legibility or reusability:
 
 Subroutines in VCL do not take arguments, nor do they return values.
 
-To call a subroutine, use the call keyword followed by the subroutine's name:
+To call a subroutine, use the call keyword followed by the subroutine's name::
 
 call pipe_if_local;
 
 Varnish has quite a few built in subroutines that are called for each
 transaction as it flows through Varnish. These builtin subroutines are all named vcl_*. Your own subroutines cannot start their name with vcl_.
+
+.. XXX:looks as bit funky as red text? benc
+
 See :ref:`vcl-built-in-subs`.
diff --git a/doc/sphinx/users-guide/vcl-variables.rst b/doc/sphinx/users-guide/vcl-variables.rst
index 20fcd4e..88ab8f4 100644
--- a/doc/sphinx/users-guide/vcl-variables.rst
+++ b/doc/sphinx/users-guide/vcl-variables.rst
@@ -1,27 +1,33 @@
 
-Requests, responses and objects
+Requests and responses as objects
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-In VCL, there several important objects.
+.. XXX: refactored headline. benc
+
+In VCL, there several important objects that you need to be aware of. These objects can be accessed and manipulated using VCL.
 
 
 *req*
- The request object. When Varnish has received the request the req object is 
- created and populated. Most of the work you do in vcl_recv you 
- do on or with the req object.
+ The request object. When Varnish has received the request the `req` object is 
+ created and populated. Most of the work you do in `vcl_recv` you 
+ do on or with the `req` object.
 
 *bereq*
  The backend request object. Varnish contructs this before sending it to the 
- backend. It is based on the req object.
+ backend. It is based on the `req` object.
+
+.. XXX:in what way? benc
 
 *beresp*
  The backend response object. It contains the headers of the object 
  coming from the backend. If you want to modify the reponse coming from the 
- server you modify this object in vcl_backend_reponse. 
+ server you modify this object in `vcl_backend_reponse`. 
 
 *resp*
  The HTTP response right before it is delivered to the client. It is
- typically modified in vcl_deliver.
+ typically modified in `vcl_deliver`.
 
 *obj* 
  The object as it is stored in cache. Mostly read only.
+.. XXX:What object? the current request? benc
+
diff --git a/doc/sphinx/users-guide/vcl.rst b/doc/sphinx/users-guide/vcl.rst
index ad65403..8585241 100644
--- a/doc/sphinx/users-guide/vcl.rst
+++ b/doc/sphinx/users-guide/vcl.rst
@@ -3,12 +3,12 @@
 VCL - Varnish Configuration Language
 ------------------------------------
 
-This section is about getting Varnish to do what you want to
+This section covers how to tell Varnish how to handle
 your HTTP traffic, using the Varnish Configuration Language (VCL).
 
 Varnish has a great configuration system. Most other systems use
 configuration directives, where you basically turn on and off lots of
-switches. Varnish uses a domain specific language called VCL for this.
+switches. We have instead chosen to use a domain specific language called VCL for this.
 
 Every inbound request flows through Varnish and you can influence how
 the request is being handled by altering the VCL code. You can direct
@@ -26,7 +26,7 @@ request, another when files are fetched from the backend server.
 
 If you don't call an action in your subroutine and it reaches the end
 Varnish will execute some built-in VCL code. You will see this VCL
-code commented out in builtin.vcl that ships with Varnish Cache.
+code commented out in the file `builtin.vcl` that ships with Varnish Cache.
 
 .. _users-guide-vcl_fetch_actions:
 
@@ -43,5 +43,7 @@ code commented out in builtin.vcl that ships with Varnish Cache.
    vcl-inline-c
    vcl-examples
    websockets
+.. XXX: websockets seems to be missing? does it refer to the last sample in the vcl index if so already included. benc
+
    devicedetection
 
diff --git a/doc/sphinx/whats-new/changes.rst b/doc/sphinx/whats-new/changes.rst
index cf027c8..274c9f5 100644
--- a/doc/sphinx/whats-new/changes.rst
+++ b/doc/sphinx/whats-new/changes.rst
@@ -3,14 +3,15 @@
 Changes in Varnish 4
 ====================
 
-Varnish 4 is quite an extensive update over Varnish 3, with some very big improvements to central parts of varnish.
+Varnish 4 is quite an extensive update to Varnish 3, with some very big improvements to central parts of varnish.
 
 Client/backend split
 --------------------
 In the past, Varnish has fetched the content from the backend in the same
-thread as the client request. The client and backend code has now been split,
-allowing for some much requested improvements.
-This split allows varnish to refresh content in the background while serving
+thread as the client request.In Varnish 4 we have  split the client and backend code into separate trheads allowing for some much requested improvements.
+This split allows Varnish to refresh content in the background while serving
 stale content quickly to the client.
 
-This split has also necessitated a change of the VCL-functions, in particular functionality has moved from the old vcl_fetch method to the two new methods vcl_backend_fetch and vcl_backend_response.
+This split has also necessitated a change of the VCL-functions, in particular functionality has moved from the old `vcl_fetch` method to the two new methods `vcl_backend_fetch` and `vcl_backend_response`.
+
+.. XXX:Here would an updated flow-diagram over functions be great. benc
diff --git a/doc/sphinx/whats-new/index.rst b/doc/sphinx/whats-new/index.rst
index a1d1d73..c9d9130 100644
--- a/doc/sphinx/whats-new/index.rst
+++ b/doc/sphinx/whats-new/index.rst
@@ -1,14 +1,16 @@
 .. _whats-new-index:
 
 %%%%%%%%%%%%%%%%%%%%%%%%%%
-What's new for Varnish 4.0
+What's new in Varnish 4.0
 %%%%%%%%%%%%%%%%%%%%%%%%%%
 
-This document describes the changes that have been made for Varnish 4. The
-first section will describe the overarching changes that have gone into
-Varnish, while the second section describes what changes you need to make to
-your configuration as well as any changes in behaviour that you need to take
-into consideration while upgrading.
+This section describes the changes that have been made for Varnish 4. The
+first subsection describes overarching changes that have gone into
+Varnish 4.0, while the second subsection describes changes you need to make to
+your current configuration (assuming you are on Varnish 3.x) as well as any changes in behaviour that you need to be aware of and take
+into consideration when upgrading.
+
+.. XXX:Heavy change of meaning above! benc
 
 .. toctree::
    :maxdepth: 2
diff --git a/doc/sphinx/whats-new/upgrading.rst b/doc/sphinx/whats-new/upgrading.rst
index 5f52318..2921835 100644
--- a/doc/sphinx/whats-new/upgrading.rst
+++ b/doc/sphinx/whats-new/upgrading.rst
@@ -37,7 +37,7 @@ Since the client director was already a special case of the hash director, it ha
 
 error() is now a return value
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-You must now explicitly return an error::
+You must explicitly return an error::
 
 	return(error(999, "Response));
 
@@ -76,11 +76,11 @@ vcl_recv should return(hash) instead of lookup now
 
 req.* not available in vcl_backend_response
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-req.* used to be available in vcl_fetch, but after the split of functionality, you only have bereq.* in vcl_backend_response.
+req.* used to be available in `vcl_fetch`, but after the split of functionality, you only have 'bereq.*' in `vcl_backend_response`.
 
 vcl_* reserved
 ~~~~~~~~~~~~~~
-Your own subs cannot be named vcl_* anymore. That is reserved for builtin subs.
+Any custom-made subs cannot be named 'vcl_*' anymore. This namespace is reserved for builtin subs.
 
 req.backend.healthy replaced by std.healthy(req.backend)
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~



From lkarsten at varnish-software.com  Tue Apr  1 13:09:51 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:51 +0200
Subject: [4.0] b4068d3 minor update
Message-ID: <E1WUyRr-00038D-Q5@project.varnish-software.com>


commit b4068d3ead5974626bfae87a01bda521a5af0fa3
Author: benc <benc at redpill-linpro.com>
Date:   Sun Mar 16 12:55:16 2014 +0100

    minor update

diff --git a/doc/sphinx/users-guide/purging.rst b/doc/sphinx/users-guide/purging.rst
index 2be32e3..214fe81 100644
--- a/doc/sphinx/users-guide/purging.rst
+++ b/doc/sphinx/users-guide/purging.rst
@@ -21,7 +21,7 @@ A *purge* is what happens when you pick out an object from the cache
 and discard it along with its variants. Usually a purge is invoked
 through HTTP with the method `PURGE`.
 
-An HTTP purge is similar to a HTTP GET request, except that the
+A HTTP purge is similar to a HTTP GET request, except that the
 *method* is `PURGE`. Actually you can call the method whatever you'd
 like, but most people refer to this as purging. Squid, for example, supports the
 same mechanism. In order to support purging in Varnish you need the



From lkarsten at varnish-software.com  Tue Apr  1 13:09:51 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:51 +0200
Subject: [4.0] 64438cd Minor cleanups
Message-ID: <E1WUyRr-00038M-RF@project.varnish-software.com>


commit 64438cd0e89378430e55f86b349159e4564aa207
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu Mar 20 12:35:38 2014 +0100

    Minor cleanups

diff --git a/doc/sphinx/installation/help.rst b/doc/sphinx/installation/help.rst
index 946e921..66433de 100644
--- a/doc/sphinx/installation/help.rst
+++ b/doc/sphinx/installation/help.rst
@@ -38,7 +38,7 @@ it mostly on topic, and please don't paste random links unless they are
 Mailing Lists
 =============
 
-Subscribing or unsubscribing to our mailing lists is handled through our Mailman_.
+Subscribing or unsubscribing to our mailing lists is handled through mailman_.
 
 If you are going to use Varnish, subscribing to our `varnish-announce`
 mailing list is a very good idea. The typical pattern is that
diff --git a/doc/sphinx/installation/install.rst b/doc/sphinx/installation/install.rst
index 348b6ef..2cddd59 100644
--- a/doc/sphinx/installation/install.rst
+++ b/doc/sphinx/installation/install.rst
@@ -106,11 +106,8 @@ packages installed:
 * `pcre-devel`
 * `pkgconfig`
 * `libedit-devel`
-
-If you're building from git, you also need the following:
-
 * `docutils`
-* `python-sphinx` (optional, if you want to build the documentation)
+* `python-sphinx`
 
 Compiling Varnish
 -----------------
@@ -144,9 +141,9 @@ And finally, the true test of a brave heart::
 
 	sudo make install
 
-Varnish will now be installed in `/usr/loca``l. The ``varnishd` binary is in
+Varnish will now be installed in `/usr/local`. The ``varnishd` binary is in
 `/usr/local/sbin/varnishd` and its default configuration will be
 `/usr/local/etc/varnish/default.vcl`.
 
-After succesful installation you are ready to proceed to the :ref:`tutorial-index`.
+After successful installation you are ready to proceed to the :ref:`tutorial-index`.
 
diff --git a/doc/sphinx/tutorial/backend_servers.rst b/doc/sphinx/tutorial/backend_servers.rst
index 1c9ab65..cb5f087 100644
--- a/doc/sphinx/tutorial/backend_servers.rst
+++ b/doc/sphinx/tutorial/backend_servers.rst
@@ -3,7 +3,7 @@
 Backend servers
 ---------------
 
-Varnish has a concept of `backend` or `origin` servers. A backend
+Varnish has a concept of `backend` or origin servers. A backend
 server is the server providing the content Varnish will accelerate via the cache.
 
 Our first task is to tell Varnish where it can find its content. Start
@@ -41,15 +41,9 @@ localhost, port 8080.::
   }
 
 
-Varnish can have several backends defined and can even join
-several backends together into clusters of backends for load balancing
-purposes, having Varnish pick one backend based on different
-algorithms. 
-
-A lot of the power of Varnish Cache comes from it's design, which
-might not be what you are expecting.
-
-.. XXX:What am I expecting? benc
+Varnish can have several backends defined and can even join several backends
+together into clusters of backends for load balancing purposes, having Varnish
+pick one backend based on different algorithms. 
 
 Next, let's have a look at some of what makes Varnish unique and what you can do with it.
 



From lkarsten at varnish-software.com  Tue Apr  1 13:09:51 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:51 +0200
Subject: [4.0] a077da7 Use pipe for chunked POST/PUT requests.
Message-ID: <E1WUyRr-00038V-TD@project.varnish-software.com>


commit a077da7c9561299a4727387785cbe8f3211d6722
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu Mar 20 12:40:40 2014 +0100

    Use pipe for chunked POST/PUT requests.
    
    We don't know how to handle incoming chunked bodies currently, but the backend
    might.

diff --git a/bin/varnishd/builtin.vcl b/bin/varnishd/builtin.vcl
index 3949689..71e01f7 100644
--- a/bin/varnishd/builtin.vcl
+++ b/bin/varnishd/builtin.vcl
@@ -55,6 +55,13 @@ sub vcl_recv {
         /* Non-RFC2616 or CONNECT which is weird. */
         return (pipe);
     }
+
+    /* We don't support chunked uploads, except when piping. */
+    if ((req.request == "POST" || req.request == "PUT") &&
+      req.http.transfer-encoding ~ "chunked") {
+        return(pipe);
+    }
+
     if (req.method != "GET" && req.method != "HEAD") {
         /* We only deal with GET and HEAD by default */
         return (pass);



From lkarsten at varnish-software.com  Tue Apr  1 13:09:51 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:51 +0200
Subject: [4.0] b966907 Squash some RST warnings
Message-ID: <E1WUyRr-00038e-Ui@project.varnish-software.com>


commit b966907afd8a2790b66a68e1d74b0e0e200bb959
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu Mar 20 13:06:55 2014 +0100

    Squash some RST warnings

diff --git a/doc/sphinx/installation/bugs.rst b/doc/sphinx/installation/bugs.rst
index dd71cc8..de433a1 100644
--- a/doc/sphinx/installation/bugs.rst
+++ b/doc/sphinx/installation/bugs.rst
@@ -83,13 +83,14 @@ general forms:
 	for HTTP headers and Cookies.
 
 	Please try that before reporting a bug.
-..  (TODO: which params to tweak)
 
 "Assert error in ..."
 	This is something bad that should never happen, and a bug
 	report is almost certainly in order. As always, if in doubt
 	ask us on IRC before opening the ticket.
 
+..  (TODO: in the ws-size note above, mention which params to tweak)
+
 In your syslog it may all be joined into one single line, but if you
 can reproduce the crash, do so while running `varnishd` manually:
 
diff --git a/doc/sphinx/installation/install.rst b/doc/sphinx/installation/install.rst
index 2cddd59..a407624 100644
--- a/doc/sphinx/installation/install.rst
+++ b/doc/sphinx/installation/install.rst
@@ -141,7 +141,7 @@ And finally, the true test of a brave heart::
 
 	sudo make install
 
-Varnish will now be installed in `/usr/local`. The ``varnishd` binary is in
+Varnish will now be installed in `/usr/local`. The ``varnishd`` binary is in
 `/usr/local/sbin/varnishd` and its default configuration will be
 `/usr/local/etc/varnish/default.vcl`.
 
diff --git a/doc/sphinx/users-guide/command-line.rst b/doc/sphinx/users-guide/command-line.rst
index 9016d51..aa8c5be 100644
--- a/doc/sphinx/users-guide/command-line.rst
+++ b/doc/sphinx/users-guide/command-line.rst
@@ -16,7 +16,7 @@ you will find the startup options here:
 
 
 '-a' *listen_address*
-^^^^^^^^^^^^^^^^^^^
+^^^^^^^^^^^^^^^^^^^^^
 
 The '-a' argument defines what address Varnish should listen to, and service HTTP requests from.
 
@@ -42,7 +42,7 @@ If your webserver runs on the same machine, you will have to move
 it to another port number first.
 
 '-f' *VCL-file* or '-b' *backend*
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 
 Varnish needs to know where to find the HTTP server it is caching for.
diff --git a/doc/sphinx/users-guide/esi.rst b/doc/sphinx/users-guide/esi.rst
index 45f8814..5c7ce72 100644
--- a/doc/sphinx/users-guide/esi.rst
+++ b/doc/sphinx/users-guide/esi.rst
@@ -82,7 +82,7 @@ For example::
   -->
 
 Doing ESI on JSON and other non-XML'ish content
-----------------------------------------------
+-----------------------------------------------
 
 Please note that Varnish will peek at the included content. If it
 doesn't start with a "<" Varnish assumes you didn't really mean to
diff --git a/doc/sphinx/users-guide/index.rst b/doc/sphinx/users-guide/index.rst
index 3d4d831..bbc7d6a 100644
--- a/doc/sphinx/users-guide/index.rst
+++ b/doc/sphinx/users-guide/index.rst
@@ -40,9 +40,8 @@ from a transactional level to aggregating statistics.
    vcl
    report
    performance
-   esi   
+   esi
    troubleshooting
-   orphans
 
 .. customizing (which is a non ideal title)
 
diff --git a/doc/sphinx/users-guide/run_security.rst b/doc/sphinx/users-guide/run_security.rst
index d7340a0..7607cfc 100644
--- a/doc/sphinx/users-guide/run_security.rst
+++ b/doc/sphinx/users-guide/run_security.rst
@@ -168,8 +168,6 @@ Furthermore you may want to look at and lock down:
 
 :ref:`ref_param_syslog_cli_traffic`
 	Log all CLI commands to `syslog(8)`, so you know what goes on.
-.. XXX: syslog(8)? benc
-
 
 :ref:`ref_param_vcc_unsafe_path`
 	Restrict VCL/VMODS to :ref:`ref_param_vcl_dir` and :ref:`ref_param_vmod_dir`
diff --git a/doc/sphinx/users-guide/troubleshooting.rst b/doc/sphinx/users-guide/troubleshooting.rst
index 9d57744..d05b9d6 100644
--- a/doc/sphinx/users-guide/troubleshooting.rst
+++ b/doc/sphinx/users-guide/troubleshooting.rst
@@ -91,10 +91,8 @@ A core dumped is usually due to a bug in Varnish. However, in order to
 debug a segfault the developers need you to provide a fair bit of
 data.
 
- * Make sure you have Varnish installed with symbols
-.. XXX:Symbols? benc
- * Make sure core dumps are enabled (ulimit)
-.. XXX:ulimit? benc
+ * Make sure you have Varnish installed with debugging symbols.
+ * Make sure core dumps are allowed in the parent shell. (``ulimit -c unlimited``)
 
 Once you have the core you open it with `gdb` and issue the command ``bt``
 to get a stack trace of the thread that caused the segfault.
diff --git a/doc/sphinx/users-guide/vcl-syntax.rst b/doc/sphinx/users-guide/vcl-syntax.rst
index 4615291..f0b427f 100644
--- a/doc/sphinx/users-guide/vcl-syntax.rst
+++ b/doc/sphinx/users-guide/vcl-syntax.rst
@@ -82,20 +82,20 @@ down for, uhm, examples.
 Subroutines
 ~~~~~~~~~~~
 
-A subroutine is used to group code for legibility or reusability:
-::
-  
+A subroutine is used to group code for legibility or reusability::
+
   sub pipe_if_local {
     if (client.ip ~ local) {
       return (pipe);
     }
   }
 
+
 Subroutines in VCL do not take arguments, nor do they return values.
 
 To call a subroutine, use the call keyword followed by the subroutine's name::
 
-call pipe_if_local;
+  call pipe_if_local;
 
 Varnish has quite a few built in subroutines that are called for each
 transaction as it flows through Varnish. These builtin subroutines are all named vcl_*. Your own subroutines cannot start their name with vcl_.
diff --git a/doc/sphinx/users-guide/vcl-variables.rst b/doc/sphinx/users-guide/vcl-variables.rst
index 88ab8f4..644c046 100644
--- a/doc/sphinx/users-guide/vcl-variables.rst
+++ b/doc/sphinx/users-guide/vcl-variables.rst
@@ -1,33 +1,35 @@
 
-Requests and responses as objects
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Request and response VCL objects
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 .. XXX: refactored headline. benc
 
-In VCL, there several important objects that you need to be aware of. These objects can be accessed and manipulated using VCL.
+In VCL, there several important objects that you need to be aware of. These
+objects can be accessed and manipulated using VCL.
 
 
 *req*
- The request object. When Varnish has received the request the `req` object is 
- created and populated. Most of the work you do in `vcl_recv` you 
+ The request object. When Varnish has received the request the `req` object is
+ created and populated. Most of the work you do in `vcl_recv` you
  do on or with the `req` object.
 
 *bereq*
- The backend request object. Varnish contructs this before sending it to the 
+ The backend request object. Varnish contructs this before sending it to the
  backend. It is based on the `req` object.
 
 .. XXX:in what way? benc
 
 *beresp*
- The backend response object. It contains the headers of the object 
- coming from the backend. If you want to modify the reponse coming from the 
- server you modify this object in `vcl_backend_reponse`. 
+ The backend response object. It contains the headers of the object
+ coming from the backend. If you want to modify the response coming from the
+ server you modify this object in `vcl_backend_reponse`.
 
 *resp*
  The HTTP response right before it is delivered to the client. It is
  typically modified in `vcl_deliver`.
 
 *obj* 
- The object as it is stored in cache. Mostly read only.
+ The object as it is stored in cache. Read only.
+
 .. XXX:What object? the current request? benc
 



From lkarsten at varnish-software.com  Tue Apr  1 13:09:51 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:51 +0200
Subject: [4.0] 8c56d06 Old habits die hard.
Message-ID: <E1WUyRr-00038o-WB@project.varnish-software.com>


commit 8c56d063c1dfd5dbdafdeae26416f0bb4bce5288
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu Mar 20 13:11:55 2014 +0100

    Old habits die hard.
    
    req.request is now called req.method.

diff --git a/bin/varnishd/builtin.vcl b/bin/varnishd/builtin.vcl
index 71e01f7..f9d3527 100644
--- a/bin/varnishd/builtin.vcl
+++ b/bin/varnishd/builtin.vcl
@@ -57,7 +57,7 @@ sub vcl_recv {
     }
 
     /* We don't support chunked uploads, except when piping. */
-    if ((req.request == "POST" || req.request == "PUT") &&
+    if ((req.method == "POST" || req.method == "PUT") &&
       req.http.transfer-encoding ~ "chunked") {
         return(pipe);
     }



From lkarsten at varnish-software.com  Tue Apr  1 13:09:52 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:52 +0200
Subject: [4.0] 5e3e98f Correct article
Message-ID: <E1WUyRs-00038x-1I@project.varnish-software.com>


commit 5e3e98f054309e34ccbef67bd030acf22369f388
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu Mar 20 13:30:22 2014 +0100

    Correct article

diff --git a/doc/sphinx/glossary/index.rst b/doc/sphinx/glossary/index.rst
index 3da5af3..574156b 100644
--- a/doc/sphinx/glossary/index.rst
+++ b/doc/sphinx/glossary/index.rst
@@ -34,11 +34,11 @@ Varnish Glossary
    backend
 	The HTTP server varnishd is caching for.  This can be
 	any sort of device that handles HTTP requests, including, but
-	not limited to: a webserver, a CMS, a load-balancer 
+	not limited to: a webserver, a CMS, a load-balancer
 	another varnishd, etc.
 
    client
-	The program which sends varnishd a HTTP request, typically
+	The program which sends varnishd an HTTP request, typically
 	a browser, but do not forget to think about spiders, robots
 	script-kiddies and criminals.
 
@@ -65,13 +65,13 @@ Varnish Glossary
    .. comment: "components of traffic ---------------------------------"
 
    header
-	A HTTP protocol header, like "Accept-Encoding:".
+	An HTTP protocol header, like "Accept-Encoding:".
 
    request
 	What the client sends to varnishd and varnishd sends to the backend.
 
    response
-	What the backend returns to varnishd and varnishd returns to 
+	What the backend returns to varnishd and varnishd returns to
 	the client.  When the response is stored in varnishd's cache,
 	we call it an object.
 
diff --git a/doc/sphinx/users-guide/increasing-your-hitrate.rst b/doc/sphinx/users-guide/increasing-your-hitrate.rst
index 2f22e3c..4be147f 100644
--- a/doc/sphinx/users-guide/increasing-your-hitrate.rst
+++ b/doc/sphinx/users-guide/increasing-your-hitrate.rst
@@ -48,7 +48,7 @@ Tool: lwp-request
 ~~~~~~~~~~~~~~~~~
 
 `lwp-request` is tool that is a part of The World-Wide Web library for Perl. It's a
-couple of really basic programs that can execute a HTTP request and
+couple of really basic programs that can execute an HTTP request and
 show you the result. We mostly use the two programs, ``GET`` and ``HEAD``.
 
 vg.no was the first site to use Varnish and the people running Varnish
diff --git a/doc/sphinx/users-guide/purging.rst b/doc/sphinx/users-guide/purging.rst
index 214fe81..c0e8381 100644
--- a/doc/sphinx/users-guide/purging.rst
+++ b/doc/sphinx/users-guide/purging.rst
@@ -21,7 +21,7 @@ A *purge* is what happens when you pick out an object from the cache
 and discard it along with its variants. Usually a purge is invoked
 through HTTP with the method `PURGE`.
 
-A HTTP purge is similar to a HTTP GET request, except that the
+An HTTP purge is similar to an HTTP GET request, except that the
 *method* is `PURGE`. Actually you can call the method whatever you'd
 like, but most people refer to this as purging. Squid, for example, supports the
 same mechanism. In order to support purging in Varnish you need the
diff --git a/doc/sphinx/users-guide/run_cli.rst b/doc/sphinx/users-guide/run_cli.rst
index 3e75ac5..40c6b4e 100644
--- a/doc/sphinx/users-guide/run_cli.rst
+++ b/doc/sphinx/users-guide/run_cli.rst
@@ -116,7 +116,7 @@ but they are banned from delivery.
 
 Instead of checking each and every cached object right away, we
 test each object against the regular expression only if and when
-a HTTP request asks for it.
+an HTTP request asks for it.
 
 Banning stuff is much cheaper than restarting Varnish to get rid
 of wronly cached content.
diff --git a/doc/sphinx/users-guide/run_security.rst b/doc/sphinx/users-guide/run_security.rst
index 7607cfc..5fe3890 100644
--- a/doc/sphinx/users-guide/run_security.rst
+++ b/doc/sphinx/users-guide/run_security.rst
@@ -6,7 +6,7 @@ Security first
 If you are the only person involved in running Varnish, or if all
 the people involved are trusted to the same degree, you can skip
 this chapter. We have protected Varnish as well as we can from
-anything which can come in through a HTTP socket.
+anything which can come in through an HTTP socket.
 
 If parts of your web infrastructure are outsourced or otherwise
 partitioned along administrative lines, you need to think about



From lkarsten at varnish-software.com  Tue Apr  1 13:09:52 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:52 +0200
Subject: [4.0] bb8e863 Reindent for spaces, explain obj changes.
Message-ID: <E1WUyRs-000397-2J@project.varnish-software.com>


commit bb8e863d89f0a8c19e245f41143f187cb91bd921
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu Mar 20 15:02:08 2014 +0100

    Reindent for spaces, explain obj changes.

diff --git a/doc/sphinx/whats-new/upgrading.rst b/doc/sphinx/whats-new/upgrading.rst
index 2921835..02e6f02 100644
--- a/doc/sphinx/whats-new/upgrading.rst
+++ b/doc/sphinx/whats-new/upgrading.rst
@@ -25,15 +25,14 @@ Use the hash director as a client director
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Since the client director was already a special case of the hash director, it has been removed, and you should use the hash director directly::
 
-	sub vcl_init {
-        	new h = directors.hash();
-        	h.add_backend(b1, 1);
-        	h.add_backend(b2, 1);
-	}
-
-	sub vcl_recv {
-		set req.backend_hint = h.backend(client.ip);
-	}
+    sub vcl_init {
+        new h = directors.hash();
+        h.add_backend(b1, 1);
+        h.add_backend(b2, 1);
+    }
+    sub vcl_recv {
+        set req.backend_hint = h.backend(client.ip);
+    }
 
 error() is now a return value
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -85,6 +84,12 @@ Any custom-made subs cannot be named 'vcl_*' anymore. This namespace is reserved
 req.backend.healthy replaced by std.healthy(req.backend)
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
+obj is now read-only
+~~~~~~~~~~~~~~~~~~~~
+
+`obj` is now read-only. `obj.hits`, if enabled in VCL, now counts per objecthead,
+not per object. `obj.last_use` has been retired.
+
 Changes to parameters
 =====================
 



From lkarsten at varnish-software.com  Tue Apr  1 13:09:52 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:52 +0200
Subject: [4.0] c8add6f Add a conventions section.
Message-ID: <E1WUyRs-00039G-3c@project.varnish-software.com>


commit c8add6f7d95d49846c23d5c21c31c6504fcbb092
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu Mar 20 15:29:52 2014 +0100

    Add a conventions section.
    
    Try to document our conventions when it comes to commands, paths, tool names
    and similar.
    
    Also write some formatting rules for maintainers, like wrapping at 80
    characters and avoiding tabs.
    
    I'm happy to discuss changing these if someone feels strongly about anything,
    the main point is just to make it easier to read though being consequent about
    it throughout the manual.

diff --git a/doc/sphinx/index.rst b/doc/sphinx/index.rst
index 830149a..11b4cce 100644
--- a/doc/sphinx/index.rst
+++ b/doc/sphinx/index.rst
@@ -1,36 +1,75 @@
+Welcome to the Varnish documentation!
+=====================================
+
+Introduction
+------------
+
+Varnish Cache is a web application accelerator also known as a caching HTTP
+reverse proxy. You install it in front of any server that speaks HTTP and
+configure it to cache the contents. Varnish Cache is really, really fast. It
+typically speeds up delivery with a factor of 300 - 1000x, depending on your
+architecture.
+
+
+To get started with Varnish-Cache we recommend that you read the installation
+guide :ref:`install-index`. Once you have Varnish up and running we recommend
+that you go through our tutorial - :ref:`tutorial-index`, and finally the
+:ref:`users_guide_index`.
+
+If you need to find out how to use a specific Varnish tool, the
+:ref:`reference-index` contains detailed documentation over the tools. Changes
+from previous versions are located in the :ref:`whats-new-index` chapter. In
+closing, we have :ref:`phk`, a collection of blog posts from Poul-Henning Kamp
+related to Varnish and HTTP.
+
+
+Conventions used
+----------------
+
+Conventions used in this manual include:
+
+  ``service varnish restart``
+    A command you can run, or a shortkey you can press. Used either in the
+    terminal or after starting one of the tools.
+
+  `/usr/local/`, `varnishadm`, `sess_timeout`
+    A utility, Varnish configurable parameter or path.
+
+  http://www.varnish-cache.org/
+    A hyperlink.
+
+Longer listings like example command output and VCL will look like this::
+
+    $ /opt/varnish/sbin/varnishd -V
+    varnishd (varnish-4.0.0-tp1 revision ddd00e1)
+    Copyright (c) 2006 Verdens Gang AS
+    Copyright (c) 2006-2011 Varnish Software AS
+
+
+.. For maintainers:
+.. * always write Varnish with a capital V: Varnish, Varnish Cache.
+.. * Write Varnish tools as their executable name: `varnishd`, `varnishadm`.
+.. * if part of a command actually runable by the reader, use double backticks:
+..   ``varnishd -f foo.c``
+.. * wrap lines at 80 characters, ident with 4 spaces. No tabs, please.
 .. We use the following header indicators
 .. For titles:
 
 .. H1
 .. %%%%%
+
 .. Title
 .. %%%%%
 
 .. H2 - H5
-.. ====================== 
+.. ======================
 .. ----------------------
 .. ~~~~~~~~~~~~~~~~~~~~~~
-.. ...................... 
+.. ......................
 
-Welcome to the Varnish documentation!
-=====================================
-
-Introduction
-------------
-
-Varnish Cache is a web application accelerator also known as a caching HTTP reverse proxy. You install it in front of any server that speaks HTTP and configure it to cache the contents. Varnish Cache is really, really fast. It typically speeds up delivery with a factor of 300 - 1000x, depending on your architecture. 
-It has its mission in front of a
-web server and cache content and it makes your web site go faster.
-
-To get started with Varnish-Cache we recommend that you read the installation guide
-:ref:`install-index`. Once you have Varnish up and running we recommend that you go through
-our tutorial - :ref:`tutorial-index`, and finally the :ref:`users_guide_index`.
-
-If you need to find out how to use a specific Varnish tool, the
-:ref:`reference-index` contains detailed documentation over the tools. Changes from previous versions are located in
-the :ref:`whats-new-index` chapter. In closing, we have :ref:`phk`, a collection
-of blog posts from Poul-Henning Kamp related to Varnish and HTTP.
 
+Table of contents
+-----------------
 
 
 .. toctree::



From lkarsten at varnish-software.com  Tue Apr  1 13:09:52 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:52 +0200
Subject: [4.0] b8f32b9 Squash more warnings
Message-ID: <E1WUyRs-00039Q-4r@project.varnish-software.com>


commit b8f32b911ef11e626ceb116851e66ea8a76a2367
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu Mar 20 15:51:24 2014 +0100

    Squash more warnings

diff --git a/doc/sphinx/index.rst b/doc/sphinx/index.rst
index 11b4cce..db8b5a5 100644
--- a/doc/sphinx/index.rst
+++ b/doc/sphinx/index.rst
@@ -14,7 +14,7 @@ architecture.
 To get started with Varnish-Cache we recommend that you read the installation
 guide :ref:`install-index`. Once you have Varnish up and running we recommend
 that you go through our tutorial - :ref:`tutorial-index`, and finally the
-:ref:`users_guide_index`.
+:ref:`users-guide-index`.
 
 If you need to find out how to use a specific Varnish tool, the
 :ref:`reference-index` contains detailed documentation over the tools. Changes
diff --git a/doc/sphinx/reference/varnishadm.rst b/doc/sphinx/reference/varnishadm.rst
index 5bb9f6e..8143abd 100644
--- a/doc/sphinx/reference/varnishadm.rst
+++ b/doc/sphinx/reference/varnishadm.rst
@@ -1,3 +1,5 @@
+.. _ref-varnishadm:
+
 ==========
 varnishadm
 ==========
diff --git a/doc/sphinx/reference/varnishhist.rst b/doc/sphinx/reference/varnishhist.rst
index 8071384..685aebe 100644
--- a/doc/sphinx/reference/varnishhist.rst
+++ b/doc/sphinx/reference/varnishhist.rst
@@ -1,3 +1,5 @@
+.. _ref-varnishhist:
+
 ===========
 varnishhist
 ===========
diff --git a/doc/sphinx/reference/varnishncsa.rst b/doc/sphinx/reference/varnishncsa.rst
index 2902f6b..77086ac 100644
--- a/doc/sphinx/reference/varnishncsa.rst
+++ b/doc/sphinx/reference/varnishncsa.rst
@@ -1,3 +1,5 @@
+.. _ref-varnishncsa:
+
 ===========
 varnishncsa
 ===========
diff --git a/doc/sphinx/reference/varnishstat.rst b/doc/sphinx/reference/varnishstat.rst
index 3505526..4178778 100644
--- a/doc/sphinx/reference/varnishstat.rst
+++ b/doc/sphinx/reference/varnishstat.rst
@@ -1,4 +1,4 @@
-.. _reference-varnishstat:
+.. _ref-varnishstat:
 
 ===========
 varnishstat
@@ -13,6 +13,8 @@ SYNOPSIS
 
 varnishstat [-1] [-x] [-j] [-f field_list] [-l] [-n varnish_name] [-N filename] [-V] [-w delay]
 
+.. TODO: autogenerate this synopsis like the others.
+
 DESCRIPTION
 ===========
 
diff --git a/doc/sphinx/reference/varnishtest.rst b/doc/sphinx/reference/varnishtest.rst
index a3a0eec..bfae5a5 100644
--- a/doc/sphinx/reference/varnishtest.rst
+++ b/doc/sphinx/reference/varnishtest.rst
@@ -1,3 +1,5 @@
+.. _ref-varnishtest:
+
 ===========
 varnishtest
 ===========
diff --git a/doc/sphinx/reference/varnishtop.rst b/doc/sphinx/reference/varnishtop.rst
index 4692143..e1abbcb 100644
--- a/doc/sphinx/reference/varnishtop.rst
+++ b/doc/sphinx/reference/varnishtop.rst
@@ -1,3 +1,5 @@
+.. _ref-varnishtop:
+
 ==========
 varnishtop
 ==========
diff --git a/doc/sphinx/users-guide/sizing-your-cache.rst b/doc/sphinx/users-guide/sizing-your-cache.rst
index 497ce0e..90621f8 100644
--- a/doc/sphinx/users-guide/sizing-your-cache.rst
+++ b/doc/sphinx/users-guide/sizing-your-cache.rst
@@ -2,26 +2,26 @@
 Sizing your cache
 -----------------
 
-Deciding on cache size can be a tricky
-task. A few things to consider:
+Deciding on cache size can be a tricky task. A few things to consider:
 
  * How big is your *hot* data set. For a portal or news site that
    would be the size of the front page with all the stuff on it, and
-   the size of all the pages and objects linked from the first page. 
+   the size of all the pages and objects linked from the first page.
+
  * How expensive is it to generate an object? Sometimes it makes sense
    to only cache images a little while or not to cache them at all if
    they are cheap to serve from the backend and you have a limited
    amount of memory.
- * Watch the `n_lru_nuked` counter with :ref:`reference-varnishstat` or
+
+ * Watch the `n_lru_nuked` counter with :ref:`ref-varnishstat` or
    some other tool. If you have a lot of LRU activity then your cache
    is evicting objects due to space constraints and you should
    consider increasing the size of the cache.
 
 Be aware that every object that is stored also carries overhead that
-is kept outside the actually storage area. So, even if you specify '-s
-malloc,16G' Varnish might actually use **double** that. Varnish has a
-overhead of about 1k per object. So, if you have lots of small objects
+is kept outside the actually storage area. So, even if you specify ``-s
+malloc,16G`` Varnish might actually use **double** that. Varnish has a
+overhead of about 1KB per object. So, if you have lots of small objects
 in your cache the overhead might be significant.
 
 .. XXX:This seems to contradict the last paragraph in "storage-backends". benc
-
diff --git a/doc/sphinx/users-guide/vcl-backends.rst b/doc/sphinx/users-guide/vcl-backends.rst
index c54d951..77347c4 100644
--- a/doc/sphinx/users-guide/vcl-backends.rst
+++ b/doc/sphinx/users-guide/vcl-backends.rst
@@ -180,34 +180,32 @@ define the backends.::
         }
     }
 
-Whats new here is the ``probe``. Varnish will check the health of each
+What is new here is the ``probe``. Varnish will check the health of each
 backend with a probe. The options are:
 
 url
     The URL Varnish will use to send a probe request.
 
 interval
-    How often should we poll
+    How often should we poll.
 
 timeout
-    What is the timeout of the probe
+    What is the timeout of the probe.
 
 window
-    Varnish will maintain a *sliding window* of the results. Here the
-    window has five checks.
+    Varnish will retain up to this many probes when considering backend health.
 
 threshold
-    How many of the '.window' last polls must be good for the backend to be declared healthy.
-
-.. XXX: .window probably means something but not to me :) benc
+    How many of the `.window` last polls must be good for the backend to be
+    declared healthy.
 
 initial
-    How many of the probes that needs to be succesful when Varnish starts - defaults
-    to the same amount as the threshold.
+    How many of the probes that needs to be succesful when Varnish starts.
+    Defaults to the same amount as the threshold.
 
-Now we define the 'director'.::
 
 .. XXX: Where and why? benc
+Now we define the 'director'::
 
     import directors;
 
@@ -222,10 +220,10 @@ unhealthy. Varnish can also serve stale content if all the backends are
 down. See :ref:`users-guide-handling_misbehaving_servers` for more
 information on how to enable this.
 
-Please note that Varnish will keep probes active for all loaded
-VCLs. Varnish will coalesce probes that seem identical - so be careful
-not to change the probe config if you do a lot of VCL
-loading. Unloading the VCL will discard the probes.
+Please note that Varnish will keep probes active for all loaded VCLs. Varnish
+will coalesce probes that seem identical - so be careful not to change the
+probe config if you do a lot of VCL loading. Unloading the VCL will discard the
+probes.
 
 For more information on how to do this please see
 ref:`reference-vcl-director`.
diff --git a/doc/sphinx/users-guide/vcl.rst b/doc/sphinx/users-guide/vcl.rst
index 8585241..86ed8cf 100644
--- a/doc/sphinx/users-guide/vcl.rst
+++ b/doc/sphinx/users-guide/vcl.rst
@@ -42,8 +42,5 @@ code commented out in the file `builtin.vcl` that ships with Varnish Cache.
    vcl-grace
    vcl-inline-c
    vcl-examples
-   websockets
-.. XXX: websockets seems to be missing? does it refer to the last sample in the vcl index if so already included. benc
-
    devicedetection
 



From lkarsten at varnish-software.com  Tue Apr  1 13:09:52 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:52 +0200
Subject: [4.0] 1309012 Skip generated files, they are included manually.
Message-ID: <E1WUyRs-00039a-60@project.varnish-software.com>


commit 1309012f56d31b29b40961e7183cb4916001cfc2
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu Mar 20 15:52:56 2014 +0100

    Skip generated files, they are included manually.
    
    We include these autogenerated files directly where they are needed.
    
    Let the main sphinx process ignore them, so it doesn't complain that they're
    not referenced anywhere.

diff --git a/doc/sphinx/conf.py.in b/doc/sphinx/conf.py.in
index f68a20f..8104bb3 100644
--- a/doc/sphinx/conf.py.in
+++ b/doc/sphinx/conf.py.in
@@ -64,7 +64,7 @@ release = '@VERSION@'
 
 # List of directories, relative to source directory, that shouldn't be searched
 # for source files.
-exclude_patterns = ['build','reference/params.rst']
+exclude_patterns = ['build','include/*.rst']
 
 # The reST default role (used for this markup: `text`) to use for all documents.
 #default_role = None



From lkarsten at varnish-software.com  Tue Apr  1 13:09:52 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:52 +0200
Subject: [4.0] 9f9d069 Squash more sphinx warnings
Message-ID: <E1WUyRs-00039j-78@project.varnish-software.com>


commit 9f9d0690b37962aa9188f23ef7204568a034b5f3
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu Mar 20 16:06:45 2014 +0100

    Squash more sphinx warnings

diff --git a/doc/sphinx/users-guide/vcl-backends.rst b/doc/sphinx/users-guide/vcl-backends.rst
index 77347c4..a51e4f6 100644
--- a/doc/sphinx/users-guide/vcl-backends.rst
+++ b/doc/sphinx/users-guide/vcl-backends.rst
@@ -205,6 +205,7 @@ initial
 
 
 .. XXX: Where and why? benc
+
 Now we define the 'director'::
 
     import directors;
@@ -214,6 +215,7 @@ Now we define the 'director'::
         { .backend = server2; }
     }
 
+
 You use this director just as you would use any other director or
 backend. Varnish will not send traffic to hosts that are marked as
 unhealthy. Varnish can also serve stale content if all the backends are
diff --git a/doc/sphinx/users-guide/vcl-syntax.rst b/doc/sphinx/users-guide/vcl-syntax.rst
index f0b427f..d4acdd7 100644
--- a/doc/sphinx/users-guide/vcl-syntax.rst
+++ b/doc/sphinx/users-guide/vcl-syntax.rst
@@ -98,8 +98,7 @@ To call a subroutine, use the call keyword followed by the subroutine's name::
   call pipe_if_local;
 
 Varnish has quite a few built in subroutines that are called for each
-transaction as it flows through Varnish. These builtin subroutines are all named vcl_*. Your own subroutines cannot start their name with vcl_.
+transaction as it flows through Varnish. These builtin subroutines are all
+named `vcl_*`. Your own subroutines cannot start their name with `vcl\_`.
 
 .. XXX:looks as bit funky as red text? benc
-
-See :ref:`vcl-built-in-subs`.



From lkarsten at varnish-software.com  Tue Apr  1 13:09:52 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:52 +0200
Subject: [4.0] b109eed Formatting, typo
Message-ID: <E1WUyRs-00039s-B1@project.varnish-software.com>


commit b109eed58c17ee0dadea7d82847945bf1137eb2f
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu Mar 20 16:22:23 2014 +0100

    Formatting, typo

diff --git a/doc/sphinx/tutorial/starting_varnish.rst b/doc/sphinx/tutorial/starting_varnish.rst
index 2896394..969e07e 100644
--- a/doc/sphinx/tutorial/starting_varnish.rst
+++ b/doc/sphinx/tutorial/starting_varnish.rst
@@ -5,12 +5,13 @@ Starting Varnish
 ----------------
 
 This tutorial will assume that you are running Varnish on Ubuntu, Debian,
-Red Hat Enterprise Linux or Centos. Those of you running on other
+Red Hat Enterprise Linux or CentOS. Those of you running on other
 platforms might have to do some mental translation exercises in order
 to follow this. Since you're on a "weird" platform you're probably used
 to it. :-)
 
-Make sure you have Varnish succesfully installed (following one of the procedures described in "Installing Varnish" above.
+Make sure you have Varnish successfully installed (following one of the
+procedures described in "Installing Varnish" above.
 
 When properly installed you start Varnish with ``service varnish start``.  This will start
 Varnish if it isn't already running.
@@ -18,15 +19,16 @@ Varnish if it isn't already running.
 .. XXX:What does it do if it is already running? benc
 
 Now you have Varnish running. Let us make sure that it works
-properly. Use your browser to go to http://127.0.0.1:6081/
-(obviously, you should replace the IP address with the IP for the machine that currently runs Varnish. 
-The default configuration will try to forward requests to a web
-application running on the same machine as Varnish was installed on. Varnish expects the web application to be exposed over http on port 8080.
+properly. Use your browser to go to http://127.0.0.1:6081/ (Replace the IP
+address with the IP for the machine that runs Varnish) The default
+configuration will try to forward requests to a web application running on the
+same machine as Varnish was installed on. Varnish expects the web application
+to be exposed over http on port 8080.
 
 If there is no web application being served up on that location Varnish will
 issue an error. Varnish Cache is very conservative about telling the
 world what is wrong so whenever something is amiss it will issue the
-same generic "Error 503 Service Unavilable".
+same generic "Error 503 Service Unavailable".
 
 You might have a web application running on some other port or some
 other machine. Let's edit the configuration and make it point to
@@ -44,7 +46,7 @@ probably look like this::
   }
 
 We'll change it and make it point to something that works. Hopefully
-www.varnish-cache.org is up. Let's use that. Replace the text with::
+http://www.varnish-cache.org/ is up. Let's use that. Replace the text with::
 
   vcl 4.0;
   
@@ -58,6 +60,6 @@ Now issue ``service varnish reload`` to make Varnish reload it's
 configuration. If that succeeded visit http://127.0.0.1:6081/ in your
 browser and you should see some directory listing. It works! The
 reason you're not seeing the Varnish official website is because your
-client isn't sending the appropriate Host: header in the request and
+client isn't sending the appropriate `Host` header in the request and
 it ends up showing a listing of the default webfolder on the machine
-usually serving up varnish-cache.org.
+usually serving up http://www.varnish-cache.org/ .



From lkarsten at varnish-software.com  Tue Apr  1 13:09:52 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:52 +0200
Subject: [4.0] 31a4724 Formatting cleanup
Message-ID: <E1WUyRs-0003A1-C8@project.varnish-software.com>


commit 31a472467f331836d82b60e3fd32a7bc6f3ac330
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu Mar 20 16:29:34 2014 +0100

    Formatting cleanup

diff --git a/doc/sphinx/installation/install.rst b/doc/sphinx/installation/install.rst
index a407624..f95f4b4 100644
--- a/doc/sphinx/installation/install.rst
+++ b/doc/sphinx/installation/install.rst
@@ -25,12 +25,12 @@ Binary package:
 From source:
 		``cd /usr/ports/varnish && make install clean``
 
-CentOS/RedHat
--------------
+Red Hat / CentOS
+----------------
 
 We try to keep the latest version available as prebuilt RPMs (el5 and el6)
 on `repo.varnish-cache.org <http://repo.varnish-cache.org/>`_.  See the online
-`RedHat installation instructions
+`Red Hat installation instructions
 <http://www.varnish-cache.org/installation/redhat>`_ for more information.
 
 Varnish is included in the `EPEL
@@ -92,10 +92,10 @@ If you're building from git, you also need the following:
 * `python-docutils`
 * `python-sphinx` (optional, if you want to build the documentation)
 
-Build dependencies on RedHat / CentOS
+Build dependencies on Red Hat / CentOS
 --------------------------------------
 
-To build Varnish on a RedHat or CentOS system you need the following
+To build Varnish on a Red Hat or CentOS system you need the following
 packages installed:
 
 * `automake`
diff --git a/doc/sphinx/tutorial/putting_varnish_on_port_80.rst b/doc/sphinx/tutorial/putting_varnish_on_port_80.rst
index 0fc2ddc..3e57917 100644
--- a/doc/sphinx/tutorial/putting_varnish_on_port_80.rst
+++ b/doc/sphinx/tutorial/putting_varnish_on_port_80.rst
@@ -2,17 +2,12 @@
 Put Varnish on port 80
 ----------------------
 
-Until now we've been running with Varnish on a high port, which is
-great for testing purposes. Lets put Varnish on the default HTTP port.
+Until now we've been running with Varnish on a high port which is great for
+testing purposes. Let's now put Varnish on the default HTTP port 80.
 
-First we stop varnish::
-
-     # service varnish stop
-
-.. XXX:This renders to a different font than other commands. it should be the double backtick format for the command. benc
-
-Now we need to edit the configuration file that starts Varnish. 
+First we stop varnish: ``service varnish stop``
 
+Now we need to edit the configuration file that starts Varnish.
 
 Debian/Ubuntu
 ~~~~~~~~~~~~~
@@ -34,17 +29,17 @@ Change it to::
                -S /etc/varnish/secret \
                -s malloc,256m"
 
-Red Hat EL / Centos
-~~~~~~~~~~~~~~~~~~~
+Red Hat Enterprise Linux / CentOS
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+On Red Hat/CentOS you can find a similar configuration file in
+`/etc/sysconfig/varnish`.
 
-On Red Hat EL / Centos
-On Red Hat/Centos it is `/etc/sysconfig/varnish`
 
+Restarting Varnish again
+------------------------
 
-Restarting Varnish
-------------------
+Once the change is done, restart Varnish: ``service varnish start``.
 
-Once the change is done. Restart varnish: ``service varnish
-restart``. Now everyone accessing your site will be accessing through
-Varnish.
+Now everyone accessing your site will be accessing through Varnish.
 



From lkarsten at varnish-software.com  Tue Apr  1 13:09:52 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:52 +0200
Subject: [4.0] c510ea2 Endless reformatting
Message-ID: <E1WUyRs-0003AA-DG@project.varnish-software.com>


commit c510ea2194d136c3f8bed3d9b359d9707af0f7d3
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu Mar 20 16:35:27 2014 +0100

    Endless reformatting

diff --git a/doc/sphinx/tutorial/introduction.rst b/doc/sphinx/tutorial/introduction.rst
index 98cfddc..2fbc43a 100644
--- a/doc/sphinx/tutorial/introduction.rst
+++ b/doc/sphinx/tutorial/introduction.rst
@@ -9,18 +9,20 @@ the request from the cache it will forward the request to the backend,
 fetch the response, store it in the cache and deliver it to the client.
 
 When Varnish has a cached response ready it is typically delivered in
-a matter of microseconds, several orders of magnitude faster than your
+a matter of microseconds, two orders of magnitude faster than your
 typical backend server, so you want to make sure to have Varnish answer
 as many of the requests as possible directly from the cache.
 
-Varnish decides whether it can store the content or not based on the
-response it gets back from the backend. The backend can instruct
-Varnish to cache the content with the HTTP response header
-`Cache-Control`. There are a few conditions where Varnish will not
-cache, the most common one being the use of cookies. Since cookies indicates a client-specific web object, Varnish will by default not cache it. 
+Varnish decides whether it can store the content or not based on the response
+it gets back from the backend. The backend can instruct Varnish to cache the
+content with the HTTP response header `Cache-Control`. There are a few
+conditions where Varnish will not cache, the most common one being the use of
+cookies. Since cookies indicates a client-specific web object, Varnish will by
+default not cache it.
+
 This behaviour as most of Varnish functionality can be changed using policies
-written in the Varnish Configuration Language. See the Users Guide
-for more information on how to do that.
+written in the Varnish Configuration Language (VCL). See
+:ref:`users-guide-index` for more information on how to do that.
 
 Performance
 ~~~~~~~~~~~
diff --git a/doc/sphinx/tutorial/starting_varnish.rst b/doc/sphinx/tutorial/starting_varnish.rst
index 969e07e..8d9b64c 100644
--- a/doc/sphinx/tutorial/starting_varnish.rst
+++ b/doc/sphinx/tutorial/starting_varnish.rst
@@ -13,8 +13,8 @@ to it. :-)
 Make sure you have Varnish successfully installed (following one of the
 procedures described in "Installing Varnish" above.
 
-When properly installed you start Varnish with ``service varnish start``.  This will start
-Varnish if it isn't already running.
+When properly installed you start Varnish with ``service varnish start``.  This
+will start Varnish if it isn't already running.
 
 .. XXX:What does it do if it is already running? benc
 



From lkarsten at varnish-software.com  Tue Apr  1 13:09:52 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:52 +0200
Subject: [4.0] af74f47 WIP, redoing structure
Message-ID: <E1WUyRs-0003AJ-EH@project.varnish-software.com>


commit af74f47154126408ad463ed9b004ab9302566b7f
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu Mar 20 17:18:32 2014 +0100

    WIP, redoing structure

diff --git a/doc/sphinx/installation/install.rst b/doc/sphinx/installation/install.rst
index f95f4b4..a8ef31c 100644
--- a/doc/sphinx/installation/install.rst
+++ b/doc/sphinx/installation/install.rst
@@ -3,11 +3,13 @@
 Installing Varnish
 ==================
 
-With open source software, you can choose to install binary packages
-or compile it yourself from source code. To install a package or compile
-from source is a matter of personal taste. If you don't know which
-method to choose, we recommend that you read this whole section and then choose the method you
-feel most comfortable with.
+.. no section heading here.
+
+With open source software, you can choose to install binary packages or compile
+it yourself from source code. To install a package or compile from source is a
+matter of personal taste. If you don't know which method to choose, we
+recommend that you read this whole section and then choose the method you feel
+most comfortable with.
 
 
 Source or packages?
@@ -60,15 +62,13 @@ want to compile Varnish from source for other reasons, follow these
 steps:
 
 Download the appropriate release tarball, which you can find on
-`repo.varnish-cache.org/source <http://repo.varnish-cache.org/source/>`_.
+http://repo.varnish-cache.org/source/ .
 
 Alternatively, if you want to hack on Varnish, you should clone our
 git repository by doing.
 
       ``git clone git://git.varnish-cache.org/varnish-cache``
 
-Please note that a git checkout will need some more build-dependencies
-than listed below, in particular the `Python Docutis` and `Sphinx`.
 
 Build dependencies on Debian / Ubuntu
 --------------------------------------
@@ -120,8 +120,9 @@ taken care of::
 	sh configure
 	make
 
-The `configure` script takes some arguments, but more likely than
-not you can forget about that for now, almost everything in Varnish can be tweaked with run time parameters.
+The `configure` script takes some arguments, but more likely than not you can
+forget about that for now, almost everything in Varnish can be tweaked with run
+time parameters.
 
 Before you install, you may want to run the test suite, make a cup of
 tea while it runs, it usually takes a couple of minutes::
@@ -139,9 +140,9 @@ Installing
 
 And finally, the true test of a brave heart::
 
-	sudo make install
+And finally, the true test of a brave heart: ``sudo make install``
 
-Varnish will now be installed in `/usr/local`. The ``varnishd`` binary is in
+Varnish will now be installed in `/usr/local`. The `varnishd` binary is in
 `/usr/local/sbin/varnishd` and its default configuration will be
 `/usr/local/etc/varnish/default.vcl`.
 



From lkarsten at varnish-software.com  Tue Apr  1 13:09:52 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:52 +0200
Subject: [4.0] cfb7fd2 C-c: nocache is honored in 4.0
Message-ID: <E1WUyRs-0003AS-FC@project.varnish-software.com>


commit cfb7fd2a30a8b1bb7931ec25ab213d8179ec00d0
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu Mar 20 17:35:02 2014 +0100

    C-c: nocache is honored in 4.0

diff --git a/doc/sphinx/users-guide/increasing-your-hitrate.rst b/doc/sphinx/users-guide/increasing-your-hitrate.rst
index 4be147f..886659e 100644
--- a/doc/sphinx/users-guide/increasing-your-hitrate.rst
+++ b/doc/sphinx/users-guide/increasing-your-hitrate.rst
@@ -222,10 +222,7 @@ Cache-Control
 
 The 'Cache-Control' header instructs caches how to handle the content. Varnish
 cares about the *max-age* parameter and uses it to calculate the TTL
-for an object. 
-
-``Cache-Control: nocache`` is ignored but if you need this you can
-easily add support for it.
+for an object.
 
 So make sure you issue a 'Cache-Control' header with a max-age
 header. You can have a look at what Varnish Software's Drupal server



From lkarsten at varnish-software.com  Tue Apr  1 13:09:52 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:52 +0200
Subject: [4.0] 9d5e807 Formatting
Message-ID: <E1WUyRs-0003Ab-GF@project.varnish-software.com>


commit 9d5e80703499aa0d175b937983ba0a3fa9fa1cd1
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu Mar 20 17:35:13 2014 +0100

    Formatting

diff --git a/doc/sphinx/users-guide/increasing-your-hitrate.rst b/doc/sphinx/users-guide/increasing-your-hitrate.rst
index 886659e..9d8e0af 100644
--- a/doc/sphinx/users-guide/increasing-your-hitrate.rst
+++ b/doc/sphinx/users-guide/increasing-your-hitrate.rst
@@ -235,9 +235,7 @@ Age
 ~~~
 
 Varnish adds an 'Age' header to indicate how long the object has been
-kept inside Varnish. You can grep out 'Age' from `varnishlog` like this::
-
-  varnishlog -i TxHeader -I ^Age
+kept inside Varnish. You can grep out 'Age' from `varnishlog` with ``varnishlog -i TxHeader -I ^Age``.
 
 Pragma
 ~~~~~~
@@ -247,10 +245,10 @@ header. You could easily add support for this header in VCL.
 
 In `vcl_backend_response`::
 
-  if (beresp.http.Pragma ~ "nocache") {
+    if (beresp.http.Pragma ~ "nocache") {
         set beresp.uncacheable = true;
-	set beresp.ttl = 120s; // how long not to cache this url.
-  }
+        set beresp.ttl = 120s; // how long not to cache this url.
+    }
 
 Authorization
 ~~~~~~~~~~~~~
@@ -258,12 +256,12 @@ Authorization
 If Varnish sees an 'Authorization' header it will pass the request. If
 this is not what you want you can unset the header.
 
-Overriding the time-to-live (ttl)
+Overriding the time-to-live (TTL)
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 Sometimes your backend will misbehave. It might, depending on your
-setup, be easier to override the ttl in Varnish than to fix your
-somewhat cumbersome backend. 
+setup, be easier to override the TTL in Varnish then to fix your
+somewhat cumbersome backend.
 
 You need VCL to identify the objects you want and then you set the
 'beresp.ttl' to whatever you want::
@@ -274,7 +272,7 @@ You need VCL to identify the objects you want and then you set the
       }
   }
 
-This example will set the ttl to 5 days for the old legacy stuff on
+This example will set the TTL to 5 days for the old legacy stuff on
 your site.
 
 Forcing caching for certain requests and certain responses



From lkarsten at varnish-software.com  Tue Apr  1 13:09:52 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:52 +0200
Subject: [4.0] 0ab1908 Redo heading structure a bit
Message-ID: <E1WUyRs-0003Ak-Hd@project.varnish-software.com>


commit 0ab19082d27e4ce559c4d17c8daf85dc0b2658c2
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu Mar 20 17:36:00 2014 +0100

    Redo heading structure a bit

diff --git a/doc/sphinx/installation/index.rst b/doc/sphinx/installation/index.rst
index 03e60b8..f758d84 100644
--- a/doc/sphinx/installation/index.rst
+++ b/doc/sphinx/installation/index.rst
@@ -1,8 +1,7 @@
 .. _install-index:
 
-%%%%%%%%%%%%%%%%%%%%
 Varnish Installation
-%%%%%%%%%%%%%%%%%%%%
+====================
 
 This section covers installation prerequisites, a step-by-step installation procedure, how and where
 to get help, and how to report bugs. It also contains a set of platform specific notes to aid you when installing Varnish on certain platforms.
diff --git a/doc/sphinx/phk/index.rst b/doc/sphinx/phk/index.rst
index 58da708..7eac4e5 100644
--- a/doc/sphinx/phk/index.rst
+++ b/doc/sphinx/phk/index.rst
@@ -1,8 +1,7 @@
 .. _phk:
 
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 Poul-Hennings random outbursts
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+==============================
 
 You may or may not want to know what Poul-Henning thinks.
 
diff --git a/doc/sphinx/tutorial/index.rst b/doc/sphinx/tutorial/index.rst
index 44008e7..edac3c9 100644
--- a/doc/sphinx/tutorial/index.rst
+++ b/doc/sphinx/tutorial/index.rst
@@ -1,8 +1,7 @@
 .. _tutorial-index:
 
-%%%%%%%%%%%%%%%%%%%%
 The Varnish Tutorial
-%%%%%%%%%%%%%%%%%%%%
+====================
 
 This section covers the Varnish basics in a tutorial form. It will cover what Varnish is and how it
 works. It also covers how to get Varnish up and running. After this section you probably would want to
diff --git a/doc/sphinx/users-guide/index.rst b/doc/sphinx/users-guide/index.rst
index bbc7d6a..e1c3e83 100644
--- a/doc/sphinx/users-guide/index.rst
+++ b/doc/sphinx/users-guide/index.rst
@@ -1,8 +1,7 @@
 .. _users-guide-index:
 
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 The Varnish Users Guide
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+=======================
 
 The Varnish documentation consists of three main documents:
 



From lkarsten at varnish-software.com  Tue Apr  1 13:09:52 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:52 +0200
Subject: [4.0] 830151a Adhere to style manual
Message-ID: <E1WUyRs-0003At-Ii@project.varnish-software.com>


commit 830151a85ef4a65b9278c6005c6834257151aa21
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu Mar 20 17:36:28 2014 +0100

    Adhere to style manual

diff --git a/doc/sphinx/installation/platformnotes.rst b/doc/sphinx/installation/platformnotes.rst
index 7f11eaa..0eca96f 100644
--- a/doc/sphinx/installation/platformnotes.rst
+++ b/doc/sphinx/installation/platformnotes.rst
@@ -12,9 +12,10 @@ Transparent hugepages on Redhat Enterprise Linux 6
 On RHEL6 Transparent Hugepage kernel support is enabled by default.
 This is known to cause sporadic crashes of Varnish.
 
-It is recommended to disable transparent hugepages on affected systems::
-
-    $ echo "never" > /sys/kernel/mm/redhat_transparent_hugepage/enabled
+It is recommended to disable transparent hugepages on affected systems. This
+can be done with 
+``echo "never" > /sys/kernel/mm/redhat_transparent_hugepage/enabled`` (runtime) and changes to
+`/etc/sysctl.conf` (persisted.)
 
 On Debian/Ubuntu systems running 3.2 kernels the default value is "madvise" and
 does not need to be changed.
@@ -34,11 +35,8 @@ the maximum stack size before starting Varnish.
 The default allocates too much memory per thread, which will make Varnish fail
 as soon as the number of threads (traffic) increases.
 
-Reduce the maximum stack size by running::
-
-    ulimit -s 256
-
-in the Varnish startup script.
+Reduce the maximum stack size by adding ``ulimit -s 256`` before starting
+Varnish in the init script.
 
 TCP keep-alive configuration
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
diff --git a/doc/sphinx/installation/prerequisites.rst b/doc/sphinx/installation/prerequisites.rst
index 7ef0bd7..05aea31 100644
--- a/doc/sphinx/installation/prerequisites.rst
+++ b/doc/sphinx/installation/prerequisites.rst
@@ -1,7 +1,6 @@
 Prerequisites
 =============
 
-
 In order for you to install Varnish you must have the following:
 
   * A recent, preferably server grade, computer.



From lkarsten at varnish-software.com  Tue Apr  1 13:09:52 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:52 +0200
Subject: [4.0] c4ee24f TOC looks OK in HTML and latex now
Message-ID: <E1WUyRs-0003B3-Jv@project.varnish-software.com>


commit c4ee24f8b09750cad8975af67ee04b695ae420c5
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu Mar 20 17:37:10 2014 +0100

    TOC looks OK in HTML and latex now

diff --git a/doc/sphinx/index.rst b/doc/sphinx/index.rst
index db8b5a5..b87a1a6 100644
--- a/doc/sphinx/index.rst
+++ b/doc/sphinx/index.rst
@@ -1,8 +1,6 @@
-Welcome to the Varnish documentation!
-=====================================
 
-Introduction
-------------
+Varnish Administrator Documentation
+===================================
 
 Varnish Cache is a web application accelerator also known as a caching HTTP
 reverse proxy. You install it in front of any server that speaks HTTP and
@@ -10,7 +8,6 @@ configure it to cache the contents. Varnish Cache is really, really fast. It
 typically speeds up delivery with a factor of 300 - 1000x, depending on your
 architecture.
 
-
 To get started with Varnish-Cache we recommend that you read the installation
 guide :ref:`install-index`. Once you have Varnish up and running we recommend
 that you go through our tutorial - :ref:`tutorial-index`, and finally the
@@ -23,9 +20,6 @@ closing, we have :ref:`phk`, a collection of blog posts from Poul-Henning Kamp
 related to Varnish and HTTP.
 
 
-Conventions used
-----------------
-
 Conventions used in this manual include:
 
   ``service varnish restart``
@@ -68,23 +62,21 @@ Longer listings like example command output and VCL will look like this::
 .. ......................
 
 
-Table of contents
------------------
+.. toctree::
+    :maxdepth: 1
 
+    installation/index.rst
+    tutorial/index.rst
+    users-guide/index.rst
+    reference/index.rst
+    whats-new/index.rst
+    phk/index.rst
+    glossary/index.rst
 
-.. toctree::
-   :maxdepth: 1
 
-   installation/index.rst
-   tutorial/index.rst
-   users-guide/index.rst
-   reference/index.rst
-   whats-new/index.rst
-   phk/index.rst
-   glossary/index.rst
 
 Indices and tables
-==================
+------------------
 
 * :ref:`genindex`
 * :ref:`search`



From tfheen at err.no  Tue Apr  1 13:09:52 2014
From: tfheen at err.no (Tollef Fog Heen)
Date: Tue, 01 Apr 2014 15:09:52 +0200
Subject: [4.0] 02e38f7 Fix two small resource leaks to make coverity happy
Message-ID: <E1WUyRs-0003BC-L9@project.varnish-software.com>


commit 02e38f7f26e35a23faf4356ab6b7d3bae264cffa
Author: Tollef Fog Heen <tfheen at fastly.com>
Date:   Thu Mar 20 20:05:41 2014 +0100

    Fix two small resource leaks to make coverity happy

diff --git a/bin/varnishadm/varnishadm.c b/bin/varnishadm/varnishadm.c
index e1686ad..90016a9 100644
--- a/bin/varnishadm/varnishadm.c
+++ b/bin/varnishadm/varnishadm.c
@@ -128,6 +128,7 @@ cli_sock(const char *T_arg, const char *S_arg)
 	if (status != CLIS_OK) {
 		fprintf(stderr, "Rejected %u\n%s\n", status, answer);
 		AZ(close(sock));
+		free(answer);
 		return (-1);
 	}
 	free(answer);
@@ -137,7 +138,8 @@ cli_sock(const char *T_arg, const char *S_arg)
 	if (status != CLIS_OK || strstr(answer, "PONG") == NULL) {
 		fprintf(stderr, "No pong received from server\n");
 		AZ(close(sock));
-		return(-1);
+		free(answer);
+		return (-1);
 	}
 	free(answer);
 



From tfheen at err.no  Tue Apr  1 13:09:52 2014
From: tfheen at err.no (Tollef Fog Heen)
Date: Tue, 01 Apr 2014 15:09:52 +0200
Subject: [4.0] 7b48fae Move freeing of topbuild later to prevent use-after-free
Message-ID: <E1WUyRs-0003BU-NC@project.varnish-software.com>


commit 7b48faecf3c71d1ed8f630878425f0e1f1d4db92
Author: Tollef Fog Heen <tfheen at fastly.com>
Date:   Fri Mar 21 08:12:25 2014 +0100

    Move freeing of topbuild later to prevent use-after-free

diff --git a/bin/varnishtest/vtc_main.c b/bin/varnishtest/vtc_main.c
index fc3b588..de08003 100644
--- a/bin/varnishtest/vtc_main.c
+++ b/bin/varnishtest/vtc_main.c
@@ -378,7 +378,6 @@ i_mode(void)
 	}
 	AN(topbuild);
 	extmacro_def("topbuild", "%s", topbuild);
-	free(topbuild);
 	/*
 	 * Build $PATH which can find all programs in the build tree
 	 */
@@ -411,6 +410,7 @@ i_mode(void)
 	} while (0);
 #include "vmods.h"
 #undef VTC_VMOD
+	free(topbuild);
 	VSB_delete(vsb);
 }
 



From tfheen at err.no  Tue Apr  1 13:09:52 2014
From: tfheen at err.no (Tollef Fog Heen)
Date: Tue, 01 Apr 2014 15:09:52 +0200
Subject: [4.0] 597f4e9 Free topbuild, makes coverity happy
Message-ID: <E1WUyRs-0003BL-MA@project.varnish-software.com>


commit 597f4e909b85a0ff7a988f6646fc3afe76d0e603
Author: Tollef Fog Heen <tfheen at fastly.com>
Date:   Thu Mar 20 20:09:26 2014 +0100

    Free topbuild, makes coverity happy

diff --git a/bin/varnishtest/vtc_main.c b/bin/varnishtest/vtc_main.c
index 3fc2b43..fc3b588 100644
--- a/bin/varnishtest/vtc_main.c
+++ b/bin/varnishtest/vtc_main.c
@@ -378,7 +378,7 @@ i_mode(void)
 	}
 	AN(topbuild);
 	extmacro_def("topbuild", "%s", topbuild);
-
+	free(topbuild);
 	/*
 	 * Build $PATH which can find all programs in the build tree
 	 */



From lkarsten at varnish-software.com  Tue Apr  1 13:09:52 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:52 +0200
Subject: [4.0] 8f176df Use system Python3 interpreter
Message-ID: <E1WUyRs-0003Bd-OQ@project.varnish-software.com>


commit 8f176df35efa4f1e04f4866ec0543719c4aeb429
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Fri Mar 21 14:18:34 2014 +0100

    Use system Python3 interpreter
    
    I don't have Python 3.1, but I do have Python 3.3.5 which should suffice.

diff --git a/lib/libvcc/generate.py b/lib/libvcc/generate.py
index 7c7e9db..84e363d 100755
--- a/lib/libvcc/generate.py
+++ b/lib/libvcc/generate.py
@@ -1,4 +1,4 @@
-#!/usr/local/bin/python3.1
+#!/usr/bin/env python3
 #-
 # Copyright (c) 2006 Verdens Gang AS
 # Copyright (c) 2006-2014 Varnish Software AS



From lkarsten at varnish-software.com  Tue Apr  1 13:09:52 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:52 +0200
Subject: [4.0] 2ebf017 Clean up for 4.0 syntax.
Message-ID: <E1WUyRs-0003Bn-Pf@project.varnish-software.com>


commit 2ebf017d5af8104e59b8876f3b3bf720c44da4bb
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Fri Mar 21 14:43:48 2014 +0100

    Clean up for 4.0 syntax.

diff --git a/doc/sphinx/reference/vcl.rst b/doc/sphinx/reference/vcl.rst
index 942d1ac..c71577b 100644
--- a/doc/sphinx/reference/vcl.rst
+++ b/doc/sphinx/reference/vcl.rst
@@ -12,36 +12,19 @@ DESCRIPTION
 ===========
 
 The VCL language is a small domain-specific language designed to be
-used to define request handling and document caching policies for
+used to describe request handling and document caching policies for
 Varnish Cache.
 
 When a new configuration is loaded, the varnishd management process
 translates the VCL code to C and compiles it to a shared object which
-is then dynamically linked into the server process.
+is then loaded into the server process.
 
 This document focuses on the syntax of the VCL language. For a full
 description of syntax and semantics, with ample examples, please see
-the users guide at https://www.varnish-cache.org/docs/
-
-VCL consists of the following elements:
- * Operators
- * Conditionals
- * Strings, booleans, time, duration, ints
- * Regular expressions
-
-In addition VCL has the following constructs:
- * Include
- * Backend definitions
- * Probes
- * Access control lists - ACLs
- * Import statement
- * Functions
- * Subroutines
+the online documentation at https://www.varnish-cache.org/docs/ .
 
-Note that are no loops or iterators of any kind in VCL.
-
-Each VCL file must start by declaring its version. For Varnish 4.0 you
-must include a "vcl 4.0;" at the top of the file.
+Starting with Varnish 4.0, each VCL file must start by declaring its version
+with a special "vcl 4.0;" marker at the top of the file.
 
 
 Operators
@@ -62,35 +45,39 @@ The following operators are available in VCL:
     Negation.
 
   &&
-    Logical and
+    Logical and.
 
   ||
-    Logical or
+    Logical or.
 
 
 Conditionals
 ------------
 
-VCL has *if* statements.
+VCL has *if* and *else* statements. Nested logic can be implemented
+with the *elseif* statement. (*elsif*/*elif*/*else if* is equivalent.)
 
+Note that are no loops or iterators of any kind in VCL.
 
-Strings, booleans, time, duration and ints
-------------------------------------------
+
+Strings, booleans, time, duration and integers
+----------------------------------------------
 
 These are the data types in Varnish. You can *set* or *unset* these.
 
 Example::
 
-  set req.http.user-agent = "unknown";
+  set req.http.User-Agent = "unknown";
+  unset req.http.Range;
 
 
 Strings
 ~~~~~~~
 
-Basic strings are enclosed in " ... ", and may not contain
+Basic strings are enclosed in double quotes (" ... "), and may not contain
 newlines. Long strings are enclosed in {" ... "}. They may contain any
-character including ", newline and other control characters except for
-the NUL (0x00) character
+character including single double quotes ("), newline and other control
+characters except for the NUL (0x00) character.
 
 Booleans
 ~~~~~~~~
@@ -107,7 +94,8 @@ formatted string.
 Durations
 ---------
 
-Durations are defined by a number and a designation. The number can be a real so 1.5w is allowed.
+Durations are defined by a number and a designation. The number can be a real
+so 1.5w is allowed.
 
   ms
     milliseconds
@@ -128,14 +116,14 @@ Durations are defined by a number and a designation. The number can be a real so
     weeks
 
 
-Ints
-----
+Integers
+--------
 
 Certain fields are integers, used as expected. In string context they
 return a string.
 
-Reals
------
+Real numbers
+------------
 
 VCL understands real numbers. As with integers, when used in a string
 context they will return a string.
@@ -144,12 +132,11 @@ context they will return a string.
 Regular Expressions
 -------------------
 
-Varnish uses PCRE - Perl-compatible regular expressions. For a
-complete description of PCRE please see the pcre(3) man page.
+Varnish uses Perl-compatible regular expressions (PCRE). For a
+complete description please see the pcre(3) man page.
 
-To send flags to the PCRE engine, such as to turn on *case insensitivity* 
-add the flag within parens following a question mark,
-like this::
+To send flags to the PCRE engine, such as to do case insensitive matching, add
+the flag within parens following a question mark, like this::
 
     # If host is NOT example dot com..
     if (req.http.host !~ "(?i)example.com$") {
@@ -162,22 +149,35 @@ Include statement
 
 To include a VCL file in another file use the include keyword::
 
-  include "foo.vcl";
+    include "foo.vcl";
+
+
+Import statement
+----------------
+
+The *import* statement is used to load Varnish Modules (VMODs.)
+
+Example::
+
+    import std;
+    sub vcl_recv {
+        std.log("foo");
+    }
+
 
 
 Backend definition
 ------------------
 
-A backend declaration creates and initializes a named backend
-object. A declaration start with the keyword *backend* followed by the
-name of the backend. The actual declaration is in curly brackets, in a
-key/value fashion.::
+A backend declaration creates and initialises a named backend object. A
+declaration start with the keyword *backend* followed by the name of the
+backend. The actual declaration is in curly brackets, in a key/value fashion.::
 
     backend name {
         .attribute = "value";
     }
 
-The only mandatory attribute is host. The attributes will inherit
+The only mandatory attribute is *host*. The attributes will inherit
 their defaults from the global parameters. The following attributes
 are available:
 
@@ -233,9 +233,9 @@ There are no mandatory options. These are the options you can set:
 
   expected_response
     The expected HTTP response code. Defaults to 200.
-            
+
   timeout
-    The timeout for the probe. Default it 2s.
+    The timeout for the probe. Default is 2s.
 
   interval
     How often the probe is run. Default is 5s.
@@ -245,10 +245,11 @@ There are no mandatory options. These are the options you can set:
     starts. Defaults to the value of threshold - 1. In this case, the
     backend starts as sick and requires one single poll to be
     considered healthy.
-            
+
   window
-    How many of the latest polls we examine to determine backend health. Defaults to 8.
-            
+    How many of the latest polls we examine to determine backend health.
+    Defaults to 8.
+
   threshold
     How many of the polls in .window must have succeeded for us to
     consider the backend healthy. If this is set to more than or equal
@@ -258,68 +259,82 @@ There are no mandatory options. These are the options you can set:
     threshold - 1.
 
 
-ACLs
-----
+Access Control List (ACL)
+-------------------------
 
-An ACL declaration creates and initializes a named access control list
-which can later be used to match client addresses::
+An Access Control List (ACL) declaration creates and initialises a named access
+control list which can later be used to match client addresses::
 
-    acl local {
+    acl localnetwork {
         "localhost";    # myself
         "192.0.2.0"/24; # and everyone on the local network
-        ! "192.0.2.23"; # except for the dialin router
+        ! "192.0.2.23"; # except for the dial-in router
     }
 
 If an ACL entry specifies a host name which Varnish is unable to
-resolve, it will match any address it is compared to.  Consequently,
+resolve, it will match any address it is compared to. Consequently,
 if it is preceded by a negation mark, it will reject any address it is
-compared to, which may not be what you intended.  If the entry is
+compared to, which may not be what you intended. If the entry is
 enclosed in parentheses, however, it will simply be ignored.
 
 To match an IP address against an ACL, simply use the match operator::
 
-    if (client.ip ~ local) {
+    if (client.ip ~ localnetwork) {
         return (pipe);
     }
 
 
+VCL objects
+-----------
+
+A VCL object can be made with the *new* keyword.
+
+Example::
+
+    sub vcl_init {
+        new b = directors.round_robin()
+        b.add_backend(node1);
+    }
+
+
 Subroutines
 -----------
 
 A subroutine is used to group code for legibility or reusability::
 
     sub pipe_if_local {
-        if (client.ip ~ local) {
+        if (client.ip ~ localnetwork) {
             return (pipe);
         }
     }
 
 Subroutines in VCL do not take arguments, nor do they return
-values. The built in subroutines all have names beginning with vcl_,
+values. The built in subroutines all have names beginning with vcl\_,
 which is reserved.
 
 To call a subroutine, use the call keyword followed by the subroutine's name::
 
-    call pipe_if_local;
+    sub vcl_recv {
+        call pipe_if_local;
+    }
 
 Return statements
 ~~~~~~~~~~~~~~~~~
 
-The subroutine executions ends when a return(*action*) statement is
-made. The *action* specifies how execution should proceed. The context
-defines which actions are available. See the user guide for information
-on what actions are available where.
+The ongoing vcl\_* subroutine execution ends when a return(*action*) statement
+is made.
+
+The *action* specifies how execution should proceed. The context defines
+which actions are available.
 
 Multiple subroutines
 ~~~~~~~~~~~~~~~~~~~~
 
-If multiple subroutines with the name of one of the builtin
-ones are defined, they are concatenated in the order in which they
-appear in the source.
-
-The default versions distributed with Varnish will be implicitly
-concatenated.
+If multiple subroutines with the name of one of the built-in ones are defined,
+they are concatenated in the order in which they appear in the source.
 
+The built-in VCL distributed with Varnish will be implicitly concatenated
+when the VCL is compiled.
 
 
 Variables
@@ -338,11 +353,35 @@ Functions
 The following built-in functions are available:
 
 ban(expression)
-  Bans all objects in cache that match the expression.
+  Invalidates all objects in cache that match the expression with the
+  ban mechanism.
+
+call(subroutine)
+  Run a VCL subroutine within the current scope.
+
+hash_data(input)
+  Adds an input to the hash input. In the built-in VCL hash_data()
+  is called on the host and URL of the *request*. Available in vcl_hash.
+
+new()
+  Instanciate a new VCL object. Available in vcl_init.
+
+purge()
+  Invalidate all variants of the current object using purge. Available in 
+  vcl_miss and vcl_hit.
 
-hash_data(str)
-  Adds a string to the hash input. In the built-in VCL hash_data()
-  is called on the host and URL of the *request*.
+return()
+  End execution of the current VCL subroutine, and continue to the next step
+  in the request handling state machine.
+
+rollback()
+  Restore request HTTP headers to their original state.
+
+synthetic(STRING)
+  Prepare a synthetic response body containing the STRING. Available in
+  vcl_synth and vcl_backend_error.
+
+.. list above comes from struct action_table[] in vcc_action.c.
 
 regsub(str, regex, sub)
   Returns a copy of str with the first occurrence of the regular
@@ -354,11 +393,13 @@ regsub(str, regex, sub)
 regsuball(str, regex, sub)
   As regsub() but this replaces all occurrences.
 
+.. regsub* is in vcc_expr.c
+
 
 EXAMPLES
 ========
 
-For examples, please see the guide guide.
+For examples, please see the online documentation.
 
 SEE ALSO
 ========
@@ -373,7 +414,7 @@ HISTORY
 VCL was developed by Poul-Henning Kamp in cooperation with Verdens
 Gang AS, Redpill Linpro and Varnish Software.  This manual page is
 written by Per Buer, Poul-Henning Kamp, Martin Blix Grydeland,
-Kristian Lyngst?l and possibly others.
+Kristian Lyngst?l, Lasse Karstensen and possibly others.
 
 COPYRIGHT
 =========



From lkarsten at varnish-software.com  Tue Apr  1 13:09:52 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:52 +0200
Subject: [4.0] 9556bb4 HTTP is an abbreviation
Message-ID: <E1WUyRs-0003Bz-Ql@project.varnish-software.com>


commit 9556bb4c23b9e8617760756d6d670831660bbe1f
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Fri Mar 21 14:57:16 2014 +0100

    HTTP is an abbreviation

diff --git a/lib/libvcc/vcc_action.c b/lib/libvcc/vcc_action.c
index 602beff..84724c1 100644
--- a/lib/libvcc/vcc_action.c
+++ b/lib/libvcc/vcc_action.c
@@ -135,7 +135,7 @@ parse_unset(struct vcc *tl)
 	assert(vp != NULL);
 	if (vp->fmt != HEADER) {
 		VSB_printf(tl->sb,
-		    "Only http header variables can be unset.\n");
+		    "Only HTTP header variables can be unset.\n");
 		vcc_ErrWhere(tl, tl->t);
 		return;
 	}



From lkarsten at varnish-software.com  Tue Apr  1 13:09:52 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:52 +0200
Subject: [4.0] cca0de0 Update copyright
Message-ID: <E1WUyRs-0003C8-Ry@project.varnish-software.com>


commit cca0de063ee656dcac52f63c0a05ed7172eca2c4
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Fri Mar 21 14:57:25 2014 +0100

    Update copyright

diff --git a/lib/libvcc/vcc_storage.c b/lib/libvcc/vcc_storage.c
index 5f9712a..0ab3f64 100644
--- a/lib/libvcc/vcc_storage.c
+++ b/lib/libvcc/vcc_storage.c
@@ -1,5 +1,5 @@
 /*-
- * Copyright (c) 2010 Varnish Software AS
+ * Copyright (c) 2010-2014 Varnish Software AS
  * All rights reserved.
  *
  * Author: Poul-Henning Kamp <phk at phk.freebsd.dk>



From lkarsten at varnish-software.com  Tue Apr  1 13:09:52 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:52 +0200
Subject: [4.0] 54e060c Move vcl_var.rst, use sane variable name for fp.
Message-ID: <E1WUyRs-0003CH-TH@project.varnish-software.com>


commit 54e060c6542fe1109b7f893cc259065f7c21e596
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Fri Mar 21 15:03:02 2014 +0100

    Move vcl_var.rst, use sane variable name for fp.
    
    Generated RST file should live in include/ like the others, not in reference/.
    
    Try to make the code slightly more readable through a descriptive variable name
    for the file handle.

diff --git a/.gitignore b/.gitignore
index a37080a..ba64e39 100644
--- a/.gitignore
+++ b/.gitignore
@@ -95,7 +95,7 @@ cscope.*out
 /doc/*.html
 /doc/sphinx/build/
 /doc/sphinx/conf.py
-/doc/sphinx/reference/vcl_var.rst
+/doc/sphinx/include/vcl_var.rst
 
 # NetBeans insists on this
 /nbproject/private/
diff --git a/doc/sphinx/reference/vcl.rst b/doc/sphinx/reference/vcl.rst
index c71577b..1268920 100644
--- a/doc/sphinx/reference/vcl.rst
+++ b/doc/sphinx/reference/vcl.rst
@@ -344,7 +344,7 @@ In VCL you have access to certain variable objects. These contain
 requests and responses currently being worked on. What variables are
 available depends on context.
 
-.. include:: vcl_var.rst
+.. include:: ../include/vcl_var.rst
 
 
 Functions
diff --git a/lib/libvcc/generate.py b/lib/libvcc/generate.py
index 84e363d..95c32eb 100755
--- a/lib/libvcc/generate.py
+++ b/lib/libvcc/generate.py
@@ -1131,7 +1131,7 @@ fo.close
 
 #######################################################################
 
-fo = open(buildroot + "/doc/sphinx/reference/vcl_var.rst", "w")
+fp_vclvar = open(buildroot + "/doc/sphinx/include/vcl_var.rst", "w")
 
 l = list()
 for i in sp_variables:
@@ -1164,14 +1164,14 @@ hdr=""
 for i in l:
 	j = i[0].split(".")
 	if j[0] != hdr:
-		fo.write("\n" + j[0] + "\n")
-		fo.write("~" * len(j[0]) + "\n")
+		fp_vclvar.write("\n" + j[0] + "\n")
+		fp_vclvar.write("~" * len(j[0]) + "\n")
 		hdr = j[0]
-	fo.write("\n" + i[0] + "\n\n")
-	fo.write("\tType: " + i[1] + "\n\n")
-	rst_where(fo, "Readable from: ", i[2])
-	rst_where(fo, "Writable from: ", i[3])
+	fp_vclvar.write("\n" + i[0] + "\n\n")
+	fp_vclvar.write("\tType: " + i[1] + "\n\n")
+	rst_where(fp_vclvar, "Readable from: ", i[2])
+	rst_where(fp_vclvar, "Writable from: ", i[3])
 	for j in i[4].split("\n"):
-		fo.write("\t" + j.strip() + "\n")
+		fp_vclvar.write("\t%s\n" % j.strip())
 
-fo.close()
+fp_vclvar.close()



From phk at FreeBSD.org  Tue Apr  1 13:09:52 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 01 Apr 2014 15:09:52 +0200
Subject: [4.0] ccc2005 Discover failure to execute varnishd faster
Message-ID: <E1WUyRs-0003CQ-UI@project.varnish-software.com>


commit ccc2005dac37c2a7d24fdba734d89e09d15310a9
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Mar 21 14:07:59 2014 +0000

    Discover failure to execute varnishd faster

diff --git a/bin/varnishtest/vtc_varnish.c b/bin/varnishtest/vtc_varnish.c
index 0b71923..19f51dd 100644
--- a/bin/varnishtest/vtc_varnish.c
+++ b/bin/varnishtest/vtc_varnish.c
@@ -339,8 +339,10 @@ varnish_thread(void *priv)
 		i = poll(fds, 1, 1000);
 		if (i == 0)
 			continue;
-		if (fds->revents & (POLLERR|POLLHUP))
+		if (fds->revents & (POLLERR|POLLHUP)) {
+			vtc_log(v->vl, 4, "STDOUT poll 0x%x", fds->revents);
 			break;
+		}
 		i = read(v->fds[0], buf, sizeof buf - 1);
 		if (i <= 0)
 			break;
@@ -426,8 +428,8 @@ varnish_launch(struct varnish *v)
 	memset(fd, 0, sizeof fd);
 	fd[0].fd = v->cli_fd;
 	fd[0].events = POLLIN;
-	fd[1].fd = v->fds[0];
-	fd[1].events = 0; /* Only care about POLLHUP, which is output-only */
+	fd[1].fd = v->fds[1];
+	fd[1].events = POLLIN;
 	i = poll(fd, 2, 10000);
 	vtc_log(v->vl, 4, "CLIPOLL %d 0x%x 0x%x",
 	    i, fd[0].revents, fd[1].revents);



From phk at FreeBSD.org  Tue Apr  1 13:09:52 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 01 Apr 2014 15:09:52 +0200
Subject: [4.0] 755a6c5 Track source changes
Message-ID: <E1WUyRs-0003CZ-VU@project.varnish-software.com>


commit 755a6c533d22a07fdfa40b00fcb020c16c7860a3
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Mar 21 14:31:36 2014 +0000

    Track source changes

diff --git a/bin/varnishtest/tests/v00018.vtc b/bin/varnishtest/tests/v00018.vtc
index 004ec15..aae3723 100644
--- a/bin/varnishtest/tests/v00018.vtc
+++ b/bin/varnishtest/tests/v00018.vtc
@@ -73,7 +73,7 @@ varnish v1 -errvcl {Symbol not found: 'mu' (expected type BOOL):} {
 	sub vcl_backend_response { set beresp.do_gzip = mu; }
 }
 
-varnish v1 -errvcl {Only http header variables can be unset.} {
+varnish v1 -errvcl {Only HTTP header variables can be unset.} {
 	backend b { .host = "127.0.0.1"; }
 	sub vcl_backend_response { unset beresp.do_gzip; }
 }



From lkarsten at varnish-software.com  Tue Apr  1 13:09:53 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:53 +0200
Subject: [4.0] d6a7f31 Cache-Control: no-cache and friends is honored.
Message-ID: <E1WUyRt-0003Ci-0O@project.varnish-software.com>


commit d6a7f3172cb882f70a17c95c550f0cb6cde48d20
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Sat Mar 22 09:56:43 2014 +0100

    Cache-Control: no-cache and friends is honored.
    
    The builtin VCL supports this now. Since so many noticed this missing
    in 3.0, I think it should be mentioned.

diff --git a/doc/sphinx/whats-new/upgrading.rst b/doc/sphinx/whats-new/upgrading.rst
index 02e6f02..402b693 100644
--- a/doc/sphinx/whats-new/upgrading.rst
+++ b/doc/sphinx/whats-new/upgrading.rst
@@ -79,7 +79,8 @@ req.* used to be available in `vcl_fetch`, but after the split of functionality,
 
 vcl_* reserved
 ~~~~~~~~~~~~~~
-Any custom-made subs cannot be named 'vcl_*' anymore. This namespace is reserved for builtin subs.
+Any custom-made subs cannot be named 'vcl_*' anymore. This namespace is
+reserved for builtin subs.
 
 req.backend.healthy replaced by std.healthy(req.backend)
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -87,8 +88,19 @@ req.backend.healthy replaced by std.healthy(req.backend)
 obj is now read-only
 ~~~~~~~~~~~~~~~~~~~~
 
-`obj` is now read-only. `obj.hits`, if enabled in VCL, now counts per objecthead,
-not per object. `obj.last_use` has been retired.
+`obj` is now read-only. `obj.hits`, if enabled in VCL, now counts per
+objecthead, not per object. `obj.last_use` has been retired.
+
+
+default/builtin VCL changes
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The VCL code that is appended to user-configured VCL automatically is now
+called the builtin VCL. (previously default.vcl)
+
+The builtin VCL now honors Cache-Control: no-cache (and friends) to indicate
+uncacheable content from the backend.
+
 
 Changes to parameters
 =====================



From phk at FreeBSD.org  Tue Apr  1 13:09:53 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 01 Apr 2014 15:09:53 +0200
Subject: [4.0] 71a357a Make objects read-only, once they have been stored.
Message-ID: <E1WUyRt-0003Cu-1h@project.varnish-software.com>


commit 71a357ab29bb4ec4f8e671665395e101ad885d2e
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Mar 24 07:59:58 2014 +0000

    Make objects read-only, once they have been stored.
    
    Now that we have a proper purge-mechanism, all the magic to
    set obj.{ttl,grace,keep} in vcl_hit{} can go away.

diff --git a/bin/varnishd/cache/cache_vrt_var.c b/bin/varnishd/cache/cache_vrt_var.c
index b51d2de..c48dc37 100644
--- a/bin/varnishd/cache/cache_vrt_var.c
+++ b/bin/varnishd/cache/cache_vrt_var.c
@@ -436,23 +436,21 @@ VRT_r_bereq_retries(const struct vrt_ctx *ctx)
  * keep are relative to ttl.
  */
 
-#define VRT_DO_EXP(which, sexp, fld, offset, now, extra)	\
+#define VRT_DO_EXP_L(which, sexp, fld)				\
 								\
 void								\
 VRT_l_##which##_##fld(const struct vrt_ctx *ctx, double a)	\
 {								\
-	double dt;						\
 								\
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);			\
-	if (a > 0.0)						\
-		sexp.fld = a + offset;				\
-	else							\
-		sexp.fld = 0;					\
-	extra;							\
-	dt = now - sexp.t_origin;				\
-	VSLb(ctx->vsl, SLT_TTL, "VCL %.0f %.0f %.0f %.0f %.0f",	\
-	    sexp.ttl - dt, sexp.grace, sexp.keep, now, dt);	\
-}								\
+	if (a < 0.0)						\
+		a = 0.0;					\
+	sexp.fld = a;						\
+	VSLb(ctx->vsl, SLT_TTL, "VCL %.0f %.0f %.0f %.0f",	\
+	    sexp.ttl, sexp.grace, sexp.keep, sexp.t_origin);	\
+}
+
+#define VRT_DO_EXP_R(which, sexp, fld, offset)			\
 								\
 double								\
 VRT_r_##which##_##fld(const struct vrt_ctx *ctx)		\
@@ -464,17 +462,17 @@ VRT_r_##which##_##fld(const struct vrt_ctx *ctx)		\
 	return(0.0);						\
 }
 
-VRT_DO_EXP(obj, ctx->req->obj->exp, grace, 0, ctx->req->t_req,
-   EXP_Rearm(ctx->req->obj, ctx->req->t_req, NAN, NAN, NAN);)
-VRT_DO_EXP(obj, ctx->req->obj->exp, ttl,
-   (ctx->req->t_req - ctx->req->obj->exp.t_origin), ctx->req->t_req,
-   EXP_Rearm(ctx->req->obj, ctx->req->t_req, NAN, NAN, NAN);)
-VRT_DO_EXP(obj, ctx->req->obj->exp, keep, 0, ctx->req->t_req,
-   EXP_Rearm(ctx->req->obj, ctx->req->t_req, NAN, NAN, NAN);)
+VRT_DO_EXP_R(obj, ctx->req->obj->exp, ttl,
+   (ctx->req->t_req - ctx->req->obj->exp.t_origin))
+VRT_DO_EXP_R(obj, ctx->req->obj->exp, grace, 0)
+VRT_DO_EXP_R(obj, ctx->req->obj->exp, keep, 0)
 
-VRT_DO_EXP(beresp, ctx->bo->exp, grace, 0, ctx->bo->exp.t_origin,)
-VRT_DO_EXP(beresp, ctx->bo->exp, ttl, 0, ctx->bo->exp.t_origin,)
-VRT_DO_EXP(beresp, ctx->bo->exp, keep, 0, ctx->bo->exp.t_origin,)
+VRT_DO_EXP_L(beresp, ctx->bo->exp, ttl)
+VRT_DO_EXP_R(beresp, ctx->bo->exp, ttl, 0)
+VRT_DO_EXP_L(beresp, ctx->bo->exp, grace)
+VRT_DO_EXP_R(beresp, ctx->bo->exp, grace, 0)
+VRT_DO_EXP_L(beresp, ctx->bo->exp, keep)
+VRT_DO_EXP_R(beresp, ctx->bo->exp, keep, 0)
 
 /*--------------------------------------------------------------------
  * [be]req.xid
diff --git a/bin/varnishtest/tests/p00001.vtc b/bin/varnishtest/tests/p00001.vtc
deleted file mode 100644
index 8c38715..0000000
--- a/bin/varnishtest/tests/p00001.vtc
+++ /dev/null
@@ -1,70 +0,0 @@
-varnishtest "Change TTL on persistent object"
-
-shell "rm -f ${tmpdir}/_.per"
-
-server s1 {
-	rxreq
-	txresp -hdr "Foo: foo"
-} -start
-
-varnish v1 \
-	-arg "-pfeature=+wait_silo" \
-	-arg "-pban_lurker_sleep=0" \
-	-arg "-spersistent,${tmpdir}/_.per,10m" \
-	-vcl+backend { } -start
-
-varnish v1 -cliok "param.set default_grace 0"
-varnish v1 -cliok "param.set default_keep 0"
-
-client c1 {
-	txreq -url "/"
-	rxresp
-	expect resp.status == 200
-	expect resp.http.X-Varnish == "1001"
-	expect resp.http.foo == "foo"
-} -run
-
-varnish v1 -stop
-
-varnish v1 -vcl+backend {
-	sub vcl_hit {
-		set obj.ttl = 1 s;
-	}
-}
-
-server s1 -wait
-
-varnish v1 -start
-varnish v1 -cliok "debug.xid 1999"
-
-client c1 {
-	txreq -url "/"
-	rxresp
-	expect resp.status == 200
-	expect resp.http.X-Varnish == "2001 1002"
-	expect resp.http.foo == "foo"
-} -run
-
-varnish v1 -stop
-
-server s1 {
-	rxreq
-	txresp -hdr "Foo: bar"
-} -start
-
-varnish v1 -vcl+backend { }
-
-delay 2
-
-varnish v1 -start
-varnish v1 -cliok "debug.xid 2999"
-
-client c1 {
-	txreq -url "/"
-	rxresp
-	expect resp.status == 200
-	expect resp.http.X-Varnish == "3001"
-	expect resp.http.foo == "bar"
-} -run
-
-# shell "rm -f /tmp/__v1/_.per"
diff --git a/bin/varnishtest/tests/r00365.vtc b/bin/varnishtest/tests/r00365.vtc
deleted file mode 100644
index 43c76b3..0000000
--- a/bin/varnishtest/tests/r00365.vtc
+++ /dev/null
@@ -1,29 +0,0 @@
-varnishtest "Test restarts in vcl_hit"
-
-server s1 {
-	rxreq
-	expect req.url == "/foo"
-	txresp -status 200 -body "1"
-	rxreq
-	expect req.url == "/foo"
-	txresp -status 200 -body "22"
-} -start
-
-varnish v1 -vcl+backend {
-	sub vcl_hit {
-		set obj.ttl = 0s;
-		return (restart);
-	}
-} -start
-
-client c1 {
-	txreq -url "/foo"
-	rxresp
-	expect resp.status == 200
-	expect resp.bodylen == 1
-
-	txreq -url "/foo"
-	rxresp
-	expect resp.status == 200
-	expect resp.bodylen == 2
-} -run
diff --git a/bin/varnishtest/tests/r00444.vtc b/bin/varnishtest/tests/r00444.vtc
deleted file mode 100644
index 899ab57..0000000
--- a/bin/varnishtest/tests/r00444.vtc
+++ /dev/null
@@ -1,59 +0,0 @@
-varnishtest "purging on POST"
-
-server s1 {
-	rxreq
-	expect req.method == "GET"
-	txresp -body "1"
-
-	rxreq
-	expect req.method == "POST"
-	txresp -body "22"
-
-	rxreq
-	expect req.method == "POST"
-	txresp -body "333"
-
-	rxreq
-	expect req.method == "GET"
-	txresp -body "4444"
-} -start
-
-varnish v1 -vcl+backend {
-	sub vcl_recv {
-		if (req.method == "POST") {
-			/* Lookup so we find any cached object */
-			return (hash);
-		}
-	}
-	sub vcl_hit {
-		if (req.method == "POST") {
-			/* Get rid of this object */
-			set obj.ttl = 0s;
-			return (pass);
-		}
-	}
-	sub vcl_miss {
-		if (req.method == "POST") {
-			/* Make sure we don't cache the POST result */
-			return (pass);
-		}
-	}
-} -start
-
-client c1 {
-	txreq
-	rxresp
-	expect resp.bodylen == 1
-
-	txreq -req POST
-	rxresp
-	expect resp.bodylen == 2
-
-	txreq -req POST
-	rxresp
-	expect resp.bodylen == 3
-
-	txreq
-	rxresp
-	expect resp.bodylen == 4
-} -run
diff --git a/bin/varnishtest/tests/r00956.vtc b/bin/varnishtest/tests/r00956.vtc
index 2744a44..85cca43 100644
--- a/bin/varnishtest/tests/r00956.vtc
+++ b/bin/varnishtest/tests/r00956.vtc
@@ -9,18 +9,12 @@ varnish v1 -vcl+backend {
 	sub vcl_backend_response {
 		set beresp.ttl = 10s;
 		set beresp.http.fooB = beresp.ttl;
-		set beresp.http.barB = "xxx";
 	}
 	sub vcl_hit {
 		set req.http.foo = obj.ttl;
-		set obj.ttl = 7s;
-		set obj.grace = 120s;
-		set obj.keep = 1h;
-		set req.http.bar = obj.ttl;
 	}
 	sub vcl_deliver {
 		set resp.http.foo = req.http.foo;
-		set resp.http.bar = req.http.bar;
 	}
 } -start
 
@@ -29,7 +23,6 @@ client c1 {
 	rxresp
 	expect resp.bodylen == 40
 	expect resp.http.fooB == 10.000
-	expect resp.http.barB == "xxx"
 
 	delay 2
 	txreq
@@ -37,19 +30,13 @@ client c1 {
 	expect resp.bodylen == 40
 	# XXX: should be: < 8
 	expect resp.http.fooB == 10.000
-	expect resp.http.barB == "xxx"
 	expect resp.http.foo != 10.000
 	expect resp.http.foo != 9.000
-	expect resp.http.bar == 7.000
 
 	delay 2
 	txreq
 	rxresp
 	expect resp.bodylen == 40
-	# XXX: should be: < 5
 	expect resp.http.fooB == 10.000
-	expect resp.http.barB == "xxx"
-	expect resp.http.foo != 7.000
-	expect resp.http.foo != 6.000
-	expect resp.http.bar == 7.000
+	expect resp.http.foo <= 5.000
 } -run
diff --git a/bin/varnishtest/tests/v00016.vtc b/bin/varnishtest/tests/v00016.vtc
index b42106d..1107f86 100644
--- a/bin/varnishtest/tests/v00016.vtc
+++ b/bin/varnishtest/tests/v00016.vtc
@@ -75,12 +75,12 @@ varnish v1 -vcl {
 
 varnish v1 -errvcl {Unknown time unit 'k'.  Legal are 'ms', 's', 'm', 'h', 'd' and 'w'} {
 	backend b { .host = "127.0.0.1"; }
-	sub vcl_backend_response { set obj.ttl = 1. k; }
+	sub vcl_backend_response { set beresp.ttl = 1. k; }
 }
 
 varnish v1 -errvcl {Expression has type DURATION, expected BOOL} {
 	backend b { .host = "127.0.0.1"; }
-	sub vcl_backend_response { if (obj.ttl *= 2) { } }
+	sub vcl_backend_response { if (beresp.ttl *= 2) { } }
 }
 
 varnish v1 -errvcl {Operator > not possible on BACKEND} {
diff --git a/bin/varnishtest/tests/v00021.vtc b/bin/varnishtest/tests/v00021.vtc
index 636207e..301384b 100644
--- a/bin/varnishtest/tests/v00021.vtc
+++ b/bin/varnishtest/tests/v00021.vtc
@@ -1,11 +1,11 @@
 varnishtest "VCL compiler coverage test: vcc_xref.c"
 
-varnish v1 -errvcl {'obj.ttl': cannot be set in method 'vcl_recv'.} {
+varnish v1 -errvcl {Variable 'obj.ttl' is read only.} {
 	backend b { .host = "127.0.0.1"; }
 	sub vcl_recv { set obj.ttl = 1 w; }
 }
 
-varnish v1 -errvcl {'obj.ttl': cannot be set from method 'foo'.} {
+varnish v1 -errvcl {Variable 'obj.ttl' is read only.} {
 	backend b { .host = "127.0.0.1"; }
 
 	sub foo { set obj.ttl = 1 w; }
diff --git a/bin/varnishtest/tests/v00023.vtc b/bin/varnishtest/tests/v00023.vtc
deleted file mode 100644
index e96585a..0000000
--- a/bin/varnishtest/tests/v00023.vtc
+++ /dev/null
@@ -1,29 +0,0 @@
-varnishtest "Test that obj.ttl = 0s prevents subsequent hits"
-
-server s1 {
-	rxreq
-	expect req.url == "/foo"
-	txresp -status 200 -body "1"
-	rxreq
-	expect req.url == "/foo"
-	txresp -status 200 -body "22"
-} -start
-
-varnish v1 -vcl+backend {
-	sub vcl_hit {
-		set obj.ttl = 0s;
-		return (restart);
-	}
-} -start
-
-client c1 {
-	txreq -url "/foo"
-	rxresp
-	expect resp.status == 200
-	expect resp.bodylen == 1
-
-	txreq -url "/foo"
-	rxresp
-	expect resp.status == 200
-	expect resp.bodylen == 2
-} -run
diff --git a/bin/varnishtest/tests/v00025.vtc b/bin/varnishtest/tests/v00025.vtc
index b090502..1be49af 100644
--- a/bin/varnishtest/tests/v00025.vtc
+++ b/bin/varnishtest/tests/v00025.vtc
@@ -24,14 +24,10 @@ varnish v1 -vcl+backend {
 
 	sub vcl_hit {
 		if (obj.grace < 3m) {
-			set obj.grace = 1m;
-		} else {
-			set obj.grace = 2m;
+			return (deliver);
 		}
 		if (obj.ttl < 3m) {
-			set obj.ttl = 2m;
-		} else {
-			set obj.ttl = 3m;
+			return (deliver);
 		}
 	}
 
diff --git a/include/tbl/vsl_tags.h b/include/tbl/vsl_tags.h
index 9217582..5f768ec 100644
--- a/include/tbl/vsl_tags.h
+++ b/include/tbl/vsl_tags.h
@@ -230,27 +230,26 @@ SLTM(TTL, 0, "TTL set on object",
 	"A TTL record is emitted whenever the ttl, grace or keep"
 	" values for an object is set.\n\n"
 	"The format is::\n\n"
-	"\t%u %s %d %d %d %d %d [ %d %u %u ]\n"
-	"\t|  |  |  |  |  |  |    |  |  |\n"
-	"\t|  |  |  |  |  |  |    |  |  +- Max-Age from Cache-Control header\n"
-	"\t|  |  |  |  |  |  |    |  +---- Expires header\n"
-	"\t|  |  |  |  |  |  |    +------- Date header\n"
-	"\t|  |  |  |  |  |  +------------ Age (incl Age: header value)\n"
-	"\t|  |  |  |  |  +--------------- Reference time for TTL\n"
-	"\t|  |  |  |  +------------------ Keep\n"
-	"\t|  |  |  +--------------------- Grace\n"
-	"\t|  |  +------------------------ TTL\n"
-	"\t|  +--------------------------- \"RFC\" or \"VCL\"\n"
-	"\t+------------------------------ object XID\n"
+	"\t%s %d %d %d %d %d [ %d %u %u ]\n"
+	"\t|  |  |  |  |  |    |  |  |\n"
+	"\t|  |  |  |  |  |    |  |  +- Max-Age from Cache-Control header\n"
+	"\t|  |  |  |  |  |    |  +---- Expires header\n"
+	"\t|  |  |  |  |  |    +------- Date header\n"
+	"\t|  |  |  |  |  +------------ Age (incl Age: header value)\n"
+	"\t|  |  |  |  +--------------- Reference time for TTL\n"
+	"\t|  |  |  +------------------ Keep\n"
+	"\t|  |  +--------------------- Grace\n"
+	"\t|  +------------------------ TTL\n"
+	"\t+--------------------------- \"RFC\" or \"VCL\"\n"
 	"\n"
-	"The last three fields are only present in \"RFC\" headers.\n\n"
+	"The last four fields are only present in \"RFC\" headers.\n\n"
 	"Examples::\n\n"
-	"\t1001 RFC 19 -1 -1 1312966109 4 0 0 23\n"
-	"\t1001 VCL 10 -1 -1 1312966109 4\n"
-	"\t1001 VCL 7 -1 -1 1312966111 6\n"
-	"\t1001 VCL 7 120 -1 1312966111 6\n"
-	"\t1001 VCL 7 120 3600 1312966111 6\n"
-	"\t1001 VCL 12 120 3600 1312966113 8\n"
+	"\tRFC 19 -1 -1 1312966109 4 0 0 23\n"
+	"\tVCL 10 -1 -1 1312966109 4\n"
+	"\tVCL 7 -1 -1 1312966111 6\n"
+	"\tVCL 7 120 -1 1312966111 6\n"
+	"\tVCL 7 120 3600 1312966111 6\n"
+	"\tVCL 12 120 3600 1312966113 8\n"
 	"\n"
 )
 
diff --git a/lib/libvcc/generate.py b/lib/libvcc/generate.py
index 95c32eb..b6ccfe2 100755
--- a/lib/libvcc/generate.py
+++ b/lib/libvcc/generate.py
@@ -528,7 +528,7 @@ sp_variables = [
 	('obj.ttl',
 		'DURATION',
 		( 'hit', ),
-		( 'hit', ), """
+		( ), """
 		The object's remaining time to live, in seconds.
 		obj.ttl is writable.
 		"""
@@ -536,14 +536,14 @@ sp_variables = [
 	('obj.grace',
 		'DURATION',
 		( 'hit', ),
-		( 'hit', ), """
+		( ), """
 		The object's grace period in seconds. obj.grace is writable.
 		"""
 	),
 	('obj.keep',
 		'DURATION',
 		( 'hit', ),
-		( 'hit', ), """
+		( ), """
 		"""
 	),
 	('obj.uncacheable',



From phk at FreeBSD.org  Tue Apr  1 13:09:53 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 01 Apr 2014 15:09:53 +0200
Subject: [4.0] 67538af Move retries behind refcount so it is cleared when
 busyobj is recycled.
Message-ID: <E1WUyRt-0003DG-3y@project.varnish-software.com>


commit 67538af2af3f4285a12b1e7d9a0044a44c614ec3
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Mar 24 09:33:06 2014 +0000

    Move retries behind refcount so it is cleared when busyobj is recycled.
    
    Spotted by:	martin

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index ac615a4..01ec775 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -523,8 +523,8 @@ struct busyobj {
 	 * All fields from refcount and down are zeroed when the busyobj
 	 * is recycled.
 	 */
-	int			retries;
 	unsigned		refcount;
+	int			retries;
 	double			t_fetch;
 	struct req		*req;
 



From phk at FreeBSD.org  Tue Apr  1 13:09:53 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 01 Apr 2014 15:09:53 +0200
Subject: [4.0] e557692 If vcl_synth{} or vcl_deliver{} sets a "Connection:
 close" header, we close from our side with the new SC_RESP_CLOSE status.
Message-ID: <E1WUyRt-0003DN-5x@project.varnish-software.com>


commit e55769294b592eb3cf7a0cf9ea6109ad08e22466
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Mar 24 10:06:48 2014 +0000

    If vcl_synth{} or vcl_deliver{} sets a "Connection: close" header,
    we close from our side with the new SC_RESP_CLOSE status.

diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index 6650c31..d3d70d6 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -153,6 +153,9 @@ cnt_deliver(struct worker *wrk, struct req *req)
 
 	V1D_Deliver(req);
 
+	if (http_HdrIs(req->resp, H_Connection, "close"))
+		req->doclose = SC_RESP_CLOSE;
+
 	if (req->obj->objcore->flags & OC_F_PASS) {
 		/*
 		 * No point in saving the body if it is hit-for-pass,
@@ -185,7 +188,7 @@ DOT }
  */
 
 static enum req_fsm_nxt
-cnt_error(struct worker *wrk, struct req *req)
+cnt_synth(struct worker *wrk, struct req *req)
 {
 	char date[40];
 	struct http *h;
@@ -234,6 +237,9 @@ cnt_error(struct worker *wrk, struct req *req)
 	}
 	assert(wrk->handling == VCL_RET_DELIVER);
 
+	if (http_HdrIs(req->resp, H_Connection, "close"))
+		req->doclose = SC_RESP_CLOSE;
+
 	V1D_Deliver_Synth(req);
 
 	VSB_delete(req->synth_body);
@@ -272,7 +278,7 @@ cnt_fetch(struct worker *wrk, struct req *req)
 
 	if (req->objcore->flags & OC_F_FAILED) {
 		req->err_code = 503;
-		req->req_step = R_STP_ERROR;
+		req->req_step = R_STP_SYNTH;
 		(void)HSH_DerefObjCore(&wrk->stats, &req->objcore);
 		req->objcore = NULL;
 		return (REQ_FSM_MORE);
@@ -422,7 +428,7 @@ cnt_lookup(struct worker *wrk, struct req *req)
 		req->req_step = R_STP_RESTART;
 		break;
 	case VCL_RET_SYNTH:
-		req->req_step = R_STP_ERROR;
+		req->req_step = R_STP_SYNTH;
 		break;
 	case VCL_RET_PASS:
 		wrk->stats.cache_hit++;
@@ -480,7 +486,7 @@ cnt_miss(struct worker *wrk, struct req *req)
 		req->req_step = R_STP_FETCH;
 		return (REQ_FSM_MORE);
 	case VCL_RET_SYNTH:
-		req->req_step = R_STP_ERROR;
+		req->req_step = R_STP_SYNTH;
 		break;
 	case VCL_RET_RESTART:
 		req->req_step = R_STP_RESTART;
@@ -528,7 +534,7 @@ cnt_pass(struct worker *wrk, struct req *req)
 	VCL_pass_method(req->vcl, wrk, req, NULL, req->http->ws);
 	switch (wrk->handling) {
 	case VCL_RET_SYNTH:
-		req->req_step = R_STP_ERROR;
+		req->req_step = R_STP_SYNTH;
 		break;
 	case VCL_RET_RESTART:
 		req->req_step = R_STP_RESTART;
@@ -620,7 +626,7 @@ cnt_restart(struct worker *wrk, struct req *req)
 	if (++req->restarts >= cache_param->max_restarts) {
 		VSLb(req->vsl, SLT_VCL_Error, "Too many restarts");
 		req->err_code = 503;
-		req->req_step = R_STP_ERROR;
+		req->req_step = R_STP_SYNTH;
 	} else {
 		wid = VXID_Get(&wrk->vxid_pool);
 		// XXX: ReqEnd + ReqAcct ?
@@ -676,7 +682,7 @@ cnt_recv(struct worker *wrk, struct req *req)
 	http_VSL_log(req->http);
 
 	if (req->err_code) {
-		req->req_step = R_STP_ERROR;
+		req->req_step = R_STP_SYNTH;
 		return (REQ_FSM_MORE);
 	}
 
@@ -755,7 +761,7 @@ cnt_recv(struct worker *wrk, struct req *req)
 		req->req_step = R_STP_PASS;
 		return (REQ_FSM_MORE);
 	case VCL_RET_SYNTH:
-		req->req_step = R_STP_ERROR;
+		req->req_step = R_STP_SYNTH;
 		return (REQ_FSM_MORE);
 	default:
 		WRONG("Illegal return from vcl_recv{}");
@@ -795,7 +801,7 @@ cnt_purge(struct worker *wrk, struct req *req)
 	AZ(HSH_DerefObjCore(&wrk->stats, &boc));
 
 	VCL_purge_method(req->vcl, wrk, req, NULL, req->http->ws);
-	req->req_step = R_STP_ERROR;
+	req->req_step = R_STP_SYNTH;
 	return (REQ_FSM_MORE);
 }
 
diff --git a/include/tbl/sess_close.h b/include/tbl/sess_close.h
index 522eb02..5b77eba 100644
--- a/include/tbl/sess_close.h
+++ b/include/tbl/sess_close.h
@@ -39,6 +39,7 @@ SESS_CLOSE(RX_TIMEOUT,	"Receive timeout")
 SESS_CLOSE(TX_PIPE,	"Piped transaction")
 SESS_CLOSE(TX_ERROR,	"Error transaction")
 SESS_CLOSE(TX_EOF,	"EOF transmission")
+SESS_CLOSE(RESP_CLOSE,	"Backend/VCL requested close")
 SESS_CLOSE(OVERLOAD,	"Out of some resource")
 SESS_CLOSE(SESS_PIPE_OVERFLOW,	"Session pipe overflow")
 
diff --git a/include/tbl/steps.h b/include/tbl/steps.h
index d0efae7..bb02fb4 100644
--- a/include/tbl/steps.h
+++ b/include/tbl/steps.h
@@ -45,7 +45,7 @@ REQ_STEP(purge,		PURGE,		(wrk, req))
 REQ_STEP(miss,		MISS,		(wrk, req))
 REQ_STEP(fetch,		FETCH,		(wrk, req))
 REQ_STEP(deliver,	DELIVER,	(wrk, req))
-REQ_STEP(error,		ERROR,		(wrk, req))
+REQ_STEP(synth,		SYNTH,		(wrk, req))
 #endif
 
 #ifdef FETCH_STEP



From phk at FreeBSD.org  Tue Apr  1 13:09:53 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 01 Apr 2014 15:09:53 +0200
Subject: [4.0] d850882 Add tests for outgoing Connection: close handling
Message-ID: <E1WUyRt-0003DX-7W@project.varnish-software.com>


commit d850882688678e514bc01766390b1f93731057ba
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Mar 24 10:22:54 2014 +0000

    Add tests for outgoing Connection: close handling

diff --git a/bin/varnishtest/tests/c00064.vtc b/bin/varnishtest/tests/c00064.vtc
new file mode 100644
index 0000000..4791597
--- /dev/null
+++ b/bin/varnishtest/tests/c00064.vtc
@@ -0,0 +1,35 @@
+varnishtest "Connection: close in vcl_synth{}"
+
+server s1 {
+	rxreq
+	txresp -status 400
+} -start
+
+varnish v1 -vcl+backend {
+	sub vcl_miss {
+		if (req.url == "/333") {
+			return (synth(333, "FOO"));
+		} else {
+			return (synth(334, "BAR"));
+		}
+	}
+
+	sub vcl_synth {
+		if (resp.status == 333) {
+			set resp.http.connection = "close";
+		}
+	}
+} -start
+
+client c1 {
+	txreq -url /334
+	rxresp
+	expect resp.status == 334
+	txreq -url /334
+	rxresp
+	expect resp.status == 334
+	txreq -url /333
+	rxresp
+	expect resp.status == 333
+	expect_close
+} -run
diff --git a/bin/varnishtest/tests/c00065.vtc b/bin/varnishtest/tests/c00065.vtc
new file mode 100644
index 0000000..aed48cd
--- /dev/null
+++ b/bin/varnishtest/tests/c00065.vtc
@@ -0,0 +1,24 @@
+varnishtest "Connection: close in vcl_deliver{}"
+
+server s1 {
+	rxreq
+	txresp
+} -start
+
+varnish v1 -vcl+backend {
+	sub vcl_deliver {
+		set resp.http.Connection  = "close";
+	}
+} -start
+
+client c1 {
+	txreq
+	rxresp
+	expect_close
+} -run
+
+client c1 {
+	txreq
+	rxresp
+	expect_close
+} -run



From phk at FreeBSD.org  Tue Apr  1 13:09:53 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 01 Apr 2014 15:09:53 +0200
Subject: [4.0] 6abce40 Reflect change of obj size
Message-ID: <E1WUyRt-0003Dg-9h@project.varnish-software.com>


commit 6abce4069f3715e8f1f0812051b43c3a0b310cd3
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Mar 24 10:56:57 2014 +0000

    Reflect change of obj size

diff --git a/bin/varnishtest/tests/c00044.vtc b/bin/varnishtest/tests/c00044.vtc
index 9bd7950..2d1d3cc 100644
--- a/bin/varnishtest/tests/c00044.vtc
+++ b/bin/varnishtest/tests/c00044.vtc
@@ -40,7 +40,7 @@ varnish v1 -expect SMA.Transient.g_bytes == 0
 varnish v1 -expect SMA.s0.g_bytes == 0
 varnish v1 -expect SMA.s0.g_space > 1000000
 varnish v1 -expect SMA.s1.g_bytes > 1000000
-varnish v1 -expect SMA.s1.g_space < 170
+varnish v1 -expect SMA.s1.g_space < 180
 varnish v1 -expect SMA.s2.g_bytes == 0
 varnish v1 -expect SMA.s2.g_space > 1000000
 
@@ -55,9 +55,9 @@ varnish v1 -expect SMA.Transient.g_bytes == 0
 varnish v1 -expect SMA.s0.g_bytes == 0
 varnish v1 -expect SMA.s0.g_space > 1000000
 varnish v1 -expect SMA.s1.g_bytes > 1000000
-varnish v1 -expect SMA.s1.g_space < 170
+varnish v1 -expect SMA.s1.g_space < 180
 varnish v1 -expect SMA.s2.g_bytes > 1000000
-varnish v1 -expect SMA.s2.g_space < 170
+varnish v1 -expect SMA.s2.g_space < 180
 
 client c1 {
 	txreq -url /burp
@@ -68,11 +68,11 @@ client c1 {
 
 varnish v1 -expect SMA.Transient.g_bytes == 0
 varnish v1 -expect SMA.s0.g_bytes > 1000000
-varnish v1 -expect SMA.s0.g_space < 170
+varnish v1 -expect SMA.s0.g_space < 180
 varnish v1 -expect SMA.s1.g_bytes > 1000000
-varnish v1 -expect SMA.s1.g_space < 170
+varnish v1 -expect SMA.s1.g_space < 180
 varnish v1 -expect SMA.s2.g_bytes > 1000000
-varnish v1 -expect SMA.s2.g_space < 170
+varnish v1 -expect SMA.s2.g_space < 180
 
 client c1 {
 	txreq -url /foo1
diff --git a/bin/varnishtest/tests/c00045.vtc b/bin/varnishtest/tests/c00045.vtc
index c653cf0..3105041 100644
--- a/bin/varnishtest/tests/c00045.vtc
+++ b/bin/varnishtest/tests/c00045.vtc
@@ -30,7 +30,7 @@ client c1 {
 
 varnish v1 -expect SMA.Transient.g_bytes == 0
 varnish v1 -expect SMA.s0.g_bytes > 1000000
-varnish v1 -expect SMA.s0.g_space < 170
+varnish v1 -expect SMA.s0.g_space < 180
 varnish v1 -expect SMA.s1.g_bytes == 0
 varnish v1 -expect SMA.s1.g_space > 1000000
 varnish v1 -expect SMA.s2.g_bytes == 0



From phk at FreeBSD.org  Tue Apr  1 13:09:53 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 01 Apr 2014 15:09:53 +0200
Subject: [4.0] d5a6e32 Bump 10 bytes further to catch all platforms
Message-ID: <E1WUyRt-0003Dr-BU@project.varnish-software.com>


commit d5a6e32141034ae08ed2e29a5c916f20b46441d9
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Mar 24 11:29:21 2014 +0000

    Bump 10 bytes further to catch all platforms

diff --git a/bin/varnishtest/tests/c00045.vtc b/bin/varnishtest/tests/c00045.vtc
index 3105041..024a3dd 100644
--- a/bin/varnishtest/tests/c00045.vtc
+++ b/bin/varnishtest/tests/c00045.vtc
@@ -30,7 +30,7 @@ client c1 {
 
 varnish v1 -expect SMA.Transient.g_bytes == 0
 varnish v1 -expect SMA.s0.g_bytes > 1000000
-varnish v1 -expect SMA.s0.g_space < 180
+varnish v1 -expect SMA.s0.g_space < 190
 varnish v1 -expect SMA.s1.g_bytes == 0
 varnish v1 -expect SMA.s1.g_space > 1000000
 varnish v1 -expect SMA.s2.g_bytes == 0



From phk at FreeBSD.org  Tue Apr  1 13:09:53 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 01 Apr 2014 15:09:53 +0200
Subject: [4.0] e135798 more tweaking
Message-ID: <E1WUyRt-0003E1-DD@project.varnish-software.com>


commit e1357988dcb4959930eb1695f877f11bda56501b
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Mar 24 12:06:39 2014 +0000

    more tweaking

diff --git a/bin/varnishtest/tests/c00045.vtc b/bin/varnishtest/tests/c00045.vtc
index 024a3dd..16f26f7 100644
--- a/bin/varnishtest/tests/c00045.vtc
+++ b/bin/varnishtest/tests/c00045.vtc
@@ -46,7 +46,7 @@ client c1 {
 varnish v1 -expect n_lru_nuked == 1
 varnish v1 -expect SMA.Transient.g_bytes == 0
 varnish v1 -expect SMA.s0.g_bytes > 1000000
-varnish v1 -expect SMA.s0.g_space < 1171
+varnish v1 -expect SMA.s0.g_space < 1180
 varnish v1 -expect SMA.s1.g_bytes == 0
 varnish v1 -expect SMA.s1.g_space > 1000000
 varnish v1 -expect SMA.s2.g_bytes == 0



From phk at FreeBSD.org  Tue Apr  1 13:09:53 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 01 Apr 2014 15:09:53 +0200
Subject: [4.0] fe32532 more tweaking
Message-ID: <E1WUyRt-0003EC-F7@project.varnish-software.com>


commit fe325325aabca22700048a3815fa8d9e23ebd115
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Mar 24 12:21:52 2014 +0000

    more tweaking

diff --git a/bin/varnishtest/tests/c00045.vtc b/bin/varnishtest/tests/c00045.vtc
index 16f26f7..126f1a9 100644
--- a/bin/varnishtest/tests/c00045.vtc
+++ b/bin/varnishtest/tests/c00045.vtc
@@ -62,7 +62,7 @@ client c1 {
 varnish v1 -expect n_lru_nuked == 2
 varnish v1 -expect SMA.Transient.g_bytes == 0
 varnish v1 -expect SMA.s0.g_bytes > 1000000
-varnish v1 -expect SMA.s0.g_space < 1172
+varnish v1 -expect SMA.s0.g_space < 1182
 varnish v1 -expect SMA.s1.g_bytes == 0
 varnish v1 -expect SMA.s1.g_space > 1000000
 varnish v1 -expect SMA.s2.g_bytes == 0



From phk at FreeBSD.org  Tue Apr  1 13:09:53 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 01 Apr 2014 15:09:53 +0200
Subject: [4.0] 836f128 Add a couple of tiny delays to stabilize this test.
Message-ID: <E1WUyRt-0003EN-Hh@project.varnish-software.com>


commit 836f1283eff963c3a73d4fe7b4c0cf9b9bb50286
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Mar 24 12:51:25 2014 +0000

    Add a couple of tiny delays to stabilize this test.

diff --git a/bin/varnishtest/tests/c00055.vtc b/bin/varnishtest/tests/c00055.vtc
index 4ecbe35..e02bc47 100644
--- a/bin/varnishtest/tests/c00055.vtc
+++ b/bin/varnishtest/tests/c00055.vtc
@@ -31,11 +31,15 @@ client c1 {
 	expect resp.bodylen == 2
 } -run
 
+delay .1
+
 client c1 {
 	txreq -req POST -nolen -hdr "Content-Length: 52"
 	delay .3
 } -run
 
+delay .1
+
 server s1 {
 	rxreq
 	txresp



From phk at FreeBSD.org  Tue Apr  1 13:09:53 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 01 Apr 2014 15:09:53 +0200
Subject: [4.0] 7a14478 Stabilize this test with a short sleep
Message-ID: <E1WUyRt-0003EX-Iu@project.varnish-software.com>


commit 7a144781c3d2a926e84bfa1cc32f44b11a4621b8
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Mar 24 13:13:01 2014 +0000

    Stabilize this test with a short sleep

diff --git a/bin/varnishtest/tests/c00041.vtc b/bin/varnishtest/tests/c00041.vtc
index 24a7871..2b7efc2 100644
--- a/bin/varnishtest/tests/c00041.vtc
+++ b/bin/varnishtest/tests/c00041.vtc
@@ -47,21 +47,26 @@ client c1 {
 	txreq -url "/1" -hdr "foo: foo1"
 	rxresp
 	expect resp.status == 200
+	expect resp.http.x-varnish == 1001
 	expect resp.bodylen == 1
 
 	txreq -url "/1" -hdr "Foo: foo2"
 	rxresp
 	expect resp.status == 200
+	expect resp.http.x-varnish == 1003
 	expect resp.bodylen == 2
+	delay .1
 
 	txreq -url "/1" -hdr "foo: foo1"
 	rxresp
 	expect resp.status == 200
+	expect resp.http.x-varnish == "1005 1002"
 	expect resp.bodylen == 1
 
 	txreq -url "/1" -hdr "Foo: foo2"
 	rxresp
 	expect resp.status == 200
+	expect resp.http.x-varnish == "1006 1004"
 	expect resp.bodylen == 2
 
 	# Purge on hit



From daghf at varnish-software.com  Tue Apr  1 13:09:53 2014
From: daghf at varnish-software.com (Dag Haavi Finstad)
Date: Tue, 01 Apr 2014 15:09:53 +0200
Subject: [4.0] 32c8acb Handle 'return (abandon);' in vcl_backend_response.
Message-ID: <E1WUyRt-0003Ei-Kd@project.varnish-software.com>


commit 32c8acbf89a0ce4f3a6fe1ed16b08ae7e9af7564
Author: Dag Haavi Finstad <daghf at varnish-software.com>
Date:   Mon Mar 24 14:54:30 2014 +0100

    Handle 'return (abandon);' in vcl_backend_response.
    
    Test case by: Federico G. Schwindt
    
    Fixes: #1417

diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index 5b5be58..93be439 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -341,6 +341,9 @@ vbf_stp_startfetch(struct worker *wrk, struct busyobj *bo)
 
 	VCL_backend_response_method(bo->vcl, wrk, NULL, bo, bo->beresp->ws);
 
+	if (wrk->handling == VCL_RET_ABANDON)
+		return (F_STP_FAIL);
+
 	if (wrk->handling == VCL_RET_RETRY) {
 		AN (bo->vbc);
 		VDI_CloseFd(&bo->vbc);
diff --git a/bin/varnishtest/tests/r01417.vtc b/bin/varnishtest/tests/r01417.vtc
new file mode 100644
index 0000000..60263c7
--- /dev/null
+++ b/bin/varnishtest/tests/r01417.vtc
@@ -0,0 +1,18 @@
+varnishtest "vcl_backend_response abandon"
+
+server s1 {
+	rxreq
+	txresp
+} -start
+
+varnish v1 -vcl+backend {
+	sub vcl_backend_response {
+		return (abandon);
+	}
+} -start
+
+client c1 {
+	txreq
+	rxresp
+	expect resp.status == 503
+} -run



From phk at FreeBSD.org  Tue Apr  1 13:09:53 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 01 Apr 2014 15:09:53 +0200
Subject: [4.0] 5a6cf6c Add more panic detail on busyobjs.
Message-ID: <E1WUyRt-0003Es-MD@project.varnish-software.com>


commit 5a6cf6c74a71ccdd3bcb5257bb1d81ff432a684d
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Mar 24 16:33:30 2014 +0000

    Add more panic detail on busyobjs.
    
    Remove a debugging message.

diff --git a/bin/varnishd/cache/cache_panic.c b/bin/varnishd/cache/cache_panic.c
index a01e771..575d049 100644
--- a/bin/varnishd/cache/cache_panic.c
+++ b/bin/varnishd/cache/cache_panic.c
@@ -283,6 +283,10 @@ pan_busyobj(const struct busyobj *bo)
 
 	VSB_printf(pan_vsp, "  busyobj = %p {\n", bo);
 	pan_ws(bo->ws, 4);
+	VSB_printf(pan_vsp, "  refcnt = %u\n", bo->refcount);
+	VSB_printf(pan_vsp, "  retries = %d\n", bo->retries);
+	VSB_printf(pan_vsp, "  failed = %d\n", bo->failed);
+	VSB_printf(pan_vsp, "  state = %d\n", (int)bo->state);
 #define BO_FLAG(l, r, w, d) if(bo->l) VSB_printf(pan_vsp, "    is_" #l "\n");
 #include "tbl/bo_flags.h"
 #undef BO_FLAG
diff --git a/bin/varnishd/cache/cache_vrt_var.c b/bin/varnishd/cache/cache_vrt_var.c
index c48dc37..b68031c 100644
--- a/bin/varnishd/cache/cache_vrt_var.c
+++ b/bin/varnishd/cache/cache_vrt_var.c
@@ -192,7 +192,6 @@ VRT_l_beresp_uncacheable(const struct vrt_ctx *ctx, unsigned a)
 		VSLb(ctx->vsl, SLT_VCL_Error,
 		    "Ignoring attempt to reset beresp.uncacheable");
 	} else if (a) {
-		VSLb(ctx->vsl, SLT_Debug, "XXXX: UNCACHEABLE %u" , a);
 		ctx->bo->uncacheable = 1;
 	}
 }



From phk at FreeBSD.org  Tue Apr  1 13:09:53 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 01 Apr 2014 15:09:53 +0200
Subject: [4.0] d94db39 Remove a debugging print
Message-ID: <E1WUyRt-0003F1-O3@project.varnish-software.com>


commit d94db39f17311760bb95e166e68a9a109fafdc78
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Mar 25 07:51:44 2014 +0000

    Remove a debugging print

diff --git a/lib/libvcc/generate.py b/lib/libvcc/generate.py
index b6ccfe2..a118513 100755
--- a/lib/libvcc/generate.py
+++ b/lib/libvcc/generate.py
@@ -1002,7 +1002,6 @@ def one_var(nm, spec):
 		if nm == i[0]:
 			fh.write("VCL_" + typ +
 			    " VRT_r_%s(const struct vrt_ctx *);\n" % cnam )
-	print(spec)
 	restrict(fo, spec[2])
 
 	if len(spec[3]) == 0:



From phk at FreeBSD.org  Tue Apr  1 13:09:53 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 01 Apr 2014 15:09:53 +0200
Subject: [4.0] 405aad9 Split the oc->flags into those locked by oh->mtx,
 and those locked by lru->mtx.
Message-ID: <E1WUyRt-0003FA-Q4@project.varnish-software.com>


commit 405aad97af2482ec28df3e248055d0cd512a7fb0
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Mar 25 08:29:05 2014 +0000

    Split the oc->flags into those locked by oh->mtx, and those locked
    by lru->mtx.
    
    Spotted by: Martin

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index 01ec775..68c74ff 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -422,17 +422,21 @@ struct objcore {
 	struct objhead		*objhead;
 	struct busyobj		*busyobj;
 	double			timer_when;
-	unsigned		flags;
+
+	uint16_t		flags;
 #define OC_F_BUSY		(1<<1)
 #define OC_F_PASS		(1<<2)
-#define OC_F_OFFLRU		(1<<4)
 #define OC_F_PRIV		(1<<5)		/* Stevedore private flag */
-#define OC_F_DYING		(1<<7)
 #define OC_F_PRIVATE		(1<<8)
 #define OC_F_FAILED		(1<<9)
-#define OC_F_MOVE		(1<<10)
-#define OC_F_INSERT		(1<<11)
-#define OC_F_EXP		(1<<12)
+
+	uint16_t		exp_flags;
+#define OC_EF_OFFLRU		(1<<4)
+#define OC_EF_MOVE		(1<<10)
+#define OC_EF_INSERT		(1<<11)
+#define OC_EF_EXP		(1<<12)
+#define OC_EF_DYING		(1<<7)
+
 	unsigned		timer_idx;
 	VTAILQ_ENTRY(objcore)	list;
 	VTAILQ_ENTRY(objcore)	lru_list;
diff --git a/bin/varnishd/cache/cache_expire.c b/bin/varnishd/cache/cache_expire.c
index 4f21152..8bd0707 100644
--- a/bin/varnishd/cache/cache_expire.c
+++ b/bin/varnishd/cache/cache_expire.c
@@ -111,9 +111,9 @@ exp_mail_it(struct objcore *oc)
 {
 	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
 
-	AN(oc->flags & OC_F_OFFLRU);
+	AN(oc->exp_flags & OC_EF_OFFLRU);
 	Lck_Lock(&exphdl->mtx);
-	if (oc->flags & OC_F_DYING)
+	if (oc->exp_flags & OC_EF_DYING)
 		VTAILQ_INSERT_HEAD(&exphdl->inbox, oc, lru_list);
 	else
 		VTAILQ_INSERT_TAIL(&exphdl->inbox, oc, lru_list);
@@ -135,8 +135,8 @@ EXP_Inject(struct objcore *oc, struct lru *lru, double when)
 
 	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
 
-	AZ(oc->flags & (OC_F_OFFLRU | OC_F_INSERT | OC_F_MOVE));
-	AZ(oc->flags & OC_F_DYING);
+	AZ(oc->exp_flags & (OC_EF_OFFLRU | OC_EF_INSERT | OC_EF_MOVE));
+	AZ(oc->exp_flags & OC_EF_DYING);
 	// AN(oc->flags & OC_F_BUSY);
 
 	if (lru == NULL)
@@ -145,9 +145,9 @@ EXP_Inject(struct objcore *oc, struct lru *lru, double when)
 
 	Lck_Lock(&lru->mtx);
 	lru->n_objcore++;
-	oc->flags |= OC_F_OFFLRU | OC_F_INSERT | OC_F_EXP;
+	oc->exp_flags |= OC_EF_OFFLRU | OC_EF_INSERT | OC_EF_EXP;
 	if (when < 0)
-		oc->flags |= OC_F_MOVE;
+		oc->exp_flags |= OC_EF_MOVE;
 	else
 		oc->timer_when = when;
 	Lck_Unlock(&lru->mtx);
@@ -205,9 +205,9 @@ EXP_Touch(struct objcore *oc, double now)
 	if (Lck_Trylock(&lru->mtx))
 		return;
 
-	AN(oc->flags & OC_F_EXP);
+	AN(oc->exp_flags & OC_EF_EXP);
 
-	if (!(oc->flags & OC_F_OFFLRU)) {
+	if (!(oc->exp_flags & OC_EF_OFFLRU)) {
 		/* Can only touch it while it's actually on the LRU list */
 		VTAILQ_REMOVE(&lru->lru_head, oc, lru_list);
 		VTAILQ_INSERT_TAIL(&lru->lru_head, oc, lru_list);
@@ -234,7 +234,7 @@ EXP_Rearm(struct object *o, double now, double ttl, double grace, double keep)
 	CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
 	assert(oc->refcnt > 0);
 
-	AN(oc->flags & OC_F_EXP);
+	AN(oc->exp_flags & OC_EF_EXP);
 
 	if (!isnan(ttl))
 		o->exp.ttl = now + ttl - o->exp.t_origin;
@@ -257,14 +257,14 @@ EXP_Rearm(struct object *o, double now, double ttl, double grace, double keep)
 	Lck_Lock(&lru->mtx);
 
 	if (!isnan(now) && when <= now)
-		oc->flags |= OC_F_DYING;
+		oc->exp_flags |= OC_EF_DYING;
 	else
-		oc->flags |= OC_F_MOVE;
+		oc->exp_flags |= OC_EF_MOVE;
 
-	if (oc->flags & OC_F_OFFLRU) {
+	if (oc->exp_flags & OC_EF_OFFLRU) {
 		oc = NULL;
 	} else {
-		oc->flags |= OC_F_OFFLRU;
+		oc->exp_flags |= OC_EF_OFFLRU;
 		VTAILQ_REMOVE(&lru->lru_head, oc, lru_list);
 	}
 	Lck_Unlock(&lru->mtx);
@@ -294,8 +294,8 @@ EXP_NukeOne(struct busyobj *bo, struct lru *lru)
 		VSLb(bo->vsl, SLT_ExpKill, "LRU_Cand p=%p f=0x%x r=%d",
 		    oc, oc->flags, oc->refcnt);
 
-		AZ(oc->flags & OC_F_OFFLRU);
-		AZ(oc->flags & OC_F_DYING);
+		AZ(oc->exp_flags & OC_EF_OFFLRU);
+		AZ(oc->exp_flags & OC_EF_DYING);
 
 		/*
 		 * It wont release any space if we cannot release the last
@@ -309,7 +309,7 @@ EXP_NukeOne(struct busyobj *bo, struct lru *lru)
 		if (Lck_Trylock(&oh->mtx))
 			continue;
 		if (oc->refcnt == 1) {
-			oc->flags |= OC_F_DYING | OC_F_OFFLRU;
+			oc->exp_flags |= OC_EF_DYING | OC_EF_OFFLRU;
 			oc->refcnt++;
 			VSC_C_main->n_lru_nuked++; // XXX per lru ?
 			VTAILQ_REMOVE(&lru->lru_head, oc, lru_list);
@@ -434,20 +434,20 @@ exp_inbox(struct exp_priv *ep, struct objcore *oc, double now)
 
 	/* Evacuate our action-flags, and put it back on the LRU list */
 	Lck_Lock(&lru->mtx);
-	flags = oc->flags;
-	AN(flags & OC_F_OFFLRU);
-	oc->flags &= ~(OC_F_INSERT | OC_F_MOVE);
+	flags = oc->exp_flags;
+	AN(flags & OC_EF_OFFLRU);
+	oc->exp_flags &= ~(OC_EF_INSERT | OC_EF_MOVE);
 	oc->last_lru = now;
-	if (!(flags & OC_F_DYING)) {
+	if (!(flags & OC_EF_DYING)) {
 		VTAILQ_INSERT_TAIL(&lru->lru_head, oc, lru_list);
-		oc->flags &= ~OC_F_OFFLRU;
+		oc->exp_flags &= ~OC_EF_OFFLRU;
 	}
 	Lck_Unlock(&lru->mtx);
 
-	if (flags & OC_F_DYING) {
+	if (flags & OC_EF_DYING) {
 		VSLb(&ep->vsl, SLT_ExpKill, "EXP_Kill p=%p e=%.9f f=0x%x", oc,
 		    oc->timer_when, oc->flags);
-		if (!(flags & OC_F_INSERT)) {
+		if (!(flags & OC_EF_INSERT)) {
 			assert(oc->timer_idx != BINHEAP_NOIDX);
 			binheap_delete(ep->heap, oc->timer_idx);
 		}
@@ -456,7 +456,7 @@ exp_inbox(struct exp_priv *ep, struct objcore *oc, double now)
 		return;
 	}
 
-	if (flags & OC_F_MOVE) {
+	if (flags & OC_EF_MOVE) {
 		o = oc_getobj(&ep->wrk->stats, oc);
 		CHECK_OBJ_NOTNULL(o, OBJECT_MAGIC);
 		oc->timer_when = exp_when(o);
@@ -472,11 +472,11 @@ exp_inbox(struct exp_priv *ep, struct objcore *oc, double now)
 	 * XXX: the next moment and rip them out again.
 	 */
 
-	if (flags & OC_F_INSERT) {
+	if (flags & OC_EF_INSERT) {
 		assert(oc->timer_idx == BINHEAP_NOIDX);
 		binheap_insert(exphdl->heap, oc);
 		assert(oc->timer_idx != BINHEAP_NOIDX);
-	} else if (flags & OC_F_MOVE) {
+	} else if (flags & OC_EF_MOVE) {
 		assert(oc->timer_idx != BINHEAP_NOIDX);
 		binheap_reorder(exphdl->heap, oc->timer_idx);
 		assert(oc->timer_idx != BINHEAP_NOIDX);
@@ -514,11 +514,11 @@ exp_expire(struct exp_priv *ep, double now)
 	CHECK_OBJ_NOTNULL(lru, LRU_MAGIC);
 	Lck_Lock(&lru->mtx);
 	// AZ(oc->flags & OC_F_BUSY);
-	oc->flags |= OC_F_DYING;
-	if (oc->flags & OC_F_OFFLRU)
+	oc->exp_flags |= OC_EF_DYING;
+	if (oc->exp_flags & OC_EF_OFFLRU)
 		oc = NULL;
 	else {
-		oc->flags |= OC_F_OFFLRU;
+		oc->exp_flags |= OC_EF_OFFLRU;
 		VTAILQ_REMOVE(&lru->lru_head, oc, lru_list);
 	}
 	Lck_Unlock(&lru->mtx);
diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index d7263e1..6cb2de8 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -401,7 +401,9 @@ HSH_Lookup(struct req *req, struct objcore **ocp, struct objcore **bocp,
 		CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
 		assert(oc->objhead == oh);
 
-		if (oc->flags & (OC_F_FAILED | OC_F_DYING))
+		if (oc->exp_flags & OC_EF_DYING)
+			continue;
+		if (oc->flags & OC_F_FAILED)
 			continue;
 
 		if (oc->flags & OC_F_BUSY) {
@@ -572,7 +574,7 @@ HSH_Purge(struct worker *wrk, struct objhead *oh, double ttl, double grace)
 	VTAILQ_FOREACH(oc, &oh->objcs, list) {
 		CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC);
 		assert(oc->objhead == oh);
-		if (oc->flags & (OC_F_BUSY|OC_F_DYING)) {
+		if (oc->flags & OC_F_BUSY) {
 			/*
 			 * We cannot purge busy objects here, because their
 			 * owners have special rights to them, and may nuke
@@ -581,6 +583,8 @@ HSH_Purge(struct worker *wrk, struct objhead *oh, double ttl, double grace)
 			 */
 			continue;
 		}
+		if (oc->exp_flags & OC_EF_DYING)
+			continue;
 		xxxassert(spc >= sizeof *ocp);
 		oc->refcnt++;
 		spc -= sizeof *ocp;
@@ -680,7 +684,7 @@ HSH_Unbusy(struct dstat *ds, struct objcore *oc)
 	if (!(oc->flags & OC_F_PRIVATE)) {
 		BAN_NewObjCore(oc);
 		EXP_Insert(oc);
-		AN(oc->flags & OC_F_EXP);
+		AN(oc->exp_flags & OC_EF_EXP);
 		AN(oc->ban);
 	}
 
diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index d3d70d6..fd9783c 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -100,7 +100,7 @@ cnt_deliver(struct worker *wrk, struct req *req)
 	assert(req->obj->objcore->refcnt > 0);
 
 	req->t_resp = W_TIM_real(wrk);
-	if (req->obj->objcore->flags & OC_F_EXP)
+	if (req->obj->objcore->exp_flags & OC_EF_EXP)
 		EXP_Touch(req->obj->objcore, req->t_resp);
 
 	HTTP_Setup(req->resp, req->ws, req->vsl, SLT_RespMethod);
@@ -399,7 +399,8 @@ cnt_lookup(struct worker *wrk, struct req *req)
 	switch (wrk->handling) {
 	case VCL_RET_DELIVER:
 		if (boc != NULL) {
-			AZ(oc->flags & (OC_F_FAILED|OC_F_DYING|OC_F_PASS));
+			AZ(oc->flags & (OC_F_FAILED|OC_F_PASS));
+			AZ(oc->exp_flags & OC_EF_DYING);
 			AZ(oc->busyobj);
 			VBF_Fetch(wrk, req, boc, o, VBF_BACKGROUND);
 		} else {



From phk at FreeBSD.org  Tue Apr  1 13:09:53 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 01 Apr 2014 15:09:53 +0200
Subject: [4.0] ac21ca0 If the backend sends "Content-Length: 0" and
 "Content_Encoding: gzip" it is buggy,
 work around it. If the backend sends chunked encoding we don't catch it and
 fail the transfer.
Message-ID: <E1WUyRt-0003FL-Rm@project.varnish-software.com>


commit ac21ca073d7a93268e9f7340519e2e9cc8db1d00
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Mar 25 10:46:35 2014 +0000

    If the backend sends "Content-Length: 0" and "Content_Encoding: gzip"
    it is buggy, work around it.  If the backend sends chunked encoding
    we don't catch it and fail the transfer.
    
    Fixes: #1320

diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index 93be439..ecbcf2e 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -416,6 +416,20 @@ vbf_stp_fetch(struct worker *wrk, struct busyobj *bo)
 	if (bo->do_gzip && !bo->is_gunzip)
 		bo->do_gzip = 0;
 
+	AN(bo->vbc);
+	est = V1F_Setup_Fetch(bo);
+
+	if (est == 0) {
+		/*
+		 * If the length is known to be zero, it's not gziped.
+		 * A similar issue exists for chunked encoding but we
+		 * don't handle that.  See #1320.
+		 */
+		http_Unset(bo->beresp, H_Content_Encoding);
+		bo->is_gzip = 0;
+		bo->is_gunzip = 1;
+	}
+
 	/* But we can't do both at the same time */
 	assert(bo->do_gzip == 0 || bo->do_gunzip == 0);
 
@@ -425,9 +439,6 @@ vbf_stp_fetch(struct worker *wrk, struct busyobj *bo)
 	else if (bo->do_gunzip)
 		http_Unset(bo->beresp, H_Content_Encoding);
 
-	AN(bo->vbc);
-	est = V1F_Setup_Fetch(bo);
-
 	if (bo->do_gunzip || (bo->is_gzip && bo->do_esi)) {
 		RFC2616_Weaken_Etag(bo->beresp);
 		VFP_Push(bo, vfp_gunzip_pull, 0);
diff --git a/bin/varnishtest/tests/r01320.vtc b/bin/varnishtest/tests/r01320.vtc
new file mode 100644
index 0000000..568699d
--- /dev/null
+++ b/bin/varnishtest/tests/r01320.vtc
@@ -0,0 +1,15 @@
+varnishtest "#1320 zero-length gzip"
+
+server s1 {
+	rxreq
+	txresp -hdr "Content-Encoding: gzip"
+} -start
+
+varnish v1 -vcl+backend { } -start
+
+
+client c1 {
+	txreq -hdr "Accept-Encoding: gzip"
+	rxresp
+	expect resp.status == 200
+} -run



From perbu at varnish-software.com  Tue Apr  1 13:09:53 2014
From: perbu at varnish-software.com (Per Buer)
Date: Tue, 01 Apr 2014 15:09:53 +0200
Subject: [4.0] c17fc43 Reorder command line options with the binary and add
 the missing ones. Reformat for easier editing.
Message-ID: <E1WUyRt-0003FT-TG@project.varnish-software.com>


commit c17fc433fa65c4294814fe810efc7f413038075d
Author: Per Buer <perbu at varnish-software.com>
Date:   Tue Mar 25 12:16:52 2014 +0100

    Reorder command line options with the binary and add the missing ones. Reformat for easier editing.

diff --git a/doc/sphinx/reference/varnishd.rst b/doc/sphinx/reference/varnishd.rst
index a93ecac..506e2e2 100644
--- a/doc/sphinx/reference/varnishd.rst
+++ b/doc/sphinx/reference/varnishd.rst
@@ -12,17 +12,19 @@ HTTP accelerator daemon
 SYNOPSIS
 ========
 
-varnishd [-a address[:port]] [-b host[:port]] [-d] [-F] [-f config]
-	 [-g group] [-h type[,options]] [-i identity]
-	 [-l shmlogsize] [-n name] [-P file] [-p param=value]
-	 [-s type[,options]] [-T address[:port]] [-t ttl]
-	 [-r param[,param...]] [-u user] [-V]
+varnishd [-a address[:port]] [-b host[:port]] [-C] [-d] [-f config]
+	 [-F] [-g group] [-h type[,options]] [-i identity]
+	 [-l shl[,free[,fill]]] [-M address:port] [-n name] 
+         [-P file] [-p param=value] [-r param[,param...]
+	 [-s [name=]kind[,options]] [-S secret-file] [-T address[:port]] 
+         [-t ttl] [-u user] [-V]
 
 DESCRIPTION
 ===========
 
-The varnishd daemon accepts HTTP requests from clients, passes them on to a backend server and caches the
-returned documents to better satisfy future requests for the same document.
+The varnishd daemon accepts HTTP requests from clients, passes them on
+to a backend server and caches the returned documents to better
+satisfy future requests for the same document.
 
 .. _ref-varnishd-options:
 
@@ -30,33 +32,39 @@ OPTIONS
 =======
 
 -a address[:port][,address[:port][...]
-            Listen for client requests on the specified address and port.  The address can be a host
-            name (?localhost?), an IPv4 dotted-quad (?127.0.0.1?), or an IPv6 address enclosed in
-            square brackets (?[::1]?).  If address is not specified, varnishd will listen on all
-            available IPv4 and IPv6 interfaces.  If port is not specified, the default HTTP port as
-            listed in /etc/services is used.  Multiple listening addresses and ports can be
-            specified as a whitespace or comma -separated list.
+            Listen for client requests on the specified address and
+            port.  The address can be a host name (?localhost?), an
+            IPv4 dotted-quad (?127.0.0.1?), or an IPv6 address
+            enclosed in square brackets (?[::1]?).  If address is not
+            specified, varnishd will listen on all available IPv4 and
+            IPv6 interfaces.  If port is not specified, the default
+            HTTP port as listed in /etc/services is used.  Multiple
+            listening addresses and ports can be specified as a
+            whitespace or comma -separated list.
 
 -b host[:port]
-            Use the specified host as backend server.  If port is not specified,
-	        the default is 8080.
+            Use the specified host as backend server.  If port is not
+	    specified, the default is 8080.
 
--C	        Print VCL code compiled to C language and exit. Specify the VCL file
-	        to compile with the -f option.
+-C	    Print VCL code compiled to C language and exit. Specify the VCL file
+	    to compile with the -f option.
 
--d          Enables debugging mode: The parent process runs in the foreground with a CLI connection
-            on stdin/stdout, and the child process must be started explicitly with a CLI command.
-            Terminating the parent process will also terminate the child.
-
--F          Run in the foreground.
+-d          Enables debugging mode: The parent process runs in the foreground
+            with a CLI connection on stdin/stdout, and the child
+            process must be started explicitly with a CLI command.
+            Terminating the parent process will also terminate the
+            child.
 
 -f config   Use the specified VCL configuration file instead of the
             builtin default.  See vcl(7) for details on VCL
             syntax. When no configuration is supplied varnishd will
             not start the cache process.
 
--g group    Specifies the name of an unprivileged group to which the child process should switch
-            before it starts accepting connections.  This is a shortcut for specifying the group
+-F          Run in the foreground.
+
+-g group    Specifies the name of an unprivileged group to which the
+            child process should switch before it starts accepting
+            connections.  This is a shortcut for specifying the group
             run-time parameter.
 
 -h type[,options]
@@ -66,14 +74,25 @@ OPTIONS
             Specify the identity of the Varnish server.  This can be accessed using server.identity
             from VCL
 
--l shmlogsize
-            Specify size of shmlog file.  Scaling suffixes like 'k', 'm' can be used up to
-            (e)tabytes.  Default is 80 Megabytes.  Specifying less than 8 Megabytes is unwise.
+-l shl[,free[,fill]]
+            Specifies size of shmlog file. shl is the store for the
+            shared memory log records [80m], free is the store for other
+            allocations [1m] and fill determines how the log is [+].
+            Scaling suffixes like 'k', 'm' can be used up to
+            (e)tabytes.  Default is 80 Megabytes.
 
--n name     Specify a name for this instance.  Amonst other things, this name is used to construct
-            the name of the directory in which varnishd keeps temporary files and persistent state.
-            If the specified name begins with a forward slash, it is interpreted as the absolute
-            path to the directory which should be used for this purpose.
+-M address:port
+            Connect to this port and offer the command line interface.
+            Think of it as a reverse shell. When running with -M and there is
+            no backend defined the child process (the cache) will not start
+            initially.
+
+-n name     Specify the name for this instance.  Amonst other things, this
+            name is used to construct the name of the directory in
+            which varnishd keeps temporary files and persistent state.
+            If the specified name begins with a forward slash, it is
+            interpreted as the absolute path to the directory which
+            should be used for this purpose.
 
 -P file     Write the process's PID to the specified file.
 
@@ -82,7 +101,13 @@ OPTIONS
             Run-Time Parameters for a list of parameters. This option can be
             used multiple times to specify multiple parameters.
 
--S file     Path to a file containing a secret used for authorizing access to the management port.
+-r param[,param...]
+            Make the listed parameters read only. This gives the
+            system administrator a way to limit what the Varnish CLI can do.
+            Consider making parameters such as *user*, *group*, *cc_command*,
+            *vcc_allow_inline_c* read only as these can potentially be used
+            to escalate privileges from the CLI.
+            Protecting *listen_address* may also be a good idea.
 
 -s [name=]type[,options]
             Use the specified storage backend. The storage backends can be one of the following:
@@ -95,27 +120,15 @@ OPTIONS
             multiple times to specify multiple storage files. Names
             are referenced in logs, vcl, statistics, etc.
 
+-S file     Path to a file containing a secret used for authorizing access to the management port.
+
 -T address[:port]
             Offer a management interface on the specified address and port.  See Management
             Interface for a list of management commands.
 
--M address:port
-            Connect to this port and offer the command line interface.
-            Think of it as a reverse shell. When running with -M and there is
-            no backend defined the child process (the cache) will not start
-            initially.
-
 -t ttl      Specifies a hard minimum time to live for cached documents. This
             is a shortcut for specifying the default_ttl run-time parameter.
 
--r param[,param...]
-            Make the listed parameters read only. This gives the
-            system administrator a way to limit what the Varnish CLI can do.
-            Consider making parameters such as *user*, *group*, *cc_command*,
-            *vcc_allow_inline_c* read only as these can potentially be used
-            to escalate privileges from the CLI.
-            Protecting *listen_address* may also be a good idea.
-
 -u user     Specifies the name of an unprivileged user to which the child
             process should switch before it starts accepting
             connections. This is a shortcut for specifying the user



From perbu at varnish-software.com  Tue Apr  1 13:09:53 2014
From: perbu at varnish-software.com (Per Buer)
Date: Tue, 01 Apr 2014 15:09:53 +0200
Subject: [4.0] 1e08789 zap a warning. Formatting error
Message-ID: <E1WUyRt-0003Fd-Ub@project.varnish-software.com>


commit 1e08789d6c99866250dc2fcab98d72eb76981152
Author: Per Buer <perbu at varnish-software.com>
Date:   Tue Mar 25 12:24:47 2014 +0100

    zap a warning. Formatting error

diff --git a/doc/sphinx/installation/install.rst b/doc/sphinx/installation/install.rst
index a8ef31c..9b61d35 100644
--- a/doc/sphinx/installation/install.rst
+++ b/doc/sphinx/installation/install.rst
@@ -138,8 +138,6 @@ figuring out what.
 Installing
 ----------
 
-And finally, the true test of a brave heart::
-
 And finally, the true test of a brave heart: ``sudo make install``
 
 Varnish will now be installed in `/usr/local`. The `varnishd` binary is in



From perbu at varnish-software.com  Tue Apr  1 13:09:53 2014
From: perbu at varnish-software.com (Per Buer)
Date: Tue, 01 Apr 2014 15:09:53 +0200
Subject: [4.0] a1a10e0 Clear up hash algorithms. Critbit is teh shit.
Message-ID: <E1WUyRt-0003Fm-WA@project.varnish-software.com>


commit a1a10e0790f9ffb2aa859e76f6f4d1c7670ef33c
Author: Per Buer <perbu at varnish-software.com>
Date:   Tue Mar 25 12:56:21 2014 +0100

    Clear up hash algorithms.
    Critbit is teh shit.

diff --git a/doc/sphinx/reference/varnishd.rst b/doc/sphinx/reference/varnishd.rst
index 506e2e2..975fa14 100644
--- a/doc/sphinx/reference/varnishd.rst
+++ b/doc/sphinx/reference/varnishd.rst
@@ -145,20 +145,22 @@ Hash Algorithms
 
 The following hash algorithms are available:
 
+critbit 
+   A self-scaling tree structure. The default hash algorithm in
+    Varnish Cache 2.1 and onwards. In comparison to a more traditional
+    B tree the critbit tree is almost completely lockless. Do not 
+    change this unless you are certain what you're doing.
+
 simple_list
     A simple doubly-linked list.  Not recommended for production use.
 
 classic[,buckets]
-    A standard hash table.  This is the default.  The hash key is the
-    CRC32 of the object's URL modulo the size of the hash table.  Each
-    table entry points to a list of elements which share the same hash
-    key. The buckets parameter specifies the number of entries in the
-    hash table.  The default is 16383.
-
-critbit
-    A self-scaling tree structure. The default hash algorithm in 2.1. In
-    comparison to a more traditional B tree the critbit tree is almost
-    completely lockless.
+    A standard hash table. The hash key is the CRC32 of the object's
+    URL modulo the size of the hash table.  Each table entry points to
+    a list of elements which share the same hash key. The buckets
+    parameter specifies the number of entries in the hash table.  The
+    default is 16383.
+
 
 Storage Types
 -------------



From perbu at varnish-software.com  Tue Apr  1 13:09:54 2014
From: perbu at varnish-software.com (Per Buer)
Date: Tue, 01 Apr 2014 15:09:54 +0200
Subject: [4.0] a343a8e Discourage use of persistent. You need to know what you
 are doing.
Message-ID: <E1WUyRu-0003Fv-1s@project.varnish-software.com>


commit a343a8ede5db704fb43da48f5047e682c93c2410
Author: Per Buer <perbu at varnish-software.com>
Date:   Tue Mar 25 12:56:45 2014 +0100

    Discourage use of persistent.
    You need to know what you are doing.

diff --git a/doc/sphinx/reference/varnishd.rst b/doc/sphinx/reference/varnishd.rst
index 975fa14..3e30d19 100644
--- a/doc/sphinx/reference/varnishd.rst
+++ b/doc/sphinx/reference/varnishd.rst
@@ -188,7 +188,9 @@ syntax: persistent,path,size
 
 Persistent storage. Varnish will store objects in a file in a manner
 that will secure the survival of *most* of the objects in the event of
-a planned or unplanned shutdown of Varnish.
+a planned or unplanned shutdown of Varnish. The persistent storage
+backend has multiple issues with it and will likely be removed from a
+future version of Varnish.
 
 
 Management Interface



From perbu at varnish-software.com  Tue Apr  1 13:09:54 2014
From: perbu at varnish-software.com (Per Buer)
Date: Tue, 01 Apr 2014 15:09:54 +0200
Subject: [4.0] 2d38d19 Too much Varnish Software here.
Message-ID: <E1WUyRu-0003G4-3C@project.varnish-software.com>


commit 2d38d194dc6f630a0f51f7b8855d539e22d7a20e
Author: Per Buer <perbu at varnish-software.com>
Date:   Tue Mar 25 12:57:02 2014 +0100

    Too much Varnish Software here.

diff --git a/doc/sphinx/reference/varnishd.rst b/doc/sphinx/reference/varnishd.rst
index 3e30d19..633ea43 100644
--- a/doc/sphinx/reference/varnishd.rst
+++ b/doc/sphinx/reference/varnishd.rst
@@ -255,7 +255,7 @@ HISTORY
 =======
 
 The varnishd daemon was developed by Poul-Henning Kamp in cooperation
-with Verdens Gang AS, Varnish Software AS and Varnish Software.
+with Verdens Gang AS and Varnish Software.
 
 This manual page was written by Dag-Erling Sm?rgrav with updates by
 Stig Sandbeck Mathisen <ssm at debian.org>.



From daghf at varnish-software.com  Tue Apr  1 13:09:54 2014
From: daghf at varnish-software.com (Dag Haavi Finstad)
Date: Tue, 01 Apr 2014 15:09:54 +0200
Subject: [4.0] 8a986ce Fix 'varnishstat -l' segmentation fault.
Message-ID: <E1WUyRu-0003GG-4c@project.varnish-software.com>


commit 8a986ce96e1396edb80fdd538d2045ca2dd2d115
Author: Dag Haavi Finstad <daghf at varnish-software.com>
Date:   Tue Mar 25 15:08:29 2014 +0100

    Fix 'varnishstat -l' segmentation fault.
    
    Fixes: #1450

diff --git a/bin/varnishstat/varnishstat.c b/bin/varnishstat/varnishstat.c
index 10f6d51..31c0a1b 100644
--- a/bin/varnishstat/varnishstat.c
+++ b/bin/varnishstat/varnishstat.c
@@ -210,6 +210,10 @@ do_list_cb(void *priv, const struct VSC_point * const pt)
 	const struct VSC_section * sec;
 
 	(void)priv;
+
+	if (pt == NULL)
+		return (0);
+
 	sec = pt->section;
 	i = 0;
 	if (strcmp(sec->fantom->type, ""))



From daghf at varnish-software.com  Tue Apr  1 13:09:54 2014
From: daghf at varnish-software.com (Dag Haavi Finstad)
Date: Tue, 01 Apr 2014 15:09:54 +0200
Subject: [4.0] 26bc79e Fix 'varnishstat -l' lack of output.
Message-ID: <E1WUyRu-0003GP-5n@project.varnish-software.com>


commit 26bc79ea9b1e743cf224883c45fd2b3fdbf7ebfb
Author: Dag Haavi Finstad <daghf at varnish-software.com>
Date:   Tue Mar 25 15:20:44 2014 +0100

    Fix 'varnishstat -l' lack of output.
    
    Postpone invoking list_fields() until after VSM_Open().

diff --git a/bin/varnishstat/varnishstat.c b/bin/varnishstat/varnishstat.c
index 31c0a1b..7b37f0b 100644
--- a/bin/varnishstat/varnishstat.c
+++ b/bin/varnishstat/varnishstat.c
@@ -273,7 +273,7 @@ main(int argc, char * const *argv)
 {
 	int c;
 	struct VSM_data *vd;
-	int delay = 1, once = 0, xml = 0, json = 0, do_repeat = 0;
+	int delay = 1, once = 0, xml = 0, json = 0, do_repeat = 0, f_list = 0;
 
 	vd = VSM_New();
 
@@ -283,8 +283,8 @@ main(int argc, char * const *argv)
 			once = 1;
 			break;
 		case 'l':
-			list_fields(vd);
-			exit(0);
+			f_list = 1;
+			break;
 		case 'V':
 			VCS_Message("varnishstat");
 			exit(0);
@@ -310,7 +310,7 @@ main(int argc, char * const *argv)
 		fprintf(stderr, "%s\n", VSM_Error(vd));
 		exit(1);
 	}
-	if (!(xml || json || once)) {
+	if (!(xml || json || once || f_list)) {
 		do_curses(vd, delay);
 		exit(0);
 	}
@@ -322,6 +322,8 @@ main(int argc, char * const *argv)
 			do_json(vd);
 		else if (once)
 			do_once(vd, VSC_Main(vd, NULL));
+		else if (f_list)
+			list_fields(vd);
 		else {
 			assert(0);
 		}



From lkarsten at varnish-software.com  Tue Apr  1 13:09:54 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:54 +0200
Subject: [4.0] f8c6b70 Document weight on the hash director
Message-ID: <E1WUyRu-0003Ga-8X@project.varnish-software.com>


commit f8c6b70e4edeed99d25f8c17521ebbb1981746af
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Tue Mar 25 17:54:09 2014 +0100

    Document weight on the hash director

diff --git a/lib/libvmod_directors/vmod.vcc b/lib/libvmod_directors/vmod.vcc
index 24e8839..3b21dc5 100644
--- a/lib/libvmod_directors/vmod.vcc
+++ b/lib/libvmod_directors/vmod.vcc
@@ -141,9 +141,13 @@ Example
 $Method VOID .add_backend(BACKEND, REAL)
 
 Description
-       Adds a backend to the director.
+	Adds a backend to the director with a certain weight.
+
+	Weight is used as in the round_robin director. Recommended value is
+	1.0 unless you have special needs.
+
 Example
-       hdir.add_backend(backend1);
+       hdir.add_backend(backend1, 1.0);
 
 $Method BACKEND .backend(STRING_LIST)
 



From lkarsten at varnish-software.com  Tue Apr  1 13:09:54 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:54 +0200
Subject: [4.0] b112e9a Remove incomplete sentence and visible XXX
Message-ID: <E1WUyRu-0003Gj-Ah@project.varnish-software.com>


commit b112e9a9e83ada9ff1f508d628e61a594cb09f41
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Tue Mar 25 17:54:31 2014 +0100

    Remove incomplete sentence and visible XXX

diff --git a/lib/libvmod_directors/vmod.vcc b/lib/libvmod_directors/vmod.vcc
index 3b21dc5..230bf7f 100644
--- a/lib/libvmod_directors/vmod.vcc
+++ b/lib/libvmod_directors/vmod.vcc
@@ -81,9 +81,9 @@ Example
 $Object fallback()
 
 Description
-        Creates a fallback director. This director will
+        Creates a fallback director.
 
-	XXX: Add description of fallback director.
+#	XXX: Add description of fallback director.
 
 Example
         new foo = directors.fallback();



From lkarsten at varnish-software.com  Tue Apr  1 13:09:54 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:54 +0200
Subject: [4.0] efc1871 Improve backend_hint + director example
Message-ID: <E1WUyRu-0003Gs-Bj@project.varnish-software.com>


commit efc187150ab61681113f5ca54825670931724c40
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Tue Mar 25 18:32:40 2014 +0100

    Improve backend_hint + director example

diff --git a/doc/sphinx/whats-new/upgrading.rst b/doc/sphinx/whats-new/upgrading.rst
index 402b693..3634a33 100644
--- a/doc/sphinx/whats-new/upgrading.rst
+++ b/doc/sphinx/whats-new/upgrading.rst
@@ -21,9 +21,26 @@ vcl_fetch is now vcl_backend_response
 Directors have been moved to the vmod_directors
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
+To make directors (backend selection logic) easier to extend, the directors
+are now defined in loadable VMODs.
+
+Setting a backend for future fetches in `vcl_recv` is now done as follows::
+
+    sub vcl_init {
+        new cluster1 = directors.round_robin();
+        cluster1.add_backend(b1, 1.0);
+        cluster1.add_backend(b2, 1.0);
+    }
+    sub vcl_recv {
+        set req.backend_hint = cluster1.backend();
+    }
+
+Note the extra `.backend()` needed after the director name.
+
 Use the hash director as a client director
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Since the client director was already a special case of the hash director, it has been removed, and you should use the hash director directly::
+Since the client director was already a special case of the hash director, it
+has been removed, and you should use the hash director directly::
 
     sub vcl_init {
         new h = directors.hash();
@@ -102,6 +119,11 @@ The builtin VCL now honors Cache-Control: no-cache (and friends) to indicate
 uncacheable content from the backend.
 
 
+The `remove` keyword is gone
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Replaced by `unset`.
+
 Changes to parameters
 =====================
 



From martin at varnish-software.com  Tue Apr  1 13:09:54 2014
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Tue, 01 Apr 2014 15:09:54 +0200
Subject: [4.0] c204366 Introduce SLT_Timestamp mechanism for request handling
 timing information.
Message-ID: <E1WUyRu-0003H1-Df@project.varnish-software.com>


commit c2043664d86377909145e1e54f84c58fc40858ad
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Tue Mar 18 17:47:14 2014 +0100

    Introduce SLT_Timestamp mechanism for request handling timing information.

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index 68c74ff..ecc2185 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -48,6 +48,7 @@
 #include <string.h>
 #include <limits.h>
 #include <unistd.h>
+#include <math.h>
 
 #if defined(HAVE_EPOLL_CTL)
 #include <sys/epoll.h>
@@ -674,8 +675,10 @@ struct req {
 
 	char			*ws_req;	/* WS above request data */
 
-	double			t_req;
-	double			t_resp;
+	/* Timestamps */
+	double			t_first;	/* First timestamp logged */
+	double			t_prev;		/* Previous timestamp logged */
+	double			t_req;		/* Headers complete */
 
 	struct http_conn	htc[1];
 	const char		*client_identity;
@@ -1138,6 +1141,17 @@ void VSLbv(struct vsl_log *, enum VSL_tag_e tag, const char *fmt, va_list va);
 void VSLb(struct vsl_log *, enum VSL_tag_e tag, const char *fmt, ...)
     __printflike(3, 4);
 void VSLbt(struct vsl_log *, enum VSL_tag_e tag, txt t);
+void VSLb_ts(struct vsl_log *, const char *event, double first, double *pprev,
+    double now);
+static inline void
+VSLb_ts_req(struct req *req, const char *event, double now)
+{
+
+	if (isnan(req->t_first) || req->t_first == 0.)
+		req->t_first = req->t_prev = now;
+	VSLb_ts(req->vsl, event, req->t_first, &req->t_prev, now);
+}
+
 
 void VSL_Flush(struct vsl_log *, int overflow);
 
diff --git a/bin/varnishd/cache/cache_esi_deliver.c b/bin/varnishd/cache/cache_esi_deliver.c
index 65ad51d..b1c3362 100644
--- a/bin/varnishd/cache/cache_esi_deliver.c
+++ b/bin/varnishd/cache/cache_esi_deliver.c
@@ -35,6 +35,7 @@
 
 #include "cache.h"
 
+#include "vtim.h"
 #include "cache_esi.h"
 #include "vend.h"
 #include "vgz.h"
@@ -98,12 +99,16 @@ ved_include(struct req *preq, const char *src, const char *host)
 	 */
 	req->req_step = R_STP_RECV;
 	req->t_req = preq->t_req;
+	assert(isnan(req->t_first));
+	assert(isnan(req->t_prev));
 	req->gzip_resp = preq->gzip_resp;
 	req->crc = preq->crc;
 	req->l_crc = preq->l_crc;
 
 	THR_SetRequest(req);
 
+	VSLb_ts_req(req, "Start", W_TIM_real(wrk));
+
 	while (1) {
 		req->wrk = wrk;
 		s = CNT_Request(wrk, req);
@@ -122,7 +127,7 @@ ved_include(struct req *preq, const char *src, const char *host)
 	/* Reset the workspace */
 	WS_Reset(wrk->aws, wrk_ws_wm);	/* XXX ? */
 
-	WRW_Reserve(preq->wrk, &preq->sp->fd, preq->vsl, preq->t_resp);
+	WRW_Reserve(preq->wrk, &preq->sp->fd, preq->vsl, preq->t_prev);
 	if (preq->res_mode & RES_CHUNKED)
 		WRW_Chunked(preq->wrk);
 
diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index ecbcf2e..e689913 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -877,6 +877,7 @@ VBF_Fetch(struct worker *wrk, struct req *req, struct objcore *oc,
 			AZ(bo->fetch_objcore->flags & OC_F_BUSY);
 		}
 	}
+	VSLb_ts_req(req, "Fetch", W_TIM_real(wrk));
 	THR_SetBusyobj(NULL);
 	VBO_DerefBusyObj(wrk, &bo);
 }
diff --git a/bin/varnishd/cache/cache_http1_deliver.c b/bin/varnishd/cache/cache_http1_deliver.c
index 231df94..e82ed31 100644
--- a/bin/varnishd/cache/cache_http1_deliver.c
+++ b/bin/varnishd/cache/cache_http1_deliver.c
@@ -282,7 +282,7 @@ V1D_Deliver(struct req *req)
 	if (req->res_mode & RES_ESI)
 		RFC2616_Weaken_Etag(req->resp);
 
-	WRW_Reserve(req->wrk, &req->sp->fd, req->vsl, req->t_resp);
+	WRW_Reserve(req->wrk, &req->sp->fd, req->vsl, req->t_prev);
 
 	/*
 	 * Send HTTP protocol header, unless interior ESI object
@@ -389,7 +389,7 @@ V1D_Deliver_Synth(struct req *req)
 			v1d_dorange(req, r);
 	}
 
-	WRW_Reserve(req->wrk, &req->sp->fd, req->vsl, req->t_resp);
+	WRW_Reserve(req->wrk, &req->sp->fd, req->vsl, req->t_prev);
 
 	/*
 	 * Send HTTP protocol header, unless interior ESI object
diff --git a/bin/varnishd/cache/cache_http1_fsm.c b/bin/varnishd/cache/cache_http1_fsm.c
index 5d0f286..37ed35a 100644
--- a/bin/varnishd/cache/cache_http1_fsm.c
+++ b/bin/varnishd/cache/cache_http1_fsm.c
@@ -103,8 +103,10 @@ http1_wait(struct sess *sp, struct worker *wrk, struct req *req)
 	AZ(req->vcl);
 	AZ(req->obj);
 	AZ(req->esi_level);
+	assert(!isnan(sp->t_idle));
+	assert(isnan(req->t_first));
+	assert(isnan(req->t_prev));
 	assert(isnan(req->t_req));
-	assert(isnan(req->t_resp));
 
 	tmo = (int)(1e3 * cache_param->timeout_linger);
 	while (1) {
@@ -120,7 +122,10 @@ http1_wait(struct sess *sp, struct worker *wrk, struct req *req)
 			hs = HTTP1_Complete(req->htc);
 		if (hs == HTTP1_COMPLETE) {
 			/* Got it, run with it */
-			req->t_req = now;
+			if (isnan(req->t_first))
+				VSLb_ts_req(req, "Start", now);
+			VSLb_ts_req(req, "Req", now);
+			req->t_req = req->t_prev;
 			return (REQ_FSM_MORE);
 		} else if (hs == HTTP1_ERROR_EOF) {
 			why = SC_REM_CLOSE;
@@ -138,7 +143,6 @@ http1_wait(struct sess *sp, struct worker *wrk, struct req *req)
 			when = sp->t_idle + cache_param->timeout_linger;
 			tmo = (int)(1e3 * (when - now));
 			if (when < now || tmo == 0) {
-				req->t_req = NAN;
 				wrk->stats.sess_herd++;
 				SES_ReleaseReq(req);
 				WAIT_Enter(sp);
@@ -146,9 +150,10 @@ http1_wait(struct sess *sp, struct worker *wrk, struct req *req)
 			}
 		} else {
 			/* Working on it */
-			if (isnan(req->t_req))
-				req->t_req = now;
-			when = req->t_req + cache_param->timeout_req;
+			if (isnan(req->t_first))
+				/* Timestamp Start on first byte received */
+				VSLb_ts_req(req, "Start", now);
+			when = sp->t_idle + cache_param->timeout_req;
 			tmo = (int)(1e3 * (when - now));
 			if (when < now || tmo == 0) {
 				why = SC_RX_TIMEOUT;
@@ -195,11 +200,15 @@ http1_cleanup(struct sess *sp, struct worker *wrk, struct req *req)
 		req->vcl = NULL;
 	}
 
-	sp->t_idle = W_TIM_real(wrk);
+	if (!isnan(req->t_prev) && req->t_prev > 0.)
+		sp->t_idle = req->t_prev;
+	else
+		sp->t_idle = W_TIM_real(wrk);
 	VSL_Flush(req->vsl, 0);
 
+	req->t_first = NAN;
+	req->t_prev = NAN;
 	req->t_req = NAN;
-	req->t_resp = NAN;
 	req->req_body_status = REQ_BODY_INIT;
 
 	// req->req_bodybytes = 0;
@@ -226,7 +235,9 @@ http1_cleanup(struct sess *sp, struct worker *wrk, struct req *req)
 	WS_Reset(wrk->aws, NULL);
 
 	if (HTTP1_Reinit(req->htc) == HTTP1_COMPLETE) {
-		req->t_req = sp->t_idle;
+		VSLb_ts_req(req, "Start", sp->t_idle);
+		VSLb_ts_req(req, "Req", sp->t_idle);
+		req->t_req = req->t_prev;
 		wrk->stats.sess_pipeline++;
 		return (SESS_DONE_RET_START);
 	} else {
@@ -522,16 +533,19 @@ HTTP1_IterateReqBody(struct req *req, req_body_iter_f *func, void *priv)
 		l = http1_iter_req_body(req, buf, sizeof buf);
 		if (l < 0) {
 			req->doclose = SC_RX_BODY;
-			return (l);
+			break;
 		}
 		if (l > 0) {
 			i = func(req, priv, buf, l);
 			if (i) {
-				return (i);
+				l = i;
+				break;
 			}
 		}
 	} while (l > 0);
-	return(0);
+	VSLb_ts_req(req, "ReqBody", VTIM_real());
+
+	return (l);
 }
 
 /*----------------------------------------------------------------------
@@ -605,7 +619,8 @@ HTTP1_CacheReqBody(struct req *req, ssize_t maxsize)
 			    req->h1.bytes_yet : cache_param->fetch_chunksize);
 			if (st == NULL) {
 				req->req_body_status = REQ_BODY_FAIL;
-				return (-1);
+				l = -1;
+				break;
 			} else {
 				VTAILQ_INSERT_TAIL(&req->body, st, list);
 			}
@@ -615,11 +630,12 @@ HTTP1_CacheReqBody(struct req *req, ssize_t maxsize)
 		l = http1_iter_req_body(req, st->ptr + st->len, l);
 		if (l < 0) {
 			req->doclose = SC_RX_BODY;
-			return (l);
+			break;
 		}
 		if (req->req_bodybytes > maxsize) {
 			req->req_body_status = REQ_BODY_FAIL;
-			return (-1);
+			l = -1;
+			break;
 		}
 		if (l > 0) {
 			st->len += l;
@@ -627,6 +643,8 @@ HTTP1_CacheReqBody(struct req *req, ssize_t maxsize)
 				st = NULL;
 		}
 	} while (l > 0);
-	req->req_body_status = REQ_BODY_CACHED;
-	return(0);
+	if (l == 0)
+		req->req_body_status = REQ_BODY_CACHED;
+	VSLb_ts_req(req, "ReqBody", VTIM_real());
+	return (l);
 }
diff --git a/bin/varnishd/cache/cache_pipe.c b/bin/varnishd/cache/cache_pipe.c
index b4403bb..1a0a44f 100644
--- a/bin/varnishd/cache/cache_pipe.c
+++ b/bin/varnishd/cache/cache_pipe.c
@@ -89,14 +89,14 @@ PipeRequest(struct req *req, struct busyobj *bo)
 
 	i = WRW_FlushRelease(wrk);
 
+	VSLb_ts_req(req, "Pipe", W_TIM_real(wrk));
+
 	if (i) {
 		SES_Close(req->sp, SC_TX_PIPE);
 		VDI_CloseFd(&vc);
 		return;
 	}
 
-	req->t_resp = VTIM_real();
-
 	memset(fds, 0, sizeof fds);
 
 	// XXX: not yet (void)VTCP_linger(vc->fd, 0);
@@ -130,6 +130,7 @@ PipeRequest(struct req *req, struct busyobj *bo)
 			fds[1].fd = -1;
 		}
 	}
+	VSLb_ts_req(req, "PipeSess", W_TIM_real(wrk));
 	SES_Close(req->sp, SC_TX_PIPE);
 	VDI_CloseFd(&vc);
 	bo->vbc = NULL;
diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index fd9783c..9826ae7 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -88,6 +88,7 @@ static enum req_fsm_nxt
 cnt_deliver(struct worker *wrk, struct req *req)
 {
 	char time_str[30];
+	double now;
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
@@ -99,9 +100,10 @@ cnt_deliver(struct worker *wrk, struct req *req)
 
 	assert(req->obj->objcore->refcnt > 0);
 
-	req->t_resp = W_TIM_real(wrk);
+	now = W_TIM_real(wrk);
+	VSLb_ts_req(req, "Process", now);
 	if (req->obj->objcore->exp_flags & OC_EF_EXP)
-		EXP_Touch(req->obj->objcore, req->t_resp);
+		EXP_Touch(req->obj->objcore, now);
 
 	HTTP_Setup(req->resp, req->ws, req->vsl, SLT_RespMethod);
 
@@ -109,7 +111,7 @@ cnt_deliver(struct worker *wrk, struct req *req)
 	http_FilterResp(req->obj->http, req->resp, 0);
 
 	http_Unset(req->resp, H_Date);
-	VTIM_format(req->t_resp, time_str);
+	VTIM_format(now, time_str);
 	http_PrintfHeader(req->resp, "Date: %s", time_str);
 
 	if (req->wrk->stats.cache_hit)
@@ -121,7 +123,7 @@ cnt_deliver(struct worker *wrk, struct req *req)
 		    "X-Varnish: %u", req->vsl->wid & VSL_IDENTMASK);
 
 	http_PrintfHeader(req->resp, "Age: %.0f",
-	    req->t_resp - req->obj->exp.t_origin);
+	    now - req->obj->exp.t_origin);
 
 	http_SetHeader(req->resp, "Via: 1.1 varnish (v4)");
 
@@ -152,6 +154,7 @@ cnt_deliver(struct worker *wrk, struct req *req)
 		http_SetResp(req->resp, "HTTP/1.1", 304, "Not Modified");
 
 	V1D_Deliver(req);
+	VSLb_ts_req(req, "Resp", W_TIM_real(wrk));
 
 	if (http_HdrIs(req->resp, H_Connection, "close"))
 		req->doclose = SC_RESP_CLOSE;
@@ -192,6 +195,7 @@ cnt_synth(struct worker *wrk, struct req *req)
 {
 	char date[40];
 	struct http *h;
+	double now;
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
@@ -200,7 +204,9 @@ cnt_synth(struct worker *wrk, struct req *req)
 
 	HTTP_Setup(req->resp, req->ws, req->vsl, SLT_RespMethod);
 	h = req->resp;
-	req->t_resp = VTIM_real();
+
+	now = W_TIM_real(wrk);
+	VSLb_ts_req(req, "Process", now);
 
 	if (req->err_code < 100 || req->err_code > 999)
 		req->err_code = 501;
@@ -208,7 +214,7 @@ cnt_synth(struct worker *wrk, struct req *req)
 	http_ClrHeader(h);
 	http_PutProtocol(h, "HTTP/1.1");
 	http_PutStatus(h, req->err_code);
-	VTIM_format(req->t_resp, date);
+	VTIM_format(now, date);
 	http_PrintfHeader(h, "Date: %s", date);
 	http_SetHeader(h, "Server: Varnish");
 	http_PrintfHeader(req->resp,
@@ -242,6 +248,8 @@ cnt_synth(struct worker *wrk, struct req *req)
 
 	V1D_Deliver_Synth(req);
 
+	VSLb_ts_req(req, "Resp", W_TIM_real(wrk));
+
 	VSB_delete(req->synth_body);
 	req->synth_body = NULL;
 
@@ -326,6 +334,7 @@ cnt_lookup(struct worker *wrk, struct req *req)
 	struct object *o;
 	struct objhead *oh;
 	enum lookup_e lr;
+	int had_objhead = 0;
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
@@ -336,6 +345,8 @@ cnt_lookup(struct worker *wrk, struct req *req)
 	VRY_Prep(req);
 
 	AZ(req->objcore);
+	if (req->hash_objhead)
+		had_objhead = 1;
 	lr = HSH_Lookup(req, &oc, &boc,
 	    req->esi_level == 0 ? 1 : 0,
 	    req->hash_always_miss ? 1 : 0
@@ -349,6 +360,8 @@ cnt_lookup(struct worker *wrk, struct req *req)
 		 */
 		return (REQ_FSM_DISEMBARK);
 	}
+	if (had_objhead)
+		VSLb_ts_req(req, "Waitinglist", W_TIM_real(wrk));
 
 	if (boc == NULL) {
 		VRY_Finish(req, DISCARD);
@@ -631,12 +644,14 @@ cnt_restart(struct worker *wrk, struct req *req)
 	} else {
 		wid = VXID_Get(&wrk->vxid_pool);
 		// XXX: ReqEnd + ReqAcct ?
+		VSLb_ts_req(req, "Restart", W_TIM_real(wrk));
 		VSLb(req->vsl, SLT_Link, "req %u restart", wid);
 		VSLb(req->vsl, SLT_End, "%s", "");
 		VSL_Flush(req->vsl, 0);
 		owid = req->vsl->wid & VSL_IDENTMASK;
 		req->vsl->wid = wid | VSL_CLIENTMARKER;
 		VSLb(req->vsl, SLT_Begin, "req %u restart", owid);
+		VSLb_ts_req(req, "Start", req->t_prev);
 		req->err_code = 0;
 		req->req_step = R_STP_RECV;
 	}
@@ -677,6 +692,10 @@ cnt_recv(struct worker *wrk, struct req *req)
 	AZ(req->obj);
 	AZ(req->objcore);
 
+	assert(!isnan(req->t_first));
+	assert(!isnan(req->t_prev));
+	assert(!isnan(req->t_req));
+
 	VSLb(req->vsl, SLT_ReqStart, "%s %s",
 	    req->sp->client_addr_str, req->sp->client_port_str);
 
@@ -882,12 +901,6 @@ CNT_Request(struct worker *wrk, struct req *req)
 			VSLb(req->vsl, SLT_Length, "%ju",
 			    (uintmax_t)req->resp_bodybytes);
 		}
-		VSLb(req->vsl, SLT_ReqEnd, "%.9f %.9f %.9f %.9f %.9f",
-		    req->t_req,
-		    req->sp->t_idle,
-		    req->sp->t_idle - req->t_resp,
-		    req->t_resp - req->t_req,
-		    req->sp->t_idle - req->t_resp);
 
 		while (!VTAILQ_EMPTY(&req->body)) {
 			st = VTAILQ_FIRST(&req->body);
diff --git a/bin/varnishd/cache/cache_session.c b/bin/varnishd/cache/cache_session.c
index 96a2507..73ed5c7 100644
--- a/bin/varnishd/cache/cache_session.c
+++ b/bin/varnishd/cache/cache_session.c
@@ -399,8 +399,9 @@ SES_GetReq(struct worker *wrk, struct sess *sp)
 
 	WS_Init(req->ws, "req", p, e - p);
 
+	req->t_first = NAN;
+	req->t_prev = NAN;
 	req->t_req = NAN;
-	req->t_resp = NAN;
 
 	VTAILQ_INIT(&req->body);
 
diff --git a/bin/varnishd/cache/cache_shmlog.c b/bin/varnishd/cache/cache_shmlog.c
index 81f5a03..19048c2 100644
--- a/bin/varnishd/cache/cache_shmlog.c
+++ b/bin/varnishd/cache/cache_shmlog.c
@@ -350,6 +350,17 @@ VSLb(struct vsl_log *vsl, enum VSL_tag_e tag, const char *fmt, ...)
 	va_end(ap);
 }
 
+void
+VSLb_ts(struct vsl_log *vsl, const char *event, double first, double *pprev,
+    double now)
+{
+
+	assert(!isnan(now) && now != 0.);
+	VSLb(vsl, SLT_Timestamp, "%s: %.9f %.9f %.9f",
+	    event, now, now - first, now - *pprev);
+	*pprev = now;
+}
+
 /*--------------------------------------------------------------------
  * Setup a VSL buffer, allocate space if none provided.
  */
diff --git a/include/tbl/vsl_tags.h b/include/tbl/vsl_tags.h
index 5f768ec..34c5c13 100644
--- a/include/tbl/vsl_tags.h
+++ b/include/tbl/vsl_tags.h
@@ -462,4 +462,21 @@ SLTM(Storage, 0, "Where object is stored",
 	"\t+---- Type (\"malloc\", \"file\", \"persistent\" etc.)\n"
 	"\n"
 )
+
+SLTM(Timestamp, 0, "Timing information",
+	"Contains timing information for the Varnish worker threads.\n\n"
+	"Time stamps are issued by Varnish on certain events,"
+	" and show the absolute time of the event, the time spent since the"
+	" start of the work unit, and the time spent since the last timestamp"
+	" was logged.\n\n"
+	"The format is::\n\n"
+	"\t%s: %f %f %f\n"
+	"\t|   |  |  |\n"
+	"\t|   |  |  +- Time since last timestamp\n"
+	"\t|   |  +---- Time since start of work unit\n"
+	"\t|   +------- Absolute time of event\n"
+	"\t+----------- Event label\n"
+	"\n"
+)
+
 #undef NODEF_NOTICE



From martin at varnish-software.com  Tue Apr  1 13:09:54 2014
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Tue, 01 Apr 2014 15:09:54 +0200
Subject: [4.0] dc66d1c Reduce logged timer precision to microseconds
Message-ID: <E1WUyRu-0003HB-FD@project.varnish-software.com>


commit dc66d1cbdcba1e5ac2ccf428ac147d2ed6788bb2
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Mon Mar 24 14:34:15 2014 +0100

    Reduce logged timer precision to microseconds

diff --git a/bin/varnishd/cache/cache_shmlog.c b/bin/varnishd/cache/cache_shmlog.c
index 19048c2..900ecda 100644
--- a/bin/varnishd/cache/cache_shmlog.c
+++ b/bin/varnishd/cache/cache_shmlog.c
@@ -356,7 +356,7 @@ VSLb_ts(struct vsl_log *vsl, const char *event, double first, double *pprev,
 {
 
 	assert(!isnan(now) && now != 0.);
-	VSLb(vsl, SLT_Timestamp, "%s: %.9f %.9f %.9f",
+	VSLb(vsl, SLT_Timestamp, "%s: %.6f %.6f %.6f",
 	    event, now, now - first, now - *pprev);
 	*pprev = now;
 }



From martin at varnish-software.com  Tue Apr  1 13:09:54 2014
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Tue, 01 Apr 2014 15:09:54 +0200
Subject: [4.0] e15f957 Backend fetch timestamps
Message-ID: <E1WUyRu-0003HK-Ga@project.varnish-software.com>


commit e15f9573f1536497716e4f6d08b58b427e4db36c
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Tue Mar 25 16:28:31 2014 +0100

    Backend fetch timestamps

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index ecc2185..ea5d990 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -569,11 +569,8 @@ struct busyobj {
 	double			between_bytes_timeout;
 
 	/* Timers */
-	double			t_start;
-	double			t_send;
-	double			t_sent;
-	double			t_hdr;
-	double			t_body;
+	double			t_first;	/* First timestamp logged */
+	double			t_prev;		/* Previous timestamp logged */
 
 	const char		*storage_hint;
 	struct director		*director;
@@ -1143,6 +1140,7 @@ void VSLb(struct vsl_log *, enum VSL_tag_e tag, const char *fmt, ...)
 void VSLbt(struct vsl_log *, enum VSL_tag_e tag, txt t);
 void VSLb_ts(struct vsl_log *, const char *event, double first, double *pprev,
     double now);
+
 static inline void
 VSLb_ts_req(struct req *req, const char *event, double now)
 {
@@ -1152,6 +1150,15 @@ VSLb_ts_req(struct req *req, const char *event, double now)
 	VSLb_ts(req->vsl, event, req->t_first, &req->t_prev, now);
 }
 
+static inline void
+VSLb_ts_busyobj(struct busyobj *bo, const char *event, double now)
+{
+
+	if (isnan(bo->t_first) || bo->t_first == 0.)
+		bo->t_first = bo->t_prev = now;
+	VSLb_ts(bo->vsl, event, bo->t_first, &bo->t_prev, now);
+}
+
 
 void VSL_Flush(struct vsl_log *, int overflow);
 
diff --git a/bin/varnishd/cache/cache_busyobj.c b/bin/varnishd/cache/cache_busyobj.c
index db1c7a1..69f849b 100644
--- a/bin/varnishd/cache/cache_busyobj.c
+++ b/bin/varnishd/cache/cache_busyobj.c
@@ -149,6 +149,8 @@ VBO_GetBusyObj(struct worker *wrk, const struct req *req)
 	VCL_Ref(bo->vcl);
 
 	bo->t_fetch = req->t_req;
+	assert(!isnan(bo->t_fetch) && bo->t_fetch != 0.);
+	bo->t_first = bo->t_prev = NAN;
 
 	return (bo);
 }
diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index e689913..cd627cd 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -226,6 +226,8 @@ vbf_stp_retry(struct worker *wrk, struct busyobj *bo)
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
 
+	VSLb_ts_busyobj(bo, "Retry", W_TIM_real(wrk));
+
 	// XXX: BereqEnd + BereqAcct ?
 	wid = VXID_Get(&wrk->vxid_pool);
 	VSLb(bo->vsl, SLT_Link, "bereq %u retry", wid);
@@ -234,6 +236,7 @@ vbf_stp_retry(struct worker *wrk, struct busyobj *bo)
 	owid = bo->vsl->wid & VSL_IDENTMASK;
 	bo->vsl->wid = wid | VSL_BACKENDMARKER;
 	VSLb(bo->vsl, SLT_Begin, "bereq %u retry", owid);
+	VSLb_ts_busyobj(bo, "Start", bo->t_prev);
 
 	return (F_STP_STARTFETCH);
 }
@@ -285,9 +288,11 @@ vbf_stp_startfetch(struct worker *wrk, struct busyobj *bo)
 	 * Do a single retry in that case.
 	 */
 	if (i == 1) {
+		VSLb_ts_busyobj(bo, "Beresp", W_TIM_real(wrk));
 		VSC_C_main->backend_retry++;
 		i = V1F_fetch_hdr(wrk, bo, bo->req);
 	}
+	VSLb_ts_busyobj(bo, "Beresp", W_TIM_real(wrk));
 
 	if (i) {
 		AZ(bo->vbc);
@@ -364,8 +369,6 @@ vbf_stp_startfetch(struct worker *wrk, struct busyobj *bo)
 
 	assert(wrk->handling == VCL_RET_DELIVER);
 
-	bo->t_body = VTIM_mono();
-
 	return (do_ims ? F_STP_CONDFETCH : F_STP_FETCH);
 }
 
@@ -525,6 +528,7 @@ vbf_stp_fetch(struct worker *wrk, struct busyobj *bo)
 		assert(bo->state == BOS_REQ_DONE);
 		HSH_Unbusy(&wrk->stats, obj->objcore);
 	}
+	VSLb_ts_busyobj(bo, "BerespBody", W_TIM_real(wrk));
 	VBO_setstate(bo, BOS_FINISHED);
 	return (F_STP_DONE);
 }
@@ -617,6 +621,7 @@ vbf_stp_condfetch(struct worker *wrk, struct busyobj *bo)
 	assert(obj->len == al);
 	EXP_Rearm(bo->ims_obj, bo->ims_obj->exp.t_origin, 0, 0, 0);
 	VBO_setstate(bo, BOS_FINISHED);
+	VSLb_ts_busyobj(bo, "BerespBody", W_TIM_real(wrk));
 	return (F_STP_DONE);
 }
 
@@ -633,6 +638,8 @@ vbf_stp_error(struct worker *wrk, struct busyobj *bo)
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
 
+	VSLb_ts_busyobj(bo, "Error", W_TIM_real(wrk));
+
 	AN(bo->fetch_objcore->flags & OC_F_BUSY);
 
 	AZ(bo->synth_body);
@@ -720,7 +727,6 @@ vbf_fetch_thread(struct worker *wrk, void *priv)
 {
 	struct busyobj *bo;
 	enum fetch_step stp;
-	double t_hdr, t_body;
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CAST_OBJ_NOTNULL(bo, priv, BUSYOBJ_MAGIC);
@@ -729,11 +735,9 @@ vbf_fetch_thread(struct worker *wrk, void *priv)
 
 	THR_SetBusyobj(bo);
 	stp = F_STP_MKBEREQ;
-	bo->t_start = VTIM_real();
-	bo->t_send = NAN;
-	bo->t_sent = NAN;
-	bo->t_hdr = NAN;
-	bo->t_body = NAN;
+	assert(isnan(bo->t_first));
+	assert(isnan(bo->t_prev));
+	VSLb_ts_busyobj(bo, "Start", W_TIM_real(wrk));
 
 	bo->stats = &wrk->stats;
 
@@ -790,14 +794,6 @@ vbf_fetch_thread(struct worker *wrk, void *priv)
 	if (bo->ims_obj != NULL)
 		(void)HSH_DerefObj(&wrk->stats, &bo->ims_obj);
 
-	t_hdr = bo->t_hdr - bo->t_sent;
-	t_body = bo->t_body - bo->t_hdr;
-	VSLb(bo->vsl, SLT_BereqEnd, "%.9f %.9f %.9f %.9f %.9f %.9f",
-	     bo->t_start,
-	     VTIM_real(),
-	     bo->t_sent - bo->t_send,
-	     t_hdr, t_body, t_hdr + t_body);
-
 	VBO_DerefBusyObj(wrk, &bo);
 	THR_SetBusyobj(NULL);
 }
diff --git a/bin/varnishd/cache/cache_http1_fetch.c b/bin/varnishd/cache/cache_http1_fetch.c
index e25ced9..317bfcb 100644
--- a/bin/varnishd/cache/cache_http1_fetch.c
+++ b/bin/varnishd/cache/cache_http1_fetch.c
@@ -311,8 +311,7 @@ V1F_fetch_hdr(struct worker *wrk, struct busyobj *bo, struct req *req)
 		VDI_AddHostHeader(bo->bereq, vc);
 
 	(void)VTCP_blocking(vc->fd);	/* XXX: we should timeout instead */
-	WRW_Reserve(wrk, &vc->fd, bo->vsl, bo->t_fetch);
-	bo->t_send = VTIM_mono();
+	WRW_Reserve(wrk, &vc->fd, bo->vsl, bo->t_prev);
 	(void)HTTP1_Write(wrk, hp, 0);	/* XXX: stats ? */
 
 	/* Deal with any message-body the request might (still) have */
@@ -334,13 +333,14 @@ V1F_fetch_hdr(struct worker *wrk, struct busyobj *bo, struct req *req)
 	if (WRW_FlushRelease(wrk) || i != 0) {
 		VSLb(bo->vsl, SLT_FetchError, "backend write error: %d (%s)",
 		    errno, strerror(errno));
+		VSLb_ts_busyobj(bo, "Bereq", W_TIM_real(wrk));
 		VDI_CloseFd(&bo->vbc);
 		/* XXX: other cleanup ? */
 		return (retry);
 	}
+	VSLb_ts_busyobj(bo, "Bereq", W_TIM_real(wrk));
 
 	VSC_C_main->backend_req++;
-	bo->t_sent = VTIM_mono();
 
 	/* Receive response */
 
@@ -377,7 +377,6 @@ V1F_fetch_hdr(struct worker *wrk, struct busyobj *bo, struct req *req)
 			    vc->between_bytes_timeout);
 		}
 	} while (hs != HTTP1_COMPLETE);
-	bo->t_hdr = VTIM_mono();
 
 	hp = bo->beresp;
 



From martin at varnish-software.com  Tue Apr  1 13:09:54 2014
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Tue, 01 Apr 2014 15:09:54 +0200
Subject: [4.0] 7b5ee1a Timestamp documentation
Message-ID: <E1WUyRu-0003HU-Hq@project.varnish-software.com>


commit 7b5ee1a64c41dbe0044a74a6a9ea562309a28f04
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Tue Mar 25 16:28:58 2014 +0100

    Timestamp documentation

diff --git a/doc/sphinx/reference/vsl.rst b/doc/sphinx/reference/vsl.rst
index 78c3456..36d51bd 100644
--- a/doc/sphinx/reference/vsl.rst
+++ b/doc/sphinx/reference/vsl.rst
@@ -20,6 +20,83 @@ VSL tags
 
 .. include:: ../../../lib/libvarnishapi/vsl-tags.rst
 
+TIMESTAMPS
+==========
+
+Timestamps are inserted in the log on completing certain events during
+the worker thread's task handling. The timestamps has a label showing
+which event was completed. The reported fields show the absolute time
+of the event, the time spent since the start of the task and the time
+spent since the last timestamp was logged.
+
+The timestamps logged automatically by Varnish are inserted after
+completing events that are expected to have delays (e.g. network IO or
+spending time on a waitinglist). Timestamps can also be inserted from
+VCL using the std.timestamp() method. If one is doing time consuming
+tasks in the VCL configuration, it's a good idea to log a timestamp
+after completing that task. This keeps the timing information in
+subsequent timestamps from including the time spent on the VCL event.
+
+Request handling timestamps
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Start
+	The start of request processing (first byte received or
+	restart).
+
+Req
+	Complete client request received.
+
+ReqBody
+	Client request body processed (discarded, cached or passed to
+	the backend).
+
+Waitinglist
+	Came off waitinglist.
+
+Fetch
+	Fetch processing finished (completely fetched or ready for
+	streaming).
+
+Process
+	Processing finished, ready to deliver the client response.
+
+Resp
+	Delivery of response to the client finished.
+
+Restart
+	Client request is being restarted.
+
+Pipe handling timestamps
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+Pipe
+	Opened a pipe to the backend and forwarded the request.
+
+PipeSess
+	The pipe session has finished.
+
+Backend fetch timestamps
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+Start
+	Start of the backend fetch processing.
+
+Bereq
+	Backend request sent.
+
+Beresp
+	Backend response headers received.
+
+BerespBody
+	Backend response body received.
+
+Retry
+	Backend request is being retried.
+
+Error
+	Backend request failed to vcl_error.
+
 
 HISTORY
 =======
diff --git a/include/tbl/vsl_tags.h b/include/tbl/vsl_tags.h
index 34c5c13..0f50a48 100644
--- a/include/tbl/vsl_tags.h
+++ b/include/tbl/vsl_tags.h
@@ -468,7 +468,8 @@ SLTM(Timestamp, 0, "Timing information",
 	"Time stamps are issued by Varnish on certain events,"
 	" and show the absolute time of the event, the time spent since the"
 	" start of the work unit, and the time spent since the last timestamp"
-	" was logged.\n\n"
+	" was logged. See vsl(7) for information about the individual"
+	" timestamps.\n\n"
 	"The format is::\n\n"
 	"\t%s: %f %f %f\n"
 	"\t|   |  |  |\n"



From martin at varnish-software.com  Tue Apr  1 13:09:54 2014
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Tue, 01 Apr 2014 15:09:54 +0200
Subject: [4.0] 7e2ac56 Add an XXX comment about turning off unnecessary
 timestamps
Message-ID: <E1WUyRu-0003Hd-Ip@project.varnish-software.com>


commit 7e2ac56eee82982ea7d0e65c4bd9b12f23fea543
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Tue Mar 25 17:16:15 2014 +0100

    Add an XXX comment about turning off unnecessary timestamps

diff --git a/bin/varnishd/cache/cache_shmlog.c b/bin/varnishd/cache/cache_shmlog.c
index 900ecda..af78bfa 100644
--- a/bin/varnishd/cache/cache_shmlog.c
+++ b/bin/varnishd/cache/cache_shmlog.c
@@ -355,6 +355,10 @@ VSLb_ts(struct vsl_log *vsl, const char *event, double first, double *pprev,
     double now)
 {
 
+	/* XXX: Make an option to turn off some unnecessary timestamp
+	   logging. This must be done carefully because some functions
+	   (e.g. WRW_Reserve) takes the last timestamp as it's inital
+	   value for timeout calculation. */
 	assert(!isnan(now) && now != 0.);
 	VSLb(vsl, SLT_Timestamp, "%s: %.6f %.6f %.6f",
 	    event, now, now - first, now - *pprev);



From martin at varnish-software.com  Tue Apr  1 13:09:54 2014
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Tue, 01 Apr 2014 15:09:54 +0200
Subject: [4.0] acc49fd Fix test cases to pass with timestamp records present
Message-ID: <E1WUyRu-0003Hm-K8@project.varnish-software.com>


commit acc49fdda0cbefa8722a4b9ce4975ee516ae7d0b
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Tue Mar 25 17:43:48 2014 +0100

    Fix test cases to pass with timestamp records present

diff --git a/bin/varnishtest/tests/l00000.vtc b/bin/varnishtest/tests/l00000.vtc
index 09423cc..1c732fc 100644
--- a/bin/varnishtest/tests/l00000.vtc
+++ b/bin/varnishtest/tests/l00000.vtc
@@ -16,14 +16,13 @@ logexpect l1 -v v1 -g session {
 	expect 0 =	End
 
 	expect 0 *	Begin		"req 1000"
-	expect 0 =	ReqStart
+	expect * =	ReqStart
 	expect 0 =	ReqMethod	GET
 	expect 0 =	ReqURL		/
 	expect 0 =	ReqProtocol	HTTP/1.1
 	expect * =	ReqHeader	"Foo: bar"
 	expect * =	Link		bereq
-	expect * =	ReqEnd
-	expect 0 =	End
+	expect * =	End
 
 	expect 0 1002	Begin		"bereq 1001"
 	expect * =	End
diff --git a/bin/varnishtest/tests/l00001.vtc b/bin/varnishtest/tests/l00001.vtc
index a161c7b..b1ec63c 100644
--- a/bin/varnishtest/tests/l00001.vtc
+++ b/bin/varnishtest/tests/l00001.vtc
@@ -22,175 +22,150 @@ client c1 {
 # Test 'eq' operator
 logexpect l1 -d 1 -g vxid -q "Begin eq 'req 1000 rxreq'" {
 	expect 0 *	Begin	req
-	expect * =	ReqEnd
 	expect * =	End
 } -run
 
 # Test 'ne' operator
 logexpect l1 -d 1 -g vxid -q "ReqProtocol ne 'HTTP/1.0'" {
 	expect 0 *	Begin	req
-	expect * =	ReqEnd
 	expect * =	End
 } -run
 
 # Test '==' operator on integers
 logexpect l1 -d 1 -g vxid -q "RespStatus == 200" {
 	expect 0 *	Begin	req
-	expect * =	ReqEnd
 	expect * =	End
 } -run
 
 # Test '==' operator on floats
 logexpect l1 -d 1 -g vxid -q "RespStatus == 200." {
 	expect 0 *	Begin	req
-	expect * =	ReqEnd
 	expect * =	End
 } -run
 
 # Test '!=' operator on integers
 logexpect l1 -d 1 -g vxid -q "RespStatus != 503" {
 	expect 0 *	Begin	req
-	expect * =	ReqEnd
 	expect * =	End
 } -run
 
 # Test '!=' operator on floats
 logexpect l1 -d 1 -g vxid -q "RespStatus != 503." {
 	expect 0 *	Begin	req
-	expect * =	ReqEnd
 	expect * =	End
 } -run
 
 # Test '<' operator on integers
 logexpect l1 -d 1 -g vxid -q "RespStatus < 201" {
 	expect 0 *	Begin	req
-	expect * =	ReqEnd
 	expect * =	End
 } -run
 
 # Test '<' operator on floats
 logexpect l1 -d 1 -g vxid -q "RespStatus < 201." {
 	expect 0 *	Begin	req
-	expect * =	ReqEnd
 	expect * =	End
 } -run
 
 # Test '>' operator on integers
 logexpect l1 -d 1 -g vxid -q "RespStatus > 199" {
 	expect 0 *	Begin	req
-	expect * =	ReqEnd
 	expect * =	End
 } -run
 
 # Test '>' operator on floats
 logexpect l1 -d 1 -g vxid -q "RespStatus > 199." {
 	expect 0 *	Begin	req
-	expect * =	ReqEnd
 	expect * =	End
 } -run
 
 # Test '<=' operator on integers
 logexpect l1 -d 1 -g vxid -q "RespStatus <= 200" {
 	expect 0 *	Begin	req
-	expect * =	ReqEnd
 	expect * =	End
 } -run
 
 # Test '<=' operator on floats
 logexpect l1 -d 1 -g vxid -q "RespStatus <= 200." {
 	expect 0 *	Begin	req
-	expect * =	ReqEnd
 	expect * =	End
 } -run
 
 # Test '>=' operator on integers
 logexpect l1 -d 1 -g vxid -q "RespStatus >= 200" {
 	expect 0 *	Begin	req
-	expect * =	ReqEnd
 	expect * =	End
 } -run
 
 # Test '>=' operator on floats
 logexpect l1 -d 1 -g vxid -q "RespStatus >= 200." {
 	expect 0 *	Begin	req
-	expect * =	ReqEnd
 	expect * =	End
 } -run
 
 # Test '~' operator
 logexpect l1 -d 1 -g vxid -q "RespStatus ~ '^200$'" {
 	expect 0 *	Begin	req
-	expect * =	ReqEnd
 	expect * =	End
 } -run
 
 # Test '!~' operator
 logexpect l1 -d 1 -g vxid -q "RespStatus !~ '^404$'" {
 	expect 0 *	Begin	req
-	expect * =	ReqEnd
 	expect * =	End
 } -run
 
 # Test boolean and
 logexpect l1 -d 1 -g vxid -q "RespStatus == 200 and RespStatus ~ '^200$'" {
 	expect 0 *	Begin	req
-	expect * =	ReqEnd
 	expect * =	End
 } -run
 
 # Test boolean or
 logexpect l1 -d 1 -g vxid -q "RespStatus == 404 or RespStatus ~ '^200$'" {
 	expect 0 *	Begin	req
-	expect * =	ReqEnd
 	expect * =	End
 } -run
 
 # Test boolean not
 logexpect l1 -d 1 -g vxid -q "RespStatus == 404 or not RespStatus ~ '^404$'" {
 	expect 0 *	Begin	req
-	expect * =	ReqEnd
 	expect * =	End
 } -run
 
 # Test grouping
 logexpect l1 -d 1 -g vxid -q "(RespStatus == 200 or RespStatus == 404) and RespStatus == 200" {
 	expect 0 *	Begin	req
-	expect * =	ReqEnd
 	expect * =	End
 } -run
 
 # Test and/or precedence
 logexpect l1 -d 1 -g vxid -q "RespStatus == 200 or RespStatus == 503 and RespStatus == 404" {
 	  expect 0 *	Begin	req
-	  expect * =	ReqEnd
 	  expect * =	End
 } -run
 
 # Test field
 logexpect l1 -d 1 -g vxid -q "RespHeader[2] == 123" {
 	  expect 0 *	Begin	req
-	  expect * =	ReqEnd
 	  expect * =	End
 } -run
 
 # Test taglists
 logexpect l1 -d 1 -g vxid -q "Debug,Resp* == 200" {
 	  expect 0 *	Begin	req
-	  expect * =	ReqEnd
 	  expect * =	End
 } -run
 
 # Test record prefix
 logexpect l1 -d 1 -g vxid -q "Resp*:x-test eq '123 321'" {
 	  expect 0 *	Begin	req
-	  expect * =	ReqEnd
 	  expect * =	End
 } -run
 
 # Test tag presence (no operator)
 logexpect l1 -d 1 -g vxid -q "RespStatus" {
 	  expect 0 *	Begin	req
-	  expect * =	ReqEnd
 	  expect * =	End
 } -run
 



From martin at varnish-software.com  Tue Apr  1 13:09:54 2014
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Tue, 01 Apr 2014 15:09:54 +0200
Subject: [4.0] d8c8c66 Timestamp test case
Message-ID: <E1WUyRu-0003Hw-LF@project.varnish-software.com>


commit d8c8c66d4c4d474af66997ece6491483f8a17cbb
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Wed Mar 26 12:27:19 2014 +0100

    Timestamp test case

diff --git a/bin/varnishtest/tests/s00004.vtc b/bin/varnishtest/tests/s00004.vtc
new file mode 100644
index 0000000..169d6ab
--- /dev/null
+++ b/bin/varnishtest/tests/s00004.vtc
@@ -0,0 +1,65 @@
+varnishtest "Timestamps"
+
+server s1 {
+	rxreq
+	expect req.url == "/1"
+	delay 1
+	txresp -nolen -hdr "Transfer-Encoding: chunked"
+	delay 1
+	chunkedlen 1000
+	chunkedlen 0
+
+	rxreq
+	expect req.url == "/2"
+	txresp
+} -start
+
+varnish v1 -vcl+backend {
+	sub vcl_backend_response {
+		set beresp.do_stream = false;
+	}
+	sub vcl_deliver {
+		if (req.url == "/1" && req.restarts == 0) {
+			return (restart);
+		}
+	}
+} -start
+
+logexpect l1 -v v1 -g request {
+	expect 0 1001	Begin		req
+	expect * =	Timestamp	{Start: \S+ 0\.000000 0\.000000}
+	expect * =	Timestamp	{Req: \S+ 0\.\d+ 0\.\d+}
+	expect * =	Timestamp	{Fetch: \S+ 2\.\d+ 2\.\d+}
+	expect * =	Timestamp	{Process: \S+ 2\.\d+ 0\.\d+}
+	expect * =	Timestamp	{Restart: \S+ 2\.\d+ 0\.\d+}
+	expect * =	End
+	expect 0 1002	Begin		bereq
+	expect * =	Timestamp	{Start: \S+ 0\.000000 0\.000000}
+	expect * =	Timestamp	{Bereq: \S+ 0\.\d+ 0\.\d+}
+	expect * =	Timestamp	{Beresp: \S+ 1\.\d+ 1\.\d+}
+	expect * =	Timestamp	{BerespBody: \S+ 2\.\d+ 1\.\d+}
+	expect * =	End
+	expect 0 1003	Begin		req 1001 restart
+	expect * =	Timestamp	{Start: \S+ 2\.\d+ 0\.\d+}
+	expect * =	Timestamp	{Process: \S+ 2\.\d+ 0\.\d+}
+	expect * =	Timestamp	{Resp: \S+ 2\.\d+ 0\.\d+}
+	expect * =	End
+
+	expect 0 1004	Begin		req
+	expect * =	Timestamp	{Start: \S+ 0\.000000 0\.000000}
+	expect * =	Timestamp	{Req: \S+ 0\.\d+ 0\.\d+}
+	expect * =	Timestamp	{ReqBody: \S+ 0\.\d+ 0\.\d+}
+	expect * =	Timestamp	{Fetch: \S+ 0\.\d+ 0\.\d+}
+	expect * =	Timestamp	{Resp: \S+ 0\.\d+ 0\.\d+}
+	expect * =	End
+} -start
+
+client c1 {
+	txreq -url "/1"
+	rxresp
+	delay 1
+	txreq -req "POST" -url "/2" -body "asdf"
+	rxresp
+} -run
+
+logexpect l1 -wait



From martin at varnish-software.com  Tue Apr  1 13:09:54 2014
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Tue, 01 Apr 2014 15:09:54 +0200
Subject: [4.0] 064f034 Add std.timestamp() method
Message-ID: <E1WUyRu-0003I5-MV@project.varnish-software.com>


commit 064f03498b6d9f11156a1f5ab0648b992495a08b
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Wed Mar 26 13:22:01 2014 +0100

    Add std.timestamp() method

diff --git a/bin/varnishtest/tests/m00013.vtc b/bin/varnishtest/tests/m00013.vtc
new file mode 100644
index 0000000..c3a6295
--- /dev/null
+++ b/bin/varnishtest/tests/m00013.vtc
@@ -0,0 +1,44 @@
+varnishtest "Timestamps from std vmod"
+
+server s1 {
+	rxreq
+	txresp
+} -start
+
+varnish v1 -vcl+backend {
+	import ${vmod_std};
+
+	sub vcl_init {
+		# Make sure that this doesn't cause ill effects
+		std.timestamp("bogus");
+	}
+
+	sub vcl_recv {
+		std.timestamp("vcl_recv");
+	}
+
+	sub vcl_backend_fetch {
+		std.timestamp("vcl_backend_fetch");
+	}
+} -start
+
+logexpect l1 -v v1 -g request {
+	expect 0 1001	Begin		req
+	expect * =	Timestamp	Start:
+	expect * =	Timestamp	{vcl_recv: \S+ 0\.\d+ 0\.\d+}
+	expect * =	Timestamp	Resp:
+	expect * =	End
+	expect 0 1002	Begin		bereq
+	expect * =	Timestamp	Start:
+	expect * =	Timestamp	{vcl_backend_fetch: \S+ 0\.\d+ 0\.\d+}
+	expect * =	Timestamp	BerespBody:
+	expect * =	End
+} -start
+
+client c1 {
+	txreq
+	rxresp
+	expect resp.status == 200
+} -run
+
+logexpect l1 -wait
diff --git a/lib/libvmod_std/vmod.vcc b/lib/libvmod_std/vmod.vcc
index dcb12ec..b1b7010 100644
--- a/lib/libvmod_std/vmod.vcc
+++ b/lib/libvmod_std/vmod.vcc
@@ -145,6 +145,15 @@ $Function INT port(IP)
 Description
 	Returns the port number of an IP address.
 
+$Function VOID timestamp(STRING)
+
+Description
+	Introduces a timestamp in the log with the current time. Uses
+	the argument as the timespamp label. This is useful to time
+	the execution of lengthy VCL procedures, and makes the
+	timestamps inserted automatically by Varnish more accurate.
+Example
+	std.timestamp("curl-request");
 
 SEE ALSO
 ========
diff --git a/lib/libvmod_std/vmod_std.c b/lib/libvmod_std/vmod_std.c
index 7301559..233c2da 100644
--- a/lib/libvmod_std/vmod_std.c
+++ b/lib/libvmod_std/vmod_std.c
@@ -41,6 +41,7 @@
 #include "vrt.h"
 #include "vtcp.h"
 #include "vsa.h"
+#include "vtim.h"
 
 #include "cache/cache.h"
 #include "cache/cache_backend.h"
@@ -204,3 +205,23 @@ vmod_port(const struct vrt_ctx *ctx, VCL_IP ip)
 		return (0);
 	return (VSA_Port(ip));
 }
+
+VCL_VOID __match_proto__(td_std_timestamp)
+vmod_timestamp(const struct vrt_ctx *ctx, VCL_STRING label)
+{
+
+	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
+	if (label == NULL)
+		return;
+	if (*label == '\0')
+		return;
+	if (ctx->bo != NULL && ctx->req == NULL) {
+		/* Called from backend vcl methods */
+		CHECK_OBJ_NOTNULL(ctx->bo, BUSYOBJ_MAGIC);
+		VSLb_ts_busyobj(ctx->bo, label, VTIM_real());
+	} else if (ctx->req != NULL) {
+		/* Called from request vcl methdos */
+		CHECK_OBJ_NOTNULL(ctx->req, REQ_MAGIC);
+		VSLb_ts_req(ctx->req, label, VTIM_real());
+	}
+}



From daghf at varnish-software.com  Tue Apr  1 13:09:54 2014
From: daghf at varnish-software.com (Dag Haavi Finstad)
Date: Tue, 01 Apr 2014 15:09:54 +0200
Subject: [4.0] 59c5e3d Update varnishncsa to read timing data from
 SLT_Timestamp records.
Message-ID: <E1WUyRu-0003IE-Nq@project.varnish-software.com>


commit 59c5e3d38ebc117b8799cd1818f7cdc634df77ce
Author: Dag Haavi Finstad <daghf at varnish-software.com>
Date:   Wed Mar 26 14:26:43 2014 +0100

    Update varnishncsa to read timing data from SLT_Timestamp records.

diff --git a/bin/varnishncsa/varnishncsa.c b/bin/varnishncsa/varnishncsa.c
index 7434070..d03f91c 100644
--- a/bin/varnishncsa/varnishncsa.c
+++ b/bin/varnishncsa/varnishncsa.c
@@ -777,10 +777,16 @@ dispatch_f(struct VSL_data *vsl, struct VSL_transaction * const pt[],
 			case SLT_Length:
 				frag_line(b, e, &CTX.frag[F_b]);
 				break;
-			case SLT_ReqEnd:
-				frag_fields(b, e, 1, &CTX.frag[F_tstart],
-				    2, &CTX.frag[F_tend], 3, &CTX.frag[F_ttfb],
-				    0, NULL);
+			case SLT_Timestamp:
+				if (isprefix(b, "Start:", e, &p)) {
+					frag_fields(p, e, 1, &CTX.frag[F_tstart], 0, NULL);
+
+				} else if (isprefix(b, "Resp:", e, &p)) {
+					frag_fields(p, e, 1, &CTX.frag[F_tend], 0, NULL);
+
+				} else if (isprefix(b, "Process:", e, &p)) {
+					frag_fields(p, e, 2, &CTX.frag[F_ttfb], 0, NULL);
+				}
 				break;
 			case SLT_ReqHeader:
 				if (isprefix(b, "Host:", e, &p))



From daghf at varnish-software.com  Tue Apr  1 13:09:54 2014
From: daghf at varnish-software.com (Dag Haavi Finstad)
Date: Tue, 01 Apr 2014 15:09:54 +0200
Subject: [4.0] b40bd3d Update varnishhist to read ttfb from VSL_Timestamp
 record.
Message-ID: <E1WUyRu-0003IN-P1@project.varnish-software.com>


commit b40bd3dc159b3cf1efbb8eaa0521abd0645800a8
Author: Dag Haavi Finstad <daghf at varnish-software.com>
Date:   Wed Mar 26 15:49:45 2014 +0100

    Update varnishhist to read ttfb from VSL_Timestamp record.
    
    Fixes: #1460

diff --git a/bin/varnishhist/varnishhist.c b/bin/varnishhist/varnishhist.c
index c7406da..91078d8 100644
--- a/bin/varnishhist/varnishhist.c
+++ b/bin/varnishhist/varnishhist.c
@@ -105,6 +105,7 @@ static int scales[] = {
 struct profile {
 	const char *name;
 	enum VSL_tag_e tag;
+	const char *prefix;
 	int field;
 	int hist_low;
 	int hist_high;
@@ -112,13 +113,15 @@ struct profile {
 profiles[] = {
 	{
 		.name = "responsetime",
-		.tag = SLT_ReqEnd,
-		.field = 5,
+		.tag = SLT_Timestamp,
+		.prefix = "Process:",
+		.field = 3,
 		.hist_low = -6,
 		.hist_high = 3
 	}, {
 		.name = "size",
 		.tag = SLT_Length,
+		.prefix = NULL,
 		.field = 1,
 		.hist_low = 1,
 		.hist_high = 8
@@ -224,6 +227,13 @@ accumulate(struct VSL_data *vsl, struct VSL_transaction * const pt[],
 			default:
 				if (tag != match_tag)
 					break;
+
+				if (active_profile->prefix &&
+				    strncmp(VSL_CDATA(tr->c->rec.ptr),
+					active_profile->prefix,
+					strlen(active_profile->prefix)) != 0)
+					break;
+
 				i = sscanf(VSL_CDATA(tr->c->rec.ptr), format,
 				    &value);
 				if (i != 1)
@@ -362,7 +372,7 @@ main(int argc, char **argv)
 	const char *profile = "responsetime";
 	pthread_t thr;
 	int fnum = -1;
-	struct profile cli_p;
+	struct profile cli_p = {0};
 	cli_p.name = 0;
 
 	VUT_Init(progname);



From lkarsten at varnish-software.com  Tue Apr  1 13:09:54 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:54 +0200
Subject: [4.0] d6298a7 Restore default_ttl to 120 seconds.
Message-ID: <E1WUyRu-0003IW-QP@project.varnish-software.com>


commit d6298a76f621b1083bb645ec82f1401f2963aa52
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Wed Mar 26 17:01:19 2014 +0100

    Restore default_ttl to 120 seconds.
    
    This was reduced to 20 seconds in 6d0c94c34c0e471b12982b207857934ec4e6ea57.
    
    The git commit message does not mention default_ttl, and since this was a
    fairly big commit, I don't believe the change was intentional.

diff --git a/bin/varnishd/mgt/mgt_param_tbl.c b/bin/varnishd/mgt/mgt_param_tbl.c
index b6ae75c..be862e2 100644
--- a/bin/varnishd/mgt/mgt_param_tbl.c
+++ b/bin/varnishd/mgt/mgt_param_tbl.c
@@ -64,7 +64,7 @@ struct parspec mgt_parspec[] = {
 		"The TTL assigned to objects if neither the backend nor "
 		"the VCL code assigns one.",
 		OBJ_STICKY,
-		"20", "seconds" },
+		"120", "seconds" },
 	{ "default_grace", tweak_timeout, &mgt_param.default_grace,
 		"0", NULL,
 		"Default grace period.  We will deliver an object "



From daghf at varnish-software.com  Tue Apr  1 13:09:54 2014
From: daghf at varnish-software.com (Dag Haavi Finstad)
Date: Tue, 01 Apr 2014 15:09:54 +0200
Subject: [4.0] 1a6bb1d Reformat to <=80 cols.
Message-ID: <E1WUyRu-0003If-RQ@project.varnish-software.com>


commit 1a6bb1d6d39d9685c9e347d3f8dc15cc3e298d70
Author: Dag Haavi Finstad <daghf at varnish-software.com>
Date:   Wed Mar 26 17:26:02 2014 +0100

    Reformat to <=80 cols.

diff --git a/bin/varnishncsa/varnishncsa.c b/bin/varnishncsa/varnishncsa.c
index d03f91c..75a0452 100644
--- a/bin/varnishncsa/varnishncsa.c
+++ b/bin/varnishncsa/varnishncsa.c
@@ -779,13 +779,16 @@ dispatch_f(struct VSL_data *vsl, struct VSL_transaction * const pt[],
 				break;
 			case SLT_Timestamp:
 				if (isprefix(b, "Start:", e, &p)) {
-					frag_fields(p, e, 1, &CTX.frag[F_tstart], 0, NULL);
+					frag_fields(p, e, 1,
+					    &CTX.frag[F_tstart], 0, NULL);
 
 				} else if (isprefix(b, "Resp:", e, &p)) {
-					frag_fields(p, e, 1, &CTX.frag[F_tend], 0, NULL);
+					frag_fields(p, e, 1,
+					    &CTX.frag[F_tend], 0, NULL);
 
 				} else if (isprefix(b, "Process:", e, &p)) {
-					frag_fields(p, e, 2, &CTX.frag[F_ttfb], 0, NULL);
+					frag_fields(p, e, 2,
+					    &CTX.frag[F_ttfb], 0, NULL);
 				}
 				break;
 			case SLT_ReqHeader:



From daghf at varnish-software.com  Tue Apr  1 13:09:54 2014
From: daghf at varnish-software.com (Dag Haavi Finstad)
Date: Tue, 01 Apr 2014 15:09:54 +0200
Subject: [4.0] 75eaba8 Varnishncsa %D format option is now in microseconds.
Message-ID: <E1WUyRu-0003Io-Sf@project.varnish-software.com>


commit 75eaba88157ddbdb3d69d699f8e5feb3d7065ae0
Author: Dag Haavi Finstad <daghf at varnish-software.com>
Date:   Thu Mar 27 11:32:52 2014 +0100

    Varnishncsa %D format option is now in microseconds.

diff --git a/bin/varnishncsa/varnishncsa.c b/bin/varnishncsa/varnishncsa.c
index 75a0452..a38264b 100644
--- a/bin/varnishncsa/varnishncsa.c
+++ b/bin/varnishncsa/varnishncsa.c
@@ -276,7 +276,7 @@ format_time(const struct format *format)
 
 	switch (format->time_type) {
 	case 'D':
-		AZ(VSB_printf(CTX.vsb, "%f", t_end - t_start));
+		AZ(VSB_printf(CTX.vsb, "%f", (t_end - t_start) * 1e6));
 		break;
 	case 't':
 		AN(format->time_fmt);



From lkarsten at varnish-software.com  Tue Apr  1 13:09:54 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:54 +0200
Subject: [4.0] 6dc740f Add manual Provides for libvarnishapi.so.1.
Message-ID: <E1WUyRu-0003Ix-Tg@project.varnish-software.com>


commit 6dc740f00a7d079d4464b0716e4f899632e712b8
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu Mar 27 12:11:10 2014 +0100

    Add manual Provides for libvarnishapi.so.1.
    
    find-provides did not pick up on libvarnishapi.so.1 for varnish-libs
    automatically, and find-requires for the main package did. This meant
    that package pair was uninstallable.
    
    I don't quite know why this is happening, so I'm working around it
    with a hardcoded Provides for now. This is on the critical path for
    4.0 release.

diff --git a/redhat/varnish.spec b/redhat/varnish.spec
index 6f35ecc..0d0537c 100644
--- a/redhat/varnish.spec
+++ b/redhat/varnish.spec
@@ -49,6 +49,7 @@ available on the following web site: http://www.varnish-cache.org/
 Summary: Libraries for %{name}
 Group: System Environment/Libraries
 BuildRequires: ncurses-devel
+Provides: libvarnishapi.so.1
 #Obsoletes: libvarnish1
 
 %description libs



From lkarsten at varnish-software.com  Tue Apr  1 13:09:54 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:54 +0200
Subject: [4.0] 6f4a02d Amending 6dc740f with additional Provides.
Message-ID: <E1WUyRu-0003J6-Ue@project.varnish-software.com>


commit 6f4a02ddda2d5bd232316c3aff445b331c6b877c
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu Mar 27 16:34:47 2014 +0100

    Amending 6dc740f with additional Provides.
    
    As Ingar mentions in #1445, this is a workaround for the
    dependency issue.
    
    Originally attempt at a shorter fix in 6dc740f didn't work out.

diff --git a/redhat/varnish.spec b/redhat/varnish.spec
index 0d0537c..9b08aa2 100644
--- a/redhat/varnish.spec
+++ b/redhat/varnish.spec
@@ -50,6 +50,10 @@ Summary: Libraries for %{name}
 Group: System Environment/Libraries
 BuildRequires: ncurses-devel
 Provides: libvarnishapi.so.1
+Provides: libvarnishapi.so.1(LIBVARNISHAPI_1.0)(64bit)
+Provides: libvarnishapi.so.1(LIBVARNISHAPI_1.1)(64bit)
+Provides: libvarnishapi.so.1(LIBVARNISHAPI_1.2)(64bit)
+Provides: libvarnishapi.so.1(LIBVARNISHAPI_1.3)(64bit)
 #Obsoletes: libvarnish1
 
 %description libs



From fgsch at lodoss.net  Tue Apr  1 13:09:54 2014
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Tue, 01 Apr 2014 15:09:54 +0200
Subject: [4.0] f021385 Use -p to pass parameters, -w is long gone
Message-ID: <E1WUyRu-0003JF-Vq@project.varnish-software.com>


commit f021385325303bca545699df961571a26374de08
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Thu Mar 27 16:05:06 2014 +0000

    Use -p to pass parameters, -w is long gone

diff --git a/redhat/varnish.sysconfig b/redhat/varnish.sysconfig
index f5846c9..aa0f8ba 100644
--- a/redhat/varnish.sysconfig
+++ b/redhat/varnish.sysconfig
@@ -100,7 +100,9 @@ DAEMON_OPTS="-a ${VARNISH_LISTEN_ADDRESS}:${VARNISH_LISTEN_PORT} \
              -f ${VARNISH_VCL_CONF} \
              -T ${VARNISH_ADMIN_LISTEN_ADDRESS}:${VARNISH_ADMIN_LISTEN_PORT} \
              -t ${VARNISH_TTL} \
-             -w ${VARNISH_MIN_THREADS},${VARNISH_MAX_THREADS},${VARNISH_THREAD_TIMEOUT} \
+             -p thread_pool_min=${VARNISH_MIN_THREADS} \
+             -p thread_pool_max=${VARNISH_MAX_THREADS} \
+             -p thread_pool_timeout=${VARNISH_THREAD_TIMEOUT} \
              -u varnish -g varnish \
              -S ${VARNISH_SECRET_FILE} \
              -s ${VARNISH_STORAGE}"



From lkarsten at varnish-software.com  Tue Apr  1 13:09:55 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:55 +0200
Subject: [4.0] 648951e Use jemalloc again.
Message-ID: <E1WUyRv-0003JO-0s@project.varnish-software.com>


commit 648951e42ec5154c975b230d1c5e6b0977938575
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu Mar 27 17:08:18 2014 +0100

    Use jemalloc again.
    
    Since we removed the bundled jemalloc, our packages have been
    using system malloc on RHEL Linux.

diff --git a/redhat/varnish.spec b/redhat/varnish.spec
index 9b08aa2..9adcaa2 100644
--- a/redhat/varnish.spec
+++ b/redhat/varnish.spec
@@ -16,11 +16,12 @@ BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n)
 # To build from git, start with a make dist, see redhat/README.redhat 
 # You will need at least automake autoconf libtool
 #BuildRequires: automake autoconf libtool
-BuildRequires: ncurses-devel groff pcre-devel pkgconfig python-docutils libedit-devel
+BuildRequires: ncurses-devel groff pcre-devel pkgconfig python-docutils libedit-devel jemalloc-devel
 Requires: varnish-libs = %{version}-%{release}
 Requires: logrotate
 Requires: ncurses
 Requires: pcre
+Requires: jemalloc
 Requires(pre): shadow-utils
 Requires(post): /sbin/chkconfig, /usr/bin/uuidgen
 Requires(preun): /sbin/chkconfig



From lkarsten at varnish-software.com  Tue Apr  1 13:09:55 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:55 +0200
Subject: [4.0] aa526ab Harmonise default storage across distributions.
Message-ID: <E1WUyRv-0003JX-2C@project.varnish-software.com>


commit aa526abd8f81b7ec9a8bb8803fdbb6a36fbff1ce
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu Mar 27 17:13:40 2014 +0100

    Harmonise default storage across distributions.
    
    The standard storage in our other Linux packages is 256 MB malloc.
    
    Change the default on Redhat systems to match this.

diff --git a/redhat/varnish.sysconfig b/redhat/varnish.sysconfig
index aa0f8ba..6aa2354 100644
--- a/redhat/varnish.sysconfig
+++ b/redhat/varnish.sysconfig
@@ -81,15 +81,12 @@ VARNISH_MAX_THREADS=1000
 # # Idle timeout for worker threads
 VARNISH_THREAD_TIMEOUT=120
 #
-# # Cache file location
-VARNISH_STORAGE_FILE=/var/lib/varnish/varnish_storage.bin
-#
 # # Cache file size: in bytes, optionally using k / M / G / T suffix,
 # # or in percentage of available disk space using the % suffix.
-VARNISH_STORAGE_SIZE=1G
+VARNISH_STORAGE_SIZE=256M
 #
 # # Backend storage specification
-VARNISH_STORAGE="file,${VARNISH_STORAGE_FILE},${VARNISH_STORAGE_SIZE}"
+VARNISH_STORAGE="malloc,${VARNISH_STORAGE_SIZE}"
 #
 # # Default TTL used when the backend does not specify one
 VARNISH_TTL=120



From daghf at varnish-software.com  Tue Apr  1 13:09:55 2014
From: daghf at varnish-software.com (Dag Haavi Finstad)
Date: Tue, 01 Apr 2014 15:09:55 +0200
Subject: [4.0] 723da93 Fix duplicate wording in example.vcl
Message-ID: <E1WUyRv-0003Jg-3K@project.varnish-software.com>


commit 723da932036b8e0046b408ffc6c8c0c0fd531e9c
Author: Dag Haavi Finstad <daghf at varnish-software.com>
Date:   Thu Mar 27 17:42:13 2014 +0100

    Fix duplicate wording in example.vcl

diff --git a/etc/example.vcl b/etc/example.vcl
index b99ec90..a2987bb 100644
--- a/etc/example.vcl
+++ b/etc/example.vcl
@@ -1,9 +1,9 @@
 #
 # This is an example VCL file for Varnish.
 #
-# It does not do anything by default, delegating control to the builtin vcl.
-# The builtin VCL is called when there is no explicit explicit return
-# statement.
+# It does not do anything by default, delegating control to the
+# builtin VCL. The builtin VCL is called when there is no explicit
+# return statement.
 #
 # See the VCL chapters in the Users Guide at https://www.varnish-cache.org/docs/
 # and http://varnish-cache.org/trac/wiki/VCLExamples for more examples.



From fgsch at lodoss.net  Tue Apr  1 13:09:55 2014
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Tue, 01 Apr 2014 15:09:55 +0200
Subject: [4.0] 1a1f1d3 Update dependencies and sort entries
Message-ID: <E1WUyRv-0003Jp-4L@project.varnish-software.com>


commit 1a1f1d30d917b50073ba1458960da833b2ddb829
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Thu Mar 27 16:47:29 2014 +0000

    Update dependencies and sort entries

diff --git a/doc/sphinx/installation/install.rst b/doc/sphinx/installation/install.rst
index 9b61d35..883cbf9 100644
--- a/doc/sphinx/installation/install.rst
+++ b/doc/sphinx/installation/install.rst
@@ -76,20 +76,20 @@ Build dependencies on Debian / Ubuntu
 In order to build Varnish from source you need a number of packages
 installed. On a Debian or Ubuntu system these are:
 
-* `autotools-dev`
-* `automake1.1`
-* `libtool`
 * `autoconf`
-* `libncurses-dev`
+* `automake1.1`
+* `autotools-dev`
 * `groff-base`
-* `libpcre3-dev`
-* `pkg-config`
 * `make`
 * `libedit-dev`
+* `libncurses-dev`
+* `libpcre3-dev`
+* `libtool`
+* `pkg-config`
+* `python-docutils`
 
 If you're building from git, you also need the following:
 
-* `python-docutils`
 * `python-sphinx` (optional, if you want to build the documentation)
 
 Build dependencies on Red Hat / CentOS
@@ -100,14 +100,17 @@ packages installed:
 
 * `automake`
 * `autoconf`
+* `groff`
+* `libedit-devel`
 * `libtool`
 * `ncurses-devel`
-* `groff`
 * `pcre-devel`
 * `pkgconfig`
-* `libedit-devel`
-* `docutils`
-* `python-sphinx`
+* `python-docutils`
+
+If you're building from git, you also need the following:
+
+* `python-sphinx` (optional, if you want to build the documentation)
 
 Compiling Varnish
 -----------------



From fgsch at lodoss.net  Tue Apr  1 13:09:55 2014
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Tue, 01 Apr 2014 15:09:55 +0200
Subject: [4.0] fa54ec5 Check if the compiler supports -Wno-address
Message-ID: <E1WUyRv-0003Jy-5e@project.varnish-software.com>


commit fa54ec5f5ce2aed29dbbce749ac637ae6180c454
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Thu Mar 27 17:54:22 2014 +0000

    Check if the compiler supports -Wno-address
    
    gcc 4.1.2 (and previous version I assume) doesn't support it.

diff --git a/configure.ac b/configure.ac
index e2a0b98..d1e203e 100644
--- a/configure.ac
+++ b/configure.ac
@@ -483,8 +483,7 @@ DEVELOPER_CFLAGS=" \
 	-Wchar-subscripts \
 	-Wnested-externs \
 	-Wextra \
-	-Wno-sign-compare \
-	-Wno-address \
+	-Wno-sign-compare
 "
 
 # These are not compliable yet
@@ -519,6 +518,10 @@ if test "x$enable_developer_warnings" != "xno"; then
 	# compiler flags not available on gcc3
 	AX_CHECK_COMPILE_FLAG([-Wno-pointer-sign],
 		[DEVELOPER_CFLAGS="${DEVELOPER_CFLAGS} -Wno-pointer-sign"], [], [])
+	# Not available in gcc 4.1.2
+	AX_CHECK_COMPILE_FLAG([-Wno-address],
+		[DEVELOPER_CFLAGS="${DEVELOPER_CFLAGS} -Wno-address"])
+
 	# no known way to specifically disabling missing-field-initializers warnings
 	# keeping the rest of Wextra
 	AX_CHECK_COMPILE_FLAG([-Wno-missing-field-initializers],



From fgsch at lodoss.net  Tue Apr  1 13:09:55 2014
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Tue, 01 Apr 2014 15:09:55 +0200
Subject: [4.0] 7bee7ba Fix previous commit
Message-ID: <E1WUyRv-0003K7-6f@project.varnish-software.com>


commit 7bee7ba694f0ed9cbd38c479be721fd2e672ff25
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Thu Mar 27 18:06:40 2014 +0000

    Fix previous commit

diff --git a/configure.ac b/configure.ac
index d1e203e..6612215 100644
--- a/configure.ac
+++ b/configure.ac
@@ -483,7 +483,7 @@ DEVELOPER_CFLAGS=" \
 	-Wchar-subscripts \
 	-Wnested-externs \
 	-Wextra \
-	-Wno-sign-compare
+	-Wno-sign-compare \
 "
 
 # These are not compliable yet



From phk at FreeBSD.org  Tue Apr  1 13:09:55 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 01 Apr 2014 15:09:55 +0200
Subject: [4.0] dc8b9fc Restrict client.ip and client.identity to req
 processing to avoid the predictable panic if they are accessed from the
 backend side.
Message-ID: <E1WUyRv-0003KG-7d@project.varnish-software.com>


commit dc8b9fc9b309106321f2cdd1793a68f36f5f0287
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Mar 28 07:30:17 2014 +0000

    Restrict client.ip and client.identity to req processing to avoid
    the predictable panic if they are accessed from the backend side.

diff --git a/lib/libvcc/generate.py b/lib/libvcc/generate.py
index a118513..06ed058 100755
--- a/lib/libvcc/generate.py
+++ b/lib/libvcc/generate.py
@@ -159,15 +159,15 @@ returns =(
 sp_variables = [
 	('client.ip',
 		'IP',
-		( 'both',),
+		( 'client',),
 		( ), """
 		The client's IP address.
 		"""
 	),
 	('client.identity',
 		'STRING',
-		( 'both',),
-		( 'both',), """
+		( 'client',),
+		( 'client',), """
 		Identification of the client, used to load balance
 		in the client director.
 		"""



From fgsch at lodoss.net  Tue Apr  1 13:09:55 2014
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Tue, 01 Apr 2014 15:09:55 +0200
Subject: [4.0] c17a6d2 Spelling, formatting and various other fixes
Message-ID: <E1WUyRv-0003KP-8n@project.varnish-software.com>


commit c17a6d23c0255d0ae11c5612be5e313078273db8
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Fri Mar 28 08:44:29 2014 +0000

    Spelling, formatting and various other fixes

diff --git a/doc/sphinx/reference/vmod.rst b/doc/sphinx/reference/vmod.rst
index 580caff..033e04b 100644
--- a/doc/sphinx/reference/vmod.rst
+++ b/doc/sphinx/reference/vmod.rst
@@ -158,7 +158,7 @@ STRING_LIST
 
 	A multi-component text-string.  We try very hard to avoid
 	doing text-processing in Varnish, and this is one way we
-	to avoid that, by not editing separate pieces of a sting
+	to avoid that, by not editing separate pieces of a string
 	together to one string, unless we have to.
 
 	Consider this contrived example::
diff --git a/doc/sphinx/users-guide/increasing-your-hitrate.rst b/doc/sphinx/users-guide/increasing-your-hitrate.rst
index 9d8e0af..a413b71 100644
--- a/doc/sphinx/users-guide/increasing-your-hitrate.rst
+++ b/doc/sphinx/users-guide/increasing-your-hitrate.rst
@@ -3,29 +3,30 @@
 Achieving a high hitrate
 ------------------------
 
-Now that Varnish is up and running, and you can access your web
-application through Varnish. Unless your application is specifically
-written to work behind a web accelerator you'll probably need to do
-some changes to either the configuration or the application in order
-to get a high hitrate in Varnish.
+Now that Varnish is up and running you can access your web application
+through Varnish. Unless your application is specifically written to
+work behind a web accelerator you'll probably need to do some
+changes to either the configuration or the application in order to
+get a high hitrate in Varnish.
 
 Varnish will not cache your data unless it's absolutely sure it is
 safe to do so. So, for you to understand how Varnish decides if and
-how to cache a page, We'll guide you through a couple of tools that you
-should find useful to understand what is happening in your Varnish setup.
+how to cache a page, we'll guide you through a couple of tools that
+you should find useful to understand what is happening in your
+Varnish setup.
 
-Note that you need a tool to see the HTTP headers that fly between Varnish and
-the backend. On the Varnish server, the easiest way to do this is to use
-`varnishlog` and `varnishtop` but sometimes a client-side tool makes
-sense. Here are the ones we commonly use.
+Note that you need a tool to see the HTTP headers that fly between
+Varnish and the backend. On the Varnish server, the easiest way to
+do this is to use `varnishlog` and `varnishtop` but sometimes a
+client-side tool makes sense. Here are the ones we commonly use.
 
 Tool: varnishtop
 ~~~~~~~~~~~~~~~~
 
 You can use varnishtop to identify what URLs are hitting the backend
-the most. ``varnishtop -i txurl`` is an essential command, showing you
-the top `txurl` requests Varnish is sending to the backend. You can
-see some other examples of `varnishtop` usage in
+the most. ``varnishtop -i txurl`` is an essential command, showing
+you the top `txurl` requests Varnish is sending to the backend. You
+can see some other examples of `varnishtop` usage in
 :ref:`users-guide-statistics`.
 
 
@@ -47,9 +48,10 @@ http://www.varnish-cache.org/trac/wiki/VCLExampleHitMissHeader
 Tool: lwp-request
 ~~~~~~~~~~~~~~~~~
 
-`lwp-request` is tool that is a part of The World-Wide Web library for Perl. It's a
-couple of really basic programs that can execute an HTTP request and
-show you the result. We mostly use the two programs, ``GET`` and ``HEAD``.
+`lwp-request` is tool that is a part of The World-Wide Web library
+for Perl. It's a couple of really basic programs that can execute
+an HTTP request and show you the result. We mostly use the two
+programs, ``GET`` and ``HEAD``.
 
 vg.no was the first site to use Varnish and the people running Varnish
 there are quite clueful. So it's interesting to look at their HTTP
@@ -59,7 +61,7 @@ Headers. Let's send a GET request for their home page::
   GET http://vg.no/
   Host: www.vg.no
   User-Agent: lwp-request/5.834 libwww-perl/5.834
-  
+
   200 OK
   Cache-Control: must-revalidate
   Refresh: 600
@@ -81,7 +83,7 @@ really care about the content, only the headers.
 As you can see, VG adds quite a bit of information in their
 headers. Some of the headers, like the 'X-Rick-Would-Never' are specific
 to vg.no and their somewhat odd sense of humour. Others, like the
-'X-VG-Webcache' are for debugging purposes. 
+'X-VG-Webcache' are for debugging purposes.
 
 So, to check whether a site sets cookies for a specific URL, just do::
 
@@ -92,10 +94,11 @@ So, to check whether a site sets cookies for a specific URL, just do::
 Tool: Live HTTP Headers
 ~~~~~~~~~~~~~~~~~~~~~~~
 
-There is also a plugin for Firefox called `Live HTTP Headers`. This plugin can show you
-what headers are being sent and recieved. `Live HTTP Headers` can be
-found at https://addons.mozilla.org/en-US/firefox/addon/3829/ or by
-googling "Live HTTP Headers".
+There is also a plugin for Firefox called `Live HTTP Headers`. This
+plugin can show you what headers are being sent and received.
+`Live HTTP Headers` can be found at
+https://addons.mozilla.org/en-US/firefox/addon/3829/ or by googling
+"Live HTTP Headers".
 
 
 The role of HTTP Headers
@@ -108,7 +111,7 @@ the content cached.
 
 Please note that when Varnish considers these headers Varnish actually
 considers itself *part of* the actual webserver. The rationale being
-that both are under your control. 
+that both are under your control.
 
 The term *surrogate origin cache* is not really well defined by the
 IETF or RFC 2616 so the various ways Varnish works might differ from
@@ -129,7 +132,7 @@ the backend.
 This can be overly conservative. A lot of sites use Google Analytics
 (GA) to analyze their traffic. GA sets a cookie to track you. This
 cookie is used by the client side javascript and is therefore of no
-interest to the server. 
+interest to the server.
 
 Cookies from the client
 ~~~~~~~~~~~~~~~~~~~~~~~
@@ -139,9 +142,9 @@ cookies unless you are accessing a special part of the web site. This
 VCL snippet in `vcl_recv` will disregard cookies unless you are
 accessing `/admin/`::
 
-  if ( !( req.url ~ ^/admin/) ) {
-    unset req.http.Cookie;
-  }
+    if (!(req.url ~ "^/admin/")) {
+        unset req.http.Cookie;
+    }
 
 Quite simple. If, however, you need to do something more complicated,
 like removing one out of several cookies, things get
@@ -152,36 +155,39 @@ whats going on. If you aren't we recommend that you either pick up a book on
 the subject, read through the *pcrepattern* man page, or read through
 one of many online guides.
 
-Lets use the Varnish Software (VS) web as an example here. Very simplified the setup VS uses can be described as a Drupal-based backend with a Varnish cache infront. VS uses some cookies for
-Google Analytics tracking and similar tools. The cookies are all set
-and used by Javascript. Varnish and Drupal doesn't need to see those
-cookies and since Varnish will cease caching of pages when the client
-sends cookies Varnish will discard these unnecessary cookies in VCL. 
+Lets use the Varnish Software (VS) web as an example here. Very
+simplified the setup VS uses can be described as a Drupal-based
+backend with a Varnish cache in front. VS uses some cookies for
+Google Analytics tracking and similar tools. The cookies are all
+set and used by Javascript. Varnish and Drupal doesn't need to see
+those cookies and since Varnish will cease caching of pages when
+the client sends cookies Varnish will discard these unnecessary
+cookies in VCL.
 
-In the following VCL we discard all cookies that start with a
+In the following VCL we discard all cookies that start with an
 underscore::
 
-  // Remove has_js and Google Analytics __* cookies.
-  set req.http.Cookie = regsuball(req.http.Cookie, "(^|;\s*)(_[_a-z]+|has_js)=[^;]*", "");
-  // Remove a ";" prefix, if present.
-  set req.http.Cookie = regsub(req.http.Cookie, "^;\s*", "");
+    # Remove has_js and Google Analytics __* cookies.
+    set req.http.Cookie = regsuball(req.http.Cookie, "(^|;\s*)(_[_a-z]+|has_js)=[^;]*", "");
+    # Remove a ";" prefix, if present.
+    set req.http.Cookie = regsub(req.http.Cookie, "^;\s*", "");
 
 Lets look at an example where we remove everything except the
 cookies named "COOKIE1" and "COOKIE2" and you can marvel at the "beauty" of it::
 
-  sub vcl_recv {
-    if (req.http.Cookie) {
-      set req.http.Cookie = ";" + req.http.Cookie;
-      set req.http.Cookie = regsuball(req.http.Cookie, "; +", ";");
-      set req.http.Cookie = regsuball(req.http.Cookie, ";(COOKIE1|COOKIE2)=", "; \1=");
-      set req.http.Cookie = regsuball(req.http.Cookie, ";[^ ][^;]*", "");
-      set req.http.Cookie = regsuball(req.http.Cookie, "^[; ]+|[; ]+$", "");
-
-      if (req.http.Cookie == "") {
-          remove req.http.Cookie;
-      }
+    sub vcl_recv {
+        if (req.http.Cookie) {
+            set req.http.Cookie = ";" + req.http.Cookie;
+            set req.http.Cookie = regsuball(req.http.Cookie, "; +", ";");
+            set req.http.Cookie = regsuball(req.http.Cookie, ";(COOKIE1|COOKIE2)=", "; \1=");
+            set req.http.Cookie = regsuball(req.http.Cookie, ";[^ ][^;]*", "");
+            set req.http.Cookie = regsuball(req.http.Cookie, "^[; ]+|[; ]+$", "");
+
+            if (req.http.Cookie == "") {
+                unset req.http.Cookie;
+            }
+        }
     }
-  }
 
 A somewhat simpler example that can accomplish almost the same functionality can be
 found below. Instead of filtering out "other" cookies it instead picks out
@@ -190,16 +196,16 @@ copies it back to the request, deleting the original cookie header.
 .. XXX:Verify correctness of request above! benc
 ::
 
-  sub vcl_recv {
-         # save the original cookie header so we can mangle it
+    sub vcl_recv {
+        # save the original cookie header so we can mangle it
         set req.http.X-Varnish-PHP_SID = req.http.Cookie;
-        # using a capturing sub pattern, extract the continuous string of 
+        # using a capturing sub pattern, extract the continuous string of
         # alphanumerics that immediately follows "PHPSESSID="
-        set req.http.X-Varnish-PHP_SID = 
+        set req.http.X-Varnish-PHP_SID =
            regsuball(req.http.X-Varnish-PHP_SID, ";? ?PHPSESSID=([a-zA-Z0-9]+)( |;| ;).*","\1");
         set req.http.Cookie = req.X-Varnish-PHP_SID;
-        remove req.X-Varnish-PHP_SID;
-   }   
+        unset req.X-Varnish-PHP_SID;
+    }
 
 There are other scary examples of what can be done in VCL in the
 Varnish Cache Wiki.
@@ -211,10 +217,10 @@ Cookies coming from the backend
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 If your backend server sets a cookie using the 'Set-Cookie' header
-Varnish will not cache the page when using the default configuration. A
-`hit-for-pass` object (see :ref:`user-guide-vcl_actions`) is created.
-So, if the backend server acts silly and sets unwanted cookies just unset
-the 'Set-Cookie' header and all should be fine. 
+Varnish will not cache the page when using the default configuration.
+A `hit-for-pass` object (see :ref:`user-guide-vcl_actions`) is
+created.  So, if the backend server acts silly and sets unwanted
+cookies just unset the 'Set-Cookie' header and all should be fine.
 
 
 Cache-Control
@@ -247,7 +253,7 @@ In `vcl_backend_response`::
 
     if (beresp.http.Pragma ~ "nocache") {
         set beresp.uncacheable = true;
-        set beresp.ttl = 120s; // how long not to cache this url.
+        set beresp.ttl = 120s; # how long not to cache this url.
     }
 
 Authorization
@@ -266,11 +272,11 @@ somewhat cumbersome backend.
 You need VCL to identify the objects you want and then you set the
 'beresp.ttl' to whatever you want::
 
-  sub vcl_backend_response {
-      if (req.url ~ "^/legacy_broken_cms/") {
-          set beresp.ttl = 5d;
-      }
-  }
+    sub vcl_backend_response {
+        if (req.url ~ "^/legacy_broken_cms/") {
+            set beresp.ttl = 5d;
+        }
+    }
 
 This example will set the TTL to 5 days for the old legacy stuff on
 your site.
@@ -288,18 +294,18 @@ the cache but it isn't really recommended.
 Normalizing your namespace
 ~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-Some sites are accessed via lots of
-hostnames. http://www.varnish-software.com/,
-http://varnish-software.com/ and http://varnishsoftware.com/ all point
-at the same site. Since Varnish doesn't know they are the same,
+Some sites are accessed via lots of hostnames.
+http://www.varnish-software.com/, http://varnish-software.com/ and
+http://varnishsoftware.com/ all point at the same site. Since Varnish
+doesn't know they are the same,
 .. XXX: heavy meaning change above. benc
 Varnish will cache different versions of every page for every
-hostname. You can mitigate this in your web server configuration by
-setting up redirects or by using the following VCL::
+hostname. You can mitigate this in your web server configuration
+by setting up redirects or by using the following VCL::
 
-  if (req.http.host ~ "(?i)^(www.)?varnish-?software.com") {
-    set req.http.host = "varnish-software.com";
-  }
+    if (req.http.host ~ "(?i)^(www.)?varnish-?software.com") {
+        set req.http.host = "varnish-software.com";
+    }
 
 
 .. _users-guide-vary:
@@ -307,38 +313,38 @@ setting up redirects or by using the following VCL::
 HTTP Vary
 ---------
 
-*HTTP Vary is not a trivial concept. It is by far the most
-misunderstood HTTP header.*
+*HTTP Vary is not a trivial concept. It is by far the most misunderstood
+HTTP header.*
 
-A lot of the response headers tell the client something about the HTTP
-object being delivered. Clients can request different variants of a
-HTTP object, based on their preference. Their preferences might cover
-stuff like encoding or language. When a client prefers UK English this
-is indicated through ``Accept-Language: en-uk``. Caches need to keep
-these different variants apart and this is done through the HTTP
-response header 'Vary'.
+A lot of the response headers tell the client something about the
+HTTP object being delivered. Clients can request different variants
+of a HTTP object, based on their preference. Their preferences might
+cover stuff like encoding or language. When a client prefers UK
+English this is indicated through ``Accept-Language: en-uk``. Caches
+need to keep these different variants apart and this is done through
+the HTTP response header 'Vary'.
 
 When a backend server issues a ``Vary: Accept-Language`` it tells
 Varnish that its needs to cache a separate version for every different
 Accept-Language that is coming from the clients.
 
-If two clients say they accept the languages "en-us, en-uk" and "da,
-de" respectively, Varnish will cache and serve two different versions
-of the page if the backend indicated that Varnish needs to vary on the
-'Accept-Language' header.
+If two clients say they accept the languages "en-us, en-uk" and
+"da, de" respectively, Varnish will cache and serve two different
+versions of the page if the backend indicated that Varnish needs
+to vary on the 'Accept-Language' header.
 
 Please note that the headers that 'Vary' refer to need to match
-*exactly* for there to be a match. So Varnish will keep two copies of
-a page if one of them was created for "en-us, en-uk" and the other for
-"en-us,en-uk". Just the lack of a whitespace will force Varnish to cache
-another version.
+*exactly* for there to be a match. So Varnish will keep two copies
+of a page if one of them was created for "en-us, en-uk" and the
+other for "en-us,en-uk". Just the lack of a whitespace will force
+Varnish to cache another version.
 
-To achieve a high hitrate whilst using Vary is there therefore crucial
-to normalize the headers the backends varies on. Remember, just a
-difference in casing can force different cache entries.
+To achieve a high hitrate whilst using Vary is there therefore
+crucial to normalize the headers the backends varies on. Remember,
+just a difference in casing can force different cache entries.
 
-The following VCL code will normalize the 'Accept-Language' headers, to
-one of either "en","de" or "fr"::
+The following VCL code will normalize the 'Accept-Language' headers,
+to one of either "en", "de" or "fr"::
 
     if (req.http.Accept-Language) {
         if (req.http.Accept-Language ~ "en") {
@@ -348,9 +354,9 @@ one of either "en","de" or "fr"::
         } elsif (req.http.Accept-Language ~ "fr") {
             set req.http.Accept-Language = "fr";
         } else {
-            # unknown language. Remove the accept-language header and 
-	    # use the backend default.
-            remove req.http.Accept-Language
+            # unknown language. Remove the accept-language header and
+            # use the backend default.
+            unset req.http.Accept-Language
         }
     }
 
@@ -360,22 +366,22 @@ gzip, deflate with a preference for gzip.
 Vary parse errors
 ~~~~~~~~~~~~~~~~~
 
-Varnish will return a "503 internal server error" page when it fails to
-parse the 'Vary' header, or if any of the client headers listed
+Varnish will return a "503 internal server error" page when it fails
+to parse the 'Vary' header, or if any of the client headers listed
 in the Vary header exceeds the limit of 65k characters. An 'SLT_Error'
 log entry is added in these cases.
 
 Pitfall - Vary: User-Agent
 ~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-Some applications or application servers send ``Vary: User-Agent`` along
-with their content. This instructs Varnish to cache a separate copy
-for every variation of 'User-Agent' there is and there are plenty. Even a
-single patchlevel of the same browser will generate at least 10
-different 'User-Agent' headers based just on what operating system they
-are running. 
+Some applications or application servers send ``Vary: User-Agent``
+along with their content. This instructs Varnish to cache a separate
+copy for every variation of 'User-Agent' there is and there are
+plenty. Even a single patchlevel of the same browser will generate
+at least 10 different 'User-Agent' headers based just on what
+operating system they are running.
 
 So if you *really* need to vary based on 'User-Agent' be sure to
-normalize the header or your hit rate will suffer badly. Use the above
-code as a template.
+normalize the header or your hit rate will suffer badly. Use the
+above code as a template.
 
diff --git a/doc/sphinx/whats-new/upgrading.rst b/doc/sphinx/whats-new/upgrading.rst
index 3634a33..d64772d 100644
--- a/doc/sphinx/whats-new/upgrading.rst
+++ b/doc/sphinx/whats-new/upgrading.rst
@@ -7,13 +7,17 @@ Upgrading to Varnish 4
 Changes to VCL
 ==============
 
-Much of the VCL syntax has changed in Varnish 4. We've tried to compile a list of changes needed to upgrade here.
+Much of the VCL syntax has changed in Varnish 4. We've tried to
+compile a list of changes needed to upgrade here.
 
 Version statement
 ~~~~~~~~~~~~~~~~~
-To make sure that people have upgraded their VCL to the current version, Varnish now requires the first line of VCL to indicate the VCL version number::
 
-	vcl 4.0;
+To make sure that people have upgraded their VCL to the current
+version, Varnish now requires the first line of VCL to indicate the
+VCL version number::
+
+    vcl 4.0;
 
 vcl_fetch is now vcl_backend_response
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -21,8 +25,8 @@ vcl_fetch is now vcl_backend_response
 Directors have been moved to the vmod_directors
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-To make directors (backend selection logic) easier to extend, the directors
-are now defined in loadable VMODs.
+To make directors (backend selection logic) easier to extend, the
+directors are now defined in loadable VMODs.
 
 Setting a backend for future fetches in `vcl_recv` is now done as follows::
 
@@ -31,6 +35,7 @@ Setting a backend for future fetches in `vcl_recv` is now done as follows::
         cluster1.add_backend(b1, 1.0);
         cluster1.add_backend(b2, 1.0);
     }
+
     sub vcl_recv {
         set req.backend_hint = cluster1.backend();
     }
@@ -47,21 +52,22 @@ has been removed, and you should use the hash director directly::
         h.add_backend(b1, 1);
         h.add_backend(b2, 1);
     }
+
     sub vcl_recv {
         set req.backend_hint = h.backend(client.ip);
     }
 
-error() is now a return value
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-You must explicitly return an error::
+error() is now synth()
+~~~~~~~~~~~~~~~~~~~~~~
 
-	return(error(999, "Response));
+And you must explicitly return it::
 
+    return (synth(999, "Response"));
 
-Synthetic responses in vcl_error
+Synthetic responses in vcl_synth
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-Setting headers on synthetic response bodies made in vcl_error are now done on
+Setting headers on synthetic response bodies made in vcl_synth are now done on
 resp.http instead of obj.http.
 
 The synthetic keyword is now a function::
@@ -73,31 +79,33 @@ The synthetic keyword is now a function::
         return (deliver);
     }
 
-
-
 hit_for_pass objects are created using beresp.uncacheable
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
 Example::
 
-	sub vcl_backend_response {
-		if(beresp.http.X-No-Cache) {
-			set beresp.uncacheable = true;
-			set beresp.ttl = 120s;
-			return(deliver);
-		}
-	}
+    sub vcl_backend_response {
+        if (beresp.http.X-No-Cache) {
+            set beresp.uncacheable = true;
+            set beresp.ttl = 120s;
+            return (deliver);
+        }
+    }
 
 vcl_recv should return(hash) instead of lookup now
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 req.* not available in vcl_backend_response
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-req.* used to be available in `vcl_fetch`, but after the split of functionality, you only have 'bereq.*' in `vcl_backend_response`.
+
+req.* used to be available in `vcl_fetch`, but after the split of
+functionality, you only have 'bereq.*' in `vcl_backend_response`.
 
 vcl_* reserved
 ~~~~~~~~~~~~~~
-Any custom-made subs cannot be named 'vcl_*' anymore. This namespace is
-reserved for builtin subs.
+
+Any custom-made subs cannot be named 'vcl_*' anymore. This namespace
+is reserved for builtin subs.
 
 req.backend.healthy replaced by std.healthy(req.backend)
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -112,15 +120,15 @@ objecthead, not per object. `obj.last_use` has been retired.
 default/builtin VCL changes
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-The VCL code that is appended to user-configured VCL automatically is now
-called the builtin VCL. (previously default.vcl)
+The VCL code that is appended to user-configured VCL automatically
+is now called the builtin VCL. (previously default.vcl)
 
-The builtin VCL now honors Cache-Control: no-cache (and friends) to indicate
-uncacheable content from the backend.
+The builtin VCL now honors Cache-Control: no-cache (and friends)
+to indicate uncacheable content from the backend.
 
 
 The `remove` keyword is gone
-~~~~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 Replaced by `unset`.
 



From phk at FreeBSD.org  Tue Apr  1 13:09:55 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 01 Apr 2014 15:09:55 +0200
Subject: [4.0] 78cbdea Fix test-case
Message-ID: <E1WUyRv-0003KZ-9w@project.varnish-software.com>


commit 78cbdeac8765f1f6e82345bf58d25fd93b1ce918
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Mar 28 08:57:42 2014 +0000

    Fix test-case

diff --git a/bin/varnishtest/tests/v00017.vtc b/bin/varnishtest/tests/v00017.vtc
index 49eec50..b0d6c20 100644
--- a/bin/varnishtest/tests/v00017.vtc
+++ b/bin/varnishtest/tests/v00017.vtc
@@ -64,7 +64,7 @@ varnish v1 -vcl {
 	sub vcl_recv { if (client.ip ~ a) { return(pass); } }
 }
 
-varnish v1 -vcl {
+varnish v1 -errvcl {'client.ip': Not available in method 'vcl_backend_response'} {
 	backend b { .host = "127.0.0.1"; }
 	sub vcl_recv { if (client.ip == "127.0.0.1") { return(pass); } }
 	sub vcl_backend_response { if (client.ip != "127.0.0.1") { return(retry); } }



From phk at FreeBSD.org  Tue Apr  1 13:09:55 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 01 Apr 2014 15:09:55 +0200
Subject: [4.0] db1d398 Fix an assert to look at the right objcore
Message-ID: <E1WUyRv-0003Ki-Ap@project.varnish-software.com>


commit db1d3987b8f5113495905a953f68b5ccf61ba28d
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Mar 28 09:10:21 2014 +0000

    Fix an assert to look at the right objcore

diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index 9826ae7..090e0b6 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -414,7 +414,7 @@ cnt_lookup(struct worker *wrk, struct req *req)
 		if (boc != NULL) {
 			AZ(oc->flags & (OC_F_FAILED|OC_F_PASS));
 			AZ(oc->exp_flags & OC_EF_DYING);
-			AZ(oc->busyobj);
+			AZ(boc->busyobj);
 			VBF_Fetch(wrk, req, boc, o, VBF_BACKGROUND);
 		} else {
 			(void)HTTP1_DiscardReqBody(req);// XXX: handle err



From daghf at varnish-software.com  Tue Apr  1 13:09:55 2014
From: daghf at varnish-software.com (Dag Haavi Finstad)
Date: Tue, 01 Apr 2014 15:09:55 +0200
Subject: [4.0] 8d3e492 VCC default vcl -> builtin vcl renames.
Message-ID: <E1WUyRv-0003Kr-Bt@project.varnish-software.com>


commit 8d3e4924058aa4f5ca90ea7c917fb183bd42a6a9
Author: Dag Haavi Finstad <daghf at varnish-software.com>
Date:   Fri Mar 28 13:01:33 2014 +0100

    VCC default vcl -> builtin vcl renames.

diff --git a/bin/varnishd/mgt/mgt_vcc.c b/bin/varnishd/mgt/mgt_vcc.c
index 09e4eb4..2c2ffd9 100644
--- a/bin/varnishd/mgt/mgt_vcc.c
+++ b/bin/varnishd/mgt/mgt_vcc.c
@@ -488,7 +488,7 @@ mgt_vcc_init(void)
 
 	vcc = VCC_New();
 	AN(vcc);
-	VCC_Default_VCL(vcc, builtin_vcl);
+	VCC_Builtin_VCL(vcc, builtin_vcl);
 	AZ(atexit(mgt_vcc_atexit));
 }
 
diff --git a/include/libvcc.h b/include/libvcc.h
index 5c50c34..6b16252 100644
--- a/include/libvcc.h
+++ b/include/libvcc.h
@@ -31,7 +31,7 @@
 struct vcc;
 
 struct vcc *VCC_New(void);
-void VCC_Default_VCL(struct vcc *, const char *str);
+void VCC_Builtin_VCL(struct vcc *, const char *str);
 void VCC_VCL_dir(struct vcc *, const char *str);
 void VCC_VMOD_dir(struct vcc *, const char *str);
 void VCC_Err_Unref(struct vcc *tl, unsigned u);
diff --git a/lib/libvcc/vcc_compile.c b/lib/libvcc/vcc_compile.c
index 91ea4c9..81a7620 100644
--- a/lib/libvcc/vcc_compile.c
+++ b/lib/libvcc/vcc_compile.c
@@ -489,7 +489,7 @@ vcc_NewVcc(const struct vcc *tl0)
 	ALLOC_OBJ(tl, VCC_MAGIC);
 	AN(tl);
 	if (tl0 != NULL) {
-		REPLACE(tl->default_vcl, tl0->default_vcl);
+		REPLACE(tl->builtin_vcl, tl0->builtin_vcl);
 		REPLACE(tl->vcl_dir, tl0->vcl_dir);
 		REPLACE(tl->vmod_dir, tl0->vmod_dir);
 		tl->vars = tl0->vars;
@@ -612,8 +612,8 @@ vcc_CompileSource(const struct vcc *tl0, struct vsb *sb, struct source *sp)
 	if (tl->err)
 		return (vcc_DestroyTokenList(tl, NULL));
 
-	/* Register and lex the default VCL */
-	sp = vcc_new_source(tl->default_vcl, NULL, "Default");
+	/* Register and lex the builtin VCL */
+	sp = vcc_new_source(tl->builtin_vcl, NULL, "Builtin");
 	assert(sp != NULL);
 	VTAILQ_INSERT_TAIL(&tl->sources, sp, list);
 	sp->idx = tl->nsources++;
@@ -736,15 +736,15 @@ VCC_New(void)
 }
 
 /*--------------------------------------------------------------------
- * Configure default VCL source code
+ * Configure builtin VCL source code
  */
 
 void
-VCC_Default_VCL(struct vcc *tl, const char *str)
+VCC_Builtin_VCL(struct vcc *tl, const char *str)
 {
 
 	CHECK_OBJ_NOTNULL(tl, VCC_MAGIC);
-	REPLACE(tl->default_vcl, str);
+	REPLACE(tl->builtin_vcl, str);
 }
 
 /*--------------------------------------------------------------------
diff --git a/lib/libvcc/vcc_compile.h b/lib/libvcc/vcc_compile.h
index bdfda05..128916b 100644
--- a/lib/libvcc/vcc_compile.h
+++ b/lib/libvcc/vcc_compile.h
@@ -160,7 +160,7 @@ struct vcc {
 #define VCC_MAGIC		0x24ad719d
 
 	/* Parameter/Template section */
-	char			*default_vcl;
+	char			*builtin_vcl;
 	char			*vcl_dir;
 	char			*vmod_dir;
 



From martin at varnish-software.com  Tue Apr  1 13:09:55 2014
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Tue, 01 Apr 2014 15:09:55 +0200
Subject: [4.0] 631f23e Fix the meaning of '=' in logexpect
Message-ID: <E1WUyRv-0003L0-Cr@project.varnish-software.com>


commit 631f23e6f043b5a6c67de2ce69dd231c2b77c4a7
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Sun Mar 30 03:36:25 2014 +0200

    Fix the meaning of '=' in logexpect
    
    Fix the meaning of '=' in logexpect to mean same value as that of the
    last successfully matched log record.

diff --git a/bin/varnishtest/vtc_logexp.c b/bin/varnishtest/vtc_logexp.c
index 5b18f0b..92fa5ae 100644
--- a/bin/varnishtest/vtc_logexp.c
+++ b/bin/varnishtest/vtc_logexp.c
@@ -42,7 +42,7 @@
  * tag:  [tagname|*|=]		Tag to match against
  * regex:			regular expression to match against (optional)
  * *:				Match anything
- * =:				Match value of last examined log record
+ * =:				Match value of last successfully matched record
  */
 
 #include "config.h"
@@ -245,6 +245,8 @@ logexp_dispatch(struct VSL_data *vsl, struct VSL_transaction * const pt[],
 			    legend, vxid, VSL_tags[tag], type, len, data);
 
 			if (ok) {
+				le->vxid_last = vxid;
+				le->tag_last = tag;
 				le->skip_cnt = 0;
 				logexp_next(le);
 				if (le->test == NULL)
@@ -253,9 +255,6 @@ logexp_dispatch(struct VSL_data *vsl, struct VSL_transaction * const pt[],
 			}
 			if (skip)
 				le->skip_cnt++;
-
-			le->vxid_last = vxid;
-			le->tag_last = tag;
 		}
 	}
 



From martin at varnish-software.com  Tue Apr  1 13:09:55 2014
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Tue, 01 Apr 2014 15:09:55 +0200
Subject: [4.0] 5b17fb2 Fix logexp record length to account for terminating
 zero.
Message-ID: <E1WUyRv-0003L9-Dw@project.varnish-software.com>


commit 5b17fb212b68dcbae35a6e693bb80d5949730313
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Sun Mar 30 12:31:30 2014 +0200

    Fix logexp record length to account for terminating zero.
    
    This makes it possible to anchor regex's at the end again.

diff --git a/bin/varnishtest/vtc_logexp.c b/bin/varnishtest/vtc_logexp.c
index 92fa5ae..e6adae7 100644
--- a/bin/varnishtest/vtc_logexp.c
+++ b/bin/varnishtest/vtc_logexp.c
@@ -196,7 +196,7 @@ logexp_dispatch(struct VSL_data *vsl, struct VSL_transaction * const pt[],
 			vxid = VSL_ID(t->c->rec.ptr);
 			tag = VSL_TAG(t->c->rec.ptr);
 			data = VSL_CDATA(t->c->rec.ptr);
-			len = VSL_LEN(t->c->rec.ptr);
+			len = VSL_LEN(t->c->rec.ptr) - 1;
 
 			if (tag == SLT__Batch)
 				continue;



From martin at varnish-software.com  Tue Apr  1 13:09:55 2014
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Tue, 01 Apr 2014 15:09:55 +0200
Subject: [4.0] 34f4b96 Remove struct sess acct_bit and bit fields from
 SessClose log record
Message-ID: <E1WUyRv-0003LI-Ev@project.varnish-software.com>


commit 34f4b96a53356f49e782d7a31957704f2129d7b2
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Fri Mar 28 11:04:12 2014 +0100

    Remove struct sess acct_bit and bit fields from SessClose log record
    
    We want to remove the req->acct_req dup counters that was feeding this
    data to reduce the size of struct req. The statistics value of the
    boolean flags was also small. So it's going away.

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index ea5d990..b793467 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -234,12 +234,6 @@ struct acct {
 #undef ACCT
 };
 
-struct acct_bit {
-#define ACCT(foo)	unsigned int	foo : 1;
-#include "tbl/acct_fields.h"
-#undef ACCT
-};
-
 /*--------------------------------------------------------------------*/
 
 #define L0(t, n)
@@ -748,8 +742,6 @@ struct sess {
 	enum sess_close		reason;
 	uint32_t		vxid;
 
-	struct acct_bit		acct_bit;
-
 	/* Cross references ------------------------------------------*/
 
 	struct sesspool		*sesspool;
diff --git a/bin/varnishd/cache/cache_session.c b/bin/varnishd/cache/cache_session.c
index 73ed5c7..8059c83 100644
--- a/bin/varnishd/cache/cache_session.c
+++ b/bin/varnishd/cache/cache_session.c
@@ -81,7 +81,6 @@ SES_Charge(struct worker *wrk, struct req *req)
 
 #define ACCT(foo)				\
 	wrk->stats.s_##foo += a->foo;		\
-	if (a->foo) sp->acct_bit.foo =1;	\
 	a->foo = 0;
 #include "tbl/acct_fields.h"
 #undef ACCT
@@ -316,7 +315,6 @@ SES_Close(struct sess *sp, enum sess_close reason)
 void
 SES_Delete(struct sess *sp, enum sess_close reason, double now)
 {
-	struct acct_bit *b;
 	struct sesspool *pp;
 
 	CHECK_OBJ_NOTNULL(sp, SESS_MAGIC);
@@ -332,10 +330,8 @@ SES_Delete(struct sess *sp, enum sess_close reason, double now)
 		now = VTIM_real();
 	assert(!isnan(sp->t_open));
 
-	b = &sp->acct_bit;
-	VSL(SLT_SessClose, sp->vxid, "%s %.3f %u %u %u %u %u %u",
-	    sess_close_2str(sp->reason, 0), now - sp->t_open, b->req,
-	    b->pipe, b->pass, b->fetch, b->hdrbytes, b->bodybytes);
+	VSL(SLT_SessClose, sp->vxid, "%s %.3f",
+	    sess_close_2str(sp->reason, 0), now - sp->t_open);
 	VSL(SLT_End, sp->vxid, "%s", "");
 
 	Lck_Delete(&sp->mtx);
diff --git a/include/tbl/vsl_tags.h b/include/tbl/vsl_tags.h
index 0f50a48..10a3a15 100644
--- a/include/tbl/vsl_tags.h
+++ b/include/tbl/vsl_tags.h
@@ -102,16 +102,10 @@ SLTM(SessOpen, 0, "Client connection opened",
 SLTM(SessClose, 0, "Client connection closed",
 	"SessionClose is the last record for any client connection.\n\n"
 	"The format is::\n\n"
-	"\t%s %f %u %u %u %u %u %u\n"
-	"\t|  |  |  |  |  |  |  |\n"
-	"\t|  |  |  |  |  |  |  +- Bool: Body bytes were sent\n"
-	"\t|  |  |  |  |  |  +---- Bool: Resp.Header bytes sent\n"
-	"\t|  |  |  |  |  +------- Bool: Backend fetches initiated\n"
-	"\t|  |  |  |  +---------- Bool: 'pass' used\n"
-	"\t|  |  |  +------------- Bool: 'pipe' used\n"
-	"\t|  |  +---------------- Bool: good requests completed\n"
-	"\t|  +------------------- How long the session was open\n"
-	"\t+---------------------- Why the connection closed\n"
+	"\t%s %f\n"
+	"\t|  |\n"
+	"\t|  +- How long the session was open\n"
+	"\t+---- Why the connection closed\n"
 	"\n"
 )
 



From martin at varnish-software.com  Tue Apr  1 13:09:55 2014
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Tue, 01 Apr 2014 15:09:55 +0200
Subject: [4.0] 6f2e1bc Request handling byte accounting
Message-ID: <E1WUyRv-0003LR-GC@project.varnish-software.com>


commit 6f2e1bcd343a7a745ac01aa61e8ce0e66b3bb6cc
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Fri Mar 28 11:25:38 2014 +0100

    Request handling byte accounting
    
    Counters show bytes received/transmitted, split between header and
    body bytes.
    
    SLT_ReqAcct logs per request counter values.

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index b793467..de7d895 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -228,9 +228,9 @@ struct http_conn {
 
 /*--------------------------------------------------------------------*/
 
-struct acct {
+struct acct_req {
 #define ACCT(foo)	uint64_t	foo;
-#include "tbl/acct_fields.h"
+#include "tbl/acct_fields_req.h"
 #undef ACCT
 };
 
@@ -654,9 +654,9 @@ struct req {
 	double			d_ttl;
 
 	unsigned char		wantbody;
-	uint64_t		req_bodybytes;
+	uint64_t		req_bodybytes;	/* Parsed req bodybytes */
 
-	uint64_t		resp_bodybytes;
+	uint64_t		resp_hdrbytes;	/* Scheduled resp hdrbytes */
 
 	uint16_t		err_code;
 	const char		*err_reason;
@@ -716,7 +716,7 @@ struct req {
 	struct vsl_log		vsl[1];
 
 	/* Temporary accounting */
-	struct acct		acct_req;
+	struct acct_req		acct;
 
 	/* Synth content in vcl_error */
 	struct vsb		*synth_body;
@@ -847,6 +847,7 @@ int HTTP1_CacheReqBody(struct req *req, ssize_t maxsize);
 int HTTP1_IterateReqBody(struct req *req, req_body_iter_f *func, void *priv);
 
 /* cache_http1_deliver.c */
+unsigned V1D_FlushReleaseAcct(struct req *req);
 void V1D_Deliver(struct req *);
 void V1D_Deliver_Synth(struct req *req);
 
@@ -892,6 +893,7 @@ VDP_pop(struct req *req, vdp_bytes *func)
 
 /* cache_req_fsm.c [CNT] */
 enum req_fsm_nxt CNT_Request(struct worker *, struct req *);
+void CNT_AcctLogCharge(struct dstat *, struct req *);
 
 /* cache_cli.c [CLI] */
 void CLI_Init(void);
@@ -1099,14 +1101,13 @@ void WRW_Chunked(const struct worker *w);
 void WRW_EndChunk(const struct worker *w);
 void WRW_Reserve(struct worker *w, int *fd, struct vsl_log *, double t0);
 unsigned WRW_Flush(const struct worker *w);
-unsigned WRW_FlushRelease(struct worker *w);
+unsigned WRW_FlushRelease(struct worker *w, ssize_t *pacc);
 unsigned WRW_Write(const struct worker *w, const void *ptr, int len);
 unsigned WRW_WriteH(const struct worker *w, const txt *hh, const char *suf);
 
 /* cache_session.c [SES] */
 void SES_Close(struct sess *sp, enum sess_close reason);
 void SES_Delete(struct sess *sp, enum sess_close reason, double now);
-void SES_Charge(struct worker *, struct req *);
 struct sesspool *SES_NewPool(struct pool *pp, unsigned pool_no);
 void SES_DeletePool(struct sesspool *sp);
 int SES_ScheduleReq(struct req *);
diff --git a/bin/varnishd/cache/cache_cli.c b/bin/varnishd/cache/cache_cli.c
index 8aa022b..ca9da16 100644
--- a/bin/varnishd/cache/cache_cli.c
+++ b/bin/varnishd/cache/cache_cli.c
@@ -126,7 +126,7 @@ cli_debug_sizeof(struct cli *cli, const char * const *av, void *priv)
 	SZOF(struct ws);
 	SZOF(struct http);
 	SZOF(struct http_conn);
-	SZOF(struct acct);
+	SZOF(struct acct_req);
 	SZOF(struct worker);
 	SZOF(struct wrk_accept);
 	SZOF(struct storage);
diff --git a/bin/varnishd/cache/cache_esi_deliver.c b/bin/varnishd/cache/cache_esi_deliver.c
index b1c3362..dbbe7b1 100644
--- a/bin/varnishd/cache/cache_esi_deliver.c
+++ b/bin/varnishd/cache/cache_esi_deliver.c
@@ -55,7 +55,7 @@ ved_include(struct req *preq, const char *src, const char *host)
 	if (preq->esi_level >= cache_param->max_esi_depth)
 		return;
 
-	(void)WRW_FlushRelease(wrk);
+	(void)V1D_FlushReleaseAcct(preq);
 
 	/* Take a workspace snapshot */
 	wrk_ws_wm = WS_Snapshot(wrk->aws); /* XXX ? */
@@ -120,9 +120,17 @@ ved_include(struct req *preq, const char *src, const char *host)
 		AZ(req->wrk);
 		(void)usleep(10000);
 	}
+	AN(WRW_IsReleased(wrk));
 
-	/* Make sure the VSL id has been released */
-	AZ(req->vsl->wid);
+	/* Flush and release the log */
+	AN(req->vsl->wid);
+	VSLb(req->vsl, SLT_End, "%s", "");
+	VSL_Flush(req->vsl, 0);
+	req->vsl->wid = 0;
+
+	/* Charge the transmitted body byte counts to the parent request */
+	preq->acct.resp_bodybytes += req->acct.resp_bodybytes;
+	req->acct.resp_bodybytes = 0;
 
 	/* Reset the workspace */
 	WS_Reset(wrk->aws, wrk_ws_wm);	/* XXX ? */
diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index 6cb2de8..1530937 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -501,7 +501,6 @@ HSH_Lookup(struct req *req, struct objcore **ocp, struct objcore **bocp,
 	}
 
 	wrk->stats.busy_sleep++;
-	SES_Charge(wrk, req);
 	/*
 	 * The objhead reference transfers to the sess, we get it
 	 * back when the sess comes off the waiting list and
diff --git a/bin/varnishd/cache/cache_http1_deliver.c b/bin/varnishd/cache/cache_http1_deliver.c
index e82ed31..5e99668 100644
--- a/bin/varnishd/cache/cache_http1_deliver.c
+++ b/bin/varnishd/cache/cache_http1_deliver.c
@@ -46,8 +46,6 @@ v1d_bytes(struct req *req, enum vdp_action act, const void *ptr, ssize_t len)
 
 	if (len > 0)
 		wl = WRW_Write(req->wrk, ptr, len);
-	if (wl > 0)
-		req->acct_req.bodybytes += wl;
 	if (act > VDP_NULL && WRW_Flush(req->wrk))
 		return (-1);
 	if (len != wl)
@@ -193,6 +191,33 @@ v1d_WriteDirObj(struct req *req)
 	return (ois);
 }
 
+/*--------------------------------------------------------------------
+ * V1D_FlushReleaseAcct()
+ * Call WRW_FlushRelease on the worker and update the requests
+ * byte accounting with the number of bytes transmitted
+ *
+ * Returns the return value from WRW_FlushRelease()
+ */
+unsigned
+V1D_FlushReleaseAcct(struct req *req)
+{
+	unsigned u;
+	ssize_t txcnt = 0, hdrbytes;
+
+	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
+	CHECK_OBJ_NOTNULL(req->wrk, WORKER_MAGIC);
+	u = WRW_FlushRelease(req->wrk, &txcnt);
+	if (req->acct.resp_hdrbytes < req->resp_hdrbytes) {
+		hdrbytes = req->resp_hdrbytes - req->acct.resp_hdrbytes;
+		if (hdrbytes > txcnt)
+			hdrbytes = txcnt;
+	} else
+		hdrbytes = 0;
+	req->acct.resp_hdrbytes += hdrbytes;
+	req->acct.resp_bodybytes += txcnt - hdrbytes;
+	return (u);
+}
+
 void
 V1D_Deliver(struct req *req)
 {
@@ -288,7 +313,7 @@ V1D_Deliver(struct req *req)
 	 * Send HTTP protocol header, unless interior ESI object
 	 */
 	if (!(req->res_mode & RES_ESI_CHILD))
-		req->acct_req.hdrbytes += HTTP1_Write(req->wrk, req->resp, 1);
+		req->resp_hdrbytes += HTTP1_Write(req->wrk, req->resp, 1);
 
 	if (req->res_mode & RES_CHUNKED)
 		WRW_Chunked(req->wrk);
@@ -320,7 +345,7 @@ V1D_Deliver(struct req *req)
 	    !(req->res_mode & RES_ESI_CHILD))
 		WRW_EndChunk(req->wrk);
 
-	if ((WRW_FlushRelease(req->wrk) || ois != OIS_DONE) && req->sp->fd >= 0)
+	if ((V1D_FlushReleaseAcct(req) || ois != OIS_DONE) && req->sp->fd >= 0)
 		SES_Close(req->sp, SC_REM_CLOSE);
 }
 
@@ -395,7 +420,7 @@ V1D_Deliver_Synth(struct req *req)
 	 * Send HTTP protocol header, unless interior ESI object
 	 */
 	if (!(req->res_mode & RES_ESI_CHILD))
-		req->acct_req.hdrbytes += HTTP1_Write(req->wrk, req->resp, 1);
+		req->resp_hdrbytes += HTTP1_Write(req->wrk, req->resp, 1);
 
 	if (req->res_mode & RES_CHUNKED)
 		WRW_Chunked(req->wrk);
@@ -418,6 +443,6 @@ V1D_Deliver_Synth(struct req *req)
 	if (req->res_mode & RES_CHUNKED && !(req->res_mode & RES_ESI_CHILD))
 		WRW_EndChunk(req->wrk);
 
-	if (WRW_FlushRelease(req->wrk) && req->sp->fd >= 0)
+	if (V1D_FlushReleaseAcct(req) && req->sp->fd >= 0)
 		SES_Close(req->sp, SC_REM_CLOSE);
 }
diff --git a/bin/varnishd/cache/cache_http1_fetch.c b/bin/varnishd/cache/cache_http1_fetch.c
index 317bfcb..82dcab6 100644
--- a/bin/varnishd/cache/cache_http1_fetch.c
+++ b/bin/varnishd/cache/cache_http1_fetch.c
@@ -330,7 +330,7 @@ V1F_fetch_hdr(struct worker *wrk, struct busyobj *bo, struct req *req)
 		}
 	}
 
-	if (WRW_FlushRelease(wrk) || i != 0) {
+	if (WRW_FlushRelease(wrk, NULL) || i != 0) {
 		VSLb(bo->vsl, SLT_FetchError, "backend write error: %d (%s)",
 		    errno, strerror(errno));
 		VSLb_ts_busyobj(bo, "Bereq", W_TIM_real(wrk));
diff --git a/bin/varnishd/cache/cache_http1_fsm.c b/bin/varnishd/cache/cache_http1_fsm.c
index 37ed35a..a46534c 100644
--- a/bin/varnishd/cache/cache_http1_fsm.c
+++ b/bin/varnishd/cache/cache_http1_fsm.c
@@ -126,6 +126,7 @@ http1_wait(struct sess *sp, struct worker *wrk, struct req *req)
 				VSLb_ts_req(req, "Start", now);
 			VSLb_ts_req(req, "Req", now);
 			req->t_req = req->t_prev;
+			req->acct.req_hdrbytes += Tlen(req->htc->rxbuf);
 			return (REQ_FSM_MORE);
 		} else if (hs == HTTP1_ERROR_EOF) {
 			why = SC_REM_CLOSE;
@@ -161,6 +162,8 @@ http1_wait(struct sess *sp, struct worker *wrk, struct req *req)
 			}
 		}
 	}
+	req->acct.req_hdrbytes += Tlen(req->htc->rxbuf);
+	CNT_AcctLogCharge(&wrk->stats, req);
 	SES_ReleaseReq(req);
 	assert(why != SC_NULL);
 	SES_Delete(sp, why, now);
@@ -200,20 +203,28 @@ http1_cleanup(struct sess *sp, struct worker *wrk, struct req *req)
 		req->vcl = NULL;
 	}
 
+	/* Charge and log byte counters */
+	AN(req->vsl->wid);
+	CNT_AcctLogCharge(&wrk->stats, req);
+	req->req_bodybytes = 0;
+	req->resp_hdrbytes = 0;
+
+	/* Nuke the VXID. http1_dissect() will allocate a new one when
+	   necessary */
+	VSLb(req->vsl, SLT_End, "%s", "");
+	VSL_Flush(req->vsl, 0);
+	req->vsl->wid = 0;
+
 	if (!isnan(req->t_prev) && req->t_prev > 0.)
 		sp->t_idle = req->t_prev;
 	else
 		sp->t_idle = W_TIM_real(wrk);
-	VSL_Flush(req->vsl, 0);
 
 	req->t_first = NAN;
 	req->t_prev = NAN;
 	req->t_req = NAN;
 	req->req_body_status = REQ_BODY_INIT;
 
-	// req->req_bodybytes = 0;
-	req->resp_bodybytes = 0;
-
 	req->hash_always_miss = 0;
 	req->hash_ignore_busy = 0;
 
@@ -235,10 +246,17 @@ http1_cleanup(struct sess *sp, struct worker *wrk, struct req *req)
 	WS_Reset(wrk->aws, NULL);
 
 	if (HTTP1_Reinit(req->htc) == HTTP1_COMPLETE) {
+		AZ(req->vsl->wid);
+		req->vsl->wid = VXID_Get(&wrk->vxid_pool) | VSL_CLIENTMARKER;
+		VSLb(req->vsl, SLT_Begin, "req %u rxreq",
+		    req->sp->vxid & VSL_IDENTMASK);
+		VSL(SLT_Link, req->sp->vxid, "req %u rxreq",
+		    req->vsl->wid & VSL_IDENTMASK);
 		VSLb_ts_req(req, "Start", sp->t_idle);
 		VSLb_ts_req(req, "Req", sp->t_idle);
 		req->t_req = req->t_prev;
 		wrk->stats.sess_pipeline++;
+		req->acct.req_hdrbytes += Tlen(req->htc->rxbuf);
 		return (SESS_DONE_RET_START);
 	} else {
 		if (Tlen(req->htc->rxbuf))
@@ -285,6 +303,7 @@ http1_dissect(struct worker *wrk, struct req *req)
 	const char *r_411 = "HTTP/1.1 411 Length Required\r\n\r\n";
 	const char *r_417 = "HTTP/1.1 417 Expectation Failed\r\n\r\n";
 	char *p;
+	ssize_t r;
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
@@ -314,7 +333,9 @@ http1_dissect(struct worker *wrk, struct req *req)
 	/* If we could not even parse the request, just close */
 	if (req->err_code != 0) {
 		wrk->stats.client_req_400++;
-		(void)write(req->sp->fd, r_400, strlen(r_400));
+		r = write(req->sp->fd, r_400, strlen(r_400));
+		if (r > 0)
+			req->acct.resp_hdrbytes += r;
 		SES_Close(req->sp, SC_RX_JUNK);
 		return (REQ_FSM_DONE);
 	}
@@ -326,7 +347,9 @@ http1_dissect(struct worker *wrk, struct req *req)
 
 	if (req->req_body_status == REQ_BODY_FAIL) {
 		wrk->stats.client_req_411++;
-		(void)write(req->sp->fd, r_411, strlen(r_411));
+		r = write(req->sp->fd, r_411, strlen(r_411));
+		if (r > 0)
+			req->acct.resp_hdrbytes += r;
 		SES_Close(req->sp, SC_RX_JUNK);
 		return (REQ_FSM_DONE);
 	}
@@ -335,21 +358,25 @@ http1_dissect(struct worker *wrk, struct req *req)
 		if (strcasecmp(p, "100-continue")) {
 			wrk->stats.client_req_417++;
 			req->err_code = 417;
-			(void)write(req->sp->fd, r_417, strlen(r_417));
+			r = write(req->sp->fd, r_417, strlen(r_417));
+			if (r > 0)
+				req->resp_hdrbytes += r;
 			SES_Close(req->sp, SC_RX_JUNK);
 			return (REQ_FSM_DONE);
 		}
-		if (strlen(r_100) != write(req->sp->fd, r_100, strlen(r_100))) {
-			// XXX: stats counter ?
+		r = write(req->sp->fd, r_100, strlen(r_100));
+		if (r > 0)
+			req->acct.resp_hdrbytes += r;
+		if (r != strlen(r_100)) {
 			SES_Close(req->sp, SC_REM_CLOSE);
 			return (REQ_FSM_DONE);
 		}
 	}
 
 	wrk->stats.client_req++;
+	wrk->stats.s_req++;
 
 	AZ(req->err_code);
-	req->acct_req.req++;
 	req->ws_req = WS_Snapshot(req->ws);
 	req->doclose = http_DoConnection(req->http);
 
@@ -475,6 +502,7 @@ http1_iter_req_body(struct req *req, void *buf, ssize_t len)
 	}
 	req->h1.bytes_done += len;
 	req->h1.bytes_yet = req->req_bodybytes - req->h1.bytes_done;
+	req->acct.req_bodybytes += len;
 	return (len);
 }
 
diff --git a/bin/varnishd/cache/cache_pipe.c b/bin/varnishd/cache/cache_pipe.c
index 1a0a44f..7d77096 100644
--- a/bin/varnishd/cache/cache_pipe.c
+++ b/bin/varnishd/cache/cache_pipe.c
@@ -66,6 +66,7 @@ PipeRequest(struct req *req, struct busyobj *bo)
 	struct worker *wrk;
 	struct pollfd fds[2];
 	int i;
+	ssize_t txcnt;
 
 	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
 	CHECK_OBJ_NOTNULL(req->sp, SESS_MAGIC);
@@ -80,14 +81,13 @@ PipeRequest(struct req *req, struct busyobj *bo)
 	(void)VTCP_blocking(vc->fd);
 
 	WRW_Reserve(wrk, &vc->fd, bo->vsl, req->t_req);
-	req->acct_req.hdrbytes += HTTP1_Write(wrk, bo->bereq, 0);
+	(void)HTTP1_Write(wrk, bo->bereq, 0);
 
 	if (req->htc->pipeline.b != NULL)
-		req->acct_req.bodybytes +=
-		    WRW_Write(wrk, req->htc->pipeline.b,
+		(void)WRW_Write(wrk, req->htc->pipeline.b,
 		    Tlen(req->htc->pipeline));
 
-	i = WRW_FlushRelease(wrk);
+	i = WRW_FlushRelease(wrk, &txcnt);
 
 	VSLb_ts_req(req, "Pipe", W_TIM_real(wrk));
 
diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index 090e0b6..8888226 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -200,7 +200,7 @@ cnt_synth(struct worker *wrk, struct req *req)
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
 
-	req->acct_req.synth++;
+	wrk->stats.s_synth++;
 
 	HTTP_Setup(req->resp, req->ws, req->vsl, SLT_RespMethod);
 	h = req->resp;
@@ -281,7 +281,7 @@ cnt_fetch(struct worker *wrk, struct req *req)
 	CHECK_OBJ_NOTNULL(req->objcore, OBJCORE_MAGIC);
 	AZ(req->obj);
 
-	req->acct_req.fetch++;
+	wrk->stats.s_fetch++;
 	(void)HTTP1_DiscardReqBody(req);
 
 	if (req->objcore->flags & OC_F_FAILED) {
@@ -554,7 +554,7 @@ cnt_pass(struct worker *wrk, struct req *req)
 		req->req_step = R_STP_RESTART;
 		break;
 	case VCL_RET_FETCH:
-		req->acct_req.pass++;
+		wrk->stats.s_pass++;
 		req->objcore = HSH_Private(wrk);
 		CHECK_OBJ_NOTNULL(req->objcore, OBJCORE_MAGIC);
 		VBF_Fetch(wrk, req, req->objcore, NULL, VBF_PASS);
@@ -593,7 +593,7 @@ cnt_pipe(struct worker *wrk, struct req *req)
 	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
 	CHECK_OBJ_NOTNULL(req->vcl, VCL_CONF_MAGIC);
 
-	req->acct_req.pipe++;
+	wrk->stats.s_pipe++;
 	bo = VBO_GetBusyObj(wrk, req);
 	HTTP_Setup(bo->bereq, bo->ws, bo->vsl, SLT_BereqMethod);
 	http_FilterReq(bo->bereq, req->http, 0);	// XXX: 0 ?
@@ -897,33 +897,43 @@ CNT_Request(struct worker *wrk, struct req *req)
 	}
 	if (nxt == REQ_FSM_DONE) {
 		/* XXX: Workaround for pipe */
-		if (req->sp->fd >= 0) {
-			VSLb(req->vsl, SLT_Length, "%ju",
-			    (uintmax_t)req->resp_bodybytes);
-		}
 
 		while (!VTAILQ_EMPTY(&req->body)) {
 			st = VTAILQ_FIRST(&req->body);
 			VTAILQ_REMOVE(&req->body, st, list);
 			STV_free(st);
 		}
-
-		/* done == 2 was charged by cache_hash.c */
-		SES_Charge(wrk, req);
-
-		/*
-		 * Nuke the VXID, cache_http1_fsm.c::http1_dissect() will
-		 * allocate a new one when necessary.
-		 */
-		VSLb(req->vsl, SLT_End, "%s", "");
-		req->vsl->wid = 0;
-		req->wrk = NULL;
 	}
-
+	req->wrk = NULL;
 	assert(WRW_IsReleased(wrk));
 	return (nxt);
 }
 
+void
+CNT_AcctLogCharge(struct dstat *ds, struct req *req)
+{
+	struct acct_req *a;
+
+	AN(ds);
+	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
+
+	a = &req->acct;
+
+	VSLb(req->vsl, SLT_ReqAcct, "%ju %ju %ju %ju %ju %ju",
+	    (uintmax_t)a->req_hdrbytes,
+	    (uintmax_t)a->req_bodybytes,
+	    (uintmax_t)(a->req_hdrbytes + a->req_bodybytes),
+	    (uintmax_t)a->resp_hdrbytes,
+	    (uintmax_t)a->resp_bodybytes,
+	    (uintmax_t)(a->resp_hdrbytes + a->resp_bodybytes));
+
+#define ACCT(foo)			\
+	ds->s_##foo += a->foo;		\
+	a->foo = 0;
+#include "tbl/acct_fields_req.h"
+#undef ACCT
+}
+
 /*
 DOT }
 */
diff --git a/bin/varnishd/cache/cache_session.c b/bin/varnishd/cache/cache_session.c
index 8059c83..0c5d2f9 100644
--- a/bin/varnishd/cache/cache_session.c
+++ b/bin/varnishd/cache/cache_session.c
@@ -62,31 +62,6 @@ struct sesspool {
 };
 
 /*--------------------------------------------------------------------
- * Charge statistics from worker to request and session.
- */
-
-void
-SES_Charge(struct worker *wrk, struct req *req)
-{
-	struct sess *sp;
-	struct acct *a;
-
-	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
-	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
-	sp = req->sp;
-	CHECK_OBJ_NOTNULL(sp, SESS_MAGIC);
-
-	a = &req->acct_req;
-	req->resp_bodybytes += a->bodybytes;
-
-#define ACCT(foo)				\
-	wrk->stats.s_##foo += a->foo;		\
-	a->foo = 0;
-#include "tbl/acct_fields.h"
-#undef ACCT
-}
-
-/*--------------------------------------------------------------------
  * Get a new session, preferably by recycling an already ready one
  *
  * Layout is:
@@ -395,6 +370,9 @@ SES_GetReq(struct worker *wrk, struct sess *sp)
 
 	WS_Init(req->ws, "req", p, e - p);
 
+	req->req_bodybytes = 0;
+	req->resp_hdrbytes = 0;
+
 	req->t_first = NAN;
 	req->t_prev = NAN;
 	req->t_req = NAN;
@@ -411,10 +389,14 @@ SES_ReleaseReq(struct req *req)
 	struct sesspool *pp;
 
 	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
-	AZ(req->vcl);
-#define ACCT(foo)	AZ(req->acct_req.foo);
-#include "tbl/acct_fields.h"
+
+	/* Make sure the request counters have all been zeroed */
+#define ACCT(foo) \
+	AZ(req->acct.foo);
+#include "tbl/acct_fields_req.h"
 #undef ACCT
+
+	AZ(req->vcl);
 	sp = req->sp;
 	CHECK_OBJ_NOTNULL(sp, SESS_MAGIC);
 	pp = sp->sesspool;
diff --git a/bin/varnishd/cache/cache_wrw.c b/bin/varnishd/cache/cache_wrw.c
index adf2f11..a418ea0 100644
--- a/bin/varnishd/cache/cache_wrw.c
+++ b/bin/varnishd/cache/cache_wrw.c
@@ -58,6 +58,7 @@ struct wrw {
 	unsigned		ciov;	/* Chunked header marker */
 	double			t0;
 	struct vsl_log		*vsl;
+	ssize_t			cnt;	/* Flushed byte count */
 };
 
 /*--------------------------------------------------------------------
@@ -101,7 +102,7 @@ WRW_Reserve(struct worker *wrk, int *fd, struct vsl_log *vsl, double t0)
 }
 
 static void
-WRW_Release(struct worker *wrk)
+wrw_release(struct worker *wrk, ssize_t *pacc)
 {
 	struct wrw *wrw;
 
@@ -109,6 +110,8 @@ WRW_Release(struct worker *wrk)
 	wrw = wrk->wrw;
 	wrk->wrw = NULL;
 	CHECK_OBJ_NOTNULL(wrw, WRW_MAGIC);
+	if (pacc != NULL)
+		*pacc += wrw->cnt;
 	WS_Release(wrk->aws, 0);
 	WS_Reset(wrk->aws, NULL);
 }
@@ -171,6 +174,8 @@ WRW_Flush(const struct worker *wrk)
 		}
 
 		i = writev(*wrw->wfd, wrw->iov, wrw->niov);
+		if (i > 0)
+			wrw->cnt += i;
 		while (i != wrw->liov && i > 0) {
 			/* Remove sent data from start of I/O vector,
 			 * then retry; we hit a timeout, but some data
@@ -195,6 +200,8 @@ WRW_Flush(const struct worker *wrk)
 
 			wrw_prune(wrw, i);
 			i = writev(*wrw->wfd, wrw->iov, wrw->niov);
+			if (i > 0)
+				wrw->cnt += i;
 		}
 		if (i <= 0) {
 			wrw->werr++;
@@ -212,14 +219,14 @@ WRW_Flush(const struct worker *wrk)
 }
 
 unsigned
-WRW_FlushRelease(struct worker *wrk)
+WRW_FlushRelease(struct worker *wrk, ssize_t *pacc)
 {
 	unsigned u;
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	AN(wrk->wrw->wfd);
 	u = WRW_Flush(wrk);
-	WRW_Release(wrk);
+	wrw_release(wrk, pacc);
 	return (u);
 }
 
diff --git a/bin/varnishtest/tests/l00002.vtc b/bin/varnishtest/tests/l00002.vtc
new file mode 100644
index 0000000..0106324
--- /dev/null
+++ b/bin/varnishtest/tests/l00002.vtc
@@ -0,0 +1,108 @@
+varnishtest "Test request byte counters"
+
+server s1 {
+	rxreq
+	expect req.url == "/1"
+	txresp -bodylen 1000
+
+	rxreq
+	expect req.url == "/2"
+	txresp -bodylen 2000
+} -start
+
+varnish v1 -vcl+backend {
+	sub vcl_deliver {
+		unset resp.http.date;
+		unset resp.http.age;
+		unset resp.http.via;
+		unset resp.http.x-varnish;
+	}
+} -start
+
+# Request (1001):
+# GET /1 HTTP/1.1\r\n		 17 bytes
+# Content-Length: 4\r\n		 19 bytes
+# \r\n				  2 bytes
+# Total:			 38 bytes
+# Response:
+# HTTP/1.1 200 Ok\r\n		 17 bytes
+# Content-Length: 1000\r\n	 22 bytes
+# Connection: keep-alive\r\n	 24 bytes
+# Accept-Ranges: bytes\r\n	 22 bytes
+# \r\n				  2 bytes
+# Total:			 87 bytes
+
+# Request (1003):
+# GET /2 HTTP/1.1\r\n		 17 bytes
+# \r\n				  2 bytes
+# Total:			 19 bytes
+# Response:
+# HTTP/1.1 200 Ok\r\n		 17 bytes
+# Content-Length: 2000\r\n	 22 bytes
+# Connection: keep-alive\r\n	 24 bytes
+# Accept-Ranges: bytes\r\n	 22 bytes
+# \r\n				  2 bytes
+# Total:			 87 bytes
+
+# Request (1005):
+# GET /2 HTTP/1.1\r\n		 17 bytes
+# \r\n				  2 bytes
+# Total:			 19 bytes
+# Response:
+# HTTP/1.1 200 Ok\r\n		 17 bytes
+# Content-Length: 2000\r\n	 22 bytes
+# Connection: keep-alive\r\n	 24 bytes
+# Accept-Ranges: bytes\r\n	 22 bytes
+# \r\n				  2 bytes
+# Total:			 87 bytes
+
+# Request (1006):
+# GET\r\n			  5 bytes
+# \r\n				  2 bytes
+# Total:			  7 bytes
+# Response:
+# HTTP/1.1 400 Bad Request\r\n	 26 bytes
+# \r\n				  2 bytes
+# Total:			 28 bytes
+logexpect l1 -v v1 -g request {
+	expect 0 1001	Begin	"^req .* rxreq"
+	expect * =	ReqAcct	"^38 4 42 87 1000 1087$"
+	expect 0 =	End
+	expect 0 1002	Begin	"^bereq "
+	expect * =	End
+	expect 0 1003	Begin	"^req .* rxreq"
+	expect * =	ReqAcct "^19 0 19 87 2000 2087$"
+	expect 0 =	End
+	expect 0 1004	Begin	"^bereq"
+	expect * =	End
+	expect 0 1005	Begin	"^req .* rxreq"
+	expect * =	ReqAcct "^19 0 19 87 2000 2087$"
+	expect 0 =	End
+	expect 0 1006	Begin	"^req .* rxreq"
+	expect * =	ReqAcct	"^7 0 7 28 0 28$"
+	expect 0 =	End
+} -start
+
+# Request 1001
+client c1 {
+	txreq -url "/1" -body "asdf"
+	rxresp
+	expect resp.status == 200
+
+	send "GET /2 HTTP/1.1\r\n\r\nGET /2 HTTP/1.1\r\n\r\n"
+	rxresp
+	expect resp.status == 200
+	rxresp
+	expect resp.status == 200
+
+	send "GET\r\n\r\n"
+	rxresp
+	expect resp.status == 400
+} -run
+
+logexpect l1 -wait
+
+varnish v1 -expect s_req_hdrbytes == 83
+varnish v1 -expect s_req_bodybytes == 4
+varnish v1 -expect s_resp_hdrbytes == 289
+varnish v1 -expect s_resp_bodybytes == 5000
diff --git a/include/Makefile.am b/include/Makefile.am
index 3813836..e996fc0 100644
--- a/include/Makefile.am
+++ b/include/Makefile.am
@@ -2,7 +2,7 @@
 
 # API headers
 nobase_pkginclude_HEADERS = \
-	tbl/acct_fields.h \
+	tbl/acct_fields_req.h \
 	tbl/backend_poll.h \
 	tbl/ban_vars.h \
 	tbl/bo_flags.h \
diff --git a/include/tbl/acct_fields.h b/include/tbl/acct_fields.h
deleted file mode 100644
index ac205a7..0000000
--- a/include/tbl/acct_fields.h
+++ /dev/null
@@ -1,40 +0,0 @@
-/*-
- * Copyright (c) 2008 Verdens Gang AS
- * Copyright (c) 2008-2009 Varnish Software AS
- * All rights reserved.
- *
- * Author: Poul-Henning Kamp <phk at phk.freebsd.dk>
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
- * 1. Redistributions of source code must retain the above copyright
- *    notice, this list of conditions and the following disclaimer.
- * 2. Redistributions in binary form must reproduce the above copyright
- *    notice, this list of conditions and the following disclaimer in the
- *    documentation and/or other materials provided with the distribution.
- *
- * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
- * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED.  IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
- * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
- * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
- * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
- * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
- * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
- * SUCH DAMAGE.
- *
- * These are the stats we keep track of per session.
- * SES_Charge() sums them into wrk->stats
- * NB: Remember to mark those in vsc_fields.h to be included in struct dstat.
- */
-
-ACCT(req)
-ACCT(pipe)
-ACCT(pass)
-ACCT(fetch)
-ACCT(synth)
-ACCT(hdrbytes)
-ACCT(bodybytes)
diff --git a/include/tbl/acct_fields_req.h b/include/tbl/acct_fields_req.h
new file mode 100644
index 0000000..a1759f5
--- /dev/null
+++ b/include/tbl/acct_fields_req.h
@@ -0,0 +1,36 @@
+/*-
+ * Copyright (c) 2008 Verdens Gang AS
+ * Copyright (c) 2008-2014 Varnish Software AS
+ * All rights reserved.
+ *
+ * Author: Poul-Henning Kamp <phk at phk.freebsd.dk>
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED.  IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ *
+ * These are the stats we keep track of per request.
+ * NB: Remember to mark those in vsc_fields.h to be included in struct dstat.
+ */
+
+ACCT(req_hdrbytes)
+ACCT(req_bodybytes)
+ACCT(resp_hdrbytes)
+ACCT(resp_bodybytes)
diff --git a/include/tbl/vsc_f_main.h b/include/tbl/vsc_f_main.h
index 3874ec1..859a2eb 100644
--- a/include/tbl/vsc_f_main.h
+++ b/include/tbl/vsc_f_main.h
@@ -384,13 +384,21 @@ VSC_F(s_synth,			uint64_t, 1, 'a', info,
     "Total synth",
 	""
 )
-VSC_F(s_hdrbytes,		uint64_t, 1, 'a', info,
-    "Total header bytes",
-	""
+VSC_F(s_req_hdrbytes,		uint64_t, 1, 'a', info,
+    "Request header bytes",
+	"Total request header bytes received"
 )
-VSC_F(s_bodybytes,		uint64_t, 1, 'a', info,
-    "Total body bytes",
-	""
+VSC_F(s_req_bodybytes,		uint64_t, 1, 'a', info,
+    "Request body bytes",
+	"Total request body bytes received"
+)
+VSC_F(s_resp_hdrbytes,		uint64_t, 1, 'a', info,
+    "Response header bytes",
+	"Total response header bytes transmitted"
+)
+VSC_F(s_resp_bodybytes,		uint64_t, 1, 'a', info,
+    "Reponse body bytes",
+	"Total response body bytes transmitted"
 )
 
 VSC_F(sess_closed,		uint64_t, 1, 'a', info,
diff --git a/include/tbl/vsl_tags.h b/include/tbl/vsl_tags.h
index 10a3a15..252b859 100644
--- a/include/tbl/vsl_tags.h
+++ b/include/tbl/vsl_tags.h
@@ -474,4 +474,18 @@ SLTM(Timestamp, 0, "Timing information",
 	"\n"
 )
 
+SLTM(ReqAcct, 0, "Request handling byte counts",
+	"Contains byte counts for the request handling.\n\n"
+	"The format is::\n\n"
+	"\t%d %d %d %d %d %d\n"
+	"\t|  |  |  |  |  |\n"
+	"\t|  |  |  |  |  +- Total bytes transmitted\n"
+	"\t|  |  |  |  +---- Body bytes transmitted\n"
+	"\t|  |  |  +------- Header bytes transmitted\n"
+	"\t|  |  +---------- Total bytes received\n"
+	"\t|  +------------- Body bytes received\n"
+	"\t+---------------- Header bytes received\n"
+	"\n"
+)
+
 #undef NODEF_NOTICE



From martin at varnish-software.com  Tue Apr  1 13:09:55 2014
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Tue, 01 Apr 2014 15:09:55 +0200
Subject: [4.0] c654dc5 ESI byte accounting
Message-ID: <E1WUyRv-0003Lb-HV@project.varnish-software.com>


commit c654dc51e6bbb5b58ab8f9cc80b2cd6b12c85e1e
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Sat Mar 29 23:04:07 2014 +0100

    ESI byte accounting
    
    Charge ESI bodybytes to the parent request.
    
    Log SLT_ESI_BodyBytes in each ESI-involved log transaction. This shows
    how many bytes each request adds to the total byte count.
    
    SLT_ReqAcct is not logged for ESI sub-requests.

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index de7d895..2e41e24 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -657,6 +657,7 @@ struct req {
 	uint64_t		req_bodybytes;	/* Parsed req bodybytes */
 
 	uint64_t		resp_hdrbytes;	/* Scheduled resp hdrbytes */
+	uint64_t		resp_bodybytes; /* Scheduled resp bodybytes */
 
 	uint16_t		err_code;
 	const char		*err_reason;
diff --git a/bin/varnishd/cache/cache_esi_deliver.c b/bin/varnishd/cache/cache_esi_deliver.c
index dbbe7b1..5aa85ee 100644
--- a/bin/varnishd/cache/cache_esi_deliver.c
+++ b/bin/varnishd/cache/cache_esi_deliver.c
@@ -218,15 +218,17 @@ ved_pretend_gzip(struct req *req, const uint8_t *p, ssize_t l)
 	while (l > 0) {
 		if (l >= 65535) {
 			lx = 65535;
-			(void)WRW_Write(req->wrk, buf1, sizeof buf1);
+			req->resp_bodybytes +=
+			    WRW_Write(req->wrk, buf1, sizeof buf1);
 		} else {
 			lx = (uint16_t)l;
 			buf2[0] = 0;
 			vle16enc(buf2 + 1, lx);
 			vle16enc(buf2 + 3, ~lx);
-			(void)WRW_Write(req->wrk, buf2, sizeof buf2);
+			req->resp_bodybytes +=
+			    WRW_Write(req->wrk, buf2, sizeof buf2);
 		}
-		(void)WRW_Write(req->wrk, p, lx);
+		req->resp_bodybytes += WRW_Write(req->wrk, p, lx);
 		req->crc = crc32(req->crc, p, lx);
 		req->l_crc += lx;
 		l -= lx;
@@ -283,7 +285,8 @@ ESI_Deliver(struct req *req)
 		if (isgzip && !(req->res_mode & RES_GUNZIP)) {
 			assert(sizeof gzip_hdr == 10);
 			/* Send out the gzip header */
-			(void)WRW_Write(req->wrk, gzip_hdr, 10);
+			req->resp_bodybytes +=
+			    WRW_Write(req->wrk, gzip_hdr, 10);
 			req->l_crc = 0;
 			req->gzip_resp = 1;
 			req->crc = crc32(0L, Z_NULL, 0);
@@ -339,8 +342,9 @@ ESI_Deliver(struct req *req)
 					 * We have a gzip'ed VEC and delivers
 					 * a gzip'ed ESI response.
 					 */
-					(void)WRW_Write(req->wrk,
-					    st->ptr + off, l2);
+					req->resp_bodybytes +=
+					    WRW_Write(req->wrk,
+						st->ptr + off, l2);
 				} else if (req->gzip_resp) {
 					/*
 					 * A gzip'ed ESI response, but the VEC
@@ -367,8 +371,9 @@ ESI_Deliver(struct req *req)
 					/*
 					 * Ungzip'ed VEC, ungzip'ed ESI response
 					 */
-					(void)WRW_Write(req->wrk,
-					    st->ptr + off, l2);
+					req->resp_bodybytes +=
+					    WRW_Write(req->wrk,
+						st->ptr + off, l2);
 				}
 				off += l2;
 				if (off == st->len) {
@@ -441,7 +446,7 @@ ESI_Deliver(struct req *req)
 		/* MOD(2^32) length */
 		vle32enc(tailbuf + 9, req->l_crc);
 
-		(void)WRW_Write(req->wrk, tailbuf, 13);
+		req->resp_bodybytes += WRW_Write(req->wrk, tailbuf, 13);
 	}
 	(void)WRW_Flush(req->wrk);
 }
@@ -451,7 +456,7 @@ ESI_Deliver(struct req *req)
  */
 
 static uint8_t
-ved_deliver_byterange(const struct req *req, ssize_t low, ssize_t high)
+ved_deliver_byterange(struct req *req, ssize_t low, ssize_t high)
 {
 	struct storage *st;
 	ssize_t l, lx;
@@ -477,7 +482,7 @@ ved_deliver_byterange(const struct req *req, ssize_t low, ssize_t high)
 			l = high - lx;
 		assert(lx >= low && lx + l <= high);
 		if (l != 0)
-			(void)WRW_Write(req->wrk, p, l);
+			req->resp_bodybytes += WRW_Write(req->wrk, p, l);
 		if (p + l < st->ptr + st->len)
 			return(p[l]);
 		lx += l;
@@ -532,7 +537,7 @@ ESI_DeliverChild(struct req *req)
 	 */
 	*dbits = ved_deliver_byterange(req, start/8, last/8);
 	*dbits &= ~(1U << (last & 7));
-	(void)WRW_Write(req->wrk, dbits, 1);
+	req->resp_bodybytes += WRW_Write(req->wrk, dbits, 1);
 	cc = ved_deliver_byterange(req, 1 + last/8, stop/8);
 	switch((int)(stop & 7)) {
 	case 0: /* xxxxxxxx */
@@ -576,7 +581,7 @@ ESI_DeliverChild(struct req *req)
 		INCOMPL();
 	}
 	if (lpad > 0)
-		(void)WRW_Write(req->wrk, dbits + 1, lpad);
+		req->resp_bodybytes += WRW_Write(req->wrk, dbits + 1, lpad);
 
 	/* We need the entire tail, but it may not be in one storage segment */
 	st = VTAILQ_LAST(&req->obj->store, storagehead);
diff --git a/bin/varnishd/cache/cache_http1_deliver.c b/bin/varnishd/cache/cache_http1_deliver.c
index 5e99668..765fd83 100644
--- a/bin/varnishd/cache/cache_http1_deliver.c
+++ b/bin/varnishd/cache/cache_http1_deliver.c
@@ -46,6 +46,7 @@ v1d_bytes(struct req *req, enum vdp_action act, const void *ptr, ssize_t len)
 
 	if (len > 0)
 		wl = WRW_Write(req->wrk, ptr, len);
+	req->resp_bodybytes += wl;
 	if (act > VDP_NULL && WRW_Flush(req->wrk))
 		return (-1);
 	if (len != wl)
diff --git a/bin/varnishd/cache/cache_http1_fsm.c b/bin/varnishd/cache/cache_http1_fsm.c
index a46534c..9bb5e79 100644
--- a/bin/varnishd/cache/cache_http1_fsm.c
+++ b/bin/varnishd/cache/cache_http1_fsm.c
@@ -208,6 +208,7 @@ http1_cleanup(struct sess *sp, struct worker *wrk, struct req *req)
 	CNT_AcctLogCharge(&wrk->stats, req);
 	req->req_bodybytes = 0;
 	req->resp_hdrbytes = 0;
+	req->resp_bodybytes = 0;
 
 	/* Nuke the VXID. http1_dissect() will allocate a new one when
 	   necessary */
diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index 8888226..2236553 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -897,6 +897,10 @@ CNT_Request(struct worker *wrk, struct req *req)
 	}
 	if (nxt == REQ_FSM_DONE) {
 		/* XXX: Workaround for pipe */
+		AN(req->vsl->wid);
+		if (req->res_mode & (RES_ESI|RES_ESI_CHILD))
+			VSLb(req->vsl, SLT_ESI_BodyBytes, "%ju",
+			    (uintmax_t)req->resp_bodybytes);
 
 		while (!VTAILQ_EMPTY(&req->body)) {
 			st = VTAILQ_FIRST(&req->body);
diff --git a/bin/varnishd/cache/cache_session.c b/bin/varnishd/cache/cache_session.c
index 0c5d2f9..0c010c1 100644
--- a/bin/varnishd/cache/cache_session.c
+++ b/bin/varnishd/cache/cache_session.c
@@ -372,6 +372,7 @@ SES_GetReq(struct worker *wrk, struct sess *sp)
 
 	req->req_bodybytes = 0;
 	req->resp_hdrbytes = 0;
+	req->resp_bodybytes = 0;
 
 	req->t_first = NAN;
 	req->t_prev = NAN;
diff --git a/bin/varnishtest/tests/l00003.vtc b/bin/varnishtest/tests/l00003.vtc
new file mode 100644
index 0000000..cb03a18
--- /dev/null
+++ b/bin/varnishtest/tests/l00003.vtc
@@ -0,0 +1,84 @@
+varnishtest "Test request byte counters with ESI"
+
+server s1 {
+	rxreq
+	expect req.url == "/"
+	txresp -body {<esi:include src="/1"/>ghi}
+
+	rxreq
+	expect req.url == "/1"
+	txresp -body {<esi:include src="/2"/>abc<esi:include src="/2"/>def}
+
+	rxreq
+	expect req.url == "/2"
+	txresp -body {123}
+} -start
+
+varnish v1 -vcl+backend {
+	sub vcl_backend_response {
+		set beresp.do_esi = true;
+	}
+	sub vcl_deliver {
+		unset resp.http.date;
+		unset resp.http.age;
+		unset resp.http.via;
+		unset resp.http.x-varnish;
+	}
+} -start
+
+# Request (1001):
+# GET / HTTP/1.1\r\n			16 bytes
+# \r\n					 2 bytes
+# Total:				18 bytes
+
+# Reponse:
+# HTTP/1.1 200 Ok\r\n			17 bytes
+# Transfer-Encoding: chunked\r\n	28 bytes
+# Connection: keep-alive\r\n		24 bytes
+# \r\n					 2 bytes
+# Total:				71 bytes
+
+# Response body:
+# 003\r\n				 5 bytes
+# 123\r\n				 5 bytes
+# 003\r\n				 5 bytes
+# abc\r\n				 5 bytes
+# 003\r\n				 5 bytes
+# 123\r\n				 5 bytes
+# 003\r\n				 5 bytes
+# def\r\n				 5 bytes
+# 003\r\n				 5 bytes
+# ghi\r\n				 5 bytes
+# 0\r\n					 3 bytes
+# \r\n					 2 bytes
+# Total:				55 bytes
+
+logexpect l1 -v v1 -g request {
+	expect 0 1001	Begin	"^req .* rxreq"
+	expect * =	ESI_BodyBytes	"^3"
+	expect * =	ReqAcct		"^18 0 18 71 55 126$"
+	expect 0 =	End
+	expect * 1003	Begin		"^req .* esi"
+	expect * =	ESI_BodyBytes	"^6$"
+	expect 0 =	End
+	expect * 1005	Begin		"^req .* esi"
+	expect * =	ESI_BodyBytes	"^3$"
+	expect 0 =	End
+	expect * 1007	Begin		"^req .* esi"
+	expect * =	ESI_BodyBytes	"^3$"
+	expect 0 =	End
+} -start
+
+client c1 {
+	txreq -url "/"
+	rxresp
+	expect resp.status == 200
+	expect resp.body == "123abc123defghi"
+} -run
+
+logexpect l1 -wait
+
+varnish v1 -expect s_req_hdrbytes == 18
+varnish v1 -expect s_req_bodybytes == 0
+varnish v1 -expect s_resp_hdrbytes == 71
+varnish v1 -expect s_resp_bodybytes == 55
diff --git a/include/tbl/vsl_tags.h b/include/tbl/vsl_tags.h
index 252b859..0213a61 100644
--- a/include/tbl/vsl_tags.h
+++ b/include/tbl/vsl_tags.h
@@ -475,7 +475,9 @@ SLTM(Timestamp, 0, "Timing information",
 )
 
 SLTM(ReqAcct, 0, "Request handling byte counts",
-	"Contains byte counts for the request handling.\n\n"
+	"Contains byte counts for the request handling. This record is not"
+	" logged for ESI sub-requests, but the sub-requests' response"
+	" body count is added to the main request.\n\n"
 	"The format is::\n\n"
 	"\t%d %d %d %d %d %d\n"
 	"\t|  |  |  |  |  |\n"
@@ -488,4 +490,14 @@ SLTM(ReqAcct, 0, "Request handling byte counts",
 	"\n"
 )
 
+SLTM(ESI_BodyBytes, 0, "ESI body fragment byte counter",
+	"Contains the body byte count for this ESI body fragment."
+	" This number does not include any transfer encoding overhead.\n\n"
+	"The format is::\n\n"
+	"\t%d\n"
+	"\t|\n"
+	"\t+- Body bytes\n"
+	"\n"
+)
+
 #undef NODEF_NOTICE



From martin at varnish-software.com  Tue Apr  1 13:09:55 2014
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Tue, 01 Apr 2014 15:09:55 +0200
Subject: [4.0] 6dfcd17 Pipe byte accounting
Message-ID: <E1WUyRv-0003Ll-IY@project.varnish-software.com>


commit 6dfcd17ba234e18314d5f9d739c93aca215a4160
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Sun Mar 30 01:52:17 2014 +0100

    Pipe byte accounting
    
    Add VSC counters for pipe traffic. Traffic is split between header
    bytes, and input and output piped bytes.
    
    Log SLT_PipeAcct records showing per pipe session byte counts.

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index 2e41e24..c58a1fc 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -699,6 +699,7 @@ struct req {
 #define RES_ESI			(1<<4)
 #define RES_ESI_CHILD		(1<<5)
 #define RES_GUNZIP		(1<<6)
+#define RES_PIPE		(1<<7)
 
 	/* Deliver pipeline */
 #define	N_VDPS			5
@@ -1088,6 +1089,7 @@ const char *reqbody_status_2str(enum req_body_state_e e);
 const char *sess_close_2str(enum sess_close sc, int want_desc);
 
 /* cache_pipe.c */
+void Pipe_Init(void);
 void PipeRequest(struct req *req, struct busyobj *bo);
 
 /* cache_pool.c */
diff --git a/bin/varnishd/cache/cache_main.c b/bin/varnishd/cache/cache_main.c
index a5674c0..c6a2599 100644
--- a/bin/varnishd/cache/cache_main.c
+++ b/bin/varnishd/cache/cache_main.c
@@ -219,6 +219,7 @@ child_main(void)
 	VBP_Init();
 	WRK_Init();
 	Pool_Init();
+	Pipe_Init();
 
 	EXP_Init();
 	HSH_Init(heritage.hash);
diff --git a/bin/varnishd/cache/cache_pipe.c b/bin/varnishd/cache/cache_pipe.c
index 7d77096..05b8d0a 100644
--- a/bin/varnishd/cache/cache_pipe.c
+++ b/bin/varnishd/cache/cache_pipe.c
@@ -40,8 +40,17 @@
 #include "vtcp.h"
 #include "vtim.h"
 
+static struct lock pipestat_mtx;
+
+struct acct_pipe {
+	ssize_t		req;
+	ssize_t		bereq;
+	ssize_t		in;
+	ssize_t		out;
+};
+
 static int
-rdf(int fd0, int fd1)
+rdf(int fd0, int fd1, ssize_t *pcnt)
 {
 	int i, j;
 	char buf[BUFSIZ], *p;
@@ -53,12 +62,30 @@ rdf(int fd0, int fd1)
 		j = write(fd1, p, i);
 		if (j <= 0)
 			return (1);
+		*pcnt += j;
 		if (i != j)
 			(void)usleep(100000);		/* XXX hack */
 	}
 	return (0);
 }
 
+static void
+pipecharge(struct req *req, const struct acct_pipe *a)
+{
+
+	VSLb(req->vsl, SLT_PipeAcct, "%ju %ju %ju %ju",
+	    (uintmax_t)a->req,
+	    (uintmax_t)a->bereq,
+	    (uintmax_t)a->in,
+	    (uintmax_t)a->out);
+
+	Lck_Lock(&pipestat_mtx);
+	VSC_C_main->s_pipe_hdrbytes += a->req;
+	VSC_C_main->s_pipe_in += a->in;
+	VSC_C_main->s_pipe_out += a->out;
+	Lck_Unlock(&pipestat_mtx);
+}
+
 void
 PipeRequest(struct req *req, struct busyobj *bo)
 {
@@ -66,7 +93,8 @@ PipeRequest(struct req *req, struct busyobj *bo)
 	struct worker *wrk;
 	struct pollfd fds[2];
 	int i;
-	ssize_t txcnt;
+	struct acct_pipe acct;
+	ssize_t hdrbytes;
 
 	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
 	CHECK_OBJ_NOTNULL(req->sp, SESS_MAGIC);
@@ -74,24 +102,38 @@ PipeRequest(struct req *req, struct busyobj *bo)
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
 
+	req->res_mode = RES_PIPE;
+
+	memset(&acct, 0, sizeof acct);
+	acct.req = req->acct.req_hdrbytes;
+	req->acct.req_hdrbytes = 0;
+
 	vc = VDI_GetFd(bo);
-	if (vc == NULL)
+	if (vc == NULL) {
+		pipecharge(req, &acct);
+		SES_Close(req->sp, SC_OVERLOAD);
 		return;
+	}
 	bo->vbc = vc;		/* For panic dumping */
 	(void)VTCP_blocking(vc->fd);
 
 	WRW_Reserve(wrk, &vc->fd, bo->vsl, req->t_req);
-	(void)HTTP1_Write(wrk, bo->bereq, 0);
+	hdrbytes = HTTP1_Write(wrk, bo->bereq, 0);
 
 	if (req->htc->pipeline.b != NULL)
 		(void)WRW_Write(wrk, req->htc->pipeline.b,
 		    Tlen(req->htc->pipeline));
 
-	i = WRW_FlushRelease(wrk, &txcnt);
+	i = WRW_FlushRelease(wrk, &acct.bereq);
+	if (acct.bereq > hdrbytes) {
+		acct.in = acct.bereq - hdrbytes;
+		acct.bereq = hdrbytes;
+	}
 
 	VSLb_ts_req(req, "Pipe", W_TIM_real(wrk));
 
 	if (i) {
+		pipecharge(req, &acct);
 		SES_Close(req->sp, SC_TX_PIPE);
 		VDI_CloseFd(&vc);
 		return;
@@ -113,7 +155,7 @@ PipeRequest(struct req *req, struct busyobj *bo)
 		i = poll(fds, 2, (int)(cache_param->pipe_timeout * 1e3));
 		if (i < 1)
 			break;
-		if (fds[0].revents && rdf(vc->fd, req->sp->fd)) {
+		if (fds[0].revents && rdf(vc->fd, req->sp->fd, &acct.out)) {
 			if (fds[1].fd == -1)
 				break;
 			(void)shutdown(vc->fd, SHUT_RD);
@@ -121,7 +163,7 @@ PipeRequest(struct req *req, struct busyobj *bo)
 			fds[0].events = 0;
 			fds[0].fd = -1;
 		}
-		if (fds[1].revents && rdf(req->sp->fd, vc->fd)) {
+		if (fds[1].revents && rdf(req->sp->fd, vc->fd, &acct.in)) {
 			if (fds[0].fd == -1)
 				break;
 			(void)shutdown(req->sp->fd, SHUT_RD);
@@ -131,7 +173,17 @@ PipeRequest(struct req *req, struct busyobj *bo)
 		}
 	}
 	VSLb_ts_req(req, "PipeSess", W_TIM_real(wrk));
+	pipecharge(req, &acct);
 	SES_Close(req->sp, SC_TX_PIPE);
 	VDI_CloseFd(&vc);
 	bo->vbc = NULL;
 }
+
+/*--------------------------------------------------------------------*/
+
+void
+Pipe_Init(void)
+{
+
+	Lck_New(&pipestat_mtx, lck_pipestat);
+}
diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index 2236553..1e9ecd5 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -896,7 +896,6 @@ CNT_Request(struct worker *wrk, struct req *req)
 		CHECK_OBJ_ORNULL(wrk->nobjhead, OBJHEAD_MAGIC);
 	}
 	if (nxt == REQ_FSM_DONE) {
-		/* XXX: Workaround for pipe */
 		AN(req->vsl->wid);
 		if (req->res_mode & (RES_ESI|RES_ESI_CHILD))
 			VSLb(req->vsl, SLT_ESI_BodyBytes, "%ju",
@@ -923,13 +922,15 @@ CNT_AcctLogCharge(struct dstat *ds, struct req *req)
 
 	a = &req->acct;
 
-	VSLb(req->vsl, SLT_ReqAcct, "%ju %ju %ju %ju %ju %ju",
-	    (uintmax_t)a->req_hdrbytes,
-	    (uintmax_t)a->req_bodybytes,
-	    (uintmax_t)(a->req_hdrbytes + a->req_bodybytes),
-	    (uintmax_t)a->resp_hdrbytes,
-	    (uintmax_t)a->resp_bodybytes,
-	    (uintmax_t)(a->resp_hdrbytes + a->resp_bodybytes));
+	if (!(req->res_mode & RES_PIPE)) {
+		VSLb(req->vsl, SLT_ReqAcct, "%ju %ju %ju %ju %ju %ju",
+		    (uintmax_t)a->req_hdrbytes,
+		    (uintmax_t)a->req_bodybytes,
+		    (uintmax_t)(a->req_hdrbytes + a->req_bodybytes),
+		    (uintmax_t)a->resp_hdrbytes,
+		    (uintmax_t)a->resp_bodybytes,
+		    (uintmax_t)(a->resp_hdrbytes + a->resp_bodybytes));
+	}
 
 #define ACCT(foo)			\
 	ds->s_##foo += a->foo;		\
diff --git a/bin/varnishtest/tests/l00004.vtc b/bin/varnishtest/tests/l00004.vtc
new file mode 100644
index 0000000..3b07e36
--- /dev/null
+++ b/bin/varnishtest/tests/l00004.vtc
@@ -0,0 +1,63 @@
+varnishtest "Test request byte counters on pipe"
+
+server s1 {
+	rxreq
+	expect req.url == "/"
+	expect req.http.test == "yes"
+	txresp -body "fdsa"
+} -start
+
+varnish v1 -vcl+backend {
+	sub vcl_recv {
+		return (pipe);
+	}
+
+	sub vcl_pipe {
+		set bereq.http.test = "yes";
+		unset bereq.http.x-forwarded-for;
+		unset bereq.http.x-varnish;
+		unset bereq.http.connection;
+	}
+} -start
+
+# req:
+# POST / HTTP/1.1\r\n			17 bytes
+# Content-Length: 4\r\n			19 bytes
+# \r\n					 2 bytes
+# Total:				38 bytes
+
+# bereq:
+# POST / HTTP/1.1\r\n			17 bytes
+# Content-Length: 4\r\n			19 bytes
+# test: yes\r\n				11 bytes
+# \r\n					 2 bytes
+# Total:				49 bytes
+
+# reqbody
+# asdf					 4 bytes
+
+# resp:
+# HTTP/1.1 200 Ok\r\n			17 bytes
+# Content-Length: 4\r\n			19 bytes
+# \r\n					 2 bytes
+# fdsa					 4 bytes
+# Total:				42 bytes
+
+logexpect l1 -v v1 -g request {
+	expect 0 1001	Begin		"^req .* rxreq"
+	expect * =	PipeAcct	"^38 49 4 42$"
+	expect 0 =	End
+} -start
+
+client c1 {
+	txreq -req "POST" -url "/" -hdr "Content-Length: 4"
+	send "asdf"
+	rxresp
+	expect resp.status == 200
+} -run
+
+logexpect l1 -wait
+
+varnish v1 -expect s_pipe_hdrbytes == 38
+varnish v1 -expect s_pipe_in == 4
+varnish v1 -expect s_pipe_out == 42
diff --git a/include/tbl/locks.h b/include/tbl/locks.h
index 8329bab..1a01782 100644
--- a/include/tbl/locks.h
+++ b/include/tbl/locks.h
@@ -53,4 +53,5 @@ LOCK(nbusyobj)
 LOCK(busyobj)
 LOCK(mempool)
 LOCK(vxid)
+LOCK(pipestat)
 /*lint -restore */
diff --git a/include/tbl/vsc_f_main.h b/include/tbl/vsc_f_main.h
index 859a2eb..4040b0d 100644
--- a/include/tbl/vsc_f_main.h
+++ b/include/tbl/vsc_f_main.h
@@ -400,6 +400,20 @@ VSC_F(s_resp_bodybytes,		uint64_t, 1, 'a', info,
     "Reponse body bytes",
 	"Total response body bytes transmitted"
 )
+VSC_F(s_pipe_hdrbytes,		uint64_t, 0, 'a', info,
+    "Pipe request header bytes",
+	"Total request bytes received for piped sessions"
+)
+VSC_F(s_pipe_in,		uint64_t, 0, 'a', info,
+    "Piped bytes from client",
+	"Total number of bytes forwarded from clients in"
+	" pipe sessions"
+)
+VSC_F(s_pipe_out,		uint64_t, 0, 'a', info,
+    "Piped bytes to client",
+	"Total number of bytes forwarded to clients in"
+	" pipe sessions"
+)
 
 VSC_F(sess_closed,		uint64_t, 1, 'a', info,
     "Session Closed",
diff --git a/include/tbl/vsl_tags.h b/include/tbl/vsl_tags.h
index 0213a61..99894cc 100644
--- a/include/tbl/vsl_tags.h
+++ b/include/tbl/vsl_tags.h
@@ -500,4 +500,16 @@ SLTM(ESI_BodyBytes, 0, "ESI body fragment byte counter",
 	"\n"
 )
 
+SLTM(PipeAcct, 0, "Pipe byte counts",
+	"Contains byte counters for pipe sessions.\n\n"
+	"The format is::\n\n"
+	"\t%d %d %d %d\n"
+	"\t|  |  |  |\n"
+	"\t|  |  |  +------- Piped bytes to client\n"
+	"\t|  |  +---------- Piped bytes from client\n"
+	"\t|  +------------- Backend request headers\n"
+	"\t+---------------- Client request headers\n"
+	"\n"
+)
+
 #undef NODEF_NOTICE



From martin at varnish-software.com  Tue Apr  1 13:09:55 2014
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Tue, 01 Apr 2014 15:09:55 +0200
Subject: [4.0] 3398877 Backend fetch byte accounting
Message-ID: <E1WUyRv-0003Lv-Ji@project.varnish-software.com>


commit 3398877276d228cae9cdcb3c2f5f9a5e7d8178a9
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Sun Mar 30 15:17:18 2014 +0200

    Backend fetch byte accounting
    
    Add SLT_BereqAcct log records showing headers and bodybytes
    received/transmitted when doing backend fetch operations.

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index c58a1fc..bf28983 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -236,6 +236,14 @@ struct acct_req {
 
 /*--------------------------------------------------------------------*/
 
+struct acct_bereq {
+#define ACCT(foo)	ssize_t		foo;
+#include "tbl/acct_fields_bereq.h"
+#undef ACCT
+};
+
+/*--------------------------------------------------------------------*/
+
 #define L0(t, n)
 #define L1(t, n)		t n;
 #define VSC_F(n,t,l,f,v,e,d)	L##l(t, n)
@@ -566,6 +574,9 @@ struct busyobj {
 	double			t_first;	/* First timestamp logged */
 	double			t_prev;		/* Previous timestamp logged */
 
+	/* Acct */
+	struct acct_bereq	acct;
+
 	const char		*storage_hint;
 	struct director		*director;
 	struct VCL_conf		*vcl;
diff --git a/bin/varnishd/cache/cache_busyobj.c b/bin/varnishd/cache/cache_busyobj.c
index 69f849b..22f6441 100644
--- a/bin/varnishd/cache/cache_busyobj.c
+++ b/bin/varnishd/cache/cache_busyobj.c
@@ -187,6 +187,14 @@ VBO_DerefBusyObj(struct worker *wrk, struct busyobj **pbo)
 	if (r)
 		return;
 
+	VSLb(bo->vsl, SLT_BereqAcct, "%ju %ju %ju %ju %ju %ju",
+	    (uintmax_t)bo->acct.bereq_hdrbytes,
+	    (uintmax_t)bo->acct.bereq_bodybytes,
+	    (uintmax_t)(bo->acct.bereq_hdrbytes + bo->acct.bereq_bodybytes),
+	    (uintmax_t)bo->acct.beresp_hdrbytes,
+	    (uintmax_t)bo->acct.beresp_bodybytes,
+	    (uintmax_t)(bo->acct.beresp_hdrbytes + bo->acct.beresp_bodybytes));
+
 	VSLb(bo->vsl, SLT_End, "%s", "");
 	VSL_Flush(bo->vsl, 0);
 
diff --git a/bin/varnishd/cache/cache_http1_fetch.c b/bin/varnishd/cache/cache_http1_fetch.c
index 82dcab6..b6fac38 100644
--- a/bin/varnishd/cache/cache_http1_fetch.c
+++ b/bin/varnishd/cache/cache_http1_fetch.c
@@ -91,6 +91,7 @@ v1f_pull_straight(struct busyobj *bo, void *p, ssize_t *lp, intptr_t *priv)
 	if (*priv < l)
 		l = *priv;
 	lr = HTTP1_Read(&bo->htc, p, l);
+	bo->acct.beresp_bodybytes += lr;
 	if (lr <= 0)
 		return (VFP_Error(bo, "straight insufficient bytes"));
 	*lp = lr;
@@ -127,8 +128,10 @@ v1f_pull_chunked(struct busyobj *bo, void *p, ssize_t *lp, intptr_t *priv)
 	if (*priv == -1) {
 		/* Skip leading whitespace */
 		do {
-			if (HTTP1_Read(&bo->htc, buf, 1) <= 0)
+			lr = HTTP1_Read(&bo->htc, buf, 1);
+			if (lr <= 0)
 				return (VFP_Error(bo, "chunked read err"));
+			bo->acct.beresp_bodybytes += lr;
 		} while (vct_islws(buf[0]));
 
 		if (!vct_ishex(buf[0]))
@@ -137,9 +140,11 @@ v1f_pull_chunked(struct busyobj *bo, void *p, ssize_t *lp, intptr_t *priv)
 		/* Collect hex digits, skipping leading zeros */
 		for (u = 1; u < sizeof buf; u++) {
 			do {
-				if (HTTP1_Read(&bo->htc, buf + u, 1) <= 0)
+				lr = HTTP1_Read(&bo->htc, buf + u, 1);
+				if (lr <= 0)
 					return (VFP_Error(bo,
 					    "chunked read err"));
+				bo->acct.beresp_bodybytes += lr;
 			} while (u == 1 && buf[0] == '0' && buf[u] == '0');
 			if (!vct_ishex(buf[u]))
 				break;
@@ -149,9 +154,12 @@ v1f_pull_chunked(struct busyobj *bo, void *p, ssize_t *lp, intptr_t *priv)
 			return (VFP_Error(bo,"chunked header too long"));
 
 		/* Skip trailing white space */
-		while(vct_islws(buf[u]) && buf[u] != '\n')
-			if (HTTP1_Read(&bo->htc, buf + u, 1) <= 0)
+		while(vct_islws(buf[u]) && buf[u] != '\n') {
+			lr = HTTP1_Read(&bo->htc, buf + u, 1);
+			if (lr <= 0)
 				return (VFP_Error(bo, "chunked read err"));
+			bo->acct.beresp_bodybytes += lr;
+		}
 
 		if (buf[u] != '\n')
 			return (VFP_Error(bo,"chunked header no NL"));
@@ -169,6 +177,7 @@ v1f_pull_chunked(struct busyobj *bo, void *p, ssize_t *lp, intptr_t *priv)
 		lr = HTTP1_Read(&bo->htc, p, l);
 		if (lr <= 0)
 			return (VFP_Error(bo, "straight insufficient bytes"));
+		bo->acct.beresp_bodybytes += lr;
 		*lp = lr;
 		*priv -= lr;
 		if (*priv == 0)
@@ -179,6 +188,7 @@ v1f_pull_chunked(struct busyobj *bo, void *p, ssize_t *lp, intptr_t *priv)
 	i = HTTP1_Read(&bo->htc, buf, 1);
 	if (i <= 0)
 		return (VFP_Error(bo, "chunked read err"));
+	bo->acct.beresp_bodybytes += i;
 	if (buf[0] == '\r' && HTTP1_Read(&bo->htc, buf, 1) <= 0)
 		return (VFP_Error(bo, "chunked read err"));
 	if (buf[0] != '\n')
@@ -281,8 +291,9 @@ V1F_fetch_hdr(struct worker *wrk, struct busyobj *bo, struct req *req)
 	struct http *hp;
 	enum htc_status_e hs;
 	int retry = -1;
-	int i, first;
+	int i, j, first;
 	struct http_conn *htc;
+	ssize_t hdrbytes;
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_ORNULL(req, REQ_MAGIC);
@@ -312,7 +323,7 @@ V1F_fetch_hdr(struct worker *wrk, struct busyobj *bo, struct req *req)
 
 	(void)VTCP_blocking(vc->fd);	/* XXX: we should timeout instead */
 	WRW_Reserve(wrk, &vc->fd, bo->vsl, bo->t_prev);
-	(void)HTTP1_Write(wrk, hp, 0);	/* XXX: stats ? */
+	hdrbytes = HTTP1_Write(wrk, hp, 0);
 
 	/* Deal with any message-body the request might (still) have */
 	i = 0;
@@ -330,7 +341,12 @@ V1F_fetch_hdr(struct worker *wrk, struct busyobj *bo, struct req *req)
 		}
 	}
 
-	if (WRW_FlushRelease(wrk, NULL) || i != 0) {
+	j = WRW_FlushRelease(wrk, &bo->acct.bereq_hdrbytes);
+	if (bo->acct.bereq_hdrbytes > hdrbytes) {
+		bo->acct.bereq_bodybytes = bo->acct.bereq_hdrbytes - hdrbytes;
+		bo->acct.bereq_hdrbytes = hdrbytes;
+	}
+	if (j != 0 || i != 0) {
 		VSLb(bo->vsl, SLT_FetchError, "backend write error: %d (%s)",
 		    errno, strerror(errno));
 		VSLb_ts_busyobj(bo, "Bereq", W_TIM_real(wrk));
@@ -356,6 +372,7 @@ V1F_fetch_hdr(struct worker *wrk, struct busyobj *bo, struct req *req)
 	do {
 		hs = HTTP1_Rx(htc);
 		if (hs == HTTP1_OVERFLOW) {
+			bo->acct.beresp_hdrbytes += Tlen(htc->rxbuf);
 			VSLb(bo->vsl, SLT_FetchError,
 			    "http %sread error: overflow",
 			    first ? "first " : "");
@@ -364,6 +381,7 @@ V1F_fetch_hdr(struct worker *wrk, struct busyobj *bo, struct req *req)
 			return (-1);
 		}
 		if (hs == HTTP1_ERROR_EOF) {
+			bo->acct.beresp_hdrbytes += Tlen(htc->rxbuf);
 			VSLb(bo->vsl, SLT_FetchError, "http %sread error: EOF",
 			    first ? "first " : "");
 			VDI_CloseFd(&bo->vbc);
@@ -377,6 +395,7 @@ V1F_fetch_hdr(struct worker *wrk, struct busyobj *bo, struct req *req)
 			    vc->between_bytes_timeout);
 		}
 	} while (hs != HTTP1_COMPLETE);
+	bo->acct.beresp_hdrbytes += Tlen(htc->rxbuf);
 
 	hp = bo->beresp;
 
diff --git a/bin/varnishtest/tests/l00005.vtc b/bin/varnishtest/tests/l00005.vtc
new file mode 100644
index 0000000..51d0d7e
--- /dev/null
+++ b/bin/varnishtest/tests/l00005.vtc
@@ -0,0 +1,68 @@
+varnishtest "Test backend fetch byte counters"
+
+server s1 {
+	rxreq
+	expect req.url == "/1"
+	txresp -bodylen 1000
+
+	rxreq
+	expect req.url == "/2"
+	send "HTTP/1.1\r\n\r\n"
+} -start
+
+varnish v1 -vcl+backend {
+	sub vcl_backend_fetch {
+		unset bereq.http.x-forwarded-for;
+		unset bereq.http.x-varnish;
+		set bereq.http.Host = "foo.bar";
+	}
+} -start
+
+# Request (1002):
+# POST /1 HTTP/1.1\r\n		 18 bytes
+# Content-Length: 4\r\n		 19 bytes
+# Host: foo.bar\r\n		 15 bytes
+# \r\n				  2 bytes
+# Total:			 54 bytes
+# Response:
+# HTTP/1.1 200 Ok\r\n		 17 bytes
+# Content-Length: 1000\r\n	 22 bytes
+# \r\n				  2 bytes
+# Total:			 41 bytes
+
+# Request (1004):
+# POST /2 HTTP/1.1\r\n		 18 bytes
+# Content-Length: 4\r\n		 19 bytes
+# Host: foo.bar\r\n		 15 bytes
+# \r\n				  2 bytes
+# Total:			 54 bytes
+# Reponse:
+# HTTP/1.1\r\n			 10 bytes
+# \r\n				  2 bytes
+# Total:			 12 bytes
+
+logexpect l1 -v v1 -g request {
+	expect 0 1001	Begin		"^req .* rxreq"
+	expect * =	End
+	expect 0 1002	Begin		"^bereq "
+	expect * =	BereqAcct	"^54 4 58 41 1000 1041$"
+	expect 0 =	End
+	expect 0 1003	Begin		"^req .* rxreq"
+	expect * =	End
+	expect 0 1004	Begin		"^bereq"
+	expect * =	BereqAcct	"^54 4 58 12 0 12$"
+	expect * =	End
+} -start
+
+# Request 1001
+client c1 {
+	txreq -req "POST" -url "/1" -body "asdf"
+	rxresp
+	expect resp.status == 200
+
+	txreq -req "POST" -url "/2" -body "asdf"
+	rxresp
+	expect resp.status == 503
+} -run
+
+logexpect l1 -wait
diff --git a/include/Makefile.am b/include/Makefile.am
index e996fc0..8291a50 100644
--- a/include/Makefile.am
+++ b/include/Makefile.am
@@ -3,6 +3,7 @@
 # API headers
 nobase_pkginclude_HEADERS = \
 	tbl/acct_fields_req.h \
+	tbl/acct_fields_bereq.h \
 	tbl/backend_poll.h \
 	tbl/ban_vars.h \
 	tbl/bo_flags.h \
diff --git a/include/tbl/acct_fields_bereq.h b/include/tbl/acct_fields_bereq.h
new file mode 100644
index 0000000..6454bad
--- /dev/null
+++ b/include/tbl/acct_fields_bereq.h
@@ -0,0 +1,35 @@
+/*-
+ * Copyright (c) 2008 Verdens Gang AS
+ * Copyright (c) 2008-2014 Varnish Software AS
+ * All rights reserved.
+ *
+ * Author: Poul-Henning Kamp <phk at phk.freebsd.dk>
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED.  IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ *
+ * These are the stats we keep track of per busyobj.
+ */
+
+ACCT(bereq_hdrbytes)
+ACCT(bereq_bodybytes)
+ACCT(beresp_hdrbytes)
+ACCT(beresp_bodybytes)
diff --git a/include/tbl/vsl_tags.h b/include/tbl/vsl_tags.h
index 99894cc..af15d07 100644
--- a/include/tbl/vsl_tags.h
+++ b/include/tbl/vsl_tags.h
@@ -512,4 +512,18 @@ SLTM(PipeAcct, 0, "Pipe byte counts",
 	"\n"
 )
 
+SLTM(BereqAcct, 0, "Backend request accounting",
+	"Contains byte counters from backend request processing.\n\n"
+	"The format is::\n\n"
+	"\t%d %d %d %d %d %d\n"
+	"\t|  |  |  |  |  |\n"
+	"\t|  |  |  |  |  +- Total bytes received\n"
+	"\t|  |  |  |  +---- Body bytes received\n"
+	"\t|  |  |  +------- Header bytes received\n"
+	"\t|  |  +---------- Total bytes transmitted\n"
+	"\t|  +------------- Body bytes transmitted\n"
+	"\t+---------------- Header bytes transmitted\n"
+	"\n"
+)
+
 #undef NODEF_NOTICE



From martin at varnish-software.com  Tue Apr  1 13:09:55 2014
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Tue, 01 Apr 2014 15:09:55 +0200
Subject: [4.0] 7de67d2 Backend fetch counters
Message-ID: <E1WUyRv-0003M5-L3@project.varnish-software.com>


commit 7de67d2d327c8e255e1900f62ee784b7dfdc4e66
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Sun Mar 30 16:00:03 2014 +0200

    Backend fetch counters
    
    Add per-backend VSC counters for traffic going in/out of this backend.

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index bf28983..c8d8d89 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -806,8 +806,8 @@ void VBE_DiscardHealth(const struct director *vdi);
 
 struct vbc *VDI_GetFd(struct busyobj *);
 int VDI_Healthy(const struct director *);
-void VDI_CloseFd(struct vbc **vbp);
-void VDI_RecycleFd(struct vbc **vbp);
+void VDI_CloseFd(struct vbc **vbp, const struct acct_bereq *);
+void VDI_RecycleFd(struct vbc **vbp, const struct acct_bereq *);
 void VDI_AddHostHeader(struct http *to, const struct vbc *vbc);
 void VBE_Poll(void);
 void VDI_Init(void);
diff --git a/bin/varnishd/cache/cache_backend.c b/bin/varnishd/cache/cache_backend.c
index 3725e89..6622f4b 100644
--- a/bin/varnishd/cache/cache_backend.c
+++ b/bin/varnishd/cache/cache_backend.c
@@ -288,7 +288,7 @@ vbe_GetVbe(struct busyobj *bo, struct vdi_simple *vs)
 		VSL_Flush(bo->vsl, 0);
 
 		VTCP_close(&vc->fd);
-		VBE_DropRefConn(bp);
+		VBE_DropRefConn(bp, NULL);
 		vc->backend = NULL;
 		VBE_ReleaseConn(vc);
 	}
diff --git a/bin/varnishd/cache/cache_backend.h b/bin/varnishd/cache/cache_backend.h
index c339563..c9ce112 100644
--- a/bin/varnishd/cache/cache_backend.h
+++ b/bin/varnishd/cache/cache_backend.h
@@ -152,9 +152,9 @@ struct vbc {
 void VBE_ReleaseConn(struct vbc *vc);
 
 /* cache_backend_cfg.c */
-void VBE_DropRefConn(struct backend *);
+void VBE_DropRefConn(struct backend *, const struct acct_bereq *);
 void VBE_DropRefVcl(struct backend *);
-void VBE_DropRefLocked(struct backend *b);
+void VBE_DropRefLocked(struct backend *b, const struct acct_bereq *);
 unsigned VBE_Healthy(const struct backend *b, double *changed);
 
 /* cache_backend_poll.c */
diff --git a/bin/varnishd/cache/cache_backend_cfg.c b/bin/varnishd/cache/cache_backend_cfg.c
index 5a43c2e..9adb616 100644
--- a/bin/varnishd/cache/cache_backend_cfg.c
+++ b/bin/varnishd/cache/cache_backend_cfg.c
@@ -97,7 +97,7 @@ VBE_Poll(void)
  */
 
 void
-VBE_DropRefLocked(struct backend *b)
+VBE_DropRefLocked(struct backend *b, const struct acct_bereq *acct)
 {
 	int i;
 	struct vbc *vbe, *vbe2;
@@ -105,6 +105,13 @@ VBE_DropRefLocked(struct backend *b)
 	CHECK_OBJ_NOTNULL(b, BACKEND_MAGIC);
 	assert(b->refcount > 0);
 
+	if (acct != NULL) {
+#define ACCT(foo) \
+		b->vsc->foo += acct->foo;
+#include "tbl/acct_fields_bereq.h"
+#undef ACCT
+	}
+
 	i = --b->refcount;
 	Lck_Unlock(&b->mtx);
 	if (i > 0)
@@ -131,11 +138,11 @@ VBE_DropRefVcl(struct backend *b)
 
 	Lck_Lock(&b->mtx);
 	b->vsc->vcls--;
-	VBE_DropRefLocked(b);
+	VBE_DropRefLocked(b, NULL);
 }
 
 void
-VBE_DropRefConn(struct backend *b)
+VBE_DropRefConn(struct backend *b, const struct acct_bereq *acct)
 {
 
 	CHECK_OBJ_NOTNULL(b, BACKEND_MAGIC);
@@ -143,7 +150,7 @@ VBE_DropRefConn(struct backend *b)
 	Lck_Lock(&b->mtx);
 	assert(b->n_conn > 0);
 	b->n_conn--;
-	VBE_DropRefLocked(b);
+	VBE_DropRefLocked(b, acct);
 }
 
 /*--------------------------------------------------------------------
diff --git a/bin/varnishd/cache/cache_dir.c b/bin/varnishd/cache/cache_dir.c
index 94ef47d..7491d44 100644
--- a/bin/varnishd/cache/cache_dir.c
+++ b/bin/varnishd/cache/cache_dir.c
@@ -40,7 +40,7 @@
 /* Close a connection ------------------------------------------------*/
 
 void
-VDI_CloseFd(struct vbc **vbp)
+VDI_CloseFd(struct vbc **vbp, const struct acct_bereq *acct)
 {
 	struct backend *bp;
 	struct vbc *vc;
@@ -64,7 +64,7 @@ VDI_CloseFd(struct vbc **vbp)
 	vc->vsl = NULL;
 
 	VTCP_close(&vc->fd);
-	VBE_DropRefConn(bp);
+	VBE_DropRefConn(bp, acct);
 	vc->backend = NULL;
 	VBE_ReleaseConn(vc);
 }
@@ -72,7 +72,7 @@ VDI_CloseFd(struct vbc **vbp)
 /* Recycle a connection ----------------------------------------------*/
 
 void
-VDI_RecycleFd(struct vbc **vbp)
+VDI_RecycleFd(struct vbc **vbp, const struct acct_bereq *acct)
 {
 	struct backend *bp;
 	struct vbc *vc;
@@ -95,7 +95,7 @@ VDI_RecycleFd(struct vbc **vbp)
 	Lck_Lock(&bp->mtx);
 	VSC_C_main->backend_recycle++;
 	VTAILQ_INSERT_HEAD(&bp->connlist, vc, list);
-	VBE_DropRefLocked(bp);
+	VBE_DropRefLocked(bp, acct);
 }
 
 /* Get a connection --------------------------------------------------*/
diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index cd627cd..775d66c 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -319,7 +319,7 @@ vbf_stp_startfetch(struct worker *wrk, struct busyobj *bo)
 
 	if (bo->htc.body_status == BS_ERROR) {
 		AN (bo->vbc);
-		VDI_CloseFd(&bo->vbc);
+		VDI_CloseFd(&bo->vbc, &bo->acct);
 		VSLb(bo->vsl, SLT_VCL_Error, "Body cannot be fetched");
 		return (F_STP_ERROR);
 	}
@@ -351,7 +351,7 @@ vbf_stp_startfetch(struct worker *wrk, struct busyobj *bo)
 
 	if (wrk->handling == VCL_RET_RETRY) {
 		AN (bo->vbc);
-		VDI_CloseFd(&bo->vbc);
+		VDI_CloseFd(&bo->vbc, &bo->acct);
 		bo->retries++;
 		if (bo->retries <= cache_param->max_retries)
 			return (F_STP_RETRY);
@@ -471,7 +471,7 @@ vbf_stp_fetch(struct worker *wrk, struct busyobj *bo)
 
 	if (vbf_beresp2obj(bo)) {
 		(void)VFP_Error(bo, "Could not get storage");
-		VDI_CloseFd(&bo->vbc);
+		VDI_CloseFd(&bo->vbc, &bo->acct);
 		return (F_STP_ERROR);
 	}
 
@@ -761,9 +761,9 @@ vbf_fetch_thread(struct worker *wrk, void *priv)
 
 	if (bo->vbc != NULL) {
 		if (bo->should_close)
-			VDI_CloseFd(&bo->vbc);
+			VDI_CloseFd(&bo->vbc, &bo->acct);
 		else
-			VDI_RecycleFd(&bo->vbc);
+			VDI_RecycleFd(&bo->vbc, &bo->acct);
 		AZ(bo->vbc);
 	}
 
diff --git a/bin/varnishd/cache/cache_http1_fetch.c b/bin/varnishd/cache/cache_http1_fetch.c
index b6fac38..4ddacb1 100644
--- a/bin/varnishd/cache/cache_http1_fetch.c
+++ b/bin/varnishd/cache/cache_http1_fetch.c
@@ -350,7 +350,7 @@ V1F_fetch_hdr(struct worker *wrk, struct busyobj *bo, struct req *req)
 		VSLb(bo->vsl, SLT_FetchError, "backend write error: %d (%s)",
 		    errno, strerror(errno));
 		VSLb_ts_busyobj(bo, "Bereq", W_TIM_real(wrk));
-		VDI_CloseFd(&bo->vbc);
+		VDI_CloseFd(&bo->vbc, &bo->acct);
 		/* XXX: other cleanup ? */
 		return (retry);
 	}
@@ -376,7 +376,7 @@ V1F_fetch_hdr(struct worker *wrk, struct busyobj *bo, struct req *req)
 			VSLb(bo->vsl, SLT_FetchError,
 			    "http %sread error: overflow",
 			    first ? "first " : "");
-			VDI_CloseFd(&bo->vbc);
+			VDI_CloseFd(&bo->vbc, &bo->acct);
 			/* XXX: other cleanup ? */
 			return (-1);
 		}
@@ -384,7 +384,7 @@ V1F_fetch_hdr(struct worker *wrk, struct busyobj *bo, struct req *req)
 			bo->acct.beresp_hdrbytes += Tlen(htc->rxbuf);
 			VSLb(bo->vsl, SLT_FetchError, "http %sread error: EOF",
 			    first ? "first " : "");
-			VDI_CloseFd(&bo->vbc);
+			VDI_CloseFd(&bo->vbc, &bo->acct);
 			/* XXX: other cleanup ? */
 			return (retry);
 		}
@@ -401,7 +401,7 @@ V1F_fetch_hdr(struct worker *wrk, struct busyobj *bo, struct req *req)
 
 	if (HTTP1_DissectResponse(hp, htc)) {
 		VSLb(bo->vsl, SLT_FetchError, "http format error");
-		VDI_CloseFd(&bo->vbc);
+		VDI_CloseFd(&bo->vbc, &bo->acct);
 		/* XXX: other cleanup ? */
 		return (-1);
 	}
diff --git a/bin/varnishd/cache/cache_pipe.c b/bin/varnishd/cache/cache_pipe.c
index 05b8d0a..02d6c21 100644
--- a/bin/varnishd/cache/cache_pipe.c
+++ b/bin/varnishd/cache/cache_pipe.c
@@ -70,7 +70,7 @@ rdf(int fd0, int fd1, ssize_t *pcnt)
 }
 
 static void
-pipecharge(struct req *req, const struct acct_pipe *a)
+pipecharge(struct req *req, const struct acct_pipe *a, struct VSC_C_vbe *b)
 {
 
 	VSLb(req->vsl, SLT_PipeAcct, "%ju %ju %ju %ju",
@@ -83,6 +83,11 @@ pipecharge(struct req *req, const struct acct_pipe *a)
 	VSC_C_main->s_pipe_hdrbytes += a->req;
 	VSC_C_main->s_pipe_in += a->in;
 	VSC_C_main->s_pipe_out += a->out;
+	if (b != NULL) {
+		b->pipe_hdrbytes += a->bereq;
+		b->pipe_out += a->in;
+		b->pipe_in += a->out;
+	}
 	Lck_Unlock(&pipestat_mtx);
 }
 
@@ -110,7 +115,7 @@ PipeRequest(struct req *req, struct busyobj *bo)
 
 	vc = VDI_GetFd(bo);
 	if (vc == NULL) {
-		pipecharge(req, &acct);
+		pipecharge(req, &acct, NULL);
 		SES_Close(req->sp, SC_OVERLOAD);
 		return;
 	}
@@ -133,9 +138,9 @@ PipeRequest(struct req *req, struct busyobj *bo)
 	VSLb_ts_req(req, "Pipe", W_TIM_real(wrk));
 
 	if (i) {
-		pipecharge(req, &acct);
+		pipecharge(req, &acct, vc->backend->vsc);
 		SES_Close(req->sp, SC_TX_PIPE);
-		VDI_CloseFd(&vc);
+		VDI_CloseFd(&vc, NULL);
 		return;
 	}
 
@@ -173,9 +178,9 @@ PipeRequest(struct req *req, struct busyobj *bo)
 		}
 	}
 	VSLb_ts_req(req, "PipeSess", W_TIM_real(wrk));
-	pipecharge(req, &acct);
+	pipecharge(req, &acct, vc->backend->vsc);
 	SES_Close(req->sp, SC_TX_PIPE);
-	VDI_CloseFd(&vc);
+	VDI_CloseFd(&vc, NULL);
 	bo->vbc = NULL;
 }
 
diff --git a/include/tbl/vsc_fields.h b/include/tbl/vsc_fields.h
index c05d428..64e63a3 100644
--- a/include/tbl/vsc_fields.h
+++ b/include/tbl/vsc_fields.h
@@ -184,6 +184,36 @@ VSC_F(happy,			uint64_t, 0, 'b', info,
     "Happy health probes",
 	""
 )
+VSC_F(bereq_hdrbytes,		uint64_t, 0, 'a', info,
+    "Request header bytes",
+	"Total backend request header bytes sent"
+)
+VSC_F(bereq_bodybytes,		uint64_t, 0, 'a', info,
+    "Request body bytes",
+	"Total backend request body bytes sent"
+)
+VSC_F(beresp_hdrbytes,		uint64_t, 0, 'a', info,
+    "Response header bytes",
+	"Total backend response header bytes received"
+)
+VSC_F(beresp_bodybytes,		uint64_t, 0, 'a', info,
+    "Response body bytes",
+	"Total backend response body bytes received"
+)
+VSC_F(pipe_hdrbytes,		uint64_t, 0, 'a', info,
+    "Pipe request header bytes",
+	"Total request bytes sent for piped sessions"
+)
+VSC_F(pipe_out,			uint64_t, 0, 'a', info,
+    "Piped bytes to backend",
+	"Total number of bytes forwarded to backend in"
+	" pipe sessions"
+)
+VSC_F(pipe_in,			uint64_t, 0, 'a', info,
+    "Piped bytes from backend",
+	"Total number of bytes forwarded from backend in"
+	" pipe sessions"
+)
 
 #endif
 



From martin at varnish-software.com  Tue Apr  1 13:09:55 2014
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Tue, 01 Apr 2014 15:09:55 +0200
Subject: [4.0] 621ac54 Use SLT_ReqAcct for length in varnishncsa
Message-ID: <E1WUyRv-0003MF-MA@project.varnish-software.com>


commit 621ac54f87220c8c7413f13fe81ada8f71ce2c25
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Sun Mar 30 20:57:49 2014 +0200

    Use SLT_ReqAcct for length in varnishncsa

diff --git a/bin/varnishncsa/varnishncsa.c b/bin/varnishncsa/varnishncsa.c
index a38264b..7a47b1b 100644
--- a/bin/varnishncsa/varnishncsa.c
+++ b/bin/varnishncsa/varnishncsa.c
@@ -774,8 +774,8 @@ dispatch_f(struct VSL_data *vsl, struct VSL_transaction * const pt[],
 			case SLT_RespStatus:
 				frag_line(b, e, &CTX.frag[F_s]);
 				break;
-			case SLT_Length:
-				frag_line(b, e, &CTX.frag[F_b]);
+			case SLT_ReqAcct:
+				frag_fields(b, e, 5, &CTX.frag[F_b], 0, NULL);
 				break;
 			case SLT_Timestamp:
 				if (isprefix(b, "Start:", e, &p)) {



From martin at varnish-software.com  Tue Apr  1 13:09:55 2014
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Tue, 01 Apr 2014 15:09:55 +0200
Subject: [4.0] 2e75328 Use SLT_ReqAcct for length in varnishhist
Message-ID: <E1WUyRv-0003MO-N4@project.varnish-software.com>


commit 2e7532887c6304608d671095b3acc474751a7ab3
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Sun Mar 30 20:59:26 2014 +0200

    Use SLT_ReqAcct for length in varnishhist

diff --git a/bin/varnishhist/varnishhist.c b/bin/varnishhist/varnishhist.c
index 91078d8..e8eb1ff 100644
--- a/bin/varnishhist/varnishhist.c
+++ b/bin/varnishhist/varnishhist.c
@@ -120,9 +120,9 @@ profiles[] = {
 		.hist_high = 3
 	}, {
 		.name = "size",
-		.tag = SLT_Length,
+		.tag = SLT_ReqAcct,
 		.prefix = NULL,
-		.field = 1,
+		.field = 5,
 		.hist_low = 1,
 		.hist_high = 8
 	}, {



From martin at varnish-software.com  Tue Apr  1 13:09:55 2014
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Tue, 01 Apr 2014 15:09:55 +0200
Subject: [4.0] d6d97bb Disable streaming on byte counter test cases
Message-ID: <E1WUyRv-0003N7-T6@project.varnish-software.com>


commit d6d97bb34290da61ba350d44309adedc07c06824
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Mon Mar 31 12:00:47 2014 +0200

    Disable streaming on byte counter test cases
    
    This makes sure no extra 'transfer-encoding' headers should pop up
    changing the byte counts.

diff --git a/bin/varnishtest/tests/l00002.vtc b/bin/varnishtest/tests/l00002.vtc
index 0106324..8c7bc35 100644
--- a/bin/varnishtest/tests/l00002.vtc
+++ b/bin/varnishtest/tests/l00002.vtc
@@ -11,6 +11,9 @@ server s1 {
 } -start
 
 varnish v1 -vcl+backend {
+	sub vcl_backend_response {
+		set beresp.do_stream = false;
+	}
 	sub vcl_deliver {
 		unset resp.http.date;
 		unset resp.http.age;
diff --git a/bin/varnishtest/tests/l00005.vtc b/bin/varnishtest/tests/l00005.vtc
index 51d0d7e..dbbea1b 100644
--- a/bin/varnishtest/tests/l00005.vtc
+++ b/bin/varnishtest/tests/l00005.vtc
@@ -16,6 +16,9 @@ varnish v1 -vcl+backend {
 		unset bereq.http.x-varnish;
 		set bereq.http.Host = "foo.bar";
 	}
+	sub vcl_backend_response {
+		set beresp.do_stream = false;
+	}
 } -start
 
 # Request (1002):



From martin at varnish-software.com  Tue Apr  1 13:09:55 2014
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Tue, 01 Apr 2014 15:09:55 +0200
Subject: [4.0] 6d468fe Don't use acct as variable name
Message-ID: <E1WUyRv-0003NG-UQ@project.varnish-software.com>


commit 6d468fe20062add9164d6070652385a5d4ac0557
Author: Martin Blix Grydeland <martin at friday>
Date:   Mon Mar 31 11:41:01 2014 +0200

    Don't use acct as variable name
    
    This triggers 'shadows global' build errors.

diff --git a/bin/varnishd/cache/cache_pipe.c b/bin/varnishd/cache/cache_pipe.c
index 02d6c21..2e83655 100644
--- a/bin/varnishd/cache/cache_pipe.c
+++ b/bin/varnishd/cache/cache_pipe.c
@@ -98,7 +98,7 @@ PipeRequest(struct req *req, struct busyobj *bo)
 	struct worker *wrk;
 	struct pollfd fds[2];
 	int i;
-	struct acct_pipe acct;
+	struct acct_pipe acct_pipe;
 	ssize_t hdrbytes;
 
 	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
@@ -109,13 +109,13 @@ PipeRequest(struct req *req, struct busyobj *bo)
 
 	req->res_mode = RES_PIPE;
 
-	memset(&acct, 0, sizeof acct);
-	acct.req = req->acct.req_hdrbytes;
+	memset(&acct_pipe, 0, sizeof acct_pipe);
+	acct_pipe.req = req->acct.req_hdrbytes;
 	req->acct.req_hdrbytes = 0;
 
 	vc = VDI_GetFd(bo);
 	if (vc == NULL) {
-		pipecharge(req, &acct, NULL);
+		pipecharge(req, &acct_pipe, NULL);
 		SES_Close(req->sp, SC_OVERLOAD);
 		return;
 	}
@@ -129,16 +129,16 @@ PipeRequest(struct req *req, struct busyobj *bo)
 		(void)WRW_Write(wrk, req->htc->pipeline.b,
 		    Tlen(req->htc->pipeline));
 
-	i = WRW_FlushRelease(wrk, &acct.bereq);
-	if (acct.bereq > hdrbytes) {
-		acct.in = acct.bereq - hdrbytes;
-		acct.bereq = hdrbytes;
+	i = WRW_FlushRelease(wrk, &acct_pipe.bereq);
+	if (acct_pipe.bereq > hdrbytes) {
+		acct_pipe.in = acct_pipe.bereq - hdrbytes;
+		acct_pipe.bereq = hdrbytes;
 	}
 
 	VSLb_ts_req(req, "Pipe", W_TIM_real(wrk));
 
 	if (i) {
-		pipecharge(req, &acct, vc->backend->vsc);
+		pipecharge(req, &acct_pipe, vc->backend->vsc);
 		SES_Close(req->sp, SC_TX_PIPE);
 		VDI_CloseFd(&vc, NULL);
 		return;
@@ -160,7 +160,8 @@ PipeRequest(struct req *req, struct busyobj *bo)
 		i = poll(fds, 2, (int)(cache_param->pipe_timeout * 1e3));
 		if (i < 1)
 			break;
-		if (fds[0].revents && rdf(vc->fd, req->sp->fd, &acct.out)) {
+		if (fds[0].revents &&
+		    rdf(vc->fd, req->sp->fd, &acct_pipe.out)) {
 			if (fds[1].fd == -1)
 				break;
 			(void)shutdown(vc->fd, SHUT_RD);
@@ -168,7 +169,8 @@ PipeRequest(struct req *req, struct busyobj *bo)
 			fds[0].events = 0;
 			fds[0].fd = -1;
 		}
-		if (fds[1].revents && rdf(req->sp->fd, vc->fd, &acct.in)) {
+		if (fds[1].revents &&
+		    rdf(req->sp->fd, vc->fd, &acct_pipe.in)) {
 			if (fds[0].fd == -1)
 				break;
 			(void)shutdown(req->sp->fd, SHUT_RD);
@@ -178,7 +180,7 @@ PipeRequest(struct req *req, struct busyobj *bo)
 		}
 	}
 	VSLb_ts_req(req, "PipeSess", W_TIM_real(wrk));
-	pipecharge(req, &acct, vc->backend->vsc);
+	pipecharge(req, &acct_pipe, vc->backend->vsc);
 	SES_Close(req->sp, SC_TX_PIPE);
 	VDI_CloseFd(&vc, NULL);
 	bo->vbc = NULL;



From phk at FreeBSD.org  Tue Apr  1 13:09:55 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 01 Apr 2014 15:09:55 +0200
Subject: [4.0] e471a84 Silence Flexelint
Message-ID: <E1WUyRv-0003NP-VP@project.varnish-software.com>


commit e471a8469d02d53a8e7679b11961f179cfb70196
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Mar 31 10:39:58 2014 +0000

    Silence Flexelint

diff --git a/include/tbl/acct_fields_bereq.h b/include/tbl/acct_fields_bereq.h
index 6454bad..7b28879 100644
--- a/include/tbl/acct_fields_bereq.h
+++ b/include/tbl/acct_fields_bereq.h
@@ -29,7 +29,9 @@
  * These are the stats we keep track of per busyobj.
  */
 
+/*lint -save -e525 -e539 */
 ACCT(bereq_hdrbytes)
 ACCT(bereq_bodybytes)
 ACCT(beresp_hdrbytes)
 ACCT(beresp_bodybytes)
+/*lint -restore */



From apj at mutt.dk  Tue Apr  1 13:09:56 2014
From: apj at mutt.dk (Andreas Plesner)
Date: Tue, 01 Apr 2014 15:09:56 +0200
Subject: [4.0] d861c18 This file was named wrong,
 and the test does not make sense anymore,
 since the backend health information is in the fetch thread
Message-ID: <E1WUyRw-0003NY-08@project.varnish-software.com>


commit d861c18fc97d74068e58337be86b306e0920b075
Author: Andreas Plesner <apj at mutt.dk>
Date:   Mon Mar 31 12:39:39 2014 +0200

    This file was named wrong, and the test does not make sense anymore, since the backend health information is in the fetch thread

diff --git a/bin/varnishtest/tests/r01228.vcl b/bin/varnishtest/tests/r01228.vcl
deleted file mode 100644
index 1cb7e01..0000000
--- a/bin/varnishtest/tests/r01228.vcl
+++ /dev/null
@@ -1,21 +0,0 @@
-varnishtest "req.backend.healthy in vcl_deliver"
-
-server s1 {
-	rxreq
-	txresp
-} -start
-
-varnish v1 -vcl+backend {
-
-	import ${vmod_std};
-
-	sub vcl_deliver {
-	    set resp.http.x-foo = std.healthy(req.backend);
-	}
-} -start
-
-client c1 {
-	txreq
-	rxresp
-	expect resp.status == 200
-} -run



From lkarsten at varnish-software.com  Tue Apr  1 13:09:56 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:56 +0200
Subject: [4.0] 54001d6 Bump release number
Message-ID: <E1WUyRw-0003Nh-13@project.varnish-software.com>


commit 54001d6cd4e62b6c58d24e9e677bcc2ff053b89d
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu Mar 13 14:44:53 2014 +0100

    Bump release number

diff --git a/configure.ac b/configure.ac
index 6612215..59a3690 100644
--- a/configure.ac
+++ b/configure.ac
@@ -2,7 +2,7 @@ AC_PREREQ(2.59)
 AC_COPYRIGHT([Copyright (c) 2006 Verdens Gang AS
 Copyright (c) 2006-2014 Varnish Software AS])
 AC_REVISION([$Id$])
-AC_INIT([Varnish], [4.0.0-tp2], [varnish-dev at varnish-cache.org])
+AC_INIT([Varnish], [4.0.0-tp3], [varnish-dev at varnish-cache.org])
 AC_CONFIG_SRCDIR(include/miniobj.h)
 AC_CONFIG_HEADERS([config.h])
 AC_CONFIG_MACRO_DIR([m4])



From lkarsten at varnish-software.com  Tue Apr  1 13:09:56 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:56 +0200
Subject: [4.0] bc17365 Revert "Bump release number"
Message-ID: <E1WUyRw-0003Nq-1x@project.varnish-software.com>


commit bc17365dbdec0c6d49b2ad99f17d51d2d94c828d
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Mon Mar 31 13:11:09 2014 +0200

    Revert "Bump release number"
    
    This reverts commit 54001d6cd4e62b6c58d24e9e677bcc2ff053b89d.

diff --git a/configure.ac b/configure.ac
index 59a3690..6612215 100644
--- a/configure.ac
+++ b/configure.ac
@@ -2,7 +2,7 @@ AC_PREREQ(2.59)
 AC_COPYRIGHT([Copyright (c) 2006 Verdens Gang AS
 Copyright (c) 2006-2014 Varnish Software AS])
 AC_REVISION([$Id$])
-AC_INIT([Varnish], [4.0.0-tp3], [varnish-dev at varnish-cache.org])
+AC_INIT([Varnish], [4.0.0-tp2], [varnish-dev at varnish-cache.org])
 AC_CONFIG_SRCDIR(include/miniobj.h)
 AC_CONFIG_HEADERS([config.h])
 AC_CONFIG_MACRO_DIR([m4])



From lkarsten at varnish-software.com  Tue Apr  1 13:09:56 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:56 +0200
Subject: [4.0] d1e3ff9 Prepare for 4.0.0 beta1 release.
Message-ID: <E1WUyRw-0003Nz-3L@project.varnish-software.com>


commit d1e3ff9ab604a22bdc0aec23ebefcae7d527a8a6
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu Mar 27 12:10:59 2014 +0100

    Prepare for 4.0.0 beta1 release.

diff --git a/configure.ac b/configure.ac
index 6612215..18e4ed0 100644
--- a/configure.ac
+++ b/configure.ac
@@ -2,7 +2,7 @@ AC_PREREQ(2.59)
 AC_COPYRIGHT([Copyright (c) 2006 Verdens Gang AS
 Copyright (c) 2006-2014 Varnish Software AS])
 AC_REVISION([$Id$])
-AC_INIT([Varnish], [4.0.0-tp2], [varnish-dev at varnish-cache.org])
+AC_INIT([Varnish], [4.0.0-beta1], [varnish-dev at varnish-cache.org])
 AC_CONFIG_SRCDIR(include/miniobj.h)
 AC_CONFIG_HEADERS([config.h])
 AC_CONFIG_MACRO_DIR([m4])
diff --git a/doc/changes.rst b/doc/changes.rst
index 25d655a..28d64da 100644
--- a/doc/changes.rst
+++ b/doc/changes.rst
@@ -1,3 +1,63 @@
+
+=====================================
+Changes from 4.0.0 TP2 to 4.0.0 beta1
+=====================================
+
+New since TP2:
+
+- Previous always-appended code called default.vcl is now called builtin.vcl.
+  The new example.vcl is recommended as a starting point for new users.
+- vcl_error is now called vcl_synth, and does not any more mandate closing the
+  client connection.
+- New VCL function vcl_backend_error, where you can change the 503 prepared if
+  all your backends are failing. This can then be cached as a regular object.
+- Keyword "remove" in VCL is replaced by "unset".
+- new timestamp and accounting records in varnishlog.
+- std.timestamp() is introduced.
+- stored objects are now read only, meaning obj.hits now counts per objecthead
+  instead. obj.lastuse saw little use and has been removed.
+- builtin VCL now does return(pipe) for chunked POST and PUT requests.
+- python-docutils and rst2man are now build requirements.
+- cli_timeout is now 60 seconds to avoid slaughtering the child process in
+  times of high IO load/scheduling latency.
+- return(purge) from vcl_recv is now valid.
+- return(hash) is now the default return action from vcl_recv.
+- req.backend is now req.backend_hint. beresp.storage is beresp.storage_hint.
+
+
+Bugs fixed
+----------
+
+* 1460_ - tools now use the new timestamp format.
+* 1450_ - varnishstat -l segmentation fault.
+* 1320_ - Work around Content-Length: 0 and Content-Encoding: gzip gracefully.
+* 1458_ - Panic on busy object.
+* 1417_ - Handle return(abandon) in vcl_backend_response.
+* 1455_ - vcl_pipe now sets Connection: close by default on backend requests.
+* 1454_ - X-Forwarded-For is now done in C, before vcl_recv is run.
+* 1436_ - Better explanation when missing an import in VCL.
+* 1440_ - Serve ESI-includes from a different backend.
+* 1441_ - Incorrect grouping when logging ESI subrequests.
+* 1434_ - std.duration can now do ms/milliseconds.
+* 1419_ - Don't put objcores on the ban list until they go non-BUSY.
+* 1405_ - Ban lurker does not always evict all objects.
+
+.. _1460: https://www.varnish-cache.org/trac/ticket/1460
+.. _1450: https://www.varnish-cache.org/trac/ticket/1450
+.. _1320: https://www.varnish-cache.org/trac/ticket/1320
+.. _1458: https://www.varnish-cache.org/trac/ticket/1458
+.. _1417: https://www.varnish-cache.org/trac/ticket/1417
+.. _1455: https://www.varnish-cache.org/trac/ticket/1455
+.. _1454: https://www.varnish-cache.org/trac/ticket/1454
+.. _1436: https://www.varnish-cache.org/trac/ticket/1436
+.. _1440: https://www.varnish-cache.org/trac/ticket/1440
+.. _1441: https://www.varnish-cache.org/trac/ticket/1441
+.. _1434: https://www.varnish-cache.org/trac/ticket/1434
+.. _1419: https://www.varnish-cache.org/trac/ticket/1419
+.. _1405: https://www.varnish-cache.org/trac/ticket/1405
+
+
+
 ===================================
 Changes from 4.0.0 TP1 to 4.0.0 TP2
 ===================================
diff --git a/redhat/varnish.spec b/redhat/varnish.spec
index 9adcaa2..c1b5f8d 100644
--- a/redhat/varnish.spec
+++ b/redhat/varnish.spec
@@ -1,11 +1,11 @@
-%define v_rc tp2
+%define v_rc beta1
 %define vd_rc %{?v_rc:-%{?v_rc}}
 %define    _use_internal_dependency_generator 0
 %define __find_provides %{_builddir}/varnish-%{version}%{?v_rc:-%{?v_rc}}/redhat/find-provides
 Summary: High-performance HTTP accelerator
 Name: varnish
 Version: 4.0.0
-Release: 0.20131129%{?v_rc}%{?dist}
+Release: 0.20140328%{?v_rc}%{?dist}
 License: BSD
 Group: System Environment/Daemons
 URL: http://www.varnish-cache.org/



From lkarsten at varnish-software.com  Tue Apr  1 13:09:56 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:56 +0200
Subject: [4.0] 24ac2e1 Merge tag 'varnish-4.0.0-beta1' into 4.0
Message-ID: <E1WUyRw-0003OA-4I@project.varnish-software.com>


commit 24ac2e1569c59fd8dbfe4caf3b53465896d5137d
Merge: bc17365 d1e3ff9
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Tue Apr 1 14:58:53 2014 +0200

    Merge tag 'varnish-4.0.0-beta1' into 4.0



From lkarsten at varnish-software.com  Tue Apr  1 13:09:56 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Tue, 01 Apr 2014 15:09:56 +0200
Subject: [4.0] 3453924 Merge branch '4.0' of
 git.varnish-cache.org:varnish-cache into 4.0
Message-ID: <E1WUyRw-0003OJ-5O@project.varnish-software.com>


commit 3453924f357a86ad8173c26b932a099cd7e375e5
Merge: 24ac2e1 0a39f22
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Tue Apr 1 15:09:37 2014 +0200

    Merge branch '4.0' of git.varnish-cache.org:varnish-cache into 4.0
    
    Conflicts:
    	configure.ac
    	doc/sphinx/reference/vcl.rst

diff --cc doc/sphinx/reference/vcl.rst
index 1268920,a4b6f7b..ca1ac48
--- a/doc/sphinx/reference/vcl.rst
+++ b/doc/sphinx/reference/vcl.rst
@@@ -54,14 -67,11 +54,14 @@@ The following operators are available i
  Conditionals
  ------------
  
 -VCL has *if* statements.
 +VCL has *if* and *else* statements. Nested logic can be implemented
- with the *elseif* statement. (*elsif*/*elif*/*else if* is equivalent.)
++with the *elseif* statement. (*elsif*/*elif*/*else if* are equivalent.)
  
 +Note that are no loops or iterators of any kind in VCL.
  
 -Strings, booleans, time, duration and ints
 -------------------------------------------
 +
 +Strings, booleans, time, duration and integers
 +----------------------------------------------
  
  These are the data types in Varnish. You can *set* or *unset* these.
  



From tfheen at fastly.com  Tue Apr  1 13:16:51 2014
From: tfheen at fastly.com (Tollef Fog Heen)
Date: Tue, 1 Apr 2014 15:16:51 +0200
Subject: [master] 3be7659 Require same version of libvarnishapi for now
In-Reply-To: <E1WUxzV-0000OJ-Gw@project.varnish-software.com>
References: <E1WUxzV-0000OJ-Gw@project.varnish-software.com>
Message-ID: <CA+nLBF1zQ91j81e5zAS_qO6hjTf1=8k0L4TJxp1p4e4rwVxMcg@mail.gmail.com>

2014-04-01 14:40 GMT+02:00 Lasse Karstensen <lkarsten at varnish-software.com>:

> -Replaces: libvarnishapi1 (<< 3.0.0-5)

Why are you removing this? It's needed for error-free upgrades from
older versions of Varnish.



From phk at FreeBSD.org  Wed Apr  2 13:54:10 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Wed, 02 Apr 2014 15:54:10 +0200
Subject: [master] 0b31b9f Make sure the thread_pool_stack default is at least
 the minimum imposed by the kernel.
Message-ID: <E1WVLcI-00017V-29@project.varnish-software.com>


commit 0b31b9f091afa34b97d1c18bfc7b185afa1be11f
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Apr 2 13:53:39 2014 +0000

    Make sure the thread_pool_stack default is at least the minimum
    imposed by the kernel.
    
    Spotted by:	ingvar (on PPC64)

diff --git a/bin/varnishd/mgt/mgt_main.c b/bin/varnishd/mgt/mgt_main.c
index 7da3d88..5dc602e 100644
--- a/bin/varnishd/mgt/mgt_main.c
+++ b/bin/varnishd/mgt/mgt_main.c
@@ -340,11 +340,12 @@ make_secret(const char *dirname)
 /*--------------------------------------------------------------------*/
 
 static char stackmin[20];
+static char stackdef[20];
 
 static void
 init_params(struct cli *cli)
 {
-	ssize_t low;
+	ssize_t def, low;
 
 	MCF_CollectParams();
 
@@ -372,6 +373,12 @@ init_params(struct cli *cli)
 	bprintf(stackmin, "%jd", (intmax_t)low);
 	MCF_SetMinimum("thread_pool_stack", stackmin);
 
+	def = 48 * 1024;
+	if (def < low)
+		def = low;
+	bprintf(stackdef, "%jd", (intmax_t)def);
+	MCF_SetDefault("thread_pool_stack", stackdef);
+
 	MCF_InitParams(cli);
 }
 
diff --git a/bin/varnishd/mgt/mgt_pool.c b/bin/varnishd/mgt/mgt_pool.c
index c3fd066..b3294f7 100644
--- a/bin/varnishd/mgt/mgt_pool.c
+++ b/bin/varnishd/mgt/mgt_pool.c
@@ -214,6 +214,6 @@ struct parspec WRK_parspec[] = {
 		"This will likely be rounded up to a multiple of 4k"
 		" (or whatever the page_size might be) by the kernel.",
 		EXPERIMENTAL,
-		"48k", "bytes" },
+		NULL, "bytes" },	// default set in mgt_main.c
 	{ NULL, NULL, NULL }
 };



From lkarsten at varnish-software.com  Wed Apr  2 14:24:52 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Wed, 02 Apr 2014 16:24:52 +0200
Subject: [master] 3d93b5c Fix bad example.
Message-ID: <E1WVM60-0003qk-RP@project.varnish-software.com>


commit 3d93b5c4b3abf5856473f454b7c43f9f24531bfa
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Wed Apr 2 16:24:07 2014 +0200

    Fix bad example.
    
    Spotted by: Martin Boer

diff --git a/doc/sphinx/whats-new/upgrading.rst b/doc/sphinx/whats-new/upgrading.rst
index 7f3d951..f975179 100644
--- a/doc/sphinx/whats-new/upgrading.rst
+++ b/doc/sphinx/whats-new/upgrading.rst
@@ -54,7 +54,7 @@ has been removed, and you should use the hash director directly::
     }
 
     sub vcl_recv {
-        set req.backend_hint = h.backend(client.ip);
+        set req.backend_hint = h.backend(client.identity);
     }
 
 error() is now synth()



From martin at varnish-software.com  Mon Apr  7 11:32:05 2014
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Mon, 07 Apr 2014 13:32:05 +0200
Subject: [master] 669197c create a hard dependency on some curses library
Message-ID: <E1WX7mX-0004bt-8m@project.varnish-software.com>


commit 669197cdc246221d4bf599b41396a3860afd662e
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Wed Mar 19 20:10:46 2014 +0100

    create a hard dependency on some curses library

diff --git a/bin/Makefile.am b/bin/Makefile.am
index 1f33d4e..8aa2ef9 100644
--- a/bin/Makefile.am
+++ b/bin/Makefile.am
@@ -3,10 +3,12 @@
 # Disabling building of the tools while api is in flux
 
 #SUBDIRS = varnishreplay
-SUBDIRS = varnishadm varnishd varnishlog varnishncsa varnishtest
-
-if VARNISH_CURSES
-SUBDIRS += varnishhist
-SUBDIRS += varnishstat
-SUBDIRS += varnishtop
-endif
+SUBDIRS = \
+	varnishadm \
+	varnishd \
+	varnishhist \
+	varnishlog \
+	varnishncsa \
+	varnishstat \
+	varnishtest \
+	varnishtop
diff --git a/configure.ac b/configure.ac
index 18e4ed0..a910e59 100644
--- a/configure.ac
+++ b/configure.ac
@@ -86,7 +86,9 @@ LIBS="${save_LIBS}"
 AC_SUBST(DL_LIBS)
 
 AX_WITH_CURSES
-AM_CONDITIONAL(VARNISH_CURSES, test x$ax_cv_curses = xyes)
+if test "x$ax_cv_curses" != xyes; then
+   AC_MSG_ERROR([requires an X/Open-compatible Curses library])
+fi
 
 save_LIBS="${LIBS}"
 LIBS=""
diff --git a/doc/sphinx/Makefile.am b/doc/sphinx/Makefile.am
index f98a044..3e3bc15 100644
--- a/doc/sphinx/Makefile.am
+++ b/doc/sphinx/Makefile.am
@@ -197,9 +197,6 @@ include/varnishlog_synopsis.rst: $(top_builddir)/bin/varnishlog/varnishlog_opt2r
 BUILT_SOURCES += include/varnishlog_options.rst \
 	include/varnishlog_synopsis.rst
 
-# tools which only get built when curses is available
-# XXX add varnishstat here when _opt2rst is ready
-if VARNISH_CURSES
 include/varnishtop_options.rst: $(top_builddir)/bin/varnishtop/varnishtop_opt2rst
 	$(top_builddir)/bin/varnishtop/varnishtop_opt2rst options > $@
 include/varnishtop_synopsis.rst: $(top_builddir)/bin/varnishtop/varnishtop_opt2rst
@@ -213,8 +210,6 @@ include/varnishhist_synopsis.rst: $(top_builddir)/bin/varnishhist/varnishhist_op
 	$(top_builddir)/bin/varnishhist/varnishhist_opt2rst synopsis > $@
 BUILT_SOURCES += include/varnishhist_options.rst \
 	 include/varnishhist_synopsis.rst
-endif
-
 
 include/vmod_std.rst: $(top_builddir)/lib/libvmod_std/vmod_std.rst
 	cp $? $@
diff --git a/man/Makefile.am b/man/Makefile.am
index ae3e40d..c25804e 100644
--- a/man/Makefile.am
+++ b/man/Makefile.am
@@ -14,10 +14,12 @@ dist_man_MANS = \
 	vsl-query.7 \
 	varnishadm.1 \
 	varnishd.1 \
+	varnishhist.1 \
 	varnishlog.1 \
 	varnishncsa.1 \
 	varnishstat.1 \
 	varnishtest.1 \
+	varnishtop.1 \
 	vmod_directors.3 \
 	vmod_std.3
 
@@ -69,12 +71,6 @@ varnishstat.1: $(top_srcdir)/doc/sphinx/reference/varnishstat.rst
 varnishtest.1: $(top_srcdir)/doc/sphinx/reference/varnishtest.rst
 	${RST2MAN} $(top_srcdir)/doc/sphinx/reference/varnishtest.rst $@
 
-# tools which only get built when curses is available
-# XXX add varnishstat here when _opt2rst is ready
-if VARNISH_CURSES
-dist_man_MANS += varnishtop.1 \
-	varnishhist.1
-
 varnishtop.1: \
 	$(top_srcdir)/doc/sphinx/reference/varnishtop.rst \
 	$(top_srcdir)/doc/sphinx/include/varnishtop_options.rst \
@@ -86,8 +82,6 @@ varnishhist.1: \
 	$(top_srcdir)/doc/sphinx/include/varnishhist_options.rst \
 	$(top_srcdir)/doc/sphinx/include/varnishhist_synopsis.rst
 	${RST2MAN} $(top_srcdir)/doc/sphinx/reference/varnishhist.rst $@
-endif
-
 
 vmod_std.3: $(top_srcdir)/lib/libvmod_std/vmod_std.man.rst
 	${RST2MAN} $? $@



From martin at varnish-software.com  Mon Apr  7 11:32:05 2014
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Mon, 07 Apr 2014 13:32:05 +0200
Subject: [master] ed234cf Re-add some XXX Makefile comments
Message-ID: <E1WX7mX-0004bz-9t@project.varnish-software.com>


commit ed234cfd631e5ed88e639bc10a99ab0cc3c459a7
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Mon Apr 7 13:31:01 2014 +0200

    Re-add some XXX Makefile comments

diff --git a/doc/sphinx/Makefile.am b/doc/sphinx/Makefile.am
index 3e3bc15..ab8d021 100644
--- a/doc/sphinx/Makefile.am
+++ b/doc/sphinx/Makefile.am
@@ -183,6 +183,8 @@ include/params.rst: $(top_builddir)/bin/varnishd/varnishd
 	$(top_builddir)/bin/varnishd/varnishd -x dumprstparam > include/params.rst
 BUILT_SOURCES = include/params.rst
 
+# XXX add varnishstat here when it's been _opt2rst'ed
+
 include/varnishncsa_options.rst: $(top_builddir)/bin/varnishncsa/varnishncsa_opt2rst
 	$(top_builddir)/bin/varnishncsa/varnishncsa_opt2rst options > $@
 include/varnishncsa_synopsis.rst: $(top_builddir)/bin/varnishncsa/varnishncsa_opt2rst
diff --git a/man/Makefile.am b/man/Makefile.am
index c25804e..e4a0bfe 100644
--- a/man/Makefile.am
+++ b/man/Makefile.am
@@ -65,6 +65,7 @@ varnishlog.1: \
 varnishreplay.1: $(top_srcdir)/doc/sphinx/reference/varnishreplay.rst
 	${RST2MAN} $(top_srcdir)/doc/sphinx/reference/varnishreplay.rst $@
 
+# XXX add _options.rst and _synopsis.rst here when it's been _opt2rst'ed
 varnishstat.1: $(top_srcdir)/doc/sphinx/reference/varnishstat.rst
 	${RST2MAN} $(top_srcdir)/doc/sphinx/reference/varnishstat.rst $@
 



From martin at varnish-software.com  Mon Apr  7 12:03:13 2014
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Mon, 07 Apr 2014 14:03:13 +0200
Subject: [master] acd718f Only grab the first matched fragment in varnishncsa
Message-ID: <E1WX8Gf-0000PP-Df@project.varnish-software.com>


commit acd718f04215a8ca0cca4cd0d7997aceaf73c5a9
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Mon Apr 7 14:02:35 2014 +0200

    Only grab the first matched fragment in varnishncsa
    
    Fixes: #1462

diff --git a/bin/varnishncsa/varnishncsa.c b/bin/varnishncsa/varnishncsa.c
index 7a47b1b..e3f2432 100644
--- a/bin/varnishncsa/varnishncsa.c
+++ b/bin/varnishncsa/varnishncsa.c
@@ -682,9 +682,12 @@ frag_fields(const char *b, const char *e, ...)
 			q++;
 
 		if (field == n) {
-			frag->gen = CTX.gen;
-			frag->b = p;
-			frag->e = q;
+			if (frag->gen != CTX.gen) {
+				/* We only grab the same matching field once */
+				frag->gen = CTX.gen;
+				frag->b = p;
+				frag->e = q;
+			}
 			field = va_arg(ap, int);
 			if (field == 0)
 				break;
@@ -698,6 +701,10 @@ static void
 frag_line(const char *b, const char *e, struct fragment *f)
 {
 
+	if (f->gen == CTX.gen)
+		/* We only grab the same matching record once */
+		return;
+
 	/* Skip leading space */
 	while (b < e && isspace(*b))
 		++b;



From martin at varnish-software.com  Tue Apr  8 09:03:27 2014
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Tue, 08 Apr 2014 11:03:27 +0200
Subject: [master] a43326d Rearm at zero TTL on failed objects
Message-ID: <E1WXRwF-0000mj-RZ@project.varnish-software.com>


commit a43326dd93b9a37428b9aecf72e785ed7675afef
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Tue Apr 1 13:47:43 2014 +0200

    Rearm at zero TTL on failed objects
    
    Fixes: #1468

diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index 54a042d..19b8fb8 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -707,6 +707,11 @@ vbf_stp_fail(struct worker *wrk, struct busyobj *bo)
 
 	assert(bo->state < BOS_FINISHED);
 	HSH_Fail(bo->fetch_objcore);
+	if (bo->fetch_objcore->exp_flags & OC_EF_EXP) {
+		/* Already unbusied - expire it */
+		AN(bo->fetch_obj);
+		EXP_Rearm(bo->fetch_obj, bo->fetch_obj->exp.t_origin, 0, 0, 0);
+	}
 	wrk->stats.fetch_failed++;
 	VBO_setstate(bo, BOS_FAILED);
 	return (F_STP_DONE);
diff --git a/bin/varnishtest/tests/r01468.vtc b/bin/varnishtest/tests/r01468.vtc
new file mode 100644
index 0000000..119c1da
--- /dev/null
+++ b/bin/varnishtest/tests/r01468.vtc
@@ -0,0 +1,53 @@
+varnishtest "#1468 - freeing failed obj"
+
+server s1 {
+	rxreq
+	expect req.url == "/1"
+	txresp -nolen -hdr "Transfer-Encoding: chunked"
+	chunked {<HTML>}
+	sema r1 sync 2
+	close
+
+	accept
+	rxreq
+	expect req.url == "/2"
+	txresp -nolen -hdr "Transfer-Encoding: chunked"
+	chunked {<HTML>}
+	sema r1 sync 2
+} -start
+
+varnish v1 -vcl+backend {
+	sub vcl_recv {
+		if (req.url == "/2") {
+			return (pass);
+		}
+	}
+	sub vcl_backend_response {
+		set beresp.ttl = 5m;
+	}
+} -start
+
+# Test the regular cached content case
+client c1 {
+	txreq -url "/1"
+	rxresphdrs
+	expect resp.status == 200
+	rxchunk
+	sema r1 sync 2
+	expect_close
+} -run
+
+# Test the pass from vcl_recv case
+client c1 {
+	txreq -url "/2"
+	rxresphdrs
+	expect resp.status == 200
+	rxchunk
+	sema r1 sync 2
+	expect_close
+} -run
+
+# Delay to allow expiry thread to do it's job
+delay 1
+
+varnish v1 -expect n_object == 0



From martin at varnish-software.com  Tue Apr  8 12:59:19 2014
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Tue, 08 Apr 2014 14:59:19 +0200
Subject: [master] 9f3100e Take offset rounding into account when rounding the
 length of segment size.
Message-ID: <E1WXVcV-0008AD-AC@project.varnish-software.com>


commit 9f3100e298e079d75fe4e448d5b1b4e26cb2606e
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Tue Apr 8 14:57:09 2014 +0200

    Take offset rounding into account when rounding the length of segment
    size.
    
    Fixes: #1469

diff --git a/bin/varnishd/storage/storage_persistent_silo.c b/bin/varnishd/storage/storage_persistent_silo.c
index 35af470..f27ccd9 100644
--- a/bin/varnishd/storage/storage_persistent_silo.c
+++ b/bin/varnishd/storage/storage_persistent_silo.c
@@ -219,6 +219,7 @@ smp_new_seg(struct smp_sc *sc)
 	sg->lru->flags |= LRU_F_DONTMOVE;
 
 	sg->p.offset = IRNUP(sc, sg->p.offset);
+	sg->p.length -= sg->p.offset - tmpsg.p.offset;
 	sg->p.length = IRNDN(sc, sg->p.length);
 	assert(sg->p.offset + sg->p.length <= tmpsg.p.offset + tmpsg.p.length);
 	sc->free_offset = sg->p.offset + sg->p.length;



From martin at varnish-software.com  Wed Apr  9 10:30:40 2014
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Wed, 09 Apr 2014 12:30:40 +0200
Subject: [3.0] dec9098 Clear the pass oc flag bits earlier in the ban lurker
Message-ID: <E1WXpmC-00041v-JY@project.varnish-software.com>


commit dec9098a2ac760d7ba4765a9a1a1091103b6eb05
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Wed Apr 9 11:28:43 2014 +0200

    Clear the pass oc flag bits earlier in the ban lurker
    
    The pass bits of the oc's flags wasn't cleared before or'ing the new
    pass on failing the busy object test in the ban lurker. This would
    cause the pass not to be matched later, making the lurker spin on the
    busy object.
    
    Fixes: #1470

diff --git a/bin/varnishd/cache_ban.c b/bin/varnishd/cache_ban.c
index 6062f98..4e4de57 100644
--- a/bin/varnishd/cache_ban.c
+++ b/bin/varnishd/cache_ban.c
@@ -844,6 +844,10 @@ ban_lurker_work(const struct sess *sp, unsigned pass)
 				continue;
 			}
 			/*
+			 * Clear this oc's pass bits.
+			 */
+			oc->flags &= ~OC_F_LURK;
+			/*
 			 * If the object is busy, we can't touch
 			 * it. Defer it to a later run.
 			 */
@@ -861,7 +865,6 @@ ban_lurker_work(const struct sess *sp, unsigned pass)
 			 */
 			AN(oc->refcnt);
 			oc->refcnt++;
-			oc->flags &= ~OC_F_LURK;
 			Lck_Unlock(&ban_mtx);
 			/*
 			 * Get the object and check it against all relevant bans



From lkarsten at varnish-software.com  Wed Apr  9 11:49:40 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Wed, 09 Apr 2014 13:49:40 +0200
Subject: [master] c91c1c3 Expand access to req.backend_hint
Message-ID: <E1WXr0e-0000lK-IL@project.varnish-software.com>


commit c91c1c3ef677b2247d589c5df94e5b4db7f7e25d
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Wed Apr 9 13:47:14 2014 +0200

    Expand access to req.backend_hint
    
    We need req.backend_hint in vcl_hit to enable different
    grace times depending on backend health.

diff --git a/lib/libvcc/generate.py b/lib/libvcc/generate.py
index 06ed058..ace50e6 100755
--- a/lib/libvcc/generate.py
+++ b/lib/libvcc/generate.py
@@ -271,11 +271,9 @@ sp_variables = [
 	),
 	('req.backend_hint',
 		'BACKEND',
-		( 'recv',),
-		( 'recv',), """
+		( 'client', ),
+		( 'client',), """
 		Set bereq.backend to this if we attempt to fetch.
-		This variable is a convenience so the overall
-		policy can be set up once and for all in vcl_recv{}.
 		"""
 	),
 	('req.hash_ignore_busy',



From martin at varnish-software.com  Wed Apr  9 13:23:01 2014
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Wed, 09 Apr 2014 15:23:01 +0200
Subject: [master] a0da22f Make varnishstat use the -w update interval argument
 in curses mode.
Message-ID: <E1WXsSz-0006j8-DR@project.varnish-software.com>


commit a0da22f9a3df031ff345b1a06311fc41a0ed3d79
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Wed Apr 9 15:21:41 2014 +0200

    Make varnishstat use the -w update interval argument in curses mode.

diff --git a/bin/varnishstat/varnishstat_curses.c b/bin/varnishstat/varnishstat_curses.c
index 6e8bae8..356a92b 100644
--- a/bin/varnishstat/varnishstat_curses.c
+++ b/bin/varnishstat/varnishstat_curses.c
@@ -863,7 +863,7 @@ do_curses(struct VSM_data *vd, int delay)
 	struct VSM_fantom f_mgt = VSM_FANTOM_NULL;
 	struct VSM_fantom f_iter = VSM_FANTOM_NULL;
 
-	(void)delay;
+	interval = delay;
 
 	initscr();
 	raw();



From martin at varnish-software.com  Wed Apr  9 13:23:01 2014
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Wed, 09 Apr 2014 15:23:01 +0200
Subject: [master] 3ba1d01 Update the varnishstat documentation to reflect the
 implementation
Message-ID: <E1WXsSz-0006jE-FQ@project.varnish-software.com>


commit 3ba1d01e7782565e110614fdeea0ce3d67781725
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Wed Apr 9 15:22:21 2014 +0200

    Update the varnishstat documentation to reflect the implementation

diff --git a/doc/sphinx/reference/varnishstat.rst b/doc/sphinx/reference/varnishstat.rst
index 4178778..02a72d0 100644
--- a/doc/sphinx/reference/varnishstat.rst
+++ b/doc/sphinx/reference/varnishstat.rst
@@ -20,47 +20,128 @@ DESCRIPTION
 
 The varnishstat utility displays statistics from a running varnishd(1) instance.
 
+OPTIONS
+=======
+
 The following options are available:
 
--1          Instead of presenting of a continuously updated display, print the statistics to stdout.
+-1
+	Instead of presenting of a continuously updated display, print
+	the statistics to stdout.
+
+-f
+	Field inclusion glob. A field glob consists of three parts,
+        type, ident and name, where ident is optional. Each part can
+        contain a '*' character at the end to match a prefix. Use
+        backslash to escape characters. If the argument starts with
+        '^' it is used as an exclusion glob. Multiple -f arguments may
+        be given, and they will be applied in order.
+
+-l
+	Lists the available fields to use with the -f option.
+
+-n
+	Specifies the name of the varnishd instance to get logs from.
+	If -n is not specified, the host name is used.
+
+-N
+	Specify a the filename of a stale VSM instance. When using
+        this option the abandonment checking is disabled.
+
+-V
+	Display the version number and exit.
+
+-w delay
+	Wait delay seconds between updates.  The default is 1. Can
+	also be used with -1, -x or -j for repeated output.
+
+-x
+	Displays the result as XML.
+
+-j	
+	Displays the result as JSON.
+
+CURSES MODE
+===========
+
+When neither -1, -j or -x options are given, the application starts up
+in curses mode. This shows a continously updated view of the counter
+values, along with their description.
+
+The top area shows process uptime information.
 
--f          Field inclusion glob. A field glob consists of three
-            parts, type, ident and name, where ident is optional. Each
-            part can contain a '*' character at the end to match a
-            prefix. Use backslash to escape characters. If the
-            argument starts with '^' it is used as an exclusion
-            glob. Multiple -f arguments may be given, and they will be
-            applied in order.
+The center area shows a list of counter values.
 
--l          Lists the available fields to use with the -f option.
+The bottom area shows the description of the currently selected
+counter.
 
--n          Specifies the name of the varnishd instance to get logs from.  If -n is not specified, the host name
-	    is used.
+Columns
+-------
 
--N          Specify a the filename of a stale VSM instance. When using this
-            option the abandonment checking is disabled.
+The following columns are displayed, from left to right:
 
--V          Display the version number and exit.
+Name
+	The name of the counter
 
--w delay    Wait delay seconds between updates.  The default is 1. Can also be used with -1, -x or -j for repeated output.
+Current
+	The current value of the counter.
 
--x          Displays the result as XML.
+Change
+	The average per second change over the last update interval.
 
--j          Displays the result as JSON.
+Average
+	The average value of this counter over the runtime of the
+	Varnish daemon, or a period if the counter can't be averaged.
 
-The columns in the main display are, from left to right:
+Avg_10
+	The moving average over the last 10 update intervals.
 
-1.   Value
-2.   Per-second average in the period since last update, or a period if the value can not be averaged
-3.   Per-second average over process lifetime, or a period if the value can not be averaged
-4.   Descriptive text
+Avg_100
+	The moving average over the last 100 update intervals.
 
-When using the -1 option, the columns in the output are, from left to right:
+Avg_1000
+	The moving average over the last 1000 update intervals.
 
-1.   Symbolic entry name
-2.   Value
-3.   Per-second average over process lifetime, or a period if the value can not be averaged
-4.   Descriptive text
+Key bindings
+------------
+
+The following keys control the interactive display:
+
+<UP>
+	Navigate the counter list one line up.
+
+<DOWN>
+	Navigate the counter list one line down.
+
+<PAGEUP> or <b>
+	Navigate the counter list one page up.
+
+<PAGEDOWN> or <SPACE>
+	Navigate the counter list one page down.
+
+<g>
+	Go to the top of the counter list.
+
+<G>
+	Go to the bottom of the counter list.
+
+<d>
+	Toggle between showing and hiding unseen counters. Unseen
+	counters are those that has been zero for the entire runtime
+	of varnishstat. Defaults to hide unseen counters.
+
+<v>
+	Cycle through the verbosity levels. Defaults to only showing
+	informational counters.
+
+<q>
+	Quit.
+
+<CTRL+T>
+	Sample now.
+
+XML Output
+==========
 
 When using the -x option, the output is::
 
@@ -70,6 +151,9 @@ When using the -x option, the output is::
     <description>FIELD DESCRIPTION</description>
   </stat>
 
+JSON OUTPUT
+===========
+
 With -j the output format is::
 
   {
@@ -109,8 +193,8 @@ HISTORY
 
 The varnishstat utility was originally developed by Poul-Henning Kamp
 <phk at phk.freebsd.dk> in cooperation with Verdens Gang AS and Varnish
-Software AS. Manual page written by Dag-Erling Sm?rgrav, Per Buer
-and Lasse Karstensen.
+Software AS. Manual page written by Dag-Erling Sm?rgrav, Per Buer,
+Lasse Karstensen and Martin Blix Grydeland.
 
 
 COPYRIGHT



From fgsch at lodoss.net  Wed Apr  9 15:42:55 2014
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Wed, 09 Apr 2014 17:42:55 +0200
Subject: [master] 69e4cfc On pipe, if we don't get a backend log it to shmlog
Message-ID: <E1WXueN-0000bI-D2@project.varnish-software.com>


commit 69e4cfc9ae931aab710e33c0fa89e35ae1e50940
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Wed Apr 9 16:42:16 2014 +0100

    On pipe, if we don't get a backend log it to shmlog

diff --git a/bin/varnishd/cache/cache_pipe.c b/bin/varnishd/cache/cache_pipe.c
index 2e83655..bb666a1 100644
--- a/bin/varnishd/cache/cache_pipe.c
+++ b/bin/varnishd/cache/cache_pipe.c
@@ -115,6 +115,7 @@ PipeRequest(struct req *req, struct busyobj *bo)
 
 	vc = VDI_GetFd(bo);
 	if (vc == NULL) {
+		VSLb(bo->vsl, SLT_FetchError, "no backend connection");
 		pipecharge(req, &acct_pipe, NULL);
 		SES_Close(req->sp, SC_OVERLOAD);
 		return;



From phk at FreeBSD.org  Wed Apr  9 20:54:45 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Wed, 09 Apr 2014 22:54:45 +0200
Subject: [master] 72d8f88 Typo
Message-ID: <E1WXzW9-0002Wv-GY@project.varnish-software.com>


commit 72d8f88ded4b304e0832906094ede8f65b7a2081
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Apr 9 20:54:34 2014 +0000

    Typo
    
    Spotted by:	Ruben

diff --git a/doc/sphinx/phk/ssl.rst b/doc/sphinx/phk/ssl.rst
index bf47970..f700f19 100644
--- a/doc/sphinx/phk/ssl.rst
+++ b/doc/sphinx/phk/ssl.rst
@@ -43,7 +43,7 @@ Varnish do with it ?
 
 We would terminate SSL sessions, and we would burn CPU cycles doing
 that.  You can kiss the highly optimized delivery path in Varnish
-goodby for SSL, we cannot simply tell the kernel to put the bytes
+goodbye for SSL, we cannot simply tell the kernel to put the bytes
 on the socket, rather, we have to corkscrew the data through
 the SSL library and then write it to the socket.
 



From fgsch at lodoss.net  Thu Apr 10 06:48:00 2014
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Thu, 10 Apr 2014 08:48:00 +0200
Subject: [master] 7135d6b Wording
Message-ID: <E1WY8mG-0003Ba-TN@project.varnish-software.com>


commit 7135d6b76addc182ee500b4494eb4dae873f15bc
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Wed Apr 9 17:14:05 2014 +0100

    Wording

diff --git a/doc/sphinx/reference/varnishstat.rst b/doc/sphinx/reference/varnishstat.rst
index 02a72d0..8940d80 100644
--- a/doc/sphinx/reference/varnishstat.rst
+++ b/doc/sphinx/reference/varnishstat.rst
@@ -26,7 +26,7 @@ OPTIONS
 The following options are available:
 
 -1
-	Instead of presenting of a continuously updated display, print
+	Instead of presenting a continuously updated display, print
 	the statistics to stdout.
 
 -f



From fgsch at lodoss.net  Thu Apr 10 07:26:31 2014
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Thu, 10 Apr 2014 09:26:31 +0200
Subject: [master] 989cfea Install cache_backend.h with the vmod headers
Message-ID: <E1WY9NX-0000BL-7V@project.varnish-software.com>


commit 989cfea0477e285e80ab9fa4c9f82ad7b9813d16
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Thu Apr 10 08:26:02 2014 +0100

    Install cache_backend.h with the vmod headers

diff --git a/bin/varnishd/Makefile.am b/bin/varnishd/Makefile.am
index d9a8e62..402cd0d 100644
--- a/bin/varnishd/Makefile.am
+++ b/bin/varnishd/Makefile.am
@@ -103,6 +103,7 @@ noinst_HEADERS = \
 pkgdataincludedir = $(pkgdatadir)/include
 nobase_pkgdatainclude_HEADERS = \
 	cache/cache.h \
+	cache/cache_backend.h \
 	common/common.h \
 	common/params.h
 



From lkarsten at varnish-software.com  Thu Apr 10 09:16:39 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 10 Apr 2014 11:16:39 +0200
Subject: [master] 2acedeb Prepare 4.0.0 release
Message-ID: <E1WYB67-0007oc-L5@project.varnish-software.com>


commit 2acedeb7d9291a9dffae637abf5fb95d4f640201
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu Apr 10 11:02:32 2014 +0200

    Prepare 4.0.0 release

diff --git a/configure.ac b/configure.ac
index a910e59..4dfb222 100644
--- a/configure.ac
+++ b/configure.ac
@@ -2,7 +2,7 @@ AC_PREREQ(2.59)
 AC_COPYRIGHT([Copyright (c) 2006 Verdens Gang AS
 Copyright (c) 2006-2014 Varnish Software AS])
 AC_REVISION([$Id$])
-AC_INIT([Varnish], [4.0.0-beta1], [varnish-dev at varnish-cache.org])
+AC_INIT([Varnish], [4.0.0], [varnish-dev at varnish-cache.org])
 AC_CONFIG_SRCDIR(include/miniobj.h)
 AC_CONFIG_HEADERS([config.h])
 AC_CONFIG_MACRO_DIR([m4])
diff --git a/doc/changes.rst b/doc/changes.rst
index 28d64da..837266d 100644
--- a/doc/changes.rst
+++ b/doc/changes.rst
@@ -1,3 +1,28 @@
+==============================================
+Changes from 4.0.0 beta1 to 4.0.0 (2014-04-10)
+==============================================
+
+New since 4.0.0-beta1:
+
+- improved varnishstat documentation.
+- In VCL, req.backend_hint is available in vcl_hit
+- ncurses is now a dependency.
+
+
+Bugs fixed
+----------
+
+* 1469_ - Fix build error on PPC
+* 1468_ - Set ttl=0 on failed objects
+* 1462_ - Handle duplicate ReqURL in varnishncsa.
+* 1467_ - Fix missing clearing of oc->busyobj on HSH_Fail.
+
+
+.. _1469: https://www.varnish-cache.org/trac/ticket/1469
+.. _1468: https://www.varnish-cache.org/trac/ticket/1468
+.. _1462: https://www.varnish-cache.org/trac/ticket/1462
+.. _1467: https://www.varnish-cache.org/trac/ticket/1467
+
 
 =====================================
 Changes from 4.0.0 TP2 to 4.0.0 beta1
diff --git a/redhat/varnish.spec b/redhat/varnish.spec
index c1b5f8d..c1f02cf 100644
--- a/redhat/varnish.spec
+++ b/redhat/varnish.spec
@@ -5,7 +5,8 @@
 Summary: High-performance HTTP accelerator
 Name: varnish
 Version: 4.0.0
-Release: 0.20140328%{?v_rc}%{?dist}
+#Release: 0.20140328%{?v_rc}%{?dist}
+Release: 1%{?v_rc}%{?dist}
 License: BSD
 Group: System Environment/Daemons
 URL: http://www.varnish-cache.org/



From lkarsten at varnish-software.com  Thu Apr 10 14:29:54 2014
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 10 Apr 2014 16:29:54 +0200
Subject: [master] 26c2dc6 Ease this up a bit not scare of users
Message-ID: <E1WYFzG-0005D0-7v@project.varnish-software.com>


commit 26c2dc60b23323ace636eb25667d64062039efd3
Author: Lasse Karstensen <lkarsten at varnish-software.com>
Date:   Thu Apr 10 16:29:40 2014 +0200

    Ease this up a bit not scare of users

diff --git a/doc/sphinx/whats-new/upgrading.rst b/doc/sphinx/whats-new/upgrading.rst
index f975179..c2417e6 100644
--- a/doc/sphinx/whats-new/upgrading.rst
+++ b/doc/sphinx/whats-new/upgrading.rst
@@ -7,7 +7,7 @@ Upgrading to Varnish 4
 Changes to VCL
 ==============
 
-Much of the VCL syntax has changed in Varnish 4. We've tried to
+The backend fetch parts of VCL have changed in Varnish 4. We've tried to
 compile a list of changes needed to upgrade here.
 
 Version statement



From apj at mutt.dk  Thu Apr 10 20:06:56 2014
From: apj at mutt.dk (Andreas Plesner)
Date: Thu, 10 Apr 2014 22:06:56 +0200
Subject: [master] 77524d9 Add some text about some of the changed parameters,
 fixes #1474 Introduce new section for new parameters
Message-ID: <E1WYLFQ-0004V2-RG@project.varnish-software.com>


commit 77524d922370f72037ac41d7980be2294c72817a
Author: Andreas Plesner <apj at mutt.dk>
Date:   Thu Apr 10 22:06:17 2014 +0200

    Add some text about some of the changed parameters, fixes #1474
    Introduce new section for new parameters

diff --git a/doc/sphinx/whats-new/upgrading.rst b/doc/sphinx/whats-new/upgrading.rst
index c2417e6..2b23621 100644
--- a/doc/sphinx/whats-new/upgrading.rst
+++ b/doc/sphinx/whats-new/upgrading.rst
@@ -132,14 +132,16 @@ The `remove` keyword is gone
 
 Replaced by `unset`.
 
-Changes to parameters
-=====================
+Changes to existing parameters
+==============================
 
-linger
-~~~~~~
+session_linger
+~~~~~~~~~~~~~~
+`session_linger` has been renamed to `timeout_linger`.
 
 sess_timeout
 ~~~~~~~~~~~~
+`sess_timeout` has been renamed to `timeout_idle`.
 
 sess_workspace
 ~~~~~~~~~~~~~~
@@ -153,8 +155,11 @@ This memory segment has been split into two in 4.0;
 In most cases where you increased `sess_workspace` before, you
 want to increase `workspace_client` now.
 
+New parameters since 3.0
+========================
+
 vcc_allow_inline_c
 ~~~~~~~~~~~~~~~~~~
 
-This parameter is new since 3.0, and prohibits the use of inline
-C code in VCL by default.
+You can now completely disable inline C in your VCL, and it is
+disabled by default.



From varnish-commit at varnish-cache.org  Thu Apr 10 23:56:14 2014
From: varnish-commit at varnish-cache.org (=?utf-8?B?5bi45YWI55Sf?=)
Date: Fri, 11 Apr 2014 07:56:14 +0800
Subject: =?utf-8?B?6YCa55+l77yb?=
Message-ID: <7DF0703ECAB404F614CCD6FD49B67832@rtaaidsgs>

varnish-commit at varnish-cache.org???????????????
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: ?????????? ???.txt
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-commit/attachments/20140411/e8fc4f84/attachment.txt>

From apj at mutt.dk  Fri Apr 11 10:57:58 2014
From: apj at mutt.dk (Andreas Plesner)
Date: Fri, 11 Apr 2014 12:57:58 +0200
Subject: [master] b241f61 Add info about client.port
Message-ID: <E1WYZ9i-0003tL-7K@project.varnish-software.com>


commit b241f613acbd78310424baba86fd70d96f6c2063
Author: Andreas Plesner <apj at mutt.dk>
Date:   Fri Apr 11 12:57:46 2014 +0200

    Add info about client.port

diff --git a/doc/sphinx/whats-new/upgrading.rst b/doc/sphinx/whats-new/upgrading.rst
index 2b23621..82298a2 100644
--- a/doc/sphinx/whats-new/upgrading.rst
+++ b/doc/sphinx/whats-new/upgrading.rst
@@ -110,6 +110,13 @@ is reserved for builtin subs.
 req.backend.healthy replaced by std.healthy(req.backend)
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
+client.port replaced by std.port(client.ip)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The `client.ip` is now a proper type, which renders as an IP address by
+default. You need to use the `std.port()` function to get the port
+number.
+
 obj is now read-only
 ~~~~~~~~~~~~~~~~~~~~
 



From apj at mutt.dk  Fri Apr 11 15:19:25 2014
From: apj at mutt.dk (Andreas Plesner)
Date: Fri, 11 Apr 2014 17:19:25 +0200
Subject: [master] ec13b93 Fix example,
 since req.* isn't available in vcl_backend_response
Message-ID: <E1WYdEj-0004sz-9u@project.varnish-software.com>


commit ec13b93699041fbf721cc0a0b4d0c4a5fe62a635
Author: Andreas Plesner <apj at mutt.dk>
Date:   Fri Apr 11 17:19:04 2014 +0200

    Fix example, since req.* isn't available in vcl_backend_response

diff --git a/doc/sphinx/users-guide/purging.rst b/doc/sphinx/users-guide/purging.rst
index c0e8381..fcd5f5b 100644
--- a/doc/sphinx/users-guide/purging.rst
+++ b/doc/sphinx/users-guide/purging.rst
@@ -122,7 +122,7 @@ object is not available in the `ban lurker` thread.
 You can use the following template to write `ban lurker` friendly bans::
 
   sub vcl_backend_response {
-    set beresp.http.x-url = req.url;
+    set beresp.http.x-url = bereq.url;
   }
 
   sub vcl_deliver {



From fgsch at lodoss.net  Fri Apr 11 18:09:24 2014
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Fri, 11 Apr 2014 20:09:24 +0200
Subject: [master] 6cb0f12 Update, mostly vcl_fetch -> vcl_backend_response
Message-ID: <E1WYftE-00010B-IP@project.varnish-software.com>


commit 6cb0f12a52c5a12b4b082ce7ea285a0a05199c8e
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Fri Apr 11 19:09:01 2014 +0100

    Update, mostly vcl_fetch -> vcl_backend_response

diff --git a/doc/sphinx/glossary/index.rst b/doc/sphinx/glossary/index.rst
index 574156b..2f9250e 100644
--- a/doc/sphinx/glossary/index.rst
+++ b/doc/sphinx/glossary/index.rst
@@ -78,7 +78,7 @@ Varnish Glossary
    backend response
         The response specifically served from a backend to
         varnishd. The backend response may be manipulated in
-        vcl_fetch.
+        vcl_backend_response.
 
    body
 	The bytes that make up the contents of the object, varnishd
diff --git a/doc/sphinx/reference/varnishtest.rst b/doc/sphinx/reference/varnishtest.rst
index bfae5a5..568d4e3 100644
--- a/doc/sphinx/reference/varnishtest.rst
+++ b/doc/sphinx/reference/varnishtest.rst
@@ -82,9 +82,9 @@ An example::
         } -start
 
         varnish v1 -vcl+backend {
-                sub vcl_fetch {
+                sub vcl_backend_response {
                         set beresp.do_esi = true;
-                        if (req.url == "/foo") {
+                        if (bereq.url == "/foo") {
                                 set beresp.ttl = 0s;
                         } else {
                                 set beresp.ttl = 10m;
diff --git a/doc/sphinx/users-guide/compression.rst b/doc/sphinx/users-guide/compression.rst
index fc84f9f..cb95619 100644
--- a/doc/sphinx/users-guide/compression.rst
+++ b/doc/sphinx/users-guide/compression.rst
@@ -29,7 +29,7 @@ content it will be stored in memory in its compressed form. If the
 backend sends content in clear text it will be stored in clear text.
 
 You can make Varnish compress content before storing it in cache in
-`vcl_fetch` by setting 'do_gzip' to true, like this::
+`vcl_backend_response` by setting 'do_gzip' to true, like this::
 
    sub vcl_backend_response {
         if (beresp.http.content-type ~ "text") {
diff --git a/doc/sphinx/users-guide/devicedetection.rst b/doc/sphinx/users-guide/devicedetection.rst
index 5067c4c..2a0cb21 100644
--- a/doc/sphinx/users-guide/devicedetection.rst
+++ b/doc/sphinx/users-guide/devicedetection.rst
@@ -75,8 +75,8 @@ VCL::
     # If the backend does not mention in Vary that it has crafted special
     # content based on the User-Agent (==X-UA-Device), add it. 
     # If your backend does set Vary: User-Agent, you may have to remove that here.
-    sub vcl_fetch {
-        if (req.http.X-UA-Device) {
+    sub vcl_backend_response {
+        if (bereq.http.X-UA-Device) {
             if (!beresp.http.Vary) { # no Vary at all
                 set beresp.http.Vary = "X-UA-Device"; 
             } elseif (beresp.http.Vary !~ "X-UA-Device") { # add to existing Vary
@@ -85,7 +85,7 @@ VCL::
         }
         # comment this out if you don't want the client to know your
         # classification
-        set beresp.http.X-UA-Device = req.http.X-UA-Device;
+        set beresp.http.X-UA-Device = bereq.http.X-UA-Device;
     }
 
     # to keep any caches in the wild from serving wrong content to client #2
@@ -131,15 +131,15 @@ VCL::
     sub vcl_pass { if (req.http.X-UA-Device) { set bereq.http.User-Agent = req.http.X-UA-Device; } }
 
     # standard Vary handling code from previous examples.
-    sub vcl_fetch {
-        if (req.http.X-UA-Device) {
+    sub vcl_backend_response {
+        if (bereq.http.X-UA-Device) {
             if (!beresp.http.Vary) { # no Vary at all
                 set beresp.http.Vary = "X-UA-Device";
             } elseif (beresp.http.Vary !~ "X-UA-Device") { # add to existing Vary
                 set beresp.http.Vary = beresp.http.Vary + ", X-UA-Device";
             }
         }
-        set beresp.http.X-UA-Device = req.http.X-UA-Device;
+        set beresp.http.X-UA-Device = bereq.http.X-UA-Device;
     }
     sub vcl_deliver {
         if ((req.http.X-UA-Device) && (resp.http.Vary)) {
@@ -186,8 +186,8 @@ VCL::
 
     # Handle redirects, otherwise standard Vary handling code from previous
     # examples.
-    sub vcl_fetch {
-        if (req.http.X-UA-Device) {
+    sub vcl_backend_response {
+        if (bereq.http.X-UA-Device) {
             if (!beresp.http.Vary) { # no Vary at all
                 set beresp.http.Vary = "X-UA-Device";
             } elseif (beresp.http.Vary !~ "X-UA-Device") { # add to existing Vary
@@ -203,7 +203,7 @@ VCL::
                 set beresp.http.location = regsub(beresp.http.location, "[?&]devicetype=.*$", "");
             }
         }
-        set beresp.http.X-UA-Device = req.http.X-UA-Device;
+        set beresp.http.X-UA-Device = bereq.http.X-UA-Device;
     }
     sub vcl_deliver {
         if ((req.http.X-UA-Device) && (resp.http.Vary)) {
diff --git a/doc/sphinx/users-guide/esi.rst b/doc/sphinx/users-guide/esi.rst
index 5c7ce72..b81d2ea 100644
--- a/doc/sphinx/users-guide/esi.rst
+++ b/doc/sphinx/users-guide/esi.rst
@@ -50,11 +50,11 @@ Now, lets have an HTML file that has an ESI include statement::
 
 For ESI to work you need to activate ESI processing in VCL, like this::
 
-    sub vcl_fetch {
-    	if (req.url == "/test.html") {
+    sub vcl_backend_response {
+    	if (bereq.url == "/test.html") {
            set beresp.do_esi = true; // Do ESI processing
            set beresp.ttl = 24 h;    // Sets the TTL on the HTML above
-    	} elseif (req.url == "/cgi-bin/date.cgi") {
+    	} elseif (bereq.url == "/cgi-bin/date.cgi") {
            set beresp.ttl = 1m;      // Sets a one minute TTL on
 	       	       	 	     // the included object
         }
diff --git a/doc/sphinx/users-guide/vcl-actions.rst b/doc/sphinx/users-guide/vcl-actions.rst
index 0775afe..d4ede58 100644
--- a/doc/sphinx/users-guide/vcl-actions.rst
+++ b/doc/sphinx/users-guide/vcl-actions.rst
@@ -8,12 +8,12 @@ The most common actions to return are these:
 .. XXX:Maybe a bit more explanation here what is an action and how it is returned? benc
 
 *pass*
- When you return pass the request and subsequent response will be passed to
- and from the backend server. It won't be cached. `pass` can be returned from
- `vcl_recv`
+  When you return pass the request and subsequent response will be passed to
+  and from the backend server. It won't be cached. `pass` can be returned from
+  `vcl_recv`.
 
-*lookup*
-  When you return lookup from `vcl_recv` you tell Varnish to deliver content 
+*hash*
+  When you return hash from `vcl_recv` you tell Varnish to deliver content 
   from cache even if the request othervise indicates that the request 
   should be passed. 
 
@@ -24,18 +24,15 @@ The most common actions to return are these:
   client and the backend connections and Varnish will just sit there
   and shuffle bytes back and forth. Varnish will not look at the data being 
   send back and forth - so your logs will be incomplete. 
-  Beware that with HTTP 1.1 a client can send several requests on the same 
-  connection and so you should instruct Varnish to add a "Connection: close"
-  header before actually returning pipe. 
 
 *deliver*
- Deliver the object to the client. Usually returned from `vcl_backend_response`. 
+  Deliver the object to the client. Usually returned from `vcl_backend_response`. 
 
 *restart*
- Restart processing of the request. You can restart the processing of
- the whole transaction. Changes to the `req` object are retained.
+  Restart processing of the request. You can restart the processing of
+  the whole transaction. Changes to the `req` object are retained.
 
 *retry*
- Retry the request against the backend. This can be called from
- `vcl_backend_response` or `vcl_backend_error` if you don't like the response 
- that the backend delivered.
+  Retry the request against the backend. This can be returned from
+  `vcl_backend_response` or `vcl_backend_error` if you don't like the response 
+  that the backend delivered.
diff --git a/doc/sphinx/users-guide/vcl-built-in-subs.rst b/doc/sphinx/users-guide/vcl-built-in-subs.rst
index 708c05e..0a90a34 100644
--- a/doc/sphinx/users-guide/vcl-built-in-subs.rst
+++ b/doc/sphinx/users-guide/vcl-built-in-subs.rst
@@ -119,7 +119,7 @@ of the following keywords:
 
   fetch
     Retrieve the requested object from the backend. Control will
-    eventually pass to `vcl_fetch`.
+    eventually pass to `vcl_backend_fetch`.
 
 vcl_hash
 ~~~~~~~~
diff --git a/doc/sphinx/users-guide/vcl-example-manipulating-responses.rst b/doc/sphinx/users-guide/vcl-example-manipulating-responses.rst
index 6719524..31e10e1 100644
--- a/doc/sphinx/users-guide/vcl-example-manipulating-responses.rst
+++ b/doc/sphinx/users-guide/vcl-example-manipulating-responses.rst
@@ -6,8 +6,8 @@ Altering the backend response
 Here we override the TTL of a object coming from the backend if it
 matches certain criteria::
 
-  sub vcl_fetch {
-     if (req.url ~ "\.(png|gif|jpg)$") {
+  sub vcl_backend_response {
+     if (bereq.url ~ "\.(png|gif|jpg)$") {
        unset beresp.http.set-cookie;
        set beresp.ttl = 1h;
     }
diff --git a/doc/sphinx/users-guide/vcl-grace.rst b/doc/sphinx/users-guide/vcl-grace.rst
index afc92bd..c3a122c 100644
--- a/doc/sphinx/users-guide/vcl-grace.rst
+++ b/doc/sphinx/users-guide/vcl-grace.rst
@@ -27,7 +27,7 @@ So, in order to serve stale content we must first have some content to
 serve. So to make Varnish keep all objects for 30 minutes beyond their
 TTL use the following VCL::
 
-  sub vcl_fetch {
+  sub vcl_backend_response {
     set beresp.grace = 30m;
   }
 
@@ -45,7 +45,7 @@ minutes if we are unable to serve them? Well, if you have enabled
 backend is sick and if it is we can serve the stale content for a bit
 longer.::
 
-   if (! req.backend.healthy) {
+   if (!std.healthy(backend)) {
       set req.grace = 5m;
    } else {
       set req.grace = 15s;



From fgsch at lodoss.net  Fri Apr 11 18:25:46 2014
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Fri, 11 Apr 2014 20:25:46 +0200
Subject: [master] cf33308 vcl_hash() returns lookup now
Message-ID: <E1WYg94-00046l-B0@project.varnish-software.com>


commit cf333085ae42d8b3f39707a55fca6b51435fec1c
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Fri Apr 11 19:24:51 2014 +0100

    vcl_hash() returns lookup now
    
    While here fmt(1) the text.

diff --git a/doc/sphinx/users-guide/vcl-hashing.rst b/doc/sphinx/users-guide/vcl-hashing.rst
index 83758ad..bdc47b0 100644
--- a/doc/sphinx/users-guide/vcl-hashing.rst
+++ b/doc/sphinx/users-guide/vcl-hashing.rst
@@ -1,25 +1,25 @@
 Hashing
 -------
 
-Internally, when Varnish stores content in the cache it stores the object together with a hash
-key to find the object again. In the default setup this key is
-calculated based on the content of the *Host* header or the IP address
-of the server and the URL.
+Internally, when Varnish stores content in the cache it stores the object
+together with a hash key to find the object again. In the default setup
+this key is calculated based on the content of the *Host* header or the
+IP address of the server and the URL.
 
 Behold the `default vcl`::
 
- sub vcl_hash {
-     hash_data(req.url);
-     if (req.http.host) {
-         hash_data(req.http.host);
-     } else {
-         hash_data(server.ip);
-     }
-     return (hash);
- }
-
-As you can see it first checks in `req.url` then `req.http.host` if it
-exists. It is worth pointing out that Varnish doesn't lowercase the
+    sub vcl_hash {
+        hash_data(req.url);
+        if (req.http.host) {
+            hash_data(req.http.host);
+        } else {
+            hash_data(server.ip);
+        }
+        return (lookup);
+    }
+
+As you can see it first checks in `req.url` then `req.http.host` if
+it exists. It is worth pointing out that Varnish doesn't lowercase the
 hostname or the URL before hashing it so in theory having "Varnish.org/"
 and "varnish.org/" would result in different cache entries. Browsers
 however, tend to lowercase hostnames.
@@ -29,22 +29,20 @@ serve up different content to different clients based on arbitrary
 criteria.
 
 Let's say you want to serve pages in different languages to your users
-based on where their IP address is located. You would need some Vmod
-to get a country code and then put it into the hash. It might look
-like this.
+based on where their IP address is located. You would need some Vmod to
+get a country code and then put it into the hash. It might look like this.
 
 In `vcl_recv`::
 
-  set req.http.X-Country-Code = geoip.lookup(client.ip);
+    set req.http.X-Country-Code = geoip.lookup(client.ip);
 
 And then add a `vcl_hash`::
 
- sub vcl_hash {
-   hash_data(req.http.X-Country-Code);
- }
+    sub vcl_hash {
+        hash_data(req.http.X-Country-Code);
+    }
 
-As the default VCL will take care of adding the host and URL to the
-hash we don't have to do anything else. Be careful calling
-``return(hash)`` as this will abort the execution of the default VCL and
-Varnish can end up returning data based on
-more or less random inputs.
+As the default VCL will take care of adding the host and URL to the hash
+we don't have to do anything else. Be careful calling ``return (lookup)``
+as this will abort the execution of the default VCL and Varnish can end
+up returning data based on more or less random inputs.



From phk at FreeBSD.org  Fri Apr 11 23:56:39 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Sat, 12 Apr 2014 01:56:39 +0200
Subject: [master] 514b371 Preaching the "Moral License" gospel,
 not just for Varnish
Message-ID: <E1WYlJH-0008Rb-8k@project.varnish-software.com>


commit 514b371ea87f974db90b82c55f4e3f518a605bc5
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Apr 11 23:55:29 2014 +0000

    Preaching the "Moral License" gospel, not just for Varnish

diff --git a/doc/sphinx/phk/dough.rst b/doc/sphinx/phk/dough.rst
new file mode 100644
index 0000000..08c5645
--- /dev/null
+++ b/doc/sphinx/phk/dough.rst
@@ -0,0 +1,266 @@
+.. _phk_dough:
+
+====================================================
+Raking in the dough on Free and Open Source Software
+====================================================
+
+I'm writing this on the third day after the "Heartbleed" bug in OpenSSL
+devasted internet security, and while I have been very critical of the
+OpenSSL source code since I first saw it, I have nothing but admiration
+for the OpenSSL crew and their effort.
+
+In particular considering what they're paid for it.
+
+Inspired by an article in `Wall Street Journal`_ which tangentially
+touches on the lack of funding for OpenSSL development, I have
+decided to write up my own experiences with funding Open Source
+Software development in some detail.
+
+I've been in the software industry for 30 years now, and I have
+made a living more or less directly from Open Source Software
+for the most recent 15 years.
+
+Sometimes the money came from helping a customer use Open Source
+Software, some times I wrote the Open Source Software for their
+needs and sometimes, as with the `Varnish Moral License`_ I get
+paid to develop and maintain Open Source Software for the greater
+common good.
+
+FreeBSD community funding
+=========================
+
+My first crowd-funding of Free and Open Source Software, was in
+2004, where I `solicited the FreeBSD community`_ for money, so that
+I could devote three to six months of my time to the FreeBSD disk-I/O
+subsystem.
+
+At that time I had spent 10 years as one of the central and key
+FreeBSD developers, so there were no question about my ability
+or suitability for the task at hand.
+
+But in 2004 crowd-funding was not yet "in", and I had to figure
+out how to do it myself.
+
+My parents brought me up to think that finances is a private matter
+but I concluded that the only way you could ask strangers to trow
+money at you, would be to run an open book, where they could see
+what happened to them, so I did open books.
+
+My next dilemma was about my rate, again I had always perceived my
+rate to be a private matter between me and my customers.
+
+My rate is about half of what most people expect -- because I wont
+work for most people: I only work on things I really *care* about.
+
+One of my worries therefore were that publishing my rate would
+undercut friends and colleagues in the FreeBSD project who made a
+living consulting.
+
+But again, there were no way around it, so I published my rate but
+made every attempt to distinguish it from a consulting rate, and
+I never heard any complaints.
+
+And so, having agonized over the exact text and sounded it off on a
+couple of close friends in the FreeBSD project, I threw the proposal
+out there -- and wondered what would happen next.
+
+I had a perfectly safe fall-back plan, you have to when you have
+two kids and a mortgage to feed, but I really had no idea what would
+happen.
+
+Worst case, I'd cause the mother of all `bikesheds`_ get thrown out
+of the FreeBSD project, and be denounced for my "ideological impurity"
+with respect to Free and Open Source Software.
+
+Best case, I expected to get maybe one or two months funded.
+
+The FreeBSD community responded overwhelmingly, my company has never
+sent as many invoices as it did in 2004, and my accountant nearly
+blew a fuse.
+
+And suddenly I found myself in a situation I had never even considered
+how to handle:  How to stop people from sending me money.
+
+I had simply set up a PayPal account, (more on that in a bit), and
+at least at that time, there were no way to prevent people from
+dropping money into it, no matter how much you wanted to stop them.
+
+In the end I managed to yell loud enough and only got overfunded
+a few percent, and I belive that my attempt to deflect the surplus
+to the FreeBSD Foundation gave them a little boost that year.
+
+So about PayPal:  The first thing they did was to shut my account,
+and demand all kinds of papers to be faxed to them, including a
+copy of my passport, despite the fact that Danish law was quite
+clear on that being illegal.  Then, as now, their dispute resolution
+process was less than user-friendly, and in the end it took an
+appeal to a high-ranking officer in PayPal and quite a bit of time
+to actually get the money people had donated.
+
+I swore to myself that next time, if there ever came a next time,
+PayPal would not be involved.  Besides, I found their fees quite
+excessive.
+
+In total I made EUR27K, and it kept my kids fed and my bank
+happy for the six months I worked on it.
+
+And work I did.
+
+I've never had a harsher boss than those six months, and it surprised
+me how much it stressed me, because I felt like I was working on a
+stage, with the entire FreeBSD project in audience, wondering if I
+were going to deliver the goods or not.
+
+As a result, the 187 donors certainly got their moneys worth,
+most of that half year I worked 80 hour weeks, which made me
+decide not to continue, despite many donors indicating that they
+were perfectly willing to fund several more months.
+
+Varnish community funding
+=========================
+
+Five years later, having developed Varnish 1.0 for Norways "Verdens
+Gang" newspaper, I decided to give community funding a go again.
+
+Wiser from experience, I structured the `Varnish Moral License`_
+to tackle the issues which had caused me grief the first time
+around:
+
+Contact first, then send money, not the other way around, and also
+a focus on fewer larger sponsors, rather than people sending me
+EUR10 or USD15 or even, in one case, the EUR1 which happened to
+remain in his PayPal Account.
+
+I ran even more open books this time, on the VML webpages you can
+see how many hours and a one-line description of what I did in them,
+for every single day I've been working under the VML since 2010.
+
+I also decided to be honest with myself and my donors, one hour
+of work was one hour of work -- nobody would benefit from me
+dying from stress.
+
+In practice it doesn't quite work like that, there are plenty of
+thinking in the shower, emails and IRC answers at all hours of the
+day and a lot of "just checking a detail" that happens off the
+clock, because I like my job, and nothing could stop me anyway.
+
+In each of 2010, 2011 and 2013 I worked around 950 hours work on
+Varnish, funded by the community.
+
+In 2012 I only worked 589 hours, because I was building a prototype
+computer cluster to do adaptive optics real-time calculations for
+the ESO `Extremely Large Telescope`_ ("ELT") -- There was no way I
+could say no to that contract :-)
+
+In 2014 I actually have hours available do even more Varnish work,
+and I have done so in the ramp up to the 4.0.0 release, but despite
+my not so subtle hints, the current outlook is still only for 800
+hours to be funded, but I'm crossing my fingers that more sponsors
+will appear now that V4 is released.  (Nudge, nudge, wink, wink,
+he said knowingly! :-)
+
+Why Free and Open Source costs money
+====================================
+
+Varnish is about 90.000 lines of code, the VML brings in about
+EUR90K a year, and that means that Varnish has me working and
+caring about issues big and small.
+
+Not that I am satisfied with our level of effort, we should have
+much better documentation, our wish-list of features is far too
+long and we take too long to close tickets.
+
+But I'm not going to complain, because the Heartbleed vulnerability
+revealed that even though OpenSSL is about three to five times
+larger in terms of code, the OpenSSL Foundation Inc. took in only
+about EUR700K last year, and most of that was for consulting and
+certification, not for "free-range" development and maintenance of
+the OpenSSL source code base.
+
+I really hope that the Heartbleed vulnerability helps bring home
+the message to other communities, that Free and Open Source Software
+does not materialize out of empty space, it is written by people.
+
+People who love what we do, which is why I'm sitting here,
+way past midnight on a friday evening, writing this phamplet.
+
+But software *is* written by people, real people with kids, cars,
+mortgages, leaky roofs, sick pets, infirm parents and all other
+kinds of perfectly normal worries of an adult human being.
+
+The best way to improve the quality of Free and Open Source Software,
+is to make it possible for these people to spend time on it.
+
+They need time to review submissions carefully, time to write and
+run test-cases, time to respond and fix to bug-reports, time to
+code and most of all, time to think about the code.
+
+But it would not even be close to morally defensible to ask these
+people to forego time to play with their kids, so that they instead
+develop and maintain the software that drives other peoples companies.
+
+The right way to go -- the moral way to go -- and by far the most
+productive way to go, is to pay the developers so they can make
+the software they love their living.
+
+How to fund Free and Open Source Software
+=========================================
+
+One way is to hire them, with the understanding that they spend
+some company time on the software.
+
+Experience has shown that these people almost invariably have highly
+desirable brains which employers love to throw at all sorts of
+interesting problems, which tends to erode the "donated" company
+time.
+
+But a lot of Free and Open Source Software has been, and still is
+developed and  maintained this way, with or without written
+agreements or even knowledge of this being the case.
+
+Another way is for software projects to set up foundations to
+collect money and hire developers.  This is a relatively complex
+thing to do, and it will only be availabel for larger projects.
+
+The Apache Foundation "adopts" smaller projects inside their field
+of interest, and I belive that works OK, but I'm not sure if it
+can easily be transplanted to different topics.
+
+The final way is to simply throw money a the developers, the
+way the FreeBSD and Varnish communities have done with me.
+
+It is a far more flexible solution with respect to level of
+engangement, national boundaries etc. etc, but in many ways it
+demands more from both sides of the deal, in particular
+with respect to paperwork, taxes and so on.
+
+Conclusion
+==========
+
+I am obiously biased, I derive a large fraction of my relatively
+modest income from community funding, for which I am the Varnish
+community deeply grateful.
+
+But biased as I may be, I belive that the Varnish community and I
+has shown that a tiny investment goes a long way in Free and Open
+Source Software.
+
+I hope to see that mutual benefit spread to other communities and
+projects, not just to OpenSSL and not just because they found a
+really bad bug the other way, but to any community around any piece
+of software which does serious work for serious companies.
+
+Thanks in advance,
+
+Poul-Henning, 2014-04-11
+
+.. _Wall Street Journal: http://online.wsj.com/news/articles/SB10001424052702303873604579491350251315132
+
+.. _Varnish Moral License: http://phk.freebsd.dk/VML
+
+.. _solicited the FreeBSD community: http://people.freebsd.org/~phk/funding.html
+
+.. _Extremely Large Telescope: http://www.eso.org/public/teles-instr/e-elt/
+
+.. _bikesheds: http://bikeshed.org/
+
diff --git a/doc/sphinx/phk/index.rst b/doc/sphinx/phk/index.rst
index 7eac4e5..40a4830 100644
--- a/doc/sphinx/phk/index.rst
+++ b/doc/sphinx/phk/index.rst
@@ -8,6 +8,7 @@ You may or may not want to know what Poul-Henning thinks.
 .. toctree::
 	:maxdepth: 1
 
+	dough.rst
 	wanton_destruction.rst
 	spdy.rst
 	http20.rst



From phk at FreeBSD.org  Sat Apr 12 00:17:13 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Sat, 12 Apr 2014 02:17:13 +0200
Subject: [master] e477eb7 git commit is magic: It makes it possible to see
 mistakes in the text.
Message-ID: <E1WYldB-0003Px-FV@project.varnish-software.com>


commit e477eb72cb6bfb2a71abd1020b7fcde8e0265d63
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Sat Apr 12 00:16:51 2014 +0000

    git commit is magic:  It makes it possible to see mistakes in the text.

diff --git a/doc/sphinx/phk/dough.rst b/doc/sphinx/phk/dough.rst
index 08c5645..ed9f6be 100644
--- a/doc/sphinx/phk/dough.rst
+++ b/doc/sphinx/phk/dough.rst
@@ -129,7 +129,7 @@ around:
 Contact first, then send money, not the other way around, and also
 a focus on fewer larger sponsors, rather than people sending me
 EUR10 or USD15 or even, in one case, the EUR1 which happened to
-remain in his PayPal Account.
+linger in his PayPal Account.
 
 I ran even more open books this time, on the VML webpages you can
 see how many hours and a one-line description of what I did in them,
@@ -173,9 +173,11 @@ long and we take too long to close tickets.
 But I'm not going to complain, because the Heartbleed vulnerability
 revealed that even though OpenSSL is about three to five times
 larger in terms of code, the OpenSSL Foundation Inc. took in only
-about EUR700K last year, and most of that was for consulting and
-certification, not for "free-range" development and maintenance of
-the OpenSSL source code base.
+about EUR700K last year.
+
+And most of that EUR700K was for consulting and certification, not
+for "free-range" development and maintenance of the OpenSSL source
+code base so badly needs.
 
 I really hope that the Heartbleed vulnerability helps bring home
 the message to other communities, that Free and Open Source Software
@@ -247,7 +249,7 @@ Source Software.
 
 I hope to see that mutual benefit spread to other communities and
 projects, not just to OpenSSL and not just because they found a
-really bad bug the other way, but to any community around any piece
+really bad bug the other day, but to any community around any piece
 of software which does serious work for serious companies.
 
 Thanks in advance,



From phk at FreeBSD.org  Sat Apr 12 07:59:24 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Sat, 12 Apr 2014 09:59:24 +0200
Subject: [master] 426474c typo
Message-ID: <E1WYsqS-0002LN-JF@project.varnish-software.com>


commit 426474c067ce0c129a4f249b6b1a7a29527f2bb2
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Sat Apr 12 07:59:17 2014 +0000

    typo

diff --git a/doc/sphinx/phk/dough.rst b/doc/sphinx/phk/dough.rst
index ed9f6be..3e71fe6 100644
--- a/doc/sphinx/phk/dough.rst
+++ b/doc/sphinx/phk/dough.rst
@@ -222,7 +222,7 @@ agreements or even knowledge of this being the case.
 
 Another way is for software projects to set up foundations to
 collect money and hire developers.  This is a relatively complex
-thing to do, and it will only be availabel for larger projects.
+thing to do, and it will only be available for larger projects.
 
 The Apache Foundation "adopts" smaller projects inside their field
 of interest, and I belive that works OK, but I'm not sure if it



From martin at varnish-software.com  Mon Apr 14 16:03:41 2014
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Mon, 14 Apr 2014 18:03:41 +0200
Subject: [master] 89870e0 Be more RFC2616 compliant,
 by keeping Date header as given from the backend. Set it on the
 object if missing.
Message-ID: <E1WZjMD-0006OC-L0@project.varnish-software.com>


commit 89870e0bbd785964c322e1e453f492d747731c88
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Mon Apr 14 14:40:11 2014 +0200

    Be more RFC2616 compliant, by keeping Date header as given from the
    backend. Set it on the object if missing.
    
    Previously we would add an explicit Date header on each response with
    the time and date of the response, overwriting any Date header on the
    object that came from the backend.
    
    Now we will add a Date header if the backend didn't supply one just
    before vcl_backend_response() is run, and not add any Date headers for
    the replies.
    
    This change means Date headers sent by Varnish is approximately the
    time and date this object was generated by the backend, and not the
    current date and time.

diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index 19b8fb8..3091d29 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -249,6 +249,8 @@ static enum fetch_step
 vbf_stp_startfetch(struct worker *wrk, struct busyobj *bo)
 {
 	int i, do_ims;
+	double now;
+	char time_str[VTIM_FORMAT_SIZE];
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
@@ -292,7 +294,8 @@ vbf_stp_startfetch(struct worker *wrk, struct busyobj *bo)
 		VSC_C_main->backend_retry++;
 		i = V1F_fetch_hdr(wrk, bo, bo->req);
 	}
-	VSLb_ts_busyobj(bo, "Beresp", W_TIM_real(wrk));
+	now = W_TIM_real(wrk);
+	VSLb_ts_busyobj(bo, "Beresp", now);
 
 	if (i) {
 		AZ(bo->vbc);
@@ -302,6 +305,22 @@ vbf_stp_startfetch(struct worker *wrk, struct busyobj *bo)
 	AN(bo->vbc);
 	http_VSL_log(bo->beresp);
 
+	if (!http_GetHdr(bo->beresp, H_Date, NULL)) {
+		/*
+		 * RFC 2616 14.18 Date: The Date general-header field
+		 * represents the date and time at which the message was
+		 * originated, having the same semantics as orig-date in
+		 * RFC 822. ... A received message that does not have a
+		 * Date header field MUST be assigned one by the recipient
+		 * if the message will be cached by that recipient or
+		 * gatewayed via a protocol which requires a Date.
+		 *
+		 * If we didn't get a Date header, we assign one here.
+		 */
+		VTIM_format(now, time_str);
+		http_PrintfHeader(bo->beresp, "Date: %s", time_str);
+	}
+
 	/*
 	 * These two headers can be spread over multiple actual headers
 	 * and we rely on their content outside of VCL, so collect them
diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index 1e9ecd5..c782d44 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -87,7 +87,6 @@ DOT deliver:deliver:s -> DONE [style=bold,color=blue]
 static enum req_fsm_nxt
 cnt_deliver(struct worker *wrk, struct req *req)
 {
-	char time_str[30];
 	double now;
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
@@ -110,10 +109,6 @@ cnt_deliver(struct worker *wrk, struct req *req)
 	http_ClrHeader(req->resp);
 	http_FilterResp(req->obj->http, req->resp, 0);
 
-	http_Unset(req->resp, H_Date);
-	VTIM_format(now, time_str);
-	http_PrintfHeader(req->resp, "Date: %s", time_str);
-
 	if (req->wrk->stats.cache_hit)
 		http_PrintfHeader(req->resp,
 		    "X-Varnish: %u %u", req->vsl->wid & VSL_IDENTMASK,
diff --git a/bin/varnishtest/tests/c00044.vtc b/bin/varnishtest/tests/c00044.vtc
index 2d1d3cc..091e8a8 100644
--- a/bin/varnishtest/tests/c00044.vtc
+++ b/bin/varnishtest/tests/c00044.vtc
@@ -25,6 +25,8 @@ varnish v1 \
 	sub vcl_backend_response {
 		set beresp.do_stream = false;
 		set beresp.storage_hint = "invalid";
+		# Unset Date header to not change the object sizes
+		unset beresp.http.Date;
 	}
 } -start
 
diff --git a/bin/varnishtest/tests/c00045.vtc b/bin/varnishtest/tests/c00045.vtc
index 126f1a9..33b7b9e 100644
--- a/bin/varnishtest/tests/c00045.vtc
+++ b/bin/varnishtest/tests/c00045.vtc
@@ -17,6 +17,8 @@ varnish v1 \
 	sub vcl_backend_response {
 		set beresp.do_stream = false;
 		set beresp.storage_hint = "s0";
+		# Unset Date header to not change the object sizes
+		unset beresp.http.Date;
 	}
 } -start
 
diff --git a/bin/varnishtest/tests/r01140.vtc b/bin/varnishtest/tests/r01140.vtc
index e38e304..11e2a72 100644
--- a/bin/varnishtest/tests/r01140.vtc
+++ b/bin/varnishtest/tests/r01140.vtc
@@ -22,6 +22,8 @@ varnish v1 -arg "-p nuke_limit=0 -p shortlived=0" \
 	-arg "-smalloc,1m" -vcl+backend {
 	sub vcl_backend_response {
 		set beresp.do_stream = false;
+		# Unset Date header to not change the object sizes
+		unset beresp.http.Date;
 	}
 } -start
 
diff --git a/bin/varnishtest/tests/r01284.vtc b/bin/varnishtest/tests/r01284.vtc
index f89b579..dc6336f 100644
--- a/bin/varnishtest/tests/r01284.vtc
+++ b/bin/varnishtest/tests/r01284.vtc
@@ -16,6 +16,8 @@ varnish v1 \
 	sub vcl_backend_response {
 		set beresp.do_stream = false;
 		set beresp.storage_hint = "Transient";
+		# Unset Date header to not change the object sizes
+		unset beresp.http.Date;
 	}
 } -start
 



From martin at varnish-software.com  Mon Apr 14 16:03:41 2014
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Mon, 14 Apr 2014 18:03:41 +0200
Subject: [master] d31fa83 Timestamp "Process" after vcl_deliver has been run.
Message-ID: <E1WZjMD-0006OJ-MC@project.varnish-software.com>


commit d31fa8397deda3697d68a381d7df48073e41556d
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Mon Apr 14 15:44:00 2014 +0200

    Timestamp "Process" after vcl_deliver has been run.
    
    The EXP_Touch and the Age header will be based off the last timestamp
    taken. This can be slightly inaccurate, but saves us from having to do
    another timestamp for them.
    
    Fixes: #1475

diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index c782d44..b6d0500 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -87,7 +87,6 @@ DOT deliver:deliver:s -> DONE [style=bold,color=blue]
 static enum req_fsm_nxt
 cnt_deliver(struct worker *wrk, struct req *req)
 {
-	double now;
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
@@ -99,10 +98,8 @@ cnt_deliver(struct worker *wrk, struct req *req)
 
 	assert(req->obj->objcore->refcnt > 0);
 
-	now = W_TIM_real(wrk);
-	VSLb_ts_req(req, "Process", now);
 	if (req->obj->objcore->exp_flags & OC_EF_EXP)
-		EXP_Touch(req->obj->objcore, now);
+		EXP_Touch(req->obj->objcore, req->t_prev);
 
 	HTTP_Setup(req->resp, req->ws, req->vsl, SLT_RespMethod);
 
@@ -117,8 +114,13 @@ cnt_deliver(struct worker *wrk, struct req *req)
 		http_PrintfHeader(req->resp,
 		    "X-Varnish: %u", req->vsl->wid & VSL_IDENTMASK);
 
+	/* We base Age calculation upon the last timestamp taken during
+	   client request processing. This gives some inaccuracy, but
+	   since Age is only full second resolution that shouldn't
+	   matter. */
+	assert(req->t_prev > req->obj->exp.t_origin);
 	http_PrintfHeader(req->resp, "Age: %.0f",
-	    now - req->obj->exp.t_origin);
+	    req->t_prev - req->obj->exp.t_origin);
 
 	http_SetHeader(req->resp, "Via: 1.1 varnish (v4)");
 
@@ -127,6 +129,7 @@ cnt_deliver(struct worker *wrk, struct req *req)
 		RFC2616_Weaken_Etag(req->resp);
 
 	VCL_deliver_method(req->vcl, wrk, req, NULL, req->http->ws);
+	VSLb_ts_req(req, "Process", W_TIM_real(wrk));
 
 	/* Stop the insanity before it turns "Hotel California" on us */
 	if (req->restarts >= cache_param->max_restarts)



From martin at varnish-software.com  Mon Apr 14 16:03:41 2014
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Mon, 14 Apr 2014 18:03:41 +0200
Subject: [master] 9919320 Use the time we get the content from the backend as
 the origin time of objects.
Message-ID: <E1WZjMD-0006OS-NL@project.varnish-software.com>


commit 9919320874d90c1a99e3a7fe9172c63fa8c24159
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Mon Apr 14 17:09:20 2014 +0200

    Use the time we get the content from the backend as the origin time of objects.
    
    Before, the origin time for the objects was defined as the time we
    received the full request from the client that initiated the
    fetch. Now we use the time we received the response headers from the
    backend.

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index c8d8d89..b0740c1 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -532,7 +532,6 @@ struct busyobj {
 	 */
 	unsigned		refcount;
 	int			retries;
-	double			t_fetch;
 	struct req		*req;
 
 	uint8_t			*vary;
@@ -1240,7 +1239,7 @@ int WS_Overflowed(const struct ws *ws);
 void *WS_Printf(struct ws *ws, const char *fmt, ...) __printflike(2, 3);
 
 /* rfc2616.c */
-void RFC2616_Ttl(struct busyobj *);
+void RFC2616_Ttl(struct busyobj *, double now);
 enum body_status RFC2616_Body(struct busyobj *, struct dstat *);
 unsigned RFC2616_Req_Gzip(const struct http *);
 int RFC2616_Do_Cond(const struct req *sp);
diff --git a/bin/varnishd/cache/cache_busyobj.c b/bin/varnishd/cache/cache_busyobj.c
index 22f6441..45e2845 100644
--- a/bin/varnishd/cache/cache_busyobj.c
+++ b/bin/varnishd/cache/cache_busyobj.c
@@ -148,8 +148,6 @@ VBO_GetBusyObj(struct worker *wrk, const struct req *req)
 	bo->vcl = req->vcl;
 	VCL_Ref(bo->vcl);
 
-	bo->t_fetch = req->t_req;
-	assert(!isnan(bo->t_fetch) && bo->t_fetch != 0.);
 	bo->t_first = bo->t_prev = NAN;
 
 	return (bo);
diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index 3091d29..2bb5feb 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -347,7 +347,7 @@ vbf_stp_startfetch(struct worker *wrk, struct busyobj *bo)
 	 * What does RFC2616 think about TTL ?
 	 */
 	EXP_Clr(&bo->exp);
-	RFC2616_Ttl(bo);
+	RFC2616_Ttl(bo, now);
 
 	/* private objects have negative TTL */
 	if (bo->fetch_objcore->flags & OC_F_PRIVATE)
@@ -670,7 +670,7 @@ vbf_stp_error(struct worker *wrk, struct busyobj *bo)
 	HTTP_Setup(bo->beresp, bo->ws, bo->vsl, SLT_BerespMethod);
 	http_SetResp(bo->beresp, "HTTP/1.1", 503, "Backend fetch failed");
 
-	bo->exp.t_origin = VTIM_real();
+	bo->exp.t_origin = bo->t_prev;
 	bo->exp.ttl = 0;
 	bo->exp.grace = 0;
 	bo->exp.keep = 0;
diff --git a/bin/varnishd/cache/cache_rfc2616.c b/bin/varnishd/cache/cache_rfc2616.c
index 5334291..bb0a85e 100644
--- a/bin/varnishd/cache/cache_rfc2616.c
+++ b/bin/varnishd/cache/cache_rfc2616.c
@@ -63,7 +63,7 @@
  */
 
 void
-RFC2616_Ttl(struct busyobj *bo)
+RFC2616_Ttl(struct busyobj *bo, double now)
 {
 	unsigned max_age, age;
 	double h_date, h_expires;
@@ -76,8 +76,8 @@ RFC2616_Ttl(struct busyobj *bo)
 
 	hp = bo->beresp;
 
-	assert(bo->t_fetch != 0.0 && !isnan(bo->t_fetch));
-	expp->t_origin = bo->t_fetch;
+	assert(now != 0.0 && !isnan(now));
+	expp->t_origin = now;
 
 	expp->ttl = cache_param->default_ttl;
 	expp->grace = cache_param->default_grace;
@@ -154,16 +154,16 @@ RFC2616_Ttl(struct busyobj *bo)
 		}
 
 		if (h_date == 0 ||
-		    fabs(h_date - bo->t_fetch) < cache_param->clock_skew) {
+		    fabs(h_date - now) < cache_param->clock_skew) {
 			/*
 			 * If we have no Date: header or if it is
 			 * sufficiently close to our clock we will
 			 * trust Expires: relative to our own clock.
 			 */
-			if (h_expires < bo->t_fetch)
+			if (h_expires < now)
 				expp->ttl = 0;
 			else
-				expp->ttl = h_expires - bo->t_fetch;
+				expp->ttl = h_expires - now;
 			break;
 		} else {
 			/*
@@ -179,7 +179,7 @@ RFC2616_Ttl(struct busyobj *bo)
 	/* calculated TTL, Our time, Date, Expires, max-age, age */
 	VSLb(bo->vsl, SLT_TTL,
 	    "RFC %.0f %.0f %.0f %.0f %.0f %.0f %.0f %u",
-	    expp->ttl, -1., -1., bo->t_fetch,
+	    expp->ttl, -1., -1., now,
 	    expp->t_origin, h_date, h_expires, max_age);
 }
 



From martin at varnish-software.com  Mon Apr 14 16:03:41 2014
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Mon, 14 Apr 2014 18:03:41 +0200
Subject: [master] 0a02a39 Add Date and Server headers also on backend error
 objects
Message-ID: <E1WZjMD-0006Oc-OX@project.varnish-software.com>


commit 0a02a3974f36929bef9212904ae2a1494bbba7c6
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Mon Apr 14 17:50:11 2014 +0200

    Add Date and Server headers also on backend error objects

diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index 2bb5feb..be664e5 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -653,11 +653,14 @@ vbf_stp_error(struct worker *wrk, struct busyobj *bo)
 {
 	struct storage *st;
 	ssize_t l;
+	double now;
+	char time_str[VTIM_FORMAT_SIZE];
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
 
-	VSLb_ts_busyobj(bo, "Error", W_TIM_real(wrk));
+	now = W_TIM_real(wrk);
+	VSLb_ts_busyobj(bo, "Error", now);
 
 	AN(bo->fetch_objcore->flags & OC_F_BUSY);
 
@@ -669,6 +672,9 @@ vbf_stp_error(struct worker *wrk, struct busyobj *bo)
 
 	HTTP_Setup(bo->beresp, bo->ws, bo->vsl, SLT_BerespMethod);
 	http_SetResp(bo->beresp, "HTTP/1.1", 503, "Backend fetch failed");
+	VTIM_format(now, time_str);
+	http_PrintfHeader(bo->beresp, "Date: %s", time_str);
+	http_SetHeader(bo->beresp, "Server: Varnish");
 
 	bo->exp.t_origin = bo->t_prev;
 	bo->exp.ttl = 0;



From martin at varnish-software.com  Mon Apr 14 16:03:41 2014
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Mon, 14 Apr 2014 18:03:41 +0200
Subject: [master] 1127242 Add test case to make sure we always emit Date
 headers
Message-ID: <E1WZjMD-0006Ol-Pq@project.varnish-software.com>


commit 1127242299c930330a389c6960bb27f254132b96
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Mon Apr 14 17:50:51 2014 +0200

    Add test case to make sure we always emit Date headers

diff --git a/bin/varnishtest/tests/c00066.vtc b/bin/varnishtest/tests/c00066.vtc
new file mode 100644
index 0000000..eeb1ff9
--- /dev/null
+++ b/bin/varnishtest/tests/c00066.vtc
@@ -0,0 +1,58 @@
+varnishtest "Check that we always deliver Date headers"
+
+server s1 {
+	rxreq
+	txresp
+} -start
+
+varnish v1 -vcl+backend {
+	backend bad { .host = "${bad_ip}"; }
+	sub vcl_recv {
+		if (req.url == "/synth") {
+			return (synth(200, "Synth test"));
+		}
+		if (req.url == "/error") {
+			set req.backend_hint = bad;
+		}
+	}
+	sub vcl_backend_response {
+		set beresp.do_stream = false;
+	}
+} -start
+
+varnish v1 -cliok "param.set connect_timeout 1"
+
+logexpect l1 -v v1 -g request {
+	expect	0 1001	Begin		"^req .* rxreq"
+	expect	* =	ReqURL		"/"
+	expect	* =	RespHeader	"^Date: "
+	expect	* =	End
+
+	expect	* 1003	Begin		"^req .* rxreq"
+	expect	* =	ReqURL		"/synth"
+	expect	* =	RespHeader	"^Date: "
+	expect	* =	End
+
+	expect	* 1004	Begin		"^req .* rxreq"
+	expect	* =	ReqURL		"/error"
+	expect	* =	RespHeader	"^Date: "
+	expect	* =	End
+} -start
+
+client c1 {
+	txreq -url "/"
+	rxresp
+	expect resp.status == 200
+	expect resp.msg == "Ok"
+
+	txreq -url "/synth"
+	rxresp
+	expect resp.status == 200
+	expect resp.msg == "Synth test"
+
+	txreq -url "/error"
+	rxresp
+	expect resp.status == 503
+} -run
+
+logexpect l1 -wait



From fgsch at lodoss.net  Wed Apr 16 16:04:31 2014
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Wed, 16 Apr 2014 18:04:31 +0200
Subject: [master] a9716fe Use bereq.url within vcl_backend_response
Message-ID: <E1WaSK7-0001PM-Nt@project.varnish-software.com>


commit a9716fe5ba673ed32f4eaeffd17ae65bbe4381e7
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Wed Apr 16 17:03:26 2014 +0100

    Use bereq.url within vcl_backend_response
    
    Pointed out by dharrigan in #varnish.

diff --git a/doc/sphinx/users-guide/increasing-your-hitrate.rst b/doc/sphinx/users-guide/increasing-your-hitrate.rst
index a413b71..231e27b 100644
--- a/doc/sphinx/users-guide/increasing-your-hitrate.rst
+++ b/doc/sphinx/users-guide/increasing-your-hitrate.rst
@@ -273,7 +273,7 @@ You need VCL to identify the objects you want and then you set the
 'beresp.ttl' to whatever you want::
 
     sub vcl_backend_response {
-        if (req.url ~ "^/legacy_broken_cms/") {
+        if (bereq.url ~ "^/legacy_broken_cms/") {
             set beresp.ttl = 5d;
         }
     }



From fgsch at lodoss.net  Thu Apr 17 22:22:45 2014
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Fri, 18 Apr 2014 00:22:45 +0200
Subject: [master] 5e1586a Fix typo in error message
Message-ID: <E1Wauhh-0004vc-N4@project.varnish-software.com>


commit 5e1586aa4c3f9bf346cac63a018ccc8d30ee0618
Author: Matt Robenolt <matt at ydekproductions.com>
Date:   Thu Apr 17 14:33:38 2014 -0700

    Fix typo in error message

diff --git a/lib/libvarnishtools/vut.c b/lib/libvarnishtools/vut.c
index f465b1f..d943d8f 100644
--- a/lib/libvarnishtools/vut.c
+++ b/lib/libvarnishtools/vut.c
@@ -304,7 +304,7 @@ VUT_Main(void)
 			VUT.vslq = VSLQ_New(VUT.vsl, &c, VUT.g_arg, VUT.q_arg);
 			AN(VUT.vslq);
 			AZ(c);
-			VUT_Error(0, "Log reaquired");
+			VUT_Error(0, "Log reacquired");
 		}
 
 		i = VSLQ_Dispatch(VUT.vslq, VUT.dispatch_f, VUT.dispatch_priv);



From apj at mutt.dk  Mon Apr 21 20:11:33 2014
From: apj at mutt.dk (Andreas Plesner)
Date: Mon, 21 Apr 2014 22:11:33 +0200
Subject: [master] e7340a6 Add more changes. Fixes #1480
Message-ID: <E1WcKYv-00061l-90@project.varnish-software.com>


commit e7340a68ecf908968f4ffbb4ca4be24347348aae
Author: Andreas Plesner <apj at mutt.dk>
Date:   Mon Apr 21 22:11:16 2014 +0200

    Add more changes.
    Fixes #1480

diff --git a/doc/sphinx/whats-new/upgrading.rst b/doc/sphinx/whats-new/upgrading.rst
index 82298a2..788764a 100644
--- a/doc/sphinx/whats-new/upgrading.rst
+++ b/doc/sphinx/whats-new/upgrading.rst
@@ -19,6 +19,12 @@ VCL version number::
 
     vcl 4.0;
 
+req.request is now req.method
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+To align better with RFC naming, `req.request` has been renamed to
+`req.method`.
+
 vcl_fetch is now vcl_backend_response
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -57,6 +63,13 @@ has been removed, and you should use the hash director directly::
         set req.backend_hint = h.backend(client.identity);
     }
 
+vcl_error is now vcl_backend_error
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+To make a distinction between internally generated errors and
+VCL synthetic responses, `vcl_backend_error` will be called when 
+varnish encounters an error when trying to fetch an object.
+
 error() is now synth()
 ~~~~~~~~~~~~~~~~~~~~~~
 
@@ -79,6 +92,13 @@ The synthetic keyword is now a function::
         return (deliver);
     }
 
+obj in vcl_error replaced by beresp in vcl_backend_error
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+To better represent a the context in which it is called, you
+should now use `beresp.*` vcl_backend_error, where you used to
+use `obj.*` in `vcl_error`.
+
 hit_for_pass objects are created using beresp.uncacheable
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -92,9 +112,6 @@ Example::
         }
     }
 
-vcl_recv should return(hash) instead of lookup now
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
 req.* not available in vcl_backend_response
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -110,12 +127,12 @@ is reserved for builtin subs.
 req.backend.healthy replaced by std.healthy(req.backend)
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-client.port replaced by std.port(client.ip)
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+client.port, and server.port replaced by respectively std.port(client.ip) and std.port(server.ip)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-The `client.ip` is now a proper type, which renders as an IP address by
-default. You need to use the `std.port()` function to get the port
-number.
+`client.ip` and `server.ip` are now proper datatypes, which renders
+as an IP address by default. You need to use the `std.port()`
+function to get the port number.
 
 obj is now read-only
 ~~~~~~~~~~~~~~~~~~~~
@@ -123,6 +140,15 @@ obj is now read-only
 `obj` is now read-only. `obj.hits`, if enabled in VCL, now counts per
 objecthead, not per object. `obj.last_use` has been retired.
 
+Some return values have been replaced
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Apart from the new `synth` return value described above, the
+following has changed:
+
+ - `vcl_recv` must now return `hash` instead of `lookup`
+ - `vcl_hash` must now return `lookup` instead of `hash`
+ - `vcl_pass` must now return `fetch` instead of `pass`
 
 default/builtin VCL changes
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -139,6 +165,8 @@ The `remove` keyword is gone
 
 Replaced by `unset`.
 
+
+
 Changes to existing parameters
 ==============================
 
@@ -170,3 +198,14 @@ vcc_allow_inline_c
 
 You can now completely disable inline C in your VCL, and it is
 disabled by default.
+
+Other changes
+=============
+
+New log filtering
+~~~~~~~~~~~~~~~~~
+
+The logging framework has a new filtering language, which means
+that the -m switch has been replaced with a new -q switch.
+See :ref:`ref-vsl-query` for more information about the new
+query language.



From phk at FreeBSD.org  Tue Apr 22 08:17:28 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 22 Apr 2014 10:17:28 +0200
Subject: [master] 203eafe Some stylistic consistency patches from Dridi
 Boukelmoune who made them with Coccinelle.
Message-ID: <E1WcVtQ-0002zT-Eb@project.varnish-software.com>


commit 203eafef5b9c9f03dd29826128e4800a17e28d39
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Apr 22 08:16:41 2014 +0000

    Some stylistic consistency patches from Dridi Boukelmoune
    who made them with Coccinelle.
    
    Thanks!

diff --git a/bin/varnishd/cache/cache_cli.c b/bin/varnishd/cache/cache_cli.c
index ca9da16..99b781c 100644
--- a/bin/varnishd/cache/cache_cli.c
+++ b/bin/varnishd/cache/cache_cli.c
@@ -214,7 +214,7 @@ ccf_panic(struct cli *cli, const char * const *av, void *priv)
 	(void)cli;
 	(void)av;
 	(void)priv;
-	assert(!strcmp("", "You asked for it"));
+	AZ(strcmp("", "You asked for it"));
 }
 
 /*--------------------------------------------------------------------*/
diff --git a/bin/varnishd/cache/cache_esi_deliver.c b/bin/varnishd/cache/cache_esi_deliver.c
index 5aa85ee..a246985 100644
--- a/bin/varnishd/cache/cache_esi_deliver.c
+++ b/bin/varnishd/cache/cache_esi_deliver.c
@@ -302,7 +302,7 @@ ESI_Deliver(struct req *req)
 		i = VGZ_Gunzip(vgz, &dp, &dl);
 		assert(i == VGZ_OK);
 		assert(VGZ_IbufEmpty(vgz));
-		assert(dl == 0);
+		AZ(dl);
 	}
 
 	st = VTAILQ_FIRST(&req->obj->store);
diff --git a/bin/varnishd/cache/cache_expire.c b/bin/varnishd/cache/cache_expire.c
index 8bd0707..b5ecfbe 100644
--- a/bin/varnishd/cache/cache_expire.c
+++ b/bin/varnishd/cache/cache_expire.c
@@ -98,7 +98,7 @@ exp_when(const struct object *o)
 	CHECK_OBJ_NOTNULL(o, OBJECT_MAGIC);
 
 	when = o->exp.t_origin + o->exp.ttl + o->exp.grace + o->exp.keep;
-	assert(!isnan(when));
+	AZ(isnan(when));
 	return (when);
 }
 
diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index be664e5..5a0a9ff 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -801,7 +801,7 @@ vbf_fetch_thread(struct worker *wrk, void *priv)
 	http_Teardown(bo->beresp);
 
 	if (bo->state == BOS_FINISHED) {
-		assert(!(bo->fetch_objcore->flags & OC_F_FAILED));
+		AZ(bo->fetch_objcore->flags & OC_F_FAILED);
 		HSH_Complete(bo->fetch_objcore);
 		VSLb(bo->vsl, SLT_Length, "%zd", bo->fetch_obj->len);
 		{
diff --git a/bin/varnishd/cache/cache_fetch_proc.c b/bin/varnishd/cache/cache_fetch_proc.c
index 8861e1d..429052d 100644
--- a/bin/varnishd/cache/cache_fetch_proc.c
+++ b/bin/varnishd/cache/cache_fetch_proc.c
@@ -230,7 +230,7 @@ VFP_Fetch_Body(struct busyobj *bo, ssize_t est)
 		AZ(bo->failed);
 		vfps = VFP_Suck(bo, st->ptr + st->len, &l);
 		if (l > 0 && vfps != VFP_ERROR) {
-			assert(!VTAILQ_EMPTY(&bo->fetch_obj->store));
+			AZ(VTAILQ_EMPTY(&bo->fetch_obj->store));
 			VBO_extend(bo, l);
 		}
 		if (st->len == st->space)
diff --git a/bin/varnishd/cache/cache_http1_fsm.c b/bin/varnishd/cache/cache_http1_fsm.c
index 9bb5e79..3702827 100644
--- a/bin/varnishd/cache/cache_http1_fsm.c
+++ b/bin/varnishd/cache/cache_http1_fsm.c
@@ -103,7 +103,7 @@ http1_wait(struct sess *sp, struct worker *wrk, struct req *req)
 	AZ(req->vcl);
 	AZ(req->obj);
 	AZ(req->esi_level);
-	assert(!isnan(sp->t_idle));
+	AZ(isnan(sp->t_idle));
 	assert(isnan(req->t_first));
 	assert(isnan(req->t_prev));
 	assert(isnan(req->t_req));
diff --git a/bin/varnishd/cache/cache_lck.c b/bin/varnishd/cache/cache_lck.c
index 54203af..288d452 100644
--- a/bin/varnishd/cache/cache_lck.c
+++ b/bin/varnishd/cache/cache_lck.c
@@ -134,8 +134,8 @@ Lck__Assert(const struct lock *lck, int held)
 		assert(ilck->held);
 		assert(pthread_equal(ilck->owner, pthread_self()));
 	} else {
-		assert(!ilck->held);
-		assert(!pthread_equal(ilck->owner, pthread_self()));
+		AZ(ilck->held);
+		AZ(pthread_equal(ilck->owner, pthread_self()));
 	}
 }
 
diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index b6d0500..2518422 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -690,9 +690,9 @@ cnt_recv(struct worker *wrk, struct req *req)
 	AZ(req->obj);
 	AZ(req->objcore);
 
-	assert(!isnan(req->t_first));
-	assert(!isnan(req->t_prev));
-	assert(!isnan(req->t_req));
+	AZ(isnan(req->t_first));
+	AZ(isnan(req->t_prev));
+	AZ(isnan(req->t_req));
 
 	VSLb(req->vsl, SLT_ReqStart, "%s %s",
 	    req->sp->client_addr_str, req->sp->client_port_str);
diff --git a/bin/varnishd/cache/cache_session.c b/bin/varnishd/cache/cache_session.c
index 0c010c1..c938b98 100644
--- a/bin/varnishd/cache/cache_session.c
+++ b/bin/varnishd/cache/cache_session.c
@@ -303,7 +303,7 @@ SES_Delete(struct sess *sp, enum sess_close reason, double now)
 
 	if (isnan(now))
 		now = VTIM_real();
-	assert(!isnan(sp->t_open));
+	AZ(isnan(sp->t_open));
 
 	VSL(SLT_SessClose, sp->vxid, "%s %.3f",
 	    sess_close_2str(sp->reason, 0), now - sp->t_open);
diff --git a/bin/varnishd/cache/cache_shmlog.c b/bin/varnishd/cache/cache_shmlog.c
index af78bfa..d0733d7 100644
--- a/bin/varnishd/cache/cache_shmlog.c
+++ b/bin/varnishd/cache/cache_shmlog.c
@@ -78,7 +78,7 @@ static inline uint32_t *
 vsl_hdr(enum VSL_tag_e tag, uint32_t *p, unsigned len, uint32_t vxid)
 {
 
-	assert(((uintptr_t)p & 0x3) == 0);
+	AZ((uintptr_t)p & 0x3);
 	assert(tag > SLT__Bogus);
 	assert(tag < SLT__Reserved);
 	AZ(len & ~VSL_LENMASK);
@@ -133,7 +133,7 @@ vsl_get(unsigned len, unsigned records, unsigned flushes)
 		AZ(err);
 	}
 	assert(vsl_ptr < vsl_end);
-	assert(((uintptr_t)vsl_ptr & 0x3) == 0);
+	AZ((uintptr_t)vsl_ptr & 0x3);
 
 	VSC_C_main->shm_writes++;
 	VSC_C_main->shm_flushes += flushes;
@@ -146,7 +146,7 @@ vsl_get(unsigned len, unsigned records, unsigned flushes)
 	p = vsl_ptr;
 	vsl_ptr = VSL_END(vsl_ptr, len);
 	assert(vsl_ptr < vsl_end);
-	assert(((uintptr_t)vsl_ptr & 0x3) == 0);
+	AZ((uintptr_t)vsl_ptr & 0x3);
 
 	*vsl_ptr = VSL_ENDMARKER;
 
diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index 0018095..e6dd179 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -250,7 +250,7 @@ VCL_Nuke(struct vcls *vcl)
 	ASSERT_CLI();
 	assert(vcl != vcl_active);
 	assert(vcl->conf->discard);
-	assert(vcl->conf->busy == 0);
+	AZ(vcl->conf->busy);
 	VTAILQ_REMOVE(&vcl_head, vcl, list);
 	ctx.method = VCL_MET_FINI;
 	ctx.handling = &hand;
@@ -438,7 +438,7 @@ VCL_##func##_method(struct VCL_conf *vcl, struct worker *wrk,		\
 	vcl_call_method(wrk, req, bo, ws, VCL_MET_ ## upper,		\
 	    vcl->func##_func);						\
 	assert((1U << wrk->handling) & bitmap);				\
-	assert(!((1U << wrk->handling) & ~bitmap));			\
+	AZ((1U << wrk->handling) & ~bitmap);			\
 }
 
 #include "tbl/vcl_returns.h"
diff --git a/bin/varnishd/cache/cache_wrw.c b/bin/varnishd/cache/cache_wrw.c
index a418ea0..09c7e26 100644
--- a/bin/varnishd/cache/cache_wrw.c
+++ b/bin/varnishd/cache/cache_wrw.c
@@ -137,7 +137,7 @@ wrw_prune(struct wrw *wrw, ssize_t bytes)
 		}
 		used += wrw->iov[j].iov_len;
 	}
-	assert(wrw->liov == 0);
+	AZ(wrw->liov);
 }
 
 unsigned
diff --git a/bin/varnishd/hash/hash_critbit.c b/bin/varnishd/hash/hash_critbit.c
index 46f7f95..4e14bdd 100644
--- a/bin/varnishd/hash/hash_critbit.c
+++ b/bin/varnishd/hash/hash_critbit.c
@@ -73,7 +73,7 @@ hcb_build_bittbl(void)
 
 	/* Quick asserts for sanity check */
 	assert(hcb_bits(0x34, 0x34) == 8);
-	assert(hcb_bits(0xaa, 0x55) == 0);
+	AZ(hcb_bits(0xaa, 0x55));
 	assert(hcb_bits(0x01, 0x22) == 2);
 	assert(hcb_bits(0x10, 0x0b) == 3);
 }
@@ -132,7 +132,7 @@ static uintptr_t
 hcb_r_node(struct objhead *n)
 {
 
-	assert(!((uintptr_t)n & (HCB_BIT_NODE|HCB_BIT_Y)));
+	AZ((uintptr_t)n & (HCB_BIT_NODE | HCB_BIT_Y));
 	return (HCB_BIT_NODE | (uintptr_t)n);
 }
 
@@ -141,7 +141,7 @@ hcb_l_node(uintptr_t u)
 {
 
 	assert(u & HCB_BIT_NODE);
-	assert(!(u & HCB_BIT_Y));
+	AZ(u & HCB_BIT_Y);
 	return ((struct objhead *)(u & ~HCB_BIT_NODE));
 }
 
@@ -150,7 +150,7 @@ hcb_r_y(struct hcb_y *y)
 {
 
 	CHECK_OBJ_NOTNULL(y, HCB_Y_MAGIC);
-	assert(!((uintptr_t)y & (HCB_BIT_NODE|HCB_BIT_Y)));
+	AZ((uintptr_t)y & (HCB_BIT_NODE | HCB_BIT_Y));
 	return (HCB_BIT_Y | (uintptr_t)y);
 }
 
@@ -158,7 +158,7 @@ static struct hcb_y *
 hcb_l_y(uintptr_t u)
 {
 
-	assert(!(u & HCB_BIT_NODE));
+	AZ(u & HCB_BIT_NODE);
 	assert(u & HCB_BIT_Y);
 	return ((struct hcb_y *)(u & ~HCB_BIT_Y));
 }
@@ -252,7 +252,7 @@ hcb_insert(struct worker *wrk, struct hcb_root *root, const uint8_t *digest,
 	s2 = 1-s2;
 
 	p = &root->origo;
-	assert(*p != 0);
+	AN(*p);
 
 	while(hcb_is_y(*p)) {
 		y = hcb_l_y(*p);
diff --git a/bin/varnishd/mgt/mgt_cli.c b/bin/varnishd/mgt/mgt_cli.c
index f977adb..3d71a14 100644
--- a/bin/varnishd/mgt/mgt_cli.c
+++ b/bin/varnishd/mgt/mgt_cli.c
@@ -116,7 +116,7 @@ mcf_panic(struct cli *cli, const char * const *av, void *priv)
 	(void)cli;
 	(void)av;
 	(void)priv;
-	assert(!strcmp("", "You asked for it"));
+	AZ(strcmp("", "You asked for it"));
 }
 
 static struct cli_proto cli_debug[] = {
diff --git a/bin/varnishd/mgt/mgt_main.c b/bin/varnishd/mgt/mgt_main.c
index 5dc602e..76b702b 100644
--- a/bin/varnishd/mgt/mgt_main.c
+++ b/bin/varnishd/mgt/mgt_main.c
@@ -300,7 +300,7 @@ cli_stdin_close(void *priv)
 	(void)close(0);
 	(void)close(1);
 	(void)close(2);
-	assert(open("/dev/null", O_RDONLY) == 0);
+	AZ(open("/dev/null", O_RDONLY));
 	assert(open("/dev/null", O_WRONLY) == 1);
 	assert(open("/dev/null", O_WRONLY) == 2);
 
diff --git a/bin/varnishd/storage/stevedore.c b/bin/varnishd/storage/stevedore.c
index 30ae453..5e54ed6 100644
--- a/bin/varnishd/storage/stevedore.c
+++ b/bin/varnishd/storage/stevedore.c
@@ -183,8 +183,7 @@ stv_alloc(struct stevedore *stv, size_t size)
 
 		size >>= 1;
 	}
-	if (st != NULL)
-		CHECK_OBJ_NOTNULL(st, STORAGE_MAGIC);
+	CHECK_OBJ_ORNULL(st, STORAGE_MAGIC);
 	return (st);
 }
 
@@ -226,8 +225,7 @@ stv_alloc_obj(struct busyobj *bo, size_t size)
 		    EXP_NukeOne(bo, stv->lru) == -1)
 			break;
 	}
-	if (st != NULL)
-		CHECK_OBJ_NOTNULL(st, STORAGE_MAGIC);
+	CHECK_OBJ_ORNULL(st, STORAGE_MAGIC);
 	return (st);
 }
 
diff --git a/bin/varnishd/storage/stevedore_utils.c b/bin/varnishd/storage/stevedore_utils.c
index 081b93a..a86d57e 100644
--- a/bin/varnishd/storage/stevedore_utils.c
+++ b/bin/varnishd/storage/stevedore_utils.c
@@ -159,7 +159,7 @@ stv_fsspace(int fd, unsigned *bs)
 	/* We use units of the larger of filesystem blocksize and pagesize */
 	if (*bs < bsize)
 		*bs = bsize;
-	xxxassert(*bs % bsize == 0);
+	XXXAZ(*bs % bsize);
 	return (bsize * bavail);
 }
 
@@ -188,7 +188,7 @@ STV_FileSize(int fd, const char *size, unsigned *granularity, const char *ctx)
 
 	bs = *granularity;
 	fssize = stv_fsspace(fd, &bs);
-	xxxassert(bs % *granularity == 0);
+	XXXAZ(bs % *granularity);
 
 	if ((size == NULL || *size == '\0') && st.st_size != 0) {
 		/*
diff --git a/bin/varnishd/storage/storage_file.c b/bin/varnishd/storage/storage_file.c
index ac0b6c4..c48200c 100644
--- a/bin/varnishd/storage/storage_file.c
+++ b/bin/varnishd/storage/storage_file.c
@@ -168,7 +168,7 @@ insfree(struct smf_sc *sc, struct smf *sp)
 	struct smf *sp2;
 	size_t ns;
 
-	assert(sp->alloc == 0);
+	AZ(sp->alloc);
 	assert(sp->flist == NULL);
 	Lck_AssertHeld(&sc->mtx);
 	b = sp->size / sc->pagesize;
@@ -182,7 +182,7 @@ insfree(struct smf_sc *sc, struct smf *sp)
 	ns = b * sc->pagesize;
 	VTAILQ_FOREACH(sp2, sp->flist, status) {
 		assert(sp2->size >= ns);
-		assert(sp2->alloc == 0);
+		AZ(sp2->alloc);
 		assert(sp2->flist == sp->flist);
 		if (sp->offset < sp2->offset)
 			break;
@@ -198,7 +198,7 @@ remfree(const struct smf_sc *sc, struct smf *sp)
 {
 	size_t b;
 
-	assert(sp->alloc == 0);
+	AZ(sp->alloc);
 	assert(sp->flist != NULL);
 	Lck_AssertHeld(&sc->mtx);
 	b = sp->size / sc->pagesize;
@@ -223,7 +223,7 @@ alloc_smf(struct smf_sc *sc, size_t bytes)
 	struct smf *sp, *sp2;
 	size_t b;
 
-	assert(!(bytes % sc->pagesize));
+	AZ(bytes % sc->pagesize);
 	b = bytes / sc->pagesize;
 	if (b >= NBUCKET)
 		b = NBUCKET - 1;
@@ -280,9 +280,9 @@ free_smf(struct smf *sp)
 	struct smf_sc *sc = sp->sc;
 
 	CHECK_OBJ_NOTNULL(sp, SMF_MAGIC);
-	assert(sp->alloc != 0);
+	AN(sp->alloc);
 	assert(sp->size > 0);
-	assert(!(sp->size % sc->pagesize));
+	AZ(sp->size % sc->pagesize);
 	VTAILQ_REMOVE(&sc->used, sp, status);
 	sp->alloc = 0;
 
@@ -324,11 +324,11 @@ trim_smf(struct smf *sp, size_t bytes)
 	struct smf *sp2;
 	struct smf_sc *sc = sp->sc;
 
-	assert(sp->alloc != 0);
+	AN(sp->alloc);
 	assert(bytes > 0);
 	assert(bytes < sp->size);
-	assert(!(bytes % sc->pagesize));
-	assert(!(sp->size % sc->pagesize));
+	AZ(bytes % sc->pagesize);
+	AZ(sp->size % sc->pagesize);
 	CHECK_OBJ_NOTNULL(sp, SMF_MAGIC);
 	sp2 = malloc(sizeof *sp2);
 	XXXAN(sp2);
@@ -353,7 +353,7 @@ new_smf(struct smf_sc *sc, unsigned char *ptr, off_t off, size_t len)
 {
 	struct smf *sp, *sp2;
 
-	assert(!(len % sc->pagesize));
+	AZ(len % sc->pagesize);
 	sp = calloc(sizeof *sp, 1);
 	XXXAN(sp);
 	sp->magic = SMF_MAGIC;
@@ -395,8 +395,8 @@ smf_open_chunk(struct smf_sc *sc, off_t sz, off_t off, off_t *fail, off_t *sum)
 	void *p;
 	off_t h;
 
-	assert(sz != 0);
-	assert(!(sz % sc->pagesize));
+	AN(sz);
+	AZ(sz % sc->pagesize);
 
 	if (*fail < (uintmax_t)sc->pagesize * MINPAGES)
 		return;
diff --git a/bin/varnishd/waiter/cache_waiter_epoll.c b/bin/varnishd/waiter/cache_waiter_epoll.c
index ba41210..ea875e7 100644
--- a/bin/varnishd/waiter/cache_waiter_epoll.c
+++ b/bin/varnishd/waiter/cache_waiter_epoll.c
@@ -124,7 +124,7 @@ vwe_eev(struct vwe *vwe, const struct epoll_event *ep, double now)
 				j++;
 				i -= sizeof ss[0];
 			}
-			assert(i == 0);
+			AZ(i);
 		}
 	} else {
 		CAST_OBJ_NOTNULL(sp, ep->data.ptr, SESS_MAGIC);
diff --git a/bin/varnishd/waiter/cache_waiter_kqueue.c b/bin/varnishd/waiter/cache_waiter_kqueue.c
index fb82e99..68f7fba 100644
--- a/bin/varnishd/waiter/cache_waiter_kqueue.c
+++ b/bin/varnishd/waiter/cache_waiter_kqueue.c
@@ -70,7 +70,7 @@ vwk_kq_flush(struct vwk *vwk)
 	if (vwk->nki == 0)
 		return;
 	i = kevent(vwk->kq, vwk->ki, vwk->nki, NULL, 0, NULL);
-	assert(i == 0);
+	AZ(i);
 	vwk->nki = 0;
 }
 
@@ -108,7 +108,7 @@ vwk_pipe_ev(struct vwk *vwk, const struct kevent *kp)
 		j++;
 		i -= sizeof ss[0];
 	}
-	assert(i == 0);
+	AZ(i);
 }
 
 /*--------------------------------------------------------------------*/
diff --git a/bin/varnishd/waiter/cache_waiter_poll.c b/bin/varnishd/waiter/cache_waiter_poll.c
index d2c29d1..de8fd58 100644
--- a/bin/varnishd/waiter/cache_waiter_poll.c
+++ b/bin/varnishd/waiter/cache_waiter_poll.c
@@ -94,8 +94,8 @@ vwp_poll(struct vwp *vwp, int fd)
 		vwp->hpoll = fd;
 
 	assert(vwp->pollfd[fd].fd == -1);
-	assert(vwp->pollfd[fd].events == 0);
-	assert(vwp->pollfd[fd].revents == 0);
+	AZ(vwp->pollfd[fd].events);
+	AZ(vwp->pollfd[fd].revents);
 
 	vwp->pollfd[fd].fd = fd;
 	vwp->pollfd[fd].events = POLLIN;
@@ -111,7 +111,7 @@ vwp_unpoll(struct vwp *vwp, int fd)
 
 	assert(vwp->pollfd[fd].fd == fd);
 	assert(vwp->pollfd[fd].events == POLLIN);
-	assert(vwp->pollfd[fd].revents == 0);
+	AZ(vwp->pollfd[fd].revents);
 
 	vwp->pollfd[fd].fd = -1;
 	vwp->pollfd[fd].events = 0;
@@ -177,7 +177,7 @@ vwp_main(void *priv)
 			v2--;
 			i = read(vwp->pipes[0], ss, sizeof ss);
 			assert(i >= 0);
-			assert(((unsigned)i % sizeof ss[0]) == 0);
+			AZ((unsigned)i % sizeof ss[0]);
 			for (j = 0; j * sizeof ss[0] < i; j++) {
 				CHECK_OBJ_NOTNULL(ss[j], SESS_MAGIC);
 				assert(ss[j]->fd >= 0);
@@ -185,7 +185,7 @@ vwp_main(void *priv)
 				vwp_poll(vwp, ss[j]->fd);
 			}
 		}
-		assert(v2 == 0);
+		AZ(v2);
 	}
 	NEEDLESS_RETURN(NULL);
 }
diff --git a/bin/varnishreplay/varnishreplay.c b/bin/varnishreplay/varnishreplay.c
index d8f2877..7fc24fe 100644
--- a/bin/varnishreplay/varnishreplay.c
+++ b/bin/varnishreplay/varnishreplay.c
@@ -246,7 +246,7 @@ static struct replay_thread *
 thread_get(int fd, void *(*thread_main)(void *))
 {
 
-	assert(fd != 0);
+	AN(fd);
 	if (fd >= nthreads) {
 		struct replay_thread **newthreads = threads;
 		size_t newnthreads = nthreads;
diff --git a/bin/varnishstat/varnishstat.c b/bin/varnishstat/varnishstat.c
index 7b37f0b..6fa7cc7 100644
--- a/bin/varnishstat/varnishstat.c
+++ b/bin/varnishstat/varnishstat.c
@@ -55,7 +55,7 @@ do_xml_cb(void *priv, const struct VSC_point * const pt)
 	(void)priv;
 	if (pt == NULL)
 		return (0);
-	assert(!strcmp(pt->desc->fmt, "uint64_t"));
+	AZ(strcmp(pt->desc->fmt, "uint64_t"));
 	val = *(const volatile uint64_t*)pt->ptr;
 	sec = pt->section;
 
@@ -100,7 +100,7 @@ do_json_cb(void *priv, const struct VSC_point * const pt)
 		return (0);
 
 	jp = priv;
-	assert(!strcmp(pt->desc->fmt, "uint64_t"));
+	AZ(strcmp(pt->desc->fmt, "uint64_t"));
 	val = *(const volatile uint64_t*)pt->ptr;
 	sec = pt->section;
 
@@ -167,7 +167,7 @@ do_once_cb(void *priv, const struct VSC_point * const pt)
 	if (pt == NULL)
 		return (0);
 	op = priv;
-	assert(!strcmp(pt->desc->fmt, "uint64_t"));
+	AZ(strcmp(pt->desc->fmt, "uint64_t"));
 	val = *(const volatile uint64_t*)pt->ptr;
 	sec = pt->section;
 	i = 0;
diff --git a/bin/varnishstat/varnishstat_curses.c b/bin/varnishstat/varnishstat_curses.c
index 356a92b..373cf09 100644
--- a/bin/varnishstat/varnishstat_curses.c
+++ b/bin/varnishstat/varnishstat_curses.c
@@ -264,7 +264,7 @@ build_pt_list_cb(void *priv, const struct VSC_point *vpt)
 
 	CAST_OBJ_NOTNULL(pt_priv, priv, PT_PRIV_MAGIC);
 
-	assert(!strcmp(vpt->desc->fmt, "uint64_t"));
+	AZ(strcmp(vpt->desc->fmt, "uint64_t"));
 	snprintf(buf, sizeof buf, "%s.%s.%s", vpt->section->type,
 	    vpt->section->ident, vpt->desc->name);
 	buf[sizeof buf - 1] = '\0';
diff --git a/bin/varnishtest/vtc.c b/bin/varnishtest/vtc.c
index 470c931..a31c83f 100644
--- a/bin/varnishtest/vtc.c
+++ b/bin/varnishtest/vtc.c
@@ -390,7 +390,7 @@ cmd_varnishtest(CMD_ARGS)
 
 	if (av == NULL)
 		return;
-	assert(!strcmp(av[0], "varnishtest"));
+	AZ(strcmp(av[0], "varnishtest"));
 
 	vtc_log(vl, 1, "TEST %s", av[1]);
 	AZ(av[2]);
@@ -414,7 +414,7 @@ cmd_shell(CMD_ARGS)
 	AZ(av[2]);
 	vtc_dump(vl, 4, "shell", av[1], -1);
 	r = system(av[1]);
-	assert(WEXITSTATUS(r) == 0);
+	AZ(WEXITSTATUS(r));
 }
 
 /**********************************************************************
diff --git a/bin/varnishtest/vtc_client.c b/bin/varnishtest/vtc_client.c
index 7eb430f..a273776 100644
--- a/bin/varnishtest/vtc_client.c
+++ b/bin/varnishtest/vtc_client.c
@@ -217,7 +217,7 @@ cmd_client(CMD_ARGS)
 		return;
 	}
 
-	assert(!strcmp(av[0], "client"));
+	AZ(strcmp(av[0], "client"));
 	av++;
 
 	VTAILQ_FOREACH(c, &clients, list)
diff --git a/bin/varnishtest/vtc_http.c b/bin/varnishtest/vtc_http.c
index 4388f73..17dba6b 100644
--- a/bin/varnishtest/vtc_http.c
+++ b/bin/varnishtest/vtc_http.c
@@ -225,7 +225,7 @@ cmd_http_expect(CMD_ARGS)
 	(void)cmd;
 	(void)vl;
 	CAST_OBJ_NOTNULL(hp, priv, HTTP_MAGIC);
-	assert(!strcmp(av[0], "expect"));
+	AZ(strcmp(av[0], "expect"));
 	av++;
 
 	AN(av[0]);
@@ -298,13 +298,13 @@ http_splitheader(struct http *hp, int req)
 	hh[n++] = p;
 	while (!vct_islws(*p))
 		p++;
-	assert(!vct_iscrlf(p));
+	AZ(vct_iscrlf(p));
 	*p++ = '\0';
 
 	/* URL/STATUS */
 	while (vct_issp(*p))		/* XXX: H space only */
 		p++;
-	assert(!vct_iscrlf(p));
+	AZ(vct_iscrlf(p));
 	hh[n++] = p;
 	while (!vct_islws(*p))
 		p++;
@@ -524,7 +524,7 @@ cmd_http_rxresp(CMD_ARGS)
 	(void)vl;
 	CAST_OBJ_NOTNULL(hp, priv, HTTP_MAGIC);
 	ONLY_CLIENT(hp, av);
-	assert(!strcmp(av[0], "rxresp"));
+	AZ(strcmp(av[0], "rxresp"));
 	av++;
 
 	for(; *av != NULL; av++)
@@ -554,7 +554,7 @@ cmd_http_rxresphdrs(CMD_ARGS)
 	(void)vl;
 	CAST_OBJ_NOTNULL(hp, priv, HTTP_MAGIC);
 	ONLY_CLIENT(hp, av);
-	assert(!strcmp(av[0], "rxresphdrs"));
+	AZ(strcmp(av[0], "rxresphdrs"));
 	av++;
 
 	for(; *av != NULL; av++)
@@ -767,7 +767,7 @@ cmd_http_txresp(CMD_ARGS)
 	(void)vl;
 	CAST_OBJ_NOTNULL(hp, priv, HTTP_MAGIC);
 	ONLY_SERVER(hp, av);
-	assert(!strcmp(av[0], "txresp"));
+	AZ(strcmp(av[0], "txresp"));
 	av++;
 
 	VSB_clear(hp->vsb);
@@ -812,7 +812,7 @@ cmd_http_rxreq(CMD_ARGS)
 	(void)vl;
 	CAST_OBJ_NOTNULL(hp, priv, HTTP_MAGIC);
 	ONLY_SERVER(hp, av);
-	assert(!strcmp(av[0], "rxreq"));
+	AZ(strcmp(av[0], "rxreq"));
 	av++;
 
 	for(; *av != NULL; av++)
@@ -832,7 +832,7 @@ cmd_http_rxreqhdrs(CMD_ARGS)
 	(void)cmd;
 	(void)vl;
 	CAST_OBJ_NOTNULL(hp, priv, HTTP_MAGIC);
-	assert(!strcmp(av[0], "rxreqhdrs"));
+	AZ(strcmp(av[0], "rxreqhdrs"));
 	av++;
 
 	for(; *av != NULL; av++)
@@ -850,7 +850,7 @@ cmd_http_rxbody(CMD_ARGS)
 	(void)vl;
 	CAST_OBJ_NOTNULL(hp, priv, HTTP_MAGIC);
 	ONLY_SERVER(hp, av);
-	assert(!strcmp(av[0], "rxbody"));
+	AZ(strcmp(av[0], "rxbody"));
 	av++;
 
 	for(; *av != NULL; av++)
@@ -895,7 +895,7 @@ cmd_http_txreq(CMD_ARGS)
 	(void)vl;
 	CAST_OBJ_NOTNULL(hp, priv, HTTP_MAGIC);
 	ONLY_CLIENT(hp, av);
-	assert(!strcmp(av[0], "txreq"));
+	AZ(strcmp(av[0], "txreq"));
 	av++;
 
 	VSB_clear(hp->vsb);
diff --git a/bin/varnishtest/vtc_logexp.c b/bin/varnishtest/vtc_logexp.c
index e6adae7..48c8f3d 100644
--- a/bin/varnishtest/vtc_logexp.c
+++ b/bin/varnishtest/vtc_logexp.c
@@ -489,7 +489,7 @@ cmd_logexp(CMD_ARGS)
 		return;
 	}
 
-	assert(!strcmp(av[0], "logexpect"));
+	AZ(strcmp(av[0], "logexpect"));
 	av++;
 
 	VTAILQ_FOREACH(le, &logexps, list) {
diff --git a/bin/varnishtest/vtc_sema.c b/bin/varnishtest/vtc_sema.c
index fd4c561..2ebfdee 100644
--- a/bin/varnishtest/vtc_sema.c
+++ b/bin/varnishtest/vtc_sema.c
@@ -129,7 +129,7 @@ cmd_sema(CMD_ARGS)
 		return;
 	}
 
-	assert(!strcmp(av[0], "sema"));
+	AZ(strcmp(av[0], "sema"));
 	av++;
 
 	AZ(pthread_mutex_lock(&sema_mtx));
diff --git a/bin/varnishtest/vtc_server.c b/bin/varnishtest/vtc_server.c
index eaa680f..86cf293 100644
--- a/bin/varnishtest/vtc_server.c
+++ b/bin/varnishtest/vtc_server.c
@@ -251,7 +251,7 @@ cmd_server(CMD_ARGS)
 		return;
 	}
 
-	assert(!strcmp(av[0], "server"));
+	AZ(strcmp(av[0], "server"));
 	av++;
 
 	VTAILQ_FOREACH(s, &servers, list)
@@ -278,7 +278,7 @@ cmd_server(CMD_ARGS)
 		if (s->run)
 			server_wait(s);
 
-		assert(s->run == 0);
+		AZ(s->run);
 		if (!strcmp(*av, "-repeat")) {
 			s->repeat = atoi(av[1]);
 			av++;
diff --git a/bin/varnishtest/vtc_varnish.c b/bin/varnishtest/vtc_varnish.c
index 19f51dd..14b94be 100644
--- a/bin/varnishtest/vtc_varnish.c
+++ b/bin/varnishtest/vtc_varnish.c
@@ -104,7 +104,7 @@ varnish_ask_cli(const struct varnish *v, const char *cmd, char **repl)
 		    cmd, i, retval, r);
 		return ((enum VCLI_status_e)retval);
 	}
-	assert(i == 0);
+	AZ(i);
 	vtc_log(v->vl, 3, "CLI RX  %u", retval);
 	vtc_dump(v->vl, 4, "CLI RX", r, -1);
 	if (repl != NULL)
@@ -402,7 +402,7 @@ varnish_launch(struct varnish *v)
 	v->pid = fork();
 	assert(v->pid >= 0);
 	if (v->pid == 0) {
-		assert(dup2(v->fds[0], 0) == 0);
+		AZ(dup2(v->fds[0], 0));
 		assert(dup2(v->fds[3], 1) == 1);
 		assert(dup2(1, 2) == 2);
 		AZ(close(v->fds[0]));
@@ -754,7 +754,7 @@ do_stat_cb(void *priv, const struct VSC_point * const pt)
 	if (strcmp(pt->desc->name, p))
 		return (0);
 
-	assert(!strcmp(pt->desc->fmt, "uint64_t"));
+	AZ(strcmp(pt->desc->fmt, "uint64_t"));
 	sp->val = *(const volatile uint64_t*)pt->ptr;
 	return (1);
 }
@@ -843,7 +843,7 @@ cmd_varnish(CMD_ARGS)
 		return;
 	}
 
-	assert(!strcmp(av[0], "varnish"));
+	AZ(strcmp(av[0], "varnish"));
 	av++;
 
 	VTAILQ_FOREACH(v, &varnishes, list)
diff --git a/lib/libvarnish/binary_heap.c b/lib/libvarnish/binary_heap.c
index 33616b5..491974b 100644
--- a/lib/libvarnish/binary_heap.c
+++ b/lib/libvarnish/binary_heap.c
@@ -215,7 +215,7 @@ binheap_new(void *priv, binheap_cmp_t *cmp_f, binheap_update_t *update_f)
 
 	bh->page_size = (unsigned)getpagesize() / sizeof (void *);
 	bh->page_mask = bh->page_size - 1;
-	assert(!(bh->page_size & bh->page_mask));	/* power of two */
+	AZ(bh->page_size & bh->page_mask);	/* power of two */
 	for (u = 1; (1U << u) != bh->page_size; u++)
 		;
 	bh->page_shift = u;
@@ -350,7 +350,7 @@ chk(const struct binheap *bh)
 
 	for (u = 2; u < bh->next; u++) {
 		v = parent(bh, u);
-		assert(!bh->cmp(bh->priv, A(bh, u), A(bh, v)));
+		AZ(bh->cmp(bh->priv, A(bh, u), A(bh, v)));
 	}
 }
 #endif
@@ -603,7 +603,7 @@ main(int argc, char **argv)
 			v = random() % N;
 			if (ff[v] != NULL) {
 				CHECK_OBJ_NOTNULL(ff[v], FOO_MAGIC);
-				assert(ff[v]->idx != 0);
+				AN(ff[v]->idx);
 				if (ff[v]->key & 1) {
 					binheap_delete(bh, ff[v]->idx);
 					assert(ff[v]->idx == BINHEAP_NOIDX);
@@ -619,7 +619,7 @@ main(int argc, char **argv)
 				ff[v]->key = random() % R;
 				binheap_insert(bh, ff[v]);
 				CHECK_OBJ_NOTNULL(ff[v], FOO_MAGIC);
-				assert(ff[v]->idx != 0);
+				AN(ff[v]->idx);
 			}
 			if (0)
 				chk2(bh);
diff --git a/lib/libvarnish/cli_serve.c b/lib/libvarnish/cli_serve.c
index 3ea59d6..7b7d9d5 100644
--- a/lib/libvarnish/cli_serve.c
+++ b/lib/libvarnish/cli_serve.c
@@ -369,7 +369,7 @@ cls_vlu(void *priv, const char *p)
 		return (0);
 	} else {
 		AN(cfd->argv[cfd->last_idx]);
-		assert(!strcmp(cfd->argv[cfd->last_idx], "<<"));
+		AZ(strcmp(cfd->argv[cfd->last_idx], "<<"));
 		AN(cfd->argv[cfd->last_idx + 1]);
 		if (strcmp(p, cfd->argv[cfd->last_idx + 1])) {
 			VSB_cat(cfd->last_arg, p);
diff --git a/lib/libvarnish/vav.c b/lib/libvarnish/vav.c
index 24b16b7..715e43c 100644
--- a/lib/libvarnish/vav.c
+++ b/lib/libvarnish/vav.c
@@ -90,7 +90,7 @@ VAV_BackSlash(const char *s, char *res)
 		break;
 	case 'x':
 		if (1 == sscanf(s + 1, "x%02x", &u)) {
-			assert(!(u & ~0xff));
+			AZ(u & ~0xff);
 			c = u;	/*lint !e734 loss of precision */
 			r = 4;
 		}
diff --git a/lib/libvarnish/vev.c b/lib/libvarnish/vev.c
index 87d1582..6429e4b 100644
--- a/lib/libvarnish/vev.c
+++ b/lib/libvarnish/vev.c
@@ -257,7 +257,7 @@ vev_add(struct vev_base *evb, struct vev *e)
 		es = &vev_sigs[e->sig];
 		if (es->vev != NULL)
 			return (EBUSY);
-		assert(es->happened == 0);
+		AZ(es->happened);
 		es->vev = e;
 		es->vevb = evb;
 		es->sigact.sa_flags = e->sig_flags;
@@ -299,7 +299,7 @@ vev_add(struct vev_base *evb, struct vev *e)
 
 	if (e->sig > 0) {
 		assert(es != NULL);
-		assert(sigaction(e->sig, &es->sigact, NULL) == 0);
+		AZ(sigaction(e->sig, &es->sigact, NULL));
 	}
 
 	return (0);
@@ -320,7 +320,7 @@ vev_del(struct vev_base *evb, struct vev *e)
 
 	if (e->__binheap_idx != 0)
 		binheap_delete(evb->binheap, e->__binheap_idx);
-	assert(e->__binheap_idx == 0);
+	AZ(e->__binheap_idx);
 
 	if (e->fd >= 0) {
 		DBG(evb, "... pidx = %d\n", e->__poll_idx);
@@ -341,7 +341,7 @@ vev_del(struct vev_base *evb, struct vev *e)
 		es->vevb = NULL;
 		es->sigact.sa_flags = e->sig_flags;
 		es->sigact.sa_handler = SIG_DFL;
-		assert(sigaction(e->sig, &es->sigact, NULL) == 0);
+		AZ(sigaction(e->sig, &es->sigact, NULL));
 		es->happened = 0;
 	}
 
@@ -517,6 +517,6 @@ vev_schedule_one(struct vev_base *evb)
 			free(e);
 		}
 	}
-	assert(i == 0);
+	AZ(i);
 	return (1);
 }
diff --git a/lib/libvarnish/vfil.c b/lib/libvarnish/vfil.c
index 5a5f087..920a06f 100644
--- a/lib/libvarnish/vfil.c
+++ b/lib/libvarnish/vfil.c
@@ -87,7 +87,7 @@ VFIL_readfd(int fd, ssize_t *sz)
 	char *f;
 	int i;
 
-	assert(0 == fstat(fd, &st));
+	AZ(fstat(fd, &st));
 	if (!S_ISREG(st.st_mode))
 		return (NULL);
 	f = malloc(st.st_size + 1);
diff --git a/lib/libvarnish/vsha256.c b/lib/libvarnish/vsha256.c
index a2208cf..6642d9f 100644
--- a/lib/libvarnish/vsha256.c
+++ b/lib/libvarnish/vsha256.c
@@ -329,6 +329,6 @@ SHA256_Test(void)
 		SHA256_Init(&c);
 		SHA256_Update(&c, p->input, strlen(p->input));
 		SHA256_Final(o, &c);
-		assert(!memcmp(o, p->output, 32));
+		AZ(memcmp(o, p->output, 32));
 	}
 }
diff --git a/lib/libvarnish/vtim.c b/lib/libvarnish/vtim.c
index 42c1558..c0efdbd 100644
--- a/lib/libvarnish/vtim.c
+++ b/lib/libvarnish/vtim.c
@@ -73,12 +73,12 @@ VTIM_mono(void)
 #elif  HAVE_CLOCK_GETTIME
 	struct timespec ts;
 
-	assert(clock_gettime(CLOCK_MONOTONIC, &ts) == 0);
+	AZ(clock_gettime(CLOCK_MONOTONIC, &ts));
 	return (ts.tv_sec + 1e-9 * ts.tv_nsec);
 #else
 	struct timeval tv;
 
-	assert(gettimeofday(&tv, NULL) == 0);
+	AZ(gettimeofday(&tv, NULL));
 	return (tv.tv_sec + 1e-6 * tv.tv_usec);
 #endif
 }
@@ -89,12 +89,12 @@ VTIM_real(void)
 #ifdef HAVE_CLOCK_GETTIME
 	struct timespec ts;
 
-	assert(clock_gettime(CLOCK_REALTIME, &ts) == 0);
+	AZ(clock_gettime(CLOCK_REALTIME, &ts));
 	return (ts.tv_sec + 1e-9 * ts.tv_nsec);
 #else
 	struct timeval tv;
 
-	assert(gettimeofday(&tv, NULL) == 0);
+	AZ(gettimeofday(&tv, NULL));
 	return (tv.tv_sec + 1e-6 * tv.tv_usec);
 #endif
 }
@@ -150,7 +150,7 @@ VTIM_parse(const char *p)
 			 * to set the timezone to UTC.  We check that.
 			 */
 			t = mktime(&tm);
-			assert(!strcmp(tzname[0], "UTC"));
+			AZ(strcmp(tzname[0], "UTC"));
 #endif
 			return (t);
 		}
diff --git a/lib/libvcc/vcc_storage.c b/lib/libvcc/vcc_storage.c
index 0ab3f64..89e63b0 100644
--- a/lib/libvcc/vcc_storage.c
+++ b/lib/libvcc/vcc_storage.c
@@ -109,7 +109,7 @@ vcc_Stv_Wildcard(struct vcc *tl, const struct token *t,
 
 	(void)wcsym;
 	assert((t->e - t->b) > strlen(PFX));
-	assert(!memcmp(t->b, PFX, strlen(PFX)));
+	AZ(memcmp(t->b, PFX, strlen(PFX)));
 
 	p = t->b + strlen(PFX);
 	for (q = p; q < t->e && *q != '.'; q++)



From phk at FreeBSD.org  Tue Apr 22 08:52:46 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 22 Apr 2014 10:52:46 +0200
Subject: [master] 3599490 One may have gcc or other things restricted,
 so that e.g. only a specific user and/or group may execute it.
 Varnish never inherited the groups of the user that has been specified by
 "-u". initgroups() will make sure that varnish gets all supplementary groups.
Message-ID: <E1WcWRa-0005QS-8j@project.varnish-software.com>


commit 3599490aed5524ea19a63cf488a60dc8ddb59365
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Apr 22 08:51:58 2014 +0000

    One may have gcc or other things restricted, so that e.g. only a
    specific user and/or group may execute it. Varnish never inherited
    the groups of the user that has been specified by "-u". initgroups()
    will make sure that varnish gets all supplementary groups.
    
    Submitted by:	Christian Ruppert
    Fixes	#1482

diff --git a/bin/varnishd/mgt/mgt_sandbox.c b/bin/varnishd/mgt/mgt_sandbox.c
index 069d660..83a2aeb 100644
--- a/bin/varnishd/mgt/mgt_sandbox.c
+++ b/bin/varnishd/mgt/mgt_sandbox.c
@@ -48,6 +48,7 @@
 #include <sys/prctl.h>
 #endif
 
+#include <grp.h>
 #include <stdio.h>
 #include <syslog.h>
 #include <unistd.h>
@@ -64,6 +65,7 @@ mgt_sandbox_unix(enum sandbox_e who)
 	(void)who;
 	if (geteuid() == 0) {
 		XXXAZ(setgid(mgt_param.gid));
+		XXXAZ(initgroups(mgt_param.user, mgt_param.gid));
 		XXXAZ(setuid(mgt_param.uid));
 	} else {
 		REPORT0(LOG_INFO, "Not running as root, no priv-sep");



From perbu at varnish-software.com  Tue Apr 22 16:10:52 2014
From: perbu at varnish-software.com (Per Buer)
Date: Tue, 22 Apr 2014 18:10:52 +0200
Subject: [master] 35308a3 Missing paren. Spotted by Alexander Kuznetcov
Message-ID: <E1WcdHY-0005Je-BF@project.varnish-software.com>


commit 35308a3c975ce3bd3690fe3afa4f25bb78b887e5
Author: Per Buer <perbu at varnish-software.com>
Date:   Tue Apr 22 18:10:50 2014 +0200

    Missing paren. Spotted by Alexander Kuznetcov

diff --git a/doc/sphinx/users-guide/purging.rst b/doc/sphinx/users-guide/purging.rst
index fcd5f5b..0c9b7c9 100644
--- a/doc/sphinx/users-guide/purging.rst
+++ b/doc/sphinx/users-guide/purging.rst
@@ -37,7 +37,7 @@ following VCL in place::
 
 	  if (req.method == "PURGE") {
 		  if (!client.ip ~ purge) {
-			  return(synth(405,"Not allowed.");
+			  return(synth(405,"Not allowed."));
 		  }
                   # jump to hit/miss
 		  return (purge);



From daghf at varnish-software.com  Wed Apr 23 14:39:45 2014
From: daghf at varnish-software.com (Dag Haavi Finstad)
Date: Wed, 23 Apr 2014 16:39:45 +0200
Subject: [master] 26546ec varnishtest txresp reason phrase 'Ok' -> 'OK'
Message-ID: <E1WcyKv-0006K6-CL@project.varnish-software.com>


commit 26546ec34d101ce0e8bd3ffbeb51e7af5af77f4b
Author: Dag Haavi Finstad <daghf at varnish-software.com>
Date:   Wed Apr 23 16:39:34 2014 +0200

    varnishtest txresp reason phrase 'Ok' -> 'OK'

diff --git a/bin/varnishtest/tests/a00001.vtc b/bin/varnishtest/tests/a00001.vtc
index 04106fe..d842287 100644
--- a/bin/varnishtest/tests/a00001.vtc
+++ b/bin/varnishtest/tests/a00001.vtc
@@ -15,7 +15,7 @@ client c1 -connect ${s1_sock} {
 	rxresp
 	expect resp.proto == HTTP/1.1
 	expect resp.status == 200
-	expect resp.msg == Ok
+	expect resp.msg == OK
 }
 
 client c1 -run
diff --git a/bin/varnishtest/tests/a00003.vtc b/bin/varnishtest/tests/a00003.vtc
index f928a42..810da48 100644
--- a/bin/varnishtest/tests/a00003.vtc
+++ b/bin/varnishtest/tests/a00003.vtc
@@ -32,7 +32,7 @@ client c2 -connect ${s2_sock} {
 	rxresp
 	expect resp.proto == HTTP/1.1
 	expect resp.status == 200
-	expect resp.msg == Ok
+	expect resp.msg == OK
 }
 
 client c1 -start
diff --git a/bin/varnishtest/tests/a00005.vtc b/bin/varnishtest/tests/a00005.vtc
index fcf4e58..6c0d634 100644
--- a/bin/varnishtest/tests/a00005.vtc
+++ b/bin/varnishtest/tests/a00005.vtc
@@ -34,7 +34,7 @@ client c1 -connect ${s2_sock} {
 	rxresp
 	expect resp.proto == HTTP/1.1
 	expect resp.status == 200
-	expect resp.msg == Ok
+	expect resp.msg == OK
 }
 
 client c1 -run
diff --git a/bin/varnishtest/tests/b00007.vtc b/bin/varnishtest/tests/b00007.vtc
index a48fa3f..02babe7 100644
--- a/bin/varnishtest/tests/b00007.vtc
+++ b/bin/varnishtest/tests/b00007.vtc
@@ -3,7 +3,7 @@ varnishtest "Check chunked encoding from backend works"
 server s1 {
 	rxreq
 	expect req.url == "/bar"
-	send "HTTP/1.1 200 Ok\r\n"
+	send "HTTP/1.1 200 OK\r\n"
 	send "Transfer-encoding: chunked\r\n"
 	send "\r\n"
 	send "00000004\r\n1234\r\n"
@@ -12,7 +12,7 @@ server s1 {
 
 	rxreq
 	expect req.url == "/foo"
-	send "HTTP/1.1 200 Ok\r\n"
+	send "HTTP/1.1 200 OK\r\n"
 	send "Transfer-encoding: chunked\r\n"
 	send "\r\n"
 	send "00000004\r\n1234\r\n"
diff --git a/bin/varnishtest/tests/b00011.vtc b/bin/varnishtest/tests/b00011.vtc
index bbd8cb3..82381a8 100644
--- a/bin/varnishtest/tests/b00011.vtc
+++ b/bin/varnishtest/tests/b00011.vtc
@@ -2,7 +2,7 @@ varnishtest "Check HTTP/1.0 EOF transmission"
 
 server s1 {
 	rxreq
-	send "HTTP/1.1 200 Ok\n"
+	send "HTTP/1.1 200 OK\n"
 	send "Connection: close\n"
 	send "\n"
 	send "Body line 1\n"
diff --git a/bin/varnishtest/tests/b00020.vtc b/bin/varnishtest/tests/b00020.vtc
index 20bc72b..1448ed5 100644
--- a/bin/varnishtest/tests/b00020.vtc
+++ b/bin/varnishtest/tests/b00020.vtc
@@ -4,7 +4,7 @@ feature SO_RCVTIMEO_WORKS
 
 server s1 {
 	rxreq
-	send "HTTP/1.1 200 Ok\r\nConnection: close\r\n\r\n"
+	send "HTTP/1.1 200 OK\r\nConnection: close\r\n\r\n"
 	delay 1.5
 	# send "Baba\n"
 } -start
@@ -29,7 +29,7 @@ varnish v1 -expect n_objectcore == 0
 
 server s1 -wait {
 	rxreq
-	send "HTTP/1.1 200 Ok\r\nConnection: close\r\n\r\n"
+	send "HTTP/1.1 200 OK\r\nConnection: close\r\n\r\n"
 	delay 0.5
 	send "Baba\n"
 	delay 0.5
diff --git a/bin/varnishtest/tests/b00021.vtc b/bin/varnishtest/tests/b00021.vtc
index 5646183..3d4e241 100644
--- a/bin/varnishtest/tests/b00021.vtc
+++ b/bin/varnishtest/tests/b00021.vtc
@@ -4,7 +4,7 @@ feature SO_RCVTIMEO_WORKS
 
 server s1 {
 	rxreq
-	send "HTTP/1.1 200 Ok\r\nConnection: close\r\n\r\n"
+	send "HTTP/1.1 200 OK\r\nConnection: close\r\n\r\n"
 	delay 4.0
 	send "Baba\n"
 } -start
@@ -26,7 +26,7 @@ client c1 {
 
 server s1 {
 	rxreq
-	send "HTTP/1.1 200 Ok\r\nConnection: close\r\n\r\n"
+	send "HTTP/1.1 200 OK\r\nConnection: close\r\n\r\n"
 	delay 1.0
 	send "Baba\n"
 	delay 1.0
diff --git a/bin/varnishtest/tests/b00022.vtc b/bin/varnishtest/tests/b00022.vtc
index 2201586..c232408 100644
--- a/bin/varnishtest/tests/b00022.vtc
+++ b/bin/varnishtest/tests/b00022.vtc
@@ -4,7 +4,7 @@ feature SO_RCVTIMEO_WORKS
 
 server s1 {
 	rxreq
-	send "HTTP/1.1 200 Ok\r\nConnection: close\r\n\r\n"
+	send "HTTP/1.1 200 OK\r\nConnection: close\r\n\r\n"
 	delay 1.5
 	send "Baba\n"
 } -start
@@ -28,7 +28,7 @@ client c1 {
 
 server s1 {
 	rxreq
-	send "HTTP/1.1 200 Ok\r\nConnection: close\r\n\r\n"
+	send "HTTP/1.1 200 OK\r\nConnection: close\r\n\r\n"
 	delay 0.5
 	send "Baba\n"
 	delay 0.5
diff --git a/bin/varnishtest/tests/b00027.vtc b/bin/varnishtest/tests/b00027.vtc
index f1907c0..e067475 100644
--- a/bin/varnishtest/tests/b00027.vtc
+++ b/bin/varnishtest/tests/b00027.vtc
@@ -6,7 +6,7 @@ server s1 {
 	rxreq
 	txresp -proto HTTP/1.0 -hdr "Connection: keep-alive"
 	rxreq
-	send "HTTP/1.1 200 Ok\n"
+	send "HTTP/1.1 200 OK\n"
 	send "Transfer-encoding: foobar\n"
 	send "\n"
 } -start
diff --git a/bin/varnishtest/tests/c00013.vtc b/bin/varnishtest/tests/c00013.vtc
index b9578aa..081ed1f 100644
--- a/bin/varnishtest/tests/c00013.vtc
+++ b/bin/varnishtest/tests/c00013.vtc
@@ -3,7 +3,7 @@ varnishtest "Test parking second request on backend delay"
 server s1 {
 	rxreq
 	expect req.url == "/foo"
-	send "HTTP/1.1 200 Ok\r\nConnection: close\r\n\r\n"
+	send "HTTP/1.1 200 OK\r\nConnection: close\r\n\r\n"
 	delay .2
 	sema r1 sync 2
 	delay .2
diff --git a/bin/varnishtest/tests/c00014.vtc b/bin/varnishtest/tests/c00014.vtc
index fffbbd7..99f2dd7 100644
--- a/bin/varnishtest/tests/c00014.vtc
+++ b/bin/varnishtest/tests/c00014.vtc
@@ -4,7 +4,7 @@ server s1 {
 	rxreq
 	expect req.url == "/foo"
 	sema r1 sync 2
-	send "HTTP/1.1 200 Ok\r\nContent-Length: 12\r\n\r\n"
+	send "HTTP/1.1 200 OK\r\nContent-Length: 12\r\n\r\n"
 	send "line1\n"
 	send "line2\n"
 
diff --git a/bin/varnishtest/tests/c00066.vtc b/bin/varnishtest/tests/c00066.vtc
index eeb1ff9..18a9b67 100644
--- a/bin/varnishtest/tests/c00066.vtc
+++ b/bin/varnishtest/tests/c00066.vtc
@@ -43,7 +43,7 @@ client c1 {
 	txreq -url "/"
 	rxresp
 	expect resp.status == 200
-	expect resp.msg == "Ok"
+	expect resp.msg == "OK"
 
 	txreq -url "/synth"
 	rxresp
diff --git a/bin/varnishtest/tests/e00007.vtc b/bin/varnishtest/tests/e00007.vtc
index 519759c..a03ef55 100644
--- a/bin/varnishtest/tests/e00007.vtc
+++ b/bin/varnishtest/tests/e00007.vtc
@@ -16,7 +16,7 @@ varnishtest "ESI spanning storage bits"
 server s1 {
 	rxreq
 	expect req.url == "/foo/bar"
-	send "HTTP/1.1 200 Ok\n"
+	send "HTTP/1.1 200 OK\n"
 	send "Connection: close\n"
 	send "\n"
 	send {
diff --git a/bin/varnishtest/tests/e00014.vtc b/bin/varnishtest/tests/e00014.vtc
index 7021c95..ec6d16b 100644
--- a/bin/varnishtest/tests/e00014.vtc
+++ b/bin/varnishtest/tests/e00014.vtc
@@ -3,7 +3,7 @@ varnishtest "Check <esi: detector"
 server s1 {
         rxreq
         expect req.url == "/foo"
-	send "HTTP/1.1 200 Ok\n"
+	send "HTTP/1.1 200 OK\n"
         send "Connection: close\n"
 	send "\n"
         send {           <a>   <esi/>                          }
diff --git a/bin/varnishtest/tests/l00002.vtc b/bin/varnishtest/tests/l00002.vtc
index 8c7bc35..72faa08 100644
--- a/bin/varnishtest/tests/l00002.vtc
+++ b/bin/varnishtest/tests/l00002.vtc
@@ -28,7 +28,7 @@ varnish v1 -vcl+backend {
 # \r\n				  2 bytes
 # Total:			 38 bytes
 # Response:
-# HTTP/1.1 200 Ok\r\n		 17 bytes
+# HTTP/1.1 200 OK\r\n		 17 bytes
 # Content-Length: 1000\r\n	 22 bytes
 # Connection: keep-alive\r\n	 24 bytes
 # Accept-Ranges: bytes\r\n	 22 bytes
@@ -40,7 +40,7 @@ varnish v1 -vcl+backend {
 # \r\n				  2 bytes
 # Total:			 19 bytes
 # Response:
-# HTTP/1.1 200 Ok\r\n		 17 bytes
+# HTTP/1.1 200 OK\r\n		 17 bytes
 # Content-Length: 2000\r\n	 22 bytes
 # Connection: keep-alive\r\n	 24 bytes
 # Accept-Ranges: bytes\r\n	 22 bytes
@@ -52,7 +52,7 @@ varnish v1 -vcl+backend {
 # \r\n				  2 bytes
 # Total:			 19 bytes
 # Response:
-# HTTP/1.1 200 Ok\r\n		 17 bytes
+# HTTP/1.1 200 OK\r\n		 17 bytes
 # Content-Length: 2000\r\n	 22 bytes
 # Connection: keep-alive\r\n	 24 bytes
 # Accept-Ranges: bytes\r\n	 22 bytes
diff --git a/bin/varnishtest/tests/l00003.vtc b/bin/varnishtest/tests/l00003.vtc
index cb03a18..86c9e2a 100644
--- a/bin/varnishtest/tests/l00003.vtc
+++ b/bin/varnishtest/tests/l00003.vtc
@@ -32,7 +32,7 @@ varnish v1 -vcl+backend {
 # Total:				18 bytes
 
 # Reponse:
-# HTTP/1.1 200 Ok\r\n			17 bytes
+# HTTP/1.1 200 OK\r\n			17 bytes
 # Transfer-Encoding: chunked\r\n	28 bytes
 # Connection: keep-alive\r\n		24 bytes
 # \r\n					 2 bytes
diff --git a/bin/varnishtest/tests/l00004.vtc b/bin/varnishtest/tests/l00004.vtc
index 3b07e36..22e5ea3 100644
--- a/bin/varnishtest/tests/l00004.vtc
+++ b/bin/varnishtest/tests/l00004.vtc
@@ -37,7 +37,7 @@ varnish v1 -vcl+backend {
 # asdf					 4 bytes
 
 # resp:
-# HTTP/1.1 200 Ok\r\n			17 bytes
+# HTTP/1.1 200 OK\r\n			17 bytes
 # Content-Length: 4\r\n			19 bytes
 # \r\n					 2 bytes
 # fdsa					 4 bytes
diff --git a/bin/varnishtest/tests/l00005.vtc b/bin/varnishtest/tests/l00005.vtc
index dbbea1b..e7472ff 100644
--- a/bin/varnishtest/tests/l00005.vtc
+++ b/bin/varnishtest/tests/l00005.vtc
@@ -28,7 +28,7 @@ varnish v1 -vcl+backend {
 # \r\n				  2 bytes
 # Total:			 54 bytes
 # Response:
-# HTTP/1.1 200 Ok\r\n		 17 bytes
+# HTTP/1.1 200 OK\r\n		 17 bytes
 # Content-Length: 1000\r\n	 22 bytes
 # \r\n				  2 bytes
 # Total:			 41 bytes
diff --git a/bin/varnishtest/tests/p00007.vtc b/bin/varnishtest/tests/p00007.vtc
index ca6d04d..5fd9e9e 100644
--- a/bin/varnishtest/tests/p00007.vtc
+++ b/bin/varnishtest/tests/p00007.vtc
@@ -3,7 +3,7 @@ varnishtest "test reload of object spanning incomplete segment"
 server s1 {
 	rxreq
 	expect req.url == "/1"
-	send "HTTP/1.1 200 Ok\n"
+	send "HTTP/1.1 200 OK\n"
 	send "Transfer-encoding: chunked\n"
 	send "\n"
 	chunkedlen 32
diff --git a/bin/varnishtest/tests/r00387.vtc b/bin/varnishtest/tests/r00387.vtc
index 5edb18a..57ec219 100644
--- a/bin/varnishtest/tests/r00387.vtc
+++ b/bin/varnishtest/tests/r00387.vtc
@@ -3,7 +3,7 @@ varnishtest "Regression test for #387: too long chunk header"
 server s1 {
 	rxreq
 	non-fatal
-	send "HTTP/1.1 200 Ok\r\n"
+	send "HTTP/1.1 200 OK\r\n"
 	send "Transfer-encoding: chunked\r\n"
 	send "\r\n"
 	send "004\r\n1234\r\n"
diff --git a/bin/varnishtest/tests/r00694.vtc b/bin/varnishtest/tests/r00694.vtc
index da2d9a5..a28f4f1 100644
--- a/bin/varnishtest/tests/r00694.vtc
+++ b/bin/varnishtest/tests/r00694.vtc
@@ -2,7 +2,7 @@ varnishtest "Wrong calculation of last storage segment length"
 
 server s1 {
 	rxreq
-	send "HTTP/1.1 200 Ok\r\n"
+	send "HTTP/1.1 200 OK\r\n"
 	send "Transfer-encoding: chunked\r\n"
 	send "\r\n"
 	# This is chunksize (128k) + 2 to force to chunks to be allocated
diff --git a/bin/varnishtest/tests/r00733.vtc b/bin/varnishtest/tests/r00733.vtc
index 4d6dd05..365f0ec 100644
--- a/bin/varnishtest/tests/r00733.vtc
+++ b/bin/varnishtest/tests/r00733.vtc
@@ -2,7 +2,7 @@ varnishtest "HTTP/1.1 Backend sends no length hint"
 
 server s1 {
 	rxreq
-	send "HTTP/1.1 200 Ok\n"
+	send "HTTP/1.1 200 OK\n"
 	send "\n"
 	send "12345"
 } -start
diff --git a/bin/varnishtest/tests/r01419.vtc b/bin/varnishtest/tests/r01419.vtc
index 5165c63..a47d424 100644
--- a/bin/varnishtest/tests/r01419.vtc
+++ b/bin/varnishtest/tests/r01419.vtc
@@ -2,7 +2,7 @@ varnishtest "Make sure banlurker skips busy objects"
 
 server s1 {
 	rxreq
-	send "HTTP/1.0 200 Ok\r\n"
+	send "HTTP/1.0 200 OK\r\n"
 	sema r1 sync 2
 	send "Foobar: blaf\r\n"
 	send "Content-Length: 10\r\n"
diff --git a/bin/varnishtest/vtc_http.c b/bin/varnishtest/vtc_http.c
index 17dba6b..966f043 100644
--- a/bin/varnishtest/vtc_http.c
+++ b/bin/varnishtest/vtc_http.c
@@ -760,7 +760,7 @@ cmd_http_txresp(CMD_ARGS)
 	struct http *hp;
 	const char *proto = "HTTP/1.1";
 	const char *status = "200";
-	const char *msg = "Ok";
+	const char *msg = "OK";
 	char* body = NULL;
 
 	(void)cmd;



From tfheen at err.no  Thu Apr 24 16:28:27 2014
From: tfheen at err.no (Tollef Fog Heen)
Date: Thu, 24 Apr 2014 18:28:27 +0200
Subject: [master] 06ba9c2 Error when no rst2man is found
Message-ID: <E1WdMVf-0002aN-EX@project.varnish-software.com>


commit 06ba9c2ae51c5496ae2fc322f4d3543f0894479b
Author: Tollef Fog Heen <tfheen at fastly.com>
Date:   Thu Apr 24 18:27:13 2014 +0200

    Error when no rst2man is found
    
    If one runs configure with --without-rst2man or no rst2man is found,
    the value is "no", not an empty string.  Handle this correctly in
    configure.
    
    Fixes: #1473

diff --git a/configure.ac b/configure.ac
index 4dfb222..82bd906 100644
--- a/configure.ac
+++ b/configure.ac
@@ -55,7 +55,7 @@ AC_ARG_WITH([rst2man],
   AS_HELP_STRING([--with-rst2man=PATH], [Location of rst2man (auto)]),
   [RST2MAN="$withval"],
   AC_CHECK_PROGS(RST2MAN, [rst2man rst2man.py], []))
-if test -z "$RST2MAN"; then
+if test "$RST2MAN" = "no"; then
   AC_MSG_ERROR(
     [rst2man is needed to build Varnish, please install python-docutils.])
 fi



From phk at FreeBSD.org  Fri Apr 25 11:42:57 2014
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Fri, 25 Apr 2014 13:42:57 +0200
Subject: [master] e99b5cf Only attempt IMS on status=200 objects.
Message-ID: <E1WdeWv-0003fq-0d@project.varnish-software.com>


commit e99b5cfd886ec38a7f883e23ba516063cf4c16f8
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Apr 25 11:42:37 2014 +0000

    Only attempt IMS on status=200 objects.

diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index 5a0a9ff..e61a007 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -199,7 +199,7 @@ vbf_stp_mkbereq(const struct worker *wrk, struct busyobj *bo)
 		http_CopyHome(bo->bereq0);
 	}
 
-	if (bo->ims_obj != NULL) {
+	if (bo->ims_obj != NULL && bo->ims_obj->http->status == 200) {
 		if (http_GetHdr(bo->ims_obj->http, H_Last_Modified, &p)) {
 			http_PrintfHeader(bo->bereq0,
 			    "If-Modified-Since: %s", p);



From martin at varnish-software.com  Fri Apr 25 13:50:41 2014
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Fri, 25 Apr 2014 15:50:41 +0200
Subject: [master] e258ca1 Remove negative Age assertion and truncate to zero
 instead.
Message-ID: <E1WdgWX-0004Iv-Er@project.varnish-software.com>


commit e258ca15604b210aad3d70960eccd21fce645778
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Thu Apr 24 14:55:06 2014 +0200

    Remove negative Age assertion and truncate to zero instead.
    
    The Age reported on response objects is calculated from the last
    request timestamp taken. For a cache hit that hasn't been on a
    waitinglist, that will be the Start timestamp. This opens a race where
    the requests' last timestamp could be before the objects t_origin.
    
    Truncate the Age to zero rather than assert in that case.
    
    Fixes: #1486

diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index 2518422..88cebcd 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -117,10 +117,12 @@ cnt_deliver(struct worker *wrk, struct req *req)
 	/* We base Age calculation upon the last timestamp taken during
 	   client request processing. This gives some inaccuracy, but
 	   since Age is only full second resolution that shouldn't
-	   matter. */
-	assert(req->t_prev > req->obj->exp.t_origin);
+	   matter. (Last request timestamp could be a Start timestamp
+	   taken before the object entered into cache leading to negative
+	   age. Truncate to zero in that case).
+	*/
 	http_PrintfHeader(req->resp, "Age: %.0f",
-	    req->t_prev - req->obj->exp.t_origin);
+	    fmax(0., req->t_prev - req->obj->exp.t_origin));
 
 	http_SetHeader(req->resp, "Via: 1.1 varnish (v4)");
 



From martin at varnish-software.com  Fri Apr 25 13:50:41 2014
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Fri, 25 Apr 2014 15:50:41 +0200
Subject: [master] e04394c Copy the status code, proto,
 status string and response message on backend IMS.
Message-ID: <E1WdgWX-0004J1-G9@project.varnish-software.com>


commit e04394c7205547193d188ca5d18e8361e57c838d
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Fri Apr 25 11:11:13 2014 +0200

    Copy the status code, proto, status string and response message on
    backend IMS.
    
    When revalidating using backend IMS, copy the status code, status
    code, status string and response message from the original object into
    the new revalidated object. This makes sure that none of the 304
    message fields gets applied to the new revalidated object.
    
    Fixes: #1485

diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index e61a007..dded999 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -356,9 +356,9 @@ vbf_stp_startfetch(struct worker *wrk, struct busyobj *bo)
 	AZ(bo->do_esi);
 
 	if (bo->ims_obj != NULL && bo->beresp->status == 304) {
-		bo->beresp->status = 200;
 		http_Merge(bo->ims_obj->http, bo->beresp,
 		    bo->ims_obj->changed_gzip);
+		assert(bo->beresp->status == 200);
 		do_ims = 1;
 	} else
 		do_ims = 0;
diff --git a/bin/varnishd/cache/cache_http.c b/bin/varnishd/cache/cache_http.c
index 6898987..7fe6c9f 100644
--- a/bin/varnishd/cache/cache_http.c
+++ b/bin/varnishd/cache/cache_http.c
@@ -637,7 +637,9 @@ http_FilterResp(const struct http *fm, struct http *to, unsigned how)
 }
 
 /*--------------------------------------------------------------------
- * Merge two HTTP headers the "wrong" way.
+ * Merge two HTTP headers the "wrong" way. Used by backend IMS to
+ * merge in the headers of the validated object with the headers of
+ * the 304 response.
  */
 
 void
@@ -646,6 +648,11 @@ http_Merge(const struct http *fm, struct http *to, int not_ce)
 	unsigned u, v;
 	const char *p;
 
+	to->status = fm->status;
+	http_SetH(to, HTTP_HDR_PROTO, fm->hd[HTTP_HDR_PROTO].b);
+	http_SetH(to, HTTP_HDR_STATUS, fm->hd[HTTP_HDR_STATUS].b);
+	http_SetH(to, HTTP_HDR_RESPONSE, fm->hd[HTTP_HDR_RESPONSE].b);
+
 	for (u = HTTP_HDR_FIRST; u < fm->nhd; u++)
 		fm->hdf[u] |= HDF_MARKER;
 	if (not_ce) {
diff --git a/bin/varnishtest/tests/r01485.vtc b/bin/varnishtest/tests/r01485.vtc
new file mode 100644
index 0000000..caf267a
--- /dev/null
+++ b/bin/varnishtest/tests/r01485.vtc
@@ -0,0 +1,32 @@
+varnishtest "#1485: Wrong response reason phrase"
+
+server s1 {
+	rxreq
+	txresp -hdr "Etag: foo"
+
+	rxreq
+	expect req.http.If-None-Match == "foo"
+	txresp -status 304 -msg "Not Modified"
+} -start
+
+varnish v1 -vcl+backend {
+	sub vcl_backend_response {
+		set beresp.ttl = 1ms;
+		set beresp.grace = 0s;
+		set beresp.keep = 1h;
+	}
+} -start
+
+client c1 {
+	txreq
+	rxresp
+	expect resp.status == 200
+	expect resp.msg == "OK"
+
+	delay 0.1
+
+	txreq
+	rxresp
+	expect resp.status == 200
+	expect resp.msg == "OK"
+} -run



From fgsch at lodoss.net  Sun Apr 27 13:11:16 2014
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Sun, 27 Apr 2014 15:11:16 +0200
Subject: [master] 5729e77 Document vcl_purge and replace vcl_erro with
 vcl_synth
Message-ID: <E1WeOrU-0008VH-6z@project.varnish-software.com>


commit 5729e773973fb1b9190bccadc88caba948b4e5f6
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Sun Apr 27 14:09:19 2014 +0100

    Document vcl_purge and replace vcl_erro with vcl_synth

diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index 88cebcd..d12f91c 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -53,7 +53,7 @@ DOT	label="Request received"
 DOT ]
 DOT ESI_REQ [ shape=hexagon ]
 DOT ESI_REQ -> recv
-DOT ERROR [shape=plaintext]
+DOT SYNTH [shape=plaintext]
 DOT RESTART [shape=plaintext]
 DOT acceptor -> recv [style=bold,color=green]
  */
@@ -178,15 +178,15 @@ VSLb(req->vsl, SLT_Debug, "XXX REF %d", req->obj->objcore->refcnt);
 }
 
 /*--------------------------------------------------------------------
- * Emit an error
+ * Emit a synthetic response
  *
-DOT subgraph xcluster_error {
-DOT	vcl_error [
+DOT subgraph xcluster_synth {
+DOT	synth [
 DOT		shape=record
-DOT		label="{{vcl_error()|resp.}|{<del>deliver?|<restart>restart?}}"
+DOT		label="{cnt_synth:|{vcl_synth\{\}|resp.}|{<del>deliver?|<restart>restart?}}"
 DOT	]
-DOT	ERROR -> vcl_error
-DOT	vcl_error:del:s -> deliver [label=deliver]
+DOT	SYNTH -> synth
+DOT	synth:del:s -> deliver [label=deliver]
 DOT }
  */
 
@@ -313,7 +313,7 @@ DOT		label="{<top>cnt_lookup:|hash lookup|{<busy>busy?|<e>exp?|<eb>exp+busy?|<h>
 DOT	]
 DOT	lookup2 [
 DOT		shape=record
-DOT		label="{<top>cnt_lookup:|{vcl_hit\{\}|req.*, obj.*}|{<deliver>deliver?|error?|restart?|<fetch>fetch?|<pass>pass?}}"
+DOT		label="{<top>cnt_lookup:|{vcl_hit\{\}|req.*, obj.*}|{<deliver>deliver?|synth?|restart?|<fetch>fetch?|<pass>pass?}}"
 DOT	]
 DOT }
 DOT lookup:busy:w -> lookup:top:w [label="(waitinglist)"]
@@ -471,7 +471,7 @@ cnt_lookup(struct worker *wrk, struct req *req)
 DOT subgraph xcluster_miss {
 DOT	miss [
 DOT		shape=record
-DOT		label="{cnt_miss:|{vcl_miss\{\}|req.*}|{<fetch>fetch?|<err>error?|<rst>restart?|<pass>pass?}}"
+DOT		label="{cnt_miss:|{vcl_miss\{\}|req.*}|{<fetch>fetch?|<synth>synth?|<rst>restart?|<pass>pass?}}"
 DOT	]
 DOT }
 DOT miss:fetch:s -> fetch [label="fetch",style=bold,color=blue]
@@ -525,7 +525,7 @@ cnt_miss(struct worker *wrk, struct req *req)
 DOT subgraph xcluster_pass {
 DOT	pass [
 DOT		shape=record
-DOT		label="{cnt_pass:|{vcl_pass\{\}|req.*}|{<fetch>fetch?|<err>error?|<rst>restart?}}"
+DOT		label="{cnt_pass:|{vcl_pass\{\}|req.*}|{<fetch>fetch?|<synth>synth?|<rst>restart?}}"
 DOT	]
 DOT }
 DOT pass:fetch:s -> fetch:n [style=bold, color=red]
@@ -573,7 +573,7 @@ cnt_pass(struct worker *wrk, struct req *req)
 DOT subgraph xcluster_pipe {
 DOT	pipe [
 DOT		shape=record
-DOT		label="{cnt_pipe:|filter req.*-\>bereq.*|{vcl_pipe()|req.*, bereq\.*}|{<pipe>pipe?|<error>error?}}"
+DOT		label="{cnt_pipe:|filter req.*-\>bereq.*|{vcl_pipe()|req.*, bereq\.*}|{<pipe>pipe?|<synth>synth?}}"
 DOT	]
 DOT	pipe_do [
 DOT		shape=ellipse
@@ -625,7 +625,7 @@ DOT }
 DOT RESTART -> restart [color=purple]
 DOT restart -> recv [color=purple]
 DOT restart -> err_restart
-DOT err_restart [label="ERROR",shape=plaintext]
+DOT err_restart [label="SYNTH",shape=plaintext]
  */
 
 static enum req_fsm_nxt
@@ -667,7 +667,7 @@ cnt_restart(struct worker *wrk, struct req *req)
 DOT subgraph xcluster_recv {
 DOT	recv [
 DOT		shape=record
-DOT		label="{cnt_recv:|{vcl_recv\{\}|req.*}|{vcl_hash\{\}|req.*}|{<lookup>lookup?|<pass>pass?|<pipe>pipe?|<error>error?|<purge>purge?}}"
+DOT		label="{cnt_recv:|{vcl_recv\{\}|req.*}|{vcl_hash\{\}|req.*}|{<lookup>lookup?|<pass>pass?|<pipe>pipe?|<synth>synth?|<purge>purge?}}"
 DOT	]
 DOT }
 DOT recv:pipe -> pipe [style=bold,color=orange]
@@ -793,6 +793,13 @@ cnt_recv(struct worker *wrk, struct req *req)
  * Find the objhead, purge it and ask VCL if we should fetch or
  * just return.
  * XXX: fetching not implemented yet.
+ *
+DOT subgraph xcluster_purge {
+DOT	purge [
+DOT		shape=record
+DOT		label="{cnt_purge:|{vcl_purge\{\}|req.*}|{<synth>synth?}}"
+DOT	]
+DOT }
  */
 
 static enum req_fsm_nxt



From martin at varnish-software.com  Mon Apr 28 13:27:01 2014
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Mon, 28 Apr 2014 15:27:01 +0200
Subject: [master] 94a3d07 Close race on req->wrk clearing introduced in
 6f2e1bcd
Message-ID: <E1WelaH-0006Qx-Gi@project.varnish-software.com>


commit 94a3d071b66d4f7aa7f94660cad42e4d58f25724
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Mon Apr 28 15:19:15 2014 +0200

    Close race on req->wrk clearing introduced in 6f2e1bcd
    
    The disembarking thread would clear the req->wrk pointer in
    CNT_Request. The req could have already been rescheduled on another
    worker at this point, causing the req->wrk to be cleared while
    processing. Change back to not clearing the pointer for
    REQ_FSM_DISEMBARK (it's already been done under mutex in HSH_Lookup
    anyways).
    
    Fixes: #1488

diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index d12f91c..d465ba4 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -913,8 +913,8 @@ CNT_Request(struct worker *wrk, struct req *req)
 			VTAILQ_REMOVE(&req->body, st, list);
 			STV_free(st);
 		}
+		req->wrk = NULL;
 	}
-	req->wrk = NULL;
 	assert(WRW_IsReleased(wrk));
 	return (nxt);
 }



From fgsch at lodoss.net  Mon Apr 28 15:36:24 2014
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Mon, 28 Apr 2014 17:36:24 +0200
Subject: [master] c8663d5 Sync with reality
Message-ID: <E1WenbU-0007rG-Me@project.varnish-software.com>


commit c8663d50d0b420dfa28d1a1b8b6fc0560b6fffd0
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Mon Apr 28 16:33:34 2014 +0100

    Sync with reality
    
    Update board members, URLs, spelling and fmt(1).

diff --git a/doc/sphinx/tutorial/introduction.rst b/doc/sphinx/tutorial/introduction.rst
index 2fbc43a..330764f 100644
--- a/doc/sphinx/tutorial/introduction.rst
+++ b/doc/sphinx/tutorial/introduction.rst
@@ -3,32 +3,32 @@
 The fundamentals of web proxy caching with Varnish
 --------------------------------------------------
 
-Varnish is a caching HTTP reverse proxy. It recieves requests from
-clients and tries to answer them from the cache. If Varnish cannot answer
-the request from the cache it will forward the request to the backend,
+Varnish is a caching HTTP reverse proxy. It receives requests from clients
+and tries to answer them from the cache. If Varnish cannot answer the
+request from the cache it will forward the request to the backend,
 fetch the response, store it in the cache and deliver it to the client.
 
-When Varnish has a cached response ready it is typically delivered in
-a matter of microseconds, two orders of magnitude faster than your
-typical backend server, so you want to make sure to have Varnish answer
-as many of the requests as possible directly from the cache.
+When Varnish has a cached response ready it is typically delivered in a
+matter of microseconds, two orders of magnitude faster than your typical
+backend server, so you want to make sure to have Varnish answer as many
+of the requests as possible directly from the cache.
 
-Varnish decides whether it can store the content or not based on the response
-it gets back from the backend. The backend can instruct Varnish to cache the
-content with the HTTP response header `Cache-Control`. There are a few
-conditions where Varnish will not cache, the most common one being the use of
-cookies. Since cookies indicates a client-specific web object, Varnish will by
-default not cache it.
+Varnish decides whether it can store the content or not based on the
+response it gets back from the backend. The backend can instruct Varnish
+to cache the content with the HTTP response header `Cache-Control`. There
+are a few conditions where Varnish will not cache, the most common one
+being the use of cookies. Since cookies indicates a client-specific web
+object, Varnish will by default not cache it.
 
-This behaviour as most of Varnish functionality can be changed using policies
-written in the Varnish Configuration Language (VCL). See
+This behaviour as most of Varnish functionality can be changed using
+policies written in the Varnish Configuration Language (VCL). See
 :ref:`users-guide-index` for more information on how to do that.
 
 Performance
 ~~~~~~~~~~~
 
-Varnish has a modern architecture and is written with performance in
-mind.  It is usually bound by the speed of the network, effectively
+Varnish has a modern architecture and is written with performance
+in mind.  It is usually bound by the speed of the network, effectively
 turning performance into a non-issue. You get to focus on how your web
 application work and you can allow yourself, to some degree, to care
 less about performance and scalability.
@@ -38,45 +38,44 @@ less about performance and scalability.
 Flexibility
 ~~~~~~~~~~~
 
-One of the key features of Varnish Cache, in addition to its
-performance, is the flexibility of its configuration language,
-VCL. VCL enables you to write policies on how incoming requests should
-be handled. 
+One of the key features of Varnish Cache, in addition to its performance,
+is the flexibility of its configuration language, VCL. VCL enables you
+to write policies on how incoming requests should be handled.
 
 In such a policy you can decide what content you want to serve, from
-where you want to get the content and how the request or response
-should be altered. 
+where you want to get the content and how the request or response should
+be altered.
 
 Supported platforms
 --------------------
 
-Varnish is written to run on modern versions of Linux and FreeBSD and
-the best experience is had on those platforms. Thanks to our
-contributors it also runs on NetBSD, OpenBSD, OS X and various
-Solaris-descendants like Oracle Solaris, OmniOS and SmartOS.
+Varnish is written to run on modern versions of Linux and FreeBSD and the
+best experience is had on those platforms. Thanks to our contributors
+it also runs on NetBSD, OpenBSD, OS X and various Solaris-descendants
+like Oracle Solaris, OmniOS and SmartOS.
 
 About the Varnish development process
 -------------------------------------
 
 Varnish is a community driven project. The development is overseen by
-the Varnish Governing Board which currently consist of Poul-Henning
-Kamp (Architect), Rogier Mulhuijzen (Fastly) and Kristian Lyngst?l
-(Varnish Software).
+the Varnish Governing Board which currently consist of Poul-Henning Kamp
+(Architect), Rogier Mulhuijzen (Fastly) and Lasse Karstensen (Varnish
+Software).
 
-Please see https://www.varnish-cache.org/trac/wiki/Contributing as
-a starting point if you would like to contribute to Varnish.
+Please see https://www.varnish-cache.org/trac/wiki/Contributing as a
+starting point if you would like to contribute to Varnish.
 
 Getting in touch
 ----------------
 
-You can get in touch with us trough many channels. For real time chat
-you can reach us on IRC trough the server irc.linpro.net on the
-#varnish and #varnish-hacking channels.
-The are two mailing lists available. One for user questions and one
-for development discussions. See https://www.varnish-cache.org/mailinglist for
-information and signup.  There is also a web forum on the same site.
+You can get in touch with us through many channels. For real time chat
+you can reach us on IRC trough the server irc.linpro.net on the #varnish
+and #varnish-hacking channels.
+There are two mailing lists available. One for user questions and one
+for development discussions. See https://www.varnish-cache.org/lists
+for information and signup.  There is also a web forum on the same site.
 
 Now that you have a vague idea on what Varnish Cache is, let see if we
 can get it up and running.
 
-.. XXX:The above three paragraphs are repetetive this is already handled in previos chapters. The only new information is Governing Board which could be moved to the introduction and the paragraphs scrapped. benc
+.. XXX:The above three paragraphs are repetitive this is already handled in previous chapters. The only new information is Governing Board which could be moved to the introduction and the paragraphs scrapped. benc



From martin at varnish-software.com  Wed Apr 30 14:00:28 2014
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Wed, 30 Apr 2014 16:00:28 +0200
Subject: [master] 9aa6eef Fix up req.esi to only be available in client
 context.
Message-ID: <E1WfV3k-0003yA-Gn@project.varnish-software.com>


commit 9aa6eef5b2f37f2197ff70bb41e74705b83f80eb
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Wed Apr 30 15:59:12 2014 +0200

    Fix up req.esi to only be available in client context.
    
    Fixes: #1489
    Spotted by: xcir

diff --git a/lib/libvcc/generate.py b/lib/libvcc/generate.py
index ace50e6..57f22ca 100755
--- a/lib/libvcc/generate.py
+++ b/lib/libvcc/generate.py
@@ -254,8 +254,8 @@ sp_variables = [
 	),
 	('req.esi',
 		'BOOL',
-		( 'recv', 'backend_response', 'deliver', 'synth',),
-		( 'recv', 'backend_response', 'deliver', 'synth',), """
+		( 'client',),
+		( 'client',), """
 		Boolean. Set to false to disable ESI processing
 		regardless of any value in beresp.do_esi. Defaults
 		to true. This variable is subject to change in