cache_fetch

From lkarsten at varnish-software.com Wed Jun 1 11:05:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Wed, 01 Jun 2016 13:05:09 +0200 Subject: [4.1] ab83560 Add 1806 to changelog. Message-ID: commit ab83560584113301ec42bfc188661b1b3dc86b3f Author: Lasse Karstensen Date: Wed Jun 1 13:04:00 2016 +0200 Add 1806 to changelog. Ref: #1806 diff --git a/doc/changes.rst b/doc/changes.rst index 7cbe430..2f7d716 100644 --- a/doc/changes.rst +++ b/doc/changes.rst @@ -22,6 +22,7 @@ Bugs fixed * 1869_ - Remove temporary directory iff called with -C * 1883_ - Only accept C identifiers as acls * 1855_ - Truncate output if it's wider than 12 chars +* 1806_ - One minute delay on return (pipe) and a POST-Request * 1725_ - Revive the backend_conn counter .. _1955: https://github.com/varnishcache/varnish-cache/issues/1955 @@ -35,6 +36,7 @@ Bugs fixed .. _1871: https://github.com/varnishcache/varnish-cache/issues/1871 .. _1869: https://github.com/varnishcache/varnish-cache/issues/1869 .. _1855: https://github.com/varnishcache/varnish-cache/issues/1855 +.. _1806: https://github.com/varnishcache/varnish-cache/issues/1806 .. _1725: https://github.com/varnishcache/varnish-cache/issues/1725 From lkarsten at varnish-software.com Wed Jun 1 11:59:06 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Wed, 01 Jun 2016 13:59:06 +0200 Subject: [master] 4fadbca Rename to fit with the other thread names. Message-ID: commit 4fadbca2b035bccebec478a2b486347e23ef4843 Author: Lasse Karstensen Date: Wed Jun 1 13:56:39 2016 +0200 Rename to fit with the other thread names. Before this change: hostname:/proc/24139$ cat task/*/comm | sort | uniq Backend poller ban-lurker cache-acceptor cache-epoll cache-main cache-timeout cache-worker hcb-cleaner MPL_busyobj MPL_req0 MPL_req1 MPL_sess0 MPL_sess1 pool_herder vsm_cleaner diff --git a/bin/varnishd/cache/cache_backend_probe.c b/bin/varnishd/cache/cache_backend_probe.c index 0adf25d..479d1fa 100644 --- a/bin/varnishd/cache/cache_backend_probe.c +++ b/bin/varnishd/cache/cache_backend_probe.c @@ -627,5 +627,5 @@ VBP_Init(void) vbp_heap = binheap_new(NULL, vbp_cmp, vbp_update); AN(vbp_heap); AZ(pthread_cond_init(&vbp_cond, NULL)); - WRK_BgThread(&thr, "Backend poller", vbp_thread, NULL); + WRK_BgThread(&thr, "backend-poller", vbp_thread, NULL); } From lkarsten at varnish-software.com Wed Jun 1 12:26:06 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Wed, 01 Jun 2016 14:26:06 +0200 Subject: [4.1] 5ddcf6a check for overrun before looking at the next vsm record Message-ID: commit 5ddcf6a2b26da27c4bd3871f4303cdedb41b3433 Author: Nils Goroll Date: Wed May 11 14:47:10 2016 +0200 check for overrun before looking at the next vsm record Fixes #1873 diff --git a/lib/libvarnishapi/vsl_cursor.c b/lib/libvarnishapi/vsl_cursor.c index e9daab0..33d08e3 100644 --- a/lib/libvarnishapi/vsl_cursor.c +++ b/lib/libvarnishapi/vsl_cursor.c @@ -126,6 +126,10 @@ vslc_vsm_next(const struct VSL_cursor *cursor) CHECK_OBJ_NOTNULL(c->vsm, VSM_MAGIC); while (1) { + i = vslc_vsm_check(&c->cursor, &c->next); + if (i <= 0) + return (-3); /* Overrun */ + t = *(volatile const uint32_t *)c->next.ptr; AN(t); @@ -147,10 +151,6 @@ vslc_vsm_next(const struct VSL_cursor *cursor) return (0); /* No new records available */ } - i = vslc_vsm_check(&c->cursor, &c->next); - if (i <= 0) - return (-3); /* Overrun */ - c->cursor.rec = c->next; c->next.ptr = VSL_NEXT(c->next.ptr); From lkarsten at varnish-software.com Wed Jun 1 12:26:06 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Wed, 01 Jun 2016 14:26:06 +0200 Subject: [4.1] 0cea407 Add 1873 to changelog. Message-ID: commit 0cea407acc94329585443234530d3bec20952221 Author: Lasse Karstensen Date: Wed Jun 1 14:25:42 2016 +0200 Add 1873 to changelog. Ref: #1873 diff --git a/doc/changes.rst b/doc/changes.rst index 2f7d716..6791048 100644 --- a/doc/changes.rst +++ b/doc/changes.rst @@ -18,6 +18,7 @@ Bugs fixed * 1885_ - [vmodtool] Method names should start with a period * 1879_ - Correct handling of duplicate headers on IMS header merge * 1878_ - Fix a ESI+gzip corner case which had escaped notice until now +* 1873_ - Check for overrun before looking at the next vsm record * 1871_ - Missing error handling code in V1F_Setup_Fetch * 1869_ - Remove temporary directory iff called with -C * 1883_ - Only accept C identifiers as acls @@ -33,6 +34,7 @@ Bugs fixed .. _1883: https://github.com/varnishcache/varnish-cache/issues/1883 .. _1879: https://github.com/varnishcache/varnish-cache/issues/1879 .. _1878: https://github.com/varnishcache/varnish-cache/issues/1878 +.. _1873: https://github.com/varnishcache/varnish-cache/issues/1873 .. _1871: https://github.com/varnishcache/varnish-cache/issues/1871 .. _1869: https://github.com/varnishcache/varnish-cache/issues/1869 .. _1855: https://github.com/varnishcache/varnish-cache/issues/1855 From lkarsten at varnish-software.com Wed Jun 1 12:41:07 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Wed, 01 Jun 2016 14:41:07 +0200 Subject: [4.1] 55a08f5 Add #1847 (https_scheme) properly to changelog. Message-ID: commit 55a08f553df8c3f0716a82e85f4c020e639b2c53 Author: Lasse Karstensen Date: Wed Jun 1 14:39:29 2016 +0200 Add #1847 (https_scheme) properly to changelog. Ref: #1847 diff --git a/doc/changes.rst b/doc/changes.rst index 6791048..a8f9920 100644 --- a/doc/changes.rst +++ b/doc/changes.rst @@ -95,7 +95,7 @@ Varnish Cache 4.1.2-beta1 (2016-02-17) * Off-by-one in WS_Reset() fixed. * "https_scheme" parameter added. Enables graceful handling of compound - request URLs with HTTPS scheme. + request URLs with HTTPS scheme. (Bug 1847_) Bugs fixed ---------- @@ -117,6 +117,7 @@ Bugs fixed .. _1851: https://www.varnish-cache.org/trac/ticket/1851 .. _1852: https://www.varnish-cache.org/trac/ticket/1852 .. _1857: https://www.varnish-cache.org/trac/ticket/1857 +.. _1847: https://www.varnish-cache.org/trac/ticket/1847 ================================ From phk at FreeBSD.org Wed Jun 1 14:57:08 2016 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Wed, 01 Jun 2016 16:57:08 +0200 Subject: [master] a78efad I set out to straighten out a number of hacks and inconsistencies in the string spec we pass to the VCC, to make future VCL/VMOD/VCC work easier. Message-ID: commit a78efad8002895e6097aeb6b1daeac0f6108b9a9 Author: Poul-Henning Kamp Date: Wed Jun 1 14:41:07 2016 +0000 I set out to straighten out a number of hacks and inconsistencies in the string spec we pass to the VCC, to make future VCL/VMOD/VCC work easier. Instead I ended up rewriting the entire .VCC file compiler, which was so much easier than the first time, because now I know what it should do. There should be no incompatibilities, but as always: Please test and report. diff --git a/lib/libvcc/vcc_action.c b/lib/libvcc/vcc_action.c index 2e1a2fb..aa78adb 100644 --- a/lib/libvcc/vcc_action.c +++ b/lib/libvcc/vcc_action.c @@ -196,23 +196,22 @@ parse_new(struct vcc *tl) /*lint -save -e448 */ /* Split the first three args */ p = sy2->args; + s_obj = p; p += strlen(p) + 1; - s_init = p; - /* - * Check for the end marked (\0\0) followed by s(truct) to avoid - * matching an ENUM half-way through and generating illegal C code. - */ - while (p[0] != '\0' || p[1] != '\0' || p[2] != 's') - p++; - p += 2; - AZ(strncmp(p, "struct vmod_", 12)); + s_struct = p; p += strlen(p) + 1; - s_fini = p + strlen(p) + 1; - while (p[0] != '\0' || p[1] != '\0') + + s_init = p; + while (p[0] != '\0' || p[1] != '\0' || p[2] != '\0') + p++; + p += 3; + + s_fini = p; + while (p[0] != '\0' || p[1] != '\0' || p[2] != '\0') p++; - p += 2; + p += 3; Fh(tl, 0, "static %s *vo_%s;\n\n", s_struct, sy1->name); @@ -238,20 +237,9 @@ parse_new(struct vcc *tl) sy3->args = p; sy3->extra = TlDup(tl, buf1); - while (p[0] != '\0' || p[1] != '\0') { - if (!memcmp(p, "ENUM\0", 5)) { - /* XXX: Special case for ENUM that has - it's own \0\0 end marker. Not exactly - elegant, we should consider - alternatives here. Maybe runlength - encode the entire block? */ - p += strlen(p) + 1; - while (p[0] != '\0' || p[1] != '\0') - p++; - } + while (p[0] != '\0' || p[1] != '\0' || p[2] != '\0') p++; - } - p += 2; + p += 3; } sy1->def_e = tl->t; /*lint -restore */ diff --git a/lib/libvcc/vcc_expr.c b/lib/libvcc/vcc_expr.c index 7ad85ee..8284618 100644 --- a/lib/libvcc/vcc_expr.c +++ b/lib/libvcc/vcc_expr.c @@ -662,20 +662,23 @@ vcc_func(struct vcc *tl, struct expr **e, const char *cfunc, p += strlen(p) + 1; continue; } - if (fa->type == ENUM) { - fa->enum_bits = p; - while (*p != '\0') + if (*p == '\1') { + fa->enum_bits = ++p; + while (*p != '\1') p += strlen(p) + 1; - p += strlen(p) + 1; + p++; + assert(*p == '\0'); + p++; } - if (*p == '\1') { + if (*p == '\2') { fa->name = p + 1; - p = strchr(p, '\0') + 1; - if (*p == '\2') { - fa->val = p + 1; - p = strchr(p, '\0') + 1; - } + p += strlen(p) + 1; + } + if (*p == '\3') { + fa->val = p + 1; + p += strlen(p) + 1; } + assert(*p == 0 || *p > ' '); } VTAILQ_FOREACH(fa, &head, list) { diff --git a/lib/libvcc/vcc_vmod.c b/lib/libvcc/vcc_vmod.c index 5d36bd3..06c2b92 100644 --- a/lib/libvcc/vcc_vmod.c +++ b/lib/libvcc/vcc_vmod.c @@ -235,7 +235,8 @@ vcc_ParseImport(struct vcc *tl) "\t\t VCL_EVENT_DISCARD);\n", p, PF(mod)); VSB_printf(ifp->event, "\t%s(ctx, &vmod_priv_%.*s, ev)", p, PF(mod)); - } else { + } else if (!strcmp(p, "$FUNC")) { + p += strlen(p) + 1; sym = VCC_AddSymbolStr(tl, p, SYM_FUNC); ERRCHK(tl); AN(sym); @@ -245,6 +246,10 @@ vcc_ParseImport(struct vcc *tl) p += strlen(p) + 1; sym->args = p; sym->fmt = VCC_arg_type(&p); + } else { + VSB_printf(tl->sb, "Internal spec error (%s)\n", p); + vcc_ErrWhere(tl, mod); + return; } } diff --git a/lib/libvcc/vmodtool.py b/lib/libvcc/vmodtool.py index 4ee6341..570a908 100755 --- a/lib/libvcc/vmodtool.py +++ b/lib/libvcc/vmodtool.py @@ -36,6 +36,7 @@ Read the vmod.vcc file (inputvcc) and produce: # This script should work with both Python 2 and Python 3. from __future__ import print_function +import os import sys import re import optparse @@ -46,6 +47,8 @@ from os.path import dirname, exists, join, realpath from pprint import pprint, pformat from tempfile import mkstemp +rstfmt=False + ctypes = { 'ACL': "VCL_ACL", 'BACKEND': "VCL_BACKEND", @@ -85,9 +88,16 @@ def write_c_file_warning(fo): def write_rst_file_warning(fo): write_file_warning(fo, "..", "..", "..") +def write_rst_hdr(fo, s, below="-", above=None): + if above != None: + fo.write(above * len(s) + "\n") + fo.write(s + "\n") + if below != None: + fo.write(below * len(s) + "\n") + ####################################################################### -def lwrap(s, width=72): +def lwrap(s, width=64): """ Wrap a C-prototype like string into a number of lines. """ @@ -109,6 +119,11 @@ def lwrap(s, width=72): def quote(s): return s.replace("\"", "\\\"") +def indent(p,n): + n = len(p.expandtabs()) + n + p = "\t" * int(n / 8) + p += " " * int(n % 8) + return p ####################################################################### @@ -130,840 +145,691 @@ class FormatError(Exception): self.details = details Exception.__init__(self) - ####################################################################### -class Token(object): - def __init__(self, ln, ch, tokstr): - self.ln = ln - self.ch = ch - self.str = tokstr +def err(str, warn=True): + if opts.strict or not warn: + print("ERROR: " + str, file = sys.stderr) + exit(1) + raise FormatError(str, "") + else: + print("WARNING: " + str, file = sys.stderr) - def __repr__(self): - return "<@%d \"%s\">" % (self.ln, self.str) +def fmt_cstruct(fo, mn, x): + a = "\ttd_" + mn + "_" + x + while len(a.expandtabs()) < 40: + a += "\t" + fo.write("%s*%s;\n" % (a, x)) ####################################################################### -class Vmod(object): - def __init__(self, nam, dnam, sec): - if not is_c_name(nam): - raise ParseError("Module name '%s' is illegal" % nam) - self.nam = nam - self.dnam = dnam - self.sec = sec - self.event = None - self.funcs = list() - self.objs = list() - self.doc_str = [] - self.doc_order = [] - - def set_event(self, nam): - if self.event is not None: - raise ParseError("Module %s already has $Event" % - self.nam) - if not is_c_name(nam): - raise ParseError("$Event name '%s' is illegal" % nam) - self.event = nam - - def add_func(self, fn): - self.funcs.append(fn) - self.doc_order.append(fn) - - def add_obj(self, o): - self.objs.append(o) - self.doc_order.append(o) - - def c_proto(self, fo): - for o in self.objs: - fo.write("/* Object %s */\n" % o.nam) - o.fixup(self.nam) - o.c_proto(fo) - fo.write("\n") - if len(self.funcs) > 0: - fo.write("/* Functions */\n") - for f in self.funcs: - for i in lwrap(f.c_proto()): - fo.write(i + "\n") - if self.event is not None: - fo.write("\n") - fo.write("#ifdef VCL_MET_MAX\n") - fo.write("vmod_event_f " + self.event + ";\n") - fo.write("#endif\n") - - def c_typedefs_(self): - l = list() - for o in self.objs: - for t in o.c_typedefs(self.nam): - l.append(t) - l.append("") - if len(self.funcs) > 0: - l.append("/* Functions */") - for f in self.funcs: - l.append(f.c_typedef(self.nam)) - l.append("") - return l - - def c_typedefs(self, fo): - for i in self.c_typedefs_(): - for j in lwrap(i): - fo.write(j + "\n") - - def c_vmod(self, fo): - cs = self.c_struct() - fo.write(cs + ';\n') - - vfn = 'Vmod_%s_Func' % self.nam - - fo.write("/*lint -esym(754, %s::*) */\n" % vfn) - fo.write("\nstatic const struct %s Vmod_Func = " % vfn) - fo.write(self.c_initializer()) - fo.write("\n") - - fo.write("\nstatic const char Vmod_Proto[] =\n") - for t in self.c_typedefs_(): - for i in lwrap(t, width=64): - fo.write('\t"' + i + '\\n"\n') - fo.write('\t"\\n"\n') - for i in (cs + ";").split("\n"): - fo.write('\n\t"' + i + '\\n"') - fo.write('\n\t"static struct ' + vfn + " " + vfn + ';";\n\n') - - fo.write(self.c_strspec()) - - fo.write("\n") +class ctype(object): + def __init__(self, vt, ct): + self.vt = vt + self.ct = ct + self.nm = None + self.defval = None + self.spec = None + + def __str__(self): + s = "<" + self.vt + if self.nm != None: + s += " " + self.nm + if self.defval != None: + s += " VAL=" + self.defval + if self.spec != None: + s += " SPEC=" + str(self.spec) + return s + ">" + + def vcl(self): + if self.vt == "STRING_LIST": + return "STRING" + if self.spec == None: + return self.vt + return self.vt + " {" + ",".join(self.spec) + "}" + + def specstr(self, fo, p): + fo.write(p + '"' + self.vt) + fo.write('\\0"\n') + p = indent(p, 4) + if self.spec != None: + fo.write(p + '"\\1"\n') + p = indent(p, 4) + for i in self.spec: + fo.write(p + '"' + i + '\\0"\n') + p = indent(p, -4) + # This terminating \1 is necessary to ensure that + # a prototype always ends with three \0's + fo.write(p + '"\\1\\0"\n') + if self.nm != None: + fo.write(p + '"\\2" "' + self.nm + '\\0"\n') + if self.defval != None: + fo.write(p + '"\\3" "' + quote(self.defval) + '\\0"\n') + +def vtype(txt): + j = len(txt) + for i in (',', ' ', '\n', '\t'): + x = txt.find(i) + if x > 0: + j = min(j, x) + t = txt[:j] + r = txt[j:].lstrip() + if t not in ctypes: + err("Did not recognize type <%s>" % txt) + ct = ctype(t, ctypes[t]) + if t != "ENUM": + return ct, r + assert r[0] == '{' + e = r[1:].split('}', maxsplit=1) + r = e[1].lstrip() + e = e[0].split(',') + ct.spec = [] + for i in e: + ct.spec.append(i.strip()) + return ct, r + +def arg(txt): + a,s = vtype(txt) + if len(s) == 0 or s[0] == ',': + return a,s + + i = s.find('=') + j = s.find(',') + if j >= 0 and j < i: + i = -1 + if i < 0: + i = s.find(',') + if i < 0: + i = len(s) + a.nm = s[:i] + s = s[i:] + return a, s + + a.nm = s[:i] + s = s[i+1:] + if s[0] == '"' or s[0] == "'": + m = re.match("(['\"]).*?(\\1)", s) + if not m: + err("Unbalanced quote") + a.defval = s[:m.end()] + s = s[m.end():] + else: + i = s.find(',') + a.defval = s[:i] + s = s[i:] - nm = "Vmod_" + self.nam + "_Data" - fo.write("/*lint -esym(759, %s) */\n" % nm) - fo.write("const struct vmod_data " + nm + " = {\n") - fo.write("\t.vrt_major = VRT_MAJOR_VERSION,\n") - fo.write("\t.vrt_minor = VRT_MINOR_VERSION,\n") - fo.write("\t.name = \"%s\",\n" % self.nam) - fo.write("\t.func = &Vmod_Func,\n") - fo.write("\t.func_len = sizeof(Vmod_Func),\n") - fo.write("\t.proto = Vmod_Proto,\n") - fo.write("\t.spec = Vmod_Spec,\n") - fo.write("\t.abi = VMOD_ABI_Version,\n") + return a,s - # NB: Sort of hackish: - # Fill file_id with random stuff, so we can tell if - # VCC and VRT_Vmod_Init() dlopens the same file - # - fo.write("\t.file_id = \"") - for i in range(32): - fo.write("%c" % random.randint(0x40, 0x5a)) - fo.write("\",\n") - fo.write("};\n") +class prototype(object): + def __init__(self, st, retval=True, prefix=""): + l = st.line[1] + while True: + a1 = l.count("(") + a2 = l.count(")") + if a1 > 0 and a1 == a2: + break + n = st.doc.split("\n", maxsplit=1) + l += n[0] + st.doc = n[1] - def c_initializer(self): - s = '{\n' - for o in self.objs: - s += o.c_initializer() + if retval: + self.retval,s = vtype(l) + else: + self.retval = None + s = l + i = s.find("(") + assert i > 0 + self.name = prefix + s[:i].strip() + s = s[i:].strip() + assert s[0] == "(" + assert s[-1] == ")" + s = s[1:-1] + self.args = [] + while len(s) > 0: + a,s = arg(s) + self.args.append(a) + if len(s) == 0: + break; + assert s[0] == ',' + s = s[1:].lstrip() + + def cname(self): + return self.name.replace(".", "_") + + def vcl_proto(self, short): + s = "" + if self.retval != None: + s += self.retval.vcl() + " " + s += self.name + "(" + l = [] + for i in self.args: + t = i.vcl() + if not short: + if i.nm != None: + t += " " + i.nm + if i.defval != None: + t += "=" + i.defval + l.append(t) + s += ", ".join(l) + ")" + return s - s += "\n\t/* Functions */\n" - for f in self.funcs: - s += f.c_initializer() + def c_ret(self): + return self.retval.ct + + def c_args(self): + if len(self.args) == 0: + return "" + l = [""] + for i in self.args: + l.append(i.ct) + return ", ".join(l) + + def specstr(self, fo, p): + p = indent(p, 4) + if self.retval == None: + fo.write(p + '"VOID\\0"\n') + else: + self.retval.specstr(fo, p) + if self.args != None: + p = indent(p, 4) + for i in self.args: + i.specstr(fo, p) + p = indent(p, -4) + fo.write(p + '"\\0"\n') - s += "\n\t/* Init/Fini */\n" - if self.event is not None: - s += "\t" + self.event + ",\n" - s += "};" +####################################################################### - return s +class stanza(object): + def __init__(self, l0, doc, vcc): + self.line = l0 + if len(doc) == 1: + self.doc = doc[0] + else: + self.doc = "" + self.vcc = vcc + self.rstlbl = None + self.methods = None + self.proto = None + self.parse() + + def dump(self): + print(type(self), self.line) + + def rstfile(self, fo, man): + if self.rstlbl != None: + fo.write(".. _" + self.rstlbl + ":\n\n") + + self.rsthead(fo, man) + self.rstmid(fo, man) + self.rsttail(fo, man) + + def rsthead(self, fo, man): + if self.proto == None: + return + if rstfmt: + s = self.proto.vcl_proto(short=False) + write_rst_hdr(fo, s, '-') + else: + write_rst_hdr(fo, self.proto.name, '-') + s = self.proto.vcl_proto(short=False) + fo.write("\n::\n\n\t%s\n" % s) - def c_struct(self): - s = 'struct Vmod_' + self.nam + '_Func {\n' - for o in self.objs: - s += o.c_struct(self.nam) + def rstmid(self, fo, man): + fo.write(self.doc + "\n") - s += "\n\t/* Functions */\n" - for f in self.funcs: - s += f.c_struct(self.nam) + def rsttail(self, fo, man): + return - s += "\n\t/* Init/Fini */\n" - if self.event is not None: - s += "\tvmod_event_f\t*_event;\n" - s += '}' - return s + def hfile(self, fo): + return - def c_strspec(self): - s = "/*lint -save -e786 -e840 */\n" - s += "static const char * const Vmod_Spec[] = {\n" + def cstruct(self, fo): + return - for o in self.objs: - s += o.c_strspec(self.nam) + ",\n\n" + def cstruct_init(self, fo): + return - if len(self.funcs) > 0: - s += "\t/* Functions */\n" - for f in self.funcs: - s += f.c_strspec(self.nam) + ',\n\n' + def specstr(self, fo): + return - if self.event is not None: - s += "\t/* Init/Fini */\n" - s += '\t"$EVENT\\0Vmod_' + self.nam + '_Func._event",\n' +####################################################################### - s += "\t0\n" - s += "};\n" - s += "/*lint -restore */\n" - return s +class s_module(stanza): + def parse(self): + a = self.line[1].split(maxsplit=2) + self.vcc.modname = a[0] + self.vcc.mansection = a[1] + self.vcc.moddesc = a[2] + self.rstlbl = "vmod_%s(%s)" % ( + self.vcc.modname, + self.vcc.mansection + ) + self.vcc.contents.append(self) + + def rsthead(self, fo, man): + + write_rst_hdr(fo, "vmod_" + self.vcc.modname, "=", "=") + fo.write("\n") - def doc(self, l): - self.doc_str.append(l) + write_rst_hdr(fo, self.vcc.moddesc, "-", "-") - def doc_dump(self, fo, suf): - fo.write(".. role:: ref(emphasis)\n\n") - i = "vmod_" + self.nam - fo.write(".. _" + i + "(" + self.sec + "):\n\n") - fo.write("=" * len(i) + "\n") - fo.write(i + "\n") - fo.write("=" * len(i) + "\n") - fo.write("\n") - i = self.dnam - fo.write("-" * len(i) + "\n") - fo.write(i + "\n") - fo.write("-" * len(i) + "\n") fo.write("\n") - fo.write(":Manual section: %s\n" % self.sec) + fo.write(":Manual section: " + self.vcc.mansection + "\n") + fo.write("\n") - fo.write("SYNOPSIS\n") - fo.write("========\n") + write_rst_hdr(fo, "SYNOPSIS", "=") fo.write("\n") - fo.write("import %s [from \"path\"] ;\n" % self.nam) + fo.write('import %s [from "path"] ;\n' % self.vcc.modname) fo.write("\n") - for i in self.doc_str: - fo.write(i + "\n") - fo.write("CONTENTS\n") - fo.write("========\n") + + def rsttail(self, fo, man): + + write_rst_hdr(fo, "CONTENTS", "=") fo.write("\n") + + if man: + for i in self.vcc.contents[1:]: + if i.rstlbl == None: + continue + fo.write("* %s\n" % + i.proto.vcl_proto(short=True)) + fo.write("\n") + return + l = [] - for i in self.funcs: - l.append(i.doc_idx(suf)) - for i in self.objs: - l += i.doc_idx(suf) + for i in self.vcc.contents[1:]: + j = i.rstlbl + if j != None: + l.append([j.split("_", maxsplit=1)[1], j]) + if i.methods == None: + continue + for x in i.methods: + j = x.rstlbl + l.append([j.split("_", maxsplit=1)[1], j]) + l.sort() for i in l: - fo.write("* " + i[1] + "\n") + fo.write("* :ref:`%s`\n" % i[1]) fo.write("\n") - for i in self.doc_order: - i.doc_dump(fo) -####################################################################### +class s_event(stanza): + def parse(self): + self.event_func = self.line[1] + self.vcc.contents.append(self) -class Func(object): - def __init__(self, nam, retval, al): - #if not is_c_name(nam): - # raise Exception("Func name '%s' is illegal" % nam) - if retval not in ctypes: - raise TypeError( - "Return type '%s' not a valid type", retval) - self.nam = nam - self.cnam = nam.replace(".", "_") - self.al = al - self.retval = retval - self.pfx = None - self.doc_str = [] - - def __repr__(self): - return "" % (self.retval, self.nam) - - def set_pfx(self, s): - self.pfx = s - - def c_proto(self, fini=False): - s = ctypes[self.retval] + " vmod_" + self.cnam + "(" - p = "" - if not fini: - s += "VRT_CTX" - p = ", " - if self.pfx is not None: - s += p + self.pfx - p = ", " - for a in self.al: - s += p + ctypes[a.typ] - p = ", " - s += ");" - return s - - def c_typedef(self, modname, fini=False): - s = "typedef " - s += ctypes[self.retval] - s += " td_" + modname + "_" + self.cnam + "(" - p = "" - if not fini: - s += "VRT_CTX" - p = ", " - if self.pfx is not None: - s += p + self.pfx - p = ", " - for a in self.al: - s += p + ctypes[a.typ] - p = ", " - s += ");" - return s + def rstfile(self, fo, man): + return - def c_struct(self, modname): - s = '\ttd_' + modname + "_" + self.cnam - if len(s.expandtabs()) >= 40: - s += "\n\t\t\t\t\t" - else: - while len(s.expandtabs()) < 40: - s += "\t" - s += "*" + self.cnam + ";\n" - return s + def hfile(self, fo): + fo.write("#ifdef VCL_MET_MAX\n") + fo.write("vmod_event_f %s;\n" % self.event_func) + fo.write("#endif\n") - def c_initializer(self): - return "\tvmod_" + self.cnam + ",\n" - - def c_strspec(self, modnam, pfx="\t"): - s = pfx + '"' + modnam + "." + self.nam + '\\0"\n' - s += pfx + '"' - s += "Vmod_" + modnam + "_Func." + self.cnam + '\\0"\n' - s += pfx + ' "' + self.retval + '\\0"\n' - for a in self.al: - s += pfx + '\t"' + a.c_strspec() + '"\n' - s += pfx + '"\\0"' - return s + def cstruct(self, fo): + fo.write("\tvmod_event_f\t\t\t*_event;\n") - def doc(self, l): - self.doc_str.append(l) + def cstruct_init(self, fo): + fo.write("\t%s,\n" % self.event_func) - def doc_proto(self): - s = self.retval + " " + self.nam + "(" - d = "" - for i in self.al: - s += d + i.typ - d = ", " - s += ")" - return s + def specstr(self, fo): + fo.write('\t"$EVENT\\0"\n\t "Vmod_%s_Func._event",\n' % + self.vcc.modname) - def doc_idx(self, suf): - if suf == "": - return (self.nam, ":ref:`func_" + self.nam + "`") - else: - return (self.nam, self.doc_proto()) +class s_function(stanza): + def parse(self): + self.proto = prototype(self) + self.rstlbl = "func_" + self.proto.name + self.vcc.contents.append(self) - def doc_dump(self, fo): - s = self.doc_proto() - fo.write(".. _func_" + self.nam + ":\n\n") - fo.write(s + "\n") - fo.write("-" * len(s) + "\n") - fo.write("\n") - fo.write("Prototype\n") - s = "\t" + self.retval + " " + self.nam + "(" - d = "" - for i in self.al: - s += d + i.typ - if i.nam is not None: - s += " " + i.nam - d = ", " - fo.write(s + ")\n") - for i in self.doc_str: + def hfile(self, fo): + fn = "vmod_" + self.proto.name + s = "%s %s(VRT_CTX" % (self.proto.c_ret(), fn) + s += self.proto.c_args() + ");" + for i in lwrap(s): fo.write(i + "\n") - -####################################################################### - -class Obj(object): - def __init__(self, nam): - self.nam = nam - self.init = None - self.fini = None - self.methods = list() - self.doc_str = [] - self.st = None - - def fixup(self, modnam): - assert self.nam is not None - self.st = "struct vmod_" + modnam + "_" + self.nam - self.init.set_pfx(self.st + " **, const char *") - self.fini.set_pfx(self.st + " **") - for m in self.methods: - m.set_pfx(self.st + " *") - - def set_init(self, f): - self.init = f - self.fini = Func(f.nam, "VOID", []) - self.init.cnam += "__init" - self.fini.cnam += "__fini" - - def add_method(self, m): - self.methods.append(m) - - def c_typedefs(self, modnam): - l = list() - l.append("/* Object " + self.nam + " */") - l.append(self.st + ";") - l.append(self.init.c_typedef(modnam) + "") - l.append(self.fini.c_typedef(modnam, fini=True) + "") - for m in self.methods: - l.append(m.c_typedef(modnam) + "") - return l - - def c_proto(self, fo): - fo.write(self.st + ";\n") - l = [] - l += lwrap(self.init.c_proto()) - l += lwrap(self.fini.c_proto(fini=True)) - for m in self.methods: - l += lwrap(m.c_proto()) - for i in l: + def cfile(self, fo): + fn = "td_" + self.vcc.modname + "_" + self.proto.name + s = "typedef %s %s(VRT_CTX" % (self.proto.c_ret(), fn) + s += self.proto.c_args() + ");" + for i in lwrap(s): fo.write(i + "\n") - def c_struct(self, modnam): - s = "\t/* Object " + self.nam + " */\n" - s += self.init.c_struct(modnam) - s += self.fini.c_struct(modnam) - for m in self.methods: - s += m.c_struct(modnam) - return s + def cstruct(self, fo): + fmt_cstruct(fo, self.vcc.modname, self.proto.cname()) - def c_initializer(self): - s = "\t/* Object " + self.nam + " */\n" - s += self.init.c_initializer() - s += self.fini.c_initializer() - for m in self.methods: - s += m.c_initializer() - return s + def cstruct_init(self, fo): + fo.write("\tvmod_" + self.proto.cname() + ",\n") - def c_strspec(self, modnam): - s = "\t/* Object " + self.nam + " */\n" - s += '\t"$OBJ\\0"\n' - s += self.init.c_strspec(modnam, pfx="\t\t") + '\n' - s += '\t\t"' + self.st + '\\0"\n' - s += self.fini.c_strspec(modnam, pfx="\t\t") + '\n' - for m in self.methods: - s += m.c_strspec(modnam, pfx="\t\t") + '\n' - s += '\t"\\0"' - return s + def specstr(self, fo): + fo.write('\t"$FUNC\\0"\n\t "%s.%s\\0"\n' % + (self.vcc.modname, self.proto.name)) + fo.write('\t\t"Vmod_%s_Func.%s\\0"\n' % + (self.vcc.modname, self.proto.cname())) + self.proto.specstr(fo, "\t\t") - def doc(self, l): - self.doc_str.append(l) + fo.write('\t"\\0",\n\n') - def doc_idx(self, suf): - l = [] - if suf == "": - l.append((self.nam, ":ref:`obj_" + self.nam + "`")) +class s_object(stanza): + def parse(self): + self.proto = prototype(self, retval=False) + self.rstlbl = "obj_" + self.proto.name + self.vcc.contents.append(self) + self.methods = [] + + def rsthead(self, fo, man): + if rstfmt: + s = self.proto.vcl_proto(short=False) + write_rst_hdr(fo, "new OBJ = " + s, '=') else: - l.append((self.nam, "Object " + self.nam)) - for i in self.methods: - l.append(i.doc_idx(suf)) - return l - - def doc_dump(self, fo): - fo.write(".. _obj_" + self.nam + ":\n\n") - s = "Object " + self.nam - fo.write(s + "\n") - fo.write("=" * len(s) + "\n") - fo.write("\n") + write_rst_hdr(fo, self.proto.name, '-') + s = "new OBJ = " + self.proto.vcl_proto(short=False) + fo.write("\n::\n\n\t%s\n" % s) - for i in self.doc_str: - fo.write(i + "\n") + fo.write(self.doc + "\n") for i in self.methods: - i.doc_dump(fo) + i.rstfile(fo, man) -####################################################################### - -class Arg(object): - def __init__(self, typ, nam=None, det=None): - self.nam = nam - self.typ = typ - self.det = det - self.val = None + def rstmid(self, fo, man): + return - def __repr__(self): - return "" % (self.nam, self.typ, str(self.det)) + def chfile(self, fo, h): + sn = "vmod_" + self.vcc.modname + "_" + self.proto.name + fo.write("struct %s;\n" % sn) - def c_strspec(self): - if self.det is None: - s = self.typ + "\\0" + if h: + def p(x): + return x + " vmod_" else: - s = self.det - if self.nam is not None: - s += '"\n\t\t "\\1' + self.nam + '\\0' - if self.val is not None: - # The space before the value is important to - # terminate the \2 escape sequence - s += '"\n\t\t\t"\\2 ' + quote(self.val) + "\\0" - return s - -####################################################################### -# -# + def p(x): + return "typedef " + x + \ + " td_%s_" % self.vcc.modname + + s = p("VCL_VOID") + "%s__init(VRT_CTX, " % self.proto.name + s += "struct %s **, const char *" % sn + s += self.proto.c_args() + ");" + for i in lwrap(s): + fo.write(i + "\n") -def parse_enum2(tl): - t = tl.get_token() - if t.str != "{": - raise ParseError("expected \"{\"") - s = "ENUM\\0" - t = None - while True: - if t is None: - t = tl.get_token() - if t.str == "}": - break - s += t.str + "\\0" - t = tl.get_token() - if t.str == ",": - t = None - elif t.str == "}": - break - else: - raise ParseError( - "Expected \"}\" or \",\" not \"%s\"" % t.str) - s += "\\0" - return Arg("ENUM", det=s) + s = p("VCL_VOID") + s += "%s__fini(struct %s **);" % (self.proto.name, sn) + for i in lwrap(s): + fo.write(i + "\n") + for i in self.methods: + cn = i.proto.cname() + s = p(i.proto.c_ret()) + s += "%s(VRT_CTX, struct %s *" % (cn, sn) + s += i.proto.c_args() + ");" + for i in lwrap(s): + fo.write(i + "\n") + fo.write("\n") -def parse_arg(tl, al): - t = tl.get_token() - assert t is not None + def hfile(self, fo): + self.chfile(fo, True) - if t.str == ")": - return t + def cfile(self, fo): + self.chfile(fo, False) - if t.str == "ENUM": - al.append(parse_enum2(tl)) - elif t.str in ctypes: - al.append(Arg(t.str)) - else: - raise Exception("ARG? %s", t.str) + def cstruct(self, fo): + td = "td_" + self.vcc.modname + "_" + self.proto.name + "_" + fmt_cstruct(fo, self.vcc.modname, self.proto.name + "__init") + fmt_cstruct(fo, self.vcc.modname, self.proto.name + "__fini") + for i in self.methods: + i.cstruct(fo) - t = tl.get_token() - if t.str == "," or t.str == ")": - return t + def cstruct_init(self, fo): + p = "\tvmod_" + fo.write(p + self.proto.name + "__init,\n") + fo.write(p + self.proto.name + "__fini,\n") + for i in self.methods: + i.cstruct_init(fo) + fo.write("\n") - if not is_c_name(t.str): - raise ParseError( - 'Expected ")", "," or argument name, not "%s"' % t.str) + def specstr(self, fo): - al[-1].nam = t.str - t = tl.get_token() + fo.write('\t"$OBJ\\0"\n\t "%s.%s\\0"\n' % + (self.vcc.modname, self.proto.name)) - if t.str == "," or t.str == ")": - return t + fo.write('\t\t"struct vmod_%s_%s\\0"\n' % + (self.vcc.modname, self.proto.name)) + fo.write("\n") - if t.str != "=": - raise ParseError( - 'Expected ")", "," or "=", not "%s"' % t.str) + fo.write('\t\t"Vmod_%s_Func.%s__init\\0"\n' % + (self.vcc.modname, self.proto.name)) + self.proto.specstr(fo, '\t\t') + fo.write('\t\t"\\0"\n\n') - t = tl.get_token() - al[-1].val = t.str + fo.write('\t\t"Vmod_%s_Func.%s__fini\\0"\n' % + (self.vcc.modname, self.proto.name)) + fo.write('\t\t "VOID\\0"\n') + fo.write('\t\t "\\0"\n') + fo.write('\t\t"\\0"\n\n') - t = tl.get_token() - return t + for i in self.methods: + i.specstr(fo) -####################################################################### -# -# + fo.write('\t"\\0",\n\n') -def parse_module(tl): - nm = tl.get_token().str - sec = tl.get_token().str - s = "" - while len(tl.tl) > 0: - s += " " + tl.get_token().str - dnm = s[1:] - return Vmod(nm, dnm, sec) - - -def parse_func(tl, rt_type=None, pobj=None): - al = list() - if rt_type is None: - t = tl.get_token() - rt_type = t.str - if rt_type not in ctypes: - raise TypeError( - "Return type '%s' not a valid type" % rt_type) - - t = tl.get_token() - fname = t.str - if pobj is not None and fname[0] == "." and is_c_name(fname[1:]): - fname = pobj + fname - elif pobj is not None and fname[0] != ".": - raise ParseError("Method name '%s' must start with ." % fname) - elif pobj is not None and not is_c_name(fname[1:]): - raise ParseError("Method name '%s' is illegal" % fname[1:]) - elif not is_c_name(fname): - raise ParseError("Function name '%s' is illegal" % fname) - - t = tl.get_token() - if t.str != "(": - raise ParseError("Expected \"(\" got \"%s\"" % t.str) - - while True: - t = parse_arg(tl, al) - if t.str == ")": - break - if t.str != ",": - raise ParseError("End Of Input looking for ')' or ','") + def dump(self): + super(s_object, self).dump() + for i in self.methods: + i.dump() + +class s_method(stanza): + def parse(self): + p = self.vcc.contents[-1] + assert type(p) == s_object + self.proto = prototype(self, prefix=p.proto.name) + self.rstlbl = "func_" + self.proto.name + p.methods.append(self) + + def cstruct(self, fo): + fmt_cstruct(fo, self.vcc.modname, self.proto.cname()) + + def cstruct_init(self, fo): + fo.write('\t' + "vmod_" + self.proto.cname() + ",\n") + + def specstr(self, fo): + fo.write('\t\t"%s.%s\\0"\n' % + (self.vcc.modname, self.proto.name)) + fo.write('\t\t "Vmod_%s_Func.%s\\0"\n' % + (self.vcc.modname, self.proto.cname())) + self.proto.specstr(fo, '\t\t ') + fo.write('\t\t"\\0"\n\n') - f = Func(fname, rt_type, al) +####################################################################### - return f +class vcc(object): + def __init__(self, inputvcc, rstdir, outputprefix): + self.inputfile = inputvcc + self.rstdir = rstdir + self.pfx = outputprefix + self.contents = [] + self.commit_files = [] + + def commit(self): + for i in self.commit_files: + os.rename(i + ".tmp", i) + + def parse(self): + a = open(self.inputfile, "r").read() + a = a.split("\n$") + for i in range(len(a)): + b = a[i].split("\n", maxsplit=1) + if i == 0: + self.copyright = a[0] + continue + c = b[0].split(maxsplit=1) + if i == 1: + if c[0] != "Module": + err("$Module must be first stanze") + if c[0] == "Module": + s_module(c, b[1:], self) + elif c[0] == "Event": + s_event(c, b[1:], self) + elif c[0] == "Function": + s_function(c, b[1:], self) + elif c[0] == "Object": + s_object(c, b[1:], self) + elif c[0] == "Method": + s_method(c, b[1:], self) + else: + err("Unknown stanze $%s" % c[0]) + + def rst_copyright(self, fo): + write_rst_hdr(fo, "COPYRIGHT", "=") + fo.write("\n::\n\n") + a = self.copyright + a = a.replace("\n#", "\n ") + if a[:2] == "#\n": + a = a[2:] + if a[:3] == "#-\n": + a = a[3:] + fo.write(a + "\n") + + def rstfile(self, man=False): + fn = self.rstdir + "/vmod_" + self.modname + if man: + fn += ".man" + fn += ".rst" + self.commit_files.append(fn) + fo = open(fn + ".tmp", "w") + write_rst_file_warning(fo) + fo.write(".. role:: ref(emphasis)\n\n") -####################################################################### -# -# + for i in self.contents: + i.rstfile(fo, man) -def parse_obj(tl): - f = parse_func(tl, "VOID") - o = Obj(f.nam) - o.set_init(f) - return o + self.rst_copyright(fo) + fo.close() -####################################################################### + def hfile(self): + fn = self.rstdir + "/" + self.pfx + ".h" + self.commit_files.append(fn) + fo = open(fn + ".tmp", "w") + write_c_file_warning(fo) + fo.write("struct vmod_priv;\n") + fo.write("\n") + fo.write("extern const struct vmod_data Vmod_debug_Data;\n") + fo.write("\n") -class FileSection(object): - """ - A section of the inputvcc, starting at a keyword. - """ - def __init__(self): - self.l = [] - self.tl = [] + for j in self.contents: + j.hfile(fo) + fo.close() - def add_line(self, ln, l): - self.l.append((ln, l)) + def cstruct(self, fo, csn): - def get_token(self): - while True: - if len(self.tl) > 0: - # print("T\t", self.tl[0]) - return self.tl.pop(0) - if len(self.l) == 0: - break - self.more_tokens() - return None + fo.write("\n%s {\n" % csn) + for j in self.contents: + j.cstruct(fo) + fo.write("};\n") - def more_tokens(self): - ln, l = self.l.pop(0) - if l == "": - return - l = re.sub("[ \t]*#.*$", "", l) - l = re.sub("[ \t]*\n", "", l) - - if re.match("['\"]", l): - m = re.match("(['\"]).*?(\\1)", l) - if not m: - raise FormatError("Unbalanced quote", - "Unbalanced quote on line %d" % ln) - self.tl.append(Token(ln, 0, l[:m.end()])) - self.l.insert(0, (ln, l[m.end():])) - return + def cstruct_init(self, fo, csn): + fo.write("\nstatic const %s Vmod_Func = {\n" % csn) + for j in self.contents: + j.cstruct_init(fo) + fo.write("};\n") - m = re.search("['\"]", l) - if m: - rest = l[m.start():] - self.l.insert(0, (ln, rest)) - l = l[:m.start()] + def specstr(self, fo): + fo.write("\n/*lint -save -e786 -e840 */\n") + fo.write("static const char * const Vmod_Spec[] = {\n") - l = re.sub("([(){},=])", r' \1 ', l) - if l == "": - return - for j in l.split(): - self.tl.append(Token(ln, 0, j)) + for j in self.contents: + j.specstr(fo) + fo.write("\t0\n") + fo.write("};\n") + fo.write("/*lint -restore */\n") + + def api(self, fo): + fo.write("\n/*lint -esym(759, Vmod_debug_Data) */\n") + fo.write("const struct vmod_data Vmod_%s_Data = {\n" % + self.modname) + fo.write("\t.vrt_major =\tVRT_MAJOR_VERSION,\n") + fo.write("\t.vrt_minor =\tVRT_MINOR_VERSION,\n") + fo.write('\t.name =\t\t"%s",\n' % self.modname) + fo.write('\t.func =\t\t&Vmod_Func,\n') + fo.write('\t.func_len =\tsizeof(Vmod_Func),\n') + fo.write('\t.proto =\tVmod_Proto,\n') + fo.write('\t.spec =\t\tVmod_Spec,\n') + fo.write('\t.abi =\t\tVMOD_ABI_Version,\n') + # NB: Sort of hackish: + # Fill file_id with random stuff, so we can tell if + # VCC and VRT_Vmod_Init() dlopens the same file + # + fo.write("\t.file_id =\t\"") + for i in range(32): + fo.write("%c" % random.randint(0x40, 0x5a)) + fo.write("\",\n") + fo.write("};\n") - def parse(self, vx): - t = self.get_token() - if t is None: - return - t0 = t.str - if t.str == "$Module": - o = parse_module(self) - vx.append(o) - elif t.str == "$Event": - x = self.get_token() - vx[0].set_event(x.str) - o = None - elif t.str == "$Function": - if len(vx) == 2: - vx.pop(-1) - o = parse_func(self) - vx[0].add_func(o) - elif t.str == "$Object": - if len(vx) == 2: - vx.pop(-1) - o = parse_obj(self) - vx[0].add_obj(o) - vx.append(o) - elif t.str == "$Method": - if len(vx) != 2: - raise FormatError("$Method outside $Object", "") - o = parse_func(self, pobj=vx[1].nam) - vx[1].add_method(o) - else: - if opts.strict: - raise FormatError("Unknown keyword: %s" % - t.str, "") - else: - print("WARNING: Unknown keyword: %s:" % - t.str, file=sys.stderr) - o = None - while len(self.tl) > 0: - self.get_token() - - assert len(self.tl) == 0 - if o is None and len(self.l) > 0: - m = "%s description is not included in .rst" % t0 - details = pformat(self.l) - if opts.strict: - raise FormatError(m, details) - else: - print("WARNING: %s:" % m, file=sys.stderr) - print(details, file=sys.stderr) - else: - for ln, i in self.l: - o.doc(i) + def cfile(self): + fn = self.rstdir + "/" + self.pfx + ".c" + self.commit_files.append(fn) + fo = open(fn + ".tmp", "w") + write_c_file_warning(fo) -####################################################################### + fn2 = fn + ".tmp2" -def polish(l): - """ - Polish the copyright message. - """ - if len(l[0]) == 0: - l.pop(0) - return True - c = l[0][0] - for i in l: - if len(i) == 0: - continue - if i[0] != c: - c = None - break - if c is not None: - for i in range(len(l)): - l[i] = l[i][1:] - return True - return False + for i in ["config", "vcl", "vrt", "vcc_if", "vmod_abi"]: + fo.write('#include "%s.h"\n' % i) + fo.write("\n") -class SimpleTestCase(unittest.TestCase): - def test_included_vccs(self): - from tempfile import mktemp - from glob import glob - tmpfile = mktemp() - bdir = dirname(realpath(__file__)) - for inputfile in glob(join(bdir, "../libvmod_*/vmod.vcc")): - runmain(inputfile, ".", tmpfile) - unlink(tmpfile + ".c") - unlink(tmpfile + ".h") + fx = open(fn2, "w") + for i in self.contents: + if type(i) == s_object: + i.cfile(fo) + i.cfile(fx) -####################################################################### -def runmain(inputvcc, rstdir, outputprefix): - # Read the file in - lines = [] - with open(inputvcc, "r") as fp: - for i in fp: - lines.append(i.rstrip()) - ln = 0 - - ####################################################################### - # First collect the copyright: All initial lines starting with '#' - - copy_right = [] - while len(lines[0]) > 0 and lines[0][0] == "#": - ln += 1 - copy_right.append(lines.pop(0)) - - if len(copy_right) > 0: - if copy_right[0] == "#-": - copy_right = [] - else: - while polish(copy_right): - continue + fx.write("/* Functions */\n") + for i in self.contents: + if type(i) == s_function: + i.cfile(fo) + i.cfile(fx) - if False: - for i in copy_right: - print("(C)\t", i) - - ####################################################################### - # Break into sections - - sl = [] - sc = FileSection() - sl.append(sc) - while len(lines) > 0: - ln += 1 - l = lines.pop(0) - j = l.split() - if len(j) > 0 and re.match("^\$", j[0]): - sc = FileSection() - sl.append(sc) - sc.add_line(ln, l) - - ####################################################################### - # Parse each section - - try: - vx = [] - for i in sl: - i.parse(vx) - assert len(i.tl) == 0 - except ParseError as e: - print("ERROR: Parse error reading \"%s\":" % inputvcc) - pprint(str(e)) - exit(-1) - except FormatError as e: - print("ERROR: Format error reading \"%s\": %s" % - (inputvcc, pformat(e.msg))) - print(e.details) - exit(-2) + csn = "Vmod_%s_Func" % self.modname - ####################################################################### - # Parsing done, now process - # - fd_cfile, fn_cfile = mkstemp(dir='.') - cfile = fdopen(fd_cfile, "w") + self.cstruct(fo, "struct " + csn) - fd_headerfile, fn_headerfile = mkstemp(dir='.') - headerfile = fdopen(fd_headerfile, "w") + self.cstruct(fx, "struct " + csn) - write_c_file_warning(cfile) - write_c_file_warning(headerfile) + fo.write("\n/*lint -esym(754, Vmod_debug_Func::*) */\n") + self.cstruct_init(fo, "struct " + csn) - headerfile.write('struct vmod_priv;\n\n') + fx.close() - headerfile.write('extern const struct vmod_data Vmod_%s_Data;\n\n' % vx[0].nam) + fo.write("\nstatic const char Vmod_Proto[] =\n") + fi = open(fn2) + for i in fi: + fo.write('\t"%s\\n"\n' % i.rstrip()) + fi.close() + fo.write('\t"static struct %s %s;";\n' % (csn, csn)) - vx[0].c_proto(headerfile) + os.remove(fn2) - cfile.write('#include "config.h"\n') - cfile.write('#include "vcl.h"\n') - cfile.write('#include "vrt.h"\n') - cfile.write('#include "%s.h"\n' % outputprefix) - cfile.write('#include "vmod_abi.h"\n') - cfile.write('\n') + self.specstr(fo) - vx[0].c_typedefs(cfile) - vx[0].c_vmod(cfile) + self.api(fo) - cfile.close() - headerfile.close() + fo.close() - for suf in ("", ".man"): - fd, fn_fp = mkstemp(dir='.') - fp = fdopen(fd, "w") - write_rst_file_warning(fp) +####################################################################### - vx[0].doc_dump(fp, suf) +def runmain(inputvcc, rstdir, outputprefix): - if len(copy_right) > 0: - fp.write("\n") - fp.write("COPYRIGHT\n") - fp.write("=========\n") - fp.write("\n::\n\n") - for i in copy_right: - fp.write(" %s\n" % i) - fp.write("\n") - fp.close() - rename(fn_fp, join(rstdir, "vmod_%s%s.rst" % (vx[0].nam, suf))) + v = vcc(inputvcc, rstdir, outputprefix) + v.parse() - # Our makefile targets the .c file so make sure to put that in place last. - rename(fn_headerfile, outputprefix + ".h") - rename(fn_cfile, outputprefix + ".c") + v.rstfile(man=False) + v.rstfile(man=True) + v.hfile() + v.cfile() + v.commit() if __name__ == "__main__": usagetext = "Usage: %prog [options] " From phk at FreeBSD.org Wed Jun 1 15:17:07 2016 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Wed, 01 Jun 2016 17:17:07 +0200 Subject: [master] 5de0971 Be python2 compatible Message-ID: commit 5de0971230b22b6ada0c70cc6a1ca575018b3667 Author: Poul-Henning Kamp Date: Wed Jun 1 15:16:24 2016 +0000 Be python2 compatible diff --git a/lib/libvcc/vmodtool.py b/lib/libvcc/vmodtool.py index 570a908..9525f2d 100755 --- a/lib/libvcc/vmodtool.py +++ b/lib/libvcc/vmodtool.py @@ -220,7 +220,7 @@ def vtype(txt): if t != "ENUM": return ct, r assert r[0] == '{' - e = r[1:].split('}', maxsplit=1) + e = r[1:].split('}', 1) r = e[1].lstrip() e = e[0].split(',') ct.spec = [] @@ -268,7 +268,7 @@ class prototype(object): a2 = l.count(")") if a1 > 0 and a1 == a2: break - n = st.doc.split("\n", maxsplit=1) + n = st.doc.split("\n", 1) l += n[0] st.doc = n[1] @@ -396,7 +396,7 @@ class stanza(object): class s_module(stanza): def parse(self): - a = self.line[1].split(maxsplit=2) + a = self.line[1].split(None, 2) self.vcc.modname = a[0] self.vcc.mansection = a[1] self.vcc.moddesc = a[2] @@ -440,12 +440,12 @@ class s_module(stanza): for i in self.vcc.contents[1:]: j = i.rstlbl if j != None: - l.append([j.split("_", maxsplit=1)[1], j]) + l.append([j.split("_", 1)[1], j]) if i.methods == None: continue for x in i.methods: j = x.rstlbl - l.append([j.split("_", maxsplit=1)[1], j]) + l.append([j.split("_", 1)[1], j]) l.sort() for i in l: @@ -657,11 +657,11 @@ class vcc(object): a = open(self.inputfile, "r").read() a = a.split("\n$") for i in range(len(a)): - b = a[i].split("\n", maxsplit=1) + b = a[i].split("\n", 1) if i == 0: self.copyright = a[0] continue - c = b[0].split(maxsplit=1) + c = b[0].split(None, 1) if i == 1: if c[0] != "Module": err("$Module must be first stanze") From guillaume at varnish-software.com Thu Jun 2 12:31:07 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Thu, 02 Jun 2016 14:31:07 +0200 Subject: [master] dfb5df8 Simplify r01941.vtc a bit Message-ID: commit dfb5df8934f5ffa31f5d969791aa3da5e469b626 Author: Guillaume Quintard Date: Wed Jun 1 19:11:54 2016 +0200 Simplify r01941.vtc a bit diff --git a/bin/varnishtest/tests/r01941.vtc b/bin/varnishtest/tests/r01941.vtc index 2531ded..188125c 100644 --- a/bin/varnishtest/tests/r01941.vtc +++ b/bin/varnishtest/tests/r01941.vtc @@ -11,100 +11,13 @@ server s1 { expect req.url == /big txresp -bodylen 1037480 rxreq - txresp -body { - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -} + txresp -nolen -hdr "Content-Length: 6545" -body "" + loop 92 { + send {} + } + send "" rxreq expect req.url == /long1234567890123456789012345678901234567890.txt txresp -body {fo} From phk at FreeBSD.org Fri Jun 3 14:54:07 2016 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Fri, 03 Jun 2016 16:54:07 +0200 Subject: [master] 160e1f1 Fix comment Message-ID: commit 160e1f1ab2a29d6b3720807ea5ebfae6b0c701da Author: Poul-Henning Kamp Date: Thu Jun 2 07:15:01 2016 +0000 Fix comment diff --git a/lib/libvcc/vcc_token.c b/lib/libvcc/vcc_token.c index c4771bd..3113722 100644 --- a/lib/libvcc/vcc_token.c +++ b/lib/libvcc/vcc_token.c @@ -457,7 +457,7 @@ vcc_Lexer(struct vcc *tl, struct source *sp) return; } - /* Match for the fixed tokens (see token.tcl) */ + /* Match for the fixed tokens (see generate.py) */ u = vcl_fixed_token(p, &q); if (u != 0) { vcc_AddToken(tl, u, p, q); From hermunn at varnish-software.com Fri Jun 3 15:05:07 2016 From: hermunn at varnish-software.com (Pål Hermunn Johansen) Date: Fri, 03 Jun 2016 17:05:07 +0200 Subject: [4.1] f07287a Correctly handle HTTP/1.1 EOF response Message-ID: commit f07287a6a3a5ebe797346e97ff0b464df5b07a3e Author: Brandon Black Date: Tue May 17 21:37:31 2016 +0000 Correctly handle HTTP/1.1 EOF response Commit e142a199c for Issue #1918 fixed up the case where the server sends an HTTP/1.1 response with no Content-Length, no Transfer-Encoding, but with Connection:close. This fixes the very similar case where all the conditions are the same except that there's also no Connection: close header, but the content is still implicitly delimited by the server closing the connection. This behavior has been observed from multiple versions of Apache with WSGI applications behind it, resulting in broken Content-Length:0 responses from Varnish. Ref: varnishcache/varnish-cache#1954 diff --git a/bin/varnishd/http1/cache_http1_proto.c b/bin/varnishd/http1/cache_http1_proto.c index 67d7b2b..ebf0d2a 100644 --- a/bin/varnishd/http1/cache_http1_proto.c +++ b/bin/varnishd/http1/cache_http1_proto.c @@ -307,7 +307,7 @@ http1_body_status(const struct http *hp, struct http_conn *htc, int request) return (cl == 0 ? BS_NONE : BS_LENGTH); } - if (hp->protover == 11 && (request || !http_HdrIs(hp, H_Connection, "close"))) + if (hp->protover == 11 && request) return (BS_NONE); if (http_HdrIs(hp, H_Connection, "keep-alive")) { diff --git a/bin/varnishtest/tests/r01918.vtc b/bin/varnishtest/tests/r01918.vtc index c0593aa..68a4d72 100644 --- a/bin/varnishtest/tests/r01918.vtc +++ b/bin/varnishtest/tests/r01918.vtc @@ -3,19 +3,15 @@ varnishtest "Check EOF responses work with HTTP/1.1" server s1 { rxreq txresp -nolen -bodylen 10 - rxreq - txresp -hdr "Connection: close" -nolen -bodylen 10 + close + accept } -start varnish v1 -vcl+backend { } -start client c1 { - txreq -url /1 - rxresp - expect resp.status == 200 - expect resp.bodylen == 0 - txreq -url /2 + txreq rxresp expect resp.status == 200 expect resp.bodylen == 10 From fgsch at lodoss.net Sat Jun 4 16:19:05 2016 From: fgsch at lodoss.net (Federico G. Schwindt) Date: Sat, 04 Jun 2016 18:19:05 +0200 Subject: [master] fd21faf Handle synopsis and options via the vut code Message-ID: commit fd21faff891451b8a1699b2bdb5c1f11977d2df2 Author: Federico G. Schwindt Date: Sat Jun 4 16:26:26 2016 +0100 Handle synopsis and options via the vut code diff --git a/bin/varnishstat/Makefile.am b/bin/varnishstat/Makefile.am index 5c0a179..82303fa 100644 --- a/bin/varnishstat/Makefile.am +++ b/bin/varnishstat/Makefile.am @@ -8,9 +8,10 @@ bin_PROGRAMS = varnishstat varnishstat_SOURCES = \ varnishstat.h \ - \ varnishstat.c \ - varnishstat_curses.c + varnishstat_curses.c \ + varnishstat_options.h \ + varnishstat_options.c varnishstat_CFLAGS = \ @SAN_CFLAGS@ diff --git a/bin/varnishstat/varnishstat.c b/bin/varnishstat/varnishstat.c index c7746d6..bf7cc5b 100644 --- a/bin/varnishstat/varnishstat.c +++ b/bin/varnishstat/varnishstat.c @@ -41,11 +41,16 @@ #include #include +#include "vapi/voptget.h" +#include "vapi/vsl.h" +#include "vdef.h" #include "vnum.h" #include "vtim.h" +#include "vut.h" #include "varnishstat.h" +static const char progname[] = "varnishstat"; /*--------------------------------------------------------------------*/ @@ -247,50 +252,38 @@ list_fields(struct VSM_data *vd) /*--------------------------------------------------------------------*/ static void -usage(void) +usage(int status) { -#define FMT " %-28s # %s\n" - fprintf(stderr, "usage: varnishstat " - "[-1lV] [-f field] [-t seconds|] " - VSC_n_USAGE "\n"); - fprintf(stderr, FMT, "-1", "Print the statistics to stdout."); - fprintf(stderr, FMT, "-f field", "Field inclusion glob"); - fprintf(stderr, FMT, "", - "If it starts with '^' it is used as an exclusion list."); - fprintf(stderr, FMT, "-l", - "Lists the available fields to use with the -f option."); - fprintf(stderr, FMT, "-n varnish_name", - "The varnishd instance to get logs from."); - fprintf(stderr, FMT, "-N filename", - "Filename of a stale VSM instance."); - fprintf(stderr, FMT, "-t seconds|", - "Timeout before returning error on initial VSM connection."); - fprintf(stderr, FMT, "-V", "Display the version number and exit."); - fprintf(stderr, FMT, "-x", - "Print statistics to stdout as XML."); - fprintf(stderr, FMT, "-j", - "Print statistics to stdout as JSON."); -#undef FMT - exit(1); + const char **opt; + + fprintf(stderr, "Usage: %s \n\n", progname); + fprintf(stderr, "Options:\n"); + for (opt = vopt_spec.vopt_usage; *opt != NULL; opt +=2) + fprintf(stderr, " %-25s %s\n", *opt, *(opt + 1)); + exit(status); } int main(int argc, char * const *argv) { - int c; struct VSM_data *vd; double t_arg = 5.0, t_start = NAN; int once = 0, xml = 0, json = 0, f_list = 0, curses = 0; + signed char opt; int i; + VUT_Init(progname, argc, argv, &vopt_spec); vd = VSM_New(); AN(vd); - while ((c = getopt(argc, argv, VSC_ARGS "1f:lVxjt:")) != -1) { - switch (c) { + while ((opt = getopt(argc, argv, vopt_spec.vopt_optstring)) != -1) { + switch (opt) { case '1': once = 1; break; + case 'h': + /* Usage help */ + usage(0); case 'l': f_list = 1; break; @@ -319,10 +312,10 @@ main(int argc, char * const *argv) json = 1; break; default: - if (VSC_Arg(vd, c, optarg) > 0) + if (VSC_Arg(vd, opt, optarg) > 0) break; fprintf(stderr, "%s\n", VSM_Error(vd)); - usage(); + usage(1); } } diff --git a/bin/varnishstat/varnishstat_options.c b/bin/varnishstat/varnishstat_options.c new file mode 100644 index 0000000..60b8d28 --- /dev/null +++ b/bin/varnishstat/varnishstat_options.c @@ -0,0 +1,34 @@ +/*- + * Copyright (c) 2016 Varnish Software AS + * All rights reserved. + * + * Author: Federico G. Schwindt + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * Option definitions for varnishstat + */ + +#include +#define VOPT_DEFINITION +#define VOPT_INC "varnishstat_options.h" +#include "vapi/voptget.h" diff --git a/bin/varnishstat/varnishstat_options.h b/bin/varnishstat/varnishstat_options.h new file mode 100644 index 0000000..10d4c7e --- /dev/null +++ b/bin/varnishstat/varnishstat_options.h @@ -0,0 +1,73 @@ +/*- + * Copyright (c) 2016 Varnish Software AS + * All rights reserved. + * + * Author: Federico G. Schwindt + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * Option definitions for varnishstat + */ + +#include "vapi/vapi_options.h" +#include "vut_options.h" + +#define STAT_OPT_1 \ + VOPT("1", "[-1]", "Print the statistics to stdout", \ + "Instead of presenting a continuously updated display," \ + " print the statistics to stdout." \ + ) +#define STAT_OPT_f \ + VOPT("f:", "[-f field]", "Field inclusion glob", \ + "Field inclusion glob." \ + " A field glob consists of three parts, type, ident and" \ + " name, where ident is optional. Each part can contain a" \ + " '*' character at the end to match a prefix. Use" \ + " backslash to escape characters. If the argument starts" \ + " with '^' it is used as an exclusion glob. Multiple -f" \ + " arguments may be given, and they will be applied in" \ + " order." \ + ) +#define STAT_OPT_j \ + VOPT("j", "[-j]", "Print statistics to stdout as JSON", \ + "Print statistics to stdout as JSON." \ + ) +#define STAT_OPT_l \ + VOPT("l", "[-l]", \ + "Lists the available fields to use with the -f option", \ + "Lists the available fields to use with the -f option." \ + ) +#define STAT_OPT_x \ + VOPT("x", "[-x]", "Print statistics to stdout as XML", \ + "Print statistics to stdout as XML." \ + ) + +STAT_OPT_1 +STAT_OPT_f +VUT_OPT_h +STAT_OPT_j +STAT_OPT_l +VUT_OPT_n +VUT_OPT_N +VUT_OPT_t +VUT_OPT_V +STAT_OPT_x diff --git a/doc/sphinx/Makefile.am b/doc/sphinx/Makefile.am index 6baf438..7a55e4b 100644 --- a/doc/sphinx/Makefile.am +++ b/doc/sphinx/Makefile.am @@ -164,6 +164,13 @@ include/varnishhist_synopsis.rst: $(top_builddir)/bin/varnishhist/varnishhist BUILT_SOURCES += include/varnishhist_options.rst \ include/varnishhist_synopsis.rst +include/varnishstat_options.rst: $(top_builddir)/bin/varnishstat/varnishstat + $(top_builddir)/bin/varnishstat/varnishstat --options > $@ +include/varnishstat_synopsis.rst: $(top_builddir)/bin/varnishstat/varnishstat + $(top_builddir)/bin/varnishstat/varnishstat --synopsis > $@ +BUILT_SOURCES += include/varnishstat_options.rst \ + include/varnishstat_synopsis.rst + include/vsl-tags.rst: $(top_builddir)/lib/libvarnishapi/vsl2rst $(top_builddir)/lib/libvarnishapi/vsl2rst > $@ BUILT_SOURCES += include/vsl-tags.rst diff --git a/doc/sphinx/reference/varnishstat.rst b/doc/sphinx/reference/varnishstat.rst index a0619ae..5901a06 100644 --- a/doc/sphinx/reference/varnishstat.rst +++ b/doc/sphinx/reference/varnishstat.rst @@ -15,60 +15,17 @@ Varnish Cache statistics SYNOPSIS ======== -varnishstat [-1] [-x] [-j] [-f field] [-l] [-n varnish_name] [-N filename] [-t seconds|] [-V] - -.. TODO: autogenerate this synopsis like the others. +.. include:: ../include/varnishstat_synopsis.rst +varnishstat |synopsis| DESCRIPTION =========== The varnishstat utility displays statistics from a running varnishd(1) instance. -OPTIONS -======= - The following options are available: --1 - Instead of presenting a continuously updated display, print - the statistics to stdout. - --f - Field inclusion glob. A field glob consists of three parts, - type, ident and name, where ident is optional. Each part can - contain a '*' character at the end to match a prefix. Use - backslash to escape characters. If the argument starts with - '^' it is used as an exclusion glob. Multiple -f arguments may - be given, and they will be applied in order. - --l - Lists the available fields to use with the -f option. - --n - Specifies the name of the varnishd instance to get logs from. - If -n is not specified, the host name is used. - --N - Specify a the filename of a stale VSM instance. When using - this option the abandonment checking is disabled. - --t seconds| - Timeout before returning error on initial VSM connection. If - set the VSM connection is retried every 0.5 seconds for this - any seconds. If zero the connection is attempted only once and - will fail immediately if unsuccessful. If set to "off", the - connection will not fail, allowing the utility to start and - wait indefinitely for the Varnish instance to appear. - Defaults to 5 seconds. - --V - Display the version number and exit. - --x - Displays the result as XML. - --j - Displays the result as JSON. +.. include:: ../include/varnishstat_options.rst CURSES MODE =========== diff --git a/man/Makefile.am b/man/Makefile.am index 05a1618..adb76c2 100644 --- a/man/Makefile.am +++ b/man/Makefile.am @@ -58,8 +58,9 @@ varnishlog.1: \ $(top_builddir)/doc/sphinx/include/varnishlog_synopsis.rst ${RST2MAN} $(RST2ANY_FLAGS) $(top_srcdir)/doc/sphinx/reference/varnishlog.rst $@ -# XXX add _options.rst and _synopsis.rst here when it's been _opt2rst'ed -varnishstat.1: $(top_srcdir)/doc/sphinx/reference/varnishstat.rst +varnishstat.1: $(top_srcdir)/doc/sphinx/reference/varnishstat.rst \ + $(top_builddir)/doc/sphinx/include/varnishstat_options.rst \ + $(top_builddir)/doc/sphinx/include/varnishstat_synopsis.rst ${RST2MAN} $(RST2ANY_FLAGS) $(top_srcdir)/doc/sphinx/reference/varnishstat.rst $@ varnishtest.1: $(top_srcdir)/doc/sphinx/reference/varnishtest.rst From fgsch at lodoss.net Sat Jun 4 16:19:05 2016 From: fgsch at lodoss.net (Federico G. Schwindt) Date: Sat, 04 Jun 2016 18:19:05 +0200 Subject: [master] 07bb17e Update Message-ID: commit 07bb17ea57c0eccf2132dde03ec7e0671f473c04 Author: Federico G. Schwindt Date: Sat Jun 4 16:27:58 2016 +0100 Update diff --git a/.gitignore b/.gitignore index 1f5e586..bb3cf3b 100644 --- a/.gitignore +++ b/.gitignore @@ -88,7 +88,6 @@ cscope.*out /bin/varnishhist/varnishhist /bin/varnishlog/varnishlog /bin/varnishncsa/varnishncsa -/bin/varnishreplay/varnishreplay /bin/varnishstat/varnishstat /bin/varnishstat/vsc2rst /bin/varnishtest/varnishtest @@ -98,10 +97,8 @@ cscope.*out /doc/*.html /doc/sphinx/build/ /doc/sphinx/conf.py -/doc/sphinx/include/vcl_var.rst # graphviz-generated /doc/graphviz/*.pdf -/doc/graphviz/*.svg # NetBeans insists on this /nbproject/private/ From fgsch at lodoss.net Sat Jun 4 16:19:05 2016 From: fgsch at lodoss.net (Federico G. Schwindt) Date: Sat, 04 Jun 2016 18:19:05 +0200 Subject: [master] 590f6e4 GC unused defines Message-ID: commit 590f6e450ef37bc42bb92f5bae086f4727171b44 Author: Federico G. Schwindt Date: Sat Jun 4 16:29:09 2016 +0100 GC unused defines diff --git a/include/vapi/vsc.h b/include/vapi/vsc.h index 609586e..83ed155 100644 --- a/include/vapi/vsc.h +++ b/include/vapi/vsc.h @@ -44,12 +44,6 @@ struct VSM_fantom; * VSC level access functions */ -#define VSC_ARGS "f:n:N:" -#define VSC_n_USAGE VSM_n_USAGE " " VSM_N_USAGE -#define VSC_f_USAGE "[-f field_name,...]" -#define VSC_USAGE VSC_n_USAGE \ - VSC_f_USAGE - int VSC_Arg(struct VSM_data *vd, int arg, const char *opt); /* * Handle standard stat-presenter arguments From fgsch at lodoss.net Sat Jun 4 16:19:05 2016 From: fgsch at lodoss.net (Federico G. Schwindt) Date: Sat, 04 Jun 2016 18:19:05 +0200 Subject: [master] 2593845 Add -h and use different exit status Message-ID: commit 25938455c411bbb1ab1ae3f11f6a3bcd972f2b7f Author: Federico G. Schwindt Date: Sat Jun 4 16:39:24 2016 +0100 Add -h and use different exit status diff --git a/bin/varnishadm/varnishadm.c b/bin/varnishadm/varnishadm.c index e426f0b..e90df71 100644 --- a/bin/varnishadm/varnishadm.c +++ b/bin/varnishadm/varnishadm.c @@ -378,13 +378,13 @@ pass(int sock) static void -usage(void) +usage(int status) { fprintf(stderr, - "Usage: varnishadm [-n ident] [-t timeout] [-S secretfile] " + "Usage: varnishadm [-h] [-n ident] [-t timeout] [-S secretfile] " "[-T [address]:port] [command [...]]\n"); fprintf(stderr, "\t-n is mutually exclusive with -S and -T\n"); - exit(1); + exit(status); } static int @@ -456,8 +456,11 @@ main(int argc, char * const *argv) * The '+' stops that from happening * See #1496 */ - while ((opt = getopt(argc, argv, "+n:S:T:t:")) != -1) { + while ((opt = getopt(argc, argv, "+hn:S:T:t:")) != -1) { switch (opt) { + case 'h': + /* Usage help */ + usage(0); case 'n': n_arg = optarg; break; @@ -470,10 +473,10 @@ main(int argc, char * const *argv) case 't': timeout = VNUM(optarg); if (isnan(timeout)) - usage(); + usage(1); break; default: - usage(); + usage(1); } } @@ -482,7 +485,7 @@ main(int argc, char * const *argv) if (n_arg != NULL) { if (T_arg != NULL || S_arg != NULL) { - usage(); + usage(1); } sock = n_arg_sock(n_arg); } else if (T_arg == NULL) { diff --git a/doc/sphinx/reference/varnishadm.rst b/doc/sphinx/reference/varnishadm.rst index 395a72a..e503671 100644 --- a/doc/sphinx/reference/varnishadm.rst +++ b/doc/sphinx/reference/varnishadm.rst @@ -14,7 +14,7 @@ Control a running Varnish instance SYNOPSIS ======== -varnishadm [-n ident] [-t timeout] [-S secretfile] [-T [address]:port] [command [...]] +varnishadm [-h] [-n ident] [-t timeout] [-S secretfile] [-T [address]:port] [command [...]] DESCRIPTION @@ -35,6 +35,9 @@ replies between the CLI socket and stdin/stdout. OPTIONS ======= +-h + Print program usage and exit. + -n ident Connect to the instance of `varnishd` with this name. From fgsch at lodoss.net Sat Jun 4 16:19:05 2016 From: fgsch at lodoss.net (Federico G. Schwindt) Date: Sat, 04 Jun 2016 18:19:05 +0200 Subject: [master] d61d82e Restructure and sync with reality Message-ID: commit d61d82eeb2cd9506d1288cd5137d551bee45f787 Author: Federico G. Schwindt Date: Sat Jun 4 16:41:10 2016 +0100 Restructure and sync with reality diff --git a/doc/sphinx/reference/varnishstat.rst b/doc/sphinx/reference/varnishstat.rst index 5901a06..09bab41 100644 --- a/doc/sphinx/reference/varnishstat.rst +++ b/doc/sphinx/reference/varnishstat.rst @@ -109,10 +109,10 @@ The following keys control the interactive display: Sample now. -XML Output -========== +OUTPUTS +======= -When using the -x option, the output is:: +The XML output format is:: @@ -127,10 +127,7 @@ When using the -x option, the output is:: [..] -JSON OUTPUT -=========== - -With -j the output format is:: +The JSON output format is:: { "timestamp": "YYYY-MM-DDTHH:mm:SS", @@ -139,10 +136,10 @@ With -j the output format is:: "type": "FIELD TYPE", "ident": "FIELD IDENT", "flag": "FIELD SEMANTICS", "format": "FIELD DISPLAY FORMAT", "value": FIELD VALUE }, - "FIELD2 NAME": { - "description": "FIELD2 DESCRIPTION", - "type": "FIELD2 TYPE", "ident": "FIELD2 IDENT", "flag": "FIELD2 SEMANTICS", "format": "FIELD2 DISPLAY FORMAT", - "value": FIELD2 VALUE + "FIELD NAME": { + "description": "FIELD DESCRIPTION", + "type": "FIELD TYPE", "ident": "FIELD IDENT", "flag": "FIELD SEMANTICS", "format": "FIELD DISPLAY FORMAT", + "value": FIELD VALUE }, [..] } @@ -151,9 +148,6 @@ With -j the output format is:: Type and ident are optional. Timestamp is the time when the report was generated by varnishstat. -Repeated output with -1, -x or -j will have a single empty line (\\n) -between each block of output. - SEE ALSO ======== From fgsch at lodoss.net Sat Jun 4 16:19:05 2016 From: fgsch at lodoss.net (Federico G. Schwindt) Date: Sat, 04 Jun 2016 18:19:05 +0200 Subject: [master] 95a1575 Be consistent wrt usage output Message-ID: commit 95a157523d562a76b6495803c5275abb56493b89 Author: Federico G. Schwindt Date: Sat Jun 4 16:42:00 2016 +0100 Be consistent wrt usage output diff --git a/bin/varnishlog/varnishlog.c b/bin/varnishlog/varnishlog.c index 9b5e909..dde6ead 100644 --- a/bin/varnishlog/varnishlog.c +++ b/bin/varnishlog/varnishlog.c @@ -69,7 +69,7 @@ usage(int status) fprintf(stderr, "Usage: %s \n\n", progname); fprintf(stderr, "Options:\n"); for (opt = vopt_spec.vopt_usage; *opt != NULL; opt += 2) - fprintf(stderr, " %-25s %s\n", *opt, *(opt + 1)); + fprintf(stderr, " %-25s %s\n", *opt, *(opt + 1)); exit(status); } From fgsch at lodoss.net Sat Jun 4 16:19:05 2016 From: fgsch at lodoss.net (Federico G. Schwindt) Date: Sat, 04 Jun 2016 18:19:05 +0200 Subject: [master] 8295fed Exit with 0 if -h was used Message-ID: commit 8295fed485b707901cb6e071c3aed8bcb6b4f8ed Author: Federico G. Schwindt Date: Sat Jun 4 16:43:16 2016 +0100 Exit with 0 if -h was used diff --git a/bin/varnishhist/varnishhist.c b/bin/varnishhist/varnishhist.c index d8a9905..d58d1ab 100644 --- a/bin/varnishhist/varnishhist.c +++ b/bin/varnishhist/varnishhist.c @@ -391,6 +391,9 @@ main(int argc, char **argv) assert(VUT_Arg('c', NULL)); while ((i = getopt(argc, argv, vopt_spec.vopt_optstring)) != -1) { switch (i) { + case 'h': + /* Usage help */ + usage(0); case 'P': colon = strchr(optarg, ':'); /* no colon, take the profile as a name*/ From fgsch at lodoss.net Sat Jun 4 16:19:06 2016 From: fgsch at lodoss.net (Federico G. Schwindt) Date: Sat, 04 Jun 2016 18:19:06 +0200 Subject: [master] affce05 Use VUT_Error() and drop stderr where not needed Message-ID: commit affce0513e141407989f08ad7905944f65d8166e Author: Federico G. Schwindt Date: Sat Jun 4 17:01:04 2016 +0100 Use VUT_Error() and drop stderr where not needed Some other minor cleanup while I'm here. diff --git a/bin/varnishstat/varnishstat.c b/bin/varnishstat/varnishstat.c index bf7cc5b..46cb300 100644 --- a/bin/varnishstat/varnishstat.c +++ b/bin/varnishstat/varnishstat.c @@ -157,7 +157,6 @@ do_json(struct VSM_data *vd) printf(" \"timestamp\": \"%s\",\n", time_stamp); (void)VSC_Iter(vd, NULL, do_json_cb, &jp); printf("\n}\n"); - fflush(stdout); } @@ -229,22 +228,22 @@ do_list_cb(void *priv, const struct VSC_point * const pt) sec = pt->section; i = 0; if (strcmp(sec->fantom->type, "")) - i += fprintf(stderr, "%s.", sec->fantom->type); + i += printf("%s.", sec->fantom->type); if (strcmp(sec->fantom->ident, "")) - i += fprintf(stderr, "%s.", sec->fantom->ident); - i += fprintf(stderr, "%s", pt->desc->name); + i += printf("%s.", sec->fantom->ident); + i += printf("%s", pt->desc->name); if (i < 30) - fprintf(stderr, "%*s", i - 30, ""); - fprintf(stderr, " %s\n", pt->desc->sdesc); + printf("%*s", i - 30, ""); + printf(" %s\n", pt->desc->sdesc); return (0); } static void list_fields(struct VSM_data *vd) { - fprintf(stderr, "Varnishstat -f option fields:\n"); - fprintf(stderr, "Field name Description\n"); - fprintf(stderr, "---------- -----------\n"); + printf("Varnishstat -f option fields:\n"); + printf("Field name Description\n"); + printf("---------- -----------\n"); (void)VSC_Iter(vd, NULL, do_list_cb, NULL); } @@ -292,14 +291,10 @@ main(int argc, char * const *argv) t_arg = -1.; else { t_arg = VNUM(optarg); - if (isnan(t_arg)) { - fprintf(stderr, "-t: Syntax error"); - exit(1); - } - if (t_arg < 0.) { - fprintf(stderr, "-t: Range error"); - exit(1); - } + if (isnan(t_arg)) + VUT_Error(1, "-t: Syntax error"); + if (t_arg < 0.) + VUT_Error(1, "-t: Range error"); } break; case 'V': @@ -312,10 +307,11 @@ main(int argc, char * const *argv) json = 1; break; default: - if (VSC_Arg(vd, opt, optarg) > 0) - break; - fprintf(stderr, "%s\n", VSM_Error(vd)); - usage(1); + i = VSC_Arg(vd, opt, optarg); + if (i < 0) + VUT_Error(1, "%s", VSM_Error(vd)); + if (!i) + usage(1); } } @@ -327,8 +323,8 @@ main(int argc, char * const *argv) if (!i) break; if (isnan(t_start) && t_arg > 0.) { - fprintf(stderr, "Can't open log -" - " retrying for %.0f seconds\n", t_arg); + VUT_Error(0, "Can't open log -" + " retrying for %.0f seconds", t_arg); t_start = VTIM_real(); } if (t_arg <= 0.) @@ -340,18 +336,14 @@ main(int argc, char * const *argv) } if (curses) { - if (i && t_arg >= 0.) { - fprintf(stderr, "%s\n", VSM_Error(vd)); - exit(1); - } + if (i && t_arg >= 0.) + VUT_Error(1, "%s", VSM_Error(vd)); do_curses(vd, 1.0); exit(0); } - if (i) { - fprintf(stderr, "%s\n", VSM_Error(vd)); - exit(1); - } + if (i) + VUT_Error(1, "%s", VSM_Error(vd)); if (xml) do_xml(vd); @@ -364,8 +356,5 @@ main(int argc, char * const *argv) else assert(0); - /* end of output block marker. */ - printf("\n"); - exit(0); } From varnish-commit at varnish-cache.org Mon Jun 6 00:34:55 2016 From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org) Date: Mon, 06 Jun 2016 02:34:55 +0200 Subject: grab your part right now Message-ID: <5754C52F.5080909@varnish-cache.org> Hello, Just wanted to know where I should send you the profits you'll be making? My team is all setup and ready to welcome you on board, so that we can start you on one of the most amazing pesos making journeys you'll ever take. >> CLICK HERE TO START YOUR JOURNEY Let's not waste another minute, click the link below and your first commissions will be coming in as early as in the next 30 minutes. >> CLICK HERE TO START EXPEDITION To your success, Hazel Merritt -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Mon Jun 6 09:01:08 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Mon, 06 Jun 2016 11:01:08 +0200 Subject: [master] 17b878b Recoding doxygen in awk is fun Message-ID: commit 17b878b34b981df821ca3589524cc9824db2a3f4 Author: Guillaume Quintard Date: Mon May 30 12:53:31 2016 +0200 Recoding doxygen in awk is fun diff --git a/doc/sphinx/vtc-syntax.vtc b/doc/sphinx/vtc-syntax.vtc new file mode 100644 index 0000000..ef67591 --- /dev/null +++ b/doc/sphinx/vtc-syntax.vtc @@ -0,0 +1,41 @@ +# end of paragraph if we hit '*/' +$0 ~ "*/" { + p = 0; +} + +# if in paragraph and no section is announced, +# concatenate +p && $0 !~ "[ /]* SECTION: " { + cl[section] = cl[section] gensub(/ \* ?/, "", "1", $0) "\n"; +} + +# section announcement +$0 ~ "[ /]* SECTION: " { + section = $3; + sl[len++] = section; + tl[section] = gensub(/[\t ]*\/?\* SECTION: [^ ]+ +/, "", "1", $0); + p = 1; +} + +# sort sections, underline titles, print +END { + asort(sl); + for (i in sl) { + section = sl[i] + print(tl[section]); + a = section + c = gsub(/\./, "", a); + if (c == 0) + r = "="; + else if (c == 1) + r = "*" + else if (c == 2) + r = "+" + else if (c == 3) + r = "-" + else + r = "." + print(gensub(/./, r, "g", tl[section])); + print(cl[section]); + } +} From guillaume at varnish-software.com Mon Jun 6 09:01:08 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Mon, 06 Jun 2016 11:01:08 +0200 Subject: [master] e2161bc Add skeleton vtc manpage Message-ID: commit e2161bcfe37ec75fcc0c6a9ceef6124770415864 Author: Guillaume Quintard Date: Mon May 30 16:16:57 2016 +0200 Add skeleton vtc manpage diff --git a/doc/sphinx/reference/index.rst b/doc/sphinx/reference/index.rst index 603d1a0..917fe60 100644 --- a/doc/sphinx/reference/index.rst +++ b/doc/sphinx/reference/index.rst @@ -26,6 +26,7 @@ The Varnish Reference Manual varnish-counters.rst vsl.rst vsl-query.rst + vtc.rst .. todo:: The programs: diff --git a/doc/sphinx/reference/vtc.rst b/doc/sphinx/reference/vtc.rst new file mode 100644 index 0000000..443015b --- /dev/null +++ b/doc/sphinx/reference/vtc.rst @@ -0,0 +1,76 @@ +.. role:: ref(emphasis) + +.. _vtc(7): + +=== +VTC +=== + +------------------------ +Varnish Test Case Syntax +------------------------ + +:Manual section: 7 + +OVERVIEW +======== + +This document describes the syntax used by Varnish Test Cases files (.vtc). +A vtc file describe a scenario with different scripted HTTP-talking entities, +and generally one or more Varnish instances to test. + +PARSING +======= + +A vtc file will be read word after word, with very little tokenization, meaning +a syntax error won't be detected until the test actually reach the relevant +action in the test. + +A parsing error will most of the time result in an assert being triggered. If +this happens, please refer yourself to the related source file and line +number. However, this guide should help you avoid the most common mistakes. + +Words and strings +~~~~~~~~~~~~~~~~~ + +The parser splits words by detecting whitespace characters and a string is a +word, or a series of words on the same line enclosed by double-quotes ("..."), +or, for multi-line strings, enclosed in curly brackets ({...}). + +Comments +~~~~~~~~ + +The leading whitespaces of lines are ignored. Empty lines (or ones consisting +only of whitespaces) are ignored too, as are the lines starting with "#" that +are comments. + +Lines and commands +~~~~~~~~~~~~~~~~~~ + +Test files take at most one command per line, with the first word of the line +being the command and the following ones being its arguments. To continue over +to a new line without breaking the argument string, you can escape the newline +character (\n) with a backslash (\). + +SYNTAX +====== + +.. include:: ../include/vtc-syntax.rst + +HISTORY +======= + +This document has been written by Guillaume Quintard. + +SEE ALSO +======== + +* :ref:`varnishtest(1)` + +COPYRIGHT +========= + +This document is licensed under the same licence as Varnish +itself. See LICENCE for details. + +* Copyright (c) 2006-2016 Varnish Software AS diff --git a/doc/sphinx/vtc-syntax.awk b/doc/sphinx/vtc-syntax.awk new file mode 100644 index 0000000..ef67591 --- /dev/null +++ b/doc/sphinx/vtc-syntax.awk @@ -0,0 +1,41 @@ +# end of paragraph if we hit '*/' +$0 ~ "*/" { + p = 0; +} + +# if in paragraph and no section is announced, +# concatenate +p && $0 !~ "[ /]* SECTION: " { + cl[section] = cl[section] gensub(/ \* ?/, "", "1", $0) "\n"; +} + +# section announcement +$0 ~ "[ /]* SECTION: " { + section = $3; + sl[len++] = section; + tl[section] = gensub(/[\t ]*\/?\* SECTION: [^ ]+ +/, "", "1", $0); + p = 1; +} + +# sort sections, underline titles, print +END { + asort(sl); + for (i in sl) { + section = sl[i] + print(tl[section]); + a = section + c = gsub(/\./, "", a); + if (c == 0) + r = "="; + else if (c == 1) + r = "*" + else if (c == 2) + r = "+" + else if (c == 3) + r = "-" + else + r = "." + print(gensub(/./, r, "g", tl[section])); + print(cl[section]); + } +} diff --git a/doc/sphinx/vtc-syntax.vtc b/doc/sphinx/vtc-syntax.vtc deleted file mode 100644 index ef67591..0000000 --- a/doc/sphinx/vtc-syntax.vtc +++ /dev/null @@ -1,41 +0,0 @@ -# end of paragraph if we hit '*/' -$0 ~ "*/" { - p = 0; -} - -# if in paragraph and no section is announced, -# concatenate -p && $0 !~ "[ /]* SECTION: " { - cl[section] = cl[section] gensub(/ \* ?/, "", "1", $0) "\n"; -} - -# section announcement -$0 ~ "[ /]* SECTION: " { - section = $3; - sl[len++] = section; - tl[section] = gensub(/[\t ]*\/?\* SECTION: [^ ]+ +/, "", "1", $0); - p = 1; -} - -# sort sections, underline titles, print -END { - asort(sl); - for (i in sl) { - section = sl[i] - print(tl[section]); - a = section - c = gsub(/\./, "", a); - if (c == 0) - r = "="; - else if (c == 1) - r = "*" - else if (c == 2) - r = "+" - else if (c == 3) - r = "-" - else - r = "." - print(gensub(/./, r, "g", tl[section])); - print(cl[section]); - } -} diff --git a/man/Makefile.am b/man/Makefile.am index adb76c2..f27934b 100644 --- a/man/Makefile.am +++ b/man/Makefile.am @@ -13,6 +13,7 @@ dist_man_MANS = \ varnishncsa.1 \ varnishstat.1 \ varnishtest.1 \ + vtc.7 \ varnishtop.1 \ vmod_directors.3 \ vmod_std.3 @@ -66,6 +67,9 @@ varnishstat.1: $(top_srcdir)/doc/sphinx/reference/varnishstat.rst \ varnishtest.1: $(top_srcdir)/doc/sphinx/reference/varnishtest.rst ${RST2MAN} $(RST2ANY_FLAGS) $(top_srcdir)/doc/sphinx/reference/varnishtest.rst $@ +vtc.7: $(top_srcdir)/doc/sphinx/reference/vtc.rst + ${RST2MAN} $(RST2ANY_FLAGS) $(top_srcdir)/doc/sphinx/reference/vtc.rst $@ + varnishtop.1: \ $(top_srcdir)/doc/sphinx/reference/varnishtop.rst \ $(top_builddir)/doc/sphinx/include/varnishtop_options.rst \ From guillaume at varnish-software.com Mon Jun 6 09:01:08 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Mon, 06 Jun 2016 11:01:08 +0200 Subject: [master] b4be194 Process c files to retrieve the syntax doc Message-ID: commit b4be194e0cdb58b9aa937052c230b2ffb48464a1 Author: Guillaume Quintard Date: Mon May 30 17:23:31 2016 +0200 Process c files to retrieve the syntax doc diff --git a/doc/sphinx/Makefile.am b/doc/sphinx/Makefile.am index 7a55e4b..2e11fa3 100644 --- a/doc/sphinx/Makefile.am +++ b/doc/sphinx/Makefile.am @@ -175,6 +175,11 @@ include/vsl-tags.rst: $(top_builddir)/lib/libvarnishapi/vsl2rst $(top_builddir)/lib/libvarnishapi/vsl2rst > $@ BUILT_SOURCES += include/vsl-tags.rst +VTCSYN_SRC = $(top_srcdir)/bin/varnishtest/vtc.c +include/vtc-syntax.rst: vtc-syntax.awk $(VTCSYN_SRC) + awk -f $(top_srcdir)/doc/sphinx/vtc-syntax.awk $(VTCSYN_SRC) > $@ +BUILT_SOURCES += include/vtc-syntax.rst + .PHONY: reference reference: test -d $@ || mkdir $@ diff --git a/man/Makefile.am b/man/Makefile.am index f27934b..da9b405 100644 --- a/man/Makefile.am +++ b/man/Makefile.am @@ -67,7 +67,8 @@ varnishstat.1: $(top_srcdir)/doc/sphinx/reference/varnishstat.rst \ varnishtest.1: $(top_srcdir)/doc/sphinx/reference/varnishtest.rst ${RST2MAN} $(RST2ANY_FLAGS) $(top_srcdir)/doc/sphinx/reference/varnishtest.rst $@ -vtc.7: $(top_srcdir)/doc/sphinx/reference/vtc.rst +vtc.7: $(top_srcdir)/doc/sphinx/reference/vtc.rst \ + $(top_srcdir)/doc/sphinx/include/vtc-syntax.rst ${RST2MAN} $(RST2ANY_FLAGS) $(top_srcdir)/doc/sphinx/reference/vtc.rst $@ varnishtop.1: \ From guillaume at varnish-software.com Mon Jun 6 09:01:08 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Mon, 06 Jun 2016 11:01:08 +0200 Subject: [master] 6b61e28 Follow style guide lines Message-ID: commit 6b61e286b08e81e89aac26aa0b0b1f46329ec55e Author: Guillaume Quintard Date: Mon May 30 17:27:17 2016 +0200 Follow style guide lines diff --git a/doc/sphinx/reference/vtc.rst b/doc/sphinx/reference/vtc.rst index 443015b..a96702a 100644 --- a/doc/sphinx/reference/vtc.rst +++ b/doc/sphinx/reference/vtc.rst @@ -31,21 +31,21 @@ this happens, please refer yourself to the related source file and line number. However, this guide should help you avoid the most common mistakes. Words and strings -~~~~~~~~~~~~~~~~~ +----------------- The parser splits words by detecting whitespace characters and a string is a word, or a series of words on the same line enclosed by double-quotes ("..."), or, for multi-line strings, enclosed in curly brackets ({...}). Comments -~~~~~~~~ +-------- The leading whitespaces of lines are ignored. Empty lines (or ones consisting only of whitespaces) are ignored too, as are the lines starting with "#" that are comments. Lines and commands -~~~~~~~~~~~~~~~~~~ +------------------ Test files take at most one command per line, with the first word of the line being the command and the following ones being its arguments. To continue over diff --git a/doc/sphinx/vtc-syntax.awk b/doc/sphinx/vtc-syntax.awk index ef67591..e481c26 100644 --- a/doc/sphinx/vtc-syntax.awk +++ b/doc/sphinx/vtc-syntax.awk @@ -26,13 +26,11 @@ END { a = section c = gsub(/\./, "", a); if (c == 0) - r = "="; + r = "-"; else if (c == 1) - r = "*" + r = "~" else if (c == 2) - r = "+" - else if (c == 3) - r = "-" + r = "." else r = "." print(gensub(/./, r, "g", tl[section])); From guillaume at varnish-software.com Mon Jun 6 09:01:08 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Mon, 06 Jun 2016 11:01:08 +0200 Subject: [master] f8a0a86 Document varnishtest Message-ID: commit f8a0a866324db8dd8f8ccd80fca65cc7c4e7b8da Author: Guillaume Quintard Date: Mon May 30 18:01:23 2016 +0200 Document varnishtest diff --git a/bin/varnishtest/vtc.c b/bin/varnishtest/vtc.c index d862554..c7bce41 100644 --- a/bin/varnishtest/vtc.c +++ b/bin/varnishtest/vtc.c @@ -392,8 +392,15 @@ reset_cmds(const struct cmds *cmd) cmd->cmd(NULL, NULL, NULL, NULL); } -/********************************************************************** - * Output test description +/* SECTION: varnishtest varnishtest + * + * This should be the first command in your vtc as it will identify the test + * case with a short yet descriptive sentence. It takes exactly one argument, a + * string, eg:: + * + * varnishtest "Check that varnishtest is actually a valid command" + * + * It will also print that string in the log. */ static void From guillaume at varnish-software.com Mon Jun 6 09:01:08 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Mon, 06 Jun 2016 11:01:08 +0200 Subject: [master] 0604ada Document shell/err_shell Message-ID: commit 0604adad58f801ed4481f2f6b2dd98faf979b5ba Author: Guillaume Quintard Date: Tue May 31 10:56:09 2016 +0200 Document shell/err_shell diff --git a/bin/varnishtest/vtc.c b/bin/varnishtest/vtc.c index c7bce41..807f8d6 100644 --- a/bin/varnishtest/vtc.c +++ b/bin/varnishtest/vtc.c @@ -419,8 +419,18 @@ cmd_varnishtest(CMD_ARGS) AZ(av[2]); } -/********************************************************************** - * Shell command execution +/* SECTION: shell shell + * + * Pass the string given as argument to a shell. If you have multiple commands + * to run, you can use curly barces to describe a multi-lines script, eg:: + * + * shell { + * echo begin + * cat /etc/fstab + * echo end + * } + * + * The vtc will fail if the return code of the shell is not 0. */ static void @@ -442,8 +452,15 @@ cmd_shell(CMD_ARGS) av[1], s, strerror(errno)); } -/********************************************************************** - * Shell command execution +/* SECTION: err_shell err_shell + * + * This is very similar to the the ``shell`` command, except it takes a first + * string as argument before the command:: + * + * err_shell "foo" "echo foo" + * + * err_shell expect the shell command to fail AND stdout to match the string, + * failing the test case otherwise. */ static void From guillaume at varnish-software.com Mon Jun 6 09:01:08 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Mon, 06 Jun 2016 11:01:08 +0200 Subject: [master] 804917f Document delay Message-ID: commit 804917f36def48428040886a7498efd35d05758c Author: Guillaume Quintard Date: Tue May 31 11:13:32 2016 +0200 Document delay diff --git a/bin/varnishtest/vtc.c b/bin/varnishtest/vtc.c index 807f8d6..615478a 100644 --- a/bin/varnishtest/vtc.c +++ b/bin/varnishtest/vtc.c @@ -507,8 +507,11 @@ cmd_err_shell(CMD_ARGS) VSB_destroy(&vsb); } -/********************************************************************** - * Dump command arguments +/* SECTION: delay delay + * + * This is the equivalent of ``sleep`` in shell: the command takes one argument + * that is the number of seconds (can be a float) to wait before continuing the + * test. */ void From guillaume at varnish-software.com Mon Jun 6 09:01:08 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Mon, 06 Jun 2016 11:01:08 +0200 Subject: [master] b6f0a69 Document random Message-ID: commit b6f0a69848cec217040c0b55989a4ee104fdd678 Author: Guillaume Quintard Date: Tue May 31 11:19:00 2016 +0200 Document random diff --git a/bin/varnishtest/vtc.c b/bin/varnishtest/vtc.c index 615478a..2f7c2a1 100644 --- a/bin/varnishtest/vtc.c +++ b/bin/varnishtest/vtc.c @@ -543,6 +543,11 @@ static const unsigned long random_expect[NRNDEXPECT] = { #define RND_NEXT_1K 0x3bdcbe30 +/* SECTION: random random + * + * Check the random number generator. + */ + static void cmd_random(CMD_ARGS) { From guillaume at varnish-software.com Mon Jun 6 09:01:08 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Mon, 06 Jun 2016 11:01:08 +0200 Subject: [master] 4a572ae Document feature Message-ID: commit 4a572aec600ec9e5f26c6fce5054c9bdc0bb06ee Author: Guillaume Quintard Date: Tue May 31 11:29:23 2016 +0200 Document feature diff --git a/bin/varnishtest/vtc.c b/bin/varnishtest/vtc.c index 2f7c2a1..20ba48f 100644 --- a/bin/varnishtest/vtc.c +++ b/bin/varnishtest/vtc.c @@ -581,8 +581,20 @@ cmd_random(CMD_ARGS) } } -/********************************************************************** - * Check features. +/* SECTION: feature feature + * + * Test that the required feature(s) for a test are available, and skip the test + * otherwise. feature takes any number of arguments from this list: + * + * - SO_RCVTIMEO_WORKS + * - 64bit + * - !OSX + * - dns + * - topbuild + * - root + * - user_varnish + * - user_vcache + * */ static void From guillaume at varnish-software.com Mon Jun 6 09:01:08 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Mon, 06 Jun 2016 11:01:08 +0200 Subject: [master] 066b057 Process vtc_barrier.c Message-ID: commit 066b057583a4e605d57b1dd65f07f1f6101a4402 Author: Guillaume Quintard Date: Tue May 31 12:24:42 2016 +0200 Process vtc_barrier.c diff --git a/doc/sphinx/Makefile.am b/doc/sphinx/Makefile.am index 2e11fa3..d2832c5 100644 --- a/doc/sphinx/Makefile.am +++ b/doc/sphinx/Makefile.am @@ -175,7 +175,8 @@ include/vsl-tags.rst: $(top_builddir)/lib/libvarnishapi/vsl2rst $(top_builddir)/lib/libvarnishapi/vsl2rst > $@ BUILT_SOURCES += include/vsl-tags.rst -VTCSYN_SRC = $(top_srcdir)/bin/varnishtest/vtc.c +VTCSYN_SRC = $(top_srcdir)/bin/varnishtest/vtc.c \ + $(top_srcdir)/bin/varnishtest/vtc_barrier.c include/vtc-syntax.rst: vtc-syntax.awk $(VTCSYN_SRC) awk -f $(top_srcdir)/doc/sphinx/vtc-syntax.awk $(VTCSYN_SRC) > $@ BUILT_SOURCES += include/vtc-syntax.rst From guillaume at varnish-software.com Mon Jun 6 09:01:08 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Mon, 06 Jun 2016 11:01:08 +0200 Subject: [master] 01b3404 Document barrier Message-ID: commit 01b3404c961d17560153759f7e01c481b092ef5b Author: Guillaume Quintard Date: Tue May 31 12:25:14 2016 +0200 Document barrier diff --git a/bin/varnishtest/vtc_barrier.c b/bin/varnishtest/vtc_barrier.c index a7bffc4..3498ee5 100644 --- a/bin/varnishtest/vtc_barrier.c +++ b/bin/varnishtest/vtc_barrier.c @@ -353,8 +353,52 @@ barrier_sync(struct barrier *b, struct vtclog *vl) } } -/********************************************************************** - * Barrier command dispatch +/* SECTION: barrier barrier + * + * NOTE: this can be used from the top-level as well as from client and server + * specifications. + * + * Barriers allows you to synchronize different threads to make sure events + * occur in the right order. It's even possible to use them in VCL. + * + * First, it's necessary to declare the barrier:: + * + * barrier bNAME TYPE NUMBER [-cyclic] + * + * With the arguments being: + * + * bNAME + * this is the name of the barrier, used to identify it when you'll + * create sync points. It must start with 'b'. + * + * TYPE + * it can be "cond" (mutex) or "sock" (socket) and sets internal + * behavior. If you don't need VCL synchronization, use cond. + * + * NUMBER + * number of sync point needed to go through the barrier. + * + * \-cyclic + * if present, the barrier will reset itself and be ready for another + * round once gotten through. + * + * Then, to add a sync point:: + * + * barrier bNAME sync + * + * This will block the parent thread until the number of sync points for bNAME + * reaches the NUMBER given in the barrier declaration. + * + * If you wish to synchronize the VCL, you need to declare a "sock" barrier. + * This will emit a macro definition named "bNAME_sock" that you can use in + * VCL (after importing the debug vmod):: + * + * debug.barrier_sync("${bNAME_sock}"); + * + * This function returns 0 if everything went well and is the equivalent of + * ``barrier bNAME sync`` at the VTC top-level. + * + * */ void From guillaume at varnish-software.com Mon Jun 6 09:01:09 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Mon, 06 Jun 2016 11:01:09 +0200 Subject: [master] af92e49 Process vtc_logexp.c Message-ID: commit af92e4971e5273844305af742173d768c25136f6 Author: Guillaume Quintard Date: Tue May 31 15:57:49 2016 +0200 Process vtc_logexp.c diff --git a/doc/sphinx/Makefile.am b/doc/sphinx/Makefile.am index d2832c5..679147a 100644 --- a/doc/sphinx/Makefile.am +++ b/doc/sphinx/Makefile.am @@ -176,7 +176,8 @@ include/vsl-tags.rst: $(top_builddir)/lib/libvarnishapi/vsl2rst BUILT_SOURCES += include/vsl-tags.rst VTCSYN_SRC = $(top_srcdir)/bin/varnishtest/vtc.c \ - $(top_srcdir)/bin/varnishtest/vtc_barrier.c + $(top_srcdir)/bin/varnishtest/vtc_barrier.c \ + $(top_srcdir)/bin/varnishtest/vtc_logexp.c include/vtc-syntax.rst: vtc-syntax.awk $(VTCSYN_SRC) awk -f $(top_srcdir)/doc/sphinx/vtc-syntax.awk $(VTCSYN_SRC) > $@ BUILT_SOURCES += include/vtc-syntax.rst From guillaume at varnish-software.com Mon Jun 6 09:01:09 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Mon, 06 Jun 2016 11:01:09 +0200 Subject: [master] 06d917d Document logexpect Message-ID: commit 06d917d4c4396c311b19622c66db8b0aeb3e3814 Author: Guillaume Quintard Date: Tue May 31 15:53:11 2016 +0200 Document logexpect diff --git a/bin/varnishtest/vtc_logexp.c b/bin/varnishtest/vtc_logexp.c index 0bc26a0..3b14d81 100644 --- a/bin/varnishtest/vtc_logexp.c +++ b/bin/varnishtest/vtc_logexp.c @@ -26,33 +26,83 @@ * SUCH DAMAGE. */ -/* - * Synopsis: - * -v - * -d <0|1> (head/tail mode) - * -g - * -q +/* SECTION: logexpect logexpect * - * vsl arguments (vsl_arg.c) - * -b Only display backend records - * -c Only display client records - * -C Caseless regular expressions - * -i Include tags - * -I <[taglist:]regex> Include by regex - * -L Incomplete transaction limit - * -T Transaction end timeout + * Reads the VSL and looks for records matching a given specification. It will + * process records trying to match the first pattern, and when done, will + * continue processing, trying to match the following pattern. If a pattern + * isn't matched, the test will fail. + * + * logexpect threads are declared this way:: * + * logexpect lNAME -v [-g ] [-d 0|1] [-q query] \ + * [vsl arguments] { + * expect + * expect + * ... + * } [-start|-wait] * - * logexpect lN -v [-g ] [-d 0|1] [-q query] [vsl arguments] { - * expect - * } + * And once declared, you can start them, or wait on them:: * - * skip: [uint|*] Max number of record to skip - * vxid: [uint|*|=] vxid to match - * tag: [tagname|*|=] Tag to match against - * regex: regular expression to match against (optional) - * *: Match anything - * =: Match value of last successfully matched record + * logexpect lNAME <-start|-wait> + * + * With: + * + * lNAME + * Name the logexpect thread, it must start with 'l'. + * + * \-v id + * Specify the varnish instance to use (most of the time, id=v1). + * + * \-g + * Start processing log records at the head of the log instead of the + * tail. + * + * \-q query + * Filter records using a query expression, see ``man vsl-query`` for + * more information. + * + * \-start + * Start the logexpect thread in the background. + * + * \-wait + * Wait for the logexpect thread to finish + * + * VSL arguments (similar to the varnishlog options): + * + * \-b|-c + * Process only backend/client records. + * + * \-C + * Use caseless regex + * + * \-i + * Include tags + * + * \-I <[taglist:]regex> + * Include by regex + * + * \-T + * Transaction end timeout + * + * And the arguments of the specifications lines are: + * + * skip: [uint|*] + * Max number of record to skip + * + * vxid: [uint|*|=] + * vxid to match + * + * tag: [tagname|*|=] + * Tag to match against + * + * regex: + * regular expression to match against (optional) ('*' is anything, '=' + * is the value of the last matched record) */ #include "config.h" From guillaume at varnish-software.com Mon Jun 6 09:01:09 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Mon, 06 Jun 2016 11:01:09 +0200 Subject: [master] 3cf371c Process vtc_varnish.c Message-ID: commit 3cf371c4bc85f0d6f3d9d4cb1d7bc16536ea10b4 Author: Guillaume Quintard Date: Wed Jun 1 12:12:15 2016 +0200 Process vtc_varnish.c diff --git a/doc/sphinx/Makefile.am b/doc/sphinx/Makefile.am index 679147a..e7278fc 100644 --- a/doc/sphinx/Makefile.am +++ b/doc/sphinx/Makefile.am @@ -177,7 +177,8 @@ BUILT_SOURCES += include/vsl-tags.rst VTCSYN_SRC = $(top_srcdir)/bin/varnishtest/vtc.c \ $(top_srcdir)/bin/varnishtest/vtc_barrier.c \ - $(top_srcdir)/bin/varnishtest/vtc_logexp.c + $(top_srcdir)/bin/varnishtest/vtc_logexp.c \ + $(top_srcdir)/bin/varnishtest/vtc_varnish.c include/vtc-syntax.rst: vtc-syntax.awk $(VTCSYN_SRC) awk -f $(top_srcdir)/doc/sphinx/vtc-syntax.awk $(VTCSYN_SRC) > $@ BUILT_SOURCES += include/vtc-syntax.rst From guillaume at varnish-software.com Mon Jun 6 09:01:09 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Mon, 06 Jun 2016 11:01:09 +0200 Subject: [master] 98fadda Document varnish Message-ID: commit 98fadda40e6e54f75ee25da581711f2555b5baff Author: Guillaume Quintard Date: Wed Jun 1 12:12:37 2016 +0200 Document varnish diff --git a/bin/varnishtest/vtc_varnish.c b/bin/varnishtest/vtc_varnish.c index d486b01..7a0c4ff 100644 --- a/bin/varnishtest/vtc_varnish.c +++ b/bin/varnishtest/vtc_varnish.c @@ -913,9 +913,79 @@ varnish_expect(const struct varnish *v, char * const *av) av[0], (uintmax_t)sp.val, av[1], av[2], (uintmax_t)ref); } } - -/********************************************************************** - * Varnish server cmd dispatch +/* SECTION: varnish varnish + * + * Define and interact with varnish instances. + * + * To define a Varnish server, you'll use this syntax:: + * + * varnish vNAME [-arg STRING] [-vcl STRING] [-vcl+backend STRING] + * [-errvcl STRING STRING] [-jail STRING] [-proto PROXY] + * + * With: + * + * vNAME + * Identify the Varnish server with a string, it must starts with 'v'. + * + * \-arg STRING + * Pass an argument to varnishd, for example "-h simple_list". + * + * \-vcl STRING + * Specify the VCL to load on this Varnish instance. You'll probably + * want to use multi-lines strings for this ({...}). + * + * \-vcl+backend STRING + * Do the exact same thing as -vcl, but adds the definition block of + * known backends (ie. already defined). + * + * \-errvcl STRING1 STRING2 + * Load STRING2 as VCL, expecting it to fail, and Varnish to send an + * error string matching STRING2 + * + * \-jail STRING + * Look at ``man varnishd`` (-j) for more information. + * + * \-proto PROXY + * Have Varnish use the proxy protocol. Note that PROXY here is the + * actual string. + * + * You can decide to start the Varnish instance and/or wait for several events:: + * + * varnish vNAME [-start] [-wait] [-wait-running] [-wait-stopped] + * + * \-start + * Start Varnish in the background, immediately processing the rest of + * the file. + * + * \-wait + * Wait for that instance to terminate. + * + * \-wait-running + * Wait for the Varnish child process to be started. + * + * \-wait-stopped + * Wait for the Varnish child process to stop. + * + * \-cleanup + * Once Varnish is stopped, clean everything after it. This is only used + * in one test and you should never need it. + * + * Once Varnish is started, you can talk to it (as you would through + * ``varnishadm``) with these additional switches:: + * + * varnish vNAME [-cli STRING] [-cliok STRING] [-clierr STRING] + * [-expect STRING OP NUMBER] + * + * \-cli|-cliok|-clierr STRING + * All three of these will send STRING to the CLI, the only difference + * is what they expect the return code to be. -cli doesn't expect + * anything, -cliok expects 200 and -clierr expects not 200. + * + * \-expect STRING OP NUMBER + * Look into the VSM and make sure the counter identified by STRING has + * a correct value. OP can be ==, >, >=, <, <=. For example:: + * + * varnish v1 -expect SMA.s1.g_space > 1000000 */ void From guillaume at varnish-software.com Mon Jun 6 09:01:09 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Mon, 06 Jun 2016 11:01:09 +0200 Subject: [master] 07cc58f Allow empty titles Message-ID: commit 07cc58ff3107a4d253ef87637b1bac2864113985 Author: Guillaume Quintard Date: Wed Jun 1 17:50:24 2016 +0200 Allow empty titles Useful to order sections without have a title. diff --git a/doc/sphinx/vtc-syntax.awk b/doc/sphinx/vtc-syntax.awk index e481c26..58ce4ad 100644 --- a/doc/sphinx/vtc-syntax.awk +++ b/doc/sphinx/vtc-syntax.awk @@ -13,7 +13,11 @@ p && $0 !~ "[ /]* SECTION: " { $0 ~ "[ /]* SECTION: " { section = $3; sl[len++] = section; - tl[section] = gensub(/[\t ]*\/?\* SECTION: [^ ]+ +/, "", "1", $0); + if ($4) { + tl[section] = gensub(/[\t ]*\/?\* SECTION: [^ ]+ +/, "", "1", $0); + } else { + tl[section] = ""; + } p = 1; } From guillaume at varnish-software.com Mon Jun 6 09:01:09 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Mon, 06 Jun 2016 11:01:09 +0200 Subject: [master] 1758887 Process vtc_http.c Message-ID: commit 175888792703fa0bd826a212dc5acbfecae5b841 Author: Guillaume Quintard Date: Wed Jun 1 17:52:36 2016 +0200 Process vtc_http.c diff --git a/doc/sphinx/Makefile.am b/doc/sphinx/Makefile.am index e7278fc..696b8ef 100644 --- a/doc/sphinx/Makefile.am +++ b/doc/sphinx/Makefile.am @@ -177,6 +177,7 @@ BUILT_SOURCES += include/vsl-tags.rst VTCSYN_SRC = $(top_srcdir)/bin/varnishtest/vtc.c \ $(top_srcdir)/bin/varnishtest/vtc_barrier.c \ + $(top_srcdir)/bin/varnishtest/vtc_http.c \ $(top_srcdir)/bin/varnishtest/vtc_logexp.c \ $(top_srcdir)/bin/varnishtest/vtc_varnish.c include/vtc-syntax.rst: vtc-syntax.awk $(VTCSYN_SRC) From guillaume at varnish-software.com Mon Jun 6 09:01:09 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Mon, 06 Jun 2016 11:01:09 +0200 Subject: [master] 78b1c95 Document client/server Message-ID: commit 78b1c95f568e7d5a27e47bbfae53bec22759d805 Author: Guillaume Quintard Date: Wed Jun 1 17:51:33 2016 +0200 Document client/server diff --git a/bin/varnishtest/vtc_http.c b/bin/varnishtest/vtc_http.c index 20f7354..7b49a13 100644 --- a/bin/varnishtest/vtc_http.c +++ b/bin/varnishtest/vtc_http.c @@ -49,6 +49,71 @@ #define MAX_HDR 50 +/* SECTION: client-server client/server + * + * Client and server threads are fake HTTP entities used to test your Varnish + * and VCL. They take any number of arguments, and the one that are not + * recognized, assuming they don't start with '-', are treated as + * specifications, laying out the actions to undertake:: + * + * client cNAME [ARGS*] + * server sNAME [ARGS*] + * + * Clients and server are identified by a string that's the first argument, + * clients' names start with 'c' and servers' names start with 's'. + * + * As the client and server commands share a good deal of arguments and + * specification actions, they are grouped in this single section, specific + * items will be explicitly marked as such. + * + * SECTION: client-server.macros Macros and automatic behaviour + * + * To make things easier in the general case, clients will connect by default + * to the first Varnish server declared and the -vcl+backend switch of the + * ``varnish`` command will add all the declared servers as backends. + * + * Be careful though, servers will by default listen to the 127.0.0.1 IP and + * will pick a random port, and publish 3 macros: sNAME_addr, sNAME_port and + * sNAME_sock, but only once they are started. For varnishtest to + * create the vcl with the correct values, the server must be started when you + * use -vcl+backend. + * + * SECTION: client-server.args Arguments + * + * \-start + * Start the thread in background, processing the last given + * specification. + * + * \-wait + * Block until the thread finishes. + * + * \-run (client only) + * Equivalent to "-start -wait". + * + * \repeat NUMBER + * Instead of processing the specification only once, do it NUMBER times. + * + * \-break (server only) + * Stop the server. + * + * \-listen STRING (server only) + * Dictate the listening socket for the server. STRING is of the form + * "IP PORT". + * + * \-connect STRING (client only) + * Indicate the server to connect to. STRING is also of the form + * "IP PORT". + * + * \-dispatch (server only, s0 only) + * Normally, to keep things simple, server threads only handle one + * connection at a time, but the -dispatch switch allows to accept + * any number of connection and handle them following the given spec. + * + * However, -dispatch is only allowed for the server name "s0". + * + * SECTION: client-server.spec Specification + */ + struct http { unsigned magic; #define HTTP_MAGIC 0x2f02169c @@ -567,11 +632,12 @@ http_rxhdr(struct http *hp) hp->body = hp->rxbuf + hp->prxbuf; } - -/********************************************************************** - * Receive a response +/* SECTION: client-server.spec.rxresp + * + * rxresp [-no_obj] (client only) + * Receive and parse a response's headers and body. If -no_obj is present, only get + * the headers. */ - static void cmd_http_rxresp(CMD_ARGS) { @@ -605,6 +671,11 @@ cmd_http_rxresp(CMD_ARGS) vtc_log(hp->vl, 4, "bodylen = %s", hp->bodylen); } +/* SECTION: client-server.spec.rxreqhdrs + * + * rxresp (client only) + * Receive and parse a response's headers. + */ static void cmd_http_rxresphdrs(CMD_ARGS) { @@ -815,6 +886,66 @@ http_tx_parse_args(char * const *av, struct vtclog *vl, struct http *hp, return (av); } +/* SECTION: client-server.spec.txre + * + * txreq|txresp [ARG*] + * Send a minimal request or response, but overload it if necessary. + * + * txreq is client-specific and txresp is server-specific. + * + * The only thing different between a request and a response, apart + * from who can send them is that the first line (request line vs + * status line), so all the options are prety much the same. + * + * \-req STRING (txreq only) + * What method to use (default: "GET". + * + * \-url STRING (txreq only) + * What location to use (default "/"). + * + * \-proto STRING + * What protocol use in the status line. + * (default: "HTTP/1.1"). + * + * \-status NUMBER (txresp only) + * What status code to return (default 200). + * + * \-msg STRING (txresp only) + * What message to put in the status line (default: "OK"). + * + * These three switches can appear in any order but must come before the + * following ones. + * + * \-nolen + * Don't include a Content-Length header in the response. + * + * \-hdr STRING + * Add STRING as a header, it must follow this format: + * "name: value". It can be called multiple times. + * + * You can then use the arguments related to the body: + * + * \-body STRING + * Input STRING as body. + * + * \-bodylen NUMBER + * Generate and input a body that is NUMBER bytes-long. + * + * \-gziplevel NUMBER + * Set the gzip level (call it before any of the other gzip + * switches). + * + * \gzipresidual NUMBER + * Add extra gzip bits. You should never need it. + * + * \-gzipbody STRING + * Zip STRING and send it as body. + * + * \-gziplen NUMBER + * Combine -body and -gzipbody: create a body of length NUMBER, + * zip it and send as body. + */ + /********************************************************************** * Transmit a response */ @@ -864,8 +995,10 @@ cmd_http_txresp(CMD_ARGS) http_write(hp, 4, "txresp"); } -/********************************************************************** - * Receive a request +/* SECTION: client-server.spec.rxreq + * + * rxreq (server only) + * Receive and parse a request's headers and body. */ static void @@ -891,6 +1024,12 @@ cmd_http_rxreq(CMD_ARGS) vtc_log(hp->vl, 4, "bodylen = %s", hp->bodylen); } +/* SECTION: client-server.spec.rxreqhdrs + * + * rxreqhdrs + * Receive and parse a request's headers (but not the body). + */ + static void cmd_http_rxreqhdrs(CMD_ARGS) { @@ -911,6 +1050,12 @@ cmd_http_rxreqhdrs(CMD_ARGS) "Multiple Content-Length headers.\n"); } +/* SECTION: client-server.spec.rxreqbody + * + * rxreqbody (server only) + * Receive a request's body. + */ + static void cmd_http_rxreqbody(CMD_ARGS) { @@ -929,6 +1074,12 @@ cmd_http_rxreqbody(CMD_ARGS) vtc_log(hp->vl, 4, "bodylen = %s", hp->bodylen); } +/* SECTION: client-server.spec.rxrespbody + * + * rxrespbody + * Receive a response's body. + */ + static void cmd_http_rxrespbody(CMD_ARGS) { @@ -947,6 +1098,12 @@ cmd_http_rxrespbody(CMD_ARGS) vtc_log(hp->vl, 4, "bodylen = %s", hp->bodylen); } +/* SECTION: client-server.spec.rxchunk + * + * rxchunk + * Receive an HTTP chunk + */ + static void cmd_http_rxchunk(CMD_ARGS) { @@ -1009,8 +1166,10 @@ cmd_http_txreq(CMD_ARGS) http_write(hp, 4, "txreq"); } -/********************************************************************** - * Receive N characters +/* SECTION: client-server.spec.recv + * + * recv NUMBER + * Read NUMBER bytes from the connection. */ static void @@ -1037,8 +1196,10 @@ cmd_http_recv(CMD_ARGS) } } -/********************************************************************** - * Send a string +/* SECTION: client-server.spec.send + * + * send STRING + * Push STRING on the connection. */ static void @@ -1059,8 +1220,10 @@ cmd_http_send(CMD_ARGS) strerror(errno)); } -/********************************************************************** - * Send a string many times +/* SECTION: client-server.spec.send_n + * + * send_n NUMBER STRING + * Write STRING on the socket NUMBER times. */ static void @@ -1087,8 +1250,10 @@ cmd_http_send_n(CMD_ARGS) } } -/********************************************************************** - * Send an OOB urgent message +/* SECTION: client-server.spec.send_urgent + * + * send_urgent STRING + * Send string as TCP OOB urgent data. You will never need this. */ static void @@ -1109,8 +1274,12 @@ cmd_http_send_urgent(CMD_ARGS) "Write error in http_send_urgent(): %s", strerror(errno)); } -/********************************************************************** - * Send a hex string +/* SECTION: client-server.spec.sendhex + * + * sendhex STRING + * Send bytes as described by STRING. STRING should consist of hex pairs + * possibly separated by whitespace or newlines. For example: + * "0F EE a5 3df2". */ static void @@ -1150,8 +1319,10 @@ cmd_http_sendhex(CMD_ARGS) } -/********************************************************************** - * Send a string as chunked encoding +/* SECTION: client-server.spec.chunked + * + * chunked STRING + * Send STRING as chunked encoding. */ static void @@ -1170,6 +1341,13 @@ cmd_http_chunked(CMD_ARGS) http_write(hp, 4, "chunked"); } +/* SECTION: client-server.spec.chunkedlen + * + * chunkedlen NUMBER + * Do as ``chunked`` except that varnishtest will generate the string + * for you, with a length of NUMBER characters. + */ + static void cmd_http_chunkedlen(CMD_ARGS) { @@ -1205,8 +1383,11 @@ cmd_http_chunkedlen(CMD_ARGS) http_write(hp, 4, "chunked"); } -/********************************************************************** - * set the timeout + +/* SECTION: client-server.spec.timeout + * + * timeout NUMBER + * Set the TCP timeout for this entity. */ static void @@ -1226,8 +1407,10 @@ cmd_http_timeout(CMD_ARGS) hp->timeout = (int)(d * 1000.0); } -/********************************************************************** - * expect other end to close (server only) +/* SECTION: client-server.spec.expect_close + * + * expect_close + * Wait for the connected client to close the connection. */ static void @@ -1268,8 +1451,10 @@ cmd_http_expect_close(CMD_ARGS) vtc_log(vl, 4, "fd=%d EOF, as expected", hp->fd); } -/********************************************************************** - * close a new connection (server only) +/* SECTION: client-server.spec.close + * + * close (server only) + * Close the active TCP connection */ static void @@ -1287,8 +1472,10 @@ cmd_http_close(CMD_ARGS) vtc_log(vl, 4, "Closed"); } -/********************************************************************** - * close and accept a new connection (server only) +/* SECTION: client-server.spec.accept_close + * + * accept (server only) + * Close the active connection (if any) and accept a new one. */ static void @@ -1311,8 +1498,10 @@ cmd_http_accept(CMD_ARGS) vtc_log(vl, 3, "Accepted socket fd is %d", hp->fd); } -/********************************************************************** - * loop operator +/* SECTION: client-server.spec.loop + * + * loop NUMBER STRING + * Process STRING as a specification, NUMBER times. */ static void @@ -1332,8 +1521,10 @@ cmd_http_loop(CMD_ARGS) } } -/********************************************************************** - * Control fatality +/* SECTION: client-server.spec.fatal + * + * fatal|non-fatal + * Control whether a failure of this entity should stop the test. */ static void @@ -1352,6 +1543,17 @@ cmd_http_fatal(CMD_ARGS) } } +/* SECTION: client-server.spec.delay + * + * delay + * Same as for the top-level delay. + * + * SECTION: client-server.spec.barrier + * + * barrier + * Same as for the top-level barrier + */ + /********************************************************************** * Execute HTTP specifications */ From guillaume at varnish-software.com Mon Jun 6 09:01:09 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Mon, 06 Jun 2016 11:01:09 +0200 Subject: [master] 75ce6ba Document expect Message-ID: commit 75ce6ba09a26ee40322e85b0cee8990e50fe4a94 Author: Guillaume Quintard Date: Wed Jun 1 18:12:35 2016 +0200 Document expect diff --git a/bin/varnishtest/vtc_http.c b/bin/varnishtest/vtc_http.c index 7b49a13..184384c 100644 --- a/bin/varnishtest/vtc_http.c +++ b/bin/varnishtest/vtc_http.c @@ -256,8 +256,40 @@ http_count_header(char * const *hh, const char *hdr) return (r); } -/********************************************************************** - * Expect +/* SECTION: client-server.spec.expect + * + * expect STRING1 OP STRING2 + * Test if "STRING1 OP STRING2" is true, and if not, fails the test. + * OP can be ==, <, <=, >, >= when STRING1 and STRING2 represent numbers + * in which case it's an order operator. If STRING1 and STRING2 are + * meant as strings OP is a matching operator, either == (exact match) + * or ~ (regex match). + * + * varnishtet will first try to resolve STRING1 and STRING2 by looking + * if they have special meanings, in which case, the resolved value is + * use for the test. Note that this value can be a string representing a + * number, allowing for tests such as:: + * + * expect req.http.x-num > 2 + * + * Here's the list of recognized strings, most should be obvious as they + * either match VCL logic, or the txreq/txresp options: + * + * - remote.ip + * - remote.port + * - req.method + * - req.url + * - req.proto + * - resp.proto + * - resp.status + * - resp.msg + * - resp.chunklen + * - req.bodylen + * - req.body + * - resp.bodylen + * - resp.body + * - req.http.NAME + * - resp.http.NAME */ static const char * From phk at FreeBSD.org Mon Jun 6 09:23:06 2016 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Mon, 06 Jun 2016 11:23:06 +0200 Subject: [master] d04bb8c sort Message-ID: commit d04bb8c37c1b23d1c5863466a9c5bedfc20c8e3c Author: Poul-Henning Kamp Date: Fri Jun 3 15:04:48 2016 +0000 sort diff --git a/lib/libvcc/Makefile.am b/lib/libvcc/Makefile.am index 6cb09fb..d0c8ece 100644 --- a/lib/libvcc/Makefile.am +++ b/lib/libvcc/Makefile.am @@ -24,13 +24,13 @@ libvcc_la_SOURCES = \ vcc_backend_util.c \ vcc_compile.c \ vcc_expr.c \ - vcc_parse.c \ vcc_fixed_token.c \ vcc_obj.c \ + vcc_parse.c \ vcc_storage.c \ - vcc_utils.c \ vcc_symb.c \ vcc_token.c \ + vcc_utils.c \ vcc_var.c \ vcc_vmod.c \ vcc_xref.c From phk at FreeBSD.org Mon Jun 6 09:23:06 2016 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Mon, 06 Jun 2016 11:23:06 +0200 Subject: [master] 7327936 Rearrange symbol table slightly, in preparation for mode invasive renovations of it. Message-ID: commit 7327936e6f6c2cd08ee6b8e8e1d6473d7cd5abb8 Author: Poul-Henning Kamp Date: Mon Jun 6 09:18:37 2016 +0000 Rearrange symbol table slightly, in preparation for mode invasive renovations of it. diff --git a/lib/libvcc/vcc_compile.c b/lib/libvcc/vcc_compile.c index 4a182cf..605aed3 100644 --- a/lib/libvcc/vcc_compile.c +++ b/lib/libvcc/vcc_compile.c @@ -575,7 +575,6 @@ vcc_NewVcc(const struct vcp *vcp) ALLOC_OBJ(tl, VCC_MAGIC); AN(tl); tl->param = vcp; - VTAILQ_INIT(&tl->symbols); VTAILQ_INIT(&tl->inifin); VTAILQ_INIT(&tl->membits); VTAILQ_INIT(&tl->tokens); @@ -606,7 +605,6 @@ vcc_DestroyTokenList(struct vcc *tl, char *ret) { struct membit *mb; struct source *sp; - struct symbol *sym; int i; while (!VTAILQ_EMPTY(&tl->membits)) { @@ -621,12 +619,6 @@ vcc_DestroyTokenList(struct vcc *tl, char *ret) vcc_destroy_source(sp); } - while (!VTAILQ_EMPTY(&tl->symbols)) { - sym = VTAILQ_FIRST(&tl->symbols); - VTAILQ_REMOVE(&tl->symbols, sym, list); - FREE_OBJ(sym); - } - VSB_destroy(&tl->fh); VSB_destroy(&tl->fc); for (i = 0; i < VCL_MET_MAX; i++) diff --git a/lib/libvcc/vcc_compile.h b/lib/libvcc/vcc_compile.h index e9e72cc..8ba41c0 100644 --- a/lib/libvcc/vcc_compile.h +++ b/lib/libvcc/vcc_compile.h @@ -113,6 +113,7 @@ struct symbol { unsigned magic; #define SYMBOL_MAGIC 0x3368c9fb VTAILQ_ENTRY(symbol) list; + VTAILQ_HEAD(,symbol) children; char *name; unsigned nlen; @@ -179,7 +180,7 @@ struct vcc { /* Parameter/Template section */ const struct vcp *param; - VTAILQ_HEAD(, symbol) symbols; + struct symbol *symbols; struct inifinhead inifin; unsigned ninifin; diff --git a/lib/libvcc/vcc_symb.c b/lib/libvcc/vcc_symb.c index 762e27b..38127a3 100644 --- a/lib/libvcc/vcc_symb.c +++ b/lib/libvcc/vcc_symb.c @@ -65,11 +65,36 @@ VCC_SymKind(struct vcc *tl, const struct symbol *s) } static struct symbol * +vcc_new_symbol(struct vcc *tl, const char *b, const char *e) +{ + struct symbol *sym; + + AN(b); + if (e == NULL) + e = strchr(b, '\0'); + AN(e); + assert(e > b); + sym = TlAlloc(tl, sizeof *sym); + INIT_OBJ(sym, SYMBOL_MAGIC); + AN(sym); + sym->name = TlAlloc(tl, (e - b) + 1L); + AN(sym->name); + memcpy(sym->name, b, (e - b)); + sym->name[e - b] = '\0'; + sym->nlen = e - b; + VTAILQ_INIT(&sym->children); + return (sym); +} + +static struct symbol * vcc_AddSymbol(struct vcc *tl, const char *nb, int l, enum symkind kind) { struct symbol *sym; - VTAILQ_FOREACH(sym, &tl->symbols, list) { + if (tl->symbols == NULL) + tl->symbols = vcc_new_symbol(tl, "", NULL); + + VTAILQ_FOREACH(sym, &tl->symbols->children, list) { if (sym->nlen != l) continue; if (memcmp(nb, sym->name, l)) @@ -81,14 +106,8 @@ vcc_AddSymbol(struct vcc *tl, const char *nb, int l, enum symkind kind) ErrInternal(tl); return (NULL); } - ALLOC_OBJ(sym, SYMBOL_MAGIC); - AN(sym); - sym->name = malloc(l + 1L); - AN(sym->name); - memcpy(sym->name, nb, l); - sym->name[l] = '\0'; - sym->nlen = l; - VTAILQ_INSERT_HEAD(&tl->symbols, sym, list); + sym = vcc_new_symbol(tl, nb, nb + l); + VTAILQ_INSERT_HEAD(&tl->symbols->children, sym, list); sym->kind = kind; return (sym); } @@ -127,7 +146,7 @@ VCC_FindSymbol(struct vcc *tl, const struct token *t, enum symkind kind) struct symbol *sym; assert(t->tok == ID); - VTAILQ_FOREACH(sym, &tl->symbols, list) { + VTAILQ_FOREACH(sym, &tl->symbols->children, list) { if (sym->kind == SYM_WILDCARD && (t->e - t->b > sym->nlen) && !memcmp(sym->name, t->b, sym->nlen)) { @@ -147,7 +166,7 @@ VCC_WalkSymbols(struct vcc *tl, symwalk_f *func, enum symkind kind) { struct symbol *sym; - VTAILQ_FOREACH(sym, &tl->symbols, list) { + VTAILQ_FOREACH(sym, &tl->symbols->children, list) { if (kind == SYM_NONE || kind == sym->kind) func(tl, sym); ERRCHK(tl); From guillaume at varnish-software.com Mon Jun 6 09:26:07 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Mon, 06 Jun 2016 11:26:07 +0200 Subject: [master] e17ba0b Add -stop doc, and be more detailed Message-ID: commit e17ba0bcae7475bd46ffb5f37bc2f8f4558440b5 Author: Guillaume Quintard Date: Mon Jun 6 11:24:09 2016 +0200 Add -stop doc, and be more detailed diff --git a/bin/varnishtest/vtc_varnish.c b/bin/varnishtest/vtc_varnish.c index 7a0c4ff..7a42324 100644 --- a/bin/varnishtest/vtc_varnish.c +++ b/bin/varnishtest/vtc_varnish.c @@ -922,6 +922,10 @@ varnish_expect(const struct varnish *v, char * const *av) * varnish vNAME [-arg STRING] [-vcl STRING] [-vcl+backend STRING] * [-errvcl STRING STRING] [-jail STRING] [-proto PROXY] * + * The first ``varnish vNAME`` invocation will start the varnishd master + * process in the background, waiting for the ``-start`` switch to actually + * start the child. + * * With: * * vNAME @@ -954,8 +958,10 @@ varnish_expect(const struct varnish *v, char * const *av) * varnish vNAME [-start] [-wait] [-wait-running] [-wait-stopped] * * \-start - * Start Varnish in the background, immediately processing the rest of - * the file. + * Start the child process. + * + * \-stop + * Stop the child process. * * \-wait * Wait for that instance to terminate. From phk at FreeBSD.org Mon Jun 6 13:09:07 2016 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Mon, 06 Jun 2016 15:09:07 +0200 Subject: [master] 0148166 Convert the VCC symbol table from flat to hierarchy. Message-ID: commit 0148166a64d8900d535ea40c68af670be9d2412c Author: Poul-Henning Kamp Date: Mon Jun 6 12:36:42 2016 +0000 Convert the VCC symbol table from flat to hierarchy. This includes various ugly workarounds for some of the things I hope to clean up with this change. diff --git a/include/tbl/symbol_kind.h b/include/tbl/symbol_kind.h index e9add4a..bd90016 100644 --- a/include/tbl/symbol_kind.h +++ b/include/tbl/symbol_kind.h @@ -32,6 +32,7 @@ VCC_SYMB(NONE, none) VCC_SYMB(ACL, acl) VCC_SYMB(BACKEND, backend) VCC_SYMB(FUNC, func) /* VMOD function/procedure */ +VCC_SYMB(INSTANCE, instance) VCC_SYMB(METHOD, method) VCC_SYMB(OBJECT, object) VCC_SYMB(PROBE, probe) diff --git a/lib/libvcc/vcc_action.c b/lib/libvcc/vcc_action.c index aa78adb..e089afd 100644 --- a/lib/libvcc/vcc_action.c +++ b/lib/libvcc/vcc_action.c @@ -174,7 +174,7 @@ parse_new(struct vcc *tl) return; } - sy1 = VCC_AddSymbolTok(tl, tl->t, SYM_NONE); // XXX: NONE ? + sy1 = VCC_AddSymbolTok(tl, tl->t, SYM_INSTANCE); XXXAN(sy1); sy1->def_b = tl->t; vcc_NextToken(tl); @@ -218,7 +218,8 @@ parse_new(struct vcc *tl) vcc_NextToken(tl); bprintf(buf1, ", &vo_%s, \"%s\"", sy1->name, sy1->name); - vcc_Eval_Func(tl, s_init, buf1, sy2->name, s_init + strlen(s_init) + 1); + vcc_Eval_Func(tl, s_init, buf1, sy2->name, s_init + strlen(s_init) + 1, + sy2->vmod); ifp = New_IniFin(tl); VSB_printf(ifp->fin, "\t\t%s(&vo_%s);", s_fini, sy1->name); ExpectErr(tl, ';'); diff --git a/lib/libvcc/vcc_compile.h b/lib/libvcc/vcc_compile.h index 8ba41c0..de1cc7e 100644 --- a/lib/libvcc/vcc_compile.h +++ b/lib/libvcc/vcc_compile.h @@ -106,8 +106,8 @@ enum symkind { typedef void sym_expr_t(struct vcc *tl, struct expr **, const struct symbol *sym, enum var_type); -typedef struct symbol *sym_wildcard_t(struct vcc *tl, const struct token *t, - const struct symbol *sym); +typedef void sym_wildcard_t(struct vcc *, struct symbol *, + const char *, const char *); struct symbol { unsigned magic; @@ -115,6 +115,8 @@ struct symbol { VTAILQ_ENTRY(symbol) list; VTAILQ_HEAD(,symbol) children; + const char *vmod; + char *name; unsigned nlen; sym_wildcard_t *wildcard; @@ -289,7 +291,7 @@ sym_expr_t vcc_Eval_Var; sym_expr_t vcc_Eval_Handle; sym_expr_t vcc_Eval_SymFunc; void vcc_Eval_Func(struct vcc *tl, const char *cfunc, const char *extra, - const char *name, const char *args); + const char *name, const char *args, const char *vmod); enum var_type VCC_arg_type(const char **p); enum symkind VCC_HandleKind(enum var_type fmt); struct symbol *VCC_HandleSymbol(struct vcc *, const struct token *, @@ -312,6 +314,8 @@ void Resolve_Sockaddr(struct vcc *tl, const char *host, const char *defport, const struct token *t_err, const char *errid); /* vcc_symb.c */ +struct symbol *VCC_Symbol(struct vcc *, struct symbol *, + const char *, const char *, enum symkind, int); struct symbol *VCC_AddSymbolStr(struct vcc *tl, const char *name, enum symkind); struct symbol *VCC_AddSymbolTok(struct vcc *tl, const struct token *t, enum symkind kind); diff --git a/lib/libvcc/vcc_expr.c b/lib/libvcc/vcc_expr.c index 8284618..90cd47e 100644 --- a/lib/libvcc/vcc_expr.c +++ b/lib/libvcc/vcc_expr.c @@ -537,18 +537,15 @@ vcc_Eval_Var(struct vcc *tl, struct expr **e, const struct symbol *sym, */ static struct expr * -vcc_priv_arg(struct vcc *tl, const char *p, const char *name) +vcc_priv_arg(struct vcc *tl, const char *p, const char *name, const char *vmod) { - const char *r; struct expr *e2; char buf[32]; struct inifin *ifp; + (void)name; if (!strcmp(p, "PRIV_VCL")) { - r = strchr(name, '.'); - AN(r); - e2 = vcc_mk_expr(VOID, "&vmod_priv_%.*s", - (int) (r - name), name); + e2 = vcc_mk_expr(VOID, "&vmod_priv_%s", vmod); } else if (!strcmp(p, "PRIV_CALL")) { bprintf(buf, "vmod_priv_%u", tl->unique++); ifp = New_IniFin(tl); @@ -556,17 +553,11 @@ vcc_priv_arg(struct vcc *tl, const char *p, const char *name) VSB_printf(ifp->fin, "\tVRT_priv_fini(&%s);", buf); e2 = vcc_mk_expr(VOID, "&%s", buf); } else if (!strcmp(p, "PRIV_TASK")) { - r = strchr(name, '.'); - AN(r); e2 = vcc_mk_expr(VOID, - "VRT_priv_task(ctx, &VGC_vmod_%.*s)", - (int) (r - name), name); + "VRT_priv_task(ctx, &VGC_vmod_%s)", vmod); } else if (!strcmp(p, "PRIV_TOP")) { - r = strchr(name, '.'); - AN(r); e2 = vcc_mk_expr(VOID, - "VRT_priv_top(ctx, &VGC_vmod_%.*s)", - (int) (r - name), name); + "VRT_priv_top(ctx, &VGC_vmod_%s)", vmod); } else { WRONG("Wrong PRIV_ type"); } @@ -633,7 +624,7 @@ vcc_do_arg(struct vcc *tl, struct func_arg *fa) static void vcc_func(struct vcc *tl, struct expr **e, const char *cfunc, - const char *extra, const char *name, const char *args) + const char *extra, const char *name, const char *args, const char *vmod) { const char *p; struct expr *e1; @@ -657,7 +648,7 @@ vcc_func(struct vcc *tl, struct expr **e, const char *cfunc, VTAILQ_INSERT_TAIL(&head, fa, list); fa->type = VCC_arg_type(&p); if (fa->type == VOID && !memcmp(p, "PRIV_", 5)) { - fa->result = vcc_priv_arg(tl, p, name); + fa->result = vcc_priv_arg(tl, p, name, vmod); fa->name = ""; p += strlen(p) + 1; continue; @@ -751,13 +742,13 @@ vcc_func(struct vcc *tl, struct expr **e, const char *cfunc, void vcc_Eval_Func(struct vcc *tl, const char *cfunc, - const char *extra, const char *name, const char *args) + const char *extra, const char *name, const char *args, const char *vmod) { struct expr *e = NULL; struct token *t1; t1 = tl->t; - vcc_func(tl, &e, cfunc, extra, name, args); + vcc_func(tl, &e, cfunc, extra, name, args, vmod); if (!tl->err) { vcc_expr_fmt(tl->fb, tl->indent, e); VSB_cat(tl->fb, ";\n"); @@ -782,7 +773,8 @@ vcc_Eval_SymFunc(struct vcc *tl, struct expr **e, const struct symbol *sym, AN(sym->name); AN(sym->args); SkipToken(tl, ID); - vcc_func(tl, e, sym->cfunc, sym->extra, sym->name, sym->args); + vcc_func(tl, e, sym->cfunc, sym->extra, sym->name, sym->args, + sym->vmod); } /*-------------------------------------------------------------------- diff --git a/lib/libvcc/vcc_storage.c b/lib/libvcc/vcc_storage.c index efc4f11..878c727 100644 --- a/lib/libvcc/vcc_storage.c +++ b/lib/libvcc/vcc_storage.c @@ -61,21 +61,18 @@ #include "vcc_compile.h" -#define PFX "storage." - /*-------------------------------------------------------------------- * */ static struct var * -vcc_Stv_mkvar(struct vcc *tl, const struct token *t, enum var_type fmt) +vcc_Stv_mkvar(struct vcc *tl, enum var_type fmt) { struct var *v; v = TlAlloc(tl, sizeof *v); AN(v); - v->name = TlDupTok(tl, t); v->r_methods = 0; #define VCL_MET_MAC(l,u,t,b) v->r_methods |= VCL_MET_##u; #include "tbl/vcl_returns.h" @@ -95,39 +92,34 @@ static struct stvars { { NULL, BOOL } }; -struct symbol * -vcc_Stv_Wildcard(struct vcc *tl, const struct token *t, - const struct symbol *wcsym) +void __match_proto__(sym_wildcard_t) +vcc_Stv_Wildcard(struct vcc *tl, struct symbol *parent, + const char *b, const char *e) { - const char *p, *q; + const char *q; struct var *v = NULL; struct symbol *sym; struct stvars *sv; char stv[1024]; char buf[1024]; - (void)wcsym; - assert((t->e - t->b) > strlen(PFX)); - AZ(memcmp(t->b, PFX, strlen(PFX))); - - p = t->b + strlen(PFX); - for (q = p; q < t->e && *q != '.'; q++) + for (q = b; q < e && *q != '.'; q++) continue; - bprintf(stv, "%.*s", (int)(q - p), p); + bprintf(stv, "%.*s", (int)(q - b), b); - if (q == t->e) { - v = vcc_Stv_mkvar(tl, t, BOOL); + if (q == e) { + v = vcc_Stv_mkvar(tl, BOOL); bprintf(buf, "VRT_Stv(\"%s\")", stv); v->rname = TlDup(tl, buf); } else { assert(*q == '.'); q++; for(sv = stvars; sv->name != NULL; sv++) { - if (strncmp(q, sv->name, t->e - q)) + if (strncmp(q, sv->name, e - q)) continue; - if (sv->name[t->e - q] != '\0') + if (sv->name[e - q] != '\0') continue; - v = vcc_Stv_mkvar(tl, t, sv->fmt); + v = vcc_Stv_mkvar(tl, sv->fmt); bprintf(buf, "VRT_Stv_%s(\"%s\")", sv->name, stv); v->rname = TlDup(tl, buf); break; @@ -135,9 +127,9 @@ vcc_Stv_Wildcard(struct vcc *tl, const struct token *t, } if (v == NULL) - return (NULL); + return; - sym = VCC_AddSymbolTok(tl, t, SYM_VAR); + sym = VCC_Symbol(tl, parent, b, e, SYM_VAR, 1); AN(sym); sym->fmt = v->fmt; sym->eval = vcc_Eval_Var; @@ -145,6 +137,4 @@ vcc_Stv_Wildcard(struct vcc *tl, const struct token *t, sym->rname = v->rname; sym->w_methods = v->w_methods; sym->lname = v->lname; - - return (sym); } diff --git a/lib/libvcc/vcc_symb.c b/lib/libvcc/vcc_symb.c index 38127a3..be40cc1 100644 --- a/lib/libvcc/vcc_symb.c +++ b/lib/libvcc/vcc_symb.c @@ -86,30 +86,83 @@ vcc_new_symbol(struct vcc *tl, const char *b, const char *e) return (sym); } -static struct symbol * -vcc_AddSymbol(struct vcc *tl, const char *nb, int l, enum symkind kind) +struct symbol * +VCC_Symbol(struct vcc *tl, struct symbol *parent, + const char *b, const char *e, enum symkind kind, int create) { - struct symbol *sym; + const char *q; + struct symbol *sym, *sym2 = NULL; + size_t l; + int i; if (tl->symbols == NULL) tl->symbols = vcc_new_symbol(tl, "", NULL); + if (parent == NULL) + parent = tl->symbols; + + AN(b); + assert(e == NULL || b < e); + if (e == NULL) + e = strchr(b, '\0'); + assert(e > b); + if (e[-1] == '.') + e--; + assert(e > b); - VTAILQ_FOREACH(sym, &tl->symbols->children, list) { - if (sym->nlen != l) + q = strchr(b, '.'); + if (q == NULL || q > e) + q = e; + l = q - b; + assert(l > 0); + + VTAILQ_FOREACH(sym, &parent->children, list) { + i = strncmp(sym->name, b, l); + if (i < 0) continue; - if (memcmp(nb, sym->name, l)) + if (i > 0 || l < sym->nlen) { + sym2 = sym; + sym = NULL; + break; + } + if (l > sym->nlen) continue; - if (kind != sym->kind) + if (q < e) + break; + if (kind != SYM_NONE && sym->kind != kind) continue; - VSB_printf(tl->sb, "Name Collision: <%.*s> <%s>\n", - l, nb, VCC_SymKind(tl, sym)); - ErrInternal(tl); - return (NULL); + if (kind == SYM_NONE && sym->kind == kind) + continue; + break; } - sym = vcc_new_symbol(tl, nb, nb + l); - VTAILQ_INSERT_HEAD(&tl->symbols->children, sym, list); - sym->kind = kind; - return (sym); + if (sym == NULL && create == 0 && parent->kind == SYM_WILDCARD) { + AN(parent->wildcard); + parent->wildcard(tl, parent, b, e); + if (tl->err) + return (NULL); + return (VCC_Symbol(tl, parent, b, e, kind, -1)); + } + if (sym == NULL && create < 1) + return (sym); + if (sym == NULL) { + sym = vcc_new_symbol(tl, b, q); + if (sym2 != NULL) + VTAILQ_INSERT_BEFORE(sym2, sym, list); + else + VTAILQ_INSERT_TAIL(&parent->children, sym, list); + if (q == e) + sym->kind = kind; + } + if (q == e) + return (sym); + assert(*q == '.'); + return (VCC_Symbol(tl, sym, ++q, e, kind, create)); +} + +static struct symbol * +vcc_AddSymbol(struct vcc *tl, const char *nb, int l, enum symkind kind) +{ + + return(VCC_Symbol(tl, NULL, nb, nb + l, kind, 1)); } struct symbol * @@ -143,36 +196,32 @@ VCC_GetSymbolTok(struct vcc *tl, const struct token *tok, enum symkind kind) struct symbol * VCC_FindSymbol(struct vcc *tl, const struct token *t, enum symkind kind) { - struct symbol *sym; assert(t->tok == ID); - VTAILQ_FOREACH(sym, &tl->symbols->children, list) { - if (sym->kind == SYM_WILDCARD && - (t->e - t->b > sym->nlen) && - !memcmp(sym->name, t->b, sym->nlen)) { - AN(sym->wildcard); - return (sym->wildcard(tl, t, sym)); - } - if (kind != SYM_NONE && kind != sym->kind) - continue; - if (vcc_IdIs(t, sym->name)) - return (sym); - } - return (NULL); + return (VCC_Symbol(tl, NULL, t->b, t->e, kind, 0)); } -void -VCC_WalkSymbols(struct vcc *tl, symwalk_f *func, enum symkind kind) +static void +vcc_walksymbols(struct vcc *tl, const struct symbol *root, + symwalk_f *func, enum symkind kind) { struct symbol *sym; - VTAILQ_FOREACH(sym, &tl->symbols->children, list) { + VTAILQ_FOREACH(sym, &root->children, list) { if (kind == SYM_NONE || kind == sym->kind) func(tl, sym); ERRCHK(tl); + vcc_walksymbols(tl, sym, func, kind); } } +void +VCC_WalkSymbols(struct vcc *tl, symwalk_f *func, enum symkind kind) +{ + + vcc_walksymbols(tl, tl->symbols, func, kind); +} + static void vcc_global(struct vcc *tl, struct symbol *sym, enum var_type fmt, const char *str, va_list ap) diff --git a/lib/libvcc/vcc_var.c b/lib/libvcc/vcc_var.c index 6bd5dec..3f4ef81 100644 --- a/lib/libvcc/vcc_var.c +++ b/lib/libvcc/vcc_var.c @@ -37,32 +37,31 @@ /*--------------------------------------------------------------------*/ -struct symbol * -vcc_Var_Wildcard(struct vcc *tl, const struct token *t, const struct symbol *wc) +void __match_proto__(sym_wildcard_t) +vcc_Var_Wildcard(struct vcc *tl, struct symbol *parent, + const char *b, const char *e) { struct symbol *sym; struct var *v; const struct var *vh; unsigned u; - const char *p, *leaf; + const char *p; struct vsb *vsb; - vh = wc->wildcard_priv; + vh = parent->wildcard_priv; assert(vh->fmt == HEADER); v = TlAlloc(tl, sizeof *v); AN(v); - v->name = TlDupTok(tl, t); v->r_methods = vh->r_methods; v->w_methods = vh->w_methods; v->fmt = vh->fmt; - leaf = v->name + vh->len; /* Create a C-name version of the header name */ vsb = VSB_new_auto(); AN(vsb); VSB_printf(vsb, "&VGC_%s_", vh->rname); - for (p = leaf, u = 1; *p != '\0'; p++, u++) + for (p = b, u = 1; p < e; p++, u++) if (vct_isalpha(*p) || vct_isdigit(*p)) VSB_putc(vsb, *p); else @@ -71,7 +70,8 @@ vcc_Var_Wildcard(struct vcc *tl, const struct token *t, const struct symbol *wc) /* Create the static identifier */ Fh(tl, 0, "static const struct gethdr_s %s =\n", VSB_data(vsb) + 1); - Fh(tl, 0, " { %s, \"\\%03o%s:\"};\n", vh->rname, u, leaf); + Fh(tl, 0, " { %s, \"\\%03o%.*s:\"};\n", + vh->rname, u, (int)(e - b), b); /* Create the symbol r/l values */ v->rname = TlDup(tl, VSB_data(vsb)); @@ -81,7 +81,7 @@ vcc_Var_Wildcard(struct vcc *tl, const struct token *t, const struct symbol *wc) v->lname = TlDup(tl, VSB_data(vsb)); VSB_destroy(&vsb); - sym = VCC_AddSymbolTok(tl, t, SYM_VAR); + sym = VCC_Symbol(tl, parent, b, e, SYM_VAR, 1); AN(sym); sym->fmt = v->fmt; sym->eval = vcc_Eval_Var; @@ -89,7 +89,6 @@ vcc_Var_Wildcard(struct vcc *tl, const struct token *t, const struct symbol *wc) sym->rname = v->rname; sym->w_methods = v->w_methods; sym->lname = v->lname; - return (sym); } /*--------------------------------------------------------------------*/ diff --git a/lib/libvcc/vcc_vmod.c b/lib/libvcc/vcc_vmod.c index 06c2b92..07995c3 100644 --- a/lib/libvcc/vcc_vmod.c +++ b/lib/libvcc/vcc_vmod.c @@ -67,6 +67,7 @@ vcc_ParseImport(struct vcc *tl) struct inifin *ifp; const char * const *spec; struct symbol *sym; + struct symbol *msym; const struct symbol *osym; const char *p; // int *modlen; @@ -96,11 +97,11 @@ vcc_ParseImport(struct vcc *tl) } bprintf(fn, "%.*s", PF(mod)); - sym = VCC_AddSymbolStr(tl, fn, SYM_VMOD); + msym = VCC_AddSymbolStr(tl, fn, SYM_VMOD); ERRCHK(tl); - AN(sym); - sym->def_b = t1; - sym->def_e = tl->t; + AN(msym); + msym->def_b = t1; + msym->def_e = tl->t; if (tl->t->tok == ID) { if (!vcc_IdIs(tl->t, "from")) { @@ -222,6 +223,7 @@ vcc_ParseImport(struct vcc *tl) sym = VCC_AddSymbolStr(tl, p, SYM_OBJECT); XXXAN(sym); sym->args = p; + sym->vmod = msym->name; } else if (!strcmp(p, "$EVENT")) { p += strlen(p) + 1; if (ifp == NULL) @@ -240,6 +242,7 @@ vcc_ParseImport(struct vcc *tl) sym = VCC_AddSymbolStr(tl, p, SYM_FUNC); ERRCHK(tl); AN(sym); + sym->vmod = msym->name; sym->eval = vcc_Eval_SymFunc; p += strlen(p) + 1; sym->cfunc = p; From phk at FreeBSD.org Mon Jun 6 13:09:07 2016 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Mon, 06 Jun 2016 15:09:07 +0200 Subject: [master] 0cd6bdd Remove unneeded struct member "len" Message-ID: commit 0cd6bddab38082f08c62cb837d74ec04d75b85d7 Author: Poul-Henning Kamp Date: Mon Jun 6 12:46:40 2016 +0000 Remove unneeded struct member "len" diff --git a/lib/libvcc/generate.py b/lib/libvcc/generate.py index 5da2c95..ed7a2eb 100755 --- a/lib/libvcc/generate.py +++ b/lib/libvcc/generate.py @@ -1140,7 +1140,7 @@ def one_var(nm, spec): cnam = i[0].replace(".", "_") ctyp = vcltypes[typ] - fo.write("\t{ \"%s\", %s, %d,\n" % (nm, typ, len(nm))) + fo.write("\t{ \"%s\", %s,\n" % (nm, typ)) if len(spec[2]) == 0: fo.write('\t NULL,\t/* No reads allowed */\n') diff --git a/lib/libvcc/vcc_compile.h b/lib/libvcc/vcc_compile.h index de1cc7e..ac9cb0a 100644 --- a/lib/libvcc/vcc_compile.h +++ b/lib/libvcc/vcc_compile.h @@ -224,7 +224,6 @@ struct vcc { struct var { const char *name; enum var_type fmt; - unsigned len; const char *rname; unsigned r_methods; const char *lname; From phk at FreeBSD.org Mon Jun 6 13:09:07 2016 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Mon, 06 Jun 2016 15:09:07 +0200 Subject: [master] 4d3c70c Unify symbol table access through a single function Message-ID: commit 4d3c70c7e274abd9dc68f3059cafc27c17213f65 Author: Poul-Henning Kamp Date: Mon Jun 6 13:08:01 2016 +0000 Unify symbol table access through a single function diff --git a/lib/libvcc/vcc_action.c b/lib/libvcc/vcc_action.c index e089afd..d81c73a 100644 --- a/lib/libvcc/vcc_action.c +++ b/lib/libvcc/vcc_action.c @@ -158,7 +158,7 @@ parse_new(struct vcc *tl) ExpectErr(tl, ID); vcc_ExpectCid(tl, "VCL object"); ERRCHK(tl); - sy1 = VCC_FindSymbol(tl, tl->t, SYM_NONE); + sy1 = VCC_SymbolTok(tl, NULL, tl->t, SYM_NONE, 0); if (sy1 != NULL) { VSB_printf(tl->sb, "Object name '%.*s' already used.\n", PF(tl->t)); @@ -174,7 +174,7 @@ parse_new(struct vcc *tl) return; } - sy1 = VCC_AddSymbolTok(tl, tl->t, SYM_INSTANCE); + sy1 = VCC_SymbolTok(tl, NULL, tl->t, SYM_INSTANCE, 1); XXXAN(sy1); sy1->def_b = tl->t; vcc_NextToken(tl); @@ -183,7 +183,7 @@ parse_new(struct vcc *tl) vcc_NextToken(tl); ExpectErr(tl, ID); - sy2 = VCC_FindSymbol(tl, tl->t, SYM_OBJECT); + sy2 = VCC_SymbolTok(tl, NULL, tl->t, SYM_OBJECT, 0); if (sy2 == NULL) { VSB_printf(tl->sb, "Symbol not found: "); vcc_ErrToken(tl, tl->t); @@ -229,7 +229,7 @@ parse_new(struct vcc *tl) while (*p != '\0') { p += strlen(s_obj); bprintf(buf2, "%s%s", sy1->name, p); - sy3 = VCC_AddSymbolStr(tl, buf2, SYM_FUNC); + sy3 = VCC_Symbol(tl, NULL, buf2, NULL, SYM_FUNC, 1); AN(sy3); sy3->eval = vcc_Eval_SymFunc; p += strlen(p) + 1; @@ -415,7 +415,7 @@ vcc_ParseAction(struct vcc *tl) return (1); } } - sym = VCC_FindSymbol(tl, tl->t, SYM_NONE); + sym = VCC_SymbolTok(tl, NULL, tl->t, SYM_NONE, 0); if (sym != NULL && sym->kind == SYM_FUNC) { vcc_Expr_Call(tl, sym); return (1); diff --git a/lib/libvcc/vcc_backend.c b/lib/libvcc/vcc_backend.c index abf454d..5847e72 100644 --- a/lib/libvcc/vcc_backend.c +++ b/lib/libvcc/vcc_backend.c @@ -388,7 +388,7 @@ vcc_ParseHostDef(struct vcc *tl, const struct token *t_be, const char *vgcname) Fb(tl, 0, "\t.probe = &%s,\n", p); ERRCHK(tl); } else if (vcc_IdIs(t_field, "probe") && tl->t->tok == ID) { - if (VCC_FindSymbol(tl, tl->t, SYM_PROBE) == NULL) { + if (!VCC_SymbolTok(tl, NULL, tl->t, SYM_PROBE, 0)) { VSB_printf(tl->sb, "Probe %.*s not found\n", PF(tl->t)); vcc_ErrWhere(tl, tl->t); diff --git a/lib/libvcc/vcc_compile.c b/lib/libvcc/vcc_compile.c index 605aed3..1e11540 100644 --- a/lib/libvcc/vcc_compile.c +++ b/lib/libvcc/vcc_compile.c @@ -650,7 +650,7 @@ vcc_CompileSource(const struct vcp * const vcp, struct vsb *sb, vcc_Expr_Init(tl); VTAILQ_FOREACH(sym, &vcp->symbols, list) { - sym2 = VCC_AddSymbolStr(tl, sym->name, sym->kind); + sym2 = VCC_Symbol(tl, NULL, sym->name, NULL, sym->kind, 1); sym2->fmt = sym->fmt; sym2->eval = sym->eval; sym2->eval_priv = sym->eval_priv; @@ -662,11 +662,12 @@ vcc_CompileSource(const struct vcp * const vcp, struct vsb *sb, for (v = vcc_vars; v->name != NULL; v++) { if (v->fmt == HEADER) { - sym = VCC_AddSymbolStr(tl, v->name, SYM_WILDCARD); + sym = VCC_Symbol(tl, NULL, v->name, NULL, + SYM_WILDCARD, 1); sym->wildcard = vcc_Var_Wildcard; sym->wildcard_priv = v; } else { - sym = VCC_AddSymbolStr(tl, v->name, SYM_VAR); + sym = VCC_Symbol(tl, NULL, v->name, NULL, SYM_VAR, 1); } sym->fmt = v->fmt; sym->eval = vcc_Eval_Var; @@ -676,7 +677,7 @@ vcc_CompileSource(const struct vcp * const vcp, struct vsb *sb, sym->lname = v->lname; } - sym = VCC_AddSymbolStr(tl, "storage.", SYM_WILDCARD); + sym = VCC_Symbol(tl, NULL, "storage", NULL, SYM_WILDCARD, 1); sym->wildcard = vcc_Stv_Wildcard; Fh(tl, 0, "/* ---===### VCC generated .h code ###===---*/\n"); diff --git a/lib/libvcc/vcc_compile.h b/lib/libvcc/vcc_compile.h index ac9cb0a..556eacf 100644 --- a/lib/libvcc/vcc_compile.h +++ b/lib/libvcc/vcc_compile.h @@ -315,13 +315,8 @@ void Resolve_Sockaddr(struct vcc *tl, const char *host, const char *defport, /* vcc_symb.c */ struct symbol *VCC_Symbol(struct vcc *, struct symbol *, const char *, const char *, enum symkind, int); -struct symbol *VCC_AddSymbolStr(struct vcc *tl, const char *name, enum symkind); -struct symbol *VCC_AddSymbolTok(struct vcc *tl, const struct token *t, - enum symkind kind); -struct symbol *VCC_GetSymbolTok(struct vcc *tl, const struct token *tok, - enum symkind); -struct symbol *VCC_FindSymbol(struct vcc *tl, - const struct token *t, enum symkind kind); +#define VCC_SymbolTok(vcc, sym, tok, kind, create) \ + VCC_Symbol(vcc, sym, (tok)->b, (tok)->e, kind, create) const char * VCC_SymKind(struct vcc *tl, const struct symbol *s); typedef void symwalk_f(struct vcc *tl, const struct symbol *s); void VCC_WalkSymbols(struct vcc *tl, symwalk_f *func, enum symkind kind); diff --git a/lib/libvcc/vcc_expr.c b/lib/libvcc/vcc_expr.c index 90cd47e..4f8cfe2 100644 --- a/lib/libvcc/vcc_expr.c +++ b/lib/libvcc/vcc_expr.c @@ -814,13 +814,13 @@ vcc_expr4(struct vcc *tl, struct expr **e, enum var_type fmt) sym = NULL; kind = VCC_HandleKind(fmt); if (kind != SYM_NONE) - sym = VCC_FindSymbol(tl, tl->t, kind); + sym = VCC_SymbolTok(tl, NULL, tl->t, kind, 0); if (sym == NULL) - sym = VCC_FindSymbol(tl, tl->t, SYM_VAR); + sym = VCC_SymbolTok(tl, NULL, tl->t, SYM_VAR, 0); if (sym == NULL) - sym = VCC_FindSymbol(tl, tl->t, SYM_FUNC); + sym = VCC_SymbolTok(tl, NULL, tl->t, SYM_FUNC, 0); if (sym == NULL) - sym = VCC_FindSymbol(tl, tl->t, SYM_NONE); + sym = VCC_SymbolTok(tl, NULL, tl->t, SYM_NONE, 0); if (sym == NULL || sym->eval == NULL) { VSB_printf(tl->sb, "Symbol not found: "); vcc_ErrToken(tl, tl->t); @@ -1409,22 +1409,22 @@ vcc_Expr_Init(struct vcc *tl) { struct symbol *sym; - sym = VCC_AddSymbolStr(tl, "regsub", SYM_FUNC); + sym = VCC_Symbol(tl, NULL, "regsub", NULL, SYM_FUNC, 1); AN(sym); sym->eval = vcc_Eval_Regsub; sym->eval_priv = NULL; - sym = VCC_AddSymbolStr(tl, "regsuball", SYM_FUNC); + sym = VCC_Symbol(tl, NULL, "regsuball", NULL, SYM_FUNC, 1); AN(sym); sym->eval = vcc_Eval_Regsub; sym->eval_priv = sym; - sym = VCC_AddSymbolStr(tl, "true", SYM_FUNC); + sym = VCC_Symbol(tl, NULL, "true", NULL, SYM_FUNC, 1); AN(sym); sym->eval = vcc_Eval_BoolConst; sym->eval_priv = sym; - sym = VCC_AddSymbolStr(tl, "false", SYM_FUNC); + sym = VCC_Symbol(tl, NULL, "false", NULL, SYM_FUNC, 1); AN(sym); sym->eval = vcc_Eval_BoolConst; sym->eval_priv = NULL; diff --git a/lib/libvcc/vcc_symb.c b/lib/libvcc/vcc_symb.c index be40cc1..79c6798 100644 --- a/lib/libvcc/vcc_symb.c +++ b/lib/libvcc/vcc_symb.c @@ -158,49 +158,6 @@ VCC_Symbol(struct vcc *tl, struct symbol *parent, return (VCC_Symbol(tl, sym, ++q, e, kind, create)); } -static struct symbol * -vcc_AddSymbol(struct vcc *tl, const char *nb, int l, enum symkind kind) -{ - - return(VCC_Symbol(tl, NULL, nb, nb + l, kind, 1)); -} - -struct symbol * -VCC_AddSymbolStr(struct vcc *tl, const char *name, enum symkind kind) -{ - - return (vcc_AddSymbol(tl, name, strlen(name), kind)); -} - -struct symbol * -VCC_AddSymbolTok(struct vcc *tl, const struct token *t, enum symkind kind) -{ - - return (vcc_AddSymbol(tl, t->b, t->e - t->b, kind)); -} - -struct symbol * -VCC_GetSymbolTok(struct vcc *tl, const struct token *tok, enum symkind kind) -{ - struct symbol *sym; - - sym = VCC_FindSymbol(tl, tok, kind); - if (sym == NULL) { - sym = vcc_AddSymbol(tl, tok->b, tok->e - tok->b, kind); - AN(sym); - sym->def_b = tok; - } - return (sym); -} - -struct symbol * -VCC_FindSymbol(struct vcc *tl, const struct token *t, enum symkind kind) -{ - - assert(t->tok == ID); - return (VCC_Symbol(tl, NULL, t->b, t->e, kind, 0)); -} - static void vcc_walksymbols(struct vcc *tl, const struct symbol *root, symwalk_f *func, enum symkind kind) @@ -272,7 +229,9 @@ VCC_HandleSymbol(struct vcc *tl, const struct token *tk, enum var_type fmt, kind = VCC_HandleKind(fmt); assert(kind != SYM_NONE); - sym = VCC_GetSymbolTok(tl, tk, kind); + sym = VCC_SymbolTok(tl, NULL, tk, kind, 1); + if (sym->def_b == NULL) + sym->def_b = tk; AN(sym); if (sym->ndef > 0) { p = VCC_SymKind(tl, sym); diff --git a/lib/libvcc/vcc_var.c b/lib/libvcc/vcc_var.c index 3f4ef81..eac452d 100644 --- a/lib/libvcc/vcc_var.c +++ b/lib/libvcc/vcc_var.c @@ -99,7 +99,7 @@ vcc_FindVar(struct vcc *tl, const struct token *t, int wr_access, { const struct symbol *sym; - sym = VCC_FindSymbol(tl, t, SYM_VAR); + sym = VCC_SymbolTok(tl, NULL, t, SYM_VAR, 0); if (sym != NULL) { if (wr_access && sym->w_methods == 0) { VSB_printf(tl->sb, "Variable "); diff --git a/lib/libvcc/vcc_vmod.c b/lib/libvcc/vcc_vmod.c index 07995c3..77eefee 100644 --- a/lib/libvcc/vcc_vmod.c +++ b/lib/libvcc/vcc_vmod.c @@ -80,7 +80,7 @@ vcc_ParseImport(struct vcc *tl) mod = tl->t; vcc_NextToken(tl); - osym = VCC_FindSymbol(tl, mod, SYM_NONE); + osym = VCC_SymbolTok(tl, NULL, mod, SYM_NONE, 0); if (osym != NULL && osym->kind != SYM_VMOD) { VSB_printf(tl->sb, "Module %.*s conflicts with other symbol.\n", PF(mod)); @@ -97,7 +97,7 @@ vcc_ParseImport(struct vcc *tl) } bprintf(fn, "%.*s", PF(mod)); - msym = VCC_AddSymbolStr(tl, fn, SYM_VMOD); + msym = VCC_Symbol(tl, NULL, fn, NULL, SYM_VMOD, 1); ERRCHK(tl); AN(msym); msym->def_b = t1; @@ -220,7 +220,7 @@ vcc_ParseImport(struct vcc *tl) p = *spec; if (!strcmp(p, "$OBJ")) { p += strlen(p) + 1; - sym = VCC_AddSymbolStr(tl, p, SYM_OBJECT); + sym = VCC_Symbol(tl, NULL, p, NULL, SYM_OBJECT, 1); XXXAN(sym); sym->args = p; sym->vmod = msym->name; @@ -239,7 +239,7 @@ vcc_ParseImport(struct vcc *tl) p, PF(mod)); } else if (!strcmp(p, "$FUNC")) { p += strlen(p) + 1; - sym = VCC_AddSymbolStr(tl, p, SYM_FUNC); + sym = VCC_Symbol(tl, NULL, p, NULL, SYM_FUNC, 1); ERRCHK(tl); AN(sym); sym->vmod = msym->name; diff --git a/lib/libvcc/vcc_xref.c b/lib/libvcc/vcc_xref.c index 5440829..989a87d 100644 --- a/lib/libvcc/vcc_xref.c +++ b/lib/libvcc/vcc_xref.c @@ -78,7 +78,7 @@ vcc_AddRef(struct vcc *tl, const struct token *t, enum symkind kind) { struct symbol *sym; - sym = VCC_GetSymbolTok(tl, t, kind); + sym = VCC_SymbolTok(tl, NULL, t, kind, 1); AN(sym); sym->nref++; } @@ -88,7 +88,9 @@ vcc_AddDef(struct vcc *tl, const struct token *t, enum symkind kind) { struct symbol *sym; - sym = VCC_GetSymbolTok(tl, t, kind); + sym = VCC_SymbolTok(tl, NULL, t, kind, 1); + if (sym->def_b == NULL) + sym->def_b = t; AN(sym); sym->ndef++; return (sym->ndef); @@ -134,7 +136,7 @@ vcc_findproc(struct vcc *tl, struct token *t) struct proc *p; - sym = VCC_GetSymbolTok(tl, t, SYM_SUB); + sym = VCC_SymbolTok(tl, NULL, t, SYM_SUB, 1); AN(sym); if (sym->proc != NULL) return (sym->proc); From hermunn at varnish-software.com Mon Jun 6 15:43:07 2016 From: hermunn at varnish-software.com (Pål Hermunn Johansen) Date: Mon, 06 Jun 2016 17:43:07 +0200 Subject: [4.1] ad32407 missing Wait_HeapDelete Message-ID: commit ad32407e360ee84892a41f362bc284484a2c4c63 Author: Nils Goroll Date: Tue May 31 17:36:56 2016 +0200 missing Wait_HeapDelete fixes #1971 diff --git a/bin/varnishd/waiter/cache_waiter_ports.c b/bin/varnishd/waiter/cache_waiter_ports.c index f9e3b52..0850441 100644 --- a/bin/varnishd/waiter/cache_waiter_ports.c +++ b/bin/varnishd/waiter/cache_waiter_ports.c @@ -3,7 +3,7 @@ * Copyright (c) 2006 Varnish Software AS * Copyright (c) 2007 OmniTI Computer Consulting, Inc. * Copyright (c) 2007 Theo Schlossnagle - * Copyright (c) 2010-2015 UPLEX, Nils Goroll + * Copyright (c) 2010-2016 UPLEX, Nils Goroll * All rights reserved. * * Redistribution and use in source and binary forms, with or without @@ -165,6 +165,7 @@ vws_thread(void *priv) } CHECK_OBJ_NOTNULL(wp, WAITED_MAGIC); vws_del(vws, wp->fd); + Wait_HeapDelete(w, wp); Wait_Call(w, wp, WAITER_TIMEOUT, now); } then = vws->next - now; From hermunn at varnish-software.com Mon Jun 6 15:43:07 2016 From: hermunn at varnish-software.com (Pål Hermunn Johansen) Date: Mon, 06 Jun 2016 17:43:07 +0200 Subject: [4.1] d5a3959 Current SmartOS also has epoll(), so we need to prefer ports ... Message-ID: commit d5a3959d9f8323deb466df5cf1fba2eeb41c9a8a Author: Nils Goroll Date: Tue May 31 16:05:29 2016 +0200 Current SmartOS also has epoll(), so we need to prefer ports ... ... in order to actually use it by default diff --git a/bin/varnishd/mgt/mgt_main.c b/bin/varnishd/mgt/mgt_main.c index dab70fb..18656dc 100644 --- a/bin/varnishd/mgt/mgt_main.c +++ b/bin/varnishd/mgt/mgt_main.c @@ -248,12 +248,12 @@ usage(void) #if defined(HAVE_KQUEUE) fprintf(stderr, FMT, "", " -W kqueue"); #endif -#if defined(HAVE_EPOLL_CTL) - fprintf(stderr, FMT, "", " -W epoll"); -#endif #if defined(HAVE_PORT_CREATE) fprintf(stderr, FMT, "", " -W ports"); #endif +#if defined(HAVE_EPOLL_CTL) + fprintf(stderr, FMT, "", " -W epoll"); +#endif fprintf(stderr, FMT, "", " -W poll"); #undef FMT diff --git a/bin/varnishd/waiter/mgt_waiter.c b/bin/varnishd/waiter/mgt_waiter.c index c0e450d..58d4307 100644 --- a/bin/varnishd/waiter/mgt_waiter.c +++ b/bin/varnishd/waiter/mgt_waiter.c @@ -40,12 +40,12 @@ static const struct choice waiter_choice[] = { #if defined(HAVE_KQUEUE) { "kqueue", &waiter_kqueue }, #endif - #if defined(HAVE_EPOLL_CTL) - { "epoll", &waiter_epoll }, - #endif #if defined(HAVE_PORT_CREATE) { "ports", &waiter_ports }, #endif + #if defined(HAVE_EPOLL_CTL) + { "epoll", &waiter_epoll }, + #endif { "poll", &waiter_poll }, { NULL, NULL} }; From phk at FreeBSD.org Mon Jun 6 21:32:06 2016 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Mon, 06 Jun 2016 23:32:06 +0200 Subject: [master] 525cbc6 Make error details conditional on there being details to show. Message-ID: commit 525cbc6e442e9ab3bb62665acb4a67e0f05b96ac Author: Poul-Henning Kamp Date: Mon Jun 6 21:30:55 2016 +0000 Make error details conditional on there being details to show. diff --git a/lib/libvcc/vcc_action.c b/lib/libvcc/vcc_action.c index d81c73a..784578a 100644 --- a/lib/libvcc/vcc_action.c +++ b/lib/libvcc/vcc_action.c @@ -163,14 +163,16 @@ parse_new(struct vcc *tl) VSB_printf(tl->sb, "Object name '%.*s' already used.\n", PF(tl->t)); - VSB_printf(tl->sb, "First usage:\n"); - AN(sy1->def_b); - if (sy1->def_e != NULL) - vcc_ErrWhere2(tl, sy1->def_b, sy1->def_e); - else - vcc_ErrWhere(tl, sy1->def_b); - VSB_printf(tl->sb, "Redefinition:\n"); - vcc_ErrWhere(tl, tl->t); + if (sy1->def_b) { + VSB_printf(tl->sb, "First usage:\n"); + AN(sy1->def_b); + if (sy1->def_e != NULL) + vcc_ErrWhere2(tl, sy1->def_b, sy1->def_e); + else + vcc_ErrWhere(tl, sy1->def_b); + VSB_printf(tl->sb, "Redefinition:\n"); + vcc_ErrWhere(tl, tl->t); + } return; } From phk at FreeBSD.org Tue Jun 7 07:37:05 2016 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Tue, 07 Jun 2016 09:37:05 +0200 Subject: [master] bd30219 More symbol table cleanup. Message-ID: commit bd30219ea12cb558ff156815d8d8c455dea5fe67 Author: Poul-Henning Kamp Date: Tue Jun 7 07:36:01 2016 +0000 More symbol table cleanup. diff --git a/bin/varnishtest/tests/r01510.vtc b/bin/varnishtest/tests/r01510.vtc index 9597240..a8eb7f7 100644 --- a/bin/varnishtest/tests/r01510.vtc +++ b/bin/varnishtest/tests/r01510.vtc @@ -1,6 +1,6 @@ varnishtest "Duplicate object names" -varnish v1 -errvcl {Object name 'first' already used.} { +varnish v1 -errvcl {Instance 'first' redefined.} { import debug; sub vcl_init { new first = debug.obj("FOO"); @@ -8,7 +8,7 @@ varnish v1 -errvcl {Object name 'first' already used.} { } } -varnish v1 -errvcl {Object name 'first' already used.} { +varnish v1 -errvcl {Name 'first' already used.} { import debug; backend first { .host = "${bad_ip}"; .port = "9080"; } diff --git a/bin/varnishtest/tests/r01569.vtc b/bin/varnishtest/tests/r01569.vtc index af3c489..e3e2b1c 100644 --- a/bin/varnishtest/tests/r01569.vtc +++ b/bin/varnishtest/tests/r01569.vtc @@ -1,6 +1,6 @@ varnishtest "symbol lookup order issue" -varnish v1 -vcl { +varnish v1 -errvcl {Name 'debug' already used.} { vcl 4.0; import debug; diff --git a/bin/varnishtest/tests/v00034.vtc b/bin/varnishtest/tests/v00034.vtc index 41626c1..b4d526f 100644 --- a/bin/varnishtest/tests/v00034.vtc +++ b/bin/varnishtest/tests/v00034.vtc @@ -7,19 +7,19 @@ server s1 { varnish v1 -vcl+backend { } -start -varnish v1 -errvcl {Function c1 redefined} { +varnish v1 -errvcl {Function 'c1' redefined} { backend foo { .host = "127.0.0.1"; } sub c1 { } sub c1 { } sub vcl_recv { call c1; } } -varnish v1 -errvcl {Backend s1 redefined} { +varnish v1 -errvcl {Backend 's1' redefined} { backend s1 { .host = "127.0.0.1"; } backend s1 { .host = "127.0.0.1"; } } -varnish v1 -errvcl {Probe p1 redefined} { +varnish v1 -errvcl {Probe 'p1' redefined} { probe p1 { } probe p1 { } backend s1 { .host = "127.0.0.1"; .probe = p1;} diff --git a/lib/libvcc/vcc_action.c b/lib/libvcc/vcc_action.c index 784578a..467f5d2 100644 --- a/lib/libvcc/vcc_action.c +++ b/lib/libvcc/vcc_action.c @@ -158,27 +158,12 @@ parse_new(struct vcc *tl) ExpectErr(tl, ID); vcc_ExpectCid(tl, "VCL object"); ERRCHK(tl); - sy1 = VCC_SymbolTok(tl, NULL, tl->t, SYM_NONE, 0); - if (sy1 != NULL) { - VSB_printf(tl->sb, "Object name '%.*s' already used.\n", - PF(tl->t)); - - if (sy1->def_b) { - VSB_printf(tl->sb, "First usage:\n"); - AN(sy1->def_b); - if (sy1->def_e != NULL) - vcc_ErrWhere2(tl, sy1->def_b, sy1->def_e); - else - vcc_ErrWhere(tl, sy1->def_b); - VSB_printf(tl->sb, "Redefinition:\n"); - vcc_ErrWhere(tl, tl->t); - } - return; - } + sy1 = VCC_HandleSymbol(tl, tl->t, INSTANCE, "XXX"); + ERRCHK(tl); + + /* We allow implicit use of VMOD objects: Pretend it's ref'ed */ + sy1->nref++; - sy1 = VCC_SymbolTok(tl, NULL, tl->t, SYM_INSTANCE, 1); - XXXAN(sy1); - sy1->def_b = tl->t; vcc_NextToken(tl); ExpectErr(tl, '='); diff --git a/lib/libvcc/vcc_compile.c b/lib/libvcc/vcc_compile.c index 1e11540..8653b30 100644 --- a/lib/libvcc/vcc_compile.c +++ b/lib/libvcc/vcc_compile.c @@ -905,11 +905,13 @@ void VCP_Stevedore(struct vcp *vcp, const char *stv_name) { struct symbol *sym; + char stv[1024]; CHECK_OBJ_NOTNULL(vcp, VCP_MAGIC); ALLOC_OBJ(sym, SYMBOL_MAGIC); AN(sym); - REPLACE(sym->name, stv_name); /* XXX storage.* ? */ + bprintf(stv, "stv.%s", stv_name); + REPLACE(sym->name, stv); /* XXX storage.* ? */ sym->kind = SYM_STEVEDORE; VCC_GlobalSymbol(sym, STEVEDORE, "VRT_stevedore(\"%s\")", stv_name); VTAILQ_INSERT_TAIL(&vcp->symbols, sym, list); diff --git a/lib/libvcc/vcc_parse.c b/lib/libvcc/vcc_parse.c index e6df4c8..bcb9586 100644 --- a/lib/libvcc/vcc_parse.c +++ b/lib/libvcc/vcc_parse.c @@ -240,7 +240,7 @@ vcc_ParseFunction(struct vcc *tl) i = vcc_AddDef(tl, tl->t, SYM_SUB); if (i > 1) { VSB_printf(tl->sb, - "Function %.*s redefined\n", PF(tl->t)); + "Function '%.*s' redefined\n", PF(tl->t)); vcc_ErrWhere(tl, tl->t); return; } diff --git a/lib/libvcc/vcc_symb.c b/lib/libvcc/vcc_symb.c index 79c6798..4b9979d 100644 --- a/lib/libvcc/vcc_symb.c +++ b/lib/libvcc/vcc_symb.c @@ -42,11 +42,12 @@ enum symkind VCC_HandleKind(enum var_type fmt) { switch(fmt) { - case ACL: return(SYM_ACL); - case BACKEND: return(SYM_BACKEND); - case PROBE: return(SYM_PROBE); - case STEVEDORE: return(SYM_STEVEDORE); - default: return(SYM_NONE); + case ACL: return(SYM_ACL); + case BACKEND: return(SYM_BACKEND); + case PROBE: return(SYM_PROBE); + case STEVEDORE: return(SYM_STEVEDORE); + case INSTANCE: return(SYM_INSTANCE); + default: return(SYM_NONE); } } @@ -223,26 +224,43 @@ VCC_HandleSymbol(struct vcc *tl, const struct token *tk, enum var_type fmt, { struct symbol *sym; enum symkind kind; - va_list ap; const char *p; + va_list ap; kind = VCC_HandleKind(fmt); assert(kind != SYM_NONE); - sym = VCC_SymbolTok(tl, NULL, tk, kind, 1); - if (sym->def_b == NULL) - sym->def_b = tk; - AN(sym); - if (sym->ndef > 0) { + sym = VCC_SymbolTok(tl, NULL, tk, SYM_NONE, 0); + if (sym != NULL && sym->def_b != NULL && kind == sym->kind) { p = VCC_SymKind(tl, sym); - VSB_printf(tl->sb, "%c%s %.*s redefined\n", - toupper(*p), p+1, PF(tk)); + VSB_printf(tl->sb, "%c%s '%.*s' redefined.\n", + toupper(*p), p + 1, PF(tk)); + vcc_ErrWhere(tl, tk); + VSB_printf(tl->sb, "First definition:\n"); + AN(sym->def_b); + vcc_ErrWhere(tl, sym->def_b); + return (sym); + } else if (sym != NULL && sym->def_b != NULL) { + VSB_printf(tl->sb, "Name '%.*s' already used.\n", PF(tk)); + vcc_ErrWhere(tl, tk); + VSB_printf(tl->sb, "First definition:\n"); + AN(sym->def_b); + vcc_ErrWhere(tl, sym->def_b); + return (sym); + } else if (sym != NULL) { + VSB_printf(tl->sb, + "Name %.*s is a reserved name.\n", PF(tk)); vcc_ErrWhere(tl, tk); return (sym); } + sym = VCC_SymbolTok(tl, NULL, tk, kind, 1); + AN(sym); + AZ(sym->ndef); va_start(ap, str); vcc_global(tl, sym, fmt, str, ap); va_end(ap); sym->ndef = 1; + if (sym->def_b == NULL) + sym->def_b = tk; return (sym); } From phk at FreeBSD.org Tue Jun 7 08:21:06 2016 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Tue, 07 Jun 2016 10:21:06 +0200 Subject: [master] 632d519 VCC have not lived in the MGT process for a long time, GC the OCD-level memory-management stuff, and drop the two-stage setup. Message-ID: commit 632d519b9fdc268a0323b4a3abb0e93c83d834a5 Author: Poul-Henning Kamp Date: Tue Jun 7 08:19:10 2016 +0000 VCC have not lived in the MGT process for a long time, GC the OCD-level memory-management stuff, and drop the two-stage setup. diff --git a/bin/varnishd/mgt/mgt_vcc.c b/bin/varnishd/mgt/mgt_vcc.c index 62b12e8..79bb4be 100644 --- a/bin/varnishd/mgt/mgt_vcc.c +++ b/bin/varnishd/mgt/mgt_vcc.c @@ -83,10 +83,10 @@ static void __match_proto__(vsub_func_f) run_vcc(void *priv) { char *csrc; - struct vsb *sb; + struct vsb *sb = NULL; struct vcc_priv *vp; int fd, i, l; - struct vcp *vcp; + struct vcc *vcc; struct stevedore *stv; VJ_subproc(JAIL_SUBPROC_VCC); @@ -94,20 +94,18 @@ run_vcc(void *priv) AZ(chdir(vp->dir)); - sb = VSB_new_auto(); - XXXAN(sb); - vcp = VCP_New(); - AN(vcp); - VCP_Builtin_VCL(vcp, builtin_vcl); - VCP_VCL_path(vcp, mgt_vcl_path); - VCP_VMOD_path(vcp, mgt_vmod_path); - VCP_Err_Unref(vcp, mgt_vcc_err_unref); - VCP_Allow_InlineC(vcp, mgt_vcc_allow_inline_c); - VCP_Unsafe_Path(vcp, mgt_vcc_unsafe_path); + vcc = VCC_New(); + AN(vcc); + VCC_Builtin_VCL(vcc, builtin_vcl); + VCC_VCL_path(vcc, mgt_vcl_path); + VCC_VMOD_path(vcc, mgt_vmod_path); + VCC_Err_Unref(vcc, mgt_vcc_err_unref); + VCC_Allow_InlineC(vcc, mgt_vcc_allow_inline_c); + VCC_Unsafe_Path(vcc, mgt_vcc_unsafe_path); VTAILQ_FOREACH(stv, &stv_stevedores, list) - VCP_Stevedore(vcp, stv->ident); - VCP_Stevedore(vcp, stv_transient->ident); - csrc = VCC_Compile(vcp, sb, vp->vclsrc, vp->vclsrcfile); + VCC_Stevedore(vcc, stv->ident); + VCC_Stevedore(vcc, stv_transient->ident); + csrc = VCC_Compile(vcc, &sb, vp->vclsrc, vp->vclsrcfile); AZ(VSB_finish(sb)); if (VSB_len(sb)) printf("%s", VSB_data(sb)); diff --git a/include/libvcc.h b/include/libvcc.h index 1ae4b8a..2b69bd8 100644 --- a/include/libvcc.h +++ b/include/libvcc.h @@ -28,16 +28,16 @@ * */ -struct vcp; +struct vcc; -struct vcp *VCP_New(void); -void VCP_Allow_InlineC(struct vcp *, unsigned); -void VCP_Builtin_VCL(struct vcp *, const char *); -void VCP_Err_Unref(struct vcp *, unsigned); -void VCP_Stevedore(struct vcp *, const char *); -void VCP_Unsafe_Path(struct vcp *, unsigned); -void VCP_VCL_path(struct vcp *, const char *); -void VCP_VMOD_path(struct vcp *, const char *); +struct vcc *VCC_New(void); +void VCC_Allow_InlineC(struct vcc *, unsigned); +void VCC_Builtin_VCL(struct vcc *, const char *); +void VCC_Err_Unref(struct vcc *, unsigned); +void VCC_Stevedore(struct vcc *, const char *); +void VCC_Unsafe_Path(struct vcc *, unsigned); +void VCC_VCL_path(struct vcc *, const char *); +void VCC_VMOD_path(struct vcc *, const char *); -char *VCC_Compile(const struct vcp *, struct vsb *, +char *VCC_Compile(struct vcc *, struct vsb **, const char *vclsrc, const char *vclsrcfile); diff --git a/lib/libvcc/vcc_acl.c b/lib/libvcc/vcc_acl.c index 18d967d..7c7ec97 100644 --- a/lib/libvcc/vcc_acl.c +++ b/lib/libvcc/vcc_acl.c @@ -364,7 +364,7 @@ vcc_acl_emit(struct vcc *tl, const char *acln, int anon) Fh(tl, 0, "\t\tVRT_acl_log(ctx, \"NO_FAM %s\");\n", acln); Fh(tl, 0, "\t\treturn(0);\n"); Fh(tl, 0, "\t}\n\n"); - if (!tl->param->err_unref && !anon) { + if (!tl->err_unref && !anon) { ifp = New_IniFin(tl); VSB_printf(ifp->ini, "\tif (0) match_acl_named_%s(0, 0);\n", acln); diff --git a/lib/libvcc/vcc_compile.c b/lib/libvcc/vcc_compile.c index 8653b30..6254794 100644 --- a/lib/libvcc/vcc_compile.c +++ b/lib/libvcc/vcc_compile.c @@ -74,26 +74,14 @@ struct method method_tab[] = { /*--------------------------------------------------------------------*/ -static void -TlDoFree(struct vcc *tl, void *p) -{ - struct membit *mb; - - mb = calloc(sizeof *mb, 1); - assert(mb != NULL); - mb->ptr = p; - VTAILQ_INSERT_TAIL(&tl->membits, mb, list); -} - - -void * +void * __match_proto__(TlAlloc) TlAlloc(struct vcc *tl, unsigned len) { void *p; + (void)tl; p = calloc(len, 1); assert(p != NULL); - TlDoFree(tl, p); return (p); } @@ -453,31 +441,21 @@ vcc_new_source(const char *b, const char *e, const char *name) return (sp); } -static void -vcc_destroy_source(struct source *sp) -{ - - if (sp->freeit != NULL) - free(sp->freeit); - free(sp->name); - free(sp); -} - /*--------------------------------------------------------------------*/ static struct source * -vcc_file_source(const struct vcp * const vcp, struct vsb *sb, const char *fn) +vcc_file_source(struct vcc *tl, const char *fn) { char *f, *fnp; struct source *sp; - if (!vcp->unsafe_path && strchr(fn, '/') != NULL) { - VSB_printf(sb, "VCL filename '%s' is unsafe.\n", fn); + if (!tl->unsafe_path && strchr(fn, '/') != NULL) { + VSB_printf(tl->sb, "VCL filename '%s' is unsafe.\n", fn); return (NULL); } f = NULL; - if (VFIL_searchpath(vcp->vcl_path, NULL, &f, fn, &fnp) || f == NULL) { - VSB_printf(sb, "Cannot read file '%s' (%s)\n", + if (VFIL_searchpath(tl->vcl_path, NULL, &f, fn, &fnp) || f == NULL) { + VSB_printf(tl->sb, "Cannot read file '%s' (%s)\n", fnp != NULL ? fnp : fn, strerror(errno)); free(fnp); return (NULL); @@ -540,10 +518,10 @@ vcc_resolve_includes(struct vcc *tl) VSB_bcat(vsb, t1->src->name, p - t1->src->name); VSB_cat(vsb, t1->dec + 1); AZ(VSB_finish(vsb)); - sp = vcc_file_source(tl->param, tl->sb, VSB_data(vsb)); + sp = vcc_file_source(tl, VSB_data(vsb)); VSB_destroy(&vsb); } else { - sp = vcc_file_source(tl->param, tl->sb, t1->dec); + sp = vcc_file_source(tl, t1->dec); } if (sp == NULL) { vcc_ErrWhere(tl, t1); @@ -563,103 +541,22 @@ vcc_resolve_includes(struct vcc *tl) } } -/*--------------------------------------------------------------------*/ - -static struct vcc * -vcc_NewVcc(const struct vcp *vcp) -{ - struct vcc *tl; - int i; - - CHECK_OBJ_NOTNULL(vcp, VCP_MAGIC); - ALLOC_OBJ(tl, VCC_MAGIC); - AN(tl); - tl->param = vcp; - VTAILQ_INIT(&tl->inifin); - VTAILQ_INIT(&tl->membits); - VTAILQ_INIT(&tl->tokens); - VTAILQ_INIT(&tl->sources); - - tl->nsources = 0; - - /* General C code */ - tl->fc = VSB_new_auto(); - assert(tl->fc != NULL); - - /* Forward decls (.h like) */ - tl->fh = VSB_new_auto(); - assert(tl->fh != NULL); - - /* body code of methods */ - for (i = 0; i < VCL_MET_MAX; i++) { - tl->fm[i] = VSB_new_auto(); - assert(tl->fm[i] != NULL); - } - return (tl); -} - -/*--------------------------------------------------------------------*/ - -static char * -vcc_DestroyTokenList(struct vcc *tl, char *ret) -{ - struct membit *mb; - struct source *sp; - int i; - - while (!VTAILQ_EMPTY(&tl->membits)) { - mb = VTAILQ_FIRST(&tl->membits); - VTAILQ_REMOVE(&tl->membits, mb, list); - free(mb->ptr); - free(mb); - } - while (!VTAILQ_EMPTY(&tl->sources)) { - sp = VTAILQ_FIRST(&tl->sources); - VTAILQ_REMOVE(&tl->sources, sp, list); - vcc_destroy_source(sp); - } - - VSB_destroy(&tl->fh); - VSB_destroy(&tl->fc); - for (i = 0; i < VCL_MET_MAX; i++) - VSB_destroy(&tl->fm[i]); - - free(tl); - return (ret); -} - /*-------------------------------------------------------------------- * Compile the VCL code from the given source and return the C-source */ static char * -vcc_CompileSource(const struct vcp * const vcp, struct vsb *sb, - struct source *sp) +vcc_CompileSource(struct vcc *tl, struct source *sp) { - struct vcc *tl; - struct symbol *sym, *sym2; + struct symbol *sym; const struct var *v; struct vsb *vsb; char *of; int i; - tl = vcc_NewVcc(vcp); - tl->sb = sb; - vcc_Expr_Init(tl); - VTAILQ_FOREACH(sym, &vcp->symbols, list) { - sym2 = VCC_Symbol(tl, NULL, sym->name, NULL, sym->kind, 1); - sym2->fmt = sym->fmt; - sym2->eval = sym->eval; - sym2->eval_priv = sym->eval_priv; - sym2->rname = sym->rname; - sym2->r_methods = sym->r_methods; - sym2->ndef = 1; - sym2->nref = 1; - } - for (v = vcc_vars; v->name != NULL; v++) { if (v->fmt == HEADER) { sym = VCC_Symbol(tl, NULL, v->name, NULL, @@ -689,32 +586,32 @@ vcc_CompileSource(const struct vcp * const vcp, struct vsb *sb, sp->idx = tl->nsources++; vcc_Lexer(tl, sp); if (tl->err) - return (vcc_DestroyTokenList(tl, NULL)); + return (NULL); /* Register and lex the builtin VCL */ - sp = vcc_new_source(tl->param->builtin_vcl, NULL, "Builtin"); + sp = vcc_new_source(tl->builtin_vcl, NULL, "Builtin"); assert(sp != NULL); VTAILQ_INSERT_TAIL(&tl->sources, sp, list); sp->idx = tl->nsources++; vcc_Lexer(tl, sp); if (tl->err) - return (vcc_DestroyTokenList(tl, NULL)); + return (NULL); /* Add "END OF INPUT" token */ vcc_AddToken(tl, EOI, sp->e, sp->e); if (tl->err) - return (vcc_DestroyTokenList(tl, NULL)); + return (NULL); /* Expand and lex any includes in the token string */ vcc_resolve_includes(tl); if (tl->err) - return (vcc_DestroyTokenList(tl, NULL)); + return (NULL); /* Parse the token string */ tl->t = VTAILQ_FIRST(&tl->tokens); vcc_Parse(tl); if (tl->err) - return (vcc_DestroyTokenList(tl, NULL)); + return (NULL); /* Check if we have any backends at all */ if (tl->default_director == NULL) { @@ -722,7 +619,7 @@ vcc_CompileSource(const struct vcp * const vcp, struct vsb *sb, "No backends or directors found in VCL program, " "at least one is necessary.\n"); tl->err = 1; - return (vcc_DestroyTokenList(tl, NULL)); + return (NULL); } /* Configure the default director */ @@ -730,15 +627,15 @@ vcc_CompileSource(const struct vcp * const vcp, struct vsb *sb, /* Check for orphans */ if (vcc_CheckReferences(tl)) - return (vcc_DestroyTokenList(tl, NULL)); + return (NULL); /* Check that all action returns are legal */ if (vcc_CheckAction(tl) || tl->err) - return (vcc_DestroyTokenList(tl, NULL)); + return (NULL); /* Check that all variable uses are legal */ if (vcc_CheckUses(tl) || tl->err) - return (vcc_DestroyTokenList(tl, NULL)); + return (NULL); /* Emit method functions */ Fh(tl, 1, "\n"); @@ -793,7 +690,7 @@ vcc_CompileSource(const struct vcp * const vcp, struct vsb *sb, VSB_destroy(&vsb); /* done */ - return (vcc_DestroyTokenList(tl, of)); + return (of); } /*-------------------------------------------------------------------- @@ -802,22 +699,23 @@ vcc_CompileSource(const struct vcp * const vcp, struct vsb *sb, */ char * -VCC_Compile(const struct vcp *vcp, struct vsb *sb, +VCC_Compile(struct vcc *tl, struct vsb **sb, const char *vclsrc, const char *vclsrcfile) { struct source *sp; - char *r; - CHECK_OBJ_NOTNULL(vcp, VCP_MAGIC); + char *r = NULL; + + CHECK_OBJ_NOTNULL(tl, VCC_MAGIC); AN(sb); AN(vclsrcfile); if (vclsrc != NULL) sp = vcc_new_source(vclsrc, NULL, vclsrcfile); else - sp = vcc_file_source(vcp, sb, vclsrcfile); - if (sp == NULL) - return (NULL); - r = vcc_CompileSource(vcp, sb, sp); + sp = vcc_file_source(tl, vclsrcfile); + if (sp != NULL) + r = vcc_CompileSource(tl, sp); + *sb = tl->sb; return (r); } @@ -825,16 +723,36 @@ VCC_Compile(const struct vcp *vcp, struct vsb *sb, * Allocate a compiler instance */ -struct vcp * -VCP_New(void) +struct vcc * +VCC_New(void) { - struct vcp *vcp; + struct vcc *tl; + int i; + + ALLOC_OBJ(tl, VCC_MAGIC); + AN(tl); + VTAILQ_INIT(&tl->inifin); + VTAILQ_INIT(&tl->tokens); + VTAILQ_INIT(&tl->sources); - ALLOC_OBJ(vcp, VCP_MAGIC); - AN(vcp); - VTAILQ_INIT(&vcp->symbols); + tl->nsources = 0; - return (vcp); + /* General C code */ + tl->fc = VSB_new_auto(); + assert(tl->fc != NULL); + + /* Forward decls (.h like) */ + tl->fh = VSB_new_auto(); + assert(tl->fh != NULL); + + /* body code of methods */ + for (i = 0; i < VCL_MET_MAX; i++) { + tl->fm[i] = VSB_new_auto(); + assert(tl->fm[i] != NULL); + } + tl->sb = VSB_new_auto(); + AN(tl->sb); + return (tl); } /*-------------------------------------------------------------------- @@ -842,11 +760,11 @@ VCP_New(void) */ void -VCP_Builtin_VCL(struct vcp *vcp, const char *str) +VCC_Builtin_VCL(struct vcc *vcc, const char *str) { - CHECK_OBJ_NOTNULL(vcp, VCP_MAGIC); - REPLACE(vcp->builtin_vcl, str); + CHECK_OBJ_NOTNULL(vcc, VCC_MAGIC); + REPLACE(vcc->builtin_vcl, str); } /*-------------------------------------------------------------------- @@ -854,11 +772,11 @@ VCP_Builtin_VCL(struct vcp *vcp, const char *str) */ void -VCP_VCL_path(struct vcp *vcp, const char *str) +VCC_VCL_path(struct vcc *vcc, const char *str) { - CHECK_OBJ_NOTNULL(vcp, VCP_MAGIC); - VFIL_setpath(&vcp->vcl_path, str); + CHECK_OBJ_NOTNULL(vcc, VCC_MAGIC); + VFIL_setpath(&vcc->vcl_path, str); } /*-------------------------------------------------------------------- @@ -866,11 +784,11 @@ VCP_VCL_path(struct vcp *vcp, const char *str) */ void -VCP_VMOD_path(struct vcp *vcp, const char *str) +VCC_VMOD_path(struct vcc *vcc, const char *str) { - CHECK_OBJ_NOTNULL(vcp, VCP_MAGIC); - VFIL_setpath(&vcp->vmod_path, str); + CHECK_OBJ_NOTNULL(vcc, VCC_MAGIC); + VFIL_setpath(&vcc->vmod_path, str); } /*-------------------------------------------------------------------- @@ -878,42 +796,47 @@ VCP_VMOD_path(struct vcp *vcp, const char *str) */ void -VCP_Err_Unref(struct vcp *vcp, unsigned u) +VCC_Err_Unref(struct vcc *vcc, unsigned u) { - CHECK_OBJ_NOTNULL(vcp, VCP_MAGIC); - vcp->err_unref = u; + CHECK_OBJ_NOTNULL(vcc, VCC_MAGIC); + vcc->err_unref = u; } void -VCP_Allow_InlineC(struct vcp *vcp, unsigned u) +VCC_Allow_InlineC(struct vcc *vcc, unsigned u) { - CHECK_OBJ_NOTNULL(vcp, VCP_MAGIC); - vcp->allow_inline_c = u; + CHECK_OBJ_NOTNULL(vcc, VCC_MAGIC); + vcc->allow_inline_c = u; } void -VCP_Unsafe_Path(struct vcp *vcp, unsigned u) +VCC_Unsafe_Path(struct vcc *vcc, unsigned u) { - CHECK_OBJ_NOTNULL(vcp, VCP_MAGIC); - vcp->unsafe_path = u; + CHECK_OBJ_NOTNULL(vcc, VCC_MAGIC); + vcc->unsafe_path = u; } void -VCP_Stevedore(struct vcp *vcp, const char *stv_name) +VCC_Stevedore(struct vcc *vcc, const char *stv_name) { +#if 0 struct symbol *sym; char stv[1024]; - CHECK_OBJ_NOTNULL(vcp, VCP_MAGIC); + CHECK_OBJ_NOTNULL(vcc, VCC_MAGIC); ALLOC_OBJ(sym, SYMBOL_MAGIC); AN(sym); bprintf(stv, "stv.%s", stv_name); REPLACE(sym->name, stv); /* XXX storage.* ? */ sym->kind = SYM_STEVEDORE; VCC_GlobalSymbol(sym, STEVEDORE, "VRT_stevedore(\"%s\")", stv_name); - VTAILQ_INSERT_TAIL(&vcp->symbols, sym, list); + VTAILQ_INSERT_TAIL(&vcc->symbols, sym, list); +#else + (void)vcc; + (void)stv_name; +#endif } diff --git a/lib/libvcc/vcc_compile.h b/lib/libvcc/vcc_compile.h index 556eacf..9999230 100644 --- a/lib/libvcc/vcc_compile.h +++ b/lib/libvcc/vcc_compile.h @@ -74,11 +74,6 @@ enum var_type { #undef VCC_TYPE }; -struct membit { - VTAILQ_ENTRY(membit) list; - void *ptr; -}; - struct source { VTAILQ_ENTRY(source) list; char *name; @@ -160,9 +155,10 @@ struct inifin { VTAILQ_HEAD(inifinhead, inifin); -struct vcp { +struct vcc { unsigned magic; -#define VCP_MAGIC 0xd90acfbc +#define VCC_MAGIC 0x24ad719d + float syntax; char *builtin_vcl; struct vfil_path *vcl_path; @@ -171,17 +167,6 @@ struct vcp { unsigned allow_inline_c; unsigned unsafe_path; - VTAILQ_HEAD(,symbol) symbols; -}; - -struct vcc { - unsigned magic; -#define VCC_MAGIC 0x24ad719d - float syntax; - - /* Parameter/Template section */ - const struct vcp *param; - struct symbol *symbols; struct inifinhead inifin; @@ -190,7 +175,6 @@ struct vcc { /* Instance section */ struct tokenhead tokens; VTAILQ_HEAD(, source) sources; - VTAILQ_HEAD(, membit) membits; unsigned nsources; struct source *src; struct token *t; diff --git a/lib/libvcc/vcc_parse.c b/lib/libvcc/vcc_parse.c index bcb9586..aa242f8 100644 --- a/lib/libvcc/vcc_parse.c +++ b/lib/libvcc/vcc_parse.c @@ -160,7 +160,7 @@ vcc_Compound(struct vcc *tl) Fb(tl, 1, "}\n"); return; case CSRC: - if (tl->param->allow_inline_c) { + if (tl->allow_inline_c) { Fb(tl, 1, "%.*s\n", (int) (tl->t->e - (tl->t->b + 2)), tl->t->b + 1); @@ -352,7 +352,7 @@ vcc_Parse(struct vcc *tl) ERRCHK(tl); switch (tl->t->tok) { case CSRC: - if (tl->param->allow_inline_c) { + if (tl->allow_inline_c) { Fc(tl, 0, "%.*s\n", (int) (tl->t->e - (tl->t->b + 4)), tl->t->b + 2); diff --git a/lib/libvcc/vcc_vmod.c b/lib/libvcc/vcc_vmod.c index 77eefee..9f6f59e 100644 --- a/lib/libvcc/vcc_vmod.c +++ b/lib/libvcc/vcc_vmod.c @@ -110,7 +110,7 @@ vcc_ParseImport(struct vcc *tl) return; } vcc_NextToken(tl); - if (!tl->param->unsafe_path && strchr(tl->t->dec, '/')) { + if (!tl->unsafe_path && strchr(tl->t->dec, '/')) { VSB_printf(tl->sb, "'import ... from path ...' is unsafe.\nAt:"); vcc_ErrToken(tl, tl->t); @@ -130,7 +130,7 @@ vcc_ParseImport(struct vcc *tl) SkipToken(tl, ';'); - if (VFIL_searchpath(tl->param->vmod_path, + if (VFIL_searchpath(tl->vmod_path, vcc_path_dlopen, &hdl, fn, &fnpx)) { VSB_printf(tl->sb, "Could not load VMOD %.*s\n", PF(mod)); VSB_printf(tl->sb, "\tFile name: %s\n", diff --git a/lib/libvcc/vcc_xref.c b/lib/libvcc/vcc_xref.c index 989a87d..e8feac2 100644 --- a/lib/libvcc/vcc_xref.c +++ b/lib/libvcc/vcc_xref.c @@ -110,7 +110,7 @@ vcc_checkref(struct vcc *tl, const struct symbol *sym) VSB_printf(tl->sb, "Unused %s %.*s, defined:\n", VCC_SymKind(tl, sym), PF(sym->def_b)); vcc_ErrWhere(tl, sym->def_b); - if (!tl->param->err_unref) { + if (!tl->err_unref) { VSB_printf(tl->sb, "(That was just a warning)\n"); tl->err = 0; } @@ -291,7 +291,7 @@ vcc_checkaction2(struct vcc *tl, const struct symbol *sym) return; VSB_printf(tl->sb, "Function unused\n"); vcc_ErrWhere(tl, p->name); - if (!tl->param->err_unref) { + if (!tl->err_unref) { VSB_printf(tl->sb, "(That was just a warning)\n"); tl->err = 0; } From phk at FreeBSD.org Tue Jun 7 08:59:06 2016 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Tue, 07 Jun 2016 10:59:06 +0200 Subject: [master] 7e56280 Forgot to add the "INSTANCE" type (for VCL "new = vmod.bla" instances) Message-ID: commit 7e56280ba77f29d29d7beb7274c1798bb6c6b642 Author: Poul-Henning Kamp Date: Tue Jun 7 08:57:32 2016 +0000 Forgot to add the "INSTANCE" type (for VCL "new = vmod.bla" instances) diff --git a/include/vrt.h b/include/vrt.h index ef679bb..4ef7dec 100644 --- a/include/vrt.h +++ b/include/vrt.h @@ -89,6 +89,7 @@ typedef double VCL_DURATION; typedef const char * VCL_ENUM; typedef const struct gethdr_s * VCL_HEADER; typedef struct http * VCL_HTTP; +typedef void VCL_INSTANCE; typedef long VCL_INT; typedef const struct suckaddr * VCL_IP; typedef const struct vrt_backend_probe * VCL_PROBE; From hermunn at varnish-software.com Tue Jun 7 12:07:06 2016 From: hermunn at varnish-software.com (Pål Hermunn Johansen) Date: Tue, 07 Jun 2016 14:07:06 +0200 Subject: [4.1] c07c58d Remove implicit line feed when using formatfile. Message-ID: commit c07c58d782d3e8fcd58ef28e9063cf19c10134fa Author: Lasse Karstensen Date: Mon May 30 13:35:53 2016 +0200 Remove implicit line feed when using formatfile. Fixes: #1967 diff --git a/bin/varnishncsa/varnishncsa.c b/bin/varnishncsa/varnishncsa.c index 76eea41..9a2e89c 100644 --- a/bin/varnishncsa/varnishncsa.c +++ b/bin/varnishncsa/varnishncsa.c @@ -1104,12 +1104,14 @@ read_format(const char *formatfile) { FILE *fmtfile; size_t len = 0; + int fmtlen; char *fmt = NULL; fmtfile = fopen(formatfile, "r"); if (fmtfile == NULL) VUT_Error(1, "Can't open format file (%s)", strerror(errno)); - if (getline(&fmt, &len, fmtfile) == -1) { + fmtlen = getline(&fmt, &len, fmtfile); + if (fmtlen == -1) { free(fmt); if (feof(fmtfile)) VUT_Error(1, "Empty format file"); @@ -1118,6 +1120,8 @@ read_format(const char *formatfile) strerror(errno)); } fclose(fmtfile); + if (fmt[fmtlen - 1] == '\n') + fmt[fmtlen - 1] = '\0'; return (fmt); } From varnish-commit at varnish-cache.org Tue Jun 7 13:39:38 2016 From: varnish-commit at varnish-cache.org (dgauhel) Date: Tue, 7 Jun 2016 21:39:38 +0800 Subject: =?utf-8?B?dmFybmlzaC1jb21taXTvvJrlt6XkuJo0LjA=?= =?utf-8?B?6IOM5pmv5LiL5Lit5Zu95Yi26YCg5Lia55qE5piO5aSp?= Message-ID: <20160607213949678326@rxle.org> ???varnish-commit:??? ??4.0??????????? ?????????????????????? ???????????????????? ??????????????????? ??????????????? ??? 2016-6-721:39:47 -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ??4.0???????????.docx Type: application/vnd.openxmlformats-officedocument.wordprocessingml.document Size: 19840 bytes Desc: not available URL: From varnish-commit at varnish-cache.org Tue Jun 7 17:03:23 2016 From: varnish-commit at varnish-cache.org (st) Date: Wed, 8 Jun 2016 01:03:23 +0800 Subject: =?utf-8?B?dmFybmlzaC1jb21taXQ65aaC5L2V5aSE55CG5ZGY?= =?utf-8?B?5bel6L+d57qq6Zeu6aKY77yM5Y+K5pyJ5pWI6LCD5bKX77yM6LCD6Jaq?= Message-ID: <20160608010333417324@atvvm.net> varnish-commit: ?? 1.???????????????????????????? 2.?????????????????? 3.???????????????????????? 4.?????????????????????????? 5.?????????????? 6.?????????????????????????? 7.?????????????????????????? 8.??????????????????? 9.??????????????????????? ??????????????????? ???????????? 2016/6/8 ???1:03:32 ????? -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ??????????????????????.docx Type: application/vnd.openxmlformats-officedocument.wordprocessingml.document Size: 27721 bytes Desc: not available URL: From phk at phk.freebsd.dk Wed Jun 8 07:42:07 2016 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Wed, 08 Jun 2016 07:42:07 +0000 Subject: [master] 17b878b Recoding doxygen in awk is fun In-Reply-To: References: Message-ID: <75975.1465371727@critter.freebsd.dk> -------- In message , Guillaume Quintard writes: > Recoding doxygen in awk is fun This seems to require gawk, which is not a dependency I think we had previously ? At the very least this should happen: --- a/doc/sphinx/Makefile.am +++ b/doc/sphinx/Makefile.am @@ -181,7 +181,7 @@ VTCSYN_SRC = $(top_srcdir)/bin/varnishtest/vtc.c \ $(top_srcdir)/bin/varnishtest/vtc_logexp.c \ $(top_srcdir)/bin/varnishtest/vtc_varnish.c include/vtc-syntax.rst: vtc-syntax.awk $(VTCSYN_SRC) - awk -f $(top_srcdir)/doc/sphinx/vtc-syntax.awk $(VTCSYN_SRC) > $@ + ${AWK} -f $(top_srcdir)/doc/sphinx/vtc-syntax.awk $(VTCSYN_SRC) > $@ BUILT_SOURCES += include/vtc-syntax.rst .PHONY: reference -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From guillaume at varnish-software.com Wed Jun 8 10:17:06 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Wed, 08 Jun 2016 12:17:06 +0200 Subject: [master] 3a263d9 use Python instead of awk Message-ID: commit 3a263d940cf24d36a85cb835291c3062677dd87b Author: Guillaume Quintard Date: Wed Jun 8 12:02:39 2016 +0200 use Python instead of awk diff --git a/doc/sphinx/Makefile.am b/doc/sphinx/Makefile.am index 696b8ef..b9713d9 100644 --- a/doc/sphinx/Makefile.am +++ b/doc/sphinx/Makefile.am @@ -180,8 +180,8 @@ VTCSYN_SRC = $(top_srcdir)/bin/varnishtest/vtc.c \ $(top_srcdir)/bin/varnishtest/vtc_http.c \ $(top_srcdir)/bin/varnishtest/vtc_logexp.c \ $(top_srcdir)/bin/varnishtest/vtc_varnish.c -include/vtc-syntax.rst: vtc-syntax.awk $(VTCSYN_SRC) - awk -f $(top_srcdir)/doc/sphinx/vtc-syntax.awk $(VTCSYN_SRC) > $@ +include/vtc-syntax.rst: vtc-syntax.py $(VTCSYN_SRC) + $(PYTHON) $(top_srcdir)/doc/sphinx/vtc-syntax.py $(VTCSYN_SRC) > $@ BUILT_SOURCES += include/vtc-syntax.rst .PHONY: reference diff --git a/doc/sphinx/vtc-syntax.awk b/doc/sphinx/vtc-syntax.awk deleted file mode 100644 index 58ce4ad..0000000 --- a/doc/sphinx/vtc-syntax.awk +++ /dev/null @@ -1,43 +0,0 @@ -# end of paragraph if we hit '*/' -$0 ~ "*/" { - p = 0; -} - -# if in paragraph and no section is announced, -# concatenate -p && $0 !~ "[ /]* SECTION: " { - cl[section] = cl[section] gensub(/ \* ?/, "", "1", $0) "\n"; -} - -# section announcement -$0 ~ "[ /]* SECTION: " { - section = $3; - sl[len++] = section; - if ($4) { - tl[section] = gensub(/[\t ]*\/?\* SECTION: [^ ]+ +/, "", "1", $0); - } else { - tl[section] = ""; - } - p = 1; -} - -# sort sections, underline titles, print -END { - asort(sl); - for (i in sl) { - section = sl[i] - print(tl[section]); - a = section - c = gsub(/\./, "", a); - if (c == 0) - r = "-"; - else if (c == 1) - r = "~" - else if (c == 2) - r = "." - else - r = "." - print(gensub(/./, r, "g", tl[section])); - print(cl[section]); - } -} diff --git a/doc/sphinx/vtc-syntax.py b/doc/sphinx/vtc-syntax.py new file mode 100644 index 0000000..f0601b3 --- /dev/null +++ b/doc/sphinx/vtc-syntax.py @@ -0,0 +1,80 @@ +#!/usr/bin/env python +#- +# Copyright (c) 2006-2016 Varnish Software AS +# All rights reserved. +# +# Author: Guillaume Quintard +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# 1. Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# 2. Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. +# +# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +# ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE +# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +# SUCH DAMAGE. +# +# Generate various .c and .h files for the VSL query expression parser +# and the interfaces for it. + +from __future__ import print_function +import sys; +import re; + +def parse_file(fn): + p = False + section = "" + resec = re.compile("[ /]\* SECTION: ") + cl = {} + tl = {} + sl = [] + + f = open(fn, 'r') + + for l in f: + if "*/" in l: + p = 0 + if resec.match(l): + a = l.split() + section = a[2] + sl.append(section) + cl[section] = [] + if len(a) > 3: + tl[section] = re.sub(r'^[\t ]*\/?\* SECTION: [^ ]+ +', + "", l) + else: + tl[section] = "" + p = 1 + elif p: + cl[section].append(re.sub(r'^ \* ?',"", l)) + f.close() + + sl.sort() + for section in sl: + print(tl[section], end=''); + a = section + c = section.count('.') + if c == 0: + r = "-" + elif c ==1: + r = "~" + else: + r = "." + print(re.sub(r'.', r, tl[section]), end='') + print("".join(cl[section])) + +if __name__ == "__main__": + for fn in sys.argv[1:]: + parse_file(fn) From guillaume at varnish-software.com Wed Jun 8 10:28:06 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Wed, 08 Jun 2016 12:28:06 +0200 Subject: [master] cf22b85 Remove syntax description from varnishtest man Message-ID: commit cf22b85525d7e97fac84ddcc5c95e25718dcee4d Author: Guillaume Quintard Date: Tue Jun 7 13:47:56 2016 +0200 Remove syntax description from varnishtest man diff --git a/doc/sphinx/reference/varnishtest.rst b/doc/sphinx/reference/varnishtest.rst index 17cdc53..680b8ff 100644 --- a/doc/sphinx/reference/varnishtest.rst +++ b/doc/sphinx/reference/varnishtest.rst @@ -67,10 +67,10 @@ If `TMPDIR` is set in the environment, varnishtest creates temporary SCRIPTS ======= -The script language used for Varnishtest is not a strictly defined -language. The best reference for writing scripts is the varnishtest program -itself. In the Varnish source code repository, under -`bin/varnishtest/tests/`, all the regression tests for Varnish are kept. +The vtc syntax is documented at length in :ref:`vtc(7)`. Should you want more +examples than the one below, you can have a look at the Varnish source code +repository, under `bin/varnishtest/tests/`, where all the regression tests for +Varnish are kept. An example:: @@ -173,156 +173,11 @@ both your VMOD and the built-in VMODs:: varnishtest -p vmod_path="$VMOD_PATH" ... - -AVAILABLE COMMANDS -================== - -server -****** - -Creates a mock server that can accept requests from Varnish and send -responses. Accepted parameters: - -\-listen - specifies address and port to listen on (e.g. "127.0.0.1:80") - -client -****** - -Creates a client instance that sends requests to Varnish and receives -responses. By default, a client will try and connect to the first -varnish server available. - -Accepted parameters: - -\-connect - specify where to connect to (e.g. "-connect ${s1_sock}"). - -server/client command arguments -******************************* - -\-repeat INT - repeats the commands INT in order -\-wait - waits for commands to complete -\-start - starts the client, and continue without waiting for completion -\-run - equivalent to -start then -wait - - -varnish -******* - -Starts Varnish instance. Accepted arguments: - -\-arg STRING - passes additional arguments to varnishd -\-cli - executes a command in CLI of running instance -\-cliok - executes a command and expect it return OK status -\-clierr - executes a command and expect it to error with given status - (e.g. "-clierr 300 panic.clear") -\-vcl STRING - specify VCL for the instance. You can create multiline strings by - encasing them in curly braces. -\-vcl+backend STRING - specifes VCL for the instance, and automatically inject backends - definition of currently defined servers. -\-errvcl - tests that invalid VCL results in an error. -\-stop - stops the instance -\-wait-stopped - waits for the varnish child to stop -\-wait-running - waits for the varnish child to start -\-wait - waits for varnish to stop -\-expect - sets up a test for asserting variables against expected results. - Syntax: "-expect " - -See tests supplied with Varnish distribution for usage examples for all -these directives. - -delay -***** - -Sleeps for specified number of seconds. - -Usage: ``delay FLOAT`` - -varnishtest -*********** - -Accepts a string as an only argument. This being a test name that is being -output into the log. By default, test name is not shown, unless it fails. - -shell -***** - -Executes a shell command. Accepts one argument as a string, and runs -the command as is. - -Usage: ``shell "CMD"`` - -err_shell -********* - -Usage: ``err_shell "STRING" "CMD"`` - -barrier -******* - -Usage: ``barrier NAME [arguments]`` - -random -****** - -Initializes random generator (need to call std.random() in vcl). See -m00002.vtc for more info. - -feature -******* - -Checks for features to be present in the test environment. If feature -is not present, test is skipped. - -Usage: ``feature STRING [STRING...]`` - -Possible checks: - -SO_RCVTIMEO_WORKS - The SO_RCVTIMEO socket option is working -64bit - The environment is 64 bits -!OSX - The environment is not OSX -dns - DNS lookups are working -topbuild - varnishtest has been started with '-i' -root - varnishtest has been invoked by the root user -user_varnish - The varnish user is present -user_vcache - The vcache user is present -group_varnish - The varnish group is present - -logexpect -********* - -This allows checking order and contents of VSL records in varnishtest. - SEE ALSO ======== * varnishtest source code repository with tests +* :ref:`vtc(7)` * :ref:`varnishhist(1)` * :ref:`varnishlog(1)` * :ref:`varnishncsa(1)` From guillaume at varnish-software.com Wed Jun 8 10:28:06 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Wed, 08 Jun 2016 12:28:06 +0200 Subject: [master] c0b7384 Explicit features Message-ID: commit c0b7384de238ab4fda1cdd708e659af898b484ff Author: Guillaume Quintard Date: Tue Jun 7 13:48:20 2016 +0200 Explicit features diff --git a/bin/varnishtest/vtc.c b/bin/varnishtest/vtc.c index 20ba48f..e7a40ef 100644 --- a/bin/varnishtest/vtc.c +++ b/bin/varnishtest/vtc.c @@ -586,15 +586,24 @@ cmd_random(CMD_ARGS) * Test that the required feature(s) for a test are available, and skip the test * otherwise. feature takes any number of arguments from this list: * - * - SO_RCVTIMEO_WORKS - * - 64bit - * - !OSX - * - dns - * - topbuild - * - root - * - user_varnish - * - user_vcache - * + * SO_RCVTIMEO_WORKS + * The SO_RCVTIMEO socket option is working + * 64bit + * The environment is 64 bits + * !OSX + * The environment is not OSX + * dns + * DNS lookups are working + * topbuild + * varnishtest has been started with '-i' + * root + * varnishtest has been invoked by the root user + * user_varnish + * The varnish user is present + * user_vcache + * The vcache user is present + * group_varnish + * The varnish group is present */ static void From fgsch at lodoss.net Wed Jun 8 10:57:06 2016 From: fgsch at lodoss.net (Federico G. Schwindt) Date: Wed, 08 Jun 2016 12:57:06 +0200 Subject: [master] ffc96d6 Correct parameter name Message-ID: commit ffc96d63330b3334ebe8ee71ec958bc4a425ac61 Author: Federico G. Schwindt Date: Wed Jun 8 11:56:05 2016 +0100 Correct parameter name diff --git a/include/tbl/vsc_f_main.h b/include/tbl/vsc_f_main.h index 4fdd691..0d98740 100644 --- a/include/tbl/vsc_f_main.h +++ b/include/tbl/vsc_f_main.h @@ -239,7 +239,7 @@ VSC_F(thread_queue_len, uint64_t, 0, 'g', 'i', info, "Length of session queue", "Length of session queue waiting for threads." " NB: Only updates once per second." - " See also parameter queue_max." + " See also parameter thread_queue_limit." ) VSC_F(busy_sleep, uint64_t, 1, 'c', 'i', info, @@ -263,13 +263,13 @@ VSC_F(busy_killed, uint64_t, 1, 'c', 'i', info, VSC_F(sess_queued, uint64_t, 0, 'c', 'i', info, "Sessions queued for thread", "Number of times session was queued waiting for a thread." - " See also parameter queue_max." + " See also parameter thread_queue_limit." ) VSC_F(sess_dropped, uint64_t, 0, 'c', 'i', info, "Sessions dropped for thread", "Number of times session was dropped because the queue were too" - " long already. See also parameter queue_max." + " long already. See also parameter thread_queue_limit." ) /*---------------------------------------------------------------------*/ From guillaume at varnish-software.com Wed Jun 8 11:29:05 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Wed, 08 Jun 2016 13:29:05 +0200 Subject: [master] 61da838 Proper ordering Message-ID: commit 61da838b6e131ffe3431caab3ee93f7eaa6b27dc Author: Guillaume Quintard Date: Wed Jun 8 13:28:28 2016 +0200 Proper ordering diff --git a/doc/sphinx/reference/varnishtest.rst b/doc/sphinx/reference/varnishtest.rst index 680b8ff..d3199d8 100644 --- a/doc/sphinx/reference/varnishtest.rst +++ b/doc/sphinx/reference/varnishtest.rst @@ -177,13 +177,13 @@ SEE ALSO ======== * varnishtest source code repository with tests -* :ref:`vtc(7)` * :ref:`varnishhist(1)` * :ref:`varnishlog(1)` * :ref:`varnishncsa(1)` * :ref:`varnishstat(1)` * :ref:`varnishtop(1)` * :ref:`vcl(7)` +* :ref:`vtc(7)` HISTORY ======= From varnish-commit at varnish-cache.org Wed Jun 8 09:38:04 2016 From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org) Date: 8 Jun 2016 03:38:04 -0600 Subject: limited release Message-ID: <6157F489F2C22B02FCD6ABD52881A856@varnish-cache.org> Hello! We are looking for employees working remotely. My name is Lamont Williams, I am the personnel manager of a large International company. Most of the work you can do from home, that is, at a distance. Salary is $2500-$5000. If you are interested in this offer, please visit Our Site Best regards! -------------- next part -------------- An HTML attachment was scrubbed... URL: From phk at FreeBSD.org Wed Jun 8 18:59:06 2016 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Wed, 08 Jun 2016 20:59:06 +0200 Subject: [master] 144840a Introduce typedef vcc_type_t Message-ID: commit 144840a65a9e78da52adcae3f5e5b913a39ae600 Author: Poul-Henning Kamp Date: Wed Jun 8 07:01:44 2016 +0000 Introduce typedef vcc_type_t diff --git a/lib/libvcc/vcc_action.c b/lib/libvcc/vcc_action.c index 467f5d2..67daa80 100644 --- a/lib/libvcc/vcc_action.c +++ b/lib/libvcc/vcc_action.c @@ -54,9 +54,9 @@ parse_call(struct vcc *tl) /*--------------------------------------------------------------------*/ static const struct arith { - enum var_type type; + vcc_type_t type; unsigned oper; - enum var_type want; + vcc_type_t want; } arith[] = { { INT, T_INCR, INT }, { INT, T_DECR, INT }, @@ -84,7 +84,7 @@ parse_set(struct vcc *tl) { const struct symbol *sym; const struct arith *ap; - enum var_type fmt; + vcc_type_t fmt; vcc_NextToken(tl); ExpectErr(tl, ID); diff --git a/lib/libvcc/vcc_compile.h b/lib/libvcc/vcc_compile.h index 9999230..ea626a8 100644 --- a/lib/libvcc/vcc_compile.h +++ b/lib/libvcc/vcc_compile.h @@ -74,6 +74,8 @@ enum var_type { #undef VCC_TYPE }; +typedef enum var_type vcc_type_t; + struct source { VTAILQ_ENTRY(source) list; char *name; @@ -100,7 +102,7 @@ enum symkind { }; typedef void sym_expr_t(struct vcc *tl, struct expr **, - const struct symbol *sym, enum var_type); + const struct symbol *sym, vcc_type_t); typedef void sym_wildcard_t(struct vcc *, struct symbol *, const char *, const char *); @@ -120,7 +122,7 @@ struct symbol { const struct token *def_b, *def_e; - enum var_type fmt; + vcc_type_t fmt; sym_expr_t *eval; void *eval_priv; @@ -207,7 +209,7 @@ struct vcc { struct var { const char *name; - enum var_type fmt; + vcc_type_t fmt; const char *rname; unsigned r_methods; const char *lname; @@ -267,7 +269,7 @@ char *TlDupTok(struct vcc *tl, const struct token *tok); double vcc_DoubleVal(struct vcc *tl); void vcc_Duration(struct vcc *tl, double *); unsigned vcc_UintVal(struct vcc *tl); -void vcc_Expr(struct vcc *tl, enum var_type typ); +void vcc_Expr(struct vcc *tl, vcc_type_t typ); void vcc_Expr_Call(struct vcc *tl, const struct symbol *sym); void vcc_Expr_Init(struct vcc *tl); sym_expr_t vcc_Eval_Var; @@ -275,10 +277,10 @@ sym_expr_t vcc_Eval_Handle; sym_expr_t vcc_Eval_SymFunc; void vcc_Eval_Func(struct vcc *tl, const char *cfunc, const char *extra, const char *name, const char *args, const char *vmod); -enum var_type VCC_arg_type(const char **p); -enum symkind VCC_HandleKind(enum var_type fmt); +vcc_type_t VCC_arg_type(const char **p); +enum symkind VCC_HandleKind(vcc_type_t fmt); struct symbol *VCC_HandleSymbol(struct vcc *, const struct token *, - enum var_type fmt, const char *str, ...); + vcc_type_t fmt, const char *str, ...); /* vcc_obj.c */ extern const struct var vcc_vars[]; @@ -304,7 +306,7 @@ struct symbol *VCC_Symbol(struct vcc *, struct symbol *, const char * VCC_SymKind(struct vcc *tl, const struct symbol *s); typedef void symwalk_f(struct vcc *tl, const struct symbol *s); void VCC_WalkSymbols(struct vcc *tl, symwalk_f *func, enum symkind kind); -void VCC_GlobalSymbol(struct symbol *, enum var_type, const char *str, ...); +void VCC_GlobalSymbol(struct symbol *, vcc_type_t, const char *str, ...); /* vcc_token.c */ void vcc_Coord(const struct vcc *tl, struct vsb *vsb, diff --git a/lib/libvcc/vcc_expr.c b/lib/libvcc/vcc_expr.c index 4f8cfe2..5e86895 100644 --- a/lib/libvcc/vcc_expr.c +++ b/lib/libvcc/vcc_expr.c @@ -40,7 +40,7 @@ #include "vcc_compile.h" static const char * -vcc_Type(enum var_type fmt) +vcc_Type(vcc_type_t fmt) { switch(fmt) { #define VCC_TYPE(a) case a: return(#a); @@ -214,7 +214,7 @@ vcc_ByteVal(struct vcc *tl, double *d) struct expr { unsigned magic; #define EXPR_MAGIC 0x38c794ab - enum var_type fmt; + vcc_type_t fmt; struct vsb *vsb; uint8_t constant; #define EXPR_VAR (1<<0) @@ -230,7 +230,7 @@ vcc_isconst(const struct expr *e) return (e->constant & EXPR_CONST); } -static void vcc_expr0(struct vcc *tl, struct expr **e, enum var_type fmt); +static void vcc_expr0(struct vcc *tl, struct expr **e, vcc_type_t fmt); static struct expr * vcc_new_expr(void) @@ -247,11 +247,11 @@ vcc_new_expr(void) } static struct expr * -vcc_mk_expr(enum var_type fmt, const char *str, ...) +vcc_mk_expr(vcc_type_t fmt, const char *str, ...) __v_printflike(2, 3); static struct expr * -vcc_mk_expr(enum var_type fmt, const char *str, ...) +vcc_mk_expr(vcc_type_t fmt, const char *str, ...) { va_list ap; struct expr *e; @@ -296,7 +296,7 @@ vcc_delete_expr(struct expr *e) */ static struct expr * -vcc_expr_edit(enum var_type fmt, const char *p, struct expr *e1, +vcc_expr_edit(vcc_type_t fmt, const char *p, struct expr *e1, struct expr *e2) { struct expr *e; @@ -381,7 +381,7 @@ vcc_expr_fmt(struct vsb *d, int ind, const struct expr *e1) /*-------------------------------------------------------------------- */ -enum var_type +vcc_type_t VCC_arg_type(const char **p) { @@ -395,7 +395,7 @@ VCC_arg_type(const char **p) */ static void -vcc_expr_tostring(struct vcc *tl, struct expr **e, enum var_type fmt) +vcc_expr_tostring(struct vcc *tl, struct expr **e, vcc_type_t fmt) { const char *p; uint8_t constant = EXPR_VAR; @@ -448,7 +448,7 @@ vcc_expr_tostring(struct vcc *tl, struct expr **e, enum var_type fmt) static void __match_proto__(sym_expr_t) vcc_Eval_Regsub(struct vcc *tl, struct expr **e, const struct symbol *sym, - enum var_type fmt) + vcc_type_t fmt) { struct expr *e2; int all = sym->eval_priv == NULL ? 0 : 1; @@ -485,7 +485,7 @@ vcc_Eval_Regsub(struct vcc *tl, struct expr **e, const struct symbol *sym, static void __match_proto__(sym_expr_t) vcc_Eval_BoolConst(struct vcc *tl, struct expr **e, const struct symbol *sym, - enum var_type fmt) + vcc_type_t fmt) { (void)fmt; @@ -499,7 +499,7 @@ vcc_Eval_BoolConst(struct vcc *tl, struct expr **e, const struct symbol *sym, void __match_proto__(sym_expr_t) vcc_Eval_Handle(struct vcc *tl, struct expr **e, const struct symbol *sym, - enum var_type fmt) + vcc_type_t fmt) { assert(sym->kind == sym->kind); @@ -522,7 +522,7 @@ vcc_Eval_Handle(struct vcc *tl, struct expr **e, const struct symbol *sym, void __match_proto__(sym_expr_t) vcc_Eval_Var(struct vcc *tl, struct expr **e, const struct symbol *sym, - enum var_type fmt) + vcc_type_t fmt) { (void)fmt; @@ -565,7 +565,7 @@ vcc_priv_arg(struct vcc *tl, const char *p, const char *name, const char *vmod) } struct func_arg { - enum var_type type; + vcc_type_t type; const char *enum_bits; const char *name; const char *val; @@ -629,7 +629,7 @@ vcc_func(struct vcc *tl, struct expr **e, const char *cfunc, const char *p; struct expr *e1; struct func_arg *fa, *fa2; - enum var_type rfmt; + vcc_type_t rfmt; VTAILQ_HEAD(,func_arg) head; struct token *t1; @@ -763,7 +763,7 @@ vcc_Eval_Func(struct vcc *tl, const char *cfunc, void __match_proto__(sym_expr_t) vcc_Eval_SymFunc(struct vcc *tl, struct expr **e, const struct symbol *sym, - enum var_type fmt) + vcc_type_t fmt) { (void)fmt; @@ -787,7 +787,7 @@ vcc_Eval_SymFunc(struct vcc *tl, struct expr **e, const struct symbol *sym, */ static void -vcc_expr4(struct vcc *tl, struct expr **e, enum var_type fmt) +vcc_expr4(struct vcc *tl, struct expr **e, vcc_type_t fmt) { struct expr *e1, *e2; const char *ip; @@ -927,10 +927,10 @@ vcc_expr4(struct vcc *tl, struct expr **e, enum var_type fmt) */ static void -vcc_expr_mul(struct vcc *tl, struct expr **e, enum var_type fmt) +vcc_expr_mul(struct vcc *tl, struct expr **e, vcc_type_t fmt) { struct expr *e2; - enum var_type f2, f3; + vcc_type_t f2, f3; struct token *tk; *e = NULL; @@ -979,7 +979,7 @@ vcc_expr_mul(struct vcc *tl, struct expr **e, enum var_type fmt) static void vcc_expr_string_add(struct vcc *tl, struct expr **e, struct expr *e2) { - enum var_type f2; + vcc_type_t f2; AN(e); AN(*e); @@ -1023,10 +1023,10 @@ vcc_expr_string_add(struct vcc *tl, struct expr **e, struct expr *e2) } static void -vcc_expr_add(struct vcc *tl, struct expr **e, enum var_type fmt) +vcc_expr_add(struct vcc *tl, struct expr **e, vcc_type_t fmt) { struct expr *e2; - enum var_type f2; + vcc_type_t f2; struct token *tk; *e = NULL; @@ -1085,7 +1085,7 @@ vcc_expr_add(struct vcc *tl, struct expr **e, enum var_type fmt) */ static void -vcc_expr_strfold(struct vcc *tl, struct expr **e, enum var_type fmt) +vcc_expr_strfold(struct vcc *tl, struct expr **e, vcc_type_t fmt) { vcc_expr_add(tl, e, fmt); @@ -1119,7 +1119,7 @@ vcc_expr_strfold(struct vcc *tl, struct expr **e, enum var_type fmt) {typ, '>', "(\v1 > \v2)" } static const struct cmps { - enum var_type fmt; + vcc_type_t fmt; unsigned token; const char *emit; } vcc_cmps[] = { @@ -1138,7 +1138,7 @@ static const struct cmps { #undef NUM_REL static void -vcc_expr_cmp(struct vcc *tl, struct expr **e, enum var_type fmt) +vcc_expr_cmp(struct vcc *tl, struct expr **e, vcc_type_t fmt) { struct expr *e2; const struct cmps *cp; @@ -1243,7 +1243,7 @@ vcc_expr_cmp(struct vcc *tl, struct expr **e, enum var_type fmt) */ static void -vcc_expr_not(struct vcc *tl, struct expr **e, enum var_type fmt) +vcc_expr_not(struct vcc *tl, struct expr **e, vcc_type_t fmt) { struct expr *e2; struct token *tk; @@ -1274,7 +1274,7 @@ vcc_expr_not(struct vcc *tl, struct expr **e, enum var_type fmt) */ static void -vcc_expr_cand(struct vcc *tl, struct expr **e, enum var_type fmt) +vcc_expr_cand(struct vcc *tl, struct expr **e, vcc_type_t fmt) { struct expr *e2; struct token *tk; @@ -1309,7 +1309,7 @@ vcc_expr_cand(struct vcc *tl, struct expr **e, enum var_type fmt) */ static void -vcc_expr0(struct vcc *tl, struct expr **e, enum var_type fmt) +vcc_expr0(struct vcc *tl, struct expr **e, vcc_type_t fmt) { struct expr *e2; struct token *tk; @@ -1349,7 +1349,7 @@ vcc_expr0(struct vcc *tl, struct expr **e, enum var_type fmt) */ void -vcc_Expr(struct vcc *tl, enum var_type fmt) +vcc_Expr(struct vcc *tl, vcc_type_t fmt) { struct expr *e; struct token *t1; diff --git a/lib/libvcc/vcc_storage.c b/lib/libvcc/vcc_storage.c index 878c727..9d17b5e 100644 --- a/lib/libvcc/vcc_storage.c +++ b/lib/libvcc/vcc_storage.c @@ -66,7 +66,7 @@ */ static struct var * -vcc_Stv_mkvar(struct vcc *tl, enum var_type fmt) +vcc_Stv_mkvar(struct vcc *tl, vcc_type_t fmt) { struct var *v; @@ -84,7 +84,7 @@ vcc_Stv_mkvar(struct vcc *tl, enum var_type fmt) static struct stvars { const char *name; - enum var_type fmt; + vcc_type_t fmt; } stvars[] = { #define VRTSTVVAR(nm, vtype, ctype, dval) { #nm, vtype }, #include "tbl/vrt_stv_var.h" diff --git a/lib/libvcc/vcc_symb.c b/lib/libvcc/vcc_symb.c index 4b9979d..ad223a7 100644 --- a/lib/libvcc/vcc_symb.c +++ b/lib/libvcc/vcc_symb.c @@ -39,7 +39,7 @@ /*--------------------------------------------------------------------*/ enum symkind -VCC_HandleKind(enum var_type fmt) +VCC_HandleKind(vcc_type_t fmt) { switch(fmt) { case ACL: return(SYM_ACL); @@ -182,7 +182,7 @@ VCC_WalkSymbols(struct vcc *tl, symwalk_f *func, enum symkind kind) static void vcc_global(struct vcc *tl, struct symbol *sym, - enum var_type fmt, const char *str, va_list ap) + vcc_type_t fmt, const char *str, va_list ap) { struct vsb *vsb; @@ -209,7 +209,7 @@ vcc_global(struct vcc *tl, struct symbol *sym, } void -VCC_GlobalSymbol(struct symbol *sym, enum var_type fmt, const char *str, ...) +VCC_GlobalSymbol(struct symbol *sym, vcc_type_t fmt, const char *str, ...) { va_list ap; @@ -219,7 +219,7 @@ VCC_GlobalSymbol(struct symbol *sym, enum var_type fmt, const char *str, ...) } struct symbol * -VCC_HandleSymbol(struct vcc *tl, const struct token *tk, enum var_type fmt, +VCC_HandleSymbol(struct vcc *tl, const struct token *tk, vcc_type_t fmt, const char *str, ...) { struct symbol *sym; From phk at FreeBSD.org Wed Jun 8 18:59:06 2016 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Wed, 08 Jun 2016 20:59:06 +0200 Subject: [master] 035cfa4 Turn VCL types into C-objects inside the VCC compiler. Message-ID: commit 035cfa4fe3b24fe01ae18eb894eca368d2dc6ba9 Author: Poul-Henning Kamp Date: Wed Jun 8 15:22:58 2016 +0000 Turn VCL types into C-objects inside the VCC compiler. (Confused ? You won't be after next weeks episode of ...) diff --git a/lib/libvcc/Makefile.am b/lib/libvcc/Makefile.am index d0c8ece..70fad80 100644 --- a/lib/libvcc/Makefile.am +++ b/lib/libvcc/Makefile.am @@ -30,6 +30,7 @@ libvcc_la_SOURCES = \ vcc_storage.c \ vcc_symb.c \ vcc_token.c \ + vcc_types.c \ vcc_utils.c \ vcc_var.c \ vcc_vmod.c \ diff --git a/lib/libvcc/vcc_compile.h b/lib/libvcc/vcc_compile.h index ea626a8..99dfcfb 100644 --- a/lib/libvcc/vcc_compile.h +++ b/lib/libvcc/vcc_compile.h @@ -68,14 +68,6 @@ struct expr; struct vcc; struct symbol; -enum var_type { -#define VCC_TYPE(foo) foo, -#include "tbl/vcc_types.h" -#undef VCC_TYPE -}; - -typedef enum var_type vcc_type_t; - struct source { VTAILQ_ENTRY(source) list; char *name; @@ -95,6 +87,22 @@ struct token { char *dec; }; +typedef const struct type *vcc_type_t; + +struct type { + unsigned magic; +#define TYPE_MAGIC 0xfae932d9 + + const char *name; + const char *tostring; + vcc_type_t multype; +}; + +#define VCC_TYPE(foo) extern const struct type foo[1]; +#include "tbl/vcc_types.h" +#undef VCC_TYPE + + enum symkind { #define VCC_SYMB(uu, ll) SYM_##uu, #include "tbl/symbol_kind.h" @@ -277,7 +285,6 @@ sym_expr_t vcc_Eval_Handle; sym_expr_t vcc_Eval_SymFunc; void vcc_Eval_Func(struct vcc *tl, const char *cfunc, const char *extra, const char *name, const char *args, const char *vmod); -vcc_type_t VCC_arg_type(const char **p); enum symkind VCC_HandleKind(vcc_type_t fmt); struct symbol *VCC_HandleSymbol(struct vcc *, const struct token *, vcc_type_t fmt, const char *str, ...); @@ -325,6 +332,9 @@ void vcc__ErrInternal(struct vcc *tl, const char *func, void vcc_AddToken(struct vcc *tl, unsigned tok, const char *b, const char *e); +/* vcc_types.c */ +vcc_type_t VCC_Type(const char *p); + /* vcc_var.c */ sym_wildcard_t vcc_Var_Wildcard; const struct symbol *vcc_FindVar(struct vcc *tl, const struct token *t, diff --git a/lib/libvcc/vcc_expr.c b/lib/libvcc/vcc_expr.c index 5e86895..1a73ce3 100644 --- a/lib/libvcc/vcc_expr.c +++ b/lib/libvcc/vcc_expr.c @@ -39,19 +39,6 @@ #include "vcc_compile.h" -static const char * -vcc_Type(vcc_type_t fmt) -{ - switch(fmt) { -#define VCC_TYPE(a) case a: return(#a); -#include "tbl/vcc_types.h" -#undef VCC_TYPE - default: - assert("Unknown Type"); - return(NULL); - } -} - /*-------------------------------------------------------------------- * Recognize and convert units of time, return seconds. */ @@ -381,19 +368,6 @@ vcc_expr_fmt(struct vsb *d, int ind, const struct expr *e1) /*-------------------------------------------------------------------- */ -vcc_type_t -VCC_arg_type(const char **p) -{ - -#define VCC_TYPE(a) if (!strcmp(#a, *p)) { *p += strlen(#a) + 1; return (a);} -#include "tbl/vcc_types.h" -#undef VCC_TYPE - return (VOID); -} - -/*-------------------------------------------------------------------- - */ - static void vcc_expr_tostring(struct vcc *tl, struct expr **e, vcc_type_t fmt) { @@ -403,41 +377,25 @@ vcc_expr_tostring(struct vcc *tl, struct expr **e, vcc_type_t fmt) CHECK_OBJ_NOTNULL(*e, EXPR_MAGIC); AN(fmt == STRING || fmt == STRING_LIST); - p = NULL; - switch((*e)->fmt) { - case BACKEND: p = "VRT_BACKEND_string(\v1)"; break; - case BOOL: p = "VRT_BOOL_string(\v1)"; break; - case DURATION: p = "VRT_REAL_string(ctx, \v1)"; break; - /* XXX: should DURATION insist on "s" suffix ? */ - case INT: + p = (*e)->fmt->tostring; + if (p == NULL && (*e)->fmt == INT) { if (vcc_isconst(*e)) { p = "\"\v1\""; constant = EXPR_CONST; } else { p = "VRT_INT_string(ctx, \v1)"; } - break; - case IP: p = "VRT_IP_string(ctx, \v1)"; break; - case BYTES: p = "VRT_REAL_string(ctx, \v1)"; break; /* XXX */ - case REAL: p = "VRT_REAL_string(ctx, \v1)"; break; - case TIME: p = "VRT_TIME_string(ctx, \v1)"; break; - case HEADER: p = "VRT_GetHdr(ctx, \v1)"; break; - case ENUM: - case STRING: - case STRING_LIST: - break; - case BLOB: - VSB_printf(tl->sb, - "Wrong use of BLOB value.\n" - "BLOBs can only be used as arguments to VMOD" - " functions.\n"); - vcc_ErrWhere2(tl, (*e)->t1, tl->t); - return; - default: - INCOMPL(); - break; + } else if (p == NULL && (*e)->fmt == BLOB) { + VSB_printf(tl->sb, + "Wrong use of BLOB value.\n" + "BLOBs can only be used as arguments to VMOD" + " functions.\n"); + vcc_ErrWhere2(tl, (*e)->t1, tl->t); + return; + } else { + AN(p); } - if (p != NULL) { + if (*p != '\0') { *e = vcc_expr_edit(fmt, p, *e, NULL); (*e)->constant = constant; } @@ -606,9 +564,9 @@ vcc_do_arg(struct vcc *tl, struct func_arg *fa) if (e2->fmt != fa->type) { VSB_printf(tl->sb, "Wrong argument type."); VSB_printf(tl->sb, " Expected %s.", - vcc_Type(fa->type)); + fa->type->name); VSB_printf(tl->sb, " Got %s.\n", - vcc_Type(e2->fmt)); + e2->fmt->name); vcc_ErrWhere2(tl, e2->t1, tl->t); return; } @@ -640,19 +598,23 @@ vcc_func(struct vcc *tl, struct expr **e, const char *cfunc, p = args; if (extra == NULL) extra = ""; - rfmt = VCC_arg_type(&p); + rfmt = VCC_Type(p); + AN(rfmt); + p += strlen(p) + 1; VTAILQ_INIT(&head); while (*p != '\0') { fa = calloc(sizeof *fa, 1); AN(fa); VTAILQ_INSERT_TAIL(&head, fa, list); - fa->type = VCC_arg_type(&p); - if (fa->type == VOID && !memcmp(p, "PRIV_", 5)) { + if (!memcmp(p, "PRIV_", 5)) { fa->result = vcc_priv_arg(tl, p, name, vmod); fa->name = ""; p += strlen(p) + 1; continue; } + fa->type = VCC_Type(p); + AN(fa->type); + p += strlen(p) + 1; if (*p == '\1') { fa->enum_bits = ++p; while (*p != '\1') @@ -825,7 +787,7 @@ vcc_expr4(struct vcc *tl, struct expr **e, vcc_type_t fmt) VSB_printf(tl->sb, "Symbol not found: "); vcc_ErrToken(tl, tl->t); VSB_printf(tl->sb, " (expected type %s):\n", - vcc_Type(fmt)); + fmt->name); vcc_ErrWhere(tl, tl->t); return; } @@ -914,7 +876,7 @@ vcc_expr4(struct vcc *tl, struct expr **e, vcc_type_t fmt) default: VSB_printf(tl->sb, "Unknown token "); vcc_ErrToken(tl, tl->t); - VSB_printf(tl->sb, " when looking for %s\n\n", vcc_Type(fmt)); + VSB_printf(tl->sb, " when looking for %s\n\n", fmt->name); vcc_ErrWhere(tl, tl->t); break; } @@ -937,18 +899,14 @@ vcc_expr_mul(struct vcc *tl, struct expr **e, vcc_type_t fmt) vcc_expr4(tl, e, fmt); ERRCHK(tl); AN(*e); - f3 = f2 = (*e)->fmt; + f3 = (*e)->fmt; - switch(f2) { - case INT: f2 = INT; break; - case DURATION: f2 = REAL; break; - case BYTES: f2 = REAL; break; - case REAL: f2 = REAL; break; - default: + f2 = f3->multype; + if (f2 == NULL) { if (tl->t->tok != '*' && tl->t->tok != '/') return; VSB_printf(tl->sb, "Operator %.*s not possible on type %s.\n", - PF(tl->t), vcc_Type(f2)); + PF(tl->t), f3->name); vcc_ErrWhere(tl, tl->t); return; } @@ -1067,7 +1025,7 @@ vcc_expr_add(struct vcc *tl, struct expr **e, vcc_type_t fmt) return; } else { VSB_printf(tl->sb, "%s %.*s %s not possible.\n", - vcc_Type((*e)->fmt), PF(tk), vcc_Type(e2->fmt)); + (*e)->fmt->name, PF(tk), e2->fmt->name); vcc_ErrWhere2(tl, tk, tl->t); return; } @@ -1166,9 +1124,9 @@ vcc_expr_cmp(struct vcc *tl, struct expr **e, vcc_type_t fmt) ERRCHK(tl); if (e2->fmt != (*e)->fmt) { /* XXX */ VSB_printf(tl->sb, "Comparison of different types: "); - VSB_printf(tl->sb, "%s ", vcc_Type((*e)->fmt)); + VSB_printf(tl->sb, "%s ", (*e)->fmt->name); vcc_ErrToken(tl, tk); - VSB_printf(tl->sb, " %s\n", vcc_Type(e2->fmt)); + VSB_printf(tl->sb, " %s\n", e2->fmt->name); vcc_ErrWhere(tl, tk); return; } @@ -1224,7 +1182,7 @@ vcc_expr_cmp(struct vcc *tl, struct expr **e, vcc_type_t fmt) case '~': case T_NOMATCH: VSB_printf(tl->sb, "Operator %.*s not possible on %s\n", - PF(tl->t), vcc_Type((*e)->fmt)); + PF(tl->t), (*e)->fmt->name); vcc_ErrWhere(tl, tl->t); return; default: @@ -1263,7 +1221,7 @@ vcc_expr_not(struct vcc *tl, struct expr **e, vcc_type_t fmt) return; } VSB_printf(tl->sb, "'!' must be followed by BOOL, found "); - VSB_printf(tl->sb, "%s.\n", vcc_Type(e2->fmt)); + VSB_printf(tl->sb, "%s.\n", e2->fmt->name); vcc_ErrWhere2(tl, tk, tl->t); } @@ -1293,7 +1251,7 @@ vcc_expr_cand(struct vcc *tl, struct expr **e, vcc_type_t fmt) if (e2->fmt != BOOL) { VSB_printf(tl->sb, "'&&' must be followed by BOOL, found "); - VSB_printf(tl->sb, "%s.\n", vcc_Type(e2->fmt)); + VSB_printf(tl->sb, "%s.\n", e2->fmt->name); vcc_ErrWhere2(tl, tk, tl->t); return; } @@ -1327,7 +1285,7 @@ vcc_expr0(struct vcc *tl, struct expr **e, vcc_type_t fmt) if (e2->fmt != BOOL) { VSB_printf(tl->sb, "'||' must be followed by BOOL, found "); - VSB_printf(tl->sb, "%s.\n", vcc_Type(e2->fmt)); + VSB_printf(tl->sb, "%s.\n", e2->fmt->name); vcc_ErrWhere2(tl, tk, tl->t); return; } @@ -1362,7 +1320,7 @@ vcc_Expr(struct vcc *tl, vcc_type_t fmt) e->t1 = t1; if (!tl->err && fmt != e->fmt) { VSB_printf(tl->sb, "Expression has type %s, expected %s\n", - vcc_Type(e->fmt), vcc_Type(fmt)); + e->fmt->name, fmt->name); tl->err = 1; } if (!tl->err) { diff --git a/lib/libvcc/vcc_symb.c b/lib/libvcc/vcc_symb.c index ad223a7..ee12862 100644 --- a/lib/libvcc/vcc_symb.c +++ b/lib/libvcc/vcc_symb.c @@ -41,14 +41,12 @@ enum symkind VCC_HandleKind(vcc_type_t fmt) { - switch(fmt) { - case ACL: return(SYM_ACL); - case BACKEND: return(SYM_BACKEND); - case PROBE: return(SYM_PROBE); - case STEVEDORE: return(SYM_STEVEDORE); - case INSTANCE: return(SYM_INSTANCE); - default: return(SYM_NONE); - } + if (fmt == ACL) return(SYM_ACL); + if (fmt == BACKEND) return(SYM_BACKEND); + if (fmt == PROBE) return(SYM_PROBE); + if (fmt == STEVEDORE) return(SYM_STEVEDORE); + if (fmt == INSTANCE) return(SYM_INSTANCE); + return(SYM_NONE); } const char * diff --git a/lib/libvcc/vcc_types.c b/lib/libvcc/vcc_types.c new file mode 100644 index 0000000..822471d --- /dev/null +++ b/lib/libvcc/vcc_types.c @@ -0,0 +1,160 @@ +/*- + * Copyright (c) 2006 Verdens Gang AS + * Copyright (c) 2006-2011 Varnish Software AS + * All rights reserved. + * + * Author: Poul-Henning Kamp + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + */ + +#include "config.h" + +#include +#include +#include +#include +#include + +#include "vcc_compile.h" + +const struct type ACL[1] = {{ + .magic = 0xfae932d9, + .name = "ACL", +}}; + +const struct type BACKEND[1] = {{ + .magic = 0xfae932d9, + .name = "BACKEND", + .tostring = "VRT_BACKEND_string(\v1)", +}}; + +const struct type BLOB[1] = {{ + .magic = 0xfae932d9, + .name = "BLOB", +}}; + +const struct type BOOL[1] = {{ + .magic = 0xfae932d9, + .name = "BOOL", + .tostring = "VRT_BOOL_string(\v1)", +}}; + +const struct type BYTES[1] = {{ + .magic = 0xfae932d9, + .name = "BYTES", + .tostring = "VRT_REAL_string(ctx, \v1)", // XXX: wrong + .multype = REAL, // XXX: wrong +}}; + +const struct type DURATION[1] = {{ + .magic = 0xfae932d9, + .name = "DURATION", + .tostring = "VRT_REAL_string(ctx, \v1)", // XXX 's' suff? + .multype = REAL, +}}; + +const struct type ENUM[1] = {{ + .magic = 0xfae932d9, + .name = "ENUM", + .tostring = "", +}}; + +const struct type HEADER[1] = {{ + .magic = 0xfae932d9, + .name = "HEADER", + .tostring = "VRT_GetHdr(ctx, \v1)", +}}; + +const struct type HTTP[1] = {{ + .magic = 0xfae932d9, + .name = "HTTP", +}}; + +const struct type INSTANCE[1] = {{ + .magic = 0xfae932d9, + .name = "INSTANCE", +}}; + +const struct type INT[1] = {{ + .magic = 0xfae932d9, + .name = "INT", + .multype = INT, +}}; + +const struct type IP[1] = {{ + .magic = 0xfae932d9, + .name = "IP", + .tostring = "VRT_IP_string(ctx, \v1)", +}}; + +const struct type PROBE[1] = {{ + .magic = 0xfae932d9, + .name = "PROBE", +}}; + +const struct type REAL[1] = {{ + .magic = 0xfae932d9, + .name = "REAL", + .tostring = "VRT_REAL_string(ctx, \v1)", + .multype = REAL, +}}; + +const struct type STEVEDORE[1] = {{ + .magic = 0xfae932d9, + .name = "STEVEDORE", +}}; + +const struct type STRING[1] = {{ + .magic = 0xfae932d9, + .name = "STRING", + .tostring = "", +}}; + +const struct type STRING_LIST[1] = {{ + .magic = 0xfae932d9, + .name = "STRING_LIST", + .tostring = "", +}}; + +const struct type TIME[1] = {{ + .magic = 0xfae932d9, + .name = "TIME", + .tostring = "VRT_TIME_string(ctx, \v1)", +}}; + +const struct type VOID[1] = {{ + .magic = 0xfae932d9, + .name = "VOID", +}}; + +vcc_type_t +VCC_Type(const char *p) +{ + +#define VCC_TYPE(foo) if (!strcmp(p, #foo)) return (foo); +#include "tbl/vcc_types.h" +#undef VCC_TYPE + return (NULL); +} + diff --git a/lib/libvcc/vcc_vmod.c b/lib/libvcc/vcc_vmod.c index 9f6f59e..c151337 100644 --- a/lib/libvcc/vcc_vmod.c +++ b/lib/libvcc/vcc_vmod.c @@ -248,7 +248,9 @@ vcc_ParseImport(struct vcc *tl) sym->cfunc = p; p += strlen(p) + 1; sym->args = p; - sym->fmt = VCC_arg_type(&p); + sym->fmt = VCC_Type(p); + AN(sym->fmt); + p += strlen(p) + 1; } else { VSB_printf(tl->sb, "Internal spec error (%s)\n", p); vcc_ErrWhere(tl, mod); From phk at FreeBSD.org Wed Jun 8 18:59:06 2016 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Wed, 08 Jun 2016 20:59:06 +0200 Subject: [master] ce467f2 Get rid of SYM_WILDCARD, check the ->wildcard field instead. Message-ID: commit ce467f20123a1e3a8a73373bd0bbfb4b93d14371 Author: Poul-Henning Kamp Date: Wed Jun 8 15:30:07 2016 +0000 Get rid of SYM_WILDCARD, check the ->wildcard field instead. diff --git a/include/tbl/symbol_kind.h b/include/tbl/symbol_kind.h index bd90016..1ec18f4 100644 --- a/include/tbl/symbol_kind.h +++ b/include/tbl/symbol_kind.h @@ -40,5 +40,4 @@ VCC_SYMB(STEVEDORE, stevedore) VCC_SYMB(SUB, sub) /* VCL subroutine */ VCC_SYMB(VAR, var) VCC_SYMB(VMOD, vmod) -VCC_SYMB(WILDCARD, wildcard) /*lint -restore */ diff --git a/lib/libvcc/vcc_compile.c b/lib/libvcc/vcc_compile.c index 6254794..35cf4f0 100644 --- a/lib/libvcc/vcc_compile.c +++ b/lib/libvcc/vcc_compile.c @@ -560,7 +560,7 @@ vcc_CompileSource(struct vcc *tl, struct source *sp) for (v = vcc_vars; v->name != NULL; v++) { if (v->fmt == HEADER) { sym = VCC_Symbol(tl, NULL, v->name, NULL, - SYM_WILDCARD, 1); + SYM_NONE, 1); sym->wildcard = vcc_Var_Wildcard; sym->wildcard_priv = v; } else { @@ -574,7 +574,7 @@ vcc_CompileSource(struct vcc *tl, struct source *sp) sym->lname = v->lname; } - sym = VCC_Symbol(tl, NULL, "storage", NULL, SYM_WILDCARD, 1); + sym = VCC_Symbol(tl, NULL, "storage", NULL, SYM_NONE, 1); sym->wildcard = vcc_Stv_Wildcard; Fh(tl, 0, "/* ---===### VCC generated .h code ###===---*/\n"); diff --git a/lib/libvcc/vcc_symb.c b/lib/libvcc/vcc_symb.c index ee12862..701555a 100644 --- a/lib/libvcc/vcc_symb.c +++ b/lib/libvcc/vcc_symb.c @@ -133,7 +133,7 @@ VCC_Symbol(struct vcc *tl, struct symbol *parent, continue; break; } - if (sym == NULL && create == 0 && parent->kind == SYM_WILDCARD) { + if (sym == NULL && create == 0 && parent->wildcard != NULL) { AN(parent->wildcard); parent->wildcard(tl, parent, b, e); if (tl->err) From phk at FreeBSD.org Wed Jun 8 18:59:06 2016 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Wed, 08 Jun 2016 20:59:06 +0200 Subject: [master] 03ca454 Moving parsing of "new" to vcc_vmod.c where it belongs. Message-ID: commit 03ca4548b8608c3cb5d0e50a66e6289343398ce8 Author: Poul-Henning Kamp Date: Wed Jun 8 16:18:13 2016 +0000 Moving parsing of "new" to vcc_vmod.c where it belongs. Minor simplifications. diff --git a/lib/libvcc/vcc_action.c b/lib/libvcc/vcc_action.c index 67daa80..37482f0 100644 --- a/lib/libvcc/vcc_action.c +++ b/lib/libvcc/vcc_action.c @@ -146,96 +146,6 @@ parse_unset(struct vcc *tl) /*--------------------------------------------------------------------*/ static void -parse_new(struct vcc *tl) -{ - struct symbol *sy1, *sy2, *sy3; - struct inifin *ifp; - const char *p, *s_obj, *s_init, *s_struct, *s_fini; - char buf1[128]; - char buf2[128]; - - vcc_NextToken(tl); - ExpectErr(tl, ID); - vcc_ExpectCid(tl, "VCL object"); - ERRCHK(tl); - sy1 = VCC_HandleSymbol(tl, tl->t, INSTANCE, "XXX"); - ERRCHK(tl); - - /* We allow implicit use of VMOD objects: Pretend it's ref'ed */ - sy1->nref++; - - vcc_NextToken(tl); - - ExpectErr(tl, '='); - vcc_NextToken(tl); - - ExpectErr(tl, ID); - sy2 = VCC_SymbolTok(tl, NULL, tl->t, SYM_OBJECT, 0); - if (sy2 == NULL) { - VSB_printf(tl->sb, "Symbol not found: "); - vcc_ErrToken(tl, tl->t); - VSB_printf(tl->sb, " at "); - vcc_ErrWhere(tl, tl->t); - return; - } - XXXAN(sy2); - - /*lint -save -e448 */ - /* Split the first three args */ - p = sy2->args; - - s_obj = p; - p += strlen(p) + 1; - - s_struct = p; - p += strlen(p) + 1; - - s_init = p; - while (p[0] != '\0' || p[1] != '\0' || p[2] != '\0') - p++; - p += 3; - - s_fini = p; - while (p[0] != '\0' || p[1] != '\0' || p[2] != '\0') - p++; - p += 3; - - Fh(tl, 0, "static %s *vo_%s;\n\n", s_struct, sy1->name); - - vcc_NextToken(tl); - - bprintf(buf1, ", &vo_%s, \"%s\"", sy1->name, sy1->name); - vcc_Eval_Func(tl, s_init, buf1, sy2->name, s_init + strlen(s_init) + 1, - sy2->vmod); - ifp = New_IniFin(tl); - VSB_printf(ifp->fin, "\t\t%s(&vo_%s);", s_fini, sy1->name); - ExpectErr(tl, ';'); - - bprintf(buf1, ", vo_%s", sy1->name); - /* Split the methods from the args */ - while (*p != '\0') { - p += strlen(s_obj); - bprintf(buf2, "%s%s", sy1->name, p); - sy3 = VCC_Symbol(tl, NULL, buf2, NULL, SYM_FUNC, 1); - AN(sy3); - sy3->eval = vcc_Eval_SymFunc; - p += strlen(p) + 1; - sy3->cfunc = p; - p += strlen(p) + 1; - - sy3->args = p; - sy3->extra = TlDup(tl, buf1); - while (p[0] != '\0' || p[1] != '\0' || p[2] != '\0') - p++; - p += 3; - } - sy1->def_e = tl->t; - /*lint -restore */ -} - -/*--------------------------------------------------------------------*/ - -static void parse_ban(struct vcc *tl) { @@ -375,7 +285,7 @@ static struct action_table { { "ban", parse_ban }, { "call", parse_call }, { "hash_data", parse_hash_data, VCL_MET_HASH }, - { "new", parse_new, VCL_MET_INIT}, + { "new", vcc_ParseNew, VCL_MET_INIT}, { "return", parse_return }, { "set", parse_set }, { "synthetic", parse_synthetic, diff --git a/lib/libvcc/vcc_compile.h b/lib/libvcc/vcc_compile.h index 99dfcfb..84ec643 100644 --- a/lib/libvcc/vcc_compile.h +++ b/lib/libvcc/vcc_compile.h @@ -133,7 +133,7 @@ struct symbol { vcc_type_t fmt; sym_expr_t *eval; - void *eval_priv; + const void *eval_priv; /* xref.c */ struct proc *proc; @@ -342,6 +342,7 @@ const struct symbol *vcc_FindVar(struct vcc *tl, const struct token *t, /* vcc_vmod.c */ void vcc_ParseImport(struct vcc *tl); +void vcc_ParseNew(struct vcc *tl); /* vcc_xref.c */ int vcc_AddDef(struct vcc *tl, const struct token *t, enum symkind type); diff --git a/lib/libvcc/vcc_expr.c b/lib/libvcc/vcc_expr.c index 1a73ce3..ff30a74 100644 --- a/lib/libvcc/vcc_expr.c +++ b/lib/libvcc/vcc_expr.c @@ -727,16 +727,23 @@ void __match_proto__(sym_expr_t) vcc_Eval_SymFunc(struct vcc *tl, struct expr **e, const struct symbol *sym, vcc_type_t fmt) { + const char *cfunc; + const char *args; + const char *p; (void)fmt; assert(sym->kind == SYM_FUNC); /* XXX */ - AN(sym->cfunc); + AN(sym->eval_priv); + AZ(sym->args); + AZ(sym->cfunc); + p = sym->eval_priv; + cfunc = p; + p += strlen(p) + 1; + args = p; AN(sym->name); - AN(sym->args); SkipToken(tl, ID); - vcc_func(tl, e, sym->cfunc, sym->extra, sym->name, sym->args, - sym->vmod); + vcc_func(tl, e, cfunc, sym->extra, sym->name, args, sym->vmod); } /*-------------------------------------------------------------------- @@ -778,10 +785,6 @@ vcc_expr4(struct vcc *tl, struct expr **e, vcc_type_t fmt) if (kind != SYM_NONE) sym = VCC_SymbolTok(tl, NULL, tl->t, kind, 0); if (sym == NULL) - sym = VCC_SymbolTok(tl, NULL, tl->t, SYM_VAR, 0); - if (sym == NULL) - sym = VCC_SymbolTok(tl, NULL, tl->t, SYM_FUNC, 0); - if (sym == NULL) sym = VCC_SymbolTok(tl, NULL, tl->t, SYM_NONE, 0); if (sym == NULL || sym->eval == NULL) { VSB_printf(tl->sb, "Symbol not found: "); diff --git a/lib/libvcc/vcc_vmod.c b/lib/libvcc/vcc_vmod.c index c151337..2e6f128 100644 --- a/lib/libvcc/vcc_vmod.c +++ b/lib/libvcc/vcc_vmod.c @@ -245,9 +245,8 @@ vcc_ParseImport(struct vcc *tl) sym->vmod = msym->name; sym->eval = vcc_Eval_SymFunc; p += strlen(p) + 1; - sym->cfunc = p; + sym->eval_priv = p; p += strlen(p) + 1; - sym->args = p; sym->fmt = VCC_Type(p); AN(sym->fmt); p += strlen(p) + 1; @@ -264,3 +263,90 @@ vcc_ParseImport(struct vcc *tl) Fh(tl, 0, "\n%s\n", vmd->proto); Fh(tl, 0, "\n/* --- END VMOD %.*s --- */\n\n", PF(mod)); } + +void +vcc_ParseNew(struct vcc *tl) +{ + struct symbol *sy1, *sy2, *sy3; + struct inifin *ifp; + const char *p, *s_obj, *s_init, *s_struct, *s_fini; + char buf1[128]; + char buf2[128]; + + vcc_NextToken(tl); + ExpectErr(tl, ID); + vcc_ExpectCid(tl, "VCL object"); + ERRCHK(tl); + sy1 = VCC_HandleSymbol(tl, tl->t, INSTANCE, "XXX"); + ERRCHK(tl); + + /* We allow implicit use of VMOD objects: Pretend it's ref'ed */ + sy1->nref++; + + vcc_NextToken(tl); + + ExpectErr(tl, '='); + vcc_NextToken(tl); + + ExpectErr(tl, ID); + sy2 = VCC_SymbolTok(tl, NULL, tl->t, SYM_OBJECT, 0); + if (sy2 == NULL) { + VSB_printf(tl->sb, "Symbol not found: "); + vcc_ErrToken(tl, tl->t); + VSB_printf(tl->sb, " at "); + vcc_ErrWhere(tl, tl->t); + return; + } + XXXAN(sy2); + + /*lint -save -e448 */ + /* Split the first three args */ + p = sy2->args; + + s_obj = p; + p += strlen(p) + 1; + + s_struct = p; + p += strlen(p) + 1; + + s_init = p; + while (p[0] != '\0' || p[1] != '\0' || p[2] != '\0') + p++; + p += 3; + + s_fini = p; + while (p[0] != '\0' || p[1] != '\0' || p[2] != '\0') + p++; + p += 3; + + Fh(tl, 0, "static %s *vo_%s;\n\n", s_struct, sy1->name); + + vcc_NextToken(tl); + + bprintf(buf1, ", &vo_%s, \"%s\"", sy1->name, sy1->name); + vcc_Eval_Func(tl, s_init, buf1, sy2->name, s_init + strlen(s_init) + 1, + sy2->vmod); + ifp = New_IniFin(tl); + VSB_printf(ifp->fin, "\t\t%s(&vo_%s);", s_fini, sy1->name); + ExpectErr(tl, ';'); + + bprintf(buf1, ", vo_%s", sy1->name); + /* Split the methods from the args */ + while (*p != '\0') { + p += strlen(s_obj); + bprintf(buf2, "%s%s", sy1->name, p); + sy3 = VCC_Symbol(tl, NULL, buf2, NULL, SYM_FUNC, 1); + AN(sy3); + sy3->eval = vcc_Eval_SymFunc; + p += strlen(p) + 1; + sy3->eval_priv = p; + p += strlen(p) + 1; + sy3->fmt = VCC_Type(p); + sy3->extra = TlDup(tl, buf1); + while (p[0] != '\0' || p[1] != '\0' || p[2] != '\0') + p++; + p += 3; + } + sy1->def_e = tl->t; + /*lint -restore */ +} From phk at FreeBSD.org Wed Jun 8 18:59:06 2016 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Wed, 08 Jun 2016 20:59:06 +0200 Subject: [master] 874841d Clean up and centralize specstring handling Message-ID: commit 874841d86d348795004cad9ff88b26b8b6694ce7 Author: Poul-Henning Kamp Date: Wed Jun 8 18:57:43 2016 +0000 Clean up and centralize specstring handling diff --git a/lib/libvcc/vcc_compile.h b/lib/libvcc/vcc_compile.h index 84ec643..b6c8f37 100644 --- a/lib/libvcc/vcc_compile.h +++ b/lib/libvcc/vcc_compile.h @@ -139,10 +139,7 @@ struct symbol { struct proc *proc; unsigned nref, ndef; - /* SYM_FUNC */ - const char *cfunc; const char *extra; - const char *args; /* SYM_VAR */ const char *rname; @@ -283,8 +280,8 @@ void vcc_Expr_Init(struct vcc *tl); sym_expr_t vcc_Eval_Var; sym_expr_t vcc_Eval_Handle; sym_expr_t vcc_Eval_SymFunc; -void vcc_Eval_Func(struct vcc *tl, const char *cfunc, const char *extra, - const char *name, const char *args, const char *vmod); +void vcc_Eval_Func(struct vcc *tl, const char *spec, + const char *extra, const struct symbol *sym); enum symkind VCC_HandleKind(vcc_type_t fmt); struct symbol *VCC_HandleSymbol(struct vcc *, const struct token *, vcc_type_t fmt, const char *str, ...); diff --git a/lib/libvcc/vcc_expr.c b/lib/libvcc/vcc_expr.c index ff30a74..f5b7391 100644 --- a/lib/libvcc/vcc_expr.c +++ b/lib/libvcc/vcc_expr.c @@ -581,33 +581,35 @@ vcc_do_arg(struct vcc *tl, struct func_arg *fa) } static void -vcc_func(struct vcc *tl, struct expr **e, const char *cfunc, - const char *extra, const char *name, const char *args, const char *vmod) +vcc_func(struct vcc *tl, struct expr **e, const char *spec, + const char *extra, const struct symbol *sym) { + vcc_type_t rfmt; + const char *args; + const char *cfunc; const char *p; struct expr *e1; struct func_arg *fa, *fa2; - vcc_type_t rfmt; VTAILQ_HEAD(,func_arg) head; struct token *t1; - AN(cfunc); - AN(args); - AN(name); + rfmt = VCC_Type(spec); + spec += strlen(spec) + 1; + cfunc = spec; + spec += strlen(spec) + 1; + args = spec; SkipToken(tl, '('); p = args; if (extra == NULL) extra = ""; - rfmt = VCC_Type(p); AN(rfmt); - p += strlen(p) + 1; VTAILQ_INIT(&head); while (*p != '\0') { fa = calloc(sizeof *fa, 1); AN(fa); VTAILQ_INSERT_TAIL(&head, fa, list); if (!memcmp(p, "PRIV_", 5)) { - fa->result = vcc_priv_arg(tl, p, name, vmod); + fa->result = vcc_priv_arg(tl, p, sym->name, sym->vmod); fa->name = ""; p += strlen(p) + 1; continue; @@ -703,14 +705,14 @@ vcc_func(struct vcc *tl, struct expr **e, const char *cfunc, */ void -vcc_Eval_Func(struct vcc *tl, const char *cfunc, - const char *extra, const char *name, const char *args, const char *vmod) +vcc_Eval_Func(struct vcc *tl, const char *spec, + const char *extra, const struct symbol *sym) { struct expr *e = NULL; struct token *t1; t1 = tl->t; - vcc_func(tl, &e, cfunc, extra, name, args, vmod); + vcc_func(tl, &e, spec, extra, sym); if (!tl->err) { vcc_expr_fmt(tl->fb, tl->indent, e); VSB_cat(tl->fb, ";\n"); @@ -727,23 +729,14 @@ void __match_proto__(sym_expr_t) vcc_Eval_SymFunc(struct vcc *tl, struct expr **e, const struct symbol *sym, vcc_type_t fmt) { - const char *cfunc; - const char *args; - const char *p; (void)fmt; assert(sym->kind == SYM_FUNC); - /* XXX */ AN(sym->eval_priv); - AZ(sym->args); - AZ(sym->cfunc); - p = sym->eval_priv; - cfunc = p; - p += strlen(p) + 1; - args = p; - AN(sym->name); + SkipToken(tl, ID); - vcc_func(tl, e, cfunc, sym->extra, sym->name, args, sym->vmod); + assert(sym->fmt == VCC_Type(sym->eval_priv)); + vcc_func(tl, e, sym->eval_priv, sym->extra, sym); } /*-------------------------------------------------------------------- diff --git a/lib/libvcc/vcc_vmod.c b/lib/libvcc/vcc_vmod.c index 2e6f128..cf7fa91 100644 --- a/lib/libvcc/vcc_vmod.c +++ b/lib/libvcc/vcc_vmod.c @@ -222,7 +222,7 @@ vcc_ParseImport(struct vcc *tl) p += strlen(p) + 1; sym = VCC_Symbol(tl, NULL, p, NULL, SYM_OBJECT, 1); XXXAN(sym); - sym->args = p; + sym->extra = p; sym->vmod = msym->name; } else if (!strcmp(p, "$EVENT")) { p += strlen(p) + 1; @@ -246,10 +246,8 @@ vcc_ParseImport(struct vcc *tl) sym->eval = vcc_Eval_SymFunc; p += strlen(p) + 1; sym->eval_priv = p; - p += strlen(p) + 1; sym->fmt = VCC_Type(p); AN(sym->fmt); - p += strlen(p) + 1; } else { VSB_printf(tl->sb, "Internal spec error (%s)\n", p); vcc_ErrWhere(tl, mod); @@ -269,7 +267,7 @@ vcc_ParseNew(struct vcc *tl) { struct symbol *sy1, *sy2, *sy3; struct inifin *ifp; - const char *p, *s_obj, *s_init, *s_struct, *s_fini; + const char *p, *s_obj; char buf1[128]; char buf2[128]; @@ -297,41 +295,34 @@ vcc_ParseNew(struct vcc *tl) vcc_ErrWhere(tl, tl->t); return; } - XXXAN(sy2); + vcc_NextToken(tl); - /*lint -save -e448 */ - /* Split the first three args */ - p = sy2->args; + p = sy2->extra; s_obj = p; p += strlen(p) + 1; - s_struct = p; + Fh(tl, 0, "static %s *vo_%s;\n\n", p, sy1->name); p += strlen(p) + 1; - s_init = p; + bprintf(buf1, ", &vo_%s, \"%s\"", sy1->name, sy1->name); + vcc_Eval_Func(tl, p, buf1, sy2); + ExpectErr(tl, ';'); + while (p[0] != '\0' || p[1] != '\0' || p[2] != '\0') p++; p += 3; - s_fini = p; + ifp = New_IniFin(tl); + p += strlen(p) + 1; + VSB_printf(ifp->fin, "\t\t%s(&vo_%s);", p, sy1->name); + while (p[0] != '\0' || p[1] != '\0' || p[2] != '\0') p++; p += 3; - Fh(tl, 0, "static %s *vo_%s;\n\n", s_struct, sy1->name); - - vcc_NextToken(tl); - - bprintf(buf1, ", &vo_%s, \"%s\"", sy1->name, sy1->name); - vcc_Eval_Func(tl, s_init, buf1, sy2->name, s_init + strlen(s_init) + 1, - sy2->vmod); - ifp = New_IniFin(tl); - VSB_printf(ifp->fin, "\t\t%s(&vo_%s);", s_fini, sy1->name); - ExpectErr(tl, ';'); - + /* Instantiate symbols for the methods */ bprintf(buf1, ", vo_%s", sy1->name); - /* Split the methods from the args */ while (*p != '\0') { p += strlen(s_obj); bprintf(buf2, "%s%s", sy1->name, p); @@ -340,7 +331,6 @@ vcc_ParseNew(struct vcc *tl) sy3->eval = vcc_Eval_SymFunc; p += strlen(p) + 1; sy3->eval_priv = p; - p += strlen(p) + 1; sy3->fmt = VCC_Type(p); sy3->extra = TlDup(tl, buf1); while (p[0] != '\0' || p[1] != '\0' || p[2] != '\0') @@ -348,5 +338,4 @@ vcc_ParseNew(struct vcc *tl) p += 3; } sy1->def_e = tl->t; - /*lint -restore */ } diff --git a/lib/libvcc/vmodtool.py b/lib/libvcc/vmodtool.py index 9525f2d..9228e82 100755 --- a/lib/libvcc/vmodtool.py +++ b/lib/libvcc/vmodtool.py @@ -324,17 +324,16 @@ class prototype(object): l.append(i.ct) return ", ".join(l) - def specstr(self, fo, p): - p = indent(p, 4) + def specstr(self, fo, cfunc, p): if self.retval == None: fo.write(p + '"VOID\\0"\n') else: self.retval.specstr(fo, p) + fo.write(p + '"' + cfunc + '\\0"\n') + p = indent(p, 4) if self.args != None: - p = indent(p, 4) for i in self.args: i.specstr(fo, p) - p = indent(p, -4) fo.write(p + '"\\0"\n') ####################################################################### @@ -472,7 +471,7 @@ class s_event(stanza): fo.write("\t%s,\n" % self.event_func) def specstr(self, fo): - fo.write('\t"$EVENT\\0"\n\t "Vmod_%s_Func._event",\n' % + fo.write('\t"$EVENT\\0"\n\t "Vmod_%s_Func._event",\n\n' % self.vcc.modname) class s_function(stanza): @@ -502,13 +501,11 @@ class s_function(stanza): fo.write("\tvmod_" + self.proto.cname() + ",\n") def specstr(self, fo): - fo.write('\t"$FUNC\\0"\n\t "%s.%s\\0"\n' % + fo.write('\t"$FUNC\\0"\t"%s.%s\\0"\n\n' % (self.vcc.modname, self.proto.name)) - fo.write('\t\t"Vmod_%s_Func.%s\\0"\n' % - (self.vcc.modname, self.proto.cname())) - self.proto.specstr(fo, "\t\t") - - fo.write('\t"\\0",\n\n') + self.proto.specstr(fo, 'Vmod_%s_Func.%s' % + (self.vcc.modname, self.proto.cname()), "\t ") + fo.write('\t "\\0",\n\n') class s_object(stanza): def parse(self): @@ -589,28 +586,27 @@ class s_object(stanza): def specstr(self, fo): - fo.write('\t"$OBJ\\0"\n\t "%s.%s\\0"\n' % + fo.write('\t"$OBJ\\0"\t"%s.%s\\0"\n\n' % (self.vcc.modname, self.proto.name)) - fo.write('\t\t"struct vmod_%s_%s\\0"\n' % + fo.write('\t "struct vmod_%s_%s\\0"\n' % (self.vcc.modname, self.proto.name)) fo.write("\n") - fo.write('\t\t"Vmod_%s_Func.%s__init\\0"\n' % - (self.vcc.modname, self.proto.name)) - self.proto.specstr(fo, '\t\t') - fo.write('\t\t"\\0"\n\n') + self.proto.specstr(fo, 'Vmod_%s_Func.%s__init' % + (self.vcc.modname, self.proto.name), '\t ') + fo.write('\t "\\0"\n\n') - fo.write('\t\t"Vmod_%s_Func.%s__fini\\0"\n' % + fo.write('\t "VOID\\0"\n') + fo.write('\t "Vmod_%s_Func.%s__fini\\0"\n' % (self.vcc.modname, self.proto.name)) - fo.write('\t\t "VOID\\0"\n') - fo.write('\t\t "\\0"\n') - fo.write('\t\t"\\0"\n\n') + fo.write('\t\t"\\0"\n') + fo.write('\t "\\0"\n\n') for i in self.methods: i.specstr(fo) - fo.write('\t"\\0",\n\n') + fo.write('\t "\\0",\n\n') def dump(self): super(s_object, self).dump() @@ -632,11 +628,10 @@ class s_method(stanza): fo.write('\t' + "vmod_" + self.proto.cname() + ",\n") def specstr(self, fo): - fo.write('\t\t"%s.%s\\0"\n' % + fo.write('\t "%s.%s\\0"\n' % (self.vcc.modname, self.proto.name)) - fo.write('\t\t "Vmod_%s_Func.%s\\0"\n' % - (self.vcc.modname, self.proto.cname())) - self.proto.specstr(fo, '\t\t ') + self.proto.specstr(fo, 'Vmod_%s_Func.%s' % + (self.vcc.modname, self.proto.cname()), '\t\t') fo.write('\t\t"\\0"\n\n') ####################################################################### From nils.goroll at uplex.de Wed Jun 8 20:20:10 2016 From: nils.goroll at uplex.de (Nils Goroll) Date: Wed, 08 Jun 2016 22:20:10 +0200 Subject: [master] c1e52e9 Finally some instructions for building on SmartOS (for a start) Message-ID: commit c1e52e95c9a82a6c6442ca8f3c4c26f50aae1e67 Author: Nils Goroll Date: Wed Jun 8 22:17:09 2016 +0200 Finally some instructions for building on SmartOS (for a start) diff --git a/doc/sphinx/installation/install.rst b/doc/sphinx/installation/install.rst index 8f99d80..f77ed32 100644 --- a/doc/sphinx/installation/install.rst +++ b/doc/sphinx/installation/install.rst @@ -110,6 +110,19 @@ packages installed: * `python-sphinx` * `graphviz` +Build dependencies on a SmartOS Zone +------------------------------------ + +As of SmartOS pkgsrc 2015Q4, install the following packages:: + + pkgin in autoconf automake libedit libtool ncurses \ + pcre graphviz py27-sphinx python27 gmake gcc49 \ + pkg-config + +Optionally, to pull from a repository:: + + pkgin in git + Compiling Varnish ----------------- From phk at FreeBSD.org Wed Jun 8 21:15:06 2016 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Wed, 08 Jun 2016 23:15:06 +0200 Subject: [master] 2d7f4ab Improve compiler typechecking and error reporting. Message-ID: commit 2d7f4abe57c0dd2bb0f15b05e0b362f7aac3ec76 Author: Poul-Henning Kamp Date: Wed Jun 8 21:13:29 2016 +0000 Improve compiler typechecking and error reporting. diff --git a/bin/varnishtest/tests/v00020.vtc b/bin/varnishtest/tests/v00020.vtc index d8e4622..e041356 100644 --- a/bin/varnishtest/tests/v00020.vtc +++ b/bin/varnishtest/tests/v00020.vtc @@ -304,3 +304,11 @@ varnish v1 -errvcl {Name of ACL, 'foo-bar', contains illegal character '-'} { varnish v1 -errvcl {Expected 'from path ...'} { import std to; } + +varnish v1 -errvcl {INT * BLOB not possible.} { + import debug; + backend b { .host = "127.0.0.1"; } + sub vcl_deliver { + set resp.status = 100 * debug.str2blob("a"); + } +} diff --git a/lib/libvcc/vcc_expr.c b/lib/libvcc/vcc_expr.c index f5b7391..3a6620d 100644 --- a/lib/libvcc/vcc_expr.c +++ b/lib/libvcc/vcc_expr.c @@ -378,23 +378,15 @@ vcc_expr_tostring(struct vcc *tl, struct expr **e, vcc_type_t fmt) AN(fmt == STRING || fmt == STRING_LIST); p = (*e)->fmt->tostring; - if (p == NULL && (*e)->fmt == INT) { - if (vcc_isconst(*e)) { - p = "\"\v1\""; - constant = EXPR_CONST; - } else { - p = "VRT_INT_string(ctx, \v1)"; - } - } else if (p == NULL && (*e)->fmt == BLOB) { + if (p == NULL && (*e)->fmt == BLOB) { VSB_printf(tl->sb, "Wrong use of BLOB value.\n" "BLOBs can only be used as arguments to VMOD" " functions.\n"); vcc_ErrWhere2(tl, (*e)->t1, tl->t); return; - } else { - AN(p); } + AN(p); if (*p != '\0') { *e = vcc_expr_edit(fmt, p, *e, NULL); (*e)->constant = constant; @@ -912,6 +904,13 @@ vcc_expr_mul(struct vcc *tl, struct expr **e, vcc_type_t fmt) vcc_NextToken(tl); vcc_expr4(tl, &e2, f2); ERRCHK(tl); + if (e2->fmt->multype == NULL) { + VSB_printf(tl->sb, + "%s %.*s %s not possible.\n", + f2->name, PF(tk), e2->fmt->name); + vcc_ErrWhere(tl, tk); + return; + } assert(e2->fmt == f2); if (tk->tok == '*') *e = vcc_expr_edit(f3, "(\v1*\v2)", *e, e2); @@ -947,10 +946,8 @@ vcc_expr_string_add(struct vcc *tl, struct expr **e, struct expr *e2) vcc_expr_mul(tl, &e2, STRING); } ERRCHK(tl); - if (e2->fmt != STRING && e2->fmt != STRING_LIST) { + if (e2->fmt != STRING && e2->fmt != STRING_LIST) vcc_expr_tostring(tl, &e2, f2); - ERRCHK(tl); - } ERRCHK(tl); assert(e2->fmt == STRING || e2->fmt == STRING_LIST); @@ -996,41 +993,46 @@ vcc_expr_add(struct vcc *tl, struct expr **e, vcc_type_t fmt) else vcc_expr_mul(tl, &e2, f2); ERRCHK(tl); - if (tk->tok == '-' && (*e)->fmt == TIME && e2->fmt == TIME) { - /* OK */ - } else if ((*e)->fmt == TIME && e2->fmt == DURATION) { - f2 = TIME; - /* OK */ - } else if ((*e)->fmt == BYTES && e2->fmt == BYTES) { - /* OK */ - } else if ((*e)->fmt == INT && e2->fmt == INT) { - /* OK */ - } else if ((*e)->fmt == REAL && e2->fmt == REAL) { - /* OK */ - } else if ((*e)->fmt == DURATION && e2->fmt == DURATION) { - /* OK */ - } else if (tk->tok == '+' && - (*e)->fmt == STRING && e2->fmt == STRING) { - vcc_expr_string_add(tl, e, e2); - return; - } else if (tk->tok == '+' && - (fmt == STRING || fmt == STRING_LIST)) { - /* Time to fold and add as string */ - vcc_expr_tostring(tl, e, STRING); - vcc_expr_string_add(tl, e, e2); - return; - } else { - VSB_printf(tl->sb, "%s %.*s %s not possible.\n", - (*e)->fmt->name, PF(tk), e2->fmt->name); - vcc_ErrWhere2(tl, tk, tl->t); - return; + +#define ADD_OK(op, a, b, c) \ + if (tk->tok == op[0] && (*e)->fmt == a && e2->fmt == b) { \ + *e = vcc_expr_edit(c, "(\v1" op "\v2)", *e, e2); \ + continue; \ + } \ + + ADD_OK("-", TIME, TIME, DURATION); + ADD_OK("+", TIME, DURATION, TIME); + ADD_OK("-", TIME, DURATION, TIME); + ADD_OK("+", DURATION, DURATION, DURATION); + ADD_OK("-", DURATION, DURATION, DURATION); + ADD_OK("+", BYTES, BYTES, BYTES); + ADD_OK("-", BYTES, BYTES, BYTES); + ADD_OK("+", INT, INT, INT); + ADD_OK("-", INT, INT, INT); + ADD_OK("+", REAL, REAL, REAL); + ADD_OK("-", REAL, REAL, REAL); + +#undef ADD_OK + + if (tk->tok == '+') { + if ((*e)->fmt == STRING && e2->fmt == STRING) { + vcc_expr_string_add(tl, e, e2); + return; + } + + if (fmt == STRING || fmt == STRING_LIST) { + /* Time to fold and add as string */ + if ((*e)->fmt != fmt) + vcc_expr_tostring(tl, e, fmt); + vcc_expr_string_add(tl, e, e2); + return; + } } - if (tk->tok == '+') - *e = vcc_expr_edit(f2, "(\v1+\v2)", *e, e2); - else if (f2 == TIME && e2->fmt == TIME) - *e = vcc_expr_edit(DURATION, "(\v1-\v2)", *e, e2); - else - *e = vcc_expr_edit(f2, "(\v1-\v2)", *e, e2); + + VSB_printf(tl->sb, "%s %.*s %s not possible.\n", + (*e)->fmt->name, PF(tk), e2->fmt->name); + vcc_ErrWhere2(tl, tk, tl->t); + return; } } @@ -1246,8 +1248,8 @@ vcc_expr_cand(struct vcc *tl, struct expr **e, vcc_type_t fmt) ERRCHK(tl); if (e2->fmt != BOOL) { VSB_printf(tl->sb, - "'&&' must be followed by BOOL, found "); - VSB_printf(tl->sb, "%s.\n", e2->fmt->name); + "'&&' must be followed by BOOL," + " found %s.\n", e2->fmt->name); vcc_ErrWhere2(tl, tk, tl->t); return; } @@ -1280,8 +1282,8 @@ vcc_expr0(struct vcc *tl, struct expr **e, vcc_type_t fmt) ERRCHK(tl); if (e2->fmt != BOOL) { VSB_printf(tl->sb, - "'||' must be followed by BOOL, found "); - VSB_printf(tl->sb, "%s.\n", e2->fmt->name); + "'||' must be followed by BOOL," + " found %s.\n", e2->fmt->name); vcc_ErrWhere2(tl, tk, tl->t); return; } diff --git a/lib/libvcc/vcc_types.c b/lib/libvcc/vcc_types.c index 822471d..65b7553 100644 --- a/lib/libvcc/vcc_types.c +++ b/lib/libvcc/vcc_types.c @@ -100,6 +100,7 @@ const struct type INT[1] = {{ .magic = 0xfae932d9, .name = "INT", .multype = INT, + .tostring = "VRT_INT_string(ctx, \v1)", }}; const struct type IP[1] = {{ From varnish-commit at varnish-cache.org Thu Jun 9 13:04:59 2016 From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org) Date: 9 Jun 2016 05:04:59 -0800 Subject: finally it has been released and it’s free Message-ID: <004301d1c252$03f8f60d$03195295$@varnish-cache.org> Hello! We are looking for employees working remotely. My name is Lawrence Lara, I am the personnel manager of a large International company. Most of the work you can do from home, that is, at a distance. Salary is $2500-$5000. If you are interested in this offer, please visit Our Site Best regards! -------------- next part -------------- An HTML attachment was scrubbed... URL: From varnish-commit at varnish-cache.org Thu Jun 9 12:31:14 2016 From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org) Date: 9 Jun 2016 18:31:14 +0600 Subject: Your membership is now activated Message-ID: <5D603159710C081924751D35484C5D60@varnish-cache.org> Hey, I Noticed you didn't got "Google Trader System" You are really missing the opportunity of Making 6 Figure Per Month You no longer need to work 9 to 5 After Getting it.. (I promise) ==>> Get it Now -------------- next part -------------- An HTML attachment was scrubbed... URL: From phk at FreeBSD.org Thu Jun 9 08:45:10 2016 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Thu, 09 Jun 2016 10:45:10 +0200 Subject: [master] f931117 Implement the storage.${stevedorename} VCL variables in a (not-quite-as-)hacky way Message-ID: commit f93111722862bcb55901420ef51a126e9a4e6a4c Author: Poul-Henning Kamp Date: Thu Jun 9 08:43:39 2016 +0000 Implement the storage.${stevedorename} VCL variables in a (not-quite-as-)hacky way diff --git a/bin/varnishd/cache/cache_vrt_priv.c b/bin/varnishd/cache/cache_vrt_priv.c index fd8215b..1cfa281 100644 --- a/bin/varnishd/cache/cache_vrt_priv.c +++ b/bin/varnishd/cache/cache_vrt_priv.c @@ -136,7 +136,7 @@ VRT_priv_top(VRT_CTX, void *vmod_id) return (VRT_priv_dynamic(ctx, id, (uintptr_t)vmod_id)); } else WRONG("PRIV_TOP is only accessible in client VCL context"); - return (NULL); + NEEDLESS_RETURN(NULL); } /*-------------------------------------------------------------------- diff --git a/bin/varnishd/storage/stevedore.c b/bin/varnishd/storage/stevedore.c index da7ece4..f977cdd 100644 --- a/bin/varnishd/storage/stevedore.c +++ b/bin/varnishd/storage/stevedore.c @@ -126,13 +126,20 @@ void STV_open(void) { struct stevedore *stv; + char buf[1024]; ASSERT_CLI(); VTAILQ_FOREACH(stv, &stv_stevedores, list) { + bprintf(buf, "storage.%s", stv->ident); + stv->vclname = strdup(buf); + AN(stv->vclname); if (stv->open != NULL) stv->open(stv); } stv = stv_transient; + bprintf(buf, "storage.%s", stv->ident); + stv->vclname = strdup(buf); + AN(stv->vclname); if (stv->open != NULL) stv->open(stv); stv_next = VTAILQ_FIRST(&stv_stevedores); @@ -231,15 +238,19 @@ VRT_Stv(const char *nm) return (0); } -struct stevedore * -VRT_stevedore(const char *nm) +const char * __match_proto__() +VRT_STEVEDORE_string(VCL_STEVEDORE s) { - struct stevedore *stv; + if (s == NULL) + return (NULL); + CHECK_OBJ_NOTNULL(s, STEVEDORE_MAGIC); + return (s->vclname); +} - VTAILQ_FOREACH(stv, &stv_stevedores, list) - if (!strcmp(stv->ident, nm)) - return (stv); - WRONG("Unknown stevedore name"); +VCL_STEVEDORE +VRT_stevedore(const char *nm) +{ + return (stv_find(nm)); } #define VRTSTVVAR(nm, vtype, ctype, dval) \ diff --git a/bin/varnishd/storage/storage.h b/bin/varnishd/storage/storage.h index 113b4da..0f85255 100644 --- a/bin/varnishd/storage/storage.h +++ b/bin/varnishd/storage/storage.h @@ -122,6 +122,7 @@ struct stevedore { VTAILQ_ENTRY(stevedore) list; char ident[16]; /* XXX: match VSM_chunk.ident */ + char *vclname; }; VTAILQ_HEAD(stevedore_head, stevedore); diff --git a/bin/varnishtest/tests/v00032.vtc b/bin/varnishtest/tests/v00032.vtc index f9adb77..47f1af8 100644 --- a/bin/varnishtest/tests/v00032.vtc +++ b/bin/varnishtest/tests/v00032.vtc @@ -8,7 +8,6 @@ server s1 { varnish v1 -vcl+backend { sub vcl_backend_response { set beresp.http.has_s0 = storage.s0; - set beresp.http.has_foo = storage.foo; set beresp.http.has_Transient = storage.Transient; } } -start @@ -18,7 +17,6 @@ varnish v1 -cliok "storage.list" client c1 { txreq rxresp - expect resp.http.has_s0 == true - expect resp.http.has_foo == false - expect resp.http.has_Transient == true + expect resp.http.has_s0 == storage.s0 + expect resp.http.has_Transient == storage.Transient } -run diff --git a/bin/varnishtest/tests/v00033.vtc b/bin/varnishtest/tests/v00033.vtc index a10fac1..ca51ece 100644 --- a/bin/varnishtest/tests/v00033.vtc +++ b/bin/varnishtest/tests/v00033.vtc @@ -12,7 +12,7 @@ varnish v1 -vcl+backend { sub vcl_backend_response { set beresp.http.foo = - storage.nowhere.free_space + + storage.Transient.used_space + 1 B + 1 KB + 1 MB + 1GB + 1TB; if (bereq.url == "/foo") { set beresp.storage_hint = "Transient"; @@ -39,13 +39,13 @@ client c1 { varnish v1 -errvcl {Expected BYTES unit (B, KB, MB...) got '"X"'} { sub vcl_recv { - if (storage.nowhere.free_space > 4 "X") { + if (storage.Transient.free_space > 4 "X") { } } } varnish v1 -errvcl {Unknown BYTES unit 'X'. Legal are 'B', 'KB', 'MB', 'GB' and 'TB'} { sub vcl_recv { - if (storage.nowhere.free_space > 4 X) { + if (storage.Transient.free_space > 4 X) { } } } diff --git a/include/vrt.h b/include/vrt.h index 4ef7dec..fa90786 100644 --- a/include/vrt.h +++ b/include/vrt.h @@ -94,7 +94,7 @@ typedef long VCL_INT; typedef const struct suckaddr * VCL_IP; typedef const struct vrt_backend_probe * VCL_PROBE; typedef double VCL_REAL; -typedef struct stevedore * VCL_STEVEDORE; +typedef const struct stevedore * VCL_STEVEDORE; typedef const char * VCL_STRING; typedef double VCL_TIME; typedef void VCL_VOID; @@ -334,7 +334,7 @@ struct vmod_priv *VRT_priv_top(VRT_CTX, void *vmod_id); /* Stevedore related functions */ int VRT_Stv(const char *nm); -struct stevedore *VRT_stevedore(const char *nm); +VCL_STEVEDORE VRT_stevedore(const char *nm); /* Convert things to string */ @@ -344,4 +344,5 @@ char *VRT_REAL_string(VRT_CTX, VCL_REAL); char *VRT_TIME_string(VRT_CTX, VCL_TIME); const char *VRT_BOOL_string(VCL_BOOL); const char *VRT_BACKEND_string(VCL_BACKEND); +const char *VRT_STEVEDORE_string(VCL_STEVEDORE); const char *VRT_CollectString(VRT_CTX, const char *p, ...); diff --git a/lib/libvcc/vcc_compile.c b/lib/libvcc/vcc_compile.c index 35cf4f0..a687cdc 100644 --- a/lib/libvcc/vcc_compile.c +++ b/lib/libvcc/vcc_compile.c @@ -444,7 +444,7 @@ vcc_new_source(const char *b, const char *e, const char *name) /*--------------------------------------------------------------------*/ static struct source * -vcc_file_source(struct vcc *tl, const char *fn) +vcc_file_source(const struct vcc *tl, const char *fn) { char *f, *fnp; struct source *sp; @@ -574,9 +574,6 @@ vcc_CompileSource(struct vcc *tl, struct source *sp) sym->lname = v->lname; } - sym = VCC_Symbol(tl, NULL, "storage", NULL, SYM_NONE, 1); - sym->wildcard = vcc_Stv_Wildcard; - Fh(tl, 0, "/* ---===### VCC generated .h code ###===---*/\n"); Fc(tl, 0, "\n/* ---===### VCC generated .c code ###===---*/\n"); Fh(tl, 0, "\nextern const struct VCL_conf VCL_conf;\n"); @@ -818,25 +815,3 @@ VCC_Unsafe_Path(struct vcc *vcc, unsigned u) CHECK_OBJ_NOTNULL(vcc, VCC_MAGIC); vcc->unsafe_path = u; } - -void -VCC_Stevedore(struct vcc *vcc, const char *stv_name) -{ -#if 0 - struct symbol *sym; - char stv[1024]; - - CHECK_OBJ_NOTNULL(vcc, VCC_MAGIC); - ALLOC_OBJ(sym, SYMBOL_MAGIC); - AN(sym); - bprintf(stv, "stv.%s", stv_name); - REPLACE(sym->name, stv); /* XXX storage.* ? */ - sym->kind = SYM_STEVEDORE; - VCC_GlobalSymbol(sym, STEVEDORE, "VRT_stevedore(\"%s\")", stv_name); - VTAILQ_INSERT_TAIL(&vcc->symbols, sym, list); -#else - (void)vcc; - (void)stv_name; -#endif -} - diff --git a/lib/libvcc/vcc_compile.h b/lib/libvcc/vcc_compile.h index b6c8f37..8a2b20e 100644 --- a/lib/libvcc/vcc_compile.h +++ b/lib/libvcc/vcc_compile.h @@ -292,9 +292,6 @@ extern const struct var vcc_vars[]; /* vcc_parse.c */ void vcc_Parse(struct vcc *tl); -/* vcc_storage.c */ -sym_wildcard_t vcc_Stv_Wildcard; - /* vcc_utils.c */ const char *vcc_regexp(struct vcc *tl); void Resolve_Sockaddr(struct vcc *tl, const char *host, const char *defport, @@ -310,7 +307,6 @@ struct symbol *VCC_Symbol(struct vcc *, struct symbol *, const char * VCC_SymKind(struct vcc *tl, const struct symbol *s); typedef void symwalk_f(struct vcc *tl, const struct symbol *s); void VCC_WalkSymbols(struct vcc *tl, symwalk_f *func, enum symkind kind); -void VCC_GlobalSymbol(struct symbol *, vcc_type_t, const char *str, ...); /* vcc_token.c */ void vcc_Coord(const struct vcc *tl, struct vsb *vsb, diff --git a/lib/libvcc/vcc_expr.c b/lib/libvcc/vcc_expr.c index 3a6620d..3edda25 100644 --- a/lib/libvcc/vcc_expr.c +++ b/lib/libvcc/vcc_expr.c @@ -386,6 +386,12 @@ vcc_expr_tostring(struct vcc *tl, struct expr **e, vcc_type_t fmt) vcc_ErrWhere2(tl, (*e)->t1, tl->t); return; } + if (p == NULL) { + VSB_printf(tl->sb, + "Cannot convert %s to STRING.\n", (*e)->fmt->name); + vcc_ErrWhere2(tl, (*e)->t1, tl->t); + return; + } AN(p); if (*p != '\0') { *e = vcc_expr_edit(fmt, p, *e, NULL); diff --git a/lib/libvcc/vcc_storage.c b/lib/libvcc/vcc_storage.c index 9d17b5e..6187db4 100644 --- a/lib/libvcc/vcc_storage.c +++ b/lib/libvcc/vcc_storage.c @@ -60,28 +60,12 @@ #include #include "vcc_compile.h" +#include "libvcc.h" // VCC_Stevedore() proto /*-------------------------------------------------------------------- * */ -static struct var * -vcc_Stv_mkvar(struct vcc *tl, vcc_type_t fmt) -{ - struct var *v; - - v = TlAlloc(tl, sizeof *v); - AN(v); - - v->r_methods = 0; -#define VCL_MET_MAC(l,u,t,b) v->r_methods |= VCL_MET_##u; -#include "tbl/vcl_returns.h" -#undef VCL_MET_MAC - v->fmt = fmt; - - return (v); -} - static struct stvars { const char *name; vcc_type_t fmt; @@ -92,49 +76,31 @@ static struct stvars { { NULL, BOOL } }; -void __match_proto__(sym_wildcard_t) -vcc_Stv_Wildcard(struct vcc *tl, struct symbol *parent, - const char *b, const char *e) +void +VCC_Stevedore(struct vcc *vcc, const char *stv_name) { - const char *q; - struct var *v = NULL; struct symbol *sym; struct stvars *sv; - char stv[1024]; char buf[1024]; - for (q = b; q < e && *q != '.'; q++) - continue; - bprintf(stv, "%.*s", (int)(q - b), b); - - if (q == e) { - v = vcc_Stv_mkvar(tl, BOOL); - bprintf(buf, "VRT_Stv(\"%s\")", stv); - v->rname = TlDup(tl, buf); - } else { - assert(*q == '.'); - q++; - for(sv = stvars; sv->name != NULL; sv++) { - if (strncmp(q, sv->name, e - q)) - continue; - if (sv->name[e - q] != '\0') - continue; - v = vcc_Stv_mkvar(tl, sv->fmt); - bprintf(buf, "VRT_Stv_%s(\"%s\")", sv->name, stv); - v->rname = TlDup(tl, buf); - break; - } - } - - if (v == NULL) - return; - - sym = VCC_Symbol(tl, parent, b, e, SYM_VAR, 1); + CHECK_OBJ_NOTNULL(vcc, VCC_MAGIC); + bprintf(buf, "storage.%s", stv_name); + sym = VCC_Symbol(vcc, NULL, buf, NULL, SYM_VAR, 1); AN(sym); - sym->fmt = v->fmt; + sym->fmt = STEVEDORE; sym->eval = vcc_Eval_Var; - sym->r_methods = v->r_methods; - sym->rname = v->rname; - sym->w_methods = v->w_methods; - sym->lname = v->lname; + bprintf(buf, "VRT_stevedore(\"%s\")", stv_name); + sym->rname = TlDup(vcc, buf); + sym->r_methods = ~0; + + for(sv = stvars; sv->name != NULL; sv++) { + bprintf(buf, "storage.%s.%s", stv_name, sv->name); + sym = VCC_Symbol(vcc, NULL, buf, NULL, SYM_VAR, 1); + AN(sym); + sym->fmt = sv->fmt; + sym->eval = vcc_Eval_Var; + bprintf(buf, "VRT_Stv_%s(\"%s\")", sv->name, stv_name); + sym->rname = TlDup(vcc, buf); + sym->r_methods = ~0; + } } diff --git a/lib/libvcc/vcc_symb.c b/lib/libvcc/vcc_symb.c index 701555a..c1aa72f 100644 --- a/lib/libvcc/vcc_symb.c +++ b/lib/libvcc/vcc_symb.c @@ -206,16 +206,6 @@ vcc_global(struct vcc *tl, struct symbol *sym, #undef VCL_MET_MAC } -void -VCC_GlobalSymbol(struct symbol *sym, vcc_type_t fmt, const char *str, ...) -{ - va_list ap; - - va_start(ap, str); - vcc_global(NULL, sym, fmt, str, ap); - va_end(ap); -} - struct symbol * VCC_HandleSymbol(struct vcc *tl, const struct token *tk, vcc_type_t fmt, const char *str, ...) diff --git a/lib/libvcc/vcc_types.c b/lib/libvcc/vcc_types.c index 65b7553..e98904c 100644 --- a/lib/libvcc/vcc_types.c +++ b/lib/libvcc/vcc_types.c @@ -124,6 +124,7 @@ const struct type REAL[1] = {{ const struct type STEVEDORE[1] = {{ .magic = 0xfae932d9, .name = "STEVEDORE", + .tostring = "VRT_STEVEDORE_string(\v1)", }}; const struct type STRING[1] = {{ From hermunn at varnish-software.com Thu Jun 9 14:28:09 2016 From: hermunn at varnish-software.com (Pål Hermunn Johansen) Date: Thu, 09 Jun 2016 16:28:09 +0200 Subject: [4.1] 65d90e3 Add support for TCP Fast Open extension Message-ID: commit 65d90e3dbed0a15e533b1633c8887f761a4eec59 Author: Federico G. Schwindt Date: Thu May 5 16:06:23 2016 +0100 Add support for TCP Fast Open extension Disabled by default. phk@ ok. diff --git a/bin/varnishd/cache/cache_acceptor.c b/bin/varnishd/cache/cache_acceptor.c index 5376203..647d37b 100644 --- a/bin/varnishd/cache/cache_acceptor.c +++ b/bin/varnishd/cache/cache_acceptor.c @@ -495,6 +495,14 @@ vca_acct(void *arg) VTAILQ_FOREACH(ls, &heritage.socks, list) { assert (ls->sock > 0); // We know where stdin is + if (cache_param->tcp_fastopen) { + int i; + i = VTCP_fastopen(ls->sock, cache_param->listen_depth); + if (i) + VSL(SLT_Error, ls->sock, + "Kernel TCP Fast Open: sock=%d, ret=%d %s", + ls->sock, i, strerror(errno)); + } AZ(listen(ls->sock, cache_param->listen_depth)); vca_tcp_opt_set(ls->sock, 1); if (cache_param->accept_filter) { diff --git a/configure.ac b/configure.ac index 8bc3f74..35a1b66 100644 --- a/configure.ac +++ b/configure.ac @@ -496,6 +496,34 @@ if test "$ac_cv_have_tcp_keep" = yes; then fi LIBS="${save_LIBS}" +# Check if the OS supports TCP_FASTOPEN socket option +save_LIBS="${LIBS}" +LIBS="${LIBS} ${NET_LIBS}" +AC_CACHE_CHECK([for TCP_FASTOPEN socket option], + [ac_cv_have_tcp_fastopen], + [AC_RUN_IFELSE( + [AC_LANG_PROGRAM([[ +#include +#include +#include +#include +#include + ]],[[ +int s = socket(AF_INET, SOCK_STREAM, 0); +int i; +i = 5; +if (setsockopt(s, SOL_TCP, TCP_FASTOPEN, &i, sizeof i)) + return (1); +return (0); + ]])], + [ac_cv_have_tcp_fastopen=yes], + [ac_cv_have_tcp_fastopen=no]) + ]) +if test "$ac_cv_have_tcp_fastopen" = yes; then + AC_DEFINE([HAVE_TCP_FASTOPEN], [1], [Define if OS supports TCP_FASTOPEN socket option]) +fi +LIBS="${save_LIBS}" + # Run-time directory VARNISH_STATE_DIR='${localstatedir}/varnish' AC_SUBST(VARNISH_STATE_DIR) diff --git a/include/tbl/params.h b/include/tbl/params.h index 402497f..42672d0 100644 --- a/include/tbl/params.h +++ b/include/tbl/params.h @@ -969,6 +969,20 @@ PARAM( /* func */ NULL ) +PARAM( + /* name */ tcp_fastopen, + /* typ */ bool, + /* min */ NULL, + /* max */ NULL, + /* default */ "off", + /* units */ "bool", + /* flags */ MUST_RESTART, + /* s-text */ + "Enable TCP Fast Open extension (if available in the kernel).", + /* l-text */ NULL, + /* func */ NULL +) + #if 0 PARAM( /* name */ tcp_keepalive_intvl, diff --git a/include/vtcp.h b/include/vtcp.h index a42e768..2937593 100644 --- a/include/vtcp.h +++ b/include/vtcp.h @@ -44,6 +44,7 @@ void VTCP_myname(int sock, char *abuf, unsigned alen, void VTCP_hisname(int sock, char *abuf, unsigned alen, char *pbuf, unsigned plen); int VTCP_filter_http(int sock); +int VTCP_fastopen(int sock, int depth); int VTCP_blocking(int sock); int VTCP_nonblocking(int sock); int VTCP_linger(int sock, int linger); diff --git a/lib/libvarnish/vtcp.c b/lib/libvarnish/vtcp.c index 09bf909..83e33b4 100644 --- a/lib/libvarnish/vtcp.c +++ b/lib/libvarnish/vtcp.c @@ -186,6 +186,30 @@ VTCP_filter_http(int sock) #endif +/*--------------------------------------------------------------------*/ + +#ifdef HAVE_TCP_FASTOPEN + +int +VTCP_fastopen(int sock, int depth) +{ + return (setsockopt(sock, SOL_TCP, TCP_FASTOPEN, + &depth, sizeof depth)); +} + +#else + +int +VTCP_fastopen(int sock, int depth) +{ + errno = EOPNOTSUPP; + (void)sock; + (void)depth; + return (-1); +} + +#endif + /*-------------------------------------------------------------------- * Functions for controlling NONBLOCK mode. * From hermunn at varnish-software.com Thu Jun 9 14:34:07 2016 From: hermunn at varnish-software.com (Pål Hermunn Johansen) Date: Thu, 09 Jun 2016 16:34:07 +0200 Subject: [4.1] 194acd4 Allow VCL set Last-Modified to be used for I-M-S processing. Message-ID: commit 194acd4f0a9e0483b8ae82ad58ed726129cd03eb Author: Poul-Henning Kamp Date: Wed May 18 07:03:30 2016 +0000 Allow VCL set Last-Modified to be used for I-M-S processing. Submitted by: rezan Fixes: #1931 diff --git a/bin/varnishd/cache/cache_rfc2616.c b/bin/varnishd/cache/cache_rfc2616.c index cb089cf..650bcf5 100644 --- a/bin/varnishd/cache/cache_rfc2616.c +++ b/bin/varnishd/cache/cache_rfc2616.c @@ -254,7 +254,7 @@ rfc2616_weak_compare(const char *p, const char *e) int RFC2616_Do_Cond(const struct req *req) { - const char *p, *e; + const char *p, *e, *l; double ims, lm; /* @@ -274,6 +274,12 @@ RFC2616_Do_Cond(const struct req *req) ims = VTIM_parse(p); if (!ims || ims > req->t_req) /* [RFC7232 3.3 p16] */ return (0); + if (http_GetHdr(req->resp, H_Last_Modified, &l)) { + lm = VTIM_parse(l); + if (!lm || lm > ims) + return (0); + return (1); + } AZ(ObjGetDouble(req->wrk, req->objcore, OA_LASTMODIFIED, &lm)); if (lm > ims) return (0); diff --git a/bin/varnishtest/tests/v00047.vtc b/bin/varnishtest/tests/v00047.vtc new file mode 100644 index 0000000..9ca2a6c --- /dev/null +++ b/bin/varnishtest/tests/v00047.vtc @@ -0,0 +1,40 @@ +varnishtest "Changing Last Modified in vcl_deliver" + +server s1 { + rxreq + txresp -hdr "Last-Modified: Wed, 27 Apr 2016 14:00:00 GMT" +} -start + +varnish v1 -vcl+backend { + sub vcl_deliver { + if (req.http.deliver == "modify") { + set resp.http.Last-Modified = "Wed, 27 Apr 2016 16:00:00 GMT"; + } + } +} -start + +client c1 { + txreq + rxresp + expect resp.status == 200 + + txreq -hdr "If-Modified-Since: Wed, 27 Apr 2016 14:00:00 GMT" + rxresp + expect resp.status == 304 + + txreq -hdr "If-Modified-Since: Wed, 27 Apr 2016 12:00:00 GMT" + rxresp + expect resp.status == 200 + + txreq -hdr "If-Modified-Since: Wed, 27 Apr 2016 14:00:00 GMT" -hdr "deliver: modify" + rxresp + expect resp.status == 200 + + txreq -hdr "If-Modified-Since: Wed, 27 Apr 2016 16:00:00 GMT" -hdr "deliver: modify" + rxresp + expect resp.status == 304 + + txreq -hdr "If-Modified-Since: Wed, 27 Apr 2016 18:00:00 GMT" -hdr "deliver: modify" + rxresp + expect resp.status == 304 +} -run From varnish-commit at varnish-cache.org Thu Jun 9 18:21:59 2016 From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org) Date: 9 Jun 2016 20:21:59 +0200 Subject: Weight Loss Plan For Those Who Want Quick Results Message-ID: Are you tired of the excess weight and would like to get back on track without any painful efforts? Brand-new formula and advanced structure of our exclusive product will compliment your results and will help you to lose weight with no side-effects. Click Here! Stock is limited, make sure to get yours in order to have a beautiful and healthy body! -------------- next part -------------- An HTML attachment was scrubbed... URL: From varnish-commit at varnish-cache.org Fri Jun 10 16:43:02 2016 From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org) Date: 10 Jun 2016 18:43:02 +0200 Subject: partial occupation Message-ID: <575AF33E.1518.25E39F@varnish-commit.varnish-cache.org> Hello! We are looking for employees working remotely. My name is Hal, am the personnel manager of a large International company. Most of the work you can do from home, that is, at a distance. Salary is $2500-$5000. If you are interested in this offer, please visit Our Site Best regards! -------------- next part -------------- An HTML attachment was scrubbed... URL: From varnish-commit at varnish-cache.org Fri Jun 10 20:48:45 2016 From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org) Date: 11 Jun 2016 02:48:45 +0600 Subject: beneficial proposition Message-ID: <575B2E3D.0466.2ABE9A@varnish-commit.varnish-cache.org> Hello! We are looking for employees working remotely. My name is Cornelius, am the personnel manager of a large International company. Most of the work you can do from home, that is, at a distance. Salary is $2500-$5000. If you are interested in this offer, please visit Our Site Best regards! -------------- next part -------------- An HTML attachment was scrubbed... URL: From varnish-commit at varnish-cache.org Sun Jun 12 20:02:18 2016 From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org) Date: 13 Jun 2016 02:02:18 +0600 Subject: Compliment Your Fitness Efforts With Exclusive Weight Loss Product Message-ID: <575DC43E.5685.D6EA86@varnish-commit.varnish-cache.org> Are you tired of the excess weight and would like to get back on track without any painful efforts? Brand-new formula and advanced structure of our exclusive product will compliment your results and will help you to lose weight with no side-effects. Click Here! Stock is limited, make sure to get yours in order to have a beautiful and healthy body! -------------- next part -------------- An HTML attachment was scrubbed... URL: From varnish-commit at varnish-cache.org Sun Jun 12 18:25:49 2016 From: varnish-commit at varnish-cache.org (pohsvnznk) Date: Mon, 13 Jun 2016 02:25:49 +0800 Subject: =?utf-8?B?dmFybmlzaC1jb21taXQ6SFLmnIDmo5jmiYs=?= =?utf-8?B?55qE5Y2B57G75qC45b+D6Zeu6aKY5Y+K6Kej5Yaz5pa55qGIIHNpcnVl?= Message-ID: <20160613022601557840@drbqx.net> ???varnish-commit??? ???????????????????????????? ??????????????? katfe??? 2016/6/13 ???2:26:00 -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 2016?HR??????60??????????.docx Type: application/vnd.openxmlformats-officedocument.wordprocessingml.document Size: 17489 bytes Desc: not available URL: From nils.goroll at uplex.de Sun Jun 12 19:39:05 2016 From: nils.goroll at uplex.de (Nils Goroll) Date: Sun, 12 Jun 2016 21:39:05 +0200 Subject: [master] 5c77581 only rsts go in the rstdir Message-ID: commit 5c77581c546ed6b6d12c8086ad4ea22ade35d9da Author: Nils Goroll Date: Sun Jun 12 21:38:16 2016 +0200 only rsts go in the rstdir diff --git a/lib/libvcc/vmodtool.py b/lib/libvcc/vmodtool.py index 9228e82..c8edb3a 100755 --- a/lib/libvcc/vmodtool.py +++ b/lib/libvcc/vmodtool.py @@ -702,7 +702,7 @@ class vcc(object): fo.close() def hfile(self): - fn = self.rstdir + "/" + self.pfx + ".h" + fn = self.pfx + ".h" self.commit_files.append(fn) fo = open(fn + ".tmp", "w") write_c_file_warning(fo) @@ -761,7 +761,7 @@ class vcc(object): fo.write("};\n") def cfile(self): - fn = self.rstdir + "/" + self.pfx + ".c" + fn = self.pfx + ".c" self.commit_files.append(fn) fo = open(fn + ".tmp", "w") write_c_file_warning(fo) From nils.goroll at uplex.de Sun Jun 12 19:42:06 2016 From: nils.goroll at uplex.de (Nils Goroll) Date: Sun, 12 Jun 2016 21:42:06 +0200 Subject: [master] f93edb5 if the prefix is not vcc_if, there is no vcc_if.h to include Message-ID: commit f93edb5be4929989be8b6cd83bf0f4138a83974a Author: Nils Goroll Date: Sun Jun 12 21:41:56 2016 +0200 if the prefix is not vcc_if, there is no vcc_if.h to include diff --git a/lib/libvcc/vmodtool.py b/lib/libvcc/vmodtool.py index c8edb3a..7b7c056 100755 --- a/lib/libvcc/vmodtool.py +++ b/lib/libvcc/vmodtool.py @@ -768,7 +768,7 @@ class vcc(object): fn2 = fn + ".tmp2" - for i in ["config", "vcl", "vrt", "vcc_if", "vmod_abi"]: + for i in ["config", "vcl", "vrt", self.pfx, "vmod_abi"]: fo.write('#include "%s.h"\n' % i) fo.write("\n") From varnish-commit at varnish-cache.org Mon Jun 13 07:25:09 2016 From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org) Date: Mon, 13 Jun 2016 07:25:09 -0000 Subject: Most Effective Nutrition Diet Designed For Women Message-ID: <875FF028392887E139964E5F4EE1875F@varnish-cache.org> Are you tired of the excess weight and would like to get back on track without any painful efforts? Brand-new formula and advanced structure of our exclusive product will compliment your results and will help you to lose weight with no side-effects. Click Here! Stock is limited, make sure to get yours in order to have a beautiful and healthy body! -------------- next part -------------- An HTML attachment was scrubbed... URL: From varnish-commit at varnish-cache.org Mon Jun 13 09:17:49 2016 From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org) Date: 13 Jun 2016 10:17:49 +0100 Subject: Hello! Message-ID: <2A3AF5DA66B5154909C6E95586267A3A@varnish-cache.org> Hello! We are looking for employees working remotely. My name is Bianca, am the personnel manager of a large International company. Most of the work you can do from home, that is, at a distance. Salary is $2500-$5000. If you are interested in this offer, please visit Our Site Best regards! -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Mon Jun 13 08:31:07 2016 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Mon, 13 Jun 2016 10:31:07 +0200 Subject: [master] dbfb06d Be pythonic Message-ID: commit dbfb06d4cd1e8bf017e8e537993863dca1acfd9b Author: Guillaume Quintard Date: Mon Jun 13 10:28:32 2016 +0200 Be pythonic diff --git a/lib/libvcc/vmodtool.py b/lib/libvcc/vmodtool.py index 7b7c056..d4b31ae 100755 --- a/lib/libvcc/vmodtool.py +++ b/lib/libvcc/vmodtool.py @@ -685,7 +685,7 @@ class vcc(object): fo.write(a + "\n") def rstfile(self, man=False): - fn = self.rstdir + "/vmod_" + self.modname + fn = join(self.rstdir, "vmod_" + self.modname) if man: fn += ".man" fn += ".rst" From nils.goroll at uplex.de Mon Jun 13 11:09:07 2016 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 13 Jun 2016 13:09:07 +0200 Subject: [master] bc13d83 vmodtool.py: support default value also for the last argument Message-ID: commit bc13d834374f6eef88912a62d189c1f8a507c423 Author: Nils Goroll Date: Thu Jun 9 17:06:22 2016 +0200 vmodtool.py: support default value also for the last argument closes #1976 diff --git a/bin/varnishtest/tests/m00019.vtc b/bin/varnishtest/tests/m00019.vtc index b640a44..98ea822 100644 --- a/bin/varnishtest/tests/m00019.vtc +++ b/bin/varnishtest/tests/m00019.vtc @@ -22,12 +22,12 @@ client c1 { txreq rxresp expect resp.bodylen == "6" - expect resp.http.foo1 == "1 2 3 ," - expect resp.http.foo2 == "1 2 3 ," - expect resp.http.foo3 == "1 2 3 ," - expect resp.http.foo4 == "1 2 3 ," - expect resp.http.foo5 == "1 2 3 ," - expect resp.http.foo6 == "1 2 3 ," + expect resp.http.foo1 == "1 2 3 , 4" + expect resp.http.foo2 == "1 2 3 , 4" + expect resp.http.foo3 == "1 2 3 , 4" + expect resp.http.foo4 == "1 2 3 , 4" + expect resp.http.foo5 == "1 2 3 , 4" + expect resp.http.foo6 == "1 2 3 , 4" } -run delay .1 @@ -48,10 +48,10 @@ varnish v1 -errvcl {Argument 'one' missing} { } } -varnish v1 -errvcl {Unknown argument 'four'} { +varnish v1 -errvcl {Unknown argument 'five'} { import debug; backend b1 {.host = "127.0.0.1";} sub vcl_deliver { - set resp.http.foo5 = debug.argtest("1", two=2.0, four="3"); + set resp.http.foo5 = debug.argtest("1", two=2.0, five="3"); } } diff --git a/lib/libvcc/vmodtool.py b/lib/libvcc/vmodtool.py index d4b31ae..e321b9b 100755 --- a/lib/libvcc/vmodtool.py +++ b/lib/libvcc/vmodtool.py @@ -235,7 +235,9 @@ def arg(txt): i = s.find('=') j = s.find(',') - if j >= 0 and j < i: + if j < 0: + j = len(s) + if j < i: i = -1 if i < 0: i = s.find(',') @@ -255,11 +257,14 @@ def arg(txt): s = s[m.end():] else: i = s.find(',') + if i < 0: + i = len(s) a.defval = s[:i] s = s[i:] return a,s +# XXX cant have ( or ) in an argument default value class prototype(object): def __init__(self, st, retval=True, prefix=""): l = st.line[1] diff --git a/lib/libvmod_debug/vmod.vcc b/lib/libvmod_debug/vmod.vcc index 1a74349..c0ae7d7 100644 --- a/lib/libvmod_debug/vmod.vcc +++ b/lib/libvmod_debug/vmod.vcc @@ -95,7 +95,7 @@ Encrypt the HTTP header with quad-ROT13 encryption, (this is approx 33% better than triple-DES). $Function STRING argtest(STRING one, REAL two=2, STRING three="3", - STRING comma=",") + STRING comma=",", INT four=4) $Function INT vre_limit() diff --git a/lib/libvmod_debug/vmod_debug.c b/lib/libvmod_debug/vmod_debug.c index 5ca628c..d58856a 100644 --- a/lib/libvmod_debug/vmod_debug.c +++ b/lib/libvmod_debug/vmod_debug.c @@ -186,11 +186,11 @@ vmod_rot52(VRT_CTX, VCL_HTTP hp) VCL_STRING vmod_argtest(VRT_CTX, VCL_STRING one, VCL_REAL two, VCL_STRING three, - VCL_STRING comma) + VCL_STRING comma, VCL_INT four) { char buf[100]; - bprintf(buf, "%s %g %s %s", one, two, three, comma); + bprintf(buf, "%s %g %s %s %ld", one, two, three, comma, four); return (WS_Copy(ctx->ws, buf, -1)); } From nils.goroll at uplex.de Mon Jun 13 11:09:07 2016 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 13 Jun 2016 13:09:07 +0200 Subject: [master] 511e83e lowercase an argument name Message-ID: commit 511e83e50d6c62ffcef7b900345f52a60539134b Author: Nils Goroll Date: Thu Jun 9 16:19:10 2016 +0200 lowercase an argument name diff --git a/lib/libvmod_debug/vmod.vcc b/lib/libvmod_debug/vmod.vcc index c0ae7d7..b172c74 100644 --- a/lib/libvmod_debug/vmod.vcc +++ b/lib/libvmod_debug/vmod.vcc @@ -123,7 +123,8 @@ $Method VOID .refresh(STRING addr, STRING port) Dynamically refresh & (always!) replace the backend by a new one. -$Function VOID workspace_allocate(ENUM { client, backend, session, thread }, INT SIZE) +$Function VOID workspace_allocate(ENUM { client, backend, session, thread }, + INT size) Allocate and zero out SIZE bytes from a workspace. From nils.goroll at uplex.de Mon Jun 13 11:09:07 2016 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 13 Jun 2016 13:09:07 +0200 Subject: [master] c4183dc improve m19.vtc: Set arguments to values differing from the defaults Message-ID: commit c4183dc57f2e86d883efd6da131ba5d9986639be Author: Nils Goroll Date: Thu Jun 9 17:10:11 2016 +0200 improve m19.vtc: Set arguments to values differing from the defaults diff --git a/bin/varnishtest/tests/m00019.vtc b/bin/varnishtest/tests/m00019.vtc index 98ea822..1b32afe 100644 --- a/bin/varnishtest/tests/m00019.vtc +++ b/bin/varnishtest/tests/m00019.vtc @@ -9,12 +9,12 @@ varnish v1 -vcl+backend { import debug; sub vcl_deliver { - set resp.http.foo1 = debug.argtest("1", 2.0, "3"); - set resp.http.foo2 = debug.argtest("1", two=2.0, three="3"); - set resp.http.foo3 = debug.argtest("1", three="3", two=2.0); - set resp.http.foo4 = debug.argtest("1", 2.0, three="3"); - set resp.http.foo5 = debug.argtest("1", 2.0); - set resp.http.foo6 = debug.argtest("1"); + set resp.http.foo1 = debug.argtest("1", 2.1, "3a"); + set resp.http.foo2 = debug.argtest("1", two=2.2, three="3b"); + set resp.http.foo3 = debug.argtest("1", three="3c", two=2.3); + set resp.http.foo4 = debug.argtest("1", 2.4, three="3d"); + set resp.http.foo5 = debug.argtest("1", 2.5); + set resp.http.foo6 = debug.argtest("1", four=6); } } -start @@ -22,12 +22,12 @@ client c1 { txreq rxresp expect resp.bodylen == "6" - expect resp.http.foo1 == "1 2 3 , 4" - expect resp.http.foo2 == "1 2 3 , 4" - expect resp.http.foo3 == "1 2 3 , 4" - expect resp.http.foo4 == "1 2 3 , 4" - expect resp.http.foo5 == "1 2 3 , 4" - expect resp.http.foo6 == "1 2 3 , 4" + expect resp.http.foo1 == "1 2.1 3a , 4" + expect resp.http.foo2 == "1 2.2 3b , 4" + expect resp.http.foo3 == "1 2.3 3c , 4" + expect resp.http.foo4 == "1 2.4 3d , 4" + expect resp.http.foo5 == "1 2.5 3 , 4" + expect resp.http.foo6 == "1 2 3 , 6" } -run delay .1 From nils.goroll at uplex.de Mon Jun 13 13:22:08 2016 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 13 Jun 2016 15:22:08 +0200 Subject: [master] cb12bdd vmodtool: fix first argument starting on new line Message-ID: commit cb12bdd8429c0d879f8b8423cabefc0fc49f2a6b Author: Nils Goroll Date: Mon Jun 13 15:20:56 2016 +0200 vmodtool: fix first argument starting on new line Fixes issue reported by @carlosabalde in #1976 diff --git a/lib/libvcc/vmodtool.py b/lib/libvcc/vmodtool.py index e321b9b..176f618 100755 --- a/lib/libvcc/vmodtool.py +++ b/lib/libvcc/vmodtool.py @@ -288,7 +288,7 @@ class prototype(object): s = s[i:].strip() assert s[0] == "(" assert s[-1] == ")" - s = s[1:-1] + s = s[1:-1].lstrip() self.args = [] while len(s) > 0: a,s = arg(s) diff --git a/lib/libvmod_debug/vmod.vcc b/lib/libvmod_debug/vmod.vcc index b172c74..b057ff2 100644 --- a/lib/libvmod_debug/vmod.vcc +++ b/lib/libvmod_debug/vmod.vcc @@ -94,7 +94,8 @@ $Function VOID rot52(HTTP hdr) Encrypt the HTTP header with quad-ROT13 encryption, (this is approx 33% better than triple-DES). -$Function STRING argtest(STRING one, REAL two=2, STRING three="3", +$Function STRING argtest( + STRING one, REAL two=2, STRING three="3", STRING comma=",", INT four=4) $Function INT vre_limit() From varnish-commit at varnish-cache.org Mon Jun 13 19:23:16 2016 From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org) Date: Mon, 13 Jun 2016 21:23:16 +0200 Subject: Flexible schedule Message-ID: <575F0824.9050705@varnish-cache.org> Hello! We are looking for employees working remotely. My name is Albert, am the personnel manager of a large International company. Most of the work you can do from home, that is, at a distance. Salary is $2500-$5000. If you are interested in this offer, please visit Our Site Best regards! -------------- next part -------------- An HTML attachment was scrubbed... URL: From nils.goroll at uplex.de Mon Jun 13 19:15:09 2016 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 13 Jun 2016 21:15:09 +0200 Subject: [master] 3655bf6 vcc fix keeping track of the first reference Message-ID: commit 3655bf6cfaa214327f6205ed063647b75f73be77 Author: Nils Goroll Date: Mon Jun 13 21:07:17 2016 +0200 vcc fix keeping track of the first reference Fixes #1979 diff --git a/lib/libvcc/vcc_compile.h b/lib/libvcc/vcc_compile.h index 8a2b20e..22b30f2 100644 --- a/lib/libvcc/vcc_compile.h +++ b/lib/libvcc/vcc_compile.h @@ -128,7 +128,7 @@ struct symbol { const void *wildcard_priv; enum symkind kind; - const struct token *def_b, *def_e; + const struct token *def_b, *def_e, *ref_b; vcc_type_t fmt; diff --git a/lib/libvcc/vcc_xref.c b/lib/libvcc/vcc_xref.c index e8feac2..8a75c25 100644 --- a/lib/libvcc/vcc_xref.c +++ b/lib/libvcc/vcc_xref.c @@ -79,6 +79,8 @@ vcc_AddRef(struct vcc *tl, const struct token *t, enum symkind kind) struct symbol *sym; sym = VCC_SymbolTok(tl, NULL, t, kind, 1); + if (sym->ref_b == NULL) + sym->ref_b = t; AN(sym); sym->nref++; } @@ -103,10 +105,12 @@ vcc_checkref(struct vcc *tl, const struct symbol *sym) { if (sym->ndef == 0 && sym->nref != 0) { + AN(sym->ref_b); VSB_printf(tl->sb, "Undefined %s %.*s, first reference:\n", - VCC_SymKind(tl, sym), PF(sym->def_b)); - vcc_ErrWhere(tl, sym->def_b); + VCC_SymKind(tl, sym), PF(sym->ref_b)); + vcc_ErrWhere(tl, sym->ref_b); } else if (sym->ndef != 0 && sym->nref == 0) { + AN(sym->def_b); VSB_printf(tl->sb, "Unused %s %.*s, defined:\n", VCC_SymKind(tl, sym), PF(sym->def_b)); vcc_ErrWhere(tl, sym->def_b); From varnish-commit at varnish-cache.org Mon Jun 13 22:04:02 2016 From: varnish-commit at varnish-cache.org (ydlchm) Date: Tue, 14 Jun 2016 06:04:02 +0800 Subject: =?utf-8?B?dmFybmlzaC1jb21taXTvvJrku47kvKDnu5/liLDnsr4=?= =?utf-8?B?55uK5YyW5Yiw5pm66IO95YyW55qE5Y2T6LaK55Sf5Lqn57O757uf5p6E5bu6?= Message-ID: <20160614060407106661@lhsbha.org> ???varnish-commit:??? ??4.0??????????? ?????????????????????? ???????????????????? ??????????????????? ??????????????? ??? 2016/6/14 ???6:04:06 -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ??4.0???????????.docx Type: application/vnd.openxmlformats-officedocument.wordprocessingml.document Size: 19903 bytes Desc: not available URL: From lkarsten at varnish-software.com Tue Jun 14 07:31:07 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 09:31:07 +0200 Subject: [4.1] 662856d Bring changelog up to speed. Message-ID: commit 662856de0de53f27f978ac1b6e23685cf1dda35e Author: Lasse Karstensen Date: Tue Jun 14 09:30:33 2016 +0200 Bring changelog up to speed. diff --git a/doc/changes.rst b/doc/changes.rst index a8f9920..2d53120 100644 --- a/doc/changes.rst +++ b/doc/changes.rst @@ -6,12 +6,20 @@ Changes since 4.1.2: * varnishncsa can now access and log backend requests. (PR #1905) +* On SmartOS, use ports instead of epoll by default. + +* Add support for TCP Fastopen where available. Disabled by default. + .. _1905: https://github.com/varnishcache/varnish-cache/pull/1905 Bugs fixed ---------- +* 1971_ - Add missing Wait_HeapDelete +* 1967_ - [ncsa] Remove implicit line feed when using formatfile. * 1955_ - 4.1.x sometimes duplicates Age and Accept-Ranges headers +* 1954_ - Correctly handle HTTP/1.1 EOF response +* 1931_ - Allow VCL set Last-Modified to be used for I-M-S processing. * 1928_ - req->task members must be set in case we get onto the waitinglist * 1918_ - Correctly handle EOF responses with HTTP/1.1 * 1912_ - Fix (insignificant) memory leak with mal-formed ESI directives. @@ -26,7 +34,11 @@ Bugs fixed * 1806_ - One minute delay on return (pipe) and a POST-Request * 1725_ - Revive the backend_conn counter +.. _1971: https://github.com/varnishcache/varnish-cache/issues/1971 +.. _1967: https://github.com/varnishcache/varnish-cache/issues/1967 .. _1955: https://github.com/varnishcache/varnish-cache/issues/1955 +.. _1954: https://github.com/varnishcache/varnish-cache/issues/1954 +.. _1931: https://github.com/varnishcache/varnish-cache/issues/1931 .. _1928: https://github.com/varnishcache/varnish-cache/issues/1928 .. _1918: https://github.com/varnishcache/varnish-cache/issues/1918 .. _1912: https://github.com/varnishcache/varnish-cache/issues/1912 From lkarsten at varnish-software.com Tue Jun 14 08:25:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 10:25:09 +0200 Subject: [4.1] fef2456 Backfill more changelog entries. Message-ID: commit fef2456466adfbf6c265c123a9abb63342d526a9 Author: Lasse Karstensen Date: Tue Jun 14 09:49:17 2016 +0200 Backfill more changelog entries. diff --git a/doc/changes.rst b/doc/changes.rst index 2d53120..66165a6 100644 --- a/doc/changes.rst +++ b/doc/changes.rst @@ -6,6 +6,10 @@ Changes since 4.1.2: * varnishncsa can now access and log backend requests. (PR #1905) +* [varnishncsa] New formatter %{VSL:Tag}x. + +* [varnishlog] Added log tag BacakendStart on backend transactions. + * On SmartOS, use ports instead of epoll by default. * Add support for TCP Fastopen where available. Disabled by default. @@ -21,8 +25,10 @@ Bugs fixed * 1954_ - Correctly handle HTTP/1.1 EOF response * 1931_ - Allow VCL set Last-Modified to be used for I-M-S processing. * 1928_ - req->task members must be set in case we get onto the waitinglist +* 1919_ - Avoid ban lurker panic with empty olist. * 1918_ - Correctly handle EOF responses with HTTP/1.1 * 1912_ - Fix (insignificant) memory leak with mal-formed ESI directives. +* 1904_ - Release memory instead of crashing on malformed ESI * 1885_ - [vmodtool] Method names should start with a period * 1879_ - Correct handling of duplicate headers on IMS header merge * 1878_ - Fix a ESI+gzip corner case which had escaped notice until now @@ -40,8 +46,10 @@ Bugs fixed .. _1954: https://github.com/varnishcache/varnish-cache/issues/1954 .. _1931: https://github.com/varnishcache/varnish-cache/issues/1931 .. _1928: https://github.com/varnishcache/varnish-cache/issues/1928 +.. _1919: https://github.com/varnishcache/varnish-cache/issues/1919 .. _1918: https://github.com/varnishcache/varnish-cache/issues/1918 .. _1912: https://github.com/varnishcache/varnish-cache/issues/1912 +.. _1904: https://github.com/varnishcache/varnish-cache/issues/1904 .. _1885: https://github.com/varnishcache/varnish-cache/issues/1885 .. _1883: https://github.com/varnishcache/varnish-cache/issues/1883 .. _1879: https://github.com/varnishcache/varnish-cache/issues/1879 From lkarsten at varnish-software.com Tue Jun 14 08:25:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 10:25:09 +0200 Subject: [4.1] f88292a Make it more easily readable on Github. Message-ID: commit f88292a59bf8f30f447eb912ecbb68eea6a46530 Author: Lasse Karstensen Date: Thu Mar 10 11:16:26 2016 +0100 Make it more easily readable on Github. (But mostly for testing the commit email system.) diff --git a/README b/README deleted file mode 100644 index 3ce0b45..0000000 --- a/README +++ /dev/null @@ -1,12 +0,0 @@ -This is Varnish Cache, the high-performance HTTP accelerator. - -Documentation and additional information about Varnish is available on -https://www.varnish-cache.org/ - -Technical questions about Varnish and this release should be addressed -to . Please see -https://www.varnish-cache.org/trac/wiki/Contributing for how to -contribute patches and report bugs. - -Questions about commercial support and services related to Varnish -should be addressed to . diff --git a/README.rst b/README.rst new file mode 100644 index 0000000..309edc9 --- /dev/null +++ b/README.rst @@ -0,0 +1,15 @@ +Varnish Cache +============= + +This is Varnish Cache, the high-performance HTTP accelerator. + +Documentation and additional information about Varnish is available on +https://www.varnish-cache.org/ + +Technical questions about Varnish and this release should be addressed +to . Please see +https://www.varnish-cache.org/trac/wiki/Contributing for how to +contribute patches and report bugs. + +Questions about commercial support and services related to Varnish +should be addressed to . From lkarsten at varnish-software.com Tue Jun 14 08:25:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 10:25:09 +0200 Subject: [4.1] 309c7de Update copyright. Message-ID: commit 309c7de4a90e39abc062f3f97883e41f315fe978 Author: Lasse Karstensen Date: Thu Mar 10 11:43:34 2016 +0100 Update copyright. (and test the commit email hook) diff --git a/configure.ac b/configure.ac index 35a1b66..60ca42d 100644 --- a/configure.ac +++ b/configure.ac @@ -1,6 +1,6 @@ AC_PREREQ(2.59) AC_COPYRIGHT([Copyright (c) 2006 Verdens Gang AS -Copyright (c) 2006-2016 Varnish Software AS]) +Copyright (c) 2006-2016 Varnish Software]) AC_REVISION([$Id$]) AC_INIT([Varnish], [4.1.3-dev], [varnish-dev at varnish-cache.org]) AC_CONFIG_SRCDIR(include/miniobj.h) From lkarsten at varnish-software.com Tue Jun 14 08:25:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 10:25:09 +0200 Subject: [4.1] 616e1b5 Clarified the documentation on vsm_overflow[ed] Message-ID: commit 616e1b5790192d4205cd7f6002d72e7f7016612f Author: P?l Hermunn Johansen Date: Wed Mar 9 16:38:29 2016 +0100 Clarified the documentation on vsm_overflow[ed] In most circumstances the counters vsm_overflow and vsm_overflowed should be zero. This patch tells the user what he should probably do when they are not. diff --git a/include/tbl/vsc_f_main.h b/include/tbl/vsc_f_main.h index 94341b5..13106f4 100644 --- a/include/tbl/vsc_f_main.h +++ b/include/tbl/vsc_f_main.h @@ -629,6 +629,8 @@ VSC_F(vsm_overflow, uint64_t, 0, 'g', 'B', diag, "Number of bytes which does not fit" " in the shared memory used to communicate" " with tools like varnishstat, varnishlog etc." + " If this counter is not zero, consider" + " increasing the runtime variable vsm_space." ) VSC_F(vsm_overflowed, uint64_t, 0, 'c', 'B', diag, @@ -636,4 +638,6 @@ VSC_F(vsm_overflowed, uint64_t, 0, 'c', 'B', diag, "Total number of bytes which did not fit" " in the shared memory used to communicate" " with tools like varnishstat, varnishlog etc." + " If this counter is not zero, consider" + " increasing the runtime variable vsm_space." ) From lkarsten at varnish-software.com Tue Jun 14 08:25:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 10:25:09 +0200 Subject: [4.1] 7614409 Invert desc/explanation in varnish-counters.7 Message-ID: commit 761440995702d5cdcf8120479fa29b73feed89e5 Author: Guillaume Quintard Date: Wed Mar 9 15:57:16 2016 +0100 Invert desc/explanation in varnish-counters.7 Spotted by: hermunn diff --git a/bin/varnishstat/vsc2rst.c b/bin/varnishstat/vsc2rst.c index 3358db4..2aaa16d 100644 --- a/bin/varnishstat/vsc2rst.c +++ b/bin/varnishstat/vsc2rst.c @@ -41,7 +41,7 @@ #define VSC_LEVEL_F(v,l,e,d) \ printf("%s ? %s\n\t%s\n\n", l, e, d); #define VSC_F(n, t, l, s, f, v, d, e) \ - printf("%s ? %s (%s)\n\t%s\n\n", #n, e, VSC_level_##v, d); + printf("%s ? %s (%s)\n\t%s\n\n", #n, d, VSC_level_##v, e); int main(int argc, char **argv) { From lkarsten at varnish-software.com Tue Jun 14 08:25:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 10:25:09 +0200 Subject: [4.1] b5ddc7a Update instructions on port change with systemd. Message-ID: commit b5ddc7ae57c1a56bb1639da16769b4714b7af7ca Author: Lasse Karstensen Date: Thu Mar 10 13:09:44 2016 +0100 Update instructions on port change with systemd. As suggested by Przemek Sobstel on github. diff --git a/doc/sphinx/tutorial/putting_varnish_on_port_80.rst b/doc/sphinx/tutorial/putting_varnish_on_port_80.rst index 3e57917..16d38bc 100644 --- a/doc/sphinx/tutorial/putting_varnish_on_port_80.rst +++ b/doc/sphinx/tutorial/putting_varnish_on_port_80.rst @@ -9,10 +9,10 @@ First we stop varnish: ``service varnish stop`` Now we need to edit the configuration file that starts Varnish. -Debian/Ubuntu -~~~~~~~~~~~~~ +Debian/Ubuntu (legacy) +~~~~~~~~~~~~~~~~~~~~~~ -On Debian/Ubuntu this is `/etc/default/varnish`. In the file you'll find +On older Debian/Ubuntu this is `/etc/default/varnish`. In the file you'll find some text that looks like this:: DAEMON_OPTS="-a :6081 \ @@ -29,6 +29,26 @@ Change it to:: -S /etc/varnish/secret \ -s malloc,256m" +Debian (v8+) / Ubuntu (v15.04+) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +On more recent Debian and Ubuntu systems this is configured in the systemd +service file. + +Applying changes to the default service is best done by creating a new file +`/etc/systemd/system/varnish.service.d/customexec.conf`:: + + [Service] + ExecStart= + ExecStart=/usr/sbin/varnishd -a :80 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -s malloc,256m + +This will override the ExecStart part of the default configuration shipped +with Varnish Cache. + +Run ``systemctl daemon-reload`` to make sure systemd picks up the new +configuration before restarting Varnish. + + Red Hat Enterprise Linux / CentOS ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -42,4 +62,3 @@ Restarting Varnish again Once the change is done, restart Varnish: ``service varnish start``. Now everyone accessing your site will be accessing through Varnish. - From lkarsten at varnish-software.com Tue Jun 14 08:25:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 10:25:09 +0200 Subject: [4.1] c55e322 Add an "expect_pattern" verb, which expects a "01234567012345..." pattern in the body. Message-ID: commit c55e322002f6bd8b18092cf99c3a454c9af78f70 Author: Poul-Henning Kamp Date: Sat Mar 12 00:17:27 2016 +0000 Add an "expect_pattern" verb, which expects a "01234567012345..." pattern in the body. diff --git a/bin/varnishtest/vtc_http.c b/bin/varnishtest/vtc_http.c index 7822ebc..5144d36 100644 --- a/bin/varnishtest/vtc_http.c +++ b/bin/varnishtest/vtc_http.c @@ -298,6 +298,30 @@ cmd_http_expect(CMD_ARGS) av[0], clhs, cmp, crhs, retval ? "match" : "failed"); } +static void +cmd_http_expect_pattern(CMD_ARGS) +{ + char *p; + struct http *hp; + char t = '0'; + + (void)cmd; + (void)vl; + CAST_OBJ_NOTNULL(hp, priv, HTTP_MAGIC); + AZ(strcmp(av[0], "expect_pattern")); + av++; + AZ(av[0]); + for (p = hp->body; *p != '\0'; p++) { + if (*p != t) + vtc_log(hp->vl, 0, + "EXPECT PATTERN FAIL @%zd should 0x%02x is 0x%02x", + p - hp->body, t, *p); + t += 1; + t &= ~0x08; + } + vtc_log(hp->vl, 4, "EXPECT PATTERN SUCCESS"); +} + /********************************************************************** * Split a HTTP protocol header */ @@ -1345,6 +1369,7 @@ static const struct cmds http_cmds[] = { { "rxrespbody", cmd_http_rxrespbody }, { "gunzip", cmd_http_gunzip_body }, { "expect", cmd_http_expect }, + { "expect_pattern", cmd_http_expect_pattern }, { "recv", cmd_http_recv }, { "send", cmd_http_send }, { "send_n", cmd_http_send_n }, From lkarsten at varnish-software.com Tue Jun 14 08:25:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 10:25:09 +0200 Subject: [4.1] 95b7d65 Update to 4.1 vtc syntax. Message-ID: commit 95b7d6506e32d835321e13f064b505abb40816d4 Author: Lasse Karstensen Date: Tue Jun 14 10:21:49 2016 +0200 Update to 4.1 vtc syntax. diff --git a/bin/varnishtest/tests/r01955.vtc b/bin/varnishtest/tests/r01955.vtc index 756ebc0..48fb58e 100644 --- a/bin/varnishtest/tests/r01955.vtc +++ b/bin/varnishtest/tests/r01955.vtc @@ -9,7 +9,7 @@ server s1 { } -start varnish v1 -vcl+backend { - import std; + import ${vmod_std}; sub vcl_deliver { std.collect(resp.http.age); } From nils.goroll at uplex.de Tue Jun 14 08:31:08 2016 From: nils.goroll at uplex.de (Nils Goroll) Date: Tue, 14 Jun 2016 10:31:08 +0200 Subject: [master] 3d7bd8b Fix typo Message-ID: commit 3d7bd8bfcecd4c87fd3d506b2a67747d0af5a9c6 Author: Magnus Hagander Date: Tue Jun 14 10:24:10 2016 +0200 Fix typo An object is delivered to the client, not to the backend. diff --git a/include/tbl/vsc_f_main.h b/include/tbl/vsc_f_main.h index 0d98740..2657086 100644 --- a/include/tbl/vsc_f_main.h +++ b/include/tbl/vsc_f_main.h @@ -102,7 +102,7 @@ VSC_F(cache_miss, uint64_t, 1, 'c', 'i', info, "Cache misses", "Count of misses" " A cache miss indicates the object was fetched from the" - " backend before delivering it to the backend." + " backend before delivering it to the client." ) /*---------------------------------------------------------------------*/ From lkarsten at varnish-software.com Tue Jun 14 09:19:08 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:19:08 +0200 Subject: [4.1] e2b8877 Add missing call to VSL_Match Message-ID: commit e2b8877b4b0e13a322ba481972d369109c86de52 Author: Federico G. Schwindt Date: Tue Mar 15 20:47:27 2016 +0000 Add missing call to VSL_Match With this you can also filter based on logexpect's vsl arguments. diff --git a/bin/varnishtest/vtc_logexp.c b/bin/varnishtest/vtc_logexp.c index 1c0ea70..8ac5332 100644 --- a/bin/varnishtest/vtc_logexp.c +++ b/bin/varnishtest/vtc_logexp.c @@ -196,11 +196,13 @@ logexp_dispatch(struct VSL_data *vsl, struct VSL_transaction * const pt[], int vxid, tag, type, len; const char *legend, *data; - (void)vsl; CAST_OBJ_NOTNULL(le, priv, LOGEXP_MAGIC); for (i = 0; (t = pt[i]); i++) { while (1 == VSL_Next(t->c)) { + if (!VSL_Match(vsl, t->c)) + continue; + CHECK_OBJ_NOTNULL(le->test, LOGEXP_TEST_MAGIC); AN(t->c->rec.ptr); vxid = VSL_ID(t->c->rec.ptr); From lkarsten at varnish-software.com Tue Jun 14 09:19:08 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:19:08 +0200 Subject: [4.1] abf1ca5 Spelling Message-ID: commit abf1ca53d3f4aca99867501313a377301b2571ab Author: Federico G. Schwindt Date: Wed Mar 16 15:54:11 2016 +0000 Spelling diff --git a/bin/varnishncsa/varnishncsa.c b/bin/varnishncsa/varnishncsa.c index 9a2e89c..f15aa69 100644 --- a/bin/varnishncsa/varnishncsa.c +++ b/bin/varnishncsa/varnishncsa.c @@ -82,7 +82,7 @@ enum e_frag { F_h, /* %h Host name / IP Address */ F_m, /* %m Method */ F_s, /* %s Status */ - F_I, /* %I Bytes recieved */ + F_I, /* %I Bytes received */ F_O, /* %O Bytes sent */ F_tstart, /* Time start */ F_tend, /* Time end */ @@ -720,7 +720,7 @@ parse_format(const char *format) case 'H': /* Protocol */ addf_fragment(&CTX.frag[F_H], "HTTP/1.0"); break; - case 'I': /* Bytes recieved */ + case 'I': /* Bytes received */ addf_fragment(&CTX.frag[F_I], "-"); break; case 'l': /* Client user ID (identd) always '-' */ From lkarsten at varnish-software.com Tue Jun 14 09:19:08 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:19:08 +0200 Subject: [4.1] 315bdbc Introduce barriers in varnishtest Message-ID: commit 315bdbcc039cc5ae489bf3e8a6f3073c60e8d9e5 Author: Dridi Boukelmoune Date: Tue Dec 22 18:16:08 2015 +0100 Introduce barriers in varnishtest They work like semaphores, except that they need explicit initialization and can optionally be shared between processes. Conflicts: bin/varnishtest/tests/c00073.vtc diff --git a/bin/varnishtest/Makefile.am b/bin/varnishtest/Makefile.am index 2506a8a..a748fee 100644 --- a/bin/varnishtest/Makefile.am +++ b/bin/varnishtest/Makefile.am @@ -27,6 +27,7 @@ varnishtest_SOURCES = \ vtc.h \ vmods.h \ programs.h \ + vtc_barrier.c \ vtc_client.c \ vtc_http.c \ vtc_main.c \ diff --git a/bin/varnishtest/Makefile.phk b/bin/varnishtest/Makefile.phk index 438bfed..5c80efd 100644 --- a/bin/varnishtest/Makefile.phk +++ b/bin/varnishtest/Makefile.phk @@ -5,6 +5,7 @@ PROG_SRC += vtc_log.c PROG_SRC += vtc_logexp.c PROG_SRC += vtc_main.c PROG_SRC += vtc_sema.c +PROG_SRC += vtc_barrier.c PROG_SRC += vtc_server.c PROG_SRC += vtc_varnish.c PROG_SRC += vtc_process.c diff --git a/bin/varnishtest/tests/a00008.vtc b/bin/varnishtest/tests/a00008.vtc index 3a581c8..6fe35c9 100644 --- a/bin/varnishtest/tests/a00008.vtc +++ b/bin/varnishtest/tests/a00008.vtc @@ -1,22 +1,25 @@ -varnishtest "Sema operations" +varnishtest "Barrier operations" + +barrier b1 cond 4 +barrier b2 cond 4 server s1 { rxreq - sema r1 sync 4 + barrier b1 sync delay .9 txresp } -start server s2 { rxreq - sema r1 sync 4 + barrier b1 sync delay .6 txresp } -start server s3 { rxreq - sema r1 sync 4 + barrier b1 sync delay .2 txresp } -start @@ -25,25 +28,25 @@ client c1 -connect ${s1_sock} { delay .2 txreq rxresp - sema r1 sync 4 + barrier b2 sync } -start client c2 -connect ${s2_sock} { delay .6 txreq rxresp - sema r1 sync 4 + barrier b2 sync } -start client c3 -connect ${s3_sock} { delay .9 txreq rxresp - sema r1 sync 4 + barrier b2 sync } -start # Wait for all servers to have received requests -sema r1 sync 4 +barrier b1 sync # Wait for all clients to have received responses -sema r1 sync 4 +barrier b2 sync diff --git a/bin/varnishtest/tests/c00073.vtc b/bin/varnishtest/tests/c00073.vtc new file mode 100644 index 0000000..4c8d4d0 --- /dev/null +++ b/bin/varnishtest/tests/c00073.vtc @@ -0,0 +1,32 @@ +varnishtest "Test object trimming" + +barrier b1 cond 2 +barrier b2 cond 2 + +server s1 { + rxreq + txresp -nolen -hdr "Transfer-encoding: chunked" + delay .2 + chunkedlen 4096 + barrier b1 sync + barrier b2 sync + chunkedlen 0 +} -start + +varnish v1 \ + -arg "-s malloc,1m" -vcl+backend { } -start + +client c1 { + txreq + rxresp +} -start + +barrier b1 sync +varnish v1 -expect SMA.s0.g_bytes > 10000 + +barrier b2 sync + +client c1 -wait + +# Removed when adopting to 4.1. VTC kept since it uses barriers. +#varnish v1 -expect SMA.s0.g_bytes < 6000 diff --git a/bin/varnishtest/vtc.c b/bin/varnishtest/vtc.c index aea1081..7c1ee4d 100644 --- a/bin/varnishtest/vtc.c +++ b/bin/varnishtest/vtc.c @@ -632,6 +632,7 @@ static const struct cmds cmds[] = { { "shell", cmd_shell }, { "err_shell", cmd_err_shell }, { "sema", cmd_sema }, + { "barrier", cmd_barrier }, { "random", cmd_random }, { "feature", cmd_feature }, { "logexpect", cmd_logexp }, @@ -655,6 +656,7 @@ exec_file(const char *fn, const char *script, const char *tmpdir, init_macro(); init_sema(); + init_barrier(); init_server(); /* Apply extmacro definitions */ diff --git a/bin/varnishtest/vtc.h b/bin/varnishtest/vtc.h index 518494c..584b1ba 100644 --- a/bin/varnishtest/vtc.h +++ b/bin/varnishtest/vtc.h @@ -62,6 +62,7 @@ cmd_f cmd_server; cmd_f cmd_client; cmd_f cmd_varnish; cmd_f cmd_sema; +cmd_f cmd_barrier; cmd_f cmd_logexp; cmd_f cmd_process; @@ -74,6 +75,7 @@ extern int vtc_witness; extern int feature_dns; void init_sema(void); +void init_barrier(void); void init_server(void); int http_process(struct vtclog *vl, const char *spec, int sock, int *sfd); diff --git a/bin/varnishtest/vtc_barrier.c b/bin/varnishtest/vtc_barrier.c new file mode 100644 index 0000000..448eb0d --- /dev/null +++ b/bin/varnishtest/vtc_barrier.c @@ -0,0 +1,239 @@ +/*- + * Copyright (c) 2005 Varnish Software AS + * All rights reserved. + * + * Author: Dridi Boukelmoune + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include "config.h" + +#include +#include +#include +#include +#include + +#include "vtc.h" + +enum barrier_e { + BARRIER_NONE = 0, + BARRIER_COND, + BARRIER_SOCK, +}; + +struct barrier { + unsigned magic; +#define BARRIER_MAGIC 0x7b54c275 + char *name; + VTAILQ_ENTRY(barrier) list; + pthread_mutex_t mtx; + pthread_cond_t cond; + + unsigned waiters; + unsigned expected; + + enum barrier_e type; +}; + +static pthread_mutex_t barrier_mtx; +static VTAILQ_HEAD(, barrier) barriers = VTAILQ_HEAD_INITIALIZER(barriers); + +static struct barrier * +barrier_new(char *name, struct vtclog *vl) +{ + struct barrier *b; + + ALLOC_OBJ(b, BARRIER_MAGIC); + AN(b); + AN(name); + if (*name != 'b') + vtc_log(vl, 0, "Barrier name must start with 'b' (%s)", name); + REPLACE(b->name, name); + + AZ(pthread_mutex_init(&b->mtx, NULL)); + AZ(pthread_cond_init(&b->cond, NULL)); + b->waiters = 0; + b->expected = 0; + VTAILQ_INSERT_TAIL(&barriers, b, list); + return (b); +} + +/********************************************************************** + * Init a barrier + */ + +static void +barrier_expect(struct barrier *b, const char *av, struct vtclog *vl) +{ + unsigned expected; + + if (b->type != BARRIER_NONE) + vtc_log(vl, 0, + "Barrier(%s) use error: already initialized", b->name); + + AZ(b->expected); + AZ(b->waiters); + expected = strtoul(av, NULL, 0); + if (expected < 2) + vtc_log(vl, 0, + "Barrier(%s) use error: wrong expectation (%u)", + b->name, expected); + + b->expected = expected; +} + +static void +barrier_cond(struct barrier *b, const char *av, struct vtclog *vl) +{ + + CHECK_OBJ_NOTNULL(b, BARRIER_MAGIC); + barrier_expect(b, av, vl); + b->type = BARRIER_COND; +} + +static void +barrier_sock(struct barrier *b, const char *av, struct vtclog *vl) +{ + + CHECK_OBJ_NOTNULL(b, BARRIER_MAGIC); + barrier_expect(b, av, vl); + b->type = BARRIER_SOCK; + INCOMPL(); +} + +/********************************************************************** + * Sync a barrier + */ + +static void +barrier_cond_sync(struct barrier *b, struct vtclog *vl) +{ + + CHECK_OBJ_NOTNULL(b, BARRIER_MAGIC); + assert(b->type == BARRIER_COND); + + assert(b->waiters <= b->expected); + if (b->waiters == b->expected) + vtc_log(vl, 0, + "Barrier(%s) use error: more waiters than the %u expected", + b->name, b->expected); + + if (++b->waiters == b->expected) { + vtc_log(vl, 4, "Barrier(%s) wake %u", b->name, b->expected); + AZ(pthread_cond_broadcast(&b->cond)); + } + else { + vtc_log(vl, 4, "Barrier(%s) wait %u of %u", + b->name, b->waiters, b->expected); + AZ(pthread_cond_wait(&b->cond, &b->mtx)); + } +} + +static void +barrier_sync(struct barrier *b, struct vtclog *vl) +{ + + CHECK_OBJ_NOTNULL(b, BARRIER_MAGIC); + + switch (b->type) { + case BARRIER_NONE: + vtc_log(vl, 0, + "Barrier(%s) use error: not initialized", b->name); + case BARRIER_COND: + barrier_cond_sync(b, vl); + break; + case BARRIER_SOCK: + INCOMPL(); + break; + default: + WRONG("Wrong barrier type"); + } +} + +/********************************************************************** + * Barrier command dispatch + */ + +void +cmd_barrier(CMD_ARGS) +{ + struct barrier *b, *b2; + + (void)priv; + (void)cmd; + + if (av == NULL) { + AZ(pthread_mutex_lock(&barrier_mtx)); + /* Reset and free */ + VTAILQ_FOREACH_SAFE(b, &barriers, list, b2) { + AZ(pthread_mutex_lock(&b->mtx)); + assert(b->type != BARRIER_NONE); + assert(b->waiters == b->expected); + AZ(pthread_mutex_unlock(&b->mtx)); + } + AZ(pthread_mutex_unlock(&barrier_mtx)); + return; + } + + AZ(strcmp(av[0], "barrier")); + av++; + + AZ(pthread_mutex_lock(&barrier_mtx)); + VTAILQ_FOREACH(b, &barriers, list) + if (!strcmp(b->name, av[0])) + break; + if (b == NULL) + b = barrier_new(av[0], vl); + av++; + AZ(pthread_mutex_lock(&b->mtx)); + AZ(pthread_mutex_unlock(&barrier_mtx)); + + for (; *av != NULL; av++) { + if (!strcmp(*av, "cond")) { + av++; + AN(*av); + barrier_cond(b, *av, vl); + continue; + } + if (!strcmp(*av, "sock")) { + av++; + AN(*av); + barrier_sock(b, *av, vl); + continue; + } + if (!strcmp(*av, "sync")) { + barrier_sync(b, vl); + continue; + } + vtc_log(vl, 0, "Unknown barrier argument: %s", *av); + } + AZ(pthread_mutex_unlock(&b->mtx)); +} + +void +init_barrier(void) +{ + + AZ(pthread_mutex_init(&barrier_mtx, NULL)); +} diff --git a/bin/varnishtest/vtc_http.c b/bin/varnishtest/vtc_http.c index 5144d36..48759de 100644 --- a/bin/varnishtest/vtc_http.c +++ b/bin/varnishtest/vtc_http.c @@ -1378,6 +1378,7 @@ static const struct cmds http_cmds[] = { { "chunked", cmd_http_chunked }, { "chunkedlen", cmd_http_chunkedlen }, { "delay", cmd_delay }, + { "barrier", cmd_barrier }, { "sema", cmd_sema }, { "expect_close", cmd_http_expect_close }, { "close", cmd_http_close }, diff --git a/doc/graphviz/cache_fetch.svg b/doc/graphviz/cache_fetch.svg deleted file mode 100644 index ecd062a..0000000 --- a/doc/graphviz/cache_fetch.svg +++ /dev/null @@ -1,240 +0,0 @@ - - - - - diff --git a/doc/graphviz/cache_http1_fsm.svg b/doc/graphviz/cache_http1_fsm.svg deleted file mode 100644 index 65617c7..0000000 --- a/doc/graphviz/cache_http1_fsm.svg +++ /dev/null @@ -1,128 +0,0 @@ - - - - - diff --git a/doc/graphviz/cache_req_fsm.svg b/doc/graphviz/cache_req_fsm.svg deleted file mode 100644 index d9976c3..0000000 --- a/doc/graphviz/cache_req_fsm.svg +++ /dev/null @@ -1,438 +0,0 @@ - - - - - From lkarsten at varnish-software.com Tue Jun 14 09:19:08 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:19:08 +0200 Subject: [4.1] 4b48aaf Barriers can be cyclic but not like semaphores Message-ID: commit 4b48aaf9375cf494cb8e3d90076742fdee3dc9c8 Author: Dridi Boukelmoune Date: Wed Dec 23 08:27:14 2015 +0100 Barriers can be cyclic but not like semaphores This is useful in HTTP loops where it's impossible to predict and use an array of barriers. All cycles expect the same number of waiters, unlike semaphores. diff --git a/bin/varnishtest/tests/a00008.vtc b/bin/varnishtest/tests/a00008.vtc index 6fe35c9..dfa84d3 100644 --- a/bin/varnishtest/tests/a00008.vtc +++ b/bin/varnishtest/tests/a00008.vtc @@ -1,25 +1,30 @@ varnishtest "Barrier operations" -barrier b1 cond 4 -barrier b2 cond 4 +# bs -> server, bc -> client, bb -> both +barrier bs cond 4 +barrier bc cond 4 +barrier bb cond 4 -cyclic server s1 { rxreq - barrier b1 sync + barrier bs sync + barrier bb sync delay .9 txresp } -start server s2 { rxreq - barrier b1 sync + barrier bs sync + barrier bb sync delay .6 txresp } -start server s3 { rxreq - barrier b1 sync + barrier bs sync + barrier bb sync delay .2 txresp } -start @@ -28,25 +33,30 @@ client c1 -connect ${s1_sock} { delay .2 txreq rxresp - barrier b2 sync + barrier bc sync + barrier bb sync } -start client c2 -connect ${s2_sock} { delay .6 txreq rxresp - barrier b2 sync + barrier bc sync + barrier bb sync } -start client c3 -connect ${s3_sock} { delay .9 txreq rxresp - barrier b2 sync + barrier bc sync + barrier bb sync } -start # Wait for all servers to have received requests -barrier b1 sync +barrier bs sync +barrier bb sync # Wait for all clients to have received responses -barrier b2 sync +barrier bc sync +barrier bb sync diff --git a/bin/varnishtest/vtc_barrier.c b/bin/varnishtest/vtc_barrier.c index 448eb0d..b170c56 100644 --- a/bin/varnishtest/vtc_barrier.c +++ b/bin/varnishtest/vtc_barrier.c @@ -52,6 +52,7 @@ struct barrier { unsigned waiters; unsigned expected; + unsigned cyclic; enum barrier_e type; }; @@ -122,6 +123,23 @@ barrier_sock(struct barrier *b, const char *av, struct vtclog *vl) INCOMPL(); } +static void +barrier_cyclic(struct barrier *b, struct vtclog *vl) +{ + + CHECK_OBJ_NOTNULL(b, BARRIER_MAGIC); + + if (b->type == BARRIER_NONE) + vtc_log(vl, 0, + "Barrier(%s) use error: not initialized", b->name); + + if (b->waiters != 0) + vtc_log(vl, 0, + "Barrier(%s) use error: already in use", b->name); + + b->cyclic = 1; +} + /********************************************************************** * Sync a barrier */ @@ -148,6 +166,9 @@ barrier_cond_sync(struct barrier *b, struct vtclog *vl) b->name, b->waiters, b->expected); AZ(pthread_cond_wait(&b->cond, &b->mtx)); } + + if (b->cyclic) + b->waiters = 0; } static void @@ -189,7 +210,10 @@ cmd_barrier(CMD_ARGS) VTAILQ_FOREACH_SAFE(b, &barriers, list, b2) { AZ(pthread_mutex_lock(&b->mtx)); assert(b->type != BARRIER_NONE); - assert(b->waiters == b->expected); + if (b->cyclic) + AZ(b->waiters); + else + assert(b->waiters == b->expected); AZ(pthread_mutex_unlock(&b->mtx)); } AZ(pthread_mutex_unlock(&barrier_mtx)); @@ -226,6 +250,10 @@ cmd_barrier(CMD_ARGS) barrier_sync(b, vl); continue; } + if (!strcmp(*av, "-cyclic")) { + barrier_cyclic(b, vl); + continue; + } vtc_log(vl, 0, "Unknown barrier argument: %s", *av); } AZ(pthread_mutex_unlock(&b->mtx)); From lkarsten at varnish-software.com Tue Jun 14 09:19:08 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:19:08 +0200 Subject: [4.1] 5fd309e Barriers can only be created on the top thread Message-ID: commit 5fd309eccc40eeef8df0d4ca798284ae536d89d8 Author: Dridi Boukelmoune Date: Wed Dec 23 16:24:02 2015 +0100 Barriers can only be created on the top thread This is similar ASSERT_CLI, in order to make socket barriers creation thread-safe. diff --git a/bin/varnishtest/vtc_barrier.c b/bin/varnishtest/vtc_barrier.c index b170c56..ba80cfb 100644 --- a/bin/varnishtest/vtc_barrier.c +++ b/bin/varnishtest/vtc_barrier.c @@ -70,6 +70,9 @@ barrier_new(char *name, struct vtclog *vl) AN(name); if (*name != 'b') vtc_log(vl, 0, "Barrier name must start with 'b' (%s)", name); + if (pthread_self() != vtc_thread) + vtc_log(vl, 0, + "Barrier %s can only be created on the top thread", name); REPLACE(b->name, name); AZ(pthread_mutex_init(&b->mtx, NULL)); From lkarsten at varnish-software.com Tue Jun 14 09:19:08 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:19:08 +0200 Subject: [4.1] 6a709f8 Implement socket barriers (shared between processes) Message-ID: commit 6a709f879f17384bbd527c095dedba8767461fd0 Author: Dridi Boukelmoune Date: Wed Dec 23 17:29:24 2015 +0100 Implement socket barriers (shared between processes) When varnishtest creates a socket barrier, it will bind a socket and listen to incoming connections. Once the number of expected connections is open, connections are closed. Barrier users only need to connect to the socket, read "nothing" and block until the connection is closed. It allows virtually any process to sync with varnishtest. The barrier will provide macros with its socket information. diff --git a/bin/varnishtest/tests/a00013.vtc b/bin/varnishtest/tests/a00013.vtc new file mode 100644 index 0000000..91bfce0 --- /dev/null +++ b/bin/varnishtest/tests/a00013.vtc @@ -0,0 +1,64 @@ +varnishtest "Barrier operations" + +# same as a00008.vtc, with socket barriers instead + +# bs -> server, bc -> client, bb -> both +barrier bs sock 4 +barrier bc sock 4 +barrier bb sock 4 -cyclic + +server s1 { + rxreq + barrier bs sync + barrier bb sync + delay .9 + txresp +} -start + +server s2 { + rxreq + barrier bs sync + barrier bb sync + delay .6 + txresp +} -start + +server s3 { + rxreq + barrier bs sync + barrier bb sync + delay .2 + txresp +} -start + +client c1 -connect ${s1_sock} { + delay .2 + txreq + rxresp + barrier bc sync + barrier bb sync +} -start + +client c2 -connect ${s2_sock} { + delay .6 + txreq + rxresp + barrier bc sync + barrier bb sync +} -start + +client c3 -connect ${s3_sock} { + delay .9 + txreq + rxresp + barrier bc sync + barrier bb sync +} -start + +# Wait for all servers to have received requests +barrier bs sync +barrier bb sync + +# Wait for all clients to have received responses +barrier bc sync +barrier bb sync diff --git a/bin/varnishtest/vtc_barrier.c b/bin/varnishtest/vtc_barrier.c index ba80cfb..47ac4dc 100644 --- a/bin/varnishtest/vtc_barrier.c +++ b/bin/varnishtest/vtc_barrier.c @@ -28,13 +28,19 @@ #include "config.h" +#include #include #include #include #include #include +#include +#include +#include + #include "vtc.h" +#include "vtcp.h" enum barrier_e { BARRIER_NONE = 0, @@ -55,6 +61,9 @@ struct barrier { unsigned cyclic; enum barrier_e type; + /* fields below are only for BARRIER_SOCK */ + pthread_t thread; + volatile unsigned active; }; static pthread_mutex_t barrier_mtx; @@ -116,6 +125,108 @@ barrier_cond(struct barrier *b, const char *av, struct vtclog *vl) b->type = BARRIER_COND; } +static void * +barrier_sock_thread(void *priv) +{ + struct barrier *b; + struct vtclog *vl; + struct timeval tmo; + const char *err; + char abuf[16], pbuf[6]; + int i, sock, *conns; + fd_set rfds; + + CAST_OBJ_NOTNULL(b, priv, BARRIER_MAGIC); + assert(b->type == BARRIER_SOCK); + + AZ(pthread_mutex_lock(&b->mtx)); + + vl = vtc_logopen(b->name); + AN(vl); + + sock = VTCP_listen_on("127.0.0.1:0", NULL, b->expected, &err); + if (sock < 0) { + pthread_cond_signal(&b->cond); + AZ(pthread_mutex_unlock(&b->mtx)); + vtc_log(vl, 0, "Barrier(%s) %s fails: %s (errno=%d)", + b->name, err, strerror(errno), errno); + } + assert(sock > 0); + (void)VTCP_nonblocking(sock); + VTCP_myname(sock, abuf, sizeof abuf, pbuf, sizeof pbuf); + + macro_def(vl, b->name, "addr", "%s", abuf); + macro_def(vl, b->name, "port", "%s", pbuf); + macro_def(vl, b->name, "sock", "%s:%s", abuf, pbuf); + + pthread_cond_signal(&b->cond); + AZ(pthread_mutex_unlock(&b->mtx)); + + conns = calloc(b->expected, sizeof *conns); + AN(conns); + + while (b->active) { + FD_ZERO(&rfds); + FD_SET(sock, &rfds); + + tmo.tv_sec = 1; + tmo.tv_usec = 0; + i = select(sock + 1, &rfds, NULL, NULL, &tmo); + if (i == 0) + continue; + if (i < 0) { + if (errno == EINTR) + continue; + AZ(close(sock)); + vtc_log(vl, 0, + "Barrier(%s) select fails: %s (errno=%d)", + b->name, strerror(errno), errno); + } + assert(i == 1); + assert(b->waiters <= b->expected); + if (b->waiters == b->expected) + vtc_log(vl, 0, + "Barrier(%s) use error: " + "more waiters than the %u expected", + b->name, b->expected); + + i = accept(sock, NULL, NULL); + if (i < 0) { + AZ(close(sock)); + vtc_log(vl, 0, + "Barrier(%s) accept fails: %s (errno=%d)", + b->name, strerror(errno), errno); + } + + /* NB. We don't keep track of the established connections, only + * that connections were made to the barrier's socket. + */ + conns[b->waiters] = i; + + if (++b->waiters < b->expected) { + vtc_log(vl, 4, "Barrier(%s) wait %u of %u", + b->name, b->waiters, b->expected); + continue; + } + + vtc_log(vl, 4, "Barrier(%s) wake %u", b->name, b->expected); + for (i = 0; i < b->expected; i++) + AZ(close(conns[i])); + + if (b->cyclic) + b->waiters = 0; + else + b->active = 0; + } + + macro_undef(vl, b->name, "addr"); + macro_undef(vl, b->name, "port"); + macro_undef(vl, b->name, "sock"); + AZ(close(sock)); + + return (NULL); +} + static void barrier_sock(struct barrier *b, const char *av, struct vtclog *vl) { @@ -123,7 +234,13 @@ barrier_sock(struct barrier *b, const char *av, struct vtclog *vl) CHECK_OBJ_NOTNULL(b, BARRIER_MAGIC); barrier_expect(b, av, vl); b->type = BARRIER_SOCK; - INCOMPL(); + b->active = 1; + + /* NB. We can use the BARRIER_COND's pthread_cond_t to wait until the + * socket is ready for convenience. + */ + AZ(pthread_create(&b->thread, NULL, barrier_sock_thread, b)); + AZ(pthread_cond_wait(&b->cond, &b->mtx)); } static void @@ -175,6 +292,46 @@ barrier_cond_sync(struct barrier *b, struct vtclog *vl) } static void +barrier_sock_sync(struct barrier *b, struct vtclog *vl) +{ + struct vsb *vsb; + const char *err; + char buf[32]; + int i, sock; + ssize_t sz; + + CHECK_OBJ_NOTNULL(b, BARRIER_MAGIC); + assert(b->type == BARRIER_SOCK); + + i = snprintf(buf, sizeof buf, "${%s_sock}", b->name); + assert(i > 0 && i < sizeof buf); + vsb = macro_expand(vl, buf); + vtc_log(vl, 4, "Barrier(%s) sync with socket", b->name); + + sock = VTCP_open(VSB_data(vsb), NULL, 0., &err); + if (sock < 0) + vtc_log(vl, 0, "Barrier(%s) connection failed: %s", + b->name, err); + + VSB_delete(vsb); + + /* emulate pthread_cond_wait's behavior */ + AZ(pthread_mutex_unlock(&b->mtx)); + sz = read(sock, buf, sizeof buf); /* XXX loop with timeout? */ + AZ(pthread_mutex_lock(&b->mtx)); + + i = errno; + AZ(close(sock)); + + if (sz < 0) + vtc_log(vl, 0, "Barrier(%s) connection failed: %s (errno=%d)", + b->name, strerror(i), i); + if (sz > 0) + vtc_log(vl, 0, "Barrier(%s) unexpected data (%ldB)", + b->name, sz); +} + +static void barrier_sync(struct barrier *b, struct vtclog *vl) { @@ -188,7 +345,7 @@ barrier_sync(struct barrier *b, struct vtclog *vl) barrier_cond_sync(b, vl); break; case BARRIER_SOCK: - INCOMPL(); + barrier_sock_sync(b, vl); break; default: WRONG("Wrong barrier type"); @@ -212,11 +369,20 @@ cmd_barrier(CMD_ARGS) /* Reset and free */ VTAILQ_FOREACH_SAFE(b, &barriers, list, b2) { AZ(pthread_mutex_lock(&b->mtx)); - assert(b->type != BARRIER_NONE); - if (b->cyclic) - AZ(b->waiters); - else - assert(b->waiters == b->expected); + switch (b->type) { + case BARRIER_COND: + if (b->cyclic) + AZ(b->waiters); + else + assert(b->waiters == b->expected); + break; + case BARRIER_SOCK: + b->active = 0; + AZ(pthread_join(b->thread, NULL)); + break; + default: + WRONG("Wrong barrier type"); + } AZ(pthread_mutex_unlock(&b->mtx)); } AZ(pthread_mutex_unlock(&barrier_mtx)); From lkarsten at varnish-software.com Tue Jun 14 09:19:08 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:19:08 +0200 Subject: [4.1] 93b77d9 Teach vmod-debug how to sync with varnishtest Message-ID: commit 93b77d9adaec55bc8c9edc8408f402fa2f769939 Author: Dridi Boukelmoune Date: Thu Dec 24 10:28:46 2015 +0100 Teach vmod-debug how to sync with varnishtest diff --git a/bin/varnishtest/tests/m00024.vtc b/bin/varnishtest/tests/m00024.vtc new file mode 100644 index 0000000..2124589 --- /dev/null +++ b/bin/varnishtest/tests/m00024.vtc @@ -0,0 +1,39 @@ +varnishtest "Test debug.barrier_sync" + +barrier b1 sock 2 +barrier b2 sock 2 + +server s1 { + rxreq + txresp +} -start + +varnish v1 -vcl+backend { + import debug; + + sub vcl_recv { + if (!debug.barrier_sync("${b1_sock}")) { + return (synth(400)); + } + } + + sub vcl_backend_response { + if (!debug.barrier_sync("${b2_sock}")) { + return (abandon); + } + } +} -start + +varnish v1 -cliok "param.set debug +syncvsl" + +client c1 { + txreq + rxresp + expect resp.status == 200 +} -start + +barrier b1 sync +delay 0.5 +barrier b2 sync + +client c1 -wait diff --git a/lib/libvmod_debug/vmod.vcc b/lib/libvmod_debug/vmod.vcc index ccf0d80..3ec9511 100644 --- a/lib/libvmod_debug/vmod.vcc +++ b/lib/libvmod_debug/vmod.vcc @@ -154,3 +154,7 @@ Hold a reference to the VCL when it goes cold for the given delay. $Function BOOL match_acl(ACL acl, IP ip) Perform an IP match against a named ACL. + +$Function BOOL barrier_sync(STRING) + +Synchronize with a varnishtest shared barrier. diff --git a/lib/libvmod_debug/vmod_debug.c b/lib/libvmod_debug/vmod_debug.c index 971e04c..40927a1 100644 --- a/lib/libvmod_debug/vmod_debug.c +++ b/lib/libvmod_debug/vmod_debug.c @@ -28,6 +28,7 @@ #include "config.h" +#include #include #include #include @@ -38,6 +39,7 @@ #include "vrt.h" #include "vsa.h" #include "vsb.h" +#include "vtcp.h" #include "vtim.h" #include "vcc_if.h" @@ -476,3 +478,34 @@ vmod_match_acl(VRT_CTX, VCL_ACL acl, VCL_IP ip) return (VRT_acl_match(ctx, acl, ip)); } + +VCL_BOOL +vmod_barrier_sync(VRT_CTX, VCL_STRING addr) +{ + const char *err; + char buf[32]; + int sock; + ssize_t sz; + + CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC); + AN(addr); + AN(*addr); + + VSLb(ctx->vsl, SLT_VCL_call, "barrier_sync(\"%s\")", addr); + sock = VTCP_open(addr, NULL, 0., &err); + if (sock < 0) { + VSLb(ctx->vsl, SLT_Error, "Barrier connection failed: %s", err); + return (0); + } + + sz = read(sock, buf, sizeof buf); + if (sz == 0) + return (1); + if (sz < 0) + VSLb(ctx->vsl, SLT_Error, + "Barrier connection failed: %s (errno=%d)", + strerror(errno), errno); + if (sz > 0) + VSLb(ctx->vsl, SLT_Error, "Barrier unexpected data (%ldB)", sz); + return (0); +} From lkarsten at varnish-software.com Tue Jun 14 09:19:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:19:09 +0200 Subject: [4.1] 78716e0 Close the socket Message-ID: commit 78716e084ef5d76cfa8285366c758bd08246396c Author: Dridi Boukelmoune Date: Thu Dec 24 12:41:57 2015 +0100 Close the socket diff --git a/lib/libvmod_debug/vmod_debug.c b/lib/libvmod_debug/vmod_debug.c index 40927a1..9f6358d 100644 --- a/lib/libvmod_debug/vmod_debug.c +++ b/lib/libvmod_debug/vmod_debug.c @@ -484,7 +484,7 @@ vmod_barrier_sync(VRT_CTX, VCL_STRING addr) { const char *err; char buf[32]; - int sock; + int sock, i; ssize_t sz; CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC); @@ -499,12 +499,13 @@ vmod_barrier_sync(VRT_CTX, VCL_STRING addr) } sz = read(sock, buf, sizeof buf); + i = errno; + AZ(close(sock)); if (sz == 0) return (1); if (sz < 0) VSLb(ctx->vsl, SLT_Error, - "Barrier connection failed: %s (errno=%d)", - strerror(errno), errno); + "Barrier connection failed: %s (errno=%d)", strerror(i), i); if (sz > 0) VSLb(ctx->vsl, SLT_Error, "Barrier unexpected data (%ldB)", sz); return (0); From lkarsten at varnish-software.com Tue Jun 14 09:19:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:19:09 +0200 Subject: [4.1] 49f2354 Add vtc barriers to changelog. Message-ID: commit 49f235404327e27005a8ac5cdf80e99633f8ba59 Author: Lasse Karstensen Date: Tue Jun 14 11:04:27 2016 +0200 Add vtc barriers to changelog. diff --git a/doc/changes.rst b/doc/changes.rst index 66165a6..e1ef37e 100644 --- a/doc/changes.rst +++ b/doc/changes.rst @@ -14,6 +14,9 @@ Changes since 4.1.2: * Add support for TCP Fastopen where available. Disabled by default. +* [varnishtest] New syncronization primitive barriers added, improving + coordination when test cases call external programs. + .. _1905: https://github.com/varnishcache/varnish-cache/pull/1905 Bugs fixed From lkarsten at varnish-software.com Tue Jun 14 09:19:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:19:09 +0200 Subject: [4.1] 10e4a12 Enable make clean target in man/ Message-ID: commit 10e4a12ad7df8ee277d662e59a5b2e1cf10f5991 Author: Lasse Karstensen Date: Tue Mar 29 15:50:51 2016 +0200 Enable make clean target in man/ diff --git a/man/Makefile.am b/man/Makefile.am index 35a8c52..0c93e1b 100644 --- a/man/Makefile.am +++ b/man/Makefile.am @@ -17,7 +17,7 @@ dist_man_MANS = \ vmod_directors.3 \ vmod_std.3 -MAINTAINERCLEANFILES = $(dist_man_MANS) +CLEANFILES = $(dist_man_MANS) RST2ANY_FLAGS = --halt=2 From lkarsten at varnish-software.com Tue Jun 14 09:19:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:19:09 +0200 Subject: [4.1] 4b586e6 Use the proper VSL record Message-ID: commit 4b586e6e8766c9fcc57f07970fb63c821aaacda2 Author: Dridi Boukelmoune Date: Tue Mar 29 18:51:38 2016 +0200 Use the proper VSL record diff --git a/lib/libvmod_debug/vmod_debug.c b/lib/libvmod_debug/vmod_debug.c index 9f6358d..2ebefd3 100644 --- a/lib/libvmod_debug/vmod_debug.c +++ b/lib/libvmod_debug/vmod_debug.c @@ -491,7 +491,7 @@ vmod_barrier_sync(VRT_CTX, VCL_STRING addr) AN(addr); AN(*addr); - VSLb(ctx->vsl, SLT_VCL_call, "barrier_sync(\"%s\")", addr); + VSLb(ctx->vsl, SLT_Debug, "barrier_sync(\"%s\")", addr); sock = VTCP_open(addr, NULL, 0., &err); if (sock < 0) { VSLb(ctx->vsl, SLT_Error, "Barrier connection failed: %s", err); From lkarsten at varnish-software.com Tue Jun 14 09:19:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:19:09 +0200 Subject: [4.1] addeabd Method names should start with a period Message-ID: commit addeabdb529da092d586fc5408a62127a7ac76f4 Author: Federico G. Schwindt Date: Tue Mar 29 22:18:10 2016 +0100 Method names should start with a period Raise a ParseError exception if not. Fixes #1885. diff --git a/lib/libvcc/vmodtool.py b/lib/libvcc/vmodtool.py index 95bf2d6..2f7253a 100755 --- a/lib/libvcc/vmodtool.py +++ b/lib/libvcc/vmodtool.py @@ -691,6 +691,8 @@ def parse_func(tl, rt_type=None, pobj=None): raise ParseError("Method name '%s' is illegal" % fname[1:]) elif not is_c_name(fname): raise ParseError("Function name '%s' is illegal" % fname) + elif pobj != None and fname[0] != ".": + raise ParseError("Method name '%s' must start with ." % fname) t = tl.get_token() if t.str != "(": From lkarsten at varnish-software.com Tue Jun 14 09:19:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:19:09 +0200 Subject: [4.1] 21ef123 Fix minor leak spotted by Coverity Scan Message-ID: commit 21ef1235cbdc6c8614483bb138d22b6338f9f79f Author: Dridi Boukelmoune Date: Thu Mar 31 08:00:18 2016 +0200 Fix minor leak spotted by Coverity Scan diff --git a/bin/varnishtest/vtc_barrier.c b/bin/varnishtest/vtc_barrier.c index 47ac4dc..ecd3ba5 100644 --- a/bin/varnishtest/vtc_barrier.c +++ b/bin/varnishtest/vtc_barrier.c @@ -223,6 +223,7 @@ barrier_sock_thread(void *priv) macro_undef(vl, b->name, "port"); macro_undef(vl, b->name, "sock"); AZ(close(sock)); + free(conns); return (NULL); } From lkarsten at varnish-software.com Tue Jun 14 09:19:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:19:09 +0200 Subject: [4.1] 1666af6 Turn false-positive INCOMPL macros into WRONG ones Message-ID: commit 1666af69b0781257f5613c3b545ee8d655bfb16b Author: Dridi Boukelmoune Date: Thu Mar 31 16:36:13 2016 +0200 Turn false-positive INCOMPL macros into WRONG ones It also applies for the 4.1 and 4.0 series. Refs #1890 diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c index 2540a74..d83f2eb 100644 --- a/bin/varnishd/cache/cache_req_fsm.c +++ b/bin/varnishd/cache/cache_req_fsm.c @@ -182,7 +182,7 @@ cnt_deliver(struct worker *wrk, struct req *req) req->req_step = R_STP_SYNTH; break; default: - INCOMPL(); + WRONG("Illegal return from vcl_deliver{}"); } return (REQ_FSM_MORE); @@ -471,7 +471,7 @@ cnt_lookup(struct worker *wrk, struct req *req) req->req_step = R_STP_PASS; break; default: - INCOMPL(); + WRONG("Illegal return from vcl_hash{}"); } /* Drop our object, we won't need it */ From lkarsten at varnish-software.com Tue Jun 14 09:19:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:19:09 +0200 Subject: [4.1] f99ec68 Fix varnishtest barrier error messages Message-ID: commit f99ec6844118844d505ff2c4a595882e75b5cfbb Author: Dridi Boukelmoune Date: Thu Mar 31 16:45:29 2016 +0200 Fix varnishtest barrier error messages Spotted by @fgsch diff --git a/bin/varnishtest/vtc_barrier.c b/bin/varnishtest/vtc_barrier.c index ecd3ba5..f0becfa 100644 --- a/bin/varnishtest/vtc_barrier.c +++ b/bin/varnishtest/vtc_barrier.c @@ -325,7 +325,7 @@ barrier_sock_sync(struct barrier *b, struct vtclog *vl) AZ(close(sock)); if (sz < 0) - vtc_log(vl, 0, "Barrier(%s) connection failed: %s (errno=%d)", + vtc_log(vl, 0, "Barrier(%s) read failed: %s (errno=%d)", b->name, strerror(i), i); if (sz > 0) vtc_log(vl, 0, "Barrier(%s) unexpected data (%ldB)", diff --git a/lib/libvmod_debug/vmod_debug.c b/lib/libvmod_debug/vmod_debug.c index 2ebefd3..f014715 100644 --- a/lib/libvmod_debug/vmod_debug.c +++ b/lib/libvmod_debug/vmod_debug.c @@ -505,7 +505,7 @@ vmod_barrier_sync(VRT_CTX, VCL_STRING addr) return (1); if (sz < 0) VSLb(ctx->vsl, SLT_Error, - "Barrier connection failed: %s (errno=%d)", strerror(i), i); + "Barrier read failed: %s (errno=%d)", strerror(i), i); if (sz > 0) VSLb(ctx->vsl, SLT_Error, "Barrier unexpected data (%ldB)", sz); return (0); From lkarsten at varnish-software.com Tue Jun 14 09:19:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:19:09 +0200 Subject: [4.1] cab1dd6 Wrong subroutine in error message Message-ID: commit cab1dd628d639dd3171a2e84a80cf1a0397b97c6 Author: Dridi Boukelmoune Date: Fri Apr 1 01:29:57 2016 +0200 Wrong subroutine in error message I shouldn't rely on auto-complete too much, especially for only two characters... diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c index d83f2eb..266b4ba 100644 --- a/bin/varnishd/cache/cache_req_fsm.c +++ b/bin/varnishd/cache/cache_req_fsm.c @@ -471,7 +471,7 @@ cnt_lookup(struct worker *wrk, struct req *req) req->req_step = R_STP_PASS; break; default: - WRONG("Illegal return from vcl_hash{}"); + WRONG("Illegal return from vcl_hit{}"); } /* Drop our object, we won't need it */ From lkarsten at varnish-software.com Tue Jun 14 09:19:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:19:09 +0200 Subject: [4.1] 0dff1ab Missing bracket Message-ID: commit 0dff1ab58b7a604325fcf6c037c9bb3d679968ff Author: Dridi Boukelmoune Date: Fri Apr 1 20:07:05 2016 +0200 Missing bracket diff --git a/bin/varnishd/cache/cache_ws.c b/bin/varnishd/cache/cache_ws.c index d18b5fc..19e8a5a 100644 --- a/bin/varnishd/cache/cache_ws.c +++ b/bin/varnishd/cache/cache_ws.c @@ -39,7 +39,7 @@ WS_Assert(const struct ws *ws) { CHECK_OBJ_NOTNULL(ws, WS_MAGIC); - DSL(DBG_WORKSPACE, 0, "WS(%p = (%s, %p %u %u %u)", + DSL(DBG_WORKSPACE, 0, "WS(%p) = (%s, %p %u %u %u)", ws, ws->id, ws->s, pdiff(ws->s, ws->f), ws->r == NULL ? 0 : pdiff(ws->f, ws->r), pdiff(ws->s, ws->e)); From lkarsten at varnish-software.com Tue Jun 14 09:19:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:19:09 +0200 Subject: [4.1] 05044be Update device detection VCL Message-ID: commit 05044beea595cbb058d318faae5d502d0390ee8e Author: Federico G. Schwindt Date: Thu Mar 31 14:57:51 2016 +0100 Update device detection VCL diff --git a/etc/devicedetect.vcl b/etc/devicedetect.vcl index d1a85d9..6c7e659 100644 --- a/etc/devicedetect.vcl +++ b/etc/devicedetect.vcl @@ -42,7 +42,7 @@ sub devicedetect { if (req.http.User-Agent ~ "$compatible; Googlebot-Mobile/2.1; \+http://www.google.com/bot.html$" || (req.http.User-Agent ~ "iPhone" && req.http.User-Agent ~ "\(compatible; Googlebot/2.1; \+http://www.google.com/bot.html")) { set req.http.X-UA-Device = "mobile-bot"; } - elsif (req.http.User-Agent ~ "(?i)(ads|google|bing|msn|yandex|baidu|ro|career|)bot" || + elsif (req.http.User-Agent ~ "(?i)(ads|google|bing|msn|yandex|baidu|ro|career|seznam|)bot" || req.http.User-Agent ~ "(?i)(baidu|jike|symantec)spider" || req.http.User-Agent ~ "(?i)scanner" || req.http.User-Agent ~ "(?i)(web)crawler") { @@ -61,7 +61,10 @@ sub devicedetect { elsif (req.http.User-Agent ~ "PlayBook; U; RIM Tablet") { set req.http.X-UA-Device = "tablet-rim"; } elsif (req.http.User-Agent ~ "hp-tablet.*TouchPad") { set req.http.X-UA-Device = "tablet-hp"; } elsif (req.http.User-Agent ~ "Kindle/3") { set req.http.X-UA-Device = "tablet-kindle"; } - elsif (req.http.User-Agent ~ "Touch.+Tablet PC") { set req.http.X-UA-Device = "tablet-microsoft"; } + elsif (req.http.User-Agent ~ "Touch.+Tablet PC" || + req.http.User-Agent ~ "Windows NT [0-9.]+; ARM;" ) { + set req.http.X-UA-Device = "tablet-microsoft"; + } elsif (req.http.User-Agent ~ "Mobile.+Firefox") { set req.http.X-UA-Device = "mobile-firefoxos"; } elsif (req.http.User-Agent ~ "^HTC" || req.http.User-Agent ~ "Fennec" || From lkarsten at varnish-software.com Tue Jun 14 09:19:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:19:09 +0200 Subject: [4.1] 6df5e61 Use correct printf spec for ssize_t Message-ID: commit 6df5e6149dcda40cef7385e0b81e4fd3d2b1f9e0 Author: Poul-Henning Kamp Date: Tue Apr 5 22:13:00 2016 +0000 Use correct printf spec for ssize_t diff --git a/lib/libvmod_debug/vmod_debug.c b/lib/libvmod_debug/vmod_debug.c index f014715..f5d5326 100644 --- a/lib/libvmod_debug/vmod_debug.c +++ b/lib/libvmod_debug/vmod_debug.c @@ -507,6 +507,6 @@ vmod_barrier_sync(VRT_CTX, VCL_STRING addr) VSLb(ctx->vsl, SLT_Error, "Barrier read failed: %s (errno=%d)", strerror(i), i); if (sz > 0) - VSLb(ctx->vsl, SLT_Error, "Barrier unexpected data (%ldB)", sz); + VSLb(ctx->vsl, SLT_Error, "Barrier unexpected data (%zdB)", sz); return (0); } From lkarsten at varnish-software.com Tue Jun 14 09:19:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:19:09 +0200 Subject: [4.1] 8285da0 Fix another size_t printf format Message-ID: commit 8285da0a45b6ae1d53b74e7563f6b6bb6c78bbbc Author: Poul-Henning Kamp Date: Tue Apr 5 22:20:31 2016 +0000 Fix another size_t printf format diff --git a/bin/varnishtest/vtc_barrier.c b/bin/varnishtest/vtc_barrier.c index f0becfa..a7bffc4 100644 --- a/bin/varnishtest/vtc_barrier.c +++ b/bin/varnishtest/vtc_barrier.c @@ -328,7 +328,7 @@ barrier_sock_sync(struct barrier *b, struct vtclog *vl) vtc_log(vl, 0, "Barrier(%s) read failed: %s (errno=%d)", b->name, strerror(i), i); if (sz > 0) - vtc_log(vl, 0, "Barrier(%s) unexpected data (%ldB)", + vtc_log(vl, 0, "Barrier(%s) unexpected data (%zdB)", b->name, sz); } From lkarsten at varnish-software.com Tue Jun 14 09:19:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:19:09 +0200 Subject: [4.1] 26743fd Sync with reality Message-ID: commit 26743fd13c7a847bc3387d58a3ff509847329536 Author: Dridi Boukelmoune Date: Thu Apr 7 10:25:32 2016 +0200 Sync with reality diff --git a/doc/sphinx/reference/varnishd.rst b/doc/sphinx/reference/varnishd.rst index 3372939..351ba15 100644 --- a/doc/sphinx/reference/varnishd.rst +++ b/doc/sphinx/reference/varnishd.rst @@ -315,7 +315,6 @@ Be aware that on 32 bit systems, certain default values are reduced relative to the values listed below, in order to conserve VM space: * workspace_client: 16k -* thread_pool_workspace: 16k * http_resp_size: 8k * http_req_size: 12k * gzip_stack_buffer: 4k From lkarsten at varnish-software.com Tue Jun 14 09:19:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:19:09 +0200 Subject: [4.1] edc2628 These programs should respect SIGHUP and die, rather pointlessly eat all CPU time. Message-ID: commit edc26280ee3222d8eda045b797c7f6b669602b06 Author: Poul-Henning Kamp Date: Fri Apr 8 22:16:34 2016 +0000 These programs should respect SIGHUP and die, rather pointlessly eat all CPU time. diff --git a/bin/varnishhist/varnishhist.c b/bin/varnishhist/varnishhist.c index 9eab9bf..f279dc3 100644 --- a/bin/varnishhist/varnishhist.c +++ b/bin/varnishhist/varnishhist.c @@ -294,7 +294,7 @@ accumulate(struct VSL_data *vsl, struct VSL_transaction * const pt[], static int __match_proto__(VUT_cb_f) sighup(void) { - return (1); + exit(1); } static void * diff --git a/bin/varnishtop/varnishtop.c b/bin/varnishtop/varnishtop.c index f56b9a7..9c555cc 100644 --- a/bin/varnishtop/varnishtop.c +++ b/bin/varnishtop/varnishtop.c @@ -183,7 +183,7 @@ accumulate(struct VSL_data *vsl, struct VSL_transaction * const pt[], static int __match_proto__(VUT_cb_f) sighup(void) { - return (1); + exit (1); } static void From lkarsten at varnish-software.com Tue Jun 14 09:19:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:19:09 +0200 Subject: [4.1] 0d95627 Adopt test case to 4.1. Message-ID: commit 0d956277147d0de9c4bff8cf54bd3d0da1535d30 Author: Lasse Karstensen Date: Tue Jun 14 11:18:43 2016 +0200 Adopt test case to 4.1. diff --git a/bin/varnishtest/tests/m00024.vtc b/bin/varnishtest/tests/m00024.vtc index 2124589..93c56ee 100644 --- a/bin/varnishtest/tests/m00024.vtc +++ b/bin/varnishtest/tests/m00024.vtc @@ -9,7 +9,7 @@ server s1 { } -start varnish v1 -vcl+backend { - import debug; + import ${vmod_debug}; sub vcl_recv { if (!debug.barrier_sync("${b1_sock}")) { From lkarsten at varnish-software.com Tue Jun 14 09:42:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:42:09 +0200 Subject: [4.1] fb375e9 Do not add empty strings to the cli history Message-ID: commit fb375e9bdd25c60c5900e0b262ff452fd95e4317 Author: Federico G. Schwindt Date: Tue Apr 19 13:16:23 2016 +0100 Do not add empty strings to the cli history diff --git a/bin/varnishadm/varnishadm.c b/bin/varnishadm/varnishadm.c index 621d696..e426f0b 100644 --- a/bin/varnishadm/varnishadm.c +++ b/bin/varnishadm/varnishadm.c @@ -190,7 +190,8 @@ static void send_line(char *l) if (l) { cli_write(_line_sock, l); cli_write(_line_sock, "\n"); - add_history(l); + if (*l) + add_history(l); rl_callback_handler_install("varnish> ", send_line); } else { RL_EXIT(0); From lkarsten at varnish-software.com Tue Jun 14 09:42:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:42:09 +0200 Subject: [4.1] 2258f10 Also remove directory if compilation failed Message-ID: commit 2258f107a485ff2fc2f7eae2a9c6104d493b2c6a Author: Federico G. Schwindt Date: Tue Apr 19 23:21:45 2016 +0100 Also remove directory if compilation failed diff --git a/bin/varnishd/mgt/mgt_vcc.c b/bin/varnishd/mgt/mgt_vcc.c index 0856123..7a88b74 100644 --- a/bin/varnishd/mgt/mgt_vcc.c +++ b/bin/varnishd/mgt/mgt_vcc.c @@ -299,8 +299,7 @@ mgt_VccCompile(struct cli *cli, const char *vclname, const char *vclsrc, if (status || C_flag) { (void)unlink(vp.libfile); free(vp.libfile); - if (C_flag) - AZ(rmdir(vp.dir)); + AZ(rmdir(vp.dir)); free(vp.dir); if (status) { VCLI_Out(cli, "VCL compilation failed"); From lkarsten at varnish-software.com Tue Jun 14 09:42:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:42:09 +0200 Subject: [4.1] 4f6b1ea dont assert on a failing rmdir Message-ID: commit 4f6b1eab5056037451b65db9a2c4363815c82fbc Author: Nils Goroll Date: Wed Apr 20 15:06:44 2016 +0200 dont assert on a failing rmdir We may not have permission to unlink vp.dir $ /etc/init.d/varnish_master restart Stopping Varnish Cache: [FAILED] Starting Varnish Cache: Assert error in mgt_VccCompile(), mgt/mgt_vcc.c line 302: Condition((rmdir(vp.dir)) == 0) not true. errno = 13 (Permission denied) diff --git a/bin/varnishd/mgt/mgt_vcc.c b/bin/varnishd/mgt/mgt_vcc.c index 7a88b74..eef0340 100644 --- a/bin/varnishd/mgt/mgt_vcc.c +++ b/bin/varnishd/mgt/mgt_vcc.c @@ -299,7 +299,7 @@ mgt_VccCompile(struct cli *cli, const char *vclname, const char *vclsrc, if (status || C_flag) { (void)unlink(vp.libfile); free(vp.libfile); - AZ(rmdir(vp.dir)); + (void)rmdir(vp.dir); free(vp.dir); if (status) { VCLI_Out(cli, "VCL compilation failed"); From lkarsten at varnish-software.com Tue Jun 14 09:42:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:42:09 +0200 Subject: [4.1] 4edc05c trivial comment fix Message-ID: commit 4edc05cc2a96338ef95de6e25c3eaff1005d50e5 Author: Nils Goroll Date: Mon Apr 25 16:24:36 2016 +0200 trivial comment fix diff --git a/bin/varnishd/cache/cache_ban_build.c b/bin/varnishd/cache/cache_ban_build.c index b3bda03..6fcbab8 100644 --- a/bin/varnishd/cache/cache_ban_build.c +++ b/bin/varnishd/cache/cache_ban_build.c @@ -48,7 +48,7 @@ struct ban_proto { }; /*-------------------------------------------------------------------- - * Variables we can purge on + * Variables we can ban on */ static const struct pvar { From lkarsten at varnish-software.com Tue Jun 14 09:42:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:42:09 +0200 Subject: [4.1] 9243971 Remove ancient outdated comment Message-ID: commit 924397136702b6306f3feae553e505fac6673672 Author: Dridi Boukelmoune Date: Thu Apr 28 16:09:21 2016 +0200 Remove ancient outdated comment diff --git a/bin/Makefile.am b/bin/Makefile.am index 157fbc8..b35764d 100644 --- a/bin/Makefile.am +++ b/bin/Makefile.am @@ -1,7 +1,5 @@ # -# Disabling building of the tools while api is in flux - SUBDIRS = \ varnishadm \ varnishd \ From lkarsten at varnish-software.com Tue Jun 14 09:42:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:42:09 +0200 Subject: [4.1] edf50ff Add another new output formatter. Fix typo. Message-ID: commit edf50ffcd65b9d4ae514817fcb328600861a0a12 Author: Lasse Karstensen Date: Tue Jun 14 11:24:42 2016 +0200 Add another new output formatter. Fix typo. diff --git a/doc/changes.rst b/doc/changes.rst index e1ef37e..6c2536b 100644 --- a/doc/changes.rst +++ b/doc/changes.rst @@ -6,9 +6,9 @@ Changes since 4.1.2: * varnishncsa can now access and log backend requests. (PR #1905) -* [varnishncsa] New formatter %{VSL:Tag}x. +* [varnishncsa] New output formatters %{Varnish:vxid}x and %{VSL:Tag}x. -* [varnishlog] Added log tag BacakendStart on backend transactions. +* [varnishlog] Added log tag BackendStart on backend transactions. * On SmartOS, use ports instead of epoll by default. From lkarsten at varnish-software.com Tue Jun 14 09:42:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:42:09 +0200 Subject: [4.1] f73ebf4 Minor fixes Message-ID: commit f73ebf4dfb0f7c6d100602393953f9fee018b3c3 Author: Federico G. Schwindt Date: Tue May 3 16:01:24 2016 +0100 Minor fixes diff --git a/bin/varnishncsa/varnishncsa.c b/bin/varnishncsa/varnishncsa.c index f15aa69..a5e1485 100644 --- a/bin/varnishncsa/varnishncsa.c +++ b/bin/varnishncsa/varnishncsa.c @@ -646,7 +646,7 @@ parse_x_format(char *buf) VUT_Error(1, "Syntax error: VSL:%s]", buf); if (i <= 0) VUT_Error(1, - "Syntax error. Field specifyer must be" + "Syntax error. Field specifier must be" " positive: %s]", buf); if (i > INT_MAX) { diff --git a/doc/sphinx/reference/varnishncsa.rst b/doc/sphinx/reference/varnishncsa.rst index 5d01d9a..94730a8 100644 --- a/doc/sphinx/reference/varnishncsa.rst +++ b/doc/sphinx/reference/varnishncsa.rst @@ -54,8 +54,7 @@ Client requests that results in a pipe (ie. return(pipe) in vcl), will not generate logging in backend mode. This is because varnish is not generating requests, but blindly passes on bytes in both directions. However, a varnishncsa instance running in normal mode can see this -case by using the formatter %{Varnish:handling}x, which will be "pipe" -(without the quotes). +case by using the formatter %{Varnish:handling}x, which will be 'pipe'. In backend mode, some of the fields in the format string get different meanings. Most notably, the byte counting formatters (%b, %I, %O) @@ -67,10 +66,8 @@ backend mode, and one in client mode, logging to different files. FORMAT ====== -Specify the log format used. If no format is specified the default log -format is used. - -The default log format is:: +Specify the log format to use. If no format is specified the default log +format is used:: %h %l %u %t "%r" %s %b "%{Referer}i" "%{User-agent}i" @@ -104,10 +101,10 @@ Supported formatters are: The contents of request header X. %l - Remote logname (always '-') + Remote logname. Always '-'. %m - Request method. Defaults to '-' if not known. + Request method. Defaults to '-' if not known. %{X}o The contents of response header X. @@ -117,7 +114,7 @@ Supported formatters are: bytes received from the backend. %q - The query string, if no query string exists, an empty string. + The query string. Defaults to an empty string if not present. %r The first line of the request. Synthesized from other fields, so it @@ -132,7 +129,6 @@ Supported formatters are: date/time format. In backend mode, time when the request was sent. %{X}t - In client mode, time when the request was received, in the format specified by X. In backend mode, time when the request was sent. The time specification format is the same as for strftime(3). @@ -143,11 +139,11 @@ Supported formatters are: been received. %U - The request URL without any query string. Defaults to '-' if not + The request URL without the query string. Defaults to '-' if not known. %u - Remote user from auth + Remote user from auth. %{X}x Extended variables. Supported variables are: @@ -166,9 +162,9 @@ Supported formatters are: pass, pipe or synth. Varnish:side - Backend or client side. One of two values, "b" or "c" (without the - quotes), depending on where the request was made. In pure backend - or client mode, this field will be constant. + Backend or client side. One of two values, 'b' or 'c', depending + on where the request was made. In pure backend or client mode, + this field will be constant. Varnish:vxid The VXID of the varnish transaction. @@ -177,13 +173,10 @@ Supported formatters are: Output value set by std.log("key:value") in VCL. VSL:tag or VSL:tag[field] - The value of the varnishlog entry with the given tag. If field is - specified, only the selected part is shown. Defaults to "-" - (without the quotes) when the key is not seen, or when the field - is out of bounds. If a key appears several times in a single - transaction, only the first occurrence is used. Example: The - formatter %{VSL:Begin[2]}x will print the second field of the - Begin tag, which is the VXID of the parent transaction. + The value of the VSL entry for the given tag. If field is specified, + only the selected part is shown. Defaults to '-' when the tag is not + seen, or when the field is out of bounds. If a tag appears several + times in a single transaction, only the first occurrence is used. SIGNALS ======= @@ -194,12 +187,21 @@ SIGHUP SIGUSR1 Flush any outstanding transactions +EXAMPLE +======= + +Log the second field of the Begin tag, corresponding to the VXID of the +parent transaction:: + + varnishncsa -F "%{VSL:Begin[2]}x" + SEE ALSO ======== :ref:`varnishd(1)` :ref:`varnishlog(1)` :ref:`varnishstat(1)` +:ref:`vsl(7)` HISTORY ======= From lkarsten at varnish-software.com Tue Jun 14 09:42:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:42:09 +0200 Subject: [4.1] c77a373 Fix distcheck on FreeBSD and possible other BSDs Message-ID: commit c77a37376df30c8b1c66ee78c51ba55cf1c3dd16 Author: Federico G. Schwindt Date: Wed May 4 06:59:56 2016 +0100 Fix distcheck on FreeBSD and possible other BSDs diff --git a/doc/sphinx/Makefile.am b/doc/sphinx/Makefile.am index e47a60d..41c02a3 100644 --- a/doc/sphinx/Makefile.am +++ b/doc/sphinx/Makefile.am @@ -174,3 +174,5 @@ BUILT_SOURCES += reference/vmod_directors.generated.rst EXTRA_DIST += $(BUILT_SOURCES) MAINTAINERCLEANFILES = $(EXTRA_DIST) + +.NOPATH: $(BUILT_SOURCES) diff --git a/lib/libvarnishapi/Makefile.am b/lib/libvarnishapi/Makefile.am index c7fb0e4..dc4252b 100644 --- a/lib/libvarnishapi/Makefile.am +++ b/lib/libvarnishapi/Makefile.am @@ -103,3 +103,5 @@ vsl_glob_test_SOURCES = \ vsl_glob_test_LDADD = @PCRE_LIBS@ ${RT_LIBS} ${LIBM} libvarnishapi.la vsl_glob_test_CFLAGS = -I$(top_srcdir)/include + +.NOPATH: $(BUILT_SOURCES) diff --git a/man/Makefile.am b/man/Makefile.am index 0c93e1b..536d7a2 100644 --- a/man/Makefile.am +++ b/man/Makefile.am @@ -82,3 +82,5 @@ vmod_std.3: $(top_builddir)/lib/libvmod_std/vmod_std.man.rst vmod_directors.3: $(top_builddir)/lib/libvmod_directors/vmod_directors.man.rst ${RST2MAN} $(RST2ANY_FLAGS) $? $@ + +.NOPATH: $(dist_man_MANS) From lkarsten at varnish-software.com Tue Jun 14 09:42:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:42:09 +0200 Subject: [4.1] 2e624aa Add a bunch of protective asserts on vcl->conf, to help flush out a family of tickets related to it (#1933 #1920 #1898 (?) #1888 and #1865 (?)) Message-ID: commit 2e624aab621f6a9f074e604f55568d5307fbc719 Author: Poul-Henning Kamp Date: Wed May 4 08:05:18 2016 +0000 Add a bunch of protective asserts on vcl->conf, to help flush out a family of tickets related to it (#1933 #1920 #1898 (?) #1888 and #1865 (?)) diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c index e167db3..8803f35 100644 --- a/bin/varnishd/cache/cache_vcl.c +++ b/bin/varnishd/cache/cache_vcl.c @@ -99,12 +99,16 @@ VCL_Panic(struct vsb *vsb, const struct vcl *vcl) VSB_printf(vsb, "vcl = {\n"); VSB_indent(vsb, 2); VSB_printf(vsb, "temp = %s\n", vcl->temp); - VSB_printf(vsb, "srcname = {\n"); - VSB_indent(vsb, 2); - for (i = 0; i < vcl->conf->nsrc; ++i) - VSB_printf(vsb, "\"%s\",\n", vcl->conf->srcname[i]); - VSB_indent(vsb, -2); - VSB_printf(vsb, "},\n"); + if (vcl->conf == NULL) { + VSB_printf(vsb, "conf = NULL\n"); + } else { + VSB_printf(vsb, "srcname = {\n"); + VSB_indent(vsb, 2); + for (i = 0; i < vcl->conf->nsrc; ++i) + VSB_printf(vsb, "\"%s\",\n", vcl->conf->srcname[i]); + VSB_indent(vsb, -2); + VSB_printf(vsb, "},\n"); + } VSB_indent(vsb, -2); VSB_printf(vsb, "},\n"); } @@ -350,6 +354,7 @@ VCL_DefaultDirector(const struct vcl *vcl) { CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC); + CHECK_OBJ_NOTNULL(vcl->conf, VCL_CONF_MAGIC); return (*vcl->conf->default_director); } @@ -366,6 +371,7 @@ VCL_DefaultProbe(const struct vcl *vcl) { CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC); + CHECK_OBJ_NOTNULL(vcl->conf, VCL_CONF_MAGIC); return (vcl->conf->default_probe); } @@ -377,6 +383,7 @@ VRT_count(VRT_CTX, unsigned u) CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC); CHECK_OBJ_NOTNULL(ctx->vcl, VCL_MAGIC); + CHECK_OBJ_NOTNULL(ctx->vcl->conf, VCL_CONF_MAGIC); assert(u < ctx->vcl->conf->nref); if (ctx->vsl != NULL) VSLb(ctx->vsl, SLT_VCL_trace, "%u %u.%u", u, @@ -460,6 +467,8 @@ vcl_setup_event(VRT_CTX, enum vcl_event_e ev) { CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC); + CHECK_OBJ_NOTNULL(ctx->vcl, VCL_MAGIC); + CHECK_OBJ_NOTNULL(ctx->vcl->conf, VCL_CONF_MAGIC); AN(ctx->handling); AN(ctx->vcl); AN(ctx->msg); @@ -473,6 +482,8 @@ vcl_failsafe_event(VRT_CTX, enum vcl_event_e ev) { CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC); + CHECK_OBJ_NOTNULL(ctx->vcl, VCL_MAGIC); + CHECK_OBJ_NOTNULL(ctx->vcl->conf, VCL_CONF_MAGIC); AN(ctx->handling); AN(ctx->vcl); assert(ev == VCL_EVENT_COLD || ev == VCL_EVENT_DISCARD || @@ -507,6 +518,8 @@ vcl_set_state(VRT_CTX, const char *state) ASSERT_CLI(); CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC); + CHECK_OBJ_NOTNULL(ctx->vcl, VCL_MAGIC); + CHECK_OBJ_NOTNULL(ctx->vcl->conf, VCL_CONF_MAGIC); AN(ctx->handling); AN(ctx->vcl); AN(state); @@ -563,6 +576,9 @@ vcl_cancel_load(VRT_CTX, struct cli *cli, const char *name, const char *step) { struct vcl *vcl = ctx->vcl; + CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC); + CHECK_OBJ_NOTNULL(vcl->conf, VCL_CONF_MAGIC); + AZ(VSB_finish(ctx->msg)); VCLI_SetResult(cli, CLIS_CANT); VCLI_Out(cli, "VCL \"%s\" Failed %s", name, step); @@ -831,6 +847,8 @@ ccf_config_show(struct cli *cli, const char * const *av, void *priv) VCLI_SetResult(cli, CLIS_PARAM); return; } + CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC); + CHECK_OBJ_NOTNULL(vcl->conf, VCL_CONF_MAGIC); if (verbose) { for (i = 0; i < vcl->conf->nsrc; i++) VCLI_Out(cli, "// VCL.SHOW %d %zd %s\n%s\n", From lkarsten at varnish-software.com Tue Jun 14 09:42:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:42:09 +0200 Subject: [4.1] 7f77962 Insist that tests start out with the magic string "varnishtest[:whitespace:]" Message-ID: commit 7f779623e1fb6cca610fd93641707e26582e20c1 Author: Poul-Henning Kamp Date: Wed May 4 13:15:34 2016 +0000 Insist that tests start out with the magic string "varnishtest[:whitespace:]" diff --git a/bin/varnishtest/vtc_main.c b/bin/varnishtest/vtc_main.c index 601f245..1db058c 100644 --- a/bin/varnishtest/vtc_main.c +++ b/bin/varnishtest/vtc_main.c @@ -34,6 +34,7 @@ #include #include +#include #include #include #include @@ -529,6 +530,13 @@ main(int argc, char * const *argv) continue; exit(2); } + if (strncmp(p, "varnishtest", 11) || !isspace(p[11])) { + fprintf(stderr, "File \"%s\" doesn't start with 'varnishtest'\n", + *argv); + if (vtc_continue) + continue; + exit(2); + } ALLOC_OBJ(tp, TST_MAGIC); AN(tp); tp->filename = *argv; From lkarsten at varnish-software.com Tue Jun 14 09:42:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:42:09 +0200 Subject: [4.1] 47a1ef0 Remove initial line feed. Message-ID: commit 47a1ef0632b7e42cf28fb88159e973fdca25467c Author: Lasse Karstensen Date: Tue Jun 14 11:32:32 2016 +0200 Remove initial line feed. diff --git a/bin/varnishtest/tests/r01941.vtc b/bin/varnishtest/tests/r01941.vtc index 2531ded..5d7441e 100644 --- a/bin/varnishtest/tests/r01941.vtc +++ b/bin/varnishtest/tests/r01941.vtc @@ -1,4 +1,3 @@ - varnishtest "ESI memory should only be fully allocated" # In this test case, the cache is almost filled. Then we force the From lkarsten at varnish-software.com Tue Jun 14 09:42:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:42:09 +0200 Subject: [4.1] 3004602 Remove superflous calls to WS_Assert() Message-ID: commit 3004602c4bd4849ddc30a03743e56520afd9c98e Author: Federico G. Schwindt Date: Wed Jan 13 19:21:28 2016 +0000 Remove superflous calls to WS_Assert() This was already called and has not been any changes since then. diff --git a/bin/varnishd/cache/cache_ws.c b/bin/varnishd/cache/cache_ws.c index 19e8a5a..c105119 100644 --- a/bin/varnishd/cache/cache_ws.c +++ b/bin/varnishd/cache/cache_ws.c @@ -134,7 +134,6 @@ WS_Alloc(struct ws *ws, unsigned bytes) assert(ws->r == NULL); if (ws->f + bytes > ws->e) { WS_MarkOverflow(ws); - WS_Assert(ws); return(NULL); } r = ws->f; @@ -160,7 +159,6 @@ WS_Copy(struct ws *ws, const void *str, int len) bytes = PRNDUP((unsigned)len); if (ws->f + bytes > ws->e) { WS_MarkOverflow(ws); - WS_Assert(ws); return(NULL); } r = ws->f; From lkarsten at varnish-software.com Tue Jun 14 09:42:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:42:09 +0200 Subject: [4.1] 9b2c32f Mostly whitespace OCD Message-ID: commit 9b2c32f9f04a686025804f5dfcd3b2ea598ace68 Author: Federico G. Schwindt Date: Thu May 5 16:19:36 2016 +0100 Mostly whitespace OCD Reduces my stash list to 62 entries. Conflicts: doc/sphinx/dev-guide/index.rst diff --git a/bin/varnishd/cache/cache_ws.c b/bin/varnishd/cache/cache_ws.c index c105119..e8f9b7f 100644 --- a/bin/varnishd/cache/cache_ws.c +++ b/bin/varnishd/cache/cache_ws.c @@ -134,7 +134,7 @@ WS_Alloc(struct ws *ws, unsigned bytes) assert(ws->r == NULL); if (ws->f + bytes > ws->e) { WS_MarkOverflow(ws); - return(NULL); + return (NULL); } r = ws->f; ws->f += bytes; @@ -159,7 +159,7 @@ WS_Copy(struct ws *ws, const void *str, int len) bytes = PRNDUP((unsigned)len); if (ws->f + bytes > ws->e) { WS_MarkOverflow(ws); - return(NULL); + return (NULL); } r = ws->f; ws->f += bytes; diff --git a/bin/varnishncsa/varnishncsa.c b/bin/varnishncsa/varnishncsa.c index a5e1485..bf41276 100644 --- a/bin/varnishncsa/varnishncsa.c +++ b/bin/varnishncsa/varnishncsa.c @@ -629,11 +629,10 @@ parse_x_format(char *buf) if (!strncmp(buf, "VSL:", 4)) { buf += 4; e = buf; - while(*e != '\0') + while (*e != '\0') e++; - if (e == buf) { + if (e == buf) VUT_Error(1, "Missing tag in VSL:"); - } if (e[-1] == ']') { r = e - 1; while (r > buf && *r != '[') diff --git a/lib/libvmod_debug/vmod_debug.c b/lib/libvmod_debug/vmod_debug.c index f5d5326..f16953b 100644 --- a/lib/libvmod_debug/vmod_debug.c +++ b/lib/libvmod_debug/vmod_debug.c @@ -191,7 +191,7 @@ vmod_argtest(VRT_CTX, VCL_STRING one, VCL_REAL two, VCL_STRING three, char buf[100]; bprintf(buf, "%s %g %s %s", one, two, three, comma); - return WS_Copy(ctx->ws, buf, -1); + return (WS_Copy(ctx->ws, buf, -1)); } VCL_INT @@ -350,10 +350,10 @@ event_function(VRT_CTX, struct vmod_priv *priv, enum vcl_event_e e) { switch (e) { - case VCL_EVENT_LOAD: return event_load(ctx, priv); - case VCL_EVENT_WARM: return event_warm(ctx, priv); - case VCL_EVENT_COLD: return event_cold(ctx, priv); - default: return (0); + case VCL_EVENT_LOAD: return event_load(ctx, priv); + case VCL_EVENT_WARM: return event_warm(ctx, priv); + case VCL_EVENT_COLD: return event_cold(ctx, priv); + default: return (0); } } @@ -365,16 +365,18 @@ vmod_sleep(VRT_CTX, VCL_DURATION t) VTIM_sleep(t); } -static struct ws *wsfind(VRT_CTX, VCL_ENUM which) { - if (!strcmp(which, "client")) { - return ctx->ws; - } else if (!strcmp(which, "backend")) { - return ctx->bo->ws; - } else if (!strcmp(which, "session")) { - return ctx->req->sp->ws; - } else if (!strcmp(which, "thread")) { - return ctx->req->wrk->aws; - } else +static struct ws * +wsfind(VRT_CTX, VCL_ENUM which) +{ + if (!strcmp(which, "client")) + return (ctx->ws); + else if (!strcmp(which, "backend")) + return (ctx->bo->ws); + else if (!strcmp(which, "session")) + return (ctx->req->sp->ws); + else if (!strcmp(which, "thread")) + return (ctx->req->wrk->aws); + else WRONG("No such workspace."); } @@ -423,6 +425,7 @@ vmod_workspace_overflowed(VRT_CTX, VCL_ENUM which) } static char *debug_ws_snap; + void vmod_workspace_snap(VRT_CTX, VCL_ENUM which) { From lkarsten at varnish-software.com Tue Jun 14 09:42:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:42:09 +0200 Subject: [4.1] a4f4d44 dont panic on pthread_cond_timedwait being interrupted Message-ID: commit a4f4d44235caf03e221a1b692b227ff06438aa21 Author: Nils Goroll Date: Mon May 9 10:18:32 2016 +0200 dont panic on pthread_cond_timedwait being interrupted As discussed with phk: Callers of Lck_CondWait with timeout should be prepared for early wakeups. diff --git a/bin/varnishd/cache/cache_lck.c b/bin/varnishd/cache/cache_lck.c index b702c2f..90190dc 100644 --- a/bin/varnishd/cache/cache_lck.c +++ b/bin/varnishd/cache/cache_lck.c @@ -198,7 +198,9 @@ Lck_CondWait(pthread_cond_t *cond, struct lock *lck, double when) ts.tv_nsec = (long)(modf(when, &t) * 1e9); ts.tv_sec = (long)t; retval = pthread_cond_timedwait(cond, &ilck->mtx, &ts); - assert(retval == 0 || retval == ETIMEDOUT); + assert(retval == 0 || + retval == ETIMEDOUT || + retval == EINTR); } AZ(ilck->held); ilck->held = 1; From lkarsten at varnish-software.com Tue Jun 14 09:42:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:42:09 +0200 Subject: [4.1] c72f24a document the default probe Message-ID: commit c72f24ad217075ca57c3fc4f06a95c8568f88ac2 Author: Nils Goroll Date: Wed May 11 16:14:17 2016 +0200 document the default probe diff --git a/doc/sphinx/reference/vcl.rst b/doc/sphinx/reference/vcl.rst index c982cb9..553fd72 100644 --- a/doc/sphinx/reference/vcl.rst +++ b/doc/sphinx/reference/vcl.rst @@ -244,6 +244,9 @@ the backend as down it they fail. A probe is defined as this:: .attribute = "value"; } +The probe named `default` is special and will be used for all backends +which do not explicitly reference a probe. + There are no mandatory options. These are the options you can set: url From lkarsten at varnish-software.com Tue Jun 14 09:42:10 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:42:10 +0200 Subject: [4.1] ab0977d allow whitespace and comments before the varnishtest marker Message-ID: commit ab0977dc3660dc52238442e029775d8cb8a1c871 Author: Nils Goroll Date: Fri May 13 08:05:31 2016 +0200 allow whitespace and comments before the varnishtest marker diff --git a/bin/varnishtest/tests/a00000.vtc b/bin/varnishtest/tests/a00000.vtc index ca02903..d3fe2fd 100644 --- a/bin/varnishtest/tests/a00000.vtc +++ b/bin/varnishtest/tests/a00000.vtc @@ -1,3 +1,6 @@ +# the first token in a varnishtest file must be "varnishtest", + # but whitespace and comments are fine + varnishtest "basic default HTTP transactions" server s1 { diff --git a/bin/varnishtest/vtc_main.c b/bin/varnishtest/vtc_main.c index 1db058c..ee87d09 100644 --- a/bin/varnishtest/vtc_main.c +++ b/bin/varnishtest/vtc_main.c @@ -55,6 +55,7 @@ #include "vsub.h" #include "vtcp.h" #include "vtim.h" +#include "vct.h" #define MAX_FILESIZE (1024 * 1024) @@ -530,6 +531,19 @@ main(int argc, char * const *argv) continue; exit(2); } + while (1) { + if (vct_islws(*p)) { + p++; + continue; + } + if (*p == '#') { + if ((p = strchr(p, '\n'))) + p++; + continue; + } + break; + } + if (strncmp(p, "varnishtest", 11) || !isspace(p[11])) { fprintf(stderr, "File \"%s\" doesn't start with 'varnishtest'\n", *argv); From lkarsten at varnish-software.com Tue Jun 14 09:42:10 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:42:10 +0200 Subject: [4.1] b9b6e19 Revert "Remove initial line feed." Message-ID: commit b9b6e1985b8a061e124747ba31b3482a2d1fb3e5 Author: Lasse Karstensen Date: Tue Jun 14 11:39:22 2016 +0200 Revert "Remove initial line feed." This reverts commit 47a1ef0632b7e42cf28fb88159e973fdca25467c. Not needed after 7dec1221ebae2e66f7be81c0311491ed0f684912. diff --git a/bin/varnishtest/tests/r01941.vtc b/bin/varnishtest/tests/r01941.vtc index 5d7441e..2531ded 100644 --- a/bin/varnishtest/tests/r01941.vtc +++ b/bin/varnishtest/tests/r01941.vtc @@ -1,3 +1,4 @@ + varnishtest "ESI memory should only be fully allocated" # In this test case, the cache is almost filled. Then we force the From lkarsten at varnish-software.com Tue Jun 14 09:46:10 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:46:10 +0200 Subject: [4.1] feb2c2e document what the default backend is Message-ID: commit feb2c2ebdfd16663fbb88a658abfc42238d7bff3 Author: Nils Goroll Date: Sat May 14 19:34:44 2016 +0200 document what the default backend is diff --git a/doc/sphinx/users-guide/vcl-backends.rst b/doc/sphinx/users-guide/vcl-backends.rst index b44ba9c..5844891 100644 --- a/doc/sphinx/users-guide/vcl-backends.rst +++ b/doc/sphinx/users-guide/vcl-backends.rst @@ -74,6 +74,10 @@ really arbitrary data. You want to send mobile devices to a different backend? No problem. ``if (req.http.User-agent ~ /mobile/) ..`` should do the trick. +Without an explicit backend selection, varnish will continue using +the `default` backend. If there is no backend named `default`, the +first backend found in the vcl will be used as the default backend. + Backends and virtual hosts in Varnish ------------------------------------- From lkarsten at varnish-software.com Tue Jun 14 09:46:10 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:46:10 +0200 Subject: [4.1] b9a817a Remove unneeded checks Message-ID: commit b9a817aeb4705d7b6d557d334eb899ba1ff89485 Author: Federico G. Schwindt Date: Mon May 16 18:37:44 2016 +0100 Remove unneeded checks diff --git a/configure.ac b/configure.ac index 60ca42d..f506ed5 100644 --- a/configure.ac +++ b/configure.ac @@ -239,9 +239,6 @@ AC_CHECK_FUNCS([strerror]) AC_FUNC_STRERROR_R AC_CHECK_FUNCS([dladdr]) AC_CHECK_FUNCS([socket]) -AC_CHECK_FUNCS([strptime]) -AC_CHECK_FUNCS([fmtcheck]) -AC_CHECK_FUNCS([getdtablesize]) AC_CHECK_FUNCS([nanosleep]) AC_CHECK_FUNCS([setppriv]) AC_CHECK_FUNCS([fallocate]) From lkarsten at varnish-software.com Tue Jun 14 09:46:10 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:46:10 +0200 Subject: [4.1] 8fabd4a Add a VSL variant that takes uses stdarg(3) Message-ID: commit 8fabd4aa60d68ee63bd743dea2b09b6eefae574f Author: Federico G. Schwindt Date: Mon May 16 14:59:48 2016 +0100 Add a VSL variant that takes uses stdarg(3) diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h index 39bdef7..0e0ca6f 100644 --- a/bin/varnishd/cache/cache.h +++ b/bin/varnishd/cache/cache.h @@ -957,6 +957,7 @@ void *VSM_Alloc(unsigned size, const char *class, const char *type, const char *ident); void VSM_Free(void *ptr); #ifdef VSL_ENDMARKER +void VSLv(enum VSL_tag_e tag, uint32_t vxid, const char *fmt, va_list va); void VSL(enum VSL_tag_e tag, uint32_t vxid, const char *fmt, ...) __v_printflike(3, 4); void VSLbv(struct vsl_log *, enum VSL_tag_e tag, const char *fmt, va_list va); diff --git a/bin/varnishd/cache/cache_shmlog.c b/bin/varnishd/cache/cache_shmlog.c index e733686..789e902 100644 --- a/bin/varnishd/cache/cache_shmlog.c +++ b/bin/varnishd/cache/cache_shmlog.c @@ -207,9 +207,8 @@ vslr(enum VSL_tag_e tag, uint32_t vxid, const char *b, unsigned len) */ void -VSL(enum VSL_tag_e tag, uint32_t vxid, const char *fmt, ...) +VSLv(enum VSL_tag_e tag, uint32_t vxid, const char *fmt, va_list ap) { - va_list ap; unsigned n, mlen = cache_param->vsl_reclen; char buf[mlen]; @@ -217,18 +216,26 @@ VSL(enum VSL_tag_e tag, uint32_t vxid, const char *fmt, ...) if (vsl_tag_is_masked(tag)) return; - if (strchr(fmt, '%') == NULL) { vslr(tag, vxid, fmt, strlen(fmt) + 1); } else { - va_start(ap, fmt); n = vsnprintf(buf, mlen, fmt, ap); - va_end(ap); if (n > mlen - 1) n = mlen - 1; buf[n++] = '\0'; /* NUL-terminated */ vslr(tag, vxid, buf, n); } + +} + +void +VSL(enum VSL_tag_e tag, uint32_t vxid, const char *fmt, ...) +{ + va_list ap; + + va_start(ap, fmt); + VSLv(tag, vxid, fmt, ap); + va_end(ap); } /*--------------------------------------------------------------------*/ From lkarsten at varnish-software.com Tue Jun 14 09:46:10 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:46:10 +0200 Subject: [4.1] 8f465d3 Make std.log() work from vcl_{init,fini} Message-ID: commit 8f465d363e4ea113ace1d13d302a2f6efbbf0f07 Author: Federico G. Schwindt Date: Mon May 16 18:40:11 2016 +0100 Make std.log() work from vcl_{init,fini} Fixes #1924. diff --git a/bin/varnishtest/tests/r01924.vtc b/bin/varnishtest/tests/r01924.vtc new file mode 100644 index 0000000..ed5095b --- /dev/null +++ b/bin/varnishtest/tests/r01924.vtc @@ -0,0 +1,22 @@ +varnishtest "Test std.log from vcl_{init,fini}" + +server s1 { + rxreq + txresp +} -start + +varnish v1 -vcl+backend { + import std; + + sub vcl_init { + std.log("init"); + } + + sub vcl_fini { + std.log("fini"); + } +} -start + +varnish v1 -vcl+backend { } + +varnish v1 -cliok "vcl.discard vcl1" diff --git a/lib/libvmod_std/vmod_std.c b/lib/libvmod_std/vmod_std.c index c4690b4..c5cdfc1 100644 --- a/lib/libvmod_std/vmod_std.c +++ b/lib/libvmod_std/vmod_std.c @@ -139,17 +139,20 @@ vmod_log(VRT_CTX, const char *fmt, ...) txt t; CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC); - u = WS_Reserve(ctx->ws, 0); - t.b = ctx->ws->f; va_start(ap, fmt); - t.e = VRT_StringList(ctx->ws->f, u, fmt, ap); + if (ctx->vsl != NULL) { + u = WS_Reserve(ctx->ws, 0); + t.b = ctx->ws->f; + t.e = VRT_StringList(ctx->ws->f, u, fmt, ap); + if (t.e != NULL) { + assert(t.e > t.b); + t.e--; + VSLbt(ctx->vsl, SLT_VCL_Log, t); + } + WS_Release(ctx->ws, 0); + } else + VSLv(SLT_VCL_Log, 0, fmt, ap); va_end(ap); - if (t.e != NULL) { - assert(t.e > t.b); - t.e--; - VSLbt(ctx->vsl, SLT_VCL_Log, t); - } - WS_Release(ctx->ws, 0); } VCL_VOID __match_proto__(td_std_syslog) From lkarsten at varnish-software.com Tue Jun 14 09:46:10 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:46:10 +0200 Subject: [4.1] 962fe5a Make std.syslog() work from vcl_{init,fini} Message-ID: commit 962fe5acf7c813dbebddc9ce240a2aad5124c38e Author: Federico G. Schwindt Date: Mon May 16 20:38:09 2016 +0100 Make std.syslog() work from vcl_{init,fini} Related to #1924. diff --git a/bin/varnishtest/tests/r01924.vtc b/bin/varnishtest/tests/r01924.vtc index ed5095b..d553124 100644 --- a/bin/varnishtest/tests/r01924.vtc +++ b/bin/varnishtest/tests/r01924.vtc @@ -1,4 +1,4 @@ -varnishtest "Test std.log from vcl_{init,fini}" +varnishtest "Test std.{log,syslog} from vcl_{init,fini}" server s1 { rxreq @@ -10,10 +10,12 @@ varnish v1 -vcl+backend { sub vcl_init { std.log("init"); + std.syslog(8 + 7, "init"); } sub vcl_fini { std.log("fini"); + std.syslog(8 + 7, "fini"); } } -start diff --git a/configure.ac b/configure.ac index f506ed5..d318408 100644 --- a/configure.ac +++ b/configure.ac @@ -243,6 +243,7 @@ AC_CHECK_FUNCS([nanosleep]) AC_CHECK_FUNCS([setppriv]) AC_CHECK_FUNCS([fallocate]) AC_CHECK_FUNCS([closefrom]) +AC_CHECK_FUNCS([vsyslog]) save_LIBS="${LIBS}" LIBS="${PTHREAD_LIBS}" diff --git a/lib/libvmod_std/vmod_std.c b/lib/libvmod_std/vmod_std.c index c5cdfc1..6cd5b8a 100644 --- a/lib/libvmod_std/vmod_std.c +++ b/lib/libvmod_std/vmod_std.c @@ -163,14 +163,17 @@ vmod_syslog(VRT_CTX, VCL_INT fac, const char *fmt, ...) txt t; CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC); - u = WS_Reserve(ctx->ws, 0); - t.b = ctx->ws->f; va_start(ap, fmt); - t.e = VRT_StringList(ctx->ws->f, u, fmt, ap); + if (ctx->ws != NULL) { + u = WS_Reserve(ctx->ws, 0); + t.b = ctx->ws->f; + t.e = VRT_StringList(ctx->ws->f, u, fmt, ap); + if (t.e != NULL) + syslog((int)fac, "%s", t.b); + WS_Release(ctx->ws, 0); + } else + vsyslog((int)fac, fmt, ap); va_end(ap); - if (t.e != NULL) - syslog((int)fac, "%s", t.b); - WS_Release(ctx->ws, 0); } VCL_VOID __match_proto__(td_std_collect) From lkarsten at varnish-software.com Tue Jun 14 09:46:10 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 11:46:10 +0200 Subject: [4.1] a8d3f1d Add #1924 to changelog. Message-ID: commit a8d3f1d2bf66006e66b0e4e6368e723c5a7a8102 Author: Lasse Karstensen Date: Tue Jun 14 11:45:06 2016 +0200 Add #1924 to changelog. Ref: #1924 diff --git a/doc/changes.rst b/doc/changes.rst index 6c2536b..7de6ec7 100644 --- a/doc/changes.rst +++ b/doc/changes.rst @@ -23,12 +23,13 @@ Bugs fixed ---------- * 1971_ - Add missing Wait_HeapDelete -* 1967_ - [ncsa] Remove implicit line feed when using formatfile. +* 1967_ - [ncsa] Remove implicit line feed when using formatfile * 1955_ - 4.1.x sometimes duplicates Age and Accept-Ranges headers * 1954_ - Correctly handle HTTP/1.1 EOF response -* 1931_ - Allow VCL set Last-Modified to be used for I-M-S processing. +* 1931_ - Allow VCL set Last-Modified to be used for I-M-S processing * 1928_ - req->task members must be set in case we get onto the waitinglist -* 1919_ - Avoid ban lurker panic with empty olist. +* 1924_ - Make std.log() and std.syslog() work from vcl_{init,fini} +* 1919_ - Avoid ban lurker panic with empty olist * 1918_ - Correctly handle EOF responses with HTTP/1.1 * 1912_ - Fix (insignificant) memory leak with mal-formed ESI directives. * 1904_ - Release memory instead of crashing on malformed ESI From lkarsten at varnish-software.com Tue Jun 14 10:00:16 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 12:00:16 +0200 Subject: [4.1] 11c0008 invalidate the cli_fd of dead varnishds Message-ID: commit 11c00080cfe0b610cbecb4ab861f089bc803becc Author: Nils Goroll Date: Tue May 17 14:38:04 2016 +0200 invalidate the cli_fd of dead varnishds In particular, this avoids a later backend.list attempt on a file descriptor to a dead process and, consequently, avoids a long delay in varnishtest when varnishd cant be started (for instance, because it is not in PATH). diff --git a/bin/varnishtest/vtc_varnish.c b/bin/varnishtest/vtc_varnish.c index 1ff780f..8f52886 100644 --- a/bin/varnishtest/vtc_varnish.c +++ b/bin/varnishtest/vtc_varnish.c @@ -466,18 +466,26 @@ varnish_launch(struct varnish *v) i, fd[0].revents, fd[1].revents); if (i == 0) { vtc_log(v->vl, 0, "FAIL timeout waiting for CLI connection"); + AZ(close(v->cli_fd)); + v->cli_fd = -1; return; } if (fd[1].revents & POLLHUP) { vtc_log(v->vl, 0, "FAIL debug pipe closed"); + AZ(close(v->cli_fd)); + v->cli_fd = -1; return; } if (!(fd[0].revents & POLLIN)) { vtc_log(v->vl, 0, "FAIL CLI connection wait failure"); + AZ(close(v->cli_fd)); + v->cli_fd = -1; return; } nfd = accept(v->cli_fd, NULL, NULL); if (nfd < 0) { + AZ(close(v->cli_fd)); + v->cli_fd = -1; vtc_log(v->vl, 0, "FAIL no CLI connection accepted"); return; } From lkarsten at varnish-software.com Tue Jun 14 10:00:16 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 12:00:16 +0200 Subject: [4.1] e90f5d7 Save the VCL syntax version, we will need it later on. Message-ID: commit e90f5d70f37216a3e5cb86e98224ea7b5f6ac71c Author: Poul-Henning Kamp Date: Thu May 19 09:59:45 2016 +0000 Save the VCL syntax version, we will need it later on. diff --git a/lib/libvcc/vcc_compile.h b/lib/libvcc/vcc_compile.h index 5da3d53..a4d3012 100644 --- a/lib/libvcc/vcc_compile.h +++ b/lib/libvcc/vcc_compile.h @@ -172,6 +172,7 @@ struct vcp { struct vcc { unsigned magic; #define VCC_MAGIC 0x24ad719d + float syntax; /* Parameter/Template section */ const struct vcp *param; diff --git a/lib/libvcc/vcc_parse.c b/lib/libvcc/vcc_parse.c index 8a94749..811c90f 100644 --- a/lib/libvcc/vcc_parse.c +++ b/lib/libvcc/vcc_parse.c @@ -291,17 +291,17 @@ vcc_ParseDirector(struct vcc *tl) static void vcc_ParseVcl(struct vcc *tl) { - double ver; struct token *tok; assert(vcc_IdIs(tl->t, "vcl")); vcc_NextToken(tl); tok = tl->t; - ver = vcc_DoubleVal(tl); + tl->syntax = vcc_DoubleVal(tl); ERRCHK(tl); - if (ver != 4.0) { + if (tl->syntax != 4.0) { // see TODO above - VSB_printf(tl->sb, "VCL version %.1f not supported.\n", ver); + VSB_printf(tl->sb, "VCL version %.1f not supported.\n", + tl->syntax); vcc_ErrWhere(tl, tok); ERRCHK(tl); } From lkarsten at varnish-software.com Tue Jun 14 10:00:16 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 12:00:16 +0200 Subject: [4.1] fc6790d Clarify the help text for d_opt/-d Message-ID: commit fc6790d41ebc8a7f37d49f482456d730e35bd964 Author: Nicole Izumi Date: Fri May 20 11:59:35 2016 +1000 Clarify the help text for d_opt/-d diff --git a/include/vut_options.h b/include/vut_options.h index a5b11ef..16258b8 100644 --- a/include/vut_options.h +++ b/include/vut_options.h @@ -30,9 +30,8 @@ /* VUT options */ #define VUT_OPT_d \ - VOPT("d", "[-d]", "Process old log entries on startup", \ - "Start processing log records at the head of the log" \ - " instead of the tail." \ + VOPT("d", "[-d]", "Process old log entries and exit", \ + "Process log records at the head of the log and exit." \ ) #define VUT_OPT_D \ From lkarsten at varnish-software.com Tue Jun 14 10:00:16 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 12:00:16 +0200 Subject: [4.1] 69897c6 Eliminate a NULL pointer issue with empty (apart from comments and whitespace) testfiles. Message-ID: commit 69897c6b911c0e4b790cf442b6dc3031312d90a3 Author: Poul-Henning Kamp Date: Sat May 21 17:01:35 2016 +0000 Eliminate a NULL pointer issue with empty (apart from comments and whitespace) testfiles. Spotted by: Coverity diff --git a/bin/varnishtest/vtc_main.c b/bin/varnishtest/vtc_main.c index ee87d09..4904be4 100644 --- a/bin/varnishtest/vtc_main.c +++ b/bin/varnishtest/vtc_main.c @@ -432,13 +432,55 @@ dns_works(void) * Main */ +static int +read_file(const char *fn, int ntest) +{ + struct vtc_tst *tp; + char *p; + + p = VFIL_readfile(NULL, fn, NULL); + if (p == NULL) { + fprintf(stderr, "Cannot stat file \"%s\": %s\n", + fn, strerror(errno)); + return (2); + } + for (;p != NULL && *p != '\0'; p++) { + if (vct_islws(*p)) + continue; + if (*p != '#') + break; + p = strchr(p, '\n'); + } + + if (p == NULL || *p == '\0') { + fprintf(stderr, "File \"%s\" has no content.\n", fn); + return (2); + } + + if (strncmp(p, "varnishtest", 11) || !isspace(p[11])) { + fprintf(stderr, + "File \"%s\" doesn't start with 'varnishtest'\n", + fn); + return(2); + } + ALLOC_OBJ(tp, TST_MAGIC); + AN(tp); + tp->filename = fn; + tp->script = p; + tp->ntodo = ntest; + VTAILQ_INSERT_TAIL(&tst_head, tp, list); + return(0); +} + +/********************************************************************** + * Main + */ + int main(int argc, char * const *argv) { int ch, i; int ntest = 1; /* Run tests this many times */ - struct vtc_tst *tp; - char *p; uintmax_t bufsiz; /* Default names of programs */ @@ -523,40 +565,10 @@ main(int argc, char * const *argv) argv += optind; for (;argc > 0; argc--, argv++) { - p = VFIL_readfile(NULL, *argv, NULL); - if (p == NULL) { - fprintf(stderr, "Cannot stat file \"%s\": %s\n", - *argv, strerror(errno)); - if (vtc_continue) - continue; - exit(2); - } - while (1) { - if (vct_islws(*p)) { - p++; - continue; - } - if (*p == '#') { - if ((p = strchr(p, '\n'))) - p++; - continue; - } - break; - } - - if (strncmp(p, "varnishtest", 11) || !isspace(p[11])) { - fprintf(stderr, "File \"%s\" doesn't start with 'varnishtest'\n", - *argv); - if (vtc_continue) - continue; + if (!read_file(*argv, ntest)) + continue; + if (!vtc_continue) exit(2); - } - ALLOC_OBJ(tp, TST_MAGIC); - AN(tp); - tp->filename = *argv; - tp->script = p; - tp->ntodo = ntest; - VTAILQ_INSERT_TAIL(&tst_head, tp, list); } feature_dns = dns_works(); From lkarsten at varnish-software.com Tue Jun 14 10:00:16 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 12:00:16 +0200 Subject: [4.1] 17f14aa Log an error if we fail to get storage Message-ID: commit 17f14aad07cbf95dc47212afe0102ec8d1ea793e Author: Federico G. Schwindt Date: Mon May 23 12:18:41 2016 +0100 Log an error if we fail to get storage Conflicts: bin/varnishd/cache/cache_fetch.c diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c index e3773e8..880fb4e 100644 --- a/bin/varnishd/cache/cache_fetch.c +++ b/bin/varnishd/cache/cache_fetch.c @@ -858,6 +858,7 @@ vbf_stp_error(struct worker *wrk, struct busyobj *bo) bo->vfc->esi_req = bo->bereq; if (vbf_beresp2obj(bo)) { + (void)VFP_Error(bo->vfc, "Could not get storage"); VSB_delete(synth_body); return (F_STP_FAIL); } From lkarsten at varnish-software.com Tue Jun 14 10:00:16 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 12:00:16 +0200 Subject: [4.1] 110a61e Minor OCD here and there Message-ID: commit 110a61e9ed00ba59513e30f9ef8a7e9b2a4e5128 Author: Federico G. Schwindt Date: Tue May 24 09:12:16 2016 +0100 Minor OCD here and there diff --git a/bin/varnishd/cache/cache_rfc2616.c b/bin/varnishd/cache/cache_rfc2616.c index 650bcf5..cf89f75 100644 --- a/bin/varnishd/cache/cache_rfc2616.c +++ b/bin/varnishd/cache/cache_rfc2616.c @@ -254,7 +254,7 @@ rfc2616_weak_compare(const char *p, const char *e) int RFC2616_Do_Cond(const struct req *req) { - const char *p, *e, *l; + const char *p, *e; double ims, lm; /* @@ -274,8 +274,8 @@ RFC2616_Do_Cond(const struct req *req) ims = VTIM_parse(p); if (!ims || ims > req->t_req) /* [RFC7232 3.3 p16] */ return (0); - if (http_GetHdr(req->resp, H_Last_Modified, &l)) { - lm = VTIM_parse(l); + if (http_GetHdr(req->resp, H_Last_Modified, &p)) { + lm = VTIM_parse(p); if (!lm || lm > ims) return (0); return (1); diff --git a/include/miniobj.h b/include/miniobj.h index 95f3bef..901143f 100644 --- a/include/miniobj.h +++ b/include/miniobj.h @@ -55,13 +55,13 @@ #define CAST_OBJ_NOTNULL(to, from, type_magic) \ do { \ (to) = (from); \ - assert((to) != NULL); \ + AN((to)); \ CHECK_OBJ((to), (type_magic)); \ } while (0) -#define TAKE_OBJ_NOTNULL(to, pfrom, type_magic) \ +#define TAKE_OBJ_NOTNULL(to, pfrom, type_magic) \ do { \ - assert((pfrom) != NULL); \ + AN((pfrom)); \ (to) = *(pfrom); \ *(pfrom) = NULL; \ CHECK_OBJ_NOTNULL((to), (type_magic)); \ diff --git a/lib/libvmod_debug/vmod_debug.c b/lib/libvmod_debug/vmod_debug.c index f16953b..cca54e8 100644 --- a/lib/libvmod_debug/vmod_debug.c +++ b/lib/libvmod_debug/vmod_debug.c @@ -350,9 +350,9 @@ event_function(VRT_CTX, struct vmod_priv *priv, enum vcl_event_e e) { switch (e) { - case VCL_EVENT_LOAD: return event_load(ctx, priv); - case VCL_EVENT_WARM: return event_warm(ctx, priv); - case VCL_EVENT_COLD: return event_cold(ctx, priv); + case VCL_EVENT_LOAD: return (event_load(ctx, priv)); + case VCL_EVENT_WARM: return (event_warm(ctx, priv)); + case VCL_EVENT_COLD: return (event_cold(ctx, priv)); default: return (0); } } @@ -409,7 +409,7 @@ vmod_workspace_free(VRT_CTX, VCL_ENUM which) WS_Assert(ws); AZ(ws->r); - return pdiff(ws->f, ws->e); + return (pdiff(ws->f, ws->e)); } VCL_BOOL From lkarsten at varnish-software.com Tue Jun 14 10:00:16 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 12:00:16 +0200 Subject: [4.1] 2f4defe Correct instructions to disable THP permanently Message-ID: commit 2f4defe51206cb76132bd8eb0c0c5a37e9bae2a5 Author: Federico G. Schwindt Date: Wed May 25 17:35:06 2016 +0100 Correct instructions to disable THP permanently Pointed out by Jean-Fran?ois Laurens. diff --git a/doc/sphinx/installation/platformnotes.rst b/doc/sphinx/installation/platformnotes.rst index 741d98b..1657af1 100644 --- a/doc/sphinx/installation/platformnotes.rst +++ b/doc/sphinx/installation/platformnotes.rst @@ -12,10 +12,11 @@ Transparent hugepages on Redhat Enterprise Linux 6 On RHEL6 Transparent Hugepage kernel support is enabled by default. This is known to cause sporadic crashes of Varnish. -It is recommended to disable transparent hugepages on affected systems. This -can be done with -``echo "never" > /sys/kernel/mm/redhat_transparent_hugepage/enabled`` (runtime) and changes to -`/etc/sysctl.conf` (persisted.) +It is recommended to disable transparent hugepages on affected +systems. This can be done with +``echo never > /sys/kernel/mm/redhat_transparent_hugepage/enabled`` +(runtime) or by adding "transparent_hugepage=never" to the kernel boot +line in the "/etc/grub.conf" file (persistent). On Debian/Ubuntu systems running 3.2 kernels the default value is "madvise" and does not need to be changed. From lkarsten at varnish-software.com Tue Jun 14 10:00:16 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 12:00:16 +0200 Subject: [4.1] 87bfffd Adopt to 4.1. Message-ID: commit 87bfffd8365c10d9b3c6fbfbeb6e634981b37f26 Author: Lasse Karstensen Date: Tue Jun 14 11:56:43 2016 +0200 Adopt to 4.1. diff --git a/bin/varnishtest/tests/r01924.vtc b/bin/varnishtest/tests/r01924.vtc index d553124..43078e0 100644 --- a/bin/varnishtest/tests/r01924.vtc +++ b/bin/varnishtest/tests/r01924.vtc @@ -6,7 +6,7 @@ server s1 { } -start varnish v1 -vcl+backend { - import std; + import ${vmod_std}; sub vcl_init { std.log("init"); From lkarsten at varnish-software.com Tue Jun 14 10:58:07 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 12:58:07 +0200 Subject: [4.1] 12b384b Improve output on PRIV_TASK/TOP in init/fini. Message-ID: commit 12b384ba0a23bcaa6767bf6804db96cb61cfa71f Author: Lasse Karstensen Date: Thu May 26 12:51:38 2016 +0200 Improve output on PRIV_TASK/TOP in init/fini. Output a proper error message instead of a more cryptic assert if the vmod writer does the wrong thing. Ref: #1811 diff --git a/bin/varnishd/cache/cache_vrt_priv.c b/bin/varnishd/cache/cache_vrt_priv.c index c499312..fd8215b 100644 --- a/bin/varnishd/cache/cache_vrt_priv.c +++ b/bin/varnishd/cache/cache_vrt_priv.c @@ -115,10 +115,11 @@ VRT_priv_task(VRT_CTX, void *vmod_id) if (ctx->req) { CHECK_OBJ_NOTNULL(ctx->req, REQ_MAGIC); id = (uintptr_t)ctx->req; - } else { + } else if (ctx->bo) { CHECK_OBJ_NOTNULL(ctx->bo, BUSYOBJ_MAGIC); id = (uintptr_t)ctx->bo; - } + } else + WRONG("PRIV_TASK is only accessible in client or backend VCL contexts"); return (VRT_priv_dynamic(ctx, id, (uintptr_t)vmod_id)); } @@ -133,7 +134,8 @@ VRT_priv_top(VRT_CTX, void *vmod_id) CHECK_OBJ_NOTNULL(ctx->req->top, REQ_MAGIC); id = (uintptr_t)&ctx->req->top->top; return (VRT_priv_dynamic(ctx, id, (uintptr_t)vmod_id)); - } + } else + WRONG("PRIV_TOP is only accessible in client VCL context"); return (NULL); } From lkarsten at varnish-software.com Tue Jun 14 10:58:07 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 12:58:07 +0200 Subject: [4.1] 0b1ff51 Implement atomic creation of output files. Message-ID: commit 0b1ff5170ed56484db73ee3a6396830dae01ebf5 Author: Lasse Karstensen Date: Thu May 26 16:49:07 2016 +0200 Implement atomic creation of output files. By writing output byte by byte, the calling parallel make would start jobs depending on the completed files too soon. Avoid this by writing to temporary files and move the completed results into place. Fixes: #1921 diff --git a/lib/libvcc/vmodtool.py b/lib/libvcc/vmodtool.py index 2f7253a..e820c2a 100755 --- a/lib/libvcc/vmodtool.py +++ b/lib/libvcc/vmodtool.py @@ -41,9 +41,10 @@ import re import optparse import unittest import random -from os import unlink +from os import fdopen, rename, unlink from os.path import dirname, exists, join, realpath from pprint import pprint, pformat +from tempfile import mkstemp ctypes = { 'ACL': "VCL_ACL", @@ -918,34 +919,37 @@ def runmain(inputvcc, rstdir, outputprefix): ####################################################################### # Parsing done, now process # + fd_cfile, fn_cfile = mkstemp() + cfile = fdopen(fd_cfile, "w") - fc = open("%s.c" % outputprefix, "w") - fh = open("%s.h" % outputprefix, "w") + fd_headerfile, fn_headerfile = mkstemp() + headerfile = fdopen(fd_headerfile, "w") - write_c_file_warning(fc) - write_c_file_warning(fh) + write_c_file_warning(cfile) + write_c_file_warning(headerfile) - fh.write('struct vmod_priv;\n\n') + headerfile.write('struct vmod_priv;\n\n') - fh.write('extern const struct vmod_data Vmod_%s_Data;\n\n' % vx[0].nam) + headerfile.write('extern const struct vmod_data Vmod_%s_Data;\n\n' % vx[0].nam) - vx[0].c_proto(fh) + vx[0].c_proto(headerfile) - fc.write('#include "config.h"\n') - fc.write('#include "vcl.h"\n') - fc.write('#include "vrt.h"\n') - fc.write('#include "%s.h"\n' % outputprefix) - fc.write('#include "vmod_abi.h"\n') - fc.write('\n') + cfile.write('#include "config.h"\n') + cfile.write('#include "vcl.h"\n') + cfile.write('#include "vrt.h"\n') + cfile.write('#include "%s.h"\n' % outputprefix) + cfile.write('#include "vmod_abi.h"\n') + cfile.write('\n') - vx[0].c_typedefs(fc) - vx[0].c_vmod(fc) + vx[0].c_typedefs(cfile) + vx[0].c_vmod(cfile) - fc.close() - fh.close() + cfile.close() + headerfile.close() for suf in ("", ".man"): - fp = open(join(rstdir, "vmod_%s%s.rst" % (vx[0].nam, suf)), "w") + fd, fn_fp = mkstemp() + fp = fdopen(fd, "w") write_rst_file_warning(fp) vx[0].doc_dump(fp, suf) @@ -958,6 +962,12 @@ def runmain(inputvcc, rstdir, outputprefix): for i in copy_right: fp.write(" %s\n" % i) fp.write("\n") + fp.close() + rename(fn_fp, join(rstdir, "vmod_%s%s.rst" % (vx[0].nam, suf))) + + # Our makefile targets the .c file so make sure to put that in place last. + rename(fn_headerfile, outputprefix + ".h") + rename(fn_cfile, outputprefix + ".h") if __name__ == "__main__": From lkarsten at varnish-software.com Tue Jun 14 10:58:07 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 12:58:07 +0200 Subject: [4.1] c21bb8d Output to the correct file name. Message-ID: commit c21bb8dc7ee11699bb7d66d21acc5f98386557f0 Author: Lasse Karstensen Date: Thu May 26 17:13:16 2016 +0200 Output to the correct file name. Last minute changes... diff --git a/lib/libvcc/vmodtool.py b/lib/libvcc/vmodtool.py index e820c2a..c2b1977 100755 --- a/lib/libvcc/vmodtool.py +++ b/lib/libvcc/vmodtool.py @@ -967,7 +967,7 @@ def runmain(inputvcc, rstdir, outputprefix): # Our makefile targets the .c file so make sure to put that in place last. rename(fn_headerfile, outputprefix + ".h") - rename(fn_cfile, outputprefix + ".h") + rename(fn_cfile, outputprefix + ".c") if __name__ == "__main__": From lkarsten at varnish-software.com Tue Jun 14 10:58:08 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 12:58:08 +0200 Subject: [4.1] 729c2c5 Create the temporary files in the same dir Message-ID: commit 729c2c598ca5c4caf101e3e8f76074e96781367d Author: Federico G. Schwindt Date: Fri May 27 08:46:23 2016 +0100 Create the temporary files in the same dir Rename will fail across different mount points. diff --git a/lib/libvcc/vmodtool.py b/lib/libvcc/vmodtool.py index c2b1977..d95fb71 100755 --- a/lib/libvcc/vmodtool.py +++ b/lib/libvcc/vmodtool.py @@ -919,10 +919,10 @@ def runmain(inputvcc, rstdir, outputprefix): ####################################################################### # Parsing done, now process # - fd_cfile, fn_cfile = mkstemp() + fd_cfile, fn_cfile = mkstemp(dir='.') cfile = fdopen(fd_cfile, "w") - fd_headerfile, fn_headerfile = mkstemp() + fd_headerfile, fn_headerfile = mkstemp(dir='.') headerfile = fdopen(fd_headerfile, "w") write_c_file_warning(cfile) @@ -948,7 +948,7 @@ def runmain(inputvcc, rstdir, outputprefix): headerfile.close() for suf in ("", ".man"): - fd, fn_fp = mkstemp() + fd, fn_fp = mkstemp(dir='.') fp = fdopen(fd, "w") write_rst_file_warning(fp) From lkarsten at varnish-software.com Tue Jun 14 10:58:08 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 12:58:08 +0200 Subject: [4.1] 7cd9826 At least one platform has a backtrace(3) which returns -1 in a size_t :-( Message-ID: commit 7cd98261f75676c02fc3109f649c082d00749839 Author: Poul-Henning Kamp Date: Sun May 29 20:27:09 2016 +0000 At least one platform has a backtrace(3) which returns -1 in a size_t :-( diff --git a/bin/varnishd/cache/cache_panic.c b/bin/varnishd/cache/cache_panic.c index eb4aa2c..4951c51 100644 --- a/bin/varnishd/cache/cache_panic.c +++ b/bin/varnishd/cache/cache_panic.c @@ -418,14 +418,16 @@ static void pan_backtrace(struct vsb *vsb) { void *array[10]; - size_t size; - size_t i; + ssize_t size; + ssize_t i; char **strings; char *p; size = backtrace (array, 10); - if (size == 0) + if (size <= 0) { + VSB_printf(vsb, "Backtrace not available (%zd)\n", size); return; + } VSB_printf(vsb, "Backtrace:\n"); VSB_indent(vsb, 2); for (i = 0; i < size; i++) { From lkarsten at varnish-software.com Tue Jun 14 10:58:08 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 12:58:08 +0200 Subject: [4.1] 8b59fe4 Make the santity check of backtrace(3) returns even more robust. Message-ID: commit 8b59fe4063c6051e7c15baa82985d315afdb74c6 Author: Poul-Henning Kamp Date: Mon May 30 07:50:34 2016 +0000 Make the santity check of backtrace(3) returns even more robust. diff --git a/bin/varnishd/cache/cache_panic.c b/bin/varnishd/cache/cache_panic.c index 4951c51..6a55208 100644 --- a/bin/varnishd/cache/cache_panic.c +++ b/bin/varnishd/cache/cache_panic.c @@ -414,18 +414,20 @@ pan_sess(struct vsb *vsb, const struct sess *sp) /*--------------------------------------------------------------------*/ +#define BACKTRACE_LEVELS 10 + static void pan_backtrace(struct vsb *vsb) { - void *array[10]; - ssize_t size; - ssize_t i; + void *array[BACKTRACE_LEVELS]; + size_t size; + size_t i; char **strings; char *p; - size = backtrace (array, 10); - if (size <= 0) { - VSB_printf(vsb, "Backtrace not available (%zd)\n", size); + size = backtrace (array, BACKTRACE_LEVELS); + if (size > BACKTRACE_LEVELS) { + VSB_printf(vsb, "Backtrace not available (ret=%zu)\n", size); return; } VSB_printf(vsb, "Backtrace:\n"); From lkarsten at varnish-software.com Tue Jun 14 10:58:08 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 12:58:08 +0200 Subject: [4.1] ae8a017 This bug shows why having both BE and LE test-boxes is important. Message-ID: commit ae8a0175de936e8095fb05fd97bb37731f92aba1 Author: Poul-Henning Kamp Date: Mon May 30 08:41:08 2016 +0000 This bug shows why having both BE and LE test-boxes is important. diff --git a/bin/varnishd/common/params.h b/bin/varnishd/common/params.h index d5ac4f5..678ed26 100644 --- a/bin/varnishd/common/params.h +++ b/bin/varnishd/common/params.h @@ -96,7 +96,7 @@ struct params { double wthread_add_delay; double wthread_fail_delay; double wthread_destroy_delay; - double wthread_stats_rate; + unsigned wthread_stats_rate; ssize_t wthread_stacksize; unsigned wthread_queue_limit; From lkarsten at varnish-software.com Tue Jun 14 10:58:08 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 12:58:08 +0200 Subject: [4.1] adea180 With FreeBSD 10.3 on a BeagleBone PCRE JIT explodes in the pcre_study() function. Message-ID: commit adea180f830c7a7bbcae0dc053a1b95ffaf94893 Author: Poul-Henning Kamp Date: Mon May 30 07:41:39 2016 +0000 With FreeBSD 10.3 on a BeagleBone PCRE JIT explodes in the pcre_study() function. Maybe we should have a configure-time test to see if it actually works, but for now, just make autogen.des handle it. Conflicts: autogen.des diff --git a/autogen.des b/autogen.des index a562b06..42e7d1e 100755 --- a/autogen.des +++ b/autogen.des @@ -6,6 +6,8 @@ set -ex make -k distclean > /dev/null 2>&1 || true +OPTS="" + # Prefer CLANG if we have it, and have not given preferences if [ -f /usr/bin/clang -a "x${CC}" = "x" ] ; then CC=clang @@ -18,6 +20,10 @@ else DST="--prefix=/opt/varnish --mandir=/opt/varnish/man" fi +if [ "x`uname -po`" = "xFreeBSD armv6" ] ; then + OPTS="${OPTS} --disable-pcre-jit" +fi + rm -f configure . ./autogen.sh 2>&1 | egrep -v "(subdir-objects|is in a subdirectory)" @@ -34,4 +40,5 @@ export CONFIG_SHELL=/bin/sh --enable-debugging-symbols \ --enable-dependency-tracking \ --enable-tests \ + $OPTS \ "$@" From lkarsten at varnish-software.com Tue Jun 14 10:58:08 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 12:58:08 +0200 Subject: [4.1] 9acf069 Check pcre_study() is working in configure Message-ID: commit 9acf0699c44b2f4e45b22789f38fbb3d00ab4eef Author: Federico G. Schwindt Date: Mon May 30 18:03:27 2016 +0100 Check pcre_study() is working in configure Conflicts: autogen.des diff --git a/autogen.des b/autogen.des index 42e7d1e..d710ab6 100755 --- a/autogen.des +++ b/autogen.des @@ -6,8 +6,6 @@ set -ex make -k distclean > /dev/null 2>&1 || true -OPTS="" - # Prefer CLANG if we have it, and have not given preferences if [ -f /usr/bin/clang -a "x${CC}" = "x" ] ; then CC=clang @@ -20,14 +18,9 @@ else DST="--prefix=/opt/varnish --mandir=/opt/varnish/man" fi -if [ "x`uname -po`" = "xFreeBSD armv6" ] ; then - OPTS="${OPTS} --disable-pcre-jit" -fi - rm -f configure . ./autogen.sh 2>&1 | egrep -v "(subdir-objects|is in a subdirectory)" - # autoconf prior to 2.62 has issues with zsh 4.2 and newer export CONFIG_SHELL=/bin/sh @@ -40,5 +33,4 @@ export CONFIG_SHELL=/bin/sh --enable-debugging-symbols \ --enable-dependency-tracking \ --enable-tests \ - $OPTS \ "$@" diff --git a/configure.ac b/configure.ac index d318408..831bab9 100644 --- a/configure.ac +++ b/configure.ac @@ -161,17 +161,29 @@ if test "$enable_pcre_jit" = yes; then AC_MSG_CHECKING(for PCRE JIT usability) save_CFLAGS="${CFLAGS}" CFLAGS="${PCRE_CFLAGS}" - AC_COMPILE_IFELSE( - [AC_LANG_PROGRAM([[#include "pcre.h" - #if PCRE_MAJOR != 8 || PCRE_MINOR < 32 - #error no jit - #endif]])], + save_LIBS="${CFLAGS}" + LIBS="${PCRE_LIBS}" + AC_RUN_IFELSE( + [AC_LANG_PROGRAM([[ +#include +#if PCRE_MAJOR != 8 || PCRE_MINOR < 32 +#error no jit +#endif + ]],[[ +const char *error; +pcre *re; +int erroroffset; +re = pcre_compile(".", 0, &error, &erroroffset, NULL); +if (!pcre_study(re, PCRE_STUDY_JIT_COMPILE, &error)) + return (1); + ]])], [AC_MSG_RESULT(yes) AC_DEFINE([USE_PCRE_JIT], [1], [Use the PCRE JIT compiler]) ], [AC_MSG_RESULT(no)] ) CFLAGS="${save_CFLAGS}" + LIBS="${save_LIBS}" fi From lkarsten at varnish-software.com Tue Jun 14 13:25:11 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 15:25:11 +0200 Subject: [4.1] 8d30d81 Simplify r01941.vtc a bit Message-ID: commit 8d30d817f544554a57d561b8c557c57a42c3604b Author: Guillaume Quintard Date: Wed Jun 1 19:11:54 2016 +0200 Simplify r01941.vtc a bit diff --git a/bin/varnishtest/tests/r01941.vtc b/bin/varnishtest/tests/r01941.vtc index 2531ded..188125c 100644 --- a/bin/varnishtest/tests/r01941.vtc +++ b/bin/varnishtest/tests/r01941.vtc @@ -11,100 +11,13 @@ server s1 { expect req.url == /big txresp -bodylen 1037480 rxreq - txresp -body { - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -} + txresp -nolen -hdr "Content-Length: 6545" -body "" + loop 92 { + send {} + } + send "" rxreq expect req.url == /long1234567890123456789012345678901234567890.txt txresp -body {fo} From lkarsten at varnish-software.com Tue Jun 14 13:25:12 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 15:25:12 +0200 Subject: [4.1] 2bb8e5a Fix comment Message-ID: commit 2bb8e5a6e412805631e17125e006f21b02d97127 Author: Poul-Henning Kamp Date: Thu Jun 2 07:15:01 2016 +0000 Fix comment diff --git a/lib/libvcc/vcc_token.c b/lib/libvcc/vcc_token.c index a8513f8..8a1ec48 100644 --- a/lib/libvcc/vcc_token.c +++ b/lib/libvcc/vcc_token.c @@ -466,7 +466,7 @@ vcc_Lexer(struct vcc *tl, struct source *sp) return; } - /* Match for the fixed tokens (see token.tcl) */ + /* Match for the fixed tokens (see generate.py) */ u = vcl_fixed_token(p, &q); if (u != 0) { vcc_AddToken(tl, u, p, q); From lkarsten at varnish-software.com Tue Jun 14 13:25:12 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 15:25:12 +0200 Subject: [4.1] e01a49e Add -h and use different exit status Message-ID: commit e01a49e157784c06d532be47862c652440c78b6a Author: Federico G. Schwindt Date: Sat Jun 4 16:39:24 2016 +0100 Add -h and use different exit status diff --git a/bin/varnishadm/varnishadm.c b/bin/varnishadm/varnishadm.c index e426f0b..e90df71 100644 --- a/bin/varnishadm/varnishadm.c +++ b/bin/varnishadm/varnishadm.c @@ -378,13 +378,13 @@ pass(int sock) static void -usage(void) +usage(int status) { fprintf(stderr, - "Usage: varnishadm [-n ident] [-t timeout] [-S secretfile] " + "Usage: varnishadm [-h] [-n ident] [-t timeout] [-S secretfile] " "[-T [address]:port] [command [...]]\n"); fprintf(stderr, "\t-n is mutually exclusive with -S and -T\n"); - exit(1); + exit(status); } static int @@ -456,8 +456,11 @@ main(int argc, char * const *argv) * The '+' stops that from happening * See #1496 */ - while ((opt = getopt(argc, argv, "+n:S:T:t:")) != -1) { + while ((opt = getopt(argc, argv, "+hn:S:T:t:")) != -1) { switch (opt) { + case 'h': + /* Usage help */ + usage(0); case 'n': n_arg = optarg; break; @@ -470,10 +473,10 @@ main(int argc, char * const *argv) case 't': timeout = VNUM(optarg); if (isnan(timeout)) - usage(); + usage(1); break; default: - usage(); + usage(1); } } @@ -482,7 +485,7 @@ main(int argc, char * const *argv) if (n_arg != NULL) { if (T_arg != NULL || S_arg != NULL) { - usage(); + usage(1); } sock = n_arg_sock(n_arg); } else if (T_arg == NULL) { diff --git a/doc/sphinx/reference/varnishadm.rst b/doc/sphinx/reference/varnishadm.rst index 395a72a..e503671 100644 --- a/doc/sphinx/reference/varnishadm.rst +++ b/doc/sphinx/reference/varnishadm.rst @@ -14,7 +14,7 @@ Control a running Varnish instance SYNOPSIS ======== -varnishadm [-n ident] [-t timeout] [-S secretfile] [-T [address]:port] [command [...]] +varnishadm [-h] [-n ident] [-t timeout] [-S secretfile] [-T [address]:port] [command [...]] DESCRIPTION @@ -35,6 +35,9 @@ replies between the CLI socket and stdin/stdout. OPTIONS ======= +-h + Print program usage and exit. + -n ident Connect to the instance of `varnishd` with this name. From lkarsten at varnish-software.com Tue Jun 14 13:25:12 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 15:25:12 +0200 Subject: [4.1] fea75b7 Exit with 0 if -h was used Message-ID: commit fea75b7516b6db76b45a546aac9c59725da5f262 Author: Federico G. Schwindt Date: Sat Jun 4 16:43:16 2016 +0100 Exit with 0 if -h was used diff --git a/bin/varnishhist/varnishhist.c b/bin/varnishhist/varnishhist.c index f279dc3..36a25e8 100644 --- a/bin/varnishhist/varnishhist.c +++ b/bin/varnishhist/varnishhist.c @@ -393,6 +393,9 @@ main(int argc, char **argv) assert(VUT_Arg('c', NULL)); while ((i = getopt(argc, argv, vopt_optstring)) != -1) { switch (i) { + case 'h': + /* Usage help */ + usage(0); case 'P': colon = strchr(optarg, ':'); /* no colon, take the profile as a name*/ From lkarsten at varnish-software.com Tue Jun 14 13:25:12 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 15:25:12 +0200 Subject: [4.1] 5d9c894 Correct parameter name Message-ID: commit 5d9c8942c4eaadbd847ef0b399b54e8726ea67af Author: Federico G. Schwindt Date: Wed Jun 8 11:56:05 2016 +0100 Correct parameter name diff --git a/include/tbl/vsc_f_main.h b/include/tbl/vsc_f_main.h index 13106f4..584d77e 100644 --- a/include/tbl/vsc_f_main.h +++ b/include/tbl/vsc_f_main.h @@ -239,7 +239,7 @@ VSC_F(thread_queue_len, uint64_t, 0, 'g', 'i', info, "Length of session queue", "Length of session queue waiting for threads." " NB: Only updates once per second." - " See also parameter queue_max." + " See also parameter thread_queue_limit." ) VSC_F(busy_sleep, uint64_t, 1, 'c', 'i', info, @@ -263,13 +263,13 @@ VSC_F(busy_killed, uint64_t, 1, 'c', 'i', info, VSC_F(sess_queued, uint64_t, 0, 'c', 'i', info, "Sessions queued for thread", "Number of times session was queued waiting for a thread." - " See also parameter queue_max." + " See also parameter thread_queue_limit." ) VSC_F(sess_dropped, uint64_t, 0, 'c', 'i', info, "Sessions dropped for thread", "Number of times session was dropped because the queue were too" - " long already. See also parameter queue_max." + " long already. See also parameter thread_queue_limit." ) /*---------------------------------------------------------------------*/ From lkarsten at varnish-software.com Tue Jun 14 13:25:12 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 15:25:12 +0200 Subject: [4.1] fd18b3b Finally some instructions for building on SmartOS (for a start) Message-ID: commit fd18b3ba004f3253f3c1479857a04964fbfc30a0 Author: Nils Goroll Date: Wed Jun 8 22:17:09 2016 +0200 Finally some instructions for building on SmartOS (for a start) diff --git a/doc/sphinx/installation/install.rst b/doc/sphinx/installation/install.rst index e455016..f7e6492 100644 --- a/doc/sphinx/installation/install.rst +++ b/doc/sphinx/installation/install.rst @@ -110,6 +110,19 @@ packages installed: * `python-sphinx` * `graphviz` +Build dependencies on a SmartOS Zone +------------------------------------ + +As of SmartOS pkgsrc 2015Q4, install the following packages:: + + pkgin in autoconf automake libedit libtool ncurses \ + pcre graphviz py27-sphinx python27 gmake gcc49 \ + pkg-config + +Optionally, to pull from a repository:: + + pkgin in git + Compiling Varnish ----------------- From lkarsten at varnish-software.com Tue Jun 14 13:25:12 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 15:25:12 +0200 Subject: [4.1] 01a9e47 Fix typo Message-ID: commit 01a9e4712c383036aec56e1ddd8e4ea12bbf922b Author: Magnus Hagander Date: Tue Jun 14 10:24:10 2016 +0200 Fix typo An object is delivered to the client, not to the backend. diff --git a/include/tbl/vsc_f_main.h b/include/tbl/vsc_f_main.h index 584d77e..31235e1 100644 --- a/include/tbl/vsc_f_main.h +++ b/include/tbl/vsc_f_main.h @@ -102,7 +102,7 @@ VSC_F(cache_miss, uint64_t, 1, 'c', 'i', info, "Cache misses", "Count of misses" " A cache miss indicates the object was fetched from the" - " backend before delivering it to the backend." + " backend before delivering it to the client." ) /*---------------------------------------------------------------------*/ From lkarsten at varnish-software.com Tue Jun 14 14:27:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 16:27:09 +0200 Subject: [4.1] 0226a3e Add forgotten link. Message-ID: commit 0226a3edb203dbb8620a506cb53cd68e8dac8238 Author: Lasse Karstensen Date: Tue Jun 14 15:25:37 2016 +0200 Add forgotten link. diff --git a/doc/changes.rst b/doc/changes.rst index 7de6ec7..8b895bf 100644 --- a/doc/changes.rst +++ b/doc/changes.rst @@ -50,6 +50,7 @@ Bugs fixed .. _1954: https://github.com/varnishcache/varnish-cache/issues/1954 .. _1931: https://github.com/varnishcache/varnish-cache/issues/1931 .. _1928: https://github.com/varnishcache/varnish-cache/issues/1928 +.. _1924: https://github.com/varnishcache/varnish-cache/issues/1924 .. _1919: https://github.com/varnishcache/varnish-cache/issues/1919 .. _1918: https://github.com/varnishcache/varnish-cache/issues/1918 .. _1912: https://github.com/varnishcache/varnish-cache/issues/1912 From lkarsten at varnish-software.com Tue Jun 14 14:27:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 16:27:09 +0200 Subject: [4.1] a38f5ee In probes treat a poll timeout as a recv error Message-ID: commit a38f5eecddb51d987af3922b5844e6192d04fa68 Author: Federico G. Schwindt Date: Fri Mar 11 23:14:07 2016 +0000 In probes treat a poll timeout as a recv error diff --git a/bin/varnishd/cache/cache_backend_probe.c b/bin/varnishd/cache/cache_backend_probe.c index e0590d8..c688cd0 100644 --- a/bin/varnishd/cache/cache_backend_probe.c +++ b/bin/varnishd/cache/cache_backend_probe.c @@ -271,6 +271,8 @@ vbp_poke(struct vbp_target *vt) if (tmo > 0) i = poll(pfd, 1, tmo); if (i == 0 || tmo <= 0) { + if (i == 0) + vt->err_recv |= 1; VTCP_close(&s); return; } From lkarsten at varnish-software.com Tue Jun 14 14:27:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 16:27:09 +0200 Subject: [4.1] 090ebb9 Dump more VCL info in panic, and add a general macro for checking miniobj magics. Message-ID: commit 090ebb9c9a2c9ac64fcb046ee46bc3f12beff20f Author: Poul-Henning Kamp Date: Wed May 4 13:02:58 2016 +0000 Dump more VCL info in panic, and add a general macro for checking miniobj magics. Conflicts: bin/varnishd/cache/cache_panic.c diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h index 0e0ca6f..9ae79d5 100644 --- a/bin/varnishd/cache/cache.h +++ b/bin/varnishd/cache/cache.h @@ -1123,6 +1123,14 @@ DO_DEBUG(enum debug_bits x) VSL(SLT_Debug, (id), __VA_ARGS__); \ } while (0) +#define PAN_CheckMagic(vsb, ptr, exp) \ + do { \ + if ((ptr)->magic != (exp)) \ + VSB_printf((vsb), \ + "MAGIC 0x%08x (Should:%s/0x%08x)\n", \ + (ptr)->magic, #exp, exp); \ + } while(0) + #ifdef VARNISHD_IS_NOT_A_VMOD # include "cache/cache_priv.h" #endif diff --git a/bin/varnishd/cache/cache_panic.c b/bin/varnishd/cache/cache_panic.c index 6a55208..08a4ad1 100644 --- a/bin/varnishd/cache/cache_panic.c +++ b/bin/varnishd/cache/cache_panic.c @@ -115,6 +115,7 @@ pan_ws(struct vsb *vsb, const struct ws *ws) VSB_printf(vsb, "ws = %p {\n", ws); VSB_indent(vsb, 2); + PAN_CheckMagic(vsb, ws, WS_MAGIC); if (!(ws->id[0] & 0x20)) VSB_printf(vsb, "OVERFLOWED "); VSB_printf(vsb, "id = \"%s\",\n", ws->id); @@ -144,6 +145,7 @@ pan_htc(struct vsb *vsb, const struct http_conn *htc) VSB_printf(vsb, "http_conn = %p {\n", htc); VSB_indent(vsb, 2); + PAN_CheckMagic(vsb, htc, HTTP_CONN_MAGIC); VSB_printf(vsb, "fd = %d,\n", htc->fd); VSB_printf(vsb, "doclose = %s,\n", sess_close_2str(htc->doclose, 0)); VSB_printf(vsb, "ws = %p,\n", htc->ws); @@ -172,6 +174,7 @@ pan_http(struct vsb *vsb, const char *id, const struct http *h) VSB_printf(vsb, "http[%s] = %p {\n", id, h); VSB_indent(vsb, 2); + PAN_CheckMagic(vsb, h, HTTP_MAGIC); VSB_printf(vsb, "ws[%s] = %p,\n", h->ws ? h->ws->id : "", h->ws); VSB_printf(vsb, "hdrs {\n"); VSB_indent(vsb, 2); @@ -195,6 +198,7 @@ pan_objcore(struct vsb *vsb, const char *typ, const struct objcore *oc) VSB_printf(vsb, "objcore[%s] = %p {\n", typ, oc); VSB_indent(vsb, 2); + PAN_CheckMagic(vsb, oc, OBJCORE_MAGIC); VSB_printf(vsb, "refcnt = %d,\n", oc->refcnt); VSB_printf(vsb, "flags = 0x%x,\n", oc->flags); VSB_printf(vsb, "exp_flags = 0x%x,\n", oc->exp_flags); @@ -224,6 +228,7 @@ pan_wrk(struct vsb *vsb, const struct worker *wrk) VSB_printf(vsb, "worker = %p {\n", wrk); VSB_indent(vsb, 2); + PAN_CheckMagic(vsb, wrk, WORKER_MAGIC); VSB_printf(vsb, "stack = {0x%jx -> 0x%jx},\n", (uintmax_t)wrk->stack_start, (uintmax_t)wrk->stack_end); pan_ws(vsb, wrk->aws); @@ -271,6 +276,7 @@ pan_busyobj(struct vsb *vsb, const struct busyobj *bo) VSB_printf(vsb, "busyobj = %p {\n", bo); VSB_indent(vsb, 2); + PAN_CheckMagic(vsb, bo, BUSYOBJ_MAGIC); pan_ws(vsb, bo->ws); VSB_printf(vsb, "refcnt = %u,\n", bo->refcount); VSB_printf(vsb, "retries = %d, ", bo->retries); @@ -324,7 +330,7 @@ pan_req(struct vsb *vsb, const struct req *req) VSB_printf(vsb, "req = %p {\n", req); VSB_indent(vsb, 2); - + PAN_CheckMagic(vsb, req, REQ_MAGIC); VSB_printf(vsb, "vxid = %u, ", VXID(req->vsl->wid)); switch (req->req_step) { @@ -393,10 +399,13 @@ pan_sess(struct vsb *vsb, const struct sess *sp) VSB_printf(vsb, "sp = %p {\n", sp); VSB_indent(vsb, 2); + PAN_CheckMagic(vsb, sp, SESS_MAGIC); VSB_printf(vsb, "fd = %d, vxid = %u,\n", sp->fd, VXID(sp->vxid)); AZ(SES_Get_client_ip(sp, &ci)); AZ(SES_Get_client_port(sp, &cp)); VSB_printf(vsb, "client = %s %s,\n", ci, cp); + VSB_printf(vsb, "t_open = %f,\n", sp->t_open); + VSB_printf(vsb, "t_idle = %f,\n", sp->t_idle); switch (sp->sess_step) { #define SESS_STEP(l, u) case S_STP_##u: stp = "S_STP_" #u; break; #include "tbl/steps.h" diff --git a/bin/varnishd/cache/cache_priv.h b/bin/varnishd/cache/cache_priv.h index 483d041..9a7ba9f 100644 --- a/bin/varnishd/cache/cache_priv.h +++ b/bin/varnishd/cache/cache_priv.h @@ -69,7 +69,6 @@ void BAN_NewObjCore(struct objcore *oc); void BAN_DestroyObj(struct objcore *oc); int BAN_CheckObject(struct worker *, struct objcore *, struct req *); - /* cache_busyobj.c */ void VBO_Init(void); diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c index 8803f35..00a27b6 100644 --- a/bin/varnishd/cache/cache_vcl.c +++ b/bin/varnishd/cache/cache_vcl.c @@ -98,10 +98,17 @@ VCL_Panic(struct vsb *vsb, const struct vcl *vcl) return; VSB_printf(vsb, "vcl = {\n"); VSB_indent(vsb, 2); - VSB_printf(vsb, "temp = %s\n", vcl->temp); + PAN_CheckMagic(vsb, vcl, VCL_MAGIC); + VSB_printf(vsb, "busy = %u\n", vcl->busy); + VSB_printf(vsb, "discard = %u,\n", vcl->discard); + VSB_printf(vsb, "state = %s,\n", vcl->state); + VSB_printf(vsb, "temp = %s,\n", vcl->temp); + VSB_printf(vsb, "conf = {\n"); + VSB_indent(vsb, 2); if (vcl->conf == NULL) { VSB_printf(vsb, "conf = NULL\n"); } else { + PAN_CheckMagic(vsb, vcl->conf, VCL_CONF_MAGIC); VSB_printf(vsb, "srcname = {\n"); VSB_indent(vsb, 2); for (i = 0; i < vcl->conf->nsrc; ++i) @@ -111,6 +118,8 @@ VCL_Panic(struct vsb *vsb, const struct vcl *vcl) } VSB_indent(vsb, -2); VSB_printf(vsb, "},\n"); + VSB_indent(vsb, -2); + VSB_printf(vsb, "},\n"); } /*--------------------------------------------------------------------*/ From lkarsten at varnish-software.com Tue Jun 14 14:27:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 16:27:09 +0200 Subject: [4.1] b7f35ba Teach VSB_quote() to hexdump. All zero bits will be elided to "0x0...0" Message-ID: commit b7f35ba4d181fe464e42cd39d5b725307527b4e2 Author: Poul-Henning Kamp Date: Wed Mar 9 23:35:04 2016 +0000 Teach VSB_quote() to hexdump. All zero bits will be elided to "0x0...0" Conflicts: include/vsb.h lib/libvarnish/vsb.c diff --git a/include/vsb.h b/include/vsb.h index 83fb602..2b242bc 100644 --- a/include/vsb.h +++ b/include/vsb.h @@ -76,8 +76,10 @@ char *VSB_data(const struct vsb *); ssize_t VSB_len(const struct vsb *); void VSB_delete(struct vsb *); void VSB_destroy(struct vsb **); +void VSB_quote(struct vsb *, const void *, int len, int how); #define VSB_QUOTE_NONL 1 -void VSB_quote(struct vsb *s, const char *p, int len, int how); +#define VSB_QUOTE_JSON 2 +#define VSB_QUOTE_HEX 4 void VSB_indent(struct vsb *, int); #ifdef __cplusplus }; diff --git a/lib/libvarnish/vsb.c b/lib/libvarnish/vsb.c index 1f360d5..b710693 100644 --- a/lib/libvarnish/vsb.c +++ b/lib/libvarnish/vsb.c @@ -500,14 +500,32 @@ VSB_destroy(struct vsb **s) * Quote a string */ void -VSB_quote(struct vsb *s, const char *p, int len, int how) +VSB_quote(struct vsb *s, const void *v, int len, int how) { + const char *p; const char *q; int quote = 0; + const unsigned char *u, *w; - (void)how; /* For future enhancements */ + assert(v != NULL); if (len == -1) - len = strlen(p); + len = strlen(v); + + if (how & VSB_QUOTE_HEX) { + u = v; + for (w = u; w < u + len; w++) + if (*w != 0x00) + break; + VSB_printf(s, "0x"); + if (w == u + len) { + VSB_printf(s, "0...0"); + } else { + for (w = u; w < u + len; w++) + VSB_printf(s, "%02x", *w); + } + return; + } + p = v; for (q = p; q < p + len; q++) { if (!isgraph(*q) || *q == '"' || *q == '\\') { From lkarsten at varnish-software.com Tue Jun 14 14:27:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 14 Jun 2016 16:27:09 +0200 Subject: [4.1] 2794a74 Only ... hexdumps if they are longer than 8 digits Message-ID: commit 2794a7421f6337715537738bde54dc198b4ff736 Author: Poul-Henning Kamp Date: Wed May 11 17:37:01 2016 +0000 Only ... hexdumps if they are longer than 8 digits diff --git a/lib/libvarnish/vsb.c b/lib/libvarnish/vsb.c index b710693..57549e7 100644 --- a/lib/libvarnish/vsb.c +++ b/lib/libvarnish/vsb.c @@ -517,7 +517,7 @@ VSB_quote(struct vsb *s, const void *v, int len, int how) if (*w != 0x00) break; VSB_printf(s, "0x"); - if (w == u + len) { + if (w == u + len && len > 4) { VSB_printf(s, "0...0"); } else { for (w = u; w < u + len; w++) From martin at varnish-software.com Tue Jun 14 15:13:09 2016 From: martin at varnish-software.com (Martin Blix Grydeland) Date: Tue, 14 Jun 2016 17:13:09 +0200 Subject: [master] f8110c0 Deal with fetch failures in ved_stripgzip Message-ID: commit f8110c098e0da201bf322699d5e366a9d64724bd Author: Martin Blix Grydeland Date: Mon May 23 14:31:36 2016 +0200 Deal with fetch failures in ved_stripgzip An object failing fetch that is waited upon by the ESI delivery code would cause asserts finding an empty object. If waiting for an object, check the status of the object and deliver zero bytes if failed. Fixes: #1953 diff --git a/bin/varnishd/cache/cache_esi_deliver.c b/bin/varnishd/cache/cache_esi_deliver.c index fa944d0..59323ef 100644 --- a/bin/varnishd/cache/cache_esi_deliver.c +++ b/bin/varnishd/cache/cache_esi_deliver.c @@ -711,6 +711,11 @@ ved_stripgzip(struct req *req, const struct boc *boc) /* OA_GZIPBITS is not valid until BOS_FINISHED */ if (boc != NULL) ObjWaitState(req->objcore, BOS_FINISHED); + if (req->objcore->flags & OC_F_FAILED) { + /* No way of signalling errors in the middle of + the ESI body. Omit this ESI fragment. */ + return; + } AN(ObjCheckFlag(req->wrk, req->objcore, OF_GZIPED)); diff --git a/bin/varnishtest/tests/r01953.vtc b/bin/varnishtest/tests/r01953.vtc new file mode 100644 index 0000000..b456782 --- /dev/null +++ b/bin/varnishtest/tests/r01953.vtc @@ -0,0 +1,29 @@ +varnishtest "#1953: ved_stripgzip and failed object" + +server s1 { + rxreq + txresp -gzipbody {<1>} + + rxreq + txresp -nolen -hdr "Content-Encoding: gzip" -hdr "Content-Length: 42" + # No body sent, will cause fetch error + # The delay allows time for the ESI deliver thread to get to the point + # where it is waiting for this body + delay 1 +} -start + +varnish v1 -vcl+backend { + sub vcl_backend_response { + if (bereq.url == "/") { + set beresp.do_esi = true; + } + } +} -start + +client c1 { + txreq -hdr "Accept-Encoding: gzip" + rxresp + expect resp.http.Content-Encoding == "gzip" + gunzip + expect resp.body == "<1>" +} -run From fgsch at lodoss.net Tue Jun 14 16:54:07 2016 From: fgsch at lodoss.net (Federico G. Schwindt) Date: Tue, 14 Jun 2016 18:54:07 +0200 Subject: [master] d9e16c9 director has been a vmod for a while Message-ID: commit d9e16c973d8a8f00b24f49c5ba88be2fb438483a Author: Federico G. Schwindt Date: Tue Jun 14 16:23:46 2016 +0100 director has been a vmod for a while Also remove OBE comment. diff --git a/lib/libvcc/vcc_parse.c b/lib/libvcc/vcc_parse.c index aa242f8..869979e 100644 --- a/lib/libvcc/vcc_parse.c +++ b/lib/libvcc/vcc_parse.c @@ -272,18 +272,6 @@ vcc_ParseFunction(struct vcc *tl) */ static void -vcc_ParseDirector(struct vcc *tl) -{ - VSB_printf(tl->sb, "\ndirectors are now in directors VMOD.\n"); - vcc_ErrWhere(tl, tl->t); -} - -/*-------------------------------------------------------------------- - * TODO when bumping the vcl version: - * #1603 remove return(fetch) from vcl_hit{} - */ - -static void vcc_ParseVcl(struct vcc *tl) { struct token *tok; @@ -324,7 +312,6 @@ static struct toplev { { "acl", vcc_ParseAcl }, { "sub", vcc_ParseFunction }, { "backend", vcc_ParseBackend }, - { "director", vcc_ParseDirector }, { "probe", vcc_ParseProbe }, { "import", vcc_ParseImport }, { "vcl", vcc_ParseVcl }, From fgsch at lodoss.net Tue Jun 14 17:22:09 2016 From: fgsch at lodoss.net (Federico G. Schwindt) Date: Tue, 14 Jun 2016 19:22:09 +0200 Subject: [master] 75f8e5e And fix previous commit Message-ID: commit 75f8e5ecc2ea639d1cb6e7eaa1d80fd67459b7e3 Author: Federico G. Schwindt Date: Tue Jun 14 18:17:14 2016 +0100 And fix previous commit I could have sworn I ran make check before pushing it. diff --git a/bin/varnishtest/tests/v00021.vtc b/bin/varnishtest/tests/v00021.vtc index 301384b..86ce100 100644 --- a/bin/varnishtest/tests/v00021.vtc +++ b/bin/varnishtest/tests/v00021.vtc @@ -65,15 +65,3 @@ varnish v1 -errvcl {Invalid return "deliver"} { return (deliver); } } - -varnish v1 -errvcl {directors are now in directors VMOD} { - backend b { .host = "127.0.0.1"; } - - director d1 random { - .backend b; - } - - sub vcl_recv { - set req.backend = d1; - } -} From phk at FreeBSD.org Wed Jun 15 08:42:08 2016 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Wed, 15 Jun 2016 10:42:08 +0200 Subject: [master] dbfa7b4 Nitpicking around the corners of vcl labels Message-ID: commit dbfa7b4233795ed5d4895dae2a57271460df2dff Author: Poul-Henning Kamp Date: Wed Jun 15 07:58:16 2016 +0000 Nitpicking around the corners of vcl labels diff --git a/bin/varnishd/mgt/mgt_vcl.c b/bin/varnishd/mgt/mgt_vcl.c index f9cc6bd..1308b97 100644 --- a/bin/varnishd/mgt/mgt_vcl.c +++ b/bin/varnishd/mgt/mgt_vcl.c @@ -437,7 +437,9 @@ mcf_vcl_discard(struct cli *cli, const char * const *av, void *priv) if (vp->label != NULL) { AN(vp->warm); VCLI_SetResult(cli, CLIS_PARAM); - VCLI_Out(cli, "Must remove label to discard VCL\n"); + VCLI_Out(cli, + "Cannot discard labeled (\"%s\") VCL program.\n", + vp->label->name); return; } (void)mgt_vcl_setstate(cli, vp, VCL_STATE_COLD); @@ -500,6 +502,12 @@ mcf_vcl_label(struct cli *cli, const char * const *av, void *priv) VCLI_Out(cli, "VCL labels cannot point to labels"); return; } + if (vpt->label != NULL) { + VCLI_SetResult(cli, CLIS_PARAM); + VCLI_Out(cli, "VCL already labeled (\"%s\")", + vpt->label->name); + return; + } vpl = mgt_vcl_byname(av[2]); if (vpl == NULL) vpl = mgt_vcl_add(av[2], NULL, VCL_STATE_LABEL); @@ -511,8 +519,10 @@ mcf_vcl_label(struct cli *cli, const char * const *av, void *priv) } vpl->warm = 1; if (vpl->label != NULL) { + assert(vpl->label->label == vpl); + /* XXX SET vp->label AUTO */ vpl->label->label = NULL; - /* XXX SET AUTO */ + vpl->label = NULL; } vpl->label = vpt; vpt->label = vpl; diff --git a/bin/varnishtest/tests/v00048.vtc b/bin/varnishtest/tests/v00048.vtc index e048ac7..744047d 100644 --- a/bin/varnishtest/tests/v00048.vtc +++ b/bin/varnishtest/tests/v00048.vtc @@ -31,7 +31,11 @@ client c1 { varnish v1 -cliok "vcl.list" varnish v1 -clierr 106 "vcl.label foo vcl0" +varnish v1 -cliok "vcl.label foo vcl2" +varnish v1 -clierr 106 "vcl.label bar vcl2" +varnish v1 -clierr 106 "vcl.discard vcl2" varnish v1 -cliok "vcl.label foo vcl1" +varnish v1 -clierr 106 "vcl.label vcl1 vcl2" varnish v1 -clierr 106 "vcl.state foo cold" varnish v1 -clierr 106 "vcl.label bar foo" varnish v1 -clierr 106 "vcl.discard vcl1" From hermunn at varnish-software.com Wed Jun 15 08:48:08 2016 From: hermunn at varnish-software.com (Pål Hermunn Johansen) Date: Wed, 15 Jun 2016 10:48:08 +0200 Subject: [4.1] c0a515b Deal with fetch failures in ved_stripgzip Message-ID: commit c0a515b061f6d75db188127b7b0da74c611144ac Author: Martin Blix Grydeland Date: Mon May 23 14:31:36 2016 +0200 Deal with fetch failures in ved_stripgzip An object failing fetch that is waited upon by the ESI delivery code would cause asserts finding an empty object. If waiting for an object, check the status of the object and deliver zero bytes if failed. Fixes: #1953 diff --git a/bin/varnishd/cache/cache_esi_deliver.c b/bin/varnishd/cache/cache_esi_deliver.c index 32c8b90..528704a 100644 --- a/bin/varnishd/cache/cache_esi_deliver.c +++ b/bin/varnishd/cache/cache_esi_deliver.c @@ -530,8 +530,14 @@ ved_stripgzip(struct req *req, struct busyobj *bo) CAST_OBJ_NOTNULL(ecx, req->transport_priv, ECX_MAGIC); preq = ecx->preq; + /* OA_GZIPBITS is not valid until BOS_FINISHED */ if (bo != NULL) VBO_waitstate(bo, BOS_FINISHED); + if (req->objcore->flags & OC_F_FAILED) { + /* No way of signalling errors in the middle of + the ESI body. Omit this ESI fragment. */ + return; + } AN(ObjCheckFlag(req->wrk, req->objcore, OF_GZIPED)); diff --git a/bin/varnishtest/tests/r01953.vtc b/bin/varnishtest/tests/r01953.vtc new file mode 100644 index 0000000..b456782 --- /dev/null +++ b/bin/varnishtest/tests/r01953.vtc @@ -0,0 +1,29 @@ +varnishtest "#1953: ved_stripgzip and failed object" + +server s1 { + rxreq + txresp -gzipbody {<1>} + + rxreq + txresp -nolen -hdr "Content-Encoding: gzip" -hdr "Content-Length: 42" + # No body sent, will cause fetch error + # The delay allows time for the ESI deliver thread to get to the point + # where it is waiting for this body + delay 1 +} -start + +varnish v1 -vcl+backend { + sub vcl_backend_response { + if (bereq.url == "/") { + set beresp.do_esi = true; + } + } +} -start + +client c1 { + txreq -hdr "Accept-Encoding: gzip" + rxresp + expect resp.http.Content-Encoding == "gzip" + gunzip + expect resp.body == "<1>" +} -run From lkarsten at varnish-software.com Wed Jun 15 11:19:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Wed, 15 Jun 2016 13:19:09 +0200 Subject: [4.1] 8c60ef6 Add changelog entry for #1953. Message-ID: commit 8c60ef6eb3275e8ef0d7e3f4df8a120a1ace2d28 Author: Lasse Karstensen Date: Wed Jun 15 13:04:48 2016 +0200 Add changelog entry for #1953. diff --git a/doc/changes.rst b/doc/changes.rst index 8b895bf..63bbe83 100644 --- a/doc/changes.rst +++ b/doc/changes.rst @@ -26,6 +26,7 @@ Bugs fixed * 1967_ - [ncsa] Remove implicit line feed when using formatfile * 1955_ - 4.1.x sometimes duplicates Age and Accept-Ranges headers * 1954_ - Correctly handle HTTP/1.1 EOF response +* 1953_ - Deal with fetch failures in ved_stripgzip * 1931_ - Allow VCL set Last-Modified to be used for I-M-S processing * 1928_ - req->task members must be set in case we get onto the waitinglist * 1924_ - Make std.log() and std.syslog() work from vcl_{init,fini} @@ -48,6 +49,7 @@ Bugs fixed .. _1967: https://github.com/varnishcache/varnish-cache/issues/1967 .. _1955: https://github.com/varnishcache/varnish-cache/issues/1955 .. _1954: https://github.com/varnishcache/varnish-cache/issues/1954 +.. _1953: https://github.com/varnishcache/varnish-cache/issues/1953 .. _1931: https://github.com/varnishcache/varnish-cache/issues/1931 .. _1928: https://github.com/varnishcache/varnish-cache/issues/1928 .. _1924: https://github.com/varnishcache/varnish-cache/issues/1924 From lkarsten at varnish-software.com Wed Jun 15 11:19:09 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Wed, 15 Jun 2016 13:19:09 +0200 Subject: [4.1] 1deaaba Prepare for 4.1.3-beta1. Message-ID: commit 1deaabada403ad4b1871b058f1b1edd518345403 Author: Lasse Karstensen Date: Wed Jun 15 13:05:24 2016 +0200 Prepare for 4.1.3-beta1. diff --git a/configure.ac b/configure.ac index 831bab9..9d84d9c 100644 --- a/configure.ac +++ b/configure.ac @@ -2,7 +2,7 @@ AC_PREREQ(2.59) AC_COPYRIGHT([Copyright (c) 2006 Verdens Gang AS Copyright (c) 2006-2016 Varnish Software]) AC_REVISION([$Id$]) -AC_INIT([Varnish], [4.1.3-dev], [varnish-dev at varnish-cache.org]) +AC_INIT([Varnish], [4.1.3-beta1], [varnish-dev at varnish-cache.org]) AC_CONFIG_SRCDIR(include/miniobj.h) AC_CONFIG_HEADERS([config.h]) AC_CONFIG_MACRO_DIR([m4]) diff --git a/doc/changes.rst b/doc/changes.rst index 63bbe83..676f3fd 100644 --- a/doc/changes.rst +++ b/doc/changes.rst @@ -1,5 +1,5 @@ ====================================== -Varnish Cache 4.1.3-beta1 (unreleased) +Varnish Cache 4.1.3-beta1 (2016-06-15) ====================================== Changes since 4.1.2: From fgsch at lodoss.net Wed Jun 15 11:41:10 2016 From: fgsch at lodoss.net (Federico G. Schwindt) Date: Wed, 15 Jun 2016 13:41:10 +0200 Subject: [4.1] 1c68d1d Fix build with bmake Message-ID: commit 1c68d1d421ef13d336e72d66cb2bc1378abc26b5 Author: Federico G. Schwindt Date: Tue May 3 16:43:05 2016 +0100 Fix build with bmake diff --git a/doc/graphviz/Makefile.am b/doc/graphviz/Makefile.am index 61c756b..dfbf7ab 100644 --- a/doc/graphviz/Makefile.am +++ b/doc/graphviz/Makefile.am @@ -1,5 +1,7 @@ # Makefile for graphviz outputs +SUFFIXES: .dot .pdf .svg + # for an out-of-tree build, sphinx needs the output in builddir # XXX is there a better way to do this? @@ -54,7 +56,7 @@ else @DOT@ -Tpdf -Gsize=$(SIZE) -Grotate=90 $< >$@ endif -%.pdf: %.dot +.dot.pdf: if ! HAVE_DOT @echo ================================================== @echo You need graphviz installed to generate pdf output @@ -64,7 +66,7 @@ else @DOT@ -Tpdf -Gsize=$(SIZE) $< >$@ endif -%.svg: %.dot +.dot.svg: if ! HAVE_DOT @echo ================================================== @echo You need graphviz installed to generate svg output From fgsch at lodoss.net Wed Jun 15 11:54:10 2016 From: fgsch at lodoss.net (Federico G. Schwindt) Date: Wed, 15 Jun 2016 13:54:10 +0200 Subject: [4.1] ce824e2 Add a very crude hack to detect what the default sphinx theme is called. Message-ID: commit ce824e24df695d73060e8bb6693992c93640b32e Author: Poul-Henning Kamp Date: Wed Mar 23 22:56:07 2016 +0000 Add a very crude hack to detect what the default sphinx theme is called. diff --git a/doc/sphinx/conf.py.in b/doc/sphinx/conf.py.in index 5e411b6..30c527a 100644 --- a/doc/sphinx/conf.py.in +++ b/doc/sphinx/conf.py.in @@ -91,7 +91,12 @@ pygments_style = 'sphinx' # The theme to use for HTML and HTML Help pages. Major themes that come with # Sphinx are currently 'default' and 'sphinxdoc'. -html_theme = 'default' + +import sphinx +if sphinx.__version__ >= '1.3.1': + html_theme = 'classic' +else: + html_theme = 'default' # Theme options are theme-specific and customize the look and feel of a theme # further. For a list of options available for each theme, see the From varnish-commit at varnish-cache.org Thu Jun 16 06:41:18 2016 From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org) Date: 16 Jun 2016 12:41:18 +0600 Subject: Celebrity Lifestyle Plan To Keep Yourself Healthy Message-ID: <004d01d1c79c$0359d52c$5020afbb$@varnish-cache.org> Are you tired of the excess weight and would like to get back on track without any painful efforts? Brand-new formula and advanced structure of our exclusive product will compliment your results and will help you to lose weight with no side-effects. Click Here! Stock is limited, make sure to get yours in order to have a beautiful and healthy body! -------------- next part -------------- An HTML attachment was scrubbed... URL: From lkarsten at varnish-software.com Thu Jun 16 07:37:07 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Thu, 16 Jun 2016 09:37:07 +0200 Subject: [master] 2f75b5b Add README.rst to the distribution. Message-ID: commit 2f75b5bd895c518d1a19b8380be1dd2dc0ce4a0b Author: Lasse Karstensen Date: Thu Jun 16 09:32:48 2016 +0200 Add README.rst to the distribution. diff --git a/Makefile.am b/Makefile.am index 04e245a..36056e4 100644 --- a/Makefile.am +++ b/Makefile.am @@ -9,7 +9,7 @@ m4dir = $(datadir)/aclocal m4_DATA = varnish.m4 CLEANFILES = cscope.in.out cscope.out cscope.po.out -EXTRA_DIST = README.Packaging LICENSE autogen.sh varnishapi.pc.in varnish.m4 +EXTRA_DIST = README.rst README.Packaging LICENSE autogen.sh varnishapi.pc.in varnish.m4 DISTCHECK_CONFIGURE_FLAGS = \ --enable-developer-warnings \ From lkarsten at varnish-software.com Thu Jun 16 07:44:07 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Thu, 16 Jun 2016 09:44:07 +0200 Subject: [4.1] 8e255c0 Add README.rst to the distribution. Message-ID: commit 8e255c01ac4dad0166b0c4d8835e855ad1012fbb Author: Lasse Karstensen Date: Thu Jun 16 09:32:48 2016 +0200 Add README.rst to the distribution. diff --git a/Makefile.am b/Makefile.am index b947ceb..e8e3a5e 100644 --- a/Makefile.am +++ b/Makefile.am @@ -9,7 +9,7 @@ m4dir = $(datadir)/aclocal m4_DATA = varnish.m4 CLEANFILES = cscope.in.out cscope.out cscope.po.out -EXTRA_DIST = README.Packaging LICENSE autogen.sh varnishapi.pc.in varnish.m4 +EXTRA_DIST = README.rst README.Packaging LICENSE autogen.sh varnishapi.pc.in varnish.m4 DISTCHECK_CONFIGURE_FLAGS = \ --enable-developer-warnings \ From nils.goroll at uplex.de Thu Jun 16 10:07:08 2016 From: nils.goroll at uplex.de (Nils Goroll) Date: Thu, 16 Jun 2016 12:07:08 +0200 Subject: [master] af36bc5 make the copyright section optional again Message-ID: commit af36bc5d9694a410342f9fd8ca3424fdd9a7bd69 Author: Nils Goroll Date: Sun Jun 12 21:10:38 2016 +0200 make the copyright section optional again VDD discussion result: - we want to make vcc files rst so that they can be read by rst parsers (github) right from the repo - vcc prototypes should be integrated into the rst - scn wants to look after this - we should make the copyright mandatory - for the time being, we tolerate a missing copyright to make existing vmod builds work again, the move to rst will change the interface anyway Closes #1978 diff --git a/lib/libvcc/vmodtool.py b/lib/libvcc/vmodtool.py index 176f618..6275e13 100755 --- a/lib/libvcc/vmodtool.py +++ b/lib/libvcc/vmodtool.py @@ -648,6 +648,7 @@ class vcc(object): self.pfx = outputprefix self.contents = [] self.commit_files = [] + self.copyright = None def commit(self): for i in self.commit_files: @@ -658,11 +659,15 @@ class vcc(object): a = a.split("\n$") for i in range(len(a)): b = a[i].split("\n", 1) + c = b[0].split(None, 1) + if i == 0: - self.copyright = a[0] + if c[0] == "$Module": + s_module(c, b[1:], self) + else: + self.copyright = a[0] continue - c = b[0].split(None, 1) - if i == 1: + if i == 1 and self.copyright != None: if c[0] != "Module": err("$Module must be first stanze") if c[0] == "Module": @@ -702,7 +707,8 @@ class vcc(object): for i in self.contents: i.rstfile(fo, man) - self.rst_copyright(fo) + if self.copyright != None: + self.rst_copyright(fo) fo.close() From nils.goroll at uplex.de Thu Jun 16 10:14:07 2016 From: nils.goroll at uplex.de (Nils Goroll) Date: Thu, 16 Jun 2016 12:14:07 +0200 Subject: [master] 9dea74f Prepare hsh_rush1 for rush limit Message-ID: commit 9dea74f7a3589fe6e3d40ca9f86f43ac3c95d457 Author: Nils Goroll Date: Wed May 11 12:20:11 2016 +0200 Prepare hsh_rush1 for rush limit diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c index c552416..ddb23fa 100644 --- a/bin/varnishd/cache/cache_hash.c +++ b/bin/varnishd/cache/cache_hash.c @@ -302,7 +302,7 @@ HSH_Insert(struct worker *wrk, const void *digest, struct objcore *oc, VTAILQ_INSERT_HEAD(&oh->objcs, oc, hsh_list); oc->flags &= ~OC_F_BUSY; if (!VTAILQ_EMPTY(&oh->waitinglist)) - hsh_rush1(wrk, oh, &rush, 0); + hsh_rush1(wrk, oh, &rush, HSH_RUSH_POLICY); Lck_Unlock(&oh->mtx); hsh_rush2(wrk, &rush); } @@ -519,17 +519,23 @@ HSH_Lookup(struct req *req, struct objcore **ocp, struct objcore **bocp, */ static void -hsh_rush1(struct worker *wrk, struct objhead *oh, struct rush *r, int all) +hsh_rush1(struct worker *wrk, struct objhead *oh, struct rush *r, int max) { unsigned u; struct req *req; + if (max == 0) + return; + if (max == HSH_RUSH_POLICY) + max = cache_param->rush_exponent; + assert(max > 0); + CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC); CHECK_OBJ_NOTNULL(oh, OBJHEAD_MAGIC); CHECK_OBJ_NOTNULL(r, RUSH_MAGIC); VTAILQ_INIT(&r->reqs); Lck_AssertHeld(&oh->mtx); - for (u = 0; u < cache_param->rush_exponent || all; u++) { + for (u = 0; u < max; u++) { req = VTAILQ_FIRST(&oh->waitinglist); if (req == NULL) break; @@ -706,7 +712,7 @@ HSH_Unbusy(struct worker *wrk, struct objcore *oc) VTAILQ_INSERT_HEAD(&oh->objcs, oc, hsh_list); oc->flags &= ~OC_F_BUSY; if (!VTAILQ_EMPTY(&oh->waitinglist)) - hsh_rush1(wrk, oh, &rush, 0); + hsh_rush1(wrk, oh, &rush, HSH_RUSH_POLICY); Lck_Unlock(&oh->mtx); if (!(oc->flags & OC_F_PRIVATE)) EXP_Insert(wrk, oc); @@ -862,7 +868,7 @@ HSH_DerefObjCore(struct worker *wrk, struct objcore **ocp) if (!r) VTAILQ_REMOVE(&oh->objcs, oc, hsh_list); if (!VTAILQ_EMPTY(&oh->waitinglist)) - hsh_rush1(wrk, oh, &rush, 0); + hsh_rush1(wrk, oh, &rush, HSH_RUSH_POLICY); Lck_Unlock(&oh->mtx); hsh_rush2(wrk, &rush); if (r != 0) @@ -913,7 +919,7 @@ HSH_DerefObjHead(struct worker *wrk, struct objhead **poh) */ Lck_Lock(&oh->mtx); while (oh->refcnt == 1 && !VTAILQ_EMPTY(&oh->waitinglist)) { - hsh_rush1(wrk, oh, &rush, 1); + hsh_rush1(wrk, oh, &rush, HSH_RUSH_ALL); Lck_Unlock(&oh->mtx); hsh_rush2(wrk, &rush); Lck_Lock(&oh->mtx); diff --git a/bin/varnishd/hash/hash_slinger.h b/bin/varnishd/hash/hash_slinger.h index 7743dec..d9e169a 100644 --- a/bin/varnishd/hash/hash_slinger.h +++ b/bin/varnishd/hash/hash_slinger.h @@ -110,6 +110,8 @@ void HSH_Unbusy(struct worker *, struct objcore *); void HSH_DeleteObjHead(struct worker *, struct objhead *oh); int HSH_DerefObjHead(struct worker *, struct objhead **poh); int HSH_DerefObjCore(struct worker *, struct objcore **ocp); +#define HSH_RUSH_POLICY -1 +#define HSH_RUSH_ALL INT_MAX #endif /* VARNISH_CACHE_CHILD */ extern const struct hash_slinger hsl_slinger; From nils.goroll at uplex.de Thu Jun 16 10:14:07 2016 From: nils.goroll at uplex.de (Nils Goroll) Date: Thu, 16 Jun 2016 12:14:07 +0200 Subject: [master] f2fdfea Rush waiters specifically Message-ID: commit f2fdfeade6b8edf3041e6d66bb0b376509894d24 Author: Nils Goroll Date: Tue May 31 14:42:28 2016 +0200 Rush waiters specifically Waiters hang off the objhead, so the simple fact that we manipulate some objcore on the same objhead does not imply that there are relevant news to the waiters. We now are more specific about when we wake up waiters and how many. Never for - expiry / ban / purge / nuke - stale objects we just happen to reference We do want to rush when - fetching a busy object failed or got aborted: We should wake up one waiter to re-attempt the fetch - we succeeded fetching a busy object or delivering: Rush more waiters by policy This fixes the general issue described in #1928 that we should not rush when expiring. Closes #1945 diff --git a/bin/varnishd/cache/cache_ban_lurker.c b/bin/varnishd/cache/cache_ban_lurker.c index 2712ef3..b6a5f89 100644 --- a/bin/varnishd/cache/cache_ban_lurker.c +++ b/bin/varnishd/cache/cache_ban_lurker.c @@ -265,7 +265,7 @@ ban_lurker_test_ban(struct worker *wrk, struct vsl_log *vsl, struct ban *bt, if (i) ObjSendEvent(wrk, oc, OEV_BANCHG); } - (void)HSH_DerefObjCore(wrk, &oc); + (void)HSH_DerefObjCore(wrk, &oc, 0); } } diff --git a/bin/varnishd/cache/cache_busyobj.c b/bin/varnishd/cache/cache_busyobj.c index cab4dd8..0e93dfb 100644 --- a/bin/varnishd/cache/cache_busyobj.c +++ b/bin/varnishd/cache/cache_busyobj.c @@ -171,7 +171,8 @@ VBO_ReleaseBusyObj(struct worker *wrk, struct busyobj **pbo) if (bo->fetch_objcore != NULL) { AN(wrk); - (void)HSH_DerefObjCore(wrk, &bo->fetch_objcore); + (void)HSH_DerefObjCore(wrk, &bo->fetch_objcore, + HSH_RUSH_POLICY); } VCL_Rel(&bo->vcl); diff --git a/bin/varnishd/cache/cache_expire.c b/bin/varnishd/cache/cache_expire.c index 969b117..f20c2a1 100644 --- a/bin/varnishd/cache/cache_expire.c +++ b/bin/varnishd/cache/cache_expire.c @@ -194,7 +194,7 @@ exp_inbox(struct exp_priv *ep, struct objcore *oc, unsigned flags) assert(oc->refcnt > 0); AZ(oc->exp_flags); ObjSendEvent(ep->wrk, oc, OEV_EXPIRE); - (void)HSH_DerefObjCore(ep->wrk, &oc); + (void)HSH_DerefObjCore(ep->wrk, &oc, 0); return; } @@ -271,7 +271,7 @@ exp_expire(struct exp_priv *ep, double now) VSLb(&ep->vsl, SLT_ExpKill, "EXP_Expired x=%u t=%.0f", ObjGetXID(ep->wrk, oc), EXP_Ttl(NULL, oc) - now); ObjSendEvent(ep->wrk, oc, OEV_EXPIRE); - (void)HSH_DerefObjCore(ep->wrk, &oc); + (void)HSH_DerefObjCore(ep->wrk, &oc, 0); } return (0); } diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c index 90235c1..0239b6e 100644 --- a/bin/varnishd/cache/cache_fetch.c +++ b/bin/varnishd/cache/cache_fetch.c @@ -952,7 +952,7 @@ vbf_fetch_thread(struct worker *wrk, void *priv) CHECK_OBJ_NOTNULL(bo->stale_oc, OBJCORE_MAGIC); /* We don't want the oc/stevedore ops in fetching thread */ if (!ObjCheckFlag(wrk, bo->stale_oc, OF_IMSCAND)) - (void)HSH_DerefObjCore(wrk, &bo->stale_oc); + (void)HSH_DerefObjCore(wrk, &bo->stale_oc, 0); } #endif @@ -984,7 +984,7 @@ vbf_fetch_thread(struct worker *wrk, void *priv) // AZ(bo->fetch_objcore->boc); // XXX if (bo->stale_oc != NULL) - (void)HSH_DerefObjCore(wrk, &bo->stale_oc); + (void)HSH_DerefObjCore(wrk, &bo->stale_oc, 0); wrk->vsl = NULL; HSH_DerefBoc(wrk, bo->fetch_objcore); @@ -1061,7 +1061,7 @@ VBF_Fetch(struct worker *wrk, struct req *req, struct objcore *oc, wrk->stats->fetch_no_thread++; (void)vbf_stp_fail(req->wrk, bo); if (bo->stale_oc != NULL) - (void)HSH_DerefObjCore(wrk, &bo->stale_oc); + (void)HSH_DerefObjCore(wrk, &bo->stale_oc, 0); HSH_DerefBoc(wrk, oc); SES_Rel(bo->sp); VBO_ReleaseBusyObj(wrk, &bo); @@ -1083,6 +1083,6 @@ VBF_Fetch(struct worker *wrk, struct req *req, struct objcore *oc, assert(oc->boc == boc); HSH_DerefBoc(wrk, oc); if (mode == VBF_BACKGROUND) - (void)HSH_DerefObjCore(wrk, &oc); + (void)HSH_DerefObjCore(wrk, &oc, HSH_RUSH_POLICY); THR_SetBusyobj(NULL); } diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c index ddb23fa..43ad55e 100644 --- a/bin/varnishd/cache/cache_hash.c +++ b/bin/varnishd/cache/cache_hash.c @@ -625,7 +625,7 @@ double keep) oc = ocp[n]; CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC); EXP_Rearm(oc, now, ttl, grace, keep); - (void)HSH_DerefObjCore(wrk, &oc); + (void)HSH_DerefObjCore(wrk, &oc, 0); } } while (more); WS_Release(wrk->aws, 0); @@ -843,7 +843,7 @@ HSH_DerefBoc(struct worker *wrk, struct objcore *oc) */ int -HSH_DerefObjCore(struct worker *wrk, struct objcore **ocp) +HSH_DerefObjCore(struct worker *wrk, struct objcore **ocp, int rushmax) { struct objcore *oc; struct objhead *oh; @@ -868,7 +868,7 @@ HSH_DerefObjCore(struct worker *wrk, struct objcore **ocp) if (!r) VTAILQ_REMOVE(&oh->objcs, oc, hsh_list); if (!VTAILQ_EMPTY(&oh->waitinglist)) - hsh_rush1(wrk, oh, &rush, HSH_RUSH_POLICY); + hsh_rush1(wrk, oh, &rush, rushmax); Lck_Unlock(&oh->mtx); hsh_rush2(wrk, &rush); if (r != 0) diff --git a/bin/varnishd/cache/cache_req_body.c b/bin/varnishd/cache/cache_req_body.c index ec57a42..acea250 100644 --- a/bin/varnishd/cache/cache_req_body.c +++ b/bin/varnishd/cache/cache_req_body.c @@ -67,7 +67,7 @@ vrb_pull(struct req *req, ssize_t maxsize, objiterate_f *func, void *priv) if (VFP_Open(vfc) < 0) { req->req_body_status = REQ_BODY_FAIL; HSH_DerefBoc(req->wrk, req->body_oc); - AZ(HSH_DerefObjCore(req->wrk, &req->body_oc)); + AZ(HSH_DerefObjCore(req->wrk, &req->body_oc, 0)); return (-1); } @@ -108,7 +108,7 @@ vrb_pull(struct req *req, ssize_t maxsize, objiterate_f *func, void *priv) VSLb_ts_req(req, "ReqBody", VTIM_real()); if (func != NULL) { HSH_DerefBoc(req->wrk, req->body_oc); - AZ(HSH_DerefObjCore(req->wrk, &req->body_oc)); + AZ(HSH_DerefObjCore(req->wrk, &req->body_oc, 0)); if (vfps != VFP_END) { req->req_body_status = REQ_BODY_FAIL; if (r == 0) @@ -123,7 +123,7 @@ vrb_pull(struct req *req, ssize_t maxsize, objiterate_f *func, void *priv) if (vfps != VFP_END) { req->req_body_status = REQ_BODY_FAIL; - AZ(HSH_DerefObjCore(req->wrk, &req->body_oc)); + AZ(HSH_DerefObjCore(req->wrk, &req->body_oc, 0)); return (-1); } @@ -239,7 +239,7 @@ VRB_Free(struct req *req) CHECK_OBJ_NOTNULL(req, REQ_MAGIC); if (req->body_oc != NULL) - AZ(HSH_DerefObjCore(req->wrk, &req->body_oc)); + AZ(HSH_DerefObjCore(req->wrk, &req->body_oc, 0)); } /*---------------------------------------------------------------------- diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c index a2d830c..f92c85b 100644 --- a/bin/varnishd/cache/cache_req_fsm.c +++ b/bin/varnishd/cache/cache_req_fsm.c @@ -109,7 +109,7 @@ cnt_deliver(struct worker *wrk, struct req *req) wrk->handling = VCL_RET_DELIVER; if (wrk->handling != VCL_RET_DELIVER) { - (void)HSH_DerefObjCore(wrk, &req->objcore); + (void)HSH_DerefObjCore(wrk, &req->objcore, HSH_RUSH_POLICY); http_Teardown(req->resp); switch (wrk->handling) { @@ -216,7 +216,7 @@ cnt_synth(struct worker *wrk, struct req *req) VSLb(req->vsl, SLT_Error, "Could not get storage"); req->doclose = SC_OVERLOAD; VSLb_ts_req(req, "Resp", W_TIM_real(wrk)); - (void)HSH_DerefObjCore(wrk, &req->objcore); + (void)HSH_DerefObjCore(wrk, &req->objcore, 1); http_Teardown(req->resp); return (REQ_FSM_DONE); } @@ -311,7 +311,7 @@ cnt_transmit(struct worker *wrk, struct req *req) if (boc != NULL) HSH_DerefBoc(wrk, req->objcore); - (void)HSH_DerefObjCore(wrk, &req->objcore); + (void)HSH_DerefObjCore(wrk, &req->objcore, HSH_RUSH_POLICY); http_Teardown(req->resp); return (REQ_FSM_DONE); @@ -335,7 +335,7 @@ cnt_fetch(struct worker *wrk, struct req *req) if (req->objcore->flags & OC_F_FAILED) { req->err_code = 503; req->req_step = R_STP_SYNTH; - (void)HSH_DerefObjCore(wrk, &req->objcore); + (void)HSH_DerefObjCore(wrk, &req->objcore, 1); AZ(req->objcore); return (REQ_FSM_MORE); } @@ -426,7 +426,8 @@ cnt_lookup(struct worker *wrk, struct req *req) req->stale_oc = oc; req->req_step = R_STP_MISS; } else { - (void)HSH_DerefObjCore(wrk, &req->objcore); + (void)HSH_DerefObjCore(wrk, &req->objcore, + HSH_RUSH_POLICY); /* * We don't have a busy object, so treat this * like a pass @@ -453,10 +454,10 @@ cnt_lookup(struct worker *wrk, struct req *req) } /* Drop our object, we won't need it */ - (void)HSH_DerefObjCore(wrk, &req->objcore); + (void)HSH_DerefObjCore(wrk, &req->objcore, HSH_RUSH_POLICY); if (busy != NULL) { - (void)HSH_DerefObjCore(wrk, &busy); + (void)HSH_DerefObjCore(wrk, &busy, 0); VRY_Clear(req); } @@ -482,7 +483,7 @@ cnt_miss(struct worker *wrk, struct req *req) wrk->stats->cache_miss++; VBF_Fetch(wrk, req, req->objcore, req->stale_oc, VBF_NORMAL); if (req->stale_oc != NULL) - (void)HSH_DerefObjCore(wrk, &req->stale_oc); + (void)HSH_DerefObjCore(wrk, &req->stale_oc, 0); req->req_step = R_STP_FETCH; return (REQ_FSM_MORE); case VCL_RET_SYNTH: @@ -499,8 +500,8 @@ cnt_miss(struct worker *wrk, struct req *req) } VRY_Clear(req); if (req->stale_oc != NULL) - (void)HSH_DerefObjCore(wrk, &req->stale_oc); - AZ(HSH_DerefObjCore(wrk, &req->objcore)); + (void)HSH_DerefObjCore(wrk, &req->stale_oc, 0); + AZ(HSH_DerefObjCore(wrk, &req->objcore, 1)); return (REQ_FSM_MORE); } @@ -766,7 +767,7 @@ cnt_purge(struct worker *wrk, struct req *req) HSH_Purge(wrk, boc->objhead, 0, 0, 0); - AZ(HSH_DerefObjCore(wrk, &boc)); + AZ(HSH_DerefObjCore(wrk, &boc, 1)); VCL_purge_method(req->vcl, wrk, req, NULL, NULL); switch (wrk->handling) { diff --git a/bin/varnishd/hash/hash_slinger.h b/bin/varnishd/hash/hash_slinger.h index d9e169a..006cab5 100644 --- a/bin/varnishd/hash/hash_slinger.h +++ b/bin/varnishd/hash/hash_slinger.h @@ -107,9 +107,9 @@ struct objhead { void HSH_Fail(struct objcore *); void HSH_Unbusy(struct worker *, struct objcore *); -void HSH_DeleteObjHead(struct worker *, struct objhead *oh); -int HSH_DerefObjHead(struct worker *, struct objhead **poh); -int HSH_DerefObjCore(struct worker *, struct objcore **ocp); +void HSH_DeleteObjHead(struct worker *, struct objhead *); +int HSH_DerefObjHead(struct worker *, struct objhead **); +int HSH_DerefObjCore(struct worker *, struct objcore **, int); #define HSH_RUSH_POLICY -1 #define HSH_RUSH_ALL INT_MAX #endif /* VARNISH_CACHE_CHILD */ diff --git a/bin/varnishd/storage/storage_lru.c b/bin/varnishd/storage/storage_lru.c index ab4844f..2d650c7 100644 --- a/bin/varnishd/storage/storage_lru.c +++ b/bin/varnishd/storage/storage_lru.c @@ -198,6 +198,6 @@ LRU_NukeOne(struct worker *wrk, struct lru *lru) ObjSlim(wrk, oc); VSLb(wrk->vsl, SLT_ExpKill, "LRU x=%u", ObjGetXID(wrk, oc)); - (void)HSH_DerefObjCore(wrk, &oc); // Ref from HSH_Snipe + (void)HSH_DerefObjCore(wrk, &oc, 0); // Ref from HSH_Snipe return (1); } diff --git a/bin/varnishd/storage/storage_persistent_silo.c b/bin/varnishd/storage/storage_persistent_silo.c index 61ae93b..b804a0f 100644 --- a/bin/varnishd/storage/storage_persistent_silo.c +++ b/bin/varnishd/storage/storage_persistent_silo.c @@ -173,7 +173,7 @@ smp_load_seg(struct worker *wrk, const struct smp_sc *sc, HSH_Insert(wrk, so->hash, oc, ban); AN(oc->ban); HSH_DerefBoc(wrk, oc); // XXX Keep it an stream resurrection? - (void)HSH_DerefObjCore(wrk, &oc); + (void)HSH_DerefObjCore(wrk, &oc, HSH_RUSH_POLICY); wrk->stats->n_vampireobject++; } Pool_Sumstat(wrk); From fgsch at lodoss.net Thu Jun 16 11:35:10 2016 From: fgsch at lodoss.net (Federico G. Schwindt) Date: Thu, 16 Jun 2016 13:35:10 +0200 Subject: [master] 84170d3 Zap extra newline Message-ID: commit 84170d3eed9c1185f93ffd0cab012cadca32c977 Author: Federico G. Schwindt Date: Thu Jun 16 12:24:21 2016 +0100 Zap extra newline diff --git a/bin/varnishd/mgt/mgt_jail_unix.c b/bin/varnishd/mgt/mgt_jail_unix.c index edb4653..03c9539 100644 --- a/bin/varnishd/mgt/mgt_jail_unix.c +++ b/bin/varnishd/mgt/mgt_jail_unix.c @@ -231,7 +231,7 @@ vju_subproc(enum jail_subproc_e jse) */ if (prctl(PR_SET_DUMPABLE, 1) != 0) { MGT_complain(C_INFO, - "Could not set dumpable bit. Core dumps turned off\n"); + "Could not set dumpable bit. Core dumps turned off"); } #endif } diff --git a/bin/varnishd/mgt/mgt_main.c b/bin/varnishd/mgt/mgt_main.c index e189e61..dcd45db 100644 --- a/bin/varnishd/mgt/mgt_main.c +++ b/bin/varnishd/mgt/mgt_main.c @@ -811,7 +811,7 @@ main(int argc, char * const *argv) assert(pfh == NULL || !VPF_Write(pfh)); - MGT_complain(C_DEBUG, "Platform: %s\n", VSB_data(vident) + 1); + MGT_complain(C_DEBUG, "Platform: %s", VSB_data(vident) + 1); mgt_pid = getpid(); /* daemon() changed this */ From fgsch at lodoss.net Thu Jun 16 11:39:08 2016 From: fgsch at lodoss.net (Federico G. Schwindt) Date: Thu, 16 Jun 2016 13:39:08 +0200 Subject: [master] 442eb46 Correct function name Message-ID: commit 442eb46006392b2e856bff5be367489f011494b6 Author: Federico G. Schwindt Date: Thu Jun 16 12:35:08 2016 +0100 Correct function name diff --git a/bin/varnishd/cache/cache_req_body.c b/bin/varnishd/cache/cache_req_body.c index acea250..2858e0f 100644 --- a/bin/varnishd/cache/cache_req_body.c +++ b/bin/varnishd/cache/cache_req_body.c @@ -182,7 +182,7 @@ VRB_Iterate(struct req *req, objiterate_f *func, void *priv) "Had failed reading req.body before."); return (-1); default: - WRONG("Wrong req_body_status in VRB_IterateReqBody()"); + WRONG("Wrong req_body_status in VRB_Iterate()"); } Lck_Lock(&req->sp->mtx); if (req->req_body_status == REQ_BODY_WITH_LEN || From hermunn at varnish-software.com Thu Jun 16 12:38:08 2016 From: hermunn at varnish-software.com (Pål Hermunn Johansen) Date: Thu, 16 Jun 2016 14:38:08 +0200 Subject: [master] d94b4a8 Introduce the parameter vsm_free_cooldown Message-ID: commit d94b4a880ec6ef132d6c7c6e6694a1e93cc80638 Author: P?l Hermunn Johansen Date: Wed Jun 15 15:21:22 2016 +0200 Introduce the parameter vsm_free_cooldown This new parameter specifies how long freed VSM memory is on the cooling list before it is actually freed. diff --git a/bin/varnishd/common/common_vsm.c b/bin/varnishd/common/common_vsm.c index 49ba1b6..c6af50d 100644 --- a/bin/varnishd/common/common_vsm.c +++ b/bin/varnishd/common/common_vsm.c @@ -41,11 +41,14 @@ #include #include "common.h" +#include "common/params.h" #include "vsm_priv.h" #include "vmb.h" #include "vtim.h" +extern volatile struct params * cache_param; + /*--------------------------------------------------------------------*/ struct vsm_range { @@ -299,7 +302,7 @@ VSM_common_free(struct vsm_sc *sc, void *ptr) vr2 = VTAILQ_NEXT(vr, list); VTAILQ_REMOVE(&sc->r_used, vr, list); VTAILQ_INSERT_TAIL(&sc->r_cooling, vr, list); - vr->cool = VTIM_real() + 60; /* XXX: param ? */ + vr->cool = VTIM_real() + cache_param->vsm_free_cooldown; if (vr2 != NULL) vr2->chunk->next = vr->chunk->next; else diff --git a/include/tbl/params.h b/include/tbl/params.h index c1340a5..61eca89 100644 --- a/include/tbl/params.h +++ b/include/tbl/params.h @@ -1332,6 +1332,21 @@ PARAM( /* func */ NULL ) +PARAM( + /* name */ vsm_free_cooldown, + /* typ */ timeout, + /* min */ "10.000", + /* max */ "600.000", + /* default */ "60.000", + /* units */ "seconds", + /* flags */ 0, + /* s-text */ + "How long VSM memory is kept warm after a deallocation " + "(granularity approximately 2 seconds).", + /* l-text */ "", + /* func */ NULL +) + #if 0 PARAM( /* name */ vcl_dir, From fgsch at lodoss.net Thu Jun 16 12:41:07 2016 From: fgsch at lodoss.net (Federico G. Schwindt) Date: Thu, 16 Jun 2016 14:41:07 +0200 Subject: [master] 9b216d2 Stringify boc_state Message-ID: commit 9b216d2e1cfb7d51debd79e38c9ce2bb54fed4a7 Author: Federico G. Schwindt Date: Thu Jun 16 13:32:04 2016 +0100 Stringify boc_state diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h index c1b90ae..8ee6f5a 100644 --- a/bin/varnishd/cache/cache.h +++ b/bin/varnishd/cache/cache.h @@ -360,12 +360,9 @@ struct storeobj { * The macro-states we expose outside the fetch code */ enum boc_state_e { - BOS_INVALID = 0, /* don't touch (yet) */ - BOS_REQ_DONE, /* beresp.* can be examined */ - BOS_PREP_STREAM, /* Prepare for streaming */ - BOS_STREAM, /* beresp.* can be examined */ - BOS_FINISHED, /* object is complete */ - BOS_FAILED, /* something went wrong */ +#define BOC_STATE(U, l) BOS_##U, +#include "tbl/boc_state.h" +#undef BOC_STATE }; struct boc { diff --git a/bin/varnishd/cache/cache_panic.c b/bin/varnishd/cache/cache_panic.c index b3171f9..285c9ff 100644 --- a/bin/varnishd/cache/cache_panic.c +++ b/bin/varnishd/cache/cache_panic.c @@ -94,6 +94,20 @@ reqbody_status_2str(enum req_body_state_e e) /*--------------------------------------------------------------------*/ +static const char * +boc_state_2str(enum boc_state_e e) +{ + switch (e) { +#define BOC_STATE(U,l) case BOS_##U: return(#l); +#include "tbl/boc_state.h" +#undef BOC_STATE + default: + return ("?"); + } +} + +/*--------------------------------------------------------------------*/ + const char * sess_close_2str(enum sess_close sc, int want_desc) { @@ -231,7 +245,7 @@ pan_boc(struct vsb *vsb, const struct boc *boc) VSB_indent(vsb, 2); PAN_CheckMagic(vsb, boc, BOC_MAGIC); VSB_printf(vsb, "refcnt = %u,\n", boc->refcount); - VSB_printf(vsb, "state = %d,\n", boc->state); + VSB_printf(vsb, "state = %s,\n", boc_state_2str(boc->state)); VSB_printf(vsb, "vary = %p,\n", boc->vary); VSB_printf(vsb, "stevedore_priv = %p,\n", boc->stevedore_priv); VSB_indent(vsb, -2); diff --git a/include/tbl/boc_state.h b/include/tbl/boc_state.h new file mode 100644 index 0000000..c3a2586 --- /dev/null +++ b/include/tbl/boc_state.h @@ -0,0 +1,36 @@ +/*- + * Copyright (c) 2016 Varnish Software AS + * All rights reserved. + * + * Author: Federico G. Schwindt + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +/*lint -save -e525 -e539 */ +BOC_STATE(INVALID, invalid) /* don't touch (yet) */ +BOC_STATE(REQ_DONE, req_done) /* beresp.* can be examined */ +BOC_STATE(PREP_STREAM, prep_stream) /* Prepare for streaming */ +BOC_STATE(STREAM, stream) /* beresp.* can be examined */ +BOC_STATE(FINISHED, finished) /* object is complete */ +BOC_STATE(FAILED, failed) /* something went wrong */ +/*lint -restore */ From martin at varnish-software.com Thu Jun 16 13:00:11 2016 From: martin at varnish-software.com (Martin Blix Grydeland) Date: Thu, 16 Jun 2016 15:00:11 +0200 Subject: [4.1] 278b361 Do not use an objcore flagged OC_F_PASS as a stale object Message-ID: commit 278b3612be256559330be6a05d800b2e30fcd364 Author: Martin Blix Grydeland Date: Fri May 27 14:30:18 2016 +0200 Do not use an objcore flagged OC_F_PASS as a stale object If an object marked OF_IMSCAND is used as the template during backend IMS revalidation, and the new object also is marked OC_F_PASS (hit-for-pass) in vcl_backend_response, the resulting object would (though ObjCopyAttr) have both OF_IMSCAND and OC_F_PASS. The object could then be used again for backend IMS revalidation (since it has the OF_IMSCAND flag), which would cause troubles as the body would by this time be deleted. Fix by not considering objects marked OC_F_PASS as a stale object candidate during lookup. Fixes: #1956 diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c index 266b4ba..6c5f95c 100644 --- a/bin/varnishd/cache/cache_req_fsm.c +++ b/bin/varnishd/cache/cache_req_fsm.c @@ -406,8 +406,9 @@ cnt_lookup(struct worker *wrk, struct req *req) if ((oc->flags & OC_F_PASS) && boc != NULL) { /* Treat a graced Hit-For-Pass as a miss */ + (void)HSH_DerefObjCore(wrk, &req->objcore); + AZ(req->objcore); req->objcore = boc; - req->stale_oc = oc; req->req_step = R_STP_MISS; return (REQ_FSM_MORE); } else if (oc->flags & OC_F_PASS) { diff --git a/bin/varnishtest/tests/r01956.vtc b/bin/varnishtest/tests/r01956.vtc new file mode 100644 index 0000000..9d2d505 --- /dev/null +++ b/bin/varnishtest/tests/r01956.vtc @@ -0,0 +1,53 @@ +varnishtest "#1956: graced hit-for-pass, backend ims and obj flags inheritance" + +server s1 { + rxreq + txresp -hdr {ETag: "foo"} -body "asdf" + + rxreq + expect req.http.if-none-match == {"foo"} + txresp -status 304 -hdr {ETag: "bar"} + + rxreq + expect req.http.if-none-match == "" + txresp -status 200 -hdr {ETag: "baz"} -body "asdf" +} -start + +varnish v1 -vcl+backend { + sub vcl_backend_response { + set beresp.ttl = 0.1s; + set beresp.keep = 60s; + if (beresp.http.etag == {""bar""}) { + set beresp.grace = 60s; + set beresp.uncacheable = true; + } else { + set beresp.grace = 0s; + } + return (deliver); + } +} -start + +client c1 { + timeout 5 + + txreq -hdr "cnt: 1" + rxresp + expect resp.http.etag == {"foo"} + expect resp.body == "asdf" + + delay 0.2 + + txreq -hdr "cnt: 2" + rxresp + expect resp.status == 200 + expect resp.http.etag == {"bar"} + expect resp.body == "asdf" + + delay 0.2 + + txreq -hdr "cnt: 3" + rxresp + expect resp.status == 200 + expect resp.http.etag == {"baz"} + expect resp.body == "asdf" +} -run From hermunn at varnish-software.com Thu Jun 16 13:07:07 2016 From: hermunn at varnish-software.com (Pål Hermunn Johansen) Date: Thu, 16 Jun 2016 15:07:07 +0200 Subject: [master] 3e72fdb Make the counter vsm_cooling act according to spec Message-ID: commit 3e72fdbbf673a7b4abeb4082c9b6b95ad730c3e9 Author: P?l Hermunn Johansen Date: Wed Jun 15 14:34:32 2016 +0200 Make the counter vsm_cooling act according to spec The counter g_cooling variable is now decremented when memory from the cooling list is freed, making the counter vsm_cooling update accordingly. Fixes: #1984 diff --git a/bin/varnishd/common/common_vsm.c b/bin/varnishd/common/common_vsm.c index c6af50d..860b32c 100644 --- a/bin/varnishd/common/common_vsm.c +++ b/bin/varnishd/common/common_vsm.c @@ -179,6 +179,7 @@ VSM_common_cleaner(struct vsm_sc *sc, struct VSC_C_main *stats) if (vr->cool > now) break; VTAILQ_REMOVE(&sc->r_cooling, vr, list); + sc->g_cooling -= vr->len; vsm_common_insert_free(sc, vr); } stats->vsm_free = sc->g_free; diff --git a/bin/varnishtest/tests/r01984.vtc b/bin/varnishtest/tests/r01984.vtc new file mode 100644 index 0000000..b8b4a12 --- /dev/null +++ b/bin/varnishtest/tests/r01984.vtc @@ -0,0 +1,18 @@ +varnishtest "g_cooldown counter test" + +server s1 {} -start +varnish v1 -vcl+backend {} -start +varnish v1 -cliok "param.set vsm_free_cooldown 10.0" + +# Load and use a new VCL, freeze the first +varnish v1 -vcl+backend {} +varnish v1 -cliok "vcl.state vcl1 cold" + +# the VSM_common_cleaner runs every 1.1 seconds. +# Wait for it to update vsm_cooling +delay 2 +varnish v1 -expect vsm_cooling > 0 + +# Wait long enough for all the cooling vsm memory to be transfered to the free list: +delay 12 +varnish v1 -expect vsm_cooling == 0 From fgsch at lodoss.net Thu Jun 16 13:19:08 2016 From: fgsch at lodoss.net (Federico G. Schwindt) Date: Thu, 16 Jun 2016 15:19:08 +0200 Subject: [master] ebb0cf4 Don't exit if creating the VCL directory fails Message-ID: commit ebb0cf443ef6e973795421a9b5be3875bea83435 Author: Federico G. Schwindt Date: Thu Jun 16 12:21:27 2016 +0100 Don't exit if creating the VCL directory fails Test by scn at . OK by @phk. Fixes #1963. diff --git a/bin/varnishd/mgt/mgt.h b/bin/varnishd/mgt/mgt.h index 20fd20d..929e130 100644 --- a/bin/varnishd/mgt/mgt.h +++ b/bin/varnishd/mgt/mgt.h @@ -91,7 +91,7 @@ enum jail_master_e { typedef int jail_init_f(char **); typedef void jail_master_f(enum jail_master_e); typedef void jail_subproc_f(enum jail_subproc_e); -typedef void jail_make_dir_f(const char *dname); +typedef int jail_make_dir_f(const char *dname); typedef void jail_fixfile_f(int fd); struct jail_tech { @@ -110,8 +110,8 @@ struct jail_tech { void VJ_Init(const char *j_arg); void VJ_master(enum jail_master_e jme); void VJ_subproc(enum jail_subproc_e jse); -void VJ_make_workdir(const char *dname); -void VJ_make_vcldir(const char *dname); +int VJ_make_workdir(const char *dname); +int VJ_make_vcldir(const char *dname); void VJ_fix_vsm_file(int fd); void VJ_fix_storage_file(int fd); diff --git a/bin/varnishd/mgt/mgt_jail.c b/bin/varnishd/mgt/mgt_jail.c index d75cc3b..4a71ea7 100644 --- a/bin/varnishd/mgt/mgt_jail.c +++ b/bin/varnishd/mgt/mgt_jail.c @@ -133,17 +133,15 @@ VJ_subproc(enum jail_subproc_e jse) vjt->subproc(jse); } -void +int VJ_make_workdir(const char *dname) { int fd; AN(dname); CHECK_OBJ_NOTNULL(vjt, JAIL_TECH_MAGIC); - if (vjt->make_workdir != NULL) { - vjt->make_workdir(dname); - return; - } + if (vjt->make_workdir != NULL) + return (vjt->make_workdir(dname)); VJ_master(JAIL_MASTER_FILE); if (mkdir(dname, 0755) < 0 && errno != EEXIST) @@ -162,22 +160,24 @@ VJ_make_workdir(const char *dname) AZ(close(fd)); AZ(unlink("_.testfile")); VJ_master(JAIL_MASTER_LOW); + return (0); } -void +int VJ_make_vcldir(const char *dname) { AN(dname); CHECK_OBJ_NOTNULL(vjt, JAIL_TECH_MAGIC); - if (vjt->make_vcldir != NULL) { - vjt->make_vcldir(dname); - return; - } + if (vjt->make_vcldir != NULL) + return (vjt->make_vcldir(dname)); - if (mkdir(dname, 0755) < 0 && errno != EEXIST) - ARGV_ERR("Cannot create VCL directory '%s': %s\n", + if (mkdir(dname, 0755) < 0 && errno != EEXIST) { + MGT_complain(C_ERR, "Cannot create VCL directory '%s': %s", dname, strerror(errno)); + return (1); + } + return (0); } void diff --git a/bin/varnishd/mgt/mgt_jail_unix.c b/bin/varnishd/mgt/mgt_jail_unix.c index 03c9539..147d00e 100644 --- a/bin/varnishd/mgt/mgt_jail_unix.c +++ b/bin/varnishd/mgt/mgt_jail_unix.c @@ -236,14 +236,19 @@ vju_subproc(enum jail_subproc_e jse) #endif } -static void __match_proto__(jail_make_dir_f) +static int __match_proto__(jail_make_dir_f) vju_make_vcldir(const char *dname) { AZ(seteuid(0)); - assert((mkdir(dname, 0755) == 0) || errno == EEXIST); + if (mkdir(dname, 0755) < 0 && errno != EEXIST) { + MGT_complain(C_ERR, "Cannot create VCL directory '%s': %s", + dname, strerror(errno)); + return (1); + } AZ(chown(dname, vju_uid, vju_gid)); AZ(seteuid(vju_uid)); + return (0); } diff --git a/bin/varnishd/mgt/mgt_main.c b/bin/varnishd/mgt/mgt_main.c index dcd45db..24a9d8b 100644 --- a/bin/varnishd/mgt/mgt_main.c +++ b/bin/varnishd/mgt/mgt_main.c @@ -743,7 +743,9 @@ main(int argc, char * const *argv) identify(i_arg); - VJ_make_workdir(dirname); + if (VJ_make_workdir(dirname)) + ARGV_ERR("Cannot create working directory (%s): %s\n", + dirname, strerror(errno)); /* XXX: should this be relative to the -n arg ? */ VJ_master(JAIL_MASTER_FILE); diff --git a/bin/varnishd/mgt/mgt_vcc.c b/bin/varnishd/mgt/mgt_vcc.c index 79bb4be..d7f3565 100644 --- a/bin/varnishd/mgt/mgt_vcc.c +++ b/bin/varnishd/mgt/mgt_vcc.c @@ -276,7 +276,14 @@ mgt_VccCompile(struct cli *cli, const char *vclname, const char *vclsrc, AZ(VSB_finish(sb)); vp.dir = strdup(VSB_data(sb)); AN(vp.dir); - VJ_make_vcldir(vp.dir); + + if (VJ_make_vcldir(vp.dir)) { + free(vp.dir); + VSB_destroy(&sb); + VCLI_Out(cli, "VCL compilation failed"); + VCLI_SetResult(cli, CLIS_PARAM); + return (NULL); + } VSB_clear(sb); VSB_printf(sb, "%s/%s", vp.dir, VGC_SRC); @@ -310,7 +317,7 @@ mgt_VccCompile(struct cli *cli, const char *vclname, const char *vclsrc, VCLI_Out(cli, "VCL compilation failed"); VCLI_SetResult(cli, CLIS_PARAM); } - return(NULL); + return (NULL); } free(vp.dir); diff --git a/bin/varnishtest/tests/b00032.vtc b/bin/varnishtest/tests/b00032.vtc index a36634f..48aa465 100644 --- a/bin/varnishtest/tests/b00032.vtc +++ b/bin/varnishtest/tests/b00032.vtc @@ -26,3 +26,4 @@ varnish v1 -clierr 106 {vcl.load vcl3 ./nonexistent.vcl} varnish v1 -cliok "vcl.discard vcl2" +varnish v1 -clierr 106 {vcl.load /invalid/name.vcl vcl4} From hermunn at varnish-software.com Thu Jun 16 14:12:11 2016 From: hermunn at varnish-software.com (Pål Hermunn Johansen) Date: Thu, 16 Jun 2016 16:12:11 +0200 Subject: [4.1] c5c2f9f Introduce the parameter vsm_free_cooldown Message-ID: commit c5c2f9f29015b42f64f9b4154f7ca1165e00454e Author: P?l Hermunn Johansen Date: Wed Jun 15 15:21:22 2016 +0200 Introduce the parameter vsm_free_cooldown This new parameter specifies how long freed VSM memory is on the cooling list before it is actually freed. diff --git a/bin/varnishd/common/common_vsm.c b/bin/varnishd/common/common_vsm.c index 0dda841..cb440c9 100644 --- a/bin/varnishd/common/common_vsm.c +++ b/bin/varnishd/common/common_vsm.c @@ -41,11 +41,14 @@ #include #include "common.h" +#include "common/params.h" #include "vsm_priv.h" #include "vmb.h" #include "vtim.h" +extern volatile struct params * cache_param; + /*--------------------------------------------------------------------*/ struct vsm_range { @@ -299,7 +302,7 @@ VSM_common_free(struct vsm_sc *sc, void *ptr) vr2 = VTAILQ_NEXT(vr, list); VTAILQ_REMOVE(&sc->r_used, vr, list); VTAILQ_INSERT_TAIL(&sc->r_cooling, vr, list); - vr->cool = VTIM_real() + 60; /* XXX: param ? */ + vr->cool = VTIM_real() + cache_param->vsm_free_cooldown; if (vr2 != NULL) vr2->chunk->next = vr->chunk->next; else diff --git a/include/tbl/params.h b/include/tbl/params.h index 42672d0..5c3f46b 100644 --- a/include/tbl/params.h +++ b/include/tbl/params.h @@ -1315,6 +1315,21 @@ PARAM( /* func */ NULL ) +PARAM( + /* name */ vsm_free_cooldown, + /* typ */ timeout, + /* min */ "10.000", + /* max */ "600.000", + /* default */ "60.000", + /* units */ "seconds", + /* flags */ 0, + /* s-text */ + "How long VSM memory is kept warm after a deallocation " + "(granularity approximately 2 seconds).", + /* l-text */ "", + /* func */ NULL +) + #if 0 PARAM( /* name */ vcl_dir, From hermunn at varnish-software.com Thu Jun 16 14:12:11 2016 From: hermunn at varnish-software.com (Pål Hermunn Johansen) Date: Thu, 16 Jun 2016 16:12:11 +0200 Subject: [4.1] 1c1603e Make the counter vsm_cooling act according to spec Message-ID: commit 1c1603ea1bef06a5a0003da7a54baa77207cc89a Author: P?l Hermunn Johansen Date: Wed Jun 15 14:34:32 2016 +0200 Make the counter vsm_cooling act according to spec The counter g_cooling variable is now decremented when memory from the cooling list is freed, making the counter vsm_cooling update accordingly. Fixes: #1984 diff --git a/bin/varnishd/common/common_vsm.c b/bin/varnishd/common/common_vsm.c index cb440c9..f24841c 100644 --- a/bin/varnishd/common/common_vsm.c +++ b/bin/varnishd/common/common_vsm.c @@ -179,6 +179,7 @@ VSM_common_cleaner(struct vsm_sc *sc, struct VSC_C_main *stats) if (vr->cool > now) break; VTAILQ_REMOVE(&sc->r_cooling, vr, list); + sc->g_cooling -= vr->len; vsm_common_insert_free(sc, vr); } stats->vsm_free = sc->g_free; diff --git a/bin/varnishtest/tests/r01984.vtc b/bin/varnishtest/tests/r01984.vtc new file mode 100644 index 0000000..b8b4a12 --- /dev/null +++ b/bin/varnishtest/tests/r01984.vtc @@ -0,0 +1,18 @@ +varnishtest "g_cooldown counter test" + +server s1 {} -start +varnish v1 -vcl+backend {} -start +varnish v1 -cliok "param.set vsm_free_cooldown 10.0" + +# Load and use a new VCL, freeze the first +varnish v1 -vcl+backend {} +varnish v1 -cliok "vcl.state vcl1 cold" + +# the VSM_common_cleaner runs every 1.1 seconds. +# Wait for it to update vsm_cooling +delay 2 +varnish v1 -expect vsm_cooling > 0 + +# Wait long enough for all the cooling vsm memory to be transfered to the free list: +delay 12 +varnish v1 -expect vsm_cooling == 0 From nils.goroll at uplex.de Thu Jun 16 14:21:09 2016 From: nils.goroll at uplex.de (Nils Goroll) Date: Thu, 16 Jun 2016 16:21:09 +0200 Subject: [master] b79d508 flexelint silencing Message-ID: commit b79d5086404164ba7dc4203dda56b0c84e4f43ec Author: Nils Goroll Date: Thu Jun 16 14:44:37 2016 +0200 flexelint silencing - tables get included multiple times - we use do ... while(1) all over the place - dont always include config.h for flint, source files need to include it so we want to see warnings if we missed to include it - tell flexelint where config.h is ok not to be used (sources with no actual code unless a feature is HAVE_ defined) diff --git a/bin/varnishd/flint.lnt b/bin/varnishd/flint.lnt index 190e4e1..53a15dd 100644 --- a/bin/varnishd/flint.lnt +++ b/bin/varnishd/flint.lnt @@ -118,6 +118,7 @@ -e458 // unprotected read -e728 // Symbol ... not explicitly initialized -e716 // while(1) ... +-e717 // do ... while(1) ... -e785 // Too few initializers for aggregate -e850 // for loop index variable '___' whose type category is '___' // is modified in body of the for loop that began at '___' diff --git a/bin/varnishd/mgt/mgt_jail_solaris.c b/bin/varnishd/mgt/mgt_jail_solaris.c index 2dd9322..1723e3e 100644 --- a/bin/varnishd/mgt/mgt_jail_solaris.c +++ b/bin/varnishd/mgt/mgt_jail_solaris.c @@ -202,6 +202,7 @@ * */ +//lint -e{766} #include "config.h" #ifdef HAVE_SETPPRIV diff --git a/bin/varnishd/storage/storage_umem.c b/bin/varnishd/storage/storage_umem.c index 4a1442f..eff2a6f 100644 --- a/bin/varnishd/storage/storage_umem.c +++ b/bin/varnishd/storage/storage_umem.c @@ -29,6 +29,7 @@ * Storage method based on umem_alloc(3MALLOC) */ +//lint -e{766} #include "config.h" #ifdef HAVE_LIBUMEM diff --git a/bin/varnishd/waiter/cache_waiter_epoll.c b/bin/varnishd/waiter/cache_waiter_epoll.c index f50ae46..3c2b42c 100644 --- a/bin/varnishd/waiter/cache_waiter_epoll.c +++ b/bin/varnishd/waiter/cache_waiter_epoll.c @@ -31,6 +31,7 @@ * write the session pointer to a pipe which the event engine monitors. */ +//lint -e{766} #include "config.h" #if defined(HAVE_EPOLL_CTL) diff --git a/bin/varnishd/waiter/cache_waiter_kqueue.c b/bin/varnishd/waiter/cache_waiter_kqueue.c index 02d7adc..e1c7642 100644 --- a/bin/varnishd/waiter/cache_waiter_kqueue.c +++ b/bin/varnishd/waiter/cache_waiter_kqueue.c @@ -28,6 +28,7 @@ * */ +//lint -e{766} #include "config.h" #if defined(HAVE_KQUEUE) diff --git a/bin/varnishd/waiter/cache_waiter_ports.c b/bin/varnishd/waiter/cache_waiter_ports.c index 87e4fc8..3b4f59c 100644 --- a/bin/varnishd/waiter/cache_waiter_ports.c +++ b/bin/varnishd/waiter/cache_waiter_ports.c @@ -61,6 +61,7 @@ * */ +//lint -e{766} #include "config.h" #if defined(HAVE_PORT_CREATE) diff --git a/flint.lnt b/flint.lnt index 1dcb478..05567ce 100644 --- a/flint.lnt +++ b/flint.lnt @@ -6,9 +6,9 @@ // build/config related -efile(451, "tbl/*.h") // No include guard +-efile(537, "tbl/*.h") // Repeated include -efile(451, ../../config.h) // No include guard --header(../../config.h) +libh(../../config.h) /////////////////////////////////////////////////////////////////////// diff --git a/lib/libvarnishcompat/daemon.c b/lib/libvarnishcompat/daemon.c index 6b3fbb1..3ab6b7d 100644 --- a/lib/libvarnishcompat/daemon.c +++ b/lib/libvarnishcompat/daemon.c @@ -29,6 +29,7 @@ * $FreeBSD: src/lib/libc/gen/daemon.c,v 1.8 2007/01/09 00:27:53 imp Exp $ */ +//lint -e{766} #include "config.h" #ifndef HAVE_DAEMON diff --git a/lib/libvarnishcompat/execinfo.c b/lib/libvarnishcompat/execinfo.c index a9b4471..17321b7 100644 --- a/lib/libvarnishcompat/execinfo.c +++ b/lib/libvarnishcompat/execinfo.c @@ -24,6 +24,7 @@ * SUCH DAMAGE. */ +//lint -e{766} #include "config.h" #ifndef HAVE_BACKTRACE From nils.goroll at uplex.de Thu Jun 16 14:21:09 2016 From: nils.goroll at uplex.de (Nils Goroll) Date: Thu, 16 Jun 2016 16:21:09 +0200 Subject: [master] af04edb superfluous include Message-ID: commit af04edb954eabee3078765799382265251237b70 Author: Nils Goroll Date: Thu Jun 16 14:57:47 2016 +0200 superfluous include Spotted by: Flexelint - phk yeah \o/ diff --git a/bin/varnishd/cache/cache_pool.h b/bin/varnishd/cache/cache_pool.h index d3b9e70..c400a31 100644 --- a/bin/varnishd/cache/cache_pool.h +++ b/bin/varnishd/cache/cache_pool.h @@ -29,8 +29,6 @@ * Private include file for the pool aware code. */ -#include "config.h" - VTAILQ_HEAD(taskhead, pool_task); struct pool { From hermunn at varnish-software.com Thu Jun 16 14:37:08 2016 From: hermunn at varnish-software.com (Pål Hermunn Johansen) Date: Thu, 16 Jun 2016 16:37:08 +0200 Subject: [4.1] 3cc70e7 Don't exit if creating the VCL directory fails Message-ID: commit 3cc70e758f08b3c7a9a922cd2f86c91a6bfbfb80 Author: Federico G. Schwindt Date: Thu Jun 16 12:21:27 2016 +0100 Don't exit if creating the VCL directory fails Test by scn at . OK by @phk. Fixes #1963. diff --git a/bin/varnishd/mgt/mgt.h b/bin/varnishd/mgt/mgt.h index fe96360..4e0e7bc 100644 --- a/bin/varnishd/mgt/mgt.h +++ b/bin/varnishd/mgt/mgt.h @@ -87,7 +87,7 @@ enum jail_master_e { typedef int jail_init_f(char **); typedef void jail_master_f(enum jail_master_e); typedef void jail_subproc_f(enum jail_subproc_e); -typedef void jail_make_dir_f(const char *dname); +typedef int jail_make_dir_f(const char *dname); typedef void jail_fixfile_f(int fd); struct jail_tech { @@ -106,8 +106,8 @@ struct jail_tech { void VJ_Init(const char *j_arg); void VJ_master(enum jail_master_e jme); void VJ_subproc(enum jail_subproc_e jse); -void VJ_make_workdir(const char *dname); -void VJ_make_vcldir(const char *dname); +int VJ_make_workdir(const char *dname); +int VJ_make_vcldir(const char *dname); void VJ_fix_vsm_file(int fd); void VJ_fix_storage_file(int fd); diff --git a/bin/varnishd/mgt/mgt_jail.c b/bin/varnishd/mgt/mgt_jail.c index d75cc3b..4a71ea7 100644 --- a/bin/varnishd/mgt/mgt_jail.c +++ b/bin/varnishd/mgt/mgt_jail.c @@ -133,17 +133,15 @@ VJ_subproc(enum jail_subproc_e jse) vjt->subproc(jse); } -void +int VJ_make_workdir(const char *dname) { int fd; AN(dname); CHECK_OBJ_NOTNULL(vjt, JAIL_TECH_MAGIC); - if (vjt->make_workdir != NULL) { - vjt->make_workdir(dname); - return; - } + if (vjt->make_workdir != NULL) + return (vjt->make_workdir(dname)); VJ_master(JAIL_MASTER_FILE); if (mkdir(dname, 0755) < 0 && errno != EEXIST) @@ -162,22 +160,24 @@ VJ_make_workdir(const char *dname) AZ(close(fd)); AZ(unlink("_.testfile")); VJ_master(JAIL_MASTER_LOW); + return (0); } -void +int VJ_make_vcldir(const char *dname) { AN(dname); CHECK_OBJ_NOTNULL(vjt, JAIL_TECH_MAGIC); - if (vjt->make_vcldir != NULL) { - vjt->make_vcldir(dname); - return; - } + if (vjt->make_vcldir != NULL) + return (vjt->make_vcldir(dname)); - if (mkdir(dname, 0755) < 0 && errno != EEXIST) - ARGV_ERR("Cannot create VCL directory '%s': %s\n", + if (mkdir(dname, 0755) < 0 && errno != EEXIST) { + MGT_complain(C_ERR, "Cannot create VCL directory '%s': %s", dname, strerror(errno)); + return (1); + } + return (0); } void diff --git a/bin/varnishd/mgt/mgt_jail_unix.c b/bin/varnishd/mgt/mgt_jail_unix.c index edb4653..3c9f658 100644 --- a/bin/varnishd/mgt/mgt_jail_unix.c +++ b/bin/varnishd/mgt/mgt_jail_unix.c @@ -236,14 +236,19 @@ vju_subproc(enum jail_subproc_e jse) #endif } -static void __match_proto__(jail_make_dir_f) +static int __match_proto__(jail_make_dir_f) vju_make_vcldir(const char *dname) { AZ(seteuid(0)); - assert((mkdir(dname, 0755) == 0) || errno == EEXIST); + if (mkdir(dname, 0755) < 0 && errno != EEXIST) { + MGT_complain(C_ERR, "Cannot create VCL directory '%s': %s", + dname, strerror(errno)); + return (1); + } AZ(chown(dname, vju_uid, vju_gid)); AZ(seteuid(vju_uid)); + return (0); } diff --git a/bin/varnishd/mgt/mgt_main.c b/bin/varnishd/mgt/mgt_main.c index 18656dc..7440302 100644 --- a/bin/varnishd/mgt/mgt_main.c +++ b/bin/varnishd/mgt/mgt_main.c @@ -739,7 +739,9 @@ main(int argc, char * const *argv) identify(i_arg); - VJ_make_workdir(dirname); + if (VJ_make_workdir(dirname)) + ARGV_ERR("Cannot create working directory (%s): %s\n", + dirname, strerror(errno)); /* XXX: should this be relative to the -n arg ? */ VJ_master(JAIL_MASTER_FILE); diff --git a/bin/varnishd/mgt/mgt_vcc.c b/bin/varnishd/mgt/mgt_vcc.c index eef0340..0aa3851 100644 --- a/bin/varnishd/mgt/mgt_vcc.c +++ b/bin/varnishd/mgt/mgt_vcc.c @@ -271,7 +271,14 @@ mgt_VccCompile(struct cli *cli, const char *vclname, const char *vclsrc, AZ(VSB_finish(sb)); vp.dir = strdup(VSB_data(sb)); AN(vp.dir); - VJ_make_vcldir(vp.dir); + + if (VJ_make_vcldir(vp.dir)) { + free(vp.dir); + VSB_destroy(&sb); + VCLI_Out(cli, "VCL compilation failed"); + VCLI_SetResult(cli, CLIS_PARAM); + return (NULL); + } VSB_clear(sb); VSB_printf(sb, "%s/%s", vp.dir, VGC_SRC); @@ -305,7 +312,7 @@ mgt_VccCompile(struct cli *cli, const char *vclname, const char *vclsrc, VCLI_Out(cli, "VCL compilation failed"); VCLI_SetResult(cli, CLIS_PARAM); } - return(NULL); + return (NULL); } free(vp.dir); diff --git a/bin/varnishtest/tests/b00032.vtc b/bin/varnishtest/tests/b00032.vtc index a36634f..48aa465 100644 --- a/bin/varnishtest/tests/b00032.vtc +++ b/bin/varnishtest/tests/b00032.vtc @@ -26,3 +26,4 @@ varnish v1 -clierr 106 {vcl.load vcl3 ./nonexistent.vcl} varnish v1 -cliok "vcl.discard vcl2" +varnish v1 -clierr 106 {vcl.load /invalid/name.vcl vcl4} From fgsch at lodoss.net Thu Jun 16 14:42:09 2016 From: fgsch at lodoss.net (Federico G. Schwindt) Date: Thu, 16 Jun 2016 16:42:09 +0200 Subject: [master] 0b41967 Document proxy_header Message-ID: commit 0b41967e07e6c716aa80ebeee52ac9bd958dcb8c Author: Federico G. Schwindt Date: Thu Jun 16 15:39:58 2016 +0100 Document proxy_header diff --git a/doc/sphinx/reference/vcl.rst b/doc/sphinx/reference/vcl.rst index 7e73504..7bd6c91 100644 --- a/doc/sphinx/reference/vcl.rst +++ b/doc/sphinx/reference/vcl.rst @@ -224,6 +224,10 @@ are available: probe Attach a probe to the backend. See `Probes`_ + proxy_header + The PROXY protocol version Varnish should use when connecting to + this backend. + max_connections Maximum number of open connections towards this backend. If Varnish reaches the maximum Varnish it will start failing From fgsch at lodoss.net Thu Jun 16 15:07:07 2016 From: fgsch at lodoss.net (Federico G. Schwindt) Date: Thu, 16 Jun 2016 17:07:07 +0200 Subject: [4.0] 57eaa1b Only look for libedit package if it is not native to the system Message-ID: commit 57eaa1bfc9014cdb77a3b17b62c65776172e4d58 Author: Poul-Henning Kamp Date: Mon Feb 9 09:20:25 2015 +0000 Only look for libedit package if it is not native to the system (...or something. Needless to say, somebody with autocrap clue should stare at this in disbelief.) diff --git a/configure.ac b/configure.ac index 4bd3ad0..765a8b4 100644 --- a/configure.ac +++ b/configure.ac @@ -140,32 +140,36 @@ fi AC_SUBST(PCRE_CFLAGS) AC_SUBST(PCRE_LIBS) -PKG_CHECK_MODULES([LIBEDIT], [libedit], - # having the module does not imply having the header - [AC_CHECK_HEADERS([editline/readline.h], - [AC_DEFINE([HAVE_LIBEDIT], [1], [Define if we have libedit])], - [AC_MSG_ERROR([Found libedit, but header file is missing. Hint: Install dev package?])])], - [ - # AX_LIB_READLINE overwrites LIBS which leads to every binary getting - # linked against libreadline uselessly. So we re-use LIBEDIT_LIBS which - # we have for libedit to add the lib specifically where needed - save_LIBS="${LIBS}" - AX_LIB_READLINE - LIBS="${save_LIBS}" - if test "$ax_cv_lib_readline" = "no"; then - AC_MSG_ERROR([neither libedit nor another readline compatible library found]) - fi - if test "x$ax_cv_lib_readline_history" != "xyes"; then - AC_MSG_ERROR([need readline history support]) - fi - LIBEDIT_LIBS="$ax_cv_lib_readline" +AC_CHECK_HEADERS([edit/readline/readline.h], + [AC_DEFINE([HAVE_LIBEDIT], [1], [Define if we have libedit]) + LIBEDIT_LIBS="-ledit"], + + [PKG_CHECK_MODULES([LIBEDIT], [libedit], + # having the module does not imply having the header + [AC_CHECK_HEADERS([editline/readline.h], + [AC_DEFINE([HAVE_LIBEDIT], [1], [Define if we have libedit])], + [AC_MSG_ERROR([Found libedit, but header file is missing. Hint: Install dev package?])])], + [ + # AX_LIB_READLINE overwrites LIBS which leads to every binary getting + # linked against libreadline uselessly. So we re-use LIBEDIT_LIBS which + # we have for libedit to add the lib specifically where needed + save_LIBS="${LIBS}" + AX_LIB_READLINE + LIBS="${save_LIBS}" + if test "$ax_cv_lib_readline" = "no"; then + AC_MSG_ERROR([neither libedit nor another readline compatible library found]) + fi + if test "x$ax_cv_lib_readline_history" != "xyes"; then + AC_MSG_ERROR([need readline history support]) + fi + LIBEDIT_LIBS="$ax_cv_lib_readline" + ]) ]) # Checks for header files. AC_HEADER_STDC AC_HEADER_SYS_WAIT AC_HEADER_TIME -AC_CHECK_HEADERS([edit/readline/readline.h]) AC_CHECK_HEADERS([sys/param.h]) AC_CHECK_HEADERS([sys/types.h]) AC_CHECK_HEADERS([sys/endian.h]) From fgsch at lodoss.net Thu Jun 16 16:17:07 2016 From: fgsch at lodoss.net (Federico G. Schwindt) Date: Thu, 16 Jun 2016 18:17:07 +0200 Subject: [master] 5e908bd Add boc_state.h Message-ID: commit 5e908bda3803a96bfda0e71d7cdf1635c6dfac65 Author: Federico G. Schwindt Date: Thu Jun 16 17:01:49 2016 +0100 Add boc_state.h diff --git a/include/Makefile.am b/include/Makefile.am index c833d6a..145d8b9 100644 --- a/include/Makefile.am +++ b/include/Makefile.am @@ -7,6 +7,7 @@ nobase_pkginclude_HEADERS = \ tbl/backend_poll.h \ tbl/ban_vars.h \ tbl/bo_flags.h \ + tbl/boc_state.h \ tbl/body_status.h \ tbl/cli_cmds.h \ tbl/debug_bits.h \ From fgsch at lodoss.net Thu Jun 16 16:17:07 2016 From: fgsch at lodoss.net (Federico G. Schwindt) Date: Thu, 16 Jun 2016 18:17:07 +0200 Subject: [master] 71365c5 Fix distcheck Message-ID: commit 71365c5dcb4042ce84e4b38e737a257e24a6c355 Author: Federico G. Schwindt Date: Thu Jun 16 17:02:09 2016 +0100 Fix distcheck diff --git a/doc/sphinx/Makefile.am b/doc/sphinx/Makefile.am index b9713d9..f929a6f 100644 --- a/doc/sphinx/Makefile.am +++ b/doc/sphinx/Makefile.am @@ -102,14 +102,15 @@ doctest: sphinx_prereq EXTRA_DIST = \ conf.py \ - index.rst \ - include \ + dev-guide \ glossary \ + include \ + index.rst \ installation \ phk \ tutorial \ users-guide \ - dev-guide \ + vtc-syntax.py \ whats-new dist-hook: From fgsch at lodoss.net Thu Jun 16 16:23:08 2016 From: fgsch at lodoss.net (Federico G. Schwindt) Date: Thu, 16 Jun 2016 18:23:08 +0200 Subject: [4.0] fb04e4c Add a very crude hack to detect what the default sphinx theme is called. Message-ID: commit fb04e4c4e4c5891a771f2829e08e96cf66f04299 Author: Poul-Henning Kamp Date: Wed Mar 23 22:56:07 2016 +0000 Add a very crude hack to detect what the default sphinx theme is called. diff --git a/doc/sphinx/conf.py.in b/doc/sphinx/conf.py.in index 5e411b6..30c527a 100644 --- a/doc/sphinx/conf.py.in +++ b/doc/sphinx/conf.py.in @@ -91,7 +91,12 @@ pygments_style = 'sphinx' # The theme to use for HTML and HTML Help pages. Major themes that come with # Sphinx are currently 'default' and 'sphinxdoc'. -html_theme = 'default' + +import sphinx +if sphinx.__version__ >= '1.3.1': + html_theme = 'classic' +else: + html_theme = 'default' # Theme options are theme-specific and customize the look and feel of a theme # further. For a list of options available for each theme, see the From varnish-commit at varnish-cache.org Thu Jun 16 17:40:52 2016 From: varnish-commit at varnish-cache.org (kj) Date: Fri, 17 Jun 2016 01:40:52 +0800 Subject: =?utf-8?B?dmFybmlzaC1jb21taXTvvJrkvIHkuJropoHnmoTmmK8=?= =?utf-8?B?5LuA5LmI77yf?= Message-ID: <20160617014106826103@powwl.com> varnish-commit: ??? 1????????????????????????? 2???????????????????/??????? 3???????/???????????????? 4??????????????????????????????????? 5??????????????????????? 6?????????????????????????????? 7?????????????????? 8?????????????????????????? 9??????????????? 10???????????????????????? ??????????????????????????????????????????????????????????????????????????????????????? ????????????????????????? -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ??????.xlsx Type: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet Size: 17127 bytes Desc: not available URL: From varnish-commit at varnish-cache.org Fri Jun 17 18:23:57 2016 From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org) Date: 18 Jun 2016 00:23:57 +0600 Subject: Get Back On Track With Our Breakthrough Solution Message-ID: <810002388.201606172441@varnish-cache.org> Have you been dreaming of a beautiful body, but had no results with diets, workouts and supplements? Our incredible solution with a breakthrough formula will deliver you the results you've been so eager to achive. Click Here! Powerful weight loss product that amplifies methabolism, stimulates positive mood and formulated for both men and women. -------------- next part -------------- An HTML attachment was scrubbed... URL: From varnish-commit at varnish-cache.org Fri Jun 17 21:38:27 2016 From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org) Date: 18 Jun 2016 00:38:27 +0300 Subject: Cooperarion with a large firm Message-ID: <003501d1c8e2$02879957$29459f9f$@varnish-cache.org> Hello! We are looking for employees working remotely. My name is Aurelio, am the personnel manager of a large International company. Most of the work you can do from home, that is, at a distance. Salary is $2500-$5000. If you are interested in this offer, please visit Our Site Best regards! -------------- next part -------------- An HTML attachment was scrubbed... URL: From phk at FreeBSD.org Sat Jun 18 17:18:06 2016 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Sat, 18 Jun 2016 19:18:06 +0200 Subject: [master] 59a8603 Polish while hunting for bugs Message-ID: commit 59a8603f4cf1254e307abbfcbe2d3afdfc763254 Author: Poul-Henning Kamp Date: Sat Jun 18 08:36:10 2016 +0000 Polish while hunting for bugs diff --git a/bin/varnishd/mgt/mgt_vcl.c b/bin/varnishd/mgt/mgt_vcl.c index 1308b97..d9cd043 100644 --- a/bin/varnishd/mgt/mgt_vcl.c +++ b/bin/varnishd/mgt/mgt_vcl.c @@ -54,12 +54,12 @@ struct vclprog { char *name; char *fname; unsigned warm; - char state[8]; + const char * state; double go_cold; struct vclprog *label; }; -static VTAILQ_HEAD(, vclprog) vclhead = VTAILQ_HEAD_INITIALIZER(vclhead); +static VTAILQ_HEAD(, vclprog) vclhead = VTAILQ_HEAD_INITIALIZER(vclhead); static struct vclprog *active_vcl; static struct vev *e_poker; @@ -70,16 +70,18 @@ mgt_vcl_add(const char *name, const char *libfile, const char *state) { struct vclprog *vp; + assert(state == VCL_STATE_WARM || + state == VCL_STATE_COLD || + state == VCL_STATE_AUTO || + state == VCL_STATE_LABEL); vp = calloc(sizeof *vp, 1); XXXAN(vp); REPLACE(vp->name, name); REPLACE(vp->fname, libfile); - if (strcmp(state, "cold")) - vp->warm = 1; - else - state = "auto"; + vp->state = state; - bprintf(vp->state, "%s", state); + if (vp->state != VCL_STATE_COLD) + vp->warm = 1; if (active_vcl == NULL) active_vcl = vp; @@ -137,7 +139,7 @@ mgt_vcl_setstate(struct cli *cli, struct vclprog *vp, const char *vs) if (vs == VCL_STATE_AUTO) { now = VTIM_mono(); vs = vp->warm ? VCL_STATE_WARM : VCL_STATE_COLD; - if (vp->go_cold > 0 && !strcmp(vp->state, "auto") && + if (vp->go_cold > 0 && vp->state == VCL_STATE_AUTO && vp->go_cold + mgt_param.vcl_cooldown < now) vs = VCL_STATE_COLD; } @@ -165,7 +167,7 @@ mgt_vcl_setstate(struct cli *cli, struct vclprog *vp, const char *vs) VCLI_Out(cli, "%s", p); } - free(p); + REPLACE(p, NULL); return (i); } @@ -188,10 +190,14 @@ mgt_new_vcl(struct cli *cli, const char *vclname, const char *vclsrc, } if (state == NULL) - state = "auto"; - - if (strcmp(state, "auto") && - strcmp(state, "cold") && strcmp(state, "warm")) { + state = VCL_STATE_AUTO; + else if (!strcmp(state, VCL_STATE_AUTO)) + state = VCL_STATE_AUTO; + else if (!strcmp(state, VCL_STATE_COLD)) + state = VCL_STATE_COLD; + else if (!strcmp(state, VCL_STATE_WARM)) + state = VCL_STATE_WARM; + else { VCLI_Out(cli, "State must be one of auto, cold or warm."); VCLI_SetResult(cli, CLIS_PARAM); return; @@ -214,7 +220,7 @@ mgt_new_vcl(struct cli *cli, const char *vclname, const char *vclsrc, VCLI_Out(cli, "%s", p); VCLI_SetResult(cli, CLIS_PARAM); } - free(p); + REPLACE(p, NULL); } /*--------------------------------------------------------------------*/ @@ -259,8 +265,7 @@ mgt_push_vcls_and_start(struct cli *cli, unsigned *status, char **p) if (mgt_cli_askchild(status, p, "vcl.load \"%s\" %s %d%s\n", vp->name, vp->fname, vp->warm, vp->state)) return (1); - free(*p); - *p = NULL; + REPLACE(*p, NULL); } VTAILQ_FOREACH(vp, &vclhead, list) { if (strcmp(vp->state, VCL_STATE_LABEL)) @@ -268,17 +273,14 @@ mgt_push_vcls_and_start(struct cli *cli, unsigned *status, char **p) if (mgt_cli_askchild(status, p, "vcl.label %s %s\n", vp->name, vp->label->name)) return (1); - free(*p); - *p = NULL; + REPLACE(*p, NULL); } if (mgt_cli_askchild(status, p, "vcl.use \"%s\"\n", active_vcl->name)) return (1); - free(*p); - *p = NULL; + REPLACE(*p, NULL); if (mgt_cli_askchild(status, p, "start\n")) return (1); - free(*p); - *p = NULL; + REPLACE(*p, NULL); return (0); } @@ -357,23 +359,23 @@ mcf_vcl_state(struct cli *cli, const char * const *av, void *priv) if (!strcmp(vp->state, av[3])) return; - if (!strcmp(av[3], "auto")) { - bprintf(vp->state, "%s", "auto"); + if (!strcmp(av[3], VCL_STATE_AUTO)) { + vp->state = VCL_STATE_AUTO; if (vp != active_vcl) { vp->go_cold = VTIM_mono(); (void)mgt_vcl_setstate(cli, vp, VCL_STATE_AUTO); } - } else if (!strcmp(av[3], "cold")) { + } else if (!strcmp(av[3], VCL_STATE_COLD)) { if (vp == active_vcl) { VCLI_Out(cli, "Cannot set the active VCL cold."); VCLI_SetResult(cli, CLIS_PARAM); return; } - bprintf(vp->state, "%s", "auto"); + vp->state = VCL_STATE_AUTO; (void)mgt_vcl_setstate(cli, vp, VCL_STATE_COLD); - } else if (!strcmp(av[3], "warm")) { + } else if (!strcmp(av[3], VCL_STATE_WARM)) { if (mgt_vcl_setstate(cli, vp, VCL_STATE_WARM) == 0) - bprintf(vp->state, "%s", av[3]); + vp->state = VCL_STATE_WARM; } else { VCLI_Out(cli, "State must be one of auto, cold or warm."); VCLI_SetResult(cli, CLIS_PARAM); @@ -410,7 +412,7 @@ mcf_vcl_use(struct cli *cli, const char * const *av, void *priv) (void)mgt_vcl_setstate(cli, vp2, VCL_STATE_AUTO); } } - free(p); + REPLACE(p, NULL); } static void __match_proto__(cli_func_t) @@ -472,7 +474,8 @@ mcf_vcl_list(struct cli *cli, const char * const *av, void *priv) VCLI_Out(cli, "%-10s %5s", vp == active_vcl ? "active" : "available", vp->state); - VCLI_Out(cli, "/%-8s", vp->warm ? "warm" : "cold"); + VCLI_Out(cli, "/%-8s", vp->warm ? + VCL_STATE_WARM : VCL_STATE_COLD); VCLI_Out(cli, " %6s %s", "", vp->name); if (vp->label != NULL) VCLI_Out(cli, " %s %s", @@ -526,8 +529,8 @@ mcf_vcl_label(struct cli *cli, const char * const *av, void *priv) } vpl->label = vpt; vpt->label = vpl; - if (!strcmp(vpt->state, "cold")) - strcpy(vpt->state, "auto"); + if (vpt->state == VCL_STATE_COLD) + vpt->state = VCL_STATE_AUTO; (void)mgt_vcl_setstate(cli, vpt, VCL_STATE_WARM); if (child_pid < 0) return; From phk at FreeBSD.org Sat Jun 18 17:18:06 2016 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Sat, 18 Jun 2016 19:18:06 +0200 Subject: [master] 68ac5a6 Some time ago I decided that it was more convenient if it were actually possible to locate the compiled VCL shared library based on the loaded VCL name, and removed the part of the subdirectory name which made it unique. Message-ID: commit 68ac5a6690f37393771f4ecf923b00286aa95a0e Author: Poul-Henning Kamp Date: Sat Jun 18 17:15:59 2016 +0000 Some time ago I decided that it was more convenient if it were actually possible to locate the compiled VCL shared library based on the loaded VCL name, and removed the part of the subdirectory name which made it unique. Bad idea. Add a comment to make sure I don't get that Idea again. Fixes #1933 diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c index 055c841..41ae747 100644 --- a/bin/varnishd/cache/cache_vcl.c +++ b/bin/varnishd/cache/cache_vcl.c @@ -299,6 +299,11 @@ VCL_Open(const char *fn, struct vsb *msg) AN(fn); AN(msg); +#ifdef RTLD_NOLOAD + /* Detect bogus caching by dlopen(3) */ + dlh = dlopen(fn, RTLD_NOW | RTLD_NOLOAD); + AZ(dlh); +#endif dlh = dlopen(fn, RTLD_NOW | RTLD_LOCAL); if (dlh == NULL) { VSB_printf(msg, "Could not load compiled VCL.\n"); diff --git a/bin/varnishd/mgt/mgt_vcc.c b/bin/varnishd/mgt/mgt_vcc.c index d7f3565..ca8ae94 100644 --- a/bin/varnishd/mgt/mgt_vcc.c +++ b/bin/varnishd/mgt/mgt_vcc.c @@ -47,6 +47,7 @@ #include "vcli_serve.h" #include "vfil.h" #include "vsub.h" +#include "vtim.h" struct vcc_priv { unsigned magic; @@ -272,7 +273,39 @@ mgt_VccCompile(struct cli *cli, const char *vclname, const char *vclsrc, vp.vclsrc = vclsrc; vp.vclsrcfile = vclsrcfile; - VSB_printf(sb, "vcl_%s", vclname); + /* + * The subdirectory must have a unique name to 100% certain evade + * the refcounting semantics of dlopen(3). + * + * Bad implementations of dlopen(3) think the shlib you are opening + * is the same, if the filename is the same as one already opened. + * + * Sensible implementations do a stat(2) and requires st_ino and + * st_dev to also match. + * + * A correct implementation would run on filesystems which tickle + * st_gen, and also insist that be the identical, before declaring + * a match. + * + * Since no correct implementations are known to exist, we are subject + * to really interesting races if you do something like: + * + * (running on 'boot' vcl) + * vcl.load foo /foo.vcl + * vcl.use foo + * few/slow requests + * vcl.use boot + * vcl.discard foo + * vcl.load foo /foo.vcl // dlopen(3) says "same-same" + * vcl.use foo + * + * Because discard of the first 'foo' lingers on non-zero reference + * count, and when it finally runs, it trashes the second 'foo' because + * dlopen(3) decided they were really the same thing. + * + * The Best way to reproduce ths is to have regexps in the VCL. + */ + VSB_printf(sb, "vcl_%s.%.9f", vclname, VTIM_real()); AZ(VSB_finish(sb)); vp.dir = strdup(VSB_data(sb)); AN(vp.dir); diff --git a/bin/varnishd/mgt/mgt_vcl.c b/bin/varnishd/mgt/mgt_vcl.c index d9cd043..d084353 100644 --- a/bin/varnishd/mgt/mgt_vcl.c +++ b/bin/varnishd/mgt/mgt_vcl.c @@ -92,16 +92,25 @@ mgt_vcl_add(const char *name, const char *libfile, const char *state) static void mgt_vcl_del(struct vclprog *vp) { - char dn[256]; + char *p; VTAILQ_REMOVE(&vclhead, vp, list); - if (strcmp(vp->state, VCL_STATE_LABEL)) - XXXAZ(unlink(vp->fname)); - bprintf(dn, "vcl_%s", vp->name); - VJ_master(JAIL_MASTER_FILE); - (void)rmdir(dn); // compiler droppings, eg gcov - VJ_master(JAIL_MASTER_LOW); - free(vp->fname); + if (strcmp(vp->state, VCL_STATE_LABEL)) { + AZ(unlink(vp->fname)); + p = strrchr(vp->fname, '/'); + AN(p); + *p = '\0'; + VJ_master(JAIL_MASTER_FILE); + /* + * This will fail if any files are dropped next to the library + * without us knowing. This happens for instance with GCOV. + * Assume developers know how to clean up after themselves + * (or alternatively: How to run out of disk space). + */ + (void)rmdir(vp->fname); + VJ_master(JAIL_MASTER_LOW); + free(vp->fname); + } free(vp->name); free(vp); } From varnish-commit at varnish-cache.org Sat Jun 18 22:21:01 2016 From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org) Date: 19 Jun 2016 01:21:01 +0300 Subject: Advanced Formula - Brand-New Progressive Product Message-ID: <937920218904722664331016@varnish-cache.org> Have you been dreaming of a beautiful body, but had no results with diets, workouts and supplements? Our incredible solution with a breakthrough formula will deliver you the results you've been so eager to achive. Click Here! Powerful weight loss product that amplifies methabolism, stimulates positive mood and formulated for both men and women. -------------- next part -------------- An HTML attachment was scrubbed... URL: From phk at FreeBSD.org Sat Jun 18 19:39:05 2016 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Sat, 18 Jun 2016 21:39:05 +0200 Subject: [master] ad58dd0 Forgot that GCC chokes on REPLACE(foo, NULL) Message-ID: commit ad58dd02c48034bd6e5484268dd6007329b3cf31 Author: Poul-Henning Kamp Date: Sat Jun 18 19:38:16 2016 +0000 Forgot that GCC chokes on REPLACE(foo, NULL) diff --git a/bin/varnishd/mgt/mgt_vcl.c b/bin/varnishd/mgt/mgt_vcl.c index d084353..61f8e7c 100644 --- a/bin/varnishd/mgt/mgt_vcl.c +++ b/bin/varnishd/mgt/mgt_vcl.c @@ -176,7 +176,7 @@ mgt_vcl_setstate(struct cli *cli, struct vclprog *vp, const char *vs) VCLI_Out(cli, "%s", p); } - REPLACE(p, NULL); + free(p); return (i); } @@ -229,7 +229,7 @@ mgt_new_vcl(struct cli *cli, const char *vclname, const char *vclsrc, VCLI_Out(cli, "%s", p); VCLI_SetResult(cli, CLIS_PARAM); } - REPLACE(p, NULL); + free(p); } /*--------------------------------------------------------------------*/ @@ -274,7 +274,8 @@ mgt_push_vcls_and_start(struct cli *cli, unsigned *status, char **p) if (mgt_cli_askchild(status, p, "vcl.load \"%s\" %s %d%s\n", vp->name, vp->fname, vp->warm, vp->state)) return (1); - REPLACE(*p, NULL); + free(*p); + *p = NULL; } VTAILQ_FOREACH(vp, &vclhead, list) { if (strcmp(vp->state, VCL_STATE_LABEL)) @@ -282,14 +283,17 @@ mgt_push_vcls_and_start(struct cli *cli, unsigned *status, char **p) if (mgt_cli_askchild(status, p, "vcl.label %s %s\n", vp->name, vp->label->name)) return (1); - REPLACE(*p, NULL); + free(*p); + *p = NULL; } if (mgt_cli_askchild(status, p, "vcl.use \"%s\"\n", active_vcl->name)) return (1); - REPLACE(*p, NULL); + free(*p); + *p = NULL; if (mgt_cli_askchild(status, p, "start\n")) return (1); - REPLACE(*p, NULL); + free(*p); + *p = NULL; return (0); } @@ -421,7 +425,7 @@ mcf_vcl_use(struct cli *cli, const char * const *av, void *priv) (void)mgt_vcl_setstate(cli, vp2, VCL_STATE_AUTO); } } - REPLACE(p, NULL); + free(p); } static void __match_proto__(cli_func_t) From varnish-commit at varnish-cache.org Sat Jun 18 18:39:14 2016 From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org) Date: 18 Jun 2016 12:39:14 -0600 Subject: Healthy Lifestyle Plan Will Help Your Get Back In Shape Message-ID: <001501d1c995$04d263c3$c0b555a3$@varnish-cache.org> Have you been dreaming of a beautiful body, but had no results with diets, workouts and supplements? Our incredible solution with a breakthrough formula will deliver you the results you've been so eager to achive. Click Here! Powerful weight loss product that amplifies methabolism, stimulates positive mood and formulated for both men and women. -------------- next part -------------- An HTML attachment was scrubbed... URL: From varnish-commit at varnish-cache.org Sun Jun 19 13:47:53 2016 From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org) Date: 19 Jun 2016 20:47:53 +0700 Subject: The beautiful work in crisis! Message-ID: <9DB0DE5CF1F332735E30B21F1DDC9DB0@varnish-cache.org> Hello! We are looking for employees working remotely. My name is Tisha, am the personnel manager of a large International company. Most of the work you can do from home, that is, at a distance. Salary is $2500-$5000. If you are interested in this offer, please visit Our Site Best regards! -------------- next part -------------- An HTML attachment was scrubbed... URL: From varnish-commit at varnish-cache.org Sun Jun 19 20:06:55 2016 From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org) Date: 20 Jun 2016 04:06:55 +0800 Subject: Most Effective Nutrition Diet Designed For Women Message-ID: <003e01d1ca6a$02b41739$d0d91db7$@varnish-cache.org> Have you been dreaming of a beautiful body, but had no results with diets, workouts and supplements? Our incredible solution with a breakthrough formula will deliver you the results you've been so eager to achive. Click Here! Powerful weight loss product that amplifies methabolism, stimulates positive mood and formulated for both men and women. -------------- next part -------------- An HTML attachment was scrubbed... URL: From fgsch at lodoss.net Sun Jun 19 13:09:05 2016 From: fgsch at lodoss.net (Federico G. Schwindt) Date: Sun, 19 Jun 2016 15:09:05 +0200 Subject: [master] a167820 Correct comments Message-ID: commit a167820281262fdec978ebb3c09405ad2e54ca9c Author: Federico G. Schwindt Date: Sun Jun 19 13:55:00 2016 +0100 Correct comments diff --git a/lib/libvarnish/vss.c b/lib/libvarnish/vss.c index 3d40473..251b420 100644 --- a/lib/libvarnish/vss.c +++ b/lib/libvarnish/vss.c @@ -56,7 +56,7 @@ * "0.0.0.0" - "0.0.0.0:80" * "[::1]" - "[::1]:80" * "[::]" - "[::]:80" - * "::1" + * "::1" - "[::1]:80" * * See also RFC5952 */ @@ -80,7 +80,10 @@ vss_parse(char *str, char **addr, char **port) if (*p != ' ' && *p != ':') return ("IPv6 address has wrong port separator"); } else { - /* IPv4 address of the form 127.0.0.1:80, or non-numeric */ + /* + * IPv4 address of the form 127.0.0.1:80, IPv6 address + * without port or non-numeric. + */ *addr = str; p = strchr(str, ' '); if (p == NULL) From fgsch at lodoss.net Sun Jun 19 13:09:05 2016 From: fgsch at lodoss.net (Federico G. Schwindt) Date: Sun, 19 Jun 2016 15:09:05 +0200 Subject: [master] 66d14ff Add -proxy1 and -proxy2 support to varnishtest Message-ID: commit 66d14ff3230122687f7277eb75ea5010cfc9082b Author: Federico G. Schwindt Date: Sun Jun 19 14:04:47 2016 +0100 Add -proxy1 and -proxy2 support to varnishtest Fixes #1989. diff --git a/bin/varnishtest/Makefile.am b/bin/varnishtest/Makefile.am index 9d8c0b5..d6dbc10 100644 --- a/bin/varnishtest/Makefile.am +++ b/bin/varnishtest/Makefile.am @@ -23,19 +23,20 @@ AM_CPPFLAGS = \ bin_PROGRAMS = varnishtest varnishtest_SOURCES = \ + programs.h \ + vmods.h \ vtc.c \ vtc.h \ - vmods.h \ - programs.h \ vtc_barrier.c \ vtc_client.c \ vtc_http.c \ - vtc_main.c \ vtc_log.c \ - vtc_server.c \ - vtc_varnish.c \ vtc_logexp.c \ - vtc_process.c + vtc_main.c \ + vtc_process.c \ + vtc_proxy.c \ + vtc_server.c \ + vtc_varnish.c varnishtest_LDADD = \ $(top_builddir)/lib/libvarnish/libvarnish.la \ diff --git a/bin/varnishtest/tests/o00000.vtc b/bin/varnishtest/tests/o00000.vtc index 1bf1e1d..5bd0a12 100644 --- a/bin/varnishtest/tests/o00000.vtc +++ b/bin/varnishtest/tests/o00000.vtc @@ -140,8 +140,7 @@ logexpect l1 -v v1 { } -start # Finally try something which works... -client c1 { - send "PROXY TCP4 1.2.3.4 5.6.7.8 1234 5678\r\n" +client c1 -proxy1 "1.2.3.4:1234 5.6.7.8:5678" { txreq -url /1 rxresp expect resp.http.url == "/1" @@ -156,8 +155,7 @@ client c1 { } -run delay .1 -client c1 { - send "PROXY TCP6 1:f::2 5:a::8 1234 5678\r\n" +client c1 -proxy1 "[1:f::2]:1234 [5:a::8]:5678" { txreq -url /2 rxresp expect resp.http.url == "/2" @@ -175,14 +173,14 @@ delay .1 logexpect l1 -wait # Try with appended request (See also: #1728) -client c1 { +client c2 { send "PROXY TCP6 1:f::3 5:a::8 1234 5678\r\nGET /3 HTTP/1.1\r\nHdr1: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\r\n\r\n" rxresp expect resp.http.url == "/3" } -run # Malformed (missing \r) -client c1 { +client c2 { send "PROXY TCP4 1.2.3.4 5.6.7.8 1234 5678\n" expect_close } -run @@ -190,7 +188,7 @@ delay .1 # Malformed, too long (106) # NB: Should check VSL for proper disposal -client c1 { +client c2 { send "PROXY TCP4 1.2.3.4 5.6.7.8 1234 5678 \r\n" expect_close } -run @@ -198,7 +196,7 @@ delay .1 # Malformed, too long (107) # NB: Should check VSL for proper disposal -client c1 { +client c2 { send "PROXY TCP4 1.2.3.4 5.6.7.8 1234 5678 \r\n" expect_close } -run @@ -206,7 +204,7 @@ delay .1 # Malformed, too long (108) # NB: Should check VSL for proper disposal -client c1 { +client c2 { send "PROXY TCP4 1.2.3.4 5.6.7.8 1234 5678 \r\n" expect_close } -run diff --git a/bin/varnishtest/tests/o00001.vtc b/bin/varnishtest/tests/o00001.vtc index c906963..02ff279 100644 --- a/bin/varnishtest/tests/o00001.vtc +++ b/bin/varnishtest/tests/o00001.vtc @@ -120,14 +120,8 @@ client c1 { } -run delay .1 -client c1 { - # good IPv4 - sendhex "0d 0a 0d 0a 00 0d 0a 51 55 49 54 0a" - sendhex "21 11 00 0c" - sendhex "01 02 03 04" - sendhex "05 06 07 08" - sendhex "09 0a" - sendhex "0b 0c" +# good IPv4 +client c1 -proxy2 "1.2.3.4:2314 5.6.7.8:2828" { txreq rxresp expect resp.status == 200 @@ -144,14 +138,9 @@ client c1 { } -run delay .1 -client c1 { - # good IPv6 - sendhex "0d 0a 0d 0a 00 0d 0a 51 55 49 54 0a" - sendhex "21 21 00 24" - sendhex "01 02 03 04 05 06 00 00 00 00 00 00 0d 0e 0f 10" - sendhex "81 82 83 84 85 86 00 00 00 00 00 00 8d 8e 8f 80" - sendhex "09 0a" - sendhex "0b 0c" +# good IPv6 +client c1 \ + -proxy2 "[102:304:506::d0e:f10]:2314 [8182:8384:8586::8d8e:8f80]:2828" { txreq rxresp expect resp.status == 200 diff --git a/bin/varnishtest/tests/o00002.vtc b/bin/varnishtest/tests/o00002.vtc index 5c6fee0..13fba7d 100644 --- a/bin/varnishtest/tests/o00002.vtc +++ b/bin/varnishtest/tests/o00002.vtc @@ -70,32 +70,28 @@ varnish v2 -proto PROXY -vcl { } } -start -client c1 -connect ${v2_sock} { - send "PROXY TCP4 1.2.3.4 5.6.7.8 1111 5678\r\n" +client c1 -connect ${v2_sock} -proxy1 "1.2.3.4:1111 5.6.7.8:5678" { txreq -url /1 rxresp expect resp.body == "proxy1" } -run delay .2 -client c1 -connect ${v2_sock} { - send "PROXY TCP4 1.2.3.4 5.6.7.8 2222 5678\r\n" +client c1 -connect ${v2_sock} -proxy1 "1.2.3.4:2222 5.6.7.8:5678" { txreq -url /2 rxresp expect resp.body == "proxy2" } -run delay .2 -client c1 -connect ${v2_sock} { - send "PROXY TCP6 1:f::2 5:a::8 3333 5678\r\n" +client c1 -connect ${v2_sock} -proxy1 "[1:f::2]:3333 [5:a::8]:5678" { txreq -url /3 rxresp expect resp.body == "proxy3" } -run delay .2 -client c1 -connect ${v2_sock} { - send "PROXY TCP6 1:f::2 5:a::8 4444 5678\r\n" +client c1 -connect ${v2_sock} -proxy1 "[1:f::2]:4444 [5:a::8]:5678" { txreq -url /4 rxresp expect resp.body == "proxy4" diff --git a/bin/varnishtest/tests/o00003.vtc b/bin/varnishtest/tests/o00003.vtc index d821ce7..c883131 100644 --- a/bin/varnishtest/tests/o00003.vtc +++ b/bin/varnishtest/tests/o00003.vtc @@ -14,8 +14,7 @@ varnish v1 -proto PROXY -vcl+backend { } } -start -client c1 { - send "PROXY TCP4 1.2.3.4 5.6.7.8 1111 5678\r\n" +client c1 -proxy1 "1.2.3.4:1111 5.6.7.8:5678" { txreq rxresp expect resp.http.li == ${v1_addr} diff --git a/bin/varnishtest/vtc.h b/bin/varnishtest/vtc.h index 23095d4..b2b730a 100644 --- a/bin/varnishtest/vtc.h +++ b/bin/varnishtest/vtc.h @@ -43,6 +43,7 @@ struct vtclog; struct cmds; +struct suckaddr; #define CMD_ARGS \ char * const *av, void *priv, const struct cmds *cmd, struct vtclog *vl @@ -90,6 +91,9 @@ void vtc_dump(struct vtclog *vl, int lvl, const char *pfx, void vtc_hexdump(struct vtclog *vl, int lvl, const char *pfx, const unsigned char *str, int len); +int vtc_send_proxy(int fd, int version, const struct suckaddr *sac, + const struct suckaddr *sas); + int exec_file(const char *fn, const char *script, const char *tmpdir, char *logbuf, unsigned loglen); diff --git a/bin/varnishtest/vtc_client.c b/bin/varnishtest/vtc_client.c index 74bc8e7..67c86d1 100644 --- a/bin/varnishtest/vtc_client.c +++ b/bin/varnishtest/vtc_client.c @@ -31,12 +31,14 @@ #include #include +#include #include #include #include #include "vtc.h" +#include "vsa.h" #include "vss.h" #include "vtcp.h" @@ -51,6 +53,9 @@ struct client { char connect[256]; + char *proxy_spec; + int proxy_version; + unsigned repeat; unsigned running; @@ -61,6 +66,47 @@ static VTAILQ_HEAD(, client) clients = VTAILQ_HEAD_INITIALIZER(clients); /********************************************************************** + * Send the proxy header + */ + +static int __match_proto__(vss_resolved_f) +proxy_cb(void *priv, const struct suckaddr *sa) +{ + struct suckaddr **addr = priv; + *addr = VSA_Clone(sa); + return (1); +} + +static void +client_proxy(struct vtclog *vl, int fd, int version, char *spec) +{ + struct suckaddr *sac, *sas; + const char *err; + char *p, *p2; + int error; + + p = strdup(spec); + AN(p); + p2 = strchr(p, ' '); + AN(p2); + *p2++ = '\0'; + + error = VSS_resolver(p, NULL, proxy_cb, &sac, &err); + if (err != NULL) + vtc_log(vl, 0, "Could not resolve client address: %s", err); + assert(error == 1); + error = VSS_resolver(p2, NULL, proxy_cb, &sas, &err); + if (err != NULL) + vtc_log(vl, 0, "Could not resolve server address: %s", err); + assert(error == 1); + if (vtc_send_proxy(fd, version, sac, sas)) + vtc_log(vl, 0, "Write failed: %s", strerror(errno)); + free(p); + free(sac); + free(sas); +} + +/********************************************************************** * Client thread */ @@ -101,6 +147,8 @@ client_thread(void *priv) VTCP_myname(fd, mabuf, sizeof mabuf, mpbuf, sizeof mpbuf); vtc_log(vl, 3, "connected fd %d from %s %s to %s", fd, mabuf, mpbuf, VSB_data(vsb)); + if (c->proxy_spec != NULL) + client_proxy(vl, fd, c->proxy_version, c->proxy_spec); fd = http_process(vl, c->spec, fd, NULL); vtc_log(vl, 3, "closing fd %d", fd); VTCP_close(&fd); @@ -146,6 +194,7 @@ client_delete(struct client *c) vtc_logclose(c->vl); free(c->spec); free(c->name); + free(c->proxy_spec); /* XXX: MEMLEAK (?)*/ FREE_OBJ(c); } @@ -247,6 +296,18 @@ cmd_client(CMD_ARGS) av++; continue; } + if (!strcmp(*av, "-proxy1")) { + REPLACE(c->proxy_spec, av[1]); + c->proxy_version = 1; + av++; + continue; + } + if (!strcmp(*av, "-proxy2")) { + REPLACE(c->proxy_spec, av[1]); + c->proxy_version = 2; + av++; + continue; + } if (!strcmp(*av, "-repeat")) { c->repeat = atoi(av[1]); av++; diff --git a/bin/varnishtest/vtc_http.c b/bin/varnishtest/vtc_http.c index 184384c..753ac8a 100644 --- a/bin/varnishtest/vtc_http.c +++ b/bin/varnishtest/vtc_http.c @@ -101,15 +101,23 @@ * "IP PORT". * * \-connect STRING (client only) - * Indicate the server to connect to. STRING is also of the form - * "IP PORT". + * Indicate the server to connect to. STRING is also of the form + * "IP PORT". * * \-dispatch (server only, s0 only) - * Normally, to keep things simple, server threads only handle one - * connection at a time, but the -dispatch switch allows to accept - * any number of connection and handle them following the given spec. + * Normally, to keep things simple, server threads only handle one + * connection at a time, but the -dispatch switch allows to accept + * any number of connection and handle them following the given spec. + * + * However, -dispatch is only allowed for the server name "s0". + * + * \-proxy1 STRING (client only) + * Use the PROXY protocol version 1 for this connection. STRING + * is of the form "CLIENT:PORT SERVER:PORT". * - * However, -dispatch is only allowed for the server name "s0". + * \-proxy2 STRING (client only) + * Use the PROXY protocol version 2 for this connection. STRING + * is of the form "CLIENT:PORT SERVER:PORT". * * SECTION: client-server.spec Specification */ diff --git a/bin/varnishtest/vtc_proxy.c b/bin/varnishtest/vtc_proxy.c new file mode 100644 index 0000000..e61658c --- /dev/null +++ b/bin/varnishtest/vtc_proxy.c @@ -0,0 +1,132 @@ +/*- + * Copyright (c) 2015 Varnish Software AS + * All rights reserved. + * + * Author: Poul-Henning Kamp + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include +#include + +#include + +#include + +#include "vtc.h" + +#include "vend.h" +#include "vsa.h" +#include "vtcp.h" + +static const char vpx1_sig[] = {'P', 'R', 'O', 'X', 'Y'}; +static const char vpx2_sig[] = { + '\r', '\n', '\r', '\n', '\0', '\r', '\n', + 'Q', 'U', 'I', 'T', '\n', +}; + +static void +vpx_enc_addr(struct vsb *vsb, int proto, const struct suckaddr *s) +{ + const struct sockaddr_in *sin4; + const struct sockaddr_in6 *sin6; + socklen_t sl; + + if (proto == PF_INET6) { + sin6 = VSA_Get_Sockaddr(s, &sl); //lint !e826 + AN(sin6); + assert(sl >= sizeof(*sin6)); + VSB_bcat(vsb, &sin6->sin6_addr, sizeof(sin6->sin6_addr)); + } else { + sin4 = VSA_Get_Sockaddr(s, &sl); //lint !e826 + AN(sin4); + assert(sl >= sizeof(*sin4)); + VSB_bcat(vsb, &sin4->sin_addr, sizeof(sin4->sin_addr)); + } +} + +static void +vpx_enc_port(struct vsb *vsb, const struct suckaddr *s) +{ + uint8_t b[2]; + + vbe16enc(b, (uint16_t)VSA_Port(s)); + VSB_bcat(vsb, b, sizeof(b)); +} + +int +vtc_send_proxy(int fd, int version, const struct suckaddr *sac, + const struct suckaddr *sas) +{ + struct vsb *vsb; + char hc[VTCP_ADDRBUFSIZE]; + char pc[VTCP_PORTBUFSIZE]; + char hs[VTCP_ADDRBUFSIZE]; + char ps[VTCP_PORTBUFSIZE]; + int i, len; + int proto; + + AN(sac); + AN(sas); + + assert(version == 1 || version == 2); + vsb = VSB_new_auto(); + AN(vsb); + + proto = VSA_Get_Proto(sas); + assert(proto == PF_INET6 || proto == PF_INET); + + if (version == 1) { + VSB_bcat(vsb, vpx1_sig, sizeof(vpx1_sig)); + if (proto == PF_INET6) + VSB_printf(vsb, " TCP6 "); + else if (proto == PF_INET) + VSB_printf(vsb, " TCP4 "); + VTCP_name(sac, hc, sizeof(hc), pc, sizeof(pc)); + VTCP_name(sas, hs, sizeof(hs), ps, sizeof(ps)); + VSB_printf(vsb, "%s %s %s %s\r\n", hc, hs, pc, ps); + } else if (version == 2) { + VSB_bcat(vsb, vpx2_sig, sizeof(vpx2_sig)); + VSB_putc(vsb, 0x21); + if (proto == PF_INET6) { + VSB_putc(vsb, 0x21); + VSB_putc(vsb, 0x00); + VSB_putc(vsb, 0x24); + } else if (proto == PF_INET) { + VSB_putc(vsb, 0x11); + VSB_putc(vsb, 0x00); + VSB_putc(vsb, 0x0c); + } + vpx_enc_addr(vsb, proto, sac); + vpx_enc_addr(vsb, proto, sas); + vpx_enc_port(vsb, sac); + vpx_enc_port(vsb, sas); + } else + WRONG("Wrong proxy version"); + + AZ(VSB_finish(vsb)); + len = VSB_len(vsb); + i = write(fd, VSB_data(vsb), len); + VSB_delete(vsb); + return (i != len); +} From fgsch at lodoss.net Sun Jun 19 13:29:05 2016 From: fgsch at lodoss.net (Federico G. Schwindt) Date: Sun, 19 Jun 2016 15:29:05 +0200 Subject: [master] 62f78f1 Correct error messages Message-ID: commit 62f78f1ea9512d9e3a0409ad134cba1dbf3a58f0 Author: Federico G. Schwindt Date: Sun Jun 19 14:20:34 2016 +0100 Correct error messages diff --git a/bin/varnishd/mgt/mgt_cli.c b/bin/varnishd/mgt/mgt_cli.c index 2fbcdb3..d7b8cb0 100644 --- a/bin/varnishd/mgt/mgt_cli.c +++ b/bin/varnishd/mgt/mgt_cli.c @@ -562,7 +562,7 @@ mgt_cli_telnet(const char *T_arg) AN(vsb); error = VSS_resolver(T_arg, NULL, mct_callback, vsb, &err); if (err != NULL) - ARGV_ERR("Could resolve -T argument to address\n\t%s\n", err); + ARGV_ERR("Could not resolve -T argument to address\n\t%s\n", err); AZ(error); AZ(VSB_finish(vsb)); if (VSB_len(vsb) == 0) @@ -681,7 +681,7 @@ mgt_cli_master(const char *M_arg) error = VSS_resolver(M_arg, NULL, marg_cb, NULL, &err); if (err != NULL) - ARGV_ERR("Could resolve -M argument to address\n\t%s\n", err); + ARGV_ERR("Could not resolve -M argument to address\n\t%s\n", err); AZ(error); if (VTAILQ_EMPTY(&m_addr_list)) ARGV_ERR("Could not resolve -M argument to address\n"); From varnish-commit at varnish-cache.org Sun Jun 19 20:42:38 2016 From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org) Date: 19 Jun 2016 23:42:38 +0300 Subject: Get Back On Track With Our Breakthrough Solution Message-ID: <004b01d1ca6d$07ab740f$e0566ebd@rdyxvit> Have you been dreaming of a beautiful body, but had no results with diets, workouts and supplements? Our incredible solution with a breakthrough formula will deliver you the results you've been so eager to achive. Click Here! Powerful weight loss product that amplifies methabolism, stimulates positive mood and formulated for both men and women. -------------- next part -------------- An HTML attachment was scrubbed... URL: From varnish-commit at varnish-cache.org Sun Jun 19 23:09:06 2016 From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org) Date: 19 Jun 2016 17:09:06 -0600 Subject: Hello! Message-ID: Hello! We are looking for employees working remotely. My name is Troy, am the personnel manager of a large International company. Most of the work you can do from home, that is, at a distance. Salary is $2500-$5000. If you are interested in this offer, please visit Our Site Best regards! -------------- next part -------------- An HTML attachment was scrubbed... URL: From phk at FreeBSD.org Mon Jun 20 07:02:07 2016 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Mon, 20 Jun 2016 09:02:07 +0200 Subject: [master] e240bd4 Flinting Message-ID: commit e240bd41ad376b664b96ea17110b18bbd14caa57 Author: Poul-Henning Kamp Date: Mon Jun 20 06:40:56 2016 +0000 Flinting diff --git a/flint.lnt b/flint.lnt index 05567ce..5359c22 100644 --- a/flint.lnt +++ b/flint.lnt @@ -5,10 +5,12 @@ /////////////////////////////////////////////////////////////////////// // build/config related --efile(451, "tbl/*.h") // No include guard --efile(537, "tbl/*.h") // Repeated include +-efile(451, "tbl/*.h") // No include guard +-efile(537, "tbl/*.h") // Repeated include + +-efile(451, ../../config.h) // No include guard +-efile(766, ../../config.h) // Header file '___' not used in module '___' --efile(451, ../../config.h) // No include guard +libh(../../config.h) /////////////////////////////////////////////////////////////////////// From noreply at github.com Mon Jun 20 10:08:07 2016 From: noreply at github.com (GitHub) Date: Mon, 20 Jun 2016 12:08:07 +0200 Subject: [master] f9e1711 Typo Message-ID: commit f9e1711cc333fd839e3e2c752af041f37b1ff44c Author: Dridi Boukelmoune Date: Mon Jun 20 12:07:25 2016 +0200 Typo diff --git a/bin/varnishd/mgt/mgt_vcc.c b/bin/varnishd/mgt/mgt_vcc.c index ca8ae94..fdbaa00 100644 --- a/bin/varnishd/mgt/mgt_vcc.c +++ b/bin/varnishd/mgt/mgt_vcc.c @@ -303,7 +303,7 @@ mgt_VccCompile(struct cli *cli, const char *vclname, const char *vclsrc, * count, and when it finally runs, it trashes the second 'foo' because * dlopen(3) decided they were really the same thing. * - * The Best way to reproduce ths is to have regexps in the VCL. + * The Best way to reproduce this is to have regexps in the VCL. */ VSB_printf(sb, "vcl_%s.%.9f", vclname, VTIM_real()); AZ(VSB_finish(sb)); From phk at FreeBSD.org Mon Jun 20 10:31:07 2016 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Mon, 20 Jun 2016 12:31:07 +0200 Subject: [master] 746ae2c Fix description of -clierr Message-ID: commit 746ae2c544a62405fdc1726aa9a100426f213903 Author: Poul-Henning Kamp Date: Mon Jun 20 09:10:33 2016 +0000 Fix description of -clierr diff --git a/bin/varnishtest/vtc_varnish.c b/bin/varnishtest/vtc_varnish.c index 7a42324..a4c4c15 100644 --- a/bin/varnishtest/vtc_varnish.c +++ b/bin/varnishtest/vtc_varnish.c @@ -982,10 +982,10 @@ varnish_expect(const struct varnish *v, char * const *av) * varnish vNAME [-cli STRING] [-cliok STRING] [-clierr STRING] * [-expect STRING OP NUMBER] * - * \-cli|-cliok|-clierr STRING + * \-cli STRING|-cliok STRING|-clierr STATUS STRING * All three of these will send STRING to the CLI, the only difference * is what they expect the return code to be. -cli doesn't expect - * anything, -cliok expects 200 and -clierr expects not 200. + * anything, -cliok expects 200 and -clierr expects STATUS * * \-expect STRING OP NUMBER * Look into the VSM and make sure the counter identified by STRING has From phk at FreeBSD.org Mon Jun 20 10:31:07 2016 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Mon, 20 Jun 2016 12:31:07 +0200 Subject: [master] c11240d Add a '-syntax' to set the desired VCL syntax version Message-ID: commit c11240df6a81ace06e89dc64b34f14508de1c440 Author: Poul-Henning Kamp Date: Mon Jun 20 09:28:56 2016 +0000 Add a '-syntax' to set the desired VCL syntax version diff --git a/bin/varnishtest/vtc_varnish.c b/bin/varnishtest/vtc_varnish.c index a4c4c15..8dfc938 100644 --- a/bin/varnishtest/vtc_varnish.c +++ b/bin/varnishtest/vtc_varnish.c @@ -68,6 +68,8 @@ struct varnish { int fds[4]; pid_t pid; + float syntax; + pthread_t tp; pthread_t tp_vsl; @@ -288,6 +290,8 @@ varnish_new(const char *name) REPLACE(v->jail, ""); + v->syntax = 4.0; + v->vl = vtc_logopen(name); AN(v->vl); @@ -714,8 +718,8 @@ varnish_vcl(struct varnish *v, const char *vcl, enum VCLI_status_e expect, vsb = VSB_new_auto(); AN(vsb); - VSB_printf(vsb, "vcl.inline vcl%d << %s\nvcl 4.0;\n%s\n%s\n", - ++v->vcl_nbr, NONSENSE, vcl, NONSENSE); + VSB_printf(vsb, "vcl.inline vcl%d << %s\nvcl %.1f;\n%s\n%s\n", + ++v->vcl_nbr, NONSENSE, v->syntax, vcl, NONSENSE); AZ(VSB_finish(vsb)); u = varnish_ask_cli(v, VSB_data(vsb), resp); @@ -758,7 +762,7 @@ varnish_vclbackend(struct varnish *v, const char *vcl) vsb2 = VSB_new_auto(); AN(vsb2); - VSB_printf(vsb2, "vcl 4.0;\n"); + VSB_printf(vsb2, "vcl %.1f;\n", v->syntax); cmd_server_genvcl(vsb2); @@ -963,6 +967,9 @@ varnish_expect(const struct varnish *v, char * const *av) * \-stop * Stop the child process. * + * \-syntax + * Set the VCL syntax level (default: 4.0) + * * \-wait * Wait for that instance to terminate. * @@ -1104,6 +1111,12 @@ cmd_varnish(CMD_ARGS) varnish_stop(v); continue; } + if (!strcmp(*av, "-syntax")) { + AN(av[1]); + v->syntax = strtod(av[1], NULL); + av++; + continue; + } if (!strcmp(*av, "-vcl")) { AN(av[1]); varnish_vcl(v, av[1], CLIS_OK, NULL); From phk at FreeBSD.org Mon Jun 20 10:31:07 2016 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Mon, 20 Jun 2016 12:31:07 +0200 Subject: [master] b2eaca0 White-space OCD Message-ID: commit b2eaca02639aa572cb07cfb4e15d61edb5d67ca5 Author: Poul-Henning Kamp Date: Mon Jun 20 09:53:26 2016 +0000 White-space OCD diff --git a/bin/varnishtest/vtc_varnish.c b/bin/varnishtest/vtc_varnish.c index 8dfc938..6f58bf1 100644 --- a/bin/varnishtest/vtc_varnish.c +++ b/bin/varnishtest/vtc_varnish.c @@ -923,8 +923,8 @@ varnish_expect(const struct varnish *v, char * const *av) * * To define a Varnish server, you'll use this syntax:: * - * varnish vNAME [-arg STRING] [-vcl STRING] [-vcl+backend STRING] - * [-errvcl STRING STRING] [-jail STRING] [-proto PROXY] + * varnish vNAME [-arg STRING] [-vcl STRING] [-vcl+backend STRING] + * [-errvcl STRING STRING] [-jail STRING] [-proto PROXY] * * The first ``varnish vNAME`` invocation will start the varnishd master * process in the background, waiting for the ``-start`` switch to actually From phk at FreeBSD.org Mon Jun 20 10:31:07 2016 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Mon, 20 Jun 2016 12:31:07 +0200 Subject: [master] 9a84137 Add code to cater for other vcl syntax numbers than 4.0, we will soon need them. Message-ID: commit 9a84137847228d4feebbc19d7dfc247f5074d84e Author: Poul-Henning Kamp Date: Mon Jun 20 10:00:27 2016 +0000 Add code to cater for other vcl syntax numbers than 4.0, we will soon need them. diff --git a/bin/varnishtest/tests/v00049.vtc b/bin/varnishtest/tests/v00049.vtc new file mode 100644 index 0000000..35a3efc --- /dev/null +++ b/bin/varnishtest/tests/v00049.vtc @@ -0,0 +1,17 @@ +varnishtest "VCL syntax numbers" + +varnish v1 -vcl {backend b1 { .host = "127.0.0.1:8080"; }} -start + +varnish v1 -syntax 3.9 -errvcl "VCL version 3.9 not supported." { + backend b1 { .host = "127.0.0.1:8080"; } +} + +varnish v1 -syntax 4.0 -errvcl "silly buggers" { + vcl 4.01; + backend b1 { .host = "127.0.0.1:8080"; } +} + +varnish v1 -syntax 4.0 -errvcl "9.9 higher than the top level version" { + vcl 9.9; + backend b1 { .host = "127.0.0.1:8080"; } +} diff --git a/lib/libvcc/vcc_compile.c b/lib/libvcc/vcc_compile.c index a687cdc..9a1a72c 100644 --- a/lib/libvcc/vcc_compile.c +++ b/lib/libvcc/vcc_compile.c @@ -481,7 +481,7 @@ vcc_resolve_includes(struct vcc *tl) continue; t1 = VTAILQ_NEXT(t, list); - assert(t1 != NULL); /* There's always an EOI */ + AN(t1); /* There's always an EOI */ if (t1->tok != CSTR) { VSB_printf(tl->sb, "include not followed by string constant.\n"); @@ -489,7 +489,7 @@ vcc_resolve_includes(struct vcc *tl) return; } t2 = VTAILQ_NEXT(t1, list); - assert(t2 != NULL); /* There's always an EOI */ + AN(t2); /* There's always an EOI */ if (t2->tok != ';') { VSB_printf(tl->sb, diff --git a/lib/libvcc/vcc_compile.h b/lib/libvcc/vcc_compile.h index 22b30f2..fc263fd 100644 --- a/lib/libvcc/vcc_compile.h +++ b/lib/libvcc/vcc_compile.h @@ -70,6 +70,7 @@ struct symbol; struct source { VTAILQ_ENTRY(source) list; + float syntax; char *name; const char *b; const char *e; diff --git a/lib/libvcc/vcc_parse.c b/lib/libvcc/vcc_parse.c index 869979e..c5caa62 100644 --- a/lib/libvcc/vcc_parse.c +++ b/lib/libvcc/vcc_parse.c @@ -279,13 +279,20 @@ vcc_ParseVcl(struct vcc *tl) assert(vcc_IdIs(tl->t, "vcl")); vcc_NextToken(tl); tok = tl->t; - tl->syntax = vcc_DoubleVal(tl); + tok->src->syntax = vcc_DoubleVal(tl); ERRCHK(tl); - if (tl->syntax != 4.0) { - // see TODO above - VSB_printf(tl->sb, "VCL version %.1f not supported.\n", - tl->syntax); - vcc_ErrWhere(tl, tok); + if (tl->t->e - tok->b > 4) { + VSB_printf(tl->sb, + "Don't play silly buggers with VCL version numbers\n"); + vcc_ErrWhere2(tl, tok, tl->t); + ERRCHK(tl); + } + if (tl->syntax != 0.0 && tok->src->syntax > tl->syntax) { + VSB_printf(tl->sb, + "VCL version %.1f higher than" + " the top level version %.1f\n", + tok->src->syntax, tl->syntax); + vcc_ErrWhere2(tl, tok, tl->t); ERRCHK(tl); } ExpectErr(tl, ';'); @@ -322,18 +329,27 @@ void vcc_Parse(struct vcc *tl) { struct toplev *tp; + struct token *tok; if (tl->t->tok != ID || !vcc_IdIs(tl->t, "vcl")) { VSB_printf(tl->sb, "VCL version declaration missing\n" "Update your VCL to Version 4 syntax, and add\n" "\tvcl 4.0;\n" - "on the first line the VCL files.\n" + "on the first line of the VCL files.\n" ); vcc_ErrWhere(tl, tl->t); ERRCHK(tl); } + tok = tl->t; vcc_ParseVcl(tl); + if (tok->src->syntax != 4.0) { + VSB_printf(tl->sb, "VCL version %.1f not supported.\n", + tok->src->syntax); + vcc_ErrWhere2(tl, tok, tl->t); + ERRCHK(tl); + } + tl->syntax = tl->t->src->syntax; ERRCHK(tl); while (tl->t->tok != EOI) { ERRCHK(tl); From phk at FreeBSD.org Mon Jun 20 10:31:07 2016 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Mon, 20 Jun 2016 12:31:07 +0200 Subject: [master] c0c1e05 Document how VCL syntax versioning works. Message-ID: commit c0c1e051151875b0914f479a5ebccdaff4359861 Author: Poul-Henning Kamp Date: Mon Jun 20 10:12:35 2016 +0000 Document how VCL syntax versioning works. diff --git a/doc/sphinx/reference/vcl.rst b/doc/sphinx/reference/vcl.rst index 7bd6c91..e47336c 100644 --- a/doc/sphinx/reference/vcl.rst +++ b/doc/sphinx/reference/vcl.rst @@ -28,7 +28,8 @@ description of syntax and semantics, with ample examples, please see the online documentation at https://www.varnish-cache.org/docs/ . Starting with Varnish 4.0, each VCL file must start by declaring its version -with a special "vcl 4.0;" marker at the top of the file. +with "vcl X.Y;" marker at the top of the file. See more about this +under Versioning below. Operators @@ -406,6 +407,26 @@ regsuball(str, regex, sub) For converting or casting VCL values between data types use the functions available in the std VMOD. +Versioning +========== + +Multiple versions of the VCL syntax can coexist within certain +constraints. + +The VCL syntax version at the start of VCL file specified with ''-f'' +sets the hard limit that cannot be exceed anywhere, and it selects +the appropriate version of the builtin VCL. + +That means that you can never include "vcl 9.1;" from "vcl 8.7;", but +the opposite *may* be possible, to the extent the compiler supports it. + +Files pulled in via ``include`` do not need to have a "vcl X.Y;" but +it may be a good idea to do it anyway, to not have surprises in the +future. The syntax version set in an included file only applies to +that file and any files it includes - unless these set their own VCL +syntax version. + +The version of Varnish this file belongs to supports syntax 4.0 only. EXAMPLES From phk at FreeBSD.org Mon Jun 20 10:31:07 2016 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Mon, 20 Jun 2016 12:31:07 +0200 Subject: [master] ff29e5d White space OCD Message-ID: commit ff29e5d4f1d4a0d066bf98c826953f359d06d92e Author: Poul-Henning Kamp Date: Mon Jun 20 10:27:36 2016 +0000 White space OCD diff --git a/doc/sphinx/vtc-syntax.py b/doc/sphinx/vtc-syntax.py index f0601b3..3a14f0d 100644 --- a/doc/sphinx/vtc-syntax.py +++ b/doc/sphinx/vtc-syntax.py @@ -68,7 +68,7 @@ def parse_file(fn): c = section.count('.') if c == 0: r = "-" - elif c ==1: + elif c ==1: r = "~" else: r = "." From phk at FreeBSD.org Mon Jun 20 10:31:07 2016 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Mon, 20 Jun 2016 12:31:07 +0200 Subject: [master] 42a876a White space OCD Message-ID: commit 42a876adbb949d4b5e479ad7b4ace32d9c8559cd Author: Poul-Henning Kamp Date: Mon Jun 20 10:29:02 2016 +0000 White space OCD diff --git a/bin/varnishtest/vtc_http.c b/bin/varnishtest/vtc_http.c index 753ac8a..670aa50 100644 --- a/bin/varnishtest/vtc_http.c +++ b/bin/varnishtest/vtc_http.c @@ -676,7 +676,7 @@ http_rxhdr(struct http *hp) * * rxresp [-no_obj] (client only) * Receive and parse a response's headers and body. If -no_obj is present, only get - * the headers. + * the headers. */ static void cmd_http_rxresp(CMD_ARGS) @@ -972,8 +972,8 @@ http_tx_parse_args(char * const *av, struct vtclog *vl, struct http *hp, * Generate and input a body that is NUMBER bytes-long. * * \-gziplevel NUMBER - * Set the gzip level (call it before any of the other gzip - * switches). + * Set the gzip level (call it before any of the other gzip + * switches). * * \gzipresidual NUMBER * Add extra gzip bits. You should never need it. @@ -1427,7 +1427,7 @@ cmd_http_chunkedlen(CMD_ARGS) /* SECTION: client-server.spec.timeout * * timeout NUMBER - * Set the TCP timeout for this entity. + * Set the TCP timeout for this entity. */ static void @@ -1584,14 +1584,14 @@ cmd_http_fatal(CMD_ARGS) } /* SECTION: client-server.spec.delay - * + * * delay - * Same as for the top-level delay. + * Same as for the top-level delay. * * SECTION: client-server.spec.barrier * * barrier - * Same as for the top-level barrier + * Same as for the top-level barrier */ /********************************************************************** diff --git a/bin/varnishtest/vtc_logexp.c b/bin/varnishtest/vtc_logexp.c index 3b14d81..338c117 100644 --- a/bin/varnishtest/vtc_logexp.c +++ b/bin/varnishtest/vtc_logexp.c @@ -32,7 +32,7 @@ * process records trying to match the first pattern, and when done, will * continue processing, trying to match the following pattern. If a pattern * isn't matched, the test will fail. - * + * * logexpect threads are declared this way:: * * logexpect lNAME -v [-g ] [-d 0|1] [-q query] \ @@ -68,7 +68,7 @@ * * \-start * Start the logexpect thread in the background. - * + * * \-wait * Wait for the logexpect thread to finish * @@ -76,7 +76,7 @@ * * \-b|-c * Process only backend/client records. - * + * * \-C * Use caseless regex * diff --git a/include/tbl/boc_state.h b/include/tbl/boc_state.h index c3a2586..5f71c7a 100644 --- a/include/tbl/boc_state.h +++ b/include/tbl/boc_state.h @@ -27,7 +27,7 @@ */ /*lint -save -e525 -e539 */ -BOC_STATE(INVALID, invalid) /* don't touch (yet) */ +BOC_STATE(INVALID, invalid) /* don't touch (yet) */ BOC_STATE(REQ_DONE, req_done) /* beresp.* can be examined */ BOC_STATE(PREP_STREAM, prep_stream) /* Prepare for streaming */ BOC_STATE(STREAM, stream) /* beresp.* can be examined */ From noreply at github.com Mon Jun 20 10:40:10 2016 From: noreply at github.com (GitHub) Date: Mon, 20 Jun 2016 12:40:10 +0200 Subject: [master] 6614d60 Typo Message-ID: commit 6614d608d8e233c9cff525d1af57cfede7dadfaf Author: Dridi Boukelmoune Date: Mon Jun 20 12:39:36 2016 +0200 Typo diff --git a/doc/sphinx/reference/vcl.rst b/doc/sphinx/reference/vcl.rst index e47336c..b68fdc6 100644 --- a/doc/sphinx/reference/vcl.rst +++ b/doc/sphinx/reference/vcl.rst @@ -414,7 +414,7 @@ Multiple versions of the VCL syntax can coexist within certain constraints. The VCL syntax version at the start of VCL file specified with ''-f'' -sets the hard limit that cannot be exceed anywhere, and it selects +sets the hard limit that cannot be exceeded anywhere, and it selects the appropriate version of the builtin VCL. That means that you can never include "vcl 9.1;" from "vcl 8.7;", but From varnish-commit at varnish-cache.org Mon Jun 20 15:25:47 2016 From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org) Date: 20 Jun 2016 17:25:47 +0200 Subject: hi! Message-ID: <001701d1cb0a$023cfd96$113c54b3$@varnish-cache.org> Hello! We are looking for employees working remotely. My name is Monique, am the personnel manager of a large International company. Most of the work you can do from home, that is, at a distance. Salary is $2500-$5000. If you are interested in this offer, please visit Our Site Best regards! -------------- next part -------------- An HTML attachment was scrubbed... URL: From fgsch at lodoss.net Mon Jun 20 16:03:07 2016 From: fgsch at lodoss.net (Federico G. Schwindt) Date: Mon, 20 Jun 2016 18:03:07 +0200 Subject: [master] bf79458 Some pep8 love Message-ID: commit bf79458df6117d9ef85e0dc53aea740dc39e79ff Author: Federico G. Schwindt Date: Mon Jun 20 16:08:43 2016 +0100 Some pep8 love diff --git a/doc/sphinx/vtc-syntax.py b/doc/sphinx/vtc-syntax.py index 3a14f0d..45a5102 100644 --- a/doc/sphinx/vtc-syntax.py +++ b/doc/sphinx/vtc-syntax.py @@ -1,5 +1,5 @@ #!/usr/bin/env python -#- +# # Copyright (c) 2006-2016 Varnish Software AS # All rights reserved. # @@ -26,12 +26,13 @@ # OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF # SUCH DAMAGE. # -# Generate various .c and .h files for the VSL query expression parser -# and the interfaces for it. +# Process various varnishtest C files and output reStructuredText to be +# included in vtc(7). from __future__ import print_function -import sys; -import re; +import sys +import re + def parse_file(fn): p = False @@ -41,7 +42,7 @@ def parse_file(fn): tl = {} sl = [] - f = open(fn, 'r') + f = open(fn, "r") for l in f: if "*/" in l: @@ -52,27 +53,28 @@ def parse_file(fn): sl.append(section) cl[section] = [] if len(a) > 3: - tl[section] = re.sub(r'^[\t ]*\/?\* SECTION: [^ ]+ +', - "", l) + tl[section] = re.sub( + r"^[\t ]*\/?\* SECTION: [^ ]+ +", + "", l) else: tl[section] = "" p = 1 elif p: - cl[section].append(re.sub(r'^ \* ?',"", l)) + cl[section].append(re.sub(r"^ \* ?", "", l)) f.close() sl.sort() for section in sl: - print(tl[section], end=''); + print(tl[section], end="") a = section - c = section.count('.') + c = section.count(".") if c == 0: r = "-" - elif c ==1: + elif c == 1: r = "~" else: r = "." - print(re.sub(r'.', r, tl[section]), end='') + print(re.sub(r".", r, tl[section]), end="") print("".join(cl[section])) if __name__ == "__main__": From fgsch at lodoss.net Mon Jun 20 16:03:07 2016 From: fgsch at lodoss.net (Federico G. Schwindt) Date: Mon, 20 Jun 2016 18:03:07 +0200 Subject: [master] 44169e7 Polish Message-ID: commit 44169e78e69d4b486797aaee175dbf2497eb97f0 Author: Federico G. Schwindt Date: Mon Jun 20 16:50:09 2016 +0100 Polish diff --git a/bin/varnishtest/vtc_http.c b/bin/varnishtest/vtc_http.c index 670aa50..f7e993f 100644 --- a/bin/varnishtest/vtc_http.c +++ b/bin/varnishtest/vtc_http.c @@ -56,8 +56,8 @@ * recognized, assuming they don't start with '-', are treated as * specifications, laying out the actions to undertake:: * - * client cNAME [ARGS*] - * server sNAME [ARGS*] + * client cNAME [...] + * server sNAME [...] * * Clients and server are identified by a string that's the first argument, * clients' names start with 'c' and servers' names start with 's'. @@ -113,11 +113,11 @@ * * \-proxy1 STRING (client only) * Use the PROXY protocol version 1 for this connection. STRING - * is of the form "CLIENT:PORT SERVER:PORT". + * is of the form "CLIENTIP:PORT SERVERIP:PORT". * * \-proxy2 STRING (client only) * Use the PROXY protocol version 2 for this connection. STRING - * is of the form "CLIENT:PORT SERVER:PORT". + * is of the form "CLIENTIP:PORT SERVERIP:PORT". * * SECTION: client-server.spec Specification */ @@ -928,7 +928,7 @@ http_tx_parse_args(char * const *av, struct vtclog *vl, struct http *hp, /* SECTION: client-server.spec.txre * - * txreq|txresp [ARG*] + * txreq|txresp [...] * Send a minimal request or response, but overload it if necessary. * * txreq is client-specific and txresp is server-specific. @@ -938,7 +938,7 @@ http_tx_parse_args(char * const *av, struct vtclog *vl, struct http *hp, * status line), so all the options are prety much the same. * * \-req STRING (txreq only) - * What method to use (default: "GET". + * What method to use (default: "GET"). * * \-url STRING (txreq only) * What location to use (default "/"). @@ -975,7 +975,7 @@ http_tx_parse_args(char * const *av, struct vtclog *vl, struct http *hp, * Set the gzip level (call it before any of the other gzip * switches). * - * \gzipresidual NUMBER + * \-gzipresidual NUMBER * Add extra gzip bits. You should never need it. * * \-gzipbody STRING From varnish-commit at varnish-cache.org Tue Jun 21 01:06:42 2016 From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org) Date: 21 Jun 2016 04:06:42 +0300 Subject: hi! Message-ID: <001a01d1cb5b$06e41726$a9a96f9b$@varnish-cache.org> Hello! We are looking for employees working remotely. My name is Hans, am the personnel manager of a large International company. Most of the work you can do from home, that is, at a distance. Salary is $2500-$5000. If you are interested in this offer, please visit Our Site Best regards! -------------- next part -------------- An HTML attachment was scrubbed... URL: From varnish-commit at varnish-cache.org Tue Jun 21 07:40:38 2016 From: varnish-commit at varnish-cache.org (znsikhqwg) Date: Tue, 21 Jun 2016 15:40:38 +0800 Subject: =?utf-8?B?dmFybmlzaC1jb21taXQ65aaC5L2V5bu656uL6LW3?= =?utf-8?B?57O757uf55qE44CB5pyJ5pWI55qERUhT566h55CG5L2T57O7?= Message-ID: <20160621154047181618@elzcuyc.org> ????????EHS???????? ?????? 2016?06?29-30????07?20-21???? ?????? ??EHS??2???????????????????? ?????? ???? + ???? + ???? +???? + ???? ?????? 4800?/1????????????????? ?????? 021-31006787 18917870808 ??? ????QQ? 320588808 ????? ???????EHS??????????????????????????EHS??????????????????????????????EHS????????????????????????????????????????????EHS??????????????EHS?????????????????? ????????????EHS?????????????????????????? ????EHS??????????? ????????EHS?????? ???????????EHS??????????????????? ???????????????????EHS????? ???EHS????????????????????????????????????????????????????????????????? ????????EHS???????????????????????????????????????EHS?????????????????????????????????????????????????????EHS??????????????????????????????????? ???? ??????? ?????????EHS?????????????????????????????????????????????????????EHS?????????EHS???????????????????????? ?????? ????????????????????EHS??????????????????EHS????????????????????????????????????????????????EHS???????????EHS??? EHS?????? ??EMS?OHSAS????????????????????????????????????????EHS??????????????????????????????????? ??EHS?????? ??EHS?????????????????????????????????EHS??????????????????EHS??????????????????????????????????????? ???? ?????? ?????????????????????????????????????????????????? EHS???? ??EHS???????EHS?????EHS????????????????????EHS????????????????? ???? ??EHS??????EHS??????????????EHS??????? ?????? ??EHS??????????????????????????????????EHS???? ???? ??????EHS???? a)????????????????? b)??????????????????? c)?????????? d)???EHS???????? e)????EHS??????? f)EHS?????????? g)??????EHS???? ????EHS???????? a)EHS?????????????????? b)??????EHS?? c)EHS???????? d)???????? e)?????????????????? f)??EHS????? g)EHS?????????? h)EHS?????????? ???? EHS?????? a)EHS?????? EHS?????? EHS??????? EHS????????? EHS????????????? ?????????????? ???????????????? b)EHS?????? ?????EHS???????? ?????????? ???????????????? c)EHS???????????? ?????????????EHS???????? EHS KPI?????????????? EHS???????????????????? ???????????EHS??????? ???????????????????? EHS????????? d)EHS????? EHS????????? EHS??????? ?EHS?????????????? EHS???? ???????????????EHS?????? e)EHS?????? ???????????????????? ?????? ???? ???????????????? ?????? ?????? ?????????????? ???????? ????????????? ????????? f)????????? g)???????????? ??????EHS??????? ?????????? ?????????? ????????????? ?????????????????? ??????????? ??????????? ?????????????????? ?????????????? ??????? ?????????? ????????????? ?????????? ???????????????? a)EHS????????????????? ??????SOP???????? ??????EHS??????????? ?????????????????????? ?????? ?????????EHS????? ???EHS???????????? b)EHS????????? ?????EHS??????? ?????????EHS?? ??TWI???????EHS???? ??????????????BBS??EHS???????? ????????? ?????????????????? ????EHS???? a??????????? b???????????? c????????????? d???????????????? e???EHS???? ?????????EHS????? ?????EHS????????? ????????? ????? f?EHS?????????? EHS????? EHS????????? ??????? ????EHS??????? a)??????EHS????????? ??????????? ??????????????? EHS??????????????? ???EHS?????????????? ???????EHS?????????????? b) EHS??????????? ????????????EHS???? ?????EHS KPI??????? ?????EHS???????????? ????????????EHS?? ?????????? EHS??????????????-????? ?????????? ??????? ???????????????? ??????????????????????? ??????????????? ????? ?????????????APP?????????????????????????????????11??????????6???????????????6???????????????????????????????????????????????????????????????????????????????????????????AON???????????????????????????????????????????????????? ?????????????????????????????????????????????????????????6????????????????????????2000????????5???????????????????????????????ISO14001&OHSAS18001???????6S??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????? ????????????????????????????????????????EHS??????????????????????????????????6S???????????????? ?????? ??/????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????? ? ??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????SEW?????????????????????????????????????????????????????NGK??????????????????????????????????????????????????????? ? ?????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????? -------------- next part -------------- An HTML attachment was scrubbed... URL: From phk at FreeBSD.org Tue Jun 21 13:30:12 2016 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Tue, 21 Jun 2016 15:30:12 +0200 Subject: [master] 14ee00a Initialize fd to -1 to catch bugs Message-ID: commit 14ee00a1e1ce29143e4eaa6ac697e1b5981e27cc Author: Poul-Henning Kamp Date: Tue Jun 21 13:29:36 2016 +0000 Initialize fd to -1 to catch bugs diff --git a/bin/varnishd/cache/cache_req.c b/bin/varnishd/cache/cache_req.c index 72c3d95..c06c10d 100644 --- a/bin/varnishd/cache/cache_req.c +++ b/bin/varnishd/cache/cache_req.c @@ -68,6 +68,7 @@ Req_New(const struct worker *wrk, struct sess *sp) req->top = req; // esi overrides INIT_OBJ(req->htc, HTTP_CONN_MAGIC); + req->htc->fd = -1; e = (char*)req + sz; p = (char*)(req + 1); From varnish-commit at varnish-cache.org Thu Jun 23 04:26:55 2016 From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org) Date: 23 Jun 2016 06:26:55 +0200 Subject: Working with partial occupancy Message-ID: <001c01d1cd0c$0337ee27$982d828b$@varnish-cache.org> Hello! We are looking for employees working remotely. My name is Casey, am the personnel manager of a large International company. Most of the work you can do from home, that is, at a distance. Salary is $2500-$5000. If you are interested in this offer, please visit Our Site Best regards! -------------- next part -------------- An HTML attachment was scrubbed... URL: From varnish-commit at varnish-cache.org Thu Jun 23 20:39:44 2016 From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org) Date: 24 Jun 2016 02:39:44 +0600 Subject: Cooperarion with a large firm Message-ID: Hello! We are looking for employees working remotely. My name is Katheryn, am the personnel manager of a large International company. Most of the work you can do from home, that is, at a distance. Salary is $2500-$5000. If you are interested in this offer, please visit Our Site Best regards! -------------- next part -------------- An HTML attachment was scrubbed... URL: From fgsch at lodoss.net Thu Jun 23 22:10:09 2016 From: fgsch at lodoss.net (Federico G. Schwindt) Date: Fri, 24 Jun 2016 00:10:09 +0200 Subject: [master] 78467a6 Tidy up generated C output Message-ID: commit 78467a608823ed62b2b52563a6fa529596a311c0 Author: Federico G. Schwindt Date: Thu Jun 23 23:03:55 2016 +0100 Tidy up generated C output diff --git a/lib/libvcc/vcc_action.c b/lib/libvcc/vcc_action.c index 37482f0..a2562f5 100644 --- a/lib/libvcc/vcc_action.c +++ b/lib/libvcc/vcc_action.c @@ -265,7 +265,7 @@ parse_synthetic(struct vcc *tl) Fb(tl, 1, "VRT_synth_page(ctx, "); vcc_Expr(tl, STRING_LIST); ERRCHK(tl); - Fb(tl, 0, ");\n"); + Fb(tl, 1, ");\n"); ExpectErr(tl, ')'); vcc_NextToken(tl); diff --git a/lib/libvcc/vcc_compile.c b/lib/libvcc/vcc_compile.c index 9a1a72c..22d8c10 100644 --- a/lib/libvcc/vcc_compile.c +++ b/lib/libvcc/vcc_compile.c @@ -312,7 +312,7 @@ EmitInitFini(const struct vcc *tl) } Fc(tl, 0, "\t(void)VGC_function_vcl_init(ctx);\n"); - Fc(tl, 0, "\treturn(*ctx->handling == VCL_RET_OK ? 0: -1);\n"); + Fc(tl, 0, "\treturn (*ctx->handling == VCL_RET_OK ? 0: -1);\n"); Fc(tl, 0, "}\n"); /* @@ -332,7 +332,7 @@ EmitInitFini(const struct vcc *tl) VSB_destroy(&p->fin); } - Fc(tl, 0, "\treturn(0);\n"); + Fc(tl, 0, "\treturn (0);\n"); Fc(tl, 0, "}\n"); if (has_event) { @@ -384,15 +384,15 @@ EmitInitFini(const struct vcc *tl) Fc(tl, 0, "VGC_Event(VRT_CTX, enum vcl_event_e ev)\n"); Fc(tl, 0, "{\n"); Fc(tl, 0, "\tif (ev == VCL_EVENT_LOAD)\n"); - Fc(tl, 0, "\t\treturn(VGC_Load(ctx));\n"); + Fc(tl, 0, "\t\treturn (VGC_Load(ctx));\n"); if (has_event) { Fc(tl, 0, "\tif (ev == VCL_EVENT_WARM)\n"); - Fc(tl, 0, "\t\treturn(VGC_Warmup(ctx, ev));\n"); + Fc(tl, 0, "\t\treturn (VGC_Warmup(ctx, ev));\n"); Fc(tl, 0, "\tif (ev == VCL_EVENT_COLD)\n"); - Fc(tl, 0, "\t\treturn(VGC_Cooldown(ctx, ev));\n"); + Fc(tl, 0, "\t\treturn (VGC_Cooldown(ctx, ev));\n"); } Fc(tl, 0, "\tif (ev == VCL_EVENT_DISCARD)\n"); - Fc(tl, 0, "\t\treturn(VGC_Discard(ctx));\n"); + Fc(tl, 0, "\t\treturn (VGC_Discard(ctx));\n"); Fc(tl, 0, "\n"); if (!has_event) Fc(tl, 0, "\t(void)vgc_warmupstep;\n"); @@ -656,7 +656,7 @@ vcc_CompileSource(struct vcc *tl, struct source *sp) Fc(tl, 1, " VRT_handling(ctx, VCL_RET_OK);\n"); Fc(tl, 1, "%s", VSB_data(tl->fm[i])); if (method_tab[i].bitval == VCL_MET_INIT) - Fc(tl, 1, " return(1);\n"); + Fc(tl, 1, " return (1);\n"); Fc(tl, 1, "}\n"); } From varnish-commit at varnish-cache.org Fri Jun 24 09:42:43 2016 From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org) Date: Fri, 24 Jun 2016 13:42:43 +0400 Subject: beneficial offer Message-ID: <576D0093.9090302@varnish-cache.org> Hello! We are looking for employees working remotely. My name is Kaye, am the personnel manager of a large International company. Most of the work you can do from home, that is, at a distance. Salary is $2500-$5000. If you are interested in this offer, please visit Our Site Best regards! -------------- next part -------------- An HTML attachment was scrubbed... URL: From varnish-commit at varnish-cache.org Fri Jun 24 18:32:04 2016 From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org) Date: Fri, 24 Jun 2016 23:32:04 +0500 Subject: beneficial proposition Message-ID: <576D7CA4.4030908@varnish-cache.org> Hello! We are looking for employees working remotely. My name is Mathew, am the personnel manager of a large International company. Most of the work you can do from home, that is, at a distance. Salary is $2500-$5000. If you are interested in this offer, please visit Our Site Best regards! -------------- next part -------------- An HTML attachment was scrubbed... URL: From varnish-commit at varnish-cache.org Fri Jun 24 22:38:13 2016 From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org) Date: 25 Jun 2016 00:38:13 +0200 Subject: The beautiful work in crisis! Message-ID: Hello! We are looking for employees working remotely. My name is Anderson, am the personnel manager of a large International company. Most of the work you can do from home, that is, at a distance. Salary is $2500-$5000. If you are interested in this offer, please visit Our Site Best regards! -------------- next part -------------- An HTML attachment was scrubbed... URL: From varnish-commit at varnish-cache.org Sat Jun 25 09:11:39 2016 From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org) Date: 25 Jun 2016 11:11:39 +0200 Subject: Flexible schedule Message-ID: <2A04C7F194E932A28C4F791C61BA2A04@QG00IY87W> Hello! We are looking for employees working remotely. My name is Theron, am the personnel manager of a large International company. Most of the work you can do from home, that is, at a distance. Salary is $2500-$5000. If you are interested in this offer, please visit Our Site Best regards! -------------- next part -------------- An HTML attachment was scrubbed... URL: From varnish-commit at varnish-cache.org Sat Jun 25 13:16:36 2016 From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org) Date: 25 Jun 2016 16:16:36 +0300 Subject: hi! Message-ID: Hello! We are looking for employees working remotely. My name is Emilio, am the personnel manager of a large International company. Most of the work you can do from home, that is, at a distance. Salary is $2500-$5000. If you are interested in this offer, please visit Our Site Best regards! -------------- next part -------------- An HTML attachment was scrubbed... URL: From fgsch at lodoss.net Sat Jun 25 22:42:06 2016 From: fgsch at lodoss.net (Federico G. Schwindt) Date: Sun, 26 Jun 2016 00:42:06 +0200 Subject: [master] 5c8a6b2 Add -L limit support to varnishncsa Message-ID: commit 5c8a6b2bc64491c8c8a9301b2fb33ec5cd907bfc Author: Federico G. Schwindt Date: Sat Jun 25 23:40:56 2016 +0100 Add -L limit support to varnishncsa Fixes #1994. diff --git a/bin/varnishncsa/varnishncsa_options.h b/bin/varnishncsa/varnishncsa_options.h index 74b0f52..0f12e89 100644 --- a/bin/varnishncsa/varnishncsa_options.h +++ b/bin/varnishncsa/varnishncsa_options.h @@ -83,6 +83,7 @@ NCSA_OPT_F NCSA_OPT_f NCSA_OPT_g VUT_OPT_h +VSL_OPT_L VUT_OPT_n VUT_OPT_N VUT_OPT_P From varnish-commit at varnish-cache.org Sun Jun 26 10:06:38 2016 From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org) Date: 26 Jun 2016 13:06:38 +0300 Subject: The crisis has finished! Work with us! Message-ID: <6891DDC48E2488976E223B71DB776891@CJ6N1K5> Hello! We are looking for employees working remotely. My name is Marcella, am the personnel manager of a large International company. Most of the work you can do from home, that is, at a distance. Salary is $2500-$5000. If you are interested in this offer, please visit Our Site Best regards! -------------- next part -------------- An HTML attachment was scrubbed... URL: From varnish-commit at varnish-cache.org Sun Jun 26 20:34:46 2016 From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org) Date: 26 Jun 2016 14:34:46 -0600 Subject: Cooperation with the great company Message-ID: <6ACA0C2CF1ACEAD171B7974A17516ACA@8OG0HTK> Hello! We are looking for employees working remotely. My name is Dexter, am the personnel manager of a large International company. Most of the work you can do from home, that is, at a distance. Salary is $2500-$5000. If you are interested in this offer, please visit Our Site Best regards! -------------- next part -------------- An HTML attachment was scrubbed... URL: From lkarsten at varnish-software.com Mon Jun 27 12:19:07 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Mon, 27 Jun 2016 14:19:07 +0200 Subject: [4.1] b06007c Add changelog entries for changes since 4.1.3-beta1. Message-ID: commit b06007cb4f8927725053713bc5f597372ec7c447 Author: Lasse Karstensen Date: Mon Jun 27 14:18:35 2016 +0200 Add changelog entries for changes since 4.1.3-beta1. diff --git a/configure.ac b/configure.ac index 9d84d9c..736180f 100644 --- a/configure.ac +++ b/configure.ac @@ -2,7 +2,7 @@ AC_PREREQ(2.59) AC_COPYRIGHT([Copyright (c) 2006 Verdens Gang AS Copyright (c) 2006-2016 Varnish Software]) AC_REVISION([$Id$]) -AC_INIT([Varnish], [4.1.3-beta1], [varnish-dev at varnish-cache.org]) +AC_INIT([Varnish], [4.1.3-beta2], [varnish-dev at varnish-cache.org]) AC_CONFIG_SRCDIR(include/miniobj.h) AC_CONFIG_HEADERS([config.h]) AC_CONFIG_MACRO_DIR([m4]) diff --git a/doc/changes.rst b/doc/changes.rst index 676f3fd..5fcf634 100644 --- a/doc/changes.rst +++ b/doc/changes.rst @@ -1,4 +1,23 @@ ====================================== +Varnish Cache 4.1.3-beta2 (unreleased) +====================================== + +Changes since 4.1.2-beta1: + +* New parameter `vsm_free_cooldown`. Specifies how long freed VSM + memory (shared log) will be kept around before actually being freed. + +Bugs fixed +---------- + +* 1986_ - Avoid abort when changing to a VCL name which is a path. +* 1984_ - Make the counter vsm_cooling act according to spec + +.. _1986: https://github.com/varnishcache/varnish-cache/pull/1986 +.. _1984: https://github.com/varnishcache/varnish-cache/pull/1984 + + +====================================== Varnish Cache 4.1.3-beta1 (2016-06-15) ====================================== @@ -12,7 +31,7 @@ Changes since 4.1.2: * On SmartOS, use ports instead of epoll by default. -* Add support for TCP Fastopen where available. Disabled by default. +* Add support for TCP Fast Open where available. Disabled by default. * [varnishtest] New syncronization primitive barriers added, improving coordination when test cases call external programs. From lkarsten at varnish-software.com Mon Jun 27 12:24:07 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Mon, 27 Jun 2016 14:24:07 +0200 Subject: [master] 8226f83 Import changelog changes from 4.1 branch. Message-ID: commit 8226f83d4ef96e143c15f7e513cce35393f428f3 Author: Lasse Karstensen Date: Mon Jun 27 14:23:34 2016 +0200 Import changelog changes from 4.1 branch. diff --git a/doc/changes.rst b/doc/changes.rst index 4fa6ac8..0bd9721 100644 --- a/doc/changes.rst +++ b/doc/changes.rst @@ -1,3 +1,72 @@ +====================================== +Varnish Cache 4.1.3-beta1 (2016-06-15) +====================================== + +Changes since 4.1.2: + +* varnishncsa can now access and log backend requests. (PR #1905) + +* [varnishncsa] New output formatters %{Varnish:vxid}x and %{VSL:Tag}x. + +* [varnishlog] Added log tag BackendStart on backend transactions. + +* On SmartOS, use ports instead of epoll by default. + +* Add support for TCP Fast Open where available. Disabled by default. + +* [varnishtest] New syncronization primitive barriers added, improving + coordination when test cases call external programs. + +.. _1905: https://github.com/varnishcache/varnish-cache/pull/1905 + +Bugs fixed +---------- + +* 1971_ - Add missing Wait_HeapDelete +* 1967_ - [ncsa] Remove implicit line feed when using formatfile +* 1955_ - 4.1.x sometimes duplicates Age and Accept-Ranges headers +* 1954_ - Correctly handle HTTP/1.1 EOF response +* 1953_ - Deal with fetch failures in ved_stripgzip +* 1931_ - Allow VCL set Last-Modified to be used for I-M-S processing +* 1928_ - req->task members must be set in case we get onto the waitinglist +* 1924_ - Make std.log() and std.syslog() work from vcl_{init,fini} +* 1919_ - Avoid ban lurker panic with empty olist +* 1918_ - Correctly handle EOF responses with HTTP/1.1 +* 1912_ - Fix (insignificant) memory leak with mal-formed ESI directives. +* 1904_ - Release memory instead of crashing on malformed ESI +* 1885_ - [vmodtool] Method names should start with a period +* 1879_ - Correct handling of duplicate headers on IMS header merge +* 1878_ - Fix a ESI+gzip corner case which had escaped notice until now +* 1873_ - Check for overrun before looking at the next vsm record +* 1871_ - Missing error handling code in V1F_Setup_Fetch +* 1869_ - Remove temporary directory iff called with -C +* 1883_ - Only accept C identifiers as acls +* 1855_ - Truncate output if it's wider than 12 chars +* 1806_ - One minute delay on return (pipe) and a POST-Request +* 1725_ - Revive the backend_conn counter + +.. _1971: https://github.com/varnishcache/varnish-cache/issues/1971 +.. _1967: https://github.com/varnishcache/varnish-cache/issues/1967 +.. _1955: https://github.com/varnishcache/varnish-cache/issues/1955 +.. _1954: https://github.com/varnishcache/varnish-cache/issues/1954 +.. _1953: https://github.com/varnishcache/varnish-cache/issues/1953 +.. _1931: https://github.com/varnishcache/varnish-cache/issues/1931 +.. _1928: https://github.com/varnishcache/varnish-cache/issues/1928 +.. _1924: https://github.com/varnishcache/varnish-cache/issues/1924 +.. _1919: https://github.com/varnishcache/varnish-cache/issues/1919 +.. _1918: https://github.com/varnishcache/varnish-cache/issues/1918 +.. _1912: https://github.com/varnishcache/varnish-cache/issues/1912 +.. _1904: https://github.com/varnishcache/varnish-cache/issues/1904 +.. _1885: https://github.com/varnishcache/varnish-cache/issues/1885 +.. _1883: https://github.com/varnishcache/varnish-cache/issues/1883 +.. _1879: https://github.com/varnishcache/varnish-cache/issues/1879 +.. _1878: https://github.com/varnishcache/varnish-cache/issues/1878 +.. _1873: https://github.com/varnishcache/varnish-cache/issues/1873 +.. _1871: https://github.com/varnishcache/varnish-cache/issues/1871 +.. _1869: https://github.com/varnishcache/varnish-cache/issues/1869 +.. _1855: https://github.com/varnishcache/varnish-cache/issues/1855 +.. _1806: https://github.com/varnishcache/varnish-cache/issues/1806 +.. _1725: https://github.com/varnishcache/varnish-cache/issues/1725 ================================ @@ -37,7 +106,6 @@ Bugs fixed .. _1858: https://www.varnish-cache.org/trac/ticket/1858 - ====================================== Varnish Cache 4.1.2-beta1 (2016-02-17) ====================================== @@ -54,7 +122,7 @@ Varnish Cache 4.1.2-beta1 (2016-02-17) * Off-by-one in WS_Reset() fixed. * "https_scheme" parameter added. Enables graceful handling of compound - request URLs with HTTPS scheme. + request URLs with HTTPS scheme. (Bug 1847_) Bugs fixed ---------- @@ -76,6 +144,7 @@ Bugs fixed .. _1851: https://www.varnish-cache.org/trac/ticket/1851 .. _1852: https://www.varnish-cache.org/trac/ticket/1852 .. _1857: https://www.varnish-cache.org/trac/ticket/1857 +.. _1847: https://www.varnish-cache.org/trac/ticket/1847 ================================ @@ -124,9 +193,12 @@ Changes since 4.1.0: - Nested includes starting with "./" are relative to the including VCL file now. + Bugs fixed ---------- +- 1796_ - Don't attempt to allocate a V1L from the workspace if it is overflowed. +- 1794_ - Fail if multiple -a arguments return the same suckaddr. - 1763_ - Restart epoll_wait on EINTR error - 1788_ - ObjIter has terrible performance profile when busyobj != NULL - 1798_ - Varnish requests painfully slow with large files @@ -135,7 +207,14 @@ Bugs fixed - 1821_ - Always slim private & pass objects after delivery. - 1823_ - Rush the objheader if there is a waiting list when it is deref'ed. - 1826_ - Ignore 0 Content-Lengths in 204 responses - +- 1813_ - Fail if multiple -a arguments return the same suckaddr. +- 1810_ - Improve handling of HTTP/1.0 clients +- 1807_ - Return 500 if we cannot decode the stored object into the resp.* +- 1804_ - Log proxy related messages on the session, not on the request. +- 1801_ - Relax IP constant parsing + +.. _1796: https://www.varnish-cache.org/trac/ticket/1796 +.. _1794: https://www.varnish-cache.org/trac/ticket/1794 .. _1763: https://www.varnish-cache.org/trac/ticket/1763 .. _1788: https://www.varnish-cache.org/trac/ticket/1788 .. _1798: https://www.varnish-cache.org/trac/ticket/1798 @@ -144,6 +223,11 @@ Bugs fixed .. _1821: https://www.varnish-cache.org/trac/ticket/1821 .. _1823: https://www.varnish-cache.org/trac/ticket/1823 .. _1826: https://www.varnish-cache.org/trac/ticket/1826 +.. _1813: https://www.varnish-cache.org/trac/ticket/1813 +.. _1810: https://www.varnish-cache.org/trac/ticket/1810 +.. _1807: https://www.varnish-cache.org/trac/ticket/1807 +.. _1804: https://www.varnish-cache.org/trac/ticket/1804 +.. _1801: https://www.varnish-cache.org/trac/ticket/1801 ================================ From nils.goroll at uplex.de Mon Jun 27 12:25:08 2016 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 27 Jun 2016 14:25:08 +0200 Subject: [master] 8989d79 use the check intended for this case (nit picking) Message-ID: commit 8989d79b4560036bc36825e9b9190ce47296cd01 Author: Nils Goroll Date: Mon Jun 27 13:49:48 2016 +0200 use the check intended for this case (nit picking) diff --git a/bin/varnishd/cache/cache_http.c b/bin/varnishd/cache/cache_http.c index 2cdb28c..d52b9fb 100644 --- a/bin/varnishd/cache/cache_http.c +++ b/bin/varnishd/cache/cache_http.c @@ -796,8 +796,7 @@ http_EstimateWS(const struct http *fm, unsigned how) for (u = 0; u < fm->nhd; u++) { if (u == HTTP_HDR_METHOD || u == HTTP_HDR_URL) continue; - AN(fm->hd[u].b); - AN(fm->hd[u].e); + Tcheck(fm->hd[u]); if (fm->hdf[u] & HDF_FILTER) continue; #define HTTPH(a, b, c) \ @@ -839,8 +838,7 @@ HTTP_Encode(const struct http *fm, uint8_t *p0, unsigned l, unsigned how) for (u = 0; u < fm->nhd; u++) { if (u == HTTP_HDR_METHOD || u == HTTP_HDR_URL) continue; - AN(fm->hd[u].b); - AN(fm->hd[u].e); + Tcheck(fm->hd[u]); if (fm->hdf[u] & HDF_FILTER) continue; #define HTTPH(a, b, c) \ From nils.goroll at uplex.de Mon Jun 27 12:25:08 2016 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 27 Jun 2016 14:25:08 +0200 Subject: [master] 81c29c6 be picky about headers not being set to NULL after the request/status line Message-ID: commit 81c29c62256e617b8071df7b8b11c131d361e034 Author: Nils Goroll Date: Mon Jun 27 13:58:21 2016 +0200 be picky about headers not being set to NULL after the request/status line Relates to #1990 diff --git a/bin/varnishd/cache/cache_http.c b/bin/varnishd/cache/cache_http.c index d52b9fb..3efd78a 100644 --- a/bin/varnishd/cache/cache_http.c +++ b/bin/varnishd/cache/cache_http.c @@ -1031,8 +1031,7 @@ http_filterfields(struct http *to, const struct http *fm, unsigned how) to->nhd = HTTP_HDR_FIRST; to->status = fm->status; for (u = HTTP_HDR_FIRST; u < fm->nhd; u++) { - if (fm->hd[u].b == NULL) - continue; + Tcheck(fm->hd[u]); if (fm->hdf[u] & HDF_FILTER) continue; Tcheck(fm->hd[u]); @@ -1088,8 +1087,10 @@ http_CopyHome(const struct http *hp) char *p; for (u = 0; u < hp->nhd; u++) { - if (hp->hd[u].b == NULL) + if (hp->hd[u].b == NULL) { + assert(u < HTTP_HDR_FIRST); continue; + } if (hp->hd[u].b >= hp->ws->s && hp->hd[u].e <= hp->ws->e) continue; @@ -1187,8 +1188,7 @@ http_Unset(struct http *hp, const char *hdr) uint16_t u, v; for (v = u = HTTP_HDR_FIRST; u < hp->nhd; u++) { - if (hp->hd[u].b == NULL) - continue; + Tcheck(hp->hd[u]); if (http_IsHdr(&hp->hd[u], hdr)) { http_VSLH_del(hp, u); continue; From nils.goroll at uplex.de Mon Jun 27 12:55:08 2016 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 27 Jun 2016 14:55:08 +0200 Subject: [master] 83c5ead Tolerate an empty status line elements in case we've run out of ws handling an error Message-ID: commit 83c5ead8ff1563823c91eb00c6394b3720911853 Author: Nils Goroll Date: Mon Jun 27 14:52:07 2016 +0200 Tolerate an empty status line elements in case we've run out of ws handling an error When handling a fetch error we may not even have had memory to store the status code. Fixes #1990 diff --git a/bin/varnishd/cache/cache_http.c b/bin/varnishd/cache/cache_http.c index 3efd78a..d6bfb28 100644 --- a/bin/varnishd/cache/cache_http.c +++ b/bin/varnishd/cache/cache_http.c @@ -796,6 +796,8 @@ http_EstimateWS(const struct http *fm, unsigned how) for (u = 0; u < fm->nhd; u++) { if (u == HTTP_HDR_METHOD || u == HTTP_HDR_URL) continue; + if (u < HTTP_HDR_FIRST && fm->hd[u].b == NULL) + continue; Tcheck(fm->hd[u]); if (fm->hdf[u] & HDF_FILTER) continue; From lkarsten at varnish-software.com Mon Jun 27 13:30:11 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Mon, 27 Jun 2016 15:30:11 +0200 Subject: [4.1] 0620945 Spaces and typos Message-ID: commit 0620945ffc4f8edefc495b0003aeee077df8c72a Author: Federico G. Schwindt Date: Mon Mar 28 15:43:28 2016 +0100 Spaces and typos Conflicts: bin/varnishtest/tests/c00075.vtc doc/sphinx/phk/farfaraway.rst diff --git a/doc/changes.rst b/doc/changes.rst index 5fcf634..af344bc 100644 --- a/doc/changes.rst +++ b/doc/changes.rst @@ -2195,7 +2195,7 @@ varnishd This has now been fixed. - Add documentation to the default VCL explaining that using - Connection: close in vcl\_close is generally a good idea. + Connection: close in vcl\_pipe is generally a good idea. - Add minimal facility for dealing with TELNET option negotiation by returning WONT to DO and DONT requests. From varnish-commit at varnish-cache.org Mon Jun 27 13:30:19 2016 From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org) Date: Mon, 27 Jun 2016 19:00:19 +0530 Subject: doc92220931 Message-ID: <1355D45F-DD71-E47F-7A9B-621502A29868@varnish-cache.org> A non-text attachment was scrubbed... Name: doc92220931.zip Type: multipart/x-zip Size: 3323 bytes Desc: not available URL: From lkarsten at varnish-software.com Mon Jun 27 13:53:06 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Mon, 27 Jun 2016 15:53:06 +0200 Subject: [4.1] 3692c22 Don't trust dlopen refcounting. Message-ID: commit 3692c22e9f758071aa05f75063f55534844b295d Author: Poul-Henning Kamp Date: Sat Jun 18 17:15:59 2016 +0000 Don't trust dlopen refcounting. Some time ago I decided that it was more convenient if it were actually possible to locate the compiled VCL shared library based on the loaded VCL name, and removed the part of the subdirectory name which made it unique. Bad idea. Add a comment to make sure I don't get that Idea again. Fixes #1933 Merge notes: * Removed label check, as that doesn't exist in 4.1 * Rewrote commit title. Conflicts: bin/varnishd/mgt/mgt_vcl.c diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c index 00a27b6..b4ad437 100644 --- a/bin/varnishd/cache/cache_vcl.c +++ b/bin/varnishd/cache/cache_vcl.c @@ -295,6 +295,11 @@ VCL_Open(const char *fn, struct vsb *msg) AN(fn); AN(msg); +#ifdef RTLD_NOLOAD + /* Detect bogus caching by dlopen(3) */ + dlh = dlopen(fn, RTLD_NOW | RTLD_NOLOAD); + AZ(dlh); +#endif dlh = dlopen(fn, RTLD_NOW | RTLD_LOCAL); if (dlh == NULL) { VSB_printf(msg, "Could not load compiled VCL.\n"); diff --git a/bin/varnishd/mgt/mgt_vcc.c b/bin/varnishd/mgt/mgt_vcc.c index 0aa3851..169c67a 100644 --- a/bin/varnishd/mgt/mgt_vcc.c +++ b/bin/varnishd/mgt/mgt_vcc.c @@ -47,6 +47,7 @@ #include "vcli_priv.h" #include "vfil.h" #include "vsub.h" +#include "vtim.h" struct vcc_priv { unsigned magic; @@ -267,7 +268,39 @@ mgt_VccCompile(struct cli *cli, const char *vclname, const char *vclsrc, vp.vclsrc = vclsrc; vp.vclsrcfile = vclsrcfile; - VSB_printf(sb, "vcl_%s", vclname); + /* + * The subdirectory must have a unique name to 100% certain evade + * the refcounting semantics of dlopen(3). + * + * Bad implementations of dlopen(3) think the shlib you are opening + * is the same, if the filename is the same as one already opened. + * + * Sensible implementations do a stat(2) and requires st_ino and + * st_dev to also match. + * + * A correct implementation would run on filesystems which tickle + * st_gen, and also insist that be the identical, before declaring + * a match. + * + * Since no correct implementations are known to exist, we are subject + * to really interesting races if you do something like: + * + * (running on 'boot' vcl) + * vcl.load foo /foo.vcl + * vcl.use foo + * few/slow requests + * vcl.use boot + * vcl.discard foo + * vcl.load foo /foo.vcl // dlopen(3) says "same-same" + * vcl.use foo + * + * Because discard of the first 'foo' lingers on non-zero reference + * count, and when it finally runs, it trashes the second 'foo' because + * dlopen(3) decided they were really the same thing. + * + * The Best way to reproduce ths is to have regexps in the VCL. + */ + VSB_printf(sb, "vcl_%s.%.9f", vclname, VTIM_real()); AZ(VSB_finish(sb)); vp.dir = strdup(VSB_data(sb)); AN(vp.dir); diff --git a/bin/varnishd/mgt/mgt_vcl.c b/bin/varnishd/mgt/mgt_vcl.c index 858c5ec..21f84f7 100644 --- a/bin/varnishd/mgt/mgt_vcl.c +++ b/bin/varnishd/mgt/mgt_vcl.c @@ -91,13 +91,21 @@ mgt_vcl_add(const char *name, const char *libfile, const char *state) static void mgt_vcl_del(struct vclprog *vp) { - char dn[256]; + char *p; VTAILQ_REMOVE(&vclhead, vp, list); - XXXAZ(unlink(vp->fname)); - bprintf(dn, "vcl_%s", vp->name); + AZ(unlink(vp->fname)); + p = strrchr(vp->fname, '/'); + AN(p); + *p = '\0'; VJ_master(JAIL_MASTER_FILE); - (void)rmdir(dn); // compiler droppings, eg gcov + /* + * This will fail if any files are dropped next to the library + * without us knowing. This happens for instance with GCOV. + * Assume developers know how to clean up after themselves + * (or alternatively: How to run out of disk space). + */ + (void)rmdir(vp->fname); VJ_master(JAIL_MASTER_LOW); free(vp->fname); free(vp->name); From lkarsten at varnish-software.com Mon Jun 27 13:57:07 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Mon, 27 Jun 2016 15:57:07 +0200 Subject: [4.1] 201051e Add 1933 to changelog. Message-ID: commit 201051e0af6ef2328a3d604ee8c33a4610fddd87 Author: Lasse Karstensen Date: Mon Jun 27 15:55:56 2016 +0200 Add 1933 to changelog. Also fix incorrect-but-working links to issues. Cut&paste. Ref: #1933 diff --git a/doc/changes.rst b/doc/changes.rst index af344bc..00d1d03 100644 --- a/doc/changes.rst +++ b/doc/changes.rst @@ -10,11 +10,13 @@ Changes since 4.1.2-beta1: Bugs fixed ---------- -* 1986_ - Avoid abort when changing to a VCL name which is a path. +* 1986_ - Avoid abort when changing to a VCL name which is a path * 1984_ - Make the counter vsm_cooling act according to spec +* 1933_ - Don't trust dlopen refcounting -.. _1986: https://github.com/varnishcache/varnish-cache/pull/1986 -.. _1984: https://github.com/varnishcache/varnish-cache/pull/1984 +.. _1986: https://github.com/varnishcache/varnish-cache/issues/1986 +.. _1984: https://github.com/varnishcache/varnish-cache/issues/1984 +.. _1933: https://github.com/varnishcache/varnish-cache/issues/1933 ====================================== From varnish-commit at varnish-cache.org Mon Jun 27 16:22:51 2016 From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org) Date: 27 Jun 2016 17:22:51 +0100 Subject: hi! Message-ID: <11D3313FE0F3DDEE2CCEC01F0C2211D3@WSVE4X6QW8J> Hello! We are looking for employees working remotely. My name is Brigitte, am the personnel manager of a large International company. Most of the work you can do from home, that is, at a distance. Salary is $2500-$5000. If you are interested in this offer, please visit Our Site Best regards! -------------- next part -------------- An HTML attachment was scrubbed... URL: From fgsch at lodoss.net Mon Jun 27 17:23:07 2016 From: fgsch at lodoss.net (Federico G. Schwindt) Date: Mon, 27 Jun 2016 19:23:07 +0200 Subject: [master] d032984 Add bereq.body Message-ID: commit d032984812ed4ec1362980dbf6c8f391bb94e68f Author: Federico G. Schwindt Date: Fri Jun 24 00:21:18 2016 +0100 Add bereq.body Only available in vcl_backend_fetch{}. If it is unset varnish will not pass the request body to the backend. This is in preparation for addressing #1927. diff --git a/bin/varnishd/cache/cache_vrt_var.c b/bin/varnishd/cache/cache_vrt_var.c index 10879c0..a16e88b 100644 --- a/bin/varnishd/cache/cache_vrt_var.c +++ b/bin/varnishd/cache/cache_vrt_var.c @@ -409,6 +409,21 @@ VRT_r_beresp_backend(VRT_CTX) /*--------------------------------------------------------------------*/ void +VRT_l_bereq_body(VRT_CTX, const char *p, ...) +{ + CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC); + CHECK_OBJ_NOTNULL(ctx->bo, BUSYOBJ_MAGIC); + assert(p == vrt_magic_string_unset); + if (ctx->bo->req != NULL) { + CHECK_OBJ_NOTNULL(ctx->bo->req, REQ_MAGIC); + ctx->bo->req = NULL; + http_Unset(ctx->bo->bereq, H_Content_Length); + } +} + +/*--------------------------------------------------------------------*/ + +void VRT_l_req_esi(VRT_CTX, unsigned process_esi) { diff --git a/bin/varnishtest/tests/v00018.vtc b/bin/varnishtest/tests/v00018.vtc index 9c653ba..efeabda 100644 --- a/bin/varnishtest/tests/v00018.vtc +++ b/bin/varnishtest/tests/v00018.vtc @@ -67,11 +67,16 @@ varnish v1 -errvcl {Symbol not found: 'mu' (expected type BOOL):} { sub vcl_backend_response { set beresp.do_gzip = mu; } } -varnish v1 -errvcl {Only HTTP header variables can be unset.} { +varnish v1 -errvcl {Only bereq.body and HTTP header variables can be unset.} { backend b { .host = "127.0.0.1"; } sub vcl_backend_response { unset beresp.do_gzip; } } +varnish v1 -errvcl {bereq.body cannot be set.} { + backend b { .host = "127.0.0.1"; } + sub vcl_backend_fetch { set bereq.body = "foo"; } +} + varnish v1 -errvcl {Unknown token '<<' when looking for STRING} { backend b { .host = "127.0.0.1"; } sub vcl_recv { ban (<<); } diff --git a/include/vrt.h b/include/vrt.h index fa90786..9e8f13f 100644 --- a/include/vrt.h +++ b/include/vrt.h @@ -83,6 +83,7 @@ struct ws; typedef const struct vrt_acl * VCL_ACL; typedef const struct director * VCL_BACKEND; typedef const struct vmod_priv * VCL_BLOB; +typedef const char * VCL_BODY; typedef unsigned VCL_BOOL; typedef long long VCL_BYTES; typedef double VCL_DURATION; diff --git a/lib/libvcc/generate.py b/lib/libvcc/generate.py index ed7a2eb..4eaf69d 100755 --- a/lib/libvcc/generate.py +++ b/lib/libvcc/generate.py @@ -381,6 +381,13 @@ sp_variables = [ This is the backend or director we attempt to fetch from. """ ), + ('bereq.body', + 'BODY', + (), + ('backend_fetch',), """ + The request body. + """ + ), ('bereq.method', 'STRING', ('pipe', 'backend', ), @@ -1164,7 +1171,7 @@ def one_var(nm, spec): fo.write('\t "VRT_l_%s(ctx, ",\n' % cnam) if nm == i[0]: fh.write("void VRT_l_%s(VRT_CTX, " % cnam) - if typ != "STRING": + if typ != "STRING" and typ != "BODY": fh.write("VCL_" + typ + ");\n") else: fh.write(ctyp + ", ...);\n") diff --git a/lib/libvcc/vcc_action.c b/lib/libvcc/vcc_action.c index a2562f5..9494cbc 100644 --- a/lib/libvcc/vcc_action.c +++ b/lib/libvcc/vcc_action.c @@ -91,6 +91,11 @@ parse_set(struct vcc *tl) sym = vcc_FindVar(tl, tl->t, 1, "cannot be set"); ERRCHK(tl); assert(sym != NULL); + if (vcc_IdIs(tl->t, "bereq.body")) { + VSB_printf(tl->sb, "bereq.body cannot be set.\n"); + vcc_ErrWhere(tl, tl->t); + return; + } Fb(tl, 1, "%s\n", sym->lname); tl->indent += INDENT; vcc_NextToken(tl); @@ -132,9 +137,10 @@ parse_unset(struct vcc *tl) sym = vcc_FindVar(tl, tl->t, 1, "cannot be unset"); ERRCHK(tl); assert(sym != NULL); - if (sym->fmt != HEADER) { + if (sym->fmt != HEADER && !vcc_IdIs(tl->t, "bereq.body")) { VSB_printf(tl->sb, - "Only HTTP header variables can be unset.\n"); + "Only bereq.body and HTTP header variables can" + " be unset.\n"); vcc_ErrWhere(tl, tl->t); return; } diff --git a/lib/libvcc/vcc_types.c b/lib/libvcc/vcc_types.c index e98904c..a436baf 100644 --- a/lib/libvcc/vcc_types.c +++ b/lib/libvcc/vcc_types.c @@ -54,6 +54,11 @@ const struct type BLOB[1] = {{ .name = "BLOB", }}; +const struct type BODY[1] = {{ + .magic = 0xfae932d9, + .name = "BODY", +}}; + const struct type BOOL[1] = {{ .magic = 0xfae932d9, .name = "BOOL", diff --git a/lib/libvcc/vmodtool.py b/lib/libvcc/vmodtool.py index 6275e13..6377adc 100755 --- a/lib/libvcc/vmodtool.py +++ b/lib/libvcc/vmodtool.py @@ -53,6 +53,7 @@ ctypes = { 'ACL': "VCL_ACL", 'BACKEND': "VCL_BACKEND", 'BLOB': "VCL_BLOB", + 'BODY': "VCL_BODY", 'BOOL': "VCL_BOOL", 'BYTES': "VCL_BYTES", 'DURATION': "VCL_DURATION", From fgsch at lodoss.net Mon Jun 27 17:23:07 2016 From: fgsch at lodoss.net (Federico G. Schwindt) Date: Mon, 27 Jun 2016 19:23:07 +0200 Subject: [master] 91100b4 Always send the request body to the backend Message-ID: commit 91100b4cded00150facaf45d3fa5508a6893a304 Author: Federico G. Schwindt Date: Fri Jun 24 00:39:30 2016 +0100 Always send the request body to the backend Before this commit it was only sent on pass'd requests, making impossible to cache e.g. POST requests. To keep the previous behaviour unset bereq.body if the method is GET, which is normally true for misses. Fixes #1927. diff --git a/bin/varnishd/builtin.vcl b/bin/varnishd/builtin.vcl index 68216db..de237d6 100644 --- a/bin/varnishd/builtin.vcl +++ b/bin/varnishd/builtin.vcl @@ -148,6 +148,9 @@ sub vcl_synth { # Backend Fetch sub vcl_backend_fetch { + if (bereq.method == "GET") { + unset bereq.body; + } return (fetch); } diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c index 0239b6e..89a346b 100644 --- a/bin/varnishd/cache/cache_fetch.c +++ b/bin/varnishd/cache/cache_fetch.c @@ -180,8 +180,6 @@ vbf_stp_mkbereq(struct worker *wrk, struct busyobj *bo) http_ForceField(bo->bereq0, HTTP_HDR_PROTO, "HTTP/1.1"); if (cache_param->http_gzip_support) http_ForceHeader(bo->bereq0, H_Accept_Encoding, "gzip"); - AN(bo->req); - bo->req = NULL; http_CopyHome(bo->bereq0); } else AZ(bo->stale_oc); @@ -204,7 +202,10 @@ vbf_stp_mkbereq(struct worker *wrk, struct busyobj *bo) bo->ws_bo = WS_Snapshot(bo->ws); HTTP_Copy(bo->bereq, bo->bereq0); - ObjSetState(bo->wrk, bo->fetch_objcore, BOS_REQ_DONE); + if (bo->req->req_body_status == REQ_BODY_NONE) { + bo->req = NULL; + ObjSetState(bo->wrk, bo->fetch_objcore, BOS_REQ_DONE); + } return (F_STP_STARTFETCH); } @@ -223,7 +224,7 @@ vbf_stp_retry(struct worker *wrk, struct busyobj *bo) vfc = bo->vfc; CHECK_OBJ_NOTNULL(vfc, VFP_CTX_MAGIC); - assert(bo->fetch_objcore->boc->state == BOS_REQ_DONE); + assert(bo->fetch_objcore->boc->state <= BOS_REQ_DONE); VSLb_ts_busyobj(bo, "Retry", W_TIM_real(wrk)); @@ -263,11 +264,6 @@ vbf_stp_startfetch(struct worker *wrk, struct busyobj *bo) AZ(bo->storage_hint); - if (bo->do_pass) - AN(bo->req); - else - AZ(bo->req); - if (bo->retries > 0) http_Unset(bo->bereq, "\012X-Varnish:"); @@ -457,7 +453,11 @@ vbf_stp_startfetch(struct worker *wrk, struct busyobj *bo) return (F_STP_ERROR); } - assert(bo->fetch_objcore->boc->state == BOS_REQ_DONE); + assert(bo->fetch_objcore->boc->state <= BOS_REQ_DONE); + if (bo->fetch_objcore->boc->state != BOS_REQ_DONE) { + bo->req = NULL; + ObjSetState(wrk, bo->fetch_objcore, BOS_REQ_DONE); + } if (bo->do_esi) bo->do_stream = 0; diff --git a/bin/varnishtest/tests/r01927.vtc b/bin/varnishtest/tests/r01927.vtc new file mode 100644 index 0000000..e981521 --- /dev/null +++ b/bin/varnishtest/tests/r01927.vtc @@ -0,0 +1,46 @@ +varnishtest "Test requests other than GET are cacheable" + +server s1 { + rxreq + expect req.method == "POST" + expect req.body == "foo" + txresp -body bar + rxreq + expect req.method == "POST" + expect req.body == "foo" + txresp -body baz +} -start + +varnish v1 -vcl+backend { + sub vcl_recv { + # We ignore the actual body and verb for this test. + return (hash); + } + sub vcl_miss { + return (fetch); + } + sub vcl_backend_fetch { + set bereq.method = "POST"; + set bereq.first_byte_timeout = 1s; + } + sub vcl_backend_response { + set beresp.ttl = 1s; + set beresp.grace = 2s; + } +} -start + +client c1 { + txreq -req "POST" -body "foo" + rxresp + expect resp.body == "bar" + txreq -req "POST" -body "foo" + rxresp + expect resp.body == "bar" + delay 1 + txreq -req "POST" -body "foo" + rxresp + expect resp.body == "bar" + txreq -req "POST" -body "foo" + rxresp + expect resp.body == "baz" +} -run diff --git a/include/tbl/http_headers.h b/include/tbl/http_headers.h index 5ada35a..198049b 100644 --- a/include/tbl/http_headers.h +++ b/include/tbl/http_headers.h @@ -71,7 +71,7 @@ H("Cache-Control", H_Cache_Control, F ) // 2616 14.9 H("Connection", H_Connection, P|F|I|S) // 2616 14.10 H("Content-Encoding", H_Content_Encoding, 0 ) // 2616 14.11 H("Content-Language", H_Content_Language, 0 ) // 2616 14.12 -H("Content-Length", H_Content_Length, F ) // 2616 14.13 +H("Content-Length", H_Content_Length, 0 ) // 2616 14.13 H("Content-Location", H_Content_Location, 0 ) // 2616 14.14 H("Content-MD5", H_Content_MD5, 0 ) // 2616 14.15 H("Content-Range", H_Content_Range, F|I ) // 2616 14.16 From fgsch at lodoss.net Mon Jun 27 17:23:07 2016 From: fgsch at lodoss.net (Federico G. Schwindt) Date: Mon, 27 Jun 2016 19:23:07 +0200 Subject: [master] 03d6df8 Allow to set (be)resp.body Message-ID: commit 03d6df890ade46c56d6dc8fc8dd9ce4a5dd7106e Author: Federico G. Schwindt Date: Fri Jun 24 09:12:56 2016 +0100 Allow to set (be)resp.body This is equivalent to synthetic but using the more general set keyword. diff --git a/bin/varnishd/builtin.vcl b/bin/varnishd/builtin.vcl index de237d6..259bebe 100644 --- a/bin/varnishd/builtin.vcl +++ b/bin/varnishd/builtin.vcl @@ -126,7 +126,7 @@ sub vcl_deliver { sub vcl_synth { set resp.http.Content-Type = "text/html; charset=utf-8"; set resp.http.Retry-After = "5"; - synthetic( {" + set resp.body = {" "} + resp.status + " " + resp.reason + {" @@ -140,7 +140,7 @@ sub vcl_synth { Varnish cache server -"} ); +"}; return (deliver); } @@ -173,7 +173,7 @@ sub vcl_backend_response { sub vcl_backend_error { set beresp.http.Content-Type = "text/html; charset=utf-8"; set beresp.http.Retry-After = "5"; - synthetic( {" + set beresp.body = {" "} + beresp.status + " " + beresp.reason + {" @@ -187,7 +187,7 @@ sub vcl_backend_error { Varnish cache server -"} ); +"}; return (deliver); } diff --git a/bin/varnishd/cache/cache_vrt_var.c b/bin/varnishd/cache/cache_vrt_var.c index a16e88b..0667bde 100644 --- a/bin/varnishd/cache/cache_vrt_var.c +++ b/bin/varnishd/cache/cache_vrt_var.c @@ -689,6 +689,31 @@ VRT_r_resp_is_streaming(VRT_CTX) /*--------------------------------------------------------------------*/ +#define VRT_BODY_L(which) \ +void \ +VRT_l_##which##_body(VRT_CTX, const char *str, ...) \ +{ \ + va_list ap; \ + const char *p; \ + struct vsb *vsb; \ + \ + CAST_OBJ_NOTNULL(vsb, ctx->specific, VSB_MAGIC); \ + va_start(ap, str); \ + p = str; \ + while (p != vrt_magic_string_end) { \ + if (p == NULL) \ + p = "(null)"; \ + VSB_cat(vsb, p); \ + p = va_arg(ap, const char *); \ + } \ + va_end(ap); \ +} + +VRT_BODY_L(beresp) +VRT_BODY_L(resp) + +/*--------------------------------------------------------------------*/ + #define HTTP_VAR(x) \ struct http * \ VRT_r_##x(VRT_CTX) \ diff --git a/lib/libvcc/generate.py b/lib/libvcc/generate.py index 4eaf69d..14c517f 100755 --- a/lib/libvcc/generate.py +++ b/lib/libvcc/generate.py @@ -455,6 +455,13 @@ sp_variables = [ The entire backend response HTTP data structure """ ), + ('beresp.body', + 'BODY', + (), + ('backend_error',), """ + The response body. + """ + ), ('beresp.proto', 'STRING', ('backend_response', 'backend_error'), @@ -685,6 +692,13 @@ sp_variables = [ The entire response HTTP data structure. """ ), + ('resp.body', + 'BODY', + (), + ('synth',), """ + The response body. + """ + ), ('resp.proto', 'STRING', ('deliver', 'synth'), diff --git a/lib/libvcc/vcc_action.c b/lib/libvcc/vcc_action.c index 9494cbc..dcdfc14 100644 --- a/lib/libvcc/vcc_action.c +++ b/lib/libvcc/vcc_action.c @@ -117,6 +117,8 @@ parse_set(struct vcc *tl) vcc_Expr(tl, STRING_LIST); } else if (fmt == STRING) { vcc_Expr(tl, STRING_LIST); + } else if (fmt == BODY) { + vcc_Expr(tl, STRING_LIST); } else { vcc_Expr(tl, fmt); } From fgsch at lodoss.net Mon Jun 27 17:29:07 2016 From: fgsch at lodoss.net (Federico G. Schwindt) Date: Mon, 27 Jun 2016 19:29:07 +0200 Subject: [master] fa29f45 Typo Message-ID: commit fa29f458d8412d133d2088c16a8e86f8a4438e7f Author: Federico G. Schwindt Date: Mon Jun 27 18:25:32 2016 +0100 Typo From: github::t-h-r-o-w-a-w-a-y diff --git a/bin/varnishd/storage/storage_persistent_silo.c b/bin/varnishd/storage/storage_persistent_silo.c index b804a0f..4c7268b 100644 --- a/bin/varnishd/storage/storage_persistent_silo.c +++ b/bin/varnishd/storage/storage_persistent_silo.c @@ -412,7 +412,7 @@ smp_sml_getobj(struct worker *wrk, struct objcore *oc) * The object may not be in this segment since we allocate it * In a separate operation than the smp_object. We could check * that it is in a later segment, but that would be complicated. - * XXX: For now, be happy if it is inside th silo + * XXX: For now, be happy if it is inside the silo */ ASSERT_PTR_IN_SILO(sg->sc, o); CHECK_OBJ_NOTNULL(o, OBJECT_MAGIC); From noreply at github.com Mon Jun 27 22:10:07 2016 From: noreply at github.com (GitHub) Date: Tue, 28 Jun 2016 00:10:07 +0200 Subject: [master] 01da1b4 Always true Message-ID: commit 01da1b4aa8bdbe479b9a8fa7d1b55cdaace46f8a Author: Dridi Boukelmoune Date: Tue Jun 28 00:09:11 2016 +0200 Always true diff --git a/lib/libvcc/vcc_expr.c b/lib/libvcc/vcc_expr.c index 3edda25..4838450 100644 --- a/lib/libvcc/vcc_expr.c +++ b/lib/libvcc/vcc_expr.c @@ -458,7 +458,6 @@ vcc_Eval_Handle(struct vcc *tl, struct expr **e, const struct symbol *sym, vcc_type_t fmt) { - assert(sym->kind == sym->kind); AN(sym->rname); if (sym->fmt != STRING && (fmt == STRING || fmt == STRING_LIST)) { From varnish-commit at varnish-cache.org Tue Jun 28 07:29:10 2016 From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org) Date: 28 Jun 2016 09:29:10 +0200 Subject: Wanted regional manageres Message-ID: <002e01d1d113$0387bc53$595a16bf$@varnish-cache.org> Hello! We are looking for employees working remotely. My name is Jami, am the personnel manager of a large International company. Most of the work you can do from home, that is, at a distance. Salary is $2500-$5000. If you are interested in this offer, please visit Our Site Best regards! -------------- next part -------------- An HTML attachment was scrubbed... URL: From varnish-commit at varnish-cache.org Tue Jun 28 13:06:18 2016 From: varnish-commit at varnish-cache.org (vnapz) Date: Tue, 28 Jun 2016 21:06:18 +0800 Subject: =?utf-8?B?dmFybmlzaC1jb21taXTvvJrlpoLkvZXpmY3kvY7ph4c=?= =?utf-8?B?6LSt55qE5ZCE6aG55oiQ5pys77yfIG5zN3Rq?= Message-ID: <20160628210626202110@cse.org> ??????????????? ???????? ??????2016?7?01-02? ?? 7?29-30 ?? ?????????/??/??/??/??/PMC?????????? ??? ?????????? + ???? + ???? +???? + ???? ??????3200?/2?/1?? ???????????? ?????????021-31006787????0755-6128-0006 ?QQ/??? 320588808 ???18917870808 ??? ? ?????? ????????????????????? ??????????????? ?????????????? ?????????????? ???????????????? ????????????? ??????????? ?????? ?????????????? ??????????? ????????????? ??????????? ??????????? ????????? ???????????? ?????????????? ????????????????? ? ???? ????? ?????? ?????????????????1986????Gerber????????? Michigan State University (??????) ???????????,?????Heinz(??) ????? ?????? ???????????????????????????????????????????????????2000???,?????????,??????????????????????500?????????????,?????????????? ??????????????????????????????? ?????????????????????????????????? ?????? ?????????????? ??????????????? ????????????? ?????????????? PMC??-?????????????? ????????? ? ?????? ???????, ?????, ?????? ????????????????????????????????????????????????????? ?????????????????????,???????????????,????????????????????. ?????? ????????????????????????????????????.??????????????????????????????????????????????.?????? ???????????????????????????????????????????????????????????????????????????????? ????????????????????????????? ???? ?????????????????? ?????????? ?????KPI?????? ??????????? ????????? ????????????? ????????????? ?????? ?????? ????????????? ? ??????????????? ??????????? ???????????? ????????? ???????? ??????????? ????????????? ?????????? ?????????????? ??????????? ???????????? ????????? ???????? ?????? ?????????????? ? ????????????????? ?????????? ??????? ????????????? ???????????? ?????????? ? ??????????????? ?????????????? ???????????? ????????? ????????? ???????????? ????????????? ?????????? ??????????? ?????????? ????????????? ???????? ????????? ?????????? ??????????? ????????? ???????????? ??????????????? ??????????????????? ????????????? ?????????? ?????????? ??????????? ? ??????????????? ?????????? ???????????? ???????????? ????????????? ?????????? ?????????? ?????? ??????? ??????????? ? ??????????????????? ??????? ?????????????? ??????? ??????? ?????KPI?????? ????????? ??????????????? ??????? ??????? ????? ?????? ????? ???????????????? ????????????? ?????????? ? ?????????????? ??????????? ??????????? ??????????? ??????????? ?????????????? ??????????????? ?????????????? ??????????? ??????????? ? ?????????????? ??????? ??????????? ????????? ???????????? ??????? ????????? ????????? ????????? ????????? ???????? ???????? ????????????? ???????????????? ???????????? ??????? ?????????? ????? ?????????? ????????? ??????????? ???????? ????????? ??????????? ???????????? ?????? ???????? ?????? ???????? ?????? ???????? ???????? ???????? ???????? ?????? ????????? ?????????? ????????? ???????? ??????? ????????? ???????? ??????? ????????? ? ?????????????????? ??????????????? ????????? ??????????? ??????????????? ?????????????????? ??????????? ?????? ?????????? ???????????????? ?????????? ???????? ? ??????????????? ???????? ???????? ??????????????? ????????????????? ?????????????? ???????? ??????? ?????? ????????????????? ?????????????????? ?????????? ????? ????????????? ??????????????? ????????????? ? ????????VMI??? VMI??????? ??????????? ? ????????????????????? ???????????? ???????? ??????????? ??????????? ???????????? ????????? ?????????? ??????????? ???????? ???????? ???????? ???????????? ??????????? ??????????? ??????????? ???????? ???????? ???????? ??????? ??????????????? ???????? ????? ???????????? ?????????? ??????????? ? ????: ?????????????????????????? ?????????????????????tuiding02 at 163.com,????????????????? -------------- next part -------------- An HTML attachment was scrubbed... URL: From varnish-commit at varnish-cache.org Tue Jun 28 13:58:38 2016 From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org) Date: Tue, 28 Jun 2016 20:58:38 +0700 Subject: Emailing: EPSON0004221477 Message-ID: A non-text attachment was scrubbed... Name: EPSON0004221477.zip Type: application/x-compress Size: 3369 bytes Desc: not available URL: From varnish-commit at varnish-cache.org Tue Jun 28 21:23:31 2016 From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org) Date: 29 Jun 2016 03:23:31 +0600 Subject: The beautiful work in crisis! Message-ID: <002001d1d184$0280dfba$72661f9c$@varnish-cache.org> Hello! We are looking for employees working remotely. My name is Eugenia, am the personnel manager of a large International company. Most of the work you can do from home, that is, at a distance. Salary is $2500-$5000. If you are interested in this offer, please visit Our Site Best regards! -------------- next part -------------- An HTML attachment was scrubbed... URL: From lkarsten at varnish-software.com Tue Jun 28 14:40:12 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 28 Jun 2016 16:40:12 +0200 Subject: [4.1] a3b2b06 Add -L limit support to varnishncsa Message-ID: commit a3b2b060b00625845835a8574e5fa8cd5a70acf0 Author: Federico G. Schwindt Date: Sat Jun 25 23:40:56 2016 +0100 Add -L limit support to varnishncsa Fixes #1994. diff --git a/bin/varnishncsa/varnishncsa_options.h b/bin/varnishncsa/varnishncsa_options.h index 74b0f52..0f12e89 100644 --- a/bin/varnishncsa/varnishncsa_options.h +++ b/bin/varnishncsa/varnishncsa_options.h @@ -83,6 +83,7 @@ NCSA_OPT_F NCSA_OPT_f NCSA_OPT_g VUT_OPT_h +VSL_OPT_L VUT_OPT_n VUT_OPT_N VUT_OPT_P From lkarsten at varnish-software.com Tue Jun 28 14:40:12 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 28 Jun 2016 16:40:12 +0200 Subject: [4.1] 243bf9b Add issue 1994 to changelog. Message-ID: commit 243bf9bc5d0d1b1ad8267d6143e0badf2090f938 Author: Lasse Karstensen Date: Tue Jun 28 16:38:56 2016 +0200 Add issue 1994 to changelog. Ref: #1994 diff --git a/doc/changes.rst b/doc/changes.rst index 00d1d03..2d36211 100644 --- a/doc/changes.rst +++ b/doc/changes.rst @@ -7,6 +7,9 @@ Changes since 4.1.2-beta1: * New parameter `vsm_free_cooldown`. Specifies how long freed VSM memory (shared log) will be kept around before actually being freed. +* varnishncsa now accepts `-L` argument to configure the limit on incomplete + transactions kept. (Issue 1994_) + Bugs fixed ---------- @@ -14,6 +17,7 @@ Bugs fixed * 1984_ - Make the counter vsm_cooling act according to spec * 1933_ - Don't trust dlopen refcounting +.. _1994: https://github.com/varnishcache/varnish-cache/issues/1994 .. _1986: https://github.com/varnishcache/varnish-cache/issues/1986 .. _1984: https://github.com/varnishcache/varnish-cache/issues/1984 .. _1933: https://github.com/varnishcache/varnish-cache/issues/1933 From lkarsten at varnish-software.com Tue Jun 28 15:59:07 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 28 Jun 2016 17:59:07 +0200 Subject: [4.1] 47861b9 Prepare for 4.1.3-beta2. Message-ID: commit 47861b9eb052ab547fce414cd9cfcf917d495419 Author: Lasse Karstensen Date: Tue Jun 28 17:57:43 2016 +0200 Prepare for 4.1.3-beta2. diff --git a/doc/changes.rst b/doc/changes.rst index 2d36211..f843df4 100644 --- a/doc/changes.rst +++ b/doc/changes.rst @@ -1,5 +1,5 @@ ====================================== -Varnish Cache 4.1.3-beta2 (unreleased) +Varnish Cache 4.1.3-beta2 (2016-06-28) ====================================== Changes since 4.1.2-beta1: From nils.goroll at uplex.de Tue Jun 28 18:37:07 2016 From: nils.goroll at uplex.de (Nils Goroll) Date: Tue, 28 Jun 2016 20:37:07 +0200 Subject: [master] ad42abd fix LostHeader message in http_PrintfHeader Message-ID: commit ad42abd533a0410be656defab542e110c1aa8338 Author: Nils Goroll Date: Tue Jun 28 20:27:39 2016 +0200 fix LostHeader message in http_PrintfHeader When out of workspace, we dumped the piece of memory at the ws free pointer, which does not necessarily contain anything sensible in this situation. diff --git a/bin/varnishd/cache/cache_http.c b/bin/varnishd/cache/cache_http.c index d6bfb28..53e4b1d 100644 --- a/bin/varnishd/cache/cache_http.c +++ b/bin/varnishd/cache/cache_http.c @@ -1149,7 +1149,8 @@ http_PrintfHeader(struct http *to, const char *fmt, ...) va_end(ap); if (n + 1 >= l || to->nhd >= to->shd) { http_fail(to); - VSLb(to->vsl, SLT_LostHeader, "%s", to->ws->f); + VSLb(to->vsl, SLT_LostHeader, "%s", + n + 1 >= l ? fmt : to->ws->f); WS_Release(to->ws, 0); return; } From varnish-commit at varnish-cache.org Wed Jun 29 12:58:07 2016 From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org) Date: 29 Jun 2016 20:58:07 +0800 Subject: beneficial offer Message-ID: <004e01d1d209$033ddb61$7c464dba$@varnish-cache.org> Hello! We are looking for employees working remotely. My name is Avery, am the personnel manager of a large International company. Most of the work you can do from home, that is, at a distance. Salary is $2500-$5000. If you are interested in this offer, please visit Our Site Best regards! -------------- next part -------------- An HTML attachment was scrubbed... URL: From phk at FreeBSD.org Thu Jun 30 08:58:09 2016 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Thu, 30 Jun 2016 10:58:09 +0200 Subject: [master] 0577f3f RFC7230/3.2.4 compliance. Message-ID: commit 0577f3fba200e45c05099427eec01610ee061436 Author: Poul-Henning Kamp Date: Thu Jun 30 08:56:22 2016 +0000 RFC7230/3.2.4 compliance. Spotted by: R?gis Leroy diff --git a/bin/varnishd/http1/cache_http1_proto.c b/bin/varnishd/http1/cache_http1_proto.c index f5dbe53..a6934e7 100644 --- a/bin/varnishd/http1/cache_http1_proto.c +++ b/bin/varnishd/http1/cache_http1_proto.c @@ -185,6 +185,18 @@ http1_dissect_hdrs(struct http *hp, char *p, struct http_conn *htc, (int)(q - p > 20 ? 20 : q - p), p); return (400); } + + for (; p < q; p++) { + if (vct_islws(*p)) { + VSLb(hp->vsl, SLT_BogoHeader, + "Space in header '%.*s'", + (int)Tlen(hp->hd[hp->nhd - 1]), + hp->hd[hp->nhd - 1].b); + return (400); + } + if (*p == ':') + break; + } } if (p < htc->rxbuf_e) p += vct_skipcrlf(p); diff --git a/bin/varnishtest/tests/b00049.vtc b/bin/varnishtest/tests/b00049.vtc new file mode 100644 index 0000000..7d2e289 --- /dev/null +++ b/bin/varnishtest/tests/b00049.vtc @@ -0,0 +1,32 @@ +varnishtest "RFC 7230 compliance" + +server s1 { + rxreq + txresp -gzipbody "FOOOOOOBAR" +} -start + +varnish v1 -vcl+backend { } -start + +client c1 { + send "GET / HTTP/1.1\r\n" + send "Host: foo\r\n" + send "\r\n" + rxresp + expect resp.status == 200 + expect resp.bodylen == 10 + + send "GET / HTTP/1.1\r\n" + send "Host: foo\r\n" + send "Accept-Encoding: gzip\r\n" + send "\r\n" + rxresp + expect resp.status == 200 + expect resp.bodylen == 33 + + send "GET / HTTP/1.1\r\n" + send "Host: foo\r\n" + send "Accept-Encoding : gzip\r\n" + send "\r\n" + rxresp + expect resp.status == 400 +} -run From lkarsten at varnish-software.com Thu Jun 30 12:28:07 2016 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Thu, 30 Jun 2016 14:28:07 +0200 Subject: [4.1] 7bf35e7 RFC7230/3.2.4 compliance. Message-ID: commit 7bf35e75c6a3aa142080d670b24a44b3f5887c60 Author: Poul-Henning Kamp Date: Thu Jun 30 08:56:22 2016 +0000 RFC7230/3.2.4 compliance. Spotted by: R?gis Leroy diff --git a/bin/varnishd/http1/cache_http1_proto.c b/bin/varnishd/http1/cache_http1_proto.c index ebf0d2a..e4c9053 100644 --- a/bin/varnishd/http1/cache_http1_proto.c +++ b/bin/varnishd/http1/cache_http1_proto.c @@ -189,6 +189,18 @@ http1_dissect_hdrs(struct http *hp, char *p, struct http_conn *htc) (int)(q - p > 20 ? 20 : q - p), p); return (400); } + + for (; p < q; p++) { + if (vct_islws(*p)) { + VSLb(hp->vsl, SLT_BogoHeader, + "Space in header '%.*s'", + (int)Tlen(hp->hd[hp->nhd - 1]), + hp->hd[hp->nhd - 1].b); + return (400); + } + if (*p == ':') + break; + } } if (p < htc->rxbuf_e) p += vct_skipcrlf(p); diff --git a/bin/varnishtest/tests/b00049.vtc b/bin/varnishtest/tests/b00049.vtc new file mode 100644 index 0000000..7d2e289 --- /dev/null +++ b/bin/varnishtest/tests/b00049.vtc @@ -0,0 +1,32 @@ +varnishtest "RFC 7230 compliance" + +server s1 { + rxreq + txresp -gzipbody "FOOOOOOBAR" +} -start + +varnish v1 -vcl+backend { } -start + +client c1 { + send "GET / HTTP/1.1\r\n" + send "Host: foo\r\n" + send "\r\n" + rxresp + expect resp.status == 200 + expect resp.bodylen == 10 + + send "GET / HTTP/1.1\r\n" + send "Host: foo\r\n" + send "Accept-Encoding: gzip\r\n" + send "\r\n" + rxresp + expect resp.status == 200 + expect resp.bodylen == 33 + + send "GET / HTTP/1.1\r\n" + send "Host: foo\r\n" + send "Accept-Encoding : gzip\r\n" + send "\r\n" + rxresp + expect resp.status == 400 +} -run From varnish-commit at varnish-cache.org Thu Jun 30 15:08:39 2016 From: varnish-commit at varnish-cache.org (varnish-commit at varnish-cache.org) Date: 30 Jun 2016 16:08:39 +0100 Subject: Flexible schedule Message-ID: <955407942.201606301636@varnish-cache.org> Hello! We are looking for employees working remotely. My name is Lynne, am the personnel manager of a large International company. Most of the work you can do from home, that is, at a distance. Salary is $2500-$5000. If you are interested in this offer, please visit Our Site Best regards! -------------- next part -------------- An HTML attachment was scrubbed... URL: