From martin at varnish-software.com Tue Apr 3 16:29:29 2012 From: martin at varnish-software.com (Martin Blix Grydeland) Date: Tue, 3 Apr 2012 18:29:29 +0200 Subject: [PATCH 1/3] Expose resp.body for inspection and testing in varnishtest Message-ID: <1333470571-31597-1-git-send-email-martin@varnish-software.com> Needed to test #1123 --- bin/varnishtest/vtc_http.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/bin/varnishtest/vtc_http.c b/bin/varnishtest/vtc_http.c index 00be0be..0678efd 100644 --- a/bin/varnishtest/vtc_http.c +++ b/bin/varnishtest/vtc_http.c @@ -190,6 +190,8 @@ cmd_var_resolve(struct http *hp, char *spec) return(hp->chunklen); if (!strcmp(spec, "resp.bodylen")) return(hp->bodylen); + if (!strcmp(spec, "resp.body")) + return(hp->body != NULL ? hp->body : spec); if (!memcmp(spec, "req.http.", 9)) { hh = hp->req; hdr = spec + 9; -- 1.7.4.1 From martin at varnish-software.com Tue Apr 3 16:29:30 2012 From: martin at varnish-software.com (Martin Blix Grydeland) Date: Tue, 3 Apr 2012 18:29:30 +0200 Subject: [PATCH 2/3] Reset output buffer on VGZ_WrwFlush() In-Reply-To: <1333470571-31597-1-git-send-email-martin@varnish-software.com> References: <1333470571-31597-1-git-send-email-martin@varnish-software.com> Message-ID: <1333470571-31597-2-git-send-email-martin@varnish-software.com> Fixes: #1123 --- bin/varnishd/cache/cache_gzip.c | 1 + bin/varnishtest/tests/r01123.vtc | 25 +++++++++++++++++++++++++ 2 files changed, 26 insertions(+), 0 deletions(-) create mode 100644 bin/varnishtest/tests/r01123.vtc diff --git a/bin/varnishd/cache/cache_gzip.c b/bin/varnishd/cache/cache_gzip.c index ca40903..f7dba8e 100644 --- a/bin/varnishd/cache/cache_gzip.c +++ b/bin/varnishd/cache/cache_gzip.c @@ -366,6 +366,7 @@ VGZ_WrwFlush(const struct worker *wrk, struct vgz *vg) (void)WRW_Write(wrk, vg->m_buf, vg->m_len); (void)WRW_Flush(wrk); vg->m_len = 0; + VGZ_Obuf(vg, vg->m_buf, vg->m_sz); } /*--------------------------------------------------------------------*/ diff --git a/bin/varnishtest/tests/r01123.vtc b/bin/varnishtest/tests/r01123.vtc new file mode 100644 index 0000000..37a9254 --- /dev/null +++ b/bin/varnishtest/tests/r01123.vtc @@ -0,0 +1,25 @@ +varnishtest "Test case for #1123 - gunzip buffer reset" + +server s1 { + rxreq + txresp -body {start end} + rxreq + expect req.url == "/included" + txresp -body {INCLUDED} +} -start + +varnish v1 -vcl+backend { + sub vcl_fetch { + if (req.url == "/") { + set beresp.do_esi = true; + } + set beresp.do_gzip = true; + } +} -start + +client c1 { + txreq + rxresp + expect resp.bodylen == 24 + expect resp.body == "start INCLUDED end" +} -run -- 1.7.4.1 From martin at varnish-software.com Tue Apr 3 16:29:31 2012 From: martin at varnish-software.com (Martin Blix Grydeland) Date: Tue, 3 Apr 2012 18:29:31 +0200 Subject: [PATCH 3/3] Fix ved_deliver_byterange() and ESI_DeliverChild() when ESI-included objects span storage chunks In-Reply-To: <1333470571-31597-1-git-send-email-martin@varnish-software.com> References: <1333470571-31597-1-git-send-email-martin@varnish-software.com> Message-ID: <1333470571-31597-3-git-send-email-martin@varnish-software.com> ved_deliver_byterange() would fail to return the correct next byte when the gzipped ESI-included object spans storage chunks. ESI_DeliverChild() would fail to extract the gzip trailer spanned storage chunks. Introduce a general STV_memcpy() function to copy bytes from objects. These bugs are present in both trunk and 3.0 Fixes: #1109 --- bin/varnishd/cache/cache.h | 2 + bin/varnishd/cache/cache_esi_deliver.c | 33 +++++++++++++----------- bin/varnishd/storage/stevedore.c | 37 ++++++++++++++++++++++++++ bin/varnishtest/tests/r01109.vtc | 44 ++++++++++++++++++++++++++++++++ 4 files changed, 101 insertions(+), 15 deletions(-) create mode 100644 bin/varnishtest/tests/r01109.vtc diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h index 073eb75..0548d1a 100644 --- a/bin/varnishd/cache/cache.h +++ b/bin/varnishd/cache/cache.h @@ -1020,6 +1020,8 @@ void STV_free(struct storage *st); void STV_open(void); void STV_close(void); void STV_Freestore(struct object *o); +void STV_Memcpy(const struct object *obj, unsigned char *dest, ssize_t start, + ssize_t n); /* storage_synth.c */ struct vsb *SMS_Makesynth(struct object *obj); diff --git a/bin/varnishd/cache/cache_esi_deliver.c b/bin/varnishd/cache/cache_esi_deliver.c index dd3d98a..53530c4 100644 --- a/bin/varnishd/cache/cache_esi_deliver.c +++ b/bin/varnishd/cache/cache_esi_deliver.c @@ -418,7 +418,7 @@ static uint8_t ved_deliver_byterange(const struct sess *sp, ssize_t low, ssize_t high) { struct storage *st; - ssize_t l, lx; + ssize_t l, l2, lx; u_char *p; //printf("BR %jd %jd\n", low, high); @@ -440,15 +440,18 @@ ved_deliver_byterange(const struct sess *sp, ssize_t low, ssize_t high) lx = low; } //printf("[1-] %jd %jd\n", lx, lx + l); - if (lx + l >= high) - l = high - lx; + l2 = l; + if (lx + l2 > high) + l2 = high - lx; //printf("[2-] %jd %jd\n", lx, lx + l); - assert(lx >= low && lx + l <= high); - if (l != 0) - (void)WRW_Write(sp->wrk, p, l); - if (lx + st->len > high) - return(p[l]); - lx += st->len; + assert(lx >= low && l2 <= l && lx + l2 <= high); + if (l2 != 0) + (void)WRW_Write(sp->wrk, p, l2); + if (l2 < l) { + assert(lx + l2 == high); + return(p[l2]); + } + lx += l; } INCOMPL(); } @@ -459,10 +462,11 @@ ESI_DeliverChild(const struct sess *sp) struct storage *st; struct object *obj; ssize_t start, last, stop, lpad; - u_char *p, cc; + u_char cc; uint32_t icrc; uint32_t ilen; uint8_t *dbits; + uint8_t buf[8]; if (!sp->req->obj->gziped) { VTAILQ_FOREACH(st, &sp->req->obj->store, list) @@ -542,12 +546,11 @@ ESI_DeliverChild(const struct sess *sp) } if (lpad > 0) (void)WRW_Write(sp->wrk, dbits + 1, lpad); - st = VTAILQ_LAST(&sp->req->obj->store, storagehead); - assert(st->len > 8); - p = st->ptr + st->len - 8; - icrc = vle32dec(p); - ilen = vle32dec(p + 4); + assert(sp->req->obj->len > 8); + STV_Memcpy(sp->req->obj, buf, sp->req->obj->len - 8, 8); + icrc = vle32dec(buf); + ilen = vle32dec(buf + 4); sp->req->crc = crc32_combine(sp->req->crc, icrc, ilen); sp->req->l_crc += ilen; } diff --git a/bin/varnishd/storage/stevedore.c b/bin/varnishd/storage/stevedore.c index f1cbe94..98466b7 100644 --- a/bin/varnishd/storage/stevedore.c +++ b/bin/varnishd/storage/stevedore.c @@ -381,6 +381,43 @@ STV_Freestore(struct object *o) } } +void +STV_Memcpy(const struct object *obj, unsigned char *dest, ssize_t start, + ssize_t n) +{ + struct storage *st; + ssize_t l, lx; + u_char *p; + + CHECK_OBJ_NOTNULL(obj, OBJECT_MAGIC); + AN(dest); + assert(n >= 0 && start + n <= obj->len); + + if (n == 0) + return; + + lx = 0; + VTAILQ_FOREACH(st, &obj->store, list) { + p = st->ptr; + l = st->len; + if (lx + l < start) { + lx += l; + continue; + } + if (lx < start) { + p += (start - lx); + l -= (start - lx); + lx = start; + } + if (lx + l > start + n) + l = start + n - lx; + assert(lx >= start && lx + l <= start + n); + memcpy(dest, p, l); + dest += l; + lx += l; + } +} + /*-------------------------------------------------------------------*/ struct storage * diff --git a/bin/varnishtest/tests/r01109.vtc b/bin/varnishtest/tests/r01109.vtc new file mode 100644 index 0000000..1a5d61d --- /dev/null +++ b/bin/varnishtest/tests/r01109.vtc @@ -0,0 +1,44 @@ +varnishtest "Test case for #1109 - Gzip+ESI broken for large included objects" + +server s1 { + rxreq + expect req.url == "/test1" + txresp -body {startend} + rxreq + expect req.url == "/include1" + # This tests ESI+gzip delivery when the ESI-included object + # has more than one storage chunk + txresp -bodylen 4082 + + rxreq + txresp -body {startend} + expect req.url == "/test2" + + rxreq + expect req.url == "/include2" + # This tests gzip trailer extraction for ESI+gzip CRC calculation + # when the trailer spans two storage chunks + txresp -bodylen 4074 +} -start + +varnish v1 -arg "-pfetch_chunksize=4k" -arg "-pgzip_level=0" -vcl+backend { + sub vcl_fetch { + if (req.url ~ "/test") { + set beresp.do_esi = true; + } + set beresp.do_gzip = true; + } +} -start + +client c1 { + txreq -url "/test1" -hdr "Accept-Encoding: gzip" + rxresp + gunzip + expect resp.bodylen == 4096 + + txreq -url "/test2" -hdr "Accept-Encoding: gzip" + rxresp + gunzip + expect resp.bodylen == 4088 +} -run + -- 1.7.4.1 From martin at varnish-software.com Wed Apr 4 09:31:55 2012 From: martin at varnish-software.com (Martin Blix Grydeland) Date: Wed, 4 Apr 2012 11:31:55 +0200 Subject: [PATCH] Fix ved_deliver_byterange() and ESI_DeliverChild() when ESI-included objects span storage chunks Message-ID: <1333531915-6340-1-git-send-email-martin@varnish-software.com> ved_deliver_byterange() would fail to return the correct next byte when the gzipped ESI-included object spans storage chunks. ESI_DeliverChild() would fail to extract the gzip trailer when it spans storage chunks. These bugs are present in both trunk and 3.0. Introduces a general STV_Memcpy() function to copy bytes from objects. Fixes: #1109 --- bin/varnishd/cache/cache.h | 2 + bin/varnishd/cache/cache_esi_deliver.c | 33 +++++++++++++----------- bin/varnishd/storage/stevedore.c | 41 +++++++++++++++++++++++++++++ bin/varnishtest/tests/r01109.vtc | 44 ++++++++++++++++++++++++++++++++ 4 files changed, 105 insertions(+), 15 deletions(-) create mode 100644 bin/varnishtest/tests/r01109.vtc diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h index 073eb75..0548d1a 100644 --- a/bin/varnishd/cache/cache.h +++ b/bin/varnishd/cache/cache.h @@ -1020,6 +1020,8 @@ void STV_free(struct storage *st); void STV_open(void); void STV_close(void); void STV_Freestore(struct object *o); +void STV_Memcpy(const struct object *obj, unsigned char *dest, ssize_t start, + ssize_t n); /* storage_synth.c */ struct vsb *SMS_Makesynth(struct object *obj); diff --git a/bin/varnishd/cache/cache_esi_deliver.c b/bin/varnishd/cache/cache_esi_deliver.c index dd3d98a..53530c4 100644 --- a/bin/varnishd/cache/cache_esi_deliver.c +++ b/bin/varnishd/cache/cache_esi_deliver.c @@ -418,7 +418,7 @@ static uint8_t ved_deliver_byterange(const struct sess *sp, ssize_t low, ssize_t high) { struct storage *st; - ssize_t l, lx; + ssize_t l, l2, lx; u_char *p; //printf("BR %jd %jd\n", low, high); @@ -440,15 +440,18 @@ ved_deliver_byterange(const struct sess *sp, ssize_t low, ssize_t high) lx = low; } //printf("[1-] %jd %jd\n", lx, lx + l); - if (lx + l >= high) - l = high - lx; + l2 = l; + if (lx + l2 > high) + l2 = high - lx; //printf("[2-] %jd %jd\n", lx, lx + l); - assert(lx >= low && lx + l <= high); - if (l != 0) - (void)WRW_Write(sp->wrk, p, l); - if (lx + st->len > high) - return(p[l]); - lx += st->len; + assert(lx >= low && l2 <= l && lx + l2 <= high); + if (l2 != 0) + (void)WRW_Write(sp->wrk, p, l2); + if (l2 < l) { + assert(lx + l2 == high); + return(p[l2]); + } + lx += l; } INCOMPL(); } @@ -459,10 +462,11 @@ ESI_DeliverChild(const struct sess *sp) struct storage *st; struct object *obj; ssize_t start, last, stop, lpad; - u_char *p, cc; + u_char cc; uint32_t icrc; uint32_t ilen; uint8_t *dbits; + uint8_t buf[8]; if (!sp->req->obj->gziped) { VTAILQ_FOREACH(st, &sp->req->obj->store, list) @@ -542,12 +546,11 @@ ESI_DeliverChild(const struct sess *sp) } if (lpad > 0) (void)WRW_Write(sp->wrk, dbits + 1, lpad); - st = VTAILQ_LAST(&sp->req->obj->store, storagehead); - assert(st->len > 8); - p = st->ptr + st->len - 8; - icrc = vle32dec(p); - ilen = vle32dec(p + 4); + assert(sp->req->obj->len > 8); + STV_Memcpy(sp->req->obj, buf, sp->req->obj->len - 8, 8); + icrc = vle32dec(buf); + ilen = vle32dec(buf + 4); sp->req->crc = crc32_combine(sp->req->crc, icrc, ilen); sp->req->l_crc += ilen; } diff --git a/bin/varnishd/storage/stevedore.c b/bin/varnishd/storage/stevedore.c index f1cbe94..c8d83ff 100644 --- a/bin/varnishd/storage/stevedore.c +++ b/bin/varnishd/storage/stevedore.c @@ -381,6 +381,47 @@ STV_Freestore(struct object *o) } } +void +STV_Memcpy(const struct object *obj, unsigned char *dest, ssize_t start, + ssize_t n) +{ + struct storage *st; + ssize_t l, lx; + u_char *p; + + CHECK_OBJ_NOTNULL(obj, OBJECT_MAGIC); + AN(dest); + assert(n >= 0 && start + n <= obj->len); + + if (n == 0) + return; + + lx = 0; + VTAILQ_FOREACH(st, &obj->store, list) { + p = st->ptr; + l = st->len; + if (lx + l < start) { + lx += l; + continue; + } + if (lx < start) { + p += (start - lx); + l -= (start - lx); + lx = start; + } + if (l > n) + l = n; + assert(lx == start && l <= n); + memcpy(dest, p, l); + dest += l; + lx += l; + start += l; + n -= l; + if (n == 0) + break; + } +} + /*-------------------------------------------------------------------*/ struct storage * diff --git a/bin/varnishtest/tests/r01109.vtc b/bin/varnishtest/tests/r01109.vtc new file mode 100644 index 0000000..1a5d61d --- /dev/null +++ b/bin/varnishtest/tests/r01109.vtc @@ -0,0 +1,44 @@ +varnishtest "Test case for #1109 - Gzip+ESI broken for large included objects" + +server s1 { + rxreq + expect req.url == "/test1" + txresp -body {startend} + rxreq + expect req.url == "/include1" + # This tests ESI+gzip delivery when the ESI-included object + # has more than one storage chunk + txresp -bodylen 4082 + + rxreq + txresp -body {startend} + expect req.url == "/test2" + + rxreq + expect req.url == "/include2" + # This tests gzip trailer extraction for ESI+gzip CRC calculation + # when the trailer spans two storage chunks + txresp -bodylen 4074 +} -start + +varnish v1 -arg "-pfetch_chunksize=4k" -arg "-pgzip_level=0" -vcl+backend { + sub vcl_fetch { + if (req.url ~ "/test") { + set beresp.do_esi = true; + } + set beresp.do_gzip = true; + } +} -start + +client c1 { + txreq -url "/test1" -hdr "Accept-Encoding: gzip" + rxresp + gunzip + expect resp.bodylen == 4096 + + txreq -url "/test2" -hdr "Accept-Encoding: gzip" + rxresp + gunzip + expect resp.bodylen == 4088 +} -run + -- 1.7.4.1 From qwe64 at inbox.lv Thu Apr 5 12:19:57 2012 From: qwe64 at inbox.lv (qwe 64) Date: Thu, 05 Apr 2012 15:19:57 +0300 Subject: a typo? Message-ID: <1333628397.4f7d8deda294d@mail.inbox.lv> An HTML attachment was scrubbed... URL: From mathieu.desnoyers at efficios.com Tue Apr 3 23:49:21 2012 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Tue, 3 Apr 2012 19:49:21 -0400 Subject: Scalability micro-conference topic proposals (LPC2012) Message-ID: <20120403234921.GB27752@Krystal> Hi, We are organizing a micro-conference on scaling both upwards (many cores) and downwards (low footprint, energy efficiency) that targets all layers of the software stack. Our intent is to bring together application, libraries and kernel developers to discuss the scalability issues they currently face, and get exposure for the ongoing work on scalability infrastructure. Suggestions of topics are welcome. If you would like to present, please let us know: we have lightnening-talk slots and a few 30 minutes slots available. Presentations should be oriented towards stimulating discussion over currently faced scalability problems and/or work in progress in the area of scalability. The micro-conference will be held between August 29-31, at LinuxCon North America 2012, in San Diego. http://www.linuxplumbersconf.org/2012/ The Scaling Micro-Conference page is available at: http://wiki.linuxplumbersconf.org/2012:scaling Best Regards, Mathieu Desnoyers & Paul E. McKenney -- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com From tfheen at varnish-software.com Tue Apr 10 08:52:37 2012 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Tue, 10 Apr 2012 10:52:37 +0200 Subject: [PATCH] add a least-connection director In-Reply-To: References: Message-ID: <20120410085237.GB27563@err.no> ]] Federico G. Schwindt Hi, > No, will check tonight. Was fine last time I checked. have you had any progress on this? I'd like to get this merged if possible. :-) Regards, -- Tollef Fog Heen Technical lead, Varnish Software t: +47 21 98 92 64 From tfheen at varnish-software.com Tue Apr 10 08:59:35 2012 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Tue, 10 Apr 2012 10:59:35 +0200 Subject: a typo? In-Reply-To: <1333628397.4f7d8deda294d@mail.inbox.lv> References: <1333628397.4f7d8deda294d@mail.inbox.lv> Message-ID: <20120410085935.GC27563@err.no> ]] qwe 64 > varnish-3.0.2 > > include/http_headers.h line 66: > HTTPH("Content-Langugae", H_Content_Language, 2, 0, 0, 0, 0) /* RFC2616 14.12 */ > is it a typo? Indeed, it is, thanks, fixed now. -- Tollef Fog Heen Technical lead, Varnish Software t: +47 21 98 92 64 From martin at varnish-software.com Tue Apr 10 14:37:47 2012 From: martin at varnish-software.com (Martin Blix Grydeland) Date: Tue, 10 Apr 2012 16:37:47 +0200 Subject: [PATCH] Reset req->err_code to zero when doing restart from vcl_fetch() Message-ID: <1334068667-21195-1-git-send-email-martin@varnish-software.com> By resetting the req->err_code upon restarting after vcl_fetch(), a proper 503 will be issued if this later triggers a max restarts error instead of whatever status code the aborted backend connection returned. Test case by: scoof Fixes: #1113 --- bin/varnishd/cache/cache_center.c | 1 + bin/varnishtest/tests/r01113.vtc | 21 +++++++++++++++++++++ 2 files changed, 22 insertions(+), 0 deletions(-) create mode 100644 bin/varnishtest/tests/r01113.vtc diff --git a/bin/varnishd/cache/cache_center.c b/bin/varnishd/cache/cache_center.c index 7104e0f..1fa8604 100644 --- a/bin/varnishd/cache/cache_center.c +++ b/bin/varnishd/cache/cache_center.c @@ -661,6 +661,7 @@ cnt_fetch(struct sess *sp, struct worker *wrk, struct req *req) switch (req->handling) { case VCL_RET_RESTART: req->restarts++; + req->err_code = 0; sp->step = STP_RECV; return (0); case VCL_RET_ERROR: diff --git a/bin/varnishtest/tests/r01113.vtc b/bin/varnishtest/tests/r01113.vtc new file mode 100644 index 0000000..efbfd55 --- /dev/null +++ b/bin/varnishtest/tests/r01113.vtc @@ -0,0 +1,21 @@ +varnishtest "HTTP status code when hitting max_restarts" + +server s1 { + rxreq + txresp + accept + rxreq + txresp +} -start + +varnish v1 -vcl+backend { + sub vcl_fetch { + return(restart); + } +} -start -cliok "param.set max_restarts 1" + +client c1 { + txreq + rxresp + expect resp.status == 503 +} -run -- 1.7.4.1 From martin at varnish-software.com Thu Apr 12 11:42:15 2012 From: martin at varnish-software.com (Martin Blix Grydeland) Date: Thu, 12 Apr 2012 13:42:15 +0200 Subject: [PATCH 1/2] Add an ERRCHK point in vcc_expr0() in the VCL compiler to catch unresolved symbols from vcc_expr4(). Message-ID: <1334230936-28445-1-git-send-email-martin@varnish-software.com> Fixes: #1125 --- bin/varnishtest/tests/r01125.vtc | 9 +++++++++ lib/libvcl/vcc_expr.c | 1 + 2 files changed, 10 insertions(+), 0 deletions(-) create mode 100644 bin/varnishtest/tests/r01125.vtc diff --git a/bin/varnishtest/tests/r01125.vtc b/bin/varnishtest/tests/r01125.vtc new file mode 100644 index 0000000..dda1171 --- /dev/null +++ b/bin/varnishtest/tests/r01125.vtc @@ -0,0 +1,9 @@ +varnishtest "#1125 unresolved first arg to regsub causes VCL compiler segfault" + +varnish v1 -badvcl { + backend b { .host = "127.0.0.1"; } + + sub vcl_recv { + set req.url = regsub(reg.url, "\?[^?]+$", ""); + } +} diff --git a/lib/libvcl/vcc_expr.c b/lib/libvcl/vcc_expr.c index 203d76f..a4ed74e 100644 --- a/lib/libvcl/vcc_expr.c +++ b/lib/libvcl/vcc_expr.c @@ -454,6 +454,7 @@ vcc_Eval_Regsub(struct vcc *tl, struct expr **e, const struct symbol *sym) SkipToken(tl, '('); vcc_expr0(tl, &e2, STRING); + ERRCHK(tl); if (e2->fmt != STRING) vcc_expr_tostring(&e2, STRING); -- 1.7.4.1 From martin at varnish-software.com Thu Apr 12 11:42:16 2012 From: martin at varnish-software.com (Martin Blix Grydeland) Date: Thu, 12 Apr 2012 13:42:16 +0200 Subject: [PATCH 2/2] Stop vcc_acl_try_netnotation() from trying to add a fifth octet to the parsed address. In-Reply-To: <1334230936-28445-1-git-send-email-martin@varnish-software.com> References: <1334230936-28445-1-git-send-email-martin@varnish-software.com> Message-ID: <1334230936-28445-2-git-send-email-martin@varnish-software.com> Fixes: #1126 --- bin/varnishtest/tests/r01126.vtc | 13 +++++++++++++ lib/libvcl/vcc_acl.c | 2 ++ 2 files changed, 15 insertions(+), 0 deletions(-) create mode 100644 bin/varnishtest/tests/r01126.vtc diff --git a/bin/varnishtest/tests/r01126.vtc b/bin/varnishtest/tests/r01126.vtc new file mode 100644 index 0000000..4f84bd3 --- /dev/null +++ b/bin/varnishtest/tests/r01126.vtc @@ -0,0 +1,13 @@ +varnishtest "#1126 - ACL with five octets causes VCL compiler to segfault" + +varnish v1 -badvcl { + backend b { .host = "127.0.0.1"; } + acl a { "127.0.0.0.1"; } + sub vcl_recv { if (client.ip ~ a) { return (pass); } } +} + +varnish v1 -badvcl { + backend b { .host = "127.0.0.1"; } + acl a { "127.0.0.0."; } + sub vcl_recv { if (client.ip ~ a) { return (pass); } } +} diff --git a/lib/libvcl/vcc_acl.c b/lib/libvcl/vcc_acl.c index e6a1065..541b91d 100644 --- a/lib/libvcl/vcc_acl.c +++ b/lib/libvcl/vcc_acl.c @@ -263,6 +263,8 @@ vcc_acl_try_netnotation(struct vcc *tl, struct acl_e *ae) return (0); p += k + 1; } + if (i > 3) + return (0); if (ae->t_mask == NULL) ae->mask = 8 + 8 * i; vcc_acl_add_entry(tl, ae, 4, b, AF_INET); -- 1.7.4.1 From martin at varnish-software.com Thu Apr 12 16:11:18 2012 From: martin at varnish-software.com (Martin Blix Grydeland) Date: Thu, 12 Apr 2012 18:11:18 +0200 Subject: [PATCH] Differentiate between CLIS_PARAM(106) and CLIS_INTERROR(503) for parse/compile errors and critical errors. Message-ID: <1334247078-13996-1-git-send-email-martin@varnish-software.com> This to enable regression testing on e.g. VCC compiler segfaults, as these previously would return CLIS_PARAM(106) (the same as any bad vcl), thus not really tripping the test case. --- bin/varnishd/mgt/mgt_vcc.c | 61 ++++++++++++++++++++++++++------------------ include/vcli.h | 3 +- include/vsub.h | 2 +- lib/libvarnish/vsub.c | 15 ++++++++--- 4 files changed, 50 insertions(+), 31 deletions(-) diff --git a/bin/varnishd/mgt/mgt_vcc.c b/bin/varnishd/mgt/mgt_vcc.c index 12433ad..a8981b3 100644 --- a/bin/varnishd/mgt/mgt_vcc.c +++ b/bin/varnishd/mgt/mgt_vcc.c @@ -215,25 +215,31 @@ run_dlopen(void *priv) } /*-------------------------------------------------------------------- - * Compile a VCL program, return shared object, errors in sb. + * Compile a VCL program, shared object in pof, errors in sb. + * Returns: + * -2: Critical error + * -1: Parse/compile error + * 0: Success (file name in 'pof') */ -static char * -mgt_run_cc(const char *vcl, struct vsb *sb, int C_flag) +static int +mgt_run_cc(const char *vcl, struct vsb *sb, int C_flag, char **pof) { char *csrc; struct vsb *cmdsb; char sf[] = "./vcl.########.c"; char of[sizeof sf + 1]; - char *retval; int sfd, i; struct vcc_priv vp; + AN(pof); + *pof = NULL; + /* Create temporary C source file */ sfd = VFIL_tmpfile(sf); if (sfd < 0) { VSB_printf(sb, "Failed to create %s: %s", sf, strerror(errno)); - return (NULL); + return (-2); } AZ(close(sfd)); @@ -242,9 +248,9 @@ mgt_run_cc(const char *vcl, struct vsb *sb, int C_flag) vp.magic = VCC_PRIV_MAGIC; vp.sf = sf; vp.vcl = vcl; - if (VSUB_run(sb, run_vcc, &vp, "VCC-compiler", -1)) { + if ((i = VSUB_run(sb, run_vcc, &vp, "VCC-compiler", -1))) { (void)unlink(sf); - return (NULL); + return (i); } if (C_flag) { @@ -276,33 +282,35 @@ mgt_run_cc(const char *vcl, struct vsb *sb, int C_flag) /* Ensure the file is readable to the unprivileged user */ if (!i) { i = chmod(of, 0755); - if (i) + if (i) { VSB_printf(sb, "Failed to set permissions on %s: %s", of, strerror(errno)); + i = -2; + } } if (i) { (void)unlink(of); - return (NULL); + return (i); } - retval = strdup(of); - XXXAN(retval); - return (retval); + *pof = strdup(of); + XXXAN(*pof); + return (i); } /*--------------------------------------------------------------------*/ -static char * -mgt_VccCompile(struct vsb **sb, const char *b, int C_flag) +static int +mgt_VccCompile(struct vsb **sb, const char *b, int C_flag, char **pvf) { - char *vf; + int i; *sb = VSB_new_auto(); XXXAN(*sb); - vf = mgt_run_cc(b, *sb, C_flag); + i = mgt_run_cc(b, *sb, C_flag, pvf); AZ(VSB_finish(*sb)); - return (vf); + return (i); } /*--------------------------------------------------------------------*/ @@ -358,7 +366,6 @@ mgt_vcc_delbyname(const char *name) } /*--------------------------------------------------------------------*/ - int mgt_vcc_default(const char *b_arg, const char *f_arg, char *vcl, int C_flag) { @@ -389,7 +396,7 @@ mgt_vcc_default(const char *b_arg, const char *f_arg, char *vcl, int C_flag) } strcpy(buf, "boot"); - vf = mgt_VccCompile(&sb, vcl, C_flag); + (void)mgt_VccCompile(&sb, vcl, C_flag, &vf); free(vcl); if (VSB_len(sb) > 0) fprintf(stderr, "%s", VSB_data(sb)); @@ -481,6 +488,7 @@ mcf_config_inline(struct cli *cli, const char * const *av, void *priv) struct vsb *sb; unsigned status; struct vclprog *vp; + int i; (void)priv; @@ -491,13 +499,14 @@ mcf_config_inline(struct cli *cli, const char * const *av, void *priv) return; } - vf = mgt_VccCompile(&sb, av[3], 0); + i = mgt_VccCompile(&sb, av[3], 0, &vf); if (VSB_len(sb) > 0) VCLI_Out(cli, "%s\n", VSB_data(sb)); VSB_delete(sb); - if (vf == NULL) { + if (i) { + AZ(vf); VCLI_Out(cli, "VCL compilation failed"); - VCLI_SetResult(cli, CLIS_PARAM); + VCLI_SetResult(cli, (i < -1 ? CLIS_INTERROR : CLIS_PARAM)); return; } VCLI_Out(cli, "VCL compiled."); @@ -519,6 +528,7 @@ mcf_config_load(struct cli *cli, const char * const *av, void *priv) unsigned status; char *p = NULL; struct vclprog *vp; + int i; (void)priv; vp = mgt_vcc_byname(av[2]); @@ -535,15 +545,16 @@ mcf_config_load(struct cli *cli, const char * const *av, void *priv) return; } - vf = mgt_VccCompile(&sb, vcl, 0); + i = mgt_VccCompile(&sb, vcl, 0, &vf); free(vcl); if (VSB_len(sb) > 0) VCLI_Out(cli, "%s", VSB_data(sb)); VSB_delete(sb); - if (vf == NULL) { + if (i) { + AZ(vf); VCLI_Out(cli, "VCL compilation failed"); - VCLI_SetResult(cli, CLIS_PARAM); + VCLI_SetResult(cli, (i < -1 ? CLIS_INTERROR : CLIS_PARAM)); return; } VCLI_Out(cli, "VCL compiled."); diff --git a/include/vcli.h b/include/vcli.h index 04ac11c..21bfa26 100644 --- a/include/vcli.h +++ b/include/vcli.h @@ -202,7 +202,8 @@ enum VCLI_status_e { CLIS_TRUNCATED = 201, CLIS_CANT = 300, CLIS_COMMS = 400, - CLIS_CLOSE = 500 + CLIS_CLOSE = 500, + CLIS_INTERROR = 503 }; /* Length of first line of response */ diff --git a/include/vsub.h b/include/vsub.h index 6705b11..ae89be5 100644 --- a/include/vsub.h +++ b/include/vsub.h @@ -28,7 +28,7 @@ * */ -/* from libvarnish/subproc.c */ +/* from libvarnish/vsub.c */ typedef void vsub_func_f(void*); int VSUB_run(struct vsb *sb, vsub_func_f *func, void *priv, const char *name, diff --git a/lib/libvarnish/vsub.c b/lib/libvarnish/vsub.c index 497119c..3550259 100644 --- a/lib/libvarnish/vsub.c +++ b/lib/libvarnish/vsub.c @@ -64,6 +64,13 @@ vsub_vlu(void *priv, const char *str) return (0); } +/* Fork and execute a function + Returns: + -2: Critical error (syscall failure or exit on signal) + -1: Non-zero return value + 0: Success (zero exit value) +*/ + int VSUB_run(struct vsb *sb, vsub_func_f *func, void *priv, const char *name, int maxlines) @@ -81,7 +88,7 @@ VSUB_run(struct vsb *sb, vsub_func_f *func, void *priv, const char *name, if (pipe(p) < 0) { VSB_printf(sb, "Starting %s: pipe() failed: %s", name, strerror(errno)); - return (-1); + return (-2); } assert(p[0] > STDERR_FILENO); assert(p[1] > STDERR_FILENO); @@ -90,7 +97,7 @@ VSUB_run(struct vsb *sb, vsub_func_f *func, void *priv, const char *name, name, strerror(errno)); AZ(close(p[0])); AZ(close(p[1])); - return (-1); + return (-2); } if (pid == 0) { AZ(close(STDIN_FILENO)); @@ -117,7 +124,7 @@ VSUB_run(struct vsb *sb, vsub_func_f *func, void *priv, const char *name, if (rv < 0 && errno != EINTR) { VSB_printf(sb, "Running %s: waitpid() failed: %s\n", name, strerror(errno)); - return (-1); + return (-2); } } while (rv < 0); if (!WIFEXITED(status) || WEXITSTATUS(status) != 0) { @@ -129,7 +136,7 @@ VSUB_run(struct vsb *sb, vsub_func_f *func, void *priv, const char *name, if (WCOREDUMP(status)) VSB_printf(sb, ", core dumped"); VSB_printf(sb, "\n"); - return (-1); + return (WIFSIGNALED(status) ? -2 : -1); } return (0); } -- 1.7.4.1 From Gauthier.Delacroix at coreye.fr Fri Apr 13 16:46:02 2012 From: Gauthier.Delacroix at coreye.fr (Delacroix, Gauthier) Date: Fri, 13 Apr 2012 18:46:02 +0200 Subject: Minimum version of curl for libvmod-curl Message-ID: <8349678DF4CCC3428201CED9DF7B4D9C11EF66D7@richter.lild01.pictime.fr> Hi ! I have some errors when building libvmod-curl : vmod_curl.c:200: error: 'CURLOPT_TIMEOUT_MS' undeclared (first use in this function) vmod_curl.c:200: error: (Each undeclared identifier is reported only once vmod_curl.c:200: error: for each function it appears in.) vmod_curl.c:203: error: 'CURLOPT_CONNECTTIMEOUT_MS' undeclared (first use in this function) It seems that my curl version doesn't provide required headers. Which is the minimum version to use ? (I have 7.15.5 on CentOS 5) Thanks ! PS : I was about to post this as an issue on github libvmod-curl page, but since there is no issue yet, I suppose best is to post here. Gauthier From martin at varnish-software.com Thu Apr 19 14:01:39 2012 From: martin at varnish-software.com (Martin Blix Grydeland) Date: Thu, 19 Apr 2012 16:01:39 +0200 Subject: [PATCH 1/2] Change the TryLock-or-fail behavior of the exp_timer() to TryLock-or-tryharder. Message-ID: <1334844100-15138-1-git-send-email-martin@varnish-software.com> On very busy systems, the exp_timer can start to lag behind on it's work if it frequently fails to acquire the LRU lock out of locking order. Change this to go the long route when the TryLock fails, and punt only if the head of the heap has changed while releasing the locks. --- bin/varnishd/cache/cache_expire.c | 18 ++++++++++++------ 1 files changed, 12 insertions(+), 6 deletions(-) diff --git a/bin/varnishd/cache/cache_expire.c b/bin/varnishd/cache/cache_expire.c index 39ded77..edbd52c 100644 --- a/bin/varnishd/cache/cache_expire.c +++ b/bin/varnishd/cache/cache_expire.c @@ -371,18 +371,24 @@ exp_timer(struct worker *wrk, void *priv) /* * It's time... - * Technically we should drop the exp_mtx, get the lru->mtx - * get the exp_mtx again and then check that the oc is still - * on the binheap. We take the shorter route and try to - * get the lru->mtx and punt if we fail. + * Try to acquire the lru->mtx. If fail, take the long route + * of releasing the lru->mtx and then getting exp_mtx and + * lru->mtx in the correct order, checking that the oc is + * still head of the heap. */ lru = oc_getlru(oc); CHECK_OBJ_NOTNULL(lru, LRU_MAGIC); if (Lck_Trylock(&lru->mtx)) { Lck_Unlock(&exp_mtx); - oc = NULL; - continue; + + Lck_Lock(&lru->mtx); + Lck_Lock(&exp_mtx); + if (oc != binheap_root(exp_heap)) { + Lck_Unlock(&exp_mtx); + Lck_Unlock(&lru->mtx); + continue; + } } /* Remove from binheap */ -- 1.7.4.1 From martin at varnish-software.com Thu Apr 19 14:01:40 2012 From: martin at varnish-software.com (Martin Blix Grydeland) Date: Thu, 19 Apr 2012 16:01:40 +0200 Subject: [PATCH 2/2] Add a runtime parameter for expiry_log_threshold. In-Reply-To: <1334844100-15138-1-git-send-email-martin@varnish-software.com> References: <1334844100-15138-1-git-send-email-martin@varnish-software.com> Message-ID: <1334844100-15138-2-git-send-email-martin@varnish-software.com> On busy servers, the logging and especially the object counters will not reflect the work done by the expiry timer is very busy and does not sleep (and flush the counters and log). This patch adds a configurable maximum number of objects expired before updating the counters and flushing the logs. --- bin/varnishd/cache/cache_expire.c | 7 ++++++- bin/varnishd/common/params.h | 1 + bin/varnishd/mgt/mgt_param.c | 6 ++++++ 3 files changed, 13 insertions(+), 1 deletions(-) diff --git a/bin/varnishd/cache/cache_expire.c b/bin/varnishd/cache/cache_expire.c index edbd52c..dbf21aa 100644 --- a/bin/varnishd/cache/cache_expire.c +++ b/bin/varnishd/cache/cache_expire.c @@ -336,15 +336,19 @@ exp_timer(struct worker *wrk, void *priv) double t; struct object *o; struct vsl_log vsl; + unsigned i = 0; (void)priv; VSL_Setup(&vsl, NULL, 0); t = VTIM_real(); oc = NULL; while (1) { - if (oc == NULL) { + if (oc == NULL || i >= cache_param->expiry_log_threshold) { VSL_Flush(&vsl, 0); WRK_SumStat(wrk); + i = 0; + } + if (oc == NULL) { VTIM_sleep(cache_param->expiry_sleep); t = VTIM_real(); } @@ -404,6 +408,7 @@ exp_timer(struct worker *wrk, void *priv) Lck_Unlock(&lru->mtx); VSC_C_main->n_expired++; + i++; CHECK_OBJ_NOTNULL(oc->objhead, OBJHEAD_MAGIC); o = oc_getobj(&wrk->stats, oc); diff --git a/bin/varnishd/common/params.h b/bin/varnishd/common/params.h index 6e502fd..b30a8b4 100644 --- a/bin/varnishd/common/params.h +++ b/bin/varnishd/common/params.h @@ -163,6 +163,7 @@ struct params { /* Expiry pacer parameters */ double expiry_sleep; + unsigned expiry_log_threshold; /* Acceptor pacer parameters */ double acceptor_sleep_max; diff --git a/bin/varnishd/mgt/mgt_param.c b/bin/varnishd/mgt/mgt_param.c index f6493ad..ae6d6d0 100644 --- a/bin/varnishd/mgt/mgt_param.c +++ b/bin/varnishd/mgt/mgt_param.c @@ -820,6 +820,12 @@ static const struct parspec input_parspec[] = { "for it to do.\n", 0, "1", "seconds" }, + { "expiry_log_threshold", tweak_uint, &mgt_param.expiry_log_threshold, + 1, UINT_MAX, + "Maximum number of objects expired without the expiry thread" + " sleeping before flushing the log.\n", + 0, + "100", "objects" }, { "pipe_timeout", tweak_timeout, &mgt_param.pipe_timeout, 0, 0, "Idle timeout for PIPE sessions. " "If nothing have been received in either direction for " -- 1.7.4.1 From martin at varnish-software.com Mon Apr 23 08:02:23 2012 From: martin at varnish-software.com (Martin Blix Grydeland) Date: Mon, 23 Apr 2012 10:02:23 +0200 Subject: [PATCH 1/2] Change the TryLock-or-fail behavior of the exp_timer() to TryLock-or-tryharder, and expire up to 10 objects at a time when they are on the same LRU. Message-ID: <1335168144-1195-1-git-send-email-martin@varnish-software.com> On very busy systems, the exp_timer can start to lag behind on it's work if it frequently fails to acquire the LRU lock out of locking order. Change this to go the long route when the TryLock fails, and punt only if the head of the heap has changed while releasing the locks. Also make the expiry thread scoop up to 10 other expired objects if they are on the same LRU. This should decrease the locking done by the expiry thread. --- bin/varnishd/cache/cache_expire.c | 65 +++++++++++++++++++++++++------------ 1 files changed, 44 insertions(+), 21 deletions(-) diff --git a/bin/varnishd/cache/cache_expire.c b/bin/varnishd/cache/cache_expire.c index 39ded77..5272f8c 100644 --- a/bin/varnishd/cache/cache_expire.c +++ b/bin/varnishd/cache/cache_expire.c @@ -328,10 +328,16 @@ EXP_Rearm(const struct object *o) * object expires, accounting also for graceability, it is killed. */ +#ifndef EXP_TIMER_N +#define EXP_TIMER_N 10 +#endif + static void * __match_proto__(bgthread_t) exp_timer(struct worker *wrk, void *priv) { struct objcore *oc; + struct objcore *oc_array[EXP_TIMER_N]; + int i, n; struct lru *lru; double t; struct object *o; @@ -371,39 +377,56 @@ exp_timer(struct worker *wrk, void *priv) /* * It's time... - * Technically we should drop the exp_mtx, get the lru->mtx - * get the exp_mtx again and then check that the oc is still - * on the binheap. We take the shorter route and try to - * get the lru->mtx and punt if we fail. + * Try to acquire the lru->mtx. If fail, take the long route + * of releasing the lru->mtx and then getting exp_mtx and + * lru->mtx in the correct order. */ lru = oc_getlru(oc); CHECK_OBJ_NOTNULL(lru, LRU_MAGIC); if (Lck_Trylock(&lru->mtx)) { Lck_Unlock(&exp_mtx); - oc = NULL; - continue; - } - /* Remove from binheap */ - assert(oc->timer_idx != BINHEAP_NOIDX); - binheap_delete(exp_heap, oc->timer_idx); - assert(oc->timer_idx == BINHEAP_NOIDX); + Lck_Lock(&lru->mtx); + Lck_Lock(&exp_mtx); + } - /* And from LRU */ - lru = oc_getlru(oc); - VTAILQ_REMOVE(&lru->lru_head, oc, lru_list); + /* Now that we have the locks, scoop up all expired + * objects while on the same LRU */ + n = 0; + while (n < EXP_TIMER_N) { + oc = binheap_root(exp_heap); + CHECK_OBJ_ORNULL(oc, OBJCORE_MAGIC); + if (oc == NULL || oc_getlru(oc) != lru) + break; + if (oc->timer_when > t) { + oc = NULL; + break; + } + assert(oc->timer_idx != BINHEAP_NOIDX); + + /* Remove from binheap and LRU */ + binheap_delete(exp_heap, oc->timer_idx); + VTAILQ_REMOVE(&lru->lru_head, oc, lru_list); + + oc_array[n++] = oc; + } Lck_Unlock(&exp_mtx); Lck_Unlock(&lru->mtx); - VSC_C_main->n_expired++; - - CHECK_OBJ_NOTNULL(oc->objhead, OBJHEAD_MAGIC); - o = oc_getobj(&wrk->stats, oc); - VSLb(&vsl, SLT_ExpKill, "%u %.0f", - oc_getxid(&wrk->stats, oc), EXP_Ttl(NULL, o) - t); - (void)HSH_Deref(&wrk->stats, oc, NULL); + /* Deref the OCs we have expired */ + for (i = 0; i < n; i++) { + oc = oc_array[i]; + CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC); + CHECK_OBJ_NOTNULL(oc->objhead, OBJHEAD_MAGIC); + assert(oc->timer_idx == BINHEAP_NOIDX); + o = oc_getobj(&wrk->stats, oc); + VSLb(&vsl, SLT_ExpKill, "%u %.0f", + oc_getxid(&wrk->stats, oc), EXP_Ttl(NULL, o) - t); + (void)HSH_Deref(&wrk->stats, oc, NULL); + } + VSC_C_main->n_expired += n; } NEEDLESS_RETURN(NULL); } -- 1.7.4.1 From martin at varnish-software.com Mon Apr 23 08:02:24 2012 From: martin at varnish-software.com (Martin Blix Grydeland) Date: Mon, 23 Apr 2012 10:02:24 +0200 Subject: [PATCH 2/2] Make the exp_timer flush the log and counters every 0.5 seconds. In-Reply-To: <1335168144-1195-1-git-send-email-martin@varnish-software.com> References: <1335168144-1195-1-git-send-email-martin@varnish-software.com> Message-ID: <1335168144-1195-2-git-send-email-martin@varnish-software.com> --- bin/varnishd/cache/cache_expire.c | 10 +++++++--- 1 files changed, 7 insertions(+), 3 deletions(-) diff --git a/bin/varnishd/cache/cache_expire.c b/bin/varnishd/cache/cache_expire.c index 5272f8c..9e7c7d3 100644 --- a/bin/varnishd/cache/cache_expire.c +++ b/bin/varnishd/cache/cache_expire.c @@ -339,18 +339,22 @@ exp_timer(struct worker *wrk, void *priv) struct objcore *oc_array[EXP_TIMER_N]; int i, n; struct lru *lru; - double t; + double t, tlog; struct object *o; struct vsl_log vsl; (void)priv; VSL_Setup(&vsl, NULL, 0); t = VTIM_real(); + tlog = 0; oc = NULL; while (1) { - if (oc == NULL) { + if (t - tlog < 0.5) { VSL_Flush(&vsl, 0); WRK_SumStat(wrk); + tlog = t; + } + if (oc == NULL) { VTIM_sleep(cache_param->expiry_sleep); t = VTIM_real(); } @@ -367,7 +371,7 @@ exp_timer(struct worker *wrk, void *priv) * We may have expired so many objects that our timestamp * got out of date, refresh it and check again. */ - if (oc->timer_when > t) + if (oc->timer_when > t || t - oc->timer_when > 0.5) t = VTIM_real(); if (oc->timer_when > t) { Lck_Unlock(&exp_mtx); -- 1.7.4.1 From brad at comstyle.com Thu Apr 26 07:01:28 2012 From: brad at comstyle.com (Brad Smith) Date: Thu, 26 Apr 2012 03:01:28 -0400 Subject: [PATCH] Fix libedit detection on *BSD OS's In-Reply-To: <4F98E794.4030902@comstyle.com> References: <20120102190345.GA24654@rox.home.comstyle.com> <4F98E794.4030902@comstyle.com> Message-ID: <20120426070128.GA26996@rox.home.comstyle.com> On Thu, Apr 26, 2012 at 02:13:40AM -0400, Brad Smith wrote: > On 02/01/12 2:03 PM, Brad wrote: > >The following patch allows the autoconf script to detect the presence of > >libedit when there isn't a pkg-config file present and thus allowing > >Varnish to detect libedit on OpenBSD/FreeBSD/NetBSD/DragonFly. > > > >diff --git a/bin/varnishadm/varnishadm.c b/bin/varnishadm/varnishadm.c > >index bb5cc8e..90532a8 100644 > >--- a/bin/varnishadm/varnishadm.c > >+++ b/bin/varnishadm/varnishadm.c > >@@ -33,8 +33,12 @@ > > > > #ifdef HAVE_LIBEDIT > > #include > >+#ifdef HAVE_READLINE_READLINE_H > >+#include > >+#else > > #include > > #endif > >+#endif > > > > #include > > #include > > This part of the diff was not commited and the tree is broken > at the moment. Here is an updated diff to fix the build on OpenBSD/NetBSD/DragonFly. diff --git a/bin/varnishadm/varnishadm.c b/bin/varnishadm/varnishadm.c index f3b8b69..aff9336 100644 --- a/bin/varnishadm/varnishadm.c +++ b/bin/varnishadm/varnishadm.c @@ -35,6 +35,8 @@ # include # ifdef HAVE_EDIT_READLINE_READLINE_H # include +# elif HAVE_READLINE_READLINE_H +# include # else # include # endif -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From phk at phk.freebsd.dk Thu Apr 26 16:10:57 2012 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Thu, 26 Apr 2012 16:10:57 +0000 Subject: [PATCH] Fix libedit detection on *BSD OS's In-Reply-To: Your message of "Thu, 26 Apr 2012 03:01:28 -0400." <20120426070128.GA26996@rox.home.comstyle.com> Message-ID: <2891.1335456657@critter.freebsd.dk> In message <20120426070128.GA26996 at rox.home.comstyle.com>, Brad Smith writes: Thanks, committed. >On Thu, Apr 26, 2012 at 02:13:40AM -0400, Brad Smith wrote: >> On 02/01/12 2:03 PM, Brad wrote: >> >The following patch allows the autoconf script to detect the presence of >> >libedit when there isn't a pkg-config file present and thus allowing >> >Varnish to detect libedit on OpenBSD/FreeBSD/NetBSD/DragonFly. >> > >> >diff --git a/bin/varnishadm/varnishadm.c b/bin/varnishadm/varnishadm.c >> >index bb5cc8e..90532a8 100644 >> >--- a/bin/varnishadm/varnishadm.c >> >+++ b/bin/varnishadm/varnishadm.c >> >@@ -33,8 +33,12 @@ >> > >> > #ifdef HAVE_LIBEDIT >> > #include >> >+#ifdef HAVE_READLINE_READLINE_H >> >+#include >> >+#else >> > #include >> > #endif >> >+#endif >> > >> > #include >> > #include >> >> This part of the diff was not commited and the tree is broken >> at the moment. > >Here is an updated diff to fix the build on OpenBSD/NetBSD/DragonFly. > > >diff --git a/bin/varnishadm/varnishadm.c b/bin/varnishadm/varnishadm.c >index f3b8b69..aff9336 100644 >--- a/bin/varnishadm/varnishadm.c >+++ b/bin/varnishadm/varnishadm.c >@@ -35,6 +35,8 @@ > # include > # ifdef HAVE_EDIT_READLINE_READLINE_H > # include >+# elif HAVE_READLINE_READLINE_H >+# include > # else > # include > # endif > >-- >This message has been scanned for viruses and >dangerous content by MailScanner, and is >believed to be clean. > > >_______________________________________________ >varnish-dev mailing list >varnish-dev at varnish-cache.org >https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev > -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From brad at comstyle.com Thu Apr 26 18:38:16 2012 From: brad at comstyle.com (Brad Smith) Date: Thu, 26 Apr 2012 14:38:16 -0400 Subject: [PATCH] Fix libedit detection on *BSD OS's In-Reply-To: <2891.1335456657@critter.freebsd.dk> References: <2891.1335456657@critter.freebsd.dk> Message-ID: <4F999618.9060708@comstyle.com> On 26/04/12 12:10 PM, Poul-Henning Kamp wrote: > In message<20120426070128.GA26996 at rox.home.comstyle.com>, Brad Smith writes: > > Thanks, committed. Thank you. -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From brad at comstyle.com Thu Apr 26 18:57:23 2012 From: brad at comstyle.com (Brad Smith) Date: Thu, 26 Apr 2012 14:57:23 -0400 Subject: [PATCH] Fix Varnish build on OpenBSD Message-ID: <20120426185723.GA9552@rox.home.comstyle.com> Unlike the 3.0.2 release checking out the latest and greatest from Git doesn't fare well building on OpenBSD. The following diff fixes the build issues. diff --git a/bin/varnishadm/varnishadm.c b/bin/varnishadm/varnishadm.c index aff9336..f4c262c 100644 --- a/bin/varnishadm/varnishadm.c +++ b/bin/varnishadm/varnishadm.c @@ -29,6 +29,7 @@ #include "config.h" +#include #include #ifdef HAVE_LIBEDIT diff --git a/bin/varnishd/cache/cache_dir_dns.c b/bin/varnishd/cache/cache_dir_dns.c index 9cad3f6..ee411dc 100644 --- a/bin/varnishd/cache/cache_dir_dns.c +++ b/bin/varnishd/cache/cache_dir_dns.c @@ -29,6 +29,8 @@ #include "config.h" +#include +#include #include #include diff --git a/bin/varnishd/cache/cache_vrt.c b/bin/varnishd/cache/cache_vrt.c index cbe5637..10658cd 100644 --- a/bin/varnishd/cache/cache_vrt.c +++ b/bin/varnishd/cache/cache_vrt.c @@ -31,6 +31,8 @@ #include "config.h" +#include +#include #include #include diff --git a/bin/varnishd/mgt/mgt_cli.c b/bin/varnishd/mgt/mgt_cli.c index cbea81f..3b0c1a7 100644 --- a/bin/varnishd/mgt/mgt_cli.c +++ b/bin/varnishd/mgt/mgt_cli.c @@ -31,6 +31,7 @@ #include "config.h" +#include #include #include diff --git a/bin/varnishd/storage/stevedore_utils.c b/bin/varnishd/storage/stevedore_utils.c index 89c6027..2364289 100644 --- a/bin/varnishd/storage/stevedore_utils.c +++ b/bin/varnishd/storage/stevedore_utils.c @@ -34,6 +34,7 @@ #include #include #ifdef HAVE_SYS_MOUNT_H +# include # include #endif #ifdef HAVE_SYS_STATVFS_H diff --git a/lib/libvarnish/vsha256.c b/lib/libvarnish/vsha256.c index 10b7a6a..0a521b2 100644 --- a/lib/libvarnish/vsha256.c +++ b/lib/libvarnish/vsha256.c @@ -29,6 +29,7 @@ #include "config.h" #ifdef HAVE_SYS_ENDIAN_H +#include #include #define VBYTE_ORDER _BYTE_ORDER #define VBIG_ENDIAN _BIG_ENDIAN diff --git a/lib/libvmod_std/vmod_std.c b/lib/libvmod_std/vmod_std.c index d1f6771..5b5c0aa 100644 --- a/lib/libvmod_std/vmod_std.c +++ b/lib/libvmod_std/vmod_std.c @@ -28,6 +28,8 @@ #include "config.h" +#include +#include #include #include -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From martin at varnish-software.com Fri Apr 27 12:46:14 2012 From: martin at varnish-software.com (Martin Blix Grydeland) Date: Fri, 27 Apr 2012 14:46:14 +0200 Subject: Improving persistence Message-ID: Hi, I've started on a project to improve the persistence storage backend in Varnish. I've created a wiki page at https://www.varnish-cache.org/trac/wiki/ImprovedPersistence with my initial ideas and implementation plan. All feedback most welcome (and won't be answered immediately as I leave for one week vacation tomorrow :) Regards, Martin -- Martin Blix Grydeland Varnish Software AS -------------- next part -------------- An HTML attachment was scrubbed... URL: From phk at phk.freebsd.dk Fri Apr 27 13:47:25 2012 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Fri, 27 Apr 2012 13:47:25 +0000 Subject: [PATCH] Fix Varnish build on OpenBSD In-Reply-To: Your message of "Thu, 26 Apr 2012 14:57:23 -0400." <20120426185723.GA9552@rox.home.comstyle.com> Message-ID: <30423.1335534445@critter.freebsd.dk> In message <20120426185723.GA9552 at rox.home.comstyle.com>, Brad Smith writes: >Unlike the 3.0.2 release checking out the latest and greatest from Git >doesn't fare well building on OpenBSD. The following diff fixes the >build issues. Committed, thanks. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From brad at comstyle.com Thu Apr 26 06:13:40 2012 From: brad at comstyle.com (Brad Smith) Date: Thu, 26 Apr 2012 02:13:40 -0400 Subject: [PATCH] Fix libedit detection on *BSD OS's In-Reply-To: <20120102190345.GA24654@rox.home.comstyle.com> References: <20120102190345.GA24654@rox.home.comstyle.com> Message-ID: <4F98E794.4030902@comstyle.com> On 02/01/12 2:03 PM, Brad wrote: > The following patch allows the autoconf script to detect the presence of > libedit when there isn't a pkg-config file present and thus allowing > Varnish to detect libedit on OpenBSD/FreeBSD/NetBSD/DragonFly. > > diff --git a/bin/varnishadm/varnishadm.c b/bin/varnishadm/varnishadm.c > index bb5cc8e..90532a8 100644 > --- a/bin/varnishadm/varnishadm.c > +++ b/bin/varnishadm/varnishadm.c > @@ -33,8 +33,12 @@ > > #ifdef HAVE_LIBEDIT > #include > +#ifdef HAVE_READLINE_READLINE_H > +#include > +#else > #include > #endif > +#endif > > #include > #include This part of the diff was not commited and the tree is broken at the moment. -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.