From tfheen at varnish-software.com Fri Apr 15 06:27:00 2011 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Fri, 15 Apr 2011 08:27:00 +0200 Subject: TCP_Check() compatibility for NetBSD In-Reply-To: <20110413121542.48140fee.daniel.bilik@neosystem.cz> (Daniel Bilik's message of "Wed, 13 Apr 2011 12:15:42 +0200") References: <20110413121542.48140fee.daniel.bilik@neosystem.cz> Message-ID: <8739lk6nd7.fsf@qurzaw.varnish-software.com> ]] Daniel Bilik Hi, | It can be fixed by simply extending the condition for macro definition, see | attached patch. Thanks, patch applied. varnish-bugs isn't a normal mailing list, it's just for use by trac, so please send to varnish-dev in the future. :-) (Or file a ticket in trac.) Regards, -- Tollef Fog Heen Varnish Software t: +47 21 98 92 64 From tfheen at varnish-software.com Fri Apr 15 08:39:55 2011 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Fri, 15 Apr 2011 10:39:55 +0200 Subject: #900 - suggested fix Message-ID: <87r5936h7o.fsf@qurzaw.varnish-software.com> Hi, I've attached my suggested fix for #900. As a side effect, it allows you to embed binary data in arbitrary strings, so you can serve PNG or GIF images from vcl_error. Comments very welcome, both on the approach and what we should allow. Regards, -- Tollef Fog Heen Varnish Software t: +47 21 98 92 64 -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Fix-up-and-support-in-VCL.patch Type: text/x-diff Size: 2006 bytes Desc: not available URL: From jbq at caraldi.com Fri Apr 15 14:26:18 2011 From: jbq at caraldi.com (Jean-Baptiste Quenot) Date: Fri, 15 Apr 2011 16:26:18 +0200 Subject: Assert error in WSLR() Message-ID: Hi there, Since upgrading from Varnish 2.0.6 to Varnish 2.1.3 (both with the Apache LogFormat patch applied) I get this error in production: Apr 15 11:54:31 gw1 varnishd[11313]: Child (9244) Panic message: Assert error in WSLR(), shmlog.c line 236: Condition(w->wlp < w->wle) not true. thread = (cache-worker) ident = Linux,2.6.35-28-server,x86_64,-sfile,-hcritbit,epoll Backtrace: 0x4235f8: /usr/sbin/varnishd() [0x4235f8] 0x4385ed: /usr/sbin/varnishd(WSLR+0x1cd) [0x4385ed] 0x4203cd: /usr/sbin/varnishd() [0x4203cd] 0x4205ce: /usr/sbin/varnishd(http_DissectRequest+0xee) [0x4205ce] 0x414360: /usr/sbin/varnishd(CNT_Session+0x670) [0x414360] 0x424da8: /usr/sbin/varnishd() [0x424da8] 0x42517b: /usr/sbin/varnishd() [0x42517b] 0x7f07dcb8f971: /lib/libpthread.so.0(+0x7971) [0x7f07dcb8f971] 0x7f07dc44e92d: /lib/libc.so.6(clone+0x6d) [0x7f07dc44e92d] sp = 0x7efb39157008 { fd = 35, id = 35, xid = 1796073229, client = 208.80.194.37:50339, step = STP_START, handling = deliver, restarts = 0, esis = 0 ws = 0x7efb39157078 { id = "sess", {s,f,r,e} = {0x7efb39157cd0,+25184,(nil),+65536}, }, http[req] = { ws = 0x7efb39157078[sess] "GET", "/some/path.html", "HTTP/1.0", "Host: some.host.com", "User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; knst; knst2007; Editor 3.0 - http://www.e-ditor.com; .NET CLR 1.1.4322)", }, worker = 0x7efb236f6be0 { ws = 0x7efb236f6d50 { id = "wrk", {s,f,r,e} = {0x7efb236e4b80,0x7efb236e4b80,(nil),+65536}, }, }, vcl = { srcname = { "input", "Default", }, }, }, Apr 15 11:54:31 gw1 varnishd[11313]: child (20985) Started Apr 15 11:54:32 gw1 varnishd[11313]: Child (20985) said Apr 15 11:54:32 gw1 varnishd[11313]: Child (20985) said Child starts Apr 15 11:54:32 gw1 varnishd[11313]: Child (20985) said managed to mmap 53687091200 bytes of 53687091200 Does it ring a bell to you? Thanks in advance, -- Jean-Baptiste Quenot From kristian at varnish-software.com Fri Apr 15 15:15:51 2011 From: kristian at varnish-software.com (Kristian Lyngstol) Date: Fri, 15 Apr 2011 17:15:51 +0200 Subject: [PATCH] Various 4GB+ fixes Message-ID: <20110415151551.GB4984@localhost.localdomain> Mainly changing int/usigned to size_t/ssize_t and a bit of %u to %lu etc. Not _too_ thoroughly tested, but it went through and the expiry didn't blast apart any more. A quick look-through before push/commit would be nice. (Ops, just spotted my white-space mix-up. I'll fix before push) --- bin/varnishd/cache.h | 8 ++++---- bin/varnishd/cache_center.c | 8 +++++--- bin/varnishd/cache_fetch.c | 4 ++-- bin/varnishd/cache_response.c | 2 +- bin/varnishd/cache_wrw.c | 6 +++--- 5 files changed, 15 insertions(+), 13 deletions(-) diff --git a/bin/varnishd/cache.h b/bin/varnishd/cache.h index a1ea70c..d691fa6 100644 --- a/bin/varnishd/cache.h +++ b/bin/varnishd/cache.h @@ -362,8 +362,8 @@ struct storage { void *priv; unsigned char *ptr; - unsigned len; - unsigned space; + ssize_t len; + ssize_t space; int fd; off_t where; @@ -807,10 +807,10 @@ void WRW_EndChunk(struct worker *w); void WRW_Reserve(struct worker *w, int *fd); unsigned WRW_Flush(struct worker *w); unsigned WRW_FlushRelease(struct worker *w); -unsigned WRW_Write(struct worker *w, const void *ptr, int len); +ssize_t WRW_Write(struct worker *w, const void *ptr, size_t len); unsigned WRW_WriteH(struct worker *w, const txt *hh, const char *suf); #ifdef SENDFILE_WORKS -void WRW_Sendfile(struct worker *w, int fd, off_t off, unsigned len); +void WRW_Sendfile(struct worker *w, int fd, off_t off, ssize_t len); #endif /* SENDFILE_WORKS */ typedef void *bgthread_t(struct sess *, void *priv); diff --git a/bin/varnishd/cache_center.c b/bin/varnishd/cache_center.c index 1c1118c..d753fa7 100644 --- a/bin/varnishd/cache_center.c +++ b/bin/varnishd/cache_center.c @@ -325,7 +325,7 @@ cnt_done(struct sess *sp) da = sp->t_end - sp->t_resp; dh = sp->t_req - sp->t_open; /* XXX: Add StatReq == StatSess */ - WSP(sp, SLT_Length, "%u", sp->acct_req.bodybytes); + WSP(sp, SLT_Length, "%lu", sp->acct_req.bodybytes); WSL(sp->wrk, SLT_ReqEnd, sp->id, "%u %.9f %.9f %.9f %.9f %.9f", sp->xid, sp->t_req, sp->t_end, dh, dp, da); } @@ -628,9 +628,11 @@ cnt_fetchbody(struct sess *sp) int i; struct http *hp, *hp2; char *b; - unsigned l, nhttp; + ssize_t l; + unsigned nhttp; struct vsb *vary = NULL; - int varyl = 0, pass; + size_t varyl = 0; + int pass; assert(sp->handling == VCL_RET_HIT_FOR_PASS || sp->handling == VCL_RET_DELIVER); diff --git a/bin/varnishd/cache_fetch.c b/bin/varnishd/cache_fetch.c index 87ce092..0f5dcc4 100644 --- a/bin/varnishd/cache_fetch.c +++ b/bin/varnishd/cache_fetch.c @@ -570,7 +570,7 @@ FetchBody(struct sess *sp) if (cls == 0 && sp->wrk->do_close) cls = 1; - WSL(sp->wrk, SLT_Length, sp->vbc->fd, "%u", sp->obj->len); + WSL(sp->wrk, SLT_Length, sp->vbc->fd, "%lu", sp->obj->len); { /* Sanity check fetch methods accounting */ @@ -585,7 +585,7 @@ FetchBody(struct sess *sp) if (mklen > 0) { http_Unset(sp->obj->http, H_Content_Length); http_PrintfHeader(sp->wrk, sp->fd, sp->obj->http, - "Content-Length: %u", sp->obj->len); + "Content-Length: %lu", sp->obj->len); } if (cls) diff --git a/bin/varnishd/cache_response.c b/bin/varnishd/cache_response.c index 2fbf120..0a2f3e9 100644 --- a/bin/varnishd/cache_response.c +++ b/bin/varnishd/cache_response.c @@ -281,7 +281,7 @@ res_WriteGunzipObj(struct sess *sp) static void res_WriteDirObj(struct sess *sp, size_t low, size_t high) { - unsigned u = 0; + ssize_t u = 0; size_t ptr, off, len; struct storage *st; diff --git a/bin/varnishd/cache_wrw.c b/bin/varnishd/cache_wrw.c index 2340aa3..8aaa5c1 100644 --- a/bin/varnishd/cache_wrw.c +++ b/bin/varnishd/cache_wrw.c @@ -175,8 +175,8 @@ WRW_WriteH(struct worker *w, const txt *hh, const char *suf) return (u); } -unsigned -WRW_Write(struct worker *w, const void *ptr, int len) +ssize_t +WRW_Write(struct worker *w, const void *ptr, size_t len) { struct wrw *wrw; @@ -237,7 +237,7 @@ WRW_EndChunk(struct worker *w) #ifdef SENDFILE_WORKS void -WRW_Sendfile(struct worker *w, int fd, off_t off, unsigned len) +WRW_Sendfile(struct worker *w, int fd, off_t off, ssize_t len) { struct wrw *wrw; -- 1.7.1 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: Digital signature URL: From l at lrowe.co.uk Mon Apr 18 17:06:49 2011 From: l at lrowe.co.uk (Laurence Rowe) Date: Mon, 18 Apr 2011 18:06:49 +0100 Subject: #900 - suggested fix In-Reply-To: <87r5936h7o.fsf@qurzaw.varnish-software.com> References: <87r5936h7o.fsf@qurzaw.varnish-software.com> Message-ID: On 15 April 2011 09:39, Tollef Fog Heen wrote:> > Hi, ?I've attached my suggested fix for #900. ?As a side effect, it > allows you to embed binary data in arbitrary strings, so you can serve > PNG or GIF images from vcl_error. > > Comments very welcome, both on the approach and what we should allow. Allowing binary data in vcl strings (I'm assuming \0 is what's missing here currently) sounds really useful in conjunction with the new vmod_digest module - I'm interested in being able to extract and validate signed authentication cookies in my VCL and the more of the string manipulation that can be done in VCL rather than C code, the easier that will be. Laurence From l at lrowe.co.uk Mon Apr 18 17:11:30 2011 From: l at lrowe.co.uk (Laurence Rowe) Date: Mon, 18 Apr 2011 18:11:30 +0100 Subject: #900 - suggested fix In-Reply-To: References: <87r5936h7o.fsf@qurzaw.varnish-software.com> Message-ID: On 18 April 2011 18:06, Laurence Rowe wrote: > On 15 April 2011 09:39, Tollef Fog Heen wrote:> >> Hi, ?I've attached my suggested fix for #900. ?As a side effect, it >> allows you to embed binary data in arbitrary strings, so you can serve >> PNG or GIF images from vcl_error. >> >> Comments very welcome, both on the approach and what we should allow. > > Allowing binary data in vcl strings (I'm assuming \0 is what's missing > here currently) sounds really useful in conjunction with the new > vmod_digest module - I'm interested in being able to extract and > validate signed authentication cookies in my VCL and the more of the > string manipulation that can be done in VCL rather than C code, the > easier that will be. Having looked at the patch, I see that you're only changing the vcl parser. Supporting arbitrary binary data in strings will require support for null bytes, but vcl strings are currently null terminated if I remember correctly? Laurence From phk at phk.freebsd.dk Mon Apr 18 10:41:59 2011 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Mon, 18 Apr 2011 10:41:59 +0000 Subject: [PATCH] Various 4GB+ fixes In-Reply-To: Your message of "Fri, 15 Apr 2011 17:15:51 +0200." <20110415151551.GB4984@localhost.localdomain> Message-ID: <3029.1303123319@critter.freebsd.dk> In message <20110415151551.GB4984 at localhost.localdomain>, Kristian Lyngstol wri tes: As I mentioned on IRC, it is intentional that struct storage is not 64-bit, there is no advantage of having storage-chunks larger than a couple of GB and asking for it may cause the kernel to do more than than good is. I've cherry picked some of these changes, be aware that we should use "%jd" printf formats, trusting "%ld" to do the right thing is not productive. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From harm.verhagen at gmail.com Sat Apr 23 12:28:20 2011 From: harm.verhagen at gmail.com (Harm Verhagen) Date: Sat, 23 Apr 2011 14:28:20 +0200 Subject: varnish not generating 304 in case multiple etags are used. Message-ID: Hi, I'm having a problem with the handling of http responses that contain multiple etag headers. I think varnish does not handle this case as effecient as it can. (And therefor I suspect a bug in varnish). What do you think? steps to reproduce ============== See request/response below. Force a reload of a cached item in browser: Cache-Control: max-age=0 result ==== HTTP 200 is returned, varnish serves file content, varnish does NOT contact backend. expected result ============ HTTP 304 is returned, varnish does not server file content, varnish does NOT contact backend. NOTE ===== This does work correct, in case the reply contains a SINGLE etag header. -> all works as expected, I get a nice 304 instead of a 200. As soon as the reply contains TWO etag headers, it seems varnish never returns a 304. Which I think is incorrect. No ? Note; This is not related to the vlc, as single etag headers are working fine. (I'm doing NOTHING with headers, etag or others in my vlc). request (anonimised) ======= GET /dam/asset/download/uuid/121a8df4-4fe8-11e0-bd65-56167b3dc56d/variant/103/ HTTP/1.1 Host: X.X.X User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:2.0) Gecko/20100101 Firefox/4.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: nl,en-us;q=0.7,en;q=0.3 Accept-Encoding: gzip, deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 115 Connection: keep-alive Cookie: __utma=1.1741069999.1296235686.1303504321.1303507491.8; If-Modified-Since: Fri, 22 Apr 2011 13:00:00 GMT If-None-Match: "d41d8cd98f00b204e9800998ecf8427e", "4a00e1-3ddf-4a1816f7a9400" Cache-Control: max-age=0 reply HTTP/1.1 200 OK Server: Apache/2.2.14 (Ubuntu) Content-Disposition: inline; filename="Seminar Speakers.jpg" Expires: Sat, 23 Apr 2011 13:00:00 GMT Etag: "d41d8cd98f00b204e9800998ecf8427e", "4a00e1-3ddf-4a1816f7a9400" Cache-Control: max-age=86400 P3P: CP="IDC DSP COR ADM DEVi TAIi PSA PSD IVAi IVDi CONi HIS OUR IND CNT" Last-Modified: Fri, 22 Apr 2011 13:00:00 GMT Content-Type: image/jpeg;charset=utf-8 Content-Length: 15839 Date: Sat, 23 Apr 2011 12:07:39 GMT X-Varnish: 2113947582 2113947501 Age: 58 Via: 1.1 varnish Connection: keep-alive version ====== varnish 2.1.5 Log transcript of a single request+response (varnishlog) ================================= 11 SessionOpen c X.X.X.220 50540 :80 11 ReqStart c X.X.X.220 50540 2113947582 11 RxRequest c GET 11 RxURL c /dam/asset/download/uuid/121a8df4-4fe8-11e0-bd65-56167b3dc56d/variant/103/ 11 RxProtocol c HTTP/1.1 11 RxHeader c Host: X.X.X 11 RxHeader c User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:2.0) Gecko/20100101 Firefox/4.0 11 RxHeader c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 11 RxHeader c Accept-Language: nl,en-us;q=0.7,en;q=0.3 11 RxHeader c Accept-Encoding: gzip, deflate 11 RxHeader c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 11 RxHeader c Keep-Alive: 115 11 RxHeader c Connection: keep-alive 11 RxHeader c Cookie: __utma=1.1741069999.1296235686.1303504321.1303507491.8; __utmz=1.1303507491.8.8.utmcsr=senzio.com|utmccn=(referral)|utmcmd=referral|utmcct=/; __utma=142895841.963436971.1296236932.1302872710.1303499754.14; __utmz=142895841.1296236932.1.1.utmcsr=(d 11 RxHeader c If-Modified-Since: Fri, 22 Apr 2011 13:00:00 GMT 11 RxHeader c If-None-Match: "d41d8cd98f00b204e9800998ecf8427e", "4a00e1-3ddf-4a1816f7a9400" 11 RxHeader c Cache-Control: max-age=0 11 VCL_call c recv 11 VCL_return c lookup 11 VCL_call c hash 11 VCL_return c hash 11 Hit c 2113947501 11 VCL_call c hit 11 LostHeader c X-Cache: 11 VCL_return c deliver 11 VCL_call c deliver 11 VCL_return c deliver 11 TxProtocol c HTTP/1.1 11 TxStatus c 200 11 TxResponse c OK 11 TxHeader c Server: Apache/2.2.14 (Ubuntu) 11 TxHeader c Content-Disposition: inline; filename="Seminar Speakers.jpg" 11 TxHeader c Expires: Sat, 23 Apr 2011 13:00:00 GMT 11 TxHeader c ETag: "d41d8cd98f00b204e9800998ecf8427e" 11 TxHeader c Cache-Control: max-age=86400 11 TxHeader c P3P: CP="IDC DSP COR ADM DEVi TAIi PSA PSD IVAi IVDi CONi HIS OUR IND CNT" 11 TxHeader c Last-Modified: Fri, 22 Apr 2011 13:00:00 GMT 11 TxHeader c ETag: "4a00e1-3ddf-4a1816f7a9400" 11 TxHeader c Content-Type: image/jpeg;charset=utf-8 11 TxHeader c Content-Length: 15839 11 TxHeader c Date: Sat, 23 Apr 2011 12:07:39 GMT 11 TxHeader c X-Varnish: 2113947582 2113947501 11 TxHeader c Age: 58 11 TxHeader c Via: 1.1 varnish 11 TxHeader c Connection: keep-alive 11 Length c 15839 11 ReqEnd c 2113947582 1303560459.219877005 1303560459.317925930 0.000497818 0.000181675 0.097867250 11 Debug c "herding" 0 ExpKill - 2113947483 -30 0 CLI - Rd ping 0 CLI - Wr 200 19 PONG 1303560459 1.0 Regards, Harm Verhagen -------------- next part -------------- An HTML attachment was scrubbed... URL: From sky at crucially.net Sat Apr 23 16:57:34 2011 From: sky at crucially.net (sky at crucially.net) Date: Sat, 23 Apr 2011 16:57:34 +0000 Subject: varnish not generating 304 in case multiple etags are used. In-Reply-To: References: Message-ID: <1849898851-1303577858-cardhu_decombobulator_blackberry.rim.net-1309323071-@bda098.bisx.prod.on.blackberry> What is it supposed to do. Sent via BlackBerry by AT&T -----Original Message----- From: Harm Verhagen Sender: varnish-dev-bounces at varnish-cache.org Date: Sat, 23 Apr 2011 14:28:20 To: Subject: varnish not generating 304 in case multiple etags are used. _______________________________________________ varnish-dev mailing list varnish-dev at varnish-cache.org http://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev From mnot at yahoo-inc.com Tue Apr 26 00:31:04 2011 From: mnot at yahoo-inc.com (Mark Nottingham) Date: Mon, 25 Apr 2011 17:31:04 -0700 Subject: varnish not generating 304 in case multiple etags are used. In-Reply-To: References: Message-ID: An ETag header can only contain one ETag. See: http://tools.ietf.org/html/draft-ietf-httpbis-p4-conditional-14#section-2.2 Cheers, On 23/04/2011, at 10:28 PM, Harm Verhagen wrote: > Hi, > > I'm having a problem with the handling of http responses that contain multiple etag headers. I think varnish does not handle this case as effecient as it can. (And therefor I suspect a bug in varnish). > What do you think? > > > steps to reproduce > ============== > See request/response below. Force a reload of a cached item in browser: Cache-Control: max-age=0 > > result > ==== > HTTP 200 is returned, varnish serves file content, varnish does NOT contact backend. > > expected result > ============ > HTTP 304 is returned, varnish does not server file content, varnish does NOT contact backend. > > NOTE > ===== > This does work correct, in case the reply contains a SINGLE etag header. -> all works as expected, I get a nice 304 instead of a 200. > As soon as the reply contains TWO etag headers, it seems varnish never returns a 304. Which I think is incorrect. No ? > > Note; This is not related to the vlc, as single etag headers are working fine. (I'm doing NOTHING with headers, etag or others in my vlc). > > request (anonimised) > ======= > GET /dam/asset/download/uuid/121a8df4-4fe8-11e0-bd65-56167b3dc56d/variant/103/ HTTP/1.1 > Host: X.X.X > User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:2.0) Gecko/20100101 Firefox/4.0 > Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 > Accept-Language: nl,en-us;q=0.7,en;q=0.3 > Accept-Encoding: gzip, deflate > Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 > Keep-Alive: 115 > Connection: keep-alive > Cookie: __utma=1.1741069999.1296235686.1303504321.1303507491.8; > If-Modified-Since: Fri, 22 Apr 2011 13:00:00 GMT > If-None-Match: "d41d8cd98f00b204e9800998ecf8427e", "4a00e1-3ddf-4a1816f7a9400" > Cache-Control: max-age=0 > > > reply > HTTP/1.1 200 OK > Server: Apache/2.2.14 (Ubuntu) > Content-Disposition: inline; filename="Seminar Speakers.jpg" > Expires: Sat, 23 Apr 2011 13:00:00 GMT > Etag: "d41d8cd98f00b204e9800998ecf8427e", "4a00e1-3ddf-4a1816f7a9400" > Cache-Control: max-age=86400 > P3P: CP="IDC DSP COR ADM DEVi TAIi PSA PSD IVAi IVDi CONi HIS OUR IND CNT" > Last-Modified: Fri, 22 Apr 2011 13:00:00 GMT > Content-Type: image/jpeg;charset=utf-8 > Content-Length: 15839 > Date: Sat, 23 Apr 2011 12:07:39 GMT > X-Varnish: 2113947582 2113947501 > Age: 58 > Via: 1.1 varnish > Connection: keep-alive > > > > > version > ====== > varnish 2.1.5 > > > > Log transcript of a single request+response (varnishlog) > ================================= > 11 SessionOpen c X.X.X.220 50540 :80 > 11 ReqStart c X.X.X.220 50540 2113947582 > 11 RxRequest c GET > 11 RxURL c /dam/asset/download/uuid/121a8df4-4fe8-11e0-bd65-56167b3dc56d/variant/103/ > 11 RxProtocol c HTTP/1.1 > 11 RxHeader c Host: X.X.X > 11 RxHeader c User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:2.0) Gecko/20100101 Firefox/4.0 > 11 RxHeader c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 > 11 RxHeader c Accept-Language: nl,en-us;q=0.7,en;q=0.3 > 11 RxHeader c Accept-Encoding: gzip, deflate > 11 RxHeader c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 > 11 RxHeader c Keep-Alive: 115 > 11 RxHeader c Connection: keep-alive > 11 RxHeader c Cookie: __utma=1.1741069999.1296235686.1303504321.1303507491.8; __utmz=1.1303507491.8.8.utmcsr=senzio.com|utmccn=(referral)|utmcmd=referral|utmcct=/; __utma=142895841.963436971.1296236932.1302872710.1303499754.14; __utmz=142895841.1296236932.1.1.utmcsr=(d > 11 RxHeader c If-Modified-Since: Fri, 22 Apr 2011 13:00:00 GMT > 11 RxHeader c If-None-Match: "d41d8cd98f00b204e9800998ecf8427e", "4a00e1-3ddf-4a1816f7a9400" > 11 RxHeader c Cache-Control: max-age=0 > 11 VCL_call c recv > 11 VCL_return c lookup > 11 VCL_call c hash > 11 VCL_return c hash > 11 Hit c 2113947501 > 11 VCL_call c hit > 11 LostHeader c X-Cache: > 11 VCL_return c deliver > 11 VCL_call c deliver > 11 VCL_return c deliver > 11 TxProtocol c HTTP/1.1 > 11 TxStatus c 200 > 11 TxResponse c OK > 11 TxHeader c Server: Apache/2.2.14 (Ubuntu) > 11 TxHeader c Content-Disposition: inline; filename="Seminar Speakers.jpg" > 11 TxHeader c Expires: Sat, 23 Apr 2011 13:00:00 GMT > 11 TxHeader c ETag: "d41d8cd98f00b204e9800998ecf8427e" > 11 TxHeader c Cache-Control: max-age=86400 > 11 TxHeader c P3P: CP="IDC DSP COR ADM DEVi TAIi PSA PSD IVAi IVDi CONi HIS OUR IND CNT" > 11 TxHeader c Last-Modified: Fri, 22 Apr 2011 13:00:00 GMT > 11 TxHeader c ETag: "4a00e1-3ddf-4a1816f7a9400" > 11 TxHeader c Content-Type: image/jpeg;charset=utf-8 > 11 TxHeader c Content-Length: 15839 > 11 TxHeader c Date: Sat, 23 Apr 2011 12:07:39 GMT > 11 TxHeader c X-Varnish: 2113947582 2113947501 > 11 TxHeader c Age: 58 > 11 TxHeader c Via: 1.1 varnish > 11 TxHeader c Connection: keep-alive > 11 Length c 15839 > 11 ReqEnd c 2113947582 1303560459.219877005 1303560459.317925930 0.000497818 0.000181675 0.097867250 > 11 Debug c "herding" > 0 ExpKill - 2113947483 -30 > 0 CLI - Rd ping > 0 CLI - Wr 200 19 PONG 1303560459 1.0 > > > Regards, > Harm Verhagen > _______________________________________________ > varnish-dev mailing list > varnish-dev at varnish-cache.org > http://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev -- Mark Nottingham mnot at yahoo-inc.com From tfheen at varnish-software.com Wed Apr 27 08:32:29 2011 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Wed, 27 Apr 2011 10:32:29 +0200 Subject: [PATCH] support for multiple -o arguments to varnish tools Message-ID: <87aafct7qa.fsf@qurzaw.varnish-software.com> The following patch allows multiple -o arguments to varnishlog, varnishncsa, varnishhist and varnishsizes. The format is -o tag:regex which will only count a transaction if it matches all the -o parameters. Feedback is most welcome, in particular I wonder if -o is a good choice or if we should go with -m(atch) or somtheing similar. -o is also somewhat confusing since some people think it means -o(r) Comments on the implementation is welcome too, of course. Regards, -------------- next part -------------- A non-text attachment was scrubbed... Name: multiple_o_support.diff Type: text/x-diff Size: 19701 bytes Desc: not available URL: -------------- next part -------------- -- Tollef Fog Heen Varnish Software t: +47 21 98 92 64 From phk at phk.freebsd.dk Wed Apr 27 08:46:16 2011 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Wed, 27 Apr 2011 08:46:16 +0000 Subject: [PATCH] support for multiple -o arguments to varnish tools In-Reply-To: Your message of "Wed, 27 Apr 2011 10:32:29 +0200." <87aafct7qa.fsf@qurzaw.varnish-software.com> Message-ID: <66952.1303893976@critter.freebsd.dk> In message <87aafct7qa.fsf at qurzaw.varnish-software.com>, Tollef Fog Heen writes : >The following patch allows multiple -o arguments to varnishlog, Just for the record: I take a pretty hands-off attitude to the OaM tools, varnish{log,stat,hist,...}, because I know even less about what people need from these than from varnishd. I have some code-quality and overall structure concerns, but with respect to what the tools do and how they do it, It's up to the community to figure things out. So please give Tollefs patch a spin and give him some feedback. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From martin at varnish-software.com Thu Apr 28 14:03:19 2011 From: martin at varnish-software.com (Martin Blix Grydeland) Date: Thu, 28 Apr 2011 16:03:19 +0200 Subject: When changing purge->ban in 3.0, should we also expose the VRT_ban_string functionality from VCL instead of relying on magic parsing the difference Message-ID: I noticed while working on a case that when implementing general ban expressions through VCL you have to trick the VCL compiler into accepting to read the whole ban expression through a header variable. The use case was like this: sub vcl_recv { # Incoming header ban-expression is a complete ban expression containing e.g. "req.url ~ \.gif" if (req.method == "BAN" && req.http.ban-expression) { ban(req.http.ban-expression); } } This will produce a compiler error of missing expected operator, as the compiler recognizes the req.http.ban-expression as a variable and goes into the mode of recognizing ban-expressions (reads 3 arguments, first a variable, 2nd an operator and third a stringval, possibly followed by && and other triplets). If the first argument isn't a variable, it will feed a string to VRT_ban_string instead for parsing at runtime. So a work around for the above is to instead do the ban like this: ban("" + req.http.ban-expression); This tricks the VCL compiler into doing runtime parsing of the expression instead, but it's not pretty and intuitive. So I was wondering if we now that we make changes to the purge->ban naming for 3.0, should also do away with the automagic context guessing. I suggest adding a ban_string() function that always does the expression parsing at runtime, and make the ban() function always do the expression parsing at compile time. Regards, Martin Blix Grydeland -- Martin Blix Grydeland Varnish Software AS -------------- next part -------------- An HTML attachment was scrubbed... URL: From jbq at caraldi.com Thu Apr 28 15:48:41 2011 From: jbq at caraldi.com (Jean-Baptiste Quenot) Date: Thu, 28 Apr 2011 17:48:41 +0200 Subject: Assert error in WSLR() In-Reply-To: References: Message-ID: Can I safely ignore this error, or shall I dig further to find the cause? 2011/4/15 Jean-Baptiste Quenot : > Hi there, > > Since upgrading from Varnish 2.0.6 to Varnish 2.1.3 (both with the > Apache LogFormat patch applied) I get this error in production: > > Apr 15 11:54:31 gw1 varnishd[11313]: Child (9244) Panic message: > Assert error in WSLR(), shmlog.c line 236: > ?Condition(w->wlp < w->wle) not true. > thread = (cache-worker) > ident = Linux,2.6.35-28-server,x86_64,-sfile,-hcritbit,epoll > Backtrace: > ?0x4235f8: /usr/sbin/varnishd() [0x4235f8] > ?0x4385ed: /usr/sbin/varnishd(WSLR+0x1cd) [0x4385ed] > ?0x4203cd: /usr/sbin/varnishd() [0x4203cd] > ?0x4205ce: /usr/sbin/varnishd(http_DissectRequest+0xee) [0x4205ce] > ?0x414360: /usr/sbin/varnishd(CNT_Session+0x670) [0x414360] > ?0x424da8: /usr/sbin/varnishd() [0x424da8] > ?0x42517b: /usr/sbin/varnishd() [0x42517b] > ?0x7f07dcb8f971: /lib/libpthread.so.0(+0x7971) [0x7f07dcb8f971] > ?0x7f07dc44e92d: /lib/libc.so.6(clone+0x6d) [0x7f07dc44e92d] > sp = 0x7efb39157008 { > ?fd = 35, id = 35, xid = 1796073229, > ?client = 208.80.194.37:50339, > ?step = STP_START, > ?handling = deliver, > ?restarts = 0, esis = 0 > ?ws = 0x7efb39157078 { > ? id = "sess", > ? {s,f,r,e} = {0x7efb39157cd0,+25184,(nil),+65536}, > ?}, > ?http[req] = { > ? ws = 0x7efb39157078[sess] > ? ? "GET", > ? ? "/some/path.html", > ? ? "HTTP/1.0", > ? ? "Host: some.host.com", > ? ? "User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; > SV1; knst; knst2007; Editor 3.0 - http://www.e-ditor.com; .NET CLR > 1.1.4322)", > ?}, > ?worker = 0x7efb236f6be0 { > ? ws = 0x7efb236f6d50 { > ? ? id = "wrk", > ? ? {s,f,r,e} = {0x7efb236e4b80,0x7efb236e4b80,(nil),+65536}, > ? }, > ? }, > ? vcl = { > ? ? srcname = { > ? ? ? "input", > ? ? ? "Default", > ? ? }, > ? }, > }, > Apr 15 11:54:31 gw1 varnishd[11313]: child (20985) Started > Apr 15 11:54:32 gw1 varnishd[11313]: Child (20985) said > Apr 15 11:54:32 gw1 varnishd[11313]: Child (20985) said Child starts > Apr 15 11:54:32 gw1 varnishd[11313]: Child (20985) said managed to > mmap 53687091200 bytes of 53687091200 > > Does it ring a bell to you? > > Thanks in advance, > -- > Jean-Baptiste Quenot > -- Jean-Baptiste Quenot From phk at phk.freebsd.dk Thu Apr 28 22:14:12 2011 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Thu, 28 Apr 2011 22:14:12 +0000 Subject: When changing purge->ban in 3.0, should we also expose the VRT_ban_string functionality from VCL instead of relying on magic parsing the difference In-Reply-To: Your message of "Thu, 28 Apr 2011 16:03:19 +0200." Message-ID: <29566.1304028852@critter.freebsd.dk> In message , Martin Blix Gr ydeland writes: >This will produce a compiler error of missing expected operator, as the >compiler recognizes the req.http.ban-expression as a variable and goes into >the mode of recognizing ban-expressions (reads 3 arguments, first a >variable, 2nd an operator and third a stringval, possibly followed by && and >other triplets). If the first argument isn't a variable, it will feed a >string to VRT_ban_string instead for parsing at runtime. I'm not happy with the way this entire thing turned out. It looked quite sensible when I started out and then it got ugly on contact with the reality. I'm sort of hessitant to mandate the one-string format, because it will force some people to collect into one string, components which the BAN code will immediately split back into those components. One possible, but ugly-ish solution, is to make the VCL::ban function var-args, and let the number of args steer the interpretation: ban(req.http.ban-expression); ban(req.http.foo, '==', req.http.ban-detail); Not sure how much work that is to implement, if it is 3.0 material or even a good idea. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From tfheen at varnish-software.com Fri Apr 29 04:55:05 2011 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Fri, 29 Apr 2011 06:55:05 +0200 Subject: When changing purge->ban in 3.0, should we also expose the VRT_ban_string functionality from VCL instead of relying on magic parsing the difference In-Reply-To: <29566.1304028852@critter.freebsd.dk> (Poul-Henning Kamp's message of "Thu, 28 Apr 2011 22:14:12 +0000") References: <29566.1304028852@critter.freebsd.dk> Message-ID: <8762pxfyhi.fsf@qurzaw.varnish-software.com> ]] "Poul-Henning Kamp" | I'm sort of hessitant to mandate the one-string format, because it | will force some people to collect into one string, components which | the BAN code will immediately split back into those components. Apart from the slight silliness and inelegance of this, does it cause any real-world problems? I agree it's not optimal, but you probably don't have a zillion bans a second so its performance is not critical and having a consistent UI is, IMO, pretty important. -- Tollef Fog Heen Varnish Software t: +47 21 98 92 64 From phk at phk.freebsd.dk Fri Apr 29 08:37:03 2011 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Fri, 29 Apr 2011 08:37:03 +0000 Subject: When changing purge->ban in 3.0, should we also expose the VRT_ban_string functionality from VCL instead of relying on magic parsing the difference In-Reply-To: Your message of "Fri, 29 Apr 2011 06:55:05 +0200." <8762pxfyhi.fsf@qurzaw.varnish-software.com> Message-ID: <3108.1304066223@critter.freebsd.dk> In message <8762pxfyhi.fsf at qurzaw.varnish-software.com>, Tollef Fog Heen writes : >| I'm sort of hessitant to mandate the one-string format, because it >| will force some people to collect into one string, components which >| the BAN code will immediately split back into those components. > >Apart from the slight silliness and inelegance of this, does it cause >any real-world problems? I worry about quoting madness... -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From tfheen at varnish-software.com Fri Apr 29 10:32:50 2011 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Fri, 29 Apr 2011 12:32:50 +0200 Subject: When changing purge->ban in 3.0, should we also expose the VRT_ban_string functionality from VCL instead of relying on magic parsing the difference In-Reply-To: <3108.1304066223@critter.freebsd.dk> (Poul-Henning Kamp's message of "Fri, 29 Apr 2011 08:37:03 +0000") References: <3108.1304066223@critter.freebsd.dk> Message-ID: <87liyte4a5.fsf@qurzaw.varnish-software.com> ]] "Poul-Henning Kamp" | In message <8762pxfyhi.fsf at qurzaw.varnish-software.com>, Tollef Fog Heen writes | : | >| I'm sort of hessitant to mandate the one-string format, because it | >| will force some people to collect into one string, components which | >| the BAN code will immediately split back into those components. | > | >Apart from the slight silliness and inelegance of this, does it cause | >any real-world problems? | | I worry about quoting madness... It'd be less quoting madness by not trying to do magic in the ban parser: ban("req.url == " + req.url); looks a whole lot more sane than: ban("req.url == req.url"); where then req.url doesn't mean the same thing on the left and right hand sides. (Unless I've misunderstood something about what the current code does). Or are you worried about the case of req.url (or whatever you ban on) containing &&, || and similar characters? Can we solve that by having a function which escapes anything special to bans, so we can do: ban("req.url == " + ban_escape(req.url)); ? Cheers, -- Tollef Fog Heen Varnish Software t: +47 21 98 92 64 From phk at phk.freebsd.dk Fri Apr 29 10:48:45 2011 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Fri, 29 Apr 2011 10:48:45 +0000 Subject: When changing purge->ban in 3.0, should we also expose the VRT_ban_string functionality from VCL instead of relying on magic parsing the difference In-Reply-To: Your message of "Fri, 29 Apr 2011 12:32:50 +0200." <87liyte4a5.fsf@qurzaw.varnish-software.com> Message-ID: <19540.1304074125@critter.freebsd.dk> In message <87liyte4a5.fsf at qurzaw.varnish-software.com>, Tollef Fog Heen writes : >looks a whole lot more sane than: > >ban("req.url == req.url"); It would be: ban(req.url, '==', req.url) Or possibly even ban("req.url", '==', req.url) >Or are you worried about the case of req.url (or whatever you ban on) >containing &&, || and similar characters? Also that. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From ticaretmeslek at projects.linpro.no Sat Apr 30 04:24:34 2011 From: ticaretmeslek at projects.linpro.no (ticaretmeslek at projects.linpro.no) Date: Sat, 30 Apr 2011 06:24:34 +0200 (CEST) Subject: Hey!! Message-ID: <20110430042434.2631B1391BF@projects.linpro.no> Hello Stranger!!! I don't know who is there, but I want to check up my destiny. I'm single, 24 years old girl from Russia and I am opened for new acquaintances. Probably it was a joke, and your address doesn't work but I trust in destiny and hope that my letter doesn't remain without attention. If you are single and want find your soul mate too, reply to me please. My mail : irishkalovesmile at rambler.ru Irina.