From brian at briansmith.org Mon May 5 19:19:29 2008 From: brian at briansmith.org (Brian Smith) Date: Mon, 5 May 2008 12:19:29 -0700 Subject: mmap'd file vs. swap Message-ID: My understanding is that Varnish uses a mmap'd file, but it doesn't re-use the file's contents when it is restarted. Why does Varnish use a mmap'd file instead of just using regular virtual memory (swap) directly? I would think that the operating system would be more eager to write the data to the mmap'd file than it would be to write to swap; on a system where the hot cache entries can be stored on disk, it would seem that the swap-based method would be superior. (I ask because I am trying to decide which method to use for a back-end system, not to be critical of Varnish's design.) Thank you, Brian From phk at phk.freebsd.dk Mon May 5 19:44:23 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Mon, 05 May 2008 19:44:23 +0000 Subject: mmap'd file vs. swap In-Reply-To: Your message of "Mon, 05 May 2008 12:19:29 MST." Message-ID: <11740.1210016663@critter.freebsd.dk> In message , "Brian Smith" writes: >My understanding is that Varnish uses a mmap'd file, but it doesn't re-use >the file's contents when it is restarted. Correct. > Why does Varnish use a mmap'd file >instead of just using regular virtual memory (swap) directly? For many small reasons, none of which are particularly good. You can use regular VM by enabling the "malloc" storage method and various people report good success with that. >I would think >that the operating system would be more eager to write the data to the >mmap'd file than it would be to write to swap; on a system where the hot >cache entries can be stored on disk, it would seem that the swap-based >method would be superior. Well, it's slightly more complex than that, we mmap with the MAP_NOSYNC flag so it should be about a wash. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From brian at briansmith.org Tue May 6 02:40:23 2008 From: brian at briansmith.org (Brian Smith) Date: Mon, 5 May 2008 19:40:23 -0700 Subject: mmap'd file vs. swap References: <11740.1210016663@critter.freebsd.dk> Message-ID: <0FEAF43CB0D04D568394C0A421ADFF15@T60> Poul-Henning Kamp wrote: > You can use regular VM by enabling the "malloc" storage method > and various people report good success with that. ... > Well, it's slightly more complex than that, we mmap with the > MAP_NOSYNC flag so it should be about a wash. Thanks. I wasn't familiar with MAP_NOSYNC since I am using Linux, which doesn't have it. In the mmap-based implementation, it looks like you have to implement malloc-like logic yourself, including thread-safety and defragmentation. Does the mmap-based storage have some advantage to normal malloc() implementations (e.g. glibc, jemalloc, TCMalloc) due to its domain-specific knowledge? Thanks, Brian From jt at endpoint.com Tue May 20 04:08:15 2008 From: jt at endpoint.com (JT Justman) Date: Mon, 19 May 2008 21:08:15 -0700 Subject: Objects for ESI includes incorrectly miss Message-ID: <48324EAF.3020007@endpoint.com> Hi, everyone. I've been doing my best to dig into the bug I opened: http://varnish.projects.linpro.no/ticket/240 Basically, every object is inserted, but the second and subsiquent objects always miss. The object goes to the MISS step in cache_center cnt_lookup() where there is a check on sp->obj->busy. This seems to be an indication that there was no object on the session. cnt_lookup is called with a session that does not have an >obj cnt_lookup is called from the 'STEP' system, from cnt_recv - which itself doen't touch the obj, except to call 'AZ' on it, which I think is a check. RECV is called from fetch/RESTART, and cnt_start. And cache_vrt_esi. So obviously ESI is expected to do something to the object that it isn't. RESTART is calling Unbusy. ESI_Deliver is setting RECV. The object is fiddled with but seems to already exist. Tried calling Unbusy in ESI_Deliver. No success. Can't modify the object until after RECV. ESI_Deliver is setting the document's object aside, and calling RECV with a NULL'ed object. Yes, the object to RECV is null. Then it passes on to LOOKUP, still with NULL. LOOKUP checks for null, allocates, and calls VCL_hash_method. I.e., the manpage lies. vcl_hash manipulates req.hash. Back in cnt_lookup. HSH_Lookup is called and assigned to local o. HSH_Lookup locates objhead (?) then loops through the objects of objhead(?). It looks to see if the object was busy, if it is uncacheable, vary condition, banning, and grace period. Okay, I guess objhead is a hash of objects. It's looking for one that it can work with. If it finds one it likes, it returns it. In our case, it is finding the object for one include, and not for the other. So how is the object getting into objhead? There is a call to HSH_Prealloc. It pre-builds the objhead and object if needed. Hmmm.... the objhead has some kind of queue functionality. So if there is no objhead yet as of HSH_Lookup, then the empty one is used, and a call to hash->lookup fills it, I guess. It only places objhead into the session if there are busy items, otherwise into the object. hash is a hash_slinger. I guess this is mapping the different hash types. So lookup seems to be accessing the store and looking for the object. I guess it walks the hash looking for a match by comparing various values. It calls HSH_Compare as the final check, then returns the matching objhead. HSH_Compare is passed a session and an objhead. First it checks the hash lenghts. Then it checks if the hash itself matches. In our case, compare is returning 0 on both requests. Back to hash->lookup. This falls to the second have of the loop. It increases the header element's refcnt. Places the objhead in roh. Unlockes mtx. Then returns roh. So head back to HSH_Lookup. Now we've got the newly-retrieved objhead. Which we are looping through. What is oh->objects exactly? It's a VTAILQ_HEAD(,object) if that helps. TTL value is broken! NaN! Staring me in the face! So I think the date is funny. RFC2616_cache_policy is being called from cnt_fetch. Afterwards TTL is correct. From jt at endpoint.com Tue May 20 14:08:32 2008 From: jt at endpoint.com (JT Justman) Date: Tue, 20 May 2008 07:08:32 -0700 Subject: Objects for ESI includes incorrectly miss In-Reply-To: <48324EAF.3020007@endpoint.com> References: <48324EAF.3020007@endpoint.com> Message-ID: <4832DB60.6080004@endpoint.com> JT Justman wrote: > Hi, everyone. I've been doing my best to dig into the bug I opened: Whoops! I accidentally sent that with a lot of notes that weren't supposed to be there. Sorry. You can safely ignore it and I'll come back when I finish this line of investigation. JT From jt at endpoint.com Tue May 20 16:45:01 2008 From: jt at endpoint.com (JT Justman) Date: Tue, 20 May 2008 09:45:01 -0700 Subject: Objects for ESI includes incorrectly miss (Solved / Patch) In-Reply-To: <48324EAF.3020007@endpoint.com> References: <48324EAF.3020007@endpoint.com> Message-ID: <4833000D.3010204@endpoint.com> JT Justman wrote: > Hi, everyone. I've been doing my best to dig into the bug I opened: > > http://varnish.projects.linpro.no/ticket/240 > > Basically, every object is inserted, but the second and subsiquent > objects always miss. I have found the cause for the bug, and a potential fix. In cache_hash.c function HSH_Lookup, there is a ttl check: if (o->ttl >= sp->t_req) This test passes on the first ESI and fails on the others. Debugging shows that the ttl is correct, but t_req is NAN. It seems that t_req is cleared, and needs to be reset somewhere in ESI or the step flow. My solution is to set sp->t_req = TIM_real() in ESI_Deliver. (See patch). Not being intimate with the program flow or timing, I don't know what other implications this could have. Suggestions? JT -------------- next part -------------- A non-text attachment was scrubbed... Name: cache_vrt_esi-jt_2008-05-20.patch Type: text/x-diff Size: 410 bytes Desc: not available URL: From charlescurley at charlescurley.com Wed May 21 18:57:14 2008 From: charlescurley at charlescurley.com (Charles Curley) Date: Wed, 21 May 2008 12:57:14 -0600 Subject: Mapping VCL variables to internal variables Message-ID: <20080521185714.GC6756@charlescurley.com> I'd like to set a variable in VCL, and then use it in a modification of Varnish. Having added to req.hash with something like: sub vcl_hash { if (req.http.Accept-Encoding ~ "gzip") { set req.hash += "gzip"; } else if (req.http.Accept-Encoding ~ "deflate") { set req.hash += "deflate"; } } where do I find that in Varnish itself? Thanks -- Charles Curley /"\ ASCII Ribbon Campaign Looking for fine software \ / Respect for open standards and/or writing? X No HTML/RTF in email http://www.charlescurley.com / \ No M$ Word docs in email Key fingerprint = CE5C 6645 A45A 64E4 94C0 809C FFF6 4C48 4ECD DFDB -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: From yu at irx.jp Thu May 22 11:24:42 2008 From: yu at irx.jp (Yutaro Shimamura) Date: Thu, 22 May 2008 20:24:42 +0900 Subject: sendfile() in varnish Message-ID: <2BEFE70A-FCB1-4E37-96BC-B6D8E5252C0A@irx.jp> Hello. I using Varnish for Ajax based API server. with -T 127.0.0.1:10411 \ -f /var/home/yu/etc/000.vcl \ -s file,/home/cache,16g \ -p thread_pool_min=512 \ -p thread_pool_max=3096 \ -p thread_pools=16 \ -p listen_depth=3096 \ -p client_http11=on \ -p backend_http11=on \ -p cc_command="cc -fpic -shared -O3 -Wl,-x -o %o %s" \ -p lru_interval=21600 \ -P /var/run/varnishd.pid \ -u www \ -a 0.0.0.0:80 value of varnishstat "Objects sent with sendfile" is almost 0. 0 0.00 0.00 Objects sent with sendfile so, write() called. 37327571 335.20 157.86 Objects sent with write when do you use sendfile()? regards, ------- Yutaro Shimamura yu at irx.jp From perbu at linpro.no Thu May 22 15:46:34 2008 From: perbu at linpro.no (Per Buer) Date: Thu, 22 May 2008 17:46:34 +0200 Subject: sendfile() in varnish In-Reply-To: <2BEFE70A-FCB1-4E37-96BC-B6D8E5252C0A@irx.jp> References: <2BEFE70A-FCB1-4E37-96BC-B6D8E5252C0A@irx.jp> Message-ID: <4835955A.5010508@linpro.no> Yutaro Shimamura skrev: > when do you use sendfile()? Sendfile is not used on Linux or FreeBSD. If I recall correctly it's because there is no sure way of knowing when the sendfile call is done with the data. Per. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 252 bytes Desc: OpenPGP digital signature URL: From phk at phk.freebsd.dk Thu May 22 20:22:18 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Thu, 22 May 2008 20:22:18 +0000 Subject: sendfile() in varnish In-Reply-To: Your message of "Thu, 22 May 2008 20:24:42 +0900." <2BEFE70A-FCB1-4E37-96BC-B6D8E5252C0A@irx.jp> Message-ID: <959.1211487738@critter.freebsd.dk> In message <2BEFE70A-FCB1-4E37-96BC-B6D8E5252C0A at irx.jp>, Yutaro Shimamura writ es: >value of varnishstat "Objects sent with sendfile" is almost 0. > 0 0.00 0.00 Objects sent with sendfile > >so, write() called. > 37327571 335.20 157.86 Objects sent with write > >when do you use sendfile()? By default: never, there are problems with all sendfile() implementations that prevent them from being usable in Varnish. Only FreeBSD-Current and later FreeBSD-8 will support sendfile(). -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From phk at phk.freebsd.dk Thu May 22 20:25:34 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Thu, 22 May 2008 20:25:34 +0000 Subject: Mapping VCL variables to internal variables In-Reply-To: Your message of "Wed, 21 May 2008 12:57:14 CST." <20080521185714.GC6756@charlescurley.com> Message-ID: <978.1211487934@critter.freebsd.dk> In message <20080521185714.GC6756 at charlescurley.com>, Charles Curley writes: >I'd like to set a variable in VCL, and then use it in a modification >of Varnish. Having added to req.hash with something like: > >sub vcl_hash >{ >if (req.http.Accept-Encoding ~ "gzip") { >set req.hash +=3D "gzip"; >} else if (req.http.Accept-Encoding ~ "deflate") { >set req.hash +=3D "deflate"; >} >} > >where do I find that in Varnish itself? See cache_center.c::cnt_lookup() [search for "VCL_hash_method(sp);"], and cache_hash.c::HSH_Copy(), HSH_Compare() and HSH_Lookup(). -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From chenxy.china at gmail.com Fri May 23 09:57:19 2008 From: chenxy.china at gmail.com (chen xiaoyong) Date: Fri, 23 May 2008 17:57:19 +0800 Subject: function vca_acct has a little bug ? Message-ID: <7ab169e90805230257u3aae7911le013b52572aef2ab@mail.gmail.com> cache_acceptor.c line 205 vca_acct(void *arg) { ...... i = accept(pfd[u].fd, addr, &l); ......... } in freebsd os , accept maybe block in some one socket if heritage.socks > 1 . because all pfd[u].fd not set nonblock -------------- next part -------------- An HTML attachment was scrubbed... URL: From phk at phk.freebsd.dk Tue May 27 07:15:25 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Tue, 27 May 2008 07:15:25 +0000 Subject: Objects for ESI includes incorrectly miss (Solved / Patch) In-Reply-To: Your message of "Tue, 20 May 2008 09:45:01 MST." <4833000D.3010204@endpoint.com> Message-ID: <51665.1211872525@critter.freebsd.dk> Can you try this patch instead: Index: cache_center.c =================================================================== --- cache_center.c (revision 2639) +++ cache_center.c (working copy) @@ -234,7 +234,6 @@ sp->xid = 0; SES_Charge(sp); sp->t_open = sp->t_end; - sp->t_req = NAN; sp->t_resp = NAN; WSL_Flush(sp->wrk, 0); @@ -242,6 +241,8 @@ if (sp->esis > 0) return (1); + sp->t_req = NAN; + if (sp->fd >= 0 && sp->doclose != NULL) vca_close_session(sp, sp->doclose); if (sp->fd < 0) { Poul-Henning In message <4833000D.3010204 at endpoint.com>, JT Justman writes: >This is a multi-part message in MIME format. >--------------020006040805080601000507 >Content-Type: text/plain; charset=ISO-8859-1; format=flowed >Content-Transfer-Encoding: 7bit > >JT Justman wrote: >> Hi, everyone. I've been doing my best to dig into the bug I opened: >> >> http://varnish.projects.linpro.no/ticket/240 >> >> Basically, every object is inserted, but the second and subsiquent >> objects always miss. > >I have found the cause for the bug, and a potential fix. > >In cache_hash.c function HSH_Lookup, there is a ttl check: > >if (o->ttl >= sp->t_req) > >This test passes on the first ESI and fails on the others. Debugging >shows that the ttl is correct, but t_req is NAN. It seems that t_req is >cleared, and needs to be reset somewhere in ESI or the step flow. > >My solution is to set sp->t_req = TIM_real() in ESI_Deliver. (See patch). > >Not being intimate with the program flow or timing, I don't know what >other implications this could have. Suggestions? > >JT > >--------------020006040805080601000507 >Content-Type: text/x-diff; > name="cache_vrt_esi-jt_2008-05-20.patch" >Content-Transfer-Encoding: 7bit >Content-Disposition: inline; > filename="cache_vrt_esi-jt_2008-05-20.patch" > >Index: cache_vrt_esi.c >=================================================================== >--- cache_vrt_esi.c (revision 2635) >+++ cache_vrt_esi.c (working copy) >@@ -773,6 +773,8 @@ > obj = sp->obj; > sp->obj = NULL; > *sp->http = *sp->http0; >+ if (isnan(sp->t_req)) >+ sp->t_req = TIM_real(); > /* XXX: reset sp->ws */ > http_SetH(sp->http, HTTP_HDR_URL, eb->include.b); > if (eb->host.b != NULL) { > >--------------020006040805080601000507 >Content-Type: text/plain; charset="us-ascii" >MIME-Version: 1.0 >Content-Transfer-Encoding: 7bit >Content-Disposition: inline > >_______________________________________________ >varnish-dev mailing list >varnish-dev at projects.linpro.no >http://projects.linpro.no/mailman/listinfo/varnish-dev > >--------------020006040805080601000507-- > -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence.