From anders at fupp.net Tue Jul 1 12:25:56 2008 From: anders at fupp.net (Anders Nordby) Date: Tue, 1 Jul 2008 14:25:56 +0200 Subject: Reloading default.vcl In-Reply-To: <64320.203.97.62.217.1214863372.squirrel@services.directender.co.nz> References: <3083.1214862841@critter.freebsd.dk> <64320.203.97.62.217.1214863372.squirrel@services.directender.co.nz> Message-ID: <20080701122556.GA17385@fupp.net> Hi, On Tue, Jul 01, 2008 at 10:02:52AM +1200, Darryl Dixon - Winterhouse Consulting wrote: >> Afterwards you can >> vcl.discard old-unique-name >> >> if you want to get rid of it. > It is pretty easy to automate this all with expect(1) as well, if an > 'automated' reload solution is required :) There's not much need for Except if you run varnishadm instead of using telnet. Also, often Varnish doesn't let you delete an old config. See http://varnish.projects.linpro.no/ticket/145. Bye, -- Anders. From tfheen at linpro.no Tue Jul 1 12:40:27 2008 From: tfheen at linpro.no (Tollef Fog Heen) Date: Tue, 01 Jul 2008 14:40:27 +0200 Subject: grace handling In-Reply-To: <4868DD0D.5070108@wiggy.net> (Wichert Akkerman's message of "Mon\, 30 Jun 2008 15\:18\:05 +0200") References: <4868DD0D.5070108@wiggy.net> Message-ID: <877ic5aibo.fsf@luxevop.linpro.no> * Wichert Akkerman | Is there any documentation on how req.grace and obj.grace are used? I | could not find any mention of grace handling in the manpages or the wiki. There's a commit, http://varnish.projects.linpro.no/changeset/2392 which explains a little bit about this. -- Tollef Fog Heen / Linpro AS t: 21 54 41 73 UNIX is user friendly, it's just picky about who its friends are From skye at F4.ca Tue Jul 1 18:16:15 2008 From: skye at F4.ca (Skye Poier Nott) Date: Tue, 1 Jul 2008 11:16:15 -0700 Subject: Strategy for large cache sets Message-ID: <11FDA9BC-9FA7-4F6A-BA49-F7EB3E3461E5@F4.ca> I want to deploy Varnish with very large cache sizes (200GB or more) for large, long lived file sets. Is it more efficient to use large swap or large mmap in this scenario? According to the FreeBSD lists, even 20GB of swap requires 200MB of kern.maxswzone just to keep track of it, so it doesn't seem like that will scale too well. Is one or the other method better for many small files vs less many big files? Thanks again... when I'm a Varnish expert I'll help the newbs :) Skye From phk at phk.freebsd.dk Tue Jul 1 18:24:47 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Tue, 01 Jul 2008 18:24:47 +0000 Subject: Strategy for large cache sets In-Reply-To: Your message of "Tue, 01 Jul 2008 11:16:15 MST." <11FDA9BC-9FA7-4F6A-BA49-F7EB3E3461E5@F4.ca> Message-ID: <2367.1214936687@critter.freebsd.dk> In message <11FDA9BC-9FA7-4F6A-BA49-F7EB3E3461E5 at F4.ca>, Skye Poier Nott writes : >I want to deploy Varnish with very large cache sizes (200GB or more) >for large, long lived file sets. Is it more efficient to use large >swap or large mmap in this scenario? We have no real-world experience with content of that size, so the answer is: we don't know. Off the bat, I would think files would be better, for exactly the reson you cite: the swap management is pretty thirsty after metadata. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From skye at F4.ca Tue Jul 1 19:13:44 2008 From: skye at F4.ca (Skye Poier Nott) Date: Tue, 1 Jul 2008 12:13:44 -0700 Subject: Strategy for large cache sets In-Reply-To: <2367.1214936687@critter.freebsd.dk> References: <2367.1214936687@critter.freebsd.dk> Message-ID: <5417D56E-8D74-49E1-8FAF-2A7EE3D3CF84@F4.ca> Thanks, that was my hunch. I'll let you know how it goes when I have some performance metrics. Skye On 1-Jul-08, at 11:24 AM, Poul-Henning Kamp wrote: > In message <11FDA9BC-9FA7-4F6A-BA49-F7EB3E3461E5 at F4.ca>, Skye Poier > Nott writes > : >> I want to deploy Varnish with very large cache sizes (200GB or more) >> for large, long lived file sets. Is it more efficient to use large >> swap or large mmap in this scenario? > > We have no real-world experience with content of that size, so > the answer is: we don't know. > > Off the bat, I would think files would be better, for exactly the > reson you cite: the swap management is pretty thirsty after metadata. > > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > phk at FreeBSD.ORG | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by > incompetence. > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc From phk at phk.freebsd.dk Tue Jul 1 19:36:49 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Tue, 01 Jul 2008 19:36:49 +0000 Subject: Strategy for large cache sets In-Reply-To: Your message of "Tue, 01 Jul 2008 12:13:44 MST." <5417D56E-8D74-49E1-8FAF-2A7EE3D3CF84@F4.ca> Message-ID: <11538.1214941009@critter.freebsd.dk> In message <5417D56E-8D74-49E1-8FAF-2A7EE3D3CF84 at F4.ca>, Skye Poier Nott writes: >Thanks, that was my hunch. I'll let you know how it goes when I have >some performance metrics. Input is very much appreciated. I can also recommend setting up Munin or similar to plot all the varnishstat and systat variables, that has proven to be quite helpful to figuring out what dynamics caused what problems. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From skye at f4.ca Tue Jul 1 19:56:19 2008 From: skye at f4.ca (Skye Poier Nott) Date: Tue, 1 Jul 2008 12:56:19 -0700 Subject: Trunk build errors Message-ID: <2B2F1223-7ACE-455F-BD54-713592917ACA@f4.ca> Weird, any thoughts? Or am I just asking for trouble with -trunk :) $ svn update (last night's) $ make clean && configure && make mkdir .libs gcc -DVARNISH_STATE_DIR=\"/usr/local/var/varnish\" -g -O2 -o .libs/ varnishd varnishd-cache_acceptor.o varnishd-cache_acceptor_epoll.o varnishd-cache_acceptor_poll.o varnishd-cache_acceptor_kqueue.o varnishd-cache_backend.o varnishd-cache_ban.o varnishd-cache_center.o varnishd-cache_cli.o varnishd-cache_dir_random.o varnishd- cache_dir_simple.o varnishd-cache_expire.o varnishd-cache_fetch.o varnishd-cache_hash.o varnishd-cache_http.o varnishd-cache_httpconn.o varnishd-cache_main.o varnishd-cache_panic.o varnishd-cache_pool.o varnishd-cache_pipe.o varnishd-cache_response.o varnishd- cache_session.o varnishd-cache_synthetic.o varnishd-cache_vary.o varnishd-cache_vcl.o varnishd-cache_vrt.o varnishd-cache_vrt_acl.o varnishd-cache_vrt_re.o varnishd-cache_vrt_esi.o varnishd-cache_ws.o varnishd-hash_simple_list.o varnishd-hash_classic.o varnishd- instance.o varnishd-mgt_child.o varnishd-mgt_cli.o varnishd- mgt_event.o varnishd-mgt_param.o varnishd-mgt_vcc.o varnishd-rfc2616.o varnishd-shmlog.o varnishd-stevedore.o varnishd-storage_file.o varnishd-storage_malloc.o varnishd-tcp.o varnishd-varnishd.o -Wl,-- export-dynamic ../../lib/libvarnish/.libs/libvarnish.so ../../lib/ libvarnishcompat/.libs/libvarnishcompat.so ../../lib/libvcl/.libs/ libvcl.so -lthr -lm -Wl,--rpath -Wl,/usr/local/lib varnishd-cache_http.o(.text+0x5a0): In function `http_GetHdr': ../../include/vct.h:47: undefined reference to `vct_typtab' varnishd-cache_http.o(.text+0x65e): In function `http_GetHdrField': ../../include/vct.h:49: undefined reference to `vct_typtab' varnishd-cache_http.o(.text+0x679):../../include/vct.h:49: undefined reference to `vct_typtab' varnishd-cache_http.o(.text+0x6a5):../../include/vct.h:49: undefined reference to `vct_typtab' varnishd-cache_http.o(.text+0x6cd):../../include/vct.h:49: undefined reference to `vct_typtab' varnishd-cache_http.o(.text+0x737):../../include/vct.h:47: more undefined references to `vct_typtab' follow *** Error code 1 $ uname -a FreeBSD XX 7.0-RELEASE FreeBSD 7.0-RELEASE #0: Sun Feb 24 10:35:36 UTC 2008 root at driscoll.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 $ $ gcc --version gcc (GCC) 4.2.1 20070719 [FreeBSD] Copyright (C) 2007 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. $ srcgrep vct_typtab ~/varnish/ varnish-cache include/vct.h:extern unsigned char vct_typtab[256]; include/vct.h: return (vct_typtab[x] & (y)); lib/libvarnish/vct.c:unsigned char vct_typtab[256] = { Today's svn barfs on configure: configure: creating ./config.status config.status: creating Makefile config.status: creating bin/Makefile config.status: creating bin/varnishadm/Makefile config.status: creating bin/varnishd/Makefile config.status: creating bin/varnishlog/Makefile config.status: creating bin/varnishhist/Makefile config.status: creating bin/varnishncsa/Makefile config.status: creating bin/varnishreplay/Makefile config.status: creating bin/varnishstat/Makefile config.status: error: cannot find input file: bin/varnishtest/ Makefile.in zsh: exit 1 configure Thanks, Skye From barry at automattic.com Wed Jul 2 15:59:07 2008 From: barry at automattic.com (Barry Abrahamson) Date: Wed, 2 Jul 2008 11:59:07 -0400 Subject: Strategy for large cache sets In-Reply-To: <11FDA9BC-9FA7-4F6A-BA49-F7EB3E3461E5@F4.ca> References: <11FDA9BC-9FA7-4F6A-BA49-F7EB3E3461E5@F4.ca> Message-ID: On Jul 1, 2008, at 2:16 PM, Skye Poier Nott wrote: > I want to deploy Varnish with very large cache sizes (200GB or more) > for large, long lived file sets. We are doing this also and running into performance problems. We have tried files and swap Running on Linux (Debian). When the cache starts to get large, we start to see huge load spikes (loads of 200+) caused by IO wait but no corresponding spikes in request rates or any of the varnish metrics (except threads running which I am pretty sure is a result of the load spike and not the cause). We are currently running 1.1.2 but have tried with trunk and the same thing happens. If you find anything useful in your testing, I would love you hear about it. -- Barry Abrahamson | Systems Wrangler | Automattic Blog: http://barry.wordpress.com From simon at darkmere.gen.nz Wed Jul 2 21:50:23 2008 From: simon at darkmere.gen.nz (Simon Lyall) Date: Thu, 3 Jul 2008 09:50:23 +1200 (NZST) Subject: Strategy for large cache sets In-Reply-To: References: <11FDA9BC-9FA7-4F6A-BA49-F7EB3E3461E5@F4.ca> Message-ID: On Wed, 2 Jul 2008, Barry Abrahamson wrote: > On Jul 1, 2008, at 2:16 PM, Skye Poier Nott wrote: >> I want to deploy Varnish with very large cache sizes (200GB or more) >> for large, long lived file sets. > > We are doing this also and running into performance problems. > about it. Personally I'm keeping with the "cache" bit in the title and only trying to keep a subset of may data in Varnish. Then again my usage patterns are more concentrated. I was wondering how large a cache people are managing to use without any problems? Maybe a survey: Version: Special Features/Config: Hits/Second: RAM: Total Cache Size: OS: Comments: BTW: I'm still on the testing stage for various non-technical reasons. -- Simon Lyall | Very Busy | Web: http://www.darkmere.gen.nz/ "To stay awake all night adds a day to your life" - Stilgar | eMT. From chris at futuredead.com Wed Jul 2 22:19:31 2008 From: chris at futuredead.com (Chris Ferry) Date: Wed, 2 Jul 2008 18:19:31 -0400 Subject: Varnish and hot linking Message-ID: I've set up varnish to handle hot linking, but I have an issue with varnish caching the "hotlink" image for the image request. Here's the relevant part of default.vcl: sub vcl_recv { # Add a unique header containing the client address remove req.http.X-Forwarded-For; set req.http.X-Forwarded-For = client.ip; if (req.request == "GET") { if ( !(req.http.referer ~ "[a-z0-9]+.domain.com| yahoo.|rssbandit.org|newsgator.com|google.|search?q=cache" ) && (req.http.host ~ "^(raw)?images.domain.") && (req.http.referer) ) { set req.url="http://images.domain.com/hotlink.gif"; pass; } lookup; } if (req.request == "PURGE") { purge_url(req.url); error 200 "OK!!"; } } The problem is this, a user hotlinks http://images.domain.com/puppy.jpeg and we return hotlink.gif. Now if a user legitimately loads puppy.jpeg he receives the cached copy of hotlink.gif in its place. I though by setting the req.url that the cached file is now hotlink.gif and we haven't even attempted to store puppy.jpeg in the cache. I need to legitimately serve puppy.jpeg even if it's being hotlinked from somewhere. Any help would be appreciated. Thanks Chris -------------- next part -------------- An HTML attachment was scrubbed... URL: From darryl.dixon at winterhouseconsulting.com Wed Jul 2 22:36:43 2008 From: darryl.dixon at winterhouseconsulting.com (Darryl Dixon - Winterhouse Consulting) Date: Thu, 3 Jul 2008 10:36:43 +1200 (NZST) Subject: Varnish and hot linking In-Reply-To: References: Message-ID: <2624.203.97.62.217.1215038203.squirrel@services.directender.co.nz> > I've set up varnish to handle hot linking, but I have an issue with > varnish caching the "hotlink" image for the image request. > > Here's the relevant part of default.vcl: > > sub vcl_recv { > > # Add a unique header containing the client address > remove req.http.X-Forwarded-For; > set req.http.X-Forwarded-For = client.ip; > if (req.request == "GET") { > if ( !(req.http.referer ~ "[a-z0-9]+.domain.com| > yahoo.|rssbandit.org|newsgator.com|google.|search?q=cache" ) && > (req.http.host ~ "^(raw)?images.domain.") && (req.http.referer) ) { > set req.url="http://images.domain.com/hotlink.gif"; > pass; > } > lookup; > } > > if (req.request == "PURGE") { > purge_url(req.url); > error 200 "OK!!"; > } > } > > The problem is this, a user hotlinks http://images.domain.com/puppy.jpeg > and we return hotlink.gif. > Now if a user legitimately loads puppy.jpeg he receives the cached > copy of hotlink.gif in its place. > I though by setting the req.url that the cached file is now > hotlink.gif and we haven't even attempted to store puppy.jpeg in the > cache. > I need to legitimately serve puppy.jpeg even if it's being hotlinked > from somewhere. > Any help would be appreciated. Hi Chris, You need to add some conditions in vcl_fetch to prevent the 'hotlink' image from being cached on its way back from being fetched from the backend. In other words, calling 'pass' in vcl_recv ensures that the current request will get a fresh copy from the backend, but during the processing of that request, the fresh copy will be 'insert'-ed into the cache during vcl_fetch, and so subsequent requests which hit the 'lookup' in vcl_recv will see the previously cached copy. You need to perform a check in vcl_fetch very similar to the one in vcl_recv to prevent the 'pass'-ed response from being cached. Hope this helps, regards, Darryl Dixon Winterhouse Consulting Ltd http://www.winterhouseconsulting.com From anders at fupp.net Thu Jul 3 04:40:20 2008 From: anders at fupp.net (Anders Nordby) Date: Thu, 3 Jul 2008 06:40:20 +0200 Subject: Strategy for large cache sets In-Reply-To: <11FDA9BC-9FA7-4F6A-BA49-F7EB3E3461E5@F4.ca> References: <11FDA9BC-9FA7-4F6A-BA49-F7EB3E3461E5@F4.ca> Message-ID: <20080703044020.GA98462@fupp.net> Hi, On Tue, Jul 01, 2008 at 11:16:15AM -0700, Skye Poier Nott wrote: > I want to deploy Varnish with very large cache sizes (200GB or more) > for large, long lived file sets. Is it more efficient to use large > swap or large mmap in this scenario? > > According to the FreeBSD lists, even 20GB of swap requires 200MB of > kern.maxswzone just to keep track of it, so it doesn't seem like that > will scale too well. Is one or the other method better for many small > files vs less many big files? My experience with Varnish on FreeBSD with long lived (~1 week) large data sets tells me that using the file storage backend easily gives you 60-70 second hangs. The malloc backend works smoother. I've been using 256 MB maxswzone on a few servers with upto 80 GB of data in the swap and did not have any problems with maxswzone beeing too small. That said, I do get large peaks in number of threads and vm faults with peak/high traffic, which makes it difficult to scale further. I don't know if this is due to bottlenecks in the VM subsystem, Varnish or if I have too little RAM. But I hope to find out more about it. I suspect there is more work needed in this area to be done by the developers. PS: FreeBSD supports swap devices upto only 32 GB, so you may need to split your disks/volumes up in many partitions. Bye, -- Anders. From calle at korjus.se Thu Jul 3 06:21:58 2008 From: calle at korjus.se (Calle Korjus) Date: Thu, 3 Jul 2008 08:21:58 +0200 Subject: Strategy for large cache sets In-Reply-To: References: <11FDA9BC-9FA7-4F6A-BA49-F7EB3E3461E5@F4.ca> Message-ID: We are running varnish with a huge dataset (+5TB) on our backends, we've been trying a lot of different setups but we've also ran into the problem with load spikes with every configuration that cahches more than the amount of ram we have, both mmap and malloc with larger swap. Right now we are running trunk with tcmalloc and 55gb cachestore (we have 64gb ram in the frontends now) we have about 70% cachehits and it's very stable, but we're still looking for a way to use mmap or malloc+swap to be able to have a cachestore over 100gb. So I'm also very interested in your results. Regards Calle -----Original Message----- From: varnish-misc-bounces at projects.linpro.no [mailto:varnish-misc-bounces at projects.linpro.no] On Behalf Of Barry Abrahamson Sent: den 2 juli 2008 17:59 To: varnish-misc at projects.linpro.no Subject: Re: Strategy for large cache sets On Jul 1, 2008, at 2:16 PM, Skye Poier Nott wrote: > I want to deploy Varnish with very large cache sizes (200GB or more) > for large, long lived file sets. We are doing this also and running into performance problems. We have tried files and swap Running on Linux (Debian). When the cache starts to get large, we start to see huge load spikes (loads of 200+) caused by IO wait but no corresponding spikes in request rates or any of the varnish metrics (except threads running which I am pretty sure is a result of the load spike and not the cause). We are currently running 1.1.2 but have tried with trunk and the same thing happens. If you find anything useful in your testing, I would love you hear about it. -- Barry Abrahamson | Systems Wrangler | Automattic Blog: http://barry.wordpress.com _______________________________________________ varnish-misc mailing list varnish-misc at projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-misc From aaraines at gmail.com Thu Jul 3 15:45:18 2008 From: aaraines at gmail.com (Drew Raines) Date: Thu, 03 Jul 2008 10:45:18 -0500 Subject: Strategy for large cache sets References: <11FDA9BC-9FA7-4F6A-BA49-F7EB3E3461E5@F4.ca> Message-ID: Simon Lyall wrote: > I was wondering how large a cache people are managing to use without any > problems? Maybe a survey: Version: trunk-r2646 Config: file backend Traffic: 90 req/s avg, 1.5-2K peak RAM: 12GB, 1GB swap Cache Size: 36GB reserved, 24GB allocated OS: Linux 2.6.18-6-amd64 (Debian 4.0r3) This represents one of our image cache boxes, with a 99.6% hitrate, in a pool fronted with nginx. Our cache files reside on a SAN where we can get around 500MB/s over iSCSI. We went that route because we could not get Linux to swap efficiently enough to make malloc feasible. Except for the occasional flood of connections which will cause varnishd to consume more RSS than we've allotted, it's been extremely stable and performant. -Drew From ottolski at web.de Thu Jul 3 16:25:30 2008 From: ottolski at web.de (Sascha Ottolski) Date: Thu, 3 Jul 2008 18:25:30 +0200 Subject: Strategy for large cache sets In-Reply-To: <11FDA9BC-9FA7-4F6A-BA49-F7EB3E3461E5@F4.ca> References: <11FDA9BC-9FA7-4F6A-BA49-F7EB3E3461E5@F4.ca> Message-ID: <200807031825.30967.ottolski@web.de> Am Dienstag 01 Juli 2008 20:16:15 schrieb Skye Poier Nott: > I want to deploy Varnish with very large cache sizes (200GB or more) > for large, long lived file sets. Is it more efficient to use large > swap or large mmap in this scenario? > > According to the FreeBSD lists, even 20GB of swap requires 200MB of > kern.maxswzone just to keep track of it, so it doesn't seem like that > will scale too well. Is one or the other method better for many > small files vs less many big files? > > Thanks again... when I'm a Varnish expert I'll help the newbs :) > > Skye I'm administrating three varnish instances for static images; cache file is setup as 517 GB (pre allocated with "dd"); currently, after 35 days uptime, it's filled more than half: 305543155712 bytes allocated 248759787520 bytes free It's running on Debian etch, 2x dual-core amd opteron, 32 GB RAM, 30 GB swap: Mem: 32969244k total, 32870944k used, 98300k free, 120672k buffers Swap: 29045480k total, 6096612k used, 22948868k free, 25752752k cached the varnish process looks like this in "top": PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ P COMMAND 3221 username 15 0 522g 27g 21g S 1 87.0 539:33.89 2 varnishd hitrate is 98% for recent short term, overall average is 95%. load of the machines is between 1 and 2 on medium traffic times, and goes up to 3 only seldom. but, the request rate is may be relatively low, compared to what others reported on the list. I don't have hard numbers unfortunately, average is 80.49 according to varnishstat. At peaks the reate may be in the order of may be 400 req/sec for a single instance. We had issues with 1.1.2 crashing, but since running on trunk (r2640), everything runs smooth. response time according to a nagios http response check is between 0.5 - 1 seconds, almost never over 1 second, even at peak times. Hope it's useful for someone, let me know if you need more details. Cheers, Sascha From skye at f4.ca Thu Jul 3 21:23:41 2008 From: skye at f4.ca (Skye Poier Nott) Date: Thu, 3 Jul 2008 14:23:41 -0700 Subject: Object leak? Message-ID: <58A9D074-0C43-4E2F-ACDC-C07ADB13B554@f4.ca> I installed Munin to start keeping track of various Varnish stat and in my test configuration which keeps requesting the same ~4,000 small objects over and over, I notice that the Varnish object count keeps climbing steadily, at a rate of about 5,000 more objects every 10 hours. VM alloc stays constant. How can I debug this issue? (using -trunk) Maybe its just a varnishstat bug... Thanks, Skye From darryl.dixon at winterhouseconsulting.com Sun Jul 6 23:13:00 2008 From: darryl.dixon at winterhouseconsulting.com (Darryl Dixon - Winterhouse Consulting) Date: Mon, 7 Jul 2008 11:13:00 +1200 (NZST) Subject: Strategy for large cache sets In-Reply-To: References: <11FDA9BC-9FA7-4F6A-BA49-F7EB3E3461E5@F4.ca> Message-ID: <12864.203.97.62.217.1215385980.squirrel@services.directender.co.nz> > Our cache files reside on a SAN where > we can get around 500MB/s over iSCSI. We went that route because we > could not get Linux to swap efficiently enough to make malloc > feasible. I know this is off-topic, but, I'm interested in the 500MB/s figure... Are you on 10gbit ethernet? Darryl Dixon Winterhouse Consulting Ltd http://www.winterhouseconsulting.com From aaraines at gmail.com Mon Jul 7 18:29:16 2008 From: aaraines at gmail.com (Drew Raines) Date: Mon, 07 Jul 2008 13:29:16 -0500 Subject: Strategy for large cache sets References: <11FDA9BC-9FA7-4F6A-BA49-F7EB3E3461E5@F4.ca> <12864.203.97.62.217.1215385980.squirrel@services.directender.co.nz> Message-ID: Darryl Dixon - Winterhouse Consulting wrote: >> Our cache files reside on a SAN where we can get around 500MB/s >> over iSCSI. We went that route because we could not get Linux to >> swap efficiently enough to make malloc feasible. > > I know this is off-topic, but, I'm interested in the 500MB/s > figure... Are you on 10gbit ethernet? No, the boxes only have GigE interfaces. I think our data center uses 10GigE switches from the rack to the SAN, however. -Drew From skye at F4.ca Tue Jul 8 00:06:28 2008 From: skye at F4.ca (Skye Poier Nott) Date: Mon, 7 Jul 2008 17:06:28 -0700 Subject: hash.purge Message-ID: <949EC45D-893F-490F-81D9-0254FB5ECC3F@F4.ca> Does hash.purge take a regex in the host section as well, or just the url section? ie is this valid (delete all jpg's for all *.bar.com hosts) hash.purge #.+\.bar\.com#\.jpg$ Thanks, Skye From skye at F4.ca Tue Jul 8 00:14:18 2008 From: skye at F4.ca (Skye Poier Nott) Date: Mon, 7 Jul 2008 17:14:18 -0700 Subject: hash.purge In-Reply-To: <949EC45D-893F-490F-81D9-0254FB5ECC3F@F4.ca> References: <949EC45D-893F-490F-81D9-0254FB5ECC3F@F4.ca> Message-ID: <5E3D7B68-A4C6-46E6-A16B-2049D71D78B1@F4.ca> Answering my own questions again... yes, regex in the host part does seem to work! Cool! However I noticed escaping dots doesn't seem to work as normal: hash.purge #w...6\.example\.com#$ 100 41 Syntax Error: Invalid backslash sequence hash.purge #w...6.example.com#$ 200 27 PURGE #w...6.example.com#$ Bug or feature? Skye On 7-Jul-08, at 5:06 PM, Skye Poier Nott wrote: > Does hash.purge take a regex in the host section as well, or just the > url section? > ie is this valid (delete all jpg's for all *.bar.com hosts) > > hash.purge #.+\.bar\.com#\.jpg$ > > Thanks, > Skye > > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc From phk at phk.freebsd.dk Tue Jul 8 05:34:30 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Tue, 08 Jul 2008 05:34:30 +0000 Subject: hash.purge In-Reply-To: Your message of "Mon, 07 Jul 2008 17:06:28 MST." <949EC45D-893F-490F-81D9-0254FB5ECC3F@F4.ca> Message-ID: <521.1215495270@critter.freebsd.dk> In message <949EC45D-893F-490F-81D9-0254FB5ECC3F at F4.ca>, Skye Poier Nott writes : >Does hash.purge take a regex in the host section as well, or just the >url section? It takes one regexp for the entire hash string, so you can match against anything you put into the hash. >ie is this valid (delete all jpg's for all *.bar.com hosts) > >hash.purge #.+\.bar\.com#\.jpg$ Yes, except you got the '#' in the wrong side of things: .+\.bar\.com#\.jpg# -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From phk at phk.freebsd.dk Tue Jul 8 06:08:42 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Tue, 08 Jul 2008 06:08:42 +0000 Subject: hash.purge In-Reply-To: Your message of "Mon, 07 Jul 2008 17:14:18 MST." <5E3D7B68-A4C6-46E6-A16B-2049D71D78B1@F4.ca> Message-ID: <685.1215497322@critter.freebsd.dk> In message <5E3D7B68-A4C6-46E6-A16B-2049D71D78B1 at F4.ca>, Skye Poier Nott writes : >Answering my own questions again... yes, regex in the host part does >seem to work! Cool! > >However I noticed escaping dots doesn't seem to work as normal: > >hash.purge #w...6\.example\.com#$ >100 41 >Syntax Error: Invalid backslash sequence You need to escape the '\' in the CLI: hash.purge #w...6\\.example\\.com#$ -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From barry at automattic.com Thu Jul 10 17:32:06 2008 From: barry at automattic.com (Barry Abrahamson) Date: Thu, 10 Jul 2008 13:32:06 -0400 Subject: Random director in varnish trunk doesn't work? Message-ID: <05ACBE9C-0B5A-4246-A402-5391A039E534@automattic.com> Is anyone successfully using the random director in varnish trunk (r2917)? I have something like this in my config: backend web1 { .host = "10.0.1.1"; .port = "8080"; } backend web2 { .host = "10.0.1.2"; .port = "8080"; } director default random { { .backend = web1; .weight = 1; } { .backend = web2; .weight = 1; } } sub vcl_recv { set req.backend = default; ..... Varnish won't serve any requests -- it looks like the child just dies. Strace shows: read(11, "Assert error in vdi_random_choos"..., 8191) = 102 write(2, "Child (30484) said Assert error "..., 84) = 84 Using a normal backend (not random director) works fine. Details: Debian Etch amd64, 2.6.18-6. I am not sure if it is an unhandled config/syntax error in my vcl or something else. -- Barry Abrahamson | Systems Wrangler | Automattic Blog: http://barry.wordpress.com From phk at phk.freebsd.dk Thu Jul 10 18:10:11 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Thu, 10 Jul 2008 18:10:11 +0000 Subject: Random director in varnish trunk doesn't work? In-Reply-To: Your message of "Thu, 10 Jul 2008 13:32:06 -0400." <05ACBE9C-0B5A-4246-A402-5391A039E534@automattic.com> Message-ID: <60064.1215713411@critter.freebsd.dk> In message <05ACBE9C-0B5A-4246-A402-5391A039E534 at automattic.com>, Barry Abraham son writes: >Is anyone successfully using the random director in varnish trunk >(r2917)? Try #2919, I have fixed an off-by one bug I introduced recently. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From barry at automattic.com Fri Jul 11 00:47:30 2008 From: barry at automattic.com (Barry Abrahamson) Date: Thu, 10 Jul 2008 20:47:30 -0400 Subject: Random director in varnish trunk doesn't work? In-Reply-To: <60064.1215713411@critter.freebsd.dk> References: <60064.1215713411@critter.freebsd.dk> Message-ID: <54FA1349-D0EC-42FB-96DE-BD45972D0CCB@automattic.com> On Jul 10, 2008, at 2:10 PM, Poul-Henning Kamp wrote: > In message <05ACBE9C-0B5A-4246-A402-5391A039E534 at automattic.com>, > Barry Abraham > son writes: >> Is anyone successfully using the random director in varnish trunk >> (r2917)? > > Try #2919, I have fixed an off-by one bug I introduced recently. Worked. Thanks. -- Barry Abrahamson | Systems Wrangler | Automattic Blog: http://barry.wordpress.com From armdan20 at gmail.com Mon Jul 14 11:25:39 2008 From: armdan20 at gmail.com (andan andan) Date: Mon, 14 Jul 2008 13:25:39 +0200 Subject: Meaning of obj.prefetch Message-ID: Hi. The obj.prefecth variable, What is it supposed to do? The default VCL set this variable to -30s, and varnishlog shows (30 seconds before that object expires): 0 ExpPick - 964711497 prefetch 0 VCL_call - prefetch 0 VCL_return - fetch 0 Debug - "Attempt Prefetch 964711497" Can someone please, explain what does it mean. Thanks in advance. BR. From phk at phk.freebsd.dk Mon Jul 14 11:26:07 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Mon, 14 Jul 2008 11:26:07 +0000 Subject: Meaning of obj.prefetch In-Reply-To: Your message of "Mon, 14 Jul 2008 13:25:39 +0200." Message-ID: <14732.1216034767@critter.freebsd.dk> In message , "anda n andan" writes: >Hi. > >The obj.prefecth variable, What is it supposed to do? Nothing yet. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From skye at F4.ca Mon Jul 14 19:02:18 2008 From: skye at F4.ca (Skye Poier Nott) Date: Mon, 14 Jul 2008 12:02:18 -0700 Subject: purge.hash in trunk Message-ID: <92E32C99-23A8-49B8-9B73-93710C33875F@F4.ca> Updated to r2945 today, is purge.hash changed or broken? This works: purge.url . purge.hash . But none of these have any effect: purge.hash .#.# 200 0 purge.hash #.#. 200 0 purge.hash $#$# 200 0 purge.hash #$#$ 200 0 (The last trunk rev I was using had #host#path but now it's host#path# ? It would be nice if 'help purge.hash' mentioned the current expected format) -Skye From phk at phk.freebsd.dk Mon Jul 14 19:33:35 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Mon, 14 Jul 2008 19:33:35 +0000 Subject: purge.hash in trunk In-Reply-To: Your message of "Mon, 14 Jul 2008 12:02:18 MST." <92E32C99-23A8-49B8-9B73-93710C33875F@F4.ca> Message-ID: <23657.1216064015@critter.freebsd.dk> In message <92E32C99-23A8-49B8-9B73-93710C33875F at F4.ca>, Skye Poier Nott writes : >Updated to r2945 today, is purge.hash changed or broken? > >This works: > >purge.url . >purge.hash . > >But none of these have any effect: > >purge.hash .#.# This would only purge single character urls on single character host names. Try: purge.hash .*#.*# for wider matching >(The last trunk rev I was using had #host#path but now it's >host#path# ? It would be nice if 'help purge.hash' mentioned the >current expected format) The expected format is what vcl_hash {} set up, and since the user can redefine that function, a constant help message would have good chances of being confusing. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From skye at F4.ca Mon Jul 14 21:51:13 2008 From: skye at F4.ca (Skye Poier Nott) Date: Mon, 14 Jul 2008 14:51:13 -0700 Subject: purge.hash in trunk In-Reply-To: <23657.1216064015@critter.freebsd.dk> References: <23657.1216064015@critter.freebsd.dk> Message-ID: <60AFD34C-2AD1-40CF-AA2A-C4D3DA10E63B@F4.ca> Oh I see, right, it matches against the whole concatenated hash string. Why is it that this purges all files from w0011.example.com purge.hash w0011.*#.* But this does nothing? purge.hash w0011.*#.*# Flipping the #'s around works: purge.hash #w0011.*#.* This also has no effect, is "+" not supported in the regex? purge.hash w0011.+#.+ purge.hash w0011.+#.+# purge.hash #w0011.+#.+ But none of these do anything: purge.hash #w0011.*#.*html purge.hash w0011.*#.*html purge.hash w0011.*#.*html# So I'm somewhat confused again. I haven't messed with vcl_hash at all. Thanks, Skye On 14-Jul-08, at 12:33 PM, Poul-Henning Kamp wrote: > In message <92E32C99-23A8-49B8-9B73-93710C33875F at F4.ca>, Skye Poier > Nott writes > : >> Updated to r2945 today, is purge.hash changed or broken? >> >> This works: >> >> purge.url . >> purge.hash . >> >> But none of these have any effect: >> >> purge.hash .#.# > > This would only purge single character urls on single character > host names. > > Try: > purge.hash .*#.*# > > for wider matching > >> (The last trunk rev I was using had #host#path but now it's >> host#path# ? It would be nice if 'help purge.hash' mentioned the >> current expected format) > > The expected format is what vcl_hash {} set up, and since the > user can redefine that function, a constant help message would > have good chances of being confusing. > > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > phk at FreeBSD.ORG | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by > incompetence. > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc From phk at phk.freebsd.dk Mon Jul 14 21:58:05 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Mon, 14 Jul 2008 21:58:05 +0000 Subject: purge.hash in trunk In-Reply-To: Your message of "Mon, 14 Jul 2008 14:51:13 MST." <60AFD34C-2AD1-40CF-AA2A-C4D3DA10E63B@F4.ca> Message-ID: <24326.1216072685@critter.freebsd.dk> In message <60AFD34C-2AD1-40CF-AA2A-C4D3DA10E63B at F4.ca>, Skye Poier Nott writes : >Oh I see, right, it matches against the whole concatenated hash string. Yes. >Why is it that this purges all files from w0011.example.com > > purge.hash w0011.*#.* Because the last ".*" matches nothing, and you havn't tied it to the front of the hash string with a "^" >But this does nothing? > > purge.hash w0011.*#.*# Because the default vcl_hash says: sub vcl_hash { set req.hash += req.url; if (req.http.host) { set req.hash += req.http.host; } else { set req.hash += server.ip; } hash; } You are trying to purge all urls that match "w0011" >Flipping the #'s around works: > > purge.hash #w0011.*#.* Yes, because now you match the '#' after the url, the hosts that are named w0011, the '#' after the hostname and nothing with the .* at the end. The correct way to purge everything from w0011 would be: purge.hash #w0011.*# -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From skye at F4.ca Mon Jul 14 22:10:28 2008 From: skye at F4.ca (Skye Poier Nott) Date: Mon, 14 Jul 2008 15:10:28 -0700 Subject: purge.hash in trunk In-Reply-To: <24326.1216072685@critter.freebsd.dk> References: <24326.1216072685@critter.freebsd.dk> Message-ID: <5B4C37D1-D3F7-4B42-8AC0-6A6D01F2ACF8@F4.ca> On 14-Jul-08, at 2:58 PM, Poul-Henning Kamp wrote: > Because the default vcl_hash says: > > sub vcl_hash { > set req.hash += req.url; > if (req.http.host) { > set req.hash += req.http.host; > } else { > set req.hash += server.ip; > } > hash; > } *blush* that would explain it, this whole time I've been trying to 'purge host#path#' when its supposed to be 'purge path#host#' Thanks, Skye From apokalyptik at apokalyptik.com Tue Jul 15 00:04:05 2008 From: apokalyptik at apokalyptik.com (Demitrious Kelly) Date: Mon, 14 Jul 2008 17:04:05 -0700 Subject: Different Caching Durations For Varnish and Client Message-ID: <487BE975.5010303@apokalyptik.com> Hey guys! Lets sat I have some dynamically generated content that I would like varnish to cache for, say a month. When this content is updated I'm invalidating the cache via a url.purge. I therefor want the world outside to check back in with varnish every 10 minutes. So in the response to varnish I have set the Expires to be 1 month from the request, and in the vcl I'm doing the following: sub vcl_deliver { remove resp.http.Expires; remove resp.http.Cache-Control; remove resp.http.Age; set resp.http.Cache-Control = "max-age=600, proxy-revalidate"; set resp.http.Age = "0"; deliver; } The problem is that this feels like the wrong way to be approaching the problem, and doesn't seem very "correct" as far as HTTP protocol observance is concerned. What would be the best way to implement such a scenario? Secondarily is there any nifty mechanism for generating an expires header with the VCL? I would imagine that the cleanest way to do this would be to drop the cache-control idea and go with an expires always set to 600 seconds from now on the vcl_deliver. Cheers! DK From ay at vg.no Tue Jul 15 08:49:09 2008 From: ay at vg.no (Audun Ytterdal) Date: Tue, 15 Jul 2008 10:49:09 +0200 Subject: Different Caching Durations For Varnish and Client In-Reply-To: <487BE975.5010303@apokalyptik.com> References: <487BE975.5010303@apokalyptik.com> Message-ID: <487C6485.2090109@vg.no> Demitrious Kelly wrote: > Hey guys! > > Lets sat I have some dynamically generated content that I would like > varnish to cache for, say a month. When this content is updated I'm > invalidating the cache via a url.purge. I therefor want the world > outside to check back in with varnish every 10 minutes. > > So in the response to varnish I have set the Expires to be 1 month from > the request, and in the vcl I'm doing the following: > > sub vcl_deliver { > remove resp.http.Expires; > remove resp.http.Cache-Control; > remove resp.http.Age; > set resp.http.Cache-Control = "max-age=600, proxy-revalidate"; > set resp.http.Age = "0"; > deliver; > } > > The problem is that this feels like the wrong way to be approaching the > problem, and doesn't seem very "correct" as far as HTTP protocol > observance is concerned. What would be the best way to implement such a > scenario? > > Secondarily is there any nifty mechanism for generating an expires > header with the VCL? I would imagine that the cleanest way to do this > would be to drop the cache-control idea and go with an expires always > set to 600 seconds from now on the vcl_deliver. > You could use smax-age instead of max-age. smax-age is shared cache max time. varnish uses smax-age and max-age. Only problem is that any other shared caches (such as a local squid in front of the client) also would use smax-age I would also like som sort of "max-age, only for varnish"-header. -- Audun ***************************************************************** Denne fotnoten bekrefter at denne e-postmeldingen ble skannet av MailSweeper og funnet fri for virus. ***************************************************************** This footnote confirms that this email message has been swept by MailSweeper for the presence of computer viruses. ***************************************************************** From kradziszewski at gmail.com Tue Jul 15 09:05:51 2008 From: kradziszewski at gmail.com (Kamil Radziszewski) Date: Tue, 15 Jul 2008 11:05:51 +0200 Subject: Backend failover Message-ID: <231965b00807150205h168a30ax409643f2406dc4a8@mail.gmail.com> Can I configure varnish backend so that it has few servers in failover mode? -------------- next part -------------- An HTML attachment was scrubbed... URL: From anders.vannman at vk.se Tue Jul 15 09:05:56 2008 From: anders.vannman at vk.se (Anders =?ISO-8859-1?Q?V=E4nnman?=) Date: Tue, 15 Jul 2008 11:05:56 +0200 Subject: varnish-misc Digest, Vol 28, Issue 8 Message-ID: Hej, Jag har semester till och med den 24 augusti. Maila till support at vk.se eller ring 090-15 10 20 om det g?ller br?dskande support?rende. Mvh Anders V -------------- next part -------------- An HTML attachment was scrubbed... URL: From anders.vannman at vk.se Tue Jul 15 10:00:06 2008 From: anders.vannman at vk.se (Anders =?ISO-8859-1?Q?V=E4nnman?=) Date: Tue, 15 Jul 2008 12:00:06 +0200 Subject: varnish-misc Digest, Vol 28, Issue 9 Message-ID: Hej, Jag har semester till och med den 24 augusti. Maila till support at vk.se eller ring 090-15 10 20 om det g?ller br?dskande support?rende. Mvh Anders V -------------- next part -------------- An HTML attachment was scrubbed... URL: From anders.vannman at vk.se Wed Jul 16 10:00:09 2008 From: anders.vannman at vk.se (Anders =?ISO-8859-1?Q?V=E4nnman?=) Date: Wed, 16 Jul 2008 12:00:09 +0200 Subject: varnish-misc Digest, Vol 28, Issue 10 Message-ID: Hej, Jag har semester till och med den 24 augusti. Maila till support at vk.se eller ring 090-15 10 20 om det g?ller br?dskande support?rende. Mvh Anders V -------------- next part -------------- An HTML attachment was scrubbed... URL: From apokalyptik at apokalyptik.com Wed Jul 16 17:55:38 2008 From: apokalyptik at apokalyptik.com (Demitrious Kelly) Date: Wed, 16 Jul 2008 10:55:38 -0700 Subject: Different Caching Durations For Varnish and Client In-Reply-To: <487C6485.2090109@vg.no> References: <487BE975.5010303@apokalyptik.com> <487C6485.2090109@vg.no> Message-ID: <487E361A.6070002@apokalyptik.com> We're actually thinking about the possibility of scrubbing Cache-Control, Age, and Expires on the varnish output, and using NGINX (already our load balancer in front of varnish) to programatically set an expires header for the client on the way out. Of course it would be much cleaner, and less human-error-prone if we could do something like this in vcl_deliver: set res.http.Expires = 300; Nobody would have to remember that one last piece of it was hanging out in the load balancer... DK Audun Ytterdal wrote: > Demitrious Kelly wrote: > >> Hey guys! >> >> Lets sat I have some dynamically generated content that I would like >> varnish to cache for, say a month. When this content is updated I'm >> invalidating the cache via a url.purge. I therefor want the world >> outside to check back in with varnish every 10 minutes. >> >> So in the response to varnish I have set the Expires to be 1 month from >> the request, and in the vcl I'm doing the following: >> >> sub vcl_deliver { >> remove resp.http.Expires; >> remove resp.http.Cache-Control; >> remove resp.http.Age; >> set resp.http.Cache-Control = "max-age=600, proxy-revalidate"; >> set resp.http.Age = "0"; >> deliver; >> } >> >> The problem is that this feels like the wrong way to be approaching the >> problem, and doesn't seem very "correct" as far as HTTP protocol >> observance is concerned. What would be the best way to implement such a >> scenario? >> >> Secondarily is there any nifty mechanism for generating an expires >> header with the VCL? I would imagine that the cleanest way to do this >> would be to drop the cache-control idea and go with an expires always >> set to 600 seconds from now on the vcl_deliver. >> >> > You could use smax-age instead of max-age. smax-age is shared cache max > time. > varnish uses smax-age and max-age. Only problem is that any other shared > caches (such as a local squid in front of the client) also would use > smax-age > > I would also like som sort of "max-age, only for varnish"-header. > > -- > Audun > > ***************************************************************** > Denne fotnoten bekrefter at denne e-postmeldingen ble > skannet av MailSweeper og funnet fri for virus. > ***************************************************************** > This footnote confirms that this email message has been > swept by MailSweeper for the presence of computer viruses. > ***************************************************************** > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From phk at phk.freebsd.dk Thu Jul 17 08:43:25 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Thu, 17 Jul 2008 08:43:25 +0000 Subject: Different Caching Durations For Varnish and Client In-Reply-To: Your message of "Tue, 15 Jul 2008 10:49:09 +0200." <487C6485.2090109@vg.no> Message-ID: <53567.1216284205@critter.freebsd.dk> In message <487C6485.2090109 at vg.no>, Audun Ytterdal writes: >Demitrious Kelly wrote: >> So in the response to varnish I have set the Expires to be 1 month from >> the request, and in the vcl I'm doing the following: >> >> sub vcl_deliver { >> remove resp.http.Expires; >> remove resp.http.Cache-Control; >> remove resp.http.Age; >> set resp.http.Cache-Control = "max-age=600, proxy-revalidate"; >> set resp.http.Age = "0"; >> deliver; >> } >> >I would also like som sort of "max-age, only for varnish"-header. Write up a proposal how it should work and lets look at it ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From wichert at wiggy.net Thu Jul 17 08:54:36 2008 From: wichert at wiggy.net (Wichert Akkerman) Date: Thu, 17 Jul 2008 10:54:36 +0200 Subject: Different Caching Durations For Varnish and Client In-Reply-To: <53567.1216284205@critter.freebsd.dk> References: <487C6485.2090109@vg.no> <53567.1216284205@critter.freebsd.dk> Message-ID: <20080717085436.GB17944@wiggy.net> Previously Poul-Henning Kamp wrote: > In message <487C6485.2090109 at vg.no>, Audun Ytterdal writes: > >Demitrious Kelly wrote: > > >> So in the response to varnish I have set the Expires to be 1 month from > >> the request, and in the vcl I'm doing the following: > >> > >> sub vcl_deliver { > >> remove resp.http.Expires; > >> remove resp.http.Cache-Control; > >> remove resp.http.Age; > >> set resp.http.Cache-Control = "max-age=600, proxy-revalidate"; > >> set resp.http.Age = "0"; > >> deliver; > >> } > >> > > > >I would also like som sort of "max-age, only for varnish"-header. > > Write up a proposal how it should work and lets look at it ? Instead of 'only for varnish' a 'only for first proxy in path' would be more generic and more explicit about the intention. Wichert. -- Wichert Akkerman It is simple to make things. http://www.wiggy.net/ It is hard to make things simple. From anders.vannman at vk.se Thu Jul 17 10:00:09 2008 From: anders.vannman at vk.se (Anders =?ISO-8859-1?Q?V=E4nnman?=) Date: Thu, 17 Jul 2008 12:00:09 +0200 Subject: varnish-misc Digest, Vol 28, Issue 11 Message-ID: Hej, Jag har semester till och med den 24 augusti. Maila till support at vk.se eller ring 090-15 10 20 om det g?ller br?dskande support?rende. Mvh Anders V -------------- next part -------------- An HTML attachment was scrubbed... URL: From havard.futseter at met.no Thu Jul 17 10:31:06 2008 From: havard.futseter at met.no (=?ISO-8859-1?Q?H=E5vard_Futs=E6ter?=) Date: Thu, 17 Jul 2008 12:31:06 +0200 Subject: expire/purge race condition fixed? Message-ID: <487F1F6A.3070508@met.no> Hi. I was wondering if this race condition will be fixed in the 2.0 release ? - H?vard Futs?ter From phk at phk.freebsd.dk Thu Jul 17 10:40:49 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Thu, 17 Jul 2008 10:40:49 +0000 Subject: expire/purge race condition fixed? In-Reply-To: Your message of "Thu, 17 Jul 2008 12:31:06 +0200." <487F1F6A.3070508@met.no> Message-ID: <69762.1216291249@critter.freebsd.dk> In message <487F1F6A.3070508 at met.no>, =?ISO-8859-1?Q?H=E5vard_Futs=E6ter?= writ es: >Hi. I was wondering if this race condition will be fixed in the 2.0 = Uhm, which race condition ? I'm not aware of any... -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From havard.futseter at met.no Thu Jul 17 10:57:35 2008 From: havard.futseter at met.no (=?ISO-8859-1?Q?H=E5vard_Futs=E6ter?=) Date: Thu, 17 Jul 2008 12:57:35 +0200 Subject: expire/purge race condition fixed? In-Reply-To: <69762.1216291249@critter.freebsd.dk> References: <69762.1216291249@critter.freebsd.dk> Message-ID: <487F259F.7080207@met.no> Poul-Henning Kamp wrote: > In message <487F1F6A.3070508 at met.no>, =?ISO-8859-1?Q?H=E5vard_Futs=E6ter?= writ > es: > >> Hi. I was wondering if this race condition will be fixed in the 2.0 = > > Uhm, which race condition ? I'm not aware of any... > We are using 1.0.4, so this might have been fixed in the latest releas. I have not seen any mention of it though. The problem is this: Varnish receives a request that triggers an url purge regexp and the object is consequently banned. If the same object has already expired, but not yet been deleted, the child process restarts itself. I don't the inner workings in play here, but this is the chain of events I get from the logs at least. Please let me know if I have misunderstood something here. - H?vard From phk at phk.freebsd.dk Thu Jul 17 11:21:10 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Thu, 17 Jul 2008 11:21:10 +0000 Subject: expire/purge race condition fixed? In-Reply-To: Your message of "Thu, 17 Jul 2008 12:57:35 +0200." <487F259F.7080207@met.no> Message-ID: <69903.1216293670@critter.freebsd.dk> In message <487F259F.7080207 at met.no>, =?ISO-8859-1?Q?H=E5vard_Futs=E6ter?= writ es: >We are using 1.0.4, so this might have been fixed in the latest releas. >I have not seen any mention of it though. > >The problem is this: >Varnish receives a request that triggers an url purge regexp and the >object is consequently banned. If the same object has already expired, >but not yet been deleted, the child process restarts itself. Indeed a lot has happened since 1.0.4, but I should probably add this very scenario to the test-cases. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From kontakt at web30.ch Thu Jul 17 11:50:14 2008 From: kontakt at web30.ch (Simon Kammerer) Date: Thu, 17 Jul 2008 13:50:14 +0200 Subject: searchable mailing list archive? Message-ID: <487F31F6.6090700@web30.ch> Hi! Is somewhere a searchable mailinglist archive? Simon From phk at phk.freebsd.dk Thu Jul 17 14:28:16 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Thu, 17 Jul 2008 14:28:16 +0000 Subject: searchable mailing list archive? In-Reply-To: Your message of "Thu, 17 Jul 2008 13:50:14 +0200." <487F31F6.6090700@web30.ch> Message-ID: <70429.1216304896@critter.freebsd.dk> In message <487F31F6.6090700 at web30.ch>, Simon Kammerer writes: >Hi! > >Is somewhere a searchable mailinglist archive? http://varnish.projects.linpro.no/wiki/MailingLists I belive google has them indexed, so searching with what I look for site:projects.linpro.no should work. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From drupalnut at gmail.com Fri Jul 18 02:36:42 2008 From: drupalnut at gmail.com (Jack Tuhman) Date: Thu, 17 Jul 2008 22:36:42 -0400 Subject: issues with page refresh Message-ID: Hi list, I am setting up Varnish to protect a drupal site. My vcl.conf file is at the bottom of this message: I am having 2 issues: 1) if a user presses refresh or F5, they do not see any images, if they click on a link, they see an image. 2) If I wanted Varnish to cache the php pages (right now I pass them) but clear its cache every 30 seconds, is there an easy way to do this? Also, does anyone have any thing they think I should add to the vcl.conf file, coming from squid, it seems very simple. Thanks for the help, Jack backend default { set backend.host = "10.10.0.10"; set backend.port = "81"; } sub vcl_recv { if (req.request == "POST") { pipe; } if (req.url ~ "\.(ico|gif|jpeg|jpg|png|swf|css|htc|js|bz2|gz|zip|xls|ppt|pdf|doc|rar)") { lookup; } pass; } sub vcl_fetch { # force minimum ttl of 180 seconds if (obj.ttl < 180s) { set obj.ttl = 180s; } } -------------- next part -------------- An HTML attachment was scrubbed... URL: From anders.vannman at vk.se Fri Jul 18 10:00:07 2008 From: anders.vannman at vk.se (Anders =?ISO-8859-1?Q?V=E4nnman?=) Date: Fri, 18 Jul 2008 12:00:07 +0200 Subject: varnish-misc Digest, Vol 28, Issue 12 Message-ID: Hej, Jag har semester till och med den 24 augusti. Maila till support at vk.se eller ring 090-15 10 20 om det g?ller br?dskande support?rende. Mvh Anders V -------------- next part -------------- An HTML attachment was scrubbed... URL: From anders.vannman at vk.se Sat Jul 19 10:00:06 2008 From: anders.vannman at vk.se (Anders =?ISO-8859-1?Q?V=E4nnman?=) Date: Sat, 19 Jul 2008 12:00:06 +0200 Subject: varnish-misc Digest, Vol 28, Issue 13 Message-ID: Hej, Jag har semester till och med den 24 augusti. Maila till support at vk.se eller ring 090-15 10 20 om det g?ller br?dskande support?rende. Mvh Anders V -------------- next part -------------- An HTML attachment was scrubbed... URL: From drupalnut at gmail.com Sun Jul 20 03:08:01 2008 From: drupalnut at gmail.com (Jack Tuhman) Date: Sat, 19 Jul 2008 23:08:01 -0400 Subject: Log Question Message-ID: Is there a way to get varnish to output (to a file) apache like logs. I looked at: http://linux.die.net/man/1/varnishlog I ran varnishlog -w /path/to/filename -D I am unable to tail the file as it has charactores in it that cause my console to beep and display marks. If I run varnishlog without the args, I get way more info that I want. I would be very happy with src_ip path_requested I would be even happier if it could tell me if it was a hit or miss. Thanks, Jack -------------- next part -------------- An HTML attachment was scrubbed... URL: From anders.vannman at vk.se Sun Jul 20 10:00:05 2008 From: anders.vannman at vk.se (Anders =?ISO-8859-1?Q?V=E4nnman?=) Date: Sun, 20 Jul 2008 12:00:05 +0200 Subject: varnish-misc Digest, Vol 28, Issue 14 Message-ID: Hej, Jag har semester till och med den 24 augusti. Maila till support at vk.se eller ring 090-15 10 20 om det g?ller br?dskande support?rende. Mvh Anders V -------------- next part -------------- An HTML attachment was scrubbed... URL: From phk at phk.freebsd.dk Sun Jul 20 13:45:04 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Sun, 20 Jul 2008 13:45:04 +0000 Subject: Log Question In-Reply-To: Your message of "Sat, 19 Jul 2008 23:08:01 -0400." Message-ID: <8977.1216561504@critter.freebsd.dk> In message , "Jack Tuhman" writes: >Is there a way to get varnish to output (to a file) apache like logs. Use the varnishncsa program. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From anders.vannman at vk.se Mon Jul 21 10:00:26 2008 From: anders.vannman at vk.se (Anders =?ISO-8859-1?Q?V=E4nnman?=) Date: Mon, 21 Jul 2008 12:00:26 +0200 Subject: varnish-misc Digest, Vol 28, Issue 15 Message-ID: Hej, Jag har semester till och med den 24 augusti. Maila till support at vk.se eller ring 090-15 10 20 om det g?ller br?dskande support?rende. Mvh Anders V -------------- next part -------------- An HTML attachment was scrubbed... URL: From ay at vg.no Mon Jul 21 13:35:30 2008 From: ay at vg.no (Audun Ytterdal) Date: Mon, 21 Jul 2008 15:35:30 +0200 Subject: Cookies, User-agents and Hashes Message-ID: <488490A2.8040201@vg.no> Lets say you have this site that when browsing with for example an iphone you are sendt to another backend with scaled down content for your eyes only. You users like this light version of you site, but some still wants to see the full version with all bells and whistles What I've done so far: in vcl_recv { if (req.url == "/" && req.http.User-Agent ~ "^Mozilla/5.0 \(iP(hone|od);" && !req.http.User-Agent ~ "somecookie" ) { set req.backend = someother; pass; } It's set to pass because otherwise it will on a hit lookup the same hash as the non-iphone users and visa-versa I could add User-Agent to my hash, but that is a bit overkill and would make my cache 10-30 times larger. A substring might help a bit here. Or I could maybe just copy all the if's in vcl_recv and add a extra number to the hash for each if. But then I would have to duplicate alot of lines. especially when adding cookies and stuff. Is this the correct way of doing things? It would be nice if you could set hash allready in vcl_recv.. -- Audun ***************************************************************** Denne fotnoten bekrefter at denne e-postmeldingen ble skannet av MailSweeper og funnet fri for virus. ***************************************************************** This footnote confirms that this email message has been swept by MailSweeper for the presence of computer viruses. ***************************************************************** From phk at phk.freebsd.dk Mon Jul 21 20:20:17 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Mon, 21 Jul 2008 20:20:17 +0000 Subject: Cookies, User-agents and Hashes In-Reply-To: Your message of "Mon, 21 Jul 2008 15:35:30 +0200." <488490A2.8040201@vg.no> Message-ID: <2111.1216671617@critter.freebsd.dk> In message <488490A2.8040201 at vg.no>, Audun Ytterdal writes: >Lets say you have this site that when browsing with for example an >iphone you are sendt to another backend with scaled down content for >your eyes only. You users like this light version of you site, but some >still wants to see the full version with all bells and whistles > >What I've done so far: > >in vcl_recv { > > if (req.url == "/" && req.http.User-Agent ~ "^Mozilla/5.0 >\(iP(hone|od);" && !req.http.User-Agent ~ "somecookie" ) { > set req.backend = someother; > pass; > } > >It's set to pass because otherwise it will on a hit lookup the same hash >as the non-iphone users and visa-versa > >I could add User-Agent to my hash, but that is a bit overkill and would >make my cache 10-30 times larger. A substring might help a bit here. How about this: sub vcl_recv { unset req.http.hash_input; // Don't confuse me if (req.url == "/" && req.http.User-Agent ~ "^Mozilla/5.0 \(iP(hone|od);" && !req.http.User-Agent ~ "somecookie" ) { set req.backend = someother; set req.http.hash_input = "iphone"; } } sub vcl_hash { set req.hash += req.http.hash_input; } -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From phk at phk.freebsd.dk Mon Jul 21 20:36:01 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Mon, 21 Jul 2008 20:36:01 +0000 Subject: issues with page refresh In-Reply-To: Your message of "Thu, 17 Jul 2008 22:36:42 -0400." Message-ID: <2223.1216672561@critter.freebsd.dk> In message , "Jack Tuhman" writes: >1) if a user presses refresh or F5, they do not see any images, if they >click on a link, they see an image. I will need more debugging info to find out what is going on. In particular a varnishlog output. >2) If I wanted Varnish to cache the php pages (right now I pass them) but >clear its cache every 30 seconds, is there an easy way to do this? In the vcl code you included, you set the TTL to 3 minutes minimum, you could do something like: if (obj.url ~ "\.php") { set obj.ttl = 30s; } >sub vcl_recv { > if (req.request == "POST") { > pipe; > } If you use a recent version of -trunk, you can & should use pass here. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From phk at phk.freebsd.dk Mon Jul 21 20:40:39 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Mon, 21 Jul 2008 20:40:39 +0000 Subject: expire/purge race condition fixed? In-Reply-To: Your message of "Thu, 17 Jul 2008 12:57:35 +0200." <487F259F.7080207@met.no> Message-ID: <5032.1216672839@critter.freebsd.dk> In message <487F259F.7080207 at met.no>, =?ISO-8859-1?Q?H=E5vard_Futs=E6ter?= writ es: >We are using 1.0.4, so this might have been fixed in the latest releas. >I have not seen any mention of it though. That is very old code by now, so I would urge an upgrade. >The problem is this: >Varnish receives a request that triggers an url purge regexp and the >object is consequently banned. If the same object has already expired, >but not yet been deleted, the child process restarts itself. I don't recall seeing anything like that in newer time. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From phk at phk.freebsd.dk Mon Jul 21 20:49:13 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Mon, 21 Jul 2008 20:49:13 +0000 Subject: Backend failover In-Reply-To: Your message of "Tue, 15 Jul 2008 11:05:51 +0200." <231965b00807150205h168a30ax409643f2406dc4a8@mail.gmail.com> Message-ID: <5116.1216673353@critter.freebsd.dk> In message <231965b00807150205h168a30ax409643f2406dc4a8 at mail.gmail.com>, "Kamil Radziszewski" writes: >Can I configure varnish backend so that it has few servers in failover mode? We have a "hackish" way to do it now with what we call "restarts", more comprehensive solutions is in the pipeline. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From olivier at qalpit.com Tue Jul 22 20:27:17 2008 From: olivier at qalpit.com (Olivier Beau) Date: Tue, 22 Jul 2008 22:27:17 +0200 Subject: Child not responding to ping, killing it Message-ID: <1216758437.488642a5413e2@www.qalpit.com> Hi all, I installed varnish (trunk) on 3 redhat5 64bit, (serving the same website) i see those logs on only one server : Jul 22 18:12:25 img-front1 varnishd[19818]: Child (27470) not responding to ping, killing it. Jul 22 18:12:25 img-front1 varnishd[19818]: Child (27470) not responding to ping, killing it. Jul 22 18:12:25 img-front1 varnishd[19818]: Child (27470) died signal=9 Jul 22 18:12:25 img-front1 varnishd[19818]: child (13971) Started Jul 22 18:12:25 img-front1 varnishd[19818]: Child (13971) said Closed fds: 4 5 6 9 10 12 13 Jul 22 18:12:25 img-front1 varnishd[19818]: Child (13971) said Child starts Jul 22 18:12:25 img-front1 varnishd[19818]: Child (13971) said managed to mmap 10737418240 bytes of 10737418240 Jul 22 18:12:25 img-front1 varnishd[19818]: Child (13971) said Ready Jul 22 19:21:05 img-front1 varnishd[19818]: Child (13971) not responding to ping, killing it. Jul 22 19:21:06 img-front1 varnishd[19818]: Child (13971) not responding to ping, killing it. Jul 22 19:21:06 img-front1 varnishd[19818]: Child (13971) died signal=9 Jul 22 19:21:06 img-front1 varnishd[19818]: child (22911) Started Jul 22 19:21:06 img-front1 varnishd[19818]: Child (22911) said Closed fds: 4 5 6 9 10 12 13 Jul 22 19:21:06 img-front1 varnishd[19818]: Child (22911) said Child starts Jul 22 19:21:06 img-front1 varnishd[19818]: Child (22911) said managed to mmap 10737418240 bytes of 10737418240 Jul 22 19:21:06 img-front1 varnishd[19818]: Child (22911) said Ready Jul 22 20:05:12 img-front1 varnishd[19818]: Child (22911) not responding to ping, killing it. Jul 22 20:05:13 img-front1 varnishd[19818]: Child (22911) not responding to ping, killing it. Jul 22 20:05:13 img-front1 varnishd[19818]: Child (22911) died signal=9 Jul 22 20:05:13 img-front1 varnishd[19818]: child (28767) Started Jul 22 20:05:13 img-front1 varnishd[19818]: Child (28767) said Closed fds: 4 5 6 9 10 12 13 Jul 22 20:05:13 img-front1 varnishd[19818]: Child (28767) said Child starts Jul 22 20:05:13 img-front1 varnishd[19818]: Child (28767) said managed to mmap 10737418240 bytes of 10737418240 Jul 22 20:05:13 img-front1 varnishd[19818]: Child (28767) said Ready everything is OK from on OS point of view (cpu,load,traffic,memory..) what could cause the child to hang ? should i try logging everything for debugging ? Olivier Beau From phk at phk.freebsd.dk Tue Jul 22 22:43:15 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Tue, 22 Jul 2008 22:43:15 +0000 Subject: experimental Varnish ACL revision (IPv6 able ?) Message-ID: <89383.1216766595@critter.freebsd.dk> I have reworked the varnish ACL code, but am not done testing. There's a patch here: http://phk.freebsd.dk/patch/varnish_acl.patch If any of you feel like beating it up, feel free to. The changes are: * Do DNS lookups at compile time instead of child load time. This makes restarts (much) faster (for long ACLS) and reports the DNS errors where & when they are needed. * Support IPv6 filtering. The ACL rules still have the same syntax, but the sematics have expanded to handle IPv6 also: acl foo { "foohost"; // Match, if the address is one of the // ipv4 or ipv6 addresses of "foohost" ! "foohost"; // Fail, if... "192.168.1.7" / 24; // Use mask for comparison: The '7' is // ignored "www.freebsd.org" / 24; // This will give compile error, because // the "www.freebsd.org" has both ipv4 // and ipv6 addresses, and using the same // mask for both kinds do not make sense. ( ... ); // Ignore this rule if DNS lookup fails. ( ! "idiot.net" ); // If we can resolve "idiot.net", then // return Failure to match, if we see them. } Please notice that DNS lookup happens *only* on VCL compilation, if a DNS record changes you need to recompile (ie: vcl.load or vcl.inline) your VCL code again, it is not enough to just switch vcl (vcl.use). (This is the same as with backend DNS lookups) -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From wichert at wiggy.net Tue Jul 22 22:45:58 2008 From: wichert at wiggy.net (Wichert Akkerman) Date: Wed, 23 Jul 2008 00:45:58 +0200 Subject: experimental Varnish ACL revision (IPv6 able ?) In-Reply-To: <89383.1216766595@critter.freebsd.dk> References: <89383.1216766595@critter.freebsd.dk> Message-ID: <48866326.2030006@wiggy.net> Poul-Henning Kamp wrote: > I have reworked the varnish ACL code, but am not done testing. > > There's a patch here: > > http://phk.freebsd.dk/patch/varnish_acl.patch > > If any of you feel like beating it up, feel free to. > > The changes are: > > * Do DNS lookups at compile time instead of child load time. This > makes restarts (much) faster (for long ACLS) and reports the > DNS errors where& when they are needed. > > * Support IPv6 filtering. > > The ACL rules still have the same syntax, but the sematics have > expanded to handle IPv6 also: > > acl foo { > > "foohost"; // Match, if the address is one of the > // ipv4 or ipv6 addresses of "foohost" > > ! "foohost"; // Fail, if... > > "192.168.1.7" / 24; // Use mask for comparison: The '7' is > // ignored > Can you also write 192.168.1/24 ? Wichert. -- Wichert Akkerman It is simple to make things. http://www.wiggy.net/ It is hard to make things simple. From phk at phk.freebsd.dk Tue Jul 22 23:58:09 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Tue, 22 Jul 2008 23:58:09 +0000 Subject: experimental Varnish ACL revision (IPv6 able ?) In-Reply-To: Your message of "Wed, 23 Jul 2008 00:45:58 +0200." <48866326.2030006@wiggy.net> Message-ID: <76820.1216771089@critter.freebsd.dk> In message <48866326.2030006 at wiggy.net>, Wichert Akkerman writes: >Can you also write 192.168.1/24 ? I use the operating systems getaddrinfo() function for the part before the '/' so that depends what happens in that case on your OS. I havn't tested that. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From armdan20 at gmail.com Wed Jul 23 11:23:48 2008 From: armdan20 at gmail.com (andan andan) Date: Wed, 23 Jul 2008 13:23:48 +0200 Subject: 503 status code Message-ID: Hi all. We have Varnish trunk rev2980 running in production. All works fine except some errors logged by varnishncsa with the 503 status code. All these errors are associated to pass rules. We have tried to increase .connectimeout without sucess. Apparently, the backend (a load balancer in front of several webservers), works fine, testing it, with stress tools, curl and others. Any ideas ? Thank You very much. Best regards. From lukas.loesche at bertelsmann.de Wed Jul 23 11:27:55 2008 From: lukas.loesche at bertelsmann.de (Loesche, Lukas, ZD-Scoyo) Date: Wed, 23 Jul 2008 13:27:55 +0200 Subject: Cache Purging via Mgmt Port vs. PURGE via HTTP Message-ID: Hi, I got a question regarding purging of cached data. As I learned there?s two ways to purge a file. Either using the management port url.purge foo, or using a VCL rule. My question is, when I use the VCL rule I transmit the PURGE /path/to/file and Host: www.foobar.tld HTTP Headers. When I purge using the management port I only transmit the url.purge ^/path/to/file$. What will happen when the same /path/to/file exists on more than one HTTP Host? Will it purge both versions, since I don?t explicitely set the HTTP Host to purge from when using the management port? Or will it purge none or only one version? If so which one? E.g. If varnish caches two files: http://www.foo.bar/path/to/file.jpg and http://user.foo.bar/path/to/file.jpg and I url.purge /path/to/file.jpg.... Which version will be purged? The one from www.foo.bar or the one from user.foo.bar or none? Thanks, -- Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From phk at phk.freebsd.dk Wed Jul 23 20:46:13 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Wed, 23 Jul 2008 20:46:13 +0000 Subject: Cache Purging via Mgmt Port vs. PURGE via HTTP In-Reply-To: Your message of "Wed, 23 Jul 2008 13:27:55 +0200." Message-ID: <1504.1216845973@critter.freebsd.dk> In message , "Loesche, Lukas, ZD-Scoyo" writes: >When I purge using the management >port I only transmit the url.purge ^/path/to/file$. What will happen when >the same /path/to/file exists on more than one HTTP Host? Will it purge both If you use purge.url yes. If you use purge.hash, you can specify a regexp for the full hash string including the Host: header (or any other fields you have put in it). -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From phk at phk.freebsd.dk Wed Jul 23 20:46:58 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Wed, 23 Jul 2008 20:46:58 +0000 Subject: 503 status code In-Reply-To: Your message of "Wed, 23 Jul 2008 13:23:48 +0200." Message-ID: <1513.1216846018@critter.freebsd.dk> In message , "andan andan" writes: >We have Varnish trunk rev2980 running in production. > >All works fine except some errors logged by varnishncsa with the 503 >status code. All these errors are associated to pass rules. > >We have tried to increase .connectimeout without sucess. > >Apparently, the backend (a load balancer in front of several >webservers), works fine, testing it, with stress tools, curl and >others. Try if you can, to capture a tcpdump ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From phk at phk.freebsd.dk Wed Jul 23 21:07:46 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Wed, 23 Jul 2008 21:07:46 +0000 Subject: Child not responding to ping, killing it In-Reply-To: Your message of "Tue, 22 Jul 2008 22:27:17 +0200." <1216758437.488642a5413e2@www.qalpit.com> Message-ID: <10355.1216847266@critter.freebsd.dk> In message <1216758437.488642a5413e2 at www.qalpit.com>, Olivier Beau writes: >Hi all, > >I installed varnish (trunk) on 3 redhat5 64bit, (serving the same website) > >i see those logs on only one server : > >Jul 22 18:12:25 img-front1 varnishd[19818]: Child (27470) not responding to >ping, killing it. >Jul 22 18:12:25 img-front1 varnishd[19818]: Child (27470) not responding to >ping, killing it. >Jul 22 18:12:25 img-front1 varnishd[19818]: Child (27470) died signal=9 Try increasing the cli_timeout and ping_interval parameters and see if that mitigaes the issue. If it does, then the child simply gets stuck for long periods of time. If it doesn't change things, then the child is stuck in an endless loop and getting a stack backtrace from a core file would be wonderful. Right now you'll not get a coredump on the signal 9, but give me a few minutes, and the coredump will be controled by the diag_bitmap paramter. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From nkinkade at creativecommons.org Thu Jul 24 00:46:26 2008 From: nkinkade at creativecommons.org (Nathan Kinkade) Date: Wed, 23 Jul 2008 17:46:26 -0700 Subject: Getting a core dump from Varnish 1.1.2 Message-ID: <1c5831080807231746w1dffe156k1b9d1a04dd6972b0@mail.gmail.com> We were having some problems with Varnish on an i386 build of Debian Etch. After reading the following ticket I decided to upgrade our servers to an amd64 build: http://varnish.projects.linpro.no/ticket/85. So, on two of the servers we are getting segmentation faults from Varnish every now and then (not too frequently). I rebuilt Varnish following the instructions for debugging at: http://varnish.projects.linpro.no/wiki/DebuggingVarnish, but Varnish doesn't seem to be outputting core dump files. I've got Varnish running in a screen session and when a child process segfaults, I see things like this in the terminal: Child said (2, 13865): <> Cache child died pid=13865 status=0xb Clean child Child cleaned start child pid 11024 Child said (2, 11024): < Message-ID: <58253.1216888950@critter.freebsd.dk> In message <1c5831080807231746w1dffe156k1b9d1a04dd6972b0 at mail.gmail.com>, "Nath an Kinkade" writes: >Child said (2, 13865): < Hi all, Is there a way in varnish to get the child's uptime ? (otherwise i'll get it with a 'ps'..) Olivier Beau From phk at phk.freebsd.dk Thu Jul 24 10:25:36 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Thu, 24 Jul 2008 10:25:36 +0000 Subject: varnish uptime In-Reply-To: Your message of "Thu, 24 Jul 2008 12:19:38 +0200." <1216894778.4888573a2409a@www.qalpit.com> Message-ID: <58664.1216895136@critter.freebsd.dk> In message <1216894778.4888573a2409a at www.qalpit.com>, Olivier Beau writes: > >Hi all, > >Is there a way in varnish to get the child's uptime ? >(otherwise i'll get it with a 'ps'..) It's at the top left corner of varnishstat output -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From olivier at qalpit.com Thu Jul 24 10:28:12 2008 From: olivier at qalpit.com (Olivier Beau) Date: Thu, 24 Jul 2008 12:28:12 +0200 Subject: varnish uptime In-Reply-To: <58664.1216895136@critter.freebsd.dk> References: <58664.1216895136@critter.freebsd.dk> Message-ID: <1216895292.4888593c82482@www.qalpit.com> > >Is there a way in varnish to get the child's uptime ? > >(otherwise i'll get it with a 'ps'..) > > It's at the top left corner of varnishstat output yes i saw, but i'ld like to get it in my monitoring script shell; and it's not available in varnishstat -1 Olivier From nkinkade at creativecommons.org Thu Jul 24 14:24:48 2008 From: nkinkade at creativecommons.org (Nathan Kinkade) Date: Thu, 24 Jul 2008 07:24:48 -0700 Subject: Getting a core dump from Varnish 1.1.2 In-Reply-To: <58253.1216888950@critter.freebsd.dk> References: <1c5831080807231746w1dffe156k1b9d1a04dd6972b0@mail.gmail.com> <58253.1216888950@critter.freebsd.dk> Message-ID: <1c5831080807240724x14e7d082o31c6b50202c1cd14@mail.gmail.com> 2008/7/24 Poul-Henning Kamp : > In message <1c5831080807231746w1dffe156k1b9d1a04dd6972b0 at mail.gmail.com>, "Nath > an Kinkade" writes: > >>Child said (2, 13865): < > I belive this message means that you run out of workspace somewhere, > but I can't remember the details for 1.1.2 any more. > > Try increasing the session workspace by a couple of kilobytes and > see if that cures it. > > Alternatively, try to update to a -trunk version, it's much better > than 1.1.2. Is there a particular revision number in trunk that is worth grabbing, or is it presumed that HEAD is stable enough for a fairly busy production machine? Nathan P.S. Is the etiquette on this list to reply to responder and the list, or just the list? From phk at phk.freebsd.dk Thu Jul 24 15:12:55 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Thu, 24 Jul 2008 15:12:55 +0000 Subject: Getting a core dump from Varnish 1.1.2 In-Reply-To: Your message of "Thu, 24 Jul 2008 07:24:48 MST." <1c5831080807240724x14e7d082o31c6b50202c1cd14@mail.gmail.com> Message-ID: <59868.1216912375@critter.freebsd.dk> In message <1c5831080807240724x14e7d082o31c6b50202c1cd14 at mail.gmail.com>, "Nath an Kinkade" writes: >> Alternatively, try to update to a -trunk version, it's much better >> than 1.1.2. > > >Is there a particular revision number in trunk that is worth grabbing, >or is it presumed that HEAD is stable enough for a fairly busy >production machine? At this time HEAD is very stable because we are trying to get closer to release 2.0 >P.S. Is the etiquette on this list to reply to responder and the list, >or just the list? "Be reasonable" is as much etiquette as we have managed to have yet :-) -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From nkinkade at creativecommons.org Thu Jul 24 18:08:37 2008 From: nkinkade at creativecommons.org (Nathan Kinkade) Date: Thu, 24 Jul 2008 11:08:37 -0700 Subject: Getting a core dump from Varnish 1.1.2 In-Reply-To: <59868.1216912375@critter.freebsd.dk> References: <1c5831080807240724x14e7d082o31c6b50202c1cd14@mail.gmail.com> <59868.1216912375@critter.freebsd.dk> Message-ID: <1c5831080807241108r916bfeao3efe3d065e322e8f@mail.gmail.com> 2008/7/24 Poul-Henning Kamp : > In message <1c5831080807240724x14e7d082o31c6b50202c1cd14 at mail.gmail.com>, "Nath > an Kinkade" writes: > >>> Alternatively, try to update to a -trunk version, it's much better >>> than 1.1.2. >> >> >>Is there a particular revision number in trunk that is worth grabbing, >>or is it presumed that HEAD is stable enough for a fairly busy >>production machine? > > At this time HEAD is very stable because we are trying to get closer to > release 2.0 Okay, I just checked out trunk, built it with debugging enabled, and it's running on one of our servers. If the segfault issue is still present, perhaps this time I'll get a core dump. Thanks, Nathan From kova70 at gmail.com Fri Jul 25 09:44:28 2008 From: kova70 at gmail.com (Raymond Hall) Date: Fri, 25 Jul 2008 04:44:28 -0500 Subject: varnishncsa and virtual hosts Message-ID: <53dee190807250244x464de7dr99d492e77d27f29d@mail.gmail.com> Hi there, I've run through pretty much all posts regarding separating varnishncsa's output per hosts, as requested in RxHeader. So far, no one has posted a reliable solution ... if only varnishlog accepted both -o and -w ... or varnishncsa RxHeader worked... :) No use redirecting varnishlog -o -c RxHeader '^Host: www.example.com' to a fifo and ther asking varnishncsa to read from it. Has anyone found a reliable solution to this? (ie. other than resorting to ugly cron-based workarounds....) I'm using varnish-2.0-tp2 TIA, Ray -- I'd rather have an inch of a dog than miles of pedigree. Dana Burnet From anders at fupp.net Fri Jul 25 13:18:04 2008 From: anders at fupp.net (Anders Nordby) Date: Fri, 25 Jul 2008 15:18:04 +0200 Subject: varnishncsa and virtual hosts In-Reply-To: <53dee190807250244x464de7dr99d492e77d27f29d@mail.gmail.com> References: <53dee190807250244x464de7dr99d492e77d27f29d@mail.gmail.com> Message-ID: <20080725131803.GA41345@fupp.net> Hi, I'm using an ugly cron-based work-aroud. ;-) Seems to work ok. But yes, I'd appreciate very much if we could write separate logs for different virtual hosts. Preferably, even without running one instance of varnishncsa for every log. On Fri, Jul 25, 2008 at 04:44:28AM -0500, Raymond Hall wrote: > I've run through pretty much all posts regarding separating > varnishncsa's output per hosts, as requested in RxHeader. > So far, no one has posted a reliable solution ... if only varnishlog > accepted both -o and -w ... or varnishncsa RxHeader worked... :) > No use redirecting varnishlog -o -c RxHeader '^Host: www.example.com' > to a fifo and ther asking varnishncsa to read from it. > > Has anyone found a reliable solution to this? (ie. other than > resorting to ugly cron-based workarounds....) > I'm using varnish-2.0-tp2 > > TIA, > Ray > > -- > I'd rather have an inch of a dog than miles of pedigree. > > Dana Burnet > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc -- Anders. From olivier at qalpit.com Fri Jul 25 15:01:45 2008 From: olivier at qalpit.com (Olivier Beau) Date: Fri, 25 Jul 2008 17:01:45 +0200 Subject: Child not responding to ping, killing it In-Reply-To: <10355.1216847266@critter.freebsd.dk> References: <10355.1216847266@critter.freebsd.dk> Message-ID: <1216998105.4889ead90d5f9@www.qalpit.com> Hello, i doubled cli_timeout's and ping_interval's defaults : # varnishadm -T localhost:6082 param.show | grep -e cli_timeout -e ping_interval cli_timeout 10 [seconds] ping_interval 6 [seconds] # Still had the problem twice today; should i try raising those params somemore ? Olivier Selon Poul-Henning Kamp : > In message <1216758437.488642a5413e2 at www.qalpit.com>, Olivier Beau writes: > >Hi all, > > > >I installed varnish (trunk) on 3 redhat5 64bit, (serving the same website) > > > >i see those logs on only one server : > > > >Jul 22 18:12:25 img-front1 varnishd[19818]: Child (27470) not responding to > >ping, killing it. > >Jul 22 18:12:25 img-front1 varnishd[19818]: Child (27470) not responding to > >ping, killing it. > >Jul 22 18:12:25 img-front1 varnishd[19818]: Child (27470) died signal=9 > > Try increasing the cli_timeout and ping_interval parameters and see if > that mitigaes the issue. > > If it does, then the child simply gets stuck for long periods of time. > > If it doesn't change things, then the child is stuck in an endless loop > and getting a stack backtrace from a core file would be wonderful. > > Right now you'll not get a coredump on the signal 9, but give me a > few minutes, and the coredump will be controled by the diag_bitmap > paramter. > > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > phk at FreeBSD.ORG | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetence. > -- Olivier Beau From phk at phk.freebsd.dk Fri Jul 25 15:06:05 2008 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Fri, 25 Jul 2008 15:06:05 +0000 Subject: Child not responding to ping, killing it In-Reply-To: Your message of "Fri, 25 Jul 2008 17:01:45 +0200." <1216998105.4889ead90d5f9@www.qalpit.com> Message-ID: <53134.1216998365@critter.freebsd.dk> In message <1216998105.4889ead90d5f9 at www.qalpit.com>, Olivier Beau writes: >i doubled cli_timeout's and ping_interval's defaults : > ># varnishadm -T localhost:6082 param.show | grep -e cli_timeout -e ping_interval >cli_timeout 10 [seconds] >ping_interval 6 [seconds] ># > >Still had the problem twice today; >should i try raising those params somemore ? If you update to trunk, it should coredump the cacher process instead, then try using gdb to get a backtrace so we can see what it is doing instead of answering pings... -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From olivier at qalpit.com Sun Jul 27 19:29:56 2008 From: olivier at qalpit.com (Olivier Beau) Date: Sun, 27 Jul 2008 21:29:56 +0200 Subject: High cpu when cache is full In-Reply-To: <53134.1216998365@critter.freebsd.dk> References: <53134.1216998365@critter.freebsd.dk> Message-ID: <1217186996.488cccb42a606@www.qalpit.com> Hi, I'm running trunk on rhel5 64 bit and have a 30G cache storage. When the cache went 100% full, varnish started to do some cleaning. -> is this a heavy operation or locking something up ? (the number of workers jump from 20 to 500 for a minute or so) The cache went down to 80% full in a couple minutes and still going down two hours later -> when will the cleaning stop ? We had relative high cpu load (went up to 5 load average instend of under 1 normaly..) I saw lots of logs like those in varnishlog: 104 VCL_call c discard 104 VCL_return c discard 0 ExpKill - 1316381337 LRU And this in varnishtop: 1614.08 VCL_retur discard 1611.10 VCL_call discard 722.34 VCL_retur deliver -> i guess the cleaning is cpu intensive; is there a way to control it or is it done automagicaly ? i'm attaching a graph of a couple metrics from varnishstat (i pretty sure the number of workers going up and down right now is caused by the cleaning/cpu load) here's some more info if needed : # varnishstat -1 | grep object n_object 1097629 . N struct object n_objecthead 1097631 . N struct objecthead n_expired 7335 . N expired objects n_lru_nuked 305646 . N LRU nuked objects n_lru_saved 0 . N LRU saved objects n_lru_moved 1330623 . N LRU moved objects n_deathrow 0 . N objects on deathrow # # ps auxw |grep varnishd root 15585 0.0 0.0 61124 724 pts/0 S+ 19:18 0:00 grep varnishd root 22799 0.0 0.0 105468 636 ? Ss Jul25 0:00 /usr/sbin/varnishd -a :80 -T localhost:6082 -f /etc/varnish/default.vcl -u varnish -g varnish -p lru_interval 3600 -h classic,150001 -s file,/data/varnish/tmp/varnish_storage.bin,30G -P /var/run/varnish.pid varnish 22802 1.2 79.0 33679432 3200060 ? Sl Jul25 46:35 /usr/sbin/varnishd -a :80 -T localhost:6082 -f /etc/varnish/default.vcl -u varnish -g varnish -p lru_interval 3600 -h classic,150001 -s file,/data/varnish/tmp/varnish_storage.bin,30G -P /var/run/varnish.pid # one hour after the previous snip above : # varnishstat -1 | grep object n_object 964181 . N struct object n_objecthead 964185 . N struct objecthead n_expired 7534 . N expired objects n_lru_nuked 480448 . N LRU nuked objects n_lru_saved 0 . N LRU saved objects n_lru_moved 1378748 . N LRU moved objects n_deathrow 0 . N objects on deathrow # Olivier -------------- next part -------------- A non-text attachment was scrubbed... Name: varnish.png Type: image/png Size: 38211 bytes Desc: not available URL: From tfheen at linpro.no Mon Jul 28 10:25:38 2008 From: tfheen at linpro.no (Tollef Fog Heen) Date: Mon, 28 Jul 2008 12:25:38 +0200 Subject: searchable mailing list archive? In-Reply-To: <487F31F6.6090700@web30.ch> (Simon Kammerer's message of "Thu, 17 Jul 2008 13:50:14 +0200") References: <487F31F6.6090700@web30.ch> Message-ID: <87myk21f19.fsf@linpro.no> ]] Simon Kammerer | Is somewhere a searchable mailinglist archive? Not directly (see Poul-Henning Kamp's followup). I have requested a search box to be set up for the mailing lists. -- Tollef Fog Heen / Linpro AS t: 21 54 41 73 UNIX is user friendly, it's just picky about who its friends are From Phuwadon at sanookonline.co.th Mon Jul 28 10:28:48 2008 From: Phuwadon at sanookonline.co.th (Phuwadon Danrahan) Date: Mon, 28 Jul 2008 17:28:48 +0700 Subject: Question about purge_url & purge_hash Message-ID: Hi all, I configured the Varnish 1.1.2-r2635 to allow purging from backoffice web servers. The first version of this concept is very easy, just send PURGE request for specific URL to Varnish servers. Configuraiton, VCL_RECV if (req.request == "PURGE") { if (client.ip ~ backoffice) { lookup; } } VCL_HIT if (req.request == "PURGE") { set obj.ttl = 0s; error 200 "Purged"; } This above configuration is working fine but the application owner would like to purge some URL that contains multiple paging in only one PURGE request. The example is http://myhost.com/path/to/index.php?id=123&page=1 http://myhost.com/path/to/index.php?id=123&page=2 http://myhost.com/path/to/index.php?id=123&page=3 Sometimes, the maximum page can be more than 1000. So, we think about purge_url(req.http) but it always return MISS when we input the URL with REGEX. What did we test the VCL configuration 1. use WFETCH to get content from varnish with the following request headers, GET /path/to/file.gif HTTP/1.0\r\n Host: myhost.com\r\n Accept-Encoding: gzip\r\n 2. use WFETCH to purge content from varnish with the following request headers, PURGE /path/to/file.gif HTTP/1.0\r\n Host: myhost.com\r\n Accept-Encoding: gzip\r\n We can purged the object in this step 3. We tried, PURGE \.gif$ HTTP/1.0\r\n Host: myhost.com\r\n Accept-Encoding: gzip\r\n But this one can not be purged which we require this kind of purging style (allow REGEX) Could anyone provide the valid use of purge_url or purge_hash in both vcl_recv and vcl_hit? and how can we input the REGEX url into purging request? Thank you. Phuwadon D. From cs at comlounge.net Mon Jul 28 10:38:37 2008 From: cs at comlounge.net (Christian Scholz) Date: Mon, 28 Jul 2008 12:38:37 +0200 Subject: searchable mailing list archive? In-Reply-To: <87myk21f19.fsf@linpro.no> References: <487F31F6.6090700@web30.ch> <87myk21f19.fsf@linpro.no> Message-ID: <488DA1AD.6040609@comlounge.net> Hi! What about: http://dir.gmane.org/gmane.comp.web.varnish.misc ? There is one directly on the page and another one in the externel Mail Archive Link further down as well. The same is true for the other lists: http://dir.gmane.org/search.php?match=varnish -- Christian Tollef Fog Heen wrote: > ]] Simon Kammerer > > | Is somewhere a searchable mailinglist archive? > > Not directly (see Poul-Henning Kamp's followup). I have requested a > search box to be set up for the mailing lists. > -- Christian Scholz Homepage: http://comlounge.net COM.lounge blog: http://mrtopf.de/blog Luetticher Strasse 10 Skype: HerrTopf 52064 Aachen Video Blog: http://comlounge.tv Tel: +49 241 400 730 0 E-Mail cs at comlounge.net Fax: +49 241 979 00 850 IRC: MrTopf, Tao_T neue Show: TOPFt?glich (http://mrtopf.de/blog/category/topf-taglich/) From shahid.azeez at gmail.com Mon Jul 28 11:05:11 2008 From: shahid.azeez at gmail.com (Shahid P.A) Date: Mon, 28 Jul 2008 16:35:11 +0530 Subject: Varnish Dought Message-ID: <8f7dd1b0807280405v650abdbcw85e08765f9b1bcd0@mail.gmail.com> Hai, I configured varnish along with Apache as web server(2 web servers) and Haproxy as load load balancer. Varnish is working when there is only one web server. The flow of rewuest is: Fist the request hits Varnish from varnish to Haproxy from here any one of the web server (web1 or web2) My varnish configuration is below backend default { # Our default backend, i.e. the web server # You can use more than 1. See docs. set backend.host = "0.0.0.0"; set backend.port = "9999"; } sub vcl_recv { if (req.request != "GET" && req.request != "HEAD") { pipe; } if (req.http.Expect) { pipe; } if (req.http.Authenticate || req.http.Cookie) { pass; } if (req.request == "GET") { lookup; } else { lookup; } } sub vcl_pipe { pipe; } sub vcl_pass { pass; } sub vcl_hash { set req.hash += req.url; set req.hash += req.http.host; hash; } sub vcl_hit { if (!obj.cacheable) { pass; } deliver; } sub vcl_miss { fetch; } sub vcl_fetch { if (!obj.valid) { error; } if (!obj.cacheable) { pass; } insert; } sub vcl_deliver { deliver; } sub vcl_timeout { discard; } sub vcl_discard { discard; } *Shahid* -------------- next part -------------- An HTML attachment was scrubbed... URL: