From des at linpro.no Tue Nov 7 12:39:33 2006 From: des at linpro.no (Dag-Erling =?iso-8859-1?Q?Sm=F8rgrav?=) Date: Tue, 07 Nov 2006 13:39:33 +0100 Subject: On 1.0.2 and the danger of feature-based releases Message-ID: I finally tagged 1.0.2, about three weeks later than I originally intended. The delay is mostly due to my indecision: am I really sure it's ready now, or should I check one more detail, close one more ticket? This is obviously a mug's game, so I suggest that in the future, point releases will be time-based - on or around the 20th of each month, if release-worthy changes have been made since the last release, a new one will be rolled. DES -- Dag-Erling Sm?rgrav Senior Software Developer Linpro AS - www.linpro.no From andrew at 2sheds.de Sun Nov 19 17:23:14 2006 From: andrew at 2sheds.de (Andrew Miehs) Date: Sun, 19 Nov 2006 18:23:14 +0100 Subject: Varnish as cache for image servers Message-ID: Dear List, I only yesterday discovered Varnish. After spending most of today reading and trawling the web on information concerning it, I am dying to try it out on our web farm tomorrow. My current problem is that I have 3 web servers, each with the same 32GB of static image files. I am currently using Lighttpd running on Linux 2.6 to deliver this static content to our customers. The web servers are 32bit machines with 4GB of RAM. (Yes 64bit would be better) :-). We currently have have approx 1000req/s per server (files about 30k each). An F5 Loadbalancer is splitting the load between these machines. # free total used free shared buffers cached Mem: 3930100 1917892 2012208 0 517032 971216 -/+ buffers/cache: 429644 3500456 Swap: 1028152 0 1028152 # The original idea was to let the linux kernel do all the caching with its filesystem cache. Unfortunately, it does not seem to want to use all the RAM available for this purpose. I was hoping that by making more effective use of the available RAM, I would be able to reduce IO Wait. I am aware that I can help do this with faster disks, bigger RAID, etc, but I was hoping to try and improve the situation by using the onboard RAM. Now for my questions: 1) Would you recommend using Varnish for this situation, or would you suggest something like Apache with Mod_cache_memory or Zeus with memory caching? 2) The ALPHA notes suggested that Varnish did not check for backend errors - has this been fixed in the 1.0.2 version? IE: Can I use it in a live situation and sleep at night? 3) Do you think running web server and Varnish on the same box would cause a problem? 4) How do you limit the size of Varnish? Is this limited by the size of the -s file? or size that you set with -s? I realise that this will be a max of 3GB with a 32bit kernel.... 5) Is there a suggested 'startup command line' that I should use in such a situation - like the one used for for your test run? and last but not least 6) As I am quite happy with the modified dates, etc that my web server is delivering, do I need to use VCL scripts? Thanks for getting through this mail! Cheers Andrew --- Andrew Miehs From andersb at vgnett.no Sun Nov 19 23:45:44 2006 From: andersb at vgnett.no (Anders Berg) Date: Mon, 20 Nov 2006 00:45:44 +0100 Subject: Varnish as cache for image servers In-Reply-To: References: Message-ID: <77FB7B41-6D6B-417A-A9F1-7DFA7CB76E45@vgnett.no> On Nov 19, 2006, at 18:23, Andrew Miehs wrote: > Dear List, > > I only yesterday discovered Varnish. After spending most of today > reading and trawling the web on information concerning it, I am > dying to try it out on our web farm tomorrow. Hi Andrew, and welcome to Varnish. > > My current problem is that I have 3 web servers, each with the same > 32GB of static image files. I am currently using Lighttpd running > on Linux 2.6 to deliver this static content to our customers. The > web servers are 32bit machines with 4GB of RAM. (Yes 64bit would be > better) :-). We currently have have approx 1000req/s per server > (files about 30k each). An F5 Loadbalancer is splitting the load > between these machines. > > # free > total used free shared buffers > cached > Mem: 3930100 1917892 2012208 0 517032 > 971216 > -/+ buffers/cache: 429644 3500456 > Swap: 1028152 0 1028152 > # > > The original idea was to let the linux kernel do all the caching > with its filesystem cache. Unfortunately, it does not seem to want > to use all the RAM available for this purpose. I was hoping that by > making more effective use of the available RAM, I would be able to > reduce IO Wait. > > I am aware that I can help do this with faster disks, bigger RAID, > etc, but I was hoping to try and improve the situation by using the > onboard RAM. > > Now for my questions: > I'll have to answer these questions based on my experience and maybe in a generic way. > 1) Would you recommend using Varnish for this situation, or would > you suggest something like Apache with Mod_cache_memory or Zeus > with memory caching? I would say that since your content is static I would in either go 2 ways based on the traffic pattern and what your data is "doing". For example, where I work we have a rig that serv flash ads, and since some tags for ads today make a uniqe URL for each serv (even if content is satic). Since caching works bad in situations like this, I have ended up using a FreeBSD with accel filtering and apache, and that works fine. Mind that is with a small "often used" dataset. But when it comes to "pure" static files I would say that Varnish in theory should be a perfect match for what you are trying to do (I say in theory since I have not used Varnish like that in a prod. evironment). The reason I say so is because Varnish uses some "state of the art" features in the OS (Even if they have been around a long time :) Like you say, let the OS do the stuff it knows how to do best.). Right now I use Varnish in a prod. environment with mixed content static/dynamic, and it works amazing. Read the attached PDF for a presentation and some stats. I wanna add stuff here about 32 bit VS 64 bit, and talk some more about you dataset instead of answering those questions later. Since the 32 bit is slowly dying out we did not put much effort into it and you will see "stupid" limits like max size on filesize (cache size). We mmap the a file and let the OS do its work, and a 32 bit machine would, as you say, have a 3 GB limit (I have not tried Varnish on a 32 bit machine, so I can't say where the limit is.). If that is "enuff" to make Varnish work at it's best for you depends on your dataset. Lets say you only "often" serv 3 GB of that static content, well, then that is not a big setback, since Varnish will slowly change data out with the default LRU algorithm (we have planned more algorithms, and Varnish is made modular so it's easy to change to one you'd like/make). If you can cache the static content for lets say a week and new static content is added at a slow pace and not 30 GB a day, not much disk IO will get done I guess. I know people wanna use their hardware to the end, but Varnish running on 64 bit shines so good, in many cases it's an immediate return of investment buying new machines and running Varnish. I changed out 12 machines running Squid with 2 64 bit machines with Varnish :) As I mentioned we wanna add more algorithms in the future, if there is need/requests for it, but right now we are gonna focues our work on fixing bugs, getting things to run rock-hard stable, and adding fetaures in VCL. We are also starting work with a spec for Varnish 2.0 (ESI/ESI'ish behaviour beeing the most popular request feature it seems), and getting the governing board down. Our list of things we wanna see in Varnish is long and uber-cool. Stay tuned. > 2) The ALPHA notes suggested that Varnish did not check for backend > errors - has this been fixed in the 1.0.2 version? IE: Can I use it > in a live situation and sleep at night? We cut some corners before launch, and VCL (thats where backend loadbalancing and some other "save-backend" features are implemented) did not get as feature-rich as we hoped. But I am happy to inform that the break we took after the 1.0 launch, is now over and features like backend-loadbalancing and checking are high on the list and "quick" to implement I am told. So again, stay tuned. Also note that I use Varnish in prod., but note that I am using FreeBSD STABLE and Varnish works great with a sometimes overloaded backend. > > 3) Do you think running web server and Varnish on the same box > would cause a problem? Have not tried it. But should work fine depending on RAM and loadtypes again. :) > 4) How do you limit the size of Varnish? Is this limited by the > size of the -s file? or size that you set with -s? I realise that > this will be a max of 3GB with a 32bit kernel.... -s uses or makes a file depending on if there was a file there in the first place. > 5) Is there a suggested 'startup command line' that I should use in > such a situation - like the one used for for your test run? > > and last but not least > > 6) As I am quite happy with the modified dates, etc that my web > server is delivering, do I need to use VCL scripts? No, the default VCL will honor your own settings in most cases. Hope that helps some. I guess I should add that Varnish was my initative and I provided the funding for Varnish 1.0 and therefore I would have to be biased :) YS Anders Berg > > > Thanks for getting through this mail! > > > Cheers > > > Andrew From andersb at vgnett.no Sun Nov 19 23:50:47 2006 From: andersb at vgnett.no (Anders Berg) Date: Mon, 20 Nov 2006 00:50:47 +0100 Subject: Varnish as cache for image servers In-Reply-To: <77FB7B41-6D6B-417A-A9F1-7DFA7CB76E45@vgnett.no> References: <77FB7B41-6D6B-417A-A9F1-7DFA7CB76E45@vgnett.no> Message-ID: <41622EC1-2667-4B14-8E5A-B7B5888EFA88@vgnett.no> And the slides :) Doh. Anders Berg -------------- next part -------------- A non-text attachment was scrubbed... Name: varnish_launch_sept_20_2006.pdf Type: application/pdf Size: 404931 bytes Desc: not available URL: -------------- next part -------------- On Nov 20, 2006, at 0:45, Anders Berg wrote: > > On Nov 19, 2006, at 18:23, Andrew Miehs wrote: > >> Dear List, >> >> I only yesterday discovered Varnish. After spending most of today >> reading and trawling the web on information concerning it, I am >> dying to try it out on our web farm tomorrow. > > Hi Andrew, and welcome to Varnish. > >> >> My current problem is that I have 3 web servers, each with the >> same 32GB of static image files. I am currently using Lighttpd >> running on Linux 2.6 to deliver this static content to our >> customers. The web servers are 32bit machines with 4GB of RAM. >> (Yes 64bit would be better) :-). We currently have have approx >> 1000req/s per server (files about 30k each). An F5 Loadbalancer is >> splitting the load between these machines. >> >> # free >> total used free shared buffers >> cached >> Mem: 3930100 1917892 2012208 0 517032 >> 971216 >> -/+ buffers/cache: 429644 3500456 >> Swap: 1028152 0 1028152 >> # >> >> The original idea was to let the linux kernel do all the caching >> with its filesystem cache. Unfortunately, it does not seem to want >> to use all the RAM available for this purpose. I was hoping that >> by making more effective use of the available RAM, I would be able >> to reduce IO Wait. >> >> I am aware that I can help do this with faster disks, bigger RAID, >> etc, but I was hoping to try and improve the situation by using >> the onboard RAM. >> >> Now for my questions: >> > > I'll have to answer these questions based on my experience and > maybe in a generic way. > >> 1) Would you recommend using Varnish for this situation, or would >> you suggest something like Apache with Mod_cache_memory or Zeus >> with memory caching? > > I would say that since your content is static I would in either go > 2 ways based on the traffic pattern and what your data is "doing". > For example, where I work we have a rig that serv flash ads, and > since some tags for ads today make a uniqe URL for each serv (even > if content is satic). Since caching works bad in situations like > this, I have ended up using a FreeBSD with accel filtering and > apache, and that works fine. Mind that is with a small "often used" > dataset. > But when it comes to "pure" static files I would say that Varnish > in theory should be a perfect match for what you are trying to do > (I say in theory since I have not used Varnish like that in a prod. > evironment). The reason I say so is because Varnish uses some > "state of the art" features in the OS (Even if they have been > around a long time :) Like you say, let the OS do the stuff it > knows how to do best.). Right now I use Varnish in a prod. > environment with mixed content static/dynamic, and it works > amazing. Read the attached PDF for a presentation and some stats. > > I wanna add stuff here about 32 bit VS 64 bit, and talk some more > about you dataset instead of answering those questions later. Since > the 32 bit is slowly dying out we did not put much effort into it > and you will see "stupid" limits like max size on filesize (cache > size). We mmap the a file and let the OS do its work, and a 32 bit > machine would, as you say, have a 3 GB limit (I have not tried > Varnish on a 32 bit machine, so I can't say where the limit is.). > If that is "enuff" to make Varnish work at it's best for you > depends on your dataset. Lets say you only "often" serv 3 GB of > that static content, well, then that is not a big setback, since > Varnish will slowly change data out with the default LRU algorithm > (we have planned more algorithms, and Varnish is made modular so > it's easy to change to one you'd like/make). If you can cache the > static content for lets say a week and new static content is added > at a slow pace and not 30 GB a day, not much disk IO will get done > I guess. I know people wanna use their hardware to the end, but > Varnish running on 64 bit shines so good, in many cases it's an > immediate return of investment buying new machines and running > Varnish. I changed out 12 machines running Squid with 2 64 bit > machines with Varnish :) > > As I mentioned we wanna add more algorithms in the future, if there > is need/requests for it, but right now we are gonna focues our work > on fixing bugs, getting things to run rock-hard stable, and adding > fetaures in VCL. We are also starting work with a spec for Varnish > 2.0 (ESI/ESI'ish behaviour beeing the most popular request feature > it seems), and getting the governing board down. Our list of things > we wanna see in Varnish is long and uber-cool. Stay tuned. > >> 2) The ALPHA notes suggested that Varnish did not check for >> backend errors - has this been fixed in the 1.0.2 version? IE: Can >> I use it in a live situation and sleep at night? > > We cut some corners before launch, and VCL (thats where backend > loadbalancing and some other "save-backend" features are > implemented) did not get as feature-rich as we hoped. But I am > happy to inform that the break we took after the 1.0 launch, is now > over and features like backend-loadbalancing and checking are high > on the list and "quick" to implement I am told. So again, stay > tuned. Also note that I use Varnish in prod., but note that I am > using FreeBSD STABLE and Varnish works great with a sometimes > overloaded backend. > >> >> 3) Do you think running web server and Varnish on the same box >> would cause a problem? > > Have not tried it. But should work fine depending on RAM and > loadtypes again. :) > >> 4) How do you limit the size of Varnish? Is this limited by the >> size of the -s file? or size that you set with -s? I realise that >> this will be a max of 3GB with a 32bit kernel.... > > -s uses or makes a file depending on if there was a file there in > the first place. > >> 5) Is there a suggested 'startup command line' that I should use >> in such a situation - like the one used for for your test run? >> >> and last but not least >> >> 6) As I am quite happy with the modified dates, etc that my web >> server is delivering, do I need to use VCL scripts? > > No, the default VCL will honor your own settings in most cases. > > > > Hope that helps some. > > I guess I should add that Varnish was my initative and I provided > the funding for Varnish 1.0 and therefore I would have to be biased :) > > > YS > Anders Berg > >> >> >> Thanks for getting through this mail! >> >> >> Cheers >> >> >> Andrew > > _______________________________________________ > varnish-dev mailing list > varnish-dev at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-dev > From netex at 163.com Mon Nov 20 03:26:48 2006 From: netex at 163.com (Jiang Hong) Date: Mon, 20 Nov 2006 11:26:48 +0800 Subject: varnishd dev is not as active as before? Message-ID: <20061120112500.94FA.NETEX@163.com> Hello List, Is varnishd project development still active (as before)? Some old problems not solved for months like #53/#55 and new ones as #70. Will these be solved after FreeBSD 6.2's release? :-)))) 1.0.2 is not quite stable in relatively high load (>1000req/s). Sorry for my bad expression, I don't want to be offensive. -- Jiang Hong From phk at phk.freebsd.dk Mon Nov 20 07:28:15 2006 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Mon, 20 Nov 2006 07:28:15 +0000 Subject: varnishd dev is not as active as before? In-Reply-To: Your message of "Mon, 20 Nov 2006 11:26:48 +0800." <20061120112500.94FA.NETEX@163.com> Message-ID: <3016.1164007695@critter.freebsd.dk> Hi Jiang, I have been trying to take a bit of vacation for the last couple of months, and therefore I have not done much on Varnish. I have spent a fair amount of time giving talks about Varnish, I just came home from Stockholm yesterday, and I also read the emails on the list, so I am not entirely out of touch. As far as I can tell, varnish V2 development will start in january and I do plan to get a couple of bug fixes in before that. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From netex at 163.com Mon Nov 20 11:14:18 2006 From: netex at 163.com (Jiang Hong) Date: Mon, 20 Nov 2006 19:14:18 +0800 Subject: varnishd dev is not as active as before? In-Reply-To: <3016.1164007695@critter.freebsd.dk> References: <20061120112500.94FA.NETEX@163.com> <3016.1164007695@critter.freebsd.dk> Message-ID: <20061120191004.669D.NETEX@163.com> Hi, Thanks for your inspiring reply. :) The problems are so annoying that I have to go back to Squid again. Do you recommend varnish on 32-bit systems? I have tons of small files (images), but 2GB should be enough form them. > Hi Jiang, > > I have been trying to take a bit of vacation for the last couple > of months, and therefore I have not done much on Varnish. > > I have spent a fair amount of time giving talks about Varnish, > I just came home from Stockholm yesterday, and I also read the > emails on the list, so I am not entirely out of touch. > > As far as I can tell, varnish V2 development will start in > january and I do plan to get a couple of bug fixes in before that. > > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > phk at FreeBSD.ORG | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetence. -- Jiang Hong