From corinl at gmx.de Tue Dec 4 10:10:07 2007 From: corinl at gmx.de (Corin Langosch) Date: Tue, 04 Dec 2007 11:10:07 +0100 Subject: save cache on exit? Message-ID: <4755277F.50602@gmx.de> Hello, I'm using vanish for some days now and really like it. My only problem is: how can I tell varnish to save the cached data to disk at shutdown? Currently it seems to start with an empty cache at each restart. I'm already using file-based storage, but it does not seem to store anything on disc. Regards, Corin From ssm at linpro.no Tue Dec 4 10:28:24 2007 From: ssm at linpro.no (Stig Sandbeck Mathisen) Date: Tue, 04 Dec 2007 11:28:24 +0100 Subject: save cache on exit? In-Reply-To: <4755277F.50602@gmx.de> (Corin Langosch's message of "Tue, 04 Dec 2007 11:10:07 +0100") References: <4755277F.50602@gmx.de> Message-ID: <7xd4tm7nzr.fsf@iostat.linpro.no> Corin Langosch writes: > How can I tell varnish to save the cached data to disk at shutdown? That is not possible. However, you can reconfigure Varnish at runtime, so there should be little reason to shutdown a varnish process once it is in production. -- Stig Sandbeck Mathisen, Linpro From duja at torlen.net Tue Dec 4 10:45:34 2007 From: duja at torlen.net (Erik) Date: Tue, 4 Dec 2007 11:45:34 +0100 Subject: Varnish and HTTPS Message-ID: <1oayyglld8xhdw1.041220071145@torlen.net> Hi, I know that Varnish doesn't handle https. But what if the request is in https? Would varnish pass it to the backend or just drop it? / Erik From ssm at linpro.no Tue Dec 4 11:56:25 2007 From: ssm at linpro.no (Stig Sandbeck Mathisen) Date: Tue, 04 Dec 2007 12:56:25 +0100 Subject: Varnish and HTTPS In-Reply-To: <1oayyglld8xhdw1.041220071145@torlen.net> (duja@torlen.net's message of "Tue, 4 Dec 2007 11:45:34 +0100") References: <1oayyglld8xhdw1.041220071145@torlen.net> Message-ID: <7xr6i265cm.fsf@iostat.linpro.no> Erik writes: > I know that Varnish doesn't handle https. But what if the request is > in https? Would varnish pass it to the backend or just drop it? It would not be a normal HTTP request, so Varnish would not try to handle it. You could place a https to http gateway between your users and varnish, so varnish can accelerate the http traffic. The pipeline would look like this: Duh Internet - https2http gw - varnish - http server -- Stig Sandbeck Mathisen, Linpro From duja at torlen.net Tue Dec 4 12:08:17 2007 From: duja at torlen.net (Erik) Date: Tue, 4 Dec 2007 13:08:17 +0100 Subject: Varnish and HTTPS Message-ID: Thanks for the info but I did just wondered if varnish pass it to the backend or if it drops the request. I did some tests and it seems like it passes the request to the backend. BTW, what program may I use to have https2http? Isn't https encrypted? / Erik Original Message ----------------------- Erik writes: > I know that Varnish doesn't handle https. But what if the request is > in https? Would varnish pass it to the backend or just drop it? It would not be a normal HTTP request, so Varnish would not try to handle it. You could place a https to http gateway between your users and varnish, so varnish can accelerate the http traffic. The pipeline would look like this: Duh Internet - https2http gw - varnish - http server -- Stig Sandbeck Mathisen, Linpro From dwetzel at nerim.net Tue Dec 4 12:59:36 2007 From: dwetzel at nerim.net (Damien Wetzel) Date: Tue, 4 Dec 2007 13:59:36 +0100 Subject: Varnish and HTTPS In-Reply-To: <7xr6i265cm.fsf@iostat.linpro.no> References: <1oayyglld8xhdw1.041220071145@torlen.net> <7xr6i265cm.fsf@iostat.linpro.no> Message-ID: <18261.20280.545915.719232@localhost.localdomain> Hello all, is https handling planned somewhere in the roadmap ? thanks, Stig Sandbeck Mathisen writes: > Erik writes: > > > I know that Varnish doesn't handle https. But what if the request is > > in https? Would varnish pass it to the backend or just drop it? > > It would not be a normal HTTP request, so Varnish would not try to > handle it. You could place a https to http gateway between your users > and varnish, so varnish can accelerate the http traffic. The pipeline > would look like this: > > Duh Internet - https2http gw - varnish - http server > > -- > Stig Sandbeck Mathisen, Linpro > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc From caleb.anthony at gmail.com Tue Dec 4 16:03:17 2007 From: caleb.anthony at gmail.com (Caleb Anthony) Date: Tue, 4 Dec 2007 09:03:17 -0700 Subject: Varnish and HTTPS In-Reply-To: References: Message-ID: <33afbbe70712040803m542bf70aq12addfe2449bd9cb@mail.gmail.com> You could use pound as an HTTPS front-end for varnish (or anything else too). http://www.apsis.ch/pound/ It works quite well. I believe that Slashdot uses it as thier HTTP front-end. On 12/4/07, Erik wrote: > Thanks for the info but I did just wondered if varnish pass it to the backend or if it drops the request. I did some tests and it seems like it passes the request to the backend. > > BTW, what program may I use to have https2http? Isn't https encrypted? > > / Erik > > Original Message ----------------------- > Erik writes: > > > I know that Varnish doesn't handle https. But what if the request is > > in https? Would varnish pass it to the backend or just drop it? > > It would not be a normal HTTP request, so Varnish would not try to > handle it. You could place a https to http gateway between your users > and varnish, so varnish can accelerate the http traffic. The pipeline > would look like this: > > Duh Internet - https2http gw - varnish - http server > > -- > Stig Sandbeck Mathisen, Linpro > > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc > From corinl at gmx.de Tue Dec 4 16:38:07 2007 From: corinl at gmx.de (Corin Langosch) Date: Tue, 04 Dec 2007 17:38:07 +0100 Subject: save cache on exit? In-Reply-To: <7xd4tm7nzr.fsf@iostat.linpro.no> References: <4755277F.50602@gmx.de> <7xd4tm7nzr.fsf@iostat.linpro.no> Message-ID: <4755826F.6080104@gmx.de> The problem is that varnishd segfaults from time to time. It then has to grep all GBs from the backends again. For a hot restart this might be good to have too (after server upgrade, server failure, ..). Stig Sandbeck Mathisen schrieb: > Corin Langosch writes: > > >> How can I tell varnish to save the cached data to disk at shutdown? >> > > That is not possible. > > However, you can reconfigure Varnish at runtime, so there should be > little reason to shutdown a varnish process once it is in production. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From varnish at lukem.org Wed Dec 5 03:33:07 2007 From: varnish at lukem.org (Luke Macpherson) Date: Wed, 5 Dec 2007 14:33:07 +1100 Subject: save cache on exit? In-Reply-To: <4755826F.6080104@gmx.de> References: <4755277F.50602@gmx.de> <7xd4tm7nzr.fsf@iostat.linpro.no> <4755826F.6080104@gmx.de> Message-ID: <29f77e8a0712041933g7749baf4m41906d6ef19d5cad@mail.gmail.com> If it has segfaulted, you'd have no guarantee it was in a consistent state, and so reusing the mmap-ed file would be a bad idea. On 12/5/07, Corin Langosch wrote: > > The problem is that varnishd segfaults from time to time. It then has to > grep all GBs from the backends again. For a hot restart this might be good > to have too (after server upgrade, server failure, ..). > > Stig Sandbeck Mathisen schrieb: > Corin Langosch writes: > > > > How can I tell varnish to save the cached data to disk at shutdown? > > That is not possible. > > However, you can reconfigure Varnish at runtime, so there should be > little reason to shutdown a varnish process once it is in production. > > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc > > From corinl at gmx.de Wed Dec 5 17:42:23 2007 From: corinl at gmx.de (Corin Langosch) Date: Wed, 05 Dec 2007 18:42:23 +0100 Subject: stable version Message-ID: <4756E2FF.2000505@gmx.de> Hi! I have the problem that varnishd is crashing every few minutes or hours. I already tried the "debian" version varnish-1.0.2 (crashes after 6-12 hours) and varnish-1.1.1 (download from hp, self build, crashes after 0-2 hours). Syslog shows errors like varnishd[12575]: segfault at 000000000000000c rip 0000000000406fd1 rsp 000000005101eaf0 error 4 Does a stable version exist? Which one is it? Are prebuild binaries available? Regards, Corin From gwenael.saint-genest at makina-corpus.com Wed Dec 5 20:14:39 2007 From: gwenael.saint-genest at makina-corpus.com (Saint-Genest Gwenael) Date: Wed, 05 Dec 2007 21:14:39 +0100 Subject: stable version In-Reply-To: <4756E2FF.2000505@gmx.de> References: <4756E2FF.2000505@gmx.de> Message-ID: <475706AF.1050808@makina-corpus.com> Corin Langosch wrote: > Hi! > > I have the problem that varnishd is crashing every few minutes or hours. > > I already tried the "debian" version varnish-1.0.2 (crashes after 6-12 > hours) and varnish-1.1.1 (download from hp, self build, crashes after > 0-2 hours). Syslog shows errors like > > varnishd[12575]: segfault at 000000000000000c rip 0000000000406fd1 rsp > 000000005101eaf0 error 4 > > Does a stable version exist? Which one is it? Are prebuild binaries > available? > > Regards, > Corin > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc Hi ! For your main question (about a "stable" version) i don't know, sorry. But maybe, we can investigate for your problem ;) Can you look into your binary to see where the program is when segfault appear ? (with objdump -d varnishd ... or something like this) G. Saint-Genest -- Saint-Genest Gwenael Makina Corpus - http://www.makina-corpus.com/ 44 boulevard des Pas enchant?s FR-44230 Saint-S?bastien-sur-Loire T : +33 (0) 2 51 79 80 80 P: +33 (0)6 81 05 03 00 From jonsml at gmail.com Wed Dec 5 21:54:10 2007 From: jonsml at gmail.com (Jon) Date: Wed, 5 Dec 2007 16:54:10 -0500 Subject: worker threads maxing out Message-ID: <5f06ff9a0712051354w7fa9bf85j853074bec9e6a49f@mail.gmail.com> Greetings, I've been battling this for several days now and need some help to figure this out. Machines stats: Ubuntu 7.10 64-bit, 4 GB RAM, 60 GB available drive space The issue I've been encountering is that every morning during peak traffic hours the N worker threads will hit the max allowed then N queued work requests hits the max and of course the server starts to reject any new incoming requests. My backend server will start to queue. This is what I have for the varnishd options "-a 0.0.0.0:80 -f /usr/local/varnish/etc/default.net.vcl -p send_timeout=30 -w 1500,3000,30" I've been changing the -w numbers around but it hasn't help. When I lowered the max to 200 it didn't cause my backend server to queue but it was rejecting requests. I've also set the ulimit -n to 65536 in the limits.conf file and in the mornings varnishd will eat up 100% cpu and restarts over and over. Then I added the ulimit -n 65536 into the startup script that's when it stopped restarting and started the problem I described above. After I remove all the load from the Varnish server the N worker threads is fixed at 1500. During non-peak hours the N worker threads is ~260. During peak it serves out 40 - 80 Mbps of images, previously using Squid and it's fine. Thanks! Jon From corinl at gmx.de Thu Dec 6 00:27:05 2007 From: corinl at gmx.de (Corin Langosch) Date: Thu, 06 Dec 2007 01:27:05 +0100 Subject: stable version In-Reply-To: <475706AF.1050808@makina-corpus.com> References: <4756E2FF.2000505@gmx.de> <475706AF.1050808@makina-corpus.com> Message-ID: <475741D9.5050102@gmx.de> Saint-Genest Gwenael schrieb: > Corin Langosch wrote: > >> Hi! >> >> I have the problem that varnishd is crashing every few minutes or hours. >> >> I already tried the "debian" version varnish-1.0.2 (crashes after 6-12 >> hours) and varnish-1.1.1 (download from hp, self build, crashes after >> 0-2 hours). Syslog shows errors like >> >> varnishd[12575]: segfault at 000000000000000c rip 0000000000406fd1 rsp >> 000000005101eaf0 error 4 >> >> Does a stable version exist? Which one is it? Are prebuild binaries >> available? >> >> Regards, >> Corin >> >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at projects.linpro.no >> http://projects.linpro.no/mailman/listinfo/varnish-misc >> > Hi ! > > For your main question (about a "stable" version) i don't know, sorry. > But maybe, we can investigate for your problem ;) Can you look into > your binary to see where the program is when segfault appear ? (with > objdump -d varnishd ... or something like this) > > G. Saint-Genest Hi! Using the "debian" version varnish-1.0.2 I took the 406fd1 from the segfault rip which points a "mov 0xc(%rdi),%edx). This is inside section "VBE_ClosedFd". Is this correct? Does varnishd hit any system limits like open files..? Regards, Corin -------------- next part -------------- An HTML attachment was scrubbed... URL: From ssm at linpro.no Thu Dec 6 05:54:42 2007 From: ssm at linpro.no (Stig Sandbeck Mathisen) Date: Thu, 06 Dec 2007 06:54:42 +0100 Subject: Varnish and HTTPS In-Reply-To: (duja@torlen.net's message of "Tue, 4 Dec 2007 13:08:17 +0100") References: Message-ID: <7xhciw2wrh.fsf@iostat.linpro.no> Erik writes: > Thanks for the info but I did just wondered if varnish pass it to > the backend or if it drops the request. I did some tests and it > seems like it passes the request to the backend. > > BTW, what program may I use to have https2http? Isn't https > encrypted? This is already answered in http://varnish.projects.linpro.no/wiki/FAQ -- Stig Sandbeck Mathisen, Linpro From duja at torlen.net Thu Dec 6 13:49:55 2007 From: duja at torlen.net (Erik) Date: Thu, 6 Dec 2007 14:49:55 +0100 Subject: Varnish Hash Message-ID: Just to make this clear, does varnish identify an object like this in vcl_hash? sub vcl_hash { set req.hash += req.url; set req.hash += req.http.host; hash; } / Erik From phk at phk.freebsd.dk Thu Dec 6 22:34:36 2007 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Thu, 06 Dec 2007 22:34:36 +0000 Subject: Varnish Hash In-Reply-To: Your message of "Thu, 06 Dec 2007 14:49:55 +0100." Message-ID: <1275.1196980476@critter.freebsd.dk> In message , Erik writes: >Just to make this clear, does varnish identify an object like this in vcl_hash? > > sub vcl_hash { > set req.hash += req.url; > set req.hash += req.http.host; > hash; > } Well, mostly. That is the primary identification, but each match can have multiple different objects, depending on the Vary header and ttl. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From ric at digitalmarbles.com Fri Dec 7 02:09:57 2007 From: ric at digitalmarbles.com (Ricardo Newbery) Date: Thu, 6 Dec 2007 18:09:57 -0800 Subject: Varnish Hash In-Reply-To: <1275.1196980476@critter.freebsd.dk> References: <1275.1196980476@critter.freebsd.dk> Message-ID: <36256474-9C9D-4C1C-9CE8-12AD5D5EE682@digitalmarbles.com> On Dec 6, 2007, at 2:34 PM, Poul-Henning Kamp wrote: > In message , Erik writes: >> Just to make this clear, does varnish identify an object like this >> in vcl_hash? >> >> sub vcl_hash { >> set req.hash += req.url; >> set req.hash += req.http.host; >> hash; >> } > > Well, mostly. That is the primary identification, but each match > can have multiple different objects, depending on the Vary header > and ttl. Apologies for butting into this thread... Multiple objects depending on ttl? Can you elaborate? Also, how are purges done when variations exist? I'm guessing all variations get purged. Is this correct? Ric From warproof at warproof.com Fri Dec 7 07:16:27 2007 From: warproof at warproof.com (Rich Christiansen) Date: Fri, 07 Dec 2007 00:16:27 -0700 Subject: Varnish Logging Message-ID: <4758F34B.8070909@warproof.com> Hello, all! First of all, thanks to everyone for making such a great product. Varnish has performed wonderfully. It really does 'just work'. I am having some problems regarding logging, however. Our specific app requires detailed logging data for every transaction. Namely, we have to track the number of bytes leaving our app so that we can charge clients based on data transfer usage. So, for using varnish, we've got two problems: 1) varnishncsa seems to only log the size (in bytes) of the file being transfered. We deal a lot with large files (30MB-100MB), so many of our file transfers are cut short. We need to be able to log exactly how many bytes were actually transfered. Is this possible? 2) We've also noticed that if the file transfer is aborted before about 2MB is transfered, the file transfer isn't logged at all to access.log. Is this by design? For the same reasons stated in #1, we need to find a reason to log these transfers as well. On a side note, I tried using Squid (yuck) and found that it satisfied point 1, but /very/ strangely also had the same 'problem' described in point 2. (Although the 'logging threshold' was lower, around 1MB or so) So weird. I'd love to hear your thoughts! If we can't get this working, I'm going to have to resort to a suboptimal solution using lighttpd and rsync...(yuck) Thanks in advance, -Rich -------------- next part -------------- An HTML attachment was scrubbed... URL: From phk at phk.freebsd.dk Fri Dec 7 08:41:29 2007 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Fri, 07 Dec 2007 08:41:29 +0000 Subject: Varnish Hash In-Reply-To: Your message of "Thu, 06 Dec 2007 18:09:57 PST." <36256474-9C9D-4C1C-9CE8-12AD5D5EE682@digitalmarbles.com> Message-ID: <3213.1197016889@critter.freebsd.dk> In message <36256474-9C9D-4C1C-9CE8-12AD5D5EE682 at digitalmarbles.com>, Ricardo N ewbery writes: >> In message , Erik writes: >>> Just to make this clear, does varnish identify an object like this >>> in vcl_hash? >>> >>> sub vcl_hash { >>> set req.hash += req.url; >>> set req.hash += req.http.host; >>> hash; >>> } >> >> Well, mostly. That is the primary identification, but each match >> can have multiple different objects, depending on the Vary header >> and ttl. > >Apologies for butting into this thread... > >Multiple objects depending on ttl? Can you elaborate? That's a technical feature we need for prefetch to work. >Also, how are purges done when variations exist? I'm guessing all >variations get purged. Is this correct? Correct. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From ebe at dmi.dk Fri Dec 7 08:42:43 2007 From: ebe at dmi.dk (Eivind Bengtsson) Date: Fri, 07 Dec 2007 09:42:43 +0100 Subject: one cpu in 100% after simple url.purge ?? Message-ID: <47590783.5040509@dmi.dk> Hi all I just noticed something I did not expect... I had to purge two small images from te cache - they have a ttl of ten minutes. I telnet into varnish and write url.purge *varsel_oversigt.png I get a 200 I write url.purge *varsel_markering.* I get a 200 I log out of telnetpromt and notice that the "root-driven" varnishd is now in 100% cpu usage it has been there since (? an hour). There is no problem for the users (I think) as the "nobody-driven" varnishd serves them and only uses around 8% cpu. What is varnishd using one whole cpu for ? Regards Eivind -- Eivind Bengtsson Systemadministrator - Cand.merc.(dat) Danmarks Meteorologiske Institut Lyngbyvej 100 2100 K?benhavn ? Direkte tlf. : 39157544 Email: ebe at dmi.dk echo 'This is not a pipe.' | cat -> /dev/tty From gwenael.saint-genest at makina-corpus.com Fri Dec 7 09:43:41 2007 From: gwenael.saint-genest at makina-corpus.com (Saint-Genest Gwenael) Date: Fri, 07 Dec 2007 10:43:41 +0100 Subject: Varnish Logging In-Reply-To: <4758F34B.8070909@warproof.com> References: <4758F34B.8070909@warproof.com> Message-ID: <475915CD.2010900@makina-corpus.com> Rich Christiansen wrote: > Hello, all! Hi ! > 1) varnishncsa seems to only log the size (in bytes) of the file being > transfered. We deal a lot with large files (30MB-100MB), so many of our > file transfers are cut short. We need to be able to log exactly how > many bytes were actually transfered. Is this possible? I think you can't get this information for the moment. In your case, file transfer is made by RES_WriteObj() (cache_response.c) with a call to WRK_Sendfile() (cache_pool.c). If you look into thins function : if (WRK_Flush(w) == 0 && sendfile(*w->wfd, fd, &off, len) != len) w->werr++; The number of bytes copied (returned by sendfile) is not saved :( With a little patch you can save this value and use it for statistics. Gwen -- Saint-Genest Gwenael Makina Corpus - http://www.makina-corpus.com/ 44 boulevard des Pas enchant?s FR-44230 Saint-S?bastien-sur-Loire T : +33 (0) 2 51 79 80 80 P: +33 (0)6 81 05 03 00 From ebe at dmi.dk Fri Dec 7 09:56:42 2007 From: ebe at dmi.dk (Eivind Bengtsson) Date: Fri, 07 Dec 2007 10:56:42 +0100 Subject: one cpu in 100% after simple url.purge ?? In-Reply-To: <47590783.5040509@dmi.dk> References: <47590783.5040509@dmi.dk> Message-ID: <475918DA.3060106@dmi.dk> Hi in reply to my own mail ... Varnishd is using one cpu for this : poll([{fd=6, events=POLLIN}, {fd=14, events=POLLIN}, {fd=10, events=POLLIN, revents=POLLNVAL}, {fd=14, events=POLLIN}], 4, 2498) = 1 clock_gettime(CLOCK_MONOTONIC, {764359, 966356511}) = 0 poll([{fd=6, events=POLLIN}, {fd=14, events=POLLIN}, {fd=10, events=POLLIN, revents=POLLNVAL}, {fd=14, events=POLLIN}], 4, 2498) = 1 clock_gettime(CLOCK_MONOTONIC, {764359, 966436511}) = 0 poll([{fd=6, events=POLLIN}, {fd=14, events=POLLIN}, {fd=10, events=POLLIN, revents=POLLNVAL}, {fd=14, events=POLLIN}], 4, 2498) = 1 clock_gettime(CLOCK_MONOTONIC, {764359, 966514511}) = 0 poll([{fd=6, events=POLLIN}, {fd=14, events=POLLIN}, {fd=10, events=POLLIN, revents=POLLNVAL}, {fd=14, events=POLLIN}], 4, 2498) = 1 clock_gettime(CLOCK_MONOTONIC, {764359, 966588511}) = 0 poll([{fd=6, events=POLLIN}, {fd=14, events=POLLIN}, {fd=10, events=POLLIN, revents=POLLNVAL}, {fd=14, events=POLLIN}], 4, 2498) = 1 clock_gettime(CLOCK_MONOTONIC, {764359, 966663511}) = 0 poll([{fd=6, events=POLLIN}, {fd=14, events=POLLIN}, {fd=10, events=POLLIN, revents=POLLNVAL}, {fd=14, events=POLLIN}], 4, 2497) = 1 clock_gettime(CLOCK_MONOTONIC, {764359, 966743511}) = 0 poll([{fd=6, events=POLLIN}, {fd=14, events=POLLIN}, {fd=10, events=POLLIN, revents=POLLNVAL}, {fd=14, events=POLLIN}], 4, 2497) = 1 clock_gettime(CLOCK_MONOTONIC, {764359, 966818511}) = 0 and so on .... phk indiates on irc that it could be caused by the telnet... Just for the record ;-)... only escape is to restart varnishd. /Eivind ps Many thanks to the developers for quick help on diagnose Eivind Bengtsson wrote: > Hi all > > I just noticed something I did not expect... I had to purge two small > images from te cache - they have a ttl of ten minutes. > > I telnet into varnish and write > > url.purge *varsel_oversigt.png > I get a 200 > > I write > > url.purge *varsel_markering.* > I get a 200 > > I log out of telnetpromt > > and notice that the "root-driven" varnishd is now in 100% cpu usage it has > been there since (? an hour). > There is no problem for the users (I think) as the "nobody-driven" varnishd > serves them and only uses around 8% cpu. > > What is varnishd using one whole cpu for ? > > Regards > Eivind > -- Eivind Bengtsson Systemadministrator - Cand.merc.(dat) Danmarks Meteorologiske Institut Lyngbyvej 100 2100 K?benhavn ? Direkte tlf. : 39157544 Email: ebe at dmi.dk echo 'This is not a pipe.' | cat -> /dev/tty From phk at phk.freebsd.dk Fri Dec 7 11:45:01 2007 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Fri, 07 Dec 2007 11:45:01 +0000 Subject: Varnish Logging In-Reply-To: Your message of "Fri, 07 Dec 2007 10:43:41 +0100." <475915CD.2010900@makina-corpus.com> Message-ID: <4177.1197027901@critter.freebsd.dk> In message <475915CD.2010900 at makina-corpus.com>, Saint-Genest Gwenael writes: >Rich Christiansen wrote: >> transfered. We deal a lot with large files (30MB-100MB), so many of our >> file transfers are cut short. We need to be able to log exactly how >> many bytes were actually transfered. Is this possible? > > if (WRK_Flush(w) == 0 && > sendfile(*w->wfd, fd, &off, len) != len) > w->werr++; > >The number of bytes copied (returned by sendfile) is not saved :( > >With a little patch you can save this value and use it for statistics. Please open a ticket for this, so we get the number properly integrated in the statistics. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From ltning at anduin.net Mon Dec 10 14:23:22 2007 From: ltning at anduin.net (=?ISO-8859-1?Q?Eirik_=D8verby?=) Date: Mon, 10 Dec 2007 15:23:22 +0100 Subject: disable sendfile in Varnish, please In-Reply-To: <5362.1178896342@critter.freebsd.dk> References: <5362.1178896342@critter.freebsd.dk> Message-ID: Hi, is this still a problem? I'm running varnish in front of a midsize site (~2m hits/day), and have seen a couple of panics like the ones attached. It refers to sendfile, and fails to do a core dump (I know core dumps in general are functioning, as I had a couple of NIC-related ones not too long ago). Thus the attached "screenshot" is all I've got. FreeBSD anduin.net 6.2-STABLE FreeBSD 6.2-STABLE #1: Sat Apr 28 12:59:34 CEST 2007 Varnish version: varnish-1.1.1_2 (from ports) /Eirik On May 11, 2007, at 17:12, Poul-Henning Kamp wrote: > > I've nailed three different operating system kernels as having > sendfile(2) issues today, so I would advice all of you to > disable sendfile to avoid the various problems we've seen. > > The easiest way is to specify > > -p sendfile_threshold=-1 > > to varnishd, or by using the CLI: > > param.set sendfile_threshold -1 > > Only those of you with high paging rates would be in risk of seing > any performance changes as result of this. > > While it is quite easy to set up a test environment to look for > sendfile trouble, we cannot do it as part of varnish runtime so I > think we will ship with sendfile_threshold at -1 for the forseeable > future. > > Once we start to see kernels where sendfile works correctly in all > instances, we can consider how to seletively enable those. > > Alternatively, if nobody sees a performance loss from disabling it > now, we might as well just forget about it in toto. > > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > phk at FreeBSD.ORG | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by > incompetence. > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: fbsd_sendfile_panic.png Type: image/png Size: 57917 bytes Desc: not available URL: From phk at phk.freebsd.dk Mon Dec 10 16:17:47 2007 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Mon, 10 Dec 2007 16:17:47 +0000 Subject: disable sendfile in Varnish, please In-Reply-To: Your message of "Mon, 10 Dec 2007 15:23:22 +0100." Message-ID: <27157.1197303467@critter.freebsd.dk> In message , =?ISO-8859-1?Q?Ei rik_=D8verby?= writes: >Hi, > >is this still a problem? Yes, sendfile is not currently usable because it does not tell us when it is _really_ done with the data we send, so it can run afoul of our reuse of the memory for short lived objects. I have not spent any time on it, as the performance hit from using writev(2) seems to be trivial if one has plenty of RAM and because the real fix for the sendfile issue is likely to mean a redefinition of the sendfile system call. Poul-Henning >> I've nailed three different operating system kernels as having >> sendfile(2) issues today, so I would advice all of you to >> disable sendfile to avoid the various problems we've seen. >> >> The easiest way is to specify >> >> -p sendfile_threshold=-1 >> >> to varnishd, or by using the CLI: >> >> param.set sendfile_threshold -1 -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From ltning at anduin.net Mon Dec 10 19:00:08 2007 From: ltning at anduin.net (=?ISO-8859-1?Q?Eirik_=D8verby?=) Date: Mon, 10 Dec 2007 20:00:08 +0100 Subject: disable sendfile in Varnish, please In-Reply-To: <27157.1197303467@critter.freebsd.dk> References: <27157.1197303467@critter.freebsd.dk> Message-ID: On Dec 10, 2007, at 17:17, Poul-Henning Kamp wrote: > In message , =? > ISO-8859-1?Q?Ei > rik_=D8verby?= writes: > >> Hi, >> >> is this still a problem? > > Yes, sendfile is not currently usable because it does not tell > us when it is _really_ done with the data we send, so it can > run afoul of our reuse of the memory for short lived objects. Thanks. What's the likelihood that the error I saw today is in fact caused by this weakness in the sendfile implementation, and that Varnish has triggered it? In other words; now that I disabled it as suggested, how surprised should I be to see a similiar error again? /Eirik > I have not spent any time on it, as the performance hit from > using writev(2) seems to be trivial if one has plenty of RAM > and because the real fix for the sendfile issue is likely > to mean a redefinition of the sendfile system call. > > Poul-Henning > >>> I've nailed three different operating system kernels as having >>> sendfile(2) issues today, so I would advice all of you to >>> disable sendfile to avoid the various problems we've seen. >>> >>> The easiest way is to specify >>> >>> -p sendfile_threshold=-1 >>> >>> to varnishd, or by using the CLI: >>> >>> param.set sendfile_threshold -1 > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > phk at FreeBSD.ORG | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by > incompetence. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ltning at anduin.net Mon Dec 10 19:04:25 2007 From: ltning at anduin.net (=?ISO-8859-1?Q?Eirik_=D8verby?=) Date: Mon, 10 Dec 2007 20:04:25 +0100 Subject: varnishncsa, -w option and the HUP signal Message-ID: Hi, I suspect I've asked this before: Quoting the man page: -w file Write log entries to file instead of displaying them. The file will be overwritten unless the -a option was specified. If varnishncsa receives a SIGHUP while writing to a file, it will reopen the file, allowing the old one to be rotated away. Put shortly, varnishncsa seems to totally ignore HUP signals. No log file reopening takes place. I see from the source that code is there to handle it, but I lack the skills and time to figure out why it's not hit, or not working as (I) expect(ed). I remember trying this some time ago as well, while varnish 1.0 was the hotness, and found the same behavior. I've meanwhile resorted to writing to a fifo buffer and reading from there using flog. Not pretty, but it works. /Eirik -------------- next part -------------- An HTML attachment was scrubbed... URL: From phk at phk.freebsd.dk Mon Dec 10 19:28:18 2007 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Mon, 10 Dec 2007 19:28:18 +0000 Subject: disable sendfile in Varnish, please In-Reply-To: Your message of "Mon, 10 Dec 2007 20:00:08 +0100." Message-ID: <28129.1197314898@critter.freebsd.dk> In message , =?ISO-8859-1?Q?Ei rik_=D8verby?= writes: >>> is this still a problem? >> >> Yes, sendfile is not currently usable because it does not tell >> us when it is _really_ done with the data we send, so it can >> run afoul of our reuse of the memory for short lived objects. > >Thanks. What's the likelihood that the error I saw today is in fact >caused by this weakness in the sendfile implementation, and that >Varnish has triggered it? In other words; now that I disabled it as >suggested, how surprised should I be to see a similiar error again? Well, the fact that you have sendfile enabled at all means that you run a pretty old[1] version of varnish, so there are a number of other bugs you may hit, but at least the one where sendfile sends wrong data to the user should be gone. Poul-Henning [1] For such a young project obviously :-) -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From caleb.anthony at gmail.com Mon Dec 10 20:07:38 2007 From: caleb.anthony at gmail.com (Caleb Anthony) Date: Mon, 10 Dec 2007 13:07:38 -0700 Subject: varnishncsa, -w option and the HUP signal In-Reply-To: References: Message-ID: <33afbbe70712101207g7f247148gde2856524c5c8d23@mail.gmail.com> Hello, I'm not sure what version of Varnish you are using, but revision 2270 changed some of the SIGHUP code in varnishncsa: http://varnish.projects.linpro.no/changeset/2270 Download the latest SVN version and give it a try... On 12/10/07, Eirik ?verby wrote: > Hi, > > I suspect I've asked this before: > > Quoting the man page: > -w file Write log entries to file instead of displaying them. The > file will be overwritten unless the -a option was > specified. > > If varnishncsa receives a SIGHUP while writing to a file, > it > will reopen the file, allowing the old one to be rotated > away. > > Put shortly, varnishncsa seems to totally ignore HUP signals. No log file > reopening takes place. I see from the source that code is there to handle > it, but I lack the skills and time to figure out why it's not hit, or not > working as (I) expect(ed). > > I remember trying this some time ago as well, while varnish 1.0 was the > hotness, and found the same behavior. I've meanwhile resorted to writing to > a fifo buffer and reading from there using flog. Not pretty, but it works. > > /Eirik > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc > > From steve at kcilink.com Mon Dec 10 20:30:44 2007 From: steve at kcilink.com (Steve Scally) Date: Mon, 10 Dec 2007 15:30:44 -0500 Subject: varnishncsa, -w option and the HUP signal In-Reply-To: References: Message-ID: <7725E04D-3199-4219-8F83-9BB22FABD2F0@kcilink.com> Eirik, What is the string you are using for varnishncsa? Steve On Dec 10, 2007, at 2:04 PM, Eirik ?verby wrote: > Hi, > > I suspect I've asked this before: > > Quoting the man page: > -w file Write log entries to file instead of displaying > them. The > file will be overwritten unless the -a option was > specified. > > If varnishncsa receives a SIGHUP while writing to > a file, it > will reopen the file, allowing the old one to be > rotated > away. > > Put shortly, varnishncsa seems to totally ignore HUP signals. No > log file reopening takes place. I see from the source that code is > there to handle it, but I lack the skills and time to figure out > why it's not hit, or not working as (I) expect(ed). > > I remember trying this some time ago as well, while varnish 1.0 was > the hotness, and found the same behavior. I've meanwhile resorted > to writing to a fifo buffer and reading from there using flog. Not > pretty, but it works. > > /Eirik > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc -------------- next part -------------- An HTML attachment was scrubbed... URL: From steinove at vg.no Tue Dec 11 09:02:46 2007 From: steinove at vg.no (Stein Ove Rosseland) Date: Tue, 11 Dec 2007 10:02:46 +0100 Subject: disable sendfile in Varnish, please In-Reply-To: <27157.1197303467@critter.freebsd.dk> References: <27157.1197303467@critter.freebsd.dk> Message-ID: <475E5236.5090205@vg.no> Poul-Henning Kamp wrote: > In message , =?ISO-8859-1?Q?Ei > rik_=D8verby?= writes: > >> Hi, >> >> is this still a problem? > > Yes, sendfile is not currently usable because it does not tell > us when it is _really_ done with the data we send, so it can > run afoul of our reuse of the memory for short lived objects. > As i understand it, the problem is in the bsd implementation of sendfile, and does not occur in the linux implementation. Or do You have another opinion on the matter? Stein Ove ***************************************************************** Denne fotnoten bekrefter at denne e-postmeldingen ble skannet av MailSweeper og funnet fri for virus. ***************************************************************** This footnote confirms that this email message has been swept by MailSweeper for the presence of computer viruses. ***************************************************************** From phk at phk.freebsd.dk Tue Dec 11 09:29:10 2007 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Tue, 11 Dec 2007 09:29:10 +0000 Subject: disable sendfile in Varnish, please In-Reply-To: Your message of "Tue, 11 Dec 2007 10:02:46 +0100." <475E5236.5090205@vg.no> Message-ID: <73119.1197365350@critter.freebsd.dk> In message <475E5236.5090205 at vg.no>, Stein Ove Rosseland writes: >As i understand it, the problem is in the bsd implementation of >sendfile, and does not occur in the linux implementation. > >Or do You have another opinion on the matter? As far as I'm aware, the linux implementation has the same shortcoming (in addition to lacking the ability to send a header) -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From andre at idium.no Tue Dec 11 12:27:03 2007 From: andre at idium.no (=?ISO-8859-1?Q?Andr=E9_=D8ien_Langvand?=) Date: Tue, 11 Dec 2007 13:27:03 +0100 Subject: disable sendfile in Varnish, please In-Reply-To: <73119.1197365350@critter.freebsd.dk> References: <73119.1197365350@critter.freebsd.dk> Message-ID: <475E8217.3070808@idium.no> Yes, I was experiencing this problem w/ varnish on linux (2.6.17). -- Andr? ?ien Langvand - PGP: 0x7B1E3468 Systemadministrator - Idium AS - http://www.idium.no Poul-Henning Kamp wrote: > In message <475E5236.5090205 at vg.no>, Stein Ove Rosseland writes: > >> As i understand it, the problem is in the bsd implementation of >> sendfile, and does not occur in the linux implementation. >> >> Or do You have another opinion on the matter? > > As far as I'm aware, the linux implementation has the same shortcoming > (in addition to lacking the ability to send a header) > From florian.schulze at gmx.net Wed Dec 12 15:55:00 2007 From: florian.schulze at gmx.net (Florian Schulze) Date: Wed, 12 Dec 2007 16:55:00 +0100 Subject: Release status References: Message-ID: On Sun, 18 Nov 2007 00:15:15 +0100, Martin Aspeli wrote: > Hi, > > When are we likely to see a new stable Varnish release? Will 1.1.2 ever > see the light of day? > > Unfortunately, Varnish 1.1.1 is unusable for me (stability issues); > trunk was fine for a while, but I'm experiencing some more stability > issues (and tracking trunk for production systems doesn't seem like a > very good idea). I'd be very happy with a stable 1.1.2, or a stable and > tested 2.0 if it's not too far away. > > Cheers, > Martin > Is there any news on the release? All we need is a 1.1.2 with the fix from r1968. It's almost been another month since this mail, and that fix is from september AFAIR. Regards, Florian Schulze From denis at zeno.org Wed Dec 12 17:04:48 2007 From: denis at zeno.org (Denis Ahrens) Date: Wed, 12 Dec 2007 18:04:48 +0100 Subject: Release status In-Reply-To: References: Message-ID: <86D38D51-158C-4932-BA7E-2D6C1754F1D1@zeno.org> On 12.12.2007, at 16:55, Florian Schulze wrote: > On Sun, 18 Nov 2007 00:15:15 +0100, Martin Aspeli > wrote: > >> Hi, >> >> When are we likely to see a new stable Varnish release? Will 1.1.2 >> ever >> see the light of day? >> >> Unfortunately, Varnish 1.1.1 is unusable for me (stability issues); >> trunk was fine for a while, but I'm experiencing some more stability >> issues (and tracking trunk for production systems doesn't seem like a >> very good idea). I'd be very happy with a stable 1.1.2, or a stable >> and >> tested 2.0 if it's not too far away. I need the fix from r1971! ;) Denis From blynch at sharpcast.com Thu Dec 13 07:52:45 2007 From: blynch at sharpcast.com (Brian Lynch) Date: Wed, 12 Dec 2007 23:52:45 -0800 Subject: backend server timeout Message-ID: <594646A31FBBB341953566459E0CADCF01445D18@alder.corp.sharpcast.com> Hi, I noticed that there used to be a parameter for adjusting the timeout for backend servers is no longer in the vcl syntax. Is there another way to adjust the amount of time varnish will wait for the response from a backend server? Thanks, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From hakon.jensen at escenic.com Thu Dec 13 09:20:40 2007 From: hakon.jensen at escenic.com (=?ISO-8859-1?Q?H=E5kon_Andreas_Jensen?=) Date: Thu, 13 Dec 2007 10:20:40 +0100 Subject: Purge multiple varnish servers Message-ID: <4760F968.7040102@escenic.com> I'm trying to figure out a way to purge multiple Varnish servers. Is it possible to have a master Varnish server that receives an URL to purge and then passes the purge on to the other servers using VCL. I know this can be done passing each server a purge command, but I would like just to communicate with the master Varnish and let it take care of the rest of the purge logic. I'm quite new in this area, so please bare with me. Thanks in advance for any hints and advices. -H?kon From phk at phk.freebsd.dk Thu Dec 13 09:30:30 2007 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Thu, 13 Dec 2007 09:30:30 +0000 Subject: Purge multiple varnish servers In-Reply-To: Your message of "Thu, 13 Dec 2007 10:20:40 +0100." <4760F968.7040102@escenic.com> Message-ID: <67379.1197538230@critter.freebsd.dk> In message <4760F968.7040102 at escenic.com>, =?ISO-8859-1?Q?H=E5kon_Andreas_Jensen?= writes: >I'm trying to figure out a way to purge multiple Varnish servers. >Is it possible to have a master Varnish server that receives an URL >to purge and then passes the purge on to the other servers using >VCL. If you have looked at that original varnish documents, you'll spot a box labeled "cluster controller" That was more or less what it was inteded for. We havn't got around to writing it yet. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From hakon.jensen at escenic.com Thu Dec 13 09:46:47 2007 From: hakon.jensen at escenic.com (=?ISO-8859-1?Q?H=E5kon_Andreas_Jensen?=) Date: Thu, 13 Dec 2007 10:46:47 +0100 Subject: Purge multiple varnish servers In-Reply-To: <67379.1197538230@critter.freebsd.dk> References: <67379.1197538230@critter.freebsd.dk> Message-ID: <4760FF87.1010909@escenic.com> Poul-Henning Kamp wrote: > In message <4760F968.7040102 at escenic.com>, =?ISO-8859-1?Q?H=E5kon_Andreas_Jensen?= writes: > > >> I'm trying to figure out a way to purge multiple Varnish servers. >> Is it possible to have a master Varnish server that receives an URL >> to purge and then passes the purge on to the other servers using >> VCL. >> > > If you have looked at that original varnish documents, you'll spot a box > labeled "cluster controller" > > That was more or less what it was inteded for. > > We havn't got around to writing it yet. > Ok. Thanks for replying so quickly. -H?kon From duja at torlen.net Thu Dec 13 11:02:38 2007 From: duja at torlen.net (Erik) Date: Thu, 13 Dec 2007 12:02:38 +0100 Subject: purge_url Message-ID: I will stop bothering the helpful guys at the irc for a minute and ask a question here ;) My VCL conf looks like this: sub vcl_recv { #Change the host header to www.mysite.com if(req.http.host == "10.1.1.54" || req.http.host == "test.mysite.com") { set req.http.host = "www.mysite.com"; } #Purge specified files from acl purge. #Purge = Delete the specified url from the cache if(req.request == "PURGE") { if(client.ip ~ purge) { purge_url(req.url); } } sub vcl_miss { #set http version 1.1 set bereq.proto = "HTTP/1.1"; if (req.request == "PURGE") { error 404 "The url could not be found in the cache."; } } sub vcl_hit { if (req.request == "PURGE") { set obj.ttl = 0s; error 200 "Purged successfully!"; } } } # End of VCL Conf I receive a 404 error "The url could not be found in the cache." when I try to PURGE the url. PURGE http://www.mysite.com/data/hello.gif HTTP/1.1 This is the varnishlog when I try to purge: 12 RxRequest c PURGE 12 RxURL c /data/hello.gif 12 RxProtocol c HTTP/1.1 12 VCL_call c recv 12 VCL_acl c MATCH purge "84.xx.xx.xx" 12 VCL_return c lookup 12 VCL_call c hash 12 VCL_return c hash 12 Debug c "Hash: /data/hello.gif#10.1.1.54:80#" 12 VCL_call c miss 0 Debug "VCL_error(404, The url could not be found in the cache.)" 12 VCL_return c error 12 Length c 453 12 TxProtocol c HTTP/1.0 12 TxStatus c 404 12 TxResponse c Not Found 12 TxHeader c Server: Varnish 12 TxHeader c Retry-After: 0 12 TxHeader c Content-Type: text/html; charset=utf-8 12 TxHeader c Content-Length: 453 12 TxHeader c Date: Thu, 13 Dec 2007 11:34:50 GMT 12 TxHeader c X-Varnish: 1188471893 12 TxHeader c Age: nan 12 TxHeader c Via: 1.1 varnish 12 TxHeader c Connection: keep-alive When I try to request it with GET i receive: 12 ReqStart c 84.xx.xx.xx 32162 1188471894 12 RxRequest c GET 12 RxURL c /data/hello.gif 12 RxProtocol c HTTP/1.1 12 RxHeader c Host: www.mysite.com 12 RxHeader c User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; sv-SE; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11 12 RxHeader c Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 12 RxHeader c Accept-Language: sv,en-us;q=0.7,en;q=0.3 12 RxHeader c Accept-Encoding: gzip,deflate 12 RxHeader c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 12 RxHeader c Keep-Alive: 300 12 RxHeader c Connection: keep-alive 12 RxHeader c Cookie:__utma=75765975.864378540.1197378455.1197453843.1197535348.6; 12 VCL_call c recv 12 VCL_return c lookup 12 VCL_call c hash 12 VCL_return c hash 12 Debug c "Hash Match: /data/hello.gif#www.mysite.com#" 12 Hit c 1188471893 12 VCL_call c hit 12 VCL_return c deliver 12 Length c 905 12 VCL_call c deliver 12 VCL_return c deliver 12 TxProtocol c HTTP/1.1 12 TxStatus c 200 12 TxResponse c OK 12 TxHeader c Server: Zeus/4.3 12 TxHeader c Content-Type: image/gif 12 TxHeader c Content-Language: sv-SE 12 TxHeader c Content-Length: 905 12 TxHeader c Date: Thu, 13 Dec 2007 11:37:26 GMT 12 TxHeader c X-Varnish: 1188471894 1188471893 12 TxHeader c Age: 43 12 TxHeader c Via: 1.1 varnish 12 TxHeader c Connection: keep-alive The strange thing is that the host is different in PURGE Match and GET Match. I have used the same host on both requests. Does it have anything to do with the vcl code where Im changing the hostheader? / Erik From jbittman at chewcorp.com Fri Dec 14 00:23:02 2007 From: jbittman at chewcorp.com (Jobe Bittman) Date: Thu, 13 Dec 2007 16:23:02 -0800 Subject: hiding varnish headers from requests In-Reply-To: <18219.16591.185911.307821@localhost.localdomain> References: <18219.16591.185911.307821@localhost.localdomain> Message-ID: <4aa1ddab0712131623y421558f3r18013df6dea528f3@mail.gmail.com> Edit your vcl file and modify deliver like so. sub vcl_deliver { remove resp.http.X-Varnish; remove resp.http.Via; } On 11/2/07, Damien Wetzel wrote: > > Hello, > I wondered if there is a way to prevent Varnish from > puting headers like : > X-varnish > Via: 1.1 varnish > in its responses to clients. > thanks, > -- > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > Damien WETZEL (ATANAR TECHNOLOGIES) ("`-/")_.-'"``-._ > http://www.atanar.com . . `; -._ )-;-,_`) > (v_,)' _ )`-.\ ``-' > Phone:+33 6 62 29 61 77 _.- _..-_/ / ((.' > - So much to do, so little time - ((,.-' ((,/ > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc > -- Jobe Bittman Chief Network Architect ZenGarden, Inc. (858) 200-6099 -------------- next part -------------- An HTML attachment was scrubbed... URL: From vilhelm at aptoma.com Fri Dec 14 09:03:34 2007 From: vilhelm at aptoma.com (=?ISO-8859-1?Q?=22Vilhelm_K=2E_Vard=F8y=22?=) Date: Fri, 14 Dec 2007 10:03:34 +0100 Subject: Release status In-Reply-To: References: Message-ID: <476246E6.706@aptoma.com> Hi, Martin Aspeli wrote: > When are we likely to see a new stable Varnish release? Will 1.1.2 ever > see the light of day? I'm also waiting for this release. It would be really helpful to get a status update (the web page isn't very helpful since it states a delay until some time late September). Thanks! -- Best regards, Vilhelm K. Vard?y Programmer - APTOMA AS Akersgata 55, 0170 Oslo, NORWAY Mobile +47 46 66 23 96 http://www.aptoma.no From lists at tomster.org Fri Dec 14 21:16:19 2007 From: lists at tomster.org (Tom Lazar) Date: Fri, 14 Dec 2007 22:16:19 +0100 Subject: Release status In-Reply-To: <476246E6.706@aptoma.com> References: <476246E6.706@aptoma.com> Message-ID: <90031C63-4B64-4327-BFF9-2BD26D7C4BF4@tomster.org> On 14.12.2007, at 10:03, Vilhelm K. Vard?y wrote: > Hi, > > Martin Aspeli wrote: >> When are we likely to see a new stable Varnish release? Will 1.1.2 >> ever >> see the light of day? > > I'm also waiting for this release. It would be really helpful to > get a > status update (the web page isn't very helpful since it states a delay > until some time late September). same here. FIW, i actually switched off two varnish 1.0.4 instances because they have been crashing every couple of days. > > > Thanks! > > -- > Best regards, > Vilhelm K. Vard?y > Programmer - APTOMA AS > Akersgata 55, 0170 Oslo, NORWAY > Mobile +47 46 66 23 96 > http://www.aptoma.no > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc > From Phil.Cryer at edwardjones.com Mon Dec 17 14:56:54 2007 From: Phil.Cryer at edwardjones.com (Cryer,Phil) Date: Mon, 17 Dec 2007 08:56:54 -0600 Subject: Release status In-Reply-To: <90031C63-4B64-4327-BFF9-2BD26D7C4BF4@tomster.org> References: <476246E6.706@aptoma.com> <90031C63-4B64-4327-BFF9-2BD26D7C4BF4@tomster.org> Message-ID: <3ECD7F7DDE95BA4FA598E8DDE71F1A5105C98CAE@nwpsrv08.edj.ad.edwardjones.com> If you are not the intended recipient of this message (including attachments), or if you have received this message in error, immediately notify us and delete it and any attachments. If you no longer wish to receive e-mail from Edward Jones, please send this request to messages at edwardjones.com. You must include the e-mail address that you wish not to receive e-mail communications. For important additional information related to this e-mail, visit www.edwardjones.com/US_email_disclosure -----Original Message----- From: varnish-misc-bounces at projects.linpro.no [mailto:varnish-misc-bounces at projects.linpro.no] On Behalf Of Tom Lazar Sent: Friday, December 14, 2007 3:16 PM To: varnish-misc at projects.linpro.no Subject: Re: Release status On 14.12.2007, at 10:03, Vilhelm K. Vard?y wrote: > Hi, > > Martin Aspeli wrote: >> When are we likely to see a new stable Varnish release? Will 1.1.2 >> ever see the light of day? > > I'm also waiting for this release. It would be really helpful to get > a status update (the web page isn't very helpful since it states a > delay until some time late September). same here. FIW, i actually switched off two varnish 1.0.4 instances because they have been crashing every couple of days. Plus, I think it looks very bad to have a message on the front page that 1.1.2 is delayed due to illness with a promise date of 3 months in the past. That doesn't instill confidence in perspective users/customers browsing the site. P From anders at fupp.net Mon Dec 17 23:54:18 2007 From: anders at fupp.net (Anders Nordby) Date: Tue, 18 Dec 2007 00:54:18 +0100 Subject: purge_url In-Reply-To: References: Message-ID: <20071217235418.GB11496@fupp.net> On Thu, Dec 13, 2007 at 12:02:38PM +0100, Erik wrote: > I will stop bothering the helpful guys at the irc for a minute and ask a question here ;) > > My VCL conf looks like this: > sub vcl_recv { > #Change the host header to www.mysite.com > if(req.http.host == "10.1.1.54" || req.http.host == "test.mysite.com") { > set req.http.host = "www.mysite.com"; > } > > #Purge specified files from acl purge. > #Purge = Delete the specified url from the cache > if(req.request == "PURGE") { > if(client.ip ~ purge) { > purge_url(req.url); > } > } I would change this part to: if(req.request == "REPURGE") { if(client.ip ~ purge) { purge_url(req.url); error 200 "Repurged." } } If you are sending a regexp purge, you will always get a miss, unless you accidentally don't use any regexp special characters - which does not make sence. In any case, I recommend using a different request for normal purge and regexp purge, but supporting both in the VCL. Bye, -- Anders. From des at linpro.no Tue Dec 18 17:21:53 2007 From: des at linpro.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Tue, 18 Dec 2007 18:21:53 +0100 Subject: The revenge of the return of 1.1.2 Message-ID: Believe it or not, I have merged a few additional fixes and finished the change log for 1.1.2, and once Stig merges his changes to the Debian package metadata, we will be all set for a December 20th release. Please test... DES -- Dag-Erling Sm?rgrav Senior Software Developer Linpro AS - www.linpro.no From florian.schulze at gmx.net Tue Dec 18 17:46:22 2007 From: florian.schulze at gmx.net (Florian Schulze) Date: Tue, 18 Dec 2007 18:46:22 +0100 Subject: The revenge of the return of 1.1.2 References: Message-ID: On Tue, 18 Dec 2007 18:21:53 +0100, Dag-Erling Sm?rgrav wrote: > Believe it or not, I have merged a few additional fixes and finished > the change log for 1.1.2, and once Stig merges his changes to the > Debian package metadata, we will be all set for a December 20th > release. Please test... > > DES /me cheers! Go, DES, go! Regards, Florian Schulze From florian.schulze at gmx.net Tue Dec 18 18:03:17 2007 From: florian.schulze at gmx.net (Florian Schulze) Date: Tue, 18 Dec 2007 19:03:17 +0100 Subject: The revenge of the return of 1.1.2 References: Message-ID: On Tue, 18 Dec 2007 18:21:53 +0100, Dag-Erling Sm?rgrav wrote: > Believe it or not, I have merged a few additional fixes and finished > the change log for 1.1.2, and once Stig merges his changes to the > Debian package metadata, we will be all set for a December 20th > release. Please test... > > DES I just fetched the 1.1 branch r2308 and tried to compile on Mac OS X Leopard. cache_pool.c:43:2: error: #error Unknown sendfile() implementation cache_pool.c:173:2: error: #error Unknown sendfile() implementation make[3]: *** [varnishd-cache_pool.o] Error 1 make[2]: *** [all-recursive] Error 1 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2 During configure I get this when looking for sendfile: checking for sendfile... yes some more configure info: checking build system type... i386-apple-darwin9.1.0 checking host system type... i386-apple-darwin9.1.0 checking target system type... i386-apple-darwin9.1.0 ... checking for ld used by gcc... /usr/libexec/gcc/i686-apple-darwin9/4.0.1/ld Some gcc info: $ gcc -v Using built-in specs. Target: i686-apple-darwin9 Configured with: /var/tmp/gcc/gcc-5465~16/src/configure --disable-checking -enable-werror --prefix=/usr --mandir=/share/man --enable-languages=c,objc,c++,obj-c++ --program-transform-name=/^[cg][^.-]*$/s/$/-4.0/ --with-gxx-include-dir=/include/c++/4.0.0 --with-slibdir=/usr/lib --build=i686-apple-darwin9 --with-arch=apple --with-tune=generic --host=i686-apple-darwin9 --target=i686-apple-darwin9 Thread model: posix gcc version 4.0.1 (Apple Inc. build 5465) Regards, Florian Schulze From jfbustarret at wat.tv Tue Dec 18 18:12:56 2007 From: jfbustarret at wat.tv (BUSTARRET, Jean-francois) Date: Tue, 18 Dec 2007 19:12:56 +0100 Subject: The revenge of the return of 1.1.2 In-Reply-To: References: Message-ID: <53C652A09719C54DA24741D0157CB2695FFAB5@TFPRDEXS1.tf1.groupetf1.fr> Wonderful news ! Thanks DES JFB > -----Message d'origine----- > De : varnish-misc-bounces at projects.linpro.no > [mailto:varnish-misc-bounces at projects.linpro.no] De la part > de Dag-Erling Sm?rgrav > Envoy? : mardi 18 d?cembre 2007 18:22 > ? : varnish-misc at projects.linpro.no > Objet : The revenge of the return of 1.1.2 > > Believe it or not, I have merged a few additional fixes and > finished the change log for 1.1.2, and once Stig merges his > changes to the Debian package metadata, we will be all set > for a December 20th release. Please test... > > DES > -- > Dag-Erling Sm?rgrav > Senior Software Developer > Linpro AS - www.linpro.no > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc > From ssm at linpro.no Wed Dec 19 07:59:36 2007 From: ssm at linpro.no (Stig Sandbeck Mathisen) Date: Wed, 19 Dec 2007 08:59:36 +0100 Subject: The revenge of the return of 1.1.2 In-Reply-To: (Dag-Erling =?utf-8?Q?Sm?= =?utf-8?Q?=C3=B8rgrav's?= message of "Tue, 18 Dec 2007 18:21:53 +0100") References: Message-ID: <7x1w9jcdxz.fsf@iostat.linpro.no> Dag-Erling Sm?rgrav writes: > [...] once Stig merges his changes to the Debian package metadata, That's now done as well. -- Stig Sandbeck Mathisen, Linpro From des at linpro.no Wed Dec 19 14:09:20 2007 From: des at linpro.no (=?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?=) Date: Wed, 19 Dec 2007 15:09:20 +0100 Subject: The revenge of the return of 1.1.2 In-Reply-To: (Florian Schulze's message of "Tue, 18 Dec 2007 19:03:17 +0100") References: Message-ID: "Florian Schulze" writes: > I just fetched the 1.1 branch r2308 and tried to compile on Mac OS X > Leopard. > > cache_pool.c:43:2: error: #error Unknown sendfile() implementation > cache_pool.c:173:2: error: #error Unknown sendfile() implementation > make[3]: *** [varnishd-cache_pool.o] Error 1 > make[2]: *** [all-recursive] Error 1 > make[1]: *** [all-recursive] Error 1 > make: *** [all] Error 2 Ah, yes. Both Linux and FreeBSD have sendfile(), but with different argument lists and slightly different semantics, so we check for __FreeBSD__ or __linux__ when using sendfile(). This didn't use to be a problem, because AFAIK Mac OS X didn't have sendfile() before Leopard. I imagine Leopard's sendfile() works like FreeBSD's, but I don't have a Leopard box to test on (my Mac is too old to run it). I can change cache_pool.c so just ignores HAVE_SENDFILE in the !(FreeBSD || Linux) case, however. DES -- Dag-Erling Sm?rgrav Senior Software Developer Linpro AS - www.linpro.no From florian.schulze at gmx.net Wed Dec 19 21:00:55 2007 From: florian.schulze at gmx.net (Florian Schulze) Date: Wed, 19 Dec 2007 22:00:55 +0100 Subject: The revenge of the return of 1.1.2 References: <7x1w9jcdxz.fsf@iostat.linpro.no> Message-ID: On Wed, 19 Dec 2007 08:59:36 +0100, Stig Sandbeck Mathisen wrote: > Dag-Erling Sm?rgrav writes: > >> [...] once Stig merges his changes to the Debian package metadata, > > That's now done as well. > Thanks for bringing the release even more close! Regards, Florian Schulze From ericlin.jnlin at gmail.com Thu Dec 20 12:11:10 2007 From: ericlin.jnlin at gmail.com (Lin Jui-Nan Eric) Date: Thu, 20 Dec 2007 20:11:10 +0800 Subject: Varnishd crash while receiving some HTTP requests In-Reply-To: <47713ee10712200235t1cd58f3en94b4fa222dbb796@mail.gmail.com> References: <47713ee10712200235t1cd58f3en94b4fa222dbb796@mail.gmail.com> Message-ID: <47713ee10712200411q575edbdel7845cd0770243445@mail.gmail.com> Dear All, I found that varnishd child crashed while receiving the below request: GET /robot.txt HTTP/1.0 \r\n\r\n\r\n While the count of "\r\n" is larger than 3, no response will be generated by varnishd. If the count of "\r\n" is 2, that's ok. I use varnishd-1.2 from svn: http://varnish.projects.linpro.no/svn/branches/1.2 The running command is: varnishd -a "[MYIP]:10000" -f /usr/local/etc/varnish/image.vcl -h classic,1048583 -n /gstripe0/varnish0 -s "file,/gstripe0/varnish0/varnish-cache.mmap,100G" -t 604800 -w 32,1024 -d The error message is: Child said (2, 65305): <> Cache child died pid=65305 status=0x6 Clean child Child cleaned start child pid 65306 Child said (2, 65306): <> From ericlin.jnlin at gmail.com Thu Dec 20 15:50:41 2007 From: ericlin.jnlin at gmail.com (Lin Jui-Nan Eric) Date: Thu, 20 Dec 2007 23:50:41 +0800 Subject: Varnishd crash while receiving some HTTP requests Message-ID: <47713ee10712200750i42f5034ctadb9efdba0e419f1@mail.gmail.com> Dear All, I found that varnishd child crashed while receiving the below request: GET /robot.txt HTTP/1.0 \r\n\r\n\r\n While the count of "\r\n" is larger than 3, no response will be generated by varnishd. If the count of "\r\n" is 2, that's ok. I use varnishd-1.2 from svn: http://varnish.projects.linpro.no/svn/branches/1.2 The running command is: varnishd -a "[MYIP]:10000" -f /usr/local/etc/varnish/image.vcl -h classic,1048583 -n /gstripe0/varnish0 -s "file,/gstripe0/varnish0/varnish-cache.mmap,100G" -t 604800 -w 32,1024 -d The error message is: Child said (2, 65305): <> Cache child died pid=65305 status=0x6 Clean child Child cleaned start child pid 65306 Child said (2, 65306): <> From ericlin at tamama.org Thu Dec 20 15:51:46 2007 From: ericlin at tamama.org (Lin Jui-Nan Eric) Date: Thu, 20 Dec 2007 23:51:46 +0800 Subject: Varnishd crash while receiving some HTTP requests In-Reply-To: <47713ee10712200750i42f5034ctadb9efdba0e419f1@mail.gmail.com> References: <47713ee10712200750i42f5034ctadb9efdba0e419f1@mail.gmail.com> Message-ID: <47713ee10712200751j23dae407u66f6ac856d3ffbed@mail.gmail.com> Dear All, I found that varnishd child crashed while receiving the below request: GET /robot.txt HTTP/1.0 \r\n\r\n\r\n While the count of "\r\n" is larger than 3, no response will be generated by varnishd. If the count of "\r\n" is 2, that's ok. I use varnishd-1.2 from svn: http://varnish.projects.linpro.no/svn/branches/1.2 The running command is: varnishd -a "[MYIP]:10000" -f /usr/local/etc/varnish/image.vcl -h classic,1048583 -n /gstripe0/varnish0 -s "file,/gstripe0/varnish0/varnish-cache.mmap,100G" -t 604800 -w 32,1024 -d The error message is: Child said (2, 65305): <> Cache child died pid=65305 status=0x6 Clean child Child cleaned start child pid 65306 Child said (2, 65306): <> From wangmd0127 at gmail.com Thu Dec 20 16:00:29 2007 From: wangmd0127 at gmail.com (=?UTF-8?B?546L5piO6L6+?=) Date: Fri, 21 Dec 2007 00:00:29 +0800 Subject: varnishd 1.1.2 not useing epoll in kernel 2.6 Message-ID: hello, I install varnishd 1.1.2 in Red Hat AS4.4(kernel: 2.6.9-42.ELsmp). I'm very strange, it doesn't useing epoll. That's the output of my configure and strace: checking for strvisx... no checking for daemon... yes checking for clock_gettime... yes checking for kqueue... no checking for epoll_ctl... yes checking for poll... yes checking whether SO_RCVTIMEO works... yes writev(9, [{"200 15 \n", 13}, {"PONG 1198165652", 15}, {"\n", 1}], 3) = 29 poll([{fd=10, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 read(10, "ping\n", 4096) = 5 time(NULL) = 1198165655 writev(9, [{"200 15 \n", 13}, {"PONG 1198165655", 15}, {"\n", 1}], 3) = 29 poll([{fd=10, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 read(10, "ping\n", 4096) = 5 time(NULL) = 1198165658 Regards! -------------- next part -------------- An HTML attachment was scrubbed... URL: From wangmd0127 at gmail.com Fri Dec 21 02:47:28 2007 From: wangmd0127 at gmail.com (mingdawang) Date: Fri, 21 Dec 2007 10:47:28 +0800 Subject: varnish 1.1.2 not use epoll in kernel 2.6 Message-ID: Hello? I have installed varnish-1.1.2 for testing,but I found a strange problem, I installed varnish with steps as followed: ./configure --prefix=/home/Varnish/ make make install I saw information when I configured: checking for strvisx... no checking for daemon... yes checking for kqueue... no checking for epoll_ctl... yes checking for poll... yes checking whether SO_RCVTIMEO works... yes After I started varnish,I used strace to trace it, then I found varnish didn't use epoll at all? writev(9, [{"200 15 \n", 13}, {"PONG 1198165652", 15}, {"\n", 1}], 3) = 29 poll([{fd=10, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 read(10, "ping\n", 4096) = 5 time(NULL) = 1198165655 writev(9, [{"200 15 \n", 13}, {"PONG 1198165655", 15}, {"\n", 1}], 3) = 29 poll([{fd=10, events=POLLIN, revents=POLLIN}], 1, 5000) = 1 read(10, "ping\n", 4096) = 5 time(NULL) = 1198165658 my os is Red hat AS4.4 with kernel 2.6.9-42.ELsmp , how to make epoll to work in varnish? Thank you very much! -------------- next part -------------- An HTML attachment was scrubbed... URL: From ericlin at tamama.org Fri Dec 21 04:36:57 2007 From: ericlin at tamama.org (Lin Jui-Nan Eric) Date: Fri, 21 Dec 2007 12:36:57 +0800 Subject: Varnishd crash while receiving some HTTP requests In-Reply-To: <47713ee10712200751j23dae407u66f6ac856d3ffbed@mail.gmail.com> References: <47713ee10712200750i42f5034ctadb9efdba0e419f1@mail.gmail.com> <47713ee10712200751j23dae407u66f6ac856d3ffbed@mail.gmail.com> Message-ID: <47713ee10712202036j6dac12ddgb23e43a4dd249e9d@mail.gmail.com> Dear All, I found that varnishd child crashed while receiving the below request: GET /robot.txt HTTP/1.0 \r\n\r\n\r\n While the count of "\r\n" is larger than 3, no response will be generated by varnishd. If the count of "\r\n" is 2, that's ok. I use varnishd-1.2 from svn: http://varnish.projects.linpro.no/svn/branches/1.2 The running command is: varnishd -a "[MYIP]:10000" -f /usr/local/etc/varnish/image.vcl -h classic,1048583 -n /gstripe0/varnish0 -s "file,/gstripe0/varnish0/varnish-cache.mmap,100G" -t 604800 -w 32,1024 -d The error message is: Child said (2, 65305): <> Cache child died pid=65305 status=0x6 Clean child Child cleaned start child pid 65306 Child said (2, 65306): <> From michael.lee at zerustech.com Fri Dec 21 04:55:28 2007 From: michael.lee at zerustech.com (Michael Lee) Date: Fri, 21 Dec 2007 12:55:28 +0800 Subject: Varnishd crash while receiving some HTTP requests In-Reply-To: <47713ee10712202036j6dac12ddgb23e43a4dd249e9d@mail.gmail.com> References: <47713ee10712200750i42f5034ctadb9efdba0e419f1@mail.gmail.com> <47713ee10712200751j23dae407u66f6ac856d3ffbed@mail.gmail.com> <47713ee10712202036j6dac12ddgb23e43a4dd249e9d@mail.gmail.com> Message-ID: <476B4740.5080606@zerustech.com> Hi Jui-Nan, Looks like you should try version 1.1.2 instead. V1.2 is a bit outdated. It was changed one month ago. Besides, I guess v1.2 is not a publish release yet. Kind regards, Michael Lin Jui-Nan Eric wrote: > Dear All, > > I found that varnishd child crashed while receiving the below request: > GET /robot.txt HTTP/1.0 \r\n\r\n\r\n > > While the count of "\r\n" is larger than 3, no response will be > generated by varnishd. > If the count of "\r\n" is 2, that's ok. > I use varnishd-1.2 from svn: http://varnish.projects.linpro.no/svn/branches/1.2 > > The running command is: > varnishd -a "[MYIP]:10000" -f /usr/local/etc/varnish/image.vcl -h > classic,1048583 -n /gstripe0/varnish0 -s > "file,/gstripe0/varnish0/varnish-cache.mmap,100G" -t 604800 -w 32,1024 > -d > > The error message is: > > Child said (2, 65305): < cache_http.c line 366: > Condition(p <= t.e) not true. > errno = 0 (Unknown error: 0) > Cache child died pid=65305 status=0x6 > Clean child > Child cleaned > start child pid 65306 > Child said (2, 65306): < managed to mmap 107374182400 bytes of 107374182400 > Ready > CLI ready > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc > > -- Michael Lee / Managing Director / ZerusTech Ltd Tel: +86 (21) 6428 3803 Fax: +86 (21) 6428 3805 Mobile: +86 139 1663 5259 Skype: zerustech Gmail: zerustech at gmail.com www.zerustech.com Suite 819-820 807 Zhaojiabang Road Shanghai P.R.China 200030 From ericlin at tamama.org Fri Dec 21 07:55:12 2007 From: ericlin at tamama.org (Lin Jui-Nan Eric) Date: Fri, 21 Dec 2007 15:55:12 +0800 Subject: Varnishd crash while receiving some HTTP requests In-Reply-To: <476B4740.5080606@zerustech.com> References: <47713ee10712200750i42f5034ctadb9efdba0e419f1@mail.gmail.com> <47713ee10712200751j23dae407u66f6ac856d3ffbed@mail.gmail.com> <47713ee10712202036j6dac12ddgb23e43a4dd249e9d@mail.gmail.com> <476B4740.5080606@zerustech.com> Message-ID: <47713ee10712202355m5a1c0a0bj2b73a6156def353d@mail.gmail.com> Dear Michael, I have tried 1.1.2 and this problem is solved. Thank you very much! On 12/21/07, Michael Lee wrote: > Hi Jui-Nan, > > Looks like you should try version 1.1.2 instead. V1.2 is a bit outdated. It was changed one month ago. > > Besides, I guess v1.2 is not a publish release yet. > > Kind regards, > > Michael > > Lin Jui-Nan Eric wrote: > > Dear All, > > > > I found that varnishd child crashed while receiving the below request: > > GET /robot.txt HTTP/1.0 \r\n\r\n\r\n > > > > While the count of "\r\n" is larger than 3, no response will be > > generated by varnishd. > > If the count of "\r\n" is 2, that's ok. > > I use varnishd-1.2 from svn: http://varnish.projects.linpro.no/svn/branches/1.2 > > > > The running command is: > > varnishd -a "[MYIP]:10000" -f /usr/local/etc/varnish/image.vcl -h > > classic,1048583 -n /gstripe0/varnish0 -s > > "file,/gstripe0/varnish0/varnish-cache.mmap,100G" -t 604800 -w 32,1024 > > -d > > > > The error message is: > > > > Child said (2, 65305): < > cache_http.c line 366: > > Condition(p <= t.e) not true. > > errno = 0 (Unknown error: 0) > > Cache child died pid=65305 status=0x6 > > Clean child > > Child cleaned > > start child pid 65306 > > Child said (2, 65306): < > managed to mmap 107374182400 bytes of 107374182400 > > Ready > > CLI ready > > _______________________________________________ > > varnish-misc mailing list > > varnish-misc at projects.linpro.no > > http://projects.linpro.no/mailman/listinfo/varnish-misc > > > > > > > -- > Michael Lee / Managing Director / ZerusTech Ltd > > Tel: +86 (21) 6428 3803 > Fax: +86 (21) 6428 3805 > Mobile: +86 139 1663 5259 > Skype: zerustech > Gmail: zerustech at gmail.com > > www.zerustech.com > > Suite 819-820 > 807 Zhaojiabang Road > Shanghai > P.R.China > 200030 > From michael.lee at zerustech.com Fri Dec 21 08:05:41 2007 From: michael.lee at zerustech.com (Michael Lee) Date: Fri, 21 Dec 2007 16:05:41 +0800 Subject: Varnishd crash while receiving some HTTP requests In-Reply-To: <47713ee10712202355m5a1c0a0bj2b73a6156def353d@mail.gmail.com> References: <47713ee10712200750i42f5034ctadb9efdba0e419f1@mail.gmail.com> <47713ee10712200751j23dae407u66f6ac856d3ffbed@mail.gmail.com> <47713ee10712202036j6dac12ddgb23e43a4dd249e9d@mail.gmail.com> <476B4740.5080606@zerustech.com> <47713ee10712202355m5a1c0a0bj2b73a6156def353d@mail.gmail.com> Message-ID: <476B73D5.8030505@zerustech.com> No problem :) Lin Jui-Nan Eric wrote: > Dear Michael, > > I have tried 1.1.2 and this problem is solved. Thank you very much! > > On 12/21/07, Michael Lee wrote: >> Hi Jui-Nan, >> >> Looks like you should try version 1.1.2 instead. V1.2 is a bit outdated. It was changed one month ago. >> >> Besides, I guess v1.2 is not a publish release yet. >> >> Kind regards, >> >> Michael >> >> Lin Jui-Nan Eric wrote: >>> Dear All, >>> >>> I found that varnishd child crashed while receiving the below request: >>> GET /robot.txt HTTP/1.0 \r\n\r\n\r\n >>> >>> While the count of "\r\n" is larger than 3, no response will be >>> generated by varnishd. >>> If the count of "\r\n" is 2, that's ok. >>> I use varnishd-1.2 from svn: http://varnish.projects.linpro.no/svn/branches/1.2 >>> >>> The running command is: >>> varnishd -a "[MYIP]:10000" -f /usr/local/etc/varnish/image.vcl -h >>> classic,1048583 -n /gstripe0/varnish0 -s >>> "file,/gstripe0/varnish0/varnish-cache.mmap,100G" -t 604800 -w 32,1024 >>> -d >>> >>> The error message is: >>> >>> Child said (2, 65305): <>> cache_http.c line 366: >>> Condition(p <= t.e) not true. >>> errno = 0 (Unknown error: 0) >>> Cache child died pid=65305 status=0x6 >>> Clean child >>> Child cleaned >>> start child pid 65306 >>> Child said (2, 65306): <>> managed to mmap 107374182400 bytes of 107374182400 >>> Ready >>> CLI ready >>> _______________________________________________ >>> varnish-misc mailing list >>> varnish-misc at projects.linpro.no >>> http://projects.linpro.no/mailman/listinfo/varnish-misc >>> >>> >> >> -- >> Michael Lee / Managing Director / ZerusTech Ltd >> >> Tel: +86 (21) 6428 3803 >> Fax: +86 (21) 6428 3805 >> Mobile: +86 139 1663 5259 >> Skype: zerustech >> Gmail: zerustech at gmail.com >> >> www.zerustech.com >> >> Suite 819-820 >> 807 Zhaojiabang Road >> Shanghai >> P.R.China >> 200030 >> > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc > > -- Michael Lee / Managing Director / ZerusTech Ltd Tel: +86 (21) 6428 3803 Fax: +86 (21) 6428 3805 Mobile: +86 139 1663 5259 Skype: zerustech Gmail: zerustech at gmail.com www.zerustech.com Suite 819-820 807 Zhaojiabang Road Shanghai P.R.China 200030 From ericlin at tamama.org Fri Dec 21 10:59:26 2007 From: ericlin at tamama.org (Lin Jui-Nan Eric) Date: Fri, 21 Dec 2007 18:59:26 +0800 Subject: Varnishd 1.1.2 returns 5xx Message-ID: <47713ee10712210259h3be8efe4t50daab5e8a347412@mail.gmail.com> Dear All, I am trying to replace squid with varnishd for reverse proxy. But I found varnishd 1.1.2 sometimes responses 5xx, while varnishd 1.2 works correctly. I tested with the command below: cat a_long_url_list | wget -i - -B http://rproxy{1,2}/ -O /dev/null -o /dev/stdout | egrep -i '(response|fail)' | grep -v 200 where rproxy1 running varnishd-1.2, and rproxy2 running varnishd-1.1.2 The output of wget which connected rproxy2 is: HTTP request sent, awaiting response... 503 Service Unavailable HTTP request sent, awaiting response... 503 Service Unavailable HTTP request sent, awaiting response... 503 Service Unavailable HTTP request sent, awaiting response... 503 Service Unavailable HTTP request sent, awaiting response... 503 Service Unavailable All OS is FreeBSD 7.0. Thank you for any suggestions! From denis at zeno.org Fri Dec 21 16:39:57 2007 From: denis at zeno.org (Denis Ahrens) Date: Fri, 21 Dec 2007 17:39:57 +0100 Subject: problems with varnish 1.1.2 Message-ID: <33667D03-8703-446B-99DE-DE10C759A0FF@zeno.org> Hi I tried today to switch to varnish 1.1.2 but have a problem. It seems that version 1.1.2 does not send the data itself. When I test it with telnet I get the headers but after some seconds I get disconnected. [Flashlie:~] denis% telnet www.test.org 80 Trying 10.1.1.2... Connected to www.test.org. Escape character is '^]'. GET http://www.test.org/index.html HTTP/1.1 HTTP/1.1 200 OK Server: ZenoServer/2.1.10 * DBSERVER/r1456 * Middleware/1444 Vary: Accept-Encoding Content-Type: text/html Cache-Control: s-maxage=300, max-age=120 Last-Modified: Fri, 21 Dec 2007 15:20:42 GMT Date: Fri, 21 Dec 2007 16:33:38 GMT X-Varnish: 1220401610 Age: 0 Via: 1.1 varnish Connection: keep-alive Connection closed by foreign host. The connection to the backend works because the Server header shows this. But it does not deliver the content. Any suggestions? I build version 1.1.1 the same way and everything works Denis From phk at phk.freebsd.dk Sat Dec 22 08:40:29 2007 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Sat, 22 Dec 2007 08:40:29 +0000 Subject: Varnishd 1.1.2 returns 5xx In-Reply-To: Your message of "Fri, 21 Dec 2007 18:59:26 +0800." <47713ee10712210259h3be8efe4t50daab5e8a347412@mail.gmail.com> Message-ID: <25995.1198312829@critter.freebsd.dk> In message <47713ee10712210259h3be8efe4t50daab5e8a347412 at mail.gmail.com>, "Lin Jui-Nan Eric" writes: >I am trying to replace squid with varnishd for reverse proxy. But I >found varnishd 1.1.2 sometimes responses 5xx, while varnishd 1.2 works >correctly. Examine your varnishlog output to see why varnish gives 503. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From ericlin at tamama.org Mon Dec 24 16:26:52 2007 From: ericlin at tamama.org (Lin Jui-Nan Eric) Date: Tue, 25 Dec 2007 00:26:52 +0800 Subject: varnishd: out of swap space (was: Varnishd 1.1.2 returns 5xx) Message-ID: <47713ee10712240826j2933765as825ff5bdd00b6c1d@mail.gmail.com> Now I encounter another problem: out of swap space. > dmesg -a | tail -10 swap_pager_getswapspace(6): failed swap_pager_getswapspace(4): failed swap_pager_getswapspace(8): failed swap_pager_getswapspace(8): failed swap_pager_getswapspace(8): failed swap_pager_getswapspace(9): failed swap_pager_getswapspace(8): failed swap_pager_getswapspace(8): failed pid 4190 (varnishd), uid 65534, was killed: out of swap space Dec 23 07:47:39 rproxy2 kernel: pid 4190 (varnishd), uid 65534, was killed: out of swap space > uname -a rproxy2 [/net/account/admin/jnlin] -jnlin- uname -a FreeBSD rproxy2 7.0-BETA2 FreeBSD 7.0-BETA2 #1: Mon Nov 12 16:06:40 CST 2007 root at rproxy2:/usr/obj/usr/src/sys/KERNEL amd64 > cat /var/run/dmesg.boot | head -22 Copyright (c) 1992-2007 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 7.0-BETA2 #1: Mon Nov 12 16:06:40 CST 2007 root at rproxy2:/usr/obj/usr/src/sys/KERNEL Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Genuine Intel(R) CPU @ 2.66GHz (2660.01-MHz K8-class CPU) Origin = "GenuineIntel" Id = 0x6f4 Stepping = 4 Features=0xbfebfbff Features2=0x4e3bd AMD Features=0x20100800 AMD Features2=0x1 Cores per package: 2 usable memory = 4285853696 (4087 MB) avail memory = 4124594176 (3933 MB) ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 cpu2 (AP): APIC ID: 6 cpu3 (AP): APIC ID: 7 Do I need to tune any sysctl values or...? Thank you for any suggestions! From Lynch_Wu at asus.com.tw Wed Dec 26 07:30:12 2007 From: Lynch_Wu at asus.com.tw (Lynch_Wu at asus.com.tw) Date: Wed, 26 Dec 2007 15:30:12 +0800 Subject: Document root Message-ID: <72D8F1BB8BA9E2479060AF035DFBF4F1690373@TP-EX-26.corpnet.asus> Hi all, Could someone clarify how to specify the document root in Varnish VCL, just like in Apache? Currently I am using WebLogic as backend server and Apache as web server, where all the images / css / javascripts are in the document root of Apache. For some reason I am trying to replace Apache with Varnish. But the first step is to figure out how to conquer the settings! Thanks in advance, Lynch From ericlin at tamama.org Wed Dec 26 11:36:06 2007 From: ericlin at tamama.org (Lin Jui-Nan Eric) Date: Wed, 26 Dec 2007 19:36:06 +0800 Subject: Document root In-Reply-To: <72D8F1BB8BA9E2479060AF035DFBF4F1690373@TP-EX-26.corpnet.asus> References: <72D8F1BB8BA9E2479060AF035DFBF4F1690373@TP-EX-26.corpnet.asus> Message-ID: <47713ee10712260336w525305e2w5a08b4628fcc2457@mail.gmail.com> Hi, Varnish is a "reverse proxy". You must have a backend HTTP Server, such as apache or lighttpd. On 12/26/07, Lynch_Wu at asus.com.tw wrote: > Hi all, > > Could someone clarify how to specify the document root in > Varnish VCL, just like in Apache? > > Currently I am using WebLogic as backend server and Apache > as web server, where all the images / css / javascripts are in the > document root of Apache. For some reason I am trying to replace > Apache with Varnish. But the first step is to figure out how to > conquer the settings! > > Thanks in advance, > Lynch > _______________________________________________ > varnish-misc mailing list > varnish-misc at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-misc > From paul.marsh at gmail.com Fri Dec 28 14:25:45 2007 From: paul.marsh at gmail.com (Paul) Date: Fri, 28 Dec 2007 09:25:45 -0500 Subject: Two New HTTP Caching Extensions Message-ID: <63a3d0be0712280625i7cdd347du7c5b215bd7b50b2b@mail.gmail.com> Found these ideas very interesting, made it into Squid 2.7. Was curious if Varnish would ever support similar functionality. http://www.mnot.net/blog/2007/12/12/stale From game at wst.net.cn Sun Dec 23 02:59:32 2007 From: game at wst.net.cn (game) Date: Sun, 23 Dec 2007 02:59:32 -0000 Subject: Varnish1.1.2 returns 200OK,but no content Message-ID: <20071223024223.M58534@wst.net.cn> An HTML attachment was scrubbed... URL: