From andersb at vgnett.no Fri Sep 1 16:36:28 2006 From: andersb at vgnett.no (Anders Berg) Date: Fri, 1 Sep 2006 18:36:28 +0200 Subject: Back on mail etc. Message-ID: Hi, just checking inn on mail again. Next week I will slowly get back to work. Not gonna write details about my back on public mailinglist, so if you wonder just send private email. Anders Berg From phk at phk.freebsd.dk Sat Sep 2 16:49:55 2006 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Sat, 02 Sep 2006 16:49:55 +0000 Subject: NUUG presentation on september 19? In-Reply-To: Your message of "Thu, 31 Aug 2006 08:39:31 +0200." <20060831063931.GA62503@totem.fix.no> Message-ID: <1391.1157215795@critter.freebsd.dk> In message <20060831063931.GA62503 at totem.fix.no>, Anders Nordby writes: >Hi, >I just got an OK from NUUG, so we're all set for september 19. > >What I need now is: > >- title of the presentation. The "Varnish" web-accellerator -- or "How to be 10 times faster without really trying". >- 1-2 paragraphs about the presentation. VG sponsored the development of a web-accellerator called "Varnish" because Squid was too slow for them. The first half of the talk will introduce Varnish and present some of the novel features it brings to the business of web-serving. The second half of the talk, using Varnish as the example, will show ways to get most performance out of modern hardware and operating systems. >- 1 paragraph about the speaker. Poul-Henning Kamp has been mucking about with computers long enough to become an old fart. When he doesn't renovate the FreeBSD kernel, he solves problems for customers at reasonable hourly rates. Poul-Henning -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From des at linpro.no Sun Sep 3 18:15:01 2006 From: des at linpro.no (Dag-Erling =?iso-8859-1?Q?Sm=F8rgrav?=) Date: Sun, 03 Sep 2006 20:15:01 +0200 Subject: NUUG presentation on september 19? References: <1391.1157215795@critter.freebsd.dk> Message-ID: "Poul-Henning Kamp" writes: > Anders Nordby writes: > >- title of the presentation. > The "Varnish" web-accellerator We've consistently used "The Varnish HTTP accelerator" so far. DES -- Dag-Erling Sm?rgrav Senior Software Developer Linpro AS - www.linpro.no From phk at phk.freebsd.dk Sun Sep 3 18:27:18 2006 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Sun, 03 Sep 2006 18:27:18 +0000 Subject: NUUG presentation on september 19? In-Reply-To: Your message of "Sun, 03 Sep 2006 20:15:01 +0200." Message-ID: <1301.1157308038@critter.freebsd.dk> In message , Dag-Erling =?iso-8859-1?Q?Sm=F8rgra v?= writes: >"Poul-Henning Kamp" writes: >> Anders Nordby writes: >> >- title of the presentation. >> The "Varnish" web-accellerator > >We've consistently used "The Varnish HTTP accelerator" so far. Ok, I'll stick to that. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From anders at fupp.net Mon Sep 4 09:18:14 2006 From: anders at fupp.net (Anders Nordby) Date: Mon, 4 Sep 2006 11:18:14 +0200 Subject: NUUG presentation on september 19? In-Reply-To: <1391.1157215795@critter.freebsd.dk> References: <20060831063931.GA62503@totem.fix.no> <1391.1157215795@critter.freebsd.dk> Message-ID: <20060904091814.GA6333@totem.fix.no> Hi, On Sat, Sep 02, 2006 at 04:49:55PM +0000, Poul-Henning Kamp wrote: >>What I need now is: > The "Varnish" web-accellerator > -- or > "How to be 10 times faster without really trying". > (..) I take it this means you will do the presentation in english? Bye, -- Anders. From phk at phk.freebsd.dk Mon Sep 4 09:22:10 2006 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Mon, 04 Sep 2006 09:22:10 +0000 Subject: NUUG presentation on september 19? In-Reply-To: Your message of "Mon, 04 Sep 2006 11:18:14 +0200." <20060904091814.GA6333@totem.fix.no> Message-ID: <4834.1157361730@critter.freebsd.dk> In message <20060904091814.GA6333 at totem.fix.no>, Anders Nordby writes: >Hi, > >On Sat, Sep 02, 2006 at 04:49:55PM +0000, Poul-Henning Kamp wrote: >>>What I need now is: >> The "Varnish" web-accellerator >> -- or >> "How to be 10 times faster without really trying". >> (..) > >I take it this means you will do the presentation in english? Yes, I am pretty sure I would talk so fast in Danish that nobody would be able to figure out what I said... -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From xing at litespeedtech.com Tue Sep 5 05:09:44 2006 From: xing at litespeedtech.com (Xing Li) Date: Mon, 04 Sep 2006 22:09:44 -0700 Subject: Bug Report..can't use Trac tickets Message-ID: <44FD0698.7030302@litespeedtech.com> Sorry for the list intrusion. Registered with Trac as "xing" but I don't have the rights to submit a bug ticket according to Trac: "TICKET_CREATE privileges are required to perform this operation". Following asserts are received when varnish was inserted into live test traffic stream situated in-between litespeed, a web server acting as a proxy, and the real server. The default VCL was modified so varnish will cache even with cookies present in request. No other change made. I have the communication logged to disk but there was too much data and I could not figure through the asserts which request was at fault. If a dev can tell me what to look for, I can research the varnish log. Xing Li LiteSpeed Technologies, Inc. Start Bug Report: --------------- CentOS 4.4, x86_64, libevent 1.1b, Varnish 0.9.1 launched with "varnishd -d -d -p 11101 -b 'myip 82' --------------- Source modification: Line 48 of varnishd/mgt_vcc.c from " if (req.http.Authenticate || req.http.Cookie) {\n" to " if (req.http.Authenticate) {\n" --------------- Error Description from varnishd: --------------- file /tmp/varnish.D85dYM (unlinked) size 83630907392 bytes (20417702 fs-blocks, 20417702 pages) rolling(1)... rolling(2)... start CLI start child pid 30671 200 0 Child said (2, 30671): <> Child said (2, 30671): <> Cache child died pid=30671 status=0x6 Clean child Child cleaned start child pid 31027 Child said (2, 31027): <> Child said (2, 31027): <> Cache child died pid=31027 status=0x6 Clean child Child cleaned start child pid 31122 Child said (2, 31122): <> Child said (2, 31122): <> Cache child died pid=31122 status=0x6 Clean child Child cleaned start child pid 31133 Child said (2, 31133): <> Child said (2, 31133): <> Cache child died pid=31133 status=0x6 Clean child Child cleaned start child pid 31160 Child said (2, 31160): <> Child said (2, 31160): <> Cache child died pid=31160 status=0x6 Clean child Child cleaned start child pid 31201 Child said (2, 31201): < Message-ID: <11816.1157438484@critter.freebsd.dk> In message <44FD0698.7030302 at litespeedtech.com>, Xing Li writes: Hi Xing, You get the honour of being the first "outsider" to report an error. Welcome! :-) >Sorry for the list intrusion. Registered with Trac as "xing" but I don't >have the rights to submit a bug ticket according to Trac: "TICKET_CREATE >privileges are required to perform this operation". (this is des@ domain, I'll leave it to him) >I have the communication logged to disk but there was too much data and >I could not figure through the asserts which request was at fault. If a >dev can tell me what to look for, I can research the varnish log. This particular assert indicates protocol trouble with a document that uses "chunked encoding", usually a CGI script, and the most likely trouble is that Varnish got out of step with the server somehow. In theory the transaction should be the last in the varnislog so if you run: varnishlog -d -w /tmp/foo (wait approx 10 seconds, then hit ctrl-C) varnishlog -r /tmp/foo | tail -200 You should get the information about which URL etc. The easiest way to debug it from there would be if I could access the server from here to pick up that URL, then I can see what happens close up with a debugger, but if that is not possible we will find another way. Poul-Henning -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From des at linpro.no Tue Sep 5 08:39:01 2006 From: des at linpro.no (Dag-Erling =?iso-8859-1?Q?Sm=F8rgrav?=) Date: Tue, 05 Sep 2006 10:39:01 +0200 Subject: Bug Report..can't use Trac tickets References: <44FD0698.7030302@litespeedtech.com> Message-ID: Xing Li writes: > Sorry for the list intrusion. Registered with Trac as "xing" but I > don't have the rights to submit a bug ticket according to Trac: > "TICKET_CREATE privileges are required to perform this operation". Sorry, should be fixed now. DES -- Dag-Erling Sm?rgrav Senior Software Developer Linpro AS - www.linpro.no From anders at fupp.net Tue Sep 5 19:47:22 2006 From: anders at fupp.net (Anders Nordby) Date: Tue, 5 Sep 2006 21:47:22 +0200 Subject: NUUG presentation on september 19 Message-ID: <20060905194722.GB2301@totem.fix.no> Hi, Should we change the title from "The Varnish HTTP accelerator" to "Releaseparty, The Varnish HTTP accelerator"? That is, do we make this a release party? It _might_ have a positive effect, when marketing this meeting. I've put the meeting up on NUUGs calendar: http://www.nuug.no/adict/?organizer=NUUG Cheers, -- Anders. From phk at phk.freebsd.dk Tue Sep 5 20:30:12 2006 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Tue, 05 Sep 2006 20:30:12 +0000 Subject: NUUG presentation on september 19 In-Reply-To: Your message of "Tue, 05 Sep 2006 21:47:22 +0200." <20060905194722.GB2301@totem.fix.no> Message-ID: <16712.1157488212@critter.freebsd.dk> In message <20060905194722.GB2301 at totem.fix.no>, Anders Nordby writes: >Hi, > >Should we change the title from "The Varnish HTTP accelerator" to >"Releaseparty, The Varnish HTTP accelerator"? That is, do we make this a >release party? Works for me :-) -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From Anders.Berg at vg.no Wed Sep 6 09:34:02 2006 From: Anders.Berg at vg.no (Anders Berg) Date: Wed, 6 Sep 2006 11:34:02 +0200 Subject: SV: NUUG presentation on september 19 References: <20060905194722.GB2301@totem.fix.no> Message-ID: I guess the right word is: Vernissage: The Varnish HTTP accelerator Consider Varnish as art, and that the name is derived from Vernissage :) But Releaseparty works fine. So we stick to that. Ab ________________________________ Fra: varnish-dev-bounces at projects.linpro.no p? vegne av Anders Nordby Sendt: ti 05.09.2006 21:47 Til: varnish-dev at projects.linpro.no Emne: NUUG presentation on september 19 Hi, Should we change the title from "The Varnish HTTP accelerator" to "Releaseparty, The Varnish HTTP accelerator"? That is, do we make this a release party? It _might_ have a positive effect, when marketing this meeting. I've put the meeting up on NUUGs calendar: http://www.nuug.no/adict/?organizer=NUUG Cheers, -- Anders. _______________________________________________ varnish-dev mailing list varnish-dev at projects.linpro.no http://projects.linpro.no/mailman/listinfo/varnish-dev ***************************************************************** Denne fotnoten bekrefter at denne e-postmeldingen ble skannet av MailSweeper og funnet fri for virus. ***************************************************************** This footnote confirms that this email message has been swept by MailSweeper for the presence of computer viruses. ***************************************************************** -------------- next part -------------- An HTML attachment was scrubbed... URL: From des at linpro.no Thu Sep 7 07:34:57 2006 From: des at linpro.no (Dag-Erling =?iso-8859-1?Q?Sm=F8rgrav?=) Date: Thu, 07 Sep 2006 09:34:57 +0200 Subject: Garbage in output Message-ID: I stopped varnishd on c21 after Arne got a corrupted front page. Here are the response headers: HTTP/1.0 200 OK Date: Thu, 07 Sep 2006 07:11:18 GMT Server: Apache/1.3.27 (Unix) Cache-Control: max-age=900 Expires: Thu, 07 Sep 2006 07:26:18 GMT Last-Modified: Thu, 07 Sep 2006 07:10:03 GMT Etag: "2b400e-22f85-44ffc5cb" Content-Type: text/html; charset=iso-8859-1 Age: 267, 549 X-Cache: HIT from www.vg.no Content-Length: 143237 X-Varnish: 1614312003 1613659966 Via: 1.1 varnish DES -- Dag-Erling Sm?rgrav Senior Software Developer Linpro AS - www.linpro.no -------------- next part -------------- A non-text attachment was scrubbed... Name: vgno Type: application/octet-stream Size: 197193 bytes Desc: not available URL: From andersb at vgnett.no Thu Sep 7 15:57:23 2006 From: andersb at vgnett.no (Anders Berg) Date: Thu, 7 Sep 2006 17:57:23 +0200 Subject: Preparation for launch Message-ID: I can't do much about most of the code so here is at least the mock- up for the T-shirt :) http://klikk.vg.no/varnish/img/VarnishT01.jpg http://klikk.vg.no/varnish/img/VarnishT02.jpg You guy's like? Due to time issue I may have to send it to print tomorrow. Anders Berg From edward at linpro.no Fri Sep 8 06:34:23 2006 From: edward at linpro.no (Edward Fjellskaal) Date: Fri, 08 Sep 2006 08:34:23 +0200 Subject: Preparation for launch In-Reply-To: References: Message-ID: Anders Berg wrote: > I can't do much about most of the code so here is at least the mock-up > for the T-shirt :) > > http://klikk.vg.no/varnish/img/VarnishT01.jpg > http://klikk.vg.no/varnish/img/VarnishT02.jpg > > You guy's like? Due to time issue I may have to send it to print tomorrow. > > Anders Berg I like it :) E -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 254 bytes Desc: OpenPGP digital signature URL: From des at linpro.no Fri Sep 8 09:38:03 2006 From: des at linpro.no (Dag-Erling =?iso-8859-1?Q?Sm=F8rgrav?=) Date: Fri, 08 Sep 2006 11:38:03 +0200 Subject: IRC meeting summary Message-ID: - 2006-09-20 confirmed as release date - all features are in present, though not necessarily complete - performance is acceptable for now - documentation is incomplete - des will take care - order of priorities for 1.0: 1) backend error handling 2) documentation 3) VCL features 4) performance - andersb will provide hardware for paralell testing on FreeBSD 6, FreeBSD 7 and Linux 2.6 (Ubuntu Dapper) I'm still not happy about the fact that we don't handle the "cache full" situation, but I guess we can live with it for now. DES -- Dag-Erling Sm?rgrav Senior Software Developer Linpro AS - www.linpro.no From stein at linpro.no Fri Sep 8 09:37:53 2006 From: stein at linpro.no (Stein Halvorsen) Date: Fri, 08 Sep 2006 11:37:53 +0200 Subject: Preparation for launch In-Reply-To: References: Message-ID: Edward Fjellskaal wrote: >> You guy's like? > I like it :) AOL! :D From phk at phk.freebsd.dk Mon Sep 11 19:48:30 2006 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Mon, 11 Sep 2006 19:48:30 +0000 Subject: I have update c21 Message-ID: <76944.1158004110@critter.freebsd.dk> I update c21 to FreeBSD RELENG_6 in order to find out if there are any of the bug fixes committed since july that affect us. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From phk at phk.freebsd.dk Mon Sep 11 19:57:54 2006 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Mon, 11 Sep 2006 19:57:54 +0000 Subject: I have update c21 In-Reply-To: Your message of "Mon, 11 Sep 2006 19:48:30 GMT." <76944.1158004110@critter.freebsd.dk> Message-ID: <77013.1158004674@critter.freebsd.dk> In message <76944.1158004110 at critter.freebsd.dk>, Poul-Henning Kamp writes: > >I update c21 to FreeBSD RELENG_6 in order to find out if there >are any of the bug fixes committed since july that affect us. > And put a couple of strategic sysctls in /etc/rc.local, we should document those. Not sure about the somaxconn thing, but we should at least mention it. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From anders at fupp.net Tue Sep 12 08:47:02 2006 From: anders at fupp.net (Anders Nordby) Date: Tue, 12 Sep 2006 10:47:02 +0200 Subject: NUUG meeting to be announced, material ready Message-ID: <20060912084702.GA79396@totem.fix.no> Hi, I just finished the web page about the meeting: http://www.nuug.no/aktiviteter/20060919-varnish/ Also there is a poster for people to hang up: http://www.nuug.no/aktiviteter/20060919-varnish/banner.ps http://www.nuug.no/aktiviteter/20060919-varnish/banner.pdf If no typos/errors/problems are reported, I will announce the meeting today. Cheers, -- Anders. From phk at phk.freebsd.dk Tue Sep 12 08:57:29 2006 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Tue, 12 Sep 2006 08:57:29 +0000 Subject: NUUG meeting to be announced, material ready In-Reply-To: Your message of "Tue, 12 Sep 2006 10:47:02 +0200." <20060912084702.GA79396@totem.fix.no> Message-ID: <80181.1158051449@critter.freebsd.dk> In message <20060912084702.GA79396 at totem.fix.no>, Anders Nordby writes: >Hi, > >I just finished the web page about the meeting: > >http://www.nuug.no/aktiviteter/20060919-varnish/ > >Also there is a poster for people to hang up: > >http://www.nuug.no/aktiviteter/20060919-varnish/banner.ps >http://www.nuug.no/aktiviteter/20060919-varnish/banner.pdf > >If no typos/errors/problems are reported, I will announce the meeting >today. I see no problems. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From xing at litespeedtech.com Tue Sep 12 17:52:22 2006 From: xing at litespeedtech.com (xing at litespeedtech.com) Date: Tue, 12 Sep 2006 10:52:22 -0700 Subject: Linux: rev 949 trunk is stable..not sure about others Message-ID: <4506F3D6.7090003@litespeedtech.com> Have been running on varnish/trunk revision 949 for a few days now. Other than a high cpu issue which is resolved by restarting varnish once a day, varnish has been very good. Just tested latest trunk build at 971 with bad result. Connections are stalling. Trying to backtrack in revision to see where it broke but I can't get the in-between revision to compile cleanly. from revision 957 to 970, builds on linux are broken in "make" stage: rm -f ".deps/varnishd-cache_pool.Tpo"; exit 1; fi cache_pool.c: In function `WRK_Sendfile': cache_pool.c:105: error: storage size of 'sfh' isn't known make[3]: *** [varnishd-cache_pool.o] Error 1 make[3]: Leaving directory `/root/varnish2/bin/varnishd' make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory `/root/varnish2/bin' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/root/varnish2' make: *** [all] Error 2 --------------------------- Rev 956 builds cleanly but results in the same connection stalling problem. So the problem has been narrowed down quite a bit. The changes somewhere between 949 and 956 are causing the connection stalling problem. I only tested latest rev and backtracked to clean build of 956. Didn't test 950-955. Will test those 5 revs later in the night PST time. Xing From phk at phk.freebsd.dk Tue Sep 12 19:20:42 2006 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Tue, 12 Sep 2006 19:20:42 +0000 Subject: Linux: rev 949 trunk is stable..not sure about others In-Reply-To: Your message of "Tue, 12 Sep 2006 10:52:22 MST." <4506F3D6.7090003@litespeedtech.com> Message-ID: <3969.1158088842@critter.freebsd.dk> In message <4506F3D6.7090003 at litespeedtech.com>, "xing at litespeedtech.com" write s: >Have been running on varnish/trunk revision 949 for a few days now. >Other than a high cpu issue which is resolved by restarting varnish once >a day, varnish has been very good. > >Just tested latest trunk build at 971 with bad result. Connections are >stalling. Ohh, that's bad. >Trying to backtrack in revision to see where it broke but I can't get >the in-between revision to compile cleanly. > >from revision 957 to 970, builds on linux are broken in "make" stage: That's just a missing ')' >The changes somewhere between 949 and 956 are causing the connection >stalling problem. I only tested latest rev and backtracked to clean >build of 956. Didn't test 950-955. Will test those 5 revs later in the >night PST time. It's a big help if you can track it down. I did some work to reduce mutex contention in the acceptor/herder and that's probably where the trouble is. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From andersb at vgnett.no Tue Sep 12 19:55:53 2006 From: andersb at vgnett.no (Anders Berg) Date: Tue, 12 Sep 2006 21:55:53 +0200 (CEST) Subject: varnishncsa Message-ID: <1511.193.69.165.4.1158090953.squirrel@denise.vg.no> Hi, thx for the quick intro into the world of gdb today, it helped me make varnishncsa more robust (found a cornercase). In all modesty I think I can say that varnishncsa might be getting darn close to alpha stage soon, but I am experiencing a major bug/designflaw somewhere, that hopefully can be fixed pretty easily. The result of this bug is something like this: 85.167.168.159 - - [12/Sep/2006:21:44:06 +0200 "GET /grafikk/frontbilder/126179.jpg HTTP/1.1" 200 2532 "http://www.vg.no/sport/fotball/ligaer/index.hbs?liga=england1div" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)" 80.213.127.56 - - [12/Sep/2006:21:44:32 +0200 "" 0 0 "-" "-" Notice that varnishsncsa halted for 26 seconds. So something in the SHM loglines I read, is causing major problems for the logger. The only way to fix it is to dump out varnishlog at the same time I run varnishncsa and compair notes to see whats causing it. This isn't a req for help, merly a state-of-code for varnishncsa. Also, I will need a base64 decoder soon to make varnishncsa follow the NCSA standard. How would we include this in Varnish DES? Is base64 decoding part of "standard" C? Or do we import the library, or do we cut paste the BSD licensed base64 code? Anders Berg From phk at phk.freebsd.dk Tue Sep 12 20:30:35 2006 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Tue, 12 Sep 2006 20:30:35 +0000 Subject: varnishncsa In-Reply-To: Your message of "Tue, 12 Sep 2006 21:55:53 +0200." <1511.193.69.165.4.1158090953.squirrel@denise.vg.no> Message-ID: <15766.1158093035@critter.freebsd.dk> In message <1511.193.69.165.4.1158090953.squirrel at denise.vg.no>, "Anders Berg" writes: >Hi, > >thx for the quick intro into the world of gdb today, it helped me make >varnishncsa more robust (found a cornercase). I've included a flexelint output below, there may be more you want to look at. You can get more info about the messages at www.gimpel.com. A fair number of them are likely to be false alarms. 137 Total 65 115_Error 44 755_Info 10 409_Warning 5 560_Warning 5 529_Warning 4 550_Warning 1 725_Info 1 719_Info 1 715_Info 1 551_Warning FlexeLint for C/C++ (Unix) Vers. 8.00u, Copyright Gimpel Software 1985-2006 --- Module: varnishncsa.c (C) _ #... __attribute__ ((noreturn)) void vdb_panic(const char *, ...) V_DEAD; ../../include/varnishapi.h 27 Error 10: Expecting ';' _ }; varnishncsa.c 54 Error 19: Useless Declaration _ static struct logline ll[65536]; varnishncsa.c 63 Error 88: Symbol 'll' is an array of empty elements _ } varnishncsa.c 114 Warning 529: Symbol 'tmpPtrc' (line 87) not subsequently referenced varnishncsa.c 87 Info 830: Location cited in prior message _ } varnishncsa.c 114 Warning 529: Symbol 'j' (line 91) not subsequently referenced varnishncsa.c 91 Info 830: Location cited in prior message _ tmpPtr = strchr(p + 4, ' '); varnishncsa.c 157 Error 64: Type mismatch (arg. no. 1) (ptrs to qualification,signed/unsigned) _ j = strlen(p + 4) - strlen(tmpPtr); // length of IP varnishncsa.c 158 Error 64: Type mismatch (arg. no. 1) (ptrs to qualification,signed/unsigned) varnishncsa.c 158 Error 64: Type mismatch (arg. no. 1) (ptrs to qualification,signed/unsigned) _ strncpy(ll[u].df_h, p + 4, j); varnishncsa.c 159 Error 115: struct/union not defined varnishncsa.c 159 Error 40: Undeclared identifier 'df_h' varnishncsa.c 159 Error 64: Type mismatch (arg. no. 2) (ptrs to qualification,signed/unsigned) _ ll[u].df_h[j] = '\0'; // put on a NULL at end of buffer. varnishncsa.c 160 Error 115: struct/union not defined varnishncsa.c 160 Error 40: Undeclared identifier 'df_h' varnishncsa.c 160 Warning 409: Expecting a pointer or array varnishncsa.c 160 Error 63: Expected an lvalue _ vsb_bcat(ob[u], p + 4, strlen(p + 4)); varnishncsa.c 176 Error 64: Type mismatch (arg. no. 1) (ptrs to qualification,signed/unsigned) _ vsb_bcat(ob[u], p + 4, strlen(p + 4)); varnishncsa.c 181 Error 64: Type mismatch (arg. no. 1) (ptrs to qualification,signed/unsigned) _ vsb_bcat(ob[u], p + 4, strlen(p + 4)); varnishncsa.c 186 Error 64: Type mismatch (arg. no. 1) (ptrs to qualification,signed/unsigned) _ ll[u].bogus_req = 1; varnishncsa.c 193 Error 115: struct/union not defined varnishncsa.c 193 Error 40: Undeclared identifier 'bogus_req' varnishncsa.c 193 Error 63: Expected an lvalue _ break; varnishncsa.c 195 Info 725: Expected positive indentation from line 148 varnishncsa.c 148 Info 830: Location cited in prior message _ vsb_bcat(ob[u], p + 4, strlen(p + 4)); varnishncsa.c 200 Error 64: Type mismatch (arg. no. 1) (ptrs to qualification,signed/unsigned) _ vsb_bcat(ob[u], p + 4, strlen(p + 4)); varnishncsa.c 207 Error 64: Type mismatch (arg. no. 1) (ptrs to qualification,signed/unsigned) _ ll[u].df_s = strdup(p + 4); varnishncsa.c 213 Error 115: struct/union not defined varnishncsa.c 213 Error 40: Undeclared identifier 'df_s' varnishncsa.c 213 Error 64: Type mismatch (arg. no. 1) (ptrs to qualification,signed/unsigned) varnishncsa.c 213 Error 63: Expected an lvalue _ ll[u].df_sfini = 1; varnishncsa.c 214 Error 115: struct/union not defined varnishncsa.c 214 Error 40: Undeclared identifier 'df_sfini' varnishncsa.c 214 Error 63: Expected an lvalue _ ll[u].df_U = strdup(p + 4); varnishncsa.c 225 Error 115: struct/union not defined varnishncsa.c 225 Error 40: Undeclared identifier 'df_U' varnishncsa.c 225 Error 64: Type mismatch (arg. no. 1) (ptrs to qualification,signed/unsigned) varnishncsa.c 225 Error 63: Expected an lvalue _ ll[u].df_U = ll[u].df_U + 12; varnishncsa.c 228 Error 115: struct/union not defined varnishncsa.c 228 Error 40: Undeclared identifier 'df_U' varnishncsa.c 228 Error 115: struct/union not defined varnishncsa.c 228 Error 40: Undeclared identifier 'df_U' varnishncsa.c 228 Error 63: Expected an lvalue _ ll[u].df_Ufini = 1; varnishncsa.c 229 Error 115: struct/union not defined varnishncsa.c 229 Error 40: Undeclared identifier 'df_Ufini' varnishncsa.c 229 Error 63: Expected an lvalue _ ll[u].df_R = strdup(p + 4); varnishncsa.c 232 Error 115: struct/union not defined varnishncsa.c 232 Error 40: Undeclared identifier 'df_R' varnishncsa.c 232 Error 64: Type mismatch (arg. no. 1) (ptrs to qualification,signed/unsigned) varnishncsa.c 232 Error 63: Expected an lvalue _ ll[u].df_R = ll[u].df_R + 9; varnishncsa.c 233 Error 115: struct/union not defined varnishncsa.c 233 Error 40: Undeclared identifier 'df_R' varnishncsa.c 233 Error 115: struct/union not defined varnishncsa.c 233 Error 40: Undeclared identifier 'df_R' varnishncsa.c 233 Error 63: Expected an lvalue _ ll[u].df_Rfini = 1; varnishncsa.c 234 Error 115: struct/union not defined varnishncsa.c 234 Error 40: Undeclared identifier 'df_Rfini' varnishncsa.c 234 Error 63: Expected an lvalue _ tmpPtra = strdup(p + 4); varnishncsa.c 243 Error 64: Type mismatch (arg. no. 1) (ptrs to qualification,signed/unsigned) _ ll[u].logline_time = make_timestring(tmpPtra); varnishncsa.c 244 Error 115: struct/union not defined varnishncsa.c 244 Error 40: Undeclared identifier 'logline_time' varnishncsa.c 244 Error 63: Expected an lvalue _ ll[u].df_rfini = 1; varnishncsa.c 246 Error 115: struct/union not defined varnishncsa.c 246 Error 40: Undeclared identifier 'df_rfini' varnishncsa.c 246 Error 63: Expected an lvalue _ ll[u].df_b = strdup(p + 4); varnishncsa.c 255 Error 115: struct/union not defined varnishncsa.c 255 Error 40: Undeclared identifier 'df_b' varnishncsa.c 255 Error 64: Type mismatch (arg. no. 1) (ptrs to qualification,signed/unsigned) varnishncsa.c 255 Error 63: Expected an lvalue _ ll[u].df_bfini = 1; varnishncsa.c 256 Error 115: struct/union not defined varnishncsa.c 256 Error 40: Undeclared identifier 'df_bfini' varnishncsa.c 256 Error 63: Expected an lvalue _ if (!atoi(ll[u].df_b)){ varnishncsa.c 257 Error 115: struct/union not defined varnishncsa.c 257 Error 40: Undeclared identifier 'df_b' _ ll[u].df_b = malloc(2); varnishncsa.c 258 Error 115: struct/union not defined varnishncsa.c 258 Error 40: Undeclared identifier 'df_b' varnishncsa.c 258 Error 63: Expected an lvalue _ ll[u].df_b[0] = '-'; varnishncsa.c 259 Error 115: struct/union not defined varnishncsa.c 259 Error 40: Undeclared identifier 'df_b' varnishncsa.c 259 Warning 409: Expecting a pointer or array varnishncsa.c 259 Error 63: Expected an lvalue _ ll[u].df_b[1] = '\0'; varnishncsa.c 260 Error 115: struct/union not defined varnishncsa.c 260 Error 40: Undeclared identifier 'df_b' varnishncsa.c 260 Warning 409: Expecting a pointer or array varnishncsa.c 260 Error 63: Expected an lvalue _ ll[u].df_hfini = 1; varnishncsa.c 271 Error 115: struct/union not defined varnishncsa.c 271 Error 40: Undeclared identifier 'df_hfini' varnishncsa.c 271 Error 63: Expected an lvalue _ if (ll[u].df_h[0] == '\0'){ varnishncsa.c 283 Error 115: struct/union not defined varnishncsa.c 283 Error 40: Undeclared identifier 'df_h' varnishncsa.c 283 Warning 409: Expecting a pointer or array _ tmpPtr = strchr(p + 4, ' '); varnishncsa.c 286 Error 64: Type mismatch (arg. no. 1) (ptrs to qualification,signed/unsigned) _ j = strlen(p + 4) - strlen(tmpPtr); // length of IP varnishncsa.c 287 Error 64: Type mismatch (arg. no. 1) (ptrs to qualification,signed/unsigned) varnishncsa.c 287 Error 64: Type mismatch (arg. no. 1) (ptrs to qualification,signed/unsigned) _ strncpy(ll[u].df_h, p + 4, j); varnishncsa.c 288 Error 115: struct/union not defined varnishncsa.c 288 Error 40: Undeclared identifier 'df_h' varnishncsa.c 288 Error 64: Type mismatch (arg. no. 2) (ptrs to qualification,signed/unsigned) _ ll[u].df_h[j] = '\0'; // put on a NULL at end of buffer. varnishncsa.c 289 Error 115: struct/union not defined varnishncsa.c 289 Error 40: Undeclared identifier 'df_h' varnishncsa.c 289 Warning 409: Expecting a pointer or array varnishncsa.c 289 Error 63: Expected an lvalue _ if (ll[u].df_rfini) { varnishncsa.c 305 Error 115: struct/union not defined varnishncsa.c 305 Error 40: Undeclared identifier 'df_rfini' _ strftime (temp_time, 28, "[%d/%b/%Y:%X %z] ", ll[u].logline_time); varnishncsa.c 314 Error 115: struct/union not defined varnishncsa.c 314 Error 40: Undeclared identifier 'logline_time' _ if (ll[u].df_h[0] == '\0' || ll[u].bogus_req){ varnishncsa.c 316 Error 115: struct/union not defined varnishncsa.c 316 Error 40: Undeclared identifier 'df_h' varnishncsa.c 316 Warning 409: Expecting a pointer or array varnishncsa.c 316 Error 115: struct/union not defined varnishncsa.c 316 Error 40: Undeclared identifier 'bogus_req' _ ll[u].bogus_req = 0; varnishncsa.c 317 Error 115: struct/union not defined varnishncsa.c 317 Error 40: Undeclared identifier 'bogus_req' varnishncsa.c 317 Error 63: Expected an lvalue _ printf("%s - - %s ", ll[u].df_h, temp_time ); varnishncsa.c 321 Error 115: struct/union not defined varnishncsa.c 321 Error 40: Undeclared identifier 'df_h' varnishncsa.c 321 Warning 560: argument no. 2 should be a pointer _ printf(" %s %s ", ll[u].df_s, ll[u].df_b, ll[u].df_R); varnishncsa.c 324 Error 115: struct/union not defined varnishncsa.c 324 Error 40: Undeclared identifier 'df_s' varnishncsa.c 324 Error 115: struct/union not defined varnishncsa.c 324 Error 40: Undeclared identifier 'df_b' varnishncsa.c 324 Error 115: struct/union not defined varnishncsa.c 324 Error 40: Undeclared identifier 'df_R' varnishncsa.c 324 Info 719: Too many arguments for format (1 too many) varnishncsa.c 324 Warning 560: argument no. 2 should be a pointer varnishncsa.c 324 Warning 560: argument no. 3 should be a pointer _ if (ll[u].df_Rfini){ varnishncsa.c 325 Error 115: struct/union not defined varnishncsa.c 325 Error 40: Undeclared identifier 'df_Rfini' _ printf(" \"%s\" ", ll[u].df_R); varnishncsa.c 326 Error 115: struct/union not defined varnishncsa.c 326 Error 40: Undeclared identifier 'df_R' varnishncsa.c 326 Warning 560: argument no. 2 should be a pointer _ if (ll[u].df_Ufini){ varnishncsa.c 332 Error 115: struct/union not defined varnishncsa.c 332 Error 40: Undeclared identifier 'df_Ufini' _ printf(" \"%s\" ", ll[u].df_U); varnishncsa.c 333 Error 115: struct/union not defined varnishncsa.c 333 Error 40: Undeclared identifier 'df_U' varnishncsa.c 333 Warning 560: argument no. 2 should be a pointer _ ll[u].df_rfini = 0; varnishncsa.c 346 Error 115: struct/union not defined varnishncsa.c 346 Error 40: Undeclared identifier 'df_rfini' varnishncsa.c 346 Error 63: Expected an lvalue _ if (ll[u].df_sfini){ varnishncsa.c 351 Error 115: struct/union not defined varnishncsa.c 351 Error 40: Undeclared identifier 'df_sfini' _ free(ll[u].df_s); varnishncsa.c 352 Error 115: struct/union not defined varnishncsa.c 352 Error 40: Undeclared identifier 'df_s' _ ll[u].df_sfini = 0; varnishncsa.c 353 Error 115: struct/union not defined varnishncsa.c 353 Error 40: Undeclared identifier 'df_sfini' varnishncsa.c 353 Error 63: Expected an lvalue _ jalla = strlen(ll[u].df_s); varnishncsa.c 355 Error 115: struct/union not defined varnishncsa.c 355 Error 40: Undeclared identifier 'df_s' _ if (ll[u].df_bfini){ varnishncsa.c 359 Error 115: struct/union not defined varnishncsa.c 359 Error 40: Undeclared identifier 'df_bfini' _ free(ll[u].df_b); varnishncsa.c 360 Error 115: struct/union not defined varnishncsa.c 360 Error 40: Undeclared identifier 'df_b' _ ll[u].df_bfini = 0; varnishncsa.c 361 Error 115: struct/union not defined varnishncsa.c 361 Error 40: Undeclared identifier 'df_bfini' varnishncsa.c 361 Error 63: Expected an lvalue _ jalla = strlen(ll[u].df_b); varnishncsa.c 363 Error 115: struct/union not defined varnishncsa.c 363 Error 40: Undeclared identifier 'df_b' _ if (ll[u].df_Ufini){ varnishncsa.c 368 Error 115: struct/union not defined varnishncsa.c 368 Error 40: Undeclared identifier 'df_Ufini' _ ll[u].df_U = ll[u].df_U - 12; varnishncsa.c 369 Error 115: struct/union not defined varnishncsa.c 369 Error 40: Undeclared identifier 'df_U' varnishncsa.c 369 Error 115: struct/union not defined varnishncsa.c 369 Error 40: Undeclared identifier 'df_U' varnishncsa.c 369 Error 63: Expected an lvalue _ free(ll[u].df_U); varnishncsa.c 370 Error 115: struct/union not defined varnishncsa.c 370 Error 40: Undeclared identifier 'df_U' _ ll[u].df_Ufini = 0; varnishncsa.c 371 Error 115: struct/union not defined varnishncsa.c 371 Error 40: Undeclared identifier 'df_Ufini' varnishncsa.c 371 Error 63: Expected an lvalue _ ll[u].df_UN[0] = '\0'; varnishncsa.c 372 Error 115: struct/union not defined varnishncsa.c 372 Error 40: Undeclared identifier 'df_UN' varnishncsa.c 372 Warning 409: Expecting a pointer or array varnishncsa.c 372 Error 63: Expected an lvalue _ ll[u].df_U[0] = '\0'; varnishncsa.c 373 Error 115: struct/union not defined varnishncsa.c 373 Error 40: Undeclared identifier 'df_U' varnishncsa.c 373 Warning 409: Expecting a pointer or array varnishncsa.c 373 Error 63: Expected an lvalue _ jalla = strlen(ll[u].df_U); varnishncsa.c 375 Error 115: struct/union not defined varnishncsa.c 375 Error 40: Undeclared identifier 'df_U' _ if (ll[u].df_Rfini){ varnishncsa.c 379 Error 115: struct/union not defined varnishncsa.c 379 Error 40: Undeclared identifier 'df_Rfini' _ ll[u].df_R = ll[u].df_R - 9; varnishncsa.c 380 Error 115: struct/union not defined varnishncsa.c 380 Error 40: Undeclared identifier 'df_R' varnishncsa.c 380 Error 115: struct/union not defined varnishncsa.c 380 Error 40: Undeclared identifier 'df_R' varnishncsa.c 380 Error 63: Expected an lvalue _ free(ll[u].df_R); varnishncsa.c 381 Error 115: struct/union not defined varnishncsa.c 381 Error 40: Undeclared identifier 'df_R' _ ll[u].df_R[0] = '\0'; varnishncsa.c 382 Error 115: struct/union not defined varnishncsa.c 382 Error 40: Undeclared identifier 'df_R' varnishncsa.c 382 Warning 409: Expecting a pointer or array varnishncsa.c 382 Error 63: Expected an lvalue _ ll[u].df_Rfini = 0; varnishncsa.c 383 Error 115: struct/union not defined varnishncsa.c 383 Error 40: Undeclared identifier 'df_Rfini' varnishncsa.c 383 Error 63: Expected an lvalue _ jalla = strlen(ll[u].df_R); varnishncsa.c 385 Error 115: struct/union not defined varnishncsa.c 385 Error 40: Undeclared identifier 'df_R' _ if (ll[u].df_hfini) { varnishncsa.c 415 Error 115: struct/union not defined varnishncsa.c 415 Error 40: Undeclared identifier 'df_hfini' _ ll[u].df_h[0] = '\0'; varnishncsa.c 419 Error 115: struct/union not defined varnishncsa.c 419 Error 40: Undeclared identifier 'df_h' varnishncsa.c 419 Warning 409: Expecting a pointer or array varnishncsa.c 419 Error 63: Expected an lvalue _ ll[u].df_hfini = 0; varnishncsa.c 421 Error 115: struct/union not defined varnishncsa.c 421 Error 40: Undeclared identifier 'df_hfini' varnishncsa.c 421 Error 63: Expected an lvalue _ } varnishncsa.c 428 Warning 550: Symbol 'jalla' (line 310) not accessed varnishncsa.c 310 Info 830: Location cited in prior message _ } varnishncsa.c 431 Warning 529: Symbol 'jalla2' (line 133) not subsequently referenced varnishncsa.c 133 Info 830: Location cited in prior message _ } varnishncsa.c 431 Info 715: Symbol 'w_opt' (line 117) not referenced varnishncsa.c 117 Info 830: Location cited in prior message _ } varnishncsa.c 431 Warning 550: Symbol 'v' (line 119) not accessed varnishncsa.c 119 Info 830: Location cited in prior message _ } varnishncsa.c 431 Warning 550: Symbol 'w' (line 119) not accessed varnishncsa.c 119 Info 830: Location cited in prior message _ } varnishncsa.c 431 Warning 529: Symbol 'df_b_set' (line 131) not subsequently referenced varnishncsa.c 131 Info 830: Location cited in prior message _ } varnishncsa.c 431 Warning 529: Symbol 'df_s_set' (line 130) not subsequently referenced varnishncsa.c 130 Info 830: Location cited in prior message _ } varnishncsa.c 506 Warning 550: Symbol 'u' (line 446) not accessed varnishncsa.c 446 Info 830: Location cited in prior message --- Wrap-up for Module: varnishncsa.c Warning 551: Symbol 'll' (line 63, file varnishncsa.c) not accessed varnishncsa.c 63 Info 830: Location cited in prior message /// Start of Pass 2 /// --- Module: varnishncsa.c (C) /// Start of Pass 3 /// --- Module: varnishncsa.c (C) --- Global Wrap-up Info 755: global macro 'VERSION' (line 150, file ../../config.h) not referenced ../../config.h 150 Info 830: Location cited in prior message Info 755: global macro 'HAVE_ASPRINTF' (line 5, file ../../config.h) not referenced ../../config.h 5 Info 830: Location cited in prior message Info 755: global macro 'HAVE_DECL_STRERROR_R' (line 9, file ../../config.h) not referenced ../../config.h 9 Info 830: Location cited in prior message Info 755: global macro 'HAVE_DLFCN_H' (line 12, file ../../config.h) not referenced ../../config.h 12 Info 830: Location cited in prior message Info 755: global macro 'HAVE_INTTYPES_H' (line 21, file ../../config.h) not referenced ../../config.h 21 Info 830: Location cited in prior message Info 755: global macro 'HAVE_KQUEUE' (line 24, file ../../config.h) not referenced ../../config.h 24 Info 830: Location cited in prior message Info 755: global macro 'HAVE_MD5_H' (line 27, file ../../config.h) not referenced ../../config.h 27 Info 830: Location cited in prior message Info 755: global macro 'HAVE_MEMORY_H' (line 30, file ../../config.h) not referenced ../../config.h 30 Info 830: Location cited in prior message Info 755: global macro 'HAVE_NETINET_IN_H' (line 33, file ../../config.h) not referenced ../../config.h 33 Info 830: Location cited in prior message Info 755: global macro 'HAVE_POLL' (line 36, file ../../config.h) not referenced ../../config.h 36 Info 830: Location cited in prior message Info 755: global macro 'HAVE_SENDFILE' (line 39, file ../../config.h) not referenced ../../config.h 39 Info 830: Location cited in prior message Info 755: global macro 'HAVE_SETPROCTITLE' (line 42, file ../../config.h) not referenced ../../config.h 42 Info 830: Location cited in prior message Info 755: global macro 'HAVE_SOCKET' (line 45, file ../../config.h) not referenced ../../config.h 45 Info 830: Location cited in prior message Info 755: global macro 'HAVE_SRANDOMDEV' (line 48, file ../../config.h) not referenced ../../config.h 48 Info 830: Location cited in prior message Info 755: global macro 'HAVE_STDDEF_H' (line 51, file ../../config.h) not referenced ../../config.h 51 Info 830: Location cited in prior message Info 755: global macro 'HAVE_STDINT_H' (line 54, file ../../config.h) not referenced ../../config.h 54 Info 830: Location cited in prior message Info 755: global macro 'HAVE_STDLIB_H' (line 57, file ../../config.h) not referenced ../../config.h 57 Info 830: Location cited in prior message Info 755: global macro 'HAVE_STRERROR' (line 60, file ../../config.h) not referenced ../../config.h 60 Info 830: Location cited in prior message Info 755: global macro 'HAVE_STRERROR_R' (line 63, file ../../config.h) not referenced ../../config.h 63 Info 830: Location cited in prior message Info 755: global macro 'HAVE_STRINGS_H' (line 66, file ../../config.h) not referenced ../../config.h 66 Info 830: Location cited in prior message Info 755: global macro 'HAVE_STRING_H' (line 69, file ../../config.h) not referenced ../../config.h 69 Info 830: Location cited in prior message Info 755: global macro 'HAVE_STRLCAT' (line 72, file ../../config.h) not referenced ../../config.h 72 Info 830: Location cited in prior message Info 755: global macro 'HAVE_STRLCPY' (line 75, file ../../config.h) not referenced ../../config.h 75 Info 830: Location cited in prior message Info 755: global macro 'HAVE_STRPTIME' (line 81, file ../../config.h) not referenced ../../config.h 81 Info 830: Location cited in prior message Info 755: global macro 'HAVE_STRUCT_SOCKADDR_SA_LEN' (line 84, file ../../config.h) not referenced ../../config.h 84 Info 830: Location cited in prior message Info 755: global macro 'HAVE_STRVIS' (line 87, file ../../config.h) not referenced ../../config.h 87 Info 830: Location cited in prior message Info 755: global macro 'HAVE_STRVISX' (line 90, file ../../config.h) not referenced ../../config.h 90 Info 830: Location cited in prior message Info 755: global macro 'HAVE_SYS_SOCKET_H' (line 93, file ../../config.h) not referenced ../../config.h 93 Info 830: Location cited in prior message Info 755: global macro 'HAVE_SYS_STAT_H' (line 96, file ../../config.h) not referenced ../../config.h 96 Info 830: Location cited in prior message Info 755: global macro 'HAVE_SYS_TYPES_H' (line 99, file ../../config.h) not referenced ../../config.h 99 Info 830: Location cited in prior message Info 755: global macro 'HAVE_SYS_WAIT_H' (line 105, file ../../config.h) not referenced ../../config.h 105 Info 830: Location cited in prior message Info 755: global macro 'HAVE_UNISTD_H' (line 108, file ../../config.h) not referenced ../../config.h 108 Info 830: Location cited in prior message Info 755: global macro 'HAVE_VASPRINTF' (line 111, file ../../config.h) not referenced ../../config.h 111 Info 830: Location cited in prior message Info 755: global macro 'HAVE_VIS' (line 114, file ../../config.h) not referenced ../../config.h 114 Info 830: Location cited in prior message Info 755: global macro 'HAVE_VPRINTF' (line 117, file ../../config.h) not referenced ../../config.h 117 Info 830: Location cited in prior message Info 755: global macro 'RETSIGTYPE' (line 138, file ../../config.h) not referenced ../../config.h 138 Info 830: Location cited in prior message Info 755: global macro 'STDC_HEADERS' (line 141, file ../../config.h) not referenced ../../config.h 141 Info 830: Location cited in prior message Info 755: global macro 'PACKAGE' (line 120, file ../../config.h) not referenced ../../config.h 120 Info 830: Location cited in prior message Info 755: global macro 'PACKAGE_BUGREPORT' (line 123, file ../../config.h) not referenced ../../config.h 123 Info 830: Location cited in prior message Info 755: global macro 'PACKAGE_NAME' (line 126, file ../../config.h) not referenced ../../config.h 126 Info 830: Location cited in prior message Info 755: global macro 'PACKAGE_STRING' (line 129, file ../../config.h) not referenced ../../config.h 129 Info 830: Location cited in prior message Info 755: global macro 'PACKAGE_TARNAME' (line 132, file ../../config.h) not referenced ../../config.h 132 Info 830: Location cited in prior message Info 755: global macro 'PACKAGE_VERSION' (line 135, file ../../config.h) not referenced ../../config.h 135 Info 830: Location cited in prior message Info 755: global macro 'TIME_WITH_SYS_TIME' (line 147, file ../../config.h) not referenced ../../config.h 147 Info 830: Location cited in prior message -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From anders at fupp.net Wed Sep 13 08:10:26 2006 From: anders at fupp.net (Anders Nordby) Date: Wed, 13 Sep 2006 10:10:26 +0200 Subject: NUUG Oslo 2006-09-19: Releaseparty, the Varnish HTTP accelerator In-Reply-To: <20060913075847.GA76535@totem.fix.no> References: <20060913075847.GA76535@totem.fix.no> Message-ID: <20060913081026.GB90185@totem.fix.no> Hi, The meeting has been announced. See you there. On Wed, Sep 13, 2006 at 09:58:47AM +0200, aktive at nuug.no wrote: > Norwegian Unix User Group (NUUG) inviterer til medlemsm?te i NUUG Oslo. > Alle medlemmer og ikke-medlemmer er velkomne. Arrangementet er gratis. > Etter foredraget (fra ca. 20:30) blir det sosialt samv?r. > > Tid: Tirsdag 19. september 2006 kl. 18:30 > Sted: H?gskolen i Oslo, HiO, - Vika, Cort Adelersgt. 30. > > M?nedens tema er "Releaseparty, the Varnish HTTP accelerator": > > VG sponsored the creation of a web-accellerator called "Varnish" > because Squid was too slow for them. Varnish is being developed by > Poul-Henning Kamp and the Norwegian Linux consultancy Linpro. This > is the releaseparty for version 1.0. > > The first half of the talk will introduce Varnish and present some > of the novel features it brings to the business of web-serving. > > The second half of the talk, using Varnish as the example, will show > ways to get the most performance out of modern hardware and operating > systems. > > Poul-Henning Kamp has been mucking about with computers long enough to > become an old fart. When he doesn't renovate the FreeBSD kernel, he > solves problems for customers at reasonable hourly rates. > > Inngangen til h?yskolen er 15 meter fra trikkeholdeplassen Vikatorvet. > Parkeringsplass finnes mellom skolen og Rusel?kka skole, men det er > sv?rt begrenset plass p? denne. > > Mer informasjon om hvordan du finner frem p?: > http://www.hio.no/content/view/full/36866. > > Heng gjerne opp reklameplakaten for dette arrangementet p? din > arbeidsplass, studieplass eller hvor det m?tte passe. Vi har den > tilgjengelig b?de i PostScript-, PDF- og LATEX-format: > > http://www.nuug.no/aktiviteter/20060919-varnish/banner.ps > http://www.nuug.no/aktiviteter/20060919-varnish/banner.pdf > http://www.nuug.no/aktiviteter/20060919-varnish/banner.tex > > Vel m?tt! > > Mvh, > > -- > Anders - NUUG-aktivist > -- > Arkiv over denne mailinglisten: http://www.nobug.no/arkiv/, > brukernavn/passord bsd/KvbS08. -- Anders. From des at linpro.no Wed Sep 13 08:29:58 2006 From: des at linpro.no (Dag-Erling =?iso-8859-1?Q?Sm=F8rgrav?=) Date: Wed, 13 Sep 2006 10:29:58 +0200 Subject: Linux: rev 949 trunk is stable..not sure about others References: <3969.1158088842@critter.freebsd.dk> Message-ID: "Poul-Henning Kamp" writes: > "xing at litespeedtech.com" writes: > > from revision 957 to 970, builds on linux are broken in "make" stage: > That's just a missing ')' No, that's the sendfile stuff. DES -- Dag-Erling Sm?rgrav Senior Software Developer Linpro AS - www.linpro.no From xing at litespeedtech.com Wed Sep 13 16:37:23 2006 From: xing at litespeedtech.com (xing at litespeedtech.com) Date: Wed, 13 Sep 2006 09:37:23 -0700 Subject: Linux: rev 949 trunk is stable..not sure about others In-Reply-To: <3969.1158088842@critter.freebsd.dk> References: <3969.1158088842@critter.freebsd.dk> Message-ID: <450833C3.3060300@litespeedtech.com> Couldn't test changes since rev 949 and before 956 to isolate problem. All of the revisions in between compile with error: vcl_compile.c:677: error: syntax error before "__unused" vcl_compile.c: In function `HeaderVar': vcl_compile.c:685: error: `t' undeclared (first use in this function) vcl_compile.c:685: error: (Each undeclared identifier is reported only once vcl_compile.c:685: error: for each function it appears in.) vcl_compile.c:692: error: `vh' undeclared (first use in this function) vcl_compile.c: At top level: vcl_compile.c:1889: error: syntax error before "__unused" vcl_compile.c: In function `VCC_T_render': vcl_compile.c:1893: error: `args' undeclared (first use in this function) vcl_compile.c:1894: error: `f' undeclared (first use in this function) vcl_compile.c: At top level: vcl_compile.c:1899: error: syntax error before "__unused" vcl_compile.c: In function `VCC_T_arginfo': vcl_compile.c:1902: error: `n' undeclared (first use in this function) vcl_compile.c:1903: error: `argtypes' undeclared (first use in this function) make[3]: *** [vcl_compile.lo] Error 1 From phk at phk.freebsd.dk Thu Sep 14 06:29:37 2006 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Thu, 14 Sep 2006 06:29:37 +0000 Subject: offline today Message-ID: <25459.1158215377@critter.freebsd.dk> Call me at my mobile if anything explodes. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From xing at litespeedtech.com Fri Sep 15 18:12:57 2006 From: xing at litespeedtech.com (xing at litespeedtech.com) Date: Fri, 15 Sep 2006 11:12:57 -0700 Subject: trunk 995..confirmed linux sendfile() fix Message-ID: <450AED29.2020804@litespeedtech.com> Upgraded 2 live varnishd instances from trunk 977 (commented out sendfile support) to rev 977 and I can confirm the sendfile() bug that existed in linux previously has been fixed. Keep up the great progress to a 1.0 launch party. =) Xing From xing at litespeedtech.com Fri Sep 15 18:18:53 2006 From: xing at litespeedtech.com (xing at litespeedtech.com) Date: Fri, 15 Sep 2006 11:18:53 -0700 Subject: varnishstat and 2 instances of varnishd Message-ID: <450AEE8D.4020906@litespeedtech.com> Since yesterday I have been running 2 instances of varnishd on the same server. So far so good. My question is, I'm not sure how to direct varnishstat to grab/display stats from a specific varnishd instance. --help only reveal -cV args. Xing From phk at phk.freebsd.dk Fri Sep 15 18:25:04 2006 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Fri, 15 Sep 2006 18:25:04 +0000 Subject: varnishstat and 2 instances of varnishd In-Reply-To: Your message of "Fri, 15 Sep 2006 11:18:53 MST." <450AEE8D.4020906@litespeedtech.com> Message-ID: <10031.1158344704@critter.freebsd.dk> In message <450AEE8D.4020906 at litespeedtech.com>, "xing at litespeedtech.com" write s: >Since yesterday I have been running 2 instances of varnishd on the same >server. > >So far so good. My question is, I'm not sure how to direct varnishstat >to grab/display stats from a specific varnishd instance. --help only >reveal -cV args. Ohh, uhm... The shared memory file is currently (rather haphazardly) called "tmp/_.vsl". At some point I planned on revisiting that but I have not thought much about it. Just including the listen port is not enough as you could have two varnishes listening on specific IP# but same port. Including the entire listen address seems needlessly cumbersome. I guess assigning some kind of instance numbers is the way to go... The instance number would have to be specified to all the tools then. Ideas welcome. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From phk at phk.freebsd.dk Fri Sep 15 18:26:25 2006 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Fri, 15 Sep 2006 18:26:25 +0000 Subject: trunk 995..confirmed linux sendfile() fix In-Reply-To: Your message of "Fri, 15 Sep 2006 11:12:57 MST." <450AED29.2020804@litespeedtech.com> Message-ID: <10041.1158344785@critter.freebsd.dk> In message <450AED29.2020804 at litespeedtech.com>, "xing at litespeedtech.com" write s: >Upgraded 2 live varnishd instances from trunk 977 (commented out >sendfile support) to rev 977 and I can confirm the sendfile() bug that >existed in linux previously has been fixed. Great! -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From xing at litespeedtech.com Sun Sep 17 00:52:18 2006 From: xing at litespeedtech.com (xing at litespeedtech.com) Date: Sat, 16 Sep 2006 17:52:18 -0700 Subject: error auto-restart problem.. Message-ID: <450C9C42.2050809@litespeedtech.com> child said (2, 26344): <> Child said (2, 26344): <> Cache child died pid=26344 status=0x6 Clean child Child cleaned start child pid 26358 Pushing vcls failed: CLI communication error unlink /tmp/vcl.XXqufVqH Usually varnish automatically restarts the cache process when it encounters the above message. However, it wasn't able to today due to "CLI communication error". Running Trunk 995 and the error happened on the second instance of varnishd. Likely result of me running 2 instances on the same server. Xing From phk at phk.freebsd.dk Sun Sep 17 09:09:50 2006 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Sun, 17 Sep 2006 09:09:50 +0000 Subject: error auto-restart problem.. In-Reply-To: Your message of "Sat, 16 Sep 2006 17:52:18 MST." <450C9C42.2050809@litespeedtech.com> Message-ID: <77774.1158484190@critter.freebsd.dk> In message <450C9C42.2050809 at litespeedtech.com>, "xing at litespeedtech.com" write s: >Pushing vcls failed: >CLI communication error >unlink /tmp/vcl.XXqufVqH I have just committed a change to make sure we get the error messages from the child in this case. If you can reproduce it I'm very interested in what it said. >[...] Likely result of me running 2 instances on the same server. I'm not so sure. But regarding the multiple-instance thing: I think my inclination is that each instance of varnish gets a user chosen "identifier". The only requirement I have is that it can be part of a filename (ie: no '/' etc). You will of course have to specify that identifier to all programs that need access to the shared memory. Is that workable ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From phk at phk.freebsd.dk Sun Sep 17 10:31:07 2006 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Sun, 17 Sep 2006 10:31:07 +0000 Subject: license edit ? Message-ID: <78019.1158489067@critter.freebsd.dk> Do we need to do a bulk edit of all source files to insert the BSD license ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From andersb at vgnett.no Sun Sep 17 13:25:58 2006 From: andersb at vgnett.no (Anders Berg) Date: Sun, 17 Sep 2006 15:25:58 +0200 (CEST) Subject: license edit ? In-Reply-To: <78019.1158489067@critter.freebsd.dk> References: <78019.1158489067@critter.freebsd.dk> Message-ID: <50718.85.167.14.93.1158499558.squirrel@denise.vg.no> > > Do we need to do a bulk edit of all source files to insert > the BSD license ? I think so. DES probably has a plan for how/what to do? Anders Berg > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > phk at FreeBSD.ORG | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by > incompetence. > _______________________________________________ > varnish-dev mailing list > varnish-dev at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-dev > > From xing at litespeedtech.com Sun Sep 17 17:20:49 2006 From: xing at litespeedtech.com (xing at litespeedtech.com) Date: Sun, 17 Sep 2006 10:20:49 -0700 Subject: error auto-restart problem.. In-Reply-To: <77774.1158484190@critter.freebsd.dk> References: <77774.1158484190@critter.freebsd.dk> Message-ID: <450D83F1.4060702@litespeedtech.com> >> Pushing vcls failed: >> CLI communication error >> unlink /tmp/vcl.XXqufVqH > > I have just committed a change to make sure we get the error messages > from the child in this case. It only happened once in the last 72+ hours. Hopefully it will resurface soon. > But regarding the multiple-instance thing: > > I think my inclination is that each instance of varnish gets a > user chosen "identifier". The only requirement I have is that > it can be part of a filename (ie: no '/' etc). This sounds good. I really do not have any preference on how instances should be tagged/identified as. > You will of course have to specify that identifier to all programs > that need access to the shared memory. That's fine. I will likely use the domain name as identifier. The custom identifier string setup is easier than using the backend ip:port and much easier to type out on the terminal so I'm all for it. Xing From des at linpro.no Mon Sep 18 07:50:08 2006 From: des at linpro.no (Dag-Erling =?iso-8859-1?Q?Sm=F8rgrav?=) Date: Mon, 18 Sep 2006 09:50:08 +0200 Subject: license edit ? References: <78019.1158489067@critter.freebsd.dk> <50718.85.167.14.93.1158499558.squirrel@denise.vg.no> Message-ID: "Anders Berg" writes: > Poul-Henning Kamp writes: > > Do we need to do a bulk edit of all source files to insert > > the BSD license ? > I think so. DES probably has a plan for how/what to do? I wasn't planning to, since we have LICENSE in the root, but maybe we should. I also think I'll put out a 0.9.9 release later today to test the releng process. DES -- Dag-Erling Sm?rgrav Senior Software Developer Linpro AS - www.linpro.no From phk at phk.freebsd.dk Mon Sep 18 07:56:51 2006 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Mon, 18 Sep 2006 07:56:51 +0000 Subject: license edit ? In-Reply-To: Your message of "Mon, 18 Sep 2006 09:50:08 +0200." Message-ID: <24588.1158566211@critter.freebsd.dk> In message , Dag-Erling =?iso-8859-1?Q?Sm=F8rgra v?= writes: >"Anders Berg" writes: >> Poul-Henning Kamp writes: >> > Do we need to do a bulk edit of all source files to insert >> > the BSD license ? >> I think so. DES probably has a plan for how/what to do? > >I wasn't planning to, since we have LICENSE in the root, but maybe we >should. I also think I'll put out a 0.9.9 release later today to test >the releng process. With the changes to the backend code today, I'm sort of in a miniature "code freeze" before the release now. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From des at linpro.no Mon Sep 18 08:00:55 2006 From: des at linpro.no (Dag-Erling =?iso-8859-1?Q?Sm=F8rgrav?=) Date: Mon, 18 Sep 2006 10:00:55 +0200 Subject: license edit ? References: <24588.1158566211@critter.freebsd.dk> Message-ID: "Poul-Henning Kamp" writes: > With the changes to the backend code today, I'm sort of in a miniature > "code freeze" before the release now. Good. I need to write some more documentation, and we should run another full-scale test today, but other than that I think we're good to go. Anders, when will the second machine be ready? DES -- Dag-Erling Sm?rgrav Senior Software Developer Linpro AS - www.linpro.no From andersb at vgnett.no Mon Sep 18 13:07:14 2006 From: andersb at vgnett.no (Anders Berg) Date: Mon, 18 Sep 2006 15:07:14 +0200 (CEST) Subject: license edit ? In-Reply-To: References: <24588.1158566211@critter.freebsd.dk> Message-ID: <2070.193.69.165.4.1158584834.squirrel@denise.vg.no> > "Poul-Henning Kamp" writes: >> With the changes to the backend code today, I'm sort of in a miniature >> "code freeze" before the release now. > > Good. I need to write some more documentation, and we should run > another full-scale test today, but other than that I think we're good > to go. Anders, when will the second machine be ready? Hi, have ordered 1 dual AMD machine with CURRENT, and 1 single Intel with Ubuntu. Hopefully by the end of the day. On wednesday, should we call the release Varnish 1.0-RC1? Ab > DES > -- > Dag-Erling Sm?rgrav > Senior Software Developer > Linpro AS - www.linpro.no > _______________________________________________ > varnish-dev mailing list > varnish-dev at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-dev > > From des at linpro.no Mon Sep 18 13:15:04 2006 From: des at linpro.no (Dag-Erling =?iso-8859-1?Q?Sm=F8rgrav?=) Date: Mon, 18 Sep 2006 15:15:04 +0200 Subject: license edit ? References: <24588.1158566211@critter.freebsd.dk> <2070.193.69.165.4.1158584834.squirrel@denise.vg.no> Message-ID: "Anders Berg" writes: > On wednesday, should we call the release Varnish 1.0-RC1? No, just 1.0. DES -- Dag-Erling Sm?rgrav Senior Software Developer Linpro AS - www.linpro.no From xing at litespeedtech.com Mon Sep 18 16:16:48 2006 From: xing at litespeedtech.com (xing at litespeedtech.com) Date: Mon, 18 Sep 2006 09:16:48 -0700 Subject: error auto-restart problem.. In-Reply-To: <77774.1158484190@critter.freebsd.dk> References: <77774.1158484190@critter.freebsd.dk> Message-ID: <450EC670.5010605@litespeedtech.com> The error happened again on instance 2 with the new build. The only meaningful config diff between instance 1 and instance 2 is that 2 is started with -s file,/raid0/varn.fp,10G Fatal error at the end. Cache child died pid=1688 status=0x6 Clean child Child cleaned start child pid 1699 Child said (2, 1699): <> Child said (2, 1699): <> Cache child died pid=1699 status=0x6 Clean child Child cleaned start child pid 1730 Child said (2, 1730): <> Child said (2, 1730): <> Cache child died pid=1730 status=0x6 Clean child Child cleaned start child pid 1740 Cache child died pid=1740 status=0x6 Clean child Child cleaned start child pid 1749 Child said (2, 1749): <> Cache child died pid=1749 status=0x6 Clean child Child cleaned start child pid 1760 Child said (2, 1760): <> Cache child died pid=1760 status=0x6 Clean child Child cleaned start child pid 1771 Pushing vcls failed: CLI communication error unlink /tmp/vcl.XXxgVLp9 Poul-Henning Kamp wrote: > In message <450C9C42.2050809 at litespeedtech.com>, "xing at litespeedtech.com" write > s: > >> Pushing vcls failed: >> CLI communication error >> unlink /tmp/vcl.XXqufVqH > > I have just committed a change to make sure we get the error messages > from the child in this case. > > If you can reproduce it I'm very interested in what it said. > >> [...] Likely result of me running 2 instances on the same server. > > I'm not so sure. > > But regarding the multiple-instance thing: > > I think my inclination is that each instance of varnish gets a > user chosen "identifier". The only requirement I have is that > it can be part of a filename (ie: no '/' etc). > > You will of course have to specify that identifier to all programs > that need access to the shared memory. > > Is that workable ? > From phk at phk.freebsd.dk Mon Sep 18 19:02:45 2006 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Mon, 18 Sep 2006 19:02:45 +0000 Subject: error auto-restart problem.. In-Reply-To: Your message of "Mon, 18 Sep 2006 09:16:48 MST." <450EC670.5010605@litespeedtech.com> Message-ID: <41717.1158606165@critter.freebsd.dk> In message <450EC670.5010605 at litespeedtech.com>, "xing at litespeedtech.com" write s: >The error happened again on instance 2 with the new build. The only >meaningful config diff between instance 1 and instance 2 is that 2 is >started with -s file,/raid0/varn.fp,10G >Child said (2, 1699): <cache_fetc > h.c line 318: > Condition(i == 0) not true. > errno = 104 (Connection reset by peer) This was a TCP connection to the backend which got broken. I need to handle that properly. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From mnot at yahoo-inc.com Wed Sep 20 18:23:25 2006 From: mnot at yahoo-inc.com (Mark Nottingham) Date: Wed, 20 Sep 2006 11:23:25 -0700 Subject: Testing Varnish Message-ID: <11A48748-CDA5-4185-9ECB-A6248A7351C3@yahoo-inc.com> Hi, The good news - my initial testing on a single 3Ghz Xeon (HT) processor Varnish box indicates that it tops out at about 18,000 1k responses a second, and it seems reasonable under overload. This compares very favourably to many other implementations, and is much improved from the last release. The bad news - testing for HTTP conformance with Co-Advisor finds a *lot* of violations. However, I'm just testing stock Varnish with no configuration (as that's not documented! ;). Can anyone comment on the state of Varnish's HTTP conformance? Cheers, P.S. I can't provide full results, but I believe Measurement Factory gives favourable terms to Open Source projects; you might want to talk to them. -- Mark Nottingham mnot at yahoo-inc.com From phk at phk.freebsd.dk Wed Sep 20 20:35:58 2006 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Wed, 20 Sep 2006 20:35:58 +0000 Subject: Testing Varnish In-Reply-To: Your message of "Wed, 20 Sep 2006 11:23:25 MST." <11A48748-CDA5-4185-9ECB-A6248A7351C3@yahoo-inc.com> Message-ID: <54456.1158784558@critter.freebsd.dk> In message <11A48748-CDA5-4185-9ECB-A6248A7351C3 at yahoo-inc.com>, Mark Nottingha m writes: >The bad news - testing for HTTP conformance with Co-Advisor coad.measurement-factory.com/> finds a *lot* of violations. However, >I'm just testing stock Varnish with no configuration (as that's not >documented! ;). Can anyone comment on the state of Varnish's HTTP >conformance? Have you tested Varnish as a proxy or as an origin server ? Varnish is not a proxy in the RFC2616 sense. Varnish is an origin server. An origin server which gets its contents using HTTP, but an origin server nontheless. RFC2616 only defines proxies and caches in a client side context and Varnish is (firmly!) a server-side facility. If in doubt, we side with the origin server, not the client. To the extent that we fail origin server requirements that affect practical deployment of Varnish, we will fix, but I cannot guarantee that we will strive for "to-the-letter-and-damn-the-cost" compliance. Poul-Henning -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From marcus at thefeed.no Wed Sep 20 20:47:00 2006 From: marcus at thefeed.no (Marcus Ramberg) Date: Wed, 20 Sep 2006 22:47:00 +0200 Subject: For reference, compiling on OSX Message-ID: <825a03c0609201347k1520a771h8c940c0704f163a9@mail.gmail.com> I was talking to Dag Henning about compiling varnish on OSX earlier today at the press conference. Just wanted to paste the current show stopper here for reference: if gcc -DHAVE_CONFIG_H -I. -I. -I../.. -I../../include -include config.h-O2 -pipe -MT varnishd-cache_acceptor.o -MD -MP -MF ".deps/varnishd-cache_acceptor.Tpo" -c -o varnishd-cache_acceptor.o `test -f 'cache_acceptor.c' || echo './'`cache_acceptor.c; \ then mv -f ".deps/varnishd-cache_acceptor.Tpo" ".deps/varnishd-cache_acceptor.Po"; else rm -f ".deps/varnishd-cache_acceptor.Tpo"; exit 1; fi cache_acceptor.c: In function 'vca_acct': cache_acceptor.c:170: error: 'CLOCK_REALTIME' undeclared (first use in this function) cache_acceptor.c:170: error: (Each undeclared identifier is reported only once cache_acceptor.c:170: error: for each function it appears in.) make[3]: *** [varnishd-cache_acceptor.o] Error 1 make[2]: *** [all-recursive] Error 1 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2 Dag Henning says this is because OSX does not have a clock_gettime system call, and that gettimeofday aint monotonous enough or something like that... :-) I will try to dig more into the issue. Other first impressions from testing out Varnish: We need more docs. Specially on making VCL policies. varnish does not seem to set X-Forwarded-For ? (Maybe it's not common to do it, but it's how my framework deals with reverse proxies) -- With regards Marcus Ramberg -------------- next part -------------- An HTML attachment was scrubbed... URL: From phk at phk.freebsd.dk Wed Sep 20 20:59:05 2006 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Wed, 20 Sep 2006 20:59:05 +0000 Subject: For reference, compiling on OSX In-Reply-To: Your message of "Wed, 20 Sep 2006 22:47:00 +0200." <825a03c0609201347k1520a771h8c940c0704f163a9@mail.gmail.com> Message-ID: <62228.1158785945@critter.freebsd.dk> In message <825a03c0609201347k1520a771h8c940c0704f163a9 at mail.gmail.com>, "Marcu s Ramberg" writes: >cache_acceptor.c:170: error: 'CLOCK_REALTIME' undeclared (first use in this >function) > >Dag Henning says this is because OSX does not have a clock_gettime system >call, and that gettimeofday aint monotonous enough or something like that... >:-) I will try to dig more into the issue. gettimeofday() will do. Just add a compat function that emulates clock_gettime() with a gettimeofday() call, and then preferably send an email to Jordan K. Hubbard at Apple and tell them to get their act together. >varnish does not seem to set X-Forwarded-For ? (Maybe it's not common to do >it, but it's how my framework deals with reverse proxies) I didn't know how widespread and widely used the X-Forwarded-For: header was, so I didn't put it in initially. If there is a desire, we can make it possible to do so from VCL. Poul-Henning -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From andersb at vgnett.no Wed Sep 20 21:11:03 2006 From: andersb at vgnett.no (Anders Berg) Date: Wed, 20 Sep 2006 23:11:03 +0200 (CEST) Subject: For reference, compiling on OSX In-Reply-To: <62228.1158785945@critter.freebsd.dk> References: Your message of "Wed, 20 Sep 2006 22:47:00 +0200."<825a03c0609201347k1520a771h8c940c0704f163a9@mail.gmail.com> <62228.1158785945@critter.freebsd.dk> Message-ID: <1308.193.213.34.102.1158786663.squirrel@denise.vg.no> > In message <825a03c0609201347k1520a771h8c940c0704f163a9 at mail.gmail.com>, > "Marcu > s Ramberg" writes: > >>cache_acceptor.c:170: error: 'CLOCK_REALTIME' undeclared (first use in >> this >>function) >> >>Dag Henning says this is because OSX does not have a clock_gettime system >>call, and that gettimeofday aint monotonous enough or something like >> that... >>:-) I will try to dig more into the issue. > > gettimeofday() will do. Just add a compat function that emulates > clock_gettime() with a gettimeofday() call, and then preferably > send an email to Jordan K. Hubbard at Apple and tell them to get > their act together. For the non-programmers out there, this should maybe be done in the buildscripts? Anders Berg From mnot at yahoo-inc.com Wed Sep 20 21:14:34 2006 From: mnot at yahoo-inc.com (Mark Nottingham) Date: Wed, 20 Sep 2006 14:14:34 -0700 Subject: Testing Varnish In-Reply-To: <54456.1158784558@critter.freebsd.dk> References: <54456.1158784558@critter.freebsd.dk> Message-ID: On 2006/09/20, at 1:35 PM, Poul-Henning Kamp wrote: > In message <11A48748-CDA5-4185-9ECB-A6248A7351C3 at yahoo-inc.com>, > Mark Nottingham writes: > >> The bad news - testing for HTTP conformance with Co-Advisor > coad.measurement-factory.com/> finds a *lot* of violations. However, >> I'm just testing stock Varnish with no configuration (as that's not >> documented! ;). Can anyone comment on the state of Varnish's HTTP >> conformance? > > Have you tested Varnish as a proxy or as an origin server ? As a gateway. > Varnish is not a proxy in the RFC2616 sense. Varnish > is an origin server. An origin server which gets its contents > using HTTP, but an origin server nontheless. That's a gateway, a.k.a. surrogate, a.k.a. reverse proxy. While it's true that many of the requirements for HTTP proxies won't apply to it, there are many HTTP requirements that do. For example, Varnish is acting as a cache, and appears to violate many of the requirements for caches. > RFC2616 only defines proxies and caches in a client side context > and Varnish is (firmly!) a server-side facility. No. From RFC2616: > cache > A program's local store of response messages and the subsystem > that controls its message storage, retrieval, and deletion. A > cache stores cacheable responses in order to reduce the response > time and network bandwidth consumption on future, equivalent > requests. Any client or server may include a cache, though a > cache > cannot be used by a server that is acting as a tunnel. > To the extent that we fail origin server requirements that affect > practical deployment of Varnish, we will fix, but I cannot guarantee > that we will strive for "to-the-letter-and-damn-the-cost" compliance. That's fine by me; perfect conformance is very difficult to obtain, and in many cases, not too useful when faced with real loads. My concerns are mostly around interoperability with other devices, and controlling how Varnish caches things. Cheers, -- Mark Nottingham mnot at yahoo-inc.com From marcus.ramberg at gmail.com Wed Sep 20 20:24:09 2006 From: marcus.ramberg at gmail.com (Marcus Ramberg) Date: Wed, 20 Sep 2006 22:24:09 +0200 Subject: Read more link points to an application/octet-stream file? Message-ID: <825a03c0609201324l759b53d4u7cef1339aaccd47c@mail.gmail.com> On the frontpage of www.varnish-cache.org :-/ -- With regards Marcus Ramberg -------------- next part -------------- An HTML attachment was scrubbed... URL: From des at linpro.no Thu Sep 21 09:31:48 2006 From: des at linpro.no (Dag-Erling =?iso-8859-1?Q?Sm=F8rgrav?=) Date: Thu, 21 Sep 2006 11:31:48 +0200 Subject: Read more link points to an application/octet-stream file? References: <825a03c0609201324l759b53d4u7cef1339aaccd47c@mail.gmail.com> Message-ID: "Marcus Ramberg" writes: > On the frontpage of www.varnish-cache.org :-/ Fixed. It didn't show up in my testing, for some reason :( DES -- Dag-Erling Sm?rgrav Senior Software Developer Linpro AS - www.linpro.no From phk at phk.freebsd.dk Thu Sep 21 10:07:11 2006 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Thu, 21 Sep 2006 10:07:11 +0000 Subject: Testing Varnish In-Reply-To: Your message of "Wed, 20 Sep 2006 14:14:34 MST." Message-ID: <1227.1158833231@critter.freebsd.dk> In message , Mark Nottingha m writes: >> Varnish is not a proxy in the RFC2616 sense. Varnish >> is an origin server. An origin server which gets its contents >> using HTTP, but an origin server nontheless. > >That's a gateway, a.k.a. surrogate, a.k.a. reverse proxy. In RFC2616 context, Varnish is an origin server and a client, but it is not a proxy or a gateway. Varnish is designed and built to be able to violate the semantic transparancy requirement for both of those. The name Varnish derives from the dictionary definition that says "To give a deceptively good appearance" (...to a CMS system). The entire point of Varnish is to make up for shortcomings in CMS systems and these shortcomings are not limited merely to speed. Varnish may be tasked with fixing up object lifetimes or anything else the site administrator is unhappy with. While varnish speaks "state of the art HTTP" on the client side, it may have to use "pidgin HTTP" with the backend. RFC2616 does not even begin to address a situation like that. For instance section 14.15 says: The Content-MD5 header field [...] Only origin servers or clients MAY generate the Content-MD5 header field; proxies and gateways MUST NOT generate it, [...] This is a perfect example of what Varnish might be asked to do: if the CMS system does not or can not produce Content-MD5 and the site ovner wants these, Varnish should be able to add them. Another example is that one of the hottest wishlist items for Varnish version 2 is ESI(-like) facilities for content composition, there is no way RFC2616 can allow us to do that, while sailing under a "gateway" flag. I want to stress, that this non-adherence to standard is not based on disrespect for RFC2616 on our part. The RFC2616 authors tacitly assume that a web-server can be fixed or improved at will. Unfortunately experience has shown that other parts of CMS systems are far more critical than the webserver part and therefore many sites which would love to, do not have that option. Therefore we are trying fill a need the authors of RFC2616 did not foresee: The need to put a glossy finish over a rubbish origin server to make it look good to the clients. >My concerns are mostly around interoperability with other devices, and >controlling how Varnish caches things. All of "policy" or "behaviour" aspects of Varnish are designed to happen in the VCL programming language. The default VCL code is intended to Do The Right Thing, so that a site administrator hit my abnormal traffic at 2:20AM can kick in a Varnish box, point it at the backend and have it cope with the spike and revisit the situation when he wakes up next time. Therefore Varnish will look a lot like a proxy or gateway when you look at it first time. But hopefully it will be much more than that to you, once you get to know it better. All that however, does not change the fact that in the separate roles of client and origin server, Varnish should comply with RFC2616 to the extent possible, reasonable and productive. Poul-Henning PS: At this point, VCL is nowhere as expressive as we dream about, but that is merely a matter of time, ideas and code. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From des at des.no Fri Sep 22 12:20:23 2006 From: des at des.no (Dag-Erling =?iso-8859-1?Q?Sm=F8rgrav?=) Date: Fri, 22 Sep 2006 14:20:23 +0200 Subject: My first try with varnishd In-Reply-To: <20060921111413.2F40.NETEX@163.com> (Jiang Hong's message of "Thu, 21 Sep 2006 11:44:04 +0800") References: <20060921111413.2F40.NETEX@163.com> Message-ID: <867izww0s8.fsf@dwp.des.no> [redirected to the mailing list] Jiang Hong writes: > First congratulations to the 1.0 release of varnishd! Thank you! > The 'mmap' line told me varnishd uses mmap, so 1.6G cache file *SEEMED* too > big for it (but isn't the limit 2GB?). Then I tried a 160MB cache file, this > time varnishd started working! It successfully mapped the entire file, but for some reason there was too little address space left for malloc(). Try a slightly smaller file, or increase maxdsiz, or switch to a 64-bit machine. > So is it possible for varnishd to use multiple mmaps to use cache files > larger than 2GB? It is enough for most cases, but when the average file > size is larger than 2GB, it's much better to use a cache_dir like Squid. No, it's a matter of address space. Varnish does not use a two-level cache like Squid, it uses a single cache which is backed by a file to allow the VM system to swap it out if it runs low on physical memory. On a 64-bit machine, you can create a cache file as big as your entire disk if you want. > Can '-s malloc' be used with '-s file'? Squid uses memory as level-1 cache > and disk storage as level-2 cache. No. Squid's two-level cache is precisely one of things that's wrong with it. It ends up fighting the VM system under heavy load. Varnish on the other hand tries to avoid doing work which the VM system already does better (like deciding which objects should be in physical memory and which should be on disk). > varnishd's VCL is much stronger than Squid's acl rules, but the sample.vcl > cannot pass the compilation: > > backend vg { > set backend.ip = 10.0.0.100; > set backend.timeout = 4s; > set backend.bandwidth = 2000Mb/s; > } I'm not sure where this code is from, but I think it was meant as an illustration of the principles of VCL, not as an example of a working configuration. > backend.ip should be backend.host/port. I can't find replacements for > backend.timeout and backend.bandwidth in 5 minutes. Is there any plan > to make a document? Yes, I'm working on it. For now, look through recent threads in the varnish-dev archive (http://projects.linpro.no/pipermail/varnish-dev/) DES -- Dag-Erling Sm?rgrav - des at des.no From mnot at yahoo-inc.com Fri Sep 22 17:19:17 2006 From: mnot at yahoo-inc.com (Mark Nottingham) Date: Fri, 22 Sep 2006 10:19:17 -0700 Subject: Testing Varnish In-Reply-To: <1227.1158833231@critter.freebsd.dk> References: <1227.1158833231@critter.freebsd.dk> Message-ID: <0F08FB20-F4EE-4345-88C3-A271B340E5B3@yahoo-inc.com> On 2006/09/21, at 3:07 AM, Poul-Henning Kamp wrote: > In message , > Mark Nottingha > m writes: > >>> Varnish is not a proxy in the RFC2616 sense. Varnish >>> is an origin server. An origin server which gets its contents >>> using HTTP, but an origin server nontheless. >> >> That's a gateway, a.k.a. surrogate, a.k.a. reverse proxy. > > In RFC2616 context, Varnish is an origin server and a client, but > it is not a proxy or a gateway. Varnish is designed and built to > be able to violate the semantic transparancy requirement for both > of those. Semantic transparency requirements are placed on caches, not gateways (or proxies, for that matter). "Reverse proxies," "accelerators," "surrogates," most "CDNs" and similar systems are all gateways; they act as the origin server, from the client's point of view. Most of them have caches that aren't semantically transparent, because they've been configured (either directly, or with protocol extensions) to do things beyond "normal" HTTP. > Therefore Varnish will look a lot like a proxy or gateway when you > look at it first time. But hopefully it will be much more than > that to you, once you get to know it better. I think this is a good expression of my concern. Most products like Varnish start from a base of being a conformant HTTP cache, and build extensions and modifications on top of that. If that isn't the intent, it needs to be said very clearly. > All that however, does not change the fact that in the separate > roles of client and origin server, Varnish should comply with > RFC2616 to the extent possible, reasonable and productive. That's good to hear. There seem to be a number of HTTP violations (that aren't specific to proxies) in Varnish; again, I'd encourage you to talk to the Measurement Factory folks and make their tests (or something like them) part of your QA suite. Cheers, -- Mark Nottingham mnot at yahoo-inc.com From phk at phk.freebsd.dk Fri Sep 22 18:09:09 2006 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Fri, 22 Sep 2006 18:09:09 +0000 Subject: Testing Varnish In-Reply-To: Your message of "Fri, 22 Sep 2006 10:19:17 MST." <0F08FB20-F4EE-4345-88C3-A271B340E5B3@yahoo-inc.com> Message-ID: <2400.1158948549@critter.freebsd.dk> In message <0F08FB20-F4EE-4345-88C3-A271B340E5B3 at yahoo-inc.com>, Mark Nottingha m writes: >Semantic transparency requirements are placed on caches, not gateways >(or proxies, for that matter). This is turning into a semantic fight, but RFC2616 clearly does presuppose that gateways are transparent, and that is too narrow a constraint for us. >> All that however, does not change the fact that in the separate >> roles of client and origin server, Varnish should comply with >> RFC2616 to the extent possible, reasonable and productive. > >That's good to hear. There seem to be a number of HTTP violations >(that aren't specific to proxies) in Varnish; again, I'd encourage >you to talk to the Measurement Factory folks and make their tests (or >something like them) part of your QA suite. Yes, we have a lot of work to do in this area, and we need to build some kind of test-rig so we can test the VCL primitives comprehensively as well. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From andersb at vgnett.no Fri Sep 22 22:08:10 2006 From: andersb at vgnett.no (Anders Berg) Date: Sat, 23 Sep 2006 00:08:10 +0200 (CEST) Subject: Testing Varnish In-Reply-To: <2400.1158948549@critter.freebsd.dk> References: Your message of "Fri, 22 Sep 2006 10:19:17 MST."<0F08FB20-F4EE-4345-88C3-A271B340E5B3@yahoo-inc.com> <2400.1158948549@critter.freebsd.dk> Message-ID: <1425.193.213.34.102.1158962890.squirrel@denise.vg.no> > In message <0F08FB20-F4EE-4345-88C3-A271B340E5B3 at yahoo-inc.com>, Mark > Nottingham writes: >>> All that however, does not change the fact that in the separate >>> roles of client and origin server, Varnish should comply with >>> RFC2616 to the extent possible, reasonable and productive. >> >>That's good to hear. There seem to be a number of HTTP violations >>(that aren't specific to proxies) in Varnish; again, I'd encourage >>you to talk to the Measurement Factory folks and make their tests (or >>something like them) part of your QA suite. > > Yes, we have a lot of work to do in this area, and we need to build > some kind of test-rig so we can test the VCL primitives comprehensively > as well. I would like to add that when it came to RFC 2616 compliance, the team discussed it a fair bit, without going into all the gory details possible. "What??!!" You may ask, "You have to do just exactly that!". Well, here comes one of the cool parts about Varnish: We discussed implementing RFC 2616 in VCL, as a default VCL config file. While, as you notice, we ended up with a not so strict interpetation as default, I (with the fear of saying something wrong, since I did not write the code) am pretty sure that VCL is so flexible as to be able to do exactly that. In other words, one can say that while Varnish is a "cache" that be anything you want it to be :) Think of it as a cache that is _quick_ at doing a lookup/serv and scale good on hardware, and at the same time has the flexibility of a "programming language" (VCL). We are fully aware that VCL itself puts a threshold on our users. Thats regretful, but we worked hard at getting it as easy as possible (Insane to say right now since the doc on VCL is hidden somewhere in /dev/null land), but at the same time very powerful. When the day is over, we/I feel that the primary users that Varnish tries to serv, are users that may need to do complex things with HTTP 1.0-1. And we give them the tool for this. Every website that will benefit alot from Varnish has the resources to learn and appreciate VCL quickly. At the same time we give the hobby website hackers a okay default run-and-forget config, we think... Thanks for pointing us towards Measurement Factory. Their tool(s) may be great for finding/making/testing vcl files for their compliance or such. I'll dig into it. I will have to stand corrected if what I say here is completly off the hook, but this is at least part of the dialog and conclusion the team has ended up with in regards to RFC 2616 and VCL. Anders Berg From andersb at vgnett.no Fri Sep 22 22:35:33 2006 From: andersb at vgnett.no (Anders Berg) Date: Sat, 23 Sep 2006 00:35:33 +0200 (CEST) Subject: My first try with varnishd In-Reply-To: <867izww0s8.fsf@dwp.des.no> References: <20060921111413.2F40.NETEX@163.com> <867izww0s8.fsf@dwp.des.no> Message-ID: <1484.193.213.34.102.1158964533.squirrel@denise.vg.no> > [redirected to the mailing list] > > Jiang Hong writes: >> First congratulations to the 1.0 release of varnishd! > > Thank you! > >> The 'mmap' line told me varnishd uses mmap, so 1.6G cache file *SEEMED* >> too >> big for it (but isn't the limit 2GB?). Then I tried a 160MB cache file, >> this >> time varnishd started working! > > It successfully mapped the entire file, but for some reason there was > too little address space left for malloc(). Try a slightly smaller > file, or increase maxdsiz, or switch to a 64-bit machine. > >> So is it possible for varnishd to use multiple mmaps to use cache files >> larger than 2GB? It is enough for most cases, but when the average file >> size is larger than 2GB, it's much better to use a cache_dir like Squid. > > No, it's a matter of address space. Varnish does not use a two-level > cache like Squid, it uses a single cache which is backed by a file to > allow the VM system to swap it out if it runs low on physical memory. > On a 64-bit machine, you can create a cache file as big as your entire > disk if you want. > >> Can '-s malloc' be used with '-s file'? Squid uses memory as level-1 >> cache >> and disk storage as level-2 cache. > > No. Squid's two-level cache is precisely one of things that's wrong > with it. It ends up fighting the VM system under heavy load. Varnish > on the other hand tries to avoid doing work which the VM system > already does better (like deciding which objects should be in physical > memory and which should be on disk). Great text/answers for the FAQ, DES. > Yes, I'm working on it. For now, look through recent threads in the > varnish-dev archive (http://projects.linpro.no/pipermail/varnish-dev/) Great. I just wrote in another email that it was in /dev/null land for now. Nothing is better than I was wrong. Anders Berg From xing at litespeedtech.com Mon Sep 25 18:03:38 2006 From: xing at litespeedtech.com (xing at litespeedtech.com) Date: Mon, 25 Sep 2006 11:03:38 -0700 Subject: x-forwarded-for Message-ID: <451819FA.1060802@litespeedtech.com> Source: http://varnish.projects.linpro.no/changeset/1121 It appears it's missing a condition where varnish need to rewrite/append to the x-forwarded-for header if it already exists in the incoming http request instead of outputting a new/duplicate header. This can happen when varnish is behind HTTP based load balancer or proxy. Xing From phk at phk.freebsd.dk Mon Sep 25 18:09:13 2006 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Mon, 25 Sep 2006 18:09:13 +0000 Subject: x-forwarded-for In-Reply-To: Your message of "Mon, 25 Sep 2006 11:03:38 MST." <451819FA.1060802@litespeedtech.com> Message-ID: <2752.1159207753@critter.freebsd.dk> In message <451819FA.1060802 at litespeedtech.com>, "xing at litespeedtech.com" write s: >Source: http://varnish.projects.linpro.no/changeset/1121 > >It appears it's missing a condition where varnish need to rewrite/append >to the x-forwarded-for header if it already exists in the incoming http >request instead of outputting a new/duplicate header. According to RFC2616 multiple headers append, they don't replace each other. So: X-Forewarded-for: 10.0.0.2, 10.0.0.3 and X-Forewarded-for: 10.0.0.2 X-Forewarded-for: 10.0.0.3 should have the same semantic meaning. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From xing at litespeedtech.com Mon Sep 25 18:11:20 2006 From: xing at litespeedtech.com (xing at litespeedtech.com) Date: Mon, 25 Sep 2006 11:11:20 -0700 Subject: x-forwarded-for In-Reply-To: <2752.1159207753@critter.freebsd.dk> References: <2752.1159207753@critter.freebsd.dk> Message-ID: <45181BC8.9020504@litespeedtech.com> Did a google search right after the email and realized my mistake. They are equivalent. RFC 2616, section 4.2: Multiple message-header fields with the same field-name MAY be present in a message if and only if the entire field-value for that header field is defined as a comma-separated list [i.e., #(values)]. It MUST be possible to combine the multiple header fields into one "field-name: field-value" pair, without changing the semantics of the message, by appending each subsequent field-value to the first, each separated by a comma. The order in which header fields with the same field-name are received is therefore significant to the interpretation of the combined field value, and thus a proxy MUST NOT change the order of these field values when a message is forwarded. Xing Poul-Henning Kamp wrote: > In message <451819FA.1060802 at litespeedtech.com>, "xing at litespeedtech.com" write > s: >> Source: http://varnish.projects.linpro.no/changeset/1121 >> >> It appears it's missing a condition where varnish need to rewrite/append >> to the x-forwarded-for header if it already exists in the incoming http >> request instead of outputting a new/duplicate header. > > According to RFC2616 multiple headers append, they don't replace > each other. > > So: > X-Forewarded-for: 10.0.0.2, 10.0.0.3 > and > X-Forewarded-for: 10.0.0.2 > X-Forewarded-for: 10.0.0.3 > > should have the same semantic meaning. > From phk at phk.freebsd.dk Mon Sep 25 18:14:54 2006 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Mon, 25 Sep 2006 18:14:54 +0000 Subject: x-forwarded-for In-Reply-To: Your message of "Mon, 25 Sep 2006 11:11:20 MST." <45181BC8.9020504@litespeedtech.com> Message-ID: <2801.1159208094@critter.freebsd.dk> In message <45181BC8.9020504 at litespeedtech.com>, "xing at litespeedtech.com" write s: >Did a google search right after the email and realized my mistake. They >are equivalent. One of the possible features for Varnish that has been talked about is a header-scrubber facility, combining multi-instance headers would be one obvious part of that task. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From xing at litespeedtech.com Thu Sep 28 20:56:35 2006 From: xing at litespeedtech.com (xing at litespeedtech.com) Date: Thu, 28 Sep 2006 13:56:35 -0700 Subject: varnishstat output Message-ID: <451C3703.70205@litespeedtech.com> Notice the "N large free smf" row. I don't think it's normal to have such a gigantic value so putting this on the list to let you guys decide. Xing running trunk r1121 Hitrate ratio: 3 3 3 Hitrate avg: 0.3745 0.3745 0.3745 869844 4.97 2.89 Client connections accepted 31200592 164.04 103.52 Client requests received 11705560 58.66 38.84 Cache hits 8200 0.00 0.03 Cache hits for pass 19476063 105.38 64.62 Cache misses 19495032 105.38 64.68 Backend connections initiated 18204089 91.47 60.40 Backend connections recyles 62 -0.99 0.00 Backend connections unused 1 0.00 0.00 N struct srcaddr 1 0.00 0.00 N active struct srcaddr 28 0.00 0.00 N struct sess_mem 11 0.00 0.00 N struct sess 76909 19.88 0.26 N struct object 77185 19.88 0.26 N struct objecthead 96081 16.90 0.32 N struct smf 30318 10.94 0.10 N small free smf 18446744073709546117 -12.92 -0.02 N large free smf 907 0.00 0.00 N struct vbe_conn 18 0.00 0.00 N worker threads 18797 0.00 0.06 N worker threads created 0 0.00 0.00 N worker threads not created 0 0.00 0.00 N worker threads limited 0 0.00 0.00 N queued work requests 18797 0.00 0.06 N overflowed work requests 0 0.00 0.00 N dropped work requests 19403966 80.53 64.38 N expired objects 1385 3.98 0.00 N objects on deathrow 0 0.00 0.00 HTTP header overflows 28797758 150.12 95.55 Objects sent with sendfile 1771479 5.97 5.88 Objects sent with write 841355 3.98 2.79 Total Sessions 31200565 155.09 103.52 Total Requests 10769 0.00 0.04 Total pipe 10213 0.00 0.03 Total pass 19473155 96.44 64.61 Total fetch 8428401331 42244.99 27964.73 Total header bytes 840988735623 4701316.51 2790330.05 Total body bytes 559633 0.00 1.86 Session Closed 0 0.00 0.00 Session Pipeline 0 0.00 0.00 Session Read Ahead 30640971 155.09 101.66 Session herd 1802143774 8685.22 5979.36 SHM records 54266433 250.54 180.05 SHM writes 19898 0.00 0.07 SHM MTX contention From andersb at vgnett.no Thu Sep 28 21:31:22 2006 From: andersb at vgnett.no (Anders Berg) Date: Thu, 28 Sep 2006 23:31:22 +0200 (CEST) Subject: varnishstat output In-Reply-To: <451C3703.70205@litespeedtech.com> References: <451C3703.70205@litespeedtech.com> Message-ID: <2285.193.69.165.4.1159479082.squirrel@denise.vg.no> > Notice the "N large free smf" row. I don't think it's normal to have > such a gigantic value so putting this on the list to let you guys decide. > > Xing > > running trunk r1121 > > Hitrate ratio: 3 3 3 > Hitrate avg: 0.3745 0.3745 0.3745 > 18446744073709546117 -12.92 -0.02 N large free smf Can confim seeing this on a 32-bit machine running CentOS 4.4. Anders Berg From andersb at vgnett.no Thu Sep 28 21:39:39 2006 From: andersb at vgnett.no (Anders Berg) Date: Thu, 28 Sep 2006 23:39:39 +0200 (CEST) Subject: varnishstat output In-Reply-To: <2285.193.69.165.4.1159479082.squirrel@denise.vg.no> References: <451C3703.70205@litespeedtech.com> <2285.193.69.165.4.1159479082.squirrel@denise.vg.no> Message-ID: <2420.193.69.165.4.1159479579.squirrel@denise.vg.no> >> Notice the "N large free smf" row. I don't think it's normal to have >> such a gigantic value so putting this on the list to let you guys >> decide. >> >> Xing >> >> running trunk r1121 >> >> Hitrate ratio: 3 3 3 >> Hitrate avg: 0.3745 0.3745 0.3745 > >> 18446744073709546117 -12.92 -0.02 N large free smf > > Can confim seeing this on a 32-bit machine running CentOS 4.4. Sorry, misread terminal name. I can confim this on a 64-bit machine, running FreeBSD 7.0 CURRENT. _Not_ the CentOS box. Anders Berg From phk at phk.freebsd.dk Thu Sep 28 21:59:26 2006 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Thu, 28 Sep 2006 21:59:26 +0000 Subject: varnishstat output In-Reply-To: Your message of "Thu, 28 Sep 2006 13:56:35 MST." <451C3703.70205@litespeedtech.com> Message-ID: <2016.1159480766@critter.freebsd.dk> In message <451C3703.70205 at litespeedtech.com>, "xing at litespeedtech.com" writes: A number of the statistics counters are not 100% reliably locked and therefore races can happen. I can't remember if this particular one is one of those, but if it isn't, it's a serious problem: 18446744073709546117 = 0xFFFFFFFFFFFFEA85 Poul-Henning >Notice the "N large free smf" row. I don't think it's normal to have >such a gigantic value so putting this on the list to let you guys decide. > >Xing > >running trunk r1121 > >Hitrate ratio: 3 3 3 >Hitrate avg: 0.3745 0.3745 0.3745 > > 869844 4.97 2.89 Client connections accepted > 31200592 164.04 103.52 Client requests received > 11705560 58.66 38.84 Cache hits > 8200 0.00 0.03 Cache hits for pass > 19476063 105.38 64.62 Cache misses > 19495032 105.38 64.68 Backend connections initiated > 18204089 91.47 60.40 Backend connections recyles > 62 -0.99 0.00 Backend connections unused > 1 0.00 0.00 N struct srcaddr > 1 0.00 0.00 N active struct srcaddr > 28 0.00 0.00 N struct sess_mem > 11 0.00 0.00 N struct sess > 76909 19.88 0.26 N struct object > 77185 19.88 0.26 N struct objecthead > 96081 16.90 0.32 N struct smf > 30318 10.94 0.10 N small free smf >18446744073709546117 -12.92 -0.02 N large free smf > 907 0.00 0.00 N struct vbe_conn > 18 0.00 0.00 N worker threads > 18797 0.00 0.06 N worker threads created > 0 0.00 0.00 N worker threads not created > 0 0.00 0.00 N worker threads limited > 0 0.00 0.00 N queued work requests > 18797 0.00 0.06 N overflowed work requests > 0 0.00 0.00 N dropped work requests > 19403966 80.53 64.38 N expired objects > 1385 3.98 0.00 N objects on deathrow > 0 0.00 0.00 HTTP header overflows > 28797758 150.12 95.55 Objects sent with sendfile > 1771479 5.97 5.88 Objects sent with write > 841355 3.98 2.79 Total Sessions > 31200565 155.09 103.52 Total Requests > 10769 0.00 0.04 Total pipe > 10213 0.00 0.03 Total pass > 19473155 96.44 64.61 Total fetch > 8428401331 42244.99 27964.73 Total header bytes >840988735623 4701316.51 2790330.05 Total body bytes > 559633 0.00 1.86 Session Closed > 0 0.00 0.00 Session Pipeline > 0 0.00 0.00 Session Read Ahead > 30640971 155.09 101.66 Session herd > 1802143774 8685.22 5979.36 SHM records > 54266433 250.54 180.05 SHM writes > 19898 0.00 0.07 SHM MTX contention > >_______________________________________________ >varnish-dev mailing list >varnish-dev at projects.linpro.no >http://projects.linpro.no/mailman/listinfo/varnish-dev > -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From xing at litespeedtech.com Thu Sep 28 22:18:33 2006 From: xing at litespeedtech.com (xing at litespeedtech.com) Date: Thu, 28 Sep 2006 15:18:33 -0700 Subject: varnishstat output In-Reply-To: <2420.193.69.165.4.1159479579.squirrel@denise.vg.no> References: <451C3703.70205@litespeedtech.com> <2285.193.69.165.4.1159479082.squirrel@denise.vg.no> <2420.193.69.165.4.1159479579.squirrel@denise.vg.no> Message-ID: <451C4A39.50304@litespeedtech.com> I'm running 64bit CentOS 4.4 btw. Anders Berg wrote: >>> Notice the "N large free smf" row. I don't think it's normal to have >>> such a gigantic value so putting this on the list to let you guys >>> decide. >>> >>> Xing >>> >>> running trunk r1121 >>> >>> Hitrate ratio: 3 3 3 >>> Hitrate avg: 0.3745 0.3745 0.3745 >>> 18446744073709546117 -12.92 -0.02 N large free smf >> Can confim seeing this on a 32-bit machine running CentOS 4.4. > > Sorry, misread terminal name. > > I can confim this on a 64-bit machine, running FreeBSD 7.0 CURRENT. > > _Not_ the CentOS box. > > Anders Berg > _______________________________________________ > varnish-dev mailing list > varnish-dev at projects.linpro.no > http://projects.linpro.no/mailman/listinfo/varnish-dev >