From fdylist at gmail.com Mon Jul 4 08:10:06 2011 From: fdylist at gmail.com (Fibrizo) Date: Mon, 4 Jul 2011 16:10:06 +0800 Subject: transient storage memleak in 3.0? Message-ID: <005401cc3a21$cd560440$68020cc0$@gmail.com> I set -s malloc,4G but varnish eat all my mem(12G), and use swap! In varnishstat , I saw SMA.s0.nbytes is 4G is right, but SMA.Transient.nbytes is almost 6G! What is SMA.Transient and why it can't be free? And how can I control transient storage size? SMA.s0.nreq 2343385 74.98 Allocator requests SMA.s0.nobj 353562 . Outstanding allocations SMA.s0.nbytes 4294967257 . Outstanding bytes SMA.s0.balloc 23949971661 . Bytes allocated SMA.s0.bfree 19655004404 . Bytes free SMA.Transient.nreq 600545 19.21 Allocator requests SMA.Transient.nobj 487027 . Outstanding allocations SMA.Transient.nbytes 6709806097 . Outstanding bytes SMA.Transient.balloc 9203517551 . Bytes allocated SMA.Transient.bfree 2493711454 . Bytes free Thank you~ my english is vary poor... -------------- next part -------------- An HTML attachment was scrubbed... URL: From nbubingo at gmail.com Fri Jul 8 07:06:06 2011 From: nbubingo at gmail.com (=?GB2312?B?0qbOsLHz?=) Date: Fri, 8 Jul 2011 15:06:06 +0800 Subject: About the vmod? Message-ID: Hi folks, I'm a freshman with the varnish and glad to see that the varnish3.0 support vmod. I want to develop a module to dispatch request to a certain backend server. The dispatch strategy is neither round robin or random. Is there anyway to develop such a director module? I have also noticed that the vmod are like callback functions. Is there any good way to do some configure work before start varnish? Thanks. Weibin Yao -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrea.campi at zephirworks.com Fri Jul 8 08:38:44 2011 From: andrea.campi at zephirworks.com (Andrea Campi) Date: Fri, 8 Jul 2011 10:38:44 +0200 Subject: About the vmod? In-Reply-To: References: Message-ID: Hi, On Fri, Jul 8, 2011 at 9:06 AM, ??? wrote: > Hi folks, > I'm a freshman with the varnish and glad to see that the varnish3.0 support > vmod. > I want to develop a module to dispatch request to a certain backend server. > The dispatch?strategy is neither round robin or random.?Is there anyway to > develop such a director module? Not currently. vmod's can define subroutines you can call from VCL, and that's pretty much in. To add a director strategy you need to hack source code directly. > I have also noticed that the vmod are like callback functions. ?Is there any > good way to do some configure work before start varnish? Your vmod can define an init function which will be called when your vmod is loaded. Look at the documentation and at existing modules. Andrea From martin at varnish-software.com Fri Jul 8 14:43:24 2011 From: martin at varnish-software.com (Martin Blix Grydeland) Date: Fri, 8 Jul 2011 16:43:24 +0200 Subject: Patch review for fix to #947 (assert on SIGHUP to varnishncsa) Message-ID: This bug is caused by an assert when usleep is interrupted by signal delivery in vsl.c (libvarnishapi). Some tools expect to receive signals (e.g. varnishncsa with -w option), so this has to be addressed. Attached patch relaxes the assert to not trigger on signal interrupt. Any comments appreciated. Regards, Martin Blix Grydeland -- Martin Blix Grydeland Varnish Software AS -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: bug-942-assert-on-sighup.patch Type: application/octet-stream Size: 733 bytes Desc: not available URL: From tfheen at varnish-software.com Mon Jul 11 07:42:45 2011 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Mon, 11 Jul 2011 09:42:45 +0200 Subject: varnishtop feature request: show _expensive_ urls In-Reply-To: (Harm Verhagen's message of "Wed, 29 Jun 2011 12:03:12 +0200") References: Message-ID: <874o2tqn4a.fsf@qurzaw.varnish-software.com> ]] Harm Verhagen | What do you guys think ? It's useful and I have a vague idea on how to implement it, it's just lacking in a bit more time spent thinking and some time spent implementing. | This would instantly give the problem areas of a website, no ? Well, it'd certainly be a useful tool for tracking certain types of problems, at least, yes. Cheers, -- Tollef Fog Heen Varnish Software t: +47 21 98 92 64 From dongyu at staff.sina.com.cn Mon Jul 4 08:06:51 2011 From: dongyu at staff.sina.com.cn (=?gb2312?B?t7+2q9Pq?=) Date: Mon, 04 Jul 2011 16:06:51 +0800 Subject: transient storage memleak in 3.0? Message-ID: <004a01cc3a21$54e11270$fea33750$@staff.sina.com.cn> I set ?Cs malloc,4G but varnish eat all my mem(12G), and use swap! In varnishstat , I saw SMA.s0.nbytes is 4G is right, but SMA.Transient.nbytes is almost 6G! What is SMA.Transient and why it can??t be free? And how can I control transient storage size? SMA.s0.nreq 2343385 74.98 Allocator requests SMA.s0.nobj 353562 . Outstanding allocations SMA.s0.nbytes 4294967257 . Outstanding bytes SMA.s0.balloc 23949971661 . Bytes allocated SMA.s0.bfree 19655004404 . Bytes free SMA.Transient.nreq 600545 19.21 Allocator requests SMA.Transient.nobj 487027 . Outstanding allocations SMA.Transient.nbytes 6709806097 . Outstanding bytes SMA.Transient.balloc 9203517551 . Bytes allocated SMA.Transient.bfree 2493711454 . Bytes free Thank you~ my english is vary poor... -------------- next part -------------- An HTML attachment was scrubbed... URL: From dongyu at staff.sina.com.cn Mon Jul 4 08:09:34 2011 From: dongyu at staff.sina.com.cn (=?gb2312?B?t7+2q9Pq?=) Date: Mon, 04 Jul 2011 16:09:34 +0800 Subject: transient storage memleak in 3.0? Message-ID: <004f01cc3a21$b61e1ba0$225a52e0$@staff.sina.com.cn> I set ?Cs malloc,4G but varnish eat all my mem(12G), and use swap! In varnishstat , I saw SMA.s0.nbytes is 4G is right, but SMA.Transient.nbytes is almost 6G! What is SMA.Transient and why it can??t be free? And how can I control transient storage size? SMA.s0.nreq 2343385 74.98 Allocator requests SMA.s0.nobj 353562 . Outstanding allocations SMA.s0.nbytes 4294967257 . Outstanding bytes SMA.s0.balloc 23949971661 . Bytes allocated SMA.s0.bfree 19655004404 . Bytes free SMA.Transient.nreq 600545 19.21 Allocator requests SMA.Transient.nobj 487027 . Outstanding allocations SMA.Transient.nbytes 6709806097 . Outstanding bytes SMA.Transient.balloc 9203517551 . Bytes allocated SMA.Transient.bfree 2493711454 . Bytes free Thank you~ my english is vary poor... -------------- next part -------------- An HTML attachment was scrubbed... URL: From tfheen at varnish-software.com Mon Jul 11 07:45:07 2011 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Mon, 11 Jul 2011 09:45:07 +0200 Subject: Patch review for fix to #947 (assert on SIGHUP to varnishncsa) In-Reply-To: (Martin Blix Grydeland's message of "Fri, 8 Jul 2011 16:43:24 +0200") References: Message-ID: <87zkklp8fw.fsf@qurzaw.varnish-software.com> ]] Martin Blix Grydeland | This bug is caused by an assert when usleep is interrupted by signal | delivery in vsl.c (libvarnishapi). Some tools expect to receive signals | (e.g. varnishncsa with -w option), so this has to be addressed. Attached | patch relaxes the assert to not trigger on signal interrupt. Any comments | appreciated. Looks reasonable to me, please commit. Regards, -- Tollef Fog Heen Varnish Software t: +47 21 98 92 64 From Stephen at projects.linpro.no Tue Jul 12 05:10:58 2011 From: Stephen at projects.linpro.no (Stephen at projects.linpro.no) Date: 12 Jul 2011 13:10:58 +0800 Subject: information Request - codebaboon.com Message-ID: <20110710071040.FD32400FF735FEB9@goodswell.net> I searched your contact Email address from codebaboon.com (web site) I am Stephen Lee and I come from China (Hong Kong). Could you let me know your business purchase / marketing department contact email ? OR Do you have purchase office in Hong Kong or China ? I thought to send you details of our Electronic display believing your business may benefit from using them. These products design are for Products Promotion and any Retail Store, Shop, Boutique or any Public Area ( like SuperMarket ). Our factory produce: A. LCD Player with Motion Sensor. ( It is use to Attract Customers in Goods Shelves area ) B. Bigger Size HD LCD Windows Display and come with internal media player. (use in Store Windows Display area) Please see attach photos These products is use to promote your products, Increase 10% - 30% sales, Attract Customers and help the customers to make decisions based on the information gained from the Electronic display. We can email you a Marketing Research Report (LCD Player can see a 30- percent increase in sales, while overall store sales can increase as much as 12 percent.) and products information to your company purchase department for reference. Thanks for your help. I am apologize for any inconvenience, appreciate your patience. Sincerely, Stephen Lee CONTACTdetail No virus found in this outgoing message. Checked by AVG - www.avg.com Version: 10.0.1209 / Virus Database: 1500/3597 - Release Date: 7/10/2011 7:10:40 -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 381ZZZZ381381381381ZZ.jpg Type: image/jpeg Size: 27974 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: PPPP0396P0396.gif Type: image/gif Size: 11884 bytes Desc: not available URL: From jbq at caraldi.com Tue Jul 12 14:47:03 2011 From: jbq at caraldi.com (Jean-Baptiste Quenot) Date: Tue, 12 Jul 2011 16:47:03 +0200 Subject: Varnish 3.0 NCSA log format missing object age In-Reply-To: References: Message-ID: Here is the patch: http://dl.dropbox.com/u/5836407/Patches/Varnish_3.0.0-Add-support-for-object-age-in-varnishncsa.patch varnishncsa is still missing a log format validator, and documentation for {Varnish:...}x fields, but object age is supported with this patch. I ported it from the logformat patch of varnish 2.1, I don't know why this field was omitted during 3.0 development. Best regards, -- Jean-Baptiste Quenot From jbq at caraldi.com Tue Jul 12 15:51:01 2011 From: jbq at caraldi.com (Jean-Baptiste Quenot) Date: Tue, 12 Jul 2011 17:51:01 +0200 Subject: varnishlog -o option Message-ID: A little discrepancy in the doc. Man page says: -o Ignored for compatibility with earlier versions. But in EXAMPLES I can find it: The following command line reads that same log file and displays requests for the front page:: $ varnishlog -r /var/log/varnish.log -c -o RxURL '^/$' -- Jean-Baptiste Quenot From jbq at caraldi.com Tue Jul 12 20:44:18 2011 From: jbq at caraldi.com (Jean-Baptiste Quenot) Date: Tue, 12 Jul 2011 22:44:18 +0200 Subject: Varnish 3.0.0 crash Message-ID: Hello, I put varnish 3.0.0 in production today, and get frequent crashes: Jul 12 22:13:47 gw1 kernel: [3379855.023726] varnishd[2181] general protection ip:7fcc7177f6be sp:7fbfdfbeb098 error:0 in libc-2.13.so[7fcc71753000+18a000] Jul 12 20:13:47 gw1 varnishd[16495]: Child (1334) Panic message: Assert error in vsl_hdr(), cache_shmlog.c line 62: Condition(((uintptr_t)p & 0x3) == 0) not true. Jul 12 20:13:47 gw1 varnishd[16495]: child (5223) Started Jul 12 20:13:47 gw1 varnishd[16495]: Child (5223) said Child starts Jul 12 20:13:47 gw1 varnishd[16495]: Child (5223) said SMF.s0 mmap'ed 53687091200 bytes of 53687091200 Jul 12 21:09:48 gw1 kernel: [3376023.128094] varnishd[1333] trap stack segment ip:431958 sp:7fbfd11eb2b0 error:0 Jul 12 19:09:48 gw1 varnishd[16495]: child (1334) Started Jul 12 19:09:48 gw1 varnishd[16495]: Child (1334) said Child starts Jul 12 19:09:48 gw1 varnishd[16495]: Child (1334) said SMF.s0 mmap'ed 53687091200 bytes of 53687091200 Jul 12 19:08:54 gw1 varnishd[16495]: child (1241) Started Jul 12 19:08:54 gw1 varnishd[16495]: Child (1241) said Child starts Jul 12 19:08:54 gw1 varnishd[16495]: Child (1241) said SMF.s0 mmap'ed 53687091200 bytes of 53687091200 Jul 12 21:08:54 gw1 kernel: [3375969.228010] varnishd[1087]: segfault at 7fbfe61ff000 ip 00007fcc717d8aab sp 00007fbfe61e4fa8 error 6 in libc-2.13.so[7fcc71753000+18a000] Jul 12 21:06:58 gw1 kernel: [3375853.809908] varnishd[991]: segfault at 7fbfe4fff000 ip 00007fcc717d8aab sp 00007fbfe4fe57e8 error 6 in libc-2.13.so[7fcc71753000+18a000] Jul 12 19:06:58 gw1 varnishd[16495]: child (1076) Started Jul 12 19:06:58 gw1 varnishd[16495]: Child (1076) said Child starts Jul 12 19:06:58 gw1 varnishd[16495]: Child (1076) said SMF.s0 mmap'ed 53687091200 bytes of 53687091200 Do you need a gdb backtrace or something? -- Jean-Baptiste Quenot From phk at phk.freebsd.dk Tue Jul 12 21:04:20 2011 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Tue, 12 Jul 2011 21:04:20 +0000 Subject: Varnish 3.0.0 crash In-Reply-To: Your message of "Tue, 12 Jul 2011 22:44:18 +0200." Message-ID: <7840.1310504660@critter.freebsd.dk> In message , Jean-Baptiste Quenot writes: >Do you need a gdb backtrace or something? Yes please. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From tfheen at varnish-software.com Fri Jul 15 06:41:48 2011 From: tfheen at varnish-software.com (Tollef Fog Heen) Date: Fri, 15 Jul 2011 08:41:48 +0200 Subject: varnishlog -o option In-Reply-To: (Jean-Baptiste Quenot's message of "Tue, 12 Jul 2011 17:51:01 +0200") References: Message-ID: <87liw059lf.fsf@qurzaw.varnish-software.com> ]] Jean-Baptiste Quenot | But in EXAMPLES I can find it: | | The following command line reads that same log file and | displays requests for the front page:: | | $ varnishlog -r /var/log/varnish.log -c -o RxURL '^/$' Thanks, already fixed. -- Tollef Fog Heen Varnish Software t: +47 21 98 92 64 From jbq at caraldi.com Fri Jul 15 19:10:55 2011 From: jbq at caraldi.com (Jean-Baptiste Quenot) Date: Fri, 15 Jul 2011 21:10:55 +0200 Subject: Varnish 3.0.0 crash In-Reply-To: <7840.1310504660@critter.freebsd.dk> References: <7840.1310504660@critter.freebsd.dk> Message-ID: These are the details I could manage to get for now: Core was generated by `/usr/sbin/varnishd -P /var/run/varnishd.pid -a :80 -t 604800 -T localhost:6082'. Program terminated with signal 11, Segmentation fault. #0 0x00007ff242c71aab in mempcpy () from /lib/x86_64-linux-gnu/libc.so.6 (gdb) bt #0 0x00007ff242c71aab in mempcpy () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x00007ff242c60bbe in _IO_default_xsputn () from /lib/x86_64-linux-gnu/libc.so.6 #2 0x00007ff242c3331d in vfprintf () from /lib/x86_64-linux-gnu/libc.so.6 #3 0x00007ff242ce8780 in __vsnprintf_chk () from /lib/x86_64-linux-gnu/libc.so.6 #4 0x000000000043193e in WSL () #5 0x0000000000429df5 in ?? () #6 0x0000000000429f8e in http_DissectRequest () #7 0x0000000000417318 in CNT_Session () #8 0x000000000042daf8 in ?? () #9 0x000000000042df81 in ?? () This is running on Ubuntu 11.04. I had to set core_pattern="/var/tmp/%p.core" in the kernel, otherwise no core was generated. I can't manage to load debug symbols from libvarnish1... -- Jean-Baptiste Quenot From jbq at caraldi.com Sat Jul 16 17:38:22 2011 From: jbq at caraldi.com (Jean-Baptiste Quenot) Date: Sat, 16 Jul 2011 19:38:22 +0200 Subject: Debug symbols broken in debian package Message-ID: Hi there, To whom it may concern, the Debian and Ubuntu packages for Varnish do not build the debug symbols correctly. Here is a patch to fix this issue: http://dl.dropbox.com/u/5836407/Patches/VarnishDeb_3.0.0-Fix-packaging-of-debug-symbols.patch Best regards, -- Jean-Baptiste Quenot From jbq at caraldi.com Sat Jul 16 18:40:36 2011 From: jbq at caraldi.com (Jean-Baptiste Quenot) Date: Sat, 16 Jul 2011 20:40:36 +0200 Subject: varnishncsa 3.0.0 crash Message-ID: This is varnishncsa with the object age patch sent to the mailing-list. Core was generated by `/usr/bin/varnishncsa -a -c -w /var/log/varnishncsa.log -D -P /var/run/var'. Program terminated with signal 11, Segmentation fault. #0 0x00007f0370e4f5d1 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 (gdb) bt #0 0x00007f0370e4f5d1 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x00007f0370e34fbe in fputs () from /lib/x86_64-linux-gnu/libc.so.6 #2 0x0000000000402861 in h_ncsa (priv=0x16b2bf0, tag=SLT_ReqEnd, fd=, len=88, spec=, ptr=, bitmap=0) at /usr/include/bits/stdio2.h:98 #3 0x00007f0371387f63 in VSL_Dispatch () from /usr/lib/libvarnishapi.so.1 #4 0x0000000000402d74 in main (argc=10, argv=) at varnishncsa.c:830 -- Jean-Baptiste Quenot From jbq at caraldi.com Sat Jul 16 18:49:41 2011 From: jbq at caraldi.com (Jean-Baptiste Quenot) Date: Sat, 16 Jul 2011 20:49:41 +0200 Subject: varnishncsa 3.0.0 crash In-Reply-To: References: Message-ID: Another one: Core was generated by `/usr/bin/varnishncsa -a -c -w /var/log/varnishncsa.log -D -P /var/run/var'. Program terminated with signal 11, Segmentation fault. #0 0x00007f4656c545d1 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 (gdb) bt #0 0x00007f4656c545d1 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x00007f4656c39fbe in fputs () from /lib/x86_64-linux-gnu/libc.so.6 #2 0x0000000000402861 in h_ncsa (priv=0x1775bf0, tag=SLT_ReqEnd, fd=, len=88, spec=, ptr=, bitmap=0) at /usr/include/bits/stdio2.h:98 #3 0x00007f465718cf63 in vbit_expand (vd=, pp=0x1775bf0, mb=0x1775bf0) at ../../include/vbm.h:57 #4 vbit_set (vd=, pp=0x1775bf0, mb=0x1775bf0) at ../../include/vbm.h:91 #5 VSL_NextLog (vd=, pp=0x1775bf0, mb=0x1775bf0) at vsl.c:226 #6 0x00000001004038b5 in ?? () #7 0x0000000000000000 in ?? () By the way the full command-line is: /usr/bin/varnishncsa -a -c -w /var/log/varnishncsa.log -D -P /var/run/varnishncsa/varnishncsa.pid -F '%{Host}i %h %l %u %t "%r" %s %b "%{Referer}i" "%{User-agent}i" %{Varnish:time_firstbyte}x %{Varnish:age}x' Cheers, -- Jean-Baptiste Quenot From jbq at caraldi.com Sat Jul 16 19:28:00 2011 From: jbq at caraldi.com (Jean-Baptiste Quenot) Date: Sat, 16 Jul 2011 21:28:00 +0200 Subject: varnishncsa 3.0.0 crash In-Reply-To: References:

Message-ID: As the backtrace mentions SLT_ReqEnd, there must be a bug along those lines: case SLT_ReqEnd: { char ttfb[64]; if (!lp->active) break; if (lp->df_ttfb != NULL || sscanf(ptr, "%*u %*u.%*u %ld.%*u %*u.%*u %s", &l, ttfb) != 2) { clean_logline(lp); break; } lp->df_ttfb = strdup(ttfb); t = l; localtime_r(&t, &lp->df_t); /* got it all */ lp->complete = 1; break; } Removing %{Varnish:time_firstbyte}x from log format does not help, still crashes. -- Jean-Baptiste Quenot From jbq at caraldi.com Sat Jul 16 19:55:32 2011 From: jbq at caraldi.com (Jean-Baptiste Quenot) Date: Sat, 16 Jul 2011 21:55:32 +0200 Subject: Varnish 3.0.0 crash In-Reply-To: References: <7840.1310504660@critter.freebsd.dk> Message-ID: OK here is the crash log with proper debug symbols: https://gist.github.com/1086698 Time for me to get back to 2.1. -- Jean-Baptiste Quenot From ssm at redpill-linpro.com Mon Jul 18 10:31:31 2011 From: ssm at redpill-linpro.com (Stig Sandbeck Mathisen) Date: Mon, 18 Jul 2011 12:31:31 +0200 Subject: Debug symbols broken in debian package In-Reply-To: (Jean-Baptiste Quenot's message of "Sat, 16 Jul 2011 19:38:22 +0200") References: Message-ID: <7x7h7fribg.fsf@fsck.linpro.no> Jean-Baptiste Quenot writes: > Hi there, > > To whom it may concern, the Debian and Ubuntu packages for Varnish do > not build the debug symbols correctly. Thanks. I've uploaded varnish 3.0.0-4 to Debian with an additional package which contains debugging symbols (varnish-dbg). > Here is a patch to fix this issue: > > http://dl.dropbox.com/u/5836407/Patches/VarnishDeb_3.0.0-Fix-packaging-of-debug-symbols.patch I think you left some pieces out, but I got the general idea. :) -- Stig Sandbeck Mathisen From jbq at caraldi.com Mon Jul 18 12:19:44 2011 From: jbq at caraldi.com (Jean-Baptiste Quenot) Date: Mon, 18 Jul 2011 14:19:44 +0200 Subject: Debug symbols broken in debian package In-Reply-To: <7x7h7fribg.fsf@fsck.linpro.no> References: <7x7h7fribg.fsf@fsck.linpro.no> Message-ID: 2011/7/18 Stig Sandbeck Mathisen : > I think you left some pieces out, but I got the general idea. :) Ooops indeed I thought the extra varnish-dbg package was coming from the stock debian package, but I just realized it's a patch I added myself a long time ago. Thanks! -- Jean-Baptiste Quenot