From daghf at varnish-software.com Mon Jul 2 08:09:59 2012 From: daghf at varnish-software.com (Dag Haavi Finstad) Date: Mon, 2 Jul 2012 10:09:59 +0200 Subject: [PATCH] Fix for an issue where varnishlog reports an incorrect "Length" entry on gunzipped delivery Message-ID: Hi I found a small bug related to updates of the 'bodybytes' counter. The problem manifests itself on gunzipped delivery, where varnishlog will display an incorrect 'Length' entry (for objects whose size is not divisible by gzip_buffer). Patch enclosed. -- Dag Haavi Finstad -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: bodybytes-gunzip.patch Type: application/octet-stream Size: 1663 bytes Desc: not available URL: From martin at varnish-software.com Wed Jul 4 11:59:21 2012 From: martin at varnish-software.com (Martin Blix Grydeland) Date: Wed, 4 Jul 2012 13:59:21 +0200 Subject: Patches to fix access of busy objects in ban lurker, expiry thread and EXP_NukeOne (bugs #1150 and #1162) Message-ID: Hunting #1150 some additional areas popped up too where attempts to access busy objects from other threads could happen, with subsequent assertion. I have not been able to create good test cases for these, as the time window is too small. Though all problems have been confirmed (and confirmed fixed by the patches), by introducing deliberate delays in Varnish during testing. Regards, Martin Blix Grydeland -- Martin Blix Grydeland Varnish Software AS -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Make-the-ban-lurker-skip-busy-objects.patch Type: application/octet-stream Size: 1294 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0002-Do-not-try-to-expire-objects-that-are-still-busy.patch Type: application/octet-stream Size: 1223 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0003-Do-not-consider-busy-objects-as-candidates-in-EXP_Nu.patch Type: application/octet-stream Size: 1066 bytes Desc: not available URL: From kristian at varnish-software.com Fri Jul 6 12:10:58 2012 From: kristian at varnish-software.com (Kristian Lyngstol) Date: Fri, 6 Jul 2012 14:10:58 +0200 Subject: experimental-ims and varnishncsa In-Reply-To: <4FE4A92D.5060000@uplex.de> References: <20120622084406.GN3661@freud.kly.no> <4FE4A92D.5060000@uplex.de> Message-ID: <20120706121058.GA26518@freud.kly.no> On Fri, Jun 22, 2012 at 07:19:41PM +0200, Geoff Simmons wrote: > On 6/22/12 10:44 AM, Kristian Lyngstol wrote: > > I ran across a bug in the experimental-ims branch that's evident in > > varnishncsa: > > > > experimental-ims: > > > > 172.24.2.41 - - [15/Jun/2012:19:13:42 -0300] "- > > http://localhost-HTTP/1.0" 0 "-" "-" > > > > vanilla: > > > > 172.22.2.41 - - [13/Jun/2012:16:06:00 -0300] "GET > > http://test.com/test.jpgHTTP/1.1" 200 0 "-" "Mozilla/5.0 (X11; > > Ubuntu; Linux i686; rv:12.0) > > > > I've tracked it down to the commit > > f7dbf53eb01d26170f6c4bafd3f0a50d00444dc5 which is a PHK-commit. > > > > Any chance you could take a look? Given that it's obviously a > > merge-related bug it seems somewhat time consuming to track down > > without intimate knowledge of the IMS-branch. > > Sure, I'll have a look on the weekend. Thanks for figuring out the > commit, makes the search much easier. Did you find some time for this issue by any chance? - Kristian From ruben at varnish-software.com Mon Jul 9 19:55:36 2012 From: ruben at varnish-software.com (=?UTF-8?Q?Rub=C3=A9n_Romero?=) Date: Mon, 9 Jul 2012 21:55:36 +0200 Subject: Welcome to VUG6 in London, United Kingdom - October 5th & 6th, 2012 Message-ID: Hello everyone, It is my pleasure to announce the Sixth Varnish User Group meeting which will be held in London on the first week of October. The meeting is kindly sponsored by the BBC. The Varnish User Day will be on Friday 5th at the BBC Media Center. The location for the Varnish Developer meeting on Saturday (Hackurday?) will be announced later. Limited seats, so if you think you are coming, register now! Details & registration are available on our community website: < http://www.varnish.org/vug6> We urge all attendees to consider presenting to the audience how you use Varnish, as well as what kind of workshops or BoFs you would be interested in having. Add your presentation by replying to this email or directly in the wiki: For those of you going to the Velocity Europe Conference, VUG6 is the excuse you were looking for to spend the weekend in London. See you there! ;-) Best regards, -- Rub?n Romero, Self-Appointed Varnish Cheerleader! Phone: +47 21 98 92 62 / Mobile: +47 959 64 088 / Skype&Twitter: ruben_varnish *Varnish makes websites fly!* Whitepapers | Video | Twitter | LinkedIn -------------- next part -------------- An HTML attachment was scrubbed... URL: From varnish-dev at projects.linpro.no Tue Jul 10 23:41:09 2012 From: varnish-dev at projects.linpro.no (varnish-dev at projects.linpro.no) Date: Wed, 11 Jul 2012 00:41:09 +0100 Subject: Information zur JOBBORSE Message-ID: <8580437907.ZRS5C6UZ513705@kuabdrtakp.uugevwcuzhr.tv> An HTML attachment was scrubbed... URL: From varnish-dev at projects.linpro.no Wed Jul 11 03:54:35 2012 From: varnish-dev at projects.linpro.no (varnish-dev at projects.linpro.no) Date: Tue, 10 Jul 2012 21:54:35 -0600 Subject: Arbeit in Deutschland Message-ID: <0479504697.T9NW9IHC994736@pictejiozwm.dbndzuxqdltiop.su> An HTML attachment was scrubbed... URL: From varnish-dev at projects.linpro.no Wed Jul 11 10:30:53 2012 From: varnish-dev at projects.linpro.no (varnish-dev at projects.linpro.no) Date: Wed, 11 Jul 2012 11:30:53 +0100 Subject: Information zur JOBBORSE Message-ID: <7080486061.9485465Q052010@xrdjnhdiu.qyzoeldpmzi.biz> An HTML attachment was scrubbed... URL: From varnish-dev at projects.linpro.no Wed Jul 11 13:26:46 2012 From: varnish-dev at projects.linpro.no (varnish-dev at projects.linpro.no) Date: Wed, 11 Jul 2012 13:26:46 +0000 Subject: Arbeitsmarkt Naturwissenschaften Message-ID: <9747758477.N986M0Z4163938@atnlgszzmgbil.xxzutnlbltmj.ua> An HTML attachment was scrubbed... URL: From varnish-bugs-owner at projects.linpro.no Wed Jul 11 14:57:47 2012 From: varnish-bugs-owner at projects.linpro.no (varnish-bugs-owner at projects.linpro.no) Date: Wed, 11 Jul 2012 15:57:47 +0100 Subject: Wir suchen einen Operationsmanager Message-ID: <9170220876.MNNH16DK558056@zbflmhpsyeu.ruyvgrtgdjlnhf.biz> An HTML attachment was scrubbed... URL: From varnish-dev at projects.linpro.no Wed Jul 11 18:15:40 2012 From: varnish-dev at projects.linpro.no (varnish-dev at projects.linpro.no) Date: Wed, 11 Jul 2012 20:15:40 +0200 Subject: Arbeitsmarkt Naturwissenschaften Message-ID: <5796626446.QOH01N6K884153@eofvhwi.jqgtiqgtlmhjgyf.su> An HTML attachment was scrubbed... URL: From martin at varnish-software.com Fri Jul 13 07:57:09 2012 From: martin at varnish-software.com (Martin Blix Grydeland) Date: Fri, 13 Jul 2012 09:57:09 +0200 Subject: [PATCH] Document the version requirements of the varnishreplay input file. Message-ID: <1342166229-11785-1-git-send-email-martin@varnish-software.com> --- doc/sphinx/reference/varnishreplay.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/doc/sphinx/reference/varnishreplay.rst b/doc/sphinx/reference/varnishreplay.rst index 4be103f..da3df36 100644 --- a/doc/sphinx/reference/varnishreplay.rst +++ b/doc/sphinx/reference/varnishreplay.rst @@ -32,8 +32,9 @@ The following options are available: -D Turn on debugging mode. --r file Parse logs from this file. This option is mandatory. - +-r file Parse logs from this file. The input file has to be from + a varnishlog of the same version as the varnishreplay + binary. This option is mandatory. SEE ALSO ======== -- 1.7.9.5 From jocelyn.delarosa at smartjog.com Fri Jul 13 14:53:46 2012 From: jocelyn.delarosa at smartjog.com (Jocelyn De La Rosa) Date: Fri, 13 Jul 2012 16:53:46 +0200 Subject: experimental-ims and varnishncsa In-Reply-To: <20120706121058.GA26518@freud.kly.no> References: <20120622084406.GN3661@freud.kly.no> <4FE4A92D.5060000@uplex.de> <20120706121058.GA26518@freud.kly.no> Message-ID: <5000367A.5060800@smartjog.com> On 07/06/2012 02:10 PM, Kristian Lyngstol wrote: > On Fri, Jun 22, 2012 at 07:19:41PM +0200, Geoff Simmons wrote: >> On 6/22/12 10:44 AM, Kristian Lyngstol wrote: >>> I ran across a bug in the experimental-ims branch that's evident in >>> varnishncsa: >>> >>> experimental-ims: >>> >>> 172.24.2.41 - - [15/Jun/2012:19:13:42 -0300] "- >>> http://localhost-HTTP/1.0" 0 "-" "-" >>> >>> vanilla: >>> >>> 172.22.2.41 - - [13/Jun/2012:16:06:00 -0300] "GET >>> http://test.com/test.jpgHTTP/1.1" 200 0 "-" "Mozilla/5.0 (X11; >>> Ubuntu; Linux i686; rv:12.0) >>> >>> I've tracked it down to the commit >>> f7dbf53eb01d26170f6c4bafd3f0a50d00444dc5 which is a PHK-commit. >>> >>> Any chance you could take a look? Given that it's obviously a >>> merge-related bug it seems somewhat time consuming to track down >>> without intimate knowledge of the IMS-branch. >> Sure, I'll have a look on the weekend. Thanks for figuring out the >> commit, makes the search much easier. > Did you find some time for this issue by any chance? > > - Kristian > > _______________________________________________ > varnish-dev mailing list > varnish-dev at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev > > Hi, I had time to investigate the problem. It existed in the master branch for the commit f7dbf53eb01d26170f6c4bafd3f0a50d00444dc5 and was fixed 3 commits later (9a63616cb869286a0a3f8139f8f0f1205ad87e80) So I cherry picked the 3 patches and added some change so it works. Here are the 3 patches. Cheers, -- Jocelyn De La Rosa - Developer, Research& Engineering SmartJog | www.smartjog.com 27 Blvd Hippolyte Marqu?s, 94200 Ivry-sur-Seine, France -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Clarify-comment-add-assert-comment.patch Type: text/x-patch Size: 1468 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0002-Eliminate-a-single-use-macro.patch Type: text/x-patch Size: 1397 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0003-Cleanup-all-the-easy-bits-of-the-VSL-api-in-varnishd.patch Type: text/x-patch Size: 33295 bytes Desc: not available URL: From geoff at uplex.de Wed Jul 18 14:19:55 2012 From: geoff at uplex.de (Geoff Simmons) Date: Wed, 18 Jul 2012 16:19:55 +0200 Subject: [PATCH] Fix the compile error in cache_waiter_ports.c for the Solaris port (ticket #1167) Message-ID: <5006C60B.2090307@uplex.de> Hello all, The enclosed patch, for the current 3.0 branch, fixes the build error for the Solaris port reported in Trac ticket #1167. Best, Geoff $ git show commit 1f4006e83a3b257c9a604dd40b857d6292608b46 Merge: b50bbd6 b164c06 Author: Dag Haavi Finstad Date: Mon Jul 16 12:36:15 2012 +0200 -- ** * * UPLEX - Nils Goroll Systemoptimierung Schwanenwik 24 22087 Hamburg Tel +49 40 2880 5731 Mob +49 176 636 90917 Fax +49 40 42949753 http://uplex.de -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: 1167.patch URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 896 bytes Desc: OpenPGP digital signature URL: From geoff at uplex.de Thu Jul 19 12:47:57 2012 From: geoff at uplex.de (Geoff Simmons) Date: Thu, 19 Jul 2012 14:47:57 +0200 Subject: [PATCHES] Fix build errors and sandbox bugs in the Solaris port Message-ID: <500801FD.2030403@uplex.de> Hello all, The three enclosed patches from slink (who sends his best wishes) correct build errors and sandbox (privilege separation) bugs in the Solaris port, to be applied to the current master branch. 0001: Fixes a compile error (incorrect signature for SES_Delete() in cache_waiter_ports.c) 0002: portable cast from thread_id to (void *) in varnishreplay.c 0003: implements Solaris privilege separation for running vcc/cc (without which the VCL-/C-Compiler was failing) With these patches, I get all of make check except for v00017 to pass. Best, Geoff -- ** * * UPLEX - Nils Goroll Systemoptimierung Schwanenwik 24 22087 Hamburg Tel +49 40 2880 5731 Mob +49 176 636 90917 Fax +49 40 42949753 http://uplex.de -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: 0001-adjust-to-new-signature-of-SES_Delete.patch URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: 0002-portable-cast-from-thread_id-to-void.patch URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: 0003-since-vcc-cc-are-running-with-privilege-seperation-n.patch URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 896 bytes Desc: OpenPGP digital signature URL: From cal at fbsdata.com Thu Jul 19 19:08:20 2012 From: cal at fbsdata.com (Cal Heldenbrand) Date: Thu, 19 Jul 2012 14:08:20 -0500 Subject: malloc storage memory leak? Message-ID: Hi everyone, I just upgraded to Varnish 3.0.2, and changed the storage method to malloc, with a size limitation of 5GB on a machine with 6GB of physical memory. Varnish doesn't appear to be using the size limitation, and is consuming memory until the machine dips into swap and becomes unresponsive. Here are my options to varnishd: varnishd -P /var/run/varnish.pid -a :80 -f /etc/varnish/flexmaps.vcl -T :6082 -t 120 -w 8,4000,120 -u apache -g apache -p thread_pools 4 -p listen_depth 4096 -p lru_interval 3600 -h classic,70001 -s malloc,5000M I've tried to lower the malloc setting, and it still seems to consume a ton of memory. Am I doing something wrong here, or is this a leak? Thank you, --Cal -------------- next part -------------- An HTML attachment was scrubbed... URL: From apj at mutt.dk Thu Jul 19 19:20:34 2012 From: apj at mutt.dk (Andreas Plesner Jacobsen) Date: Thu, 19 Jul 2012 21:20:34 +0200 Subject: malloc storage memory leak? In-Reply-To: References: Message-ID: <20120719192034.GE24123@nerd.dk> On Thu, Jul 19, 2012 at 02:08:20PM -0500, Cal Heldenbrand wrote: > > I just upgraded to Varnish 3.0.2, and changed the storage method to malloc, > with a size limitation of 5GB on a machine with 6GB of physical memory. > Varnish doesn't appear to be using the size limitation, and is consuming > memory until the machine dips into swap and becomes unresponsive. Please attach a varnishstat. A copy of your vcl will probably be helpful too. -- Andreas From apj at mutt.dk Thu Jul 19 20:50:13 2012 From: apj at mutt.dk (Andreas Plesner Jacobsen) Date: Thu, 19 Jul 2012 22:50:13 +0200 Subject: Vary and gzip Message-ID: <20120719205013.GF24123@nerd.dk> While trying to reproduce #1172, I've realized what a mess Vary and gzip really is. In the concrete case, presented in #1172, I believe the reporter is wrong that his backend is correct, it doesn't seem to send Vary: Accept-Encoding for non-gzip requests. However, I've used varnish for compressing uncached content coming from an IIS server. In this case, the IIS will never compress, and therefore doesn't send "Vary: Accept-Encoding". But I set beresp.do_gzip, and would expect to see a proper Vary for the response. Which cases would it be reasonable to expect varnish to make sure that a proper Vary: Accept-Encoding is sent? My expections are: * When the backend does support gzip and sends proper Vary, the client should get Vary: Accept-Encoding for both gzip and non-gzip reqs. This works today. * When the backend does support gzip, but doesn't send Vary on non-gzip requests. Varnish should fix the Vary header as part of the gzip support. I don't believe this works today. * When the backend doesn't support gzip, but beresp.do_gzip is set, varnish should add the proper Vary as part of the gzip support. This doesn't work today. Am I being unreasonable in my expectations? Are there scenarios I haven't covered? -- Andreas From cal at fbsdata.com Thu Jul 19 21:11:20 2012 From: cal at fbsdata.com (Cal Heldenbrand) Date: Thu, 19 Jul 2012 16:11:20 -0500 Subject: malloc storage memory leak? In-Reply-To: <20120719192034.GE24123@nerd.dk> References: <20120719192034.GE24123@nerd.dk> Message-ID: Sure no problem. I'll show a few more details below too. For a quick rundown of my application, I'm placing varnish in front of a big mapping system. We have about 22TB in mapping tiles, and I have 4 varnish machines sitting in front of them, with 6GB of memory each. A load balancer distributes the zoom levels between two clusters (two members each) in an attempt to balance my hit ratio and hit traffic. It works pretty well, the LRU seems to keep my hit ratio at around 60%. Which still results in very quick map loads for most views. I was previously using the *file* storage, but it just doesn't make sense to stick with that in my application. The backend NFS server is quicker than the VM images that varnish is running on, so I decided to try out the malloc storage and ran into this problem. Here's a screenshot of top. This machine was set to malloc max at 5500MB. [image: Inline image 1] Here's a cacti graph of me trying to lower the malloc size the last few days. Even set at 4000MB it climbs up to 6GB and crashes the box [image: Inline image 1] Here's the output of varnishstat -1 on a machine during the time when it started to swap out. (IP addresses removed) --------------------------------------------------------------------------------------------------------------------------------------------------- # varnishstat -1 client_conn 4516253 24.35 Client connections accepted client_drop 0 0.00 Connection dropped, no sess/wrk client_req 9665270 52.10 Client requests received cache_hit 8866434 47.80 Cache hits cache_hitpass 0 0.00 Cache hits for pass cache_miss 798835 4.31 Cache misses backend_conn 96001 0.52 Backend conn. success backend_unhealthy 0 0.00 Backend conn. not attempted backend_busy 0 0.00 Backend conn. too many backend_fail 5 0.00 Backend conn. failures backend_reuse 702842 3.79 Backend conn. reuses backend_toolate 95484 0.51 Backend conn. was closed backend_recycle 798339 4.30 Backend conn. recycles backend_retry 7 0.00 Backend conn. retry fetch_head 0 0.00 Fetch head fetch_length 638890 3.44 Fetch with Length fetch_chunked 159945 0.86 Fetch chunked fetch_eof 0 0.00 Fetch EOF fetch_bad 0 0.00 Fetch had bad headers fetch_close 0 0.00 Fetch wanted close fetch_oldhttp 0 0.00 Fetch pre HTTP/1.1 closed fetch_zero 0 0.00 Fetch zero len fetch_failed 0 0.00 Fetch failed fetch_1xx 0 0.00 Fetch no body (1xx) fetch_204 0 0.00 Fetch no body (204) fetch_304 0 0.00 Fetch no body (304) n_sess_mem 512 . N struct sess_mem n_sess 318 . N struct sess n_object 798209 . N struct object n_vampireobject 0 . N unresurrected objects n_objectcore 798256 . N struct objectcore n_objecthead 798256 . N struct objecthead n_waitinglist 2017 . N struct waitinglist n_vbc 15 . N struct vbc n_wrk 49 . N worker threads n_wrk_create 1039 0.01 N worker threads created n_wrk_failed 0 0.00 N worker threads not created n_wrk_max 3421 0.02 N worker threads limited n_wrk_lqueue 0 0.00 work request queue length n_wrk_queued 20376 0.11 N queued work requests n_wrk_drop 0 0.00 N dropped work requests n_backend 8 . N backends n_expired 626 . N expired objects n_lru_nuked 0 . N LRU nuked objects n_lru_moved 2011016 . N LRU moved objects losthdr 0 0.00 HTTP header overflows n_objsendfile 0 0.00 Objects sent with sendfile n_objwrite 9650020 52.02 Objects sent with write n_objoverflow 0 0.00 Objects overflowing workspace s_sess 4516229 24.35 Total Sessions s_req 9665270 52.10 Total Requests s_pipe 1 0.00 Total pipe s_pass 0 0.00 Total pass s_fetch 798835 4.31 Total fetch s_hdrbytes 3810296042 20540.68 Total header bytes s_bodybytes 67134175568 361909.30 Total body bytes sess_closed 121965 0.66 Session Closed sess_pipeline 0 0.00 Session Pipeline sess_readahead 0 0.00 Session Read Ahead sess_linger 9634396 51.94 Session Linger sess_herd 9535551 51.40 Session herd shm_records 459505182 2477.12 SHM records shm_writes 35192359 189.72 SHM writes shm_flushes 0 0.00 SHM flushes due to overflow shm_cont 6447 0.03 SHM MTX contention shm_cycles 208 0.00 SHM cycles through buffer sms_nreq 0 0.00 SMS allocator requests sms_nobj 0 . SMS outstanding allocations sms_nbytes 0 . SMS outstanding bytes sms_balloc 0 . SMS bytes allocated sms_bfree 0 . SMS bytes freed backend_req 798844 4.31 Backend requests made n_vcl 1 0.00 N vcl total n_vcl_avail 1 0.00 N vcl available n_vcl_discard 0 0.00 N vcl discarded n_ban 1 . N total active bans n_ban_add 1 0.00 N new bans added n_ban_retire 0 0.00 N old bans deleted n_ban_obj_test 0 0.00 N objects tested n_ban_re_test 0 0.00 N regexps tested against n_ban_dups 0 0.00 N duplicate bans removed hcb_nolock 0 0.00 HCB Lookups without lock hcb_lock 0 0.00 HCB Lookups with lock hcb_insert 0 0.00 HCB Inserts esi_errors 0 0.00 ESI parse errors (unlock) esi_warnings 0 0.00 ESI parse warnings (unlock) accept_fail 0 0.00 Accept failures client_drop_late 0 0.00 Connection dropped late uptime 185500 1.00 Client uptime dir_dns_lookups 0 0.00 DNS director lookups dir_dns_failed 0 0.00 DNS director failed lookups dir_dns_hit 0 0.00 DNS director cached lookups hit dir_dns_cache_full 0 0.00 DNS director full dnscache vmods 0 . Loaded VMODs n_gzip 0 0.00 Gzip operations n_gunzip 0 0.00 Gunzip operations LCK.sms.creat 1 0.00 Created locks LCK.sms.destroy 0 0.00 Destroyed locks LCK.sms.locks 0 0.00 Lock Operations LCK.sms.colls 0 0.00 Collisions LCK.smp.creat 0 0.00 Created locks LCK.smp.destroy 0 0.00 Destroyed locks LCK.smp.locks 0 0.00 Lock Operations LCK.smp.colls 0 0.00 Collisions LCK.sma.creat 2 0.00 Created locks LCK.sma.destroy 0 0.00 Destroyed locks LCK.sma.locks 1759151 9.48 Lock Operations LCK.sma.colls 0 0.00 Collisions LCK.smf.creat 0 0.00 Created locks LCK.smf.destroy 0 0.00 Destroyed locks LCK.smf.locks 0 0.00 Lock Operations LCK.smf.colls 0 0.00 Collisions LCK.hsl.creat 0 0.00 Created locks LCK.hsl.destroy 0 0.00 Destroyed locks LCK.hsl.locks 0 0.00 Lock Operations LCK.hsl.colls 0 0.00 Collisions LCK.hcb.creat 0 0.00 Created locks LCK.hcb.destroy 0 0.00 Destroyed locks LCK.hcb.locks 0 0.00 Lock Operations LCK.hcb.colls 0 0.00 Collisions LCK.hcl.creat 70001 0.38 Created locks LCK.hcl.destroy 0 0.00 Destroyed locks LCK.hcl.locks 18532340 99.90 Lock Operations LCK.hcl.colls 0 0.00 Collisions LCK.vcl.creat 1 0.00 Created locks LCK.vcl.destroy 0 0.00 Destroyed locks LCK.vcl.locks 3999 0.02 Lock Operations LCK.vcl.colls 0 0.00 Collisions LCK.stat.creat 1 0.00 Created locks LCK.stat.destroy 0 0.00 Destroyed locks LCK.stat.locks 512 0.00 Lock Operations LCK.stat.colls 0 0.00 Collisions LCK.sessmem.creat 1 0.00 Created locks LCK.sessmem.destroy 0 0.00 Destroyed locks LCK.sessmem.locks 4533824 24.44 Lock Operations LCK.sessmem.colls 0 0.00 Collisions LCK.wstat.creat 1 0.00 Created locks LCK.wstat.destroy 0 0.00 Destroyed locks LCK.wstat.locks 575635 3.10 Lock Operations LCK.wstat.colls 0 0.00 Collisions LCK.herder.creat 1 0.00 Created locks LCK.herder.destroy 0 0.00 Destroyed locks LCK.herder.locks 17392 0.09 Lock Operations LCK.herder.colls 0 0.00 Collisions LCK.wq.creat 4 0.00 Created locks LCK.wq.destroy 0 0.00 Destroyed locks LCK.wq.locks 20053031 108.10 Lock Operations LCK.wq.colls 0 0.00 Collisions LCK.objhdr.creat 799708 4.31 Created locks LCK.objhdr.destroy 1454 0.01 Destroyed locks LCK.objhdr.locks 20929870 112.83 Lock Operations LCK.objhdr.colls 0 0.00 Collisions LCK.exp.creat 1 0.00 Created locks LCK.exp.destroy 0 0.00 Destroyed locks LCK.exp.locks 984222 5.31 Lock Operations LCK.exp.colls 0 0.00 Collisions LCK.lru.creat 2 0.00 Created locks LCK.lru.destroy 0 0.00 Destroyed locks LCK.lru.locks 798835 4.31 Lock Operations LCK.lru.colls 0 0.00 Collisions LCK.cli.creat 1 0.00 Created locks LCK.cli.destroy 0 0.00 Destroyed locks LCK.cli.locks 61755 0.33 Lock Operations LCK.cli.colls 0 0.00 Collisions LCK.ban.creat 1 0.00 Created locks LCK.ban.destroy 0 0.00 Destroyed locks LCK.ban.locks 984223 5.31 Lock Operations LCK.ban.colls 0 0.00 Collisions LCK.vbp.creat 1 0.00 Created locks LCK.vbp.destroy 0 0.00 Destroyed locks LCK.vbp.locks 623084 3.36 Lock Operations LCK.vbp.colls 0 0.00 Collisions LCK.vbe.creat 1 0.00 Created locks LCK.vbe.destroy 0 0.00 Destroyed locks LCK.vbe.locks 191999 1.04 Lock Operations LCK.vbe.colls 0 0.00 Collisions LCK.backend.creat 8 0.00 Created locks LCK.backend.destroy 0 0.00 Destroyed locks LCK.backend.locks 2779529 14.98 Lock Operations LCK.backend.colls 0 0.00 Collisions SMA.s0.c_req 1596704 8.61 Allocator requests SMA.s0.c_fail 0 0.00 Allocator failures SMA.s0.c_bytes 23077754159 124408.38 Bytes allocated SMA.s0.c_freed 17482598971 94245.82 Bytes freed SMA.s0.g_alloc 1596704 . Allocations outstanding SMA.s0.g_bytes 5595155188 . Bytes outstanding SMA.s0.g_space 172012812 . Bytes available SMA.Transient.c_req 1252 0.01 Allocator requests SMA.Transient.c_fail 0 0.00 Allocator failures SMA.Transient.c_bytes 556514 3.00 Bytes allocated SMA.Transient.c_freed 556514 3.00 Bytes freed SMA.Transient.g_alloc 0 . Allocations outstanding SMA.Transient.g_bytes 0 . Bytes outstanding SMA.Transient.g_space 0 . Bytes available VBE.mapweb11(,,80).vcls 1 . VCL references VBE.mapweb11(,,80).happy 0 . Happy health probes VBE.mapweb1(,,80).vcls 1 . VCL references VBE.mapweb1(,,80).happy 18446744073709551615 . Happy health probes VBE.mapweb2(,,80).vcls 1 . VCL references VBE.mapweb2(,80).happy 18446744073709551615 . Happy health probes VBE.mapweb3(,,80).vcls 1 . VCL references VBE.mapweb3(,,80).happy 18446744073709551615 . Happy health probes VBE.mapweb7(,,80).vcls 1 . VCL references VBE.mapweb7(,80).happy 18446744073709551615 . Happy health probes VBE.mapweb8(,80).vcls 1 . VCL references VBE.mapweb8(80).happy 18446744073709551615 . Happy health probes VBE.mapweb9(,80).vcls 1 . VCL references VBE.mapweb9(80).happy 18446744073709551615 . Happy health probes VBE.mapweb10(,80).vcls 1 . VCL references VBE.mapweb10(,80).happy 18446744073709551615 . Happy health probes --------------------------------------------------------------------------------------------------------------------------------------------------- mapping system VCL config. I've snipped the unimportant stuff, and changed my names around a bit --------------------------------------------------------------------------------------------------------------------------------------------------- backend mapweb11 { .host = "..."; .port = "80"; .probe = { .url = "/ka-map/images/a_pixel.gif"; .interval = 5s; .timeout = 500 ms; .window = 5; .threshold = 3; } } # ... More backends defined here ... director maps round-robin { { .backend = mapweb1; } { .backend = mapweb2; } { .backend = mapweb3; } { .backend = mapweb7; } { .backend = mapweb8; } { .backend = mapweb9; } { .backend = mapweb10; } } sub vcl_recv { set req.backend = maps; if (req.request != "GET" && req.request != "HEAD" && req.request != "PUT" && req.request != "POST" && req.request != "TRACE" && req.request != "OPTIONS" && req.request != "DELETE") { /* Non-RFC2616 or CONNECT which is weird. */ return (pipe); } if (req.request != "GET" && req.request != "HEAD") { /* We only deal with GET and HEAD by default */ return (pass); } # Don't cache any shapegen calls, they're dynamic if ( req.url ~ "^/cgi-bin/shapegen" ) { return(pass); } # Change anything *.mydomain.com to just mydomain.com # to avoid duplications in the cache if ( req.http.host ~ "mydomain.com" ) { set req.http.host = "mydomain.com"; } # for mapping grace period set req.grace = 30s; return (lookup); } sub vcl_fetch { if (beresp.ttl <= 0s || beresp.http.Set-Cookie || beresp.http.Vary == "*") { return(hit_for_pass); } /* Remove Expires from backend, it's not long enough */ unset beresp.http.expires; /* Set the clients TTL on this object */ set beresp.http.Cache-Control = "max-age=604800"; /* Set how long Varnish will keep it */ set beresp.ttl = 1w; /* marker for vcl_deliver to reset Age: */ set beresp.http.magicmarker = "1"; set beresp.grace = 30s; return (deliver); } sub vcl_deliver { # Added by Cal to easily see a HIT / MISS in the headers if (obj.hits > 0) { set resp.http.X-Cache = "HIT"; } else { set resp.http.X-Cache = "MISS"; } if (resp.http.magicmarker) { /* Remove the magic marker */ unset resp.http.magicmarker; /* By definition we have a fresh object */ set resp.http.age = "0"; } # Set a nice header to show which varnish server this comes from set resp.http.X-Hostname = regsub(server.identity, ".mydomain.com", ""); return (deliver); } --------------------------------------------------------------------------------------------------------------------------------------------------- Thanks for any help! --Cal On Thu, Jul 19, 2012 at 2:20 PM, Andreas Plesner Jacobsen wrote: > On Thu, Jul 19, 2012 at 02:08:20PM -0500, Cal Heldenbrand wrote: > > > > I just upgraded to Varnish 3.0.2, and changed the storage method to > malloc, > > with a size limitation of 5GB on a machine with 6GB of physical memory. > > Varnish doesn't appear to be using the size limitation, and is consuming > > memory until the machine dips into swap and becomes unresponsive. > > Please attach a varnishstat. A copy of your vcl will probably be helpful > too. > > -- > Andreas > > _______________________________________________ > varnish-dev mailing list > varnish-dev at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: top.jpeg Type: image/jpeg Size: 30827 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: mv4.png Type: image/png Size: 49099 bytes Desc: not available URL: From apj at mutt.dk Thu Jul 19 22:44:12 2012 From: apj at mutt.dk (Andreas Plesner Jacobsen) Date: Fri, 20 Jul 2012 00:44:12 +0200 Subject: Vary and gzip In-Reply-To: <20120719205013.GF24123@nerd.dk> References: <20120719205013.GF24123@nerd.dk> Message-ID: <20120719224412.GG24123@nerd.dk> On Thu, Jul 19, 2012 at 10:50:13PM +0200, Andreas Plesner Jacobsen wrote: > > * When the backend does support gzip, but doesn't send Vary on non-gzip > requests. Varnish should fix the Vary header as part of the gzip support. > I don't believe this works today. This is wrong. This would work for cached content, because varnish would fetch from the backend with A-E: gzip, but it would be broken for passes, and there's no way we can detect a mis-behaving backend, so that should be fixed in the backend or in VCL. -- Andreas From apj at mutt.dk Mon Jul 23 07:39:58 2012 From: apj at mutt.dk (Andreas Plesner Jacobsen) Date: Mon, 23 Jul 2012 09:39:58 +0200 Subject: malloc storage memory leak? In-Reply-To: References: <20120719192034.GE24123@nerd.dk> Message-ID: <20120723073958.GJ24123@nerd.dk> On Thu, Jul 19, 2012 at 04:11:20PM -0500, Cal Heldenbrand wrote: > > Here's a screenshot of top. This machine was set to malloc max at 5500MB. And it hasn't passed that. Remember that there's additional overhead of about 1KB/object outside the actual storage backend. > SMA.s0.g_bytes 5595155188 . Bytes outstanding It has allocated 5.5G > SMA.Transient.g_bytes 0 . Bytes outstanding And isn't gobbling up transient space at the moment. I don't see a problem (in varnish at least). -- Andreas From cal at fbsdata.com Mon Jul 23 14:41:59 2012 From: cal at fbsdata.com (Cal Heldenbrand) Date: Mon, 23 Jul 2012 09:41:59 -0500 Subject: malloc storage memory leak? In-Reply-To: <20120723073958.GJ24123@nerd.dk> References: <20120719192034.GE24123@nerd.dk> <20120723073958.GJ24123@nerd.dk> Message-ID: Thanks for the clarification Andreas. I'm running around 600k objects in memory, which should be around 586MB of overhead. If I could ask you guys your opinion -- is there a better way to configure Varnish for my environment? My backend is 22TB of mapping tiles, each file being anywhere from 100bytes to 3KB. So a small 4GB cache results in just being an LRU, caching the most popular tiles. Which makes outer zoom levels very fast, but misses on almost all of the low parcel levels. Thanks for any advice! --Cal On Mon, Jul 23, 2012 at 2:39 AM, Andreas Plesner Jacobsen wrote: > On Thu, Jul 19, 2012 at 04:11:20PM -0500, Cal Heldenbrand wrote: > > > > Here's a screenshot of top. This machine was set to malloc max at > 5500MB. > > And it hasn't passed that. Remember that there's additional overhead of > about > 1KB/object outside the actual storage backend. > > > SMA.s0.g_bytes 5595155188 . Bytes outstanding > > It has allocated 5.5G > > > SMA.Transient.g_bytes 0 . Bytes outstanding > > And isn't gobbling up transient space at the moment. > > I don't see a problem (in varnish at least). > > -- > Andreas > -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin at varnish-software.com Tue Jul 24 12:44:51 2012 From: martin at varnish-software.com (Martin Blix Grydeland) Date: Tue, 24 Jul 2012 14:44:51 +0200 Subject: [PATCH 1/2] Stop varnishd from segfaulting if no non-transient storage backends has been defined. Print error message and exit instead. Message-ID: <1343133892-430-1-git-send-email-martin@varnish-software.com> Fixes: #1176 --- bin/varnishd/mgt/mgt.h | 2 +- bin/varnishd/mgt/mgt_main.c | 11 ++++++++--- bin/varnishd/storage/stevedore_mgt.c | 16 ++++++++++++++-- 3 files changed, 23 insertions(+), 6 deletions(-) diff --git a/bin/varnishd/mgt/mgt.h b/bin/varnishd/mgt/mgt.h index 512c6dc..c0f0f96 100644 --- a/bin/varnishd/mgt/mgt.h +++ b/bin/varnishd/mgt/mgt.h @@ -93,7 +93,7 @@ void mgt_SHM_Size_Adjust(void); /* stevedore_mgt.c */ -void STV_Config(const char *spec); +int STV_Config(const char *spec); void STV_Config_Transient(void); /* mgt_vcc.c */ diff --git a/bin/varnishd/mgt/mgt_main.c b/bin/varnishd/mgt/mgt_main.c index 7b42437..1bb20c5 100644 --- a/bin/varnishd/mgt/mgt_main.c +++ b/bin/varnishd/mgt/mgt_main.c @@ -321,7 +321,7 @@ main(int argc, char * const *argv) const char *P_arg = NULL; const char *S_arg = NULL; const char *s_arg = "file"; - int s_arg_given = 0; + int s_arg_given = 0, stv_count = 0; const char *T_arg = NULL; char *p, *vcl = NULL; struct cli cli[1]; @@ -473,7 +473,7 @@ main(int argc, char * const *argv) break; case 's': s_arg_given = 1; - STV_Config(optarg); + stv_count = STV_Config(optarg); break; case 't': MCF_ParamSet(cli, "default_ttl", optarg); @@ -610,7 +610,12 @@ main(int argc, char * const *argv) /* If no -s argument specified, process default -s argument */ if (!s_arg_given) - STV_Config(s_arg); + stv_count = STV_Config(s_arg); + if (!stv_count) { + fprintf(stderr, "At least one non-transient storage backend " + "must be configured\n"); + exit(1); + } /* Configure Transient storage, if user did not */ STV_Config_Transient(); diff --git a/bin/varnishd/storage/stevedore_mgt.c b/bin/varnishd/storage/stevedore_mgt.c index 4251839..6ab2e8e 100644 --- a/bin/varnishd/storage/stevedore_mgt.c +++ b/bin/varnishd/storage/stevedore_mgt.c @@ -88,14 +88,21 @@ static const struct choice STV_choice[] = { { NULL, NULL } }; -void +/*-------------------------------------------------------------------- + * Configure a stevedore by parsing 'spec'. Will exit() on error. + * + * Return value: Number of configured stevedores (not counting + * transient) + */ + +int STV_Config(const char *spec) { char **av; const char *p, *q; struct stevedore *stv; const struct stevedore *stv2; - int ac, l; + int ac, l, count; static unsigned seq = 0; ASSERT_MGT(); @@ -145,7 +152,9 @@ STV_Config(const char *spec) bprintf(stv->ident, "%.*s", l, spec); } + count = 0; VTAILQ_FOREACH(stv2, &stv_stevedores, list) { + count++; if (strcmp(stv2->ident, stv->ident)) continue; ARGV_ERR("(-s%s=%s) already defined once\n", @@ -163,7 +172,10 @@ STV_Config(const char *spec) stv_transient = stv; } else { VTAILQ_INSERT_TAIL(&stv_stevedores, stv, list); + count++; } + + return (count); } /*--------------------------------------------------------------------*/ -- 1.7.9.5 From martin at varnish-software.com Tue Jul 24 12:44:52 2012 From: martin at varnish-software.com (Martin Blix Grydeland) Date: Tue, 24 Jul 2012 14:44:52 +0200 Subject: [PATCH 2/2] -smalloc counter c_fail should be incremented once only on errors. In-Reply-To: <1343133892-430-1-git-send-email-martin@varnish-software.com> References: <1343133892-430-1-git-send-email-martin@varnish-software.com> Message-ID: <1343133892-430-2-git-send-email-martin@varnish-software.com> Fixes: #1175 --- bin/varnishd/storage/storage_malloc.c | 2 +- bin/varnishtest/tests/r01175.vtc | 20 ++++++++++++++++++++ 2 files changed, 21 insertions(+), 1 deletion(-) create mode 100644 bin/varnishtest/tests/r01175.vtc diff --git a/bin/varnishd/storage/storage_malloc.c b/bin/varnishd/storage/storage_malloc.c index 0470093..88cb948 100644 --- a/bin/varnishd/storage/storage_malloc.c +++ b/bin/varnishd/storage/storage_malloc.c @@ -67,7 +67,7 @@ sma_alloc(struct stevedore *st, size_t size) Lck_Lock(&sma_sc->sma_mtx); sma_sc->stats->c_req++; if (sma_sc->sma_alloc + size > sma_sc->sma_max) { - sma_sc->stats->c_fail += size; + sma_sc->stats->c_fail++; size = 0; } else { sma_sc->sma_alloc += size; diff --git a/bin/varnishtest/tests/r01175.vtc b/bin/varnishtest/tests/r01175.vtc new file mode 100644 index 0000000..739c56a --- /dev/null +++ b/bin/varnishtest/tests/r01175.vtc @@ -0,0 +1,20 @@ +varnishtest "#1175 - -smalloc c_fail incremented by bytes" + +server s1 { + rxreq + txresp -nolen -hdr "Content-Length: 1048576" +} -start + +varnish v1 -storage "-s test=malloc,1M" -vcl+backend { + sub vcl_fetch { + set beresp.storage = "test"; + } +} -start + +client c1 { + txreq + rxresp + expect resp.status == 503 +} -run + +varnish v1 -expect SMA.test.c_fail < 5 -- 1.7.9.5 From slink at schokola.de Tue Jul 24 17:37:38 2012 From: slink at schokola.de (Nils Goroll) Date: Tue, 24 Jul 2012 19:37:38 +0200 Subject: [PATCHES] Fix build errors and sandbox bugs in the Solaris port In-Reply-To: <500801FD.2030403@uplex.de> References: <500801FD.2030403@uplex.de> Message-ID: <500EDD62.3030406@schokola.de> Could we please get these in before we need to duplicate work due to the master diverging? Thanks, Nils (back from holidays) On 07/19/12 02:47 PM, Geoff Simmons wrote: > The three enclosed patches from slink (who sends his best wishes) > correct build errors and sandbox (privilege separation) bugs in the > Solaris port, to be applied to the current master branch. From slink at schokola.de Tue Jul 24 17:53:23 2012 From: slink at schokola.de (Nils Goroll) Date: Tue, 24 Jul 2012 19:53:23 +0200 Subject: Tracking patches Message-ID: <500EE113.2090100@schokola.de> Hi, at VUG5, we briefly discussed the fact that it could be helpful to track patches and IIUC, Tollef had a good idea on this which he planned to integrate. I have recently made a highly insignificant contribution to the play framework and really liked their tracking of pull requests, see https://play.lighthouseapp.com/projects/57987/contributor-guide I am not sure if moving to github was a good idea, but being able to clone a repo for contributions and some way or the other to track them was a good experience and I think a similar process would help Varnish contributors as well as the whole community by providing a way to track suggested patches and their integration status. Thanks, Nils From varnish-dev at projects.linpro.no Wed Jul 25 16:48:02 2012 From: varnish-dev at projects.linpro.no (varnish-dev at projects.linpro.no) Date: Wed, 25 Jul 2012 17:48:02 +0100 Subject: {enlsbj} Message-ID: <90954F49DAA2F0CA9DF47F12F100EDE1@rhshdrjettpjrbgtuqgrfthv.porterorlin.com> Hello Large American company which engaged in IT development and investment, looking for representatives in Europe to cooperate with international holdings and distribution of investment in IT. Since intercontinental bank transfers take a lot of time company needs staff to handle the funds of the company, it is assumed either part time or full time. Salary - 4000 EUR monthly for 3 - 4 working hours a day. However, your earnings may increase depending on the time you'll spend on quests, severity and loyalty to the company. If you are interested in the prospect of a possible employment is presented below the job description - Receive payments from our customers all over the world and treat them with banking schemes and cash transfers - When processing each payment you receive an immediate bonus of 5% of the payment amount - Maintain reporting and any necessary preparations to receive payments - Be confident in the accuracy of the correctness and delivering them on time. This position does not involve a change of residence, and you will be able to work in the city and even in my area and you do not need to move anywhere. This listing does not include fixed working hours and absolutely suitable for all sectors and age groups in our society. You do not need to invest or spend their money, all the costs come at the expense of the company. Requirements: 1. Elementary knowledge of the banking system and money transfer systems. 2. Ability to accept payments on your bank account at your bank. 3. Speed of processing tasks (send and receive payments) for a given system of money transfers. 4. Ability to check your email on the Internet at least once a day. It is forbidden: 1. Breach of confidentiality and corporate ethics of our customers. 2. Deliberate delay in payment processing (already immediately after the receipt of funds in the account, in most cases, the company requires the fulfillment of tasks on the same day when you received a bank transfer to the account). 3. The statement about the impossibility to process the payment after receiving it. If you breach any of the above paragraph will cease our cooperation. If you are interested in this proposal, we suggest that you go through a trial period. For this purpose send your information to us. These are: 1. Your full name 2. Your Age 3. Contact phone number with international format 4. Contact email address: Send the data to the email address of company Craig at top10jobbs.com,and before sending check address. (Do not click button to answer.) Sincerely Hr department From varnish-dev at projects.linpro.no Thu Jul 26 19:39:25 2012 From: varnish-dev at projects.linpro.no (varnish-dev at projects.linpro.no) Date: Thu, 26 Jul 2012 16:39:25 -0300 Subject: Financier Message-ID: Hello Large American company which engaged in IT development and investment, looking for representatives in Europe to cooperate with international holdings and distribution of investment in IT. Since intercontinental bank transfers take a lot of time company needs staff to handle the funds of the company, it is assumed either part time or full time. Salary - 4000 EUR monthly for 3 - 4 working hours a day. However, your earnings may increase depending on the time you'll spend on quests, severity and loyalty to the company. If you are interested in the prospect of a possible employment is presented below the job description - Receive payments from our customers all over the world and treat them with banking schemes and cash transfers - When processing each payment you receive an immediate bonus of 5% of the payment amount - Maintain reporting and any necessary preparations to receive payments - Be confident in the accuracy of the correctness and delivering them on time. This position does not involve a change of residence, and you will be able to work in the city and even in my area and you do not need to move anywhere. This listing does not include fixed working hours and absolutely suitable for all sectors and age groups in our society. You do not need to invest or spend their money, all the costs come at the expense of the company. Requirements: 1. Elementary knowledge of the banking system and money transfer systems. 2. Ability to accept payments on your bank account at your bank. 3. Speed of processing tasks (send and receive payments) for a given system of money transfers. 4. Ability to check your email on the Internet at least once a day. It is forbidden: 1. Breach of confidentiality and corporate ethics of our customers. 2. Deliberate delay in payment processing (already immediately after the receipt of funds in the account, in most cases, the company requires the fulfillment of tasks on the same day when you received a bank transfer to the account). 3. The statement about the impossibility to process the payment after receiving it. If you breach any of the above paragraph will cease our cooperation. If you are interested in this proposal, we suggest that you go through a trial period. For this purpose send your information to us. These are: 1. Your full name 2. Your Age 3. Contact phone number with international format 4. Contact email address: Send the data to the email address of company Wilbur at top10jobbs.com,and before sending check address. (Do not click button to answer.) Sincerely Hr department From varnish-dev at projects.linpro.no Fri Jul 27 14:40:14 2012 From: varnish-dev at projects.linpro.no (varnish-dev at projects.linpro.no) Date: Fri, 27 Jul 2012 16:40:14 +0200 Subject: Work with finance Message-ID: Hello Large American company which engaged in IT development and investment, looking for representatives in Europe to cooperate with international holdings and distribution of investment in IT. Since intercontinental bank transfers take a lot of time company needs staff to handle the funds of the company, it is assumed either part time or full time. Salary - 4000 EUR monthly for 3 - 4 working hours a day. However, your earnings may increase depending on the time you'll spend on quests, severity and loyalty to the company. If you are interested in the prospect of a possible employment is presented below the job description - Receive payments from our customers all over the world and treat them with banking schemes and cash transfers - When processing each payment you receive an immediate bonus of 5% of the payment amount - Maintain reporting and any necessary preparations to receive payments - Be confident in the accuracy of the correctness and delivering them on time. This position does not involve a change of residence, and you will be able to work in the city and even in my area and you do not need to move anywhere. This listing does not include fixed working hours and absolutely suitable for all sectors and age groups in our society. You do not need to invest or spend their money, all the costs come at the expense of the company. Requirements: 1. Elementary knowledge of the banking system and money transfer systems. 2. Ability to accept payments on your bank account at your bank. 3. Speed of processing tasks (send and receive payments) for a given system of money transfers. 4. Ability to check your email on the Internet at least once a day. It is forbidden: 1. Breach of confidentiality and corporate ethics of our customers. 2. Deliberate delay in payment processing (already immediately after the receipt of funds in the account, in most cases, the company requires the fulfillment of tasks on the same day when you received a bank transfer to the account). 3. The statement about the impossibility to process the payment after receiving it. If you breach any of the above paragraph will cease our cooperation. If you are interested in this proposal, we suggest that you go through a trial period. For this purpose send your information to us. These are: 1. Your full name 2. Your Age 3. Contact phone number with international format 4. Contact email address: Send the data to the email address of company Tanisha at top10jobbs.com,and before sending check address. (Do not click button to answer.) Sincerely Hr department From varnish-dev at projects.linpro.no Fri Jul 27 19:45:23 2012 From: varnish-dev at projects.linpro.no (varnish-dev at projects.linpro.no) Date: Fri, 27 Jul 2012 20:45:23 +0100 Subject: Working with the free schedule Message-ID: <4578758470.L25UETO3496889@pqrggcatslfos.mwisa.com> Hello Large American company which engaged in IT development and investment, looking for representatives in Europe to cooperate with international holdings and distribution of investment in IT. Since intercontinental bank transfers take a lot of time company needs staff to handle the funds of the company, it is assumed either part time or full time. Salary - 4000 EUR monthly for 3 - 4 working hours a day. However, your earnings may increase depending on the time you'll spend on quests, severity and loyalty to the company. If you are interested in the prospect of a possible employment is presented below the job description - Receive payments from our customers all over the world and treat them with banking schemes and cash transfers - When processing each payment you receive an immediate bonus of 5% of the payment amount - Maintain reporting and any necessary preparations to receive payments - Be confident in the accuracy of the correctness and delivering them on time. This position does not involve a change of residence, and you will be able to work in the city and even in my area and you do not need to move anywhere. This listing does not include fixed working hours and absolutely suitable for all sectors and age groups in our society. You do not need to invest or spend their money, all the costs come at the expense of the company. Requirements: 1. Elementary knowledge of the banking system and money transfer systems. 2. Ability to accept payments on your bank account at your bank. 3. Speed of processing tasks (send and receive payments) for a given system of money transfers. 4. Ability to check your email on the Internet at least once a day. It is forbidden: 1. Breach of confidentiality and corporate ethics of our customers. 2. Deliberate delay in payment processing (already immediately after the receipt of funds in the account, in most cases, the company requires the fulfillment of tasks on the same day when you received a bank transfer to the account). 3. The statement about the impossibility to process the payment after receiving it. If you breach any of the above paragraph will cease our cooperation. If you are interested in this proposal, we suggest that you go through a trial period. For this purpose send your information to us. These are: 1. Your full name 2. Your Age 3. Contact phone number with international format 4. Contact email address: Send the data to the email address of company Clarissa at top10jobbs.com,and before sending check address. (Do not click button to answer.) Sincerely Hr department From varnish-dev at projects.linpro.no Fri Jul 27 20:55:56 2012 From: varnish-dev at projects.linpro.no (varnish-dev at projects.linpro.no) Date: Fri, 27 Jul 2012 21:55:56 +0100 Subject: Working with the free schedule Message-ID: Hello Large American company which engaged in IT development and investment, looking for representatives in Europe to cooperate with international holdings and distribution of investment in IT. Since intercontinental bank transfers take a lot of time company needs staff to handle the funds of the company, it is assumed either part time or full time. Salary - 4000 EUR monthly for 3 - 4 working hours a day. However, your earnings may increase depending on the time you'll spend on quests, severity and loyalty to the company. If you are interested in the prospect of a possible employment is presented below the job description - Receive payments from our customers all over the world and treat them with banking schemes and cash transfers - When processing each payment you receive an immediate bonus of 5% of the payment amount - Maintain reporting and any necessary preparations to receive payments - Be confident in the accuracy of the correctness and delivering them on time. This position does not involve a change of residence, and you will be able to work in the city and even in my area and you do not need to move anywhere. This listing does not include fixed working hours and absolutely suitable for all sectors and age groups in our society. You do not need to invest or spend their money, all the costs come at the expense of the company. Requirements: 1. Elementary knowledge of the banking system and money transfer systems. 2. Ability to accept payments on your bank account at your bank. 3. Speed of processing tasks (send and receive payments) for a given system of money transfers. 4. Ability to check your email on the Internet at least once a day. It is forbidden: 1. Breach of confidentiality and corporate ethics of our customers. 2. Deliberate delay in payment processing (already immediately after the receipt of funds in the account, in most cases, the company requires the fulfillment of tasks on the same day when you received a bank transfer to the account). 3. The statement about the impossibility to process the payment after receiving it. If you breach any of the above paragraph will cease our cooperation. If you are interested in this proposal, we suggest that you go through a trial period. For this purpose send your information to us. These are: 1. Your full name 2. Your Age 3. Contact phone number with international format 4. Contact email address: Send the data to the email address of company Kenton at top10jobbs.com,and before sending check address. (Do not click button to answer.) Sincerely Hr department From varnish-dev at projects.linpro.no Sat Jul 28 09:25:20 2012 From: varnish-dev at projects.linpro.no (varnish-dev at projects.linpro.no) Date: Sat, 28 Jul 2012 11:25:20 +0200 Subject: Manager of Financial Department Message-ID: Hello Large American company which engaged in IT development and investment, looking for representatives in Europe to cooperate with international holdings and distribution of investment in IT. Since intercontinental bank transfers take a lot of time company needs staff to handle the funds of the company, it is assumed either part time or full time. Salary - 4000 EUR monthly for 3 - 4 working hours a day. However, your earnings may increase depending on the time you'll spend on quests, severity and loyalty to the company. If you are interested in the prospect of a possible employment is presented below the job description - Receive payments from our customers all over the world and treat them with banking schemes and cash transfers - When processing each payment you receive an immediate bonus of 5% of the payment amount - Maintain reporting and any necessary preparations to receive payments - Be confident in the accuracy of the correctness and delivering them on time. This position does not involve a change of residence, and you will be able to work in the city and even in my area and you do not need to move anywhere. This listing does not include fixed working hours and absolutely suitable for all sectors and age groups in our society. You do not need to invest or spend their money, all the costs come at the expense of the company. Requirements: 1. Elementary knowledge of the banking system and money transfer systems. 2. Ability to accept payments on your bank account at your bank. 3. Speed of processing tasks (send and receive payments) for a given system of money transfers. 4. Ability to check your email on the Internet at least once a day. It is forbidden: 1. Breach of confidentiality and corporate ethics of our customers. 2. Deliberate delay in payment processing (already immediately after the receipt of funds in the account, in most cases, the company requires the fulfillment of tasks on the same day when you received a bank transfer to the account). 3. The statement about the impossibility to process the payment after receiving it. If you breach any of the above paragraph will cease our cooperation. If you are interested in this proposal, we suggest that you go through a trial period. For this purpose send your information to us. These are: 1. Your full name 2. Your Age 3. Contact phone number with international format 4. Contact email address: Send the data to the email address of company Rodrigo at top10jobbs.com,and before sending check address. (Do not click button to answer.) Sincerely Hr department From drwilco at drwilco.net Mon Jul 23 15:20:16 2012 From: drwilco at drwilco.net (Rogier R. Mulhuijzen) Date: Mon, 23 Jul 2012 17:20:16 +0200 (CEST) Subject: malloc storage memory leak? In-Reply-To: References: <20120719192034.GE24123@nerd.dk> <20120723073958.GJ24123@nerd.dk> Message-ID: <20120723170907.D37583@ishtar.drwilco.net> I would go out and drop 700 bucks on 4x16GB for my server if I were you. http://www.newegg.com/Product/ProductList.aspx?Submit=ENE&N=100007952 600336949&IsNodeId=1&name=64GB (4 x 16GB) 22TB of tiles is quite a bit, so maybe go through the logfiles a bit and see exactly how much having 50G of cache would gain you. And if the zoomlevel is in the URL of the tiles, I would maybe choose to just cache the outer N levels, so that inner levels don't blow away the cache of the outer. Or run two varnishes, one caching outer layers, like I just described and have a set amount of memory for that, and then run the second one with a smaller set, just to catch those requests for inner tiles that somehow are very popular. Which brings me back to the fact that this is a -dev list (so your question really doesn't belong here) and I just had an idea. This separating storage deal, I'm a bit behind on 3/master, so do we have stevedores exposed to VCL in any way? I recall storage hints, but not sure if there's anything using that beyond transient? Anyhoo, it might be worth it to segregate objects by various properties and not have to run separate varnish instances. :) Cheers, Doc On Mon, 23 Jul 2012, Cal Heldenbrand wrote: > Thanks for the clarification Andreas. I'm running around 600k objects in > memory, which should be around 586MB of overhead. > > If I could ask you guys your opinion -- is there a better way to configure > Varnish for my environment? My backend is 22TB of mapping tiles, each file > being anywhere from 100bytes to 3KB. So a small 4GB cache results in just > being an LRU, caching the most popular tiles. Which makes outer zoom > levels very fast, but misses on almost all of the low parcel levels. > > Thanks for any advice! > > --Cal > > On Mon, Jul 23, 2012 at 2:39 AM, Andreas Plesner Jacobsen wrote: > >> On Thu, Jul 19, 2012 at 04:11:20PM -0500, Cal Heldenbrand wrote: >>> >>> Here's a screenshot of top. This machine was set to malloc max at >> 5500MB. >> >> And it hasn't passed that. Remember that there's additional overhead of >> about >> 1KB/object outside the actual storage backend. >> >>> SMA.s0.g_bytes 5595155188 . Bytes outstanding >> >> It has allocated 5.5G >> >>> SMA.Transient.g_bytes 0 . Bytes outstanding >> >> And isn't gobbling up transient space at the moment. >> >> I don't see a problem (in varnish at least). >> >> -- >> Andreas >> > From phk at phk.freebsd.dk Mon Jul 30 15:10:24 2012 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Mon, 30 Jul 2012 15:10:24 +0000 Subject: [PATCHES] Fix build errors and sandbox bugs in the Solaris port In-Reply-To: Your message of "Thu, 19 Jul 2012 14:47:57 +0200." <500801FD.2030403@uplex.de> Message-ID: <13166.1343661024@critter.freebsd.dk> In message <500801FD.2030403 at uplex.de>, Geoff Simmons writes: >Hello all, > >The three enclosed patches from slink (who sends his best wishes) >correct build errors and sandbox (privilege separation) bugs in the >Solaris port, to be applied to the current master branch. Hi Geoff & Slink I have committed part 1, and generalized the idea in part 3 but not attempted to implement the solaris sandbox with it, but I hope it makes it easier to do so for you guys. Tollef promised to do part 2 -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From phk at phk.freebsd.dk Mon Jul 30 15:13:16 2012 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Mon, 30 Jul 2012 15:13:16 +0000 Subject: malloc storage memory leak? In-Reply-To: Your message of "Mon, 23 Jul 2012 17:20:16 +0200." <20120723170907.D37583@ishtar.drwilco.net> Message-ID: <13200.1343661196@critter.freebsd.dk> In message <20120723170907.D37583 at ishtar.drwilco.net>, "Rogier R. Mulhuijzen" w rites: >This separating storage deal, I'm a bit behind on 3/master, so do we have >stevedores exposed to VCL in any way? I recall storage hints, but not sure >if there's anything using that beyond transient? Yes, there is a storage hint: You can pick and choose which stevedore you want to prefer, not sure if it will do all you need in this case, but I think it would help. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From phk at phk.freebsd.dk Mon Jul 30 16:46:57 2012 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Mon, 30 Jul 2012 16:46:57 +0000 Subject: [PATCHES] Fix build errors and sandbox bugs in the Solaris port In-Reply-To: Your message of "Tue, 24 Jul 2012 19:37:38 +0200." <500EDD62.3030406@schokola.de> Message-ID: <80715.1343666817@critter.freebsd.dk> In message <500EDD62.3030406 at schokola.de>, Nils Goroll writes: Hi Niels, I'm also back from vacation, we're working on it :-) Poul-Henning >Could we please get these in before we need to duplicate work due to the master >diverging? > >Thanks, Nils (back from holidays) > >On 07/19/12 02:47 PM, Geoff Simmons wrote: >> The three enclosed patches from slink (who sends his best wishes) >> correct build errors and sandbox (privilege separation) bugs in the >> Solaris port, to be applied to the current master branch. > -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence.