From slink at schokola.de Mon Nov 4 17:44:32 2013 From: slink at schokola.de (Nils Goroll) Date: Mon, 04 Nov 2013 18:44:32 +0100 Subject: [PATCH] 3.0.4 solaris needs config.h Message-ID: <5277DD00.6000100@schokola.de> long standing item... -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: 0001-always-include-config.h-first-solaris-and-others.patch URL: From slink at schokola.de Mon Nov 4 17:46:17 2013 From: slink at schokola.de (Nils Goroll) Date: Mon, 04 Nov 2013 18:46:17 +0100 Subject: [PATCH] portable cast from thread_id to (void *) In-Reply-To: <50EDB951.6090400@schokola.de> References: <50EDB951.6090400@schokola.de> Message-ID: <5277DD69.8040004@schokola.de> can we come back to this one ? Only casting to (void *) gives a warning: cast to pointer from integer of different size and I like -Wall Thanks, Nils From slink at schokola.de Mon Nov 4 17:55:06 2013 From: slink at schokola.de (Nils Goroll) Date: Mon, 04 Nov 2013 18:55:06 +0100 Subject: [PATCH] portable cast from thread_id to (void *) In-Reply-To: <5277DD69.8040004@schokola.de> References: <50EDB951.6090400@schokola.de> <5277DD69.8040004@schokola.de> Message-ID: <5277DF7A.7030408@schokola.de> now with attachment On 11/ 4/13 06:46 PM, Nils Goroll wrote: > can we come back to this one ? > > Only casting to (void *) gives a > > warning: cast to pointer from integer of different size > > and I like -Wall > > Thanks, Nils -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: 0002-portable-cast-from-thread_id-to-void.patch URL: From ingvar at redpill-linpro.com Tue Nov 5 12:20:06 2013 From: ingvar at redpill-linpro.com (Ingvar Hagelund) Date: Tue, 05 Nov 2013 13:20:06 +0100 Subject: CVE number needed for Varnish DoS, also heads-up In-Reply-To: <20131031071147.GC15674@err.no> References: <20131030132753.GC10999@err.no> <877gcu3bhe.fsf@mid.deneb.enyo.de> <20131031071147.GC15674@err.no> Message-ID: <5278E276.6090704@redpill-linpro.com> Den 31. okt. 2013 08:11, skrev Tollef Fog Heen: > ]] Florian Weimer >> * Tollef Fog Heen: >>> we've had a denial of service attack reported in Varnish. I believe >>> we should get this fixed in stable (we're working on a patch), but >>> I'd like a CVE # to go with the advisory. Draft advisory at >>> http://etherpad.wikimedia.org/p/WnwRT4FH6e >> is this link already public? If not, what's your disclosure schedule? > Yes, see > https://www.varnish-cache.org/lists/pipermail/varnish-announce/2013-October/000686.html > for our advisory. Diff is > https://www.varnish-cache.org/trac/changeset/4bd5b7991bf602a6c46dd0d65fc04d4b8d9667a6?format=diff&new=4bd5b7991bf602a6c46dd0d65fc04d4b8d9667a6 > ||Fedora/EPEL's tracking bug is here: https://bugzilla.redhat.com/show_bug.cgi?id=1025127 For Fedora, I'll just wait for 3.0.5, I think. f18 and f19 have 3.0.3. I recently commited 3.0.4 to rawhide, but I won't build packages for f18 and f19 now, if 3.0.5 is out in a few days. epel5 has varnish-2.0.6. epel6 has 2.1.5. I have produced a backport for 2.0.6 available here: http://users.linpro.no/ingvar/varnish/varnish.fix_CVE-2013-4484.patch.txt . I've added some changes for http_DissectRequest too (a check for Duplicated Host headers), though I cant say for sure if these are necessary. It compiles and runs tests/r01367.vtc fine without them. Please review this. If it seems appropriate, I'll do one for varnish-2.1.5 too. Ingvar From lkarsten at varnish-software.com Tue Nov 5 14:47:44 2013 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Tue, 5 Nov 2013 15:47:44 +0100 Subject: Memory management in VMODS: sp->ws vs. sp->wrk->ws In-Reply-To: <99BDCB7B-580B-4CB9-BEAA-6C988AFCD3E3@gmail.com> References: <99BDCB7B-580B-4CB9-BEAA-6C988AFCD3E3@gmail.com> Message-ID: <20131105144743.GB12008@immer.varnish-software.com> On Wed, Oct 30, 2013 at 05:33:19PM +0100, Carlos Abalde wrote: > After reading all the available documentation and also reviewing implementations of almost all VMODs in https://www.varnish-cache.org/vmods, I'm not sure what is the difference between using sp->ws and sp->wrk->ws when allocating memory in a VMOD method. It seems sp->wrk->sp is the way to go when using WS_Reserve() and WS_Release(). However, when using WS_Alloc() or WS_Dup() I've seen modules using both sp->ws and sp->wrk->sp. I guess it's not a random choice, but I haven't been able to identify any rule. > Is there any difference? Any hint on when to use sp->ws and when sp->wrk->sp? Hi Carlos. Use sp->ws. In the case of the request being waitinglisted, it will be wrk-less until woken up again. -- With regards, Lasse Karstensen Varnish Software AS From lkarsten at varnish-software.com Mon Nov 11 15:34:14 2013 From: lkarsten at varnish-software.com (Lasse Karstensen) Date: Mon, 11 Nov 2013 16:34:14 +0100 Subject: More on the HAProxy proxy protocol Message-ID: <20131111153413.GA27258@immer.varnish-software.com> [ watch out, this is became rather long ] Hi all. Recently I've been looking into merging Mark Bergsma's proxy protocol[1] work into Varnish. The goal of this email is to write down our thoughts up until now, and get a small discussion going on how it should look when it is done. This is a protocol where a small header is written first in a TCP connection, telling the next hop what the client ip/port was as seen by the proxy in front. Typical use is SSL termination where the SSL terminator does not know, or want to know for performance reasons, anything about the inner protocol. (no x-f-f) There are two header formats, proxy (text based) and proxy2 (binary). The binary one is preferred in the specification. I'm only considering frontside proxy support here. We have the standard xff (or now just f-f) to indicate this information to the backend. The proxy protocol also supports domain sockets, UDP and other oddities. I've only considered TCP over IPv4 and IPv6 here. I've discussed this with Tollef for a bit, and we've come up with the following. 1. Extend the -a startup argument with a protocol definition: Current behaviour: varnishd -a 192.0.2.10:80 -f /etc/varnish/foo.vcl varnishd -a plain at 192.0.2.10:80 -f /etc/varnish/foo.vcl # New equivalent. Force proxy Proxy or Proxy2 on incoming connections: varnishd -a proxy at 192.0.2.10:80 -f /etc/varnish/foo.vcl varnishd -a proxy2 at 192.0.2.10:80 -f /etc/varnish/foo.vcl Per the specification any connection not sending a proxy header to such a socket should be a hard error. It might be necessary to filter what clients are allowed to connect to this socket, to avoid security implications of outside clients writing the proxy header themselves. This can perhaps allow only localhost in the default case, and a predefined ACL must be set/extended in VCL to allow external clients. (needs discussion/thought) 2. VCL interface In VCL we now have client.ip, server.ip and server.port available. These are now (as I understand it) picked directly from the socket endpoints. New in the proxy protocol is that we have: a) proxy front connection source IPv4/IPv6. b) proxy front connection source port. c) proxy front connection destination IPv4/IPv6. (your SSL IP) d) proxy front connection destination port. (often 443) This information, or parts of it, needs to be put into VCL somehow, so we can use it for policy decisions like ACL matching. Mark took the stoic approach: a) VCL_IP req.proxy.client.ip b) VCL_INT req.proxy.client.port c) VCL_IP req.proxy.server.ip d) VCL_INT req.proxy.server.port e) bool req.proxy (proxy in use or not) This feels confusing; req.proxy.server.ip, is that the connection with the envelope on or what? Why is it in req, it is a session concept not a request concept; we don't have req.client.ip, we have client.ip. To keep VCL consise, and I think this is the least surprising behaviour for a user, we suggest that VCL client.ip is set to (a) if the socket is a proxy socket. This makes all the ACLs and logging behave like you expect them to. No cluttered VCL in the default case. If we do this, we're not sure what server.ip should be any more. Should it be (c), or should it be the local endpoint IP? Input appreciated. We also need to make something new that is the local socket endpoint information, since client.ip/server.ip is modified. Maybe a tcp.(client|server).(ip|port), if that prefix makes sense. I think the two ways forward are: 1) treat proxy as an exception to the rule, and make the 5 new proxy variables in VCL. (but removing req.) 2) change client.ip behind the scenes, and create 5 (?) new tcp.XXX variables in VCL to store what would be in client.ip/server.ip. Any comments or inputs on this are appreciated. 1: http://haproxy.1wt.eu/download/1.5/doc/proxy-protocol.txt -- With regards, Lasse Karstensen Varnish Software AS From phk at phk.freebsd.dk Mon Nov 11 19:19:08 2013 From: phk at phk.freebsd.dk (Poul-Henning Kamp) Date: Mon, 11 Nov 2013 19:19:08 +0000 Subject: More on the HAProxy proxy protocol In-Reply-To: <20131111153413.GA27258@immer.varnish-software.com> References: <20131111153413.GA27258@immer.varnish-software.com> Message-ID: <38901.1384197548@critter.freebsd.dk> In message <20131111153413.GA27258 at immer.varnish-software.com>, Lasse Karstense n writes: >1. Extend the -a startup argument with a protocol definition: We may be facing the same kind of issue with HTTP/2.0 but my best idea so far was: -a :80+http1+http2 > varnishd -a proxy at 192.0.2.10:80 -f /etc/varnish/foo.vcl > varnishd -a proxy2 at 192.0.2.10:80 -f /etc/varnish/foo.vcl I wish people would learn to name things sensibly. Calling a protocol "proxy" is a recipe for confusion >Per the specification any connection not sending a proxy header to such a >socket should be a hard error. Yes, I noticed that too, and that sort of made me lean towards allocating a specific command line argument (-p ?) >It might be necessary to filter what clients are allowed to connect to this >socket [...] I don't think that is really our job, but see below. >2. VCL interface > >In VCL we now have client.ip, server.ip and server.port available. These >are now (as I understand it) picked directly from the socket endpoints. I've been thinking about something like this: remote.ip // [IP Other end of TCP connection remote.port // [INT Our sockets peer-address local.ip // [IP own end of the TCP connection local.port // [INT sockets local address client.ip // [IP] Which IP$ client to connected to our end from. // if proto == PROXY // set from PROXY.hdr // else // set from remote.ip server.ip // [IP] Which IP# client connected to in our end. server.port // [INT] // if proto == PROXY // set from PROXY.hdr // else // set from our.* client.identity // Best case ultimate client identity // if X-F-F: // set from X-F-F // else // set from client.ip I'm somewhat tempted to make client.identity a STRING, rather than an IP, to make it clear to people that running it through an ACL is a bad idea. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk at FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From martin at varnish-software.com Tue Nov 19 09:02:15 2013 From: martin at varnish-software.com (Martin Blix Grydeland) Date: Tue, 19 Nov 2013 10:02:15 +0100 Subject: [PATCH] Backport #1337: weird http status codes In-Reply-To: References: Message-ID: Hi Kacper, In Varnishtest, a server or client is always handling a single connection (albeit for the server it's possible to do a new 'accept' statement, the client doesn't have anything similar). Since your test case fails on the client, and Varnish always closes on the client when going through vcl_error, it's not possible to reuse the same connection for the two tests. So you'll need to write the client parts of the tests like this: client c1 { txreq -url /low rxresp expect resp.status == 503 } -run client c1 { txreq -url /high rxresp expect resp.status == 503 } -run Martin On 11 October 2013 20:38, Kacper Wysocki wrote: > Hia all, > this is a backport of out-of-range backend status codes handling to > varnish-3.0.4. > > Martin mentioned maybe that fix needed backporting, and I couldn't wait. > Still can't get the test to pass, but if I break the two requests into two > tests, they pass. A case of the noobs? > > -- > http://u.delta9.pl > Too much order is its own chaos. > Employ no technique to gain supreme enlightment. > > > _______________________________________________ > varnish-dev mailing list > varnish-dev at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev > -- *Martin Blix Grydeland* Senior Developer | Varnish Software AS Cell: +47 21 98 92 60 We Make Websites Fly! -------------- next part -------------- An HTML attachment was scrubbed... URL: From ruben at varnish-software.com Thu Nov 21 14:12:48 2013 From: ruben at varnish-software.com (=?UTF-8?Q?Rub=C3=A9n_Romero?=) Date: Thu, 21 Nov 2013 15:12:48 +0100 Subject: Reminder: Eight Varnish User Group Meeting in Berlin next week #VUG8 Message-ID: Hallo, Alles gut? Sch?n. Das ist gut. ** Please help us to spread the word to all Varnish & Web Performance enthusiasts you know ** In behalf of the Varnish Team, it is my pleasure to announce to you the next Varnish User Group Meeting, the eight of its kind, which is to be held in Berlin, Germany. #VUG8 is possible thanks to the sponsorship of Axel Springer, Fastly, Uplex and Varnish Software. * Varnish User Day is open for everyone and it will be at Axel Springer HQ on Thursday, November 28th, 2013. Agenda and general information is up [1] and there are a few free tickets left. Get one now! [2] * Varnish (Core) Developer meeting: will happen on Wednesday, November 27th, 2013. This event is invite only. See and use the wiki to see the venue, add discussion topics and other details [3] * Hacking Day / Community Roundtable: New for #VUG8! It's open for everyone (we have a few free seats left) and takes place on Friday, November 29th, 2013. Discussion topics are Better documentation, V4 Release & Parties, Varnish Local User Groups, Varnish Security Firewall, VMODs, Utilities, improving http://varnish.org and more. We are very much looking forward to meet the Varnish community in Berlin and spending time with you while making Varnish Cache better. Before I let you go: Do not forget to let your colleagues and friends know about the event AND use the #VUG8 tag in social media. Thanks! Wir sehen uns in Berlin! Links: [1] [2] [3] [4] [5] All the best, -- *Rub?n Romero* Community & Sales | Varnish Software AS Cell: +47 95964088 / Office: +47 21989260 Skype & Twitter: ruben_varnish We Make Websites Fly!Winner of the 2013 Red Herring Top 100 Global Awards -------------- next part -------------- An HTML attachment was scrubbed... URL: From apassos at cs.umass.edu Tue Nov 19 18:55:52 2013 From: apassos at cs.umass.edu (Alexandre Passos) Date: Tue, 19 Nov 2013 13:55:52 -0500 Subject: Compilation problems and test failures on OSX, recent macbook air Message-ID: Hi, I got some compilation problems and test failures on a recent macbook air. The compilation problems happened because at many points varnish does a printf with format string %ju and passes it an uint64_t, but it expects an uintmax_t. The attached patch seems to fix this. After compiling I get two test errors when running make check, and I've attached the test log too. The errors seem to be unrelated to what I did to make it compile. I hope these were helpful, and thanks for the patience. Thanks -- - Alexandre Passos -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: fix.patch Type: application/octet-stream Size: 1548 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: test-suite.log Type: application/octet-stream Size: 17765 bytes Desc: not available URL: