From fgsch at lodoss.net  Thu Aug  1 01:03:14 2013
From: fgsch at lodoss.net (Federico Schwindt)
Date: Thu, 1 Aug 2013 02:03:14 +0100
Subject: [PATCH] Implement std.ip() to simplify ACL checking in VCL
In-Reply-To: <20130731134648.GB14202@immer.varnish-software.com>
References: <20130731134648.GB14202@immer.varnish-software.com>
Message-ID: <CAJV_h0aaqeTQM8FjtSQp3zGOKdm4c3nhgPhDpQzz3XD7tM+7-w@mail.gmail.com>

On Wed, Jul 31, 2013 at 2:46 PM, Lasse Karstensen <
lkarsten at varnish-software.com> wrote:

> Hello all.
>
> I've extended the std vmod to include an ip() method, which
> converts a string into VCL IP. The result can be used for
> matching against a VCL ACL.
>
> Attached is a patch against current master. Documentation and
> test case included.
>
> Please consider this for merging into the 4.0 release.
>

A few comments:

- rp leaks if WS_Reserve() fails.

- WS_Reserve() is cheaper that getaddrinfo(), so I'd check first if there
is space and then do the getaddrinfo() call. That'd simplify the error path
too.

- Missing CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC).

- You could just check for getaddrinfo() != 0 instead of using s = .. since
it's not used anywhere else.

- Using VCL_IP for the fallback parameter restricts what you can use to
client.ip or server.ip. This might or might not be a problem.
I wrote a similar function a while ago that was using a STRING parameter as
suggested by Tollef. Not sure if this is still required.

Cheers,

f.-
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20130801/b065e8c9/attachment.html>

From fgsch at lodoss.net  Thu Aug  1 01:11:00 2013
From: fgsch at lodoss.net (Federico Schwindt)
Date: Thu, 1 Aug 2013 02:11:00 +0100
Subject: [PATCH] Update graph
Message-ID: <CAJV_h0bmQhu+dd8pLpO8uyghpQx=Rm6scNCxGVKTEz-JJtwY+Q@mail.gmail.com>

And against master:

diff --git a/bin/varnishd/cache/cache_req_fsm.c
b/bin/varnishd/cache/cache_req_fsm.c
index 5ee6d14..ec7505c 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -762,7 +762,7 @@ cnt_restart(const struct worker *wrk, struct req *req)
 DOT subgraph xcluster_recv {
 DOT    recv [
 DOT            shape=record
-DOT
 label="{cnt_recv:|{vcl_recv\{\}|req.*}|{<pipe>pipe?|<pass>pass?|<error>error?|<lookup>lookup?}}"
+DOT
 label="{cnt_recv:|{vcl_recv\{\}|req.*}|{<pass>pass?|<lookup>lookup?|<pipe>pipe?|<error>error?}}"
 DOT    ]
 DOT }
 DOT subgraph xcluster_hash {
@@ -771,12 +771,14 @@ DOT               shape=record
 DOT            label="{cnt_recv:|{vcl_hash\{\}|req.*}}"
 DOT    ]
 DOT }
-DOT recv:pipe -> pipe [style=bold,color=orange]
-DOT recv:pass -> pass [style=bold,color=red]
+DOT recv:pipe -> hash [style=bold,color=orange]
+DOT recv:pass -> hash [style=bold,color=red]
 #DOT recv:error -> err_recv
 #DOT err_recv [label="ERROR",shape=plaintext]
 DOT recv:lookup -> hash [style=bold,color=green]
-DOT hash -> lookup [label="hash",style=bold,color=green]
+DOT hash -> lookup [style=bold,color=green]
+DOT hash -> pipe [style=bold,color=orange]
+DOT hash -> pass [style=bold,color=red]
  */

 static enum req_fsm_nxt
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20130801/76acd3e1/attachment.html>

From lkarsten at varnish-software.com  Thu Aug  1 10:41:13 2013
From: lkarsten at varnish-software.com (Lasse Karstensen)
Date: Thu, 1 Aug 2013 12:41:13 +0200
Subject: [PATCH] Implement std.ip() to simplify ACL checking in VCL
In-Reply-To: <CAJV_h0aaqeTQM8FjtSQp3zGOKdm4c3nhgPhDpQzz3XD7tM+7-w@mail.gmail.com>
References: <20130731134648.GB14202@immer.varnish-software.com>
 <CAJV_h0aaqeTQM8FjtSQp3zGOKdm4c3nhgPhDpQzz3XD7tM+7-w@mail.gmail.com>
Message-ID: <20130801104113.GD14202@immer.varnish-software.com>

On Thu, Aug 01, 2013 at 02:03:14AM +0100, Federico Schwindt wrote:
> On Wed, Jul 31, 2013 at 2:46 PM, Lasse Karstensen <
> lkarsten at varnish-software.com> wrote:
> > I've extended the std vmod to include an ip() method, which
> > converts a string into VCL IP. The result can be used for
> > matching against a VCL ACL.
> A few comments:
> - rp leaks if WS_Reserve() fails.
> - WS_Reserve() is cheaper that getaddrinfo(), so I'd check first if there
> is space and then do the getaddrinfo() call. That'd simplify the error path
> too.
> - Missing CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC).
> - You could just check for getaddrinfo() != 0 instead of using s = .. since
> it's not used anywhere else.

Hi Federico.
Thank you for taking the time to review this.

I've implemented the changes you proposed. Updated (full) patch is attached.


> - Using VCL_IP for the fallback parameter restricts what you can use to
> client.ip or server.ip. This might or might not be a problem.
> I wrote a similar function a while ago that was using a STRING parameter as
> suggested by Tollef. Not sure if this is still required.

Yes, we discussed this for a bit in the office.
I don't have strong opinions on either side.

You can of course nest them to get an arbitrary fallback:
	std.ip(req.http.X-Forwarded-For, std.ip("127.255.255.255"));

The error handling when the fallback conversion fails doesn't seem to have
any obvious solution. If the user gets to pick a fallback by him/herself,
then at least they know clearly what to check for.


-- 
With regards,
Lasse Karstensen
Varnish Software AS
-------------- next part --------------
A non-text attachment was scrubbed...
Name: add-ip-std.2013-08-01.patch
Type: text/x-diff
Size: 3545 bytes
Desc: not available
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20130801/91465361/attachment.bin>

From bilbo at hobbiton.org  Thu Aug  1 15:21:26 2013
From: bilbo at hobbiton.org (Leif Pedersen)
Date: Thu, 1 Aug 2013 10:21:26 -0500
Subject: [PATCH] Implement std.ip() to simplify ACL checking in VCL
In-Reply-To: <20130801104113.GD14202@immer.varnish-software.com>
References: <20130731134648.GB14202@immer.varnish-software.com>
 <CAJV_h0aaqeTQM8FjtSQp3zGOKdm4c3nhgPhDpQzz3XD7tM+7-w@mail.gmail.com>
 <20130801104113.GD14202@immer.varnish-software.com>
Message-ID: <CAK-wPOj1RtSKwDzvO5pvZ2NMXEXHCiB2N+MDokppM5crHKqyaA@mail.gmail.com>

On Thu, Aug 1, 2013 at 5:41 AM, Lasse Karstensen <
lkarsten at varnish-software.com> wrote:

> On Thu, Aug 01, 2013 at 02:03:14AM +0100, Federico Schwindt wrote:
> > On Wed, Jul 31, 2013 at 2:46 PM, Lasse Karstensen <
> lkarsten at varnish-software.com> wrote:
>
> > - Using VCL_IP for the fallback parameter restricts what you can use to
> > client.ip or server.ip. This might or might not be a problem.
> > I wrote a similar function a while ago that was using a STRING parameter
> as
> > suggested by Tollef. Not sure if this is still required.
>
> You can of course nest them to get an arbitrary fallback:
>         std.ip(req.http.X-Forwarded-For, std.ip("127.255.255.255"));
>
>
I've noticed that client.ip and server.ip can be used implicitly as
strings. (Is this without caveats?) So if std.ip accepts a string as the
fall-back, then the VCL never need specify the conversion explicitly, which
would make VCL code more succinct. That is, by accepting a string, both of
these would work, which I think would be helpful:

std.ip(req.http.X-Forwarded-For, "127.255.255.255");
std.ip(req.http.X-Forwarded-For, client.ip);

Of course, your point about a fallback fallback still stands if the string
is formatted badly. I suppose I'd just fall back to 0.0.0.0 or something in
that case.

PS. Useful feature, thanks. I wrote a couple of pieces of VCL code where I
had to resort to matching an IP received from a header against a range with
a regex instead of an ACL. This oughtta clean that up for me.


-- 

As implied by email protocols, the information in this message is
not confidential.  Any middle-man or recipient may inspect, modify,
copy, forward, reply to, delete, or filter email for any purpose unless
said parties are otherwise obligated.  As the sender, I acknowledge that
I have a lower expectation of the control and privacy of this message
than I would a post-card.  Further, nothing in this message is
legally binding without cryptographic evidence of its integrity.

http://bilbo.hobbiton.org/wiki/Eat_My_Sig
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20130801/677f22b5/attachment.html>

From bilbo at hobbiton.org  Thu Aug  1 15:57:42 2013
From: bilbo at hobbiton.org (Leif Pedersen)
Date: Thu, 1 Aug 2013 10:57:42 -0500
Subject: rdbms as backend
In-Reply-To: <51F8D021.8020105@gmail.com>
References: <51F79C4B.5050506@gmail.com>
 <CAK-wPOg6Fy5O04xL5mpeinjL-iT4761kUU9JZzxB5-ao_xhSVg@mail.gmail.com>
 <51F7BF0B.3050002@gmail.com>
 <CAK-wPOjYH1heFPS2Bpa_pX_WFDD5rM_frnsf82Q-j7Ym6mh6MQ@mail.gmail.com>
 <51F8D021.8020105@gmail.com>
Message-ID: <CAK-wPOjk-2K2ie3xr3ehjgxfgnpTgTntjzKUr-kCx0ten2GxYQ@mail.gmail.com>

Hm, lemmie step back a sec. So as I understand, you currently get 30k
frontend requests per sec, and with Varnish to cache results, you have
about 7k backend requests per sec. Does this line up now?

Seems to me that if you're provisioning the middleware for 7k rqs instead
of 30k rqs, the problem is much easier to solve. It may require a few
machines, but it sounds like your DB costs are so high that saving you 76%
on database traffic would be an easy budget to meet. You've done FAPWS3 on
one machine at 3k rps? How about simply running that solution on 3 machines
plus a spare or two, which should provision it for about 9k rps? One nice
thing about this approach is that it's usually far easier to add (and fail
over) HTTP nodes and database clients than to add database servers.

I'm pragmatic in this. If the middleware costs a lot more than a custom
vmod to connect to the DBs, then I'd most likely do that. My skepticism is
just that it doesn't seem likely. So I won't answer your question ("you
would not connect to DB directly either?") in the absolute affirmative.
However, with an experienced guess and only a little information about your
problem space, that is my inclination, yes. And I wouldn't build the vmod
for purity or fun -- it sounds to me like a daunting gnarly thing with lots
of maintainability issues. But don't let me tell you not to, if you really
believe in the cause. :)

With deference to the authors, I'd be a bit astonished to see such a vmod
in Varnish's distribution. But if it's worth it in comparison to a
middleware solution (be it Python, node.js, C++, or whatever), the results
would certainly be interesting as a third-party vmod if you don't mind
sharing.

- Leif


-- 

As implied by email protocols, the information in this message is
not confidential.  Any middle-man or recipient may inspect, modify,
copy, forward, reply to, delete, or filter email for any purpose unless
said parties are otherwise obligated.  As the sender, I acknowledge that
I have a lower expectation of the control and privacy of this message
than I would a post-card.  Further, nothing in this message is
legally binding without cryptographic evidence of its integrity.

http://bilbo.hobbiton.org/wiki/Eat_My_Sig
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20130801/cfce71d6/attachment.html>

From mrkafk at gmail.com  Thu Aug  1 16:52:34 2013
From: mrkafk at gmail.com (Marcin Krol)
Date: Thu, 01 Aug 2013 18:52:34 +0200
Subject: rdbms as backend
In-Reply-To: <CAK-wPOjk-2K2ie3xr3ehjgxfgnpTgTntjzKUr-kCx0ten2GxYQ@mail.gmail.com>
References: <51F79C4B.5050506@gmail.com>
 <CAK-wPOg6Fy5O04xL5mpeinjL-iT4761kUU9JZzxB5-ao_xhSVg@mail.gmail.com>
 <51F7BF0B.3050002@gmail.com>
 <CAK-wPOjYH1heFPS2Bpa_pX_WFDD5rM_frnsf82Q-j7Ym6mh6MQ@mail.gmail.com>
 <51F8D021.8020105@gmail.com>
 <CAK-wPOjk-2K2ie3xr3ehjgxfgnpTgTntjzKUr-kCx0ten2GxYQ@mail.gmail.com>
Message-ID: <51FA9252.4080503@gmail.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello Leif,

Thanks for interest in this topic! Nobody else picked it up it seems. :-)


To tell the truth it's somewhat different:

1. 30K rps *total* is what we have to handle at typical hrs.


2. 7K rps is what I managed to get ONE INSTANCE of Varnish to achieve
*as cache for one of our HTTP backends*. (our legacy solution caches
Oracle, MySQL *and* HTTP backends, transforming it all into JSON/HTTP
responses, used by lots of different client libraries and subsystems -
this is all internal infrastructure).


3. 20-30 instances of node.js spread across several machines + nginx
at the front would be doable for obvious reasons of load balancing and
failover (that's what tests of working prototype implemented in
node.js suggest). That's sort of plan A. I'm investigating plan B.

What I don't like about plan A is that it's not only heavy and costly,
what's even worse is that we have to build our own HA and load
balancing into it that like most of infrastructual stuff somebody
develops for themselves is half-baked and coded in haste. Another
piece of old unmaintainable cruft in the making, after telling
ourselves once again "this time it will be different" (no it won't be
unless we take a different approach).

Varnish has probes, load balancing, random and round robin failover
and load balancing that's (probably?) battle-tested by many people and
companies and frankly the bulk of dev costs is on somebody else's
shoulders.

I do not like building caching server myself anymore than I like
building nginx or apache replacements myself.

All clients talk http anyway. Some backends talk http anyway.

So the only thing I'd have to do to achieve nirvana would be making
databases translate their result sets into JSON and make them
available over http. Which eventually we have to do anyway at some
place (lots of different clients, can't rewrite and upgrade them all
anyway on version change of mysql from X to Y).

We have already done loose coupling in the databases: to avoid having
to rewrite SQL on every upgrade or possibly switch to another DB, we
implement everything possible in dbs as stored procedures and use dirt
simple queries calling those stored procedures, so SQL "frontend"
stays the same while you can tweak stored procedure behind it to your
liking. There's only a single step from there to query result
uniformization.

Admittedly, this sort of thing - plugging db into varnish - looks
weird, even outlandish. But it's so logical and fits so well I have
trouble giving up this thought!


I may give up though and simply add another layer between varnish and
databases.

Regards,
MK











W dniu 8/1/2013 17:57, Leif Pedersen pisze:
> Hm, lemmie step back a sec. So as I understand, you currently get
> 30k frontend requests per sec, and with Varnish to cache results,
> you have about 7k backend requests per sec. Does this line up now?
> 
> Seems to me that if you're provisioning the middleware for 7k rqs 
> instead of 30k rqs, the problem is much easier to solve. It may
> require a few machines, but it sounds like your DB costs are so
> high that saving you 76% on database traffic would be an easy
> budget to meet. You've done FAPWS3 on one machine at 3k rps? How
> about simply running that solution on 3 machines plus a spare or
> two, which should provision it for about 9k rps? One nice thing
> about this approach is that it's usually far easier to add (and
> fail over) HTTP nodes and database clients than to add database
> servers.
> 
> I'm pragmatic in this. If the middleware costs a lot more than a
> custom vmod to connect to the DBs, then I'd most likely do that. My
> skepticism is just that it doesn't seem likely. So I won't answer
> your question ("you would not connect to DB directly either?") in
> the absolute affirmative. However, with an experienced guess and
> only a little information about your problem space, that is my
> inclination, yes. And I wouldn't build the vmod for purity or fun
> -- it sounds to me like a daunting gnarly thing with lots of
> maintainability issues. But don't let me tell you not to, if you
> really believe in the cause. :)
> 
> With deference to the authors, I'd be a bit astonished to see such
> a vmod in Varnish's distribution. But if it's worth it in
> comparison to a middleware solution (be it Python, node.js, C++, or
> whatever), the results would certainly be interesting as a
> third-party vmod if you don't mind sharing.
> 
> - Leif
> 
> 
> --
> 
> As implied by email protocols, the information in this message is 
> not confidential.  Any middle-man or recipient may inspect,
> modify, copy, forward, reply to, delete, or filter email for any
> purpose unless said parties are otherwise obligated.  As the
> sender, I acknowledge that I have a lower expectation of the
> control and privacy of this message than I would a post-card.
> Further, nothing in this message is legally binding without
> cryptographic evidence of its integrity.
> 
> http://bilbo.hobbiton.org/wiki/Eat_My_Sig
> 
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.20 (MingW32)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJR+pJSAAoJEFMgHzhQQ7hOEjIH/R3MrpOdXSPeBDZVTgrhg63U
lUjsuobvDJDYXeMSoBNI24aeFUdCwnWrPJjNHN1qG9TAZl7+Rtq0muLkHb/ToAlx
n4L7A18omg1Lqp9SAHboL4+OHgpBSfNOLDtuuXS6L1NoOxkdWJxAyBVCrz1x+QqO
HxvKWPy0pVUYwX6P9tdjTTNIjlzJpNrshV036MCe3cdnFRMvlR1sFGOKc8DGQjEb
6VZ7pxQEzCFj6D9cYfe4a8X3x46nRMshlD+k3su2Zsp7t/450mEKS4LiJf9H6Aht
rqhZl1LHbXm5dAghJxC4FZOhsr2HTgpQQSdHgA79gv+Ej1GhkO9iAHcjI2XPouA=
=92FJ
-----END PGP SIGNATURE-----



From slaweuk at gmail.com  Thu Aug  1 17:18:03 2013
From: slaweuk at gmail.com (slaweuk)
Date: Thu, 1 Aug 2013 18:18:03 +0100
Subject: rdbms as backend
In-Reply-To: <51FA9252.4080503@gmail.com>
References: <51F79C4B.5050506@gmail.com>
 <CAK-wPOg6Fy5O04xL5mpeinjL-iT4761kUU9JZzxB5-ao_xhSVg@mail.gmail.com>
 <51F7BF0B.3050002@gmail.com>
 <CAK-wPOjYH1heFPS2Bpa_pX_WFDD5rM_frnsf82Q-j7Ym6mh6MQ@mail.gmail.com>
 <51F8D021.8020105@gmail.com>
 <CAK-wPOjk-2K2ie3xr3ehjgxfgnpTgTntjzKUr-kCx0ten2GxYQ@mail.gmail.com>
 <51FA9252.4080503@gmail.com>
Message-ID: <78CA71F8-3D1F-4C8F-87C9-B1F52AACEFD7@gmail.com>

On 1 Aug 2013, at 17:52, Marcin Krol <mrkafk at gmail.com> wrote:
> So the only thing I'd have to do to achieve nirvana would be making
> databases translate their result sets into JSON and make them
> available over http. Which eventually we have to do anyway at some
> place (lots of different clients, can't rewrite and upgrade them all
> anyway on version change of mysql from X to Y).


Hi Marcin,

There are some projects already covering this subject:

http://code.nytimes.com/projects/dbslayer
http://code.google.com/p/mod-ndb/
http://www.slashdb.com/
http://jersey.java.net/
http://restsql.org/

I would also consider adding HAProxy behind the Varnish to support more complex proxy configurations (also better insights in connections metrics).

Best Regards,
Slawek
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20130801/6f9c1703/attachment.html>

From fgsch at lodoss.net  Fri Aug  2 01:43:29 2013
From: fgsch at lodoss.net (Federico Schwindt)
Date: Fri, 2 Aug 2013 02:43:29 +0100
Subject: [PATCH] Implement std.ip() to simplify ACL checking in VCL
In-Reply-To: <20130801104113.GD14202@immer.varnish-software.com>
References: <20130731134648.GB14202@immer.varnish-software.com>
 <CAJV_h0aaqeTQM8FjtSQp3zGOKdm4c3nhgPhDpQzz3XD7tM+7-w@mail.gmail.com>
 <20130801104113.GD14202@immer.varnish-software.com>
Message-ID: <CAJV_h0b+dKo17BAVOvwum8x2MttjfAJu28BDz-Yroig687spAQ@mail.gmail.com>

On Thu, Aug 1, 2013 at 11:41 AM, Lasse Karstensen <
lkarsten at varnish-software.com> wrote:

> On Thu, Aug 01, 2013 at 02:03:14AM +0100, Federico Schwindt wrote:
> > On Wed, Jul 31, 2013 at 2:46 PM, Lasse Karstensen <
> > lkarsten at varnish-software.com> wrote:
> > > I've extended the std vmod to include an ip() method, which
> > > converts a string into VCL IP. The result can be used for
> > > matching against a VCL ACL.
> > A few comments:
> > - rp leaks if WS_Reserve() fails.
> > - WS_Reserve() is cheaper that getaddrinfo(), so I'd check first if there
> > is space and then do the getaddrinfo() call. That'd simplify the error
> path
> > too.
> > - Missing CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC).
> > - You could just check for getaddrinfo() != 0 instead of using s = ..
> since
> > it's not used anywhere else.
>
> Hi Federico.
> Thank you for taking the time to review this.
>
> I've implemented the changes you proposed. Updated (full) patch is
> attached.


Style aside looks good to me.


> > - Using VCL_IP for the fallback parameter restricts what you can use to
> > client.ip or server.ip. This might or might not be a problem.
> > I wrote a similar function a while ago that was using a STRING parameter
> as
> > suggested by Tollef. Not sure if this is still required.
>
> Yes, we discussed this for a bit in the office.
> I don't have strong opinions on either side.
>
> You can of course nest them to get an arbitrary fallback:
>         std.ip(req.http.X-Forwarded-For, std.ip("127.255.255.255"));


I take you meant:

std.ip(req.http.X-Forwarded-For, std.ip("127.255.255.255", server.ip));

or similar above.


> The error handling when the fallback conversion fails doesn't seem to have
> any obvious solution. If the user gets to pick a fallback by him/herself,
> then at least they know clearly what to check for.


True. Perhaps this is a good candidate for the VCL Examples wiki page.

f.-
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20130802/a2ab4c9a/attachment.html>

From fgsch at lodoss.net  Fri Aug  2 02:02:29 2013
From: fgsch at lodoss.net (Federico Schwindt)
Date: Fri, 2 Aug 2013 03:02:29 +0100
Subject: [PATCH] Implement std.ip() to simplify ACL checking in VCL
In-Reply-To: <CAK-wPOj1RtSKwDzvO5pvZ2NMXEXHCiB2N+MDokppM5crHKqyaA@mail.gmail.com>
References: <20130731134648.GB14202@immer.varnish-software.com>
 <CAJV_h0aaqeTQM8FjtSQp3zGOKdm4c3nhgPhDpQzz3XD7tM+7-w@mail.gmail.com>
 <20130801104113.GD14202@immer.varnish-software.com>
 <CAK-wPOj1RtSKwDzvO5pvZ2NMXEXHCiB2N+MDokppM5crHKqyaA@mail.gmail.com>
Message-ID: <CAJV_h0ZnNc+PfT8Xd1arkECKV2aAcVpeXsFSedQZUQw1CYxuQg@mail.gmail.com>

On Thu, Aug 1, 2013 at 4:21 PM, Leif Pedersen <bilbo at hobbiton.org> wrote:

> On Thu, Aug 1, 2013 at 5:41 AM, Lasse Karstensen <
> lkarsten at varnish-software.com> wrote:
>
>> On Thu, Aug 01, 2013 at 02:03:14AM +0100, Federico Schwindt wrote:
>> > On Wed, Jul 31, 2013 at 2:46 PM, Lasse Karstensen <
>> lkarsten at varnish-software.com> wrote:
>>
>> > - Using VCL_IP for the fallback parameter restricts what you can use to
>> > client.ip or server.ip. This might or might not be a problem.
>> > I wrote a similar function a while ago that was using a STRING
>> parameter as
>> > suggested by Tollef. Not sure if this is still required.
>>
>> You can of course nest them to get an arbitrary fallback:
>>         std.ip(req.http.X-Forwarded-For, std.ip("127.255.255.255"));
>>
>>
> I've noticed that client.ip and server.ip can be used implicitly as
> strings. (Is this without caveats?) So if std.ip accepts a string as the
> fall-back, then the VCL never need specify the conversion explicitly, which
> would make VCL code more succinct. That is, by accepting a string, both of
> these would work, which I think would be helpful:
>
> std.ip(req.http.X-Forwarded-For, "127.255.255.255");
> std.ip(req.http.X-Forwarded-For, client.ip);
>
 [..]
>

Explicit conversion won't happen in the case above. That'd need to be
implemented first if fallback would change to STRING.

f.-
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20130802/340e1cb5/attachment.html>

From phk at phk.freebsd.dk  Mon Aug  5 05:12:11 2013
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Mon, 05 Aug 2013 05:12:11 +0000
Subject: [PATCH] Implement std.ip() to simplify ACL checking in VCL
In-Reply-To: <CAK-wPOj1RtSKwDzvO5pvZ2NMXEXHCiB2N+MDokppM5crHKqyaA@mail.gmail.com>
References: <20130731134648.GB14202@immer.varnish-software.com>
 <CAJV_h0aaqeTQM8FjtSQp3zGOKdm4c3nhgPhDpQzz3XD7tM+7-w@mail.gmail.com>
 <20130801104113.GD14202@immer.varnish-software.com>
 <CAK-wPOj1RtSKwDzvO5pvZ2NMXEXHCiB2N+MDokppM5crHKqyaA@mail.gmail.com>
Message-ID: <11698.1375679531@critter.freebsd.dk>

In message <CAK-wPOj1RtSKwDzvO5pvZ2NMXEXHCiB2N+MDokppM5crHKqyaA at mail.gmail.com>
, Leif Pedersen writes:

>> > - Using VCL_IP for the fallback parameter restricts what you can use to
>> > client.ip or server.ip. This might or might not be a problem.
>> > I wrote a similar function a while ago that was using a STRING parameter as
>> > suggested by Tollef. Not sure if this is still required.

VCL_IP is the right type.

I can't remember if VCC has code to type-convert a STRING to IP, but
if not, that should be added.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.



From phk at phk.freebsd.dk  Mon Aug  5 06:06:44 2013
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Mon, 05 Aug 2013 06:06:44 +0000
Subject: [PATCH] Implement std.ip() to simplify ACL checking in VCL
In-Reply-To: <11698.1375679531@critter.freebsd.dk>
References: <20130731134648.GB14202@immer.varnish-software.com>
 <CAJV_h0aaqeTQM8FjtSQp3zGOKdm4c3nhgPhDpQzz3XD7tM+7-w@mail.gmail.com>
 <20130801104113.GD14202@immer.varnish-software.com>
 <CAK-wPOj1RtSKwDzvO5pvZ2NMXEXHCiB2N+MDokppM5crHKqyaA@mail.gmail.com>
 <11698.1375679531@critter.freebsd.dk>
Message-ID: <33379.1375682804@critter.freebsd.dk>

In message <11698.1375679531 at critter.freebsd.dk>, "Poul-Henning Kamp" writes:
>In message <CAK-wPOj1RtSKwDzvO5pvZ2NMXEXHCiB2N+MDokppM5crHKqyaA at mail.gmail.com>
>, Leif Pedersen writes:
>
>>> > - Using VCL_IP for the fallback parameter restricts what you can use to
>>> > client.ip or server.ip. This might or might not be a problem.
>>> > I wrote a similar function a while ago that was using a STRING parameter as
>>> > suggested by Tollef. Not sure if this is still required.
>
>VCL_IP is the right type.
>
>I can't remember if VCC has code to type-convert a STRING to IP, but
>if not, that should be added.

I looked, it doesn't, I'm adding it now.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.



From christophe.borot at gmail.com  Fri Aug  2 16:57:16 2013
From: christophe.borot at gmail.com (Christophe Borot)
Date: Fri, 2 Aug 2013 18:57:16 +0200
Subject: Varnish is working, but only for each instance of a browser...
Message-ID: <CAOoRGECsXamgVnbG3Fi75dV8N=jpbQU0gzHdJUONbuA1i8xphQ@mail.gmail.com>

Dear Mailing List,

I'm experiencing a weird problem.

Let's say I'm on Firefox, the cache is working (1st visit: cache is built,
2nd visit, cache is sent, 3rd visit, cache in sent...,  etc...)
Then if I go on Chrome, Varnish is recalculating the page for this "new"
visitor, instead of sending the first calculated page.

It's like if Varnish was working, but only for each instance of a browser.

If anyone can tell me if i'm doing something wrong ?

Thank you very much,

Christophe
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20130802/5659810b/attachment.html>

From varnish at bsdchicks.com  Mon Aug  5 12:42:39 2013
From: varnish at bsdchicks.com (Rogier 'DocWilco' Mulhuijzen)
Date: Mon, 5 Aug 2013 14:42:39 +0200
Subject: Varnish is working, but only for each instance of a browser...
In-Reply-To: <CAOoRGECsXamgVnbG3Fi75dV8N=jpbQU0gzHdJUONbuA1i8xphQ@mail.gmail.com>
References: <CAOoRGECsXamgVnbG3Fi75dV8N=jpbQU0gzHdJUONbuA1i8xphQ@mail.gmail.com>
Message-ID: <CANTTouXDCA_ieH7r5RES43n4O8RTHGPB2f=JrLsddVm1Mmb0yQ@mail.gmail.com>

First of all, this is a mailing list for developers of Varnish, not a list
for users. I would like to point you at:

https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

Also, when mailing that, make sure to include your VCL (sanitized if
there's any sensitive information in it) and the output of the varnishlog
utility for requests by both Firefox and Chrome.

My guess is that your backend sends a Vary: User-Agent or the likes, though.

Cheers,

      Doc


On Fri, Aug 2, 2013 at 6:57 PM, Christophe Borot <christophe.borot at gmail.com
> wrote:

> Dear Mailing List,
>
> I'm experiencing a weird problem.
>
> Let's say I'm on Firefox, the cache is working (1st visit: cache is built,
> 2nd visit, cache is sent, 3rd visit, cache in sent...,  etc...)
> Then if I go on Chrome, Varnish is recalculating the page for this "new"
> visitor, instead of sending the first calculated page.
>
> It's like if Varnish was working, but only for each instance of a browser.
>
> If anyone can tell me if i'm doing something wrong ?
>
> Thank you very much,
>
> Christophe
>
>
>
> _______________________________________________
> varnish-dev mailing list
> varnish-dev at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20130805/56a70c72/attachment.html>

From slink at schokola.de  Tue Aug  6 07:37:50 2013
From: slink at schokola.de (Nils Goroll)
Date: Tue, 06 Aug 2013 09:37:50 +0200
Subject: VDD13Q3: Friday September 6th @ Hamburg
Message-ID: <5200A7CE.2050806@schokola.de>

Hi,

after double checking with bz, we now have a date with just one "Ifneedbe" and 
no real "No", so I've fixed the date to September 6th: 
https://www.varnish-cache.org/trac/wiki/VDD13Q3

Please confirm your participation on the wiki page.

If anyone needs assistance finding accommodation, please let me know. Also 
everyone is welcome to arrive early or leave late to explore Hamburg's nightlife 
with us.


I am very much looking forward to welcoming you to Hamburg and spend a day with 
some collaborative Varnish hacking.

See you, Nils



From russ at eatnumber1.com  Mon Aug 12 17:57:35 2013
From: russ at eatnumber1.com (Russell Harmon)
Date: Mon, 12 Aug 2013 10:57:35 -0700
Subject: Fwd: Requests with the same ID interleaving in varnishlog output
In-Reply-To: <CA+zrezSQQYr=LppPDbFJ=shmZgwNszPOirc9OZdTiujddd6H6g@mail.gmail.com>
References: <CA+zrezSQQYr=LppPDbFJ=shmZgwNszPOirc9OZdTiujddd6H6g@mail.gmail.com>
Message-ID: <CA+zrezTHHnGkM3qQOqhcfbv57So+oSdszrsmiALVioDMc+xMfw@mail.gmail.com>

Forwarding this to varnish-dev since varnish-misc had no answers for me.

---------- Forwarded message ----------
From: Russell Harmon <russ at eatnumber1.com>
Date: Thu, Aug 8, 2013 at 11:11 AM
Subject: Requests with the same ID interleaving in varnishlog output
To: varnish-misc at varnish-cache.org


I'm writing a parser for varnishlog output and am seeing some odd behavior.
See the log here:
https://gist.github.com/eatnumber1/1dc55700324d9ee4dfb4which shows a
request start with xid 1254156398, later followed by a
request end with xid 1254156353 (which is the xid of the request before
1254156398), later (not shown) followed by the ReqEnd of 1254156353. What's
the deal? How can I tell what request all the lines with id 7 are for?

--
Russell Harmon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20130812/953a66e0/attachment.html>

From srmelody at gmail.com  Wed Aug 21 17:03:05 2013
From: srmelody at gmail.com (Sean Melody)
Date: Wed, 21 Aug 2013 13:03:05 -0400
Subject: Streaming branch
Message-ID: <CAPrJR8sTdkrLJxUP-w8RBXu9KFjZyjhpkFEEZHVouV7kMB7WBA@mail.gmail.com>

Hi all,

First time poster, long time fan.  :)

Were the commits that Martin made in his fork:
https://github.com/mbgrydeland/varnish-cache-streaming merged and/or ported
to the 3.0.4 release?  If I wanted to cache streaming video, can I used
3.0.4 with range requests and the do_stream flag turned on?


Thanks!

Sean
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20130821/16979248/attachment.html>

From c.ruppert at babiel.com  Thu Aug 29 12:19:57 2013
From: c.ruppert at babiel.com (Christian Ruppert)
Date: Thu, 29 Aug 2013 14:19:57 +0200
Subject: [PATCH]: Debian init script - Verify the VCL before restart
Message-ID: <D76AF9B3DDF5FF49833AC0A13965490603B298A1@s554.babiel.com>

Hi guys,

I enhanced the Debian init script a bit. It now verifies the config
prior restarting (or actually stopping) the current running varnish
process.
I couldn't find the Debian stuff in the Varnish Git repository so if
there is a other location for it please let me know.
The patch is against the current Varnish version (3.0.4) of
repo.varnish-cache.org.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: varnish.init.patch
Type: application/octet-stream
Size: 523 bytes
Desc: varnish.init.patch
URL: <https://www.varnish-cache.org/lists/pipermail/vinyl-dev/attachments/20130829/cd941799/attachment.obj>